INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) 2025-12-17 16:13:03,941 - lava_callback - INFO - Received callback for node 694037c8575bc060ce157e06 2025-12-17 16:13:03,966 - lava_callback - INFO - Starting async job submit for node 694037c8575bc060ce157e06 INFO: 172.21.0.1:48308 - "POST /node/694037c8575bc060ce157e06 HTTP/1.0" 202 Accepted 2025-12-17 16:13:04,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:04,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037c8575bc060ce157e06 HTTP/1.1" 200 None 2025-12-17 16:13:04,117 - lava_callback - DEBUG - Processing node 694037c8575bc060ce157e06: baseline-nfs-arm64 2025-12-17 16:13:05,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:13:06,110 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:13:06,111 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694037c8575bc060ce157e06/log.txt.gz 2025-12-17 16:13:06,111 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:13:06,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:13:06,398 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:13:06,399 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694037c8575bc060ce157e06/lava_callback.json.gz INFO: 172.21.0.1:48318 - "GET / HTTP/1.0" 200 OK DEBUG: Setup failed for 694037c8575bc060ce157e06. Transitting job node result: incomplete -> fail 2025-12-17 16:13:06,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:06,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037c8575bc060ce157e06 HTTP/1.1" 200 None 2025-12-17 16:13:06,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:06,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693bd0faf552eae6fe201eb6 HTTP/1.1" 200 None 2025-12-17 16:13:06,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:06,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037c8575bc060ce157e06 HTTP/1.1" 200 None 2025-12-17 16:13:06,966 - lava_callback - INFO - Completed processing callback for node 694037c8575bc060ce157e06 INFO: 172.21.0.1:39022 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:13:14,357 - lava_callback - INFO - Received callback for node 69403386575bc060ce1570e4 INFO: 172.21.0.1:39030 - "POST /node/69403386575bc060ce1570e4 HTTP/1.0" 202 Accepted 2025-12-17 16:13:14,364 - lava_callback - INFO - Starting async job submit for node 69403386575bc060ce1570e4 2025-12-17 16:13:14,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:14,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403386575bc060ce1570e4 HTTP/1.1" 200 None 2025-12-17 16:13:14,379 - lava_callback - DEBUG - Processing node 69403386575bc060ce1570e4: baseline-arm-baylibre 2025-12-17 16:13:14,379 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403386575bc060ce1570e4 2025-12-17 16:13:14,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:14,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403386575bc060ce1570e4 HTTP/1.1" 200 None 2025-12-17 16:13:14,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:14,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:14,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:14,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403386575bc060ce1570e4 HTTP/1.1" 200 None 2025-12-17 16:13:14,418 - lava_callback - INFO - Completed processing callback for node 69403386575bc060ce1570e4 2025-12-17 16:13:15,019 - lava_callback - INFO - Received callback for node 69403389575bc060ce1570ef INFO: 172.21.0.1:39044 - "POST /node/69403389575bc060ce1570ef HTTP/1.0" 202 Accepted 2025-12-17 16:13:15,028 - lava_callback - INFO - Starting async job submit for node 69403389575bc060ce1570ef 2025-12-17 16:13:15,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:15,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403389575bc060ce1570ef HTTP/1.1" 200 None 2025-12-17 16:13:15,039 - lava_callback - DEBUG - Processing node 69403389575bc060ce1570ef: baseline-arm-baylibre 2025-12-17 16:13:15,039 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403389575bc060ce1570ef 2025-12-17 16:13:15,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:15,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403389575bc060ce1570ef HTTP/1.1" 200 None 2025-12-17 16:13:15,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:15,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:15,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:15,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403389575bc060ce1570ef HTTP/1.1" 200 None 2025-12-17 16:13:15,086 - lava_callback - INFO - Completed processing callback for node 69403389575bc060ce1570ef INFO: 172.21.0.1:39048 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:13:15,658 - lava_callback - INFO - Received callback for node 6940338a575bc060ce1570f2 INFO: 172.21.0.1:39064 - "POST /node/6940338a575bc060ce1570f2 HTTP/1.0" 202 Accepted 2025-12-17 16:13:15,663 - lava_callback - INFO - Starting async job submit for node 6940338a575bc060ce1570f2 2025-12-17 16:13:15,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:15,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940338a575bc060ce1570f2 HTTP/1.1" 200 None 2025-12-17 16:13:15,674 - lava_callback - DEBUG - Processing node 6940338a575bc060ce1570f2: baseline-arm-baylibre 2025-12-17 16:13:15,674 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940338a575bc060ce1570f2 2025-12-17 16:13:15,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:15,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940338a575bc060ce1570f2 HTTP/1.1" 200 None 2025-12-17 16:13:15,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:15,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:15,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:15,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940338a575bc060ce1570f2 HTTP/1.1" 200 None 2025-12-17 16:13:15,721 - lava_callback - INFO - Completed processing callback for node 6940338a575bc060ce1570f2 2025-12-17 16:13:16,320 - lava_callback - INFO - Received callback for node 6940338b575bc060ce1570f5 INFO: 172.21.0.1:39070 - "POST /node/6940338b575bc060ce1570f5 HTTP/1.0" 202 Accepted 2025-12-17 16:13:16,325 - lava_callback - INFO - Starting async job submit for node 6940338b575bc060ce1570f5 2025-12-17 16:13:16,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:16,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940338b575bc060ce1570f5 HTTP/1.1" 200 None 2025-12-17 16:13:16,341 - lava_callback - DEBUG - Processing node 6940338b575bc060ce1570f5: baseline-arm-baylibre 2025-12-17 16:13:16,341 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940338b575bc060ce1570f5 2025-12-17 16:13:16,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:16,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940338b575bc060ce1570f5 HTTP/1.1" 200 None 2025-12-17 16:13:16,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:16,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:16,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:16,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940338b575bc060ce1570f5 HTTP/1.1" 200 None 2025-12-17 16:13:16,375 - lava_callback - INFO - Completed processing callback for node 6940338b575bc060ce1570f5 INFO: 172.21.0.1:39082 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:13:16,983 - lava_callback - INFO - Received callback for node 6940338c575bc060ce1570f8 INFO: 172.21.0.1:39086 - "POST /node/6940338c575bc060ce1570f8 HTTP/1.0" 202 Accepted 2025-12-17 16:13:16,989 - lava_callback - INFO - Starting async job submit for node 6940338c575bc060ce1570f8 2025-12-17 16:13:16,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:17,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940338c575bc060ce1570f8 HTTP/1.1" 200 None 2025-12-17 16:13:17,000 - lava_callback - DEBUG - Processing node 6940338c575bc060ce1570f8: baseline-arm-baylibre 2025-12-17 16:13:17,000 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940338c575bc060ce1570f8 2025-12-17 16:13:17,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:17,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940338c575bc060ce1570f8 HTTP/1.1" 200 None 2025-12-17 16:13:17,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:17,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:17,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:17,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940338c575bc060ce1570f8 HTTP/1.1" 200 None 2025-12-17 16:13:17,038 - lava_callback - INFO - Completed processing callback for node 6940338c575bc060ce1570f8 2025-12-17 16:13:17,674 - lava_callback - INFO - Received callback for node 6940338e575bc060ce1570fb INFO: 172.21.0.1:39090 - "POST /node/6940338e575bc060ce1570fb HTTP/1.0" 202 Accepted 2025-12-17 16:13:17,684 - lava_callback - INFO - Starting async job submit for node 6940338e575bc060ce1570fb 2025-12-17 16:13:17,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:17,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940338e575bc060ce1570fb HTTP/1.1" 200 None 2025-12-17 16:13:17,696 - lava_callback - DEBUG - Processing node 6940338e575bc060ce1570fb: baseline-arm-baylibre 2025-12-17 16:13:17,696 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940338e575bc060ce1570fb 2025-12-17 16:13:17,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:17,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940338e575bc060ce1570fb HTTP/1.1" 200 None 2025-12-17 16:13:17,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:17,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:17,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:17,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940338e575bc060ce1570fb HTTP/1.1" 200 None 2025-12-17 16:13:17,751 - lava_callback - INFO - Completed processing callback for node 6940338e575bc060ce1570fb 2025-12-17 16:13:18,314 - lava_callback - INFO - Received callback for node 6940338f575bc060ce157100 INFO: 172.21.0.1:39104 - "POST /node/6940338f575bc060ce157100 HTTP/1.0" 202 Accepted 2025-12-17 16:13:18,324 - lava_callback - INFO - Starting async job submit for node 6940338f575bc060ce157100 2025-12-17 16:13:18,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:18,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940338f575bc060ce157100 HTTP/1.1" 200 None 2025-12-17 16:13:18,430 - lava_callback - DEBUG - Processing node 6940338f575bc060ce157100: baseline-arm-baylibre 2025-12-17 16:13:18,430 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940338f575bc060ce157100 2025-12-17 16:13:18,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:18,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940338f575bc060ce157100 HTTP/1.1" 200 None 2025-12-17 16:13:18,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:18,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:18,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:18,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940338f575bc060ce157100 HTTP/1.1" 200 None 2025-12-17 16:13:18,469 - lava_callback - INFO - Completed processing callback for node 6940338f575bc060ce157100 2025-12-17 16:13:19,006 - lava_callback - INFO - Received callback for node 69403392575bc060ce15710c INFO: 172.21.0.1:39118 - "POST /node/69403392575bc060ce15710c HTTP/1.0" 202 Accepted 2025-12-17 16:13:19,016 - lava_callback - INFO - Starting async job submit for node 69403392575bc060ce15710c 2025-12-17 16:13:19,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:19,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403392575bc060ce15710c HTTP/1.1" 200 None 2025-12-17 16:13:19,027 - lava_callback - DEBUG - Processing node 69403392575bc060ce15710c: baseline-arm-baylibre 2025-12-17 16:13:19,027 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403392575bc060ce15710c 2025-12-17 16:13:19,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:19,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403392575bc060ce15710c HTTP/1.1" 200 None 2025-12-17 16:13:19,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:19,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:19,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:19,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403392575bc060ce15710c HTTP/1.1" 200 None 2025-12-17 16:13:19,068 - lava_callback - INFO - Completed processing callback for node 69403392575bc060ce15710c 2025-12-17 16:13:19,669 - lava_callback - INFO - Received callback for node 69403393575bc060ce15710f INFO: 172.21.0.1:39132 - "POST /node/69403393575bc060ce15710f HTTP/1.0" 202 Accepted 2025-12-17 16:13:19,675 - lava_callback - INFO - Starting async job submit for node 69403393575bc060ce15710f 2025-12-17 16:13:19,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:19,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403393575bc060ce15710f HTTP/1.1" 200 None 2025-12-17 16:13:19,686 - lava_callback - DEBUG - Processing node 69403393575bc060ce15710f: baseline-arm-baylibre 2025-12-17 16:13:19,686 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403393575bc060ce15710f 2025-12-17 16:13:19,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:19,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403393575bc060ce15710f HTTP/1.1" 200 None 2025-12-17 16:13:19,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:19,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:19,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:19,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403393575bc060ce15710f HTTP/1.1" 200 None 2025-12-17 16:13:19,725 - lava_callback - INFO - Completed processing callback for node 69403393575bc060ce15710f 2025-12-17 16:13:20,319 - lava_callback - INFO - Received callback for node 69403395575bc060ce157112 INFO: 172.21.0.1:39500 - "POST /node/69403395575bc060ce157112 HTTP/1.0" 202 Accepted 2025-12-17 16:13:20,325 - lava_callback - INFO - Starting async job submit for node 69403395575bc060ce157112 2025-12-17 16:13:20,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:20,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403395575bc060ce157112 HTTP/1.1" 200 None 2025-12-17 16:13:20,339 - lava_callback - DEBUG - Processing node 69403395575bc060ce157112: baseline-arm-baylibre 2025-12-17 16:13:20,340 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403395575bc060ce157112 2025-12-17 16:13:20,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:20,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403395575bc060ce157112 HTTP/1.1" 200 None 2025-12-17 16:13:20,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:20,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:20,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:20,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403395575bc060ce157112 HTTP/1.1" 200 None 2025-12-17 16:13:20,399 - lava_callback - INFO - Completed processing callback for node 69403395575bc060ce157112 2025-12-17 16:13:24,755 - lava_callback - INFO - Received callback for node 69403963575bc060ce1580dd INFO: 172.21.0.1:39516 - "POST /node/69403963575bc060ce1580dd HTTP/1.0" 202 Accepted 2025-12-17 16:13:24,764 - lava_callback - INFO - Starting async job submit for node 69403963575bc060ce1580dd 2025-12-17 16:13:24,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:24,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403963575bc060ce1580dd HTTP/1.1" 200 None 2025-12-17 16:13:24,876 - lava_callback - DEBUG - Processing node 69403963575bc060ce1580dd: baseline-nfs-arm64 2025-12-17 16:13:26,047 - lava_callback - INFO - Received callback for node 69403396575bc060ce157115 2025-12-17 16:13:26,068 - lava_callback - INFO - Starting async job submit for node 69403396575bc060ce157115 2025-12-17 16:13:26,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:39520 - "POST /node/69403396575bc060ce157115 HTTP/1.0" 202 Accepted 2025-12-17 16:13:26,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403396575bc060ce157115 HTTP/1.1" 200 None 2025-12-17 16:13:26,151 - lava_callback - DEBUG - Processing node 69403396575bc060ce157115: baseline-arm-broonie 2025-12-17 16:13:26,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403396575bc060ce157115 2025-12-17 16:13:26,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:26,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403396575bc060ce157115 HTTP/1.1" 200 None 2025-12-17 16:13:26,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:26,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:13:26,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:26,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:26,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403396575bc060ce157115 HTTP/1.1" 200 None 2025-12-17 16:13:26,284 - lava_callback - INFO - Completed processing callback for node 69403396575bc060ce157115 INFO: 172.21.0.1:39522 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:13:26,510 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:13:26,511 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69403963575bc060ce1580dd/log.txt.gz 2025-12-17 16:13:26,511 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:13:26,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:13:26,733 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:13:26,734 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69403963575bc060ce1580dd/lava_callback.json.gz 2025-12-17 16:13:26,875 - lava_callback - INFO - Received callback for node 69403397575bc060ce157118 INFO: 172.21.0.1:39536 - "POST /node/69403397575bc060ce157118 HTTP/1.0" 202 Accepted 2025-12-17 16:13:26,888 - lava_callback - INFO - Starting async job submit for node 69403397575bc060ce157118 2025-12-17 16:13:26,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 DEBUG: Setup failed for 69403963575bc060ce1580dd. Transitting job node result: incomplete -> fail 2025-12-17 16:13:26,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:26,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403397575bc060ce157118 HTTP/1.1" 200 None 2025-12-17 16:13:26,908 - lava_callback - DEBUG - Processing node 69403397575bc060ce157118: baseline-arm-broonie 2025-12-17 16:13:26,908 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403397575bc060ce157118 2025-12-17 16:13:26,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:26,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403963575bc060ce1580dd HTTP/1.1" 200 None 2025-12-17 16:13:26,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:26,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403397575bc060ce157118 HTTP/1.1" 200 None 2025-12-17 16:13:26,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:26,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d1eabfad9ee307f5cfc12 HTTP/1.1" 200 None 2025-12-17 16:13:26,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:26,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:26,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:26,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403397575bc060ce157118 HTTP/1.1" 200 None 2025-12-17 16:13:26,952 - lava_callback - INFO - Completed processing callback for node 69403397575bc060ce157118 2025-12-17 16:13:26,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403963575bc060ce1580dd HTTP/1.1" 200 None 2025-12-17 16:13:26,989 - lava_callback - INFO - Completed processing callback for node 69403963575bc060ce1580dd 2025-12-17 16:13:27,638 - lava_callback - INFO - Received callback for node 69403398575bc060ce15711b INFO: 172.21.0.1:39552 - "POST /node/69403398575bc060ce15711b HTTP/1.0" 202 Accepted 2025-12-17 16:13:27,644 - lava_callback - INFO - Starting async job submit for node 69403398575bc060ce15711b 2025-12-17 16:13:27,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:27,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403398575bc060ce15711b HTTP/1.1" 200 None 2025-12-17 16:13:27,653 - lava_callback - DEBUG - Processing node 69403398575bc060ce15711b: baseline-arm-broonie 2025-12-17 16:13:27,653 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403398575bc060ce15711b 2025-12-17 16:13:27,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:27,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403398575bc060ce15711b HTTP/1.1" 200 None 2025-12-17 16:13:27,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:27,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:27,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:27,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403398575bc060ce15711b HTTP/1.1" 200 None 2025-12-17 16:13:27,684 - lava_callback - INFO - Completed processing callback for node 69403398575bc060ce15711b 2025-12-17 16:13:28,986 - lava_callback - INFO - Received callback for node 6940339d575bc060ce157124 INFO: 172.21.0.1:39554 - "POST /node/6940339d575bc060ce157124 HTTP/1.0" 202 Accepted 2025-12-17 16:13:28,992 - lava_callback - INFO - Starting async job submit for node 6940339d575bc060ce157124 2025-12-17 16:13:28,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:29,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940339d575bc060ce157124 HTTP/1.1" 200 None 2025-12-17 16:13:29,002 - lava_callback - DEBUG - Processing node 6940339d575bc060ce157124: baseline-arm-clabbe 2025-12-17 16:13:29,002 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940339d575bc060ce157124 2025-12-17 16:13:29,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:29,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940339d575bc060ce157124 HTTP/1.1" 200 None 2025-12-17 16:13:29,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:29,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:29,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:29,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940339d575bc060ce157124 HTTP/1.1" 200 None 2025-12-17 16:13:29,037 - lava_callback - INFO - Completed processing callback for node 6940339d575bc060ce157124 2025-12-17 16:13:29,666 - lava_callback - INFO - Received callback for node 6940339f575bc060ce157127 INFO: 172.21.0.1:39560 - "POST /node/6940339f575bc060ce157127 HTTP/1.0" 202 Accepted 2025-12-17 16:13:29,673 - lava_callback - INFO - Starting async job submit for node 6940339f575bc060ce157127 2025-12-17 16:13:29,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:29,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940339f575bc060ce157127 HTTP/1.1" 200 None 2025-12-17 16:13:29,682 - lava_callback - DEBUG - Processing node 6940339f575bc060ce157127: baseline-arm-clabbe 2025-12-17 16:13:29,682 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940339f575bc060ce157127 2025-12-17 16:13:29,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:29,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940339f575bc060ce157127 HTTP/1.1" 200 None 2025-12-17 16:13:29,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:29,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:29,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:29,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940339f575bc060ce157127 HTTP/1.1" 200 None 2025-12-17 16:13:29,728 - lava_callback - INFO - Completed processing callback for node 6940339f575bc060ce157127 2025-12-17 16:13:30,323 - lava_callback - INFO - Received callback for node 694033a1575bc060ce15712a INFO: 172.21.0.1:51118 - "POST /node/694033a1575bc060ce15712a HTTP/1.0" 202 Accepted 2025-12-17 16:13:30,329 - lava_callback - INFO - Starting async job submit for node 694033a1575bc060ce15712a 2025-12-17 16:13:30,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:30,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033a1575bc060ce15712a HTTP/1.1" 200 None 2025-12-17 16:13:30,342 - lava_callback - DEBUG - Processing node 694033a1575bc060ce15712a: baseline-arm-clabbe 2025-12-17 16:13:30,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033a1575bc060ce15712a 2025-12-17 16:13:30,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:30,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033a1575bc060ce15712a HTTP/1.1" 200 None 2025-12-17 16:13:30,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:30,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:30,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:30,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033a1575bc060ce15712a HTTP/1.1" 200 None 2025-12-17 16:13:30,377 - lava_callback - INFO - Completed processing callback for node 694033a1575bc060ce15712a 2025-12-17 16:13:30,970 - lava_callback - INFO - Received callback for node 694033a2575bc060ce15712d INFO: 172.21.0.1:51120 - "POST /node/694033a2575bc060ce15712d HTTP/1.0" 202 Accepted 2025-12-17 16:13:30,977 - lava_callback - INFO - Starting async job submit for node 694033a2575bc060ce15712d 2025-12-17 16:13:30,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:30,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033a2575bc060ce15712d HTTP/1.1" 200 None 2025-12-17 16:13:30,986 - lava_callback - DEBUG - Processing node 694033a2575bc060ce15712d: baseline-arm-clabbe 2025-12-17 16:13:30,986 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033a2575bc060ce15712d 2025-12-17 16:13:30,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:30,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033a2575bc060ce15712d HTTP/1.1" 200 None 2025-12-17 16:13:30,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:31,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:31,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:31,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033a2575bc060ce15712d HTTP/1.1" 200 None 2025-12-17 16:13:31,033 - lava_callback - INFO - Completed processing callback for node 694033a2575bc060ce15712d 2025-12-17 16:13:31,643 - lava_callback - INFO - Received callback for node 694033a3575bc060ce157132 INFO: 172.21.0.1:51126 - "POST /node/694033a3575bc060ce157132 HTTP/1.0" 202 Accepted 2025-12-17 16:13:31,649 - lava_callback - INFO - Starting async job submit for node 694033a3575bc060ce157132 2025-12-17 16:13:31,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:31,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033a3575bc060ce157132 HTTP/1.1" 200 None 2025-12-17 16:13:31,658 - lava_callback - DEBUG - Processing node 694033a3575bc060ce157132: baseline-arm-clabbe 2025-12-17 16:13:31,658 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033a3575bc060ce157132 2025-12-17 16:13:31,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:31,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033a3575bc060ce157132 HTTP/1.1" 200 None 2025-12-17 16:13:31,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:31,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:31,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:31,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033a3575bc060ce157132 HTTP/1.1" 200 None 2025-12-17 16:13:31,690 - lava_callback - INFO - Completed processing callback for node 694033a3575bc060ce157132 2025-12-17 16:13:32,300 - lava_callback - INFO - Received callback for node 694033a5575bc060ce15713a INFO: 172.21.0.1:51132 - "POST /node/694033a5575bc060ce15713a HTTP/1.0" 202 Accepted 2025-12-17 16:13:32,308 - lava_callback - INFO - Starting async job submit for node 694033a5575bc060ce15713a 2025-12-17 16:13:32,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:32,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033a5575bc060ce15713a HTTP/1.1" 200 None 2025-12-17 16:13:32,318 - lava_callback - DEBUG - Processing node 694033a5575bc060ce15713a: baseline-arm-clabbe 2025-12-17 16:13:32,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033a5575bc060ce15713a 2025-12-17 16:13:32,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:32,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033a5575bc060ce15713a HTTP/1.1" 200 None 2025-12-17 16:13:32,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:32,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:32,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:32,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033a5575bc060ce15713a HTTP/1.1" 200 None 2025-12-17 16:13:32,371 - lava_callback - INFO - Completed processing callback for node 694033a5575bc060ce15713a 2025-12-17 16:13:32,978 - lava_callback - INFO - Received callback for node 694033a6575bc060ce157142 INFO: 172.21.0.1:51148 - "POST /node/694033a6575bc060ce157142 HTTP/1.0" 202 Accepted 2025-12-17 16:13:32,984 - lava_callback - INFO - Starting async job submit for node 694033a6575bc060ce157142 2025-12-17 16:13:32,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:32,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033a6575bc060ce157142 HTTP/1.1" 200 None 2025-12-17 16:13:32,993 - lava_callback - DEBUG - Processing node 694033a6575bc060ce157142: baseline-arm-clabbe 2025-12-17 16:13:32,993 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033a6575bc060ce157142 2025-12-17 16:13:32,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:33,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033a6575bc060ce157142 HTTP/1.1" 200 None 2025-12-17 16:13:33,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:33,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:33,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:33,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033a6575bc060ce157142 HTTP/1.1" 200 None 2025-12-17 16:13:33,029 - lava_callback - INFO - Completed processing callback for node 694033a6575bc060ce157142 2025-12-17 16:13:33,621 - lava_callback - INFO - Received callback for node 694033a8575bc060ce157146 INFO: 172.21.0.1:51150 - "POST /node/694033a8575bc060ce157146 HTTP/1.0" 202 Accepted 2025-12-17 16:13:33,628 - lava_callback - INFO - Starting async job submit for node 694033a8575bc060ce157146 2025-12-17 16:13:33,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:33,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033a8575bc060ce157146 HTTP/1.1" 200 None 2025-12-17 16:13:33,638 - lava_callback - DEBUG - Processing node 694033a8575bc060ce157146: baseline-arm-clabbe 2025-12-17 16:13:33,638 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033a8575bc060ce157146 2025-12-17 16:13:33,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:33,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033a8575bc060ce157146 HTTP/1.1" 200 None 2025-12-17 16:13:33,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:33,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:33,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:33,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033a8575bc060ce157146 HTTP/1.1" 200 None 2025-12-17 16:13:33,694 - lava_callback - INFO - Completed processing callback for node 694033a8575bc060ce157146 INFO: 172.21.0.1:51152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36530 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:13:48,996 - lava_callback - INFO - Received callback for node 694033a9575bc060ce157149 INFO: 172.21.0.1:36540 - "POST /node/694033a9575bc060ce157149 HTTP/1.0" 202 Accepted 2025-12-17 16:13:49,004 - lava_callback - INFO - Starting async job submit for node 694033a9575bc060ce157149 2025-12-17 16:13:49,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:49,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033a9575bc060ce157149 HTTP/1.1" 200 None 2025-12-17 16:13:49,015 - lava_callback - DEBUG - Processing node 694033a9575bc060ce157149: baseline-arm-clabbe 2025-12-17 16:13:49,015 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033a9575bc060ce157149 2025-12-17 16:13:49,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:49,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033a9575bc060ce157149 HTTP/1.1" 200 None 2025-12-17 16:13:49,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:49,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:49,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:49,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033a9575bc060ce157149 HTTP/1.1" 200 None 2025-12-17 16:13:49,060 - lava_callback - INFO - Completed processing callback for node 694033a9575bc060ce157149 2025-12-17 16:13:49,666 - lava_callback - INFO - Received callback for node 694033aa575bc060ce15714c INFO: 172.21.0.1:36556 - "POST /node/694033aa575bc060ce15714c HTTP/1.0" 202 Accepted 2025-12-17 16:13:49,675 - lava_callback - INFO - Starting async job submit for node 694033aa575bc060ce15714c 2025-12-17 16:13:49,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:49,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033aa575bc060ce15714c HTTP/1.1" 200 None 2025-12-17 16:13:49,912 - lava_callback - DEBUG - Processing node 694033aa575bc060ce15714c: baseline-arm-clabbe 2025-12-17 16:13:49,912 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033aa575bc060ce15714c 2025-12-17 16:13:49,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:49,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033aa575bc060ce15714c HTTP/1.1" 200 None 2025-12-17 16:13:49,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:49,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:49,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:49,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033aa575bc060ce15714c HTTP/1.1" 200 None 2025-12-17 16:13:49,950 - lava_callback - INFO - Completed processing callback for node 694033aa575bc060ce15714c 2025-12-17 16:13:50,333 - lava_callback - INFO - Received callback for node 694033ab575bc060ce15714f INFO: 172.21.0.1:49120 - "POST /node/694033ab575bc060ce15714f HTTP/1.0" 202 Accepted 2025-12-17 16:13:50,340 - lava_callback - INFO - Starting async job submit for node 694033ab575bc060ce15714f 2025-12-17 16:13:50,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:50,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033ab575bc060ce15714f HTTP/1.1" 200 None 2025-12-17 16:13:50,349 - lava_callback - DEBUG - Processing node 694033ab575bc060ce15714f: baseline-arm-clabbe 2025-12-17 16:13:50,350 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033ab575bc060ce15714f 2025-12-17 16:13:50,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:50,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033ab575bc060ce15714f HTTP/1.1" 200 None 2025-12-17 16:13:50,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:50,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:50,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:50,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033ab575bc060ce15714f HTTP/1.1" 200 None 2025-12-17 16:13:50,380 - lava_callback - INFO - Completed processing callback for node 694033ab575bc060ce15714f 2025-12-17 16:13:50,999 - lava_callback - INFO - Received callback for node 694033ad575bc060ce157152 INFO: 172.21.0.1:49126 - "POST /node/694033ad575bc060ce157152 HTTP/1.0" 202 Accepted 2025-12-17 16:13:51,007 - lava_callback - INFO - Starting async job submit for node 694033ad575bc060ce157152 2025-12-17 16:13:51,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:51,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033ad575bc060ce157152 HTTP/1.1" 200 None 2025-12-17 16:13:51,018 - lava_callback - DEBUG - Processing node 694033ad575bc060ce157152: baseline-arm-clabbe 2025-12-17 16:13:51,019 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033ad575bc060ce157152 2025-12-17 16:13:51,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:51,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033ad575bc060ce157152 HTTP/1.1" 200 None 2025-12-17 16:13:51,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:51,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:13:51,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:51,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033ad575bc060ce157152 HTTP/1.1" 200 None 2025-12-17 16:13:51,060 - lava_callback - INFO - Completed processing callback for node 694033ad575bc060ce157152 2025-12-17 16:13:54,369 - lava_callback - INFO - Received callback for node 694033ae575bc060ce157155 INFO: 172.21.0.1:49138 - "POST /node/694033ae575bc060ce157155 HTTP/1.0" 202 Accepted 2025-12-17 16:13:54,378 - lava_callback - INFO - Starting async job submit for node 694033ae575bc060ce157155 2025-12-17 16:13:54,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:54,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033ae575bc060ce157155 HTTP/1.1" 200 None 2025-12-17 16:13:54,401 - lava_callback - DEBUG - Processing node 694033ae575bc060ce157155: baseline-arm64-baylibre 2025-12-17 16:13:54,402 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033ae575bc060ce157155 2025-12-17 16:13:54,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:54,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033ae575bc060ce157155 HTTP/1.1" 200 None 2025-12-17 16:13:54,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:54,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:13:54,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:54,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033ae575bc060ce157155 HTTP/1.1" 200 None 2025-12-17 16:13:54,449 - lava_callback - INFO - Completed processing callback for node 694033ae575bc060ce157155 2025-12-17 16:13:55,030 - lava_callback - INFO - Received callback for node 694033b1575bc060ce157158 INFO: 172.21.0.1:49144 - "POST /node/694033b1575bc060ce157158 HTTP/1.0" 202 Accepted 2025-12-17 16:13:55,038 - lava_callback - INFO - Starting async job submit for node 694033b1575bc060ce157158 2025-12-17 16:13:55,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:55,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033b1575bc060ce157158 HTTP/1.1" 200 None 2025-12-17 16:13:55,047 - lava_callback - DEBUG - Processing node 694033b1575bc060ce157158: baseline-arm64-baylibre 2025-12-17 16:13:55,047 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033b1575bc060ce157158 2025-12-17 16:13:55,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:55,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033b1575bc060ce157158 HTTP/1.1" 200 None 2025-12-17 16:13:55,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:55,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:13:55,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:55,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033b1575bc060ce157158 HTTP/1.1" 200 None 2025-12-17 16:13:55,081 - lava_callback - INFO - Completed processing callback for node 694033b1575bc060ce157158 2025-12-17 16:13:55,702 - lava_callback - INFO - Received callback for node 694033b3575bc060ce15715b INFO: 172.21.0.1:49154 - "POST /node/694033b3575bc060ce15715b HTTP/1.0" 202 Accepted 2025-12-17 16:13:55,707 - lava_callback - INFO - Starting async job submit for node 694033b3575bc060ce15715b 2025-12-17 16:13:55,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:55,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033b3575bc060ce15715b HTTP/1.1" 200 None 2025-12-17 16:13:55,717 - lava_callback - DEBUG - Processing node 694033b3575bc060ce15715b: baseline-arm64-baylibre 2025-12-17 16:13:55,717 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033b3575bc060ce15715b 2025-12-17 16:13:55,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:55,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033b3575bc060ce15715b HTTP/1.1" 200 None 2025-12-17 16:13:55,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:55,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:13:55,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:55,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033b3575bc060ce15715b HTTP/1.1" 200 None 2025-12-17 16:13:55,797 - lava_callback - INFO - Completed processing callback for node 694033b3575bc060ce15715b 2025-12-17 16:13:56,371 - lava_callback - INFO - Received callback for node 694033b4575bc060ce15715e INFO: 172.21.0.1:49160 - "POST /node/694033b4575bc060ce15715e HTTP/1.0" 202 Accepted 2025-12-17 16:13:56,378 - lava_callback - INFO - Starting async job submit for node 694033b4575bc060ce15715e 2025-12-17 16:13:56,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:56,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033b4575bc060ce15715e HTTP/1.1" 200 None 2025-12-17 16:13:56,387 - lava_callback - DEBUG - Processing node 694033b4575bc060ce15715e: baseline-arm64-baylibre 2025-12-17 16:13:56,387 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033b4575bc060ce15715e 2025-12-17 16:13:56,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:56,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033b4575bc060ce15715e HTTP/1.1" 200 None 2025-12-17 16:13:56,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:56,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:13:56,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:56,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033b4575bc060ce15715e HTTP/1.1" 200 None 2025-12-17 16:13:56,431 - lava_callback - INFO - Completed processing callback for node 694033b4575bc060ce15715e INFO: 172.21.0.1:49166 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:13:57,039 - lava_callback - INFO - Received callback for node 694033b7575bc060ce157161 INFO: 172.21.0.1:49176 - "POST /node/694033b7575bc060ce157161 HTTP/1.0" 202 Accepted 2025-12-17 16:13:57,045 - lava_callback - INFO - Starting async job submit for node 694033b7575bc060ce157161 2025-12-17 16:13:57,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:57,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033b7575bc060ce157161 HTTP/1.1" 200 None 2025-12-17 16:13:57,054 - lava_callback - DEBUG - Processing node 694033b7575bc060ce157161: baseline-arm64-baylibre 2025-12-17 16:13:57,055 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033b7575bc060ce157161 2025-12-17 16:13:57,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:57,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033b7575bc060ce157161 HTTP/1.1" 200 None 2025-12-17 16:13:57,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:57,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:13:57,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:57,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033b7575bc060ce157161 HTTP/1.1" 200 None 2025-12-17 16:13:57,085 - lava_callback - INFO - Completed processing callback for node 694033b7575bc060ce157161 2025-12-17 16:13:57,702 - lava_callback - INFO - Received callback for node 694033bb575bc060ce157173 INFO: 172.21.0.1:49186 - "POST /node/694033bb575bc060ce157173 HTTP/1.0" 202 Accepted 2025-12-17 16:13:57,709 - lava_callback - INFO - Starting async job submit for node 694033bb575bc060ce157173 2025-12-17 16:13:57,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:57,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033bb575bc060ce157173 HTTP/1.1" 200 None 2025-12-17 16:13:57,718 - lava_callback - DEBUG - Processing node 694033bb575bc060ce157173: baseline-arm64-baylibre 2025-12-17 16:13:57,718 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033bb575bc060ce157173 2025-12-17 16:13:57,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:57,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033bb575bc060ce157173 HTTP/1.1" 200 None 2025-12-17 16:13:57,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:57,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:13:57,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:57,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033bb575bc060ce157173 HTTP/1.1" 200 None 2025-12-17 16:13:57,755 - lava_callback - INFO - Completed processing callback for node 694033bb575bc060ce157173 2025-12-17 16:13:58,344 - lava_callback - INFO - Received callback for node 694033bd575bc060ce157176 INFO: 172.21.0.1:49188 - "POST /node/694033bd575bc060ce157176 HTTP/1.0" 202 Accepted 2025-12-17 16:13:58,350 - lava_callback - INFO - Starting async job submit for node 694033bd575bc060ce157176 2025-12-17 16:13:58,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:58,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033bd575bc060ce157176 HTTP/1.1" 200 None 2025-12-17 16:13:58,360 - lava_callback - DEBUG - Processing node 694033bd575bc060ce157176: baseline-arm64-baylibre 2025-12-17 16:13:58,360 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033bd575bc060ce157176 2025-12-17 16:13:58,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:58,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033bd575bc060ce157176 HTTP/1.1" 200 None 2025-12-17 16:13:58,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:58,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:13:58,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:58,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033bd575bc060ce157176 HTTP/1.1" 200 None 2025-12-17 16:13:58,394 - lava_callback - INFO - Completed processing callback for node 694033bd575bc060ce157176 2025-12-17 16:13:59,000 - lava_callback - INFO - Received callback for node 694033bf575bc060ce157179 INFO: 172.21.0.1:49198 - "POST /node/694033bf575bc060ce157179 HTTP/1.0" 202 Accepted 2025-12-17 16:13:59,009 - lava_callback - INFO - Starting async job submit for node 694033bf575bc060ce157179 2025-12-17 16:13:59,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:59,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033bf575bc060ce157179 HTTP/1.1" 200 None 2025-12-17 16:13:59,019 - lava_callback - DEBUG - Processing node 694033bf575bc060ce157179: baseline-arm64-baylibre 2025-12-17 16:13:59,019 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033bf575bc060ce157179 2025-12-17 16:13:59,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:59,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033bf575bc060ce157179 HTTP/1.1" 200 None 2025-12-17 16:13:59,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:59,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:13:59,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:13:59,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033bf575bc060ce157179 HTTP/1.1" 200 None 2025-12-17 16:13:59,065 - lava_callback - INFO - Completed processing callback for node 694033bf575bc060ce157179 2025-12-17 16:14:00,728 - lava_callback - INFO - Received callback for node 694034bb575bc060ce1573a7 INFO: 172.21.0.1:38250 - "POST /node/694034bb575bc060ce1573a7 HTTP/1.0" 202 Accepted 2025-12-17 16:14:00,738 - lava_callback - INFO - Starting async job submit for node 694034bb575bc060ce1573a7 2025-12-17 16:14:00,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:00,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034bb575bc060ce1573a7 HTTP/1.1" 200 None 2025-12-17 16:14:00,773 - lava_callback - DEBUG - Processing node 694034bb575bc060ce1573a7: baseline-arm 2025-12-17 16:14:00,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:14:01,121 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:14:01,122 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-694034bb575bc060ce1573a7/log.txt.gz 2025-12-17 16:14:01,122 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:14:01,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:14:01,256 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:14:01,257 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-694034bb575bc060ce1573a7/lava_callback.json.gz 2025-12-17 16:14:01,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:01,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034bb575bc060ce1573a7 HTTP/1.1" 200 None 2025-12-17 16:14:01,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:01,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:14:01,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:01,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034bb575bc060ce1573a7 HTTP/1.1" 200 None 2025-12-17 16:14:01,327 - lava_callback - INFO - Completed processing callback for node 694034bb575bc060ce1573a7 2025-12-17 16:14:02,198 - lava_callback - INFO - Received callback for node 6942a79c1b61f1c6005fd563 INFO: 172.21.0.1:38262 - "POST /node/6942a79c1b61f1c6005fd563 HTTP/1.0" 202 Accepted 2025-12-17 16:14:02,206 - lava_callback - INFO - Starting async job submit for node 6942a79c1b61f1c6005fd563 2025-12-17 16:14:02,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:02,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942a79c1b61f1c6005fd563 HTTP/1.1" 200 None 2025-12-17 16:14:02,248 - lava_callback - DEBUG - Processing node 6942a79c1b61f1c6005fd563: baseline-arm-broonie 2025-12-17 16:14:02,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:14:02,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:14:02,665 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-6942a79c1b61f1c6005fd563/log.txt.gz 2025-12-17 16:14:02,666 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:14:02,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:14:02,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:14:02,821 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-6942a79c1b61f1c6005fd563/lava_callback.json.gz 2025-12-17 16:14:02,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:02,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942a79c1b61f1c6005fd563 HTTP/1.1" 200 None 2025-12-17 16:14:02,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:02,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941f6b6bb27571da24aebf5 HTTP/1.1" 200 None 2025-12-17 16:14:02,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:02,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942a79c1b61f1c6005fd563 HTTP/1.1" 200 None 2025-12-17 16:14:02,921 - lava_callback - INFO - Completed processing callback for node 6942a79c1b61f1c6005fd563 INFO: 172.21.0.1:38270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58454 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:14:14,410 - lava_callback - INFO - Received callback for node 694033c9575bc060ce157188 INFO: 172.21.0.1:58468 - "POST /node/694033c9575bc060ce157188 HTTP/1.0" 202 Accepted 2025-12-17 16:14:14,417 - lava_callback - INFO - Starting async job submit for node 694033c9575bc060ce157188 2025-12-17 16:14:14,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:14,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033c9575bc060ce157188 HTTP/1.1" 200 None 2025-12-17 16:14:14,425 - lava_callback - DEBUG - Processing node 694033c9575bc060ce157188: baseline-arm64-baylibre 2025-12-17 16:14:14,425 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033c9575bc060ce157188 2025-12-17 16:14:14,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:14,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033c9575bc060ce157188 HTTP/1.1" 200 None 2025-12-17 16:14:14,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:14,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:14,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:14,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033c9575bc060ce157188 HTTP/1.1" 200 None 2025-12-17 16:14:14,459 - lava_callback - INFO - Completed processing callback for node 694033c9575bc060ce157188 2025-12-17 16:14:15,073 - lava_callback - INFO - Received callback for node 694033c1575bc060ce15717c INFO: 172.21.0.1:58482 - "POST /node/694033c1575bc060ce15717c HTTP/1.0" 202 Accepted 2025-12-17 16:14:15,080 - lava_callback - INFO - Starting async job submit for node 694033c1575bc060ce15717c 2025-12-17 16:14:15,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:15,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033c1575bc060ce15717c HTTP/1.1" 200 None 2025-12-17 16:14:15,088 - lava_callback - DEBUG - Processing node 694033c1575bc060ce15717c: baseline-arm64-baylibre 2025-12-17 16:14:15,089 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033c1575bc060ce15717c 2025-12-17 16:14:15,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:15,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033c1575bc060ce15717c HTTP/1.1" 200 None 2025-12-17 16:14:15,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:15,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:15,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:15,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033c1575bc060ce15717c HTTP/1.1" 200 None 2025-12-17 16:14:15,147 - lava_callback - INFO - Completed processing callback for node 694033c1575bc060ce15717c INFO: 172.21.0.1:58490 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:14:15,739 - lava_callback - INFO - Received callback for node 694033c3575bc060ce15717f INFO: 172.21.0.1:58498 - "POST /node/694033c3575bc060ce15717f HTTP/1.0" 202 Accepted 2025-12-17 16:14:15,747 - lava_callback - INFO - Starting async job submit for node 694033c3575bc060ce15717f 2025-12-17 16:14:15,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:15,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033c3575bc060ce15717f HTTP/1.1" 200 None 2025-12-17 16:14:15,760 - lava_callback - DEBUG - Processing node 694033c3575bc060ce15717f: baseline-arm64-baylibre 2025-12-17 16:14:15,760 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033c3575bc060ce15717f 2025-12-17 16:14:15,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:15,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033c3575bc060ce15717f HTTP/1.1" 200 None 2025-12-17 16:14:15,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:15,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:15,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:15,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033c3575bc060ce15717f HTTP/1.1" 200 None 2025-12-17 16:14:15,796 - lava_callback - INFO - Completed processing callback for node 694033c3575bc060ce15717f 2025-12-17 16:14:16,397 - lava_callback - INFO - Received callback for node 694033c5575bc060ce157182 INFO: 172.21.0.1:58500 - "POST /node/694033c5575bc060ce157182 HTTP/1.0" 202 Accepted 2025-12-17 16:14:16,403 - lava_callback - INFO - Starting async job submit for node 694033c5575bc060ce157182 2025-12-17 16:14:16,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:16,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033c5575bc060ce157182 HTTP/1.1" 200 None 2025-12-17 16:14:16,412 - lava_callback - DEBUG - Processing node 694033c5575bc060ce157182: baseline-arm64-baylibre 2025-12-17 16:14:16,412 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033c5575bc060ce157182 2025-12-17 16:14:16,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:16,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033c5575bc060ce157182 HTTP/1.1" 200 None 2025-12-17 16:14:16,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:16,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:16,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:16,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033c5575bc060ce157182 HTTP/1.1" 200 None 2025-12-17 16:14:16,445 - lava_callback - INFO - Completed processing callback for node 694033c5575bc060ce157182 INFO: 172.21.0.1:58506 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:14:17,070 - lava_callback - INFO - Received callback for node 694033c7575bc060ce157185 INFO: 172.21.0.1:58514 - "POST /node/694033c7575bc060ce157185 HTTP/1.0" 202 Accepted 2025-12-17 16:14:17,076 - lava_callback - INFO - Starting async job submit for node 694033c7575bc060ce157185 2025-12-17 16:14:17,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:17,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033c7575bc060ce157185 HTTP/1.1" 200 None 2025-12-17 16:14:17,105 - lava_callback - DEBUG - Processing node 694033c7575bc060ce157185: baseline-arm64-baylibre 2025-12-17 16:14:17,105 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033c7575bc060ce157185 2025-12-17 16:14:17,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:17,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033c7575bc060ce157185 HTTP/1.1" 200 None 2025-12-17 16:14:17,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:17,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:17,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:17,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033c7575bc060ce157185 HTTP/1.1" 200 None 2025-12-17 16:14:17,143 - lava_callback - INFO - Completed processing callback for node 694033c7575bc060ce157185 2025-12-17 16:14:17,752 - lava_callback - INFO - Received callback for node 694033cb575bc060ce15718d INFO: 172.21.0.1:58518 - "POST /node/694033cb575bc060ce15718d HTTP/1.0" 202 Accepted 2025-12-17 16:14:17,764 - lava_callback - INFO - Starting async job submit for node 694033cb575bc060ce15718d 2025-12-17 16:14:17,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:17,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033cb575bc060ce15718d HTTP/1.1" 200 None 2025-12-17 16:14:17,783 - lava_callback - DEBUG - Processing node 694033cb575bc060ce15718d: baseline-arm64-baylibre 2025-12-17 16:14:17,783 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033cb575bc060ce15718d 2025-12-17 16:14:17,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:17,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033cb575bc060ce15718d HTTP/1.1" 200 None 2025-12-17 16:14:17,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:17,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:17,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:17,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033cb575bc060ce15718d HTTP/1.1" 200 None 2025-12-17 16:14:17,828 - lava_callback - INFO - Completed processing callback for node 694033cb575bc060ce15718d 2025-12-17 16:14:18,424 - lava_callback - INFO - Received callback for node 694033cd575bc060ce157196 INFO: 172.21.0.1:58530 - "POST /node/694033cd575bc060ce157196 HTTP/1.0" 202 Accepted 2025-12-17 16:14:18,431 - lava_callback - INFO - Starting async job submit for node 694033cd575bc060ce157196 2025-12-17 16:14:18,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:18,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033cd575bc060ce157196 HTTP/1.1" 200 None 2025-12-17 16:14:18,445 - lava_callback - DEBUG - Processing node 694033cd575bc060ce157196: baseline-arm64-baylibre 2025-12-17 16:14:18,445 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033cd575bc060ce157196 2025-12-17 16:14:18,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:18,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033cd575bc060ce157196 HTTP/1.1" 200 None 2025-12-17 16:14:18,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:18,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:18,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:18,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033cd575bc060ce157196 HTTP/1.1" 200 None 2025-12-17 16:14:18,482 - lava_callback - INFO - Completed processing callback for node 694033cd575bc060ce157196 2025-12-17 16:14:19,090 - lava_callback - INFO - Received callback for node 694033d1575bc060ce1571a2 INFO: 172.21.0.1:58544 - "POST /node/694033d1575bc060ce1571a2 HTTP/1.0" 202 Accepted 2025-12-17 16:14:19,097 - lava_callback - INFO - Starting async job submit for node 694033d1575bc060ce1571a2 2025-12-17 16:14:19,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:19,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033d1575bc060ce1571a2 HTTP/1.1" 200 None 2025-12-17 16:14:19,106 - lava_callback - DEBUG - Processing node 694033d1575bc060ce1571a2: baseline-arm64-baylibre 2025-12-17 16:14:19,106 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033d1575bc060ce1571a2 2025-12-17 16:14:19,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:19,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033d1575bc060ce1571a2 HTTP/1.1" 200 None 2025-12-17 16:14:19,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:19,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:19,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:19,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033d1575bc060ce1571a2 HTTP/1.1" 200 None 2025-12-17 16:14:19,141 - lava_callback - INFO - Completed processing callback for node 694033d1575bc060ce1571a2 2025-12-17 16:14:19,766 - lava_callback - INFO - Received callback for node 694033d3575bc060ce1571a5 INFO: 172.21.0.1:52078 - "POST /node/694033d3575bc060ce1571a5 HTTP/1.0" 202 Accepted 2025-12-17 16:14:19,774 - lava_callback - INFO - Starting async job submit for node 694033d3575bc060ce1571a5 2025-12-17 16:14:19,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:19,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033d3575bc060ce1571a5 HTTP/1.1" 200 None 2025-12-17 16:14:19,782 - lava_callback - DEBUG - Processing node 694033d3575bc060ce1571a5: baseline-arm64-baylibre 2025-12-17 16:14:19,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033d3575bc060ce1571a5 2025-12-17 16:14:19,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:19,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033d3575bc060ce1571a5 HTTP/1.1" 200 None 2025-12-17 16:14:19,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:19,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:19,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:19,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033d3575bc060ce1571a5 HTTP/1.1" 200 None 2025-12-17 16:14:19,819 - lava_callback - INFO - Completed processing callback for node 694033d3575bc060ce1571a5 INFO: 172.21.0.1:52080 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:14:29,007 - lava_callback - INFO - Received callback for node 694033d9575bc060ce1571ae INFO: 172.21.0.1:52094 - "POST /node/694033d9575bc060ce1571ae HTTP/1.0" 202 Accepted 2025-12-17 16:14:29,015 - lava_callback - INFO - Starting async job submit for node 694033d9575bc060ce1571ae 2025-12-17 16:14:29,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:29,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033d9575bc060ce1571ae HTTP/1.1" 200 None 2025-12-17 16:14:29,025 - lava_callback - DEBUG - Processing node 694033d9575bc060ce1571ae: baseline-arm64-clabbe 2025-12-17 16:14:29,025 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033d9575bc060ce1571ae 2025-12-17 16:14:29,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:29,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033d9575bc060ce1571ae HTTP/1.1" 200 None 2025-12-17 16:14:29,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:29,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:29,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:29,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033d9575bc060ce1571ae HTTP/1.1" 200 None 2025-12-17 16:14:29,061 - lava_callback - INFO - Completed processing callback for node 694033d9575bc060ce1571ae 2025-12-17 16:14:29,814 - lava_callback - INFO - Received callback for node 694033dc575bc060ce1571b1 INFO: 172.21.0.1:47752 - "POST /node/694033dc575bc060ce1571b1 HTTP/1.0" 202 Accepted 2025-12-17 16:14:29,823 - lava_callback - INFO - Starting async job submit for node 694033dc575bc060ce1571b1 2025-12-17 16:14:29,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:29,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033dc575bc060ce1571b1 HTTP/1.1" 200 None 2025-12-17 16:14:29,834 - lava_callback - DEBUG - Processing node 694033dc575bc060ce1571b1: baseline-arm64-clabbe 2025-12-17 16:14:29,834 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033dc575bc060ce1571b1 2025-12-17 16:14:29,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:29,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033dc575bc060ce1571b1 HTTP/1.1" 200 None 2025-12-17 16:14:29,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:29,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:29,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:29,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033dc575bc060ce1571b1 HTTP/1.1" 200 None 2025-12-17 16:14:29,884 - lava_callback - INFO - Completed processing callback for node 694033dc575bc060ce1571b1 2025-12-17 16:14:30,467 - lava_callback - INFO - Received callback for node 694033de575bc060ce1571b4 INFO: 172.21.0.1:47766 - "POST /node/694033de575bc060ce1571b4 HTTP/1.0" 202 Accepted 2025-12-17 16:14:30,475 - lava_callback - INFO - Starting async job submit for node 694033de575bc060ce1571b4 2025-12-17 16:14:30,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:30,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033de575bc060ce1571b4 HTTP/1.1" 200 None 2025-12-17 16:14:30,485 - lava_callback - DEBUG - Processing node 694033de575bc060ce1571b4: baseline-arm64-clabbe 2025-12-17 16:14:30,485 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033de575bc060ce1571b4 2025-12-17 16:14:30,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:30,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033de575bc060ce1571b4 HTTP/1.1" 200 None 2025-12-17 16:14:30,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:30,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:30,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:30,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033de575bc060ce1571b4 HTTP/1.1" 200 None 2025-12-17 16:14:30,526 - lava_callback - INFO - Completed processing callback for node 694033de575bc060ce1571b4 2025-12-17 16:14:34,403 - lava_callback - INFO - Received callback for node 694033d5575bc060ce1571a8 INFO: 172.21.0.1:47774 - "POST /node/694033d5575bc060ce1571a8 HTTP/1.0" 202 Accepted 2025-12-17 16:14:34,410 - lava_callback - INFO - Starting async job submit for node 694033d5575bc060ce1571a8 2025-12-17 16:14:34,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:34,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033d5575bc060ce1571a8 HTTP/1.1" 200 None 2025-12-17 16:14:34,419 - lava_callback - DEBUG - Processing node 694033d5575bc060ce1571a8: baseline-arm64-baylibre 2025-12-17 16:14:34,419 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033d5575bc060ce1571a8 2025-12-17 16:14:34,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:34,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033d5575bc060ce1571a8 HTTP/1.1" 200 None 2025-12-17 16:14:34,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:34,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:34,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:34,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033d5575bc060ce1571a8 HTTP/1.1" 200 None 2025-12-17 16:14:34,456 - lava_callback - INFO - Completed processing callback for node 694033d5575bc060ce1571a8 INFO: 172.21.0.1:47790 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:14:46,103 - lava_callback - INFO - Received callback for node 694033f3575bc060ce1571d6 INFO: 172.21.0.1:37392 - "POST /node/694033f3575bc060ce1571d6 HTTP/1.0" 202 Accepted 2025-12-17 16:14:46,110 - lava_callback - INFO - Starting async job submit for node 694033f3575bc060ce1571d6 2025-12-17 16:14:46,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:46,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033f3575bc060ce1571d6 HTTP/1.1" 200 None 2025-12-17 16:14:46,119 - lava_callback - DEBUG - Processing node 694033f3575bc060ce1571d6: ltp-sched 2025-12-17 16:14:46,119 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033f3575bc060ce1571d6 2025-12-17 16:14:46,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:46,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033f3575bc060ce1571d6 HTTP/1.1" 200 None 2025-12-17 16:14:46,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:46,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad967fb52d1dfff36c7c1 HTTP/1.1" 200 None 2025-12-17 16:14:46,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:46,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033f3575bc060ce1571d6 HTTP/1.1" 200 None 2025-12-17 16:14:46,149 - lava_callback - INFO - Completed processing callback for node 694033f3575bc060ce1571d6 INFO: 172.21.0.1:37400 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:14:46,873 - lava_callback - INFO - Received callback for node 694033f4575bc060ce1571d9 INFO: 172.21.0.1:37410 - "POST /node/694033f4575bc060ce1571d9 HTTP/1.0" 202 Accepted 2025-12-17 16:14:46,879 - lava_callback - INFO - Starting async job submit for node 694033f4575bc060ce1571d9 2025-12-17 16:14:46,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:46,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033f4575bc060ce1571d9 HTTP/1.1" 200 None 2025-12-17 16:14:46,896 - lava_callback - DEBUG - Processing node 694033f4575bc060ce1571d9: ltp-sched 2025-12-17 16:14:46,896 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033f4575bc060ce1571d9 2025-12-17 16:14:46,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:46,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033f4575bc060ce1571d9 HTTP/1.1" 200 None 2025-12-17 16:14:46,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:46,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad967fb52d1dfff36c7c1 HTTP/1.1" 200 None 2025-12-17 16:14:46,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:46,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033f4575bc060ce1571d9 HTTP/1.1" 200 None 2025-12-17 16:14:46,930 - lava_callback - INFO - Completed processing callback for node 694033f4575bc060ce1571d9 2025-12-17 16:14:47,611 - lava_callback - INFO - Received callback for node 694033f5575bc060ce1571dc INFO: 172.21.0.1:37426 - "POST /node/694033f5575bc060ce1571dc HTTP/1.0" 202 Accepted 2025-12-17 16:14:47,617 - lava_callback - INFO - Starting async job submit for node 694033f5575bc060ce1571dc 2025-12-17 16:14:47,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:47,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033f5575bc060ce1571dc HTTP/1.1" 200 None 2025-12-17 16:14:47,626 - lava_callback - DEBUG - Processing node 694033f5575bc060ce1571dc: ltp-syscalls 2025-12-17 16:14:47,626 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033f5575bc060ce1571dc 2025-12-17 16:14:47,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:47,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033f5575bc060ce1571dc HTTP/1.1" 200 None 2025-12-17 16:14:47,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:47,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:14:47,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:47,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033f5575bc060ce1571dc HTTP/1.1" 200 None 2025-12-17 16:14:47,662 - lava_callback - INFO - Completed processing callback for node 694033f5575bc060ce1571dc 2025-12-17 16:14:49,060 - lava_callback - INFO - Received callback for node 694033e5575bc060ce1571c1 INFO: 172.21.0.1:37430 - "POST /node/694033e5575bc060ce1571c1 HTTP/1.0" 202 Accepted 2025-12-17 16:14:49,065 - lava_callback - INFO - Starting async job submit for node 694033e5575bc060ce1571c1 2025-12-17 16:14:49,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:49,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033e5575bc060ce1571c1 HTTP/1.1" 200 None 2025-12-17 16:14:49,075 - lava_callback - DEBUG - Processing node 694033e5575bc060ce1571c1: baseline-arm64-clabbe 2025-12-17 16:14:49,075 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033e5575bc060ce1571c1 2025-12-17 16:14:49,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:49,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033e5575bc060ce1571c1 HTTP/1.1" 200 None 2025-12-17 16:14:49,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:49,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:49,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:49,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033e5575bc060ce1571c1 HTTP/1.1" 200 None 2025-12-17 16:14:49,104 - lava_callback - INFO - Completed processing callback for node 694033e5575bc060ce1571c1 2025-12-17 16:14:49,730 - lava_callback - INFO - Received callback for node 694033e7575bc060ce1571c4 INFO: 172.21.0.1:38814 - "POST /node/694033e7575bc060ce1571c4 HTTP/1.0" 202 Accepted 2025-12-17 16:14:49,736 - lava_callback - INFO - Starting async job submit for node 694033e7575bc060ce1571c4 2025-12-17 16:14:49,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:49,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033e7575bc060ce1571c4 HTTP/1.1" 200 None 2025-12-17 16:14:49,746 - lava_callback - DEBUG - Processing node 694033e7575bc060ce1571c4: baseline-arm64-clabbe 2025-12-17 16:14:49,746 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033e7575bc060ce1571c4 2025-12-17 16:14:49,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:49,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033e7575bc060ce1571c4 HTTP/1.1" 200 None 2025-12-17 16:14:49,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:49,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:49,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:49,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033e7575bc060ce1571c4 HTTP/1.1" 200 None 2025-12-17 16:14:49,786 - lava_callback - INFO - Completed processing callback for node 694033e7575bc060ce1571c4 2025-12-17 16:14:50,404 - lava_callback - INFO - Received callback for node 694033e9575bc060ce1571c7 INFO: 172.21.0.1:38816 - "POST /node/694033e9575bc060ce1571c7 HTTP/1.0" 202 Accepted 2025-12-17 16:14:50,410 - lava_callback - INFO - Starting async job submit for node 694033e9575bc060ce1571c7 2025-12-17 16:14:50,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:50,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033e9575bc060ce1571c7 HTTP/1.1" 200 None 2025-12-17 16:14:50,419 - lava_callback - DEBUG - Processing node 694033e9575bc060ce1571c7: baseline-arm64-clabbe 2025-12-17 16:14:50,419 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033e9575bc060ce1571c7 2025-12-17 16:14:50,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:50,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033e9575bc060ce1571c7 HTTP/1.1" 200 None 2025-12-17 16:14:50,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:50,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:50,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:50,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033e9575bc060ce1571c7 HTTP/1.1" 200 None 2025-12-17 16:14:50,454 - lava_callback - INFO - Completed processing callback for node 694033e9575bc060ce1571c7 2025-12-17 16:14:51,081 - lava_callback - INFO - Received callback for node 694033ea575bc060ce1571ca INFO: 172.21.0.1:38830 - "POST /node/694033ea575bc060ce1571ca HTTP/1.0" 202 Accepted 2025-12-17 16:14:51,088 - lava_callback - INFO - Starting async job submit for node 694033ea575bc060ce1571ca 2025-12-17 16:14:51,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:51,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033ea575bc060ce1571ca HTTP/1.1" 200 None 2025-12-17 16:14:51,099 - lava_callback - DEBUG - Processing node 694033ea575bc060ce1571ca: baseline-arm64-clabbe 2025-12-17 16:14:51,099 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033ea575bc060ce1571ca 2025-12-17 16:14:51,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:51,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033ea575bc060ce1571ca HTTP/1.1" 200 None 2025-12-17 16:14:51,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:51,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:51,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:51,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033ea575bc060ce1571ca HTTP/1.1" 200 None 2025-12-17 16:14:51,149 - lava_callback - INFO - Completed processing callback for node 694033ea575bc060ce1571ca 2025-12-17 16:14:51,747 - lava_callback - INFO - Received callback for node 694033ec575bc060ce1571cd INFO: 172.21.0.1:38838 - "POST /node/694033ec575bc060ce1571cd HTTP/1.0" 202 Accepted 2025-12-17 16:14:51,756 - lava_callback - INFO - Starting async job submit for node 694033ec575bc060ce1571cd 2025-12-17 16:14:51,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:51,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033ec575bc060ce1571cd HTTP/1.1" 200 None 2025-12-17 16:14:51,767 - lava_callback - DEBUG - Processing node 694033ec575bc060ce1571cd: baseline-arm64-clabbe 2025-12-17 16:14:51,767 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033ec575bc060ce1571cd 2025-12-17 16:14:51,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:51,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033ec575bc060ce1571cd HTTP/1.1" 200 None 2025-12-17 16:14:51,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:51,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:51,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:51,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033ec575bc060ce1571cd HTTP/1.1" 200 None 2025-12-17 16:14:51,810 - lava_callback - INFO - Completed processing callback for node 694033ec575bc060ce1571cd 2025-12-17 16:14:52,420 - lava_callback - INFO - Received callback for node 694033ee575bc060ce1571d0 INFO: 172.21.0.1:38842 - "POST /node/694033ee575bc060ce1571d0 HTTP/1.0" 202 Accepted 2025-12-17 16:14:52,428 - lava_callback - INFO - Starting async job submit for node 694033ee575bc060ce1571d0 2025-12-17 16:14:52,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:52,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033ee575bc060ce1571d0 HTTP/1.1" 200 None 2025-12-17 16:14:52,438 - lava_callback - DEBUG - Processing node 694033ee575bc060ce1571d0: baseline-arm64-clabbe 2025-12-17 16:14:52,438 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033ee575bc060ce1571d0 2025-12-17 16:14:52,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:52,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033ee575bc060ce1571d0 HTTP/1.1" 200 None 2025-12-17 16:14:52,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:52,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:52,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:52,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033ee575bc060ce1571d0 HTTP/1.1" 200 None 2025-12-17 16:14:52,755 - lava_callback - INFO - Completed processing callback for node 694033ee575bc060ce1571d0 2025-12-17 16:14:53,085 - lava_callback - INFO - Received callback for node 694033f0575bc060ce1571d3 INFO: 172.21.0.1:38850 - "POST /node/694033f0575bc060ce1571d3 HTTP/1.0" 202 Accepted 2025-12-17 16:14:53,091 - lava_callback - INFO - Starting async job submit for node 694033f0575bc060ce1571d3 2025-12-17 16:14:53,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:53,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033f0575bc060ce1571d3 HTTP/1.1" 200 None 2025-12-17 16:14:53,099 - lava_callback - DEBUG - Processing node 694033f0575bc060ce1571d3: baseline-arm64-clabbe 2025-12-17 16:14:53,099 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033f0575bc060ce1571d3 2025-12-17 16:14:53,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:53,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033f0575bc060ce1571d3 HTTP/1.1" 200 None 2025-12-17 16:14:53,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:53,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:53,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:53,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033f0575bc060ce1571d3 HTTP/1.1" 200 None 2025-12-17 16:14:53,135 - lava_callback - INFO - Completed processing callback for node 694033f0575bc060ce1571d3 2025-12-17 16:14:54,410 - lava_callback - INFO - Received callback for node 694033fa575bc060ce1571ed INFO: 172.21.0.1:38866 - "POST /node/694033fa575bc060ce1571ed HTTP/1.0" 202 Accepted 2025-12-17 16:14:54,420 - lava_callback - INFO - Starting async job submit for node 694033fa575bc060ce1571ed 2025-12-17 16:14:54,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:54,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033fa575bc060ce1571ed HTTP/1.1" 200 None 2025-12-17 16:14:54,432 - lava_callback - DEBUG - Processing node 694033fa575bc060ce1571ed: baseline-arm64-baylibre 2025-12-17 16:14:54,432 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033fa575bc060ce1571ed 2025-12-17 16:14:54,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:54,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033fa575bc060ce1571ed HTTP/1.1" 200 None 2025-12-17 16:14:54,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:54,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:14:54,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:54,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033fa575bc060ce1571ed HTTP/1.1" 200 None 2025-12-17 16:14:54,466 - lava_callback - INFO - Completed processing callback for node 694033fa575bc060ce1571ed 2025-12-17 16:14:54,661 - lava_callback - INFO - Received callback for node 694033f6575bc060ce1571df INFO: 172.21.0.1:38876 - "POST /node/694033f6575bc060ce1571df HTTP/1.0" 202 Accepted 2025-12-17 16:14:54,667 - lava_callback - INFO - Starting async job submit for node 694033f6575bc060ce1571df 2025-12-17 16:14:54,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:54,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033f6575bc060ce1571df HTTP/1.1" 200 None 2025-12-17 16:14:54,676 - lava_callback - DEBUG - Processing node 694033f6575bc060ce1571df: baseline-arm-pengutronix 2025-12-17 16:14:54,676 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033f6575bc060ce1571df 2025-12-17 16:14:54,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:54,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033f6575bc060ce1571df HTTP/1.1" 200 None 2025-12-17 16:14:54,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:54,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af125fee8383e92b69ab4 HTTP/1.1" 200 None 2025-12-17 16:14:54,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:14:54,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033f6575bc060ce1571df HTTP/1.1" 200 None 2025-12-17 16:14:54,717 - lava_callback - INFO - Completed processing callback for node 694033f6575bc060ce1571df INFO: 172.21.0.1:38888 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:15:06,060 - lava_callback - INFO - Received callback for node 694033f8575bc060ce1571e5 INFO: 172.21.0.1:55952 - "POST /node/694033f8575bc060ce1571e5 HTTP/1.0" 202 Accepted 2025-12-17 16:15:06,067 - lava_callback - INFO - Starting async job submit for node 694033f8575bc060ce1571e5 2025-12-17 16:15:06,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:06,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033f8575bc060ce1571e5 HTTP/1.1" 200 None 2025-12-17 16:15:06,076 - lava_callback - DEBUG - Processing node 694033f8575bc060ce1571e5: ltp-syscalls 2025-12-17 16:15:06,076 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033f8575bc060ce1571e5 2025-12-17 16:15:06,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:06,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033f8575bc060ce1571e5 HTTP/1.1" 200 None 2025-12-17 16:15:06,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:06,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:15:06,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:06,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033f8575bc060ce1571e5 HTTP/1.1" 200 None 2025-12-17 16:15:06,121 - lava_callback - INFO - Completed processing callback for node 694033f8575bc060ce1571e5 INFO: 172.21.0.1:55968 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:15:06,825 - lava_callback - INFO - Received callback for node 694033fa575bc060ce1571ea INFO: 172.21.0.1:55984 - "POST /node/694033fa575bc060ce1571ea HTTP/1.0" 202 Accepted 2025-12-17 16:15:06,832 - lava_callback - INFO - Starting async job submit for node 694033fa575bc060ce1571ea 2025-12-17 16:15:06,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:06,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033fa575bc060ce1571ea HTTP/1.1" 200 None 2025-12-17 16:15:06,844 - lava_callback - DEBUG - Processing node 694033fa575bc060ce1571ea: ltp-syscalls 2025-12-17 16:15:06,844 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694033fa575bc060ce1571ea 2025-12-17 16:15:06,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:06,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694033fa575bc060ce1571ea HTTP/1.1" 200 None 2025-12-17 16:15:06,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:06,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:15:06,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:06,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694033fa575bc060ce1571ea HTTP/1.1" 200 None 2025-12-17 16:15:06,880 - lava_callback - INFO - Completed processing callback for node 694033fa575bc060ce1571ea INFO: 172.21.0.1:33606 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:15:14,443 - lava_callback - INFO - Received callback for node 69403401575bc060ce1571f0 INFO: 172.21.0.1:33612 - "POST /node/69403401575bc060ce1571f0 HTTP/1.0" 202 Accepted 2025-12-17 16:15:14,449 - lava_callback - INFO - Starting async job submit for node 69403401575bc060ce1571f0 2025-12-17 16:15:14,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:14,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403401575bc060ce1571f0 HTTP/1.1" 200 None 2025-12-17 16:15:14,467 - lava_callback - DEBUG - Processing node 69403401575bc060ce1571f0: baseline-arm64-baylibre 2025-12-17 16:15:14,467 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403401575bc060ce1571f0 2025-12-17 16:15:14,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:14,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403401575bc060ce1571f0 HTTP/1.1" 200 None 2025-12-17 16:15:14,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:14,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:15:14,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:14,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403401575bc060ce1571f0 HTTP/1.1" 200 None 2025-12-17 16:15:14,514 - lava_callback - INFO - Completed processing callback for node 69403401575bc060ce1571f0 2025-12-17 16:15:15,091 - lava_callback - INFO - Received callback for node 69403404575bc060ce1571f8 INFO: 172.21.0.1:33622 - "POST /node/69403404575bc060ce1571f8 HTTP/1.0" 202 Accepted 2025-12-17 16:15:15,097 - lava_callback - INFO - Starting async job submit for node 69403404575bc060ce1571f8 2025-12-17 16:15:15,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:15,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403404575bc060ce1571f8 HTTP/1.1" 200 None 2025-12-17 16:15:15,107 - lava_callback - DEBUG - Processing node 69403404575bc060ce1571f8: baseline-arm64-baylibre 2025-12-17 16:15:15,107 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403404575bc060ce1571f8 2025-12-17 16:15:15,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:15,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403404575bc060ce1571f8 HTTP/1.1" 200 None 2025-12-17 16:15:15,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:15,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:15:15,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:15,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403404575bc060ce1571f8 HTTP/1.1" 200 None 2025-12-17 16:15:15,148 - lava_callback - INFO - Completed processing callback for node 69403404575bc060ce1571f8 INFO: 172.21.0.1:33630 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:15:15,740 - lava_callback - INFO - Received callback for node 69403409575bc060ce1571ff INFO: 172.21.0.1:33632 - "POST /node/69403409575bc060ce1571ff HTTP/1.0" 202 Accepted 2025-12-17 16:15:15,747 - lava_callback - INFO - Starting async job submit for node 69403409575bc060ce1571ff 2025-12-17 16:15:15,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:15,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403409575bc060ce1571ff HTTP/1.1" 200 None 2025-12-17 16:15:15,756 - lava_callback - DEBUG - Processing node 69403409575bc060ce1571ff: baseline-arm64-baylibre 2025-12-17 16:15:15,756 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403409575bc060ce1571ff 2025-12-17 16:15:15,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:15,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403409575bc060ce1571ff HTTP/1.1" 200 None 2025-12-17 16:15:15,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:15,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:15:15,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:15,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403409575bc060ce1571ff HTTP/1.1" 200 None 2025-12-17 16:15:15,795 - lava_callback - INFO - Completed processing callback for node 69403409575bc060ce1571ff 2025-12-17 16:15:16,401 - lava_callback - INFO - Received callback for node 6940340c575bc060ce157202 INFO: 172.21.0.1:33646 - "POST /node/6940340c575bc060ce157202 HTTP/1.0" 202 Accepted 2025-12-17 16:15:16,408 - lava_callback - INFO - Starting async job submit for node 6940340c575bc060ce157202 2025-12-17 16:15:16,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:16,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940340c575bc060ce157202 HTTP/1.1" 200 None 2025-12-17 16:15:16,417 - lava_callback - DEBUG - Processing node 6940340c575bc060ce157202: baseline-arm64-baylibre 2025-12-17 16:15:16,417 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940340c575bc060ce157202 2025-12-17 16:15:16,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:16,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940340c575bc060ce157202 HTTP/1.1" 200 None 2025-12-17 16:15:16,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:16,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:15:16,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:16,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940340c575bc060ce157202 HTTP/1.1" 200 None 2025-12-17 16:15:16,452 - lava_callback - INFO - Completed processing callback for node 6940340c575bc060ce157202 INFO: 172.21.0.1:33650 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:15:17,061 - lava_callback - INFO - Received callback for node 6940340e575bc060ce15720b INFO: 172.21.0.1:33666 - "POST /node/6940340e575bc060ce15720b HTTP/1.0" 202 Accepted 2025-12-17 16:15:17,069 - lava_callback - INFO - Starting async job submit for node 6940340e575bc060ce15720b 2025-12-17 16:15:17,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:17,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940340e575bc060ce15720b HTTP/1.1" 200 None 2025-12-17 16:15:17,081 - lava_callback - DEBUG - Processing node 6940340e575bc060ce15720b: baseline-arm64-baylibre 2025-12-17 16:15:17,081 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940340e575bc060ce15720b 2025-12-17 16:15:17,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:17,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940340e575bc060ce15720b HTTP/1.1" 200 None 2025-12-17 16:15:17,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:17,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:15:17,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:17,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940340e575bc060ce15720b HTTP/1.1" 200 None 2025-12-17 16:15:17,118 - lava_callback - INFO - Completed processing callback for node 6940340e575bc060ce15720b 2025-12-17 16:15:17,763 - lava_callback - INFO - Received callback for node 69403410575bc060ce15720e INFO: 172.21.0.1:33672 - "POST /node/69403410575bc060ce15720e HTTP/1.0" 202 Accepted 2025-12-17 16:15:17,769 - lava_callback - INFO - Starting async job submit for node 69403410575bc060ce15720e 2025-12-17 16:15:17,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:17,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403410575bc060ce15720e HTTP/1.1" 200 None 2025-12-17 16:15:17,778 - lava_callback - DEBUG - Processing node 69403410575bc060ce15720e: baseline-arm64-baylibre 2025-12-17 16:15:17,778 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403410575bc060ce15720e 2025-12-17 16:15:17,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:17,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403410575bc060ce15720e HTTP/1.1" 200 None 2025-12-17 16:15:17,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:17,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:15:17,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:17,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403410575bc060ce15720e HTTP/1.1" 200 None 2025-12-17 16:15:17,849 - lava_callback - INFO - Completed processing callback for node 69403410575bc060ce15720e INFO: 172.21.0.1:53424 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:15:29,053 - lava_callback - INFO - Received callback for node 69403418575bc060ce157221 INFO: 172.21.0.1:53430 - "POST /node/69403418575bc060ce157221 HTTP/1.0" 202 Accepted 2025-12-17 16:15:29,062 - lava_callback - INFO - Starting async job submit for node 69403418575bc060ce157221 2025-12-17 16:15:29,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:29,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403418575bc060ce157221 HTTP/1.1" 200 None 2025-12-17 16:15:29,073 - lava_callback - DEBUG - Processing node 69403418575bc060ce157221: baseline-arm64-clabbe 2025-12-17 16:15:29,073 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403418575bc060ce157221 2025-12-17 16:15:29,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:29,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403418575bc060ce157221 HTTP/1.1" 200 None 2025-12-17 16:15:29,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:29,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:15:29,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:29,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403418575bc060ce157221 HTTP/1.1" 200 None 2025-12-17 16:15:29,116 - lava_callback - INFO - Completed processing callback for node 69403418575bc060ce157221 2025-12-17 16:15:29,724 - lava_callback - INFO - Received callback for node 6940341b575bc060ce157224 INFO: 172.21.0.1:55944 - "POST /node/6940341b575bc060ce157224 HTTP/1.0" 202 Accepted 2025-12-17 16:15:29,733 - lava_callback - INFO - Starting async job submit for node 6940341b575bc060ce157224 2025-12-17 16:15:29,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:29,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940341b575bc060ce157224 HTTP/1.1" 200 None 2025-12-17 16:15:29,746 - lava_callback - DEBUG - Processing node 6940341b575bc060ce157224: baseline-arm64-clabbe 2025-12-17 16:15:29,746 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940341b575bc060ce157224 2025-12-17 16:15:29,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:29,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940341b575bc060ce157224 HTTP/1.1" 200 None 2025-12-17 16:15:29,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:29,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:15:29,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:29,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940341b575bc060ce157224 HTTP/1.1" 200 None 2025-12-17 16:15:29,829 - lava_callback - INFO - Completed processing callback for node 6940341b575bc060ce157224 2025-12-17 16:15:30,407 - lava_callback - INFO - Received callback for node 6940341d575bc060ce157227 INFO: 172.21.0.1:55960 - "POST /node/6940341d575bc060ce157227 HTTP/1.0" 202 Accepted 2025-12-17 16:15:30,416 - lava_callback - INFO - Starting async job submit for node 6940341d575bc060ce157227 2025-12-17 16:15:30,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:30,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940341d575bc060ce157227 HTTP/1.1" 200 None 2025-12-17 16:15:30,434 - lava_callback - DEBUG - Processing node 6940341d575bc060ce157227: baseline-arm64-clabbe 2025-12-17 16:15:30,434 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940341d575bc060ce157227 2025-12-17 16:15:30,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:30,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940341d575bc060ce157227 HTTP/1.1" 200 None 2025-12-17 16:15:30,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:30,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:15:30,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:30,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940341d575bc060ce157227 HTTP/1.1" 200 None 2025-12-17 16:15:30,476 - lava_callback - INFO - Completed processing callback for node 6940341d575bc060ce157227 2025-12-17 16:15:31,076 - lava_callback - INFO - Received callback for node 69403420575bc060ce15722a INFO: 172.21.0.1:55970 - "POST /node/69403420575bc060ce15722a HTTP/1.0" 202 Accepted 2025-12-17 16:15:31,082 - lava_callback - INFO - Starting async job submit for node 69403420575bc060ce15722a 2025-12-17 16:15:31,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:31,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403420575bc060ce15722a HTTP/1.1" 200 None 2025-12-17 16:15:31,092 - lava_callback - DEBUG - Processing node 69403420575bc060ce15722a: baseline-arm64-clabbe 2025-12-17 16:15:31,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403420575bc060ce15722a 2025-12-17 16:15:31,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:31,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403420575bc060ce15722a HTTP/1.1" 200 None 2025-12-17 16:15:31,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:31,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:15:31,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:31,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403420575bc060ce15722a HTTP/1.1" 200 None 2025-12-17 16:15:31,130 - lava_callback - INFO - Completed processing callback for node 69403420575bc060ce15722a 2025-12-17 16:15:34,476 - lava_callback - INFO - Received callback for node 69403412575bc060ce157211 INFO: 172.21.0.1:55980 - "POST /node/69403412575bc060ce157211 HTTP/1.0" 202 Accepted 2025-12-17 16:15:34,483 - lava_callback - INFO - Starting async job submit for node 69403412575bc060ce157211 2025-12-17 16:15:34,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:34,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403412575bc060ce157211 HTTP/1.1" 200 None 2025-12-17 16:15:34,496 - lava_callback - DEBUG - Processing node 69403412575bc060ce157211: baseline-arm64-baylibre 2025-12-17 16:15:34,496 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403412575bc060ce157211 2025-12-17 16:15:34,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:34,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403412575bc060ce157211 HTTP/1.1" 200 None 2025-12-17 16:15:34,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:34,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:15:34,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:34,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403412575bc060ce157211 HTTP/1.1" 200 None 2025-12-17 16:15:34,538 - lava_callback - INFO - Completed processing callback for node 69403412575bc060ce157211 2025-12-17 16:15:34,763 - lava_callback - INFO - Received callback for node 69403423575bc060ce157230 INFO: 172.21.0.1:55986 - "POST /node/69403423575bc060ce157230 HTTP/1.0" 202 Accepted 2025-12-17 16:15:34,770 - lava_callback - INFO - Starting async job submit for node 69403423575bc060ce157230 2025-12-17 16:15:34,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:34,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403423575bc060ce157230 HTTP/1.1" 200 None 2025-12-17 16:15:34,780 - lava_callback - DEBUG - Processing node 69403423575bc060ce157230: baseline-arm 2025-12-17 16:15:34,781 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403423575bc060ce157230 2025-12-17 16:15:34,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:34,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403423575bc060ce157230 HTTP/1.1" 200 None 2025-12-17 16:15:34,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:34,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:15:34,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:34,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403423575bc060ce157230 HTTP/1.1" 200 None 2025-12-17 16:15:34,832 - lava_callback - INFO - Completed processing callback for node 69403423575bc060ce157230 2025-12-17 16:15:35,144 - lava_callback - INFO - Received callback for node 69403414575bc060ce157217 INFO: 172.21.0.1:56000 - "POST /node/69403414575bc060ce157217 HTTP/1.0" 202 Accepted 2025-12-17 16:15:35,153 - lava_callback - INFO - Starting async job submit for node 69403414575bc060ce157217 2025-12-17 16:15:35,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:35,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403414575bc060ce157217 HTTP/1.1" 200 None 2025-12-17 16:15:35,163 - lava_callback - DEBUG - Processing node 69403414575bc060ce157217: baseline-arm64-baylibre 2025-12-17 16:15:35,163 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403414575bc060ce157217 2025-12-17 16:15:35,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:35,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403414575bc060ce157217 HTTP/1.1" 200 None 2025-12-17 16:15:35,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:35,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:15:35,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:35,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403414575bc060ce157217 HTTP/1.1" 200 None 2025-12-17 16:15:35,207 - lava_callback - INFO - Completed processing callback for node 69403414575bc060ce157217 2025-12-17 16:15:35,825 - lava_callback - INFO - Received callback for node 69403424575bc060ce157233 INFO: 172.21.0.1:56012 - "POST /node/69403424575bc060ce157233 HTTP/1.0" 202 Accepted 2025-12-17 16:15:35,832 - lava_callback - INFO - Starting async job submit for node 69403424575bc060ce157233 2025-12-17 16:15:35,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:35,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403424575bc060ce157233 HTTP/1.1" 200 None 2025-12-17 16:15:35,843 - lava_callback - DEBUG - Processing node 69403424575bc060ce157233: baseline-arm-baylibre 2025-12-17 16:15:35,843 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403424575bc060ce157233 2025-12-17 16:15:35,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:35,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403424575bc060ce157233 HTTP/1.1" 200 None 2025-12-17 16:15:35,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:35,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:15:35,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:35,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403424575bc060ce157233 HTTP/1.1" 200 None 2025-12-17 16:15:35,881 - lava_callback - INFO - Completed processing callback for node 69403424575bc060ce157233 INFO: 172.21.0.1:56020 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:15:41,203 - lava_callback - INFO - Received callback for node 694034bc575bc060ce1573aa INFO: 172.21.0.1:59566 - "POST /node/694034bc575bc060ce1573aa HTTP/1.0" 202 Accepted 2025-12-17 16:15:41,212 - lava_callback - INFO - Starting async job submit for node 694034bc575bc060ce1573aa 2025-12-17 16:15:41,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:41,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034bc575bc060ce1573aa HTTP/1.1" 200 None 2025-12-17 16:15:41,258 - lava_callback - DEBUG - Processing node 694034bc575bc060ce1573aa: baseline-arm 2025-12-17 16:15:41,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:15:41,686 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:15:41,687 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-694034bc575bc060ce1573aa/log.txt.gz 2025-12-17 16:15:41,687 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:15:41,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:15:41,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:15:41,860 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-694034bc575bc060ce1573aa/lava_callback.json.gz 2025-12-17 16:15:41,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:41,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034bc575bc060ce1573aa HTTP/1.1" 200 None 2025-12-17 16:15:41,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:41,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:15:41,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:41,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034bc575bc060ce1573aa HTTP/1.1" 200 None 2025-12-17 16:15:41,936 - lava_callback - INFO - Completed processing callback for node 694034bc575bc060ce1573aa INFO: 172.21.0.1:59582 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:15:49,109 - lava_callback - INFO - Received callback for node 69403422575bc060ce15722d INFO: 172.21.0.1:59586 - "POST /node/69403422575bc060ce15722d HTTP/1.0" 202 Accepted 2025-12-17 16:15:49,116 - lava_callback - INFO - Starting async job submit for node 69403422575bc060ce15722d 2025-12-17 16:15:49,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:49,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403422575bc060ce15722d HTTP/1.1" 200 None 2025-12-17 16:15:49,134 - lava_callback - DEBUG - Processing node 69403422575bc060ce15722d: baseline-arm64-clabbe 2025-12-17 16:15:49,134 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403422575bc060ce15722d 2025-12-17 16:15:49,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:49,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403422575bc060ce15722d HTTP/1.1" 200 None 2025-12-17 16:15:49,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:49,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d218c07de2c0cbc264e3f HTTP/1.1" 200 None 2025-12-17 16:15:49,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:49,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403422575bc060ce15722d HTTP/1.1" 200 None 2025-12-17 16:15:49,170 - lava_callback - INFO - Completed processing callback for node 69403422575bc060ce15722d 2025-12-17 16:15:54,484 - lava_callback - INFO - Received callback for node 69403426575bc060ce157236 INFO: 172.21.0.1:51422 - "POST /node/69403426575bc060ce157236 HTTP/1.0" 202 Accepted 2025-12-17 16:15:54,490 - lava_callback - INFO - Starting async job submit for node 69403426575bc060ce157236 2025-12-17 16:15:54,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:54,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403426575bc060ce157236 HTTP/1.1" 200 None 2025-12-17 16:15:54,501 - lava_callback - DEBUG - Processing node 69403426575bc060ce157236: baseline-arm-baylibre 2025-12-17 16:15:54,501 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403426575bc060ce157236 2025-12-17 16:15:54,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:54,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403426575bc060ce157236 HTTP/1.1" 200 None 2025-12-17 16:15:54,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:54,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:15:54,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:54,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403426575bc060ce157236 HTTP/1.1" 200 None 2025-12-17 16:15:54,545 - lava_callback - INFO - Completed processing callback for node 69403426575bc060ce157236 2025-12-17 16:15:54,716 - lava_callback - INFO - Received callback for node 69403429575bc060ce15723f INFO: 172.21.0.1:51432 - "POST /node/69403429575bc060ce15723f HTTP/1.0" 202 Accepted 2025-12-17 16:15:54,725 - lava_callback - INFO - Starting async job submit for node 69403429575bc060ce15723f 2025-12-17 16:15:54,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:54,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403429575bc060ce15723f HTTP/1.1" 200 None 2025-12-17 16:15:54,786 - lava_callback - DEBUG - Processing node 69403429575bc060ce15723f: baseline-arm 2025-12-17 16:15:54,786 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403429575bc060ce15723f 2025-12-17 16:15:54,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:54,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403429575bc060ce15723f HTTP/1.1" 200 None 2025-12-17 16:15:54,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:54,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:15:54,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:54,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403429575bc060ce15723f HTTP/1.1" 200 None 2025-12-17 16:15:54,832 - lava_callback - INFO - Completed processing callback for node 69403429575bc060ce15723f 2025-12-17 16:15:55,167 - lava_callback - INFO - Received callback for node 69403427575bc060ce157239 INFO: 172.21.0.1:51434 - "POST /node/69403427575bc060ce157239 HTTP/1.0" 202 Accepted 2025-12-17 16:15:55,175 - lava_callback - INFO - Starting async job submit for node 69403427575bc060ce157239 2025-12-17 16:15:55,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:55,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403427575bc060ce157239 HTTP/1.1" 200 None 2025-12-17 16:15:55,186 - lava_callback - DEBUG - Processing node 69403427575bc060ce157239: baseline-arm-baylibre 2025-12-17 16:15:55,186 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403427575bc060ce157239 2025-12-17 16:15:55,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:55,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403427575bc060ce157239 HTTP/1.1" 200 None 2025-12-17 16:15:55,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:55,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:15:55,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:55,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403427575bc060ce157239 HTTP/1.1" 200 None 2025-12-17 16:15:55,236 - lava_callback - INFO - Completed processing callback for node 69403427575bc060ce157239 2025-12-17 16:15:55,419 - lava_callback - INFO - Received callback for node 6940342b575bc060ce157242 INFO: 172.21.0.1:51446 - "POST /node/6940342b575bc060ce157242 HTTP/1.0" 202 Accepted 2025-12-17 16:15:55,427 - lava_callback - INFO - Starting async job submit for node 6940342b575bc060ce157242 2025-12-17 16:15:55,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:55,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940342b575bc060ce157242 HTTP/1.1" 200 None 2025-12-17 16:15:55,441 - lava_callback - DEBUG - Processing node 6940342b575bc060ce157242: baseline-arm 2025-12-17 16:15:55,442 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940342b575bc060ce157242 2025-12-17 16:15:55,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:55,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940342b575bc060ce157242 HTTP/1.1" 200 None 2025-12-17 16:15:55,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:55,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:15:55,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:55,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940342b575bc060ce157242 HTTP/1.1" 200 None 2025-12-17 16:15:55,482 - lava_callback - INFO - Completed processing callback for node 6940342b575bc060ce157242 2025-12-17 16:15:55,815 - lava_callback - INFO - Received callback for node 69403428575bc060ce15723c INFO: 172.21.0.1:51456 - "POST /node/69403428575bc060ce15723c HTTP/1.0" 202 Accepted 2025-12-17 16:15:55,824 - lava_callback - INFO - Starting async job submit for node 69403428575bc060ce15723c 2025-12-17 16:15:55,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:55,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403428575bc060ce15723c HTTP/1.1" 200 None 2025-12-17 16:15:55,920 - lava_callback - DEBUG - Processing node 69403428575bc060ce15723c: baseline-arm-baylibre 2025-12-17 16:15:55,920 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403428575bc060ce15723c 2025-12-17 16:15:55,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:55,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403428575bc060ce15723c HTTP/1.1" 200 None 2025-12-17 16:15:55,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:55,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:15:55,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:55,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403428575bc060ce15723c HTTP/1.1" 200 None 2025-12-17 16:15:55,984 - lava_callback - INFO - Completed processing callback for node 69403428575bc060ce15723c 2025-12-17 16:15:56,469 - lava_callback - INFO - Received callback for node 6940342c575bc060ce157245 INFO: 172.21.0.1:51460 - "POST /node/6940342c575bc060ce157245 HTTP/1.0" 202 Accepted 2025-12-17 16:15:56,477 - lava_callback - INFO - Starting async job submit for node 6940342c575bc060ce157245 2025-12-17 16:15:56,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:56,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940342c575bc060ce157245 HTTP/1.1" 200 None 2025-12-17 16:15:56,488 - lava_callback - DEBUG - Processing node 6940342c575bc060ce157245: baseline-arm-baylibre 2025-12-17 16:15:56,488 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940342c575bc060ce157245 2025-12-17 16:15:56,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:56,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940342c575bc060ce157245 HTTP/1.1" 200 None 2025-12-17 16:15:56,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:51468 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:15:56,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:15:56,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:56,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940342c575bc060ce157245 HTTP/1.1" 200 None 2025-12-17 16:15:56,531 - lava_callback - INFO - Completed processing callback for node 6940342c575bc060ce157245 2025-12-17 16:15:57,123 - lava_callback - INFO - Received callback for node 6940342d575bc060ce157248 INFO: 172.21.0.1:51470 - "POST /node/6940342d575bc060ce157248 HTTP/1.0" 202 Accepted 2025-12-17 16:15:57,130 - lava_callback - INFO - Starting async job submit for node 6940342d575bc060ce157248 2025-12-17 16:15:57,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:57,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940342d575bc060ce157248 HTTP/1.1" 200 None 2025-12-17 16:15:57,139 - lava_callback - DEBUG - Processing node 6940342d575bc060ce157248: baseline-arm-baylibre 2025-12-17 16:15:57,139 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940342d575bc060ce157248 2025-12-17 16:15:57,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:57,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940342d575bc060ce157248 HTTP/1.1" 200 None 2025-12-17 16:15:57,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:57,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:15:57,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:57,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940342d575bc060ce157248 HTTP/1.1" 200 None 2025-12-17 16:15:57,173 - lava_callback - INFO - Completed processing callback for node 6940342d575bc060ce157248 2025-12-17 16:15:57,786 - lava_callback - INFO - Received callback for node 6940342e575bc060ce15724b INFO: 172.21.0.1:51478 - "POST /node/6940342e575bc060ce15724b HTTP/1.0" 202 Accepted 2025-12-17 16:15:57,794 - lava_callback - INFO - Starting async job submit for node 6940342e575bc060ce15724b 2025-12-17 16:15:57,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:57,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940342e575bc060ce15724b HTTP/1.1" 200 None 2025-12-17 16:15:57,804 - lava_callback - DEBUG - Processing node 6940342e575bc060ce15724b: baseline-arm-baylibre 2025-12-17 16:15:57,804 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940342e575bc060ce15724b 2025-12-17 16:15:57,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:57,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940342e575bc060ce15724b HTTP/1.1" 200 None 2025-12-17 16:15:57,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:57,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:15:57,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:57,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940342e575bc060ce15724b HTTP/1.1" 200 None 2025-12-17 16:15:57,842 - lava_callback - INFO - Completed processing callback for node 6940342e575bc060ce15724b 2025-12-17 16:15:58,440 - lava_callback - INFO - Received callback for node 69403430575bc060ce15724e INFO: 172.21.0.1:51490 - "POST /node/69403430575bc060ce15724e HTTP/1.0" 202 Accepted 2025-12-17 16:15:58,446 - lava_callback - INFO - Starting async job submit for node 69403430575bc060ce15724e 2025-12-17 16:15:58,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:58,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403430575bc060ce15724e HTTP/1.1" 200 None 2025-12-17 16:15:58,456 - lava_callback - DEBUG - Processing node 69403430575bc060ce15724e: baseline-arm-baylibre 2025-12-17 16:15:58,456 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403430575bc060ce15724e 2025-12-17 16:15:58,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:58,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403430575bc060ce15724e HTTP/1.1" 200 None 2025-12-17 16:15:58,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:58,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:15:58,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:58,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403430575bc060ce15724e HTTP/1.1" 200 None 2025-12-17 16:15:58,498 - lava_callback - INFO - Completed processing callback for node 69403430575bc060ce15724e 2025-12-17 16:15:59,093 - lava_callback - INFO - Received callback for node 69403431575bc060ce157251 INFO: 172.21.0.1:51492 - "POST /node/69403431575bc060ce157251 HTTP/1.0" 202 Accepted 2025-12-17 16:15:59,101 - lava_callback - INFO - Starting async job submit for node 69403431575bc060ce157251 2025-12-17 16:15:59,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:59,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403431575bc060ce157251 HTTP/1.1" 200 None 2025-12-17 16:15:59,111 - lava_callback - DEBUG - Processing node 69403431575bc060ce157251: baseline-arm-baylibre 2025-12-17 16:15:59,111 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403431575bc060ce157251 2025-12-17 16:15:59,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:59,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403431575bc060ce157251 HTTP/1.1" 200 None 2025-12-17 16:15:59,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:59,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:15:59,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:59,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403431575bc060ce157251 HTTP/1.1" 200 None 2025-12-17 16:15:59,155 - lava_callback - INFO - Completed processing callback for node 69403431575bc060ce157251 2025-12-17 16:15:59,771 - lava_callback - INFO - Received callback for node 69403432575bc060ce157254 INFO: 172.21.0.1:38398 - "POST /node/69403432575bc060ce157254 HTTP/1.0" 202 Accepted 2025-12-17 16:15:59,777 - lava_callback - INFO - Starting async job submit for node 69403432575bc060ce157254 2025-12-17 16:15:59,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:59,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403432575bc060ce157254 HTTP/1.1" 200 None 2025-12-17 16:15:59,790 - lava_callback - DEBUG - Processing node 69403432575bc060ce157254: baseline-arm-baylibre 2025-12-17 16:15:59,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403432575bc060ce157254 2025-12-17 16:15:59,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:59,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403432575bc060ce157254 HTTP/1.1" 200 None 2025-12-17 16:15:59,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:59,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:15:59,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:15:59,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403432575bc060ce157254 HTTP/1.1" 200 None 2025-12-17 16:15:59,838 - lava_callback - INFO - Completed processing callback for node 69403432575bc060ce157254 2025-12-17 16:16:00,452 - lava_callback - INFO - Received callback for node 69403434575bc060ce157257 INFO: 172.21.0.1:38406 - "POST /node/69403434575bc060ce157257 HTTP/1.0" 202 Accepted 2025-12-17 16:16:00,460 - lava_callback - INFO - Starting async job submit for node 69403434575bc060ce157257 2025-12-17 16:16:00,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:00,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403434575bc060ce157257 HTTP/1.1" 200 None 2025-12-17 16:16:00,473 - lava_callback - DEBUG - Processing node 69403434575bc060ce157257: baseline-arm-baylibre 2025-12-17 16:16:00,473 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403434575bc060ce157257 2025-12-17 16:16:00,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:00,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403434575bc060ce157257 HTTP/1.1" 200 None 2025-12-17 16:16:00,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:00,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:16:00,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:00,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403434575bc060ce157257 HTTP/1.1" 200 None 2025-12-17 16:16:00,518 - lava_callback - INFO - Completed processing callback for node 69403434575bc060ce157257 2025-12-17 16:16:01,090 - lava_callback - INFO - Received callback for node 69403435575bc060ce15725a INFO: 172.21.0.1:38410 - "POST /node/69403435575bc060ce15725a HTTP/1.0" 202 Accepted 2025-12-17 16:16:01,099 - lava_callback - INFO - Starting async job submit for node 69403435575bc060ce15725a 2025-12-17 16:16:01,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:01,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403435575bc060ce15725a HTTP/1.1" 200 None 2025-12-17 16:16:01,118 - lava_callback - DEBUG - Processing node 69403435575bc060ce15725a: baseline-arm-baylibre 2025-12-17 16:16:01,119 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403435575bc060ce15725a 2025-12-17 16:16:01,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:01,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403435575bc060ce15725a HTTP/1.1" 200 None 2025-12-17 16:16:01,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:01,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:16:01,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:01,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403435575bc060ce15725a HTTP/1.1" 200 None 2025-12-17 16:16:01,163 - lava_callback - INFO - Completed processing callback for node 69403435575bc060ce15725a 2025-12-17 16:16:01,750 - lava_callback - INFO - Received callback for node 69403436575bc060ce15725d INFO: 172.21.0.1:38416 - "POST /node/69403436575bc060ce15725d HTTP/1.0" 202 Accepted 2025-12-17 16:16:01,757 - lava_callback - INFO - Starting async job submit for node 69403436575bc060ce15725d 2025-12-17 16:16:01,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:01,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403436575bc060ce15725d HTTP/1.1" 200 None 2025-12-17 16:16:01,767 - lava_callback - DEBUG - Processing node 69403436575bc060ce15725d: baseline-arm-baylibre 2025-12-17 16:16:01,767 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403436575bc060ce15725d 2025-12-17 16:16:01,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:01,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403436575bc060ce15725d HTTP/1.1" 200 None 2025-12-17 16:16:01,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:01,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:16:01,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:01,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403436575bc060ce15725d HTTP/1.1" 200 None 2025-12-17 16:16:01,816 - lava_callback - INFO - Completed processing callback for node 69403436575bc060ce15725d 2025-12-17 16:16:02,401 - lava_callback - INFO - Received callback for node 69403437575bc060ce157260 INFO: 172.21.0.1:38418 - "POST /node/69403437575bc060ce157260 HTTP/1.0" 202 Accepted 2025-12-17 16:16:02,409 - lava_callback - INFO - Starting async job submit for node 69403437575bc060ce157260 2025-12-17 16:16:02,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:02,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403437575bc060ce157260 HTTP/1.1" 200 None 2025-12-17 16:16:02,419 - lava_callback - DEBUG - Processing node 69403437575bc060ce157260: baseline-arm-baylibre 2025-12-17 16:16:02,419 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403437575bc060ce157260 2025-12-17 16:16:02,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:02,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403437575bc060ce157260 HTTP/1.1" 200 None 2025-12-17 16:16:02,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:02,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:16:02,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:02,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403437575bc060ce157260 HTTP/1.1" 200 None 2025-12-17 16:16:02,464 - lava_callback - INFO - Completed processing callback for node 69403437575bc060ce157260 2025-12-17 16:16:03,078 - lava_callback - INFO - Received callback for node 69403438575bc060ce157263 INFO: 172.21.0.1:38426 - "POST /node/69403438575bc060ce157263 HTTP/1.0" 202 Accepted 2025-12-17 16:16:03,088 - lava_callback - INFO - Starting async job submit for node 69403438575bc060ce157263 2025-12-17 16:16:03,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:03,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403438575bc060ce157263 HTTP/1.1" 200 None 2025-12-17 16:16:03,098 - lava_callback - DEBUG - Processing node 69403438575bc060ce157263: baseline-arm-baylibre 2025-12-17 16:16:03,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403438575bc060ce157263 2025-12-17 16:16:03,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:03,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403438575bc060ce157263 HTTP/1.1" 200 None 2025-12-17 16:16:03,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:03,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:16:03,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:03,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403438575bc060ce157263 HTTP/1.1" 200 None 2025-12-17 16:16:03,137 - lava_callback - INFO - Completed processing callback for node 69403438575bc060ce157263 2025-12-17 16:16:05,968 - lava_callback - INFO - Received callback for node 6940343a575bc060ce157266 INFO: 172.21.0.1:38438 - "POST /node/6940343a575bc060ce157266 HTTP/1.0" 202 Accepted 2025-12-17 16:16:05,975 - lava_callback - INFO - Starting async job submit for node 6940343a575bc060ce157266 2025-12-17 16:16:05,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:05,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940343a575bc060ce157266 HTTP/1.1" 200 None 2025-12-17 16:16:05,985 - lava_callback - DEBUG - Processing node 6940343a575bc060ce157266: baseline-arm-broonie 2025-12-17 16:16:05,985 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940343a575bc060ce157266 2025-12-17 16:16:05,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:05,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940343a575bc060ce157266 HTTP/1.1" 200 None 2025-12-17 16:16:05,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:06,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:16:06,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:06,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940343a575bc060ce157266 HTTP/1.1" 200 None 2025-12-17 16:16:06,052 - lava_callback - INFO - Completed processing callback for node 6940343a575bc060ce157266 INFO: 172.21.0.1:38446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56700 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:16:14,512 - lava_callback - INFO - Received callback for node 6940343b575bc060ce157269 INFO: 172.21.0.1:56716 - "POST /node/6940343b575bc060ce157269 HTTP/1.0" 202 Accepted 2025-12-17 16:16:14,520 - lava_callback - INFO - Starting async job submit for node 6940343b575bc060ce157269 2025-12-17 16:16:14,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:14,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940343b575bc060ce157269 HTTP/1.1" 200 None 2025-12-17 16:16:14,529 - lava_callback - DEBUG - Processing node 6940343b575bc060ce157269: baseline-arm-baylibre 2025-12-17 16:16:14,530 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940343b575bc060ce157269 2025-12-17 16:16:14,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:14,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940343b575bc060ce157269 HTTP/1.1" 200 None 2025-12-17 16:16:14,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:14,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:16:14,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:14,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940343b575bc060ce157269 HTTP/1.1" 200 None 2025-12-17 16:16:14,568 - lava_callback - INFO - Completed processing callback for node 6940343b575bc060ce157269 INFO: 172.21.0.1:56726 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:16:15,170 - lava_callback - INFO - Received callback for node 6940343c575bc060ce15726c INFO: 172.21.0.1:56738 - "POST /node/6940343c575bc060ce15726c HTTP/1.0" 202 Accepted 2025-12-17 16:16:15,179 - lava_callback - INFO - Starting async job submit for node 6940343c575bc060ce15726c 2025-12-17 16:16:15,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:15,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940343c575bc060ce15726c HTTP/1.1" 200 None 2025-12-17 16:16:15,192 - lava_callback - DEBUG - Processing node 6940343c575bc060ce15726c: baseline-arm-baylibre 2025-12-17 16:16:15,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940343c575bc060ce15726c 2025-12-17 16:16:15,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:15,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940343c575bc060ce15726c HTTP/1.1" 200 None 2025-12-17 16:16:15,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:15,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:16:15,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:15,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940343c575bc060ce15726c HTTP/1.1" 200 None 2025-12-17 16:16:15,240 - lava_callback - INFO - Completed processing callback for node 6940343c575bc060ce15726c 2025-12-17 16:16:15,827 - lava_callback - INFO - Received callback for node 69403442575bc060ce157278 INFO: 172.21.0.1:56742 - "POST /node/69403442575bc060ce157278 HTTP/1.0" 202 Accepted 2025-12-17 16:16:15,835 - lava_callback - INFO - Starting async job submit for node 69403442575bc060ce157278 2025-12-17 16:16:15,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:15,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403442575bc060ce157278 HTTP/1.1" 200 None 2025-12-17 16:16:15,845 - lava_callback - DEBUG - Processing node 69403442575bc060ce157278: baseline-arm64-baylibre 2025-12-17 16:16:15,845 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403442575bc060ce157278 2025-12-17 16:16:15,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:15,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403442575bc060ce157278 HTTP/1.1" 200 None 2025-12-17 16:16:15,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:15,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:15,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:15,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403442575bc060ce157278 HTTP/1.1" 200 None 2025-12-17 16:16:15,899 - lava_callback - INFO - Completed processing callback for node 69403442575bc060ce157278 2025-12-17 16:16:16,500 - lava_callback - INFO - Received callback for node 69403446575bc060ce15727e INFO: 172.21.0.1:56744 - "POST /node/69403446575bc060ce15727e HTTP/1.0" 202 Accepted 2025-12-17 16:16:16,509 - lava_callback - INFO - Starting async job submit for node 69403446575bc060ce15727e 2025-12-17 16:16:16,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:56756 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:16:16,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403446575bc060ce15727e HTTP/1.1" 200 None 2025-12-17 16:16:16,522 - lava_callback - DEBUG - Processing node 69403446575bc060ce15727e: baseline-arm64-baylibre 2025-12-17 16:16:16,522 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403446575bc060ce15727e 2025-12-17 16:16:16,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:16,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403446575bc060ce15727e HTTP/1.1" 200 None 2025-12-17 16:16:16,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:16,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:16,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:16,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403446575bc060ce15727e HTTP/1.1" 200 None 2025-12-17 16:16:16,577 - lava_callback - INFO - Completed processing callback for node 69403446575bc060ce15727e 2025-12-17 16:16:17,174 - lava_callback - INFO - Received callback for node 69403448575bc060ce157281 INFO: 172.21.0.1:56762 - "POST /node/69403448575bc060ce157281 HTTP/1.0" 202 Accepted 2025-12-17 16:16:17,182 - lava_callback - INFO - Starting async job submit for node 69403448575bc060ce157281 2025-12-17 16:16:17,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:17,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403448575bc060ce157281 HTTP/1.1" 200 None 2025-12-17 16:16:17,199 - lava_callback - DEBUG - Processing node 69403448575bc060ce157281: baseline-arm64-baylibre 2025-12-17 16:16:17,199 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403448575bc060ce157281 2025-12-17 16:16:17,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:17,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403448575bc060ce157281 HTTP/1.1" 200 None 2025-12-17 16:16:17,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:17,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:17,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:17,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403448575bc060ce157281 HTTP/1.1" 200 None 2025-12-17 16:16:17,251 - lava_callback - INFO - Completed processing callback for node 69403448575bc060ce157281 2025-12-17 16:16:17,826 - lava_callback - INFO - Received callback for node 6940344a575bc060ce15728a INFO: 172.21.0.1:56774 - "POST /node/6940344a575bc060ce15728a HTTP/1.0" 202 Accepted 2025-12-17 16:16:17,835 - lava_callback - INFO - Starting async job submit for node 6940344a575bc060ce15728a 2025-12-17 16:16:17,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:17,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940344a575bc060ce15728a HTTP/1.1" 200 None 2025-12-17 16:16:17,847 - lava_callback - DEBUG - Processing node 6940344a575bc060ce15728a: baseline-arm64-baylibre 2025-12-17 16:16:17,847 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940344a575bc060ce15728a 2025-12-17 16:16:17,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:17,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940344a575bc060ce15728a HTTP/1.1" 200 None 2025-12-17 16:16:17,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:17,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:17,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:17,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940344a575bc060ce15728a HTTP/1.1" 200 None 2025-12-17 16:16:17,910 - lava_callback - INFO - Completed processing callback for node 6940344a575bc060ce15728a 2025-12-17 16:16:18,514 - lava_callback - INFO - Received callback for node 6940344c575bc060ce157293 INFO: 172.21.0.1:56786 - "POST /node/6940344c575bc060ce157293 HTTP/1.0" 202 Accepted 2025-12-17 16:16:18,523 - lava_callback - INFO - Starting async job submit for node 6940344c575bc060ce157293 2025-12-17 16:16:18,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:18,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940344c575bc060ce157293 HTTP/1.1" 200 None 2025-12-17 16:16:18,535 - lava_callback - DEBUG - Processing node 6940344c575bc060ce157293: baseline-arm64-baylibre 2025-12-17 16:16:18,535 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940344c575bc060ce157293 2025-12-17 16:16:18,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:18,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940344c575bc060ce157293 HTTP/1.1" 200 None 2025-12-17 16:16:18,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:18,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:18,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:18,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940344c575bc060ce157293 HTTP/1.1" 200 None 2025-12-17 16:16:18,606 - lava_callback - INFO - Completed processing callback for node 6940344c575bc060ce157293 INFO: 172.21.0.1:46176 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:16:34,495 - lava_callback - INFO - Received callback for node 6940344e575bc060ce157296 INFO: 172.21.0.1:34078 - "POST /node/6940344e575bc060ce157296 HTTP/1.0" 202 Accepted 2025-12-17 16:16:34,501 - lava_callback - INFO - Starting async job submit for node 6940344e575bc060ce157296 2025-12-17 16:16:34,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:34,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940344e575bc060ce157296 HTTP/1.1" 200 None 2025-12-17 16:16:34,510 - lava_callback - DEBUG - Processing node 6940344e575bc060ce157296: baseline-arm64-baylibre 2025-12-17 16:16:34,510 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940344e575bc060ce157296 2025-12-17 16:16:34,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:34,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940344e575bc060ce157296 HTTP/1.1" 200 None 2025-12-17 16:16:34,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:34,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:34,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:34,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940344e575bc060ce157296 HTTP/1.1" 200 None 2025-12-17 16:16:34,554 - lava_callback - INFO - Completed processing callback for node 6940344e575bc060ce157296 2025-12-17 16:16:35,187 - lava_callback - INFO - Received callback for node 69403450575bc060ce157299 INFO: 172.21.0.1:34082 - "POST /node/69403450575bc060ce157299 HTTP/1.0" 202 Accepted 2025-12-17 16:16:35,194 - lava_callback - INFO - Starting async job submit for node 69403450575bc060ce157299 2025-12-17 16:16:35,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:35,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403450575bc060ce157299 HTTP/1.1" 200 None 2025-12-17 16:16:35,205 - lava_callback - DEBUG - Processing node 69403450575bc060ce157299: baseline-arm64-baylibre 2025-12-17 16:16:35,205 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403450575bc060ce157299 2025-12-17 16:16:35,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:35,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403450575bc060ce157299 HTTP/1.1" 200 None 2025-12-17 16:16:35,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:35,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:35,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:35,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403450575bc060ce157299 HTTP/1.1" 200 None 2025-12-17 16:16:35,239 - lava_callback - INFO - Completed processing callback for node 69403450575bc060ce157299 2025-12-17 16:16:35,850 - lava_callback - INFO - Received callback for node 69403452575bc060ce15729c INFO: 172.21.0.1:34088 - "POST /node/69403452575bc060ce15729c HTTP/1.0" 202 Accepted 2025-12-17 16:16:35,856 - lava_callback - INFO - Starting async job submit for node 69403452575bc060ce15729c 2025-12-17 16:16:35,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:35,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403452575bc060ce15729c HTTP/1.1" 200 None 2025-12-17 16:16:35,870 - lava_callback - DEBUG - Processing node 69403452575bc060ce15729c: baseline-arm64-baylibre 2025-12-17 16:16:35,870 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403452575bc060ce15729c 2025-12-17 16:16:35,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:35,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403452575bc060ce15729c HTTP/1.1" 200 None 2025-12-17 16:16:35,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:35,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:35,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:35,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403452575bc060ce15729c HTTP/1.1" 200 None 2025-12-17 16:16:35,916 - lava_callback - INFO - Completed processing callback for node 69403452575bc060ce15729c 2025-12-17 16:16:36,501 - lava_callback - INFO - Received callback for node 69403454575bc060ce15729f INFO: 172.21.0.1:34092 - "POST /node/69403454575bc060ce15729f HTTP/1.0" 202 Accepted 2025-12-17 16:16:36,509 - lava_callback - INFO - Starting async job submit for node 69403454575bc060ce15729f 2025-12-17 16:16:36,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:34106 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:16:36,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403454575bc060ce15729f HTTP/1.1" 200 None 2025-12-17 16:16:36,520 - lava_callback - DEBUG - Processing node 69403454575bc060ce15729f: baseline-arm64-baylibre 2025-12-17 16:16:36,520 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403454575bc060ce15729f 2025-12-17 16:16:36,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:36,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403454575bc060ce15729f HTTP/1.1" 200 None 2025-12-17 16:16:36,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:36,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:36,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:36,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403454575bc060ce15729f HTTP/1.1" 200 None 2025-12-17 16:16:36,562 - lava_callback - INFO - Completed processing callback for node 69403454575bc060ce15729f 2025-12-17 16:16:37,187 - lava_callback - INFO - Received callback for node 6940345a575bc060ce1572a2 INFO: 172.21.0.1:34110 - "POST /node/6940345a575bc060ce1572a2 HTTP/1.0" 202 Accepted 2025-12-17 16:16:37,194 - lava_callback - INFO - Starting async job submit for node 6940345a575bc060ce1572a2 2025-12-17 16:16:37,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:37,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940345a575bc060ce1572a2 HTTP/1.1" 200 None 2025-12-17 16:16:37,215 - lava_callback - DEBUG - Processing node 6940345a575bc060ce1572a2: baseline-arm64-baylibre 2025-12-17 16:16:37,215 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940345a575bc060ce1572a2 2025-12-17 16:16:37,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:37,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940345a575bc060ce1572a2 HTTP/1.1" 200 None 2025-12-17 16:16:37,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:37,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:37,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:37,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940345a575bc060ce1572a2 HTTP/1.1" 200 None 2025-12-17 16:16:37,251 - lava_callback - INFO - Completed processing callback for node 6940345a575bc060ce1572a2 2025-12-17 16:16:37,849 - lava_callback - INFO - Received callback for node 6940345c575bc060ce1572a5 INFO: 172.21.0.1:34118 - "POST /node/6940345c575bc060ce1572a5 HTTP/1.0" 202 Accepted 2025-12-17 16:16:37,857 - lava_callback - INFO - Starting async job submit for node 6940345c575bc060ce1572a5 2025-12-17 16:16:37,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:37,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940345c575bc060ce1572a5 HTTP/1.1" 200 None 2025-12-17 16:16:37,867 - lava_callback - DEBUG - Processing node 6940345c575bc060ce1572a5: baseline-arm64-baylibre 2025-12-17 16:16:37,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940345c575bc060ce1572a5 2025-12-17 16:16:37,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:37,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940345c575bc060ce1572a5 HTTP/1.1" 200 None 2025-12-17 16:16:37,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:37,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:37,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:37,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940345c575bc060ce1572a5 HTTP/1.1" 200 None 2025-12-17 16:16:37,911 - lava_callback - INFO - Completed processing callback for node 6940345c575bc060ce1572a5 2025-12-17 16:16:38,518 - lava_callback - INFO - Received callback for node 6940345e575bc060ce1572ab INFO: 172.21.0.1:34128 - "POST /node/6940345e575bc060ce1572ab HTTP/1.0" 202 Accepted 2025-12-17 16:16:38,526 - lava_callback - INFO - Starting async job submit for node 6940345e575bc060ce1572ab 2025-12-17 16:16:38,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:38,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940345e575bc060ce1572ab HTTP/1.1" 200 None 2025-12-17 16:16:38,536 - lava_callback - DEBUG - Processing node 6940345e575bc060ce1572ab: baseline-arm64-baylibre 2025-12-17 16:16:38,537 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940345e575bc060ce1572ab 2025-12-17 16:16:38,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:38,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940345e575bc060ce1572ab HTTP/1.1" 200 None 2025-12-17 16:16:38,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:38,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:38,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:38,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940345e575bc060ce1572ab HTTP/1.1" 200 None 2025-12-17 16:16:38,601 - lava_callback - INFO - Completed processing callback for node 6940345e575bc060ce1572ab 2025-12-17 16:16:39,190 - lava_callback - INFO - Received callback for node 69403460575bc060ce1572ae INFO: 172.21.0.1:34140 - "POST /node/69403460575bc060ce1572ae HTTP/1.0" 202 Accepted 2025-12-17 16:16:39,199 - lava_callback - INFO - Starting async job submit for node 69403460575bc060ce1572ae 2025-12-17 16:16:39,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:39,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403460575bc060ce1572ae HTTP/1.1" 200 None 2025-12-17 16:16:39,210 - lava_callback - DEBUG - Processing node 69403460575bc060ce1572ae: baseline-arm64-baylibre 2025-12-17 16:16:39,210 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403460575bc060ce1572ae 2025-12-17 16:16:39,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:39,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403460575bc060ce1572ae HTTP/1.1" 200 None 2025-12-17 16:16:39,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:39,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:39,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:39,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403460575bc060ce1572ae HTTP/1.1" 200 None 2025-12-17 16:16:39,254 - lava_callback - INFO - Completed processing callback for node 69403460575bc060ce1572ae INFO: 172.21.0.1:54452 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:16:49,145 - lava_callback - INFO - Received callback for node 69403467575bc060ce1572b7 INFO: 172.21.0.1:54468 - "POST /node/69403467575bc060ce1572b7 HTTP/1.0" 202 Accepted 2025-12-17 16:16:49,152 - lava_callback - INFO - Starting async job submit for node 69403467575bc060ce1572b7 2025-12-17 16:16:49,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:49,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403467575bc060ce1572b7 HTTP/1.1" 200 None 2025-12-17 16:16:49,162 - lava_callback - DEBUG - Processing node 69403467575bc060ce1572b7: baseline-arm64-clabbe 2025-12-17 16:16:49,162 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403467575bc060ce1572b7 2025-12-17 16:16:49,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:49,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403467575bc060ce1572b7 HTTP/1.1" 200 None 2025-12-17 16:16:49,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:49,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:49,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:49,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403467575bc060ce1572b7 HTTP/1.1" 200 None 2025-12-17 16:16:49,210 - lava_callback - INFO - Completed processing callback for node 69403467575bc060ce1572b7 2025-12-17 16:16:54,528 - lava_callback - INFO - Received callback for node 69403463575bc060ce1572b1 INFO: 172.21.0.1:45784 - "POST /node/69403463575bc060ce1572b1 HTTP/1.0" 202 Accepted 2025-12-17 16:16:54,536 - lava_callback - INFO - Starting async job submit for node 69403463575bc060ce1572b1 2025-12-17 16:16:54,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:54,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403463575bc060ce1572b1 HTTP/1.1" 200 None 2025-12-17 16:16:54,546 - lava_callback - DEBUG - Processing node 69403463575bc060ce1572b1: baseline-arm64-baylibre 2025-12-17 16:16:54,546 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403463575bc060ce1572b1 2025-12-17 16:16:54,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:54,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403463575bc060ce1572b1 HTTP/1.1" 200 None 2025-12-17 16:16:54,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:54,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:54,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:54,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403463575bc060ce1572b1 HTTP/1.1" 200 None 2025-12-17 16:16:54,584 - lava_callback - INFO - Completed processing callback for node 69403463575bc060ce1572b1 2025-12-17 16:16:55,193 - lava_callback - INFO - Received callback for node 69403465575bc060ce1572b4 INFO: 172.21.0.1:45796 - "POST /node/69403465575bc060ce1572b4 HTTP/1.0" 202 Accepted 2025-12-17 16:16:55,201 - lava_callback - INFO - Starting async job submit for node 69403465575bc060ce1572b4 2025-12-17 16:16:55,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:55,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403465575bc060ce1572b4 HTTP/1.1" 200 None 2025-12-17 16:16:55,210 - lava_callback - DEBUG - Processing node 69403465575bc060ce1572b4: baseline-arm64-baylibre 2025-12-17 16:16:55,210 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403465575bc060ce1572b4 2025-12-17 16:16:55,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:55,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403465575bc060ce1572b4 HTTP/1.1" 200 None 2025-12-17 16:16:55,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:55,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:55,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:55,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403465575bc060ce1572b4 HTTP/1.1" 200 None 2025-12-17 16:16:55,251 - lava_callback - INFO - Completed processing callback for node 69403465575bc060ce1572b4 2025-12-17 16:16:55,863 - lava_callback - INFO - Received callback for node 6940346a575bc060ce1572ba INFO: 172.21.0.1:45800 - "POST /node/6940346a575bc060ce1572ba HTTP/1.0" 202 Accepted 2025-12-17 16:16:55,873 - lava_callback - INFO - Starting async job submit for node 6940346a575bc060ce1572ba 2025-12-17 16:16:55,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:55,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940346a575bc060ce1572ba HTTP/1.1" 200 None 2025-12-17 16:16:55,882 - lava_callback - DEBUG - Processing node 6940346a575bc060ce1572ba: baseline-arm64-baylibre 2025-12-17 16:16:55,882 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940346a575bc060ce1572ba 2025-12-17 16:16:55,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:55,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940346a575bc060ce1572ba HTTP/1.1" 200 None 2025-12-17 16:16:55,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:55,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:55,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:55,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940346a575bc060ce1572ba HTTP/1.1" 200 None 2025-12-17 16:16:55,919 - lava_callback - INFO - Completed processing callback for node 6940346a575bc060ce1572ba INFO: 172.21.0.1:45808 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:16:56,532 - lava_callback - INFO - Received callback for node 69403472575bc060ce1572c9 INFO: 172.21.0.1:45810 - "POST /node/69403472575bc060ce1572c9 HTTP/1.0" 202 Accepted 2025-12-17 16:16:56,538 - lava_callback - INFO - Starting async job submit for node 69403472575bc060ce1572c9 2025-12-17 16:16:56,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:56,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403472575bc060ce1572c9 HTTP/1.1" 200 None 2025-12-17 16:16:56,548 - lava_callback - DEBUG - Processing node 69403472575bc060ce1572c9: baseline-arm64-baylibre 2025-12-17 16:16:56,548 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403472575bc060ce1572c9 2025-12-17 16:16:56,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:56,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403472575bc060ce1572c9 HTTP/1.1" 200 None 2025-12-17 16:16:56,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:56,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:16:56,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:16:56,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403472575bc060ce1572c9 HTTP/1.1" 200 None 2025-12-17 16:16:56,579 - lava_callback - INFO - Completed processing callback for node 69403472575bc060ce1572c9 INFO: 172.21.0.1:52822 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:17:09,110 - lava_callback - INFO - Received callback for node 69403470575bc060ce1572c1 INFO: 172.21.0.1:52836 - "POST /node/69403470575bc060ce1572c1 HTTP/1.0" 202 Accepted 2025-12-17 16:17:09,116 - lava_callback - INFO - Starting async job submit for node 69403470575bc060ce1572c1 2025-12-17 16:17:09,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:09,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403470575bc060ce1572c1 HTTP/1.1" 200 None 2025-12-17 16:17:09,126 - lava_callback - DEBUG - Processing node 69403470575bc060ce1572c1: baseline-arm64-clabbe 2025-12-17 16:17:09,126 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403470575bc060ce1572c1 2025-12-17 16:17:09,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:09,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403470575bc060ce1572c1 HTTP/1.1" 200 None 2025-12-17 16:17:09,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:09,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:17:09,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:09,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403470575bc060ce1572c1 HTTP/1.1" 200 None 2025-12-17 16:17:09,166 - lava_callback - INFO - Completed processing callback for node 69403470575bc060ce1572c1 2025-12-17 16:17:09,777 - lava_callback - INFO - Received callback for node 69403474575bc060ce1572d1 INFO: 172.21.0.1:43430 - "POST /node/69403474575bc060ce1572d1 HTTP/1.0" 202 Accepted 2025-12-17 16:17:09,785 - lava_callback - INFO - Starting async job submit for node 69403474575bc060ce1572d1 2025-12-17 16:17:09,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:09,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403474575bc060ce1572d1 HTTP/1.1" 200 None 2025-12-17 16:17:09,819 - lava_callback - DEBUG - Processing node 69403474575bc060ce1572d1: baseline-arm64-clabbe 2025-12-17 16:17:09,819 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403474575bc060ce1572d1 2025-12-17 16:17:09,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:09,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403474575bc060ce1572d1 HTTP/1.1" 200 None 2025-12-17 16:17:09,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:09,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:17:09,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:09,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403474575bc060ce1572d1 HTTP/1.1" 200 None 2025-12-17 16:17:09,924 - lava_callback - INFO - Completed processing callback for node 69403474575bc060ce1572d1 2025-12-17 16:17:10,443 - lava_callback - INFO - Received callback for node 6940347f575bc060ce1572e8 INFO: 172.21.0.1:43440 - "POST /node/6940347f575bc060ce1572e8 HTTP/1.0" 202 Accepted 2025-12-17 16:17:10,449 - lava_callback - INFO - Starting async job submit for node 6940347f575bc060ce1572e8 2025-12-17 16:17:10,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:10,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940347f575bc060ce1572e8 HTTP/1.1" 200 None 2025-12-17 16:17:10,458 - lava_callback - DEBUG - Processing node 6940347f575bc060ce1572e8: baseline-arm64-clabbe 2025-12-17 16:17:10,458 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940347f575bc060ce1572e8 2025-12-17 16:17:10,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:10,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940347f575bc060ce1572e8 HTTP/1.1" 200 None 2025-12-17 16:17:10,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:10,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:17:10,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:10,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940347f575bc060ce1572e8 HTTP/1.1" 200 None 2025-12-17 16:17:10,492 - lava_callback - INFO - Completed processing callback for node 6940347f575bc060ce1572e8 INFO: 172.21.0.1:43450 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:17:14,533 - lava_callback - INFO - Received callback for node 69403476575bc060ce1572d4 INFO: 172.21.0.1:43458 - "POST /node/69403476575bc060ce1572d4 HTTP/1.0" 202 Accepted 2025-12-17 16:17:14,539 - lava_callback - INFO - Starting async job submit for node 69403476575bc060ce1572d4 2025-12-17 16:17:14,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:14,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403476575bc060ce1572d4 HTTP/1.1" 200 None 2025-12-17 16:17:14,550 - lava_callback - DEBUG - Processing node 69403476575bc060ce1572d4: baseline-arm64-baylibre 2025-12-17 16:17:14,550 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403476575bc060ce1572d4 2025-12-17 16:17:14,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:14,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403476575bc060ce1572d4 HTTP/1.1" 200 None 2025-12-17 16:17:14,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:14,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:17:14,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:14,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403476575bc060ce1572d4 HTTP/1.1" 200 None 2025-12-17 16:17:14,589 - lava_callback - INFO - Completed processing callback for node 69403476575bc060ce1572d4 INFO: 172.21.0.1:43464 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:17:15,209 - lava_callback - INFO - Received callback for node 6940347c575bc060ce1572e5 INFO: 172.21.0.1:43476 - "POST /node/6940347c575bc060ce1572e5 HTTP/1.0" 202 Accepted 2025-12-17 16:17:15,215 - lava_callback - INFO - Starting async job submit for node 6940347c575bc060ce1572e5 2025-12-17 16:17:15,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:15,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940347c575bc060ce1572e5 HTTP/1.1" 200 None 2025-12-17 16:17:15,227 - lava_callback - DEBUG - Processing node 6940347c575bc060ce1572e5: baseline-arm64-baylibre 2025-12-17 16:17:15,228 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940347c575bc060ce1572e5 2025-12-17 16:17:15,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:15,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940347c575bc060ce1572e5 HTTP/1.1" 200 None 2025-12-17 16:17:15,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:15,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:17:15,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:15,252 - lava_callback - INFO - Received callback for node 69403486575bc060ce1572f7 INFO: 172.21.0.1:43492 - "POST /node/69403486575bc060ce1572f7 HTTP/1.0" 202 Accepted 2025-12-17 16:17:15,260 - lava_callback - INFO - Starting async job submit for node 69403486575bc060ce1572f7 2025-12-17 16:17:15,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:15,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403486575bc060ce1572f7 HTTP/1.1" 200 None 2025-12-17 16:17:15,270 - lava_callback - DEBUG - Processing node 69403486575bc060ce1572f7: baseline-arm 2025-12-17 16:17:15,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403486575bc060ce1572f7 2025-12-17 16:17:15,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:15,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940347c575bc060ce1572e5 HTTP/1.1" 200 None 2025-12-17 16:17:15,281 - lava_callback - INFO - Completed processing callback for node 6940347c575bc060ce1572e5 2025-12-17 16:17:15,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403486575bc060ce1572f7 HTTP/1.1" 200 None 2025-12-17 16:17:15,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:15,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7ecfb52d1dfff36c48a HTTP/1.1" 200 None 2025-12-17 16:17:15,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:15,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403486575bc060ce1572f7 HTTP/1.1" 200 None 2025-12-17 16:17:15,346 - lava_callback - INFO - Completed processing callback for node 69403486575bc060ce1572f7 2025-12-17 16:17:15,873 - lava_callback - INFO - Received callback for node 69403480575bc060ce1572eb INFO: 172.21.0.1:43496 - "POST /node/69403480575bc060ce1572eb HTTP/1.0" 202 Accepted 2025-12-17 16:17:15,879 - lava_callback - INFO - Starting async job submit for node 69403480575bc060ce1572eb 2025-12-17 16:17:15,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:15,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403480575bc060ce1572eb HTTP/1.1" 200 None 2025-12-17 16:17:15,892 - lava_callback - DEBUG - Processing node 69403480575bc060ce1572eb: baseline-arm64-baylibre 2025-12-17 16:17:15,892 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403480575bc060ce1572eb 2025-12-17 16:17:15,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:15,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403480575bc060ce1572eb HTTP/1.1" 200 None 2025-12-17 16:17:15,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:15,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:17:15,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:15,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403480575bc060ce1572eb HTTP/1.1" 200 None 2025-12-17 16:17:15,928 - lava_callback - INFO - Completed processing callback for node 69403480575bc060ce1572eb INFO: 172.21.0.1:43506 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:17:26,236 - lava_callback - INFO - Received callback for node 69403482575bc060ce1572ee INFO: 172.21.0.1:47486 - "POST /node/69403482575bc060ce1572ee HTTP/1.0" 202 Accepted 2025-12-17 16:17:26,245 - lava_callback - INFO - Starting async job submit for node 69403482575bc060ce1572ee 2025-12-17 16:17:26,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:26,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403482575bc060ce1572ee HTTP/1.1" 200 None 2025-12-17 16:17:26,256 - lava_callback - DEBUG - Processing node 69403482575bc060ce1572ee: baseline-arm-broonie 2025-12-17 16:17:26,256 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403482575bc060ce1572ee 2025-12-17 16:17:26,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:26,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403482575bc060ce1572ee HTTP/1.1" 200 None 2025-12-17 16:17:26,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:26,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:17:26,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:26,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403482575bc060ce1572ee HTTP/1.1" 200 None 2025-12-17 16:17:26,301 - lava_callback - INFO - Completed processing callback for node 69403482575bc060ce1572ee INFO: 172.21.0.1:47498 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:17:26,971 - lava_callback - INFO - Received callback for node 69403483575bc060ce1572f1 INFO: 172.21.0.1:47510 - "POST /node/69403483575bc060ce1572f1 HTTP/1.0" 202 Accepted 2025-12-17 16:17:26,978 - lava_callback - INFO - Starting async job submit for node 69403483575bc060ce1572f1 2025-12-17 16:17:26,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:26,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403483575bc060ce1572f1 HTTP/1.1" 200 None 2025-12-17 16:17:26,988 - lava_callback - DEBUG - Processing node 69403483575bc060ce1572f1: baseline-arm-broonie 2025-12-17 16:17:26,988 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403483575bc060ce1572f1 2025-12-17 16:17:26,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:26,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403483575bc060ce1572f1 HTTP/1.1" 200 None 2025-12-17 16:17:26,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:27,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:17:27,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:27,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403483575bc060ce1572f1 HTTP/1.1" 200 None 2025-12-17 16:17:27,023 - lava_callback - INFO - Completed processing callback for node 69403483575bc060ce1572f1 2025-12-17 16:17:29,120 - lava_callback - INFO - Received callback for node 69403485575bc060ce1572f4 INFO: 172.21.0.1:47514 - "POST /node/69403485575bc060ce1572f4 HTTP/1.0" 202 Accepted 2025-12-17 16:17:29,126 - lava_callback - INFO - Starting async job submit for node 69403485575bc060ce1572f4 2025-12-17 16:17:29,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:29,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403485575bc060ce1572f4 HTTP/1.1" 200 None 2025-12-17 16:17:29,136 - lava_callback - DEBUG - Processing node 69403485575bc060ce1572f4: baseline-arm-clabbe 2025-12-17 16:17:29,137 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403485575bc060ce1572f4 2025-12-17 16:17:29,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:29,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403485575bc060ce1572f4 HTTP/1.1" 200 None 2025-12-17 16:17:29,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:29,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:17:29,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:29,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403485575bc060ce1572f4 HTTP/1.1" 200 None 2025-12-17 16:17:29,171 - lava_callback - INFO - Completed processing callback for node 69403485575bc060ce1572f4 2025-12-17 16:17:29,788 - lava_callback - INFO - Received callback for node 6940348b575bc060ce157300 INFO: 172.21.0.1:32820 - "POST /node/6940348b575bc060ce157300 HTTP/1.0" 202 Accepted 2025-12-17 16:17:29,797 - lava_callback - INFO - Starting async job submit for node 6940348b575bc060ce157300 2025-12-17 16:17:29,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:29,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940348b575bc060ce157300 HTTP/1.1" 200 None 2025-12-17 16:17:29,810 - lava_callback - DEBUG - Processing node 6940348b575bc060ce157300: baseline-arm-clabbe 2025-12-17 16:17:29,810 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940348b575bc060ce157300 2025-12-17 16:17:29,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:29,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940348b575bc060ce157300 HTTP/1.1" 200 None 2025-12-17 16:17:29,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:29,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:17:29,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:29,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940348b575bc060ce157300 HTTP/1.1" 200 None 2025-12-17 16:17:29,852 - lava_callback - INFO - Completed processing callback for node 6940348b575bc060ce157300 2025-12-17 16:17:30,591 - lava_callback - INFO - Received callback for node 6940348c575bc060ce157303 INFO: 172.21.0.1:32836 - "POST /node/6940348c575bc060ce157303 HTTP/1.0" 202 Accepted 2025-12-17 16:17:30,598 - lava_callback - INFO - Starting async job submit for node 6940348c575bc060ce157303 2025-12-17 16:17:30,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:30,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940348c575bc060ce157303 HTTP/1.1" 200 None 2025-12-17 16:17:30,613 - lava_callback - DEBUG - Processing node 6940348c575bc060ce157303: baseline-arm-clabbe 2025-12-17 16:17:30,614 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940348c575bc060ce157303 2025-12-17 16:17:30,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:30,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940348c575bc060ce157303 HTTP/1.1" 200 None 2025-12-17 16:17:30,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:30,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:17:30,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:30,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940348c575bc060ce157303 HTTP/1.1" 200 None 2025-12-17 16:17:30,656 - lava_callback - INFO - Completed processing callback for node 6940348c575bc060ce157303 2025-12-17 16:17:31,242 - lava_callback - INFO - Received callback for node 6940348e575bc060ce157306 INFO: 172.21.0.1:32848 - "POST /node/6940348e575bc060ce157306 HTTP/1.0" 202 Accepted 2025-12-17 16:17:31,253 - lava_callback - INFO - Starting async job submit for node 6940348e575bc060ce157306 2025-12-17 16:17:31,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:31,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940348e575bc060ce157306 HTTP/1.1" 200 None 2025-12-17 16:17:31,273 - lava_callback - DEBUG - Processing node 6940348e575bc060ce157306: baseline-arm-clabbe 2025-12-17 16:17:31,273 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940348e575bc060ce157306 2025-12-17 16:17:31,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:31,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940348e575bc060ce157306 HTTP/1.1" 200 None 2025-12-17 16:17:31,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:31,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:17:31,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:31,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940348e575bc060ce157306 HTTP/1.1" 200 None 2025-12-17 16:17:31,337 - lava_callback - INFO - Completed processing callback for node 6940348e575bc060ce157306 2025-12-17 16:17:31,911 - lava_callback - INFO - Received callback for node 6940348f575bc060ce15730d INFO: 172.21.0.1:32862 - "POST /node/6940348f575bc060ce15730d HTTP/1.0" 202 Accepted 2025-12-17 16:17:31,917 - lava_callback - INFO - Starting async job submit for node 6940348f575bc060ce15730d 2025-12-17 16:17:31,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:31,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940348f575bc060ce15730d HTTP/1.1" 200 None 2025-12-17 16:17:31,925 - lava_callback - DEBUG - Processing node 6940348f575bc060ce15730d: baseline-arm-clabbe 2025-12-17 16:17:31,926 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940348f575bc060ce15730d 2025-12-17 16:17:31,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:31,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940348f575bc060ce15730d HTTP/1.1" 200 None 2025-12-17 16:17:31,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:31,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:17:31,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:31,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940348f575bc060ce15730d HTTP/1.1" 200 None 2025-12-17 16:17:31,959 - lava_callback - INFO - Completed processing callback for node 6940348f575bc060ce15730d 2025-12-17 16:17:32,568 - lava_callback - INFO - Received callback for node 69403490575bc060ce157310 INFO: 172.21.0.1:32874 - "POST /node/69403490575bc060ce157310 HTTP/1.0" 202 Accepted 2025-12-17 16:17:32,577 - lava_callback - INFO - Starting async job submit for node 69403490575bc060ce157310 2025-12-17 16:17:32,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:32,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403490575bc060ce157310 HTTP/1.1" 200 None 2025-12-17 16:17:32,589 - lava_callback - DEBUG - Processing node 69403490575bc060ce157310: baseline-arm-clabbe 2025-12-17 16:17:32,589 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403490575bc060ce157310 2025-12-17 16:17:32,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:32,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403490575bc060ce157310 HTTP/1.1" 200 None 2025-12-17 16:17:32,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:32,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:17:32,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:32,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403490575bc060ce157310 HTTP/1.1" 200 None 2025-12-17 16:17:32,629 - lava_callback - INFO - Completed processing callback for node 69403490575bc060ce157310 2025-12-17 16:17:33,246 - lava_callback - INFO - Received callback for node 69403492575bc060ce157313 INFO: 172.21.0.1:32888 - "POST /node/69403492575bc060ce157313 HTTP/1.0" 202 Accepted 2025-12-17 16:17:33,254 - lava_callback - INFO - Starting async job submit for node 69403492575bc060ce157313 2025-12-17 16:17:33,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:33,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403492575bc060ce157313 HTTP/1.1" 200 None 2025-12-17 16:17:33,263 - lava_callback - DEBUG - Processing node 69403492575bc060ce157313: baseline-arm-clabbe 2025-12-17 16:17:33,263 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403492575bc060ce157313 2025-12-17 16:17:33,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:33,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403492575bc060ce157313 HTTP/1.1" 200 None 2025-12-17 16:17:33,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:33,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:17:33,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:33,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403492575bc060ce157313 HTTP/1.1" 200 None 2025-12-17 16:17:33,299 - lava_callback - INFO - Completed processing callback for node 69403492575bc060ce157313 2025-12-17 16:17:33,910 - lava_callback - INFO - Received callback for node 69403493575bc060ce157316 INFO: 172.21.0.1:32892 - "POST /node/69403493575bc060ce157316 HTTP/1.0" 202 Accepted 2025-12-17 16:17:33,916 - lava_callback - INFO - Starting async job submit for node 69403493575bc060ce157316 2025-12-17 16:17:33,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:33,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403493575bc060ce157316 HTTP/1.1" 200 None 2025-12-17 16:17:33,926 - lava_callback - DEBUG - Processing node 69403493575bc060ce157316: baseline-arm-clabbe 2025-12-17 16:17:33,926 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403493575bc060ce157316 2025-12-17 16:17:33,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:33,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403493575bc060ce157316 HTTP/1.1" 200 None 2025-12-17 16:17:33,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:33,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:17:33,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:33,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403493575bc060ce157316 HTTP/1.1" 200 None 2025-12-17 16:17:33,966 - lava_callback - INFO - Completed processing callback for node 69403493575bc060ce157316 2025-12-17 16:17:34,565 - lava_callback - INFO - Received callback for node 69403494575bc060ce15731d INFO: 172.21.0.1:32904 - "POST /node/69403494575bc060ce15731d HTTP/1.0" 202 Accepted 2025-12-17 16:17:34,578 - lava_callback - INFO - Starting async job submit for node 69403494575bc060ce15731d 2025-12-17 16:17:34,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:34,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403494575bc060ce15731d HTTP/1.1" 200 None 2025-12-17 16:17:34,602 - lava_callback - DEBUG - Processing node 69403494575bc060ce15731d: baseline-arm-clabbe 2025-12-17 16:17:34,602 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403494575bc060ce15731d 2025-12-17 16:17:34,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:34,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403494575bc060ce15731d HTTP/1.1" 200 None 2025-12-17 16:17:34,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:34,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:17:34,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:34,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403494575bc060ce15731d HTTP/1.1" 200 None 2025-12-17 16:17:34,639 - lava_callback - INFO - Completed processing callback for node 69403494575bc060ce15731d 2025-12-17 16:17:34,858 - lava_callback - INFO - Received callback for node 69403499575bc060ce15732b INFO: 172.21.0.1:32912 - "POST /node/69403499575bc060ce15732b HTTP/1.0" 202 Accepted 2025-12-17 16:17:34,865 - lava_callback - INFO - Starting async job submit for node 69403499575bc060ce15732b 2025-12-17 16:17:34,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:34,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403499575bc060ce15732b HTTP/1.1" 200 None 2025-12-17 16:17:34,874 - lava_callback - DEBUG - Processing node 69403499575bc060ce15732b: baseline-arm 2025-12-17 16:17:34,874 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403499575bc060ce15732b 2025-12-17 16:17:34,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:34,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403499575bc060ce15732b HTTP/1.1" 200 None 2025-12-17 16:17:34,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:34,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:17:34,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:34,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403499575bc060ce15732b HTTP/1.1" 200 None 2025-12-17 16:17:34,909 - lava_callback - INFO - Completed processing callback for node 69403499575bc060ce15732b 2025-12-17 16:17:35,243 - lava_callback - INFO - Received callback for node 69403496575bc060ce157322 INFO: 172.21.0.1:32922 - "POST /node/69403496575bc060ce157322 HTTP/1.0" 202 Accepted 2025-12-17 16:17:35,250 - lava_callback - INFO - Starting async job submit for node 69403496575bc060ce157322 2025-12-17 16:17:35,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:35,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403496575bc060ce157322 HTTP/1.1" 200 None 2025-12-17 16:17:35,260 - lava_callback - DEBUG - Processing node 69403496575bc060ce157322: baseline-arm-clabbe 2025-12-17 16:17:35,260 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403496575bc060ce157322 2025-12-17 16:17:35,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:35,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403496575bc060ce157322 HTTP/1.1" 200 None 2025-12-17 16:17:35,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:35,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:17:35,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:35,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403496575bc060ce157322 HTTP/1.1" 200 None 2025-12-17 16:17:35,295 - lava_callback - INFO - Completed processing callback for node 69403496575bc060ce157322 2025-12-17 16:17:35,546 - lava_callback - INFO - Received callback for node 6940349b575bc060ce15732e INFO: 172.21.0.1:32934 - "POST /node/6940349b575bc060ce15732e HTTP/1.0" 202 Accepted 2025-12-17 16:17:35,552 - lava_callback - INFO - Starting async job submit for node 6940349b575bc060ce15732e 2025-12-17 16:17:35,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:35,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940349b575bc060ce15732e HTTP/1.1" 200 None 2025-12-17 16:17:35,566 - lava_callback - DEBUG - Processing node 6940349b575bc060ce15732e: baseline-arm 2025-12-17 16:17:35,566 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940349b575bc060ce15732e 2025-12-17 16:17:35,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:35,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940349b575bc060ce15732e HTTP/1.1" 200 None 2025-12-17 16:17:35,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:35,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:17:35,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:35,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940349b575bc060ce15732e HTTP/1.1" 200 None 2025-12-17 16:17:35,602 - lava_callback - INFO - Completed processing callback for node 6940349b575bc060ce15732e 2025-12-17 16:17:35,912 - lava_callback - INFO - Received callback for node 69403497575bc060ce157325 INFO: 172.21.0.1:32950 - "POST /node/69403497575bc060ce157325 HTTP/1.0" 202 Accepted 2025-12-17 16:17:35,922 - lava_callback - INFO - Starting async job submit for node 69403497575bc060ce157325 2025-12-17 16:17:35,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:35,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403497575bc060ce157325 HTTP/1.1" 200 None 2025-12-17 16:17:35,943 - lava_callback - DEBUG - Processing node 69403497575bc060ce157325: baseline-arm-clabbe 2025-12-17 16:17:35,943 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403497575bc060ce157325 2025-12-17 16:17:35,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:35,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403497575bc060ce157325 HTTP/1.1" 200 None 2025-12-17 16:17:35,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:35,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:17:35,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:36,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403497575bc060ce157325 HTTP/1.1" 200 None 2025-12-17 16:17:36,016 - lava_callback - INFO - Completed processing callback for node 69403497575bc060ce157325 2025-12-17 16:17:36,237 - lava_callback - INFO - Received callback for node 6940349c575bc060ce157331 INFO: 172.21.0.1:32952 - "POST /node/6940349c575bc060ce157331 HTTP/1.0" 202 Accepted 2025-12-17 16:17:36,245 - lava_callback - INFO - Starting async job submit for node 6940349c575bc060ce157331 2025-12-17 16:17:36,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:36,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940349c575bc060ce157331 HTTP/1.1" 200 None 2025-12-17 16:17:36,259 - lava_callback - DEBUG - Processing node 6940349c575bc060ce157331: baseline-arm 2025-12-17 16:17:36,259 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940349c575bc060ce157331 2025-12-17 16:17:36,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:36,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940349c575bc060ce157331 HTTP/1.1" 200 None 2025-12-17 16:17:36,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:36,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:17:36,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:36,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940349c575bc060ce157331 HTTP/1.1" 200 None 2025-12-17 16:17:36,295 - lava_callback - INFO - Completed processing callback for node 6940349c575bc060ce157331 INFO: 172.21.0.1:32960 - "GET /node/6940258f575bc060ce1540f6%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:32968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52386 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:17:49,178 - lava_callback - INFO - Received callback for node 694034a3575bc060ce15733f INFO: 172.21.0.1:52392 - "POST /node/694034a3575bc060ce15733f HTTP/1.0" 202 Accepted 2025-12-17 16:17:49,184 - lava_callback - INFO - Starting async job submit for node 694034a3575bc060ce15733f 2025-12-17 16:17:49,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:49,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034a3575bc060ce15733f HTTP/1.1" 200 None 2025-12-17 16:17:49,193 - lava_callback - DEBUG - Processing node 694034a3575bc060ce15733f: baseline-arm64-clabbe 2025-12-17 16:17:49,194 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034a3575bc060ce15733f 2025-12-17 16:17:49,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:49,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034a3575bc060ce15733f HTTP/1.1" 200 None 2025-12-17 16:17:49,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:49,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:17:49,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:49,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034a3575bc060ce15733f HTTP/1.1" 200 None 2025-12-17 16:17:49,236 - lava_callback - INFO - Completed processing callback for node 694034a3575bc060ce15733f 2025-12-17 16:17:49,858 - lava_callback - INFO - Received callback for node 694034a7575bc060ce157347 INFO: 172.21.0.1:40060 - "POST /node/694034a7575bc060ce157347 HTTP/1.0" 202 Accepted 2025-12-17 16:17:49,867 - lava_callback - INFO - Starting async job submit for node 694034a7575bc060ce157347 2025-12-17 16:17:49,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:49,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034a7575bc060ce157347 HTTP/1.1" 200 None 2025-12-17 16:17:49,875 - lava_callback - DEBUG - Processing node 694034a7575bc060ce157347: baseline-arm64-clabbe 2025-12-17 16:17:49,875 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034a7575bc060ce157347 2025-12-17 16:17:49,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:49,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034a7575bc060ce157347 HTTP/1.1" 200 None 2025-12-17 16:17:49,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:49,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:17:49,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:49,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034a7575bc060ce157347 HTTP/1.1" 200 None 2025-12-17 16:17:49,911 - lava_callback - INFO - Completed processing callback for node 694034a7575bc060ce157347 2025-12-17 16:17:50,541 - lava_callback - INFO - Received callback for node 69403498575bc060ce157328 INFO: 172.21.0.1:40070 - "POST /node/69403498575bc060ce157328 HTTP/1.0" 202 Accepted 2025-12-17 16:17:50,548 - lava_callback - INFO - Starting async job submit for node 69403498575bc060ce157328 2025-12-17 16:17:50,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:50,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403498575bc060ce157328 HTTP/1.1" 200 None 2025-12-17 16:17:50,558 - lava_callback - DEBUG - Processing node 69403498575bc060ce157328: baseline-arm-clabbe 2025-12-17 16:17:50,558 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403498575bc060ce157328 2025-12-17 16:17:50,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:50,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403498575bc060ce157328 HTTP/1.1" 200 None 2025-12-17 16:17:50,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:50,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:17:50,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:50,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403498575bc060ce157328 HTTP/1.1" 200 None 2025-12-17 16:17:50,596 - lava_callback - INFO - Completed processing callback for node 69403498575bc060ce157328 2025-12-17 16:17:51,213 - lava_callback - INFO - Received callback for node 6940349f575bc060ce157339 INFO: 172.21.0.1:40082 - "POST /node/6940349f575bc060ce157339 HTTP/1.0" 202 Accepted 2025-12-17 16:17:51,219 - lava_callback - INFO - Starting async job submit for node 6940349f575bc060ce157339 2025-12-17 16:17:51,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:51,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940349f575bc060ce157339 HTTP/1.1" 200 None 2025-12-17 16:17:51,229 - lava_callback - DEBUG - Processing node 6940349f575bc060ce157339: baseline-arm64-clabbe 2025-12-17 16:17:51,229 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940349f575bc060ce157339 2025-12-17 16:17:51,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:51,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940349f575bc060ce157339 HTTP/1.1" 200 None 2025-12-17 16:17:51,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:51,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:17:51,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:51,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940349f575bc060ce157339 HTTP/1.1" 200 None 2025-12-17 16:17:51,270 - lava_callback - INFO - Completed processing callback for node 6940349f575bc060ce157339 2025-12-17 16:17:54,537 - lava_callback - INFO - Received callback for node 6940349d575bc060ce157336 INFO: 172.21.0.1:40092 - "POST /node/6940349d575bc060ce157336 HTTP/1.0" 202 Accepted 2025-12-17 16:17:54,545 - lava_callback - INFO - Starting async job submit for node 6940349d575bc060ce157336 2025-12-17 16:17:54,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:54,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940349d575bc060ce157336 HTTP/1.1" 200 None 2025-12-17 16:17:54,554 - lava_callback - DEBUG - Processing node 6940349d575bc060ce157336: baseline-arm-baylibre 2025-12-17 16:17:54,554 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940349d575bc060ce157336 2025-12-17 16:17:54,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:54,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940349d575bc060ce157336 HTTP/1.1" 200 None 2025-12-17 16:17:54,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:54,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:17:54,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:54,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940349d575bc060ce157336 HTTP/1.1" 200 None 2025-12-17 16:17:54,600 - lava_callback - INFO - Completed processing callback for node 6940349d575bc060ce157336 2025-12-17 16:17:55,223 - lava_callback - INFO - Received callback for node 694034a1575bc060ce15733c INFO: 172.21.0.1:40106 - "POST /node/694034a1575bc060ce15733c HTTP/1.0" 202 Accepted 2025-12-17 16:17:55,231 - lava_callback - INFO - Starting async job submit for node 694034a1575bc060ce15733c 2025-12-17 16:17:55,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:55,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034a1575bc060ce15733c HTTP/1.1" 200 None 2025-12-17 16:17:55,240 - lava_callback - DEBUG - Processing node 694034a1575bc060ce15733c: baseline-arm64-baylibre 2025-12-17 16:17:55,240 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034a1575bc060ce15733c 2025-12-17 16:17:55,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:55,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034a1575bc060ce15733c HTTP/1.1" 200 None 2025-12-17 16:17:55,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:55,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:17:55,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:55,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034a1575bc060ce15733c HTTP/1.1" 200 None 2025-12-17 16:17:55,283 - lava_callback - INFO - Completed processing callback for node 694034a1575bc060ce15733c 2025-12-17 16:17:55,865 - lava_callback - INFO - Received callback for node 694034a5575bc060ce157342 INFO: 172.21.0.1:40112 - "POST /node/694034a5575bc060ce157342 HTTP/1.0" 202 Accepted 2025-12-17 16:17:55,871 - lava_callback - INFO - Starting async job submit for node 694034a5575bc060ce157342 2025-12-17 16:17:55,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:55,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034a5575bc060ce157342 HTTP/1.1" 200 None 2025-12-17 16:17:55,881 - lava_callback - DEBUG - Processing node 694034a5575bc060ce157342: baseline-arm64-baylibre 2025-12-17 16:17:55,881 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034a5575bc060ce157342 2025-12-17 16:17:55,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:55,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034a5575bc060ce157342 HTTP/1.1" 200 None 2025-12-17 16:17:55,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:55,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:17:55,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:55,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034a5575bc060ce157342 HTTP/1.1" 200 None 2025-12-17 16:17:55,968 - lava_callback - INFO - Completed processing callback for node 694034a5575bc060ce157342 INFO: 172.21.0.1:40122 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:17:56,546 - lava_callback - INFO - Received callback for node 694034a9575bc060ce157350 INFO: 172.21.0.1:40132 - "POST /node/694034a9575bc060ce157350 HTTP/1.0" 202 Accepted 2025-12-17 16:17:56,553 - lava_callback - INFO - Starting async job submit for node 694034a9575bc060ce157350 2025-12-17 16:17:56,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:56,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034a9575bc060ce157350 HTTP/1.1" 200 None 2025-12-17 16:17:56,563 - lava_callback - DEBUG - Processing node 694034a9575bc060ce157350: baseline-arm64-baylibre 2025-12-17 16:17:56,563 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034a9575bc060ce157350 2025-12-17 16:17:56,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:56,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034a9575bc060ce157350 HTTP/1.1" 200 None 2025-12-17 16:17:56,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:56,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:17:56,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:56,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034a9575bc060ce157350 HTTP/1.1" 200 None 2025-12-17 16:17:56,611 - lava_callback - INFO - Completed processing callback for node 694034a9575bc060ce157350 2025-12-17 16:17:57,233 - lava_callback - INFO - Received callback for node 694034af575bc060ce157392 INFO: 172.21.0.1:40144 - "POST /node/694034af575bc060ce157392 HTTP/1.0" 202 Accepted 2025-12-17 16:17:57,239 - lava_callback - INFO - Starting async job submit for node 694034af575bc060ce157392 2025-12-17 16:17:57,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:57,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034af575bc060ce157392 HTTP/1.1" 200 None 2025-12-17 16:17:57,252 - lava_callback - DEBUG - Processing node 694034af575bc060ce157392: baseline-arm64-baylibre 2025-12-17 16:17:57,253 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034af575bc060ce157392 2025-12-17 16:17:57,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:57,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034af575bc060ce157392 HTTP/1.1" 200 None 2025-12-17 16:17:57,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:57,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:17:57,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:17:57,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034af575bc060ce157392 HTTP/1.1" 200 None 2025-12-17 16:17:57,337 - lava_callback - INFO - Completed processing callback for node 694034af575bc060ce157392 2025-12-17 16:18:06,396 - lava_callback - INFO - Received callback for node 6940c4f41e3c24e95681d5e8 INFO: 172.21.0.1:57912 - "POST /node/6940c4f41e3c24e95681d5e8 HTTP/1.0" 202 Accepted 2025-12-17 16:18:06,404 - lava_callback - INFO - Starting async job submit for node 6940c4f41e3c24e95681d5e8 2025-12-17 16:18:06,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:06,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c4f41e3c24e95681d5e8 HTTP/1.1" 200 None 2025-12-17 16:18:06,476 - lava_callback - DEBUG - Processing node 6940c4f41e3c24e95681d5e8: baseline-x86 INFO: 172.21.0.1:57926 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:18:07,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:18:07,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:18:07,584 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-6940c4f41e3c24e95681d5e8/log.txt.gz 2025-12-17 16:18:07,584 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:18:07,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:18:07,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:18:07,825 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-6940c4f41e3c24e95681d5e8/lava_callback.json.gz DEBUG: Setup failed for 6940c4f41e3c24e95681d5e8. Transitting job node result: incomplete -> fail 2025-12-17 16:18:07,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:07,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c4f41e3c24e95681d5e8 HTTP/1.1" 200 None 2025-12-17 16:18:07,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:07,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21a207de2c0cbc264e9a HTTP/1.1" 200 None 2025-12-17 16:18:07,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:07,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940c4f41e3c24e95681d5e8 HTTP/1.1" 200 None 2025-12-17 16:18:07,987 - lava_callback - INFO - Completed processing callback for node 6940c4f41e3c24e95681d5e8 2025-12-17 16:18:09,173 - lava_callback - INFO - Received callback for node 694034ad575bc060ce15738f INFO: 172.21.0.1:57938 - "POST /node/694034ad575bc060ce15738f HTTP/1.0" 202 Accepted 2025-12-17 16:18:09,179 - lava_callback - INFO - Starting async job submit for node 694034ad575bc060ce15738f 2025-12-17 16:18:09,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:09,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ad575bc060ce15738f HTTP/1.1" 200 None 2025-12-17 16:18:09,188 - lava_callback - DEBUG - Processing node 694034ad575bc060ce15738f: baseline-arm64-clabbe 2025-12-17 16:18:09,188 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034ad575bc060ce15738f 2025-12-17 16:18:09,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:09,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ad575bc060ce15738f HTTP/1.1" 200 None 2025-12-17 16:18:09,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:09,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:18:09,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:09,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034ad575bc060ce15738f HTTP/1.1" 200 None 2025-12-17 16:18:09,253 - lava_callback - INFO - Completed processing callback for node 694034ad575bc060ce15738f 2025-12-17 16:18:09,840 - lava_callback - INFO - Received callback for node 694034b2575bc060ce157395 INFO: 172.21.0.1:46658 - "POST /node/694034b2575bc060ce157395 HTTP/1.0" 202 Accepted 2025-12-17 16:18:09,850 - lava_callback - INFO - Starting async job submit for node 694034b2575bc060ce157395 2025-12-17 16:18:09,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:09,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034b2575bc060ce157395 HTTP/1.1" 200 None 2025-12-17 16:18:09,862 - lava_callback - DEBUG - Processing node 694034b2575bc060ce157395: baseline-arm64-clabbe 2025-12-17 16:18:09,862 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034b2575bc060ce157395 2025-12-17 16:18:09,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:09,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034b2575bc060ce157395 HTTP/1.1" 200 None 2025-12-17 16:18:09,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:09,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:18:09,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:09,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034b2575bc060ce157395 HTTP/1.1" 200 None 2025-12-17 16:18:09,909 - lava_callback - INFO - Completed processing callback for node 694034b2575bc060ce157395 2025-12-17 16:18:10,515 - lava_callback - INFO - Received callback for node 694034be575bc060ce1573b6 INFO: 172.21.0.1:46660 - "POST /node/694034be575bc060ce1573b6 HTTP/1.0" 202 Accepted 2025-12-17 16:18:10,524 - lava_callback - INFO - Starting async job submit for node 694034be575bc060ce1573b6 2025-12-17 16:18:10,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:10,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034be575bc060ce1573b6 HTTP/1.1" 200 None 2025-12-17 16:18:10,535 - lava_callback - DEBUG - Processing node 694034be575bc060ce1573b6: baseline-arm64-clabbe 2025-12-17 16:18:10,535 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034be575bc060ce1573b6 2025-12-17 16:18:10,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:10,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034be575bc060ce1573b6 HTTP/1.1" 200 None 2025-12-17 16:18:10,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:10,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:18:10,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:10,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034be575bc060ce1573b6 HTTP/1.1" 200 None 2025-12-17 16:18:10,584 - lava_callback - INFO - Completed processing callback for node 694034be575bc060ce1573b6 INFO: 172.21.0.1:46664 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:18:14,567 - lava_callback - INFO - Received callback for node 694034b4575bc060ce157398 INFO: 172.21.0.1:46678 - "POST /node/694034b4575bc060ce157398 HTTP/1.0" 202 Accepted 2025-12-17 16:18:14,576 - lava_callback - INFO - Starting async job submit for node 694034b4575bc060ce157398 2025-12-17 16:18:14,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:14,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034b4575bc060ce157398 HTTP/1.1" 200 None 2025-12-17 16:18:14,593 - lava_callback - DEBUG - Processing node 694034b4575bc060ce157398: baseline-arm64-baylibre 2025-12-17 16:18:14,593 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034b4575bc060ce157398 2025-12-17 16:18:14,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:14,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034b4575bc060ce157398 HTTP/1.1" 200 None 2025-12-17 16:18:14,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:14,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:18:14,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:14,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034b4575bc060ce157398 HTTP/1.1" 200 None 2025-12-17 16:18:14,636 - lava_callback - INFO - Completed processing callback for node 694034b4575bc060ce157398 INFO: 172.21.0.1:46684 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:18:15,224 - lava_callback - INFO - Received callback for node 694034b6575bc060ce15739b INFO: 172.21.0.1:46686 - "POST /node/694034b6575bc060ce15739b HTTP/1.0" 202 Accepted 2025-12-17 16:18:15,234 - lava_callback - INFO - Starting async job submit for node 694034b6575bc060ce15739b 2025-12-17 16:18:15,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:15,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034b6575bc060ce15739b HTTP/1.1" 200 None 2025-12-17 16:18:15,247 - lava_callback - DEBUG - Processing node 694034b6575bc060ce15739b: baseline-arm-baylibre 2025-12-17 16:18:15,248 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034b6575bc060ce15739b 2025-12-17 16:18:15,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:15,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034b6575bc060ce15739b HTTP/1.1" 200 None 2025-12-17 16:18:15,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:15,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:18:15,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:15,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034b6575bc060ce15739b HTTP/1.1" 200 None 2025-12-17 16:18:15,297 - lava_callback - INFO - Completed processing callback for node 694034b6575bc060ce15739b 2025-12-17 16:18:15,907 - lava_callback - INFO - Received callback for node 694034b7575bc060ce15739e INFO: 172.21.0.1:46690 - "POST /node/694034b7575bc060ce15739e HTTP/1.0" 202 Accepted 2025-12-17 16:18:15,917 - lava_callback - INFO - Starting async job submit for node 694034b7575bc060ce15739e 2025-12-17 16:18:15,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:15,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034b7575bc060ce15739e HTTP/1.1" 200 None 2025-12-17 16:18:15,935 - lava_callback - DEBUG - Processing node 694034b7575bc060ce15739e: baseline-arm64-baylibre 2025-12-17 16:18:15,935 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034b7575bc060ce15739e 2025-12-17 16:18:15,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:15,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034b7575bc060ce15739e HTTP/1.1" 200 None 2025-12-17 16:18:15,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:15,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:18:15,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:15,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034b7575bc060ce15739e HTTP/1.1" 200 None 2025-12-17 16:18:15,985 - lava_callback - INFO - Completed processing callback for node 694034b7575bc060ce15739e INFO: 172.21.0.1:46700 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:18:16,574 - lava_callback - INFO - Received callback for node 694034b9575bc060ce1573a1 INFO: 172.21.0.1:46702 - "POST /node/694034b9575bc060ce1573a1 HTTP/1.0" 202 Accepted 2025-12-17 16:18:16,584 - lava_callback - INFO - Starting async job submit for node 694034b9575bc060ce1573a1 2025-12-17 16:18:16,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:16,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034b9575bc060ce1573a1 HTTP/1.1" 200 None 2025-12-17 16:18:16,597 - lava_callback - DEBUG - Processing node 694034b9575bc060ce1573a1: baseline-arm64-baylibre 2025-12-17 16:18:16,597 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034b9575bc060ce1573a1 2025-12-17 16:18:16,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:16,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034b9575bc060ce1573a1 HTTP/1.1" 200 None 2025-12-17 16:18:16,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:16,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:18:16,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:16,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034b9575bc060ce1573a1 HTTP/1.1" 200 None 2025-12-17 16:18:16,644 - lava_callback - INFO - Completed processing callback for node 694034b9575bc060ce1573a1 2025-12-17 16:18:17,440 - lava_callback - INFO - Received callback for node 694034c4575bc060ce1573c1 INFO: 172.21.0.1:46712 - "POST /node/694034c4575bc060ce1573c1 HTTP/1.0" 202 Accepted 2025-12-17 16:18:17,449 - lava_callback - INFO - Starting async job submit for node 694034c4575bc060ce1573c1 2025-12-17 16:18:17,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:17,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034c4575bc060ce1573c1 HTTP/1.1" 200 None 2025-12-17 16:18:17,460 - lava_callback - DEBUG - Processing node 694034c4575bc060ce1573c1: baseline-arm-baylibre 2025-12-17 16:18:17,460 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034c4575bc060ce1573c1 2025-12-17 16:18:17,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:17,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034c4575bc060ce1573c1 HTTP/1.1" 200 None 2025-12-17 16:18:17,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:17,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:18:17,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:17,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034c4575bc060ce1573c1 HTTP/1.1" 200 None 2025-12-17 16:18:17,522 - lava_callback - INFO - Completed processing callback for node 694034c4575bc060ce1573c1 2025-12-17 16:18:26,275 - lava_callback - INFO - Received callback for node 694034bd575bc060ce1573b0 INFO: 172.21.0.1:37980 - "POST /node/694034bd575bc060ce1573b0 HTTP/1.0" 202 Accepted 2025-12-17 16:18:26,281 - lava_callback - INFO - Starting async job submit for node 694034bd575bc060ce1573b0 2025-12-17 16:18:26,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:26,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034bd575bc060ce1573b0 HTTP/1.1" 200 None 2025-12-17 16:18:26,291 - lava_callback - DEBUG - Processing node 694034bd575bc060ce1573b0: ltp-smoketest 2025-12-17 16:18:26,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034bd575bc060ce1573b0 2025-12-17 16:18:26,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:26,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034bd575bc060ce1573b0 HTTP/1.1" 200 None 2025-12-17 16:18:26,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:26,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:18:26,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:26,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034bd575bc060ce1573b0 HTTP/1.1" 200 None 2025-12-17 16:18:26,350 - lava_callback - INFO - Completed processing callback for node 694034bd575bc060ce1573b0 INFO: 172.21.0.1:37992 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:18:27,074 - lava_callback - INFO - Received callback for node 694034bd575bc060ce1573b3 INFO: 172.21.0.1:37994 - "POST /node/694034bd575bc060ce1573b3 HTTP/1.0" 202 Accepted 2025-12-17 16:18:27,081 - lava_callback - INFO - Starting async job submit for node 694034bd575bc060ce1573b3 2025-12-17 16:18:27,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:27,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034bd575bc060ce1573b3 HTTP/1.1" 200 None 2025-12-17 16:18:27,091 - lava_callback - DEBUG - Processing node 694034bd575bc060ce1573b3: ltp-smoketest 2025-12-17 16:18:27,091 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034bd575bc060ce1573b3 2025-12-17 16:18:27,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:27,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034bd575bc060ce1573b3 HTTP/1.1" 200 None 2025-12-17 16:18:27,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:27,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:18:27,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:27,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034bd575bc060ce1573b3 HTTP/1.1" 200 None 2025-12-17 16:18:27,126 - lava_callback - INFO - Completed processing callback for node 694034bd575bc060ce1573b3 2025-12-17 16:18:28,008 - lava_callback - INFO - Received callback for node 694034cb575bc060ce1573cd INFO: 172.21.0.1:38006 - "POST /node/694034cb575bc060ce1573cd HTTP/1.0" 202 Accepted 2025-12-17 16:18:28,014 - lava_callback - INFO - Starting async job submit for node 694034cb575bc060ce1573cd 2025-12-17 16:18:28,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:28,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034cb575bc060ce1573cd HTTP/1.1" 200 None 2025-12-17 16:18:28,024 - lava_callback - DEBUG - Processing node 694034cb575bc060ce1573cd: baseline-arm 2025-12-17 16:18:28,024 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034cb575bc060ce1573cd 2025-12-17 16:18:28,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:28,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034cb575bc060ce1573cd HTTP/1.1" 200 None 2025-12-17 16:18:28,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:28,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:18:28,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:28,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034cb575bc060ce1573cd HTTP/1.1" 200 None 2025-12-17 16:18:28,058 - lava_callback - INFO - Completed processing callback for node 694034cb575bc060ce1573cd 2025-12-17 16:18:28,851 - lava_callback - INFO - Received callback for node 694034cc575bc060ce1573d0 INFO: 172.21.0.1:38014 - "POST /node/694034cc575bc060ce1573d0 HTTP/1.0" 202 Accepted 2025-12-17 16:18:28,857 - lava_callback - INFO - Starting async job submit for node 694034cc575bc060ce1573d0 2025-12-17 16:18:28,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:28,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034cc575bc060ce1573d0 HTTP/1.1" 200 None 2025-12-17 16:18:28,868 - lava_callback - DEBUG - Processing node 694034cc575bc060ce1573d0: baseline-arm 2025-12-17 16:18:28,868 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034cc575bc060ce1573d0 2025-12-17 16:18:28,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:28,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034cc575bc060ce1573d0 HTTP/1.1" 200 None 2025-12-17 16:18:28,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:28,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:18:28,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:28,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034cc575bc060ce1573d0 HTTP/1.1" 200 None 2025-12-17 16:18:28,909 - lava_callback - INFO - Completed processing callback for node 694034cc575bc060ce1573d0 2025-12-17 16:18:29,190 - lava_callback - INFO - Received callback for node 694034c0575bc060ce1573b9 INFO: 172.21.0.1:38018 - "POST /node/694034c0575bc060ce1573b9 HTTP/1.0" 202 Accepted 2025-12-17 16:18:29,196 - lava_callback - INFO - Starting async job submit for node 694034c0575bc060ce1573b9 2025-12-17 16:18:29,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:29,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034c0575bc060ce1573b9 HTTP/1.1" 200 None 2025-12-17 16:18:29,208 - lava_callback - DEBUG - Processing node 694034c0575bc060ce1573b9: baseline-arm64-clabbe 2025-12-17 16:18:29,208 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034c0575bc060ce1573b9 2025-12-17 16:18:29,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:29,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034c0575bc060ce1573b9 HTTP/1.1" 200 None 2025-12-17 16:18:29,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:29,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:18:29,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:29,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034c0575bc060ce1573b9 HTTP/1.1" 200 None 2025-12-17 16:18:29,290 - lava_callback - INFO - Completed processing callback for node 694034c0575bc060ce1573b9 2025-12-17 16:18:29,645 - lava_callback - INFO - Received callback for node 694034cc575bc060ce1573d3 INFO: 172.21.0.1:38022 - "POST /node/694034cc575bc060ce1573d3 HTTP/1.0" 202 Accepted 2025-12-17 16:18:29,651 - lava_callback - INFO - Starting async job submit for node 694034cc575bc060ce1573d3 2025-12-17 16:18:29,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:29,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034cc575bc060ce1573d3 HTTP/1.1" 200 None 2025-12-17 16:18:29,661 - lava_callback - DEBUG - Processing node 694034cc575bc060ce1573d3: baseline-arm 2025-12-17 16:18:29,661 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034cc575bc060ce1573d3 2025-12-17 16:18:29,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:29,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034cc575bc060ce1573d3 HTTP/1.1" 200 None 2025-12-17 16:18:29,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:29,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:18:29,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:29,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034cc575bc060ce1573d3 HTTP/1.1" 200 None 2025-12-17 16:18:29,703 - lava_callback - INFO - Completed processing callback for node 694034cc575bc060ce1573d3 2025-12-17 16:18:29,852 - lava_callback - INFO - Received callback for node 694034c2575bc060ce1573be INFO: 172.21.0.1:34354 - "POST /node/694034c2575bc060ce1573be HTTP/1.0" 202 Accepted 2025-12-17 16:18:29,862 - lava_callback - INFO - Starting async job submit for node 694034c2575bc060ce1573be 2025-12-17 16:18:29,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:29,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034c2575bc060ce1573be HTTP/1.1" 200 None 2025-12-17 16:18:29,873 - lava_callback - DEBUG - Processing node 694034c2575bc060ce1573be: baseline-arm64-clabbe 2025-12-17 16:18:29,873 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034c2575bc060ce1573be 2025-12-17 16:18:29,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:29,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034c2575bc060ce1573be HTTP/1.1" 200 None 2025-12-17 16:18:29,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:29,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:18:29,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:29,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034c2575bc060ce1573be HTTP/1.1" 200 None 2025-12-17 16:18:29,911 - lava_callback - INFO - Completed processing callback for node 694034c2575bc060ce1573be 2025-12-17 16:18:30,448 - lava_callback - INFO - Received callback for node 694034cc575bc060ce1573d6 INFO: 172.21.0.1:34366 - "POST /node/694034cc575bc060ce1573d6 HTTP/1.0" 202 Accepted 2025-12-17 16:18:30,455 - lava_callback - INFO - Starting async job submit for node 694034cc575bc060ce1573d6 2025-12-17 16:18:30,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:30,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034cc575bc060ce1573d6 HTTP/1.1" 200 None 2025-12-17 16:18:30,466 - lava_callback - DEBUG - Processing node 694034cc575bc060ce1573d6: ltp-smoketest 2025-12-17 16:18:30,466 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034cc575bc060ce1573d6 2025-12-17 16:18:30,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:30,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034cc575bc060ce1573d6 HTTP/1.1" 200 None 2025-12-17 16:18:30,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:30,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:18:30,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:30,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034cc575bc060ce1573d6 HTTP/1.1" 200 None 2025-12-17 16:18:30,507 - lava_callback - INFO - Completed processing callback for node 694034cc575bc060ce1573d6 2025-12-17 16:18:30,514 - lava_callback - INFO - Received callback for node 694034c6575bc060ce1573c4 INFO: 172.21.0.1:34378 - "POST /node/694034c6575bc060ce1573c4 HTTP/1.0" 202 Accepted 2025-12-17 16:18:30,524 - lava_callback - INFO - Starting async job submit for node 694034c6575bc060ce1573c4 2025-12-17 16:18:30,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:30,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034c6575bc060ce1573c4 HTTP/1.1" 200 None 2025-12-17 16:18:30,547 - lava_callback - DEBUG - Processing node 694034c6575bc060ce1573c4: baseline-arm64-clabbe 2025-12-17 16:18:30,547 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034c6575bc060ce1573c4 2025-12-17 16:18:30,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:30,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034c6575bc060ce1573c4 HTTP/1.1" 200 None 2025-12-17 16:18:30,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:30,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:18:30,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:30,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034c6575bc060ce1573c4 HTTP/1.1" 200 None 2025-12-17 16:18:30,589 - lava_callback - INFO - Completed processing callback for node 694034c6575bc060ce1573c4 2025-12-17 16:18:31,174 - lava_callback - INFO - Received callback for node 694034c8575bc060ce1573c7 INFO: 172.21.0.1:34380 - "POST /node/694034c8575bc060ce1573c7 HTTP/1.0" 202 Accepted 2025-12-17 16:18:31,180 - lava_callback - INFO - Starting async job submit for node 694034c8575bc060ce1573c7 2025-12-17 16:18:31,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:31,182 - lava_callback - INFO - Received callback for node 694034cd575bc060ce1573d9 INFO: 172.21.0.1:34386 - "POST /node/694034cd575bc060ce1573d9 HTTP/1.0" 202 Accepted 2025-12-17 16:18:31,188 - lava_callback - INFO - Starting async job submit for node 694034cd575bc060ce1573d9 2025-12-17 16:18:31,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:31,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034c8575bc060ce1573c7 HTTP/1.1" 200 None 2025-12-17 16:18:31,196 - lava_callback - DEBUG - Processing node 694034c8575bc060ce1573c7: baseline-arm64-clabbe 2025-12-17 16:18:31,196 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034c8575bc060ce1573c7 2025-12-17 16:18:31,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:31,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034cd575bc060ce1573d9 HTTP/1.1" 200 None 2025-12-17 16:18:31,209 - lava_callback - DEBUG - Processing node 694034cd575bc060ce1573d9: ltp-smoketest 2025-12-17 16:18:31,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034cd575bc060ce1573d9 2025-12-17 16:18:31,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:31,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034cd575bc060ce1573d9 HTTP/1.1" 200 None 2025-12-17 16:18:31,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:31,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034c8575bc060ce1573c7 HTTP/1.1" 200 None 2025-12-17 16:18:31,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:31,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:18:31,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:31,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:18:31,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:31,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034cd575bc060ce1573d9 HTTP/1.1" 200 None 2025-12-17 16:18:31,255 - lava_callback - INFO - Completed processing callback for node 694034cd575bc060ce1573d9 2025-12-17 16:18:31,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034c8575bc060ce1573c7 HTTP/1.1" 200 None 2025-12-17 16:18:31,289 - lava_callback - INFO - Completed processing callback for node 694034c8575bc060ce1573c7 2025-12-17 16:18:31,865 - lava_callback - INFO - Received callback for node 694034ca575bc060ce1573ca INFO: 172.21.0.1:34396 - "POST /node/694034ca575bc060ce1573ca HTTP/1.0" 202 Accepted 2025-12-17 16:18:31,872 - lava_callback - INFO - Starting async job submit for node 694034ca575bc060ce1573ca 2025-12-17 16:18:31,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:31,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ca575bc060ce1573ca HTTP/1.1" 200 None 2025-12-17 16:18:31,885 - lava_callback - DEBUG - Processing node 694034ca575bc060ce1573ca: baseline-arm64-clabbe 2025-12-17 16:18:31,885 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034ca575bc060ce1573ca 2025-12-17 16:18:31,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:31,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ca575bc060ce1573ca HTTP/1.1" 200 None 2025-12-17 16:18:31,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:31,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d221a07de2c0cbc264f7a HTTP/1.1" 200 None 2025-12-17 16:18:31,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:31,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034ca575bc060ce1573ca HTTP/1.1" 200 None 2025-12-17 16:18:31,925 - lava_callback - INFO - Completed processing callback for node 694034ca575bc060ce1573ca 2025-12-17 16:18:31,948 - lava_callback - INFO - Received callback for node 694034cd575bc060ce1573dc INFO: 172.21.0.1:34410 - "POST /node/694034cd575bc060ce1573dc HTTP/1.0" 202 Accepted 2025-12-17 16:18:31,959 - lava_callback - INFO - Starting async job submit for node 694034cd575bc060ce1573dc 2025-12-17 16:18:31,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:31,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034cd575bc060ce1573dc HTTP/1.1" 200 None 2025-12-17 16:18:31,970 - lava_callback - DEBUG - Processing node 694034cd575bc060ce1573dc: ltp-smoketest 2025-12-17 16:18:31,970 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034cd575bc060ce1573dc 2025-12-17 16:18:31,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:31,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034cd575bc060ce1573dc HTTP/1.1" 200 None 2025-12-17 16:18:31,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:32,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:18:32,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:32,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034cd575bc060ce1573dc HTTP/1.1" 200 None 2025-12-17 16:18:32,024 - lava_callback - INFO - Completed processing callback for node 694034cd575bc060ce1573dc 2025-12-17 16:18:32,747 - lava_callback - INFO - Received callback for node 694034ce575bc060ce1573df INFO: 172.21.0.1:34422 - "POST /node/694034ce575bc060ce1573df HTTP/1.0" 202 Accepted 2025-12-17 16:18:32,753 - lava_callback - INFO - Starting async job submit for node 694034ce575bc060ce1573df 2025-12-17 16:18:32,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:32,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ce575bc060ce1573df HTTP/1.1" 200 None 2025-12-17 16:18:32,763 - lava_callback - DEBUG - Processing node 694034ce575bc060ce1573df: baseline-arm 2025-12-17 16:18:32,763 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034ce575bc060ce1573df 2025-12-17 16:18:32,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:32,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ce575bc060ce1573df HTTP/1.1" 200 None 2025-12-17 16:18:32,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:32,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:18:32,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:32,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034ce575bc060ce1573df HTTP/1.1" 200 None 2025-12-17 16:18:32,804 - lava_callback - INFO - Completed processing callback for node 694034ce575bc060ce1573df 2025-12-17 16:18:33,501 - lava_callback - INFO - Received callback for node 694034ce575bc060ce1573e2 INFO: 172.21.0.1:34426 - "POST /node/694034ce575bc060ce1573e2 HTTP/1.0" 202 Accepted 2025-12-17 16:18:33,507 - lava_callback - INFO - Starting async job submit for node 694034ce575bc060ce1573e2 2025-12-17 16:18:33,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:33,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ce575bc060ce1573e2 HTTP/1.1" 200 None 2025-12-17 16:18:33,766 - lava_callback - DEBUG - Processing node 694034ce575bc060ce1573e2: baseline-arm 2025-12-17 16:18:33,766 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034ce575bc060ce1573e2 2025-12-17 16:18:33,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:33,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ce575bc060ce1573e2 HTTP/1.1" 200 None 2025-12-17 16:18:33,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:33,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:18:33,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:33,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034ce575bc060ce1573e2 HTTP/1.1" 200 None 2025-12-17 16:18:33,807 - lava_callback - INFO - Completed processing callback for node 694034ce575bc060ce1573e2 2025-12-17 16:18:34,231 - lava_callback - INFO - Received callback for node 694034d0575bc060ce1573ea INFO: 172.21.0.1:34430 - "POST /node/694034d0575bc060ce1573ea HTTP/1.0" 202 Accepted 2025-12-17 16:18:34,237 - lava_callback - INFO - Starting async job submit for node 694034d0575bc060ce1573ea 2025-12-17 16:18:34,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:34,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d0575bc060ce1573ea HTTP/1.1" 200 None 2025-12-17 16:18:34,247 - lava_callback - DEBUG - Processing node 694034d0575bc060ce1573ea: baseline-arm 2025-12-17 16:18:34,247 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034d0575bc060ce1573ea 2025-12-17 16:18:34,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:34,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d0575bc060ce1573ea HTTP/1.1" 200 None 2025-12-17 16:18:34,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:34,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:18:34,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:34,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034d0575bc060ce1573ea HTTP/1.1" 200 None 2025-12-17 16:18:34,290 - lava_callback - INFO - Completed processing callback for node 694034d0575bc060ce1573ea 2025-12-17 16:18:34,553 - lava_callback - INFO - Received callback for node 694034cf575bc060ce1573e5 INFO: 172.21.0.1:34444 - "POST /node/694034cf575bc060ce1573e5 HTTP/1.0" 202 Accepted 2025-12-17 16:18:34,560 - lava_callback - INFO - Starting async job submit for node 694034cf575bc060ce1573e5 2025-12-17 16:18:34,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:34,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034cf575bc060ce1573e5 HTTP/1.1" 200 None 2025-12-17 16:18:34,576 - lava_callback - DEBUG - Processing node 694034cf575bc060ce1573e5: baseline-arm-baylibre 2025-12-17 16:18:34,576 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034cf575bc060ce1573e5 2025-12-17 16:18:34,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:34,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034cf575bc060ce1573e5 HTTP/1.1" 200 None 2025-12-17 16:18:34,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:34,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:18:34,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:34,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034cf575bc060ce1573e5 HTTP/1.1" 200 None 2025-12-17 16:18:34,611 - lava_callback - INFO - Completed processing callback for node 694034cf575bc060ce1573e5 2025-12-17 16:18:34,992 - lava_callback - INFO - Received callback for node 694034d1575bc060ce1573ed INFO: 172.21.0.1:34452 - "POST /node/694034d1575bc060ce1573ed HTTP/1.0" 202 Accepted 2025-12-17 16:18:34,998 - lava_callback - INFO - Starting async job submit for node 694034d1575bc060ce1573ed 2025-12-17 16:18:34,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:35,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d1575bc060ce1573ed HTTP/1.1" 200 None 2025-12-17 16:18:35,009 - lava_callback - DEBUG - Processing node 694034d1575bc060ce1573ed: ltp-smoketest 2025-12-17 16:18:35,009 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034d1575bc060ce1573ed 2025-12-17 16:18:35,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:35,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d1575bc060ce1573ed HTTP/1.1" 200 None 2025-12-17 16:18:35,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:35,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:18:35,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:35,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034d1575bc060ce1573ed HTTP/1.1" 200 None 2025-12-17 16:18:35,051 - lava_callback - INFO - Completed processing callback for node 694034d1575bc060ce1573ed 2025-12-17 16:18:35,226 - lava_callback - INFO - Received callback for node 694034d5575bc060ce15740b INFO: 172.21.0.1:34460 - "POST /node/694034d5575bc060ce15740b HTTP/1.0" 202 Accepted 2025-12-17 16:18:35,233 - lava_callback - INFO - Starting async job submit for node 694034d5575bc060ce15740b 2025-12-17 16:18:35,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:35,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d5575bc060ce15740b HTTP/1.1" 200 None 2025-12-17 16:18:35,252 - lava_callback - DEBUG - Processing node 694034d5575bc060ce15740b: baseline-arm-baylibre 2025-12-17 16:18:35,252 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034d5575bc060ce15740b 2025-12-17 16:18:35,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:35,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d5575bc060ce15740b HTTP/1.1" 200 None 2025-12-17 16:18:35,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:35,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:18:35,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:35,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034d5575bc060ce15740b HTTP/1.1" 200 None 2025-12-17 16:18:35,287 - lava_callback - INFO - Completed processing callback for node 694034d5575bc060ce15740b 2025-12-17 16:18:35,764 - lava_callback - INFO - Received callback for node 694034d1575bc060ce1573f1 INFO: 172.21.0.1:34468 - "POST /node/694034d1575bc060ce1573f1 HTTP/1.0" 202 Accepted 2025-12-17 16:18:35,770 - lava_callback - INFO - Starting async job submit for node 694034d1575bc060ce1573f1 2025-12-17 16:18:35,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:35,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d1575bc060ce1573f1 HTTP/1.1" 200 None 2025-12-17 16:18:35,781 - lava_callback - DEBUG - Processing node 694034d1575bc060ce1573f1: ltp-smoketest 2025-12-17 16:18:35,781 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034d1575bc060ce1573f1 2025-12-17 16:18:35,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:35,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d1575bc060ce1573f1 HTTP/1.1" 200 None 2025-12-17 16:18:35,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:35,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:18:35,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:35,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034d1575bc060ce1573f1 HTTP/1.1" 200 None 2025-12-17 16:18:35,818 - lava_callback - INFO - Completed processing callback for node 694034d1575bc060ce1573f1 2025-12-17 16:18:35,904 - lava_callback - INFO - Received callback for node 694034d6575bc060ce157412 INFO: 172.21.0.1:34470 - "POST /node/694034d6575bc060ce157412 HTTP/1.0" 202 Accepted 2025-12-17 16:18:35,911 - lava_callback - INFO - Starting async job submit for node 694034d6575bc060ce157412 2025-12-17 16:18:35,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:35,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d6575bc060ce157412 HTTP/1.1" 200 None 2025-12-17 16:18:35,926 - lava_callback - DEBUG - Processing node 694034d6575bc060ce157412: baseline-arm-baylibre 2025-12-17 16:18:35,926 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034d6575bc060ce157412 2025-12-17 16:18:35,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:35,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d6575bc060ce157412 HTTP/1.1" 200 None 2025-12-17 16:18:35,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:35,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:18:35,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:35,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034d6575bc060ce157412 HTTP/1.1" 200 None 2025-12-17 16:18:35,959 - lava_callback - INFO - Completed processing callback for node 694034d6575bc060ce157412 INFO: 172.21.0.1:34476 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:18:36,554 - lava_callback - INFO - Received callback for node 694034d2575bc060ce1573f7 INFO: 172.21.0.1:34480 - "POST /node/694034d2575bc060ce1573f7 HTTP/1.0" 202 Accepted 2025-12-17 16:18:36,559 - lava_callback - INFO - Starting async job submit for node 694034d2575bc060ce1573f7 2025-12-17 16:18:36,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:36,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d2575bc060ce1573f7 HTTP/1.1" 200 None 2025-12-17 16:18:36,575 - lava_callback - DEBUG - Processing node 694034d2575bc060ce1573f7: baseline-arm 2025-12-17 16:18:36,575 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034d2575bc060ce1573f7 2025-12-17 16:18:36,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:36,578 - lava_callback - INFO - Received callback for node 694034d8575bc060ce157415 INFO: 172.21.0.1:34492 - "POST /node/694034d8575bc060ce157415 HTTP/1.0" 202 Accepted 2025-12-17 16:18:36,584 - lava_callback - INFO - Starting async job submit for node 694034d8575bc060ce157415 2025-12-17 16:18:36,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:36,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d2575bc060ce1573f7 HTTP/1.1" 200 None 2025-12-17 16:18:36,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:36,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d8575bc060ce157415 HTTP/1.1" 200 None 2025-12-17 16:18:36,593 - lava_callback - DEBUG - Processing node 694034d8575bc060ce157415: baseline-arm-baylibre 2025-12-17 16:18:36,593 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034d8575bc060ce157415 2025-12-17 16:18:36,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:36,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:18:36,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d8575bc060ce157415 HTTP/1.1" 200 None 2025-12-17 16:18:36,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:36,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:36,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:18:36,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:36,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034d2575bc060ce1573f7 HTTP/1.1" 200 None 2025-12-17 16:18:36,631 - lava_callback - INFO - Completed processing callback for node 694034d2575bc060ce1573f7 2025-12-17 16:18:36,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034d8575bc060ce157415 HTTP/1.1" 200 None 2025-12-17 16:18:36,658 - lava_callback - INFO - Completed processing callback for node 694034d8575bc060ce157415 2025-12-17 16:18:37,278 - lava_callback - INFO - Received callback for node 694034d2575bc060ce1573fa INFO: 172.21.0.1:34498 - "POST /node/694034d2575bc060ce1573fa HTTP/1.0" 202 Accepted 2025-12-17 16:18:37,284 - lava_callback - INFO - Starting async job submit for node 694034d2575bc060ce1573fa 2025-12-17 16:18:37,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:37,286 - lava_callback - INFO - Received callback for node 694034d9575bc060ce157418 INFO: 172.21.0.1:34502 - "POST /node/694034d9575bc060ce157418 HTTP/1.0" 202 Accepted 2025-12-17 16:18:37,292 - lava_callback - INFO - Starting async job submit for node 694034d9575bc060ce157418 2025-12-17 16:18:37,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:37,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d2575bc060ce1573fa HTTP/1.1" 200 None 2025-12-17 16:18:37,301 - lava_callback - DEBUG - Processing node 694034d2575bc060ce1573fa: baseline-arm 2025-12-17 16:18:37,301 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034d2575bc060ce1573fa 2025-12-17 16:18:37,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:37,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d9575bc060ce157418 HTTP/1.1" 200 None 2025-12-17 16:18:37,303 - lava_callback - DEBUG - Processing node 694034d9575bc060ce157418: baseline-arm-baylibre 2025-12-17 16:18:37,303 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034d9575bc060ce157418 2025-12-17 16:18:37,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:37,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d2575bc060ce1573fa HTTP/1.1" 200 None 2025-12-17 16:18:37,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:37,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d9575bc060ce157418 HTTP/1.1" 200 None 2025-12-17 16:18:37,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:37,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:18:37,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:37,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:18:37,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:37,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034d2575bc060ce1573fa HTTP/1.1" 200 None 2025-12-17 16:18:37,347 - lava_callback - INFO - Completed processing callback for node 694034d2575bc060ce1573fa 2025-12-17 16:18:37,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034d9575bc060ce157418 HTTP/1.1" 200 None 2025-12-17 16:18:37,357 - lava_callback - INFO - Completed processing callback for node 694034d9575bc060ce157418 2025-12-17 16:18:38,009 - lava_callback - INFO - Received callback for node 694034d3575bc060ce1573fd INFO: 172.21.0.1:34518 - "POST /node/694034d3575bc060ce1573fd HTTP/1.0" 202 Accepted 2025-12-17 16:18:38,015 - lava_callback - INFO - Starting async job submit for node 694034d3575bc060ce1573fd 2025-12-17 16:18:38,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:38,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d3575bc060ce1573fd HTTP/1.1" 200 None 2025-12-17 16:18:38,025 - lava_callback - DEBUG - Processing node 694034d3575bc060ce1573fd: baseline-arm 2025-12-17 16:18:38,025 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034d3575bc060ce1573fd 2025-12-17 16:18:38,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:38,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d3575bc060ce1573fd HTTP/1.1" 200 None 2025-12-17 16:18:38,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:38,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:18:38,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:38,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034d3575bc060ce1573fd HTTP/1.1" 200 None 2025-12-17 16:18:38,074 - lava_callback - INFO - Completed processing callback for node 694034d3575bc060ce1573fd 2025-12-17 16:18:38,750 - lava_callback - INFO - Received callback for node 694034d3575bc060ce157400 INFO: 172.21.0.1:34528 - "POST /node/694034d3575bc060ce157400 HTTP/1.0" 202 Accepted 2025-12-17 16:18:38,756 - lava_callback - INFO - Starting async job submit for node 694034d3575bc060ce157400 2025-12-17 16:18:38,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:38,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d3575bc060ce157400 HTTP/1.1" 200 None 2025-12-17 16:18:38,765 - lava_callback - DEBUG - Processing node 694034d3575bc060ce157400: ltp-smoketest 2025-12-17 16:18:38,765 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034d3575bc060ce157400 2025-12-17 16:18:38,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:38,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d3575bc060ce157400 HTTP/1.1" 200 None 2025-12-17 16:18:38,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:38,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:18:38,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:38,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034d3575bc060ce157400 HTTP/1.1" 200 None 2025-12-17 16:18:38,800 - lava_callback - INFO - Completed processing callback for node 694034d3575bc060ce157400 2025-12-17 16:18:39,510 - lava_callback - INFO - Received callback for node 694034d4575bc060ce157403 INFO: 172.21.0.1:34530 - "POST /node/694034d4575bc060ce157403 HTTP/1.0" 202 Accepted 2025-12-17 16:18:39,516 - lava_callback - INFO - Starting async job submit for node 694034d4575bc060ce157403 2025-12-17 16:18:39,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:39,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d4575bc060ce157403 HTTP/1.1" 200 None 2025-12-17 16:18:39,526 - lava_callback - DEBUG - Processing node 694034d4575bc060ce157403: ltp-smoketest 2025-12-17 16:18:39,526 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034d4575bc060ce157403 2025-12-17 16:18:39,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:39,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d4575bc060ce157403 HTTP/1.1" 200 None 2025-12-17 16:18:39,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:39,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:18:39,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:39,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034d4575bc060ce157403 HTTP/1.1" 200 None 2025-12-17 16:18:39,564 - lava_callback - INFO - Completed processing callback for node 694034d4575bc060ce157403 2025-12-17 16:18:40,227 - lava_callback - INFO - Received callback for node 694034d4575bc060ce157406 INFO: 172.21.0.1:48236 - "POST /node/694034d4575bc060ce157406 HTTP/1.0" 202 Accepted 2025-12-17 16:18:40,233 - lava_callback - INFO - Starting async job submit for node 694034d4575bc060ce157406 2025-12-17 16:18:40,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:40,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d4575bc060ce157406 HTTP/1.1" 200 None 2025-12-17 16:18:40,242 - lava_callback - DEBUG - Processing node 694034d4575bc060ce157406: ltp-smoketest 2025-12-17 16:18:40,243 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034d4575bc060ce157406 2025-12-17 16:18:40,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:40,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d4575bc060ce157406 HTTP/1.1" 200 None 2025-12-17 16:18:40,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:40,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:18:40,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:40,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034d4575bc060ce157406 HTTP/1.1" 200 None 2025-12-17 16:18:40,283 - lava_callback - INFO - Completed processing callback for node 694034d4575bc060ce157406 2025-12-17 16:18:46,166 - lava_callback - INFO - Received callback for node 694034d1575bc060ce1573f4 INFO: 172.21.0.1:48238 - "POST /node/694034d1575bc060ce1573f4 HTTP/1.0" 202 Accepted 2025-12-17 16:18:46,172 - lava_callback - INFO - Starting async job submit for node 694034d1575bc060ce1573f4 2025-12-17 16:18:46,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:46,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d1575bc060ce1573f4 HTTP/1.1" 200 None 2025-12-17 16:18:46,212 - lava_callback - DEBUG - Processing node 694034d1575bc060ce1573f4: ltp-smoketest 2025-12-17 16:18:46,212 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034d1575bc060ce1573f4 2025-12-17 16:18:46,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:46,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034d1575bc060ce1573f4 HTTP/1.1" 200 None 2025-12-17 16:18:46,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:46,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:18:46,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:46,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034d1575bc060ce1573f4 HTTP/1.1" 200 None 2025-12-17 16:18:46,256 - lava_callback - INFO - Completed processing callback for node 694034d1575bc060ce1573f4 INFO: 172.21.0.1:48254 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:18:54,639 - lava_callback - INFO - Received callback for node 694034da575bc060ce15741b INFO: 172.21.0.1:35106 - "POST /node/694034da575bc060ce15741b HTTP/1.0" 202 Accepted 2025-12-17 16:18:54,651 - lava_callback - INFO - Starting async job submit for node 694034da575bc060ce15741b 2025-12-17 16:18:54,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:54,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034da575bc060ce15741b HTTP/1.1" 200 None 2025-12-17 16:18:54,691 - lava_callback - DEBUG - Processing node 694034da575bc060ce15741b: baseline-arm-baylibre 2025-12-17 16:18:54,691 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034da575bc060ce15741b 2025-12-17 16:18:54,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:54,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034da575bc060ce15741b HTTP/1.1" 200 None 2025-12-17 16:18:54,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:54,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:18:54,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:54,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034da575bc060ce15741b HTTP/1.1" 200 None 2025-12-17 16:18:54,766 - lava_callback - INFO - Completed processing callback for node 694034da575bc060ce15741b 2025-12-17 16:18:55,333 - lava_callback - INFO - Received callback for node 694034dc575bc060ce157422 INFO: 172.21.0.1:35120 - "POST /node/694034dc575bc060ce157422 HTTP/1.0" 202 Accepted 2025-12-17 16:18:55,344 - lava_callback - INFO - Starting async job submit for node 694034dc575bc060ce157422 2025-12-17 16:18:55,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:55,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034dc575bc060ce157422 HTTP/1.1" 200 None 2025-12-17 16:18:55,363 - lava_callback - DEBUG - Processing node 694034dc575bc060ce157422: baseline-arm-baylibre 2025-12-17 16:18:55,363 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034dc575bc060ce157422 2025-12-17 16:18:55,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:55,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034dc575bc060ce157422 HTTP/1.1" 200 None 2025-12-17 16:18:55,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:55,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:18:55,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:55,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034dc575bc060ce157422 HTTP/1.1" 200 None 2025-12-17 16:18:55,433 - lava_callback - INFO - Completed processing callback for node 694034dc575bc060ce157422 2025-12-17 16:18:55,997 - lava_callback - INFO - Received callback for node 694034dd575bc060ce157425 INFO: 172.21.0.1:35128 - "POST /node/694034dd575bc060ce157425 HTTP/1.0" 202 Accepted 2025-12-17 16:18:56,007 - lava_callback - INFO - Starting async job submit for node 694034dd575bc060ce157425 2025-12-17 16:18:56,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:56,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034dd575bc060ce157425 HTTP/1.1" 200 None 2025-12-17 16:18:56,022 - lava_callback - DEBUG - Processing node 694034dd575bc060ce157425: baseline-arm-baylibre 2025-12-17 16:18:56,022 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034dd575bc060ce157425 2025-12-17 16:18:56,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:56,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034dd575bc060ce157425 HTTP/1.1" 200 None 2025-12-17 16:18:56,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:56,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:18:56,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:56,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034dd575bc060ce157425 HTTP/1.1" 200 None 2025-12-17 16:18:56,096 - lava_callback - INFO - Completed processing callback for node 694034dd575bc060ce157425 INFO: 172.21.0.1:35134 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:18:56,685 - lava_callback - INFO - Received callback for node 694034df575bc060ce157428 INFO: 172.21.0.1:35142 - "POST /node/694034df575bc060ce157428 HTTP/1.0" 202 Accepted 2025-12-17 16:18:56,698 - lava_callback - INFO - Starting async job submit for node 694034df575bc060ce157428 2025-12-17 16:18:56,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:56,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034df575bc060ce157428 HTTP/1.1" 200 None 2025-12-17 16:18:56,718 - lava_callback - DEBUG - Processing node 694034df575bc060ce157428: baseline-arm-baylibre 2025-12-17 16:18:56,718 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034df575bc060ce157428 2025-12-17 16:18:56,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:56,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034df575bc060ce157428 HTTP/1.1" 200 None 2025-12-17 16:18:56,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:56,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:18:56,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:56,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034df575bc060ce157428 HTTP/1.1" 200 None 2025-12-17 16:18:56,792 - lava_callback - INFO - Completed processing callback for node 694034df575bc060ce157428 2025-12-17 16:18:57,337 - lava_callback - INFO - Received callback for node 694034e0575bc060ce157430 INFO: 172.21.0.1:35156 - "POST /node/694034e0575bc060ce157430 HTTP/1.0" 202 Accepted 2025-12-17 16:18:57,348 - lava_callback - INFO - Starting async job submit for node 694034e0575bc060ce157430 2025-12-17 16:18:57,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:57,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034e0575bc060ce157430 HTTP/1.1" 200 None 2025-12-17 16:18:57,363 - lava_callback - DEBUG - Processing node 694034e0575bc060ce157430: baseline-arm-baylibre 2025-12-17 16:18:57,363 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034e0575bc060ce157430 2025-12-17 16:18:57,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:57,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034e0575bc060ce157430 HTTP/1.1" 200 None 2025-12-17 16:18:57,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:57,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:18:57,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:57,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034e0575bc060ce157430 HTTP/1.1" 200 None 2025-12-17 16:18:57,423 - lava_callback - INFO - Completed processing callback for node 694034e0575bc060ce157430 2025-12-17 16:18:58,034 - lava_callback - INFO - Received callback for node 694034e1575bc060ce157433 INFO: 172.21.0.1:35164 - "POST /node/694034e1575bc060ce157433 HTTP/1.0" 202 Accepted 2025-12-17 16:18:58,045 - lava_callback - INFO - Starting async job submit for node 694034e1575bc060ce157433 2025-12-17 16:18:58,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:58,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034e1575bc060ce157433 HTTP/1.1" 200 None 2025-12-17 16:18:58,060 - lava_callback - DEBUG - Processing node 694034e1575bc060ce157433: baseline-arm-baylibre 2025-12-17 16:18:58,061 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034e1575bc060ce157433 2025-12-17 16:18:58,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:58,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034e1575bc060ce157433 HTTP/1.1" 200 None 2025-12-17 16:18:58,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:58,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:18:58,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:58,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034e1575bc060ce157433 HTTP/1.1" 200 None 2025-12-17 16:18:58,160 - lava_callback - INFO - Completed processing callback for node 694034e1575bc060ce157433 2025-12-17 16:18:58,713 - lava_callback - INFO - Received callback for node 694034e2575bc060ce157436 INFO: 172.21.0.1:35166 - "POST /node/694034e2575bc060ce157436 HTTP/1.0" 202 Accepted 2025-12-17 16:18:58,724 - lava_callback - INFO - Starting async job submit for node 694034e2575bc060ce157436 2025-12-17 16:18:58,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:58,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034e2575bc060ce157436 HTTP/1.1" 200 None 2025-12-17 16:18:58,767 - lava_callback - DEBUG - Processing node 694034e2575bc060ce157436: baseline-arm-baylibre 2025-12-17 16:18:58,767 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034e2575bc060ce157436 2025-12-17 16:18:58,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:58,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034e2575bc060ce157436 HTTP/1.1" 200 None 2025-12-17 16:18:58,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:58,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:18:58,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:58,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034e2575bc060ce157436 HTTP/1.1" 200 None 2025-12-17 16:18:58,822 - lava_callback - INFO - Completed processing callback for node 694034e2575bc060ce157436 2025-12-17 16:18:59,380 - lava_callback - INFO - Received callback for node 694034e4575bc060ce15743b INFO: 172.21.0.1:35168 - "POST /node/694034e4575bc060ce15743b HTTP/1.0" 202 Accepted 2025-12-17 16:18:59,392 - lava_callback - INFO - Starting async job submit for node 694034e4575bc060ce15743b 2025-12-17 16:18:59,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:59,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034e4575bc060ce15743b HTTP/1.1" 200 None 2025-12-17 16:18:59,405 - lava_callback - DEBUG - Processing node 694034e4575bc060ce15743b: baseline-arm-baylibre 2025-12-17 16:18:59,405 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034e4575bc060ce15743b 2025-12-17 16:18:59,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:59,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034e4575bc060ce15743b HTTP/1.1" 200 None 2025-12-17 16:18:59,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:59,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:18:59,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:18:59,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034e4575bc060ce15743b HTTP/1.1" 200 None 2025-12-17 16:18:59,478 - lava_callback - INFO - Completed processing callback for node 694034e4575bc060ce15743b 2025-12-17 16:19:00,074 - lava_callback - INFO - Received callback for node 694034e5575bc060ce157442 INFO: 172.21.0.1:33472 - "POST /node/694034e5575bc060ce157442 HTTP/1.0" 202 Accepted 2025-12-17 16:19:00,084 - lava_callback - INFO - Starting async job submit for node 694034e5575bc060ce157442 2025-12-17 16:19:00,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:00,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034e5575bc060ce157442 HTTP/1.1" 200 None 2025-12-17 16:19:00,101 - lava_callback - DEBUG - Processing node 694034e5575bc060ce157442: baseline-arm-baylibre 2025-12-17 16:19:00,101 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034e5575bc060ce157442 2025-12-17 16:19:00,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:00,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034e5575bc060ce157442 HTTP/1.1" 200 None 2025-12-17 16:19:00,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:00,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:00,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:00,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034e5575bc060ce157442 HTTP/1.1" 200 None 2025-12-17 16:19:00,169 - lava_callback - INFO - Completed processing callback for node 694034e5575bc060ce157442 2025-12-17 16:19:00,760 - lava_callback - INFO - Received callback for node 694034e6575bc060ce157445 INFO: 172.21.0.1:33476 - "POST /node/694034e6575bc060ce157445 HTTP/1.0" 202 Accepted 2025-12-17 16:19:00,771 - lava_callback - INFO - Starting async job submit for node 694034e6575bc060ce157445 2025-12-17 16:19:00,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:00,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034e6575bc060ce157445 HTTP/1.1" 200 None 2025-12-17 16:19:00,795 - lava_callback - DEBUG - Processing node 694034e6575bc060ce157445: baseline-arm-baylibre 2025-12-17 16:19:00,795 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034e6575bc060ce157445 2025-12-17 16:19:00,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:00,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034e6575bc060ce157445 HTTP/1.1" 200 None 2025-12-17 16:19:00,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:00,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:00,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:00,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034e6575bc060ce157445 HTTP/1.1" 200 None 2025-12-17 16:19:00,847 - lava_callback - INFO - Completed processing callback for node 694034e6575bc060ce157445 2025-12-17 16:19:01,469 - lava_callback - INFO - Received callback for node 694034e7575bc060ce157449 INFO: 172.21.0.1:33490 - "POST /node/694034e7575bc060ce157449 HTTP/1.0" 202 Accepted 2025-12-17 16:19:01,480 - lava_callback - INFO - Starting async job submit for node 694034e7575bc060ce157449 2025-12-17 16:19:01,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:01,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034e7575bc060ce157449 HTTP/1.1" 200 None 2025-12-17 16:19:01,494 - lava_callback - DEBUG - Processing node 694034e7575bc060ce157449: baseline-arm-baylibre 2025-12-17 16:19:01,494 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034e7575bc060ce157449 2025-12-17 16:19:01,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:01,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034e7575bc060ce157449 HTTP/1.1" 200 None 2025-12-17 16:19:01,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:01,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:01,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:01,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034e7575bc060ce157449 HTTP/1.1" 200 None 2025-12-17 16:19:01,571 - lava_callback - INFO - Completed processing callback for node 694034e7575bc060ce157449 2025-12-17 16:19:06,347 - lava_callback - INFO - Received callback for node 694034ee575bc060ce157459 INFO: 172.21.0.1:33498 - "POST /node/694034ee575bc060ce157459 HTTP/1.0" 202 Accepted 2025-12-17 16:19:06,357 - lava_callback - INFO - Starting async job submit for node 694034ee575bc060ce157459 2025-12-17 16:19:06,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:06,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ee575bc060ce157459 HTTP/1.1" 200 None 2025-12-17 16:19:06,369 - lava_callback - DEBUG - Processing node 694034ee575bc060ce157459: baseline-arm-broonie 2025-12-17 16:19:06,369 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034ee575bc060ce157459 2025-12-17 16:19:06,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:06,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ee575bc060ce157459 HTTP/1.1" 200 None 2025-12-17 16:19:06,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:06,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:06,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:06,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034ee575bc060ce157459 HTTP/1.1" 200 None 2025-12-17 16:19:06,420 - lava_callback - INFO - Completed processing callback for node 694034ee575bc060ce157459 INFO: 172.21.0.1:33504 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:19:07,108 - lava_callback - INFO - Received callback for node 694034e9575bc060ce15744d INFO: 172.21.0.1:33510 - "POST /node/694034e9575bc060ce15744d HTTP/1.0" 202 Accepted 2025-12-17 16:19:07,119 - lava_callback - INFO - Starting async job submit for node 694034e9575bc060ce15744d 2025-12-17 16:19:07,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:07,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034e9575bc060ce15744d HTTP/1.1" 200 None 2025-12-17 16:19:07,133 - lava_callback - DEBUG - Processing node 694034e9575bc060ce15744d: baseline-arm-broonie 2025-12-17 16:19:07,133 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034e9575bc060ce15744d 2025-12-17 16:19:07,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:07,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034e9575bc060ce15744d HTTP/1.1" 200 None 2025-12-17 16:19:07,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:07,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:07,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:07,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034e9575bc060ce15744d HTTP/1.1" 200 None 2025-12-17 16:19:07,190 - lava_callback - INFO - Completed processing callback for node 694034e9575bc060ce15744d 2025-12-17 16:19:07,872 - lava_callback - INFO - Received callback for node 694034ea575bc060ce157450 INFO: 172.21.0.1:33520 - "POST /node/694034ea575bc060ce157450 HTTP/1.0" 202 Accepted 2025-12-17 16:19:07,880 - lava_callback - INFO - Starting async job submit for node 694034ea575bc060ce157450 2025-12-17 16:19:07,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:07,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ea575bc060ce157450 HTTP/1.1" 200 None 2025-12-17 16:19:07,890 - lava_callback - DEBUG - Processing node 694034ea575bc060ce157450: baseline-arm-broonie 2025-12-17 16:19:07,890 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034ea575bc060ce157450 2025-12-17 16:19:07,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:07,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ea575bc060ce157450 HTTP/1.1" 200 None 2025-12-17 16:19:07,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:07,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:07,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:07,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034ea575bc060ce157450 HTTP/1.1" 200 None 2025-12-17 16:19:07,936 - lava_callback - INFO - Completed processing callback for node 694034ea575bc060ce157450 2025-12-17 16:19:08,583 - lava_callback - INFO - Received callback for node 694034eb575bc060ce157453 INFO: 172.21.0.1:33528 - "POST /node/694034eb575bc060ce157453 HTTP/1.0" 202 Accepted 2025-12-17 16:19:08,592 - lava_callback - INFO - Starting async job submit for node 694034eb575bc060ce157453 2025-12-17 16:19:08,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:08,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034eb575bc060ce157453 HTTP/1.1" 200 None 2025-12-17 16:19:08,602 - lava_callback - DEBUG - Processing node 694034eb575bc060ce157453: baseline-arm-broonie 2025-12-17 16:19:08,602 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034eb575bc060ce157453 2025-12-17 16:19:08,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:08,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034eb575bc060ce157453 HTTP/1.1" 200 None 2025-12-17 16:19:08,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:08,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:08,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:08,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034eb575bc060ce157453 HTTP/1.1" 200 None 2025-12-17 16:19:08,642 - lava_callback - INFO - Completed processing callback for node 694034eb575bc060ce157453 2025-12-17 16:19:09,234 - lava_callback - INFO - Received callback for node 694034f0575bc060ce15745f INFO: 172.21.0.1:33544 - "POST /node/694034f0575bc060ce15745f HTTP/1.0" 202 Accepted 2025-12-17 16:19:09,242 - lava_callback - INFO - Starting async job submit for node 694034f0575bc060ce15745f 2025-12-17 16:19:09,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:09,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034f0575bc060ce15745f HTTP/1.1" 200 None 2025-12-17 16:19:09,257 - lava_callback - DEBUG - Processing node 694034f0575bc060ce15745f: baseline-arm-clabbe 2025-12-17 16:19:09,257 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034f0575bc060ce15745f 2025-12-17 16:19:09,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:09,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034f0575bc060ce15745f HTTP/1.1" 200 None 2025-12-17 16:19:09,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:09,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:09,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:09,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034f0575bc060ce15745f HTTP/1.1" 200 None 2025-12-17 16:19:09,319 - lava_callback - INFO - Completed processing callback for node 694034f0575bc060ce15745f 2025-12-17 16:19:09,347 - lava_callback - INFO - Received callback for node 694034ec575bc060ce157456 INFO: 172.21.0.1:33560 - "POST /node/694034ec575bc060ce157456 HTTP/1.0" 202 Accepted 2025-12-17 16:19:09,357 - lava_callback - INFO - Starting async job submit for node 694034ec575bc060ce157456 2025-12-17 16:19:09,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:09,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ec575bc060ce157456 HTTP/1.1" 200 None 2025-12-17 16:19:09,372 - lava_callback - DEBUG - Processing node 694034ec575bc060ce157456: baseline-arm-broonie 2025-12-17 16:19:09,372 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034ec575bc060ce157456 2025-12-17 16:19:09,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:09,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ec575bc060ce157456 HTTP/1.1" 200 None 2025-12-17 16:19:09,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:09,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:09,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:09,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034ec575bc060ce157456 HTTP/1.1" 200 None 2025-12-17 16:19:09,426 - lava_callback - INFO - Completed processing callback for node 694034ec575bc060ce157456 2025-12-17 16:19:09,879 - lava_callback - INFO - Received callback for node 694034f2575bc060ce157462 INFO: 172.21.0.1:50942 - "POST /node/694034f2575bc060ce157462 HTTP/1.0" 202 Accepted 2025-12-17 16:19:09,886 - lava_callback - INFO - Starting async job submit for node 694034f2575bc060ce157462 2025-12-17 16:19:09,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:09,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034f2575bc060ce157462 HTTP/1.1" 200 None 2025-12-17 16:19:09,896 - lava_callback - DEBUG - Processing node 694034f2575bc060ce157462: baseline-arm-clabbe 2025-12-17 16:19:09,896 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034f2575bc060ce157462 2025-12-17 16:19:09,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:09,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034f2575bc060ce157462 HTTP/1.1" 200 None 2025-12-17 16:19:09,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:09,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:09,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:09,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034f2575bc060ce157462 HTTP/1.1" 200 None 2025-12-17 16:19:09,934 - lava_callback - INFO - Completed processing callback for node 694034f2575bc060ce157462 2025-12-17 16:19:10,091 - lava_callback - INFO - Received callback for node 694034ef575bc060ce15745c INFO: 172.21.0.1:50944 - "POST /node/694034ef575bc060ce15745c HTTP/1.0" 202 Accepted 2025-12-17 16:19:10,099 - lava_callback - INFO - Starting async job submit for node 694034ef575bc060ce15745c 2025-12-17 16:19:10,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:10,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ef575bc060ce15745c HTTP/1.1" 200 None 2025-12-17 16:19:10,112 - lava_callback - DEBUG - Processing node 694034ef575bc060ce15745c: baseline-arm-broonie 2025-12-17 16:19:10,112 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034ef575bc060ce15745c 2025-12-17 16:19:10,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:10,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ef575bc060ce15745c HTTP/1.1" 200 None 2025-12-17 16:19:10,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:10,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:10,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:10,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034ef575bc060ce15745c HTTP/1.1" 200 None 2025-12-17 16:19:10,154 - lava_callback - INFO - Completed processing callback for node 694034ef575bc060ce15745c 2025-12-17 16:19:10,523 - lava_callback - INFO - Received callback for node 694034f4575bc060ce157465 INFO: 172.21.0.1:50954 - "POST /node/694034f4575bc060ce157465 HTTP/1.0" 202 Accepted 2025-12-17 16:19:10,529 - lava_callback - INFO - Starting async job submit for node 694034f4575bc060ce157465 2025-12-17 16:19:10,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:10,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034f4575bc060ce157465 HTTP/1.1" 200 None 2025-12-17 16:19:10,538 - lava_callback - DEBUG - Processing node 694034f4575bc060ce157465: baseline-arm-clabbe 2025-12-17 16:19:10,538 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034f4575bc060ce157465 2025-12-17 16:19:10,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:10,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034f4575bc060ce157465 HTTP/1.1" 200 None 2025-12-17 16:19:10,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:10,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:10,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:10,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034f4575bc060ce157465 HTTP/1.1" 200 None 2025-12-17 16:19:10,579 - lava_callback - INFO - Completed processing callback for node 694034f4575bc060ce157465 INFO: 172.21.0.1:50966 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:19:11,176 - lava_callback - INFO - Received callback for node 694034f5575bc060ce157468 INFO: 172.21.0.1:50982 - "POST /node/694034f5575bc060ce157468 HTTP/1.0" 202 Accepted 2025-12-17 16:19:11,183 - lava_callback - INFO - Starting async job submit for node 694034f5575bc060ce157468 2025-12-17 16:19:11,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:11,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034f5575bc060ce157468 HTTP/1.1" 200 None 2025-12-17 16:19:11,195 - lava_callback - DEBUG - Processing node 694034f5575bc060ce157468: baseline-arm-clabbe 2025-12-17 16:19:11,195 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034f5575bc060ce157468 2025-12-17 16:19:11,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:11,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034f5575bc060ce157468 HTTP/1.1" 200 None 2025-12-17 16:19:11,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:11,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:11,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:11,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034f5575bc060ce157468 HTTP/1.1" 200 None 2025-12-17 16:19:11,237 - lava_callback - INFO - Completed processing callback for node 694034f5575bc060ce157468 2025-12-17 16:19:11,842 - lava_callback - INFO - Received callback for node 694034f6575bc060ce15746b INFO: 172.21.0.1:50986 - "POST /node/694034f6575bc060ce15746b HTTP/1.0" 202 Accepted 2025-12-17 16:19:11,849 - lava_callback - INFO - Starting async job submit for node 694034f6575bc060ce15746b 2025-12-17 16:19:11,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:11,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034f6575bc060ce15746b HTTP/1.1" 200 None 2025-12-17 16:19:11,864 - lava_callback - DEBUG - Processing node 694034f6575bc060ce15746b: baseline-arm-clabbe 2025-12-17 16:19:11,864 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034f6575bc060ce15746b 2025-12-17 16:19:11,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:11,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034f6575bc060ce15746b HTTP/1.1" 200 None 2025-12-17 16:19:11,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:11,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:11,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:11,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034f6575bc060ce15746b HTTP/1.1" 200 None 2025-12-17 16:19:11,907 - lava_callback - INFO - Completed processing callback for node 694034f6575bc060ce15746b 2025-12-17 16:19:12,493 - lava_callback - INFO - Received callback for node 694034f8575bc060ce157470 INFO: 172.21.0.1:50998 - "POST /node/694034f8575bc060ce157470 HTTP/1.0" 202 Accepted 2025-12-17 16:19:12,500 - lava_callback - INFO - Starting async job submit for node 694034f8575bc060ce157470 2025-12-17 16:19:12,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:12,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034f8575bc060ce157470 HTTP/1.1" 200 None 2025-12-17 16:19:12,509 - lava_callback - DEBUG - Processing node 694034f8575bc060ce157470: baseline-arm-clabbe 2025-12-17 16:19:12,509 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034f8575bc060ce157470 2025-12-17 16:19:12,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:12,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034f8575bc060ce157470 HTTP/1.1" 200 None 2025-12-17 16:19:12,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:12,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:12,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:12,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034f8575bc060ce157470 HTTP/1.1" 200 None 2025-12-17 16:19:12,547 - lava_callback - INFO - Completed processing callback for node 694034f8575bc060ce157470 2025-12-17 16:19:13,163 - lava_callback - INFO - Received callback for node 694034f9575bc060ce157477 INFO: 172.21.0.1:51012 - "POST /node/694034f9575bc060ce157477 HTTP/1.0" 202 Accepted 2025-12-17 16:19:13,170 - lava_callback - INFO - Starting async job submit for node 694034f9575bc060ce157477 2025-12-17 16:19:13,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:13,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034f9575bc060ce157477 HTTP/1.1" 200 None 2025-12-17 16:19:13,179 - lava_callback - DEBUG - Processing node 694034f9575bc060ce157477: baseline-arm-clabbe 2025-12-17 16:19:13,179 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034f9575bc060ce157477 2025-12-17 16:19:13,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:13,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034f9575bc060ce157477 HTTP/1.1" 200 None 2025-12-17 16:19:13,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:13,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:13,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:13,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034f9575bc060ce157477 HTTP/1.1" 200 None 2025-12-17 16:19:13,217 - lava_callback - INFO - Completed processing callback for node 694034f9575bc060ce157477 2025-12-17 16:19:13,810 - lava_callback - INFO - Received callback for node 694034fa575bc060ce15747e INFO: 172.21.0.1:51028 - "POST /node/694034fa575bc060ce15747e HTTP/1.0" 202 Accepted 2025-12-17 16:19:13,820 - lava_callback - INFO - Starting async job submit for node 694034fa575bc060ce15747e 2025-12-17 16:19:13,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:13,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034fa575bc060ce15747e HTTP/1.1" 200 None 2025-12-17 16:19:13,831 - lava_callback - DEBUG - Processing node 694034fa575bc060ce15747e: baseline-arm-clabbe 2025-12-17 16:19:13,831 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034fa575bc060ce15747e 2025-12-17 16:19:13,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:13,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034fa575bc060ce15747e HTTP/1.1" 200 None 2025-12-17 16:19:13,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:13,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:13,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:13,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034fa575bc060ce15747e HTTP/1.1" 200 None 2025-12-17 16:19:13,873 - lava_callback - INFO - Completed processing callback for node 694034fa575bc060ce15747e 2025-12-17 16:19:14,472 - lava_callback - INFO - Received callback for node 694034fc575bc060ce157483 INFO: 172.21.0.1:51042 - "POST /node/694034fc575bc060ce157483 HTTP/1.0" 202 Accepted 2025-12-17 16:19:14,482 - lava_callback - INFO - Starting async job submit for node 694034fc575bc060ce157483 2025-12-17 16:19:14,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:14,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034fc575bc060ce157483 HTTP/1.1" 200 None 2025-12-17 16:19:14,495 - lava_callback - DEBUG - Processing node 694034fc575bc060ce157483: baseline-arm-clabbe 2025-12-17 16:19:14,495 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034fc575bc060ce157483 2025-12-17 16:19:14,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:14,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034fc575bc060ce157483 HTTP/1.1" 200 None 2025-12-17 16:19:14,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:14,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:14,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:14,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034fc575bc060ce157483 HTTP/1.1" 200 None 2025-12-17 16:19:14,539 - lava_callback - INFO - Completed processing callback for node 694034fc575bc060ce157483 INFO: 172.21.0.1:51054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51070 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:19:26,451 - lava_callback - INFO - Received callback for node 69403501575bc060ce15748f INFO: 172.21.0.1:42892 - "POST /node/69403501575bc060ce15748f HTTP/1.0" 202 Accepted 2025-12-17 16:19:26,458 - lava_callback - INFO - Starting async job submit for node 69403501575bc060ce15748f 2025-12-17 16:19:26,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:26,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403501575bc060ce15748f HTTP/1.1" 200 None 2025-12-17 16:19:26,468 - lava_callback - DEBUG - Processing node 69403501575bc060ce15748f: ltp-cap-bounds 2025-12-17 16:19:26,468 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403501575bc060ce15748f 2025-12-17 16:19:26,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:26,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403501575bc060ce15748f HTTP/1.1" 200 None 2025-12-17 16:19:26,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:26,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:26,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:26,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403501575bc060ce15748f HTTP/1.1" 200 None 2025-12-17 16:19:26,504 - lava_callback - INFO - Completed processing callback for node 69403501575bc060ce15748f INFO: 172.21.0.1:42896 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:19:27,201 - lava_callback - INFO - Received callback for node 69403502575bc060ce157492 INFO: 172.21.0.1:42912 - "POST /node/69403502575bc060ce157492 HTTP/1.0" 202 Accepted 2025-12-17 16:19:27,209 - lava_callback - INFO - Starting async job submit for node 69403502575bc060ce157492 2025-12-17 16:19:27,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:27,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403502575bc060ce157492 HTTP/1.1" 200 None 2025-12-17 16:19:27,219 - lava_callback - DEBUG - Processing node 69403502575bc060ce157492: ltp-cap-bounds 2025-12-17 16:19:27,219 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403502575bc060ce157492 2025-12-17 16:19:27,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:27,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403502575bc060ce157492 HTTP/1.1" 200 None 2025-12-17 16:19:27,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:27,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:27,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:27,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403502575bc060ce157492 HTTP/1.1" 200 None 2025-12-17 16:19:27,274 - lava_callback - INFO - Completed processing callback for node 69403502575bc060ce157492 2025-12-17 16:19:27,927 - lava_callback - INFO - Received callback for node 6940350a575bc060ce1574a4 INFO: 172.21.0.1:42920 - "POST /node/6940350a575bc060ce1574a4 HTTP/1.0" 202 Accepted 2025-12-17 16:19:27,933 - lava_callback - INFO - Starting async job submit for node 6940350a575bc060ce1574a4 2025-12-17 16:19:27,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:27,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940350a575bc060ce1574a4 HTTP/1.1" 200 None 2025-12-17 16:19:27,943 - lava_callback - DEBUG - Processing node 6940350a575bc060ce1574a4: ltp-containers 2025-12-17 16:19:27,944 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940350a575bc060ce1574a4 2025-12-17 16:19:27,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:27,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940350a575bc060ce1574a4 HTTP/1.1" 200 None 2025-12-17 16:19:27,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:27,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:27,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:27,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940350a575bc060ce1574a4 HTTP/1.1" 200 None 2025-12-17 16:19:27,976 - lava_callback - INFO - Completed processing callback for node 6940350a575bc060ce1574a4 2025-12-17 16:19:28,691 - lava_callback - INFO - Received callback for node 6940350c575bc060ce1574a9 INFO: 172.21.0.1:42924 - "POST /node/6940350c575bc060ce1574a9 HTTP/1.0" 202 Accepted 2025-12-17 16:19:28,700 - lava_callback - INFO - Starting async job submit for node 6940350c575bc060ce1574a9 2025-12-17 16:19:28,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:28,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940350c575bc060ce1574a9 HTTP/1.1" 200 None 2025-12-17 16:19:28,721 - lava_callback - DEBUG - Processing node 6940350c575bc060ce1574a9: ltp-containers 2025-12-17 16:19:28,721 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940350c575bc060ce1574a9 2025-12-17 16:19:28,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:28,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940350c575bc060ce1574a9 HTTP/1.1" 200 None 2025-12-17 16:19:28,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:28,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:28,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:28,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940350c575bc060ce1574a9 HTTP/1.1" 200 None 2025-12-17 16:19:28,760 - lava_callback - INFO - Completed processing callback for node 6940350c575bc060ce1574a9 2025-12-17 16:19:29,228 - lava_callback - INFO - Received callback for node 694034fd575bc060ce157486 INFO: 172.21.0.1:42936 - "POST /node/694034fd575bc060ce157486 HTTP/1.0" 202 Accepted 2025-12-17 16:19:29,235 - lava_callback - INFO - Starting async job submit for node 694034fd575bc060ce157486 2025-12-17 16:19:29,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:29,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034fd575bc060ce157486 HTTP/1.1" 200 None 2025-12-17 16:19:29,251 - lava_callback - DEBUG - Processing node 694034fd575bc060ce157486: baseline-arm-clabbe 2025-12-17 16:19:29,251 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034fd575bc060ce157486 2025-12-17 16:19:29,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:29,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034fd575bc060ce157486 HTTP/1.1" 200 None 2025-12-17 16:19:29,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:29,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:29,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:29,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034fd575bc060ce157486 HTTP/1.1" 200 None 2025-12-17 16:19:29,296 - lava_callback - INFO - Completed processing callback for node 694034fd575bc060ce157486 2025-12-17 16:19:29,415 - lava_callback - INFO - Received callback for node 6940350d575bc060ce1574b0 INFO: 172.21.0.1:42940 - "POST /node/6940350d575bc060ce1574b0 HTTP/1.0" 202 Accepted 2025-12-17 16:19:29,423 - lava_callback - INFO - Starting async job submit for node 6940350d575bc060ce1574b0 2025-12-17 16:19:29,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:29,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940350d575bc060ce1574b0 HTTP/1.1" 200 None 2025-12-17 16:19:29,433 - lava_callback - DEBUG - Processing node 6940350d575bc060ce1574b0: ltp-containers 2025-12-17 16:19:29,433 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940350d575bc060ce1574b0 2025-12-17 16:19:29,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:29,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940350d575bc060ce1574b0 HTTP/1.1" 200 None 2025-12-17 16:19:29,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:29,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:29,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:29,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940350d575bc060ce1574b0 HTTP/1.1" 200 None 2025-12-17 16:19:29,483 - lava_callback - INFO - Completed processing callback for node 6940350d575bc060ce1574b0 2025-12-17 16:19:29,914 - lava_callback - INFO - Received callback for node 694034fe575bc060ce157489 INFO: 172.21.0.1:49882 - "POST /node/694034fe575bc060ce157489 HTTP/1.0" 202 Accepted 2025-12-17 16:19:29,920 - lava_callback - INFO - Starting async job submit for node 694034fe575bc060ce157489 2025-12-17 16:19:29,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:29,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034fe575bc060ce157489 HTTP/1.1" 200 None 2025-12-17 16:19:29,931 - lava_callback - DEBUG - Processing node 694034fe575bc060ce157489: baseline-arm-clabbe 2025-12-17 16:19:29,931 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034fe575bc060ce157489 2025-12-17 16:19:29,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:29,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034fe575bc060ce157489 HTTP/1.1" 200 None 2025-12-17 16:19:29,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:29,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:29,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:29,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034fe575bc060ce157489 HTTP/1.1" 200 None 2025-12-17 16:19:29,976 - lava_callback - INFO - Completed processing callback for node 694034fe575bc060ce157489 2025-12-17 16:19:30,186 - lava_callback - INFO - Received callback for node 69403510575bc060ce1574ba INFO: 172.21.0.1:49890 - "POST /node/69403510575bc060ce1574ba HTTP/1.0" 202 Accepted 2025-12-17 16:19:30,194 - lava_callback - INFO - Starting async job submit for node 69403510575bc060ce1574ba 2025-12-17 16:19:30,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:30,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403510575bc060ce1574ba HTTP/1.1" 200 None 2025-12-17 16:19:30,205 - lava_callback - DEBUG - Processing node 69403510575bc060ce1574ba: ltp-controllers 2025-12-17 16:19:30,205 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403510575bc060ce1574ba 2025-12-17 16:19:30,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:30,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403510575bc060ce1574ba HTTP/1.1" 200 None 2025-12-17 16:19:30,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:30,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:30,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:30,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403510575bc060ce1574ba HTTP/1.1" 200 None 2025-12-17 16:19:30,246 - lava_callback - INFO - Completed processing callback for node 69403510575bc060ce1574ba 2025-12-17 16:19:30,574 - lava_callback - INFO - Received callback for node 694034ff575bc060ce15748c INFO: 172.21.0.1:49906 - "POST /node/694034ff575bc060ce15748c HTTP/1.0" 202 Accepted 2025-12-17 16:19:30,580 - lava_callback - INFO - Starting async job submit for node 694034ff575bc060ce15748c 2025-12-17 16:19:30,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:30,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ff575bc060ce15748c HTTP/1.1" 200 None 2025-12-17 16:19:30,594 - lava_callback - DEBUG - Processing node 694034ff575bc060ce15748c: baseline-arm-clabbe 2025-12-17 16:19:30,594 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694034ff575bc060ce15748c 2025-12-17 16:19:30,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:30,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034ff575bc060ce15748c HTTP/1.1" 200 None 2025-12-17 16:19:30,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:30,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:30,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:30,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034ff575bc060ce15748c HTTP/1.1" 200 None 2025-12-17 16:19:30,634 - lava_callback - INFO - Completed processing callback for node 694034ff575bc060ce15748c 2025-12-17 16:19:30,919 - lava_callback - INFO - Received callback for node 69403511575bc060ce1574bd INFO: 172.21.0.1:49914 - "POST /node/69403511575bc060ce1574bd HTTP/1.0" 202 Accepted 2025-12-17 16:19:30,926 - lava_callback - INFO - Starting async job submit for node 69403511575bc060ce1574bd 2025-12-17 16:19:30,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:30,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403511575bc060ce1574bd HTTP/1.1" 200 None 2025-12-17 16:19:30,935 - lava_callback - DEBUG - Processing node 69403511575bc060ce1574bd: ltp-controllers 2025-12-17 16:19:30,935 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403511575bc060ce1574bd 2025-12-17 16:19:30,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:30,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403511575bc060ce1574bd HTTP/1.1" 200 None 2025-12-17 16:19:30,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:30,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:30,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:30,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403511575bc060ce1574bd HTTP/1.1" 200 None 2025-12-17 16:19:30,973 - lava_callback - INFO - Completed processing callback for node 69403511575bc060ce1574bd 2025-12-17 16:19:31,267 - lava_callback - INFO - Received callback for node 69403506575bc060ce15749b INFO: 172.21.0.1:49918 - "POST /node/69403506575bc060ce15749b HTTP/1.0" 202 Accepted 2025-12-17 16:19:31,275 - lava_callback - INFO - Starting async job submit for node 69403506575bc060ce15749b 2025-12-17 16:19:31,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:31,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403506575bc060ce15749b HTTP/1.1" 200 None 2025-12-17 16:19:31,289 - lava_callback - DEBUG - Processing node 69403506575bc060ce15749b: baseline-arm-clabbe 2025-12-17 16:19:31,289 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403506575bc060ce15749b 2025-12-17 16:19:31,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:31,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403506575bc060ce15749b HTTP/1.1" 200 None 2025-12-17 16:19:31,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:31,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:31,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:31,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403506575bc060ce15749b HTTP/1.1" 200 None 2025-12-17 16:19:31,326 - lava_callback - INFO - Completed processing callback for node 69403506575bc060ce15749b 2025-12-17 16:19:31,926 - lava_callback - INFO - Received callback for node 69403508575bc060ce15749e INFO: 172.21.0.1:49920 - "POST /node/69403508575bc060ce15749e HTTP/1.0" 202 Accepted 2025-12-17 16:19:31,932 - lava_callback - INFO - Starting async job submit for node 69403508575bc060ce15749e 2025-12-17 16:19:31,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:31,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403508575bc060ce15749e HTTP/1.1" 200 None 2025-12-17 16:19:31,943 - lava_callback - DEBUG - Processing node 69403508575bc060ce15749e: baseline-arm-clabbe 2025-12-17 16:19:31,943 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403508575bc060ce15749e 2025-12-17 16:19:31,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:31,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403508575bc060ce15749e HTTP/1.1" 200 None 2025-12-17 16:19:31,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:31,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:31,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:31,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403508575bc060ce15749e HTTP/1.1" 200 None 2025-12-17 16:19:31,981 - lava_callback - INFO - Completed processing callback for node 69403508575bc060ce15749e 2025-12-17 16:19:32,579 - lava_callback - INFO - Received callback for node 69403509575bc060ce1574a1 INFO: 172.21.0.1:49922 - "POST /node/69403509575bc060ce1574a1 HTTP/1.0" 202 Accepted 2025-12-17 16:19:32,587 - lava_callback - INFO - Starting async job submit for node 69403509575bc060ce1574a1 2025-12-17 16:19:32,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:32,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403509575bc060ce1574a1 HTTP/1.1" 200 None 2025-12-17 16:19:32,598 - lava_callback - DEBUG - Processing node 69403509575bc060ce1574a1: baseline-arm-clabbe 2025-12-17 16:19:32,598 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403509575bc060ce1574a1 2025-12-17 16:19:32,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:32,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403509575bc060ce1574a1 HTTP/1.1" 200 None 2025-12-17 16:19:32,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:32,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:32,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:32,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403509575bc060ce1574a1 HTTP/1.1" 200 None 2025-12-17 16:19:32,633 - lava_callback - INFO - Completed processing callback for node 69403509575bc060ce1574a1 2025-12-17 16:19:34,878 - lava_callback - INFO - Received callback for node 69403503575bc060ce157495 INFO: 172.21.0.1:49936 - "POST /node/69403503575bc060ce157495 HTTP/1.0" 202 Accepted 2025-12-17 16:19:34,886 - lava_callback - INFO - Starting async job submit for node 69403503575bc060ce157495 2025-12-17 16:19:34,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:34,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403503575bc060ce157495 HTTP/1.1" 200 None 2025-12-17 16:19:34,897 - lava_callback - DEBUG - Processing node 69403503575bc060ce157495: baseline-arm-pengutronix 2025-12-17 16:19:34,897 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403503575bc060ce157495 2025-12-17 16:19:34,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:34,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403503575bc060ce157495 HTTP/1.1" 200 None 2025-12-17 16:19:34,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:34,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:34,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:34,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403503575bc060ce157495 HTTP/1.1" 200 None 2025-12-17 16:19:34,943 - lava_callback - INFO - Completed processing callback for node 69403503575bc060ce157495 2025-12-17 16:19:35,495 - lava_callback - INFO - Received callback for node 69403505575bc060ce157498 INFO: 172.21.0.1:49948 - "POST /node/69403505575bc060ce157498 HTTP/1.0" 202 Accepted 2025-12-17 16:19:35,502 - lava_callback - INFO - Starting async job submit for node 69403505575bc060ce157498 2025-12-17 16:19:35,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:35,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403505575bc060ce157498 HTTP/1.1" 200 None 2025-12-17 16:19:35,516 - lava_callback - DEBUG - Processing node 69403505575bc060ce157498: baseline-arm-pengutronix 2025-12-17 16:19:35,516 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403505575bc060ce157498 2025-12-17 16:19:35,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:35,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403505575bc060ce157498 HTTP/1.1" 200 None 2025-12-17 16:19:35,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:35,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:35,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:35,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403505575bc060ce157498 HTTP/1.1" 200 None 2025-12-17 16:19:35,588 - lava_callback - INFO - Completed processing callback for node 69403505575bc060ce157498 INFO: 172.21.0.1:49954 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:19:46,242 - lava_callback - INFO - Received callback for node 69403513575bc060ce1574c0 INFO: 172.21.0.1:35754 - "POST /node/69403513575bc060ce1574c0 HTTP/1.0" 202 Accepted 2025-12-17 16:19:46,248 - lava_callback - INFO - Starting async job submit for node 69403513575bc060ce1574c0 2025-12-17 16:19:46,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:46,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403513575bc060ce1574c0 HTTP/1.1" 200 None 2025-12-17 16:19:46,257 - lava_callback - DEBUG - Processing node 69403513575bc060ce1574c0: ltp-controllers 2025-12-17 16:19:46,258 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403513575bc060ce1574c0 2025-12-17 16:19:46,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:46,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403513575bc060ce1574c0 HTTP/1.1" 200 None 2025-12-17 16:19:46,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:46,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:46,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:46,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403513575bc060ce1574c0 HTTP/1.1" 200 None 2025-12-17 16:19:46,294 - lava_callback - INFO - Completed processing callback for node 69403513575bc060ce1574c0 INFO: 172.21.0.1:35758 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:19:46,989 - lava_callback - INFO - Received callback for node 69403514575bc060ce1574c3 INFO: 172.21.0.1:35770 - "POST /node/69403514575bc060ce1574c3 HTTP/1.0" 202 Accepted 2025-12-17 16:19:46,997 - lava_callback - INFO - Starting async job submit for node 69403514575bc060ce1574c3 2025-12-17 16:19:46,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:47,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403514575bc060ce1574c3 HTTP/1.1" 200 None 2025-12-17 16:19:47,006 - lava_callback - DEBUG - Processing node 69403514575bc060ce1574c3: ltp-cve 2025-12-17 16:19:47,007 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403514575bc060ce1574c3 2025-12-17 16:19:47,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:47,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403514575bc060ce1574c3 HTTP/1.1" 200 None 2025-12-17 16:19:47,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:47,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:47,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:47,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403514575bc060ce1574c3 HTTP/1.1" 200 None 2025-12-17 16:19:47,043 - lava_callback - INFO - Completed processing callback for node 69403514575bc060ce1574c3 2025-12-17 16:19:47,704 - lava_callback - INFO - Received callback for node 69403515575bc060ce1574c6 INFO: 172.21.0.1:35784 - "POST /node/69403515575bc060ce1574c6 HTTP/1.0" 202 Accepted 2025-12-17 16:19:47,714 - lava_callback - INFO - Starting async job submit for node 69403515575bc060ce1574c6 2025-12-17 16:19:47,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:47,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403515575bc060ce1574c6 HTTP/1.1" 200 None 2025-12-17 16:19:47,726 - lava_callback - DEBUG - Processing node 69403515575bc060ce1574c6: ltp-cve 2025-12-17 16:19:47,726 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403515575bc060ce1574c6 2025-12-17 16:19:47,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:47,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403515575bc060ce1574c6 HTTP/1.1" 200 None 2025-12-17 16:19:47,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:47,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:47,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:47,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403515575bc060ce1574c6 HTTP/1.1" 200 None 2025-12-17 16:19:47,773 - lava_callback - INFO - Completed processing callback for node 69403515575bc060ce1574c6 2025-12-17 16:19:48,444 - lava_callback - INFO - Received callback for node 69403516575bc060ce1574c9 INFO: 172.21.0.1:35786 - "POST /node/69403516575bc060ce1574c9 HTTP/1.0" 202 Accepted 2025-12-17 16:19:48,452 - lava_callback - INFO - Starting async job submit for node 69403516575bc060ce1574c9 2025-12-17 16:19:48,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:48,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403516575bc060ce1574c9 HTTP/1.1" 200 None 2025-12-17 16:19:48,461 - lava_callback - DEBUG - Processing node 69403516575bc060ce1574c9: ltp-cve 2025-12-17 16:19:48,462 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403516575bc060ce1574c9 2025-12-17 16:19:48,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:48,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403516575bc060ce1574c9 HTTP/1.1" 200 None 2025-12-17 16:19:48,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:48,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:48,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:48,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403516575bc060ce1574c9 HTTP/1.1" 200 None 2025-12-17 16:19:48,502 - lava_callback - INFO - Completed processing callback for node 69403516575bc060ce1574c9 2025-12-17 16:19:49,194 - lava_callback - INFO - Received callback for node 69403518575bc060ce1574cc INFO: 172.21.0.1:35798 - "POST /node/69403518575bc060ce1574cc HTTP/1.0" 202 Accepted 2025-12-17 16:19:49,200 - lava_callback - INFO - Starting async job submit for node 69403518575bc060ce1574cc 2025-12-17 16:19:49,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:49,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403518575bc060ce1574cc HTTP/1.1" 200 None 2025-12-17 16:19:49,209 - lava_callback - DEBUG - Processing node 69403518575bc060ce1574cc: ltp-fs-bind 2025-12-17 16:19:49,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403518575bc060ce1574cc 2025-12-17 16:19:49,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:49,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403518575bc060ce1574cc HTTP/1.1" 200 None 2025-12-17 16:19:49,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:49,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:49,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:49,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403518575bc060ce1574cc HTTP/1.1" 200 None 2025-12-17 16:19:49,245 - lava_callback - INFO - Completed processing callback for node 69403518575bc060ce1574cc 2025-12-17 16:19:49,966 - lava_callback - INFO - Received callback for node 69403519575bc060ce1574cf INFO: 172.21.0.1:36642 - "POST /node/69403519575bc060ce1574cf HTTP/1.0" 202 Accepted 2025-12-17 16:19:49,973 - lava_callback - INFO - Starting async job submit for node 69403519575bc060ce1574cf 2025-12-17 16:19:49,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:49,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403519575bc060ce1574cf HTTP/1.1" 200 None 2025-12-17 16:19:49,989 - lava_callback - DEBUG - Processing node 69403519575bc060ce1574cf: ltp-fs-bind 2025-12-17 16:19:49,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403519575bc060ce1574cf 2025-12-17 16:19:49,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:49,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403519575bc060ce1574cf HTTP/1.1" 200 None 2025-12-17 16:19:49,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:50,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:50,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:50,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403519575bc060ce1574cf HTTP/1.1" 200 None 2025-12-17 16:19:50,033 - lava_callback - INFO - Completed processing callback for node 69403519575bc060ce1574cf 2025-12-17 16:19:50,713 - lava_callback - INFO - Received callback for node 6940351a575bc060ce1574d2 INFO: 172.21.0.1:36650 - "POST /node/6940351a575bc060ce1574d2 HTTP/1.0" 202 Accepted 2025-12-17 16:19:50,726 - lava_callback - INFO - Starting async job submit for node 6940351a575bc060ce1574d2 2025-12-17 16:19:50,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:50,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940351a575bc060ce1574d2 HTTP/1.1" 200 None 2025-12-17 16:19:50,737 - lava_callback - DEBUG - Processing node 6940351a575bc060ce1574d2: ltp-fsx 2025-12-17 16:19:50,737 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940351a575bc060ce1574d2 2025-12-17 16:19:50,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:50,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940351a575bc060ce1574d2 HTTP/1.1" 200 None 2025-12-17 16:19:50,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:50,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:50,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:50,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940351a575bc060ce1574d2 HTTP/1.1" 200 None 2025-12-17 16:19:50,781 - lava_callback - INFO - Completed processing callback for node 6940351a575bc060ce1574d2 2025-12-17 16:19:51,435 - lava_callback - INFO - Received callback for node 6940351c575bc060ce1574d5 INFO: 172.21.0.1:36664 - "POST /node/6940351c575bc060ce1574d5 HTTP/1.0" 202 Accepted 2025-12-17 16:19:51,442 - lava_callback - INFO - Starting async job submit for node 6940351c575bc060ce1574d5 2025-12-17 16:19:51,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:51,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940351c575bc060ce1574d5 HTTP/1.1" 200 None 2025-12-17 16:19:51,451 - lava_callback - DEBUG - Processing node 6940351c575bc060ce1574d5: ltp-fs-bind 2025-12-17 16:19:51,451 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940351c575bc060ce1574d5 2025-12-17 16:19:51,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:51,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940351c575bc060ce1574d5 HTTP/1.1" 200 None 2025-12-17 16:19:51,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:51,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:51,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:51,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940351c575bc060ce1574d5 HTTP/1.1" 200 None 2025-12-17 16:19:51,485 - lava_callback - INFO - Completed processing callback for node 6940351c575bc060ce1574d5 2025-12-17 16:19:52,204 - lava_callback - INFO - Received callback for node 6940351d575bc060ce1574d8 INFO: 172.21.0.1:36674 - "POST /node/6940351d575bc060ce1574d8 HTTP/1.0" 202 Accepted 2025-12-17 16:19:52,212 - lava_callback - INFO - Starting async job submit for node 6940351d575bc060ce1574d8 2025-12-17 16:19:52,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:52,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940351d575bc060ce1574d8 HTTP/1.1" 200 None 2025-12-17 16:19:52,228 - lava_callback - DEBUG - Processing node 6940351d575bc060ce1574d8: ltp-fsx 2025-12-17 16:19:52,228 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940351d575bc060ce1574d8 2025-12-17 16:19:52,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:52,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940351d575bc060ce1574d8 HTTP/1.1" 200 None 2025-12-17 16:19:52,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:52,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:52,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:52,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940351d575bc060ce1574d8 HTTP/1.1" 200 None 2025-12-17 16:19:52,259 - lava_callback - INFO - Completed processing callback for node 6940351d575bc060ce1574d8 2025-12-17 16:19:52,924 - lava_callback - INFO - Received callback for node 6940351e575bc060ce1574db INFO: 172.21.0.1:36684 - "POST /node/6940351e575bc060ce1574db HTTP/1.0" 202 Accepted 2025-12-17 16:19:52,931 - lava_callback - INFO - Starting async job submit for node 6940351e575bc060ce1574db 2025-12-17 16:19:52,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:52,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940351e575bc060ce1574db HTTP/1.1" 200 None 2025-12-17 16:19:52,940 - lava_callback - DEBUG - Processing node 6940351e575bc060ce1574db: ltp-input 2025-12-17 16:19:52,940 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940351e575bc060ce1574db 2025-12-17 16:19:52,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:52,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940351e575bc060ce1574db HTTP/1.1" 200 None 2025-12-17 16:19:52,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:52,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:52,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:52,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940351e575bc060ce1574db HTTP/1.1" 200 None 2025-12-17 16:19:52,976 - lava_callback - INFO - Completed processing callback for node 6940351e575bc060ce1574db 2025-12-17 16:19:53,661 - lava_callback - INFO - Received callback for node 69403520575bc060ce1574e0 INFO: 172.21.0.1:36698 - "POST /node/69403520575bc060ce1574e0 HTTP/1.0" 202 Accepted 2025-12-17 16:19:53,667 - lava_callback - INFO - Starting async job submit for node 69403520575bc060ce1574e0 2025-12-17 16:19:53,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:53,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403520575bc060ce1574e0 HTTP/1.1" 200 None 2025-12-17 16:19:53,686 - lava_callback - DEBUG - Processing node 69403520575bc060ce1574e0: ltp-fsx 2025-12-17 16:19:53,686 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403520575bc060ce1574e0 2025-12-17 16:19:53,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:53,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403520575bc060ce1574e0 HTTP/1.1" 200 None 2025-12-17 16:19:53,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:53,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:53,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:53,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403520575bc060ce1574e0 HTTP/1.1" 200 None 2025-12-17 16:19:53,739 - lava_callback - INFO - Completed processing callback for node 69403520575bc060ce1574e0 2025-12-17 16:19:54,375 - lava_callback - INFO - Received callback for node 69403521575bc060ce1574e7 INFO: 172.21.0.1:36706 - "POST /node/69403521575bc060ce1574e7 HTTP/1.0" 202 Accepted 2025-12-17 16:19:54,382 - lava_callback - INFO - Starting async job submit for node 69403521575bc060ce1574e7 2025-12-17 16:19:54,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:54,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403521575bc060ce1574e7 HTTP/1.1" 200 None 2025-12-17 16:19:54,393 - lava_callback - DEBUG - Processing node 69403521575bc060ce1574e7: ltp-input 2025-12-17 16:19:54,393 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403521575bc060ce1574e7 2025-12-17 16:19:54,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:54,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403521575bc060ce1574e7 HTTP/1.1" 200 None 2025-12-17 16:19:54,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:54,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:54,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:54,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403521575bc060ce1574e7 HTTP/1.1" 200 None 2025-12-17 16:19:54,433 - lava_callback - INFO - Completed processing callback for node 69403521575bc060ce1574e7 2025-12-17 16:19:55,097 - lava_callback - INFO - Received callback for node 69403523575bc060ce1574f0 INFO: 172.21.0.1:36718 - "POST /node/69403523575bc060ce1574f0 HTTP/1.0" 202 Accepted 2025-12-17 16:19:55,108 - lava_callback - INFO - Starting async job submit for node 69403523575bc060ce1574f0 2025-12-17 16:19:55,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:55,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403523575bc060ce1574f0 HTTP/1.1" 200 None 2025-12-17 16:19:55,122 - lava_callback - DEBUG - Processing node 69403523575bc060ce1574f0: ltp-ipc 2025-12-17 16:19:55,122 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403523575bc060ce1574f0 2025-12-17 16:19:55,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:55,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403523575bc060ce1574f0 HTTP/1.1" 200 None 2025-12-17 16:19:55,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:55,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:55,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:55,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403523575bc060ce1574f0 HTTP/1.1" 200 None 2025-12-17 16:19:55,176 - lava_callback - INFO - Completed processing callback for node 69403523575bc060ce1574f0 2025-12-17 16:19:55,817 - lava_callback - INFO - Received callback for node 69403524575bc060ce1574f4 INFO: 172.21.0.1:36730 - "POST /node/69403524575bc060ce1574f4 HTTP/1.0" 202 Accepted 2025-12-17 16:19:55,826 - lava_callback - INFO - Starting async job submit for node 69403524575bc060ce1574f4 2025-12-17 16:19:55,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:55,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403524575bc060ce1574f4 HTTP/1.1" 200 None 2025-12-17 16:19:55,838 - lava_callback - DEBUG - Processing node 69403524575bc060ce1574f4: ltp-input 2025-12-17 16:19:55,838 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403524575bc060ce1574f4 2025-12-17 16:19:55,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:55,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403524575bc060ce1574f4 HTTP/1.1" 200 None 2025-12-17 16:19:55,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:55,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:55,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:55,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403524575bc060ce1574f4 HTTP/1.1" 200 None 2025-12-17 16:19:55,882 - lava_callback - INFO - Completed processing callback for node 69403524575bc060ce1574f4 INFO: 172.21.0.1:36732 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:19:56,592 - lava_callback - INFO - Received callback for node 69403525575bc060ce1574f9 INFO: 172.21.0.1:36736 - "POST /node/69403525575bc060ce1574f9 HTTP/1.0" 202 Accepted 2025-12-17 16:19:56,602 - lava_callback - INFO - Starting async job submit for node 69403525575bc060ce1574f9 2025-12-17 16:19:56,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:56,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403525575bc060ce1574f9 HTTP/1.1" 200 None 2025-12-17 16:19:56,618 - lava_callback - DEBUG - Processing node 69403525575bc060ce1574f9: ltp-ipc 2025-12-17 16:19:56,618 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403525575bc060ce1574f9 2025-12-17 16:19:56,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:56,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403525575bc060ce1574f9 HTTP/1.1" 200 None 2025-12-17 16:19:56,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:56,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:56,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:56,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403525575bc060ce1574f9 HTTP/1.1" 200 None 2025-12-17 16:19:56,659 - lava_callback - INFO - Completed processing callback for node 69403525575bc060ce1574f9 2025-12-17 16:19:57,337 - lava_callback - INFO - Received callback for node 69403526575bc060ce1574fe INFO: 172.21.0.1:36748 - "POST /node/69403526575bc060ce1574fe HTTP/1.0" 202 Accepted 2025-12-17 16:19:57,347 - lava_callback - INFO - Starting async job submit for node 69403526575bc060ce1574fe 2025-12-17 16:19:57,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:57,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403526575bc060ce1574fe HTTP/1.1" 200 None 2025-12-17 16:19:57,358 - lava_callback - DEBUG - Processing node 69403526575bc060ce1574fe: ltp-pty 2025-12-17 16:19:57,358 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403526575bc060ce1574fe 2025-12-17 16:19:57,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:57,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403526575bc060ce1574fe HTTP/1.1" 200 None 2025-12-17 16:19:57,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:57,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:57,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:57,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403526575bc060ce1574fe HTTP/1.1" 200 None 2025-12-17 16:19:57,406 - lava_callback - INFO - Completed processing callback for node 69403526575bc060ce1574fe 2025-12-17 16:19:58,062 - lava_callback - INFO - Received callback for node 69403528575bc060ce157501 INFO: 172.21.0.1:36752 - "POST /node/69403528575bc060ce157501 HTTP/1.0" 202 Accepted 2025-12-17 16:19:58,071 - lava_callback - INFO - Starting async job submit for node 69403528575bc060ce157501 2025-12-17 16:19:58,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:58,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403528575bc060ce157501 HTTP/1.1" 200 None 2025-12-17 16:19:58,081 - lava_callback - DEBUG - Processing node 69403528575bc060ce157501: ltp-ipc 2025-12-17 16:19:58,081 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403528575bc060ce157501 2025-12-17 16:19:58,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:58,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403528575bc060ce157501 HTTP/1.1" 200 None 2025-12-17 16:19:58,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:58,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:58,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:58,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403528575bc060ce157501 HTTP/1.1" 200 None 2025-12-17 16:19:58,124 - lava_callback - INFO - Completed processing callback for node 69403528575bc060ce157501 2025-12-17 16:19:59,256 - lava_callback - INFO - Received callback for node 69403529575bc060ce157504 INFO: 172.21.0.1:36768 - "POST /node/69403529575bc060ce157504 HTTP/1.0" 202 Accepted 2025-12-17 16:19:59,272 - lava_callback - INFO - Starting async job submit for node 69403529575bc060ce157504 2025-12-17 16:19:59,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:59,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403529575bc060ce157504 HTTP/1.1" 200 None 2025-12-17 16:19:59,296 - lava_callback - DEBUG - Processing node 69403529575bc060ce157504: ltp-sched 2025-12-17 16:19:59,296 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403529575bc060ce157504 2025-12-17 16:19:59,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:59,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403529575bc060ce157504 HTTP/1.1" 200 None 2025-12-17 16:19:59,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:59,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:19:59,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:19:59,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403529575bc060ce157504 HTTP/1.1" 200 None 2025-12-17 16:19:59,346 - lava_callback - INFO - Completed processing callback for node 69403529575bc060ce157504 2025-12-17 16:19:59,987 - lava_callback - INFO - Received callback for node 6940352a575bc060ce157507 INFO: 172.21.0.1:34498 - "POST /node/6940352a575bc060ce157507 HTTP/1.0" 202 Accepted 2025-12-17 16:19:59,996 - lava_callback - INFO - Starting async job submit for node 6940352a575bc060ce157507 2025-12-17 16:19:59,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:00,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940352a575bc060ce157507 HTTP/1.1" 200 None 2025-12-17 16:20:00,006 - lava_callback - DEBUG - Processing node 6940352a575bc060ce157507: ltp-pty 2025-12-17 16:20:00,006 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940352a575bc060ce157507 2025-12-17 16:20:00,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:00,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940352a575bc060ce157507 HTTP/1.1" 200 None 2025-12-17 16:20:00,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:00,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:00,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:00,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940352a575bc060ce157507 HTTP/1.1" 200 None 2025-12-17 16:20:00,087 - lava_callback - INFO - Completed processing callback for node 6940352a575bc060ce157507 2025-12-17 16:20:00,712 - lava_callback - INFO - Received callback for node 6940352b575bc060ce15750a INFO: 172.21.0.1:34508 - "POST /node/6940352b575bc060ce15750a HTTP/1.0" 202 Accepted 2025-12-17 16:20:00,723 - lava_callback - INFO - Starting async job submit for node 6940352b575bc060ce15750a 2025-12-17 16:20:00,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:00,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940352b575bc060ce15750a HTTP/1.1" 200 None 2025-12-17 16:20:00,734 - lava_callback - DEBUG - Processing node 6940352b575bc060ce15750a: ltp-pty 2025-12-17 16:20:00,734 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940352b575bc060ce15750a 2025-12-17 16:20:00,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:00,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940352b575bc060ce15750a HTTP/1.1" 200 None 2025-12-17 16:20:00,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:00,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:00,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:00,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940352b575bc060ce15750a HTTP/1.1" 200 None 2025-12-17 16:20:00,790 - lava_callback - INFO - Completed processing callback for node 6940352b575bc060ce15750a 2025-12-17 16:20:01,438 - lava_callback - INFO - Received callback for node 6940352d575bc060ce15750d INFO: 172.21.0.1:34512 - "POST /node/6940352d575bc060ce15750d HTTP/1.0" 202 Accepted 2025-12-17 16:20:01,444 - lava_callback - INFO - Starting async job submit for node 6940352d575bc060ce15750d 2025-12-17 16:20:01,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:01,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940352d575bc060ce15750d HTTP/1.1" 200 None 2025-12-17 16:20:01,454 - lava_callback - DEBUG - Processing node 6940352d575bc060ce15750d: ltp-smoketest 2025-12-17 16:20:01,454 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940352d575bc060ce15750d 2025-12-17 16:20:01,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:01,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940352d575bc060ce15750d HTTP/1.1" 200 None 2025-12-17 16:20:01,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:01,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:01,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:01,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940352d575bc060ce15750d HTTP/1.1" 200 None 2025-12-17 16:20:01,487 - lava_callback - INFO - Completed processing callback for node 6940352d575bc060ce15750d 2025-12-17 16:20:02,241 - lava_callback - INFO - Received callback for node 6940352e575bc060ce157510 INFO: 172.21.0.1:34528 - "POST /node/6940352e575bc060ce157510 HTTP/1.0" 202 Accepted 2025-12-17 16:20:02,252 - lava_callback - INFO - Starting async job submit for node 6940352e575bc060ce157510 2025-12-17 16:20:02,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:02,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940352e575bc060ce157510 HTTP/1.1" 200 None 2025-12-17 16:20:02,263 - lava_callback - DEBUG - Processing node 6940352e575bc060ce157510: ltp-sched 2025-12-17 16:20:02,263 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940352e575bc060ce157510 2025-12-17 16:20:02,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:02,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940352e575bc060ce157510 HTTP/1.1" 200 None 2025-12-17 16:20:02,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:02,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:02,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:02,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940352e575bc060ce157510 HTTP/1.1" 200 None 2025-12-17 16:20:02,310 - lava_callback - INFO - Completed processing callback for node 6940352e575bc060ce157510 2025-12-17 16:20:03,046 - lava_callback - INFO - Received callback for node 6940352f575bc060ce157513 INFO: 172.21.0.1:34530 - "POST /node/6940352f575bc060ce157513 HTTP/1.0" 202 Accepted 2025-12-17 16:20:03,056 - lava_callback - INFO - Starting async job submit for node 6940352f575bc060ce157513 2025-12-17 16:20:03,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:03,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940352f575bc060ce157513 HTTP/1.1" 200 None 2025-12-17 16:20:03,068 - lava_callback - DEBUG - Processing node 6940352f575bc060ce157513: ltp-sched 2025-12-17 16:20:03,068 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940352f575bc060ce157513 2025-12-17 16:20:03,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:03,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940352f575bc060ce157513 HTTP/1.1" 200 None 2025-12-17 16:20:03,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:03,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:03,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:03,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940352f575bc060ce157513 HTTP/1.1" 200 None 2025-12-17 16:20:03,117 - lava_callback - INFO - Completed processing callback for node 6940352f575bc060ce157513 2025-12-17 16:20:03,787 - lava_callback - INFO - Received callback for node 69403530575bc060ce157516 INFO: 172.21.0.1:34546 - "POST /node/69403530575bc060ce157516 HTTP/1.0" 202 Accepted 2025-12-17 16:20:03,797 - lava_callback - INFO - Starting async job submit for node 69403530575bc060ce157516 2025-12-17 16:20:03,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:03,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403530575bc060ce157516 HTTP/1.1" 200 None 2025-12-17 16:20:03,810 - lava_callback - DEBUG - Processing node 69403530575bc060ce157516: ltp-syscalls 2025-12-17 16:20:03,810 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403530575bc060ce157516 2025-12-17 16:20:03,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:03,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403530575bc060ce157516 HTTP/1.1" 200 None 2025-12-17 16:20:03,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:03,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:03,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:03,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403530575bc060ce157516 HTTP/1.1" 200 None 2025-12-17 16:20:03,858 - lava_callback - INFO - Completed processing callback for node 69403530575bc060ce157516 2025-12-17 16:20:04,533 - lava_callback - INFO - Received callback for node 69403531575bc060ce157519 INFO: 172.21.0.1:34548 - "POST /node/69403531575bc060ce157519 HTTP/1.0" 202 Accepted 2025-12-17 16:20:04,543 - lava_callback - INFO - Starting async job submit for node 69403531575bc060ce157519 2025-12-17 16:20:04,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:04,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403531575bc060ce157519 HTTP/1.1" 200 None 2025-12-17 16:20:04,556 - lava_callback - DEBUG - Processing node 69403531575bc060ce157519: ltp-smoketest 2025-12-17 16:20:04,556 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403531575bc060ce157519 2025-12-17 16:20:04,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:04,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403531575bc060ce157519 HTTP/1.1" 200 None 2025-12-17 16:20:04,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:04,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:04,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:04,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403531575bc060ce157519 HTTP/1.1" 200 None 2025-12-17 16:20:04,600 - lava_callback - INFO - Completed processing callback for node 69403531575bc060ce157519 2025-12-17 16:20:05,283 - lava_callback - INFO - Received callback for node 69403533575bc060ce15751c INFO: 172.21.0.1:34560 - "POST /node/69403533575bc060ce15751c HTTP/1.0" 202 Accepted 2025-12-17 16:20:05,290 - lava_callback - INFO - Starting async job submit for node 69403533575bc060ce15751c 2025-12-17 16:20:05,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:05,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403533575bc060ce15751c HTTP/1.1" 200 None 2025-12-17 16:20:05,299 - lava_callback - DEBUG - Processing node 69403533575bc060ce15751c: ltp-smoketest 2025-12-17 16:20:05,299 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403533575bc060ce15751c 2025-12-17 16:20:05,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:05,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403533575bc060ce15751c HTTP/1.1" 200 None 2025-12-17 16:20:05,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:05,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:05,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:05,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403533575bc060ce15751c HTTP/1.1" 200 None 2025-12-17 16:20:05,342 - lava_callback - INFO - Completed processing callback for node 69403533575bc060ce15751c 2025-12-17 16:20:06,008 - lava_callback - INFO - Received callback for node 69403534575bc060ce157521 INFO: 172.21.0.1:34576 - "POST /node/69403534575bc060ce157521 HTTP/1.0" 202 Accepted 2025-12-17 16:20:06,014 - lava_callback - INFO - Starting async job submit for node 69403534575bc060ce157521 2025-12-17 16:20:06,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:06,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403534575bc060ce157521 HTTP/1.1" 200 None 2025-12-17 16:20:06,023 - lava_callback - DEBUG - Processing node 69403534575bc060ce157521: ltp-syscalls-ipc 2025-12-17 16:20:06,024 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403534575bc060ce157521 2025-12-17 16:20:06,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:06,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403534575bc060ce157521 HTTP/1.1" 200 None 2025-12-17 16:20:06,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:06,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:06,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:06,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403534575bc060ce157521 HTTP/1.1" 200 None 2025-12-17 16:20:06,058 - lava_callback - INFO - Completed processing callback for node 69403534575bc060ce157521 INFO: 172.21.0.1:34588 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:20:08,162 - lava_callback - INFO - Received callback for node 69403536575bc060ce15752a INFO: 172.21.0.1:34596 - "POST /node/69403536575bc060ce15752a HTTP/1.0" 202 Accepted 2025-12-17 16:20:08,168 - lava_callback - INFO - Starting async job submit for node 69403536575bc060ce15752a 2025-12-17 16:20:08,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:08,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403536575bc060ce15752a HTTP/1.1" 200 None 2025-12-17 16:20:08,360 - lava_callback - DEBUG - Processing node 69403536575bc060ce15752a: ltp-syscalls 2025-12-17 16:20:08,361 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403536575bc060ce15752a 2025-12-17 16:20:08,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:08,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403536575bc060ce15752a HTTP/1.1" 200 None 2025-12-17 16:20:08,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:08,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:08,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:08,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403536575bc060ce15752a HTTP/1.1" 200 None 2025-12-17 16:20:08,398 - lava_callback - INFO - Completed processing callback for node 69403536575bc060ce15752a 2025-12-17 16:20:08,945 - lava_callback - INFO - Received callback for node 69403537575bc060ce157531 INFO: 172.21.0.1:34604 - "POST /node/69403537575bc060ce157531 HTTP/1.0" 202 Accepted 2025-12-17 16:20:08,951 - lava_callback - INFO - Starting async job submit for node 69403537575bc060ce157531 2025-12-17 16:20:08,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:08,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403537575bc060ce157531 HTTP/1.1" 200 None 2025-12-17 16:20:08,961 - lava_callback - DEBUG - Processing node 69403537575bc060ce157531: ltp-syscalls 2025-12-17 16:20:08,961 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403537575bc060ce157531 2025-12-17 16:20:08,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:08,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403537575bc060ce157531 HTTP/1.1" 200 None 2025-12-17 16:20:08,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:08,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:08,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:08,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403537575bc060ce157531 HTTP/1.1" 200 None 2025-12-17 16:20:08,997 - lava_callback - INFO - Completed processing callback for node 69403537575bc060ce157531 2025-12-17 16:20:09,707 - lava_callback - INFO - Received callback for node 69403538575bc060ce157534 INFO: 172.21.0.1:47372 - "POST /node/69403538575bc060ce157534 HTTP/1.0" 202 Accepted 2025-12-17 16:20:09,715 - lava_callback - INFO - Starting async job submit for node 69403538575bc060ce157534 2025-12-17 16:20:09,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:09,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403538575bc060ce157534 HTTP/1.1" 200 None 2025-12-17 16:20:09,725 - lava_callback - DEBUG - Processing node 69403538575bc060ce157534: ltp-timers 2025-12-17 16:20:09,725 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403538575bc060ce157534 2025-12-17 16:20:09,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:09,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403538575bc060ce157534 HTTP/1.1" 200 None 2025-12-17 16:20:09,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:09,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:09,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:09,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403538575bc060ce157534 HTTP/1.1" 200 None 2025-12-17 16:20:09,765 - lava_callback - INFO - Completed processing callback for node 69403538575bc060ce157534 INFO: 172.21.0.1:47376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47394 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:20:21,114 - lava_callback - INFO - Received callback for node 6940396a575bc060ce1580e9 INFO: 172.21.0.1:39440 - "POST /node/6940396a575bc060ce1580e9 HTTP/1.0" 202 Accepted 2025-12-17 16:20:21,123 - lava_callback - INFO - Starting async job submit for node 6940396a575bc060ce1580e9 2025-12-17 16:20:21,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:21,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940396a575bc060ce1580e9 HTTP/1.1" 200 None 2025-12-17 16:20:21,226 - lava_callback - DEBUG - Processing node 6940396a575bc060ce1580e9: baseline-nfs-arm64 2025-12-17 16:20:22,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:20:22,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:20:22,994 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6940396a575bc060ce1580e9/log.txt.gz 2025-12-17 16:20:22,995 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:20:23,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:20:23,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:20:23,206 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6940396a575bc060ce1580e9/lava_callback.json.gz DEBUG: Setup failed for 6940396a575bc060ce1580e9. Transitting job node result: incomplete -> fail 2025-12-17 16:20:23,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:23,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940396a575bc060ce1580e9 HTTP/1.1" 200 None 2025-12-17 16:20:23,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:23,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:20:23,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:23,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940396a575bc060ce1580e9 HTTP/1.1" 200 None 2025-12-17 16:20:23,403 - lava_callback - INFO - Completed processing callback for node 6940396a575bc060ce1580e9 INFO: 172.21.0.1:39450 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:20:27,600 - lava_callback - INFO - Received callback for node 69403539575bc060ce157539 INFO: 172.21.0.1:39456 - "POST /node/69403539575bc060ce157539 HTTP/1.0" 202 Accepted 2025-12-17 16:20:27,608 - lava_callback - INFO - Starting async job submit for node 69403539575bc060ce157539 2025-12-17 16:20:27,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:27,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403539575bc060ce157539 HTTP/1.1" 200 None 2025-12-17 16:20:27,617 - lava_callback - DEBUG - Processing node 69403539575bc060ce157539: ltp-syscalls-ipc 2025-12-17 16:20:27,617 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403539575bc060ce157539 2025-12-17 16:20:27,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:27,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403539575bc060ce157539 HTTP/1.1" 200 None 2025-12-17 16:20:27,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:27,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:27,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:27,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403539575bc060ce157539 HTTP/1.1" 200 None 2025-12-17 16:20:27,688 - lava_callback - INFO - Completed processing callback for node 69403539575bc060ce157539 2025-12-17 16:20:28,318 - lava_callback - INFO - Received callback for node 6940353b575bc060ce15753c INFO: 172.21.0.1:39462 - "POST /node/6940353b575bc060ce15753c HTTP/1.0" 202 Accepted 2025-12-17 16:20:28,326 - lava_callback - INFO - Starting async job submit for node 6940353b575bc060ce15753c 2025-12-17 16:20:28,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:28,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940353b575bc060ce15753c HTTP/1.1" 200 None 2025-12-17 16:20:28,336 - lava_callback - DEBUG - Processing node 6940353b575bc060ce15753c: ltp-syscalls-ipc 2025-12-17 16:20:28,336 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940353b575bc060ce15753c 2025-12-17 16:20:28,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:28,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940353b575bc060ce15753c HTTP/1.1" 200 None 2025-12-17 16:20:28,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:28,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:28,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:28,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940353b575bc060ce15753c HTTP/1.1" 200 None 2025-12-17 16:20:28,371 - lava_callback - INFO - Completed processing callback for node 6940353b575bc060ce15753c 2025-12-17 16:20:29,047 - lava_callback - INFO - Received callback for node 6940353c575bc060ce15753f INFO: 172.21.0.1:39464 - "POST /node/6940353c575bc060ce15753f HTTP/1.0" 202 Accepted 2025-12-17 16:20:29,064 - lava_callback - INFO - Starting async job submit for node 6940353c575bc060ce15753f 2025-12-17 16:20:29,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:29,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940353c575bc060ce15753f HTTP/1.1" 200 None 2025-12-17 16:20:29,082 - lava_callback - DEBUG - Processing node 6940353c575bc060ce15753f: ltp-watchqueue 2025-12-17 16:20:29,082 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940353c575bc060ce15753f 2025-12-17 16:20:29,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:29,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940353c575bc060ce15753f HTTP/1.1" 200 None 2025-12-17 16:20:29,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:29,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:29,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:29,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940353c575bc060ce15753f HTTP/1.1" 200 None 2025-12-17 16:20:29,122 - lava_callback - INFO - Completed processing callback for node 6940353c575bc060ce15753f 2025-12-17 16:20:29,806 - lava_callback - INFO - Received callback for node 6940353d575bc060ce157542 INFO: 172.21.0.1:52022 - "POST /node/6940353d575bc060ce157542 HTTP/1.0" 202 Accepted 2025-12-17 16:20:29,815 - lava_callback - INFO - Starting async job submit for node 6940353d575bc060ce157542 2025-12-17 16:20:29,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:29,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940353d575bc060ce157542 HTTP/1.1" 200 None 2025-12-17 16:20:29,824 - lava_callback - DEBUG - Processing node 6940353d575bc060ce157542: ltp-timers 2025-12-17 16:20:29,824 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940353d575bc060ce157542 2025-12-17 16:20:29,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:29,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940353d575bc060ce157542 HTTP/1.1" 200 None 2025-12-17 16:20:29,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:29,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:29,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:29,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940353d575bc060ce157542 HTTP/1.1" 200 None 2025-12-17 16:20:29,861 - lava_callback - INFO - Completed processing callback for node 6940353d575bc060ce157542 2025-12-17 16:20:30,531 - lava_callback - INFO - Received callback for node 6940353f575bc060ce157545 INFO: 172.21.0.1:52032 - "POST /node/6940353f575bc060ce157545 HTTP/1.0" 202 Accepted 2025-12-17 16:20:30,540 - lava_callback - INFO - Starting async job submit for node 6940353f575bc060ce157545 2025-12-17 16:20:30,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:30,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940353f575bc060ce157545 HTTP/1.1" 200 None 2025-12-17 16:20:30,556 - lava_callback - DEBUG - Processing node 6940353f575bc060ce157545: ltp-timers 2025-12-17 16:20:30,556 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940353f575bc060ce157545 2025-12-17 16:20:30,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:30,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940353f575bc060ce157545 HTTP/1.1" 200 None 2025-12-17 16:20:30,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:30,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:30,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:30,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940353f575bc060ce157545 HTTP/1.1" 200 None 2025-12-17 16:20:30,596 - lava_callback - INFO - Completed processing callback for node 6940353f575bc060ce157545 2025-12-17 16:20:31,336 - lava_callback - INFO - Received callback for node 69403540575bc060ce157548 INFO: 172.21.0.1:52040 - "POST /node/69403540575bc060ce157548 HTTP/1.0" 202 Accepted 2025-12-17 16:20:31,343 - lava_callback - INFO - Starting async job submit for node 69403540575bc060ce157548 2025-12-17 16:20:31,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:31,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403540575bc060ce157548 HTTP/1.1" 200 None 2025-12-17 16:20:31,352 - lava_callback - DEBUG - Processing node 69403540575bc060ce157548: ltp-watchqueue 2025-12-17 16:20:31,352 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403540575bc060ce157548 2025-12-17 16:20:31,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:31,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403540575bc060ce157548 HTTP/1.1" 200 None 2025-12-17 16:20:31,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:31,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:20:31,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:31,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403540575bc060ce157548 HTTP/1.1" 200 None 2025-12-17 16:20:31,387 - lava_callback - INFO - Completed processing callback for node 69403540575bc060ce157548 2025-12-17 16:20:32,134 - lava_callback - INFO - Received callback for node 69403541575bc060ce15754b INFO: 172.21.0.1:52044 - "POST /node/69403541575bc060ce15754b HTTP/1.0" 202 Accepted 2025-12-17 16:20:32,142 - lava_callback - INFO - Starting async job submit for node 69403541575bc060ce15754b 2025-12-17 16:20:32,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:32,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403541575bc060ce15754b HTTP/1.1" 200 None 2025-12-17 16:20:32,152 - lava_callback - DEBUG - Processing node 69403541575bc060ce15754b: ltp-syscalls 2025-12-17 16:20:32,152 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403541575bc060ce15754b 2025-12-17 16:20:32,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:32,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403541575bc060ce15754b HTTP/1.1" 200 None 2025-12-17 16:20:32,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:32,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:20:32,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:32,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403541575bc060ce15754b HTTP/1.1" 200 None 2025-12-17 16:20:32,191 - lava_callback - INFO - Completed processing callback for node 69403541575bc060ce15754b 2025-12-17 16:20:32,868 - lava_callback - INFO - Received callback for node 69403542575bc060ce15754e INFO: 172.21.0.1:52060 - "POST /node/69403542575bc060ce15754e HTTP/1.0" 202 Accepted 2025-12-17 16:20:32,875 - lava_callback - INFO - Starting async job submit for node 69403542575bc060ce15754e 2025-12-17 16:20:32,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:32,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403542575bc060ce15754e HTTP/1.1" 200 None 2025-12-17 16:20:32,887 - lava_callback - DEBUG - Processing node 69403542575bc060ce15754e: ltp-syscalls 2025-12-17 16:20:32,887 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403542575bc060ce15754e 2025-12-17 16:20:32,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:32,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403542575bc060ce15754e HTTP/1.1" 200 None 2025-12-17 16:20:32,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:32,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:20:32,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:32,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403542575bc060ce15754e HTTP/1.1" 200 None 2025-12-17 16:20:32,924 - lava_callback - INFO - Completed processing callback for node 69403542575bc060ce15754e 2025-12-17 16:20:33,608 - lava_callback - INFO - Received callback for node 69403544575bc060ce157551 INFO: 172.21.0.1:52068 - "POST /node/69403544575bc060ce157551 HTTP/1.0" 202 Accepted 2025-12-17 16:20:33,615 - lava_callback - INFO - Starting async job submit for node 69403544575bc060ce157551 2025-12-17 16:20:33,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:33,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403544575bc060ce157551 HTTP/1.1" 200 None 2025-12-17 16:20:33,624 - lava_callback - DEBUG - Processing node 69403544575bc060ce157551: ltp-syscalls 2025-12-17 16:20:33,624 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403544575bc060ce157551 2025-12-17 16:20:33,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:33,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403544575bc060ce157551 HTTP/1.1" 200 None 2025-12-17 16:20:33,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:33,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:20:33,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:33,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403544575bc060ce157551 HTTP/1.1" 200 None 2025-12-17 16:20:33,678 - lava_callback - INFO - Completed processing callback for node 69403544575bc060ce157551 2025-12-17 16:20:34,675 - lava_callback - INFO - Received callback for node 69403546575bc060ce157557 INFO: 172.21.0.1:52084 - "POST /node/69403546575bc060ce157557 HTTP/1.0" 202 Accepted 2025-12-17 16:20:34,681 - lava_callback - INFO - Starting async job submit for node 69403546575bc060ce157557 2025-12-17 16:20:34,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:34,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403546575bc060ce157557 HTTP/1.1" 200 None 2025-12-17 16:20:34,692 - lava_callback - DEBUG - Processing node 69403546575bc060ce157557: baseline-arm64-baylibre 2025-12-17 16:20:34,692 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403546575bc060ce157557 2025-12-17 16:20:34,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:34,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403546575bc060ce157557 HTTP/1.1" 200 None 2025-12-17 16:20:34,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:34,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:20:34,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:34,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403546575bc060ce157557 HTTP/1.1" 200 None 2025-12-17 16:20:34,731 - lava_callback - INFO - Completed processing callback for node 69403546575bc060ce157557 2025-12-17 16:20:35,367 - lava_callback - INFO - Received callback for node 69403549575bc060ce15755c INFO: 172.21.0.1:52096 - "POST /node/69403549575bc060ce15755c HTTP/1.0" 202 Accepted 2025-12-17 16:20:35,373 - lava_callback - INFO - Starting async job submit for node 69403549575bc060ce15755c 2025-12-17 16:20:35,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:35,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403549575bc060ce15755c HTTP/1.1" 200 None 2025-12-17 16:20:35,383 - lava_callback - DEBUG - Processing node 69403549575bc060ce15755c: baseline-arm64-baylibre 2025-12-17 16:20:35,383 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403549575bc060ce15755c 2025-12-17 16:20:35,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:35,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403549575bc060ce15755c HTTP/1.1" 200 None 2025-12-17 16:20:35,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:35,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:20:35,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:35,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403549575bc060ce15755c HTTP/1.1" 200 None 2025-12-17 16:20:35,418 - lava_callback - INFO - Completed processing callback for node 69403549575bc060ce15755c 2025-12-17 16:20:36,050 - lava_callback - INFO - Received callback for node 6940354b575bc060ce15755f INFO: 172.21.0.1:52112 - "POST /node/6940354b575bc060ce15755f HTTP/1.0" 202 Accepted 2025-12-17 16:20:36,057 - lava_callback - INFO - Starting async job submit for node 6940354b575bc060ce15755f 2025-12-17 16:20:36,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:36,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940354b575bc060ce15755f HTTP/1.1" 200 None 2025-12-17 16:20:36,066 - lava_callback - DEBUG - Processing node 6940354b575bc060ce15755f: baseline-arm64-baylibre 2025-12-17 16:20:36,066 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940354b575bc060ce15755f 2025-12-17 16:20:36,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:36,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940354b575bc060ce15755f HTTP/1.1" 200 None 2025-12-17 16:20:36,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:36,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:20:36,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:36,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940354b575bc060ce15755f HTTP/1.1" 200 None 2025-12-17 16:20:36,113 - lava_callback - INFO - Completed processing callback for node 6940354b575bc060ce15755f INFO: 172.21.0.1:52116 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:20:36,717 - lava_callback - INFO - Received callback for node 6940354d575bc060ce157564 INFO: 172.21.0.1:52130 - "POST /node/6940354d575bc060ce157564 HTTP/1.0" 202 Accepted 2025-12-17 16:20:36,723 - lava_callback - INFO - Starting async job submit for node 6940354d575bc060ce157564 2025-12-17 16:20:36,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:36,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940354d575bc060ce157564 HTTP/1.1" 200 None 2025-12-17 16:20:36,732 - lava_callback - DEBUG - Processing node 6940354d575bc060ce157564: baseline-arm64-baylibre 2025-12-17 16:20:36,732 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940354d575bc060ce157564 2025-12-17 16:20:36,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:36,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940354d575bc060ce157564 HTTP/1.1" 200 None 2025-12-17 16:20:36,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:36,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:20:36,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:36,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940354d575bc060ce157564 HTTP/1.1" 200 None 2025-12-17 16:20:36,764 - lava_callback - INFO - Completed processing callback for node 6940354d575bc060ce157564 2025-12-17 16:20:37,361 - lava_callback - INFO - Received callback for node 6940354f575bc060ce15756f INFO: 172.21.0.1:52144 - "POST /node/6940354f575bc060ce15756f HTTP/1.0" 202 Accepted 2025-12-17 16:20:37,368 - lava_callback - INFO - Starting async job submit for node 6940354f575bc060ce15756f 2025-12-17 16:20:37,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:37,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940354f575bc060ce15756f HTTP/1.1" 200 None 2025-12-17 16:20:37,377 - lava_callback - DEBUG - Processing node 6940354f575bc060ce15756f: baseline-arm64-baylibre 2025-12-17 16:20:37,378 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940354f575bc060ce15756f 2025-12-17 16:20:37,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:37,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940354f575bc060ce15756f HTTP/1.1" 200 None 2025-12-17 16:20:37,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:37,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:20:37,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:37,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940354f575bc060ce15756f HTTP/1.1" 200 None 2025-12-17 16:20:37,419 - lava_callback - INFO - Completed processing callback for node 6940354f575bc060ce15756f INFO: 172.21.0.1:35094 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:20:54,708 - lava_callback - INFO - Received callback for node 69403552575bc060ce157578 INFO: 172.21.0.1:47800 - "POST /node/69403552575bc060ce157578 HTTP/1.0" 202 Accepted 2025-12-17 16:20:54,714 - lava_callback - INFO - Starting async job submit for node 69403552575bc060ce157578 2025-12-17 16:20:54,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:54,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403552575bc060ce157578 HTTP/1.1" 200 None 2025-12-17 16:20:54,725 - lava_callback - DEBUG - Processing node 69403552575bc060ce157578: baseline-arm64-baylibre 2025-12-17 16:20:54,725 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403552575bc060ce157578 2025-12-17 16:20:54,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:54,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403552575bc060ce157578 HTTP/1.1" 200 None 2025-12-17 16:20:54,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:54,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:20:54,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:54,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403552575bc060ce157578 HTTP/1.1" 200 None 2025-12-17 16:20:54,757 - lava_callback - INFO - Completed processing callback for node 69403552575bc060ce157578 2025-12-17 16:20:55,364 - lava_callback - INFO - Received callback for node 69403554575bc060ce15757b INFO: 172.21.0.1:47812 - "POST /node/69403554575bc060ce15757b HTTP/1.0" 202 Accepted 2025-12-17 16:20:55,373 - lava_callback - INFO - Starting async job submit for node 69403554575bc060ce15757b 2025-12-17 16:20:55,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:55,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403554575bc060ce15757b HTTP/1.1" 200 None 2025-12-17 16:20:55,381 - lava_callback - DEBUG - Processing node 69403554575bc060ce15757b: baseline-arm64-baylibre 2025-12-17 16:20:55,381 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403554575bc060ce15757b 2025-12-17 16:20:55,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:55,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403554575bc060ce15757b HTTP/1.1" 200 None 2025-12-17 16:20:55,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:55,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:20:55,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:55,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403554575bc060ce15757b HTTP/1.1" 200 None 2025-12-17 16:20:55,431 - lava_callback - INFO - Completed processing callback for node 69403554575bc060ce15757b 2025-12-17 16:20:56,009 - lava_callback - INFO - Received callback for node 69403556575bc060ce15757e INFO: 172.21.0.1:47814 - "POST /node/69403556575bc060ce15757e HTTP/1.0" 202 Accepted 2025-12-17 16:20:56,015 - lava_callback - INFO - Starting async job submit for node 69403556575bc060ce15757e 2025-12-17 16:20:56,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:56,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403556575bc060ce15757e HTTP/1.1" 200 None 2025-12-17 16:20:56,024 - lava_callback - DEBUG - Processing node 69403556575bc060ce15757e: baseline-arm64-baylibre 2025-12-17 16:20:56,024 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403556575bc060ce15757e 2025-12-17 16:20:56,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:56,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403556575bc060ce15757e HTTP/1.1" 200 None 2025-12-17 16:20:56,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:56,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:20:56,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:56,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403556575bc060ce15757e HTTP/1.1" 200 None 2025-12-17 16:20:56,054 - lava_callback - INFO - Completed processing callback for node 69403556575bc060ce15757e INFO: 172.21.0.1:47820 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:20:56,676 - lava_callback - INFO - Received callback for node 69403558575bc060ce157581 INFO: 172.21.0.1:47836 - "POST /node/69403558575bc060ce157581 HTTP/1.0" 202 Accepted 2025-12-17 16:20:56,686 - lava_callback - INFO - Starting async job submit for node 69403558575bc060ce157581 2025-12-17 16:20:56,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:56,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403558575bc060ce157581 HTTP/1.1" 200 None 2025-12-17 16:20:56,698 - lava_callback - DEBUG - Processing node 69403558575bc060ce157581: baseline-arm64-baylibre 2025-12-17 16:20:56,698 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403558575bc060ce157581 2025-12-17 16:20:56,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:56,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403558575bc060ce157581 HTTP/1.1" 200 None 2025-12-17 16:20:56,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:56,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:20:56,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:56,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403558575bc060ce157581 HTTP/1.1" 200 None 2025-12-17 16:20:56,734 - lava_callback - INFO - Completed processing callback for node 69403558575bc060ce157581 2025-12-17 16:20:57,352 - lava_callback - INFO - Received callback for node 6940355a575bc060ce157584 INFO: 172.21.0.1:47838 - "POST /node/6940355a575bc060ce157584 HTTP/1.0" 202 Accepted 2025-12-17 16:20:57,358 - lava_callback - INFO - Starting async job submit for node 6940355a575bc060ce157584 2025-12-17 16:20:57,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:57,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940355a575bc060ce157584 HTTP/1.1" 200 None 2025-12-17 16:20:57,367 - lava_callback - DEBUG - Processing node 6940355a575bc060ce157584: baseline-arm64-baylibre 2025-12-17 16:20:57,368 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940355a575bc060ce157584 2025-12-17 16:20:57,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:57,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940355a575bc060ce157584 HTTP/1.1" 200 None 2025-12-17 16:20:57,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:57,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:20:57,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:57,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940355a575bc060ce157584 HTTP/1.1" 200 None 2025-12-17 16:20:57,405 - lava_callback - INFO - Completed processing callback for node 6940355a575bc060ce157584 2025-12-17 16:20:58,041 - lava_callback - INFO - Received callback for node 6940355c575bc060ce157587 INFO: 172.21.0.1:47852 - "POST /node/6940355c575bc060ce157587 HTTP/1.0" 202 Accepted 2025-12-17 16:20:58,048 - lava_callback - INFO - Starting async job submit for node 6940355c575bc060ce157587 2025-12-17 16:20:58,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:58,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940355c575bc060ce157587 HTTP/1.1" 200 None 2025-12-17 16:20:58,058 - lava_callback - DEBUG - Processing node 6940355c575bc060ce157587: baseline-arm64-baylibre 2025-12-17 16:20:58,058 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940355c575bc060ce157587 2025-12-17 16:20:58,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:58,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940355c575bc060ce157587 HTTP/1.1" 200 None 2025-12-17 16:20:58,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:58,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:20:58,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:58,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940355c575bc060ce157587 HTTP/1.1" 200 None 2025-12-17 16:20:58,098 - lava_callback - INFO - Completed processing callback for node 6940355c575bc060ce157587 2025-12-17 16:20:58,706 - lava_callback - INFO - Received callback for node 6940355e575bc060ce157590 INFO: 172.21.0.1:47860 - "POST /node/6940355e575bc060ce157590 HTTP/1.0" 202 Accepted 2025-12-17 16:20:58,717 - lava_callback - INFO - Starting async job submit for node 6940355e575bc060ce157590 2025-12-17 16:20:58,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:58,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940355e575bc060ce157590 HTTP/1.1" 200 None 2025-12-17 16:20:58,748 - lava_callback - DEBUG - Processing node 6940355e575bc060ce157590: baseline-arm64-baylibre 2025-12-17 16:20:58,748 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940355e575bc060ce157590 2025-12-17 16:20:58,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:58,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940355e575bc060ce157590 HTTP/1.1" 200 None 2025-12-17 16:20:58,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:58,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:20:58,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:58,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940355e575bc060ce157590 HTTP/1.1" 200 None 2025-12-17 16:20:58,782 - lava_callback - INFO - Completed processing callback for node 6940355e575bc060ce157590 2025-12-17 16:20:59,383 - lava_callback - INFO - Received callback for node 69403560575bc060ce157594 INFO: 172.21.0.1:47866 - "POST /node/69403560575bc060ce157594 HTTP/1.0" 202 Accepted 2025-12-17 16:20:59,390 - lava_callback - INFO - Starting async job submit for node 69403560575bc060ce157594 2025-12-17 16:20:59,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:59,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403560575bc060ce157594 HTTP/1.1" 200 None 2025-12-17 16:20:59,399 - lava_callback - DEBUG - Processing node 69403560575bc060ce157594: baseline-arm64-baylibre 2025-12-17 16:20:59,399 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403560575bc060ce157594 2025-12-17 16:20:59,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:59,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403560575bc060ce157594 HTTP/1.1" 200 None 2025-12-17 16:20:59,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:59,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:20:59,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:20:59,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403560575bc060ce157594 HTTP/1.1" 200 None 2025-12-17 16:20:59,439 - lava_callback - INFO - Completed processing callback for node 69403560575bc060ce157594 2025-12-17 16:21:00,053 - lava_callback - INFO - Received callback for node 69403562575bc060ce15759d INFO: 172.21.0.1:37364 - "POST /node/69403562575bc060ce15759d HTTP/1.0" 202 Accepted 2025-12-17 16:21:00,062 - lava_callback - INFO - Starting async job submit for node 69403562575bc060ce15759d 2025-12-17 16:21:00,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:00,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403562575bc060ce15759d HTTP/1.1" 200 None 2025-12-17 16:21:00,071 - lava_callback - DEBUG - Processing node 69403562575bc060ce15759d: baseline-arm64-baylibre 2025-12-17 16:21:00,072 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403562575bc060ce15759d 2025-12-17 16:21:00,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:00,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403562575bc060ce15759d HTTP/1.1" 200 None 2025-12-17 16:21:00,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:00,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:00,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:00,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403562575bc060ce15759d HTTP/1.1" 200 None 2025-12-17 16:21:00,139 - lava_callback - INFO - Completed processing callback for node 69403562575bc060ce15759d 2025-12-17 16:21:00,729 - lava_callback - INFO - Received callback for node 69403564575bc060ce1575a1 INFO: 172.21.0.1:37378 - "POST /node/69403564575bc060ce1575a1 HTTP/1.0" 202 Accepted 2025-12-17 16:21:00,736 - lava_callback - INFO - Starting async job submit for node 69403564575bc060ce1575a1 2025-12-17 16:21:00,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:00,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403564575bc060ce1575a1 HTTP/1.1" 200 None 2025-12-17 16:21:00,744 - lava_callback - DEBUG - Processing node 69403564575bc060ce1575a1: baseline-arm64-baylibre 2025-12-17 16:21:00,745 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403564575bc060ce1575a1 2025-12-17 16:21:00,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:00,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403564575bc060ce1575a1 HTTP/1.1" 200 None 2025-12-17 16:21:00,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:00,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:00,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:00,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403564575bc060ce1575a1 HTTP/1.1" 200 None 2025-12-17 16:21:00,784 - lava_callback - INFO - Completed processing callback for node 69403564575bc060ce1575a1 INFO: 172.21.0.1:37388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42690 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:21:14,678 - lava_callback - INFO - Received callback for node 69403566575bc060ce1575a4 INFO: 172.21.0.1:42706 - "POST /node/69403566575bc060ce1575a4 HTTP/1.0" 202 Accepted 2025-12-17 16:21:14,689 - lava_callback - INFO - Starting async job submit for node 69403566575bc060ce1575a4 2025-12-17 16:21:14,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:14,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403566575bc060ce1575a4 HTTP/1.1" 200 None 2025-12-17 16:21:14,717 - lava_callback - DEBUG - Processing node 69403566575bc060ce1575a4: baseline-arm64-baylibre 2025-12-17 16:21:14,717 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403566575bc060ce1575a4 2025-12-17 16:21:14,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:14,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403566575bc060ce1575a4 HTTP/1.1" 200 None 2025-12-17 16:21:14,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:14,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:14,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:14,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403566575bc060ce1575a4 HTTP/1.1" 200 None 2025-12-17 16:21:14,756 - lava_callback - INFO - Completed processing callback for node 69403566575bc060ce1575a4 INFO: 172.21.0.1:42716 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:21:15,351 - lava_callback - INFO - Received callback for node 69403568575bc060ce1575a8 INFO: 172.21.0.1:42720 - "POST /node/69403568575bc060ce1575a8 HTTP/1.0" 202 Accepted 2025-12-17 16:21:15,358 - lava_callback - INFO - Starting async job submit for node 69403568575bc060ce1575a8 2025-12-17 16:21:15,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:15,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403568575bc060ce1575a8 HTTP/1.1" 200 None 2025-12-17 16:21:15,368 - lava_callback - DEBUG - Processing node 69403568575bc060ce1575a8: baseline-arm64-baylibre 2025-12-17 16:21:15,368 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403568575bc060ce1575a8 2025-12-17 16:21:15,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:15,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403568575bc060ce1575a8 HTTP/1.1" 200 None 2025-12-17 16:21:15,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:15,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:15,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:15,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403568575bc060ce1575a8 HTTP/1.1" 200 None 2025-12-17 16:21:15,404 - lava_callback - INFO - Completed processing callback for node 69403568575bc060ce1575a8 2025-12-17 16:21:16,034 - lava_callback - INFO - Received callback for node 6940356a575bc060ce1575ab INFO: 172.21.0.1:42722 - "POST /node/6940356a575bc060ce1575ab HTTP/1.0" 202 Accepted 2025-12-17 16:21:16,043 - lava_callback - INFO - Starting async job submit for node 6940356a575bc060ce1575ab 2025-12-17 16:21:16,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:16,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940356a575bc060ce1575ab HTTP/1.1" 200 None 2025-12-17 16:21:16,061 - lava_callback - DEBUG - Processing node 6940356a575bc060ce1575ab: baseline-arm64-baylibre 2025-12-17 16:21:16,061 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940356a575bc060ce1575ab 2025-12-17 16:21:16,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:16,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940356a575bc060ce1575ab HTTP/1.1" 200 None 2025-12-17 16:21:16,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:16,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:16,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:16,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940356a575bc060ce1575ab HTTP/1.1" 200 None 2025-12-17 16:21:16,095 - lava_callback - INFO - Completed processing callback for node 6940356a575bc060ce1575ab INFO: 172.21.0.1:42730 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:21:16,684 - lava_callback - INFO - Received callback for node 6940356c575bc060ce1575ae INFO: 172.21.0.1:42746 - "POST /node/6940356c575bc060ce1575ae HTTP/1.0" 202 Accepted 2025-12-17 16:21:16,691 - lava_callback - INFO - Starting async job submit for node 6940356c575bc060ce1575ae 2025-12-17 16:21:16,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:16,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940356c575bc060ce1575ae HTTP/1.1" 200 None 2025-12-17 16:21:16,701 - lava_callback - DEBUG - Processing node 6940356c575bc060ce1575ae: baseline-arm64-baylibre 2025-12-17 16:21:16,701 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940356c575bc060ce1575ae 2025-12-17 16:21:16,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:16,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940356c575bc060ce1575ae HTTP/1.1" 200 None 2025-12-17 16:21:16,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:16,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:16,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:16,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940356c575bc060ce1575ae HTTP/1.1" 200 None 2025-12-17 16:21:16,740 - lava_callback - INFO - Completed processing callback for node 6940356c575bc060ce1575ae 2025-12-17 16:21:17,353 - lava_callback - INFO - Received callback for node 6940356e575bc060ce1575b1 INFO: 172.21.0.1:42762 - "POST /node/6940356e575bc060ce1575b1 HTTP/1.0" 202 Accepted 2025-12-17 16:21:17,360 - lava_callback - INFO - Starting async job submit for node 6940356e575bc060ce1575b1 2025-12-17 16:21:17,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:17,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940356e575bc060ce1575b1 HTTP/1.1" 200 None 2025-12-17 16:21:17,383 - lava_callback - DEBUG - Processing node 6940356e575bc060ce1575b1: baseline-arm64-baylibre 2025-12-17 16:21:17,383 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940356e575bc060ce1575b1 2025-12-17 16:21:17,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:17,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940356e575bc060ce1575b1 HTTP/1.1" 200 None 2025-12-17 16:21:17,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:17,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:17,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:17,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940356e575bc060ce1575b1 HTTP/1.1" 200 None 2025-12-17 16:21:17,437 - lava_callback - INFO - Completed processing callback for node 6940356e575bc060ce1575b1 2025-12-17 16:21:18,022 - lava_callback - INFO - Received callback for node 69403570575bc060ce1575b8 INFO: 172.21.0.1:42778 - "POST /node/69403570575bc060ce1575b8 HTTP/1.0" 202 Accepted 2025-12-17 16:21:18,028 - lava_callback - INFO - Starting async job submit for node 69403570575bc060ce1575b8 2025-12-17 16:21:18,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:18,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403570575bc060ce1575b8 HTTP/1.1" 200 None 2025-12-17 16:21:18,038 - lava_callback - DEBUG - Processing node 69403570575bc060ce1575b8: baseline-arm64-baylibre 2025-12-17 16:21:18,038 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403570575bc060ce1575b8 2025-12-17 16:21:18,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:18,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403570575bc060ce1575b8 HTTP/1.1" 200 None 2025-12-17 16:21:18,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:18,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:18,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:18,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403570575bc060ce1575b8 HTTP/1.1" 200 None 2025-12-17 16:21:18,078 - lava_callback - INFO - Completed processing callback for node 69403570575bc060ce1575b8 2025-12-17 16:21:18,677 - lava_callback - INFO - Received callback for node 69403572575bc060ce1575c3 INFO: 172.21.0.1:42784 - "POST /node/69403572575bc060ce1575c3 HTTP/1.0" 202 Accepted 2025-12-17 16:21:18,683 - lava_callback - INFO - Starting async job submit for node 69403572575bc060ce1575c3 2025-12-17 16:21:18,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:18,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403572575bc060ce1575c3 HTTP/1.1" 200 None 2025-12-17 16:21:18,696 - lava_callback - DEBUG - Processing node 69403572575bc060ce1575c3: baseline-arm64-baylibre 2025-12-17 16:21:18,696 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403572575bc060ce1575c3 2025-12-17 16:21:18,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:18,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403572575bc060ce1575c3 HTTP/1.1" 200 None 2025-12-17 16:21:18,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:18,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:18,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:18,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403572575bc060ce1575c3 HTTP/1.1" 200 None 2025-12-17 16:21:18,732 - lava_callback - INFO - Completed processing callback for node 69403572575bc060ce1575c3 2025-12-17 16:21:19,365 - lava_callback - INFO - Received callback for node 69403575575bc060ce1575ca INFO: 172.21.0.1:42792 - "POST /node/69403575575bc060ce1575ca HTTP/1.0" 202 Accepted 2025-12-17 16:21:19,371 - lava_callback - INFO - Starting async job submit for node 69403575575bc060ce1575ca 2025-12-17 16:21:19,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:19,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403575575bc060ce1575ca HTTP/1.1" 200 None 2025-12-17 16:21:19,381 - lava_callback - DEBUG - Processing node 69403575575bc060ce1575ca: baseline-arm64-baylibre 2025-12-17 16:21:19,381 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403575575bc060ce1575ca 2025-12-17 16:21:19,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:19,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403575575bc060ce1575ca HTTP/1.1" 200 None 2025-12-17 16:21:19,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:19,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:19,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:19,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403575575bc060ce1575ca HTTP/1.1" 200 None 2025-12-17 16:21:19,417 - lava_callback - INFO - Completed processing callback for node 69403575575bc060ce1575ca 2025-12-17 16:21:20,016 - lava_callback - INFO - Received callback for node 69403577575bc060ce1575cf INFO: 172.21.0.1:54148 - "POST /node/69403577575bc060ce1575cf HTTP/1.0" 202 Accepted 2025-12-17 16:21:20,022 - lava_callback - INFO - Starting async job submit for node 69403577575bc060ce1575cf 2025-12-17 16:21:20,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:20,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403577575bc060ce1575cf HTTP/1.1" 200 None 2025-12-17 16:21:20,031 - lava_callback - DEBUG - Processing node 69403577575bc060ce1575cf: baseline-arm64-baylibre 2025-12-17 16:21:20,031 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403577575bc060ce1575cf 2025-12-17 16:21:20,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:20,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403577575bc060ce1575cf HTTP/1.1" 200 None 2025-12-17 16:21:20,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:20,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:20,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:20,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403577575bc060ce1575cf HTTP/1.1" 200 None 2025-12-17 16:21:20,078 - lava_callback - INFO - Completed processing callback for node 69403577575bc060ce1575cf 2025-12-17 16:21:20,685 - lava_callback - INFO - Received callback for node 69403579575bc060ce1575d2 INFO: 172.21.0.1:54156 - "POST /node/69403579575bc060ce1575d2 HTTP/1.0" 202 Accepted 2025-12-17 16:21:20,694 - lava_callback - INFO - Starting async job submit for node 69403579575bc060ce1575d2 2025-12-17 16:21:20,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:20,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403579575bc060ce1575d2 HTTP/1.1" 200 None 2025-12-17 16:21:20,703 - lava_callback - DEBUG - Processing node 69403579575bc060ce1575d2: baseline-arm64-baylibre 2025-12-17 16:21:20,703 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403579575bc060ce1575d2 2025-12-17 16:21:20,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:20,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403579575bc060ce1575d2 HTTP/1.1" 200 None 2025-12-17 16:21:20,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:20,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:20,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:20,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403579575bc060ce1575d2 HTTP/1.1" 200 None 2025-12-17 16:21:20,744 - lava_callback - INFO - Completed processing callback for node 69403579575bc060ce1575d2 2025-12-17 16:21:23,380 - lava_callback - INFO - Received callback for node 694039ba575bc060ce1581f8 INFO: 172.21.0.1:54158 - "POST /node/694039ba575bc060ce1581f8 HTTP/1.0" 202 Accepted 2025-12-17 16:21:23,392 - lava_callback - INFO - Starting async job submit for node 694039ba575bc060ce1581f8 2025-12-17 16:21:23,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:23,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039ba575bc060ce1581f8 HTTP/1.1" 200 None 2025-12-17 16:21:23,480 - lava_callback - DEBUG - Processing node 694039ba575bc060ce1581f8: baseline-nfs-arm64 2025-12-17 16:21:24,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:21:25,297 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:21:25,298 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694039ba575bc060ce1581f8/log.txt.gz 2025-12-17 16:21:25,298 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:21:25,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:21:25,555 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:21:25,556 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694039ba575bc060ce1581f8/lava_callback.json.gz DEBUG: Setup failed for 694039ba575bc060ce1581f8. Transitting job node result: incomplete -> fail 2025-12-17 16:21:25,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:25,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039ba575bc060ce1581f8 HTTP/1.1" 200 None 2025-12-17 16:21:25,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:25,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693bd0faf552eae6fe201eb6 HTTP/1.1" 200 None 2025-12-17 16:21:25,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:25,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039ba575bc060ce1581f8 HTTP/1.1" 200 None 2025-12-17 16:21:25,766 - lava_callback - INFO - Completed processing callback for node 694039ba575bc060ce1581f8 INFO: 172.21.0.1:54160 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:21:29,335 - lava_callback - INFO - Received callback for node 69403580575bc060ce1575df INFO: 172.21.0.1:54170 - "POST /node/69403580575bc060ce1575df HTTP/1.0" 202 Accepted 2025-12-17 16:21:29,341 - lava_callback - INFO - Starting async job submit for node 69403580575bc060ce1575df 2025-12-17 16:21:29,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:29,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403580575bc060ce1575df HTTP/1.1" 200 None 2025-12-17 16:21:29,351 - lava_callback - DEBUG - Processing node 69403580575bc060ce1575df: baseline-arm64-clabbe 2025-12-17 16:21:29,351 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403580575bc060ce1575df 2025-12-17 16:21:29,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:29,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403580575bc060ce1575df HTTP/1.1" 200 None 2025-12-17 16:21:29,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:29,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:29,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:29,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403580575bc060ce1575df HTTP/1.1" 200 None 2025-12-17 16:21:29,386 - lava_callback - INFO - Completed processing callback for node 69403580575bc060ce1575df 2025-12-17 16:21:30,002 - lava_callback - INFO - Received callback for node 69403583575bc060ce1575e2 INFO: 172.21.0.1:49520 - "POST /node/69403583575bc060ce1575e2 HTTP/1.0" 202 Accepted 2025-12-17 16:21:30,009 - lava_callback - INFO - Starting async job submit for node 69403583575bc060ce1575e2 2025-12-17 16:21:30,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:30,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403583575bc060ce1575e2 HTTP/1.1" 200 None 2025-12-17 16:21:30,020 - lava_callback - DEBUG - Processing node 69403583575bc060ce1575e2: baseline-arm64-clabbe 2025-12-17 16:21:30,020 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403583575bc060ce1575e2 2025-12-17 16:21:30,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:30,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403583575bc060ce1575e2 HTTP/1.1" 200 None 2025-12-17 16:21:30,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:30,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:30,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:30,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403583575bc060ce1575e2 HTTP/1.1" 200 None 2025-12-17 16:21:30,059 - lava_callback - INFO - Completed processing callback for node 69403583575bc060ce1575e2 2025-12-17 16:21:30,668 - lava_callback - INFO - Received callback for node 69403587575bc060ce1575e8 INFO: 172.21.0.1:49526 - "POST /node/69403587575bc060ce1575e8 HTTP/1.0" 202 Accepted 2025-12-17 16:21:30,674 - lava_callback - INFO - Starting async job submit for node 69403587575bc060ce1575e8 2025-12-17 16:21:30,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:30,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403587575bc060ce1575e8 HTTP/1.1" 200 None 2025-12-17 16:21:30,684 - lava_callback - DEBUG - Processing node 69403587575bc060ce1575e8: baseline-arm64-clabbe 2025-12-17 16:21:30,684 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403587575bc060ce1575e8 2025-12-17 16:21:30,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:30,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403587575bc060ce1575e8 HTTP/1.1" 200 None 2025-12-17 16:21:30,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:30,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:30,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:30,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403587575bc060ce1575e8 HTTP/1.1" 200 None 2025-12-17 16:21:30,718 - lava_callback - INFO - Completed processing callback for node 69403587575bc060ce1575e8 2025-12-17 16:21:34,702 - lava_callback - INFO - Received callback for node 6940357b575bc060ce1575d9 INFO: 172.21.0.1:49538 - "POST /node/6940357b575bc060ce1575d9 HTTP/1.0" 202 Accepted 2025-12-17 16:21:34,709 - lava_callback - INFO - Starting async job submit for node 6940357b575bc060ce1575d9 2025-12-17 16:21:34,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:34,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940357b575bc060ce1575d9 HTTP/1.1" 200 None 2025-12-17 16:21:34,718 - lava_callback - DEBUG - Processing node 6940357b575bc060ce1575d9: baseline-arm64-baylibre 2025-12-17 16:21:34,718 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940357b575bc060ce1575d9 2025-12-17 16:21:34,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:34,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940357b575bc060ce1575d9 HTTP/1.1" 200 None 2025-12-17 16:21:34,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:34,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:34,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:34,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940357b575bc060ce1575d9 HTTP/1.1" 200 None 2025-12-17 16:21:34,753 - lava_callback - INFO - Completed processing callback for node 6940357b575bc060ce1575d9 2025-12-17 16:21:35,377 - lava_callback - INFO - Received callback for node 6940357d575bc060ce1575dc INFO: 172.21.0.1:49540 - "POST /node/6940357d575bc060ce1575dc HTTP/1.0" 202 Accepted 2025-12-17 16:21:35,383 - lava_callback - INFO - Starting async job submit for node 6940357d575bc060ce1575dc 2025-12-17 16:21:35,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:35,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940357d575bc060ce1575dc HTTP/1.1" 200 None 2025-12-17 16:21:35,394 - lava_callback - DEBUG - Processing node 6940357d575bc060ce1575dc: baseline-arm64-baylibre 2025-12-17 16:21:35,395 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940357d575bc060ce1575dc 2025-12-17 16:21:35,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:35,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940357d575bc060ce1575dc HTTP/1.1" 200 None 2025-12-17 16:21:35,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:35,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:35,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:35,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940357d575bc060ce1575dc HTTP/1.1" 200 None 2025-12-17 16:21:35,445 - lava_callback - INFO - Completed processing callback for node 6940357d575bc060ce1575dc INFO: 172.21.0.1:49542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60940 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:21:49,302 - lava_callback - INFO - Received callback for node 69403589575bc060ce1575ec INFO: 172.21.0.1:60942 - "POST /node/69403589575bc060ce1575ec HTTP/1.0" 202 Accepted 2025-12-17 16:21:49,308 - lava_callback - INFO - Starting async job submit for node 69403589575bc060ce1575ec 2025-12-17 16:21:49,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:49,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403589575bc060ce1575ec HTTP/1.1" 200 None 2025-12-17 16:21:49,320 - lava_callback - DEBUG - Processing node 69403589575bc060ce1575ec: baseline-arm64-clabbe 2025-12-17 16:21:49,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403589575bc060ce1575ec 2025-12-17 16:21:49,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:49,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403589575bc060ce1575ec HTTP/1.1" 200 None 2025-12-17 16:21:49,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:49,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:49,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:49,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403589575bc060ce1575ec HTTP/1.1" 200 None 2025-12-17 16:21:49,361 - lava_callback - INFO - Completed processing callback for node 69403589575bc060ce1575ec 2025-12-17 16:21:49,957 - lava_callback - INFO - Received callback for node 6940358b575bc060ce1575f4 INFO: 172.21.0.1:59952 - "POST /node/6940358b575bc060ce1575f4 HTTP/1.0" 202 Accepted 2025-12-17 16:21:49,965 - lava_callback - INFO - Starting async job submit for node 6940358b575bc060ce1575f4 2025-12-17 16:21:49,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:49,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940358b575bc060ce1575f4 HTTP/1.1" 200 None 2025-12-17 16:21:49,978 - lava_callback - DEBUG - Processing node 6940358b575bc060ce1575f4: baseline-arm64-clabbe 2025-12-17 16:21:49,978 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940358b575bc060ce1575f4 2025-12-17 16:21:49,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:49,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940358b575bc060ce1575f4 HTTP/1.1" 200 None 2025-12-17 16:21:49,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:50,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7fefb52d1dfff36c4c4 HTTP/1.1" 200 None 2025-12-17 16:21:50,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:50,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940358b575bc060ce1575f4 HTTP/1.1" 200 None 2025-12-17 16:21:50,024 - lava_callback - INFO - Completed processing callback for node 6940358b575bc060ce1575f4 2025-12-17 16:21:50,600 - lava_callback - INFO - Received callback for node 6940358c575bc060ce1575f7 INFO: 172.21.0.1:59966 - "POST /node/6940358c575bc060ce1575f7 HTTP/1.0" 202 Accepted 2025-12-17 16:21:50,605 - lava_callback - INFO - Starting async job submit for node 6940358c575bc060ce1575f7 2025-12-17 16:21:50,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:50,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940358c575bc060ce1575f7 HTTP/1.1" 200 None 2025-12-17 16:21:50,614 - lava_callback - DEBUG - Processing node 6940358c575bc060ce1575f7: baseline-arm-clabbe 2025-12-17 16:21:50,614 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940358c575bc060ce1575f7 2025-12-17 16:21:50,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:50,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940358c575bc060ce1575f7 HTTP/1.1" 200 None 2025-12-17 16:21:50,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:50,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad967fb52d1dfff36c7c1 HTTP/1.1" 200 None 2025-12-17 16:21:50,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:50,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940358c575bc060ce1575f7 HTTP/1.1" 200 None 2025-12-17 16:21:50,661 - lava_callback - INFO - Completed processing callback for node 6940358c575bc060ce1575f7 2025-12-17 16:21:51,253 - lava_callback - INFO - Received callback for node 6940358e575bc060ce1575fa INFO: 172.21.0.1:59978 - "POST /node/6940358e575bc060ce1575fa HTTP/1.0" 202 Accepted 2025-12-17 16:21:51,261 - lava_callback - INFO - Starting async job submit for node 6940358e575bc060ce1575fa 2025-12-17 16:21:51,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:51,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940358e575bc060ce1575fa HTTP/1.1" 200 None 2025-12-17 16:21:51,270 - lava_callback - DEBUG - Processing node 6940358e575bc060ce1575fa: baseline-arm-clabbe 2025-12-17 16:21:51,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940358e575bc060ce1575fa 2025-12-17 16:21:51,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:51,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940358e575bc060ce1575fa HTTP/1.1" 200 None 2025-12-17 16:21:51,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:51,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad967fb52d1dfff36c7c1 HTTP/1.1" 200 None 2025-12-17 16:21:51,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:51,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940358e575bc060ce1575fa HTTP/1.1" 200 None 2025-12-17 16:21:51,307 - lava_callback - INFO - Completed processing callback for node 6940358e575bc060ce1575fa 2025-12-17 16:21:51,939 - lava_callback - INFO - Received callback for node 69403593575bc060ce157601 INFO: 172.21.0.1:59984 - "POST /node/69403593575bc060ce157601 HTTP/1.0" 202 Accepted 2025-12-17 16:21:51,947 - lava_callback - INFO - Starting async job submit for node 69403593575bc060ce157601 2025-12-17 16:21:51,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:51,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403593575bc060ce157601 HTTP/1.1" 200 None 2025-12-17 16:21:51,960 - lava_callback - DEBUG - Processing node 69403593575bc060ce157601: baseline-arm-clabbe 2025-12-17 16:21:51,961 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403593575bc060ce157601 2025-12-17 16:21:51,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:51,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403593575bc060ce157601 HTTP/1.1" 200 None 2025-12-17 16:21:51,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:51,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad967fb52d1dfff36c7c1 HTTP/1.1" 200 None 2025-12-17 16:21:51,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:52,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403593575bc060ce157601 HTTP/1.1" 200 None 2025-12-17 16:21:52,011 - lava_callback - INFO - Completed processing callback for node 69403593575bc060ce157601 2025-12-17 16:21:52,597 - lava_callback - INFO - Received callback for node 69403594575bc060ce157604 INFO: 172.21.0.1:59994 - "POST /node/69403594575bc060ce157604 HTTP/1.0" 202 Accepted 2025-12-17 16:21:52,604 - lava_callback - INFO - Starting async job submit for node 69403594575bc060ce157604 2025-12-17 16:21:52,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:52,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403594575bc060ce157604 HTTP/1.1" 200 None 2025-12-17 16:21:52,618 - lava_callback - DEBUG - Processing node 69403594575bc060ce157604: baseline-arm-clabbe 2025-12-17 16:21:52,618 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403594575bc060ce157604 2025-12-17 16:21:52,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:52,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403594575bc060ce157604 HTTP/1.1" 200 None 2025-12-17 16:21:52,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:52,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad967fb52d1dfff36c7c1 HTTP/1.1" 200 None 2025-12-17 16:21:52,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:52,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403594575bc060ce157604 HTTP/1.1" 200 None 2025-12-17 16:21:52,673 - lava_callback - INFO - Completed processing callback for node 69403594575bc060ce157604 2025-12-17 16:21:53,273 - lava_callback - INFO - Received callback for node 69403596575bc060ce157607 INFO: 172.21.0.1:60008 - "POST /node/69403596575bc060ce157607 HTTP/1.0" 202 Accepted 2025-12-17 16:21:53,279 - lava_callback - INFO - Starting async job submit for node 69403596575bc060ce157607 2025-12-17 16:21:53,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:53,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403596575bc060ce157607 HTTP/1.1" 200 None 2025-12-17 16:21:53,288 - lava_callback - DEBUG - Processing node 69403596575bc060ce157607: baseline-arm-clabbe 2025-12-17 16:21:53,288 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403596575bc060ce157607 2025-12-17 16:21:53,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:53,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403596575bc060ce157607 HTTP/1.1" 200 None 2025-12-17 16:21:53,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:53,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad967fb52d1dfff36c7c1 HTTP/1.1" 200 None 2025-12-17 16:21:53,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:53,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403596575bc060ce157607 HTTP/1.1" 200 None 2025-12-17 16:21:53,327 - lava_callback - INFO - Completed processing callback for node 69403596575bc060ce157607 2025-12-17 16:21:53,949 - lava_callback - INFO - Received callback for node 69403597575bc060ce15760a INFO: 172.21.0.1:60018 - "POST /node/69403597575bc060ce15760a HTTP/1.0" 202 Accepted 2025-12-17 16:21:53,956 - lava_callback - INFO - Starting async job submit for node 69403597575bc060ce15760a 2025-12-17 16:21:53,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:53,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403597575bc060ce15760a HTTP/1.1" 200 None 2025-12-17 16:21:53,965 - lava_callback - DEBUG - Processing node 69403597575bc060ce15760a: baseline-arm-clabbe 2025-12-17 16:21:53,965 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403597575bc060ce15760a 2025-12-17 16:21:53,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:53,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403597575bc060ce15760a HTTP/1.1" 200 None 2025-12-17 16:21:53,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:53,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad967fb52d1dfff36c7c1 HTTP/1.1" 200 None 2025-12-17 16:21:53,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:54,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403597575bc060ce15760a HTTP/1.1" 200 None 2025-12-17 16:21:54,009 - lava_callback - INFO - Completed processing callback for node 69403597575bc060ce15760a 2025-12-17 16:21:54,612 - lava_callback - INFO - Received callback for node 69403598575bc060ce15760d INFO: 172.21.0.1:60026 - "POST /node/69403598575bc060ce15760d HTTP/1.0" 202 Accepted 2025-12-17 16:21:54,619 - lava_callback - INFO - Starting async job submit for node 69403598575bc060ce15760d 2025-12-17 16:21:54,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:54,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403598575bc060ce15760d HTTP/1.1" 200 None 2025-12-17 16:21:54,629 - lava_callback - DEBUG - Processing node 69403598575bc060ce15760d: baseline-arm-clabbe 2025-12-17 16:21:54,630 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403598575bc060ce15760d 2025-12-17 16:21:54,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:54,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403598575bc060ce15760d HTTP/1.1" 200 None 2025-12-17 16:21:54,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:54,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad967fb52d1dfff36c7c1 HTTP/1.1" 200 None 2025-12-17 16:21:54,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:54,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403598575bc060ce15760d HTTP/1.1" 200 None 2025-12-17 16:21:54,671 - lava_callback - INFO - Completed processing callback for node 69403598575bc060ce15760d INFO: 172.21.0.1:60028 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:21:56,550 - lava_callback - INFO - Received callback for node 694035a3575bc060ce157627 INFO: 172.21.0.1:60042 - "POST /node/694035a3575bc060ce157627 HTTP/1.0" 202 Accepted 2025-12-17 16:21:56,558 - lava_callback - INFO - Starting async job submit for node 694035a3575bc060ce157627 2025-12-17 16:21:56,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:56,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035a3575bc060ce157627 HTTP/1.1" 200 None 2025-12-17 16:21:56,569 - lava_callback - DEBUG - Processing node 694035a3575bc060ce157627: baseline-arm 2025-12-17 16:21:56,569 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035a3575bc060ce157627 2025-12-17 16:21:56,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:56,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035a3575bc060ce157627 HTTP/1.1" 200 None 2025-12-17 16:21:56,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:56,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:21:56,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:21:56,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035a3575bc060ce157627 HTTP/1.1" 200 None 2025-12-17 16:21:56,612 - lava_callback - INFO - Completed processing callback for node 694035a3575bc060ce157627 INFO: 172.21.0.1:51398 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:22:07,744 - lava_callback - INFO - Received callback for node 6940359a575bc060ce157610 INFO: 172.21.0.1:51412 - "POST /node/6940359a575bc060ce157610 HTTP/1.0" 202 Accepted 2025-12-17 16:22:07,750 - lava_callback - INFO - Starting async job submit for node 6940359a575bc060ce157610 2025-12-17 16:22:07,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:07,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940359a575bc060ce157610 HTTP/1.1" 200 None 2025-12-17 16:22:07,760 - lava_callback - DEBUG - Processing node 6940359a575bc060ce157610: ltp-syscalls 2025-12-17 16:22:07,760 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940359a575bc060ce157610 2025-12-17 16:22:07,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:07,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940359a575bc060ce157610 HTTP/1.1" 200 None 2025-12-17 16:22:07,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:07,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad967fb52d1dfff36c7c1 HTTP/1.1" 200 None 2025-12-17 16:22:07,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:07,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940359a575bc060ce157610 HTTP/1.1" 200 None 2025-12-17 16:22:07,794 - lava_callback - INFO - Completed processing callback for node 6940359a575bc060ce157610 2025-12-17 16:22:08,521 - lava_callback - INFO - Received callback for node 6940359f575bc060ce15761b INFO: 172.21.0.1:51416 - "POST /node/6940359f575bc060ce15761b HTTP/1.0" 202 Accepted 2025-12-17 16:22:08,529 - lava_callback - INFO - Starting async job submit for node 6940359f575bc060ce15761b 2025-12-17 16:22:08,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:08,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940359f575bc060ce15761b HTTP/1.1" 200 None 2025-12-17 16:22:08,539 - lava_callback - DEBUG - Processing node 6940359f575bc060ce15761b: ltp-syscalls 2025-12-17 16:22:08,539 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940359f575bc060ce15761b 2025-12-17 16:22:08,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:08,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940359f575bc060ce15761b HTTP/1.1" 200 None 2025-12-17 16:22:08,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:08,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad967fb52d1dfff36c7c1 HTTP/1.1" 200 None 2025-12-17 16:22:08,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:08,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940359f575bc060ce15761b HTTP/1.1" 200 None 2025-12-17 16:22:08,578 - lava_callback - INFO - Completed processing callback for node 6940359f575bc060ce15761b 2025-12-17 16:22:09,315 - lava_callback - INFO - Received callback for node 694035a0575bc060ce15761e INFO: 172.21.0.1:51428 - "POST /node/694035a0575bc060ce15761e HTTP/1.0" 202 Accepted 2025-12-17 16:22:09,322 - lava_callback - INFO - Starting async job submit for node 694035a0575bc060ce15761e 2025-12-17 16:22:09,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:09,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035a0575bc060ce15761e HTTP/1.1" 200 None 2025-12-17 16:22:09,333 - lava_callback - DEBUG - Processing node 694035a0575bc060ce15761e: baseline-arm-clabbe 2025-12-17 16:22:09,333 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035a0575bc060ce15761e 2025-12-17 16:22:09,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:09,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035a0575bc060ce15761e HTTP/1.1" 200 None 2025-12-17 16:22:09,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:09,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad967fb52d1dfff36c7c1 HTTP/1.1" 200 None 2025-12-17 16:22:09,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:09,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035a0575bc060ce15761e HTTP/1.1" 200 None 2025-12-17 16:22:09,383 - lava_callback - INFO - Completed processing callback for node 694035a0575bc060ce15761e 2025-12-17 16:22:09,979 - lava_callback - INFO - Received callback for node 694035a1575bc060ce157621 INFO: 172.21.0.1:52204 - "POST /node/694035a1575bc060ce157621 HTTP/1.0" 202 Accepted 2025-12-17 16:22:09,985 - lava_callback - INFO - Starting async job submit for node 694035a1575bc060ce157621 2025-12-17 16:22:09,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:09,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035a1575bc060ce157621 HTTP/1.1" 200 None 2025-12-17 16:22:09,995 - lava_callback - DEBUG - Processing node 694035a1575bc060ce157621: baseline-arm-clabbe 2025-12-17 16:22:09,995 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035a1575bc060ce157621 2025-12-17 16:22:09,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:10,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035a1575bc060ce157621 HTTP/1.1" 200 None 2025-12-17 16:22:10,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:10,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad967fb52d1dfff36c7c1 HTTP/1.1" 200 None 2025-12-17 16:22:10,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:10,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035a1575bc060ce157621 HTTP/1.1" 200 None 2025-12-17 16:22:10,033 - lava_callback - INFO - Completed processing callback for node 694035a1575bc060ce157621 INFO: 172.21.0.1:52210 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:22:10,686 - lava_callback - INFO - Received callback for node 694035a2575bc060ce157624 INFO: 172.21.0.1:52216 - "POST /node/694035a2575bc060ce157624 HTTP/1.0" 202 Accepted 2025-12-17 16:22:10,695 - lava_callback - INFO - Starting async job submit for node 694035a2575bc060ce157624 2025-12-17 16:22:10,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:10,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035a2575bc060ce157624 HTTP/1.1" 200 None 2025-12-17 16:22:10,709 - lava_callback - DEBUG - Processing node 694035a2575bc060ce157624: baseline-arm-clabbe 2025-12-17 16:22:10,709 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035a2575bc060ce157624 2025-12-17 16:22:10,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:10,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035a2575bc060ce157624 HTTP/1.1" 200 None 2025-12-17 16:22:10,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:10,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad967fb52d1dfff36c7c1 HTTP/1.1" 200 None 2025-12-17 16:22:10,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:10,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035a2575bc060ce157624 HTTP/1.1" 200 None 2025-12-17 16:22:10,744 - lava_callback - INFO - Completed processing callback for node 694035a2575bc060ce157624 2025-12-17 16:22:14,751 - lava_callback - INFO - Received callback for node 694035a5575bc060ce15762a INFO: 172.21.0.1:52230 - "POST /node/694035a5575bc060ce15762a HTTP/1.0" 202 Accepted 2025-12-17 16:22:14,758 - lava_callback - INFO - Starting async job submit for node 694035a5575bc060ce15762a 2025-12-17 16:22:14,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:14,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035a5575bc060ce15762a HTTP/1.1" 200 None 2025-12-17 16:22:14,771 - lava_callback - DEBUG - Processing node 694035a5575bc060ce15762a: baseline-arm64-baylibre 2025-12-17 16:22:14,771 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035a5575bc060ce15762a 2025-12-17 16:22:14,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:14,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035a5575bc060ce15762a HTTP/1.1" 200 None 2025-12-17 16:22:14,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:14,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:22:14,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:14,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035a5575bc060ce15762a HTTP/1.1" 200 None 2025-12-17 16:22:14,816 - lava_callback - INFO - Completed processing callback for node 694035a5575bc060ce15762a INFO: 172.21.0.1:52232 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:22:15,398 - lava_callback - INFO - Received callback for node 694035a7575bc060ce15762d INFO: 172.21.0.1:52238 - "POST /node/694035a7575bc060ce15762d HTTP/1.0" 202 Accepted 2025-12-17 16:22:15,404 - lava_callback - INFO - Starting async job submit for node 694035a7575bc060ce15762d 2025-12-17 16:22:15,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:15,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035a7575bc060ce15762d HTTP/1.1" 200 None 2025-12-17 16:22:15,425 - lava_callback - DEBUG - Processing node 694035a7575bc060ce15762d: baseline-arm64-baylibre 2025-12-17 16:22:15,425 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035a7575bc060ce15762d 2025-12-17 16:22:15,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:15,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035a7575bc060ce15762d HTTP/1.1" 200 None 2025-12-17 16:22:15,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:15,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:22:15,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:15,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035a7575bc060ce15762d HTTP/1.1" 200 None 2025-12-17 16:22:15,485 - lava_callback - INFO - Completed processing callback for node 694035a7575bc060ce15762d 2025-12-17 16:22:16,072 - lava_callback - INFO - Received callback for node 694035a9575bc060ce157634 INFO: 172.21.0.1:52242 - "POST /node/694035a9575bc060ce157634 HTTP/1.0" 202 Accepted 2025-12-17 16:22:16,077 - lava_callback - INFO - Starting async job submit for node 694035a9575bc060ce157634 2025-12-17 16:22:16,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:16,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035a9575bc060ce157634 HTTP/1.1" 200 None 2025-12-17 16:22:16,086 - lava_callback - DEBUG - Processing node 694035a9575bc060ce157634: baseline-arm64-baylibre 2025-12-17 16:22:16,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035a9575bc060ce157634 2025-12-17 16:22:16,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:16,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035a9575bc060ce157634 HTTP/1.1" 200 None 2025-12-17 16:22:16,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:16,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:22:16,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:16,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035a9575bc060ce157634 HTTP/1.1" 200 None 2025-12-17 16:22:16,125 - lava_callback - INFO - Completed processing callback for node 694035a9575bc060ce157634 INFO: 172.21.0.1:52254 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:22:16,736 - lava_callback - INFO - Received callback for node 694035ab575bc060ce157637 INFO: 172.21.0.1:52260 - "POST /node/694035ab575bc060ce157637 HTTP/1.0" 202 Accepted 2025-12-17 16:22:16,742 - lava_callback - INFO - Starting async job submit for node 694035ab575bc060ce157637 2025-12-17 16:22:16,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:16,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035ab575bc060ce157637 HTTP/1.1" 200 None 2025-12-17 16:22:16,753 - lava_callback - DEBUG - Processing node 694035ab575bc060ce157637: baseline-arm64-baylibre 2025-12-17 16:22:16,753 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035ab575bc060ce157637 2025-12-17 16:22:16,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:16,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035ab575bc060ce157637 HTTP/1.1" 200 None 2025-12-17 16:22:16,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:16,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:22:16,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:16,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035ab575bc060ce157637 HTTP/1.1" 200 None 2025-12-17 16:22:16,800 - lava_callback - INFO - Completed processing callback for node 694035ab575bc060ce157637 2025-12-17 16:22:17,386 - lava_callback - INFO - Received callback for node 694035ad575bc060ce15763a INFO: 172.21.0.1:52264 - "POST /node/694035ad575bc060ce15763a HTTP/1.0" 202 Accepted 2025-12-17 16:22:17,394 - lava_callback - INFO - Starting async job submit for node 694035ad575bc060ce15763a 2025-12-17 16:22:17,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:17,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035ad575bc060ce15763a HTTP/1.1" 200 None 2025-12-17 16:22:17,404 - lava_callback - DEBUG - Processing node 694035ad575bc060ce15763a: baseline-arm64-baylibre 2025-12-17 16:22:17,404 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035ad575bc060ce15763a 2025-12-17 16:22:17,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:17,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035ad575bc060ce15763a HTTP/1.1" 200 None 2025-12-17 16:22:17,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:17,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:22:17,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:17,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035ad575bc060ce15763a HTTP/1.1" 200 None 2025-12-17 16:22:17,452 - lava_callback - INFO - Completed processing callback for node 694035ad575bc060ce15763a 2025-12-17 16:22:18,062 - lava_callback - INFO - Received callback for node 694035af575bc060ce15763d INFO: 172.21.0.1:52276 - "POST /node/694035af575bc060ce15763d HTTP/1.0" 202 Accepted 2025-12-17 16:22:18,069 - lava_callback - INFO - Starting async job submit for node 694035af575bc060ce15763d 2025-12-17 16:22:18,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:18,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035af575bc060ce15763d HTTP/1.1" 200 None 2025-12-17 16:22:18,079 - lava_callback - DEBUG - Processing node 694035af575bc060ce15763d: baseline-arm64-baylibre 2025-12-17 16:22:18,079 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035af575bc060ce15763d 2025-12-17 16:22:18,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:18,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035af575bc060ce15763d HTTP/1.1" 200 None 2025-12-17 16:22:18,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:18,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:22:18,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:18,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035af575bc060ce15763d HTTP/1.1" 200 None 2025-12-17 16:22:18,116 - lava_callback - INFO - Completed processing callback for node 694035af575bc060ce15763d 2025-12-17 16:22:18,710 - lava_callback - INFO - Received callback for node 694035b1575bc060ce157640 INFO: 172.21.0.1:52278 - "POST /node/694035b1575bc060ce157640 HTTP/1.0" 202 Accepted 2025-12-17 16:22:18,719 - lava_callback - INFO - Starting async job submit for node 694035b1575bc060ce157640 2025-12-17 16:22:18,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:18,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035b1575bc060ce157640 HTTP/1.1" 200 None 2025-12-17 16:22:18,749 - lava_callback - DEBUG - Processing node 694035b1575bc060ce157640: baseline-arm64-baylibre 2025-12-17 16:22:18,749 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035b1575bc060ce157640 2025-12-17 16:22:18,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:18,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035b1575bc060ce157640 HTTP/1.1" 200 None 2025-12-17 16:22:18,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:18,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:22:18,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:18,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035b1575bc060ce157640 HTTP/1.1" 200 None 2025-12-17 16:22:18,796 - lava_callback - INFO - Completed processing callback for node 694035b1575bc060ce157640 2025-12-17 16:22:19,377 - lava_callback - INFO - Received callback for node 694035b3575bc060ce157643 INFO: 172.21.0.1:52290 - "POST /node/694035b3575bc060ce157643 HTTP/1.0" 202 Accepted 2025-12-17 16:22:19,384 - lava_callback - INFO - Starting async job submit for node 694035b3575bc060ce157643 2025-12-17 16:22:19,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:19,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035b3575bc060ce157643 HTTP/1.1" 200 None 2025-12-17 16:22:19,394 - lava_callback - DEBUG - Processing node 694035b3575bc060ce157643: baseline-arm64-baylibre 2025-12-17 16:22:19,394 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035b3575bc060ce157643 2025-12-17 16:22:19,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:19,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035b3575bc060ce157643 HTTP/1.1" 200 None 2025-12-17 16:22:19,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:19,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:22:19,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:19,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035b3575bc060ce157643 HTTP/1.1" 200 None 2025-12-17 16:22:19,433 - lava_callback - INFO - Completed processing callback for node 694035b3575bc060ce157643 INFO: 172.21.0.1:36312 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:22:34,742 - lava_callback - INFO - Received callback for node 694035c3575bc060ce1576bd INFO: 172.21.0.1:41974 - "POST /node/694035c3575bc060ce1576bd HTTP/1.0" 202 Accepted 2025-12-17 16:22:34,748 - lava_callback - INFO - Starting async job submit for node 694035c3575bc060ce1576bd 2025-12-17 16:22:34,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:34,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035c3575bc060ce1576bd HTTP/1.1" 200 None 2025-12-17 16:22:34,759 - lava_callback - DEBUG - Processing node 694035c3575bc060ce1576bd: baseline-arm64-baylibre 2025-12-17 16:22:34,759 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035c3575bc060ce1576bd 2025-12-17 16:22:34,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:34,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035c3575bc060ce1576bd HTTP/1.1" 200 None 2025-12-17 16:22:34,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:34,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:22:34,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:34,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035c3575bc060ce1576bd HTTP/1.1" 200 None 2025-12-17 16:22:34,806 - lava_callback - INFO - Completed processing callback for node 694035c3575bc060ce1576bd 2025-12-17 16:22:35,407 - lava_callback - INFO - Received callback for node 694035c5575bc060ce1576c2 INFO: 172.21.0.1:41984 - "POST /node/694035c5575bc060ce1576c2 HTTP/1.0" 202 Accepted 2025-12-17 16:22:35,414 - lava_callback - INFO - Starting async job submit for node 694035c5575bc060ce1576c2 2025-12-17 16:22:35,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:35,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035c5575bc060ce1576c2 HTTP/1.1" 200 None 2025-12-17 16:22:35,424 - lava_callback - DEBUG - Processing node 694035c5575bc060ce1576c2: baseline-arm64-baylibre 2025-12-17 16:22:35,424 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035c5575bc060ce1576c2 2025-12-17 16:22:35,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:35,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035c5575bc060ce1576c2 HTTP/1.1" 200 None 2025-12-17 16:22:35,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:35,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693fc41c575bc060ce148104 HTTP/1.1" 200 None 2025-12-17 16:22:35,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:35,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035c5575bc060ce1576c2 HTTP/1.1" 200 None 2025-12-17 16:22:35,474 - lava_callback - INFO - Completed processing callback for node 694035c5575bc060ce1576c2 2025-12-17 16:22:36,075 - lava_callback - INFO - Received callback for node 694035c7575bc060ce1576c5 INFO: 172.21.0.1:41992 - "POST /node/694035c7575bc060ce1576c5 HTTP/1.0" 202 Accepted 2025-12-17 16:22:36,085 - lava_callback - INFO - Starting async job submit for node 694035c7575bc060ce1576c5 2025-12-17 16:22:36,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:36,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035c7575bc060ce1576c5 HTTP/1.1" 200 None 2025-12-17 16:22:36,095 - lava_callback - DEBUG - Processing node 694035c7575bc060ce1576c5: baseline-arm64-baylibre 2025-12-17 16:22:36,095 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035c7575bc060ce1576c5 2025-12-17 16:22:36,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:36,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035c7575bc060ce1576c5 HTTP/1.1" 200 None 2025-12-17 16:22:36,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:36,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693fc41c575bc060ce148104 HTTP/1.1" 200 None 2025-12-17 16:22:36,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:36,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035c7575bc060ce1576c5 HTTP/1.1" 200 None 2025-12-17 16:22:36,145 - lava_callback - INFO - Completed processing callback for node 694035c7575bc060ce1576c5 INFO: 172.21.0.1:42002 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:22:36,716 - lava_callback - INFO - Received callback for node 694035c9575bc060ce1576c8 INFO: 172.21.0.1:42010 - "POST /node/694035c9575bc060ce1576c8 HTTP/1.0" 202 Accepted 2025-12-17 16:22:36,729 - lava_callback - INFO - Starting async job submit for node 694035c9575bc060ce1576c8 2025-12-17 16:22:36,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:36,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035c9575bc060ce1576c8 HTTP/1.1" 200 None 2025-12-17 16:22:36,751 - lava_callback - DEBUG - Processing node 694035c9575bc060ce1576c8: baseline-arm64-baylibre 2025-12-17 16:22:36,751 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035c9575bc060ce1576c8 2025-12-17 16:22:36,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:36,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035c9575bc060ce1576c8 HTTP/1.1" 200 None 2025-12-17 16:22:36,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:36,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693fc41c575bc060ce148104 HTTP/1.1" 200 None 2025-12-17 16:22:36,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:36,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035c9575bc060ce1576c8 HTTP/1.1" 200 None 2025-12-17 16:22:36,796 - lava_callback - INFO - Completed processing callback for node 694035c9575bc060ce1576c8 INFO: 172.21.0.1:58202 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:22:54,773 - lava_callback - INFO - Received callback for node 694035cb575bc060ce1576cb INFO: 172.21.0.1:46164 - "POST /node/694035cb575bc060ce1576cb HTTP/1.0" 202 Accepted 2025-12-17 16:22:54,781 - lava_callback - INFO - Starting async job submit for node 694035cb575bc060ce1576cb 2025-12-17 16:22:54,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:54,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035cb575bc060ce1576cb HTTP/1.1" 200 None 2025-12-17 16:22:54,792 - lava_callback - DEBUG - Processing node 694035cb575bc060ce1576cb: baseline-arm64-baylibre 2025-12-17 16:22:54,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035cb575bc060ce1576cb 2025-12-17 16:22:54,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:54,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035cb575bc060ce1576cb HTTP/1.1" 200 None 2025-12-17 16:22:54,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:54,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693fc41c575bc060ce148104 HTTP/1.1" 200 None 2025-12-17 16:22:54,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:54,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035cb575bc060ce1576cb HTTP/1.1" 200 None 2025-12-17 16:22:54,847 - lava_callback - INFO - Completed processing callback for node 694035cb575bc060ce1576cb 2025-12-17 16:22:55,445 - lava_callback - INFO - Received callback for node 694035cd575bc060ce1576ce INFO: 172.21.0.1:46174 - "POST /node/694035cd575bc060ce1576ce HTTP/1.0" 202 Accepted 2025-12-17 16:22:55,453 - lava_callback - INFO - Starting async job submit for node 694035cd575bc060ce1576ce 2025-12-17 16:22:55,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:55,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035cd575bc060ce1576ce HTTP/1.1" 200 None 2025-12-17 16:22:55,468 - lava_callback - DEBUG - Processing node 694035cd575bc060ce1576ce: baseline-arm64-baylibre 2025-12-17 16:22:55,469 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035cd575bc060ce1576ce 2025-12-17 16:22:55,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:55,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035cd575bc060ce1576ce HTTP/1.1" 200 None 2025-12-17 16:22:55,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:55,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693fc41c575bc060ce148104 HTTP/1.1" 200 None 2025-12-17 16:22:55,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:55,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035cd575bc060ce1576ce HTTP/1.1" 200 None 2025-12-17 16:22:55,520 - lava_callback - INFO - Completed processing callback for node 694035cd575bc060ce1576ce 2025-12-17 16:22:56,081 - lava_callback - INFO - Received callback for node 694035cf575bc060ce1576d1 INFO: 172.21.0.1:46190 - "POST /node/694035cf575bc060ce1576d1 HTTP/1.0" 202 Accepted 2025-12-17 16:22:56,089 - lava_callback - INFO - Starting async job submit for node 694035cf575bc060ce1576d1 2025-12-17 16:22:56,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:56,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035cf575bc060ce1576d1 HTTP/1.1" 200 None 2025-12-17 16:22:56,102 - lava_callback - DEBUG - Processing node 694035cf575bc060ce1576d1: baseline-arm64-baylibre 2025-12-17 16:22:56,102 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035cf575bc060ce1576d1 2025-12-17 16:22:56,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:56,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035cf575bc060ce1576d1 HTTP/1.1" 200 None 2025-12-17 16:22:56,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:56,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693fc41c575bc060ce148104 HTTP/1.1" 200 None 2025-12-17 16:22:56,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:56,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035cf575bc060ce1576d1 HTTP/1.1" 200 None 2025-12-17 16:22:56,163 - lava_callback - INFO - Completed processing callback for node 694035cf575bc060ce1576d1 INFO: 172.21.0.1:46206 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:22:56,754 - lava_callback - INFO - Received callback for node 694035d2575bc060ce1576d4 INFO: 172.21.0.1:46208 - "POST /node/694035d2575bc060ce1576d4 HTTP/1.0" 202 Accepted 2025-12-17 16:22:56,761 - lava_callback - INFO - Starting async job submit for node 694035d2575bc060ce1576d4 2025-12-17 16:22:56,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:56,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035d2575bc060ce1576d4 HTTP/1.1" 200 None 2025-12-17 16:22:56,776 - lava_callback - DEBUG - Processing node 694035d2575bc060ce1576d4: baseline-arm64-baylibre 2025-12-17 16:22:56,776 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035d2575bc060ce1576d4 2025-12-17 16:22:56,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:56,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035d2575bc060ce1576d4 HTTP/1.1" 200 None 2025-12-17 16:22:56,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:56,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693fc41c575bc060ce148104 HTTP/1.1" 200 None 2025-12-17 16:22:56,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:56,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035d2575bc060ce1576d4 HTTP/1.1" 200 None 2025-12-17 16:22:56,827 - lava_callback - INFO - Completed processing callback for node 694035d2575bc060ce1576d4 2025-12-17 16:22:57,424 - lava_callback - INFO - Received callback for node 694035d4575bc060ce1576d7 INFO: 172.21.0.1:46222 - "POST /node/694035d4575bc060ce1576d7 HTTP/1.0" 202 Accepted 2025-12-17 16:22:57,430 - lava_callback - INFO - Starting async job submit for node 694035d4575bc060ce1576d7 2025-12-17 16:22:57,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:57,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035d4575bc060ce1576d7 HTTP/1.1" 200 None 2025-12-17 16:22:57,440 - lava_callback - DEBUG - Processing node 694035d4575bc060ce1576d7: baseline-arm64-baylibre 2025-12-17 16:22:57,440 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035d4575bc060ce1576d7 2025-12-17 16:22:57,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:57,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035d4575bc060ce1576d7 HTTP/1.1" 200 None 2025-12-17 16:22:57,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:57,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693fc41c575bc060ce148104 HTTP/1.1" 200 None 2025-12-17 16:22:57,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:57,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035d4575bc060ce1576d7 HTTP/1.1" 200 None 2025-12-17 16:22:57,511 - lava_callback - INFO - Completed processing callback for node 694035d4575bc060ce1576d7 2025-12-17 16:22:58,064 - lava_callback - INFO - Received callback for node 694035d6575bc060ce1576db INFO: 172.21.0.1:46238 - "POST /node/694035d6575bc060ce1576db HTTP/1.0" 202 Accepted 2025-12-17 16:22:58,074 - lava_callback - INFO - Starting async job submit for node 694035d6575bc060ce1576db 2025-12-17 16:22:58,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:58,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035d6575bc060ce1576db HTTP/1.1" 200 None 2025-12-17 16:22:58,086 - lava_callback - DEBUG - Processing node 694035d6575bc060ce1576db: baseline-arm64-baylibre 2025-12-17 16:22:58,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035d6575bc060ce1576db 2025-12-17 16:22:58,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:58,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035d6575bc060ce1576db HTTP/1.1" 200 None 2025-12-17 16:22:58,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:58,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693fc41c575bc060ce148104 HTTP/1.1" 200 None 2025-12-17 16:22:58,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:22:58,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035d6575bc060ce1576db HTTP/1.1" 200 None 2025-12-17 16:22:58,129 - lava_callback - INFO - Completed processing callback for node 694035d6575bc060ce1576db INFO: 172.21.0.1:33946 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:23:09,375 - lava_callback - INFO - Received callback for node 694035e4575bc060ce1576fc INFO: 172.21.0.1:33960 - "POST /node/694035e4575bc060ce1576fc HTTP/1.0" 202 Accepted 2025-12-17 16:23:09,382 - lava_callback - INFO - Starting async job submit for node 694035e4575bc060ce1576fc 2025-12-17 16:23:09,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:09,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035e4575bc060ce1576fc HTTP/1.1" 200 None 2025-12-17 16:23:09,392 - lava_callback - DEBUG - Processing node 694035e4575bc060ce1576fc: baseline-arm64-clabbe 2025-12-17 16:23:09,393 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035e4575bc060ce1576fc 2025-12-17 16:23:09,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:09,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035e4575bc060ce1576fc HTTP/1.1" 200 None 2025-12-17 16:23:09,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:09,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693fc41c575bc060ce148104 HTTP/1.1" 200 None 2025-12-17 16:23:09,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:09,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035e4575bc060ce1576fc HTTP/1.1" 200 None 2025-12-17 16:23:09,431 - lava_callback - INFO - Completed processing callback for node 694035e4575bc060ce1576fc 2025-12-17 16:23:10,060 - lava_callback - INFO - Received callback for node 694035e7575bc060ce1576ff INFO: 172.21.0.1:50740 - "POST /node/694035e7575bc060ce1576ff HTTP/1.0" 202 Accepted 2025-12-17 16:23:10,067 - lava_callback - INFO - Starting async job submit for node 694035e7575bc060ce1576ff 2025-12-17 16:23:10,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:10,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035e7575bc060ce1576ff HTTP/1.1" 200 None 2025-12-17 16:23:10,076 - lava_callback - DEBUG - Processing node 694035e7575bc060ce1576ff: baseline-arm64-clabbe 2025-12-17 16:23:10,076 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035e7575bc060ce1576ff 2025-12-17 16:23:10,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:10,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035e7575bc060ce1576ff HTTP/1.1" 200 None 2025-12-17 16:23:10,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:10,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693fc41c575bc060ce148104 HTTP/1.1" 200 None 2025-12-17 16:23:10,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:10,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035e7575bc060ce1576ff HTTP/1.1" 200 None 2025-12-17 16:23:10,110 - lava_callback - INFO - Completed processing callback for node 694035e7575bc060ce1576ff INFO: 172.21.0.1:50746 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:23:10,728 - lava_callback - INFO - Received callback for node 694035e9575bc060ce157708 INFO: 172.21.0.1:50750 - "POST /node/694035e9575bc060ce157708 HTTP/1.0" 202 Accepted 2025-12-17 16:23:10,734 - lava_callback - INFO - Starting async job submit for node 694035e9575bc060ce157708 2025-12-17 16:23:10,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:10,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035e9575bc060ce157708 HTTP/1.1" 200 None 2025-12-17 16:23:10,747 - lava_callback - DEBUG - Processing node 694035e9575bc060ce157708: baseline-arm64-clabbe 2025-12-17 16:23:10,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035e9575bc060ce157708 2025-12-17 16:23:10,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:10,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035e9575bc060ce157708 HTTP/1.1" 200 None 2025-12-17 16:23:10,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:10,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693fc41c575bc060ce148104 HTTP/1.1" 200 None 2025-12-17 16:23:10,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:10,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035e9575bc060ce157708 HTTP/1.1" 200 None 2025-12-17 16:23:10,782 - lava_callback - INFO - Completed processing callback for node 694035e9575bc060ce157708 2025-12-17 16:23:11,373 - lava_callback - INFO - Received callback for node 694035eb575bc060ce157713 INFO: 172.21.0.1:50764 - "POST /node/694035eb575bc060ce157713 HTTP/1.0" 202 Accepted 2025-12-17 16:23:11,379 - lava_callback - INFO - Starting async job submit for node 694035eb575bc060ce157713 2025-12-17 16:23:11,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:11,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035eb575bc060ce157713 HTTP/1.1" 200 None 2025-12-17 16:23:11,388 - lava_callback - DEBUG - Processing node 694035eb575bc060ce157713: baseline-arm64-clabbe 2025-12-17 16:23:11,388 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035eb575bc060ce157713 2025-12-17 16:23:11,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:11,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035eb575bc060ce157713 HTTP/1.1" 200 None 2025-12-17 16:23:11,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:11,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693fc41c575bc060ce148104 HTTP/1.1" 200 None 2025-12-17 16:23:11,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:11,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035eb575bc060ce157713 HTTP/1.1" 200 None 2025-12-17 16:23:11,430 - lava_callback - INFO - Completed processing callback for node 694035eb575bc060ce157713 INFO: 172.21.0.1:50770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41382 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:23:29,384 - lava_callback - INFO - Received callback for node 694035ed575bc060ce15771c INFO: 172.21.0.1:41390 - "POST /node/694035ed575bc060ce15771c HTTP/1.0" 202 Accepted 2025-12-17 16:23:29,391 - lava_callback - INFO - Starting async job submit for node 694035ed575bc060ce15771c 2025-12-17 16:23:29,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:29,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035ed575bc060ce15771c HTTP/1.1" 200 None 2025-12-17 16:23:29,410 - lava_callback - DEBUG - Processing node 694035ed575bc060ce15771c: baseline-arm64-clabbe 2025-12-17 16:23:29,410 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035ed575bc060ce15771c 2025-12-17 16:23:29,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:29,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035ed575bc060ce15771c HTTP/1.1" 200 None 2025-12-17 16:23:29,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:29,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693fc41c575bc060ce148104 HTTP/1.1" 200 None 2025-12-17 16:23:29,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:29,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035ed575bc060ce15771c HTTP/1.1" 200 None 2025-12-17 16:23:29,478 - lava_callback - INFO - Completed processing callback for node 694035ed575bc060ce15771c 2025-12-17 16:23:30,069 - lava_callback - INFO - Received callback for node 694035ef575bc060ce157723 INFO: 172.21.0.1:58732 - "POST /node/694035ef575bc060ce157723 HTTP/1.0" 202 Accepted 2025-12-17 16:23:30,083 - lava_callback - INFO - Starting async job submit for node 694035ef575bc060ce157723 2025-12-17 16:23:30,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:30,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035ef575bc060ce157723 HTTP/1.1" 200 None 2025-12-17 16:23:30,102 - lava_callback - DEBUG - Processing node 694035ef575bc060ce157723: baseline-arm64-clabbe 2025-12-17 16:23:30,102 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035ef575bc060ce157723 2025-12-17 16:23:30,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:30,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035ef575bc060ce157723 HTTP/1.1" 200 None 2025-12-17 16:23:30,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:30,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:23:30,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:30,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035ef575bc060ce157723 HTTP/1.1" 200 None 2025-12-17 16:23:30,169 - lava_callback - INFO - Completed processing callback for node 694035ef575bc060ce157723 2025-12-17 16:23:30,731 - lava_callback - INFO - Received callback for node 694035f2575bc060ce157726 INFO: 172.21.0.1:58742 - "POST /node/694035f2575bc060ce157726 HTTP/1.0" 202 Accepted 2025-12-17 16:23:30,739 - lava_callback - INFO - Starting async job submit for node 694035f2575bc060ce157726 2025-12-17 16:23:30,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:30,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035f2575bc060ce157726 HTTP/1.1" 200 None 2025-12-17 16:23:30,748 - lava_callback - DEBUG - Processing node 694035f2575bc060ce157726: baseline-arm64-clabbe 2025-12-17 16:23:30,748 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035f2575bc060ce157726 2025-12-17 16:23:30,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:30,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035f2575bc060ce157726 HTTP/1.1" 200 None 2025-12-17 16:23:30,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:30,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:23:30,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:30,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035f2575bc060ce157726 HTTP/1.1" 200 None 2025-12-17 16:23:30,807 - lava_callback - INFO - Completed processing callback for node 694035f2575bc060ce157726 2025-12-17 16:23:31,388 - lava_callback - INFO - Received callback for node 694035f5575bc060ce15772c INFO: 172.21.0.1:58754 - "POST /node/694035f5575bc060ce15772c HTTP/1.0" 202 Accepted 2025-12-17 16:23:31,402 - lava_callback - INFO - Starting async job submit for node 694035f5575bc060ce15772c 2025-12-17 16:23:31,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:31,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035f5575bc060ce15772c HTTP/1.1" 200 None 2025-12-17 16:23:31,415 - lava_callback - DEBUG - Processing node 694035f5575bc060ce15772c: baseline-arm64-clabbe 2025-12-17 16:23:31,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035f5575bc060ce15772c 2025-12-17 16:23:31,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:31,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035f5575bc060ce15772c HTTP/1.1" 200 None 2025-12-17 16:23:31,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:31,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:23:31,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:31,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035f5575bc060ce15772c HTTP/1.1" 200 None 2025-12-17 16:23:31,472 - lava_callback - INFO - Completed processing callback for node 694035f5575bc060ce15772c 2025-12-17 16:23:31,783 - lava_callback - INFO - Received callback for node 6942d8e2e43d2b76e9021f33 INFO: 172.21.0.1:58758 - "POST /node/6942d8e2e43d2b76e9021f33 HTTP/1.0" 202 Accepted 2025-12-17 16:23:31,799 - lava_callback - INFO - Starting async job submit for node 6942d8e2e43d2b76e9021f33 2025-12-17 16:23:31,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:31,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d8e2e43d2b76e9021f33 HTTP/1.1" 200 None 2025-12-17 16:23:31,846 - lava_callback - DEBUG - Processing node 6942d8e2e43d2b76e9021f33: baseline-x86 2025-12-17 16:23:32,055 - lava_callback - INFO - Received callback for node 694035fb575bc060ce15772f INFO: 172.21.0.1:58768 - "POST /node/694035fb575bc060ce15772f HTTP/1.0" 202 Accepted 2025-12-17 16:23:32,064 - lava_callback - INFO - Starting async job submit for node 694035fb575bc060ce15772f 2025-12-17 16:23:32,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:32,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035fb575bc060ce15772f HTTP/1.1" 200 None 2025-12-17 16:23:32,094 - lava_callback - DEBUG - Processing node 694035fb575bc060ce15772f: baseline-arm64-clabbe 2025-12-17 16:23:32,094 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035fb575bc060ce15772f 2025-12-17 16:23:32,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:32,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035fb575bc060ce15772f HTTP/1.1" 200 None 2025-12-17 16:23:32,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:32,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:23:32,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:32,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:23:32,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035fb575bc060ce15772f HTTP/1.1" 200 None 2025-12-17 16:23:32,204 - lava_callback - INFO - Completed processing callback for node 694035fb575bc060ce15772f 2025-12-17 16:23:32,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:23:32,504 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-6942d8e2e43d2b76e9021f33/log.txt.gz 2025-12-17 16:23:32,505 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:23:32,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:23:32,717 - lava_callback - INFO - Received callback for node 694035fe575bc060ce157734 INFO: 172.21.0.1:58780 - "POST /node/694035fe575bc060ce157734 HTTP/1.0" 202 Accepted 2025-12-17 16:23:32,730 - lava_callback - INFO - Starting async job submit for node 694035fe575bc060ce157734 2025-12-17 16:23:32,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:32,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035fe575bc060ce157734 HTTP/1.1" 200 None 2025-12-17 16:23:32,747 - lava_callback - DEBUG - Processing node 694035fe575bc060ce157734: baseline-arm64-clabbe 2025-12-17 16:23:32,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035fe575bc060ce157734 2025-12-17 16:23:32,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:32,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035fe575bc060ce157734 HTTP/1.1" 200 None 2025-12-17 16:23:32,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:32,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:23:32,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:32,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035fe575bc060ce157734 HTTP/1.1" 200 None 2025-12-17 16:23:32,798 - lava_callback - INFO - Completed processing callback for node 694035fe575bc060ce157734 2025-12-17 16:23:33,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:23:33,046 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-6942d8e2e43d2b76e9021f33/lava_callback.json.gz 2025-12-17 16:23:33,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:33,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d8e2e43d2b76e9021f33 HTTP/1.1" 200 None 2025-12-17 16:23:33,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:33,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4da1b61f1c600601441 HTTP/1.1" 200 None 2025-12-17 16:23:33,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:33,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942d8e2e43d2b76e9021f33 HTTP/1.1" 200 None 2025-12-17 16:23:33,152 - lava_callback - INFO - Completed processing callback for node 6942d8e2e43d2b76e9021f33 2025-12-17 16:23:34,826 - lava_callback - INFO - Received callback for node 694035f3575bc060ce157729 INFO: 172.21.0.1:58786 - "POST /node/694035f3575bc060ce157729 HTTP/1.0" 202 Accepted 2025-12-17 16:23:34,836 - lava_callback - INFO - Starting async job submit for node 694035f3575bc060ce157729 2025-12-17 16:23:34,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:34,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035f3575bc060ce157729 HTTP/1.1" 200 None 2025-12-17 16:23:34,849 - lava_callback - DEBUG - Processing node 694035f3575bc060ce157729: baseline-arm-baylibre 2025-12-17 16:23:34,849 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694035f3575bc060ce157729 2025-12-17 16:23:34,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:34,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694035f3575bc060ce157729 HTTP/1.1" 200 None 2025-12-17 16:23:34,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:34,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:23:34,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:34,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694035f3575bc060ce157729 HTTP/1.1" 200 None 2025-12-17 16:23:34,899 - lava_callback - INFO - Completed processing callback for node 694035f3575bc060ce157729 2025-12-17 16:23:35,504 - lava_callback - INFO - Received callback for node 69403601575bc060ce15773b INFO: 172.21.0.1:58796 - "POST /node/69403601575bc060ce15773b HTTP/1.0" 202 Accepted 2025-12-17 16:23:35,512 - lava_callback - INFO - Starting async job submit for node 69403601575bc060ce15773b 2025-12-17 16:23:35,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:35,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403601575bc060ce15773b HTTP/1.1" 200 None 2025-12-17 16:23:35,523 - lava_callback - DEBUG - Processing node 69403601575bc060ce15773b: baseline-arm-baylibre 2025-12-17 16:23:35,523 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403601575bc060ce15773b 2025-12-17 16:23:35,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:35,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403601575bc060ce15773b HTTP/1.1" 200 None 2025-12-17 16:23:35,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:35,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:23:35,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:35,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403601575bc060ce15773b HTTP/1.1" 200 None 2025-12-17 16:23:35,565 - lava_callback - INFO - Completed processing callback for node 69403601575bc060ce15773b 2025-12-17 16:23:36,186 - lava_callback - INFO - Received callback for node 69403602575bc060ce157744 INFO: 172.21.0.1:58812 - "POST /node/69403602575bc060ce157744 HTTP/1.0" 202 Accepted 2025-12-17 16:23:36,194 - lava_callback - INFO - Starting async job submit for node 69403602575bc060ce157744 2025-12-17 16:23:36,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:36,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403602575bc060ce157744 HTTP/1.1" 200 None 2025-12-17 16:23:36,209 - lava_callback - DEBUG - Processing node 69403602575bc060ce157744: baseline-arm64-baylibre 2025-12-17 16:23:36,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403602575bc060ce157744 2025-12-17 16:23:36,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:36,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403602575bc060ce157744 HTTP/1.1" 200 None 2025-12-17 16:23:36,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:36,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:23:36,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:36,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403602575bc060ce157744 HTTP/1.1" 200 None 2025-12-17 16:23:36,245 - lava_callback - INFO - Completed processing callback for node 69403602575bc060ce157744 INFO: 172.21.0.1:58820 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:23:36,839 - lava_callback - INFO - Received callback for node 69403604575bc060ce15774b INFO: 172.21.0.1:58830 - "POST /node/69403604575bc060ce15774b HTTP/1.0" 202 Accepted 2025-12-17 16:23:36,847 - lava_callback - INFO - Starting async job submit for node 69403604575bc060ce15774b 2025-12-17 16:23:36,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:36,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403604575bc060ce15774b HTTP/1.1" 200 None 2025-12-17 16:23:36,857 - lava_callback - DEBUG - Processing node 69403604575bc060ce15774b: baseline-arm64-baylibre 2025-12-17 16:23:36,857 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403604575bc060ce15774b 2025-12-17 16:23:36,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:36,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403604575bc060ce15774b HTTP/1.1" 200 None 2025-12-17 16:23:36,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:36,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:23:36,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:36,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403604575bc060ce15774b HTTP/1.1" 200 None 2025-12-17 16:23:36,902 - lava_callback - INFO - Completed processing callback for node 69403604575bc060ce15774b 2025-12-17 16:23:43,047 - lava_callback - INFO - Received callback for node 6942d8e1e43d2b76e9021f23 INFO: 172.21.0.1:50622 - "POST /node/6942d8e1e43d2b76e9021f23 HTTP/1.0" 202 Accepted 2025-12-17 16:23:43,055 - lava_callback - INFO - Starting async job submit for node 6942d8e1e43d2b76e9021f23 2025-12-17 16:23:43,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:43,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d8e1e43d2b76e9021f23 HTTP/1.1" 200 None 2025-12-17 16:23:43,088 - lava_callback - DEBUG - Processing node 6942d8e1e43d2b76e9021f23: baseline-x86 2025-12-17 16:23:43,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:23:43,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:23:43,483 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-6942d8e1e43d2b76e9021f23/log.txt.gz 2025-12-17 16:23:43,483 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:23:43,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:23:43,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:23:43,616 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-6942d8e1e43d2b76e9021f23/lava_callback.json.gz 2025-12-17 16:23:43,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:43,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d8e1e43d2b76e9021f23 HTTP/1.1" 200 None 2025-12-17 16:23:43,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:43,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4da1b61f1c600601441 HTTP/1.1" 200 None 2025-12-17 16:23:43,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:43,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942d8e1e43d2b76e9021f23 HTTP/1.1" 200 None 2025-12-17 16:23:43,677 - lava_callback - INFO - Completed processing callback for node 6942d8e1e43d2b76e9021f23 INFO: 172.21.0.1:50632 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:23:47,702 - lava_callback - INFO - Received callback for node 69403600575bc060ce157738 INFO: 172.21.0.1:50646 - "POST /node/69403600575bc060ce157738 HTTP/1.0" 202 Accepted 2025-12-17 16:23:47,711 - lava_callback - INFO - Starting async job submit for node 69403600575bc060ce157738 2025-12-17 16:23:47,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:47,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403600575bc060ce157738 HTTP/1.1" 200 None 2025-12-17 16:23:47,724 - lava_callback - DEBUG - Processing node 69403600575bc060ce157738: ltp-sched 2025-12-17 16:23:47,724 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403600575bc060ce157738 2025-12-17 16:23:47,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:47,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403600575bc060ce157738 HTTP/1.1" 200 None 2025-12-17 16:23:47,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:47,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad967fb52d1dfff36c7c1 HTTP/1.1" 200 None 2025-12-17 16:23:47,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:47,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403600575bc060ce157738 HTTP/1.1" 200 None 2025-12-17 16:23:47,769 - lava_callback - INFO - Completed processing callback for node 69403600575bc060ce157738 2025-12-17 16:23:54,821 - lava_callback - INFO - Received callback for node 69403606575bc060ce15774e INFO: 172.21.0.1:56680 - "POST /node/69403606575bc060ce15774e HTTP/1.0" 202 Accepted 2025-12-17 16:23:54,829 - lava_callback - INFO - Starting async job submit for node 69403606575bc060ce15774e 2025-12-17 16:23:54,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:54,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403606575bc060ce15774e HTTP/1.1" 200 None 2025-12-17 16:23:54,841 - lava_callback - DEBUG - Processing node 69403606575bc060ce15774e: baseline-arm64-baylibre 2025-12-17 16:23:54,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403606575bc060ce15774e 2025-12-17 16:23:54,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:54,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403606575bc060ce15774e HTTP/1.1" 200 None 2025-12-17 16:23:54,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:54,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:23:54,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:54,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403606575bc060ce15774e HTTP/1.1" 200 None 2025-12-17 16:23:54,877 - lava_callback - INFO - Completed processing callback for node 69403606575bc060ce15774e 2025-12-17 16:23:55,475 - lava_callback - INFO - Received callback for node 69403608575bc060ce157751 INFO: 172.21.0.1:56682 - "POST /node/69403608575bc060ce157751 HTTP/1.0" 202 Accepted 2025-12-17 16:23:55,481 - lava_callback - INFO - Starting async job submit for node 69403608575bc060ce157751 2025-12-17 16:23:55,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:55,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403608575bc060ce157751 HTTP/1.1" 200 None 2025-12-17 16:23:55,510 - lava_callback - DEBUG - Processing node 69403608575bc060ce157751: baseline-arm-baylibre 2025-12-17 16:23:55,510 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403608575bc060ce157751 2025-12-17 16:23:55,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:55,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403608575bc060ce157751 HTTP/1.1" 200 None 2025-12-17 16:23:55,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:55,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8c4fb52d1dfff36c6b1 HTTP/1.1" 200 None 2025-12-17 16:23:55,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:55,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403608575bc060ce157751 HTTP/1.1" 200 None 2025-12-17 16:23:55,542 - lava_callback - INFO - Completed processing callback for node 69403608575bc060ce157751 2025-12-17 16:23:56,132 - lava_callback - INFO - Received callback for node 6940360a575bc060ce157754 INFO: 172.21.0.1:56696 - "POST /node/6940360a575bc060ce157754 HTTP/1.0" 202 Accepted 2025-12-17 16:23:56,138 - lava_callback - INFO - Starting async job submit for node 6940360a575bc060ce157754 2025-12-17 16:23:56,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:56,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940360a575bc060ce157754 HTTP/1.1" 200 None 2025-12-17 16:23:56,148 - lava_callback - DEBUG - Processing node 6940360a575bc060ce157754: baseline-arm64-baylibre 2025-12-17 16:23:56,148 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940360a575bc060ce157754 2025-12-17 16:23:56,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:56,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940360a575bc060ce157754 HTTP/1.1" 200 None 2025-12-17 16:23:56,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:56,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:23:56,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:56,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940360a575bc060ce157754 HTTP/1.1" 200 None 2025-12-17 16:23:56,186 - lava_callback - INFO - Completed processing callback for node 6940360a575bc060ce157754 INFO: 172.21.0.1:56702 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:23:56,799 - lava_callback - INFO - Received callback for node 6940360b575bc060ce157757 INFO: 172.21.0.1:56710 - "POST /node/6940360b575bc060ce157757 HTTP/1.0" 202 Accepted 2025-12-17 16:23:56,806 - lava_callback - INFO - Starting async job submit for node 6940360b575bc060ce157757 2025-12-17 16:23:56,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:56,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940360b575bc060ce157757 HTTP/1.1" 200 None 2025-12-17 16:23:56,815 - lava_callback - DEBUG - Processing node 6940360b575bc060ce157757: baseline-arm64-baylibre 2025-12-17 16:23:56,815 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940360b575bc060ce157757 2025-12-17 16:23:56,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:56,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940360b575bc060ce157757 HTTP/1.1" 200 None 2025-12-17 16:23:56,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:56,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:23:56,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:56,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940360b575bc060ce157757 HTTP/1.1" 200 None 2025-12-17 16:23:56,907 - lava_callback - INFO - Completed processing callback for node 6940360b575bc060ce157757 2025-12-17 16:23:57,456 - lava_callback - INFO - Received callback for node 6940360d575bc060ce15775a INFO: 172.21.0.1:56724 - "POST /node/6940360d575bc060ce15775a HTTP/1.0" 202 Accepted 2025-12-17 16:23:57,463 - lava_callback - INFO - Starting async job submit for node 6940360d575bc060ce15775a 2025-12-17 16:23:57,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:57,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940360d575bc060ce15775a HTTP/1.1" 200 None 2025-12-17 16:23:57,472 - lava_callback - DEBUG - Processing node 6940360d575bc060ce15775a: baseline-arm64-baylibre 2025-12-17 16:23:57,472 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940360d575bc060ce15775a 2025-12-17 16:23:57,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:57,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940360d575bc060ce15775a HTTP/1.1" 200 None 2025-12-17 16:23:57,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:57,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:23:57,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:57,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940360d575bc060ce15775a HTTP/1.1" 200 None 2025-12-17 16:23:57,507 - lava_callback - INFO - Completed processing callback for node 6940360d575bc060ce15775a 2025-12-17 16:23:58,144 - lava_callback - INFO - Received callback for node 6940360f575bc060ce15775d INFO: 172.21.0.1:56740 - "POST /node/6940360f575bc060ce15775d HTTP/1.0" 202 Accepted 2025-12-17 16:23:58,151 - lava_callback - INFO - Starting async job submit for node 6940360f575bc060ce15775d 2025-12-17 16:23:58,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:58,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940360f575bc060ce15775d HTTP/1.1" 200 None 2025-12-17 16:23:58,164 - lava_callback - DEBUG - Processing node 6940360f575bc060ce15775d: baseline-arm64-baylibre 2025-12-17 16:23:58,164 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940360f575bc060ce15775d 2025-12-17 16:23:58,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:58,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940360f575bc060ce15775d HTTP/1.1" 200 None 2025-12-17 16:23:58,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:58,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:23:58,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:58,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940360f575bc060ce15775d HTTP/1.1" 200 None 2025-12-17 16:23:58,209 - lava_callback - INFO - Completed processing callback for node 6940360f575bc060ce15775d 2025-12-17 16:23:58,806 - lava_callback - INFO - Received callback for node 69403612575bc060ce157766 INFO: 172.21.0.1:56750 - "POST /node/69403612575bc060ce157766 HTTP/1.0" 202 Accepted 2025-12-17 16:23:58,812 - lava_callback - INFO - Starting async job submit for node 69403612575bc060ce157766 2025-12-17 16:23:58,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:58,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403612575bc060ce157766 HTTP/1.1" 200 None 2025-12-17 16:23:58,821 - lava_callback - DEBUG - Processing node 69403612575bc060ce157766: baseline-arm64-baylibre 2025-12-17 16:23:58,821 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403612575bc060ce157766 2025-12-17 16:23:58,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:58,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403612575bc060ce157766 HTTP/1.1" 200 None 2025-12-17 16:23:58,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:58,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:23:58,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:58,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403612575bc060ce157766 HTTP/1.1" 200 None 2025-12-17 16:23:58,852 - lava_callback - INFO - Completed processing callback for node 69403612575bc060ce157766 2025-12-17 16:23:59,450 - lava_callback - INFO - Received callback for node 69403618575bc060ce157776 INFO: 172.21.0.1:56754 - "POST /node/69403618575bc060ce157776 HTTP/1.0" 202 Accepted 2025-12-17 16:23:59,456 - lava_callback - INFO - Starting async job submit for node 69403618575bc060ce157776 2025-12-17 16:23:59,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:59,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403618575bc060ce157776 HTTP/1.1" 200 None 2025-12-17 16:23:59,466 - lava_callback - DEBUG - Processing node 69403618575bc060ce157776: baseline-arm64-baylibre 2025-12-17 16:23:59,466 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403618575bc060ce157776 2025-12-17 16:23:59,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:59,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403618575bc060ce157776 HTTP/1.1" 200 None 2025-12-17 16:23:59,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:59,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:23:59,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:23:59,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403618575bc060ce157776 HTTP/1.1" 200 None 2025-12-17 16:23:59,506 - lava_callback - INFO - Completed processing callback for node 69403618575bc060ce157776 INFO: 172.21.0.1:47684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44400 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:24:14,836 - lava_callback - INFO - Received callback for node 6940361b575bc060ce157779 INFO: 172.21.0.1:44410 - "POST /node/6940361b575bc060ce157779 HTTP/1.0" 202 Accepted 2025-12-17 16:24:14,843 - lava_callback - INFO - Starting async job submit for node 6940361b575bc060ce157779 2025-12-17 16:24:14,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:14,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940361b575bc060ce157779 HTTP/1.1" 200 None 2025-12-17 16:24:14,852 - lava_callback - DEBUG - Processing node 6940361b575bc060ce157779: baseline-arm64-baylibre 2025-12-17 16:24:14,852 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940361b575bc060ce157779 2025-12-17 16:24:14,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:14,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940361b575bc060ce157779 HTTP/1.1" 200 None 2025-12-17 16:24:14,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:14,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:14,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:14,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940361b575bc060ce157779 HTTP/1.1" 200 None 2025-12-17 16:24:14,898 - lava_callback - INFO - Completed processing callback for node 6940361b575bc060ce157779 INFO: 172.21.0.1:44426 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:24:15,475 - lava_callback - INFO - Received callback for node 6940361d575bc060ce15777c INFO: 172.21.0.1:44428 - "POST /node/6940361d575bc060ce15777c HTTP/1.0" 202 Accepted 2025-12-17 16:24:15,482 - lava_callback - INFO - Starting async job submit for node 6940361d575bc060ce15777c 2025-12-17 16:24:15,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:15,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940361d575bc060ce15777c HTTP/1.1" 200 None 2025-12-17 16:24:15,492 - lava_callback - DEBUG - Processing node 6940361d575bc060ce15777c: baseline-arm64-baylibre 2025-12-17 16:24:15,492 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940361d575bc060ce15777c 2025-12-17 16:24:15,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:15,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940361d575bc060ce15777c HTTP/1.1" 200 None 2025-12-17 16:24:15,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:15,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:15,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:15,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940361d575bc060ce15777c HTTP/1.1" 200 None 2025-12-17 16:24:15,532 - lava_callback - INFO - Completed processing callback for node 6940361d575bc060ce15777c 2025-12-17 16:24:16,125 - lava_callback - INFO - Received callback for node 6940361f575bc060ce15777f INFO: 172.21.0.1:44434 - "POST /node/6940361f575bc060ce15777f HTTP/1.0" 202 Accepted 2025-12-17 16:24:16,130 - lava_callback - INFO - Starting async job submit for node 6940361f575bc060ce15777f 2025-12-17 16:24:16,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:16,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940361f575bc060ce15777f HTTP/1.1" 200 None 2025-12-17 16:24:16,140 - lava_callback - DEBUG - Processing node 6940361f575bc060ce15777f: baseline-arm64-baylibre 2025-12-17 16:24:16,140 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940361f575bc060ce15777f 2025-12-17 16:24:16,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:16,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940361f575bc060ce15777f HTTP/1.1" 200 None 2025-12-17 16:24:16,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:16,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:16,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:16,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940361f575bc060ce15777f HTTP/1.1" 200 None 2025-12-17 16:24:16,179 - lava_callback - INFO - Completed processing callback for node 6940361f575bc060ce15777f INFO: 172.21.0.1:44440 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:24:16,790 - lava_callback - INFO - Received callback for node 69403624575bc060ce157787 INFO: 172.21.0.1:44442 - "POST /node/69403624575bc060ce157787 HTTP/1.0" 202 Accepted 2025-12-17 16:24:16,795 - lava_callback - INFO - Starting async job submit for node 69403624575bc060ce157787 2025-12-17 16:24:16,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:16,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403624575bc060ce157787 HTTP/1.1" 200 None 2025-12-17 16:24:16,805 - lava_callback - DEBUG - Processing node 69403624575bc060ce157787: baseline-arm64-baylibre 2025-12-17 16:24:16,805 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403624575bc060ce157787 2025-12-17 16:24:16,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:16,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403624575bc060ce157787 HTTP/1.1" 200 None 2025-12-17 16:24:16,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:16,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:16,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:16,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403624575bc060ce157787 HTTP/1.1" 200 None 2025-12-17 16:24:16,837 - lava_callback - INFO - Completed processing callback for node 69403624575bc060ce157787 2025-12-17 16:24:17,457 - lava_callback - INFO - Received callback for node 69403627575bc060ce15778d INFO: 172.21.0.1:44444 - "POST /node/69403627575bc060ce15778d HTTP/1.0" 202 Accepted 2025-12-17 16:24:17,463 - lava_callback - INFO - Starting async job submit for node 69403627575bc060ce15778d 2025-12-17 16:24:17,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:17,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403627575bc060ce15778d HTTP/1.1" 200 None 2025-12-17 16:24:17,473 - lava_callback - DEBUG - Processing node 69403627575bc060ce15778d: baseline-arm64-baylibre 2025-12-17 16:24:17,473 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403627575bc060ce15778d 2025-12-17 16:24:17,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:17,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403627575bc060ce15778d HTTP/1.1" 200 None 2025-12-17 16:24:17,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:17,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:17,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:17,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403627575bc060ce15778d HTTP/1.1" 200 None 2025-12-17 16:24:17,514 - lava_callback - INFO - Completed processing callback for node 69403627575bc060ce15778d 2025-12-17 16:24:18,136 - lava_callback - INFO - Received callback for node 69403629575bc060ce157790 INFO: 172.21.0.1:44452 - "POST /node/69403629575bc060ce157790 HTTP/1.0" 202 Accepted 2025-12-17 16:24:18,143 - lava_callback - INFO - Starting async job submit for node 69403629575bc060ce157790 2025-12-17 16:24:18,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:18,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403629575bc060ce157790 HTTP/1.1" 200 None 2025-12-17 16:24:18,153 - lava_callback - DEBUG - Processing node 69403629575bc060ce157790: baseline-arm64-baylibre 2025-12-17 16:24:18,153 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403629575bc060ce157790 2025-12-17 16:24:18,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:18,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403629575bc060ce157790 HTTP/1.1" 200 None 2025-12-17 16:24:18,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:18,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:18,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:18,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403629575bc060ce157790 HTTP/1.1" 200 None 2025-12-17 16:24:18,195 - lava_callback - INFO - Completed processing callback for node 69403629575bc060ce157790 2025-12-17 16:24:18,799 - lava_callback - INFO - Received callback for node 6940362b575bc060ce157793 INFO: 172.21.0.1:44468 - "POST /node/6940362b575bc060ce157793 HTTP/1.0" 202 Accepted 2025-12-17 16:24:18,807 - lava_callback - INFO - Starting async job submit for node 6940362b575bc060ce157793 2025-12-17 16:24:18,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:18,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940362b575bc060ce157793 HTTP/1.1" 200 None 2025-12-17 16:24:18,817 - lava_callback - DEBUG - Processing node 6940362b575bc060ce157793: baseline-arm64-baylibre 2025-12-17 16:24:18,817 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940362b575bc060ce157793 2025-12-17 16:24:18,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:18,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940362b575bc060ce157793 HTTP/1.1" 200 None 2025-12-17 16:24:18,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:18,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:18,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:18,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940362b575bc060ce157793 HTTP/1.1" 200 None 2025-12-17 16:24:18,867 - lava_callback - INFO - Completed processing callback for node 6940362b575bc060ce157793 2025-12-17 16:24:19,484 - lava_callback - INFO - Received callback for node 6940362d575bc060ce157796 INFO: 172.21.0.1:44472 - "POST /node/6940362d575bc060ce157796 HTTP/1.0" 202 Accepted 2025-12-17 16:24:19,491 - lava_callback - INFO - Starting async job submit for node 6940362d575bc060ce157796 2025-12-17 16:24:19,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:19,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940362d575bc060ce157796 HTTP/1.1" 200 None 2025-12-17 16:24:19,793 - lava_callback - DEBUG - Processing node 6940362d575bc060ce157796: baseline-arm64-baylibre 2025-12-17 16:24:19,794 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940362d575bc060ce157796 2025-12-17 16:24:19,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:19,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940362d575bc060ce157796 HTTP/1.1" 200 None 2025-12-17 16:24:19,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:19,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:19,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:19,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940362d575bc060ce157796 HTTP/1.1" 200 None 2025-12-17 16:24:19,848 - lava_callback - INFO - Completed processing callback for node 6940362d575bc060ce157796 INFO: 172.21.0.1:58846 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:24:29,457 - lava_callback - INFO - Received callback for node 69403631575bc060ce1577a1 INFO: 172.21.0.1:58856 - "POST /node/69403631575bc060ce1577a1 HTTP/1.0" 202 Accepted 2025-12-17 16:24:29,464 - lava_callback - INFO - Starting async job submit for node 69403631575bc060ce1577a1 2025-12-17 16:24:29,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:29,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403631575bc060ce1577a1 HTTP/1.1" 200 None 2025-12-17 16:24:29,474 - lava_callback - DEBUG - Processing node 69403631575bc060ce1577a1: baseline-arm64-clabbe 2025-12-17 16:24:29,474 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403631575bc060ce1577a1 2025-12-17 16:24:29,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:29,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403631575bc060ce1577a1 HTTP/1.1" 200 None 2025-12-17 16:24:29,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:29,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:29,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:29,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403631575bc060ce1577a1 HTTP/1.1" 200 None 2025-12-17 16:24:29,512 - lava_callback - INFO - Completed processing callback for node 69403631575bc060ce1577a1 2025-12-17 16:24:30,141 - lava_callback - INFO - Received callback for node 69403634575bc060ce1577a4 INFO: 172.21.0.1:59906 - "POST /node/69403634575bc060ce1577a4 HTTP/1.0" 202 Accepted 2025-12-17 16:24:30,151 - lava_callback - INFO - Starting async job submit for node 69403634575bc060ce1577a4 2025-12-17 16:24:30,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:30,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403634575bc060ce1577a4 HTTP/1.1" 200 None 2025-12-17 16:24:30,162 - lava_callback - DEBUG - Processing node 69403634575bc060ce1577a4: baseline-arm64-clabbe 2025-12-17 16:24:30,163 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403634575bc060ce1577a4 2025-12-17 16:24:30,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:30,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403634575bc060ce1577a4 HTTP/1.1" 200 None 2025-12-17 16:24:30,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:30,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:30,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:30,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403634575bc060ce1577a4 HTTP/1.1" 200 None 2025-12-17 16:24:30,196 - lava_callback - INFO - Completed processing callback for node 69403634575bc060ce1577a4 2025-12-17 16:24:34,857 - lava_callback - INFO - Received callback for node 6940362f575bc060ce157799 INFO: 172.21.0.1:59912 - "POST /node/6940362f575bc060ce157799 HTTP/1.0" 202 Accepted 2025-12-17 16:24:34,864 - lava_callback - INFO - Starting async job submit for node 6940362f575bc060ce157799 2025-12-17 16:24:34,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:34,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940362f575bc060ce157799 HTTP/1.1" 200 None 2025-12-17 16:24:34,874 - lava_callback - DEBUG - Processing node 6940362f575bc060ce157799: baseline-arm64-baylibre 2025-12-17 16:24:34,874 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940362f575bc060ce157799 2025-12-17 16:24:34,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:34,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940362f575bc060ce157799 HTTP/1.1" 200 None 2025-12-17 16:24:34,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:34,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:34,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:34,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940362f575bc060ce157799 HTTP/1.1" 200 None 2025-12-17 16:24:34,910 - lava_callback - INFO - Completed processing callback for node 6940362f575bc060ce157799 2025-12-17 16:24:35,518 - lava_callback - INFO - Received callback for node 69403636575bc060ce1577a7 INFO: 172.21.0.1:59918 - "POST /node/69403636575bc060ce1577a7 HTTP/1.0" 202 Accepted 2025-12-17 16:24:35,524 - lava_callback - INFO - Starting async job submit for node 69403636575bc060ce1577a7 2025-12-17 16:24:35,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:35,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403636575bc060ce1577a7 HTTP/1.1" 200 None 2025-12-17 16:24:35,533 - lava_callback - DEBUG - Processing node 69403636575bc060ce1577a7: baseline-arm-baylibre 2025-12-17 16:24:35,533 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403636575bc060ce1577a7 2025-12-17 16:24:35,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:35,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403636575bc060ce1577a7 HTTP/1.1" 200 None 2025-12-17 16:24:35,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:35,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:24:35,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:35,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403636575bc060ce1577a7 HTTP/1.1" 200 None 2025-12-17 16:24:35,571 - lava_callback - INFO - Completed processing callback for node 69403636575bc060ce1577a7 2025-12-17 16:24:36,210 - lava_callback - INFO - Received callback for node 69403637575bc060ce1577aa INFO: 172.21.0.1:59934 - "POST /node/69403637575bc060ce1577aa HTTP/1.0" 202 Accepted 2025-12-17 16:24:36,216 - lava_callback - INFO - Starting async job submit for node 69403637575bc060ce1577aa 2025-12-17 16:24:36,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:36,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403637575bc060ce1577aa HTTP/1.1" 200 None 2025-12-17 16:24:36,226 - lava_callback - DEBUG - Processing node 69403637575bc060ce1577aa: baseline-arm-baylibre 2025-12-17 16:24:36,226 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403637575bc060ce1577aa 2025-12-17 16:24:36,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:36,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403637575bc060ce1577aa HTTP/1.1" 200 None 2025-12-17 16:24:36,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:36,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:24:36,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:36,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403637575bc060ce1577aa HTTP/1.1" 200 None 2025-12-17 16:24:36,264 - lava_callback - INFO - Completed processing callback for node 69403637575bc060ce1577aa INFO: 172.21.0.1:59938 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:24:36,869 - lava_callback - INFO - Received callback for node 69403639575bc060ce1577ad INFO: 172.21.0.1:59940 - "POST /node/69403639575bc060ce1577ad HTTP/1.0" 202 Accepted 2025-12-17 16:24:36,875 - lava_callback - INFO - Starting async job submit for node 69403639575bc060ce1577ad 2025-12-17 16:24:36,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:36,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403639575bc060ce1577ad HTTP/1.1" 200 None 2025-12-17 16:24:36,885 - lava_callback - DEBUG - Processing node 69403639575bc060ce1577ad: baseline-arm-baylibre 2025-12-17 16:24:36,885 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403639575bc060ce1577ad 2025-12-17 16:24:36,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:36,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403639575bc060ce1577ad HTTP/1.1" 200 None 2025-12-17 16:24:36,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:36,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:24:36,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:36,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403639575bc060ce1577ad HTTP/1.1" 200 None 2025-12-17 16:24:36,920 - lava_callback - INFO - Completed processing callback for node 69403639575bc060ce1577ad 2025-12-17 16:24:37,559 - lava_callback - INFO - Received callback for node 6940363a575bc060ce1577b1 INFO: 172.21.0.1:59946 - "POST /node/6940363a575bc060ce1577b1 HTTP/1.0" 202 Accepted 2025-12-17 16:24:37,567 - lava_callback - INFO - Starting async job submit for node 6940363a575bc060ce1577b1 2025-12-17 16:24:37,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:37,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940363a575bc060ce1577b1 HTTP/1.1" 200 None 2025-12-17 16:24:37,576 - lava_callback - DEBUG - Processing node 6940363a575bc060ce1577b1: baseline-arm-baylibre 2025-12-17 16:24:37,576 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940363a575bc060ce1577b1 2025-12-17 16:24:37,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:37,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940363a575bc060ce1577b1 HTTP/1.1" 200 None 2025-12-17 16:24:37,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:37,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:24:37,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:37,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940363a575bc060ce1577b1 HTTP/1.1" 200 None 2025-12-17 16:24:37,618 - lava_callback - INFO - Completed processing callback for node 6940363a575bc060ce1577b1 2025-12-17 16:24:38,235 - lava_callback - INFO - Received callback for node 6940363b575bc060ce1577b6 INFO: 172.21.0.1:59950 - "POST /node/6940363b575bc060ce1577b6 HTTP/1.0" 202 Accepted 2025-12-17 16:24:38,245 - lava_callback - INFO - Starting async job submit for node 6940363b575bc060ce1577b6 2025-12-17 16:24:38,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:38,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940363b575bc060ce1577b6 HTTP/1.1" 200 None 2025-12-17 16:24:38,256 - lava_callback - DEBUG - Processing node 6940363b575bc060ce1577b6: baseline-arm-baylibre 2025-12-17 16:24:38,256 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940363b575bc060ce1577b6 2025-12-17 16:24:38,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:38,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940363b575bc060ce1577b6 HTTP/1.1" 200 None 2025-12-17 16:24:38,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:38,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:24:38,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:38,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940363b575bc060ce1577b6 HTTP/1.1" 200 None 2025-12-17 16:24:38,286 - lava_callback - INFO - Completed processing callback for node 6940363b575bc060ce1577b6 2025-12-17 16:24:38,893 - lava_callback - INFO - Received callback for node 6940363c575bc060ce1577b9 INFO: 172.21.0.1:59966 - "POST /node/6940363c575bc060ce1577b9 HTTP/1.0" 202 Accepted 2025-12-17 16:24:38,899 - lava_callback - INFO - Starting async job submit for node 6940363c575bc060ce1577b9 2025-12-17 16:24:38,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:38,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940363c575bc060ce1577b9 HTTP/1.1" 200 None 2025-12-17 16:24:38,909 - lava_callback - DEBUG - Processing node 6940363c575bc060ce1577b9: baseline-arm-baylibre 2025-12-17 16:24:38,909 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940363c575bc060ce1577b9 2025-12-17 16:24:38,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:38,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940363c575bc060ce1577b9 HTTP/1.1" 200 None 2025-12-17 16:24:38,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:38,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:24:38,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:38,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940363c575bc060ce1577b9 HTTP/1.1" 200 None 2025-12-17 16:24:38,943 - lava_callback - INFO - Completed processing callback for node 6940363c575bc060ce1577b9 INFO: 172.21.0.1:49148 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:24:47,818 - lava_callback - INFO - Received callback for node 69403640575bc060ce1577bf INFO: 172.21.0.1:49156 - "POST /node/69403640575bc060ce1577bf HTTP/1.0" 202 Accepted 2025-12-17 16:24:47,824 - lava_callback - INFO - Starting async job submit for node 69403640575bc060ce1577bf 2025-12-17 16:24:47,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:47,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403640575bc060ce1577bf HTTP/1.1" 200 None 2025-12-17 16:24:47,833 - lava_callback - DEBUG - Processing node 69403640575bc060ce1577bf: baseline-arm-broonie 2025-12-17 16:24:47,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403640575bc060ce1577bf 2025-12-17 16:24:47,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:47,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403640575bc060ce1577bf HTTP/1.1" 200 None 2025-12-17 16:24:47,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:47,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:24:47,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:47,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403640575bc060ce1577bf HTTP/1.1" 200 None 2025-12-17 16:24:47,872 - lava_callback - INFO - Completed processing callback for node 69403640575bc060ce1577bf 2025-12-17 16:24:49,413 - lava_callback - INFO - Received callback for node 6940363e575bc060ce1577bc INFO: 172.21.0.1:49164 - "POST /node/6940363e575bc060ce1577bc HTTP/1.0" 202 Accepted 2025-12-17 16:24:49,418 - lava_callback - INFO - Starting async job submit for node 6940363e575bc060ce1577bc 2025-12-17 16:24:49,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:49,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940363e575bc060ce1577bc HTTP/1.1" 200 None 2025-12-17 16:24:49,430 - lava_callback - DEBUG - Processing node 6940363e575bc060ce1577bc: baseline-arm64-clabbe 2025-12-17 16:24:49,430 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940363e575bc060ce1577bc 2025-12-17 16:24:49,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:49,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940363e575bc060ce1577bc HTTP/1.1" 200 None 2025-12-17 16:24:49,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:49,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:49,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:49,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940363e575bc060ce1577bc HTTP/1.1" 200 None 2025-12-17 16:24:49,470 - lava_callback - INFO - Completed processing callback for node 6940363e575bc060ce1577bc 2025-12-17 16:24:50,087 - lava_callback - INFO - Received callback for node 69403641575bc060ce1577c2 INFO: 172.21.0.1:54182 - "POST /node/69403641575bc060ce1577c2 HTTP/1.0" 202 Accepted 2025-12-17 16:24:50,094 - lava_callback - INFO - Starting async job submit for node 69403641575bc060ce1577c2 2025-12-17 16:24:50,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:50,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403641575bc060ce1577c2 HTTP/1.1" 200 None 2025-12-17 16:24:50,104 - lava_callback - DEBUG - Processing node 69403641575bc060ce1577c2: baseline-arm64-clabbe 2025-12-17 16:24:50,104 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403641575bc060ce1577c2 2025-12-17 16:24:50,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:50,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403641575bc060ce1577c2 HTTP/1.1" 200 None 2025-12-17 16:24:50,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:50,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:50,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:50,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403641575bc060ce1577c2 HTTP/1.1" 200 None 2025-12-17 16:24:50,145 - lava_callback - INFO - Completed processing callback for node 69403641575bc060ce1577c2 2025-12-17 16:24:50,784 - lava_callback - INFO - Received callback for node 69403643575bc060ce1577c5 INFO: 172.21.0.1:54186 - "POST /node/69403643575bc060ce1577c5 HTTP/1.0" 202 Accepted 2025-12-17 16:24:50,793 - lava_callback - INFO - Starting async job submit for node 69403643575bc060ce1577c5 2025-12-17 16:24:50,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:50,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403643575bc060ce1577c5 HTTP/1.1" 200 None 2025-12-17 16:24:50,804 - lava_callback - DEBUG - Processing node 69403643575bc060ce1577c5: baseline-arm64-clabbe 2025-12-17 16:24:50,804 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403643575bc060ce1577c5 2025-12-17 16:24:50,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:50,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403643575bc060ce1577c5 HTTP/1.1" 200 None 2025-12-17 16:24:50,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:50,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:50,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:50,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403643575bc060ce1577c5 HTTP/1.1" 200 None 2025-12-17 16:24:50,852 - lava_callback - INFO - Completed processing callback for node 69403643575bc060ce1577c5 2025-12-17 16:24:51,463 - lava_callback - INFO - Received callback for node 69403645575bc060ce1577c8 INFO: 172.21.0.1:54194 - "POST /node/69403645575bc060ce1577c8 HTTP/1.0" 202 Accepted 2025-12-17 16:24:51,471 - lava_callback - INFO - Starting async job submit for node 69403645575bc060ce1577c8 2025-12-17 16:24:51,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:51,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403645575bc060ce1577c8 HTTP/1.1" 200 None 2025-12-17 16:24:51,480 - lava_callback - DEBUG - Processing node 69403645575bc060ce1577c8: baseline-arm64-clabbe 2025-12-17 16:24:51,480 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403645575bc060ce1577c8 2025-12-17 16:24:51,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:51,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403645575bc060ce1577c8 HTTP/1.1" 200 None 2025-12-17 16:24:51,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:51,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:51,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:51,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403645575bc060ce1577c8 HTTP/1.1" 200 None 2025-12-17 16:24:51,521 - lava_callback - INFO - Completed processing callback for node 69403645575bc060ce1577c8 2025-12-17 16:24:52,109 - lava_callback - INFO - Received callback for node 69403648575bc060ce1577cb INFO: 172.21.0.1:54206 - "POST /node/69403648575bc060ce1577cb HTTP/1.0" 202 Accepted 2025-12-17 16:24:52,116 - lava_callback - INFO - Starting async job submit for node 69403648575bc060ce1577cb 2025-12-17 16:24:52,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:52,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403648575bc060ce1577cb HTTP/1.1" 200 None 2025-12-17 16:24:52,128 - lava_callback - DEBUG - Processing node 69403648575bc060ce1577cb: baseline-arm64-clabbe 2025-12-17 16:24:52,128 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403648575bc060ce1577cb 2025-12-17 16:24:52,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:52,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403648575bc060ce1577cb HTTP/1.1" 200 None 2025-12-17 16:24:52,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:52,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:52,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:52,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403648575bc060ce1577cb HTTP/1.1" 200 None 2025-12-17 16:24:52,167 - lava_callback - INFO - Completed processing callback for node 69403648575bc060ce1577cb 2025-12-17 16:24:52,790 - lava_callback - INFO - Received callback for node 6940364a575bc060ce1577ce INFO: 172.21.0.1:54210 - "POST /node/6940364a575bc060ce1577ce HTTP/1.0" 202 Accepted 2025-12-17 16:24:52,797 - lava_callback - INFO - Starting async job submit for node 6940364a575bc060ce1577ce 2025-12-17 16:24:52,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:52,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940364a575bc060ce1577ce HTTP/1.1" 200 None 2025-12-17 16:24:52,809 - lava_callback - DEBUG - Processing node 6940364a575bc060ce1577ce: baseline-arm64-clabbe 2025-12-17 16:24:52,809 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940364a575bc060ce1577ce 2025-12-17 16:24:52,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:52,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940364a575bc060ce1577ce HTTP/1.1" 200 None 2025-12-17 16:24:52,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:52,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:52,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:52,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940364a575bc060ce1577ce HTTP/1.1" 200 None 2025-12-17 16:24:52,856 - lava_callback - INFO - Completed processing callback for node 6940364a575bc060ce1577ce 2025-12-17 16:24:53,471 - lava_callback - INFO - Received callback for node 6940364b575bc060ce1577d1 INFO: 172.21.0.1:54224 - "POST /node/6940364b575bc060ce1577d1 HTTP/1.0" 202 Accepted 2025-12-17 16:24:53,481 - lava_callback - INFO - Starting async job submit for node 6940364b575bc060ce1577d1 2025-12-17 16:24:53,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:53,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940364b575bc060ce1577d1 HTTP/1.1" 200 None 2025-12-17 16:24:53,492 - lava_callback - DEBUG - Processing node 6940364b575bc060ce1577d1: baseline-arm-clabbe 2025-12-17 16:24:53,492 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940364b575bc060ce1577d1 2025-12-17 16:24:53,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:53,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940364b575bc060ce1577d1 HTTP/1.1" 200 None 2025-12-17 16:24:53,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:53,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:24:53,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:53,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940364b575bc060ce1577d1 HTTP/1.1" 200 None 2025-12-17 16:24:53,535 - lava_callback - INFO - Completed processing callback for node 6940364b575bc060ce1577d1 2025-12-17 16:24:54,141 - lava_callback - INFO - Received callback for node 6940364d575bc060ce1577d4 INFO: 172.21.0.1:54236 - "POST /node/6940364d575bc060ce1577d4 HTTP/1.0" 202 Accepted 2025-12-17 16:24:54,147 - lava_callback - INFO - Starting async job submit for node 6940364d575bc060ce1577d4 2025-12-17 16:24:54,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:54,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940364d575bc060ce1577d4 HTTP/1.1" 200 None 2025-12-17 16:24:54,157 - lava_callback - DEBUG - Processing node 6940364d575bc060ce1577d4: baseline-arm64-clabbe 2025-12-17 16:24:54,157 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940364d575bc060ce1577d4 2025-12-17 16:24:54,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:54,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940364d575bc060ce1577d4 HTTP/1.1" 200 None 2025-12-17 16:24:54,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:54,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:24:54,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:54,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940364d575bc060ce1577d4 HTTP/1.1" 200 None 2025-12-17 16:24:54,192 - lava_callback - INFO - Completed processing callback for node 6940364d575bc060ce1577d4 2025-12-17 16:24:54,372 - lava_callback - INFO - Received callback for node 69403652575bc060ce1577eb INFO: 172.21.0.1:54240 - "POST /node/69403652575bc060ce1577eb HTTP/1.0" 202 Accepted 2025-12-17 16:24:54,380 - lava_callback - INFO - Starting async job submit for node 69403652575bc060ce1577eb 2025-12-17 16:24:54,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:54,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403652575bc060ce1577eb HTTP/1.1" 200 None 2025-12-17 16:24:54,389 - lava_callback - DEBUG - Processing node 69403652575bc060ce1577eb: baseline-x86-kcidebug-intel 2025-12-17 16:24:54,389 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403652575bc060ce1577eb 2025-12-17 16:24:54,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:54,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403652575bc060ce1577eb HTTP/1.1" 200 None 2025-12-17 16:24:54,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:54,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6938484ea4c94886b94c4060 HTTP/1.1" 200 None 2025-12-17 16:24:54,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:54,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403652575bc060ce1577eb HTTP/1.1" 200 None 2025-12-17 16:24:54,442 - lava_callback - INFO - Completed processing callback for node 69403652575bc060ce1577eb 2025-12-17 16:24:54,827 - lava_callback - INFO - Received callback for node 69403650575bc060ce1577e1 INFO: 172.21.0.1:54252 - "POST /node/69403650575bc060ce1577e1 HTTP/1.0" 202 Accepted 2025-12-17 16:24:54,834 - lava_callback - INFO - Starting async job submit for node 69403650575bc060ce1577e1 2025-12-17 16:24:54,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:54,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403650575bc060ce1577e1 HTTP/1.1" 200 None 2025-12-17 16:24:54,899 - lava_callback - DEBUG - Processing node 69403650575bc060ce1577e1: baseline-arm-clabbe 2025-12-17 16:24:54,899 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403650575bc060ce1577e1 2025-12-17 16:24:54,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:54,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403650575bc060ce1577e1 HTTP/1.1" 200 None 2025-12-17 16:24:54,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:54,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:24:54,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:24:54,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403650575bc060ce1577e1 HTTP/1.1" 200 None 2025-12-17 16:24:54,944 - lava_callback - INFO - Completed processing callback for node 69403650575bc060ce1577e1 INFO: 172.21.0.1:54254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41912 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:25:07,902 - lava_callback - INFO - Received callback for node 69403652575bc060ce1577e8 INFO: 172.21.0.1:41916 - "POST /node/69403652575bc060ce1577e8 HTTP/1.0" 202 Accepted 2025-12-17 16:25:07,910 - lava_callback - INFO - Starting async job submit for node 69403652575bc060ce1577e8 2025-12-17 16:25:07,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:07,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403652575bc060ce1577e8 HTTP/1.1" 200 None 2025-12-17 16:25:07,920 - lava_callback - DEBUG - Processing node 69403652575bc060ce1577e8: ltp-syscalls 2025-12-17 16:25:07,920 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403652575bc060ce1577e8 2025-12-17 16:25:07,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:07,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403652575bc060ce1577e8 HTTP/1.1" 200 None 2025-12-17 16:25:07,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:07,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:07,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:07,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403652575bc060ce1577e8 HTTP/1.1" 200 None 2025-12-17 16:25:07,956 - lava_callback - INFO - Completed processing callback for node 69403652575bc060ce1577e8 2025-12-17 16:25:08,641 - lava_callback - INFO - Received callback for node 6940365e575bc060ce15780c INFO: 172.21.0.1:41932 - "POST /node/6940365e575bc060ce15780c HTTP/1.0" 202 Accepted 2025-12-17 16:25:08,649 - lava_callback - INFO - Starting async job submit for node 6940365e575bc060ce15780c 2025-12-17 16:25:08,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:08,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940365e575bc060ce15780c HTTP/1.1" 200 None 2025-12-17 16:25:08,658 - lava_callback - DEBUG - Processing node 6940365e575bc060ce15780c: baseline-arm-broonie 2025-12-17 16:25:08,658 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940365e575bc060ce15780c 2025-12-17 16:25:08,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:08,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940365e575bc060ce15780c HTTP/1.1" 200 None 2025-12-17 16:25:08,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:08,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:25:08,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:08,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940365e575bc060ce15780c HTTP/1.1" 200 None 2025-12-17 16:25:08,700 - lava_callback - INFO - Completed processing callback for node 6940365e575bc060ce15780c 2025-12-17 16:25:09,413 - lava_callback - INFO - Received callback for node 69403653575bc060ce1577ee INFO: 172.21.0.1:41942 - "POST /node/69403653575bc060ce1577ee HTTP/1.0" 202 Accepted 2025-12-17 16:25:09,419 - lava_callback - INFO - Starting async job submit for node 69403653575bc060ce1577ee 2025-12-17 16:25:09,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:09,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403653575bc060ce1577ee HTTP/1.1" 200 None 2025-12-17 16:25:09,431 - lava_callback - DEBUG - Processing node 69403653575bc060ce1577ee: baseline-arm64-clabbe 2025-12-17 16:25:09,431 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403653575bc060ce1577ee 2025-12-17 16:25:09,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:09,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403653575bc060ce1577ee HTTP/1.1" 200 None 2025-12-17 16:25:09,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:09,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f9fb52d1dfff36c4b3 HTTP/1.1" 200 None 2025-12-17 16:25:09,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:09,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403653575bc060ce1577ee HTTP/1.1" 200 None 2025-12-17 16:25:09,467 - lava_callback - INFO - Completed processing callback for node 69403653575bc060ce1577ee 2025-12-17 16:25:10,098 - lava_callback - INFO - Received callback for node 69403655575bc060ce1577f1 INFO: 172.21.0.1:47064 - "POST /node/69403655575bc060ce1577f1 HTTP/1.0" 202 Accepted 2025-12-17 16:25:10,105 - lava_callback - INFO - Starting async job submit for node 69403655575bc060ce1577f1 2025-12-17 16:25:10,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:10,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403655575bc060ce1577f1 HTTP/1.1" 200 None 2025-12-17 16:25:10,119 - lava_callback - DEBUG - Processing node 69403655575bc060ce1577f1: baseline-arm-clabbe 2025-12-17 16:25:10,119 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403655575bc060ce1577f1 2025-12-17 16:25:10,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:10,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403655575bc060ce1577f1 HTTP/1.1" 200 None 2025-12-17 16:25:10,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:10,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:10,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:10,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403655575bc060ce1577f1 HTTP/1.1" 200 None 2025-12-17 16:25:10,160 - lava_callback - INFO - Completed processing callback for node 69403655575bc060ce1577f1 INFO: 172.21.0.1:47078 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:25:10,790 - lava_callback - INFO - Received callback for node 69403656575bc060ce1577f4 INFO: 172.21.0.1:47092 - "POST /node/69403656575bc060ce1577f4 HTTP/1.0" 202 Accepted 2025-12-17 16:25:10,798 - lava_callback - INFO - Starting async job submit for node 69403656575bc060ce1577f4 2025-12-17 16:25:10,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:10,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403656575bc060ce1577f4 HTTP/1.1" 200 None 2025-12-17 16:25:10,813 - lava_callback - DEBUG - Processing node 69403656575bc060ce1577f4: baseline-arm-clabbe 2025-12-17 16:25:10,813 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403656575bc060ce1577f4 2025-12-17 16:25:10,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:10,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403656575bc060ce1577f4 HTTP/1.1" 200 None 2025-12-17 16:25:10,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:10,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:10,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:10,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403656575bc060ce1577f4 HTTP/1.1" 200 None 2025-12-17 16:25:10,852 - lava_callback - INFO - Completed processing callback for node 69403656575bc060ce1577f4 2025-12-17 16:25:11,471 - lava_callback - INFO - Received callback for node 69403657575bc060ce1577f7 INFO: 172.21.0.1:47096 - "POST /node/69403657575bc060ce1577f7 HTTP/1.0" 202 Accepted 2025-12-17 16:25:11,478 - lava_callback - INFO - Starting async job submit for node 69403657575bc060ce1577f7 2025-12-17 16:25:11,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:11,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403657575bc060ce1577f7 HTTP/1.1" 200 None 2025-12-17 16:25:11,490 - lava_callback - DEBUG - Processing node 69403657575bc060ce1577f7: baseline-arm-clabbe 2025-12-17 16:25:11,490 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403657575bc060ce1577f7 2025-12-17 16:25:11,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:11,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403657575bc060ce1577f7 HTTP/1.1" 200 None 2025-12-17 16:25:11,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:11,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:11,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:11,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403657575bc060ce1577f7 HTTP/1.1" 200 None 2025-12-17 16:25:11,524 - lava_callback - INFO - Completed processing callback for node 69403657575bc060ce1577f7 2025-12-17 16:25:12,143 - lava_callback - INFO - Received callback for node 69403663575bc060ce157820 INFO: 172.21.0.1:47112 - "POST /node/69403663575bc060ce157820 HTTP/1.0" 202 Accepted 2025-12-17 16:25:12,152 - lava_callback - INFO - Starting async job submit for node 69403663575bc060ce157820 2025-12-17 16:25:12,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:12,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403663575bc060ce157820 HTTP/1.1" 200 None 2025-12-17 16:25:12,177 - lava_callback - DEBUG - Processing node 69403663575bc060ce157820: baseline-arm-clabbe 2025-12-17 16:25:12,177 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403663575bc060ce157820 2025-12-17 16:25:12,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:12,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403663575bc060ce157820 HTTP/1.1" 200 None 2025-12-17 16:25:12,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:12,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:25:12,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:12,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403663575bc060ce157820 HTTP/1.1" 200 None 2025-12-17 16:25:12,215 - lava_callback - INFO - Completed processing callback for node 69403663575bc060ce157820 2025-12-17 16:25:12,801 - lava_callback - INFO - Received callback for node 69403664575bc060ce157827 INFO: 172.21.0.1:47114 - "POST /node/69403664575bc060ce157827 HTTP/1.0" 202 Accepted 2025-12-17 16:25:12,808 - lava_callback - INFO - Starting async job submit for node 69403664575bc060ce157827 2025-12-17 16:25:12,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:12,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403664575bc060ce157827 HTTP/1.1" 200 None 2025-12-17 16:25:12,816 - lava_callback - DEBUG - Processing node 69403664575bc060ce157827: baseline-arm-clabbe 2025-12-17 16:25:12,816 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403664575bc060ce157827 2025-12-17 16:25:12,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:12,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403664575bc060ce157827 HTTP/1.1" 200 None 2025-12-17 16:25:12,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:12,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:25:12,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:12,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403664575bc060ce157827 HTTP/1.1" 200 None 2025-12-17 16:25:12,858 - lava_callback - INFO - Completed processing callback for node 69403664575bc060ce157827 2025-12-17 16:25:14,888 - lava_callback - INFO - Received callback for node 69403658575bc060ce1577fa INFO: 172.21.0.1:47116 - "POST /node/69403658575bc060ce1577fa HTTP/1.0" 202 Accepted 2025-12-17 16:25:14,895 - lava_callback - INFO - Starting async job submit for node 69403658575bc060ce1577fa 2025-12-17 16:25:14,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:14,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403658575bc060ce1577fa HTTP/1.1" 200 None 2025-12-17 16:25:14,904 - lava_callback - DEBUG - Processing node 69403658575bc060ce1577fa: baseline-arm-baylibre 2025-12-17 16:25:14,905 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403658575bc060ce1577fa 2025-12-17 16:25:14,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:14,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403658575bc060ce1577fa HTTP/1.1" 200 None 2025-12-17 16:25:14,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:14,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:25:14,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:14,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403658575bc060ce1577fa HTTP/1.1" 200 None 2025-12-17 16:25:14,944 - lava_callback - INFO - Completed processing callback for node 69403658575bc060ce1577fa 2025-12-17 16:25:14,988 - lava_callback - INFO - Received callback for node 69403669575bc060ce157837 INFO: 172.21.0.1:47126 - "POST /node/69403669575bc060ce157837 HTTP/1.0" 202 Accepted 2025-12-17 16:25:14,998 - lava_callback - INFO - Starting async job submit for node 69403669575bc060ce157837 2025-12-17 16:25:14,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:15,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403669575bc060ce157837 HTTP/1.1" 200 None 2025-12-17 16:25:15,019 - lava_callback - DEBUG - Processing node 69403669575bc060ce157837: baseline-arm 2025-12-17 16:25:15,019 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403669575bc060ce157837 2025-12-17 16:25:15,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:15,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403669575bc060ce157837 HTTP/1.1" 200 None 2025-12-17 16:25:15,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:15,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:15,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:15,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403669575bc060ce157837 HTTP/1.1" 200 None 2025-12-17 16:25:15,070 - lava_callback - INFO - Completed processing callback for node 69403669575bc060ce157837 INFO: 172.21.0.1:47138 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:25:15,547 - lava_callback - INFO - Received callback for node 6940365a575bc060ce157801 INFO: 172.21.0.1:47152 - "POST /node/6940365a575bc060ce157801 HTTP/1.0" 202 Accepted 2025-12-17 16:25:15,557 - lava_callback - INFO - Starting async job submit for node 6940365a575bc060ce157801 2025-12-17 16:25:15,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:15,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940365a575bc060ce157801 HTTP/1.1" 200 None 2025-12-17 16:25:15,568 - lava_callback - DEBUG - Processing node 6940365a575bc060ce157801: baseline-arm-baylibre 2025-12-17 16:25:15,568 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940365a575bc060ce157801 2025-12-17 16:25:15,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:15,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940365a575bc060ce157801 HTTP/1.1" 200 None 2025-12-17 16:25:15,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:15,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:25:15,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:15,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940365a575bc060ce157801 HTTP/1.1" 200 None 2025-12-17 16:25:15,610 - lava_callback - INFO - Completed processing callback for node 6940365a575bc060ce157801 2025-12-17 16:25:16,221 - lava_callback - INFO - Received callback for node 6940365b575bc060ce157806 INFO: 172.21.0.1:47158 - "POST /node/6940365b575bc060ce157806 HTTP/1.0" 202 Accepted 2025-12-17 16:25:16,229 - lava_callback - INFO - Starting async job submit for node 6940365b575bc060ce157806 2025-12-17 16:25:16,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:16,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940365b575bc060ce157806 HTTP/1.1" 200 None 2025-12-17 16:25:16,239 - lava_callback - DEBUG - Processing node 6940365b575bc060ce157806: baseline-arm-baylibre 2025-12-17 16:25:16,239 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940365b575bc060ce157806 2025-12-17 16:25:16,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:16,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940365b575bc060ce157806 HTTP/1.1" 200 None 2025-12-17 16:25:16,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:16,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:25:16,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:16,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940365b575bc060ce157806 HTTP/1.1" 200 None 2025-12-17 16:25:16,279 - lava_callback - INFO - Completed processing callback for node 6940365b575bc060ce157806 INFO: 172.21.0.1:47174 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:25:16,921 - lava_callback - INFO - Received callback for node 6940365c575bc060ce157809 INFO: 172.21.0.1:47180 - "POST /node/6940365c575bc060ce157809 HTTP/1.0" 202 Accepted 2025-12-17 16:25:16,927 - lava_callback - INFO - Starting async job submit for node 6940365c575bc060ce157809 2025-12-17 16:25:16,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:16,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940365c575bc060ce157809 HTTP/1.1" 200 None 2025-12-17 16:25:16,936 - lava_callback - DEBUG - Processing node 6940365c575bc060ce157809: baseline-arm-baylibre 2025-12-17 16:25:16,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940365c575bc060ce157809 2025-12-17 16:25:16,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:16,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940365c575bc060ce157809 HTTP/1.1" 200 None 2025-12-17 16:25:16,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:16,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:25:16,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:16,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940365c575bc060ce157809 HTTP/1.1" 200 None 2025-12-17 16:25:16,969 - lava_callback - INFO - Completed processing callback for node 6940365c575bc060ce157809 2025-12-17 16:25:17,570 - lava_callback - INFO - Received callback for node 6940365f575bc060ce15780f INFO: 172.21.0.1:47196 - "POST /node/6940365f575bc060ce15780f HTTP/1.0" 202 Accepted 2025-12-17 16:25:17,580 - lava_callback - INFO - Starting async job submit for node 6940365f575bc060ce15780f 2025-12-17 16:25:17,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:17,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940365f575bc060ce15780f HTTP/1.1" 200 None 2025-12-17 16:25:17,605 - lava_callback - DEBUG - Processing node 6940365f575bc060ce15780f: baseline-arm-baylibre 2025-12-17 16:25:17,605 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940365f575bc060ce15780f 2025-12-17 16:25:17,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:17,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940365f575bc060ce15780f HTTP/1.1" 200 None 2025-12-17 16:25:17,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:17,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:25:17,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:17,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940365f575bc060ce15780f HTTP/1.1" 200 None 2025-12-17 16:25:17,643 - lava_callback - INFO - Completed processing callback for node 6940365f575bc060ce15780f 2025-12-17 16:25:18,240 - lava_callback - INFO - Received callback for node 69403660575bc060ce157815 INFO: 172.21.0.1:47212 - "POST /node/69403660575bc060ce157815 HTTP/1.0" 202 Accepted 2025-12-17 16:25:18,247 - lava_callback - INFO - Starting async job submit for node 69403660575bc060ce157815 2025-12-17 16:25:18,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:18,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403660575bc060ce157815 HTTP/1.1" 200 None 2025-12-17 16:25:18,256 - lava_callback - DEBUG - Processing node 69403660575bc060ce157815: baseline-arm-baylibre 2025-12-17 16:25:18,256 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403660575bc060ce157815 2025-12-17 16:25:18,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:18,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403660575bc060ce157815 HTTP/1.1" 200 None 2025-12-17 16:25:18,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:18,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:25:18,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:18,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403660575bc060ce157815 HTTP/1.1" 200 None 2025-12-17 16:25:18,290 - lava_callback - INFO - Completed processing callback for node 69403660575bc060ce157815 INFO: 172.21.0.1:39202 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:25:28,406 - lava_callback - INFO - Received callback for node 6940366e575bc060ce157840 INFO: 172.21.0.1:39208 - "POST /node/6940366e575bc060ce157840 HTTP/1.0" 202 Accepted 2025-12-17 16:25:28,413 - lava_callback - INFO - Starting async job submit for node 6940366e575bc060ce157840 2025-12-17 16:25:28,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:28,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940366e575bc060ce157840 HTTP/1.1" 200 None 2025-12-17 16:25:28,423 - lava_callback - DEBUG - Processing node 6940366e575bc060ce157840: ltp-syscalls 2025-12-17 16:25:28,423 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940366e575bc060ce157840 2025-12-17 16:25:28,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:28,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940366e575bc060ce157840 HTTP/1.1" 200 None 2025-12-17 16:25:28,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:28,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:25:28,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:28,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940366e575bc060ce157840 HTTP/1.1" 200 None 2025-12-17 16:25:28,457 - lava_callback - INFO - Completed processing callback for node 6940366e575bc060ce157840 2025-12-17 16:25:29,174 - lava_callback - INFO - Received callback for node 69403673575bc060ce15784c INFO: 172.21.0.1:39216 - "POST /node/69403673575bc060ce15784c HTTP/1.0" 202 Accepted 2025-12-17 16:25:29,181 - lava_callback - INFO - Starting async job submit for node 69403673575bc060ce15784c 2025-12-17 16:25:29,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:29,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403673575bc060ce15784c HTTP/1.1" 200 None 2025-12-17 16:25:29,190 - lava_callback - DEBUG - Processing node 69403673575bc060ce15784c: baseline-arm-broonie 2025-12-17 16:25:29,190 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403673575bc060ce15784c 2025-12-17 16:25:29,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:29,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403673575bc060ce15784c HTTP/1.1" 200 None 2025-12-17 16:25:29,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:29,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:29,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:29,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403673575bc060ce15784c HTTP/1.1" 200 None 2025-12-17 16:25:29,237 - lava_callback - INFO - Completed processing callback for node 69403673575bc060ce15784c 2025-12-17 16:25:29,466 - lava_callback - INFO - Received callback for node 69403665575bc060ce15782e INFO: 172.21.0.1:39230 - "POST /node/69403665575bc060ce15782e HTTP/1.0" 202 Accepted 2025-12-17 16:25:29,473 - lava_callback - INFO - Starting async job submit for node 69403665575bc060ce15782e 2025-12-17 16:25:29,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:29,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403665575bc060ce15782e HTTP/1.1" 200 None 2025-12-17 16:25:29,483 - lava_callback - DEBUG - Processing node 69403665575bc060ce15782e: baseline-arm-clabbe 2025-12-17 16:25:29,483 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403665575bc060ce15782e 2025-12-17 16:25:29,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:29,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403665575bc060ce15782e HTTP/1.1" 200 None 2025-12-17 16:25:29,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:29,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:25:29,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:29,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403665575bc060ce15782e HTTP/1.1" 200 None 2025-12-17 16:25:29,515 - lava_callback - INFO - Completed processing callback for node 69403665575bc060ce15782e 2025-12-17 16:25:30,147 - lava_callback - INFO - Received callback for node 69403667575bc060ce157831 INFO: 172.21.0.1:34894 - "POST /node/69403667575bc060ce157831 HTTP/1.0" 202 Accepted 2025-12-17 16:25:30,154 - lava_callback - INFO - Starting async job submit for node 69403667575bc060ce157831 2025-12-17 16:25:30,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:30,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403667575bc060ce157831 HTTP/1.1" 200 None 2025-12-17 16:25:30,163 - lava_callback - DEBUG - Processing node 69403667575bc060ce157831: baseline-arm-clabbe 2025-12-17 16:25:30,163 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403667575bc060ce157831 2025-12-17 16:25:30,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:30,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403667575bc060ce157831 HTTP/1.1" 200 None 2025-12-17 16:25:30,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:30,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:25:30,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:30,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403667575bc060ce157831 HTTP/1.1" 200 None 2025-12-17 16:25:30,209 - lava_callback - INFO - Completed processing callback for node 69403667575bc060ce157831 2025-12-17 16:25:30,812 - lava_callback - INFO - Received callback for node 69403668575bc060ce157834 INFO: 172.21.0.1:34904 - "POST /node/69403668575bc060ce157834 HTTP/1.0" 202 Accepted 2025-12-17 16:25:30,821 - lava_callback - INFO - Starting async job submit for node 69403668575bc060ce157834 2025-12-17 16:25:30,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:30,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403668575bc060ce157834 HTTP/1.1" 200 None 2025-12-17 16:25:30,831 - lava_callback - DEBUG - Processing node 69403668575bc060ce157834: baseline-arm-clabbe 2025-12-17 16:25:30,831 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403668575bc060ce157834 2025-12-17 16:25:30,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:30,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403668575bc060ce157834 HTTP/1.1" 200 None 2025-12-17 16:25:30,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:30,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:25:30,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:30,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403668575bc060ce157834 HTTP/1.1" 200 None 2025-12-17 16:25:30,868 - lava_callback - INFO - Completed processing callback for node 69403668575bc060ce157834 2025-12-17 16:25:34,392 - lava_callback - INFO - Received callback for node 6940366a575bc060ce15783a INFO: 172.21.0.1:34906 - "POST /node/6940366a575bc060ce15783a HTTP/1.0" 202 Accepted 2025-12-17 16:25:34,399 - lava_callback - INFO - Starting async job submit for node 6940366a575bc060ce15783a 2025-12-17 16:25:34,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:34,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940366a575bc060ce15783a HTTP/1.1" 200 None 2025-12-17 16:25:34,408 - lava_callback - DEBUG - Processing node 6940366a575bc060ce15783a: baseline-arm 2025-12-17 16:25:34,408 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940366a575bc060ce15783a 2025-12-17 16:25:34,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:34,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940366a575bc060ce15783a HTTP/1.1" 200 None 2025-12-17 16:25:34,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:34,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:34,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:34,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940366a575bc060ce15783a HTTP/1.1" 200 None 2025-12-17 16:25:34,448 - lava_callback - INFO - Completed processing callback for node 6940366a575bc060ce15783a 2025-12-17 16:25:34,921 - lava_callback - INFO - Received callback for node 6940366b575bc060ce15783d INFO: 172.21.0.1:34910 - "POST /node/6940366b575bc060ce15783d HTTP/1.0" 202 Accepted 2025-12-17 16:25:34,934 - lava_callback - INFO - Starting async job submit for node 6940366b575bc060ce15783d 2025-12-17 16:25:34,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:34,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940366b575bc060ce15783d HTTP/1.1" 200 None 2025-12-17 16:25:34,952 - lava_callback - DEBUG - Processing node 6940366b575bc060ce15783d: baseline-arm-baylibre 2025-12-17 16:25:34,952 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940366b575bc060ce15783d 2025-12-17 16:25:34,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:34,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940366b575bc060ce15783d HTTP/1.1" 200 None 2025-12-17 16:25:34,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:34,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:34,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:35,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940366b575bc060ce15783d HTTP/1.1" 200 None 2025-12-17 16:25:35,012 - lava_callback - INFO - Completed processing callback for node 6940366b575bc060ce15783d 2025-12-17 16:25:35,581 - lava_callback - INFO - Received callback for node 6940366f575bc060ce157843 INFO: 172.21.0.1:34914 - "POST /node/6940366f575bc060ce157843 HTTP/1.0" 202 Accepted 2025-12-17 16:25:35,587 - lava_callback - INFO - Starting async job submit for node 6940366f575bc060ce157843 2025-12-17 16:25:35,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:35,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940366f575bc060ce157843 HTTP/1.1" 200 None 2025-12-17 16:25:35,603 - lava_callback - DEBUG - Processing node 6940366f575bc060ce157843: baseline-arm-baylibre 2025-12-17 16:25:35,603 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940366f575bc060ce157843 2025-12-17 16:25:35,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:35,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940366f575bc060ce157843 HTTP/1.1" 200 None 2025-12-17 16:25:35,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:35,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:35,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:35,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940366f575bc060ce157843 HTTP/1.1" 200 None 2025-12-17 16:25:35,636 - lava_callback - INFO - Completed processing callback for node 6940366f575bc060ce157843 2025-12-17 16:25:36,247 - lava_callback - INFO - Received callback for node 69403670575bc060ce157846 INFO: 172.21.0.1:34918 - "POST /node/69403670575bc060ce157846 HTTP/1.0" 202 Accepted 2025-12-17 16:25:36,253 - lava_callback - INFO - Starting async job submit for node 69403670575bc060ce157846 2025-12-17 16:25:36,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:36,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403670575bc060ce157846 HTTP/1.1" 200 None 2025-12-17 16:25:36,266 - lava_callback - DEBUG - Processing node 69403670575bc060ce157846: baseline-arm-baylibre 2025-12-17 16:25:36,266 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403670575bc060ce157846 2025-12-17 16:25:36,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:36,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403670575bc060ce157846 HTTP/1.1" 200 None 2025-12-17 16:25:36,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:36,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:36,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:36,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403670575bc060ce157846 HTTP/1.1" 200 None 2025-12-17 16:25:36,300 - lava_callback - INFO - Completed processing callback for node 69403670575bc060ce157846 INFO: 172.21.0.1:34926 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:25:36,889 - lava_callback - INFO - Received callback for node 69403671575bc060ce157849 INFO: 172.21.0.1:34942 - "POST /node/69403671575bc060ce157849 HTTP/1.0" 202 Accepted 2025-12-17 16:25:36,895 - lava_callback - INFO - Starting async job submit for node 69403671575bc060ce157849 2025-12-17 16:25:36,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:36,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403671575bc060ce157849 HTTP/1.1" 200 None 2025-12-17 16:25:36,905 - lava_callback - DEBUG - Processing node 69403671575bc060ce157849: baseline-arm-baylibre 2025-12-17 16:25:36,905 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403671575bc060ce157849 2025-12-17 16:25:36,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:36,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403671575bc060ce157849 HTTP/1.1" 200 None 2025-12-17 16:25:36,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:36,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:36,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:36,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403671575bc060ce157849 HTTP/1.1" 200 None 2025-12-17 16:25:36,947 - lava_callback - INFO - Completed processing callback for node 69403671575bc060ce157849 2025-12-17 16:25:37,536 - lava_callback - INFO - Received callback for node 69403674575bc060ce157856 INFO: 172.21.0.1:34944 - "POST /node/69403674575bc060ce157856 HTTP/1.0" 202 Accepted 2025-12-17 16:25:37,541 - lava_callback - INFO - Starting async job submit for node 69403674575bc060ce157856 2025-12-17 16:25:37,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:37,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403674575bc060ce157856 HTTP/1.1" 200 None 2025-12-17 16:25:37,551 - lava_callback - DEBUG - Processing node 69403674575bc060ce157856: baseline-arm-baylibre 2025-12-17 16:25:37,551 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403674575bc060ce157856 2025-12-17 16:25:37,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:37,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403674575bc060ce157856 HTTP/1.1" 200 None 2025-12-17 16:25:37,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:37,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:37,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:37,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403674575bc060ce157856 HTTP/1.1" 200 None 2025-12-17 16:25:37,630 - lava_callback - INFO - Completed processing callback for node 69403674575bc060ce157856 2025-12-17 16:25:38,206 - lava_callback - INFO - Received callback for node 69403679575bc060ce157865 INFO: 172.21.0.1:34948 - "POST /node/69403679575bc060ce157865 HTTP/1.0" 202 Accepted 2025-12-17 16:25:38,213 - lava_callback - INFO - Starting async job submit for node 69403679575bc060ce157865 2025-12-17 16:25:38,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:38,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403679575bc060ce157865 HTTP/1.1" 200 None 2025-12-17 16:25:38,222 - lava_callback - DEBUG - Processing node 69403679575bc060ce157865: baseline-arm-baylibre 2025-12-17 16:25:38,223 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403679575bc060ce157865 2025-12-17 16:25:38,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:38,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403679575bc060ce157865 HTTP/1.1" 200 None 2025-12-17 16:25:38,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:38,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:38,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:38,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403679575bc060ce157865 HTTP/1.1" 200 None 2025-12-17 16:25:38,265 - lava_callback - INFO - Completed processing callback for node 69403679575bc060ce157865 2025-12-17 16:25:38,880 - lava_callback - INFO - Received callback for node 6940367b575bc060ce157868 INFO: 172.21.0.1:34964 - "POST /node/6940367b575bc060ce157868 HTTP/1.0" 202 Accepted 2025-12-17 16:25:38,888 - lava_callback - INFO - Starting async job submit for node 6940367b575bc060ce157868 2025-12-17 16:25:38,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:38,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940367b575bc060ce157868 HTTP/1.1" 200 None 2025-12-17 16:25:38,897 - lava_callback - DEBUG - Processing node 6940367b575bc060ce157868: baseline-arm-baylibre 2025-12-17 16:25:38,897 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940367b575bc060ce157868 2025-12-17 16:25:38,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:38,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940367b575bc060ce157868 HTTP/1.1" 200 None 2025-12-17 16:25:38,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:38,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:38,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:38,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940367b575bc060ce157868 HTTP/1.1" 200 None 2025-12-17 16:25:38,934 - lava_callback - INFO - Completed processing callback for node 6940367b575bc060ce157868 2025-12-17 16:25:39,555 - lava_callback - INFO - Received callback for node 6940367c575bc060ce15786b INFO: 172.21.0.1:34968 - "POST /node/6940367c575bc060ce15786b HTTP/1.0" 202 Accepted 2025-12-17 16:25:39,561 - lava_callback - INFO - Starting async job submit for node 6940367c575bc060ce15786b 2025-12-17 16:25:39,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:39,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940367c575bc060ce15786b HTTP/1.1" 200 None 2025-12-17 16:25:39,570 - lava_callback - DEBUG - Processing node 6940367c575bc060ce15786b: baseline-arm-baylibre 2025-12-17 16:25:39,571 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940367c575bc060ce15786b 2025-12-17 16:25:39,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:39,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940367c575bc060ce15786b HTTP/1.1" 200 None 2025-12-17 16:25:39,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:39,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:39,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:39,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940367c575bc060ce15786b HTTP/1.1" 200 None 2025-12-17 16:25:39,601 - lava_callback - INFO - Completed processing callback for node 6940367c575bc060ce15786b 2025-12-17 16:25:40,239 - lava_callback - INFO - Received callback for node 6940367d575bc060ce15786e INFO: 172.21.0.1:44334 - "POST /node/6940367d575bc060ce15786e HTTP/1.0" 202 Accepted 2025-12-17 16:25:40,246 - lava_callback - INFO - Starting async job submit for node 6940367d575bc060ce15786e 2025-12-17 16:25:40,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:40,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940367d575bc060ce15786e HTTP/1.1" 200 None 2025-12-17 16:25:40,255 - lava_callback - DEBUG - Processing node 6940367d575bc060ce15786e: baseline-arm-baylibre 2025-12-17 16:25:40,255 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940367d575bc060ce15786e 2025-12-17 16:25:40,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:40,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940367d575bc060ce15786e HTTP/1.1" 200 None 2025-12-17 16:25:40,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:40,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:40,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:40,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940367d575bc060ce15786e HTTP/1.1" 200 None 2025-12-17 16:25:40,299 - lava_callback - INFO - Completed processing callback for node 6940367d575bc060ce15786e 2025-12-17 16:25:41,216 - lava_callback - INFO - Received callback for node 6942d910e43d2b76e9021fb0 INFO: 172.21.0.1:44346 - "POST /node/6942d910e43d2b76e9021fb0 HTTP/1.0" 202 Accepted 2025-12-17 16:25:41,224 - lava_callback - INFO - Starting async job submit for node 6942d910e43d2b76e9021fb0 2025-12-17 16:25:41,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:41,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d910e43d2b76e9021fb0 HTTP/1.1" 200 None 2025-12-17 16:25:41,260 - lava_callback - DEBUG - Processing node 6942d910e43d2b76e9021fb0: baseline-riscv-broonie 2025-12-17 16:25:41,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:25:41,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:25:41,726 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-riscv-broonie-6942d910e43d2b76e9021fb0/log.txt.gz 2025-12-17 16:25:41,726 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:25:41,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:25:41,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:25:41,857 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-riscv-broonie-6942d910e43d2b76e9021fb0/lava_callback.json.gz 2025-12-17 16:25:41,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:41,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d910e43d2b76e9021fb0 HTTP/1.1" 200 None 2025-12-17 16:25:41,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:41,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4d71b61f1c600601436 HTTP/1.1" 200 None 2025-12-17 16:25:41,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:41,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942d910e43d2b76e9021fb0 HTTP/1.1" 200 None 2025-12-17 16:25:41,961 - lava_callback - INFO - Completed processing callback for node 6942d910e43d2b76e9021fb0 2025-12-17 16:25:43,275 - lava_callback - INFO - Received callback for node 6940330b575bc060ce156f77 INFO: 172.21.0.1:44362 - "POST /node/6940330b575bc060ce156f77 HTTP/1.0" 202 Accepted 2025-12-17 16:25:43,322 - lava_callback - INFO - Starting async job submit for node 6940330b575bc060ce156f77 2025-12-17 16:25:43,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:43,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940330b575bc060ce156f77 HTTP/1.1" 200 None 2025-12-17 16:25:43,387 - lava_callback - DEBUG - Processing node 6940330b575bc060ce156f77: baseline-arm-pengutronix 2025-12-17 16:25:43,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:25:43,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:25:43,669 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-6940330b575bc060ce156f77/log.txt.gz 2025-12-17 16:25:43,669 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:25:43,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:25:43,804 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:25:43,805 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-6940330b575bc060ce156f77/lava_callback.json.gz DEBUG: Setup failed for 6940330b575bc060ce156f77. Transitting job node result: incomplete -> fail 2025-12-17 16:25:43,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:43,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940330b575bc060ce156f77 HTTP/1.1" 200 None 2025-12-17 16:25:43,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:43,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:25:43,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:43,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940330b575bc060ce156f77 HTTP/1.1" 200 None 2025-12-17 16:25:43,918 - lava_callback - INFO - Completed processing callback for node 6940330b575bc060ce156f77 INFO: 172.21.0.1:44376 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:25:47,361 - lava_callback - INFO - Received callback for node 6942d911e43d2b76e9021fb3 INFO: 172.21.0.1:44392 - "POST /node/6942d911e43d2b76e9021fb3 HTTP/1.0" 202 Accepted 2025-12-17 16:25:47,371 - lava_callback - INFO - Starting async job submit for node 6942d911e43d2b76e9021fb3 2025-12-17 16:25:47,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:47,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d911e43d2b76e9021fb3 HTTP/1.1" 200 None 2025-12-17 16:25:47,420 - lava_callback - DEBUG - Processing node 6942d911e43d2b76e9021fb3: baseline-riscv-collabora 2025-12-17 16:25:47,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:25:47,847 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:25:47,848 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-riscv-collabora-6942d911e43d2b76e9021fb3/log.txt.gz 2025-12-17 16:25:47,848 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:25:47,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:25:47,909 - lava_callback - INFO - Received callback for node 69403682575bc060ce15787a INFO: 172.21.0.1:44404 - "POST /node/69403682575bc060ce15787a HTTP/1.0" 202 Accepted 2025-12-17 16:25:47,916 - lava_callback - INFO - Starting async job submit for node 69403682575bc060ce15787a 2025-12-17 16:25:47,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:47,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403682575bc060ce15787a HTTP/1.1" 200 None 2025-12-17 16:25:47,927 - lava_callback - DEBUG - Processing node 69403682575bc060ce15787a: baseline-arm-broonie 2025-12-17 16:25:47,927 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403682575bc060ce15787a 2025-12-17 16:25:47,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:47,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403682575bc060ce15787a HTTP/1.1" 200 None 2025-12-17 16:25:47,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:47,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:47,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:47,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403682575bc060ce15787a HTTP/1.1" 200 None 2025-12-17 16:25:47,972 - lava_callback - INFO - Completed processing callback for node 69403682575bc060ce15787a 2025-12-17 16:25:48,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:25:48,279 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-riscv-collabora-6942d911e43d2b76e9021fb3/lava_callback.json.gz 2025-12-17 16:25:48,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:48,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d911e43d2b76e9021fb3 HTTP/1.1" 200 None 2025-12-17 16:25:48,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:48,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4d71b61f1c600601436 HTTP/1.1" 200 None 2025-12-17 16:25:48,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:48,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942d911e43d2b76e9021fb3 HTTP/1.1" 200 None 2025-12-17 16:25:48,360 - lava_callback - INFO - Completed processing callback for node 6942d911e43d2b76e9021fb3 2025-12-17 16:25:49,431 - lava_callback - INFO - Received callback for node 69403683575bc060ce15787d INFO: 172.21.0.1:44410 - "POST /node/69403683575bc060ce15787d HTTP/1.0" 202 Accepted 2025-12-17 16:25:49,440 - lava_callback - INFO - Starting async job submit for node 69403683575bc060ce15787d 2025-12-17 16:25:49,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:49,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403683575bc060ce15787d HTTP/1.1" 200 None 2025-12-17 16:25:49,452 - lava_callback - DEBUG - Processing node 69403683575bc060ce15787d: baseline-arm-clabbe 2025-12-17 16:25:49,453 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403683575bc060ce15787d 2025-12-17 16:25:49,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:49,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403683575bc060ce15787d HTTP/1.1" 200 None 2025-12-17 16:25:49,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:49,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:49,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:49,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403683575bc060ce15787d HTTP/1.1" 200 None 2025-12-17 16:25:49,494 - lava_callback - INFO - Completed processing callback for node 69403683575bc060ce15787d 2025-12-17 16:25:50,091 - lava_callback - INFO - Received callback for node 69403686575bc060ce157880 INFO: 172.21.0.1:53312 - "POST /node/69403686575bc060ce157880 HTTP/1.0" 202 Accepted 2025-12-17 16:25:50,100 - lava_callback - INFO - Starting async job submit for node 69403686575bc060ce157880 2025-12-17 16:25:50,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:50,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403686575bc060ce157880 HTTP/1.1" 200 None 2025-12-17 16:25:50,115 - lava_callback - DEBUG - Processing node 69403686575bc060ce157880: baseline-arm-clabbe 2025-12-17 16:25:50,115 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403686575bc060ce157880 2025-12-17 16:25:50,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:50,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403686575bc060ce157880 HTTP/1.1" 200 None 2025-12-17 16:25:50,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:50,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:50,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:50,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403686575bc060ce157880 HTTP/1.1" 200 None 2025-12-17 16:25:50,156 - lava_callback - INFO - Completed processing callback for node 69403686575bc060ce157880 2025-12-17 16:25:50,744 - lava_callback - INFO - Received callback for node 69403687575bc060ce157883 INFO: 172.21.0.1:53318 - "POST /node/69403687575bc060ce157883 HTTP/1.0" 202 Accepted 2025-12-17 16:25:50,753 - lava_callback - INFO - Starting async job submit for node 69403687575bc060ce157883 2025-12-17 16:25:50,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:50,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403687575bc060ce157883 HTTP/1.1" 200 None 2025-12-17 16:25:50,766 - lava_callback - DEBUG - Processing node 69403687575bc060ce157883: baseline-arm-clabbe 2025-12-17 16:25:50,766 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403687575bc060ce157883 2025-12-17 16:25:50,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:50,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403687575bc060ce157883 HTTP/1.1" 200 None 2025-12-17 16:25:50,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:50,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:50,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:50,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403687575bc060ce157883 HTTP/1.1" 200 None 2025-12-17 16:25:50,818 - lava_callback - INFO - Completed processing callback for node 69403687575bc060ce157883 2025-12-17 16:25:51,424 - lava_callback - INFO - Received callback for node 69403688575bc060ce15788a INFO: 172.21.0.1:53334 - "POST /node/69403688575bc060ce15788a HTTP/1.0" 202 Accepted 2025-12-17 16:25:51,436 - lava_callback - INFO - Starting async job submit for node 69403688575bc060ce15788a 2025-12-17 16:25:51,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:51,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403688575bc060ce15788a HTTP/1.1" 200 None 2025-12-17 16:25:51,448 - lava_callback - DEBUG - Processing node 69403688575bc060ce15788a: baseline-arm-clabbe 2025-12-17 16:25:51,448 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403688575bc060ce15788a 2025-12-17 16:25:51,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:51,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403688575bc060ce15788a HTTP/1.1" 200 None 2025-12-17 16:25:51,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:51,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:51,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:51,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403688575bc060ce15788a HTTP/1.1" 200 None 2025-12-17 16:25:51,502 - lava_callback - INFO - Completed processing callback for node 69403688575bc060ce15788a 2025-12-17 16:25:52,119 - lava_callback - INFO - Received callback for node 6940368a575bc060ce157893 INFO: 172.21.0.1:53338 - "POST /node/6940368a575bc060ce157893 HTTP/1.0" 202 Accepted 2025-12-17 16:25:52,129 - lava_callback - INFO - Starting async job submit for node 6940368a575bc060ce157893 2025-12-17 16:25:52,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:52,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940368a575bc060ce157893 HTTP/1.1" 200 None 2025-12-17 16:25:52,146 - lava_callback - DEBUG - Processing node 6940368a575bc060ce157893: baseline-arm-clabbe 2025-12-17 16:25:52,146 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940368a575bc060ce157893 2025-12-17 16:25:52,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:52,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940368a575bc060ce157893 HTTP/1.1" 200 None 2025-12-17 16:25:52,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:52,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:52,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:52,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940368a575bc060ce157893 HTTP/1.1" 200 None 2025-12-17 16:25:52,213 - lava_callback - INFO - Completed processing callback for node 6940368a575bc060ce157893 2025-12-17 16:25:54,920 - lava_callback - INFO - Received callback for node 6940367e575bc060ce157871 INFO: 172.21.0.1:53342 - "POST /node/6940367e575bc060ce157871 HTTP/1.0" 202 Accepted 2025-12-17 16:25:54,929 - lava_callback - INFO - Starting async job submit for node 6940367e575bc060ce157871 2025-12-17 16:25:54,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:54,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940367e575bc060ce157871 HTTP/1.1" 200 None 2025-12-17 16:25:54,941 - lava_callback - DEBUG - Processing node 6940367e575bc060ce157871: baseline-arm-baylibre 2025-12-17 16:25:54,941 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940367e575bc060ce157871 2025-12-17 16:25:54,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:54,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940367e575bc060ce157871 HTTP/1.1" 200 None 2025-12-17 16:25:54,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:54,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:54,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:55,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940367e575bc060ce157871 HTTP/1.1" 200 None 2025-12-17 16:25:55,001 - lava_callback - INFO - Completed processing callback for node 6940367e575bc060ce157871 2025-12-17 16:25:55,608 - lava_callback - INFO - Received callback for node 69403680575bc060ce157874 INFO: 172.21.0.1:53346 - "POST /node/69403680575bc060ce157874 HTTP/1.0" 202 Accepted 2025-12-17 16:25:55,617 - lava_callback - INFO - Starting async job submit for node 69403680575bc060ce157874 2025-12-17 16:25:55,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:55,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403680575bc060ce157874 HTTP/1.1" 200 None 2025-12-17 16:25:55,633 - lava_callback - DEBUG - Processing node 69403680575bc060ce157874: baseline-arm-baylibre 2025-12-17 16:25:55,634 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403680575bc060ce157874 2025-12-17 16:25:55,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:55,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403680575bc060ce157874 HTTP/1.1" 200 None 2025-12-17 16:25:55,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:55,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:55,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:55,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403680575bc060ce157874 HTTP/1.1" 200 None 2025-12-17 16:25:55,688 - lava_callback - INFO - Completed processing callback for node 69403680575bc060ce157874 2025-12-17 16:25:56,293 - lava_callback - INFO - Received callback for node 69403681575bc060ce157877 INFO: 172.21.0.1:53360 - "POST /node/69403681575bc060ce157877 HTTP/1.0" 202 Accepted 2025-12-17 16:25:56,300 - lava_callback - INFO - Starting async job submit for node 69403681575bc060ce157877 2025-12-17 16:25:56,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:56,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403681575bc060ce157877 HTTP/1.1" 200 None 2025-12-17 16:25:56,310 - lava_callback - DEBUG - Processing node 69403681575bc060ce157877: baseline-arm-baylibre 2025-12-17 16:25:56,310 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403681575bc060ce157877 2025-12-17 16:25:56,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:56,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403681575bc060ce157877 HTTP/1.1" 200 None 2025-12-17 16:25:56,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:56,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:25:56,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:25:56,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403681575bc060ce157877 HTTP/1.1" 200 None 2025-12-17 16:25:56,357 - lava_callback - INFO - Completed processing callback for node 69403681575bc060ce157877 INFO: 172.21.0.1:53366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50218 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:26:07,933 - lava_callback - INFO - Received callback for node 6940369d575bc060ce1578b8 INFO: 172.21.0.1:50230 - "POST /node/6940369d575bc060ce1578b8 HTTP/1.0" 202 Accepted 2025-12-17 16:26:07,940 - lava_callback - INFO - Starting async job submit for node 6940369d575bc060ce1578b8 2025-12-17 16:26:07,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:07,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940369d575bc060ce1578b8 HTTP/1.1" 200 None 2025-12-17 16:26:07,952 - lava_callback - DEBUG - Processing node 6940369d575bc060ce1578b8: ltp-syscalls 2025-12-17 16:26:07,952 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940369d575bc060ce1578b8 2025-12-17 16:26:07,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:07,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940369d575bc060ce1578b8 HTTP/1.1" 200 None 2025-12-17 16:26:07,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:07,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:26:07,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:07,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940369d575bc060ce1578b8 HTTP/1.1" 200 None 2025-12-17 16:26:07,995 - lava_callback - INFO - Completed processing callback for node 6940369d575bc060ce1578b8 2025-12-17 16:26:09,459 - lava_callback - INFO - Received callback for node 6940368c575bc060ce157899 INFO: 172.21.0.1:50234 - "POST /node/6940368c575bc060ce157899 HTTP/1.0" 202 Accepted 2025-12-17 16:26:09,468 - lava_callback - INFO - Starting async job submit for node 6940368c575bc060ce157899 2025-12-17 16:26:09,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:09,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940368c575bc060ce157899 HTTP/1.1" 200 None 2025-12-17 16:26:09,480 - lava_callback - DEBUG - Processing node 6940368c575bc060ce157899: baseline-arm-clabbe 2025-12-17 16:26:09,480 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940368c575bc060ce157899 2025-12-17 16:26:09,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:09,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940368c575bc060ce157899 HTTP/1.1" 200 None 2025-12-17 16:26:09,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:09,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:26:09,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:09,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940368c575bc060ce157899 HTTP/1.1" 200 None 2025-12-17 16:26:09,546 - lava_callback - INFO - Completed processing callback for node 6940368c575bc060ce157899 2025-12-17 16:26:10,132 - lava_callback - INFO - Received callback for node 6940368e575bc060ce1578a5 INFO: 172.21.0.1:42692 - "POST /node/6940368e575bc060ce1578a5 HTTP/1.0" 202 Accepted 2025-12-17 16:26:10,142 - lava_callback - INFO - Starting async job submit for node 6940368e575bc060ce1578a5 2025-12-17 16:26:10,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:10,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940368e575bc060ce1578a5 HTTP/1.1" 200 None 2025-12-17 16:26:10,155 - lava_callback - DEBUG - Processing node 6940368e575bc060ce1578a5: baseline-arm-clabbe 2025-12-17 16:26:10,155 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940368e575bc060ce1578a5 2025-12-17 16:26:10,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:10,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940368e575bc060ce1578a5 HTTP/1.1" 200 None 2025-12-17 16:26:10,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:10,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:26:10,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:10,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940368e575bc060ce1578a5 HTTP/1.1" 200 None 2025-12-17 16:26:10,198 - lava_callback - INFO - Completed processing callback for node 6940368e575bc060ce1578a5 INFO: 172.21.0.1:42696 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:26:10,817 - lava_callback - INFO - Received callback for node 69403690575bc060ce1578a9 INFO: 172.21.0.1:42708 - "POST /node/69403690575bc060ce1578a9 HTTP/1.0" 202 Accepted 2025-12-17 16:26:10,825 - lava_callback - INFO - Starting async job submit for node 69403690575bc060ce1578a9 2025-12-17 16:26:10,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:10,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403690575bc060ce1578a9 HTTP/1.1" 200 None 2025-12-17 16:26:10,837 - lava_callback - DEBUG - Processing node 69403690575bc060ce1578a9: baseline-arm-clabbe 2025-12-17 16:26:10,837 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403690575bc060ce1578a9 2025-12-17 16:26:10,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:10,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403690575bc060ce1578a9 HTTP/1.1" 200 None 2025-12-17 16:26:10,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:10,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:26:10,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:10,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403690575bc060ce1578a9 HTTP/1.1" 200 None 2025-12-17 16:26:10,877 - lava_callback - INFO - Completed processing callback for node 69403690575bc060ce1578a9 2025-12-17 16:26:11,493 - lava_callback - INFO - Received callback for node 69403692575bc060ce1578ac INFO: 172.21.0.1:42724 - "POST /node/69403692575bc060ce1578ac HTTP/1.0" 202 Accepted 2025-12-17 16:26:11,502 - lava_callback - INFO - Starting async job submit for node 69403692575bc060ce1578ac 2025-12-17 16:26:11,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:11,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403692575bc060ce1578ac HTTP/1.1" 200 None 2025-12-17 16:26:11,515 - lava_callback - DEBUG - Processing node 69403692575bc060ce1578ac: baseline-arm-clabbe 2025-12-17 16:26:11,515 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403692575bc060ce1578ac 2025-12-17 16:26:11,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:11,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403692575bc060ce1578ac HTTP/1.1" 200 None 2025-12-17 16:26:11,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:11,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:26:11,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:11,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403692575bc060ce1578ac HTTP/1.1" 200 None 2025-12-17 16:26:11,583 - lava_callback - INFO - Completed processing callback for node 69403692575bc060ce1578ac 2025-12-17 16:26:12,159 - lava_callback - INFO - Received callback for node 69403694575bc060ce1578af INFO: 172.21.0.1:42740 - "POST /node/69403694575bc060ce1578af HTTP/1.0" 202 Accepted 2025-12-17 16:26:12,168 - lava_callback - INFO - Starting async job submit for node 69403694575bc060ce1578af 2025-12-17 16:26:12,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:12,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403694575bc060ce1578af HTTP/1.1" 200 None 2025-12-17 16:26:12,180 - lava_callback - DEBUG - Processing node 69403694575bc060ce1578af: baseline-arm-clabbe 2025-12-17 16:26:12,180 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403694575bc060ce1578af 2025-12-17 16:26:12,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:12,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403694575bc060ce1578af HTTP/1.1" 200 None 2025-12-17 16:26:12,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:12,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:26:12,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:12,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403694575bc060ce1578af HTTP/1.1" 200 None 2025-12-17 16:26:12,218 - lava_callback - INFO - Completed processing callback for node 69403694575bc060ce1578af 2025-12-17 16:26:14,665 - lava_callback - INFO - Received callback for node 694036a2575bc060ce1578c6 INFO: 172.21.0.1:42742 - "POST /node/694036a2575bc060ce1578c6 HTTP/1.0" 202 Accepted 2025-12-17 16:26:14,675 - lava_callback - INFO - Starting async job submit for node 694036a2575bc060ce1578c6 2025-12-17 16:26:14,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:14,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036a2575bc060ce1578c6 HTTP/1.1" 200 None 2025-12-17 16:26:14,687 - lava_callback - DEBUG - Processing node 694036a2575bc060ce1578c6: baseline-x86-kcidebug-intel 2025-12-17 16:26:14,687 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036a2575bc060ce1578c6 2025-12-17 16:26:14,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:14,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036a2575bc060ce1578c6 HTTP/1.1" 200 None 2025-12-17 16:26:14,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:14,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d8e44c18b79b848b5fc61 HTTP/1.1" 200 None 2025-12-17 16:26:14,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:14,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036a2575bc060ce1578c6 HTTP/1.1" 200 None 2025-12-17 16:26:14,730 - lava_callback - INFO - Completed processing callback for node 694036a2575bc060ce1578c6 INFO: 172.21.0.1:42752 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:26:15,419 - lava_callback - INFO - Received callback for node 694036a2575bc060ce1578cb INFO: 172.21.0.1:42764 - "POST /node/694036a2575bc060ce1578cb HTTP/1.0" 202 Accepted 2025-12-17 16:26:15,425 - lava_callback - INFO - Starting async job submit for node 694036a2575bc060ce1578cb 2025-12-17 16:26:15,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:15,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036a2575bc060ce1578cb HTTP/1.1" 200 None 2025-12-17 16:26:15,435 - lava_callback - DEBUG - Processing node 694036a2575bc060ce1578cb: baseline-arm 2025-12-17 16:26:15,435 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036a2575bc060ce1578cb 2025-12-17 16:26:15,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:15,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036a2575bc060ce1578cb HTTP/1.1" 200 None 2025-12-17 16:26:15,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:15,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:26:15,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:15,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036a2575bc060ce1578cb HTTP/1.1" 200 None 2025-12-17 16:26:15,468 - lava_callback - INFO - Completed processing callback for node 694036a2575bc060ce1578cb 2025-12-17 16:26:16,172 - lava_callback - INFO - Received callback for node 694036a4575bc060ce1578d2 INFO: 172.21.0.1:42770 - "POST /node/694036a4575bc060ce1578d2 HTTP/1.0" 202 Accepted 2025-12-17 16:26:16,179 - lava_callback - INFO - Starting async job submit for node 694036a4575bc060ce1578d2 2025-12-17 16:26:16,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:16,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036a4575bc060ce1578d2 HTTP/1.1" 200 None 2025-12-17 16:26:16,189 - lava_callback - DEBUG - Processing node 694036a4575bc060ce1578d2: baseline-arm 2025-12-17 16:26:16,190 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036a4575bc060ce1578d2 2025-12-17 16:26:16,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:16,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036a4575bc060ce1578d2 HTTP/1.1" 200 None 2025-12-17 16:26:16,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:16,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:26:16,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:16,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036a4575bc060ce1578d2 HTTP/1.1" 200 None 2025-12-17 16:26:16,234 - lava_callback - INFO - Completed processing callback for node 694036a4575bc060ce1578d2 INFO: 172.21.0.1:42784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54584 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:26:27,888 - lava_callback - INFO - Received callback for node 694036a1575bc060ce1578bf INFO: 172.21.0.1:54600 - "POST /node/694036a1575bc060ce1578bf HTTP/1.0" 202 Accepted 2025-12-17 16:26:27,898 - lava_callback - INFO - Starting async job submit for node 694036a1575bc060ce1578bf 2025-12-17 16:26:27,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:27,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036a1575bc060ce1578bf HTTP/1.1" 200 None 2025-12-17 16:26:27,918 - lava_callback - DEBUG - Processing node 694036a1575bc060ce1578bf: ltp-syscalls 2025-12-17 16:26:27,918 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036a1575bc060ce1578bf 2025-12-17 16:26:27,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:27,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036a1575bc060ce1578bf HTTP/1.1" 200 None 2025-12-17 16:26:27,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:27,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:26:27,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:27,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036a1575bc060ce1578bf HTTP/1.1" 200 None 2025-12-17 16:26:27,969 - lava_callback - INFO - Completed processing callback for node 694036a1575bc060ce1578bf 2025-12-17 16:26:34,951 - lava_callback - INFO - Received callback for node 694036a6575bc060ce1578d5 INFO: 172.21.0.1:49268 - "POST /node/694036a6575bc060ce1578d5 HTTP/1.0" 202 Accepted 2025-12-17 16:26:34,960 - lava_callback - INFO - Starting async job submit for node 694036a6575bc060ce1578d5 2025-12-17 16:26:34,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:34,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036a6575bc060ce1578d5 HTTP/1.1" 200 None 2025-12-17 16:26:34,980 - lava_callback - DEBUG - Processing node 694036a6575bc060ce1578d5: baseline-arm-baylibre 2025-12-17 16:26:34,980 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036a6575bc060ce1578d5 2025-12-17 16:26:34,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:34,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036a6575bc060ce1578d5 HTTP/1.1" 200 None 2025-12-17 16:26:34,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:35,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:26:35,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:35,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036a6575bc060ce1578d5 HTTP/1.1" 200 None 2025-12-17 16:26:35,041 - lava_callback - INFO - Completed processing callback for node 694036a6575bc060ce1578d5 2025-12-17 16:26:35,643 - lava_callback - INFO - Received callback for node 694036a8575bc060ce1578d8 INFO: 172.21.0.1:49272 - "POST /node/694036a8575bc060ce1578d8 HTTP/1.0" 202 Accepted 2025-12-17 16:26:35,654 - lava_callback - INFO - Starting async job submit for node 694036a8575bc060ce1578d8 2025-12-17 16:26:35,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:35,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036a8575bc060ce1578d8 HTTP/1.1" 200 None 2025-12-17 16:26:35,666 - lava_callback - DEBUG - Processing node 694036a8575bc060ce1578d8: baseline-arm-baylibre 2025-12-17 16:26:35,666 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036a8575bc060ce1578d8 2025-12-17 16:26:35,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:35,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036a8575bc060ce1578d8 HTTP/1.1" 200 None 2025-12-17 16:26:35,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:35,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:26:35,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:35,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036a8575bc060ce1578d8 HTTP/1.1" 200 None 2025-12-17 16:26:35,723 - lava_callback - INFO - Completed processing callback for node 694036a8575bc060ce1578d8 2025-12-17 16:26:36,317 - lava_callback - INFO - Received callback for node 694036aa575bc060ce1578db INFO: 172.21.0.1:49276 - "POST /node/694036aa575bc060ce1578db HTTP/1.0" 202 Accepted 2025-12-17 16:26:36,330 - lava_callback - INFO - Starting async job submit for node 694036aa575bc060ce1578db 2025-12-17 16:26:36,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:36,332 - lava_callback - INFO - Received callback for node 694039bb575bc060ce1581fe INFO: 172.21.0.1:49288 - "POST /node/694039bb575bc060ce1581fe HTTP/1.0" 202 Accepted 2025-12-17 16:26:36,361 - lava_callback - INFO - Starting async job submit for node 694039bb575bc060ce1581fe 2025-12-17 16:26:36,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036aa575bc060ce1578db HTTP/1.1" 200 None 2025-12-17 16:26:36,374 - lava_callback - DEBUG - Processing node 694036aa575bc060ce1578db: baseline-arm-baylibre 2025-12-17 16:26:36,376 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036aa575bc060ce1578db 2025-12-17 16:26:36,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:36,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036aa575bc060ce1578db HTTP/1.1" 200 None 2025-12-17 16:26:36,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:36,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:26:36,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:36,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:36,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039bb575bc060ce1581fe HTTP/1.1" 200 None 2025-12-17 16:26:36,554 - lava_callback - DEBUG - Processing node 694039bb575bc060ce1581fe: baseline-nfs-arm64 2025-12-17 16:26:36,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036aa575bc060ce1578db HTTP/1.1" 200 None 2025-12-17 16:26:36,723 - lava_callback - INFO - Completed processing callback for node 694036aa575bc060ce1578db INFO: 172.21.0.1:49300 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:26:37,014 - lava_callback - INFO - Received callback for node 694036ab575bc060ce1578de INFO: 172.21.0.1:49304 - "POST /node/694036ab575bc060ce1578de HTTP/1.0" 202 Accepted 2025-12-17 16:26:37,034 - lava_callback - INFO - Starting async job submit for node 694036ab575bc060ce1578de 2025-12-17 16:26:37,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:37,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ab575bc060ce1578de HTTP/1.1" 200 None 2025-12-17 16:26:37,072 - lava_callback - DEBUG - Processing node 694036ab575bc060ce1578de: baseline-arm-baylibre 2025-12-17 16:26:37,072 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036ab575bc060ce1578de 2025-12-17 16:26:37,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:37,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ab575bc060ce1578de HTTP/1.1" 200 None 2025-12-17 16:26:37,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:37,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:26:37,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:37,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036ab575bc060ce1578de HTTP/1.1" 200 None 2025-12-17 16:26:37,249 - lava_callback - INFO - Completed processing callback for node 694036ab575bc060ce1578de 2025-12-17 16:26:37,733 - lava_callback - INFO - Received callback for node 694036ad575bc060ce1578e1 INFO: 172.21.0.1:49314 - "POST /node/694036ad575bc060ce1578e1 HTTP/1.0" 202 Accepted 2025-12-17 16:26:37,742 - lava_callback - INFO - Starting async job submit for node 694036ad575bc060ce1578e1 2025-12-17 16:26:37,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:37,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ad575bc060ce1578e1 HTTP/1.1" 200 None 2025-12-17 16:26:37,800 - lava_callback - DEBUG - Processing node 694036ad575bc060ce1578e1: baseline-arm-baylibre 2025-12-17 16:26:37,800 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036ad575bc060ce1578e1 2025-12-17 16:26:37,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:37,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ad575bc060ce1578e1 HTTP/1.1" 200 None 2025-12-17 16:26:37,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:37,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:26:37,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:37,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036ad575bc060ce1578e1 HTTP/1.1" 200 None 2025-12-17 16:26:37,950 - lava_callback - INFO - Completed processing callback for node 694036ad575bc060ce1578e1 2025-12-17 16:26:38,429 - lava_callback - INFO - Received callback for node 694036af575bc060ce1578e4 2025-12-17 16:26:38,451 - lava_callback - INFO - Starting async job submit for node 694036af575bc060ce1578e4 2025-12-17 16:26:38,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:49326 - "POST /node/694036af575bc060ce1578e4 HTTP/1.0" 202 Accepted 2025-12-17 16:26:38,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036af575bc060ce1578e4 HTTP/1.1" 200 None 2025-12-17 16:26:38,495 - lava_callback - DEBUG - Processing node 694036af575bc060ce1578e4: baseline-arm-baylibre 2025-12-17 16:26:38,496 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036af575bc060ce1578e4 2025-12-17 16:26:38,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:38,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036af575bc060ce1578e4 HTTP/1.1" 200 None 2025-12-17 16:26:38,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:38,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:26:38,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:38,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036af575bc060ce1578e4 HTTP/1.1" 200 None 2025-12-17 16:26:38,673 - lava_callback - INFO - Completed processing callback for node 694036af575bc060ce1578e4 2025-12-17 16:26:38,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:26:39,136 - lava_callback - INFO - Received callback for node 694036b1575bc060ce1578e9 INFO: 172.21.0.1:49330 - "POST /node/694036b1575bc060ce1578e9 HTTP/1.0" 202 Accepted 2025-12-17 16:26:39,145 - lava_callback - INFO - Starting async job submit for node 694036b1575bc060ce1578e9 2025-12-17 16:26:39,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:39,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036b1575bc060ce1578e9 HTTP/1.1" 200 None 2025-12-17 16:26:39,160 - lava_callback - DEBUG - Processing node 694036b1575bc060ce1578e9: baseline-arm-baylibre 2025-12-17 16:26:39,160 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036b1575bc060ce1578e9 2025-12-17 16:26:39,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:39,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036b1575bc060ce1578e9 HTTP/1.1" 200 None 2025-12-17 16:26:39,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:39,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:26:39,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:39,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036b1575bc060ce1578e9 HTTP/1.1" 200 None 2025-12-17 16:26:39,239 - lava_callback - INFO - Completed processing callback for node 694036b1575bc060ce1578e9 2025-12-17 16:26:39,825 - lava_callback - INFO - Received callback for node 694036b3575bc060ce1578ec INFO: 172.21.0.1:33798 - "POST /node/694036b3575bc060ce1578ec HTTP/1.0" 202 Accepted 2025-12-17 16:26:39,835 - lava_callback - INFO - Starting async job submit for node 694036b3575bc060ce1578ec 2025-12-17 16:26:39,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:39,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036b3575bc060ce1578ec HTTP/1.1" 200 None 2025-12-17 16:26:39,850 - lava_callback - DEBUG - Processing node 694036b3575bc060ce1578ec: baseline-arm-baylibre 2025-12-17 16:26:39,850 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036b3575bc060ce1578ec 2025-12-17 16:26:39,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:39,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036b3575bc060ce1578ec HTTP/1.1" 200 None 2025-12-17 16:26:39,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:39,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:26:39,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:39,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036b3575bc060ce1578ec HTTP/1.1" 200 None 2025-12-17 16:26:39,907 - lava_callback - INFO - Completed processing callback for node 694036b3575bc060ce1578ec 2025-12-17 16:26:40,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:26:40,101 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694039bb575bc060ce1581fe/log.txt.gz 2025-12-17 16:26:40,101 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:26:40,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:26:40,499 - lava_callback - INFO - Received callback for node 694036b5575bc060ce1578ef INFO: 172.21.0.1:33804 - "POST /node/694036b5575bc060ce1578ef HTTP/1.0" 202 Accepted 2025-12-17 16:26:40,509 - lava_callback - INFO - Starting async job submit for node 694036b5575bc060ce1578ef 2025-12-17 16:26:40,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:40,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036b5575bc060ce1578ef HTTP/1.1" 200 None 2025-12-17 16:26:40,524 - lava_callback - DEBUG - Processing node 694036b5575bc060ce1578ef: baseline-arm-baylibre 2025-12-17 16:26:40,524 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036b5575bc060ce1578ef 2025-12-17 16:26:40,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:40,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036b5575bc060ce1578ef HTTP/1.1" 200 None 2025-12-17 16:26:40,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:40,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:26:40,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:40,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036b5575bc060ce1578ef HTTP/1.1" 200 None 2025-12-17 16:26:40,582 - lava_callback - INFO - Completed processing callback for node 694036b5575bc060ce1578ef 2025-12-17 16:26:41,599 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:26:41,601 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694039bb575bc060ce1581fe/lava_callback.json.gz DEBUG: Setup failed for 694039bb575bc060ce1581fe. Transitting job node result: incomplete -> fail 2025-12-17 16:26:41,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:41,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039bb575bc060ce1581fe HTTP/1.1" 200 None 2025-12-17 16:26:41,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:41,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031f2575bc060ce156c76 HTTP/1.1" 200 None 2025-12-17 16:26:41,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:41,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039bb575bc060ce1581fe HTTP/1.1" 200 None 2025-12-17 16:26:41,901 - lava_callback - INFO - Completed processing callback for node 694039bb575bc060ce1581fe INFO: 172.21.0.1:33810 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:26:54,976 - lava_callback - INFO - Received callback for node 694036bc575bc060ce15790a INFO: 172.21.0.1:58476 - "POST /node/694036bc575bc060ce15790a HTTP/1.0" 202 Accepted 2025-12-17 16:26:54,985 - lava_callback - INFO - Starting async job submit for node 694036bc575bc060ce15790a 2025-12-17 16:26:54,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:54,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036bc575bc060ce15790a HTTP/1.1" 200 None 2025-12-17 16:26:54,996 - lava_callback - DEBUG - Processing node 694036bc575bc060ce15790a: baseline-arm64-baylibre 2025-12-17 16:26:54,996 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036bc575bc060ce15790a 2025-12-17 16:26:54,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:55,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036bc575bc060ce15790a HTTP/1.1" 200 None 2025-12-17 16:26:55,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:55,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:26:55,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:55,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036bc575bc060ce15790a HTTP/1.1" 200 None 2025-12-17 16:26:55,073 - lava_callback - INFO - Completed processing callback for node 694036bc575bc060ce15790a 2025-12-17 16:26:55,641 - lava_callback - INFO - Received callback for node 694036ba575bc060ce157905 INFO: 172.21.0.1:58490 - "POST /node/694036ba575bc060ce157905 HTTP/1.0" 202 Accepted 2025-12-17 16:26:55,649 - lava_callback - INFO - Starting async job submit for node 694036ba575bc060ce157905 2025-12-17 16:26:55,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:55,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ba575bc060ce157905 HTTP/1.1" 200 None 2025-12-17 16:26:55,662 - lava_callback - DEBUG - Processing node 694036ba575bc060ce157905: baseline-arm-baylibre 2025-12-17 16:26:55,662 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036ba575bc060ce157905 2025-12-17 16:26:55,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:55,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ba575bc060ce157905 HTTP/1.1" 200 None 2025-12-17 16:26:55,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:55,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:26:55,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:55,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036ba575bc060ce157905 HTTP/1.1" 200 None 2025-12-17 16:26:55,708 - lava_callback - INFO - Completed processing callback for node 694036ba575bc060ce157905 2025-12-17 16:26:56,332 - lava_callback - INFO - Received callback for node 694036be575bc060ce15790d INFO: 172.21.0.1:58502 - "POST /node/694036be575bc060ce15790d HTTP/1.0" 202 Accepted 2025-12-17 16:26:56,342 - lava_callback - INFO - Starting async job submit for node 694036be575bc060ce15790d 2025-12-17 16:26:56,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:56,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036be575bc060ce15790d HTTP/1.1" 200 None 2025-12-17 16:26:56,358 - lava_callback - DEBUG - Processing node 694036be575bc060ce15790d: baseline-arm-baylibre 2025-12-17 16:26:56,359 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036be575bc060ce15790d 2025-12-17 16:26:56,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:56,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036be575bc060ce15790d HTTP/1.1" 200 None 2025-12-17 16:26:56,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:56,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:26:56,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:56,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036be575bc060ce15790d HTTP/1.1" 200 None 2025-12-17 16:26:56,414 - lava_callback - INFO - Completed processing callback for node 694036be575bc060ce15790d INFO: 172.21.0.1:58518 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:26:57,006 - lava_callback - INFO - Received callback for node 694036c0575bc060ce157910 INFO: 172.21.0.1:58524 - "POST /node/694036c0575bc060ce157910 HTTP/1.0" 202 Accepted 2025-12-17 16:26:57,014 - lava_callback - INFO - Starting async job submit for node 694036c0575bc060ce157910 2025-12-17 16:26:57,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:57,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036c0575bc060ce157910 HTTP/1.1" 200 None 2025-12-17 16:26:57,054 - lava_callback - DEBUG - Processing node 694036c0575bc060ce157910: baseline-arm64-baylibre 2025-12-17 16:26:57,054 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036c0575bc060ce157910 2025-12-17 16:26:57,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:57,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036c0575bc060ce157910 HTTP/1.1" 200 None 2025-12-17 16:26:57,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:57,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:26:57,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:57,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036c0575bc060ce157910 HTTP/1.1" 200 None 2025-12-17 16:26:57,125 - lava_callback - INFO - Completed processing callback for node 694036c0575bc060ce157910 2025-12-17 16:26:57,700 - lava_callback - INFO - Received callback for node 694036c3575bc060ce157913 INFO: 172.21.0.1:58534 - "POST /node/694036c3575bc060ce157913 HTTP/1.0" 202 Accepted 2025-12-17 16:26:57,712 - lava_callback - INFO - Starting async job submit for node 694036c3575bc060ce157913 2025-12-17 16:26:57,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:57,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036c3575bc060ce157913 HTTP/1.1" 200 None 2025-12-17 16:26:57,731 - lava_callback - DEBUG - Processing node 694036c3575bc060ce157913: baseline-arm-baylibre 2025-12-17 16:26:57,731 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036c3575bc060ce157913 2025-12-17 16:26:57,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:57,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036c3575bc060ce157913 HTTP/1.1" 200 None 2025-12-17 16:26:57,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:57,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:26:57,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:57,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036c3575bc060ce157913 HTTP/1.1" 200 None 2025-12-17 16:26:57,786 - lava_callback - INFO - Completed processing callback for node 694036c3575bc060ce157913 2025-12-17 16:26:58,385 - lava_callback - INFO - Received callback for node 694036c5575bc060ce157916 INFO: 172.21.0.1:58538 - "POST /node/694036c5575bc060ce157916 HTTP/1.0" 202 Accepted 2025-12-17 16:26:58,394 - lava_callback - INFO - Starting async job submit for node 694036c5575bc060ce157916 2025-12-17 16:26:58,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:58,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036c5575bc060ce157916 HTTP/1.1" 200 None 2025-12-17 16:26:58,653 - lava_callback - DEBUG - Processing node 694036c5575bc060ce157916: baseline-arm64-baylibre 2025-12-17 16:26:58,653 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036c5575bc060ce157916 2025-12-17 16:26:58,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:58,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036c5575bc060ce157916 HTTP/1.1" 200 None 2025-12-17 16:26:58,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:58,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:26:58,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:58,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036c5575bc060ce157916 HTTP/1.1" 200 None 2025-12-17 16:26:58,704 - lava_callback - INFO - Completed processing callback for node 694036c5575bc060ce157916 2025-12-17 16:26:59,054 - lava_callback - INFO - Received callback for node 694036c8575bc060ce15791e INFO: 172.21.0.1:58542 - "POST /node/694036c8575bc060ce15791e HTTP/1.0" 202 Accepted 2025-12-17 16:26:59,064 - lava_callback - INFO - Starting async job submit for node 694036c8575bc060ce15791e 2025-12-17 16:26:59,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:59,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036c8575bc060ce15791e HTTP/1.1" 200 None 2025-12-17 16:26:59,078 - lava_callback - DEBUG - Processing node 694036c8575bc060ce15791e: baseline-arm64-baylibre 2025-12-17 16:26:59,078 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036c8575bc060ce15791e 2025-12-17 16:26:59,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:59,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036c8575bc060ce15791e HTTP/1.1" 200 None 2025-12-17 16:26:59,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:59,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:26:59,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:59,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036c8575bc060ce15791e HTTP/1.1" 200 None 2025-12-17 16:26:59,133 - lava_callback - INFO - Completed processing callback for node 694036c8575bc060ce15791e 2025-12-17 16:26:59,749 - lava_callback - INFO - Received callback for node 694036cb575bc060ce157925 INFO: 172.21.0.1:44170 - "POST /node/694036cb575bc060ce157925 HTTP/1.0" 202 Accepted 2025-12-17 16:26:59,758 - lava_callback - INFO - Starting async job submit for node 694036cb575bc060ce157925 2025-12-17 16:26:59,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:59,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036cb575bc060ce157925 HTTP/1.1" 200 None 2025-12-17 16:26:59,770 - lava_callback - DEBUG - Processing node 694036cb575bc060ce157925: baseline-arm64-baylibre 2025-12-17 16:26:59,770 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036cb575bc060ce157925 2025-12-17 16:26:59,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:59,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036cb575bc060ce157925 HTTP/1.1" 200 None 2025-12-17 16:26:59,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:59,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:26:59,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:26:59,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036cb575bc060ce157925 HTTP/1.1" 200 None 2025-12-17 16:26:59,817 - lava_callback - INFO - Completed processing callback for node 694036cb575bc060ce157925 INFO: 172.21.0.1:44184 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:27:08,058 - lava_callback - INFO - Received callback for node 694036d4575bc060ce157932 INFO: 172.21.0.1:44186 - "POST /node/694036d4575bc060ce157932 HTTP/1.0" 202 Accepted 2025-12-17 16:27:08,072 - lava_callback - INFO - Starting async job submit for node 694036d4575bc060ce157932 2025-12-17 16:27:08,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:08,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036d4575bc060ce157932 HTTP/1.1" 200 None 2025-12-17 16:27:08,087 - lava_callback - DEBUG - Processing node 694036d4575bc060ce157932: baseline-arm-broonie 2025-12-17 16:27:08,087 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036d4575bc060ce157932 2025-12-17 16:27:08,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:08,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036d4575bc060ce157932 HTTP/1.1" 200 None 2025-12-17 16:27:08,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:08,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:27:08,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:08,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036d4575bc060ce157932 HTTP/1.1" 200 None 2025-12-17 16:27:08,139 - lava_callback - INFO - Completed processing callback for node 694036d4575bc060ce157932 2025-12-17 16:27:08,944 - lava_callback - INFO - Received callback for node 694036d9575bc060ce15793c INFO: 172.21.0.1:44202 - "POST /node/694036d9575bc060ce15793c HTTP/1.0" 202 Accepted 2025-12-17 16:27:08,958 - lava_callback - INFO - Starting async job submit for node 694036d9575bc060ce15793c 2025-12-17 16:27:08,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:08,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036d9575bc060ce15793c HTTP/1.1" 200 None 2025-12-17 16:27:08,979 - lava_callback - DEBUG - Processing node 694036d9575bc060ce15793c: baseline-arm-broonie 2025-12-17 16:27:08,979 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036d9575bc060ce15793c 2025-12-17 16:27:08,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:08,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036d9575bc060ce15793c HTTP/1.1" 200 None 2025-12-17 16:27:08,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:09,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:27:09,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:09,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036d9575bc060ce15793c HTTP/1.1" 200 None 2025-12-17 16:27:09,045 - lava_callback - INFO - Completed processing callback for node 694036d9575bc060ce15793c 2025-12-17 16:27:09,499 - lava_callback - INFO - Received callback for node 694036da575bc060ce15793f INFO: 172.21.0.1:44212 - "POST /node/694036da575bc060ce15793f HTTP/1.0" 202 Accepted 2025-12-17 16:27:09,511 - lava_callback - INFO - Starting async job submit for node 694036da575bc060ce15793f 2025-12-17 16:27:09,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:09,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036da575bc060ce15793f HTTP/1.1" 200 None 2025-12-17 16:27:09,537 - lava_callback - DEBUG - Processing node 694036da575bc060ce15793f: baseline-arm-clabbe 2025-12-17 16:27:09,537 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036da575bc060ce15793f 2025-12-17 16:27:09,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:09,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036da575bc060ce15793f HTTP/1.1" 200 None 2025-12-17 16:27:09,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:09,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:27:09,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:09,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036da575bc060ce15793f HTTP/1.1" 200 None 2025-12-17 16:27:09,592 - lava_callback - INFO - Completed processing callback for node 694036da575bc060ce15793f INFO: 172.21.0.1:37268 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:27:15,009 - lava_callback - INFO - Received callback for node 694036cf575bc060ce157928 INFO: 172.21.0.1:37284 - "POST /node/694036cf575bc060ce157928 HTTP/1.0" 202 Accepted 2025-12-17 16:27:15,019 - lava_callback - INFO - Starting async job submit for node 694036cf575bc060ce157928 2025-12-17 16:27:15,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:15,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036cf575bc060ce157928 HTTP/1.1" 200 None 2025-12-17 16:27:15,034 - lava_callback - DEBUG - Processing node 694036cf575bc060ce157928: baseline-arm64-baylibre 2025-12-17 16:27:15,034 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036cf575bc060ce157928 2025-12-17 16:27:15,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:15,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036cf575bc060ce157928 HTTP/1.1" 200 None 2025-12-17 16:27:15,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:15,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:27:15,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:15,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036cf575bc060ce157928 HTTP/1.1" 200 None 2025-12-17 16:27:15,115 - lava_callback - INFO - Completed processing callback for node 694036cf575bc060ce157928 INFO: 172.21.0.1:37294 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:27:15,710 - lava_callback - INFO - Received callback for node 694036d1575bc060ce15792b INFO: 172.21.0.1:37304 - "POST /node/694036d1575bc060ce15792b HTTP/1.0" 202 Accepted 2025-12-17 16:27:15,718 - lava_callback - INFO - Starting async job submit for node 694036d1575bc060ce15792b 2025-12-17 16:27:15,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:15,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036d1575bc060ce15792b HTTP/1.1" 200 None 2025-12-17 16:27:15,736 - lava_callback - DEBUG - Processing node 694036d1575bc060ce15792b: baseline-arm64-baylibre 2025-12-17 16:27:15,736 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036d1575bc060ce15792b 2025-12-17 16:27:15,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:15,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036d1575bc060ce15792b HTTP/1.1" 200 None 2025-12-17 16:27:15,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:15,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:27:15,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:15,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036d1575bc060ce15792b HTTP/1.1" 200 None 2025-12-17 16:27:15,782 - lava_callback - INFO - Completed processing callback for node 694036d1575bc060ce15792b 2025-12-17 16:27:16,410 - lava_callback - INFO - Received callback for node 694036d6575bc060ce157935 INFO: 172.21.0.1:37308 - "POST /node/694036d6575bc060ce157935 HTTP/1.0" 202 Accepted 2025-12-17 16:27:16,418 - lava_callback - INFO - Starting async job submit for node 694036d6575bc060ce157935 2025-12-17 16:27:16,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:16,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036d6575bc060ce157935 HTTP/1.1" 200 None 2025-12-17 16:27:16,435 - lava_callback - DEBUG - Processing node 694036d6575bc060ce157935: baseline-arm64-baylibre 2025-12-17 16:27:16,435 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036d6575bc060ce157935 2025-12-17 16:27:16,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:16,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036d6575bc060ce157935 HTTP/1.1" 200 None 2025-12-17 16:27:16,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:16,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:27:16,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:16,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036d6575bc060ce157935 HTTP/1.1" 200 None 2025-12-17 16:27:16,505 - lava_callback - INFO - Completed processing callback for node 694036d6575bc060ce157935 INFO: 172.21.0.1:37322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53920 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:27:27,885 - lava_callback - INFO - Received callback for node 694036e4575bc060ce15796d INFO: 172.21.0.1:53922 - "POST /node/694036e4575bc060ce15796d HTTP/1.0" 202 Accepted 2025-12-17 16:27:27,894 - lava_callback - INFO - Starting async job submit for node 694036e4575bc060ce15796d 2025-12-17 16:27:27,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:27,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036e4575bc060ce15796d HTTP/1.1" 200 None 2025-12-17 16:27:27,908 - lava_callback - DEBUG - Processing node 694036e4575bc060ce15796d: baseline-arm-broonie 2025-12-17 16:27:27,908 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036e4575bc060ce15796d 2025-12-17 16:27:27,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:27,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036e4575bc060ce15796d HTTP/1.1" 200 None 2025-12-17 16:27:27,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:27,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:27,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:27,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036e4575bc060ce15796d HTTP/1.1" 200 None 2025-12-17 16:27:27,967 - lava_callback - INFO - Completed processing callback for node 694036e4575bc060ce15796d 2025-12-17 16:27:28,653 - lava_callback - INFO - Received callback for node 694036e6575bc060ce157977 INFO: 172.21.0.1:53938 - "POST /node/694036e6575bc060ce157977 HTTP/1.0" 202 Accepted 2025-12-17 16:27:28,664 - lava_callback - INFO - Starting async job submit for node 694036e6575bc060ce157977 2025-12-17 16:27:28,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:28,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036e6575bc060ce157977 HTTP/1.1" 200 None 2025-12-17 16:27:28,680 - lava_callback - DEBUG - Processing node 694036e6575bc060ce157977: baseline-arm-broonie 2025-12-17 16:27:28,680 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036e6575bc060ce157977 2025-12-17 16:27:28,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:28,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036e6575bc060ce157977 HTTP/1.1" 200 None 2025-12-17 16:27:28,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:28,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:28,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:28,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036e6575bc060ce157977 HTTP/1.1" 200 None 2025-12-17 16:27:28,725 - lava_callback - INFO - Completed processing callback for node 694036e6575bc060ce157977 2025-12-17 16:27:29,459 - lava_callback - INFO - Received callback for node 694036ec575bc060ce1579b5 INFO: 172.21.0.1:53944 - "POST /node/694036ec575bc060ce1579b5 HTTP/1.0" 202 Accepted 2025-12-17 16:27:29,468 - lava_callback - INFO - Starting async job submit for node 694036ec575bc060ce1579b5 2025-12-17 16:27:29,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:29,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ec575bc060ce1579b5 HTTP/1.1" 200 None 2025-12-17 16:27:29,482 - lava_callback - DEBUG - Processing node 694036ec575bc060ce1579b5: ltp-cap-bounds 2025-12-17 16:27:29,482 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036ec575bc060ce1579b5 2025-12-17 16:27:29,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:29,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ec575bc060ce1579b5 HTTP/1.1" 200 None 2025-12-17 16:27:29,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:29,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:29,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:29,513 - lava_callback - INFO - Received callback for node 694036dd575bc060ce157942 INFO: 172.21.0.1:53948 - "POST /node/694036dd575bc060ce157942 HTTP/1.0" 202 Accepted 2025-12-17 16:27:29,523 - lava_callback - INFO - Starting async job submit for node 694036dd575bc060ce157942 2025-12-17 16:27:29,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:29,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036dd575bc060ce157942 HTTP/1.1" 200 None 2025-12-17 16:27:29,545 - lava_callback - DEBUG - Processing node 694036dd575bc060ce157942: baseline-arm-clabbe 2025-12-17 16:27:29,545 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036dd575bc060ce157942 2025-12-17 16:27:29,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:29,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036ec575bc060ce1579b5 HTTP/1.1" 200 None 2025-12-17 16:27:29,555 - lava_callback - INFO - Completed processing callback for node 694036ec575bc060ce1579b5 2025-12-17 16:27:29,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036dd575bc060ce157942 HTTP/1.1" 200 None 2025-12-17 16:27:29,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:29,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:27:29,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:29,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036dd575bc060ce157942 HTTP/1.1" 200 None 2025-12-17 16:27:29,678 - lava_callback - INFO - Completed processing callback for node 694036dd575bc060ce157942 2025-12-17 16:27:30,182 - lava_callback - INFO - Received callback for node 694036df575bc060ce157945 INFO: 172.21.0.1:48594 - "POST /node/694036df575bc060ce157945 HTTP/1.0" 202 Accepted 2025-12-17 16:27:30,192 - lava_callback - INFO - Starting async job submit for node 694036df575bc060ce157945 2025-12-17 16:27:30,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:30,242 - lava_callback - INFO - Received callback for node 694036ec575bc060ce1579b9 INFO: 172.21.0.1:48602 - "POST /node/694036ec575bc060ce1579b9 HTTP/1.0" 202 Accepted 2025-12-17 16:27:30,252 - lava_callback - INFO - Starting async job submit for node 694036ec575bc060ce1579b9 2025-12-17 16:27:30,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:30,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036df575bc060ce157945 HTTP/1.1" 200 None 2025-12-17 16:27:30,668 - lava_callback - DEBUG - Processing node 694036df575bc060ce157945: baseline-arm-clabbe 2025-12-17 16:27:30,668 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036df575bc060ce157945 2025-12-17 16:27:30,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:30,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ec575bc060ce1579b9 HTTP/1.1" 200 None 2025-12-17 16:27:30,670 - lava_callback - DEBUG - Processing node 694036ec575bc060ce1579b9: ltp-containers 2025-12-17 16:27:30,670 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036ec575bc060ce1579b9 2025-12-17 16:27:30,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:30,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036df575bc060ce157945 HTTP/1.1" 200 None 2025-12-17 16:27:30,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:30,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ec575bc060ce1579b9 HTTP/1.1" 200 None 2025-12-17 16:27:30,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:30,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:27:30,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:30,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:30,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:30,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036df575bc060ce157945 HTTP/1.1" 200 None 2025-12-17 16:27:30,728 - lava_callback - INFO - Completed processing callback for node 694036df575bc060ce157945 2025-12-17 16:27:30,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036ec575bc060ce1579b9 HTTP/1.1" 200 None 2025-12-17 16:27:30,743 - lava_callback - INFO - Completed processing callback for node 694036ec575bc060ce1579b9 2025-12-17 16:27:30,849 - lava_callback - INFO - Received callback for node 694036e1575bc060ce15794a INFO: 172.21.0.1:48612 - "POST /node/694036e1575bc060ce15794a HTTP/1.0" 202 Accepted 2025-12-17 16:27:30,865 - lava_callback - INFO - Starting async job submit for node 694036e1575bc060ce15794a 2025-12-17 16:27:30,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:30,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036e1575bc060ce15794a HTTP/1.1" 200 None 2025-12-17 16:27:30,888 - lava_callback - DEBUG - Processing node 694036e1575bc060ce15794a: baseline-arm-clabbe 2025-12-17 16:27:30,888 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036e1575bc060ce15794a 2025-12-17 16:27:30,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:30,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036e1575bc060ce15794a HTTP/1.1" 200 None 2025-12-17 16:27:30,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:30,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:27:30,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:31,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036e1575bc060ce15794a HTTP/1.1" 200 None 2025-12-17 16:27:31,006 - lava_callback - INFO - Completed processing callback for node 694036e1575bc060ce15794a 2025-12-17 16:27:31,026 - lava_callback - INFO - Received callback for node 694036ed575bc060ce1579bc INFO: 172.21.0.1:48614 - "POST /node/694036ed575bc060ce1579bc HTTP/1.0" 202 Accepted 2025-12-17 16:27:31,045 - lava_callback - INFO - Starting async job submit for node 694036ed575bc060ce1579bc 2025-12-17 16:27:31,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:31,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ed575bc060ce1579bc HTTP/1.1" 200 None 2025-12-17 16:27:31,068 - lava_callback - DEBUG - Processing node 694036ed575bc060ce1579bc: ltp-controllers 2025-12-17 16:27:31,068 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036ed575bc060ce1579bc 2025-12-17 16:27:31,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:31,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ed575bc060ce1579bc HTTP/1.1" 200 None 2025-12-17 16:27:31,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:31,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:31,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:31,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036ed575bc060ce1579bc HTTP/1.1" 200 None 2025-12-17 16:27:31,157 - lava_callback - INFO - Completed processing callback for node 694036ed575bc060ce1579bc 2025-12-17 16:27:31,557 - lava_callback - INFO - Received callback for node 694036e2575bc060ce15795b INFO: 172.21.0.1:48626 - "POST /node/694036e2575bc060ce15795b HTTP/1.0" 202 Accepted 2025-12-17 16:27:31,568 - lava_callback - INFO - Starting async job submit for node 694036e2575bc060ce15795b 2025-12-17 16:27:31,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:31,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036e2575bc060ce15795b HTTP/1.1" 200 None 2025-12-17 16:27:31,585 - lava_callback - DEBUG - Processing node 694036e2575bc060ce15795b: baseline-arm-clabbe 2025-12-17 16:27:31,585 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036e2575bc060ce15795b 2025-12-17 16:27:31,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:31,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036e2575bc060ce15795b HTTP/1.1" 200 None 2025-12-17 16:27:31,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:31,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:27:31,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:31,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036e2575bc060ce15795b HTTP/1.1" 200 None 2025-12-17 16:27:31,642 - lava_callback - INFO - Completed processing callback for node 694036e2575bc060ce15795b 2025-12-17 16:27:31,789 - lava_callback - INFO - Received callback for node 694036ed575bc060ce1579c1 INFO: 172.21.0.1:48628 - "POST /node/694036ed575bc060ce1579c1 HTTP/1.0" 202 Accepted 2025-12-17 16:27:31,812 - lava_callback - INFO - Starting async job submit for node 694036ed575bc060ce1579c1 2025-12-17 16:27:31,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:31,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ed575bc060ce1579c1 HTTP/1.1" 200 None 2025-12-17 16:27:31,912 - lava_callback - DEBUG - Processing node 694036ed575bc060ce1579c1: ltp-cve 2025-12-17 16:27:31,912 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036ed575bc060ce1579c1 2025-12-17 16:27:31,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:31,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ed575bc060ce1579c1 HTTP/1.1" 200 None 2025-12-17 16:27:31,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:31,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:31,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:31,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036ed575bc060ce1579c1 HTTP/1.1" 200 None 2025-12-17 16:27:31,969 - lava_callback - INFO - Completed processing callback for node 694036ed575bc060ce1579c1 2025-12-17 16:27:32,228 - lava_callback - INFO - Received callback for node 694036e7575bc060ce157984 INFO: 172.21.0.1:48632 - "POST /node/694036e7575bc060ce157984 HTTP/1.0" 202 Accepted 2025-12-17 16:27:32,237 - lava_callback - INFO - Starting async job submit for node 694036e7575bc060ce157984 2025-12-17 16:27:32,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:32,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036e7575bc060ce157984 HTTP/1.1" 200 None 2025-12-17 16:27:32,249 - lava_callback - DEBUG - Processing node 694036e7575bc060ce157984: baseline-arm-clabbe 2025-12-17 16:27:32,249 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036e7575bc060ce157984 2025-12-17 16:27:32,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:32,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036e7575bc060ce157984 HTTP/1.1" 200 None 2025-12-17 16:27:32,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:32,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:27:32,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:32,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036e7575bc060ce157984 HTTP/1.1" 200 None 2025-12-17 16:27:32,341 - lava_callback - INFO - Completed processing callback for node 694036e7575bc060ce157984 2025-12-17 16:27:32,554 - lava_callback - INFO - Received callback for node 694036ee575bc060ce1579c8 INFO: 172.21.0.1:48646 - "POST /node/694036ee575bc060ce1579c8 HTTP/1.0" 202 Accepted 2025-12-17 16:27:32,563 - lava_callback - INFO - Starting async job submit for node 694036ee575bc060ce1579c8 2025-12-17 16:27:32,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:32,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ee575bc060ce1579c8 HTTP/1.1" 200 None 2025-12-17 16:27:32,578 - lava_callback - DEBUG - Processing node 694036ee575bc060ce1579c8: ltp-fs-bind 2025-12-17 16:27:32,579 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036ee575bc060ce1579c8 2025-12-17 16:27:32,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:32,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ee575bc060ce1579c8 HTTP/1.1" 200 None 2025-12-17 16:27:32,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:32,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:32,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:32,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036ee575bc060ce1579c8 HTTP/1.1" 200 None 2025-12-17 16:27:32,623 - lava_callback - INFO - Completed processing callback for node 694036ee575bc060ce1579c8 2025-12-17 16:27:32,902 - lava_callback - INFO - Received callback for node 694036e8575bc060ce157990 INFO: 172.21.0.1:48658 - "POST /node/694036e8575bc060ce157990 HTTP/1.0" 202 Accepted 2025-12-17 16:27:32,911 - lava_callback - INFO - Starting async job submit for node 694036e8575bc060ce157990 2025-12-17 16:27:32,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:32,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036e8575bc060ce157990 HTTP/1.1" 200 None 2025-12-17 16:27:32,927 - lava_callback - DEBUG - Processing node 694036e8575bc060ce157990: baseline-arm-clabbe 2025-12-17 16:27:32,927 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036e8575bc060ce157990 2025-12-17 16:27:32,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:32,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036e8575bc060ce157990 HTTP/1.1" 200 None 2025-12-17 16:27:32,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:32,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:27:32,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:32,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036e8575bc060ce157990 HTTP/1.1" 200 None 2025-12-17 16:27:32,966 - lava_callback - INFO - Completed processing callback for node 694036e8575bc060ce157990 2025-12-17 16:27:33,297 - lava_callback - INFO - Received callback for node 694036ef575bc060ce1579cb INFO: 172.21.0.1:48674 - "POST /node/694036ef575bc060ce1579cb HTTP/1.0" 202 Accepted 2025-12-17 16:27:33,311 - lava_callback - INFO - Starting async job submit for node 694036ef575bc060ce1579cb 2025-12-17 16:27:33,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:33,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ef575bc060ce1579cb HTTP/1.1" 200 None 2025-12-17 16:27:33,329 - lava_callback - DEBUG - Processing node 694036ef575bc060ce1579cb: ltp-fsx 2025-12-17 16:27:33,329 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036ef575bc060ce1579cb 2025-12-17 16:27:33,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:33,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ef575bc060ce1579cb HTTP/1.1" 200 None 2025-12-17 16:27:33,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:33,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:33,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:33,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036ef575bc060ce1579cb HTTP/1.1" 200 None 2025-12-17 16:27:33,388 - lava_callback - INFO - Completed processing callback for node 694036ef575bc060ce1579cb 2025-12-17 16:27:33,561 - lava_callback - INFO - Received callback for node 694036ea575bc060ce1579a2 INFO: 172.21.0.1:48686 - "POST /node/694036ea575bc060ce1579a2 HTTP/1.0" 202 Accepted 2025-12-17 16:27:33,576 - lava_callback - INFO - Starting async job submit for node 694036ea575bc060ce1579a2 2025-12-17 16:27:33,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:33,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ea575bc060ce1579a2 HTTP/1.1" 200 None 2025-12-17 16:27:33,596 - lava_callback - DEBUG - Processing node 694036ea575bc060ce1579a2: baseline-arm-clabbe 2025-12-17 16:27:33,596 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036ea575bc060ce1579a2 2025-12-17 16:27:33,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:33,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ea575bc060ce1579a2 HTTP/1.1" 200 None 2025-12-17 16:27:33,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:33,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:27:33,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:33,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036ea575bc060ce1579a2 HTTP/1.1" 200 None 2025-12-17 16:27:33,653 - lava_callback - INFO - Completed processing callback for node 694036ea575bc060ce1579a2 2025-12-17 16:27:34,044 - lava_callback - INFO - Received callback for node 694036ef575bc060ce1579ce INFO: 172.21.0.1:48692 - "POST /node/694036ef575bc060ce1579ce HTTP/1.0" 202 Accepted 2025-12-17 16:27:34,054 - lava_callback - INFO - Starting async job submit for node 694036ef575bc060ce1579ce 2025-12-17 16:27:34,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ef575bc060ce1579ce HTTP/1.1" 200 None 2025-12-17 16:27:34,068 - lava_callback - DEBUG - Processing node 694036ef575bc060ce1579ce: ltp-input 2025-12-17 16:27:34,068 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036ef575bc060ce1579ce 2025-12-17 16:27:34,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ef575bc060ce1579ce HTTP/1.1" 200 None 2025-12-17 16:27:34,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:34,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036ef575bc060ce1579ce HTTP/1.1" 200 None 2025-12-17 16:27:34,122 - lava_callback - INFO - Completed processing callback for node 694036ef575bc060ce1579ce 2025-12-17 16:27:34,220 - lava_callback - INFO - Received callback for node 694036ec575bc060ce1579b2 INFO: 172.21.0.1:48698 - "POST /node/694036ec575bc060ce1579b2 HTTP/1.0" 202 Accepted 2025-12-17 16:27:34,232 - lava_callback - INFO - Starting async job submit for node 694036ec575bc060ce1579b2 2025-12-17 16:27:34,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ec575bc060ce1579b2 HTTP/1.1" 200 None 2025-12-17 16:27:34,264 - lava_callback - DEBUG - Processing node 694036ec575bc060ce1579b2: baseline-arm-clabbe 2025-12-17 16:27:34,264 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036ec575bc060ce1579b2 2025-12-17 16:27:34,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ec575bc060ce1579b2 HTTP/1.1" 200 None 2025-12-17 16:27:34,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:27:34,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036ec575bc060ce1579b2 HTTP/1.1" 200 None 2025-12-17 16:27:34,332 - lava_callback - INFO - Completed processing callback for node 694036ec575bc060ce1579b2 2025-12-17 16:27:34,595 - lava_callback - INFO - Received callback for node 694036e3575bc060ce157962 INFO: 172.21.0.1:48714 - "POST /node/694036e3575bc060ce157962 HTTP/1.0" 202 Accepted 2025-12-17 16:27:34,609 - lava_callback - INFO - Starting async job submit for node 694036e3575bc060ce157962 2025-12-17 16:27:34,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036e3575bc060ce157962 HTTP/1.1" 200 None 2025-12-17 16:27:34,627 - lava_callback - DEBUG - Processing node 694036e3575bc060ce157962: baseline-arm 2025-12-17 16:27:34,627 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036e3575bc060ce157962 2025-12-17 16:27:34,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036e3575bc060ce157962 HTTP/1.1" 200 None 2025-12-17 16:27:34,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:34,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036e3575bc060ce157962 HTTP/1.1" 200 None 2025-12-17 16:27:34,673 - lava_callback - INFO - Completed processing callback for node 694036e3575bc060ce157962 2025-12-17 16:27:34,912 - lava_callback - INFO - Received callback for node 694036ed575bc060ce1579bf INFO: 172.21.0.1:48726 - "POST /node/694036ed575bc060ce1579bf HTTP/1.0" 202 Accepted 2025-12-17 16:27:34,921 - lava_callback - INFO - Starting async job submit for node 694036ed575bc060ce1579bf 2025-12-17 16:27:34,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,929 - lava_callback - INFO - Received callback for node 694036f0575bc060ce1579d1 INFO: 172.21.0.1:48738 - "POST /node/694036f0575bc060ce1579d1 HTTP/1.0" 202 Accepted 2025-12-17 16:27:34,940 - lava_callback - INFO - Starting async job submit for node 694036f0575bc060ce1579d1 2025-12-17 16:27:34,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ed575bc060ce1579bf HTTP/1.1" 200 None 2025-12-17 16:27:34,949 - lava_callback - DEBUG - Processing node 694036ed575bc060ce1579bf: baseline-arm64-clabbe 2025-12-17 16:27:34,949 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036ed575bc060ce1579bf 2025-12-17 16:27:34,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f0575bc060ce1579d1 HTTP/1.1" 200 None 2025-12-17 16:27:34,959 - lava_callback - DEBUG - Processing node 694036f0575bc060ce1579d1: ltp-ipc 2025-12-17 16:27:34,959 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036f0575bc060ce1579d1 2025-12-17 16:27:34,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036ed575bc060ce1579bf HTTP/1.1" 200 None 2025-12-17 16:27:34,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f0575bc060ce1579d1 HTTP/1.1" 200 None 2025-12-17 16:27:34,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:27:34,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:34,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:34,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:35,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036ed575bc060ce1579bf HTTP/1.1" 200 None 2025-12-17 16:27:35,008 - lava_callback - INFO - Completed processing callback for node 694036ed575bc060ce1579bf 2025-12-17 16:27:35,013 - lava_callback - INFO - Received callback for node 694036e4575bc060ce15796a INFO: 172.21.0.1:48748 - "POST /node/694036e4575bc060ce15796a HTTP/1.0" 202 Accepted 2025-12-17 16:27:35,036 - lava_callback - INFO - Starting async job submit for node 694036e4575bc060ce15796a 2025-12-17 16:27:35,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:35,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036f0575bc060ce1579d1 HTTP/1.1" 200 None 2025-12-17 16:27:35,050 - lava_callback - INFO - Completed processing callback for node 694036f0575bc060ce1579d1 2025-12-17 16:27:35,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036e4575bc060ce15796a HTTP/1.1" 200 None 2025-12-17 16:27:35,083 - lava_callback - DEBUG - Processing node 694036e4575bc060ce15796a: baseline-arm64-baylibre 2025-12-17 16:27:35,083 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036e4575bc060ce15796a 2025-12-17 16:27:35,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:35,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036e4575bc060ce15796a HTTP/1.1" 200 None 2025-12-17 16:27:35,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:35,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:27:35,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:35,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036e4575bc060ce15796a HTTP/1.1" 200 None 2025-12-17 16:27:35,201 - lava_callback - INFO - Completed processing callback for node 694036e4575bc060ce15796a 2025-12-17 16:27:35,591 - lava_callback - INFO - Received callback for node 694036f0575bc060ce1579d3 INFO: 172.21.0.1:48758 - "POST /node/694036f0575bc060ce1579d3 HTTP/1.0" 202 Accepted 2025-12-17 16:27:35,602 - lava_callback - INFO - Starting async job submit for node 694036f0575bc060ce1579d3 2025-12-17 16:27:35,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:35,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f0575bc060ce1579d3 HTTP/1.1" 200 None 2025-12-17 16:27:35,615 - lava_callback - DEBUG - Processing node 694036f0575bc060ce1579d3: baseline-arm-clabbe 2025-12-17 16:27:35,615 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036f0575bc060ce1579d3 2025-12-17 16:27:35,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:35,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f0575bc060ce1579d3 HTTP/1.1" 200 None 2025-12-17 16:27:35,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:35,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:27:35,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:35,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036f0575bc060ce1579d3 HTTP/1.1" 200 None 2025-12-17 16:27:35,664 - lava_callback - INFO - Completed processing callback for node 694036f0575bc060ce1579d3 2025-12-17 16:27:35,776 - lava_callback - INFO - Received callback for node 694036f0575bc060ce1579d6 INFO: 172.21.0.1:48770 - "POST /node/694036f0575bc060ce1579d6 HTTP/1.0" 202 Accepted 2025-12-17 16:27:35,788 - lava_callback - INFO - Starting async job submit for node 694036f0575bc060ce1579d6 2025-12-17 16:27:35,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:35,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f0575bc060ce1579d6 HTTP/1.1" 200 None 2025-12-17 16:27:35,802 - lava_callback - DEBUG - Processing node 694036f0575bc060ce1579d6: ltp-pty 2025-12-17 16:27:35,802 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036f0575bc060ce1579d6 2025-12-17 16:27:35,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:35,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f0575bc060ce1579d6 HTTP/1.1" 200 None 2025-12-17 16:27:35,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:35,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:35,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:35,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036f0575bc060ce1579d6 HTTP/1.1" 200 None 2025-12-17 16:27:35,872 - lava_callback - INFO - Completed processing callback for node 694036f0575bc060ce1579d6 INFO: 172.21.0.1:48782 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:27:36,541 - lava_callback - INFO - Received callback for node 694036f1575bc060ce1579da INFO: 172.21.0.1:48794 - "POST /node/694036f1575bc060ce1579da HTTP/1.0" 202 Accepted 2025-12-17 16:27:36,556 - lava_callback - INFO - Starting async job submit for node 694036f1575bc060ce1579da 2025-12-17 16:27:36,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:36,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f1575bc060ce1579da HTTP/1.1" 200 None 2025-12-17 16:27:36,585 - lava_callback - DEBUG - Processing node 694036f1575bc060ce1579da: ltp-sched 2025-12-17 16:27:36,585 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036f1575bc060ce1579da 2025-12-17 16:27:36,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:36,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f1575bc060ce1579da HTTP/1.1" 200 None 2025-12-17 16:27:36,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:36,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:36,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:36,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036f1575bc060ce1579da HTTP/1.1" 200 None 2025-12-17 16:27:36,648 - lava_callback - INFO - Completed processing callback for node 694036f1575bc060ce1579da 2025-12-17 16:27:37,358 - lava_callback - INFO - Received callback for node 694036f1575bc060ce1579dd INFO: 172.21.0.1:48798 - "POST /node/694036f1575bc060ce1579dd HTTP/1.0" 202 Accepted 2025-12-17 16:27:37,367 - lava_callback - INFO - Starting async job submit for node 694036f1575bc060ce1579dd 2025-12-17 16:27:37,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:37,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f1575bc060ce1579dd HTTP/1.1" 200 None 2025-12-17 16:27:37,384 - lava_callback - DEBUG - Processing node 694036f1575bc060ce1579dd: ltp-smoketest 2025-12-17 16:27:37,384 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036f1575bc060ce1579dd 2025-12-17 16:27:37,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:37,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f1575bc060ce1579dd HTTP/1.1" 200 None 2025-12-17 16:27:37,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:37,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:37,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:37,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036f1575bc060ce1579dd HTTP/1.1" 200 None 2025-12-17 16:27:37,447 - lava_callback - INFO - Completed processing callback for node 694036f1575bc060ce1579dd 2025-12-17 16:27:38,108 - lava_callback - INFO - Received callback for node 694036f2575bc060ce1579e0 INFO: 172.21.0.1:48802 - "POST /node/694036f2575bc060ce1579e0 HTTP/1.0" 202 Accepted 2025-12-17 16:27:38,119 - lava_callback - INFO - Starting async job submit for node 694036f2575bc060ce1579e0 2025-12-17 16:27:38,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:38,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f2575bc060ce1579e0 HTTP/1.1" 200 None 2025-12-17 16:27:38,133 - lava_callback - DEBUG - Processing node 694036f2575bc060ce1579e0: ltp-syscalls 2025-12-17 16:27:38,133 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036f2575bc060ce1579e0 2025-12-17 16:27:38,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:38,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f2575bc060ce1579e0 HTTP/1.1" 200 None 2025-12-17 16:27:38,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:38,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:38,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:38,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036f2575bc060ce1579e0 HTTP/1.1" 200 None 2025-12-17 16:27:38,190 - lava_callback - INFO - Completed processing callback for node 694036f2575bc060ce1579e0 2025-12-17 16:27:38,855 - lava_callback - INFO - Received callback for node 694036f2575bc060ce1579e5 INFO: 172.21.0.1:48804 - "POST /node/694036f2575bc060ce1579e5 HTTP/1.0" 202 Accepted 2025-12-17 16:27:38,865 - lava_callback - INFO - Starting async job submit for node 694036f2575bc060ce1579e5 2025-12-17 16:27:38,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:38,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f2575bc060ce1579e5 HTTP/1.1" 200 None 2025-12-17 16:27:38,877 - lava_callback - DEBUG - Processing node 694036f2575bc060ce1579e5: ltp-syscalls-ipc 2025-12-17 16:27:38,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036f2575bc060ce1579e5 2025-12-17 16:27:38,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:38,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f2575bc060ce1579e5 HTTP/1.1" 200 None 2025-12-17 16:27:38,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:38,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:38,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:38,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036f2575bc060ce1579e5 HTTP/1.1" 200 None 2025-12-17 16:27:38,963 - lava_callback - INFO - Completed processing callback for node 694036f2575bc060ce1579e5 2025-12-17 16:27:39,576 - lava_callback - INFO - Received callback for node 694036f3575bc060ce1579e9 INFO: 172.21.0.1:48808 - "POST /node/694036f3575bc060ce1579e9 HTTP/1.0" 202 Accepted 2025-12-17 16:27:39,587 - lava_callback - INFO - Starting async job submit for node 694036f3575bc060ce1579e9 2025-12-17 16:27:39,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:39,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f3575bc060ce1579e9 HTTP/1.1" 200 None 2025-12-17 16:27:39,604 - lava_callback - DEBUG - Processing node 694036f3575bc060ce1579e9: ltp-timers 2025-12-17 16:27:39,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036f3575bc060ce1579e9 2025-12-17 16:27:39,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:39,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f3575bc060ce1579e9 HTTP/1.1" 200 None 2025-12-17 16:27:39,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:39,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:39,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:39,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036f3575bc060ce1579e9 HTTP/1.1" 200 None 2025-12-17 16:27:39,650 - lava_callback - INFO - Completed processing callback for node 694036f3575bc060ce1579e9 2025-12-17 16:27:40,642 - lava_callback - INFO - Received callback for node 694036f3575bc060ce1579ec INFO: 172.21.0.1:34718 - "POST /node/694036f3575bc060ce1579ec HTTP/1.0" 202 Accepted 2025-12-17 16:27:40,654 - lava_callback - INFO - Starting async job submit for node 694036f3575bc060ce1579ec 2025-12-17 16:27:40,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:40,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f3575bc060ce1579ec HTTP/1.1" 200 None 2025-12-17 16:27:40,670 - lava_callback - DEBUG - Processing node 694036f3575bc060ce1579ec: ltp-watchqueue 2025-12-17 16:27:40,670 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036f3575bc060ce1579ec 2025-12-17 16:27:40,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:40,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f3575bc060ce1579ec HTTP/1.1" 200 None 2025-12-17 16:27:40,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:40,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:27:40,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:40,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036f3575bc060ce1579ec HTTP/1.1" 200 None 2025-12-17 16:27:40,728 - lava_callback - INFO - Completed processing callback for node 694036f3575bc060ce1579ec 2025-12-17 16:27:41,593 - lava_callback - INFO - Received callback for node 694036f9575bc060ce1579f5 INFO: 172.21.0.1:34732 - "POST /node/694036f9575bc060ce1579f5 HTTP/1.0" 202 Accepted 2025-12-17 16:27:41,605 - lava_callback - INFO - Starting async job submit for node 694036f9575bc060ce1579f5 2025-12-17 16:27:41,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:41,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f9575bc060ce1579f5 HTTP/1.1" 200 None 2025-12-17 16:27:41,629 - lava_callback - DEBUG - Processing node 694036f9575bc060ce1579f5: ltp-syscalls 2025-12-17 16:27:41,629 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036f9575bc060ce1579f5 2025-12-17 16:27:41,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:41,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f9575bc060ce1579f5 HTTP/1.1" 200 None 2025-12-17 16:27:41,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:41,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:27:41,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:41,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036f9575bc060ce1579f5 HTTP/1.1" 200 None 2025-12-17 16:27:41,711 - lava_callback - INFO - Completed processing callback for node 694036f9575bc060ce1579f5 2025-12-17 16:27:42,507 - lava_callback - INFO - Received callback for node 694036fa575bc060ce1579f8 INFO: 172.21.0.1:34734 - "POST /node/694036fa575bc060ce1579f8 HTTP/1.0" 202 Accepted 2025-12-17 16:27:42,516 - lava_callback - INFO - Starting async job submit for node 694036fa575bc060ce1579f8 2025-12-17 16:27:42,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:42,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036fa575bc060ce1579f8 HTTP/1.1" 200 None 2025-12-17 16:27:42,532 - lava_callback - DEBUG - Processing node 694036fa575bc060ce1579f8: ltp-syscalls 2025-12-17 16:27:42,532 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036fa575bc060ce1579f8 2025-12-17 16:27:42,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:42,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036fa575bc060ce1579f8 HTTP/1.1" 200 None 2025-12-17 16:27:42,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:42,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:27:42,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:42,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036fa575bc060ce1579f8 HTTP/1.1" 200 None 2025-12-17 16:27:42,585 - lava_callback - INFO - Completed processing callback for node 694036fa575bc060ce1579f8 INFO: 172.21.0.1:34740 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:27:49,529 - lava_callback - INFO - Received callback for node 694036f2575bc060ce1579e2 INFO: 172.21.0.1:34750 - "POST /node/694036f2575bc060ce1579e2 HTTP/1.0" 202 Accepted 2025-12-17 16:27:49,540 - lava_callback - INFO - Starting async job submit for node 694036f2575bc060ce1579e2 2025-12-17 16:27:49,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:49,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f2575bc060ce1579e2 HTTP/1.1" 200 None 2025-12-17 16:27:49,559 - lava_callback - DEBUG - Processing node 694036f2575bc060ce1579e2: baseline-arm64-clabbe 2025-12-17 16:27:49,559 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036f2575bc060ce1579e2 2025-12-17 16:27:49,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:49,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f2575bc060ce1579e2 HTTP/1.1" 200 None 2025-12-17 16:27:49,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:49,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:27:49,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:49,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036f2575bc060ce1579e2 HTTP/1.1" 200 None 2025-12-17 16:27:49,618 - lava_callback - INFO - Completed processing callback for node 694036f2575bc060ce1579e2 2025-12-17 16:27:50,177 - lava_callback - INFO - Received callback for node 694036f4575bc060ce1579ef INFO: 172.21.0.1:41562 - "POST /node/694036f4575bc060ce1579ef HTTP/1.0" 202 Accepted 2025-12-17 16:27:50,187 - lava_callback - INFO - Starting async job submit for node 694036f4575bc060ce1579ef 2025-12-17 16:27:50,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:50,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f4575bc060ce1579ef HTTP/1.1" 200 None 2025-12-17 16:27:50,205 - lava_callback - DEBUG - Processing node 694036f4575bc060ce1579ef: baseline-arm64-clabbe 2025-12-17 16:27:50,205 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036f4575bc060ce1579ef 2025-12-17 16:27:50,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:50,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f4575bc060ce1579ef HTTP/1.1" 200 None 2025-12-17 16:27:50,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:50,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:27:50,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:50,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036f4575bc060ce1579ef HTTP/1.1" 200 None 2025-12-17 16:27:50,263 - lava_callback - INFO - Completed processing callback for node 694036f4575bc060ce1579ef 2025-12-17 16:27:50,838 - lava_callback - INFO - Received callback for node 694036f6575bc060ce1579f2 INFO: 172.21.0.1:41578 - "POST /node/694036f6575bc060ce1579f2 HTTP/1.0" 202 Accepted 2025-12-17 16:27:50,849 - lava_callback - INFO - Starting async job submit for node 694036f6575bc060ce1579f2 2025-12-17 16:27:50,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:50,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f6575bc060ce1579f2 HTTP/1.1" 200 None 2025-12-17 16:27:50,863 - lava_callback - DEBUG - Processing node 694036f6575bc060ce1579f2: baseline-arm64-clabbe 2025-12-17 16:27:50,863 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036f6575bc060ce1579f2 2025-12-17 16:27:50,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:50,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036f6575bc060ce1579f2 HTTP/1.1" 200 None 2025-12-17 16:27:50,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:50,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:27:50,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:50,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036f6575bc060ce1579f2 HTTP/1.1" 200 None 2025-12-17 16:27:50,927 - lava_callback - INFO - Completed processing callback for node 694036f6575bc060ce1579f2 2025-12-17 16:27:51,510 - lava_callback - INFO - Received callback for node 694036fb575bc060ce1579fb INFO: 172.21.0.1:41582 - "POST /node/694036fb575bc060ce1579fb HTTP/1.0" 202 Accepted 2025-12-17 16:27:51,519 - lava_callback - INFO - Starting async job submit for node 694036fb575bc060ce1579fb 2025-12-17 16:27:51,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:51,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036fb575bc060ce1579fb HTTP/1.1" 200 None 2025-12-17 16:27:51,534 - lava_callback - DEBUG - Processing node 694036fb575bc060ce1579fb: baseline-arm64-clabbe 2025-12-17 16:27:51,534 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036fb575bc060ce1579fb 2025-12-17 16:27:51,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:51,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036fb575bc060ce1579fb HTTP/1.1" 200 None 2025-12-17 16:27:51,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:51,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:27:51,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:51,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036fb575bc060ce1579fb HTTP/1.1" 200 None 2025-12-17 16:27:51,601 - lava_callback - INFO - Completed processing callback for node 694036fb575bc060ce1579fb 2025-12-17 16:27:54,412 - lava_callback - INFO - Received callback for node 69403703575bc060ce157a21 INFO: 172.21.0.1:41596 - "POST /node/69403703575bc060ce157a21 HTTP/1.0" 202 Accepted 2025-12-17 16:27:54,420 - lava_callback - INFO - Starting async job submit for node 69403703575bc060ce157a21 2025-12-17 16:27:54,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:54,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403703575bc060ce157a21 HTTP/1.1" 200 None 2025-12-17 16:27:54,434 - lava_callback - DEBUG - Processing node 69403703575bc060ce157a21: baseline-x86-kcidebug-intel 2025-12-17 16:27:54,434 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403703575bc060ce157a21 2025-12-17 16:27:54,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:54,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403703575bc060ce157a21 HTTP/1.1" 200 None 2025-12-17 16:27:54,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:54,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693aebc7fee8383e92b666f7 HTTP/1.1" 200 None 2025-12-17 16:27:54,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:54,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403703575bc060ce157a21 HTTP/1.1" 200 None 2025-12-17 16:27:54,573 - lava_callback - INFO - Completed processing callback for node 69403703575bc060ce157a21 2025-12-17 16:27:54,994 - lava_callback - INFO - Received callback for node 69403704575bc060ce157a28 INFO: 172.21.0.1:41604 - "POST /node/69403704575bc060ce157a28 HTTP/1.0" 202 Accepted 2025-12-17 16:27:55,003 - lava_callback - INFO - Starting async job submit for node 69403704575bc060ce157a28 2025-12-17 16:27:55,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:55,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403704575bc060ce157a28 HTTP/1.1" 200 None 2025-12-17 16:27:55,021 - lava_callback - DEBUG - Processing node 69403704575bc060ce157a28: baseline-arm64-baylibre 2025-12-17 16:27:55,022 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403704575bc060ce157a28 2025-12-17 16:27:55,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:55,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403704575bc060ce157a28 HTTP/1.1" 200 None 2025-12-17 16:27:55,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:55,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:27:55,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:55,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403704575bc060ce157a28 HTTP/1.1" 200 None 2025-12-17 16:27:55,091 - lava_callback - INFO - Completed processing callback for node 69403704575bc060ce157a28 2025-12-17 16:27:55,339 - lava_callback - INFO - Received callback for node 694036fd575bc060ce157a02 INFO: 172.21.0.1:41610 - "POST /node/694036fd575bc060ce157a02 HTTP/1.0" 202 Accepted 2025-12-17 16:27:55,351 - lava_callback - INFO - Starting async job submit for node 694036fd575bc060ce157a02 2025-12-17 16:27:55,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:55,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036fd575bc060ce157a02 HTTP/1.1" 200 None 2025-12-17 16:27:55,367 - lava_callback - DEBUG - Processing node 694036fd575bc060ce157a02: baseline-arm-pengutronix 2025-12-17 16:27:55,367 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694036fd575bc060ce157a02 2025-12-17 16:27:55,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:55,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036fd575bc060ce157a02 HTTP/1.1" 200 None 2025-12-17 16:27:55,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:55,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d219f07de2c0cbc264e94 HTTP/1.1" 200 None 2025-12-17 16:27:55,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:55,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036fd575bc060ce157a02 HTTP/1.1" 200 None 2025-12-17 16:27:55,419 - lava_callback - INFO - Completed processing callback for node 694036fd575bc060ce157a02 2025-12-17 16:27:55,673 - lava_callback - INFO - Received callback for node 69403709575bc060ce157a4d INFO: 172.21.0.1:41626 - "POST /node/69403709575bc060ce157a4d HTTP/1.0" 202 Accepted 2025-12-17 16:27:55,682 - lava_callback - INFO - Starting async job submit for node 69403709575bc060ce157a4d 2025-12-17 16:27:55,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:55,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403709575bc060ce157a4d HTTP/1.1" 200 None 2025-12-17 16:27:55,696 - lava_callback - DEBUG - Processing node 69403709575bc060ce157a4d: baseline-arm64-baylibre 2025-12-17 16:27:55,696 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403709575bc060ce157a4d 2025-12-17 16:27:55,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:55,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403709575bc060ce157a4d HTTP/1.1" 200 None 2025-12-17 16:27:55,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:55,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:27:55,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:27:55,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403709575bc060ce157a4d HTTP/1.1" 200 None 2025-12-17 16:27:55,748 - lava_callback - INFO - Completed processing callback for node 69403709575bc060ce157a4d INFO: 172.21.0.1:41634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49966 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:28:15,011 - lava_callback - INFO - Received callback for node 6940370f575bc060ce157a60 INFO: 172.21.0.1:49976 - "POST /node/6940370f575bc060ce157a60 HTTP/1.0" 202 Accepted 2025-12-17 16:28:15,024 - lava_callback - INFO - Starting async job submit for node 6940370f575bc060ce157a60 2025-12-17 16:28:15,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:15,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940370f575bc060ce157a60 HTTP/1.1" 200 None 2025-12-17 16:28:15,040 - lava_callback - DEBUG - Processing node 6940370f575bc060ce157a60: baseline-arm64-baylibre 2025-12-17 16:28:15,040 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940370f575bc060ce157a60 2025-12-17 16:28:15,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:15,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940370f575bc060ce157a60 HTTP/1.1" 200 None 2025-12-17 16:28:15,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:15,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:28:15,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:15,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940370f575bc060ce157a60 HTTP/1.1" 200 None 2025-12-17 16:28:15,097 - lava_callback - INFO - Completed processing callback for node 6940370f575bc060ce157a60 INFO: 172.21.0.1:49986 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:28:15,677 - lava_callback - INFO - Received callback for node 69403711575bc060ce157a63 INFO: 172.21.0.1:49988 - "POST /node/69403711575bc060ce157a63 HTTP/1.0" 202 Accepted 2025-12-17 16:28:15,692 - lava_callback - INFO - Starting async job submit for node 69403711575bc060ce157a63 2025-12-17 16:28:15,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:15,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403711575bc060ce157a63 HTTP/1.1" 200 None 2025-12-17 16:28:15,704 - lava_callback - DEBUG - Processing node 69403711575bc060ce157a63: baseline-arm64-baylibre 2025-12-17 16:28:15,704 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403711575bc060ce157a63 2025-12-17 16:28:15,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:15,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403711575bc060ce157a63 HTTP/1.1" 200 None 2025-12-17 16:28:15,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:15,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:28:15,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:15,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403711575bc060ce157a63 HTTP/1.1" 200 None 2025-12-17 16:28:15,770 - lava_callback - INFO - Completed processing callback for node 69403711575bc060ce157a63 2025-12-17 16:28:16,356 - lava_callback - INFO - Received callback for node 69403717575bc060ce157a74 INFO: 172.21.0.1:50004 - "POST /node/69403717575bc060ce157a74 HTTP/1.0" 202 Accepted 2025-12-17 16:28:16,366 - lava_callback - INFO - Starting async job submit for node 69403717575bc060ce157a74 2025-12-17 16:28:16,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:16,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403717575bc060ce157a74 HTTP/1.1" 200 None 2025-12-17 16:28:16,378 - lava_callback - DEBUG - Processing node 69403717575bc060ce157a74: baseline-arm64-baylibre 2025-12-17 16:28:16,378 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403717575bc060ce157a74 2025-12-17 16:28:16,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:16,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403717575bc060ce157a74 HTTP/1.1" 200 None 2025-12-17 16:28:16,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:16,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:28:16,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:16,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403717575bc060ce157a74 HTTP/1.1" 200 None 2025-12-17 16:28:16,449 - lava_callback - INFO - Completed processing callback for node 69403717575bc060ce157a74 INFO: 172.21.0.1:50014 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:28:17,049 - lava_callback - INFO - Received callback for node 69403719575bc060ce157a77 INFO: 172.21.0.1:50016 - "POST /node/69403719575bc060ce157a77 HTTP/1.0" 202 Accepted 2025-12-17 16:28:17,056 - lava_callback - INFO - Starting async job submit for node 69403719575bc060ce157a77 2025-12-17 16:28:17,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:17,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403719575bc060ce157a77 HTTP/1.1" 200 None 2025-12-17 16:28:17,074 - lava_callback - DEBUG - Processing node 69403719575bc060ce157a77: baseline-arm64-baylibre 2025-12-17 16:28:17,074 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403719575bc060ce157a77 2025-12-17 16:28:17,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:17,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403719575bc060ce157a77 HTTP/1.1" 200 None 2025-12-17 16:28:17,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:17,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:28:17,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:17,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403719575bc060ce157a77 HTTP/1.1" 200 None 2025-12-17 16:28:17,119 - lava_callback - INFO - Completed processing callback for node 69403719575bc060ce157a77 2025-12-17 16:28:17,712 - lava_callback - INFO - Received callback for node 6940371b575bc060ce157a7a INFO: 172.21.0.1:50022 - "POST /node/6940371b575bc060ce157a7a HTTP/1.0" 202 Accepted 2025-12-17 16:28:17,722 - lava_callback - INFO - Starting async job submit for node 6940371b575bc060ce157a7a 2025-12-17 16:28:17,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:17,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940371b575bc060ce157a7a HTTP/1.1" 200 None 2025-12-17 16:28:17,738 - lava_callback - DEBUG - Processing node 6940371b575bc060ce157a7a: baseline-arm64-baylibre 2025-12-17 16:28:17,738 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940371b575bc060ce157a7a 2025-12-17 16:28:17,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:17,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940371b575bc060ce157a7a HTTP/1.1" 200 None 2025-12-17 16:28:17,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:17,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:28:17,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:17,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940371b575bc060ce157a7a HTTP/1.1" 200 None 2025-12-17 16:28:17,784 - lava_callback - INFO - Completed processing callback for node 6940371b575bc060ce157a7a 2025-12-17 16:28:18,390 - lava_callback - INFO - Received callback for node 6940371d575bc060ce157a7d INFO: 172.21.0.1:50028 - "POST /node/6940371d575bc060ce157a7d HTTP/1.0" 202 Accepted 2025-12-17 16:28:18,399 - lava_callback - INFO - Starting async job submit for node 6940371d575bc060ce157a7d 2025-12-17 16:28:18,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:18,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940371d575bc060ce157a7d HTTP/1.1" 200 None 2025-12-17 16:28:18,413 - lava_callback - DEBUG - Processing node 6940371d575bc060ce157a7d: baseline-arm64-baylibre 2025-12-17 16:28:18,413 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940371d575bc060ce157a7d 2025-12-17 16:28:18,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:18,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940371d575bc060ce157a7d HTTP/1.1" 200 None 2025-12-17 16:28:18,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:18,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:28:18,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:18,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940371d575bc060ce157a7d HTTP/1.1" 200 None 2025-12-17 16:28:18,452 - lava_callback - INFO - Completed processing callback for node 6940371d575bc060ce157a7d 2025-12-17 16:28:25,112 - lava_callback - INFO - Received callback for node 6942da22e43d2b76e90222a1 INFO: 172.21.0.1:37722 - "POST /node/6942da22e43d2b76e90222a1 HTTP/1.0" 202 Accepted 2025-12-17 16:28:25,124 - lava_callback - INFO - Starting async job submit for node 6942da22e43d2b76e90222a1 2025-12-17 16:28:25,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:25,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da22e43d2b76e90222a1 HTTP/1.1" 200 None 2025-12-17 16:28:25,154 - lava_callback - DEBUG - Processing node 6942da22e43d2b76e90222a1: baseline-arm 2025-12-17 16:28:25,164 - lava_callback - WARNING - Failed to upload log 2025-12-17 16:28:25,164 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:28:25,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:28:25,360 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:28:25,361 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942da22e43d2b76e90222a1/lava_callback.json.gz 2025-12-17 16:28:25,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:25,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da22e43d2b76e90222a1 HTTP/1.1" 200 None 2025-12-17 16:28:25,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:25,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:28:25,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:25,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da22e43d2b76e90222a1 HTTP/1.1" 200 None 2025-12-17 16:28:25,455 - lava_callback - INFO - Completed processing callback for node 6942da22e43d2b76e90222a1 2025-12-17 16:28:25,942 - lava_callback - INFO - Received callback for node 69403a0f575bc060ce1582e4 INFO: 172.21.0.1:37726 - "POST /node/69403a0f575bc060ce1582e4 HTTP/1.0" 202 Accepted 2025-12-17 16:28:25,951 - lava_callback - INFO - Starting async job submit for node 69403a0f575bc060ce1582e4 2025-12-17 16:28:26,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:26,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403a0f575bc060ce1582e4 HTTP/1.1" 200 None 2025-12-17 16:28:26,051 - lava_callback - DEBUG - Processing node 69403a0f575bc060ce1582e4: baseline-nfs-arm64 INFO: 172.21.0.1:37738 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:28:27,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:28:27,692 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:28:27,692 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69403a0f575bc060ce1582e4/log.txt.gz 2025-12-17 16:28:27,693 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:28:27,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:28:27,903 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:28:27,904 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69403a0f575bc060ce1582e4/lava_callback.json.gz DEBUG: Setup failed for 69403a0f575bc060ce1582e4. Transitting job node result: incomplete -> fail 2025-12-17 16:28:28,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:28,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403a0f575bc060ce1582e4 HTTP/1.1" 200 None 2025-12-17 16:28:28,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:28,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693bd0faf552eae6fe201eb6 HTTP/1.1" 200 None 2025-12-17 16:28:28,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:28,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403a0f575bc060ce1582e4 HTTP/1.1" 200 None 2025-12-17 16:28:28,197 - lava_callback - INFO - Completed processing callback for node 69403a0f575bc060ce1582e4 2025-12-17 16:28:35,027 - lava_callback - INFO - Received callback for node 6940371f575bc060ce157a80 INFO: 172.21.0.1:47260 - "POST /node/6940371f575bc060ce157a80 HTTP/1.0" 202 Accepted 2025-12-17 16:28:35,045 - lava_callback - INFO - Starting async job submit for node 6940371f575bc060ce157a80 2025-12-17 16:28:35,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:35,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940371f575bc060ce157a80 HTTP/1.1" 200 None 2025-12-17 16:28:35,066 - lava_callback - DEBUG - Processing node 6940371f575bc060ce157a80: baseline-arm64-baylibre 2025-12-17 16:28:35,066 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940371f575bc060ce157a80 2025-12-17 16:28:35,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:35,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940371f575bc060ce157a80 HTTP/1.1" 200 None 2025-12-17 16:28:35,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:35,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:28:35,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:35,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940371f575bc060ce157a80 HTTP/1.1" 200 None 2025-12-17 16:28:35,136 - lava_callback - INFO - Completed processing callback for node 6940371f575bc060ce157a80 2025-12-17 16:28:35,723 - lava_callback - INFO - Received callback for node 69403722575bc060ce157ae5 INFO: 172.21.0.1:47274 - "POST /node/69403722575bc060ce157ae5 HTTP/1.0" 202 Accepted 2025-12-17 16:28:35,729 - lava_callback - INFO - Starting async job submit for node 69403722575bc060ce157ae5 2025-12-17 16:28:35,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:35,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403722575bc060ce157ae5 HTTP/1.1" 200 None 2025-12-17 16:28:35,739 - lava_callback - DEBUG - Processing node 69403722575bc060ce157ae5: baseline-arm64-baylibre 2025-12-17 16:28:35,739 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403722575bc060ce157ae5 2025-12-17 16:28:35,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:35,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403722575bc060ce157ae5 HTTP/1.1" 200 None 2025-12-17 16:28:35,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:35,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:28:35,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:35,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403722575bc060ce157ae5 HTTP/1.1" 200 None 2025-12-17 16:28:35,775 - lava_callback - INFO - Completed processing callback for node 69403722575bc060ce157ae5 2025-12-17 16:28:36,375 - lava_callback - INFO - Received callback for node 69403724575bc060ce157ae8 INFO: 172.21.0.1:47282 - "POST /node/69403724575bc060ce157ae8 HTTP/1.0" 202 Accepted 2025-12-17 16:28:36,387 - lava_callback - INFO - Starting async job submit for node 69403724575bc060ce157ae8 2025-12-17 16:28:36,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:36,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403724575bc060ce157ae8 HTTP/1.1" 200 None 2025-12-17 16:28:36,400 - lava_callback - DEBUG - Processing node 69403724575bc060ce157ae8: baseline-arm64-baylibre 2025-12-17 16:28:36,400 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403724575bc060ce157ae8 2025-12-17 16:28:36,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:36,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403724575bc060ce157ae8 HTTP/1.1" 200 None 2025-12-17 16:28:36,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:36,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:28:36,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:36,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403724575bc060ce157ae8 HTTP/1.1" 200 None 2025-12-17 16:28:36,451 - lava_callback - INFO - Completed processing callback for node 69403724575bc060ce157ae8 INFO: 172.21.0.1:47284 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:28:37,074 - lava_callback - INFO - Received callback for node 69403726575bc060ce157aeb INFO: 172.21.0.1:47292 - "POST /node/69403726575bc060ce157aeb HTTP/1.0" 202 Accepted 2025-12-17 16:28:37,081 - lava_callback - INFO - Starting async job submit for node 69403726575bc060ce157aeb 2025-12-17 16:28:37,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:37,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403726575bc060ce157aeb HTTP/1.1" 200 None 2025-12-17 16:28:37,093 - lava_callback - DEBUG - Processing node 69403726575bc060ce157aeb: baseline-arm64-baylibre 2025-12-17 16:28:37,093 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403726575bc060ce157aeb 2025-12-17 16:28:37,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:37,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403726575bc060ce157aeb HTTP/1.1" 200 None 2025-12-17 16:28:37,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:37,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:28:37,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:37,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403726575bc060ce157aeb HTTP/1.1" 200 None 2025-12-17 16:28:37,137 - lava_callback - INFO - Completed processing callback for node 69403726575bc060ce157aeb 2025-12-17 16:28:37,750 - lava_callback - INFO - Received callback for node 69403728575bc060ce157b00 INFO: 172.21.0.1:47296 - "POST /node/69403728575bc060ce157b00 HTTP/1.0" 202 Accepted 2025-12-17 16:28:37,756 - lava_callback - INFO - Starting async job submit for node 69403728575bc060ce157b00 2025-12-17 16:28:37,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:37,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403728575bc060ce157b00 HTTP/1.1" 200 None 2025-12-17 16:28:37,765 - lava_callback - DEBUG - Processing node 69403728575bc060ce157b00: baseline-arm64-baylibre 2025-12-17 16:28:37,765 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403728575bc060ce157b00 2025-12-17 16:28:37,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:37,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403728575bc060ce157b00 HTTP/1.1" 200 None 2025-12-17 16:28:37,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:37,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:28:37,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:37,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403728575bc060ce157b00 HTTP/1.1" 200 None 2025-12-17 16:28:37,837 - lava_callback - INFO - Completed processing callback for node 69403728575bc060ce157b00 2025-12-17 16:28:38,412 - lava_callback - INFO - Received callback for node 6940372b575bc060ce157c68 INFO: 172.21.0.1:47300 - "POST /node/6940372b575bc060ce157c68 HTTP/1.0" 202 Accepted 2025-12-17 16:28:38,422 - lava_callback - INFO - Starting async job submit for node 6940372b575bc060ce157c68 2025-12-17 16:28:38,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:38,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940372b575bc060ce157c68 HTTP/1.1" 200 None 2025-12-17 16:28:38,434 - lava_callback - DEBUG - Processing node 6940372b575bc060ce157c68: baseline-arm64-baylibre 2025-12-17 16:28:38,434 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940372b575bc060ce157c68 2025-12-17 16:28:38,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:38,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940372b575bc060ce157c68 HTTP/1.1" 200 None 2025-12-17 16:28:38,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:38,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:28:38,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:38,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940372b575bc060ce157c68 HTTP/1.1" 200 None 2025-12-17 16:28:38,473 - lava_callback - INFO - Completed processing callback for node 6940372b575bc060ce157c68 2025-12-17 16:28:39,078 - lava_callback - INFO - Received callback for node 6940372d575bc060ce157c70 INFO: 172.21.0.1:47306 - "POST /node/6940372d575bc060ce157c70 HTTP/1.0" 202 Accepted 2025-12-17 16:28:39,086 - lava_callback - INFO - Starting async job submit for node 6940372d575bc060ce157c70 2025-12-17 16:28:39,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:39,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940372d575bc060ce157c70 HTTP/1.1" 200 None 2025-12-17 16:28:39,095 - lava_callback - DEBUG - Processing node 6940372d575bc060ce157c70: baseline-arm64-baylibre 2025-12-17 16:28:39,095 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940372d575bc060ce157c70 2025-12-17 16:28:39,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:39,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940372d575bc060ce157c70 HTTP/1.1" 200 None 2025-12-17 16:28:39,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:39,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:28:39,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:39,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940372d575bc060ce157c70 HTTP/1.1" 200 None 2025-12-17 16:28:39,140 - lava_callback - INFO - Completed processing callback for node 6940372d575bc060ce157c70 2025-12-17 16:28:39,720 - lava_callback - INFO - Received callback for node 69403730575bc060ce157c74 INFO: 172.21.0.1:53354 - "POST /node/69403730575bc060ce157c74 HTTP/1.0" 202 Accepted 2025-12-17 16:28:39,729 - lava_callback - INFO - Starting async job submit for node 69403730575bc060ce157c74 2025-12-17 16:28:39,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:39,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403730575bc060ce157c74 HTTP/1.1" 200 None 2025-12-17 16:28:39,743 - lava_callback - DEBUG - Processing node 69403730575bc060ce157c74: baseline-arm64-baylibre 2025-12-17 16:28:39,743 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403730575bc060ce157c74 2025-12-17 16:28:39,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:39,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403730575bc060ce157c74 HTTP/1.1" 200 None 2025-12-17 16:28:39,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:39,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:28:39,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:39,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403730575bc060ce157c74 HTTP/1.1" 200 None 2025-12-17 16:28:39,796 - lava_callback - INFO - Completed processing callback for node 69403730575bc060ce157c74 2025-12-17 16:28:44,408 - lava_callback - INFO - Received callback for node 69403b84575bc060ce1584ae INFO: 172.21.0.1:53356 - "POST /node/69403b84575bc060ce1584ae HTTP/1.0" 202 Accepted 2025-12-17 16:28:44,419 - lava_callback - INFO - Starting async job submit for node 69403b84575bc060ce1584ae 2025-12-17 16:28:44,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:44,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b84575bc060ce1584ae HTTP/1.1" 200 None 2025-12-17 16:28:44,501 - lava_callback - DEBUG - Processing node 69403b84575bc060ce1584ae: baseline-nfs-arm64 2025-12-17 16:28:46,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:28:46,210 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:28:46,211 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69403b84575bc060ce1584ae/log.txt.gz 2025-12-17 16:28:46,211 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:28:46,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:28:46,454 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:28:46,455 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69403b84575bc060ce1584ae/lava_callback.json.gz INFO: 172.21.0.1:53368 - "GET / HTTP/1.0" 200 OK DEBUG: Setup failed for 69403b84575bc060ce1584ae. Transitting job node result: incomplete -> fail 2025-12-17 16:28:46,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:46,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b84575bc060ce1584ae HTTP/1.1" 200 None 2025-12-17 16:28:46,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:46,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d1eabfad9ee307f5cfc12 HTTP/1.1" 200 None 2025-12-17 16:28:46,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:46,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b84575bc060ce1584ae HTTP/1.1" 200 None 2025-12-17 16:28:46,701 - lava_callback - INFO - Completed processing callback for node 69403b84575bc060ce1584ae 2025-12-17 16:28:55,052 - lava_callback - INFO - Received callback for node 69403737575bc060ce157c87 INFO: 172.21.0.1:52490 - "POST /node/69403737575bc060ce157c87 HTTP/1.0" 202 Accepted 2025-12-17 16:28:55,061 - lava_callback - INFO - Starting async job submit for node 69403737575bc060ce157c87 2025-12-17 16:28:55,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:55,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403737575bc060ce157c87 HTTP/1.1" 200 None 2025-12-17 16:28:55,072 - lava_callback - DEBUG - Processing node 69403737575bc060ce157c87: baseline-arm64-baylibre 2025-12-17 16:28:55,072 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403737575bc060ce157c87 2025-12-17 16:28:55,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:55,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403737575bc060ce157c87 HTTP/1.1" 200 None 2025-12-17 16:28:55,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:55,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:28:55,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:55,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403737575bc060ce157c87 HTTP/1.1" 200 None 2025-12-17 16:28:55,119 - lava_callback - INFO - Completed processing callback for node 69403737575bc060ce157c87 2025-12-17 16:28:55,725 - lava_callback - INFO - Received callback for node 6940373a575bc060ce157c8a INFO: 172.21.0.1:52492 - "POST /node/6940373a575bc060ce157c8a HTTP/1.0" 202 Accepted 2025-12-17 16:28:55,732 - lava_callback - INFO - Starting async job submit for node 6940373a575bc060ce157c8a 2025-12-17 16:28:55,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:55,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940373a575bc060ce157c8a HTTP/1.1" 200 None 2025-12-17 16:28:55,741 - lava_callback - DEBUG - Processing node 6940373a575bc060ce157c8a: baseline-arm64-baylibre 2025-12-17 16:28:55,741 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940373a575bc060ce157c8a 2025-12-17 16:28:55,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:55,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940373a575bc060ce157c8a HTTP/1.1" 200 None 2025-12-17 16:28:55,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:55,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:28:55,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:55,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940373a575bc060ce157c8a HTTP/1.1" 200 None 2025-12-17 16:28:55,787 - lava_callback - INFO - Completed processing callback for node 6940373a575bc060ce157c8a 2025-12-17 16:28:56,402 - lava_callback - INFO - Received callback for node 6940373c575bc060ce157c8d INFO: 172.21.0.1:52494 - "POST /node/6940373c575bc060ce157c8d HTTP/1.0" 202 Accepted 2025-12-17 16:28:56,411 - lava_callback - INFO - Starting async job submit for node 6940373c575bc060ce157c8d 2025-12-17 16:28:56,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:56,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940373c575bc060ce157c8d HTTP/1.1" 200 None 2025-12-17 16:28:56,423 - lava_callback - DEBUG - Processing node 6940373c575bc060ce157c8d: baseline-arm64-baylibre 2025-12-17 16:28:56,423 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940373c575bc060ce157c8d 2025-12-17 16:28:56,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:56,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940373c575bc060ce157c8d HTTP/1.1" 200 None 2025-12-17 16:28:56,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:56,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:28:56,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:56,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940373c575bc060ce157c8d HTTP/1.1" 200 None 2025-12-17 16:28:56,462 - lava_callback - INFO - Completed processing callback for node 6940373c575bc060ce157c8d INFO: 172.21.0.1:52498 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:28:56,583 - lava_callback - INFO - Received callback for node 6942da24e43d2b76e90222b8 INFO: 172.21.0.1:52508 - "POST /node/6942da24e43d2b76e90222b8 HTTP/1.0" 202 Accepted 2025-12-17 16:28:56,631 - lava_callback - INFO - Starting async job submit for node 6942da24e43d2b76e90222b8 2025-12-17 16:28:56,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:56,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da24e43d2b76e90222b8 HTTP/1.1" 200 None 2025-12-17 16:28:56,666 - lava_callback - DEBUG - Processing node 6942da24e43d2b76e90222b8: baseline-arm 2025-12-17 16:28:56,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:28:57,070 - lava_callback - INFO - Received callback for node 6940373e575bc060ce157c9c INFO: 172.21.0.1:52516 - "POST /node/6940373e575bc060ce157c9c HTTP/1.0" 202 Accepted 2025-12-17 16:28:57,081 - lava_callback - INFO - Starting async job submit for node 6940373e575bc060ce157c9c 2025-12-17 16:28:57,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:57,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940373e575bc060ce157c9c HTTP/1.1" 200 None 2025-12-17 16:28:57,092 - lava_callback - DEBUG - Processing node 6940373e575bc060ce157c9c: baseline-arm64-baylibre 2025-12-17 16:28:57,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940373e575bc060ce157c9c 2025-12-17 16:28:57,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:57,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940373e575bc060ce157c9c HTTP/1.1" 200 None 2025-12-17 16:28:57,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:57,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:28:57,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:57,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940373e575bc060ce157c9c HTTP/1.1" 200 None 2025-12-17 16:28:57,137 - lava_callback - INFO - Completed processing callback for node 6940373e575bc060ce157c9c 2025-12-17 16:28:57,720 - lava_callback - INFO - Received callback for node 69403740575bc060ce157c9f INFO: 172.21.0.1:52532 - "POST /node/69403740575bc060ce157c9f HTTP/1.0" 202 Accepted 2025-12-17 16:28:57,728 - lava_callback - INFO - Starting async job submit for node 69403740575bc060ce157c9f 2025-12-17 16:28:57,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:57,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403740575bc060ce157c9f HTTP/1.1" 200 None 2025-12-17 16:28:57,740 - lava_callback - DEBUG - Processing node 69403740575bc060ce157c9f: baseline-arm64-baylibre 2025-12-17 16:28:57,740 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403740575bc060ce157c9f 2025-12-17 16:28:57,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:57,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403740575bc060ce157c9f HTTP/1.1" 200 None 2025-12-17 16:28:57,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:57,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:28:57,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:57,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403740575bc060ce157c9f HTTP/1.1" 200 None 2025-12-17 16:28:57,785 - lava_callback - INFO - Completed processing callback for node 69403740575bc060ce157c9f 2025-12-17 16:28:58,405 - lava_callback - INFO - Received callback for node 69403742575bc060ce157ca2 INFO: 172.21.0.1:52534 - "POST /node/69403742575bc060ce157ca2 HTTP/1.0" 202 Accepted 2025-12-17 16:28:58,411 - lava_callback - INFO - Starting async job submit for node 69403742575bc060ce157ca2 2025-12-17 16:28:58,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:58,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403742575bc060ce157ca2 HTTP/1.1" 200 None 2025-12-17 16:28:58,423 - lava_callback - DEBUG - Processing node 69403742575bc060ce157ca2: baseline-arm64-baylibre 2025-12-17 16:28:58,423 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403742575bc060ce157ca2 2025-12-17 16:28:58,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:58,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403742575bc060ce157ca2 HTTP/1.1" 200 None 2025-12-17 16:28:58,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:58,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:28:58,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:58,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403742575bc060ce157ca2 HTTP/1.1" 200 None 2025-12-17 16:28:58,460 - lava_callback - INFO - Completed processing callback for node 69403742575bc060ce157ca2 2025-12-17 16:28:59,085 - lava_callback - INFO - Received callback for node 69403745575bc060ce157cd5 INFO: 172.21.0.1:52550 - "POST /node/69403745575bc060ce157cd5 HTTP/1.0" 202 Accepted 2025-12-17 16:28:59,094 - lava_callback - INFO - Starting async job submit for node 69403745575bc060ce157cd5 2025-12-17 16:28:59,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:59,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403745575bc060ce157cd5 HTTP/1.1" 200 None 2025-12-17 16:28:59,104 - lava_callback - DEBUG - Processing node 69403745575bc060ce157cd5: baseline-arm64-baylibre 2025-12-17 16:28:59,104 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403745575bc060ce157cd5 2025-12-17 16:28:59,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:59,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403745575bc060ce157cd5 HTTP/1.1" 200 None 2025-12-17 16:28:59,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:59,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:28:59,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:59,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403745575bc060ce157cd5 HTTP/1.1" 200 None 2025-12-17 16:28:59,141 - lava_callback - INFO - Completed processing callback for node 69403745575bc060ce157cd5 2025-12-17 16:28:59,247 - lava_callback - INFO - Received callback for node 6942da21e43d2b76e9022298 INFO: 172.21.0.1:52558 - "POST /node/6942da21e43d2b76e9022298 HTTP/1.0" 202 Accepted 2025-12-17 16:28:59,295 - lava_callback - INFO - Starting async job submit for node 6942da21e43d2b76e9022298 2025-12-17 16:28:59,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:28:59,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da21e43d2b76e9022298 HTTP/1.1" 200 None 2025-12-17 16:28:59,350 - lava_callback - DEBUG - Processing node 6942da21e43d2b76e9022298: baseline-arm 2025-12-17 16:28:59,458 - lava_callback - WARNING - Failed to upload log 2025-12-17 16:28:59,458 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:28:59,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:29:00,649 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:00,650 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942da21e43d2b76e9022298/lava_callback.json.gz 2025-12-17 16:29:00,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:00,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:00,708 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942da24e43d2b76e90222b8/log.txt.gz 2025-12-17 16:29:00,708 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:29:00,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da21e43d2b76e9022298 HTTP/1.1" 200 None 2025-12-17 16:29:00,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:00,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:29:00,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:29:00,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:00,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da21e43d2b76e9022298 HTTP/1.1" 200 None 2025-12-17 16:29:00,740 - lava_callback - INFO - Completed processing callback for node 6942da21e43d2b76e9022298 2025-12-17 16:29:00,930 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:00,931 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942da24e43d2b76e90222b8/lava_callback.json.gz 2025-12-17 16:29:00,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:00,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da24e43d2b76e90222b8 HTTP/1.1" 200 None 2025-12-17 16:29:00,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:00,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:29:00,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:01,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da24e43d2b76e90222b8 HTTP/1.1" 200 None 2025-12-17 16:29:01,010 - lava_callback - INFO - Completed processing callback for node 6942da24e43d2b76e90222b8 INFO: 172.21.0.1:34196 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:29:07,292 - lava_callback - INFO - Received callback for node 69403698575bc060ce1578b2 INFO: 172.21.0.1:34208 - "POST /node/69403698575bc060ce1578b2 HTTP/1.0" 202 Accepted 2025-12-17 16:29:07,303 - lava_callback - INFO - Starting async job submit for node 69403698575bc060ce1578b2 2025-12-17 16:29:07,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:07,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403698575bc060ce1578b2 HTTP/1.1" 200 None 2025-12-17 16:29:07,341 - lava_callback - DEBUG - Processing node 69403698575bc060ce1578b2: blktests-ddp-x86 2025-12-17 16:29:08,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:29:08,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:08,596 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/blktests-ddp-x86-69403698575bc060ce1578b2/log.txt.gz 2025-12-17 16:29:08,596 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:29:08,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:29:08,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:08,842 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/blktests-ddp-x86-69403698575bc060ce1578b2/lava_callback.json.gz 2025-12-17 16:29:08,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:08,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403698575bc060ce1578b2 HTTP/1.1" 200 None 2025-12-17 16:29:08,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:08,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693b5e1c4b47c3f872674fdf HTTP/1.1" 200 None 2025-12-17 16:29:08,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:08,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403698575bc060ce1578b2 HTTP/1.1" 200 None 2025-12-17 16:29:08,933 - lava_callback - INFO - Completed processing callback for node 69403698575bc060ce1578b2 2025-12-17 16:29:09,568 - lava_callback - INFO - Received callback for node 69403752575bc060ce157cf3 INFO: 172.21.0.1:34212 - "POST /node/69403752575bc060ce157cf3 HTTP/1.0" 202 Accepted 2025-12-17 16:29:09,575 - lava_callback - INFO - Starting async job submit for node 69403752575bc060ce157cf3 2025-12-17 16:29:09,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:09,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403752575bc060ce157cf3 HTTP/1.1" 200 None 2025-12-17 16:29:09,600 - lava_callback - DEBUG - Processing node 69403752575bc060ce157cf3: baseline-arm64-clabbe 2025-12-17 16:29:09,601 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403752575bc060ce157cf3 2025-12-17 16:29:09,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:09,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403752575bc060ce157cf3 HTTP/1.1" 200 None 2025-12-17 16:29:09,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:09,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:09,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:09,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403752575bc060ce157cf3 HTTP/1.1" 200 None 2025-12-17 16:29:09,645 - lava_callback - INFO - Completed processing callback for node 69403752575bc060ce157cf3 INFO: 172.21.0.1:57030 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:29:15,070 - lava_callback - INFO - Received callback for node 69403749575bc060ce157cdf INFO: 172.21.0.1:57034 - "POST /node/69403749575bc060ce157cdf HTTP/1.0" 202 Accepted 2025-12-17 16:29:15,077 - lava_callback - INFO - Starting async job submit for node 69403749575bc060ce157cdf 2025-12-17 16:29:15,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:15,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403749575bc060ce157cdf HTTP/1.1" 200 None 2025-12-17 16:29:15,085 - lava_callback - DEBUG - Processing node 69403749575bc060ce157cdf: baseline-arm64-baylibre 2025-12-17 16:29:15,085 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403749575bc060ce157cdf 2025-12-17 16:29:15,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:15,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403749575bc060ce157cdf HTTP/1.1" 200 None 2025-12-17 16:29:15,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:15,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:15,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:15,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403749575bc060ce157cdf HTTP/1.1" 200 None 2025-12-17 16:29:15,128 - lava_callback - INFO - Completed processing callback for node 69403749575bc060ce157cdf INFO: 172.21.0.1:57050 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:29:15,764 - lava_callback - INFO - Received callback for node 6940374b575bc060ce157ce2 INFO: 172.21.0.1:57052 - "POST /node/6940374b575bc060ce157ce2 HTTP/1.0" 202 Accepted 2025-12-17 16:29:15,770 - lava_callback - INFO - Starting async job submit for node 6940374b575bc060ce157ce2 2025-12-17 16:29:15,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:15,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940374b575bc060ce157ce2 HTTP/1.1" 200 None 2025-12-17 16:29:15,785 - lava_callback - DEBUG - Processing node 6940374b575bc060ce157ce2: baseline-arm64-baylibre 2025-12-17 16:29:15,785 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940374b575bc060ce157ce2 2025-12-17 16:29:15,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:15,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940374b575bc060ce157ce2 HTTP/1.1" 200 None 2025-12-17 16:29:15,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:15,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:15,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:15,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940374b575bc060ce157ce2 HTTP/1.1" 200 None 2025-12-17 16:29:15,816 - lava_callback - INFO - Completed processing callback for node 6940374b575bc060ce157ce2 2025-12-17 16:29:16,423 - lava_callback - INFO - Received callback for node 6940374d575bc060ce157ce5 INFO: 172.21.0.1:57068 - "POST /node/6940374d575bc060ce157ce5 HTTP/1.0" 202 Accepted 2025-12-17 16:29:16,430 - lava_callback - INFO - Starting async job submit for node 6940374d575bc060ce157ce5 2025-12-17 16:29:16,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:16,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940374d575bc060ce157ce5 HTTP/1.1" 200 None 2025-12-17 16:29:16,438 - lava_callback - DEBUG - Processing node 6940374d575bc060ce157ce5: baseline-arm64-baylibre 2025-12-17 16:29:16,438 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940374d575bc060ce157ce5 2025-12-17 16:29:16,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:16,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940374d575bc060ce157ce5 HTTP/1.1" 200 None 2025-12-17 16:29:16,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:16,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:16,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:16,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940374d575bc060ce157ce5 HTTP/1.1" 200 None 2025-12-17 16:29:16,478 - lava_callback - INFO - Completed processing callback for node 6940374d575bc060ce157ce5 INFO: 172.21.0.1:57080 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:29:17,076 - lava_callback - INFO - Received callback for node 6940374f575bc060ce157ce8 INFO: 172.21.0.1:57090 - "POST /node/6940374f575bc060ce157ce8 HTTP/1.0" 202 Accepted 2025-12-17 16:29:17,082 - lava_callback - INFO - Starting async job submit for node 6940374f575bc060ce157ce8 2025-12-17 16:29:17,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:17,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940374f575bc060ce157ce8 HTTP/1.1" 200 None 2025-12-17 16:29:17,091 - lava_callback - DEBUG - Processing node 6940374f575bc060ce157ce8: baseline-arm64-baylibre 2025-12-17 16:29:17,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940374f575bc060ce157ce8 2025-12-17 16:29:17,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:17,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940374f575bc060ce157ce8 HTTP/1.1" 200 None 2025-12-17 16:29:17,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:17,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:17,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:17,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940374f575bc060ce157ce8 HTTP/1.1" 200 None 2025-12-17 16:29:17,127 - lava_callback - INFO - Completed processing callback for node 6940374f575bc060ce157ce8 INFO: 172.21.0.1:36832 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:29:26,656 - lava_callback - INFO - Received callback for node 6942da2ae43d2b76e90222f2 INFO: 172.21.0.1:36848 - "POST /node/6942da2ae43d2b76e90222f2 HTTP/1.0" 202 Accepted 2025-12-17 16:29:26,662 - lava_callback - INFO - Starting async job submit for node 6942da2ae43d2b76e90222f2 2025-12-17 16:29:26,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:26,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da2ae43d2b76e90222f2 HTTP/1.1" 200 None 2025-12-17 16:29:26,697 - lava_callback - DEBUG - Processing node 6942da2ae43d2b76e90222f2: baseline-arm-pengutronix 2025-12-17 16:29:26,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:29:27,033 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:27,033 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-6942da2ae43d2b76e90222f2/log.txt.gz 2025-12-17 16:29:27,034 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:29:27,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:29:27,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:27,199 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-6942da2ae43d2b76e90222f2/lava_callback.json.gz 2025-12-17 16:29:27,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:27,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da2ae43d2b76e90222f2 HTTP/1.1" 200 None 2025-12-17 16:29:27,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:27,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:29:27,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:27,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da2ae43d2b76e90222f2 HTTP/1.1" 200 None 2025-12-17 16:29:27,264 - lava_callback - INFO - Completed processing callback for node 6942da2ae43d2b76e90222f2 2025-12-17 16:29:27,772 - lava_callback - INFO - Received callback for node 6942da29e43d2b76e90222ee INFO: 172.21.0.1:36862 - "POST /node/6942da29e43d2b76e90222ee HTTP/1.0" 202 Accepted 2025-12-17 16:29:27,780 - lava_callback - INFO - Starting async job submit for node 6942da29e43d2b76e90222ee 2025-12-17 16:29:27,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:27,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da29e43d2b76e90222ee HTTP/1.1" 200 None 2025-12-17 16:29:27,811 - lava_callback - DEBUG - Processing node 6942da29e43d2b76e90222ee: baseline-arm-pengutronix 2025-12-17 16:29:27,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:29:28,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:28,131 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-6942da29e43d2b76e90222ee/log.txt.gz 2025-12-17 16:29:28,131 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:29:28,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:29:28,299 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:28,300 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-6942da29e43d2b76e90222ee/lava_callback.json.gz 2025-12-17 16:29:28,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:28,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da29e43d2b76e90222ee HTTP/1.1" 200 None 2025-12-17 16:29:28,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:28,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:29:28,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:28,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da29e43d2b76e90222ee HTTP/1.1" 200 None 2025-12-17 16:29:28,362 - lava_callback - INFO - Completed processing callback for node 6942da29e43d2b76e90222ee 2025-12-17 16:29:29,566 - lava_callback - INFO - Received callback for node 69403754575bc060ce157cfc INFO: 172.21.0.1:36872 - "POST /node/69403754575bc060ce157cfc HTTP/1.0" 202 Accepted 2025-12-17 16:29:29,573 - lava_callback - INFO - Starting async job submit for node 69403754575bc060ce157cfc 2025-12-17 16:29:29,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:29,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403754575bc060ce157cfc HTTP/1.1" 200 None 2025-12-17 16:29:29,584 - lava_callback - DEBUG - Processing node 69403754575bc060ce157cfc: baseline-arm64-clabbe 2025-12-17 16:29:29,584 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403754575bc060ce157cfc 2025-12-17 16:29:29,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:29,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403754575bc060ce157cfc HTTP/1.1" 200 None 2025-12-17 16:29:29,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:29,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:29,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:29,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403754575bc060ce157cfc HTTP/1.1" 200 None 2025-12-17 16:29:29,622 - lava_callback - INFO - Completed processing callback for node 69403754575bc060ce157cfc 2025-12-17 16:29:29,905 - lava_callback - INFO - Received callback for node 6942da2be43d2b76e90222f9 INFO: 172.21.0.1:36964 - "POST /node/6942da2be43d2b76e90222f9 HTTP/1.0" 202 Accepted 2025-12-17 16:29:29,957 - lava_callback - INFO - Starting async job submit for node 6942da2be43d2b76e90222f9 2025-12-17 16:29:29,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:29,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da2be43d2b76e90222f9 HTTP/1.1" 200 None 2025-12-17 16:29:29,990 - lava_callback - DEBUG - Processing node 6942da2be43d2b76e90222f9: baseline-arm-pengutronix 2025-12-17 16:29:30,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:29:30,244 - lava_callback - INFO - Received callback for node 69403756575bc060ce157cff INFO: 172.21.0.1:36976 - "POST /node/69403756575bc060ce157cff HTTP/1.0" 202 Accepted 2025-12-17 16:29:30,258 - lava_callback - INFO - Starting async job submit for node 69403756575bc060ce157cff 2025-12-17 16:29:30,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:30,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:30,274 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-6942da2be43d2b76e90222f9/log.txt.gz 2025-12-17 16:29:30,274 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:29:30,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:29:30,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403756575bc060ce157cff HTTP/1.1" 200 None 2025-12-17 16:29:30,328 - lava_callback - DEBUG - Processing node 69403756575bc060ce157cff: baseline-arm64-clabbe 2025-12-17 16:29:30,328 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403756575bc060ce157cff 2025-12-17 16:29:30,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:30,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403756575bc060ce157cff HTTP/1.1" 200 None 2025-12-17 16:29:30,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:30,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:30,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:30,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403756575bc060ce157cff HTTP/1.1" 200 None 2025-12-17 16:29:30,377 - lava_callback - INFO - Completed processing callback for node 69403756575bc060ce157cff 2025-12-17 16:29:30,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:30,460 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-6942da2be43d2b76e90222f9/lava_callback.json.gz 2025-12-17 16:29:30,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:30,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da2be43d2b76e90222f9 HTTP/1.1" 200 None 2025-12-17 16:29:30,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:30,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:29:30,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:30,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da2be43d2b76e90222f9 HTTP/1.1" 200 None 2025-12-17 16:29:30,530 - lava_callback - INFO - Completed processing callback for node 6942da2be43d2b76e90222f9 2025-12-17 16:29:30,941 - lava_callback - INFO - Received callback for node 69403758575bc060ce157d02 INFO: 172.21.0.1:36982 - "POST /node/69403758575bc060ce157d02 HTTP/1.0" 202 Accepted 2025-12-17 16:29:30,948 - lava_callback - INFO - Starting async job submit for node 69403758575bc060ce157d02 2025-12-17 16:29:30,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:30,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403758575bc060ce157d02 HTTP/1.1" 200 None 2025-12-17 16:29:30,958 - lava_callback - DEBUG - Processing node 69403758575bc060ce157d02: baseline-arm64-clabbe 2025-12-17 16:29:30,958 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403758575bc060ce157d02 2025-12-17 16:29:30,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:30,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403758575bc060ce157d02 HTTP/1.1" 200 None 2025-12-17 16:29:30,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:30,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:30,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:30,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403758575bc060ce157d02 HTTP/1.1" 200 None 2025-12-17 16:29:31,004 - lava_callback - INFO - Completed processing callback for node 69403758575bc060ce157d02 2025-12-17 16:29:31,605 - lava_callback - INFO - Received callback for node 6940375b575bc060ce157d05 INFO: 172.21.0.1:36984 - "POST /node/6940375b575bc060ce157d05 HTTP/1.0" 202 Accepted 2025-12-17 16:29:31,613 - lava_callback - INFO - Starting async job submit for node 6940375b575bc060ce157d05 2025-12-17 16:29:31,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:31,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940375b575bc060ce157d05 HTTP/1.1" 200 None 2025-12-17 16:29:31,622 - lava_callback - DEBUG - Processing node 6940375b575bc060ce157d05: baseline-arm64-clabbe 2025-12-17 16:29:31,622 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940375b575bc060ce157d05 2025-12-17 16:29:31,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:31,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940375b575bc060ce157d05 HTTP/1.1" 200 None 2025-12-17 16:29:31,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:31,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:31,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:31,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940375b575bc060ce157d05 HTTP/1.1" 200 None 2025-12-17 16:29:31,659 - lava_callback - INFO - Completed processing callback for node 6940375b575bc060ce157d05 2025-12-17 16:29:35,089 - lava_callback - INFO - Received callback for node 69403760575bc060ce157d12 INFO: 172.21.0.1:36994 - "POST /node/69403760575bc060ce157d12 HTTP/1.0" 202 Accepted 2025-12-17 16:29:35,095 - lava_callback - INFO - Starting async job submit for node 69403760575bc060ce157d12 2025-12-17 16:29:35,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:35,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403760575bc060ce157d12 HTTP/1.1" 200 None 2025-12-17 16:29:35,106 - lava_callback - DEBUG - Processing node 69403760575bc060ce157d12: baseline-arm64-baylibre 2025-12-17 16:29:35,106 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403760575bc060ce157d12 2025-12-17 16:29:35,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:35,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403760575bc060ce157d12 HTTP/1.1" 200 None 2025-12-17 16:29:35,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:35,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:35,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:35,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403760575bc060ce157d12 HTTP/1.1" 200 None 2025-12-17 16:29:35,143 - lava_callback - INFO - Completed processing callback for node 69403760575bc060ce157d12 2025-12-17 16:29:35,776 - lava_callback - INFO - Received callback for node 69403763575bc060ce157d24 INFO: 172.21.0.1:36998 - "POST /node/69403763575bc060ce157d24 HTTP/1.0" 202 Accepted 2025-12-17 16:29:35,785 - lava_callback - INFO - Starting async job submit for node 69403763575bc060ce157d24 2025-12-17 16:29:35,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:35,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403763575bc060ce157d24 HTTP/1.1" 200 None 2025-12-17 16:29:35,795 - lava_callback - DEBUG - Processing node 69403763575bc060ce157d24: baseline-arm64-baylibre 2025-12-17 16:29:35,796 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403763575bc060ce157d24 2025-12-17 16:29:35,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:35,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403763575bc060ce157d24 HTTP/1.1" 200 None 2025-12-17 16:29:35,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:35,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:35,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:35,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403763575bc060ce157d24 HTTP/1.1" 200 None 2025-12-17 16:29:35,834 - lava_callback - INFO - Completed processing callback for node 69403763575bc060ce157d24 2025-12-17 16:29:36,445 - lava_callback - INFO - Received callback for node 69403765575bc060ce157d2b INFO: 172.21.0.1:37014 - "POST /node/69403765575bc060ce157d2b HTTP/1.0" 202 Accepted 2025-12-17 16:29:36,452 - lava_callback - INFO - Starting async job submit for node 69403765575bc060ce157d2b 2025-12-17 16:29:36,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:36,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403765575bc060ce157d2b HTTP/1.1" 200 None 2025-12-17 16:29:36,462 - lava_callback - DEBUG - Processing node 69403765575bc060ce157d2b: baseline-arm64-baylibre 2025-12-17 16:29:36,462 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403765575bc060ce157d2b 2025-12-17 16:29:36,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:36,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403765575bc060ce157d2b HTTP/1.1" 200 None 2025-12-17 16:29:36,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:36,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:36,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:36,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403765575bc060ce157d2b HTTP/1.1" 200 None 2025-12-17 16:29:36,509 - lava_callback - INFO - Completed processing callback for node 69403765575bc060ce157d2b INFO: 172.21.0.1:37018 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:29:37,081 - lava_callback - INFO - Received callback for node 69403767575bc060ce157d36 INFO: 172.21.0.1:37026 - "POST /node/69403767575bc060ce157d36 HTTP/1.0" 202 Accepted 2025-12-17 16:29:37,090 - lava_callback - INFO - Starting async job submit for node 69403767575bc060ce157d36 2025-12-17 16:29:37,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:37,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403767575bc060ce157d36 HTTP/1.1" 200 None 2025-12-17 16:29:37,100 - lava_callback - DEBUG - Processing node 69403767575bc060ce157d36: baseline-arm64-baylibre 2025-12-17 16:29:37,101 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403767575bc060ce157d36 2025-12-17 16:29:37,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:37,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403767575bc060ce157d36 HTTP/1.1" 200 None 2025-12-17 16:29:37,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:37,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:37,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:37,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403767575bc060ce157d36 HTTP/1.1" 200 None 2025-12-17 16:29:37,156 - lava_callback - INFO - Completed processing callback for node 69403767575bc060ce157d36 2025-12-17 16:29:37,754 - lava_callback - INFO - Received callback for node 69403769575bc060ce157d39 INFO: 172.21.0.1:37028 - "POST /node/69403769575bc060ce157d39 HTTP/1.0" 202 Accepted 2025-12-17 16:29:37,764 - lava_callback - INFO - Starting async job submit for node 69403769575bc060ce157d39 2025-12-17 16:29:37,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:38,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403769575bc060ce157d39 HTTP/1.1" 200 None 2025-12-17 16:29:38,126 - lava_callback - DEBUG - Processing node 69403769575bc060ce157d39: baseline-arm64-baylibre 2025-12-17 16:29:38,127 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403769575bc060ce157d39 2025-12-17 16:29:38,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:38,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403769575bc060ce157d39 HTTP/1.1" 200 None 2025-12-17 16:29:38,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:38,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:38,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:38,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403769575bc060ce157d39 HTTP/1.1" 200 None 2025-12-17 16:29:38,176 - lava_callback - INFO - Completed processing callback for node 69403769575bc060ce157d39 2025-12-17 16:29:38,422 - lava_callback - INFO - Received callback for node 6940376b575bc060ce157d3c INFO: 172.21.0.1:37034 - "POST /node/6940376b575bc060ce157d3c HTTP/1.0" 202 Accepted 2025-12-17 16:29:38,431 - lava_callback - INFO - Starting async job submit for node 6940376b575bc060ce157d3c 2025-12-17 16:29:38,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:38,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940376b575bc060ce157d3c HTTP/1.1" 200 None 2025-12-17 16:29:38,444 - lava_callback - DEBUG - Processing node 6940376b575bc060ce157d3c: baseline-arm64-baylibre 2025-12-17 16:29:38,444 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940376b575bc060ce157d3c 2025-12-17 16:29:38,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:38,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940376b575bc060ce157d3c HTTP/1.1" 200 None 2025-12-17 16:29:38,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:38,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:38,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:38,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940376b575bc060ce157d3c HTTP/1.1" 200 None 2025-12-17 16:29:38,488 - lava_callback - INFO - Completed processing callback for node 6940376b575bc060ce157d3c 2025-12-17 16:29:39,101 - lava_callback - INFO - Received callback for node 6940376d575bc060ce157d3f INFO: 172.21.0.1:37042 - "POST /node/6940376d575bc060ce157d3f HTTP/1.0" 202 Accepted 2025-12-17 16:29:39,111 - lava_callback - INFO - Starting async job submit for node 6940376d575bc060ce157d3f 2025-12-17 16:29:39,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:39,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940376d575bc060ce157d3f HTTP/1.1" 200 None 2025-12-17 16:29:39,125 - lava_callback - DEBUG - Processing node 6940376d575bc060ce157d3f: baseline-arm64-baylibre 2025-12-17 16:29:39,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940376d575bc060ce157d3f 2025-12-17 16:29:39,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:39,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940376d575bc060ce157d3f HTTP/1.1" 200 None 2025-12-17 16:29:39,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:39,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:39,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:39,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940376d575bc060ce157d3f HTTP/1.1" 200 None 2025-12-17 16:29:39,176 - lava_callback - INFO - Completed processing callback for node 6940376d575bc060ce157d3f 2025-12-17 16:29:42,374 - lava_callback - INFO - Received callback for node 6942da25e43d2b76e90222be INFO: 172.21.0.1:39148 - "POST /node/6942da25e43d2b76e90222be HTTP/1.0" 202 Accepted 2025-12-17 16:29:42,383 - lava_callback - INFO - Starting async job submit for node 6942da25e43d2b76e90222be 2025-12-17 16:29:42,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:42,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da25e43d2b76e90222be HTTP/1.1" 200 None 2025-12-17 16:29:42,418 - lava_callback - DEBUG - Processing node 6942da25e43d2b76e90222be: baseline-arm-broonie 2025-12-17 16:29:42,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:29:42,855 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:42,856 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-6942da25e43d2b76e90222be/log.txt.gz 2025-12-17 16:29:42,856 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:29:42,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:29:43,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:43,005 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-6942da25e43d2b76e90222be/lava_callback.json.gz 2025-12-17 16:29:43,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:43,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da25e43d2b76e90222be HTTP/1.1" 200 None 2025-12-17 16:29:43,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:43,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:29:43,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:43,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da25e43d2b76e90222be HTTP/1.1" 200 None 2025-12-17 16:29:43,074 - lava_callback - INFO - Completed processing callback for node 6942da25e43d2b76e90222be INFO: 172.21.0.1:39162 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:29:47,845 - lava_callback - INFO - Received callback for node 6942da23e43d2b76e90222b5 INFO: 172.21.0.1:39176 - "POST /node/6942da23e43d2b76e90222b5 HTTP/1.0" 202 Accepted 2025-12-17 16:29:47,855 - lava_callback - INFO - Starting async job submit for node 6942da23e43d2b76e90222b5 2025-12-17 16:29:47,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:47,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da23e43d2b76e90222b5 HTTP/1.1" 200 None 2025-12-17 16:29:47,896 - lava_callback - DEBUG - Processing node 6942da23e43d2b76e90222b5: baseline-arm 2025-12-17 16:29:48,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:29:48,453 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:48,454 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942da23e43d2b76e90222b5/log.txt.gz 2025-12-17 16:29:48,454 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:29:48,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:29:48,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:48,607 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942da23e43d2b76e90222b5/lava_callback.json.gz 2025-12-17 16:29:48,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:48,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da23e43d2b76e90222b5 HTTP/1.1" 200 None 2025-12-17 16:29:48,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:48,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:29:48,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:48,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da23e43d2b76e90222b5 HTTP/1.1" 200 None 2025-12-17 16:29:48,678 - lava_callback - INFO - Completed processing callback for node 6942da23e43d2b76e90222b5 2025-12-17 16:29:49,707 - lava_callback - INFO - Received callback for node 6942da22e43d2b76e90222a9 INFO: 172.21.0.1:42388 - "POST /node/6942da22e43d2b76e90222a9 HTTP/1.0" 202 Accepted 2025-12-17 16:29:49,716 - lava_callback - INFO - Starting async job submit for node 6942da22e43d2b76e90222a9 2025-12-17 16:29:49,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:49,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da22e43d2b76e90222a9 HTTP/1.1" 200 None 2025-12-17 16:29:49,754 - lava_callback - DEBUG - Processing node 6942da22e43d2b76e90222a9: baseline-arm 2025-12-17 16:29:50,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:29:50,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:50,199 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942da22e43d2b76e90222a9/log.txt.gz 2025-12-17 16:29:50,200 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:29:50,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:29:50,363 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:29:50,363 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942da22e43d2b76e90222a9/lava_callback.json.gz 2025-12-17 16:29:50,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:50,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da22e43d2b76e90222a9 HTTP/1.1" 200 None 2025-12-17 16:29:50,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:50,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:29:50,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:50,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da22e43d2b76e90222a9 HTTP/1.1" 200 None 2025-12-17 16:29:50,434 - lava_callback - INFO - Completed processing callback for node 6942da22e43d2b76e90222a9 2025-12-17 16:29:55,061 - lava_callback - INFO - Received callback for node 6940376f575bc060ce157d46 INFO: 172.21.0.1:42400 - "POST /node/6940376f575bc060ce157d46 HTTP/1.0" 202 Accepted 2025-12-17 16:29:55,070 - lava_callback - INFO - Starting async job submit for node 6940376f575bc060ce157d46 2025-12-17 16:29:55,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:55,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940376f575bc060ce157d46 HTTP/1.1" 200 None 2025-12-17 16:29:55,080 - lava_callback - DEBUG - Processing node 6940376f575bc060ce157d46: baseline-arm64-baylibre 2025-12-17 16:29:55,080 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940376f575bc060ce157d46 2025-12-17 16:29:55,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:55,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940376f575bc060ce157d46 HTTP/1.1" 200 None 2025-12-17 16:29:55,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:55,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:55,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:55,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940376f575bc060ce157d46 HTTP/1.1" 200 None 2025-12-17 16:29:55,120 - lava_callback - INFO - Completed processing callback for node 6940376f575bc060ce157d46 2025-12-17 16:29:55,758 - lava_callback - INFO - Received callback for node 69403771575bc060ce157d49 INFO: 172.21.0.1:42412 - "POST /node/69403771575bc060ce157d49 HTTP/1.0" 202 Accepted 2025-12-17 16:29:55,765 - lava_callback - INFO - Starting async job submit for node 69403771575bc060ce157d49 2025-12-17 16:29:55,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:55,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403771575bc060ce157d49 HTTP/1.1" 200 None 2025-12-17 16:29:55,775 - lava_callback - DEBUG - Processing node 69403771575bc060ce157d49: baseline-arm64-baylibre 2025-12-17 16:29:55,775 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403771575bc060ce157d49 2025-12-17 16:29:55,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:55,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403771575bc060ce157d49 HTTP/1.1" 200 None 2025-12-17 16:29:55,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:55,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:55,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:55,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403771575bc060ce157d49 HTTP/1.1" 200 None 2025-12-17 16:29:55,813 - lava_callback - INFO - Completed processing callback for node 69403771575bc060ce157d49 2025-12-17 16:29:56,412 - lava_callback - INFO - Received callback for node 69403777575bc060ce157d50 INFO: 172.21.0.1:42428 - "POST /node/69403777575bc060ce157d50 HTTP/1.0" 202 Accepted 2025-12-17 16:29:56,418 - lava_callback - INFO - Starting async job submit for node 69403777575bc060ce157d50 2025-12-17 16:29:56,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:56,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403777575bc060ce157d50 HTTP/1.1" 200 None 2025-12-17 16:29:56,428 - lava_callback - DEBUG - Processing node 69403777575bc060ce157d50: baseline-arm64-baylibre 2025-12-17 16:29:56,429 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403777575bc060ce157d50 2025-12-17 16:29:56,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:56,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403777575bc060ce157d50 HTTP/1.1" 200 None 2025-12-17 16:29:56,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:56,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:56,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:56,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403777575bc060ce157d50 HTTP/1.1" 200 None 2025-12-17 16:29:56,488 - lava_callback - INFO - Completed processing callback for node 69403777575bc060ce157d50 INFO: 172.21.0.1:42442 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:29:57,066 - lava_callback - INFO - Received callback for node 6940377a575bc060ce157d5b INFO: 172.21.0.1:42454 - "POST /node/6940377a575bc060ce157d5b HTTP/1.0" 202 Accepted 2025-12-17 16:29:57,073 - lava_callback - INFO - Starting async job submit for node 6940377a575bc060ce157d5b 2025-12-17 16:29:57,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:57,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940377a575bc060ce157d5b HTTP/1.1" 200 None 2025-12-17 16:29:57,082 - lava_callback - DEBUG - Processing node 6940377a575bc060ce157d5b: baseline-arm64-baylibre 2025-12-17 16:29:57,082 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940377a575bc060ce157d5b 2025-12-17 16:29:57,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:57,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940377a575bc060ce157d5b HTTP/1.1" 200 None 2025-12-17 16:29:57,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:57,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:57,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:57,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940377a575bc060ce157d5b HTTP/1.1" 200 None 2025-12-17 16:29:57,117 - lava_callback - INFO - Completed processing callback for node 6940377a575bc060ce157d5b 2025-12-17 16:29:57,741 - lava_callback - INFO - Received callback for node 6940377c575bc060ce157d5e INFO: 172.21.0.1:42460 - "POST /node/6940377c575bc060ce157d5e HTTP/1.0" 202 Accepted 2025-12-17 16:29:57,748 - lava_callback - INFO - Starting async job submit for node 6940377c575bc060ce157d5e 2025-12-17 16:29:57,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:57,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940377c575bc060ce157d5e HTTP/1.1" 200 None 2025-12-17 16:29:57,757 - lava_callback - DEBUG - Processing node 6940377c575bc060ce157d5e: baseline-arm64-baylibre 2025-12-17 16:29:57,757 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940377c575bc060ce157d5e 2025-12-17 16:29:57,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:57,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940377c575bc060ce157d5e HTTP/1.1" 200 None 2025-12-17 16:29:57,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:57,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:29:57,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:29:57,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940377c575bc060ce157d5e HTTP/1.1" 200 None 2025-12-17 16:29:57,797 - lava_callback - INFO - Completed processing callback for node 6940377c575bc060ce157d5e INFO: 172.21.0.1:43192 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:30:09,317 - lava_callback - INFO - Received callback for node 6942da28e43d2b76e90222dc INFO: 172.21.0.1:43204 - "POST /node/6942da28e43d2b76e90222dc HTTP/1.0" 202 Accepted 2025-12-17 16:30:09,323 - lava_callback - INFO - Starting async job submit for node 6942da28e43d2b76e90222dc 2025-12-17 16:30:09,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:09,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da28e43d2b76e90222dc HTTP/1.1" 200 None 2025-12-17 16:30:09,535 - lava_callback - DEBUG - Processing node 6942da28e43d2b76e90222dc: baseline-arm-kontron 2025-12-17 16:30:09,657 - lava_callback - INFO - Received callback for node 6940377e575bc060ce157d65 INFO: 172.21.0.1:43206 - "POST /node/6940377e575bc060ce157d65 HTTP/1.0" 202 Accepted 2025-12-17 16:30:09,669 - lava_callback - INFO - Starting async job submit for node 6940377e575bc060ce157d65 2025-12-17 16:30:09,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:09,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940377e575bc060ce157d65 HTTP/1.1" 200 None 2025-12-17 16:30:09,758 - lava_callback - DEBUG - Processing node 6940377e575bc060ce157d65: baseline-arm64-clabbe 2025-12-17 16:30:09,758 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940377e575bc060ce157d65 2025-12-17 16:30:09,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:09,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940377e575bc060ce157d65 HTTP/1.1" 200 None 2025-12-17 16:30:09,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:09,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:30:09,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:09,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:09,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940377e575bc060ce157d65 HTTP/1.1" 200 None 2025-12-17 16:30:09,840 - lava_callback - INFO - Completed processing callback for node 6940377e575bc060ce157d65 2025-12-17 16:30:09,998 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:30:09,999 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-6942da28e43d2b76e90222dc/log.txt.gz 2025-12-17 16:30:09,999 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:30:10,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:30:10,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:30:10,199 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-6942da28e43d2b76e90222dc/lava_callback.json.gz 2025-12-17 16:30:10,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:10,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da28e43d2b76e90222dc HTTP/1.1" 200 None 2025-12-17 16:30:10,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:10,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:30:10,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:10,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da28e43d2b76e90222dc HTTP/1.1" 200 None 2025-12-17 16:30:10,270 - lava_callback - INFO - Completed processing callback for node 6942da28e43d2b76e90222dc 2025-12-17 16:30:10,323 - lava_callback - INFO - Received callback for node 69403782575bc060ce157d78 INFO: 172.21.0.1:50560 - "POST /node/69403782575bc060ce157d78 HTTP/1.0" 202 Accepted 2025-12-17 16:30:10,332 - lava_callback - INFO - Starting async job submit for node 69403782575bc060ce157d78 2025-12-17 16:30:10,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:10,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403782575bc060ce157d78 HTTP/1.1" 200 None 2025-12-17 16:30:10,343 - lava_callback - DEBUG - Processing node 69403782575bc060ce157d78: baseline-arm64-clabbe 2025-12-17 16:30:10,344 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403782575bc060ce157d78 2025-12-17 16:30:10,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:10,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403782575bc060ce157d78 HTTP/1.1" 200 None 2025-12-17 16:30:10,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:10,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:10,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:10,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403782575bc060ce157d78 HTTP/1.1" 200 None 2025-12-17 16:30:10,393 - lava_callback - INFO - Completed processing callback for node 69403782575bc060ce157d78 INFO: 172.21.0.1:50576 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:30:10,988 - lava_callback - INFO - Received callback for node 69403783575bc060ce157d7b INFO: 172.21.0.1:50588 - "POST /node/69403783575bc060ce157d7b HTTP/1.0" 202 Accepted 2025-12-17 16:30:10,995 - lava_callback - INFO - Starting async job submit for node 69403783575bc060ce157d7b 2025-12-17 16:30:10,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:11,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403783575bc060ce157d7b HTTP/1.1" 200 None 2025-12-17 16:30:11,005 - lava_callback - DEBUG - Processing node 69403783575bc060ce157d7b: baseline-arm64-clabbe 2025-12-17 16:30:11,005 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403783575bc060ce157d7b 2025-12-17 16:30:11,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:11,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403783575bc060ce157d7b HTTP/1.1" 200 None 2025-12-17 16:30:11,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:11,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:11,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:11,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403783575bc060ce157d7b HTTP/1.1" 200 None 2025-12-17 16:30:11,037 - lava_callback - INFO - Completed processing callback for node 69403783575bc060ce157d7b 2025-12-17 16:30:11,686 - lava_callback - INFO - Received callback for node 69403785575bc060ce157d7e INFO: 172.21.0.1:50604 - "POST /node/69403785575bc060ce157d7e HTTP/1.0" 202 Accepted 2025-12-17 16:30:11,693 - lava_callback - INFO - Starting async job submit for node 69403785575bc060ce157d7e 2025-12-17 16:30:11,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:11,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403785575bc060ce157d7e HTTP/1.1" 200 None 2025-12-17 16:30:11,702 - lava_callback - DEBUG - Processing node 69403785575bc060ce157d7e: baseline-arm64-clabbe 2025-12-17 16:30:11,702 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403785575bc060ce157d7e 2025-12-17 16:30:11,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:11,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403785575bc060ce157d7e HTTP/1.1" 200 None 2025-12-17 16:30:11,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:11,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:11,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:11,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403785575bc060ce157d7e HTTP/1.1" 200 None 2025-12-17 16:30:11,745 - lava_callback - INFO - Completed processing callback for node 69403785575bc060ce157d7e 2025-12-17 16:30:15,110 - lava_callback - INFO - Received callback for node 69403787575bc060ce157d81 INFO: 172.21.0.1:50616 - "POST /node/69403787575bc060ce157d81 HTTP/1.0" 202 Accepted 2025-12-17 16:30:15,116 - lava_callback - INFO - Starting async job submit for node 69403787575bc060ce157d81 2025-12-17 16:30:15,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:15,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403787575bc060ce157d81 HTTP/1.1" 200 None 2025-12-17 16:30:15,125 - lava_callback - DEBUG - Processing node 69403787575bc060ce157d81: baseline-arm64-baylibre 2025-12-17 16:30:15,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403787575bc060ce157d81 2025-12-17 16:30:15,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:15,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403787575bc060ce157d81 HTTP/1.1" 200 None 2025-12-17 16:30:15,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:15,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:15,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:15,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403787575bc060ce157d81 HTTP/1.1" 200 None 2025-12-17 16:30:15,160 - lava_callback - INFO - Completed processing callback for node 69403787575bc060ce157d81 INFO: 172.21.0.1:50624 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:30:15,777 - lava_callback - INFO - Received callback for node 6940378a575bc060ce157d84 INFO: 172.21.0.1:50638 - "POST /node/6940378a575bc060ce157d84 HTTP/1.0" 202 Accepted 2025-12-17 16:30:15,784 - lava_callback - INFO - Starting async job submit for node 6940378a575bc060ce157d84 2025-12-17 16:30:15,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:15,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940378a575bc060ce157d84 HTTP/1.1" 200 None 2025-12-17 16:30:15,793 - lava_callback - DEBUG - Processing node 6940378a575bc060ce157d84: baseline-arm64-baylibre 2025-12-17 16:30:15,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940378a575bc060ce157d84 2025-12-17 16:30:15,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:15,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940378a575bc060ce157d84 HTTP/1.1" 200 None 2025-12-17 16:30:15,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:15,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:15,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:15,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940378a575bc060ce157d84 HTTP/1.1" 200 None 2025-12-17 16:30:15,839 - lava_callback - INFO - Completed processing callback for node 6940378a575bc060ce157d84 2025-12-17 16:30:16,432 - lava_callback - INFO - Received callback for node 6940378d575bc060ce157d8b INFO: 172.21.0.1:50646 - "POST /node/6940378d575bc060ce157d8b HTTP/1.0" 202 Accepted 2025-12-17 16:30:16,439 - lava_callback - INFO - Starting async job submit for node 6940378d575bc060ce157d8b 2025-12-17 16:30:16,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:16,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940378d575bc060ce157d8b HTTP/1.1" 200 None 2025-12-17 16:30:16,448 - lava_callback - DEBUG - Processing node 6940378d575bc060ce157d8b: baseline-arm64-baylibre 2025-12-17 16:30:16,448 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940378d575bc060ce157d8b 2025-12-17 16:30:16,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:16,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940378d575bc060ce157d8b HTTP/1.1" 200 None 2025-12-17 16:30:16,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:16,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:16,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:16,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940378d575bc060ce157d8b HTTP/1.1" 200 None 2025-12-17 16:30:16,482 - lava_callback - INFO - Completed processing callback for node 6940378d575bc060ce157d8b INFO: 172.21.0.1:50662 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:30:17,104 - lava_callback - INFO - Received callback for node 69403790575bc060ce157d95 INFO: 172.21.0.1:50664 - "POST /node/69403790575bc060ce157d95 HTTP/1.0" 202 Accepted 2025-12-17 16:30:17,113 - lava_callback - INFO - Starting async job submit for node 69403790575bc060ce157d95 2025-12-17 16:30:17,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:17,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403790575bc060ce157d95 HTTP/1.1" 200 None 2025-12-17 16:30:17,124 - lava_callback - DEBUG - Processing node 69403790575bc060ce157d95: baseline-arm64-baylibre 2025-12-17 16:30:17,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403790575bc060ce157d95 2025-12-17 16:30:17,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:17,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403790575bc060ce157d95 HTTP/1.1" 200 None 2025-12-17 16:30:17,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:17,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:17,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:17,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403790575bc060ce157d95 HTTP/1.1" 200 None 2025-12-17 16:30:17,192 - lava_callback - INFO - Completed processing callback for node 69403790575bc060ce157d95 2025-12-17 16:30:17,799 - lava_callback - INFO - Received callback for node 69403792575bc060ce157d9c INFO: 172.21.0.1:50676 - "POST /node/69403792575bc060ce157d9c HTTP/1.0" 202 Accepted 2025-12-17 16:30:17,806 - lava_callback - INFO - Starting async job submit for node 69403792575bc060ce157d9c 2025-12-17 16:30:17,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:17,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403792575bc060ce157d9c HTTP/1.1" 200 None 2025-12-17 16:30:17,817 - lava_callback - DEBUG - Processing node 69403792575bc060ce157d9c: baseline-arm64-baylibre 2025-12-17 16:30:17,817 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403792575bc060ce157d9c 2025-12-17 16:30:17,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:17,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403792575bc060ce157d9c HTTP/1.1" 200 None 2025-12-17 16:30:17,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:17,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:17,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:17,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403792575bc060ce157d9c HTTP/1.1" 200 None 2025-12-17 16:30:17,854 - lava_callback - INFO - Completed processing callback for node 69403792575bc060ce157d9c 2025-12-17 16:30:18,471 - lava_callback - INFO - Received callback for node 69403794575bc060ce157d9f INFO: 172.21.0.1:50692 - "POST /node/69403794575bc060ce157d9f HTTP/1.0" 202 Accepted 2025-12-17 16:30:18,477 - lava_callback - INFO - Starting async job submit for node 69403794575bc060ce157d9f 2025-12-17 16:30:18,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:18,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403794575bc060ce157d9f HTTP/1.1" 200 None 2025-12-17 16:30:18,487 - lava_callback - DEBUG - Processing node 69403794575bc060ce157d9f: baseline-arm64-baylibre 2025-12-17 16:30:18,487 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403794575bc060ce157d9f 2025-12-17 16:30:18,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:18,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403794575bc060ce157d9f HTTP/1.1" 200 None 2025-12-17 16:30:18,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:18,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:18,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:18,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403794575bc060ce157d9f HTTP/1.1" 200 None 2025-12-17 16:30:18,521 - lava_callback - INFO - Completed processing callback for node 69403794575bc060ce157d9f 2025-12-17 16:30:23,743 - lava_callback - INFO - Received callback for node 6942da21e43d2b76e902229c INFO: 172.21.0.1:59718 - "POST /node/6942da21e43d2b76e902229c HTTP/1.0" 202 Accepted 2025-12-17 16:30:23,753 - lava_callback - INFO - Starting async job submit for node 6942da21e43d2b76e902229c 2025-12-17 16:30:23,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:23,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da21e43d2b76e902229c HTTP/1.1" 200 None 2025-12-17 16:30:23,796 - lava_callback - DEBUG - Processing node 6942da21e43d2b76e902229c: baseline-arm 2025-12-17 16:30:24,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:30:24,337 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:30:24,338 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942da21e43d2b76e902229c/log.txt.gz 2025-12-17 16:30:24,338 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:30:24,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:30:24,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:30:24,516 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942da21e43d2b76e902229c/lava_callback.json.gz 2025-12-17 16:30:24,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:24,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da21e43d2b76e902229c HTTP/1.1" 200 None 2025-12-17 16:30:24,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:24,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:30:24,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:24,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da21e43d2b76e902229c HTTP/1.1" 200 None 2025-12-17 16:30:24,603 - lava_callback - INFO - Completed processing callback for node 6942da21e43d2b76e902229c INFO: 172.21.0.1:59734 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:30:29,334 - lava_callback - INFO - Received callback for node 6942da27e43d2b76e90222d8 INFO: 172.21.0.1:59750 - "POST /node/6942da27e43d2b76e90222d8 HTTP/1.0" 202 Accepted 2025-12-17 16:30:29,342 - lava_callback - INFO - Starting async job submit for node 6942da27e43d2b76e90222d8 2025-12-17 16:30:29,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:29,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da27e43d2b76e90222d8 HTTP/1.1" 200 None 2025-12-17 16:30:29,377 - lava_callback - DEBUG - Processing node 6942da27e43d2b76e90222d8: baseline-arm-kontron 2025-12-17 16:30:29,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:30:29,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:30:29,729 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-6942da27e43d2b76e90222d8/log.txt.gz 2025-12-17 16:30:29,730 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:30:29,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:30:29,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:30:29,887 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-6942da27e43d2b76e90222d8/lava_callback.json.gz 2025-12-17 16:30:29,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:29,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da27e43d2b76e90222d8 HTTP/1.1" 200 None 2025-12-17 16:30:29,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:29,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:30:29,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:29,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da27e43d2b76e90222d8 HTTP/1.1" 200 None 2025-12-17 16:30:29,946 - lava_callback - INFO - Completed processing callback for node 6942da27e43d2b76e90222d8 2025-12-17 16:30:35,110 - lava_callback - INFO - Received callback for node 69403796575bc060ce157da2 INFO: 172.21.0.1:60710 - "POST /node/69403796575bc060ce157da2 HTTP/1.0" 202 Accepted 2025-12-17 16:30:35,117 - lava_callback - INFO - Starting async job submit for node 69403796575bc060ce157da2 2025-12-17 16:30:35,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:35,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403796575bc060ce157da2 HTTP/1.1" 200 None 2025-12-17 16:30:35,127 - lava_callback - DEBUG - Processing node 69403796575bc060ce157da2: baseline-arm64-baylibre 2025-12-17 16:30:35,127 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403796575bc060ce157da2 2025-12-17 16:30:35,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:35,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403796575bc060ce157da2 HTTP/1.1" 200 None 2025-12-17 16:30:35,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:35,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:35,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:35,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403796575bc060ce157da2 HTTP/1.1" 200 None 2025-12-17 16:30:35,162 - lava_callback - INFO - Completed processing callback for node 69403796575bc060ce157da2 2025-12-17 16:30:35,788 - lava_callback - INFO - Received callback for node 69403798575bc060ce157da7 INFO: 172.21.0.1:60722 - "POST /node/69403798575bc060ce157da7 HTTP/1.0" 202 Accepted 2025-12-17 16:30:35,794 - lava_callback - INFO - Starting async job submit for node 69403798575bc060ce157da7 2025-12-17 16:30:35,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:35,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403798575bc060ce157da7 HTTP/1.1" 200 None 2025-12-17 16:30:35,804 - lava_callback - DEBUG - Processing node 69403798575bc060ce157da7: baseline-arm64-baylibre 2025-12-17 16:30:35,804 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403798575bc060ce157da7 2025-12-17 16:30:35,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:35,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403798575bc060ce157da7 HTTP/1.1" 200 None 2025-12-17 16:30:35,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:35,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:35,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:35,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403798575bc060ce157da7 HTTP/1.1" 200 None 2025-12-17 16:30:35,840 - lava_callback - INFO - Completed processing callback for node 69403798575bc060ce157da7 2025-12-17 16:30:36,476 - lava_callback - INFO - Received callback for node 6940379a575bc060ce157daa INFO: 172.21.0.1:60734 - "POST /node/6940379a575bc060ce157daa HTTP/1.0" 202 Accepted 2025-12-17 16:30:36,484 - lava_callback - INFO - Starting async job submit for node 6940379a575bc060ce157daa 2025-12-17 16:30:36,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:36,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940379a575bc060ce157daa HTTP/1.1" 200 None 2025-12-17 16:30:36,492 - lava_callback - DEBUG - Processing node 6940379a575bc060ce157daa: baseline-arm64-baylibre 2025-12-17 16:30:36,492 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940379a575bc060ce157daa 2025-12-17 16:30:36,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:36,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940379a575bc060ce157daa HTTP/1.1" 200 None 2025-12-17 16:30:36,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:36,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:36,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:60744 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:30:36,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940379a575bc060ce157daa HTTP/1.1" 200 None 2025-12-17 16:30:36,532 - lava_callback - INFO - Completed processing callback for node 6940379a575bc060ce157daa 2025-12-17 16:30:37,143 - lava_callback - INFO - Received callback for node 6940379c575bc060ce157db2 INFO: 172.21.0.1:60750 - "POST /node/6940379c575bc060ce157db2 HTTP/1.0" 202 Accepted 2025-12-17 16:30:37,150 - lava_callback - INFO - Starting async job submit for node 6940379c575bc060ce157db2 2025-12-17 16:30:37,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:37,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940379c575bc060ce157db2 HTTP/1.1" 200 None 2025-12-17 16:30:37,159 - lava_callback - DEBUG - Processing node 6940379c575bc060ce157db2: baseline-arm64-baylibre 2025-12-17 16:30:37,159 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940379c575bc060ce157db2 2025-12-17 16:30:37,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:37,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940379c575bc060ce157db2 HTTP/1.1" 200 None 2025-12-17 16:30:37,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:37,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:37,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:37,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940379c575bc060ce157db2 HTTP/1.1" 200 None 2025-12-17 16:30:37,197 - lava_callback - INFO - Completed processing callback for node 6940379c575bc060ce157db2 2025-12-17 16:30:37,831 - lava_callback - INFO - Received callback for node 6940379e575bc060ce157db5 INFO: 172.21.0.1:60760 - "POST /node/6940379e575bc060ce157db5 HTTP/1.0" 202 Accepted 2025-12-17 16:30:37,838 - lava_callback - INFO - Starting async job submit for node 6940379e575bc060ce157db5 2025-12-17 16:30:37,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:37,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940379e575bc060ce157db5 HTTP/1.1" 200 None 2025-12-17 16:30:37,848 - lava_callback - DEBUG - Processing node 6940379e575bc060ce157db5: baseline-arm64-baylibre 2025-12-17 16:30:37,848 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940379e575bc060ce157db5 2025-12-17 16:30:37,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:37,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940379e575bc060ce157db5 HTTP/1.1" 200 None 2025-12-17 16:30:37,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:37,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:37,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:37,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940379e575bc060ce157db5 HTTP/1.1" 200 None 2025-12-17 16:30:37,887 - lava_callback - INFO - Completed processing callback for node 6940379e575bc060ce157db5 2025-12-17 16:30:38,525 - lava_callback - INFO - Received callback for node 694037a0575bc060ce157dbb INFO: 172.21.0.1:60772 - "POST /node/694037a0575bc060ce157dbb HTTP/1.0" 202 Accepted 2025-12-17 16:30:38,538 - lava_callback - INFO - Starting async job submit for node 694037a0575bc060ce157dbb 2025-12-17 16:30:38,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:38,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037a0575bc060ce157dbb HTTP/1.1" 200 None 2025-12-17 16:30:38,553 - lava_callback - DEBUG - Processing node 694037a0575bc060ce157dbb: baseline-arm64-baylibre 2025-12-17 16:30:38,553 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037a0575bc060ce157dbb 2025-12-17 16:30:38,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:38,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037a0575bc060ce157dbb HTTP/1.1" 200 None 2025-12-17 16:30:38,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:38,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:38,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:38,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037a0575bc060ce157dbb HTTP/1.1" 200 None 2025-12-17 16:30:38,593 - lava_callback - INFO - Completed processing callback for node 694037a0575bc060ce157dbb 2025-12-17 16:30:39,170 - lava_callback - INFO - Received callback for node 694037a3575bc060ce157dc5 INFO: 172.21.0.1:60786 - "POST /node/694037a3575bc060ce157dc5 HTTP/1.0" 202 Accepted 2025-12-17 16:30:39,179 - lava_callback - INFO - Starting async job submit for node 694037a3575bc060ce157dc5 2025-12-17 16:30:39,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:39,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037a3575bc060ce157dc5 HTTP/1.1" 200 None 2025-12-17 16:30:39,191 - lava_callback - DEBUG - Processing node 694037a3575bc060ce157dc5: baseline-arm64-baylibre 2025-12-17 16:30:39,191 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037a3575bc060ce157dc5 2025-12-17 16:30:39,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:39,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037a3575bc060ce157dc5 HTTP/1.1" 200 None 2025-12-17 16:30:39,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:39,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:39,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:39,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037a3575bc060ce157dc5 HTTP/1.1" 200 None 2025-12-17 16:30:39,232 - lava_callback - INFO - Completed processing callback for node 694037a3575bc060ce157dc5 2025-12-17 16:30:39,474 - lava_callback - INFO - Received callback for node 6942da2ee43d2b76e9022314 INFO: 172.21.0.1:60794 - "POST /node/6942da2ee43d2b76e9022314 HTTP/1.0" 202 Accepted 2025-12-17 16:30:39,482 - lava_callback - INFO - Starting async job submit for node 6942da2ee43d2b76e9022314 2025-12-17 16:30:39,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:39,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da2ee43d2b76e9022314 HTTP/1.1" 200 None 2025-12-17 16:30:39,536 - lava_callback - DEBUG - Processing node 6942da2ee43d2b76e9022314: ltp-dio 2025-12-17 16:30:39,875 - lava_callback - INFO - Received callback for node 694037a5575bc060ce157dc8 INFO: 172.21.0.1:44794 - "POST /node/694037a5575bc060ce157dc8 HTTP/1.0" 202 Accepted 2025-12-17 16:30:39,893 - lava_callback - INFO - Starting async job submit for node 694037a5575bc060ce157dc8 2025-12-17 16:30:39,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:39,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037a5575bc060ce157dc8 HTTP/1.1" 200 None 2025-12-17 16:30:39,978 - lava_callback - DEBUG - Processing node 694037a5575bc060ce157dc8: baseline-arm64-baylibre 2025-12-17 16:30:39,979 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037a5575bc060ce157dc8 2025-12-17 16:30:39,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:40,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037a5575bc060ce157dc8 HTTP/1.1" 200 None 2025-12-17 16:30:40,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:40,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:40,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:40,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037a5575bc060ce157dc8 HTTP/1.1" 200 None 2025-12-17 16:30:40,200 - lava_callback - INFO - Completed processing callback for node 694037a5575bc060ce157dc8 2025-12-17 16:30:40,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:30:40,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:30:40,382 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-6942da2ee43d2b76e9022314/log.txt.gz 2025-12-17 16:30:40,382 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:30:40,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:30:40,674 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:30:40,675 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-6942da2ee43d2b76e9022314/lava_callback.json.gz 2025-12-17 16:30:40,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:40,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da2ee43d2b76e9022314 HTTP/1.1" 200 None 2025-12-17 16:30:40,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:40,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:30:40,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:40,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da2ee43d2b76e9022314 HTTP/1.1" 200 None 2025-12-17 16:30:40,814 - lava_callback - INFO - Completed processing callback for node 6942da2ee43d2b76e9022314 INFO: 172.21.0.1:44804 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:30:49,624 - lava_callback - INFO - Received callback for node 694037a7575bc060ce157dcd INFO: 172.21.0.1:44810 - "POST /node/694037a7575bc060ce157dcd HTTP/1.0" 202 Accepted 2025-12-17 16:30:49,630 - lava_callback - INFO - Starting async job submit for node 694037a7575bc060ce157dcd 2025-12-17 16:30:49,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:49,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037a7575bc060ce157dcd HTTP/1.1" 200 None 2025-12-17 16:30:49,640 - lava_callback - DEBUG - Processing node 694037a7575bc060ce157dcd: baseline-arm64-clabbe 2025-12-17 16:30:49,640 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037a7575bc060ce157dcd 2025-12-17 16:30:49,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:49,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037a7575bc060ce157dcd HTTP/1.1" 200 None 2025-12-17 16:30:49,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:49,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:49,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:49,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037a7575bc060ce157dcd HTTP/1.1" 200 None 2025-12-17 16:30:49,681 - lava_callback - INFO - Completed processing callback for node 694037a7575bc060ce157dcd 2025-12-17 16:30:50,278 - lava_callback - INFO - Received callback for node 694037aa575bc060ce157dd0 INFO: 172.21.0.1:58354 - "POST /node/694037aa575bc060ce157dd0 HTTP/1.0" 202 Accepted 2025-12-17 16:30:50,285 - lava_callback - INFO - Starting async job submit for node 694037aa575bc060ce157dd0 2025-12-17 16:30:50,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:50,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037aa575bc060ce157dd0 HTTP/1.1" 200 None 2025-12-17 16:30:50,294 - lava_callback - DEBUG - Processing node 694037aa575bc060ce157dd0: baseline-arm64-clabbe 2025-12-17 16:30:50,294 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037aa575bc060ce157dd0 2025-12-17 16:30:50,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:50,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037aa575bc060ce157dd0 HTTP/1.1" 200 None 2025-12-17 16:30:50,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:50,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:50,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:50,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037aa575bc060ce157dd0 HTTP/1.1" 200 None 2025-12-17 16:30:50,332 - lava_callback - INFO - Completed processing callback for node 694037aa575bc060ce157dd0 2025-12-17 16:30:50,935 - lava_callback - INFO - Received callback for node 694037ac575bc060ce157dd3 INFO: 172.21.0.1:58370 - "POST /node/694037ac575bc060ce157dd3 HTTP/1.0" 202 Accepted 2025-12-17 16:30:50,942 - lava_callback - INFO - Starting async job submit for node 694037ac575bc060ce157dd3 2025-12-17 16:30:50,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:50,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037ac575bc060ce157dd3 HTTP/1.1" 200 None 2025-12-17 16:30:50,976 - lava_callback - DEBUG - Processing node 694037ac575bc060ce157dd3: baseline-arm64-clabbe 2025-12-17 16:30:50,977 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037ac575bc060ce157dd3 2025-12-17 16:30:50,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:50,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037ac575bc060ce157dd3 HTTP/1.1" 200 None 2025-12-17 16:30:50,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:50,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:50,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:51,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037ac575bc060ce157dd3 HTTP/1.1" 200 None 2025-12-17 16:30:51,016 - lava_callback - INFO - Completed processing callback for node 694037ac575bc060ce157dd3 2025-12-17 16:30:51,618 - lava_callback - INFO - Received callback for node 694037ae575bc060ce157dd6 INFO: 172.21.0.1:58382 - "POST /node/694037ae575bc060ce157dd6 HTTP/1.0" 202 Accepted 2025-12-17 16:30:51,626 - lava_callback - INFO - Starting async job submit for node 694037ae575bc060ce157dd6 2025-12-17 16:30:51,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:51,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037ae575bc060ce157dd6 HTTP/1.1" 200 None 2025-12-17 16:30:51,638 - lava_callback - DEBUG - Processing node 694037ae575bc060ce157dd6: baseline-arm64-clabbe 2025-12-17 16:30:51,638 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037ae575bc060ce157dd6 2025-12-17 16:30:51,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:51,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037ae575bc060ce157dd6 HTTP/1.1" 200 None 2025-12-17 16:30:51,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:51,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:51,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:51,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037ae575bc060ce157dd6 HTTP/1.1" 200 None 2025-12-17 16:30:51,674 - lava_callback - INFO - Completed processing callback for node 694037ae575bc060ce157dd6 2025-12-17 16:30:52,293 - lava_callback - INFO - Received callback for node 694037b0575bc060ce157dd9 INFO: 172.21.0.1:58394 - "POST /node/694037b0575bc060ce157dd9 HTTP/1.0" 202 Accepted 2025-12-17 16:30:52,301 - lava_callback - INFO - Starting async job submit for node 694037b0575bc060ce157dd9 2025-12-17 16:30:52,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:52,303 - lava_callback - INFO - Received callback for node 69403377575bc060ce1570bc INFO: 172.21.0.1:58404 - "POST /node/69403377575bc060ce1570bc HTTP/1.0" 202 Accepted 2025-12-17 16:30:52,310 - lava_callback - INFO - Starting async job submit for node 69403377575bc060ce1570bc 2025-12-17 16:30:52,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037b0575bc060ce157dd9 HTTP/1.1" 200 None 2025-12-17 16:30:52,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:52,353 - lava_callback - DEBUG - Processing node 694037b0575bc060ce157dd9: baseline-arm64-clabbe 2025-12-17 16:30:52,354 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037b0575bc060ce157dd9 2025-12-17 16:30:52,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:52,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403377575bc060ce1570bc HTTP/1.1" 200 None 2025-12-17 16:30:52,367 - lava_callback - DEBUG - Processing node 69403377575bc060ce1570bc: baseline-arm 2025-12-17 16:30:52,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037b0575bc060ce157dd9 HTTP/1.1" 200 None 2025-12-17 16:30:52,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:52,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:52,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:52,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037b0575bc060ce157dd9 HTTP/1.1" 200 None 2025-12-17 16:30:52,545 - lava_callback - INFO - Completed processing callback for node 694037b0575bc060ce157dd9 2025-12-17 16:30:52,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:30:52,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:30:52,921 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69403377575bc060ce1570bc/log.txt.gz 2025-12-17 16:30:52,921 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:30:52,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:30:52,973 - lava_callback - INFO - Received callback for node 694037b2575bc060ce157ddc INFO: 172.21.0.1:58410 - "POST /node/694037b2575bc060ce157ddc HTTP/1.0" 202 Accepted 2025-12-17 16:30:52,984 - lava_callback - INFO - Starting async job submit for node 694037b2575bc060ce157ddc 2025-12-17 16:30:52,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:52,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037b2575bc060ce157ddc HTTP/1.1" 200 None 2025-12-17 16:30:52,994 - lava_callback - DEBUG - Processing node 694037b2575bc060ce157ddc: baseline-arm64-clabbe 2025-12-17 16:30:52,994 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037b2575bc060ce157ddc 2025-12-17 16:30:52,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:53,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037b2575bc060ce157ddc HTTP/1.1" 200 None 2025-12-17 16:30:53,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:53,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:53,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:53,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037b2575bc060ce157ddc HTTP/1.1" 200 None 2025-12-17 16:30:53,037 - lava_callback - INFO - Completed processing callback for node 694037b2575bc060ce157ddc 2025-12-17 16:30:53,105 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:30:53,106 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69403377575bc060ce1570bc/lava_callback.json.gz DEBUG: Setup failed for 69403377575bc060ce1570bc. Transitting job node result: incomplete -> fail 2025-12-17 16:30:53,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:53,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403377575bc060ce1570bc HTTP/1.1" 200 None 2025-12-17 16:30:53,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:53,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad816fb52d1dfff36c512 HTTP/1.1" 200 None 2025-12-17 16:30:53,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:53,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403377575bc060ce1570bc HTTP/1.1" 200 None 2025-12-17 16:30:53,243 - lava_callback - INFO - Completed processing callback for node 69403377575bc060ce1570bc 2025-12-17 16:30:53,662 - lava_callback - INFO - Received callback for node 694037b4575bc060ce157de1 INFO: 172.21.0.1:58412 - "POST /node/694037b4575bc060ce157de1 HTTP/1.0" 202 Accepted 2025-12-17 16:30:53,671 - lava_callback - INFO - Starting async job submit for node 694037b4575bc060ce157de1 2025-12-17 16:30:53,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:53,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037b4575bc060ce157de1 HTTP/1.1" 200 None 2025-12-17 16:30:53,684 - lava_callback - DEBUG - Processing node 694037b4575bc060ce157de1: baseline-arm64-clabbe 2025-12-17 16:30:53,684 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037b4575bc060ce157de1 2025-12-17 16:30:53,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:53,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037b4575bc060ce157de1 HTTP/1.1" 200 None 2025-12-17 16:30:53,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:53,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:53,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:53,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037b4575bc060ce157de1 HTTP/1.1" 200 None 2025-12-17 16:30:53,732 - lava_callback - INFO - Completed processing callback for node 694037b4575bc060ce157de1 2025-12-17 16:30:54,351 - lava_callback - INFO - Received callback for node 694037b6575bc060ce157de6 INFO: 172.21.0.1:58416 - "POST /node/694037b6575bc060ce157de6 HTTP/1.0" 202 Accepted 2025-12-17 16:30:54,359 - lava_callback - INFO - Starting async job submit for node 694037b6575bc060ce157de6 2025-12-17 16:30:54,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:54,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037b6575bc060ce157de6 HTTP/1.1" 200 None 2025-12-17 16:30:54,370 - lava_callback - DEBUG - Processing node 694037b6575bc060ce157de6: baseline-arm64-clabbe 2025-12-17 16:30:54,370 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037b6575bc060ce157de6 2025-12-17 16:30:54,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:54,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037b6575bc060ce157de6 HTTP/1.1" 200 None 2025-12-17 16:30:54,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:54,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:54,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:54,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037b6575bc060ce157de6 HTTP/1.1" 200 None 2025-12-17 16:30:54,420 - lava_callback - INFO - Completed processing callback for node 694037b6575bc060ce157de6 2025-12-17 16:30:54,689 - lava_callback - INFO - Received callback for node 6942da94e43d2b76e90224a2 INFO: 172.21.0.1:58424 - "POST /node/6942da94e43d2b76e90224a2 HTTP/1.0" 202 Accepted 2025-12-17 16:30:54,738 - lava_callback - INFO - Starting async job submit for node 6942da94e43d2b76e90224a2 2025-12-17 16:30:54,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:54,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da94e43d2b76e90224a2 HTTP/1.1" 200 None 2025-12-17 16:30:54,766 - lava_callback - DEBUG - Processing node 6942da94e43d2b76e90224a2: baseline-arm 2025-12-17 16:30:54,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:30:55,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:30:55,101 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942da94e43d2b76e90224a2/log.txt.gz 2025-12-17 16:30:55,102 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:30:55,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:30:55,128 - lava_callback - INFO - Received callback for node 694037bc575bc060ce157df3 INFO: 172.21.0.1:58430 - "POST /node/694037bc575bc060ce157df3 HTTP/1.0" 202 Accepted 2025-12-17 16:30:55,136 - lava_callback - INFO - Starting async job submit for node 694037bc575bc060ce157df3 2025-12-17 16:30:55,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:55,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037bc575bc060ce157df3 HTTP/1.1" 200 None 2025-12-17 16:30:55,146 - lava_callback - DEBUG - Processing node 694037bc575bc060ce157df3: baseline-arm64-baylibre 2025-12-17 16:30:55,147 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037bc575bc060ce157df3 2025-12-17 16:30:55,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:55,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037bc575bc060ce157df3 HTTP/1.1" 200 None 2025-12-17 16:30:55,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:55,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:30:55,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:55,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037bc575bc060ce157df3 HTTP/1.1" 200 None 2025-12-17 16:30:55,196 - lava_callback - INFO - Completed processing callback for node 694037bc575bc060ce157df3 2025-12-17 16:30:55,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:30:55,280 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942da94e43d2b76e90224a2/lava_callback.json.gz 2025-12-17 16:30:55,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:55,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da94e43d2b76e90224a2 HTTP/1.1" 200 None 2025-12-17 16:30:55,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:55,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:30:55,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:55,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da94e43d2b76e90224a2 HTTP/1.1" 200 None 2025-12-17 16:30:55,346 - lava_callback - INFO - Completed processing callback for node 6942da94e43d2b76e90224a2 INFO: 172.21.0.1:58442 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:30:57,835 - lava_callback - INFO - Received callback for node 6942da94e43d2b76e90224a5 INFO: 172.21.0.1:58446 - "POST /node/6942da94e43d2b76e90224a5 HTTP/1.0" 202 Accepted 2025-12-17 16:30:57,882 - lava_callback - INFO - Starting async job submit for node 6942da94e43d2b76e90224a5 2025-12-17 16:30:57,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:57,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da94e43d2b76e90224a5 HTTP/1.1" 200 None 2025-12-17 16:30:57,907 - lava_callback - DEBUG - Processing node 6942da94e43d2b76e90224a5: baseline-arm 2025-12-17 16:30:58,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:30:58,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:30:58,159 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942da94e43d2b76e90224a5/log.txt.gz 2025-12-17 16:30:58,159 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:30:58,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:30:58,274 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:30:58,275 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942da94e43d2b76e90224a5/lava_callback.json.gz 2025-12-17 16:30:58,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:58,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da94e43d2b76e90224a5 HTTP/1.1" 200 None 2025-12-17 16:30:58,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:58,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:30:58,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:30:58,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da94e43d2b76e90224a5 HTTP/1.1" 200 None 2025-12-17 16:30:58,348 - lava_callback - INFO - Completed processing callback for node 6942da94e43d2b76e90224a5 INFO: 172.21.0.1:50282 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:31:09,665 - lava_callback - INFO - Received callback for node 694037c4575bc060ce157e00 INFO: 172.21.0.1:50294 - "POST /node/694037c4575bc060ce157e00 HTTP/1.0" 202 Accepted 2025-12-17 16:31:09,672 - lava_callback - INFO - Starting async job submit for node 694037c4575bc060ce157e00 2025-12-17 16:31:09,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:09,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037c4575bc060ce157e00 HTTP/1.1" 200 None 2025-12-17 16:31:09,683 - lava_callback - DEBUG - Processing node 694037c4575bc060ce157e00: baseline-arm64-clabbe 2025-12-17 16:31:09,683 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037c4575bc060ce157e00 2025-12-17 16:31:09,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:09,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037c4575bc060ce157e00 HTTP/1.1" 200 None 2025-12-17 16:31:09,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:09,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:09,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:09,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037c4575bc060ce157e00 HTTP/1.1" 200 None 2025-12-17 16:31:09,725 - lava_callback - INFO - Completed processing callback for node 694037c4575bc060ce157e00 2025-12-17 16:31:10,326 - lava_callback - INFO - Received callback for node 694037c6575bc060ce157e03 INFO: 172.21.0.1:43374 - "POST /node/694037c6575bc060ce157e03 HTTP/1.0" 202 Accepted 2025-12-17 16:31:10,335 - lava_callback - INFO - Starting async job submit for node 694037c6575bc060ce157e03 2025-12-17 16:31:10,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:10,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037c6575bc060ce157e03 HTTP/1.1" 200 None 2025-12-17 16:31:10,347 - lava_callback - DEBUG - Processing node 694037c6575bc060ce157e03: baseline-arm64-clabbe 2025-12-17 16:31:10,347 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037c6575bc060ce157e03 2025-12-17 16:31:10,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:10,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037c6575bc060ce157e03 HTTP/1.1" 200 None 2025-12-17 16:31:10,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:10,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:10,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:10,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037c6575bc060ce157e03 HTTP/1.1" 200 None 2025-12-17 16:31:10,405 - lava_callback - INFO - Completed processing callback for node 694037c6575bc060ce157e03 INFO: 172.21.0.1:43388 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:31:15,151 - lava_callback - INFO - Received callback for node 694037be575bc060ce157df7 INFO: 172.21.0.1:43398 - "POST /node/694037be575bc060ce157df7 HTTP/1.0" 202 Accepted 2025-12-17 16:31:15,158 - lava_callback - INFO - Starting async job submit for node 694037be575bc060ce157df7 2025-12-17 16:31:15,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:43406 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:31:15,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037be575bc060ce157df7 HTTP/1.1" 200 None 2025-12-17 16:31:15,166 - lava_callback - DEBUG - Processing node 694037be575bc060ce157df7: baseline-arm64-baylibre 2025-12-17 16:31:15,166 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037be575bc060ce157df7 2025-12-17 16:31:15,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:15,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037be575bc060ce157df7 HTTP/1.1" 200 None 2025-12-17 16:31:15,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:15,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:15,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:15,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037be575bc060ce157df7 HTTP/1.1" 200 None 2025-12-17 16:31:15,204 - lava_callback - INFO - Completed processing callback for node 694037be575bc060ce157df7 2025-12-17 16:31:15,822 - lava_callback - INFO - Received callback for node 694037c0575bc060ce157dfa INFO: 172.21.0.1:43420 - "POST /node/694037c0575bc060ce157dfa HTTP/1.0" 202 Accepted 2025-12-17 16:31:15,829 - lava_callback - INFO - Starting async job submit for node 694037c0575bc060ce157dfa 2025-12-17 16:31:15,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:15,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037c0575bc060ce157dfa HTTP/1.1" 200 None 2025-12-17 16:31:15,837 - lava_callback - DEBUG - Processing node 694037c0575bc060ce157dfa: baseline-arm64-baylibre 2025-12-17 16:31:15,837 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037c0575bc060ce157dfa 2025-12-17 16:31:15,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:15,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037c0575bc060ce157dfa HTTP/1.1" 200 None 2025-12-17 16:31:15,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:15,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:15,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:15,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037c0575bc060ce157dfa HTTP/1.1" 200 None 2025-12-17 16:31:15,879 - lava_callback - INFO - Completed processing callback for node 694037c0575bc060ce157dfa 2025-12-17 16:31:16,493 - lava_callback - INFO - Received callback for node 694037c2575bc060ce157dfd INFO: 172.21.0.1:43430 - "POST /node/694037c2575bc060ce157dfd HTTP/1.0" 202 Accepted 2025-12-17 16:31:16,500 - lava_callback - INFO - Starting async job submit for node 694037c2575bc060ce157dfd 2025-12-17 16:31:16,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:16,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037c2575bc060ce157dfd HTTP/1.1" 200 None 2025-12-17 16:31:16,510 - lava_callback - DEBUG - Processing node 694037c2575bc060ce157dfd: baseline-arm64-baylibre 2025-12-17 16:31:16,511 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037c2575bc060ce157dfd 2025-12-17 16:31:16,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:43442 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:31:16,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037c2575bc060ce157dfd HTTP/1.1" 200 None 2025-12-17 16:31:16,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:16,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:16,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:16,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037c2575bc060ce157dfd HTTP/1.1" 200 None 2025-12-17 16:31:16,550 - lava_callback - INFO - Completed processing callback for node 694037c2575bc060ce157dfd 2025-12-17 16:31:17,177 - lava_callback - INFO - Received callback for node 694037ca575bc060ce157e09 INFO: 172.21.0.1:43456 - "POST /node/694037ca575bc060ce157e09 HTTP/1.0" 202 Accepted 2025-12-17 16:31:17,184 - lava_callback - INFO - Starting async job submit for node 694037ca575bc060ce157e09 2025-12-17 16:31:17,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:17,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037ca575bc060ce157e09 HTTP/1.1" 200 None 2025-12-17 16:31:17,193 - lava_callback - DEBUG - Processing node 694037ca575bc060ce157e09: baseline-arm64-baylibre 2025-12-17 16:31:17,193 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037ca575bc060ce157e09 2025-12-17 16:31:17,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:17,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037ca575bc060ce157e09 HTTP/1.1" 200 None 2025-12-17 16:31:17,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:17,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:17,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:17,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037ca575bc060ce157e09 HTTP/1.1" 200 None 2025-12-17 16:31:17,234 - lava_callback - INFO - Completed processing callback for node 694037ca575bc060ce157e09 2025-12-17 16:31:17,835 - lava_callback - INFO - Received callback for node 694037cd575bc060ce157e0c INFO: 172.21.0.1:43460 - "POST /node/694037cd575bc060ce157e0c HTTP/1.0" 202 Accepted 2025-12-17 16:31:17,842 - lava_callback - INFO - Starting async job submit for node 694037cd575bc060ce157e0c 2025-12-17 16:31:17,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:17,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037cd575bc060ce157e0c HTTP/1.1" 200 None 2025-12-17 16:31:17,854 - lava_callback - DEBUG - Processing node 694037cd575bc060ce157e0c: baseline-arm64-baylibre 2025-12-17 16:31:17,854 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037cd575bc060ce157e0c 2025-12-17 16:31:17,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:17,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037cd575bc060ce157e0c HTTP/1.1" 200 None 2025-12-17 16:31:17,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:17,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:17,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:17,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037cd575bc060ce157e0c HTTP/1.1" 200 None 2025-12-17 16:31:17,890 - lava_callback - INFO - Completed processing callback for node 694037cd575bc060ce157e0c 2025-12-17 16:31:18,514 - lava_callback - INFO - Received callback for node 694037cf575bc060ce157e15 INFO: 172.21.0.1:43466 - "POST /node/694037cf575bc060ce157e15 HTTP/1.0" 202 Accepted 2025-12-17 16:31:18,519 - lava_callback - INFO - Starting async job submit for node 694037cf575bc060ce157e15 2025-12-17 16:31:18,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:18,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037cf575bc060ce157e15 HTTP/1.1" 200 None 2025-12-17 16:31:18,529 - lava_callback - DEBUG - Processing node 694037cf575bc060ce157e15: baseline-arm64-baylibre 2025-12-17 16:31:18,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037cf575bc060ce157e15 2025-12-17 16:31:18,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:18,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037cf575bc060ce157e15 HTTP/1.1" 200 None 2025-12-17 16:31:18,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:18,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:18,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:18,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037cf575bc060ce157e15 HTTP/1.1" 200 None 2025-12-17 16:31:18,562 - lava_callback - INFO - Completed processing callback for node 694037cf575bc060ce157e15 2025-12-17 16:31:19,183 - lava_callback - INFO - Received callback for node 694037d1575bc060ce157e1a INFO: 172.21.0.1:43470 - "POST /node/694037d1575bc060ce157e1a HTTP/1.0" 202 Accepted 2025-12-17 16:31:19,190 - lava_callback - INFO - Starting async job submit for node 694037d1575bc060ce157e1a 2025-12-17 16:31:19,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:19,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037d1575bc060ce157e1a HTTP/1.1" 200 None 2025-12-17 16:31:19,204 - lava_callback - DEBUG - Processing node 694037d1575bc060ce157e1a: baseline-arm64-baylibre 2025-12-17 16:31:19,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037d1575bc060ce157e1a 2025-12-17 16:31:19,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:19,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037d1575bc060ce157e1a HTTP/1.1" 200 None 2025-12-17 16:31:19,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:19,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:19,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:19,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037d1575bc060ce157e1a HTTP/1.1" 200 None 2025-12-17 16:31:19,261 - lava_callback - INFO - Completed processing callback for node 694037d1575bc060ce157e1a INFO: 172.21.0.1:33784 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:31:29,647 - lava_callback - INFO - Received callback for node 694037d5575bc060ce157e20 INFO: 172.21.0.1:33798 - "POST /node/694037d5575bc060ce157e20 HTTP/1.0" 202 Accepted 2025-12-17 16:31:29,653 - lava_callback - INFO - Starting async job submit for node 694037d5575bc060ce157e20 2025-12-17 16:31:29,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:29,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037d5575bc060ce157e20 HTTP/1.1" 200 None 2025-12-17 16:31:29,668 - lava_callback - DEBUG - Processing node 694037d5575bc060ce157e20: baseline-arm64-clabbe 2025-12-17 16:31:29,668 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037d5575bc060ce157e20 2025-12-17 16:31:29,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:29,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037d5575bc060ce157e20 HTTP/1.1" 200 None 2025-12-17 16:31:29,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:29,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:29,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:29,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037d5575bc060ce157e20 HTTP/1.1" 200 None 2025-12-17 16:31:29,701 - lava_callback - INFO - Completed processing callback for node 694037d5575bc060ce157e20 2025-12-17 16:31:30,307 - lava_callback - INFO - Received callback for node 694037dd575bc060ce157e2c INFO: 172.21.0.1:33230 - "POST /node/694037dd575bc060ce157e2c HTTP/1.0" 202 Accepted 2025-12-17 16:31:30,315 - lava_callback - INFO - Starting async job submit for node 694037dd575bc060ce157e2c 2025-12-17 16:31:30,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:30,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037dd575bc060ce157e2c HTTP/1.1" 200 None 2025-12-17 16:31:30,327 - lava_callback - DEBUG - Processing node 694037dd575bc060ce157e2c: baseline-arm64-clabbe 2025-12-17 16:31:30,327 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037dd575bc060ce157e2c 2025-12-17 16:31:30,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:30,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037dd575bc060ce157e2c HTTP/1.1" 200 None 2025-12-17 16:31:30,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:30,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:30,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:30,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037dd575bc060ce157e2c HTTP/1.1" 200 None 2025-12-17 16:31:30,365 - lava_callback - INFO - Completed processing callback for node 694037dd575bc060ce157e2c 2025-12-17 16:31:31,000 - lava_callback - INFO - Received callback for node 694037df575bc060ce157e2f INFO: 172.21.0.1:33246 - "POST /node/694037df575bc060ce157e2f HTTP/1.0" 202 Accepted 2025-12-17 16:31:31,006 - lava_callback - INFO - Starting async job submit for node 694037df575bc060ce157e2f 2025-12-17 16:31:31,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:31,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037df575bc060ce157e2f HTTP/1.1" 200 None 2025-12-17 16:31:31,015 - lava_callback - DEBUG - Processing node 694037df575bc060ce157e2f: baseline-arm64-clabbe 2025-12-17 16:31:31,015 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037df575bc060ce157e2f 2025-12-17 16:31:31,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:31,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037df575bc060ce157e2f HTTP/1.1" 200 None 2025-12-17 16:31:31,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:31,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:31:31,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:31,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037df575bc060ce157e2f HTTP/1.1" 200 None 2025-12-17 16:31:31,049 - lava_callback - INFO - Completed processing callback for node 694037df575bc060ce157e2f 2025-12-17 16:31:35,156 - lava_callback - INFO - Received callback for node 694037d3575bc060ce157e1d INFO: 172.21.0.1:33258 - "POST /node/694037d3575bc060ce157e1d HTTP/1.0" 202 Accepted 2025-12-17 16:31:35,162 - lava_callback - INFO - Starting async job submit for node 694037d3575bc060ce157e1d 2025-12-17 16:31:35,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:35,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037d3575bc060ce157e1d HTTP/1.1" 200 None 2025-12-17 16:31:35,171 - lava_callback - DEBUG - Processing node 694037d3575bc060ce157e1d: baseline-arm64-baylibre 2025-12-17 16:31:35,172 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037d3575bc060ce157e1d 2025-12-17 16:31:35,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:35,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037d3575bc060ce157e1d HTTP/1.1" 200 None 2025-12-17 16:31:35,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:35,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:35,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:35,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037d3575bc060ce157e1d HTTP/1.1" 200 None 2025-12-17 16:31:35,205 - lava_callback - INFO - Completed processing callback for node 694037d3575bc060ce157e1d 2025-12-17 16:31:35,787 - lava_callback - INFO - Received callback for node 694037d7575bc060ce157e23 INFO: 172.21.0.1:33268 - "POST /node/694037d7575bc060ce157e23 HTTP/1.0" 202 Accepted 2025-12-17 16:31:35,794 - lava_callback - INFO - Starting async job submit for node 694037d7575bc060ce157e23 2025-12-17 16:31:35,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:35,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037d7575bc060ce157e23 HTTP/1.1" 200 None 2025-12-17 16:31:35,803 - lava_callback - DEBUG - Processing node 694037d7575bc060ce157e23: baseline-arm64-baylibre 2025-12-17 16:31:35,803 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037d7575bc060ce157e23 2025-12-17 16:31:35,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:35,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037d7575bc060ce157e23 HTTP/1.1" 200 None 2025-12-17 16:31:35,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:35,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:35,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:35,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037d7575bc060ce157e23 HTTP/1.1" 200 None 2025-12-17 16:31:35,856 - lava_callback - INFO - Completed processing callback for node 694037d7575bc060ce157e23 INFO: 172.21.0.1:33282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57772 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:31:49,661 - lava_callback - INFO - Received callback for node 694037e1575bc060ce157e33 INFO: 172.21.0.1:57774 - "POST /node/694037e1575bc060ce157e33 HTTP/1.0" 202 Accepted 2025-12-17 16:31:49,667 - lava_callback - INFO - Starting async job submit for node 694037e1575bc060ce157e33 2025-12-17 16:31:49,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:49,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037e1575bc060ce157e33 HTTP/1.1" 200 None 2025-12-17 16:31:49,677 - lava_callback - DEBUG - Processing node 694037e1575bc060ce157e33: baseline-arm64-clabbe 2025-12-17 16:31:49,677 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037e1575bc060ce157e33 2025-12-17 16:31:49,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:49,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037e1575bc060ce157e33 HTTP/1.1" 200 None 2025-12-17 16:31:49,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:49,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:31:49,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:49,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037e1575bc060ce157e33 HTTP/1.1" 200 None 2025-12-17 16:31:49,715 - lava_callback - INFO - Completed processing callback for node 694037e1575bc060ce157e33 2025-12-17 16:31:50,327 - lava_callback - INFO - Received callback for node 694037e3575bc060ce157e3d INFO: 172.21.0.1:47860 - "POST /node/694037e3575bc060ce157e3d HTTP/1.0" 202 Accepted 2025-12-17 16:31:50,337 - lava_callback - INFO - Starting async job submit for node 694037e3575bc060ce157e3d 2025-12-17 16:31:50,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:50,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037e3575bc060ce157e3d HTTP/1.1" 200 None 2025-12-17 16:31:50,349 - lava_callback - DEBUG - Processing node 694037e3575bc060ce157e3d: baseline-arm64-clabbe 2025-12-17 16:31:50,349 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037e3575bc060ce157e3d 2025-12-17 16:31:50,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:50,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037e3575bc060ce157e3d HTTP/1.1" 200 None 2025-12-17 16:31:50,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:50,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:31:50,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:50,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037e3575bc060ce157e3d HTTP/1.1" 200 None 2025-12-17 16:31:50,398 - lava_callback - INFO - Completed processing callback for node 694037e3575bc060ce157e3d 2025-12-17 16:31:51,003 - lava_callback - INFO - Received callback for node 694037e5575bc060ce157e44 INFO: 172.21.0.1:47872 - "POST /node/694037e5575bc060ce157e44 HTTP/1.0" 202 Accepted 2025-12-17 16:31:51,009 - lava_callback - INFO - Starting async job submit for node 694037e5575bc060ce157e44 2025-12-17 16:31:51,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:51,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037e5575bc060ce157e44 HTTP/1.1" 200 None 2025-12-17 16:31:51,018 - lava_callback - DEBUG - Processing node 694037e5575bc060ce157e44: baseline-arm64-clabbe 2025-12-17 16:31:51,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037e5575bc060ce157e44 2025-12-17 16:31:51,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:51,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037e5575bc060ce157e44 HTTP/1.1" 200 None 2025-12-17 16:31:51,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:51,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:31:51,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:51,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037e5575bc060ce157e44 HTTP/1.1" 200 None 2025-12-17 16:31:51,060 - lava_callback - INFO - Completed processing callback for node 694037e5575bc060ce157e44 2025-12-17 16:31:51,673 - lava_callback - INFO - Received callback for node 694037e7575bc060ce157e47 INFO: 172.21.0.1:47886 - "POST /node/694037e7575bc060ce157e47 HTTP/1.0" 202 Accepted 2025-12-17 16:31:51,681 - lava_callback - INFO - Starting async job submit for node 694037e7575bc060ce157e47 2025-12-17 16:31:51,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:51,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037e7575bc060ce157e47 HTTP/1.1" 200 None 2025-12-17 16:31:51,698 - lava_callback - DEBUG - Processing node 694037e7575bc060ce157e47: baseline-arm64-clabbe 2025-12-17 16:31:51,698 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037e7575bc060ce157e47 2025-12-17 16:31:51,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:51,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037e7575bc060ce157e47 HTTP/1.1" 200 None 2025-12-17 16:31:51,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:51,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:31:51,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:51,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037e7575bc060ce157e47 HTTP/1.1" 200 None 2025-12-17 16:31:51,762 - lava_callback - INFO - Completed processing callback for node 694037e7575bc060ce157e47 2025-12-17 16:31:52,369 - lava_callback - INFO - Received callback for node 694037e9575bc060ce157e4a INFO: 172.21.0.1:47888 - "POST /node/694037e9575bc060ce157e4a HTTP/1.0" 202 Accepted 2025-12-17 16:31:52,379 - lava_callback - INFO - Starting async job submit for node 694037e9575bc060ce157e4a 2025-12-17 16:31:52,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:52,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037e9575bc060ce157e4a HTTP/1.1" 200 None 2025-12-17 16:31:52,390 - lava_callback - DEBUG - Processing node 694037e9575bc060ce157e4a: baseline-arm64-clabbe 2025-12-17 16:31:52,390 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037e9575bc060ce157e4a 2025-12-17 16:31:52,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:52,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037e9575bc060ce157e4a HTTP/1.1" 200 None 2025-12-17 16:31:52,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:52,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:52,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:52,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037e9575bc060ce157e4a HTTP/1.1" 200 None 2025-12-17 16:31:52,423 - lava_callback - INFO - Completed processing callback for node 694037e9575bc060ce157e4a 2025-12-17 16:31:53,043 - lava_callback - INFO - Received callback for node 694037eb575bc060ce157e4d INFO: 172.21.0.1:47896 - "POST /node/694037eb575bc060ce157e4d HTTP/1.0" 202 Accepted 2025-12-17 16:31:53,051 - lava_callback - INFO - Starting async job submit for node 694037eb575bc060ce157e4d 2025-12-17 16:31:53,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:53,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037eb575bc060ce157e4d HTTP/1.1" 200 None 2025-12-17 16:31:53,065 - lava_callback - DEBUG - Processing node 694037eb575bc060ce157e4d: baseline-arm64-clabbe 2025-12-17 16:31:53,065 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037eb575bc060ce157e4d 2025-12-17 16:31:53,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:53,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037eb575bc060ce157e4d HTTP/1.1" 200 None 2025-12-17 16:31:53,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:53,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:53,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:53,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037eb575bc060ce157e4d HTTP/1.1" 200 None 2025-12-17 16:31:53,117 - lava_callback - INFO - Completed processing callback for node 694037eb575bc060ce157e4d 2025-12-17 16:31:53,730 - lava_callback - INFO - Received callback for node 694037ee575bc060ce157e50 INFO: 172.21.0.1:47904 - "POST /node/694037ee575bc060ce157e50 HTTP/1.0" 202 Accepted 2025-12-17 16:31:53,738 - lava_callback - INFO - Starting async job submit for node 694037ee575bc060ce157e50 2025-12-17 16:31:53,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:53,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037ee575bc060ce157e50 HTTP/1.1" 200 None 2025-12-17 16:31:53,748 - lava_callback - DEBUG - Processing node 694037ee575bc060ce157e50: baseline-arm64-clabbe 2025-12-17 16:31:53,749 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037ee575bc060ce157e50 2025-12-17 16:31:53,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:53,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037ee575bc060ce157e50 HTTP/1.1" 200 None 2025-12-17 16:31:53,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:53,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:53,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:53,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037ee575bc060ce157e50 HTTP/1.1" 200 None 2025-12-17 16:31:53,795 - lava_callback - INFO - Completed processing callback for node 694037ee575bc060ce157e50 2025-12-17 16:31:53,917 - lava_callback - INFO - Received callback for node 6942da23e43d2b76e90222ac INFO: 172.21.0.1:47918 - "POST /node/6942da23e43d2b76e90222ac HTTP/1.0" 202 Accepted 2025-12-17 16:31:53,926 - lava_callback - INFO - Starting async job submit for node 6942da23e43d2b76e90222ac 2025-12-17 16:31:53,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:53,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da23e43d2b76e90222ac HTTP/1.1" 200 None 2025-12-17 16:31:53,970 - lava_callback - DEBUG - Processing node 6942da23e43d2b76e90222ac: baseline-arm 2025-12-17 16:31:54,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:31:54,396 - lava_callback - INFO - Received callback for node 694037f0575bc060ce157e53 2025-12-17 16:31:54,409 - lava_callback - INFO - Starting async job submit for node 694037f0575bc060ce157e53 INFO: 172.21.0.1:47930 - "POST /node/694037f0575bc060ce157e53 HTTP/1.0" 202 Accepted 2025-12-17 16:31:54,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:54,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:31:54,412 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942da23e43d2b76e90222ac/log.txt.gz 2025-12-17 16:31:54,412 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:31:54,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:31:54,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037f0575bc060ce157e53 HTTP/1.1" 200 None 2025-12-17 16:31:54,421 - lava_callback - DEBUG - Processing node 694037f0575bc060ce157e53: baseline-arm64-clabbe 2025-12-17 16:31:54,421 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037f0575bc060ce157e53 2025-12-17 16:31:54,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:54,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037f0575bc060ce157e53 HTTP/1.1" 200 None 2025-12-17 16:31:54,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:54,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:54,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:54,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037f0575bc060ce157e53 HTTP/1.1" 200 None 2025-12-17 16:31:54,468 - lava_callback - INFO - Completed processing callback for node 694037f0575bc060ce157e53 2025-12-17 16:31:54,641 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:31:54,642 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942da23e43d2b76e90222ac/lava_callback.json.gz 2025-12-17 16:31:54,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:54,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da23e43d2b76e90222ac HTTP/1.1" 200 None 2025-12-17 16:31:54,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:54,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:31:54,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:54,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da23e43d2b76e90222ac HTTP/1.1" 200 None 2025-12-17 16:31:54,735 - lava_callback - INFO - Completed processing callback for node 6942da23e43d2b76e90222ac 2025-12-17 16:31:55,071 - lava_callback - INFO - Received callback for node 694037f2575bc060ce157e56 INFO: 172.21.0.1:47946 - "POST /node/694037f2575bc060ce157e56 HTTP/1.0" 202 Accepted 2025-12-17 16:31:55,078 - lava_callback - INFO - Starting async job submit for node 694037f2575bc060ce157e56 2025-12-17 16:31:55,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:55,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037f2575bc060ce157e56 HTTP/1.1" 200 None 2025-12-17 16:31:55,086 - lava_callback - DEBUG - Processing node 694037f2575bc060ce157e56: baseline-arm64-clabbe 2025-12-17 16:31:55,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037f2575bc060ce157e56 2025-12-17 16:31:55,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:55,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037f2575bc060ce157e56 HTTP/1.1" 200 None 2025-12-17 16:31:55,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:55,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:31:55,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:31:55,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037f2575bc060ce157e56 HTTP/1.1" 200 None 2025-12-17 16:31:55,127 - lava_callback - INFO - Completed processing callback for node 694037f2575bc060ce157e56 INFO: 172.21.0.1:47952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59082 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:32:09,707 - lava_callback - INFO - Received callback for node 694037f6575bc060ce157e5b INFO: 172.21.0.1:33730 - "POST /node/694037f6575bc060ce157e5b HTTP/1.0" 202 Accepted 2025-12-17 16:32:09,712 - lava_callback - INFO - Starting async job submit for node 694037f6575bc060ce157e5b 2025-12-17 16:32:09,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:09,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037f6575bc060ce157e5b HTTP/1.1" 200 None 2025-12-17 16:32:09,723 - lava_callback - DEBUG - Processing node 694037f6575bc060ce157e5b: baseline-arm64-clabbe 2025-12-17 16:32:09,723 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037f6575bc060ce157e5b 2025-12-17 16:32:09,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:09,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037f6575bc060ce157e5b HTTP/1.1" 200 None 2025-12-17 16:32:09,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:09,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:32:09,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:09,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037f6575bc060ce157e5b HTTP/1.1" 200 None 2025-12-17 16:32:09,760 - lava_callback - INFO - Completed processing callback for node 694037f6575bc060ce157e5b 2025-12-17 16:32:10,370 - lava_callback - INFO - Received callback for node 694037f8575bc060ce157e64 INFO: 172.21.0.1:33740 - "POST /node/694037f8575bc060ce157e64 HTTP/1.0" 202 Accepted 2025-12-17 16:32:10,376 - lava_callback - INFO - Starting async job submit for node 694037f8575bc060ce157e64 2025-12-17 16:32:10,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:10,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037f8575bc060ce157e64 HTTP/1.1" 200 None 2025-12-17 16:32:10,385 - lava_callback - DEBUG - Processing node 694037f8575bc060ce157e64: baseline-arm64-clabbe 2025-12-17 16:32:10,385 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037f8575bc060ce157e64 2025-12-17 16:32:10,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:10,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037f8575bc060ce157e64 HTTP/1.1" 200 None 2025-12-17 16:32:10,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:10,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:32:10,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:10,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037f8575bc060ce157e64 HTTP/1.1" 200 None 2025-12-17 16:32:10,417 - lava_callback - INFO - Completed processing callback for node 694037f8575bc060ce157e64 INFO: 172.21.0.1:33746 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:32:11,030 - lava_callback - INFO - Received callback for node 694037fa575bc060ce157e67 INFO: 172.21.0.1:33758 - "POST /node/694037fa575bc060ce157e67 HTTP/1.0" 202 Accepted 2025-12-17 16:32:11,036 - lava_callback - INFO - Starting async job submit for node 694037fa575bc060ce157e67 2025-12-17 16:32:11,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:11,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037fa575bc060ce157e67 HTTP/1.1" 200 None 2025-12-17 16:32:11,050 - lava_callback - DEBUG - Processing node 694037fa575bc060ce157e67: baseline-arm64-clabbe 2025-12-17 16:32:11,050 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037fa575bc060ce157e67 2025-12-17 16:32:11,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:11,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037fa575bc060ce157e67 HTTP/1.1" 200 None 2025-12-17 16:32:11,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:11,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:32:11,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:11,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037fa575bc060ce157e67 HTTP/1.1" 200 None 2025-12-17 16:32:11,089 - lava_callback - INFO - Completed processing callback for node 694037fa575bc060ce157e67 2025-12-17 16:32:11,746 - lava_callback - INFO - Received callback for node 694037fc575bc060ce157e6a INFO: 172.21.0.1:33772 - "POST /node/694037fc575bc060ce157e6a HTTP/1.0" 202 Accepted 2025-12-17 16:32:11,752 - lava_callback - INFO - Starting async job submit for node 694037fc575bc060ce157e6a 2025-12-17 16:32:11,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:11,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037fc575bc060ce157e6a HTTP/1.1" 200 None 2025-12-17 16:32:11,761 - lava_callback - DEBUG - Processing node 694037fc575bc060ce157e6a: baseline-arm64-clabbe 2025-12-17 16:32:11,762 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694037fc575bc060ce157e6a 2025-12-17 16:32:11,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:11,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694037fc575bc060ce157e6a HTTP/1.1" 200 None 2025-12-17 16:32:11,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:11,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21c507de2c0cbc264ec3 HTTP/1.1" 200 None 2025-12-17 16:32:11,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:11,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694037fc575bc060ce157e6a HTTP/1.1" 200 None 2025-12-17 16:32:11,796 - lava_callback - INFO - Completed processing callback for node 694037fc575bc060ce157e6a INFO: 172.21.0.1:33774 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:32:15,192 - lava_callback - INFO - Received callback for node 69403807575bc060ce157e78 INFO: 172.21.0.1:33780 - "POST /node/69403807575bc060ce157e78 HTTP/1.0" 202 Accepted 2025-12-17 16:32:15,202 - lava_callback - INFO - Starting async job submit for node 69403807575bc060ce157e78 2025-12-17 16:32:15,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:15,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403807575bc060ce157e78 HTTP/1.1" 200 None 2025-12-17 16:32:15,215 - lava_callback - DEBUG - Processing node 69403807575bc060ce157e78: baseline-arm64-baylibre 2025-12-17 16:32:15,215 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403807575bc060ce157e78 2025-12-17 16:32:15,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:15,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403807575bc060ce157e78 HTTP/1.1" 200 None 2025-12-17 16:32:15,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:15,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:32:15,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:15,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403807575bc060ce157e78 HTTP/1.1" 200 None 2025-12-17 16:32:15,248 - lava_callback - INFO - Completed processing callback for node 69403807575bc060ce157e78 2025-12-17 16:32:15,299 - lava_callback - INFO - Received callback for node 69403809575bc060ce157e7b INFO: 172.21.0.1:33792 - "POST /node/69403809575bc060ce157e7b HTTP/1.0" 202 Accepted 2025-12-17 16:32:15,308 - lava_callback - INFO - Starting async job submit for node 69403809575bc060ce157e7b 2025-12-17 16:32:15,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:15,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403809575bc060ce157e7b HTTP/1.1" 200 None 2025-12-17 16:32:15,317 - lava_callback - DEBUG - Processing node 69403809575bc060ce157e7b: baseline-x86-kcidebug-intel 2025-12-17 16:32:15,317 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403809575bc060ce157e7b 2025-12-17 16:32:15,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:15,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403809575bc060ce157e7b HTTP/1.1" 200 None 2025-12-17 16:32:15,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:15,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af14dfee8383e92b69bdc HTTP/1.1" 200 None 2025-12-17 16:32:15,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:15,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403809575bc060ce157e7b HTTP/1.1" 200 None 2025-12-17 16:32:15,355 - lava_callback - INFO - Completed processing callback for node 69403809575bc060ce157e7b 2025-12-17 16:32:15,866 - lava_callback - INFO - Received callback for node 69403804575bc060ce157e6f INFO: 172.21.0.1:33806 - "POST /node/69403804575bc060ce157e6f HTTP/1.0" 202 Accepted 2025-12-17 16:32:15,872 - lava_callback - INFO - Starting async job submit for node 69403804575bc060ce157e6f 2025-12-17 16:32:15,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:15,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403804575bc060ce157e6f HTTP/1.1" 200 None 2025-12-17 16:32:15,882 - lava_callback - DEBUG - Processing node 69403804575bc060ce157e6f: baseline-arm64-baylibre 2025-12-17 16:32:15,883 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403804575bc060ce157e6f 2025-12-17 16:32:15,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:15,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403804575bc060ce157e6f HTTP/1.1" 200 None 2025-12-17 16:32:15,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:15,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:32:15,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:15,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403804575bc060ce157e6f HTTP/1.1" 200 None 2025-12-17 16:32:15,913 - lava_callback - INFO - Completed processing callback for node 69403804575bc060ce157e6f INFO: 172.21.0.1:33814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45032 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:32:35,208 - lava_callback - INFO - Received callback for node 69403818575bc060ce157e80 INFO: 172.21.0.1:36710 - "POST /node/69403818575bc060ce157e80 HTTP/1.0" 202 Accepted 2025-12-17 16:32:35,215 - lava_callback - INFO - Starting async job submit for node 69403818575bc060ce157e80 2025-12-17 16:32:35,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:35,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403818575bc060ce157e80 HTTP/1.1" 200 None 2025-12-17 16:32:35,224 - lava_callback - DEBUG - Processing node 69403818575bc060ce157e80: baseline-arm64-baylibre 2025-12-17 16:32:35,224 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403818575bc060ce157e80 2025-12-17 16:32:35,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:35,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403818575bc060ce157e80 HTTP/1.1" 200 None 2025-12-17 16:32:35,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:35,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:32:35,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:35,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403818575bc060ce157e80 HTTP/1.1" 200 None 2025-12-17 16:32:35,264 - lava_callback - INFO - Completed processing callback for node 69403818575bc060ce157e80 2025-12-17 16:32:35,867 - lava_callback - INFO - Received callback for node 6940381b575bc060ce157e8b INFO: 172.21.0.1:36720 - "POST /node/6940381b575bc060ce157e8b HTTP/1.0" 202 Accepted 2025-12-17 16:32:35,874 - lava_callback - INFO - Starting async job submit for node 6940381b575bc060ce157e8b 2025-12-17 16:32:35,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:35,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940381b575bc060ce157e8b HTTP/1.1" 200 None 2025-12-17 16:32:35,885 - lava_callback - DEBUG - Processing node 6940381b575bc060ce157e8b: baseline-arm64-baylibre 2025-12-17 16:32:35,885 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940381b575bc060ce157e8b 2025-12-17 16:32:35,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:35,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940381b575bc060ce157e8b HTTP/1.1" 200 None 2025-12-17 16:32:35,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:35,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:32:35,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:35,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940381b575bc060ce157e8b HTTP/1.1" 200 None 2025-12-17 16:32:35,919 - lava_callback - INFO - Completed processing callback for node 6940381b575bc060ce157e8b INFO: 172.21.0.1:36734 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:32:36,541 - lava_callback - INFO - Received callback for node 6940381d575bc060ce157e8e INFO: 172.21.0.1:36738 - "POST /node/6940381d575bc060ce157e8e HTTP/1.0" 202 Accepted 2025-12-17 16:32:36,548 - lava_callback - INFO - Starting async job submit for node 6940381d575bc060ce157e8e 2025-12-17 16:32:36,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:36,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940381d575bc060ce157e8e HTTP/1.1" 200 None 2025-12-17 16:32:36,559 - lava_callback - DEBUG - Processing node 6940381d575bc060ce157e8e: baseline-arm64-baylibre 2025-12-17 16:32:36,559 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940381d575bc060ce157e8e 2025-12-17 16:32:36,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:36,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940381d575bc060ce157e8e HTTP/1.1" 200 None 2025-12-17 16:32:36,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:36,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:32:36,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:36,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940381d575bc060ce157e8e HTTP/1.1" 200 None 2025-12-17 16:32:36,595 - lava_callback - INFO - Completed processing callback for node 6940381d575bc060ce157e8e INFO: 172.21.0.1:47856 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:32:55,267 - lava_callback - INFO - Received callback for node 69403823575bc060ce157e91 INFO: 172.21.0.1:42432 - "POST /node/69403823575bc060ce157e91 HTTP/1.0" 202 Accepted 2025-12-17 16:32:55,274 - lava_callback - INFO - Starting async job submit for node 69403823575bc060ce157e91 2025-12-17 16:32:55,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:55,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403823575bc060ce157e91 HTTP/1.1" 200 None 2025-12-17 16:32:55,284 - lava_callback - DEBUG - Processing node 69403823575bc060ce157e91: baseline-arm64-baylibre 2025-12-17 16:32:55,284 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403823575bc060ce157e91 2025-12-17 16:32:55,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:55,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403823575bc060ce157e91 HTTP/1.1" 200 None 2025-12-17 16:32:55,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:55,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:32:55,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:55,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403823575bc060ce157e91 HTTP/1.1" 200 None 2025-12-17 16:32:55,324 - lava_callback - INFO - Completed processing callback for node 69403823575bc060ce157e91 2025-12-17 16:32:55,944 - lava_callback - INFO - Received callback for node 69403825575bc060ce157e98 INFO: 172.21.0.1:42446 - "POST /node/69403825575bc060ce157e98 HTTP/1.0" 202 Accepted 2025-12-17 16:32:55,951 - lava_callback - INFO - Starting async job submit for node 69403825575bc060ce157e98 2025-12-17 16:32:55,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:55,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403825575bc060ce157e98 HTTP/1.1" 200 None 2025-12-17 16:32:55,968 - lava_callback - DEBUG - Processing node 69403825575bc060ce157e98: baseline-arm64-baylibre 2025-12-17 16:32:55,968 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403825575bc060ce157e98 2025-12-17 16:32:55,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:55,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403825575bc060ce157e98 HTTP/1.1" 200 None 2025-12-17 16:32:55,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:55,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:32:55,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:56,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403825575bc060ce157e98 HTTP/1.1" 200 None 2025-12-17 16:32:56,008 - lava_callback - INFO - Completed processing callback for node 69403825575bc060ce157e98 INFO: 172.21.0.1:42452 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:32:56,602 - lava_callback - INFO - Received callback for node 6940382c575bc060ce157e9d INFO: 172.21.0.1:42464 - "POST /node/6940382c575bc060ce157e9d HTTP/1.0" 202 Accepted 2025-12-17 16:32:56,608 - lava_callback - INFO - Starting async job submit for node 6940382c575bc060ce157e9d 2025-12-17 16:32:56,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:56,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940382c575bc060ce157e9d HTTP/1.1" 200 None 2025-12-17 16:32:56,620 - lava_callback - DEBUG - Processing node 6940382c575bc060ce157e9d: baseline-arm64-baylibre 2025-12-17 16:32:56,621 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940382c575bc060ce157e9d 2025-12-17 16:32:56,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:56,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940382c575bc060ce157e9d HTTP/1.1" 200 None 2025-12-17 16:32:56,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:56,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:32:56,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:56,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940382c575bc060ce157e9d HTTP/1.1" 200 None 2025-12-17 16:32:56,664 - lava_callback - INFO - Completed processing callback for node 6940382c575bc060ce157e9d 2025-12-17 16:32:57,254 - lava_callback - INFO - Received callback for node 6940382f575bc060ce157ea8 INFO: 172.21.0.1:42468 - "POST /node/6940382f575bc060ce157ea8 HTTP/1.0" 202 Accepted 2025-12-17 16:32:57,261 - lava_callback - INFO - Starting async job submit for node 6940382f575bc060ce157ea8 2025-12-17 16:32:57,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:57,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940382f575bc060ce157ea8 HTTP/1.1" 200 None 2025-12-17 16:32:57,270 - lava_callback - DEBUG - Processing node 6940382f575bc060ce157ea8: baseline-arm64-baylibre 2025-12-17 16:32:57,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940382f575bc060ce157ea8 2025-12-17 16:32:57,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:57,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940382f575bc060ce157ea8 HTTP/1.1" 200 None 2025-12-17 16:32:57,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:57,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:32:57,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:57,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940382f575bc060ce157ea8 HTTP/1.1" 200 None 2025-12-17 16:32:57,305 - lava_callback - INFO - Completed processing callback for node 6940382f575bc060ce157ea8 2025-12-17 16:32:57,932 - lava_callback - INFO - Received callback for node 69403831575bc060ce157eaf INFO: 172.21.0.1:42484 - "POST /node/69403831575bc060ce157eaf HTTP/1.0" 202 Accepted 2025-12-17 16:32:57,939 - lava_callback - INFO - Starting async job submit for node 69403831575bc060ce157eaf 2025-12-17 16:32:57,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:57,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403831575bc060ce157eaf HTTP/1.1" 200 None 2025-12-17 16:32:57,948 - lava_callback - DEBUG - Processing node 69403831575bc060ce157eaf: baseline-arm64-baylibre 2025-12-17 16:32:57,948 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403831575bc060ce157eaf 2025-12-17 16:32:57,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:57,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403831575bc060ce157eaf HTTP/1.1" 200 None 2025-12-17 16:32:57,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:57,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:32:57,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:57,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403831575bc060ce157eaf HTTP/1.1" 200 None 2025-12-17 16:32:57,982 - lava_callback - INFO - Completed processing callback for node 69403831575bc060ce157eaf 2025-12-17 16:32:58,613 - lava_callback - INFO - Received callback for node 69403833575bc060ce157eb2 INFO: 172.21.0.1:42492 - "POST /node/69403833575bc060ce157eb2 HTTP/1.0" 202 Accepted 2025-12-17 16:32:58,619 - lava_callback - INFO - Starting async job submit for node 69403833575bc060ce157eb2 2025-12-17 16:32:58,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:58,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403833575bc060ce157eb2 HTTP/1.1" 200 None 2025-12-17 16:32:58,629 - lava_callback - DEBUG - Processing node 69403833575bc060ce157eb2: baseline-arm64-baylibre 2025-12-17 16:32:58,629 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403833575bc060ce157eb2 2025-12-17 16:32:58,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:58,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403833575bc060ce157eb2 HTTP/1.1" 200 None 2025-12-17 16:32:58,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:58,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:32:58,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:58,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403833575bc060ce157eb2 HTTP/1.1" 200 None 2025-12-17 16:32:58,672 - lava_callback - INFO - Completed processing callback for node 69403833575bc060ce157eb2 2025-12-17 16:32:59,276 - lava_callback - INFO - Received callback for node 69403835575bc060ce157eb5 INFO: 172.21.0.1:42508 - "POST /node/69403835575bc060ce157eb5 HTTP/1.0" 202 Accepted 2025-12-17 16:32:59,282 - lava_callback - INFO - Starting async job submit for node 69403835575bc060ce157eb5 2025-12-17 16:32:59,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:59,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403835575bc060ce157eb5 HTTP/1.1" 200 None 2025-12-17 16:32:59,293 - lava_callback - DEBUG - Processing node 69403835575bc060ce157eb5: baseline-arm64-baylibre 2025-12-17 16:32:59,293 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403835575bc060ce157eb5 2025-12-17 16:32:59,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:59,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403835575bc060ce157eb5 HTTP/1.1" 200 None 2025-12-17 16:32:59,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:59,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:32:59,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:32:59,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403835575bc060ce157eb5 HTTP/1.1" 200 None 2025-12-17 16:32:59,329 - lava_callback - INFO - Completed processing callback for node 69403835575bc060ce157eb5 INFO: 172.21.0.1:38346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40366 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:33:15,256 - lava_callback - INFO - Received callback for node 6940383c575bc060ce157ecb INFO: 172.21.0.1:40378 - "POST /node/6940383c575bc060ce157ecb HTTP/1.0" 202 Accepted 2025-12-17 16:33:15,261 - lava_callback - INFO - Starting async job submit for node 6940383c575bc060ce157ecb 2025-12-17 16:33:15,262 - lava_callback - INFO - Received callback for node 69403837575bc060ce157eb8 INFO: 172.21.0.1:40394 - "POST /node/69403837575bc060ce157eb8 HTTP/1.0" 202 Accepted 2025-12-17 16:33:15,267 - lava_callback - INFO - Starting async job submit for node 69403837575bc060ce157eb8 2025-12-17 16:33:15,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:15,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:15,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940383c575bc060ce157ecb HTTP/1.1" 200 None 2025-12-17 16:33:15,278 - lava_callback - DEBUG - Processing node 6940383c575bc060ce157ecb: baseline-x86-kcidebug-intel 2025-12-17 16:33:15,278 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940383c575bc060ce157ecb 2025-12-17 16:33:15,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:15,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403837575bc060ce157eb8 HTTP/1.1" 200 None 2025-12-17 16:33:15,280 - lava_callback - DEBUG - Processing node 69403837575bc060ce157eb8: baseline-arm64-baylibre 2025-12-17 16:33:15,280 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403837575bc060ce157eb8 2025-12-17 16:33:15,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:15,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940383c575bc060ce157ecb HTTP/1.1" 200 None 2025-12-17 16:33:15,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:15,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403837575bc060ce157eb8 HTTP/1.1" 200 None 2025-12-17 16:33:15,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:15,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403217575bc060ce156caa HTTP/1.1" 200 None 2025-12-17 16:33:15,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:15,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:33:15,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:15,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940383c575bc060ce157ecb HTTP/1.1" 200 None 2025-12-17 16:33:15,322 - lava_callback - INFO - Completed processing callback for node 6940383c575bc060ce157ecb 2025-12-17 16:33:15,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403837575bc060ce157eb8 HTTP/1.1" 200 None 2025-12-17 16:33:15,330 - lava_callback - INFO - Completed processing callback for node 69403837575bc060ce157eb8 2025-12-17 16:33:15,956 - lava_callback - INFO - Received callback for node 69403839575bc060ce157ebb INFO: 172.21.0.1:40402 - "POST /node/69403839575bc060ce157ebb HTTP/1.0" 202 Accepted 2025-12-17 16:33:15,962 - lava_callback - INFO - Starting async job submit for node 69403839575bc060ce157ebb 2025-12-17 16:33:15,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:15,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403839575bc060ce157ebb HTTP/1.1" 200 None 2025-12-17 16:33:15,972 - lava_callback - DEBUG - Processing node 69403839575bc060ce157ebb: baseline-arm64-baylibre 2025-12-17 16:33:15,972 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403839575bc060ce157ebb 2025-12-17 16:33:15,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:15,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403839575bc060ce157ebb HTTP/1.1" 200 None 2025-12-17 16:33:15,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:15,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:33:15,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:16,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403839575bc060ce157ebb HTTP/1.1" 200 None 2025-12-17 16:33:16,002 - lava_callback - INFO - Completed processing callback for node 69403839575bc060ce157ebb INFO: 172.21.0.1:40406 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:33:16,623 - lava_callback - INFO - Received callback for node 69403840575bc060ce157ed0 INFO: 172.21.0.1:40422 - "POST /node/69403840575bc060ce157ed0 HTTP/1.0" 202 Accepted 2025-12-17 16:33:16,628 - lava_callback - INFO - Starting async job submit for node 69403840575bc060ce157ed0 2025-12-17 16:33:16,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:16,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403840575bc060ce157ed0 HTTP/1.1" 200 None 2025-12-17 16:33:16,650 - lava_callback - DEBUG - Processing node 69403840575bc060ce157ed0: baseline-arm64-baylibre 2025-12-17 16:33:16,650 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403840575bc060ce157ed0 2025-12-17 16:33:16,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:16,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403840575bc060ce157ed0 HTTP/1.1" 200 None 2025-12-17 16:33:16,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:16,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:33:16,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:16,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403840575bc060ce157ed0 HTTP/1.1" 200 None 2025-12-17 16:33:16,685 - lava_callback - INFO - Completed processing callback for node 69403840575bc060ce157ed0 2025-12-17 16:33:17,304 - lava_callback - INFO - Received callback for node 69403843575bc060ce157ed7 INFO: 172.21.0.1:40428 - "POST /node/69403843575bc060ce157ed7 HTTP/1.0" 202 Accepted 2025-12-17 16:33:17,311 - lava_callback - INFO - Starting async job submit for node 69403843575bc060ce157ed7 2025-12-17 16:33:17,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:17,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403843575bc060ce157ed7 HTTP/1.1" 200 None 2025-12-17 16:33:17,322 - lava_callback - DEBUG - Processing node 69403843575bc060ce157ed7: baseline-arm64-baylibre 2025-12-17 16:33:17,322 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403843575bc060ce157ed7 2025-12-17 16:33:17,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:17,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403843575bc060ce157ed7 HTTP/1.1" 200 None 2025-12-17 16:33:17,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:17,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:33:17,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:17,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403843575bc060ce157ed7 HTTP/1.1" 200 None 2025-12-17 16:33:17,359 - lava_callback - INFO - Completed processing callback for node 69403843575bc060ce157ed7 2025-12-17 16:33:17,971 - lava_callback - INFO - Received callback for node 69403845575bc060ce157eda INFO: 172.21.0.1:40430 - "POST /node/69403845575bc060ce157eda HTTP/1.0" 202 Accepted 2025-12-17 16:33:17,977 - lava_callback - INFO - Starting async job submit for node 69403845575bc060ce157eda 2025-12-17 16:33:17,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:17,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403845575bc060ce157eda HTTP/1.1" 200 None 2025-12-17 16:33:17,987 - lava_callback - DEBUG - Processing node 69403845575bc060ce157eda: baseline-arm64-baylibre 2025-12-17 16:33:17,987 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403845575bc060ce157eda 2025-12-17 16:33:17,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:17,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403845575bc060ce157eda HTTP/1.1" 200 None 2025-12-17 16:33:17,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:18,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:33:18,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:18,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403845575bc060ce157eda HTTP/1.1" 200 None 2025-12-17 16:33:18,020 - lava_callback - INFO - Completed processing callback for node 69403845575bc060ce157eda 2025-12-17 16:33:21,538 - lava_callback - INFO - Received callback for node 694034bc575bc060ce1573ad INFO: 172.21.0.1:37354 - "POST /node/694034bc575bc060ce1573ad HTTP/1.0" 202 Accepted 2025-12-17 16:33:21,547 - lava_callback - INFO - Starting async job submit for node 694034bc575bc060ce1573ad 2025-12-17 16:33:21,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:21,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034bc575bc060ce1573ad HTTP/1.1" 200 None 2025-12-17 16:33:21,647 - lava_callback - DEBUG - Processing node 694034bc575bc060ce1573ad: ltp-smoketest 2025-12-17 16:33:22,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:33:22,276 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:33:22,277 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoketest-694034bc575bc060ce1573ad/log.txt.gz 2025-12-17 16:33:22,277 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:33:22,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:33:22,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:33:22,457 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoketest-694034bc575bc060ce1573ad/lava_callback.json.gz 2025-12-17 16:33:22,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:22,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694034bc575bc060ce1573ad HTTP/1.1" 200 None 2025-12-17 16:33:22,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:22,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0aafad9ee307f5b1dcd HTTP/1.1" 200 None 2025-12-17 16:33:22,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:22,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694034bc575bc060ce1573ad HTTP/1.1" 200 None 2025-12-17 16:33:22,654 - lava_callback - INFO - Completed processing callback for node 694034bc575bc060ce1573ad INFO: 172.21.0.1:37360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58788 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:33:39,883 - lava_callback - INFO - Received callback for node 6940c4f71e3c24e95681d5f2 INFO: 172.21.0.1:47876 - "POST /node/6940c4f71e3c24e95681d5f2 HTTP/1.0" 202 Accepted 2025-12-17 16:33:39,890 - lava_callback - INFO - Starting async job submit for node 6940c4f71e3c24e95681d5f2 2025-12-17 16:33:39,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:39,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c4f71e3c24e95681d5f2 HTTP/1.1" 200 None 2025-12-17 16:33:39,944 - lava_callback - DEBUG - Processing node 6940c4f71e3c24e95681d5f2: baseline-x86 2025-12-17 16:33:40,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:33:41,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:33:41,037 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-6940c4f71e3c24e95681d5f2/log.txt.gz 2025-12-17 16:33:41,037 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:33:41,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:33:41,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:33:41,426 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-6940c4f71e3c24e95681d5f2/lava_callback.json.gz DEBUG: Setup failed for 6940c4f71e3c24e95681d5f2. Transitting job node result: incomplete -> fail 2025-12-17 16:33:41,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:41,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c4f71e3c24e95681d5f2 HTTP/1.1" 200 None 2025-12-17 16:33:41,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:41,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21eb07de2c0cbc264f2b HTTP/1.1" 200 None 2025-12-17 16:33:41,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:41,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940c4f71e3c24e95681d5f2 HTTP/1.1" 200 None 2025-12-17 16:33:41,583 - lava_callback - INFO - Completed processing callback for node 6940c4f71e3c24e95681d5f2 2025-12-17 16:33:42,983 - lava_callback - INFO - Received callback for node 69404105575bc060ce158a9e INFO: 172.21.0.1:47888 - "POST /node/69404105575bc060ce158a9e HTTP/1.0" 202 Accepted 2025-12-17 16:33:42,991 - lava_callback - INFO - Starting async job submit for node 69404105575bc060ce158a9e 2025-12-17 16:33:43,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:43,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404105575bc060ce158a9e HTTP/1.1" 200 None 2025-12-17 16:33:43,033 - lava_callback - DEBUG - Processing node 69404105575bc060ce158a9e: blktests-ddp-x86 2025-12-17 16:33:43,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:33:44,113 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:33:44,114 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/blktests-ddp-x86-69404105575bc060ce158a9e/log.txt.gz 2025-12-17 16:33:44,114 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:33:44,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:33:44,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:33:44,330 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/blktests-ddp-x86-69404105575bc060ce158a9e/lava_callback.json.gz 2025-12-17 16:33:44,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:44,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404105575bc060ce158a9e HTTP/1.1" 200 None 2025-12-17 16:33:44,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:44,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693b5e1c4b47c3f872674fdf HTTP/1.1" 200 None 2025-12-17 16:33:44,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:44,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69404105575bc060ce158a9e HTTP/1.1" 200 None 2025-12-17 16:33:44,392 - lava_callback - INFO - Completed processing callback for node 69404105575bc060ce158a9e INFO: 172.21.0.1:47904 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:33:49,773 - lava_callback - INFO - Received callback for node 6940385a575bc060ce157ee5 INFO: 172.21.0.1:57698 - "POST /node/6940385a575bc060ce157ee5 HTTP/1.0" 202 Accepted 2025-12-17 16:33:49,780 - lava_callback - INFO - Starting async job submit for node 6940385a575bc060ce157ee5 2025-12-17 16:33:49,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:49,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940385a575bc060ce157ee5 HTTP/1.1" 200 None 2025-12-17 16:33:49,793 - lava_callback - DEBUG - Processing node 6940385a575bc060ce157ee5: baseline-arm64-clabbe 2025-12-17 16:33:49,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940385a575bc060ce157ee5 2025-12-17 16:33:49,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:49,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940385a575bc060ce157ee5 HTTP/1.1" 200 None 2025-12-17 16:33:49,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:49,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:33:49,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:49,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940385a575bc060ce157ee5 HTTP/1.1" 200 None 2025-12-17 16:33:49,845 - lava_callback - INFO - Completed processing callback for node 6940385a575bc060ce157ee5 2025-12-17 16:33:50,435 - lava_callback - INFO - Received callback for node 6940385d575bc060ce157eea INFO: 172.21.0.1:57714 - "POST /node/6940385d575bc060ce157eea HTTP/1.0" 202 Accepted 2025-12-17 16:33:50,443 - lava_callback - INFO - Starting async job submit for node 6940385d575bc060ce157eea 2025-12-17 16:33:50,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:50,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940385d575bc060ce157eea HTTP/1.1" 200 None 2025-12-17 16:33:50,455 - lava_callback - DEBUG - Processing node 6940385d575bc060ce157eea: baseline-arm64-clabbe 2025-12-17 16:33:50,455 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940385d575bc060ce157eea 2025-12-17 16:33:50,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:50,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940385d575bc060ce157eea HTTP/1.1" 200 None 2025-12-17 16:33:50,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:50,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:33:50,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:50,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940385d575bc060ce157eea HTTP/1.1" 200 None 2025-12-17 16:33:50,493 - lava_callback - INFO - Completed processing callback for node 6940385d575bc060ce157eea 2025-12-17 16:33:51,092 - lava_callback - INFO - Received callback for node 6940385f575bc060ce157eed INFO: 172.21.0.1:57730 - "POST /node/6940385f575bc060ce157eed HTTP/1.0" 202 Accepted 2025-12-17 16:33:51,099 - lava_callback - INFO - Starting async job submit for node 6940385f575bc060ce157eed 2025-12-17 16:33:51,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:51,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940385f575bc060ce157eed HTTP/1.1" 200 None 2025-12-17 16:33:51,113 - lava_callback - DEBUG - Processing node 6940385f575bc060ce157eed: baseline-arm64-clabbe 2025-12-17 16:33:51,113 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940385f575bc060ce157eed 2025-12-17 16:33:51,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:51,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940385f575bc060ce157eed HTTP/1.1" 200 None 2025-12-17 16:33:51,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:51,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:33:51,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:33:51,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940385f575bc060ce157eed HTTP/1.1" 200 None 2025-12-17 16:33:51,162 - lava_callback - INFO - Completed processing callback for node 6940385f575bc060ce157eed INFO: 172.21.0.1:57734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60618 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:34:08,380 - lava_callback - INFO - Received callback for node 6940387d575bc060ce157f40 INFO: 172.21.0.1:60630 - "POST /node/6940387d575bc060ce157f40 HTTP/1.0" 202 Accepted 2025-12-17 16:34:08,390 - lava_callback - INFO - Starting async job submit for node 6940387d575bc060ce157f40 2025-12-17 16:34:08,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:08,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387d575bc060ce157f40 HTTP/1.1" 200 None 2025-12-17 16:34:08,405 - lava_callback - DEBUG - Processing node 6940387d575bc060ce157f40: baseline-arm 2025-12-17 16:34:08,405 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940387d575bc060ce157f40 2025-12-17 16:34:08,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:08,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387d575bc060ce157f40 HTTP/1.1" 200 None 2025-12-17 16:34:08,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:08,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:08,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:08,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940387d575bc060ce157f40 HTTP/1.1" 200 None 2025-12-17 16:34:08,460 - lava_callback - INFO - Completed processing callback for node 6940387d575bc060ce157f40 2025-12-17 16:34:09,168 - lava_callback - INFO - Received callback for node 6940387c575bc060ce157f3a INFO: 172.21.0.1:60640 - "POST /node/6940387c575bc060ce157f3a HTTP/1.0" 202 Accepted 2025-12-17 16:34:09,174 - lava_callback - INFO - Starting async job submit for node 6940387c575bc060ce157f3a 2025-12-17 16:34:09,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:09,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387c575bc060ce157f3a HTTP/1.1" 200 None 2025-12-17 16:34:09,183 - lava_callback - DEBUG - Processing node 6940387c575bc060ce157f3a: baseline-arm 2025-12-17 16:34:09,183 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940387c575bc060ce157f3a 2025-12-17 16:34:09,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:09,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387c575bc060ce157f3a HTTP/1.1" 200 None 2025-12-17 16:34:09,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:09,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:09,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:09,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940387c575bc060ce157f3a HTTP/1.1" 200 None 2025-12-17 16:34:09,227 - lava_callback - INFO - Completed processing callback for node 6940387c575bc060ce157f3a 2025-12-17 16:34:09,768 - lava_callback - INFO - Received callback for node 69403870575bc060ce157efe INFO: 172.21.0.1:51512 - "POST /node/69403870575bc060ce157efe HTTP/1.0" 202 Accepted 2025-12-17 16:34:09,778 - lava_callback - INFO - Starting async job submit for node 69403870575bc060ce157efe 2025-12-17 16:34:09,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:09,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403870575bc060ce157efe HTTP/1.1" 200 None 2025-12-17 16:34:09,791 - lava_callback - DEBUG - Processing node 69403870575bc060ce157efe: baseline-arm64-clabbe 2025-12-17 16:34:09,791 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403870575bc060ce157efe 2025-12-17 16:34:09,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:09,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403870575bc060ce157efe HTTP/1.1" 200 None 2025-12-17 16:34:09,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:09,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:34:09,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:09,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403870575bc060ce157efe HTTP/1.1" 200 None 2025-12-17 16:34:09,842 - lava_callback - INFO - Completed processing callback for node 69403870575bc060ce157efe 2025-12-17 16:34:09,973 - lava_callback - INFO - Received callback for node 6940387c575bc060ce157f3d INFO: 172.21.0.1:51526 - "POST /node/6940387c575bc060ce157f3d HTTP/1.0" 202 Accepted 2025-12-17 16:34:09,982 - lava_callback - INFO - Starting async job submit for node 6940387c575bc060ce157f3d 2025-12-17 16:34:09,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:09,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387c575bc060ce157f3d HTTP/1.1" 200 None 2025-12-17 16:34:09,992 - lava_callback - DEBUG - Processing node 6940387c575bc060ce157f3d: baseline-arm 2025-12-17 16:34:09,992 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940387c575bc060ce157f3d 2025-12-17 16:34:09,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:10,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387c575bc060ce157f3d HTTP/1.1" 200 None 2025-12-17 16:34:10,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:10,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:10,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:10,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940387c575bc060ce157f3d HTTP/1.1" 200 None 2025-12-17 16:34:10,036 - lava_callback - INFO - Completed processing callback for node 6940387c575bc060ce157f3d 2025-12-17 16:34:10,441 - lava_callback - INFO - Received callback for node 69403874575bc060ce157f2b INFO: 172.21.0.1:51540 - "POST /node/69403874575bc060ce157f2b HTTP/1.0" 202 Accepted 2025-12-17 16:34:10,447 - lava_callback - INFO - Starting async job submit for node 69403874575bc060ce157f2b 2025-12-17 16:34:10,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:10,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403874575bc060ce157f2b HTTP/1.1" 200 None 2025-12-17 16:34:10,458 - lava_callback - DEBUG - Processing node 69403874575bc060ce157f2b: baseline-arm64-clabbe 2025-12-17 16:34:10,458 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403874575bc060ce157f2b 2025-12-17 16:34:10,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:10,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403874575bc060ce157f2b HTTP/1.1" 200 None 2025-12-17 16:34:10,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:10,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:34:10,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:10,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403874575bc060ce157f2b HTTP/1.1" 200 None 2025-12-17 16:34:10,492 - lava_callback - INFO - Completed processing callback for node 69403874575bc060ce157f2b 2025-12-17 16:34:10,688 - lava_callback - INFO - Received callback for node 6940387d575bc060ce157f43 INFO: 172.21.0.1:51548 - "POST /node/6940387d575bc060ce157f43 HTTP/1.0" 202 Accepted 2025-12-17 16:34:10,696 - lava_callback - INFO - Starting async job submit for node 6940387d575bc060ce157f43 2025-12-17 16:34:10,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:10,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387d575bc060ce157f43 HTTP/1.1" 200 None 2025-12-17 16:34:10,705 - lava_callback - DEBUG - Processing node 6940387d575bc060ce157f43: baseline-arm 2025-12-17 16:34:10,705 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940387d575bc060ce157f43 2025-12-17 16:34:10,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:10,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387d575bc060ce157f43 HTTP/1.1" 200 None 2025-12-17 16:34:10,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:10,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:10,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:10,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940387d575bc060ce157f43 HTTP/1.1" 200 None 2025-12-17 16:34:10,755 - lava_callback - INFO - Completed processing callback for node 6940387d575bc060ce157f43 INFO: 172.21.0.1:51564 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:34:11,103 - lava_callback - INFO - Received callback for node 69403876575bc060ce157f2e INFO: 172.21.0.1:51568 - "POST /node/69403876575bc060ce157f2e HTTP/1.0" 202 Accepted 2025-12-17 16:34:11,109 - lava_callback - INFO - Starting async job submit for node 69403876575bc060ce157f2e 2025-12-17 16:34:11,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:11,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403876575bc060ce157f2e HTTP/1.1" 200 None 2025-12-17 16:34:11,119 - lava_callback - DEBUG - Processing node 69403876575bc060ce157f2e: baseline-arm64-clabbe 2025-12-17 16:34:11,119 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403876575bc060ce157f2e 2025-12-17 16:34:11,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:11,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403876575bc060ce157f2e HTTP/1.1" 200 None 2025-12-17 16:34:11,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:11,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:34:11,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:11,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403876575bc060ce157f2e HTTP/1.1" 200 None 2025-12-17 16:34:11,156 - lava_callback - INFO - Completed processing callback for node 69403876575bc060ce157f2e 2025-12-17 16:34:11,509 - lava_callback - INFO - Received callback for node 6940387e575bc060ce157f46 INFO: 172.21.0.1:51582 - "POST /node/6940387e575bc060ce157f46 HTTP/1.0" 202 Accepted 2025-12-17 16:34:11,516 - lava_callback - INFO - Starting async job submit for node 6940387e575bc060ce157f46 2025-12-17 16:34:11,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:11,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387e575bc060ce157f46 HTTP/1.1" 200 None 2025-12-17 16:34:11,531 - lava_callback - DEBUG - Processing node 6940387e575bc060ce157f46: ltp-smoketest 2025-12-17 16:34:11,531 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940387e575bc060ce157f46 2025-12-17 16:34:11,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:11,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387e575bc060ce157f46 HTTP/1.1" 200 None 2025-12-17 16:34:11,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:11,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:11,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:11,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940387e575bc060ce157f46 HTTP/1.1" 200 None 2025-12-17 16:34:11,572 - lava_callback - INFO - Completed processing callback for node 6940387e575bc060ce157f46 2025-12-17 16:34:11,744 - lava_callback - INFO - Received callback for node 69403878575bc060ce157f31 INFO: 172.21.0.1:51584 - "POST /node/69403878575bc060ce157f31 HTTP/1.0" 202 Accepted 2025-12-17 16:34:11,753 - lava_callback - INFO - Starting async job submit for node 69403878575bc060ce157f31 2025-12-17 16:34:11,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:11,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403878575bc060ce157f31 HTTP/1.1" 200 None 2025-12-17 16:34:11,769 - lava_callback - DEBUG - Processing node 69403878575bc060ce157f31: baseline-arm64-clabbe 2025-12-17 16:34:11,769 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403878575bc060ce157f31 2025-12-17 16:34:11,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:11,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403878575bc060ce157f31 HTTP/1.1" 200 None 2025-12-17 16:34:11,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:11,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:34:11,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:11,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403878575bc060ce157f31 HTTP/1.1" 200 None 2025-12-17 16:34:11,832 - lava_callback - INFO - Completed processing callback for node 69403878575bc060ce157f31 2025-12-17 16:34:12,250 - lava_callback - INFO - Received callback for node 6940387e575bc060ce157f49 INFO: 172.21.0.1:51596 - "POST /node/6940387e575bc060ce157f49 HTTP/1.0" 202 Accepted 2025-12-17 16:34:12,257 - lava_callback - INFO - Starting async job submit for node 6940387e575bc060ce157f49 2025-12-17 16:34:12,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:12,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387e575bc060ce157f49 HTTP/1.1" 200 None 2025-12-17 16:34:12,267 - lava_callback - DEBUG - Processing node 6940387e575bc060ce157f49: baseline-arm 2025-12-17 16:34:12,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940387e575bc060ce157f49 2025-12-17 16:34:12,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:12,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387e575bc060ce157f49 HTTP/1.1" 200 None 2025-12-17 16:34:12,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:12,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:12,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:12,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940387e575bc060ce157f49 HTTP/1.1" 200 None 2025-12-17 16:34:12,320 - lava_callback - INFO - Completed processing callback for node 6940387e575bc060ce157f49 2025-12-17 16:34:12,404 - lava_callback - INFO - Received callback for node 6940387a575bc060ce157f34 INFO: 172.21.0.1:51610 - "POST /node/6940387a575bc060ce157f34 HTTP/1.0" 202 Accepted 2025-12-17 16:34:12,410 - lava_callback - INFO - Starting async job submit for node 6940387a575bc060ce157f34 2025-12-17 16:34:12,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:12,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387a575bc060ce157f34 HTTP/1.1" 200 None 2025-12-17 16:34:12,426 - lava_callback - DEBUG - Processing node 6940387a575bc060ce157f34: baseline-arm64-clabbe 2025-12-17 16:34:12,426 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940387a575bc060ce157f34 2025-12-17 16:34:12,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:12,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387a575bc060ce157f34 HTTP/1.1" 200 None 2025-12-17 16:34:12,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:12,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:34:12,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:12,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940387a575bc060ce157f34 HTTP/1.1" 200 None 2025-12-17 16:34:12,462 - lava_callback - INFO - Completed processing callback for node 6940387a575bc060ce157f34 2025-12-17 16:34:12,999 - lava_callback - INFO - Received callback for node 6940387f575bc060ce157f4c INFO: 172.21.0.1:51614 - "POST /node/6940387f575bc060ce157f4c HTTP/1.0" 202 Accepted 2025-12-17 16:34:13,007 - lava_callback - INFO - Starting async job submit for node 6940387f575bc060ce157f4c 2025-12-17 16:34:13,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:13,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387f575bc060ce157f4c HTTP/1.1" 200 None 2025-12-17 16:34:13,018 - lava_callback - DEBUG - Processing node 6940387f575bc060ce157f4c: ltp-smoketest 2025-12-17 16:34:13,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940387f575bc060ce157f4c 2025-12-17 16:34:13,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:13,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387f575bc060ce157f4c HTTP/1.1" 200 None 2025-12-17 16:34:13,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:13,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:13,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:13,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940387f575bc060ce157f4c HTTP/1.1" 200 None 2025-12-17 16:34:13,063 - lava_callback - INFO - Completed processing callback for node 6940387f575bc060ce157f4c 2025-12-17 16:34:13,730 - lava_callback - INFO - Received callback for node 6940387f575bc060ce157f4f INFO: 172.21.0.1:51626 - "POST /node/6940387f575bc060ce157f4f HTTP/1.0" 202 Accepted 2025-12-17 16:34:13,739 - lava_callback - INFO - Starting async job submit for node 6940387f575bc060ce157f4f 2025-12-17 16:34:13,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:13,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387f575bc060ce157f4f HTTP/1.1" 200 None 2025-12-17 16:34:13,748 - lava_callback - DEBUG - Processing node 6940387f575bc060ce157f4f: ltp-smoketest 2025-12-17 16:34:13,748 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940387f575bc060ce157f4f 2025-12-17 16:34:13,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:13,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387f575bc060ce157f4f HTTP/1.1" 200 None 2025-12-17 16:34:13,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:13,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:13,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:13,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940387f575bc060ce157f4f HTTP/1.1" 200 None 2025-12-17 16:34:13,791 - lava_callback - INFO - Completed processing callback for node 6940387f575bc060ce157f4f 2025-12-17 16:34:14,521 - lava_callback - INFO - Received callback for node 6940387f575bc060ce157f52 INFO: 172.21.0.1:51634 - "POST /node/6940387f575bc060ce157f52 HTTP/1.0" 202 Accepted 2025-12-17 16:34:14,529 - lava_callback - INFO - Starting async job submit for node 6940387f575bc060ce157f52 2025-12-17 16:34:14,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:14,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387f575bc060ce157f52 HTTP/1.1" 200 None 2025-12-17 16:34:14,574 - lava_callback - DEBUG - Processing node 6940387f575bc060ce157f52: ltp-smoketest 2025-12-17 16:34:14,574 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940387f575bc060ce157f52 2025-12-17 16:34:14,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:14,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387f575bc060ce157f52 HTTP/1.1" 200 None 2025-12-17 16:34:14,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:14,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:14,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:14,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940387f575bc060ce157f52 HTTP/1.1" 200 None 2025-12-17 16:34:14,624 - lava_callback - INFO - Completed processing callback for node 6940387f575bc060ce157f52 INFO: 172.21.0.1:51638 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:34:15,319 - lava_callback - INFO - Received callback for node 69403880575bc060ce157f55 INFO: 172.21.0.1:51650 - "POST /node/69403880575bc060ce157f55 HTTP/1.0" 202 Accepted 2025-12-17 16:34:15,326 - lava_callback - INFO - Starting async job submit for node 69403880575bc060ce157f55 2025-12-17 16:34:15,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:15,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403880575bc060ce157f55 HTTP/1.1" 200 None 2025-12-17 16:34:15,340 - lava_callback - DEBUG - Processing node 69403880575bc060ce157f55: ltp-smoketest 2025-12-17 16:34:15,340 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403880575bc060ce157f55 2025-12-17 16:34:15,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:15,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403880575bc060ce157f55 HTTP/1.1" 200 None 2025-12-17 16:34:15,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:15,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:15,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:15,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403880575bc060ce157f55 HTTP/1.1" 200 None 2025-12-17 16:34:15,390 - lava_callback - INFO - Completed processing callback for node 69403880575bc060ce157f55 2025-12-17 16:34:15,679 - lava_callback - INFO - Received callback for node 69403886575bc060ce157f75 INFO: 172.21.0.1:51664 - "POST /node/69403886575bc060ce157f75 HTTP/1.0" 202 Accepted 2025-12-17 16:34:15,687 - lava_callback - INFO - Starting async job submit for node 69403886575bc060ce157f75 2025-12-17 16:34:15,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:15,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403886575bc060ce157f75 HTTP/1.1" 200 None 2025-12-17 16:34:15,697 - lava_callback - DEBUG - Processing node 69403886575bc060ce157f75: baseline-x86 2025-12-17 16:34:15,697 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403886575bc060ce157f75 2025-12-17 16:34:15,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:15,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403886575bc060ce157f75 HTTP/1.1" 200 None 2025-12-17 16:34:15,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:15,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d5fad9ee307f5b1e66 HTTP/1.1" 200 None 2025-12-17 16:34:15,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:15,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403886575bc060ce157f75 HTTP/1.1" 200 None 2025-12-17 16:34:15,759 - lava_callback - INFO - Completed processing callback for node 69403886575bc060ce157f75 2025-12-17 16:34:16,106 - lava_callback - INFO - Received callback for node 69403880575bc060ce157f58 INFO: 172.21.0.1:51674 - "POST /node/69403880575bc060ce157f58 HTTP/1.0" 202 Accepted 2025-12-17 16:34:16,112 - lava_callback - INFO - Starting async job submit for node 69403880575bc060ce157f58 2025-12-17 16:34:16,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:16,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403880575bc060ce157f58 HTTP/1.1" 200 None 2025-12-17 16:34:16,124 - lava_callback - DEBUG - Processing node 69403880575bc060ce157f58: ltp-smoketest 2025-12-17 16:34:16,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403880575bc060ce157f58 2025-12-17 16:34:16,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:16,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403880575bc060ce157f58 HTTP/1.1" 200 None 2025-12-17 16:34:16,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:16,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:16,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:16,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403880575bc060ce157f58 HTTP/1.1" 200 None 2025-12-17 16:34:16,169 - lava_callback - INFO - Completed processing callback for node 69403880575bc060ce157f58 INFO: 172.21.0.1:51686 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:34:16,893 - lava_callback - INFO - Received callback for node 69403881575bc060ce157f5f INFO: 172.21.0.1:51688 - "POST /node/69403881575bc060ce157f5f HTTP/1.0" 202 Accepted 2025-12-17 16:34:16,900 - lava_callback - INFO - Starting async job submit for node 69403881575bc060ce157f5f 2025-12-17 16:34:16,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:16,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403881575bc060ce157f5f HTTP/1.1" 200 None 2025-12-17 16:34:16,909 - lava_callback - DEBUG - Processing node 69403881575bc060ce157f5f: baseline-arm 2025-12-17 16:34:16,909 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403881575bc060ce157f5f 2025-12-17 16:34:16,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:16,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403881575bc060ce157f5f HTTP/1.1" 200 None 2025-12-17 16:34:16,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:16,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:16,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:16,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403881575bc060ce157f5f HTTP/1.1" 200 None 2025-12-17 16:34:16,949 - lava_callback - INFO - Completed processing callback for node 69403881575bc060ce157f5f 2025-12-17 16:34:17,682 - lava_callback - INFO - Received callback for node 69403881575bc060ce157f64 INFO: 172.21.0.1:51694 - "POST /node/69403881575bc060ce157f64 HTTP/1.0" 202 Accepted 2025-12-17 16:34:17,690 - lava_callback - INFO - Starting async job submit for node 69403881575bc060ce157f64 2025-12-17 16:34:17,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:17,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403881575bc060ce157f64 HTTP/1.1" 200 None 2025-12-17 16:34:17,702 - lava_callback - DEBUG - Processing node 69403881575bc060ce157f64: baseline-arm 2025-12-17 16:34:17,703 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403881575bc060ce157f64 2025-12-17 16:34:17,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:17,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403881575bc060ce157f64 HTTP/1.1" 200 None 2025-12-17 16:34:17,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:17,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:17,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:17,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403881575bc060ce157f64 HTTP/1.1" 200 None 2025-12-17 16:34:17,748 - lava_callback - INFO - Completed processing callback for node 69403881575bc060ce157f64 2025-12-17 16:34:18,512 - lava_callback - INFO - Received callback for node 69403882575bc060ce157f67 INFO: 172.21.0.1:51702 - "POST /node/69403882575bc060ce157f67 HTTP/1.0" 202 Accepted 2025-12-17 16:34:18,520 - lava_callback - INFO - Starting async job submit for node 69403882575bc060ce157f67 2025-12-17 16:34:18,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:18,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403882575bc060ce157f67 HTTP/1.1" 200 None 2025-12-17 16:34:18,530 - lava_callback - DEBUG - Processing node 69403882575bc060ce157f67: baseline-arm 2025-12-17 16:34:18,530 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403882575bc060ce157f67 2025-12-17 16:34:18,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:18,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403882575bc060ce157f67 HTTP/1.1" 200 None 2025-12-17 16:34:18,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:18,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:18,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:18,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403882575bc060ce157f67 HTTP/1.1" 200 None 2025-12-17 16:34:18,581 - lava_callback - INFO - Completed processing callback for node 69403882575bc060ce157f67 2025-12-17 16:34:19,266 - lava_callback - INFO - Received callback for node 69403882575bc060ce157f6c INFO: 172.21.0.1:51712 - "POST /node/69403882575bc060ce157f6c HTTP/1.0" 202 Accepted 2025-12-17 16:34:19,274 - lava_callback - INFO - Starting async job submit for node 69403882575bc060ce157f6c 2025-12-17 16:34:19,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:19,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403882575bc060ce157f6c HTTP/1.1" 200 None 2025-12-17 16:34:19,285 - lava_callback - DEBUG - Processing node 69403882575bc060ce157f6c: ltp-smoketest 2025-12-17 16:34:19,285 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403882575bc060ce157f6c 2025-12-17 16:34:19,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:19,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403882575bc060ce157f6c HTTP/1.1" 200 None 2025-12-17 16:34:19,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:19,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:19,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:19,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403882575bc060ce157f6c HTTP/1.1" 200 None 2025-12-17 16:34:19,328 - lava_callback - INFO - Completed processing callback for node 69403882575bc060ce157f6c 2025-12-17 16:34:19,997 - lava_callback - INFO - Received callback for node 69403883575bc060ce157f6f INFO: 172.21.0.1:49234 - "POST /node/69403883575bc060ce157f6f HTTP/1.0" 202 Accepted 2025-12-17 16:34:20,018 - lava_callback - INFO - Starting async job submit for node 69403883575bc060ce157f6f 2025-12-17 16:34:20,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:20,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403883575bc060ce157f6f HTTP/1.1" 200 None 2025-12-17 16:34:20,042 - lava_callback - DEBUG - Processing node 69403883575bc060ce157f6f: ltp-smoketest 2025-12-17 16:34:20,042 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403883575bc060ce157f6f 2025-12-17 16:34:20,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:20,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403883575bc060ce157f6f HTTP/1.1" 200 None 2025-12-17 16:34:20,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:20,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:20,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:20,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403883575bc060ce157f6f HTTP/1.1" 200 None 2025-12-17 16:34:20,136 - lava_callback - INFO - Completed processing callback for node 69403883575bc060ce157f6f 2025-12-17 16:34:20,822 - lava_callback - INFO - Received callback for node 69403883575bc060ce157f72 INFO: 172.21.0.1:49242 - "POST /node/69403883575bc060ce157f72 HTTP/1.0" 202 Accepted 2025-12-17 16:34:20,829 - lava_callback - INFO - Starting async job submit for node 69403883575bc060ce157f72 2025-12-17 16:34:20,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:20,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403883575bc060ce157f72 HTTP/1.1" 200 None 2025-12-17 16:34:20,908 - lava_callback - DEBUG - Processing node 69403883575bc060ce157f72: ltp-smoketest 2025-12-17 16:34:20,908 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403883575bc060ce157f72 2025-12-17 16:34:20,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:20,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403883575bc060ce157f72 HTTP/1.1" 200 None 2025-12-17 16:34:20,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:20,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:34:20,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:20,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403883575bc060ce157f72 HTTP/1.1" 200 None 2025-12-17 16:34:20,967 - lava_callback - INFO - Completed processing callback for node 69403883575bc060ce157f72 INFO: 172.21.0.1:49258 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:34:29,800 - lava_callback - INFO - Received callback for node 69403890575bc060ce157f80 INFO: 172.21.0.1:56514 - "POST /node/69403890575bc060ce157f80 HTTP/1.0" 202 Accepted 2025-12-17 16:34:29,813 - lava_callback - INFO - Starting async job submit for node 69403890575bc060ce157f80 2025-12-17 16:34:29,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:29,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403890575bc060ce157f80 HTTP/1.1" 200 None 2025-12-17 16:34:29,828 - lava_callback - DEBUG - Processing node 69403890575bc060ce157f80: baseline-arm64-clabbe 2025-12-17 16:34:29,828 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403890575bc060ce157f80 2025-12-17 16:34:29,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:29,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403890575bc060ce157f80 HTTP/1.1" 200 None 2025-12-17 16:34:29,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:29,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:34:29,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:29,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403890575bc060ce157f80 HTTP/1.1" 200 None 2025-12-17 16:34:29,888 - lava_callback - INFO - Completed processing callback for node 69403890575bc060ce157f80 2025-12-17 16:34:30,486 - lava_callback - INFO - Received callback for node 69403893575bc060ce157f88 INFO: 172.21.0.1:56522 - "POST /node/69403893575bc060ce157f88 HTTP/1.0" 202 Accepted 2025-12-17 16:34:30,496 - lava_callback - INFO - Starting async job submit for node 69403893575bc060ce157f88 2025-12-17 16:34:30,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:30,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403893575bc060ce157f88 HTTP/1.1" 200 None 2025-12-17 16:34:30,510 - lava_callback - DEBUG - Processing node 69403893575bc060ce157f88: baseline-arm64-clabbe 2025-12-17 16:34:30,510 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403893575bc060ce157f88 2025-12-17 16:34:30,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:30,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403893575bc060ce157f88 HTTP/1.1" 200 None 2025-12-17 16:34:30,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:30,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:34:30,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:30,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403893575bc060ce157f88 HTTP/1.1" 200 None 2025-12-17 16:34:30,565 - lava_callback - INFO - Completed processing callback for node 69403893575bc060ce157f88 2025-12-17 16:34:35,309 - lava_callback - INFO - Received callback for node 69403895575bc060ce157f8b INFO: 172.21.0.1:56536 - "POST /node/69403895575bc060ce157f8b HTTP/1.0" 202 Accepted 2025-12-17 16:34:35,319 - lava_callback - INFO - Starting async job submit for node 69403895575bc060ce157f8b 2025-12-17 16:34:35,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:35,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403895575bc060ce157f8b HTTP/1.1" 200 None 2025-12-17 16:34:35,333 - lava_callback - DEBUG - Processing node 69403895575bc060ce157f8b: baseline-arm64-baylibre 2025-12-17 16:34:35,333 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403895575bc060ce157f8b 2025-12-17 16:34:35,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:35,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403895575bc060ce157f8b HTTP/1.1" 200 None 2025-12-17 16:34:35,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:35,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:34:35,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:35,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403895575bc060ce157f8b HTTP/1.1" 200 None 2025-12-17 16:34:35,389 - lava_callback - INFO - Completed processing callback for node 69403895575bc060ce157f8b 2025-12-17 16:34:35,414 - lava_callback - INFO - Received callback for node 6940388d575bc060ce157f78 INFO: 172.21.0.1:56542 - "POST /node/6940388d575bc060ce157f78 HTTP/1.0" 202 Accepted 2025-12-17 16:34:35,423 - lava_callback - INFO - Starting async job submit for node 6940388d575bc060ce157f78 2025-12-17 16:34:35,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:35,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940388d575bc060ce157f78 HTTP/1.1" 200 None 2025-12-17 16:34:35,441 - lava_callback - DEBUG - Processing node 6940388d575bc060ce157f78: baseline-x86 2025-12-17 16:34:35,441 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940388d575bc060ce157f78 2025-12-17 16:34:35,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:35,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940388d575bc060ce157f78 HTTP/1.1" 200 None 2025-12-17 16:34:35,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:35,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d5fad9ee307f5b1e66 HTTP/1.1" 200 None 2025-12-17 16:34:35,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:35,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940388d575bc060ce157f78 HTTP/1.1" 200 None 2025-12-17 16:34:35,520 - lava_callback - INFO - Completed processing callback for node 6940388d575bc060ce157f78 2025-12-17 16:34:36,009 - lava_callback - INFO - Received callback for node 69403898575bc060ce157f8e INFO: 172.21.0.1:56558 - "POST /node/69403898575bc060ce157f8e HTTP/1.0" 202 Accepted 2025-12-17 16:34:36,019 - lava_callback - INFO - Starting async job submit for node 69403898575bc060ce157f8e 2025-12-17 16:34:36,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:36,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403898575bc060ce157f8e HTTP/1.1" 200 None 2025-12-17 16:34:36,036 - lava_callback - DEBUG - Processing node 69403898575bc060ce157f8e: baseline-arm64-baylibre 2025-12-17 16:34:36,036 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403898575bc060ce157f8e 2025-12-17 16:34:36,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:36,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403898575bc060ce157f8e HTTP/1.1" 200 None 2025-12-17 16:34:36,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:36,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:34:36,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:36,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403898575bc060ce157f8e HTTP/1.1" 200 None 2025-12-17 16:34:36,092 - lava_callback - INFO - Completed processing callback for node 69403898575bc060ce157f8e 2025-12-17 16:34:36,118 - lava_callback - INFO - Received callback for node 6940388f575bc060ce157f7b INFO: 172.21.0.1:56574 - "POST /node/6940388f575bc060ce157f7b HTTP/1.0" 202 Accepted 2025-12-17 16:34:36,133 - lava_callback - INFO - Starting async job submit for node 6940388f575bc060ce157f7b 2025-12-17 16:34:36,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:36,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940388f575bc060ce157f7b HTTP/1.1" 200 None 2025-12-17 16:34:36,165 - lava_callback - DEBUG - Processing node 6940388f575bc060ce157f7b: baseline-x86 2025-12-17 16:34:36,165 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940388f575bc060ce157f7b 2025-12-17 16:34:36,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:36,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940388f575bc060ce157f7b HTTP/1.1" 200 None 2025-12-17 16:34:36,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:36,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d5fad9ee307f5b1e66 HTTP/1.1" 200 None 2025-12-17 16:34:36,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:36,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940388f575bc060ce157f7b HTTP/1.1" 200 None 2025-12-17 16:34:36,238 - lava_callback - INFO - Completed processing callback for node 6940388f575bc060ce157f7b INFO: 172.21.0.1:56586 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:34:36,691 - lava_callback - INFO - Received callback for node 6940389a575bc060ce157f91 INFO: 172.21.0.1:56596 - "POST /node/6940389a575bc060ce157f91 HTTP/1.0" 202 Accepted 2025-12-17 16:34:36,700 - lava_callback - INFO - Starting async job submit for node 6940389a575bc060ce157f91 2025-12-17 16:34:36,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:36,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940389a575bc060ce157f91 HTTP/1.1" 200 None 2025-12-17 16:34:36,712 - lava_callback - DEBUG - Processing node 6940389a575bc060ce157f91: baseline-arm64-baylibre 2025-12-17 16:34:36,712 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940389a575bc060ce157f91 2025-12-17 16:34:36,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:36,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940389a575bc060ce157f91 HTTP/1.1" 200 None 2025-12-17 16:34:36,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:36,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:34:36,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:36,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940389a575bc060ce157f91 HTTP/1.1" 200 None 2025-12-17 16:34:36,784 - lava_callback - INFO - Completed processing callback for node 6940389a575bc060ce157f91 INFO: 172.21.0.1:54810 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:34:48,724 - lava_callback - INFO - Received callback for node 6942dabce43d2b76e9022524 INFO: 172.21.0.1:54820 - "POST /node/6942dabce43d2b76e9022524 HTTP/1.0" 202 Accepted 2025-12-17 16:34:48,774 - lava_callback - INFO - Starting async job submit for node 6942dabce43d2b76e9022524 2025-12-17 16:34:48,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:48,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dabce43d2b76e9022524 HTTP/1.1" 200 None 2025-12-17 16:34:48,840 - lava_callback - DEBUG - Processing node 6942dabce43d2b76e9022524: baseline-arm 2025-12-17 16:34:49,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:34:49,574 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:34:49,575 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942dabce43d2b76e9022524/log.txt.gz 2025-12-17 16:34:49,575 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:34:49,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:34:49,746 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:34:49,747 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942dabce43d2b76e9022524/lava_callback.json.gz 2025-12-17 16:34:49,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:49,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dabce43d2b76e9022524 HTTP/1.1" 200 None 2025-12-17 16:34:49,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:49,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:34:49,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:49,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942dabce43d2b76e9022524 HTTP/1.1" 200 None 2025-12-17 16:34:49,815 - lava_callback - INFO - Completed processing callback for node 6942dabce43d2b76e9022524 2025-12-17 16:34:55,323 - lava_callback - INFO - Received callback for node 694038a4575bc060ce157f9b INFO: 172.21.0.1:46608 - "POST /node/694038a4575bc060ce157f9b HTTP/1.0" 202 Accepted 2025-12-17 16:34:55,334 - lava_callback - INFO - Starting async job submit for node 694038a4575bc060ce157f9b 2025-12-17 16:34:55,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:55,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038a4575bc060ce157f9b HTTP/1.1" 200 None 2025-12-17 16:34:55,348 - lava_callback - DEBUG - Processing node 694038a4575bc060ce157f9b: baseline-arm64-baylibre 2025-12-17 16:34:55,348 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038a4575bc060ce157f9b 2025-12-17 16:34:55,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:55,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038a4575bc060ce157f9b HTTP/1.1" 200 None 2025-12-17 16:34:55,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:55,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:34:55,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:55,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038a4575bc060ce157f9b HTTP/1.1" 200 None 2025-12-17 16:34:55,406 - lava_callback - INFO - Completed processing callback for node 694038a4575bc060ce157f9b 2025-12-17 16:34:55,995 - lava_callback - INFO - Received callback for node 694038a7575bc060ce157fa4 INFO: 172.21.0.1:46624 - "POST /node/694038a7575bc060ce157fa4 HTTP/1.0" 202 Accepted 2025-12-17 16:34:56,004 - lava_callback - INFO - Starting async job submit for node 694038a7575bc060ce157fa4 2025-12-17 16:34:56,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:56,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038a7575bc060ce157fa4 HTTP/1.1" 200 None 2025-12-17 16:34:56,015 - lava_callback - DEBUG - Processing node 694038a7575bc060ce157fa4: baseline-arm64-baylibre 2025-12-17 16:34:56,015 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038a7575bc060ce157fa4 2025-12-17 16:34:56,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:56,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038a7575bc060ce157fa4 HTTP/1.1" 200 None 2025-12-17 16:34:56,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:56,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:34:56,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:56,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038a7575bc060ce157fa4 HTTP/1.1" 200 None 2025-12-17 16:34:56,060 - lava_callback - INFO - Completed processing callback for node 694038a7575bc060ce157fa4 INFO: 172.21.0.1:46638 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:34:56,646 - lava_callback - INFO - Received callback for node 694038a9575bc060ce157fa7 INFO: 172.21.0.1:46646 - "POST /node/694038a9575bc060ce157fa7 HTTP/1.0" 202 Accepted 2025-12-17 16:34:56,655 - lava_callback - INFO - Starting async job submit for node 694038a9575bc060ce157fa7 2025-12-17 16:34:56,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:56,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038a9575bc060ce157fa7 HTTP/1.1" 200 None 2025-12-17 16:34:56,665 - lava_callback - DEBUG - Processing node 694038a9575bc060ce157fa7: baseline-arm64-baylibre 2025-12-17 16:34:56,665 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038a9575bc060ce157fa7 2025-12-17 16:34:56,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:56,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038a9575bc060ce157fa7 HTTP/1.1" 200 None 2025-12-17 16:34:56,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:56,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:34:56,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:34:56,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038a9575bc060ce157fa7 HTTP/1.1" 200 None 2025-12-17 16:34:56,714 - lava_callback - INFO - Completed processing callback for node 694038a9575bc060ce157fa7 2025-12-17 16:35:01,206 - lava_callback - INFO - Received callback for node 6940387b575bc060ce157f37 INFO: 172.21.0.1:41736 - "POST /node/6940387b575bc060ce157f37 HTTP/1.0" 202 Accepted 2025-12-17 16:35:01,218 - lava_callback - INFO - Starting async job submit for node 6940387b575bc060ce157f37 2025-12-17 16:35:01,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:01,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387b575bc060ce157f37 HTTP/1.1" 200 None 2025-12-17 16:35:01,272 - lava_callback - DEBUG - Processing node 6940387b575bc060ce157f37: baseline-arm 2025-12-17 16:35:01,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:35:02,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:35:02,101 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6940387b575bc060ce157f37/log.txt.gz 2025-12-17 16:35:02,101 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:35:02,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:35:02,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:35:02,319 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6940387b575bc060ce157f37/lava_callback.json.gz 2025-12-17 16:35:02,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:02,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940387b575bc060ce157f37 HTTP/1.1" 200 None 2025-12-17 16:35:02,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:02,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:35:02,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:02,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940387b575bc060ce157f37 HTTP/1.1" 200 None 2025-12-17 16:35:02,446 - lava_callback - INFO - Completed processing callback for node 6940387b575bc060ce157f37 INFO: 172.21.0.1:41750 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:35:08,397 - lava_callback - INFO - Received callback for node 694038ab575bc060ce157faa INFO: 172.21.0.1:41758 - "POST /node/694038ab575bc060ce157faa HTTP/1.0" 202 Accepted 2025-12-17 16:35:08,409 - lava_callback - INFO - Starting async job submit for node 694038ab575bc060ce157faa 2025-12-17 16:35:08,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:08,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038ab575bc060ce157faa HTTP/1.1" 200 None 2025-12-17 16:35:08,427 - lava_callback - DEBUG - Processing node 694038ab575bc060ce157faa: baseline-arm-broonie 2025-12-17 16:35:08,427 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038ab575bc060ce157faa 2025-12-17 16:35:08,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:08,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038ab575bc060ce157faa HTTP/1.1" 200 None 2025-12-17 16:35:08,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:08,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d8e27c18b79b848b5fc10 HTTP/1.1" 200 None 2025-12-17 16:35:08,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:08,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038ab575bc060ce157faa HTTP/1.1" 200 None 2025-12-17 16:35:08,481 - lava_callback - INFO - Completed processing callback for node 694038ab575bc060ce157faa INFO: 172.21.0.1:58484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58496 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:35:15,369 - lava_callback - INFO - Received callback for node 694038b8575bc060ce157faf INFO: 172.21.0.1:58504 - "POST /node/694038b8575bc060ce157faf HTTP/1.0" 202 Accepted 2025-12-17 16:35:15,377 - lava_callback - INFO - Starting async job submit for node 694038b8575bc060ce157faf 2025-12-17 16:35:15,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:15,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038b8575bc060ce157faf HTTP/1.1" 200 None 2025-12-17 16:35:15,390 - lava_callback - DEBUG - Processing node 694038b8575bc060ce157faf: baseline-arm64-baylibre 2025-12-17 16:35:15,390 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038b8575bc060ce157faf 2025-12-17 16:35:15,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:15,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038b8575bc060ce157faf HTTP/1.1" 200 None 2025-12-17 16:35:15,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:15,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:35:15,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:15,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038b8575bc060ce157faf HTTP/1.1" 200 None 2025-12-17 16:35:15,448 - lava_callback - INFO - Completed processing callback for node 694038b8575bc060ce157faf 2025-12-17 16:35:15,712 - lava_callback - INFO - Received callback for node 694038bf575bc060ce157fc0 INFO: 172.21.0.1:58518 - "POST /node/694038bf575bc060ce157fc0 HTTP/1.0" 202 Accepted 2025-12-17 16:35:15,721 - lava_callback - INFO - Starting async job submit for node 694038bf575bc060ce157fc0 2025-12-17 16:35:15,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:15,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038bf575bc060ce157fc0 HTTP/1.1" 200 None 2025-12-17 16:35:15,736 - lava_callback - DEBUG - Processing node 694038bf575bc060ce157fc0: baseline-arm-pengutronix 2025-12-17 16:35:15,737 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038bf575bc060ce157fc0 2025-12-17 16:35:15,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:15,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038bf575bc060ce157fc0 HTTP/1.1" 200 None 2025-12-17 16:35:15,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:15,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d8e27c18b79b848b5fc10 HTTP/1.1" 200 None 2025-12-17 16:35:15,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:15,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038bf575bc060ce157fc0 HTTP/1.1" 200 None 2025-12-17 16:35:15,798 - lava_callback - INFO - Completed processing callback for node 694038bf575bc060ce157fc0 2025-12-17 16:35:16,054 - lava_callback - INFO - Received callback for node 694038bb575bc060ce157fb6 INFO: 172.21.0.1:58520 - "POST /node/694038bb575bc060ce157fb6 HTTP/1.0" 202 Accepted 2025-12-17 16:35:16,067 - lava_callback - INFO - Starting async job submit for node 694038bb575bc060ce157fb6 2025-12-17 16:35:16,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:16,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038bb575bc060ce157fb6 HTTP/1.1" 200 None 2025-12-17 16:35:16,082 - lava_callback - DEBUG - Processing node 694038bb575bc060ce157fb6: baseline-arm64-baylibre 2025-12-17 16:35:16,083 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038bb575bc060ce157fb6 2025-12-17 16:35:16,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:16,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038bb575bc060ce157fb6 HTTP/1.1" 200 None 2025-12-17 16:35:16,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:16,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:35:16,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:16,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038bb575bc060ce157fb6 HTTP/1.1" 200 None 2025-12-17 16:35:16,135 - lava_callback - INFO - Completed processing callback for node 694038bb575bc060ce157fb6 INFO: 172.21.0.1:58536 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:35:16,749 - lava_callback - INFO - Received callback for node 694038bd575bc060ce157fbb INFO: 172.21.0.1:58546 - "POST /node/694038bd575bc060ce157fbb HTTP/1.0" 202 Accepted 2025-12-17 16:35:16,758 - lava_callback - INFO - Starting async job submit for node 694038bd575bc060ce157fbb 2025-12-17 16:35:16,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:16,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038bd575bc060ce157fbb HTTP/1.1" 200 None 2025-12-17 16:35:16,774 - lava_callback - DEBUG - Processing node 694038bd575bc060ce157fbb: baseline-arm64-baylibre 2025-12-17 16:35:16,774 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038bd575bc060ce157fbb 2025-12-17 16:35:16,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:16,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038bd575bc060ce157fbb HTTP/1.1" 200 None 2025-12-17 16:35:16,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:16,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:35:16,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:16,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038bd575bc060ce157fbb HTTP/1.1" 200 None 2025-12-17 16:35:16,829 - lava_callback - INFO - Completed processing callback for node 694038bd575bc060ce157fbb INFO: 172.21.0.1:39058 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:35:28,257 - lava_callback - INFO - Received callback for node 694038cf575bc060ce157fca INFO: 172.21.0.1:39062 - "POST /node/694038cf575bc060ce157fca HTTP/1.0" 202 Accepted 2025-12-17 16:35:28,266 - lava_callback - INFO - Starting async job submit for node 694038cf575bc060ce157fca 2025-12-17 16:35:28,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:28,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038cf575bc060ce157fca HTTP/1.1" 200 None 2025-12-17 16:35:28,276 - lava_callback - DEBUG - Processing node 694038cf575bc060ce157fca: ltp-syscalls 2025-12-17 16:35:28,276 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038cf575bc060ce157fca 2025-12-17 16:35:28,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:28,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038cf575bc060ce157fca HTTP/1.1" 200 None 2025-12-17 16:35:28,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:28,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d8e27c18b79b848b5fc10 HTTP/1.1" 200 None 2025-12-17 16:35:28,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:28,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038cf575bc060ce157fca HTTP/1.1" 200 None 2025-12-17 16:35:28,326 - lava_callback - INFO - Completed processing callback for node 694038cf575bc060ce157fca 2025-12-17 16:35:29,884 - lava_callback - INFO - Received callback for node 694038c1575bc060ce157fc3 INFO: 172.21.0.1:34554 - "POST /node/694038c1575bc060ce157fc3 HTTP/1.0" 202 Accepted 2025-12-17 16:35:29,893 - lava_callback - INFO - Starting async job submit for node 694038c1575bc060ce157fc3 2025-12-17 16:35:29,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:29,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038c1575bc060ce157fc3 HTTP/1.1" 200 None 2025-12-17 16:35:29,906 - lava_callback - DEBUG - Processing node 694038c1575bc060ce157fc3: baseline-arm64-clabbe 2025-12-17 16:35:29,906 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038c1575bc060ce157fc3 2025-12-17 16:35:29,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:29,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038c1575bc060ce157fc3 HTTP/1.1" 200 None 2025-12-17 16:35:29,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:29,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:35:29,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:29,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038c1575bc060ce157fc3 HTTP/1.1" 200 None 2025-12-17 16:35:29,944 - lava_callback - INFO - Completed processing callback for node 694038c1575bc060ce157fc3 INFO: 172.21.0.1:34570 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:35:41,035 - lava_callback - INFO - Received callback for node 69403bbf575bc060ce158501 INFO: 172.21.0.1:44524 - "POST /node/69403bbf575bc060ce158501 HTTP/1.0" 202 Accepted 2025-12-17 16:35:41,052 - lava_callback - INFO - Starting async job submit for node 69403bbf575bc060ce158501 2025-12-17 16:35:41,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:41,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bbf575bc060ce158501 HTTP/1.1" 200 None 2025-12-17 16:35:41,209 - lava_callback - DEBUG - Processing node 69403bbf575bc060ce158501: baseline-nfs-arm64 2025-12-17 16:35:43,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:35:43,639 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:35:43,640 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69403bbf575bc060ce158501/log.txt.gz 2025-12-17 16:35:43,640 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:35:43,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:35:44,284 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:35:44,285 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69403bbf575bc060ce158501/lava_callback.json.gz DEBUG: Setup failed for 69403bbf575bc060ce158501. Transitting job node result: incomplete -> fail 2025-12-17 16:35:44,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:44,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bbf575bc060ce158501 HTTP/1.1" 200 None 2025-12-17 16:35:44,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:44,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d1eabfad9ee307f5cfc12 HTTP/1.1" 200 None 2025-12-17 16:35:44,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:44,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403bbf575bc060ce158501 HTTP/1.1" 200 None 2025-12-17 16:35:44,541 - lava_callback - INFO - Completed processing callback for node 69403bbf575bc060ce158501 INFO: 172.21.0.1:44540 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:35:48,331 - lava_callback - INFO - Received callback for node 694038d0575bc060ce157fd1 INFO: 172.21.0.1:44554 - "POST /node/694038d0575bc060ce157fd1 HTTP/1.0" 202 Accepted 2025-12-17 16:35:48,341 - lava_callback - INFO - Starting async job submit for node 694038d0575bc060ce157fd1 2025-12-17 16:35:48,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:48,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038d0575bc060ce157fd1 HTTP/1.1" 200 None 2025-12-17 16:35:48,360 - lava_callback - DEBUG - Processing node 694038d0575bc060ce157fd1: ltp-syscalls-ipc 2025-12-17 16:35:48,360 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038d0575bc060ce157fd1 2025-12-17 16:35:48,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:48,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038d0575bc060ce157fd1 HTTP/1.1" 200 None 2025-12-17 16:35:48,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:48,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d8e27c18b79b848b5fc10 HTTP/1.1" 200 None 2025-12-17 16:35:48,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:48,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038d0575bc060ce157fd1 HTTP/1.1" 200 None 2025-12-17 16:35:48,427 - lava_callback - INFO - Completed processing callback for node 694038d0575bc060ce157fd1 2025-12-17 16:35:49,151 - lava_callback - INFO - Received callback for node 694038d1575bc060ce157fd6 INFO: 172.21.0.1:44558 - "POST /node/694038d1575bc060ce157fd6 HTTP/1.0" 202 Accepted 2025-12-17 16:35:49,160 - lava_callback - INFO - Starting async job submit for node 694038d1575bc060ce157fd6 2025-12-17 16:35:49,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:49,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038d1575bc060ce157fd6 HTTP/1.1" 200 None 2025-12-17 16:35:49,178 - lava_callback - DEBUG - Processing node 694038d1575bc060ce157fd6: ltp-timers 2025-12-17 16:35:49,178 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038d1575bc060ce157fd6 2025-12-17 16:35:49,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:49,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038d1575bc060ce157fd6 HTTP/1.1" 200 None 2025-12-17 16:35:49,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:49,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d8e27c18b79b848b5fc10 HTTP/1.1" 200 None 2025-12-17 16:35:49,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:49,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038d1575bc060ce157fd6 HTTP/1.1" 200 None 2025-12-17 16:35:49,236 - lava_callback - INFO - Completed processing callback for node 694038d1575bc060ce157fd6 2025-12-17 16:35:49,808 - lava_callback - INFO - Received callback for node 694038d4575bc060ce157fe2 INFO: 172.21.0.1:52768 - "POST /node/694038d4575bc060ce157fe2 HTTP/1.0" 202 Accepted 2025-12-17 16:35:49,820 - lava_callback - INFO - Starting async job submit for node 694038d4575bc060ce157fe2 2025-12-17 16:35:49,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:49,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038d4575bc060ce157fe2 HTTP/1.1" 200 None 2025-12-17 16:35:49,846 - lava_callback - DEBUG - Processing node 694038d4575bc060ce157fe2: baseline-arm64-clabbe 2025-12-17 16:35:49,846 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038d4575bc060ce157fe2 2025-12-17 16:35:49,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:49,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038d4575bc060ce157fe2 HTTP/1.1" 200 None 2025-12-17 16:35:49,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:49,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:35:49,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:49,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038d4575bc060ce157fe2 HTTP/1.1" 200 None 2025-12-17 16:35:49,911 - lava_callback - INFO - Completed processing callback for node 694038d4575bc060ce157fe2 2025-12-17 16:35:49,930 - lava_callback - INFO - Received callback for node 694038d3575bc060ce157fdd INFO: 172.21.0.1:52772 - "POST /node/694038d3575bc060ce157fdd HTTP/1.0" 202 Accepted 2025-12-17 16:35:49,947 - lava_callback - INFO - Starting async job submit for node 694038d3575bc060ce157fdd 2025-12-17 16:35:49,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:49,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038d3575bc060ce157fdd HTTP/1.1" 200 None 2025-12-17 16:35:49,984 - lava_callback - DEBUG - Processing node 694038d3575bc060ce157fdd: ltp-watchqueue 2025-12-17 16:35:49,984 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038d3575bc060ce157fdd 2025-12-17 16:35:49,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:50,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038d3575bc060ce157fdd HTTP/1.1" 200 None 2025-12-17 16:35:50,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:50,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d8e27c18b79b848b5fc10 HTTP/1.1" 200 None 2025-12-17 16:35:50,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:50,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038d3575bc060ce157fdd HTTP/1.1" 200 None 2025-12-17 16:35:50,100 - lava_callback - INFO - Completed processing callback for node 694038d3575bc060ce157fdd 2025-12-17 16:35:50,495 - lava_callback - INFO - Received callback for node 694038d7575bc060ce157fe5 INFO: 172.21.0.1:52788 - "POST /node/694038d7575bc060ce157fe5 HTTP/1.0" 202 Accepted 2025-12-17 16:35:50,505 - lava_callback - INFO - Starting async job submit for node 694038d7575bc060ce157fe5 2025-12-17 16:35:50,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:50,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038d7575bc060ce157fe5 HTTP/1.1" 200 None 2025-12-17 16:35:50,523 - lava_callback - DEBUG - Processing node 694038d7575bc060ce157fe5: baseline-arm64-clabbe 2025-12-17 16:35:50,523 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038d7575bc060ce157fe5 2025-12-17 16:35:50,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:50,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038d7575bc060ce157fe5 HTTP/1.1" 200 None 2025-12-17 16:35:50,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:50,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:35:50,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:50,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038d7575bc060ce157fe5 HTTP/1.1" 200 None 2025-12-17 16:35:50,579 - lava_callback - INFO - Completed processing callback for node 694038d7575bc060ce157fe5 2025-12-17 16:35:51,148 - lava_callback - INFO - Received callback for node 694038d9575bc060ce157fe8 INFO: 172.21.0.1:52796 - "POST /node/694038d9575bc060ce157fe8 HTTP/1.0" 202 Accepted 2025-12-17 16:35:51,157 - lava_callback - INFO - Starting async job submit for node 694038d9575bc060ce157fe8 2025-12-17 16:35:51,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:51,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038d9575bc060ce157fe8 HTTP/1.1" 200 None 2025-12-17 16:35:51,176 - lava_callback - DEBUG - Processing node 694038d9575bc060ce157fe8: baseline-arm64-clabbe 2025-12-17 16:35:51,176 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038d9575bc060ce157fe8 2025-12-17 16:35:51,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:51,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038d9575bc060ce157fe8 HTTP/1.1" 200 None 2025-12-17 16:35:51,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:51,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:35:51,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:51,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038d9575bc060ce157fe8 HTTP/1.1" 200 None 2025-12-17 16:35:51,250 - lava_callback - INFO - Completed processing callback for node 694038d9575bc060ce157fe8 2025-12-17 16:35:51,805 - lava_callback - INFO - Received callback for node 694038db575bc060ce157feb INFO: 172.21.0.1:52800 - "POST /node/694038db575bc060ce157feb HTTP/1.0" 202 Accepted 2025-12-17 16:35:51,818 - lava_callback - INFO - Starting async job submit for node 694038db575bc060ce157feb 2025-12-17 16:35:51,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:51,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038db575bc060ce157feb HTTP/1.1" 200 None 2025-12-17 16:35:51,847 - lava_callback - DEBUG - Processing node 694038db575bc060ce157feb: baseline-arm64-clabbe 2025-12-17 16:35:51,847 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038db575bc060ce157feb 2025-12-17 16:35:51,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:51,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038db575bc060ce157feb HTTP/1.1" 200 None 2025-12-17 16:35:51,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:51,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8e3fb52d1dfff36c707 HTTP/1.1" 200 None 2025-12-17 16:35:51,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:51,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038db575bc060ce157feb HTTP/1.1" 200 None 2025-12-17 16:35:51,926 - lava_callback - INFO - Completed processing callback for node 694038db575bc060ce157feb 2025-12-17 16:35:55,393 - lava_callback - INFO - Received callback for node 694038e0575bc060ce157ff0 INFO: 172.21.0.1:52806 - "POST /node/694038e0575bc060ce157ff0 HTTP/1.0" 202 Accepted 2025-12-17 16:35:55,402 - lava_callback - INFO - Starting async job submit for node 694038e0575bc060ce157ff0 2025-12-17 16:35:55,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:55,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038e0575bc060ce157ff0 HTTP/1.1" 200 None 2025-12-17 16:35:55,415 - lava_callback - DEBUG - Processing node 694038e0575bc060ce157ff0: baseline-arm64-baylibre 2025-12-17 16:35:55,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038e0575bc060ce157ff0 2025-12-17 16:35:55,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:55,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038e0575bc060ce157ff0 HTTP/1.1" 200 None 2025-12-17 16:35:55,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:55,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:35:55,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:55,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038e0575bc060ce157ff0 HTTP/1.1" 200 None 2025-12-17 16:35:55,514 - lava_callback - INFO - Completed processing callback for node 694038e0575bc060ce157ff0 2025-12-17 16:35:56,077 - lava_callback - INFO - Received callback for node 694038e3575bc060ce157ff5 INFO: 172.21.0.1:52812 - "POST /node/694038e3575bc060ce157ff5 HTTP/1.0" 202 Accepted 2025-12-17 16:35:56,086 - lava_callback - INFO - Starting async job submit for node 694038e3575bc060ce157ff5 2025-12-17 16:35:56,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:52814 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:35:56,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038e3575bc060ce157ff5 HTTP/1.1" 200 None 2025-12-17 16:35:56,560 - lava_callback - DEBUG - Processing node 694038e3575bc060ce157ff5: baseline-arm64-baylibre 2025-12-17 16:35:56,560 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038e3575bc060ce157ff5 2025-12-17 16:35:56,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:56,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038e3575bc060ce157ff5 HTTP/1.1" 200 None 2025-12-17 16:35:56,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:56,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:35:56,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:35:56,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038e3575bc060ce157ff5 HTTP/1.1" 200 None 2025-12-17 16:35:56,610 - lava_callback - INFO - Completed processing callback for node 694038e3575bc060ce157ff5 INFO: 172.21.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40914 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:36:15,431 - lava_callback - INFO - Received callback for node 694038f4575bc060ce157ffb INFO: 172.21.0.1:40922 - "POST /node/694038f4575bc060ce157ffb HTTP/1.0" 202 Accepted 2025-12-17 16:36:15,437 - lava_callback - INFO - Starting async job submit for node 694038f4575bc060ce157ffb 2025-12-17 16:36:15,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:15,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038f4575bc060ce157ffb HTTP/1.1" 200 None 2025-12-17 16:36:15,446 - lava_callback - DEBUG - Processing node 694038f4575bc060ce157ffb: baseline-arm64-baylibre 2025-12-17 16:36:15,446 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038f4575bc060ce157ffb 2025-12-17 16:36:15,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:15,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038f4575bc060ce157ffb HTTP/1.1" 200 None 2025-12-17 16:36:15,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:15,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:36:15,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:15,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038f4575bc060ce157ffb HTTP/1.1" 200 None 2025-12-17 16:36:15,481 - lava_callback - INFO - Completed processing callback for node 694038f4575bc060ce157ffb 2025-12-17 16:36:16,094 - lava_callback - INFO - Received callback for node 694038f7575bc060ce158006 INFO: 172.21.0.1:40926 - "POST /node/694038f7575bc060ce158006 HTTP/1.0" 202 Accepted 2025-12-17 16:36:16,107 - lava_callback - INFO - Starting async job submit for node 694038f7575bc060ce158006 2025-12-17 16:36:16,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:16,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038f7575bc060ce158006 HTTP/1.1" 200 None 2025-12-17 16:36:16,122 - lava_callback - DEBUG - Processing node 694038f7575bc060ce158006: baseline-arm64-baylibre 2025-12-17 16:36:16,122 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038f7575bc060ce158006 2025-12-17 16:36:16,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:16,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038f7575bc060ce158006 HTTP/1.1" 200 None 2025-12-17 16:36:16,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:16,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:36:16,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:16,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038f7575bc060ce158006 HTTP/1.1" 200 None 2025-12-17 16:36:16,157 - lava_callback - INFO - Completed processing callback for node 694038f7575bc060ce158006 INFO: 172.21.0.1:40932 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:36:16,776 - lava_callback - INFO - Received callback for node 694038f9575bc060ce158009 INFO: 172.21.0.1:40934 - "POST /node/694038f9575bc060ce158009 HTTP/1.0" 202 Accepted 2025-12-17 16:36:16,788 - lava_callback - INFO - Starting async job submit for node 694038f9575bc060ce158009 2025-12-17 16:36:16,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:16,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038f9575bc060ce158009 HTTP/1.1" 200 None 2025-12-17 16:36:16,802 - lava_callback - DEBUG - Processing node 694038f9575bc060ce158009: baseline-arm64-baylibre 2025-12-17 16:36:16,802 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038f9575bc060ce158009 2025-12-17 16:36:16,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:16,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038f9575bc060ce158009 HTTP/1.1" 200 None 2025-12-17 16:36:16,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:16,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:36:16,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:16,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038f9575bc060ce158009 HTTP/1.1" 200 None 2025-12-17 16:36:16,840 - lava_callback - INFO - Completed processing callback for node 694038f9575bc060ce158009 2025-12-17 16:36:17,477 - lava_callback - INFO - Received callback for node 694038fb575bc060ce15800c INFO: 172.21.0.1:40948 - "POST /node/694038fb575bc060ce15800c HTTP/1.0" 202 Accepted 2025-12-17 16:36:17,487 - lava_callback - INFO - Starting async job submit for node 694038fb575bc060ce15800c 2025-12-17 16:36:17,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:17,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038fb575bc060ce15800c HTTP/1.1" 200 None 2025-12-17 16:36:17,498 - lava_callback - DEBUG - Processing node 694038fb575bc060ce15800c: baseline-arm64-baylibre 2025-12-17 16:36:17,498 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038fb575bc060ce15800c 2025-12-17 16:36:17,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:17,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038fb575bc060ce15800c HTTP/1.1" 200 None 2025-12-17 16:36:17,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:17,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:36:17,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:17,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038fb575bc060ce15800c HTTP/1.1" 200 None 2025-12-17 16:36:17,548 - lava_callback - INFO - Completed processing callback for node 694038fb575bc060ce15800c 2025-12-17 16:36:18,136 - lava_callback - INFO - Received callback for node 694038fd575bc060ce15800f INFO: 172.21.0.1:40956 - "POST /node/694038fd575bc060ce15800f HTTP/1.0" 202 Accepted 2025-12-17 16:36:18,144 - lava_callback - INFO - Starting async job submit for node 694038fd575bc060ce15800f 2025-12-17 16:36:18,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:18,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038fd575bc060ce15800f HTTP/1.1" 200 None 2025-12-17 16:36:18,162 - lava_callback - DEBUG - Processing node 694038fd575bc060ce15800f: baseline-arm64-baylibre 2025-12-17 16:36:18,162 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694038fd575bc060ce15800f 2025-12-17 16:36:18,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:18,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694038fd575bc060ce15800f HTTP/1.1" 200 None 2025-12-17 16:36:18,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:18,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:36:18,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:18,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694038fd575bc060ce15800f HTTP/1.1" 200 None 2025-12-17 16:36:18,240 - lava_callback - INFO - Completed processing callback for node 694038fd575bc060ce15800f INFO: 172.21.0.1:53012 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:36:35,407 - lava_callback - INFO - Received callback for node 69403908575bc060ce158014 INFO: 172.21.0.1:60152 - "POST /node/69403908575bc060ce158014 HTTP/1.0" 202 Accepted 2025-12-17 16:36:35,418 - lava_callback - INFO - Starting async job submit for node 69403908575bc060ce158014 2025-12-17 16:36:35,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:35,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403908575bc060ce158014 HTTP/1.1" 200 None 2025-12-17 16:36:35,431 - lava_callback - DEBUG - Processing node 69403908575bc060ce158014: baseline-arm64-baylibre 2025-12-17 16:36:35,431 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403908575bc060ce158014 2025-12-17 16:36:35,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:35,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403908575bc060ce158014 HTTP/1.1" 200 None 2025-12-17 16:36:35,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:35,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:36:35,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:35,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403908575bc060ce158014 HTTP/1.1" 200 None 2025-12-17 16:36:35,491 - lava_callback - INFO - Completed processing callback for node 69403908575bc060ce158014 2025-12-17 16:36:36,067 - lava_callback - INFO - Received callback for node 6940390b575bc060ce158019 INFO: 172.21.0.1:60158 - "POST /node/6940390b575bc060ce158019 HTTP/1.0" 202 Accepted 2025-12-17 16:36:36,076 - lava_callback - INFO - Starting async job submit for node 6940390b575bc060ce158019 2025-12-17 16:36:36,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:36,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940390b575bc060ce158019 HTTP/1.1" 200 None 2025-12-17 16:36:36,089 - lava_callback - DEBUG - Processing node 6940390b575bc060ce158019: baseline-arm64-baylibre 2025-12-17 16:36:36,089 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940390b575bc060ce158019 2025-12-17 16:36:36,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:36,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940390b575bc060ce158019 HTTP/1.1" 200 None 2025-12-17 16:36:36,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:36,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:36:36,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:36,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940390b575bc060ce158019 HTTP/1.1" 200 None 2025-12-17 16:36:36,128 - lava_callback - INFO - Completed processing callback for node 6940390b575bc060ce158019 INFO: 172.21.0.1:60170 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:36:42,596 - lava_callback - INFO - Received callback for node 6942bccf1b61f1c6005fe4f2 INFO: 172.21.0.1:42516 - "POST /node/6942bccf1b61f1c6005fe4f2 HTTP/1.0" 202 Accepted 2025-12-17 16:36:42,605 - lava_callback - INFO - Starting async job submit for node 6942bccf1b61f1c6005fe4f2 2025-12-17 16:36:42,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:42,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942bccf1b61f1c6005fe4f2 HTTP/1.1" 200 None 2025-12-17 16:36:42,649 - lava_callback - DEBUG - Processing node 6942bccf1b61f1c6005fe4f2: baseline-arm-broonie 2025-12-17 16:36:42,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:36:43,106 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:36:43,107 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-6942bccf1b61f1c6005fe4f2/log.txt.gz 2025-12-17 16:36:43,108 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:36:43,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:36:43,254 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:36:43,255 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-6942bccf1b61f1c6005fe4f2/lava_callback.json.gz 2025-12-17 16:36:43,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:43,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942bccf1b61f1c6005fe4f2 HTTP/1.1" 200 None 2025-12-17 16:36:43,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:43,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694263e5bb27571da24bb9ec HTTP/1.1" 200 None 2025-12-17 16:36:43,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:43,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942bccf1b61f1c6005fe4f2 HTTP/1.1" 200 None 2025-12-17 16:36:43,330 - lava_callback - INFO - Completed processing callback for node 6942bccf1b61f1c6005fe4f2 INFO: 172.21.0.1:42520 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:36:49,901 - lava_callback - INFO - Received callback for node 6940390e575bc060ce15801e INFO: 172.21.0.1:42944 - "POST /node/6940390e575bc060ce15801e HTTP/1.0" 202 Accepted 2025-12-17 16:36:49,908 - lava_callback - INFO - Starting async job submit for node 6940390e575bc060ce15801e 2025-12-17 16:36:49,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:49,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940390e575bc060ce15801e HTTP/1.1" 200 None 2025-12-17 16:36:49,917 - lava_callback - DEBUG - Processing node 6940390e575bc060ce15801e: baseline-arm64-clabbe 2025-12-17 16:36:49,917 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940390e575bc060ce15801e 2025-12-17 16:36:49,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:49,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940390e575bc060ce15801e HTTP/1.1" 200 None 2025-12-17 16:36:49,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:49,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:36:49,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:49,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940390e575bc060ce15801e HTTP/1.1" 200 None 2025-12-17 16:36:49,955 - lava_callback - INFO - Completed processing callback for node 6940390e575bc060ce15801e 2025-12-17 16:36:50,550 - lava_callback - INFO - Received callback for node 69403911575bc060ce158031 INFO: 172.21.0.1:42960 - "POST /node/69403911575bc060ce158031 HTTP/1.0" 202 Accepted 2025-12-17 16:36:50,557 - lava_callback - INFO - Starting async job submit for node 69403911575bc060ce158031 2025-12-17 16:36:50,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:50,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403911575bc060ce158031 HTTP/1.1" 200 None 2025-12-17 16:36:50,565 - lava_callback - DEBUG - Processing node 69403911575bc060ce158031: baseline-arm64-clabbe 2025-12-17 16:36:50,565 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403911575bc060ce158031 2025-12-17 16:36:50,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:50,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403911575bc060ce158031 HTTP/1.1" 200 None 2025-12-17 16:36:50,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:50,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:36:50,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:50,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403911575bc060ce158031 HTTP/1.1" 200 None 2025-12-17 16:36:50,600 - lava_callback - INFO - Completed processing callback for node 69403911575bc060ce158031 2025-12-17 16:36:51,216 - lava_callback - INFO - Received callback for node 69403913575bc060ce158034 INFO: 172.21.0.1:42972 - "POST /node/69403913575bc060ce158034 HTTP/1.0" 202 Accepted 2025-12-17 16:36:51,222 - lava_callback - INFO - Starting async job submit for node 69403913575bc060ce158034 2025-12-17 16:36:51,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:51,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403913575bc060ce158034 HTTP/1.1" 200 None 2025-12-17 16:36:51,231 - lava_callback - DEBUG - Processing node 69403913575bc060ce158034: baseline-arm64-clabbe 2025-12-17 16:36:51,231 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403913575bc060ce158034 2025-12-17 16:36:51,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:51,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403913575bc060ce158034 HTTP/1.1" 200 None 2025-12-17 16:36:51,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:51,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:36:51,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:51,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403913575bc060ce158034 HTTP/1.1" 200 None 2025-12-17 16:36:51,272 - lava_callback - INFO - Completed processing callback for node 69403913575bc060ce158034 2025-12-17 16:36:51,902 - lava_callback - INFO - Received callback for node 69403917575bc060ce15803a INFO: 172.21.0.1:42978 - "POST /node/69403917575bc060ce15803a HTTP/1.0" 202 Accepted 2025-12-17 16:36:51,909 - lava_callback - INFO - Starting async job submit for node 69403917575bc060ce15803a 2025-12-17 16:36:51,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:51,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403917575bc060ce15803a HTTP/1.1" 200 None 2025-12-17 16:36:51,918 - lava_callback - DEBUG - Processing node 69403917575bc060ce15803a: baseline-arm64-clabbe 2025-12-17 16:36:51,918 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403917575bc060ce15803a 2025-12-17 16:36:51,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:51,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403917575bc060ce15803a HTTP/1.1" 200 None 2025-12-17 16:36:51,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:51,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:36:51,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:51,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403917575bc060ce15803a HTTP/1.1" 200 None 2025-12-17 16:36:51,962 - lava_callback - INFO - Completed processing callback for node 69403917575bc060ce15803a 2025-12-17 16:36:52,558 - lava_callback - INFO - Received callback for node 69403919575bc060ce158041 INFO: 172.21.0.1:42990 - "POST /node/69403919575bc060ce158041 HTTP/1.0" 202 Accepted 2025-12-17 16:36:52,564 - lava_callback - INFO - Starting async job submit for node 69403919575bc060ce158041 2025-12-17 16:36:52,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:52,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403919575bc060ce158041 HTTP/1.1" 200 None 2025-12-17 16:36:52,578 - lava_callback - DEBUG - Processing node 69403919575bc060ce158041: baseline-arm64-clabbe 2025-12-17 16:36:52,578 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403919575bc060ce158041 2025-12-17 16:36:52,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:52,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403919575bc060ce158041 HTTP/1.1" 200 None 2025-12-17 16:36:52,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:52,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:36:52,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:52,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403919575bc060ce158041 HTTP/1.1" 200 None 2025-12-17 16:36:52,610 - lava_callback - INFO - Completed processing callback for node 69403919575bc060ce158041 2025-12-17 16:36:55,476 - lava_callback - INFO - Received callback for node 6940391c575bc060ce158049 INFO: 172.21.0.1:43004 - "POST /node/6940391c575bc060ce158049 HTTP/1.0" 202 Accepted 2025-12-17 16:36:55,485 - lava_callback - INFO - Starting async job submit for node 6940391c575bc060ce158049 2025-12-17 16:36:55,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:55,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940391c575bc060ce158049 HTTP/1.1" 200 None 2025-12-17 16:36:55,494 - lava_callback - DEBUG - Processing node 6940391c575bc060ce158049: baseline-arm64-baylibre 2025-12-17 16:36:55,494 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940391c575bc060ce158049 2025-12-17 16:36:55,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:55,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940391c575bc060ce158049 HTTP/1.1" 200 None 2025-12-17 16:36:55,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:55,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:36:55,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:55,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940391c575bc060ce158049 HTTP/1.1" 200 None 2025-12-17 16:36:55,530 - lava_callback - INFO - Completed processing callback for node 6940391c575bc060ce158049 2025-12-17 16:36:55,768 - lava_callback - INFO - Received callback for node 69403915575bc060ce158037 INFO: 172.21.0.1:43012 - "POST /node/69403915575bc060ce158037 HTTP/1.0" 202 Accepted 2025-12-17 16:36:55,775 - lava_callback - INFO - Starting async job submit for node 69403915575bc060ce158037 2025-12-17 16:36:55,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:55,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403915575bc060ce158037 HTTP/1.1" 200 None 2025-12-17 16:36:55,784 - lava_callback - DEBUG - Processing node 69403915575bc060ce158037: baseline-arm 2025-12-17 16:36:55,784 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403915575bc060ce158037 2025-12-17 16:36:55,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:55,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403915575bc060ce158037 HTTP/1.1" 200 None 2025-12-17 16:36:55,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:55,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8bbfb52d1dfff36c6a3 HTTP/1.1" 200 None 2025-12-17 16:36:55,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:36:55,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403915575bc060ce158037 HTTP/1.1" 200 None 2025-12-17 16:36:55,817 - lava_callback - INFO - Completed processing callback for node 69403915575bc060ce158037 INFO: 172.21.0.1:43020 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:37:02,448 - lava_callback - INFO - Received callback for node 6940391f575bc060ce15804e INFO: 172.21.0.1:41386 - "POST /node/6940391f575bc060ce15804e HTTP/1.0" 202 Accepted 2025-12-17 16:37:02,458 - lava_callback - INFO - Starting async job submit for node 6940391f575bc060ce15804e 2025-12-17 16:37:02,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:02,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940391f575bc060ce15804e HTTP/1.1" 200 None 2025-12-17 16:37:02,497 - lava_callback - DEBUG - Processing node 6940391f575bc060ce15804e: baseline-arm 2025-12-17 16:37:02,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:37:02,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:37:02,873 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6940391f575bc060ce15804e/log.txt.gz 2025-12-17 16:37:02,873 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:37:02,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:37:03,032 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:37:03,033 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6940391f575bc060ce15804e/lava_callback.json.gz 2025-12-17 16:37:03,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:03,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940391f575bc060ce15804e HTTP/1.1" 200 None 2025-12-17 16:37:03,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:03,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922877ecffe173f95b6b HTTP/1.1" 200 None 2025-12-17 16:37:03,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:03,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940391f575bc060ce15804e HTTP/1.1" 200 None 2025-12-17 16:37:03,115 - lava_callback - INFO - Completed processing callback for node 6940391f575bc060ce15804e INFO: 172.21.0.1:41388 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:37:08,472 - lava_callback - INFO - Received callback for node 69403920575bc060ce158058 INFO: 172.21.0.1:41392 - "POST /node/69403920575bc060ce158058 HTTP/1.0" 202 Accepted 2025-12-17 16:37:08,480 - lava_callback - INFO - Starting async job submit for node 69403920575bc060ce158058 2025-12-17 16:37:08,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:08,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403920575bc060ce158058 HTTP/1.1" 200 None 2025-12-17 16:37:08,490 - lava_callback - DEBUG - Processing node 69403920575bc060ce158058: baseline-arm 2025-12-17 16:37:08,490 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403920575bc060ce158058 2025-12-17 16:37:08,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:08,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403920575bc060ce158058 HTTP/1.1" 200 None 2025-12-17 16:37:08,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:08,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:08,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:08,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403920575bc060ce158058 HTTP/1.1" 200 None 2025-12-17 16:37:08,528 - lava_callback - INFO - Completed processing callback for node 69403920575bc060ce158058 2025-12-17 16:37:09,275 - lava_callback - INFO - Received callback for node 69403920575bc060ce15805d INFO: 172.21.0.1:41408 - "POST /node/69403920575bc060ce15805d HTTP/1.0" 202 Accepted 2025-12-17 16:37:09,281 - lava_callback - INFO - Starting async job submit for node 69403920575bc060ce15805d 2025-12-17 16:37:09,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:09,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403920575bc060ce15805d HTTP/1.1" 200 None 2025-12-17 16:37:09,291 - lava_callback - DEBUG - Processing node 69403920575bc060ce15805d: baseline-arm 2025-12-17 16:37:09,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403920575bc060ce15805d 2025-12-17 16:37:09,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:09,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403920575bc060ce15805d HTTP/1.1" 200 None 2025-12-17 16:37:09,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:09,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:09,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:09,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403920575bc060ce15805d HTTP/1.1" 200 None 2025-12-17 16:37:09,322 - lava_callback - INFO - Completed processing callback for node 69403920575bc060ce15805d 2025-12-17 16:37:10,044 - lava_callback - INFO - Received callback for node 69403921575bc060ce158062 INFO: 172.21.0.1:58938 - "POST /node/69403921575bc060ce158062 HTTP/1.0" 202 Accepted 2025-12-17 16:37:10,051 - lava_callback - INFO - Starting async job submit for node 69403921575bc060ce158062 2025-12-17 16:37:10,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:10,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403921575bc060ce158062 HTTP/1.1" 200 None 2025-12-17 16:37:10,063 - lava_callback - DEBUG - Processing node 69403921575bc060ce158062: baseline-arm 2025-12-17 16:37:10,063 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403921575bc060ce158062 2025-12-17 16:37:10,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:10,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403921575bc060ce158062 HTTP/1.1" 200 None 2025-12-17 16:37:10,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:10,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:10,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:10,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403921575bc060ce158062 HTTP/1.1" 200 None 2025-12-17 16:37:10,100 - lava_callback - INFO - Completed processing callback for node 69403921575bc060ce158062 2025-12-17 16:37:10,792 - lava_callback - INFO - Received callback for node 69403922575bc060ce158067 INFO: 172.21.0.1:58944 - "POST /node/69403922575bc060ce158067 HTTP/1.0" 202 Accepted 2025-12-17 16:37:10,800 - lava_callback - INFO - Starting async job submit for node 69403922575bc060ce158067 2025-12-17 16:37:10,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:10,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403922575bc060ce158067 HTTP/1.1" 200 None 2025-12-17 16:37:10,809 - lava_callback - DEBUG - Processing node 69403922575bc060ce158067: baseline-arm 2025-12-17 16:37:10,809 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403922575bc060ce158067 2025-12-17 16:37:10,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:10,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403922575bc060ce158067 HTTP/1.1" 200 None 2025-12-17 16:37:10,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:10,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:10,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:10,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403922575bc060ce158067 HTTP/1.1" 200 None 2025-12-17 16:37:10,843 - lava_callback - INFO - Completed processing callback for node 69403922575bc060ce158067 INFO: 172.21.0.1:58956 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:37:11,504 - lava_callback - INFO - Received callback for node 69403923575bc060ce15806c INFO: 172.21.0.1:58962 - "POST /node/69403923575bc060ce15806c HTTP/1.0" 202 Accepted 2025-12-17 16:37:11,511 - lava_callback - INFO - Starting async job submit for node 69403923575bc060ce15806c 2025-12-17 16:37:11,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:11,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403923575bc060ce15806c HTTP/1.1" 200 None 2025-12-17 16:37:11,546 - lava_callback - DEBUG - Processing node 69403923575bc060ce15806c: baseline-arm 2025-12-17 16:37:11,546 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403923575bc060ce15806c 2025-12-17 16:37:11,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:11,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403923575bc060ce15806c HTTP/1.1" 200 None 2025-12-17 16:37:11,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:11,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:11,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:11,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403923575bc060ce15806c HTTP/1.1" 200 None 2025-12-17 16:37:11,593 - lava_callback - INFO - Completed processing callback for node 69403923575bc060ce15806c 2025-12-17 16:37:12,618 - lava_callback - INFO - Received callback for node 69403923575bc060ce158071 INFO: 172.21.0.1:58970 - "POST /node/69403923575bc060ce158071 HTTP/1.0" 202 Accepted 2025-12-17 16:37:12,624 - lava_callback - INFO - Starting async job submit for node 69403923575bc060ce158071 2025-12-17 16:37:12,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:12,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403923575bc060ce158071 HTTP/1.1" 200 None 2025-12-17 16:37:12,632 - lava_callback - DEBUG - Processing node 69403923575bc060ce158071: ltp-smoketest 2025-12-17 16:37:12,632 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403923575bc060ce158071 2025-12-17 16:37:12,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:12,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403923575bc060ce158071 HTTP/1.1" 200 None 2025-12-17 16:37:12,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:12,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:12,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:12,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403923575bc060ce158071 HTTP/1.1" 200 None 2025-12-17 16:37:12,669 - lava_callback - INFO - Completed processing callback for node 69403923575bc060ce158071 2025-12-17 16:37:14,064 - lava_callback - INFO - Received callback for node 69403924575bc060ce158076 INFO: 172.21.0.1:58976 - "POST /node/69403924575bc060ce158076 HTTP/1.0" 202 Accepted 2025-12-17 16:37:14,071 - lava_callback - INFO - Starting async job submit for node 69403924575bc060ce158076 2025-12-17 16:37:14,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:14,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403924575bc060ce158076 HTTP/1.1" 200 None 2025-12-17 16:37:14,080 - lava_callback - DEBUG - Processing node 69403924575bc060ce158076: baseline-arm 2025-12-17 16:37:14,080 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403924575bc060ce158076 2025-12-17 16:37:14,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:14,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403924575bc060ce158076 HTTP/1.1" 200 None 2025-12-17 16:37:14,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:14,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:14,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:14,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403924575bc060ce158076 HTTP/1.1" 200 None 2025-12-17 16:37:14,119 - lava_callback - INFO - Completed processing callback for node 69403924575bc060ce158076 INFO: 172.21.0.1:58990 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:37:15,295 - lava_callback - INFO - Received callback for node 69403925575bc060ce15807b INFO: 172.21.0.1:59000 - "POST /node/69403925575bc060ce15807b HTTP/1.0" 202 Accepted 2025-12-17 16:37:15,301 - lava_callback - INFO - Starting async job submit for node 69403925575bc060ce15807b 2025-12-17 16:37:15,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:15,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403925575bc060ce15807b HTTP/1.1" 200 None 2025-12-17 16:37:15,311 - lava_callback - DEBUG - Processing node 69403925575bc060ce15807b: ltp-smoketest 2025-12-17 16:37:15,312 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403925575bc060ce15807b 2025-12-17 16:37:15,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:15,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403925575bc060ce15807b HTTP/1.1" 200 None 2025-12-17 16:37:15,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:15,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:15,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:15,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403925575bc060ce15807b HTTP/1.1" 200 None 2025-12-17 16:37:15,346 - lava_callback - INFO - Completed processing callback for node 69403925575bc060ce15807b 2025-12-17 16:37:16,035 - lava_callback - INFO - Received callback for node 69403926575bc060ce158080 INFO: 172.21.0.1:59016 - "POST /node/69403926575bc060ce158080 HTTP/1.0" 202 Accepted 2025-12-17 16:37:16,042 - lava_callback - INFO - Starting async job submit for node 69403926575bc060ce158080 2025-12-17 16:37:16,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:16,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403926575bc060ce158080 HTTP/1.1" 200 None 2025-12-17 16:37:16,054 - lava_callback - DEBUG - Processing node 69403926575bc060ce158080: ltp-smoketest 2025-12-17 16:37:16,054 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403926575bc060ce158080 2025-12-17 16:37:16,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:16,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403926575bc060ce158080 HTTP/1.1" 200 None 2025-12-17 16:37:16,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:16,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:16,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:16,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403926575bc060ce158080 HTTP/1.1" 200 None 2025-12-17 16:37:16,099 - lava_callback - INFO - Completed processing callback for node 69403926575bc060ce158080 INFO: 172.21.0.1:59018 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:37:16,814 - lava_callback - INFO - Received callback for node 69403926575bc060ce158086 INFO: 172.21.0.1:59024 - "POST /node/69403926575bc060ce158086 HTTP/1.0" 202 Accepted 2025-12-17 16:37:16,821 - lava_callback - INFO - Starting async job submit for node 69403926575bc060ce158086 2025-12-17 16:37:16,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:16,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403926575bc060ce158086 HTTP/1.1" 200 None 2025-12-17 16:37:16,835 - lava_callback - DEBUG - Processing node 69403926575bc060ce158086: ltp-smoketest 2025-12-17 16:37:16,835 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403926575bc060ce158086 2025-12-17 16:37:16,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:16,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403926575bc060ce158086 HTTP/1.1" 200 None 2025-12-17 16:37:16,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:16,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:16,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:16,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403926575bc060ce158086 HTTP/1.1" 200 None 2025-12-17 16:37:16,876 - lava_callback - INFO - Completed processing callback for node 69403926575bc060ce158086 2025-12-17 16:37:17,533 - lava_callback - INFO - Received callback for node 69403927575bc060ce15808b INFO: 172.21.0.1:59040 - "POST /node/69403927575bc060ce15808b HTTP/1.0" 202 Accepted 2025-12-17 16:37:17,538 - lava_callback - INFO - Starting async job submit for node 69403927575bc060ce15808b 2025-12-17 16:37:17,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:17,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403927575bc060ce15808b HTTP/1.1" 200 None 2025-12-17 16:37:17,548 - lava_callback - DEBUG - Processing node 69403927575bc060ce15808b: ltp-smoketest 2025-12-17 16:37:17,548 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403927575bc060ce15808b 2025-12-17 16:37:17,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:17,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403927575bc060ce15808b HTTP/1.1" 200 None 2025-12-17 16:37:17,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:17,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:17,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:17,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403927575bc060ce15808b HTTP/1.1" 200 None 2025-12-17 16:37:17,580 - lava_callback - INFO - Completed processing callback for node 69403927575bc060ce15808b 2025-12-17 16:37:18,965 - lava_callback - INFO - Received callback for node 69403928575bc060ce158090 INFO: 172.21.0.1:59050 - "POST /node/69403928575bc060ce158090 HTTP/1.0" 202 Accepted 2025-12-17 16:37:18,972 - lava_callback - INFO - Starting async job submit for node 69403928575bc060ce158090 2025-12-17 16:37:18,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:18,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403928575bc060ce158090 HTTP/1.1" 200 None 2025-12-17 16:37:18,982 - lava_callback - DEBUG - Processing node 69403928575bc060ce158090: ltp-smoketest 2025-12-17 16:37:18,982 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403928575bc060ce158090 2025-12-17 16:37:18,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:18,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403928575bc060ce158090 HTTP/1.1" 200 None 2025-12-17 16:37:18,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:19,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:19,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:19,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403928575bc060ce158090 HTTP/1.1" 200 None 2025-12-17 16:37:19,020 - lava_callback - INFO - Completed processing callback for node 69403928575bc060ce158090 2025-12-17 16:37:19,761 - lava_callback - INFO - Received callback for node 69403932575bc060ce158096 INFO: 172.21.0.1:60008 - "POST /node/69403932575bc060ce158096 HTTP/1.0" 202 Accepted 2025-12-17 16:37:19,769 - lava_callback - INFO - Starting async job submit for node 69403932575bc060ce158096 2025-12-17 16:37:19,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:19,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403932575bc060ce158096 HTTP/1.1" 200 None 2025-12-17 16:37:19,779 - lava_callback - DEBUG - Processing node 69403932575bc060ce158096: baseline-arm 2025-12-17 16:37:19,779 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403932575bc060ce158096 2025-12-17 16:37:19,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:19,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403932575bc060ce158096 HTTP/1.1" 200 None 2025-12-17 16:37:19,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:19,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:19,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:19,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403932575bc060ce158096 HTTP/1.1" 200 None 2025-12-17 16:37:19,816 - lava_callback - INFO - Completed processing callback for node 69403932575bc060ce158096 2025-12-17 16:37:20,583 - lava_callback - INFO - Received callback for node 69403932575bc060ce15809b INFO: 172.21.0.1:60018 - "POST /node/69403932575bc060ce15809b HTTP/1.0" 202 Accepted 2025-12-17 16:37:20,589 - lava_callback - INFO - Starting async job submit for node 69403932575bc060ce15809b 2025-12-17 16:37:20,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:20,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403932575bc060ce15809b HTTP/1.1" 200 None 2025-12-17 16:37:20,598 - lava_callback - DEBUG - Processing node 69403932575bc060ce15809b: baseline-arm 2025-12-17 16:37:20,598 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403932575bc060ce15809b 2025-12-17 16:37:20,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:20,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403932575bc060ce15809b HTTP/1.1" 200 None 2025-12-17 16:37:20,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:20,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:20,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:20,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403932575bc060ce15809b HTTP/1.1" 200 None 2025-12-17 16:37:20,633 - lava_callback - INFO - Completed processing callback for node 69403932575bc060ce15809b 2025-12-17 16:37:21,691 - lava_callback - INFO - Received callback for node 69403933575bc060ce1580a0 INFO: 172.21.0.1:60026 - "POST /node/69403933575bc060ce1580a0 HTTP/1.0" 202 Accepted 2025-12-17 16:37:21,698 - lava_callback - INFO - Starting async job submit for node 69403933575bc060ce1580a0 2025-12-17 16:37:21,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:21,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403933575bc060ce1580a0 HTTP/1.1" 200 None 2025-12-17 16:37:21,716 - lava_callback - DEBUG - Processing node 69403933575bc060ce1580a0: baseline-arm 2025-12-17 16:37:21,716 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403933575bc060ce1580a0 2025-12-17 16:37:21,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:21,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403933575bc060ce1580a0 HTTP/1.1" 200 None 2025-12-17 16:37:21,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:21,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:21,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:21,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403933575bc060ce1580a0 HTTP/1.1" 200 None 2025-12-17 16:37:21,766 - lava_callback - INFO - Completed processing callback for node 69403933575bc060ce1580a0 2025-12-17 16:37:22,410 - lava_callback - INFO - Received callback for node 69403934575bc060ce1580a5 INFO: 172.21.0.1:60034 - "POST /node/69403934575bc060ce1580a5 HTTP/1.0" 202 Accepted 2025-12-17 16:37:22,418 - lava_callback - INFO - Starting async job submit for node 69403934575bc060ce1580a5 2025-12-17 16:37:22,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:22,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403934575bc060ce1580a5 HTTP/1.1" 200 None 2025-12-17 16:37:22,426 - lava_callback - DEBUG - Processing node 69403934575bc060ce1580a5: ltp-smoketest 2025-12-17 16:37:22,427 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403934575bc060ce1580a5 2025-12-17 16:37:22,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:22,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403934575bc060ce1580a5 HTTP/1.1" 200 None 2025-12-17 16:37:22,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:22,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:22,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:22,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403934575bc060ce1580a5 HTTP/1.1" 200 None 2025-12-17 16:37:22,470 - lava_callback - INFO - Completed processing callback for node 69403934575bc060ce1580a5 2025-12-17 16:37:23,233 - lava_callback - INFO - Received callback for node 69403935575bc060ce1580aa INFO: 172.21.0.1:60038 - "POST /node/69403935575bc060ce1580aa HTTP/1.0" 202 Accepted 2025-12-17 16:37:23,242 - lava_callback - INFO - Starting async job submit for node 69403935575bc060ce1580aa 2025-12-17 16:37:23,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:23,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403935575bc060ce1580aa HTTP/1.1" 200 None 2025-12-17 16:37:23,252 - lava_callback - DEBUG - Processing node 69403935575bc060ce1580aa: ltp-smoketest 2025-12-17 16:37:23,252 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403935575bc060ce1580aa 2025-12-17 16:37:23,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:23,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403935575bc060ce1580aa HTTP/1.1" 200 None 2025-12-17 16:37:23,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:23,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:23,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:23,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403935575bc060ce1580aa HTTP/1.1" 200 None 2025-12-17 16:37:23,297 - lava_callback - INFO - Completed processing callback for node 69403935575bc060ce1580aa 2025-12-17 16:37:23,976 - lava_callback - INFO - Received callback for node 69403935575bc060ce1580af INFO: 172.21.0.1:60042 - "POST /node/69403935575bc060ce1580af HTTP/1.0" 202 Accepted 2025-12-17 16:37:23,990 - lava_callback - INFO - Starting async job submit for node 69403935575bc060ce1580af 2025-12-17 16:37:23,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:24,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403935575bc060ce1580af HTTP/1.1" 200 None 2025-12-17 16:37:24,015 - lava_callback - DEBUG - Processing node 69403935575bc060ce1580af: ltp-smoketest 2025-12-17 16:37:24,015 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403935575bc060ce1580af 2025-12-17 16:37:24,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:24,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403935575bc060ce1580af HTTP/1.1" 200 None 2025-12-17 16:37:24,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:24,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0abfad9ee307f5b1dd2 HTTP/1.1" 200 None 2025-12-17 16:37:24,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:24,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403935575bc060ce1580af HTTP/1.1" 200 None 2025-12-17 16:37:24,051 - lava_callback - INFO - Completed processing callback for node 69403935575bc060ce1580af INFO: 172.21.0.1:60050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34402 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:37:55,452 - lava_callback - INFO - Received callback for node 69403958575bc060ce1580c6 INFO: 172.21.0.1:59792 - "POST /node/69403958575bc060ce1580c6 HTTP/1.0" 202 Accepted 2025-12-17 16:37:55,458 - lava_callback - INFO - Starting async job submit for node 69403958575bc060ce1580c6 2025-12-17 16:37:55,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:55,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403958575bc060ce1580c6 HTTP/1.1" 200 None 2025-12-17 16:37:55,468 - lava_callback - DEBUG - Processing node 69403958575bc060ce1580c6: baseline-arm64-baylibre 2025-12-17 16:37:55,468 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403958575bc060ce1580c6 2025-12-17 16:37:55,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:55,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403958575bc060ce1580c6 HTTP/1.1" 200 None 2025-12-17 16:37:55,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:55,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:37:55,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:55,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403958575bc060ce1580c6 HTTP/1.1" 200 None 2025-12-17 16:37:55,500 - lava_callback - INFO - Completed processing callback for node 69403958575bc060ce1580c6 2025-12-17 16:37:56,109 - lava_callback - INFO - Received callback for node 6940395b575bc060ce1580cd INFO: 172.21.0.1:59798 - "POST /node/6940395b575bc060ce1580cd HTTP/1.0" 202 Accepted 2025-12-17 16:37:56,116 - lava_callback - INFO - Starting async job submit for node 6940395b575bc060ce1580cd 2025-12-17 16:37:56,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:56,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940395b575bc060ce1580cd HTTP/1.1" 200 None 2025-12-17 16:37:56,125 - lava_callback - DEBUG - Processing node 6940395b575bc060ce1580cd: baseline-arm64-baylibre 2025-12-17 16:37:56,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940395b575bc060ce1580cd 2025-12-17 16:37:56,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:56,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940395b575bc060ce1580cd HTTP/1.1" 200 None 2025-12-17 16:37:56,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:56,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:37:56,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:56,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940395b575bc060ce1580cd HTTP/1.1" 200 None 2025-12-17 16:37:56,167 - lava_callback - INFO - Completed processing callback for node 6940395b575bc060ce1580cd INFO: 172.21.0.1:59806 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:37:56,785 - lava_callback - INFO - Received callback for node 6940395d575bc060ce1580d0 INFO: 172.21.0.1:59808 - "POST /node/6940395d575bc060ce1580d0 HTTP/1.0" 202 Accepted 2025-12-17 16:37:56,792 - lava_callback - INFO - Starting async job submit for node 6940395d575bc060ce1580d0 2025-12-17 16:37:56,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:56,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940395d575bc060ce1580d0 HTTP/1.1" 200 None 2025-12-17 16:37:56,801 - lava_callback - DEBUG - Processing node 6940395d575bc060ce1580d0: baseline-arm64-baylibre 2025-12-17 16:37:56,801 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940395d575bc060ce1580d0 2025-12-17 16:37:56,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:56,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940395d575bc060ce1580d0 HTTP/1.1" 200 None 2025-12-17 16:37:56,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:56,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:37:56,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:37:56,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940395d575bc060ce1580d0 HTTP/1.1" 200 None 2025-12-17 16:37:56,837 - lava_callback - INFO - Completed processing callback for node 6940395d575bc060ce1580d0 2025-12-17 16:38:04,044 - lava_callback - INFO - Received callback for node 6942c1d51b61f1c6005febf0 INFO: 172.21.0.1:36362 - "POST /node/6942c1d51b61f1c6005febf0 HTTP/1.0" 202 Accepted 2025-12-17 16:38:04,052 - lava_callback - INFO - Starting async job submit for node 6942c1d51b61f1c6005febf0 2025-12-17 16:38:04,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:04,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942c1d51b61f1c6005febf0 HTTP/1.1" 200 None 2025-12-17 16:38:04,087 - lava_callback - DEBUG - Processing node 6942c1d51b61f1c6005febf0: baseline-arm-broonie 2025-12-17 16:38:04,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:38:04,486 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:38:04,487 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-6942c1d51b61f1c6005febf0/log.txt.gz 2025-12-17 16:38:04,487 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:38:04,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:38:04,623 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:38:04,624 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-6942c1d51b61f1c6005febf0/lava_callback.json.gz 2025-12-17 16:38:04,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:04,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942c1d51b61f1c6005febf0 HTTP/1.1" 200 None 2025-12-17 16:38:04,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:04,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694267041b61f1c6005f4e99 HTTP/1.1" 200 None 2025-12-17 16:38:04,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:04,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942c1d51b61f1c6005febf0 HTTP/1.1" 200 None 2025-12-17 16:38:04,706 - lava_callback - INFO - Completed processing callback for node 6942c1d51b61f1c6005febf0 INFO: 172.21.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34048 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:38:15,482 - lava_callback - INFO - Received callback for node 6940396d575bc060ce1580f0 INFO: 172.21.0.1:34056 - "POST /node/6940396d575bc060ce1580f0 HTTP/1.0" 202 Accepted 2025-12-17 16:38:15,492 - lava_callback - INFO - Starting async job submit for node 6940396d575bc060ce1580f0 2025-12-17 16:38:15,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:15,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940396d575bc060ce1580f0 HTTP/1.1" 200 None 2025-12-17 16:38:15,506 - lava_callback - DEBUG - Processing node 6940396d575bc060ce1580f0: baseline-arm-baylibre 2025-12-17 16:38:15,506 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940396d575bc060ce1580f0 2025-12-17 16:38:15,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:15,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940396d575bc060ce1580f0 HTTP/1.1" 200 None 2025-12-17 16:38:15,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:15,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:38:15,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:15,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940396d575bc060ce1580f0 HTTP/1.1" 200 None 2025-12-17 16:38:15,566 - lava_callback - INFO - Completed processing callback for node 6940396d575bc060ce1580f0 2025-12-17 16:38:16,149 - lava_callback - INFO - Received callback for node 69403970575bc060ce1580fd INFO: 172.21.0.1:34058 - "POST /node/69403970575bc060ce1580fd HTTP/1.0" 202 Accepted 2025-12-17 16:38:16,159 - lava_callback - INFO - Starting async job submit for node 69403970575bc060ce1580fd 2025-12-17 16:38:16,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:16,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403970575bc060ce1580fd HTTP/1.1" 200 None 2025-12-17 16:38:16,176 - lava_callback - DEBUG - Processing node 69403970575bc060ce1580fd: baseline-arm64-baylibre 2025-12-17 16:38:16,176 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403970575bc060ce1580fd 2025-12-17 16:38:16,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:16,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403970575bc060ce1580fd HTTP/1.1" 200 None 2025-12-17 16:38:16,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:16,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:38:16,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:16,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403970575bc060ce1580fd HTTP/1.1" 200 None 2025-12-17 16:38:16,227 - lava_callback - INFO - Completed processing callback for node 69403970575bc060ce1580fd INFO: 172.21.0.1:34064 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:38:16,821 - lava_callback - INFO - Received callback for node 69403972575bc060ce158100 INFO: 172.21.0.1:34080 - "POST /node/69403972575bc060ce158100 HTTP/1.0" 202 Accepted 2025-12-17 16:38:16,832 - lava_callback - INFO - Starting async job submit for node 69403972575bc060ce158100 2025-12-17 16:38:16,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:16,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403972575bc060ce158100 HTTP/1.1" 200 None 2025-12-17 16:38:16,848 - lava_callback - DEBUG - Processing node 69403972575bc060ce158100: baseline-arm-baylibre 2025-12-17 16:38:16,848 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403972575bc060ce158100 2025-12-17 16:38:16,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:16,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403972575bc060ce158100 HTTP/1.1" 200 None 2025-12-17 16:38:16,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:16,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:38:16,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:16,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403972575bc060ce158100 HTTP/1.1" 200 None 2025-12-17 16:38:16,905 - lava_callback - INFO - Completed processing callback for node 69403972575bc060ce158100 2025-12-17 16:38:17,485 - lava_callback - INFO - Received callback for node 69403974575bc060ce158103 INFO: 172.21.0.1:34090 - "POST /node/69403974575bc060ce158103 HTTP/1.0" 202 Accepted 2025-12-17 16:38:17,495 - lava_callback - INFO - Starting async job submit for node 69403974575bc060ce158103 2025-12-17 16:38:17,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:17,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403974575bc060ce158103 HTTP/1.1" 200 None 2025-12-17 16:38:17,523 - lava_callback - DEBUG - Processing node 69403974575bc060ce158103: baseline-arm-baylibre 2025-12-17 16:38:17,523 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403974575bc060ce158103 2025-12-17 16:38:17,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:17,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403974575bc060ce158103 HTTP/1.1" 200 None 2025-12-17 16:38:17,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:17,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:38:17,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:17,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403974575bc060ce158103 HTTP/1.1" 200 None 2025-12-17 16:38:17,568 - lava_callback - INFO - Completed processing callback for node 69403974575bc060ce158103 2025-12-17 16:38:18,155 - lava_callback - INFO - Received callback for node 69403976575bc060ce158106 INFO: 172.21.0.1:34102 - "POST /node/69403976575bc060ce158106 HTTP/1.0" 202 Accepted 2025-12-17 16:38:18,170 - lava_callback - INFO - Starting async job submit for node 69403976575bc060ce158106 2025-12-17 16:38:18,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:18,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403976575bc060ce158106 HTTP/1.1" 200 None 2025-12-17 16:38:18,195 - lava_callback - DEBUG - Processing node 69403976575bc060ce158106: baseline-arm-baylibre 2025-12-17 16:38:18,195 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403976575bc060ce158106 2025-12-17 16:38:18,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:18,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403976575bc060ce158106 HTTP/1.1" 200 None 2025-12-17 16:38:18,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:18,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:38:18,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:18,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403976575bc060ce158106 HTTP/1.1" 200 None 2025-12-17 16:38:18,248 - lava_callback - INFO - Completed processing callback for node 69403976575bc060ce158106 INFO: 172.21.0.1:55052 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:38:28,415 - lava_callback - INFO - Received callback for node 6942dbcfe43d2b76e9022765 INFO: 172.21.0.1:55056 - "POST /node/6942dbcfe43d2b76e9022765 HTTP/1.0" 202 Accepted 2025-12-17 16:38:28,427 - lava_callback - INFO - Starting async job submit for node 6942dbcfe43d2b76e9022765 2025-12-17 16:38:28,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:28,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dbcfe43d2b76e9022765 HTTP/1.1" 200 None 2025-12-17 16:38:28,483 - lava_callback - DEBUG - Processing node 6942dbcfe43d2b76e9022765: baseline-arm 2025-12-17 16:38:29,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:38:29,458 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:38:29,459 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942dbcfe43d2b76e9022765/log.txt.gz 2025-12-17 16:38:29,460 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:38:29,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:38:30,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:38:30,289 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942dbcfe43d2b76e9022765/lava_callback.json.gz 2025-12-17 16:38:30,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:30,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dbcfe43d2b76e9022765 HTTP/1.1" 200 None 2025-12-17 16:38:30,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:30,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:38:30,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:30,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942dbcfe43d2b76e9022765 HTTP/1.1" 200 None 2025-12-17 16:38:30,450 - lava_callback - INFO - Completed processing callback for node 6942dbcfe43d2b76e9022765 2025-12-17 16:38:35,499 - lava_callback - INFO - Received callback for node 69403981575bc060ce15816e INFO: 172.21.0.1:59516 - "POST /node/69403981575bc060ce15816e HTTP/1.0" 202 Accepted 2025-12-17 16:38:35,510 - lava_callback - INFO - Starting async job submit for node 69403981575bc060ce15816e 2025-12-17 16:38:35,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:35,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403981575bc060ce15816e HTTP/1.1" 200 None 2025-12-17 16:38:35,526 - lava_callback - DEBUG - Processing node 69403981575bc060ce15816e: baseline-arm64-baylibre 2025-12-17 16:38:35,526 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403981575bc060ce15816e 2025-12-17 16:38:35,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:35,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403981575bc060ce15816e HTTP/1.1" 200 None 2025-12-17 16:38:35,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:35,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:38:35,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:35,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403981575bc060ce15816e HTTP/1.1" 200 None 2025-12-17 16:38:35,584 - lava_callback - INFO - Completed processing callback for node 69403981575bc060ce15816e 2025-12-17 16:38:36,180 - lava_callback - INFO - Received callback for node 69403984575bc060ce158173 INFO: 172.21.0.1:59520 - "POST /node/69403984575bc060ce158173 HTTP/1.0" 202 Accepted 2025-12-17 16:38:36,189 - lava_callback - INFO - Starting async job submit for node 69403984575bc060ce158173 2025-12-17 16:38:36,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:36,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403984575bc060ce158173 HTTP/1.1" 200 None 2025-12-17 16:38:36,204 - lava_callback - DEBUG - Processing node 69403984575bc060ce158173: baseline-arm-baylibre 2025-12-17 16:38:36,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403984575bc060ce158173 2025-12-17 16:38:36,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:36,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403984575bc060ce158173 HTTP/1.1" 200 None 2025-12-17 16:38:36,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:36,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:38:36,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:36,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403984575bc060ce158173 HTTP/1.1" 200 None 2025-12-17 16:38:36,256 - lava_callback - INFO - Completed processing callback for node 69403984575bc060ce158173 INFO: 172.21.0.1:59530 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:38:36,828 - lava_callback - INFO - Received callback for node 69403986575bc060ce158176 INFO: 172.21.0.1:59542 - "POST /node/69403986575bc060ce158176 HTTP/1.0" 202 Accepted 2025-12-17 16:38:36,837 - lava_callback - INFO - Starting async job submit for node 69403986575bc060ce158176 2025-12-17 16:38:36,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:36,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403986575bc060ce158176 HTTP/1.1" 200 None 2025-12-17 16:38:36,850 - lava_callback - DEBUG - Processing node 69403986575bc060ce158176: baseline-arm64-baylibre 2025-12-17 16:38:36,850 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403986575bc060ce158176 2025-12-17 16:38:36,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:36,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403986575bc060ce158176 HTTP/1.1" 200 None 2025-12-17 16:38:36,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:36,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:38:36,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:36,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403986575bc060ce158176 HTTP/1.1" 200 None 2025-12-17 16:38:36,900 - lava_callback - INFO - Completed processing callback for node 69403986575bc060ce158176 INFO: 172.21.0.1:33378 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:38:55,517 - lava_callback - INFO - Received callback for node 69403997575bc060ce158182 INFO: 172.21.0.1:59236 - "POST /node/69403997575bc060ce158182 HTTP/1.0" 202 Accepted 2025-12-17 16:38:55,523 - lava_callback - INFO - Starting async job submit for node 69403997575bc060ce158182 2025-12-17 16:38:55,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:55,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403997575bc060ce158182 HTTP/1.1" 200 None 2025-12-17 16:38:55,533 - lava_callback - DEBUG - Processing node 69403997575bc060ce158182: baseline-arm64-baylibre 2025-12-17 16:38:55,534 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403997575bc060ce158182 2025-12-17 16:38:55,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:55,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403997575bc060ce158182 HTTP/1.1" 200 None 2025-12-17 16:38:55,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:55,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:38:55,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:55,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403997575bc060ce158182 HTTP/1.1" 200 None 2025-12-17 16:38:55,595 - lava_callback - INFO - Completed processing callback for node 69403997575bc060ce158182 2025-12-17 16:38:56,176 - lava_callback - INFO - Received callback for node 69403994575bc060ce15817b INFO: 172.21.0.1:59238 - "POST /node/69403994575bc060ce15817b HTTP/1.0" 202 Accepted 2025-12-17 16:38:56,185 - lava_callback - INFO - Starting async job submit for node 69403994575bc060ce15817b 2025-12-17 16:38:56,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:56,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403994575bc060ce15817b HTTP/1.1" 200 None 2025-12-17 16:38:56,202 - lava_callback - DEBUG - Processing node 69403994575bc060ce15817b: baseline-arm-baylibre 2025-12-17 16:38:56,202 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403994575bc060ce15817b 2025-12-17 16:38:56,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:56,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403994575bc060ce15817b HTTP/1.1" 200 None 2025-12-17 16:38:56,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:56,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:38:56,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:56,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403994575bc060ce15817b HTTP/1.1" 200 None 2025-12-17 16:38:56,241 - lava_callback - INFO - Completed processing callback for node 69403994575bc060ce15817b INFO: 172.21.0.1:59246 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:38:56,845 - lava_callback - INFO - Received callback for node 6940399a575bc060ce158185 INFO: 172.21.0.1:59250 - "POST /node/6940399a575bc060ce158185 HTTP/1.0" 202 Accepted 2025-12-17 16:38:56,851 - lava_callback - INFO - Starting async job submit for node 6940399a575bc060ce158185 2025-12-17 16:38:56,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:56,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940399a575bc060ce158185 HTTP/1.1" 200 None 2025-12-17 16:38:56,863 - lava_callback - DEBUG - Processing node 6940399a575bc060ce158185: baseline-arm64-baylibre 2025-12-17 16:38:56,863 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940399a575bc060ce158185 2025-12-17 16:38:56,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:56,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940399a575bc060ce158185 HTTP/1.1" 200 None 2025-12-17 16:38:56,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:56,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:38:56,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:38:56,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940399a575bc060ce158185 HTTP/1.1" 200 None 2025-12-17 16:38:56,951 - lava_callback - INFO - Completed processing callback for node 6940399a575bc060ce158185 INFO: 172.21.0.1:33880 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:39:08,353 - lava_callback - INFO - Received callback for node 694039aa575bc060ce158196 INFO: 172.21.0.1:33896 - "POST /node/694039aa575bc060ce158196 HTTP/1.0" 202 Accepted 2025-12-17 16:39:08,359 - lava_callback - INFO - Starting async job submit for node 694039aa575bc060ce158196 2025-12-17 16:39:08,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:08,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039aa575bc060ce158196 HTTP/1.1" 200 None 2025-12-17 16:39:08,371 - lava_callback - DEBUG - Processing node 694039aa575bc060ce158196: baseline-arm-broonie 2025-12-17 16:39:08,371 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039aa575bc060ce158196 2025-12-17 16:39:08,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:08,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039aa575bc060ce158196 HTTP/1.1" 200 None 2025-12-17 16:39:08,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:08,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:39:08,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:08,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039aa575bc060ce158196 HTTP/1.1" 200 None 2025-12-17 16:39:08,412 - lava_callback - INFO - Completed processing callback for node 694039aa575bc060ce158196 INFO: 172.21.0.1:39638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39654 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:39:16,143 - lava_callback - INFO - Received callback for node 694039a2575bc060ce158191 INFO: 172.21.0.1:39660 - "POST /node/694039a2575bc060ce158191 HTTP/1.0" 202 Accepted 2025-12-17 16:39:16,148 - lava_callback - INFO - Starting async job submit for node 694039a2575bc060ce158191 2025-12-17 16:39:16,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:16,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039a2575bc060ce158191 HTTP/1.1" 200 None 2025-12-17 16:39:16,171 - lava_callback - DEBUG - Processing node 694039a2575bc060ce158191: baseline-arm 2025-12-17 16:39:16,171 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039a2575bc060ce158191 2025-12-17 16:39:16,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:16,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039a2575bc060ce158191 HTTP/1.1" 200 None 2025-12-17 16:39:16,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:16,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae22cfee8383e92b63e8f HTTP/1.1" 200 None 2025-12-17 16:39:16,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:16,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039a2575bc060ce158191 HTTP/1.1" 200 None 2025-12-17 16:39:16,206 - lava_callback - INFO - Completed processing callback for node 694039a2575bc060ce158191 INFO: 172.21.0.1:39668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50998 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:39:30,020 - lava_callback - INFO - Received callback for node 694039ae575bc060ce15819d INFO: 172.21.0.1:34792 - "POST /node/694039ae575bc060ce15819d HTTP/1.0" 202 Accepted 2025-12-17 16:39:30,025 - lava_callback - INFO - Starting async job submit for node 694039ae575bc060ce15819d 2025-12-17 16:39:30,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:30,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039ae575bc060ce15819d HTTP/1.1" 200 None 2025-12-17 16:39:30,035 - lava_callback - DEBUG - Processing node 694039ae575bc060ce15819d: baseline-arm64-clabbe 2025-12-17 16:39:30,035 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039ae575bc060ce15819d 2025-12-17 16:39:30,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:30,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039ae575bc060ce15819d HTTP/1.1" 200 None 2025-12-17 16:39:30,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:30,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:39:30,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:30,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039ae575bc060ce15819d HTTP/1.1" 200 None 2025-12-17 16:39:30,119 - lava_callback - INFO - Completed processing callback for node 694039ae575bc060ce15819d 2025-12-17 16:39:30,692 - lava_callback - INFO - Received callback for node 694039b1575bc060ce1581a0 INFO: 172.21.0.1:34806 - "POST /node/694039b1575bc060ce1581a0 HTTP/1.0" 202 Accepted 2025-12-17 16:39:30,702 - lava_callback - INFO - Starting async job submit for node 694039b1575bc060ce1581a0 2025-12-17 16:39:30,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:30,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039b1575bc060ce1581a0 HTTP/1.1" 200 None 2025-12-17 16:39:30,710 - lava_callback - DEBUG - Processing node 694039b1575bc060ce1581a0: baseline-arm-clabbe 2025-12-17 16:39:30,711 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039b1575bc060ce1581a0 2025-12-17 16:39:30,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:30,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039b1575bc060ce1581a0 HTTP/1.1" 200 None 2025-12-17 16:39:30,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:30,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:39:30,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:30,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039b1575bc060ce1581a0 HTTP/1.1" 200 None 2025-12-17 16:39:30,746 - lava_callback - INFO - Completed processing callback for node 694039b1575bc060ce1581a0 INFO: 172.21.0.1:34820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44958 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:39:50,004 - lava_callback - INFO - Received callback for node 694039c2575bc060ce158232 INFO: 172.21.0.1:41652 - "POST /node/694039c2575bc060ce158232 HTTP/1.0" 202 Accepted 2025-12-17 16:39:50,017 - lava_callback - INFO - Starting async job submit for node 694039c2575bc060ce158232 2025-12-17 16:39:50,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:50,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039c2575bc060ce158232 HTTP/1.1" 200 None 2025-12-17 16:39:50,049 - lava_callback - DEBUG - Processing node 694039c2575bc060ce158232: baseline-arm-clabbe 2025-12-17 16:39:50,049 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039c2575bc060ce158232 2025-12-17 16:39:50,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:50,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039c2575bc060ce158232 HTTP/1.1" 200 None 2025-12-17 16:39:50,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:50,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:39:50,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:50,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039c2575bc060ce158232 HTTP/1.1" 200 None 2025-12-17 16:39:50,119 - lava_callback - INFO - Completed processing callback for node 694039c2575bc060ce158232 2025-12-17 16:39:50,688 - lava_callback - INFO - Received callback for node 694039c5575bc060ce15824b INFO: 172.21.0.1:41660 - "POST /node/694039c5575bc060ce15824b HTTP/1.0" 202 Accepted 2025-12-17 16:39:50,697 - lava_callback - INFO - Starting async job submit for node 694039c5575bc060ce15824b 2025-12-17 16:39:50,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:50,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039c5575bc060ce15824b HTTP/1.1" 200 None 2025-12-17 16:39:50,708 - lava_callback - DEBUG - Processing node 694039c5575bc060ce15824b: baseline-arm64-clabbe 2025-12-17 16:39:50,708 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039c5575bc060ce15824b 2025-12-17 16:39:50,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:50,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039c5575bc060ce15824b HTTP/1.1" 200 None 2025-12-17 16:39:50,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:50,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:39:50,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:50,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039c5575bc060ce15824b HTTP/1.1" 200 None 2025-12-17 16:39:50,760 - lava_callback - INFO - Completed processing callback for node 694039c5575bc060ce15824b 2025-12-17 16:39:51,388 - lava_callback - INFO - Received callback for node 694039c7575bc060ce15825b INFO: 172.21.0.1:41668 - "POST /node/694039c7575bc060ce15825b HTTP/1.0" 202 Accepted 2025-12-17 16:39:51,397 - lava_callback - INFO - Starting async job submit for node 694039c7575bc060ce15825b 2025-12-17 16:39:51,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:51,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039c7575bc060ce15825b HTTP/1.1" 200 None 2025-12-17 16:39:51,411 - lava_callback - DEBUG - Processing node 694039c7575bc060ce15825b: baseline-arm64-clabbe 2025-12-17 16:39:51,411 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039c7575bc060ce15825b 2025-12-17 16:39:51,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:51,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039c7575bc060ce15825b HTTP/1.1" 200 None 2025-12-17 16:39:51,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:51,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:39:51,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:51,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039c7575bc060ce15825b HTTP/1.1" 200 None 2025-12-17 16:39:51,469 - lava_callback - INFO - Completed processing callback for node 694039c7575bc060ce15825b 2025-12-17 16:39:55,591 - lava_callback - INFO - Received callback for node 694039d1575bc060ce158273 INFO: 172.21.0.1:41672 - "POST /node/694039d1575bc060ce158273 HTTP/1.0" 202 Accepted 2025-12-17 16:39:55,602 - lava_callback - INFO - Starting async job submit for node 694039d1575bc060ce158273 2025-12-17 16:39:55,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:55,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039d1575bc060ce158273 HTTP/1.1" 200 None 2025-12-17 16:39:55,617 - lava_callback - DEBUG - Processing node 694039d1575bc060ce158273: baseline-arm64-baylibre 2025-12-17 16:39:55,617 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039d1575bc060ce158273 2025-12-17 16:39:55,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:55,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039d1575bc060ce158273 HTTP/1.1" 200 None 2025-12-17 16:39:55,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:55,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:39:55,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:55,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039d1575bc060ce158273 HTTP/1.1" 200 None 2025-12-17 16:39:55,676 - lava_callback - INFO - Completed processing callback for node 694039d1575bc060ce158273 2025-12-17 16:39:56,265 - lava_callback - INFO - Received callback for node 694039d3575bc060ce15827a INFO: 172.21.0.1:41682 - "POST /node/694039d3575bc060ce15827a HTTP/1.0" 202 Accepted 2025-12-17 16:39:56,274 - lava_callback - INFO - Starting async job submit for node 694039d3575bc060ce15827a 2025-12-17 16:39:56,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:56,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039d3575bc060ce15827a HTTP/1.1" 200 None 2025-12-17 16:39:56,286 - lava_callback - DEBUG - Processing node 694039d3575bc060ce15827a: baseline-arm64-baylibre 2025-12-17 16:39:56,286 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039d3575bc060ce15827a 2025-12-17 16:39:56,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:56,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039d3575bc060ce15827a HTTP/1.1" 200 None 2025-12-17 16:39:56,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:56,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:39:56,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:56,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039d3575bc060ce15827a HTTP/1.1" 200 None 2025-12-17 16:39:56,343 - lava_callback - INFO - Completed processing callback for node 694039d3575bc060ce15827a INFO: 172.21.0.1:41684 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:39:56,965 - lava_callback - INFO - Received callback for node 694039d6575bc060ce158284 INFO: 172.21.0.1:41686 - "POST /node/694039d6575bc060ce158284 HTTP/1.0" 202 Accepted 2025-12-17 16:39:56,974 - lava_callback - INFO - Starting async job submit for node 694039d6575bc060ce158284 2025-12-17 16:39:56,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:56,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039d6575bc060ce158284 HTTP/1.1" 200 None 2025-12-17 16:39:56,987 - lava_callback - DEBUG - Processing node 694039d6575bc060ce158284: baseline-arm64-baylibre 2025-12-17 16:39:56,987 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039d6575bc060ce158284 2025-12-17 16:39:56,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:56,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039d6575bc060ce158284 HTTP/1.1" 200 None 2025-12-17 16:39:56,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:57,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:39:57,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:39:57,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039d6575bc060ce158284 HTTP/1.1" 200 None 2025-12-17 16:39:57,032 - lava_callback - INFO - Completed processing callback for node 694039d6575bc060ce158284 INFO: 172.21.0.1:40146 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:40:10,023 - lava_callback - INFO - Received callback for node 694039dd575bc060ce15828b INFO: 172.21.0.1:36922 - "POST /node/694039dd575bc060ce15828b HTTP/1.0" 202 Accepted 2025-12-17 16:40:10,033 - lava_callback - INFO - Starting async job submit for node 694039dd575bc060ce15828b 2025-12-17 16:40:10,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:10,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039dd575bc060ce15828b HTTP/1.1" 200 None 2025-12-17 16:40:10,054 - lava_callback - DEBUG - Processing node 694039dd575bc060ce15828b: baseline-arm-clabbe 2025-12-17 16:40:10,055 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039dd575bc060ce15828b 2025-12-17 16:40:10,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:10,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039dd575bc060ce15828b HTTP/1.1" 200 None 2025-12-17 16:40:10,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:10,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:40:10,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:10,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039dd575bc060ce15828b HTTP/1.1" 200 None 2025-12-17 16:40:10,105 - lava_callback - INFO - Completed processing callback for node 694039dd575bc060ce15828b INFO: 172.21.0.1:36936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36942 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:40:15,608 - lava_callback - INFO - Received callback for node 694039e5575bc060ce158294 INFO: 172.21.0.1:36956 - "POST /node/694039e5575bc060ce158294 HTTP/1.0" 202 Accepted 2025-12-17 16:40:15,617 - lava_callback - INFO - Starting async job submit for node 694039e5575bc060ce158294 2025-12-17 16:40:15,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:15,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039e5575bc060ce158294 HTTP/1.1" 200 None 2025-12-17 16:40:15,629 - lava_callback - DEBUG - Processing node 694039e5575bc060ce158294: baseline-arm64-baylibre 2025-12-17 16:40:15,629 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039e5575bc060ce158294 2025-12-17 16:40:15,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:15,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039e5575bc060ce158294 HTTP/1.1" 200 None 2025-12-17 16:40:15,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:15,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:40:15,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:15,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039e5575bc060ce158294 HTTP/1.1" 200 None 2025-12-17 16:40:15,705 - lava_callback - INFO - Completed processing callback for node 694039e5575bc060ce158294 2025-12-17 16:40:16,287 - lava_callback - INFO - Received callback for node 694039e7575bc060ce15829f INFO: 172.21.0.1:36968 - "POST /node/694039e7575bc060ce15829f HTTP/1.0" 202 Accepted 2025-12-17 16:40:16,296 - lava_callback - INFO - Starting async job submit for node 694039e7575bc060ce15829f 2025-12-17 16:40:16,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:16,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039e7575bc060ce15829f HTTP/1.1" 200 None 2025-12-17 16:40:16,310 - lava_callback - DEBUG - Processing node 694039e7575bc060ce15829f: baseline-arm64-baylibre 2025-12-17 16:40:16,310 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039e7575bc060ce15829f 2025-12-17 16:40:16,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:16,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039e7575bc060ce15829f HTTP/1.1" 200 None 2025-12-17 16:40:16,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:16,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:40:16,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:16,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039e7575bc060ce15829f HTTP/1.1" 200 None 2025-12-17 16:40:16,356 - lava_callback - INFO - Completed processing callback for node 694039e7575bc060ce15829f INFO: 172.21.0.1:36972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35558 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:40:30,065 - lava_callback - INFO - Received callback for node 694039eb575bc060ce1582ab INFO: 172.21.0.1:38014 - "POST /node/694039eb575bc060ce1582ab HTTP/1.0" 202 Accepted 2025-12-17 16:40:30,074 - lava_callback - INFO - Starting async job submit for node 694039eb575bc060ce1582ab 2025-12-17 16:40:30,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:30,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039eb575bc060ce1582ab HTTP/1.1" 200 None 2025-12-17 16:40:30,092 - lava_callback - DEBUG - Processing node 694039eb575bc060ce1582ab: baseline-arm-clabbe 2025-12-17 16:40:30,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039eb575bc060ce1582ab 2025-12-17 16:40:30,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:30,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039eb575bc060ce1582ab HTTP/1.1" 200 None 2025-12-17 16:40:30,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:30,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:40:30,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:30,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039eb575bc060ce1582ab HTTP/1.1" 200 None 2025-12-17 16:40:30,144 - lava_callback - INFO - Completed processing callback for node 694039eb575bc060ce1582ab 2025-12-17 16:40:30,737 - lava_callback - INFO - Received callback for node 694039ed575bc060ce1582ae INFO: 172.21.0.1:38018 - "POST /node/694039ed575bc060ce1582ae HTTP/1.0" 202 Accepted 2025-12-17 16:40:30,746 - lava_callback - INFO - Starting async job submit for node 694039ed575bc060ce1582ae 2025-12-17 16:40:30,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:30,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039ed575bc060ce1582ae HTTP/1.1" 200 None 2025-12-17 16:40:30,758 - lava_callback - DEBUG - Processing node 694039ed575bc060ce1582ae: baseline-arm64-clabbe 2025-12-17 16:40:30,758 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039ed575bc060ce1582ae 2025-12-17 16:40:30,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:30,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039ed575bc060ce1582ae HTTP/1.1" 200 None 2025-12-17 16:40:30,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:30,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:40:30,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:30,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039ed575bc060ce1582ae HTTP/1.1" 200 None 2025-12-17 16:40:30,802 - lava_callback - INFO - Completed processing callback for node 694039ed575bc060ce1582ae 2025-12-17 16:40:31,431 - lava_callback - INFO - Received callback for node 694039f3575bc060ce1582b1 INFO: 172.21.0.1:38022 - "POST /node/694039f3575bc060ce1582b1 HTTP/1.0" 202 Accepted 2025-12-17 16:40:31,441 - lava_callback - INFO - Starting async job submit for node 694039f3575bc060ce1582b1 2025-12-17 16:40:31,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:31,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039f3575bc060ce1582b1 HTTP/1.1" 200 None 2025-12-17 16:40:31,453 - lava_callback - DEBUG - Processing node 694039f3575bc060ce1582b1: baseline-arm64-clabbe 2025-12-17 16:40:31,453 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039f3575bc060ce1582b1 2025-12-17 16:40:31,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:31,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039f3575bc060ce1582b1 HTTP/1.1" 200 None 2025-12-17 16:40:31,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:31,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:40:31,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:31,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039f3575bc060ce1582b1 HTTP/1.1" 200 None 2025-12-17 16:40:31,494 - lava_callback - INFO - Completed processing callback for node 694039f3575bc060ce1582b1 2025-12-17 16:40:35,593 - lava_callback - INFO - Received callback for node 694039f9575bc060ce1582ba INFO: 172.21.0.1:38028 - "POST /node/694039f9575bc060ce1582ba HTTP/1.0" 202 Accepted 2025-12-17 16:40:35,602 - lava_callback - INFO - Starting async job submit for node 694039f9575bc060ce1582ba 2025-12-17 16:40:35,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:35,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039f9575bc060ce1582ba HTTP/1.1" 200 None 2025-12-17 16:40:35,614 - lava_callback - DEBUG - Processing node 694039f9575bc060ce1582ba: baseline-arm64-baylibre 2025-12-17 16:40:35,614 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039f9575bc060ce1582ba 2025-12-17 16:40:35,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:35,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039f9575bc060ce1582ba HTTP/1.1" 200 None 2025-12-17 16:40:35,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:35,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:40:35,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:35,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039f9575bc060ce1582ba HTTP/1.1" 200 None 2025-12-17 16:40:35,678 - lava_callback - INFO - Completed processing callback for node 694039f9575bc060ce1582ba 2025-12-17 16:40:36,245 - lava_callback - INFO - Received callback for node 694039fb575bc060ce1582c3 INFO: 172.21.0.1:38042 - "POST /node/694039fb575bc060ce1582c3 HTTP/1.0" 202 Accepted 2025-12-17 16:40:36,254 - lava_callback - INFO - Starting async job submit for node 694039fb575bc060ce1582c3 2025-12-17 16:40:36,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:36,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039fb575bc060ce1582c3 HTTP/1.1" 200 None 2025-12-17 16:40:36,269 - lava_callback - DEBUG - Processing node 694039fb575bc060ce1582c3: baseline-arm64-baylibre 2025-12-17 16:40:36,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039fb575bc060ce1582c3 2025-12-17 16:40:36,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:36,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039fb575bc060ce1582c3 HTTP/1.1" 200 None 2025-12-17 16:40:36,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:36,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:40:36,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:36,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039fb575bc060ce1582c3 HTTP/1.1" 200 None 2025-12-17 16:40:36,335 - lava_callback - INFO - Completed processing callback for node 694039fb575bc060ce1582c3 INFO: 172.21.0.1:38056 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:40:42,156 - lava_callback - INFO - Received callback for node 6942d5f01b61f1c600601759 INFO: 172.21.0.1:53192 - "POST /node/6942d5f01b61f1c600601759 HTTP/1.0" 202 Accepted 2025-12-17 16:40:42,167 - lava_callback - INFO - Starting async job submit for node 6942d5f01b61f1c600601759 2025-12-17 16:40:42,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:42,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d5f01b61f1c600601759 HTTP/1.1" 200 None 2025-12-17 16:40:42,238 - lava_callback - DEBUG - Processing node 6942d5f01b61f1c600601759: baseline-arm-broonie 2025-12-17 16:40:42,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:40:42,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:40:42,981 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-6942d5f01b61f1c600601759/log.txt.gz 2025-12-17 16:40:42,982 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:40:42,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:40:43,354 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:40:43,356 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-6942d5f01b61f1c600601759/lava_callback.json.gz 2025-12-17 16:40:43,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:43,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d5f01b61f1c600601759 HTTP/1.1" 200 None 2025-12-17 16:40:43,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:43,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69418391459ff5fc64d300a5 HTTP/1.1" 200 None 2025-12-17 16:40:43,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:43,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942d5f01b61f1c600601759 HTTP/1.1" 200 None 2025-12-17 16:40:43,453 - lava_callback - INFO - Completed processing callback for node 6942d5f01b61f1c600601759 INFO: 172.21.0.1:53194 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:40:50,046 - lava_callback - INFO - Received callback for node 694039fe575bc060ce1582cc INFO: 172.21.0.1:59504 - "POST /node/694039fe575bc060ce1582cc HTTP/1.0" 202 Accepted 2025-12-17 16:40:50,055 - lava_callback - INFO - Starting async job submit for node 694039fe575bc060ce1582cc 2025-12-17 16:40:50,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:50,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039fe575bc060ce1582cc HTTP/1.1" 200 None 2025-12-17 16:40:50,066 - lava_callback - DEBUG - Processing node 694039fe575bc060ce1582cc: baseline-arm-clabbe 2025-12-17 16:40:50,066 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694039fe575bc060ce1582cc 2025-12-17 16:40:50,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:50,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694039fe575bc060ce1582cc HTTP/1.1" 200 None 2025-12-17 16:40:50,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:50,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:40:50,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:50,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694039fe575bc060ce1582cc HTTP/1.1" 200 None 2025-12-17 16:40:50,148 - lava_callback - INFO - Completed processing callback for node 694039fe575bc060ce1582cc 2025-12-17 16:40:55,637 - lava_callback - INFO - Received callback for node 69403a0d575bc060ce1582da INFO: 172.21.0.1:59514 - "POST /node/69403a0d575bc060ce1582da HTTP/1.0" 202 Accepted 2025-12-17 16:40:55,642 - lava_callback - INFO - Starting async job submit for node 69403a0d575bc060ce1582da 2025-12-17 16:40:55,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:55,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403a0d575bc060ce1582da HTTP/1.1" 200 None 2025-12-17 16:40:55,652 - lava_callback - DEBUG - Processing node 69403a0d575bc060ce1582da: baseline-arm64-baylibre 2025-12-17 16:40:55,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403a0d575bc060ce1582da 2025-12-17 16:40:55,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:55,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403a0d575bc060ce1582da HTTP/1.1" 200 None 2025-12-17 16:40:55,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:55,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:40:55,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:40:55,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403a0d575bc060ce1582da HTTP/1.1" 200 None 2025-12-17 16:40:55,684 - lava_callback - INFO - Completed processing callback for node 69403a0d575bc060ce1582da INFO: 172.21.0.1:59528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59360 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:41:10,046 - lava_callback - INFO - Received callback for node 69403a14575bc060ce1582f8 INFO: 172.21.0.1:33848 - "POST /node/69403a14575bc060ce1582f8 HTTP/1.0" 202 Accepted 2025-12-17 16:41:10,051 - lava_callback - INFO - Starting async job submit for node 69403a14575bc060ce1582f8 2025-12-17 16:41:10,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:10,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403a14575bc060ce1582f8 HTTP/1.1" 200 None 2025-12-17 16:41:10,061 - lava_callback - DEBUG - Processing node 69403a14575bc060ce1582f8: baseline-arm64-clabbe 2025-12-17 16:41:10,061 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403a14575bc060ce1582f8 2025-12-17 16:41:10,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:10,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403a14575bc060ce1582f8 HTTP/1.1" 200 None 2025-12-17 16:41:10,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:10,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:41:10,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:10,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403a14575bc060ce1582f8 HTTP/1.1" 200 None 2025-12-17 16:41:10,095 - lava_callback - INFO - Completed processing callback for node 69403a14575bc060ce1582f8 2025-12-17 16:41:10,694 - lava_callback - INFO - Received callback for node 69403a17575bc060ce1582fb INFO: 172.21.0.1:33864 - "POST /node/69403a17575bc060ce1582fb HTTP/1.0" 202 Accepted 2025-12-17 16:41:10,702 - lava_callback - INFO - Starting async job submit for node 69403a17575bc060ce1582fb 2025-12-17 16:41:10,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:10,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403a17575bc060ce1582fb HTTP/1.1" 200 None 2025-12-17 16:41:10,712 - lava_callback - DEBUG - Processing node 69403a17575bc060ce1582fb: baseline-arm64-clabbe 2025-12-17 16:41:10,712 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403a17575bc060ce1582fb 2025-12-17 16:41:10,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:10,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403a17575bc060ce1582fb HTTP/1.1" 200 None 2025-12-17 16:41:10,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:10,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:41:10,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:10,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403a17575bc060ce1582fb HTTP/1.1" 200 None 2025-12-17 16:41:10,744 - lava_callback - INFO - Completed processing callback for node 69403a17575bc060ce1582fb INFO: 172.21.0.1:33870 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:41:11,370 - lava_callback - INFO - Received callback for node 69403a19575bc060ce15830f INFO: 172.21.0.1:33882 - "POST /node/69403a19575bc060ce15830f HTTP/1.0" 202 Accepted 2025-12-17 16:41:11,378 - lava_callback - INFO - Starting async job submit for node 69403a19575bc060ce15830f 2025-12-17 16:41:11,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:11,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403a19575bc060ce15830f HTTP/1.1" 200 None 2025-12-17 16:41:11,389 - lava_callback - DEBUG - Processing node 69403a19575bc060ce15830f: baseline-arm64-clabbe 2025-12-17 16:41:11,390 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403a19575bc060ce15830f 2025-12-17 16:41:11,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:11,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403a19575bc060ce15830f HTTP/1.1" 200 None 2025-12-17 16:41:11,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:11,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:41:11,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:11,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403a19575bc060ce15830f HTTP/1.1" 200 None 2025-12-17 16:41:11,426 - lava_callback - INFO - Completed processing callback for node 69403a19575bc060ce15830f INFO: 172.21.0.1:33894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33900 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:41:26,108 - lava_callback - INFO - Received callback for node 6940474c575bc060ce158cad INFO: 172.21.0.1:43448 - "POST /node/6940474c575bc060ce158cad HTTP/1.0" 202 Accepted 2025-12-17 16:41:26,115 - lava_callback - INFO - Starting async job submit for node 6940474c575bc060ce158cad 2025-12-17 16:41:26,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:26,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940474c575bc060ce158cad HTTP/1.1" 200 None 2025-12-17 16:41:26,163 - lava_callback - DEBUG - Processing node 6940474c575bc060ce158cad: baseline-nfs-arm64 INFO: 172.21.0.1:43460 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:41:26,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:41:27,107 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:41:27,108 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6940474c575bc060ce158cad/log.txt.gz 2025-12-17 16:41:27,109 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:41:27,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:41:27,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:41:27,277 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6940474c575bc060ce158cad/lava_callback.json.gz DEBUG: Setup failed for 6940474c575bc060ce158cad. Transitting job node result: incomplete -> fail 2025-12-17 16:41:27,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:27,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940474c575bc060ce158cad HTTP/1.1" 200 None 2025-12-17 16:41:27,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:27,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693bd221f552eae6fe2029cc HTTP/1.1" 200 None 2025-12-17 16:41:27,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:27,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940474c575bc060ce158cad HTTP/1.1" 200 None 2025-12-17 16:41:27,400 - lava_callback - INFO - Completed processing callback for node 6940474c575bc060ce158cad INFO: 172.21.0.1:56326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32852 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:41:47,855 - lava_callback - INFO - Received callback for node 6942dc94e43d2b76e9022c5f INFO: 172.21.0.1:32864 - "POST /node/6942dc94e43d2b76e9022c5f HTTP/1.0" 202 Accepted 2025-12-17 16:41:47,861 - lava_callback - INFO - Starting async job submit for node 6942dc94e43d2b76e9022c5f 2025-12-17 16:41:47,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:47,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dc94e43d2b76e9022c5f HTTP/1.1" 200 None 2025-12-17 16:41:47,892 - lava_callback - DEBUG - Processing node 6942dc94e43d2b76e9022c5f: baseline-arm 2025-12-17 16:41:48,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:41:48,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:41:48,266 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942dc94e43d2b76e9022c5f/log.txt.gz 2025-12-17 16:41:48,266 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:41:48,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:41:48,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:41:48,432 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942dc94e43d2b76e9022c5f/lava_callback.json.gz 2025-12-17 16:41:48,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:48,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dc94e43d2b76e9022c5f HTTP/1.1" 200 None 2025-12-17 16:41:48,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:48,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:41:48,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:41:48,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942dc94e43d2b76e9022c5f HTTP/1.1" 200 None 2025-12-17 16:41:48,533 - lava_callback - INFO - Completed processing callback for node 6942dc94e43d2b76e9022c5f INFO: 172.21.0.1:50884 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:42:04,407 - lava_callback - INFO - Received callback for node 6942da25e43d2b76e90222c1 INFO: 172.21.0.1:51550 - "POST /node/6942da25e43d2b76e90222c1 HTTP/1.0" 202 Accepted 2025-12-17 16:42:04,413 - lava_callback - INFO - Starting async job submit for node 6942da25e43d2b76e90222c1 2025-12-17 16:42:04,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:42:04,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da25e43d2b76e90222c1 HTTP/1.1" 200 None 2025-12-17 16:42:04,444 - lava_callback - DEBUG - Processing node 6942da25e43d2b76e90222c1: baseline-arm-broonie 2025-12-17 16:42:04,544 - lava_callback - INFO - Received callback for node 69404571575bc060ce158c21 INFO: 172.21.0.1:51554 - "POST /node/69404571575bc060ce158c21 HTTP/1.0" 202 Accepted 2025-12-17 16:42:04,584 - lava_callback - INFO - Starting async job submit for node 69404571575bc060ce158c21 2025-12-17 16:42:04,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:42:04,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:42:04,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404571575bc060ce158c21 HTTP/1.1" 200 None 2025-12-17 16:42:04,758 - lava_callback - DEBUG - Processing node 69404571575bc060ce158c21: baseline-nfs-arm64 2025-12-17 16:42:04,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:42:04,941 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-6942da25e43d2b76e90222c1/log.txt.gz 2025-12-17 16:42:04,951 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:42:05,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:42:05,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:42:05,402 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-6942da25e43d2b76e90222c1/lava_callback.json.gz 2025-12-17 16:42:05,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:42:05,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da25e43d2b76e90222c1 HTTP/1.1" 200 None 2025-12-17 16:42:05,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:42:05,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:42:05,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:42:05,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da25e43d2b76e90222c1 HTTP/1.1" 200 None 2025-12-17 16:42:05,701 - lava_callback - INFO - Completed processing callback for node 6942da25e43d2b76e90222c1 2025-12-17 16:42:06,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:42:06,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:42:06,196 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69404571575bc060ce158c21/log.txt.gz 2025-12-17 16:42:06,196 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:42:06,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:42:06,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:42:06,420 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69404571575bc060ce158c21/lava_callback.json.gz INFO: 172.21.0.1:51560 - "GET / HTTP/1.0" 200 OK DEBUG: Setup failed for 69404571575bc060ce158c21. Transitting job node result: incomplete -> fail 2025-12-17 16:42:06,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:42:06,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404571575bc060ce158c21 HTTP/1.1" 200 None 2025-12-17 16:42:06,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:42:06,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693bd221f552eae6fe2029cc HTTP/1.1" 200 None 2025-12-17 16:42:06,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:42:06,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69404571575bc060ce158c21 HTTP/1.1" 200 None 2025-12-17 16:42:06,587 - lava_callback - INFO - Completed processing callback for node 69404571575bc060ce158c21 INFO: 172.21.0.1:35130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53040 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:42:48,594 - lava_callback - INFO - Received callback for node 69403a87575bc060ce158374 INFO: 172.21.0.1:53052 - "POST /node/69403a87575bc060ce158374 HTTP/1.0" 202 Accepted 2025-12-17 16:42:48,599 - lava_callback - INFO - Starting async job submit for node 69403a87575bc060ce158374 2025-12-17 16:42:48,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:42:48,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403a87575bc060ce158374 HTTP/1.1" 200 None 2025-12-17 16:42:48,632 - lava_callback - DEBUG - Processing node 69403a87575bc060ce158374: ltp-syscalls 2025-12-17 16:42:48,633 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403a87575bc060ce158374 2025-12-17 16:42:48,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:42:48,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403a87575bc060ce158374 HTTP/1.1" 200 None 2025-12-17 16:42:48,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:42:48,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:42:48,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:42:48,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403a87575bc060ce158374 HTTP/1.1" 200 None 2025-12-17 16:42:48,681 - lava_callback - INFO - Completed processing callback for node 69403a87575bc060ce158374 INFO: 172.21.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36088 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:43:10,190 - lava_callback - INFO - Received callback for node 69403a8a575bc060ce15837d INFO: 172.21.0.1:43578 - "POST /node/69403a8a575bc060ce15837d HTTP/1.0" 202 Accepted 2025-12-17 16:43:10,196 - lava_callback - INFO - Starting async job submit for node 69403a8a575bc060ce15837d 2025-12-17 16:43:10,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:43:10,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403a8a575bc060ce15837d HTTP/1.1" 200 None 2025-12-17 16:43:10,221 - lava_callback - DEBUG - Processing node 69403a8a575bc060ce15837d: baseline-arm64-clabbe 2025-12-17 16:43:10,221 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403a8a575bc060ce15837d 2025-12-17 16:43:10,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:43:10,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403a8a575bc060ce15837d HTTP/1.1" 200 None 2025-12-17 16:43:10,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:43:10,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:43:10,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:43:10,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403a8a575bc060ce15837d HTTP/1.1" 200 None 2025-12-17 16:43:10,264 - lava_callback - INFO - Completed processing callback for node 69403a8a575bc060ce15837d 2025-12-17 16:43:10,850 - lava_callback - INFO - Received callback for node 69403a8d575bc060ce158382 INFO: 172.21.0.1:43594 - "POST /node/69403a8d575bc060ce158382 HTTP/1.0" 202 Accepted 2025-12-17 16:43:10,856 - lava_callback - INFO - Starting async job submit for node 69403a8d575bc060ce158382 2025-12-17 16:43:10,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:43:10,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403a8d575bc060ce158382 HTTP/1.1" 200 None 2025-12-17 16:43:10,889 - lava_callback - DEBUG - Processing node 69403a8d575bc060ce158382: baseline-arm64-clabbe 2025-12-17 16:43:10,889 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403a8d575bc060ce158382 2025-12-17 16:43:10,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:43:10,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403a8d575bc060ce158382 HTTP/1.1" 200 None 2025-12-17 16:43:10,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:43:10,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:43:10,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:43:10,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403a8d575bc060ce158382 HTTP/1.1" 200 None 2025-12-17 16:43:10,933 - lava_callback - INFO - Completed processing callback for node 69403a8d575bc060ce158382 INFO: 172.21.0.1:43602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53682 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:43:50,279 - lava_callback - INFO - Received callback for node 6942dd4ce43d2b76e9022d0e INFO: 172.21.0.1:43574 - "POST /node/6942dd4ce43d2b76e9022d0e HTTP/1.0" 202 Accepted 2025-12-17 16:43:50,286 - lava_callback - INFO - Starting async job submit for node 6942dd4ce43d2b76e9022d0e 2025-12-17 16:43:50,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:43:50,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dd4ce43d2b76e9022d0e HTTP/1.1" 200 None 2025-12-17 16:43:50,317 - lava_callback - DEBUG - Processing node 6942dd4ce43d2b76e9022d0e: baseline-arm 2025-12-17 16:43:50,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:43:50,757 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:43:50,758 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942dd4ce43d2b76e9022d0e/log.txt.gz 2025-12-17 16:43:50,758 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:43:50,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:43:50,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:43:50,926 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942dd4ce43d2b76e9022d0e/lava_callback.json.gz 2025-12-17 16:43:50,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:43:50,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dd4ce43d2b76e9022d0e HTTP/1.1" 200 None 2025-12-17 16:43:50,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:43:51,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:43:51,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:43:51,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942dd4ce43d2b76e9022d0e HTTP/1.1" 200 None 2025-12-17 16:43:51,038 - lava_callback - INFO - Completed processing callback for node 6942dd4ce43d2b76e9022d0e 2025-12-17 16:43:55,764 - lava_callback - INFO - Received callback for node 69403ac1575bc060ce158396 INFO: 172.21.0.1:43580 - "POST /node/69403ac1575bc060ce158396 HTTP/1.0" 202 Accepted 2025-12-17 16:43:55,770 - lava_callback - INFO - Starting async job submit for node 69403ac1575bc060ce158396 2025-12-17 16:43:55,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:43:55,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ac1575bc060ce158396 HTTP/1.1" 200 None 2025-12-17 16:43:55,782 - lava_callback - DEBUG - Processing node 69403ac1575bc060ce158396: baseline-arm64-baylibre 2025-12-17 16:43:55,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ac1575bc060ce158396 2025-12-17 16:43:55,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:43:55,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ac1575bc060ce158396 HTTP/1.1" 200 None 2025-12-17 16:43:55,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:43:55,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:43:55,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:43:55,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ac1575bc060ce158396 HTTP/1.1" 200 None 2025-12-17 16:43:55,812 - lava_callback - INFO - Completed processing callback for node 69403ac1575bc060ce158396 INFO: 172.21.0.1:43594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60948 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:44:15,764 - lava_callback - INFO - Received callback for node 69403ad5575bc060ce1583a8 INFO: 172.21.0.1:60954 - "POST /node/69403ad5575bc060ce1583a8 HTTP/1.0" 202 Accepted 2025-12-17 16:44:15,774 - lava_callback - INFO - Starting async job submit for node 69403ad5575bc060ce1583a8 2025-12-17 16:44:15,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:15,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ad5575bc060ce1583a8 HTTP/1.1" 200 None 2025-12-17 16:44:15,786 - lava_callback - DEBUG - Processing node 69403ad5575bc060ce1583a8: baseline-arm64-baylibre 2025-12-17 16:44:15,786 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ad5575bc060ce1583a8 2025-12-17 16:44:15,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:15,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ad5575bc060ce1583a8 HTTP/1.1" 200 None 2025-12-17 16:44:15,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:15,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:44:15,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:15,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ad5575bc060ce1583a8 HTTP/1.1" 200 None 2025-12-17 16:44:15,829 - lava_callback - INFO - Completed processing callback for node 69403ad5575bc060ce1583a8 2025-12-17 16:44:16,426 - lava_callback - INFO - Received callback for node 69403ad8575bc060ce1583b6 INFO: 172.21.0.1:60962 - "POST /node/69403ad8575bc060ce1583b6 HTTP/1.0" 202 Accepted 2025-12-17 16:44:16,432 - lava_callback - INFO - Starting async job submit for node 69403ad8575bc060ce1583b6 2025-12-17 16:44:16,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:16,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ad8575bc060ce1583b6 HTTP/1.1" 200 None 2025-12-17 16:44:16,441 - lava_callback - DEBUG - Processing node 69403ad8575bc060ce1583b6: baseline-arm64-baylibre 2025-12-17 16:44:16,441 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ad8575bc060ce1583b6 2025-12-17 16:44:16,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:16,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ad8575bc060ce1583b6 HTTP/1.1" 200 None 2025-12-17 16:44:16,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:16,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:44:16,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:16,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ad8575bc060ce1583b6 HTTP/1.1" 200 None 2025-12-17 16:44:16,473 - lava_callback - INFO - Completed processing callback for node 69403ad8575bc060ce1583b6 INFO: 172.21.0.1:60964 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:44:24,100 - lava_callback - INFO - Received callback for node 6942ddc6e43d2b76e9022d32 INFO: 172.21.0.1:47994 - "POST /node/6942ddc6e43d2b76e9022d32 HTTP/1.0" 202 Accepted 2025-12-17 16:44:24,107 - lava_callback - INFO - Starting async job submit for node 6942ddc6e43d2b76e9022d32 2025-12-17 16:44:24,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:24,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942ddc6e43d2b76e9022d32 HTTP/1.1" 200 None 2025-12-17 16:44:24,136 - lava_callback - DEBUG - Processing node 6942ddc6e43d2b76e9022d32: baseline-x86 2025-12-17 16:44:24,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:44:24,442 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:44:24,443 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-6942ddc6e43d2b76e9022d32/log.txt.gz 2025-12-17 16:44:24,444 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:44:24,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:44:24,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:44:24,602 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-6942ddc6e43d2b76e9022d32/lava_callback.json.gz 2025-12-17 16:44:24,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:24,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942ddc6e43d2b76e9022d32 HTTP/1.1" 200 None 2025-12-17 16:44:24,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:24,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d79ce43d2b76e9021b80 HTTP/1.1" 200 None 2025-12-17 16:44:24,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:24,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942ddc6e43d2b76e9022d32 HTTP/1.1" 200 None 2025-12-17 16:44:24,666 - lava_callback - INFO - Completed processing callback for node 6942ddc6e43d2b76e9022d32 INFO: 172.21.0.1:48010 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:44:30,764 - lava_callback - INFO - Received callback for node 6942ddc7e43d2b76e9022d42 INFO: 172.21.0.1:37826 - "POST /node/6942ddc7e43d2b76e9022d42 HTTP/1.0" 202 Accepted 2025-12-17 16:44:30,770 - lava_callback - INFO - Starting async job submit for node 6942ddc7e43d2b76e9022d42 2025-12-17 16:44:30,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:30,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942ddc7e43d2b76e9022d42 HTTP/1.1" 200 None 2025-12-17 16:44:30,804 - lava_callback - DEBUG - Processing node 6942ddc7e43d2b76e9022d42: baseline-x86 2025-12-17 16:44:30,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:44:31,122 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:44:31,123 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-6942ddc7e43d2b76e9022d42/log.txt.gz 2025-12-17 16:44:31,123 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:44:31,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:44:31,282 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:44:31,283 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-6942ddc7e43d2b76e9022d42/lava_callback.json.gz 2025-12-17 16:44:31,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:31,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942ddc7e43d2b76e9022d42 HTTP/1.1" 200 None 2025-12-17 16:44:31,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:31,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d79ce43d2b76e9021b80 HTTP/1.1" 200 None 2025-12-17 16:44:31,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:31,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942ddc7e43d2b76e9022d42 HTTP/1.1" 200 None 2025-12-17 16:44:31,367 - lava_callback - INFO - Completed processing callback for node 6942ddc7e43d2b76e9022d42 2025-12-17 16:44:35,790 - lava_callback - INFO - Received callback for node 69403adf575bc060ce1583bf INFO: 172.21.0.1:37828 - "POST /node/69403adf575bc060ce1583bf HTTP/1.0" 202 Accepted 2025-12-17 16:44:35,796 - lava_callback - INFO - Starting async job submit for node 69403adf575bc060ce1583bf 2025-12-17 16:44:35,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:35,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403adf575bc060ce1583bf HTTP/1.1" 200 None 2025-12-17 16:44:35,805 - lava_callback - DEBUG - Processing node 69403adf575bc060ce1583bf: baseline-arm64-baylibre 2025-12-17 16:44:35,805 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403adf575bc060ce1583bf 2025-12-17 16:44:35,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:35,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403adf575bc060ce1583bf HTTP/1.1" 200 None 2025-12-17 16:44:35,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:35,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:44:35,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:35,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403adf575bc060ce1583bf HTTP/1.1" 200 None 2025-12-17 16:44:35,846 - lava_callback - INFO - Completed processing callback for node 69403adf575bc060ce1583bf 2025-12-17 16:44:36,448 - lava_callback - INFO - Received callback for node 69403ae2575bc060ce1583c2 INFO: 172.21.0.1:37834 - "POST /node/69403ae2575bc060ce1583c2 HTTP/1.0" 202 Accepted 2025-12-17 16:44:36,454 - lava_callback - INFO - Starting async job submit for node 69403ae2575bc060ce1583c2 2025-12-17 16:44:36,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:36,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ae2575bc060ce1583c2 HTTP/1.1" 200 None 2025-12-17 16:44:36,463 - lava_callback - DEBUG - Processing node 69403ae2575bc060ce1583c2: baseline-arm64-baylibre 2025-12-17 16:44:36,463 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ae2575bc060ce1583c2 2025-12-17 16:44:36,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:36,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ae2575bc060ce1583c2 HTTP/1.1" 200 None 2025-12-17 16:44:36,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:36,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:44:36,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:36,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ae2575bc060ce1583c2 HTTP/1.1" 200 None 2025-12-17 16:44:36,502 - lava_callback - INFO - Completed processing callback for node 69403ae2575bc060ce1583c2 INFO: 172.21.0.1:37840 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:44:37,105 - lava_callback - INFO - Received callback for node 69403ae9575bc060ce1583c7 INFO: 172.21.0.1:37844 - "POST /node/69403ae9575bc060ce1583c7 HTTP/1.0" 202 Accepted 2025-12-17 16:44:37,112 - lava_callback - INFO - Starting async job submit for node 69403ae9575bc060ce1583c7 2025-12-17 16:44:37,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:37,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ae9575bc060ce1583c7 HTTP/1.1" 200 None 2025-12-17 16:44:37,123 - lava_callback - DEBUG - Processing node 69403ae9575bc060ce1583c7: baseline-arm64-baylibre 2025-12-17 16:44:37,123 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ae9575bc060ce1583c7 2025-12-17 16:44:37,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:37,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ae9575bc060ce1583c7 HTTP/1.1" 200 None 2025-12-17 16:44:37,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:37,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:44:37,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:37,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ae9575bc060ce1583c7 HTTP/1.1" 200 None 2025-12-17 16:44:37,160 - lava_callback - INFO - Completed processing callback for node 69403ae9575bc060ce1583c7 2025-12-17 16:44:37,745 - lava_callback - INFO - Received callback for node 69403aec575bc060ce1583d0 INFO: 172.21.0.1:37848 - "POST /node/69403aec575bc060ce1583d0 HTTP/1.0" 202 Accepted 2025-12-17 16:44:37,753 - lava_callback - INFO - Starting async job submit for node 69403aec575bc060ce1583d0 2025-12-17 16:44:37,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:37,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403aec575bc060ce1583d0 HTTP/1.1" 200 None 2025-12-17 16:44:37,765 - lava_callback - DEBUG - Processing node 69403aec575bc060ce1583d0: baseline-arm64-baylibre 2025-12-17 16:44:37,766 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403aec575bc060ce1583d0 2025-12-17 16:44:37,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:37,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403aec575bc060ce1583d0 HTTP/1.1" 200 None 2025-12-17 16:44:37,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:37,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad8edfb52d1dfff36c71b HTTP/1.1" 200 None 2025-12-17 16:44:37,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:37,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403aec575bc060ce1583d0 HTTP/1.1" 200 None 2025-12-17 16:44:37,819 - lava_callback - INFO - Completed processing callback for node 69403aec575bc060ce1583d0 2025-12-17 16:44:38,426 - lava_callback - INFO - Received callback for node 69403aef575bc060ce1583d3 INFO: 172.21.0.1:37854 - "POST /node/69403aef575bc060ce1583d3 HTTP/1.0" 202 Accepted 2025-12-17 16:44:38,434 - lava_callback - INFO - Starting async job submit for node 69403aef575bc060ce1583d3 2025-12-17 16:44:38,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:38,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403aef575bc060ce1583d3 HTTP/1.1" 200 None 2025-12-17 16:44:38,443 - lava_callback - DEBUG - Processing node 69403aef575bc060ce1583d3: baseline-arm64-baylibre 2025-12-17 16:44:38,443 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403aef575bc060ce1583d3 2025-12-17 16:44:38,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:38,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403aef575bc060ce1583d3 HTTP/1.1" 200 None 2025-12-17 16:44:38,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:38,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:44:38,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:38,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403aef575bc060ce1583d3 HTTP/1.1" 200 None 2025-12-17 16:44:38,485 - lava_callback - INFO - Completed processing callback for node 69403aef575bc060ce1583d3 2025-12-17 16:44:39,103 - lava_callback - INFO - Received callback for node 69403af2575bc060ce1583d6 INFO: 172.21.0.1:37866 - "POST /node/69403af2575bc060ce1583d6 HTTP/1.0" 202 Accepted 2025-12-17 16:44:39,111 - lava_callback - INFO - Starting async job submit for node 69403af2575bc060ce1583d6 2025-12-17 16:44:39,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:39,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403af2575bc060ce1583d6 HTTP/1.1" 200 None 2025-12-17 16:44:39,120 - lava_callback - DEBUG - Processing node 69403af2575bc060ce1583d6: baseline-arm64-baylibre 2025-12-17 16:44:39,120 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403af2575bc060ce1583d6 2025-12-17 16:44:39,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:39,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403af2575bc060ce1583d6 HTTP/1.1" 200 None 2025-12-17 16:44:39,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:39,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:44:39,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:39,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403af2575bc060ce1583d6 HTTP/1.1" 200 None 2025-12-17 16:44:39,151 - lava_callback - INFO - Completed processing callback for node 69403af2575bc060ce1583d6 INFO: 172.21.0.1:36896 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:44:55,828 - lava_callback - INFO - Received callback for node 69403afd575bc060ce1583db INFO: 172.21.0.1:35020 - "POST /node/69403afd575bc060ce1583db HTTP/1.0" 202 Accepted 2025-12-17 16:44:55,833 - lava_callback - INFO - Starting async job submit for node 69403afd575bc060ce1583db 2025-12-17 16:44:55,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:55,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403afd575bc060ce1583db HTTP/1.1" 200 None 2025-12-17 16:44:55,843 - lava_callback - DEBUG - Processing node 69403afd575bc060ce1583db: baseline-arm64-baylibre 2025-12-17 16:44:55,843 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403afd575bc060ce1583db 2025-12-17 16:44:55,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:55,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403afd575bc060ce1583db HTTP/1.1" 200 None 2025-12-17 16:44:55,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:55,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:44:55,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:55,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403afd575bc060ce1583db HTTP/1.1" 200 None 2025-12-17 16:44:55,881 - lava_callback - INFO - Completed processing callback for node 69403afd575bc060ce1583db 2025-12-17 16:44:56,474 - lava_callback - INFO - Received callback for node 69403b00575bc060ce1583e3 INFO: 172.21.0.1:35034 - "POST /node/69403b00575bc060ce1583e3 HTTP/1.0" 202 Accepted 2025-12-17 16:44:56,480 - lava_callback - INFO - Starting async job submit for node 69403b00575bc060ce1583e3 2025-12-17 16:44:56,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:56,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b00575bc060ce1583e3 HTTP/1.1" 200 None 2025-12-17 16:44:56,490 - lava_callback - DEBUG - Processing node 69403b00575bc060ce1583e3: baseline-arm64-baylibre 2025-12-17 16:44:56,490 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b00575bc060ce1583e3 2025-12-17 16:44:56,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:56,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b00575bc060ce1583e3 HTTP/1.1" 200 None 2025-12-17 16:44:56,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:35038 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:44:56,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:44:56,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:56,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b00575bc060ce1583e3 HTTP/1.1" 200 None 2025-12-17 16:44:56,526 - lava_callback - INFO - Completed processing callback for node 69403b00575bc060ce1583e3 2025-12-17 16:44:57,131 - lava_callback - INFO - Received callback for node 69403b03575bc060ce1583e6 INFO: 172.21.0.1:35044 - "POST /node/69403b03575bc060ce1583e6 HTTP/1.0" 202 Accepted 2025-12-17 16:44:57,141 - lava_callback - INFO - Starting async job submit for node 69403b03575bc060ce1583e6 2025-12-17 16:44:57,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:57,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b03575bc060ce1583e6 HTTP/1.1" 200 None 2025-12-17 16:44:57,150 - lava_callback - DEBUG - Processing node 69403b03575bc060ce1583e6: baseline-arm64-baylibre 2025-12-17 16:44:57,150 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b03575bc060ce1583e6 2025-12-17 16:44:57,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:57,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b03575bc060ce1583e6 HTTP/1.1" 200 None 2025-12-17 16:44:57,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:57,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:44:57,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:44:57,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b03575bc060ce1583e6 HTTP/1.1" 200 None 2025-12-17 16:44:57,194 - lava_callback - INFO - Completed processing callback for node 69403b03575bc060ce1583e6 2025-12-17 16:45:04,253 - lava_callback - INFO - Received callback for node 694036eb575bc060ce1579a9 INFO: 172.21.0.1:45868 - "POST /node/694036eb575bc060ce1579a9 HTTP/1.0" 202 Accepted 2025-12-17 16:45:04,301 - lava_callback - INFO - Starting async job submit for node 694036eb575bc060ce1579a9 2025-12-17 16:45:04,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:04,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036eb575bc060ce1579a9 HTTP/1.1" 200 None 2025-12-17 16:45:04,334 - lava_callback - DEBUG - Processing node 694036eb575bc060ce1579a9: baseline-arm-pengutronix 2025-12-17 16:45:04,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:45:04,552 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:45:04,553 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-694036eb575bc060ce1579a9/log.txt.gz 2025-12-17 16:45:04,553 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:45:04,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:45:04,698 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:45:04,698 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-694036eb575bc060ce1579a9/lava_callback.json.gz DEBUG: Setup failed for 694036eb575bc060ce1579a9. Transitting job node result: incomplete -> fail 2025-12-17 16:45:04,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:04,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694036eb575bc060ce1579a9 HTTP/1.1" 200 None 2025-12-17 16:45:04,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:04,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694031ed575bc060ce156c65 HTTP/1.1" 200 None 2025-12-17 16:45:04,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:04,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694036eb575bc060ce1579a9 HTTP/1.1" 200 None 2025-12-17 16:45:04,786 - lava_callback - INFO - Completed processing callback for node 694036eb575bc060ce1579a9 INFO: 172.21.0.1:45882 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:45:07,230 - lava_callback - INFO - Received callback for node 6942da22e43d2b76e90222a4 INFO: 172.21.0.1:45884 - "POST /node/6942da22e43d2b76e90222a4 HTTP/1.0" 202 Accepted 2025-12-17 16:45:07,237 - lava_callback - INFO - Starting async job submit for node 6942da22e43d2b76e90222a4 2025-12-17 16:45:07,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:07,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da22e43d2b76e90222a4 HTTP/1.1" 200 None 2025-12-17 16:45:07,299 - lava_callback - DEBUG - Processing node 6942da22e43d2b76e90222a4: baseline-arm 2025-12-17 16:45:07,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:45:08,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:45:08,021 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942da22e43d2b76e90222a4/log.txt.gz 2025-12-17 16:45:08,021 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:45:08,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:45:08,191 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:45:08,192 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942da22e43d2b76e90222a4/lava_callback.json.gz DEBUG: Setup failed for 6942da22e43d2b76e90222a4. Transitting job node result: incomplete -> fail 2025-12-17 16:45:08,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:08,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942da22e43d2b76e90222a4 HTTP/1.1" 200 None 2025-12-17 16:45:08,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:08,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 16:45:08,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:08,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942da22e43d2b76e90222a4 HTTP/1.1" 200 None 2025-12-17 16:45:08,344 - lava_callback - INFO - Completed processing callback for node 6942da22e43d2b76e90222a4 INFO: 172.21.0.1:58188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58202 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:45:15,847 - lava_callback - INFO - Received callback for node 69403b11575bc060ce1583f5 INFO: 172.21.0.1:58218 - "POST /node/69403b11575bc060ce1583f5 HTTP/1.0" 202 Accepted 2025-12-17 16:45:15,853 - lava_callback - INFO - Starting async job submit for node 69403b11575bc060ce1583f5 2025-12-17 16:45:15,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:15,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b11575bc060ce1583f5 HTTP/1.1" 200 None 2025-12-17 16:45:15,862 - lava_callback - DEBUG - Processing node 69403b11575bc060ce1583f5: baseline-arm64-baylibre 2025-12-17 16:45:15,862 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b11575bc060ce1583f5 2025-12-17 16:45:15,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:15,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b11575bc060ce1583f5 HTTP/1.1" 200 None 2025-12-17 16:45:15,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:15,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:45:15,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:15,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b11575bc060ce1583f5 HTTP/1.1" 200 None 2025-12-17 16:45:15,896 - lava_callback - INFO - Completed processing callback for node 69403b11575bc060ce1583f5 INFO: 172.21.0.1:58230 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:45:16,514 - lava_callback - INFO - Received callback for node 69403b13575bc060ce158401 INFO: 172.21.0.1:58232 - "POST /node/69403b13575bc060ce158401 HTTP/1.0" 202 Accepted 2025-12-17 16:45:16,520 - lava_callback - INFO - Starting async job submit for node 69403b13575bc060ce158401 2025-12-17 16:45:16,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:16,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b13575bc060ce158401 HTTP/1.1" 200 None 2025-12-17 16:45:16,529 - lava_callback - DEBUG - Processing node 69403b13575bc060ce158401: baseline-arm64-baylibre 2025-12-17 16:45:16,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b13575bc060ce158401 2025-12-17 16:45:16,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:16,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b13575bc060ce158401 HTTP/1.1" 200 None 2025-12-17 16:45:16,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:16,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:45:16,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:16,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b13575bc060ce158401 HTTP/1.1" 200 None 2025-12-17 16:45:16,564 - lava_callback - INFO - Completed processing callback for node 69403b13575bc060ce158401 2025-12-17 16:45:17,196 - lava_callback - INFO - Received callback for node 69403b15575bc060ce158406 INFO: 172.21.0.1:58238 - "POST /node/69403b15575bc060ce158406 HTTP/1.0" 202 Accepted 2025-12-17 16:45:17,203 - lava_callback - INFO - Starting async job submit for node 69403b15575bc060ce158406 2025-12-17 16:45:17,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:17,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b15575bc060ce158406 HTTP/1.1" 200 None 2025-12-17 16:45:17,212 - lava_callback - DEBUG - Processing node 69403b15575bc060ce158406: baseline-arm64-baylibre 2025-12-17 16:45:17,212 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b15575bc060ce158406 2025-12-17 16:45:17,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:17,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b15575bc060ce158406 HTTP/1.1" 200 None 2025-12-17 16:45:17,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:17,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:45:17,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:17,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b15575bc060ce158406 HTTP/1.1" 200 None 2025-12-17 16:45:17,256 - lava_callback - INFO - Completed processing callback for node 69403b15575bc060ce158406 2025-12-17 16:45:17,873 - lava_callback - INFO - Received callback for node 69403b17575bc060ce15840b INFO: 172.21.0.1:58240 - "POST /node/69403b17575bc060ce15840b HTTP/1.0" 202 Accepted 2025-12-17 16:45:17,881 - lava_callback - INFO - Starting async job submit for node 69403b17575bc060ce15840b 2025-12-17 16:45:17,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:17,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b17575bc060ce15840b HTTP/1.1" 200 None 2025-12-17 16:45:17,900 - lava_callback - DEBUG - Processing node 69403b17575bc060ce15840b: baseline-arm64-baylibre 2025-12-17 16:45:17,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b17575bc060ce15840b 2025-12-17 16:45:17,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:17,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b17575bc060ce15840b HTTP/1.1" 200 None 2025-12-17 16:45:17,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:17,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:45:17,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:17,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b17575bc060ce15840b HTTP/1.1" 200 None 2025-12-17 16:45:17,936 - lava_callback - INFO - Completed processing callback for node 69403b17575bc060ce15840b 2025-12-17 16:45:18,535 - lava_callback - INFO - Received callback for node 69403b19575bc060ce15840e INFO: 172.21.0.1:58242 - "POST /node/69403b19575bc060ce15840e HTTP/1.0" 202 Accepted 2025-12-17 16:45:18,543 - lava_callback - INFO - Starting async job submit for node 69403b19575bc060ce15840e 2025-12-17 16:45:18,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:18,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b19575bc060ce15840e HTTP/1.1" 200 None 2025-12-17 16:45:18,553 - lava_callback - DEBUG - Processing node 69403b19575bc060ce15840e: baseline-arm64-baylibre 2025-12-17 16:45:18,553 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b19575bc060ce15840e 2025-12-17 16:45:18,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:18,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b19575bc060ce15840e HTTP/1.1" 200 None 2025-12-17 16:45:18,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:18,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:45:18,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:18,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b19575bc060ce15840e HTTP/1.1" 200 None 2025-12-17 16:45:18,597 - lava_callback - INFO - Completed processing callback for node 69403b19575bc060ce15840e INFO: 172.21.0.1:58112 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:45:30,347 - lava_callback - INFO - Received callback for node 69403b1e575bc060ce158414 INFO: 172.21.0.1:40630 - "POST /node/69403b1e575bc060ce158414 HTTP/1.0" 202 Accepted 2025-12-17 16:45:30,352 - lava_callback - INFO - Starting async job submit for node 69403b1e575bc060ce158414 2025-12-17 16:45:30,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:30,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b1e575bc060ce158414 HTTP/1.1" 200 None 2025-12-17 16:45:30,362 - lava_callback - DEBUG - Processing node 69403b1e575bc060ce158414: baseline-arm64-clabbe 2025-12-17 16:45:30,362 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b1e575bc060ce158414 2025-12-17 16:45:30,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:30,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b1e575bc060ce158414 HTTP/1.1" 200 None 2025-12-17 16:45:30,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:30,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:45:30,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:30,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b1e575bc060ce158414 HTTP/1.1" 200 None 2025-12-17 16:45:30,393 - lava_callback - INFO - Completed processing callback for node 69403b1e575bc060ce158414 2025-12-17 16:45:35,869 - lava_callback - INFO - Received callback for node 69403b1c575bc060ce158411 INFO: 172.21.0.1:40644 - "POST /node/69403b1c575bc060ce158411 HTTP/1.0" 202 Accepted 2025-12-17 16:45:35,878 - lava_callback - INFO - Starting async job submit for node 69403b1c575bc060ce158411 2025-12-17 16:45:35,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:35,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b1c575bc060ce158411 HTTP/1.1" 200 None 2025-12-17 16:45:35,891 - lava_callback - DEBUG - Processing node 69403b1c575bc060ce158411: baseline-arm64-baylibre 2025-12-17 16:45:35,892 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b1c575bc060ce158411 2025-12-17 16:45:35,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:35,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b1c575bc060ce158411 HTTP/1.1" 200 None 2025-12-17 16:45:35,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:35,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:45:35,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:35,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b1c575bc060ce158411 HTTP/1.1" 200 None 2025-12-17 16:45:35,954 - lava_callback - INFO - Completed processing callback for node 69403b1c575bc060ce158411 INFO: 172.21.0.1:40660 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:45:36,548 - lava_callback - INFO - Received callback for node 69403b25575bc060ce158421 INFO: 172.21.0.1:40670 - "POST /node/69403b25575bc060ce158421 HTTP/1.0" 202 Accepted 2025-12-17 16:45:36,555 - lava_callback - INFO - Starting async job submit for node 69403b25575bc060ce158421 2025-12-17 16:45:36,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:36,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b25575bc060ce158421 HTTP/1.1" 200 None 2025-12-17 16:45:36,565 - lava_callback - DEBUG - Processing node 69403b25575bc060ce158421: baseline-arm64-baylibre 2025-12-17 16:45:36,566 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b25575bc060ce158421 2025-12-17 16:45:36,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:36,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b25575bc060ce158421 HTTP/1.1" 200 None 2025-12-17 16:45:36,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:36,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:45:36,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:36,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b25575bc060ce158421 HTTP/1.1" 200 None 2025-12-17 16:45:36,608 - lava_callback - INFO - Completed processing callback for node 69403b25575bc060ce158421 2025-12-17 16:45:37,203 - lava_callback - INFO - Received callback for node 69403b27575bc060ce158429 INFO: 172.21.0.1:40684 - "POST /node/69403b27575bc060ce158429 HTTP/1.0" 202 Accepted 2025-12-17 16:45:37,215 - lava_callback - INFO - Starting async job submit for node 69403b27575bc060ce158429 2025-12-17 16:45:37,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:37,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b27575bc060ce158429 HTTP/1.1" 200 None 2025-12-17 16:45:37,236 - lava_callback - DEBUG - Processing node 69403b27575bc060ce158429: baseline-arm64-baylibre 2025-12-17 16:45:37,236 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b27575bc060ce158429 2025-12-17 16:45:37,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:37,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b27575bc060ce158429 HTTP/1.1" 200 None 2025-12-17 16:45:37,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:37,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:45:37,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:37,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b27575bc060ce158429 HTTP/1.1" 200 None 2025-12-17 16:45:37,282 - lava_callback - INFO - Completed processing callback for node 69403b27575bc060ce158429 2025-12-17 16:45:37,880 - lava_callback - INFO - Received callback for node 69403b29575bc060ce15842e INFO: 172.21.0.1:40686 - "POST /node/69403b29575bc060ce15842e HTTP/1.0" 202 Accepted 2025-12-17 16:45:37,889 - lava_callback - INFO - Starting async job submit for node 69403b29575bc060ce15842e 2025-12-17 16:45:37,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:37,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b29575bc060ce15842e HTTP/1.1" 200 None 2025-12-17 16:45:37,904 - lava_callback - DEBUG - Processing node 69403b29575bc060ce15842e: baseline-arm64-baylibre 2025-12-17 16:45:37,904 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b29575bc060ce15842e 2025-12-17 16:45:37,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:37,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b29575bc060ce15842e HTTP/1.1" 200 None 2025-12-17 16:45:37,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:37,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:45:37,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:37,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b29575bc060ce15842e HTTP/1.1" 200 None 2025-12-17 16:45:37,954 - lava_callback - INFO - Completed processing callback for node 69403b29575bc060ce15842e INFO: 172.21.0.1:47812 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:45:50,304 - lava_callback - INFO - Received callback for node 69403b2c575bc060ce158435 INFO: 172.21.0.1:55106 - "POST /node/69403b2c575bc060ce158435 HTTP/1.0" 202 Accepted 2025-12-17 16:45:50,311 - lava_callback - INFO - Starting async job submit for node 69403b2c575bc060ce158435 2025-12-17 16:45:50,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:50,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b2c575bc060ce158435 HTTP/1.1" 200 None 2025-12-17 16:45:50,322 - lava_callback - DEBUG - Processing node 69403b2c575bc060ce158435: baseline-arm64-clabbe 2025-12-17 16:45:50,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b2c575bc060ce158435 2025-12-17 16:45:50,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:50,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b2c575bc060ce158435 HTTP/1.1" 200 None 2025-12-17 16:45:50,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:50,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:45:50,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:50,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b2c575bc060ce158435 HTTP/1.1" 200 None 2025-12-17 16:45:50,362 - lava_callback - INFO - Completed processing callback for node 69403b2c575bc060ce158435 2025-12-17 16:45:50,981 - lava_callback - INFO - Received callback for node 69403b2d575bc060ce158438 INFO: 172.21.0.1:55120 - "POST /node/69403b2d575bc060ce158438 HTTP/1.0" 202 Accepted 2025-12-17 16:45:50,992 - lava_callback - INFO - Starting async job submit for node 69403b2d575bc060ce158438 2025-12-17 16:45:50,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:51,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b2d575bc060ce158438 HTTP/1.1" 200 None 2025-12-17 16:45:51,014 - lava_callback - DEBUG - Processing node 69403b2d575bc060ce158438: baseline-arm64-clabbe 2025-12-17 16:45:51,014 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b2d575bc060ce158438 2025-12-17 16:45:51,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:55,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69403b2d575bc060ce158438'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 16:45:55,030 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -5] No address associated with hostname)")': /latest/node/69403b2d575bc060ce158438 2025-12-17 16:45:55,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 16:45:55,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b2d575bc060ce158438 HTTP/1.1" 200 None 2025-12-17 16:45:55,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:55,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:45:55,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:55,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b2d575bc060ce158438 HTTP/1.1" 200 None 2025-12-17 16:45:55,063 - lava_callback - INFO - Completed processing callback for node 69403b2d575bc060ce158438 2025-12-17 16:45:55,838 - lava_callback - INFO - Received callback for node 69403b3e575bc060ce158451 INFO: 172.21.0.1:55128 - "POST /node/69403b3e575bc060ce158451 HTTP/1.0" 202 Accepted 2025-12-17 16:45:55,848 - lava_callback - INFO - Starting async job submit for node 69403b3e575bc060ce158451 2025-12-17 16:45:55,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:55,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b3e575bc060ce158451 HTTP/1.1" 200 None 2025-12-17 16:45:55,861 - lava_callback - DEBUG - Processing node 69403b3e575bc060ce158451: baseline-arm64-baylibre 2025-12-17 16:45:55,861 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b3e575bc060ce158451 2025-12-17 16:45:55,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:55,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b3e575bc060ce158451 HTTP/1.1" 200 None 2025-12-17 16:45:55,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:55,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:45:55,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:55,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b3e575bc060ce158451 HTTP/1.1" 200 None 2025-12-17 16:45:55,900 - lava_callback - INFO - Completed processing callback for node 69403b3e575bc060ce158451 INFO: 172.21.0.1:55140 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:45:56,536 - lava_callback - INFO - Received callback for node 69403b39575bc060ce158441 INFO: 172.21.0.1:55150 - "POST /node/69403b39575bc060ce158441 HTTP/1.0" 202 Accepted 2025-12-17 16:45:56,544 - lava_callback - INFO - Starting async job submit for node 69403b39575bc060ce158441 2025-12-17 16:45:56,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:56,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b39575bc060ce158441 HTTP/1.1" 200 None 2025-12-17 16:45:56,554 - lava_callback - DEBUG - Processing node 69403b39575bc060ce158441: baseline-arm64-baylibre 2025-12-17 16:45:56,554 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b39575bc060ce158441 2025-12-17 16:45:56,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:56,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b39575bc060ce158441 HTTP/1.1" 200 None 2025-12-17 16:45:56,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:56,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:45:56,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:56,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b39575bc060ce158441 HTTP/1.1" 200 None 2025-12-17 16:45:56,594 - lava_callback - INFO - Completed processing callback for node 69403b39575bc060ce158441 2025-12-17 16:45:57,228 - lava_callback - INFO - Received callback for node 69403b3c575bc060ce15844e INFO: 172.21.0.1:55152 - "POST /node/69403b3c575bc060ce15844e HTTP/1.0" 202 Accepted 2025-12-17 16:45:57,236 - lava_callback - INFO - Starting async job submit for node 69403b3c575bc060ce15844e 2025-12-17 16:45:57,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:57,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b3c575bc060ce15844e HTTP/1.1" 200 None 2025-12-17 16:45:57,249 - lava_callback - DEBUG - Processing node 69403b3c575bc060ce15844e: baseline-arm64-baylibre 2025-12-17 16:45:57,250 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b3c575bc060ce15844e 2025-12-17 16:45:57,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:57,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b3c575bc060ce15844e HTTP/1.1" 200 None 2025-12-17 16:45:57,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:57,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:45:57,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:57,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b3c575bc060ce15844e HTTP/1.1" 200 None 2025-12-17 16:45:57,298 - lava_callback - INFO - Completed processing callback for node 69403b3c575bc060ce15844e 2025-12-17 16:45:57,892 - lava_callback - INFO - Received callback for node 69403b42575bc060ce158459 INFO: 172.21.0.1:55168 - "POST /node/69403b42575bc060ce158459 HTTP/1.0" 202 Accepted 2025-12-17 16:45:57,901 - lava_callback - INFO - Starting async job submit for node 69403b42575bc060ce158459 2025-12-17 16:45:57,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:57,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b42575bc060ce158459 HTTP/1.1" 200 None 2025-12-17 16:45:57,970 - lava_callback - DEBUG - Processing node 69403b42575bc060ce158459: baseline-arm64-baylibre 2025-12-17 16:45:57,971 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b42575bc060ce158459 2025-12-17 16:45:57,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:57,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b42575bc060ce158459 HTTP/1.1" 200 None 2025-12-17 16:45:57,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:58,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:45:58,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:45:58,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b42575bc060ce158459 HTTP/1.1" 200 None 2025-12-17 16:45:58,050 - lava_callback - INFO - Completed processing callback for node 69403b42575bc060ce158459 INFO: 172.21.0.1:38684 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:46:10,320 - lava_callback - INFO - Received callback for node 69403b46575bc060ce15845f INFO: 172.21.0.1:55260 - "POST /node/69403b46575bc060ce15845f HTTP/1.0" 202 Accepted 2025-12-17 16:46:10,327 - lava_callback - INFO - Starting async job submit for node 69403b46575bc060ce15845f 2025-12-17 16:46:10,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:10,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b46575bc060ce15845f HTTP/1.1" 200 None 2025-12-17 16:46:10,337 - lava_callback - DEBUG - Processing node 69403b46575bc060ce15845f: baseline-arm64-clabbe 2025-12-17 16:46:10,337 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b46575bc060ce15845f 2025-12-17 16:46:10,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:10,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b46575bc060ce15845f HTTP/1.1" 200 None 2025-12-17 16:46:10,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:10,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:46:10,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:10,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b46575bc060ce15845f HTTP/1.1" 200 None 2025-12-17 16:46:10,370 - lava_callback - INFO - Completed processing callback for node 69403b46575bc060ce15845f INFO: 172.21.0.1:55262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55270 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:46:15,859 - lava_callback - INFO - Received callback for node 69403b44575bc060ce15845c INFO: 172.21.0.1:55278 - "POST /node/69403b44575bc060ce15845c HTTP/1.0" 202 Accepted 2025-12-17 16:46:15,865 - lava_callback - INFO - Starting async job submit for node 69403b44575bc060ce15845c 2025-12-17 16:46:15,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:15,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b44575bc060ce15845c HTTP/1.1" 200 None 2025-12-17 16:46:15,875 - lava_callback - DEBUG - Processing node 69403b44575bc060ce15845c: baseline-arm64-baylibre 2025-12-17 16:46:15,875 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b44575bc060ce15845c 2025-12-17 16:46:15,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:15,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b44575bc060ce15845c HTTP/1.1" 200 None 2025-12-17 16:46:15,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:15,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:46:15,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:15,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b44575bc060ce15845c HTTP/1.1" 200 None 2025-12-17 16:46:15,912 - lava_callback - INFO - Completed processing callback for node 69403b44575bc060ce15845c INFO: 172.21.0.1:55284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36322 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:46:30,370 - lava_callback - INFO - Received callback for node 69403b52575bc060ce158464 INFO: 172.21.0.1:50550 - "POST /node/69403b52575bc060ce158464 HTTP/1.0" 202 Accepted 2025-12-17 16:46:30,376 - lava_callback - INFO - Starting async job submit for node 69403b52575bc060ce158464 2025-12-17 16:46:30,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:30,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b52575bc060ce158464 HTTP/1.1" 200 None 2025-12-17 16:46:30,392 - lava_callback - DEBUG - Processing node 69403b52575bc060ce158464: baseline-arm64-clabbe 2025-12-17 16:46:30,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b52575bc060ce158464 2025-12-17 16:46:30,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:30,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b52575bc060ce158464 HTTP/1.1" 200 None 2025-12-17 16:46:30,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:30,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:46:30,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:30,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b52575bc060ce158464 HTTP/1.1" 200 None 2025-12-17 16:46:30,431 - lava_callback - INFO - Completed processing callback for node 69403b52575bc060ce158464 2025-12-17 16:46:31,034 - lava_callback - INFO - Received callback for node 69403b54575bc060ce15846d INFO: 172.21.0.1:50554 - "POST /node/69403b54575bc060ce15846d HTTP/1.0" 202 Accepted 2025-12-17 16:46:31,043 - lava_callback - INFO - Starting async job submit for node 69403b54575bc060ce15846d 2025-12-17 16:46:31,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:31,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b54575bc060ce15846d HTTP/1.1" 200 None 2025-12-17 16:46:31,054 - lava_callback - DEBUG - Processing node 69403b54575bc060ce15846d: baseline-arm64-clabbe 2025-12-17 16:46:31,054 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b54575bc060ce15846d 2025-12-17 16:46:31,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:31,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b54575bc060ce15846d HTTP/1.1" 200 None 2025-12-17 16:46:31,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:31,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:46:31,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:31,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b54575bc060ce15846d HTTP/1.1" 200 None 2025-12-17 16:46:31,113 - lava_callback - INFO - Completed processing callback for node 69403b54575bc060ce15846d 2025-12-17 16:46:31,735 - lava_callback - INFO - Received callback for node 69403b56575bc060ce158470 INFO: 172.21.0.1:50570 - "POST /node/69403b56575bc060ce158470 HTTP/1.0" 202 Accepted 2025-12-17 16:46:31,742 - lava_callback - INFO - Starting async job submit for node 69403b56575bc060ce158470 2025-12-17 16:46:31,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:31,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b56575bc060ce158470 HTTP/1.1" 200 None 2025-12-17 16:46:31,752 - lava_callback - DEBUG - Processing node 69403b56575bc060ce158470: baseline-arm64-clabbe 2025-12-17 16:46:31,752 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b56575bc060ce158470 2025-12-17 16:46:31,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:31,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b56575bc060ce158470 HTTP/1.1" 200 None 2025-12-17 16:46:31,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:31,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:46:31,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:31,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b56575bc060ce158470 HTTP/1.1" 200 None 2025-12-17 16:46:31,785 - lava_callback - INFO - Completed processing callback for node 69403b56575bc060ce158470 2025-12-17 16:46:32,409 - lava_callback - INFO - Received callback for node 69403b58575bc060ce158473 INFO: 172.21.0.1:50572 - "POST /node/69403b58575bc060ce158473 HTTP/1.0" 202 Accepted 2025-12-17 16:46:32,420 - lava_callback - INFO - Starting async job submit for node 69403b58575bc060ce158473 2025-12-17 16:46:32,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:32,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b58575bc060ce158473 HTTP/1.1" 200 None 2025-12-17 16:46:32,433 - lava_callback - DEBUG - Processing node 69403b58575bc060ce158473: baseline-arm64-clabbe 2025-12-17 16:46:32,433 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b58575bc060ce158473 2025-12-17 16:46:32,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:32,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b58575bc060ce158473 HTTP/1.1" 200 None 2025-12-17 16:46:32,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:32,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:46:32,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:32,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b58575bc060ce158473 HTTP/1.1" 200 None 2025-12-17 16:46:32,474 - lava_callback - INFO - Completed processing callback for node 69403b58575bc060ce158473 INFO: 172.21.0.1:50582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39434 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:46:50,359 - lava_callback - INFO - Received callback for node 69403b66575bc060ce15847f INFO: 172.21.0.1:46504 - "POST /node/69403b66575bc060ce15847f HTTP/1.0" 202 Accepted 2025-12-17 16:46:50,368 - lava_callback - INFO - Starting async job submit for node 69403b66575bc060ce15847f 2025-12-17 16:46:50,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:50,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b66575bc060ce15847f HTTP/1.1" 200 None 2025-12-17 16:46:50,382 - lava_callback - DEBUG - Processing node 69403b66575bc060ce15847f: baseline-arm64-clabbe 2025-12-17 16:46:50,382 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b66575bc060ce15847f 2025-12-17 16:46:50,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:50,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b66575bc060ce15847f HTTP/1.1" 200 None 2025-12-17 16:46:50,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:50,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:46:50,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:50,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b66575bc060ce15847f HTTP/1.1" 200 None 2025-12-17 16:46:50,448 - lava_callback - INFO - Completed processing callback for node 69403b66575bc060ce15847f 2025-12-17 16:46:51,018 - lava_callback - INFO - Received callback for node 69403b68575bc060ce158486 INFO: 172.21.0.1:46520 - "POST /node/69403b68575bc060ce158486 HTTP/1.0" 202 Accepted 2025-12-17 16:46:51,027 - lava_callback - INFO - Starting async job submit for node 69403b68575bc060ce158486 2025-12-17 16:46:51,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:51,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b68575bc060ce158486 HTTP/1.1" 200 None 2025-12-17 16:46:51,048 - lava_callback - DEBUG - Processing node 69403b68575bc060ce158486: baseline-arm64-clabbe 2025-12-17 16:46:51,048 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b68575bc060ce158486 2025-12-17 16:46:51,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:51,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b68575bc060ce158486 HTTP/1.1" 200 None 2025-12-17 16:46:51,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:51,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:46:51,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:51,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b68575bc060ce158486 HTTP/1.1" 200 None 2025-12-17 16:46:51,114 - lava_callback - INFO - Completed processing callback for node 69403b68575bc060ce158486 2025-12-17 16:46:51,703 - lava_callback - INFO - Received callback for node 69403b6a575bc060ce158489 INFO: 172.21.0.1:46526 - "POST /node/69403b6a575bc060ce158489 HTTP/1.0" 202 Accepted 2025-12-17 16:46:51,709 - lava_callback - INFO - Starting async job submit for node 69403b6a575bc060ce158489 2025-12-17 16:46:51,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:51,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b6a575bc060ce158489 HTTP/1.1" 200 None 2025-12-17 16:46:51,746 - lava_callback - DEBUG - Processing node 69403b6a575bc060ce158489: baseline-arm64-clabbe 2025-12-17 16:46:51,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b6a575bc060ce158489 2025-12-17 16:46:51,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:51,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b6a575bc060ce158489 HTTP/1.1" 200 None 2025-12-17 16:46:51,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:51,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:46:51,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:46:51,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b6a575bc060ce158489 HTTP/1.1" 200 None 2025-12-17 16:46:51,810 - lava_callback - INFO - Completed processing callback for node 69403b6a575bc060ce158489 INFO: 172.21.0.1:46542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44216 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:47:10,456 - lava_callback - INFO - Received callback for node 69403b7a575bc060ce15849a INFO: 172.21.0.1:54096 - "POST /node/69403b7a575bc060ce15849a HTTP/1.0" 202 Accepted 2025-12-17 16:47:10,465 - lava_callback - INFO - Starting async job submit for node 69403b7a575bc060ce15849a 2025-12-17 16:47:10,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:10,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b7a575bc060ce15849a HTTP/1.1" 200 None 2025-12-17 16:47:10,477 - lava_callback - DEBUG - Processing node 69403b7a575bc060ce15849a: baseline-arm64-clabbe 2025-12-17 16:47:10,477 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b7a575bc060ce15849a 2025-12-17 16:47:10,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:10,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b7a575bc060ce15849a HTTP/1.1" 200 None 2025-12-17 16:47:10,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:10,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:47:10,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:10,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b7a575bc060ce15849a HTTP/1.1" 200 None 2025-12-17 16:47:10,527 - lava_callback - INFO - Completed processing callback for node 69403b7a575bc060ce15849a INFO: 172.21.0.1:54106 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:47:11,146 - lava_callback - INFO - Received callback for node 69403b7e575bc060ce1584a3 INFO: 172.21.0.1:54110 - "POST /node/69403b7e575bc060ce1584a3 HTTP/1.0" 202 Accepted 2025-12-17 16:47:11,153 - lava_callback - INFO - Starting async job submit for node 69403b7e575bc060ce1584a3 2025-12-17 16:47:11,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:11,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b7e575bc060ce1584a3 HTTP/1.1" 200 None 2025-12-17 16:47:11,174 - lava_callback - DEBUG - Processing node 69403b7e575bc060ce1584a3: baseline-arm64-clabbe 2025-12-17 16:47:11,174 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b7e575bc060ce1584a3 2025-12-17 16:47:11,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:11,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b7e575bc060ce1584a3 HTTP/1.1" 200 None 2025-12-17 16:47:11,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:11,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ae242fee8383e92b63eb8 HTTP/1.1" 200 None 2025-12-17 16:47:11,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:11,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b7e575bc060ce1584a3 HTTP/1.1" 200 None 2025-12-17 16:47:11,219 - lava_callback - INFO - Completed processing callback for node 69403b7e575bc060ce1584a3 2025-12-17 16:47:11,809 - lava_callback - INFO - Received callback for node 69403b80575bc060ce1584a8 INFO: 172.21.0.1:54116 - "POST /node/69403b80575bc060ce1584a8 HTTP/1.0" 202 Accepted 2025-12-17 16:47:11,818 - lava_callback - INFO - Starting async job submit for node 69403b80575bc060ce1584a8 2025-12-17 16:47:11,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:11,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b80575bc060ce1584a8 HTTP/1.1" 200 None 2025-12-17 16:47:11,828 - lava_callback - DEBUG - Processing node 69403b80575bc060ce1584a8: baseline-arm64-clabbe 2025-12-17 16:47:11,828 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b80575bc060ce1584a8 2025-12-17 16:47:11,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:11,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b80575bc060ce1584a8 HTTP/1.1" 200 None 2025-12-17 16:47:11,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:11,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:47:11,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:11,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b80575bc060ce1584a8 HTTP/1.1" 200 None 2025-12-17 16:47:11,873 - lava_callback - INFO - Completed processing callback for node 69403b80575bc060ce1584a8 2025-12-17 16:47:12,467 - lava_callback - INFO - Received callback for node 69403b82575bc060ce1584ab INFO: 172.21.0.1:54130 - "POST /node/69403b82575bc060ce1584ab HTTP/1.0" 202 Accepted 2025-12-17 16:47:12,479 - lava_callback - INFO - Starting async job submit for node 69403b82575bc060ce1584ab 2025-12-17 16:47:12,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:12,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b82575bc060ce1584ab HTTP/1.1" 200 None 2025-12-17 16:47:12,494 - lava_callback - DEBUG - Processing node 69403b82575bc060ce1584ab: baseline-arm64-clabbe 2025-12-17 16:47:12,494 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b82575bc060ce1584ab 2025-12-17 16:47:12,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:12,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b82575bc060ce1584ab HTTP/1.1" 200 None 2025-12-17 16:47:12,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:12,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:47:12,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:12,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b82575bc060ce1584ab HTTP/1.1" 200 None 2025-12-17 16:47:12,556 - lava_callback - INFO - Completed processing callback for node 69403b82575bc060ce1584ab INFO: 172.21.0.1:54146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46522 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:47:35,946 - lava_callback - INFO - Received callback for node 69403b9d575bc060ce1584cb INFO: 172.21.0.1:40638 - "POST /node/69403b9d575bc060ce1584cb HTTP/1.0" 202 Accepted 2025-12-17 16:47:35,955 - lava_callback - INFO - Starting async job submit for node 69403b9d575bc060ce1584cb 2025-12-17 16:47:35,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:35,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b9d575bc060ce1584cb HTTP/1.1" 200 None 2025-12-17 16:47:35,967 - lava_callback - DEBUG - Processing node 69403b9d575bc060ce1584cb: baseline-arm64-baylibre 2025-12-17 16:47:35,967 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403b9d575bc060ce1584cb 2025-12-17 16:47:35,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:35,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b9d575bc060ce1584cb HTTP/1.1" 200 None 2025-12-17 16:47:35,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:35,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:47:35,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:36,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b9d575bc060ce1584cb HTTP/1.1" 200 None 2025-12-17 16:47:36,002 - lava_callback - INFO - Completed processing callback for node 69403b9d575bc060ce1584cb INFO: 172.21.0.1:40650 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:47:36,612 - lava_callback - INFO - Received callback for node 69403ba0575bc060ce1584d4 INFO: 172.21.0.1:40652 - "POST /node/69403ba0575bc060ce1584d4 HTTP/1.0" 202 Accepted 2025-12-17 16:47:36,618 - lava_callback - INFO - Starting async job submit for node 69403ba0575bc060ce1584d4 2025-12-17 16:47:36,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:36,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ba0575bc060ce1584d4 HTTP/1.1" 200 None 2025-12-17 16:47:36,642 - lava_callback - DEBUG - Processing node 69403ba0575bc060ce1584d4: baseline-arm64-baylibre 2025-12-17 16:47:36,642 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ba0575bc060ce1584d4 2025-12-17 16:47:36,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:36,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ba0575bc060ce1584d4 HTTP/1.1" 200 None 2025-12-17 16:47:36,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:36,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:47:36,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:36,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ba0575bc060ce1584d4 HTTP/1.1" 200 None 2025-12-17 16:47:36,679 - lava_callback - INFO - Completed processing callback for node 69403ba0575bc060ce1584d4 2025-12-17 16:47:37,280 - lava_callback - INFO - Received callback for node 69403ba2575bc060ce1584d7 INFO: 172.21.0.1:40656 - "POST /node/69403ba2575bc060ce1584d7 HTTP/1.0" 202 Accepted 2025-12-17 16:47:37,286 - lava_callback - INFO - Starting async job submit for node 69403ba2575bc060ce1584d7 2025-12-17 16:47:37,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:37,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ba2575bc060ce1584d7 HTTP/1.1" 200 None 2025-12-17 16:47:37,297 - lava_callback - DEBUG - Processing node 69403ba2575bc060ce1584d7: baseline-arm64-baylibre 2025-12-17 16:47:37,297 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ba2575bc060ce1584d7 2025-12-17 16:47:37,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:37,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ba2575bc060ce1584d7 HTTP/1.1" 200 None 2025-12-17 16:47:37,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:37,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:47:37,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:37,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ba2575bc060ce1584d7 HTTP/1.1" 200 None 2025-12-17 16:47:37,368 - lava_callback - INFO - Completed processing callback for node 69403ba2575bc060ce1584d7 2025-12-17 16:47:37,934 - lava_callback - INFO - Received callback for node 69403ba4575bc060ce1584da INFO: 172.21.0.1:40670 - "POST /node/69403ba4575bc060ce1584da HTTP/1.0" 202 Accepted 2025-12-17 16:47:37,944 - lava_callback - INFO - Starting async job submit for node 69403ba4575bc060ce1584da 2025-12-17 16:47:37,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:37,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ba4575bc060ce1584da HTTP/1.1" 200 None 2025-12-17 16:47:37,959 - lava_callback - DEBUG - Processing node 69403ba4575bc060ce1584da: baseline-arm64-baylibre 2025-12-17 16:47:37,959 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ba4575bc060ce1584da 2025-12-17 16:47:37,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:37,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ba4575bc060ce1584da HTTP/1.1" 200 None 2025-12-17 16:47:37,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:37,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:47:37,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:38,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ba4575bc060ce1584da HTTP/1.1" 200 None 2025-12-17 16:47:38,012 - lava_callback - INFO - Completed processing callback for node 69403ba4575bc060ce1584da INFO: 172.21.0.1:39468 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:47:55,987 - lava_callback - INFO - Received callback for node 69403bb1575bc060ce1584e3 INFO: 172.21.0.1:59580 - "POST /node/69403bb1575bc060ce1584e3 HTTP/1.0" 202 Accepted 2025-12-17 16:47:55,992 - lava_callback - INFO - Starting async job submit for node 69403bb1575bc060ce1584e3 2025-12-17 16:47:55,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:59584 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:47:56,644 - lava_callback - INFO - Received callback for node 69403bb3575bc060ce1584ee INFO: 172.21.0.1:59586 - "POST /node/69403bb3575bc060ce1584ee HTTP/1.0" 202 Accepted 2025-12-17 16:47:56,653 - lava_callback - INFO - Starting async job submit for node 69403bb3575bc060ce1584ee 2025-12-17 16:47:56,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:56,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bb3575bc060ce1584ee HTTP/1.1" 200 None 2025-12-17 16:47:56,666 - lava_callback - DEBUG - Processing node 69403bb3575bc060ce1584ee: baseline-arm64-baylibre 2025-12-17 16:47:56,666 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403bb3575bc060ce1584ee 2025-12-17 16:47:56,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:56,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bb3575bc060ce1584ee HTTP/1.1" 200 None 2025-12-17 16:47:56,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:56,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:47:56,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:56,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403bb3575bc060ce1584ee HTTP/1.1" 200 None 2025-12-17 16:47:56,728 - lava_callback - INFO - Completed processing callback for node 69403bb3575bc060ce1584ee 2025-12-17 16:47:57,315 - lava_callback - INFO - Received callback for node 69403bb5575bc060ce1584f1 INFO: 172.21.0.1:59594 - "POST /node/69403bb5575bc060ce1584f1 HTTP/1.0" 202 Accepted 2025-12-17 16:47:57,322 - lava_callback - INFO - Starting async job submit for node 69403bb5575bc060ce1584f1 2025-12-17 16:47:57,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:57,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bb5575bc060ce1584f1 HTTP/1.1" 200 None 2025-12-17 16:47:57,337 - lava_callback - DEBUG - Processing node 69403bb5575bc060ce1584f1: baseline-arm64-baylibre 2025-12-17 16:47:57,337 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403bb5575bc060ce1584f1 2025-12-17 16:47:57,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:57,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bb5575bc060ce1584f1 HTTP/1.1" 200 None 2025-12-17 16:47:57,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:57,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:47:57,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:57,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403bb5575bc060ce1584f1 HTTP/1.1" 200 None 2025-12-17 16:47:57,379 - lava_callback - INFO - Completed processing callback for node 69403bb5575bc060ce1584f1 2025-12-17 16:47:58,004 - lava_callback - INFO - Received callback for node 69403bb8575bc060ce1584f6 INFO: 172.21.0.1:59596 - "POST /node/69403bb8575bc060ce1584f6 HTTP/1.0" 202 Accepted 2025-12-17 16:47:58,010 - lava_callback - INFO - Starting async job submit for node 69403bb8575bc060ce1584f6 2025-12-17 16:47:58,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:58,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bb8575bc060ce1584f6 HTTP/1.1" 200 None 2025-12-17 16:47:58,019 - lava_callback - DEBUG - Processing node 69403bb8575bc060ce1584f6: baseline-arm64-baylibre 2025-12-17 16:47:58,019 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403bb8575bc060ce1584f6 2025-12-17 16:47:58,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:58,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bb8575bc060ce1584f6 HTTP/1.1" 200 None 2025-12-17 16:47:58,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:58,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:47:58,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:58,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403bb8575bc060ce1584f6 HTTP/1.1" 200 None 2025-12-17 16:47:58,065 - lava_callback - INFO - Completed processing callback for node 69403bb8575bc060ce1584f6 2025-12-17 16:47:58,654 - lava_callback - INFO - Received callback for node 69403bba575bc060ce1584f9 INFO: 172.21.0.1:59612 - "POST /node/69403bba575bc060ce1584f9 HTTP/1.0" 202 Accepted 2025-12-17 16:47:58,663 - lava_callback - INFO - Starting async job submit for node 69403bba575bc060ce1584f9 2025-12-17 16:47:58,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:58,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bba575bc060ce1584f9 HTTP/1.1" 200 None 2025-12-17 16:47:58,688 - lava_callback - DEBUG - Processing node 69403bba575bc060ce1584f9: baseline-arm64-baylibre 2025-12-17 16:47:58,688 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403bba575bc060ce1584f9 2025-12-17 16:47:58,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:58,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bba575bc060ce1584f9 HTTP/1.1" 200 None 2025-12-17 16:47:58,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:58,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:47:58,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:47:58,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403bba575bc060ce1584f9 HTTP/1.1" 200 None 2025-12-17 16:47:58,774 - lava_callback - INFO - Completed processing callback for node 69403bba575bc060ce1584f9 2025-12-17 16:48:00,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bb1575bc060ce1584e3 HTTP/1.1" 200 None 2025-12-17 16:48:00,006 - lava_callback - DEBUG - Processing node 69403bb1575bc060ce1584e3: baseline-arm64-baylibre 2025-12-17 16:48:00,006 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403bb1575bc060ce1584e3 2025-12-17 16:48:00,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:00,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bb1575bc060ce1584e3 HTTP/1.1" 200 None 2025-12-17 16:48:00,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:00,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:48:00,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:00,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403bb1575bc060ce1584e3 HTTP/1.1" 200 None 2025-12-17 16:48:00,080 - lava_callback - INFO - Completed processing callback for node 69403bb1575bc060ce1584e3 INFO: 172.21.0.1:56424 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:48:10,545 - lava_callback - INFO - Received callback for node 69403bbc575bc060ce1584fe INFO: 172.21.0.1:54226 - "POST /node/69403bbc575bc060ce1584fe HTTP/1.0" 202 Accepted 2025-12-17 16:48:10,561 - lava_callback - INFO - Starting async job submit for node 69403bbc575bc060ce1584fe 2025-12-17 16:48:10,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:10,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bbc575bc060ce1584fe HTTP/1.1" 200 None 2025-12-17 16:48:10,591 - lava_callback - DEBUG - Processing node 69403bbc575bc060ce1584fe: baseline-arm64-clabbe 2025-12-17 16:48:10,591 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403bbc575bc060ce1584fe 2025-12-17 16:48:10,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:10,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bbc575bc060ce1584fe HTTP/1.1" 200 None 2025-12-17 16:48:10,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:10,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:48:10,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:10,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403bbc575bc060ce1584fe HTTP/1.1" 200 None 2025-12-17 16:48:10,661 - lava_callback - INFO - Completed processing callback for node 69403bbc575bc060ce1584fe INFO: 172.21.0.1:54236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54244 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:48:15,970 - lava_callback - INFO - Received callback for node 69403bc5575bc060ce158506 INFO: 172.21.0.1:54260 - "POST /node/69403bc5575bc060ce158506 HTTP/1.0" 202 Accepted 2025-12-17 16:48:15,980 - lava_callback - INFO - Starting async job submit for node 69403bc5575bc060ce158506 2025-12-17 16:48:15,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:15,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bc5575bc060ce158506 HTTP/1.1" 200 None 2025-12-17 16:48:15,996 - lava_callback - DEBUG - Processing node 69403bc5575bc060ce158506: baseline-arm64-baylibre 2025-12-17 16:48:15,996 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403bc5575bc060ce158506 2025-12-17 16:48:15,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:16,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bc5575bc060ce158506 HTTP/1.1" 200 None 2025-12-17 16:48:16,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:16,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:48:16,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:16,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403bc5575bc060ce158506 HTTP/1.1" 200 None 2025-12-17 16:48:16,041 - lava_callback - INFO - Completed processing callback for node 69403bc5575bc060ce158506 INFO: 172.21.0.1:54274 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:48:16,663 - lava_callback - INFO - Received callback for node 69403bc7575bc060ce15850b INFO: 172.21.0.1:54282 - "POST /node/69403bc7575bc060ce15850b HTTP/1.0" 202 Accepted 2025-12-17 16:48:16,672 - lava_callback - INFO - Starting async job submit for node 69403bc7575bc060ce15850b 2025-12-17 16:48:16,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:16,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bc7575bc060ce15850b HTTP/1.1" 200 None 2025-12-17 16:48:16,685 - lava_callback - DEBUG - Processing node 69403bc7575bc060ce15850b: baseline-arm64-baylibre 2025-12-17 16:48:16,685 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403bc7575bc060ce15850b 2025-12-17 16:48:16,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:16,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bc7575bc060ce15850b HTTP/1.1" 200 None 2025-12-17 16:48:16,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:16,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:48:16,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:16,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403bc7575bc060ce15850b HTTP/1.1" 200 None 2025-12-17 16:48:16,733 - lava_callback - INFO - Completed processing callback for node 69403bc7575bc060ce15850b INFO: 172.21.0.1:58328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59846 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:48:37,266 - lava_callback - INFO - Received callback for node 6942debfe43d2b76e9022e9d INFO: 172.21.0.1:59848 - "POST /node/6942debfe43d2b76e9022e9d HTTP/1.0" 202 Accepted 2025-12-17 16:48:37,320 - lava_callback - INFO - Starting async job submit for node 6942debfe43d2b76e9022e9d 2025-12-17 16:48:37,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:37,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942debfe43d2b76e9022e9d HTTP/1.1" 200 None 2025-12-17 16:48:37,402 - lava_callback - DEBUG - Processing node 6942debfe43d2b76e9022e9d: baseline-arm 2025-12-17 16:48:37,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:48:37,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:48:37,910 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942debfe43d2b76e9022e9d/log.txt.gz 2025-12-17 16:48:37,910 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:48:37,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:48:38,038 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:48:38,039 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942debfe43d2b76e9022e9d/lava_callback.json.gz 2025-12-17 16:48:38,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:38,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942debfe43d2b76e9022e9d HTTP/1.1" 200 None 2025-12-17 16:48:38,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:38,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:48:38,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:38,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942debfe43d2b76e9022e9d HTTP/1.1" 200 None 2025-12-17 16:48:38,136 - lava_callback - INFO - Completed processing callback for node 6942debfe43d2b76e9022e9d INFO: 172.21.0.1:54648 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:48:50,324 - lava_callback - INFO - Received callback for node 69403bdf575bc060ce15851a INFO: 172.21.0.1:57808 - "POST /node/69403bdf575bc060ce15851a HTTP/1.0" 202 Accepted 2025-12-17 16:48:50,332 - lava_callback - INFO - Starting async job submit for node 69403bdf575bc060ce15851a 2025-12-17 16:48:50,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:50,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bdf575bc060ce15851a HTTP/1.1" 200 None 2025-12-17 16:48:50,340 - lava_callback - DEBUG - Processing node 69403bdf575bc060ce15851a: baseline-arm64-clabbe 2025-12-17 16:48:50,340 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403bdf575bc060ce15851a 2025-12-17 16:48:50,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:50,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bdf575bc060ce15851a HTTP/1.1" 200 None 2025-12-17 16:48:50,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:50,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:48:50,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:50,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403bdf575bc060ce15851a HTTP/1.1" 200 None 2025-12-17 16:48:50,388 - lava_callback - INFO - Completed processing callback for node 69403bdf575bc060ce15851a 2025-12-17 16:48:50,985 - lava_callback - INFO - Received callback for node 69403be1575bc060ce158521 INFO: 172.21.0.1:57818 - "POST /node/69403be1575bc060ce158521 HTTP/1.0" 202 Accepted 2025-12-17 16:48:51,002 - lava_callback - INFO - Starting async job submit for node 69403be1575bc060ce158521 2025-12-17 16:48:51,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:51,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403be1575bc060ce158521 HTTP/1.1" 200 None 2025-12-17 16:48:51,038 - lava_callback - DEBUG - Processing node 69403be1575bc060ce158521: baseline-arm64-clabbe 2025-12-17 16:48:51,038 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403be1575bc060ce158521 2025-12-17 16:48:51,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:51,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403be1575bc060ce158521 HTTP/1.1" 200 None 2025-12-17 16:48:51,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:51,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:48:51,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:51,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403be1575bc060ce158521 HTTP/1.1" 200 None 2025-12-17 16:48:51,089 - lava_callback - INFO - Completed processing callback for node 69403be1575bc060ce158521 2025-12-17 16:48:51,662 - lava_callback - INFO - Received callback for node 69403be3575bc060ce158524 INFO: 172.21.0.1:57834 - "POST /node/69403be3575bc060ce158524 HTTP/1.0" 202 Accepted 2025-12-17 16:48:51,669 - lava_callback - INFO - Starting async job submit for node 69403be3575bc060ce158524 2025-12-17 16:48:51,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:51,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403be3575bc060ce158524 HTTP/1.1" 200 None 2025-12-17 16:48:51,678 - lava_callback - DEBUG - Processing node 69403be3575bc060ce158524: baseline-arm64-clabbe 2025-12-17 16:48:51,678 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403be3575bc060ce158524 2025-12-17 16:48:51,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:51,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403be3575bc060ce158524 HTTP/1.1" 200 None 2025-12-17 16:48:51,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:51,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:48:51,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:51,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403be3575bc060ce158524 HTTP/1.1" 200 None 2025-12-17 16:48:51,723 - lava_callback - INFO - Completed processing callback for node 69403be3575bc060ce158524 2025-12-17 16:48:52,317 - lava_callback - INFO - Received callback for node 69403be5575bc060ce158527 INFO: 172.21.0.1:57842 - "POST /node/69403be5575bc060ce158527 HTTP/1.0" 202 Accepted 2025-12-17 16:48:52,330 - lava_callback - INFO - Starting async job submit for node 69403be5575bc060ce158527 2025-12-17 16:48:52,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:52,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403be5575bc060ce158527 HTTP/1.1" 200 None 2025-12-17 16:48:52,346 - lava_callback - DEBUG - Processing node 69403be5575bc060ce158527: baseline-arm64-clabbe 2025-12-17 16:48:52,346 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403be5575bc060ce158527 2025-12-17 16:48:52,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:52,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403be5575bc060ce158527 HTTP/1.1" 200 None 2025-12-17 16:48:52,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:52,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:48:52,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:52,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403be5575bc060ce158527 HTTP/1.1" 200 None 2025-12-17 16:48:52,393 - lava_callback - INFO - Completed processing callback for node 69403be5575bc060ce158527 2025-12-17 16:48:56,053 - lava_callback - INFO - Received callback for node 69403bed575bc060ce158530 INFO: 172.21.0.1:57854 - "POST /node/69403bed575bc060ce158530 HTTP/1.0" 202 Accepted 2025-12-17 16:48:56,062 - lava_callback - INFO - Starting async job submit for node 69403bed575bc060ce158530 2025-12-17 16:48:56,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:56,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bed575bc060ce158530 HTTP/1.1" 200 None 2025-12-17 16:48:56,081 - lava_callback - DEBUG - Processing node 69403bed575bc060ce158530: baseline-arm64-baylibre 2025-12-17 16:48:56,082 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403bed575bc060ce158530 2025-12-17 16:48:56,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:56,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bed575bc060ce158530 HTTP/1.1" 200 None 2025-12-17 16:48:56,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:56,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:48:56,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:56,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403bed575bc060ce158530 HTTP/1.1" 200 None 2025-12-17 16:48:56,160 - lava_callback - INFO - Completed processing callback for node 69403bed575bc060ce158530 INFO: 172.21.0.1:57856 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:48:56,704 - lava_callback - INFO - Received callback for node 69403bef575bc060ce158537 INFO: 172.21.0.1:57864 - "POST /node/69403bef575bc060ce158537 HTTP/1.0" 202 Accepted 2025-12-17 16:48:56,713 - lava_callback - INFO - Starting async job submit for node 69403bef575bc060ce158537 2025-12-17 16:48:56,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:56,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bef575bc060ce158537 HTTP/1.1" 200 None 2025-12-17 16:48:56,728 - lava_callback - DEBUG - Processing node 69403bef575bc060ce158537: baseline-arm64-baylibre 2025-12-17 16:48:56,728 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403bef575bc060ce158537 2025-12-17 16:48:56,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:56,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bef575bc060ce158537 HTTP/1.1" 200 None 2025-12-17 16:48:56,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:56,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:48:56,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:56,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403bef575bc060ce158537 HTTP/1.1" 200 None 2025-12-17 16:48:56,785 - lava_callback - INFO - Completed processing callback for node 69403bef575bc060ce158537 2025-12-17 16:48:57,388 - lava_callback - INFO - Received callback for node 69403bf1575bc060ce15853a INFO: 172.21.0.1:57874 - "POST /node/69403bf1575bc060ce15853a HTTP/1.0" 202 Accepted 2025-12-17 16:48:57,399 - lava_callback - INFO - Starting async job submit for node 69403bf1575bc060ce15853a 2025-12-17 16:48:57,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:57,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bf1575bc060ce15853a HTTP/1.1" 200 None 2025-12-17 16:48:57,419 - lava_callback - DEBUG - Processing node 69403bf1575bc060ce15853a: baseline-arm64-baylibre 2025-12-17 16:48:57,419 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403bf1575bc060ce15853a 2025-12-17 16:48:57,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:57,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403bf1575bc060ce15853a HTTP/1.1" 200 None 2025-12-17 16:48:57,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:57,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:48:57,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:48:57,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403bf1575bc060ce15853a HTTP/1.1" 200 None 2025-12-17 16:48:57,475 - lava_callback - INFO - Completed processing callback for node 69403bf1575bc060ce15853a INFO: 172.21.0.1:38962 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:49:07,120 - lava_callback - INFO - Received callback for node 6942decee43d2b76e9022f1f INFO: 172.21.0.1:38978 - "POST /node/6942decee43d2b76e9022f1f HTTP/1.0" 202 Accepted 2025-12-17 16:49:07,129 - lava_callback - INFO - Starting async job submit for node 6942decee43d2b76e9022f1f 2025-12-17 16:49:07,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:07,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942decee43d2b76e9022f1f HTTP/1.1" 200 None 2025-12-17 16:49:07,186 - lava_callback - DEBUG - Processing node 6942decee43d2b76e9022f1f: baseline-arm-pengutronix 2025-12-17 16:49:07,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:07,651 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:07,652 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-6942decee43d2b76e9022f1f/log.txt.gz 2025-12-17 16:49:07,652 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:49:07,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:08,084 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:08,085 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-6942decee43d2b76e9022f1f/lava_callback.json.gz 2025-12-17 16:49:08,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:08,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942decee43d2b76e9022f1f HTTP/1.1" 200 None 2025-12-17 16:49:08,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:08,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:49:08,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:08,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942decee43d2b76e9022f1f HTTP/1.1" 200 None 2025-12-17 16:49:08,188 - lava_callback - INFO - Completed processing callback for node 6942decee43d2b76e9022f1f 2025-12-17 16:49:08,240 - lava_callback - INFO - Received callback for node 6942decfe43d2b76e9022f23 INFO: 172.21.0.1:38986 - "POST /node/6942decfe43d2b76e9022f23 HTTP/1.0" 202 Accepted 2025-12-17 16:49:08,253 - lava_callback - INFO - Starting async job submit for node 6942decfe43d2b76e9022f23 2025-12-17 16:49:08,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:08,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942decfe43d2b76e9022f23 HTTP/1.1" 200 None 2025-12-17 16:49:08,296 - lava_callback - DEBUG - Processing node 6942decfe43d2b76e9022f23: baseline-arm-pengutronix 2025-12-17 16:49:08,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:08,977 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:08,977 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-6942decfe43d2b76e9022f23/log.txt.gz 2025-12-17 16:49:08,978 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:49:08,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:09,137 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:09,138 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-6942decfe43d2b76e9022f23/lava_callback.json.gz 2025-12-17 16:49:09,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:09,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942decfe43d2b76e9022f23 HTTP/1.1" 200 None 2025-12-17 16:49:09,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:09,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:49:09,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:09,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942decfe43d2b76e9022f23 HTTP/1.1" 200 None 2025-12-17 16:49:09,212 - lava_callback - INFO - Completed processing callback for node 6942decfe43d2b76e9022f23 INFO: 172.21.0.1:43438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43452 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:49:16,059 - lava_callback - INFO - Received callback for node 69403c01575bc060ce15853f INFO: 172.21.0.1:43466 - "POST /node/69403c01575bc060ce15853f HTTP/1.0" 202 Accepted 2025-12-17 16:49:16,065 - lava_callback - INFO - Starting async job submit for node 69403c01575bc060ce15853f 2025-12-17 16:49:16,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:16,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c01575bc060ce15853f HTTP/1.1" 200 None 2025-12-17 16:49:16,097 - lava_callback - DEBUG - Processing node 69403c01575bc060ce15853f: baseline-arm64-baylibre 2025-12-17 16:49:16,097 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c01575bc060ce15853f 2025-12-17 16:49:16,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:16,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c01575bc060ce15853f HTTP/1.1" 200 None 2025-12-17 16:49:16,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:16,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:49:16,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:16,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c01575bc060ce15853f HTTP/1.1" 200 None 2025-12-17 16:49:16,156 - lava_callback - INFO - Completed processing callback for node 69403c01575bc060ce15853f INFO: 172.21.0.1:43468 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:49:16,730 - lava_callback - INFO - Received callback for node 69403c04575bc060ce158544 INFO: 172.21.0.1:43470 - "POST /node/69403c04575bc060ce158544 HTTP/1.0" 202 Accepted 2025-12-17 16:49:16,743 - lava_callback - INFO - Starting async job submit for node 69403c04575bc060ce158544 2025-12-17 16:49:16,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:16,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c04575bc060ce158544 HTTP/1.1" 200 None 2025-12-17 16:49:16,760 - lava_callback - DEBUG - Processing node 69403c04575bc060ce158544: baseline-arm64-baylibre 2025-12-17 16:49:16,760 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c04575bc060ce158544 2025-12-17 16:49:16,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:16,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c04575bc060ce158544 HTTP/1.1" 200 None 2025-12-17 16:49:16,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:16,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:49:16,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:16,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c04575bc060ce158544 HTTP/1.1" 200 None 2025-12-17 16:49:16,832 - lava_callback - INFO - Completed processing callback for node 69403c04575bc060ce158544 2025-12-17 16:49:17,109 - lava_callback - INFO - Received callback for node 6942decfe43d2b76e9022f2c INFO: 172.21.0.1:43474 - "POST /node/6942decfe43d2b76e9022f2c HTTP/1.0" 202 Accepted 2025-12-17 16:49:17,161 - lava_callback - INFO - Starting async job submit for node 6942decfe43d2b76e9022f2c 2025-12-17 16:49:17,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:17,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942decfe43d2b76e9022f2c HTTP/1.1" 200 None 2025-12-17 16:49:17,223 - lava_callback - DEBUG - Processing node 6942decfe43d2b76e9022f2c: baseline-arm-pengutronix 2025-12-17 16:49:17,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:17,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:17,774 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-6942decfe43d2b76e9022f2c/log.txt.gz 2025-12-17 16:49:17,774 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:49:17,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:17,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:17,925 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-6942decfe43d2b76e9022f2c/lava_callback.json.gz 2025-12-17 16:49:17,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:17,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942decfe43d2b76e9022f2c HTTP/1.1" 200 None 2025-12-17 16:49:17,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:17,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:49:17,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:17,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942decfe43d2b76e9022f2c HTTP/1.1" 200 None 2025-12-17 16:49:17,995 - lava_callback - INFO - Completed processing callback for node 6942decfe43d2b76e9022f2c 2025-12-17 16:49:19,976 - lava_callback - INFO - Received callback for node 6940c4fa1e3c24e95681d5f9 INFO: 172.21.0.1:38798 - "POST /node/6940c4fa1e3c24e95681d5f9 HTTP/1.0" 202 Accepted 2025-12-17 16:49:19,989 - lava_callback - INFO - Starting async job submit for node 6940c4fa1e3c24e95681d5f9 2025-12-17 16:49:20,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:20,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c4fa1e3c24e95681d5f9 HTTP/1.1" 200 None 2025-12-17 16:49:20,132 - lava_callback - DEBUG - Processing node 6940c4fa1e3c24e95681d5f9: baseline-x86 2025-12-17 16:49:21,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:21,602 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:21,603 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-6940c4fa1e3c24e95681d5f9/log.txt.gz 2025-12-17 16:49:21,603 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:49:21,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:22,299 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:22,300 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-6940c4fa1e3c24e95681d5f9/lava_callback.json.gz 2025-12-17 16:49:22,423 - lava_callback - INFO - Received callback for node 6942dec3e43d2b76e9022ec4 INFO: 172.21.0.1:38812 - "POST /node/6942dec3e43d2b76e9022ec4 HTTP/1.0" 202 Accepted 2025-12-17 16:49:22,435 - lava_callback - INFO - Starting async job submit for node 6942dec3e43d2b76e9022ec4 2025-12-17 16:49:22,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 DEBUG: Setup failed for 6940c4fa1e3c24e95681d5f9. Transitting job node result: incomplete -> fail 2025-12-17 16:49:22,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:22,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dec3e43d2b76e9022ec4 HTTP/1.1" 200 None 2025-12-17 16:49:22,564 - lava_callback - DEBUG - Processing node 6942dec3e43d2b76e9022ec4: baseline-arm-broonie 2025-12-17 16:49:22,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c4fa1e3c24e95681d5f9 HTTP/1.1" 200 None 2025-12-17 16:49:22,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:22,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21eb07de2c0cbc264f2b HTTP/1.1" 200 None 2025-12-17 16:49:22,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:22,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940c4fa1e3c24e95681d5f9 HTTP/1.1" 200 None 2025-12-17 16:49:22,758 - lava_callback - INFO - Completed processing callback for node 6940c4fa1e3c24e95681d5f9 2025-12-17 16:49:23,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:23,349 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:23,350 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-6942dec3e43d2b76e9022ec4/log.txt.gz 2025-12-17 16:49:23,350 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:49:23,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:23,510 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:23,511 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-6942dec3e43d2b76e9022ec4/lava_callback.json.gz 2025-12-17 16:49:23,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:23,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dec3e43d2b76e9022ec4 HTTP/1.1" 200 None 2025-12-17 16:49:23,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:23,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:49:23,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:23,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942dec3e43d2b76e9022ec4 HTTP/1.1" 200 None 2025-12-17 16:49:23,615 - lava_callback - INFO - Completed processing callback for node 6942dec3e43d2b76e9022ec4 2025-12-17 16:49:24,233 - lava_callback - INFO - Received callback for node 6942dec4e43d2b76e9022ec8 INFO: 172.21.0.1:38824 - "POST /node/6942dec4e43d2b76e9022ec8 HTTP/1.0" 202 Accepted 2025-12-17 16:49:24,246 - lava_callback - INFO - Starting async job submit for node 6942dec4e43d2b76e9022ec8 2025-12-17 16:49:24,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:24,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dec4e43d2b76e9022ec8 HTTP/1.1" 200 None 2025-12-17 16:49:24,290 - lava_callback - DEBUG - Processing node 6942dec4e43d2b76e9022ec8: baseline-arm-broonie 2025-12-17 16:49:24,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:25,004 - lava_callback - INFO - Received callback for node 6942debce43d2b76e9022e7a INFO: 172.21.0.1:38830 - "POST /node/6942debce43d2b76e9022e7a HTTP/1.0" 202 Accepted 2025-12-17 16:49:25,015 - lava_callback - INFO - Starting async job submit for node 6942debce43d2b76e9022e7a 2025-12-17 16:49:25,058 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:25,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:25,061 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-6942dec4e43d2b76e9022ec8/log.txt.gz 2025-12-17 16:49:25,062 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:49:25,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:25,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942debce43d2b76e9022e7a HTTP/1.1" 200 None 2025-12-17 16:49:25,071 - lava_callback - DEBUG - Processing node 6942debce43d2b76e9022e7a: baseline-arm 2025-12-17 16:49:25,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:25,727 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:25,728 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-6942dec4e43d2b76e9022ec8/lava_callback.json.gz 2025-12-17 16:49:25,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:25,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dec4e43d2b76e9022ec8 HTTP/1.1" 200 None 2025-12-17 16:49:25,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:25,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:49:25,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:25,817 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:25,817 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942debce43d2b76e9022e7a/log.txt.gz 2025-12-17 16:49:25,818 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:49:25,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:25,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942dec4e43d2b76e9022ec8 HTTP/1.1" 200 None 2025-12-17 16:49:25,841 - lava_callback - INFO - Completed processing callback for node 6942dec4e43d2b76e9022ec8 2025-12-17 16:49:26,076 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:26,078 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942debce43d2b76e9022e7a/lava_callback.json.gz 2025-12-17 16:49:26,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:26,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942debce43d2b76e9022e7a HTTP/1.1" 200 None 2025-12-17 16:49:26,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:26,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:49:26,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:26,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942debce43d2b76e9022e7a HTTP/1.1" 200 None 2025-12-17 16:49:26,183 - lava_callback - INFO - Completed processing callback for node 6942debce43d2b76e9022e7a INFO: 172.21.0.1:38832 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:49:29,376 - lava_callback - INFO - Received callback for node 6942debde43d2b76e9022e8c INFO: 172.21.0.1:38844 - "POST /node/6942debde43d2b76e9022e8c HTTP/1.0" 202 Accepted 2025-12-17 16:49:29,387 - lava_callback - INFO - Starting async job submit for node 6942debde43d2b76e9022e8c 2025-12-17 16:49:29,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:29,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942debde43d2b76e9022e8c HTTP/1.1" 200 None 2025-12-17 16:49:29,436 - lava_callback - DEBUG - Processing node 6942debde43d2b76e9022e8c: baseline-arm 2025-12-17 16:49:29,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:29,987 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:29,988 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942debde43d2b76e9022e8c/log.txt.gz 2025-12-17 16:49:29,988 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:49:29,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:30,175 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:30,175 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942debde43d2b76e9022e8c/lava_callback.json.gz 2025-12-17 16:49:30,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:30,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942debde43d2b76e9022e8c HTTP/1.1" 200 None 2025-12-17 16:49:30,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:30,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:49:30,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:30,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942debde43d2b76e9022e8c HTTP/1.1" 200 None 2025-12-17 16:49:30,272 - lava_callback - INFO - Completed processing callback for node 6942debde43d2b76e9022e8c 2025-12-17 16:49:31,158 - lava_callback - INFO - Received callback for node 694030a2575bc060ce1568df INFO: 172.21.0.1:41546 - "POST /node/694030a2575bc060ce1568df HTTP/1.0" 202 Accepted 2025-12-17 16:49:31,209 - lava_callback - INFO - Starting async job submit for node 694030a2575bc060ce1568df 2025-12-17 16:49:33,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:33,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694030a2575bc060ce1568df HTTP/1.1" 200 None 2025-12-17 16:49:33,576 - lava_callback - DEBUG - Processing node 694030a2575bc060ce1568df: ltp-syscalls 2025-12-17 16:49:34,750 - lava_callback - INFO - Received callback for node 6942debfe43d2b76e9022e9a 2025-12-17 16:49:34,750 - lava_callback - INFO - Received callback for node 6942debce43d2b76e9022e82 INFO: 172.21.0.1:41550 - "POST /node/6942debfe43d2b76e9022e9a HTTP/1.0" 202 Accepted 2025-12-17 16:49:34,771 - lava_callback - INFO - Starting async job submit for node 6942debfe43d2b76e9022e9a 2025-12-17 16:49:34,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:34,858 - lava_callback - INFO - Starting async job submit for node 6942debce43d2b76e9022e82 INFO: 172.21.0.1:41560 - "POST /node/6942debce43d2b76e9022e82 HTTP/1.0" 202 Accepted 2025-12-17 16:49:34,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:35,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942debfe43d2b76e9022e9a HTTP/1.1" 200 None 2025-12-17 16:49:35,498 - lava_callback - DEBUG - Processing node 6942debfe43d2b76e9022e9a: baseline-arm 2025-12-17 16:49:35,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942debce43d2b76e9022e82 HTTP/1.1" 200 None 2025-12-17 16:49:35,504 - lava_callback - DEBUG - Processing node 6942debce43d2b76e9022e82: baseline-arm 2025-12-17 16:49:36,168 - lava_callback - INFO - Received callback for node 69403c15575bc060ce15854d 2025-12-17 16:49:36,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:36,247 - lava_callback - INFO - Starting async job submit for node 69403c15575bc060ce15854d INFO: 172.21.0.1:41564 - "POST /node/69403c15575bc060ce15854d HTTP/1.0" 202 Accepted 2025-12-17 16:49:36,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:36,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c15575bc060ce15854d HTTP/1.1" 200 None 2025-12-17 16:49:36,402 - lava_callback - DEBUG - Processing node 69403c15575bc060ce15854d: baseline-arm64-baylibre 2025-12-17 16:49:36,402 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c15575bc060ce15854d 2025-12-17 16:49:36,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:36,533 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:36,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c15575bc060ce15854d HTTP/1.1" 200 None 2025-12-17 16:49:36,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:41574 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:49:36,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:49:36,682 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942debce43d2b76e9022e82/log.txt.gz 2025-12-17 16:49:36,687 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:49:36,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:36,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:36,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c15575bc060ce15854d HTTP/1.1" 200 None 2025-12-17 16:49:36,849 - lava_callback - INFO - Completed processing callback for node 69403c15575bc060ce15854d 2025-12-17 16:49:37,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:37,342 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:37,344 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942debce43d2b76e9022e82/lava_callback.json.gz 2025-12-17 16:49:37,398 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:37,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:37,447 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942debfe43d2b76e9022e9a/log.txt.gz 2025-12-17 16:49:37,447 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:49:37,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:37,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942debce43d2b76e9022e82 HTTP/1.1" 200 None 2025-12-17 16:49:37,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:37,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:49:37,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:37,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942debce43d2b76e9022e82 HTTP/1.1" 200 None 2025-12-17 16:49:37,635 - lava_callback - INFO - Completed processing callback for node 6942debce43d2b76e9022e82 2025-12-17 16:49:38,236 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:38,237 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942debfe43d2b76e9022e9a/lava_callback.json.gz 2025-12-17 16:49:38,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:38,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942debfe43d2b76e9022e9a HTTP/1.1" 200 None 2025-12-17 16:49:38,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:38,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:49:38,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:38,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942debfe43d2b76e9022e9a HTTP/1.1" 200 None 2025-12-17 16:49:38,527 - lava_callback - INFO - Completed processing callback for node 6942debfe43d2b76e9022e9a 2025-12-17 16:49:39,299 - lava_callback - INFO - Received callback for node 6942dec9e43d2b76e9022eed INFO: 172.21.0.1:41578 - "POST /node/6942dec9e43d2b76e9022eed HTTP/1.0" 202 Accepted 2025-12-17 16:49:39,325 - lava_callback - INFO - Starting async job submit for node 6942dec9e43d2b76e9022eed 2025-12-17 16:49:39,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:39,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dec9e43d2b76e9022eed HTTP/1.1" 200 None 2025-12-17 16:49:39,424 - lava_callback - DEBUG - Processing node 6942dec9e43d2b76e9022eed: baseline-arm-clabbe 2025-12-17 16:49:40,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:40,702 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:40,749 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-6942dec9e43d2b76e9022eed/log.txt.gz 2025-12-17 16:49:40,760 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:49:40,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:41,394 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:41,446 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-6942dec9e43d2b76e9022eed/lava_callback.json.gz 2025-12-17 16:49:41,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:41,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dec9e43d2b76e9022eed HTTP/1.1" 200 None 2025-12-17 16:49:41,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:41,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:49:41,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:41,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942dec9e43d2b76e9022eed HTTP/1.1" 200 None 2025-12-17 16:49:41,670 - lava_callback - INFO - Completed processing callback for node 6942dec9e43d2b76e9022eed 2025-12-17 16:49:45,226 - lava_callback - INFO - Received callback for node 6942dec8e43d2b76e9022ee9 INFO: 172.21.0.1:35460 - "POST /node/6942dec8e43d2b76e9022ee9 HTTP/1.0" 202 Accepted 2025-12-17 16:49:45,253 - lava_callback - INFO - Starting async job submit for node 6942dec8e43d2b76e9022ee9 2025-12-17 16:49:45,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:45,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dec8e43d2b76e9022ee9 HTTP/1.1" 200 None 2025-12-17 16:49:45,356 - lava_callback - DEBUG - Processing node 6942dec8e43d2b76e9022ee9: baseline-arm-clabbe 2025-12-17 16:49:46,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:35472 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:49:46,530 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:46,531 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-6942dec8e43d2b76e9022ee9/log.txt.gz 2025-12-17 16:49:46,531 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:49:46,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:46,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:46,811 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-6942dec8e43d2b76e9022ee9/lava_callback.json.gz 2025-12-17 16:49:47,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:47,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dec8e43d2b76e9022ee9 HTTP/1.1" 200 None 2025-12-17 16:49:47,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:47,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:49:47,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:47,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:47,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942dec8e43d2b76e9022ee9 HTTP/1.1" 200 None 2025-12-17 16:49:47,362 - lava_callback - INFO - Completed processing callback for node 6942dec8e43d2b76e9022ee9 2025-12-17 16:49:47,922 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:47,923 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-694030a2575bc060ce1568df/log.txt.gz 2025-12-17 16:49:47,923 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:49:48,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:48,628 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:48,629 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-694030a2575bc060ce1568df/lava_callback.json.gz DEBUG: 694030a2575bc060ce1568df Transitting job node result: pass -> fail 2025-12-17 16:49:48,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:48,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694030a2575bc060ce1568df HTTP/1.1" 200 None 2025-12-17 16:49:48,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:48,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad820fb52d1dfff36c53b HTTP/1.1" 200 None 2025-12-17 16:49:48,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:50,021 - lava_callback - INFO - Received callback for node 6942decce43d2b76e9022f06 INFO: 172.21.0.1:48162 - "POST /node/6942decce43d2b76e9022f06 HTTP/1.0" 202 Accepted 2025-12-17 16:49:50,029 - lava_callback - INFO - Starting async job submit for node 6942decce43d2b76e9022f06 2025-12-17 16:49:50,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:50,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942decce43d2b76e9022f06 HTTP/1.1" 200 None 2025-12-17 16:49:50,074 - lava_callback - DEBUG - Processing node 6942decce43d2b76e9022f06: baseline-arm-kontron 2025-12-17 16:49:50,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:50,561 - lava_callback - INFO - Received callback for node 69403c1a575bc060ce158558 INFO: 172.21.0.1:48164 - "POST /node/69403c1a575bc060ce158558 HTTP/1.0" 202 Accepted 2025-12-17 16:49:50,571 - lava_callback - INFO - Starting async job submit for node 69403c1a575bc060ce158558 2025-12-17 16:49:50,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:50,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c1a575bc060ce158558 HTTP/1.1" 200 None 2025-12-17 16:49:50,586 - lava_callback - DEBUG - Processing node 69403c1a575bc060ce158558: baseline-arm64-clabbe 2025-12-17 16:49:50,586 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c1a575bc060ce158558 2025-12-17 16:49:50,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:50,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c1a575bc060ce158558 HTTP/1.1" 200 None 2025-12-17 16:49:50,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:50,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:49:50,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:50,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c1a575bc060ce158558 HTTP/1.1" 200 None 2025-12-17 16:49:50,640 - lava_callback - INFO - Completed processing callback for node 69403c1a575bc060ce158558 2025-12-17 16:49:50,855 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:50,856 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-6942decce43d2b76e9022f06/log.txt.gz 2025-12-17 16:49:50,856 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:49:50,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:49:51,040 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:49:51,040 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-6942decce43d2b76e9022f06/lava_callback.json.gz 2025-12-17 16:49:51,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:51,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942decce43d2b76e9022f06 HTTP/1.1" 200 None 2025-12-17 16:49:51,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:51,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:49:51,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:51,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942decce43d2b76e9022f06 HTTP/1.1" 200 None 2025-12-17 16:49:51,150 - lava_callback - INFO - Completed processing callback for node 6942decce43d2b76e9022f06 2025-12-17 16:49:51,245 - lava_callback - INFO - Received callback for node 69403c1d575bc060ce15855e INFO: 172.21.0.1:48176 - "POST /node/69403c1d575bc060ce15855e HTTP/1.0" 202 Accepted 2025-12-17 16:49:51,257 - lava_callback - INFO - Starting async job submit for node 69403c1d575bc060ce15855e 2025-12-17 16:49:51,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:51,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c1d575bc060ce15855e HTTP/1.1" 200 None 2025-12-17 16:49:51,281 - lava_callback - DEBUG - Processing node 69403c1d575bc060ce15855e: baseline-arm64-clabbe 2025-12-17 16:49:51,281 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c1d575bc060ce15855e 2025-12-17 16:49:51,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:51,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c1d575bc060ce15855e HTTP/1.1" 200 None 2025-12-17 16:49:51,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:51,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:49:51,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:51,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c1d575bc060ce15855e HTTP/1.1" 200 None 2025-12-17 16:49:51,352 - lava_callback - INFO - Completed processing callback for node 69403c1d575bc060ce15855e 2025-12-17 16:49:52,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694030a2575bc060ce1568df HTTP/1.1" 200 None 2025-12-17 16:49:52,311 - lava_callback - INFO - Completed processing callback for node 694030a2575bc060ce1568df 2025-12-17 16:49:56,061 - lava_callback - INFO - Received callback for node 69403c29575bc060ce158563 INFO: 172.21.0.1:48192 - "POST /node/69403c29575bc060ce158563 HTTP/1.0" 202 Accepted 2025-12-17 16:49:56,073 - lava_callback - INFO - Starting async job submit for node 69403c29575bc060ce158563 2025-12-17 16:49:56,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:56,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c29575bc060ce158563 HTTP/1.1" 200 None 2025-12-17 16:49:56,088 - lava_callback - DEBUG - Processing node 69403c29575bc060ce158563: baseline-arm-baylibre 2025-12-17 16:49:56,088 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c29575bc060ce158563 2025-12-17 16:49:56,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:56,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c29575bc060ce158563 HTTP/1.1" 200 None 2025-12-17 16:49:56,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:56,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:49:56,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:56,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c29575bc060ce158563 HTTP/1.1" 200 None 2025-12-17 16:49:56,159 - lava_callback - INFO - Completed processing callback for node 69403c29575bc060ce158563 2025-12-17 16:49:56,474 - lava_callback - INFO - Received callback for node 69403c33575bc060ce158580 INFO: 172.21.0.1:48200 - "POST /node/69403c33575bc060ce158580 HTTP/1.0" 202 Accepted 2025-12-17 16:49:56,483 - lava_callback - INFO - Starting async job submit for node 69403c33575bc060ce158580 2025-12-17 16:49:56,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:56,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c33575bc060ce158580 HTTP/1.1" 200 None 2025-12-17 16:49:56,498 - lava_callback - DEBUG - Processing node 69403c33575bc060ce158580: baseline-x86-kcidebug-intel 2025-12-17 16:49:56,498 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c33575bc060ce158580 2025-12-17 16:49:56,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:48204 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:49:56,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c33575bc060ce158580 HTTP/1.1" 200 None 2025-12-17 16:49:56,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:56,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693aebc7fee8383e92b666f7 HTTP/1.1" 200 None 2025-12-17 16:49:56,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:56,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c33575bc060ce158580 HTTP/1.1" 200 None 2025-12-17 16:49:56,550 - lava_callback - INFO - Completed processing callback for node 69403c33575bc060ce158580 2025-12-17 16:49:56,726 - lava_callback - INFO - Received callback for node 69403c2b575bc060ce15856a INFO: 172.21.0.1:48206 - "POST /node/69403c2b575bc060ce15856a HTTP/1.0" 202 Accepted 2025-12-17 16:49:56,735 - lava_callback - INFO - Starting async job submit for node 69403c2b575bc060ce15856a 2025-12-17 16:49:56,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:56,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c2b575bc060ce15856a HTTP/1.1" 200 None 2025-12-17 16:49:56,755 - lava_callback - DEBUG - Processing node 69403c2b575bc060ce15856a: baseline-arm-baylibre 2025-12-17 16:49:56,755 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c2b575bc060ce15856a 2025-12-17 16:49:56,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:56,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c2b575bc060ce15856a HTTP/1.1" 200 None 2025-12-17 16:49:56,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:56,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:49:56,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:56,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c2b575bc060ce15856a HTTP/1.1" 200 None 2025-12-17 16:49:56,815 - lava_callback - INFO - Completed processing callback for node 69403c2b575bc060ce15856a 2025-12-17 16:49:57,384 - lava_callback - INFO - Received callback for node 69403c2c575bc060ce15856d INFO: 172.21.0.1:48216 - "POST /node/69403c2c575bc060ce15856d HTTP/1.0" 202 Accepted 2025-12-17 16:49:57,393 - lava_callback - INFO - Starting async job submit for node 69403c2c575bc060ce15856d 2025-12-17 16:49:57,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:57,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c2c575bc060ce15856d HTTP/1.1" 200 None 2025-12-17 16:49:57,403 - lava_callback - DEBUG - Processing node 69403c2c575bc060ce15856d: baseline-arm-baylibre 2025-12-17 16:49:57,403 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c2c575bc060ce15856d 2025-12-17 16:49:57,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:57,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c2c575bc060ce15856d HTTP/1.1" 200 None 2025-12-17 16:49:57,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:57,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:49:57,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:49:57,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c2c575bc060ce15856d HTTP/1.1" 200 None 2025-12-17 16:49:57,446 - lava_callback - INFO - Completed processing callback for node 69403c2c575bc060ce15856d INFO: 172.21.0.1:53038 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:50:07,091 - lava_callback - INFO - Received callback for node 6942debce43d2b76e9022e7d INFO: 172.21.0.1:53044 - "POST /node/6942debce43d2b76e9022e7d HTTP/1.0" 202 Accepted 2025-12-17 16:50:07,103 - lava_callback - INFO - Starting async job submit for node 6942debce43d2b76e9022e7d 2025-12-17 16:50:07,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:07,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942debce43d2b76e9022e7d HTTP/1.1" 200 None 2025-12-17 16:50:07,273 - lava_callback - DEBUG - Processing node 6942debce43d2b76e9022e7d: baseline-arm 2025-12-17 16:50:07,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:50:08,121 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:50:08,122 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942debce43d2b76e9022e7d/log.txt.gz 2025-12-17 16:50:08,122 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:50:08,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:50:08,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:50:08,576 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942debce43d2b76e9022e7d/lava_callback.json.gz 2025-12-17 16:50:08,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:08,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942debce43d2b76e9022e7d HTTP/1.1" 200 None 2025-12-17 16:50:08,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:08,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:50:08,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:08,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942debce43d2b76e9022e7d HTTP/1.1" 200 None 2025-12-17 16:50:08,667 - lava_callback - INFO - Completed processing callback for node 6942debce43d2b76e9022e7d 2025-12-17 16:50:10,044 - lava_callback - INFO - Received callback for node 6942decce43d2b76e9022f0d INFO: 172.21.0.1:44732 - "POST /node/6942decce43d2b76e9022f0d HTTP/1.0" 202 Accepted 2025-12-17 16:50:10,054 - lava_callback - INFO - Starting async job submit for node 6942decce43d2b76e9022f0d 2025-12-17 16:50:10,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:10,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942decce43d2b76e9022f0d HTTP/1.1" 200 None 2025-12-17 16:50:10,114 - lava_callback - DEBUG - Processing node 6942decce43d2b76e9022f0d: baseline-arm-kontron 2025-12-17 16:50:10,389 - lava_callback - INFO - Received callback for node 69403c2e575bc060ce158572 INFO: 172.21.0.1:44748 - "POST /node/69403c2e575bc060ce158572 HTTP/1.0" 202 Accepted 2025-12-17 16:50:10,409 - lava_callback - INFO - Starting async job submit for node 69403c2e575bc060ce158572 2025-12-17 16:50:10,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:10,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c2e575bc060ce158572 HTTP/1.1" 200 None 2025-12-17 16:50:10,442 - lava_callback - DEBUG - Processing node 69403c2e575bc060ce158572: baseline-arm64-clabbe 2025-12-17 16:50:10,442 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c2e575bc060ce158572 2025-12-17 16:50:10,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:10,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c2e575bc060ce158572 HTTP/1.1" 200 None 2025-12-17 16:50:10,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:10,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:50:10,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:50:10,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:10,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c2e575bc060ce158572 HTTP/1.1" 200 None 2025-12-17 16:50:10,549 - lava_callback - INFO - Completed processing callback for node 69403c2e575bc060ce158572 2025-12-17 16:50:10,865 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:50:10,866 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-6942decce43d2b76e9022f0d/log.txt.gz 2025-12-17 16:50:10,866 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:50:10,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:50:11,069 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:50:11,070 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-6942decce43d2b76e9022f0d/lava_callback.json.gz 2025-12-17 16:50:11,081 - lava_callback - INFO - Received callback for node 69403c31575bc060ce15857d INFO: 172.21.0.1:44752 - "POST /node/69403c31575bc060ce15857d HTTP/1.0" 202 Accepted 2025-12-17 16:50:11,102 - lava_callback - INFO - Starting async job submit for node 69403c31575bc060ce15857d 2025-12-17 16:50:11,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:44768 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:50:11,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c31575bc060ce15857d HTTP/1.1" 200 None 2025-12-17 16:50:11,136 - lava_callback - DEBUG - Processing node 69403c31575bc060ce15857d: baseline-arm64-clabbe 2025-12-17 16:50:11,137 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c31575bc060ce15857d 2025-12-17 16:50:11,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:11,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:11,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c31575bc060ce15857d HTTP/1.1" 200 None 2025-12-17 16:50:11,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:11,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:50:11,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942decce43d2b76e9022f0d HTTP/1.1" 200 None 2025-12-17 16:50:11,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:11,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:11,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:50:11,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:11,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c31575bc060ce15857d HTTP/1.1" 200 None 2025-12-17 16:50:11,231 - lava_callback - INFO - Completed processing callback for node 69403c31575bc060ce15857d 2025-12-17 16:50:11,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942decce43d2b76e9022f0d HTTP/1.1" 200 None 2025-12-17 16:50:11,286 - lava_callback - INFO - Completed processing callback for node 6942decce43d2b76e9022f0d 2025-12-17 16:50:11,770 - lava_callback - INFO - Received callback for node 69403c34575bc060ce158583 INFO: 172.21.0.1:44778 - "POST /node/69403c34575bc060ce158583 HTTP/1.0" 202 Accepted 2025-12-17 16:50:11,779 - lava_callback - INFO - Starting async job submit for node 69403c34575bc060ce158583 2025-12-17 16:50:11,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:11,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c34575bc060ce158583 HTTP/1.1" 200 None 2025-12-17 16:50:11,790 - lava_callback - DEBUG - Processing node 69403c34575bc060ce158583: baseline-arm64-clabbe 2025-12-17 16:50:11,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c34575bc060ce158583 2025-12-17 16:50:11,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:11,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c34575bc060ce158583 HTTP/1.1" 200 None 2025-12-17 16:50:11,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:11,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad850fb52d1dfff36c5cb HTTP/1.1" 200 None 2025-12-17 16:50:11,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:11,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c34575bc060ce158583 HTTP/1.1" 200 None 2025-12-17 16:50:11,838 - lava_callback - INFO - Completed processing callback for node 69403c34575bc060ce158583 2025-12-17 16:50:12,760 - lava_callback - INFO - Received callback for node 6940389e575bc060ce157f96 INFO: 172.21.0.1:44782 - "POST /node/6940389e575bc060ce157f96 HTTP/1.0" 202 Accepted 2025-12-17 16:50:12,770 - lava_callback - INFO - Starting async job submit for node 6940389e575bc060ce157f96 2025-12-17 16:50:12,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:12,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940389e575bc060ce157f96 HTTP/1.1" 200 None 2025-12-17 16:50:12,854 - lava_callback - DEBUG - Processing node 6940389e575bc060ce157f96: baseline-arm 2025-12-17 16:50:13,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:50:13,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:50:13,672 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6940389e575bc060ce157f96/log.txt.gz 2025-12-17 16:50:13,672 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:50:13,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:50:13,829 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:50:13,830 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6940389e575bc060ce157f96/lava_callback.json.gz DEBUG: Setup failed for 6940389e575bc060ce157f96. Transitting job node result: incomplete -> fail 2025-12-17 16:50:13,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:13,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940389e575bc060ce157f96 HTTP/1.1" 200 None 2025-12-17 16:50:13,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:13,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d8e27c18b79b848b5fc10 HTTP/1.1" 200 None 2025-12-17 16:50:13,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:13,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940389e575bc060ce157f96 HTTP/1.1" 200 None 2025-12-17 16:50:13,995 - lava_callback - INFO - Completed processing callback for node 6940389e575bc060ce157f96 INFO: 172.21.0.1:44794 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:50:16,115 - lava_callback - INFO - Received callback for node 69403c3d575bc060ce158591 INFO: 172.21.0.1:44808 - "POST /node/69403c3d575bc060ce158591 HTTP/1.0" 202 Accepted 2025-12-17 16:50:16,125 - lava_callback - INFO - Starting async job submit for node 69403c3d575bc060ce158591 2025-12-17 16:50:16,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:16,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c3d575bc060ce158591 HTTP/1.1" 200 None 2025-12-17 16:50:16,136 - lava_callback - DEBUG - Processing node 69403c3d575bc060ce158591: baseline-arm-baylibre 2025-12-17 16:50:16,136 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c3d575bc060ce158591 2025-12-17 16:50:16,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:16,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c3d575bc060ce158591 HTTP/1.1" 200 None 2025-12-17 16:50:16,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:16,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:50:16,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:16,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c3d575bc060ce158591 HTTP/1.1" 200 None 2025-12-17 16:50:16,173 - lava_callback - INFO - Completed processing callback for node 69403c3d575bc060ce158591 INFO: 172.21.0.1:44822 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:50:16,784 - lava_callback - INFO - Received callback for node 69403c3f575bc060ce15859a INFO: 172.21.0.1:44824 - "POST /node/69403c3f575bc060ce15859a HTTP/1.0" 202 Accepted 2025-12-17 16:50:16,791 - lava_callback - INFO - Starting async job submit for node 69403c3f575bc060ce15859a 2025-12-17 16:50:16,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:16,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c3f575bc060ce15859a HTTP/1.1" 200 None 2025-12-17 16:50:16,801 - lava_callback - DEBUG - Processing node 69403c3f575bc060ce15859a: baseline-arm-baylibre 2025-12-17 16:50:16,801 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c3f575bc060ce15859a 2025-12-17 16:50:16,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:16,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c3f575bc060ce15859a HTTP/1.1" 200 None 2025-12-17 16:50:16,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:16,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:50:16,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:16,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c3f575bc060ce15859a HTTP/1.1" 200 None 2025-12-17 16:50:16,838 - lava_callback - INFO - Completed processing callback for node 69403c3f575bc060ce15859a 2025-12-17 16:50:17,451 - lava_callback - INFO - Received callback for node 69403c40575bc060ce15859d INFO: 172.21.0.1:44826 - "POST /node/69403c40575bc060ce15859d HTTP/1.0" 202 Accepted 2025-12-17 16:50:17,460 - lava_callback - INFO - Starting async job submit for node 69403c40575bc060ce15859d 2025-12-17 16:50:17,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:17,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c40575bc060ce15859d HTTP/1.1" 200 None 2025-12-17 16:50:17,472 - lava_callback - DEBUG - Processing node 69403c40575bc060ce15859d: baseline-arm-baylibre 2025-12-17 16:50:17,472 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c40575bc060ce15859d 2025-12-17 16:50:17,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:17,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c40575bc060ce15859d HTTP/1.1" 200 None 2025-12-17 16:50:17,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:17,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:50:17,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:17,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c40575bc060ce15859d HTTP/1.1" 200 None 2025-12-17 16:50:17,526 - lava_callback - INFO - Completed processing callback for node 69403c40575bc060ce15859d 2025-12-17 16:50:21,132 - lava_callback - INFO - Received callback for node 6942ded2e43d2b76e9022f49 INFO: 172.21.0.1:50972 - "POST /node/6942ded2e43d2b76e9022f49 HTTP/1.0" 202 Accepted 2025-12-17 16:50:21,139 - lava_callback - INFO - Starting async job submit for node 6942ded2e43d2b76e9022f49 2025-12-17 16:50:21,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:21,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942ded2e43d2b76e9022f49 HTTP/1.1" 200 None 2025-12-17 16:50:21,182 - lava_callback - DEBUG - Processing node 6942ded2e43d2b76e9022f49: ltp-dio 2025-12-17 16:50:21,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:50:21,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:50:21,988 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-6942ded2e43d2b76e9022f49/log.txt.gz 2025-12-17 16:50:21,989 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:50:21,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:50:22,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:50:22,165 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-6942ded2e43d2b76e9022f49/lava_callback.json.gz 2025-12-17 16:50:22,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:22,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942ded2e43d2b76e9022f49 HTTP/1.1" 200 None 2025-12-17 16:50:22,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:22,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:50:22,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:22,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942ded2e43d2b76e9022f49 HTTP/1.1" 200 None 2025-12-17 16:50:22,280 - lava_callback - INFO - Completed processing callback for node 6942ded2e43d2b76e9022f49 INFO: 172.21.0.1:50978 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:50:29,198 - lava_callback - INFO - Received callback for node 69403c41575bc060ce1585a0 INFO: 172.21.0.1:50988 - "POST /node/69403c41575bc060ce1585a0 HTTP/1.0" 202 Accepted 2025-12-17 16:50:29,208 - lava_callback - INFO - Starting async job submit for node 69403c41575bc060ce1585a0 2025-12-17 16:50:29,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:29,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c41575bc060ce1585a0 HTTP/1.1" 200 None 2025-12-17 16:50:29,219 - lava_callback - DEBUG - Processing node 69403c41575bc060ce1585a0: baseline-arm-broonie 2025-12-17 16:50:29,219 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c41575bc060ce1585a0 2025-12-17 16:50:29,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:29,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c41575bc060ce1585a0 HTTP/1.1" 200 None 2025-12-17 16:50:29,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:29,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:50:29,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:29,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c41575bc060ce1585a0 HTTP/1.1" 200 None 2025-12-17 16:50:29,331 - lava_callback - INFO - Completed processing callback for node 69403c41575bc060ce1585a0 2025-12-17 16:50:29,965 - lava_callback - INFO - Received callback for node 69403c54575bc060ce1585c1 INFO: 172.21.0.1:50012 - "POST /node/69403c54575bc060ce1585c1 HTTP/1.0" 202 Accepted 2025-12-17 16:50:29,974 - lava_callback - INFO - Starting async job submit for node 69403c54575bc060ce1585c1 2025-12-17 16:50:29,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:29,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c54575bc060ce1585c1 HTTP/1.1" 200 None 2025-12-17 16:50:29,989 - lava_callback - DEBUG - Processing node 69403c54575bc060ce1585c1: ltp-cve 2025-12-17 16:50:29,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c54575bc060ce1585c1 2025-12-17 16:50:29,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:29,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c54575bc060ce1585c1 HTTP/1.1" 200 None 2025-12-17 16:50:30,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:30,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:50:30,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:30,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c54575bc060ce1585c1 HTTP/1.1" 200 None 2025-12-17 16:50:30,043 - lava_callback - INFO - Completed processing callback for node 69403c54575bc060ce1585c1 2025-12-17 16:50:30,474 - lava_callback - INFO - Received callback for node 69403c43575bc060ce1585a7 INFO: 172.21.0.1:50026 - "POST /node/69403c43575bc060ce1585a7 HTTP/1.0" 202 Accepted 2025-12-17 16:50:30,480 - lava_callback - INFO - Starting async job submit for node 69403c43575bc060ce1585a7 2025-12-17 16:50:30,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:30,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c43575bc060ce1585a7 HTTP/1.1" 200 None 2025-12-17 16:50:30,500 - lava_callback - DEBUG - Processing node 69403c43575bc060ce1585a7: baseline-arm-clabbe 2025-12-17 16:50:30,500 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c43575bc060ce1585a7 2025-12-17 16:50:30,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:30,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c43575bc060ce1585a7 HTTP/1.1" 200 None 2025-12-17 16:50:30,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:30,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:50:30,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:30,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c43575bc060ce1585a7 HTTP/1.1" 200 None 2025-12-17 16:50:30,537 - lava_callback - INFO - Completed processing callback for node 69403c43575bc060ce1585a7 2025-12-17 16:50:31,139 - lava_callback - INFO - Received callback for node 69403c44575bc060ce1585ae INFO: 172.21.0.1:50034 - "POST /node/69403c44575bc060ce1585ae HTTP/1.0" 202 Accepted 2025-12-17 16:50:31,146 - lava_callback - INFO - Starting async job submit for node 69403c44575bc060ce1585ae 2025-12-17 16:50:31,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:31,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c44575bc060ce1585ae HTTP/1.1" 200 None 2025-12-17 16:50:31,156 - lava_callback - DEBUG - Processing node 69403c44575bc060ce1585ae: baseline-arm-clabbe 2025-12-17 16:50:31,157 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c44575bc060ce1585ae 2025-12-17 16:50:31,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:31,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c44575bc060ce1585ae HTTP/1.1" 200 None 2025-12-17 16:50:31,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:31,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:50:31,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:31,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c44575bc060ce1585ae HTTP/1.1" 200 None 2025-12-17 16:50:31,212 - lava_callback - INFO - Completed processing callback for node 69403c44575bc060ce1585ae 2025-12-17 16:50:31,796 - lava_callback - INFO - Received callback for node 69403c45575bc060ce1585b1 INFO: 172.21.0.1:50050 - "POST /node/69403c45575bc060ce1585b1 HTTP/1.0" 202 Accepted 2025-12-17 16:50:31,804 - lava_callback - INFO - Starting async job submit for node 69403c45575bc060ce1585b1 2025-12-17 16:50:31,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:31,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c45575bc060ce1585b1 HTTP/1.1" 200 None 2025-12-17 16:50:31,827 - lava_callback - DEBUG - Processing node 69403c45575bc060ce1585b1: baseline-arm-clabbe 2025-12-17 16:50:31,827 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c45575bc060ce1585b1 2025-12-17 16:50:31,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:31,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c45575bc060ce1585b1 HTTP/1.1" 200 None 2025-12-17 16:50:31,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:31,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:50:31,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:31,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c45575bc060ce1585b1 HTTP/1.1" 200 None 2025-12-17 16:50:31,862 - lava_callback - INFO - Completed processing callback for node 69403c45575bc060ce1585b1 2025-12-17 16:50:32,473 - lava_callback - INFO - Received callback for node 69403c47575bc060ce1585b4 INFO: 172.21.0.1:50060 - "POST /node/69403c47575bc060ce1585b4 HTTP/1.0" 202 Accepted 2025-12-17 16:50:32,479 - lava_callback - INFO - Starting async job submit for node 69403c47575bc060ce1585b4 2025-12-17 16:50:32,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:32,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c47575bc060ce1585b4 HTTP/1.1" 200 None 2025-12-17 16:50:32,494 - lava_callback - DEBUG - Processing node 69403c47575bc060ce1585b4: baseline-arm-clabbe 2025-12-17 16:50:32,494 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c47575bc060ce1585b4 2025-12-17 16:50:32,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:32,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c47575bc060ce1585b4 HTTP/1.1" 200 None 2025-12-17 16:50:32,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:32,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:50:32,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:32,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c47575bc060ce1585b4 HTTP/1.1" 200 None 2025-12-17 16:50:32,529 - lava_callback - INFO - Completed processing callback for node 69403c47575bc060ce1585b4 2025-12-17 16:50:36,488 - lava_callback - INFO - Received callback for node 69403c56575bc060ce1585c6 INFO: 172.21.0.1:50072 - "POST /node/69403c56575bc060ce1585c6 HTTP/1.0" 202 Accepted 2025-12-17 16:50:36,497 - lava_callback - INFO - Starting async job submit for node 69403c56575bc060ce1585c6 2025-12-17 16:50:36,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:36,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c56575bc060ce1585c6 HTTP/1.1" 200 None 2025-12-17 16:50:36,517 - lava_callback - DEBUG - Processing node 69403c56575bc060ce1585c6: baseline-arm-pengutronix 2025-12-17 16:50:36,517 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c56575bc060ce1585c6 2025-12-17 16:50:36,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:36,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c56575bc060ce1585c6 HTTP/1.1" 200 None 2025-12-17 16:50:36,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:36,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:50:36,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:36,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c56575bc060ce1585c6 HTTP/1.1" 200 None 2025-12-17 16:50:36,598 - lava_callback - INFO - Completed processing callback for node 69403c56575bc060ce1585c6 INFO: 172.21.0.1:50080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49080 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:50:48,999 - lava_callback - INFO - Received callback for node 69403c67575bc060ce1585d2 INFO: 172.21.0.1:49094 - "POST /node/69403c67575bc060ce1585d2 HTTP/1.0" 202 Accepted 2025-12-17 16:50:49,008 - lava_callback - INFO - Starting async job submit for node 69403c67575bc060ce1585d2 2025-12-17 16:50:49,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:49,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c67575bc060ce1585d2 HTTP/1.1" 200 None 2025-12-17 16:50:49,020 - lava_callback - DEBUG - Processing node 69403c67575bc060ce1585d2: ltp-fs-bind 2025-12-17 16:50:49,021 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c67575bc060ce1585d2 2025-12-17 16:50:49,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:49,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c67575bc060ce1585d2 HTTP/1.1" 200 None 2025-12-17 16:50:49,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:49,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:50:49,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:49,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c67575bc060ce1585d2 HTTP/1.1" 200 None 2025-12-17 16:50:49,075 - lava_callback - INFO - Completed processing callback for node 69403c67575bc060ce1585d2 2025-12-17 16:50:50,549 - lava_callback - INFO - Received callback for node 69403c58575bc060ce1585cb INFO: 172.21.0.1:49404 - "POST /node/69403c58575bc060ce1585cb HTTP/1.0" 202 Accepted 2025-12-17 16:50:50,558 - lava_callback - INFO - Starting async job submit for node 69403c58575bc060ce1585cb 2025-12-17 16:50:50,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:50,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c58575bc060ce1585cb HTTP/1.1" 200 None 2025-12-17 16:50:50,576 - lava_callback - DEBUG - Processing node 69403c58575bc060ce1585cb: baseline-arm-clabbe 2025-12-17 16:50:50,576 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c58575bc060ce1585cb 2025-12-17 16:50:50,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:50,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c58575bc060ce1585cb HTTP/1.1" 200 None 2025-12-17 16:50:50,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:50,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:50:50,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:50:50,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c58575bc060ce1585cb HTTP/1.1" 200 None 2025-12-17 16:50:50,609 - lava_callback - INFO - Completed processing callback for node 69403c58575bc060ce1585cb INFO: 172.21.0.1:49416 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:51:01,860 - lava_callback - INFO - Received callback for node 69404854575bc060ce158cf9 INFO: 172.21.0.1:36510 - "POST /node/69404854575bc060ce158cf9 HTTP/1.0" 202 Accepted 2025-12-17 16:51:01,870 - lava_callback - INFO - Starting async job submit for node 69404854575bc060ce158cf9 2025-12-17 16:51:01,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:01,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404854575bc060ce158cf9 HTTP/1.1" 200 None 2025-12-17 16:51:01,985 - lava_callback - DEBUG - Processing node 69404854575bc060ce158cf9: baseline-nfs-arm64 2025-12-17 16:51:03,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:51:04,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:51:04,076 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69404854575bc060ce158cf9/log.txt.gz 2025-12-17 16:51:04,076 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:51:04,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:51:04,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:51:04,516 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69404854575bc060ce158cf9/lava_callback.json.gz DEBUG: Setup failed for 69404854575bc060ce158cf9. Transitting job node result: incomplete -> fail 2025-12-17 16:51:04,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:04,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404854575bc060ce158cf9 HTTP/1.1" 200 None 2025-12-17 16:51:04,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:04,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d651ec18b79b848b57427 HTTP/1.1" 200 None 2025-12-17 16:51:04,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:04,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69404854575bc060ce158cf9 HTTP/1.1" 200 None 2025-12-17 16:51:04,717 - lava_callback - INFO - Completed processing callback for node 69404854575bc060ce158cf9 INFO: 172.21.0.1:36512 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:51:09,022 - lava_callback - INFO - Received callback for node 69403c69575bc060ce1585d7 INFO: 172.21.0.1:36526 - "POST /node/69403c69575bc060ce1585d7 HTTP/1.0" 202 Accepted 2025-12-17 16:51:09,031 - lava_callback - INFO - Starting async job submit for node 69403c69575bc060ce1585d7 2025-12-17 16:51:09,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:09,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c69575bc060ce1585d7 HTTP/1.1" 200 None 2025-12-17 16:51:09,044 - lava_callback - DEBUG - Processing node 69403c69575bc060ce1585d7: ltp-fsx 2025-12-17 16:51:09,044 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c69575bc060ce1585d7 2025-12-17 16:51:09,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:09,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c69575bc060ce1585d7 HTTP/1.1" 200 None 2025-12-17 16:51:09,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:09,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:51:09,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:09,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c69575bc060ce1585d7 HTTP/1.1" 200 None 2025-12-17 16:51:09,098 - lava_callback - INFO - Completed processing callback for node 69403c69575bc060ce1585d7 2025-12-17 16:51:09,652 - lava_callback - INFO - Received callback for node 69403c18575bc060ce158553 INFO: 172.21.0.1:36542 - "POST /node/69403c18575bc060ce158553 HTTP/1.0" 202 Accepted 2025-12-17 16:51:09,676 - lava_callback - INFO - Starting async job submit for node 69403c18575bc060ce158553 2025-12-17 16:51:09,747 - lava_callback - INFO - Received callback for node 69403c6a575bc060ce1585de INFO: 172.21.0.1:54440 - "POST /node/69403c6a575bc060ce1585de HTTP/1.0" 202 Accepted 2025-12-17 16:51:09,756 - lava_callback - INFO - Starting async job submit for node 69403c6a575bc060ce1585de 2025-12-17 16:51:09,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:09,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:09,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c18575bc060ce158553 HTTP/1.1" 200 None 2025-12-17 16:51:09,875 - lava_callback - DEBUG - Processing node 69403c18575bc060ce158553: baseline-arm-broonie 2025-12-17 16:51:09,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c6a575bc060ce1585de HTTP/1.1" 200 None 2025-12-17 16:51:09,886 - lava_callback - DEBUG - Processing node 69403c6a575bc060ce1585de: ltp-input 2025-12-17 16:51:09,886 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c6a575bc060ce1585de 2025-12-17 16:51:09,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:09,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c6a575bc060ce1585de HTTP/1.1" 200 None 2025-12-17 16:51:09,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:09,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:51:09,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:10,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c6a575bc060ce1585de HTTP/1.1" 200 None 2025-12-17 16:51:10,060 - lava_callback - INFO - Completed processing callback for node 69403c6a575bc060ce1585de 2025-12-17 16:51:10,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:51:10,476 - lava_callback - INFO - Received callback for node 69403c6b575bc060ce1585e5 INFO: 172.21.0.1:54452 - "POST /node/69403c6b575bc060ce1585e5 HTTP/1.0" 202 Accepted 2025-12-17 16:51:10,488 - lava_callback - INFO - Starting async job submit for node 69403c6b575bc060ce1585e5 2025-12-17 16:51:10,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:10,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c6b575bc060ce1585e5 HTTP/1.1" 200 None 2025-12-17 16:51:10,501 - lava_callback - DEBUG - Processing node 69403c6b575bc060ce1585e5: ltp-ipc 2025-12-17 16:51:10,501 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c6b575bc060ce1585e5 2025-12-17 16:51:10,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:10,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c6b575bc060ce1585e5 HTTP/1.1" 200 None 2025-12-17 16:51:10,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:10,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:51:10,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:10,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c6b575bc060ce1585e5 HTTP/1.1" 200 None 2025-12-17 16:51:10,573 - lava_callback - INFO - Completed processing callback for node 69403c6b575bc060ce1585e5 2025-12-17 16:51:10,899 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:51:10,900 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69403c18575bc060ce158553/log.txt.gz 2025-12-17 16:51:10,900 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:51:10,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:51:11,050 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:51:11,051 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69403c18575bc060ce158553/lava_callback.json.gz 2025-12-17 16:51:11,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:54454 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:51:11,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c18575bc060ce158553 HTTP/1.1" 200 None 2025-12-17 16:51:11,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:11,264 - lava_callback - INFO - Received callback for node 69403c6c575bc060ce1585ea INFO: 172.21.0.1:54470 - "POST /node/69403c6c575bc060ce1585ea HTTP/1.0" 202 Accepted 2025-12-17 16:51:11,274 - lava_callback - INFO - Starting async job submit for node 69403c6c575bc060ce1585ea 2025-12-17 16:51:11,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:11,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693aeba3fee8383e92b66614 HTTP/1.1" 200 None 2025-12-17 16:51:11,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:11,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c6c575bc060ce1585ea HTTP/1.1" 200 None 2025-12-17 16:51:11,294 - lava_callback - DEBUG - Processing node 69403c6c575bc060ce1585ea: ltp-pty 2025-12-17 16:51:11,294 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c6c575bc060ce1585ea 2025-12-17 16:51:11,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:11,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c6c575bc060ce1585ea HTTP/1.1" 200 None 2025-12-17 16:51:11,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:11,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c18575bc060ce158553 HTTP/1.1" 200 None 2025-12-17 16:51:11,329 - lava_callback - INFO - Completed processing callback for node 69403c18575bc060ce158553 2025-12-17 16:51:11,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:51:11,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:11,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c6c575bc060ce1585ea HTTP/1.1" 200 None 2025-12-17 16:51:11,398 - lava_callback - INFO - Completed processing callback for node 69403c6c575bc060ce1585ea 2025-12-17 16:51:11,998 - lava_callback - INFO - Received callback for node 69403c6e575bc060ce1585f1 INFO: 172.21.0.1:54484 - "POST /node/69403c6e575bc060ce1585f1 HTTP/1.0" 202 Accepted 2025-12-17 16:51:12,012 - lava_callback - INFO - Starting async job submit for node 69403c6e575bc060ce1585f1 2025-12-17 16:51:12,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:12,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c6e575bc060ce1585f1 HTTP/1.1" 200 None 2025-12-17 16:51:12,024 - lava_callback - DEBUG - Processing node 69403c6e575bc060ce1585f1: ltp-sched 2025-12-17 16:51:12,025 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c6e575bc060ce1585f1 2025-12-17 16:51:12,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:12,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c6e575bc060ce1585f1 HTTP/1.1" 200 None 2025-12-17 16:51:12,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:12,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:51:12,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:12,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c6e575bc060ce1585f1 HTTP/1.1" 200 None 2025-12-17 16:51:12,072 - lava_callback - INFO - Completed processing callback for node 69403c6e575bc060ce1585f1 2025-12-17 16:51:12,789 - lava_callback - INFO - Received callback for node 69403c6f575bc060ce1585f8 INFO: 172.21.0.1:54500 - "POST /node/69403c6f575bc060ce1585f8 HTTP/1.0" 202 Accepted 2025-12-17 16:51:12,811 - lava_callback - INFO - Starting async job submit for node 69403c6f575bc060ce1585f8 2025-12-17 16:51:12,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:12,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c6f575bc060ce1585f8 HTTP/1.1" 200 None 2025-12-17 16:51:12,826 - lava_callback - DEBUG - Processing node 69403c6f575bc060ce1585f8: ltp-smoketest 2025-12-17 16:51:12,826 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c6f575bc060ce1585f8 2025-12-17 16:51:12,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:12,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c6f575bc060ce1585f8 HTTP/1.1" 200 None 2025-12-17 16:51:12,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:12,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:51:12,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:12,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c6f575bc060ce1585f8 HTTP/1.1" 200 None 2025-12-17 16:51:12,887 - lava_callback - INFO - Completed processing callback for node 69403c6f575bc060ce1585f8 2025-12-17 16:51:13,533 - lava_callback - INFO - Received callback for node 69403c70575bc060ce1585fb INFO: 172.21.0.1:54508 - "POST /node/69403c70575bc060ce1585fb HTTP/1.0" 202 Accepted 2025-12-17 16:51:13,547 - lava_callback - INFO - Starting async job submit for node 69403c70575bc060ce1585fb 2025-12-17 16:51:13,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:13,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c70575bc060ce1585fb HTTP/1.1" 200 None 2025-12-17 16:51:13,568 - lava_callback - DEBUG - Processing node 69403c70575bc060ce1585fb: ltp-syscalls 2025-12-17 16:51:13,568 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c70575bc060ce1585fb 2025-12-17 16:51:13,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:13,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c70575bc060ce1585fb HTTP/1.1" 200 None 2025-12-17 16:51:13,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:13,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:51:13,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:13,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c70575bc060ce1585fb HTTP/1.1" 200 None 2025-12-17 16:51:13,627 - lava_callback - INFO - Completed processing callback for node 69403c70575bc060ce1585fb 2025-12-17 16:51:14,307 - lava_callback - INFO - Received callback for node 69403c72575bc060ce1585fe INFO: 172.21.0.1:54512 - "POST /node/69403c72575bc060ce1585fe HTTP/1.0" 202 Accepted 2025-12-17 16:51:14,313 - lava_callback - INFO - Starting async job submit for node 69403c72575bc060ce1585fe 2025-12-17 16:51:14,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:14,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c72575bc060ce1585fe HTTP/1.1" 200 None 2025-12-17 16:51:14,322 - lava_callback - DEBUG - Processing node 69403c72575bc060ce1585fe: ltp-syscalls-ipc 2025-12-17 16:51:14,322 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c72575bc060ce1585fe 2025-12-17 16:51:14,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:14,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c72575bc060ce1585fe HTTP/1.1" 200 None 2025-12-17 16:51:14,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:14,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:51:14,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:14,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c72575bc060ce1585fe HTTP/1.1" 200 None 2025-12-17 16:51:14,363 - lava_callback - INFO - Completed processing callback for node 69403c72575bc060ce1585fe 2025-12-17 16:51:15,089 - lava_callback - INFO - Received callback for node 69403c73575bc060ce158601 INFO: 172.21.0.1:54528 - "POST /node/69403c73575bc060ce158601 HTTP/1.0" 202 Accepted 2025-12-17 16:51:15,098 - lava_callback - INFO - Starting async job submit for node 69403c73575bc060ce158601 2025-12-17 16:51:15,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:15,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c73575bc060ce158601 HTTP/1.1" 200 None 2025-12-17 16:51:15,116 - lava_callback - DEBUG - Processing node 69403c73575bc060ce158601: ltp-timers 2025-12-17 16:51:15,116 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c73575bc060ce158601 2025-12-17 16:51:15,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:15,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c73575bc060ce158601 HTTP/1.1" 200 None 2025-12-17 16:51:15,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:15,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:51:15,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:54538 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:51:15,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c73575bc060ce158601 HTTP/1.1" 200 None 2025-12-17 16:51:15,176 - lava_callback - INFO - Completed processing callback for node 69403c73575bc060ce158601 2025-12-17 16:51:15,828 - lava_callback - INFO - Received callback for node 69403c74575bc060ce158604 INFO: 172.21.0.1:54552 - "POST /node/69403c74575bc060ce158604 HTTP/1.0" 202 Accepted 2025-12-17 16:51:15,838 - lava_callback - INFO - Starting async job submit for node 69403c74575bc060ce158604 2025-12-17 16:51:15,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:15,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c74575bc060ce158604 HTTP/1.1" 200 None 2025-12-17 16:51:15,855 - lava_callback - DEBUG - Processing node 69403c74575bc060ce158604: ltp-watchqueue 2025-12-17 16:51:15,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c74575bc060ce158604 2025-12-17 16:51:15,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:15,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c74575bc060ce158604 HTTP/1.1" 200 None 2025-12-17 16:51:15,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:15,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 16:51:15,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:15,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c74575bc060ce158604 HTTP/1.1" 200 None 2025-12-17 16:51:15,920 - lava_callback - INFO - Completed processing callback for node 69403c74575bc060ce158604 INFO: 172.21.0.1:54568 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:51:16,553 - lava_callback - INFO - Received callback for node 69403c76575bc060ce158607 INFO: 172.21.0.1:54574 - "POST /node/69403c76575bc060ce158607 HTTP/1.0" 202 Accepted 2025-12-17 16:51:16,559 - lava_callback - INFO - Starting async job submit for node 69403c76575bc060ce158607 2025-12-17 16:51:16,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:16,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c76575bc060ce158607 HTTP/1.1" 200 None 2025-12-17 16:51:16,569 - lava_callback - DEBUG - Processing node 69403c76575bc060ce158607: ltp-syscalls 2025-12-17 16:51:16,570 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c76575bc060ce158607 2025-12-17 16:51:16,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:16,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c76575bc060ce158607 HTTP/1.1" 200 None 2025-12-17 16:51:16,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:16,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693aeba3fee8383e92b66614 HTTP/1.1" 200 None 2025-12-17 16:51:16,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:16,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c76575bc060ce158607 HTTP/1.1" 200 None 2025-12-17 16:51:16,605 - lava_callback - INFO - Completed processing callback for node 69403c76575bc060ce158607 2025-12-17 16:51:24,096 - lava_callback - INFO - Received callback for node 6942decae43d2b76e9022ef6 INFO: 172.21.0.1:60960 - "POST /node/6942decae43d2b76e9022ef6 HTTP/1.0" 202 Accepted 2025-12-17 16:51:24,107 - lava_callback - INFO - Starting async job submit for node 6942decae43d2b76e9022ef6 2025-12-17 16:51:24,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:24,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942decae43d2b76e9022ef6 HTTP/1.1" 200 None 2025-12-17 16:51:24,161 - lava_callback - DEBUG - Processing node 6942decae43d2b76e9022ef6: baseline-arm-clabbe 2025-12-17 16:51:24,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:51:24,782 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:51:24,783 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-6942decae43d2b76e9022ef6/log.txt.gz 2025-12-17 16:51:24,783 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:51:24,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:51:25,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:51:25,258 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-6942decae43d2b76e9022ef6/lava_callback.json.gz 2025-12-17 16:51:25,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:25,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942decae43d2b76e9022ef6 HTTP/1.1" 200 None 2025-12-17 16:51:25,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:25,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:51:25,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:25,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942decae43d2b76e9022ef6 HTTP/1.1" 200 None 2025-12-17 16:51:25,351 - lava_callback - INFO - Completed processing callback for node 6942decae43d2b76e9022ef6 INFO: 172.21.0.1:60970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48042 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:51:48,781 - lava_callback - INFO - Received callback for node 6942debee43d2b76e9022e91 INFO: 172.21.0.1:48048 - "POST /node/6942debee43d2b76e9022e91 HTTP/1.0" 202 Accepted 2025-12-17 16:51:48,792 - lava_callback - INFO - Starting async job submit for node 6942debee43d2b76e9022e91 2025-12-17 16:51:48,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:48,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942debee43d2b76e9022e91 HTTP/1.1" 200 None 2025-12-17 16:51:48,846 - lava_callback - DEBUG - Processing node 6942debee43d2b76e9022e91: baseline-arm 2025-12-17 16:51:48,964 - lava_callback - INFO - Received callback for node 69403c99575bc060ce158613 INFO: 172.21.0.1:48058 - "POST /node/69403c99575bc060ce158613 HTTP/1.0" 202 Accepted 2025-12-17 16:51:48,976 - lava_callback - INFO - Starting async job submit for node 69403c99575bc060ce158613 2025-12-17 16:51:48,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:49,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c99575bc060ce158613 HTTP/1.1" 200 None 2025-12-17 16:51:49,052 - lava_callback - DEBUG - Processing node 69403c99575bc060ce158613: ltp-sched 2025-12-17 16:51:49,052 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c99575bc060ce158613 2025-12-17 16:51:49,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:49,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c99575bc060ce158613 HTTP/1.1" 200 None 2025-12-17 16:51:49,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:49,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d8e27c18b79b848b5fc10 HTTP/1.1" 200 None 2025-12-17 16:51:49,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:49,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c99575bc060ce158613 HTTP/1.1" 200 None 2025-12-17 16:51:49,196 - lava_callback - INFO - Completed processing callback for node 69403c99575bc060ce158613 2025-12-17 16:51:49,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:51:49,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:51:49,662 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942debee43d2b76e9022e91/log.txt.gz 2025-12-17 16:51:49,663 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:51:49,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:51:49,835 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:51:49,836 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942debee43d2b76e9022e91/lava_callback.json.gz 2025-12-17 16:51:49,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:49,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942debee43d2b76e9022e91 HTTP/1.1" 200 None 2025-12-17 16:51:49,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:49,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:51:49,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:49,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942debee43d2b76e9022e91 HTTP/1.1" 200 None 2025-12-17 16:51:49,999 - lava_callback - INFO - Completed processing callback for node 6942debee43d2b76e9022e91 2025-12-17 16:51:56,228 - lava_callback - INFO - Received callback for node 69403ca1575bc060ce158621 INFO: 172.21.0.1:33666 - "POST /node/69403ca1575bc060ce158621 HTTP/1.0" 202 Accepted 2025-12-17 16:51:56,234 - lava_callback - INFO - Starting async job submit for node 69403ca1575bc060ce158621 2025-12-17 16:51:56,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:56,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ca1575bc060ce158621 HTTP/1.1" 200 None 2025-12-17 16:51:56,242 - lava_callback - DEBUG - Processing node 69403ca1575bc060ce158621: baseline-arm-baylibre 2025-12-17 16:51:56,243 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ca1575bc060ce158621 2025-12-17 16:51:56,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:56,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ca1575bc060ce158621 HTTP/1.1" 200 None 2025-12-17 16:51:56,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:56,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:51:56,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:56,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ca1575bc060ce158621 HTTP/1.1" 200 None 2025-12-17 16:51:56,272 - lava_callback - INFO - Completed processing callback for node 69403ca1575bc060ce158621 INFO: 172.21.0.1:33680 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:51:56,661 - lava_callback - INFO - Received callback for node 69403c9a575bc060ce158618 INFO: 172.21.0.1:33686 - "POST /node/69403c9a575bc060ce158618 HTTP/1.0" 202 Accepted 2025-12-17 16:51:56,666 - lava_callback - INFO - Starting async job submit for node 69403c9a575bc060ce158618 2025-12-17 16:51:56,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:56,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c9a575bc060ce158618 HTTP/1.1" 200 None 2025-12-17 16:51:56,676 - lava_callback - DEBUG - Processing node 69403c9a575bc060ce158618: baseline-arm 2025-12-17 16:51:56,676 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403c9a575bc060ce158618 2025-12-17 16:51:56,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:56,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c9a575bc060ce158618 HTTP/1.1" 200 None 2025-12-17 16:51:56,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:56,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:51:56,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:56,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c9a575bc060ce158618 HTTP/1.1" 200 None 2025-12-17 16:51:56,705 - lava_callback - INFO - Completed processing callback for node 69403c9a575bc060ce158618 2025-12-17 16:51:56,889 - lava_callback - INFO - Received callback for node 69403ca3575bc060ce158628 INFO: 172.21.0.1:33688 - "POST /node/69403ca3575bc060ce158628 HTTP/1.0" 202 Accepted 2025-12-17 16:51:56,895 - lava_callback - INFO - Starting async job submit for node 69403ca3575bc060ce158628 2025-12-17 16:51:56,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:56,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ca3575bc060ce158628 HTTP/1.1" 200 None 2025-12-17 16:51:56,904 - lava_callback - DEBUG - Processing node 69403ca3575bc060ce158628: baseline-arm-baylibre 2025-12-17 16:51:56,904 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ca3575bc060ce158628 2025-12-17 16:51:56,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:56,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ca3575bc060ce158628 HTTP/1.1" 200 None 2025-12-17 16:51:56,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:56,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:51:56,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:56,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ca3575bc060ce158628 HTTP/1.1" 200 None 2025-12-17 16:51:56,938 - lava_callback - INFO - Completed processing callback for node 69403ca3575bc060ce158628 2025-12-17 16:51:57,543 - lava_callback - INFO - Received callback for node 69403ca4575bc060ce15862d INFO: 172.21.0.1:33700 - "POST /node/69403ca4575bc060ce15862d HTTP/1.0" 202 Accepted 2025-12-17 16:51:57,550 - lava_callback - INFO - Starting async job submit for node 69403ca4575bc060ce15862d 2025-12-17 16:51:57,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:57,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ca4575bc060ce15862d HTTP/1.1" 200 None 2025-12-17 16:51:57,560 - lava_callback - DEBUG - Processing node 69403ca4575bc060ce15862d: baseline-arm-baylibre 2025-12-17 16:51:57,560 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ca4575bc060ce15862d 2025-12-17 16:51:57,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:57,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ca4575bc060ce15862d HTTP/1.1" 200 None 2025-12-17 16:51:57,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:57,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:51:57,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:57,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ca4575bc060ce15862d HTTP/1.1" 200 None 2025-12-17 16:51:57,596 - lava_callback - INFO - Completed processing callback for node 69403ca4575bc060ce15862d 2025-12-17 16:51:58,207 - lava_callback - INFO - Received callback for node 69403ca5575bc060ce158630 INFO: 172.21.0.1:33706 - "POST /node/69403ca5575bc060ce158630 HTTP/1.0" 202 Accepted 2025-12-17 16:51:58,212 - lava_callback - INFO - Starting async job submit for node 69403ca5575bc060ce158630 2025-12-17 16:51:58,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:58,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ca5575bc060ce158630 HTTP/1.1" 200 None 2025-12-17 16:51:58,222 - lava_callback - DEBUG - Processing node 69403ca5575bc060ce158630: baseline-arm-baylibre 2025-12-17 16:51:58,222 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ca5575bc060ce158630 2025-12-17 16:51:58,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:58,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ca5575bc060ce158630 HTTP/1.1" 200 None 2025-12-17 16:51:58,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:58,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:51:58,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:51:58,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ca5575bc060ce158630 HTTP/1.1" 200 None 2025-12-17 16:51:58,253 - lava_callback - INFO - Completed processing callback for node 69403ca5575bc060ce158630 INFO: 172.21.0.1:36884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49456 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:52:16,193 - lava_callback - INFO - Received callback for node 69403cb5575bc060ce158635 INFO: 172.21.0.1:49466 - "POST /node/69403cb5575bc060ce158635 HTTP/1.0" 202 Accepted 2025-12-17 16:52:16,198 - lava_callback - INFO - Starting async job submit for node 69403cb5575bc060ce158635 2025-12-17 16:52:16,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:16,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403cb5575bc060ce158635 HTTP/1.1" 200 None 2025-12-17 16:52:16,210 - lava_callback - DEBUG - Processing node 69403cb5575bc060ce158635: baseline-arm-baylibre 2025-12-17 16:52:16,210 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403cb5575bc060ce158635 2025-12-17 16:52:16,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:16,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403cb5575bc060ce158635 HTTP/1.1" 200 None 2025-12-17 16:52:16,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:16,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:52:16,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:16,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403cb5575bc060ce158635 HTTP/1.1" 200 None 2025-12-17 16:52:16,245 - lava_callback - INFO - Completed processing callback for node 69403cb5575bc060ce158635 INFO: 172.21.0.1:49472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56834 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:52:36,247 - lava_callback - INFO - Received callback for node 69403cc9575bc060ce158693 INFO: 172.21.0.1:37562 - "POST /node/69403cc9575bc060ce158693 HTTP/1.0" 202 Accepted 2025-12-17 16:52:36,253 - lava_callback - INFO - Starting async job submit for node 69403cc9575bc060ce158693 2025-12-17 16:52:36,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:36,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403cc9575bc060ce158693 HTTP/1.1" 200 None 2025-12-17 16:52:36,270 - lava_callback - DEBUG - Processing node 69403cc9575bc060ce158693: baseline-arm-baylibre 2025-12-17 16:52:36,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403cc9575bc060ce158693 2025-12-17 16:52:36,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:36,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403cc9575bc060ce158693 HTTP/1.1" 200 None 2025-12-17 16:52:36,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:36,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:52:36,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:36,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403cc9575bc060ce158693 HTTP/1.1" 200 None 2025-12-17 16:52:36,496 - lava_callback - INFO - Completed processing callback for node 69403cc9575bc060ce158693 INFO: 172.21.0.1:37566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56418 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:52:49,781 - lava_callback - INFO - Received callback for node 69403ccc575bc060ce158698 INFO: 172.21.0.1:52606 - "POST /node/69403ccc575bc060ce158698 HTTP/1.0" 202 Accepted 2025-12-17 16:52:49,787 - lava_callback - INFO - Starting async job submit for node 69403ccc575bc060ce158698 2025-12-17 16:52:49,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:49,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ccc575bc060ce158698 HTTP/1.1" 200 None 2025-12-17 16:52:49,818 - lava_callback - DEBUG - Processing node 69403ccc575bc060ce158698: baseline-arm-broonie 2025-12-17 16:52:50,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:52:50,171 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:52:50,172 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69403ccc575bc060ce158698/log.txt.gz 2025-12-17 16:52:50,173 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:52:50,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:52:50,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:52:50,349 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69403ccc575bc060ce158698/lava_callback.json.gz 2025-12-17 16:52:50,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:50,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ccc575bc060ce158698 HTTP/1.1" 200 None 2025-12-17 16:52:50,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:50,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:52:50,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:50,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ccc575bc060ce158698 HTTP/1.1" 200 None 2025-12-17 16:52:50,408 - lava_callback - INFO - Completed processing callback for node 69403ccc575bc060ce158698 2025-12-17 16:52:55,144 - lava_callback - INFO - Received callback for node 6942decbe43d2b76e9022efe INFO: 172.21.0.1:52618 - "POST /node/6942decbe43d2b76e9022efe HTTP/1.0" 202 Accepted 2025-12-17 16:52:55,150 - lava_callback - INFO - Starting async job submit for node 6942decbe43d2b76e9022efe 2025-12-17 16:52:55,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:55,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942decbe43d2b76e9022efe HTTP/1.1" 200 None 2025-12-17 16:52:55,180 - lava_callback - DEBUG - Processing node 6942decbe43d2b76e9022efe: baseline-arm-clabbe 2025-12-17 16:52:55,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:52:55,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:52:55,537 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-6942decbe43d2b76e9022efe/log.txt.gz 2025-12-17 16:52:55,537 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:52:55,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:52:55,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:52:55,690 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-6942decbe43d2b76e9022efe/lava_callback.json.gz 2025-12-17 16:52:55,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:55,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942decbe43d2b76e9022efe HTTP/1.1" 200 None 2025-12-17 16:52:55,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:55,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:52:55,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:55,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942decbe43d2b76e9022efe HTTP/1.1" 200 None 2025-12-17 16:52:55,753 - lava_callback - INFO - Completed processing callback for node 6942decbe43d2b76e9022efe 2025-12-17 16:52:56,243 - lava_callback - INFO - Received callback for node 69403cdd575bc060ce158801 INFO: 172.21.0.1:52632 - "POST /node/69403cdd575bc060ce158801 HTTP/1.0" 202 Accepted 2025-12-17 16:52:56,250 - lava_callback - INFO - Starting async job submit for node 69403cdd575bc060ce158801 2025-12-17 16:52:56,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:56,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403cdd575bc060ce158801 HTTP/1.1" 200 None 2025-12-17 16:52:56,260 - lava_callback - DEBUG - Processing node 69403cdd575bc060ce158801: baseline-arm-baylibre 2025-12-17 16:52:56,260 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403cdd575bc060ce158801 2025-12-17 16:52:56,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:56,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403cdd575bc060ce158801 HTTP/1.1" 200 None 2025-12-17 16:52:56,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:56,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:52:56,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:56,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403cdd575bc060ce158801 HTTP/1.1" 200 None 2025-12-17 16:52:56,297 - lava_callback - INFO - Completed processing callback for node 69403cdd575bc060ce158801 2025-12-17 16:52:56,322 - lava_callback - INFO - Received callback for node 69403cd6575bc060ce15869d INFO: 172.21.0.1:52642 - "POST /node/69403cd6575bc060ce15869d HTTP/1.0" 202 Accepted 2025-12-17 16:52:56,331 - lava_callback - INFO - Starting async job submit for node 69403cd6575bc060ce15869d 2025-12-17 16:52:56,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:56,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403cd6575bc060ce15869d HTTP/1.1" 200 None 2025-12-17 16:52:56,348 - lava_callback - DEBUG - Processing node 69403cd6575bc060ce15869d: baseline-arm 2025-12-17 16:52:56,348 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403cd6575bc060ce15869d 2025-12-17 16:52:56,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:56,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403cd6575bc060ce15869d HTTP/1.1" 200 None 2025-12-17 16:52:56,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:56,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:52:56,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:52:56,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403cd6575bc060ce15869d HTTP/1.1" 200 None 2025-12-17 16:52:56,400 - lava_callback - INFO - Completed processing callback for node 69403cd6575bc060ce15869d INFO: 172.21.0.1:52656 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:53:03,501 - lava_callback - INFO - Received callback for node 694048dd575bc060ce158d0c INFO: 172.21.0.1:50404 - "POST /node/694048dd575bc060ce158d0c HTTP/1.0" 202 Accepted 2025-12-17 16:53:03,510 - lava_callback - INFO - Starting async job submit for node 694048dd575bc060ce158d0c 2025-12-17 16:53:03,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:03,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694048dd575bc060ce158d0c HTTP/1.1" 200 None 2025-12-17 16:53:03,581 - lava_callback - DEBUG - Processing node 694048dd575bc060ce158d0c: baseline-nfs-arm64 2025-12-17 16:53:04,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:53:04,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:53:04,898 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694048dd575bc060ce158d0c/log.txt.gz 2025-12-17 16:53:04,898 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:53:04,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:53:05,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:53:05,085 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694048dd575bc060ce158d0c/lava_callback.json.gz DEBUG: Setup failed for 694048dd575bc060ce158d0c. Transitting job node result: incomplete -> fail 2025-12-17 16:53:05,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:05,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694048dd575bc060ce158d0c HTTP/1.1" 200 None 2025-12-17 16:53:05,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:05,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693bd221f552eae6fe2029cc HTTP/1.1" 200 None 2025-12-17 16:53:05,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:05,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694048dd575bc060ce158d0c HTTP/1.1" 200 None 2025-12-17 16:53:05,281 - lava_callback - INFO - Completed processing callback for node 694048dd575bc060ce158d0c INFO: 172.21.0.1:50420 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:53:10,669 - lava_callback - INFO - Received callback for node 69403ce2575bc060ce158806 INFO: 172.21.0.1:35180 - "POST /node/69403ce2575bc060ce158806 HTTP/1.0" 202 Accepted 2025-12-17 16:53:10,674 - lava_callback - INFO - Starting async job submit for node 69403ce2575bc060ce158806 2025-12-17 16:53:10,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:10,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ce2575bc060ce158806 HTTP/1.1" 200 None 2025-12-17 16:53:10,705 - lava_callback - DEBUG - Processing node 69403ce2575bc060ce158806: baseline-arm-clabbe 2025-12-17 16:53:10,705 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ce2575bc060ce158806 2025-12-17 16:53:10,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:10,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ce2575bc060ce158806 HTTP/1.1" 200 None 2025-12-17 16:53:10,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:10,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:53:10,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:10,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ce2575bc060ce158806 HTTP/1.1" 200 None 2025-12-17 16:53:10,748 - lava_callback - INFO - Completed processing callback for node 69403ce2575bc060ce158806 INFO: 172.21.0.1:35194 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:53:11,341 - lava_callback - INFO - Received callback for node 69403ce5575bc060ce15880d INFO: 172.21.0.1:35206 - "POST /node/69403ce5575bc060ce15880d HTTP/1.0" 202 Accepted 2025-12-17 16:53:11,347 - lava_callback - INFO - Starting async job submit for node 69403ce5575bc060ce15880d 2025-12-17 16:53:11,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:11,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ce5575bc060ce15880d HTTP/1.1" 200 None 2025-12-17 16:53:11,356 - lava_callback - DEBUG - Processing node 69403ce5575bc060ce15880d: baseline-arm-clabbe 2025-12-17 16:53:11,356 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ce5575bc060ce15880d 2025-12-17 16:53:11,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:11,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ce5575bc060ce15880d HTTP/1.1" 200 None 2025-12-17 16:53:11,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:11,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:53:11,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:11,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ce5575bc060ce15880d HTTP/1.1" 200 None 2025-12-17 16:53:11,387 - lava_callback - INFO - Completed processing callback for node 69403ce5575bc060ce15880d 2025-12-17 16:53:12,076 - lava_callback - INFO - Received callback for node 69403ce6575bc060ce158810 INFO: 172.21.0.1:35208 - "POST /node/69403ce6575bc060ce158810 HTTP/1.0" 202 Accepted 2025-12-17 16:53:12,081 - lava_callback - INFO - Starting async job submit for node 69403ce6575bc060ce158810 2025-12-17 16:53:12,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:12,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ce6575bc060ce158810 HTTP/1.1" 200 None 2025-12-17 16:53:12,093 - lava_callback - DEBUG - Processing node 69403ce6575bc060ce158810: baseline-arm-clabbe 2025-12-17 16:53:12,093 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ce6575bc060ce158810 2025-12-17 16:53:12,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:12,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ce6575bc060ce158810 HTTP/1.1" 200 None 2025-12-17 16:53:12,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:12,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:53:12,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:12,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ce6575bc060ce158810 HTTP/1.1" 200 None 2025-12-17 16:53:12,148 - lava_callback - INFO - Completed processing callback for node 69403ce6575bc060ce158810 INFO: 172.21.0.1:35210 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:53:16,283 - lava_callback - INFO - Received callback for node 69403cf1575bc060ce15881e INFO: 172.21.0.1:35226 - "POST /node/69403cf1575bc060ce15881e HTTP/1.0" 202 Accepted 2025-12-17 16:53:16,289 - lava_callback - INFO - Starting async job submit for node 69403cf1575bc060ce15881e 2025-12-17 16:53:16,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:35230 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:53:16,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403cf1575bc060ce15881e HTTP/1.1" 200 None 2025-12-17 16:53:16,539 - lava_callback - DEBUG - Processing node 69403cf1575bc060ce15881e: baseline-arm-baylibre 2025-12-17 16:53:16,540 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403cf1575bc060ce15881e 2025-12-17 16:53:16,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:16,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403cf1575bc060ce15881e HTTP/1.1" 200 None 2025-12-17 16:53:16,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:16,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:53:16,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:16,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403cf1575bc060ce15881e HTTP/1.1" 200 None 2025-12-17 16:53:16,599 - lava_callback - INFO - Completed processing callback for node 69403cf1575bc060ce15881e 2025-12-17 16:53:16,965 - lava_callback - INFO - Received callback for node 69403cf3575bc060ce158825 INFO: 172.21.0.1:35236 - "POST /node/69403cf3575bc060ce158825 HTTP/1.0" 202 Accepted 2025-12-17 16:53:16,977 - lava_callback - INFO - Starting async job submit for node 69403cf3575bc060ce158825 2025-12-17 16:53:16,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:16,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403cf3575bc060ce158825 HTTP/1.1" 200 None 2025-12-17 16:53:16,997 - lava_callback - DEBUG - Processing node 69403cf3575bc060ce158825: baseline-arm-baylibre 2025-12-17 16:53:16,997 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403cf3575bc060ce158825 2025-12-17 16:53:16,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:17,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403cf3575bc060ce158825 HTTP/1.1" 200 None 2025-12-17 16:53:17,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:17,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:53:17,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:17,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403cf3575bc060ce158825 HTTP/1.1" 200 None 2025-12-17 16:53:17,043 - lava_callback - INFO - Completed processing callback for node 69403cf3575bc060ce158825 2025-12-17 16:53:17,641 - lava_callback - INFO - Received callback for node 69403cf4575bc060ce158828 INFO: 172.21.0.1:35238 - "POST /node/69403cf4575bc060ce158828 HTTP/1.0" 202 Accepted 2025-12-17 16:53:17,650 - lava_callback - INFO - Starting async job submit for node 69403cf4575bc060ce158828 2025-12-17 16:53:17,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:17,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403cf4575bc060ce158828 HTTP/1.1" 200 None 2025-12-17 16:53:17,658 - lava_callback - DEBUG - Processing node 69403cf4575bc060ce158828: baseline-arm-baylibre 2025-12-17 16:53:17,659 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403cf4575bc060ce158828 2025-12-17 16:53:17,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:17,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403cf4575bc060ce158828 HTTP/1.1" 200 None 2025-12-17 16:53:17,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:17,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:53:17,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:17,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403cf4575bc060ce158828 HTTP/1.1" 200 None 2025-12-17 16:53:17,699 - lava_callback - INFO - Completed processing callback for node 69403cf4575bc060ce158828 INFO: 172.21.0.1:51352 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:53:36,286 - lava_callback - INFO - Received callback for node 69403d05575bc060ce158831 INFO: 172.21.0.1:40908 - "POST /node/69403d05575bc060ce158831 HTTP/1.0" 202 Accepted 2025-12-17 16:53:36,292 - lava_callback - INFO - Starting async job submit for node 69403d05575bc060ce158831 2025-12-17 16:53:36,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:36,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d05575bc060ce158831 HTTP/1.1" 200 None 2025-12-17 16:53:36,301 - lava_callback - DEBUG - Processing node 69403d05575bc060ce158831: baseline-arm-baylibre 2025-12-17 16:53:36,301 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403d05575bc060ce158831 2025-12-17 16:53:36,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:36,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d05575bc060ce158831 HTTP/1.1" 200 None 2025-12-17 16:53:36,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:36,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:53:36,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:36,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403d05575bc060ce158831 HTTP/1.1" 200 None 2025-12-17 16:53:36,336 - lava_callback - INFO - Completed processing callback for node 69403d05575bc060ce158831 INFO: 172.21.0.1:40914 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:53:36,944 - lava_callback - INFO - Received callback for node 69403d07575bc060ce158836 INFO: 172.21.0.1:40924 - "POST /node/69403d07575bc060ce158836 HTTP/1.0" 202 Accepted 2025-12-17 16:53:36,951 - lava_callback - INFO - Starting async job submit for node 69403d07575bc060ce158836 2025-12-17 16:53:36,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:36,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d07575bc060ce158836 HTTP/1.1" 200 None 2025-12-17 16:53:36,962 - lava_callback - DEBUG - Processing node 69403d07575bc060ce158836: baseline-arm-baylibre 2025-12-17 16:53:36,962 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403d07575bc060ce158836 2025-12-17 16:53:36,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:36,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d07575bc060ce158836 HTTP/1.1" 200 None 2025-12-17 16:53:36,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:36,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:53:36,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:36,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403d07575bc060ce158836 HTTP/1.1" 200 None 2025-12-17 16:53:36,997 - lava_callback - INFO - Completed processing callback for node 69403d07575bc060ce158836 INFO: 172.21.0.1:48030 - "GET /node/694025a2575bc060ce154113%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:48032 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:53:48,358 - lava_callback - INFO - Received callback for node 6942dfa7e43d2b76e90236c3 INFO: 172.21.0.1:48042 - "POST /node/6942dfa7e43d2b76e90236c3 HTTP/1.0" 202 Accepted 2025-12-17 16:53:48,367 - lava_callback - INFO - Starting async job submit for node 6942dfa7e43d2b76e90236c3 2025-12-17 16:53:48,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:48,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dfa7e43d2b76e90236c3 HTTP/1.1" 200 None 2025-12-17 16:53:48,404 - lava_callback - DEBUG - Processing node 6942dfa7e43d2b76e90236c3: baseline-arm 2025-12-17 16:53:48,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:53:48,906 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:53:48,907 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942dfa7e43d2b76e90236c3/log.txt.gz 2025-12-17 16:53:48,907 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:53:48,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:53:49,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:53:49,133 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942dfa7e43d2b76e90236c3/lava_callback.json.gz 2025-12-17 16:53:49,147 - lava_callback - INFO - Received callback for node 69403d1b575bc060ce158845 INFO: 172.21.0.1:48054 - "POST /node/69403d1b575bc060ce158845 HTTP/1.0" 202 Accepted 2025-12-17 16:53:49,161 - lava_callback - INFO - Starting async job submit for node 69403d1b575bc060ce158845 2025-12-17 16:53:49,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:49,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:49,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d1b575bc060ce158845 HTTP/1.1" 200 None 2025-12-17 16:53:49,190 - lava_callback - DEBUG - Processing node 69403d1b575bc060ce158845: baseline-arm-broonie 2025-12-17 16:53:49,190 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403d1b575bc060ce158845 2025-12-17 16:53:49,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:49,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942dfa7e43d2b76e90236c3 HTTP/1.1" 200 None 2025-12-17 16:53:49,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:49,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d1b575bc060ce158845 HTTP/1.1" 200 None 2025-12-17 16:53:49,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:49,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 16:53:49,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:49,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:53:49,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:49,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403d1b575bc060ce158845 HTTP/1.1" 200 None 2025-12-17 16:53:49,253 - lava_callback - INFO - Completed processing callback for node 69403d1b575bc060ce158845 2025-12-17 16:53:49,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942dfa7e43d2b76e90236c3 HTTP/1.1" 200 None 2025-12-17 16:53:49,272 - lava_callback - INFO - Completed processing callback for node 6942dfa7e43d2b76e90236c3 2025-12-17 16:53:50,680 - lava_callback - INFO - Received callback for node 69403d0a575bc060ce15883b INFO: 172.21.0.1:54208 - "POST /node/69403d0a575bc060ce15883b HTTP/1.0" 202 Accepted 2025-12-17 16:53:50,689 - lava_callback - INFO - Starting async job submit for node 69403d0a575bc060ce15883b 2025-12-17 16:53:50,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:50,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d0a575bc060ce15883b HTTP/1.1" 200 None 2025-12-17 16:53:50,700 - lava_callback - DEBUG - Processing node 69403d0a575bc060ce15883b: baseline-arm-clabbe 2025-12-17 16:53:50,700 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403d0a575bc060ce15883b 2025-12-17 16:53:50,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:50,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d0a575bc060ce15883b HTTP/1.1" 200 None 2025-12-17 16:53:50,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:50,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:53:50,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:50,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403d0a575bc060ce15883b HTTP/1.1" 200 None 2025-12-17 16:53:50,748 - lava_callback - INFO - Completed processing callback for node 69403d0a575bc060ce15883b 2025-12-17 16:53:51,363 - lava_callback - INFO - Received callback for node 69403d0d575bc060ce158840 INFO: 172.21.0.1:54216 - "POST /node/69403d0d575bc060ce158840 HTTP/1.0" 202 Accepted 2025-12-17 16:53:51,372 - lava_callback - INFO - Starting async job submit for node 69403d0d575bc060ce158840 2025-12-17 16:53:51,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:51,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d0d575bc060ce158840 HTTP/1.1" 200 None 2025-12-17 16:53:51,384 - lava_callback - DEBUG - Processing node 69403d0d575bc060ce158840: baseline-arm-clabbe 2025-12-17 16:53:51,384 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403d0d575bc060ce158840 2025-12-17 16:53:51,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:51,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d0d575bc060ce158840 HTTP/1.1" 200 None 2025-12-17 16:53:51,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:51,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:53:51,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:53:51,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403d0d575bc060ce158840 HTTP/1.1" 200 None 2025-12-17 16:53:51,448 - lava_callback - INFO - Completed processing callback for node 69403d0d575bc060ce158840 INFO: 172.21.0.1:54218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38298 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:54:10,701 - lava_callback - INFO - Received callback for node 69403d1e575bc060ce15884b INFO: 172.21.0.1:34250 - "POST /node/69403d1e575bc060ce15884b HTTP/1.0" 202 Accepted 2025-12-17 16:54:10,710 - lava_callback - INFO - Starting async job submit for node 69403d1e575bc060ce15884b 2025-12-17 16:54:10,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:10,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d1e575bc060ce15884b HTTP/1.1" 200 None 2025-12-17 16:54:10,725 - lava_callback - DEBUG - Processing node 69403d1e575bc060ce15884b: baseline-arm-clabbe 2025-12-17 16:54:10,725 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403d1e575bc060ce15884b 2025-12-17 16:54:10,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:10,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d1e575bc060ce15884b HTTP/1.1" 200 None 2025-12-17 16:54:10,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:10,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:54:10,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:10,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403d1e575bc060ce15884b HTTP/1.1" 200 None 2025-12-17 16:54:10,778 - lava_callback - INFO - Completed processing callback for node 69403d1e575bc060ce15884b INFO: 172.21.0.1:34252 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:54:11,362 - lava_callback - INFO - Received callback for node 69403d21575bc060ce158850 INFO: 172.21.0.1:34260 - "POST /node/69403d21575bc060ce158850 HTTP/1.0" 202 Accepted 2025-12-17 16:54:11,371 - lava_callback - INFO - Starting async job submit for node 69403d21575bc060ce158850 2025-12-17 16:54:11,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:11,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d21575bc060ce158850 HTTP/1.1" 200 None 2025-12-17 16:54:11,386 - lava_callback - DEBUG - Processing node 69403d21575bc060ce158850: baseline-arm-clabbe 2025-12-17 16:54:11,386 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403d21575bc060ce158850 2025-12-17 16:54:11,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:11,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d21575bc060ce158850 HTTP/1.1" 200 None 2025-12-17 16:54:11,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:11,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:54:11,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:11,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403d21575bc060ce158850 HTTP/1.1" 200 None 2025-12-17 16:54:11,435 - lava_callback - INFO - Completed processing callback for node 69403d21575bc060ce158850 INFO: 172.21.0.1:34276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41094 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:54:30,714 - lava_callback - INFO - Received callback for node 69403d33575bc060ce158859 INFO: 172.21.0.1:40150 - "POST /node/69403d33575bc060ce158859 HTTP/1.0" 202 Accepted 2025-12-17 16:54:30,723 - lava_callback - INFO - Starting async job submit for node 69403d33575bc060ce158859 2025-12-17 16:54:30,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:30,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d33575bc060ce158859 HTTP/1.1" 200 None 2025-12-17 16:54:30,735 - lava_callback - DEBUG - Processing node 69403d33575bc060ce158859: baseline-arm-clabbe 2025-12-17 16:54:30,735 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403d33575bc060ce158859 2025-12-17 16:54:30,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:30,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d33575bc060ce158859 HTTP/1.1" 200 None 2025-12-17 16:54:30,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:30,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:54:30,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:30,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403d33575bc060ce158859 HTTP/1.1" 200 None 2025-12-17 16:54:30,793 - lava_callback - INFO - Completed processing callback for node 69403d33575bc060ce158859 2025-12-17 16:54:31,385 - lava_callback - INFO - Received callback for node 69403d35575bc060ce158860 INFO: 172.21.0.1:40154 - "POST /node/69403d35575bc060ce158860 HTTP/1.0" 202 Accepted 2025-12-17 16:54:31,402 - lava_callback - INFO - Starting async job submit for node 69403d35575bc060ce158860 2025-12-17 16:54:31,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:31,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d35575bc060ce158860 HTTP/1.1" 200 None 2025-12-17 16:54:31,429 - lava_callback - DEBUG - Processing node 69403d35575bc060ce158860: baseline-arm-clabbe 2025-12-17 16:54:31,429 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403d35575bc060ce158860 2025-12-17 16:54:31,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:31,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d35575bc060ce158860 HTTP/1.1" 200 None 2025-12-17 16:54:31,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:31,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:54:31,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:31,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403d35575bc060ce158860 HTTP/1.1" 200 None 2025-12-17 16:54:31,497 - lava_callback - INFO - Completed processing callback for node 69403d35575bc060ce158860 2025-12-17 16:54:32,083 - lava_callback - INFO - Received callback for node 69403d37575bc060ce158863 INFO: 172.21.0.1:40158 - "POST /node/69403d37575bc060ce158863 HTTP/1.0" 202 Accepted 2025-12-17 16:54:32,092 - lava_callback - INFO - Starting async job submit for node 69403d37575bc060ce158863 2025-12-17 16:54:32,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:32,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d37575bc060ce158863 HTTP/1.1" 200 None 2025-12-17 16:54:32,118 - lava_callback - DEBUG - Processing node 69403d37575bc060ce158863: baseline-arm-clabbe 2025-12-17 16:54:32,118 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403d37575bc060ce158863 2025-12-17 16:54:32,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:32,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d37575bc060ce158863 HTTP/1.1" 200 None 2025-12-17 16:54:32,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:32,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:54:32,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:32,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403d37575bc060ce158863 HTTP/1.1" 200 None 2025-12-17 16:54:32,163 - lava_callback - INFO - Completed processing callback for node 69403d37575bc060ce158863 INFO: 172.21.0.1:40170 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:54:42,396 - lava_callback - INFO - Received callback for node 69403920575bc060ce158055 INFO: 172.21.0.1:33490 - "POST /node/69403920575bc060ce158055 HTTP/1.0" 202 Accepted 2025-12-17 16:54:42,407 - lava_callback - INFO - Starting async job submit for node 69403920575bc060ce158055 2025-12-17 16:54:42,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:42,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403920575bc060ce158055 HTTP/1.1" 200 None 2025-12-17 16:54:42,557 - lava_callback - DEBUG - Processing node 69403920575bc060ce158055: ltp-smoketest 2025-12-17 16:54:43,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:54:43,703 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:54:43,704 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoketest-69403920575bc060ce158055/log.txt.gz 2025-12-17 16:54:43,704 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:54:43,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:54:44,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:54:44,004 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoketest-69403920575bc060ce158055/lava_callback.json.gz 2025-12-17 16:54:44,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:44,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403920575bc060ce158055 HTTP/1.1" 200 None 2025-12-17 16:54:44,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:44,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922877ecffe173f95b6b HTTP/1.1" 200 None 2025-12-17 16:54:44,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:44,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403920575bc060ce158055 HTTP/1.1" 200 None 2025-12-17 16:54:44,272 - lava_callback - INFO - Completed processing callback for node 69403920575bc060ce158055 INFO: 172.21.0.1:33498 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:54:49,029 - lava_callback - INFO - Received callback for node 69403d44575bc060ce158868 INFO: 172.21.0.1:33514 - "POST /node/69403d44575bc060ce158868 HTTP/1.0" 202 Accepted 2025-12-17 16:54:49,038 - lava_callback - INFO - Starting async job submit for node 69403d44575bc060ce158868 2025-12-17 16:54:49,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:49,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d44575bc060ce158868 HTTP/1.1" 200 None 2025-12-17 16:54:49,054 - lava_callback - DEBUG - Processing node 69403d44575bc060ce158868: ltp-syscalls 2025-12-17 16:54:49,054 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403d44575bc060ce158868 2025-12-17 16:54:49,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:49,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d44575bc060ce158868 HTTP/1.1" 200 None 2025-12-17 16:54:49,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:49,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:54:49,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:49,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403d44575bc060ce158868 HTTP/1.1" 200 None 2025-12-17 16:54:49,102 - lava_callback - INFO - Completed processing callback for node 69403d44575bc060ce158868 INFO: 172.21.0.1:51370 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:54:59,590 - lava_callback - INFO - Received callback for node 69405cac575bc060ce1595d1 INFO: 172.21.0.1:51378 - "POST /node/69405cac575bc060ce1595d1 HTTP/1.0" 202 Accepted 2025-12-17 16:54:59,604 - lava_callback - INFO - Starting async job submit for node 69405cac575bc060ce1595d1 2025-12-17 16:54:59,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:54:59,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69405cac575bc060ce1595d1 HTTP/1.1" 200 None 2025-12-17 16:54:59,666 - lava_callback - DEBUG - Processing node 69405cac575bc060ce1595d1: baseline-arm64 2025-12-17 16:55:00,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:55:00,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:55:00,606 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69405cac575bc060ce1595d1/log.txt.gz 2025-12-17 16:55:00,606 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:55:00,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:55:00,778 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:55:00,779 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69405cac575bc060ce1595d1/lava_callback.json.gz 2025-12-17 16:55:00,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:55:00,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69405cac575bc060ce1595d1 HTTP/1.1" 200 None 2025-12-17 16:55:00,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:55:00,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693f507d8b4d8f471e43f50c HTTP/1.1" 200 None 2025-12-17 16:55:00,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:55:00,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69405cac575bc060ce1595d1 HTTP/1.1" 200 None 2025-12-17 16:55:00,852 - lava_callback - INFO - Completed processing callback for node 69405cac575bc060ce1595d1 INFO: 172.21.0.1:45352 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:55:09,140 - lava_callback - INFO - Received callback for node 69403d58575bc060ce158875 INFO: 172.21.0.1:45368 - "POST /node/69403d58575bc060ce158875 HTTP/1.0" 202 Accepted 2025-12-17 16:55:09,150 - lava_callback - INFO - Starting async job submit for node 69403d58575bc060ce158875 2025-12-17 16:55:09,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:55:09,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d58575bc060ce158875 HTTP/1.1" 200 None 2025-12-17 16:55:09,164 - lava_callback - DEBUG - Processing node 69403d58575bc060ce158875: ltp-syscalls 2025-12-17 16:55:09,164 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403d58575bc060ce158875 2025-12-17 16:55:09,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:55:09,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d58575bc060ce158875 HTTP/1.1" 200 None 2025-12-17 16:55:09,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:55:09,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:55:09,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:55:09,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403d58575bc060ce158875 HTTP/1.1" 200 None 2025-12-17 16:55:09,215 - lava_callback - INFO - Completed processing callback for node 69403d58575bc060ce158875 INFO: 172.21.0.1:50296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50314 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:55:16,738 - lava_callback - INFO - Received callback for node 69403d68575bc060ce158887 INFO: 172.21.0.1:50326 - "POST /node/69403d68575bc060ce158887 HTTP/1.0" 202 Accepted 2025-12-17 16:55:16,746 - lava_callback - INFO - Starting async job submit for node 69403d68575bc060ce158887 2025-12-17 16:55:16,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:55:16,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d68575bc060ce158887 HTTP/1.1" 200 None 2025-12-17 16:55:16,758 - lava_callback - DEBUG - Processing node 69403d68575bc060ce158887: baseline-x86-kcidebug-intel 2025-12-17 16:55:16,758 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403d68575bc060ce158887 2025-12-17 16:55:16,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:55:16,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d68575bc060ce158887 HTTP/1.1" 200 None 2025-12-17 16:55:16,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:55:16,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693aebc7fee8383e92b666f7 HTTP/1.1" 200 None 2025-12-17 16:55:16,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:55:16,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403d68575bc060ce158887 HTTP/1.1" 200 None 2025-12-17 16:55:16,807 - lava_callback - INFO - Completed processing callback for node 69403d68575bc060ce158887 2025-12-17 16:55:17,415 - lava_callback - INFO - Received callback for node 69403d63575bc060ce158880 INFO: 172.21.0.1:50336 - "POST /node/69403d63575bc060ce158880 HTTP/1.0" 202 Accepted 2025-12-17 16:55:17,425 - lava_callback - INFO - Starting async job submit for node 69403d63575bc060ce158880 2025-12-17 16:55:17,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:55:17,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d63575bc060ce158880 HTTP/1.1" 200 None 2025-12-17 16:55:17,440 - lava_callback - DEBUG - Processing node 69403d63575bc060ce158880: baseline-arm 2025-12-17 16:55:17,440 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403d63575bc060ce158880 2025-12-17 16:55:17,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:55:17,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403d63575bc060ce158880 HTTP/1.1" 200 None 2025-12-17 16:55:17,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:55:17,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693aeba3fee8383e92b66614 HTTP/1.1" 200 None 2025-12-17 16:55:17,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:55:17,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403d63575bc060ce158880 HTTP/1.1" 200 None 2025-12-17 16:55:17,568 - lava_callback - INFO - Completed processing callback for node 69403d63575bc060ce158880 INFO: 172.21.0.1:47850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35998 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:56:07,388 - lava_callback - INFO - Received callback for node 69406801575bc060ce159e8b INFO: 172.21.0.1:36000 - "POST /node/69406801575bc060ce159e8b HTTP/1.0" 202 Accepted 2025-12-17 16:56:07,400 - lava_callback - INFO - Starting async job submit for node 69406801575bc060ce159e8b 2025-12-17 16:56:07,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:56:07,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69406801575bc060ce159e8b HTTP/1.1" 200 None 2025-12-17 16:56:07,444 - lava_callback - DEBUG - Processing node 69406801575bc060ce159e8b: baseline-arm64 2025-12-17 16:56:07,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:56:08,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:56:08,289 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69406801575bc060ce159e8b/log.txt.gz 2025-12-17 16:56:08,289 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 16:56:08,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 16:56:08,544 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 16:56:08,545 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69406801575bc060ce159e8b/lava_callback.json.gz 2025-12-17 16:56:08,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:56:08,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69406801575bc060ce159e8b HTTP/1.1" 200 None 2025-12-17 16:56:08,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:56:08,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693f54220d8ce4018da8984f HTTP/1.1" 200 None 2025-12-17 16:56:08,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:56:08,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69406801575bc060ce159e8b HTTP/1.1" 200 None 2025-12-17 16:56:08,642 - lava_callback - INFO - Completed processing callback for node 69406801575bc060ce159e8b INFO: 172.21.0.1:60788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60798 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:56:16,896 - lava_callback - INFO - Received callback for node 69403daf575bc060ce1588b2 INFO: 172.21.0.1:60810 - "POST /node/69403daf575bc060ce1588b2 HTTP/1.0" 202 Accepted 2025-12-17 16:56:16,902 - lava_callback - INFO - Starting async job submit for node 69403daf575bc060ce1588b2 2025-12-17 16:56:16,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:56:16,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403daf575bc060ce1588b2 HTTP/1.1" 200 None 2025-12-17 16:56:16,915 - lava_callback - DEBUG - Processing node 69403daf575bc060ce1588b2: baseline-arm 2025-12-17 16:56:16,915 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403daf575bc060ce1588b2 2025-12-17 16:56:16,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:56:16,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403daf575bc060ce1588b2 HTTP/1.1" 200 None 2025-12-17 16:56:16,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:56:16,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad949fb52d1dfff36c7ab HTTP/1.1" 200 None 2025-12-17 16:56:16,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:56:16,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403daf575bc060ce1588b2 HTTP/1.1" 200 None 2025-12-17 16:56:16,960 - lava_callback - INFO - Completed processing callback for node 69403daf575bc060ce1588b2 INFO: 172.21.0.1:39090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55114 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:57:29,195 - lava_callback - INFO - Received callback for node 69403df1575bc060ce1588cb INFO: 172.21.0.1:55124 - "POST /node/69403df1575bc060ce1588cb HTTP/1.0" 202 Accepted 2025-12-17 16:57:29,201 - lava_callback - INFO - Starting async job submit for node 69403df1575bc060ce1588cb 2025-12-17 16:57:29,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:57:29,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403df1575bc060ce1588cb HTTP/1.1" 200 None 2025-12-17 16:57:29,210 - lava_callback - DEBUG - Processing node 69403df1575bc060ce1588cb: ltp-sched 2025-12-17 16:57:29,210 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403df1575bc060ce1588cb 2025-12-17 16:57:29,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:57:29,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403df1575bc060ce1588cb HTTP/1.1" 200 None 2025-12-17 16:57:29,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:57:29,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af17afee8383e92b69dc1 HTTP/1.1" 200 None 2025-12-17 16:57:29,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:57:29,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403df1575bc060ce1588cb HTTP/1.1" 200 None 2025-12-17 16:57:29,242 - lava_callback - INFO - Completed processing callback for node 69403df1575bc060ce1588cb INFO: 172.21.0.1:42616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55868 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:57:56,472 - lava_callback - INFO - Received callback for node 69403e0a575bc060ce1588da INFO: 172.21.0.1:42170 - "POST /node/69403e0a575bc060ce1588da HTTP/1.0" 202 Accepted 2025-12-17 16:57:56,477 - lava_callback - INFO - Starting async job submit for node 69403e0a575bc060ce1588da 2025-12-17 16:57:56,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:57:56,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e0a575bc060ce1588da HTTP/1.1" 200 None 2025-12-17 16:57:56,487 - lava_callback - DEBUG - Processing node 69403e0a575bc060ce1588da: baseline-arm64-baylibre 2025-12-17 16:57:56,487 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e0a575bc060ce1588da 2025-12-17 16:57:56,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:57:56,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e0a575bc060ce1588da HTTP/1.1" 200 None 2025-12-17 16:57:56,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:57:56,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:57:56,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:57:56,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e0a575bc060ce1588da HTTP/1.1" 200 None 2025-12-17 16:57:56,519 - lava_callback - INFO - Completed processing callback for node 69403e0a575bc060ce1588da INFO: 172.21.0.1:42184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37532 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 16:58:16,475 - lava_callback - INFO - Received callback for node 69403e20575bc060ce1588fd INFO: 172.21.0.1:37536 - "POST /node/69403e20575bc060ce1588fd HTTP/1.0" 202 Accepted 2025-12-17 16:58:16,481 - lava_callback - INFO - Starting async job submit for node 69403e20575bc060ce1588fd 2025-12-17 16:58:16,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:16,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e20575bc060ce1588fd HTTP/1.1" 200 None 2025-12-17 16:58:16,491 - lava_callback - DEBUG - Processing node 69403e20575bc060ce1588fd: baseline-arm64-baylibre 2025-12-17 16:58:16,491 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e20575bc060ce1588fd 2025-12-17 16:58:16,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:16,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e20575bc060ce1588fd HTTP/1.1" 200 None 2025-12-17 16:58:16,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:16,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:58:16,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:37548 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:58:16,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e20575bc060ce1588fd HTTP/1.1" 200 None 2025-12-17 16:58:16,526 - lava_callback - INFO - Completed processing callback for node 69403e20575bc060ce1588fd 2025-12-17 16:58:16,946 - lava_callback - INFO - Received callback for node 69403e17575bc060ce1588e7 INFO: 172.21.0.1:37552 - "POST /node/69403e17575bc060ce1588e7 HTTP/1.0" 202 Accepted 2025-12-17 16:58:16,951 - lava_callback - INFO - Starting async job submit for node 69403e17575bc060ce1588e7 2025-12-17 16:58:16,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:16,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e17575bc060ce1588e7 HTTP/1.1" 200 None 2025-12-17 16:58:16,961 - lava_callback - DEBUG - Processing node 69403e17575bc060ce1588e7: baseline-x86 2025-12-17 16:58:16,962 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e17575bc060ce1588e7 2025-12-17 16:58:16,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:16,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e17575bc060ce1588e7 HTTP/1.1" 200 None 2025-12-17 16:58:16,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:16,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d5fad9ee307f5b1e66 HTTP/1.1" 200 None 2025-12-17 16:58:16,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:16,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e17575bc060ce1588e7 HTTP/1.1" 200 None 2025-12-17 16:58:16,996 - lava_callback - INFO - Completed processing callback for node 69403e17575bc060ce1588e7 2025-12-17 16:58:17,122 - lava_callback - INFO - Received callback for node 69403e23575bc060ce158900 INFO: 172.21.0.1:37554 - "POST /node/69403e23575bc060ce158900 HTTP/1.0" 202 Accepted 2025-12-17 16:58:17,131 - lava_callback - INFO - Starting async job submit for node 69403e23575bc060ce158900 2025-12-17 16:58:17,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:17,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e23575bc060ce158900 HTTP/1.1" 200 None 2025-12-17 16:58:17,142 - lava_callback - DEBUG - Processing node 69403e23575bc060ce158900: baseline-arm64-baylibre 2025-12-17 16:58:17,142 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e23575bc060ce158900 2025-12-17 16:58:17,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:17,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e23575bc060ce158900 HTTP/1.1" 200 None 2025-12-17 16:58:17,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:17,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:58:17,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:17,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e23575bc060ce158900 HTTP/1.1" 200 None 2025-12-17 16:58:17,179 - lava_callback - INFO - Completed processing callback for node 69403e23575bc060ce158900 2025-12-17 16:58:17,626 - lava_callback - INFO - Received callback for node 69403e1a575bc060ce1588ea INFO: 172.21.0.1:37566 - "POST /node/69403e1a575bc060ce1588ea HTTP/1.0" 202 Accepted 2025-12-17 16:58:17,631 - lava_callback - INFO - Starting async job submit for node 69403e1a575bc060ce1588ea 2025-12-17 16:58:17,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:17,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e1a575bc060ce1588ea HTTP/1.1" 200 None 2025-12-17 16:58:17,640 - lava_callback - DEBUG - Processing node 69403e1a575bc060ce1588ea: baseline-x86 2025-12-17 16:58:17,641 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e1a575bc060ce1588ea 2025-12-17 16:58:17,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:17,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e1a575bc060ce1588ea HTTP/1.1" 200 None 2025-12-17 16:58:17,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:17,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d5fad9ee307f5b1e66 HTTP/1.1" 200 None 2025-12-17 16:58:17,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:17,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e1a575bc060ce1588ea HTTP/1.1" 200 None 2025-12-17 16:58:17,675 - lava_callback - INFO - Completed processing callback for node 69403e1a575bc060ce1588ea 2025-12-17 16:58:17,762 - lava_callback - INFO - Received callback for node 69403e24575bc060ce158903 INFO: 172.21.0.1:37576 - "POST /node/69403e24575bc060ce158903 HTTP/1.0" 202 Accepted 2025-12-17 16:58:17,771 - lava_callback - INFO - Starting async job submit for node 69403e24575bc060ce158903 2025-12-17 16:58:17,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:17,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e24575bc060ce158903 HTTP/1.1" 200 None 2025-12-17 16:58:17,781 - lava_callback - DEBUG - Processing node 69403e24575bc060ce158903: baseline-arm64-baylibre 2025-12-17 16:58:17,781 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e24575bc060ce158903 2025-12-17 16:58:17,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:17,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e24575bc060ce158903 HTTP/1.1" 200 None 2025-12-17 16:58:17,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:17,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:58:17,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:17,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e24575bc060ce158903 HTTP/1.1" 200 None 2025-12-17 16:58:17,819 - lava_callback - INFO - Completed processing callback for node 69403e24575bc060ce158903 2025-12-17 16:58:18,298 - lava_callback - INFO - Received callback for node 69403e1d575bc060ce1588ed INFO: 172.21.0.1:37588 - "POST /node/69403e1d575bc060ce1588ed HTTP/1.0" 202 Accepted 2025-12-17 16:58:18,306 - lava_callback - INFO - Starting async job submit for node 69403e1d575bc060ce1588ed 2025-12-17 16:58:18,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:18,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e1d575bc060ce1588ed HTTP/1.1" 200 None 2025-12-17 16:58:18,319 - lava_callback - DEBUG - Processing node 69403e1d575bc060ce1588ed: baseline-x86 2025-12-17 16:58:18,319 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e1d575bc060ce1588ed 2025-12-17 16:58:18,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:18,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e1d575bc060ce1588ed HTTP/1.1" 200 None 2025-12-17 16:58:18,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:18,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d5fad9ee307f5b1e66 HTTP/1.1" 200 None 2025-12-17 16:58:18,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:18,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e1d575bc060ce1588ed HTTP/1.1" 200 None 2025-12-17 16:58:18,355 - lava_callback - INFO - Completed processing callback for node 69403e1d575bc060ce1588ed 2025-12-17 16:58:18,393 - lava_callback - INFO - Received callback for node 69403e26575bc060ce158906 INFO: 172.21.0.1:37596 - "POST /node/69403e26575bc060ce158906 HTTP/1.0" 202 Accepted 2025-12-17 16:58:18,400 - lava_callback - INFO - Starting async job submit for node 69403e26575bc060ce158906 2025-12-17 16:58:18,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:18,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e26575bc060ce158906 HTTP/1.1" 200 None 2025-12-17 16:58:18,409 - lava_callback - DEBUG - Processing node 69403e26575bc060ce158906: baseline-arm64-baylibre 2025-12-17 16:58:18,409 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e26575bc060ce158906 2025-12-17 16:58:18,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:18,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e26575bc060ce158906 HTTP/1.1" 200 None 2025-12-17 16:58:18,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:18,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:58:18,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:18,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e26575bc060ce158906 HTTP/1.1" 200 None 2025-12-17 16:58:18,448 - lava_callback - INFO - Completed processing callback for node 69403e26575bc060ce158906 2025-12-17 16:58:18,974 - lava_callback - INFO - Received callback for node 69403e1f575bc060ce1588f8 INFO: 172.21.0.1:37608 - "POST /node/69403e1f575bc060ce1588f8 HTTP/1.0" 202 Accepted 2025-12-17 16:58:18,981 - lava_callback - INFO - Starting async job submit for node 69403e1f575bc060ce1588f8 2025-12-17 16:58:18,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:18,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e1f575bc060ce1588f8 HTTP/1.1" 200 None 2025-12-17 16:58:18,991 - lava_callback - DEBUG - Processing node 69403e1f575bc060ce1588f8: baseline-x86 2025-12-17 16:58:18,991 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e1f575bc060ce1588f8 2025-12-17 16:58:18,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:18,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e1f575bc060ce1588f8 HTTP/1.1" 200 None 2025-12-17 16:58:19,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:19,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d5fad9ee307f5b1e66 HTTP/1.1" 200 None 2025-12-17 16:58:19,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:19,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e1f575bc060ce1588f8 HTTP/1.1" 200 None 2025-12-17 16:58:19,025 - lava_callback - INFO - Completed processing callback for node 69403e1f575bc060ce1588f8 INFO: 172.21.0.1:34306 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:58:36,491 - lava_callback - INFO - Received callback for node 69403e29575bc060ce158909 INFO: 172.21.0.1:58830 - "POST /node/69403e29575bc060ce158909 HTTP/1.0" 202 Accepted 2025-12-17 16:58:36,497 - lava_callback - INFO - Starting async job submit for node 69403e29575bc060ce158909 2025-12-17 16:58:36,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:36,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e29575bc060ce158909 HTTP/1.1" 200 None 2025-12-17 16:58:36,507 - lava_callback - DEBUG - Processing node 69403e29575bc060ce158909: baseline-arm64-baylibre 2025-12-17 16:58:36,507 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e29575bc060ce158909 2025-12-17 16:58:36,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:36,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e29575bc060ce158909 HTTP/1.1" 200 None 2025-12-17 16:58:36,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:58836 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:58:36,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:58:36,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:36,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e29575bc060ce158909 HTTP/1.1" 200 None 2025-12-17 16:58:36,549 - lava_callback - INFO - Completed processing callback for node 69403e29575bc060ce158909 2025-12-17 16:58:37,159 - lava_callback - INFO - Received callback for node 69403e32575bc060ce158912 INFO: 172.21.0.1:58852 - "POST /node/69403e32575bc060ce158912 HTTP/1.0" 202 Accepted 2025-12-17 16:58:37,165 - lava_callback - INFO - Starting async job submit for node 69403e32575bc060ce158912 2025-12-17 16:58:37,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:37,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e32575bc060ce158912 HTTP/1.1" 200 None 2025-12-17 16:58:37,175 - lava_callback - DEBUG - Processing node 69403e32575bc060ce158912: baseline-arm64-baylibre 2025-12-17 16:58:37,175 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e32575bc060ce158912 2025-12-17 16:58:37,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:37,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e32575bc060ce158912 HTTP/1.1" 200 None 2025-12-17 16:58:37,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:37,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:58:37,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:37,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e32575bc060ce158912 HTTP/1.1" 200 None 2025-12-17 16:58:37,209 - lava_callback - INFO - Completed processing callback for node 69403e32575bc060ce158912 2025-12-17 16:58:37,811 - lava_callback - INFO - Received callback for node 69403e34575bc060ce158919 INFO: 172.21.0.1:58860 - "POST /node/69403e34575bc060ce158919 HTTP/1.0" 202 Accepted 2025-12-17 16:58:37,819 - lava_callback - INFO - Starting async job submit for node 69403e34575bc060ce158919 2025-12-17 16:58:37,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:37,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e34575bc060ce158919 HTTP/1.1" 200 None 2025-12-17 16:58:37,829 - lava_callback - DEBUG - Processing node 69403e34575bc060ce158919: baseline-arm64-baylibre 2025-12-17 16:58:37,829 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e34575bc060ce158919 2025-12-17 16:58:37,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:37,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e34575bc060ce158919 HTTP/1.1" 200 None 2025-12-17 16:58:37,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:37,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:58:37,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:37,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e34575bc060ce158919 HTTP/1.1" 200 None 2025-12-17 16:58:37,864 - lava_callback - INFO - Completed processing callback for node 69403e34575bc060ce158919 2025-12-17 16:58:38,473 - lava_callback - INFO - Received callback for node 69403e36575bc060ce15891c INFO: 172.21.0.1:58874 - "POST /node/69403e36575bc060ce15891c HTTP/1.0" 202 Accepted 2025-12-17 16:58:38,480 - lava_callback - INFO - Starting async job submit for node 69403e36575bc060ce15891c 2025-12-17 16:58:38,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:38,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e36575bc060ce15891c HTTP/1.1" 200 None 2025-12-17 16:58:38,489 - lava_callback - DEBUG - Processing node 69403e36575bc060ce15891c: baseline-arm64-baylibre 2025-12-17 16:58:38,489 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e36575bc060ce15891c 2025-12-17 16:58:38,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:38,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e36575bc060ce15891c HTTP/1.1" 200 None 2025-12-17 16:58:38,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:38,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 16:58:38,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:58:38,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e36575bc060ce15891c HTTP/1.1" 200 None 2025-12-17 16:58:38,524 - lava_callback - INFO - Completed processing callback for node 69403e36575bc060ce15891c INFO: 172.21.0.1:41786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58508 - "GET / HTTP/1.0" 200 OK 2025-12-17 16:59:16,585 - lava_callback - INFO - Received callback for node 69403e53575bc060ce15892d INFO: 172.21.0.1:58516 - "POST /node/69403e53575bc060ce15892d HTTP/1.0" 202 Accepted 2025-12-17 16:59:16,591 - lava_callback - INFO - Starting async job submit for node 69403e53575bc060ce15892d 2025-12-17 16:59:16,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:16,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e53575bc060ce15892d HTTP/1.1" 200 None 2025-12-17 16:59:16,600 - lava_callback - DEBUG - Processing node 69403e53575bc060ce15892d: baseline-x86 2025-12-17 16:59:16,600 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e53575bc060ce15892d 2025-12-17 16:59:16,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:16,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e53575bc060ce15892d HTTP/1.1" 200 None 2025-12-17 16:59:16,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:16,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d5fad9ee307f5b1e66 HTTP/1.1" 200 None 2025-12-17 16:59:16,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:16,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e53575bc060ce15892d HTTP/1.1" 200 None 2025-12-17 16:59:16,653 - lava_callback - INFO - Completed processing callback for node 69403e53575bc060ce15892d 2025-12-17 16:59:17,282 - lava_callback - INFO - Received callback for node 69403e56575bc060ce158930 INFO: 172.21.0.1:58528 - "POST /node/69403e56575bc060ce158930 HTTP/1.0" 202 Accepted 2025-12-17 16:59:17,287 - lava_callback - INFO - Starting async job submit for node 69403e56575bc060ce158930 2025-12-17 16:59:17,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:17,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e56575bc060ce158930 HTTP/1.1" 200 None 2025-12-17 16:59:17,301 - lava_callback - DEBUG - Processing node 69403e56575bc060ce158930: baseline-x86 2025-12-17 16:59:17,301 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e56575bc060ce158930 2025-12-17 16:59:17,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:17,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e56575bc060ce158930 HTTP/1.1" 200 None 2025-12-17 16:59:17,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:17,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d5fad9ee307f5b1e66 HTTP/1.1" 200 None 2025-12-17 16:59:17,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:17,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e56575bc060ce158930 HTTP/1.1" 200 None 2025-12-17 16:59:17,343 - lava_callback - INFO - Completed processing callback for node 69403e56575bc060ce158930 2025-12-17 16:59:17,970 - lava_callback - INFO - Received callback for node 69403e58575bc060ce158933 INFO: 172.21.0.1:58534 - "POST /node/69403e58575bc060ce158933 HTTP/1.0" 202 Accepted 2025-12-17 16:59:17,975 - lava_callback - INFO - Starting async job submit for node 69403e58575bc060ce158933 2025-12-17 16:59:17,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:17,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e58575bc060ce158933 HTTP/1.1" 200 None 2025-12-17 16:59:17,989 - lava_callback - DEBUG - Processing node 69403e58575bc060ce158933: baseline-x86 2025-12-17 16:59:17,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e58575bc060ce158933 2025-12-17 16:59:17,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:17,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e58575bc060ce158933 HTTP/1.1" 200 None 2025-12-17 16:59:17,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:18,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d5fad9ee307f5b1e66 HTTP/1.1" 200 None 2025-12-17 16:59:18,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:18,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e58575bc060ce158933 HTTP/1.1" 200 None 2025-12-17 16:59:18,020 - lava_callback - INFO - Completed processing callback for node 69403e58575bc060ce158933 2025-12-17 16:59:18,663 - lava_callback - INFO - Received callback for node 69403e5b575bc060ce158936 INFO: 172.21.0.1:58536 - "POST /node/69403e5b575bc060ce158936 HTTP/1.0" 202 Accepted 2025-12-17 16:59:18,668 - lava_callback - INFO - Starting async job submit for node 69403e5b575bc060ce158936 2025-12-17 16:59:18,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:18,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e5b575bc060ce158936 HTTP/1.1" 200 None 2025-12-17 16:59:18,678 - lava_callback - DEBUG - Processing node 69403e5b575bc060ce158936: baseline-x86 2025-12-17 16:59:18,678 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e5b575bc060ce158936 2025-12-17 16:59:18,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:18,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e5b575bc060ce158936 HTTP/1.1" 200 None 2025-12-17 16:59:18,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:18,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d5fad9ee307f5b1e66 HTTP/1.1" 200 None 2025-12-17 16:59:18,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:18,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e5b575bc060ce158936 HTTP/1.1" 200 None 2025-12-17 16:59:18,712 - lava_callback - INFO - Completed processing callback for node 69403e5b575bc060ce158936 2025-12-17 16:59:19,301 - lava_callback - INFO - Received callback for node 69403e5e575bc060ce158939 INFO: 172.21.0.1:58546 - "POST /node/69403e5e575bc060ce158939 HTTP/1.0" 202 Accepted 2025-12-17 16:59:19,306 - lava_callback - INFO - Starting async job submit for node 69403e5e575bc060ce158939 2025-12-17 16:59:19,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:19,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e5e575bc060ce158939 HTTP/1.1" 200 None 2025-12-17 16:59:19,316 - lava_callback - DEBUG - Processing node 69403e5e575bc060ce158939: baseline-x86 2025-12-17 16:59:19,316 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e5e575bc060ce158939 2025-12-17 16:59:19,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:19,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e5e575bc060ce158939 HTTP/1.1" 200 None 2025-12-17 16:59:19,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:19,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d5fad9ee307f5b1e66 HTTP/1.1" 200 None 2025-12-17 16:59:19,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 16:59:19,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e5e575bc060ce158939 HTTP/1.1" 200 None 2025-12-17 16:59:19,361 - lava_callback - INFO - Completed processing callback for node 69403e5e575bc060ce158939 INFO: 172.21.0.1:51660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56074 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:00:17,878 - lava_callback - INFO - Received callback for node 69403e8f575bc060ce158958 INFO: 172.21.0.1:56084 - "POST /node/69403e8f575bc060ce158958 HTTP/1.0" 202 Accepted 2025-12-17 17:00:17,886 - lava_callback - INFO - Starting async job submit for node 69403e8f575bc060ce158958 2025-12-17 17:00:17,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:00:17,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e8f575bc060ce158958 HTTP/1.1" 200 None 2025-12-17 17:00:17,899 - lava_callback - DEBUG - Processing node 69403e8f575bc060ce158958: baseline-x86 2025-12-17 17:00:17,899 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e8f575bc060ce158958 2025-12-17 17:00:17,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:00:17,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e8f575bc060ce158958 HTTP/1.1" 200 None 2025-12-17 17:00:17,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:00:17,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:00:17,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:00:17,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e8f575bc060ce158958 HTTP/1.1" 200 None 2025-12-17 17:00:17,948 - lava_callback - INFO - Completed processing callback for node 69403e8f575bc060ce158958 2025-12-17 17:00:18,600 - lava_callback - INFO - Received callback for node 69403e92575bc060ce15895b INFO: 172.21.0.1:56090 - "POST /node/69403e92575bc060ce15895b HTTP/1.0" 202 Accepted 2025-12-17 17:00:18,608 - lava_callback - INFO - Starting async job submit for node 69403e92575bc060ce15895b 2025-12-17 17:00:18,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:00:18,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e92575bc060ce15895b HTTP/1.1" 200 None 2025-12-17 17:00:18,628 - lava_callback - DEBUG - Processing node 69403e92575bc060ce15895b: baseline-x86 2025-12-17 17:00:18,628 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e92575bc060ce15895b 2025-12-17 17:00:18,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:00:18,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e92575bc060ce15895b HTTP/1.1" 200 None 2025-12-17 17:00:18,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:00:18,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:00:18,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:00:18,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e92575bc060ce15895b HTTP/1.1" 200 None 2025-12-17 17:00:18,665 - lava_callback - INFO - Completed processing callback for node 69403e92575bc060ce15895b 2025-12-17 17:00:19,315 - lava_callback - INFO - Received callback for node 69403e95575bc060ce15895e INFO: 172.21.0.1:56106 - "POST /node/69403e95575bc060ce15895e HTTP/1.0" 202 Accepted 2025-12-17 17:00:19,323 - lava_callback - INFO - Starting async job submit for node 69403e95575bc060ce15895e 2025-12-17 17:00:19,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:00:19,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e95575bc060ce15895e HTTP/1.1" 200 None 2025-12-17 17:00:19,335 - lava_callback - DEBUG - Processing node 69403e95575bc060ce15895e: baseline-x86 2025-12-17 17:00:19,336 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403e95575bc060ce15895e 2025-12-17 17:00:19,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:00:19,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403e95575bc060ce15895e HTTP/1.1" 200 None 2025-12-17 17:00:19,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:00:19,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:00:19,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:00:19,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403e95575bc060ce15895e HTTP/1.1" 200 None 2025-12-17 17:00:19,388 - lava_callback - INFO - Completed processing callback for node 69403e95575bc060ce15895e INFO: 172.21.0.1:48960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41678 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:01:10,986 - lava_callback - INFO - Received callback for node 69403ec3575bc060ce158968 INFO: 172.21.0.1:55406 - "POST /node/69403ec3575bc060ce158968 HTTP/1.0" 202 Accepted 2025-12-17 17:01:10,997 - lava_callback - INFO - Starting async job submit for node 69403ec3575bc060ce158968 2025-12-17 17:01:10,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:11,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ec3575bc060ce158968 HTTP/1.1" 200 None 2025-12-17 17:01:11,012 - lava_callback - DEBUG - Processing node 69403ec3575bc060ce158968: baseline-arm64-clabbe 2025-12-17 17:01:11,012 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ec3575bc060ce158968 2025-12-17 17:01:11,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:11,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ec3575bc060ce158968 HTTP/1.1" 200 None 2025-12-17 17:01:11,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:11,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:01:11,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:11,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ec3575bc060ce158968 HTTP/1.1" 200 None 2025-12-17 17:01:11,071 - lava_callback - INFO - Completed processing callback for node 69403ec3575bc060ce158968 INFO: 172.21.0.1:55414 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:01:11,657 - lava_callback - INFO - Received callback for node 69403ec6575bc060ce15896f INFO: 172.21.0.1:55420 - "POST /node/69403ec6575bc060ce15896f HTTP/1.0" 202 Accepted 2025-12-17 17:01:11,667 - lava_callback - INFO - Starting async job submit for node 69403ec6575bc060ce15896f 2025-12-17 17:01:11,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:11,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ec6575bc060ce15896f HTTP/1.1" 200 None 2025-12-17 17:01:11,683 - lava_callback - DEBUG - Processing node 69403ec6575bc060ce15896f: baseline-arm64-clabbe 2025-12-17 17:01:11,683 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ec6575bc060ce15896f 2025-12-17 17:01:11,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:11,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ec6575bc060ce15896f HTTP/1.1" 200 None 2025-12-17 17:01:11,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:11,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:01:11,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:11,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ec6575bc060ce15896f HTTP/1.1" 200 None 2025-12-17 17:01:11,754 - lava_callback - INFO - Completed processing callback for node 69403ec6575bc060ce15896f 2025-12-17 17:01:12,323 - lava_callback - INFO - Received callback for node 69403ec8575bc060ce158972 INFO: 172.21.0.1:55434 - "POST /node/69403ec8575bc060ce158972 HTTP/1.0" 202 Accepted 2025-12-17 17:01:12,333 - lava_callback - INFO - Starting async job submit for node 69403ec8575bc060ce158972 2025-12-17 17:01:12,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:12,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ec8575bc060ce158972 HTTP/1.1" 200 None 2025-12-17 17:01:12,347 - lava_callback - DEBUG - Processing node 69403ec8575bc060ce158972: baseline-arm64-clabbe 2025-12-17 17:01:12,347 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ec8575bc060ce158972 2025-12-17 17:01:12,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:12,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ec8575bc060ce158972 HTTP/1.1" 200 None 2025-12-17 17:01:12,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:12,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:01:12,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:12,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ec8575bc060ce158972 HTTP/1.1" 200 None 2025-12-17 17:01:12,394 - lava_callback - INFO - Completed processing callback for node 69403ec8575bc060ce158972 INFO: 172.21.0.1:55448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55456 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:01:16,637 - lava_callback - INFO - Received callback for node 69403ed2575bc060ce15897b INFO: 172.21.0.1:55470 - "POST /node/69403ed2575bc060ce15897b HTTP/1.0" 202 Accepted 2025-12-17 17:01:16,647 - lava_callback - INFO - Starting async job submit for node 69403ed2575bc060ce15897b 2025-12-17 17:01:16,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:16,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ed2575bc060ce15897b HTTP/1.1" 200 None 2025-12-17 17:01:16,671 - lava_callback - DEBUG - Processing node 69403ed2575bc060ce15897b: baseline-arm64-baylibre 2025-12-17 17:01:16,671 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ed2575bc060ce15897b 2025-12-17 17:01:16,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:16,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ed2575bc060ce15897b HTTP/1.1" 200 None 2025-12-17 17:01:16,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:16,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:01:16,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:16,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ed2575bc060ce15897b HTTP/1.1" 200 None 2025-12-17 17:01:16,732 - lava_callback - INFO - Completed processing callback for node 69403ed2575bc060ce15897b 2025-12-17 17:01:17,296 - lava_callback - INFO - Received callback for node 69403ed4575bc060ce158982 INFO: 172.21.0.1:55472 - "POST /node/69403ed4575bc060ce158982 HTTP/1.0" 202 Accepted 2025-12-17 17:01:17,305 - lava_callback - INFO - Starting async job submit for node 69403ed4575bc060ce158982 2025-12-17 17:01:17,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:17,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ed4575bc060ce158982 HTTP/1.1" 200 None 2025-12-17 17:01:17,320 - lava_callback - DEBUG - Processing node 69403ed4575bc060ce158982: baseline-arm64-baylibre 2025-12-17 17:01:17,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ed4575bc060ce158982 2025-12-17 17:01:17,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:17,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ed4575bc060ce158982 HTTP/1.1" 200 None 2025-12-17 17:01:17,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:17,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:01:17,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:17,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ed4575bc060ce158982 HTTP/1.1" 200 None 2025-12-17 17:01:17,372 - lava_callback - INFO - Completed processing callback for node 69403ed4575bc060ce158982 2025-12-17 17:01:17,953 - lava_callback - INFO - Received callback for node 69403ed6575bc060ce158985 INFO: 172.21.0.1:55484 - "POST /node/69403ed6575bc060ce158985 HTTP/1.0" 202 Accepted 2025-12-17 17:01:17,965 - lava_callback - INFO - Starting async job submit for node 69403ed6575bc060ce158985 2025-12-17 17:01:17,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:17,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ed6575bc060ce158985 HTTP/1.1" 200 None 2025-12-17 17:01:17,989 - lava_callback - DEBUG - Processing node 69403ed6575bc060ce158985: baseline-arm64-baylibre 2025-12-17 17:01:17,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ed6575bc060ce158985 2025-12-17 17:01:17,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:18,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ed6575bc060ce158985 HTTP/1.1" 200 None 2025-12-17 17:01:18,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:18,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:01:18,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:18,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ed6575bc060ce158985 HTTP/1.1" 200 None 2025-12-17 17:01:18,053 - lava_callback - INFO - Completed processing callback for node 69403ed6575bc060ce158985 INFO: 172.21.0.1:50598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41188 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:01:36,670 - lava_callback - INFO - Received callback for node 69403ee6575bc060ce15898a INFO: 172.21.0.1:41192 - "POST /node/69403ee6575bc060ce15898a HTTP/1.0" 202 Accepted 2025-12-17 17:01:36,680 - lava_callback - INFO - Starting async job submit for node 69403ee6575bc060ce15898a 2025-12-17 17:01:36,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:36,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ee6575bc060ce15898a HTTP/1.1" 200 None 2025-12-17 17:01:36,693 - lava_callback - DEBUG - Processing node 69403ee6575bc060ce15898a: baseline-arm64-baylibre 2025-12-17 17:01:36,693 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ee6575bc060ce15898a 2025-12-17 17:01:36,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:36,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ee6575bc060ce15898a HTTP/1.1" 200 None 2025-12-17 17:01:36,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:36,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:01:36,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:36,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ee6575bc060ce15898a HTTP/1.1" 200 None 2025-12-17 17:01:36,748 - lava_callback - INFO - Completed processing callback for node 69403ee6575bc060ce15898a 2025-12-17 17:01:37,320 - lava_callback - INFO - Received callback for node 69403ee8575bc060ce158991 INFO: 172.21.0.1:41198 - "POST /node/69403ee8575bc060ce158991 HTTP/1.0" 202 Accepted 2025-12-17 17:01:37,329 - lava_callback - INFO - Starting async job submit for node 69403ee8575bc060ce158991 2025-12-17 17:01:37,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:37,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ee8575bc060ce158991 HTTP/1.1" 200 None 2025-12-17 17:01:37,342 - lava_callback - DEBUG - Processing node 69403ee8575bc060ce158991: baseline-arm64-baylibre 2025-12-17 17:01:37,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403ee8575bc060ce158991 2025-12-17 17:01:37,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:37,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403ee8575bc060ce158991 HTTP/1.1" 200 None 2025-12-17 17:01:37,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:37,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:01:37,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:37,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403ee8575bc060ce158991 HTTP/1.1" 200 None 2025-12-17 17:01:37,397 - lava_callback - INFO - Completed processing callback for node 69403ee8575bc060ce158991 2025-12-17 17:01:38,062 - lava_callback - INFO - Received callback for node 69403eea575bc060ce158994 INFO: 172.21.0.1:41202 - "POST /node/69403eea575bc060ce158994 HTTP/1.0" 202 Accepted 2025-12-17 17:01:38,073 - lava_callback - INFO - Starting async job submit for node 69403eea575bc060ce158994 2025-12-17 17:01:38,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:38,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403eea575bc060ce158994 HTTP/1.1" 200 None 2025-12-17 17:01:38,087 - lava_callback - DEBUG - Processing node 69403eea575bc060ce158994: baseline-arm64-baylibre 2025-12-17 17:01:38,087 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403eea575bc060ce158994 2025-12-17 17:01:38,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:38,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403eea575bc060ce158994 HTTP/1.1" 200 None 2025-12-17 17:01:38,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:38,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:01:38,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:38,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403eea575bc060ce158994 HTTP/1.1" 200 None 2025-12-17 17:01:38,151 - lava_callback - INFO - Completed processing callback for node 69403eea575bc060ce158994 INFO: 172.21.0.1:60544 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:01:50,982 - lava_callback - INFO - Received callback for node 69403eec575bc060ce15899b INFO: 172.21.0.1:43400 - "POST /node/69403eec575bc060ce15899b HTTP/1.0" 202 Accepted 2025-12-17 17:01:50,995 - lava_callback - INFO - Starting async job submit for node 69403eec575bc060ce15899b 2025-12-17 17:01:50,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:51,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403eec575bc060ce15899b HTTP/1.1" 200 None 2025-12-17 17:01:51,008 - lava_callback - DEBUG - Processing node 69403eec575bc060ce15899b: baseline-arm64-clabbe 2025-12-17 17:01:51,008 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403eec575bc060ce15899b 2025-12-17 17:01:51,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:51,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403eec575bc060ce15899b HTTP/1.1" 200 None 2025-12-17 17:01:51,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:51,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:01:51,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:51,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403eec575bc060ce15899b HTTP/1.1" 200 None 2025-12-17 17:01:51,055 - lava_callback - INFO - Completed processing callback for node 69403eec575bc060ce15899b 2025-12-17 17:01:51,629 - lava_callback - INFO - Received callback for node 69403eef575bc060ce15899e INFO: 172.21.0.1:43414 - "POST /node/69403eef575bc060ce15899e HTTP/1.0" 202 Accepted 2025-12-17 17:01:51,640 - lava_callback - INFO - Starting async job submit for node 69403eef575bc060ce15899e 2025-12-17 17:01:51,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:51,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403eef575bc060ce15899e HTTP/1.1" 200 None 2025-12-17 17:01:51,658 - lava_callback - DEBUG - Processing node 69403eef575bc060ce15899e: baseline-arm64-clabbe 2025-12-17 17:01:51,658 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403eef575bc060ce15899e 2025-12-17 17:01:51,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:51,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403eef575bc060ce15899e HTTP/1.1" 200 None 2025-12-17 17:01:51,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:51,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:01:51,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:51,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403eef575bc060ce15899e HTTP/1.1" 200 None 2025-12-17 17:01:51,711 - lava_callback - INFO - Completed processing callback for node 69403eef575bc060ce15899e INFO: 172.21.0.1:43416 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:01:56,686 - lava_callback - INFO - Received callback for node 69403efa575bc060ce1589a3 INFO: 172.21.0.1:43426 - "POST /node/69403efa575bc060ce1589a3 HTTP/1.0" 202 Accepted 2025-12-17 17:01:56,693 - lava_callback - INFO - Starting async job submit for node 69403efa575bc060ce1589a3 2025-12-17 17:01:56,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:56,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403efa575bc060ce1589a3 HTTP/1.1" 200 None 2025-12-17 17:01:56,702 - lava_callback - DEBUG - Processing node 69403efa575bc060ce1589a3: baseline-arm64-baylibre 2025-12-17 17:01:56,702 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403efa575bc060ce1589a3 2025-12-17 17:01:56,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:56,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403efa575bc060ce1589a3 HTTP/1.1" 200 None 2025-12-17 17:01:56,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:56,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:01:56,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:56,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403efa575bc060ce1589a3 HTTP/1.1" 200 None 2025-12-17 17:01:56,746 - lava_callback - INFO - Completed processing callback for node 69403efa575bc060ce1589a3 2025-12-17 17:01:57,355 - lava_callback - INFO - Received callback for node 69403efd575bc060ce1589aa INFO: 172.21.0.1:43436 - "POST /node/69403efd575bc060ce1589aa HTTP/1.0" 202 Accepted 2025-12-17 17:01:57,363 - lava_callback - INFO - Starting async job submit for node 69403efd575bc060ce1589aa 2025-12-17 17:01:57,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:57,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403efd575bc060ce1589aa HTTP/1.1" 200 None 2025-12-17 17:01:57,372 - lava_callback - DEBUG - Processing node 69403efd575bc060ce1589aa: baseline-arm64-baylibre 2025-12-17 17:01:57,372 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403efd575bc060ce1589aa 2025-12-17 17:01:57,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:57,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403efd575bc060ce1589aa HTTP/1.1" 200 None 2025-12-17 17:01:57,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:57,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:01:57,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:57,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403efd575bc060ce1589aa HTTP/1.1" 200 None 2025-12-17 17:01:57,422 - lava_callback - INFO - Completed processing callback for node 69403efd575bc060ce1589aa 2025-12-17 17:01:58,001 - lava_callback - INFO - Received callback for node 69403eff575bc060ce1589ad INFO: 172.21.0.1:43448 - "POST /node/69403eff575bc060ce1589ad HTTP/1.0" 202 Accepted 2025-12-17 17:01:58,008 - lava_callback - INFO - Starting async job submit for node 69403eff575bc060ce1589ad 2025-12-17 17:01:58,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:58,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403eff575bc060ce1589ad HTTP/1.1" 200 None 2025-12-17 17:01:58,018 - lava_callback - DEBUG - Processing node 69403eff575bc060ce1589ad: baseline-arm64-baylibre 2025-12-17 17:01:58,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403eff575bc060ce1589ad 2025-12-17 17:01:58,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:58,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403eff575bc060ce1589ad HTTP/1.1" 200 None 2025-12-17 17:01:58,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:58,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:01:58,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:01:58,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403eff575bc060ce1589ad HTTP/1.1" 200 None 2025-12-17 17:01:58,058 - lava_callback - INFO - Completed processing callback for node 69403eff575bc060ce1589ad INFO: 172.21.0.1:34556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57758 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:02:47,750 - lava_callback - INFO - Received callback for node 6942de14e43d2b76e9022dba INFO: 172.21.0.1:57774 - "POST /node/6942de14e43d2b76e9022dba HTTP/1.0" 202 Accepted 2025-12-17 17:02:47,757 - lava_callback - INFO - Starting async job submit for node 6942de14e43d2b76e9022dba 2025-12-17 17:02:47,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:02:47,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942de14e43d2b76e9022dba HTTP/1.1" 200 None 2025-12-17 17:02:47,825 - lava_callback - DEBUG - Processing node 6942de14e43d2b76e9022dba: baseline-arm 2025-12-17 17:02:48,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:02:48,592 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:02:48,593 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942de14e43d2b76e9022dba/log.txt.gz 2025-12-17 17:02:48,593 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:02:48,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:02:48,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:02:48,774 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942de14e43d2b76e9022dba/lava_callback.json.gz DEBUG: Setup failed for 6942de14e43d2b76e9022dba. Transitting job node result: incomplete -> fail 2025-12-17 17:02:48,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:02:48,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942de14e43d2b76e9022dba HTTP/1.1" 200 None 2025-12-17 17:02:48,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:02:48,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 17:02:48,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:02:48,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942de14e43d2b76e9022dba HTTP/1.1" 200 None 2025-12-17 17:02:48,929 - lava_callback - INFO - Completed processing callback for node 6942de14e43d2b76e9022dba 2025-12-17 17:02:49,561 - lava_callback - INFO - Received callback for node 69403f37575bc060ce1589b7 INFO: 172.21.0.1:57788 - "POST /node/69403f37575bc060ce1589b7 HTTP/1.0" 202 Accepted 2025-12-17 17:02:49,572 - lava_callback - INFO - Starting async job submit for node 69403f37575bc060ce1589b7 2025-12-17 17:02:49,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:02:49,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f37575bc060ce1589b7 HTTP/1.1" 200 None 2025-12-17 17:02:49,591 - lava_callback - DEBUG - Processing node 69403f37575bc060ce1589b7: ltp-smoketest 2025-12-17 17:02:49,591 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403f37575bc060ce1589b7 2025-12-17 17:02:49,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:02:49,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f37575bc060ce1589b7 HTTP/1.1" 200 None 2025-12-17 17:02:49,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:02:49,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad7f3fb52d1dfff36c4a3 HTTP/1.1" 200 None 2025-12-17 17:02:49,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:02:49,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403f37575bc060ce1589b7 HTTP/1.1" 200 None 2025-12-17 17:02:49,642 - lava_callback - INFO - Completed processing callback for node 69403f37575bc060ce1589b7 2025-12-17 17:02:51,080 - lava_callback - INFO - Received callback for node 69403f27575bc060ce1589b2 INFO: 172.21.0.1:51998 - "POST /node/69403f27575bc060ce1589b2 HTTP/1.0" 202 Accepted 2025-12-17 17:02:51,089 - lava_callback - INFO - Starting async job submit for node 69403f27575bc060ce1589b2 2025-12-17 17:02:51,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:02:51,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f27575bc060ce1589b2 HTTP/1.1" 200 None 2025-12-17 17:02:51,099 - lava_callback - DEBUG - Processing node 69403f27575bc060ce1589b2: baseline-arm64-clabbe 2025-12-17 17:02:51,099 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403f27575bc060ce1589b2 2025-12-17 17:02:51,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:02:51,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f27575bc060ce1589b2 HTTP/1.1" 200 None 2025-12-17 17:02:51,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:02:51,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:02:51,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:02:51,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403f27575bc060ce1589b2 HTTP/1.1" 200 None 2025-12-17 17:02:51,136 - lava_callback - INFO - Completed processing callback for node 69403f27575bc060ce1589b2 INFO: 172.21.0.1:52000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48890 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:03:11,063 - lava_callback - INFO - Received callback for node 69403f3b575bc060ce1589be INFO: 172.21.0.1:40204 - "POST /node/69403f3b575bc060ce1589be HTTP/1.0" 202 Accepted 2025-12-17 17:03:11,073 - lava_callback - INFO - Starting async job submit for node 69403f3b575bc060ce1589be 2025-12-17 17:03:11,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:11,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f3b575bc060ce1589be HTTP/1.1" 200 None 2025-12-17 17:03:11,090 - lava_callback - DEBUG - Processing node 69403f3b575bc060ce1589be: baseline-arm64-clabbe 2025-12-17 17:03:11,090 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403f3b575bc060ce1589be 2025-12-17 17:03:11,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:11,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f3b575bc060ce1589be HTTP/1.1" 200 None 2025-12-17 17:03:11,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:11,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:03:11,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:11,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403f3b575bc060ce1589be HTTP/1.1" 200 None 2025-12-17 17:03:11,136 - lava_callback - INFO - Completed processing callback for node 69403f3b575bc060ce1589be INFO: 172.21.0.1:40214 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:03:11,741 - lava_callback - INFO - Received callback for node 69403f3f575bc060ce1589c3 INFO: 172.21.0.1:40222 - "POST /node/69403f3f575bc060ce1589c3 HTTP/1.0" 202 Accepted 2025-12-17 17:03:11,748 - lava_callback - INFO - Starting async job submit for node 69403f3f575bc060ce1589c3 2025-12-17 17:03:11,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:11,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f3f575bc060ce1589c3 HTTP/1.1" 200 None 2025-12-17 17:03:11,758 - lava_callback - DEBUG - Processing node 69403f3f575bc060ce1589c3: baseline-arm64-clabbe 2025-12-17 17:03:11,758 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403f3f575bc060ce1589c3 2025-12-17 17:03:11,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:11,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f3f575bc060ce1589c3 HTTP/1.1" 200 None 2025-12-17 17:03:11,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:11,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:03:11,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:11,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403f3f575bc060ce1589c3 HTTP/1.1" 200 None 2025-12-17 17:03:11,817 - lava_callback - INFO - Completed processing callback for node 69403f3f575bc060ce1589c3 2025-12-17 17:03:12,412 - lava_callback - INFO - Received callback for node 69403f41575bc060ce1589c6 INFO: 172.21.0.1:40228 - "POST /node/69403f41575bc060ce1589c6 HTTP/1.0" 202 Accepted 2025-12-17 17:03:12,421 - lava_callback - INFO - Starting async job submit for node 69403f41575bc060ce1589c6 2025-12-17 17:03:12,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:12,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f41575bc060ce1589c6 HTTP/1.1" 200 None 2025-12-17 17:03:12,433 - lava_callback - DEBUG - Processing node 69403f41575bc060ce1589c6: baseline-arm64-clabbe 2025-12-17 17:03:12,433 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403f41575bc060ce1589c6 2025-12-17 17:03:12,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:12,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f41575bc060ce1589c6 HTTP/1.1" 200 None 2025-12-17 17:03:12,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:12,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:03:12,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:12,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403f41575bc060ce1589c6 HTTP/1.1" 200 None 2025-12-17 17:03:12,477 - lava_callback - INFO - Completed processing callback for node 69403f41575bc060ce1589c6 INFO: 172.21.0.1:40232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51504 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:03:29,621 - lava_callback - INFO - Received callback for node 69403f5f575bc060ce1589d0 INFO: 172.21.0.1:51520 - "POST /node/69403f5f575bc060ce1589d0 HTTP/1.0" 202 Accepted 2025-12-17 17:03:29,630 - lava_callback - INFO - Starting async job submit for node 69403f5f575bc060ce1589d0 2025-12-17 17:03:29,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:29,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f5f575bc060ce1589d0 HTTP/1.1" 200 None 2025-12-17 17:03:29,672 - lava_callback - DEBUG - Processing node 69403f5f575bc060ce1589d0: baseline-arm-broonie 2025-12-17 17:03:29,672 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403f5f575bc060ce1589d0 2025-12-17 17:03:29,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:29,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f5f575bc060ce1589d0 HTTP/1.1" 200 None 2025-12-17 17:03:29,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:29,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af17afee8383e92b69dc1 HTTP/1.1" 200 None 2025-12-17 17:03:29,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:29,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403f5f575bc060ce1589d0 HTTP/1.1" 200 None 2025-12-17 17:03:29,743 - lava_callback - INFO - Completed processing callback for node 69403f5f575bc060ce1589d0 2025-12-17 17:03:31,069 - lava_callback - INFO - Received callback for node 69403f4f575bc060ce1589cb INFO: 172.21.0.1:37796 - "POST /node/69403f4f575bc060ce1589cb HTTP/1.0" 202 Accepted 2025-12-17 17:03:31,078 - lava_callback - INFO - Starting async job submit for node 69403f4f575bc060ce1589cb 2025-12-17 17:03:31,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:31,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f4f575bc060ce1589cb HTTP/1.1" 200 None 2025-12-17 17:03:31,090 - lava_callback - DEBUG - Processing node 69403f4f575bc060ce1589cb: baseline-arm64-clabbe 2025-12-17 17:03:31,090 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403f4f575bc060ce1589cb 2025-12-17 17:03:31,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:31,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f4f575bc060ce1589cb HTTP/1.1" 200 None 2025-12-17 17:03:31,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:31,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693ad843fb52d1dfff36c5ac HTTP/1.1" 200 None 2025-12-17 17:03:31,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:03:31,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403f4f575bc060ce1589cb HTTP/1.1" 200 None 2025-12-17 17:03:31,134 - lava_callback - INFO - Completed processing callback for node 69403f4f575bc060ce1589cb INFO: 172.21.0.1:37812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41760 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:04:16,806 - lava_callback - INFO - Received callback for node 69403f86575bc060ce1589de INFO: 172.21.0.1:41762 - "POST /node/69403f86575bc060ce1589de HTTP/1.0" 202 Accepted 2025-12-17 17:04:16,815 - lava_callback - INFO - Starting async job submit for node 69403f86575bc060ce1589de 2025-12-17 17:04:16,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:16,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f86575bc060ce1589de HTTP/1.1" 200 None 2025-12-17 17:04:16,827 - lava_callback - DEBUG - Processing node 69403f86575bc060ce1589de: baseline-arm64-baylibre 2025-12-17 17:04:16,827 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403f86575bc060ce1589de 2025-12-17 17:04:16,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:16,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f86575bc060ce1589de HTTP/1.1" 200 None 2025-12-17 17:04:16,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:16,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d923377ecffe173f95b8b HTTP/1.1" 200 None 2025-12-17 17:04:16,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:16,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403f86575bc060ce1589de HTTP/1.1" 200 None 2025-12-17 17:04:16,865 - lava_callback - INFO - Completed processing callback for node 69403f86575bc060ce1589de 2025-12-17 17:04:17,470 - lava_callback - INFO - Received callback for node 69403f88575bc060ce1589e7 INFO: 172.21.0.1:41776 - "POST /node/69403f88575bc060ce1589e7 HTTP/1.0" 202 Accepted 2025-12-17 17:04:17,476 - lava_callback - INFO - Starting async job submit for node 69403f88575bc060ce1589e7 2025-12-17 17:04:17,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:17,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f88575bc060ce1589e7 HTTP/1.1" 200 None 2025-12-17 17:04:17,486 - lava_callback - DEBUG - Processing node 69403f88575bc060ce1589e7: baseline-arm64-baylibre 2025-12-17 17:04:17,486 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403f88575bc060ce1589e7 2025-12-17 17:04:17,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:17,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f88575bc060ce1589e7 HTTP/1.1" 200 None 2025-12-17 17:04:17,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:17,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d923377ecffe173f95b8b HTTP/1.1" 200 None 2025-12-17 17:04:17,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:17,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403f88575bc060ce1589e7 HTTP/1.1" 200 None 2025-12-17 17:04:17,521 - lava_callback - INFO - Completed processing callback for node 69403f88575bc060ce1589e7 2025-12-17 17:04:18,117 - lava_callback - INFO - Received callback for node 69403f8a575bc060ce1589ea INFO: 172.21.0.1:41780 - "POST /node/69403f8a575bc060ce1589ea HTTP/1.0" 202 Accepted 2025-12-17 17:04:18,139 - lava_callback - INFO - Starting async job submit for node 69403f8a575bc060ce1589ea 2025-12-17 17:04:18,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:18,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f8a575bc060ce1589ea HTTP/1.1" 200 None 2025-12-17 17:04:18,164 - lava_callback - DEBUG - Processing node 69403f8a575bc060ce1589ea: baseline-arm64-baylibre 2025-12-17 17:04:18,164 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403f8a575bc060ce1589ea 2025-12-17 17:04:18,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:18,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f8a575bc060ce1589ea HTTP/1.1" 200 None 2025-12-17 17:04:18,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:18,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d923377ecffe173f95b8b HTTP/1.1" 200 None 2025-12-17 17:04:18,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:18,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403f8a575bc060ce1589ea HTTP/1.1" 200 None 2025-12-17 17:04:18,209 - lava_callback - INFO - Completed processing callback for node 69403f8a575bc060ce1589ea 2025-12-17 17:04:18,764 - lava_callback - INFO - Received callback for node 69403f8c575bc060ce1589ed INFO: 172.21.0.1:41784 - "POST /node/69403f8c575bc060ce1589ed HTTP/1.0" 202 Accepted 2025-12-17 17:04:18,773 - lava_callback - INFO - Starting async job submit for node 69403f8c575bc060ce1589ed 2025-12-17 17:04:18,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:18,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f8c575bc060ce1589ed HTTP/1.1" 200 None 2025-12-17 17:04:18,790 - lava_callback - DEBUG - Processing node 69403f8c575bc060ce1589ed: baseline-arm64-baylibre 2025-12-17 17:04:18,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403f8c575bc060ce1589ed 2025-12-17 17:04:18,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:18,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f8c575bc060ce1589ed HTTP/1.1" 200 None 2025-12-17 17:04:18,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:18,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d923377ecffe173f95b8b HTTP/1.1" 200 None 2025-12-17 17:04:18,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:18,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403f8c575bc060ce1589ed HTTP/1.1" 200 None 2025-12-17 17:04:18,824 - lava_callback - INFO - Completed processing callback for node 69403f8c575bc060ce1589ed 2025-12-17 17:04:25,142 - lava_callback - INFO - Received callback for node 69403b5c575bc060ce15847a INFO: 172.21.0.1:60842 - "POST /node/69403b5c575bc060ce15847a HTTP/1.0" 202 Accepted 2025-12-17 17:04:25,201 - lava_callback - INFO - Starting async job submit for node 69403b5c575bc060ce15847a 2025-12-17 17:04:25,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:25,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b5c575bc060ce15847a HTTP/1.1" 200 None 2025-12-17 17:04:25,266 - lava_callback - DEBUG - Processing node 69403b5c575bc060ce15847a: baseline-arm-pengutronix 2025-12-17 17:04:25,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:04:25,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:04:25,605 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69403b5c575bc060ce15847a/log.txt.gz 2025-12-17 17:04:25,605 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:04:25,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:04:25,767 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:04:25,767 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69403b5c575bc060ce15847a/lava_callback.json.gz DEBUG: Setup failed for 69403b5c575bc060ce15847a. Transitting job node result: incomplete -> fail 2025-12-17 17:04:25,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:25,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403b5c575bc060ce15847a HTTP/1.1" 200 None 2025-12-17 17:04:25,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:25,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af125fee8383e92b69ab4 HTTP/1.1" 200 None 2025-12-17 17:04:25,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:25,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403b5c575bc060ce15847a HTTP/1.1" 200 None 2025-12-17 17:04:25,894 - lava_callback - INFO - Completed processing callback for node 69403b5c575bc060ce15847a INFO: 172.21.0.1:60852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47974 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:04:36,822 - lava_callback - INFO - Received callback for node 69403f9a575bc060ce1589f2 INFO: 172.21.0.1:47984 - "POST /node/69403f9a575bc060ce1589f2 HTTP/1.0" 202 Accepted 2025-12-17 17:04:36,828 - lava_callback - INFO - Starting async job submit for node 69403f9a575bc060ce1589f2 2025-12-17 17:04:36,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:36,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f9a575bc060ce1589f2 HTTP/1.1" 200 None 2025-12-17 17:04:36,837 - lava_callback - DEBUG - Processing node 69403f9a575bc060ce1589f2: baseline-arm64-baylibre 2025-12-17 17:04:36,837 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403f9a575bc060ce1589f2 2025-12-17 17:04:36,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:36,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f9a575bc060ce1589f2 HTTP/1.1" 200 None 2025-12-17 17:04:36,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:36,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d923377ecffe173f95b8b HTTP/1.1" 200 None 2025-12-17 17:04:36,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:36,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403f9a575bc060ce1589f2 HTTP/1.1" 200 None 2025-12-17 17:04:36,914 - lava_callback - INFO - Completed processing callback for node 69403f9a575bc060ce1589f2 2025-12-17 17:04:37,454 - lava_callback - INFO - Received callback for node 69403f9d575bc060ce1589ff INFO: 172.21.0.1:47994 - "POST /node/69403f9d575bc060ce1589ff HTTP/1.0" 202 Accepted 2025-12-17 17:04:37,465 - lava_callback - INFO - Starting async job submit for node 69403f9d575bc060ce1589ff 2025-12-17 17:04:37,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:37,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f9d575bc060ce1589ff HTTP/1.1" 200 None 2025-12-17 17:04:37,479 - lava_callback - DEBUG - Processing node 69403f9d575bc060ce1589ff: baseline-arm64-baylibre 2025-12-17 17:04:37,479 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403f9d575bc060ce1589ff 2025-12-17 17:04:37,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:37,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f9d575bc060ce1589ff HTTP/1.1" 200 None 2025-12-17 17:04:37,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:37,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d923377ecffe173f95b8b HTTP/1.1" 200 None 2025-12-17 17:04:37,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:37,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403f9d575bc060ce1589ff HTTP/1.1" 200 None 2025-12-17 17:04:37,524 - lava_callback - INFO - Completed processing callback for node 69403f9d575bc060ce1589ff 2025-12-17 17:04:38,115 - lava_callback - INFO - Received callback for node 69403f9f575bc060ce158a04 INFO: 172.21.0.1:47996 - "POST /node/69403f9f575bc060ce158a04 HTTP/1.0" 202 Accepted 2025-12-17 17:04:38,123 - lava_callback - INFO - Starting async job submit for node 69403f9f575bc060ce158a04 2025-12-17 17:04:38,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:38,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f9f575bc060ce158a04 HTTP/1.1" 200 None 2025-12-17 17:04:38,136 - lava_callback - DEBUG - Processing node 69403f9f575bc060ce158a04: baseline-arm64-baylibre 2025-12-17 17:04:38,137 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403f9f575bc060ce158a04 2025-12-17 17:04:38,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:38,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403f9f575bc060ce158a04 HTTP/1.1" 200 None 2025-12-17 17:04:38,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:38,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d923377ecffe173f95b8b HTTP/1.1" 200 None 2025-12-17 17:04:38,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:38,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403f9f575bc060ce158a04 HTTP/1.1" 200 None 2025-12-17 17:04:38,203 - lava_callback - INFO - Completed processing callback for node 69403f9f575bc060ce158a04 2025-12-17 17:04:38,770 - lava_callback - INFO - Received callback for node 69403fa3575bc060ce158a12 INFO: 172.21.0.1:48006 - "POST /node/69403fa3575bc060ce158a12 HTTP/1.0" 202 Accepted 2025-12-17 17:04:38,779 - lava_callback - INFO - Starting async job submit for node 69403fa3575bc060ce158a12 2025-12-17 17:04:38,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:38,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403fa3575bc060ce158a12 HTTP/1.1" 200 None 2025-12-17 17:04:38,791 - lava_callback - DEBUG - Processing node 69403fa3575bc060ce158a12: baseline-arm64-baylibre 2025-12-17 17:04:38,791 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403fa3575bc060ce158a12 2025-12-17 17:04:38,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:38,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403fa3575bc060ce158a12 HTTP/1.1" 200 None 2025-12-17 17:04:38,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:38,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d923377ecffe173f95b8b HTTP/1.1" 200 None 2025-12-17 17:04:38,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:38,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403fa3575bc060ce158a12 HTTP/1.1" 200 None 2025-12-17 17:04:38,843 - lava_callback - INFO - Completed processing callback for node 69403fa3575bc060ce158a12 2025-12-17 17:04:42,749 - lava_callback - INFO - Received callback for node 6942debde43d2b76e9022e85 INFO: 172.21.0.1:58914 - "POST /node/6942debde43d2b76e9022e85 HTTP/1.0" 202 Accepted 2025-12-17 17:04:42,757 - lava_callback - INFO - Starting async job submit for node 6942debde43d2b76e9022e85 2025-12-17 17:04:42,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:42,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942debde43d2b76e9022e85 HTTP/1.1" 200 None 2025-12-17 17:04:42,877 - lava_callback - DEBUG - Processing node 6942debde43d2b76e9022e85: baseline-arm 2025-12-17 17:04:43,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:04:44,051 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:04:44,052 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942debde43d2b76e9022e85/log.txt.gz 2025-12-17 17:04:44,053 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:04:44,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:04:44,284 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:04:44,285 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942debde43d2b76e9022e85/lava_callback.json.gz DEBUG: Setup failed for 6942debde43d2b76e9022e85. Transitting job node result: incomplete -> fail 2025-12-17 17:04:44,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:44,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942debde43d2b76e9022e85 HTTP/1.1" 200 None 2025-12-17 17:04:44,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:44,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 17:04:44,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:44,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942debde43d2b76e9022e85 HTTP/1.1" 200 None 2025-12-17 17:04:44,512 - lava_callback - INFO - Completed processing callback for node 6942debde43d2b76e9022e85 INFO: 172.21.0.1:58922 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:04:51,095 - lava_callback - INFO - Received callback for node 69403fa7575bc060ce158a18 INFO: 172.21.0.1:36434 - "POST /node/69403fa7575bc060ce158a18 HTTP/1.0" 202 Accepted 2025-12-17 17:04:51,101 - lava_callback - INFO - Starting async job submit for node 69403fa7575bc060ce158a18 2025-12-17 17:04:51,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:51,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403fa7575bc060ce158a18 HTTP/1.1" 200 None 2025-12-17 17:04:51,125 - lava_callback - DEBUG - Processing node 69403fa7575bc060ce158a18: baseline-arm64-clabbe 2025-12-17 17:04:51,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403fa7575bc060ce158a18 2025-12-17 17:04:51,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:51,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403fa7575bc060ce158a18 HTTP/1.1" 200 None 2025-12-17 17:04:51,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:51,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d923377ecffe173f95b8b HTTP/1.1" 200 None 2025-12-17 17:04:51,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:51,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403fa7575bc060ce158a18 HTTP/1.1" 200 None 2025-12-17 17:04:51,161 - lava_callback - INFO - Completed processing callback for node 69403fa7575bc060ce158a18 2025-12-17 17:04:51,755 - lava_callback - INFO - Received callback for node 69403fab575bc060ce158a28 INFO: 172.21.0.1:36440 - "POST /node/69403fab575bc060ce158a28 HTTP/1.0" 202 Accepted 2025-12-17 17:04:51,766 - lava_callback - INFO - Starting async job submit for node 69403fab575bc060ce158a28 2025-12-17 17:04:51,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:51,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403fab575bc060ce158a28 HTTP/1.1" 200 None 2025-12-17 17:04:51,777 - lava_callback - DEBUG - Processing node 69403fab575bc060ce158a28: baseline-arm64-clabbe 2025-12-17 17:04:51,777 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403fab575bc060ce158a28 2025-12-17 17:04:51,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:51,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403fab575bc060ce158a28 HTTP/1.1" 200 None 2025-12-17 17:04:51,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:51,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d923377ecffe173f95b8b HTTP/1.1" 200 None 2025-12-17 17:04:51,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:51,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403fab575bc060ce158a28 HTTP/1.1" 200 None 2025-12-17 17:04:51,825 - lava_callback - INFO - Completed processing callback for node 69403fab575bc060ce158a28 2025-12-17 17:04:52,439 - lava_callback - INFO - Received callback for node 69403fad575bc060ce158a2b INFO: 172.21.0.1:36452 - "POST /node/69403fad575bc060ce158a2b HTTP/1.0" 202 Accepted 2025-12-17 17:04:52,445 - lava_callback - INFO - Starting async job submit for node 69403fad575bc060ce158a2b 2025-12-17 17:04:52,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:52,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403fad575bc060ce158a2b HTTP/1.1" 200 None 2025-12-17 17:04:52,454 - lava_callback - DEBUG - Processing node 69403fad575bc060ce158a2b: baseline-arm64-clabbe 2025-12-17 17:04:52,454 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403fad575bc060ce158a2b 2025-12-17 17:04:52,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:52,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403fad575bc060ce158a2b HTTP/1.1" 200 None 2025-12-17 17:04:52,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:52,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d923377ecffe173f95b8b HTTP/1.1" 200 None 2025-12-17 17:04:52,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:52,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403fad575bc060ce158a2b HTTP/1.1" 200 None 2025-12-17 17:04:52,488 - lava_callback - INFO - Completed processing callback for node 69403fad575bc060ce158a2b 2025-12-17 17:04:53,114 - lava_callback - INFO - Received callback for node 69403faf575bc060ce158a2e INFO: 172.21.0.1:36464 - "POST /node/69403faf575bc060ce158a2e HTTP/1.0" 202 Accepted 2025-12-17 17:04:53,130 - lava_callback - INFO - Starting async job submit for node 69403faf575bc060ce158a2e 2025-12-17 17:04:53,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:53,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403faf575bc060ce158a2e HTTP/1.1" 200 None 2025-12-17 17:04:53,162 - lava_callback - DEBUG - Processing node 69403faf575bc060ce158a2e: baseline-arm64-clabbe 2025-12-17 17:04:53,162 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403faf575bc060ce158a2e 2025-12-17 17:04:53,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:53,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403faf575bc060ce158a2e HTTP/1.1" 200 None 2025-12-17 17:04:53,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:53,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d923377ecffe173f95b8b HTTP/1.1" 200 None 2025-12-17 17:04:53,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:53,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403faf575bc060ce158a2e HTTP/1.1" 200 None 2025-12-17 17:04:53,209 - lava_callback - INFO - Completed processing callback for node 69403faf575bc060ce158a2e 2025-12-17 17:04:53,765 - lava_callback - INFO - Received callback for node 69403fb1575bc060ce158a31 INFO: 172.21.0.1:36472 - "POST /node/69403fb1575bc060ce158a31 HTTP/1.0" 202 Accepted 2025-12-17 17:04:53,776 - lava_callback - INFO - Starting async job submit for node 69403fb1575bc060ce158a31 2025-12-17 17:04:53,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:53,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403fb1575bc060ce158a31 HTTP/1.1" 200 None 2025-12-17 17:04:53,789 - lava_callback - DEBUG - Processing node 69403fb1575bc060ce158a31: baseline-arm64-clabbe 2025-12-17 17:04:53,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403fb1575bc060ce158a31 2025-12-17 17:04:53,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:53,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403fb1575bc060ce158a31 HTTP/1.1" 200 None 2025-12-17 17:04:53,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:53,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d923377ecffe173f95b8b HTTP/1.1" 200 None 2025-12-17 17:04:53,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:53,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403fb1575bc060ce158a31 HTTP/1.1" 200 None 2025-12-17 17:04:53,863 - lava_callback - INFO - Completed processing callback for node 69403fb1575bc060ce158a31 INFO: 172.21.0.1:36474 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:04:56,831 - lava_callback - INFO - Received callback for node 69403fa5575bc060ce158a15 INFO: 172.21.0.1:36490 - "POST /node/69403fa5575bc060ce158a15 HTTP/1.0" 202 Accepted 2025-12-17 17:04:56,839 - lava_callback - INFO - Starting async job submit for node 69403fa5575bc060ce158a15 2025-12-17 17:04:56,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:56,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403fa5575bc060ce158a15 HTTP/1.1" 200 None 2025-12-17 17:04:56,848 - lava_callback - DEBUG - Processing node 69403fa5575bc060ce158a15: baseline-arm64-baylibre 2025-12-17 17:04:56,848 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69403fa5575bc060ce158a15 2025-12-17 17:04:56,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:56,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403fa5575bc060ce158a15 HTTP/1.1" 200 None 2025-12-17 17:04:56,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:56,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d923377ecffe173f95b8b HTTP/1.1" 200 None 2025-12-17 17:04:56,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:04:56,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403fa5575bc060ce158a15 HTTP/1.1" 200 None 2025-12-17 17:04:56,884 - lava_callback - INFO - Completed processing callback for node 69403fa5575bc060ce158a15 INFO: 172.21.0.1:44764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34522 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:05:12,231 - lava_callback - INFO - Received callback for node 6940c4fc1e3c24e95681d5ff INFO: 172.21.0.1:34528 - "POST /node/6940c4fc1e3c24e95681d5ff HTTP/1.0" 202 Accepted 2025-12-17 17:05:12,238 - lava_callback - INFO - Starting async job submit for node 6940c4fc1e3c24e95681d5ff 2025-12-17 17:05:12,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:05:12,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c4fc1e3c24e95681d5ff HTTP/1.1" 200 None 2025-12-17 17:05:12,312 - lava_callback - DEBUG - Processing node 6940c4fc1e3c24e95681d5ff: baseline-x86 2025-12-17 17:05:13,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:05:13,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:05:13,409 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-6940c4fc1e3c24e95681d5ff/log.txt.gz 2025-12-17 17:05:13,410 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:05:13,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:05:13,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:05:13,670 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-6940c4fc1e3c24e95681d5ff/lava_callback.json.gz DEBUG: Setup failed for 6940c4fc1e3c24e95681d5ff. Transitting job node result: incomplete -> fail 2025-12-17 17:05:13,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:05:13,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c4fc1e3c24e95681d5ff HTTP/1.1" 200 None 2025-12-17 17:05:13,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:05:13,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d21eb07de2c0cbc264f2b HTTP/1.1" 200 None 2025-12-17 17:05:13,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:05:13,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940c4fc1e3c24e95681d5ff HTTP/1.1" 200 None 2025-12-17 17:05:13,829 - lava_callback - INFO - Completed processing callback for node 6940c4fc1e3c24e95681d5ff INFO: 172.21.0.1:34538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45692 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:05:53,648 - lava_callback - INFO - Received callback for node 69404189575bc060ce158aab INFO: 172.21.0.1:34892 - "POST /node/69404189575bc060ce158aab HTTP/1.0" 202 Accepted 2025-12-17 17:05:53,654 - lava_callback - INFO - Starting async job submit for node 69404189575bc060ce158aab 2025-12-17 17:05:53,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:05:53,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404189575bc060ce158aab HTTP/1.1" 200 None 2025-12-17 17:05:53,692 - lava_callback - DEBUG - Processing node 69404189575bc060ce158aab: baseline-x86 2025-12-17 17:05:54,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:05:54,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:05:54,242 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69404189575bc060ce158aab/log.txt.gz 2025-12-17 17:05:54,242 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:05:54,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:05:54,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:05:54,381 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69404189575bc060ce158aab/lava_callback.json.gz 2025-12-17 17:05:54,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:05:54,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404189575bc060ce158aab HTTP/1.1" 200 None 2025-12-17 17:05:54,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:05:54,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af208fee8383e92b6a1a2 HTTP/1.1" 200 None 2025-12-17 17:05:54,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:05:54,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69404189575bc060ce158aab HTTP/1.1" 200 None 2025-12-17 17:05:54,440 - lava_callback - INFO - Completed processing callback for node 69404189575bc060ce158aab INFO: 172.21.0.1:34902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42786 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:06:18,869 - lava_callback - INFO - Received callback for node 6942e2e8e43d2b76e90238d0 INFO: 172.21.0.1:42800 - "POST /node/6942e2e8e43d2b76e90238d0 HTTP/1.0" 202 Accepted 2025-12-17 17:06:18,879 - lava_callback - INFO - Starting async job submit for node 6942e2e8e43d2b76e90238d0 2025-12-17 17:06:18,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:18,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e8e43d2b76e90238d0 HTTP/1.1" 200 None 2025-12-17 17:06:18,929 - lava_callback - DEBUG - Processing node 6942e2e8e43d2b76e90238d0: baseline-arm64 2025-12-17 17:06:19,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:06:19,544 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:06:19,545 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e8e43d2b76e90238d0/log.txt.gz 2025-12-17 17:06:19,545 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:06:19,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:06:19,793 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:06:19,794 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e8e43d2b76e90238d0/lava_callback.json.gz 2025-12-17 17:06:19,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:19,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e8e43d2b76e90238d0 HTTP/1.1" 200 None 2025-12-17 17:06:19,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:19,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:06:19,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:19,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2e8e43d2b76e90238d0 HTTP/1.1" 200 None 2025-12-17 17:06:19,896 - lava_callback - INFO - Completed processing callback for node 6942e2e8e43d2b76e90238d0 INFO: 172.21.0.1:52070 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:06:36,164 - lava_callback - INFO - Received callback for node 6942e2ede43d2b76e90238fd INFO: 172.21.0.1:37994 - "POST /node/6942e2ede43d2b76e90238fd HTTP/1.0" 202 Accepted 2025-12-17 17:06:36,174 - lava_callback - INFO - Starting async job submit for node 6942e2ede43d2b76e90238fd 2025-12-17 17:06:36,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:36,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2ede43d2b76e90238fd HTTP/1.1" 200 None 2025-12-17 17:06:36,219 - lava_callback - DEBUG - Processing node 6942e2ede43d2b76e90238fd: baseline-nfs-arm64 2025-12-17 17:06:36,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:38008 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:06:36,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:06:36,730 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2ede43d2b76e90238fd/log.txt.gz 2025-12-17 17:06:36,730 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:06:36,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:06:36,926 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:06:36,927 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2ede43d2b76e90238fd/lava_callback.json.gz 2025-12-17 17:06:36,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:36,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2ede43d2b76e90238fd HTTP/1.1" 200 None 2025-12-17 17:06:36,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:36,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:06:36,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:37,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2ede43d2b76e90238fd HTTP/1.1" 200 None 2025-12-17 17:06:37,031 - lava_callback - INFO - Completed processing callback for node 6942e2ede43d2b76e90238fd 2025-12-17 17:06:43,600 - lava_callback - INFO - Received callback for node 6942e2e4e43d2b76e90238b8 INFO: 172.21.0.1:53766 - "POST /node/6942e2e4e43d2b76e90238b8 HTTP/1.0" 202 Accepted 2025-12-17 17:06:43,607 - lava_callback - INFO - Starting async job submit for node 6942e2e4e43d2b76e90238b8 2025-12-17 17:06:43,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:43,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e4e43d2b76e90238b8 HTTP/1.1" 200 None 2025-12-17 17:06:43,657 - lava_callback - DEBUG - Processing node 6942e2e4e43d2b76e90238b8: baseline-arm64 2025-12-17 17:06:43,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:06:44,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:06:44,221 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e4e43d2b76e90238b8/log.txt.gz 2025-12-17 17:06:44,221 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:06:44,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:06:44,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:06:44,877 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e4e43d2b76e90238b8/lava_callback.json.gz 2025-12-17 17:06:44,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:44,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e4e43d2b76e90238b8 HTTP/1.1" 200 None 2025-12-17 17:06:44,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:44,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:06:44,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:44,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2e4e43d2b76e90238b8 HTTP/1.1" 200 None 2025-12-17 17:06:44,954 - lava_callback - INFO - Completed processing callback for node 6942e2e4e43d2b76e90238b8 INFO: 172.21.0.1:53776 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:06:55,226 - lava_callback - INFO - Received callback for node 6942e2f9e43d2b76e902394a INFO: 172.21.0.1:58024 - "POST /node/6942e2f9e43d2b76e902394a HTTP/1.0" 202 Accepted 2025-12-17 17:06:55,233 - lava_callback - INFO - Starting async job submit for node 6942e2f9e43d2b76e902394a 2025-12-17 17:06:55,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:55,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f9e43d2b76e902394a HTTP/1.1" 200 None 2025-12-17 17:06:55,273 - lava_callback - DEBUG - Processing node 6942e2f9e43d2b76e902394a: baseline-arm64-foundriesio 2025-12-17 17:06:55,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:06:55,850 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:06:55,851 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-6942e2f9e43d2b76e902394a/log.txt.gz 2025-12-17 17:06:55,851 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:06:55,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:06:56,120 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:06:56,121 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-6942e2f9e43d2b76e902394a/lava_callback.json.gz 2025-12-17 17:06:56,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:56,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f9e43d2b76e902394a HTTP/1.1" 200 None 2025-12-17 17:06:56,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:56,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:06:56,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:56,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2f9e43d2b76e902394a HTTP/1.1" 200 None 2025-12-17 17:06:56,192 - lava_callback - INFO - Completed processing callback for node 6942e2f9e43d2b76e902394a INFO: 172.21.0.1:58028 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:06:56,560 - lava_callback - INFO - Received callback for node 6942e2fae43d2b76e902394e INFO: 172.21.0.1:58044 - "POST /node/6942e2fae43d2b76e902394e HTTP/1.0" 202 Accepted 2025-12-17 17:06:56,573 - lava_callback - INFO - Starting async job submit for node 6942e2fae43d2b76e902394e 2025-12-17 17:06:56,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:56,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2fae43d2b76e902394e HTTP/1.1" 200 None 2025-12-17 17:06:56,643 - lava_callback - DEBUG - Processing node 6942e2fae43d2b76e902394e: baseline-arm64-foundriesio 2025-12-17 17:06:57,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:06:57,577 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:06:57,578 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-6942e2fae43d2b76e902394e/log.txt.gz 2025-12-17 17:06:57,578 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:06:57,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:06:57,766 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:06:57,767 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-6942e2fae43d2b76e902394e/lava_callback.json.gz 2025-12-17 17:06:57,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:57,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2fae43d2b76e902394e HTTP/1.1" 200 None 2025-12-17 17:06:57,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:57,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:06:57,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:06:57,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2fae43d2b76e902394e HTTP/1.1" 200 None 2025-12-17 17:06:57,845 - lava_callback - INFO - Completed processing callback for node 6942e2fae43d2b76e902394e 2025-12-17 17:07:05,706 - lava_callback - INFO - Received callback for node 6942e2e4e43d2b76e90238be INFO: 172.21.0.1:50074 - "POST /node/6942e2e4e43d2b76e90238be HTTP/1.0" 202 Accepted 2025-12-17 17:07:05,717 - lava_callback - INFO - Starting async job submit for node 6942e2e4e43d2b76e90238be 2025-12-17 17:07:05,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:05,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e4e43d2b76e90238be HTTP/1.1" 200 None 2025-12-17 17:07:05,774 - lava_callback - DEBUG - Processing node 6942e2e4e43d2b76e90238be: baseline-arm64 2025-12-17 17:07:06,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:50084 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:07:06,541 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:06,542 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e4e43d2b76e90238be/log.txt.gz 2025-12-17 17:07:06,542 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:07:06,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:06,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:06,762 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e4e43d2b76e90238be/lava_callback.json.gz 2025-12-17 17:07:06,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:06,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e4e43d2b76e90238be HTTP/1.1" 200 None 2025-12-17 17:07:06,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:06,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:07:06,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:06,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2e4e43d2b76e90238be HTTP/1.1" 200 None 2025-12-17 17:07:06,824 - lava_callback - INFO - Completed processing callback for node 6942e2e4e43d2b76e90238be 2025-12-17 17:07:07,227 - lava_callback - INFO - Received callback for node 6942e2f7e43d2b76e9023940 INFO: 172.21.0.1:50088 - "POST /node/6942e2f7e43d2b76e9023940 HTTP/1.0" 202 Accepted 2025-12-17 17:07:07,242 - lava_callback - INFO - Starting async job submit for node 6942e2f7e43d2b76e9023940 2025-12-17 17:07:07,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:07,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f7e43d2b76e9023940 HTTP/1.1" 200 None 2025-12-17 17:07:07,300 - lava_callback - DEBUG - Processing node 6942e2f7e43d2b76e9023940: baseline-arm64-pengutronix 2025-12-17 17:07:07,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:07,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:07,685 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-6942e2f7e43d2b76e9023940/log.txt.gz 2025-12-17 17:07:07,685 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:07:07,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:07,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:07,813 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-6942e2f7e43d2b76e9023940/lava_callback.json.gz 2025-12-17 17:07:07,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:07,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f7e43d2b76e9023940 HTTP/1.1" 200 None 2025-12-17 17:07:07,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:07,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:07:07,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:07,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2f7e43d2b76e9023940 HTTP/1.1" 200 None 2025-12-17 17:07:07,875 - lava_callback - INFO - Completed processing callback for node 6942e2f7e43d2b76e9023940 2025-12-17 17:07:09,828 - lava_callback - INFO - Received callback for node 6940402a575bc060ce158a5f INFO: 172.21.0.1:48620 - "POST /node/6940402a575bc060ce158a5f HTTP/1.0" 202 Accepted 2025-12-17 17:07:09,838 - lava_callback - INFO - Starting async job submit for node 6940402a575bc060ce158a5f 2025-12-17 17:07:09,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:09,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940402a575bc060ce158a5f HTTP/1.1" 200 None 2025-12-17 17:07:09,852 - lava_callback - DEBUG - Processing node 6940402a575bc060ce158a5f: baseline-arm-broonie 2025-12-17 17:07:09,852 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940402a575bc060ce158a5f 2025-12-17 17:07:09,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:09,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940402a575bc060ce158a5f HTTP/1.1" 200 None 2025-12-17 17:07:09,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:09,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693aeba3fee8383e92b66614 HTTP/1.1" 200 None 2025-12-17 17:07:09,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:09,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940402a575bc060ce158a5f HTTP/1.1" 200 None 2025-12-17 17:07:09,909 - lava_callback - INFO - Completed processing callback for node 6940402a575bc060ce158a5f INFO: 172.21.0.1:48626 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:07:14,980 - lava_callback - INFO - Received callback for node 6942e2f8e43d2b76e9023943 INFO: 172.21.0.1:48640 - "POST /node/6942e2f8e43d2b76e9023943 HTTP/1.0" 202 Accepted 2025-12-17 17:07:14,990 - lava_callback - INFO - Starting async job submit for node 6942e2f8e43d2b76e9023943 2025-12-17 17:07:15,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:15,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f8e43d2b76e9023943 HTTP/1.1" 200 None 2025-12-17 17:07:15,082 - lava_callback - DEBUG - Processing node 6942e2f8e43d2b76e9023943: baseline-arm64-pengutronix INFO: 172.21.0.1:48650 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 17:07:15,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:15,428 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:15,429 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-6942e2f8e43d2b76e9023943/log.txt.gz 2025-12-17 17:07:15,430 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:07:15,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:15,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:15,826 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-6942e2f8e43d2b76e9023943/lava_callback.json.gz 2025-12-17 17:07:15,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:15,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f8e43d2b76e9023943 HTTP/1.1" 200 None 2025-12-17 17:07:15,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:15,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:07:15,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:15,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2f8e43d2b76e9023943 HTTP/1.1" 200 None 2025-12-17 17:07:15,918 - lava_callback - INFO - Completed processing callback for node 6942e2f8e43d2b76e9023943 INFO: 172.21.0.1:48658 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:07:17,372 - lava_callback - INFO - Received callback for node 6942e2e5e43d2b76e90238c4 INFO: 172.21.0.1:48674 - "POST /node/6942e2e5e43d2b76e90238c4 HTTP/1.0" 202 Accepted 2025-12-17 17:07:17,380 - lava_callback - INFO - Starting async job submit for node 6942e2e5e43d2b76e90238c4 2025-12-17 17:07:17,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:17,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e5e43d2b76e90238c4 HTTP/1.1" 200 None 2025-12-17 17:07:17,422 - lava_callback - DEBUG - Processing node 6942e2e5e43d2b76e90238c4: baseline-arm64 2025-12-17 17:07:17,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:17,948 - lava_callback - INFO - Received callback for node 6942e2e9e43d2b76e90238d9 INFO: 172.21.0.1:48686 - "POST /node/6942e2e9e43d2b76e90238d9 HTTP/1.0" 202 Accepted 2025-12-17 17:07:17,963 - lava_callback - INFO - Starting async job submit for node 6942e2e9e43d2b76e90238d9 2025-12-17 17:07:17,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:18,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e9e43d2b76e90238d9 HTTP/1.1" 200 None 2025-12-17 17:07:18,013 - lava_callback - DEBUG - Processing node 6942e2e9e43d2b76e90238d9: baseline-arm64 2025-12-17 17:07:18,114 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:18,151 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e5e43d2b76e90238c4/log.txt.gz 2025-12-17 17:07:18,156 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:07:18,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:18,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:18,766 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:18,767 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e5e43d2b76e90238c4/lava_callback.json.gz 2025-12-17 17:07:18,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:18,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:18,826 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e9e43d2b76e90238d9/log.txt.gz 2025-12-17 17:07:18,826 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:07:18,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e5e43d2b76e90238c4 HTTP/1.1" 200 None 2025-12-17 17:07:18,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:18,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:18,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:07:18,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:18,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2e5e43d2b76e90238c4 HTTP/1.1" 200 None 2025-12-17 17:07:18,863 - lava_callback - INFO - Completed processing callback for node 6942e2e5e43d2b76e90238c4 2025-12-17 17:07:19,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:19,038 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e9e43d2b76e90238d9/lava_callback.json.gz 2025-12-17 17:07:19,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:19,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e9e43d2b76e90238d9 HTTP/1.1" 200 None 2025-12-17 17:07:19,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:19,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:07:19,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:19,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2e9e43d2b76e90238d9 HTTP/1.1" 200 None 2025-12-17 17:07:19,110 - lava_callback - INFO - Completed processing callback for node 6942e2e9e43d2b76e90238d9 2025-12-17 17:07:19,766 - lava_callback - INFO - Received callback for node 6942e2e4e43d2b76e90238bb INFO: 172.21.0.1:45736 - "POST /node/6942e2e4e43d2b76e90238bb HTTP/1.0" 202 Accepted 2025-12-17 17:07:19,776 - lava_callback - INFO - Starting async job submit for node 6942e2e4e43d2b76e90238bb 2025-12-17 17:07:19,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:19,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e4e43d2b76e90238bb HTTP/1.1" 200 None 2025-12-17 17:07:19,827 - lava_callback - DEBUG - Processing node 6942e2e4e43d2b76e90238bb: baseline-arm64 2025-12-17 17:07:20,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:20,385 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:20,386 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e4e43d2b76e90238bb/log.txt.gz 2025-12-17 17:07:20,386 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:07:20,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:20,533 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:20,534 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e4e43d2b76e90238bb/lava_callback.json.gz 2025-12-17 17:07:20,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:20,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e4e43d2b76e90238bb HTTP/1.1" 200 None 2025-12-17 17:07:20,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:20,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:07:20,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:20,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2e4e43d2b76e90238bb HTTP/1.1" 200 None 2025-12-17 17:07:20,598 - lava_callback - INFO - Completed processing callback for node 6942e2e4e43d2b76e90238bb 2025-12-17 17:07:22,529 - lava_callback - INFO - Received callback for node 6942e2e8e43d2b76e90238d6 INFO: 172.21.0.1:45742 - "POST /node/6942e2e8e43d2b76e90238d6 HTTP/1.0" 202 Accepted 2025-12-17 17:07:22,536 - lava_callback - INFO - Starting async job submit for node 6942e2e8e43d2b76e90238d6 2025-12-17 17:07:22,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:22,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e8e43d2b76e90238d6 HTTP/1.1" 200 None 2025-12-17 17:07:22,572 - lava_callback - DEBUG - Processing node 6942e2e8e43d2b76e90238d6: baseline-arm64 2025-12-17 17:07:22,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:23,175 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:23,176 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e8e43d2b76e90238d6/log.txt.gz 2025-12-17 17:07:23,176 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:07:23,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:23,343 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:23,344 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e8e43d2b76e90238d6/lava_callback.json.gz 2025-12-17 17:07:23,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:23,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e8e43d2b76e90238d6 HTTP/1.1" 200 None 2025-12-17 17:07:23,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:23,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:07:23,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:23,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2e8e43d2b76e90238d6 HTTP/1.1" 200 None 2025-12-17 17:07:23,401 - lava_callback - INFO - Completed processing callback for node 6942e2e8e43d2b76e90238d6 INFO: 172.21.0.1:45754 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:07:29,985 - lava_callback - INFO - Received callback for node 6942e2e5e43d2b76e90238c1 INFO: 172.21.0.1:44610 - "POST /node/6942e2e5e43d2b76e90238c1 HTTP/1.0" 202 Accepted 2025-12-17 17:07:29,995 - lava_callback - INFO - Starting async job submit for node 6942e2e5e43d2b76e90238c1 2025-12-17 17:07:30,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:30,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e5e43d2b76e90238c1 HTTP/1.1" 200 None 2025-12-17 17:07:30,036 - lava_callback - DEBUG - Processing node 6942e2e5e43d2b76e90238c1: baseline-arm64 2025-12-17 17:07:30,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:30,460 - lava_callback - INFO - Received callback for node 6942e2e6e43d2b76e90238ca INFO: 172.21.0.1:44620 - "POST /node/6942e2e6e43d2b76e90238ca HTTP/1.0" 202 Accepted 2025-12-17 17:07:30,470 - lava_callback - INFO - Starting async job submit for node 6942e2e6e43d2b76e90238ca 2025-12-17 17:07:30,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:30,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e6e43d2b76e90238ca HTTP/1.1" 200 None 2025-12-17 17:07:30,532 - lava_callback - DEBUG - Processing node 6942e2e6e43d2b76e90238ca: baseline-arm64 2025-12-17 17:07:30,692 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:30,693 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e5e43d2b76e90238c1/log.txt.gz 2025-12-17 17:07:30,694 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:07:30,699 - lava_callback - INFO - Received callback for node 6942e2f5e43d2b76e9023937 2025-12-17 17:07:30,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:44624 - "POST /node/6942e2f5e43d2b76e9023937 HTTP/1.0" 202 Accepted 2025-12-17 17:07:30,725 - lava_callback - INFO - Starting async job submit for node 6942e2f5e43d2b76e9023937 2025-12-17 17:07:30,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:30,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f5e43d2b76e9023937 HTTP/1.1" 200 None 2025-12-17 17:07:30,888 - lava_callback - DEBUG - Processing node 6942e2f5e43d2b76e9023937: baseline-arm64-kontron 2025-12-17 17:07:31,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:31,154 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e5e43d2b76e90238c1/lava_callback.json.gz 2025-12-17 17:07:31,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:31,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e5e43d2b76e90238c1 HTTP/1.1" 200 None 2025-12-17 17:07:31,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:31,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:31,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:07:31,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:31,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2e5e43d2b76e90238c1 HTTP/1.1" 200 None 2025-12-17 17:07:31,524 - lava_callback - INFO - Completed processing callback for node 6942e2e5e43d2b76e90238c1 2025-12-17 17:07:31,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:31,656 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:31,657 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e6e43d2b76e90238ca/log.txt.gz 2025-12-17 17:07:31,657 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:07:31,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:31,727 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:31,728 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-6942e2f5e43d2b76e9023937/log.txt.gz 2025-12-17 17:07:31,728 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:07:31,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:31,829 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:31,830 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e6e43d2b76e90238ca/lava_callback.json.gz 2025-12-17 17:07:31,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:31,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e6e43d2b76e90238ca HTTP/1.1" 200 None 2025-12-17 17:07:31,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:31,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:07:31,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:31,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2e6e43d2b76e90238ca HTTP/1.1" 200 None 2025-12-17 17:07:31,926 - lava_callback - INFO - Completed processing callback for node 6942e2e6e43d2b76e90238ca 2025-12-17 17:07:31,999 - lava_callback - INFO - Received callback for node 6942e2e7e43d2b76e90238cd INFO: 172.21.0.1:44628 - "POST /node/6942e2e7e43d2b76e90238cd HTTP/1.0" 202 Accepted 2025-12-17 17:07:32,013 - lava_callback - INFO - Starting async job submit for node 6942e2e7e43d2b76e90238cd 2025-12-17 17:07:32,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:32,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e7e43d2b76e90238cd HTTP/1.1" 200 None 2025-12-17 17:07:32,072 - lava_callback - DEBUG - Processing node 6942e2e7e43d2b76e90238cd: baseline-arm64 2025-12-17 17:07:32,204 - lava_callback - INFO - Received callback for node 6942e2f7e43d2b76e902393d INFO: 172.21.0.1:44644 - "POST /node/6942e2f7e43d2b76e902393d HTTP/1.0" 202 Accepted 2025-12-17 17:07:32,213 - lava_callback - INFO - Starting async job submit for node 6942e2f7e43d2b76e902393d 2025-12-17 17:07:32,225 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:32,253 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-6942e2f5e43d2b76e9023937/lava_callback.json.gz 2025-12-17 17:07:32,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:32,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:32,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f7e43d2b76e902393d HTTP/1.1" 200 None 2025-12-17 17:07:32,448 - lava_callback - DEBUG - Processing node 6942e2f7e43d2b76e902393d: baseline-arm64-kontron 2025-12-17 17:07:32,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f5e43d2b76e9023937 HTTP/1.1" 200 None 2025-12-17 17:07:32,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:32,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:07:32,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:32,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2f5e43d2b76e9023937 HTTP/1.1" 200 None 2025-12-17 17:07:32,958 - lava_callback - INFO - Completed processing callback for node 6942e2f5e43d2b76e9023937 2025-12-17 17:07:33,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:33,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:33,396 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:33,397 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-6942e2f7e43d2b76e902393d/log.txt.gz 2025-12-17 17:07:33,397 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:07:33,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:33,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:33,426 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e7e43d2b76e90238cd/log.txt.gz 2025-12-17 17:07:33,426 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:07:33,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:33,781 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:33,782 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-6942e2f7e43d2b76e902393d/lava_callback.json.gz 2025-12-17 17:07:33,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:33,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f7e43d2b76e902393d HTTP/1.1" 200 None 2025-12-17 17:07:33,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:33,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:07:33,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:33,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2f7e43d2b76e902393d HTTP/1.1" 200 None 2025-12-17 17:07:33,875 - lava_callback - INFO - Completed processing callback for node 6942e2f7e43d2b76e902393d 2025-12-17 17:07:33,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:33,914 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e7e43d2b76e90238cd/lava_callback.json.gz 2025-12-17 17:07:33,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:33,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e7e43d2b76e90238cd HTTP/1.1" 200 None 2025-12-17 17:07:33,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:33,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:07:33,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:34,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2e7e43d2b76e90238cd HTTP/1.1" 200 None 2025-12-17 17:07:34,017 - lava_callback - INFO - Completed processing callback for node 6942e2e7e43d2b76e90238cd INFO: 172.21.0.1:44660 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:07:39,145 - lava_callback - INFO - Received callback for node 6942e2eee43d2b76e9023909 INFO: 172.21.0.1:44666 - "POST /node/6942e2eee43d2b76e9023909 HTTP/1.0" 202 Accepted 2025-12-17 17:07:39,155 - lava_callback - INFO - Starting async job submit for node 6942e2eee43d2b76e9023909 2025-12-17 17:07:39,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:39,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2eee43d2b76e9023909 HTTP/1.1" 200 None 2025-12-17 17:07:39,217 - lava_callback - DEBUG - Processing node 6942e2eee43d2b76e9023909: baseline-nfs-arm64 2025-12-17 17:07:39,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:40,064 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:40,065 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2eee43d2b76e9023909/log.txt.gz 2025-12-17 17:07:40,066 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:07:40,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:40,252 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:40,253 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2eee43d2b76e9023909/lava_callback.json.gz 2025-12-17 17:07:40,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:40,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2eee43d2b76e9023909 HTTP/1.1" 200 None 2025-12-17 17:07:40,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:40,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:07:40,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:40,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2eee43d2b76e9023909 HTTP/1.1" 200 None 2025-12-17 17:07:40,314 - lava_callback - INFO - Completed processing callback for node 6942e2eee43d2b76e9023909 INFO: 172.21.0.1:45608 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:07:47,215 - lava_callback - INFO - Received callback for node 6942e2ece43d2b76e90238f7 INFO: 172.21.0.1:45616 - "POST /node/6942e2ece43d2b76e90238f7 HTTP/1.0" 202 Accepted 2025-12-17 17:07:47,222 - lava_callback - INFO - Starting async job submit for node 6942e2ece43d2b76e90238f7 2025-12-17 17:07:47,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:47,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2ece43d2b76e90238f7 HTTP/1.1" 200 None 2025-12-17 17:07:47,278 - lava_callback - DEBUG - Processing node 6942e2ece43d2b76e90238f7: baseline-nfs-arm64 2025-12-17 17:07:47,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:48,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:48,069 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2ece43d2b76e90238f7/log.txt.gz 2025-12-17 17:07:48,069 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:07:48,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:48,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:48,472 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2ece43d2b76e90238f7/lava_callback.json.gz 2025-12-17 17:07:48,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:48,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2ece43d2b76e90238f7 HTTP/1.1" 200 None 2025-12-17 17:07:48,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:48,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:07:48,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:48,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2ece43d2b76e90238f7 HTTP/1.1" 200 None 2025-12-17 17:07:48,561 - lava_callback - INFO - Completed processing callback for node 6942e2ece43d2b76e90238f7 2025-12-17 17:07:49,976 - lava_callback - INFO - Received callback for node 69404064575bc060ce158a6a INFO: 172.21.0.1:44470 - "POST /node/69404064575bc060ce158a6a HTTP/1.0" 202 Accepted 2025-12-17 17:07:49,982 - lava_callback - INFO - Starting async job submit for node 69404064575bc060ce158a6a 2025-12-17 17:07:49,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:49,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404064575bc060ce158a6a HTTP/1.1" 200 None 2025-12-17 17:07:49,991 - lava_callback - DEBUG - Processing node 69404064575bc060ce158a6a: baseline-arm-broonie 2025-12-17 17:07:49,991 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69404064575bc060ce158a6a 2025-12-17 17:07:49,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:49,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404064575bc060ce158a6a HTTP/1.1" 200 None 2025-12-17 17:07:50,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:50,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693aeba3fee8383e92b66614 HTTP/1.1" 200 None 2025-12-17 17:07:50,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:50,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69404064575bc060ce158a6a HTTP/1.1" 200 None 2025-12-17 17:07:50,022 - lava_callback - INFO - Completed processing callback for node 69404064575bc060ce158a6a 2025-12-17 17:07:50,970 - lava_callback - INFO - Received callback for node 6942e2e9e43d2b76e90238dc INFO: 172.21.0.1:44476 - "POST /node/6942e2e9e43d2b76e90238dc HTTP/1.0" 202 Accepted 2025-12-17 17:07:50,980 - lava_callback - INFO - Starting async job submit for node 6942e2e9e43d2b76e90238dc 2025-12-17 17:07:51,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:51,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e9e43d2b76e90238dc HTTP/1.1" 200 None 2025-12-17 17:07:51,055 - lava_callback - DEBUG - Processing node 6942e2e9e43d2b76e90238dc: baseline-arm64 2025-12-17 17:07:51,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:51,733 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:51,734 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e9e43d2b76e90238dc/log.txt.gz 2025-12-17 17:07:51,734 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:07:51,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:52,054 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:52,055 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e9e43d2b76e90238dc/lava_callback.json.gz 2025-12-17 17:07:52,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:52,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e9e43d2b76e90238dc HTTP/1.1" 200 None 2025-12-17 17:07:52,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:52,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:07:52,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:52,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2e9e43d2b76e90238dc HTTP/1.1" 200 None 2025-12-17 17:07:52,128 - lava_callback - INFO - Completed processing callback for node 6942e2e9e43d2b76e90238dc INFO: 172.21.0.1:44482 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:07:56,928 - lava_callback - INFO - Received callback for node 6940405a575bc060ce158a65 INFO: 172.21.0.1:44498 - "POST /node/6940405a575bc060ce158a65 HTTP/1.0" 202 Accepted 2025-12-17 17:07:56,934 - lava_callback - INFO - Starting async job submit for node 6940405a575bc060ce158a65 2025-12-17 17:07:56,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:56,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940405a575bc060ce158a65 HTTP/1.1" 200 None 2025-12-17 17:07:56,942 - lava_callback - DEBUG - Processing node 6940405a575bc060ce158a65: baseline-arm 2025-12-17 17:07:56,942 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940405a575bc060ce158a65 2025-12-17 17:07:56,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:56,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940405a575bc060ce158a65 HTTP/1.1" 200 None 2025-12-17 17:07:56,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:56,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693aeba3fee8383e92b66614 HTTP/1.1" 200 None 2025-12-17 17:07:56,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:56,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940405a575bc060ce158a65 HTTP/1.1" 200 None 2025-12-17 17:07:56,985 - lava_callback - INFO - Completed processing callback for node 6940405a575bc060ce158a65 2025-12-17 17:07:57,933 - lava_callback - INFO - Received callback for node 6942e2ede43d2b76e9023903 INFO: 172.21.0.1:44512 - "POST /node/6942e2ede43d2b76e9023903 HTTP/1.0" 202 Accepted 2025-12-17 17:07:57,941 - lava_callback - INFO - Starting async job submit for node 6942e2ede43d2b76e9023903 2025-12-17 17:07:57,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:58,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2ede43d2b76e9023903 HTTP/1.1" 200 None 2025-12-17 17:07:58,006 - lava_callback - DEBUG - Processing node 6942e2ede43d2b76e9023903: baseline-nfs-arm64 2025-12-17 17:07:58,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:58,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:58,618 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2ede43d2b76e9023903/log.txt.gz 2025-12-17 17:07:58,618 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:07:58,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:07:58,805 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:07:58,805 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2ede43d2b76e9023903/lava_callback.json.gz 2025-12-17 17:07:58,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:58,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2ede43d2b76e9023903 HTTP/1.1" 200 None 2025-12-17 17:07:58,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:58,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:07:58,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:58,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2ede43d2b76e9023903 HTTP/1.1" 200 None 2025-12-17 17:07:58,873 - lava_callback - INFO - Completed processing callback for node 6942e2ede43d2b76e9023903 2025-12-17 17:07:59,950 - lava_callback - INFO - Received callback for node 6942e2ece43d2b76e90238fa INFO: 172.21.0.1:59668 - "POST /node/6942e2ece43d2b76e90238fa HTTP/1.0" 202 Accepted 2025-12-17 17:07:59,957 - lava_callback - INFO - Starting async job submit for node 6942e2ece43d2b76e90238fa 2025-12-17 17:07:59,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:07:59,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2ece43d2b76e90238fa HTTP/1.1" 200 None 2025-12-17 17:07:59,990 - lava_callback - DEBUG - Processing node 6942e2ece43d2b76e90238fa: baseline-nfs-arm64 2025-12-17 17:08:00,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:08:00,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:08:00,849 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2ece43d2b76e90238fa/log.txt.gz 2025-12-17 17:08:00,849 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:08:00,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:08:01,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:08:01,016 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2ece43d2b76e90238fa/lava_callback.json.gz 2025-12-17 17:08:01,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:01,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2ece43d2b76e90238fa HTTP/1.1" 200 None 2025-12-17 17:08:01,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:01,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:08:01,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:01,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2ece43d2b76e90238fa HTTP/1.1" 200 None 2025-12-17 17:08:01,114 - lava_callback - INFO - Completed processing callback for node 6942e2ece43d2b76e90238fa INFO: 172.21.0.1:59680 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:08:07,961 - lava_callback - INFO - Received callback for node 6942e2e8e43d2b76e90238d3 INFO: 172.21.0.1:59694 - "POST /node/6942e2e8e43d2b76e90238d3 HTTP/1.0" 202 Accepted 2025-12-17 17:08:07,973 - lava_callback - INFO - Starting async job submit for node 6942e2e8e43d2b76e90238d3 2025-12-17 17:08:08,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:08,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e8e43d2b76e90238d3 HTTP/1.1" 200 None 2025-12-17 17:08:08,018 - lava_callback - DEBUG - Processing node 6942e2e8e43d2b76e90238d3: baseline-arm64 2025-12-17 17:08:08,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:08:08,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:08:08,814 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e8e43d2b76e90238d3/log.txt.gz 2025-12-17 17:08:08,814 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:08:08,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:08:09,038 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:08:09,038 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e2e8e43d2b76e90238d3/lava_callback.json.gz 2025-12-17 17:08:09,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:09,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2e8e43d2b76e90238d3 HTTP/1.1" 200 None 2025-12-17 17:08:09,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:09,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:08:09,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:09,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2e8e43d2b76e90238d3 HTTP/1.1" 200 None 2025-12-17 17:08:09,144 - lava_callback - INFO - Completed processing callback for node 6942e2e8e43d2b76e90238d3 INFO: 172.21.0.1:43534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43538 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 17:08:15,970 - lava_callback - INFO - Received callback for node 6942e2ebe43d2b76e90238f1 INFO: 172.21.0.1:43550 - "POST /node/6942e2ebe43d2b76e90238f1 HTTP/1.0" 202 Accepted 2025-12-17 17:08:15,980 - lava_callback - INFO - Starting async job submit for node 6942e2ebe43d2b76e90238f1 2025-12-17 17:08:16,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:16,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2ebe43d2b76e90238f1 HTTP/1.1" 200 None 2025-12-17 17:08:16,052 - lava_callback - DEBUG - Processing node 6942e2ebe43d2b76e90238f1: baseline-nfs-arm64 INFO: 172.21.0.1:43562 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:08:16,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:08:16,770 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:08:16,771 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2ebe43d2b76e90238f1/log.txt.gz 2025-12-17 17:08:16,771 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:08:16,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:08:16,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:08:16,995 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2ebe43d2b76e90238f1/lava_callback.json.gz 2025-12-17 17:08:17,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:17,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2ebe43d2b76e90238f1 HTTP/1.1" 200 None 2025-12-17 17:08:17,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:17,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:08:17,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:17,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2ebe43d2b76e90238f1 HTTP/1.1" 200 None 2025-12-17 17:08:17,084 - lava_callback - INFO - Completed processing callback for node 6942e2ebe43d2b76e90238f1 2025-12-17 17:08:19,787 - lava_callback - INFO - Received callback for node 6942e2ebe43d2b76e90238ee INFO: 172.21.0.1:59386 - "POST /node/6942e2ebe43d2b76e90238ee HTTP/1.0" 202 Accepted 2025-12-17 17:08:19,794 - lava_callback - INFO - Starting async job submit for node 6942e2ebe43d2b76e90238ee 2025-12-17 17:08:19,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:19,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2ebe43d2b76e90238ee HTTP/1.1" 200 None 2025-12-17 17:08:19,869 - lava_callback - DEBUG - Processing node 6942e2ebe43d2b76e90238ee: baseline-nfs-arm64 2025-12-17 17:08:20,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:08:20,479 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:08:20,480 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2ebe43d2b76e90238ee/log.txt.gz 2025-12-17 17:08:20,480 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:08:20,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:08:20,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:08:20,620 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2ebe43d2b76e90238ee/lava_callback.json.gz 2025-12-17 17:08:20,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:20,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2ebe43d2b76e90238ee HTTP/1.1" 200 None 2025-12-17 17:08:20,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:20,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:08:20,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:20,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2ebe43d2b76e90238ee HTTP/1.1" 200 None 2025-12-17 17:08:20,682 - lava_callback - INFO - Completed processing callback for node 6942e2ebe43d2b76e90238ee INFO: 172.21.0.1:59398 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:08:28,499 - lava_callback - INFO - Received callback for node 6942e2eee43d2b76e9023906 INFO: 172.21.0.1:59406 - "POST /node/6942e2eee43d2b76e9023906 HTTP/1.0" 202 Accepted 2025-12-17 17:08:28,510 - lava_callback - INFO - Starting async job submit for node 6942e2eee43d2b76e9023906 2025-12-17 17:08:28,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:28,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2eee43d2b76e9023906 HTTP/1.1" 200 None 2025-12-17 17:08:28,565 - lava_callback - DEBUG - Processing node 6942e2eee43d2b76e9023906: baseline-nfs-arm64 2025-12-17 17:08:29,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:08:29,330 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:08:29,331 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2eee43d2b76e9023906/log.txt.gz 2025-12-17 17:08:29,331 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:08:29,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:08:29,545 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:08:29,546 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2eee43d2b76e9023906/lava_callback.json.gz 2025-12-17 17:08:29,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:29,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2eee43d2b76e9023906 HTTP/1.1" 200 None 2025-12-17 17:08:29,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:29,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:08:29,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:29,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2eee43d2b76e9023906 HTTP/1.1" 200 None 2025-12-17 17:08:29,638 - lava_callback - INFO - Completed processing callback for node 6942e2eee43d2b76e9023906 2025-12-17 17:08:30,906 - lava_callback - INFO - Received callback for node 6942e2f6e43d2b76e902393a INFO: 172.21.0.1:48738 - "POST /node/6942e2f6e43d2b76e902393a HTTP/1.0" 202 Accepted 2025-12-17 17:08:30,916 - lava_callback - INFO - Starting async job submit for node 6942e2f6e43d2b76e902393a 2025-12-17 17:08:30,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:30,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f6e43d2b76e902393a HTTP/1.1" 200 None 2025-12-17 17:08:30,971 - lava_callback - DEBUG - Processing node 6942e2f6e43d2b76e902393a: baseline-arm64-kontron 2025-12-17 17:08:31,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:08:31,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:08:31,711 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-6942e2f6e43d2b76e902393a/log.txt.gz 2025-12-17 17:08:31,711 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:08:31,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:08:31,891 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:08:31,892 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-6942e2f6e43d2b76e902393a/lava_callback.json.gz 2025-12-17 17:08:31,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:31,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f6e43d2b76e902393a HTTP/1.1" 200 None 2025-12-17 17:08:31,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:31,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:08:31,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:32,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2f6e43d2b76e902393a HTTP/1.1" 200 None 2025-12-17 17:08:32,012 - lava_callback - INFO - Completed processing callback for node 6942e2f6e43d2b76e902393a INFO: 172.21.0.1:48742 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:08:36,980 - lava_callback - INFO - Received callback for node 6942e36ee43d2b76e90239c5 INFO: 172.21.0.1:48752 - "POST /node/6942e36ee43d2b76e90239c5 HTTP/1.0" 202 Accepted 2025-12-17 17:08:36,988 - lava_callback - INFO - Starting async job submit for node 6942e36ee43d2b76e90239c5 2025-12-17 17:08:37,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:37,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e36ee43d2b76e90239c5 HTTP/1.1" 200 None 2025-12-17 17:08:37,016 - lava_callback - DEBUG - Processing node 6942e36ee43d2b76e90239c5: baseline-arm64 2025-12-17 17:08:37,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:08:37,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:08:37,309 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e36ee43d2b76e90239c5/log.txt.gz 2025-12-17 17:08:37,309 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:08:37,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:08:37,493 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:08:37,494 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e36ee43d2b76e90239c5/lava_callback.json.gz 2025-12-17 17:08:37,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:37,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e36ee43d2b76e90239c5 HTTP/1.1" 200 None 2025-12-17 17:08:37,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:37,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:08:37,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:37,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e36ee43d2b76e90239c5 HTTP/1.1" 200 None 2025-12-17 17:08:37,570 - lava_callback - INFO - Completed processing callback for node 6942e36ee43d2b76e90239c5 2025-12-17 17:08:40,349 - lava_callback - INFO - Received callback for node 6942e2ede43d2b76e9023900 INFO: 172.21.0.1:45776 - "POST /node/6942e2ede43d2b76e9023900 HTTP/1.0" 202 Accepted 2025-12-17 17:08:40,359 - lava_callback - INFO - Starting async job submit for node 6942e2ede43d2b76e9023900 2025-12-17 17:08:40,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:40,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2ede43d2b76e9023900 HTTP/1.1" 200 None 2025-12-17 17:08:40,398 - lava_callback - DEBUG - Processing node 6942e2ede43d2b76e9023900: baseline-nfs-arm64 2025-12-17 17:08:40,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:08:41,141 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:08:41,142 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2ede43d2b76e9023900/log.txt.gz 2025-12-17 17:08:41,142 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:08:41,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:08:41,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:08:41,369 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e2ede43d2b76e9023900/lava_callback.json.gz 2025-12-17 17:08:41,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:41,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2ede43d2b76e9023900 HTTP/1.1" 200 None 2025-12-17 17:08:41,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:41,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:08:41,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:41,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2ede43d2b76e9023900 HTTP/1.1" 200 None 2025-12-17 17:08:41,484 - lava_callback - INFO - Completed processing callback for node 6942e2ede43d2b76e9023900 INFO: 172.21.0.1:45788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48202 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:08:57,381 - lava_callback - INFO - Received callback for node 694040a3575bc060ce158a77 INFO: 172.21.0.1:48206 - "POST /node/694040a3575bc060ce158a77 HTTP/1.0" 202 Accepted 2025-12-17 17:08:57,387 - lava_callback - INFO - Starting async job submit for node 694040a3575bc060ce158a77 2025-12-17 17:08:57,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:57,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694040a3575bc060ce158a77 HTTP/1.1" 200 None 2025-12-17 17:08:57,585 - lava_callback - DEBUG - Processing node 694040a3575bc060ce158a77: baseline-arm-pengutronix 2025-12-17 17:08:57,585 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694040a3575bc060ce158a77 2025-12-17 17:08:57,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:57,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694040a3575bc060ce158a77 HTTP/1.1" 200 None 2025-12-17 17:08:57,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:57,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af17afee8383e92b69dc1 HTTP/1.1" 200 None 2025-12-17 17:08:57,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:08:57,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694040a3575bc060ce158a77 HTTP/1.1" 200 None 2025-12-17 17:08:57,821 - lava_callback - INFO - Completed processing callback for node 694040a3575bc060ce158a77 INFO: 172.21.0.1:46218 - "GET /node/68bb85ce981cab3aaf28eae9%5Cn HTTP/1.0" 405 Method Not Allowed 2025-12-17 17:09:00,754 - lava_callback - INFO - Received callback for node 6942e371e43d2b76e90239e1 INFO: 172.21.0.1:46220 - "POST /node/6942e371e43d2b76e90239e1 HTTP/1.0" 202 Accepted 2025-12-17 17:09:00,761 - lava_callback - INFO - Starting async job submit for node 6942e371e43d2b76e90239e1 2025-12-17 17:09:00,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:00,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e371e43d2b76e90239e1 HTTP/1.1" 200 None 2025-12-17 17:09:00,790 - lava_callback - DEBUG - Processing node 6942e371e43d2b76e90239e1: baseline-arm64 2025-12-17 17:09:00,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:01,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:01,146 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e371e43d2b76e90239e1/log.txt.gz 2025-12-17 17:09:01,146 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:01,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:01,320 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:01,321 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e371e43d2b76e90239e1/lava_callback.json.gz 2025-12-17 17:09:01,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:01,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e371e43d2b76e90239e1 HTTP/1.1" 200 None 2025-12-17 17:09:01,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:01,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:09:01,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:01,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e371e43d2b76e90239e1 HTTP/1.1" 200 None 2025-12-17 17:09:01,380 - lava_callback - INFO - Completed processing callback for node 6942e371e43d2b76e90239e1 2025-12-17 17:09:03,078 - lava_callback - INFO - Received callback for node 6942e377e43d2b76e9023a0f INFO: 172.21.0.1:46228 - "POST /node/6942e377e43d2b76e9023a0f HTTP/1.0" 202 Accepted 2025-12-17 17:09:03,085 - lava_callback - INFO - Starting async job submit for node 6942e377e43d2b76e9023a0f 2025-12-17 17:09:03,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:03,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e377e43d2b76e9023a0f HTTP/1.1" 200 None 2025-12-17 17:09:03,114 - lava_callback - DEBUG - Processing node 6942e377e43d2b76e9023a0f: baseline-nfs-arm64 2025-12-17 17:09:03,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:03,422 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:03,423 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e377e43d2b76e9023a0f/log.txt.gz 2025-12-17 17:09:03,423 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:03,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:03,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:03,605 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e377e43d2b76e9023a0f/lava_callback.json.gz 2025-12-17 17:09:03,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:03,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e377e43d2b76e9023a0f HTTP/1.1" 200 None 2025-12-17 17:09:03,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:03,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:09:03,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:03,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e377e43d2b76e9023a0f HTTP/1.1" 200 None 2025-12-17 17:09:03,661 - lava_callback - INFO - Completed processing callback for node 6942e377e43d2b76e9023a0f INFO: 172.21.0.1:46230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55544 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:09:23,821 - lava_callback - INFO - Received callback for node 6942e36fe43d2b76e90239cb INFO: 172.21.0.1:38686 - "POST /node/6942e36fe43d2b76e90239cb HTTP/1.0" 202 Accepted 2025-12-17 17:09:23,828 - lava_callback - INFO - Starting async job submit for node 6942e36fe43d2b76e90239cb 2025-12-17 17:09:23,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:23,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e36fe43d2b76e90239cb HTTP/1.1" 200 None 2025-12-17 17:09:23,859 - lava_callback - DEBUG - Processing node 6942e36fe43d2b76e90239cb: baseline-arm64 2025-12-17 17:09:24,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:24,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:24,236 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e36fe43d2b76e90239cb/log.txt.gz 2025-12-17 17:09:24,237 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:24,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:24,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:24,369 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e36fe43d2b76e90239cb/lava_callback.json.gz 2025-12-17 17:09:24,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:24,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e36fe43d2b76e90239cb HTTP/1.1" 200 None 2025-12-17 17:09:24,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:24,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:09:24,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:24,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e36fe43d2b76e90239cb HTTP/1.1" 200 None 2025-12-17 17:09:24,460 - lava_callback - INFO - Completed processing callback for node 6942e36fe43d2b76e90239cb INFO: 172.21.0.1:38698 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:09:26,693 - lava_callback - INFO - Received callback for node 6942e389e43d2b76e9023aa9 INFO: 172.21.0.1:38706 - "POST /node/6942e389e43d2b76e9023aa9 HTTP/1.0" 202 Accepted 2025-12-17 17:09:26,699 - lava_callback - INFO - Starting async job submit for node 6942e389e43d2b76e9023aa9 2025-12-17 17:09:26,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:26,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e389e43d2b76e9023aa9 HTTP/1.1" 200 None 2025-12-17 17:09:26,728 - lava_callback - DEBUG - Processing node 6942e389e43d2b76e9023aa9: baseline-arm64-pengutronix 2025-12-17 17:09:26,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:27,026 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:27,026 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-6942e389e43d2b76e9023aa9/log.txt.gz 2025-12-17 17:09:27,027 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:27,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:27,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:27,156 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-6942e389e43d2b76e9023aa9/lava_callback.json.gz 2025-12-17 17:09:27,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:27,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e389e43d2b76e9023aa9 HTTP/1.1" 200 None 2025-12-17 17:09:27,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:27,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:09:27,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:27,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e389e43d2b76e9023aa9 HTTP/1.1" 200 None 2025-12-17 17:09:27,233 - lava_callback - INFO - Completed processing callback for node 6942e389e43d2b76e9023aa9 2025-12-17 17:09:30,365 - lava_callback - INFO - Received callback for node 6942e370e43d2b76e90239d1 INFO: 172.21.0.1:38640 - "POST /node/6942e370e43d2b76e90239d1 HTTP/1.0" 202 Accepted 2025-12-17 17:09:30,373 - lava_callback - INFO - Starting async job submit for node 6942e370e43d2b76e90239d1 2025-12-17 17:09:30,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:30,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e370e43d2b76e90239d1 HTTP/1.1" 200 None 2025-12-17 17:09:30,434 - lava_callback - DEBUG - Processing node 6942e370e43d2b76e90239d1: baseline-arm64 2025-12-17 17:09:30,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:30,871 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:30,872 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e370e43d2b76e90239d1/log.txt.gz 2025-12-17 17:09:30,872 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:30,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:31,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:31,030 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e370e43d2b76e90239d1/lava_callback.json.gz 2025-12-17 17:09:31,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:31,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e370e43d2b76e90239d1 HTTP/1.1" 200 None 2025-12-17 17:09:31,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:31,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:09:31,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:31,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e370e43d2b76e90239d1 HTTP/1.1" 200 None 2025-12-17 17:09:31,092 - lava_callback - INFO - Completed processing callback for node 6942e370e43d2b76e90239d1 2025-12-17 17:09:33,029 - lava_callback - INFO - Received callback for node 6942e36fe43d2b76e90239c8 INFO: 172.21.0.1:38654 - "POST /node/6942e36fe43d2b76e90239c8 HTTP/1.0" 202 Accepted 2025-12-17 17:09:33,036 - lava_callback - INFO - Starting async job submit for node 6942e36fe43d2b76e90239c8 2025-12-17 17:09:33,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:33,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e36fe43d2b76e90239c8 HTTP/1.1" 200 None 2025-12-17 17:09:33,067 - lava_callback - DEBUG - Processing node 6942e36fe43d2b76e90239c8: baseline-arm64 2025-12-17 17:09:33,153 - lava_callback - INFO - Received callback for node 69403c36575bc060ce158588 INFO: 172.21.0.1:38656 - "POST /node/69403c36575bc060ce158588 HTTP/1.0" 202 Accepted 2025-12-17 17:09:33,178 - lava_callback - INFO - Starting async job submit for node 69403c36575bc060ce158588 2025-12-17 17:09:33,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:33,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c36575bc060ce158588 HTTP/1.1" 200 None 2025-12-17 17:09:33,309 - lava_callback - DEBUG - Processing node 69403c36575bc060ce158588: baseline-arm 2025-12-17 17:09:33,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:33,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:33,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:33,736 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e36fe43d2b76e90239c8/log.txt.gz 2025-12-17 17:09:33,736 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:33,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:33,834 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:33,835 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69403c36575bc060ce158588/log.txt.gz 2025-12-17 17:09:33,835 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:33,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:33,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:33,885 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e36fe43d2b76e90239c8/lava_callback.json.gz 2025-12-17 17:09:33,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:33,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e36fe43d2b76e90239c8 HTTP/1.1" 200 None 2025-12-17 17:09:33,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:33,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:09:33,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:33,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e36fe43d2b76e90239c8 HTTP/1.1" 200 None 2025-12-17 17:09:33,981 - lava_callback - INFO - Completed processing callback for node 6942e36fe43d2b76e90239c8 2025-12-17 17:09:34,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:34,003 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69403c36575bc060ce158588/lava_callback.json.gz DEBUG: Setup failed for 69403c36575bc060ce158588. Transitting job node result: incomplete -> fail 2025-12-17 17:09:34,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:34,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403c36575bc060ce158588 HTTP/1.1" 200 None 2025-12-17 17:09:34,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:34,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d922277ecffe173f95b5a HTTP/1.1" 200 None 2025-12-17 17:09:34,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:34,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403c36575bc060ce158588 HTTP/1.1" 200 None 2025-12-17 17:09:34,150 - lava_callback - INFO - Completed processing callback for node 69403c36575bc060ce158588 2025-12-17 17:09:36,238 - lava_callback - INFO - Received callback for node 6942e373e43d2b76e90239eb INFO: 172.21.0.1:38662 - "POST /node/6942e373e43d2b76e90239eb HTTP/1.0" 202 Accepted 2025-12-17 17:09:36,245 - lava_callback - INFO - Starting async job submit for node 6942e373e43d2b76e90239eb 2025-12-17 17:09:36,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:36,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e373e43d2b76e90239eb HTTP/1.1" 200 None 2025-12-17 17:09:36,278 - lava_callback - DEBUG - Processing node 6942e373e43d2b76e90239eb: baseline-arm64 INFO: 172.21.0.1:38668 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:09:36,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:36,739 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:36,739 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e373e43d2b76e90239eb/log.txt.gz 2025-12-17 17:09:36,740 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:36,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:36,891 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:36,892 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e373e43d2b76e90239eb/lava_callback.json.gz 2025-12-17 17:09:36,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:36,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e373e43d2b76e90239eb HTTP/1.1" 200 None 2025-12-17 17:09:36,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:36,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:09:36,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:36,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e373e43d2b76e90239eb HTTP/1.1" 200 None 2025-12-17 17:09:36,953 - lava_callback - INFO - Completed processing callback for node 6942e373e43d2b76e90239eb 2025-12-17 17:09:39,281 - lava_callback - INFO - Received callback for node 6942e38be43d2b76e9023ab4 INFO: 172.21.0.1:38672 - "POST /node/6942e38be43d2b76e9023ab4 HTTP/1.0" 202 Accepted 2025-12-17 17:09:39,287 - lava_callback - INFO - Starting async job submit for node 6942e38be43d2b76e9023ab4 2025-12-17 17:09:39,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:39,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e38be43d2b76e9023ab4 HTTP/1.1" 200 None 2025-12-17 17:09:39,317 - lava_callback - DEBUG - Processing node 6942e38be43d2b76e9023ab4: baseline-arm64-foundriesio 2025-12-17 17:09:39,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:39,575 - lava_callback - INFO - Received callback for node 6942e373e43d2b76e90239ee INFO: 172.21.0.1:38684 - "POST /node/6942e373e43d2b76e90239ee HTTP/1.0" 202 Accepted 2025-12-17 17:09:39,581 - lava_callback - INFO - Starting async job submit for node 6942e373e43d2b76e90239ee 2025-12-17 17:09:39,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:39,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e373e43d2b76e90239ee HTTP/1.1" 200 None 2025-12-17 17:09:39,612 - lava_callback - DEBUG - Processing node 6942e373e43d2b76e90239ee: baseline-arm64 2025-12-17 17:09:39,639 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:39,701 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-6942e38be43d2b76e9023ab4/log.txt.gz 2025-12-17 17:09:39,707 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:39,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:39,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:40,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:40,037 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e373e43d2b76e90239ee/log.txt.gz 2025-12-17 17:09:40,037 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:40,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:40,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:40,046 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-6942e38be43d2b76e9023ab4/lava_callback.json.gz 2025-12-17 17:09:40,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:40,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e38be43d2b76e9023ab4 HTTP/1.1" 200 None 2025-12-17 17:09:40,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:40,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:09:40,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:40,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e38be43d2b76e9023ab4 HTTP/1.1" 200 None 2025-12-17 17:09:40,125 - lava_callback - INFO - Completed processing callback for node 6942e38be43d2b76e9023ab4 2025-12-17 17:09:40,230 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:40,231 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e373e43d2b76e90239ee/lava_callback.json.gz 2025-12-17 17:09:40,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:40,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e373e43d2b76e90239ee HTTP/1.1" 200 None 2025-12-17 17:09:40,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:40,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:09:40,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:40,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e373e43d2b76e90239ee HTTP/1.1" 200 None 2025-12-17 17:09:40,311 - lava_callback - INFO - Completed processing callback for node 6942e373e43d2b76e90239ee 2025-12-17 17:09:40,721 - lava_callback - INFO - Received callback for node 6942e371e43d2b76e90239d7 INFO: 172.21.0.1:47708 - "POST /node/6942e371e43d2b76e90239d7 HTTP/1.0" 202 Accepted 2025-12-17 17:09:40,728 - lava_callback - INFO - Starting async job submit for node 6942e371e43d2b76e90239d7 2025-12-17 17:09:40,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:40,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e371e43d2b76e90239d7 HTTP/1.1" 200 None 2025-12-17 17:09:40,763 - lava_callback - DEBUG - Processing node 6942e371e43d2b76e90239d7: baseline-arm64 2025-12-17 17:09:41,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:41,258 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:41,259 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e371e43d2b76e90239d7/log.txt.gz 2025-12-17 17:09:41,259 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:41,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:41,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:41,483 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e371e43d2b76e90239d7/lava_callback.json.gz 2025-12-17 17:09:41,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:41,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e371e43d2b76e90239d7 HTTP/1.1" 200 None 2025-12-17 17:09:41,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:41,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:09:41,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:41,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e371e43d2b76e90239d7 HTTP/1.1" 200 None 2025-12-17 17:09:41,541 - lava_callback - INFO - Completed processing callback for node 6942e371e43d2b76e90239d7 INFO: 172.21.0.1:47712 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:09:47,857 - lava_callback - INFO - Received callback for node 6942e38ae43d2b76e9023aac INFO: 172.21.0.1:47720 - "POST /node/6942e38ae43d2b76e9023aac HTTP/1.0" 202 Accepted 2025-12-17 17:09:47,863 - lava_callback - INFO - Starting async job submit for node 6942e38ae43d2b76e9023aac 2025-12-17 17:09:47,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:47,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e38ae43d2b76e9023aac HTTP/1.1" 200 None 2025-12-17 17:09:47,891 - lava_callback - DEBUG - Processing node 6942e38ae43d2b76e9023aac: baseline-arm64-pengutronix 2025-12-17 17:09:48,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:48,207 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:48,207 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-6942e38ae43d2b76e9023aac/log.txt.gz 2025-12-17 17:09:48,208 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:48,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:48,343 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:48,344 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-6942e38ae43d2b76e9023aac/lava_callback.json.gz 2025-12-17 17:09:48,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:48,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e38ae43d2b76e9023aac HTTP/1.1" 200 None 2025-12-17 17:09:48,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:48,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:09:48,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:48,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e38ae43d2b76e9023aac HTTP/1.1" 200 None 2025-12-17 17:09:48,415 - lava_callback - INFO - Completed processing callback for node 6942e38ae43d2b76e9023aac 2025-12-17 17:09:49,971 - lava_callback - INFO - Received callback for node 694040c9575bc060ce158a81 INFO: 172.21.0.1:58138 - "POST /node/694040c9575bc060ce158a81 HTTP/1.0" 202 Accepted 2025-12-17 17:09:49,977 - lava_callback - INFO - Starting async job submit for node 694040c9575bc060ce158a81 2025-12-17 17:09:49,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:49,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694040c9575bc060ce158a81 HTTP/1.1" 200 None 2025-12-17 17:09:49,986 - lava_callback - DEBUG - Processing node 694040c9575bc060ce158a81: ltp-syscalls 2025-12-17 17:09:49,986 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694040c9575bc060ce158a81 2025-12-17 17:09:49,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:49,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694040c9575bc060ce158a81 HTTP/1.1" 200 None 2025-12-17 17:09:49,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:50,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693aeba3fee8383e92b66614 HTTP/1.1" 200 None 2025-12-17 17:09:50,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:50,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694040c9575bc060ce158a81 HTTP/1.1" 200 None 2025-12-17 17:09:50,024 - lava_callback - INFO - Completed processing callback for node 694040c9575bc060ce158a81 2025-12-17 17:09:50,239 - lava_callback - INFO - Received callback for node 69405cbc575bc060ce1595f3 INFO: 172.21.0.1:58148 - "POST /node/69405cbc575bc060ce1595f3 HTTP/1.0" 202 Accepted 2025-12-17 17:09:50,248 - lava_callback - INFO - Starting async job submit for node 69405cbc575bc060ce1595f3 2025-12-17 17:09:50,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:50,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69405cbc575bc060ce1595f3 HTTP/1.1" 200 None 2025-12-17 17:09:50,348 - lava_callback - DEBUG - Processing node 69405cbc575bc060ce1595f3: baseline-nfs-arm64 2025-12-17 17:09:50,984 - lava_callback - INFO - Received callback for node 6942e388e43d2b76e9023a9c INFO: 172.21.0.1:58158 - "POST /node/6942e388e43d2b76e9023a9c HTTP/1.0" 202 Accepted 2025-12-17 17:09:51,013 - lava_callback - INFO - Starting async job submit for node 6942e388e43d2b76e9023a9c 2025-12-17 17:09:51,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:51,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e388e43d2b76e9023a9c HTTP/1.1" 200 None 2025-12-17 17:09:51,152 - lava_callback - DEBUG - Processing node 6942e388e43d2b76e9023a9c: baseline-arm64-kontron 2025-12-17 17:09:51,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:51,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:51,944 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-6942e388e43d2b76e9023a9c/log.txt.gz 2025-12-17 17:09:51,949 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:51,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:51,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:52,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:52,149 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-6942e388e43d2b76e9023a9c/lava_callback.json.gz 2025-12-17 17:09:52,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:52,180 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:52,181 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69405cbc575bc060ce1595f3/log.txt.gz 2025-12-17 17:09:52,181 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:52,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e388e43d2b76e9023a9c HTTP/1.1" 200 None 2025-12-17 17:09:52,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:52,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:52,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:09:52,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:52,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e388e43d2b76e9023a9c HTTP/1.1" 200 None 2025-12-17 17:09:52,231 - lava_callback - INFO - Completed processing callback for node 6942e388e43d2b76e9023a9c 2025-12-17 17:09:52,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:52,433 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69405cbc575bc060ce1595f3/lava_callback.json.gz 2025-12-17 17:09:52,550 - lava_callback - INFO - Received callback for node 6942e389e43d2b76e9023a9f INFO: 172.21.0.1:58164 - "POST /node/6942e389e43d2b76e9023a9f HTTP/1.0" 202 Accepted 2025-12-17 17:09:52,572 - lava_callback - INFO - Starting async job submit for node 6942e389e43d2b76e9023a9f DEBUG: Setup failed for 69405cbc575bc060ce1595f3. Transitting job node result: incomplete -> fail 2025-12-17 17:09:52,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:52,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:52,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e389e43d2b76e9023a9f HTTP/1.1" 200 None 2025-12-17 17:09:52,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69405cbc575bc060ce1595f3 HTTP/1.1" 200 None 2025-12-17 17:09:52,623 - lava_callback - DEBUG - Processing node 6942e389e43d2b76e9023a9f: baseline-arm64-kontron 2025-12-17 17:09:52,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:52,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693f507d8b4d8f471e43f50c HTTP/1.1" 200 None 2025-12-17 17:09:52,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:52,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69405cbc575bc060ce1595f3 HTTP/1.1" 200 None 2025-12-17 17:09:52,829 - lava_callback - INFO - Completed processing callback for node 69405cbc575bc060ce1595f3 2025-12-17 17:09:52,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:53,041 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:53,042 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-6942e389e43d2b76e9023a9f/log.txt.gz 2025-12-17 17:09:53,042 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:53,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:53,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:53,174 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-6942e389e43d2b76e9023a9f/lava_callback.json.gz 2025-12-17 17:09:53,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:53,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e389e43d2b76e9023a9f HTTP/1.1" 200 None 2025-12-17 17:09:53,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:53,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:09:53,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:53,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e389e43d2b76e9023a9f HTTP/1.1" 200 None 2025-12-17 17:09:53,241 - lava_callback - INFO - Completed processing callback for node 6942e389e43d2b76e9023a9f 2025-12-17 17:09:53,903 - lava_callback - INFO - Received callback for node 6942e389e43d2b76e9023aa6 INFO: 172.21.0.1:58176 - "POST /node/6942e389e43d2b76e9023aa6 HTTP/1.0" 202 Accepted 2025-12-17 17:09:53,909 - lava_callback - INFO - Starting async job submit for node 6942e389e43d2b76e9023aa6 2025-12-17 17:09:53,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:53,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e389e43d2b76e9023aa6 HTTP/1.1" 200 None 2025-12-17 17:09:53,943 - lava_callback - DEBUG - Processing node 6942e389e43d2b76e9023aa6: baseline-arm64-kontron 2025-12-17 17:09:54,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:54,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:54,366 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-6942e389e43d2b76e9023aa6/log.txt.gz 2025-12-17 17:09:54,367 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:54,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:54,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:54,505 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-6942e389e43d2b76e9023aa6/lava_callback.json.gz 2025-12-17 17:09:54,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:54,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e389e43d2b76e9023aa6 HTTP/1.1" 200 None 2025-12-17 17:09:54,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:54,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:09:54,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:54,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e389e43d2b76e9023aa6 HTTP/1.1" 200 None 2025-12-17 17:09:54,575 - lava_callback - INFO - Completed processing callback for node 6942e389e43d2b76e9023aa6 2025-12-17 17:09:55,299 - lava_callback - INFO - Received callback for node 6942e377e43d2b76e9023a15 INFO: 172.21.0.1:58178 - "POST /node/6942e377e43d2b76e9023a15 HTTP/1.0" 202 Accepted 2025-12-17 17:09:55,306 - lava_callback - INFO - Starting async job submit for node 6942e377e43d2b76e9023a15 2025-12-17 17:09:55,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:55,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e377e43d2b76e9023a15 HTTP/1.1" 200 None 2025-12-17 17:09:55,354 - lava_callback - DEBUG - Processing node 6942e377e43d2b76e9023a15: baseline-nfs-arm64 2025-12-17 17:09:55,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:55,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:55,939 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e377e43d2b76e9023a15/log.txt.gz 2025-12-17 17:09:55,939 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:55,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:56,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:56,092 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e377e43d2b76e9023a15/lava_callback.json.gz 2025-12-17 17:09:56,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:56,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e377e43d2b76e9023a15 HTTP/1.1" 200 None 2025-12-17 17:09:56,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:56,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:09:56,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:56,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e377e43d2b76e9023a15 HTTP/1.1" 200 None 2025-12-17 17:09:56,174 - lava_callback - INFO - Completed processing callback for node 6942e377e43d2b76e9023a15 2025-12-17 17:09:56,504 - lava_callback - INFO - Received callback for node 69404c69575bc060ce158df7 INFO: 172.21.0.1:58190 - "POST /node/69404c69575bc060ce158df7 HTTP/1.0" 202 Accepted 2025-12-17 17:09:56,512 - lava_callback - INFO - Starting async job submit for node 69404c69575bc060ce158df7 INFO: 172.21.0.1:58202 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:09:56,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:56,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404c69575bc060ce158df7 HTTP/1.1" 200 None 2025-12-17 17:09:56,587 - lava_callback - DEBUG - Processing node 69404c69575bc060ce158df7: baseline-nfs-arm64 2025-12-17 17:09:57,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:57,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:57,939 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69404c69575bc060ce158df7/log.txt.gz 2025-12-17 17:09:57,940 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:09:57,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:09:58,123 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:09:58,124 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69404c69575bc060ce158df7/lava_callback.json.gz DEBUG: Setup failed for 69404c69575bc060ce158df7. Transitting job node result: incomplete -> fail 2025-12-17 17:09:58,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:58,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404c69575bc060ce158df7 HTTP/1.1" 200 None 2025-12-17 17:09:58,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:58,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d651ec18b79b848b57427 HTTP/1.1" 200 None 2025-12-17 17:09:58,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:09:58,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69404c69575bc060ce158df7 HTTP/1.1" 200 None 2025-12-17 17:09:58,329 - lava_callback - INFO - Completed processing callback for node 69404c69575bc060ce158df7 2025-12-17 17:10:01,887 - lava_callback - INFO - Received callback for node 6942e376e43d2b76e9023a09 INFO: 172.21.0.1:56734 - "POST /node/6942e376e43d2b76e9023a09 HTTP/1.0" 202 Accepted 2025-12-17 17:10:01,897 - lava_callback - INFO - Starting async job submit for node 6942e376e43d2b76e9023a09 2025-12-17 17:10:01,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:01,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e376e43d2b76e9023a09 HTTP/1.1" 200 None 2025-12-17 17:10:01,941 - lava_callback - DEBUG - Processing node 6942e376e43d2b76e9023a09: baseline-nfs-arm64 2025-12-17 17:10:02,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:02,563 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:02,564 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e376e43d2b76e9023a09/log.txt.gz 2025-12-17 17:10:02,564 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:10:02,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:02,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:02,721 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e376e43d2b76e9023a09/lava_callback.json.gz 2025-12-17 17:10:02,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:02,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e376e43d2b76e9023a09 HTTP/1.1" 200 None 2025-12-17 17:10:02,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:02,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:10:02,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:02,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e376e43d2b76e9023a09 HTTP/1.1" 200 None 2025-12-17 17:10:02,780 - lava_callback - INFO - Completed processing callback for node 6942e376e43d2b76e9023a09 2025-12-17 17:10:04,753 - lava_callback - INFO - Received callback for node 69407115575bc060ce15a9d5 INFO: 172.21.0.1:56744 - "POST /node/69407115575bc060ce15a9d5 HTTP/1.0" 202 Accepted 2025-12-17 17:10:04,759 - lava_callback - INFO - Starting async job submit for node 69407115575bc060ce15a9d5 2025-12-17 17:10:04,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:04,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407115575bc060ce15a9d5 HTTP/1.1" 200 None 2025-12-17 17:10:04,797 - lava_callback - DEBUG - Processing node 69407115575bc060ce15a9d5: baseline-arm64 2025-12-17 17:10:05,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:05,282 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:05,282 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69407115575bc060ce15a9d5/log.txt.gz 2025-12-17 17:10:05,283 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:10:05,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:05,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:05,433 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69407115575bc060ce15a9d5/lava_callback.json.gz 2025-12-17 17:10:05,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:05,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407115575bc060ce15a9d5 HTTP/1.1" 200 None 2025-12-17 17:10:05,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:05,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693fc0ed0d8ce4018da9f5d2 HTTP/1.1" 200 None 2025-12-17 17:10:05,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:05,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69407115575bc060ce15a9d5 HTTP/1.1" 200 None 2025-12-17 17:10:05,497 - lava_callback - INFO - Completed processing callback for node 69407115575bc060ce15a9d5 INFO: 172.21.0.1:56748 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:10:07,985 - lava_callback - INFO - Received callback for node 6942e375e43d2b76e9023a00 INFO: 172.21.0.1:56754 - "POST /node/6942e375e43d2b76e9023a00 HTTP/1.0" 202 Accepted 2025-12-17 17:10:07,992 - lava_callback - INFO - Starting async job submit for node 6942e375e43d2b76e9023a00 2025-12-17 17:10:08,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:08,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e375e43d2b76e9023a00 HTTP/1.1" 200 None 2025-12-17 17:10:08,025 - lava_callback - DEBUG - Processing node 6942e375e43d2b76e9023a00: baseline-nfs-arm64 2025-12-17 17:10:08,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:08,465 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:08,465 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e375e43d2b76e9023a00/log.txt.gz 2025-12-17 17:10:08,466 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:10:08,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:08,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:08,605 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e375e43d2b76e9023a00/lava_callback.json.gz 2025-12-17 17:10:08,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:08,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e375e43d2b76e9023a00 HTTP/1.1" 200 None 2025-12-17 17:10:08,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:08,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:10:08,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:08,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e375e43d2b76e9023a00 HTTP/1.1" 200 None 2025-12-17 17:10:08,674 - lava_callback - INFO - Completed processing callback for node 6942e375e43d2b76e9023a00 2025-12-17 17:10:11,141 - lava_callback - INFO - Received callback for node 694040f1575bc060ce158a94 INFO: 172.21.0.1:48784 - "POST /node/694040f1575bc060ce158a94 HTTP/1.0" 202 Accepted 2025-12-17 17:10:11,147 - lava_callback - INFO - Starting async job submit for node 694040f1575bc060ce158a94 2025-12-17 17:10:11,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:11,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694040f1575bc060ce158a94 HTTP/1.1" 200 None 2025-12-17 17:10:11,156 - lava_callback - DEBUG - Processing node 694040f1575bc060ce158a94: ltp-syscalls 2025-12-17 17:10:11,156 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694040f1575bc060ce158a94 2025-12-17 17:10:11,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:11,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694040f1575bc060ce158a94 HTTP/1.1" 200 None 2025-12-17 17:10:11,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:11,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693aeba3fee8383e92b66614 HTTP/1.1" 200 None 2025-12-17 17:10:11,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:11,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694040f1575bc060ce158a94 HTTP/1.1" 200 None 2025-12-17 17:10:11,220 - lava_callback - INFO - Completed processing callback for node 694040f1575bc060ce158a94 INFO: 172.21.0.1:48792 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:10:11,415 - lava_callback - INFO - Received callback for node 6942e383e43d2b76e9023a7d INFO: 172.21.0.1:48802 - "POST /node/6942e383e43d2b76e9023a7d HTTP/1.0" 202 Accepted 2025-12-17 17:10:11,423 - lava_callback - INFO - Starting async job submit for node 6942e383e43d2b76e9023a7d 2025-12-17 17:10:11,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:11,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e383e43d2b76e9023a7d HTTP/1.1" 200 None 2025-12-17 17:10:11,476 - lava_callback - DEBUG - Processing node 6942e383e43d2b76e9023a7d: baseline-arm64-clabbe 2025-12-17 17:10:11,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:11,911 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:11,912 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-6942e383e43d2b76e9023a7d/log.txt.gz 2025-12-17 17:10:11,912 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:10:11,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:12,235 - lava_callback - INFO - Received callback for node 6942e376e43d2b76e9023a0c INFO: 172.21.0.1:48808 - "POST /node/6942e376e43d2b76e9023a0c HTTP/1.0" 202 Accepted 2025-12-17 17:10:12,243 - lava_callback - INFO - Starting async job submit for node 6942e376e43d2b76e9023a0c 2025-12-17 17:10:12,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:12,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e376e43d2b76e9023a0c HTTP/1.1" 200 None 2025-12-17 17:10:12,290 - lava_callback - DEBUG - Processing node 6942e376e43d2b76e9023a0c: baseline-nfs-arm64 2025-12-17 17:10:12,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:12,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:13,000 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e376e43d2b76e9023a0c/log.txt.gz 2025-12-17 17:10:13,000 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:10:13,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:13,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:13,168 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e376e43d2b76e9023a0c/lava_callback.json.gz 2025-12-17 17:10:13,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:13,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e376e43d2b76e9023a0c HTTP/1.1" 200 None 2025-12-17 17:10:13,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:13,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:10:13,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:13,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e376e43d2b76e9023a0c HTTP/1.1" 200 None 2025-12-17 17:10:13,230 - lava_callback - INFO - Completed processing callback for node 6942e376e43d2b76e9023a0c 2025-12-17 17:10:13,430 - lava_callback - INFO - Received callback for node 6942e387e43d2b76e9023a99 INFO: 172.21.0.1:48818 - "POST /node/6942e387e43d2b76e9023a99 HTTP/1.0" 202 Accepted 2025-12-17 17:10:13,445 - lava_callback - INFO - Starting async job submit for node 6942e387e43d2b76e9023a99 2025-12-17 17:10:13,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:13,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e387e43d2b76e9023a99 HTTP/1.1" 200 None 2025-12-17 17:10:13,516 - lava_callback - DEBUG - Processing node 6942e387e43d2b76e9023a99: baseline-arm64-clabbe 2025-12-17 17:10:13,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:13,896 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:13,897 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-6942e387e43d2b76e9023a99/log.txt.gz 2025-12-17 17:10:13,897 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:10:13,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:14,026 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:14,027 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-6942e387e43d2b76e9023a99/lava_callback.json.gz 2025-12-17 17:10:14,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:14,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e387e43d2b76e9023a99 HTTP/1.1" 200 None 2025-12-17 17:10:14,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:14,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:10:14,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:14,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e387e43d2b76e9023a99 HTTP/1.1" 200 None 2025-12-17 17:10:14,090 - lava_callback - INFO - Completed processing callback for node 6942e387e43d2b76e9023a99 INFO: 172.21.0.1:48822 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 17:10:16,507 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 INFO: 172.21.0.1:48828 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:10:16,508 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-6942e383e43d2b76e9023a7d/lava_callback.json.gz 2025-12-17 17:10:16,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:16,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e383e43d2b76e9023a7d HTTP/1.1" 200 None 2025-12-17 17:10:16,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:16,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:10:16,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:16,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e383e43d2b76e9023a7d HTTP/1.1" 200 None 2025-12-17 17:10:16,568 - lava_callback - INFO - Completed processing callback for node 6942e383e43d2b76e9023a7d 2025-12-17 17:10:17,014 - lava_callback - INFO - Received callback for node 6942e372e43d2b76e90239e5 INFO: 172.21.0.1:48832 - "POST /node/6942e372e43d2b76e90239e5 HTTP/1.0" 202 Accepted 2025-12-17 17:10:17,021 - lava_callback - INFO - Starting async job submit for node 6942e372e43d2b76e90239e5 2025-12-17 17:10:17,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:17,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e372e43d2b76e90239e5 HTTP/1.1" 200 None 2025-12-17 17:10:17,052 - lava_callback - DEBUG - Processing node 6942e372e43d2b76e90239e5: baseline-arm64 2025-12-17 17:10:17,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:17,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:17,601 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e372e43d2b76e90239e5/log.txt.gz 2025-12-17 17:10:17,602 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:10:17,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:17,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:17,753 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e372e43d2b76e90239e5/lava_callback.json.gz 2025-12-17 17:10:17,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:17,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e372e43d2b76e90239e5 HTTP/1.1" 200 None 2025-12-17 17:10:17,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:17,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:10:17,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:17,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e372e43d2b76e90239e5 HTTP/1.1" 200 None 2025-12-17 17:10:17,813 - lava_callback - INFO - Completed processing callback for node 6942e372e43d2b76e90239e5 INFO: 172.21.0.1:36276 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:10:31,229 - lava_callback - INFO - Received callback for node 6942e372e43d2b76e90239e8 INFO: 172.21.0.1:34270 - "POST /node/6942e372e43d2b76e90239e8 HTTP/1.0" 202 Accepted 2025-12-17 17:10:31,238 - lava_callback - INFO - Starting async job submit for node 6942e372e43d2b76e90239e8 2025-12-17 17:10:31,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:31,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e372e43d2b76e90239e8 HTTP/1.1" 200 None 2025-12-17 17:10:31,275 - lava_callback - DEBUG - Processing node 6942e372e43d2b76e90239e8: baseline-arm64 2025-12-17 17:10:31,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:31,693 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:31,694 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e372e43d2b76e90239e8/log.txt.gz 2025-12-17 17:10:31,694 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:10:31,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:31,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:31,859 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e372e43d2b76e90239e8/lava_callback.json.gz 2025-12-17 17:10:31,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:31,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e372e43d2b76e90239e8 HTTP/1.1" 200 None 2025-12-17 17:10:31,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:31,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:10:31,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:31,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e372e43d2b76e90239e8 HTTP/1.1" 200 None 2025-12-17 17:10:31,933 - lava_callback - INFO - Completed processing callback for node 6942e372e43d2b76e90239e8 INFO: 172.21.0.1:34284 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:10:38,453 - lava_callback - INFO - Received callback for node 6942e36fe43d2b76e90239ce INFO: 172.21.0.1:34290 - "POST /node/6942e36fe43d2b76e90239ce HTTP/1.0" 202 Accepted 2025-12-17 17:10:38,460 - lava_callback - INFO - Starting async job submit for node 6942e36fe43d2b76e90239ce 2025-12-17 17:10:38,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:38,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e36fe43d2b76e90239ce HTTP/1.1" 200 None 2025-12-17 17:10:38,490 - lava_callback - DEBUG - Processing node 6942e36fe43d2b76e90239ce: baseline-arm64 2025-12-17 17:10:38,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:38,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:38,860 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e36fe43d2b76e90239ce/log.txt.gz 2025-12-17 17:10:38,860 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:10:38,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:38,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:38,993 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e36fe43d2b76e90239ce/lava_callback.json.gz 2025-12-17 17:10:39,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:39,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e36fe43d2b76e90239ce HTTP/1.1" 200 None 2025-12-17 17:10:39,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:39,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:10:39,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:39,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e36fe43d2b76e90239ce HTTP/1.1" 200 None 2025-12-17 17:10:39,058 - lava_callback - INFO - Completed processing callback for node 6942e36fe43d2b76e90239ce INFO: 172.21.0.1:45836 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:10:51,785 - lava_callback - INFO - Received callback for node 6942e377e43d2b76e9023a12 INFO: 172.21.0.1:60258 - "POST /node/6942e377e43d2b76e9023a12 HTTP/1.0" 202 Accepted 2025-12-17 17:10:51,792 - lava_callback - INFO - Starting async job submit for node 6942e377e43d2b76e9023a12 2025-12-17 17:10:51,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:51,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e377e43d2b76e9023a12 HTTP/1.1" 200 None 2025-12-17 17:10:51,824 - lava_callback - DEBUG - Processing node 6942e377e43d2b76e9023a12: baseline-nfs-arm64 2025-12-17 17:10:52,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:52,395 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:52,395 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e377e43d2b76e9023a12/log.txt.gz 2025-12-17 17:10:52,396 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:10:52,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:52,585 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:52,586 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e377e43d2b76e9023a12/lava_callback.json.gz 2025-12-17 17:10:52,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:52,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e377e43d2b76e9023a12 HTTP/1.1" 200 None 2025-12-17 17:10:52,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:52,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:10:52,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:52,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e377e43d2b76e9023a12 HTTP/1.1" 200 None 2025-12-17 17:10:52,650 - lava_callback - INFO - Completed processing callback for node 6942e377e43d2b76e9023a12 2025-12-17 17:10:54,405 - lava_callback - INFO - Received callback for node 6942e375e43d2b76e9023a03 INFO: 172.21.0.1:60260 - "POST /node/6942e375e43d2b76e9023a03 HTTP/1.0" 202 Accepted 2025-12-17 17:10:54,415 - lava_callback - INFO - Starting async job submit for node 6942e375e43d2b76e9023a03 2025-12-17 17:10:54,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:54,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e375e43d2b76e9023a03 HTTP/1.1" 200 None 2025-12-17 17:10:54,448 - lava_callback - DEBUG - Processing node 6942e375e43d2b76e9023a03: baseline-nfs-arm64 2025-12-17 17:10:54,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:54,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:54,933 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e375e43d2b76e9023a03/log.txt.gz 2025-12-17 17:10:54,934 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:10:54,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:55,078 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:55,079 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e375e43d2b76e9023a03/lava_callback.json.gz 2025-12-17 17:10:55,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:55,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e375e43d2b76e9023a03 HTTP/1.1" 200 None 2025-12-17 17:10:55,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:55,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:10:55,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:55,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e375e43d2b76e9023a03 HTTP/1.1" 200 None 2025-12-17 17:10:55,144 - lava_callback - INFO - Completed processing callback for node 6942e375e43d2b76e9023a03 INFO: 172.21.0.1:60274 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:10:57,496 - lava_callback - INFO - Received callback for node 6942e378e43d2b76e9023a1b INFO: 172.21.0.1:60276 - "POST /node/6942e378e43d2b76e9023a1b HTTP/1.0" 202 Accepted 2025-12-17 17:10:57,507 - lava_callback - INFO - Starting async job submit for node 6942e378e43d2b76e9023a1b 2025-12-17 17:10:57,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:57,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e378e43d2b76e9023a1b HTTP/1.1" 200 None 2025-12-17 17:10:57,557 - lava_callback - DEBUG - Processing node 6942e378e43d2b76e9023a1b: baseline-nfs-arm64 2025-12-17 17:10:57,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:58,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:58,099 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e378e43d2b76e9023a1b/log.txt.gz 2025-12-17 17:10:58,100 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:10:58,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:58,254 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:58,255 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e378e43d2b76e9023a1b/lava_callback.json.gz 2025-12-17 17:10:58,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:58,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e378e43d2b76e9023a1b HTTP/1.1" 200 None 2025-12-17 17:10:58,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:58,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:10:58,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:58,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e378e43d2b76e9023a1b HTTP/1.1" 200 None 2025-12-17 17:10:58,322 - lava_callback - INFO - Completed processing callback for node 6942e378e43d2b76e9023a1b 2025-12-17 17:10:58,774 - lava_callback - INFO - Received callback for node 6942e371e43d2b76e90239de INFO: 172.21.0.1:60278 - "POST /node/6942e371e43d2b76e90239de HTTP/1.0" 202 Accepted 2025-12-17 17:10:58,783 - lava_callback - INFO - Starting async job submit for node 6942e371e43d2b76e90239de 2025-12-17 17:10:58,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:58,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e371e43d2b76e90239de HTTP/1.1" 200 None 2025-12-17 17:10:58,815 - lava_callback - DEBUG - Processing node 6942e371e43d2b76e90239de: baseline-arm64 2025-12-17 17:10:59,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:59,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:59,365 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e371e43d2b76e90239de/log.txt.gz 2025-12-17 17:10:59,366 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:10:59,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:10:59,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:10:59,530 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e371e43d2b76e90239de/lava_callback.json.gz 2025-12-17 17:10:59,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:59,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e371e43d2b76e90239de HTTP/1.1" 200 None 2025-12-17 17:10:59,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:59,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:10:59,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:10:59,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e371e43d2b76e90239de HTTP/1.1" 200 None 2025-12-17 17:10:59,641 - lava_callback - INFO - Completed processing callback for node 6942e371e43d2b76e90239de INFO: 172.21.0.1:40800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53814 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:11:11,460 - lava_callback - INFO - Received callback for node 6942e394e43d2b76e9023ac9 INFO: 172.21.0.1:53826 - "POST /node/6942e394e43d2b76e9023ac9 HTTP/1.0" 202 Accepted 2025-12-17 17:11:11,466 - lava_callback - INFO - Starting async job submit for node 6942e394e43d2b76e9023ac9 2025-12-17 17:11:11,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:11:11,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e394e43d2b76e9023ac9 HTTP/1.1" 200 None 2025-12-17 17:11:11,520 - lava_callback - DEBUG - Processing node 6942e394e43d2b76e9023ac9: baseline-arm64-kontron 2025-12-17 17:11:11,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:11:11,917 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:11:11,918 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-6942e394e43d2b76e9023ac9/log.txt.gz 2025-12-17 17:11:11,918 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:11:11,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:11:12,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:11:12,066 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-6942e394e43d2b76e9023ac9/lava_callback.json.gz 2025-12-17 17:11:12,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:11:12,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e394e43d2b76e9023ac9 HTTP/1.1" 200 None 2025-12-17 17:11:12,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:11:12,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:11:12,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:11:12,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e394e43d2b76e9023ac9 HTTP/1.1" 200 None 2025-12-17 17:11:12,151 - lava_callback - INFO - Completed processing callback for node 6942e394e43d2b76e9023ac9 INFO: 172.21.0.1:53840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51276 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:12:02,308 - lava_callback - INFO - Received callback for node 6942e386e43d2b76e9023a8d INFO: 172.21.0.1:41898 - "POST /node/6942e386e43d2b76e9023a8d HTTP/1.0" 202 Accepted 2025-12-17 17:12:02,316 - lava_callback - INFO - Starting async job submit for node 6942e386e43d2b76e9023a8d 2025-12-17 17:12:02,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:12:02,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e386e43d2b76e9023a8d HTTP/1.1" 200 None 2025-12-17 17:12:02,350 - lava_callback - DEBUG - Processing node 6942e386e43d2b76e9023a8d: baseline-arm64-clabbe 2025-12-17 17:12:02,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:12:02,962 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:12:02,963 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-6942e386e43d2b76e9023a8d/log.txt.gz 2025-12-17 17:12:02,963 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:12:02,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:12:03,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:12:03,128 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-6942e386e43d2b76e9023a8d/lava_callback.json.gz 2025-12-17 17:12:03,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:12:03,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e386e43d2b76e9023a8d HTTP/1.1" 200 None 2025-12-17 17:12:03,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:12:03,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:12:03,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:12:03,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e386e43d2b76e9023a8d HTTP/1.1" 200 None 2025-12-17 17:12:03,185 - lava_callback - INFO - Completed processing callback for node 6942e386e43d2b76e9023a8d INFO: 172.21.0.1:41908 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:12:07,679 - lava_callback - INFO - Received callback for node 6942e38ce43d2b76e9023ab9 INFO: 172.21.0.1:41912 - "POST /node/6942e38ce43d2b76e9023ab9 HTTP/1.0" 202 Accepted 2025-12-17 17:12:07,686 - lava_callback - INFO - Starting async job submit for node 6942e38ce43d2b76e9023ab9 2025-12-17 17:12:07,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:12:07,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e38ce43d2b76e9023ab9 HTTP/1.1" 200 None 2025-12-17 17:12:07,738 - lava_callback - DEBUG - Processing node 6942e38ce43d2b76e9023ab9: baseline-arm64-foundriesio 2025-12-17 17:12:08,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:12:08,641 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:12:08,642 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-6942e38ce43d2b76e9023ab9/log.txt.gz 2025-12-17 17:12:08,642 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:12:08,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:12:08,835 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:12:08,836 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-6942e38ce43d2b76e9023ab9/lava_callback.json.gz DEBUG: 6942e38ce43d2b76e9023ab9 Transitting job node result: pass -> incomplete 2025-12-17 17:12:08,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:12:08,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e38ce43d2b76e9023ab9 HTTP/1.1" 200 None 2025-12-17 17:12:08,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:12:08,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:12:08,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:12:08,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e38ce43d2b76e9023ab9 HTTP/1.1" 200 None 2025-12-17 17:12:08,913 - lava_callback - INFO - Completed processing callback for node 6942e38ce43d2b76e9023ab9 INFO: 172.21.0.1:57454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55544 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:12:39,427 - lava_callback - INFO - Received callback for node 6942e378e43d2b76e9023a18 INFO: 172.21.0.1:55552 - "POST /node/6942e378e43d2b76e9023a18 HTTP/1.0" 202 Accepted 2025-12-17 17:12:39,434 - lava_callback - INFO - Starting async job submit for node 6942e378e43d2b76e9023a18 2025-12-17 17:12:39,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:12:39,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e378e43d2b76e9023a18 HTTP/1.1" 200 None 2025-12-17 17:12:39,487 - lava_callback - DEBUG - Processing node 6942e378e43d2b76e9023a18: baseline-nfs-arm64 2025-12-17 17:12:39,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:12:40,079 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:12:40,080 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e378e43d2b76e9023a18/log.txt.gz 2025-12-17 17:12:40,080 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:12:40,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:12:40,264 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:12:40,265 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e378e43d2b76e9023a18/lava_callback.json.gz 2025-12-17 17:12:40,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:12:40,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e378e43d2b76e9023a18 HTTP/1.1" 200 None 2025-12-17 17:12:40,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:12:40,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:12:40,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:12:40,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e378e43d2b76e9023a18 HTTP/1.1" 200 None 2025-12-17 17:12:40,382 - lava_callback - INFO - Completed processing callback for node 6942e378e43d2b76e9023a18 INFO: 172.21.0.1:59560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50296 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:13:00,669 - lava_callback - INFO - Received callback for node 6940680d575bc060ce159eae INFO: 172.21.0.1:46798 - "POST /node/6940680d575bc060ce159eae HTTP/1.0" 202 Accepted 2025-12-17 17:13:00,677 - lava_callback - INFO - Starting async job submit for node 6940680d575bc060ce159eae 2025-12-17 17:13:00,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:13:00,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940680d575bc060ce159eae HTTP/1.1" 200 None 2025-12-17 17:13:00,742 - lava_callback - DEBUG - Processing node 6940680d575bc060ce159eae: baseline-nfs-arm64 2025-12-17 17:13:01,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:13:02,024 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:13:02,024 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6940680d575bc060ce159eae/log.txt.gz 2025-12-17 17:13:02,025 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:13:02,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:13:02,202 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:13:02,202 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6940680d575bc060ce159eae/lava_callback.json.gz DEBUG: Setup failed for 6940680d575bc060ce159eae. Transitting job node result: incomplete -> fail 2025-12-17 17:13:02,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:13:02,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940680d575bc060ce159eae HTTP/1.1" 200 None 2025-12-17 17:13:02,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:13:02,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693f54220d8ce4018da8984f HTTP/1.1" 200 None 2025-12-17 17:13:02,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:13:02,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940680d575bc060ce159eae HTTP/1.1" 200 None 2025-12-17 17:13:02,347 - lava_callback - INFO - Completed processing callback for node 6940680d575bc060ce159eae INFO: 172.21.0.1:46804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40580 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:13:14,401 - lava_callback - INFO - Received callback for node 6942e423e43d2b76e9023b6f INFO: 172.21.0.1:40594 - "POST /node/6942e423e43d2b76e9023b6f HTTP/1.0" 202 Accepted 2025-12-17 17:13:14,407 - lava_callback - INFO - Starting async job submit for node 6942e423e43d2b76e9023b6f 2025-12-17 17:13:14,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:13:14,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e423e43d2b76e9023b6f HTTP/1.1" 200 None 2025-12-17 17:13:14,438 - lava_callback - DEBUG - Processing node 6942e423e43d2b76e9023b6f: baseline-arm64 2025-12-17 17:13:14,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:13:15,032 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:13:15,033 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e423e43d2b76e9023b6f/log.txt.gz 2025-12-17 17:13:15,033 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:13:15,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:40608 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 17:13:15,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:13:15,237 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e423e43d2b76e9023b6f/lava_callback.json.gz 2025-12-17 17:13:15,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:13:15,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e423e43d2b76e9023b6f HTTP/1.1" 200 None 2025-12-17 17:13:15,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:13:15,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:13:15,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:13:15,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e423e43d2b76e9023b6f HTTP/1.1" 200 None 2025-12-17 17:13:15,356 - lava_callback - INFO - Completed processing callback for node 6942e423e43d2b76e9023b6f INFO: 172.21.0.1:40616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40530 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:13:29,165 - lava_callback - INFO - Received callback for node 694070d7575bc060ce15a990 INFO: 172.21.0.1:40546 - "POST /node/694070d7575bc060ce15a990 HTTP/1.0" 202 Accepted 2025-12-17 17:13:29,174 - lava_callback - INFO - Starting async job submit for node 694070d7575bc060ce15a990 2025-12-17 17:13:29,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:13:29,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d7575bc060ce15a990 HTTP/1.1" 200 None 2025-12-17 17:13:29,245 - lava_callback - DEBUG - Processing node 694070d7575bc060ce15a990: baseline-nfs-arm64 2025-12-17 17:13:30,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:13:30,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:13:30,559 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694070d7575bc060ce15a990/log.txt.gz 2025-12-17 17:13:30,559 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:13:30,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:13:30,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:13:30,753 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694070d7575bc060ce15a990/lava_callback.json.gz DEBUG: Setup failed for 694070d7575bc060ce15a990. Transitting job node result: incomplete -> fail 2025-12-17 17:13:30,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:13:30,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d7575bc060ce15a990 HTTP/1.1" 200 None 2025-12-17 17:13:30,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:13:30,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693bcc6941fb4c950977a448 HTTP/1.1" 200 None 2025-12-17 17:13:30,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:13:30,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694070d7575bc060ce15a990 HTTP/1.1" 200 None 2025-12-17 17:13:30,927 - lava_callback - INFO - Completed processing callback for node 694070d7575bc060ce15a990 INFO: 172.21.0.1:45690 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:13:44,822 - lava_callback - INFO - Received callback for node 694042c9575bc060ce158b24 INFO: 172.21.0.1:35926 - "POST /node/694042c9575bc060ce158b24 HTTP/1.0" 202 Accepted 2025-12-17 17:13:44,828 - lava_callback - INFO - Starting async job submit for node 694042c9575bc060ce158b24 2025-12-17 17:13:44,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:13:44,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042c9575bc060ce158b24 HTTP/1.1" 200 None 2025-12-17 17:13:44,869 - lava_callback - DEBUG - Processing node 694042c9575bc060ce158b24: baseline-x86 2025-12-17 17:13:45,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:13:45,327 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:13:45,328 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-694042c9575bc060ce158b24/log.txt.gz 2025-12-17 17:13:45,328 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:13:45,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:13:45,509 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:13:45,510 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-694042c9575bc060ce158b24/lava_callback.json.gz 2025-12-17 17:13:45,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:13:45,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042c9575bc060ce158b24 HTTP/1.1" 200 None 2025-12-17 17:13:45,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:13:45,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:13:45,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:13:45,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694042c9575bc060ce158b24 HTTP/1.1" 200 None 2025-12-17 17:13:45,568 - lava_callback - INFO - Completed processing callback for node 694042c9575bc060ce158b24 INFO: 172.21.0.1:35936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41070 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:14:13,153 - lava_callback - INFO - Received callback for node 6942e4abe43d2b76e9023b95 INFO: 172.21.0.1:41082 - "POST /node/6942e4abe43d2b76e9023b95 HTTP/1.0" 202 Accepted 2025-12-17 17:14:13,159 - lava_callback - INFO - Starting async job submit for node 6942e4abe43d2b76e9023b95 2025-12-17 17:14:13,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:14:13,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e4abe43d2b76e9023b95 HTTP/1.1" 200 None 2025-12-17 17:14:13,189 - lava_callback - DEBUG - Processing node 6942e4abe43d2b76e9023b95: baseline-arm64 2025-12-17 17:14:13,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:14:13,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:14:13,710 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e4abe43d2b76e9023b95/log.txt.gz 2025-12-17 17:14:13,710 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:14:13,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:14:13,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:14:13,919 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-6942e4abe43d2b76e9023b95/lava_callback.json.gz 2025-12-17 17:14:13,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:14:13,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e4abe43d2b76e9023b95 HTTP/1.1" 200 None 2025-12-17 17:14:13,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:14:13,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:14:13,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:14:13,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e4abe43d2b76e9023b95 HTTP/1.1" 200 None 2025-12-17 17:14:13,980 - lava_callback - INFO - Completed processing callback for node 6942e4abe43d2b76e9023b95 INFO: 172.21.0.1:41096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56946 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:14:50,207 - lava_callback - INFO - Received callback for node 694041f4575bc060ce158ad5 INFO: 172.21.0.1:38712 - "POST /node/694041f4575bc060ce158ad5 HTTP/1.0" 202 Accepted 2025-12-17 17:14:50,213 - lava_callback - INFO - Starting async job submit for node 694041f4575bc060ce158ad5 2025-12-17 17:14:50,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:14:50,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694041f4575bc060ce158ad5 HTTP/1.1" 200 None 2025-12-17 17:14:50,225 - lava_callback - DEBUG - Processing node 694041f4575bc060ce158ad5: baseline-arm-broonie 2025-12-17 17:14:50,225 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694041f4575bc060ce158ad5 2025-12-17 17:14:50,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:14:50,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694041f4575bc060ce158ad5 HTTP/1.1" 200 None 2025-12-17 17:14:50,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:14:50,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af125fee8383e92b69ab4 HTTP/1.1" 200 None 2025-12-17 17:14:50,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:14:50,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694041f4575bc060ce158ad5 HTTP/1.1" 200 None 2025-12-17 17:14:50,263 - lava_callback - INFO - Completed processing callback for node 694041f4575bc060ce158ad5 INFO: 172.21.0.1:38726 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:14:57,214 - lava_callback - INFO - Received callback for node 694041ff575bc060ce158adc INFO: 172.21.0.1:38728 - "POST /node/694041ff575bc060ce158adc HTTP/1.0" 202 Accepted 2025-12-17 17:14:57,223 - lava_callback - INFO - Starting async job submit for node 694041ff575bc060ce158adc 2025-12-17 17:14:57,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:14:57,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694041ff575bc060ce158adc HTTP/1.1" 200 None 2025-12-17 17:14:57,236 - lava_callback - DEBUG - Processing node 694041ff575bc060ce158adc: baseline-arm 2025-12-17 17:14:57,236 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694041ff575bc060ce158adc 2025-12-17 17:14:57,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:14:57,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694041ff575bc060ce158adc HTTP/1.1" 200 None 2025-12-17 17:14:57,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:14:57,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af17afee8383e92b69dc1 HTTP/1.1" 200 None 2025-12-17 17:14:57,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:14:57,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694041ff575bc060ce158adc HTTP/1.1" 200 None 2025-12-17 17:14:57,293 - lava_callback - INFO - Completed processing callback for node 694041ff575bc060ce158adc 2025-12-17 17:15:04,516 - lava_callback - INFO - Received callback for node 694042cc575bc060ce158b2d INFO: 172.21.0.1:40816 - "POST /node/694042cc575bc060ce158b2d HTTP/1.0" 202 Accepted 2025-12-17 17:15:04,522 - lava_callback - INFO - Starting async job submit for node 694042cc575bc060ce158b2d 2025-12-17 17:15:04,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:15:04,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042cc575bc060ce158b2d HTTP/1.1" 200 None 2025-12-17 17:15:04,553 - lava_callback - DEBUG - Processing node 694042cc575bc060ce158b2d: baseline-x86 2025-12-17 17:15:04,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:15:05,154 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:15:05,154 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-694042cc575bc060ce158b2d/log.txt.gz 2025-12-17 17:15:05,155 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:15:05,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:15:05,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:15:05,548 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-694042cc575bc060ce158b2d/lava_callback.json.gz 2025-12-17 17:15:05,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:15:05,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042cc575bc060ce158b2d HTTP/1.1" 200 None 2025-12-17 17:15:05,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:15:05,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:15:05,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:15:05,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694042cc575bc060ce158b2d HTTP/1.1" 200 None 2025-12-17 17:15:05,649 - lava_callback - INFO - Completed processing callback for node 694042cc575bc060ce158b2d INFO: 172.21.0.1:40828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35954 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:15:57,391 - lava_callback - INFO - Received callback for node 69404248575bc060ce158b01 INFO: 172.21.0.1:35970 - "POST /node/69404248575bc060ce158b01 HTTP/1.0" 202 Accepted 2025-12-17 17:15:57,402 - lava_callback - INFO - Starting async job submit for node 69404248575bc060ce158b01 2025-12-17 17:15:57,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:15:57,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404248575bc060ce158b01 HTTP/1.1" 200 None 2025-12-17 17:15:57,418 - lava_callback - DEBUG - Processing node 69404248575bc060ce158b01: baseline-arm 2025-12-17 17:15:57,419 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69404248575bc060ce158b01 2025-12-17 17:15:57,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:15:57,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404248575bc060ce158b01 HTTP/1.1" 200 None 2025-12-17 17:15:57,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:15:57,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693aeba3fee8383e92b66614 HTTP/1.1" 200 None 2025-12-17 17:15:57,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:15:57,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69404248575bc060ce158b01 HTTP/1.1" 200 None 2025-12-17 17:15:57,475 - lava_callback - INFO - Completed processing callback for node 69404248575bc060ce158b01 2025-12-17 17:16:00,299 - lava_callback - INFO - Received callback for node 6940c4ff1e3c24e95681d605 INFO: 172.21.0.1:59122 - "POST /node/6940c4ff1e3c24e95681d605 HTTP/1.0" 202 Accepted 2025-12-17 17:16:00,305 - lava_callback - INFO - Starting async job submit for node 6940c4ff1e3c24e95681d605 2025-12-17 17:16:00,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:00,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c4ff1e3c24e95681d605 HTTP/1.1" 200 None 2025-12-17 17:16:00,366 - lava_callback - DEBUG - Processing node 6940c4ff1e3c24e95681d605: baseline-x86 2025-12-17 17:16:01,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:16:01,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:16:01,213 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-6940c4ff1e3c24e95681d605/log.txt.gz 2025-12-17 17:16:01,214 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:16:01,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:16:01,582 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:16:01,584 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-6940c4ff1e3c24e95681d605/lava_callback.json.gz DEBUG: Setup failed for 6940c4ff1e3c24e95681d605. Transitting job node result: incomplete -> fail 2025-12-17 17:16:01,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:01,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c4ff1e3c24e95681d605 HTTP/1.1" 200 None 2025-12-17 17:16:01,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:01,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d5fad9ee307f5b1e66 HTTP/1.1" 200 None 2025-12-17 17:16:01,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:01,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940c4ff1e3c24e95681d605 HTTP/1.1" 200 None 2025-12-17 17:16:01,719 - lava_callback - INFO - Completed processing callback for node 6940c4ff1e3c24e95681d605 INFO: 172.21.0.1:59130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43962 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:16:23,994 - lava_callback - INFO - Received callback for node 6942e488e43d2b76e9023b89 INFO: 172.21.0.1:36452 - "POST /node/6942e488e43d2b76e9023b89 HTTP/1.0" 202 Accepted 2025-12-17 17:16:24,001 - lava_callback - INFO - Starting async job submit for node 6942e488e43d2b76e9023b89 2025-12-17 17:16:24,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:24,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e488e43d2b76e9023b89 HTTP/1.1" 200 None 2025-12-17 17:16:24,035 - lava_callback - DEBUG - Processing node 6942e488e43d2b76e9023b89: baseline-nfs-arm64 2025-12-17 17:16:24,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:16:24,597 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:16:24,598 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e488e43d2b76e9023b89/log.txt.gz 2025-12-17 17:16:24,599 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:16:24,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:16:24,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:16:24,744 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e488e43d2b76e9023b89/lava_callback.json.gz 2025-12-17 17:16:24,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:24,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e488e43d2b76e9023b89 HTTP/1.1" 200 None 2025-12-17 17:16:24,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:24,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:16:24,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:24,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e488e43d2b76e9023b89 HTTP/1.1" 200 None 2025-12-17 17:16:24,862 - lava_callback - INFO - Completed processing callback for node 6942e488e43d2b76e9023b89 2025-12-17 17:16:25,813 - lava_callback - INFO - Received callback for node 694042d0575bc060ce158b3b INFO: 172.21.0.1:36466 - "POST /node/694042d0575bc060ce158b3b HTTP/1.0" 202 Accepted 2025-12-17 17:16:25,821 - lava_callback - INFO - Starting async job submit for node 694042d0575bc060ce158b3b 2025-12-17 17:16:25,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:25,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042d0575bc060ce158b3b HTTP/1.1" 200 None 2025-12-17 17:16:25,856 - lava_callback - DEBUG - Processing node 694042d0575bc060ce158b3b: baseline-x86 2025-12-17 17:16:26,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:16:26,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:16:26,290 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-694042d0575bc060ce158b3b/log.txt.gz 2025-12-17 17:16:26,290 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:16:26,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:16:26,420 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:16:26,421 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-694042d0575bc060ce158b3b/lava_callback.json.gz 2025-12-17 17:16:26,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:26,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042d0575bc060ce158b3b HTTP/1.1" 200 None 2025-12-17 17:16:26,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:26,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:16:26,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:26,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694042d0575bc060ce158b3b HTTP/1.1" 200 None 2025-12-17 17:16:26,477 - lava_callback - INFO - Completed processing callback for node 694042d0575bc060ce158b3b INFO: 172.21.0.1:36468 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:16:30,203 - lava_callback - INFO - Received callback for node 69404259575bc060ce158b06 INFO: 172.21.0.1:42882 - "POST /node/69404259575bc060ce158b06 HTTP/1.0" 202 Accepted 2025-12-17 17:16:30,210 - lava_callback - INFO - Starting async job submit for node 69404259575bc060ce158b06 2025-12-17 17:16:30,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:30,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404259575bc060ce158b06 HTTP/1.1" 200 None 2025-12-17 17:16:30,223 - lava_callback - DEBUG - Processing node 69404259575bc060ce158b06: ltp-syscalls 2025-12-17 17:16:30,223 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69404259575bc060ce158b06 2025-12-17 17:16:30,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:30,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404259575bc060ce158b06 HTTP/1.1" 200 None 2025-12-17 17:16:30,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:30,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af125fee8383e92b69ab4 HTTP/1.1" 200 None 2025-12-17 17:16:30,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:30,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69404259575bc060ce158b06 HTTP/1.1" 200 None 2025-12-17 17:16:30,265 - lava_callback - INFO - Completed processing callback for node 69404259575bc060ce158b06 INFO: 172.21.0.1:42892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34876 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:16:57,376 - lava_callback - INFO - Received callback for node 6942e469e43d2b76e9023b81 INFO: 172.21.0.1:34880 - "POST /node/6942e469e43d2b76e9023b81 HTTP/1.0" 202 Accepted 2025-12-17 17:16:57,384 - lava_callback - INFO - Starting async job submit for node 6942e469e43d2b76e9023b81 2025-12-17 17:16:57,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:57,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e469e43d2b76e9023b81 HTTP/1.1" 200 None 2025-12-17 17:16:57,449 - lava_callback - DEBUG - Processing node 6942e469e43d2b76e9023b81: baseline-arm64-foundriesio 2025-12-17 17:16:57,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:16:58,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:16:58,005 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-6942e469e43d2b76e9023b81/log.txt.gz 2025-12-17 17:16:58,005 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:16:58,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:16:58,152 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:16:58,153 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-6942e469e43d2b76e9023b81/lava_callback.json.gz 2025-12-17 17:16:58,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:58,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e469e43d2b76e9023b81 HTTP/1.1" 200 None 2025-12-17 17:16:58,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:58,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:16:58,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:16:58,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e469e43d2b76e9023b81 HTTP/1.1" 200 None 2025-12-17 17:16:58,221 - lava_callback - INFO - Completed processing callback for node 6942e469e43d2b76e9023b81 INFO: 172.21.0.1:50996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33954 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:17:50,146 - lava_callback - INFO - Received callback for node 694042d1575bc060ce158b42 INFO: 172.21.0.1:52208 - "POST /node/694042d1575bc060ce158b42 HTTP/1.0" 202 Accepted 2025-12-17 17:17:50,151 - lava_callback - INFO - Starting async job submit for node 694042d1575bc060ce158b42 2025-12-17 17:17:50,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:17:50,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042d1575bc060ce158b42 HTTP/1.1" 200 None 2025-12-17 17:17:50,203 - lava_callback - DEBUG - Processing node 694042d1575bc060ce158b42: baseline-x86 2025-12-17 17:17:50,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:17:50,635 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:17:50,636 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-694042d1575bc060ce158b42/log.txt.gz 2025-12-17 17:17:50,636 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:17:50,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:17:50,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:17:50,796 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-694042d1575bc060ce158b42/lava_callback.json.gz 2025-12-17 17:17:50,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:17:50,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042d1575bc060ce158b42 HTTP/1.1" 200 None 2025-12-17 17:17:50,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:17:50,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403212575bc060ce156c9f HTTP/1.1" 200 None 2025-12-17 17:17:50,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:17:50,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694042d1575bc060ce158b42 HTTP/1.1" 200 None 2025-12-17 17:17:50,868 - lava_callback - INFO - Completed processing callback for node 694042d1575bc060ce158b42 INFO: 172.21.0.1:52220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39656 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:18:17,550 - lava_callback - INFO - Received callback for node 694042d5575bc060ce158b4e INFO: 172.21.0.1:39662 - "POST /node/694042d5575bc060ce158b4e HTTP/1.0" 202 Accepted 2025-12-17 17:18:17,555 - lava_callback - INFO - Starting async job submit for node 694042d5575bc060ce158b4e 2025-12-17 17:18:17,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:17,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042d5575bc060ce158b4e HTTP/1.1" 200 None 2025-12-17 17:18:17,565 - lava_callback - DEBUG - Processing node 694042d5575bc060ce158b4e: baseline-x86 2025-12-17 17:18:17,565 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694042d5575bc060ce158b4e 2025-12-17 17:18:17,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:17,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042d5575bc060ce158b4e HTTP/1.1" 200 None 2025-12-17 17:18:17,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:17,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:18:17,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:17,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694042d5575bc060ce158b4e HTTP/1.1" 200 None 2025-12-17 17:18:17,630 - lava_callback - INFO - Completed processing callback for node 694042d5575bc060ce158b4e INFO: 172.21.0.1:44008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60662 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:18:37,463 - lava_callback - INFO - Received callback for node 694042db575bc060ce158b54 INFO: 172.21.0.1:60666 - "POST /node/694042db575bc060ce158b54 HTTP/1.0" 202 Accepted 2025-12-17 17:18:37,468 - lava_callback - INFO - Starting async job submit for node 694042db575bc060ce158b54 2025-12-17 17:18:37,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:37,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042db575bc060ce158b54 HTTP/1.1" 200 None 2025-12-17 17:18:37,498 - lava_callback - DEBUG - Processing node 694042db575bc060ce158b54: baseline-x86 2025-12-17 17:18:37,498 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694042db575bc060ce158b54 2025-12-17 17:18:37,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:37,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042db575bc060ce158b54 HTTP/1.1" 200 None 2025-12-17 17:18:37,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:37,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:18:37,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:37,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694042db575bc060ce158b54 HTTP/1.1" 200 None 2025-12-17 17:18:37,528 - lava_callback - INFO - Completed processing callback for node 694042db575bc060ce158b54 2025-12-17 17:18:38,156 - lava_callback - INFO - Received callback for node 694042dd575bc060ce158b57 INFO: 172.21.0.1:60672 - "POST /node/694042dd575bc060ce158b57 HTTP/1.0" 202 Accepted 2025-12-17 17:18:38,162 - lava_callback - INFO - Starting async job submit for node 694042dd575bc060ce158b57 2025-12-17 17:18:38,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:38,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042dd575bc060ce158b57 HTTP/1.1" 200 None 2025-12-17 17:18:38,171 - lava_callback - DEBUG - Processing node 694042dd575bc060ce158b57: baseline-x86 2025-12-17 17:18:38,171 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694042dd575bc060ce158b57 2025-12-17 17:18:38,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:38,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042dd575bc060ce158b57 HTTP/1.1" 200 None 2025-12-17 17:18:38,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:38,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:18:38,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:38,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694042dd575bc060ce158b57 HTTP/1.1" 200 None 2025-12-17 17:18:38,201 - lava_callback - INFO - Completed processing callback for node 694042dd575bc060ce158b57 2025-12-17 17:18:38,865 - lava_callback - INFO - Received callback for node 694042e0575bc060ce158b5a INFO: 172.21.0.1:60686 - "POST /node/694042e0575bc060ce158b5a HTTP/1.0" 202 Accepted 2025-12-17 17:18:38,870 - lava_callback - INFO - Starting async job submit for node 694042e0575bc060ce158b5a 2025-12-17 17:18:38,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:38,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042e0575bc060ce158b5a HTTP/1.1" 200 None 2025-12-17 17:18:38,881 - lava_callback - DEBUG - Processing node 694042e0575bc060ce158b5a: baseline-x86 2025-12-17 17:18:38,882 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694042e0575bc060ce158b5a 2025-12-17 17:18:38,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:38,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042e0575bc060ce158b5a HTTP/1.1" 200 None 2025-12-17 17:18:38,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:38,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:18:38,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:38,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694042e0575bc060ce158b5a HTTP/1.1" 200 None 2025-12-17 17:18:38,915 - lava_callback - INFO - Completed processing callback for node 694042e0575bc060ce158b5a 2025-12-17 17:18:39,541 - lava_callback - INFO - Received callback for node 694042e3575bc060ce158b5d INFO: 172.21.0.1:60700 - "POST /node/694042e3575bc060ce158b5d HTTP/1.0" 202 Accepted 2025-12-17 17:18:39,546 - lava_callback - INFO - Starting async job submit for node 694042e3575bc060ce158b5d 2025-12-17 17:18:39,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:39,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042e3575bc060ce158b5d HTTP/1.1" 200 None 2025-12-17 17:18:39,555 - lava_callback - DEBUG - Processing node 694042e3575bc060ce158b5d: baseline-x86 2025-12-17 17:18:39,555 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694042e3575bc060ce158b5d 2025-12-17 17:18:39,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:39,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042e3575bc060ce158b5d HTTP/1.1" 200 None 2025-12-17 17:18:39,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:39,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:18:39,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:39,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694042e3575bc060ce158b5d HTTP/1.1" 200 None 2025-12-17 17:18:39,590 - lava_callback - INFO - Completed processing callback for node 694042e3575bc060ce158b5d INFO: 172.21.0.1:58476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55590 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:18:57,297 - lava_callback - INFO - Received callback for node 694042f6575bc060ce158b70 INFO: 172.21.0.1:55600 - "POST /node/694042f6575bc060ce158b70 HTTP/1.0" 202 Accepted 2025-12-17 17:18:57,302 - lava_callback - INFO - Starting async job submit for node 694042f6575bc060ce158b70 2025-12-17 17:18:57,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:57,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042f6575bc060ce158b70 HTTP/1.1" 200 None 2025-12-17 17:18:57,311 - lava_callback - DEBUG - Processing node 694042f6575bc060ce158b70: baseline-x86 2025-12-17 17:18:57,311 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694042f6575bc060ce158b70 2025-12-17 17:18:57,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:57,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042f6575bc060ce158b70 HTTP/1.1" 200 None 2025-12-17 17:18:57,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:57,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:18:57,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:57,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694042f6575bc060ce158b70 HTTP/1.1" 200 None 2025-12-17 17:18:57,347 - lava_callback - INFO - Completed processing callback for node 694042f6575bc060ce158b70 2025-12-17 17:18:58,004 - lava_callback - INFO - Received callback for node 694042f8575bc060ce158b73 INFO: 172.21.0.1:55616 - "POST /node/694042f8575bc060ce158b73 HTTP/1.0" 202 Accepted 2025-12-17 17:18:58,009 - lava_callback - INFO - Starting async job submit for node 694042f8575bc060ce158b73 2025-12-17 17:18:58,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:58,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042f8575bc060ce158b73 HTTP/1.1" 200 None 2025-12-17 17:18:58,018 - lava_callback - DEBUG - Processing node 694042f8575bc060ce158b73: baseline-x86 2025-12-17 17:18:58,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694042f8575bc060ce158b73 2025-12-17 17:18:58,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:58,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042f8575bc060ce158b73 HTTP/1.1" 200 None 2025-12-17 17:18:58,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:58,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:18:58,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:18:58,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694042f8575bc060ce158b73 HTTP/1.1" 200 None 2025-12-17 17:18:58,070 - lava_callback - INFO - Completed processing callback for node 694042f8575bc060ce158b73 2025-12-17 17:19:00,914 - lava_callback - INFO - Received callback for node 6942e569e43d2b76e9023bcd INFO: 172.21.0.1:48656 - "POST /node/6942e569e43d2b76e9023bcd HTTP/1.0" 202 Accepted 2025-12-17 17:19:00,920 - lava_callback - INFO - Starting async job submit for node 6942e569e43d2b76e9023bcd 2025-12-17 17:19:00,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:00,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e569e43d2b76e9023bcd HTTP/1.1" 200 None 2025-12-17 17:19:00,955 - lava_callback - DEBUG - Processing node 6942e569e43d2b76e9023bcd: baseline-nfs-arm64 2025-12-17 17:19:01,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:19:01,538 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:19:01,539 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e569e43d2b76e9023bcd/log.txt.gz 2025-12-17 17:19:01,539 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:19:01,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:19:01,758 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:19:01,759 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e569e43d2b76e9023bcd/lava_callback.json.gz 2025-12-17 17:19:01,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:01,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e569e43d2b76e9023bcd HTTP/1.1" 200 None 2025-12-17 17:19:01,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:01,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:19:01,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:01,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e569e43d2b76e9023bcd HTTP/1.1" 200 None 2025-12-17 17:19:01,869 - lava_callback - INFO - Completed processing callback for node 6942e569e43d2b76e9023bcd 2025-12-17 17:19:04,976 - lava_callback - INFO - Received callback for node 694042d3575bc060ce158b4b INFO: 172.21.0.1:48660 - "POST /node/694042d3575bc060ce158b4b HTTP/1.0" 202 Accepted 2025-12-17 17:19:04,982 - lava_callback - INFO - Starting async job submit for node 694042d3575bc060ce158b4b 2025-12-17 17:19:04,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:05,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042d3575bc060ce158b4b HTTP/1.1" 200 None 2025-12-17 17:19:05,007 - lava_callback - DEBUG - Processing node 694042d3575bc060ce158b4b: baseline-x86 2025-12-17 17:19:05,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:19:05,447 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:19:05,447 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-694042d3575bc060ce158b4b/log.txt.gz 2025-12-17 17:19:05,448 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:19:05,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:19:05,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:19:05,602 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-694042d3575bc060ce158b4b/lava_callback.json.gz 2025-12-17 17:19:05,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:05,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042d3575bc060ce158b4b HTTP/1.1" 200 None 2025-12-17 17:19:05,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:05,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:19:05,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:05,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694042d3575bc060ce158b4b HTTP/1.1" 200 None 2025-12-17 17:19:05,661 - lava_callback - INFO - Completed processing callback for node 694042d3575bc060ce158b4b INFO: 172.21.0.1:48664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38116 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:19:17,685 - lava_callback - INFO - Received callback for node 69404307575bc060ce158b83 INFO: 172.21.0.1:38130 - "POST /node/69404307575bc060ce158b83 HTTP/1.0" 202 Accepted 2025-12-17 17:19:17,690 - lava_callback - INFO - Starting async job submit for node 69404307575bc060ce158b83 2025-12-17 17:19:17,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:17,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404307575bc060ce158b83 HTTP/1.1" 200 None 2025-12-17 17:19:17,700 - lava_callback - DEBUG - Processing node 69404307575bc060ce158b83: baseline-x86 2025-12-17 17:19:17,700 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69404307575bc060ce158b83 2025-12-17 17:19:17,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:17,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404307575bc060ce158b83 HTTP/1.1" 200 None 2025-12-17 17:19:17,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:17,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:19:17,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:17,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69404307575bc060ce158b83 HTTP/1.1" 200 None 2025-12-17 17:19:17,733 - lava_callback - INFO - Completed processing callback for node 69404307575bc060ce158b83 2025-12-17 17:19:18,386 - lava_callback - INFO - Received callback for node 6940430a575bc060ce158b86 INFO: 172.21.0.1:38144 - "POST /node/6940430a575bc060ce158b86 HTTP/1.0" 202 Accepted 2025-12-17 17:19:18,392 - lava_callback - INFO - Starting async job submit for node 6940430a575bc060ce158b86 2025-12-17 17:19:18,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:18,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940430a575bc060ce158b86 HTTP/1.1" 200 None 2025-12-17 17:19:18,401 - lava_callback - DEBUG - Processing node 6940430a575bc060ce158b86: baseline-x86 2025-12-17 17:19:18,401 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940430a575bc060ce158b86 2025-12-17 17:19:18,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:18,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940430a575bc060ce158b86 HTTP/1.1" 200 None 2025-12-17 17:19:18,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:18,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:19:18,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:18,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940430a575bc060ce158b86 HTTP/1.1" 200 None 2025-12-17 17:19:18,481 - lava_callback - INFO - Completed processing callback for node 6940430a575bc060ce158b86 INFO: 172.21.0.1:40032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43460 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:19:37,227 - lava_callback - INFO - Received callback for node 69404319575bc060ce158b8f INFO: 172.21.0.1:43464 - "POST /node/69404319575bc060ce158b8f HTTP/1.0" 202 Accepted 2025-12-17 17:19:37,232 - lava_callback - INFO - Starting async job submit for node 69404319575bc060ce158b8f 2025-12-17 17:19:37,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:37,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404319575bc060ce158b8f HTTP/1.1" 200 None 2025-12-17 17:19:37,242 - lava_callback - DEBUG - Processing node 69404319575bc060ce158b8f: baseline-x86 2025-12-17 17:19:37,242 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69404319575bc060ce158b8f 2025-12-17 17:19:37,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:37,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404319575bc060ce158b8f HTTP/1.1" 200 None 2025-12-17 17:19:37,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:37,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:19:37,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:37,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69404319575bc060ce158b8f HTTP/1.1" 200 None 2025-12-17 17:19:37,277 - lava_callback - INFO - Completed processing callback for node 69404319575bc060ce158b8f 2025-12-17 17:19:40,111 - lava_callback - INFO - Received callback for node 694042d1575bc060ce158b48 INFO: 172.21.0.1:34242 - "POST /node/694042d1575bc060ce158b48 HTTP/1.0" 202 Accepted 2025-12-17 17:19:40,119 - lava_callback - INFO - Starting async job submit for node 694042d1575bc060ce158b48 2025-12-17 17:19:40,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:40,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042d1575bc060ce158b48 HTTP/1.1" 200 None 2025-12-17 17:19:40,149 - lava_callback - DEBUG - Processing node 694042d1575bc060ce158b48: blktests-ddp-x86 2025-12-17 17:19:40,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:19:41,124 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:19:41,125 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/blktests-ddp-x86-694042d1575bc060ce158b48/log.txt.gz 2025-12-17 17:19:41,125 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:19:41,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:19:41,330 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:19:41,331 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/blktests-ddp-x86-694042d1575bc060ce158b48/lava_callback.json.gz 2025-12-17 17:19:41,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:41,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694042d1575bc060ce158b48 HTTP/1.1" 200 None 2025-12-17 17:19:41,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:41,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403212575bc060ce156c9f HTTP/1.1" 200 None 2025-12-17 17:19:41,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:41,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694042d1575bc060ce158b48 HTTP/1.1" 200 None 2025-12-17 17:19:41,419 - lava_callback - INFO - Completed processing callback for node 694042d1575bc060ce158b48 INFO: 172.21.0.1:34244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41824 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:19:57,546 - lava_callback - INFO - Received callback for node 6940432e575bc060ce158b94 INFO: 172.21.0.1:41836 - "POST /node/6940432e575bc060ce158b94 HTTP/1.0" 202 Accepted 2025-12-17 17:19:57,552 - lava_callback - INFO - Starting async job submit for node 6940432e575bc060ce158b94 2025-12-17 17:19:57,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:57,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940432e575bc060ce158b94 HTTP/1.1" 200 None 2025-12-17 17:19:57,584 - lava_callback - DEBUG - Processing node 6940432e575bc060ce158b94: baseline-x86 2025-12-17 17:19:57,584 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940432e575bc060ce158b94 2025-12-17 17:19:57,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:57,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940432e575bc060ce158b94 HTTP/1.1" 200 None 2025-12-17 17:19:57,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:57,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:19:57,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:19:57,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940432e575bc060ce158b94 HTTP/1.1" 200 None 2025-12-17 17:19:57,636 - lava_callback - INFO - Completed processing callback for node 6940432e575bc060ce158b94 INFO: 172.21.0.1:57838 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:20:11,243 - lava_callback - INFO - Received callback for node 69404349575bc060ce158ba0 INFO: 172.21.0.1:55042 - "POST /node/69404349575bc060ce158ba0 HTTP/1.0" 202 Accepted 2025-12-17 17:20:11,251 - lava_callback - INFO - Starting async job submit for node 69404349575bc060ce158ba0 2025-12-17 17:20:11,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:11,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404349575bc060ce158ba0 HTTP/1.1" 200 None 2025-12-17 17:20:11,262 - lava_callback - DEBUG - Processing node 69404349575bc060ce158ba0: baseline-arm-broonie 2025-12-17 17:20:11,262 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69404349575bc060ce158ba0 2025-12-17 17:20:11,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:11,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404349575bc060ce158ba0 HTTP/1.1" 200 None 2025-12-17 17:20:11,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:11,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af17afee8383e92b69dc1 HTTP/1.1" 200 None 2025-12-17 17:20:11,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:11,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69404349575bc060ce158ba0 HTTP/1.1" 200 None 2025-12-17 17:20:11,317 - lava_callback - INFO - Completed processing callback for node 69404349575bc060ce158ba0 INFO: 172.21.0.1:55052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55072 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:20:17,723 - lava_callback - INFO - Received callback for node 6940433f575bc060ce158b99 INFO: 172.21.0.1:55080 - "POST /node/6940433f575bc060ce158b99 HTTP/1.0" 202 Accepted 2025-12-17 17:20:17,729 - lava_callback - INFO - Starting async job submit for node 6940433f575bc060ce158b99 2025-12-17 17:20:17,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:17,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940433f575bc060ce158b99 HTTP/1.1" 200 None 2025-12-17 17:20:17,740 - lava_callback - DEBUG - Processing node 6940433f575bc060ce158b99: baseline-arm 2025-12-17 17:20:17,740 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940433f575bc060ce158b99 2025-12-17 17:20:17,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:17,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940433f575bc060ce158b99 HTTP/1.1" 200 None 2025-12-17 17:20:17,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:17,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af17afee8383e92b69dc1 HTTP/1.1" 200 None 2025-12-17 17:20:17,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:17,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940433f575bc060ce158b99 HTTP/1.1" 200 None 2025-12-17 17:20:17,772 - lava_callback - INFO - Completed processing callback for node 6940433f575bc060ce158b99 2025-12-17 17:20:24,786 - lava_callback - INFO - Received callback for node 69404305575bc060ce158b80 INFO: 172.21.0.1:38584 - "POST /node/69404305575bc060ce158b80 HTTP/1.0" 202 Accepted 2025-12-17 17:20:24,792 - lava_callback - INFO - Starting async job submit for node 69404305575bc060ce158b80 2025-12-17 17:20:24,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:24,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404305575bc060ce158b80 HTTP/1.1" 200 None 2025-12-17 17:20:24,823 - lava_callback - DEBUG - Processing node 69404305575bc060ce158b80: baseline-x86 2025-12-17 17:20:25,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:20:25,254 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:20:25,254 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69404305575bc060ce158b80/log.txt.gz 2025-12-17 17:20:25,255 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:20:25,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:20:25,398 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:20:25,399 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69404305575bc060ce158b80/lava_callback.json.gz 2025-12-17 17:20:25,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:25,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404305575bc060ce158b80 HTTP/1.1" 200 None 2025-12-17 17:20:25,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:25,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d7fad9ee307f5b1e72 HTTP/1.1" 200 None 2025-12-17 17:20:25,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:25,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69404305575bc060ce158b80 HTTP/1.1" 200 None 2025-12-17 17:20:25,463 - lava_callback - INFO - Completed processing callback for node 69404305575bc060ce158b80 INFO: 172.21.0.1:38600 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:20:28,752 - lava_callback - INFO - Received callback for node 6942e239e43d2b76e902383b INFO: 172.21.0.1:38606 - "POST /node/6942e239e43d2b76e902383b HTTP/1.0" 202 Accepted 2025-12-17 17:20:28,758 - lava_callback - INFO - Starting async job submit for node 6942e239e43d2b76e902383b 2025-12-17 17:20:28,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:28,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e239e43d2b76e902383b HTTP/1.1" 200 None 2025-12-17 17:20:28,820 - lava_callback - DEBUG - Processing node 6942e239e43d2b76e902383b: baseline-arm 2025-12-17 17:20:29,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:20:29,538 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:20:29,539 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942e239e43d2b76e902383b/log.txt.gz 2025-12-17 17:20:29,540 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:20:29,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:20:29,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:20:29,685 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942e239e43d2b76e902383b/lava_callback.json.gz DEBUG: Setup failed for 6942e239e43d2b76e902383b. Transitting job node result: incomplete -> fail 2025-12-17 17:20:29,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:29,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e239e43d2b76e902383b HTTP/1.1" 200 None 2025-12-17 17:20:29,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:29,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c11b61f1c6006013f5 HTTP/1.1" 200 None 2025-12-17 17:20:29,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:29,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e239e43d2b76e902383b HTTP/1.1" 200 None 2025-12-17 17:20:29,853 - lava_callback - INFO - Completed processing callback for node 6942e239e43d2b76e902383b 2025-12-17 17:20:30,498 - lava_callback - INFO - Received callback for node 6940434b575bc060ce158ba9 INFO: 172.21.0.1:48024 - "POST /node/6940434b575bc060ce158ba9 HTTP/1.0" 202 Accepted 2025-12-17 17:20:30,503 - lava_callback - INFO - Starting async job submit for node 6940434b575bc060ce158ba9 2025-12-17 17:20:30,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:30,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940434b575bc060ce158ba9 HTTP/1.1" 200 None 2025-12-17 17:20:30,513 - lava_callback - DEBUG - Processing node 6940434b575bc060ce158ba9: baseline-arm-broonie 2025-12-17 17:20:30,513 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940434b575bc060ce158ba9 2025-12-17 17:20:30,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:30,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940434b575bc060ce158ba9 HTTP/1.1" 200 None 2025-12-17 17:20:30,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:30,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af17afee8383e92b69dc1 HTTP/1.1" 200 None 2025-12-17 17:20:30,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:20:30,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940434b575bc060ce158ba9 HTTP/1.1" 200 None 2025-12-17 17:20:30,547 - lava_callback - INFO - Completed processing callback for node 6940434b575bc060ce158ba9 INFO: 172.21.0.1:48040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45542 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:21:30,952 - lava_callback - INFO - Received callback for node 69404385575bc060ce158bb9 INFO: 172.21.0.1:54034 - "POST /node/69404385575bc060ce158bb9 HTTP/1.0" 202 Accepted 2025-12-17 17:21:30,958 - lava_callback - INFO - Starting async job submit for node 69404385575bc060ce158bb9 2025-12-17 17:21:30,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:21:30,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404385575bc060ce158bb9 HTTP/1.1" 200 None 2025-12-17 17:21:30,969 - lava_callback - DEBUG - Processing node 69404385575bc060ce158bb9: ltp-syscalls 2025-12-17 17:21:30,969 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69404385575bc060ce158bb9 2025-12-17 17:21:30,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:21:30,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404385575bc060ce158bb9 HTTP/1.1" 200 None 2025-12-17 17:21:30,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:21:30,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af17afee8383e92b69dc1 HTTP/1.1" 200 None 2025-12-17 17:21:30,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:21:31,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69404385575bc060ce158bb9 HTTP/1.1" 200 None 2025-12-17 17:21:31,008 - lava_callback - INFO - Completed processing callback for node 69404385575bc060ce158bb9 2025-12-17 17:21:33,632 - lava_callback - INFO - Received callback for node 6942e606e43d2b76e9023bee INFO: 172.21.0.1:54048 - "POST /node/6942e606e43d2b76e9023bee HTTP/1.0" 202 Accepted 2025-12-17 17:21:33,638 - lava_callback - INFO - Starting async job submit for node 6942e606e43d2b76e9023bee 2025-12-17 17:21:33,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:21:33,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e606e43d2b76e9023bee HTTP/1.1" 200 None 2025-12-17 17:21:33,673 - lava_callback - DEBUG - Processing node 6942e606e43d2b76e9023bee: baseline-nfs-arm64 2025-12-17 17:21:34,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:21:34,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:21:34,251 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e606e43d2b76e9023bee/log.txt.gz 2025-12-17 17:21:34,251 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:21:34,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:21:34,440 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:21:34,441 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6942e606e43d2b76e9023bee/lava_callback.json.gz 2025-12-17 17:21:34,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:21:34,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e606e43d2b76e9023bee HTTP/1.1" 200 None 2025-12-17 17:21:34,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:21:34,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:21:34,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:21:34,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e606e43d2b76e9023bee HTTP/1.1" 200 None 2025-12-17 17:21:34,533 - lava_callback - INFO - Completed processing callback for node 6942e606e43d2b76e9023bee INFO: 172.21.0.1:54058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60752 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:22:31,182 - lava_callback - INFO - Received callback for node 694043c5575bc060ce158bc2 INFO: 172.21.0.1:52086 - "POST /node/694043c5575bc060ce158bc2 HTTP/1.0" 202 Accepted 2025-12-17 17:22:31,187 - lava_callback - INFO - Starting async job submit for node 694043c5575bc060ce158bc2 2025-12-17 17:22:31,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:22:31,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694043c5575bc060ce158bc2 HTTP/1.1" 200 None 2025-12-17 17:22:31,199 - lava_callback - DEBUG - Processing node 694043c5575bc060ce158bc2: ltp-syscalls 2025-12-17 17:22:31,199 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694043c5575bc060ce158bc2 2025-12-17 17:22:31,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:22:31,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694043c5575bc060ce158bc2 HTTP/1.1" 200 None 2025-12-17 17:22:31,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:22:31,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af17afee8383e92b69dc1 HTTP/1.1" 200 None 2025-12-17 17:22:31,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:22:31,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694043c5575bc060ce158bc2 HTTP/1.1" 200 None 2025-12-17 17:22:31,231 - lava_callback - INFO - Completed processing callback for node 694043c5575bc060ce158bc2 INFO: 172.21.0.1:52094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48114 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:22:50,572 - lava_callback - INFO - Received callback for node 6942e2afe43d2b76e902388d INFO: 172.21.0.1:37316 - "POST /node/6942e2afe43d2b76e902388d HTTP/1.0" 202 Accepted 2025-12-17 17:22:50,579 - lava_callback - INFO - Starting async job submit for node 6942e2afe43d2b76e902388d 2025-12-17 17:22:50,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:22:50,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2afe43d2b76e902388d HTTP/1.1" 200 None 2025-12-17 17:22:50,641 - lava_callback - DEBUG - Processing node 6942e2afe43d2b76e902388d: baseline-arm 2025-12-17 17:22:51,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:22:51,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:22:51,371 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942e2afe43d2b76e902388d/log.txt.gz 2025-12-17 17:22:51,372 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:22:51,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:22:51,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:22:51,730 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942e2afe43d2b76e902388d/lava_callback.json.gz DEBUG: Setup failed for 6942e2afe43d2b76e902388d. Transitting job node result: incomplete -> fail 2025-12-17 17:22:51,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:22:51,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2afe43d2b76e902388d HTTP/1.1" 200 None 2025-12-17 17:22:51,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:22:51,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 17:22:51,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:22:51,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2afe43d2b76e902388d HTTP/1.1" 200 None 2025-12-17 17:22:51,903 - lava_callback - INFO - Completed processing callback for node 6942e2afe43d2b76e902388d INFO: 172.21.0.1:37328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47666 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:23:46,291 - lava_callback - INFO - Received callback for node 69403fe4575bc060ce158a3f INFO: 172.21.0.1:36496 - "POST /node/69403fe4575bc060ce158a3f HTTP/1.0" 202 Accepted 2025-12-17 17:23:46,337 - lava_callback - INFO - Starting async job submit for node 69403fe4575bc060ce158a3f 2025-12-17 17:23:46,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:23:46,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403fe4575bc060ce158a3f HTTP/1.1" 200 None 2025-12-17 17:23:46,371 - lava_callback - DEBUG - Processing node 69403fe4575bc060ce158a3f: baseline-arm-pengutronix 2025-12-17 17:23:46,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:36506 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:23:46,617 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:23:46,617 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69403fe4575bc060ce158a3f/log.txt.gz 2025-12-17 17:23:46,618 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:23:46,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:23:46,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:23:46,748 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69403fe4575bc060ce158a3f/lava_callback.json.gz DEBUG: Setup failed for 69403fe4575bc060ce158a3f. Transitting job node result: incomplete -> fail 2025-12-17 17:23:46,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:23:46,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69403fe4575bc060ce158a3f HTTP/1.1" 200 None 2025-12-17 17:23:46,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:23:46,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af17afee8383e92b69dc1 HTTP/1.1" 200 None 2025-12-17 17:23:46,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:23:46,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69403fe4575bc060ce158a3f HTTP/1.1" 200 None 2025-12-17 17:23:46,838 - lava_callback - INFO - Completed processing callback for node 69403fe4575bc060ce158a3f INFO: 172.21.0.1:36358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60748 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:24:11,201 - lava_callback - INFO - Received callback for node 69404439575bc060ce158bda INFO: 172.21.0.1:59786 - "POST /node/69404439575bc060ce158bda HTTP/1.0" 202 Accepted 2025-12-17 17:24:11,207 - lava_callback - INFO - Starting async job submit for node 69404439575bc060ce158bda 2025-12-17 17:24:11,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:24:11,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404439575bc060ce158bda HTTP/1.1" 200 None 2025-12-17 17:24:11,218 - lava_callback - DEBUG - Processing node 69404439575bc060ce158bda: baseline-arm-broonie 2025-12-17 17:24:11,218 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69404439575bc060ce158bda 2025-12-17 17:24:11,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:24:11,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404439575bc060ce158bda HTTP/1.1" 200 None 2025-12-17 17:24:11,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:24:11,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af125fee8383e92b69ab4 HTTP/1.1" 200 None 2025-12-17 17:24:11,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:24:11,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69404439575bc060ce158bda HTTP/1.1" 200 None 2025-12-17 17:24:11,252 - lava_callback - INFO - Completed processing callback for node 69404439575bc060ce158bda INFO: 172.21.0.1:59800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59812 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:24:17,581 - lava_callback - INFO - Received callback for node 6940442f575bc060ce158bcf INFO: 172.21.0.1:59828 - "POST /node/6940442f575bc060ce158bcf HTTP/1.0" 202 Accepted 2025-12-17 17:24:17,587 - lava_callback - INFO - Starting async job submit for node 6940442f575bc060ce158bcf 2025-12-17 17:24:17,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:24:17,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940442f575bc060ce158bcf HTTP/1.1" 200 None 2025-12-17 17:24:17,601 - lava_callback - DEBUG - Processing node 6940442f575bc060ce158bcf: baseline-arm 2025-12-17 17:24:17,601 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940442f575bc060ce158bcf 2025-12-17 17:24:17,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:24:17,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940442f575bc060ce158bcf HTTP/1.1" 200 None 2025-12-17 17:24:17,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:24:17,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af125fee8383e92b69ab4 HTTP/1.1" 200 None 2025-12-17 17:24:17,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:24:17,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940442f575bc060ce158bcf HTTP/1.1" 200 None 2025-12-17 17:24:17,643 - lava_callback - INFO - Completed processing callback for node 6940442f575bc060ce158bcf INFO: 172.21.0.1:34270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39504 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:25:05,108 - lava_callback - INFO - Received callback for node 6940714f575bc060ce15aa97 INFO: 172.21.0.1:50834 - "POST /node/6940714f575bc060ce15aa97 HTTP/1.0" 202 Accepted 2025-12-17 17:25:05,117 - lava_callback - INFO - Starting async job submit for node 6940714f575bc060ce15aa97 2025-12-17 17:25:05,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:05,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940714f575bc060ce15aa97 HTTP/1.1" 200 None 2025-12-17 17:25:05,191 - lava_callback - DEBUG - Processing node 6940714f575bc060ce15aa97: baseline-nfs-arm64 2025-12-17 17:25:06,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:25:06,494 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:25:06,495 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6940714f575bc060ce15aa97/log.txt.gz 2025-12-17 17:25:06,495 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' INFO: 172.21.0.1:50844 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:25:06,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:25:06,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:25:06,708 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6940714f575bc060ce15aa97/lava_callback.json.gz DEBUG: Setup failed for 6940714f575bc060ce15aa97. Transitting job node result: incomplete -> fail 2025-12-17 17:25:06,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:06,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940714f575bc060ce15aa97 HTTP/1.1" 200 None 2025-12-17 17:25:06,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:06,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693fc0ed0d8ce4018da9f5d2 HTTP/1.1" 200 None 2025-12-17 17:25:06,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:06,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940714f575bc060ce15aa97 HTTP/1.1" 200 None 2025-12-17 17:25:06,894 - lava_callback - INFO - Completed processing callback for node 6940714f575bc060ce15aa97 2025-12-17 17:25:10,905 - lava_callback - INFO - Received callback for node 69404474575bc060ce158bec INFO: 172.21.0.1:56356 - "POST /node/69404474575bc060ce158bec HTTP/1.0" 202 Accepted 2025-12-17 17:25:10,911 - lava_callback - INFO - Starting async job submit for node 69404474575bc060ce158bec 2025-12-17 17:25:10,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:10,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404474575bc060ce158bec HTTP/1.1" 200 None 2025-12-17 17:25:10,924 - lava_callback - DEBUG - Processing node 69404474575bc060ce158bec: baseline-arm-broonie 2025-12-17 17:25:10,924 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69404474575bc060ce158bec 2025-12-17 17:25:10,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:10,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404474575bc060ce158bec HTTP/1.1" 200 None 2025-12-17 17:25:10,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:10,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af125fee8383e92b69ab4 HTTP/1.1" 200 None 2025-12-17 17:25:10,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:10,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69404474575bc060ce158bec HTTP/1.1" 200 None 2025-12-17 17:25:10,956 - lava_callback - INFO - Completed processing callback for node 69404474575bc060ce158bec INFO: 172.21.0.1:56364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56384 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:25:18,125 - lava_callback - INFO - Received callback for node 6940446b575bc060ce158be4 INFO: 172.21.0.1:56396 - "POST /node/6940446b575bc060ce158be4 HTTP/1.0" 202 Accepted 2025-12-17 17:25:18,130 - lava_callback - INFO - Starting async job submit for node 6940446b575bc060ce158be4 2025-12-17 17:25:18,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:18,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940446b575bc060ce158be4 HTTP/1.1" 200 None 2025-12-17 17:25:18,140 - lava_callback - DEBUG - Processing node 6940446b575bc060ce158be4: baseline-arm 2025-12-17 17:25:18,141 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 6940446b575bc060ce158be4 2025-12-17 17:25:18,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:18,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940446b575bc060ce158be4 HTTP/1.1" 200 None 2025-12-17 17:25:18,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:18,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af125fee8383e92b69ab4 HTTP/1.1" 200 None 2025-12-17 17:25:18,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:18,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940446b575bc060ce158be4 HTTP/1.1" 200 None 2025-12-17 17:25:18,171 - lava_callback - INFO - Completed processing callback for node 6940446b575bc060ce158be4 2025-12-17 17:25:24,765 - lava_callback - INFO - Received callback for node 694071a2575bc060ce15ab4a INFO: 172.21.0.1:52644 - "POST /node/694071a2575bc060ce15ab4a HTTP/1.0" 202 Accepted 2025-12-17 17:25:24,773 - lava_callback - INFO - Starting async job submit for node 694071a2575bc060ce15ab4a 2025-12-17 17:25:24,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:24,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071a2575bc060ce15ab4a HTTP/1.1" 200 None 2025-12-17 17:25:24,844 - lava_callback - DEBUG - Processing node 694071a2575bc060ce15ab4a: baseline-nfs-arm64 2025-12-17 17:25:25,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:25:26,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:25:26,169 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694071a2575bc060ce15ab4a/log.txt.gz 2025-12-17 17:25:26,170 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:25:26,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:25:26,356 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:25:26,356 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694071a2575bc060ce15ab4a/lava_callback.json.gz DEBUG: Setup failed for 694071a2575bc060ce15ab4a. Transitting job node result: incomplete -> fail 2025-12-17 17:25:26,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:26,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071a2575bc060ce15ab4a HTTP/1.1" 200 None 2025-12-17 17:25:26,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:26,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693bcc2f41fb4c950977a37a HTTP/1.1" 200 None 2025-12-17 17:25:26,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:52658 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:25:26,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694071a2575bc060ce15ab4a HTTP/1.1" 200 None 2025-12-17 17:25:26,528 - lava_callback - INFO - Completed processing callback for node 694071a2575bc060ce15ab4a 2025-12-17 17:25:27,659 - lava_callback - INFO - Received callback for node 694071f8575bc060ce15abec INFO: 172.21.0.1:52664 - "POST /node/694071f8575bc060ce15abec HTTP/1.0" 202 Accepted 2025-12-17 17:25:27,667 - lava_callback - INFO - Starting async job submit for node 694071f8575bc060ce15abec 2025-12-17 17:25:27,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:27,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071f8575bc060ce15abec HTTP/1.1" 200 None 2025-12-17 17:25:27,738 - lava_callback - DEBUG - Processing node 694071f8575bc060ce15abec: baseline-nfs-arm64 2025-12-17 17:25:28,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:25:29,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:25:29,020 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694071f8575bc060ce15abec/log.txt.gz 2025-12-17 17:25:29,020 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:25:29,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:25:29,206 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:25:29,207 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694071f8575bc060ce15abec/lava_callback.json.gz DEBUG: Setup failed for 694071f8575bc060ce15abec. Transitting job node result: incomplete -> fail 2025-12-17 17:25:29,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:29,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071f8575bc060ce15abec HTTP/1.1" 200 None 2025-12-17 17:25:29,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:29,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693bcc6941fb4c950977a448 HTTP/1.1" 200 None 2025-12-17 17:25:29,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:25:29,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694071f8575bc060ce15abec HTTP/1.1" 200 None 2025-12-17 17:25:29,379 - lava_callback - INFO - Completed processing callback for node 694071f8575bc060ce15abec INFO: 172.21.0.1:53912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49704 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:26:28,900 - lava_callback - INFO - Received callback for node 694041c4575bc060ce158abb INFO: 172.21.0.1:49708 - "POST /node/694041c4575bc060ce158abb HTTP/1.0" 202 Accepted 2025-12-17 17:26:28,906 - lava_callback - INFO - Starting async job submit for node 694041c4575bc060ce158abb 2025-12-17 17:26:28,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:26:28,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694041c4575bc060ce158abb HTTP/1.1" 200 None 2025-12-17 17:26:28,945 - lava_callback - DEBUG - Processing node 694041c4575bc060ce158abb: baseline-x86-kcidebug-intel 2025-12-17 17:26:29,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:26:29,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:26:29,339 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-694041c4575bc060ce158abb/log.txt.gz 2025-12-17 17:26:29,339 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:26:29,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:26:29,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:26:29,501 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-694041c4575bc060ce158abb/lava_callback.json.gz DEBUG: Setup failed for 694041c4575bc060ce158abb. Transitting job node result: incomplete -> fail 2025-12-17 17:26:29,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:26:29,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694041c4575bc060ce158abb HTTP/1.1" 200 None 2025-12-17 17:26:29,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:26:29,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af218fee8383e92b6a1e5 HTTP/1.1" 200 None 2025-12-17 17:26:29,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:26:29,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694041c4575bc060ce158abb HTTP/1.1" 200 None 2025-12-17 17:26:29,650 - lava_callback - INFO - Completed processing callback for node 694041c4575bc060ce158abb INFO: 172.21.0.1:48446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44856 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:26:51,392 - lava_callback - INFO - Received callback for node 694044d9575bc060ce158bfb INFO: 172.21.0.1:52104 - "POST /node/694044d9575bc060ce158bfb HTTP/1.0" 202 Accepted 2025-12-17 17:26:51,397 - lava_callback - INFO - Starting async job submit for node 694044d9575bc060ce158bfb 2025-12-17 17:26:51,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:26:51,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694044d9575bc060ce158bfb HTTP/1.1" 200 None 2025-12-17 17:26:51,407 - lava_callback - DEBUG - Processing node 694044d9575bc060ce158bfb: ltp-syscalls 2025-12-17 17:26:51,407 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694044d9575bc060ce158bfb 2025-12-17 17:26:51,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:26:51,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694044d9575bc060ce158bfb HTTP/1.1" 200 None 2025-12-17 17:26:51,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:26:51,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af125fee8383e92b69ab4 HTTP/1.1" 200 None 2025-12-17 17:26:51,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:26:51,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694044d9575bc060ce158bfb HTTP/1.1" 200 None 2025-12-17 17:26:51,462 - lava_callback - INFO - Completed processing callback for node 694044d9575bc060ce158bfb INFO: 172.21.0.1:52112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49028 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:27:31,398 - lava_callback - INFO - Received callback for node 6940423e575bc060ce158af3 INFO: 172.21.0.1:47814 - "POST /node/6940423e575bc060ce158af3 HTTP/1.0" 202 Accepted 2025-12-17 17:27:31,404 - lava_callback - INFO - Starting async job submit for node 6940423e575bc060ce158af3 2025-12-17 17:27:31,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:27:31,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940423e575bc060ce158af3 HTTP/1.1" 200 None 2025-12-17 17:27:31,469 - lava_callback - DEBUG - Processing node 6940423e575bc060ce158af3: baseline-x86-kcidebug-intel 2025-12-17 17:27:31,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:27:31,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:27:31,860 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-6940423e575bc060ce158af3/log.txt.gz 2025-12-17 17:27:31,860 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:27:31,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:27:32,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:27:32,021 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-6940423e575bc060ce158af3/lava_callback.json.gz DEBUG: Setup failed for 6940423e575bc060ce158af3. Transitting job node result: incomplete -> fail 2025-12-17 17:27:32,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:27:32,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940423e575bc060ce158af3 HTTP/1.1" 200 None 2025-12-17 17:27:32,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:27:32,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af14dfee8383e92b69bdc HTTP/1.1" 200 None 2025-12-17 17:27:32,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:27:32,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940423e575bc060ce158af3 HTTP/1.1" 200 None 2025-12-17 17:27:32,145 - lava_callback - INFO - Completed processing callback for node 6940423e575bc060ce158af3 INFO: 172.21.0.1:47830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41844 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:27:50,844 - lava_callback - INFO - Received callback for node 69404515575bc060ce158c04 INFO: 172.21.0.1:42136 - "POST /node/69404515575bc060ce158c04 HTTP/1.0" 202 Accepted 2025-12-17 17:27:50,850 - lava_callback - INFO - Starting async job submit for node 69404515575bc060ce158c04 2025-12-17 17:27:50,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:27:50,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404515575bc060ce158c04 HTTP/1.1" 200 None 2025-12-17 17:27:50,859 - lava_callback - DEBUG - Processing node 69404515575bc060ce158c04: ltp-syscalls 2025-12-17 17:27:50,859 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69404515575bc060ce158c04 2025-12-17 17:27:50,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:27:50,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69404515575bc060ce158c04 HTTP/1.1" 200 None 2025-12-17 17:27:50,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:27:50,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af125fee8383e92b69ab4 HTTP/1.1" 200 None 2025-12-17 17:27:50,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:27:50,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69404515575bc060ce158c04 HTTP/1.1" 200 None 2025-12-17 17:27:50,890 - lava_callback - INFO - Completed processing callback for node 69404515575bc060ce158c04 INFO: 172.21.0.1:42138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54084 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:28:29,259 - lava_callback - INFO - Received callback for node 694071fa575bc060ce15abf2 INFO: 172.21.0.1:54088 - "POST /node/694071fa575bc060ce15abf2 HTTP/1.0" 202 Accepted 2025-12-17 17:28:29,267 - lava_callback - INFO - Starting async job submit for node 694071fa575bc060ce15abf2 2025-12-17 17:28:29,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:28:29,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071fa575bc060ce15abf2 HTTP/1.1" 200 None 2025-12-17 17:28:29,337 - lava_callback - DEBUG - Processing node 694071fa575bc060ce15abf2: baseline-nfs-arm64 2025-12-17 17:28:30,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:28:30,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:28:30,662 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694071fa575bc060ce15abf2/log.txt.gz 2025-12-17 17:28:30,662 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:28:30,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:28:30,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:28:30,880 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694071fa575bc060ce15abf2/lava_callback.json.gz DEBUG: Setup failed for 694071fa575bc060ce15abf2. Transitting job node result: incomplete -> fail 2025-12-17 17:28:31,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:28:31,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071fa575bc060ce15abf2 HTTP/1.1" 200 None 2025-12-17 17:28:31,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:28:31,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693bcc6941fb4c950977a448 HTTP/1.1" 200 None 2025-12-17 17:28:31,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:28:31,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694071fa575bc060ce15abf2 HTTP/1.1" 200 None 2025-12-17 17:28:31,051 - lava_callback - INFO - Completed processing callback for node 694071fa575bc060ce15abf2 INFO: 172.21.0.1:60940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56320 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:28:59,868 - lava_callback - INFO - Received callback for node 694041eb575bc060ce158ad0 INFO: 172.21.0.1:35054 - "POST /node/694041eb575bc060ce158ad0 HTTP/1.0" 202 Accepted 2025-12-17 17:28:59,874 - lava_callback - INFO - Starting async job submit for node 694041eb575bc060ce158ad0 2025-12-17 17:28:59,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:28:59,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694041eb575bc060ce158ad0 HTTP/1.1" 200 None 2025-12-17 17:28:59,914 - lava_callback - DEBUG - Processing node 694041eb575bc060ce158ad0: baseline-arm 2025-12-17 17:29:00,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:29:00,372 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:29:00,373 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-694041eb575bc060ce158ad0/log.txt.gz 2025-12-17 17:29:00,373 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:29:00,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:29:00,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:29:00,509 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-694041eb575bc060ce158ad0/lava_callback.json.gz DEBUG: Setup failed for 694041eb575bc060ce158ad0. Transitting job node result: incomplete -> fail 2025-12-17 17:29:00,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:29:00,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694041eb575bc060ce158ad0 HTTP/1.1" 200 None 2025-12-17 17:29:00,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:29:00,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af125fee8383e92b69ab4 HTTP/1.1" 200 None 2025-12-17 17:29:00,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:29:00,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694041eb575bc060ce158ad0 HTTP/1.1" 200 None 2025-12-17 17:29:00,606 - lava_callback - INFO - Completed processing callback for node 694041eb575bc060ce158ad0 INFO: 172.21.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52866 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:31:47,418 - lava_callback - INFO - Received callback for node 6940c5011e3c24e95681d60a INFO: 172.21.0.1:52874 - "POST /node/6940c5011e3c24e95681d60a HTTP/1.0" 202 Accepted 2025-12-17 17:31:47,425 - lava_callback - INFO - Starting async job submit for node 6940c5011e3c24e95681d60a 2025-12-17 17:31:47,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:31:47,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5011e3c24e95681d60a HTTP/1.1" 200 None 2025-12-17 17:31:47,477 - lava_callback - DEBUG - Processing node 6940c5011e3c24e95681d60a: baseline-x86 2025-12-17 17:31:48,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:31:48,445 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:31:48,446 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-6940c5011e3c24e95681d60a/log.txt.gz 2025-12-17 17:31:48,446 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:31:48,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:31:48,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:31:48,669 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-6940c5011e3c24e95681d60a/lava_callback.json.gz DEBUG: Setup failed for 6940c5011e3c24e95681d60a. Transitting job node result: incomplete -> fail 2025-12-17 17:31:48,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:31:48,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5011e3c24e95681d60a HTTP/1.1" 200 None 2025-12-17 17:31:48,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:31:48,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693cb0d5fad9ee307f5b1e66 HTTP/1.1" 200 None 2025-12-17 17:31:48,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:31:48,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940c5011e3c24e95681d60a HTTP/1.1" 200 None 2025-12-17 17:31:48,809 - lava_callback - INFO - Completed processing callback for node 6940c5011e3c24e95681d60a 2025-12-17 17:31:55,420 - lava_callback - INFO - Received callback for node 694071fc575bc060ce15abf5 INFO: 172.21.0.1:40074 - "POST /node/694071fc575bc060ce15abf5 HTTP/1.0" 202 Accepted 2025-12-17 17:31:55,428 - lava_callback - INFO - Starting async job submit for node 694071fc575bc060ce15abf5 2025-12-17 17:31:55,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:31:55,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071fc575bc060ce15abf5 HTTP/1.1" 200 None 2025-12-17 17:31:55,482 - lava_callback - DEBUG - Processing node 694071fc575bc060ce15abf5: baseline-nfs-arm64 INFO: 172.21.0.1:40084 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:31:56,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:31:56,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:31:56,925 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694071fc575bc060ce15abf5/log.txt.gz 2025-12-17 17:31:56,926 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:31:56,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:31:57,146 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:31:57,147 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-694071fc575bc060ce15abf5/lava_callback.json.gz DEBUG: Setup failed for 694071fc575bc060ce15abf5. Transitting job node result: incomplete -> fail 2025-12-17 17:31:57,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:31:57,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071fc575bc060ce15abf5 HTTP/1.1" 200 None 2025-12-17 17:31:57,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:31:57,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693bcc2f41fb4c950977a37a HTTP/1.1" 200 None 2025-12-17 17:31:57,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:31:57,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694071fc575bc060ce15abf5 HTTP/1.1" 200 None 2025-12-17 17:31:57,281 - lava_callback - INFO - Completed processing callback for node 694071fc575bc060ce15abf5 INFO: 172.21.0.1:45908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48418 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:35:38,029 - lava_callback - INFO - Received callback for node 694046e4575bc060ce158c98 INFO: 172.21.0.1:48422 - "POST /node/694046e4575bc060ce158c98 HTTP/1.0" 202 Accepted 2025-12-17 17:35:38,034 - lava_callback - INFO - Starting async job submit for node 694046e4575bc060ce158c98 2025-12-17 17:35:38,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:35:38,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694046e4575bc060ce158c98 HTTP/1.1" 200 None 2025-12-17 17:35:38,043 - lava_callback - DEBUG - Processing node 694046e4575bc060ce158c98: baseline-arm 2025-12-17 17:35:38,043 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 694046e4575bc060ce158c98 2025-12-17 17:35:38,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:35:38,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694046e4575bc060ce158c98 HTTP/1.1" 200 None 2025-12-17 17:35:38,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:35:38,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693af17afee8383e92b69dc1 HTTP/1.1" 200 None 2025-12-17 17:35:38,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:35:38,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/694046e4575bc060ce158c98 HTTP/1.1" 200 None 2025-12-17 17:35:38,077 - lava_callback - INFO - Completed processing callback for node 694046e4575bc060ce158c98 INFO: 172.21.0.1:45908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35126 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:40:33,172 - lava_callback - INFO - Received callback for node 6942e6ece43d2b76e9023c40 INFO: 172.21.0.1:47680 - "POST /node/6942e6ece43d2b76e9023c40 HTTP/1.0" 202 Accepted 2025-12-17 17:40:33,179 - lava_callback - INFO - Starting async job submit for node 6942e6ece43d2b76e9023c40 2025-12-17 17:40:33,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:40:33,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e6ece43d2b76e9023c40 HTTP/1.1" 200 None 2025-12-17 17:40:33,262 - lava_callback - DEBUG - Processing node 6942e6ece43d2b76e9023c40: baseline-arm 2025-12-17 17:40:33,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:40:34,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:40:34,019 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-6942e6ece43d2b76e9023c40/log.txt.gz 2025-12-17 17:40:34,019 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:40:34,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:40:34,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:40:34,216 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-6942e6ece43d2b76e9023c40/lava_callback.json.gz DEBUG: Setup failed for 6942e6ece43d2b76e9023c40. Transitting job node result: incomplete -> fail 2025-12-17 17:40:34,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:40:34,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e6ece43d2b76e9023c40 HTTP/1.1" 200 None 2025-12-17 17:40:34,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:40:34,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d778e43d2b76e9021b19 HTTP/1.1" 200 None 2025-12-17 17:40:34,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:40:34,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e6ece43d2b76e9023c40 HTTP/1.1" 200 None 2025-12-17 17:40:34,373 - lava_callback - INFO - Completed processing callback for node 6942e6ece43d2b76e9023c40 INFO: 172.21.0.1:47688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45648 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:40:47,322 - lava_callback - INFO - Received callback for node 6940758b575bc060ce15b029 INFO: 172.21.0.1:45660 - "POST /node/6940758b575bc060ce15b029 HTTP/1.0" 202 Accepted 2025-12-17 17:40:47,330 - lava_callback - INFO - Starting async job submit for node 6940758b575bc060ce15b029 2025-12-17 17:40:47,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:40:47,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940758b575bc060ce15b029 HTTP/1.1" 200 None 2025-12-17 17:40:47,400 - lava_callback - DEBUG - Processing node 6940758b575bc060ce15b029: baseline-nfs-arm64 2025-12-17 17:40:48,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:40:48,777 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:40:48,778 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6940758b575bc060ce15b029/log.txt.gz 2025-12-17 17:40:48,778 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:40:48,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:40:48,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:40:48,992 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-6940758b575bc060ce15b029/lava_callback.json.gz DEBUG: Setup failed for 6940758b575bc060ce15b029. Transitting job node result: incomplete -> fail 2025-12-17 17:40:49,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:40:49,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940758b575bc060ce15b029 HTTP/1.1" 200 None 2025-12-17 17:40:49,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:40:49,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693d1ed3fad9ee307f5cfc5b HTTP/1.1" 200 None 2025-12-17 17:40:49,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:40:49,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6940758b575bc060ce15b029 HTTP/1.1" 200 None 2025-12-17 17:40:49,166 - lava_callback - INFO - Completed processing callback for node 6940758b575bc060ce15b029 INFO: 172.21.0.1:59434 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:41:06,534 - lava_callback - INFO - Received callback for node 6942e2f1e43d2b76e902391b INFO: 172.21.0.1:54782 - "POST /node/6942e2f1e43d2b76e902391b HTTP/1.0" 202 Accepted 2025-12-17 17:41:06,540 - lava_callback - INFO - Starting async job submit for node 6942e2f1e43d2b76e902391b 2025-12-17 17:41:06,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:41:06,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f1e43d2b76e902391b HTTP/1.1" 200 None 2025-12-17 17:41:06,570 - lava_callback - DEBUG - Processing node 6942e2f1e43d2b76e902391b: baseline-arm64-broonie INFO: 172.21.0.1:54790 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:41:06,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:41:07,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:41:07,003 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-6942e2f1e43d2b76e902391b/log.txt.gz 2025-12-17 17:41:07,003 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:41:07,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:41:07,149 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:41:07,150 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-6942e2f1e43d2b76e902391b/lava_callback.json.gz 2025-12-17 17:41:07,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:41:07,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f1e43d2b76e902391b HTTP/1.1" 200 None 2025-12-17 17:41:07,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:41:07,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:41:07,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:41:07,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2f1e43d2b76e902391b HTTP/1.1" 200 None 2025-12-17 17:41:07,212 - lava_callback - INFO - Completed processing callback for node 6942e2f1e43d2b76e902391b INFO: 172.21.0.1:52492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50148 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:41:31,484 - lava_callback - INFO - Received callback for node 6942e2f5e43d2b76e9023934 INFO: 172.21.0.1:55092 - "POST /node/6942e2f5e43d2b76e9023934 HTTP/1.0" 202 Accepted 2025-12-17 17:41:31,490 - lava_callback - INFO - Starting async job submit for node 6942e2f5e43d2b76e9023934 2025-12-17 17:41:31,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:41:31,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f5e43d2b76e9023934 HTTP/1.1" 200 None 2025-12-17 17:41:31,530 - lava_callback - DEBUG - Processing node 6942e2f5e43d2b76e9023934: baseline-arm64-broonie 2025-12-17 17:41:31,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:41:31,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:41:31,909 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-6942e2f5e43d2b76e9023934/log.txt.gz 2025-12-17 17:41:31,910 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:41:31,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:41:32,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:41:32,049 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-6942e2f5e43d2b76e9023934/lava_callback.json.gz 2025-12-17 17:41:32,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:41:32,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f5e43d2b76e9023934 HTTP/1.1" 200 None 2025-12-17 17:41:32,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:41:32,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d4c41b61f1c6006013fe HTTP/1.1" 200 None 2025-12-17 17:41:32,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:41:32,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e2f5e43d2b76e9023934 HTTP/1.1" 200 None 2025-12-17 17:41:32,109 - lava_callback - INFO - Completed processing callback for node 6942e2f5e43d2b76e9023934 INFO: 172.21.0.1:55108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60480 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:41:58,674 - lava_callback - INFO - Received callback for node 69407459575bc060ce15aec7 INFO: 172.21.0.1:60488 - "POST /node/69407459575bc060ce15aec7 HTTP/1.0" 202 Accepted 2025-12-17 17:41:58,685 - lava_callback - INFO - Starting async job submit for node 69407459575bc060ce15aec7 2025-12-17 17:41:58,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:41:59,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407459575bc060ce15aec7 HTTP/1.1" 200 None 2025-12-17 17:41:59,279 - lava_callback - DEBUG - Processing node 69407459575bc060ce15aec7: baseline-nfs-arm64 2025-12-17 17:42:00,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:42:00,852 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:42:00,853 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69407459575bc060ce15aec7/log.txt.gz 2025-12-17 17:42:00,853 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:42:00,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:42:01,086 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:42:01,087 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69407459575bc060ce15aec7/lava_callback.json.gz DEBUG: Setup failed for 69407459575bc060ce15aec7. Transitting job node result: incomplete -> fail 2025-12-17 17:42:01,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:42:01,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407459575bc060ce15aec7 HTTP/1.1" 200 None 2025-12-17 17:42:01,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:42:03,732 - lava_callback - INFO - Received callback for node 6942e383e43d2b76e9023a7a INFO: 172.21.0.1:42552 - "POST /node/6942e383e43d2b76e9023a7a HTTP/1.0" 202 Accepted 2025-12-17 17:42:03,739 - lava_callback - INFO - Starting async job submit for node 6942e383e43d2b76e9023a7a 2025-12-17 17:42:03,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:42:05,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/693bcc2f41fb4c950977a37a HTTP/1.1" 200 None 2025-12-17 17:42:05,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:42:05,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e383e43d2b76e9023a7a HTTP/1.1" 200 None 2025-12-17 17:42:05,459 - lava_callback - DEBUG - Processing node 6942e383e43d2b76e9023a7a: baseline-arm64-broonie 2025-12-17 17:42:05,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69407459575bc060ce15aec7 HTTP/1.1" 200 None 2025-12-17 17:42:05,699 - lava_callback - INFO - Completed processing callback for node 69407459575bc060ce15aec7 2025-12-17 17:42:05,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:42:06,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:42:06,021 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-6942e383e43d2b76e9023a7a/log.txt.gz 2025-12-17 17:42:06,021 - lava_callback - WARNING - Failed to upload lava-logs.yaml: 'LogParser' object has no attribute 'get_data' 2025-12-17 17:42:06,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-12-17 17:42:06,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-12-17 17:42:06,217 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-6942e383e43d2b76e9023a7a/lava_callback.json.gz 2025-12-17 17:42:06,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:42:06,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e383e43d2b76e9023a7a HTTP/1.1" 200 None 2025-12-17 17:42:06,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:42:06,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d789e43d2b76e9021b40 HTTP/1.1" 200 None 2025-12-17 17:42:06,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:42:06,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/6942e383e43d2b76e9023a7a HTTP/1.1" 200 None 2025-12-17 17:42:06,394 - lava_callback - INFO - Completed processing callback for node 6942e383e43d2b76e9023a7a INFO: 172.21.0.1:42564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49936 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:42:17,767 - lava_callback - INFO - Received callback for node 694075c7575bc060ce15b055 INFO: 172.21.0.1:49952 - "POST /node/694075c7575bc060ce15b055 HTTP/1.0" 202 Accepted 2025-12-17 17:42:17,777 - lava_callback - INFO - Starting async job submit for node 694075c7575bc060ce15b055 2025-12-17 17:42:17,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:42:24,986 - lava_callback - INFO - Received callback for node 6942e65de43d2b76e9023c14 INFO: 172.21.0.1:36562 - "POST /node/6942e65de43d2b76e9023c14 HTTP/1.0" 202 Accepted 2025-12-17 17:42:25,001 - lava_callback - INFO - Starting async job submit for node 6942e65de43d2b76e9023c14 2025-12-17 17:42:25,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:36570 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:42:37,240 - lava_callback - INFO - Received callback for node 694047e6575bc060ce158cd6 INFO: 172.21.0.1:36632 - "POST /node/694047e6575bc060ce158cd6 HTTP/1.0" 202 Accepted 2025-12-17 17:42:39,993 - lava_callback - INFO - Starting async job submit for node 694047e6575bc060ce158cd6 INFO: 172.21.0.1:49406 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:42:51,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:45100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45112 - "GET /node/69402583575bc060ce1540e7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:45126 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:43:12,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694047e6575bc060ce158cd6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:43:13,647 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694047e6575bc060ce158cd6 2025-12-17 17:43:14,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:60066 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:43:18,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694075c7575bc060ce15b055'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:43:21,598 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694075c7575bc060ce15b055 2025-12-17 17:43:22,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:60080 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 17:43:25,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e65de43d2b76e9023c14'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:43:27,148 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6942e65de43d2b76e9023c14 INFO: 172.21.0.1:60094 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:43:29,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:52758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52766 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:43:36,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694047e6575bc060ce158cd6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:43:38,849 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694047e6575bc060ce158cd6 2025-12-17 17:43:40,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:39638 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:43:44,104 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694075c7575bc060ce15b055'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:43:45,149 - lava_callback - INFO - Received callback for node 694048c0575bc060ce158d04 2025-12-17 17:43:46,552 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694075c7575bc060ce15b055 INFO: 172.21.0.1:39650 - "POST /node/694048c0575bc060ce158d04 HTTP/1.0" 202 Accepted 2025-12-17 17:43:47,404 - lava_callback - INFO - Starting async job submit for node 694048c0575bc060ce158d04 2025-12-17 17:43:47,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 17:43:51,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e65de43d2b76e9023c14'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:43:53,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:60146 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:43:55,151 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/6942e65de43d2b76e9023c14 2025-12-17 17:43:57,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 17:44:00,049 - lava_callback - INFO - Received callback for node 694075c9575bc060ce15b058 2025-12-17 17:44:02,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694047e6575bc060ce158cd6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60156 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:44:08,752 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694047e6575bc060ce158cd6 2025-12-17 17:44:09,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694075c7575bc060ce15b055'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:44:13,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 17:44:13,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694048c0575bc060ce158d04'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:44:14,144 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694048c0575bc060ce158d04 INFO: 172.21.0.1:60148 - "POST /node/694075c9575bc060ce15b058 HTTP/1.0" 202 Accepted 2025-12-17 17:44:14,093 - lava_callback - INFO - Starting async job submit for node 694075c9575bc060ce15b058 2025-12-17 17:44:15,741 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694075c7575bc060ce15b055 2025-12-17 17:44:16,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 17:44:19,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 17:44:20,105 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e65de43d2b76e9023c14'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:44:25,795 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/6942e65de43d2b76e9023c14 INFO: 172.21.0.1:54620 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:44:26,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:54634 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 17:44:33,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:44:34,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694047e6575bc060ce158cd6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40244 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:44:39,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694048c0575bc060ce158d04'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40256 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:44:42,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694075c7575bc060ce15b055'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:44:41,953 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694048c0575bc060ce158d04 2025-12-17 17:44:43,452 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694047e6575bc060ce158cd6 2025-12-17 17:44:44,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 17:44:45,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 17:44:47,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e65de43d2b76e9023c14'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:44:50,494 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694075c7575bc060ce15b055 2025-12-17 17:44:53,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 17:44:55,256 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694075c9575bc060ce15b058'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:44:55,704 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694075c9575bc060ce15b058 2025-12-17 17:44:56,805 - lava_callback - INFO - Received callback for node 6942e37fe43d2b76e9023a5d 2025-12-17 17:44:56,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 17:44:57,652 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/6942e65de43d2b76e9023c14 2025-12-17 17:44:59,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 17:45:02,642 - lava_callback - INFO - Starting async job submit for node 6942e37fe43d2b76e9023a5d INFO: 172.21.0.1:58802 - "POST /node/6942e37fe43d2b76e9023a5d HTTP/1.0" 202 Accepted INFO: 172.21.0.1:58816 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:45:06,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694048c0575bc060ce158d04'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:45:07,149 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694047e6575bc060ce158cd6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:45:08,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:52772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52782 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:45:11,835 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694048c0575bc060ce158d04 2025-12-17 17:45:12,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 17:45:15,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694075c7575bc060ce15b055'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:45:17,199 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694075c9575bc060ce15b058'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56562 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 17:45:20,299 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694075c9575bc060ce15b058 2025-12-17 17:45:21,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 17:45:22,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e65de43d2b76e9023c14'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:45:24,799 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694047e6575bc060ce158cd6 2025-12-17 17:45:26,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 17:45:30,155 - lava_callback - INFO - Received callback for node 69404924575bc060ce158d22 2025-12-17 17:45:31,850 - lava_callback - INFO - Starting async job submit for node 69404924575bc060ce158d22 INFO: 172.21.0.1:47872 - "POST /node/69404924575bc060ce158d22 HTTP/1.0" 202 Accepted 2025-12-17 17:45:31,897 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694075c7575bc060ce15b055 2025-12-17 17:45:33,051 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694048c0575bc060ce158d04'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:45:34,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 17:45:35,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e37fe43d2b76e9023a5d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:45:37,592 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/6942e37fe43d2b76e9023a5d 2025-12-17 17:45:40,999 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/6942e65de43d2b76e9023c14 2025-12-17 17:45:43,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 17:45:43,401 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694048c0575bc060ce158d04 2025-12-17 17:45:45,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694075c9575bc060ce15b058'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:45:46,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 17:45:51,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 17:45:54,946 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694075c9575bc060ce15b058 2025-12-17 17:45:54,850 - lava_callback - ERROR - Error processing callback for node 694047e6575bc060ce158cd6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694047e6575bc060ce158cd6 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 204, in _new_conn sock = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 60, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 978, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -3] Temporary failure in name resolution The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_e File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 759, in connect self.sock = sock = self._new_conn() ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 211, in _new_conn raise NameResolutionError(self.host, self, e) from e urllib3.exceptions.NameResolutionError: HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694047e6575bc060ce158cd6 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694047e6575bc060ce158cd6 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) 2025-12-17 17:45:57,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:45:57,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 17:45:58,807 - lava_callback - ERROR - Error processing callback for node 694075c7575bc060ce15b055: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694075c7575bc060ce15b055 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 204, in _new_conn sock = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 60, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 978, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -3] Temporary failure in name resolution The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_e File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 759, in connect self.sock = sock = self._new_conn() ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 211, in _new_conn raise NameResolutionError(self.host, self, e) from e urllib3.exceptions.NameResolutionError: HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694075c7575bc060ce15b055 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694075c7575bc060ce15b055 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) 2025-12-17 17:46:05,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e37fe43d2b76e9023a5d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:46:17,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694048c0575bc060ce158d04'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:46:24,641 - lava_callback - ERROR - Error processing callback for node 6942e65de43d2b76e9023c14: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e65de43d2b76e9023c14 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 204, in _new_conn sock = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 60, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 978, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -3] Temporary failure in name resolution The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_e File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 759, in connect self.sock = sock = self._new_conn() ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 211, in _new_conn raise NameResolutionError(self.host, self, e) from e urllib3.exceptions.NameResolutionError: HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e65de43d2b76e9023c14 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e65de43d2b76e9023c14 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) INFO: 172.21.0.1:36404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36406 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:47:17,603 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/6942e37fe43d2b76e9023a5d 2025-12-17 17:47:17,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 17:47:31,948 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694048c0575bc060ce158d04 2025-12-17 17:47:35,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:55940 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:47:44,008 - lava_callback - INFO - Received callback for node 69404d4f575bc060ce158e50 2025-12-17 17:47:47,797 - lava_callback - INFO - Starting async job submit for node 69404d4f575bc060ce158e50 INFO: 172.21.0.1:55952 - "POST /node/69404d4f575bc060ce158e50 HTTP/1.0" 202 Accepted 2025-12-17 17:47:57,496 - lava_callback - ERROR - Error processing callback for node 694048c0575bc060ce158d04: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694048c0575bc060ce158d04 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 204, in _new_conn sock = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 60, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 978, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -3] Temporary failure in name resolution The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_e File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 759, in connect self.sock = sock = self._new_conn() ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 211, in _new_conn raise NameResolutionError(self.host, self, e) from e urllib3.exceptions.NameResolutionError: HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694048c0575bc060ce158d04 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694048c0575bc060ce158d04 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) INFO: 172.21.0.1:50778 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:47:59,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:48:09,304 - lava_callback - INFO - Received callback for node 69404766575bc060ce158cb4 2025-12-17 17:48:11,847 - lava_callback - INFO - Starting async job submit for node 69404766575bc060ce158cb4 INFO: 172.21.0.1:38282 - "POST /node/69404766575bc060ce158cb4 HTTP/1.0" 202 Accepted 2025-12-17 17:48:18,210 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694075c9575bc060ce15b058'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:48:18,099 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404924575bc060ce158d22'): Retry(total=4, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53536 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:48:20,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e37fe43d2b76e9023a5d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:48:22,051 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69404924575bc060ce158d22 2025-12-17 17:48:22,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 17:48:22,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 17:48:25,158 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404d4f575bc060ce158e50'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:48:26,098 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404d4f575bc060ce158e50 2025-12-17 17:48:31,401 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694075c9575bc060ce15b058 2025-12-17 17:48:30,945 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6942e37fe43d2b76e9023a5d 2025-12-17 17:48:34,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 17:48:35,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 17:48:36,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:53556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60226 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:48:42,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404924575bc060ce158d22'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:48:43,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404766575bc060ce158cb4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:48:44,952 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404766575bc060ce158cb4 2025-12-17 17:48:45,809 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404924575bc060ce158d22 2025-12-17 17:48:46,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 17:48:47,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:51682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47394 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:48:58,391 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694075c9575bc060ce15b058'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:48:58,294 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e37fe43d2b76e9023a5d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:48:58,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404d4f575bc060ce158e50'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47396 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:49:04,148 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404d4f575bc060ce158e50 2025-12-17 17:49:04,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 17:49:08,793 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/6942e37fe43d2b76e9023a5d 2025-12-17 17:49:10,303 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404766575bc060ce158cb4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:49:08,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404924575bc060ce158d22'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:49:14,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 17:49:14,346 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404766575bc060ce158cb4 2025-12-17 17:49:15,947 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/694075c9575bc060ce15b058 2025-12-17 17:49:15,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 17:49:18,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 17:49:18,993 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404924575bc060ce158d22 2025-12-17 17:49:22,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 17:49:26,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404d4f575bc060ce158e50'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43326 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 17:49:33,042 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404d4f575bc060ce158e50 2025-12-17 17:49:35,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 17:49:38,242 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e37fe43d2b76e9023a5d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:49:39,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404766575bc060ce158cb4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:49:42,949 - lava_callback - ERROR - Error processing callback for node 694075c9575bc060ce15b058: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694075c9575bc060ce15b058 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 204, in _new_conn sock = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 60, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 978, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -3] Temporary failure in name resolution The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_e File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 759, in connect self.sock = sock = self._new_conn() ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 211, in _new_conn raise NameResolutionError(self.host, self, e) from e urllib3.exceptions.NameResolutionError: HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694075c9575bc060ce15b058 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694075c9575bc060ce15b058 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) 2025-12-17 17:49:44,407 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404924575bc060ce158d22'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:49:46,952 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404766575bc060ce158cb4 2025-12-17 17:49:59,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404d4f575bc060ce158e50'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:49:59,347 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/6942e37fe43d2b76e9023a5d 2025-12-17 17:50:16,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 17:50:27,199 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404d4f575bc060ce158e50 2025-12-17 17:50:25,149 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404924575bc060ce158d22 2025-12-17 17:50:35,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 17:50:36,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 17:50:40,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:56544 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:50:55,206 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404766575bc060ce158cb4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:50:59,257 - lava_callback - ERROR - Error processing callback for node 6942e37fe43d2b76e9023a5d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e37fe43d2b76e9023a5d (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 204, in _new_conn sock = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 60, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 978, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -3] Temporary failure in name resolution The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_e File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 759, in connect self.sock = sock = self._new_conn() ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 211, in _new_conn raise NameResolutionError(self.host, self, e) from e urllib3.exceptions.NameResolutionError: HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e37fe43d2b76e9023a5d (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e37fe43d2b76e9023a5d (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) 2025-12-17 17:51:02,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404d4f575bc060ce158e50'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:51:05,206 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404924575bc060ce158d22'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:51:05,449 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404766575bc060ce158cb4 2025-12-17 17:51:05,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:42778 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:51:20,760 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404d4f575bc060ce158e50 2025-12-17 17:51:21,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:42794 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 17:51:21,507 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404924575bc060ce158d22 2025-12-17 17:51:22,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 17:51:42,649 - lava_callback - ERROR - Error processing callback for node 69404d4f575bc060ce158e50: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69404d4f575bc060ce158e50 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 204, in _new_conn sock = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 60, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 978, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -3] Temporary failure in name resolution The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_e File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 759, in connect self.sock = sock = self._new_conn() ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 211, in _new_conn raise NameResolutionError(self.host, self, e) from e urllib3.exceptions.NameResolutionError: HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69404d4f575bc060ce158e50 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69404d4f575bc060ce158e50 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) 2025-12-17 17:51:44,409 - lava_callback - ERROR - Error processing callback for node 69404924575bc060ce158d22: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69404924575bc060ce158d22 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 204, in _new_conn sock = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 60, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 978, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -3] Temporary failure in name resolution The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_e File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 759, in connect self.sock = sock = self._new_conn() ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 211, in _new_conn raise NameResolutionError(self.host, self, e) from e urllib3.exceptions.NameResolutionError: HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69404924575bc060ce158d22 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69404924575bc060ce158d22 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) 2025-12-17 17:52:11,260 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404766575bc060ce158cb4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:52:31,850 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69404766575bc060ce158cb4 2025-12-17 17:52:41,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:36270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54476 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:53:07,624 - lava_callback - ERROR - Error processing callback for node 69404766575bc060ce158cb4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69404766575bc060ce158cb4 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 204, in _new_conn sock = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 60, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 978, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -3] Temporary failure in name resolution The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_e File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 759, in connect self.sock = sock = self._new_conn() ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 211, in _new_conn raise NameResolutionError(self.host, self, e) from e urllib3.exceptions.NameResolutionError: HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69404766575bc060ce158cb4 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69404766575bc060ce158cb4 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) INFO: 172.21.0.1:43574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44644 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:57:32,806 - lava_callback - INFO - Received callback for node 69404c00575bc060ce158de5 INFO: 172.21.0.1:46726 - "POST /node/69404c00575bc060ce158de5 HTTP/1.0" 202 Accepted 2025-12-17 17:57:32,822 - lava_callback - INFO - Starting async job submit for node 69404c00575bc060ce158de5 2025-12-17 17:57:32,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:38992 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:58:31,416 - lava_callback - INFO - Received callback for node 6942eb12e43d2b76e9023ca7 2025-12-17 17:58:36,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404c00575bc060ce158de5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:58:41,014 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69404c00575bc060ce158de5 2025-12-17 17:58:44,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:35358 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 17:58:49,701 - lava_callback - INFO - Starting async job submit for node 6942eb12e43d2b76e9023ca7 INFO: 172.21.0.1:35350 - "POST /node/6942eb12e43d2b76e9023ca7 HTTP/1.0" 202 Accepted 2025-12-17 17:59:07,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404c00575bc060ce158de5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:59:13,253 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404c00575bc060ce158de5 2025-12-17 17:59:16,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 17:59:18,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:57452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49642 - "GET / HTTP/1.0" 200 OK 2025-12-17 17:59:38,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404c00575bc060ce158de5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:59:41,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942eb12e43d2b76e9023ca7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 17:59:42,953 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/6942eb12e43d2b76e9023ca7 2025-12-17 17:59:43,804 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404c00575bc060ce158de5 2025-12-17 17:59:44,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 17:59:44,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 18:00:09,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404c00575bc060ce158de5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 18:00:09,066 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942eb12e43d2b76e9023ca7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 18:00:16,548 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/6942eb12e43d2b76e9023ca7 2025-12-17 18:00:19,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 18:00:20,148 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404c00575bc060ce158de5 2025-12-17 18:00:22,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:43816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57164 - "GET / HTTP/1.0" 200 OK 2025-12-17 18:00:43,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942eb12e43d2b76e9023ca7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 18:00:43,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69404c00575bc060ce158de5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 18:00:49,610 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/6942eb12e43d2b76e9023ca7 2025-12-17 18:00:51,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 18:01:04,302 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/69404c00575bc060ce158de5 2025-12-17 18:01:11,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 18:01:12,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942eb12e43d2b76e9023ca7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 18:01:21,152 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")': /latest/node/6942eb12e43d2b76e9023ca7 2025-12-17 18:01:27,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:42740 - "GET / HTTP/1.0" 200 OK 2025-12-17 18:01:31,256 - lava_callback - ERROR - Error processing callback for node 69404c00575bc060ce158de5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69404c00575bc060ce158de5 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 204, in _new_conn sock = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 60, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 978, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -3] Temporary failure in name resolution The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_e File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 759, in connect self.sock = sock = self._new_conn() ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 211, in _new_conn raise NameResolutionError(self.host, self, e) from e urllib3.exceptions.NameResolutionError: HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69404c00575bc060ce158de5 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69404c00575bc060ce158de5 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) INFO: 172.21.0.1:42754 - "GET / HTTP/1.0" 200 OK 2025-12-17 18:02:34,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942eb12e43d2b76e9023ca7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46440 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 18:02:56,215 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6942eb12e43d2b76e9023ca7 2025-12-17 18:02:57,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 18:03:18,617 - lava_callback - ERROR - Error processing callback for node 6942eb12e43d2b76e9023ca7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942eb12e43d2b76e9023ca7 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 204, in _new_conn sock = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 60, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 978, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -3] Temporary failure in name resolution The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_e File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 759, in connect self.sock = sock = self._new_conn() ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 211, in _new_conn raise NameResolutionError(self.host, self, e) from e urllib3.exceptions.NameResolutionError: HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942eb12e43d2b76e9023ca7 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942eb12e43d2b76e9023ca7 (Caused by NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -3] Temporary failure in name resolution)")) INFO: 172.21.0.1:45686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44590 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:50:58,208 - lava_callback - INFO - Received callback for node 6940c51e1e3c24e95681d663 INFO: 172.21.0.1:44602 - "POST /node/6940c51e1e3c24e95681d663 HTTP/1.0" 202 Accepted 2025-12-17 19:50:58,218 - lava_callback - INFO - Starting async job submit for node 6940c51e1e3c24e95681d663 2025-12-17 19:50:58,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:50:58,765 - lava_callback - INFO - Received callback for node 69408166575bc060ce15cac5 INFO: 172.21.0.1:44610 - "POST /node/69408166575bc060ce15cac5 HTTP/1.0" 202 Accepted 2025-12-17 19:50:58,780 - lava_callback - INFO - Starting async job submit for node 69408166575bc060ce15cac5 2025-12-17 19:50:58,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:45898 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:51:02,986 - lava_callback - INFO - Received callback for node 69408198575bc060ce15cb53 INFO: 172.21.0.1:45902 - "POST /node/69408198575bc060ce15cb53 HTTP/1.0" 202 Accepted 2025-12-17 19:51:02,994 - lava_callback - INFO - Starting async job submit for node 69408198575bc060ce15cb53 2025-12-17 19:51:02,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:51:04,014 - lava_callback - INFO - Received callback for node 6940642b575bc060ce159c8d INFO: 172.21.0.1:45918 - "POST /node/6940642b575bc060ce159c8d HTTP/1.0" 202 Accepted 2025-12-17 19:51:04,022 - lava_callback - INFO - Starting async job submit for node 6940642b575bc060ce159c8d 2025-12-17 19:51:04,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:51:04,117 - lava_callback - INFO - Received callback for node 6940816d575bc060ce15cad7 INFO: 172.21.0.1:45924 - "POST /node/6940816d575bc060ce15cad7 HTTP/1.0" 202 Accepted 2025-12-17 19:51:04,124 - lava_callback - INFO - Starting async job submit for node 6940816d575bc060ce15cad7 2025-12-17 19:51:04,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:45938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48646 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:51:28,796 - lava_callback - INFO - Received callback for node 6940625e575bc060ce159ad6 INFO: 172.21.0.1:48656 - "POST /node/6940625e575bc060ce159ad6 HTTP/1.0" 202 Accepted 2025-12-17 19:51:28,802 - lava_callback - INFO - Starting async job submit for node 6940625e575bc060ce159ad6 2025-12-17 19:51:28,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:51:35,951 - lava_callback - INFO - Received callback for node 69406261575bc060ce159af1 INFO: 172.21.0.1:34748 - "POST /node/69406261575bc060ce159af1 HTTP/1.0" 202 Accepted 2025-12-17 19:51:35,958 - lava_callback - INFO - Starting async job submit for node 69406261575bc060ce159af1 2025-12-17 19:51:35,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:34750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49294 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:51:58,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c51e1e3c24e95681d663'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:51:58,317 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940c51e1e3c24e95681d663 2025-12-17 19:51:58,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 19:51:58,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408166575bc060ce15cac5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:51:58,920 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408166575bc060ce15cac5 2025-12-17 19:51:58,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:47366 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:52:03,068 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408198575bc060ce15cb53'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:52:03,068 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408198575bc060ce15cb53 2025-12-17 19:52:03,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 19:52:04,079 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940642b575bc060ce159c8d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:52:04,080 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940642b575bc060ce159c8d 2025-12-17 19:52:04,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 19:52:04,143 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816d575bc060ce15cad7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:52:04,144 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940816d575bc060ce15cad7 2025-12-17 19:52:04,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:47370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55118 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:52:28,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940625e575bc060ce159ad6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:52:28,868 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940625e575bc060ce159ad6 2025-12-17 19:52:28,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 19:52:36,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406261575bc060ce159af1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:52:36,016 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406261575bc060ce159af1 2025-12-17 19:52:36,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:50556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39238 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:52:58,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c51e1e3c24e95681d663'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:52:58,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408166575bc060ce15cac5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58232 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:53:00,215 - lava_callback - INFO - Received callback for node 69407240575bc060ce15ac1e 2025-12-17 19:53:00,221 - lava_callback - INFO - Starting async job submit for node 69407240575bc060ce15ac1e INFO: 172.21.0.1:58246 - "POST /node/69407240575bc060ce15ac1e HTTP/1.0" 202 Accepted 2025-12-17 19:53:00,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:53:00,357 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940c51e1e3c24e95681d663 2025-12-17 19:53:00,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 19:53:00,984 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408166575bc060ce15cac5 2025-12-17 19:53:00,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 19:53:03,120 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408198575bc060ce15cb53'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:53:04,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940642b575bc060ce159c8d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:53:04,175 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816d575bc060ce15cad7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:53:05,121 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408198575bc060ce15cb53 2025-12-17 19:53:05,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 19:53:06,124 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940642b575bc060ce159c8d 2025-12-17 19:53:06,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 19:53:06,176 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940816d575bc060ce15cad7 2025-12-17 19:53:06,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:58258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40850 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:53:28,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940625e575bc060ce159ad6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:53:30,934 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940625e575bc060ce159ad6 2025-12-17 19:53:30,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 19:53:36,051 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406261575bc060ce159af1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35514 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:53:38,052 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406261575bc060ce159af1 2025-12-17 19:53:38,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:39382 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:53:56,047 - lava_callback - INFO - Received callback for node 69405fe7575bc060ce1598dd 2025-12-17 19:53:56,053 - lava_callback - INFO - Starting async job submit for node 69405fe7575bc060ce1598dd 2025-12-17 19:53:56,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:57420 - "POST /node/69405fe7575bc060ce1598dd HTTP/1.0" 202 Accepted INFO: 172.21.0.1:57434 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:53:56,976 - lava_callback - INFO - Received callback for node 69406260575bc060ce159aee 2025-12-17 19:53:56,981 - lava_callback - INFO - Starting async job submit for node 69406260575bc060ce159aee INFO: 172.21.0.1:57448 - "POST /node/69406260575bc060ce159aee HTTP/1.0" 202 Accepted 2025-12-17 19:53:56,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:59338 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:54:00,302 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407240575bc060ce15ac1e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:54:00,302 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407240575bc060ce15ac1e 2025-12-17 19:54:00,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 19:54:00,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c51e1e3c24e95681d663'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:54:01,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408166575bc060ce15cac5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:54:04,420 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940c51e1e3c24e95681d663 2025-12-17 19:54:04,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 19:54:05,044 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408166575bc060ce15cac5 2025-12-17 19:54:05,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 19:54:05,170 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408198575bc060ce15cb53'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:54:06,191 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940642b575bc060ce159c8d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:54:06,259 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816d575bc060ce15cad7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59342 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:54:09,171 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408198575bc060ce15cb53 2025-12-17 19:54:09,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 19:54:10,192 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940642b575bc060ce159c8d 2025-12-17 19:54:10,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 19:54:10,260 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940816d575bc060ce15cad7 2025-12-17 19:54:10,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:33962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36390 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:54:31,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940625e575bc060ce159ad6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:54:35,003 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940625e575bc060ce159ad6 2025-12-17 19:54:35,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:33890 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:54:38,127 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406261575bc060ce159af1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:54:42,128 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406261575bc060ce159af1 2025-12-17 19:54:42,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:42974 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:54:48,584 - lava_callback - INFO - Received callback for node 694063c3575bc060ce159c18 2025-12-17 19:54:48,590 - lava_callback - INFO - Starting async job submit for node 694063c3575bc060ce159c18 INFO: 172.21.0.1:42976 - "POST /node/694063c3575bc060ce159c18 HTTP/1.0" 202 Accepted 2025-12-17 19:54:48,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:54:56,122 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69405fe7575bc060ce1598dd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:54:56,122 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69405fe7575bc060ce1598dd 2025-12-17 19:54:56,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:50760 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:54:56,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406260575bc060ce159aee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:54:57,000 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406260575bc060ce159aee 2025-12-17 19:54:57,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:45482 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:55:00,371 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407240575bc060ce15ac1e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:55:02,372 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407240575bc060ce15ac1e 2025-12-17 19:55:02,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 19:55:04,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c51e1e3c24e95681d663'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:55:05,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408166575bc060ce15cac5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45492 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:55:09,190 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408198575bc060ce15cb53'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:55:10,270 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940642b575bc060ce159c8d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:55:10,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816d575bc060ce15cad7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:55:12,471 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940c51e1e3c24e95681d663 2025-12-17 19:55:12,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 19:55:13,104 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408166575bc060ce15cac5 2025-12-17 19:55:13,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:36886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36894 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:55:17,191 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408198575bc060ce15cb53 2025-12-17 19:55:17,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 19:55:18,270 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940642b575bc060ce159c8d 2025-12-17 19:55:18,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 19:55:18,320 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940816d575bc060ce15cad7 2025-12-17 19:55:18,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:43704 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:55:35,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940625e575bc060ce159ad6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37092 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:55:36,509 - lava_callback - INFO - Received callback for node 69406ee8575bc060ce15a4e3 2025-12-17 19:55:36,516 - lava_callback - INFO - Starting async job submit for node 69406ee8575bc060ce15a4e3 INFO: 172.21.0.1:37094 - "POST /node/69406ee8575bc060ce15a4e3 HTTP/1.0" 202 Accepted 2025-12-17 19:55:36,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:55:42,031 - lava_callback - INFO - Received callback for node 69407203575bc060ce15ac00 2025-12-17 19:55:42,039 - lava_callback - INFO - Starting async job submit for node 69407203575bc060ce15ac00 INFO: 172.21.0.1:47902 - "POST /node/69407203575bc060ce15ac00 HTTP/1.0" 202 Accepted 2025-12-17 19:55:42,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:55:42,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406261575bc060ce159af1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:55:43,044 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940625e575bc060ce159ad6 2025-12-17 19:55:43,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:47914 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:55:48,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694063c3575bc060ce159c18'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:55:48,652 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694063c3575bc060ce159c18 2025-12-17 19:55:48,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 19:55:50,183 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406261575bc060ce159af1 2025-12-17 19:55:50,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 19:55:56,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69405fe7575bc060ce1598dd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56438 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:55:57,065 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406260575bc060ce159aee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:55:58,183 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69405fe7575bc060ce1598dd 2025-12-17 19:55:58,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 19:55:59,065 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406260575bc060ce159aee 2025-12-17 19:55:59,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:38034 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:56:02,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407240575bc060ce15ac1e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:56:06,412 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407240575bc060ce15ac1e 2025-12-17 19:56:06,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:38044 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:56:12,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c51e1e3c24e95681d663'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:56:13,167 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408166575bc060ce15cac5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36654 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:56:16,939 - lava_callback - INFO - Received callback for node 69406ee4575bc060ce15a4da 2025-12-17 19:56:16,945 - lava_callback - INFO - Starting async job submit for node 69406ee4575bc060ce15a4da INFO: 172.21.0.1:36664 - "POST /node/69406ee4575bc060ce15a4da HTTP/1.0" 202 Accepted 2025-12-17 19:56:16,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:56:17,267 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408198575bc060ce15cb53'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:56:18,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940642b575bc060ce159c8d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:56:18,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816d575bc060ce15cad7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60796 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:56:28,545 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940c51e1e3c24e95681d663 2025-12-17 19:56:28,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 19:56:29,168 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408166575bc060ce15cac5 2025-12-17 19:56:29,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 19:56:33,268 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408198575bc060ce15cb53 2025-12-17 19:56:33,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 19:56:34,342 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940642b575bc060ce159c8d 2025-12-17 19:56:34,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 19:56:34,384 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940816d575bc060ce15cad7 2025-12-17 19:56:34,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:50198 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:56:36,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ee8575bc060ce15a4e3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:56:36,578 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ee8575bc060ce15a4e3 2025-12-17 19:56:36,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 19:56:42,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407203575bc060ce15ac00'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:56:42,101 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407203575bc060ce15ac00 2025-12-17 19:56:42,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 19:56:43,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940625e575bc060ce159ad6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40562 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:56:48,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694063c3575bc060ce159c18'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:56:50,223 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406261575bc060ce159af1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:56:50,688 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694063c3575bc060ce159c18 2025-12-17 19:56:50,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:34970 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:56:58,230 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69405fe7575bc060ce1598dd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:56:59,103 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940625e575bc060ce159ad6 2025-12-17 19:56:59,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 19:56:59,165 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406260575bc060ce159aee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34392 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:57:02,230 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69405fe7575bc060ce1598dd 2025-12-17 19:57:02,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 19:57:03,165 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406260575bc060ce159aee 2025-12-17 19:57:03,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 19:57:06,224 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406261575bc060ce159af1 2025-12-17 19:57:06,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 19:57:06,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407240575bc060ce15ac1e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34402 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:57:14,436 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407240575bc060ce15ac1e 2025-12-17 19:57:14,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:50482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50490 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:57:16,986 - lava_callback - INFO - Received callback for node 6940680f575bc060ce159eb1 2025-12-17 19:57:16,992 - lava_callback - INFO - Starting async job submit for node 6940680f575bc060ce159eb1 2025-12-17 19:57:16,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:50506 - "POST /node/6940680f575bc060ce159eb1 HTTP/1.0" 202 Accepted 2025-12-17 19:57:17,057 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ee4575bc060ce15a4da'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:57:17,057 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ee4575bc060ce15a4da 2025-12-17 19:57:17,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 19:57:17,643 - lava_callback - INFO - Received callback for node 69406811575bc060ce159eb4 2025-12-17 19:57:17,650 - lava_callback - INFO - Starting async job submit for node 69406811575bc060ce159eb4 2025-12-17 19:57:17,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:50520 - "POST /node/69406811575bc060ce159eb4 HTTP/1.0" 202 Accepted 2025-12-17 19:57:18,299 - lava_callback - INFO - Received callback for node 69406813575bc060ce159eb7 INFO: 172.21.0.1:50532 - "POST /node/69406813575bc060ce159eb7 HTTP/1.0" 202 Accepted 2025-12-17 19:57:18,945 - lava_callback - INFO - Received callback for node 69406815575bc060ce159eba INFO: 172.21.0.1:50542 - "POST /node/69406815575bc060ce159eba HTTP/1.0" 202 Accepted 2025-12-17 19:57:19,592 - lava_callback - INFO - Received callback for node 69406817575bc060ce159ebd INFO: 172.21.0.1:50550 - "POST /node/69406817575bc060ce159ebd HTTP/1.0" 202 Accepted INFO: 172.21.0.1:53160 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:57:28,636 - lava_callback - ERROR - Error processing callback for node 6940c51e1e3c24e95681d663: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c51e1e3c24e95681d663 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c51e1e3c24e95681d663 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c51e1e3c24e95681d663 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 19:57:28,652 - lava_callback - INFO - Starting async job submit for node 69406813575bc060ce159eb7 2025-12-17 19:57:28,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:57:29,244 - lava_callback - ERROR - Error processing callback for node 69408166575bc060ce15cac5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408166575bc060ce15cac5 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408166575bc060ce15cac5 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408166575bc060ce15cac5 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 19:57:29,246 - lava_callback - INFO - Starting async job submit for node 69406815575bc060ce159eba 2025-12-17 19:57:29,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:57:33,364 - lava_callback - ERROR - Error processing callback for node 69408198575bc060ce15cb53: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408198575bc060ce15cb53 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408198575bc060ce15cb53 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408198575bc060ce15cb53 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 19:57:33,367 - lava_callback - INFO - Starting async job submit for node 69406817575bc060ce159ebd 2025-12-17 19:57:33,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:57:34,440 - lava_callback - ERROR - Error processing callback for node 6940642b575bc060ce159c8d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940642b575bc060ce159c8d (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940642b575bc060ce159c8d (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940642b575bc060ce159c8d (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 19:57:34,448 - lava_callback - ERROR - Error processing callback for node 6940816d575bc060ce15cad7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940816d575bc060ce15cad7 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940816d575bc060ce15cad7 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940816d575bc060ce15cad7 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:53190 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:57:36,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ee8575bc060ce15a4e3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:57:38,624 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ee8575bc060ce15a4e3 2025-12-17 19:57:38,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 19:57:42,159 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407203575bc060ce15ac00'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:57:44,160 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407203575bc060ce15ac00 2025-12-17 19:57:44,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:56374 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:57:50,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694063c3575bc060ce159c18'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:57:54,751 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694063c3575bc060ce159c18 2025-12-17 19:57:54,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:53198 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:57:57,028 - lava_callback - INFO - Received callback for node 6940683f575bc060ce159ee5 INFO: 172.21.0.1:53202 - "POST /node/6940683f575bc060ce159ee5 HTTP/1.0" 202 Accepted 2025-12-17 19:57:57,034 - lava_callback - INFO - Starting async job submit for node 6940683f575bc060ce159ee5 2025-12-17 19:57:57,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:57:57,704 - lava_callback - INFO - Received callback for node 69406842575bc060ce159ef0 INFO: 172.21.0.1:53212 - "POST /node/69406842575bc060ce159ef0 HTTP/1.0" 202 Accepted 2025-12-17 19:57:57,712 - lava_callback - INFO - Starting async job submit for node 69406842575bc060ce159ef0 2025-12-17 19:57:57,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:57:59,161 - lava_callback - ERROR - Error processing callback for node 6940625e575bc060ce159ad6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940625e575bc060ce159ad6 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940625e575bc060ce159ad6 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940625e575bc060ce159ad6 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:38144 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:58:02,254 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69405fe7575bc060ce1598dd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:58:03,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406260575bc060ce159aee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:58:06,288 - lava_callback - ERROR - Error processing callback for node 69406261575bc060ce159af1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406261575bc060ce159af1 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406261575bc060ce159af1 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406261575bc060ce159af1 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:38152 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:58:08,660 - lava_callback - INFO - Received callback for node 6940684c575bc060ce159f03 INFO: 172.21.0.1:38162 - "POST /node/6940684c575bc060ce159f03 HTTP/1.0" 202 Accepted 2025-12-17 19:58:08,666 - lava_callback - INFO - Starting async job submit for node 6940684c575bc060ce159f03 2025-12-17 19:58:08,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:58:09,320 - lava_callback - INFO - Received callback for node 6940684f575bc060ce159f06 INFO: 172.21.0.1:38166 - "POST /node/6940684f575bc060ce159f06 HTTP/1.0" 202 Accepted 2025-12-17 19:58:09,325 - lava_callback - INFO - Starting async job submit for node 6940684f575bc060ce159f06 2025-12-17 19:58:09,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:58:10,255 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69405fe7575bc060ce1598dd 2025-12-17 19:58:10,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 19:58:11,184 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406260575bc060ce159aee 2025-12-17 19:58:11,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 19:58:14,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407240575bc060ce15ac1e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34876 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:58:17,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940680f575bc060ce159eb1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:58:17,020 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940680f575bc060ce159eb1 2025-12-17 19:58:17,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 19:58:17,040 - lava_callback - INFO - Received callback for node 69406844575bc060ce159ef7 INFO: 172.21.0.1:34882 - "POST /node/69406844575bc060ce159ef7 HTTP/1.0" 202 Accepted 2025-12-17 19:58:17,091 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ee4575bc060ce15a4da'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:58:17,716 - lava_callback - INFO - Received callback for node 69406846575bc060ce159efa INFO: 172.21.0.1:34886 - "POST /node/69406846575bc060ce159efa HTTP/1.0" 202 Accepted 2025-12-17 19:58:17,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406811575bc060ce159eb4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:58:17,724 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406811575bc060ce159eb4 2025-12-17 19:58:17,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 19:58:19,092 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ee4575bc060ce15a4da 2025-12-17 19:58:19,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:43014 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:58:28,668 - lava_callback - INFO - Received callback for node 69406851575bc060ce159f09 INFO: 172.21.0.1:43018 - "POST /node/69406851575bc060ce159f09 HTTP/1.0" 202 Accepted 2025-12-17 19:58:28,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406813575bc060ce159eb7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:58:28,708 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406813575bc060ce159eb7 2025-12-17 19:58:28,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 19:58:29,290 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406815575bc060ce159eba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:58:29,291 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406815575bc060ce159eba 2025-12-17 19:58:29,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 19:58:29,327 - lava_callback - INFO - Received callback for node 69406853575bc060ce159f0c INFO: 172.21.0.1:43020 - "POST /node/69406853575bc060ce159f0c HTTP/1.0" 202 Accepted 2025-12-17 19:58:30,006 - lava_callback - INFO - Received callback for node 69406855575bc060ce159f0f INFO: 172.21.0.1:35920 - "POST /node/69406855575bc060ce159f0f HTTP/1.0" 202 Accepted 2025-12-17 19:58:30,520 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407240575bc060ce15ac1e 2025-12-17 19:58:30,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 19:58:33,437 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406817575bc060ce159ebd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:58:33,437 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406817575bc060ce159ebd 2025-12-17 19:58:33,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:35932 - "GET /node/693ae396fee8383e92b64194%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:35934 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:58:38,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ee8575bc060ce15a4e3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:58:42,677 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ee8575bc060ce15a4e3 2025-12-17 19:58:42,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 19:58:44,211 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407203575bc060ce15ac00'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50094 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:58:48,212 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407203575bc060ce15ac00 2025-12-17 19:58:48,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 19:58:54,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694063c3575bc060ce159c18'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44280 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:58:57,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940683f575bc060ce159ee5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:58:57,103 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940683f575bc060ce159ee5 2025-12-17 19:58:57,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 19:58:57,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406842575bc060ce159ef0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:58:57,786 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406842575bc060ce159ef0 2025-12-17 19:58:57,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:51230 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:59:02,788 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694063c3575bc060ce159c18 2025-12-17 19:59:02,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:51240 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:59:08,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940684c575bc060ce159f03'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:59:08,720 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940684c575bc060ce159f03 2025-12-17 19:59:08,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 19:59:09,391 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940684f575bc060ce159f06'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:59:09,392 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940684f575bc060ce159f06 2025-12-17 19:59:09,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 19:59:10,307 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69405fe7575bc060ce1598dd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:59:11,247 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406260575bc060ce159aee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55210 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:59:17,110 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940680f575bc060ce159eb1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:59:17,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406811575bc060ce159eb4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:59:19,110 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940680f575bc060ce159eb1 2025-12-17 19:59:19,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 19:59:19,159 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ee4575bc060ce15a4da'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:59:19,810 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406811575bc060ce159eb4 2025-12-17 19:59:19,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 19:59:23,160 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ee4575bc060ce15a4da 2025-12-17 19:59:23,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 19:59:26,308 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69405fe7575bc060ce1598dd 2025-12-17 19:59:26,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:34108 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:59:27,248 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406260575bc060ce159aee 2025-12-17 19:59:27,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 19:59:28,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406813575bc060ce159eb7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:59:29,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406815575bc060ce159eba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:59:30,560 - lava_callback - ERROR - Error processing callback for node 69407240575bc060ce15ac1e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407240575bc060ce15ac1e (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407240575bc060ce15ac1e (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407240575bc060ce15ac1e (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 19:59:30,562 - lava_callback - INFO - Starting async job submit for node 69406844575bc060ce159ef7 2025-12-17 19:59:30,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 19:59:30,757 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406813575bc060ce159eb7 2025-12-17 19:59:30,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 19:59:31,360 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406815575bc060ce159eba 2025-12-17 19:59:31,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 19:59:33,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406817575bc060ce159ebd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:59:35,488 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406817575bc060ce159ebd 2025-12-17 19:59:35,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:39018 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:59:42,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ee8575bc060ce15a4e3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41802 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:59:48,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407203575bc060ce15ac00'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:59:50,768 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ee8575bc060ce15a4e3 2025-12-17 19:59:50,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 19:59:56,284 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407203575bc060ce15ac00 2025-12-17 19:59:56,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:56068 - "GET / HTTP/1.0" 200 OK 2025-12-17 19:59:57,147 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940683f575bc060ce159ee5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:59:57,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406842575bc060ce159ef0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 19:59:59,147 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940683f575bc060ce159ee5 2025-12-17 19:59:59,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 19:59:59,815 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406842575bc060ce159ef0 2025-12-17 19:59:59,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:49946 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:00:02,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694063c3575bc060ce159c18'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49960 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:00:08,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940684c575bc060ce159f03'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:00:09,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940684f575bc060ce159f06'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:00:10,160 - lava_callback - INFO - Received callback for node 694066fb575bc060ce159df6 INFO: 172.21.0.1:54382 - "POST /node/694066fb575bc060ce159df6 HTTP/1.0" 202 Accepted 2025-12-17 20:00:10,748 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940684c575bc060ce159f03 2025-12-17 20:00:10,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:00:11,458 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940684f575bc060ce159f06 2025-12-17 20:00:11,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:54388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54396 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:00:17,073 - lava_callback - INFO - Received callback for node 69406b21575bc060ce15a0e6 INFO: 172.21.0.1:54400 - "POST /node/69406b21575bc060ce15a0e6 HTTP/1.0" 202 Accepted 2025-12-17 20:00:18,842 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694063c3575bc060ce159c18 2025-12-17 20:00:18,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:00:19,168 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940680f575bc060ce159eb1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:00:19,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406811575bc060ce159eb4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:00:23,168 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940680f575bc060ce159eb1 2025-12-17 20:00:23,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:00:23,223 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ee4575bc060ce15a4da'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:00:23,726 - lava_callback - INFO - Received callback for node 69407206575bc060ce15ac03 INFO: 172.21.0.1:43316 - "POST /node/69407206575bc060ce15ac03 HTTP/1.0" 202 Accepted 2025-12-17 20:00:23,876 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406811575bc060ce159eb4 2025-12-17 20:00:23,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:00:26,368 - lava_callback - ERROR - Error processing callback for node 69405fe7575bc060ce1598dd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69405fe7575bc060ce1598dd (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69405fe7575bc060ce1598dd (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69405fe7575bc060ce1598dd (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:00:26,370 - lava_callback - INFO - Starting async job submit for node 69406846575bc060ce159efa 2025-12-17 20:00:26,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:43324 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:00:27,312 - lava_callback - ERROR - Error processing callback for node 69406260575bc060ce159aee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406260575bc060ce159aee (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406260575bc060ce159aee (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406260575bc060ce159aee (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:00:27,314 - lava_callback - INFO - Starting async job submit for node 69406851575bc060ce159f09 2025-12-17 20:00:27,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:00:30,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406844575bc060ce159ef7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:00:30,613 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406844575bc060ce159ef7 2025-12-17 20:00:30,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:00:30,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406813575bc060ce159eb7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:00:31,224 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ee4575bc060ce15a4da 2025-12-17 20:00:31,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:00:31,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406815575bc060ce159eba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:00:34,813 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406813575bc060ce159eb7 2025-12-17 20:00:34,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:00:35,424 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406815575bc060ce159eba 2025-12-17 20:00:35,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:00:35,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406817575bc060ce159ebd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52172 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:00:39,580 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406817575bc060ce159ebd 2025-12-17 20:00:39,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:45212 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:00:50,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ee8575bc060ce15a4e3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:00:56,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407203575bc060ce15ac00'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55368 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:00:57,232 - lava_callback - INFO - Received callback for node 69406ef0575bc060ce15a4f5 INFO: 172.21.0.1:55374 - "POST /node/69406ef0575bc060ce15a4f5 HTTP/1.0" 202 Accepted 2025-12-17 20:00:59,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940683f575bc060ce159ee5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:00:59,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406842575bc060ce159ef0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49916 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:01:03,216 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940683f575bc060ce159ee5 2025-12-17 20:01:03,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:01:03,883 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406842575bc060ce159ef0 2025-12-17 20:01:03,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:49924 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:01:06,835 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ee8575bc060ce15a4e3 2025-12-17 20:01:06,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:01:10,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940684c575bc060ce159f03'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:01:11,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940684f575bc060ce159f06'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:01:12,352 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407203575bc060ce15ac00 2025-12-17 20:01:12,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:01:14,813 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940684c575bc060ce159f03 2025-12-17 20:01:14,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:44272 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:01:15,529 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940684f575bc060ce159f06 2025-12-17 20:01:15,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:44286 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:01:18,916 - lava_callback - ERROR - Error processing callback for node 694063c3575bc060ce159c18: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694063c3575bc060ce159c18 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694063c3575bc060ce159c18 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694063c3575bc060ce159c18 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:01:18,918 - lava_callback - INFO - Starting async job submit for node 69406853575bc060ce159f0c 2025-12-17 20:01:18,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:01:23,231 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940680f575bc060ce159eb1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:01:23,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406811575bc060ce159eb4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:01:26,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406846575bc060ce159efa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:01:26,420 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406846575bc060ce159efa 2025-12-17 20:01:26,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:58944 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:01:27,377 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406851575bc060ce159f09'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:01:27,378 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406851575bc060ce159f09 2025-12-17 20:01:27,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:01:30,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406844575bc060ce159ef7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:01:31,232 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940680f575bc060ce159eb1 2025-12-17 20:01:31,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:01:31,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ee4575bc060ce15a4da'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:01:31,927 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406811575bc060ce159eb4 2025-12-17 20:01:31,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:01:32,701 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406844575bc060ce159ef7 2025-12-17 20:01:32,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:01:34,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406813575bc060ce159eb7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:01:35,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406815575bc060ce159eba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45346 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:01:39,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406817575bc060ce159ebd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:01:42,903 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406813575bc060ce159eb7 2025-12-17 20:01:42,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:01:43,463 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406815575bc060ce159eba 2025-12-17 20:01:43,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:48436 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:01:47,285 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ee4575bc060ce15a4da 2025-12-17 20:01:47,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:01:47,664 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406817575bc060ce159ebd 2025-12-17 20:01:47,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53826 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:02:03,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940683f575bc060ce159ee5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:02:03,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406842575bc060ce159ef0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53832 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:02:06,880 - lava_callback - ERROR - Error processing callback for node 69406ee8575bc060ce15a4e3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ee8575bc060ce15a4e3 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ee8575bc060ce15a4e3 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ee8575bc060ce15a4e3 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:02:06,882 - lava_callback - INFO - Starting async job submit for node 69406855575bc060ce159f0f 2025-12-17 20:02:06,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:02:11,264 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940683f575bc060ce159ee5 2025-12-17 20:02:11,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:02:11,956 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406842575bc060ce159ef0 2025-12-17 20:02:11,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:02:12,407 - lava_callback - ERROR - Error processing callback for node 69407203575bc060ce15ac00: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407203575bc060ce15ac00 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407203575bc060ce15ac00 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407203575bc060ce15ac00 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:02:12,409 - lava_callback - INFO - Starting async job submit for node 694066fb575bc060ce159df6 2025-12-17 20:02:12,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:02:14,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940684c575bc060ce159f03'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33278 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:02:15,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940684f575bc060ce159f06'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33288 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:02:18,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406853575bc060ce159f0c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:02:18,984 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406853575bc060ce159f0c 2025-12-17 20:02:18,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:02:22,904 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940684c575bc060ce159f03 2025-12-17 20:02:22,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:02:23,600 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940684f575bc060ce159f06 2025-12-17 20:02:23,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:02:26,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406846575bc060ce159efa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51784 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:02:27,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406851575bc060ce159f09'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:02:28,504 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406846575bc060ce159efa 2025-12-17 20:02:28,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:02:29,409 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406851575bc060ce159f09 2025-12-17 20:02:29,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:02:31,297 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940680f575bc060ce159eb1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:02:32,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406811575bc060ce159eb4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:02:32,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406844575bc060ce159ef7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60842 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:02:36,764 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406844575bc060ce159ef7 2025-12-17 20:02:36,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:02:42,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406813575bc060ce159eb7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:02:43,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406815575bc060ce159eba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59818 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:02:47,298 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940680f575bc060ce159eb1 2025-12-17 20:02:47,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:02:47,375 - lava_callback - ERROR - Error processing callback for node 69406ee4575bc060ce15a4da: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ee4575bc060ce15a4da (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ee4575bc060ce15a4da (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ee4575bc060ce15a4da (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:02:47,377 - lava_callback - INFO - Starting async job submit for node 69406b21575bc060ce15a0e6 2025-12-17 20:02:47,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:02:47,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406817575bc060ce159ebd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:02:48,000 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406811575bc060ce159eb4 2025-12-17 20:02:48,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:54652 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:02:58,999 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406813575bc060ce159eb7 2025-12-17 20:02:58,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:02:59,545 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406815575bc060ce159eba 2025-12-17 20:02:59,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:35314 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:03:03,731 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406817575bc060ce159ebd 2025-12-17 20:03:03,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:35326 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:03:06,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406855575bc060ce159f0f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:03:06,948 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406855575bc060ce159f0f 2025-12-17 20:03:06,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:03:11,335 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940683f575bc060ce159ee5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:03:12,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406842575bc060ce159ef0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:03:12,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694066fb575bc060ce159df6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:03:12,521 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694066fb575bc060ce159df6 2025-12-17 20:03:12,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:47330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47340 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:03:19,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406853575bc060ce159f0c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:03:21,014 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406853575bc060ce159f0c 2025-12-17 20:03:21,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:03:22,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940684c575bc060ce159f03'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:03:23,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940684f575bc060ce159f06'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47658 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:03:27,335 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940683f575bc060ce159ee5 2025-12-17 20:03:27,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:03:28,030 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406842575bc060ce159ef0 2025-12-17 20:03:28,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:03:28,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406846575bc060ce159efa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:03:29,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406851575bc060ce159f09'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:03:32,552 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406846575bc060ce159efa 2025-12-17 20:03:32,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:03:33,438 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406851575bc060ce159f09 2025-12-17 20:03:33,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:39624 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:03:36,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406844575bc060ce159ef7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:03:38,928 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940684c575bc060ce159f03 2025-12-17 20:03:38,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:03:39,631 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940684f575bc060ce159f06 2025-12-17 20:03:39,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:03:44,835 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406844575bc060ce159ef7 2025-12-17 20:03:44,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:45854 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:03:47,358 - lava_callback - ERROR - Error processing callback for node 6940680f575bc060ce159eb1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940680f575bc060ce159eb1 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940680f575bc060ce159eb1 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940680f575bc060ce159eb1 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:03:47,360 - lava_callback - INFO - Starting async job submit for node 69407206575bc060ce15ac03 2025-12-17 20:03:47,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:03:47,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406b21575bc060ce15a0e6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:03:47,455 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406b21575bc060ce15a0e6 2025-12-17 20:03:47,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:03:48,064 - lava_callback - ERROR - Error processing callback for node 69406811575bc060ce159eb4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406811575bc060ce159eb4 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406811575bc060ce159eb4 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406811575bc060ce159eb4 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:03:48,066 - lava_callback - INFO - Starting async job submit for node 69406ef0575bc060ce15a4f5 2025-12-17 20:03:48,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:35038 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:03:59,076 - lava_callback - ERROR - Error processing callback for node 69406813575bc060ce159eb7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406813575bc060ce159eb7 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406813575bc060ce159eb7 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406813575bc060ce159eb7 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:03:59,613 - lava_callback - ERROR - Error processing callback for node 69406815575bc060ce159eba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406815575bc060ce159eba (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406815575bc060ce159eba (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406815575bc060ce159eba (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:58848 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:04:03,796 - lava_callback - ERROR - Error processing callback for node 69406817575bc060ce159ebd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406817575bc060ce159ebd (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406817575bc060ce159ebd (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406817575bc060ce159ebd (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:58860 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:04:07,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406855575bc060ce159f0f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:04:09,012 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406855575bc060ce159f0f 2025-12-17 20:04:09,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:04:12,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694066fb575bc060ce159df6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:04:14,574 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694066fb575bc060ce159df6 2025-12-17 20:04:14,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:39248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39252 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:04:21,077 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406853575bc060ce159f0c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:04:25,077 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406853575bc060ce159f0c 2025-12-17 20:04:25,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:46202 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:04:27,394 - lava_callback - ERROR - Error processing callback for node 6940683f575bc060ce159ee5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940683f575bc060ce159ee5 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940683f575bc060ce159ee5 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940683f575bc060ce159ee5 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:04:28,122 - lava_callback - ERROR - Error processing callback for node 69406842575bc060ce159ef0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406842575bc060ce159ef0 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406842575bc060ce159ef0 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406842575bc060ce159ef0 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:04:32,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406846575bc060ce159efa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:04:33,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406851575bc060ce159f09'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49660 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:04:38,968 - lava_callback - ERROR - Error processing callback for node 6940684c575bc060ce159f03: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940684c575bc060ce159f03 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940684c575bc060ce159f03 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940684c575bc060ce159f03 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:04:39,664 - lava_callback - ERROR - Error processing callback for node 6940684f575bc060ce159f06: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940684f575bc060ce159f06 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940684f575bc060ce159f06 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940684f575bc060ce159f06 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:04:40,560 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406846575bc060ce159efa 2025-12-17 20:04:40,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:04:41,493 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406851575bc060ce159f09 2025-12-17 20:04:41,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:04:44,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406844575bc060ce159ef7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41092 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:04:47,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407206575bc060ce15ac03'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:04:47,408 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407206575bc060ce15ac03 2025-12-17 20:04:47,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:04:47,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406b21575bc060ce15a0e6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:04:48,105 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef0575bc060ce15a4f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:04:48,106 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef0575bc060ce15a4f5 2025-12-17 20:04:48,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:04:49,472 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406b21575bc060ce15a0e6 2025-12-17 20:04:49,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:56300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53898 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:05:00,848 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406844575bc060ce159ef7 2025-12-17 20:05:00,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:53912 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:05:09,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406855575bc060ce159f0f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:05:13,071 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406855575bc060ce159f0f 2025-12-17 20:05:13,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:05:14,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694066fb575bc060ce159df6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37902 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:05:17,321 - lava_callback - INFO - Received callback for node 69406ef1575bc060ce15a4fc INFO: 172.21.0.1:37912 - "POST /node/69406ef1575bc060ce15a4fc HTTP/1.0" 202 Accepted 2025-12-17 20:05:17,329 - lava_callback - INFO - Starting async job submit for node 69406ef1575bc060ce15a4fc 2025-12-17 20:05:17,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:05:18,634 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694066fb575bc060ce159df6 2025-12-17 20:05:18,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:05:22,512 - lava_callback - INFO - Received callback for node 69407209575bc060ce15ac06 INFO: 172.21.0.1:59184 - "POST /node/69407209575bc060ce15ac06 HTTP/1.0" 202 Accepted 2025-12-17 20:05:22,520 - lava_callback - INFO - Starting async job submit for node 69407209575bc060ce15ac06 2025-12-17 20:05:22,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:05:25,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406853575bc060ce159f0c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59190 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:05:33,124 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406853575bc060ce159f0c 2025-12-17 20:05:33,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:56290 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:05:40,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406846575bc060ce159efa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:05:41,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406851575bc060ce159f09'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38358 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:05:47,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407206575bc060ce15ac03'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:05:48,185 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef0575bc060ce15a4f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:05:49,465 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407206575bc060ce15ac03 2025-12-17 20:05:49,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:05:49,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406b21575bc060ce15a0e6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:05:50,185 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef0575bc060ce15a4f5 2025-12-17 20:05:50,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:05:53,500 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406b21575bc060ce15a0e6 2025-12-17 20:05:53,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:05:55,138 - lava_callback - INFO - Received callback for node 694081d4575bc060ce15cbd3 INFO: 172.21.0.1:41774 - "POST /node/694081d4575bc060ce15cbd3 HTTP/1.0" 202 Accepted 2025-12-17 20:05:55,146 - lava_callback - INFO - Starting async job submit for node 694081d4575bc060ce15cbd3 2025-12-17 20:05:55,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:41784 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:05:56,602 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406846575bc060ce159efa 2025-12-17 20:05:56,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:05:57,604 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406851575bc060ce159f09 2025-12-17 20:05:57,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:50984 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:06:00,912 - lava_callback - ERROR - Error processing callback for node 69406844575bc060ce159ef7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406844575bc060ce159ef7 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406844575bc060ce159ef7 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406844575bc060ce159ef7 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:50996 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:06:13,159 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406855575bc060ce159f0f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:06:15,015 - lava_callback - INFO - Received callback for node 69408208575bc060ce15ccb8 INFO: 172.21.0.1:41306 - "POST /node/69408208575bc060ce15ccb8 HTTP/1.0" 202 Accepted 2025-12-17 20:06:15,024 - lava_callback - INFO - Starting async job submit for node 69408208575bc060ce15ccb8 2025-12-17 20:06:15,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:41308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41324 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:06:17,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef1575bc060ce15a4fc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:06:17,456 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef1575bc060ce15a4fc 2025-12-17 20:06:17,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:06:18,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694066fb575bc060ce159df6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:06:21,160 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406855575bc060ce159f0f 2025-12-17 20:06:21,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:06:22,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407209575bc060ce15ac06'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:06:22,560 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407209575bc060ce15ac06 2025-12-17 20:06:22,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:52500 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:06:26,686 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694066fb575bc060ce159df6 2025-12-17 20:06:26,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:06:29,683 - lava_callback - INFO - Received callback for node 6940830d575bc060ce15d202 INFO: 172.21.0.1:52510 - "POST /node/6940830d575bc060ce15d202 HTTP/1.0" 202 Accepted 2025-12-17 20:06:29,691 - lava_callback - INFO - Starting async job submit for node 6940830d575bc060ce15d202 2025-12-17 20:06:29,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:06:31,928 - lava_callback - INFO - Received callback for node 6940830f575bc060ce15d205 INFO: 172.21.0.1:42560 - "POST /node/6940830f575bc060ce15d205 HTTP/1.0" 202 Accepted 2025-12-17 20:06:31,936 - lava_callback - INFO - Starting async job submit for node 6940830f575bc060ce15d205 2025-12-17 20:06:31,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:06:33,151 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406853575bc060ce159f0c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51400 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:06:49,152 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406853575bc060ce159f0c 2025-12-17 20:06:49,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:06:49,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407206575bc060ce15ac03'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:06:50,223 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef0575bc060ce15a4f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:06:53,067 - lava_callback - INFO - Received callback for node 6940c5201e3c24e95681d66c INFO: 172.21.0.1:34284 - "POST /node/6940c5201e3c24e95681d66c HTTP/1.0" 202 Accepted 2025-12-17 20:06:53,074 - lava_callback - INFO - Starting async job submit for node 6940c5201e3c24e95681d66c 2025-12-17 20:06:53,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:06:53,520 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407206575bc060ce15ac03 2025-12-17 20:06:53,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:06:53,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406b21575bc060ce15a0e6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:06:54,224 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef0575bc060ce15a4f5 2025-12-17 20:06:54,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:06:55,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081d4575bc060ce15cbd3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:06:55,284 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694081d4575bc060ce15cbd3 2025-12-17 20:06:55,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:34298 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:06:56,656 - lava_callback - ERROR - Error processing callback for node 69406846575bc060ce159efa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406846575bc060ce159efa (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406846575bc060ce159efa (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406846575bc060ce159efa (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:06:57,688 - lava_callback - ERROR - Error processing callback for node 69406851575bc060ce159f09: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406851575bc060ce159f09 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406851575bc060ce159f09 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406851575bc060ce159f09 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:52332 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:07:01,568 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406b21575bc060ce15a0e6 2025-12-17 20:07:01,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:52334 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:07:15,149 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408208575bc060ce15ccb8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:07:15,150 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408208575bc060ce15ccb8 2025-12-17 20:07:15,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:39354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39358 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:07:17,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef1575bc060ce15a4fc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:07:19,503 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef1575bc060ce15a4fc 2025-12-17 20:07:19,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:07:21,205 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406855575bc060ce159f0f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:07:22,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407209575bc060ce15ac06'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:07:24,627 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407209575bc060ce15ac06 2025-12-17 20:07:24,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:48788 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:07:26,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694066fb575bc060ce159df6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:07:28,974 - lava_callback - INFO - Received callback for node 694081cd575bc060ce15cbc0 INFO: 172.21.0.1:48802 - "POST /node/694081cd575bc060ce15cbc0 HTTP/1.0" 202 Accepted 2025-12-17 20:07:28,983 - lava_callback - INFO - Starting async job submit for node 694081cd575bc060ce15cbc0 2025-12-17 20:07:29,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:07:29,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830d575bc060ce15d202'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:07:29,791 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940830d575bc060ce15d202 2025-12-17 20:07:29,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:07:32,079 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830f575bc060ce15d205'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:07:32,080 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940830f575bc060ce15d205 2025-12-17 20:07:32,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:59806 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:07:37,205 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406855575bc060ce159f0f 2025-12-17 20:07:37,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:07:42,736 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694066fb575bc060ce159df6 2025-12-17 20:07:42,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:57198 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:07:49,235 - lava_callback - ERROR - Error processing callback for node 69406853575bc060ce159f0c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406853575bc060ce159f0c (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406853575bc060ce159f0c (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406853575bc060ce159f0c (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:07:53,203 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5201e3c24e95681d66c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:07:53,204 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940c5201e3c24e95681d66c 2025-12-17 20:07:53,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:07:53,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407206575bc060ce15ac03'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:07:54,287 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef0575bc060ce15a4f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:07:55,371 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081d4575bc060ce15cbd3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36218 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:07:57,372 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694081d4575bc060ce15cbd3 2025-12-17 20:07:57,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:43232 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:08:01,584 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407206575bc060ce15ac03 2025-12-17 20:08:01,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:08:01,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406b21575bc060ce15a0e6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:08:02,288 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef0575bc060ce15a4f5 2025-12-17 20:08:02,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:08:04,636 - lava_callback - INFO - Received callback for node 6940758b575bc060ce15b026 INFO: 172.21.0.1:43238 - "POST /node/6940758b575bc060ce15b026 HTTP/1.0" 202 Accepted 2025-12-17 20:08:04,642 - lava_callback - INFO - Starting async job submit for node 6940758b575bc060ce15b026 2025-12-17 20:08:04,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:43240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51034 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:08:15,199 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408208575bc060ce15ccb8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51048 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:08:17,200 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408208575bc060ce15ccb8 2025-12-17 20:08:17,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:08:17,638 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406b21575bc060ce15a0e6 2025-12-17 20:08:17,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:08:19,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef1575bc060ce15a4fc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:08:23,577 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef1575bc060ce15a4fc 2025-12-17 20:08:23,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:08:24,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407209575bc060ce15ac06'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44574 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:08:28,686 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407209575bc060ce15ac06 2025-12-17 20:08:28,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:08:29,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081cd575bc060ce15cbc0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:08:29,104 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694081cd575bc060ce15cbc0 2025-12-17 20:08:29,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:08:29,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830d575bc060ce15d202'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:08:31,843 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940830d575bc060ce15d202 2025-12-17 20:08:31,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:08:32,152 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830f575bc060ce15d205'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:08:34,153 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940830f575bc060ce15d205 2025-12-17 20:08:34,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:59760 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:08:37,292 - lava_callback - ERROR - Error processing callback for node 69406855575bc060ce159f0f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406855575bc060ce159f0f (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406855575bc060ce159f0f (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406855575bc060ce159f0f (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:08:42,793 - lava_callback - ERROR - Error processing callback for node 694066fb575bc060ce159df6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694066fb575bc060ce159df6 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694066fb575bc060ce159df6 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694066fb575bc060ce159df6 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:60682 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:08:53,224 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5201e3c24e95681d66c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:08:55,224 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940c5201e3c24e95681d66c 2025-12-17 20:08:55,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:36608 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:08:57,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081d4575bc060ce15cbd3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49896 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:09:01,423 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694081d4575bc060ce15cbd3 2025-12-17 20:09:01,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:09:01,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407206575bc060ce15ac03'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:09:02,352 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef0575bc060ce15a4f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:09:04,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940758b575bc060ce15b026'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:09:04,720 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940758b575bc060ce15b026 2025-12-17 20:09:04,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:49912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52618 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:09:17,231 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408208575bc060ce15ccb8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:09:17,646 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407206575bc060ce15ac03 2025-12-17 20:09:17,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:09:17,712 - lava_callback - ERROR - Error processing callback for node 69406b21575bc060ce15a0e6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406b21575bc060ce15a0e6 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406b21575bc060ce15a0e6 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406b21575bc060ce15a0e6 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:09:18,352 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef0575bc060ce15a4f5 2025-12-17 20:09:18,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:09:21,232 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408208575bc060ce15ccb8 2025-12-17 20:09:21,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:09:23,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef1575bc060ce15a4fc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53244 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:09:28,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407209575bc060ce15ac06'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:09:29,149 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081cd575bc060ce15cbc0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:09:31,149 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694081cd575bc060ce15cbc0 2025-12-17 20:09:31,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:09:31,624 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef1575bc060ce15a4fc 2025-12-17 20:09:31,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:09:31,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830d575bc060ce15d202'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:09:34,201 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830f575bc060ce15d205'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:09:35,919 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940830d575bc060ce15d202 2025-12-17 20:09:35,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:49314 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:09:36,775 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407209575bc060ce15ac06 2025-12-17 20:09:36,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:09:38,201 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940830f575bc060ce15d205 2025-12-17 20:09:38,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:53988 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:09:55,278 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5201e3c24e95681d66c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45830 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:09:59,278 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940c5201e3c24e95681d66c 2025-12-17 20:09:59,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:50190 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:10:01,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081d4575bc060ce15cbd3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:10:04,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940758b575bc060ce15b026'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50200 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:10:06,783 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940758b575bc060ce15b026 2025-12-17 20:10:06,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:10:09,482 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694081d4575bc060ce15cbd3 2025-12-17 20:10:09,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:10:09,792 - lava_callback - INFO - Received callback for node 69406ef3575bc060ce15a503 INFO: 172.21.0.1:42518 - "POST /node/69406ef3575bc060ce15a503 HTTP/1.0" 202 Accepted 2025-12-17 20:10:09,804 - lava_callback - INFO - Starting async job submit for node 69406ef3575bc060ce15a503 2025-12-17 20:10:10,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:10:12,469 - lava_callback - INFO - Received callback for node 69406d3c575bc060ce15a348 INFO: 172.21.0.1:42524 - "POST /node/69406d3c575bc060ce15a348 HTTP/1.0" 202 Accepted 2025-12-17 20:10:12,517 - lava_callback - INFO - Starting async job submit for node 69406d3c575bc060ce15a348 2025-12-17 20:10:12,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:10:13,073 - lava_callback - INFO - Received callback for node 6940730c575bc060ce15ad92 INFO: 172.21.0.1:42532 - "POST /node/6940730c575bc060ce15ad92 HTTP/1.0" 202 Accepted 2025-12-17 20:10:13,081 - lava_callback - INFO - Starting async job submit for node 6940730c575bc060ce15ad92 2025-12-17 20:10:13,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:42546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42548 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:10:17,708 - lava_callback - ERROR - Error processing callback for node 69407206575bc060ce15ac03: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407206575bc060ce15ac03 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407206575bc060ce15ac03 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407206575bc060ce15ac03 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:10:18,421 - lava_callback - ERROR - Error processing callback for node 69406ef0575bc060ce15a4f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef0575bc060ce15a4f5 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef0575bc060ce15a4f5 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef0575bc060ce15a4f5 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:10:21,302 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408208575bc060ce15ccb8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47470 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:10:29,302 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408208575bc060ce15ccb8 2025-12-17 20:10:29,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:10:31,203 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081cd575bc060ce15cbc0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:10:31,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef1575bc060ce15a4fc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:10:35,204 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694081cd575bc060ce15cbc0 2025-12-17 20:10:35,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:10:35,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830d575bc060ce15d202'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40004 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:10:36,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407209575bc060ce15ac06'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:10:38,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830f575bc060ce15d205'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:10:43,984 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940830d575bc060ce15d202 2025-12-17 20:10:43,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:10:46,265 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940830f575bc060ce15d205 2025-12-17 20:10:46,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:34302 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:10:47,712 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef1575bc060ce15a4fc 2025-12-17 20:10:47,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:10:51,796 - lava_callback - INFO - Received callback for node 69406ee2575bc060ce15a4d7 INFO: 172.21.0.1:44260 - "POST /node/69406ee2575bc060ce15a4d7 HTTP/1.0" 202 Accepted 2025-12-17 20:10:51,802 - lava_callback - INFO - Starting async job submit for node 69406ee2575bc060ce15a4d7 2025-12-17 20:10:51,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:10:52,852 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407209575bc060ce15ac06 2025-12-17 20:10:52,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:44276 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:10:59,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5201e3c24e95681d66c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51624 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:11:06,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940758b575bc060ce15b026'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:11:07,355 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940c5201e3c24e95681d66c 2025-12-17 20:11:07,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:11:09,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081d4575bc060ce15cbd3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:11:10,235 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef3575bc060ce15a503'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:11:10,235 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef3575bc060ce15a503 2025-12-17 20:11:10,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:11:10,853 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940758b575bc060ce15b026 2025-12-17 20:11:10,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:11:12,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406d3c575bc060ce15a348'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:11:12,552 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406d3c575bc060ce15a348 2025-12-17 20:11:12,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:11:13,163 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940730c575bc060ce15ad92'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:11:13,164 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940730c575bc060ce15ad92 2025-12-17 20:11:13,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:42642 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:11:16,153 - lava_callback - INFO - Received callback for node 6940758a575bc060ce15b023 INFO: 172.21.0.1:42648 - "POST /node/6940758a575bc060ce15b023 HTTP/1.0" 202 Accepted 2025-12-17 20:11:16,160 - lava_callback - INFO - Starting async job submit for node 6940758a575bc060ce15b023 2025-12-17 20:11:16,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:42650 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:11:17,479 - lava_callback - INFO - Received callback for node 6940758a575bc060ce15b020 INFO: 172.21.0.1:42664 - "POST /node/6940758a575bc060ce15b020 HTTP/1.0" 202 Accepted 2025-12-17 20:11:17,485 - lava_callback - INFO - Starting async job submit for node 6940758a575bc060ce15b020 2025-12-17 20:11:17,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:11:25,563 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694081d4575bc060ce15cbd3 2025-12-17 20:11:25,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:33160 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:11:29,389 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408208575bc060ce15ccb8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:11:35,269 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081cd575bc060ce15cbc0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47720 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:11:43,269 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694081cd575bc060ce15cbc0 2025-12-17 20:11:43,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:11:43,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830d575bc060ce15d202'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:11:45,390 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408208575bc060ce15ccb8 2025-12-17 20:11:45,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:11:46,288 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830f575bc060ce15d205'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43828 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:11:47,760 - lava_callback - ERROR - Error processing callback for node 69406ef1575bc060ce15a4fc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef1575bc060ce15a4fc (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef1575bc060ce15a4fc (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef1575bc060ce15a4fc (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:11:51,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ee2575bc060ce15a4d7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:11:51,873 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ee2575bc060ce15a4d7 2025-12-17 20:11:51,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:11:52,900 - lava_callback - ERROR - Error processing callback for node 69407209575bc060ce15ac06: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407209575bc060ce15ac06 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407209575bc060ce15ac06 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407209575bc060ce15ac06 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:53224 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:11:59,996 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940830d575bc060ce15d202 2025-12-17 20:11:59,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:45196 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:12:02,288 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940830f575bc060ce15d205 2025-12-17 20:12:02,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:45210 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:12:07,390 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5201e3c24e95681d66c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:12:10,287 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef3575bc060ce15a503'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:12:10,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940758b575bc060ce15b026'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:12:12,288 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef3575bc060ce15a503 2025-12-17 20:12:12,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:12:12,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406d3c575bc060ce15a348'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:12:13,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940730c575bc060ce15ad92'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:12:14,592 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406d3c575bc060ce15a348 2025-12-17 20:12:14,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:52172 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:12:15,195 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940730c575bc060ce15ad92 2025-12-17 20:12:15,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:12:16,241 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940758a575bc060ce15b023'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:12:16,242 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940758a575bc060ce15b023 2025-12-17 20:12:16,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:52176 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:12:17,473 - lava_callback - INFO - Received callback for node 69406ef5575bc060ce15a509 INFO: 172.21.0.1:52178 - "POST /node/69406ef5575bc060ce15a509 HTTP/1.0" 202 Accepted 2025-12-17 20:12:17,480 - lava_callback - INFO - Starting async job submit for node 69406ef5575bc060ce15a509 2025-12-17 20:12:17,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940758a575bc060ce15b020'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:12:17,580 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940758a575bc060ce15b020 2025-12-17 20:12:17,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:12:17,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:12:18,896 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940758b575bc060ce15b026 2025-12-17 20:12:18,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:12:23,391 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940c5201e3c24e95681d66c 2025-12-17 20:12:23,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:12:25,640 - lava_callback - ERROR - Error processing callback for node 694081d4575bc060ce15cbd3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081d4575bc060ce15cbd3 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081d4575bc060ce15cbd3 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081d4575bc060ce15cbd3 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:44456 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:12:29,455 - lava_callback - INFO - Received callback for node 69406a4d575bc060ce159ffb INFO: 172.21.0.1:44464 - "POST /node/69406a4d575bc060ce159ffb HTTP/1.0" 202 Accepted 2025-12-17 20:12:29,462 - lava_callback - INFO - Starting async job submit for node 69406a4d575bc060ce159ffb 2025-12-17 20:12:29,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:49028 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:12:42,763 - lava_callback - INFO - Received callback for node 69407740575bc060ce15b2d8 INFO: 172.21.0.1:34710 - "POST /node/69407740575bc060ce15b2d8 HTTP/1.0" 202 Accepted 2025-12-17 20:12:42,769 - lava_callback - INFO - Starting async job submit for node 69407740575bc060ce15b2d8 2025-12-17 20:12:42,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:12:43,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081cd575bc060ce15cbc0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:12:43,959 - lava_callback - INFO - Received callback for node 6940773d575bc060ce15b2d5 INFO: 172.21.0.1:34724 - "POST /node/6940773d575bc060ce15b2d5 HTTP/1.0" 202 Accepted 2025-12-17 20:12:43,965 - lava_callback - INFO - Starting async job submit for node 6940773d575bc060ce15b2d5 2025-12-17 20:12:43,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:12:45,451 - lava_callback - ERROR - Error processing callback for node 69408208575bc060ce15ccb8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408208575bc060ce15ccb8 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408208575bc060ce15ccb8 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408208575bc060ce15ccb8 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:34736 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:12:51,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ee2575bc060ce15a4d7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:12:53,952 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ee2575bc060ce15a4d7 2025-12-17 20:12:53,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:56500 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:12:59,357 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694081cd575bc060ce15cbc0 2025-12-17 20:12:59,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:13:00,072 - lava_callback - ERROR - Error processing callback for node 6940830d575bc060ce15d202: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940830d575bc060ce15d202 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940830d575bc060ce15d202 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940830d575bc060ce15d202 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:50972 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:13:02,374 - lava_callback - ERROR - Error processing callback for node 6940830f575bc060ce15d205: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940830f575bc060ce15d205 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940830f575bc060ce15d205 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940830f575bc060ce15d205 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:50980 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:13:12,372 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef3575bc060ce15a503'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:13:14,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406d3c575bc060ce15a348'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45138 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:13:15,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940730c575bc060ce15ad92'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:13:16,294 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940758a575bc060ce15b023'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:13:16,373 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef3575bc060ce15a503 2025-12-17 20:13:16,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:45154 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:13:17,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940758a575bc060ce15b020'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:13:17,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef5575bc060ce15a509'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:13:17,652 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef5575bc060ce15a509 2025-12-17 20:13:17,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:13:18,295 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940758a575bc060ce15b023 2025-12-17 20:13:18,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:13:18,650 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406d3c575bc060ce15a348 2025-12-17 20:13:18,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:13:18,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940758b575bc060ce15b026'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:13:19,244 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940730c575bc060ce15ad92 2025-12-17 20:13:19,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:13:19,647 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940758a575bc060ce15b020 2025-12-17 20:13:19,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:13:23,448 - lava_callback - ERROR - Error processing callback for node 6940c5201e3c24e95681d66c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5201e3c24e95681d66c (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5201e3c24e95681d66c (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5201e3c24e95681d66c (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:42230 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:13:29,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406a4d575bc060ce159ffb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:13:29,604 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406a4d575bc060ce159ffb 2025-12-17 20:13:29,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:13:34,960 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940758b575bc060ce15b026 2025-12-17 20:13:34,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:39654 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:13:42,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407740575bc060ce15b2d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:13:42,844 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407740575bc060ce15b2d8 2025-12-17 20:13:42,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:13:44,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940773d575bc060ce15b2d5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:13:44,000 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940773d575bc060ce15b2d5 2025-12-17 20:13:44,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:38128 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:13:54,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ee2575bc060ce15a4d7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:32992 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:13:58,008 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ee2575bc060ce15a4d7 2025-12-17 20:13:58,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:13:59,394 - lava_callback - ERROR - Error processing callback for node 694081cd575bc060ce15cbc0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081cd575bc060ce15cbc0 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081cd575bc060ce15cbc0 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081cd575bc060ce15cbc0 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:44002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38888 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:14:16,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef3575bc060ce15a503'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38904 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:14:17,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef5575bc060ce15a509'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:14:18,378 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940758a575bc060ce15b023'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:14:18,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406d3c575bc060ce15a348'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:14:19,312 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940730c575bc060ce15ad92'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:14:19,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940758a575bc060ce15b020'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:14:19,748 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef5575bc060ce15a509 2025-12-17 20:14:19,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:14:22,378 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940758a575bc060ce15b023 2025-12-17 20:14:22,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:14:23,705 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940758a575bc060ce15b020 2025-12-17 20:14:23,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:14:24,444 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef3575bc060ce15a503 2025-12-17 20:14:24,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:55344 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:14:26,679 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406d3c575bc060ce15a348 2025-12-17 20:14:26,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:14:27,312 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940730c575bc060ce15ad92 2025-12-17 20:14:27,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:14:29,551 - lava_callback - INFO - Received callback for node 69406ef6575bc060ce15a50c INFO: 172.21.0.1:55348 - "POST /node/69406ef6575bc060ce15a50c HTTP/1.0" 202 Accepted 2025-12-17 20:14:29,558 - lava_callback - INFO - Starting async job submit for node 69406ef6575bc060ce15a50c 2025-12-17 20:14:29,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:14:29,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406a4d575bc060ce159ffb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:14:31,666 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406a4d575bc060ce159ffb 2025-12-17 20:14:31,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:14:31,833 - lava_callback - INFO - Received callback for node 694071a1575bc060ce15ab44 INFO: 172.21.0.1:44380 - "POST /node/694071a1575bc060ce15ab44 HTTP/1.0" 202 Accepted 2025-12-17 20:14:31,839 - lava_callback - INFO - Starting async job submit for node 694071a1575bc060ce15ab44 2025-12-17 20:14:31,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:14:35,018 - lava_callback - ERROR - Error processing callback for node 6940758b575bc060ce15b026: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940758b575bc060ce15b026 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940758b575bc060ce15b026 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940758b575bc060ce15b026 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:44392 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:14:40,481 - lava_callback - INFO - Received callback for node 69407742575bc060ce15b2db INFO: 172.21.0.1:46380 - "POST /node/69407742575bc060ce15b2db HTTP/1.0" 202 Accepted 2025-12-17 20:14:40,486 - lava_callback - INFO - Starting async job submit for node 69407742575bc060ce15b2db 2025-12-17 20:14:40,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:14:42,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407740575bc060ce15b2d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:14:44,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940773d575bc060ce15b2d5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:14:44,889 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407740575bc060ce15b2d8 2025-12-17 20:14:44,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:14:46,010 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940773d575bc060ce15b2d5 2025-12-17 20:14:46,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:46396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45732 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:14:58,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ee2575bc060ce15a4d7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51714 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:15:02,220 - lava_callback - INFO - Received callback for node 69407512575bc060ce15afc7 INFO: 172.21.0.1:51718 - "POST /node/69407512575bc060ce15afc7 HTTP/1.0" 202 Accepted 2025-12-17 20:15:02,227 - lava_callback - INFO - Starting async job submit for node 69407512575bc060ce15afc7 2025-12-17 20:15:02,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:15:05,661 - lava_callback - INFO - Received callback for node 69407778575bc060ce15b305 INFO: 172.21.0.1:51732 - "POST /node/69407778575bc060ce15b305 HTTP/1.0" 202 Accepted 2025-12-17 20:15:05,667 - lava_callback - INFO - Starting async job submit for node 69407778575bc060ce15b305 2025-12-17 20:15:05,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:15:06,022 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ee2575bc060ce15a4d7 2025-12-17 20:15:06,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:51734 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:15:06,865 - lava_callback - INFO - Received callback for node 69407775575bc060ce15b302 INFO: 172.21.0.1:51748 - "POST /node/69407775575bc060ce15b302 HTTP/1.0" 202 Accepted 2025-12-17 20:15:06,871 - lava_callback - INFO - Starting async job submit for node 69407775575bc060ce15b302 2025-12-17 20:15:06,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:36408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36410 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:15:19,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef5575bc060ce15a509'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:15:22,425 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940758a575bc060ce15b023'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:15:23,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940758a575bc060ce15b020'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:15:23,807 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef5575bc060ce15a509 2025-12-17 20:15:23,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:15:24,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef3575bc060ce15a503'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48340 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:15:26,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406d3c575bc060ce15a348'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:15:27,379 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940730c575bc060ce15ad92'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:15:29,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef6575bc060ce15a50c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:15:29,620 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef6575bc060ce15a50c 2025-12-17 20:15:29,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:15:30,425 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940758a575bc060ce15b023 2025-12-17 20:15:30,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:15:31,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406a4d575bc060ce159ffb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:15:31,768 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940758a575bc060ce15b020 2025-12-17 20:15:31,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:15:31,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071a1575bc060ce15ab44'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:15:31,896 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694071a1575bc060ce15ab44 2025-12-17 20:15:31,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:15:35,734 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406a4d575bc060ce159ffb 2025-12-17 20:15:35,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:48810 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:15:40,471 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef3575bc060ce15a503 2025-12-17 20:15:40,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:15:40,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407742575bc060ce15b2db'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:15:40,592 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407742575bc060ce15b2db 2025-12-17 20:15:40,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:15:42,715 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406d3c575bc060ce15a348 2025-12-17 20:15:42,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:15:43,380 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940730c575bc060ce15ad92 2025-12-17 20:15:43,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:15:44,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407740575bc060ce15b2d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:15:46,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940773d575bc060ce15b2d5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54288 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:15:48,946 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407740575bc060ce15b2d8 2025-12-17 20:15:48,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:15:50,096 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940773d575bc060ce15b2d5 2025-12-17 20:15:50,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:51692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49418 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:16:02,318 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407512575bc060ce15afc7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:16:02,318 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407512575bc060ce15afc7 2025-12-17 20:16:02,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:16:05,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407778575bc060ce15b305'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:16:05,760 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407778575bc060ce15b305 2025-12-17 20:16:05,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:16:06,064 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ee2575bc060ce15a4d7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49434 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:16:06,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407775575bc060ce15b302'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:16:06,952 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407775575bc060ce15b302 2025-12-17 20:16:06,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:35210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35222 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:16:20,448 - lava_callback - INFO - Received callback for node 6940777d575bc060ce15b30b INFO: 172.21.0.1:33006 - "POST /node/6940777d575bc060ce15b30b HTTP/1.0" 202 Accepted 2025-12-17 20:16:22,064 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ee2575bc060ce15a4d7 2025-12-17 20:16:22,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:16:23,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef5575bc060ce15a509'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33012 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:16:29,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef6575bc060ce15a50c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:16:30,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940758a575bc060ce15b023'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:16:31,674 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef6575bc060ce15a50c 2025-12-17 20:16:31,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:16:31,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940758a575bc060ce15b020'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:16:31,872 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef5575bc060ce15a509 2025-12-17 20:16:31,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:16:31,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071a1575bc060ce15ab44'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:16:33,988 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694071a1575bc060ce15ab44 2025-12-17 20:16:33,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:16:35,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406a4d575bc060ce159ffb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57778 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:16:37,666 - lava_callback - INFO - Received callback for node 69406ef8575bc060ce15a50f INFO: 172.21.0.1:57790 - "POST /node/69406ef8575bc060ce15a50f HTTP/1.0" 202 Accepted 2025-12-17 20:16:40,506 - lava_callback - ERROR - Error processing callback for node 69406ef3575bc060ce15a503: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef3575bc060ce15a503 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef3575bc060ce15a503 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef3575bc060ce15a503 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:16:40,508 - lava_callback - INFO - Starting async job submit for node 6940777d575bc060ce15b30b 2025-12-17 20:16:40,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:16:40,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407742575bc060ce15b2db'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:16:42,659 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407742575bc060ce15b2db 2025-12-17 20:16:42,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:16:42,768 - lava_callback - ERROR - Error processing callback for node 69406d3c575bc060ce15a348: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406d3c575bc060ce15a348 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406d3c575bc060ce15a348 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406d3c575bc060ce15a348 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:16:42,770 - lava_callback - INFO - Starting async job submit for node 69406ef8575bc060ce15a50f 2025-12-17 20:16:42,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:16:43,444 - lava_callback - ERROR - Error processing callback for node 6940730c575bc060ce15ad92: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940730c575bc060ce15ad92 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940730c575bc060ce15ad92 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940730c575bc060ce15ad92 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:16:43,796 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406a4d575bc060ce159ffb 2025-12-17 20:16:43,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:16:46,488 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940758a575bc060ce15b023 2025-12-17 20:16:46,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:45542 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:16:47,802 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940758a575bc060ce15b020 2025-12-17 20:16:47,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:16:49,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407740575bc060ce15b2d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:16:50,159 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940773d575bc060ce15b2d5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48170 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:16:57,013 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407740575bc060ce15b2d8 2025-12-17 20:16:57,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:16:58,160 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940773d575bc060ce15b2d5 2025-12-17 20:16:58,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:47502 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:17:02,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407512575bc060ce15afc7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:17:02,947 - lava_callback - INFO - Received callback for node 694079de575bc060ce15b75f INFO: 172.21.0.1:47518 - "POST /node/694079de575bc060ce15b75f HTTP/1.0" 202 Accepted 2025-12-17 20:17:02,953 - lava_callback - INFO - Starting async job submit for node 694079de575bc060ce15b75f 2025-12-17 20:17:02,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:17:04,134 - lava_callback - INFO - Received callback for node 6940793c575bc060ce15b6a1 INFO: 172.21.0.1:47532 - "POST /node/6940793c575bc060ce15b6a1 HTTP/1.0" 202 Accepted 2025-12-17 20:17:04,404 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407512575bc060ce15afc7 2025-12-17 20:17:04,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:17:05,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407778575bc060ce15b305'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47538 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:17:06,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407775575bc060ce15b302'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:17:07,852 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407778575bc060ce15b305 2025-12-17 20:17:07,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:17:08,992 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407775575bc060ce15b302 2025-12-17 20:17:08,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:47542 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:45662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45678 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:17:22,117 - lava_callback - ERROR - Error processing callback for node 69406ee2575bc060ce15a4d7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ee2575bc060ce15a4d7 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ee2575bc060ce15a4d7 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ee2575bc060ce15a4d7 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:17:22,119 - lava_callback - INFO - Starting async job submit for node 6940793c575bc060ce15b6a1 2025-12-17 20:17:22,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:42506 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:17:31,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef6575bc060ce15a50c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:17:31,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef5575bc060ce15a509'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:17:34,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071a1575bc060ce15ab44'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:17:35,752 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef6575bc060ce15a50c 2025-12-17 20:17:35,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:58538 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:17:38,011 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694071a1575bc060ce15ab44 2025-12-17 20:17:38,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:17:40,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940777d575bc060ce15b30b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:17:40,534 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940777d575bc060ce15b30b 2025-12-17 20:17:40,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:17:42,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407742575bc060ce15b2db'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:17:42,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef8575bc060ce15a50f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:17:42,891 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef8575bc060ce15a50f 2025-12-17 20:17:42,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:17:43,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406a4d575bc060ce159ffb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58414 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:17:46,576 - lava_callback - ERROR - Error processing callback for node 6940758a575bc060ce15b023: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940758a575bc060ce15b023 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940758a575bc060ce15b023 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940758a575bc060ce15b023 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:17:46,682 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407742575bc060ce15b2db 2025-12-17 20:17:46,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:17:47,860 - lava_callback - ERROR - Error processing callback for node 6940758a575bc060ce15b020: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940758a575bc060ce15b020 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940758a575bc060ce15b020 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940758a575bc060ce15b020 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:17:47,916 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef5575bc060ce15a509 2025-12-17 20:17:47,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:37278 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:17:57,065 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407740575bc060ce15b2d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:17:58,248 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940773d575bc060ce15b2d5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:17:59,856 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406a4d575bc060ce159ffb 2025-12-17 20:17:59,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:42512 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:18:03,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694079de575bc060ce15b75f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:18:03,034 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694079de575bc060ce15b75f 2025-12-17 20:18:03,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:18:04,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407512575bc060ce15afc7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42528 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:18:07,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407778575bc060ce15b305'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:18:08,493 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407512575bc060ce15afc7 2025-12-17 20:18:08,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:18:09,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407775575bc060ce15b302'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:18:11,912 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407778575bc060ce15b305 2025-12-17 20:18:11,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:18:13,046 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407775575bc060ce15b302 2025-12-17 20:18:13,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:18:13,065 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407740575bc060ce15b2d8 2025-12-17 20:18:13,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:18:14,248 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940773d575bc060ce15b2d5 2025-12-17 20:18:14,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:36240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36248 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:18:22,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940793c575bc060ce15b6a1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:18:22,200 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940793c575bc060ce15b6a1 2025-12-17 20:18:22,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:56066 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:18:28,529 - lava_callback - INFO - Received callback for node 694079e1575bc060ce15b769 INFO: 172.21.0.1:56072 - "POST /node/694079e1575bc060ce15b769 HTTP/1.0" 202 Accepted 2025-12-17 20:18:28,535 - lava_callback - INFO - Starting async job submit for node 694079e1575bc060ce15b769 2025-12-17 20:18:28,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:18:35,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef6575bc060ce15a50c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58628 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:18:38,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071a1575bc060ce15ab44'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:18:40,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940777d575bc060ce15b30b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:18:42,613 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940777d575bc060ce15b30b 2025-12-17 20:18:42,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:18:42,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef8575bc060ce15a50f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:18:43,818 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef6575bc060ce15a50c 2025-12-17 20:18:43,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:18:44,952 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef8575bc060ce15a50f 2025-12-17 20:18:44,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:18:46,049 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694071a1575bc060ce15ab44 2025-12-17 20:18:46,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:38320 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:18:46,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407742575bc060ce15b2db'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:18:47,952 - lava_callback - ERROR - Error processing callback for node 69406ef5575bc060ce15a509: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef5575bc060ce15a509 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef5575bc060ce15a509 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef5575bc060ce15a509 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:18:48,007 - lava_callback - INFO - Received callback for node 69407be8575bc060ce15bc1a INFO: 172.21.0.1:38322 - "POST /node/69407be8575bc060ce15bc1a HTTP/1.0" 202 Accepted 2025-12-17 20:18:48,014 - lava_callback - INFO - Starting async job submit for node 69407be8575bc060ce15bc1a 2025-12-17 20:18:48,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:18:49,601 - lava_callback - INFO - Received callback for node 69406ef9575bc060ce15a512 INFO: 172.21.0.1:38334 - "POST /node/69406ef9575bc060ce15a512 HTTP/1.0" 202 Accepted 2025-12-17 20:18:49,608 - lava_callback - INFO - Starting async job submit for node 69406ef9575bc060ce15a512 2025-12-17 20:18:49,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:18:54,774 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407742575bc060ce15b2db 2025-12-17 20:18:54,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:59730 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:18:57,432 - lava_callback - INFO - Received callback for node 69407be5575bc060ce15bc13 INFO: 172.21.0.1:59736 - "POST /node/69407be5575bc060ce15bc13 HTTP/1.0" 202 Accepted 2025-12-17 20:18:59,921 - lava_callback - ERROR - Error processing callback for node 69406a4d575bc060ce159ffb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406a4d575bc060ce159ffb (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406a4d575bc060ce159ffb (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406a4d575bc060ce159ffb (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:18:59,924 - lava_callback - INFO - Starting async job submit for node 69407be5575bc060ce15bc13 2025-12-17 20:18:59,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:42836 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:19:03,075 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694079de575bc060ce15b75f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:19:05,076 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694079de575bc060ce15b75f 2025-12-17 20:19:05,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:42848 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:19:08,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407512575bc060ce15afc7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:19:11,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407778575bc060ce15b305'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:19:13,111 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407775575bc060ce15b302'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:19:13,112 - lava_callback - ERROR - Error processing callback for node 69407740575bc060ce15b2d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407740575bc060ce15b2d8 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407740575bc060ce15b2d8 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407740575bc060ce15b2d8 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:19:14,288 - lava_callback - ERROR - Error processing callback for node 6940773d575bc060ce15b2d5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940773d575bc060ce15b2d5 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940773d575bc060ce15b2d5 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940773d575bc060ce15b2d5 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:43808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43820 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:19:16,581 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407512575bc060ce15afc7 2025-12-17 20:19:16,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:19:19,956 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407778575bc060ce15b305 2025-12-17 20:19:19,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:19:21,112 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407775575bc060ce15b302 2025-12-17 20:19:21,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:19:22,216 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940793c575bc060ce15b6a1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:19:24,216 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940793c575bc060ce15b6a1 2025-12-17 20:19:24,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:58350 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:19:28,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694079e1575bc060ce15b769'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:19:28,616 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694079e1575bc060ce15b769 2025-12-17 20:19:28,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:41350 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:19:42,528 - lava_callback - INFO - Received callback for node 69407606575bc060ce15b06b INFO: 172.21.0.1:44946 - "POST /node/69407606575bc060ce15b06b HTTP/1.0" 202 Accepted 2025-12-17 20:19:42,536 - lava_callback - INFO - Starting async job submit for node 69407606575bc060ce15b06b 2025-12-17 20:19:42,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:19:42,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940777d575bc060ce15b30b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:19:43,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef6575bc060ce15a50c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:19:44,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef8575bc060ce15a50f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:19:46,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071a1575bc060ce15ab44'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44958 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:19:46,688 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940777d575bc060ce15b30b 2025-12-17 20:19:46,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:19:48,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be8575bc060ce15bc1a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:19:48,104 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407be8575bc060ce15bc1a 2025-12-17 20:19:48,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:19:48,983 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef8575bc060ce15a50f 2025-12-17 20:19:48,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:19:49,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef9575bc060ce15a512'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:19:49,670 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef9575bc060ce15a512 2025-12-17 20:19:49,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:19:54,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407742575bc060ce15b2db'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47270 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:19:59,868 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef6575bc060ce15a50c 2025-12-17 20:19:59,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:19:59,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be5575bc060ce15bc13'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:19:59,996 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407be5575bc060ce15bc13 2025-12-17 20:19:59,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:58132 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:20:02,118 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694071a1575bc060ce15ab44 2025-12-17 20:20:02,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:20:05,106 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694079de575bc060ce15b75f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58138 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:20:09,106 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694079de575bc060ce15b75f 2025-12-17 20:20:09,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:20:10,834 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407742575bc060ce15b2db 2025-12-17 20:20:10,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:20:12,005 - lava_callback - INFO - Received callback for node 69407c84575bc060ce15be30 INFO: 172.21.0.1:40754 - "POST /node/69407c84575bc060ce15be30 HTTP/1.0" 202 Accepted 2025-12-17 20:20:12,011 - lava_callback - INFO - Starting async job submit for node 69407c84575bc060ce15be30 2025-12-17 20:20:12,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:20:13,202 - lava_callback - INFO - Received callback for node 69407c8e575bc060ce15be63 INFO: 172.21.0.1:40766 - "POST /node/69407c8e575bc060ce15be63 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:40768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40772 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:20:16,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407512575bc060ce15afc7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:20:17,972 - lava_callback - INFO - Received callback for node 69406ef4575bc060ce15a506 INFO: 172.21.0.1:40778 - "POST /node/69406ef4575bc060ce15a506 HTTP/1.0" 202 Accepted 2025-12-17 20:20:20,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407778575bc060ce15b305'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:20:21,185 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407775575bc060ce15b302'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:20:23,150 - lava_callback - INFO - Received callback for node 69407bea575bc060ce15bc1d INFO: 172.21.0.1:47670 - "POST /node/69407bea575bc060ce15bc1d HTTP/1.0" 202 Accepted 2025-12-17 20:20:24,279 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940793c575bc060ce15b6a1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47686 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:20:28,280 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940793c575bc060ce15b6a1 2025-12-17 20:20:28,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:20:28,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694079e1575bc060ce15b769'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:20:30,672 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694079e1575bc060ce15b769 2025-12-17 20:20:30,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:20:32,631 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407512575bc060ce15afc7 2025-12-17 20:20:32,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:20:36,026 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407778575bc060ce15b305 2025-12-17 20:20:36,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:40796 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:20:37,186 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407775575bc060ce15b302 2025-12-17 20:20:37,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:20:42,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407606575bc060ce15b06b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:20:42,639 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407606575bc060ce15b06b 2025-12-17 20:20:42,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:50732 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:20:46,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940777d575bc060ce15b30b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:20:48,203 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be8575bc060ce15bc1a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:20:49,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef8575bc060ce15a50f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:20:49,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef9575bc060ce15a512'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:20:50,204 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407be8575bc060ce15bc1a 2025-12-17 20:20:50,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:20:51,736 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef9575bc060ce15a512 2025-12-17 20:20:51,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:20:54,703 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940777d575bc060ce15b30b 2025-12-17 20:20:54,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:52664 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:20:57,072 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef8575bc060ce15a50f 2025-12-17 20:20:57,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:20:59,899 - lava_callback - ERROR - Error processing callback for node 69406ef6575bc060ce15a50c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef6575bc060ce15a50c (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef6575bc060ce15a50c (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef6575bc060ce15a50c (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:20:59,902 - lava_callback - INFO - Starting async job submit for node 69407c8e575bc060ce15be63 2025-12-17 20:20:59,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:21:00,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be5575bc060ce15bc13'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48976 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:21:02,034 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407be5575bc060ce15bc13 2025-12-17 20:21:02,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:21:02,191 - lava_callback - ERROR - Error processing callback for node 694071a1575bc060ce15ab44: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694071a1575bc060ce15ab44 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694071a1575bc060ce15ab44 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694071a1575bc060ce15ab44 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:21:02,193 - lava_callback - INFO - Starting async job submit for node 69406ef4575bc060ce15a506 2025-12-17 20:21:02,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:48986 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:21:09,167 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694079de575bc060ce15b75f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:21:10,108 - lava_callback - INFO - Received callback for node 694083d5575bc060ce15d50f INFO: 172.21.0.1:38084 - "POST /node/694083d5575bc060ce15d50f HTTP/1.0" 202 Accepted 2025-12-17 20:21:10,852 - lava_callback - ERROR - Error processing callback for node 69407742575bc060ce15b2db: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407742575bc060ce15b2db (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407742575bc060ce15b2db (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407742575bc060ce15b2db (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:21:10,854 - lava_callback - INFO - Starting async job submit for node 69407bea575bc060ce15bc1d 2025-12-17 20:21:10,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:21:12,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c84575bc060ce15be30'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:21:12,116 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407c84575bc060ce15be30 2025-12-17 20:21:12,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:38096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38102 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:21:17,168 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694079de575bc060ce15b75f 2025-12-17 20:21:17,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:42544 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:21:28,328 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940793c575bc060ce15b6a1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:21:30,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694079e1575bc060ce15b769'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:21:31,662 - lava_callback - INFO - Received callback for node 69408452575bc060ce15d719 INFO: 172.21.0.1:44258 - "POST /node/69408452575bc060ce15d719 HTTP/1.0" 202 Accepted 2025-12-17 20:21:32,720 - lava_callback - ERROR - Error processing callback for node 69407512575bc060ce15afc7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407512575bc060ce15afc7 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407512575bc060ce15afc7 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407512575bc060ce15afc7 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:21:32,722 - lava_callback - INFO - Starting async job submit for node 694083d5575bc060ce15d50f 2025-12-17 20:21:32,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:21:34,704 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694079e1575bc060ce15b769 2025-12-17 20:21:34,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:21:36,068 - lava_callback - ERROR - Error processing callback for node 69407778575bc060ce15b305: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407778575bc060ce15b305 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407778575bc060ce15b305 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407778575bc060ce15b305 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:21:36,071 - lava_callback - INFO - Starting async job submit for node 69408452575bc060ce15d719 2025-12-17 20:21:36,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:21:36,328 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940793c575bc060ce15b6a1 2025-12-17 20:21:36,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:44266 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:21:37,253 - lava_callback - ERROR - Error processing callback for node 69407775575bc060ce15b302: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407775575bc060ce15b302 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407775575bc060ce15b302 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407775575bc060ce15b302 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:21:42,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407606575bc060ce15b06b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:21:44,704 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407606575bc060ce15b06b 2025-12-17 20:21:44,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:42830 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:21:48,724 - lava_callback - INFO - Received callback for node 69408519575bc060ce15da7a INFO: 172.21.0.1:42842 - "POST /node/69408519575bc060ce15da7a HTTP/1.0" 202 Accepted 2025-12-17 20:21:48,733 - lava_callback - INFO - Starting async job submit for node 69408519575bc060ce15da7a 2025-12-17 20:21:48,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:21:50,294 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be8575bc060ce15bc1a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:21:51,199 - lava_callback - INFO - Received callback for node 69407c90575bc060ce15be66 INFO: 172.21.0.1:55082 - "POST /node/69407c90575bc060ce15be66 HTTP/1.0" 202 Accepted 2025-12-17 20:21:51,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef9575bc060ce15a512'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:21:54,295 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407be8575bc060ce15bc1a 2025-12-17 20:21:54,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:21:54,416 - lava_callback - INFO - Received callback for node 694084b1575bc060ce15d884 INFO: 172.21.0.1:55084 - "POST /node/694084b1575bc060ce15d884 HTTP/1.0" 202 Accepted 2025-12-17 20:21:54,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940777d575bc060ce15b30b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:21:55,799 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef9575bc060ce15a512 2025-12-17 20:21:55,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:55086 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:21:57,139 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef8575bc060ce15a50f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:22:00,174 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c8e575bc060ce15be63'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:22:00,175 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407c8e575bc060ce15be63 2025-12-17 20:22:00,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:60312 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:22:02,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be5575bc060ce15bc13'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:22:02,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef4575bc060ce15a506'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:22:02,317 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef4575bc060ce15a506 2025-12-17 20:22:02,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:22:06,096 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407be5575bc060ce15bc13 2025-12-17 20:22:06,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:60314 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:22:10,746 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940777d575bc060ce15b30b 2025-12-17 20:22:10,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:22:10,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bea575bc060ce15bc1d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:22:10,937 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407bea575bc060ce15bc1d 2025-12-17 20:22:10,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:22:12,163 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c84575bc060ce15be30'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:22:13,140 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef8575bc060ce15a50f 2025-12-17 20:22:13,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:22:14,164 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407c84575bc060ce15be30 2025-12-17 20:22:14,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:42338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42346 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:22:17,221 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694079de575bc060ce15b75f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46924 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:22:32,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d5575bc060ce15d50f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:22:32,835 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694083d5575bc060ce15d50f 2025-12-17 20:22:32,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:22:33,221 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694079de575bc060ce15b75f 2025-12-17 20:22:33,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:22:34,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694079e1575bc060ce15b769'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:22:36,180 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408452575bc060ce15d719'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:22:36,181 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408452575bc060ce15d719 2025-12-17 20:22:36,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:22:36,367 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940793c575bc060ce15b6a1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42604 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:22:42,755 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694079e1575bc060ce15b769 2025-12-17 20:22:42,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:22:44,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407606575bc060ce15b06b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45674 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:22:48,748 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407606575bc060ce15b06b 2025-12-17 20:22:48,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:22:48,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408519575bc060ce15da7a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:22:48,805 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408519575bc060ce15da7a 2025-12-17 20:22:48,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:22:52,368 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940793c575bc060ce15b6a1 2025-12-17 20:22:52,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:22:54,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be8575bc060ce15bc1a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:22:55,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef9575bc060ce15a512'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39766 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:23:00,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c8e575bc060ce15be63'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56614 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:23:02,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef4575bc060ce15a506'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:23:02,421 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407c8e575bc060ce15be63 2025-12-17 20:23:02,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:23:02,528 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407be8575bc060ce15bc1a 2025-12-17 20:23:02,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:23:03,828 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef9575bc060ce15a512 2025-12-17 20:23:03,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:23:04,394 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef4575bc060ce15a506 2025-12-17 20:23:04,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:23:06,300 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be5575bc060ce15bc13'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56630 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:23:10,422 - lava_callback - INFO - Received callback for node 69406efd575bc060ce15a51c INFO: 172.21.0.1:57440 - "POST /node/69406efd575bc060ce15a51c HTTP/1.0" 202 Accepted 2025-12-17 20:23:10,816 - lava_callback - ERROR - Error processing callback for node 6940777d575bc060ce15b30b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940777d575bc060ce15b30b (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940777d575bc060ce15b30b (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940777d575bc060ce15b30b (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:23:10,818 - lava_callback - INFO - Starting async job submit for node 69407c90575bc060ce15be66 2025-12-17 20:23:10,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:23:11,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bea575bc060ce15bc1d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:23:13,008 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407bea575bc060ce15bc1d 2025-12-17 20:23:13,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:23:13,147 - lava_callback - ERROR - Error processing callback for node 69406ef8575bc060ce15a50f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef8575bc060ce15a50f (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef8575bc060ce15a50f (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef8575bc060ce15a50f (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:23:13,149 - lava_callback - INFO - Starting async job submit for node 694084b1575bc060ce15d884 2025-12-17 20:23:13,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:23:14,300 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407be5575bc060ce15bc13 2025-12-17 20:23:14,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:23:14,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c84575bc060ce15be30'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57456 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:23:18,394 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407c84575bc060ce15be30 2025-12-17 20:23:18,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:51404 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:23:32,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d5575bc060ce15d50f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:23:33,296 - lava_callback - ERROR - Error processing callback for node 694079de575bc060ce15b75f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694079de575bc060ce15b75f (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694079de575bc060ce15b75f (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694079de575bc060ce15b75f (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:23:33,298 - lava_callback - INFO - Starting async job submit for node 69406efd575bc060ce15a51c 2025-12-17 20:23:33,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:23:34,903 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694083d5575bc060ce15d50f 2025-12-17 20:23:34,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:23:36,231 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408452575bc060ce15d719'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55512 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:23:38,231 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408452575bc060ce15d719 2025-12-17 20:23:38,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:23:42,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694079e1575bc060ce15b769'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35834 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:23:48,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407606575bc060ce15b06b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:23:48,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408519575bc060ce15da7a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:23:50,867 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408519575bc060ce15da7a 2025-12-17 20:23:50,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:23:52,423 - lava_callback - ERROR - Error processing callback for node 6940793c575bc060ce15b6a1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940793c575bc060ce15b6a1 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940793c575bc060ce15b6a1 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940793c575bc060ce15b6a1 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:45650 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:23:56,820 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407606575bc060ce15b06b 2025-12-17 20:23:56,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:23:58,821 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694079e1575bc060ce15b769 2025-12-17 20:23:58,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:51518 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:24:02,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c8e575bc060ce15be63'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:24:02,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be8575bc060ce15bc1a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:24:03,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef9575bc060ce15a512'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:24:04,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef4575bc060ce15a506'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:24:06,480 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407c8e575bc060ce15be63 2025-12-17 20:24:06,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:51526 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:24:08,456 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef4575bc060ce15a506 2025-12-17 20:24:08,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:24:10,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c90575bc060ce15be66'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:24:10,937 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407c90575bc060ce15be66 2025-12-17 20:24:10,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:24:13,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bea575bc060ce15bc1d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:24:13,271 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b1575bc060ce15d884'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:24:13,271 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694084b1575bc060ce15d884 2025-12-17 20:24:13,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:24:14,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be5575bc060ce15bc13'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54046 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:24:17,072 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407bea575bc060ce15bc1d 2025-12-17 20:24:17,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:24:17,932 - lava_callback - INFO - Received callback for node 69406efb575bc060ce15a518 INFO: 172.21.0.1:54048 - "POST /node/69406efb575bc060ce15a518 HTTP/1.0" 202 Accepted 2025-12-17 20:24:17,938 - lava_callback - INFO - Starting async job submit for node 69406efb575bc060ce15a518 2025-12-17 20:24:17,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:24:18,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c84575bc060ce15be30'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:24:18,599 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407be8575bc060ce15bc1a 2025-12-17 20:24:18,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:24:19,906 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef9575bc060ce15a512 2025-12-17 20:24:19,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:24:21,808 - lava_callback - INFO - Received callback for node 6940777a575bc060ce15b308 INFO: 172.21.0.1:34954 - "POST /node/6940777a575bc060ce15b308 HTTP/1.0" 202 Accepted 2025-12-17 20:24:25,786 - lava_callback - INFO - Received callback for node 6940852f575bc060ce15daaf INFO: 172.21.0.1:34960 - "POST /node/6940852f575bc060ce15daaf HTTP/1.0" 202 Accepted 2025-12-17 20:24:26,490 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407c84575bc060ce15be30 2025-12-17 20:24:26,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:34974 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:24:30,343 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407be5575bc060ce15bc13 2025-12-17 20:24:30,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:24:33,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efd575bc060ce15a51c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:24:33,394 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406efd575bc060ce15a51c 2025-12-17 20:24:33,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:24:34,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d5575bc060ce15d50f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55716 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:24:38,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408452575bc060ce15d719'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:24:38,992 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694083d5575bc060ce15d50f 2025-12-17 20:24:38,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:24:42,320 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408452575bc060ce15d719 2025-12-17 20:24:42,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:45504 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:24:50,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408519575bc060ce15da7a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:24:54,874 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408519575bc060ce15da7a 2025-12-17 20:24:54,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:59320 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:24:56,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407606575bc060ce15b06b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:24:58,860 - lava_callback - ERROR - Error processing callback for node 694079e1575bc060ce15b769: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694079e1575bc060ce15b769 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694079e1575bc060ce15b769 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694079e1575bc060ce15b769 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:24:58,863 - lava_callback - INFO - Starting async job submit for node 6940777a575bc060ce15b308 2025-12-17 20:24:58,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:57106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57122 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:25:06,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c8e575bc060ce15be63'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:25:08,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef4575bc060ce15a506'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:25:11,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c90575bc060ce15be66'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:25:12,884 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407606575bc060ce15b06b 2025-12-17 20:25:12,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:25:13,021 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407c90575bc060ce15be66 2025-12-17 20:25:13,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:25:13,340 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b1575bc060ce15d884'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:25:14,538 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407c8e575bc060ce15be63 2025-12-17 20:25:14,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:35052 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:25:15,340 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694084b1575bc060ce15d884 2025-12-17 20:25:15,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:25:16,480 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef4575bc060ce15a506 2025-12-17 20:25:16,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:35062 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:25:17,131 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bea575bc060ce15bc1d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:25:18,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efb575bc060ce15a518'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:25:18,060 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406efb575bc060ce15a518 2025-12-17 20:25:18,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:25:18,661 - lava_callback - ERROR - Error processing callback for node 69407be8575bc060ce15bc1a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407be8575bc060ce15bc1a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407be8575bc060ce15bc1a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407be8575bc060ce15bc1a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:25:18,663 - lava_callback - INFO - Starting async job submit for node 6940852f575bc060ce15daaf 2025-12-17 20:25:18,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:25:19,963 - lava_callback - ERROR - Error processing callback for node 69406ef9575bc060ce15a512: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef9575bc060ce15a512 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef9575bc060ce15a512 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef9575bc060ce15a512 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:25:25,132 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407bea575bc060ce15bc1d 2025-12-17 20:25:25,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:58276 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:25:26,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c84575bc060ce15be30'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:25:30,404 - lava_callback - ERROR - Error processing callback for node 69407be5575bc060ce15bc13: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407be5575bc060ce15bc13 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407be5575bc060ce15bc13 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407be5575bc060ce15bc13 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:25:33,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efd575bc060ce15a51c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:25:35,448 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406efd575bc060ce15a51c 2025-12-17 20:25:35,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:33506 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:25:39,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d5575bc060ce15d50f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:25:42,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408452575bc060ce15d719'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:25:42,536 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407c84575bc060ce15be30 2025-12-17 20:25:42,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:45180 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:25:47,032 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694083d5575bc060ce15d50f 2025-12-17 20:25:47,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:25:50,409 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408452575bc060ce15d719 2025-12-17 20:25:50,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:25:54,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408519575bc060ce15da7a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59194 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:25:58,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940777a575bc060ce15b308'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:25:58,960 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940777a575bc060ce15b308 2025-12-17 20:25:58,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:41762 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:26:02,940 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408519575bc060ce15da7a 2025-12-17 20:26:02,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:41766 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:26:12,943 - lava_callback - ERROR - Error processing callback for node 69407606575bc060ce15b06b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407606575bc060ce15b06b (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407606575bc060ce15b06b (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407606575bc060ce15b06b (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:26:13,083 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c90575bc060ce15be66'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:26:14,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c8e575bc060ce15be63'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35974 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:26:15,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b1575bc060ce15d884'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35984 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:26:16,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406ef4575bc060ce15a506'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:26:17,084 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407c90575bc060ce15be66 2025-12-17 20:26:17,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:26:17,959 - lava_callback - INFO - Received callback for node 69406efa575bc060ce15a515 INFO: 172.21.0.1:36000 - "POST /node/69406efa575bc060ce15a515 HTTP/1.0" 202 Accepted 2025-12-17 20:26:17,966 - lava_callback - INFO - Starting async job submit for node 69406efa575bc060ce15a515 2025-12-17 20:26:18,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:26:18,138 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efb575bc060ce15a518'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:26:18,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852f575bc060ce15daaf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:26:18,793 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940852f575bc060ce15daaf 2025-12-17 20:26:18,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:26:19,394 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694084b1575bc060ce15d884 2025-12-17 20:26:19,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:26:20,139 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406efb575bc060ce15a518 2025-12-17 20:26:20,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:26:25,145 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bea575bc060ce15bc1d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34150 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:26:30,576 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407c8e575bc060ce15be63 2025-12-17 20:26:30,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:26:32,553 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406ef4575bc060ce15a506 2025-12-17 20:26:32,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:26:35,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efd575bc060ce15a51c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40914 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:26:39,495 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406efd575bc060ce15a51c 2025-12-17 20:26:39,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:26:41,145 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407bea575bc060ce15bc1d 2025-12-17 20:26:41,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:26:42,581 - lava_callback - ERROR - Error processing callback for node 69407c84575bc060ce15be30: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c84575bc060ce15be30 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c84575bc060ce15be30 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c84575bc060ce15be30 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:53228 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:26:47,087 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d5575bc060ce15d50f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:26:50,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408452575bc060ce15d719'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36104 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:26:59,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940777a575bc060ce15b308'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44452 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:27:01,026 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940777a575bc060ce15b308 2025-12-17 20:27:01,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:27:02,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408519575bc060ce15da7a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:27:03,088 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694083d5575bc060ce15d50f 2025-12-17 20:27:03,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:27:06,476 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408452575bc060ce15d719 2025-12-17 20:27:06,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:44462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56330 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:27:17,141 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c90575bc060ce15be66'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:27:18,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efa575bc060ce15a515'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:27:18,073 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406efa575bc060ce15a515 2025-12-17 20:27:18,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:27:18,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852f575bc060ce15daaf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:27:18,984 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408519575bc060ce15da7a 2025-12-17 20:27:18,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:27:19,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b1575bc060ce15d884'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:27:20,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efb575bc060ce15a518'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:27:20,832 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940852f575bc060ce15daaf 2025-12-17 20:27:20,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:27:24,179 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406efb575bc060ce15a518 2025-12-17 20:27:24,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:27:25,142 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407c90575bc060ce15be66 2025-12-17 20:27:25,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:57978 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:27:27,445 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694084b1575bc060ce15d884 2025-12-17 20:27:27,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:27:29,832 - lava_callback - INFO - Received callback for node 69406f1f575bc060ce15a6b3 INFO: 172.21.0.1:47054 - "POST /node/69406f1f575bc060ce15a6b3 HTTP/1.0" 202 Accepted 2025-12-17 20:27:29,839 - lava_callback - INFO - Starting async job submit for node 69406f1f575bc060ce15a6b3 2025-12-17 20:27:29,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:27:30,629 - lava_callback - ERROR - Error processing callback for node 69407c8e575bc060ce15be63: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c8e575bc060ce15be63 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c8e575bc060ce15be63 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c8e575bc060ce15be63 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:27:32,604 - lava_callback - ERROR - Error processing callback for node 69406ef4575bc060ce15a506: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef4575bc060ce15a506 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef4575bc060ce15a506 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406ef4575bc060ce15a506 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:47066 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:27:39,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efd575bc060ce15a51c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:27:41,183 - lava_callback - ERROR - Error processing callback for node 69407bea575bc060ce15bc1d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bea575bc060ce15bc1d (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bea575bc060ce15bc1d (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bea575bc060ce15bc1d (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:46388 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:27:47,549 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406efd575bc060ce15a51c 2025-12-17 20:27:47,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:42266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59686 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:28:01,092 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940777a575bc060ce15b308'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:28:03,139 - lava_callback - ERROR - Error processing callback for node 694083d5575bc060ce15d50f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083d5575bc060ce15d50f (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083d5575bc060ce15d50f (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083d5575bc060ce15d50f (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:28:05,093 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940777a575bc060ce15b308 2025-12-17 20:28:05,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:59696 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:28:06,539 - lava_callback - ERROR - Error processing callback for node 69408452575bc060ce15d719: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408452575bc060ce15d719 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408452575bc060ce15d719 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408452575bc060ce15d719 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:54202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54216 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:28:18,139 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efa575bc060ce15a515'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:28:19,045 - lava_callback - ERROR - Error processing callback for node 69408519575bc060ce15da7a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408519575bc060ce15da7a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408519575bc060ce15da7a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408519575bc060ce15da7a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:28:20,139 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406efa575bc060ce15a515 2025-12-17 20:28:20,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:28:20,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852f575bc060ce15daaf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:28:24,247 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efb575bc060ce15a518'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:28:24,922 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940852f575bc060ce15daaf 2025-12-17 20:28:24,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:28:25,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c90575bc060ce15be66'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45572 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:28:27,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b1575bc060ce15d884'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:28:27,919 - lava_callback - INFO - Received callback for node 69406f5c575bc060ce15a83e INFO: 172.21.0.1:45586 - "POST /node/69406f5c575bc060ce15a83e HTTP/1.0" 202 Accepted 2025-12-17 20:28:27,925 - lava_callback - INFO - Starting async job submit for node 69406f5c575bc060ce15a83e 2025-12-17 20:28:27,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:28:29,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406f1f575bc060ce15a6b3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:28:29,918 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406f1f575bc060ce15a6b3 2025-12-17 20:28:29,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:28:32,248 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406efb575bc060ce15a518 2025-12-17 20:28:32,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:52506 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:28:38,108 - lava_callback - INFO - Received callback for node 69406f21575bc060ce15a6b6 INFO: 172.21.0.1:52516 - "POST /node/69406f21575bc060ce15a6b6 HTTP/1.0" 202 Accepted 2025-12-17 20:28:38,114 - lava_callback - INFO - Starting async job submit for node 69406f21575bc060ce15a6b6 2025-12-17 20:28:38,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:28:41,195 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407c90575bc060ce15be66 2025-12-17 20:28:41,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:28:43,472 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694084b1575bc060ce15d884 2025-12-17 20:28:43,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:53518 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:28:47,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efd575bc060ce15a51c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50514 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:28:58,268 - lava_callback - INFO - Received callback for node 69406eff575bc060ce15a526 INFO: 172.21.0.1:50518 - "POST /node/69406eff575bc060ce15a526 HTTP/1.0" 202 Accepted 2025-12-17 20:28:58,275 - lava_callback - INFO - Starting async job submit for node 69406eff575bc060ce15a526 2025-12-17 20:28:58,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:39632 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:29:02,839 - lava_callback - INFO - Received callback for node 69407780575bc060ce15b30e INFO: 172.21.0.1:39648 - "POST /node/69407780575bc060ce15b30e HTTP/1.0" 202 Accepted 2025-12-17 20:29:02,847 - lava_callback - INFO - Starting async job submit for node 69407780575bc060ce15b30e 2025-12-17 20:29:02,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:29:03,632 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406efd575bc060ce15a51c 2025-12-17 20:29:03,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:29:05,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940777a575bc060ce15b308'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39658 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:29:13,178 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940777a575bc060ce15b308 2025-12-17 20:29:13,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:41420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41428 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:29:20,199 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efa575bc060ce15a515'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:29:24,200 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406efa575bc060ce15a515 2025-12-17 20:29:24,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:29:25,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852f575bc060ce15daaf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41310 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:29:27,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406f5c575bc060ce15a83e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:29:27,992 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406f5c575bc060ce15a83e 2025-12-17 20:29:27,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:29:29,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406f1f575bc060ce15a6b3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:29:31,968 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406f1f575bc060ce15a6b3 2025-12-17 20:29:31,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:29:32,322 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efb575bc060ce15a518'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:29:33,008 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940852f575bc060ce15daaf 2025-12-17 20:29:33,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:41556 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:29:37,591 - lava_callback - INFO - Received callback for node 69406eee575bc060ce15a4f2 INFO: 172.21.0.1:41560 - "POST /node/69406eee575bc060ce15a4f2 HTTP/1.0" 202 Accepted 2025-12-17 20:29:37,638 - lava_callback - INFO - Starting async job submit for node 69406eee575bc060ce15a4f2 2025-12-17 20:29:37,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:29:38,224 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406f21575bc060ce15a6b6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:29:38,224 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406f21575bc060ce15a6b6 2025-12-17 20:29:38,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:29:41,285 - lava_callback - ERROR - Error processing callback for node 69407c90575bc060ce15be66: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c90575bc060ce15be66 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c90575bc060ce15be66 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c90575bc060ce15be66 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:29:41,313 - lava_callback - INFO - Received callback for node 6940c5231e3c24e95681d677 INFO: 172.21.0.1:45112 - "POST /node/6940c5231e3c24e95681d677 HTTP/1.0" 202 Accepted 2025-12-17 20:29:41,321 - lava_callback - INFO - Starting async job submit for node 6940c5231e3c24e95681d677 2025-12-17 20:29:41,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:29:43,544 - lava_callback - ERROR - Error processing callback for node 694084b1575bc060ce15d884: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b1575bc060ce15d884 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b1575bc060ce15d884 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b1575bc060ce15d884 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:45126 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:29:48,322 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406efb575bc060ce15a518 2025-12-17 20:29:48,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:45532 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:29:58,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406eff575bc060ce15a526'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:29:58,439 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406eff575bc060ce15a526 2025-12-17 20:29:58,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:60788 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:30:02,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407780575bc060ce15b30e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:30:02,952 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407780575bc060ce15b30e 2025-12-17 20:30:02,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:30:03,696 - lava_callback - ERROR - Error processing callback for node 69406efd575bc060ce15a51c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406efd575bc060ce15a51c (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406efd575bc060ce15a51c (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406efd575bc060ce15a51c (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:60804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48330 - "GET /node/693ae713fee8383e92b65020%5Cn HTTP/1.0" 405 Method Not Allowed 2025-12-17 20:30:13,244 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940777a575bc060ce15b308'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48352 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:30:21,272 - lava_callback - INFO - Received callback for node 6940708a575bc060ce15a8be INFO: 172.21.0.1:50272 - "POST /node/6940708a575bc060ce15a8be HTTP/1.0" 202 Accepted 2025-12-17 20:30:21,279 - lava_callback - INFO - Starting async job submit for node 6940708a575bc060ce15a8be 2025-12-17 20:30:21,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:30:24,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efa575bc060ce15a515'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50282 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:30:28,075 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406f5c575bc060ce15a83e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:30:29,244 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940777a575bc060ce15b308 2025-12-17 20:30:29,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:30:30,076 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406f5c575bc060ce15a83e 2025-12-17 20:30:30,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:30:32,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406f1f575bc060ce15a6b3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:30:32,243 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406efa575bc060ce15a515 2025-12-17 20:30:32,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:30:33,065 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852f575bc060ce15daaf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:30:36,008 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406f1f575bc060ce15a6b3 2025-12-17 20:30:36,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:51388 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:30:37,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406eee575bc060ce15a4f2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:30:37,699 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406eee575bc060ce15a4f2 2025-12-17 20:30:37,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:30:38,256 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406f21575bc060ce15a6b6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:30:39,133 - lava_callback - INFO - Received callback for node 694070cb575bc060ce15a96a INFO: 172.21.0.1:51398 - "POST /node/694070cb575bc060ce15a96a HTTP/1.0" 202 Accepted 2025-12-17 20:30:39,140 - lava_callback - INFO - Starting async job submit for node 694070cb575bc060ce15a96a 2025-12-17 20:30:39,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:30:40,256 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406f21575bc060ce15a6b6 2025-12-17 20:30:40,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:30:41,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5231e3c24e95681d677'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:30:41,431 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940c5231e3c24e95681d677 2025-12-17 20:30:41,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:47476 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:30:48,392 - lava_callback - ERROR - Error processing callback for node 69406efb575bc060ce15a518: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406efb575bc060ce15a518 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406efb575bc060ce15a518 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406efb575bc060ce15a518 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:30:49,065 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940852f575bc060ce15daaf 2025-12-17 20:30:49,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:36704 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:30:58,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406eff575bc060ce15a526'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:31:00,480 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406eff575bc060ce15a526 2025-12-17 20:31:00,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:40074 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:31:03,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407780575bc060ce15b30e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:31:05,015 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407780575bc060ce15b30e 2025-12-17 20:31:05,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:40080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36698 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:31:21,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940708a575bc060ce15a8be'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:31:21,376 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940708a575bc060ce15a8be 2025-12-17 20:31:21,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:42576 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:31:29,285 - lava_callback - ERROR - Error processing callback for node 6940777a575bc060ce15b308: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940777a575bc060ce15b308 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940777a575bc060ce15b308 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940777a575bc060ce15b308 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:31:30,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406f5c575bc060ce15a83e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:31:32,271 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efa575bc060ce15a515'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:31:34,116 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406f5c575bc060ce15a83e 2025-12-17 20:31:34,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:31:36,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406f1f575bc060ce15a6b3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46968 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:31:37,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406eee575bc060ce15a4f2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:31:39,226 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070cb575bc060ce15a96a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:31:39,226 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070cb575bc060ce15a96a 2025-12-17 20:31:39,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:31:39,760 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406eee575bc060ce15a4f2 2025-12-17 20:31:39,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:31:40,311 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406f21575bc060ce15a6b6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:31:41,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5231e3c24e95681d677'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:31:43,488 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940c5231e3c24e95681d677 2025-12-17 20:31:43,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:31:44,028 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406f1f575bc060ce15a6b3 2025-12-17 20:31:44,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:31:44,312 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406f21575bc060ce15a6b6 2025-12-17 20:31:44,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:57882 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:31:48,272 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406efa575bc060ce15a515 2025-12-17 20:31:48,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:31:49,136 - lava_callback - ERROR - Error processing callback for node 6940852f575bc060ce15daaf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940852f575bc060ce15daaf (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940852f575bc060ce15daaf (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940852f575bc060ce15daaf (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:40308 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:32:00,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406eff575bc060ce15a526'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55372 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:32:04,545 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406eff575bc060ce15a526 2025-12-17 20:32:04,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:32:05,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407780575bc060ce15b30e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55374 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:32:09,072 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407780575bc060ce15b30e 2025-12-17 20:32:09,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:32:10,562 - lava_callback - INFO - Received callback for node 694071a2575bc060ce15ab47 INFO: 172.21.0.1:52110 - "POST /node/694071a2575bc060ce15ab47 HTTP/1.0" 202 Accepted 2025-12-17 20:32:10,569 - lava_callback - INFO - Starting async job submit for node 694071a2575bc060ce15ab47 2025-12-17 20:32:10,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:52112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52122 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:32:21,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940708a575bc060ce15a8be'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:32:23,440 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940708a575bc060ce15a8be 2025-12-17 20:32:23,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:43016 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:32:34,145 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406f5c575bc060ce15a83e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51458 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:32:39,277 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070cb575bc060ce15a96a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:32:39,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406eee575bc060ce15a4f2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:32:41,278 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070cb575bc060ce15a96a 2025-12-17 20:32:41,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:32:42,145 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406f5c575bc060ce15a83e 2025-12-17 20:32:42,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:32:43,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5231e3c24e95681d677'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:32:43,824 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406eee575bc060ce15a4f2 2025-12-17 20:32:43,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:32:44,092 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406f1f575bc060ce15a6b3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:32:44,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406f21575bc060ce15a6b6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51938 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:32:47,552 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940c5231e3c24e95681d677 2025-12-17 20:32:47,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:32:48,304 - lava_callback - ERROR - Error processing callback for node 69406efa575bc060ce15a515: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406efa575bc060ce15a515 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406efa575bc060ce15a515 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406efa575bc060ce15a515 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:32:52,369 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406f21575bc060ce15a6b6 2025-12-17 20:32:52,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:42786 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:32:58,486 - lava_callback - INFO - Received callback for node 694070ce575bc060ce15a974 INFO: 172.21.0.1:42794 - "POST /node/694070ce575bc060ce15a974 HTTP/1.0" 202 Accepted 2025-12-17 20:32:58,493 - lava_callback - INFO - Starting async job submit for node 694070ce575bc060ce15a974 2025-12-17 20:32:58,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:33:00,092 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406f1f575bc060ce15a6b3 2025-12-17 20:33:00,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:38666 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:33:04,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406eff575bc060ce15a526'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38668 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:33:09,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407780575bc060ce15b30e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:33:10,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071a2575bc060ce15ab47'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:33:10,648 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694071a2575bc060ce15ab47 2025-12-17 20:33:10,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:33:12,609 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406eff575bc060ce15a526 2025-12-17 20:33:12,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:55116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55118 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:33:17,118 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407780575bc060ce15b30e 2025-12-17 20:33:17,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:33:23,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940708a575bc060ce15a8be'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40822 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:33:27,494 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940708a575bc060ce15a8be 2025-12-17 20:33:27,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:35830 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:33:41,327 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070cb575bc060ce15a96a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:33:42,197 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406f5c575bc060ce15a83e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:33:43,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406eee575bc060ce15a4f2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:33:45,237 - lava_callback - INFO - Received callback for node 6940778f575bc060ce15b313 INFO: 172.21.0.1:50238 - "POST /node/6940778f575bc060ce15b313 HTTP/1.0" 202 Accepted 2025-12-17 20:33:45,245 - lava_callback - INFO - Starting async job submit for node 6940778f575bc060ce15b313 2025-12-17 20:33:45,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:33:45,328 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070cb575bc060ce15a96a 2025-12-17 20:33:45,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:50248 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:33:47,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5231e3c24e95681d677'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:33:51,912 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406eee575bc060ce15a4f2 2025-12-17 20:33:51,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:33:52,406 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406f21575bc060ce15a6b6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:33:55,584 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940c5231e3c24e95681d677 2025-12-17 20:33:55,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:42794 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:33:58,198 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406f5c575bc060ce15a83e 2025-12-17 20:33:58,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:33:58,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070ce575bc060ce15a974'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:33:58,612 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070ce575bc060ce15a974 2025-12-17 20:33:58,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:34:00,153 - lava_callback - ERROR - Error processing callback for node 69406f1f575bc060ce15a6b3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406f1f575bc060ce15a6b3 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406f1f575bc060ce15a6b3 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406f1f575bc060ce15a6b3 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) INFO: 172.21.0.1:41856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41864 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:34:08,406 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406f21575bc060ce15a6b6 2025-12-17 20:34:08,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:34:10,136 - lava_callback - INFO - Received callback for node 6940741c575bc060ce15ae64 INFO: 172.21.0.1:36850 - "POST /node/6940741c575bc060ce15ae64 HTTP/1.0" 202 Accepted 2025-12-17 20:34:10,142 - lava_callback - INFO - Starting async job submit for node 6940741c575bc060ce15ae64 2025-12-17 20:34:10,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:34:10,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071a2575bc060ce15ab47'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:34:12,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406eff575bc060ce15a526'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:34:12,708 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694071a2575bc060ce15ab47 2025-12-17 20:34:12,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:36866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36872 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:34:17,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407780575bc060ce15b30e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:34:18,502 - lava_callback - INFO - Received callback for node 694070cf575bc060ce15a977 INFO: 172.21.0.1:36876 - "POST /node/694070cf575bc060ce15a977 HTTP/1.0" 202 Accepted 2025-12-17 20:34:18,510 - lava_callback - INFO - Starting async job submit for node 694070cf575bc060ce15a977 2025-12-17 20:34:18,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:34:24,410 - lava_callback - INFO - Received callback for node 694070c6575bc060ce15a940 INFO: 172.21.0.1:44180 - "POST /node/694070c6575bc060ce15a940 HTTP/1.0" 202 Accepted 2025-12-17 20:34:24,416 - lava_callback - INFO - Starting async job submit for node 694070c6575bc060ce15a940 2025-12-17 20:34:24,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:44194 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:34:27,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940708a575bc060ce15a8be'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:34:28,698 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406eff575bc060ce15a526 2025-12-17 20:34:28,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:34:33,216 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407780575bc060ce15b30e 2025-12-17 20:34:33,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:34:35,550 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940708a575bc060ce15a8be 2025-12-17 20:34:35,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:57958 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:34:41,577 - lava_callback - INFO - Received callback for node 694070d5575bc060ce15a98a INFO: 172.21.0.1:53594 - "POST /node/694070d5575bc060ce15a98a HTTP/1.0" 202 Accepted 2025-12-17 20:34:41,582 - lava_callback - INFO - Starting async job submit for node 694070d5575bc060ce15a98a 2025-12-17 20:34:41,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:34:42,319 - lava_callback - INFO - Received callback for node 694070d6575bc060ce15a98d INFO: 172.21.0.1:53608 - "POST /node/694070d6575bc060ce15a98d HTTP/1.0" 202 Accepted 2025-12-17 20:34:42,325 - lava_callback - INFO - Starting async job submit for node 694070d6575bc060ce15a98d 2025-12-17 20:34:42,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:34:45,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940778f575bc060ce15b313'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:34:45,331 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940778f575bc060ce15b313 2025-12-17 20:34:45,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:34:45,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070cb575bc060ce15a96a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53624 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:34:51,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406eee575bc060ce15a4f2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:34:53,394 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070cb575bc060ce15a96a 2025-12-17 20:34:53,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:34:55,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5231e3c24e95681d677'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60900 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:34:58,252 - lava_callback - ERROR - Error processing callback for node 69406f5c575bc060ce15a83e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406f5c575bc060ce15a83e (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406f5c575bc060ce15a83e (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406f5c575bc060ce15a83e (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:34:58,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070ce575bc060ce15a974'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:35:00,628 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070ce575bc060ce15a974 2025-12-17 20:35:00,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:49950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49954 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:35:07,993 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69406eee575bc060ce15a4f2 2025-12-17 20:35:07,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:35:08,460 - lava_callback - ERROR - Error processing callback for node 69406f21575bc060ce15a6b6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406f21575bc060ce15a6b6 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406f21575bc060ce15a6b6 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406f21575bc060ce15a6b6 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:35:10,254 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741c575bc060ce15ae64'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:35:10,254 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940741c575bc060ce15ae64 2025-12-17 20:35:10,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:35:11,632 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940c5231e3c24e95681d677 2025-12-17 20:35:11,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:35:12,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071a2575bc060ce15ab47'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60406 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:35:16,768 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694071a2575bc060ce15ab47 2025-12-17 20:35:16,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:35:18,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070cf575bc060ce15a977'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:35:18,681 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070cf575bc060ce15a977 2025-12-17 20:35:18,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:35:21,555 - lava_callback - INFO - Received callback for node 69407105575bc060ce15a9ba INFO: 172.21.0.1:38538 - "POST /node/69407105575bc060ce15a9ba HTTP/1.0" 202 Accepted 2025-12-17 20:35:21,561 - lava_callback - INFO - Starting async job submit for node 69407105575bc060ce15a9ba 2025-12-17 20:35:21,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:35:24,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070c6575bc060ce15a940'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:35:24,506 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070c6575bc060ce15a940 2025-12-17 20:35:24,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:38554 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:35:28,712 - lava_callback - ERROR - Error processing callback for node 69406eff575bc060ce15a526: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406eff575bc060ce15a526 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406eff575bc060ce15a526 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406eff575bc060ce15a526 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:35:33,281 - lava_callback - ERROR - Error processing callback for node 69407780575bc060ce15b30e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407780575bc060ce15b30e (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407780575bc060ce15b30e (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407780575bc060ce15b30e (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:35:35,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940708a575bc060ce15a8be'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57714 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:35:38,509 - lava_callback - INFO - Received callback for node 694070d0575bc060ce15a97a INFO: 172.21.0.1:57726 - "POST /node/694070d0575bc060ce15a97a HTTP/1.0" 202 Accepted 2025-12-17 20:35:38,516 - lava_callback - INFO - Starting async job submit for node 694070d0575bc060ce15a97a 2025-12-17 20:35:38,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:35:41,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d5575bc060ce15a98a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:35:41,629 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070d5575bc060ce15a98a 2025-12-17 20:35:41,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:35:42,184 - lava_callback - INFO - Received callback for node 69407114575bc060ce15a9d2 INFO: 172.21.0.1:52436 - "POST /node/69407114575bc060ce15a9d2 HTTP/1.0" 202 Accepted 2025-12-17 20:35:42,190 - lava_callback - INFO - Starting async job submit for node 69407114575bc060ce15a9d2 2025-12-17 20:35:42,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:35:42,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d6575bc060ce15a98d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:35:42,397 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070d6575bc060ce15a98d 2025-12-17 20:35:42,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:35:45,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940778f575bc060ce15b313'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52440 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:35:47,382 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940778f575bc060ce15b313 2025-12-17 20:35:47,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:35:51,615 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940708a575bc060ce15a8be 2025-12-17 20:35:51,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:35:53,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070cb575bc060ce15a96a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54338 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:36:00,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070ce575bc060ce15a974'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48644 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:36:04,659 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070ce575bc060ce15a974 2025-12-17 20:36:04,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:48654 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:36:08,059 - lava_callback - ERROR - Error processing callback for node 69406eee575bc060ce15a4f2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406eee575bc060ce15a4f2 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406eee575bc060ce15a4f2 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406eee575bc060ce15a4f2 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:36:09,435 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070cb575bc060ce15a96a 2025-12-17 20:36:09,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:36:10,041 - lava_callback - INFO - Received callback for node 6940741d575bc060ce15ae6b INFO: 172.21.0.1:47954 - "POST /node/6940741d575bc060ce15ae6b HTTP/1.0" 202 Accepted 2025-12-17 20:36:10,048 - lava_callback - INFO - Starting async job submit for node 6940741d575bc060ce15ae6b 2025-12-17 20:36:10,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:36:10,287 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741c575bc060ce15ae64'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:36:11,694 - lava_callback - ERROR - Error processing callback for node 6940c5231e3c24e95681d677: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5231e3c24e95681d677 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5231e3c24e95681d677 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5231e3c24e95681d677 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:36:12,288 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940741c575bc060ce15ae64 2025-12-17 20:36:12,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:47966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47972 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:36:16,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071a2575bc060ce15ab47'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:36:18,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070cf575bc060ce15a977'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:36:20,744 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070cf575bc060ce15a977 2025-12-17 20:36:20,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:36:21,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407105575bc060ce15a9ba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:36:21,628 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407105575bc060ce15a9ba 2025-12-17 20:36:21,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:36:21,888 - lava_callback - INFO - Received callback for node 69407142575bc060ce15aa71 INFO: 172.21.0.1:45138 - "POST /node/69407142575bc060ce15aa71 HTTP/1.0" 202 Accepted 2025-12-17 20:36:21,894 - lava_callback - INFO - Starting async job submit for node 69407142575bc060ce15aa71 2025-12-17 20:36:21,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:36:22,678 - lava_callback - INFO - Received callback for node 69407144575bc060ce15aa79 INFO: 172.21.0.1:45140 - "POST /node/69407144575bc060ce15aa79 HTTP/1.0" 202 Accepted 2025-12-17 20:36:22,683 - lava_callback - INFO - Starting async job submit for node 69407144575bc060ce15aa79 2025-12-17 20:36:22,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:36:24,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070c6575bc060ce15a940'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:36:24,835 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694071a2575bc060ce15ab47 2025-12-17 20:36:24,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:45142 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:36:26,550 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070c6575bc060ce15a940 2025-12-17 20:36:26,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:36:31,432 - lava_callback - INFO - Received callback for node 694085aa575bc060ce15dce8 INFO: 172.21.0.1:52966 - "POST /node/694085aa575bc060ce15dce8 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:52980 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:36:38,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d0575bc060ce15a97a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:36:38,675 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070d0575bc060ce15a97a 2025-12-17 20:36:38,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:36:41,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d5575bc060ce15a98a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:36:41,684 - lava_callback - INFO - Received callback for node 6940714a575bc060ce15aa8a INFO: 172.21.0.1:43358 - "POST /node/6940714a575bc060ce15aa8a HTTP/1.0" 202 Accepted 2025-12-17 20:36:42,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407114575bc060ce15a9d2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:36:42,280 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407114575bc060ce15a9d2 2025-12-17 20:36:42,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:36:42,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d6575bc060ce15a98d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:36:42,499 - lava_callback - INFO - Received callback for node 69407151575bc060ce15aa9a INFO: 172.21.0.1:43374 - "POST /node/69407151575bc060ce15aa9a HTTP/1.0" 202 Accepted 2025-12-17 20:36:43,266 - lava_callback - INFO - Received callback for node 69407152575bc060ce15aa9d INFO: 172.21.0.1:43378 - "POST /node/69407152575bc060ce15aa9d HTTP/1.0" 202 Accepted 2025-12-17 20:36:43,680 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070d5575bc060ce15a98a 2025-12-17 20:36:43,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:36:44,053 - lava_callback - INFO - Received callback for node 69407154575bc060ce15aaa0 INFO: 172.21.0.1:43392 - "POST /node/69407154575bc060ce15aaa0 HTTP/1.0" 202 Accepted 2025-12-17 20:36:44,464 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070d6575bc060ce15a98d 2025-12-17 20:36:44,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:43400 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:36:47,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940778f575bc060ce15b313'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:36:50,549 - lava_callback - INFO - Received callback for node 694070cc575bc060ce15a96f INFO: 172.21.0.1:38710 - "POST /node/694070cc575bc060ce15a96f HTTP/1.0" 202 Accepted 2025-12-17 20:36:51,448 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940778f575bc060ce15b313 2025-12-17 20:36:51,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:36:51,685 - lava_callback - ERROR - Error processing callback for node 6940708a575bc060ce15a8be: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940708a575bc060ce15a8be (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940708a575bc060ce15a8be (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940708a575bc060ce15a8be (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:36:51,687 - lava_callback - INFO - Starting async job submit for node 694085aa575bc060ce15dce8 2025-12-17 20:36:51,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:36:55,427 - lava_callback - INFO - Received callback for node 69408623575bc060ce15df03 INFO: 172.21.0.1:38716 - "POST /node/69408623575bc060ce15df03 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:38726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46652 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:37:02,182 - lava_callback - INFO - Received callback for node 69407168575bc060ce15aab6 INFO: 172.21.0.1:46660 - "POST /node/69407168575bc060ce15aab6 HTTP/1.0" 202 Accepted 2025-12-17 20:37:02,934 - lava_callback - INFO - Received callback for node 69407169575bc060ce15aab9 INFO: 172.21.0.1:46664 - "POST /node/69407169575bc060ce15aab9 HTTP/1.0" 202 Accepted 2025-12-17 20:37:04,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070ce575bc060ce15a974'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46666 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:37:09,517 - lava_callback - ERROR - Error processing callback for node 694070cb575bc060ce15a96a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070cb575bc060ce15a96a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070cb575bc060ce15a96a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070cb575bc060ce15a96a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:37:09,519 - lava_callback - INFO - Starting async job submit for node 6940714a575bc060ce15aa8a 2025-12-17 20:37:09,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:37:10,138 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741d575bc060ce15ae6b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:37:10,139 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940741d575bc060ce15ae6b 2025-12-17 20:37:10,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:37:12,339 - lava_callback - INFO - Received callback for node 6940863a575bc060ce15df89 INFO: 172.21.0.1:50896 - "POST /node/6940863a575bc060ce15df89 HTTP/1.0" 202 Accepted 2025-12-17 20:37:12,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741c575bc060ce15ae64'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:37:12,679 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070ce575bc060ce15a974 2025-12-17 20:37:12,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:37:14,766 - lava_callback - INFO - Received callback for node 69408683575bc060ce15e082 INFO: 172.21.0.1:50902 - "POST /node/69408683575bc060ce15e082 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:50908 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:37:16,352 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940741c575bc060ce15ae64 2025-12-17 20:37:16,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:50924 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:37:20,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070cf575bc060ce15a977'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:37:21,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407105575bc060ce15a9ba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:37:21,830 - lava_callback - INFO - Received callback for node 69407180575bc060ce15aaf0 INFO: 172.21.0.1:39084 - "POST /node/69407180575bc060ce15aaf0 HTTP/1.0" 202 Accepted 2025-12-17 20:37:21,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407142575bc060ce15aa71'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:37:21,967 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407142575bc060ce15aa71 2025-12-17 20:37:21,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:37:22,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407144575bc060ce15aa79'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:37:22,736 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407144575bc060ce15aa79 2025-12-17 20:37:22,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:37:23,676 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407105575bc060ce15a9ba 2025-12-17 20:37:23,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:37:24,772 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070cf575bc060ce15a977 2025-12-17 20:37:24,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:37:24,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071a2575bc060ce15ab47'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39096 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:37:26,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070c6575bc060ce15a940'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:37:30,607 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070c6575bc060ce15a940 2025-12-17 20:37:30,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:38186 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:37:38,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d0575bc060ce15a97a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:37:40,740 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070d0575bc060ce15a97a 2025-12-17 20:37:40,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:37:40,893 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694071a2575bc060ce15ab47 2025-12-17 20:37:40,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:37:41,662 - lava_callback - INFO - Received callback for node 69407181575bc060ce15aafb INFO: 172.21.0.1:41870 - "POST /node/69407181575bc060ce15aafb HTTP/1.0" 202 Accepted 2025-12-17 20:37:42,311 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407114575bc060ce15a9d2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:37:42,417 - lava_callback - INFO - Received callback for node 69407183575bc060ce15aafe INFO: 172.21.0.1:41872 - "POST /node/69407183575bc060ce15aafe HTTP/1.0" 202 Accepted 2025-12-17 20:37:43,191 - lava_callback - INFO - Received callback for node 69407184575bc060ce15ab01 INFO: 172.21.0.1:41884 - "POST /node/69407184575bc060ce15ab01 HTTP/1.0" 202 Accepted 2025-12-17 20:37:43,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d5575bc060ce15a98a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:37:44,012 - lava_callback - INFO - Received callback for node 69407185575bc060ce15ab08 INFO: 172.21.0.1:41888 - "POST /node/69407185575bc060ce15ab08 HTTP/1.0" 202 Accepted 2025-12-17 20:37:44,312 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407114575bc060ce15a9d2 2025-12-17 20:37:44,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:37:44,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d6575bc060ce15a98d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:37:44,768 - lava_callback - INFO - Received callback for node 69407187575bc060ce15ab0b INFO: 172.21.0.1:41900 - "POST /node/69407187575bc060ce15ab0b HTTP/1.0" 202 Accepted 2025-12-17 20:37:45,520 - lava_callback - INFO - Received callback for node 69407188575bc060ce15ab0e INFO: 172.21.0.1:41910 - "POST /node/69407188575bc060ce15ab0e HTTP/1.0" 202 Accepted 2025-12-17 20:37:46,320 - lava_callback - INFO - Received callback for node 69407189575bc060ce15ab11 INFO: 172.21.0.1:41920 - "POST /node/69407189575bc060ce15ab11 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:41934 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:37:47,241 - lava_callback - INFO - Received callback for node 6940718a575bc060ce15ab14 INFO: 172.21.0.1:41944 - "POST /node/6940718a575bc060ce15ab14 HTTP/1.0" 202 Accepted 2025-12-17 20:37:47,751 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070d5575bc060ce15a98a 2025-12-17 20:37:47,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:37:48,000 - lava_callback - INFO - Received callback for node 6940718c575bc060ce15ab17 INFO: 172.21.0.1:41946 - "POST /node/6940718c575bc060ce15ab17 HTTP/1.0" 202 Accepted 2025-12-17 20:37:48,520 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070d6575bc060ce15a98d 2025-12-17 20:37:48,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:37:48,889 - lava_callback - INFO - Received callback for node 6940718d575bc060ce15ab1a INFO: 172.21.0.1:41948 - "POST /node/6940718d575bc060ce15ab1a HTTP/1.0" 202 Accepted 2025-12-17 20:37:49,692 - lava_callback - INFO - Received callback for node 6940718e575bc060ce15ab1d INFO: 172.21.0.1:41954 - "POST /node/6940718e575bc060ce15ab1d HTTP/1.0" 202 Accepted 2025-12-17 20:37:50,431 - lava_callback - INFO - Received callback for node 6940718f575bc060ce15ab20 INFO: 172.21.0.1:32974 - "POST /node/6940718f575bc060ce15ab20 HTTP/1.0" 202 Accepted 2025-12-17 20:37:51,098 - lava_callback - INFO - Received callback for node 6940741d575bc060ce15ae76 INFO: 172.21.0.1:32986 - "POST /node/6940741d575bc060ce15ae76 HTTP/1.0" 202 Accepted 2025-12-17 20:37:51,176 - lava_callback - INFO - Received callback for node 69407190575bc060ce15ab23 INFO: 172.21.0.1:32990 - "POST /node/69407190575bc060ce15ab23 HTTP/1.0" 202 Accepted 2025-12-17 20:37:51,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940778f575bc060ce15b313'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:37:51,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085aa575bc060ce15dce8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:37:51,854 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694085aa575bc060ce15dce8 2025-12-17 20:37:51,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:37:51,960 - lava_callback - INFO - Received callback for node 69407192575bc060ce15ab26 INFO: 172.21.0.1:32996 - "POST /node/69407192575bc060ce15ab26 HTTP/1.0" 202 Accepted 2025-12-17 20:37:52,747 - lava_callback - INFO - Received callback for node 69407193575bc060ce15ab29 INFO: 172.21.0.1:33006 - "POST /node/69407193575bc060ce15ab29 HTTP/1.0" 202 Accepted 2025-12-17 20:37:53,551 - lava_callback - INFO - Received callback for node 69407194575bc060ce15ab2c INFO: 172.21.0.1:33018 - "POST /node/69407194575bc060ce15ab2c HTTP/1.0" 202 Accepted 2025-12-17 20:37:54,296 - lava_callback - INFO - Received callback for node 69407195575bc060ce15ab2f INFO: 172.21.0.1:33022 - "POST /node/69407195575bc060ce15ab2f HTTP/1.0" 202 Accepted 2025-12-17 20:37:55,052 - lava_callback - INFO - Received callback for node 69407196575bc060ce15ab32 INFO: 172.21.0.1:33038 - "POST /node/69407196575bc060ce15ab32 HTTP/1.0" 202 Accepted 2025-12-17 20:37:55,758 - lava_callback - INFO - Received callback for node 69407197575bc060ce15ab35 INFO: 172.21.0.1:33042 - "POST /node/69407197575bc060ce15ab35 HTTP/1.0" 202 Accepted 2025-12-17 20:37:56,511 - lava_callback - INFO - Received callback for node 69407198575bc060ce15ab38 INFO: 172.21.0.1:33044 - "POST /node/69407198575bc060ce15ab38 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:33046 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:37:57,457 - lava_callback - INFO - Received callback for node 6940719a575bc060ce15ab3b INFO: 172.21.0.1:33056 - "POST /node/6940719a575bc060ce15ab3b HTTP/1.0" 202 Accepted 2025-12-17 20:37:58,204 - lava_callback - INFO - Received callback for node 6940719b575bc060ce15ab3e INFO: 172.21.0.1:33072 - "POST /node/6940719b575bc060ce15ab3e HTTP/1.0" 202 Accepted 2025-12-17 20:37:58,933 - lava_callback - INFO - Received callback for node 694071a1575bc060ce15ab41 INFO: 172.21.0.1:33076 - "POST /node/694071a1575bc060ce15ab41 HTTP/1.0" 202 Accepted 2025-12-17 20:37:59,509 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940778f575bc060ce15b313 2025-12-17 20:37:59,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:60018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60034 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:38:09,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940714a575bc060ce15aa8a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:38:09,590 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940714a575bc060ce15aa8a 2025-12-17 20:38:09,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:38:10,228 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741d575bc060ce15ae6b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:38:12,229 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940741d575bc060ce15ae6b 2025-12-17 20:38:12,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:38:12,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070ce575bc060ce15a974'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58420 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:38:16,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741c575bc060ce15ae64'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58430 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:38:22,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407142575bc060ce15aa71'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:38:22,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407144575bc060ce15aa79'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:38:23,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407105575bc060ce15a9ba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:38:24,020 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407142575bc060ce15aa71 2025-12-17 20:38:24,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:38:24,403 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940741c575bc060ce15ae64 2025-12-17 20:38:24,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:38:24,781 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407144575bc060ce15aa79 2025-12-17 20:38:24,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:38:24,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070cf575bc060ce15a977'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40924 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:38:27,741 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407105575bc060ce15a9ba 2025-12-17 20:38:27,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:38:28,732 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070ce575bc060ce15a974 2025-12-17 20:38:28,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:38:30,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070c6575bc060ce15a940'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:38:32,838 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070cf575bc060ce15a977 2025-12-17 20:38:32,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:38088 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:38:38,674 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070c6575bc060ce15a940 2025-12-17 20:38:38,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:38:40,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d0575bc060ce15a97a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:38:40,974 - lava_callback - ERROR - Error processing callback for node 694071a2575bc060ce15ab47: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694071a2575bc060ce15ab47 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694071a2575bc060ce15ab47 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694071a2575bc060ce15ab47 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:38:40,976 - lava_callback - INFO - Starting async job submit for node 69407151575bc060ce15aa9a 2025-12-17 20:38:40,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:38:44,356 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407114575bc060ce15a9d2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:38:44,606 - lava_callback - INFO - Received callback for node 6940780c575bc060ce15b3fd INFO: 172.21.0.1:37280 - "POST /node/6940780c575bc060ce15b3fd HTTP/1.0" 202 Accepted 2025-12-17 20:38:44,770 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070d0575bc060ce15a97a 2025-12-17 20:38:44,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:37292 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:38:47,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d5575bc060ce15a98a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:38:48,356 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407114575bc060ce15a9d2 2025-12-17 20:38:48,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:38:48,388 - lava_callback - INFO - Received callback for node 694071cd575bc060ce15abd5 INFO: 172.21.0.1:37308 - "POST /node/694071cd575bc060ce15abd5 HTTP/1.0" 202 Accepted 2025-12-17 20:38:48,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d6575bc060ce15a98d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:38:51,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085aa575bc060ce15dce8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:38:53,935 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694085aa575bc060ce15dce8 2025-12-17 20:38:53,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:38:55,808 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070d5575bc060ce15a98a 2025-12-17 20:38:55,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:41958 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:38:56,608 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070d6575bc060ce15a98d 2025-12-17 20:38:56,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:38:59,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940778f575bc060ce15b313'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37056 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:39:09,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940714a575bc060ce15aa8a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:39:11,656 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940714a575bc060ce15aa8a 2025-12-17 20:39:11,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:39:12,299 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741d575bc060ce15ae6b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48400 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:39:15,575 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940778f575bc060ce15b313 2025-12-17 20:39:15,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:39:16,300 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940741d575bc060ce15ae6b 2025-12-17 20:39:16,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:48412 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:39:24,098 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407142575bc060ce15aa71'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:39:24,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741c575bc060ce15ae64'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:39:24,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407144575bc060ce15aa79'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60224 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:39:27,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407105575bc060ce15a9ba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:39:28,098 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407142575bc060ce15aa71 2025-12-17 20:39:28,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:39:28,788 - lava_callback - ERROR - Error processing callback for node 694070ce575bc060ce15a974: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070ce575bc060ce15a974 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070ce575bc060ce15a974 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070ce575bc060ce15a974 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:39:28,790 - lava_callback - INFO - Starting async job submit for node 69407152575bc060ce15aa9d 2025-12-17 20:39:28,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:39:28,848 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407144575bc060ce15aa79 2025-12-17 20:39:28,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:39:32,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070cf575bc060ce15a977'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:39:35,777 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407105575bc060ce15a9ba 2025-12-17 20:39:35,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:52250 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:39:38,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070c6575bc060ce15a940'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:39:40,492 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940741c575bc060ce15ae64 2025-12-17 20:39:40,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:39:41,039 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407151575bc060ce15aa9a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:39:41,040 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407151575bc060ce15aa9a 2025-12-17 20:39:41,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:39:44,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d0575bc060ce15a97a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56552 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:39:48,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407114575bc060ce15a9d2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:39:48,930 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070cf575bc060ce15a977 2025-12-17 20:39:48,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:39:52,836 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070d0575bc060ce15a97a 2025-12-17 20:39:52,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:39:53,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085aa575bc060ce15dce8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:39:54,744 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070c6575bc060ce15a940 2025-12-17 20:39:54,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:39:55,153 - lava_callback - INFO - Received callback for node 694087b7575bc060ce15e4ae INFO: 172.21.0.1:59160 - "POST /node/694087b7575bc060ce15e4ae HTTP/1.0" 202 Accepted 2025-12-17 20:39:55,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d5575bc060ce15a98a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:39:56,400 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407114575bc060ce15a9d2 2025-12-17 20:39:56,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:59176 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:39:56,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d6575bc060ce15a98d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:39:57,980 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694085aa575bc060ce15dce8 2025-12-17 20:39:57,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:39:58,797 - lava_callback - INFO - Received callback for node 69407113575bc060ce15a9cf INFO: 172.21.0.1:59192 - "POST /node/69407113575bc060ce15a9cf HTTP/1.0" 202 Accepted INFO: 172.21.0.1:36494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36506 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:40:11,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940714a575bc060ce15aa8a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:40:11,888 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070d5575bc060ce15a98a 2025-12-17 20:40:11,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:40:12,641 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070d6575bc060ce15a98d 2025-12-17 20:40:12,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:48638 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:40:15,632 - lava_callback - ERROR - Error processing callback for node 6940778f575bc060ce15b313: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940778f575bc060ce15b313 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940778f575bc060ce15b313 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940778f575bc060ce15b313 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:40:15,634 - lava_callback - INFO - Starting async job submit for node 69407154575bc060ce15aaa0 2025-12-17 20:40:15,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:40:15,734 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940714a575bc060ce15aa8a 2025-12-17 20:40:15,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:40:16,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741d575bc060ce15ae6b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48654 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:40:24,320 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940741d575bc060ce15ae6b 2025-12-17 20:40:24,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:40398 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:40:28,166 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407142575bc060ce15aa71'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:40:28,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407152575bc060ce15aa9d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:40:28,820 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407152575bc060ce15aa9d 2025-12-17 20:40:28,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:40:28,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407144575bc060ce15aa79'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:40:35,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407105575bc060ce15a9ba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:40:36,166 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407142575bc060ce15aa71 2025-12-17 20:40:36,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:33390 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:40:36,891 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407144575bc060ce15aa79 2025-12-17 20:40:36,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:40:40,551 - lava_callback - ERROR - Error processing callback for node 6940741c575bc060ce15ae64: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940741c575bc060ce15ae64 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940741c575bc060ce15ae64 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940741c575bc060ce15ae64 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:40:40,553 - lava_callback - INFO - Starting async job submit for node 694070cc575bc060ce15a96f 2025-12-17 20:40:40,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:40:41,065 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407151575bc060ce15aa9a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:40:43,065 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407151575bc060ce15aa9a 2025-12-17 20:40:43,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:58172 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:40:48,986 - lava_callback - ERROR - Error processing callback for node 694070cf575bc060ce15a977: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070cf575bc060ce15a977 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070cf575bc060ce15a977 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070cf575bc060ce15a977 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:40:48,988 - lava_callback - INFO - Starting async job submit for node 69408623575bc060ce15df03 2025-12-17 20:40:49,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:40:51,860 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407105575bc060ce15a9ba 2025-12-17 20:40:51,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:40:52,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d0575bc060ce15a97a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:40:54,814 - lava_callback - ERROR - Error processing callback for node 694070c6575bc060ce15a940: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070c6575bc060ce15a940 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070c6575bc060ce15a940 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070c6575bc060ce15a940 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:40:54,816 - lava_callback - INFO - Starting async job submit for node 69407168575bc060ce15aab6 2025-12-17 20:40:54,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:40:56,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407114575bc060ce15a9d2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53828 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:40:58,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085aa575bc060ce15dce8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48966 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:41:06,012 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694085aa575bc060ce15dce8 2025-12-17 20:41:06,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:48976 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:41:08,922 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070d0575bc060ce15a97a 2025-12-17 20:41:08,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:41:11,904 - lava_callback - ERROR - Error processing callback for node 694070d5575bc060ce15a98a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d5575bc060ce15a98a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d5575bc060ce15a98a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d5575bc060ce15a98a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:41:11,906 - lava_callback - INFO - Starting async job submit for node 69407169575bc060ce15aab9 2025-12-17 20:41:11,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:41:12,466 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407114575bc060ce15a9d2 2025-12-17 20:41:12,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:41:12,699 - lava_callback - ERROR - Error processing callback for node 694070d6575bc060ce15a98d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d6575bc060ce15a98d (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d6575bc060ce15a98d (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d6575bc060ce15a98d (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:41:12,701 - lava_callback - INFO - Starting async job submit for node 6940863a575bc060ce15df89 2025-12-17 20:41:12,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:41:15,026 - lava_callback - INFO - Received callback for node 69407cc3575bc060ce15bea3 INFO: 172.21.0.1:39600 - "POST /node/69407cc3575bc060ce15bea3 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:39612 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:41:15,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407154575bc060ce15aaa0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:41:15,676 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407154575bc060ce15aaa0 2025-12-17 20:41:15,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:41:15,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940714a575bc060ce15aa8a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39626 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:41:23,769 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940714a575bc060ce15aa8a 2025-12-17 20:41:23,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:41:24,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741d575bc060ce15ae6b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:41:25,696 - lava_callback - INFO - Received callback for node 69407272575bc060ce15ac47 INFO: 172.21.0.1:59020 - "POST /node/69407272575bc060ce15ac47 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:59034 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:41:28,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407152575bc060ce15aa9d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:41:29,565 - lava_callback - INFO - Received callback for node 69407cc3575bc060ce15bea9 INFO: 172.21.0.1:59044 - "POST /node/69407cc3575bc060ce15bea9 HTTP/1.0" 202 Accepted 2025-12-17 20:41:30,832 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407152575bc060ce15aa9d 2025-12-17 20:41:30,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:41:36,222 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407142575bc060ce15aa71'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53792 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:41:37,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407144575bc060ce15aa79'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:41:39,454 - lava_callback - INFO - Received callback for node 69407274575bc060ce15ac4e INFO: 172.21.0.1:53798 - "POST /node/69407274575bc060ce15ac4e HTTP/1.0" 202 Accepted 2025-12-17 20:41:40,114 - lava_callback - INFO - Received callback for node 69407275575bc060ce15ac55 INFO: 172.21.0.1:34942 - "POST /node/69407275575bc060ce15ac55 HTTP/1.0" 202 Accepted 2025-12-17 20:41:40,348 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940741d575bc060ce15ae6b 2025-12-17 20:41:40,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:41:40,746 - lava_callback - INFO - Received callback for node 69407277575bc060ce15ac58 INFO: 172.21.0.1:34958 - "POST /node/69407277575bc060ce15ac58 HTTP/1.0" 202 Accepted 2025-12-17 20:41:40,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070cc575bc060ce15a96f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:41:40,951 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070cc575bc060ce15a96f 2025-12-17 20:41:40,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:41:41,379 - lava_callback - INFO - Received callback for node 69407278575bc060ce15ac5b INFO: 172.21.0.1:34974 - "POST /node/69407278575bc060ce15ac5b HTTP/1.0" 202 Accepted 2025-12-17 20:41:41,857 - lava_callback - INFO - Received callback for node 69407280575bc060ce15ac6d INFO: 172.21.0.1:34986 - "POST /node/69407280575bc060ce15ac6d HTTP/1.0" 202 Accepted 2025-12-17 20:41:42,041 - lava_callback - INFO - Received callback for node 69407279575bc060ce15ac5e INFO: 172.21.0.1:35002 - "POST /node/69407279575bc060ce15ac5e HTTP/1.0" 202 Accepted 2025-12-17 20:41:42,693 - lava_callback - INFO - Received callback for node 6940727a575bc060ce15ac61 INFO: 172.21.0.1:35016 - "POST /node/6940727a575bc060ce15ac61 HTTP/1.0" 202 Accepted 2025-12-17 20:41:43,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407151575bc060ce15aa9a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35032 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:41:47,088 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407151575bc060ce15aa9a 2025-12-17 20:41:47,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:41:49,119 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408623575bc060ce15df03'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:41:49,119 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408623575bc060ce15df03 2025-12-17 20:41:49,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:41:51,899 - lava_callback - ERROR - Error processing callback for node 69407105575bc060ce15a9ba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407105575bc060ce15a9ba (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407105575bc060ce15a9ba (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407105575bc060ce15a9ba (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:41:51,901 - lava_callback - INFO - Starting async job submit for node 69408683575bc060ce15e082 2025-12-17 20:41:51,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:41:52,222 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407142575bc060ce15aa71 2025-12-17 20:41:52,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:41:53,124 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407144575bc060ce15aa79 2025-12-17 20:41:53,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:41:54,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407168575bc060ce15aab6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:41:54,916 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407168575bc060ce15aab6 2025-12-17 20:41:54,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:54248 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:41:58,345 - lava_callback - INFO - Received callback for node 6940727c575bc060ce15ac64 INFO: 172.21.0.1:54250 - "POST /node/6940727c575bc060ce15ac64 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:46700 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:42:06,079 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085aa575bc060ce15dce8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46706 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:42:08,944 - lava_callback - ERROR - Error processing callback for node 694070d0575bc060ce15a97a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d0575bc060ce15a97a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d0575bc060ce15a97a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d0575bc060ce15a97a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:42:08,946 - lava_callback - INFO - Starting async job submit for node 69407180575bc060ce15aaf0 2025-12-17 20:42:08,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:42:11,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407169575bc060ce15aab9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:42:11,982 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407169575bc060ce15aab9 2025-12-17 20:42:11,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:42:12,528 - lava_callback - ERROR - Error processing callback for node 69407114575bc060ce15a9d2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407114575bc060ce15a9d2 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407114575bc060ce15a9d2 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407114575bc060ce15a9d2 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:42:12,530 - lava_callback - INFO - Starting async job submit for node 69407181575bc060ce15aafb 2025-12-17 20:42:12,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:42:12,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863a575bc060ce15df89'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:42:12,799 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940863a575bc060ce15df89 2025-12-17 20:42:12,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:49202 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:42:15,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407154575bc060ce15aaa0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49212 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:42:17,740 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407154575bc060ce15aaa0 2025-12-17 20:42:17,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:42:19,229 - lava_callback - INFO - Received callback for node 694070d3575bc060ce15a984 INFO: 172.21.0.1:49228 - "POST /node/694070d3575bc060ce15a984 HTTP/1.0" 202 Accepted 2025-12-17 20:42:22,080 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694085aa575bc060ce15dce8 2025-12-17 20:42:22,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:42:23,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940714a575bc060ce15aa8a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51236 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:42:29,028 - lava_callback - INFO - Received callback for node 69407d88575bc060ce15c12d INFO: 172.21.0.1:51244 - "POST /node/69407d88575bc060ce15c12d HTTP/1.0" 202 Accepted 2025-12-17 20:42:30,160 - lava_callback - INFO - Received callback for node 69407cc4575bc060ce15beac INFO: 172.21.0.1:36556 - "POST /node/69407cc4575bc060ce15beac HTTP/1.0" 202 Accepted 2025-12-17 20:42:30,509 - lava_callback - INFO - Received callback for node 694072b2575bc060ce15acb0 INFO: 172.21.0.1:36572 - "POST /node/694072b2575bc060ce15acb0 HTTP/1.0" 202 Accepted 2025-12-17 20:42:30,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407152575bc060ce15aa9d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:42:31,171 - lava_callback - INFO - Received callback for node 694072b5575bc060ce15acb3 INFO: 172.21.0.1:36586 - "POST /node/694072b5575bc060ce15acb3 HTTP/1.0" 202 Accepted 2025-12-17 20:42:34,841 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407152575bc060ce15aa9d 2025-12-17 20:42:34,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:36596 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:42:39,836 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940714a575bc060ce15aa8a 2025-12-17 20:42:39,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:42:40,432 - lava_callback - ERROR - Error processing callback for node 6940741d575bc060ce15ae6b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940741d575bc060ce15ae6b (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940741d575bc060ce15ae6b (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940741d575bc060ce15ae6b (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:42:40,434 - lava_callback - INFO - Starting async job submit for node 69407183575bc060ce15aafe 2025-12-17 20:42:40,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:42:41,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070cc575bc060ce15a96f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:42:42,218 - lava_callback - INFO - Received callback for node 694072bf575bc060ce15accc INFO: 172.21.0.1:60560 - "POST /node/694072bf575bc060ce15accc HTTP/1.0" 202 Accepted 2025-12-17 20:42:43,000 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070cc575bc060ce15a96f 2025-12-17 20:42:43,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:42:43,080 - lava_callback - INFO - Received callback for node 694072c0575bc060ce15accf INFO: 172.21.0.1:60566 - "POST /node/694072c0575bc060ce15accf HTTP/1.0" 202 Accepted 2025-12-17 20:42:43,787 - lava_callback - INFO - Received callback for node 694072c2575bc060ce15acd6 INFO: 172.21.0.1:60568 - "POST /node/694072c2575bc060ce15acd6 HTTP/1.0" 202 Accepted 2025-12-17 20:42:44,578 - lava_callback - INFO - Received callback for node 694072c3575bc060ce15acd9 INFO: 172.21.0.1:60584 - "POST /node/694072c3575bc060ce15acd9 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:60590 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:42:47,119 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407151575bc060ce15aa9a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:42:48,621 - lava_callback - INFO - Received callback for node 694072ba575bc060ce15acc3 INFO: 172.21.0.1:60604 - "POST /node/694072ba575bc060ce15acc3 HTTP/1.0" 202 Accepted 2025-12-17 20:42:49,173 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408623575bc060ce15df03'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:42:50,507 - lava_callback - INFO - Received callback for node 694072b6575bc060ce15acba INFO: 172.21.0.1:39120 - "POST /node/694072b6575bc060ce15acba HTTP/1.0" 202 Accepted 2025-12-17 20:42:51,169 - lava_callback - INFO - Received callback for node 694072b7575bc060ce15acbd INFO: 172.21.0.1:39132 - "POST /node/694072b7575bc060ce15acbd HTTP/1.0" 202 Accepted 2025-12-17 20:42:51,175 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408623575bc060ce15df03 2025-12-17 20:42:51,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:42:51,835 - lava_callback - INFO - Received callback for node 694072b9575bc060ce15acc0 INFO: 172.21.0.1:39146 - "POST /node/694072b9575bc060ce15acc0 HTTP/1.0" 202 Accepted 2025-12-17 20:42:52,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408683575bc060ce15e082'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:42:52,028 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408683575bc060ce15e082 2025-12-17 20:42:52,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:42:52,287 - lava_callback - ERROR - Error processing callback for node 69407142575bc060ce15aa71: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407142575bc060ce15aa71 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407142575bc060ce15aa71 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407142575bc060ce15aa71 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:42:52,289 - lava_callback - INFO - Starting async job submit for node 69407184575bc060ce15ab01 2025-12-17 20:42:52,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:42:53,190 - lava_callback - ERROR - Error processing callback for node 69407144575bc060ce15aa79: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407144575bc060ce15aa79 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407144575bc060ce15aa79 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407144575bc060ce15aa79 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:42:53,192 - lava_callback - INFO - Starting async job submit for node 69407185575bc060ce15ab08 2025-12-17 20:42:53,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:42:54,479 - lava_callback - INFO - Received callback for node 69407e04575bc060ce15c205 INFO: 172.21.0.1:39158 - "POST /node/69407e04575bc060ce15c205 HTTP/1.0" 202 Accepted 2025-12-17 20:42:54,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407168575bc060ce15aab6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:42:55,120 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407151575bc060ce15aa9a 2025-12-17 20:42:55,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:39172 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:42:56,938 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407168575bc060ce15aab6 2025-12-17 20:42:56,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:50490 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:43:01,863 - lava_callback - INFO - Received callback for node 694072c4575bc060ce15acdc INFO: 172.21.0.1:50504 - "POST /node/694072c4575bc060ce15acdc HTTP/1.0" 202 Accepted 2025-12-17 20:43:02,676 - lava_callback - INFO - Received callback for node 694072c5575bc060ce15acdf INFO: 172.21.0.1:50518 - "POST /node/694072c5575bc060ce15acdf HTTP/1.0" 202 Accepted 2025-12-17 20:43:03,648 - lava_callback - INFO - Received callback for node 694072c7575bc060ce15ace2 INFO: 172.21.0.1:50524 - "POST /node/694072c7575bc060ce15ace2 HTTP/1.0" 202 Accepted 2025-12-17 20:43:04,412 - lava_callback - INFO - Received callback for node 694072c8575bc060ce15ace9 INFO: 172.21.0.1:50528 - "POST /node/694072c8575bc060ce15ace9 HTTP/1.0" 202 Accepted 2025-12-17 20:43:05,197 - lava_callback - INFO - Received callback for node 694072c9575bc060ce15acec INFO: 172.21.0.1:50536 - "POST /node/694072c9575bc060ce15acec HTTP/1.0" 202 Accepted 2025-12-17 20:43:05,986 - lava_callback - INFO - Received callback for node 694072ca575bc060ce15acfa INFO: 172.21.0.1:50542 - "POST /node/694072ca575bc060ce15acfa HTTP/1.0" 202 Accepted INFO: 172.21.0.1:50554 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:43:06,721 - lava_callback - INFO - Received callback for node 694072cc575bc060ce15acfd INFO: 172.21.0.1:50564 - "POST /node/694072cc575bc060ce15acfd HTTP/1.0" 202 Accepted 2025-12-17 20:43:07,472 - lava_callback - INFO - Received callback for node 694072cd575bc060ce15ad00 INFO: 172.21.0.1:50580 - "POST /node/694072cd575bc060ce15ad00 HTTP/1.0" 202 Accepted 2025-12-17 20:43:09,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407180575bc060ce15aaf0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:43:09,031 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407180575bc060ce15aaf0 2025-12-17 20:43:09,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:43:12,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407169575bc060ce15aab9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:43:12,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407181575bc060ce15aafb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:43:12,572 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407181575bc060ce15aafb 2025-12-17 20:43:12,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:43:12,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863a575bc060ce15df89'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:43:14,038 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407169575bc060ce15aab9 2025-12-17 20:43:14,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:43:14,864 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940863a575bc060ce15df89 2025-12-17 20:43:14,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:40790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40792 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:43:17,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407154575bc060ce15aaa0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:43:21,824 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407154575bc060ce15aaa0 2025-12-17 20:43:21,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:43:22,144 - lava_callback - ERROR - Error processing callback for node 694085aa575bc060ce15dce8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085aa575bc060ce15dce8 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085aa575bc060ce15dce8 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085aa575bc060ce15dce8 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:43:22,147 - lava_callback - INFO - Starting async job submit for node 69407187575bc060ce15ab0b 2025-12-17 20:43:22,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:41142 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:43:34,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407152575bc060ce15aa9d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38958 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:43:39,873 - lava_callback - ERROR - Error processing callback for node 6940714a575bc060ce15aa8a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940714a575bc060ce15aa8a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940714a575bc060ce15aa8a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940714a575bc060ce15aa8a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:43:39,876 - lava_callback - INFO - Starting async job submit for node 69407188575bc060ce15ab0e 2025-12-17 20:43:39,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:43:40,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407183575bc060ce15aafe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:43:40,472 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407183575bc060ce15aafe 2025-12-17 20:43:40,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:43:41,930 - lava_callback - INFO - Received callback for node 694072ec575bc060ce15ad72 INFO: 172.21.0.1:59722 - "POST /node/694072ec575bc060ce15ad72 HTTP/1.0" 202 Accepted 2025-12-17 20:43:42,915 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407152575bc060ce15aa9d 2025-12-17 20:43:42,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:43:43,055 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070cc575bc060ce15a96f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:43:45,127 - lava_callback - INFO - Received callback for node 6940788b575bc060ce15b4e8 INFO: 172.21.0.1:59724 - "POST /node/6940788b575bc060ce15b4e8 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:59740 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:43:47,056 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070cc575bc060ce15a96f 2025-12-17 20:43:47,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:43:49,427 - lava_callback - INFO - Received callback for node 69407e07575bc060ce15c210 INFO: 172.21.0.1:59752 - "POST /node/69407e07575bc060ce15c210 HTTP/1.0" 202 Accepted 2025-12-17 20:43:51,247 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408623575bc060ce15df03'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:43:52,094 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408683575bc060ce15e082'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:43:52,352 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407184575bc060ce15ab01'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:43:52,352 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407184575bc060ce15ab01 2025-12-17 20:43:52,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:43:53,259 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407185575bc060ce15ab08'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:43:53,259 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407185575bc060ce15ab08 2025-12-17 20:43:53,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:43:54,095 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408683575bc060ce15e082 2025-12-17 20:43:54,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:43:55,163 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407151575bc060ce15aa9a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:43:55,248 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408623575bc060ce15df03 2025-12-17 20:43:55,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:45674 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:43:56,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407168575bc060ce15aab6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:43:59,041 - lava_callback - INFO - Received callback for node 694070d4575bc060ce15a987 INFO: 172.21.0.1:45690 - "POST /node/694070d4575bc060ce15a987 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:51542 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:44:00,996 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407168575bc060ce15aab6 2025-12-17 20:44:00,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:51548 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:44:09,065 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407180575bc060ce15aaf0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:44:11,065 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407180575bc060ce15aaf0 2025-12-17 20:44:11,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:44:11,164 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407151575bc060ce15aa9a 2025-12-17 20:44:11,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:44:12,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407181575bc060ce15aafb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:44:14,106 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407169575bc060ce15aab9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:44:14,630 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407181575bc060ce15aafb 2025-12-17 20:44:14,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:44:14,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863a575bc060ce15df89'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41436 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:44:18,106 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407169575bc060ce15aab9 2025-12-17 20:44:18,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:44:18,934 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940863a575bc060ce15df89 2025-12-17 20:44:18,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:44:20,067 - lava_callback - INFO - Received callback for node 69406efe575bc060ce15a521 INFO: 172.21.0.1:42994 - "POST /node/69406efe575bc060ce15a521 HTTP/1.0" 202 Accepted 2025-12-17 20:44:21,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407154575bc060ce15aaa0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:44:22,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407187575bc060ce15ab0b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:44:22,196 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407187575bc060ce15ab0b 2025-12-17 20:44:22,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:43008 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:44:29,916 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407154575bc060ce15aaa0 2025-12-17 20:44:29,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:34242 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:44:39,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407188575bc060ce15ab0e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:44:39,941 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407188575bc060ce15ab0e 2025-12-17 20:44:39,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:44:40,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407183575bc060ce15aafe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:44:42,535 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407183575bc060ce15aafe 2025-12-17 20:44:42,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:44:42,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407152575bc060ce15aa9d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36044 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:44:47,119 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070cc575bc060ce15a96f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:44:52,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407184575bc060ce15ab01'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:44:53,324 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407185575bc060ce15ab08'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:44:54,141 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408683575bc060ce15e082'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:44:54,400 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407184575bc060ce15ab01 2025-12-17 20:44:54,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:44:55,120 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070cc575bc060ce15a96f 2025-12-17 20:44:55,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:44:55,303 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408623575bc060ce15df03'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:44:55,325 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407185575bc060ce15ab08 2025-12-17 20:44:55,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:58456 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:44:58,141 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408683575bc060ce15e082 2025-12-17 20:44:58,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:44:58,978 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407152575bc060ce15aa9d 2025-12-17 20:44:58,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:38410 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:45:01,081 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407168575bc060ce15aab6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:45:03,303 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408623575bc060ce15df03 2025-12-17 20:45:03,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:38424 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:45:09,082 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407168575bc060ce15aab6 2025-12-17 20:45:09,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:45:11,127 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407180575bc060ce15aaf0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:45:11,231 - lava_callback - ERROR - Error processing callback for node 69407151575bc060ce15aa9a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407151575bc060ce15aa9a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407151575bc060ce15aa9a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407151575bc060ce15aa9a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:45:11,234 - lava_callback - INFO - Starting async job submit for node 69407189575bc060ce15ab11 2025-12-17 20:45:11,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:45:14,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407181575bc060ce15aafb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:45:15,127 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407180575bc060ce15aaf0 2025-12-17 20:45:15,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:45438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45446 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:45:18,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407169575bc060ce15aab9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:45:18,707 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407181575bc060ce15aafb 2025-12-17 20:45:18,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:45:18,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863a575bc060ce15df89'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:45:22,250 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407187575bc060ce15ab0b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:45:24,250 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407187575bc060ce15ab0b 2025-12-17 20:45:24,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:45:26,194 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407169575bc060ce15aab9 2025-12-17 20:45:26,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:56202 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:45:26,993 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940863a575bc060ce15df89 2025-12-17 20:45:26,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:45:29,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407154575bc060ce15aaa0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43592 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:45:37,205 - lava_callback - INFO - Received callback for node 6940c5261e3c24e95681d67d INFO: 172.21.0.1:43600 - "POST /node/6940c5261e3c24e95681d67d HTTP/1.0" 202 Accepted 2025-12-17 20:45:39,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407188575bc060ce15ab0e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:45:41,980 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407188575bc060ce15ab0e 2025-12-17 20:45:41,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:45:42,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407183575bc060ce15aafe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:45:45,982 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407154575bc060ce15aaa0 2025-12-17 20:45:45,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:40120 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:45:46,584 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407183575bc060ce15aafe 2025-12-17 20:45:46,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:45:54,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407184575bc060ce15ab01'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:45:55,196 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070cc575bc060ce15a96f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:45:55,352 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407185575bc060ce15ab08'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46630 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:45:58,234 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408683575bc060ce15e082'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:45:58,442 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407184575bc060ce15ab01 2025-12-17 20:45:58,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:45:58,498 - lava_callback - INFO - Received callback for node 69407686575bc060ce15b0e8 INFO: 172.21.0.1:46646 - "POST /node/69407686575bc060ce15b0e8 HTTP/1.0" 202 Accepted 2025-12-17 20:45:59,025 - lava_callback - ERROR - Error processing callback for node 69407152575bc060ce15aa9d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407152575bc060ce15aa9d (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407152575bc060ce15aa9d (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407152575bc060ce15aa9d (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:45:59,027 - lava_callback - INFO - Starting async job submit for node 6940718a575bc060ce15ab14 2025-12-17 20:45:59,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:45:59,352 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407185575bc060ce15ab08 2025-12-17 20:45:59,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:41996 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:46:03,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408623575bc060ce15df03'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:46:06,234 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408683575bc060ce15e082 2025-12-17 20:46:06,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:42010 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:46:09,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407168575bc060ce15aab6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:46:10,524 - lava_callback - INFO - Received callback for node 69407165575bc060ce15aab0 INFO: 172.21.0.1:58274 - "POST /node/69407165575bc060ce15aab0 HTTP/1.0" 202 Accepted 2025-12-17 20:46:11,196 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694070cc575bc060ce15a96f 2025-12-17 20:46:11,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:46:11,293 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407189575bc060ce15ab11'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:46:11,293 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407189575bc060ce15ab11 2025-12-17 20:46:11,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:58288 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:46:15,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407180575bc060ce15aaf0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58300 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:46:18,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407181575bc060ce15aafb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:46:18,952 - lava_callback - INFO - Received callback for node 694072bd575bc060ce15acc6 INFO: 172.21.0.1:58310 - "POST /node/694072bd575bc060ce15acc6 HTTP/1.0" 202 Accepted 2025-12-17 20:46:19,359 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408623575bc060ce15df03 2025-12-17 20:46:19,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:46:23,216 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407180575bc060ce15aaf0 2025-12-17 20:46:23,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:46:24,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407187575bc060ce15ab0b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:46:25,104 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407168575bc060ce15aab6 2025-12-17 20:46:25,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:46:26,259 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407169575bc060ce15aab9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53232 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:46:26,768 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407181575bc060ce15aafb 2025-12-17 20:46:26,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:46:27,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863a575bc060ce15df89'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:46:28,321 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407187575bc060ce15ab0b 2025-12-17 20:46:28,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:43278 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:46:42,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407188575bc060ce15ab0e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:46:42,260 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407169575bc060ce15aab9 2025-12-17 20:46:42,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:46:43,059 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940863a575bc060ce15df89 2025-12-17 20:46:43,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:46:46,044 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407188575bc060ce15ab0e 2025-12-17 20:46:46,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:46:46,071 - lava_callback - ERROR - Error processing callback for node 69407154575bc060ce15aaa0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407154575bc060ce15aaa0 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407154575bc060ce15aaa0 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407154575bc060ce15aaa0 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:46:46,074 - lava_callback - INFO - Starting async job submit for node 6940718c575bc060ce15ab17 2025-12-17 20:46:46,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:59964 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:46:46,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407183575bc060ce15aafe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:46:54,602 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407183575bc060ce15aafe 2025-12-17 20:46:54,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:36978 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:46:58,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407184575bc060ce15ab01'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:46:59,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718a575bc060ce15ab14'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:46:59,114 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718a575bc060ce15ab14 2025-12-17 20:46:59,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:46:59,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407185575bc060ce15ab08'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55056 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:47:06,299 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408683575bc060ce15e082'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55062 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:47:06,544 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407184575bc060ce15ab01 2025-12-17 20:47:06,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:47:07,420 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407185575bc060ce15ab08 2025-12-17 20:47:07,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:47:11,245 - lava_callback - ERROR - Error processing callback for node 694070cc575bc060ce15a96f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070cc575bc060ce15a96f (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070cc575bc060ce15a96f (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070cc575bc060ce15a96f (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:47:11,248 - lava_callback - INFO - Starting async job submit for node 6940718d575bc060ce15ab1a 2025-12-17 20:47:11,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:47:11,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407189575bc060ce15ab11'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:47:13,345 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407189575bc060ce15ab11 2025-12-17 20:47:13,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:51266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51278 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:47:18,943 - lava_callback - INFO - Received callback for node 694072be575bc060ce15acc9 INFO: 172.21.0.1:51294 - "POST /node/694072be575bc060ce15acc9 HTTP/1.0" 202 Accepted 2025-12-17 20:47:19,428 - lava_callback - ERROR - Error processing callback for node 69408623575bc060ce15df03: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408623575bc060ce15df03 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408623575bc060ce15df03 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408623575bc060ce15df03 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:47:19,430 - lava_callback - INFO - Starting async job submit for node 6940718e575bc060ce15ab1d 2025-12-17 20:47:19,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:47:22,300 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69408683575bc060ce15e082 2025-12-17 20:47:22,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:47:23,261 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407180575bc060ce15aaf0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:47:25,180 - lava_callback - ERROR - Error processing callback for node 69407168575bc060ce15aab6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407168575bc060ce15aab6 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407168575bc060ce15aab6 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407168575bc060ce15aab6 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:47:25,182 - lava_callback - INFO - Starting async job submit for node 6940718f575bc060ce15ab20 2025-12-17 20:47:25,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:33068 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:47:26,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407181575bc060ce15aafb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:47:28,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407187575bc060ce15ab0b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:47:36,383 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407187575bc060ce15ab0b 2025-12-17 20:47:36,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:44676 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:47:38,842 - lava_callback - INFO - Received callback for node 69407687575bc060ce15b0eb INFO: 172.21.0.1:44678 - "POST /node/69407687575bc060ce15b0eb HTTP/1.0" 202 Accepted 2025-12-17 20:47:39,261 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407180575bc060ce15aaf0 2025-12-17 20:47:39,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:47:42,292 - lava_callback - ERROR - Error processing callback for node 69407169575bc060ce15aab9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407169575bc060ce15aab9 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407169575bc060ce15aab9 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407169575bc060ce15aab9 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:47:42,294 - lava_callback - INFO - Starting async job submit for node 6940741d575bc060ce15ae76 2025-12-17 20:47:42,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:47:42,811 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407181575bc060ce15aafb 2025-12-17 20:47:42,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:47:43,129 - lava_callback - ERROR - Error processing callback for node 6940863a575bc060ce15df89: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940863a575bc060ce15df89 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940863a575bc060ce15df89 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940863a575bc060ce15df89 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:47:43,132 - lava_callback - INFO - Starting async job submit for node 69407190575bc060ce15ab23 2025-12-17 20:47:43,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:47:46,110 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407188575bc060ce15ab0e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:47:46,111 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718c575bc060ce15ab17'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:47:46,111 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718c575bc060ce15ab17 2025-12-17 20:47:46,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:54402 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:47:54,110 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407188575bc060ce15ab0e 2025-12-17 20:47:54,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:47:54,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407183575bc060ce15aafe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59332 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:47:59,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718a575bc060ce15ab14'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57914 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:48:01,178 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718a575bc060ce15ab14 2025-12-17 20:48:01,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:57916 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:48:06,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407184575bc060ce15ab01'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:48:07,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407185575bc060ce15ab08'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:48:10,664 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407183575bc060ce15aafe 2025-12-17 20:48:10,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:48:11,038 - lava_callback - INFO - Received callback for node 69407681575bc060ce15b0dd INFO: 172.21.0.1:56476 - "POST /node/69407681575bc060ce15b0dd HTTP/1.0" 202 Accepted 2025-12-17 20:48:11,292 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718d575bc060ce15ab1a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:48:11,292 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718d575bc060ce15ab1a 2025-12-17 20:48:11,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:48:13,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407189575bc060ce15ab11'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56492 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:48:17,393 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407189575bc060ce15ab11 2025-12-17 20:48:17,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:48:18,625 - lava_callback - INFO - Received callback for node 69407683575bc060ce15b0e0 INFO: 172.21.0.1:56494 - "POST /node/69407683575bc060ce15b0e0 HTTP/1.0" 202 Accepted 2025-12-17 20:48:19,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718e575bc060ce15ab1d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:48:19,480 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718e575bc060ce15ab1d 2025-12-17 20:48:19,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:48:22,352 - lava_callback - ERROR - Error processing callback for node 69408683575bc060ce15e082: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408683575bc060ce15e082 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408683575bc060ce15e082 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408683575bc060ce15e082 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:48:22,354 - lava_callback - INFO - Starting async job submit for node 69407192575bc060ce15ab26 2025-12-17 20:48:22,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:48:22,631 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407184575bc060ce15ab01 2025-12-17 20:48:22,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:48:23,486 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407185575bc060ce15ab08 2025-12-17 20:48:23,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:48:25,269 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718f575bc060ce15ab20'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:48:25,270 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718f575bc060ce15ab20 2025-12-17 20:48:25,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:42862 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:48:36,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407187575bc060ce15ab0b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46002 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:48:39,274 - lava_callback - ERROR - Error processing callback for node 69407180575bc060ce15aaf0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407180575bc060ce15aaf0 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407180575bc060ce15aaf0 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407180575bc060ce15aaf0 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:48:39,277 - lava_callback - INFO - Starting async job submit for node 69407193575bc060ce15ab29 2025-12-17 20:48:39,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:48:42,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741d575bc060ce15ae76'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:48:42,360 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940741d575bc060ce15ae76 2025-12-17 20:48:42,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:48:42,366 - lava_callback - INFO - Received callback for node 6940741e575bc060ce15ae7c INFO: 172.21.0.1:43878 - "POST /node/6940741e575bc060ce15ae7c HTTP/1.0" 202 Accepted 2025-12-17 20:48:42,832 - lava_callback - ERROR - Error processing callback for node 69407181575bc060ce15aafb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407181575bc060ce15aafb (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407181575bc060ce15aafb (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407181575bc060ce15aafb (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:48:42,834 - lava_callback - INFO - Starting async job submit for node 69407194575bc060ce15ab2c 2025-12-17 20:48:42,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:48:43,184 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407190575bc060ce15ab23'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:48:43,184 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407190575bc060ce15ab23 2025-12-17 20:48:43,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:48:43,385 - lava_callback - INFO - Received callback for node 69407a59575bc060ce15b7e9 INFO: 172.21.0.1:43892 - "POST /node/69407a59575bc060ce15b7e9 HTTP/1.0" 202 Accepted 2025-12-17 20:48:46,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718c575bc060ce15ab17'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43908 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:48:48,178 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718c575bc060ce15ab17 2025-12-17 20:48:48,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:48:52,449 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407187575bc060ce15ab0b 2025-12-17 20:48:52,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:48:54,147 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407188575bc060ce15ab0e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:48:54,254 - lava_callback - INFO - Received callback for node 694070c8575bc060ce15a966 INFO: 172.21.0.1:34614 - "POST /node/694070c8575bc060ce15a966 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:34616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56448 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:49:01,226 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718a575bc060ce15ab14'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:49:05,226 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718a575bc060ce15ab14 2025-12-17 20:49:05,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:56460 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:49:10,148 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407188575bc060ce15ab0e 2025-12-17 20:49:10,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:49:10,740 - lava_callback - ERROR - Error processing callback for node 69407183575bc060ce15aafe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407183575bc060ce15aafe (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407183575bc060ce15aafe (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407183575bc060ce15aafe (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:49:10,742 - lava_callback - INFO - Starting async job submit for node 69407195575bc060ce15ab2f 2025-12-17 20:49:10,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:49:11,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718d575bc060ce15ab1a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:49:13,345 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718d575bc060ce15ab1a 2025-12-17 20:49:13,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:53394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53398 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:49:17,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407189575bc060ce15ab11'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:49:18,630 - lava_callback - INFO - Received callback for node 69407684575bc060ce15b0e3 INFO: 172.21.0.1:53408 - "POST /node/69407684575bc060ce15b0e3 HTTP/1.0" 202 Accepted 2025-12-17 20:49:19,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718e575bc060ce15ab1d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:49:20,120 - lava_callback - INFO - Received callback for node 69407688575bc060ce15b0ee INFO: 172.21.0.1:51436 - "POST /node/69407688575bc060ce15b0ee HTTP/1.0" 202 Accepted 2025-12-17 20:49:21,552 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718e575bc060ce15ab1d 2025-12-17 20:49:21,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:49:22,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407192575bc060ce15ab26'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:49:22,422 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407192575bc060ce15ab26 2025-12-17 20:49:22,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:49:22,698 - lava_callback - ERROR - Error processing callback for node 69407184575bc060ce15ab01: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407184575bc060ce15ab01 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407184575bc060ce15ab01 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407184575bc060ce15ab01 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:49:22,700 - lava_callback - INFO - Starting async job submit for node 69407196575bc060ce15ab32 2025-12-17 20:49:22,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:49:23,572 - lava_callback - ERROR - Error processing callback for node 69407185575bc060ce15ab08: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407185575bc060ce15ab08 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407185575bc060ce15ab08 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407185575bc060ce15ab08 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:49:23,574 - lava_callback - INFO - Starting async job submit for node 69407197575bc060ce15ab35 2025-12-17 20:49:23,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:49:25,318 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718f575bc060ce15ab20'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:49:25,462 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407189575bc060ce15ab11 2025-12-17 20:49:25,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:51448 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:49:27,319 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718f575bc060ce15ab20 2025-12-17 20:49:27,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:50582 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:49:39,333 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407193575bc060ce15ab29'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:49:39,333 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407193575bc060ce15ab29 2025-12-17 20:49:39,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:49:40,810 - lava_callback - INFO - Received callback for node 6940708b575bc060ce15a8c1 INFO: 172.21.0.1:54580 - "POST /node/6940708b575bc060ce15a8c1 HTTP/1.0" 202 Accepted 2025-12-17 20:49:42,360 - lava_callback - INFO - Received callback for node 69407458575bc060ce15aea6 INFO: 172.21.0.1:54594 - "POST /node/69407458575bc060ce15aea6 HTTP/1.0" 202 Accepted 2025-12-17 20:49:42,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741d575bc060ce15ae76'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:49:42,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407194575bc060ce15ab2c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:49:42,908 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407194575bc060ce15ab2c 2025-12-17 20:49:42,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:49:43,224 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407190575bc060ce15ab23'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:49:44,448 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940741d575bc060ce15ae76 2025-12-17 20:49:44,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:49:45,224 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407190575bc060ce15ab23 2025-12-17 20:49:45,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:54610 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:49:48,274 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718c575bc060ce15ab17'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:49:52,274 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718c575bc060ce15ab17 2025-12-17 20:49:52,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:49:52,515 - lava_callback - ERROR - Error processing callback for node 69407187575bc060ce15ab0b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407187575bc060ce15ab0b (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407187575bc060ce15ab0b (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407187575bc060ce15ab0b (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:49:52,517 - lava_callback - INFO - Starting async job submit for node 69407198575bc060ce15ab38 2025-12-17 20:49:52,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:48810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34064 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:50:05,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718a575bc060ce15ab14'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34078 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:50:10,224 - lava_callback - ERROR - Error processing callback for node 69407188575bc060ce15ab0e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407188575bc060ce15ab0e (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407188575bc060ce15ab0e (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407188575bc060ce15ab0e (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:50:10,226 - lava_callback - INFO - Starting async job submit for node 6940719a575bc060ce15ab3b 2025-12-17 20:50:10,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:50:10,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407195575bc060ce15ab2f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:50:10,804 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407195575bc060ce15ab2f 2025-12-17 20:50:10,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:50:13,280 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718a575bc060ce15ab14 2025-12-17 20:50:13,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:50:13,407 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718d575bc060ce15ab1a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60082 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:50:17,408 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718d575bc060ce15ab1a 2025-12-17 20:50:17,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:50:21,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718e575bc060ce15ab1d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:50:22,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407192575bc060ce15ab26'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:50:22,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407196575bc060ce15ab32'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:50:22,790 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407196575bc060ce15ab32 2025-12-17 20:50:22,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:50:23,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407197575bc060ce15ab35'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:50:23,641 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407197575bc060ce15ab35 2025-12-17 20:50:23,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:50:24,477 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407192575bc060ce15ab26 2025-12-17 20:50:24,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:50:25,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407189575bc060ce15ab11'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:50:25,596 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718e575bc060ce15ab1d 2025-12-17 20:50:25,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:49918 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:50:27,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718f575bc060ce15ab20'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:50:31,376 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718f575bc060ce15ab20 2025-12-17 20:50:31,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:55692 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:50:39,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407193575bc060ce15ab29'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:50:41,394 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407193575bc060ce15ab29 2025-12-17 20:50:41,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:50:41,512 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407189575bc060ce15ab11 2025-12-17 20:50:41,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:50:42,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407194575bc060ce15ab2c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:50:44,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741d575bc060ce15ae76'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:50:44,949 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407194575bc060ce15ab2c 2025-12-17 20:50:44,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:50:45,296 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407190575bc060ce15ab23'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51878 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:50:48,464 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940741d575bc060ce15ae76 2025-12-17 20:50:48,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:50:49,296 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407190575bc060ce15ab23 2025-12-17 20:50:49,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:50:52,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718c575bc060ce15ab17'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:50:52,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407198575bc060ce15ab38'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:50:52,578 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407198575bc060ce15ab38 2025-12-17 20:50:52,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 INFO: 172.21.0.1:46974 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:50:58,717 - lava_callback - INFO - Received callback for node 69407692575bc060ce15b112 INFO: 172.21.0.1:46976 - "POST /node/69407692575bc060ce15b112 HTTP/1.0" 202 Accepted 2025-12-17 20:51:00,332 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718c575bc060ce15ab17 2025-12-17 20:51:00,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:56810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56818 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:51:10,259 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940719a575bc060ce15ab3b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:51:10,259 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940719a575bc060ce15ab3b 2025-12-17 20:51:10,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:51:10,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407195575bc060ce15ab2f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:51:12,864 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407195575bc060ce15ab2f 2025-12-17 20:51:12,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:51:13,372 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718a575bc060ce15ab14'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43696 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:51:17,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718d575bc060ce15ab1a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:51:22,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407196575bc060ce15ab32'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:51:23,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407197575bc060ce15ab35'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:51:24,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407192575bc060ce15ab26'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:51:24,896 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407196575bc060ce15ab32 2025-12-17 20:51:24,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:51:25,492 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718d575bc060ce15ab1a 2025-12-17 20:51:25,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:51:25,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718e575bc060ce15ab1d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:51:25,705 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407197575bc060ce15ab35 2025-12-17 20:51:25,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:53972 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:51:28,517 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407192575bc060ce15ab26 2025-12-17 20:51:28,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:51:29,373 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718a575bc060ce15ab14 2025-12-17 20:51:29,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:51:31,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718f575bc060ce15ab20'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:51:33,677 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718e575bc060ce15ab1d 2025-12-17 20:51:33,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:34252 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:51:39,424 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718f575bc060ce15ab20 2025-12-17 20:51:39,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:51:41,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407193575bc060ce15ab29'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:51:41,576 - lava_callback - ERROR - Error processing callback for node 69407189575bc060ce15ab11: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407189575bc060ce15ab11 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407189575bc060ce15ab11 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407189575bc060ce15ab11 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:51:41,577 - lava_callback - INFO - Starting async job submit for node 6940719b575bc060ce15ab3e 2025-12-17 20:51:41,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:51:45,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407194575bc060ce15ab2c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:51:45,425 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407193575bc060ce15ab29 2025-12-17 20:51:45,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:40414 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:51:48,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741d575bc060ce15ae76'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:51:49,003 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407194575bc060ce15ab2c 2025-12-17 20:51:49,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:51:49,385 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407190575bc060ce15ab23'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:51:49,758 - lava_callback - INFO - Received callback for node 694087c8575bc060ce15e4c2 INFO: 172.21.0.1:59656 - "POST /node/694087c8575bc060ce15e4c2 HTTP/1.0" 202 Accepted 2025-12-17 20:51:52,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407198575bc060ce15ab38'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:51:54,642 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407198575bc060ce15ab38 2025-12-17 20:51:54,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:51:56,492 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940741d575bc060ce15ae76 2025-12-17 20:51:56,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:59662 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:51:57,385 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407190575bc060ce15ab23 2025-12-17 20:51:57,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:52:00,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718c575bc060ce15ab17'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52132 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:52:10,322 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940719a575bc060ce15ab3b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:52:12,322 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940719a575bc060ce15ab3b 2025-12-17 20:52:12,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:52:12,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407195575bc060ce15ab2f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:52:14,795 - lava_callback - INFO - Received callback for node 694087cd575bc060ce15e4d4 INFO: 172.21.0.1:39150 - "POST /node/694087cd575bc060ce15e4d4 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:39162 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:52:16,400 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718c575bc060ce15ab17 2025-12-17 20:52:16,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:39168 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:52:16,917 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407195575bc060ce15ab2f 2025-12-17 20:52:16,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:52:18,876 - lava_callback - INFO - Received callback for node 69407693575bc060ce15b119 INFO: 172.21.0.1:39180 - "POST /node/69407693575bc060ce15b119 HTTP/1.0" 202 Accepted 2025-12-17 20:52:25,135 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407196575bc060ce15ab32'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:52:25,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718d575bc060ce15ab1a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:52:25,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407197575bc060ce15ab35'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41944 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:52:28,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407192575bc060ce15ab26'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:52:28,641 - lava_callback - INFO - Received callback for node 694087d4575bc060ce15e4dd INFO: 172.21.0.1:41960 - "POST /node/694087d4575bc060ce15e4dd HTTP/1.0" 202 Accepted 2025-12-17 20:52:29,136 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407196575bc060ce15ab32 2025-12-17 20:52:29,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:52:29,435 - lava_callback - ERROR - Error processing callback for node 6940718a575bc060ce15ab14: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940718a575bc060ce15ab14 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940718a575bc060ce15ab14 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940718a575bc060ce15ab14 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:52:29,437 - lava_callback - INFO - Starting async job submit for node 694071a1575bc060ce15ab41 2025-12-17 20:52:29,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:52:29,753 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407197575bc060ce15ab35 2025-12-17 20:52:29,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:52:30,839 - lava_callback - INFO - Received callback for node 6940768c575bc060ce15b0f7 INFO: 172.21.0.1:48360 - "POST /node/6940768c575bc060ce15b0f7 HTTP/1.0" 202 Accepted 2025-12-17 20:52:33,647 - lava_callback - INFO - Received callback for node 6940881e575bc060ce15e5e2 INFO: 172.21.0.1:48372 - "POST /node/6940881e575bc060ce15e5e2 HTTP/1.0" 202 Accepted 2025-12-17 20:52:33,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718e575bc060ce15ab1d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48386 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:52:36,579 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407192575bc060ce15ab26 2025-12-17 20:52:36,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:52:39,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940718f575bc060ce15ab20'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:52:41,568 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718d575bc060ce15ab1a 2025-12-17 20:52:41,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:52:41,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940719b575bc060ce15ab3e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:52:41,624 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940719b575bc060ce15ab3e 2025-12-17 20:52:41,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:52:45,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407193575bc060ce15ab29'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39348 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:52:49,065 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407194575bc060ce15ab2c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:52:49,741 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718e575bc060ce15ab1d 2025-12-17 20:52:49,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:52:53,472 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407193575bc060ce15ab29 2025-12-17 20:52:53,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:52:54,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407198575bc060ce15ab38'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:52:55,472 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940718f575bc060ce15ab20 2025-12-17 20:52:55,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:53562 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:52:56,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741d575bc060ce15ae76'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:52:57,065 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407194575bc060ce15ab2c 2025-12-17 20:52:57,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:52:57,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407190575bc060ce15ab23'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:52:58,708 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407198575bc060ce15ab38 2025-12-17 20:52:58,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 INFO: 172.21.0.1:51844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51846 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:53:12,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940719a575bc060ce15ab3b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:53:12,580 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940741d575bc060ce15ae76 2025-12-17 20:53:12,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:53:13,453 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407190575bc060ce15ab23 2025-12-17 20:53:13,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 INFO: 172.21.0.1:42874 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:53:16,461 - lava_callback - ERROR - Error processing callback for node 6940718c575bc060ce15ab17: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940718c575bc060ce15ab17 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940718c575bc060ce15ab17 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940718c575bc060ce15ab17 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:53:16,463 - lava_callback - INFO - Starting async job submit for node 6940780c575bc060ce15b3fd 2025-12-17 20:53:16,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:42888 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:53:16,517 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940719a575bc060ce15ab3b 2025-12-17 20:53:16,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:53:17,167 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407195575bc060ce15ab2f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:53:19,188 - lava_callback - INFO - Received callback for node 6940768d575bc060ce15b0fa INFO: 172.21.0.1:42894 - "POST /node/6940768d575bc060ce15b0fa HTTP/1.0" 202 Accepted 2025-12-17 20:53:23,410 - lava_callback - INFO - Received callback for node 69407bed575bc060ce15bc22 INFO: 172.21.0.1:59688 - "POST /node/69407bed575bc060ce15bc22 HTTP/1.0" 202 Accepted 2025-12-17 20:53:25,168 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407195575bc060ce15ab2f 2025-12-17 20:53:25,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 INFO: 172.21.0.1:59702 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:53:29,190 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407196575bc060ce15ab32'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:53:29,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071a1575bc060ce15ab41'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:53:29,507 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694071a1575bc060ce15ab41 2025-12-17 20:53:29,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:53:29,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407197575bc060ce15ab35'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:53:33,850 - lava_callback - INFO - Received callback for node 69408a8a575bc060ce15eea5 INFO: 172.21.0.1:46402 - "POST /node/69408a8a575bc060ce15eea5 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:46406 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:53:36,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407192575bc060ce15ab26'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:53:37,191 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407196575bc060ce15ab32 2025-12-17 20:53:37,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:53:37,807 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407197575bc060ce15ab35 2025-12-17 20:53:37,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:53:39,196 - lava_callback - INFO - Received callback for node 6940768f575bc060ce15b101 INFO: 172.21.0.1:46414 - "POST /node/6940768f575bc060ce15b101 HTTP/1.0" 202 Accepted 2025-12-17 20:53:41,628 - lava_callback - ERROR - Error processing callback for node 6940718d575bc060ce15ab1a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940718d575bc060ce15ab1a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940718d575bc060ce15ab1a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940718d575bc060ce15ab1a (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:53:41,630 - lava_callback - INFO - Starting async job submit for node 694071cd575bc060ce15abd5 2025-12-17 20:53:41,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:53:41,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940719b575bc060ce15ab3e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:53:43,698 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940719b575bc060ce15ab3e 2025-12-17 20:53:43,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:49972 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:53:49,943 - lava_callback - ERROR - Error processing callback for node 6940718e575bc060ce15ab1d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940718e575bc060ce15ab1d (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940718e575bc060ce15ab1d (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940718e575bc060ce15ab1d (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:53:49,945 - lava_callback - INFO - Starting async job submit for node 694087b7575bc060ce15e4ae 2025-12-17 20:53:50,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:53:52,776 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407192575bc060ce15ab26 2025-12-17 20:53:52,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:53:53,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407193575bc060ce15ab29'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:53:55,708 - lava_callback - ERROR - Error processing callback for node 6940718f575bc060ce15ab20: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940718f575bc060ce15ab20 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940718f575bc060ce15ab20 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940718f575bc060ce15ab20 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:53:55,710 - lava_callback - INFO - Starting async job submit for node 69407113575bc060ce15a9cf 2025-12-17 20:53:55,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:47462 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:53:57,301 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407194575bc060ce15ab2c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:53:58,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407198575bc060ce15ab38'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:53:59,029 - lava_callback - INFO - Received callback for node 69407695575bc060ce15b11c INFO: 172.21.0.1:47468 - "POST /node/69407695575bc060ce15b11c HTTP/1.0" 202 Accepted INFO: 172.21.0.1:54922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54934 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:54:06,741 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407198575bc060ce15ab38 2025-12-17 20:54:06,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:54:09,558 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407193575bc060ce15ab29 2025-12-17 20:54:09,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:54:12,631 - lava_callback - ERROR - Error processing callback for node 6940741d575bc060ce15ae76: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940741d575bc060ce15ae76 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940741d575bc060ce15ae76 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940741d575bc060ce15ae76 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:54:12,633 - lava_callback - INFO - Starting async job submit for node 69407cc3575bc060ce15bea3 2025-12-17 20:54:12,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:54:13,302 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407194575bc060ce15ab2c 2025-12-17 20:54:13,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:54:13,464 - lava_callback - ERROR - Error processing callback for node 69407190575bc060ce15ab23: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407190575bc060ce15ab23 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407190575bc060ce15ab23 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407190575bc060ce15ab23 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:54:13,466 - lava_callback - INFO - Starting async job submit for node 69407272575bc060ce15ac47 2025-12-17 20:54:13,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:41362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41374 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:54:16,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940780c575bc060ce15b3fd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:54:16,549 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940780c575bc060ce15b3fd 2025-12-17 20:54:16,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:54:16,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940719a575bc060ce15ab3b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:54:24,550 - urllib3.connectionpool - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940719a575bc060ce15ab3b 2025-12-17 20:54:24,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (5): staging.kernelci.org:9000 2025-12-17 20:54:25,234 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407195575bc060ce15ab2f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46618 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:54:29,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071a1575bc060ce15ab41'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:54:31,552 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694071a1575bc060ce15ab41 2025-12-17 20:54:31,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 INFO: 172.21.0.1:59878 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:54:37,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407196575bc060ce15ab32'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:54:37,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407197575bc060ce15ab35'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:54:41,234 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407195575bc060ce15ab2f 2025-12-17 20:54:41,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:54:41,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071cd575bc060ce15abd5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:54:41,682 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694071cd575bc060ce15abd5 2025-12-17 20:54:41,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:54:43,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940719b575bc060ce15ab3e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46222 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:54:47,775 - urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940719b575bc060ce15ab3e 2025-12-17 20:54:47,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (4): staging.kernelci.org:9000 2025-12-17 20:54:50,075 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b7575bc060ce15e4ae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:54:50,076 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/694087b7575bc060ce15e4ae 2025-12-17 20:54:50,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:54:52,850 - lava_callback - ERROR - Error processing callback for node 69407192575bc060ce15ab26: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407192575bc060ce15ab26 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407192575bc060ce15ab26 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407192575bc060ce15ab26 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:54:52,852 - lava_callback - INFO - Starting async job submit for node 69407cc3575bc060ce15bea9 2025-12-17 20:54:52,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:54:53,244 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407196575bc060ce15ab32 2025-12-17 20:54:53,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:54:53,859 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407197575bc060ce15ab35 2025-12-17 20:54:53,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:54:55,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940719a575bc060ce15ab3b HTTP/1.1" 502 150 2025-12-17 20:54:55,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940719a575bc060ce15ab3b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:54:55,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407113575bc060ce15a9cf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:54:55,799 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407113575bc060ce15a9cf 2025-12-17 20:54:55,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:54:55,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407113575bc060ce15a9cf HTTP/1.1" 502 150 2025-12-17 20:54:55,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407113575bc060ce15a9cf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54544 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:54:56,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407195575bc060ce15ab2f HTTP/1.1" 502 150 2025-12-17 20:54:56,688 - lava_callback - ERROR - Error processing callback for node 69407195575bc060ce15ab2f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407195575bc060ce15ab2f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407195575bc060ce15ab2f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407195575bc060ce15ab2f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:54:56,690 - lava_callback - INFO - Starting async job submit for node 69407274575bc060ce15ac4e 2025-12-17 20:54:56,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:54:56,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407274575bc060ce15ac4e HTTP/1.1" 502 150 2025-12-17 20:54:56,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407274575bc060ce15ac4e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:54:56,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407274575bc060ce15ac4e 2025-12-17 20:54:56,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407274575bc060ce15ac4e HTTP/1.1" 502 150 2025-12-17 20:54:56,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407274575bc060ce15ac4e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:54:57,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407113575bc060ce15a9cf 2025-12-17 20:54:57,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407113575bc060ce15a9cf HTTP/1.1" 502 150 2025-12-17 20:54:57,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407113575bc060ce15a9cf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:54:58,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407274575bc060ce15ac4e 2025-12-17 20:54:58,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407274575bc060ce15ac4e HTTP/1.1" 502 150 2025-12-17 20:54:58,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407274575bc060ce15ac4e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37798 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:55:01,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407113575bc060ce15a9cf 2025-12-17 20:55:02,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071a1575bc060ce15ab41 HTTP/1.1" 502 150 2025-12-17 20:55:02,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071a1575bc060ce15ab41'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:02,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407274575bc060ce15ac4e 2025-12-17 20:55:06,320 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694071a1575bc060ce15ab41 INFO: 172.21.0.1:37814 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:55:06,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407198575bc060ce15ab38'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:09,603 - lava_callback - ERROR - Error processing callback for node 69407193575bc060ce15ab29: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407193575bc060ce15ab29 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407193575bc060ce15ab29 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407193575bc060ce15ab29 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:55:09,605 - lava_callback - INFO - Starting async job submit for node 69407275575bc060ce15ac55 2025-12-17 20:55:09,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:55:11,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940719a575bc060ce15ab3b 2025-12-17 20:55:12,018 - lava_callback - INFO - Received callback for node 69408830575bc060ce15e64f INFO: 172.21.0.1:55608 - "POST /node/69408830575bc060ce15e64f HTTP/1.0" 202 Accepted 2025-12-17 20:55:12,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071cd575bc060ce15abd5 HTTP/1.1" 502 150 2025-12-17 20:55:12,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071cd575bc060ce15abd5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:12,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cc3575bc060ce15bea3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:12,731 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407cc3575bc060ce15bea3 2025-12-17 20:55:12,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:55:12,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc3575bc060ce15bea3 HTTP/1.1" 502 150 2025-12-17 20:55:12,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cc3575bc060ce15bea3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:13,334 - lava_callback - ERROR - Error processing callback for node 69407194575bc060ce15ab2c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407194575bc060ce15ab2c (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 571, in getresponse httplib_response = super().getresponse() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1430, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 720, in readinto return self._sock.recv_into(b) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1251, in recv_into return self.read(nbytes, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1103, in read return self._sslobj.read(len, buffer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407194575bc060ce15ab2c (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407194575bc060ce15ab2c (Caused by ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")) 2025-12-17 20:55:13,336 - lava_callback - INFO - Starting async job submit for node 69407277575bc060ce15ac58 2025-12-17 20:55:13,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:55:13,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407277575bc060ce15ac58 HTTP/1.1" 502 150 2025-12-17 20:55:13,364 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407277575bc060ce15ac58'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:13,364 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407277575bc060ce15ac58 2025-12-17 20:55:13,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407277575bc060ce15ac58 HTTP/1.1" 502 150 2025-12-17 20:55:13,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407277575bc060ce15ac58'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:13,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407272575bc060ce15ac47'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:13,532 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407272575bc060ce15ac47 2025-12-17 20:55:13,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 20:55:13,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407272575bc060ce15ac47 HTTP/1.1" 502 150 2025-12-17 20:55:13,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407272575bc060ce15ac47'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:14,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694071cd575bc060ce15abd5 2025-12-17 20:55:14,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071cd575bc060ce15abd5 HTTP/1.1" 502 150 2025-12-17 20:55:14,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071cd575bc060ce15abd5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:14,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cc3575bc060ce15bea3 2025-12-17 20:55:14,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc3575bc060ce15bea3 HTTP/1.1" 502 150 2025-12-17 20:55:14,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cc3575bc060ce15bea3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55618 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:55:15,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407277575bc060ce15ac58 2025-12-17 20:55:15,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407277575bc060ce15ac58 HTTP/1.1" 502 150 2025-12-17 20:55:15,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407277575bc060ce15ac58'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:15,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407272575bc060ce15ac47 2025-12-17 20:55:15,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407272575bc060ce15ac47 HTTP/1.1" 502 150 2025-12-17 20:55:15,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407272575bc060ce15ac47'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55632 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:55:16,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940780c575bc060ce15b3fd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:17,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407113575bc060ce15a9cf HTTP/1.1" 502 150 2025-12-17 20:55:17,168 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407113575bc060ce15a9cf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:18,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407274575bc060ce15ac4e HTTP/1.1" 502 150 2025-12-17 20:55:18,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407274575bc060ce15ac4e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:18,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940719b575bc060ce15ab3e HTTP/1.1" 502 150 2025-12-17 20:55:18,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940719b575bc060ce15ab3e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:18,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694071cd575bc060ce15abd5 2025-12-17 20:55:18,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071cd575bc060ce15abd5 HTTP/1.1" 502 150 2025-12-17 20:55:18,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071cd575bc060ce15abd5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:18,632 - urllib3.connectionpool - WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/6940780c575bc060ce15b3fd 2025-12-17 20:55:18,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (3): staging.kernelci.org:9000 2025-12-17 20:55:18,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940780c575bc060ce15b3fd HTTP/1.1" 502 150 2025-12-17 20:55:18,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940780c575bc060ce15b3fd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:18,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cc3575bc060ce15bea3 2025-12-17 20:55:18,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc3575bc060ce15bea3 HTTP/1.1" 502 150 2025-12-17 20:55:18,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cc3575bc060ce15bea3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:19,367 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407277575bc060ce15ac58 2025-12-17 20:55:19,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407277575bc060ce15ac58 HTTP/1.1" 502 150 2025-12-17 20:55:19,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407277575bc060ce15ac58'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:19,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407272575bc060ce15ac47 2025-12-17 20:55:19,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407272575bc060ce15ac47 HTTP/1.1" 502 150 2025-12-17 20:55:19,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407272575bc060ce15ac47'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:20,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b7575bc060ce15e4ae HTTP/1.1" 502 150 2025-12-17 20:55:20,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b7575bc060ce15e4ae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:22,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940780c575bc060ce15b3fd 2025-12-17 20:55:22,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b7575bc060ce15e4ae 2025-12-17 20:55:22,815 - urllib3.connectionpool - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Read timed out. (read timeout=60.0)")': /latest/node/69407198575bc060ce15ab38 2025-12-17 20:55:22,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (6): staging.kernelci.org:9000 2025-12-17 20:55:23,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc3575bc060ce15bea9 HTTP/1.1" 502 150 2025-12-17 20:55:23,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cc3575bc060ce15bea9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:23,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cc3575bc060ce15bea9 2025-12-17 20:55:24,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407196575bc060ce15ab32 HTTP/1.1" 502 150 2025-12-17 20:55:24,080 - lava_callback - ERROR - Error processing callback for node 69407196575bc060ce15ab32: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407196575bc060ce15ab32 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407196575bc060ce15ab32 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407196575bc060ce15ab32 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:55:24,082 - lava_callback - INFO - Starting async job submit for node 69407278575bc060ce15ac5b 2025-12-17 20:55:24,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:55:24,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407197575bc060ce15ab35 HTTP/1.1" 502 150 2025-12-17 20:55:24,592 - lava_callback - ERROR - Error processing callback for node 69407197575bc060ce15ab35: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407197575bc060ce15ab35 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407197575bc060ce15ab35 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407197575bc060ce15ab35 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:55:24,593 - lava_callback - INFO - Starting async job submit for node 69407280575bc060ce15ac6d 2025-12-17 20:55:24,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:55:25,168 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407113575bc060ce15a9cf 2025-12-17 20:55:26,192 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407274575bc060ce15ac4e 2025-12-17 20:55:26,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940719b575bc060ce15ab3e INFO: 172.21.0.1:42388 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:55:26,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694071cd575bc060ce15abd5 2025-12-17 20:55:26,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cc3575bc060ce15bea3 2025-12-17 20:55:27,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407277575bc060ce15ac58 2025-12-17 20:55:27,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407272575bc060ce15ac47 2025-12-17 20:55:31,201 - lava_callback - INFO - Received callback for node 6940741e575bc060ce15ae79 INFO: 172.21.0.1:55158 - "POST /node/6940741e575bc060ce15ae79 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:55170 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:55:37,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071a1575bc060ce15ab41 HTTP/1.1" 502 150 2025-12-17 20:55:37,140 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071a1575bc060ce15ab41'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:38,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940780c575bc060ce15b3fd HTTP/1.1" 502 150 2025-12-17 20:55:38,160 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940780c575bc060ce15b3fd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:38,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407198575bc060ce15ab38 HTTP/1.1" 502 150 2025-12-17 20:55:38,161 - lava_callback - ERROR - Error processing callback for node 69407198575bc060ce15ab38: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407198575bc060ce15ab38 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407198575bc060ce15ab38 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407198575bc060ce15ab38 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:55:38,162 - lava_callback - INFO - Starting async job submit for node 69407279575bc060ce15ac5e 2025-12-17 20:55:38,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:55:38,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b7575bc060ce15e4ae HTTP/1.1" 502 150 2025-12-17 20:55:38,164 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b7575bc060ce15e4ae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:38,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407279575bc060ce15ac5e HTTP/1.1" 502 150 2025-12-17 20:55:38,173 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407279575bc060ce15ac5e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:38,173 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407279575bc060ce15ac5e 2025-12-17 20:55:38,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407279575bc060ce15ac5e HTTP/1.1" 502 150 2025-12-17 20:55:38,174 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407279575bc060ce15ac5e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:38,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc3575bc060ce15bea9 HTTP/1.1" 502 150 2025-12-17 20:55:38,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cc3575bc060ce15bea9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:39,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407278575bc060ce15ac5b HTTP/1.1" 502 150 2025-12-17 20:55:39,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407278575bc060ce15ac5b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:39,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407278575bc060ce15ac5b 2025-12-17 20:55:39,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407278575bc060ce15ac5b HTTP/1.1" 502 150 2025-12-17 20:55:39,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407278575bc060ce15ac5b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:39,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407280575bc060ce15ac6d HTTP/1.1" 502 150 2025-12-17 20:55:39,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407280575bc060ce15ac6d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:39,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407280575bc060ce15ac6d 2025-12-17 20:55:39,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407280575bc060ce15ac6d HTTP/1.1" 502 150 2025-12-17 20:55:39,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407280575bc060ce15ac6d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:40,175 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407279575bc060ce15ac5e 2025-12-17 20:55:40,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407279575bc060ce15ac5e HTTP/1.1" 502 150 2025-12-17 20:55:40,176 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407279575bc060ce15ac5e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:40,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407275575bc060ce15ac55 HTTP/1.1" 502 150 2025-12-17 20:55:40,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407275575bc060ce15ac55'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:40,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407275575bc060ce15ac55 2025-12-17 20:55:40,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407113575bc060ce15a9cf HTTP/1.1" 502 150 2025-12-17 20:55:40,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407113575bc060ce15a9cf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:40,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407275575bc060ce15ac55 HTTP/1.1" 502 150 2025-12-17 20:55:40,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407275575bc060ce15ac55'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:40,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cc3575bc060ce15bea9 2025-12-17 20:55:40,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc3575bc060ce15bea9 HTTP/1.1" 502 150 2025-12-17 20:55:40,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cc3575bc060ce15bea9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:41,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407278575bc060ce15ac5b 2025-12-17 20:55:41,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407278575bc060ce15ac5b HTTP/1.1" 502 150 2025-12-17 20:55:41,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407278575bc060ce15ac5b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:41,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407274575bc060ce15ac4e HTTP/1.1" 502 150 2025-12-17 20:55:41,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407274575bc060ce15ac4e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:41,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940719b575bc060ce15ab3e HTTP/1.1" 502 150 2025-12-17 20:55:41,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940719b575bc060ce15ab3e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:41,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407280575bc060ce15ac6d 2025-12-17 20:55:41,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407280575bc060ce15ac6d HTTP/1.1" 502 150 2025-12-17 20:55:41,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407280575bc060ce15ac6d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:42,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc3575bc060ce15bea3 HTTP/1.1" 502 150 2025-12-17 20:55:42,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cc3575bc060ce15bea3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:42,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071cd575bc060ce15abd5 HTTP/1.1" 502 150 2025-12-17 20:55:42,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071cd575bc060ce15abd5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:42,164 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b7575bc060ce15e4ae 2025-12-17 20:55:42,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b7575bc060ce15e4ae HTTP/1.1" 502 150 2025-12-17 20:55:42,167 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b7575bc060ce15e4ae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:42,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940719a575bc060ce15ab3b HTTP/1.1" 502 150 2025-12-17 20:55:42,256 - lava_callback - ERROR - Error processing callback for node 6940719a575bc060ce15ab3b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940719a575bc060ce15ab3b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 1 more time] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940719a575bc060ce15ab3b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940719a575bc060ce15ab3b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:55:42,258 - lava_callback - INFO - Starting async job submit for node 6940727a575bc060ce15ac61 2025-12-17 20:55:42,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:55:42,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940727a575bc060ce15ac61 HTTP/1.1" 502 150 2025-12-17 20:55:42,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940727a575bc060ce15ac61'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:42,264 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940727a575bc060ce15ac61 2025-12-17 20:55:42,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940727a575bc060ce15ac61 HTTP/1.1" 502 150 2025-12-17 20:55:42,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940727a575bc060ce15ac61'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:42,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407275575bc060ce15ac55 2025-12-17 20:55:42,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407275575bc060ce15ac55 HTTP/1.1" 502 150 2025-12-17 20:55:42,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407275575bc060ce15ac55'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:42,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407277575bc060ce15ac58 HTTP/1.1" 502 150 2025-12-17 20:55:42,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407277575bc060ce15ac58'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:43,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407272575bc060ce15ac47 HTTP/1.1" 502 150 2025-12-17 20:55:43,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407272575bc060ce15ac47'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:44,176 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407279575bc060ce15ac5e 2025-12-17 20:55:44,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407279575bc060ce15ac5e HTTP/1.1" 502 150 2025-12-17 20:55:44,177 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407279575bc060ce15ac5e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:44,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940727a575bc060ce15ac61 2025-12-17 20:55:44,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940727a575bc060ce15ac61 HTTP/1.1" 502 150 2025-12-17 20:55:44,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940727a575bc060ce15ac61'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:44,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cc3575bc060ce15bea9 2025-12-17 20:55:44,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc3575bc060ce15bea9 HTTP/1.1" 502 150 2025-12-17 20:55:44,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cc3575bc060ce15bea9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:45,140 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694071a1575bc060ce15ab41 2025-12-17 20:55:45,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071a1575bc060ce15ab41 HTTP/1.1" 502 150 2025-12-17 20:55:45,141 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694071a1575bc060ce15ab41'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:45,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407278575bc060ce15ac5b 2025-12-17 20:55:45,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407278575bc060ce15ac5b HTTP/1.1" 502 150 2025-12-17 20:55:45,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407278575bc060ce15ac5b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:45,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407280575bc060ce15ac6d 2025-12-17 20:55:45,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407280575bc060ce15ac6d HTTP/1.1" 502 150 2025-12-17 20:55:45,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407280575bc060ce15ac6d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:46,161 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940780c575bc060ce15b3fd 2025-12-17 20:55:46,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940780c575bc060ce15b3fd HTTP/1.1" 502 150 2025-12-17 20:55:46,162 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940780c575bc060ce15b3fd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:46,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407275575bc060ce15ac55 2025-12-17 20:55:46,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407275575bc060ce15ac55 HTTP/1.1" 502 150 2025-12-17 20:55:46,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407275575bc060ce15ac55'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51896 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:55:48,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940727a575bc060ce15ac61 2025-12-17 20:55:50,167 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b7575bc060ce15e4ae 2025-12-17 20:55:50,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b7575bc060ce15e4ae HTTP/1.1" 502 150 2025-12-17 20:55:50,168 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b7575bc060ce15e4ae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:52,177 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407279575bc060ce15ac5e 2025-12-17 20:55:52,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407279575bc060ce15ac5e HTTP/1.1" 502 150 2025-12-17 20:55:52,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407279575bc060ce15ac5e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:52,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cc3575bc060ce15bea9 2025-12-17 20:55:52,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc3575bc060ce15bea9 HTTP/1.1" 502 150 2025-12-17 20:55:52,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cc3575bc060ce15bea9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:53,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407278575bc060ce15ac5b 2025-12-17 20:55:53,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407278575bc060ce15ac5b HTTP/1.1" 502 150 2025-12-17 20:55:53,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407278575bc060ce15ac5b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:53,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407280575bc060ce15ac6d 2025-12-17 20:55:53,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407280575bc060ce15ac6d HTTP/1.1" 502 150 2025-12-17 20:55:53,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407280575bc060ce15ac6d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:54,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407275575bc060ce15ac55 2025-12-17 20:55:54,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407275575bc060ce15ac55 HTTP/1.1" 502 150 2025-12-17 20:55:54,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407275575bc060ce15ac55'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:56,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407113575bc060ce15a9cf INFO: 172.21.0.1:40368 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:55:57,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407274575bc060ce15ac4e 2025-12-17 20:55:57,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407274575bc060ce15ac4e HTTP/1.1" 502 150 2025-12-17 20:55:57,490 - lava_callback - ERROR - Error processing callback for node 69407274575bc060ce15ac4e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407274575bc060ce15ac4e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407274575bc060ce15ac4e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407274575bc060ce15ac4e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:55:57,492 - lava_callback - INFO - Starting async job submit for node 6940727c575bc060ce15ac64 2025-12-17 20:55:57,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:55:57,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940727c575bc060ce15ac64 HTTP/1.1" 502 150 2025-12-17 20:55:57,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940727c575bc060ce15ac64'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:57,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940727c575bc060ce15ac64 2025-12-17 20:55:57,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940727c575bc060ce15ac64 HTTP/1.1" 502 150 2025-12-17 20:55:57,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940727c575bc060ce15ac64'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:57,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940719b575bc060ce15ab3e 2025-12-17 20:55:57,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940719b575bc060ce15ab3e HTTP/1.1" 502 150 2025-12-17 20:55:57,749 - lava_callback - ERROR - Error processing callback for node 6940719b575bc060ce15ab3e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940719b575bc060ce15ab3e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940719b575bc060ce15ab3e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940719b575bc060ce15ab3e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:55:57,750 - lava_callback - INFO - Starting async job submit for node 694070d3575bc060ce15a984 2025-12-17 20:55:58,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cc3575bc060ce15bea3 2025-12-17 20:55:58,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694071cd575bc060ce15abd5 2025-12-17 20:55:58,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc3575bc060ce15bea3 HTTP/1.1" 502 150 2025-12-17 20:55:58,027 - lava_callback - ERROR - Error processing callback for node 69407cc3575bc060ce15bea3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cc3575bc060ce15bea3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 1 more time] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cc3575bc060ce15bea3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cc3575bc060ce15bea3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:55:58,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071cd575bc060ce15abd5 HTTP/1.1" 502 150 2025-12-17 20:55:58,084 - lava_callback - ERROR - Error processing callback for node 694071cd575bc060ce15abd5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694071cd575bc060ce15abd5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 1 more time] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694071cd575bc060ce15abd5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694071cd575bc060ce15abd5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:55:58,086 - lava_callback - INFO - Starting async job submit for node 69407cc4575bc060ce15beac 2025-12-17 20:55:58,073 - lava_callback - INFO - Starting async job submit for node 69407d88575bc060ce15c12d 2025-12-17 20:55:58,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:55:58,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:55:58,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:55:58,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d3575bc060ce15a984 HTTP/1.1" 502 150 2025-12-17 20:55:58,227 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d3575bc060ce15a984'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:58,227 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070d3575bc060ce15a984 2025-12-17 20:55:58,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d3575bc060ce15a984 HTTP/1.1" 502 150 2025-12-17 20:55:58,228 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d3575bc060ce15a984'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:58,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d88575bc060ce15c12d HTTP/1.1" 502 150 2025-12-17 20:55:58,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d88575bc060ce15c12d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:58,243 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d88575bc060ce15c12d 2025-12-17 20:55:58,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d88575bc060ce15c12d HTTP/1.1" 502 150 2025-12-17 20:55:58,244 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d88575bc060ce15c12d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:58,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc4575bc060ce15beac HTTP/1.1" 502 150 2025-12-17 20:55:58,251 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cc4575bc060ce15beac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:58,251 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cc4575bc060ce15beac 2025-12-17 20:55:58,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc4575bc060ce15beac HTTP/1.1" 502 150 2025-12-17 20:55:58,252 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cc4575bc060ce15beac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:58,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407277575bc060ce15ac58 2025-12-17 20:55:58,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407277575bc060ce15ac58 HTTP/1.1" 502 150 2025-12-17 20:55:58,770 - lava_callback - ERROR - Error processing callback for node 69407277575bc060ce15ac58: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407277575bc060ce15ac58 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407277575bc060ce15ac58 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407277575bc060ce15ac58 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:55:58,772 - lava_callback - INFO - Starting async job submit for node 694072b2575bc060ce15acb0 2025-12-17 20:55:58,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:55:58,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b2575bc060ce15acb0 HTTP/1.1" 502 150 2025-12-17 20:55:58,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b2575bc060ce15acb0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:58,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b2575bc060ce15acb0 2025-12-17 20:55:58,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b2575bc060ce15acb0 HTTP/1.1" 502 150 2025-12-17 20:55:58,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b2575bc060ce15acb0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:59,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407272575bc060ce15ac47 2025-12-17 20:55:59,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407272575bc060ce15ac47 HTTP/1.1" 502 150 2025-12-17 20:55:59,026 - lava_callback - ERROR - Error processing callback for node 69407272575bc060ce15ac47: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407272575bc060ce15ac47 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 1 more time] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407272575bc060ce15ac47 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407272575bc060ce15ac47 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:55:59,027 - lava_callback - INFO - Starting async job submit for node 694072b5575bc060ce15acb3 2025-12-17 20:55:59,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:55:59,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b5575bc060ce15acb3 HTTP/1.1" 502 150 2025-12-17 20:55:59,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b5575bc060ce15acb3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:59,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b5575bc060ce15acb3 2025-12-17 20:55:59,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b5575bc060ce15acb3 HTTP/1.1" 502 150 2025-12-17 20:55:59,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b5575bc060ce15acb3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:55:59,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940727c575bc060ce15ac64 2025-12-17 20:55:59,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940727c575bc060ce15ac64 HTTP/1.1" 502 150 2025-12-17 20:55:59,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940727c575bc060ce15ac64'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:00,228 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070d3575bc060ce15a984 2025-12-17 20:56:00,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d3575bc060ce15a984 HTTP/1.1" 502 150 2025-12-17 20:56:00,229 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d3575bc060ce15a984'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:00,244 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d88575bc060ce15c12d 2025-12-17 20:56:00,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d88575bc060ce15c12d HTTP/1.1" 502 150 2025-12-17 20:56:00,245 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d88575bc060ce15c12d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:00,252 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cc4575bc060ce15beac 2025-12-17 20:56:00,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc4575bc060ce15beac HTTP/1.1" 502 150 2025-12-17 20:56:00,253 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cc4575bc060ce15beac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:00,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b2575bc060ce15acb0 2025-12-17 20:56:00,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b2575bc060ce15acb0 HTTP/1.1" 502 150 2025-12-17 20:56:00,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b2575bc060ce15acb0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41870 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:56:01,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b5575bc060ce15acb3 2025-12-17 20:56:01,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b5575bc060ce15acb3 HTTP/1.1" 502 150 2025-12-17 20:56:01,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b5575bc060ce15acb3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:01,142 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694071a1575bc060ce15ab41 2025-12-17 20:56:01,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694071a1575bc060ce15ab41 HTTP/1.1" 502 150 2025-12-17 20:56:01,143 - lava_callback - ERROR - Error processing callback for node 694071a1575bc060ce15ab41: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694071a1575bc060ce15ab41 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694071a1575bc060ce15ab41 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694071a1575bc060ce15ab41 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:01,144 - lava_callback - INFO - Starting async job submit for node 694072bf575bc060ce15accc 2025-12-17 20:56:01,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:01,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072bf575bc060ce15accc HTTP/1.1" 502 150 2025-12-17 20:56:01,151 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072bf575bc060ce15accc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:01,151 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072bf575bc060ce15accc 2025-12-17 20:56:01,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072bf575bc060ce15accc HTTP/1.1" 502 150 2025-12-17 20:56:01,152 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072bf575bc060ce15accc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:02,162 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940780c575bc060ce15b3fd 2025-12-17 20:56:02,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940780c575bc060ce15b3fd HTTP/1.1" 502 150 2025-12-17 20:56:02,164 - lava_callback - ERROR - Error processing callback for node 6940780c575bc060ce15b3fd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940780c575bc060ce15b3fd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940780c575bc060ce15b3fd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940780c575bc060ce15b3fd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:02,165 - lava_callback - INFO - Starting async job submit for node 694072c0575bc060ce15accf 2025-12-17 20:56:02,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:02,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c0575bc060ce15accf HTTP/1.1" 502 150 2025-12-17 20:56:02,172 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c0575bc060ce15accf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:02,172 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c0575bc060ce15accf 2025-12-17 20:56:02,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c0575bc060ce15accf HTTP/1.1" 502 150 2025-12-17 20:56:02,173 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c0575bc060ce15accf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:03,086 - lava_callback - INFO - Received callback for node 69407bf0575bc060ce15bc25 INFO: 172.21.0.1:41882 - "POST /node/69407bf0575bc060ce15bc25 HTTP/1.0" 202 Accepted 2025-12-17 20:56:03,152 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072bf575bc060ce15accc 2025-12-17 20:56:03,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940727c575bc060ce15ac64 2025-12-17 20:56:03,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940727c575bc060ce15ac64 HTTP/1.1" 502 150 2025-12-17 20:56:03,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940727c575bc060ce15ac64'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:03,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940727a575bc060ce15ac61 HTTP/1.1" 502 150 2025-12-17 20:56:03,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940727a575bc060ce15ac61'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:04,173 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c0575bc060ce15accf 2025-12-17 20:56:04,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c0575bc060ce15accf HTTP/1.1" 502 150 2025-12-17 20:56:04,174 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c0575bc060ce15accf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:04,229 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070d3575bc060ce15a984 2025-12-17 20:56:04,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d3575bc060ce15a984 HTTP/1.1" 502 150 2025-12-17 20:56:04,231 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d3575bc060ce15a984'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:04,245 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d88575bc060ce15c12d 2025-12-17 20:56:04,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d88575bc060ce15c12d HTTP/1.1" 502 150 2025-12-17 20:56:04,250 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d88575bc060ce15c12d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:04,253 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cc4575bc060ce15beac 2025-12-17 20:56:04,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc4575bc060ce15beac HTTP/1.1" 502 150 2025-12-17 20:56:04,254 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cc4575bc060ce15beac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:04,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b2575bc060ce15acb0 2025-12-17 20:56:04,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b2575bc060ce15acb0 HTTP/1.1" 502 150 2025-12-17 20:56:04,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b2575bc060ce15acb0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:05,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b5575bc060ce15acb3 2025-12-17 20:56:05,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b5575bc060ce15acb3 HTTP/1.1" 502 150 2025-12-17 20:56:05,037 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b5575bc060ce15acb3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:06,169 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b7575bc060ce15e4ae 2025-12-17 20:56:06,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b7575bc060ce15e4ae HTTP/1.1" 502 150 2025-12-17 20:56:06,170 - lava_callback - ERROR - Error processing callback for node 694087b7575bc060ce15e4ae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b7575bc060ce15e4ae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 1 more time] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b7575bc060ce15e4ae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b7575bc060ce15e4ae (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:06,172 - lava_callback - INFO - Starting async job submit for node 694072c2575bc060ce15acd6 2025-12-17 20:56:06,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:06,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c2575bc060ce15acd6 HTTP/1.1" 502 150 2025-12-17 20:56:06,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c2575bc060ce15acd6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:06,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c2575bc060ce15acd6 2025-12-17 20:56:06,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c2575bc060ce15acd6 HTTP/1.1" 502 150 2025-12-17 20:56:06,201 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c2575bc060ce15acd6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41884 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:56:08,174 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c0575bc060ce15accf 2025-12-17 20:56:08,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c0575bc060ce15accf HTTP/1.1" 502 150 2025-12-17 20:56:08,175 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c0575bc060ce15accf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:08,179 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407279575bc060ce15ac5e 2025-12-17 20:56:08,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407279575bc060ce15ac5e HTTP/1.1" 502 150 2025-12-17 20:56:08,180 - lava_callback - ERROR - Error processing callback for node 69407279575bc060ce15ac5e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407279575bc060ce15ac5e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407279575bc060ce15ac5e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407279575bc060ce15ac5e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:08,182 - lava_callback - INFO - Starting async job submit for node 694072c3575bc060ce15acd9 2025-12-17 20:56:08,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:08,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c3575bc060ce15acd9 HTTP/1.1" 502 150 2025-12-17 20:56:08,188 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c3575bc060ce15acd9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:08,188 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c3575bc060ce15acd9 2025-12-17 20:56:08,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c3575bc060ce15acd9 HTTP/1.1" 502 150 2025-12-17 20:56:08,188 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c3575bc060ce15acd9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:08,201 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c2575bc060ce15acd6 2025-12-17 20:56:08,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c2575bc060ce15acd6 HTTP/1.1" 502 150 2025-12-17 20:56:08,202 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c2575bc060ce15acd6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:08,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cc3575bc060ce15bea9 2025-12-17 20:56:08,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc3575bc060ce15bea9 HTTP/1.1" 502 150 2025-12-17 20:56:08,938 - lava_callback - ERROR - Error processing callback for node 69407cc3575bc060ce15bea9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cc3575bc060ce15bea9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cc3575bc060ce15bea9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cc3575bc060ce15bea9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:08,940 - lava_callback - INFO - Starting async job submit for node 694072ba575bc060ce15acc3 2025-12-17 20:56:08,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:08,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ba575bc060ce15acc3 HTTP/1.1" 502 150 2025-12-17 20:56:08,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072ba575bc060ce15acc3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:08,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072ba575bc060ce15acc3 2025-12-17 20:56:08,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ba575bc060ce15acc3 HTTP/1.1" 502 150 2025-12-17 20:56:08,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072ba575bc060ce15acc3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:09,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407278575bc060ce15ac5b 2025-12-17 20:56:09,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407278575bc060ce15ac5b HTTP/1.1" 502 150 2025-12-17 20:56:09,453 - lava_callback - ERROR - Error processing callback for node 69407278575bc060ce15ac5b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407278575bc060ce15ac5b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407278575bc060ce15ac5b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407278575bc060ce15ac5b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:09,455 - lava_callback - INFO - Starting async job submit for node 694072b6575bc060ce15acba 2025-12-17 20:56:09,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:09,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b6575bc060ce15acba HTTP/1.1" 502 150 2025-12-17 20:56:09,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b6575bc060ce15acba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:09,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b6575bc060ce15acba 2025-12-17 20:56:09,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b6575bc060ce15acba HTTP/1.1" 502 150 2025-12-17 20:56:09,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b6575bc060ce15acba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:09,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407280575bc060ce15ac6d 2025-12-17 20:56:09,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407280575bc060ce15ac6d HTTP/1.1" 502 150 2025-12-17 20:56:09,967 - lava_callback - ERROR - Error processing callback for node 69407280575bc060ce15ac6d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407280575bc060ce15ac6d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407280575bc060ce15ac6d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407280575bc060ce15ac6d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:09,970 - lava_callback - INFO - Starting async job submit for node 694072b7575bc060ce15acbd 2025-12-17 20:56:09,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:09,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b7575bc060ce15acbd HTTP/1.1" 502 150 2025-12-17 20:56:09,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b7575bc060ce15acbd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:09,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b7575bc060ce15acbd 2025-12-17 20:56:09,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b7575bc060ce15acbd HTTP/1.1" 502 150 2025-12-17 20:56:09,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b7575bc060ce15acbd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:10,189 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c3575bc060ce15acd9 2025-12-17 20:56:10,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c3575bc060ce15acd9 HTTP/1.1" 502 150 2025-12-17 20:56:10,190 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c3575bc060ce15acd9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:10,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407275575bc060ce15ac55 2025-12-17 20:56:10,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407275575bc060ce15ac55 HTTP/1.1" 502 150 2025-12-17 20:56:10,471 - lava_callback - ERROR - Error processing callback for node 69407275575bc060ce15ac55: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407275575bc060ce15ac55 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407275575bc060ce15ac55 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407275575bc060ce15ac55 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:10,474 - lava_callback - INFO - Starting async job submit for node 694072b9575bc060ce15acc0 2025-12-17 20:56:10,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:10,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b9575bc060ce15acc0 HTTP/1.1" 502 150 2025-12-17 20:56:10,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b9575bc060ce15acc0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:10,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b9575bc060ce15acc0 2025-12-17 20:56:10,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b9575bc060ce15acc0 HTTP/1.1" 502 150 2025-12-17 20:56:10,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b9575bc060ce15acc0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:10,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072ba575bc060ce15acc3 2025-12-17 20:56:10,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ba575bc060ce15acc3 HTTP/1.1" 502 150 2025-12-17 20:56:10,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072ba575bc060ce15acc3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:11,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b6575bc060ce15acba 2025-12-17 20:56:11,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b6575bc060ce15acba HTTP/1.1" 502 150 2025-12-17 20:56:11,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b6575bc060ce15acba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:11,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940727c575bc060ce15ac64 2025-12-17 20:56:11,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940727c575bc060ce15ac64 HTTP/1.1" 502 150 2025-12-17 20:56:11,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940727c575bc060ce15ac64'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:11,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940727a575bc060ce15ac61 2025-12-17 20:56:11,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940727a575bc060ce15ac61 HTTP/1.1" 502 150 2025-12-17 20:56:11,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940727a575bc060ce15ac61'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:11,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407113575bc060ce15a9cf HTTP/1.1" 502 150 2025-12-17 20:56:11,953 - lava_callback - ERROR - Error processing callback for node 69407113575bc060ce15a9cf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407113575bc060ce15a9cf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 1 more time] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407113575bc060ce15a9cf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407113575bc060ce15a9cf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:11,955 - lava_callback - INFO - Starting async job submit for node 69407e04575bc060ce15c205 2025-12-17 20:56:11,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b7575bc060ce15acbd 2025-12-17 20:56:11,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:11,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b7575bc060ce15acbd HTTP/1.1" 502 150 2025-12-17 20:56:11,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b7575bc060ce15acbd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:11,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e04575bc060ce15c205 HTTP/1.1" 502 150 2025-12-17 20:56:11,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e04575bc060ce15c205'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:11,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e04575bc060ce15c205 2025-12-17 20:56:11,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e04575bc060ce15c205 HTTP/1.1" 502 150 2025-12-17 20:56:11,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e04575bc060ce15c205'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:12,202 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c2575bc060ce15acd6 2025-12-17 20:56:12,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c2575bc060ce15acd6 HTTP/1.1" 502 150 2025-12-17 20:56:12,203 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c2575bc060ce15acd6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:12,231 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070d3575bc060ce15a984 2025-12-17 20:56:12,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d3575bc060ce15a984 HTTP/1.1" 502 150 2025-12-17 20:56:12,232 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d3575bc060ce15a984'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:12,250 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d88575bc060ce15c12d 2025-12-17 20:56:12,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d88575bc060ce15c12d HTTP/1.1" 502 150 2025-12-17 20:56:12,251 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d88575bc060ce15c12d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:12,254 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cc4575bc060ce15beac 2025-12-17 20:56:12,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc4575bc060ce15beac HTTP/1.1" 502 150 2025-12-17 20:56:12,255 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cc4575bc060ce15beac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:12,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b9575bc060ce15acc0 2025-12-17 20:56:12,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b9575bc060ce15acc0 HTTP/1.1" 502 150 2025-12-17 20:56:12,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b9575bc060ce15acc0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:12,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b2575bc060ce15acb0 2025-12-17 20:56:12,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b2575bc060ce15acb0 HTTP/1.1" 502 150 2025-12-17 20:56:12,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b2575bc060ce15acb0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:13,037 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b5575bc060ce15acb3 2025-12-17 20:56:13,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b5575bc060ce15acb3 HTTP/1.1" 502 150 2025-12-17 20:56:13,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b5575bc060ce15acb3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:13,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e04575bc060ce15c205 2025-12-17 20:56:14,190 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c3575bc060ce15acd9 2025-12-17 20:56:14,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c3575bc060ce15acd9 HTTP/1.1" 502 150 2025-12-17 20:56:14,191 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c3575bc060ce15acd9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:14,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072ba575bc060ce15acc3 2025-12-17 20:56:14,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ba575bc060ce15acc3 HTTP/1.1" 502 150 2025-12-17 20:56:14,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072ba575bc060ce15acc3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48880 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:56:15,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b6575bc060ce15acba 2025-12-17 20:56:15,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b6575bc060ce15acba HTTP/1.1" 502 150 2025-12-17 20:56:15,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b6575bc060ce15acba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:15,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b7575bc060ce15acbd 2025-12-17 20:56:15,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b7575bc060ce15acbd HTTP/1.1" 502 150 2025-12-17 20:56:15,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b7575bc060ce15acbd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:16,175 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c0575bc060ce15accf 2025-12-17 20:56:16,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c0575bc060ce15accf HTTP/1.1" 502 150 2025-12-17 20:56:16,177 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c0575bc060ce15accf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:16,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b9575bc060ce15acc0 2025-12-17 20:56:16,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b9575bc060ce15acc0 HTTP/1.1" 502 150 2025-12-17 20:56:16,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b9575bc060ce15acc0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48884 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:56:18,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072bf575bc060ce15accc HTTP/1.1" 502 150 2025-12-17 20:56:18,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072bf575bc060ce15accc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:20,204 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c2575bc060ce15acd6 2025-12-17 20:56:20,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c2575bc060ce15acd6 HTTP/1.1" 502 150 2025-12-17 20:56:20,205 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c2575bc060ce15acd6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:22,192 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c3575bc060ce15acd9 2025-12-17 20:56:22,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072bf575bc060ce15accc 2025-12-17 20:56:22,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072bf575bc060ce15accc HTTP/1.1" 502 150 2025-12-17 20:56:22,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072bf575bc060ce15accc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:22,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072ba575bc060ce15acc3 2025-12-17 20:56:22,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ba575bc060ce15acc3 HTTP/1.1" 502 150 2025-12-17 20:56:22,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072ba575bc060ce15acc3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:23,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b6575bc060ce15acba 2025-12-17 20:56:23,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b6575bc060ce15acba HTTP/1.1" 502 150 2025-12-17 20:56:23,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b6575bc060ce15acba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:23,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b7575bc060ce15acbd 2025-12-17 20:56:23,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b7575bc060ce15acbd HTTP/1.1" 502 150 2025-12-17 20:56:23,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b7575bc060ce15acbd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:24,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b9575bc060ce15acc0 2025-12-17 20:56:24,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b9575bc060ce15acc0 HTTP/1.1" 502 150 2025-12-17 20:56:24,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072b9575bc060ce15acc0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37536 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:56:27,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940727c575bc060ce15ac64 2025-12-17 20:56:27,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940727c575bc060ce15ac64 HTTP/1.1" 502 150 2025-12-17 20:56:27,542 - lava_callback - ERROR - Error processing callback for node 6940727c575bc060ce15ac64: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940727c575bc060ce15ac64 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940727c575bc060ce15ac64 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940727c575bc060ce15ac64 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:27,544 - lava_callback - INFO - Starting async job submit for node 694072c4575bc060ce15acdc 2025-12-17 20:56:27,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:27,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c4575bc060ce15acdc HTTP/1.1" 502 150 2025-12-17 20:56:27,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c4575bc060ce15acdc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:27,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c4575bc060ce15acdc 2025-12-17 20:56:27,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c4575bc060ce15acdc HTTP/1.1" 502 150 2025-12-17 20:56:27,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c4575bc060ce15acdc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:27,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940727a575bc060ce15ac61 2025-12-17 20:56:27,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940727a575bc060ce15ac61 HTTP/1.1" 502 150 2025-12-17 20:56:27,764 - lava_callback - ERROR - Error processing callback for node 6940727a575bc060ce15ac61: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940727a575bc060ce15ac61 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940727a575bc060ce15ac61 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940727a575bc060ce15ac61 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:27,766 - lava_callback - INFO - Starting async job submit for node 694072c5575bc060ce15acdf 2025-12-17 20:56:27,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:27,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c5575bc060ce15acdf HTTP/1.1" 502 150 2025-12-17 20:56:27,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c5575bc060ce15acdf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:27,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c5575bc060ce15acdf 2025-12-17 20:56:27,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c5575bc060ce15acdf HTTP/1.1" 502 150 2025-12-17 20:56:27,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c5575bc060ce15acdf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:28,232 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070d3575bc060ce15a984 2025-12-17 20:56:28,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d3575bc060ce15a984 HTTP/1.1" 502 150 2025-12-17 20:56:28,234 - lava_callback - ERROR - Error processing callback for node 694070d3575bc060ce15a984: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d3575bc060ce15a984 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d3575bc060ce15a984 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d3575bc060ce15a984 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:28,236 - lava_callback - INFO - Starting async job submit for node 694072c7575bc060ce15ace2 2025-12-17 20:56:28,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:28,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c7575bc060ce15ace2 HTTP/1.1" 502 150 2025-12-17 20:56:28,242 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c7575bc060ce15ace2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:28,242 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c7575bc060ce15ace2 2025-12-17 20:56:28,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c7575bc060ce15ace2 HTTP/1.1" 502 150 2025-12-17 20:56:28,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c7575bc060ce15ace2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:28,251 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d88575bc060ce15c12d 2025-12-17 20:56:28,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d88575bc060ce15c12d HTTP/1.1" 502 150 2025-12-17 20:56:28,252 - lava_callback - ERROR - Error processing callback for node 69407d88575bc060ce15c12d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d88575bc060ce15c12d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d88575bc060ce15c12d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d88575bc060ce15c12d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:28,254 - lava_callback - INFO - Starting async job submit for node 694072c8575bc060ce15ace9 2025-12-17 20:56:28,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:28,255 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cc4575bc060ce15beac 2025-12-17 20:56:28,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cc4575bc060ce15beac HTTP/1.1" 502 150 2025-12-17 20:56:28,256 - lava_callback - ERROR - Error processing callback for node 69407cc4575bc060ce15beac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cc4575bc060ce15beac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cc4575bc060ce15beac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cc4575bc060ce15beac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:28,258 - lava_callback - INFO - Starting async job submit for node 694072c9575bc060ce15acec 2025-12-17 20:56:28,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:28,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c8575bc060ce15ace9 HTTP/1.1" 502 150 2025-12-17 20:56:28,261 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c8575bc060ce15ace9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:28,261 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c8575bc060ce15ace9 2025-12-17 20:56:28,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c8575bc060ce15ace9 HTTP/1.1" 502 150 2025-12-17 20:56:28,262 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c8575bc060ce15ace9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:28,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c9575bc060ce15acec HTTP/1.1" 502 150 2025-12-17 20:56:28,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c9575bc060ce15acec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:28,264 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c9575bc060ce15acec 2025-12-17 20:56:28,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c9575bc060ce15acec HTTP/1.1" 502 150 2025-12-17 20:56:28,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c9575bc060ce15acec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:28,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b2575bc060ce15acb0 2025-12-17 20:56:28,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b2575bc060ce15acb0 HTTP/1.1" 502 150 2025-12-17 20:56:28,785 - lava_callback - ERROR - Error processing callback for node 694072b2575bc060ce15acb0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072b2575bc060ce15acb0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072b2575bc060ce15acb0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072b2575bc060ce15acb0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:28,787 - lava_callback - INFO - Starting async job submit for node 694072ca575bc060ce15acfa 2025-12-17 20:56:28,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:28,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ca575bc060ce15acfa HTTP/1.1" 502 150 2025-12-17 20:56:28,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072ca575bc060ce15acfa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:28,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072ca575bc060ce15acfa 2025-12-17 20:56:28,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ca575bc060ce15acfa HTTP/1.1" 502 150 2025-12-17 20:56:28,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072ca575bc060ce15acfa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:29,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b5575bc060ce15acb3 2025-12-17 20:56:29,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e04575bc060ce15c205 HTTP/1.1" 502 150 2025-12-17 20:56:29,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e04575bc060ce15c205'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:29,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c4575bc060ce15acdc 2025-12-17 20:56:29,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c4575bc060ce15acdc HTTP/1.1" 502 150 2025-12-17 20:56:29,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c4575bc060ce15acdc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:29,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c5575bc060ce15acdf 2025-12-17 20:56:29,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c5575bc060ce15acdf HTTP/1.1" 502 150 2025-12-17 20:56:29,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c5575bc060ce15acdf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:30,243 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c7575bc060ce15ace2 2025-12-17 20:56:30,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c7575bc060ce15ace2 HTTP/1.1" 502 150 2025-12-17 20:56:30,244 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c7575bc060ce15ace2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:30,262 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c8575bc060ce15ace9 2025-12-17 20:56:30,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c8575bc060ce15ace9 HTTP/1.1" 502 150 2025-12-17 20:56:30,263 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c8575bc060ce15ace9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:30,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c9575bc060ce15acec 2025-12-17 20:56:30,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c9575bc060ce15acec HTTP/1.1" 502 150 2025-12-17 20:56:30,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c9575bc060ce15acec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:30,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072bf575bc060ce15accc 2025-12-17 20:56:30,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072bf575bc060ce15accc HTTP/1.1" 502 150 2025-12-17 20:56:30,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072bf575bc060ce15accc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:30,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072ca575bc060ce15acfa 2025-12-17 20:56:30,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ca575bc060ce15acfa HTTP/1.1" 502 150 2025-12-17 20:56:30,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072ca575bc060ce15acfa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:32,177 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c0575bc060ce15accf 2025-12-17 20:56:32,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c0575bc060ce15accf HTTP/1.1" 502 150 2025-12-17 20:56:32,178 - lava_callback - ERROR - Error processing callback for node 694072c0575bc060ce15accf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c0575bc060ce15accf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c0575bc060ce15accf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c0575bc060ce15accf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:32,180 - lava_callback - INFO - Starting async job submit for node 694072cc575bc060ce15acfd 2025-12-17 20:56:32,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:32,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072cc575bc060ce15acfd HTTP/1.1" 502 150 2025-12-17 20:56:32,211 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072cc575bc060ce15acfd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:32,211 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072cc575bc060ce15acfd 2025-12-17 20:56:32,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072cc575bc060ce15acfd HTTP/1.1" 502 150 2025-12-17 20:56:32,212 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072cc575bc060ce15acfd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:33,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e04575bc060ce15c205 2025-12-17 20:56:33,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c4575bc060ce15acdc 2025-12-17 20:56:33,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c4575bc060ce15acdc HTTP/1.1" 502 150 2025-12-17 20:56:33,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c4575bc060ce15acdc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:33,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c5575bc060ce15acdf 2025-12-17 20:56:33,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c5575bc060ce15acdf HTTP/1.1" 502 150 2025-12-17 20:56:33,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c5575bc060ce15acdf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:34,212 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072cc575bc060ce15acfd 2025-12-17 20:56:34,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072cc575bc060ce15acfd HTTP/1.1" 502 150 2025-12-17 20:56:34,213 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072cc575bc060ce15acfd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:34,244 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c7575bc060ce15ace2 2025-12-17 20:56:34,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c7575bc060ce15ace2 HTTP/1.1" 502 150 2025-12-17 20:56:34,245 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c7575bc060ce15ace2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:34,263 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c8575bc060ce15ace9 2025-12-17 20:56:34,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c8575bc060ce15ace9 HTTP/1.1" 502 150 2025-12-17 20:56:34,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c8575bc060ce15ace9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:34,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c9575bc060ce15acec 2025-12-17 20:56:34,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c9575bc060ce15acec HTTP/1.1" 502 150 2025-12-17 20:56:34,267 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c9575bc060ce15acec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:34,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072ca575bc060ce15acfa 2025-12-17 20:56:34,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ca575bc060ce15acfa HTTP/1.1" 502 150 2025-12-17 20:56:34,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072ca575bc060ce15acfa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:36,205 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c2575bc060ce15acd6 2025-12-17 20:56:36,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c2575bc060ce15acd6 HTTP/1.1" 502 150 2025-12-17 20:56:36,206 - lava_callback - ERROR - Error processing callback for node 694072c2575bc060ce15acd6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c2575bc060ce15acd6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c2575bc060ce15acd6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c2575bc060ce15acd6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:36,208 - lava_callback - INFO - Starting async job submit for node 694072cd575bc060ce15ad00 2025-12-17 20:56:36,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:36,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072cd575bc060ce15ad00 HTTP/1.1" 502 150 2025-12-17 20:56:36,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072cd575bc060ce15ad00'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:36,215 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072cd575bc060ce15ad00 2025-12-17 20:56:36,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072cd575bc060ce15ad00 HTTP/1.1" 502 150 2025-12-17 20:56:36,216 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072cd575bc060ce15ad00'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59612 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:56:37,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c3575bc060ce15acd9 HTTP/1.1" 502 150 2025-12-17 20:56:37,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c3575bc060ce15acd9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:38,214 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072cc575bc060ce15acfd 2025-12-17 20:56:38,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072cc575bc060ce15acfd HTTP/1.1" 502 150 2025-12-17 20:56:38,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072cc575bc060ce15acfd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:38,216 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072cd575bc060ce15ad00 2025-12-17 20:56:38,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072cd575bc060ce15ad00 HTTP/1.1" 502 150 2025-12-17 20:56:38,217 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072cd575bc060ce15ad00'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:38,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072ba575bc060ce15acc3 2025-12-17 20:56:38,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ba575bc060ce15acc3 HTTP/1.1" 502 150 2025-12-17 20:56:38,962 - lava_callback - ERROR - Error processing callback for node 694072ba575bc060ce15acc3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072ba575bc060ce15acc3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072ba575bc060ce15acc3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072ba575bc060ce15acc3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:38,964 - lava_callback - INFO - Starting async job submit for node 694072ec575bc060ce15ad72 2025-12-17 20:56:38,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:38,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ec575bc060ce15ad72 HTTP/1.1" 502 150 2025-12-17 20:56:38,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072ec575bc060ce15ad72'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:38,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072ec575bc060ce15ad72 2025-12-17 20:56:38,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ec575bc060ce15ad72 HTTP/1.1" 502 150 2025-12-17 20:56:38,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072ec575bc060ce15ad72'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:39,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b6575bc060ce15acba 2025-12-17 20:56:39,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b6575bc060ce15acba HTTP/1.1" 502 150 2025-12-17 20:56:39,470 - lava_callback - ERROR - Error processing callback for node 694072b6575bc060ce15acba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072b6575bc060ce15acba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072b6575bc060ce15acba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072b6575bc060ce15acba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:39,471 - lava_callback - INFO - Starting async job submit for node 6940788b575bc060ce15b4e8 2025-12-17 20:56:39,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:39,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940788b575bc060ce15b4e8 HTTP/1.1" 502 150 2025-12-17 20:56:39,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940788b575bc060ce15b4e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:39,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940788b575bc060ce15b4e8 2025-12-17 20:56:39,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940788b575bc060ce15b4e8 HTTP/1.1" 502 150 2025-12-17 20:56:39,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940788b575bc060ce15b4e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:39,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b7575bc060ce15acbd 2025-12-17 20:56:39,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b7575bc060ce15acbd HTTP/1.1" 502 150 2025-12-17 20:56:39,991 - lava_callback - ERROR - Error processing callback for node 694072b7575bc060ce15acbd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072b7575bc060ce15acbd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072b7575bc060ce15acbd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072b7575bc060ce15acbd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:39,993 - lava_callback - INFO - Starting async job submit for node 69407e07575bc060ce15c210 2025-12-17 20:56:40,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:40,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072b9575bc060ce15acc0 2025-12-17 20:56:40,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b9575bc060ce15acc0 HTTP/1.1" 502 150 2025-12-17 20:56:40,495 - lava_callback - ERROR - Error processing callback for node 694072b9575bc060ce15acc0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072b9575bc060ce15acc0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072b9575bc060ce15acc0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072b9575bc060ce15acc0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:40,497 - lava_callback - INFO - Starting async job submit for node 694070d4575bc060ce15a987 2025-12-17 20:56:40,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:40,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d4575bc060ce15a987 HTTP/1.1" 502 150 2025-12-17 20:56:40,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d4575bc060ce15a987'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:40,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070d4575bc060ce15a987 2025-12-17 20:56:40,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d4575bc060ce15a987 HTTP/1.1" 502 150 2025-12-17 20:56:40,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d4575bc060ce15a987'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:40,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072ec575bc060ce15ad72 2025-12-17 20:56:40,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ec575bc060ce15ad72 HTTP/1.1" 502 150 2025-12-17 20:56:40,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072ec575bc060ce15ad72'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:41,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940788b575bc060ce15b4e8 2025-12-17 20:56:41,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940788b575bc060ce15b4e8 HTTP/1.1" 502 150 2025-12-17 20:56:41,501 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940788b575bc060ce15b4e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:41,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c4575bc060ce15acdc 2025-12-17 20:56:41,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c4575bc060ce15acdc HTTP/1.1" 502 150 2025-12-17 20:56:41,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c4575bc060ce15acdc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:41,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c5575bc060ce15acdf 2025-12-17 20:56:41,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c5575bc060ce15acdf HTTP/1.1" 502 150 2025-12-17 20:56:41,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c5575bc060ce15acdf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:42,217 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072cd575bc060ce15ad00 2025-12-17 20:56:42,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072cd575bc060ce15ad00 HTTP/1.1" 502 150 2025-12-17 20:56:42,219 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072cd575bc060ce15ad00'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:42,245 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c7575bc060ce15ace2 2025-12-17 20:56:42,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c7575bc060ce15ace2 HTTP/1.1" 502 150 2025-12-17 20:56:42,246 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c7575bc060ce15ace2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:42,264 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c8575bc060ce15ace9 2025-12-17 20:56:42,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c8575bc060ce15ace9 HTTP/1.1" 502 150 2025-12-17 20:56:42,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c8575bc060ce15ace9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:42,268 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c9575bc060ce15acec 2025-12-17 20:56:42,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c9575bc060ce15acec HTTP/1.1" 502 150 2025-12-17 20:56:42,269 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072c9575bc060ce15acec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:42,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072ca575bc060ce15acfa 2025-12-17 20:56:42,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ca575bc060ce15acfa HTTP/1.1" 502 150 2025-12-17 20:56:42,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072ca575bc060ce15acfa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:42,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070d4575bc060ce15a987 2025-12-17 20:56:42,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d4575bc060ce15a987 HTTP/1.1" 502 150 2025-12-17 20:56:42,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d4575bc060ce15a987'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:44,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072b5575bc060ce15acb3 HTTP/1.1" 502 150 2025-12-17 20:56:44,465 - lava_callback - ERROR - Error processing callback for node 694072b5575bc060ce15acb3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072b5575bc060ce15acb3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072b5575bc060ce15acb3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072b5575bc060ce15acb3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:44,467 - lava_callback - INFO - Starting async job submit for node 69406efe575bc060ce15a521 2025-12-17 20:56:45,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072ec575bc060ce15ad72 2025-12-17 20:56:45,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ec575bc060ce15ad72 HTTP/1.1" 502 150 2025-12-17 20:56:45,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072ec575bc060ce15ad72'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:45,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940788b575bc060ce15b4e8 2025-12-17 20:56:45,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940788b575bc060ce15b4e8 HTTP/1.1" 502 150 2025-12-17 20:56:45,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940788b575bc060ce15b4e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:46,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:46,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69406efe575bc060ce15a521 HTTP/1.1" 502 150 2025-12-17 20:56:46,053 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efe575bc060ce15a521'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:46,054 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69406efe575bc060ce15a521 2025-12-17 20:56:46,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69406efe575bc060ce15a521 HTTP/1.1" 502 150 2025-12-17 20:56:46,054 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efe575bc060ce15a521'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:46,215 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072cc575bc060ce15acfd 2025-12-17 20:56:46,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072cc575bc060ce15acfd HTTP/1.1" 502 150 2025-12-17 20:56:46,216 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072cc575bc060ce15acfd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50244 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:56:46,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072bf575bc060ce15accc 2025-12-17 20:56:46,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072bf575bc060ce15accc HTTP/1.1" 502 150 2025-12-17 20:56:46,614 - lava_callback - ERROR - Error processing callback for node 694072bf575bc060ce15accc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072bf575bc060ce15accc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072bf575bc060ce15accc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072bf575bc060ce15accc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:46,616 - lava_callback - INFO - Starting async job submit for node 6940c5261e3c24e95681d67d 2025-12-17 20:56:46,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:46,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5261e3c24e95681d67d HTTP/1.1" 502 150 2025-12-17 20:56:46,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5261e3c24e95681d67d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:46,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5261e3c24e95681d67d 2025-12-17 20:56:46,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5261e3c24e95681d67d HTTP/1.1" 502 150 2025-12-17 20:56:46,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5261e3c24e95681d67d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:46,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070d4575bc060ce15a987 2025-12-17 20:56:46,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d4575bc060ce15a987 HTTP/1.1" 502 150 2025-12-17 20:56:46,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d4575bc060ce15a987'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:48,054 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69406efe575bc060ce15a521 2025-12-17 20:56:48,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5261e3c24e95681d67d 2025-12-17 20:56:48,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5261e3c24e95681d67d HTTP/1.1" 502 150 2025-12-17 20:56:48,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5261e3c24e95681d67d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:48,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e04575bc060ce15c205 HTTP/1.1" 502 150 2025-12-17 20:56:48,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e04575bc060ce15c205'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:50,219 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072cd575bc060ce15ad00 2025-12-17 20:56:50,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072cd575bc060ce15ad00 HTTP/1.1" 502 150 2025-12-17 20:56:50,220 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072cd575bc060ce15ad00'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:52,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5261e3c24e95681d67d 2025-12-17 20:56:52,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5261e3c24e95681d67d HTTP/1.1" 502 150 2025-12-17 20:56:52,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5261e3c24e95681d67d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:53,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072ec575bc060ce15ad72 2025-12-17 20:56:53,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ec575bc060ce15ad72 HTTP/1.1" 502 150 2025-12-17 20:56:53,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072ec575bc060ce15ad72'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:53,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940788b575bc060ce15b4e8 2025-12-17 20:56:53,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940788b575bc060ce15b4e8 HTTP/1.1" 502 150 2025-12-17 20:56:53,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940788b575bc060ce15b4e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:53,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c3575bc060ce15acd9 2025-12-17 20:56:53,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c3575bc060ce15acd9 HTTP/1.1" 502 150 2025-12-17 20:56:53,554 - lava_callback - ERROR - Error processing callback for node 694072c3575bc060ce15acd9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c3575bc060ce15acd9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c3575bc060ce15acd9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c3575bc060ce15acd9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:53,555 - lava_callback - INFO - Starting async job submit for node 69407686575bc060ce15b0e8 2025-12-17 20:56:53,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:53,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407686575bc060ce15b0e8 HTTP/1.1" 502 150 2025-12-17 20:56:53,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407686575bc060ce15b0e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:53,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407686575bc060ce15b0e8 2025-12-17 20:56:53,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407686575bc060ce15b0e8 HTTP/1.1" 502 150 2025-12-17 20:56:53,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407686575bc060ce15b0e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:54,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070d4575bc060ce15a987 2025-12-17 20:56:54,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d4575bc060ce15a987 HTTP/1.1" 502 150 2025-12-17 20:56:54,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d4575bc060ce15a987'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:55,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e07575bc060ce15c210 HTTP/1.1" 502 150 2025-12-17 20:56:55,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e07575bc060ce15c210'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:55,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e07575bc060ce15c210 2025-12-17 20:56:55,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e07575bc060ce15c210 HTTP/1.1" 502 150 2025-12-17 20:56:55,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e07575bc060ce15c210'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:55,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407686575bc060ce15b0e8 2025-12-17 20:56:55,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407686575bc060ce15b0e8 HTTP/1.1" 502 150 2025-12-17 20:56:55,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407686575bc060ce15b0e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53668 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:56:56,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e04575bc060ce15c205 2025-12-17 20:56:56,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e04575bc060ce15c205 HTTP/1.1" 502 150 2025-12-17 20:56:56,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e04575bc060ce15c205'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:57,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e07575bc060ce15c210 2025-12-17 20:56:57,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e07575bc060ce15c210 HTTP/1.1" 502 150 2025-12-17 20:56:57,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e07575bc060ce15c210'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:57,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c4575bc060ce15acdc 2025-12-17 20:56:57,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c4575bc060ce15acdc HTTP/1.1" 502 150 2025-12-17 20:56:57,567 - lava_callback - ERROR - Error processing callback for node 694072c4575bc060ce15acdc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c4575bc060ce15acdc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c4575bc060ce15acdc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c4575bc060ce15acdc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:57,569 - lava_callback - INFO - Starting async job submit for node 69407165575bc060ce15aab0 2025-12-17 20:56:57,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:57,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407165575bc060ce15aab0 HTTP/1.1" 502 150 2025-12-17 20:56:57,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407165575bc060ce15aab0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:57,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407165575bc060ce15aab0 2025-12-17 20:56:57,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407165575bc060ce15aab0 HTTP/1.1" 502 150 2025-12-17 20:56:57,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407165575bc060ce15aab0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:57,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c5575bc060ce15acdf 2025-12-17 20:56:57,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c5575bc060ce15acdf HTTP/1.1" 502 150 2025-12-17 20:56:57,794 - lava_callback - ERROR - Error processing callback for node 694072c5575bc060ce15acdf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c5575bc060ce15acdf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c5575bc060ce15acdf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c5575bc060ce15acdf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:57,796 - lava_callback - INFO - Starting async job submit for node 694072bd575bc060ce15acc6 2025-12-17 20:56:57,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:57,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072bd575bc060ce15acc6 HTTP/1.1" 502 150 2025-12-17 20:56:57,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072bd575bc060ce15acc6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:57,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072bd575bc060ce15acc6 2025-12-17 20:56:57,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072bd575bc060ce15acc6 HTTP/1.1" 502 150 2025-12-17 20:56:57,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072bd575bc060ce15acc6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:58,247 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c7575bc060ce15ace2 2025-12-17 20:56:58,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c7575bc060ce15ace2 HTTP/1.1" 502 150 2025-12-17 20:56:58,248 - lava_callback - ERROR - Error processing callback for node 694072c7575bc060ce15ace2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c7575bc060ce15ace2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c7575bc060ce15ace2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c7575bc060ce15ace2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:58,250 - lava_callback - INFO - Starting async job submit for node 694072be575bc060ce15acc9 2025-12-17 20:56:58,271 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c8575bc060ce15ace9 2025-12-17 20:56:58,276 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072c9575bc060ce15acec 2025-12-17 20:56:58,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c8575bc060ce15ace9 HTTP/1.1" 502 150 2025-12-17 20:56:58,287 - lava_callback - ERROR - Error processing callback for node 694072c8575bc060ce15ace9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c8575bc060ce15ace9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c8575bc060ce15ace9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c8575bc060ce15ace9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:58,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072c9575bc060ce15acec HTTP/1.1" 502 150 2025-12-17 20:56:58,335 - lava_callback - INFO - Starting async job submit for node 69407687575bc060ce15b0eb 2025-12-17 20:56:58,340 - lava_callback - ERROR - Error processing callback for node 694072c9575bc060ce15acec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c9575bc060ce15acec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c9575bc060ce15acec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072c9575bc060ce15acec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:58,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:58,382 - lava_callback - INFO - Starting async job submit for node 69407681575bc060ce15b0dd 2025-12-17 20:56:58,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:58,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:58,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072be575bc060ce15acc9 HTTP/1.1" 502 150 2025-12-17 20:56:58,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072be575bc060ce15acc9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:58,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072be575bc060ce15acc9 2025-12-17 20:56:58,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407687575bc060ce15b0eb HTTP/1.1" 502 150 2025-12-17 20:56:58,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407687575bc060ce15b0eb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:58,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407687575bc060ce15b0eb 2025-12-17 20:56:58,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072be575bc060ce15acc9 HTTP/1.1" 502 150 2025-12-17 20:56:58,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072be575bc060ce15acc9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:58,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407687575bc060ce15b0eb HTTP/1.1" 502 150 2025-12-17 20:56:58,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407687575bc060ce15b0eb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:58,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407681575bc060ce15b0dd HTTP/1.1" 502 150 2025-12-17 20:56:58,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407681575bc060ce15b0dd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:58,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407681575bc060ce15b0dd 2025-12-17 20:56:58,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407681575bc060ce15b0dd HTTP/1.1" 502 150 2025-12-17 20:56:58,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407681575bc060ce15b0dd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:58,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072ca575bc060ce15acfa 2025-12-17 20:56:58,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ca575bc060ce15acfa HTTP/1.1" 502 150 2025-12-17 20:56:58,801 - lava_callback - ERROR - Error processing callback for node 694072ca575bc060ce15acfa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072ca575bc060ce15acfa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072ca575bc060ce15acfa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072ca575bc060ce15acfa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:56:58,803 - lava_callback - INFO - Starting async job submit for node 69407683575bc060ce15b0e0 2025-12-17 20:56:58,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:56:58,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407683575bc060ce15b0e0 HTTP/1.1" 502 150 2025-12-17 20:56:58,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407683575bc060ce15b0e0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:58,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407683575bc060ce15b0e0 2025-12-17 20:56:58,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407683575bc060ce15b0e0 HTTP/1.1" 502 150 2025-12-17 20:56:58,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407683575bc060ce15b0e0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:59,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407686575bc060ce15b0e8 2025-12-17 20:56:59,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407686575bc060ce15b0e8 HTTP/1.1" 502 150 2025-12-17 20:56:59,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407686575bc060ce15b0e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:59,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407165575bc060ce15aab0 2025-12-17 20:56:59,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407165575bc060ce15aab0 HTTP/1.1" 502 150 2025-12-17 20:56:59,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407165575bc060ce15aab0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:56:59,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072bd575bc060ce15acc6 2025-12-17 20:56:59,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072bd575bc060ce15acc6 HTTP/1.1" 502 150 2025-12-17 20:56:59,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072bd575bc060ce15acc6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:00,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072be575bc060ce15acc9 2025-12-17 20:57:00,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407687575bc060ce15b0eb 2025-12-17 20:57:00,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072be575bc060ce15acc9 HTTP/1.1" 502 150 2025-12-17 20:57:00,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072be575bc060ce15acc9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:00,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407687575bc060ce15b0eb HTTP/1.1" 502 150 2025-12-17 20:57:00,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407687575bc060ce15b0eb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:00,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407681575bc060ce15b0dd 2025-12-17 20:57:00,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407681575bc060ce15b0dd HTTP/1.1" 502 150 2025-12-17 20:57:00,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407681575bc060ce15b0dd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:00,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5261e3c24e95681d67d 2025-12-17 20:57:00,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5261e3c24e95681d67d HTTP/1.1" 502 150 2025-12-17 20:57:00,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5261e3c24e95681d67d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:00,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407683575bc060ce15b0e0 2025-12-17 20:57:00,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407683575bc060ce15b0e0 HTTP/1.1" 502 150 2025-12-17 20:57:00,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407683575bc060ce15b0e0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55584 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:57:01,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e07575bc060ce15c210 2025-12-17 20:57:01,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e07575bc060ce15c210 HTTP/1.1" 502 150 2025-12-17 20:57:01,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e07575bc060ce15c210'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:02,216 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072cc575bc060ce15acfd 2025-12-17 20:57:02,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072cc575bc060ce15acfd HTTP/1.1" 502 150 2025-12-17 20:57:02,218 - lava_callback - ERROR - Error processing callback for node 694072cc575bc060ce15acfd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072cc575bc060ce15acfd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072cc575bc060ce15acfd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072cc575bc060ce15acfd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:02,220 - lava_callback - INFO - Starting async job submit for node 6940741e575bc060ce15ae7c 2025-12-17 20:57:02,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:02,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940741e575bc060ce15ae7c HTTP/1.1" 502 150 2025-12-17 20:57:02,226 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741e575bc060ce15ae7c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:02,226 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940741e575bc060ce15ae7c 2025-12-17 20:57:02,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940741e575bc060ce15ae7c HTTP/1.1" 502 150 2025-12-17 20:57:02,227 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741e575bc060ce15ae7c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:03,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69406efe575bc060ce15a521 HTTP/1.1" 502 150 2025-12-17 20:57:03,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efe575bc060ce15a521'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:03,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407165575bc060ce15aab0 2025-12-17 20:57:03,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407165575bc060ce15aab0 HTTP/1.1" 502 150 2025-12-17 20:57:03,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407165575bc060ce15aab0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:03,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072bd575bc060ce15acc6 2025-12-17 20:57:03,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072bd575bc060ce15acc6 HTTP/1.1" 502 150 2025-12-17 20:57:03,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072bd575bc060ce15acc6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:04,227 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940741e575bc060ce15ae7c 2025-12-17 20:57:04,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940741e575bc060ce15ae7c HTTP/1.1" 502 150 2025-12-17 20:57:04,228 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741e575bc060ce15ae7c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:04,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072be575bc060ce15acc9 2025-12-17 20:57:04,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407687575bc060ce15b0eb 2025-12-17 20:57:04,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072be575bc060ce15acc9 HTTP/1.1" 502 150 2025-12-17 20:57:04,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072be575bc060ce15acc9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:04,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407681575bc060ce15b0dd 2025-12-17 20:57:04,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407687575bc060ce15b0eb HTTP/1.1" 502 150 2025-12-17 20:57:04,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407687575bc060ce15b0eb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:04,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407681575bc060ce15b0dd HTTP/1.1" 502 150 2025-12-17 20:57:04,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407681575bc060ce15b0dd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:04,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407683575bc060ce15b0e0 2025-12-17 20:57:04,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407683575bc060ce15b0e0 HTTP/1.1" 502 150 2025-12-17 20:57:04,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407683575bc060ce15b0e0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:06,220 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072cd575bc060ce15ad00 2025-12-17 20:57:06,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072cd575bc060ce15ad00 HTTP/1.1" 502 150 2025-12-17 20:57:06,222 - lava_callback - ERROR - Error processing callback for node 694072cd575bc060ce15ad00: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072cd575bc060ce15ad00 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072cd575bc060ce15ad00 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072cd575bc060ce15ad00 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:06,224 - lava_callback - INFO - Starting async job submit for node 69407a59575bc060ce15b7e9 2025-12-17 20:57:06,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:06,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a59575bc060ce15b7e9 HTTP/1.1" 502 150 2025-12-17 20:57:06,255 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a59575bc060ce15b7e9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:06,255 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a59575bc060ce15b7e9 2025-12-17 20:57:06,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a59575bc060ce15b7e9 HTTP/1.1" 502 150 2025-12-17 20:57:06,256 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a59575bc060ce15b7e9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55600 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:57:07,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69406efe575bc060ce15a521 2025-12-17 20:57:07,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69406efe575bc060ce15a521 HTTP/1.1" 502 150 2025-12-17 20:57:07,410 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efe575bc060ce15a521'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:07,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407686575bc060ce15b0e8 2025-12-17 20:57:07,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407686575bc060ce15b0e8 HTTP/1.1" 502 150 2025-12-17 20:57:07,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407686575bc060ce15b0e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:08,228 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940741e575bc060ce15ae7c 2025-12-17 20:57:08,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940741e575bc060ce15ae7c HTTP/1.1" 502 150 2025-12-17 20:57:08,230 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741e575bc060ce15ae7c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:08,256 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a59575bc060ce15b7e9 2025-12-17 20:57:08,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a59575bc060ce15b7e9 HTTP/1.1" 502 150 2025-12-17 20:57:08,257 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a59575bc060ce15b7e9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:09,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072ec575bc060ce15ad72 2025-12-17 20:57:09,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072ec575bc060ce15ad72 HTTP/1.1" 502 150 2025-12-17 20:57:09,022 - lava_callback - ERROR - Error processing callback for node 694072ec575bc060ce15ad72: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072ec575bc060ce15ad72 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072ec575bc060ce15ad72 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072ec575bc060ce15ad72 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:09,024 - lava_callback - INFO - Starting async job submit for node 694070c8575bc060ce15a966 2025-12-17 20:57:09,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:09,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070c8575bc060ce15a966 HTTP/1.1" 502 150 2025-12-17 20:57:09,076 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070c8575bc060ce15a966'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:09,076 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070c8575bc060ce15a966 2025-12-17 20:57:09,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070c8575bc060ce15a966 HTTP/1.1" 502 150 2025-12-17 20:57:09,077 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070c8575bc060ce15a966'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:09,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e07575bc060ce15c210 2025-12-17 20:57:09,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e07575bc060ce15c210 HTTP/1.1" 502 150 2025-12-17 20:57:09,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e07575bc060ce15c210'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:09,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940788b575bc060ce15b4e8 2025-12-17 20:57:09,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940788b575bc060ce15b4e8 HTTP/1.1" 502 150 2025-12-17 20:57:09,521 - lava_callback - ERROR - Error processing callback for node 6940788b575bc060ce15b4e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940788b575bc060ce15b4e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940788b575bc060ce15b4e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940788b575bc060ce15b4e8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:09,522 - lava_callback - INFO - Starting async job submit for node 69407684575bc060ce15b0e3 2025-12-17 20:57:09,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:09,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407684575bc060ce15b0e3 HTTP/1.1" 502 150 2025-12-17 20:57:09,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407684575bc060ce15b0e3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:09,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407684575bc060ce15b0e3 2025-12-17 20:57:09,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407684575bc060ce15b0e3 HTTP/1.1" 502 150 2025-12-17 20:57:09,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407684575bc060ce15b0e3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:10,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070d4575bc060ce15a987 2025-12-17 20:57:10,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d4575bc060ce15a987 HTTP/1.1" 502 150 2025-12-17 20:57:10,882 - lava_callback - ERROR - Error processing callback for node 694070d4575bc060ce15a987: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d4575bc060ce15a987 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d4575bc060ce15a987 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d4575bc060ce15a987 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:10,884 - lava_callback - INFO - Starting async job submit for node 69407688575bc060ce15b0ee 2025-12-17 20:57:10,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:10,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407688575bc060ce15b0ee HTTP/1.1" 502 150 2025-12-17 20:57:10,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407688575bc060ce15b0ee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:10,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407688575bc060ce15b0ee 2025-12-17 20:57:10,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407688575bc060ce15b0ee HTTP/1.1" 502 150 2025-12-17 20:57:10,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407688575bc060ce15b0ee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:11,077 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070c8575bc060ce15a966 2025-12-17 20:57:11,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070c8575bc060ce15a966 HTTP/1.1" 502 150 2025-12-17 20:57:11,078 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070c8575bc060ce15a966'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:11,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407684575bc060ce15b0e3 2025-12-17 20:57:11,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407684575bc060ce15b0e3 HTTP/1.1" 502 150 2025-12-17 20:57:11,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407684575bc060ce15b0e3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:11,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407165575bc060ce15aab0 2025-12-17 20:57:11,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407165575bc060ce15aab0 HTTP/1.1" 502 150 2025-12-17 20:57:11,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407165575bc060ce15aab0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:11,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072bd575bc060ce15acc6 2025-12-17 20:57:11,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072bd575bc060ce15acc6 HTTP/1.1" 502 150 2025-12-17 20:57:11,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072bd575bc060ce15acc6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:12,258 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a59575bc060ce15b7e9 2025-12-17 20:57:12,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a59575bc060ce15b7e9 HTTP/1.1" 502 150 2025-12-17 20:57:12,259 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a59575bc060ce15b7e9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:12,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072be575bc060ce15acc9 2025-12-17 20:57:12,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407687575bc060ce15b0eb 2025-12-17 20:57:12,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407681575bc060ce15b0dd 2025-12-17 20:57:12,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072be575bc060ce15acc9 HTTP/1.1" 502 150 2025-12-17 20:57:12,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694072be575bc060ce15acc9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:12,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407687575bc060ce15b0eb HTTP/1.1" 502 150 2025-12-17 20:57:12,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407687575bc060ce15b0eb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:12,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407681575bc060ce15b0dd HTTP/1.1" 502 150 2025-12-17 20:57:12,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407681575bc060ce15b0dd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:12,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e04575bc060ce15c205 2025-12-17 20:57:12,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e04575bc060ce15c205 HTTP/1.1" 502 150 2025-12-17 20:57:12,819 - lava_callback - ERROR - Error processing callback for node 69407e04575bc060ce15c205: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e04575bc060ce15c205 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e04575bc060ce15c205 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e04575bc060ce15c205 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:12,821 - lava_callback - INFO - Starting async job submit for node 6940708b575bc060ce15a8c1 2025-12-17 20:57:12,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407683575bc060ce15b0e0 2025-12-17 20:57:12,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:12,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407683575bc060ce15b0e0 HTTP/1.1" 502 150 2025-12-17 20:57:12,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407683575bc060ce15b0e0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:12,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940708b575bc060ce15a8c1 HTTP/1.1" 502 150 2025-12-17 20:57:12,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940708b575bc060ce15a8c1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:12,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940708b575bc060ce15a8c1 2025-12-17 20:57:12,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940708b575bc060ce15a8c1 HTTP/1.1" 502 150 2025-12-17 20:57:12,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940708b575bc060ce15a8c1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:12,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407688575bc060ce15b0ee 2025-12-17 20:57:12,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407688575bc060ce15b0ee HTTP/1.1" 502 150 2025-12-17 20:57:12,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407688575bc060ce15b0ee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:14,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940708b575bc060ce15a8c1 2025-12-17 20:57:14,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940708b575bc060ce15a8c1 HTTP/1.1" 502 150 2025-12-17 20:57:14,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940708b575bc060ce15a8c1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:15,079 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070c8575bc060ce15a966 2025-12-17 20:57:15,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070c8575bc060ce15a966 HTTP/1.1" 502 150 2025-12-17 20:57:15,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070c8575bc060ce15a966'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59222 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:57:15,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69406efe575bc060ce15a521 2025-12-17 20:57:15,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69406efe575bc060ce15a521 HTTP/1.1" 502 150 2025-12-17 20:57:15,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69406efe575bc060ce15a521'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:15,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407684575bc060ce15b0e3 2025-12-17 20:57:15,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407684575bc060ce15b0e3 HTTP/1.1" 502 150 2025-12-17 20:57:15,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407684575bc060ce15b0e3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:16,230 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940741e575bc060ce15ae7c 2025-12-17 20:57:16,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940741e575bc060ce15ae7c HTTP/1.1" 502 150 2025-12-17 20:57:16,231 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741e575bc060ce15ae7c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59238 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:57:16,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5261e3c24e95681d67d 2025-12-17 20:57:16,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5261e3c24e95681d67d HTTP/1.1" 502 150 2025-12-17 20:57:16,673 - lava_callback - ERROR - Error processing callback for node 6940c5261e3c24e95681d67d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5261e3c24e95681d67d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5261e3c24e95681d67d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5261e3c24e95681d67d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:16,674 - lava_callback - INFO - Starting async job submit for node 69407458575bc060ce15aea6 2025-12-17 20:57:16,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:16,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407458575bc060ce15aea6 HTTP/1.1" 502 150 2025-12-17 20:57:16,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407458575bc060ce15aea6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:16,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407458575bc060ce15aea6 2025-12-17 20:57:16,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407458575bc060ce15aea6 HTTP/1.1" 502 150 2025-12-17 20:57:16,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407458575bc060ce15aea6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:16,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407688575bc060ce15b0ee 2025-12-17 20:57:16,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407688575bc060ce15b0ee HTTP/1.1" 502 150 2025-12-17 20:57:16,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407688575bc060ce15b0ee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:18,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407458575bc060ce15aea6 2025-12-17 20:57:18,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407458575bc060ce15aea6 HTTP/1.1" 502 150 2025-12-17 20:57:18,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407458575bc060ce15aea6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:18,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940708b575bc060ce15a8c1 2025-12-17 20:57:18,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940708b575bc060ce15a8c1 HTTP/1.1" 502 150 2025-12-17 20:57:18,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940708b575bc060ce15a8c1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:20,259 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a59575bc060ce15b7e9 2025-12-17 20:57:20,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a59575bc060ce15b7e9 HTTP/1.1" 502 150 2025-12-17 20:57:20,260 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a59575bc060ce15b7e9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:22,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407458575bc060ce15aea6 2025-12-17 20:57:22,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407458575bc060ce15aea6 HTTP/1.1" 502 150 2025-12-17 20:57:22,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407458575bc060ce15aea6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:23,080 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070c8575bc060ce15a966 2025-12-17 20:57:23,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070c8575bc060ce15a966 HTTP/1.1" 502 150 2025-12-17 20:57:23,082 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070c8575bc060ce15a966'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:23,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407684575bc060ce15b0e3 2025-12-17 20:57:23,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407684575bc060ce15b0e3 HTTP/1.1" 502 150 2025-12-17 20:57:23,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407684575bc060ce15b0e3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:23,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407686575bc060ce15b0e8 2025-12-17 20:57:23,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407686575bc060ce15b0e8 HTTP/1.1" 502 150 2025-12-17 20:57:23,590 - lava_callback - ERROR - Error processing callback for node 69407686575bc060ce15b0e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407686575bc060ce15b0e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407686575bc060ce15b0e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407686575bc060ce15b0e8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:23,593 - lava_callback - INFO - Starting async job submit for node 69407692575bc060ce15b112 2025-12-17 20:57:23,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:23,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407692575bc060ce15b112 HTTP/1.1" 502 150 2025-12-17 20:57:23,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407692575bc060ce15b112'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:23,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407692575bc060ce15b112 2025-12-17 20:57:23,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407692575bc060ce15b112 HTTP/1.1" 502 150 2025-12-17 20:57:23,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407692575bc060ce15b112'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:24,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407688575bc060ce15b0ee 2025-12-17 20:57:24,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407688575bc060ce15b0ee HTTP/1.1" 502 150 2025-12-17 20:57:24,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407688575bc060ce15b0ee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:25,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e07575bc060ce15c210 2025-12-17 20:57:25,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e07575bc060ce15c210 HTTP/1.1" 502 150 2025-12-17 20:57:25,479 - lava_callback - ERROR - Error processing callback for node 69407e07575bc060ce15c210: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e07575bc060ce15c210 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e07575bc060ce15c210 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e07575bc060ce15c210 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:25,481 - lava_callback - INFO - Starting async job submit for node 694087c8575bc060ce15e4c2 2025-12-17 20:57:25,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:25,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087c8575bc060ce15e4c2 HTTP/1.1" 502 150 2025-12-17 20:57:25,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087c8575bc060ce15e4c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:25,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087c8575bc060ce15e4c2 2025-12-17 20:57:25,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087c8575bc060ce15e4c2 HTTP/1.1" 502 150 2025-12-17 20:57:25,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087c8575bc060ce15e4c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:25,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407692575bc060ce15b112 2025-12-17 20:57:25,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407692575bc060ce15b112 HTTP/1.1" 502 150 2025-12-17 20:57:25,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407692575bc060ce15b112'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54752 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:57:26,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940708b575bc060ce15a8c1 2025-12-17 20:57:26,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940708b575bc060ce15a8c1 HTTP/1.1" 502 150 2025-12-17 20:57:26,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940708b575bc060ce15a8c1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:27,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087c8575bc060ce15e4c2 2025-12-17 20:57:27,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087c8575bc060ce15e4c2 HTTP/1.1" 502 150 2025-12-17 20:57:27,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087c8575bc060ce15e4c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:27,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407165575bc060ce15aab0 2025-12-17 20:57:27,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407165575bc060ce15aab0 HTTP/1.1" 502 150 2025-12-17 20:57:27,633 - lava_callback - ERROR - Error processing callback for node 69407165575bc060ce15aab0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407165575bc060ce15aab0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407165575bc060ce15aab0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407165575bc060ce15aab0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:27,635 - lava_callback - INFO - Starting async job submit for node 694087cd575bc060ce15e4d4 2025-12-17 20:57:27,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:27,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087cd575bc060ce15e4d4 HTTP/1.1" 502 150 2025-12-17 20:57:27,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087cd575bc060ce15e4d4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:27,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087cd575bc060ce15e4d4 2025-12-17 20:57:27,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087cd575bc060ce15e4d4 HTTP/1.1" 502 150 2025-12-17 20:57:27,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087cd575bc060ce15e4d4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:27,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072bd575bc060ce15acc6 2025-12-17 20:57:27,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072bd575bc060ce15acc6 HTTP/1.1" 502 150 2025-12-17 20:57:27,850 - lava_callback - ERROR - Error processing callback for node 694072bd575bc060ce15acc6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072bd575bc060ce15acc6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072bd575bc060ce15acc6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072bd575bc060ce15acc6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:27,851 - lava_callback - INFO - Starting async job submit for node 69407693575bc060ce15b119 2025-12-17 20:57:27,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:27,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407693575bc060ce15b119 HTTP/1.1" 502 150 2025-12-17 20:57:27,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407693575bc060ce15b119'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:27,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407693575bc060ce15b119 2025-12-17 20:57:27,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407693575bc060ce15b119 HTTP/1.1" 502 150 2025-12-17 20:57:27,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407693575bc060ce15b119'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:28,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694072be575bc060ce15acc9 2025-12-17 20:57:28,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407687575bc060ce15b0eb 2025-12-17 20:57:28,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407681575bc060ce15b0dd 2025-12-17 20:57:28,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694072be575bc060ce15acc9 HTTP/1.1" 502 150 2025-12-17 20:57:28,421 - lava_callback - ERROR - Error processing callback for node 694072be575bc060ce15acc9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072be575bc060ce15acc9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072be575bc060ce15acc9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694072be575bc060ce15acc9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:28,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407687575bc060ce15b0eb HTTP/1.1" 502 150 2025-12-17 20:57:28,424 - lava_callback - ERROR - Error processing callback for node 69407687575bc060ce15b0eb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407687575bc060ce15b0eb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407687575bc060ce15b0eb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407687575bc060ce15b0eb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:28,424 - lava_callback - INFO - Starting async job submit for node 694087d4575bc060ce15e4dd 2025-12-17 20:57:28,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407681575bc060ce15b0dd HTTP/1.1" 502 150 2025-12-17 20:57:28,426 - lava_callback - INFO - Starting async job submit for node 6940768c575bc060ce15b0f7 2025-12-17 20:57:28,431 - lava_callback - ERROR - Error processing callback for node 69407681575bc060ce15b0dd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407681575bc060ce15b0dd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407681575bc060ce15b0dd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407681575bc060ce15b0dd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:28,493 - lava_callback - INFO - Starting async job submit for node 6940881e575bc060ce15e5e2 2025-12-17 20:57:28,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:28,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:28,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:28,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087d4575bc060ce15e4dd HTTP/1.1" 502 150 2025-12-17 20:57:28,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087d4575bc060ce15e4dd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:28,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087d4575bc060ce15e4dd 2025-12-17 20:57:28,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087d4575bc060ce15e4dd HTTP/1.1" 502 150 2025-12-17 20:57:28,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087d4575bc060ce15e4dd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:28,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768c575bc060ce15b0f7 HTTP/1.1" 502 150 2025-12-17 20:57:28,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768c575bc060ce15b0f7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:28,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768c575bc060ce15b0f7 2025-12-17 20:57:28,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768c575bc060ce15b0f7 HTTP/1.1" 502 150 2025-12-17 20:57:28,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768c575bc060ce15b0f7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:28,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881e575bc060ce15e5e2 HTTP/1.1" 502 150 2025-12-17 20:57:28,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881e575bc060ce15e5e2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:28,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881e575bc060ce15e5e2 2025-12-17 20:57:28,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881e575bc060ce15e5e2 HTTP/1.1" 502 150 2025-12-17 20:57:28,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881e575bc060ce15e5e2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:28,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407683575bc060ce15b0e0 2025-12-17 20:57:28,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407683575bc060ce15b0e0 HTTP/1.1" 502 150 2025-12-17 20:57:28,855 - lava_callback - ERROR - Error processing callback for node 69407683575bc060ce15b0e0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407683575bc060ce15b0e0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407683575bc060ce15b0e0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407683575bc060ce15b0e0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:28,857 - lava_callback - INFO - Starting async job submit for node 6940768d575bc060ce15b0fa 2025-12-17 20:57:28,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:28,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768d575bc060ce15b0fa HTTP/1.1" 502 150 2025-12-17 20:57:28,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768d575bc060ce15b0fa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:28,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768d575bc060ce15b0fa 2025-12-17 20:57:28,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768d575bc060ce15b0fa HTTP/1.1" 502 150 2025-12-17 20:57:28,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768d575bc060ce15b0fa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:29,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407692575bc060ce15b112 2025-12-17 20:57:29,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407692575bc060ce15b112 HTTP/1.1" 502 150 2025-12-17 20:57:29,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407692575bc060ce15b112'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:29,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087cd575bc060ce15e4d4 2025-12-17 20:57:29,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087cd575bc060ce15e4d4 HTTP/1.1" 502 150 2025-12-17 20:57:29,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087cd575bc060ce15e4d4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:29,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407693575bc060ce15b119 2025-12-17 20:57:29,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407693575bc060ce15b119 HTTP/1.1" 502 150 2025-12-17 20:57:29,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407693575bc060ce15b119'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:30,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087d4575bc060ce15e4dd 2025-12-17 20:57:30,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087d4575bc060ce15e4dd HTTP/1.1" 502 150 2025-12-17 20:57:30,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087d4575bc060ce15e4dd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:30,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768c575bc060ce15b0f7 2025-12-17 20:57:30,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768c575bc060ce15b0f7 HTTP/1.1" 502 150 2025-12-17 20:57:30,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768c575bc060ce15b0f7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:30,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881e575bc060ce15e5e2 2025-12-17 20:57:30,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881e575bc060ce15e5e2 HTTP/1.1" 502 150 2025-12-17 20:57:30,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881e575bc060ce15e5e2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:30,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407458575bc060ce15aea6 2025-12-17 20:57:30,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407458575bc060ce15aea6 HTTP/1.1" 502 150 2025-12-17 20:57:30,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407458575bc060ce15aea6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:30,812 - lava_callback - INFO - Received callback for node 694078d8575bc060ce15b57c INFO: 172.21.0.1:36024 - "POST /node/694078d8575bc060ce15b57c HTTP/1.0" 202 Accepted 2025-12-17 20:57:30,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768d575bc060ce15b0fa 2025-12-17 20:57:30,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768d575bc060ce15b0fa HTTP/1.1" 502 150 2025-12-17 20:57:30,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768d575bc060ce15b0fa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:31,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69406efe575bc060ce15a521 2025-12-17 20:57:31,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69406efe575bc060ce15a521 HTTP/1.1" 502 150 2025-12-17 20:57:31,413 - lava_callback - ERROR - Error processing callback for node 69406efe575bc060ce15a521: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406efe575bc060ce15a521 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406efe575bc060ce15a521 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69406efe575bc060ce15a521 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:31,417 - lava_callback - INFO - Starting async job submit for node 69407bed575bc060ce15bc22 2025-12-17 20:57:31,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:31,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bed575bc060ce15bc22 HTTP/1.1" 502 150 2025-12-17 20:57:31,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bed575bc060ce15bc22'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:31,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bed575bc060ce15bc22 2025-12-17 20:57:31,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bed575bc060ce15bc22 HTTP/1.1" 502 150 2025-12-17 20:57:31,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bed575bc060ce15bc22'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:31,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087c8575bc060ce15e4c2 2025-12-17 20:57:31,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087c8575bc060ce15e4c2 HTTP/1.1" 502 150 2025-12-17 20:57:31,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087c8575bc060ce15e4c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:32,231 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940741e575bc060ce15ae7c 2025-12-17 20:57:33,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bed575bc060ce15bc22 2025-12-17 20:57:33,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bed575bc060ce15bc22 HTTP/1.1" 502 150 2025-12-17 20:57:33,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bed575bc060ce15bc22'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:33,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087cd575bc060ce15e4d4 2025-12-17 20:57:33,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087cd575bc060ce15e4d4 HTTP/1.1" 502 150 2025-12-17 20:57:33,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087cd575bc060ce15e4d4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:33,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407693575bc060ce15b119 2025-12-17 20:57:33,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407693575bc060ce15b119 HTTP/1.1" 502 150 2025-12-17 20:57:33,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407693575bc060ce15b119'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:34,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087d4575bc060ce15e4dd 2025-12-17 20:57:34,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087d4575bc060ce15e4dd HTTP/1.1" 502 150 2025-12-17 20:57:34,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087d4575bc060ce15e4dd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:34,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768c575bc060ce15b0f7 2025-12-17 20:57:34,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768c575bc060ce15b0f7 HTTP/1.1" 502 150 2025-12-17 20:57:34,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768c575bc060ce15b0f7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:34,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881e575bc060ce15e5e2 2025-12-17 20:57:34,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881e575bc060ce15e5e2 HTTP/1.1" 502 150 2025-12-17 20:57:34,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881e575bc060ce15e5e2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:34,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768d575bc060ce15b0fa 2025-12-17 20:57:34,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768d575bc060ce15b0fa HTTP/1.1" 502 150 2025-12-17 20:57:34,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768d575bc060ce15b0fa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:36,261 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a59575bc060ce15b7e9 INFO: 172.21.0.1:36028 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:57:37,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bed575bc060ce15bc22 2025-12-17 20:57:37,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bed575bc060ce15bc22 HTTP/1.1" 502 150 2025-12-17 20:57:37,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bed575bc060ce15bc22'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:37,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407692575bc060ce15b112 2025-12-17 20:57:37,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407692575bc060ce15b112 HTTP/1.1" 502 150 2025-12-17 20:57:37,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407692575bc060ce15b112'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:39,082 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070c8575bc060ce15a966 2025-12-17 20:57:39,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070c8575bc060ce15a966 HTTP/1.1" 502 150 2025-12-17 20:57:39,083 - lava_callback - ERROR - Error processing callback for node 694070c8575bc060ce15a966: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070c8575bc060ce15a966 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070c8575bc060ce15a966 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070c8575bc060ce15a966 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:39,085 - lava_callback - INFO - Starting async job submit for node 69408a8a575bc060ce15eea5 2025-12-17 20:57:39,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:39,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8a575bc060ce15eea5 HTTP/1.1" 502 150 2025-12-17 20:57:39,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8a575bc060ce15eea5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:39,113 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8a575bc060ce15eea5 2025-12-17 20:57:39,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8a575bc060ce15eea5 HTTP/1.1" 502 150 2025-12-17 20:57:39,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8a575bc060ce15eea5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:39,223 - lava_callback - INFO - Received callback for node 69407691575bc060ce15b107 INFO: 172.21.0.1:36042 - "POST /node/69407691575bc060ce15b107 HTTP/1.0" 202 Accepted 2025-12-17 20:57:39,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087c8575bc060ce15e4c2 2025-12-17 20:57:39,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407684575bc060ce15b0e3 2025-12-17 20:57:39,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087c8575bc060ce15e4c2 HTTP/1.1" 502 150 2025-12-17 20:57:39,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087c8575bc060ce15e4c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:39,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407684575bc060ce15b0e3 HTTP/1.1" 502 150 2025-12-17 20:57:39,558 - lava_callback - ERROR - Error processing callback for node 69407684575bc060ce15b0e3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407684575bc060ce15b0e3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407684575bc060ce15b0e3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407684575bc060ce15b0e3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:39,560 - lava_callback - INFO - Starting async job submit for node 6940768f575bc060ce15b101 2025-12-17 20:57:39,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:39,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768f575bc060ce15b101 HTTP/1.1" 502 150 2025-12-17 20:57:39,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768f575bc060ce15b101'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:39,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768f575bc060ce15b101 2025-12-17 20:57:39,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768f575bc060ce15b101 HTTP/1.1" 502 150 2025-12-17 20:57:39,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768f575bc060ce15b101'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:40,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407688575bc060ce15b0ee 2025-12-17 20:57:40,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407688575bc060ce15b0ee HTTP/1.1" 502 150 2025-12-17 20:57:40,918 - lava_callback - ERROR - Error processing callback for node 69407688575bc060ce15b0ee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407688575bc060ce15b0ee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407688575bc060ce15b0ee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407688575bc060ce15b0ee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:40,921 - lava_callback - INFO - Starting async job submit for node 69407695575bc060ce15b11c 2025-12-17 20:57:40,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:40,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407695575bc060ce15b11c HTTP/1.1" 502 150 2025-12-17 20:57:40,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407695575bc060ce15b11c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:40,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407695575bc060ce15b11c 2025-12-17 20:57:40,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407695575bc060ce15b11c HTTP/1.1" 502 150 2025-12-17 20:57:40,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407695575bc060ce15b11c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:41,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8a575bc060ce15eea5 2025-12-17 20:57:41,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8a575bc060ce15eea5 HTTP/1.1" 502 150 2025-12-17 20:57:41,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8a575bc060ce15eea5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:41,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768f575bc060ce15b101 2025-12-17 20:57:41,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768f575bc060ce15b101 HTTP/1.1" 502 150 2025-12-17 20:57:41,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768f575bc060ce15b101'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:41,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087cd575bc060ce15e4d4 2025-12-17 20:57:41,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087cd575bc060ce15e4d4 HTTP/1.1" 502 150 2025-12-17 20:57:41,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087cd575bc060ce15e4d4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:41,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407693575bc060ce15b119 2025-12-17 20:57:41,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407693575bc060ce15b119 HTTP/1.1" 502 150 2025-12-17 20:57:41,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407693575bc060ce15b119'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:42,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087d4575bc060ce15e4dd 2025-12-17 20:57:42,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768c575bc060ce15b0f7 2025-12-17 20:57:42,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087d4575bc060ce15e4dd HTTP/1.1" 502 150 2025-12-17 20:57:42,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087d4575bc060ce15e4dd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:42,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768c575bc060ce15b0f7 HTTP/1.1" 502 150 2025-12-17 20:57:42,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768c575bc060ce15b0f7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:42,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881e575bc060ce15e5e2 2025-12-17 20:57:42,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881e575bc060ce15e5e2 HTTP/1.1" 502 150 2025-12-17 20:57:42,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881e575bc060ce15e5e2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:42,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940708b575bc060ce15a8c1 2025-12-17 20:57:42,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940708b575bc060ce15a8c1 HTTP/1.1" 502 150 2025-12-17 20:57:42,871 - lava_callback - ERROR - Error processing callback for node 6940708b575bc060ce15a8c1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940708b575bc060ce15a8c1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940708b575bc060ce15a8c1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940708b575bc060ce15a8c1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:42,873 - lava_callback - INFO - Starting async job submit for node 69408830575bc060ce15e64f 2025-12-17 20:57:42,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:42,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408830575bc060ce15e64f HTTP/1.1" 502 150 2025-12-17 20:57:42,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408830575bc060ce15e64f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:42,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408830575bc060ce15e64f 2025-12-17 20:57:42,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408830575bc060ce15e64f HTTP/1.1" 502 150 2025-12-17 20:57:42,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408830575bc060ce15e64f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:42,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407695575bc060ce15b11c 2025-12-17 20:57:42,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407695575bc060ce15b11c HTTP/1.1" 502 150 2025-12-17 20:57:42,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407695575bc060ce15b11c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:42,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768d575bc060ce15b0fa 2025-12-17 20:57:42,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768d575bc060ce15b0fa HTTP/1.1" 502 150 2025-12-17 20:57:42,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768d575bc060ce15b0fa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:44,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408830575bc060ce15e64f 2025-12-17 20:57:44,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408830575bc060ce15e64f HTTP/1.1" 502 150 2025-12-17 20:57:44,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408830575bc060ce15e64f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:45,115 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8a575bc060ce15eea5 2025-12-17 20:57:45,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8a575bc060ce15eea5 HTTP/1.1" 502 150 2025-12-17 20:57:45,116 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8a575bc060ce15eea5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:45,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bed575bc060ce15bc22 2025-12-17 20:57:45,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bed575bc060ce15bc22 HTTP/1.1" 502 150 2025-12-17 20:57:45,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bed575bc060ce15bc22'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:45,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768f575bc060ce15b101 2025-12-17 20:57:45,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768f575bc060ce15b101 HTTP/1.1" 502 150 2025-12-17 20:57:45,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768f575bc060ce15b101'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36220 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:57:46,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407458575bc060ce15aea6 2025-12-17 20:57:46,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407458575bc060ce15aea6 HTTP/1.1" 502 150 2025-12-17 20:57:46,693 - lava_callback - ERROR - Error processing callback for node 69407458575bc060ce15aea6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407458575bc060ce15aea6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407458575bc060ce15aea6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407458575bc060ce15aea6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:46,696 - lava_callback - INFO - Starting async job submit for node 6940741e575bc060ce15ae79 2025-12-17 20:57:46,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:46,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940741e575bc060ce15ae79 HTTP/1.1" 502 150 2025-12-17 20:57:46,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741e575bc060ce15ae79'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:46,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940741e575bc060ce15ae79 2025-12-17 20:57:46,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940741e575bc060ce15ae79 HTTP/1.1" 502 150 2025-12-17 20:57:46,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741e575bc060ce15ae79'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:46,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407695575bc060ce15b11c 2025-12-17 20:57:46,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407695575bc060ce15b11c HTTP/1.1" 502 150 2025-12-17 20:57:46,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407695575bc060ce15b11c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:47,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940741e575bc060ce15ae7c HTTP/1.1" 502 150 2025-12-17 20:57:47,696 - lava_callback - ERROR - Error processing callback for node 6940741e575bc060ce15ae7c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940741e575bc060ce15ae7c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940741e575bc060ce15ae7c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940741e575bc060ce15ae7c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:47,698 - lava_callback - INFO - Starting async job submit for node 69407bf0575bc060ce15bc25 2025-12-17 20:57:47,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:47,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf0575bc060ce15bc25 HTTP/1.1" 502 150 2025-12-17 20:57:47,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf0575bc060ce15bc25'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:47,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf0575bc060ce15bc25 2025-12-17 20:57:47,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf0575bc060ce15bc25 HTTP/1.1" 502 150 2025-12-17 20:57:47,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf0575bc060ce15bc25'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:48,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940741e575bc060ce15ae79 2025-12-17 20:57:48,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940741e575bc060ce15ae79 HTTP/1.1" 502 150 2025-12-17 20:57:48,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741e575bc060ce15ae79'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:48,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408830575bc060ce15e64f 2025-12-17 20:57:48,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408830575bc060ce15e64f HTTP/1.1" 502 150 2025-12-17 20:57:48,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408830575bc060ce15e64f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:49,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf0575bc060ce15bc25 2025-12-17 20:57:49,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf0575bc060ce15bc25 HTTP/1.1" 502 150 2025-12-17 20:57:49,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf0575bc060ce15bc25'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:51,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a59575bc060ce15b7e9 HTTP/1.1" 502 150 2025-12-17 20:57:51,536 - lava_callback - ERROR - Error processing callback for node 69407a59575bc060ce15b7e9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407a59575bc060ce15b7e9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407a59575bc060ce15b7e9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407a59575bc060ce15b7e9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:51,538 - lava_callback - INFO - Starting async job submit for node 694078d8575bc060ce15b57c 2025-12-17 20:57:51,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:51,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078d8575bc060ce15b57c HTTP/1.1" 502 150 2025-12-17 20:57:51,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078d8575bc060ce15b57c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:51,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078d8575bc060ce15b57c 2025-12-17 20:57:51,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078d8575bc060ce15b57c HTTP/1.1" 502 150 2025-12-17 20:57:51,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078d8575bc060ce15b57c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:52,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940741e575bc060ce15ae79 2025-12-17 20:57:52,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940741e575bc060ce15ae79 HTTP/1.1" 502 150 2025-12-17 20:57:52,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741e575bc060ce15ae79'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:53,116 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8a575bc060ce15eea5 2025-12-17 20:57:53,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8a575bc060ce15eea5 HTTP/1.1" 502 150 2025-12-17 20:57:53,118 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8a575bc060ce15eea5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:53,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078d8575bc060ce15b57c 2025-12-17 20:57:53,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078d8575bc060ce15b57c HTTP/1.1" 502 150 2025-12-17 20:57:53,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078d8575bc060ce15b57c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:53,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768f575bc060ce15b101 2025-12-17 20:57:53,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768f575bc060ce15b101 HTTP/1.1" 502 150 2025-12-17 20:57:53,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768f575bc060ce15b101'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:53,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407692575bc060ce15b112 2025-12-17 20:57:53,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407692575bc060ce15b112 HTTP/1.1" 502 150 2025-12-17 20:57:53,646 - lava_callback - ERROR - Error processing callback for node 69407692575bc060ce15b112: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407692575bc060ce15b112 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407692575bc060ce15b112 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407692575bc060ce15b112 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:53,649 - lava_callback - INFO - Starting async job submit for node 69407691575bc060ce15b107 2025-12-17 20:57:53,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:57:53,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407691575bc060ce15b107 HTTP/1.1" 502 150 2025-12-17 20:57:53,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407691575bc060ce15b107'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:53,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407691575bc060ce15b107 2025-12-17 20:57:53,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407691575bc060ce15b107 HTTP/1.1" 502 150 2025-12-17 20:57:53,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407691575bc060ce15b107'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:53,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf0575bc060ce15bc25 2025-12-17 20:57:53,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf0575bc060ce15bc25 HTTP/1.1" 502 150 2025-12-17 20:57:53,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf0575bc060ce15bc25'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:54,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407695575bc060ce15b11c 2025-12-17 20:57:54,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407695575bc060ce15b11c HTTP/1.1" 502 150 2025-12-17 20:57:54,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407695575bc060ce15b11c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:55,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087c8575bc060ce15e4c2 2025-12-17 20:57:55,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087c8575bc060ce15e4c2 HTTP/1.1" 502 150 2025-12-17 20:57:55,559 - lava_callback - ERROR - Error processing callback for node 694087c8575bc060ce15e4c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087c8575bc060ce15e4c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087c8575bc060ce15e4c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087c8575bc060ce15e4c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:55,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407691575bc060ce15b107 2025-12-17 20:57:55,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407691575bc060ce15b107 HTTP/1.1" 502 150 2025-12-17 20:57:55,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407691575bc060ce15b107'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59848 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:57:56,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408830575bc060ce15e64f 2025-12-17 20:57:56,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408830575bc060ce15e64f HTTP/1.1" 502 150 2025-12-17 20:57:56,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408830575bc060ce15e64f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:57,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078d8575bc060ce15b57c 2025-12-17 20:57:57,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078d8575bc060ce15b57c HTTP/1.1" 502 150 2025-12-17 20:57:57,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078d8575bc060ce15b57c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:57:57,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087cd575bc060ce15e4d4 2025-12-17 20:57:57,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087cd575bc060ce15e4d4 HTTP/1.1" 502 150 2025-12-17 20:57:57,737 - lava_callback - ERROR - Error processing callback for node 694087cd575bc060ce15e4d4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087cd575bc060ce15e4d4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087cd575bc060ce15e4d4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087cd575bc060ce15e4d4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:57,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407693575bc060ce15b119 2025-12-17 20:57:57,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407693575bc060ce15b119 HTTP/1.1" 502 150 2025-12-17 20:57:57,886 - lava_callback - ERROR - Error processing callback for node 69407693575bc060ce15b119: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407693575bc060ce15b119 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407693575bc060ce15b119 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407693575bc060ce15b119 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:58,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087d4575bc060ce15e4dd 2025-12-17 20:57:58,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768c575bc060ce15b0f7 2025-12-17 20:57:58,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087d4575bc060ce15e4dd HTTP/1.1" 502 150 2025-12-17 20:57:58,595 - lava_callback - ERROR - Error processing callback for node 694087d4575bc060ce15e4dd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087d4575bc060ce15e4dd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087d4575bc060ce15e4dd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087d4575bc060ce15e4dd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:58,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881e575bc060ce15e5e2 2025-12-17 20:57:58,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768c575bc060ce15b0f7 HTTP/1.1" 502 150 2025-12-17 20:57:58,597 - lava_callback - ERROR - Error processing callback for node 6940768c575bc060ce15b0f7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940768c575bc060ce15b0f7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940768c575bc060ce15b0f7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940768c575bc060ce15b0f7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:58,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881e575bc060ce15e5e2 HTTP/1.1" 502 150 2025-12-17 20:57:58,600 - lava_callback - ERROR - Error processing callback for node 6940881e575bc060ce15e5e2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940881e575bc060ce15e5e2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940881e575bc060ce15e5e2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940881e575bc060ce15e5e2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:58,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768d575bc060ce15b0fa 2025-12-17 20:57:58,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768d575bc060ce15b0fa HTTP/1.1" 502 150 2025-12-17 20:57:58,973 - lava_callback - ERROR - Error processing callback for node 6940768d575bc060ce15b0fa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940768d575bc060ce15b0fa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940768d575bc060ce15b0fa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940768d575bc060ce15b0fa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:57:59,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407691575bc060ce15b107 2025-12-17 20:57:59,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407691575bc060ce15b107 HTTP/1.1" 502 150 2025-12-17 20:57:59,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407691575bc060ce15b107'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:00,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940741e575bc060ce15ae79 2025-12-17 20:58:00,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940741e575bc060ce15ae79 HTTP/1.1" 502 150 2025-12-17 20:58:00,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940741e575bc060ce15ae79'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36424 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:58:01,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bed575bc060ce15bc22 2025-12-17 20:58:01,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bed575bc060ce15bc22 HTTP/1.1" 502 150 2025-12-17 20:58:01,460 - lava_callback - ERROR - Error processing callback for node 69407bed575bc060ce15bc22: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bed575bc060ce15bc22 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bed575bc060ce15bc22 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bed575bc060ce15bc22 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:58:01,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf0575bc060ce15bc25 2025-12-17 20:58:01,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf0575bc060ce15bc25 HTTP/1.1" 502 150 2025-12-17 20:58:01,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf0575bc060ce15bc25'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:05,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078d8575bc060ce15b57c 2025-12-17 20:58:05,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078d8575bc060ce15b57c HTTP/1.1" 502 150 2025-12-17 20:58:05,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078d8575bc060ce15b57c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36438 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:58:07,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407691575bc060ce15b107 2025-12-17 20:58:07,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407691575bc060ce15b107 HTTP/1.1" 502 150 2025-12-17 20:58:07,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407691575bc060ce15b107'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:09,118 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8a575bc060ce15eea5 2025-12-17 20:58:09,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8a575bc060ce15eea5 HTTP/1.1" 502 150 2025-12-17 20:58:09,119 - lava_callback - ERROR - Error processing callback for node 69408a8a575bc060ce15eea5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a8a575bc060ce15eea5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a8a575bc060ce15eea5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a8a575bc060ce15eea5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:58:09,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768f575bc060ce15b101 2025-12-17 20:58:09,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768f575bc060ce15b101 HTTP/1.1" 502 150 2025-12-17 20:58:09,614 - lava_callback - ERROR - Error processing callback for node 6940768f575bc060ce15b101: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940768f575bc060ce15b101 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940768f575bc060ce15b101 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940768f575bc060ce15b101 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:58:10,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407695575bc060ce15b11c 2025-12-17 20:58:10,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407695575bc060ce15b11c HTTP/1.1" 502 150 2025-12-17 20:58:10,964 - lava_callback - ERROR - Error processing callback for node 69407695575bc060ce15b11c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407695575bc060ce15b11c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407695575bc060ce15b11c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407695575bc060ce15b11c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:58:12,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408830575bc060ce15e64f 2025-12-17 20:58:12,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408830575bc060ce15e64f HTTP/1.1" 502 150 2025-12-17 20:58:12,950 - lava_callback - ERROR - Error processing callback for node 69408830575bc060ce15e64f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408830575bc060ce15e64f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408830575bc060ce15e64f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408830575bc060ce15e64f (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:59152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59162 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:58:16,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940741e575bc060ce15ae79 2025-12-17 20:58:16,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940741e575bc060ce15ae79 HTTP/1.1" 502 150 2025-12-17 20:58:16,815 - lava_callback - ERROR - Error processing callback for node 6940741e575bc060ce15ae79: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940741e575bc060ce15ae79 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940741e575bc060ce15ae79 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940741e575bc060ce15ae79 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:58:17,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf0575bc060ce15bc25 2025-12-17 20:58:17,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf0575bc060ce15bc25 HTTP/1.1" 502 150 2025-12-17 20:58:17,738 - lava_callback - ERROR - Error processing callback for node 69407bf0575bc060ce15bc25: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bf0575bc060ce15bc25 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bf0575bc060ce15bc25 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bf0575bc060ce15bc25 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:58:21,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078d8575bc060ce15b57c 2025-12-17 20:58:21,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078d8575bc060ce15b57c HTTP/1.1" 502 150 2025-12-17 20:58:21,574 - lava_callback - ERROR - Error processing callback for node 694078d8575bc060ce15b57c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078d8575bc060ce15b57c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078d8575bc060ce15b57c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078d8575bc060ce15b57c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:58:23,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407691575bc060ce15b107 2025-12-17 20:58:23,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407691575bc060ce15b107 HTTP/1.1" 502 150 2025-12-17 20:58:23,701 - lava_callback - ERROR - Error processing callback for node 69407691575bc060ce15b107: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407691575bc060ce15b107 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407691575bc060ce15b107 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407691575bc060ce15b107 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:43740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54498 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:58:39,686 - lava_callback - INFO - Received callback for node 694076bc575bc060ce15b177 INFO: 172.21.0.1:54512 - "POST /node/694076bc575bc060ce15b177 HTTP/1.0" 202 Accepted 2025-12-17 20:58:39,694 - lava_callback - INFO - Starting async job submit for node 694076bc575bc060ce15b177 2025-12-17 20:58:39,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:58:39,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bc575bc060ce15b177 HTTP/1.1" 502 150 2025-12-17 20:58:39,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076bc575bc060ce15b177'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:39,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076bc575bc060ce15b177 2025-12-17 20:58:39,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bc575bc060ce15b177 HTTP/1.1" 502 150 2025-12-17 20:58:39,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076bc575bc060ce15b177'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:41,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076bc575bc060ce15b177 2025-12-17 20:58:42,992 - lava_callback - INFO - Received callback for node 69407c09575bc060ce15bc88 INFO: 172.21.0.1:41724 - "POST /node/69407c09575bc060ce15bc88 HTTP/1.0" 202 Accepted 2025-12-17 20:58:43,000 - lava_callback - INFO - Starting async job submit for node 69407c09575bc060ce15bc88 2025-12-17 20:58:43,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:58:43,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c09575bc060ce15bc88 HTTP/1.1" 502 150 2025-12-17 20:58:43,050 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c09575bc060ce15bc88'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:43,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c09575bc060ce15bc88 2025-12-17 20:58:43,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c09575bc060ce15bc88 HTTP/1.1" 502 150 2025-12-17 20:58:43,051 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c09575bc060ce15bc88'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:45,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c09575bc060ce15bc88 2025-12-17 20:58:45,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c09575bc060ce15bc88 HTTP/1.1" 502 150 2025-12-17 20:58:45,053 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c09575bc060ce15bc88'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:46,058 - lava_callback - INFO - Received callback for node 69407674575bc060ce15b095 INFO: 172.21.0.1:41736 - "POST /node/69407674575bc060ce15b095 HTTP/1.0" 202 Accepted 2025-12-17 20:58:46,065 - lava_callback - INFO - Starting async job submit for node 69407674575bc060ce15b095 2025-12-17 20:58:46,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:58:46,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407674575bc060ce15b095 HTTP/1.1" 502 150 2025-12-17 20:58:46,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407674575bc060ce15b095'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:46,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407674575bc060ce15b095 2025-12-17 20:58:46,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407674575bc060ce15b095 HTTP/1.1" 502 150 2025-12-17 20:58:46,074 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407674575bc060ce15b095'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41752 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:58:46,721 - lava_callback - INFO - Received callback for node 6940767d575bc060ce15b0b6 INFO: 172.21.0.1:41766 - "POST /node/6940767d575bc060ce15b0b6 HTTP/1.0" 202 Accepted 2025-12-17 20:58:46,727 - lava_callback - INFO - Starting async job submit for node 6940767d575bc060ce15b0b6 2025-12-17 20:58:46,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:58:46,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940767d575bc060ce15b0b6 HTTP/1.1" 502 150 2025-12-17 20:58:46,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940767d575bc060ce15b0b6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:46,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940767d575bc060ce15b0b6 2025-12-17 20:58:46,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940767d575bc060ce15b0b6 HTTP/1.1" 502 150 2025-12-17 20:58:46,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940767d575bc060ce15b0b6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:47,395 - lava_callback - INFO - Received callback for node 6940767f575bc060ce15b0d1 INFO: 172.21.0.1:41782 - "POST /node/6940767f575bc060ce15b0d1 HTTP/1.0" 202 Accepted 2025-12-17 20:58:47,401 - lava_callback - INFO - Starting async job submit for node 6940767f575bc060ce15b0d1 2025-12-17 20:58:47,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:58:47,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940767f575bc060ce15b0d1 HTTP/1.1" 502 150 2025-12-17 20:58:47,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940767f575bc060ce15b0d1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:47,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940767f575bc060ce15b0d1 2025-12-17 20:58:47,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940767f575bc060ce15b0d1 HTTP/1.1" 502 150 2025-12-17 20:58:47,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940767f575bc060ce15b0d1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:48,069 - lava_callback - INFO - Received callback for node 69407680575bc060ce15b0d6 INFO: 172.21.0.1:41798 - "POST /node/69407680575bc060ce15b0d6 HTTP/1.0" 202 Accepted 2025-12-17 20:58:48,075 - lava_callback - INFO - Starting async job submit for node 69407680575bc060ce15b0d6 2025-12-17 20:58:48,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:58:48,076 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407674575bc060ce15b095 2025-12-17 20:58:48,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407674575bc060ce15b095 HTTP/1.1" 502 150 2025-12-17 20:58:48,077 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407674575bc060ce15b095'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:48,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407680575bc060ce15b0d6 HTTP/1.1" 502 150 2025-12-17 20:58:48,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407680575bc060ce15b0d6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:48,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407680575bc060ce15b0d6 2025-12-17 20:58:48,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407680575bc060ce15b0d6 HTTP/1.1" 502 150 2025-12-17 20:58:48,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407680575bc060ce15b0d6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:48,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940767d575bc060ce15b0b6 2025-12-17 20:58:48,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940767d575bc060ce15b0b6 HTTP/1.1" 502 150 2025-12-17 20:58:48,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940767d575bc060ce15b0b6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:49,053 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c09575bc060ce15bc88 2025-12-17 20:58:49,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c09575bc060ce15bc88 HTTP/1.1" 502 150 2025-12-17 20:58:49,054 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c09575bc060ce15bc88'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:49,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940767f575bc060ce15b0d1 2025-12-17 20:58:49,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940767f575bc060ce15b0d1 HTTP/1.1" 502 150 2025-12-17 20:58:49,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940767f575bc060ce15b0d1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:50,115 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407680575bc060ce15b0d6 2025-12-17 20:58:50,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407680575bc060ce15b0d6 HTTP/1.1" 502 150 2025-12-17 20:58:50,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407680575bc060ce15b0d6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:52,077 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407674575bc060ce15b095 2025-12-17 20:58:52,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940767d575bc060ce15b0b6 2025-12-17 20:58:52,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940767d575bc060ce15b0b6 HTTP/1.1" 502 150 2025-12-17 20:58:52,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940767d575bc060ce15b0b6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:53,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940767f575bc060ce15b0d1 2025-12-17 20:58:53,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940767f575bc060ce15b0d1 HTTP/1.1" 502 150 2025-12-17 20:58:53,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940767f575bc060ce15b0d1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:54,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407680575bc060ce15b0d6 2025-12-17 20:58:54,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407680575bc060ce15b0d6 HTTP/1.1" 502 150 2025-12-17 20:58:54,118 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407680575bc060ce15b0d6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44762 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:58:57,054 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c09575bc060ce15bc88 2025-12-17 20:58:57,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c09575bc060ce15bc88 HTTP/1.1" 502 150 2025-12-17 20:58:57,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c09575bc060ce15bc88'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:58:57,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bc575bc060ce15b177 HTTP/1.1" 502 150 2025-12-17 20:58:57,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076bc575bc060ce15b177'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:00,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940767d575bc060ce15b0b6 2025-12-17 20:59:00,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940767d575bc060ce15b0b6 HTTP/1.1" 502 150 2025-12-17 20:59:00,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940767d575bc060ce15b0b6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34306 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:59:01,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076bc575bc060ce15b177 2025-12-17 20:59:01,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bc575bc060ce15b177 HTTP/1.1" 502 150 2025-12-17 20:59:01,074 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076bc575bc060ce15b177'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:01,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940767f575bc060ce15b0d1 2025-12-17 20:59:01,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940767f575bc060ce15b0d1 HTTP/1.1" 502 150 2025-12-17 20:59:01,425 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940767f575bc060ce15b0d1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:02,118 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407680575bc060ce15b0d6 INFO: 172.21.0.1:34320 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:59:07,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407674575bc060ce15b095 HTTP/1.1" 502 150 2025-12-17 20:59:07,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407674575bc060ce15b095'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:09,074 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076bc575bc060ce15b177 2025-12-17 20:59:09,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bc575bc060ce15b177 HTTP/1.1" 502 150 2025-12-17 20:59:09,075 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076bc575bc060ce15b177'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:09,410 - lava_callback - INFO - Received callback for node 69407689575bc060ce15b0f1 INFO: 172.21.0.1:34324 - "POST /node/69407689575bc060ce15b0f1 HTTP/1.0" 202 Accepted 2025-12-17 20:59:09,416 - lava_callback - INFO - Starting async job submit for node 69407689575bc060ce15b0f1 2025-12-17 20:59:09,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:59:09,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407689575bc060ce15b0f1 HTTP/1.1" 502 150 2025-12-17 20:59:09,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407689575bc060ce15b0f1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:09,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407689575bc060ce15b0f1 2025-12-17 20:59:09,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407689575bc060ce15b0f1 HTTP/1.1" 502 150 2025-12-17 20:59:09,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407689575bc060ce15b0f1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:10,050 - lava_callback - INFO - Received callback for node 6940768b575bc060ce15b0f4 INFO: 172.21.0.1:51882 - "POST /node/6940768b575bc060ce15b0f4 HTTP/1.0" 202 Accepted 2025-12-17 20:59:10,056 - lava_callback - INFO - Starting async job submit for node 6940768b575bc060ce15b0f4 2025-12-17 20:59:10,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:59:10,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768b575bc060ce15b0f4 HTTP/1.1" 502 150 2025-12-17 20:59:10,064 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768b575bc060ce15b0f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:10,064 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768b575bc060ce15b0f4 2025-12-17 20:59:10,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768b575bc060ce15b0f4 HTTP/1.1" 502 150 2025-12-17 20:59:10,065 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768b575bc060ce15b0f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:10,674 - lava_callback - INFO - Received callback for node 69407698575bc060ce15b126 INFO: 172.21.0.1:51886 - "POST /node/69407698575bc060ce15b126 HTTP/1.0" 202 Accepted 2025-12-17 20:59:10,679 - lava_callback - INFO - Starting async job submit for node 69407698575bc060ce15b126 2025-12-17 20:59:10,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:59:10,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407698575bc060ce15b126 HTTP/1.1" 502 150 2025-12-17 20:59:10,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407698575bc060ce15b126'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:10,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407698575bc060ce15b126 2025-12-17 20:59:10,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407698575bc060ce15b126 HTTP/1.1" 502 150 2025-12-17 20:59:10,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407698575bc060ce15b126'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:11,311 - lava_callback - INFO - Received callback for node 69407699575bc060ce15b129 INFO: 172.21.0.1:51898 - "POST /node/69407699575bc060ce15b129 HTTP/1.0" 202 Accepted 2025-12-17 20:59:11,317 - lava_callback - INFO - Starting async job submit for node 69407699575bc060ce15b129 2025-12-17 20:59:11,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:59:11,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407699575bc060ce15b129 HTTP/1.1" 502 150 2025-12-17 20:59:11,325 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407699575bc060ce15b129'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:11,325 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407699575bc060ce15b129 2025-12-17 20:59:11,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407699575bc060ce15b129 HTTP/1.1" 502 150 2025-12-17 20:59:11,326 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407699575bc060ce15b129'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:11,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407689575bc060ce15b0f1 2025-12-17 20:59:11,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407689575bc060ce15b0f1 HTTP/1.1" 502 150 2025-12-17 20:59:11,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407689575bc060ce15b0f1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:12,065 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768b575bc060ce15b0f4 2025-12-17 20:59:12,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768b575bc060ce15b0f4 HTTP/1.1" 502 150 2025-12-17 20:59:12,067 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768b575bc060ce15b0f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:12,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407698575bc060ce15b126 2025-12-17 20:59:12,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407698575bc060ce15b126 HTTP/1.1" 502 150 2025-12-17 20:59:12,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407698575bc060ce15b126'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:13,056 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c09575bc060ce15bc88 2025-12-17 20:59:13,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c09575bc060ce15bc88 HTTP/1.1" 502 150 2025-12-17 20:59:13,058 - lava_callback - ERROR - Error processing callback for node 69407c09575bc060ce15bc88: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c09575bc060ce15bc88 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c09575bc060ce15bc88 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c09575bc060ce15bc88 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:59:13,326 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407699575bc060ce15b129 2025-12-17 20:59:13,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407699575bc060ce15b129 HTTP/1.1" 502 150 2025-12-17 20:59:13,327 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407699575bc060ce15b129'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51904 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 20:59:15,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407689575bc060ce15b0f1 2025-12-17 20:59:15,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407689575bc060ce15b0f1 HTTP/1.1" 502 150 2025-12-17 20:59:15,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407689575bc060ce15b0f1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:15,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407674575bc060ce15b095 2025-12-17 20:59:15,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407674575bc060ce15b095 HTTP/1.1" 502 150 2025-12-17 20:59:15,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407674575bc060ce15b095'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:16,067 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768b575bc060ce15b0f4 2025-12-17 20:59:16,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768b575bc060ce15b0f4 HTTP/1.1" 502 150 2025-12-17 20:59:16,068 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768b575bc060ce15b0f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51918 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:59:16,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407698575bc060ce15b126 2025-12-17 20:59:16,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407698575bc060ce15b126 HTTP/1.1" 502 150 2025-12-17 20:59:16,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407698575bc060ce15b126'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:16,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940767d575bc060ce15b0b6 2025-12-17 20:59:16,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940767d575bc060ce15b0b6 HTTP/1.1" 502 150 2025-12-17 20:59:16,762 - lava_callback - ERROR - Error processing callback for node 6940767d575bc060ce15b0b6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940767d575bc060ce15b0b6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940767d575bc060ce15b0b6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940767d575bc060ce15b0b6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:59:17,328 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407699575bc060ce15b129 2025-12-17 20:59:17,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940767f575bc060ce15b0d1 2025-12-17 20:59:17,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940767f575bc060ce15b0d1 HTTP/1.1" 502 150 2025-12-17 20:59:17,427 - lava_callback - ERROR - Error processing callback for node 6940767f575bc060ce15b0d1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940767f575bc060ce15b0d1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940767f575bc060ce15b0d1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940767f575bc060ce15b0d1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:59:17,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407680575bc060ce15b0d6 HTTP/1.1" 502 150 2025-12-17 20:59:17,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407680575bc060ce15b0d6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:23,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407689575bc060ce15b0f1 2025-12-17 20:59:23,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407689575bc060ce15b0f1 HTTP/1.1" 502 150 2025-12-17 20:59:23,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407689575bc060ce15b0f1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:24,069 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768b575bc060ce15b0f4 2025-12-17 20:59:24,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768b575bc060ce15b0f4 HTTP/1.1" 502 150 2025-12-17 20:59:24,070 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940768b575bc060ce15b0f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:24,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407698575bc060ce15b126 2025-12-17 20:59:24,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407698575bc060ce15b126 HTTP/1.1" 502 150 2025-12-17 20:59:24,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407698575bc060ce15b126'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:25,075 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076bc575bc060ce15b177 2025-12-17 20:59:25,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bc575bc060ce15b177 HTTP/1.1" 502 150 2025-12-17 20:59:25,081 - lava_callback - ERROR - Error processing callback for node 694076bc575bc060ce15b177: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076bc575bc060ce15b177 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076bc575bc060ce15b177 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076bc575bc060ce15b177 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:58216 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:59:30,923 - lava_callback - INFO - Received callback for node 694078d9575bc060ce15b582 INFO: 172.21.0.1:60518 - "POST /node/694078d9575bc060ce15b582 HTTP/1.0" 202 Accepted 2025-12-17 20:59:30,929 - lava_callback - INFO - Starting async job submit for node 694078d9575bc060ce15b582 2025-12-17 20:59:30,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 20:59:30,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078d9575bc060ce15b582 HTTP/1.1" 502 150 2025-12-17 20:59:30,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078d9575bc060ce15b582'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:30,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078d9575bc060ce15b582 2025-12-17 20:59:30,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078d9575bc060ce15b582 HTTP/1.1" 502 150 2025-12-17 20:59:30,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078d9575bc060ce15b582'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:31,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407674575bc060ce15b095 2025-12-17 20:59:31,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407674575bc060ce15b095 HTTP/1.1" 502 150 2025-12-17 20:59:31,572 - lava_callback - ERROR - Error processing callback for node 69407674575bc060ce15b095: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407674575bc060ce15b095 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407674575bc060ce15b095 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407674575bc060ce15b095 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:59:32,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407699575bc060ce15b129 HTTP/1.1" 502 150 2025-12-17 20:59:32,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407699575bc060ce15b129'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:32,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078d9575bc060ce15b582 2025-12-17 20:59:32,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078d9575bc060ce15b582 HTTP/1.1" 502 150 2025-12-17 20:59:32,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078d9575bc060ce15b582'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:33,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407680575bc060ce15b0d6 2025-12-17 20:59:33,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407680575bc060ce15b0d6 HTTP/1.1" 502 150 2025-12-17 20:59:33,554 - lava_callback - ERROR - Error processing callback for node 69407680575bc060ce15b0d6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407680575bc060ce15b0d6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407680575bc060ce15b0d6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407680575bc060ce15b0d6 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:60530 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:59:36,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078d9575bc060ce15b582 2025-12-17 20:59:36,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078d9575bc060ce15b582 HTTP/1.1" 502 150 2025-12-17 20:59:36,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078d9575bc060ce15b582'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:39,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407689575bc060ce15b0f1 2025-12-17 20:59:39,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407689575bc060ce15b0f1 HTTP/1.1" 502 150 2025-12-17 20:59:39,452 - lava_callback - ERROR - Error processing callback for node 69407689575bc060ce15b0f1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407689575bc060ce15b0f1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407689575bc060ce15b0f1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407689575bc060ce15b0f1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:59:40,070 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940768b575bc060ce15b0f4 2025-12-17 20:59:40,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940768b575bc060ce15b0f4 HTTP/1.1" 502 150 2025-12-17 20:59:40,072 - lava_callback - ERROR - Error processing callback for node 6940768b575bc060ce15b0f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940768b575bc060ce15b0f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940768b575bc060ce15b0f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940768b575bc060ce15b0f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:59:40,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407699575bc060ce15b129 2025-12-17 20:59:40,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407699575bc060ce15b129 HTTP/1.1" 502 150 2025-12-17 20:59:40,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407699575bc060ce15b129'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 20:59:40,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407698575bc060ce15b126 2025-12-17 20:59:40,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407698575bc060ce15b126 HTTP/1.1" 502 150 2025-12-17 20:59:40,693 - lava_callback - ERROR - Error processing callback for node 69407698575bc060ce15b126: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407698575bc060ce15b126 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407698575bc060ce15b126 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407698575bc060ce15b126 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 20:59:44,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078d9575bc060ce15b582 2025-12-17 20:59:44,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078d9575bc060ce15b582 HTTP/1.1" 502 150 2025-12-17 20:59:44,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078d9575bc060ce15b582'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49368 - "GET / HTTP/1.0" 200 OK 2025-12-17 20:59:56,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407699575bc060ce15b129 2025-12-17 21:00:00,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078d9575bc060ce15b582 2025-12-17 21:00:00,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078d9575bc060ce15b582 HTTP/1.1" 502 150 2025-12-17 21:00:00,965 - lava_callback - ERROR - Error processing callback for node 694078d9575bc060ce15b582: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078d9575bc060ce15b582 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078d9575bc060ce15b582 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078d9575bc060ce15b582 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:45090 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:00:02,855 - lava_callback - INFO - Received callback for node 694076c0575bc060ce15b19f INFO: 172.21.0.1:45106 - "POST /node/694076c0575bc060ce15b19f HTTP/1.0" 202 Accepted 2025-12-17 21:00:02,863 - lava_callback - INFO - Starting async job submit for node 694076c0575bc060ce15b19f 2025-12-17 21:00:02,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:02,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c0575bc060ce15b19f HTTP/1.1" 502 150 2025-12-17 21:00:02,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c0575bc060ce15b19f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:02,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c0575bc060ce15b19f 2025-12-17 21:00:02,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c0575bc060ce15b19f HTTP/1.1" 502 150 2025-12-17 21:00:02,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c0575bc060ce15b19f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:03,661 - lava_callback - INFO - Received callback for node 694076c1575bc060ce15b1a2 INFO: 172.21.0.1:45122 - "POST /node/694076c1575bc060ce15b1a2 HTTP/1.0" 202 Accepted 2025-12-17 21:00:03,667 - lava_callback - INFO - Starting async job submit for node 694076c1575bc060ce15b1a2 2025-12-17 21:00:03,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:03,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c1575bc060ce15b1a2 HTTP/1.1" 502 150 2025-12-17 21:00:03,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c1575bc060ce15b1a2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:03,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c1575bc060ce15b1a2 2025-12-17 21:00:03,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c1575bc060ce15b1a2 HTTP/1.1" 502 150 2025-12-17 21:00:03,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c1575bc060ce15b1a2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:04,485 - lava_callback - INFO - Received callback for node 694076c3575bc060ce15b1a5 INFO: 172.21.0.1:45138 - "POST /node/694076c3575bc060ce15b1a5 HTTP/1.0" 202 Accepted 2025-12-17 21:00:04,495 - lava_callback - INFO - Starting async job submit for node 694076c3575bc060ce15b1a5 2025-12-17 21:00:04,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:04,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c3575bc060ce15b1a5 HTTP/1.1" 502 150 2025-12-17 21:00:04,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c3575bc060ce15b1a5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:04,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c3575bc060ce15b1a5 2025-12-17 21:00:04,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c3575bc060ce15b1a5 HTTP/1.1" 502 150 2025-12-17 21:00:04,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c3575bc060ce15b1a5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:04,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c0575bc060ce15b19f 2025-12-17 21:00:04,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c0575bc060ce15b19f HTTP/1.1" 502 150 2025-12-17 21:00:04,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c0575bc060ce15b19f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:05,269 - lava_callback - INFO - Received callback for node 694076c4575bc060ce15b1a8 INFO: 172.21.0.1:45146 - "POST /node/694076c4575bc060ce15b1a8 HTTP/1.0" 202 Accepted 2025-12-17 21:00:05,275 - lava_callback - INFO - Starting async job submit for node 694076c4575bc060ce15b1a8 2025-12-17 21:00:05,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:05,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c4575bc060ce15b1a8 HTTP/1.1" 502 150 2025-12-17 21:00:05,292 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c4575bc060ce15b1a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:05,293 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c4575bc060ce15b1a8 2025-12-17 21:00:05,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c4575bc060ce15b1a8 HTTP/1.1" 502 150 2025-12-17 21:00:05,293 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c4575bc060ce15b1a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:05,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c1575bc060ce15b1a2 2025-12-17 21:00:05,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c1575bc060ce15b1a2 HTTP/1.1" 502 150 2025-12-17 21:00:05,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c1575bc060ce15b1a2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:06,063 - lava_callback - INFO - Received callback for node 694076c5575bc060ce15b1ab INFO: 172.21.0.1:45150 - "POST /node/694076c5575bc060ce15b1ab HTTP/1.0" 202 Accepted 2025-12-17 21:00:06,073 - lava_callback - INFO - Starting async job submit for node 694076c5575bc060ce15b1ab 2025-12-17 21:00:06,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:06,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c5575bc060ce15b1ab HTTP/1.1" 502 150 2025-12-17 21:00:06,085 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c5575bc060ce15b1ab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:06,085 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c5575bc060ce15b1ab 2025-12-17 21:00:06,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c5575bc060ce15b1ab HTTP/1.1" 502 150 2025-12-17 21:00:06,086 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c5575bc060ce15b1ab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45164 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:00:06,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c3575bc060ce15b1a5 2025-12-17 21:00:06,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c3575bc060ce15b1a5 HTTP/1.1" 502 150 2025-12-17 21:00:06,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c3575bc060ce15b1a5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:06,806 - lava_callback - INFO - Received callback for node 694076c6575bc060ce15b1b2 INFO: 172.21.0.1:45180 - "POST /node/694076c6575bc060ce15b1b2 HTTP/1.0" 202 Accepted 2025-12-17 21:00:06,815 - lava_callback - INFO - Starting async job submit for node 694076c6575bc060ce15b1b2 2025-12-17 21:00:06,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:06,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c6575bc060ce15b1b2 HTTP/1.1" 502 150 2025-12-17 21:00:06,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c6575bc060ce15b1b2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:06,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c6575bc060ce15b1b2 2025-12-17 21:00:06,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c6575bc060ce15b1b2 HTTP/1.1" 502 150 2025-12-17 21:00:06,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c6575bc060ce15b1b2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:07,294 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c4575bc060ce15b1a8 2025-12-17 21:00:07,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c4575bc060ce15b1a8 HTTP/1.1" 502 150 2025-12-17 21:00:07,295 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c4575bc060ce15b1a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:07,561 - lava_callback - INFO - Received callback for node 694076c8575bc060ce15b1b5 INFO: 172.21.0.1:45190 - "POST /node/694076c8575bc060ce15b1b5 HTTP/1.0" 202 Accepted 2025-12-17 21:00:07,567 - lava_callback - INFO - Starting async job submit for node 694076c8575bc060ce15b1b5 2025-12-17 21:00:07,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:07,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c8575bc060ce15b1b5 HTTP/1.1" 502 150 2025-12-17 21:00:07,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c8575bc060ce15b1b5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:07,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c8575bc060ce15b1b5 2025-12-17 21:00:07,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c8575bc060ce15b1b5 HTTP/1.1" 502 150 2025-12-17 21:00:07,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c8575bc060ce15b1b5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:08,086 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c5575bc060ce15b1ab 2025-12-17 21:00:08,361 - lava_callback - INFO - Received callback for node 694076c9575bc060ce15b1b8 INFO: 172.21.0.1:45200 - "POST /node/694076c9575bc060ce15b1b8 HTTP/1.0" 202 Accepted 2025-12-17 21:00:08,367 - lava_callback - INFO - Starting async job submit for node 694076c9575bc060ce15b1b8 2025-12-17 21:00:08,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:08,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c9575bc060ce15b1b8 HTTP/1.1" 502 150 2025-12-17 21:00:08,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c9575bc060ce15b1b8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:08,375 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c9575bc060ce15b1b8 2025-12-17 21:00:08,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c9575bc060ce15b1b8 HTTP/1.1" 502 150 2025-12-17 21:00:08,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c9575bc060ce15b1b8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:08,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c6575bc060ce15b1b2 2025-12-17 21:00:08,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c6575bc060ce15b1b2 HTTP/1.1" 502 150 2025-12-17 21:00:08,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c6575bc060ce15b1b2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:08,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c0575bc060ce15b19f 2025-12-17 21:00:08,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c0575bc060ce15b19f HTTP/1.1" 502 150 2025-12-17 21:00:08,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c0575bc060ce15b19f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:09,317 - lava_callback - INFO - Received callback for node 694076ca575bc060ce15b1bb INFO: 172.21.0.1:45206 - "POST /node/694076ca575bc060ce15b1bb HTTP/1.0" 202 Accepted 2025-12-17 21:00:09,323 - lava_callback - INFO - Starting async job submit for node 694076ca575bc060ce15b1bb 2025-12-17 21:00:09,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:09,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ca575bc060ce15b1bb HTTP/1.1" 502 150 2025-12-17 21:00:09,330 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ca575bc060ce15b1bb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:09,330 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ca575bc060ce15b1bb 2025-12-17 21:00:09,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ca575bc060ce15b1bb HTTP/1.1" 502 150 2025-12-17 21:00:09,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ca575bc060ce15b1bb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:09,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c8575bc060ce15b1b5 2025-12-17 21:00:09,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c8575bc060ce15b1b5 HTTP/1.1" 502 150 2025-12-17 21:00:09,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c8575bc060ce15b1b5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:09,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c1575bc060ce15b1a2 2025-12-17 21:00:09,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c1575bc060ce15b1a2 HTTP/1.1" 502 150 2025-12-17 21:00:09,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c1575bc060ce15b1a2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:10,083 - lava_callback - INFO - Received callback for node 694076cc575bc060ce15b1be INFO: 172.21.0.1:50816 - "POST /node/694076cc575bc060ce15b1be HTTP/1.0" 202 Accepted 2025-12-17 21:00:10,089 - lava_callback - INFO - Starting async job submit for node 694076cc575bc060ce15b1be 2025-12-17 21:00:10,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:10,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cc575bc060ce15b1be HTTP/1.1" 502 150 2025-12-17 21:00:10,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076cc575bc060ce15b1be'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:10,095 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076cc575bc060ce15b1be 2025-12-17 21:00:10,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cc575bc060ce15b1be HTTP/1.1" 502 150 2025-12-17 21:00:10,096 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076cc575bc060ce15b1be'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:10,376 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c9575bc060ce15b1b8 2025-12-17 21:00:10,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c9575bc060ce15b1b8 HTTP/1.1" 502 150 2025-12-17 21:00:10,377 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c9575bc060ce15b1b8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:10,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c3575bc060ce15b1a5 2025-12-17 21:00:10,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c3575bc060ce15b1a5 HTTP/1.1" 502 150 2025-12-17 21:00:10,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c3575bc060ce15b1a5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:10,913 - lava_callback - INFO - Received callback for node 694076cd575bc060ce15b1c1 INFO: 172.21.0.1:50828 - "POST /node/694076cd575bc060ce15b1c1 HTTP/1.0" 202 Accepted 2025-12-17 21:00:10,919 - lava_callback - INFO - Starting async job submit for node 694076cd575bc060ce15b1c1 2025-12-17 21:00:10,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:10,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cd575bc060ce15b1c1 HTTP/1.1" 502 150 2025-12-17 21:00:10,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076cd575bc060ce15b1c1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:10,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076cd575bc060ce15b1c1 2025-12-17 21:00:10,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cd575bc060ce15b1c1 HTTP/1.1" 502 150 2025-12-17 21:00:10,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076cd575bc060ce15b1c1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:11,295 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c4575bc060ce15b1a8 2025-12-17 21:00:11,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c4575bc060ce15b1a8 HTTP/1.1" 502 150 2025-12-17 21:00:11,297 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c4575bc060ce15b1a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:11,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ca575bc060ce15b1bb 2025-12-17 21:00:11,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ca575bc060ce15b1bb HTTP/1.1" 502 150 2025-12-17 21:00:11,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ca575bc060ce15b1bb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:11,653 - lava_callback - INFO - Received callback for node 694076ce575bc060ce15b1c4 INFO: 172.21.0.1:50844 - "POST /node/694076ce575bc060ce15b1c4 HTTP/1.0" 202 Accepted 2025-12-17 21:00:11,659 - lava_callback - INFO - Starting async job submit for node 694076ce575bc060ce15b1c4 2025-12-17 21:00:11,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:11,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ce575bc060ce15b1c4 HTTP/1.1" 502 150 2025-12-17 21:00:11,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ce575bc060ce15b1c4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:11,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ce575bc060ce15b1c4 2025-12-17 21:00:11,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ce575bc060ce15b1c4 HTTP/1.1" 502 150 2025-12-17 21:00:11,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ce575bc060ce15b1c4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:12,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407699575bc060ce15b129 HTTP/1.1" 502 150 2025-12-17 21:00:12,080 - lava_callback - ERROR - Error processing callback for node 69407699575bc060ce15b129: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407699575bc060ce15b129 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407699575bc060ce15b129 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407699575bc060ce15b129 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:00:12,096 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076cc575bc060ce15b1be 2025-12-17 21:00:12,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cc575bc060ce15b1be HTTP/1.1" 502 150 2025-12-17 21:00:12,097 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076cc575bc060ce15b1be'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:12,477 - lava_callback - INFO - Received callback for node 694076cf575bc060ce15b1c9 INFO: 172.21.0.1:50848 - "POST /node/694076cf575bc060ce15b1c9 HTTP/1.0" 202 Accepted 2025-12-17 21:00:12,482 - lava_callback - INFO - Starting async job submit for node 694076cf575bc060ce15b1c9 2025-12-17 21:00:12,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:12,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cf575bc060ce15b1c9 HTTP/1.1" 502 150 2025-12-17 21:00:12,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076cf575bc060ce15b1c9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:12,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076cf575bc060ce15b1c9 2025-12-17 21:00:12,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cf575bc060ce15b1c9 HTTP/1.1" 502 150 2025-12-17 21:00:12,495 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076cf575bc060ce15b1c9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:12,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c6575bc060ce15b1b2 2025-12-17 21:00:12,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c6575bc060ce15b1b2 HTTP/1.1" 502 150 2025-12-17 21:00:12,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c6575bc060ce15b1b2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:12,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076cd575bc060ce15b1c1 2025-12-17 21:00:12,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cd575bc060ce15b1c1 HTTP/1.1" 502 150 2025-12-17 21:00:12,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076cd575bc060ce15b1c1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:13,577 - lava_callback - INFO - Received callback for node 694076d1575bc060ce15b1ce INFO: 172.21.0.1:50850 - "POST /node/694076d1575bc060ce15b1ce HTTP/1.0" 202 Accepted 2025-12-17 21:00:13,583 - lava_callback - INFO - Starting async job submit for node 694076d1575bc060ce15b1ce 2025-12-17 21:00:13,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:13,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d1575bc060ce15b1ce HTTP/1.1" 502 150 2025-12-17 21:00:13,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d1575bc060ce15b1ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:13,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d1575bc060ce15b1ce 2025-12-17 21:00:13,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d1575bc060ce15b1ce HTTP/1.1" 502 150 2025-12-17 21:00:13,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d1575bc060ce15b1ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:13,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c8575bc060ce15b1b5 2025-12-17 21:00:13,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c8575bc060ce15b1b5 HTTP/1.1" 502 150 2025-12-17 21:00:13,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c8575bc060ce15b1b5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:13,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ce575bc060ce15b1c4 2025-12-17 21:00:13,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ce575bc060ce15b1c4 HTTP/1.1" 502 150 2025-12-17 21:00:13,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ce575bc060ce15b1c4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:14,326 - lava_callback - INFO - Received callback for node 694076d2575bc060ce15b1d5 INFO: 172.21.0.1:50858 - "POST /node/694076d2575bc060ce15b1d5 HTTP/1.0" 202 Accepted 2025-12-17 21:00:14,332 - lava_callback - INFO - Starting async job submit for node 694076d2575bc060ce15b1d5 2025-12-17 21:00:14,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:14,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d2575bc060ce15b1d5 HTTP/1.1" 502 150 2025-12-17 21:00:14,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d2575bc060ce15b1d5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:14,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d2575bc060ce15b1d5 2025-12-17 21:00:14,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d2575bc060ce15b1d5 HTTP/1.1" 502 150 2025-12-17 21:00:14,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d2575bc060ce15b1d5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:14,377 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c9575bc060ce15b1b8 2025-12-17 21:00:14,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c9575bc060ce15b1b8 HTTP/1.1" 502 150 2025-12-17 21:00:14,378 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c9575bc060ce15b1b8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:14,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076cf575bc060ce15b1c9 2025-12-17 21:00:14,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cf575bc060ce15b1c9 HTTP/1.1" 502 150 2025-12-17 21:00:14,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076cf575bc060ce15b1c9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:15,085 - lava_callback - INFO - Received callback for node 694076d3575bc060ce15b1db INFO: 172.21.0.1:50864 - "POST /node/694076d3575bc060ce15b1db HTTP/1.0" 202 Accepted 2025-12-17 21:00:15,091 - lava_callback - INFO - Starting async job submit for node 694076d3575bc060ce15b1db 2025-12-17 21:00:15,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:15,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d3575bc060ce15b1db HTTP/1.1" 502 150 2025-12-17 21:00:15,097 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d3575bc060ce15b1db'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:15,098 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d3575bc060ce15b1db 2025-12-17 21:00:15,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d3575bc060ce15b1db HTTP/1.1" 502 150 2025-12-17 21:00:15,098 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d3575bc060ce15b1db'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50878 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:00:15,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ca575bc060ce15b1bb 2025-12-17 21:00:15,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ca575bc060ce15b1bb HTTP/1.1" 502 150 2025-12-17 21:00:15,333 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ca575bc060ce15b1bb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:15,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d1575bc060ce15b1ce 2025-12-17 21:00:15,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d1575bc060ce15b1ce HTTP/1.1" 502 150 2025-12-17 21:00:15,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d1575bc060ce15b1ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:15,894 - lava_callback - INFO - Received callback for node 694076d4575bc060ce15b1e1 INFO: 172.21.0.1:50882 - "POST /node/694076d4575bc060ce15b1e1 HTTP/1.0" 202 Accepted 2025-12-17 21:00:16,097 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076cc575bc060ce15b1be 2025-12-17 21:00:16,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cc575bc060ce15b1be HTTP/1.1" 502 150 2025-12-17 21:00:16,098 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076cc575bc060ce15b1be'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:16,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d2575bc060ce15b1d5 2025-12-17 21:00:16,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d2575bc060ce15b1d5 HTTP/1.1" 502 150 2025-12-17 21:00:16,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d2575bc060ce15b1d5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50894 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:00:16,646 - lava_callback - INFO - Received callback for node 694076d6575bc060ce15b1e6 INFO: 172.21.0.1:50904 - "POST /node/694076d6575bc060ce15b1e6 HTTP/1.0" 202 Accepted 2025-12-17 21:00:16,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c0575bc060ce15b19f 2025-12-17 21:00:16,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c0575bc060ce15b19f HTTP/1.1" 502 150 2025-12-17 21:00:16,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c0575bc060ce15b19f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:16,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076cd575bc060ce15b1c1 2025-12-17 21:00:16,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cd575bc060ce15b1c1 HTTP/1.1" 502 150 2025-12-17 21:00:16,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076cd575bc060ce15b1c1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:17,099 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d3575bc060ce15b1db 2025-12-17 21:00:17,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d3575bc060ce15b1db HTTP/1.1" 502 150 2025-12-17 21:00:17,100 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d3575bc060ce15b1db'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:17,404 - lava_callback - INFO - Received callback for node 694076d7575bc060ce15b1e9 INFO: 172.21.0.1:50908 - "POST /node/694076d7575bc060ce15b1e9 HTTP/1.0" 202 Accepted 2025-12-17 21:00:17,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ce575bc060ce15b1c4 2025-12-17 21:00:17,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ce575bc060ce15b1c4 HTTP/1.1" 502 150 2025-12-17 21:00:17,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ce575bc060ce15b1c4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:17,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c1575bc060ce15b1a2 2025-12-17 21:00:17,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c1575bc060ce15b1a2 HTTP/1.1" 502 150 2025-12-17 21:00:17,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c1575bc060ce15b1a2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:18,173 - lava_callback - INFO - Received callback for node 694076d8575bc060ce15b1ec INFO: 172.21.0.1:50918 - "POST /node/694076d8575bc060ce15b1ec HTTP/1.0" 202 Accepted 2025-12-17 21:00:18,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076cf575bc060ce15b1c9 2025-12-17 21:00:18,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cf575bc060ce15b1c9 HTTP/1.1" 502 150 2025-12-17 21:00:18,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076cf575bc060ce15b1c9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:18,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c3575bc060ce15b1a5 2025-12-17 21:00:18,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c3575bc060ce15b1a5 HTTP/1.1" 502 150 2025-12-17 21:00:18,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c3575bc060ce15b1a5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:18,940 - lava_callback - INFO - Received callback for node 694076da575bc060ce15b1f3 INFO: 172.21.0.1:50920 - "POST /node/694076da575bc060ce15b1f3 HTTP/1.0" 202 Accepted 2025-12-17 21:00:19,297 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c4575bc060ce15b1a8 2025-12-17 21:00:19,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c4575bc060ce15b1a8 HTTP/1.1" 502 150 2025-12-17 21:00:19,299 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c4575bc060ce15b1a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:19,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d1575bc060ce15b1ce 2025-12-17 21:00:19,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d1575bc060ce15b1ce HTTP/1.1" 502 150 2025-12-17 21:00:19,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d1575bc060ce15b1ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:19,728 - lava_callback - INFO - Received callback for node 694076db575bc060ce15b1f6 INFO: 172.21.0.1:43138 - "POST /node/694076db575bc060ce15b1f6 HTTP/1.0" 202 Accepted 2025-12-17 21:00:20,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d2575bc060ce15b1d5 2025-12-17 21:00:20,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d2575bc060ce15b1d5 HTTP/1.1" 502 150 2025-12-17 21:00:20,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d2575bc060ce15b1d5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:20,511 - lava_callback - INFO - Received callback for node 694076dd575bc060ce15b1fc INFO: 172.21.0.1:43142 - "POST /node/694076dd575bc060ce15b1fc HTTP/1.0" 202 Accepted 2025-12-17 21:00:20,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c6575bc060ce15b1b2 2025-12-17 21:00:20,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c6575bc060ce15b1b2 HTTP/1.1" 502 150 2025-12-17 21:00:20,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c6575bc060ce15b1b2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:21,100 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d3575bc060ce15b1db 2025-12-17 21:00:21,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d3575bc060ce15b1db HTTP/1.1" 502 150 2025-12-17 21:00:21,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d3575bc060ce15b1db'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:21,288 - lava_callback - INFO - Received callback for node 694076df575bc060ce15b1ff INFO: 172.21.0.1:43158 - "POST /node/694076df575bc060ce15b1ff HTTP/1.0" 202 Accepted 2025-12-17 21:00:21,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c8575bc060ce15b1b5 2025-12-17 21:00:21,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c8575bc060ce15b1b5 HTTP/1.1" 502 150 2025-12-17 21:00:21,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c8575bc060ce15b1b5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:22,041 - lava_callback - INFO - Received callback for node 694076e3575bc060ce15b202 INFO: 172.21.0.1:43160 - "POST /node/694076e3575bc060ce15b202 HTTP/1.0" 202 Accepted 2025-12-17 21:00:22,378 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c9575bc060ce15b1b8 2025-12-17 21:00:22,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c9575bc060ce15b1b8 HTTP/1.1" 502 150 2025-12-17 21:00:22,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c9575bc060ce15b1b8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:22,804 - lava_callback - INFO - Received callback for node 694076e4575bc060ce15b207 INFO: 172.21.0.1:43174 - "POST /node/694076e4575bc060ce15b207 HTTP/1.0" 202 Accepted 2025-12-17 21:00:23,334 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ca575bc060ce15b1bb 2025-12-17 21:00:23,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c5575bc060ce15b1ab HTTP/1.1" 502 150 2025-12-17 21:00:23,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c5575bc060ce15b1ab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:23,715 - lava_callback - INFO - Received callback for node 694076e5575bc060ce15b20e INFO: 172.21.0.1:43178 - "POST /node/694076e5575bc060ce15b20e HTTP/1.0" 202 Accepted 2025-12-17 21:00:24,099 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076cc575bc060ce15b1be 2025-12-17 21:00:24,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cc575bc060ce15b1be HTTP/1.1" 502 150 2025-12-17 21:00:24,100 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076cc575bc060ce15b1be'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:24,537 - lava_callback - INFO - Received callback for node 694076e7575bc060ce15b215 INFO: 172.21.0.1:43190 - "POST /node/694076e7575bc060ce15b215 HTTP/1.0" 202 Accepted 2025-12-17 21:00:24,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076cd575bc060ce15b1c1 2025-12-17 21:00:24,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cd575bc060ce15b1c1 HTTP/1.1" 502 150 2025-12-17 21:00:24,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076cd575bc060ce15b1c1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:25,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ce575bc060ce15b1c4 2025-12-17 21:00:25,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ce575bc060ce15b1c4 HTTP/1.1" 502 150 2025-12-17 21:00:25,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ce575bc060ce15b1c4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:26,242 - lava_callback - INFO - Received callback for node 694076e8575bc060ce15b21a INFO: 172.21.0.1:43206 - "POST /node/694076e8575bc060ce15b21a HTTP/1.0" 202 Accepted 2025-12-17 21:00:26,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076cf575bc060ce15b1c9 2025-12-17 21:00:26,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cf575bc060ce15b1c9 HTTP/1.1" 502 150 2025-12-17 21:00:26,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076cf575bc060ce15b1c9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43218 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:00:27,014 - lava_callback - INFO - Received callback for node 694076e9575bc060ce15b221 INFO: 172.21.0.1:43228 - "POST /node/694076e9575bc060ce15b221 HTTP/1.0" 202 Accepted 2025-12-17 21:00:27,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c5575bc060ce15b1ab 2025-12-17 21:00:27,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c5575bc060ce15b1ab HTTP/1.1" 502 150 2025-12-17 21:00:27,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c5575bc060ce15b1ab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:27,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d1575bc060ce15b1ce 2025-12-17 21:00:27,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d1575bc060ce15b1ce HTTP/1.1" 502 150 2025-12-17 21:00:27,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d1575bc060ce15b1ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:28,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d2575bc060ce15b1d5 2025-12-17 21:00:28,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d2575bc060ce15b1d5 HTTP/1.1" 502 150 2025-12-17 21:00:28,350 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d2575bc060ce15b1d5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:29,102 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d3575bc060ce15b1db 2025-12-17 21:00:29,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d3575bc060ce15b1db HTTP/1.1" 502 150 2025-12-17 21:00:29,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d3575bc060ce15b1db'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:32,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c0575bc060ce15b19f 2025-12-17 21:00:32,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c0575bc060ce15b19f HTTP/1.1" 502 150 2025-12-17 21:00:32,879 - lava_callback - ERROR - Error processing callback for node 694076c0575bc060ce15b19f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c0575bc060ce15b19f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c0575bc060ce15b19f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c0575bc060ce15b19f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:00:32,881 - lava_callback - INFO - Starting async job submit for node 694076d4575bc060ce15b1e1 2025-12-17 21:00:32,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:32,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d4575bc060ce15b1e1 HTTP/1.1" 502 150 2025-12-17 21:00:32,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d4575bc060ce15b1e1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:32,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d4575bc060ce15b1e1 2025-12-17 21:00:32,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d4575bc060ce15b1e1 HTTP/1.1" 502 150 2025-12-17 21:00:32,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d4575bc060ce15b1e1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:33,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c1575bc060ce15b1a2 2025-12-17 21:00:33,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c1575bc060ce15b1a2 HTTP/1.1" 502 150 2025-12-17 21:00:33,681 - lava_callback - ERROR - Error processing callback for node 694076c1575bc060ce15b1a2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c1575bc060ce15b1a2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c1575bc060ce15b1a2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c1575bc060ce15b1a2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:00:33,682 - lava_callback - INFO - Starting async job submit for node 694076d6575bc060ce15b1e6 2025-12-17 21:00:33,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:33,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d6575bc060ce15b1e6 HTTP/1.1" 502 150 2025-12-17 21:00:33,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d6575bc060ce15b1e6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:33,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d6575bc060ce15b1e6 2025-12-17 21:00:33,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d6575bc060ce15b1e6 HTTP/1.1" 502 150 2025-12-17 21:00:33,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d6575bc060ce15b1e6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:34,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c3575bc060ce15b1a5 2025-12-17 21:00:34,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d4575bc060ce15b1e1 2025-12-17 21:00:34,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d4575bc060ce15b1e1 HTTP/1.1" 502 150 2025-12-17 21:00:34,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d4575bc060ce15b1e1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:35,299 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c4575bc060ce15b1a8 2025-12-17 21:00:35,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c4575bc060ce15b1a8 HTTP/1.1" 502 150 2025-12-17 21:00:35,301 - lava_callback - ERROR - Error processing callback for node 694076c4575bc060ce15b1a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c4575bc060ce15b1a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c4575bc060ce15b1a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c4575bc060ce15b1a8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:00:35,302 - lava_callback - INFO - Starting async job submit for node 694076d7575bc060ce15b1e9 2025-12-17 21:00:35,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:35,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d7575bc060ce15b1e9 HTTP/1.1" 502 150 2025-12-17 21:00:35,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d7575bc060ce15b1e9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:35,317 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d7575bc060ce15b1e9 2025-12-17 21:00:35,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d7575bc060ce15b1e9 HTTP/1.1" 502 150 2025-12-17 21:00:35,318 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d7575bc060ce15b1e9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:35,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c5575bc060ce15b1ab 2025-12-17 21:00:35,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c5575bc060ce15b1ab HTTP/1.1" 502 150 2025-12-17 21:00:35,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076c5575bc060ce15b1ab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:35,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d6575bc060ce15b1e6 2025-12-17 21:00:35,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d6575bc060ce15b1e6 HTTP/1.1" 502 150 2025-12-17 21:00:35,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d6575bc060ce15b1e6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35324 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:00:36,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c6575bc060ce15b1b2 2025-12-17 21:00:36,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c6575bc060ce15b1b2 HTTP/1.1" 502 150 2025-12-17 21:00:36,839 - lava_callback - ERROR - Error processing callback for node 694076c6575bc060ce15b1b2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c6575bc060ce15b1b2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c6575bc060ce15b1b2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c6575bc060ce15b1b2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:00:36,842 - lava_callback - INFO - Starting async job submit for node 694076d8575bc060ce15b1ec 2025-12-17 21:00:36,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:36,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d8575bc060ce15b1ec HTTP/1.1" 502 150 2025-12-17 21:00:36,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d8575bc060ce15b1ec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:36,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d8575bc060ce15b1ec 2025-12-17 21:00:36,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d8575bc060ce15b1ec HTTP/1.1" 502 150 2025-12-17 21:00:36,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d8575bc060ce15b1ec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:37,318 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d7575bc060ce15b1e9 2025-12-17 21:00:37,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d7575bc060ce15b1e9 HTTP/1.1" 502 150 2025-12-17 21:00:37,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d7575bc060ce15b1e9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:37,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c8575bc060ce15b1b5 2025-12-17 21:00:37,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c8575bc060ce15b1b5 HTTP/1.1" 502 150 2025-12-17 21:00:37,604 - lava_callback - ERROR - Error processing callback for node 694076c8575bc060ce15b1b5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c8575bc060ce15b1b5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c8575bc060ce15b1b5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c8575bc060ce15b1b5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:00:37,609 - lava_callback - INFO - Starting async job submit for node 694076da575bc060ce15b1f3 2025-12-17 21:00:37,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:37,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076da575bc060ce15b1f3 HTTP/1.1" 502 150 2025-12-17 21:00:37,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076da575bc060ce15b1f3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:37,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076da575bc060ce15b1f3 2025-12-17 21:00:37,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076da575bc060ce15b1f3 HTTP/1.1" 502 150 2025-12-17 21:00:37,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076da575bc060ce15b1f3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:38,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c9575bc060ce15b1b8 2025-12-17 21:00:38,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c9575bc060ce15b1b8 HTTP/1.1" 502 150 2025-12-17 21:00:38,381 - lava_callback - ERROR - Error processing callback for node 694076c9575bc060ce15b1b8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c9575bc060ce15b1b8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c9575bc060ce15b1b8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c9575bc060ce15b1b8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:00:38,384 - lava_callback - INFO - Starting async job submit for node 694076db575bc060ce15b1f6 2025-12-17 21:00:38,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:38,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076db575bc060ce15b1f6 HTTP/1.1" 502 150 2025-12-17 21:00:38,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076db575bc060ce15b1f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:38,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076db575bc060ce15b1f6 2025-12-17 21:00:38,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076db575bc060ce15b1f6 HTTP/1.1" 502 150 2025-12-17 21:00:38,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076db575bc060ce15b1f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:38,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ca575bc060ce15b1bb HTTP/1.1" 502 150 2025-12-17 21:00:38,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ca575bc060ce15b1bb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:38,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d8575bc060ce15b1ec 2025-12-17 21:00:38,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d8575bc060ce15b1ec HTTP/1.1" 502 150 2025-12-17 21:00:38,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d8575bc060ce15b1ec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:38,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d4575bc060ce15b1e1 2025-12-17 21:00:38,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d4575bc060ce15b1e1 HTTP/1.1" 502 150 2025-12-17 21:00:38,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d4575bc060ce15b1e1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:39,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076da575bc060ce15b1f3 2025-12-17 21:00:39,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076da575bc060ce15b1f3 HTTP/1.1" 502 150 2025-12-17 21:00:39,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076da575bc060ce15b1f3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:39,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d6575bc060ce15b1e6 2025-12-17 21:00:39,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d6575bc060ce15b1e6 HTTP/1.1" 502 150 2025-12-17 21:00:39,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d6575bc060ce15b1e6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:40,100 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076cc575bc060ce15b1be 2025-12-17 21:00:40,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cc575bc060ce15b1be HTTP/1.1" 502 150 2025-12-17 21:00:40,102 - lava_callback - ERROR - Error processing callback for node 694076cc575bc060ce15b1be: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076cc575bc060ce15b1be (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076cc575bc060ce15b1be (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076cc575bc060ce15b1be (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:00:40,104 - lava_callback - INFO - Starting async job submit for node 694076dd575bc060ce15b1fc 2025-12-17 21:00:40,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:40,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076dd575bc060ce15b1fc HTTP/1.1" 502 150 2025-12-17 21:00:40,111 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076dd575bc060ce15b1fc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:40,111 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076dd575bc060ce15b1fc 2025-12-17 21:00:40,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076dd575bc060ce15b1fc HTTP/1.1" 502 150 2025-12-17 21:00:40,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076dd575bc060ce15b1fc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:40,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076db575bc060ce15b1f6 2025-12-17 21:00:40,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076db575bc060ce15b1f6 HTTP/1.1" 502 150 2025-12-17 21:00:40,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076db575bc060ce15b1f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:40,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076cd575bc060ce15b1c1 2025-12-17 21:00:40,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cd575bc060ce15b1c1 HTTP/1.1" 502 150 2025-12-17 21:00:40,932 - lava_callback - ERROR - Error processing callback for node 694076cd575bc060ce15b1c1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076cd575bc060ce15b1c1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076cd575bc060ce15b1c1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076cd575bc060ce15b1c1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:00:40,933 - lava_callback - INFO - Starting async job submit for node 694076df575bc060ce15b1ff 2025-12-17 21:00:40,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:40,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076df575bc060ce15b1ff HTTP/1.1" 502 150 2025-12-17 21:00:40,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076df575bc060ce15b1ff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:40,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076df575bc060ce15b1ff 2025-12-17 21:00:40,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076df575bc060ce15b1ff HTTP/1.1" 502 150 2025-12-17 21:00:40,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076df575bc060ce15b1ff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:41,320 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d7575bc060ce15b1e9 2025-12-17 21:00:41,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d7575bc060ce15b1e9 HTTP/1.1" 502 150 2025-12-17 21:00:41,321 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d7575bc060ce15b1e9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:41,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ce575bc060ce15b1c4 2025-12-17 21:00:41,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ce575bc060ce15b1c4 HTTP/1.1" 502 150 2025-12-17 21:00:41,672 - lava_callback - ERROR - Error processing callback for node 694076ce575bc060ce15b1c4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ce575bc060ce15b1c4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ce575bc060ce15b1c4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ce575bc060ce15b1c4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:00:41,674 - lava_callback - INFO - Starting async job submit for node 694076e3575bc060ce15b202 2025-12-17 21:00:41,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:41,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e3575bc060ce15b202 HTTP/1.1" 502 150 2025-12-17 21:00:41,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e3575bc060ce15b202'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:41,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e3575bc060ce15b202 2025-12-17 21:00:41,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e3575bc060ce15b202 HTTP/1.1" 502 150 2025-12-17 21:00:41,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e3575bc060ce15b202'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:42,112 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076dd575bc060ce15b1fc 2025-12-17 21:00:42,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076dd575bc060ce15b1fc HTTP/1.1" 502 150 2025-12-17 21:00:42,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076dd575bc060ce15b1fc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:42,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076cf575bc060ce15b1c9 2025-12-17 21:00:42,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076cf575bc060ce15b1c9 HTTP/1.1" 502 150 2025-12-17 21:00:42,501 - lava_callback - ERROR - Error processing callback for node 694076cf575bc060ce15b1c9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076cf575bc060ce15b1c9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076cf575bc060ce15b1c9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076cf575bc060ce15b1c9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:00:42,503 - lava_callback - INFO - Starting async job submit for node 694076e4575bc060ce15b207 2025-12-17 21:00:42,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:42,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e4575bc060ce15b207 HTTP/1.1" 502 150 2025-12-17 21:00:42,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e4575bc060ce15b207'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:42,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e4575bc060ce15b207 2025-12-17 21:00:42,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e4575bc060ce15b207 HTTP/1.1" 502 150 2025-12-17 21:00:42,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e4575bc060ce15b207'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:42,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d8575bc060ce15b1ec 2025-12-17 21:00:42,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d8575bc060ce15b1ec HTTP/1.1" 502 150 2025-12-17 21:00:42,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d8575bc060ce15b1ec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:42,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076df575bc060ce15b1ff 2025-12-17 21:00:42,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076df575bc060ce15b1ff HTTP/1.1" 502 150 2025-12-17 21:00:42,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076df575bc060ce15b1ff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:43,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d1575bc060ce15b1ce 2025-12-17 21:00:43,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d1575bc060ce15b1ce HTTP/1.1" 502 150 2025-12-17 21:00:43,598 - lava_callback - ERROR - Error processing callback for node 694076d1575bc060ce15b1ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d1575bc060ce15b1ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d1575bc060ce15b1ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d1575bc060ce15b1ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:00:43,600 - lava_callback - INFO - Starting async job submit for node 694076e5575bc060ce15b20e 2025-12-17 21:00:43,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:43,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e5575bc060ce15b20e HTTP/1.1" 502 150 2025-12-17 21:00:43,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e5575bc060ce15b20e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:43,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e5575bc060ce15b20e 2025-12-17 21:00:43,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e5575bc060ce15b20e HTTP/1.1" 502 150 2025-12-17 21:00:43,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e5575bc060ce15b20e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:43,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076da575bc060ce15b1f3 2025-12-17 21:00:43,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076da575bc060ce15b1f3 HTTP/1.1" 502 150 2025-12-17 21:00:43,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076da575bc060ce15b1f3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:43,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e3575bc060ce15b202 2025-12-17 21:00:43,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e3575bc060ce15b202 HTTP/1.1" 502 150 2025-12-17 21:00:43,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e3575bc060ce15b202'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:44,350 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d2575bc060ce15b1d5 2025-12-17 21:00:44,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d2575bc060ce15b1d5 HTTP/1.1" 502 150 2025-12-17 21:00:44,351 - lava_callback - ERROR - Error processing callback for node 694076d2575bc060ce15b1d5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d2575bc060ce15b1d5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d2575bc060ce15b1d5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d2575bc060ce15b1d5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:00:44,353 - lava_callback - INFO - Starting async job submit for node 694076e7575bc060ce15b215 2025-12-17 21:00:44,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:44,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e7575bc060ce15b215 HTTP/1.1" 502 150 2025-12-17 21:00:44,361 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e7575bc060ce15b215'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:44,361 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e7575bc060ce15b215 2025-12-17 21:00:44,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e7575bc060ce15b215 HTTP/1.1" 502 150 2025-12-17 21:00:44,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e7575bc060ce15b215'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:44,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076db575bc060ce15b1f6 2025-12-17 21:00:44,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076db575bc060ce15b1f6 HTTP/1.1" 502 150 2025-12-17 21:00:44,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076db575bc060ce15b1f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:44,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e4575bc060ce15b207 2025-12-17 21:00:44,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e4575bc060ce15b207 HTTP/1.1" 502 150 2025-12-17 21:00:44,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e4575bc060ce15b207'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:45,103 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d3575bc060ce15b1db 2025-12-17 21:00:45,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e5575bc060ce15b20e 2025-12-17 21:00:45,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e5575bc060ce15b20e HTTP/1.1" 502 150 2025-12-17 21:00:45,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e5575bc060ce15b20e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:46,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076dd575bc060ce15b1fc 2025-12-17 21:00:46,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076dd575bc060ce15b1fc HTTP/1.1" 502 150 2025-12-17 21:00:46,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076dd575bc060ce15b1fc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:46,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e7575bc060ce15b215 2025-12-17 21:00:46,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e7575bc060ce15b215 HTTP/1.1" 502 150 2025-12-17 21:00:46,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e7575bc060ce15b215'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37430 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:00:46,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d4575bc060ce15b1e1 2025-12-17 21:00:46,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d4575bc060ce15b1e1 HTTP/1.1" 502 150 2025-12-17 21:00:46,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d4575bc060ce15b1e1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:46,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076df575bc060ce15b1ff 2025-12-17 21:00:46,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076df575bc060ce15b1ff HTTP/1.1" 502 150 2025-12-17 21:00:46,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076df575bc060ce15b1ff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:47,458 - lava_callback - INFO - Received callback for node 694076eb575bc060ce15b228 INFO: 172.21.0.1:37446 - "POST /node/694076eb575bc060ce15b228 HTTP/1.0" 202 Accepted 2025-12-17 21:00:47,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e3575bc060ce15b202 2025-12-17 21:00:47,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e3575bc060ce15b202 HTTP/1.1" 502 150 2025-12-17 21:00:47,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e3575bc060ce15b202'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:47,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d6575bc060ce15b1e6 2025-12-17 21:00:47,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d6575bc060ce15b1e6 HTTP/1.1" 502 150 2025-12-17 21:00:47,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d6575bc060ce15b1e6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:48,373 - lava_callback - INFO - Received callback for node 694076ec575bc060ce15b22d INFO: 172.21.0.1:37458 - "POST /node/694076ec575bc060ce15b22d HTTP/1.0" 202 Accepted 2025-12-17 21:00:48,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e4575bc060ce15b207 2025-12-17 21:00:48,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e4575bc060ce15b207 HTTP/1.1" 502 150 2025-12-17 21:00:48,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e4575bc060ce15b207'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:49,320 - lava_callback - INFO - Received callback for node 694076ed575bc060ce15b230 INFO: 172.21.0.1:37472 - "POST /node/694076ed575bc060ce15b230 HTTP/1.0" 202 Accepted 2025-12-17 21:00:49,326 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d7575bc060ce15b1e9 2025-12-17 21:00:49,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e5575bc060ce15b20e 2025-12-17 21:00:49,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e5575bc060ce15b20e HTTP/1.1" 502 150 2025-12-17 21:00:49,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e5575bc060ce15b20e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:49,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c3575bc060ce15b1a5 HTTP/1.1" 502 150 2025-12-17 21:00:49,972 - lava_callback - ERROR - Error processing callback for node 694076c3575bc060ce15b1a5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c3575bc060ce15b1a5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c3575bc060ce15b1a5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c3575bc060ce15b1a5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:00:49,974 - lava_callback - INFO - Starting async job submit for node 694076e8575bc060ce15b21a 2025-12-17 21:00:49,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:49,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e8575bc060ce15b21a HTTP/1.1" 502 150 2025-12-17 21:00:49,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e8575bc060ce15b21a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:49,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e8575bc060ce15b21a 2025-12-17 21:00:49,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e8575bc060ce15b21a HTTP/1.1" 502 150 2025-12-17 21:00:49,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e8575bc060ce15b21a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:50,226 - lava_callback - INFO - Received callback for node 694076ee575bc060ce15b237 INFO: 172.21.0.1:51454 - "POST /node/694076ee575bc060ce15b237 HTTP/1.0" 202 Accepted 2025-12-17 21:00:50,364 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e7575bc060ce15b215 2025-12-17 21:00:50,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e7575bc060ce15b215 HTTP/1.1" 502 150 2025-12-17 21:00:50,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e7575bc060ce15b215'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:50,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d8575bc060ce15b1ec 2025-12-17 21:00:50,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d8575bc060ce15b1ec HTTP/1.1" 502 150 2025-12-17 21:00:50,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d8575bc060ce15b1ec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:50,986 - lava_callback - INFO - Received callback for node 694076f0575bc060ce15b23a INFO: 172.21.0.1:51458 - "POST /node/694076f0575bc060ce15b23a HTTP/1.0" 202 Accepted 2025-12-17 21:00:51,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076c5575bc060ce15b1ab 2025-12-17 21:00:51,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076c5575bc060ce15b1ab HTTP/1.1" 502 150 2025-12-17 21:00:51,349 - lava_callback - ERROR - Error processing callback for node 694076c5575bc060ce15b1ab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c5575bc060ce15b1ab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c5575bc060ce15b1ab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076c5575bc060ce15b1ab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:00:51,351 - lava_callback - INFO - Starting async job submit for node 694076e9575bc060ce15b221 2025-12-17 21:00:51,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:51,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e9575bc060ce15b221 HTTP/1.1" 502 150 2025-12-17 21:00:51,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e9575bc060ce15b221'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:51,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e9575bc060ce15b221 2025-12-17 21:00:51,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e9575bc060ce15b221 HTTP/1.1" 502 150 2025-12-17 21:00:51,385 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e9575bc060ce15b221'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:51,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076da575bc060ce15b1f3 2025-12-17 21:00:51,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076da575bc060ce15b1f3 HTTP/1.1" 502 150 2025-12-17 21:00:51,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076da575bc060ce15b1f3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:51,741 - lava_callback - INFO - Received callback for node 694076f1575bc060ce15b23d INFO: 172.21.0.1:51468 - "POST /node/694076f1575bc060ce15b23d HTTP/1.0" 202 Accepted 2025-12-17 21:00:51,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e8575bc060ce15b21a 2025-12-17 21:00:51,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e8575bc060ce15b21a HTTP/1.1" 502 150 2025-12-17 21:00:51,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e8575bc060ce15b21a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:52,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076db575bc060ce15b1f6 2025-12-17 21:00:52,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076db575bc060ce15b1f6 HTTP/1.1" 502 150 2025-12-17 21:00:52,406 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076db575bc060ce15b1f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:52,569 - lava_callback - INFO - Received callback for node 694076f6575bc060ce15b240 INFO: 172.21.0.1:51484 - "POST /node/694076f6575bc060ce15b240 HTTP/1.0" 202 Accepted 2025-12-17 21:00:53,311 - lava_callback - INFO - Received callback for node 694076f7575bc060ce15b245 INFO: 172.21.0.1:51500 - "POST /node/694076f7575bc060ce15b245 HTTP/1.0" 202 Accepted 2025-12-17 21:00:53,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e9575bc060ce15b221 2025-12-17 21:00:53,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e9575bc060ce15b221 HTTP/1.1" 502 150 2025-12-17 21:00:53,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e9575bc060ce15b221'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:54,115 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076dd575bc060ce15b1fc 2025-12-17 21:00:54,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076dd575bc060ce15b1fc HTTP/1.1" 502 150 2025-12-17 21:00:54,116 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076dd575bc060ce15b1fc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:54,249 - lava_callback - INFO - Received callback for node 694076f9575bc060ce15b269 INFO: 172.21.0.1:51504 - "POST /node/694076f9575bc060ce15b269 HTTP/1.0" 202 Accepted 2025-12-17 21:00:54,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ca575bc060ce15b1bb 2025-12-17 21:00:54,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ca575bc060ce15b1bb HTTP/1.1" 502 150 2025-12-17 21:00:54,710 - lava_callback - ERROR - Error processing callback for node 694076ca575bc060ce15b1bb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ca575bc060ce15b1bb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ca575bc060ce15b1bb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ca575bc060ce15b1bb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:00:54,712 - lava_callback - INFO - Starting async job submit for node 694076eb575bc060ce15b228 2025-12-17 21:00:54,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:00:54,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076eb575bc060ce15b228 HTTP/1.1" 502 150 2025-12-17 21:00:54,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076eb575bc060ce15b228'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:54,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076eb575bc060ce15b228 2025-12-17 21:00:54,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076eb575bc060ce15b228 HTTP/1.1" 502 150 2025-12-17 21:00:54,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076eb575bc060ce15b228'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:54,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076df575bc060ce15b1ff 2025-12-17 21:00:54,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076df575bc060ce15b1ff HTTP/1.1" 502 150 2025-12-17 21:00:54,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076df575bc060ce15b1ff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:55,053 - lava_callback - INFO - Received callback for node 694076fb575bc060ce15b26e INFO: 172.21.0.1:51520 - "POST /node/694076fb575bc060ce15b26e HTTP/1.0" 202 Accepted 2025-12-17 21:00:55,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e3575bc060ce15b202 2025-12-17 21:00:55,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e3575bc060ce15b202 HTTP/1.1" 502 150 2025-12-17 21:00:55,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e3575bc060ce15b202'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:55,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e8575bc060ce15b21a 2025-12-17 21:00:55,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e8575bc060ce15b21a HTTP/1.1" 502 150 2025-12-17 21:00:55,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e8575bc060ce15b21a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:56,480 - lava_callback - INFO - Received callback for node 694076fc575bc060ce15b271 INFO: 172.21.0.1:51524 - "POST /node/694076fc575bc060ce15b271 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:51538 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:00:56,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e4575bc060ce15b207 2025-12-17 21:00:56,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e4575bc060ce15b207 HTTP/1.1" 502 150 2025-12-17 21:00:56,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e4575bc060ce15b207'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:56,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076eb575bc060ce15b228 2025-12-17 21:00:56,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076eb575bc060ce15b228 HTTP/1.1" 502 150 2025-12-17 21:00:56,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076eb575bc060ce15b228'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:57,260 - lava_callback - INFO - Received callback for node 694076fd575bc060ce15b274 INFO: 172.21.0.1:51554 - "POST /node/694076fd575bc060ce15b274 HTTP/1.0" 202 Accepted 2025-12-17 21:00:57,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e9575bc060ce15b221 2025-12-17 21:00:57,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e9575bc060ce15b221 HTTP/1.1" 502 150 2025-12-17 21:00:57,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e9575bc060ce15b221'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:57,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e5575bc060ce15b20e 2025-12-17 21:00:57,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e5575bc060ce15b20e HTTP/1.1" 502 150 2025-12-17 21:00:57,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e5575bc060ce15b20e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:58,018 - lava_callback - INFO - Received callback for node 694076ff575bc060ce15b277 INFO: 172.21.0.1:51564 - "POST /node/694076ff575bc060ce15b277 HTTP/1.0" 202 Accepted 2025-12-17 21:00:58,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e7575bc060ce15b215 2025-12-17 21:00:58,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e7575bc060ce15b215 HTTP/1.1" 502 150 2025-12-17 21:00:58,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e7575bc060ce15b215'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:00:58,922 - lava_callback - INFO - Received callback for node 69407700575bc060ce15b27a INFO: 172.21.0.1:51580 - "POST /node/69407700575bc060ce15b27a HTTP/1.0" 202 Accepted 2025-12-17 21:00:59,840 - lava_callback - INFO - Received callback for node 69407701575bc060ce15b27d INFO: 172.21.0.1:35570 - "POST /node/69407701575bc060ce15b27d HTTP/1.0" 202 Accepted 2025-12-17 21:01:00,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d3575bc060ce15b1db HTTP/1.1" 502 150 2025-12-17 21:01:00,464 - lava_callback - ERROR - Error processing callback for node 694076d3575bc060ce15b1db: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d3575bc060ce15b1db (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d3575bc060ce15b1db (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d3575bc060ce15b1db (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:00,466 - lava_callback - INFO - Starting async job submit for node 694076ec575bc060ce15b22d 2025-12-17 21:01:00,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:00,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ec575bc060ce15b22d HTTP/1.1" 502 150 2025-12-17 21:01:00,495 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ec575bc060ce15b22d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:00,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ec575bc060ce15b22d 2025-12-17 21:01:00,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ec575bc060ce15b22d HTTP/1.1" 502 150 2025-12-17 21:01:00,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ec575bc060ce15b22d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:00,642 - lava_callback - INFO - Received callback for node 69407703575bc060ce15b280 INFO: 172.21.0.1:35576 - "POST /node/69407703575bc060ce15b280 HTTP/1.0" 202 Accepted 2025-12-17 21:01:00,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076eb575bc060ce15b228 2025-12-17 21:01:00,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076eb575bc060ce15b228 HTTP/1.1" 502 150 2025-12-17 21:01:00,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076eb575bc060ce15b228'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35590 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:01:01,510 - lava_callback - INFO - Received callback for node 69407704575bc060ce15b283 INFO: 172.21.0.1:35598 - "POST /node/69407704575bc060ce15b283 HTTP/1.0" 202 Accepted 2025-12-17 21:01:02,259 - lava_callback - INFO - Received callback for node 69407705575bc060ce15b286 INFO: 172.21.0.1:35602 - "POST /node/69407705575bc060ce15b286 HTTP/1.0" 202 Accepted 2025-12-17 21:01:02,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ec575bc060ce15b22d 2025-12-17 21:01:02,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ec575bc060ce15b22d HTTP/1.1" 502 150 2025-12-17 21:01:02,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ec575bc060ce15b22d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:02,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d4575bc060ce15b1e1 2025-12-17 21:01:02,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d4575bc060ce15b1e1 HTTP/1.1" 502 150 2025-12-17 21:01:02,898 - lava_callback - ERROR - Error processing callback for node 694076d4575bc060ce15b1e1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d4575bc060ce15b1e1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d4575bc060ce15b1e1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d4575bc060ce15b1e1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:02,900 - lava_callback - INFO - Starting async job submit for node 694076ed575bc060ce15b230 2025-12-17 21:01:02,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:02,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ed575bc060ce15b230 HTTP/1.1" 502 150 2025-12-17 21:01:02,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ed575bc060ce15b230'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:02,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ed575bc060ce15b230 2025-12-17 21:01:02,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ed575bc060ce15b230 HTTP/1.1" 502 150 2025-12-17 21:01:02,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ed575bc060ce15b230'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:03,149 - lava_callback - INFO - Received callback for node 69407706575bc060ce15b289 INFO: 172.21.0.1:35614 - "POST /node/69407706575bc060ce15b289 HTTP/1.0" 202 Accepted 2025-12-17 21:01:03,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d6575bc060ce15b1e6 2025-12-17 21:01:03,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d6575bc060ce15b1e6 HTTP/1.1" 502 150 2025-12-17 21:01:03,704 - lava_callback - ERROR - Error processing callback for node 694076d6575bc060ce15b1e6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d6575bc060ce15b1e6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d6575bc060ce15b1e6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d6575bc060ce15b1e6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:03,706 - lava_callback - INFO - Starting async job submit for node 694076ee575bc060ce15b237 2025-12-17 21:01:03,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:03,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ee575bc060ce15b237 HTTP/1.1" 502 150 2025-12-17 21:01:03,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ee575bc060ce15b237'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:03,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ee575bc060ce15b237 2025-12-17 21:01:03,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ee575bc060ce15b237 HTTP/1.1" 502 150 2025-12-17 21:01:03,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ee575bc060ce15b237'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:03,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e8575bc060ce15b21a 2025-12-17 21:01:03,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e8575bc060ce15b21a HTTP/1.1" 502 150 2025-12-17 21:01:03,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e8575bc060ce15b21a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:03,999 - lava_callback - INFO - Received callback for node 69407707575bc060ce15b28c INFO: 172.21.0.1:35620 - "POST /node/69407707575bc060ce15b28c HTTP/1.0" 202 Accepted 2025-12-17 21:01:04,771 - lava_callback - INFO - Received callback for node 69407709575bc060ce15b28f INFO: 172.21.0.1:35628 - "POST /node/69407709575bc060ce15b28f HTTP/1.0" 202 Accepted 2025-12-17 21:01:04,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d7575bc060ce15b1e9 HTTP/1.1" 502 150 2025-12-17 21:01:04,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076d7575bc060ce15b1e9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:04,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ed575bc060ce15b230 2025-12-17 21:01:04,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ed575bc060ce15b230 HTTP/1.1" 502 150 2025-12-17 21:01:04,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ed575bc060ce15b230'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:05,388 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e9575bc060ce15b221 2025-12-17 21:01:05,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e9575bc060ce15b221 HTTP/1.1" 502 150 2025-12-17 21:01:05,389 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076e9575bc060ce15b221'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:05,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ee575bc060ce15b237 2025-12-17 21:01:05,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ee575bc060ce15b237 HTTP/1.1" 502 150 2025-12-17 21:01:05,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ee575bc060ce15b237'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:05,733 - lava_callback - INFO - Received callback for node 6940770a575bc060ce15b292 INFO: 172.21.0.1:35640 - "POST /node/6940770a575bc060ce15b292 HTTP/1.0" 202 Accepted 2025-12-17 21:01:06,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ec575bc060ce15b22d 2025-12-17 21:01:06,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ec575bc060ce15b22d HTTP/1.1" 502 150 2025-12-17 21:01:06,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ec575bc060ce15b22d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35642 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:01:06,626 - lava_callback - INFO - Received callback for node 6940770b575bc060ce15b295 INFO: 172.21.0.1:35656 - "POST /node/6940770b575bc060ce15b295 HTTP/1.0" 202 Accepted 2025-12-17 21:01:06,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d8575bc060ce15b1ec 2025-12-17 21:01:06,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d8575bc060ce15b1ec HTTP/1.1" 502 150 2025-12-17 21:01:06,857 - lava_callback - ERROR - Error processing callback for node 694076d8575bc060ce15b1ec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d8575bc060ce15b1ec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d8575bc060ce15b1ec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d8575bc060ce15b1ec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:06,859 - lava_callback - INFO - Starting async job submit for node 694076f0575bc060ce15b23a 2025-12-17 21:01:06,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:06,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f0575bc060ce15b23a HTTP/1.1" 502 150 2025-12-17 21:01:06,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f0575bc060ce15b23a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:06,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f0575bc060ce15b23a 2025-12-17 21:01:06,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f0575bc060ce15b23a HTTP/1.1" 502 150 2025-12-17 21:01:06,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f0575bc060ce15b23a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:07,403 - lava_callback - INFO - Received callback for node 6940770c575bc060ce15b298 INFO: 172.21.0.1:35664 - "POST /node/6940770c575bc060ce15b298 HTTP/1.0" 202 Accepted 2025-12-17 21:01:07,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076da575bc060ce15b1f3 2025-12-17 21:01:07,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076da575bc060ce15b1f3 HTTP/1.1" 502 150 2025-12-17 21:01:07,624 - lava_callback - ERROR - Error processing callback for node 694076da575bc060ce15b1f3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076da575bc060ce15b1f3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076da575bc060ce15b1f3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076da575bc060ce15b1f3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:07,626 - lava_callback - INFO - Starting async job submit for node 694076f1575bc060ce15b23d 2025-12-17 21:01:07,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:07,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f1575bc060ce15b23d HTTP/1.1" 502 150 2025-12-17 21:01:07,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f1575bc060ce15b23d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:07,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f1575bc060ce15b23d 2025-12-17 21:01:07,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f1575bc060ce15b23d HTTP/1.1" 502 150 2025-12-17 21:01:07,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f1575bc060ce15b23d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:08,200 - lava_callback - INFO - Received callback for node 6940770e575bc060ce15b29b INFO: 172.21.0.1:35668 - "POST /node/6940770e575bc060ce15b29b HTTP/1.0" 202 Accepted 2025-12-17 21:01:08,406 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076db575bc060ce15b1f6 2025-12-17 21:01:08,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076db575bc060ce15b1f6 HTTP/1.1" 502 150 2025-12-17 21:01:08,407 - lava_callback - ERROR - Error processing callback for node 694076db575bc060ce15b1f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076db575bc060ce15b1f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076db575bc060ce15b1f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076db575bc060ce15b1f6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:08,409 - lava_callback - INFO - Starting async job submit for node 694076f6575bc060ce15b240 2025-12-17 21:01:08,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:08,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f6575bc060ce15b240 HTTP/1.1" 502 150 2025-12-17 21:01:08,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f6575bc060ce15b240'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:08,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f6575bc060ce15b240 2025-12-17 21:01:08,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f6575bc060ce15b240 HTTP/1.1" 502 150 2025-12-17 21:01:08,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f6575bc060ce15b240'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:08,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076eb575bc060ce15b228 2025-12-17 21:01:08,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076eb575bc060ce15b228 HTTP/1.1" 502 150 2025-12-17 21:01:08,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076eb575bc060ce15b228'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:08,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f0575bc060ce15b23a 2025-12-17 21:01:08,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f0575bc060ce15b23a HTTP/1.1" 502 150 2025-12-17 21:01:08,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f0575bc060ce15b23a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:08,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ed575bc060ce15b230 2025-12-17 21:01:08,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ed575bc060ce15b230 HTTP/1.1" 502 150 2025-12-17 21:01:08,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ed575bc060ce15b230'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:09,045 - lava_callback - INFO - Received callback for node 6940770f575bc060ce15b29e INFO: 172.21.0.1:35676 - "POST /node/6940770f575bc060ce15b29e HTTP/1.0" 202 Accepted 2025-12-17 21:01:09,545 - lava_callback - INFO - Received callback for node 69407710575bc060ce15b2a1 INFO: 172.21.0.1:35682 - "POST /node/69407710575bc060ce15b2a1 HTTP/1.0" 202 Accepted 2025-12-17 21:01:09,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f1575bc060ce15b23d 2025-12-17 21:01:09,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f1575bc060ce15b23d HTTP/1.1" 502 150 2025-12-17 21:01:09,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f1575bc060ce15b23d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:09,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ee575bc060ce15b237 2025-12-17 21:01:09,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ee575bc060ce15b237 HTTP/1.1" 502 150 2025-12-17 21:01:09,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ee575bc060ce15b237'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:10,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076dd575bc060ce15b1fc 2025-12-17 21:01:10,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076dd575bc060ce15b1fc HTTP/1.1" 502 150 2025-12-17 21:01:10,118 - lava_callback - ERROR - Error processing callback for node 694076dd575bc060ce15b1fc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076dd575bc060ce15b1fc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076dd575bc060ce15b1fc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076dd575bc060ce15b1fc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:10,121 - lava_callback - INFO - Starting async job submit for node 694076f7575bc060ce15b245 2025-12-17 21:01:10,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:10,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f7575bc060ce15b245 HTTP/1.1" 502 150 2025-12-17 21:01:10,155 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f7575bc060ce15b245'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:10,155 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f7575bc060ce15b245 2025-12-17 21:01:10,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f7575bc060ce15b245 HTTP/1.1" 502 150 2025-12-17 21:01:10,156 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f7575bc060ce15b245'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:10,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f6575bc060ce15b240 2025-12-17 21:01:10,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f6575bc060ce15b240 HTTP/1.1" 502 150 2025-12-17 21:01:10,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f6575bc060ce15b240'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:10,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076df575bc060ce15b1ff 2025-12-17 21:01:10,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076df575bc060ce15b1ff HTTP/1.1" 502 150 2025-12-17 21:01:10,952 - lava_callback - ERROR - Error processing callback for node 694076df575bc060ce15b1ff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076df575bc060ce15b1ff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076df575bc060ce15b1ff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076df575bc060ce15b1ff (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:10,955 - lava_callback - INFO - Starting async job submit for node 694076f9575bc060ce15b269 2025-12-17 21:01:10,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:10,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f9575bc060ce15b269 HTTP/1.1" 502 150 2025-12-17 21:01:10,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f9575bc060ce15b269'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:10,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f9575bc060ce15b269 2025-12-17 21:01:10,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f9575bc060ce15b269 HTTP/1.1" 502 150 2025-12-17 21:01:10,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f9575bc060ce15b269'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:11,214 - lava_callback - INFO - Received callback for node 694078da575bc060ce15b588 INFO: 172.21.0.1:46124 - "POST /node/694078da575bc060ce15b588 HTTP/1.0" 202 Accepted 2025-12-17 21:01:11,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e3575bc060ce15b202 2025-12-17 21:01:11,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e3575bc060ce15b202 HTTP/1.1" 502 150 2025-12-17 21:01:11,687 - lava_callback - ERROR - Error processing callback for node 694076e3575bc060ce15b202: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e3575bc060ce15b202 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e3575bc060ce15b202 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e3575bc060ce15b202 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:11,690 - lava_callback - INFO - Starting async job submit for node 694076fb575bc060ce15b26e 2025-12-17 21:01:11,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:11,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fb575bc060ce15b26e HTTP/1.1" 502 150 2025-12-17 21:01:11,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076fb575bc060ce15b26e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:11,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076fb575bc060ce15b26e 2025-12-17 21:01:11,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fb575bc060ce15b26e HTTP/1.1" 502 150 2025-12-17 21:01:11,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076fb575bc060ce15b26e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:12,157 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f7575bc060ce15b245 2025-12-17 21:01:12,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f7575bc060ce15b245 HTTP/1.1" 502 150 2025-12-17 21:01:12,158 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f7575bc060ce15b245'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:12,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e4575bc060ce15b207 2025-12-17 21:01:12,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e4575bc060ce15b207 HTTP/1.1" 502 150 2025-12-17 21:01:12,543 - lava_callback - ERROR - Error processing callback for node 694076e4575bc060ce15b207: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e4575bc060ce15b207 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e4575bc060ce15b207 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e4575bc060ce15b207 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:12,545 - lava_callback - INFO - Starting async job submit for node 694076fc575bc060ce15b271 2025-12-17 21:01:12,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:12,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fc575bc060ce15b271 HTTP/1.1" 502 150 2025-12-17 21:01:12,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076fc575bc060ce15b271'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:12,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076fc575bc060ce15b271 2025-12-17 21:01:12,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fc575bc060ce15b271 HTTP/1.1" 502 150 2025-12-17 21:01:12,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076fc575bc060ce15b271'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:12,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f0575bc060ce15b23a 2025-12-17 21:01:12,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f0575bc060ce15b23a HTTP/1.1" 502 150 2025-12-17 21:01:12,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f0575bc060ce15b23a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:12,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f9575bc060ce15b269 2025-12-17 21:01:12,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f9575bc060ce15b269 HTTP/1.1" 502 150 2025-12-17 21:01:12,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f9575bc060ce15b269'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:13,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e5575bc060ce15b20e 2025-12-17 21:01:13,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e5575bc060ce15b20e HTTP/1.1" 502 150 2025-12-17 21:01:13,613 - lava_callback - ERROR - Error processing callback for node 694076e5575bc060ce15b20e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e5575bc060ce15b20e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e5575bc060ce15b20e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e5575bc060ce15b20e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:13,615 - lava_callback - INFO - Starting async job submit for node 694076fd575bc060ce15b274 2025-12-17 21:01:13,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:13,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fd575bc060ce15b274 HTTP/1.1" 502 150 2025-12-17 21:01:13,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076fd575bc060ce15b274'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:13,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076fd575bc060ce15b274 2025-12-17 21:01:13,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fd575bc060ce15b274 HTTP/1.1" 502 150 2025-12-17 21:01:13,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076fd575bc060ce15b274'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:13,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f1575bc060ce15b23d 2025-12-17 21:01:13,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f1575bc060ce15b23d HTTP/1.1" 502 150 2025-12-17 21:01:13,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f1575bc060ce15b23d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:13,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076fb575bc060ce15b26e 2025-12-17 21:01:13,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fb575bc060ce15b26e HTTP/1.1" 502 150 2025-12-17 21:01:13,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076fb575bc060ce15b26e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:14,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e7575bc060ce15b215 2025-12-17 21:01:14,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e7575bc060ce15b215 HTTP/1.1" 502 150 2025-12-17 21:01:14,368 - lava_callback - ERROR - Error processing callback for node 694076e7575bc060ce15b215: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e7575bc060ce15b215 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e7575bc060ce15b215 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e7575bc060ce15b215 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:14,370 - lava_callback - INFO - Starting async job submit for node 694076ff575bc060ce15b277 2025-12-17 21:01:14,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:14,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ff575bc060ce15b277 HTTP/1.1" 502 150 2025-12-17 21:01:14,377 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ff575bc060ce15b277'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:14,377 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ff575bc060ce15b277 2025-12-17 21:01:14,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ff575bc060ce15b277 HTTP/1.1" 502 150 2025-12-17 21:01:14,378 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ff575bc060ce15b277'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:14,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f6575bc060ce15b240 2025-12-17 21:01:14,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f6575bc060ce15b240 HTTP/1.1" 502 150 2025-12-17 21:01:14,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f6575bc060ce15b240'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:14,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ec575bc060ce15b22d 2025-12-17 21:01:14,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ec575bc060ce15b22d HTTP/1.1" 502 150 2025-12-17 21:01:14,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ec575bc060ce15b22d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:14,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076fc575bc060ce15b271 2025-12-17 21:01:14,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fc575bc060ce15b271 HTTP/1.1" 502 150 2025-12-17 21:01:14,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076fc575bc060ce15b271'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46126 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:01:15,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076fd575bc060ce15b274 2025-12-17 21:01:15,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fd575bc060ce15b274 HTTP/1.1" 502 150 2025-12-17 21:01:15,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076fd575bc060ce15b274'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:16,158 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f7575bc060ce15b245 2025-12-17 21:01:16,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f7575bc060ce15b245 HTTP/1.1" 502 150 2025-12-17 21:01:16,159 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f7575bc060ce15b245'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:16,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ff575bc060ce15b277 2025-12-17 21:01:16,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ff575bc060ce15b277 HTTP/1.1" 502 150 2025-12-17 21:01:16,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ff575bc060ce15b277'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46140 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:01:16,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ed575bc060ce15b230 2025-12-17 21:01:16,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ed575bc060ce15b230 HTTP/1.1" 502 150 2025-12-17 21:01:16,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ed575bc060ce15b230'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:16,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f9575bc060ce15b269 2025-12-17 21:01:16,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f9575bc060ce15b269 HTTP/1.1" 502 150 2025-12-17 21:01:16,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f9575bc060ce15b269'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:17,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ee575bc060ce15b237 2025-12-17 21:01:17,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ee575bc060ce15b237 HTTP/1.1" 502 150 2025-12-17 21:01:17,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ee575bc060ce15b237'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:17,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076fb575bc060ce15b26e 2025-12-17 21:01:17,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fb575bc060ce15b26e HTTP/1.1" 502 150 2025-12-17 21:01:17,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076fb575bc060ce15b26e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:18,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076fc575bc060ce15b271 2025-12-17 21:01:18,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fc575bc060ce15b271 HTTP/1.1" 502 150 2025-12-17 21:01:18,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076fc575bc060ce15b271'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:19,622 - lava_callback - INFO - Received callback for node 6940c5291e3c24e95681d686 INFO: 172.21.0.1:46150 - "POST /node/6940c5291e3c24e95681d686 HTTP/1.0" 202 Accepted 2025-12-17 21:01:19,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076fd575bc060ce15b274 2025-12-17 21:01:19,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fd575bc060ce15b274 HTTP/1.1" 502 150 2025-12-17 21:01:19,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076fd575bc060ce15b274'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:19,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e8575bc060ce15b21a 2025-12-17 21:01:19,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e8575bc060ce15b21a HTTP/1.1" 502 150 2025-12-17 21:01:19,987 - lava_callback - ERROR - Error processing callback for node 694076e8575bc060ce15b21a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e8575bc060ce15b21a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e8575bc060ce15b21a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e8575bc060ce15b21a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:19,989 - lava_callback - INFO - Starting async job submit for node 69407700575bc060ce15b27a 2025-12-17 21:01:19,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:20,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407700575bc060ce15b27a HTTP/1.1" 502 150 2025-12-17 21:01:20,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407700575bc060ce15b27a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:20,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407700575bc060ce15b27a 2025-12-17 21:01:20,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407700575bc060ce15b27a HTTP/1.1" 502 150 2025-12-17 21:01:20,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407700575bc060ce15b27a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:20,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ff575bc060ce15b277 2025-12-17 21:01:20,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ff575bc060ce15b277 HTTP/1.1" 502 150 2025-12-17 21:01:20,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ff575bc060ce15b277'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:20,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076d7575bc060ce15b1e9 2025-12-17 21:01:20,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076d7575bc060ce15b1e9 HTTP/1.1" 502 150 2025-12-17 21:01:20,818 - lava_callback - ERROR - Error processing callback for node 694076d7575bc060ce15b1e9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d7575bc060ce15b1e9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d7575bc060ce15b1e9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076d7575bc060ce15b1e9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:20,820 - lava_callback - INFO - Starting async job submit for node 69407701575bc060ce15b27d 2025-12-17 21:01:20,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:20,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407701575bc060ce15b27d HTTP/1.1" 502 150 2025-12-17 21:01:20,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407701575bc060ce15b27d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:20,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407701575bc060ce15b27d 2025-12-17 21:01:20,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407701575bc060ce15b27d HTTP/1.1" 502 150 2025-12-17 21:01:20,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407701575bc060ce15b27d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:20,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f0575bc060ce15b23a 2025-12-17 21:01:20,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f0575bc060ce15b23a HTTP/1.1" 502 150 2025-12-17 21:01:20,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f0575bc060ce15b23a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:21,389 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076e9575bc060ce15b221 2025-12-17 21:01:21,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076e9575bc060ce15b221 HTTP/1.1" 502 150 2025-12-17 21:01:21,391 - lava_callback - ERROR - Error processing callback for node 694076e9575bc060ce15b221: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e9575bc060ce15b221 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e9575bc060ce15b221 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076e9575bc060ce15b221 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:21,393 - lava_callback - INFO - Starting async job submit for node 69407703575bc060ce15b280 2025-12-17 21:01:21,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:21,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407703575bc060ce15b280 HTTP/1.1" 502 150 2025-12-17 21:01:21,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407703575bc060ce15b280'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:21,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407703575bc060ce15b280 2025-12-17 21:01:21,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407703575bc060ce15b280 HTTP/1.1" 502 150 2025-12-17 21:01:21,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407703575bc060ce15b280'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:21,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f1575bc060ce15b23d 2025-12-17 21:01:21,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f1575bc060ce15b23d HTTP/1.1" 502 150 2025-12-17 21:01:21,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f1575bc060ce15b23d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:22,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407700575bc060ce15b27a 2025-12-17 21:01:22,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407700575bc060ce15b27a HTTP/1.1" 502 150 2025-12-17 21:01:22,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407700575bc060ce15b27a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:22,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f6575bc060ce15b240 2025-12-17 21:01:22,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f6575bc060ce15b240 HTTP/1.1" 502 150 2025-12-17 21:01:22,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f6575bc060ce15b240'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:22,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407701575bc060ce15b27d 2025-12-17 21:01:22,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407701575bc060ce15b27d HTTP/1.1" 502 150 2025-12-17 21:01:22,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407701575bc060ce15b27d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:23,098 - lava_callback - INFO - Received callback for node 69407dcf575bc060ce15c18f INFO: 172.21.0.1:51592 - "POST /node/69407dcf575bc060ce15c18f HTTP/1.0" 202 Accepted 2025-12-17 21:01:23,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407703575bc060ce15b280 2025-12-17 21:01:23,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407703575bc060ce15b280 HTTP/1.1" 502 150 2025-12-17 21:01:23,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407703575bc060ce15b280'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:24,160 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f7575bc060ce15b245 2025-12-17 21:01:24,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f7575bc060ce15b245 HTTP/1.1" 502 150 2025-12-17 21:01:24,161 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f7575bc060ce15b245'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:24,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076eb575bc060ce15b228 2025-12-17 21:01:24,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076eb575bc060ce15b228 HTTP/1.1" 502 150 2025-12-17 21:01:24,736 - lava_callback - ERROR - Error processing callback for node 694076eb575bc060ce15b228: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076eb575bc060ce15b228 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076eb575bc060ce15b228 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076eb575bc060ce15b228 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:24,738 - lava_callback - INFO - Starting async job submit for node 69407704575bc060ce15b283 2025-12-17 21:01:24,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:24,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407704575bc060ce15b283 HTTP/1.1" 502 150 2025-12-17 21:01:24,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407704575bc060ce15b283'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:24,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407704575bc060ce15b283 2025-12-17 21:01:24,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407704575bc060ce15b283 HTTP/1.1" 502 150 2025-12-17 21:01:24,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407704575bc060ce15b283'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:24,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f9575bc060ce15b269 2025-12-17 21:01:24,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f9575bc060ce15b269 HTTP/1.1" 502 150 2025-12-17 21:01:24,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076f9575bc060ce15b269'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:25,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076fb575bc060ce15b26e 2025-12-17 21:01:25,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fb575bc060ce15b26e HTTP/1.1" 502 150 2025-12-17 21:01:25,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076fb575bc060ce15b26e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:26,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407700575bc060ce15b27a INFO: 172.21.0.1:51606 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:01:26,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076fc575bc060ce15b271 2025-12-17 21:01:26,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fc575bc060ce15b271 HTTP/1.1" 502 150 2025-12-17 21:01:26,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076fc575bc060ce15b271'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:26,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407704575bc060ce15b283 2025-12-17 21:01:26,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407704575bc060ce15b283 HTTP/1.1" 502 150 2025-12-17 21:01:26,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407704575bc060ce15b283'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:26,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407701575bc060ce15b27d 2025-12-17 21:01:26,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407701575bc060ce15b27d HTTP/1.1" 502 150 2025-12-17 21:01:26,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407701575bc060ce15b27d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:27,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407703575bc060ce15b280 2025-12-17 21:01:27,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407703575bc060ce15b280 HTTP/1.1" 502 150 2025-12-17 21:01:27,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407703575bc060ce15b280'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:27,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076fd575bc060ce15b274 2025-12-17 21:01:27,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fd575bc060ce15b274 HTTP/1.1" 502 150 2025-12-17 21:01:27,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076fd575bc060ce15b274'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:28,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ff575bc060ce15b277 2025-12-17 21:01:28,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ff575bc060ce15b277 HTTP/1.1" 502 150 2025-12-17 21:01:28,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076ff575bc060ce15b277'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:30,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ec575bc060ce15b22d 2025-12-17 21:01:30,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ec575bc060ce15b22d HTTP/1.1" 502 150 2025-12-17 21:01:30,503 - lava_callback - ERROR - Error processing callback for node 694076ec575bc060ce15b22d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ec575bc060ce15b22d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ec575bc060ce15b22d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ec575bc060ce15b22d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:30,505 - lava_callback - INFO - Starting async job submit for node 69407705575bc060ce15b286 2025-12-17 21:01:30,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:30,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407705575bc060ce15b286 HTTP/1.1" 502 150 2025-12-17 21:01:30,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407705575bc060ce15b286'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:30,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407705575bc060ce15b286 2025-12-17 21:01:30,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407705575bc060ce15b286 HTTP/1.1" 502 150 2025-12-17 21:01:30,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407705575bc060ce15b286'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:30,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407704575bc060ce15b283 2025-12-17 21:01:30,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407704575bc060ce15b283 HTTP/1.1" 502 150 2025-12-17 21:01:30,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407704575bc060ce15b283'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:32,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407705575bc060ce15b286 2025-12-17 21:01:32,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407705575bc060ce15b286 HTTP/1.1" 502 150 2025-12-17 21:01:32,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407705575bc060ce15b286'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:32,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ed575bc060ce15b230 2025-12-17 21:01:32,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ed575bc060ce15b230 HTTP/1.1" 502 150 2025-12-17 21:01:32,916 - lava_callback - ERROR - Error processing callback for node 694076ed575bc060ce15b230: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ed575bc060ce15b230 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ed575bc060ce15b230 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ed575bc060ce15b230 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:32,917 - lava_callback - INFO - Starting async job submit for node 69407706575bc060ce15b289 2025-12-17 21:01:32,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:32,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407706575bc060ce15b289 HTTP/1.1" 502 150 2025-12-17 21:01:32,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407706575bc060ce15b289'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:32,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407706575bc060ce15b289 2025-12-17 21:01:32,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407706575bc060ce15b289 HTTP/1.1" 502 150 2025-12-17 21:01:32,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407706575bc060ce15b289'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:33,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ee575bc060ce15b237 2025-12-17 21:01:33,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ee575bc060ce15b237 HTTP/1.1" 502 150 2025-12-17 21:01:33,720 - lava_callback - ERROR - Error processing callback for node 694076ee575bc060ce15b237: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ee575bc060ce15b237 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ee575bc060ce15b237 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ee575bc060ce15b237 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:33,722 - lava_callback - INFO - Starting async job submit for node 69407707575bc060ce15b28c 2025-12-17 21:01:33,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:33,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407707575bc060ce15b28c HTTP/1.1" 502 150 2025-12-17 21:01:33,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407707575bc060ce15b28c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:33,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407707575bc060ce15b28c 2025-12-17 21:01:33,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407707575bc060ce15b28c HTTP/1.1" 502 150 2025-12-17 21:01:33,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407707575bc060ce15b28c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:34,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407701575bc060ce15b27d 2025-12-17 21:01:34,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407701575bc060ce15b27d HTTP/1.1" 502 150 2025-12-17 21:01:34,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407701575bc060ce15b27d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:34,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407706575bc060ce15b289 2025-12-17 21:01:34,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407706575bc060ce15b289 HTTP/1.1" 502 150 2025-12-17 21:01:34,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407706575bc060ce15b289'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:35,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407703575bc060ce15b280 2025-12-17 21:01:35,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407703575bc060ce15b280 HTTP/1.1" 502 150 2025-12-17 21:01:35,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407703575bc060ce15b280'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:35,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407707575bc060ce15b28c 2025-12-17 21:01:35,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407707575bc060ce15b28c HTTP/1.1" 502 150 2025-12-17 21:01:35,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407707575bc060ce15b28c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46108 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:01:36,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407705575bc060ce15b286 2025-12-17 21:01:36,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407705575bc060ce15b286 HTTP/1.1" 502 150 2025-12-17 21:01:36,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407705575bc060ce15b286'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:36,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f0575bc060ce15b23a 2025-12-17 21:01:36,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f0575bc060ce15b23a HTTP/1.1" 502 150 2025-12-17 21:01:36,883 - lava_callback - ERROR - Error processing callback for node 694076f0575bc060ce15b23a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076f0575bc060ce15b23a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076f0575bc060ce15b23a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076f0575bc060ce15b23a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:36,885 - lava_callback - INFO - Starting async job submit for node 69407709575bc060ce15b28f 2025-12-17 21:01:36,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:36,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407709575bc060ce15b28f HTTP/1.1" 502 150 2025-12-17 21:01:36,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407709575bc060ce15b28f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:36,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407709575bc060ce15b28f 2025-12-17 21:01:36,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407709575bc060ce15b28f HTTP/1.1" 502 150 2025-12-17 21:01:36,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407709575bc060ce15b28f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:37,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f1575bc060ce15b23d 2025-12-17 21:01:38,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f6575bc060ce15b240 2025-12-17 21:01:38,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f6575bc060ce15b240 HTTP/1.1" 502 150 2025-12-17 21:01:38,425 - lava_callback - ERROR - Error processing callback for node 694076f6575bc060ce15b240: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076f6575bc060ce15b240 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076f6575bc060ce15b240 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076f6575bc060ce15b240 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:38,427 - lava_callback - INFO - Starting async job submit for node 6940770a575bc060ce15b292 2025-12-17 21:01:38,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:38,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770a575bc060ce15b292 HTTP/1.1" 502 150 2025-12-17 21:01:38,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770a575bc060ce15b292'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:38,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770a575bc060ce15b292 2025-12-17 21:01:38,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770a575bc060ce15b292 HTTP/1.1" 502 150 2025-12-17 21:01:38,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770a575bc060ce15b292'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:38,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407704575bc060ce15b283 2025-12-17 21:01:38,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407704575bc060ce15b283 HTTP/1.1" 502 150 2025-12-17 21:01:38,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407704575bc060ce15b283'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:38,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407709575bc060ce15b28f 2025-12-17 21:01:38,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407709575bc060ce15b28f HTTP/1.1" 502 150 2025-12-17 21:01:38,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407709575bc060ce15b28f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:38,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407706575bc060ce15b289 2025-12-17 21:01:38,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407706575bc060ce15b289 HTTP/1.1" 502 150 2025-12-17 21:01:38,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407706575bc060ce15b289'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:39,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407707575bc060ce15b28c 2025-12-17 21:01:39,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407707575bc060ce15b28c HTTP/1.1" 502 150 2025-12-17 21:01:39,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407707575bc060ce15b28c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:40,161 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f7575bc060ce15b245 2025-12-17 21:01:40,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f7575bc060ce15b245 HTTP/1.1" 502 150 2025-12-17 21:01:40,163 - lava_callback - ERROR - Error processing callback for node 694076f7575bc060ce15b245: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076f7575bc060ce15b245 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076f7575bc060ce15b245 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076f7575bc060ce15b245 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:40,165 - lava_callback - INFO - Starting async job submit for node 6940770b575bc060ce15b295 2025-12-17 21:01:40,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:40,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770b575bc060ce15b295 HTTP/1.1" 502 150 2025-12-17 21:01:40,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770b575bc060ce15b295'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:40,178 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770b575bc060ce15b295 2025-12-17 21:01:40,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770b575bc060ce15b295 HTTP/1.1" 502 150 2025-12-17 21:01:40,179 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770b575bc060ce15b295'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:40,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770a575bc060ce15b292 2025-12-17 21:01:40,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770a575bc060ce15b292 HTTP/1.1" 502 150 2025-12-17 21:01:40,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770a575bc060ce15b292'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:40,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076f9575bc060ce15b269 2025-12-17 21:01:40,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f9575bc060ce15b269 HTTP/1.1" 502 150 2025-12-17 21:01:40,979 - lava_callback - ERROR - Error processing callback for node 694076f9575bc060ce15b269: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076f9575bc060ce15b269 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076f9575bc060ce15b269 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076f9575bc060ce15b269 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:40,981 - lava_callback - INFO - Starting async job submit for node 6940770c575bc060ce15b298 2025-12-17 21:01:40,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:41,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407700575bc060ce15b27a HTTP/1.1" 502 150 2025-12-17 21:01:41,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407700575bc060ce15b27a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:41,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076fb575bc060ce15b26e 2025-12-17 21:01:41,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fb575bc060ce15b26e HTTP/1.1" 502 150 2025-12-17 21:01:41,731 - lava_callback - ERROR - Error processing callback for node 694076fb575bc060ce15b26e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076fb575bc060ce15b26e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076fb575bc060ce15b26e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076fb575bc060ce15b26e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:41,734 - lava_callback - INFO - Starting async job submit for node 6940770e575bc060ce15b29b 2025-12-17 21:01:41,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:41,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770e575bc060ce15b29b HTTP/1.1" 502 150 2025-12-17 21:01:41,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770e575bc060ce15b29b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:41,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770e575bc060ce15b29b 2025-12-17 21:01:41,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770e575bc060ce15b29b HTTP/1.1" 502 150 2025-12-17 21:01:41,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770e575bc060ce15b29b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:42,179 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770b575bc060ce15b295 2025-12-17 21:01:42,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770b575bc060ce15b295 HTTP/1.1" 502 150 2025-12-17 21:01:42,181 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770b575bc060ce15b295'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:42,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076fc575bc060ce15b271 2025-12-17 21:01:42,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fc575bc060ce15b271 HTTP/1.1" 502 150 2025-12-17 21:01:42,580 - lava_callback - ERROR - Error processing callback for node 694076fc575bc060ce15b271: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076fc575bc060ce15b271 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076fc575bc060ce15b271 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076fc575bc060ce15b271 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:42,582 - lava_callback - INFO - Starting async job submit for node 6940770f575bc060ce15b29e 2025-12-17 21:01:42,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:42,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770f575bc060ce15b29e HTTP/1.1" 502 150 2025-12-17 21:01:42,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770f575bc060ce15b29e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:42,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770f575bc060ce15b29e 2025-12-17 21:01:42,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770f575bc060ce15b29e HTTP/1.1" 502 150 2025-12-17 21:01:42,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770f575bc060ce15b29e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:42,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407709575bc060ce15b28f 2025-12-17 21:01:42,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407709575bc060ce15b28f HTTP/1.1" 502 150 2025-12-17 21:01:42,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407709575bc060ce15b28f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:43,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076fd575bc060ce15b274 2025-12-17 21:01:43,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076fd575bc060ce15b274 HTTP/1.1" 502 150 2025-12-17 21:01:43,633 - lava_callback - ERROR - Error processing callback for node 694076fd575bc060ce15b274: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076fd575bc060ce15b274 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076fd575bc060ce15b274 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076fd575bc060ce15b274 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:43,636 - lava_callback - INFO - Starting async job submit for node 69407710575bc060ce15b2a1 2025-12-17 21:01:43,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:43,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407710575bc060ce15b2a1 HTTP/1.1" 502 150 2025-12-17 21:01:43,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407710575bc060ce15b2a1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:43,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407710575bc060ce15b2a1 2025-12-17 21:01:43,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407710575bc060ce15b2a1 HTTP/1.1" 502 150 2025-12-17 21:01:43,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407710575bc060ce15b2a1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:43,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770e575bc060ce15b29b 2025-12-17 21:01:43,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770e575bc060ce15b29b HTTP/1.1" 502 150 2025-12-17 21:01:43,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770e575bc060ce15b29b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:44,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076ff575bc060ce15b277 2025-12-17 21:01:44,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076ff575bc060ce15b277 HTTP/1.1" 502 150 2025-12-17 21:01:44,386 - lava_callback - ERROR - Error processing callback for node 694076ff575bc060ce15b277: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ff575bc060ce15b277 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ff575bc060ce15b277 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076ff575bc060ce15b277 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:44,388 - lava_callback - INFO - Starting async job submit for node 694078da575bc060ce15b588 2025-12-17 21:01:44,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:44,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078da575bc060ce15b588 HTTP/1.1" 502 150 2025-12-17 21:01:44,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078da575bc060ce15b588'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:44,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078da575bc060ce15b588 2025-12-17 21:01:44,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078da575bc060ce15b588 HTTP/1.1" 502 150 2025-12-17 21:01:44,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078da575bc060ce15b588'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:44,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770a575bc060ce15b292 2025-12-17 21:01:44,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770a575bc060ce15b292 HTTP/1.1" 502 150 2025-12-17 21:01:44,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770a575bc060ce15b292'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:44,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407705575bc060ce15b286 2025-12-17 21:01:44,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407705575bc060ce15b286 HTTP/1.1" 502 150 2025-12-17 21:01:44,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407705575bc060ce15b286'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:44,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770f575bc060ce15b29e 2025-12-17 21:01:44,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770f575bc060ce15b29e HTTP/1.1" 502 150 2025-12-17 21:01:44,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770f575bc060ce15b29e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:45,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407710575bc060ce15b2a1 2025-12-17 21:01:45,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407710575bc060ce15b2a1 HTTP/1.1" 502 150 2025-12-17 21:01:45,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407710575bc060ce15b2a1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:46,181 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770b575bc060ce15b295 2025-12-17 21:01:46,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770b575bc060ce15b295 HTTP/1.1" 502 150 2025-12-17 21:01:46,182 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770b575bc060ce15b295'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:46,357 - lava_callback - INFO - Received callback for node 69407736575bc060ce15b2c1 INFO: 172.21.0.1:42962 - "POST /node/69407736575bc060ce15b2c1 HTTP/1.0" 202 Accepted 2025-12-17 21:01:46,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078da575bc060ce15b588 2025-12-17 21:01:46,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078da575bc060ce15b588 HTTP/1.1" 502 150 2025-12-17 21:01:46,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078da575bc060ce15b588'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42968 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:01:46,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407706575bc060ce15b289 2025-12-17 21:01:46,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407706575bc060ce15b289 HTTP/1.1" 502 150 2025-12-17 21:01:46,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407706575bc060ce15b289'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:47,036 - lava_callback - INFO - Received callback for node 69407738575bc060ce15b2cf INFO: 172.21.0.1:42984 - "POST /node/69407738575bc060ce15b2cf HTTP/1.0" 202 Accepted 2025-12-17 21:01:47,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407707575bc060ce15b28c 2025-12-17 21:01:47,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407707575bc060ce15b28c HTTP/1.1" 502 150 2025-12-17 21:01:47,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407707575bc060ce15b28c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:47,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770e575bc060ce15b29b 2025-12-17 21:01:47,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770e575bc060ce15b29b HTTP/1.1" 502 150 2025-12-17 21:01:47,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770e575bc060ce15b29b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:48,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770f575bc060ce15b29e 2025-12-17 21:01:48,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770f575bc060ce15b29e HTTP/1.1" 502 150 2025-12-17 21:01:48,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770f575bc060ce15b29e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:49,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407700575bc060ce15b27a 2025-12-17 21:01:49,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407700575bc060ce15b27a HTTP/1.1" 502 150 2025-12-17 21:01:49,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407700575bc060ce15b27a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:49,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407710575bc060ce15b2a1 2025-12-17 21:01:49,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407710575bc060ce15b2a1 HTTP/1.1" 502 150 2025-12-17 21:01:49,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407710575bc060ce15b2a1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:50,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078da575bc060ce15b588 2025-12-17 21:01:50,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078da575bc060ce15b588 HTTP/1.1" 502 150 2025-12-17 21:01:50,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078da575bc060ce15b588'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:50,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407701575bc060ce15b27d 2025-12-17 21:01:50,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407701575bc060ce15b27d HTTP/1.1" 502 150 2025-12-17 21:01:50,832 - lava_callback - ERROR - Error processing callback for node 69407701575bc060ce15b27d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407701575bc060ce15b27d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407701575bc060ce15b27d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407701575bc060ce15b27d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:50,835 - lava_callback - INFO - Starting async job submit for node 6940c5291e3c24e95681d686 2025-12-17 21:01:50,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:50,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407709575bc060ce15b28f 2025-12-17 21:01:50,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407709575bc060ce15b28f HTTP/1.1" 502 150 2025-12-17 21:01:50,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407709575bc060ce15b28f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:50,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5291e3c24e95681d686 HTTP/1.1" 502 150 2025-12-17 21:01:50,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5291e3c24e95681d686'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:50,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5291e3c24e95681d686 2025-12-17 21:01:50,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5291e3c24e95681d686 HTTP/1.1" 502 150 2025-12-17 21:01:50,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5291e3c24e95681d686'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:51,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407703575bc060ce15b280 2025-12-17 21:01:51,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407703575bc060ce15b280 HTTP/1.1" 502 150 2025-12-17 21:01:51,406 - lava_callback - ERROR - Error processing callback for node 69407703575bc060ce15b280: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407703575bc060ce15b280 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407703575bc060ce15b280 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407703575bc060ce15b280 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:51,408 - lava_callback - INFO - Starting async job submit for node 69407dcf575bc060ce15c18f 2025-12-17 21:01:51,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:51,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dcf575bc060ce15c18f HTTP/1.1" 502 150 2025-12-17 21:01:51,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dcf575bc060ce15c18f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:51,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dcf575bc060ce15c18f 2025-12-17 21:01:51,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dcf575bc060ce15c18f HTTP/1.1" 502 150 2025-12-17 21:01:51,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dcf575bc060ce15c18f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:52,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770a575bc060ce15b292 2025-12-17 21:01:52,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5291e3c24e95681d686 2025-12-17 21:01:52,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5291e3c24e95681d686 HTTP/1.1" 502 150 2025-12-17 21:01:52,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5291e3c24e95681d686'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:52,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076f1575bc060ce15b23d HTTP/1.1" 502 150 2025-12-17 21:01:52,944 - lava_callback - ERROR - Error processing callback for node 694076f1575bc060ce15b23d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076f1575bc060ce15b23d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076f1575bc060ce15b23d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076f1575bc060ce15b23d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:52,946 - lava_callback - INFO - Starting async job submit for node 69407736575bc060ce15b2c1 2025-12-17 21:01:52,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:52,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407736575bc060ce15b2c1 HTTP/1.1" 502 150 2025-12-17 21:01:52,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407736575bc060ce15b2c1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:52,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407736575bc060ce15b2c1 2025-12-17 21:01:52,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407736575bc060ce15b2c1 HTTP/1.1" 502 150 2025-12-17 21:01:52,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407736575bc060ce15b2c1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:53,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dcf575bc060ce15c18f 2025-12-17 21:01:53,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dcf575bc060ce15c18f HTTP/1.1" 502 150 2025-12-17 21:01:53,437 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dcf575bc060ce15c18f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:54,182 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770b575bc060ce15b295 2025-12-17 21:01:54,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770b575bc060ce15b295 HTTP/1.1" 502 150 2025-12-17 21:01:54,184 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770b575bc060ce15b295'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:54,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407704575bc060ce15b283 2025-12-17 21:01:54,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407704575bc060ce15b283 HTTP/1.1" 502 150 2025-12-17 21:01:54,756 - lava_callback - ERROR - Error processing callback for node 69407704575bc060ce15b283: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407704575bc060ce15b283 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407704575bc060ce15b283 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407704575bc060ce15b283 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:01:54,758 - lava_callback - INFO - Starting async job submit for node 69407738575bc060ce15b2cf 2025-12-17 21:01:54,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:01:54,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407738575bc060ce15b2cf HTTP/1.1" 502 150 2025-12-17 21:01:54,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407738575bc060ce15b2cf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:54,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407738575bc060ce15b2cf 2025-12-17 21:01:54,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407738575bc060ce15b2cf HTTP/1.1" 502 150 2025-12-17 21:01:54,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407738575bc060ce15b2cf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:54,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407736575bc060ce15b2c1 2025-12-17 21:01:54,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407736575bc060ce15b2c1 HTTP/1.1" 502 150 2025-12-17 21:01:54,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407736575bc060ce15b2c1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:55,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770e575bc060ce15b29b 2025-12-17 21:01:55,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770e575bc060ce15b29b HTTP/1.1" 502 150 2025-12-17 21:01:55,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770e575bc060ce15b29b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37408 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:01:56,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770f575bc060ce15b29e 2025-12-17 21:01:56,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770f575bc060ce15b29e HTTP/1.1" 502 150 2025-12-17 21:01:56,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770f575bc060ce15b29e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:56,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407738575bc060ce15b2cf 2025-12-17 21:01:56,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407738575bc060ce15b2cf HTTP/1.1" 502 150 2025-12-17 21:01:56,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407738575bc060ce15b2cf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:56,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5291e3c24e95681d686 2025-12-17 21:01:56,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5291e3c24e95681d686 HTTP/1.1" 502 150 2025-12-17 21:01:56,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5291e3c24e95681d686'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:57,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dcf575bc060ce15c18f 2025-12-17 21:01:57,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dcf575bc060ce15c18f HTTP/1.1" 502 150 2025-12-17 21:01:57,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dcf575bc060ce15c18f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:57,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407710575bc060ce15b2a1 2025-12-17 21:01:57,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407710575bc060ce15b2a1 HTTP/1.1" 502 150 2025-12-17 21:01:57,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407710575bc060ce15b2a1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:58,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078da575bc060ce15b588 2025-12-17 21:01:58,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078da575bc060ce15b588 HTTP/1.1" 502 150 2025-12-17 21:01:58,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078da575bc060ce15b588'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:01:58,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407736575bc060ce15b2c1 2025-12-17 21:01:58,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407736575bc060ce15b2c1 HTTP/1.1" 502 150 2025-12-17 21:01:58,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407736575bc060ce15b2c1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:00,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407705575bc060ce15b286 2025-12-17 21:02:00,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407705575bc060ce15b286 HTTP/1.1" 502 150 2025-12-17 21:02:00,527 - lava_callback - ERROR - Error processing callback for node 69407705575bc060ce15b286: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407705575bc060ce15b286 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407705575bc060ce15b286 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407705575bc060ce15b286 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:00,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407738575bc060ce15b2cf 2025-12-17 21:02:00,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407738575bc060ce15b2cf HTTP/1.1" 502 150 2025-12-17 21:02:00,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407738575bc060ce15b2cf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55186 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:02:02,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407706575bc060ce15b289 2025-12-17 21:02:02,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407706575bc060ce15b289 HTTP/1.1" 502 150 2025-12-17 21:02:02,930 - lava_callback - ERROR - Error processing callback for node 69407706575bc060ce15b289: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407706575bc060ce15b289 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407706575bc060ce15b289 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407706575bc060ce15b289 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:03,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407707575bc060ce15b28c 2025-12-17 21:02:03,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407707575bc060ce15b28c HTTP/1.1" 502 150 2025-12-17 21:02:03,735 - lava_callback - ERROR - Error processing callback for node 69407707575bc060ce15b28c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407707575bc060ce15b28c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407707575bc060ce15b28c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407707575bc060ce15b28c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:04,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5291e3c24e95681d686 2025-12-17 21:02:04,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5291e3c24e95681d686 HTTP/1.1" 502 150 2025-12-17 21:02:04,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5291e3c24e95681d686'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:05,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407700575bc060ce15b27a 2025-12-17 21:02:05,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407700575bc060ce15b27a HTTP/1.1" 502 150 2025-12-17 21:02:05,432 - lava_callback - ERROR - Error processing callback for node 69407700575bc060ce15b27a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407700575bc060ce15b27a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407700575bc060ce15b27a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407700575bc060ce15b27a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:05,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dcf575bc060ce15c18f 2025-12-17 21:02:05,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dcf575bc060ce15c18f HTTP/1.1" 502 150 2025-12-17 21:02:05,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dcf575bc060ce15c18f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55200 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:02:06,679 - lava_callback - INFO - Received callback for node 69407739575bc060ce15b2d2 INFO: 172.21.0.1:55208 - "POST /node/69407739575bc060ce15b2d2 HTTP/1.0" 202 Accepted 2025-12-17 21:02:06,685 - lava_callback - INFO - Starting async job submit for node 69407739575bc060ce15b2d2 2025-12-17 21:02:06,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:02:06,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407739575bc060ce15b2d2 HTTP/1.1" 502 150 2025-12-17 21:02:06,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407739575bc060ce15b2d2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:06,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407739575bc060ce15b2d2 2025-12-17 21:02:06,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407739575bc060ce15b2d2 HTTP/1.1" 502 150 2025-12-17 21:02:06,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407739575bc060ce15b2d2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:06,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407709575bc060ce15b28f 2025-12-17 21:02:06,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407709575bc060ce15b28f HTTP/1.1" 502 150 2025-12-17 21:02:06,901 - lava_callback - ERROR - Error processing callback for node 69407709575bc060ce15b28f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407709575bc060ce15b28f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407709575bc060ce15b28f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407709575bc060ce15b28f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:06,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407736575bc060ce15b2c1 2025-12-17 21:02:06,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407736575bc060ce15b2c1 HTTP/1.1" 502 150 2025-12-17 21:02:06,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407736575bc060ce15b2c1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:07,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770a575bc060ce15b292 HTTP/1.1" 502 150 2025-12-17 21:02:07,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770a575bc060ce15b292'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:08,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407739575bc060ce15b2d2 2025-12-17 21:02:08,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407739575bc060ce15b2d2 HTTP/1.1" 502 150 2025-12-17 21:02:08,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407739575bc060ce15b2d2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:08,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407738575bc060ce15b2cf 2025-12-17 21:02:08,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407738575bc060ce15b2cf HTTP/1.1" 502 150 2025-12-17 21:02:08,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407738575bc060ce15b2cf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:10,184 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770b575bc060ce15b295 2025-12-17 21:02:10,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770b575bc060ce15b295 HTTP/1.1" 502 150 2025-12-17 21:02:10,191 - lava_callback - ERROR - Error processing callback for node 6940770b575bc060ce15b295: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940770b575bc060ce15b295 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940770b575bc060ce15b295 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940770b575bc060ce15b295 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:10,788 - lava_callback - INFO - Received callback for node 69407743575bc060ce15b2de INFO: 172.21.0.1:51230 - "POST /node/69407743575bc060ce15b2de HTTP/1.0" 202 Accepted 2025-12-17 21:02:10,794 - lava_callback - INFO - Starting async job submit for node 69407743575bc060ce15b2de 2025-12-17 21:02:10,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:02:10,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407743575bc060ce15b2de HTTP/1.1" 502 150 2025-12-17 21:02:10,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407743575bc060ce15b2de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:10,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407743575bc060ce15b2de 2025-12-17 21:02:10,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407743575bc060ce15b2de HTTP/1.1" 502 150 2025-12-17 21:02:10,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407743575bc060ce15b2de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:11,553 - lava_callback - INFO - Received callback for node 69407744575bc060ce15b2e1 INFO: 172.21.0.1:51242 - "POST /node/69407744575bc060ce15b2e1 HTTP/1.0" 202 Accepted 2025-12-17 21:02:11,563 - lava_callback - INFO - Starting async job submit for node 69407744575bc060ce15b2e1 2025-12-17 21:02:11,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:02:11,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407744575bc060ce15b2e1 HTTP/1.1" 502 150 2025-12-17 21:02:11,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407744575bc060ce15b2e1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:11,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407744575bc060ce15b2e1 2025-12-17 21:02:11,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407744575bc060ce15b2e1 HTTP/1.1" 502 150 2025-12-17 21:02:11,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407744575bc060ce15b2e1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:11,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770c575bc060ce15b298 HTTP/1.1" 502 150 2025-12-17 21:02:11,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770c575bc060ce15b298'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:11,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770c575bc060ce15b298 2025-12-17 21:02:11,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770c575bc060ce15b298 HTTP/1.1" 502 150 2025-12-17 21:02:11,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770c575bc060ce15b298'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:11,649 - lava_callback - INFO - Received callback for node 69407690575bc060ce15b104 INFO: 172.21.0.1:51256 - "POST /node/69407690575bc060ce15b104 HTTP/1.0" 202 Accepted 2025-12-17 21:02:11,660 - lava_callback - INFO - Starting async job submit for node 69407690575bc060ce15b104 2025-12-17 21:02:11,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770e575bc060ce15b29b 2025-12-17 21:02:11,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770e575bc060ce15b29b HTTP/1.1" 502 150 2025-12-17 21:02:11,764 - lava_callback - ERROR - Error processing callback for node 6940770e575bc060ce15b29b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940770e575bc060ce15b29b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940770e575bc060ce15b29b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940770e575bc060ce15b29b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:12,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:02:12,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407690575bc060ce15b104 HTTP/1.1" 502 150 2025-12-17 21:02:12,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407690575bc060ce15b104'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:12,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407690575bc060ce15b104 2025-12-17 21:02:12,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407690575bc060ce15b104 HTTP/1.1" 502 150 2025-12-17 21:02:12,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407690575bc060ce15b104'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:12,319 - lava_callback - INFO - Received callback for node 69407745575bc060ce15b2e4 INFO: 172.21.0.1:51268 - "POST /node/69407745575bc060ce15b2e4 HTTP/1.0" 202 Accepted 2025-12-17 21:02:12,327 - lava_callback - INFO - Starting async job submit for node 69407745575bc060ce15b2e4 2025-12-17 21:02:12,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:02:12,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407745575bc060ce15b2e4 HTTP/1.1" 502 150 2025-12-17 21:02:12,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407745575bc060ce15b2e4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:12,359 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407745575bc060ce15b2e4 2025-12-17 21:02:12,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407745575bc060ce15b2e4 HTTP/1.1" 502 150 2025-12-17 21:02:12,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407745575bc060ce15b2e4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:12,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770f575bc060ce15b29e 2025-12-17 21:02:12,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770f575bc060ce15b29e HTTP/1.1" 502 150 2025-12-17 21:02:12,596 - lava_callback - ERROR - Error processing callback for node 6940770f575bc060ce15b29e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940770f575bc060ce15b29e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940770f575bc060ce15b29e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940770f575bc060ce15b29e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:12,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407739575bc060ce15b2d2 2025-12-17 21:02:12,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407739575bc060ce15b2d2 HTTP/1.1" 502 150 2025-12-17 21:02:12,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407739575bc060ce15b2d2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:12,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407743575bc060ce15b2de 2025-12-17 21:02:12,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407743575bc060ce15b2de HTTP/1.1" 502 150 2025-12-17 21:02:12,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407743575bc060ce15b2de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:13,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407744575bc060ce15b2e1 2025-12-17 21:02:13,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407744575bc060ce15b2e1 HTTP/1.1" 502 150 2025-12-17 21:02:13,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407744575bc060ce15b2e1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:13,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770c575bc060ce15b298 2025-12-17 21:02:13,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770c575bc060ce15b298 HTTP/1.1" 502 150 2025-12-17 21:02:13,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770c575bc060ce15b298'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:13,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407710575bc060ce15b2a1 2025-12-17 21:02:13,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407710575bc060ce15b2a1 HTTP/1.1" 502 150 2025-12-17 21:02:13,652 - lava_callback - ERROR - Error processing callback for node 69407710575bc060ce15b2a1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407710575bc060ce15b2a1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407710575bc060ce15b2a1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407710575bc060ce15b2a1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:14,115 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407690575bc060ce15b104 2025-12-17 21:02:14,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407745575bc060ce15b2e4 2025-12-17 21:02:14,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407745575bc060ce15b2e4 HTTP/1.1" 502 150 2025-12-17 21:02:14,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407745575bc060ce15b2e4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:14,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078da575bc060ce15b588 2025-12-17 21:02:14,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078da575bc060ce15b588 HTTP/1.1" 502 150 2025-12-17 21:02:14,424 - lava_callback - ERROR - Error processing callback for node 694078da575bc060ce15b588: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078da575bc060ce15b588 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078da575bc060ce15b588 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078da575bc060ce15b588 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:51270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51280 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:02:16,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407743575bc060ce15b2de 2025-12-17 21:02:16,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407743575bc060ce15b2de HTTP/1.1" 502 150 2025-12-17 21:02:16,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407743575bc060ce15b2de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:17,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407744575bc060ce15b2e1 2025-12-17 21:02:17,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407744575bc060ce15b2e1 HTTP/1.1" 502 150 2025-12-17 21:02:17,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407744575bc060ce15b2e1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:17,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770c575bc060ce15b298 2025-12-17 21:02:17,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770c575bc060ce15b298 HTTP/1.1" 502 150 2025-12-17 21:02:17,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770c575bc060ce15b298'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:18,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407745575bc060ce15b2e4 2025-12-17 21:02:18,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407745575bc060ce15b2e4 HTTP/1.1" 502 150 2025-12-17 21:02:18,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407745575bc060ce15b2e4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:20,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407739575bc060ce15b2d2 2025-12-17 21:02:20,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407739575bc060ce15b2d2 HTTP/1.1" 502 150 2025-12-17 21:02:20,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407739575bc060ce15b2d2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:20,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5291e3c24e95681d686 2025-12-17 21:02:20,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5291e3c24e95681d686 HTTP/1.1" 502 150 2025-12-17 21:02:20,911 - lava_callback - ERROR - Error processing callback for node 6940c5291e3c24e95681d686: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5291e3c24e95681d686 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5291e3c24e95681d686 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5291e3c24e95681d686 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:21,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dcf575bc060ce15c18f 2025-12-17 21:02:21,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dcf575bc060ce15c18f HTTP/1.1" 502 150 2025-12-17 21:02:21,442 - lava_callback - ERROR - Error processing callback for node 69407dcf575bc060ce15c18f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dcf575bc060ce15c18f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dcf575bc060ce15c18f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dcf575bc060ce15c18f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:22,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407736575bc060ce15b2c1 2025-12-17 21:02:22,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407736575bc060ce15b2c1 HTTP/1.1" 502 150 2025-12-17 21:02:22,960 - lava_callback - ERROR - Error processing callback for node 69407736575bc060ce15b2c1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407736575bc060ce15b2c1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407736575bc060ce15b2c1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407736575bc060ce15b2c1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:23,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770a575bc060ce15b292 2025-12-17 21:02:23,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770a575bc060ce15b292 HTTP/1.1" 502 150 2025-12-17 21:02:23,794 - lava_callback - ERROR - Error processing callback for node 6940770a575bc060ce15b292: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940770a575bc060ce15b292 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940770a575bc060ce15b292 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940770a575bc060ce15b292 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:24,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407738575bc060ce15b2cf 2025-12-17 21:02:24,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407738575bc060ce15b2cf HTTP/1.1" 502 150 2025-12-17 21:02:24,771 - lava_callback - ERROR - Error processing callback for node 69407738575bc060ce15b2cf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407738575bc060ce15b2cf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407738575bc060ce15b2cf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407738575bc060ce15b2cf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:24,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407743575bc060ce15b2de 2025-12-17 21:02:24,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407743575bc060ce15b2de HTTP/1.1" 502 150 2025-12-17 21:02:24,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407743575bc060ce15b2de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:25,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407744575bc060ce15b2e1 2025-12-17 21:02:25,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770c575bc060ce15b298 2025-12-17 21:02:26,363 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407745575bc060ce15b2e4 2025-12-17 21:02:26,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407745575bc060ce15b2e4 HTTP/1.1" 502 150 2025-12-17 21:02:26,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407745575bc060ce15b2e4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54672 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:02:36,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407739575bc060ce15b2d2 2025-12-17 21:02:36,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407739575bc060ce15b2d2 HTTP/1.1" 502 150 2025-12-17 21:02:36,699 - lava_callback - ERROR - Error processing callback for node 69407739575bc060ce15b2d2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407739575bc060ce15b2d2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407739575bc060ce15b2d2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407739575bc060ce15b2d2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:39,487 - lava_callback - INFO - Received callback for node 694076bd575bc060ce15b199 INFO: 172.21.0.1:54676 - "POST /node/694076bd575bc060ce15b199 HTTP/1.0" 202 Accepted 2025-12-17 21:02:39,495 - lava_callback - INFO - Starting async job submit for node 694076bd575bc060ce15b199 2025-12-17 21:02:39,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:02:39,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bd575bc060ce15b199 HTTP/1.1" 502 150 2025-12-17 21:02:39,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076bd575bc060ce15b199'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:39,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076bd575bc060ce15b199 2025-12-17 21:02:39,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bd575bc060ce15b199 HTTP/1.1" 502 150 2025-12-17 21:02:39,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076bd575bc060ce15b199'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:40,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407743575bc060ce15b2de 2025-12-17 21:02:41,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770c575bc060ce15b298 HTTP/1.1" 502 150 2025-12-17 21:02:41,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940770c575bc060ce15b298'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:41,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407744575bc060ce15b2e1 HTTP/1.1" 502 150 2025-12-17 21:02:41,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407744575bc060ce15b2e1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:41,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076bd575bc060ce15b199 2025-12-17 21:02:41,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bd575bc060ce15b199 HTTP/1.1" 502 150 2025-12-17 21:02:41,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076bd575bc060ce15b199'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:42,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407745575bc060ce15b2e4 2025-12-17 21:02:42,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407745575bc060ce15b2e4 HTTP/1.1" 502 150 2025-12-17 21:02:42,367 - lava_callback - ERROR - Error processing callback for node 69407745575bc060ce15b2e4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407745575bc060ce15b2e4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407745575bc060ce15b2e4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407745575bc060ce15b2e4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:44,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407690575bc060ce15b104 HTTP/1.1" 502 150 2025-12-17 21:02:44,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407690575bc060ce15b104'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:45,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076bd575bc060ce15b199 2025-12-17 21:02:45,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bd575bc060ce15b199 HTTP/1.1" 502 150 2025-12-17 21:02:45,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076bd575bc060ce15b199'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33760 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:02:48,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407690575bc060ce15b104 2025-12-17 21:02:48,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407690575bc060ce15b104 HTTP/1.1" 502 150 2025-12-17 21:02:48,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407690575bc060ce15b104'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:51,573 - lava_callback - INFO - Received callback for node 69407e32575bc060ce15c24b INFO: 172.21.0.1:55550 - "POST /node/69407e32575bc060ce15c24b HTTP/1.0" 202 Accepted 2025-12-17 21:02:51,583 - lava_callback - INFO - Starting async job submit for node 69407e32575bc060ce15c24b 2025-12-17 21:02:51,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:02:51,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e32575bc060ce15c24b HTTP/1.1" 502 150 2025-12-17 21:02:51,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e32575bc060ce15c24b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:51,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e32575bc060ce15c24b 2025-12-17 21:02:51,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e32575bc060ce15c24b HTTP/1.1" 502 150 2025-12-17 21:02:51,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e32575bc060ce15c24b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:53,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076bd575bc060ce15b199 2025-12-17 21:02:53,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bd575bc060ce15b199 HTTP/1.1" 502 150 2025-12-17 21:02:53,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076bd575bc060ce15b199'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:53,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e32575bc060ce15c24b 2025-12-17 21:02:53,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e32575bc060ce15c24b HTTP/1.1" 502 150 2025-12-17 21:02:53,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e32575bc060ce15c24b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:56,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407743575bc060ce15b2de HTTP/1.1" 502 150 2025-12-17 21:02:56,176 - lava_callback - ERROR - Error processing callback for node 69407743575bc060ce15b2de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407743575bc060ce15b2de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407743575bc060ce15b2de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407743575bc060ce15b2de (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:55564 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:02:56,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407690575bc060ce15b104 2025-12-17 21:02:56,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407690575bc060ce15b104 HTTP/1.1" 502 150 2025-12-17 21:02:56,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407690575bc060ce15b104'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:02:57,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940770c575bc060ce15b298 2025-12-17 21:02:57,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407744575bc060ce15b2e1 2025-12-17 21:02:57,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407744575bc060ce15b2e1 HTTP/1.1" 502 150 2025-12-17 21:02:57,074 - lava_callback - ERROR - Error processing callback for node 69407744575bc060ce15b2e1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407744575bc060ce15b2e1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407744575bc060ce15b2e1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407744575bc060ce15b2e1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:02:57,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e32575bc060ce15c24b 2025-12-17 21:02:57,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e32575bc060ce15c24b HTTP/1.1" 502 150 2025-12-17 21:02:57,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e32575bc060ce15c24b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45446 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:03:05,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e32575bc060ce15c24b 2025-12-17 21:03:05,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e32575bc060ce15c24b HTTP/1.1" 502 150 2025-12-17 21:03:05,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e32575bc060ce15c24b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45462 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:03:09,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076bd575bc060ce15b199 2025-12-17 21:03:09,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bd575bc060ce15b199 HTTP/1.1" 502 150 2025-12-17 21:03:09,626 - lava_callback - ERROR - Error processing callback for node 694076bd575bc060ce15b199: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076bd575bc060ce15b199 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076bd575bc060ce15b199 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076bd575bc060ce15b199 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:03:12,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940770c575bc060ce15b298 HTTP/1.1" 502 150 2025-12-17 21:03:12,560 - lava_callback - ERROR - Error processing callback for node 6940770c575bc060ce15b298: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940770c575bc060ce15b298 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940770c575bc060ce15b298 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940770c575bc060ce15b298 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:03:12,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407690575bc060ce15b104 2025-12-17 21:03:12,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407690575bc060ce15b104 HTTP/1.1" 502 150 2025-12-17 21:03:12,917 - lava_callback - ERROR - Error processing callback for node 69407690575bc060ce15b104: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407690575bc060ce15b104 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407690575bc060ce15b104 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407690575bc060ce15b104 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:41422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41438 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:03:21,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e32575bc060ce15c24b INFO: 172.21.0.1:39814 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:03:30,565 - lava_callback - INFO - Received callback for node 69407e33575bc060ce15c250 INFO: 172.21.0.1:43312 - "POST /node/69407e33575bc060ce15c250 HTTP/1.0" 202 Accepted 2025-12-17 21:03:30,571 - lava_callback - INFO - Starting async job submit for node 69407e33575bc060ce15c250 2025-12-17 21:03:30,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:43322 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:03:46,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e33575bc060ce15c250 HTTP/1.1" 502 150 2025-12-17 21:03:46,096 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e33575bc060ce15c250'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:03:46,096 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e33575bc060ce15c250 2025-12-17 21:03:46,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e33575bc060ce15c250 HTTP/1.1" 502 150 2025-12-17 21:03:46,102 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e33575bc060ce15c250'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33464 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:03:48,102 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e33575bc060ce15c250 2025-12-17 21:03:48,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e33575bc060ce15c250 HTTP/1.1" 502 150 2025-12-17 21:03:48,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e33575bc060ce15c250'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:03:50,881 - lava_callback - INFO - Received callback for node 694077b0575bc060ce15b32b INFO: 172.21.0.1:36934 - "POST /node/694077b0575bc060ce15b32b HTTP/1.0" 202 Accepted 2025-12-17 21:03:50,887 - lava_callback - INFO - Starting async job submit for node 694077b0575bc060ce15b32b 2025-12-17 21:03:50,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:03:50,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b0575bc060ce15b32b HTTP/1.1" 502 150 2025-12-17 21:03:50,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b0575bc060ce15b32b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:03:50,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b0575bc060ce15b32b 2025-12-17 21:03:50,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b0575bc060ce15b32b HTTP/1.1" 502 150 2025-12-17 21:03:50,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b0575bc060ce15b32b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:03:51,719 - lava_callback - INFO - Received callback for node 694077b1575bc060ce15b345 INFO: 172.21.0.1:36948 - "POST /node/694077b1575bc060ce15b345 HTTP/1.0" 202 Accepted 2025-12-17 21:03:51,728 - lava_callback - INFO - Starting async job submit for node 694077b1575bc060ce15b345 2025-12-17 21:03:51,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:03:51,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b1575bc060ce15b345 HTTP/1.1" 502 150 2025-12-17 21:03:51,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b1575bc060ce15b345'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:03:51,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b1575bc060ce15b345 2025-12-17 21:03:51,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b1575bc060ce15b345 HTTP/1.1" 502 150 2025-12-17 21:03:51,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b1575bc060ce15b345'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:03:52,103 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e33575bc060ce15c250 2025-12-17 21:03:52,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e33575bc060ce15c250 HTTP/1.1" 502 150 2025-12-17 21:03:52,105 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e33575bc060ce15c250'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:03:52,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e32575bc060ce15c24b HTTP/1.1" 502 150 2025-12-17 21:03:52,240 - lava_callback - ERROR - Error processing callback for node 69407e32575bc060ce15c24b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e32575bc060ce15c24b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e32575bc060ce15c24b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e32575bc060ce15c24b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:03:52,496 - lava_callback - INFO - Received callback for node 694077b3575bc060ce15b34b INFO: 172.21.0.1:36964 - "POST /node/694077b3575bc060ce15b34b HTTP/1.0" 202 Accepted 2025-12-17 21:03:52,506 - lava_callback - INFO - Starting async job submit for node 694077b3575bc060ce15b34b 2025-12-17 21:03:52,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:03:52,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b3575bc060ce15b34b HTTP/1.1" 502 150 2025-12-17 21:03:52,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b3575bc060ce15b34b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:03:52,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b3575bc060ce15b34b 2025-12-17 21:03:52,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b3575bc060ce15b34b HTTP/1.1" 502 150 2025-12-17 21:03:52,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b3575bc060ce15b34b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:03:52,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b0575bc060ce15b32b 2025-12-17 21:03:52,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b0575bc060ce15b32b HTTP/1.1" 502 150 2025-12-17 21:03:52,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b0575bc060ce15b32b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:03:53,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b1575bc060ce15b345 2025-12-17 21:03:53,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b1575bc060ce15b345 HTTP/1.1" 502 150 2025-12-17 21:03:53,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b1575bc060ce15b345'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:03:54,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b3575bc060ce15b34b 2025-12-17 21:03:54,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b3575bc060ce15b34b HTTP/1.1" 502 150 2025-12-17 21:03:54,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b3575bc060ce15b34b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36970 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:03:56,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b0575bc060ce15b32b 2025-12-17 21:03:56,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b0575bc060ce15b32b HTTP/1.1" 502 150 2025-12-17 21:03:56,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b0575bc060ce15b32b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:03:57,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b1575bc060ce15b345 2025-12-17 21:03:57,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b1575bc060ce15b345 HTTP/1.1" 502 150 2025-12-17 21:03:57,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b1575bc060ce15b345'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:03:58,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b3575bc060ce15b34b 2025-12-17 21:03:58,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b3575bc060ce15b34b HTTP/1.1" 502 150 2025-12-17 21:03:58,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b3575bc060ce15b34b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:00,105 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e33575bc060ce15c250 INFO: 172.21.0.1:53258 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:04:04,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b0575bc060ce15b32b 2025-12-17 21:04:04,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b0575bc060ce15b32b HTTP/1.1" 502 150 2025-12-17 21:04:04,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b0575bc060ce15b32b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:05,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b1575bc060ce15b345 2025-12-17 21:04:05,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b1575bc060ce15b345 HTTP/1.1" 502 150 2025-12-17 21:04:05,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b1575bc060ce15b345'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53260 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:04:06,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b3575bc060ce15b34b 2025-12-17 21:04:06,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b3575bc060ce15b34b HTTP/1.1" 502 150 2025-12-17 21:04:06,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b3575bc060ce15b34b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:07,792 - lava_callback - INFO - Received callback for node 69407dd2575bc060ce15c192 INFO: 172.21.0.1:53266 - "POST /node/69407dd2575bc060ce15c192 HTTP/1.0" 202 Accepted 2025-12-17 21:04:07,803 - lava_callback - INFO - Starting async job submit for node 69407dd2575bc060ce15c192 2025-12-17 21:04:07,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:04:07,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dd2575bc060ce15c192 HTTP/1.1" 502 150 2025-12-17 21:04:07,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dd2575bc060ce15c192'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:07,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dd2575bc060ce15c192 2025-12-17 21:04:07,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dd2575bc060ce15c192 HTTP/1.1" 502 150 2025-12-17 21:04:07,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dd2575bc060ce15c192'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:09,667 - lava_callback - INFO - Received callback for node 694077b4575bc060ce15b34e INFO: 172.21.0.1:53272 - "POST /node/694077b4575bc060ce15b34e HTTP/1.0" 202 Accepted 2025-12-17 21:04:09,691 - lava_callback - INFO - Starting async job submit for node 694077b4575bc060ce15b34e 2025-12-17 21:04:09,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:04:09,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b4575bc060ce15b34e HTTP/1.1" 502 150 2025-12-17 21:04:09,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b4575bc060ce15b34e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:09,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b4575bc060ce15b34e 2025-12-17 21:04:09,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b4575bc060ce15b34e HTTP/1.1" 502 150 2025-12-17 21:04:09,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b4575bc060ce15b34e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:09,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dd2575bc060ce15c192 2025-12-17 21:04:09,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dd2575bc060ce15c192 HTTP/1.1" 502 150 2025-12-17 21:04:09,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dd2575bc060ce15c192'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:10,292 - lava_callback - INFO - Received callback for node 694077b6575bc060ce15b351 INFO: 172.21.0.1:56292 - "POST /node/694077b6575bc060ce15b351 HTTP/1.0" 202 Accepted 2025-12-17 21:04:10,298 - lava_callback - INFO - Starting async job submit for node 694077b6575bc060ce15b351 2025-12-17 21:04:10,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:04:10,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b6575bc060ce15b351 HTTP/1.1" 502 150 2025-12-17 21:04:10,326 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b6575bc060ce15b351'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:10,327 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b6575bc060ce15b351 2025-12-17 21:04:10,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b6575bc060ce15b351 HTTP/1.1" 502 150 2025-12-17 21:04:10,327 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b6575bc060ce15b351'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:10,816 - lava_callback - INFO - Received callback for node 69407e33575bc060ce15c255 INFO: 172.21.0.1:56294 - "POST /node/69407e33575bc060ce15c255 HTTP/1.0" 202 Accepted 2025-12-17 21:04:10,825 - lava_callback - INFO - Starting async job submit for node 69407e33575bc060ce15c255 2025-12-17 21:04:10,869 - lava_callback - INFO - Received callback for node 694077bb575bc060ce15b35d INFO: 172.21.0.1:56308 - "POST /node/694077bb575bc060ce15b35d HTTP/1.0" 202 Accepted 2025-12-17 21:04:10,887 - lava_callback - INFO - Starting async job submit for node 694077bb575bc060ce15b35d 2025-12-17 21:04:10,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:04:10,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:04:10,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bb575bc060ce15b35d HTTP/1.1" 502 150 2025-12-17 21:04:10,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077bb575bc060ce15b35d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:10,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077bb575bc060ce15b35d 2025-12-17 21:04:10,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bb575bc060ce15b35d HTTP/1.1" 502 150 2025-12-17 21:04:10,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077bb575bc060ce15b35d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:10,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e33575bc060ce15c255 HTTP/1.1" 502 150 2025-12-17 21:04:10,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e33575bc060ce15c255'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:10,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e33575bc060ce15c255 2025-12-17 21:04:10,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e33575bc060ce15c255 HTTP/1.1" 502 150 2025-12-17 21:04:10,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e33575bc060ce15c255'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:10,925 - lava_callback - INFO - Received callback for node 694077b8575bc060ce15b357 INFO: 172.21.0.1:56312 - "POST /node/694077b8575bc060ce15b357 HTTP/1.0" 202 Accepted 2025-12-17 21:04:10,935 - lava_callback - INFO - Starting async job submit for node 694077b8575bc060ce15b357 2025-12-17 21:04:10,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:04:10,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b8575bc060ce15b357 HTTP/1.1" 502 150 2025-12-17 21:04:10,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b8575bc060ce15b357'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:10,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b8575bc060ce15b357 2025-12-17 21:04:10,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b8575bc060ce15b357 HTTP/1.1" 502 150 2025-12-17 21:04:10,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b8575bc060ce15b357'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:11,725 - lava_callback - INFO - Received callback for node 694077bd575bc060ce15b360 2025-12-17 21:04:11,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b4575bc060ce15b34e INFO: 172.21.0.1:56326 - "POST /node/694077bd575bc060ce15b360 HTTP/1.0" 202 Accepted 2025-12-17 21:04:11,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b4575bc060ce15b34e HTTP/1.1" 502 150 2025-12-17 21:04:11,736 - lava_callback - INFO - Starting async job submit for node 694077bd575bc060ce15b360 2025-12-17 21:04:11,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b4575bc060ce15b34e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:11,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:04:11,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bd575bc060ce15b360 HTTP/1.1" 502 150 2025-12-17 21:04:11,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077bd575bc060ce15b360'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:11,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077bd575bc060ce15b360 2025-12-17 21:04:11,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bd575bc060ce15b360 HTTP/1.1" 502 150 2025-12-17 21:04:11,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077bd575bc060ce15b360'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:12,328 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b6575bc060ce15b351 2025-12-17 21:04:12,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b6575bc060ce15b351 HTTP/1.1" 502 150 2025-12-17 21:04:12,329 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b6575bc060ce15b351'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:12,761 - lava_callback - INFO - Received callback for node 694077be575bc060ce15b363 INFO: 172.21.0.1:56338 - "POST /node/694077be575bc060ce15b363 HTTP/1.0" 202 Accepted 2025-12-17 21:04:12,767 - lava_callback - INFO - Starting async job submit for node 694077be575bc060ce15b363 2025-12-17 21:04:12,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:04:12,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077be575bc060ce15b363 HTTP/1.1" 502 150 2025-12-17 21:04:12,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077be575bc060ce15b363'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:12,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077be575bc060ce15b363 2025-12-17 21:04:12,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077be575bc060ce15b363 HTTP/1.1" 502 150 2025-12-17 21:04:12,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077be575bc060ce15b363'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:12,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077bb575bc060ce15b35d 2025-12-17 21:04:12,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bb575bc060ce15b35d HTTP/1.1" 502 150 2025-12-17 21:04:12,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077bb575bc060ce15b35d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:12,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e33575bc060ce15c255 2025-12-17 21:04:12,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e33575bc060ce15c255 HTTP/1.1" 502 150 2025-12-17 21:04:12,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e33575bc060ce15c255'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:12,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b8575bc060ce15b357 2025-12-17 21:04:12,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b8575bc060ce15b357 HTTP/1.1" 502 150 2025-12-17 21:04:12,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b8575bc060ce15b357'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:13,527 - lava_callback - INFO - Received callback for node 694077bf575bc060ce15b366 INFO: 172.21.0.1:56344 - "POST /node/694077bf575bc060ce15b366 HTTP/1.0" 202 Accepted 2025-12-17 21:04:13,533 - lava_callback - INFO - Starting async job submit for node 694077bf575bc060ce15b366 2025-12-17 21:04:13,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:04:13,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077bd575bc060ce15b360 2025-12-17 21:04:13,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bd575bc060ce15b360 HTTP/1.1" 502 150 2025-12-17 21:04:13,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077bd575bc060ce15b360'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:13,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dd2575bc060ce15c192 2025-12-17 21:04:13,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dd2575bc060ce15c192 HTTP/1.1" 502 150 2025-12-17 21:04:13,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dd2575bc060ce15c192'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:14,342 - lava_callback - INFO - Received callback for node 694077c0575bc060ce15b369 INFO: 172.21.0.1:56352 - "POST /node/694077c0575bc060ce15b369 HTTP/1.0" 202 Accepted 2025-12-17 21:04:14,348 - lava_callback - INFO - Starting async job submit for node 694077c0575bc060ce15b369 2025-12-17 21:04:14,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:04:14,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077c0575bc060ce15b369 HTTP/1.1" 502 150 2025-12-17 21:04:14,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077c0575bc060ce15b369'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:14,355 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077c0575bc060ce15b369 2025-12-17 21:04:14,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077c0575bc060ce15b369 HTTP/1.1" 502 150 2025-12-17 21:04:14,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077c0575bc060ce15b369'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:14,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077be575bc060ce15b363 2025-12-17 21:04:14,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077be575bc060ce15b363 HTTP/1.1" 502 150 2025-12-17 21:04:14,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077be575bc060ce15b363'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:15,139 - lava_callback - INFO - Received callback for node 694077c1575bc060ce15b36c INFO: 172.21.0.1:56356 - "POST /node/694077c1575bc060ce15b36c HTTP/1.0" 202 Accepted 2025-12-17 21:04:15,149 - lava_callback - INFO - Starting async job submit for node 694077c1575bc060ce15b36c 2025-12-17 21:04:15,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:56364 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:04:15,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e33575bc060ce15c250 HTTP/1.1" 502 150 2025-12-17 21:04:15,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e33575bc060ce15c250'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:15,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b4575bc060ce15b34e 2025-12-17 21:04:15,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b4575bc060ce15b34e HTTP/1.1" 502 150 2025-12-17 21:04:15,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b4575bc060ce15b34e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:16,329 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b6575bc060ce15b351 2025-12-17 21:04:16,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b6575bc060ce15b351 HTTP/1.1" 502 150 2025-12-17 21:04:16,330 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b6575bc060ce15b351'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:16,356 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077c0575bc060ce15b369 2025-12-17 21:04:16,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077c0575bc060ce15b369 HTTP/1.1" 502 150 2025-12-17 21:04:16,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077c0575bc060ce15b369'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56366 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:04:16,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077bb575bc060ce15b35d 2025-12-17 21:04:16,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bb575bc060ce15b35d HTTP/1.1" 502 150 2025-12-17 21:04:16,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e33575bc060ce15c255 2025-12-17 21:04:16,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077bb575bc060ce15b35d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:16,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e33575bc060ce15c255 HTTP/1.1" 502 150 2025-12-17 21:04:16,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e33575bc060ce15c255'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:16,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b8575bc060ce15b357 2025-12-17 21:04:16,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b8575bc060ce15b357 HTTP/1.1" 502 150 2025-12-17 21:04:16,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b8575bc060ce15b357'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:17,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077bd575bc060ce15b360 2025-12-17 21:04:17,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bd575bc060ce15b360 HTTP/1.1" 502 150 2025-12-17 21:04:17,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077bd575bc060ce15b360'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:18,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077be575bc060ce15b363 2025-12-17 21:04:18,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077be575bc060ce15b363 HTTP/1.1" 502 150 2025-12-17 21:04:18,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077be575bc060ce15b363'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:20,357 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077c0575bc060ce15b369 2025-12-17 21:04:20,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077c0575bc060ce15b369 HTTP/1.1" 502 150 2025-12-17 21:04:20,358 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077c0575bc060ce15b369'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:20,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b0575bc060ce15b32b 2025-12-17 21:04:20,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b0575bc060ce15b32b HTTP/1.1" 502 150 2025-12-17 21:04:20,923 - lava_callback - ERROR - Error processing callback for node 694077b0575bc060ce15b32b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b0575bc060ce15b32b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b0575bc060ce15b32b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b0575bc060ce15b32b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:04:21,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b1575bc060ce15b345 2025-12-17 21:04:21,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b1575bc060ce15b345 HTTP/1.1" 502 150 2025-12-17 21:04:21,744 - lava_callback - ERROR - Error processing callback for node 694077b1575bc060ce15b345: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b1575bc060ce15b345 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b1575bc060ce15b345 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b1575bc060ce15b345 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:04:21,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dd2575bc060ce15c192 2025-12-17 21:04:21,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dd2575bc060ce15c192 HTTP/1.1" 502 150 2025-12-17 21:04:21,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dd2575bc060ce15c192'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:22,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b3575bc060ce15b34b 2025-12-17 21:04:22,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b3575bc060ce15b34b HTTP/1.1" 502 150 2025-12-17 21:04:22,535 - lava_callback - ERROR - Error processing callback for node 694077b3575bc060ce15b34b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b3575bc060ce15b34b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b3575bc060ce15b34b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b3575bc060ce15b34b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:04:23,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b4575bc060ce15b34e 2025-12-17 21:04:23,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b4575bc060ce15b34e HTTP/1.1" 502 150 2025-12-17 21:04:23,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b4575bc060ce15b34e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:24,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b6575bc060ce15b351 2025-12-17 21:04:24,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b6575bc060ce15b351 HTTP/1.1" 502 150 2025-12-17 21:04:24,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b6575bc060ce15b351'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:24,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077bb575bc060ce15b35d 2025-12-17 21:04:24,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bb575bc060ce15b35d HTTP/1.1" 502 150 2025-12-17 21:04:24,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077bb575bc060ce15b35d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:24,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e33575bc060ce15c255 2025-12-17 21:04:24,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e33575bc060ce15c255 HTTP/1.1" 502 150 2025-12-17 21:04:24,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e33575bc060ce15c255'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:24,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b8575bc060ce15b357 2025-12-17 21:04:24,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b8575bc060ce15b357 HTTP/1.1" 502 150 2025-12-17 21:04:24,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b8575bc060ce15b357'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:25,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077bd575bc060ce15b360 2025-12-17 21:04:25,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bd575bc060ce15b360 HTTP/1.1" 502 150 2025-12-17 21:04:25,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077bd575bc060ce15b360'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38214 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:04:26,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077be575bc060ce15b363 2025-12-17 21:04:26,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077be575bc060ce15b363 HTTP/1.1" 502 150 2025-12-17 21:04:26,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077be575bc060ce15b363'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:28,358 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077c0575bc060ce15b369 2025-12-17 21:04:28,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077c0575bc060ce15b369 HTTP/1.1" 502 150 2025-12-17 21:04:28,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077c0575bc060ce15b369'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:28,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bf575bc060ce15b366 HTTP/1.1" 502 150 2025-12-17 21:04:28,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077bf575bc060ce15b366'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:28,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077bf575bc060ce15b366 2025-12-17 21:04:28,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bf575bc060ce15b366 HTTP/1.1" 502 150 2025-12-17 21:04:28,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077bf575bc060ce15b366'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:30,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077c1575bc060ce15b36c HTTP/1.1" 502 150 2025-12-17 21:04:30,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077c1575bc060ce15b36c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:30,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077c1575bc060ce15b36c 2025-12-17 21:04:30,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077c1575bc060ce15b36c HTTP/1.1" 502 150 2025-12-17 21:04:30,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077c1575bc060ce15b36c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:30,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077bf575bc060ce15b366 2025-12-17 21:04:30,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bf575bc060ce15b366 HTTP/1.1" 502 150 2025-12-17 21:04:30,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077bf575bc060ce15b366'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:31,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e33575bc060ce15c250 2025-12-17 21:04:31,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e33575bc060ce15c250 HTTP/1.1" 502 150 2025-12-17 21:04:31,542 - lava_callback - ERROR - Error processing callback for node 69407e33575bc060ce15c250: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e33575bc060ce15c250 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e33575bc060ce15c250 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e33575bc060ce15c250 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:04:32,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077c1575bc060ce15b36c 2025-12-17 21:04:32,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077c1575bc060ce15b36c HTTP/1.1" 502 150 2025-12-17 21:04:32,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077c1575bc060ce15b36c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:34,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077bf575bc060ce15b366 2025-12-17 21:04:34,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bf575bc060ce15b366 HTTP/1.1" 502 150 2025-12-17 21:04:34,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077bf575bc060ce15b366'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37690 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:04:36,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077c1575bc060ce15b36c 2025-12-17 21:04:36,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077c1575bc060ce15b36c HTTP/1.1" 502 150 2025-12-17 21:04:36,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077c1575bc060ce15b36c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:37,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dd2575bc060ce15c192 2025-12-17 21:04:37,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dd2575bc060ce15c192 HTTP/1.1" 502 150 2025-12-17 21:04:37,847 - lava_callback - ERROR - Error processing callback for node 69407dd2575bc060ce15c192: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dd2575bc060ce15c192 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dd2575bc060ce15c192 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dd2575bc060ce15c192 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:04:39,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b4575bc060ce15b34e 2025-12-17 21:04:39,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b4575bc060ce15b34e HTTP/1.1" 502 150 2025-12-17 21:04:39,741 - lava_callback - ERROR - Error processing callback for node 694077b4575bc060ce15b34e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b4575bc060ce15b34e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b4575bc060ce15b34e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b4575bc060ce15b34e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:04:40,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b6575bc060ce15b351 2025-12-17 21:04:40,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b6575bc060ce15b351 HTTP/1.1" 502 150 2025-12-17 21:04:40,334 - lava_callback - ERROR - Error processing callback for node 694077b6575bc060ce15b351: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b6575bc060ce15b351 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b6575bc060ce15b351 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b6575bc060ce15b351 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:04:40,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077bb575bc060ce15b35d 2025-12-17 21:04:40,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e33575bc060ce15c255 2025-12-17 21:04:40,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bb575bc060ce15b35d HTTP/1.1" 502 150 2025-12-17 21:04:40,912 - lava_callback - ERROR - Error processing callback for node 694077bb575bc060ce15b35d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077bb575bc060ce15b35d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077bb575bc060ce15b35d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077bb575bc060ce15b35d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:04:40,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e33575bc060ce15c255 HTTP/1.1" 502 150 2025-12-17 21:04:40,914 - lava_callback - ERROR - Error processing callback for node 69407e33575bc060ce15c255: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e33575bc060ce15c255 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e33575bc060ce15c255 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e33575bc060ce15c255 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:04:40,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b8575bc060ce15b357 2025-12-17 21:04:40,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b8575bc060ce15b357 HTTP/1.1" 502 150 2025-12-17 21:04:40,971 - lava_callback - ERROR - Error processing callback for node 694077b8575bc060ce15b357: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b8575bc060ce15b357 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b8575bc060ce15b357 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b8575bc060ce15b357 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:04:41,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077bd575bc060ce15b360 2025-12-17 21:04:41,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bd575bc060ce15b360 HTTP/1.1" 502 150 2025-12-17 21:04:41,751 - lava_callback - ERROR - Error processing callback for node 694077bd575bc060ce15b360: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077bd575bc060ce15b360 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077bd575bc060ce15b360 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077bd575bc060ce15b360 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:04:42,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077be575bc060ce15b363 2025-12-17 21:04:42,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077be575bc060ce15b363 HTTP/1.1" 502 150 2025-12-17 21:04:42,780 - lava_callback - ERROR - Error processing callback for node 694077be575bc060ce15b363: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077be575bc060ce15b363 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077be575bc060ce15b363 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077be575bc060ce15b363 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:04:42,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077bf575bc060ce15b366 2025-12-17 21:04:42,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bf575bc060ce15b366 HTTP/1.1" 502 150 2025-12-17 21:04:42,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077bf575bc060ce15b366'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:44,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077c0575bc060ce15b369 2025-12-17 21:04:44,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077c0575bc060ce15b369 HTTP/1.1" 502 150 2025-12-17 21:04:44,361 - lava_callback - ERROR - Error processing callback for node 694077c0575bc060ce15b369: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077c0575bc060ce15b369 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077c0575bc060ce15b369 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077c0575bc060ce15b369 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:04:44,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077c1575bc060ce15b36c 2025-12-17 21:04:44,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077c1575bc060ce15b36c HTTP/1.1" 502 150 2025-12-17 21:04:44,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077c1575bc060ce15b36c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35628 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:04:50,806 - lava_callback - INFO - Received callback for node 694077ee575bc060ce15b3a7 INFO: 172.21.0.1:51724 - "POST /node/694077ee575bc060ce15b3a7 HTTP/1.0" 202 Accepted 2025-12-17 21:04:50,811 - lava_callback - INFO - Starting async job submit for node 694077ee575bc060ce15b3a7 2025-12-17 21:04:50,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:04:50,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ee575bc060ce15b3a7 HTTP/1.1" 502 150 2025-12-17 21:04:50,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077ee575bc060ce15b3a7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:50,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077ee575bc060ce15b3a7 2025-12-17 21:04:50,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ee575bc060ce15b3a7 HTTP/1.1" 502 150 2025-12-17 21:04:50,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077ee575bc060ce15b3a7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:51,574 - lava_callback - INFO - Received callback for node 694077f0575bc060ce15b3b9 INFO: 172.21.0.1:51728 - "POST /node/694077f0575bc060ce15b3b9 HTTP/1.0" 202 Accepted 2025-12-17 21:04:51,580 - lava_callback - INFO - Starting async job submit for node 694077f0575bc060ce15b3b9 2025-12-17 21:04:51,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:04:51,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f0575bc060ce15b3b9 HTTP/1.1" 502 150 2025-12-17 21:04:51,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f0575bc060ce15b3b9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:51,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f0575bc060ce15b3b9 2025-12-17 21:04:51,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f0575bc060ce15b3b9 HTTP/1.1" 502 150 2025-12-17 21:04:51,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f0575bc060ce15b3b9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:52,315 - lava_callback - INFO - Received callback for node 694077f1575bc060ce15b3be INFO: 172.21.0.1:51744 - "POST /node/694077f1575bc060ce15b3be HTTP/1.0" 202 Accepted 2025-12-17 21:04:52,320 - lava_callback - INFO - Starting async job submit for node 694077f1575bc060ce15b3be 2025-12-17 21:04:52,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:04:52,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077ee575bc060ce15b3a7 2025-12-17 21:04:53,087 - lava_callback - INFO - Received callback for node 694077f2575bc060ce15b3c2 INFO: 172.21.0.1:51758 - "POST /node/694077f2575bc060ce15b3c2 HTTP/1.0" 202 Accepted 2025-12-17 21:04:53,093 - lava_callback - INFO - Starting async job submit for node 694077f2575bc060ce15b3c2 2025-12-17 21:04:53,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:04:53,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f2575bc060ce15b3c2 HTTP/1.1" 502 150 2025-12-17 21:04:53,099 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f2575bc060ce15b3c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:53,099 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f2575bc060ce15b3c2 2025-12-17 21:04:53,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f2575bc060ce15b3c2 HTTP/1.1" 502 150 2025-12-17 21:04:53,100 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f2575bc060ce15b3c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:53,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f0575bc060ce15b3b9 2025-12-17 21:04:53,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f0575bc060ce15b3b9 HTTP/1.1" 502 150 2025-12-17 21:04:53,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f0575bc060ce15b3b9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:53,857 - lava_callback - INFO - Received callback for node 694077f4575bc060ce15b3c5 INFO: 172.21.0.1:51770 - "POST /node/694077f4575bc060ce15b3c5 HTTP/1.0" 202 Accepted 2025-12-17 21:04:53,863 - lava_callback - INFO - Starting async job submit for node 694077f4575bc060ce15b3c5 2025-12-17 21:04:53,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:04:53,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f4575bc060ce15b3c5 HTTP/1.1" 502 150 2025-12-17 21:04:53,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f4575bc060ce15b3c5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:53,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f4575bc060ce15b3c5 2025-12-17 21:04:53,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f4575bc060ce15b3c5 HTTP/1.1" 502 150 2025-12-17 21:04:53,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f4575bc060ce15b3c5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:54,616 - lava_callback - INFO - Received callback for node 694077f5575bc060ce15b3c8 INFO: 172.21.0.1:51784 - "POST /node/694077f5575bc060ce15b3c8 HTTP/1.0" 202 Accepted 2025-12-17 21:04:54,622 - lava_callback - INFO - Starting async job submit for node 694077f5575bc060ce15b3c8 2025-12-17 21:04:54,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:04:54,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f5575bc060ce15b3c8 HTTP/1.1" 502 150 2025-12-17 21:04:54,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f5575bc060ce15b3c8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:54,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f5575bc060ce15b3c8 2025-12-17 21:04:54,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f5575bc060ce15b3c8 HTTP/1.1" 502 150 2025-12-17 21:04:54,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f5575bc060ce15b3c8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:55,100 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f2575bc060ce15b3c2 2025-12-17 21:04:55,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f2575bc060ce15b3c2 HTTP/1.1" 502 150 2025-12-17 21:04:55,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f2575bc060ce15b3c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:55,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f4575bc060ce15b3c5 2025-12-17 21:04:55,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f4575bc060ce15b3c5 HTTP/1.1" 502 150 2025-12-17 21:04:55,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f4575bc060ce15b3c5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51800 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:04:56,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f5575bc060ce15b3c8 2025-12-17 21:04:56,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f5575bc060ce15b3c8 HTTP/1.1" 502 150 2025-12-17 21:04:56,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f5575bc060ce15b3c8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:57,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f0575bc060ce15b3b9 2025-12-17 21:04:57,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f0575bc060ce15b3b9 HTTP/1.1" 502 150 2025-12-17 21:04:57,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f0575bc060ce15b3b9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:58,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077bf575bc060ce15b366 2025-12-17 21:04:58,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077bf575bc060ce15b366 HTTP/1.1" 502 150 2025-12-17 21:04:58,861 - lava_callback - ERROR - Error processing callback for node 694077bf575bc060ce15b366: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077bf575bc060ce15b366 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077bf575bc060ce15b366 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077bf575bc060ce15b366 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:04:59,101 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f2575bc060ce15b3c2 2025-12-17 21:04:59,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f2575bc060ce15b3c2 HTTP/1.1" 502 150 2025-12-17 21:04:59,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f2575bc060ce15b3c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:04:59,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f4575bc060ce15b3c5 2025-12-17 21:04:59,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f4575bc060ce15b3c5 HTTP/1.1" 502 150 2025-12-17 21:04:59,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f4575bc060ce15b3c5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:00,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f5575bc060ce15b3c8 2025-12-17 21:05:00,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f5575bc060ce15b3c8 HTTP/1.1" 502 150 2025-12-17 21:05:00,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f5575bc060ce15b3c8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:00,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077c1575bc060ce15b36c 2025-12-17 21:05:00,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077c1575bc060ce15b36c HTTP/1.1" 502 150 2025-12-17 21:05:00,647 - lava_callback - ERROR - Error processing callback for node 694077c1575bc060ce15b36c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077c1575bc060ce15b36c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077c1575bc060ce15b36c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077c1575bc060ce15b36c (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:56934 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:05:05,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f0575bc060ce15b3b9 2025-12-17 21:05:05,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f0575bc060ce15b3b9 HTTP/1.1" 502 150 2025-12-17 21:05:05,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f0575bc060ce15b3b9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56946 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:05:07,103 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f2575bc060ce15b3c2 2025-12-17 21:05:07,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f1575bc060ce15b3be HTTP/1.1" 502 150 2025-12-17 21:05:07,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f1575bc060ce15b3be'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:07,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f1575bc060ce15b3be 2025-12-17 21:05:07,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f1575bc060ce15b3be HTTP/1.1" 502 150 2025-12-17 21:05:07,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f1575bc060ce15b3be'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:07,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f4575bc060ce15b3c5 2025-12-17 21:05:07,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f4575bc060ce15b3c5 HTTP/1.1" 502 150 2025-12-17 21:05:07,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f4575bc060ce15b3c5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:08,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ee575bc060ce15b3a7 HTTP/1.1" 502 150 2025-12-17 21:05:08,275 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077ee575bc060ce15b3a7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:08,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f5575bc060ce15b3c8 2025-12-17 21:05:08,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f5575bc060ce15b3c8 HTTP/1.1" 502 150 2025-12-17 21:05:08,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f5575bc060ce15b3c8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:09,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f1575bc060ce15b3be 2025-12-17 21:05:09,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f1575bc060ce15b3be HTTP/1.1" 502 150 2025-12-17 21:05:09,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f1575bc060ce15b3be'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:10,820 - lava_callback - INFO - Received callback for node 694077f6575bc060ce15b3cb INFO: 172.21.0.1:55856 - "POST /node/694077f6575bc060ce15b3cb HTTP/1.0" 202 Accepted 2025-12-17 21:05:10,826 - lava_callback - INFO - Starting async job submit for node 694077f6575bc060ce15b3cb 2025-12-17 21:05:10,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:10,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f6575bc060ce15b3cb HTTP/1.1" 502 150 2025-12-17 21:05:10,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f6575bc060ce15b3cb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:10,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f6575bc060ce15b3cb 2025-12-17 21:05:10,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f6575bc060ce15b3cb HTTP/1.1" 502 150 2025-12-17 21:05:10,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f6575bc060ce15b3cb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:11,645 - lava_callback - INFO - Received callback for node 694077f7575bc060ce15b3ce INFO: 172.21.0.1:55862 - "POST /node/694077f7575bc060ce15b3ce HTTP/1.0" 202 Accepted 2025-12-17 21:05:11,651 - lava_callback - INFO - Starting async job submit for node 694077f7575bc060ce15b3ce 2025-12-17 21:05:11,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:11,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f7575bc060ce15b3ce HTTP/1.1" 502 150 2025-12-17 21:05:11,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f7575bc060ce15b3ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:11,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f7575bc060ce15b3ce 2025-12-17 21:05:11,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f7575bc060ce15b3ce HTTP/1.1" 502 150 2025-12-17 21:05:11,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f7575bc060ce15b3ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:12,275 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077ee575bc060ce15b3a7 2025-12-17 21:05:12,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ee575bc060ce15b3a7 HTTP/1.1" 502 150 2025-12-17 21:05:12,276 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077ee575bc060ce15b3a7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:12,407 - lava_callback - INFO - Received callback for node 694077f9575bc060ce15b3d1 INFO: 172.21.0.1:55876 - "POST /node/694077f9575bc060ce15b3d1 HTTP/1.0" 202 Accepted 2025-12-17 21:05:12,416 - lava_callback - INFO - Starting async job submit for node 694077f9575bc060ce15b3d1 2025-12-17 21:05:12,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:12,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f9575bc060ce15b3d1 HTTP/1.1" 502 150 2025-12-17 21:05:12,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f9575bc060ce15b3d1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:12,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f9575bc060ce15b3d1 2025-12-17 21:05:12,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f9575bc060ce15b3d1 HTTP/1.1" 502 150 2025-12-17 21:05:12,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f9575bc060ce15b3d1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:12,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f6575bc060ce15b3cb 2025-12-17 21:05:12,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f6575bc060ce15b3cb HTTP/1.1" 502 150 2025-12-17 21:05:12,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f6575bc060ce15b3cb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:13,175 - lava_callback - INFO - Received callback for node 694077fa575bc060ce15b3d4 INFO: 172.21.0.1:55888 - "POST /node/694077fa575bc060ce15b3d4 HTTP/1.0" 202 Accepted 2025-12-17 21:05:13,184 - lava_callback - INFO - Starting async job submit for node 694077fa575bc060ce15b3d4 2025-12-17 21:05:13,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:13,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fa575bc060ce15b3d4 HTTP/1.1" 502 150 2025-12-17 21:05:13,214 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fa575bc060ce15b3d4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:13,214 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fa575bc060ce15b3d4 2025-12-17 21:05:13,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fa575bc060ce15b3d4 HTTP/1.1" 502 150 2025-12-17 21:05:13,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fa575bc060ce15b3d4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:13,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f7575bc060ce15b3ce 2025-12-17 21:05:13,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f7575bc060ce15b3ce HTTP/1.1" 502 150 2025-12-17 21:05:13,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f7575bc060ce15b3ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:13,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f1575bc060ce15b3be 2025-12-17 21:05:13,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f1575bc060ce15b3be HTTP/1.1" 502 150 2025-12-17 21:05:13,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f1575bc060ce15b3be'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:13,917 - lava_callback - INFO - Received callback for node 694077fb575bc060ce15b3d7 INFO: 172.21.0.1:55902 - "POST /node/694077fb575bc060ce15b3d7 HTTP/1.0" 202 Accepted 2025-12-17 21:05:13,927 - lava_callback - INFO - Starting async job submit for node 694077fb575bc060ce15b3d7 2025-12-17 21:05:13,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:13,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fb575bc060ce15b3d7 HTTP/1.1" 502 150 2025-12-17 21:05:13,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fb575bc060ce15b3d7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:13,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fb575bc060ce15b3d7 2025-12-17 21:05:13,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fb575bc060ce15b3d7 HTTP/1.1" 502 150 2025-12-17 21:05:13,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fb575bc060ce15b3d7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:14,427 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f9575bc060ce15b3d1 2025-12-17 21:05:14,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f9575bc060ce15b3d1 HTTP/1.1" 502 150 2025-12-17 21:05:14,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f9575bc060ce15b3d1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:14,673 - lava_callback - INFO - Received callback for node 694077fd575bc060ce15b3da INFO: 172.21.0.1:55918 - "POST /node/694077fd575bc060ce15b3da HTTP/1.0" 202 Accepted 2025-12-17 21:05:14,680 - lava_callback - INFO - Starting async job submit for node 694077fd575bc060ce15b3da 2025-12-17 21:05:14,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:14,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fd575bc060ce15b3da HTTP/1.1" 502 150 2025-12-17 21:05:14,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fd575bc060ce15b3da'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:14,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fd575bc060ce15b3da 2025-12-17 21:05:14,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fd575bc060ce15b3da HTTP/1.1" 502 150 2025-12-17 21:05:14,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fd575bc060ce15b3da'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55930 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:05:15,215 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fa575bc060ce15b3d4 2025-12-17 21:05:15,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fa575bc060ce15b3d4 HTTP/1.1" 502 150 2025-12-17 21:05:15,225 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fa575bc060ce15b3d4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:15,433 - lava_callback - INFO - Received callback for node 694077fe575bc060ce15b3dd INFO: 172.21.0.1:55936 - "POST /node/694077fe575bc060ce15b3dd HTTP/1.0" 202 Accepted 2025-12-17 21:05:15,443 - lava_callback - INFO - Starting async job submit for node 694077fe575bc060ce15b3dd 2025-12-17 21:05:15,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:15,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fe575bc060ce15b3dd HTTP/1.1" 502 150 2025-12-17 21:05:15,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fe575bc060ce15b3dd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:15,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fe575bc060ce15b3dd 2025-12-17 21:05:15,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fe575bc060ce15b3dd HTTP/1.1" 502 150 2025-12-17 21:05:15,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fe575bc060ce15b3dd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:15,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fb575bc060ce15b3d7 2025-12-17 21:05:15,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fb575bc060ce15b3d7 HTTP/1.1" 502 150 2025-12-17 21:05:15,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fb575bc060ce15b3d7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:16,174 - lava_callback - INFO - Received callback for node 694077ff575bc060ce15b3e0 INFO: 172.21.0.1:55942 - "POST /node/694077ff575bc060ce15b3e0 HTTP/1.0" 202 Accepted 2025-12-17 21:05:16,184 - lava_callback - INFO - Starting async job submit for node 694077ff575bc060ce15b3e0 2025-12-17 21:05:16,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:16,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ff575bc060ce15b3e0 HTTP/1.1" 502 150 2025-12-17 21:05:16,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077ff575bc060ce15b3e0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:16,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077ff575bc060ce15b3e0 2025-12-17 21:05:16,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ff575bc060ce15b3e0 HTTP/1.1" 502 150 2025-12-17 21:05:16,197 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077ff575bc060ce15b3e0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55946 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:05:16,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fd575bc060ce15b3da 2025-12-17 21:05:16,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fd575bc060ce15b3da HTTP/1.1" 502 150 2025-12-17 21:05:16,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fd575bc060ce15b3da'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:16,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f6575bc060ce15b3cb 2025-12-17 21:05:16,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f6575bc060ce15b3cb HTTP/1.1" 502 150 2025-12-17 21:05:16,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f6575bc060ce15b3cb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:16,922 - lava_callback - INFO - Received callback for node 69407800575bc060ce15b3e3 INFO: 172.21.0.1:55954 - "POST /node/69407800575bc060ce15b3e3 HTTP/1.0" 202 Accepted 2025-12-17 21:05:16,929 - lava_callback - INFO - Starting async job submit for node 69407800575bc060ce15b3e3 2025-12-17 21:05:16,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:16,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407800575bc060ce15b3e3 HTTP/1.1" 502 150 2025-12-17 21:05:16,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407800575bc060ce15b3e3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:16,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407800575bc060ce15b3e3 2025-12-17 21:05:16,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407800575bc060ce15b3e3 HTTP/1.1" 502 150 2025-12-17 21:05:16,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407800575bc060ce15b3e3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:17,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fe575bc060ce15b3dd 2025-12-17 21:05:17,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fe575bc060ce15b3dd HTTP/1.1" 502 150 2025-12-17 21:05:17,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fe575bc060ce15b3dd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:17,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f7575bc060ce15b3ce 2025-12-17 21:05:17,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f7575bc060ce15b3ce HTTP/1.1" 502 150 2025-12-17 21:05:17,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f7575bc060ce15b3ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:17,708 - lava_callback - INFO - Received callback for node 69407802575bc060ce15b3e6 INFO: 172.21.0.1:55964 - "POST /node/69407802575bc060ce15b3e6 HTTP/1.0" 202 Accepted 2025-12-17 21:05:17,715 - lava_callback - INFO - Starting async job submit for node 69407802575bc060ce15b3e6 2025-12-17 21:05:17,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:17,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407802575bc060ce15b3e6 HTTP/1.1" 502 150 2025-12-17 21:05:17,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407802575bc060ce15b3e6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:17,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407802575bc060ce15b3e6 2025-12-17 21:05:17,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407802575bc060ce15b3e6 HTTP/1.1" 502 150 2025-12-17 21:05:17,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407802575bc060ce15b3e6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:18,197 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077ff575bc060ce15b3e0 2025-12-17 21:05:18,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f9575bc060ce15b3d1 2025-12-17 21:05:18,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f9575bc060ce15b3d1 HTTP/1.1" 502 150 2025-12-17 21:05:18,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f9575bc060ce15b3d1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:18,530 - lava_callback - INFO - Received callback for node 69407803575bc060ce15b3eb INFO: 172.21.0.1:55972 - "POST /node/69407803575bc060ce15b3eb HTTP/1.0" 202 Accepted 2025-12-17 21:05:18,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407800575bc060ce15b3e3 2025-12-17 21:05:18,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407800575bc060ce15b3e3 HTTP/1.1" 502 150 2025-12-17 21:05:18,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407800575bc060ce15b3e3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:19,225 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fa575bc060ce15b3d4 2025-12-17 21:05:19,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fa575bc060ce15b3d4 HTTP/1.1" 502 150 2025-12-17 21:05:19,226 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fa575bc060ce15b3d4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:19,281 - lava_callback - INFO - Received callback for node 69407804575bc060ce15b3ee INFO: 172.21.0.1:55978 - "POST /node/69407804575bc060ce15b3ee HTTP/1.0" 202 Accepted 2025-12-17 21:05:19,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407802575bc060ce15b3e6 2025-12-17 21:05:19,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407802575bc060ce15b3e6 HTTP/1.1" 502 150 2025-12-17 21:05:19,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407802575bc060ce15b3e6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:19,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fb575bc060ce15b3d7 2025-12-17 21:05:19,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fb575bc060ce15b3d7 HTTP/1.1" 502 150 2025-12-17 21:05:19,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fb575bc060ce15b3d7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:20,184 - lava_callback - INFO - Received callback for node 69407806575bc060ce15b3f1 INFO: 172.21.0.1:50706 - "POST /node/69407806575bc060ce15b3f1 HTTP/1.0" 202 Accepted 2025-12-17 21:05:20,277 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077ee575bc060ce15b3a7 2025-12-17 21:05:20,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ee575bc060ce15b3a7 HTTP/1.1" 502 150 2025-12-17 21:05:20,278 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077ee575bc060ce15b3a7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:20,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fd575bc060ce15b3da 2025-12-17 21:05:20,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fd575bc060ce15b3da HTTP/1.1" 502 150 2025-12-17 21:05:20,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fd575bc060ce15b3da'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:20,932 - lava_callback - INFO - Received callback for node 69407807575bc060ce15b3f4 INFO: 172.21.0.1:50718 - "POST /node/69407807575bc060ce15b3f4 HTTP/1.0" 202 Accepted 2025-12-17 21:05:21,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fe575bc060ce15b3dd 2025-12-17 21:05:21,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fe575bc060ce15b3dd HTTP/1.1" 502 150 2025-12-17 21:05:21,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fe575bc060ce15b3dd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:21,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f0575bc060ce15b3b9 2025-12-17 21:05:21,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f0575bc060ce15b3b9 HTTP/1.1" 502 150 2025-12-17 21:05:21,596 - lava_callback - ERROR - Error processing callback for node 694077f0575bc060ce15b3b9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f0575bc060ce15b3b9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f0575bc060ce15b3b9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f0575bc060ce15b3b9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:21,598 - lava_callback - INFO - Starting async job submit for node 69407803575bc060ce15b3eb 2025-12-17 21:05:21,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:21,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407803575bc060ce15b3eb HTTP/1.1" 502 150 2025-12-17 21:05:21,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407803575bc060ce15b3eb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:21,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407803575bc060ce15b3eb 2025-12-17 21:05:21,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407803575bc060ce15b3eb HTTP/1.1" 502 150 2025-12-17 21:05:21,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407803575bc060ce15b3eb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:21,676 - lava_callback - INFO - Received callback for node 69407808575bc060ce15b3f7 INFO: 172.21.0.1:50728 - "POST /node/69407808575bc060ce15b3f7 HTTP/1.0" 202 Accepted 2025-12-17 21:05:21,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f1575bc060ce15b3be 2025-12-17 21:05:21,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f1575bc060ce15b3be HTTP/1.1" 502 150 2025-12-17 21:05:21,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f1575bc060ce15b3be'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:22,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f2575bc060ce15b3c2 HTTP/1.1" 502 150 2025-12-17 21:05:22,352 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f2575bc060ce15b3c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:22,425 - lava_callback - INFO - Received callback for node 69407809575bc060ce15b3fa INFO: 172.21.0.1:50742 - "POST /node/69407809575bc060ce15b3fa HTTP/1.0" 202 Accepted 2025-12-17 21:05:22,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407800575bc060ce15b3e3 2025-12-17 21:05:22,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407800575bc060ce15b3e3 HTTP/1.1" 502 150 2025-12-17 21:05:22,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407800575bc060ce15b3e3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:23,223 - lava_callback - INFO - Received callback for node 6940780d575bc060ce15b400 INFO: 172.21.0.1:50750 - "POST /node/6940780d575bc060ce15b400 HTTP/1.0" 202 Accepted 2025-12-17 21:05:23,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407803575bc060ce15b3eb 2025-12-17 21:05:23,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407803575bc060ce15b3eb HTTP/1.1" 502 150 2025-12-17 21:05:23,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407803575bc060ce15b3eb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:23,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407802575bc060ce15b3e6 2025-12-17 21:05:23,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407802575bc060ce15b3e6 HTTP/1.1" 502 150 2025-12-17 21:05:23,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407802575bc060ce15b3e6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:23,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f4575bc060ce15b3c5 2025-12-17 21:05:23,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f4575bc060ce15b3c5 HTTP/1.1" 502 150 2025-12-17 21:05:23,895 - lava_callback - ERROR - Error processing callback for node 694077f4575bc060ce15b3c5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f4575bc060ce15b3c5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f4575bc060ce15b3c5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f4575bc060ce15b3c5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:23,897 - lava_callback - INFO - Starting async job submit for node 69407804575bc060ce15b3ee 2025-12-17 21:05:23,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:23,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407804575bc060ce15b3ee HTTP/1.1" 502 150 2025-12-17 21:05:23,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407804575bc060ce15b3ee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:23,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407804575bc060ce15b3ee 2025-12-17 21:05:23,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407804575bc060ce15b3ee HTTP/1.1" 502 150 2025-12-17 21:05:23,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407804575bc060ce15b3ee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:24,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f5575bc060ce15b3c8 2025-12-17 21:05:24,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f5575bc060ce15b3c8 HTTP/1.1" 502 150 2025-12-17 21:05:24,638 - lava_callback - ERROR - Error processing callback for node 694077f5575bc060ce15b3c8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f5575bc060ce15b3c8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f5575bc060ce15b3c8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f5575bc060ce15b3c8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:24,640 - lava_callback - INFO - Starting async job submit for node 69407806575bc060ce15b3f1 2025-12-17 21:05:24,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:24,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407806575bc060ce15b3f1 HTTP/1.1" 502 150 2025-12-17 21:05:24,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407806575bc060ce15b3f1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:24,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407806575bc060ce15b3f1 2025-12-17 21:05:24,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407806575bc060ce15b3f1 HTTP/1.1" 502 150 2025-12-17 21:05:24,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407806575bc060ce15b3f1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:24,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f6575bc060ce15b3cb 2025-12-17 21:05:24,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f6575bc060ce15b3cb HTTP/1.1" 502 150 2025-12-17 21:05:24,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f6575bc060ce15b3cb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:25,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f7575bc060ce15b3ce 2025-12-17 21:05:25,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f7575bc060ce15b3ce HTTP/1.1" 502 150 2025-12-17 21:05:25,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f7575bc060ce15b3ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:25,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407804575bc060ce15b3ee 2025-12-17 21:05:25,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407804575bc060ce15b3ee HTTP/1.1" 502 150 2025-12-17 21:05:25,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407804575bc060ce15b3ee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:26,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f9575bc060ce15b3d1 2025-12-17 21:05:26,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f9575bc060ce15b3d1 HTTP/1.1" 502 150 2025-12-17 21:05:26,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077f9575bc060ce15b3d1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50760 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:05:26,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407806575bc060ce15b3f1 2025-12-17 21:05:26,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407806575bc060ce15b3f1 HTTP/1.1" 502 150 2025-12-17 21:05:26,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407806575bc060ce15b3f1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:27,226 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fa575bc060ce15b3d4 2025-12-17 21:05:27,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fa575bc060ce15b3d4 HTTP/1.1" 502 150 2025-12-17 21:05:27,227 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fa575bc060ce15b3d4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:27,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407803575bc060ce15b3eb 2025-12-17 21:05:27,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407803575bc060ce15b3eb HTTP/1.1" 502 150 2025-12-17 21:05:27,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407803575bc060ce15b3eb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:27,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fb575bc060ce15b3d7 2025-12-17 21:05:27,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fb575bc060ce15b3d7 HTTP/1.1" 502 150 2025-12-17 21:05:27,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fb575bc060ce15b3d7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:28,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fd575bc060ce15b3da 2025-12-17 21:05:28,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fd575bc060ce15b3da HTTP/1.1" 502 150 2025-12-17 21:05:28,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fd575bc060ce15b3da'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:29,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fe575bc060ce15b3dd 2025-12-17 21:05:29,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fe575bc060ce15b3dd HTTP/1.1" 502 150 2025-12-17 21:05:29,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077fe575bc060ce15b3dd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:29,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407804575bc060ce15b3ee 2025-12-17 21:05:29,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407804575bc060ce15b3ee HTTP/1.1" 502 150 2025-12-17 21:05:29,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407804575bc060ce15b3ee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:30,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407806575bc060ce15b3f1 2025-12-17 21:05:30,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407806575bc060ce15b3f1 HTTP/1.1" 502 150 2025-12-17 21:05:30,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407806575bc060ce15b3f1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:30,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407800575bc060ce15b3e3 2025-12-17 21:05:30,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407800575bc060ce15b3e3 HTTP/1.1" 502 150 2025-12-17 21:05:30,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407800575bc060ce15b3e3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:31,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407802575bc060ce15b3e6 2025-12-17 21:05:31,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407802575bc060ce15b3e6 HTTP/1.1" 502 150 2025-12-17 21:05:31,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407802575bc060ce15b3e6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:33,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ff575bc060ce15b3e0 HTTP/1.1" 502 150 2025-12-17 21:05:33,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077ff575bc060ce15b3e0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:35,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407803575bc060ce15b3eb 2025-12-17 21:05:35,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407803575bc060ce15b3eb HTTP/1.1" 502 150 2025-12-17 21:05:35,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407803575bc060ce15b3eb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:36,278 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077ee575bc060ce15b3a7 2025-12-17 21:05:36,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ee575bc060ce15b3a7 HTTP/1.1" 502 150 2025-12-17 21:05:36,279 - lava_callback - ERROR - Error processing callback for node 694077ee575bc060ce15b3a7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077ee575bc060ce15b3a7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077ee575bc060ce15b3a7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077ee575bc060ce15b3a7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:36,281 - lava_callback - INFO - Starting async job submit for node 69407807575bc060ce15b3f4 2025-12-17 21:05:36,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:36,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407807575bc060ce15b3f4 HTTP/1.1" 502 150 2025-12-17 21:05:36,309 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407807575bc060ce15b3f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:36,310 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407807575bc060ce15b3f4 2025-12-17 21:05:36,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407807575bc060ce15b3f4 HTTP/1.1" 502 150 2025-12-17 21:05:36,310 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407807575bc060ce15b3f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57634 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:05:37,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077ff575bc060ce15b3e0 2025-12-17 21:05:37,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ff575bc060ce15b3e0 HTTP/1.1" 502 150 2025-12-17 21:05:37,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077ff575bc060ce15b3e0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:37,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f1575bc060ce15b3be 2025-12-17 21:05:37,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f1575bc060ce15b3be HTTP/1.1" 502 150 2025-12-17 21:05:37,771 - lava_callback - ERROR - Error processing callback for node 694077f1575bc060ce15b3be: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f1575bc060ce15b3be (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f1575bc060ce15b3be (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f1575bc060ce15b3be (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:37,772 - lava_callback - INFO - Starting async job submit for node 69407808575bc060ce15b3f7 2025-12-17 21:05:37,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:37,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407808575bc060ce15b3f7 HTTP/1.1" 502 150 2025-12-17 21:05:37,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407808575bc060ce15b3f7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:37,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407808575bc060ce15b3f7 2025-12-17 21:05:37,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407808575bc060ce15b3f7 HTTP/1.1" 502 150 2025-12-17 21:05:37,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407808575bc060ce15b3f7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:37,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407804575bc060ce15b3ee 2025-12-17 21:05:37,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407804575bc060ce15b3ee HTTP/1.1" 502 150 2025-12-17 21:05:37,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407804575bc060ce15b3ee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:38,310 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407807575bc060ce15b3f4 2025-12-17 21:05:38,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407807575bc060ce15b3f4 HTTP/1.1" 502 150 2025-12-17 21:05:38,312 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407807575bc060ce15b3f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:38,352 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f2575bc060ce15b3c2 2025-12-17 21:05:38,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f2575bc060ce15b3c2 HTTP/1.1" 502 150 2025-12-17 21:05:38,353 - lava_callback - ERROR - Error processing callback for node 694077f2575bc060ce15b3c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f2575bc060ce15b3c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f2575bc060ce15b3c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f2575bc060ce15b3c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:38,355 - lava_callback - INFO - Starting async job submit for node 69407809575bc060ce15b3fa 2025-12-17 21:05:38,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:38,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407809575bc060ce15b3fa HTTP/1.1" 502 150 2025-12-17 21:05:38,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407809575bc060ce15b3fa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:38,363 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407809575bc060ce15b3fa 2025-12-17 21:05:38,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407809575bc060ce15b3fa HTTP/1.1" 502 150 2025-12-17 21:05:38,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407809575bc060ce15b3fa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:38,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407806575bc060ce15b3f1 2025-12-17 21:05:38,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407806575bc060ce15b3f1 HTTP/1.1" 502 150 2025-12-17 21:05:38,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407806575bc060ce15b3f1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:39,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407808575bc060ce15b3f7 2025-12-17 21:05:39,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407808575bc060ce15b3f7 HTTP/1.1" 502 150 2025-12-17 21:05:39,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407808575bc060ce15b3f7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:40,364 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407809575bc060ce15b3fa 2025-12-17 21:05:40,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407809575bc060ce15b3fa HTTP/1.1" 502 150 2025-12-17 21:05:40,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407809575bc060ce15b3fa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:40,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f6575bc060ce15b3cb 2025-12-17 21:05:40,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f6575bc060ce15b3cb HTTP/1.1" 502 150 2025-12-17 21:05:40,866 - lava_callback - ERROR - Error processing callback for node 694077f6575bc060ce15b3cb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f6575bc060ce15b3cb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f6575bc060ce15b3cb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f6575bc060ce15b3cb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:40,868 - lava_callback - INFO - Starting async job submit for node 6940780d575bc060ce15b400 2025-12-17 21:05:40,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:40,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940780d575bc060ce15b400 HTTP/1.1" 502 150 2025-12-17 21:05:40,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940780d575bc060ce15b400'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:40,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940780d575bc060ce15b400 2025-12-17 21:05:40,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940780d575bc060ce15b400 HTTP/1.1" 502 150 2025-12-17 21:05:40,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940780d575bc060ce15b400'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:41,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f7575bc060ce15b3ce 2025-12-17 21:05:41,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f7575bc060ce15b3ce HTTP/1.1" 502 150 2025-12-17 21:05:41,687 - lava_callback - ERROR - Error processing callback for node 694077f7575bc060ce15b3ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f7575bc060ce15b3ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f7575bc060ce15b3ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f7575bc060ce15b3ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:42,312 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407807575bc060ce15b3f4 2025-12-17 21:05:42,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407807575bc060ce15b3f4 HTTP/1.1" 502 150 2025-12-17 21:05:42,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407807575bc060ce15b3f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:42,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077f9575bc060ce15b3d1 2025-12-17 21:05:42,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077f9575bc060ce15b3d1 HTTP/1.1" 502 150 2025-12-17 21:05:42,435 - lava_callback - ERROR - Error processing callback for node 694077f9575bc060ce15b3d1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f9575bc060ce15b3d1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f9575bc060ce15b3d1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077f9575bc060ce15b3d1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:42,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940780d575bc060ce15b400 2025-12-17 21:05:42,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940780d575bc060ce15b400 HTTP/1.1" 502 150 2025-12-17 21:05:42,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940780d575bc060ce15b400'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:43,228 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fa575bc060ce15b3d4 2025-12-17 21:05:43,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fa575bc060ce15b3d4 HTTP/1.1" 502 150 2025-12-17 21:05:43,229 - lava_callback - ERROR - Error processing callback for node 694077fa575bc060ce15b3d4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077fa575bc060ce15b3d4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077fa575bc060ce15b3d4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077fa575bc060ce15b3d4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:43,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407808575bc060ce15b3f7 2025-12-17 21:05:43,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407808575bc060ce15b3f7 HTTP/1.1" 502 150 2025-12-17 21:05:43,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407808575bc060ce15b3f7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:43,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fb575bc060ce15b3d7 2025-12-17 21:05:43,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fb575bc060ce15b3d7 HTTP/1.1" 502 150 2025-12-17 21:05:43,959 - lava_callback - ERROR - Error processing callback for node 694077fb575bc060ce15b3d7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077fb575bc060ce15b3d7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077fb575bc060ce15b3d7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077fb575bc060ce15b3d7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:44,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407809575bc060ce15b3fa 2025-12-17 21:05:44,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fd575bc060ce15b3da 2025-12-17 21:05:44,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fd575bc060ce15b3da HTTP/1.1" 502 150 2025-12-17 21:05:44,693 - lava_callback - ERROR - Error processing callback for node 694077fd575bc060ce15b3da: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077fd575bc060ce15b3da (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077fd575bc060ce15b3da (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077fd575bc060ce15b3da (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:45,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077fe575bc060ce15b3dd 2025-12-17 21:05:45,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077ff575bc060ce15b3e0 2025-12-17 21:05:45,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ff575bc060ce15b3e0 HTTP/1.1" 502 150 2025-12-17 21:05:45,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077ff575bc060ce15b3e0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48696 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:05:46,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940780d575bc060ce15b400 2025-12-17 21:05:46,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940780d575bc060ce15b400 HTTP/1.1" 502 150 2025-12-17 21:05:46,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940780d575bc060ce15b400'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:46,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407800575bc060ce15b3e3 2025-12-17 21:05:46,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407800575bc060ce15b3e3 HTTP/1.1" 502 150 2025-12-17 21:05:46,947 - lava_callback - ERROR - Error processing callback for node 69407800575bc060ce15b3e3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407800575bc060ce15b3e3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407800575bc060ce15b3e3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407800575bc060ce15b3e3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:47,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407802575bc060ce15b3e6 2025-12-17 21:05:47,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407802575bc060ce15b3e6 HTTP/1.1" 502 150 2025-12-17 21:05:47,728 - lava_callback - ERROR - Error processing callback for node 69407802575bc060ce15b3e6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407802575bc060ce15b3e6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407802575bc060ce15b3e6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407802575bc060ce15b3e6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:50,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407807575bc060ce15b3f4 2025-12-17 21:05:50,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407807575bc060ce15b3f4 HTTP/1.1" 502 150 2025-12-17 21:05:50,315 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407807575bc060ce15b3f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:50,811 - lava_callback - INFO - Received callback for node 6940782c575bc060ce15b430 INFO: 172.21.0.1:59762 - "POST /node/6940782c575bc060ce15b430 HTTP/1.0" 202 Accepted 2025-12-17 21:05:50,817 - lava_callback - INFO - Starting async job submit for node 6940782c575bc060ce15b430 2025-12-17 21:05:50,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:50,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782c575bc060ce15b430 HTTP/1.1" 502 150 2025-12-17 21:05:50,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940782c575bc060ce15b430'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:50,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940782c575bc060ce15b430 2025-12-17 21:05:50,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782c575bc060ce15b430 HTTP/1.1" 502 150 2025-12-17 21:05:50,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940782c575bc060ce15b430'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:51,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407803575bc060ce15b3eb 2025-12-17 21:05:51,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407803575bc060ce15b3eb HTTP/1.1" 502 150 2025-12-17 21:05:51,613 - lava_callback - ERROR - Error processing callback for node 69407803575bc060ce15b3eb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407803575bc060ce15b3eb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407803575bc060ce15b3eb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407803575bc060ce15b3eb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:51,632 - lava_callback - INFO - Received callback for node 6940782d575bc060ce15b434 INFO: 172.21.0.1:59778 - "POST /node/6940782d575bc060ce15b434 HTTP/1.0" 202 Accepted 2025-12-17 21:05:51,638 - lava_callback - INFO - Starting async job submit for node 6940782d575bc060ce15b434 2025-12-17 21:05:51,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:51,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782d575bc060ce15b434 HTTP/1.1" 502 150 2025-12-17 21:05:51,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940782d575bc060ce15b434'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:51,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940782d575bc060ce15b434 2025-12-17 21:05:51,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782d575bc060ce15b434 HTTP/1.1" 502 150 2025-12-17 21:05:51,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940782d575bc060ce15b434'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:51,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407808575bc060ce15b3f7 2025-12-17 21:05:51,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407808575bc060ce15b3f7 HTTP/1.1" 502 150 2025-12-17 21:05:51,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407808575bc060ce15b3f7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:52,417 - lava_callback - INFO - Received callback for node 6940782f575bc060ce15b437 INFO: 172.21.0.1:59792 - "POST /node/6940782f575bc060ce15b437 HTTP/1.0" 202 Accepted 2025-12-17 21:05:52,424 - lava_callback - INFO - Starting async job submit for node 6940782f575bc060ce15b437 2025-12-17 21:05:52,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:52,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782f575bc060ce15b437 HTTP/1.1" 502 150 2025-12-17 21:05:52,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940782f575bc060ce15b437'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:52,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940782f575bc060ce15b437 2025-12-17 21:05:52,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782f575bc060ce15b437 HTTP/1.1" 502 150 2025-12-17 21:05:52,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940782f575bc060ce15b437'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:52,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940782c575bc060ce15b430 2025-12-17 21:05:52,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782c575bc060ce15b430 HTTP/1.1" 502 150 2025-12-17 21:05:52,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940782c575bc060ce15b430'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:53,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940782d575bc060ce15b434 2025-12-17 21:05:53,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782d575bc060ce15b434 HTTP/1.1" 502 150 2025-12-17 21:05:53,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940782d575bc060ce15b434'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:53,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407804575bc060ce15b3ee 2025-12-17 21:05:53,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407804575bc060ce15b3ee HTTP/1.1" 502 150 2025-12-17 21:05:53,913 - lava_callback - ERROR - Error processing callback for node 69407804575bc060ce15b3ee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407804575bc060ce15b3ee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407804575bc060ce15b3ee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407804575bc060ce15b3ee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:54,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940782f575bc060ce15b437 2025-12-17 21:05:54,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782f575bc060ce15b437 HTTP/1.1" 502 150 2025-12-17 21:05:54,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940782f575bc060ce15b437'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:54,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407806575bc060ce15b3f1 2025-12-17 21:05:54,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407806575bc060ce15b3f1 HTTP/1.1" 502 150 2025-12-17 21:05:54,674 - lava_callback - ERROR - Error processing callback for node 69407806575bc060ce15b3f1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407806575bc060ce15b3f1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407806575bc060ce15b3f1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407806575bc060ce15b3f1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:05:54,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940780d575bc060ce15b400 2025-12-17 21:05:54,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940780d575bc060ce15b400 HTTP/1.1" 502 150 2025-12-17 21:05:54,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940780d575bc060ce15b400'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59802 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:05:56,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940782c575bc060ce15b430 2025-12-17 21:05:56,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782c575bc060ce15b430 HTTP/1.1" 502 150 2025-12-17 21:05:56,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940782c575bc060ce15b430'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:57,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940782d575bc060ce15b434 2025-12-17 21:05:57,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782d575bc060ce15b434 HTTP/1.1" 502 150 2025-12-17 21:05:57,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940782d575bc060ce15b434'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:58,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940782f575bc060ce15b437 2025-12-17 21:05:58,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782f575bc060ce15b437 HTTP/1.1" 502 150 2025-12-17 21:05:58,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940782f575bc060ce15b437'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:59,586 - lava_callback - INFO - Received callback for node 694070d1575bc060ce15a981 INFO: 172.21.0.1:59818 - "POST /node/694070d1575bc060ce15a981 HTTP/1.0" 202 Accepted 2025-12-17 21:05:59,595 - lava_callback - INFO - Starting async job submit for node 694070d1575bc060ce15a981 2025-12-17 21:05:59,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:05:59,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d1575bc060ce15a981 HTTP/1.1" 502 150 2025-12-17 21:05:59,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d1575bc060ce15a981'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:59,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070d1575bc060ce15a981 2025-12-17 21:05:59,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d1575bc060ce15a981 HTTP/1.1" 502 150 2025-12-17 21:05:59,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d1575bc060ce15a981'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:05:59,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407809575bc060ce15b3fa HTTP/1.1" 502 150 2025-12-17 21:05:59,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407809575bc060ce15b3fa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:06:00,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077fe575bc060ce15b3dd HTTP/1.1" 502 150 2025-12-17 21:06:00,752 - lava_callback - ERROR - Error processing callback for node 694077fe575bc060ce15b3dd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077fe575bc060ce15b3dd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077fe575bc060ce15b3dd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077fe575bc060ce15b3dd (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:32804 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:06:01,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077ff575bc060ce15b3e0 2025-12-17 21:06:01,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ff575bc060ce15b3e0 HTTP/1.1" 502 150 2025-12-17 21:06:01,621 - lava_callback - ERROR - Error processing callback for node 694077ff575bc060ce15b3e0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077ff575bc060ce15b3e0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077ff575bc060ce15b3e0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077ff575bc060ce15b3e0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:06:01,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070d1575bc060ce15a981 2025-12-17 21:06:01,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d1575bc060ce15a981 HTTP/1.1" 502 150 2025-12-17 21:06:01,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d1575bc060ce15a981'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:06:04,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940782c575bc060ce15b430 2025-12-17 21:06:04,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782c575bc060ce15b430 HTTP/1.1" 502 150 2025-12-17 21:06:04,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940782c575bc060ce15b430'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:06:05,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940782d575bc060ce15b434 2025-12-17 21:06:05,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782d575bc060ce15b434 HTTP/1.1" 502 150 2025-12-17 21:06:05,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940782d575bc060ce15b434'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:06:05,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070d1575bc060ce15a981 2025-12-17 21:06:05,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d1575bc060ce15a981 HTTP/1.1" 502 150 2025-12-17 21:06:05,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d1575bc060ce15a981'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:06:06,315 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407807575bc060ce15b3f4 2025-12-17 21:06:06,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407807575bc060ce15b3f4 HTTP/1.1" 502 150 2025-12-17 21:06:06,317 - lava_callback - ERROR - Error processing callback for node 69407807575bc060ce15b3f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407807575bc060ce15b3f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407807575bc060ce15b3f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407807575bc060ce15b3f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:06:06,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940782f575bc060ce15b437 2025-12-17 21:06:06,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782f575bc060ce15b437 HTTP/1.1" 502 150 2025-12-17 21:06:06,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940782f575bc060ce15b437'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:32816 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:06:07,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407809575bc060ce15b3fa 2025-12-17 21:06:07,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407809575bc060ce15b3fa HTTP/1.1" 502 150 2025-12-17 21:06:07,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407809575bc060ce15b3fa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:06:07,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407808575bc060ce15b3f7 2025-12-17 21:06:07,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407808575bc060ce15b3f7 HTTP/1.1" 502 150 2025-12-17 21:06:07,786 - lava_callback - ERROR - Error processing callback for node 69407808575bc060ce15b3f7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407808575bc060ce15b3f7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407808575bc060ce15b3f7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407808575bc060ce15b3f7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:06:10,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940780d575bc060ce15b400 2025-12-17 21:06:10,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940780d575bc060ce15b400 HTTP/1.1" 502 150 2025-12-17 21:06:10,882 - lava_callback - ERROR - Error processing callback for node 6940780d575bc060ce15b400: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940780d575bc060ce15b400 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940780d575bc060ce15b400 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940780d575bc060ce15b400 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:06:11,547 - lava_callback - INFO - Received callback for node 69407831575bc060ce15b43d INFO: 172.21.0.1:43668 - "POST /node/69407831575bc060ce15b43d HTTP/1.0" 202 Accepted 2025-12-17 21:06:11,552 - lava_callback - INFO - Starting async job submit for node 69407831575bc060ce15b43d 2025-12-17 21:06:11,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:06:13,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070d1575bc060ce15a981 2025-12-17 21:06:13,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d1575bc060ce15a981 HTTP/1.1" 502 150 2025-12-17 21:06:13,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694070d1575bc060ce15a981'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43700 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:06:20,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940782c575bc060ce15b430 2025-12-17 21:06:20,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782c575bc060ce15b430 HTTP/1.1" 502 150 2025-12-17 21:06:20,840 - lava_callback - ERROR - Error processing callback for node 6940782c575bc060ce15b430: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940782c575bc060ce15b430 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940782c575bc060ce15b430 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940782c575bc060ce15b430 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:06:21,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940782d575bc060ce15b434 2025-12-17 21:06:21,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782d575bc060ce15b434 HTTP/1.1" 502 150 2025-12-17 21:06:21,677 - lava_callback - ERROR - Error processing callback for node 6940782d575bc060ce15b434: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940782d575bc060ce15b434 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940782d575bc060ce15b434 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940782d575bc060ce15b434 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:06:22,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940782f575bc060ce15b437 2025-12-17 21:06:22,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940782f575bc060ce15b437 HTTP/1.1" 502 150 2025-12-17 21:06:22,457 - lava_callback - ERROR - Error processing callback for node 6940782f575bc060ce15b437: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940782f575bc060ce15b437 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940782f575bc060ce15b437 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940782f575bc060ce15b437 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:06:23,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407809575bc060ce15b3fa 2025-12-17 21:06:23,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407809575bc060ce15b3fa HTTP/1.1" 502 150 2025-12-17 21:06:23,731 - lava_callback - ERROR - Error processing callback for node 69407809575bc060ce15b3fa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407809575bc060ce15b3fa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407809575bc060ce15b3fa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407809575bc060ce15b3fa (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:45016 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:06:26,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407831575bc060ce15b43d HTTP/1.1" 502 150 2025-12-17 21:06:26,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407831575bc060ce15b43d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:06:26,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407831575bc060ce15b43d 2025-12-17 21:06:26,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407831575bc060ce15b43d HTTP/1.1" 502 150 2025-12-17 21:06:26,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407831575bc060ce15b43d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:06:28,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407831575bc060ce15b43d 2025-12-17 21:06:28,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407831575bc060ce15b43d HTTP/1.1" 502 150 2025-12-17 21:06:28,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407831575bc060ce15b43d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:06:29,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694070d1575bc060ce15a981 2025-12-17 21:06:29,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694070d1575bc060ce15a981 HTTP/1.1" 502 150 2025-12-17 21:06:29,727 - lava_callback - ERROR - Error processing callback for node 694070d1575bc060ce15a981: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d1575bc060ce15a981 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d1575bc060ce15a981 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694070d1575bc060ce15a981 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:06:32,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407831575bc060ce15b43d 2025-12-17 21:06:32,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407831575bc060ce15b43d HTTP/1.1" 502 150 2025-12-17 21:06:32,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407831575bc060ce15b43d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47684 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:06:40,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407831575bc060ce15b43d 2025-12-17 21:06:40,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407831575bc060ce15b43d HTTP/1.1" 502 150 2025-12-17 21:06:40,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407831575bc060ce15b43d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36110 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:06:51,423 - lava_callback - INFO - Received callback for node 694077b7575bc060ce15b354 INFO: 172.21.0.1:39856 - "POST /node/694077b7575bc060ce15b354 HTTP/1.0" 202 Accepted 2025-12-17 21:06:51,430 - lava_callback - INFO - Starting async job submit for node 694077b7575bc060ce15b354 2025-12-17 21:06:51,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:06:51,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b7575bc060ce15b354 HTTP/1.1" 502 150 2025-12-17 21:06:51,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b7575bc060ce15b354'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:06:51,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b7575bc060ce15b354 2025-12-17 21:06:51,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b7575bc060ce15b354 HTTP/1.1" 502 150 2025-12-17 21:06:51,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b7575bc060ce15b354'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:06:53,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b7575bc060ce15b354 2025-12-17 21:06:53,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b7575bc060ce15b354 HTTP/1.1" 502 150 2025-12-17 21:06:53,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b7575bc060ce15b354'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39866 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:06:56,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407831575bc060ce15b43d 2025-12-17 21:06:56,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407831575bc060ce15b43d HTTP/1.1" 502 150 2025-12-17 21:06:56,872 - lava_callback - ERROR - Error processing callback for node 69407831575bc060ce15b43d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407831575bc060ce15b43d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407831575bc060ce15b43d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407831575bc060ce15b43d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:06:57,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b7575bc060ce15b354 2025-12-17 21:06:57,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b7575bc060ce15b354 HTTP/1.1" 502 150 2025-12-17 21:06:57,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b7575bc060ce15b354'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:06:59,667 - lava_callback - INFO - Received callback for node 69407446575bc060ce15ae91 INFO: 172.21.0.1:39876 - "POST /node/69407446575bc060ce15ae91 HTTP/1.0" 202 Accepted 2025-12-17 21:06:59,674 - lava_callback - INFO - Starting async job submit for node 69407446575bc060ce15ae91 2025-12-17 21:06:59,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:06:59,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407446575bc060ce15ae91 HTTP/1.1" 502 150 2025-12-17 21:06:59,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407446575bc060ce15ae91'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:06:59,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407446575bc060ce15ae91 2025-12-17 21:06:59,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407446575bc060ce15ae91 HTTP/1.1" 502 150 2025-12-17 21:06:59,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407446575bc060ce15ae91'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49130 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:07:01,293 - lava_callback - INFO - Received callback for node 694077ba575bc060ce15b35a INFO: 172.21.0.1:49144 - "POST /node/694077ba575bc060ce15b35a HTTP/1.0" 202 Accepted 2025-12-17 21:07:01,299 - lava_callback - INFO - Starting async job submit for node 694077ba575bc060ce15b35a 2025-12-17 21:07:01,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:01,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ba575bc060ce15b35a HTTP/1.1" 502 150 2025-12-17 21:07:01,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077ba575bc060ce15b35a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:01,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077ba575bc060ce15b35a 2025-12-17 21:07:01,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ba575bc060ce15b35a HTTP/1.1" 502 150 2025-12-17 21:07:01,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077ba575bc060ce15b35a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:01,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407446575bc060ce15ae91 2025-12-17 21:07:01,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407446575bc060ce15ae91 HTTP/1.1" 502 150 2025-12-17 21:07:01,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407446575bc060ce15ae91'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:03,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077ba575bc060ce15b35a 2025-12-17 21:07:03,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ba575bc060ce15b35a HTTP/1.1" 502 150 2025-12-17 21:07:03,333 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077ba575bc060ce15b35a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:04,073 - lava_callback - INFO - Received callback for node 69407e09575bc060ce15c213 INFO: 172.21.0.1:49154 - "POST /node/69407e09575bc060ce15c213 HTTP/1.0" 202 Accepted 2025-12-17 21:07:04,083 - lava_callback - INFO - Starting async job submit for node 69407e09575bc060ce15c213 2025-12-17 21:07:04,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:05,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b7575bc060ce15b354 2025-12-17 21:07:05,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b7575bc060ce15b354 HTTP/1.1" 502 150 2025-12-17 21:07:05,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077b7575bc060ce15b354'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:05,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407446575bc060ce15ae91 2025-12-17 21:07:05,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407446575bc060ce15ae91 HTTP/1.1" 502 150 2025-12-17 21:07:05,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407446575bc060ce15ae91'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49160 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:07:06,825 - lava_callback - INFO - Received callback for node 69407870575bc060ce15b4a1 INFO: 172.21.0.1:49172 - "POST /node/69407870575bc060ce15b4a1 HTTP/1.0" 202 Accepted 2025-12-17 21:07:06,834 - lava_callback - INFO - Starting async job submit for node 69407870575bc060ce15b4a1 2025-12-17 21:07:06,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:06,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407870575bc060ce15b4a1 HTTP/1.1" 502 150 2025-12-17 21:07:06,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407870575bc060ce15b4a1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:06,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407870575bc060ce15b4a1 2025-12-17 21:07:06,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407870575bc060ce15b4a1 HTTP/1.1" 502 150 2025-12-17 21:07:06,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407870575bc060ce15b4a1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:07,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077ba575bc060ce15b35a 2025-12-17 21:07:07,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ba575bc060ce15b35a HTTP/1.1" 502 150 2025-12-17 21:07:07,336 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077ba575bc060ce15b35a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:08,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407870575bc060ce15b4a1 2025-12-17 21:07:08,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407870575bc060ce15b4a1 HTTP/1.1" 502 150 2025-12-17 21:07:08,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407870575bc060ce15b4a1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:12,285 - lava_callback - INFO - Received callback for node 69407873575bc060ce15b4ac INFO: 172.21.0.1:36032 - "POST /node/69407873575bc060ce15b4ac HTTP/1.0" 202 Accepted 2025-12-17 21:07:12,291 - lava_callback - INFO - Starting async job submit for node 69407873575bc060ce15b4ac 2025-12-17 21:07:12,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:12,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407873575bc060ce15b4ac HTTP/1.1" 502 150 2025-12-17 21:07:12,297 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407873575bc060ce15b4ac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:12,298 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407873575bc060ce15b4ac 2025-12-17 21:07:12,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407873575bc060ce15b4ac HTTP/1.1" 502 150 2025-12-17 21:07:12,298 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407873575bc060ce15b4ac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:12,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407870575bc060ce15b4a1 2025-12-17 21:07:12,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407870575bc060ce15b4a1 HTTP/1.1" 502 150 2025-12-17 21:07:12,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407870575bc060ce15b4a1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:13,069 - lava_callback - INFO - Received callback for node 69407875575bc060ce15b4b7 INFO: 172.21.0.1:36038 - "POST /node/69407875575bc060ce15b4b7 HTTP/1.0" 202 Accepted 2025-12-17 21:07:13,075 - lava_callback - INFO - Starting async job submit for node 69407875575bc060ce15b4b7 2025-12-17 21:07:13,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:13,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407875575bc060ce15b4b7 HTTP/1.1" 502 150 2025-12-17 21:07:13,100 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407875575bc060ce15b4b7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:13,100 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407875575bc060ce15b4b7 2025-12-17 21:07:13,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407875575bc060ce15b4b7 HTTP/1.1" 502 150 2025-12-17 21:07:13,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407875575bc060ce15b4b7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:13,391 - lava_callback - INFO - Received callback for node 69408928575bc060ce15e992 INFO: 172.21.0.1:36050 - "POST /node/69408928575bc060ce15e992 HTTP/1.0" 202 Accepted 2025-12-17 21:07:13,400 - lava_callback - INFO - Starting async job submit for node 69408928575bc060ce15e992 2025-12-17 21:07:13,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:13,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408928575bc060ce15e992 HTTP/1.1" 502 150 2025-12-17 21:07:13,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408928575bc060ce15e992'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:13,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408928575bc060ce15e992 2025-12-17 21:07:13,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408928575bc060ce15e992 HTTP/1.1" 502 150 2025-12-17 21:07:13,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408928575bc060ce15e992'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:13,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407446575bc060ce15ae91 2025-12-17 21:07:13,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407446575bc060ce15ae91 HTTP/1.1" 502 150 2025-12-17 21:07:13,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407446575bc060ce15ae91'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:13,861 - lava_callback - INFO - Received callback for node 69407876575bc060ce15b4bc INFO: 172.21.0.1:36066 - "POST /node/69407876575bc060ce15b4bc HTTP/1.0" 202 Accepted 2025-12-17 21:07:13,867 - lava_callback - INFO - Starting async job submit for node 69407876575bc060ce15b4bc 2025-12-17 21:07:13,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:13,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407876575bc060ce15b4bc HTTP/1.1" 502 150 2025-12-17 21:07:13,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407876575bc060ce15b4bc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:13,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407876575bc060ce15b4bc 2025-12-17 21:07:13,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407876575bc060ce15b4bc HTTP/1.1" 502 150 2025-12-17 21:07:13,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407876575bc060ce15b4bc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:14,299 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407873575bc060ce15b4ac 2025-12-17 21:07:14,597 - lava_callback - INFO - Received callback for node 69407877575bc060ce15b4c0 INFO: 172.21.0.1:36074 - "POST /node/69407877575bc060ce15b4c0 HTTP/1.0" 202 Accepted 2025-12-17 21:07:14,603 - lava_callback - INFO - Starting async job submit for node 69407877575bc060ce15b4c0 2025-12-17 21:07:14,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:14,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407877575bc060ce15b4c0 HTTP/1.1" 502 150 2025-12-17 21:07:14,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407877575bc060ce15b4c0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:14,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407877575bc060ce15b4c0 2025-12-17 21:07:14,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407877575bc060ce15b4c0 HTTP/1.1" 502 150 2025-12-17 21:07:14,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407877575bc060ce15b4c0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:15,101 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407875575bc060ce15b4b7 2025-12-17 21:07:15,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407875575bc060ce15b4b7 HTTP/1.1" 502 150 2025-12-17 21:07:15,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407875575bc060ce15b4b7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36076 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:07:15,336 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077ba575bc060ce15b35a 2025-12-17 21:07:15,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ba575bc060ce15b35a HTTP/1.1" 502 150 2025-12-17 21:07:15,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694077ba575bc060ce15b35a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:15,407 - lava_callback - INFO - Received callback for node 69407879575bc060ce15b4c3 INFO: 172.21.0.1:36086 - "POST /node/69407879575bc060ce15b4c3 HTTP/1.0" 202 Accepted 2025-12-17 21:07:15,413 - lava_callback - INFO - Starting async job submit for node 69407879575bc060ce15b4c3 2025-12-17 21:07:15,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:15,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407879575bc060ce15b4c3 HTTP/1.1" 502 150 2025-12-17 21:07:15,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407879575bc060ce15b4c3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:15,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407879575bc060ce15b4c3 2025-12-17 21:07:15,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407879575bc060ce15b4c3 HTTP/1.1" 502 150 2025-12-17 21:07:15,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407879575bc060ce15b4c3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:15,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408928575bc060ce15e992 2025-12-17 21:07:15,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408928575bc060ce15e992 HTTP/1.1" 502 150 2025-12-17 21:07:15,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408928575bc060ce15e992'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:15,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407876575bc060ce15b4bc 2025-12-17 21:07:15,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407876575bc060ce15b4bc HTTP/1.1" 502 150 2025-12-17 21:07:15,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407876575bc060ce15b4bc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:16,147 - lava_callback - INFO - Received callback for node 6940787a575bc060ce15b4c6 INFO: 172.21.0.1:36098 - "POST /node/6940787a575bc060ce15b4c6 HTTP/1.0" 202 Accepted 2025-12-17 21:07:16,153 - lava_callback - INFO - Starting async job submit for node 6940787a575bc060ce15b4c6 2025-12-17 21:07:16,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:36104 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:07:16,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407877575bc060ce15b4c0 2025-12-17 21:07:16,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407877575bc060ce15b4c0 HTTP/1.1" 502 150 2025-12-17 21:07:16,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407877575bc060ce15b4c0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:16,918 - lava_callback - INFO - Received callback for node 6940787b575bc060ce15b4c9 INFO: 172.21.0.1:36114 - "POST /node/6940787b575bc060ce15b4c9 HTTP/1.0" 202 Accepted 2025-12-17 21:07:16,924 - lava_callback - INFO - Starting async job submit for node 6940787b575bc060ce15b4c9 2025-12-17 21:07:16,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:16,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787b575bc060ce15b4c9 HTTP/1.1" 502 150 2025-12-17 21:07:16,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787b575bc060ce15b4c9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:16,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787b575bc060ce15b4c9 2025-12-17 21:07:16,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787b575bc060ce15b4c9 HTTP/1.1" 502 150 2025-12-17 21:07:16,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787b575bc060ce15b4c9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:17,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407879575bc060ce15b4c3 2025-12-17 21:07:17,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407879575bc060ce15b4c3 HTTP/1.1" 502 150 2025-12-17 21:07:17,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407879575bc060ce15b4c3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:17,762 - lava_callback - INFO - Received callback for node 6940787c575bc060ce15b4cc INFO: 172.21.0.1:36128 - "POST /node/6940787c575bc060ce15b4cc HTTP/1.0" 202 Accepted 2025-12-17 21:07:17,767 - lava_callback - INFO - Starting async job submit for node 6940787c575bc060ce15b4cc 2025-12-17 21:07:17,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:17,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787c575bc060ce15b4cc HTTP/1.1" 502 150 2025-12-17 21:07:17,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787c575bc060ce15b4cc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:17,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787c575bc060ce15b4cc 2025-12-17 21:07:17,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787c575bc060ce15b4cc HTTP/1.1" 502 150 2025-12-17 21:07:17,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787c575bc060ce15b4cc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:18,540 - lava_callback - INFO - Received callback for node 6940787d575bc060ce15b4cf INFO: 172.21.0.1:36138 - "POST /node/6940787d575bc060ce15b4cf HTTP/1.0" 202 Accepted 2025-12-17 21:07:18,546 - lava_callback - INFO - Starting async job submit for node 6940787d575bc060ce15b4cf 2025-12-17 21:07:18,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:18,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787d575bc060ce15b4cf HTTP/1.1" 502 150 2025-12-17 21:07:18,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787d575bc060ce15b4cf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:18,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787d575bc060ce15b4cf 2025-12-17 21:07:18,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787d575bc060ce15b4cf HTTP/1.1" 502 150 2025-12-17 21:07:18,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787d575bc060ce15b4cf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:18,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787b575bc060ce15b4c9 2025-12-17 21:07:18,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787b575bc060ce15b4c9 HTTP/1.1" 502 150 2025-12-17 21:07:18,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787b575bc060ce15b4c9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:19,103 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407875575bc060ce15b4b7 2025-12-17 21:07:19,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407875575bc060ce15b4b7 HTTP/1.1" 502 150 2025-12-17 21:07:19,104 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407875575bc060ce15b4b7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:19,295 - lava_callback - INFO - Received callback for node 6940787e575bc060ce15b4d2 INFO: 172.21.0.1:36140 - "POST /node/6940787e575bc060ce15b4d2 HTTP/1.0" 202 Accepted 2025-12-17 21:07:19,301 - lava_callback - INFO - Starting async job submit for node 6940787e575bc060ce15b4d2 2025-12-17 21:07:19,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:19,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787e575bc060ce15b4d2 HTTP/1.1" 502 150 2025-12-17 21:07:19,308 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787e575bc060ce15b4d2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:19,308 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787e575bc060ce15b4d2 2025-12-17 21:07:19,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787e575bc060ce15b4d2 HTTP/1.1" 502 150 2025-12-17 21:07:19,309 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787e575bc060ce15b4d2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:19,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408928575bc060ce15e992 2025-12-17 21:07:19,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408928575bc060ce15e992 HTTP/1.1" 502 150 2025-12-17 21:07:19,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408928575bc060ce15e992'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:19,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e09575bc060ce15c213 HTTP/1.1" 502 150 2025-12-17 21:07:19,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e09575bc060ce15c213'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:19,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e09575bc060ce15c213 2025-12-17 21:07:19,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e09575bc060ce15c213 HTTP/1.1" 502 150 2025-12-17 21:07:19,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e09575bc060ce15c213'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:19,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787c575bc060ce15b4cc 2025-12-17 21:07:19,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787c575bc060ce15b4cc HTTP/1.1" 502 150 2025-12-17 21:07:19,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787c575bc060ce15b4cc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:19,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407876575bc060ce15b4bc 2025-12-17 21:07:19,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407876575bc060ce15b4bc HTTP/1.1" 502 150 2025-12-17 21:07:19,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407876575bc060ce15b4bc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:20,087 - lava_callback - INFO - Received callback for node 69407880575bc060ce15b4d8 INFO: 172.21.0.1:60826 - "POST /node/69407880575bc060ce15b4d8 HTTP/1.0" 202 Accepted 2025-12-17 21:07:20,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787d575bc060ce15b4cf 2025-12-17 21:07:20,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787d575bc060ce15b4cf HTTP/1.1" 502 150 2025-12-17 21:07:20,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787d575bc060ce15b4cf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:20,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407877575bc060ce15b4c0 2025-12-17 21:07:20,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407877575bc060ce15b4c0 HTTP/1.1" 502 150 2025-12-17 21:07:20,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407877575bc060ce15b4c0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:20,841 - lava_callback - INFO - Received callback for node 69407883575bc060ce15b4de INFO: 172.21.0.1:60832 - "POST /node/69407883575bc060ce15b4de HTTP/1.0" 202 Accepted 2025-12-17 21:07:20,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407870575bc060ce15b4a1 2025-12-17 21:07:20,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407870575bc060ce15b4a1 HTTP/1.1" 502 150 2025-12-17 21:07:20,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407870575bc060ce15b4a1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:21,309 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787e575bc060ce15b4d2 2025-12-17 21:07:21,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787e575bc060ce15b4d2 HTTP/1.1" 502 150 2025-12-17 21:07:21,310 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787e575bc060ce15b4d2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:21,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407879575bc060ce15b4c3 2025-12-17 21:07:21,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407879575bc060ce15b4c3 HTTP/1.1" 502 150 2025-12-17 21:07:21,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407879575bc060ce15b4c3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:21,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077b7575bc060ce15b354 2025-12-17 21:07:21,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077b7575bc060ce15b354 HTTP/1.1" 502 150 2025-12-17 21:07:21,469 - lava_callback - ERROR - Error processing callback for node 694077b7575bc060ce15b354: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b7575bc060ce15b354 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b7575bc060ce15b354 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077b7575bc060ce15b354 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:07:21,470 - lava_callback - INFO - Starting async job submit for node 69407880575bc060ce15b4d8 2025-12-17 21:07:21,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:21,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407880575bc060ce15b4d8 HTTP/1.1" 502 150 2025-12-17 21:07:21,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407880575bc060ce15b4d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:21,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407880575bc060ce15b4d8 2025-12-17 21:07:21,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407880575bc060ce15b4d8 HTTP/1.1" 502 150 2025-12-17 21:07:21,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407880575bc060ce15b4d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:21,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e09575bc060ce15c213 2025-12-17 21:07:21,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e09575bc060ce15c213 HTTP/1.1" 502 150 2025-12-17 21:07:21,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e09575bc060ce15c213'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:22,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787b575bc060ce15b4c9 2025-12-17 21:07:22,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787b575bc060ce15b4c9 HTTP/1.1" 502 150 2025-12-17 21:07:22,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787b575bc060ce15b4c9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:23,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407880575bc060ce15b4d8 2025-12-17 21:07:23,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407880575bc060ce15b4d8 HTTP/1.1" 502 150 2025-12-17 21:07:23,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407880575bc060ce15b4d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:23,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787c575bc060ce15b4cc 2025-12-17 21:07:23,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787c575bc060ce15b4cc HTTP/1.1" 502 150 2025-12-17 21:07:23,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787c575bc060ce15b4cc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:24,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787d575bc060ce15b4cf 2025-12-17 21:07:24,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787d575bc060ce15b4cf HTTP/1.1" 502 150 2025-12-17 21:07:24,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787d575bc060ce15b4cf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:25,310 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787e575bc060ce15b4d2 2025-12-17 21:07:25,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787e575bc060ce15b4d2 HTTP/1.1" 502 150 2025-12-17 21:07:25,312 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787e575bc060ce15b4d2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:25,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e09575bc060ce15c213 2025-12-17 21:07:25,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e09575bc060ce15c213 HTTP/1.1" 502 150 2025-12-17 21:07:25,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e09575bc060ce15c213'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:26,498 - lava_callback - INFO - Received callback for node 6940787f575bc060ce15b4d5 INFO: 172.21.0.1:60840 - "POST /node/6940787f575bc060ce15b4d5 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:60844 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:07:27,104 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407875575bc060ce15b4b7 2025-12-17 21:07:27,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407875575bc060ce15b4b7 HTTP/1.1" 502 150 2025-12-17 21:07:27,105 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407875575bc060ce15b4b7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:27,166 - lava_callback - INFO - Received callback for node 69407882575bc060ce15b4db INFO: 172.21.0.1:60852 - "POST /node/69407882575bc060ce15b4db HTTP/1.0" 202 Accepted 2025-12-17 21:07:27,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408928575bc060ce15e992 2025-12-17 21:07:27,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408928575bc060ce15e992 HTTP/1.1" 502 150 2025-12-17 21:07:27,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408928575bc060ce15e992'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:27,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407880575bc060ce15b4d8 2025-12-17 21:07:27,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407880575bc060ce15b4d8 HTTP/1.1" 502 150 2025-12-17 21:07:27,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407880575bc060ce15b4d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:27,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407876575bc060ce15b4bc 2025-12-17 21:07:27,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407876575bc060ce15b4bc HTTP/1.1" 502 150 2025-12-17 21:07:27,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407876575bc060ce15b4bc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:28,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407877575bc060ce15b4c0 2025-12-17 21:07:28,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407877575bc060ce15b4c0 HTTP/1.1" 502 150 2025-12-17 21:07:28,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407877575bc060ce15b4c0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:29,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407879575bc060ce15b4c3 2025-12-17 21:07:29,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407879575bc060ce15b4c3 HTTP/1.1" 502 150 2025-12-17 21:07:29,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407879575bc060ce15b4c3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:29,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407873575bc060ce15b4ac HTTP/1.1" 502 150 2025-12-17 21:07:29,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407873575bc060ce15b4ac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:29,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407446575bc060ce15ae91 2025-12-17 21:07:29,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407446575bc060ce15ae91 HTTP/1.1" 502 150 2025-12-17 21:07:29,807 - lava_callback - ERROR - Error processing callback for node 69407446575bc060ce15ae91: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407446575bc060ce15ae91 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407446575bc060ce15ae91 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407446575bc060ce15ae91 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:07:29,809 - lava_callback - INFO - Starting async job submit for node 69407883575bc060ce15b4de 2025-12-17 21:07:29,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:29,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407883575bc060ce15b4de HTTP/1.1" 502 150 2025-12-17 21:07:29,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407883575bc060ce15b4de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:29,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407883575bc060ce15b4de 2025-12-17 21:07:29,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407883575bc060ce15b4de HTTP/1.1" 502 150 2025-12-17 21:07:29,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407883575bc060ce15b4de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:30,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787b575bc060ce15b4c9 2025-12-17 21:07:30,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787b575bc060ce15b4c9 HTTP/1.1" 502 150 2025-12-17 21:07:30,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787b575bc060ce15b4c9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:31,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694077ba575bc060ce15b35a 2025-12-17 21:07:31,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694077ba575bc060ce15b35a HTTP/1.1" 502 150 2025-12-17 21:07:31,339 - lava_callback - ERROR - Error processing callback for node 694077ba575bc060ce15b35a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077ba575bc060ce15b35a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077ba575bc060ce15b35a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694077ba575bc060ce15b35a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:07:31,341 - lava_callback - INFO - Starting async job submit for node 6940787f575bc060ce15b4d5 2025-12-17 21:07:31,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:31,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787f575bc060ce15b4d5 HTTP/1.1" 502 150 2025-12-17 21:07:31,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787f575bc060ce15b4d5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:31,369 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787f575bc060ce15b4d5 2025-12-17 21:07:31,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787f575bc060ce15b4d5 HTTP/1.1" 502 150 2025-12-17 21:07:31,370 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787f575bc060ce15b4d5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:31,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787a575bc060ce15b4c6 HTTP/1.1" 502 150 2025-12-17 21:07:31,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787a575bc060ce15b4c6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:31,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787a575bc060ce15b4c6 2025-12-17 21:07:31,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787a575bc060ce15b4c6 HTTP/1.1" 502 150 2025-12-17 21:07:31,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787a575bc060ce15b4c6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:31,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787c575bc060ce15b4cc 2025-12-17 21:07:31,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787c575bc060ce15b4cc HTTP/1.1" 502 150 2025-12-17 21:07:31,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787c575bc060ce15b4cc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:31,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407883575bc060ce15b4de 2025-12-17 21:07:31,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407883575bc060ce15b4de HTTP/1.1" 502 150 2025-12-17 21:07:31,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407883575bc060ce15b4de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:32,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787d575bc060ce15b4cf 2025-12-17 21:07:32,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787d575bc060ce15b4cf HTTP/1.1" 502 150 2025-12-17 21:07:32,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787d575bc060ce15b4cf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:33,312 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787e575bc060ce15b4d2 2025-12-17 21:07:33,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787e575bc060ce15b4d2 HTTP/1.1" 502 150 2025-12-17 21:07:33,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787e575bc060ce15b4d2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:33,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787f575bc060ce15b4d5 2025-12-17 21:07:33,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787f575bc060ce15b4d5 HTTP/1.1" 502 150 2025-12-17 21:07:33,371 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787f575bc060ce15b4d5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:33,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407873575bc060ce15b4ac 2025-12-17 21:07:33,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407873575bc060ce15b4ac HTTP/1.1" 502 150 2025-12-17 21:07:33,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407873575bc060ce15b4ac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:33,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e09575bc060ce15c213 2025-12-17 21:07:33,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e09575bc060ce15c213 HTTP/1.1" 502 150 2025-12-17 21:07:33,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e09575bc060ce15c213'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:33,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787a575bc060ce15b4c6 2025-12-17 21:07:33,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787a575bc060ce15b4c6 HTTP/1.1" 502 150 2025-12-17 21:07:33,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787a575bc060ce15b4c6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:33,731 - lava_callback - INFO - Received callback for node 69408939575bc060ce15e9a6 INFO: 172.21.0.1:34082 - "POST /node/69408939575bc060ce15e9a6 HTTP/1.0" 202 Accepted 2025-12-17 21:07:35,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407880575bc060ce15b4d8 2025-12-17 21:07:35,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407880575bc060ce15b4d8 HTTP/1.1" 502 150 2025-12-17 21:07:35,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407880575bc060ce15b4d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:35,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407883575bc060ce15b4de 2025-12-17 21:07:35,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407883575bc060ce15b4de HTTP/1.1" 502 150 2025-12-17 21:07:35,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407883575bc060ce15b4de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34096 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:07:36,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407870575bc060ce15b4a1 2025-12-17 21:07:36,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407870575bc060ce15b4a1 HTTP/1.1" 502 150 2025-12-17 21:07:36,850 - lava_callback - ERROR - Error processing callback for node 69407870575bc060ce15b4a1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407870575bc060ce15b4a1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407870575bc060ce15b4a1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407870575bc060ce15b4a1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:07:36,852 - lava_callback - INFO - Starting async job submit for node 69407882575bc060ce15b4db 2025-12-17 21:07:36,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:36,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407882575bc060ce15b4db HTTP/1.1" 502 150 2025-12-17 21:07:36,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407882575bc060ce15b4db'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:36,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407882575bc060ce15b4db 2025-12-17 21:07:36,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407882575bc060ce15b4db HTTP/1.1" 502 150 2025-12-17 21:07:36,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407882575bc060ce15b4db'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:37,371 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787f575bc060ce15b4d5 2025-12-17 21:07:37,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787f575bc060ce15b4d5 HTTP/1.1" 502 150 2025-12-17 21:07:37,373 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787f575bc060ce15b4d5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:37,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787a575bc060ce15b4c6 2025-12-17 21:07:37,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787a575bc060ce15b4c6 HTTP/1.1" 502 150 2025-12-17 21:07:37,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787a575bc060ce15b4c6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:38,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407882575bc060ce15b4db 2025-12-17 21:07:38,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407882575bc060ce15b4db HTTP/1.1" 502 150 2025-12-17 21:07:38,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407882575bc060ce15b4db'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:41,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407873575bc060ce15b4ac 2025-12-17 21:07:41,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407873575bc060ce15b4ac HTTP/1.1" 502 150 2025-12-17 21:07:41,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407873575bc060ce15b4ac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:42,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407882575bc060ce15b4db 2025-12-17 21:07:42,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407882575bc060ce15b4db HTTP/1.1" 502 150 2025-12-17 21:07:42,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407882575bc060ce15b4db'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:43,106 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407875575bc060ce15b4b7 2025-12-17 21:07:43,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407875575bc060ce15b4b7 HTTP/1.1" 502 150 2025-12-17 21:07:43,107 - lava_callback - ERROR - Error processing callback for node 69407875575bc060ce15b4b7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407875575bc060ce15b4b7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407875575bc060ce15b4b7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407875575bc060ce15b4b7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:07:43,109 - lava_callback - INFO - Starting async job submit for node 69408939575bc060ce15e9a6 2025-12-17 21:07:43,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:43,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408939575bc060ce15e9a6 HTTP/1.1" 502 150 2025-12-17 21:07:43,187 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408939575bc060ce15e9a6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:43,187 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408939575bc060ce15e9a6 2025-12-17 21:07:43,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408939575bc060ce15e9a6 HTTP/1.1" 502 150 2025-12-17 21:07:43,189 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408939575bc060ce15e9a6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:43,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408928575bc060ce15e992 2025-12-17 21:07:43,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408928575bc060ce15e992 HTTP/1.1" 502 150 2025-12-17 21:07:43,475 - lava_callback - ERROR - Error processing callback for node 69408928575bc060ce15e992: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408928575bc060ce15e992 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408928575bc060ce15e992 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408928575bc060ce15e992 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:07:43,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407883575bc060ce15b4de 2025-12-17 21:07:43,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407883575bc060ce15b4de HTTP/1.1" 502 150 2025-12-17 21:07:43,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407883575bc060ce15b4de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:43,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407876575bc060ce15b4bc 2025-12-17 21:07:43,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407876575bc060ce15b4bc HTTP/1.1" 502 150 2025-12-17 21:07:43,896 - lava_callback - ERROR - Error processing callback for node 69407876575bc060ce15b4bc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407876575bc060ce15b4bc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407876575bc060ce15b4bc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407876575bc060ce15b4bc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:07:44,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407877575bc060ce15b4c0 2025-12-17 21:07:44,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407877575bc060ce15b4c0 HTTP/1.1" 502 150 2025-12-17 21:07:44,616 - lava_callback - ERROR - Error processing callback for node 69407877575bc060ce15b4c0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407877575bc060ce15b4c0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407877575bc060ce15b4c0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407877575bc060ce15b4c0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:07:45,189 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408939575bc060ce15e9a6 2025-12-17 21:07:45,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408939575bc060ce15e9a6 HTTP/1.1" 502 150 2025-12-17 21:07:45,190 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408939575bc060ce15e9a6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:45,373 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787f575bc060ce15b4d5 2025-12-17 21:07:45,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787f575bc060ce15b4d5 HTTP/1.1" 502 150 2025-12-17 21:07:45,374 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787f575bc060ce15b4d5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:45,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407879575bc060ce15b4c3 2025-12-17 21:07:45,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407879575bc060ce15b4c3 HTTP/1.1" 502 150 2025-12-17 21:07:45,425 - lava_callback - ERROR - Error processing callback for node 69407879575bc060ce15b4c3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407879575bc060ce15b4c3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407879575bc060ce15b4c3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407879575bc060ce15b4c3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:07:45,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787a575bc060ce15b4c6 2025-12-17 21:07:45,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787a575bc060ce15b4c6 HTTP/1.1" 502 150 2025-12-17 21:07:45,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940787a575bc060ce15b4c6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57448 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:07:46,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787b575bc060ce15b4c9 2025-12-17 21:07:46,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787b575bc060ce15b4c9 HTTP/1.1" 502 150 2025-12-17 21:07:46,939 - lava_callback - ERROR - Error processing callback for node 6940787b575bc060ce15b4c9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787b575bc060ce15b4c9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787b575bc060ce15b4c9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787b575bc060ce15b4c9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:07:47,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787c575bc060ce15b4cc 2025-12-17 21:07:47,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787c575bc060ce15b4cc HTTP/1.1" 502 150 2025-12-17 21:07:47,781 - lava_callback - ERROR - Error processing callback for node 6940787c575bc060ce15b4cc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787c575bc060ce15b4cc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787c575bc060ce15b4cc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787c575bc060ce15b4cc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:07:48,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787d575bc060ce15b4cf 2025-12-17 21:07:48,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787d575bc060ce15b4cf HTTP/1.1" 502 150 2025-12-17 21:07:48,581 - lava_callback - ERROR - Error processing callback for node 6940787d575bc060ce15b4cf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787d575bc060ce15b4cf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787d575bc060ce15b4cf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787d575bc060ce15b4cf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:07:49,190 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408939575bc060ce15e9a6 2025-12-17 21:07:49,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408939575bc060ce15e9a6 HTTP/1.1" 502 150 2025-12-17 21:07:49,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408939575bc060ce15e9a6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:49,313 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787e575bc060ce15b4d2 2025-12-17 21:07:49,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787e575bc060ce15b4d2 HTTP/1.1" 502 150 2025-12-17 21:07:49,315 - lava_callback - ERROR - Error processing callback for node 6940787e575bc060ce15b4d2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787e575bc060ce15b4d2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787e575bc060ce15b4d2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787e575bc060ce15b4d2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:07:49,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e09575bc060ce15c213 2025-12-17 21:07:49,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e09575bc060ce15c213 HTTP/1.1" 502 150 2025-12-17 21:07:49,606 - lava_callback - ERROR - Error processing callback for node 69407e09575bc060ce15c213: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e09575bc060ce15c213 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e09575bc060ce15c213 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e09575bc060ce15c213 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:07:50,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407882575bc060ce15b4db 2025-12-17 21:07:50,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407882575bc060ce15b4db HTTP/1.1" 502 150 2025-12-17 21:07:50,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407882575bc060ce15b4db'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:51,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407880575bc060ce15b4d8 2025-12-17 21:07:56,454 - lava_callback - INFO - Received callback for node 69408a94575bc060ce15eee4 INFO: 172.21.0.1:44020 - "POST /node/69408a94575bc060ce15eee4 HTTP/1.0" 202 Accepted 2025-12-17 21:07:56,463 - lava_callback - INFO - Starting async job submit for node 69408a94575bc060ce15eee4 INFO: 172.21.0.1:44022 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:07:56,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:07:56,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a94575bc060ce15eee4 HTTP/1.1" 502 150 2025-12-17 21:07:56,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a94575bc060ce15eee4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:56,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a94575bc060ce15eee4 2025-12-17 21:07:56,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a94575bc060ce15eee4 HTTP/1.1" 502 150 2025-12-17 21:07:56,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a94575bc060ce15eee4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:57,192 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408939575bc060ce15e9a6 2025-12-17 21:07:57,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408939575bc060ce15e9a6 HTTP/1.1" 502 150 2025-12-17 21:07:57,193 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408939575bc060ce15e9a6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:57,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407873575bc060ce15b4ac 2025-12-17 21:07:57,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407873575bc060ce15b4ac HTTP/1.1" 502 150 2025-12-17 21:07:57,589 - lava_callback - ERROR - Error processing callback for node 69407873575bc060ce15b4ac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407873575bc060ce15b4ac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407873575bc060ce15b4ac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407873575bc060ce15b4ac (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:44026 - "GET /node/693a81a4e17cfbe560b12c8a%5Cn HTTP/1.0" 405 Method Not Allowed 2025-12-17 21:07:58,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a94575bc060ce15eee4 2025-12-17 21:07:58,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a94575bc060ce15eee4 HTTP/1.1" 502 150 2025-12-17 21:07:58,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a94575bc060ce15eee4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:07:59,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407883575bc060ce15b4de 2025-12-17 21:07:59,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407883575bc060ce15b4de HTTP/1.1" 502 150 2025-12-17 21:07:59,825 - lava_callback - ERROR - Error processing callback for node 69407883575bc060ce15b4de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407883575bc060ce15b4de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407883575bc060ce15b4de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407883575bc060ce15b4de (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:43474 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:08:01,374 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787f575bc060ce15b4d5 2025-12-17 21:08:01,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787f575bc060ce15b4d5 HTTP/1.1" 502 150 2025-12-17 21:08:01,376 - lava_callback - ERROR - Error processing callback for node 6940787f575bc060ce15b4d5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787f575bc060ce15b4d5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787f575bc060ce15b4d5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787f575bc060ce15b4d5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:01,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940787a575bc060ce15b4c6 2025-12-17 21:08:01,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940787a575bc060ce15b4c6 HTTP/1.1" 502 150 2025-12-17 21:08:01,639 - lava_callback - ERROR - Error processing callback for node 6940787a575bc060ce15b4c6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787a575bc060ce15b4c6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787a575bc060ce15b4c6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940787a575bc060ce15b4c6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:02,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a94575bc060ce15eee4 2025-12-17 21:08:02,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a94575bc060ce15eee4 HTTP/1.1" 502 150 2025-12-17 21:08:02,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a94575bc060ce15eee4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43480 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:08:06,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407882575bc060ce15b4db 2025-12-17 21:08:06,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407880575bc060ce15b4d8 HTTP/1.1" 502 150 2025-12-17 21:08:06,960 - lava_callback - ERROR - Error processing callback for node 69407880575bc060ce15b4d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407880575bc060ce15b4d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407880575bc060ce15b4d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407880575bc060ce15b4d8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:09,775 - lava_callback - INFO - Received callback for node 694078ad575bc060ce15b53d INFO: 172.21.0.1:36964 - "POST /node/694078ad575bc060ce15b53d HTTP/1.0" 202 Accepted 2025-12-17 21:08:09,784 - lava_callback - INFO - Starting async job submit for node 694078ad575bc060ce15b53d 2025-12-17 21:08:09,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:09,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ad575bc060ce15b53d HTTP/1.1" 502 150 2025-12-17 21:08:09,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ad575bc060ce15b53d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:09,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ad575bc060ce15b53d 2025-12-17 21:08:09,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ad575bc060ce15b53d HTTP/1.1" 502 150 2025-12-17 21:08:09,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ad575bc060ce15b53d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:10,383 - lava_callback - INFO - Received callback for node 694078b0575bc060ce15b547 INFO: 172.21.0.1:36972 - "POST /node/694078b0575bc060ce15b547 HTTP/1.0" 202 Accepted 2025-12-17 21:08:10,389 - lava_callback - INFO - Starting async job submit for node 694078b0575bc060ce15b547 2025-12-17 21:08:10,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:10,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b0575bc060ce15b547 HTTP/1.1" 502 150 2025-12-17 21:08:10,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b0575bc060ce15b547'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:10,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b0575bc060ce15b547 2025-12-17 21:08:10,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b0575bc060ce15b547 HTTP/1.1" 502 150 2025-12-17 21:08:10,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b0575bc060ce15b547'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:10,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a94575bc060ce15eee4 2025-12-17 21:08:10,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a94575bc060ce15eee4 HTTP/1.1" 502 150 2025-12-17 21:08:10,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a94575bc060ce15eee4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:11,004 - lava_callback - INFO - Received callback for node 694078b1575bc060ce15b54f INFO: 172.21.0.1:36988 - "POST /node/694078b1575bc060ce15b54f HTTP/1.0" 202 Accepted 2025-12-17 21:08:11,014 - lava_callback - INFO - Starting async job submit for node 694078b1575bc060ce15b54f 2025-12-17 21:08:11,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:11,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b1575bc060ce15b54f HTTP/1.1" 502 150 2025-12-17 21:08:11,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b1575bc060ce15b54f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:11,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b1575bc060ce15b54f 2025-12-17 21:08:11,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b1575bc060ce15b54f HTTP/1.1" 502 150 2025-12-17 21:08:11,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b1575bc060ce15b54f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:11,031 - lava_callback - INFO - Received callback for node 694078a8575bc060ce15b52e INFO: 172.21.0.1:36998 - "POST /node/694078a8575bc060ce15b52e HTTP/1.0" 202 Accepted 2025-12-17 21:08:11,040 - lava_callback - INFO - Starting async job submit for node 694078a8575bc060ce15b52e 2025-12-17 21:08:11,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:11,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078a8575bc060ce15b52e HTTP/1.1" 502 150 2025-12-17 21:08:11,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078a8575bc060ce15b52e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:11,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078a8575bc060ce15b52e 2025-12-17 21:08:11,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078a8575bc060ce15b52e HTTP/1.1" 502 150 2025-12-17 21:08:11,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078a8575bc060ce15b52e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:11,801 - lava_callback - INFO - Received callback for node 694078a9575bc060ce15b534 INFO: 172.21.0.1:37006 - "POST /node/694078a9575bc060ce15b534 HTTP/1.0" 202 Accepted 2025-12-17 21:08:11,808 - lava_callback - INFO - Starting async job submit for node 694078a9575bc060ce15b534 2025-12-17 21:08:11,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:11,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ad575bc060ce15b53d 2025-12-17 21:08:11,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ad575bc060ce15b53d HTTP/1.1" 502 150 2025-12-17 21:08:11,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ad575bc060ce15b53d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:11,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078a9575bc060ce15b534 HTTP/1.1" 502 150 2025-12-17 21:08:11,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078a9575bc060ce15b534'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:11,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078a9575bc060ce15b534 2025-12-17 21:08:11,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078a9575bc060ce15b534 HTTP/1.1" 502 150 2025-12-17 21:08:11,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078a9575bc060ce15b534'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:12,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b0575bc060ce15b547 2025-12-17 21:08:12,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b0575bc060ce15b547 HTTP/1.1" 502 150 2025-12-17 21:08:12,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b0575bc060ce15b547'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:12,538 - lava_callback - INFO - Received callback for node 694078ab575bc060ce15b537 INFO: 172.21.0.1:37016 - "POST /node/694078ab575bc060ce15b537 HTTP/1.0" 202 Accepted 2025-12-17 21:08:12,547 - lava_callback - INFO - Starting async job submit for node 694078ab575bc060ce15b537 2025-12-17 21:08:12,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:12,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ab575bc060ce15b537 HTTP/1.1" 502 150 2025-12-17 21:08:12,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ab575bc060ce15b537'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:12,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ab575bc060ce15b537 2025-12-17 21:08:12,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ab575bc060ce15b537 HTTP/1.1" 502 150 2025-12-17 21:08:12,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ab575bc060ce15b537'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:13,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b1575bc060ce15b54f 2025-12-17 21:08:13,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b1575bc060ce15b54f HTTP/1.1" 502 150 2025-12-17 21:08:13,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b1575bc060ce15b54f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:13,049 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078a8575bc060ce15b52e 2025-12-17 21:08:13,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078a8575bc060ce15b52e HTTP/1.1" 502 150 2025-12-17 21:08:13,051 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078a8575bc060ce15b52e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:13,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408939575bc060ce15e9a6 2025-12-17 21:08:13,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408939575bc060ce15e9a6 HTTP/1.1" 502 150 2025-12-17 21:08:13,195 - lava_callback - ERROR - Error processing callback for node 69408939575bc060ce15e9a6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408939575bc060ce15e9a6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408939575bc060ce15e9a6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408939575bc060ce15e9a6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:13,378 - lava_callback - INFO - Received callback for node 694078ac575bc060ce15b53a INFO: 172.21.0.1:37032 - "POST /node/694078ac575bc060ce15b53a HTTP/1.0" 202 Accepted 2025-12-17 21:08:13,388 - lava_callback - INFO - Starting async job submit for node 694078ac575bc060ce15b53a 2025-12-17 21:08:13,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:13,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ac575bc060ce15b53a HTTP/1.1" 502 150 2025-12-17 21:08:13,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ac575bc060ce15b53a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:13,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ac575bc060ce15b53a 2025-12-17 21:08:13,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ac575bc060ce15b53a HTTP/1.1" 502 150 2025-12-17 21:08:13,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ac575bc060ce15b53a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:13,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078a9575bc060ce15b534 2025-12-17 21:08:13,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078a9575bc060ce15b534 HTTP/1.1" 502 150 2025-12-17 21:08:13,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078a9575bc060ce15b534'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:14,120 - lava_callback - INFO - Received callback for node 694078af575bc060ce15b541 INFO: 172.21.0.1:37040 - "POST /node/694078af575bc060ce15b541 HTTP/1.0" 202 Accepted 2025-12-17 21:08:14,129 - lava_callback - INFO - Starting async job submit for node 694078af575bc060ce15b541 2025-12-17 21:08:14,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:14,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078af575bc060ce15b541 HTTP/1.1" 502 150 2025-12-17 21:08:14,136 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078af575bc060ce15b541'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:14,136 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078af575bc060ce15b541 2025-12-17 21:08:14,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078af575bc060ce15b541 HTTP/1.1" 502 150 2025-12-17 21:08:14,137 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078af575bc060ce15b541'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:14,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ab575bc060ce15b537 2025-12-17 21:08:14,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ab575bc060ce15b537 HTTP/1.1" 502 150 2025-12-17 21:08:14,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ab575bc060ce15b537'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:14,606 - lava_callback - INFO - Received callback for node 6940764e575bc060ce15b08c INFO: 172.21.0.1:37050 - "POST /node/6940764e575bc060ce15b08c HTTP/1.0" 202 Accepted 2025-12-17 21:08:14,657 - lava_callback - INFO - Starting async job submit for node 6940764e575bc060ce15b08c 2025-12-17 21:08:14,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:14,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940764e575bc060ce15b08c HTTP/1.1" 502 150 2025-12-17 21:08:14,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940764e575bc060ce15b08c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:14,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940764e575bc060ce15b08c 2025-12-17 21:08:14,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940764e575bc060ce15b08c HTTP/1.1" 502 150 2025-12-17 21:08:14,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940764e575bc060ce15b08c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:14,867 - lava_callback - INFO - Received callback for node 694078b3575bc060ce15b553 INFO: 172.21.0.1:37066 - "POST /node/694078b3575bc060ce15b553 HTTP/1.0" 202 Accepted 2025-12-17 21:08:14,873 - lava_callback - INFO - Starting async job submit for node 694078b3575bc060ce15b553 2025-12-17 21:08:14,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:14,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b3575bc060ce15b553 HTTP/1.1" 502 150 2025-12-17 21:08:14,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b3575bc060ce15b553'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:14,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b3575bc060ce15b553 2025-12-17 21:08:14,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b3575bc060ce15b553 HTTP/1.1" 502 150 2025-12-17 21:08:14,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b3575bc060ce15b553'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37080 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:08:15,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ac575bc060ce15b53a 2025-12-17 21:08:15,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ac575bc060ce15b53a HTTP/1.1" 502 150 2025-12-17 21:08:15,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ac575bc060ce15b53a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:15,657 - lava_callback - INFO - Received callback for node 694078b4575bc060ce15b556 INFO: 172.21.0.1:37094 - "POST /node/694078b4575bc060ce15b556 HTTP/1.0" 202 Accepted 2025-12-17 21:08:15,663 - lava_callback - INFO - Starting async job submit for node 694078b4575bc060ce15b556 2025-12-17 21:08:15,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:15,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b4575bc060ce15b556 HTTP/1.1" 502 150 2025-12-17 21:08:15,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b4575bc060ce15b556'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:15,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b4575bc060ce15b556 2025-12-17 21:08:15,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b4575bc060ce15b556 HTTP/1.1" 502 150 2025-12-17 21:08:15,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b4575bc060ce15b556'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:15,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ad575bc060ce15b53d 2025-12-17 21:08:15,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ad575bc060ce15b53d HTTP/1.1" 502 150 2025-12-17 21:08:15,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ad575bc060ce15b53d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:16,137 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078af575bc060ce15b541 2025-12-17 21:08:16,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078af575bc060ce15b541 HTTP/1.1" 502 150 2025-12-17 21:08:16,138 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078af575bc060ce15b541'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:16,376 - lava_callback - INFO - Received callback for node 694078b6575bc060ce15b559 INFO: 172.21.0.1:37106 - "POST /node/694078b6575bc060ce15b559 HTTP/1.0" 202 Accepted 2025-12-17 21:08:16,427 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b0575bc060ce15b547 2025-12-17 21:08:16,427 - lava_callback - INFO - Starting async job submit for node 694078b6575bc060ce15b559 2025-12-17 21:08:16,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:16,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b0575bc060ce15b547 HTTP/1.1" 502 150 2025-12-17 21:08:16,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b0575bc060ce15b547'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:16,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b6575bc060ce15b559 HTTP/1.1" 502 150 2025-12-17 21:08:16,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b6575bc060ce15b559'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:16,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b6575bc060ce15b559 2025-12-17 21:08:16,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b6575bc060ce15b559 HTTP/1.1" 502 150 2025-12-17 21:08:16,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b6575bc060ce15b559'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37120 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:08:16,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940764e575bc060ce15b08c 2025-12-17 21:08:16,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940764e575bc060ce15b08c HTTP/1.1" 502 150 2025-12-17 21:08:16,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940764e575bc060ce15b08c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:16,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b3575bc060ce15b553 2025-12-17 21:08:16,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b3575bc060ce15b553 HTTP/1.1" 502 150 2025-12-17 21:08:16,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b3575bc060ce15b553'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:17,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b1575bc060ce15b54f 2025-12-17 21:08:17,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078a8575bc060ce15b52e 2025-12-17 21:08:17,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078a8575bc060ce15b52e HTTP/1.1" 502 150 2025-12-17 21:08:17,052 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078a8575bc060ce15b52e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:17,179 - lava_callback - INFO - Received callback for node 694078b7575bc060ce15b55c INFO: 172.21.0.1:37122 - "POST /node/694078b7575bc060ce15b55c HTTP/1.0" 202 Accepted 2025-12-17 21:08:17,185 - lava_callback - INFO - Starting async job submit for node 694078b7575bc060ce15b55c 2025-12-17 21:08:17,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:17,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b7575bc060ce15b55c HTTP/1.1" 502 150 2025-12-17 21:08:17,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b7575bc060ce15b55c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:17,192 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b7575bc060ce15b55c 2025-12-17 21:08:17,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b7575bc060ce15b55c HTTP/1.1" 502 150 2025-12-17 21:08:17,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b7575bc060ce15b55c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:17,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b4575bc060ce15b556 2025-12-17 21:08:17,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b4575bc060ce15b556 HTTP/1.1" 502 150 2025-12-17 21:08:17,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b4575bc060ce15b556'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:17,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078a9575bc060ce15b534 2025-12-17 21:08:17,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078a9575bc060ce15b534 HTTP/1.1" 502 150 2025-12-17 21:08:17,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078a9575bc060ce15b534'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:17,932 - lava_callback - INFO - Received callback for node 694078b8575bc060ce15b55f INFO: 172.21.0.1:37134 - "POST /node/694078b8575bc060ce15b55f HTTP/1.0" 202 Accepted 2025-12-17 21:08:17,938 - lava_callback - INFO - Starting async job submit for node 694078b8575bc060ce15b55f 2025-12-17 21:08:17,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:17,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b8575bc060ce15b55f HTTP/1.1" 502 150 2025-12-17 21:08:17,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b8575bc060ce15b55f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:17,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b8575bc060ce15b55f 2025-12-17 21:08:17,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b8575bc060ce15b55f HTTP/1.1" 502 150 2025-12-17 21:08:17,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b8575bc060ce15b55f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:18,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b6575bc060ce15b559 2025-12-17 21:08:18,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b6575bc060ce15b559 HTTP/1.1" 502 150 2025-12-17 21:08:18,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b6575bc060ce15b559'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:18,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ab575bc060ce15b537 2025-12-17 21:08:18,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ab575bc060ce15b537 HTTP/1.1" 502 150 2025-12-17 21:08:18,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ab575bc060ce15b537'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:18,685 - lava_callback - INFO - Received callback for node 694078ba575bc060ce15b562 INFO: 172.21.0.1:37144 - "POST /node/694078ba575bc060ce15b562 HTTP/1.0" 202 Accepted 2025-12-17 21:08:19,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b7575bc060ce15b55c 2025-12-17 21:08:19,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b7575bc060ce15b55c HTTP/1.1" 502 150 2025-12-17 21:08:19,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b7575bc060ce15b55c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:19,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ac575bc060ce15b53a 2025-12-17 21:08:19,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ac575bc060ce15b53a HTTP/1.1" 502 150 2025-12-17 21:08:19,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ac575bc060ce15b53a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:19,557 - lava_callback - INFO - Received callback for node 694078bb575bc060ce15b565 INFO: 172.21.0.1:37146 - "POST /node/694078bb575bc060ce15b565 HTTP/1.0" 202 Accepted 2025-12-17 21:08:19,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b8575bc060ce15b55f 2025-12-17 21:08:19,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b8575bc060ce15b55f HTTP/1.1" 502 150 2025-12-17 21:08:19,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b8575bc060ce15b55f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:20,139 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078af575bc060ce15b541 2025-12-17 21:08:20,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078af575bc060ce15b541 HTTP/1.1" 502 150 2025-12-17 21:08:20,140 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078af575bc060ce15b541'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:20,284 - lava_callback - INFO - Received callback for node 694078bc575bc060ce15b568 INFO: 172.21.0.1:37078 - "POST /node/694078bc575bc060ce15b568 HTTP/1.0" 202 Accepted 2025-12-17 21:08:20,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940764e575bc060ce15b08c 2025-12-17 21:08:20,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940764e575bc060ce15b08c HTTP/1.1" 502 150 2025-12-17 21:08:20,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940764e575bc060ce15b08c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:20,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b3575bc060ce15b553 2025-12-17 21:08:20,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b3575bc060ce15b553 HTTP/1.1" 502 150 2025-12-17 21:08:20,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b3575bc060ce15b553'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:21,128 - lava_callback - INFO - Received callback for node 694078bd575bc060ce15b56b INFO: 172.21.0.1:37082 - "POST /node/694078bd575bc060ce15b56b HTTP/1.0" 202 Accepted 2025-12-17 21:08:21,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b4575bc060ce15b556 2025-12-17 21:08:21,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b4575bc060ce15b556 HTTP/1.1" 502 150 2025-12-17 21:08:21,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b4575bc060ce15b556'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:21,917 - lava_callback - INFO - Received callback for node 694078bf575bc060ce15b56e INFO: 172.21.0.1:37098 - "POST /node/694078bf575bc060ce15b56e HTTP/1.0" 202 Accepted 2025-12-17 21:08:22,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b6575bc060ce15b559 2025-12-17 21:08:22,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b6575bc060ce15b559 HTTP/1.1" 502 150 2025-12-17 21:08:22,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b6575bc060ce15b559'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:22,726 - lava_callback - INFO - Received callback for node 694078c0575bc060ce15b571 INFO: 172.21.0.1:37110 - "POST /node/694078c0575bc060ce15b571 HTTP/1.0" 202 Accepted 2025-12-17 21:08:23,194 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b7575bc060ce15b55c 2025-12-17 21:08:23,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b7575bc060ce15b55c HTTP/1.1" 502 150 2025-12-17 21:08:23,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b7575bc060ce15b55c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:23,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ad575bc060ce15b53d 2025-12-17 21:08:23,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ad575bc060ce15b53d HTTP/1.1" 502 150 2025-12-17 21:08:23,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ad575bc060ce15b53d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:23,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b8575bc060ce15b55f 2025-12-17 21:08:23,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b8575bc060ce15b55f HTTP/1.1" 502 150 2025-12-17 21:08:23,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b8575bc060ce15b55f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:24,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b0575bc060ce15b547 2025-12-17 21:08:24,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b0575bc060ce15b547 HTTP/1.1" 502 150 2025-12-17 21:08:24,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b0575bc060ce15b547'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:25,052 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078a8575bc060ce15b52e 2025-12-17 21:08:25,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078a8575bc060ce15b52e HTTP/1.1" 502 150 2025-12-17 21:08:25,053 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078a8575bc060ce15b52e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:25,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078a9575bc060ce15b534 2025-12-17 21:08:25,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078a9575bc060ce15b534 HTTP/1.1" 502 150 2025-12-17 21:08:25,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078a9575bc060ce15b534'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37126 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:08:26,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a94575bc060ce15eee4 2025-12-17 21:08:26,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a94575bc060ce15eee4 HTTP/1.1" 502 150 2025-12-17 21:08:26,549 - lava_callback - ERROR - Error processing callback for node 69408a94575bc060ce15eee4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a94575bc060ce15eee4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a94575bc060ce15eee4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a94575bc060ce15eee4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:26,552 - lava_callback - INFO - Starting async job submit for node 694078ba575bc060ce15b562 2025-12-17 21:08:26,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:26,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ba575bc060ce15b562 HTTP/1.1" 502 150 2025-12-17 21:08:26,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ba575bc060ce15b562'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:26,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ba575bc060ce15b562 2025-12-17 21:08:26,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ba575bc060ce15b562 HTTP/1.1" 502 150 2025-12-17 21:08:26,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ba575bc060ce15b562'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:26,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ab575bc060ce15b537 2025-12-17 21:08:26,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ab575bc060ce15b537 HTTP/1.1" 502 150 2025-12-17 21:08:26,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ab575bc060ce15b537'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:27,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ac575bc060ce15b53a 2025-12-17 21:08:27,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ac575bc060ce15b53a HTTP/1.1" 502 150 2025-12-17 21:08:27,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ac575bc060ce15b53a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:28,140 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078af575bc060ce15b541 2025-12-17 21:08:28,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078af575bc060ce15b541 HTTP/1.1" 502 150 2025-12-17 21:08:28,141 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078af575bc060ce15b541'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:28,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ba575bc060ce15b562 2025-12-17 21:08:28,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ba575bc060ce15b562 HTTP/1.1" 502 150 2025-12-17 21:08:28,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ba575bc060ce15b562'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:28,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940764e575bc060ce15b08c 2025-12-17 21:08:28,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940764e575bc060ce15b08c HTTP/1.1" 502 150 2025-12-17 21:08:28,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940764e575bc060ce15b08c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:28,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b3575bc060ce15b553 2025-12-17 21:08:28,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b3575bc060ce15b553 HTTP/1.1" 502 150 2025-12-17 21:08:28,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b3575bc060ce15b553'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:29,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b4575bc060ce15b556 2025-12-17 21:08:29,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b4575bc060ce15b556 HTTP/1.1" 502 150 2025-12-17 21:08:29,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b4575bc060ce15b556'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:30,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b6575bc060ce15b559 2025-12-17 21:08:30,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b6575bc060ce15b559 HTTP/1.1" 502 150 2025-12-17 21:08:30,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b6575bc060ce15b559'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:31,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b7575bc060ce15b55c 2025-12-17 21:08:31,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b7575bc060ce15b55c HTTP/1.1" 502 150 2025-12-17 21:08:31,197 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b7575bc060ce15b55c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:31,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b8575bc060ce15b55f 2025-12-17 21:08:31,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b8575bc060ce15b55f HTTP/1.1" 502 150 2025-12-17 21:08:31,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b8575bc060ce15b55f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:32,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b1575bc060ce15b54f HTTP/1.1" 502 150 2025-12-17 21:08:32,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b1575bc060ce15b54f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:32,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ba575bc060ce15b562 2025-12-17 21:08:32,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ba575bc060ce15b562 HTTP/1.1" 502 150 2025-12-17 21:08:32,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ba575bc060ce15b562'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35774 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:08:37,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407882575bc060ce15b4db HTTP/1.1" 502 150 2025-12-17 21:08:37,680 - lava_callback - ERROR - Error processing callback for node 69407882575bc060ce15b4db: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407882575bc060ce15b4db (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407882575bc060ce15b4db (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407882575bc060ce15b4db (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:37,682 - lava_callback - INFO - Starting async job submit for node 694078bb575bc060ce15b565 2025-12-17 21:08:37,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:37,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bb575bc060ce15b565 HTTP/1.1" 502 150 2025-12-17 21:08:37,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bb575bc060ce15b565'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:37,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bb575bc060ce15b565 2025-12-17 21:08:37,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bb575bc060ce15b565 HTTP/1.1" 502 150 2025-12-17 21:08:37,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bb575bc060ce15b565'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:39,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bb575bc060ce15b565 2025-12-17 21:08:39,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bb575bc060ce15b565 HTTP/1.1" 502 150 2025-12-17 21:08:39,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bb575bc060ce15b565'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:39,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ad575bc060ce15b53d 2025-12-17 21:08:39,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ad575bc060ce15b53d HTTP/1.1" 502 150 2025-12-17 21:08:39,826 - lava_callback - ERROR - Error processing callback for node 694078ad575bc060ce15b53d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ad575bc060ce15b53d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ad575bc060ce15b53d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ad575bc060ce15b53d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:39,828 - lava_callback - INFO - Starting async job submit for node 694078bc575bc060ce15b568 2025-12-17 21:08:39,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:39,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bc575bc060ce15b568 HTTP/1.1" 502 150 2025-12-17 21:08:39,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bc575bc060ce15b568'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:39,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bc575bc060ce15b568 2025-12-17 21:08:39,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bc575bc060ce15b568 HTTP/1.1" 502 150 2025-12-17 21:08:39,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bc575bc060ce15b568'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:40,304 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b1575bc060ce15b54f 2025-12-17 21:08:40,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b1575bc060ce15b54f HTTP/1.1" 502 150 2025-12-17 21:08:40,306 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078b1575bc060ce15b54f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:40,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b0575bc060ce15b547 2025-12-17 21:08:40,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b0575bc060ce15b547 HTTP/1.1" 502 150 2025-12-17 21:08:40,433 - lava_callback - ERROR - Error processing callback for node 694078b0575bc060ce15b547: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b0575bc060ce15b547 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b0575bc060ce15b547 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b0575bc060ce15b547 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:40,435 - lava_callback - INFO - Starting async job submit for node 694078bd575bc060ce15b56b 2025-12-17 21:08:40,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:40,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bd575bc060ce15b56b HTTP/1.1" 502 150 2025-12-17 21:08:40,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bd575bc060ce15b56b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:40,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bd575bc060ce15b56b 2025-12-17 21:08:40,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bd575bc060ce15b56b HTTP/1.1" 502 150 2025-12-17 21:08:40,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bd575bc060ce15b56b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:40,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ba575bc060ce15b562 2025-12-17 21:08:40,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ba575bc060ce15b562 HTTP/1.1" 502 150 2025-12-17 21:08:40,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ba575bc060ce15b562'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:41,054 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078a8575bc060ce15b52e 2025-12-17 21:08:41,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078a8575bc060ce15b52e HTTP/1.1" 502 150 2025-12-17 21:08:41,055 - lava_callback - ERROR - Error processing callback for node 694078a8575bc060ce15b52e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078a8575bc060ce15b52e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078a8575bc060ce15b52e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078a8575bc060ce15b52e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:41,057 - lava_callback - INFO - Starting async job submit for node 694078bf575bc060ce15b56e 2025-12-17 21:08:41,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:41,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bf575bc060ce15b56e HTTP/1.1" 502 150 2025-12-17 21:08:41,064 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bf575bc060ce15b56e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:41,065 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bf575bc060ce15b56e 2025-12-17 21:08:41,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bf575bc060ce15b56e HTTP/1.1" 502 150 2025-12-17 21:08:41,065 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bf575bc060ce15b56e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:41,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078a9575bc060ce15b534 2025-12-17 21:08:41,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078a9575bc060ce15b534 HTTP/1.1" 502 150 2025-12-17 21:08:41,834 - lava_callback - ERROR - Error processing callback for node 694078a9575bc060ce15b534: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078a9575bc060ce15b534 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078a9575bc060ce15b534 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078a9575bc060ce15b534 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:41,836 - lava_callback - INFO - Starting async job submit for node 694078c0575bc060ce15b571 2025-12-17 21:08:41,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:41,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078c0575bc060ce15b571 HTTP/1.1" 502 150 2025-12-17 21:08:41,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078c0575bc060ce15b571'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:41,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078c0575bc060ce15b571 2025-12-17 21:08:41,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bc575bc060ce15b568 2025-12-17 21:08:41,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078c0575bc060ce15b571 HTTP/1.1" 502 150 2025-12-17 21:08:41,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078c0575bc060ce15b571'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:41,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bc575bc060ce15b568 HTTP/1.1" 502 150 2025-12-17 21:08:41,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bc575bc060ce15b568'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:42,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bd575bc060ce15b56b 2025-12-17 21:08:42,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bd575bc060ce15b56b HTTP/1.1" 502 150 2025-12-17 21:08:42,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bd575bc060ce15b56b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:42,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ab575bc060ce15b537 2025-12-17 21:08:42,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ab575bc060ce15b537 HTTP/1.1" 502 150 2025-12-17 21:08:42,587 - lava_callback - ERROR - Error processing callback for node 694078ab575bc060ce15b537: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ab575bc060ce15b537 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ab575bc060ce15b537 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ab575bc060ce15b537 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:43,066 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bf575bc060ce15b56e 2025-12-17 21:08:43,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ac575bc060ce15b53a 2025-12-17 21:08:43,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ac575bc060ce15b53a HTTP/1.1" 502 150 2025-12-17 21:08:43,406 - lava_callback - ERROR - Error processing callback for node 694078ac575bc060ce15b53a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ac575bc060ce15b53a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ac575bc060ce15b53a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ac575bc060ce15b53a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:43,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bb575bc060ce15b565 2025-12-17 21:08:43,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bb575bc060ce15b565 HTTP/1.1" 502 150 2025-12-17 21:08:43,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bb575bc060ce15b565'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:43,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078c0575bc060ce15b571 2025-12-17 21:08:43,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078c0575bc060ce15b571 HTTP/1.1" 502 150 2025-12-17 21:08:43,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078c0575bc060ce15b571'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:44,141 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078af575bc060ce15b541 2025-12-17 21:08:44,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078af575bc060ce15b541 HTTP/1.1" 502 150 2025-12-17 21:08:44,143 - lava_callback - ERROR - Error processing callback for node 694078af575bc060ce15b541: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078af575bc060ce15b541 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078af575bc060ce15b541 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078af575bc060ce15b541 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:44,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940764e575bc060ce15b08c 2025-12-17 21:08:44,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940764e575bc060ce15b08c HTTP/1.1" 502 150 2025-12-17 21:08:44,726 - lava_callback - ERROR - Error processing callback for node 6940764e575bc060ce15b08c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940764e575bc060ce15b08c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940764e575bc060ce15b08c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940764e575bc060ce15b08c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:44,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b3575bc060ce15b553 2025-12-17 21:08:44,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b3575bc060ce15b553 HTTP/1.1" 502 150 2025-12-17 21:08:44,887 - lava_callback - ERROR - Error processing callback for node 694078b3575bc060ce15b553: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b3575bc060ce15b553 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b3575bc060ce15b553 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b3575bc060ce15b553 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:45,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b4575bc060ce15b556 2025-12-17 21:08:45,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b4575bc060ce15b556 HTTP/1.1" 502 150 2025-12-17 21:08:45,681 - lava_callback - ERROR - Error processing callback for node 694078b4575bc060ce15b556: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b4575bc060ce15b556 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b4575bc060ce15b556 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b4575bc060ce15b556 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:45,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bc575bc060ce15b568 2025-12-17 21:08:45,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bc575bc060ce15b568 HTTP/1.1" 502 150 2025-12-17 21:08:45,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bc575bc060ce15b568'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:46,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b6575bc060ce15b559 2025-12-17 21:08:46,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b6575bc060ce15b559 HTTP/1.1" 502 150 2025-12-17 21:08:46,442 - lava_callback - ERROR - Error processing callback for node 694078b6575bc060ce15b559: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b6575bc060ce15b559 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b6575bc060ce15b559 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b6575bc060ce15b559 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:46,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bd575bc060ce15b56b 2025-12-17 21:08:46,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bd575bc060ce15b56b HTTP/1.1" 502 150 2025-12-17 21:08:46,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bd575bc060ce15b56b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41308 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:08:47,205 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b7575bc060ce15b55c 2025-12-17 21:08:47,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b7575bc060ce15b55c HTTP/1.1" 502 150 2025-12-17 21:08:47,208 - lava_callback - ERROR - Error processing callback for node 694078b7575bc060ce15b55c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b7575bc060ce15b55c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b7575bc060ce15b55c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b7575bc060ce15b55c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:47,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078c0575bc060ce15b571 2025-12-17 21:08:47,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078c0575bc060ce15b571 HTTP/1.1" 502 150 2025-12-17 21:08:47,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078c0575bc060ce15b571'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:47,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b8575bc060ce15b55f 2025-12-17 21:08:47,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b8575bc060ce15b55f HTTP/1.1" 502 150 2025-12-17 21:08:47,956 - lava_callback - ERROR - Error processing callback for node 694078b8575bc060ce15b55f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b8575bc060ce15b55f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b8575bc060ce15b55f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b8575bc060ce15b55f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:49,620 - lava_callback - INFO - Received callback for node 69408b6f575bc060ce15f1e4 INFO: 172.21.0.1:41324 - "POST /node/69408b6f575bc060ce15f1e4 HTTP/1.0" 202 Accepted 2025-12-17 21:08:49,628 - lava_callback - INFO - Starting async job submit for node 69408b6f575bc060ce15f1e4 2025-12-17 21:08:49,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:49,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6f575bc060ce15f1e4 HTTP/1.1" 502 150 2025-12-17 21:08:49,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6f575bc060ce15f1e4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:49,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6f575bc060ce15f1e4 2025-12-17 21:08:49,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6f575bc060ce15f1e4 HTTP/1.1" 502 150 2025-12-17 21:08:49,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6f575bc060ce15f1e4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:50,990 - lava_callback - INFO - Received callback for node 694078dc575bc060ce15b592 INFO: 172.21.0.1:41836 - "POST /node/694078dc575bc060ce15b592 HTTP/1.0" 202 Accepted 2025-12-17 21:08:50,996 - lava_callback - INFO - Starting async job submit for node 694078dc575bc060ce15b592 2025-12-17 21:08:50,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:51,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078dc575bc060ce15b592 HTTP/1.1" 502 150 2025-12-17 21:08:51,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078dc575bc060ce15b592'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:51,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078dc575bc060ce15b592 2025-12-17 21:08:51,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078dc575bc060ce15b592 HTTP/1.1" 502 150 2025-12-17 21:08:51,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078dc575bc060ce15b592'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:51,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bb575bc060ce15b565 2025-12-17 21:08:51,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bb575bc060ce15b565 HTTP/1.1" 502 150 2025-12-17 21:08:51,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bb575bc060ce15b565'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:51,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6f575bc060ce15f1e4 2025-12-17 21:08:51,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6f575bc060ce15f1e4 HTTP/1.1" 502 150 2025-12-17 21:08:51,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6f575bc060ce15f1e4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:51,726 - lava_callback - INFO - Received callback for node 694078dc575bc060ce15b597 INFO: 172.21.0.1:41846 - "POST /node/694078dc575bc060ce15b597 HTTP/1.0" 202 Accepted 2025-12-17 21:08:51,732 - lava_callback - INFO - Starting async job submit for node 694078dc575bc060ce15b597 2025-12-17 21:08:51,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:08:51,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078dc575bc060ce15b597 HTTP/1.1" 502 150 2025-12-17 21:08:51,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078dc575bc060ce15b597'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:51,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078dc575bc060ce15b597 2025-12-17 21:08:51,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078dc575bc060ce15b597 HTTP/1.1" 502 150 2025-12-17 21:08:51,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078dc575bc060ce15b597'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:53,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078dc575bc060ce15b592 2025-12-17 21:08:53,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078dc575bc060ce15b592 HTTP/1.1" 502 150 2025-12-17 21:08:53,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078dc575bc060ce15b592'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:53,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078dc575bc060ce15b597 2025-12-17 21:08:53,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078dc575bc060ce15b597 HTTP/1.1" 502 150 2025-12-17 21:08:53,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078dc575bc060ce15b597'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:53,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bc575bc060ce15b568 2025-12-17 21:08:53,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bc575bc060ce15b568 HTTP/1.1" 502 150 2025-12-17 21:08:53,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bc575bc060ce15b568'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:54,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bd575bc060ce15b56b 2025-12-17 21:08:54,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bd575bc060ce15b56b HTTP/1.1" 502 150 2025-12-17 21:08:54,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bd575bc060ce15b56b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:55,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6f575bc060ce15f1e4 2025-12-17 21:08:55,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6f575bc060ce15f1e4 HTTP/1.1" 502 150 2025-12-17 21:08:55,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6f575bc060ce15f1e4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:55,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078c0575bc060ce15b571 2025-12-17 21:08:55,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078c0575bc060ce15b571 HTTP/1.1" 502 150 2025-12-17 21:08:55,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078c0575bc060ce15b571'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:56,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078b1575bc060ce15b54f 2025-12-17 21:08:56,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078b1575bc060ce15b54f HTTP/1.1" 502 150 2025-12-17 21:08:56,307 - lava_callback - ERROR - Error processing callback for node 694078b1575bc060ce15b54f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b1575bc060ce15b54f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b1575bc060ce15b54f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078b1575bc060ce15b54f (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:41854 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:08:56,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ba575bc060ce15b562 2025-12-17 21:08:56,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ba575bc060ce15b562 HTTP/1.1" 502 150 2025-12-17 21:08:56,566 - lava_callback - ERROR - Error processing callback for node 694078ba575bc060ce15b562: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ba575bc060ce15b562 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ba575bc060ce15b562 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ba575bc060ce15b562 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:08:57,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078dc575bc060ce15b592 2025-12-17 21:08:57,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078dc575bc060ce15b592 HTTP/1.1" 502 150 2025-12-17 21:08:57,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078dc575bc060ce15b592'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:57,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078dc575bc060ce15b597 2025-12-17 21:08:57,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078dc575bc060ce15b597 HTTP/1.1" 502 150 2025-12-17 21:08:57,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078dc575bc060ce15b597'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:08:58,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bf575bc060ce15b56e HTTP/1.1" 502 150 2025-12-17 21:08:58,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bf575bc060ce15b56e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37278 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:09:02,276 - lava_callback - INFO - Received callback for node 69407504575bc060ce15afa4 INFO: 172.21.0.1:37290 - "POST /node/69407504575bc060ce15afa4 HTTP/1.0" 202 Accepted 2025-12-17 21:09:02,282 - lava_callback - INFO - Starting async job submit for node 69407504575bc060ce15afa4 2025-12-17 21:09:02,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:02,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407504575bc060ce15afa4 HTTP/1.1" 502 150 2025-12-17 21:09:02,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407504575bc060ce15afa4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:02,320 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407504575bc060ce15afa4 2025-12-17 21:09:02,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407504575bc060ce15afa4 HTTP/1.1" 502 150 2025-12-17 21:09:02,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407504575bc060ce15afa4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:02,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bf575bc060ce15b56e 2025-12-17 21:09:02,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bf575bc060ce15b56e HTTP/1.1" 502 150 2025-12-17 21:09:02,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bf575bc060ce15b56e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:03,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6f575bc060ce15f1e4 2025-12-17 21:09:03,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6f575bc060ce15f1e4 HTTP/1.1" 502 150 2025-12-17 21:09:03,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6f575bc060ce15f1e4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:04,320 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407504575bc060ce15afa4 2025-12-17 21:09:04,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407504575bc060ce15afa4 HTTP/1.1" 502 150 2025-12-17 21:09:04,322 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407504575bc060ce15afa4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:05,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078dc575bc060ce15b592 2025-12-17 21:09:05,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078dc575bc060ce15b592 HTTP/1.1" 502 150 2025-12-17 21:09:05,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078dc575bc060ce15b592'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:05,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078dc575bc060ce15b597 2025-12-17 21:09:05,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078dc575bc060ce15b597 HTTP/1.1" 502 150 2025-12-17 21:09:05,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078dc575bc060ce15b597'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37298 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:09:07,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bb575bc060ce15b565 2025-12-17 21:09:07,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bb575bc060ce15b565 HTTP/1.1" 502 150 2025-12-17 21:09:07,699 - lava_callback - ERROR - Error processing callback for node 694078bb575bc060ce15b565: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078bb575bc060ce15b565 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078bb575bc060ce15b565 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078bb575bc060ce15b565 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:08,322 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407504575bc060ce15afa4 2025-12-17 21:09:08,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407504575bc060ce15afa4 HTTP/1.1" 502 150 2025-12-17 21:09:08,323 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407504575bc060ce15afa4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:09,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bc575bc060ce15b568 2025-12-17 21:09:09,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bc575bc060ce15b568 HTTP/1.1" 502 150 2025-12-17 21:09:09,862 - lava_callback - ERROR - Error processing callback for node 694078bc575bc060ce15b568: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078bc575bc060ce15b568 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078bc575bc060ce15b568 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078bc575bc060ce15b568 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:10,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bf575bc060ce15b56e 2025-12-17 21:09:10,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bf575bc060ce15b56e HTTP/1.1" 502 150 2025-12-17 21:09:10,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078bf575bc060ce15b56e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:10,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bd575bc060ce15b56b 2025-12-17 21:09:10,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bd575bc060ce15b56b HTTP/1.1" 502 150 2025-12-17 21:09:10,485 - lava_callback - ERROR - Error processing callback for node 694078bd575bc060ce15b56b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078bd575bc060ce15b56b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078bd575bc060ce15b56b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078bd575bc060ce15b56b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:10,992 - lava_callback - INFO - Received callback for node 694078e5575bc060ce15b5b3 INFO: 172.21.0.1:45310 - "POST /node/694078e5575bc060ce15b5b3 HTTP/1.0" 202 Accepted 2025-12-17 21:09:10,998 - lava_callback - INFO - Starting async job submit for node 694078e5575bc060ce15b5b3 2025-12-17 21:09:10,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:11,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e5575bc060ce15b5b3 HTTP/1.1" 502 150 2025-12-17 21:09:11,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e5575bc060ce15b5b3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:11,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e5575bc060ce15b5b3 2025-12-17 21:09:11,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e5575bc060ce15b5b3 HTTP/1.1" 502 150 2025-12-17 21:09:11,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e5575bc060ce15b5b3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:11,790 - lava_callback - INFO - Received callback for node 694078e5575bc060ce15b5c7 INFO: 172.21.0.1:45324 - "POST /node/694078e5575bc060ce15b5c7 HTTP/1.0" 202 Accepted 2025-12-17 21:09:11,795 - lava_callback - INFO - Starting async job submit for node 694078e5575bc060ce15b5c7 2025-12-17 21:09:11,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:11,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e5575bc060ce15b5c7 HTTP/1.1" 502 150 2025-12-17 21:09:11,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e5575bc060ce15b5c7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:11,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e5575bc060ce15b5c7 2025-12-17 21:09:11,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e5575bc060ce15b5c7 HTTP/1.1" 502 150 2025-12-17 21:09:11,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e5575bc060ce15b5c7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:11,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078c0575bc060ce15b571 2025-12-17 21:09:11,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078c0575bc060ce15b571 HTTP/1.1" 502 150 2025-12-17 21:09:11,863 - lava_callback - ERROR - Error processing callback for node 694078c0575bc060ce15b571: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078c0575bc060ce15b571 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078c0575bc060ce15b571 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078c0575bc060ce15b571 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:12,530 - lava_callback - INFO - Received callback for node 694078e6575bc060ce15b5d9 INFO: 172.21.0.1:45334 - "POST /node/694078e6575bc060ce15b5d9 HTTP/1.0" 202 Accepted 2025-12-17 21:09:12,535 - lava_callback - INFO - Starting async job submit for node 694078e6575bc060ce15b5d9 2025-12-17 21:09:12,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:12,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e6575bc060ce15b5d9 HTTP/1.1" 502 150 2025-12-17 21:09:12,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e6575bc060ce15b5d9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:12,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e6575bc060ce15b5d9 2025-12-17 21:09:12,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e6575bc060ce15b5d9 HTTP/1.1" 502 150 2025-12-17 21:09:12,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e6575bc060ce15b5d9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:13,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e5575bc060ce15b5b3 2025-12-17 21:09:13,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e5575bc060ce15b5b3 HTTP/1.1" 502 150 2025-12-17 21:09:13,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e5575bc060ce15b5b3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:13,453 - lava_callback - INFO - Received callback for node 694078e6575bc060ce15b5e6 INFO: 172.21.0.1:45350 - "POST /node/694078e6575bc060ce15b5e6 HTTP/1.0" 202 Accepted 2025-12-17 21:09:13,459 - lava_callback - INFO - Starting async job submit for node 694078e6575bc060ce15b5e6 2025-12-17 21:09:13,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:13,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e6575bc060ce15b5e6 HTTP/1.1" 502 150 2025-12-17 21:09:13,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e6575bc060ce15b5e6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:13,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e6575bc060ce15b5e6 2025-12-17 21:09:13,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e6575bc060ce15b5e6 HTTP/1.1" 502 150 2025-12-17 21:09:13,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e6575bc060ce15b5e6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:13,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e5575bc060ce15b5c7 2025-12-17 21:09:13,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e5575bc060ce15b5c7 HTTP/1.1" 502 150 2025-12-17 21:09:13,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e5575bc060ce15b5c7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:14,243 - lava_callback - INFO - Received callback for node 694078e7575bc060ce15b5ef INFO: 172.21.0.1:45366 - "POST /node/694078e7575bc060ce15b5ef HTTP/1.0" 202 Accepted 2025-12-17 21:09:14,257 - lava_callback - INFO - Starting async job submit for node 694078e7575bc060ce15b5ef 2025-12-17 21:09:14,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:14,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e7575bc060ce15b5ef HTTP/1.1" 502 150 2025-12-17 21:09:14,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e7575bc060ce15b5ef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:14,280 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e7575bc060ce15b5ef 2025-12-17 21:09:14,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e7575bc060ce15b5ef HTTP/1.1" 502 150 2025-12-17 21:09:14,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e7575bc060ce15b5ef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:14,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e6575bc060ce15b5d9 2025-12-17 21:09:14,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e6575bc060ce15b5d9 HTTP/1.1" 502 150 2025-12-17 21:09:14,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e6575bc060ce15b5d9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:15,048 - lava_callback - INFO - Received callback for node 694078e7575bc060ce15b5f2 INFO: 172.21.0.1:45370 - "POST /node/694078e7575bc060ce15b5f2 HTTP/1.0" 202 Accepted 2025-12-17 21:09:15,054 - lava_callback - INFO - Starting async job submit for node 694078e7575bc060ce15b5f2 2025-12-17 21:09:15,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:15,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e7575bc060ce15b5f2 HTTP/1.1" 502 150 2025-12-17 21:09:15,084 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e7575bc060ce15b5f2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:15,085 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e7575bc060ce15b5f2 2025-12-17 21:09:15,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e7575bc060ce15b5f2 HTTP/1.1" 502 150 2025-12-17 21:09:15,085 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e7575bc060ce15b5f2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45384 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:09:15,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e6575bc060ce15b5e6 2025-12-17 21:09:15,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e6575bc060ce15b5e6 HTTP/1.1" 502 150 2025-12-17 21:09:15,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e6575bc060ce15b5e6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:15,789 - lava_callback - INFO - Received callback for node 694078e9575bc060ce15b5f5 INFO: 172.21.0.1:45400 - "POST /node/694078e9575bc060ce15b5f5 HTTP/1.0" 202 Accepted 2025-12-17 21:09:15,795 - lava_callback - INFO - Starting async job submit for node 694078e9575bc060ce15b5f5 2025-12-17 21:09:15,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:15,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e9575bc060ce15b5f5 HTTP/1.1" 502 150 2025-12-17 21:09:15,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e9575bc060ce15b5f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:15,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e9575bc060ce15b5f5 2025-12-17 21:09:15,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e9575bc060ce15b5f5 HTTP/1.1" 502 150 2025-12-17 21:09:15,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e9575bc060ce15b5f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:16,284 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e7575bc060ce15b5ef 2025-12-17 21:09:16,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e7575bc060ce15b5ef HTTP/1.1" 502 150 2025-12-17 21:09:16,285 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e7575bc060ce15b5ef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:16,324 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407504575bc060ce15afa4 2025-12-17 21:09:16,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407504575bc060ce15afa4 HTTP/1.1" 502 150 2025-12-17 21:09:16,325 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407504575bc060ce15afa4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45406 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:09:16,559 - lava_callback - INFO - Received callback for node 694078ea575bc060ce15b5f8 INFO: 172.21.0.1:45416 - "POST /node/694078ea575bc060ce15b5f8 HTTP/1.0" 202 Accepted 2025-12-17 21:09:16,565 - lava_callback - INFO - Starting async job submit for node 694078ea575bc060ce15b5f8 2025-12-17 21:09:16,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:16,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ea575bc060ce15b5f8 HTTP/1.1" 502 150 2025-12-17 21:09:16,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ea575bc060ce15b5f8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:16,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ea575bc060ce15b5f8 2025-12-17 21:09:16,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ea575bc060ce15b5f8 HTTP/1.1" 502 150 2025-12-17 21:09:16,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ea575bc060ce15b5f8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:17,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e5575bc060ce15b5b3 2025-12-17 21:09:17,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e5575bc060ce15b5b3 HTTP/1.1" 502 150 2025-12-17 21:09:17,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e5575bc060ce15b5b3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:17,086 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e7575bc060ce15b5f2 2025-12-17 21:09:17,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e7575bc060ce15b5f2 HTTP/1.1" 502 150 2025-12-17 21:09:17,087 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e7575bc060ce15b5f2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:17,323 - lava_callback - INFO - Received callback for node 694078ec575bc060ce15b5ff INFO: 172.21.0.1:45420 - "POST /node/694078ec575bc060ce15b5ff HTTP/1.0" 202 Accepted 2025-12-17 21:09:17,329 - lava_callback - INFO - Starting async job submit for node 694078ec575bc060ce15b5ff 2025-12-17 21:09:17,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:17,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ec575bc060ce15b5ff HTTP/1.1" 502 150 2025-12-17 21:09:17,336 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ec575bc060ce15b5ff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:17,336 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ec575bc060ce15b5ff 2025-12-17 21:09:17,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ec575bc060ce15b5ff HTTP/1.1" 502 150 2025-12-17 21:09:17,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ec575bc060ce15b5ff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:17,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e5575bc060ce15b5c7 2025-12-17 21:09:17,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e5575bc060ce15b5c7 HTTP/1.1" 502 150 2025-12-17 21:09:17,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e5575bc060ce15b5c7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:17,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e9575bc060ce15b5f5 2025-12-17 21:09:17,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e9575bc060ce15b5f5 HTTP/1.1" 502 150 2025-12-17 21:09:17,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e9575bc060ce15b5f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:18,133 - lava_callback - INFO - Received callback for node 694078ee575bc060ce15b602 INFO: 172.21.0.1:45424 - "POST /node/694078ee575bc060ce15b602 HTTP/1.0" 202 Accepted 2025-12-17 21:09:18,142 - lava_callback - INFO - Starting async job submit for node 694078ee575bc060ce15b602 2025-12-17 21:09:18,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:18,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ee575bc060ce15b602 HTTP/1.1" 502 150 2025-12-17 21:09:18,151 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ee575bc060ce15b602'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:18,151 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ee575bc060ce15b602 2025-12-17 21:09:18,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ee575bc060ce15b602 HTTP/1.1" 502 150 2025-12-17 21:09:18,152 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ee575bc060ce15b602'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:18,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e6575bc060ce15b5d9 2025-12-17 21:09:18,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e6575bc060ce15b5d9 HTTP/1.1" 502 150 2025-12-17 21:09:18,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e6575bc060ce15b5d9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:18,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ea575bc060ce15b5f8 2025-12-17 21:09:18,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ea575bc060ce15b5f8 HTTP/1.1" 502 150 2025-12-17 21:09:18,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ea575bc060ce15b5f8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:18,882 - lava_callback - INFO - Received callback for node 694078f3575bc060ce15b606 INFO: 172.21.0.1:45436 - "POST /node/694078f3575bc060ce15b606 HTTP/1.0" 202 Accepted 2025-12-17 21:09:18,888 - lava_callback - INFO - Starting async job submit for node 694078f3575bc060ce15b606 2025-12-17 21:09:18,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:18,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f3575bc060ce15b606 HTTP/1.1" 502 150 2025-12-17 21:09:18,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f3575bc060ce15b606'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:18,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f3575bc060ce15b606 2025-12-17 21:09:18,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f3575bc060ce15b606 HTTP/1.1" 502 150 2025-12-17 21:09:18,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f3575bc060ce15b606'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:19,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ec575bc060ce15b5ff 2025-12-17 21:09:19,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ec575bc060ce15b5ff HTTP/1.1" 502 150 2025-12-17 21:09:19,339 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ec575bc060ce15b5ff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:19,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e6575bc060ce15b5e6 2025-12-17 21:09:19,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e6575bc060ce15b5e6 HTTP/1.1" 502 150 2025-12-17 21:09:19,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e6575bc060ce15b5e6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:19,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6f575bc060ce15f1e4 2025-12-17 21:09:19,729 - lava_callback - INFO - Received callback for node 694078f5575bc060ce15b609 2025-12-17 21:09:19,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6f575bc060ce15f1e4 HTTP/1.1" 502 150 INFO: 172.21.0.1:46732 - "POST /node/694078f5575bc060ce15b609 HTTP/1.0" 202 Accepted 2025-12-17 21:09:19,735 - lava_callback - ERROR - Error processing callback for node 69408b6f575bc060ce15f1e4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b6f575bc060ce15f1e4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b6f575bc060ce15f1e4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b6f575bc060ce15f1e4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:19,738 - lava_callback - INFO - Starting async job submit for node 694078f5575bc060ce15b609 2025-12-17 21:09:19,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:19,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f5575bc060ce15b609 HTTP/1.1" 502 150 2025-12-17 21:09:19,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f5575bc060ce15b609'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:19,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f5575bc060ce15b609 2025-12-17 21:09:19,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f5575bc060ce15b609 HTTP/1.1" 502 150 2025-12-17 21:09:19,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f5575bc060ce15b609'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:20,152 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ee575bc060ce15b602 2025-12-17 21:09:20,285 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e7575bc060ce15b5ef 2025-12-17 21:09:20,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e7575bc060ce15b5ef HTTP/1.1" 502 150 2025-12-17 21:09:20,286 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e7575bc060ce15b5ef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:20,499 - lava_callback - INFO - Received callback for node 694078f6575bc060ce15b60c INFO: 172.21.0.1:46746 - "POST /node/694078f6575bc060ce15b60c HTTP/1.0" 202 Accepted 2025-12-17 21:09:20,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f3575bc060ce15b606 2025-12-17 21:09:20,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f3575bc060ce15b606 HTTP/1.1" 502 150 2025-12-17 21:09:20,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f3575bc060ce15b606'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:21,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078dc575bc060ce15b592 2025-12-17 21:09:21,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078dc575bc060ce15b592 HTTP/1.1" 502 150 2025-12-17 21:09:21,015 - lava_callback - ERROR - Error processing callback for node 694078dc575bc060ce15b592: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078dc575bc060ce15b592 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078dc575bc060ce15b592 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078dc575bc060ce15b592 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:21,017 - lava_callback - INFO - Starting async job submit for node 694078f6575bc060ce15b60c 2025-12-17 21:09:21,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:21,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f6575bc060ce15b60c HTTP/1.1" 502 150 2025-12-17 21:09:21,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f6575bc060ce15b60c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:21,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f6575bc060ce15b60c 2025-12-17 21:09:21,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f6575bc060ce15b60c HTTP/1.1" 502 150 2025-12-17 21:09:21,047 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f6575bc060ce15b60c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:21,087 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e7575bc060ce15b5f2 2025-12-17 21:09:21,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e7575bc060ce15b5f2 HTTP/1.1" 502 150 2025-12-17 21:09:21,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e7575bc060ce15b5f2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:21,239 - lava_callback - INFO - Received callback for node 694078f7575bc060ce15b61c INFO: 172.21.0.1:46762 - "POST /node/694078f7575bc060ce15b61c HTTP/1.0" 202 Accepted 2025-12-17 21:09:21,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f5575bc060ce15b609 2025-12-17 21:09:21,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f5575bc060ce15b609 HTTP/1.1" 502 150 2025-12-17 21:09:21,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f5575bc060ce15b609'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:21,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078dc575bc060ce15b597 2025-12-17 21:09:21,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078dc575bc060ce15b597 HTTP/1.1" 502 150 2025-12-17 21:09:21,755 - lava_callback - ERROR - Error processing callback for node 694078dc575bc060ce15b597: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078dc575bc060ce15b597 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078dc575bc060ce15b597 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078dc575bc060ce15b597 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:21,757 - lava_callback - INFO - Starting async job submit for node 694078f7575bc060ce15b61c 2025-12-17 21:09:21,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:21,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f7575bc060ce15b61c HTTP/1.1" 502 150 2025-12-17 21:09:21,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f7575bc060ce15b61c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:21,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f7575bc060ce15b61c 2025-12-17 21:09:21,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f7575bc060ce15b61c HTTP/1.1" 502 150 2025-12-17 21:09:21,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f7575bc060ce15b61c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:21,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e9575bc060ce15b5f5 2025-12-17 21:09:21,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e9575bc060ce15b5f5 HTTP/1.1" 502 150 2025-12-17 21:09:21,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e9575bc060ce15b5f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:22,004 - lava_callback - INFO - Received callback for node 694078f8575bc060ce15b61f INFO: 172.21.0.1:46770 - "POST /node/694078f8575bc060ce15b61f HTTP/1.0" 202 Accepted 2025-12-17 21:09:22,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ea575bc060ce15b5f8 2025-12-17 21:09:22,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ea575bc060ce15b5f8 HTTP/1.1" 502 150 2025-12-17 21:09:22,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ea575bc060ce15b5f8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:22,751 - lava_callback - INFO - Received callback for node 694078fa575bc060ce15b622 INFO: 172.21.0.1:46774 - "POST /node/694078fa575bc060ce15b622 HTTP/1.0" 202 Accepted 2025-12-17 21:09:23,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f6575bc060ce15b60c 2025-12-17 21:09:23,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f6575bc060ce15b60c HTTP/1.1" 502 150 2025-12-17 21:09:23,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f6575bc060ce15b60c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:23,339 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ec575bc060ce15b5ff 2025-12-17 21:09:23,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ec575bc060ce15b5ff HTTP/1.1" 502 150 2025-12-17 21:09:23,340 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ec575bc060ce15b5ff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:23,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f7575bc060ce15b61c 2025-12-17 21:09:23,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f7575bc060ce15b61c HTTP/1.1" 502 150 2025-12-17 21:09:23,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f7575bc060ce15b61c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:23,854 - lava_callback - INFO - Received callback for node 694078fb575bc060ce15b625 INFO: 172.21.0.1:46790 - "POST /node/694078fb575bc060ce15b625 HTTP/1.0" 202 Accepted 2025-12-17 21:09:24,684 - lava_callback - INFO - Received callback for node 694078fc575bc060ce15b628 INFO: 172.21.0.1:46806 - "POST /node/694078fc575bc060ce15b628 HTTP/1.0" 202 Accepted 2025-12-17 21:09:24,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f3575bc060ce15b606 2025-12-17 21:09:24,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f3575bc060ce15b606 HTTP/1.1" 502 150 2025-12-17 21:09:24,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f3575bc060ce15b606'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:25,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e5575bc060ce15b5b3 2025-12-17 21:09:25,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e5575bc060ce15b5b3 HTTP/1.1" 502 150 2025-12-17 21:09:25,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e5575bc060ce15b5b3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:25,428 - lava_callback - INFO - Received callback for node 694078fd575bc060ce15b62b INFO: 172.21.0.1:46820 - "POST /node/694078fd575bc060ce15b62b HTTP/1.0" 202 Accepted 2025-12-17 21:09:25,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f5575bc060ce15b609 2025-12-17 21:09:25,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f5575bc060ce15b609 HTTP/1.1" 502 150 2025-12-17 21:09:25,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f5575bc060ce15b609'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:25,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e5575bc060ce15b5c7 2025-12-17 21:09:25,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e5575bc060ce15b5c7 HTTP/1.1" 502 150 2025-12-17 21:09:25,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e5575bc060ce15b5c7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:26,183 - lava_callback - INFO - Received callback for node 694078ff575bc060ce15b62e INFO: 172.21.0.1:46836 - "POST /node/694078ff575bc060ce15b62e HTTP/1.0" 202 Accepted 2025-12-17 21:09:26,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078bf575bc060ce15b56e INFO: 172.21.0.1:46842 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:09:26,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e6575bc060ce15b5d9 2025-12-17 21:09:26,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e6575bc060ce15b5d9 HTTP/1.1" 502 150 2025-12-17 21:09:26,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e6575bc060ce15b5d9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:27,049 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f6575bc060ce15b60c 2025-12-17 21:09:27,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f6575bc060ce15b60c HTTP/1.1" 502 150 2025-12-17 21:09:27,050 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f6575bc060ce15b60c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:27,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e6575bc060ce15b5e6 2025-12-17 21:09:27,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e6575bc060ce15b5e6 HTTP/1.1" 502 150 2025-12-17 21:09:27,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e6575bc060ce15b5e6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:27,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f7575bc060ce15b61c 2025-12-17 21:09:27,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f7575bc060ce15b61c HTTP/1.1" 502 150 2025-12-17 21:09:27,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f7575bc060ce15b61c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:28,287 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e7575bc060ce15b5ef 2025-12-17 21:09:28,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e7575bc060ce15b5ef HTTP/1.1" 502 150 2025-12-17 21:09:28,288 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e7575bc060ce15b5ef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:29,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e7575bc060ce15b5f2 2025-12-17 21:09:29,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e7575bc060ce15b5f2 HTTP/1.1" 502 150 2025-12-17 21:09:29,096 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e7575bc060ce15b5f2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:29,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e9575bc060ce15b5f5 2025-12-17 21:09:29,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e9575bc060ce15b5f5 HTTP/1.1" 502 150 2025-12-17 21:09:29,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078e9575bc060ce15b5f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:30,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ea575bc060ce15b5f8 2025-12-17 21:09:30,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ea575bc060ce15b5f8 HTTP/1.1" 502 150 2025-12-17 21:09:30,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ea575bc060ce15b5f8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:31,340 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ec575bc060ce15b5ff 2025-12-17 21:09:31,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ec575bc060ce15b5ff HTTP/1.1" 502 150 2025-12-17 21:09:31,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ec575bc060ce15b5ff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:32,325 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407504575bc060ce15afa4 2025-12-17 21:09:32,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407504575bc060ce15afa4 HTTP/1.1" 502 150 2025-12-17 21:09:32,326 - lava_callback - ERROR - Error processing callback for node 69407504575bc060ce15afa4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407504575bc060ce15afa4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407504575bc060ce15afa4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407504575bc060ce15afa4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:32,329 - lava_callback - INFO - Starting async job submit for node 694078f8575bc060ce15b61f 2025-12-17 21:09:32,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:32,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f8575bc060ce15b61f HTTP/1.1" 502 150 2025-12-17 21:09:32,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f8575bc060ce15b61f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:32,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f8575bc060ce15b61f 2025-12-17 21:09:32,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f8575bc060ce15b61f HTTP/1.1" 502 150 2025-12-17 21:09:32,338 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f8575bc060ce15b61f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:32,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f3575bc060ce15b606 2025-12-17 21:09:32,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f3575bc060ce15b606 HTTP/1.1" 502 150 2025-12-17 21:09:32,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f3575bc060ce15b606'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:33,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f5575bc060ce15b609 2025-12-17 21:09:33,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f5575bc060ce15b609 HTTP/1.1" 502 150 2025-12-17 21:09:33,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f5575bc060ce15b609'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:34,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f8575bc060ce15b61f 2025-12-17 21:09:34,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f8575bc060ce15b61f HTTP/1.1" 502 150 2025-12-17 21:09:34,339 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f8575bc060ce15b61f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:35,050 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f6575bc060ce15b60c 2025-12-17 21:09:35,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f6575bc060ce15b60c HTTP/1.1" 502 150 2025-12-17 21:09:35,051 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f6575bc060ce15b60c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:35,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ee575bc060ce15b602 HTTP/1.1" 502 150 2025-12-17 21:09:35,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ee575bc060ce15b602'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:35,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f7575bc060ce15b61c 2025-12-17 21:09:35,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f7575bc060ce15b61c HTTP/1.1" 502 150 2025-12-17 21:09:35,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f7575bc060ce15b61c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39878 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:09:38,339 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f8575bc060ce15b61f 2025-12-17 21:09:38,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f8575bc060ce15b61f HTTP/1.1" 502 150 2025-12-17 21:09:38,341 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f8575bc060ce15b61f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:39,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ee575bc060ce15b602 2025-12-17 21:09:39,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ee575bc060ce15b602 HTTP/1.1" 502 150 2025-12-17 21:09:39,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ee575bc060ce15b602'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:41,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e5575bc060ce15b5b3 2025-12-17 21:09:41,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e5575bc060ce15b5b3 HTTP/1.1" 502 150 2025-12-17 21:09:41,013 - lava_callback - ERROR - Error processing callback for node 694078e5575bc060ce15b5b3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e5575bc060ce15b5b3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e5575bc060ce15b5b3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e5575bc060ce15b5b3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:41,015 - lava_callback - INFO - Starting async job submit for node 694078fa575bc060ce15b622 2025-12-17 21:09:41,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:41,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fa575bc060ce15b622 HTTP/1.1" 502 150 2025-12-17 21:09:41,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fa575bc060ce15b622'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:41,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fa575bc060ce15b622 2025-12-17 21:09:41,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fa575bc060ce15b622 HTTP/1.1" 502 150 2025-12-17 21:09:41,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fa575bc060ce15b622'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:41,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078bf575bc060ce15b56e HTTP/1.1" 502 150 2025-12-17 21:09:41,685 - lava_callback - ERROR - Error processing callback for node 694078bf575bc060ce15b56e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078bf575bc060ce15b56e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078bf575bc060ce15b56e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078bf575bc060ce15b56e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:41,687 - lava_callback - INFO - Starting async job submit for node 694078fb575bc060ce15b625 2025-12-17 21:09:41,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:41,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fb575bc060ce15b625 HTTP/1.1" 502 150 2025-12-17 21:09:41,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fb575bc060ce15b625'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:41,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fb575bc060ce15b625 2025-12-17 21:09:41,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fb575bc060ce15b625 HTTP/1.1" 502 150 2025-12-17 21:09:41,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fb575bc060ce15b625'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:41,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e5575bc060ce15b5c7 2025-12-17 21:09:41,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e5575bc060ce15b5c7 HTTP/1.1" 502 150 2025-12-17 21:09:41,811 - lava_callback - ERROR - Error processing callback for node 694078e5575bc060ce15b5c7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e5575bc060ce15b5c7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e5575bc060ce15b5c7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e5575bc060ce15b5c7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:41,813 - lava_callback - INFO - Starting async job submit for node 694078fc575bc060ce15b628 2025-12-17 21:09:41,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:41,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fc575bc060ce15b628 HTTP/1.1" 502 150 2025-12-17 21:09:41,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fc575bc060ce15b628'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:41,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fc575bc060ce15b628 2025-12-17 21:09:41,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fc575bc060ce15b628 HTTP/1.1" 502 150 2025-12-17 21:09:41,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fc575bc060ce15b628'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:42,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e6575bc060ce15b5d9 2025-12-17 21:09:42,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e6575bc060ce15b5d9 HTTP/1.1" 502 150 2025-12-17 21:09:42,574 - lava_callback - ERROR - Error processing callback for node 694078e6575bc060ce15b5d9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e6575bc060ce15b5d9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e6575bc060ce15b5d9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e6575bc060ce15b5d9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:42,576 - lava_callback - INFO - Starting async job submit for node 694078fd575bc060ce15b62b 2025-12-17 21:09:42,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:42,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fd575bc060ce15b62b HTTP/1.1" 502 150 2025-12-17 21:09:42,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fd575bc060ce15b62b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:42,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fd575bc060ce15b62b 2025-12-17 21:09:42,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fd575bc060ce15b62b HTTP/1.1" 502 150 2025-12-17 21:09:42,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fd575bc060ce15b62b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:43,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fa575bc060ce15b622 2025-12-17 21:09:43,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fa575bc060ce15b622 HTTP/1.1" 502 150 2025-12-17 21:09:43,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fa575bc060ce15b622'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:43,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e6575bc060ce15b5e6 2025-12-17 21:09:43,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e6575bc060ce15b5e6 HTTP/1.1" 502 150 2025-12-17 21:09:43,472 - lava_callback - ERROR - Error processing callback for node 694078e6575bc060ce15b5e6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e6575bc060ce15b5e6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e6575bc060ce15b5e6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e6575bc060ce15b5e6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:43,474 - lava_callback - INFO - Starting async job submit for node 694078ff575bc060ce15b62e 2025-12-17 21:09:43,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:43,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ff575bc060ce15b62e HTTP/1.1" 502 150 2025-12-17 21:09:43,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ff575bc060ce15b62e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:43,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ff575bc060ce15b62e 2025-12-17 21:09:43,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ff575bc060ce15b62e HTTP/1.1" 502 150 2025-12-17 21:09:43,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ff575bc060ce15b62e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:43,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fb575bc060ce15b625 2025-12-17 21:09:43,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fb575bc060ce15b625 HTTP/1.1" 502 150 2025-12-17 21:09:43,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fb575bc060ce15b625'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:43,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fc575bc060ce15b628 2025-12-17 21:09:43,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fc575bc060ce15b628 HTTP/1.1" 502 150 2025-12-17 21:09:43,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fc575bc060ce15b628'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:43,914 - lava_callback - INFO - Received callback for node 69407e0c575bc060ce15c21a INFO: 172.21.0.1:48990 - "POST /node/69407e0c575bc060ce15c21a HTTP/1.0" 202 Accepted 2025-12-17 21:09:44,288 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e7575bc060ce15b5ef 2025-12-17 21:09:44,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e7575bc060ce15b5ef HTTP/1.1" 502 150 2025-12-17 21:09:44,290 - lava_callback - ERROR - Error processing callback for node 694078e7575bc060ce15b5ef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e7575bc060ce15b5ef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e7575bc060ce15b5ef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e7575bc060ce15b5ef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:44,292 - lava_callback - INFO - Starting async job submit for node 69407e0c575bc060ce15c21a 2025-12-17 21:09:44,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:09:44,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0c575bc060ce15c21a HTTP/1.1" 502 150 2025-12-17 21:09:44,341 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0c575bc060ce15c21a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:44,341 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0c575bc060ce15c21a 2025-12-17 21:09:44,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0c575bc060ce15c21a HTTP/1.1" 502 150 2025-12-17 21:09:44,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0c575bc060ce15c21a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:44,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fd575bc060ce15b62b 2025-12-17 21:09:44,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fd575bc060ce15b62b HTTP/1.1" 502 150 2025-12-17 21:09:44,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fd575bc060ce15b62b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:45,097 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e7575bc060ce15b5f2 2025-12-17 21:09:45,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e7575bc060ce15b5f2 HTTP/1.1" 502 150 2025-12-17 21:09:45,098 - lava_callback - ERROR - Error processing callback for node 694078e7575bc060ce15b5f2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e7575bc060ce15b5f2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e7575bc060ce15b5f2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e7575bc060ce15b5f2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:45,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ff575bc060ce15b62e 2025-12-17 21:09:45,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ff575bc060ce15b62e HTTP/1.1" 502 150 2025-12-17 21:09:45,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ff575bc060ce15b62e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:45,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078e9575bc060ce15b5f5 2025-12-17 21:09:45,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078e9575bc060ce15b5f5 HTTP/1.1" 502 150 2025-12-17 21:09:45,819 - lava_callback - ERROR - Error processing callback for node 694078e9575bc060ce15b5f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e9575bc060ce15b5f5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e9575bc060ce15b5f5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078e9575bc060ce15b5f5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:46,341 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f8575bc060ce15b61f 2025-12-17 21:09:46,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f8575bc060ce15b61f HTTP/1.1" 502 150 2025-12-17 21:09:46,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078f8575bc060ce15b61f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:46,342 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0c575bc060ce15c21a 2025-12-17 21:09:46,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0c575bc060ce15c21a HTTP/1.1" 502 150 2025-12-17 21:09:46,343 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0c575bc060ce15c21a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48998 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:09:46,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ea575bc060ce15b5f8 2025-12-17 21:09:46,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ea575bc060ce15b5f8 HTTP/1.1" 502 150 2025-12-17 21:09:46,580 - lava_callback - ERROR - Error processing callback for node 694078ea575bc060ce15b5f8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ea575bc060ce15b5f8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ea575bc060ce15b5f8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ea575bc060ce15b5f8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:47,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fa575bc060ce15b622 2025-12-17 21:09:47,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fa575bc060ce15b622 HTTP/1.1" 502 150 2025-12-17 21:09:47,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fa575bc060ce15b622'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:47,342 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ec575bc060ce15b5ff 2025-12-17 21:09:47,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ec575bc060ce15b5ff HTTP/1.1" 502 150 2025-12-17 21:09:47,343 - lava_callback - ERROR - Error processing callback for node 694078ec575bc060ce15b5ff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ec575bc060ce15b5ff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ec575bc060ce15b5ff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ec575bc060ce15b5ff (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:47,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ee575bc060ce15b602 2025-12-17 21:09:47,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ee575bc060ce15b602 HTTP/1.1" 502 150 2025-12-17 21:09:47,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ee575bc060ce15b602'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:47,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fb575bc060ce15b625 2025-12-17 21:09:47,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fb575bc060ce15b625 HTTP/1.1" 502 150 2025-12-17 21:09:47,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fb575bc060ce15b625'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:47,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fc575bc060ce15b628 2025-12-17 21:09:47,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fc575bc060ce15b628 HTTP/1.1" 502 150 2025-12-17 21:09:47,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fc575bc060ce15b628'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:48,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fd575bc060ce15b62b 2025-12-17 21:09:48,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fd575bc060ce15b62b HTTP/1.1" 502 150 2025-12-17 21:09:48,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fd575bc060ce15b62b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:48,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f3575bc060ce15b606 2025-12-17 21:09:48,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f3575bc060ce15b606 HTTP/1.1" 502 150 2025-12-17 21:09:48,903 - lava_callback - ERROR - Error processing callback for node 694078f3575bc060ce15b606: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078f3575bc060ce15b606 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078f3575bc060ce15b606 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078f3575bc060ce15b606 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:49,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ff575bc060ce15b62e 2025-12-17 21:09:49,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ff575bc060ce15b62e HTTP/1.1" 502 150 2025-12-17 21:09:49,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ff575bc060ce15b62e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:49,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f5575bc060ce15b609 2025-12-17 21:09:49,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f5575bc060ce15b609 HTTP/1.1" 502 150 2025-12-17 21:09:49,754 - lava_callback - ERROR - Error processing callback for node 694078f5575bc060ce15b609: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078f5575bc060ce15b609 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078f5575bc060ce15b609 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078f5575bc060ce15b609 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:50,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0c575bc060ce15c21a 2025-12-17 21:09:50,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0c575bc060ce15c21a HTTP/1.1" 502 150 2025-12-17 21:09:50,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0c575bc060ce15c21a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:51,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f6575bc060ce15b60c 2025-12-17 21:09:51,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f6575bc060ce15b60c HTTP/1.1" 502 150 2025-12-17 21:09:51,053 - lava_callback - ERROR - Error processing callback for node 694078f6575bc060ce15b60c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078f6575bc060ce15b60c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078f6575bc060ce15b60c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078f6575bc060ce15b60c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:51,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f7575bc060ce15b61c 2025-12-17 21:09:51,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f7575bc060ce15b61c HTTP/1.1" 502 150 2025-12-17 21:09:51,794 - lava_callback - ERROR - Error processing callback for node 694078f7575bc060ce15b61c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078f7575bc060ce15b61c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078f7575bc060ce15b61c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078f7575bc060ce15b61c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:09:55,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fa575bc060ce15b622 2025-12-17 21:09:55,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fa575bc060ce15b622 HTTP/1.1" 502 150 2025-12-17 21:09:55,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fa575bc060ce15b622'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:55,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fb575bc060ce15b625 2025-12-17 21:09:55,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fb575bc060ce15b625 HTTP/1.1" 502 150 2025-12-17 21:09:55,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fb575bc060ce15b625'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:55,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fc575bc060ce15b628 2025-12-17 21:09:55,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fc575bc060ce15b628 HTTP/1.1" 502 150 2025-12-17 21:09:55,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fc575bc060ce15b628'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38408 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:09:56,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fd575bc060ce15b62b 2025-12-17 21:09:56,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fd575bc060ce15b62b HTTP/1.1" 502 150 2025-12-17 21:09:56,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078fd575bc060ce15b62b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:09:57,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ff575bc060ce15b62e 2025-12-17 21:09:58,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0c575bc060ce15c21a 2025-12-17 21:09:58,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0c575bc060ce15c21a HTTP/1.1" 502 150 2025-12-17 21:09:58,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0c575bc060ce15c21a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41970 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:10:02,342 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078f8575bc060ce15b61f 2025-12-17 21:10:02,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078f8575bc060ce15b61f HTTP/1.1" 502 150 2025-12-17 21:10:02,349 - lava_callback - ERROR - Error processing callback for node 694078f8575bc060ce15b61f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078f8575bc060ce15b61f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078f8575bc060ce15b61f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078f8575bc060ce15b61f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:03,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ee575bc060ce15b602 2025-12-17 21:10:03,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ee575bc060ce15b602 HTTP/1.1" 502 150 2025-12-17 21:10:03,540 - lava_callback - ERROR - Error processing callback for node 694078ee575bc060ce15b602: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ee575bc060ce15b602 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ee575bc060ce15b602 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ee575bc060ce15b602 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:41982 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:10:10,976 - lava_callback - INFO - Received callback for node 69407924575bc060ce15b669 INFO: 172.21.0.1:38840 - "POST /node/69407924575bc060ce15b669 HTTP/1.0" 202 Accepted 2025-12-17 21:10:10,982 - lava_callback - INFO - Starting async job submit for node 69407924575bc060ce15b669 2025-12-17 21:10:10,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:10:11,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407924575bc060ce15b669 HTTP/1.1" 502 150 2025-12-17 21:10:11,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407924575bc060ce15b669'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:11,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407924575bc060ce15b669 2025-12-17 21:10:11,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407924575bc060ce15b669 HTTP/1.1" 502 150 2025-12-17 21:10:11,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407924575bc060ce15b669'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:11,028 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fa575bc060ce15b622 2025-12-17 21:10:11,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fa575bc060ce15b622 HTTP/1.1" 502 150 2025-12-17 21:10:11,029 - lava_callback - ERROR - Error processing callback for node 694078fa575bc060ce15b622: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078fa575bc060ce15b622 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078fa575bc060ce15b622 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078fa575bc060ce15b622 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:11,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fb575bc060ce15b625 2025-12-17 21:10:11,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fb575bc060ce15b625 HTTP/1.1" 502 150 2025-12-17 21:10:11,705 - lava_callback - ERROR - Error processing callback for node 694078fb575bc060ce15b625: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078fb575bc060ce15b625 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078fb575bc060ce15b625 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078fb575bc060ce15b625 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:11,744 - lava_callback - INFO - Received callback for node 69407928575bc060ce15b677 INFO: 172.21.0.1:38850 - "POST /node/69407928575bc060ce15b677 HTTP/1.0" 202 Accepted 2025-12-17 21:10:11,750 - lava_callback - INFO - Starting async job submit for node 69407928575bc060ce15b677 2025-12-17 21:10:11,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:10:11,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407928575bc060ce15b677 HTTP/1.1" 502 150 2025-12-17 21:10:11,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407928575bc060ce15b677'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:11,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407928575bc060ce15b677 2025-12-17 21:10:11,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407928575bc060ce15b677 HTTP/1.1" 502 150 2025-12-17 21:10:11,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407928575bc060ce15b677'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:11,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fc575bc060ce15b628 2025-12-17 21:10:11,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fc575bc060ce15b628 HTTP/1.1" 502 150 2025-12-17 21:10:11,829 - lava_callback - ERROR - Error processing callback for node 694078fc575bc060ce15b628: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078fc575bc060ce15b628 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078fc575bc060ce15b628 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078fc575bc060ce15b628 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:12,506 - lava_callback - INFO - Received callback for node 69407929575bc060ce15b67a INFO: 172.21.0.1:38858 - "POST /node/69407929575bc060ce15b67a HTTP/1.0" 202 Accepted 2025-12-17 21:10:12,512 - lava_callback - INFO - Starting async job submit for node 69407929575bc060ce15b67a 2025-12-17 21:10:12,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:10:12,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407929575bc060ce15b67a HTTP/1.1" 502 150 2025-12-17 21:10:12,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407929575bc060ce15b67a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:12,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407929575bc060ce15b67a 2025-12-17 21:10:12,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407929575bc060ce15b67a HTTP/1.1" 502 150 2025-12-17 21:10:12,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407929575bc060ce15b67a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:12,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078fd575bc060ce15b62b 2025-12-17 21:10:12,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078fd575bc060ce15b62b HTTP/1.1" 502 150 2025-12-17 21:10:12,589 - lava_callback - ERROR - Error processing callback for node 694078fd575bc060ce15b62b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078fd575bc060ce15b62b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078fd575bc060ce15b62b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078fd575bc060ce15b62b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:12,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ff575bc060ce15b62e HTTP/1.1" 502 150 2025-12-17 21:10:12,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078ff575bc060ce15b62e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:13,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407924575bc060ce15b669 2025-12-17 21:10:13,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407924575bc060ce15b669 HTTP/1.1" 502 150 2025-12-17 21:10:13,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407924575bc060ce15b669'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:13,242 - lava_callback - INFO - Received callback for node 6940792b575bc060ce15b67d INFO: 172.21.0.1:38860 - "POST /node/6940792b575bc060ce15b67d HTTP/1.0" 202 Accepted 2025-12-17 21:10:13,248 - lava_callback - INFO - Starting async job submit for node 6940792b575bc060ce15b67d 2025-12-17 21:10:13,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:10:13,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792b575bc060ce15b67d HTTP/1.1" 502 150 2025-12-17 21:10:13,255 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792b575bc060ce15b67d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:13,255 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792b575bc060ce15b67d 2025-12-17 21:10:13,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792b575bc060ce15b67d HTTP/1.1" 502 150 2025-12-17 21:10:13,256 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792b575bc060ce15b67d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:13,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407928575bc060ce15b677 2025-12-17 21:10:13,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407928575bc060ce15b677 HTTP/1.1" 502 150 2025-12-17 21:10:13,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407928575bc060ce15b677'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:14,031 - lava_callback - INFO - Received callback for node 6940792c575bc060ce15b680 INFO: 172.21.0.1:38868 - "POST /node/6940792c575bc060ce15b680 HTTP/1.0" 202 Accepted 2025-12-17 21:10:14,037 - lava_callback - INFO - Starting async job submit for node 6940792c575bc060ce15b680 2025-12-17 21:10:14,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:10:14,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792c575bc060ce15b680 HTTP/1.1" 502 150 2025-12-17 21:10:14,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792c575bc060ce15b680'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:14,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792c575bc060ce15b680 2025-12-17 21:10:14,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792c575bc060ce15b680 HTTP/1.1" 502 150 2025-12-17 21:10:14,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792c575bc060ce15b680'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:14,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0c575bc060ce15c21a 2025-12-17 21:10:14,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0c575bc060ce15c21a HTTP/1.1" 502 150 2025-12-17 21:10:14,348 - lava_callback - ERROR - Error processing callback for node 69407e0c575bc060ce15c21a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e0c575bc060ce15c21a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e0c575bc060ce15c21a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e0c575bc060ce15c21a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:14,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407929575bc060ce15b67a 2025-12-17 21:10:14,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407929575bc060ce15b67a HTTP/1.1" 502 150 2025-12-17 21:10:14,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407929575bc060ce15b67a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:14,850 - lava_callback - INFO - Received callback for node 6940792e575bc060ce15b683 INFO: 172.21.0.1:38876 - "POST /node/6940792e575bc060ce15b683 HTTP/1.0" 202 Accepted 2025-12-17 21:10:14,856 - lava_callback - INFO - Starting async job submit for node 6940792e575bc060ce15b683 2025-12-17 21:10:14,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:10:14,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792e575bc060ce15b683 HTTP/1.1" 502 150 2025-12-17 21:10:14,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792e575bc060ce15b683'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:14,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792e575bc060ce15b683 2025-12-17 21:10:14,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792e575bc060ce15b683 HTTP/1.1" 502 150 2025-12-17 21:10:14,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792e575bc060ce15b683'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38882 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:10:15,256 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792b575bc060ce15b67d 2025-12-17 21:10:15,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792b575bc060ce15b67d HTTP/1.1" 502 150 2025-12-17 21:10:15,258 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792b575bc060ce15b67d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:15,583 - lava_callback - INFO - Received callback for node 6940792f575bc060ce15b686 INFO: 172.21.0.1:38890 - "POST /node/6940792f575bc060ce15b686 HTTP/1.0" 202 Accepted 2025-12-17 21:10:15,592 - lava_callback - INFO - Starting async job submit for node 6940792f575bc060ce15b686 2025-12-17 21:10:15,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:10:15,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792f575bc060ce15b686 HTTP/1.1" 502 150 2025-12-17 21:10:15,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792f575bc060ce15b686'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:15,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792f575bc060ce15b686 2025-12-17 21:10:15,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792f575bc060ce15b686 HTTP/1.1" 502 150 2025-12-17 21:10:15,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792f575bc060ce15b686'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:16,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792c575bc060ce15b680 2025-12-17 21:10:16,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792c575bc060ce15b680 HTTP/1.1" 502 150 2025-12-17 21:10:16,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792c575bc060ce15b680'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:16,340 - lava_callback - INFO - Received callback for node 69407930575bc060ce15b689 INFO: 172.21.0.1:38896 - "POST /node/69407930575bc060ce15b689 HTTP/1.0" 202 Accepted 2025-12-17 21:10:16,346 - lava_callback - INFO - Starting async job submit for node 69407930575bc060ce15b689 2025-12-17 21:10:16,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:10:16,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407930575bc060ce15b689 HTTP/1.1" 502 150 2025-12-17 21:10:16,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407930575bc060ce15b689'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:16,359 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407930575bc060ce15b689 2025-12-17 21:10:16,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407930575bc060ce15b689 HTTP/1.1" 502 150 2025-12-17 21:10:16,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407930575bc060ce15b689'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38910 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:10:16,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792e575bc060ce15b683 2025-12-17 21:10:16,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792e575bc060ce15b683 HTTP/1.1" 502 150 2025-12-17 21:10:16,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792e575bc060ce15b683'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:17,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407924575bc060ce15b669 2025-12-17 21:10:17,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407924575bc060ce15b669 HTTP/1.1" 502 150 2025-12-17 21:10:17,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407924575bc060ce15b669'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:17,225 - lava_callback - INFO - Received callback for node 69407931575bc060ce15b68c INFO: 172.21.0.1:38916 - "POST /node/69407931575bc060ce15b68c HTTP/1.0" 202 Accepted 2025-12-17 21:10:17,231 - lava_callback - INFO - Starting async job submit for node 69407931575bc060ce15b68c 2025-12-17 21:10:17,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:10:17,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407931575bc060ce15b68c HTTP/1.1" 502 150 2025-12-17 21:10:17,238 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407931575bc060ce15b68c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:17,238 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407931575bc060ce15b68c 2025-12-17 21:10:17,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407931575bc060ce15b68c HTTP/1.1" 502 150 2025-12-17 21:10:17,239 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407931575bc060ce15b68c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:17,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792f575bc060ce15b686 2025-12-17 21:10:17,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792f575bc060ce15b686 HTTP/1.1" 502 150 2025-12-17 21:10:17,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792f575bc060ce15b686'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:17,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407928575bc060ce15b677 2025-12-17 21:10:17,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407928575bc060ce15b677 HTTP/1.1" 502 150 2025-12-17 21:10:17,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407928575bc060ce15b677'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:17,987 - lava_callback - INFO - Received callback for node 69407933575bc060ce15b68f INFO: 172.21.0.1:38928 - "POST /node/69407933575bc060ce15b68f HTTP/1.0" 202 Accepted 2025-12-17 21:10:17,993 - lava_callback - INFO - Starting async job submit for node 69407933575bc060ce15b68f 2025-12-17 21:10:17,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:10:18,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407930575bc060ce15b689 2025-12-17 21:10:18,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407930575bc060ce15b689 HTTP/1.1" 502 150 2025-12-17 21:10:18,361 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407930575bc060ce15b689'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:18,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407929575bc060ce15b67a 2025-12-17 21:10:18,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407929575bc060ce15b67a HTTP/1.1" 502 150 2025-12-17 21:10:18,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407929575bc060ce15b67a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:18,827 - lava_callback - INFO - Received callback for node 69407934575bc060ce15b692 INFO: 172.21.0.1:38930 - "POST /node/69407934575bc060ce15b692 HTTP/1.0" 202 Accepted 2025-12-17 21:10:18,832 - lava_callback - INFO - Starting async job submit for node 69407934575bc060ce15b692 2025-12-17 21:10:18,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:10:18,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407934575bc060ce15b692 HTTP/1.1" 502 150 2025-12-17 21:10:18,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407934575bc060ce15b692'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:18,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407934575bc060ce15b692 2025-12-17 21:10:18,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407934575bc060ce15b692 HTTP/1.1" 502 150 2025-12-17 21:10:18,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407934575bc060ce15b692'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:19,239 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407931575bc060ce15b68c 2025-12-17 21:10:19,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407931575bc060ce15b68c HTTP/1.1" 502 150 2025-12-17 21:10:19,240 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407931575bc060ce15b68c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:19,258 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792b575bc060ce15b67d 2025-12-17 21:10:19,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792b575bc060ce15b67d HTTP/1.1" 502 150 2025-12-17 21:10:19,259 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792b575bc060ce15b67d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:19,569 - lava_callback - INFO - Received callback for node 69407936575bc060ce15b695 INFO: 172.21.0.1:38940 - "POST /node/69407936575bc060ce15b695 HTTP/1.0" 202 Accepted 2025-12-17 21:10:19,575 - lava_callback - INFO - Starting async job submit for node 69407936575bc060ce15b695 2025-12-17 21:10:19,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:10:19,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407936575bc060ce15b695 HTTP/1.1" 502 150 2025-12-17 21:10:19,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407936575bc060ce15b695'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:19,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407936575bc060ce15b695 2025-12-17 21:10:19,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407936575bc060ce15b695 HTTP/1.1" 502 150 2025-12-17 21:10:19,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407936575bc060ce15b695'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:20,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792c575bc060ce15b680 2025-12-17 21:10:20,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792c575bc060ce15b680 HTTP/1.1" 502 150 2025-12-17 21:10:20,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792c575bc060ce15b680'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:20,500 - lava_callback - INFO - Received callback for node 69407937575bc060ce15b698 INFO: 172.21.0.1:58182 - "POST /node/69407937575bc060ce15b698 HTTP/1.0" 202 Accepted 2025-12-17 21:10:20,506 - lava_callback - INFO - Starting async job submit for node 69407937575bc060ce15b698 2025-12-17 21:10:20,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:10:20,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407937575bc060ce15b698 HTTP/1.1" 502 150 2025-12-17 21:10:20,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407937575bc060ce15b698'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:20,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407937575bc060ce15b698 2025-12-17 21:10:20,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407937575bc060ce15b698 HTTP/1.1" 502 150 2025-12-17 21:10:20,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407937575bc060ce15b698'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:20,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407934575bc060ce15b692 2025-12-17 21:10:20,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407934575bc060ce15b692 HTTP/1.1" 502 150 2025-12-17 21:10:20,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407934575bc060ce15b692'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:20,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792e575bc060ce15b683 2025-12-17 21:10:20,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792e575bc060ce15b683 HTTP/1.1" 502 150 2025-12-17 21:10:20,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792e575bc060ce15b683'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:21,236 - lava_callback - INFO - Received callback for node 69407938575bc060ce15b69b INFO: 172.21.0.1:58196 - "POST /node/69407938575bc060ce15b69b HTTP/1.0" 202 Accepted 2025-12-17 21:10:21,242 - lava_callback - INFO - Starting async job submit for node 69407938575bc060ce15b69b 2025-12-17 21:10:21,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:10:21,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407938575bc060ce15b69b HTTP/1.1" 502 150 2025-12-17 21:10:21,271 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407938575bc060ce15b69b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:21,271 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407938575bc060ce15b69b 2025-12-17 21:10:21,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407938575bc060ce15b69b HTTP/1.1" 502 150 2025-12-17 21:10:21,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407938575bc060ce15b69b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:21,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407936575bc060ce15b695 2025-12-17 21:10:21,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407936575bc060ce15b695 HTTP/1.1" 502 150 2025-12-17 21:10:21,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407936575bc060ce15b695'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:21,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792f575bc060ce15b686 2025-12-17 21:10:21,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792f575bc060ce15b686 HTTP/1.1" 502 150 2025-12-17 21:10:21,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792f575bc060ce15b686'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:21,981 - lava_callback - INFO - Received callback for node 69407939575bc060ce15b69e INFO: 172.21.0.1:58200 - "POST /node/69407939575bc060ce15b69e HTTP/1.0" 202 Accepted 2025-12-17 21:10:21,987 - lava_callback - INFO - Starting async job submit for node 69407939575bc060ce15b69e 2025-12-17 21:10:21,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:10:21,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407939575bc060ce15b69e HTTP/1.1" 502 150 2025-12-17 21:10:21,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407939575bc060ce15b69e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:21,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407939575bc060ce15b69e 2025-12-17 21:10:21,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407939575bc060ce15b69e HTTP/1.1" 502 150 2025-12-17 21:10:21,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407939575bc060ce15b69e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:22,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407930575bc060ce15b689 2025-12-17 21:10:22,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407930575bc060ce15b689 HTTP/1.1" 502 150 2025-12-17 21:10:22,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407930575bc060ce15b689'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:22,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407937575bc060ce15b698 2025-12-17 21:10:22,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407937575bc060ce15b698 HTTP/1.1" 502 150 2025-12-17 21:10:22,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407937575bc060ce15b698'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:23,240 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407931575bc060ce15b68c 2025-12-17 21:10:23,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407938575bc060ce15b69b 2025-12-17 21:10:23,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407938575bc060ce15b69b HTTP/1.1" 502 150 2025-12-17 21:10:23,273 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407938575bc060ce15b69b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:23,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407939575bc060ce15b69e 2025-12-17 21:10:23,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407939575bc060ce15b69e HTTP/1.1" 502 150 2025-12-17 21:10:23,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407939575bc060ce15b69e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:24,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407934575bc060ce15b692 2025-12-17 21:10:24,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407934575bc060ce15b692 HTTP/1.1" 502 150 2025-12-17 21:10:24,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407934575bc060ce15b692'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:25,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407924575bc060ce15b669 2025-12-17 21:10:25,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407924575bc060ce15b669 HTTP/1.1" 502 150 2025-12-17 21:10:25,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407924575bc060ce15b669'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:25,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407936575bc060ce15b695 2025-12-17 21:10:25,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407936575bc060ce15b695 HTTP/1.1" 502 150 2025-12-17 21:10:25,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407936575bc060ce15b695'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:25,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407928575bc060ce15b677 2025-12-17 21:10:25,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407928575bc060ce15b677 HTTP/1.1" 502 150 2025-12-17 21:10:25,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407928575bc060ce15b677'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58212 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:10:26,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407937575bc060ce15b698 2025-12-17 21:10:26,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407937575bc060ce15b698 HTTP/1.1" 502 150 2025-12-17 21:10:26,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407937575bc060ce15b698'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:26,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407929575bc060ce15b67a 2025-12-17 21:10:26,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407929575bc060ce15b67a HTTP/1.1" 502 150 2025-12-17 21:10:26,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407929575bc060ce15b67a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:27,260 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792b575bc060ce15b67d 2025-12-17 21:10:27,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792b575bc060ce15b67d HTTP/1.1" 502 150 2025-12-17 21:10:27,261 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792b575bc060ce15b67d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:27,273 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407938575bc060ce15b69b 2025-12-17 21:10:27,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407938575bc060ce15b69b HTTP/1.1" 502 150 2025-12-17 21:10:27,274 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407938575bc060ce15b69b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:27,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407939575bc060ce15b69e 2025-12-17 21:10:27,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407939575bc060ce15b69e HTTP/1.1" 502 150 2025-12-17 21:10:27,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407939575bc060ce15b69e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:28,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792c575bc060ce15b680 2025-12-17 21:10:28,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792c575bc060ce15b680 HTTP/1.1" 502 150 2025-12-17 21:10:28,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792c575bc060ce15b680'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:28,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792e575bc060ce15b683 2025-12-17 21:10:28,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792e575bc060ce15b683 HTTP/1.1" 502 150 2025-12-17 21:10:28,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792e575bc060ce15b683'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:28,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078ff575bc060ce15b62e 2025-12-17 21:10:28,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078ff575bc060ce15b62e HTTP/1.1" 502 150 2025-12-17 21:10:28,914 - lava_callback - ERROR - Error processing callback for node 694078ff575bc060ce15b62e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ff575bc060ce15b62e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ff575bc060ce15b62e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078ff575bc060ce15b62e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:29,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792f575bc060ce15b686 2025-12-17 21:10:29,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792f575bc060ce15b686 HTTP/1.1" 502 150 2025-12-17 21:10:29,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940792f575bc060ce15b686'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:30,363 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407930575bc060ce15b689 2025-12-17 21:10:30,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407930575bc060ce15b689 HTTP/1.1" 502 150 2025-12-17 21:10:30,364 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407930575bc060ce15b689'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:32,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407934575bc060ce15b692 2025-12-17 21:10:32,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407934575bc060ce15b692 HTTP/1.1" 502 150 2025-12-17 21:10:32,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407934575bc060ce15b692'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:33,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407933575bc060ce15b68f HTTP/1.1" 502 150 2025-12-17 21:10:33,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407933575bc060ce15b68f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:33,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407933575bc060ce15b68f 2025-12-17 21:10:33,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407933575bc060ce15b68f HTTP/1.1" 502 150 2025-12-17 21:10:33,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407933575bc060ce15b68f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:33,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407936575bc060ce15b695 2025-12-17 21:10:33,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407936575bc060ce15b695 HTTP/1.1" 502 150 2025-12-17 21:10:33,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407936575bc060ce15b695'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:34,082 - lava_callback - INFO - Received callback for node 69408ba8575bc060ce15f2cd INFO: 172.21.0.1:54926 - "POST /node/69408ba8575bc060ce15f2cd HTTP/1.0" 202 Accepted 2025-12-17 21:10:34,090 - lava_callback - INFO - Starting async job submit for node 69408ba8575bc060ce15f2cd 2025-12-17 21:10:34,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:10:34,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407937575bc060ce15b698 2025-12-17 21:10:34,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407937575bc060ce15b698 HTTP/1.1" 502 150 2025-12-17 21:10:34,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407937575bc060ce15b698'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:35,274 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407938575bc060ce15b69b 2025-12-17 21:10:35,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407938575bc060ce15b69b HTTP/1.1" 502 150 2025-12-17 21:10:35,276 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407938575bc060ce15b69b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:35,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407933575bc060ce15b68f 2025-12-17 21:10:35,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407933575bc060ce15b68f HTTP/1.1" 502 150 2025-12-17 21:10:35,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407933575bc060ce15b68f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:35,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407939575bc060ce15b69e 2025-12-17 21:10:35,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407939575bc060ce15b69e HTTP/1.1" 502 150 2025-12-17 21:10:35,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407939575bc060ce15b69e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54940 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:10:38,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407931575bc060ce15b68c HTTP/1.1" 502 150 2025-12-17 21:10:38,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407931575bc060ce15b68c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:39,395 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407933575bc060ce15b68f 2025-12-17 21:10:39,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407933575bc060ce15b68f HTTP/1.1" 502 150 2025-12-17 21:10:39,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407933575bc060ce15b68f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:41,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407924575bc060ce15b669 2025-12-17 21:10:41,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407924575bc060ce15b669 HTTP/1.1" 502 150 2025-12-17 21:10:41,014 - lava_callback - ERROR - Error processing callback for node 69407924575bc060ce15b669: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407924575bc060ce15b669 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407924575bc060ce15b669 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407924575bc060ce15b669 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:41,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407928575bc060ce15b677 2025-12-17 21:10:41,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407928575bc060ce15b677 HTTP/1.1" 502 150 2025-12-17 21:10:41,766 - lava_callback - ERROR - Error processing callback for node 69407928575bc060ce15b677: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407928575bc060ce15b677 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407928575bc060ce15b677 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407928575bc060ce15b677 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:42,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407929575bc060ce15b67a 2025-12-17 21:10:42,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407929575bc060ce15b67a HTTP/1.1" 502 150 2025-12-17 21:10:42,525 - lava_callback - ERROR - Error processing callback for node 69407929575bc060ce15b67a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407929575bc060ce15b67a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407929575bc060ce15b67a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407929575bc060ce15b67a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:43,261 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792b575bc060ce15b67d 2025-12-17 21:10:43,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792b575bc060ce15b67d HTTP/1.1" 502 150 2025-12-17 21:10:43,262 - lava_callback - ERROR - Error processing callback for node 6940792b575bc060ce15b67d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940792b575bc060ce15b67d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940792b575bc060ce15b67d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940792b575bc060ce15b67d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:44,049 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792c575bc060ce15b680 2025-12-17 21:10:44,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792e575bc060ce15b683 2025-12-17 21:10:44,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792e575bc060ce15b683 HTTP/1.1" 502 150 2025-12-17 21:10:44,869 - lava_callback - ERROR - Error processing callback for node 6940792e575bc060ce15b683: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940792e575bc060ce15b683 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940792e575bc060ce15b683 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940792e575bc060ce15b683 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:45,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940792f575bc060ce15b686 2025-12-17 21:10:45,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792f575bc060ce15b686 HTTP/1.1" 502 150 2025-12-17 21:10:45,618 - lava_callback - ERROR - Error processing callback for node 6940792f575bc060ce15b686: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940792f575bc060ce15b686 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940792f575bc060ce15b686 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940792f575bc060ce15b686 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:46,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407930575bc060ce15b689 2025-12-17 21:10:46,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407930575bc060ce15b689 HTTP/1.1" 502 150 2025-12-17 21:10:46,373 - lava_callback - ERROR - Error processing callback for node 69407930575bc060ce15b689: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407930575bc060ce15b689 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407930575bc060ce15b689 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407930575bc060ce15b689 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:46516 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:10:46,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407931575bc060ce15b68c 2025-12-17 21:10:46,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407931575bc060ce15b68c HTTP/1.1" 502 150 2025-12-17 21:10:46,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407931575bc060ce15b68c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:47,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407933575bc060ce15b68f 2025-12-17 21:10:47,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407933575bc060ce15b68f HTTP/1.1" 502 150 2025-12-17 21:10:47,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407933575bc060ce15b68f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:48,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407934575bc060ce15b692 2025-12-17 21:10:48,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407934575bc060ce15b692 HTTP/1.1" 502 150 2025-12-17 21:10:48,847 - lava_callback - ERROR - Error processing callback for node 69407934575bc060ce15b692: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407934575bc060ce15b692 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407934575bc060ce15b692 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407934575bc060ce15b692 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:49,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba8575bc060ce15f2cd HTTP/1.1" 502 150 2025-12-17 21:10:49,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba8575bc060ce15f2cd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:49,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba8575bc060ce15f2cd 2025-12-17 21:10:49,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba8575bc060ce15f2cd HTTP/1.1" 502 150 2025-12-17 21:10:49,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba8575bc060ce15f2cd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:49,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407936575bc060ce15b695 2025-12-17 21:10:49,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407936575bc060ce15b695 HTTP/1.1" 502 150 2025-12-17 21:10:49,592 - lava_callback - ERROR - Error processing callback for node 69407936575bc060ce15b695: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407936575bc060ce15b695 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407936575bc060ce15b695 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407936575bc060ce15b695 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:50,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407937575bc060ce15b698 2025-12-17 21:10:50,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407937575bc060ce15b698 HTTP/1.1" 502 150 2025-12-17 21:10:50,518 - lava_callback - ERROR - Error processing callback for node 69407937575bc060ce15b698: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407937575bc060ce15b698 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407937575bc060ce15b698 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407937575bc060ce15b698 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:51,276 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407938575bc060ce15b69b 2025-12-17 21:10:51,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407938575bc060ce15b69b HTTP/1.1" 502 150 2025-12-17 21:10:51,277 - lava_callback - ERROR - Error processing callback for node 69407938575bc060ce15b69b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407938575bc060ce15b69b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407938575bc060ce15b69b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407938575bc060ce15b69b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:51,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba8575bc060ce15f2cd 2025-12-17 21:10:51,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba8575bc060ce15f2cd HTTP/1.1" 502 150 2025-12-17 21:10:51,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba8575bc060ce15f2cd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:10:51,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407939575bc060ce15b69e 2025-12-17 21:10:52,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407939575bc060ce15b69e HTTP/1.1" 502 150 2025-12-17 21:10:52,001 - lava_callback - ERROR - Error processing callback for node 69407939575bc060ce15b69e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407939575bc060ce15b69e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407939575bc060ce15b69e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407939575bc060ce15b69e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:10:55,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba8575bc060ce15f2cd 2025-12-17 21:10:55,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba8575bc060ce15f2cd HTTP/1.1" 502 150 2025-12-17 21:10:55,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba8575bc060ce15f2cd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40688 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:10:59,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940792c575bc060ce15b680 HTTP/1.1" 502 150 2025-12-17 21:10:59,504 - lava_callback - ERROR - Error processing callback for node 6940792c575bc060ce15b680: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940792c575bc060ce15b680 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940792c575bc060ce15b680 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940792c575bc060ce15b680 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:58970 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:11:02,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407931575bc060ce15b68c 2025-12-17 21:11:02,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407931575bc060ce15b68c HTTP/1.1" 502 150 2025-12-17 21:11:02,515 - lava_callback - ERROR - Error processing callback for node 69407931575bc060ce15b68c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407931575bc060ce15b68c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407931575bc060ce15b68c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407931575bc060ce15b68c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:11:03,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407933575bc060ce15b68f 2025-12-17 21:11:03,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407933575bc060ce15b68f HTTP/1.1" 502 150 2025-12-17 21:11:03,400 - lava_callback - ERROR - Error processing callback for node 69407933575bc060ce15b68f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407933575bc060ce15b68f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407933575bc060ce15b68f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407933575bc060ce15b68f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:11:03,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba8575bc060ce15f2cd 2025-12-17 21:11:03,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba8575bc060ce15f2cd HTTP/1.1" 502 150 2025-12-17 21:11:03,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba8575bc060ce15f2cd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58984 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:11:10,160 - lava_callback - INFO - Received callback for node 69407961575bc060ce15b6db INFO: 172.21.0.1:50574 - "POST /node/69407961575bc060ce15b6db HTTP/1.0" 202 Accepted 2025-12-17 21:11:10,170 - lava_callback - INFO - Starting async job submit for node 69407961575bc060ce15b6db 2025-12-17 21:11:10,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:50578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50586 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:11:19,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba8575bc060ce15f2cd 2025-12-17 21:11:19,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba8575bc060ce15f2cd HTTP/1.1" 502 150 2025-12-17 21:11:19,530 - lava_callback - ERROR - Error processing callback for node 69408ba8575bc060ce15f2cd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba8575bc060ce15f2cd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba8575bc060ce15f2cd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba8575bc060ce15f2cd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:11:19,693 - lava_callback - INFO - Received callback for node 6940786f575bc060ce15b49e INFO: 172.21.0.1:50598 - "POST /node/6940786f575bc060ce15b49e HTTP/1.0" 202 Accepted 2025-12-17 21:11:19,700 - lava_callback - INFO - Starting async job submit for node 6940786f575bc060ce15b49e 2025-12-17 21:11:19,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:11:19,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940786f575bc060ce15b49e HTTP/1.1" 502 150 2025-12-17 21:11:19,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940786f575bc060ce15b49e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:11:19,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940786f575bc060ce15b49e 2025-12-17 21:11:19,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940786f575bc060ce15b49e HTTP/1.1" 502 150 2025-12-17 21:11:19,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940786f575bc060ce15b49e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:11:21,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940786f575bc060ce15b49e 2025-12-17 21:11:21,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940786f575bc060ce15b49e HTTP/1.1" 502 150 2025-12-17 21:11:21,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940786f575bc060ce15b49e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:11:25,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407961575bc060ce15b6db HTTP/1.1" 502 150 2025-12-17 21:11:25,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407961575bc060ce15b6db'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:11:25,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407961575bc060ce15b6db 2025-12-17 21:11:25,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407961575bc060ce15b6db HTTP/1.1" 502 150 2025-12-17 21:11:25,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407961575bc060ce15b6db'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:11:25,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940786f575bc060ce15b49e 2025-12-17 21:11:25,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940786f575bc060ce15b49e HTTP/1.1" 502 150 2025-12-17 21:11:25,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940786f575bc060ce15b49e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39106 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:11:27,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407961575bc060ce15b6db 2025-12-17 21:11:31,580 - lava_callback - INFO - Received callback for node 6940786a575bc060ce15b496 INFO: 172.21.0.1:39170 - "POST /node/6940786a575bc060ce15b496 HTTP/1.0" 202 Accepted 2025-12-17 21:11:31,586 - lava_callback - INFO - Starting async job submit for node 6940786a575bc060ce15b496 2025-12-17 21:11:31,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:11:31,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940786a575bc060ce15b496 HTTP/1.1" 502 150 2025-12-17 21:11:31,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940786a575bc060ce15b496'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:11:31,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940786a575bc060ce15b496 2025-12-17 21:11:31,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940786a575bc060ce15b496 HTTP/1.1" 502 150 2025-12-17 21:11:31,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940786a575bc060ce15b496'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:11:33,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940786a575bc060ce15b496 2025-12-17 21:11:33,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940786a575bc060ce15b496 HTTP/1.1" 502 150 2025-12-17 21:11:33,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940786a575bc060ce15b496'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:11:33,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940786f575bc060ce15b49e 2025-12-17 21:11:33,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940786f575bc060ce15b49e HTTP/1.1" 502 150 2025-12-17 21:11:33,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940786f575bc060ce15b49e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39172 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:11:37,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940786a575bc060ce15b496 2025-12-17 21:11:39,917 - lava_callback - INFO - Received callback for node 69407872575bc060ce15b4a4 INFO: 172.21.0.1:33824 - "POST /node/69407872575bc060ce15b4a4 HTTP/1.0" 202 Accepted 2025-12-17 21:11:39,924 - lava_callback - INFO - Starting async job submit for node 69407872575bc060ce15b4a4 2025-12-17 21:11:39,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:11:40,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407872575bc060ce15b4a4 HTTP/1.1" 502 150 2025-12-17 21:11:40,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407872575bc060ce15b4a4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:11:40,042 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407872575bc060ce15b4a4 2025-12-17 21:11:40,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407872575bc060ce15b4a4 HTTP/1.1" 502 150 2025-12-17 21:11:40,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407872575bc060ce15b4a4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:11:42,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407872575bc060ce15b4a4 2025-12-17 21:11:42,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407872575bc060ce15b4a4 HTTP/1.1" 502 150 2025-12-17 21:11:42,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407872575bc060ce15b4a4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:11:43,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407961575bc060ce15b6db HTTP/1.1" 502 150 2025-12-17 21:11:43,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407961575bc060ce15b6db'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:11:46,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407872575bc060ce15b4a4 2025-12-17 21:11:46,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407872575bc060ce15b4a4 HTTP/1.1" 502 150 2025-12-17 21:11:46,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407872575bc060ce15b4a4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33834 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:11:47,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407961575bc060ce15b6db 2025-12-17 21:11:47,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407961575bc060ce15b6db HTTP/1.1" 502 150 2025-12-17 21:11:47,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407961575bc060ce15b6db'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:11:49,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940786f575bc060ce15b49e 2025-12-17 21:11:49,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940786f575bc060ce15b49e HTTP/1.1" 502 150 2025-12-17 21:11:49,747 - lava_callback - ERROR - Error processing callback for node 6940786f575bc060ce15b49e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940786f575bc060ce15b49e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940786f575bc060ce15b49e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940786f575bc060ce15b49e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:11:53,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940786a575bc060ce15b496 HTTP/1.1" 502 150 2025-12-17 21:11:53,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940786a575bc060ce15b496'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:11:54,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407872575bc060ce15b4a4 2025-12-17 21:11:55,028 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407961575bc060ce15b6db 2025-12-17 21:11:55,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407961575bc060ce15b6db HTTP/1.1" 502 150 2025-12-17 21:11:55,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407961575bc060ce15b6db'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42984 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:12:01,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940786a575bc060ce15b496 2025-12-17 21:12:01,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940786a575bc060ce15b496 HTTP/1.1" 502 150 2025-12-17 21:12:01,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940786a575bc060ce15b496'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47606 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:12:09,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407872575bc060ce15b4a4 HTTP/1.1" 502 150 2025-12-17 21:12:09,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407872575bc060ce15b4a4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:12:11,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407961575bc060ce15b6db 2025-12-17 21:12:11,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407961575bc060ce15b6db HTTP/1.1" 502 150 2025-12-17 21:12:11,030 - lava_callback - ERROR - Error processing callback for node 69407961575bc060ce15b6db: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407961575bc060ce15b6db (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407961575bc060ce15b6db (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407961575bc060ce15b6db (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:38914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38928 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:12:17,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940786a575bc060ce15b496 2025-12-17 21:12:17,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940786a575bc060ce15b496 HTTP/1.1" 502 150 2025-12-17 21:12:17,015 - lava_callback - ERROR - Error processing callback for node 6940786a575bc060ce15b496: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940786a575bc060ce15b496 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940786a575bc060ce15b496 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940786a575bc060ce15b496 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:12:24,184 - lava_callback - INFO - Received callback for node 69408025575bc060ce15c6cd INFO: 172.21.0.1:52264 - "POST /node/69408025575bc060ce15c6cd HTTP/1.0" 202 Accepted 2025-12-17 21:12:24,192 - lava_callback - INFO - Starting async job submit for node 69408025575bc060ce15c6cd 2025-12-17 21:12:24,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:12:24,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408025575bc060ce15c6cd HTTP/1.1" 502 150 2025-12-17 21:12:24,240 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408025575bc060ce15c6cd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:12:24,240 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408025575bc060ce15c6cd 2025-12-17 21:12:24,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408025575bc060ce15c6cd HTTP/1.1" 502 150 2025-12-17 21:12:24,241 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408025575bc060ce15c6cd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:12:25,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407872575bc060ce15b4a4 2025-12-17 21:12:25,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407872575bc060ce15b4a4 HTTP/1.1" 502 150 2025-12-17 21:12:25,394 - lava_callback - ERROR - Error processing callback for node 69407872575bc060ce15b4a4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407872575bc060ce15b4a4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407872575bc060ce15b4a4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407872575bc060ce15b4a4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:12:26,241 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408025575bc060ce15c6cd 2025-12-17 21:12:26,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408025575bc060ce15c6cd HTTP/1.1" 502 150 2025-12-17 21:12:26,242 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408025575bc060ce15c6cd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52266 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:12:30,242 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408025575bc060ce15c6cd 2025-12-17 21:12:30,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408025575bc060ce15c6cd HTTP/1.1" 502 150 2025-12-17 21:12:30,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408025575bc060ce15c6cd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42646 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:12:38,244 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408025575bc060ce15c6cd 2025-12-17 21:12:38,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408025575bc060ce15c6cd HTTP/1.1" 502 150 2025-12-17 21:12:38,245 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408025575bc060ce15c6cd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37318 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:12:54,245 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408025575bc060ce15c6cd 2025-12-17 21:12:54,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408025575bc060ce15c6cd HTTP/1.1" 502 150 2025-12-17 21:12:54,247 - lava_callback - ERROR - Error processing callback for node 69408025575bc060ce15c6cd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408025575bc060ce15c6cd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408025575bc060ce15c6cd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408025575bc060ce15c6cd (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:42380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36560 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:13:19,548 - lava_callback - INFO - Received callback for node 69407a94575bc060ce15b82f INFO: 172.21.0.1:36562 - "POST /node/69407a94575bc060ce15b82f HTTP/1.0" 202 Accepted 2025-12-17 21:13:19,557 - lava_callback - INFO - Starting async job submit for node 69407a94575bc060ce15b82f 2025-12-17 21:13:19,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:39120 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:13:34,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a94575bc060ce15b82f HTTP/1.1" 502 150 2025-12-17 21:13:34,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a94575bc060ce15b82f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:13:34,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a94575bc060ce15b82f 2025-12-17 21:13:34,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a94575bc060ce15b82f HTTP/1.1" 502 150 2025-12-17 21:13:34,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a94575bc060ce15b82f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40940 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:13:36,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a94575bc060ce15b82f 2025-12-17 21:13:36,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a94575bc060ce15b82f HTTP/1.1" 502 150 2025-12-17 21:13:36,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a94575bc060ce15b82f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:13:40,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a94575bc060ce15b82f 2025-12-17 21:13:40,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a94575bc060ce15b82f HTTP/1.1" 502 150 2025-12-17 21:13:40,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a94575bc060ce15b82f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40840 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:13:48,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a94575bc060ce15b82f 2025-12-17 21:13:48,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a94575bc060ce15b82f HTTP/1.1" 502 150 2025-12-17 21:13:48,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a94575bc060ce15b82f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42478 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:13:59,573 - lava_callback - INFO - Received callback for node 69407ad0575bc060ce15b86d INFO: 172.21.0.1:42480 - "POST /node/69407ad0575bc060ce15b86d HTTP/1.0" 202 Accepted 2025-12-17 21:13:59,580 - lava_callback - INFO - Starting async job submit for node 69407ad0575bc060ce15b86d 2025-12-17 21:13:59,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:13:59,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad0575bc060ce15b86d HTTP/1.1" 502 150 2025-12-17 21:13:59,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad0575bc060ce15b86d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:13:59,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad0575bc060ce15b86d 2025-12-17 21:13:59,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad0575bc060ce15b86d HTTP/1.1" 502 150 2025-12-17 21:13:59,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad0575bc060ce15b86d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43950 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:14:01,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad0575bc060ce15b86d 2025-12-17 21:14:01,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad0575bc060ce15b86d HTTP/1.1" 502 150 2025-12-17 21:14:01,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad0575bc060ce15b86d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:14:04,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a94575bc060ce15b82f 2025-12-17 21:14:04,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a94575bc060ce15b82f HTTP/1.1" 502 150 2025-12-17 21:14:04,903 - lava_callback - ERROR - Error processing callback for node 69407a94575bc060ce15b82f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407a94575bc060ce15b82f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407a94575bc060ce15b82f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407a94575bc060ce15b82f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:14:05,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad0575bc060ce15b86d 2025-12-17 21:14:05,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad0575bc060ce15b86d HTTP/1.1" 502 150 2025-12-17 21:14:05,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad0575bc060ce15b86d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43958 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:14:13,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad0575bc060ce15b86d 2025-12-17 21:14:13,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad0575bc060ce15b86d HTTP/1.1" 502 150 2025-12-17 21:14:13,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad0575bc060ce15b86d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44862 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:14:29,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad0575bc060ce15b86d 2025-12-17 21:14:29,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad0575bc060ce15b86d HTTP/1.1" 502 150 2025-12-17 21:14:29,615 - lava_callback - ERROR - Error processing callback for node 69407ad0575bc060ce15b86d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad0575bc060ce15b86d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad0575bc060ce15b86d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad0575bc060ce15b86d (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:54246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44588 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:15:03,894 - lava_callback - INFO - Received callback for node 69408451575bc060ce15d716 INFO: 172.21.0.1:44598 - "POST /node/69408451575bc060ce15d716 HTTP/1.0" 202 Accepted 2025-12-17 21:15:03,902 - lava_callback - INFO - Starting async job submit for node 69408451575bc060ce15d716 2025-12-17 21:15:03,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:15:03,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408451575bc060ce15d716 HTTP/1.1" 502 150 2025-12-17 21:15:03,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408451575bc060ce15d716'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:15:03,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408451575bc060ce15d716 2025-12-17 21:15:03,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408451575bc060ce15d716 HTTP/1.1" 502 150 2025-12-17 21:15:03,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408451575bc060ce15d716'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:15:05,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408451575bc060ce15d716 2025-12-17 21:15:05,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408451575bc060ce15d716 HTTP/1.1" 502 150 2025-12-17 21:15:05,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408451575bc060ce15d716'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44600 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:15:09,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408451575bc060ce15d716 2025-12-17 21:15:09,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408451575bc060ce15d716 HTTP/1.1" 502 150 2025-12-17 21:15:09,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408451575bc060ce15d716'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57714 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:15:17,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408451575bc060ce15d716 2025-12-17 21:15:17,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408451575bc060ce15d716 HTTP/1.1" 502 150 2025-12-17 21:15:17,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408451575bc060ce15d716'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39928 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:15:33,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408451575bc060ce15d716 2025-12-17 21:15:33,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408451575bc060ce15d716 HTTP/1.1" 502 150 2025-12-17 21:15:33,957 - lava_callback - ERROR - Error processing callback for node 69408451575bc060ce15d716: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408451575bc060ce15d716 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408451575bc060ce15d716 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408451575bc060ce15d716 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:56014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37586 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:16:07,210 - lava_callback - INFO - Received callback for node 69407a96575bc060ce15b834 INFO: 172.21.0.1:37596 - "POST /node/69407a96575bc060ce15b834 HTTP/1.0" 202 Accepted 2025-12-17 21:16:07,215 - lava_callback - INFO - Starting async job submit for node 69407a96575bc060ce15b834 2025-12-17 21:16:07,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:16:07,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a96575bc060ce15b834 HTTP/1.1" 502 150 2025-12-17 21:16:07,247 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a96575bc060ce15b834'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:16:07,247 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a96575bc060ce15b834 2025-12-17 21:16:07,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a96575bc060ce15b834 HTTP/1.1" 502 150 2025-12-17 21:16:07,248 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a96575bc060ce15b834'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:16:09,248 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a96575bc060ce15b834 2025-12-17 21:16:09,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a96575bc060ce15b834 HTTP/1.1" 502 150 2025-12-17 21:16:09,249 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a96575bc060ce15b834'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:16:13,249 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a96575bc060ce15b834 2025-12-17 21:16:13,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a96575bc060ce15b834 HTTP/1.1" 502 150 2025-12-17 21:16:13,250 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a96575bc060ce15b834'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53344 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:16:19,888 - lava_callback - INFO - Received callback for node 69407830575bc060ce15b43a INFO: 172.21.0.1:44578 - "POST /node/69407830575bc060ce15b43a HTTP/1.0" 202 Accepted 2025-12-17 21:16:19,895 - lava_callback - INFO - Starting async job submit for node 69407830575bc060ce15b43a 2025-12-17 21:16:19,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:16:19,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407830575bc060ce15b43a HTTP/1.1" 502 150 2025-12-17 21:16:19,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407830575bc060ce15b43a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:16:19,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407830575bc060ce15b43a 2025-12-17 21:16:19,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407830575bc060ce15b43a HTTP/1.1" 502 150 2025-12-17 21:16:19,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407830575bc060ce15b43a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:16:21,251 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a96575bc060ce15b834 2025-12-17 21:16:21,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a96575bc060ce15b834 HTTP/1.1" 502 150 2025-12-17 21:16:21,252 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a96575bc060ce15b834'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:16:21,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407830575bc060ce15b43a 2025-12-17 21:16:21,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407830575bc060ce15b43a HTTP/1.1" 502 150 2025-12-17 21:16:21,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407830575bc060ce15b43a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:16:25,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407830575bc060ce15b43a 2025-12-17 21:16:25,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407830575bc060ce15b43a HTTP/1.1" 502 150 2025-12-17 21:16:25,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407830575bc060ce15b43a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44592 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:16:26,924 - lava_callback - INFO - Received callback for node 69407a97575bc060ce15b838 INFO: 172.21.0.1:44600 - "POST /node/69407a97575bc060ce15b838 HTTP/1.0" 202 Accepted 2025-12-17 21:16:26,933 - lava_callback - INFO - Starting async job submit for node 69407a97575bc060ce15b838 2025-12-17 21:16:26,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:16:26,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a97575bc060ce15b838 HTTP/1.1" 502 150 2025-12-17 21:16:26,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a97575bc060ce15b838'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:16:26,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a97575bc060ce15b838 2025-12-17 21:16:26,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a97575bc060ce15b838 HTTP/1.1" 502 150 2025-12-17 21:16:26,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a97575bc060ce15b838'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:16:27,633 - lava_callback - INFO - Received callback for node 69407a98575bc060ce15b83d INFO: 172.21.0.1:44608 - "POST /node/69407a98575bc060ce15b83d HTTP/1.0" 202 Accepted 2025-12-17 21:16:27,638 - lava_callback - INFO - Starting async job submit for node 69407a98575bc060ce15b83d 2025-12-17 21:16:27,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:16:27,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a98575bc060ce15b83d HTTP/1.1" 502 150 2025-12-17 21:16:27,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a98575bc060ce15b83d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:16:27,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a98575bc060ce15b83d 2025-12-17 21:16:27,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a98575bc060ce15b83d HTTP/1.1" 502 150 2025-12-17 21:16:27,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a98575bc060ce15b83d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:16:28,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a97575bc060ce15b838 2025-12-17 21:16:28,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a97575bc060ce15b838 HTTP/1.1" 502 150 2025-12-17 21:16:28,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a97575bc060ce15b838'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:16:29,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a98575bc060ce15b83d 2025-12-17 21:16:29,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a98575bc060ce15b83d HTTP/1.1" 502 150 2025-12-17 21:16:29,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a98575bc060ce15b83d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:16:32,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a97575bc060ce15b838 2025-12-17 21:16:32,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a97575bc060ce15b838 HTTP/1.1" 502 150 2025-12-17 21:16:32,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a97575bc060ce15b838'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:16:33,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a98575bc060ce15b83d 2025-12-17 21:16:33,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a98575bc060ce15b83d HTTP/1.1" 502 150 2025-12-17 21:16:33,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a98575bc060ce15b83d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:16:33,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407830575bc060ce15b43a 2025-12-17 21:16:33,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407830575bc060ce15b43a HTTP/1.1" 502 150 2025-12-17 21:16:33,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407830575bc060ce15b43a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58158 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:16:37,252 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a96575bc060ce15b834 2025-12-17 21:16:37,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a96575bc060ce15b834 HTTP/1.1" 502 150 2025-12-17 21:16:37,254 - lava_callback - ERROR - Error processing callback for node 69407a96575bc060ce15b834: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407a96575bc060ce15b834 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407a96575bc060ce15b834 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407a96575bc060ce15b834 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:16:40,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a97575bc060ce15b838 2025-12-17 21:16:41,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a98575bc060ce15b83d 2025-12-17 21:16:41,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a98575bc060ce15b83d HTTP/1.1" 502 150 2025-12-17 21:16:41,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a98575bc060ce15b83d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:32892 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:16:49,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407830575bc060ce15b43a 2025-12-17 21:16:49,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407830575bc060ce15b43a HTTP/1.1" 502 150 2025-12-17 21:16:49,997 - lava_callback - ERROR - Error processing callback for node 69407830575bc060ce15b43a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407830575bc060ce15b43a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407830575bc060ce15b43a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407830575bc060ce15b43a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:16:56,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a97575bc060ce15b838 HTTP/1.1" 502 150 2025-12-17 21:16:56,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407a97575bc060ce15b838'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33430 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:16:57,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a98575bc060ce15b83d 2025-12-17 21:16:57,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a98575bc060ce15b83d HTTP/1.1" 502 150 2025-12-17 21:16:57,652 - lava_callback - ERROR - Error processing callback for node 69407a98575bc060ce15b83d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407a98575bc060ce15b83d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407a98575bc060ce15b83d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407a98575bc060ce15b83d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:16:58,173 - lava_callback - INFO - Received callback for node 6940c52b1e3c24e95681d68d INFO: 172.21.0.1:33440 - "POST /node/6940c52b1e3c24e95681d68d HTTP/1.0" 202 Accepted 2025-12-17 21:16:58,181 - lava_callback - INFO - Starting async job submit for node 6940c52b1e3c24e95681d68d 2025-12-17 21:16:58,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:16:58,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c52b1e3c24e95681d68d HTTP/1.1" 502 150 2025-12-17 21:16:58,236 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c52b1e3c24e95681d68d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:16:58,236 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c52b1e3c24e95681d68d 2025-12-17 21:16:58,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c52b1e3c24e95681d68d HTTP/1.1" 502 150 2025-12-17 21:16:58,236 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c52b1e3c24e95681d68d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:00,237 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c52b1e3c24e95681d68d 2025-12-17 21:17:00,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c52b1e3c24e95681d68d HTTP/1.1" 502 150 2025-12-17 21:17:00,238 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c52b1e3c24e95681d68d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37448 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:17:04,238 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c52b1e3c24e95681d68d 2025-12-17 21:17:04,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c52b1e3c24e95681d68d HTTP/1.1" 502 150 2025-12-17 21:17:04,239 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c52b1e3c24e95681d68d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37456 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:17:12,112 - lava_callback - INFO - Received callback for node 69407ad3575bc060ce15b87c INFO: 172.21.0.1:54430 - "POST /node/69407ad3575bc060ce15b87c HTTP/1.0" 202 Accepted 2025-12-17 21:17:12,118 - lava_callback - INFO - Starting async job submit for node 69407ad3575bc060ce15b87c 2025-12-17 21:17:12,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:17:12,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad3575bc060ce15b87c HTTP/1.1" 502 150 2025-12-17 21:17:12,139 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad3575bc060ce15b87c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:12,139 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad3575bc060ce15b87c 2025-12-17 21:17:12,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad3575bc060ce15b87c HTTP/1.1" 502 150 2025-12-17 21:17:12,140 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad3575bc060ce15b87c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:12,239 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c52b1e3c24e95681d68d 2025-12-17 21:17:12,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c52b1e3c24e95681d68d HTTP/1.1" 502 150 2025-12-17 21:17:12,240 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c52b1e3c24e95681d68d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:12,369 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407a97575bc060ce15b838 2025-12-17 21:17:12,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407a97575bc060ce15b838 HTTP/1.1" 502 150 2025-12-17 21:17:12,370 - lava_callback - ERROR - Error processing callback for node 69407a97575bc060ce15b838: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407a97575bc060ce15b838 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407a97575bc060ce15b838 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407a97575bc060ce15b838 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:17:12,870 - lava_callback - INFO - Received callback for node 69407ad4575bc060ce15b87f INFO: 172.21.0.1:54432 - "POST /node/69407ad4575bc060ce15b87f HTTP/1.0" 202 Accepted 2025-12-17 21:17:12,876 - lava_callback - INFO - Starting async job submit for node 69407ad4575bc060ce15b87f 2025-12-17 21:17:12,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:17:12,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad4575bc060ce15b87f HTTP/1.1" 502 150 2025-12-17 21:17:12,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad4575bc060ce15b87f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:12,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad4575bc060ce15b87f 2025-12-17 21:17:12,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad4575bc060ce15b87f HTTP/1.1" 502 150 2025-12-17 21:17:12,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad4575bc060ce15b87f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:13,645 - lava_callback - INFO - Received callback for node 69407ad6575bc060ce15b882 INFO: 172.21.0.1:54446 - "POST /node/69407ad6575bc060ce15b882 HTTP/1.0" 202 Accepted 2025-12-17 21:17:13,652 - lava_callback - INFO - Starting async job submit for node 69407ad6575bc060ce15b882 2025-12-17 21:17:13,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:17:13,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad6575bc060ce15b882 HTTP/1.1" 502 150 2025-12-17 21:17:13,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad6575bc060ce15b882'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:13,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad6575bc060ce15b882 2025-12-17 21:17:13,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad6575bc060ce15b882 HTTP/1.1" 502 150 2025-12-17 21:17:13,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad6575bc060ce15b882'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:13,821 - lava_callback - INFO - Received callback for node 69407ad2575bc060ce15b879 INFO: 172.21.0.1:54452 - "POST /node/69407ad2575bc060ce15b879 HTTP/1.0" 202 Accepted 2025-12-17 21:17:13,828 - lava_callback - INFO - Starting async job submit for node 69407ad2575bc060ce15b879 2025-12-17 21:17:13,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:17:13,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad2575bc060ce15b879 HTTP/1.1" 502 150 2025-12-17 21:17:13,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad2575bc060ce15b879'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:13,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad2575bc060ce15b879 2025-12-17 21:17:13,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad2575bc060ce15b879 HTTP/1.1" 502 150 2025-12-17 21:17:13,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad2575bc060ce15b879'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:14,140 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad3575bc060ce15b87c 2025-12-17 21:17:14,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad3575bc060ce15b87c HTTP/1.1" 502 150 2025-12-17 21:17:14,141 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad3575bc060ce15b87c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:14,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad4575bc060ce15b87f 2025-12-17 21:17:14,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad4575bc060ce15b87f HTTP/1.1" 502 150 2025-12-17 21:17:14,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad4575bc060ce15b87f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54464 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:17:15,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad6575bc060ce15b882 2025-12-17 21:17:15,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad6575bc060ce15b882 HTTP/1.1" 502 150 2025-12-17 21:17:15,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad6575bc060ce15b882'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:15,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad2575bc060ce15b879 2025-12-17 21:17:15,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad2575bc060ce15b879 HTTP/1.1" 502 150 2025-12-17 21:17:15,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad2575bc060ce15b879'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54474 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:17:18,141 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad3575bc060ce15b87c 2025-12-17 21:17:18,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad3575bc060ce15b87c HTTP/1.1" 502 150 2025-12-17 21:17:18,142 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad3575bc060ce15b87c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:18,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad4575bc060ce15b87f 2025-12-17 21:17:18,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad4575bc060ce15b87f HTTP/1.1" 502 150 2025-12-17 21:17:18,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad4575bc060ce15b87f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:19,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad6575bc060ce15b882 2025-12-17 21:17:19,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad6575bc060ce15b882 HTTP/1.1" 502 150 2025-12-17 21:17:19,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad6575bc060ce15b882'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:19,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad2575bc060ce15b879 2025-12-17 21:17:19,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad2575bc060ce15b879 HTTP/1.1" 502 150 2025-12-17 21:17:19,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad2575bc060ce15b879'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:26,143 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad3575bc060ce15b87c 2025-12-17 21:17:26,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad3575bc060ce15b87c HTTP/1.1" 502 150 2025-12-17 21:17:26,144 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad3575bc060ce15b87c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56812 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:17:26,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad4575bc060ce15b87f 2025-12-17 21:17:26,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad4575bc060ce15b87f HTTP/1.1" 502 150 2025-12-17 21:17:26,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad4575bc060ce15b87f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:27,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad6575bc060ce15b882 2025-12-17 21:17:27,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad6575bc060ce15b882 HTTP/1.1" 502 150 2025-12-17 21:17:27,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad6575bc060ce15b882'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:27,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad2575bc060ce15b879 2025-12-17 21:17:27,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad2575bc060ce15b879 HTTP/1.1" 502 150 2025-12-17 21:17:27,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad2575bc060ce15b879'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:28,241 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c52b1e3c24e95681d68d 2025-12-17 21:17:30,277 - lava_callback - INFO - Received callback for node 69407ad7575bc060ce15b88d INFO: 172.21.0.1:35614 - "POST /node/69407ad7575bc060ce15b88d HTTP/1.0" 202 Accepted 2025-12-17 21:17:30,283 - lava_callback - INFO - Starting async job submit for node 69407ad7575bc060ce15b88d 2025-12-17 21:17:30,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:17:30,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad7575bc060ce15b88d HTTP/1.1" 502 150 2025-12-17 21:17:30,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad7575bc060ce15b88d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:30,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad7575bc060ce15b88d 2025-12-17 21:17:30,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad7575bc060ce15b88d HTTP/1.1" 502 150 2025-12-17 21:17:30,314 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad7575bc060ce15b88d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:30,923 - lava_callback - INFO - Received callback for node 69407ad9575bc060ce15b890 INFO: 172.21.0.1:35622 - "POST /node/69407ad9575bc060ce15b890 HTTP/1.0" 202 Accepted 2025-12-17 21:17:30,929 - lava_callback - INFO - Starting async job submit for node 69407ad9575bc060ce15b890 2025-12-17 21:17:30,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:17:30,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad9575bc060ce15b890 HTTP/1.1" 502 150 2025-12-17 21:17:30,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad9575bc060ce15b890'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:30,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad9575bc060ce15b890 2025-12-17 21:17:30,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad9575bc060ce15b890 HTTP/1.1" 502 150 2025-12-17 21:17:30,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad9575bc060ce15b890'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:31,553 - lava_callback - INFO - Received callback for node 69407ade575bc060ce15b893 INFO: 172.21.0.1:35636 - "POST /node/69407ade575bc060ce15b893 HTTP/1.0" 202 Accepted 2025-12-17 21:17:31,558 - lava_callback - INFO - Starting async job submit for node 69407ade575bc060ce15b893 2025-12-17 21:17:31,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:17:31,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ade575bc060ce15b893 HTTP/1.1" 502 150 2025-12-17 21:17:31,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ade575bc060ce15b893'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:31,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ade575bc060ce15b893 2025-12-17 21:17:31,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ade575bc060ce15b893 HTTP/1.1" 502 150 2025-12-17 21:17:31,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ade575bc060ce15b893'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:32,167 - lava_callback - INFO - Received callback for node 69407ae0575bc060ce15b896 INFO: 172.21.0.1:35640 - "POST /node/69407ae0575bc060ce15b896 HTTP/1.0" 202 Accepted 2025-12-17 21:17:32,173 - lava_callback - INFO - Starting async job submit for node 69407ae0575bc060ce15b896 2025-12-17 21:17:32,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:17:32,315 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad7575bc060ce15b88d 2025-12-17 21:17:32,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad7575bc060ce15b88d HTTP/1.1" 502 150 2025-12-17 21:17:32,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad7575bc060ce15b88d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:32,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad9575bc060ce15b890 2025-12-17 21:17:32,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad9575bc060ce15b890 HTTP/1.1" 502 150 2025-12-17 21:17:32,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad9575bc060ce15b890'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:33,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ade575bc060ce15b893 2025-12-17 21:17:33,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ade575bc060ce15b893 HTTP/1.1" 502 150 2025-12-17 21:17:33,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ade575bc060ce15b893'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:36,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad7575bc060ce15b88d 2025-12-17 21:17:36,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad7575bc060ce15b88d HTTP/1.1" 502 150 2025-12-17 21:17:36,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad7575bc060ce15b88d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35642 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:17:36,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad9575bc060ce15b890 2025-12-17 21:17:36,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad9575bc060ce15b890 HTTP/1.1" 502 150 2025-12-17 21:17:36,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad9575bc060ce15b890'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:37,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ade575bc060ce15b893 2025-12-17 21:17:37,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ade575bc060ce15b893 HTTP/1.1" 502 150 2025-12-17 21:17:37,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ade575bc060ce15b893'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:40,075 - lava_callback - INFO - Received callback for node 694076bf575bc060ce15b19c INFO: 172.21.0.1:54810 - "POST /node/694076bf575bc060ce15b19c HTTP/1.0" 202 Accepted 2025-12-17 21:17:40,082 - lava_callback - INFO - Starting async job submit for node 694076bf575bc060ce15b19c 2025-12-17 21:17:40,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:17:40,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bf575bc060ce15b19c HTTP/1.1" 502 150 2025-12-17 21:17:40,231 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076bf575bc060ce15b19c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:40,231 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076bf575bc060ce15b19c 2025-12-17 21:17:40,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bf575bc060ce15b19c HTTP/1.1" 502 150 2025-12-17 21:17:40,232 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076bf575bc060ce15b19c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:41,693 - lava_callback - INFO - Received callback for node 69407b0f575bc060ce15b926 INFO: 172.21.0.1:54824 - "POST /node/69407b0f575bc060ce15b926 HTTP/1.0" 202 Accepted 2025-12-17 21:17:41,700 - lava_callback - INFO - Starting async job submit for node 69407b0f575bc060ce15b926 2025-12-17 21:17:41,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:17:41,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b0f575bc060ce15b926 HTTP/1.1" 502 150 2025-12-17 21:17:41,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b0f575bc060ce15b926'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:41,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b0f575bc060ce15b926 2025-12-17 21:17:41,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b0f575bc060ce15b926 HTTP/1.1" 502 150 2025-12-17 21:17:41,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b0f575bc060ce15b926'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:42,144 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad3575bc060ce15b87c 2025-12-17 21:17:42,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad3575bc060ce15b87c HTTP/1.1" 502 150 2025-12-17 21:17:42,146 - lava_callback - ERROR - Error processing callback for node 69407ad3575bc060ce15b87c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad3575bc060ce15b87c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad3575bc060ce15b87c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad3575bc060ce15b87c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:17:42,232 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076bf575bc060ce15b19c 2025-12-17 21:17:42,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bf575bc060ce15b19c HTTP/1.1" 502 150 2025-12-17 21:17:42,233 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076bf575bc060ce15b19c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:42,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad4575bc060ce15b87f 2025-12-17 21:17:42,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad4575bc060ce15b87f HTTP/1.1" 502 150 2025-12-17 21:17:42,891 - lava_callback - ERROR - Error processing callback for node 69407ad4575bc060ce15b87f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad4575bc060ce15b87f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad4575bc060ce15b87f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad4575bc060ce15b87f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:17:43,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad6575bc060ce15b882 2025-12-17 21:17:43,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad6575bc060ce15b882 HTTP/1.1" 502 150 2025-12-17 21:17:43,680 - lava_callback - ERROR - Error processing callback for node 69407ad6575bc060ce15b882: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad6575bc060ce15b882 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad6575bc060ce15b882 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad6575bc060ce15b882 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:17:43,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c52b1e3c24e95681d68d HTTP/1.1" 502 150 2025-12-17 21:17:43,728 - lava_callback - ERROR - Error processing callback for node 6940c52b1e3c24e95681d68d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c52b1e3c24e95681d68d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c52b1e3c24e95681d68d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c52b1e3c24e95681d68d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:17:43,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b0f575bc060ce15b926 2025-12-17 21:17:43,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b0f575bc060ce15b926 HTTP/1.1" 502 150 2025-12-17 21:17:43,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b0f575bc060ce15b926'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:43,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad2575bc060ce15b879 2025-12-17 21:17:43,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad2575bc060ce15b879 HTTP/1.1" 502 150 2025-12-17 21:17:43,875 - lava_callback - ERROR - Error processing callback for node 69407ad2575bc060ce15b879: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad2575bc060ce15b879 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad2575bc060ce15b879 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad2575bc060ce15b879 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:17:44,093 - lava_callback - INFO - Received callback for node 694089b2575bc060ce15eb9f INFO: 172.21.0.1:54826 - "POST /node/694089b2575bc060ce15eb9f HTTP/1.0" 202 Accepted 2025-12-17 21:17:44,106 - lava_callback - INFO - Starting async job submit for node 694089b2575bc060ce15eb9f 2025-12-17 21:17:44,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:17:44,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b2575bc060ce15eb9f HTTP/1.1" 502 150 2025-12-17 21:17:44,155 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b2575bc060ce15eb9f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:44,155 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b2575bc060ce15eb9f 2025-12-17 21:17:44,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b2575bc060ce15eb9f HTTP/1.1" 502 150 2025-12-17 21:17:44,156 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b2575bc060ce15eb9f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:44,317 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad7575bc060ce15b88d 2025-12-17 21:17:44,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad7575bc060ce15b88d HTTP/1.1" 502 150 2025-12-17 21:17:44,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad7575bc060ce15b88d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:44,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad9575bc060ce15b890 2025-12-17 21:17:44,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad9575bc060ce15b890 HTTP/1.1" 502 150 2025-12-17 21:17:44,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ad9575bc060ce15b890'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:45,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ade575bc060ce15b893 2025-12-17 21:17:45,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ade575bc060ce15b893 HTTP/1.1" 502 150 2025-12-17 21:17:45,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ade575bc060ce15b893'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:46,156 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b2575bc060ce15eb9f 2025-12-17 21:17:46,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b2575bc060ce15eb9f HTTP/1.1" 502 150 2025-12-17 21:17:46,157 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b2575bc060ce15eb9f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:46,233 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076bf575bc060ce15b19c 2025-12-17 21:17:46,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bf575bc060ce15b19c HTTP/1.1" 502 150 2025-12-17 21:17:46,234 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076bf575bc060ce15b19c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54842 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:17:47,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ae0575bc060ce15b896 HTTP/1.1" 502 150 2025-12-17 21:17:47,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ae0575bc060ce15b896'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:47,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ae0575bc060ce15b896 2025-12-17 21:17:47,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ae0575bc060ce15b896 HTTP/1.1" 502 150 2025-12-17 21:17:47,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ae0575bc060ce15b896'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:47,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b0f575bc060ce15b926 2025-12-17 21:17:47,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b0f575bc060ce15b926 HTTP/1.1" 502 150 2025-12-17 21:17:47,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b0f575bc060ce15b926'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:49,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ae0575bc060ce15b896 2025-12-17 21:17:49,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ae0575bc060ce15b896 HTTP/1.1" 502 150 2025-12-17 21:17:49,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ae0575bc060ce15b896'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:50,157 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b2575bc060ce15eb9f 2025-12-17 21:17:50,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b2575bc060ce15eb9f HTTP/1.1" 502 150 2025-12-17 21:17:50,159 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b2575bc060ce15eb9f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:53,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ae0575bc060ce15b896 2025-12-17 21:17:53,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ae0575bc060ce15b896 HTTP/1.1" 502 150 2025-12-17 21:17:53,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ae0575bc060ce15b896'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:54,235 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076bf575bc060ce15b19c 2025-12-17 21:17:54,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bf575bc060ce15b19c HTTP/1.1" 502 150 2025-12-17 21:17:54,236 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076bf575bc060ce15b19c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:17:55,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b0f575bc060ce15b926 2025-12-17 21:17:55,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b0f575bc060ce15b926 HTTP/1.1" 502 150 2025-12-17 21:17:55,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b0f575bc060ce15b926'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40886 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:17:58,159 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b2575bc060ce15eb9f 2025-12-17 21:18:00,319 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad7575bc060ce15b88d 2025-12-17 21:18:00,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad7575bc060ce15b88d HTTP/1.1" 502 150 2025-12-17 21:18:00,320 - lava_callback - ERROR - Error processing callback for node 69407ad7575bc060ce15b88d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad7575bc060ce15b88d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad7575bc060ce15b88d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad7575bc060ce15b88d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:18:00,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ad9575bc060ce15b890 2025-12-17 21:18:00,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ad9575bc060ce15b890 HTTP/1.1" 502 150 2025-12-17 21:18:00,968 - lava_callback - ERROR - Error processing callback for node 69407ad9575bc060ce15b890: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad9575bc060ce15b890 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad9575bc060ce15b890 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ad9575bc060ce15b890 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:38596 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:18:01,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ae0575bc060ce15b896 2025-12-17 21:18:01,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ae0575bc060ce15b896 HTTP/1.1" 502 150 2025-12-17 21:18:01,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ae0575bc060ce15b896'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:01,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ade575bc060ce15b893 2025-12-17 21:18:01,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ade575bc060ce15b893 HTTP/1.1" 502 150 2025-12-17 21:18:01,599 - lava_callback - ERROR - Error processing callback for node 69407ade575bc060ce15b893: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ade575bc060ce15b893 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ade575bc060ce15b893 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ade575bc060ce15b893 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:38610 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:18:10,236 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076bf575bc060ce15b19c 2025-12-17 21:18:10,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076bf575bc060ce15b19c HTTP/1.1" 502 150 2025-12-17 21:18:10,238 - lava_callback - ERROR - Error processing callback for node 694076bf575bc060ce15b19c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076bf575bc060ce15b19c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076bf575bc060ce15b19c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076bf575bc060ce15b19c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:18:11,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b0f575bc060ce15b926 2025-12-17 21:18:11,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b0f575bc060ce15b926 HTTP/1.1" 502 150 2025-12-17 21:18:11,742 - lava_callback - ERROR - Error processing callback for node 69407b0f575bc060ce15b926: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b0f575bc060ce15b926 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b0f575bc060ce15b926 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b0f575bc060ce15b926 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:18:13,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b2575bc060ce15eb9f HTTP/1.1" 502 150 2025-12-17 21:18:13,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b2575bc060ce15eb9f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52236 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:18:17,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ae0575bc060ce15b896 2025-12-17 21:18:17,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ae0575bc060ce15b896 HTTP/1.1" 502 150 2025-12-17 21:18:17,576 - lava_callback - ERROR - Error processing callback for node 69407ae0575bc060ce15b896: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ae0575bc060ce15b896 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ae0575bc060ce15b896 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ae0575bc060ce15b896 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:59020 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:18:29,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b2575bc060ce15eb9f 2025-12-17 21:18:29,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b2575bc060ce15eb9f HTTP/1.1" 502 150 2025-12-17 21:18:29,426 - lava_callback - ERROR - Error processing callback for node 694089b2575bc060ce15eb9f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b2575bc060ce15eb9f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b2575bc060ce15eb9f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b2575bc060ce15eb9f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:18:31,428 - lava_callback - INFO - Received callback for node 69407b16575bc060ce15b942 INFO: 172.21.0.1:36986 - "POST /node/69407b16575bc060ce15b942 HTTP/1.0" 202 Accepted 2025-12-17 21:18:31,433 - lava_callback - INFO - Starting async job submit for node 69407b16575bc060ce15b942 2025-12-17 21:18:31,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:18:31,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b16575bc060ce15b942 HTTP/1.1" 502 150 2025-12-17 21:18:31,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b16575bc060ce15b942'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:31,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b16575bc060ce15b942 2025-12-17 21:18:31,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b16575bc060ce15b942 HTTP/1.1" 502 150 2025-12-17 21:18:31,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b16575bc060ce15b942'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:32,185 - lava_callback - INFO - Received callback for node 69407b17575bc060ce15b945 INFO: 172.21.0.1:37000 - "POST /node/69407b17575bc060ce15b945 HTTP/1.0" 202 Accepted 2025-12-17 21:18:32,191 - lava_callback - INFO - Starting async job submit for node 69407b17575bc060ce15b945 2025-12-17 21:18:32,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:18:32,979 - lava_callback - INFO - Received callback for node 69407b1d575bc060ce15b94a INFO: 172.21.0.1:37008 - "POST /node/69407b1d575bc060ce15b94a HTTP/1.0" 202 Accepted 2025-12-17 21:18:32,985 - lava_callback - INFO - Starting async job submit for node 69407b1d575bc060ce15b94a 2025-12-17 21:18:32,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:18:32,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1d575bc060ce15b94a HTTP/1.1" 502 150 2025-12-17 21:18:32,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b1d575bc060ce15b94a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:32,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b1d575bc060ce15b94a 2025-12-17 21:18:32,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1d575bc060ce15b94a HTTP/1.1" 502 150 2025-12-17 21:18:32,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b1d575bc060ce15b94a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:33,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b16575bc060ce15b942 2025-12-17 21:18:33,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b16575bc060ce15b942 HTTP/1.1" 502 150 2025-12-17 21:18:33,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b16575bc060ce15b942'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:33,794 - lava_callback - INFO - Received callback for node 69407b1e575bc060ce15b94d INFO: 172.21.0.1:37012 - "POST /node/69407b1e575bc060ce15b94d HTTP/1.0" 202 Accepted 2025-12-17 21:18:33,800 - lava_callback - INFO - Starting async job submit for node 69407b1e575bc060ce15b94d 2025-12-17 21:18:33,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:18:33,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1e575bc060ce15b94d HTTP/1.1" 502 150 2025-12-17 21:18:33,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b1e575bc060ce15b94d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:33,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b1e575bc060ce15b94d 2025-12-17 21:18:33,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1e575bc060ce15b94d HTTP/1.1" 502 150 2025-12-17 21:18:33,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b1e575bc060ce15b94d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:34,598 - lava_callback - INFO - Received callback for node 69407b1f575bc060ce15b950 INFO: 172.21.0.1:37020 - "POST /node/69407b1f575bc060ce15b950 HTTP/1.0" 202 Accepted 2025-12-17 21:18:34,604 - lava_callback - INFO - Starting async job submit for node 69407b1f575bc060ce15b950 2025-12-17 21:18:34,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:18:34,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1f575bc060ce15b950 HTTP/1.1" 502 150 2025-12-17 21:18:34,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b1f575bc060ce15b950'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:34,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b1f575bc060ce15b950 2025-12-17 21:18:34,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1f575bc060ce15b950 HTTP/1.1" 502 150 2025-12-17 21:18:34,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b1f575bc060ce15b950'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:34,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b1d575bc060ce15b94a 2025-12-17 21:18:34,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1d575bc060ce15b94a HTTP/1.1" 502 150 2025-12-17 21:18:34,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b1d575bc060ce15b94a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:35,367 - lava_callback - INFO - Received callback for node 69407b21575bc060ce15b953 INFO: 172.21.0.1:37022 - "POST /node/69407b21575bc060ce15b953 HTTP/1.0" 202 Accepted 2025-12-17 21:18:35,373 - lava_callback - INFO - Starting async job submit for node 69407b21575bc060ce15b953 2025-12-17 21:18:35,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:18:35,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b21575bc060ce15b953 HTTP/1.1" 502 150 2025-12-17 21:18:35,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b21575bc060ce15b953'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:35,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b21575bc060ce15b953 2025-12-17 21:18:35,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b21575bc060ce15b953 HTTP/1.1" 502 150 2025-12-17 21:18:35,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b21575bc060ce15b953'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:35,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b1e575bc060ce15b94d 2025-12-17 21:18:35,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1e575bc060ce15b94d HTTP/1.1" 502 150 2025-12-17 21:18:35,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b1e575bc060ce15b94d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:36,145 - lava_callback - INFO - Received callback for node 69407b22575bc060ce15b956 INFO: 172.21.0.1:37030 - "POST /node/69407b22575bc060ce15b956 HTTP/1.0" 202 Accepted 2025-12-17 21:18:36,151 - lava_callback - INFO - Starting async job submit for node 69407b22575bc060ce15b956 2025-12-17 21:18:36,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:18:36,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b22575bc060ce15b956 HTTP/1.1" 502 150 2025-12-17 21:18:36,158 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b22575bc060ce15b956'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:36,158 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b22575bc060ce15b956 2025-12-17 21:18:36,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b22575bc060ce15b956 HTTP/1.1" 502 150 2025-12-17 21:18:36,159 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b22575bc060ce15b956'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37036 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:18:36,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b1f575bc060ce15b950 2025-12-17 21:18:36,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1f575bc060ce15b950 HTTP/1.1" 502 150 2025-12-17 21:18:36,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b1f575bc060ce15b950'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:36,905 - lava_callback - INFO - Received callback for node 69407b23575bc060ce15b959 INFO: 172.21.0.1:37052 - "POST /node/69407b23575bc060ce15b959 HTTP/1.0" 202 Accepted 2025-12-17 21:18:36,911 - lava_callback - INFO - Starting async job submit for node 69407b23575bc060ce15b959 2025-12-17 21:18:36,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:18:36,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b23575bc060ce15b959 HTTP/1.1" 502 150 2025-12-17 21:18:36,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b23575bc060ce15b959'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:36,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b23575bc060ce15b959 2025-12-17 21:18:36,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b23575bc060ce15b959 HTTP/1.1" 502 150 2025-12-17 21:18:36,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b23575bc060ce15b959'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:37,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b21575bc060ce15b953 2025-12-17 21:18:37,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b21575bc060ce15b953 HTTP/1.1" 502 150 2025-12-17 21:18:37,385 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b21575bc060ce15b953'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:37,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b16575bc060ce15b942 2025-12-17 21:18:37,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b16575bc060ce15b942 HTTP/1.1" 502 150 2025-12-17 21:18:37,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b16575bc060ce15b942'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:37,677 - lava_callback - INFO - Received callback for node 69407b25575bc060ce15b95c INFO: 172.21.0.1:37062 - "POST /node/69407b25575bc060ce15b95c HTTP/1.0" 202 Accepted 2025-12-17 21:18:37,684 - lava_callback - INFO - Starting async job submit for node 69407b25575bc060ce15b95c 2025-12-17 21:18:37,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:18:37,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b25575bc060ce15b95c HTTP/1.1" 502 150 2025-12-17 21:18:37,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b25575bc060ce15b95c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:37,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b25575bc060ce15b95c 2025-12-17 21:18:37,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b25575bc060ce15b95c HTTP/1.1" 502 150 2025-12-17 21:18:37,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b25575bc060ce15b95c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:38,159 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b22575bc060ce15b956 2025-12-17 21:18:38,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b22575bc060ce15b956 HTTP/1.1" 502 150 2025-12-17 21:18:38,160 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b22575bc060ce15b956'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:38,504 - lava_callback - INFO - Received callback for node 69407b26575bc060ce15b95f INFO: 172.21.0.1:37072 - "POST /node/69407b26575bc060ce15b95f HTTP/1.0" 202 Accepted 2025-12-17 21:18:38,514 - lava_callback - INFO - Starting async job submit for node 69407b26575bc060ce15b95f 2025-12-17 21:18:38,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:18:38,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b26575bc060ce15b95f HTTP/1.1" 502 150 2025-12-17 21:18:38,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b26575bc060ce15b95f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:38,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b26575bc060ce15b95f 2025-12-17 21:18:38,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b26575bc060ce15b95f HTTP/1.1" 502 150 2025-12-17 21:18:38,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b26575bc060ce15b95f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:38,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b23575bc060ce15b959 2025-12-17 21:18:38,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b23575bc060ce15b959 HTTP/1.1" 502 150 2025-12-17 21:18:38,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b23575bc060ce15b959'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:38,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b1d575bc060ce15b94a 2025-12-17 21:18:39,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1d575bc060ce15b94a HTTP/1.1" 502 150 2025-12-17 21:18:39,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b1d575bc060ce15b94a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:39,260 - lava_callback - INFO - Received callback for node 69407b27575bc060ce15b962 INFO: 172.21.0.1:37082 - "POST /node/69407b27575bc060ce15b962 HTTP/1.0" 202 Accepted 2025-12-17 21:18:39,266 - lava_callback - INFO - Starting async job submit for node 69407b27575bc060ce15b962 2025-12-17 21:18:39,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:18:39,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b25575bc060ce15b95c 2025-12-17 21:18:39,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b25575bc060ce15b95c HTTP/1.1" 502 150 2025-12-17 21:18:39,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b25575bc060ce15b95c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:39,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b1e575bc060ce15b94d 2025-12-17 21:18:39,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1e575bc060ce15b94d HTTP/1.1" 502 150 2025-12-17 21:18:39,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b1e575bc060ce15b94d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:39,849 - lava_callback - INFO - Received callback for node 69407b10575bc060ce15b933 INFO: 172.21.0.1:47752 - "POST /node/69407b10575bc060ce15b933 HTTP/1.0" 202 Accepted 2025-12-17 21:18:39,856 - lava_callback - INFO - Starting async job submit for node 69407b10575bc060ce15b933 2025-12-17 21:18:39,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:18:39,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b10575bc060ce15b933 HTTP/1.1" 502 150 2025-12-17 21:18:39,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b10575bc060ce15b933'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:39,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b10575bc060ce15b933 2025-12-17 21:18:39,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b10575bc060ce15b933 HTTP/1.1" 502 150 2025-12-17 21:18:39,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b10575bc060ce15b933'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:40,415 - lava_callback - INFO - Received callback for node 69407b29575bc060ce15b965 INFO: 172.21.0.1:47766 - "POST /node/69407b29575bc060ce15b965 HTTP/1.0" 202 Accepted 2025-12-17 21:18:40,421 - lava_callback - INFO - Starting async job submit for node 69407b29575bc060ce15b965 2025-12-17 21:18:40,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:18:40,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b29575bc060ce15b965 HTTP/1.1" 502 150 2025-12-17 21:18:40,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b29575bc060ce15b965'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:40,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b29575bc060ce15b965 2025-12-17 21:18:40,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b29575bc060ce15b965 HTTP/1.1" 502 150 2025-12-17 21:18:40,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b29575bc060ce15b965'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:40,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b26575bc060ce15b95f 2025-12-17 21:18:40,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b26575bc060ce15b95f HTTP/1.1" 502 150 2025-12-17 21:18:40,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b26575bc060ce15b95f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:40,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b1f575bc060ce15b950 2025-12-17 21:18:40,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1f575bc060ce15b950 HTTP/1.1" 502 150 2025-12-17 21:18:40,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b1f575bc060ce15b950'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:41,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b21575bc060ce15b953 2025-12-17 21:18:41,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b21575bc060ce15b953 HTTP/1.1" 502 150 2025-12-17 21:18:41,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b21575bc060ce15b953'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:41,397 - lava_callback - INFO - Received callback for node 69407b2a575bc060ce15b968 INFO: 172.21.0.1:47780 - "POST /node/69407b2a575bc060ce15b968 HTTP/1.0" 202 Accepted 2025-12-17 21:18:41,407 - lava_callback - INFO - Starting async job submit for node 69407b2a575bc060ce15b968 2025-12-17 21:18:41,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:18:41,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2a575bc060ce15b968 HTTP/1.1" 502 150 2025-12-17 21:18:41,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2a575bc060ce15b968'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:41,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2a575bc060ce15b968 2025-12-17 21:18:41,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2a575bc060ce15b968 HTTP/1.1" 502 150 2025-12-17 21:18:41,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2a575bc060ce15b968'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:41,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b10575bc060ce15b933 2025-12-17 21:18:41,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b10575bc060ce15b933 HTTP/1.1" 502 150 2025-12-17 21:18:41,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b10575bc060ce15b933'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:42,160 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b22575bc060ce15b956 2025-12-17 21:18:42,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b22575bc060ce15b956 HTTP/1.1" 502 150 2025-12-17 21:18:42,162 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b22575bc060ce15b956'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:42,317 - lava_callback - INFO - Received callback for node 69407b2b575bc060ce15b96b INFO: 172.21.0.1:47790 - "POST /node/69407b2b575bc060ce15b96b HTTP/1.0" 202 Accepted 2025-12-17 21:18:42,327 - lava_callback - INFO - Starting async job submit for node 69407b2b575bc060ce15b96b 2025-12-17 21:18:42,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:18:42,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2b575bc060ce15b96b HTTP/1.1" 502 150 2025-12-17 21:18:42,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2b575bc060ce15b96b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:42,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2b575bc060ce15b96b 2025-12-17 21:18:42,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2b575bc060ce15b96b HTTP/1.1" 502 150 2025-12-17 21:18:42,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2b575bc060ce15b96b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:42,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b29575bc060ce15b965 2025-12-17 21:18:42,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b29575bc060ce15b965 HTTP/1.1" 502 150 2025-12-17 21:18:42,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b29575bc060ce15b965'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:42,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b23575bc060ce15b959 2025-12-17 21:18:42,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b23575bc060ce15b959 HTTP/1.1" 502 150 2025-12-17 21:18:42,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b23575bc060ce15b959'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:43,269 - lava_callback - INFO - Received callback for node 69407b2c575bc060ce15b96e INFO: 172.21.0.1:47792 - "POST /node/69407b2c575bc060ce15b96e HTTP/1.0" 202 Accepted 2025-12-17 21:18:43,276 - lava_callback - INFO - Starting async job submit for node 69407b2c575bc060ce15b96e 2025-12-17 21:18:43,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:18:43,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2a575bc060ce15b968 2025-12-17 21:18:43,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2a575bc060ce15b968 HTTP/1.1" 502 150 2025-12-17 21:18:43,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2a575bc060ce15b968'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:43,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b25575bc060ce15b95c 2025-12-17 21:18:43,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b25575bc060ce15b95c HTTP/1.1" 502 150 2025-12-17 21:18:43,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b25575bc060ce15b95c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:44,108 - lava_callback - INFO - Received callback for node 69407b2d575bc060ce15b971 INFO: 172.21.0.1:47800 - "POST /node/69407b2d575bc060ce15b971 HTTP/1.0" 202 Accepted 2025-12-17 21:18:44,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2b575bc060ce15b96b 2025-12-17 21:18:44,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2b575bc060ce15b96b HTTP/1.1" 502 150 2025-12-17 21:18:44,340 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2b575bc060ce15b96b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:44,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b26575bc060ce15b95f 2025-12-17 21:18:44,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b26575bc060ce15b95f HTTP/1.1" 502 150 2025-12-17 21:18:44,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b26575bc060ce15b95f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:44,848 - lava_callback - INFO - Received callback for node 69407b32575bc060ce15b978 INFO: 172.21.0.1:47814 - "POST /node/69407b32575bc060ce15b978 HTTP/1.0" 202 Accepted 2025-12-17 21:18:45,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b16575bc060ce15b942 2025-12-17 21:18:45,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b16575bc060ce15b942 HTTP/1.1" 502 150 2025-12-17 21:18:45,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b16575bc060ce15b942'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:45,600 - lava_callback - INFO - Received callback for node 69407b33575bc060ce15b97d INFO: 172.21.0.1:47822 - "POST /node/69407b33575bc060ce15b97d HTTP/1.0" 202 Accepted 2025-12-17 21:18:45,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b10575bc060ce15b933 2025-12-17 21:18:45,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b10575bc060ce15b933 HTTP/1.1" 502 150 2025-12-17 21:18:45,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b10575bc060ce15b933'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:46,318 - lava_callback - INFO - Received callback for node 69407b34575bc060ce15b982 INFO: 172.21.0.1:47830 - "POST /node/69407b34575bc060ce15b982 HTTP/1.0" 202 Accepted 2025-12-17 21:18:46,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b29575bc060ce15b965 2025-12-17 21:18:46,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b29575bc060ce15b965 HTTP/1.1" 502 150 2025-12-17 21:18:46,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b29575bc060ce15b965'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47836 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:18:47,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b1d575bc060ce15b94a 2025-12-17 21:18:47,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1d575bc060ce15b94a HTTP/1.1" 502 150 2025-12-17 21:18:47,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b1d575bc060ce15b94a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:47,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2a575bc060ce15b968 2025-12-17 21:18:47,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2a575bc060ce15b968 HTTP/1.1" 502 150 2025-12-17 21:18:47,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2a575bc060ce15b968'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:47,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b17575bc060ce15b945 HTTP/1.1" 502 150 2025-12-17 21:18:47,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b17575bc060ce15b945'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:47,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b17575bc060ce15b945 2025-12-17 21:18:47,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b17575bc060ce15b945 HTTP/1.1" 502 150 2025-12-17 21:18:47,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b17575bc060ce15b945'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:47,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b1e575bc060ce15b94d 2025-12-17 21:18:47,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1e575bc060ce15b94d HTTP/1.1" 502 150 2025-12-17 21:18:47,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b1e575bc060ce15b94d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:48,340 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2b575bc060ce15b96b 2025-12-17 21:18:48,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2b575bc060ce15b96b HTTP/1.1" 502 150 2025-12-17 21:18:48,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2b575bc060ce15b96b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:48,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b1f575bc060ce15b950 2025-12-17 21:18:48,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1f575bc060ce15b950 HTTP/1.1" 502 150 2025-12-17 21:18:48,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b1f575bc060ce15b950'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:49,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b21575bc060ce15b953 2025-12-17 21:18:49,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b21575bc060ce15b953 HTTP/1.1" 502 150 2025-12-17 21:18:49,388 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b21575bc060ce15b953'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:49,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b17575bc060ce15b945 2025-12-17 21:18:49,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b17575bc060ce15b945 HTTP/1.1" 502 150 2025-12-17 21:18:49,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b17575bc060ce15b945'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:50,162 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b22575bc060ce15b956 2025-12-17 21:18:50,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b23575bc060ce15b959 2025-12-17 21:18:50,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b23575bc060ce15b959 HTTP/1.1" 502 150 2025-12-17 21:18:50,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b23575bc060ce15b959'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:51,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b25575bc060ce15b95c 2025-12-17 21:18:51,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b25575bc060ce15b95c HTTP/1.1" 502 150 2025-12-17 21:18:51,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b25575bc060ce15b95c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:51,804 - lava_callback - INFO - Received callback for node 694078db575bc060ce15b58d INFO: 172.21.0.1:54366 - "POST /node/694078db575bc060ce15b58d HTTP/1.0" 202 Accepted 2025-12-17 21:18:52,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b26575bc060ce15b95f 2025-12-17 21:18:52,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b26575bc060ce15b95f HTTP/1.1" 502 150 2025-12-17 21:18:52,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b26575bc060ce15b95f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:53,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b17575bc060ce15b945 2025-12-17 21:18:53,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b17575bc060ce15b945 HTTP/1.1" 502 150 2025-12-17 21:18:53,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b17575bc060ce15b945'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:53,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b10575bc060ce15b933 2025-12-17 21:18:53,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b10575bc060ce15b933 HTTP/1.1" 502 150 2025-12-17 21:18:53,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b10575bc060ce15b933'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:54,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b29575bc060ce15b965 2025-12-17 21:18:54,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b29575bc060ce15b965 HTTP/1.1" 502 150 2025-12-17 21:18:54,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b29575bc060ce15b965'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:54,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b27575bc060ce15b962 HTTP/1.1" 502 150 2025-12-17 21:18:54,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b27575bc060ce15b962'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:54,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b27575bc060ce15b962 2025-12-17 21:18:54,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b27575bc060ce15b962 HTTP/1.1" 502 150 2025-12-17 21:18:54,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b27575bc060ce15b962'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:55,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2a575bc060ce15b968 2025-12-17 21:18:55,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2a575bc060ce15b968 HTTP/1.1" 502 150 2025-12-17 21:18:55,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2a575bc060ce15b968'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:56,342 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2b575bc060ce15b96b 2025-12-17 21:18:56,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2b575bc060ce15b96b HTTP/1.1" 502 150 2025-12-17 21:18:56,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2b575bc060ce15b96b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54372 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:18:56,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b27575bc060ce15b962 2025-12-17 21:18:56,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b27575bc060ce15b962 HTTP/1.1" 502 150 2025-12-17 21:18:56,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b27575bc060ce15b962'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:58,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2c575bc060ce15b96e HTTP/1.1" 502 150 2025-12-17 21:18:58,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2c575bc060ce15b96e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:18:58,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2c575bc060ce15b96e 2025-12-17 21:18:58,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2c575bc060ce15b96e HTTP/1.1" 502 150 2025-12-17 21:18:58,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2c575bc060ce15b96e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:00,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b27575bc060ce15b962 2025-12-17 21:19:00,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b27575bc060ce15b962 HTTP/1.1" 502 150 2025-12-17 21:19:00,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b27575bc060ce15b962'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:00,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2c575bc060ce15b96e 2025-12-17 21:19:00,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2c575bc060ce15b96e HTTP/1.1" 502 150 2025-12-17 21:19:00,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2c575bc060ce15b96e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48398 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:19:01,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b16575bc060ce15b942 2025-12-17 21:19:01,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b16575bc060ce15b942 HTTP/1.1" 502 150 2025-12-17 21:19:01,446 - lava_callback - ERROR - Error processing callback for node 69407b16575bc060ce15b942: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b16575bc060ce15b942 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b16575bc060ce15b942 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b16575bc060ce15b942 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:01,448 - lava_callback - INFO - Starting async job submit for node 69407b2d575bc060ce15b971 2025-12-17 21:19:01,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:01,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b17575bc060ce15b945 2025-12-17 21:19:01,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2d575bc060ce15b971 HTTP/1.1" 502 150 2025-12-17 21:19:01,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2d575bc060ce15b971'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:01,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2d575bc060ce15b971 2025-12-17 21:19:01,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b17575bc060ce15b945 HTTP/1.1" 502 150 2025-12-17 21:19:01,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b17575bc060ce15b945'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:01,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2d575bc060ce15b971 HTTP/1.1" 502 150 2025-12-17 21:19:01,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2d575bc060ce15b971'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:03,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b1d575bc060ce15b94a 2025-12-17 21:19:03,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1d575bc060ce15b94a HTTP/1.1" 502 150 2025-12-17 21:19:03,003 - lava_callback - ERROR - Error processing callback for node 69407b1d575bc060ce15b94a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b1d575bc060ce15b94a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b1d575bc060ce15b94a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b1d575bc060ce15b94a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:03,006 - lava_callback - INFO - Starting async job submit for node 69407b32575bc060ce15b978 2025-12-17 21:19:03,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:03,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b32575bc060ce15b978 HTTP/1.1" 502 150 2025-12-17 21:19:03,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b32575bc060ce15b978'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:03,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b32575bc060ce15b978 2025-12-17 21:19:03,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b32575bc060ce15b978 HTTP/1.1" 502 150 2025-12-17 21:19:03,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b32575bc060ce15b978'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:03,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2d575bc060ce15b971 2025-12-17 21:19:03,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2d575bc060ce15b971 HTTP/1.1" 502 150 2025-12-17 21:19:03,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2d575bc060ce15b971'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:03,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b1e575bc060ce15b94d 2025-12-17 21:19:03,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1e575bc060ce15b94d HTTP/1.1" 502 150 2025-12-17 21:19:03,814 - lava_callback - ERROR - Error processing callback for node 69407b1e575bc060ce15b94d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b1e575bc060ce15b94d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b1e575bc060ce15b94d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b1e575bc060ce15b94d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:03,816 - lava_callback - INFO - Starting async job submit for node 69407b33575bc060ce15b97d 2025-12-17 21:19:03,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:03,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b33575bc060ce15b97d HTTP/1.1" 502 150 2025-12-17 21:19:03,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b33575bc060ce15b97d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:03,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b33575bc060ce15b97d 2025-12-17 21:19:03,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b33575bc060ce15b97d HTTP/1.1" 502 150 2025-12-17 21:19:03,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b33575bc060ce15b97d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:04,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b1f575bc060ce15b950 2025-12-17 21:19:04,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b1f575bc060ce15b950 HTTP/1.1" 502 150 2025-12-17 21:19:04,623 - lava_callback - ERROR - Error processing callback for node 69407b1f575bc060ce15b950: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b1f575bc060ce15b950 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b1f575bc060ce15b950 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b1f575bc060ce15b950 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:04,625 - lava_callback - INFO - Starting async job submit for node 69407b34575bc060ce15b982 2025-12-17 21:19:04,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:04,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b34575bc060ce15b982 HTTP/1.1" 502 150 2025-12-17 21:19:04,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b34575bc060ce15b982'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:04,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b34575bc060ce15b982 2025-12-17 21:19:04,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b34575bc060ce15b982 HTTP/1.1" 502 150 2025-12-17 21:19:04,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b34575bc060ce15b982'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:04,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2c575bc060ce15b96e 2025-12-17 21:19:04,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2c575bc060ce15b96e HTTP/1.1" 502 150 2025-12-17 21:19:04,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2c575bc060ce15b96e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:05,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b32575bc060ce15b978 2025-12-17 21:19:05,388 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b21575bc060ce15b953 2025-12-17 21:19:05,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b21575bc060ce15b953 HTTP/1.1" 502 150 2025-12-17 21:19:05,390 - lava_callback - ERROR - Error processing callback for node 69407b21575bc060ce15b953: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b21575bc060ce15b953 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b21575bc060ce15b953 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b21575bc060ce15b953 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:05,392 - lava_callback - INFO - Starting async job submit for node 694078db575bc060ce15b58d 2025-12-17 21:19:05,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:05,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078db575bc060ce15b58d HTTP/1.1" 502 150 2025-12-17 21:19:05,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078db575bc060ce15b58d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:05,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078db575bc060ce15b58d 2025-12-17 21:19:05,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078db575bc060ce15b58d HTTP/1.1" 502 150 2025-12-17 21:19:05,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078db575bc060ce15b58d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:05,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b22575bc060ce15b956 HTTP/1.1" 502 150 2025-12-17 21:19:05,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b22575bc060ce15b956'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:05,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b33575bc060ce15b97d 2025-12-17 21:19:05,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b33575bc060ce15b97d HTTP/1.1" 502 150 2025-12-17 21:19:05,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b33575bc060ce15b97d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48404 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:19:06,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b34575bc060ce15b982 2025-12-17 21:19:06,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b34575bc060ce15b982 HTTP/1.1" 502 150 2025-12-17 21:19:06,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b34575bc060ce15b982'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:06,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b23575bc060ce15b959 2025-12-17 21:19:06,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b23575bc060ce15b959 HTTP/1.1" 502 150 2025-12-17 21:19:06,949 - lava_callback - ERROR - Error processing callback for node 69407b23575bc060ce15b959: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b23575bc060ce15b959 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b23575bc060ce15b959 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b23575bc060ce15b959 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:07,246 - lava_callback - INFO - Received callback for node 69407b38575bc060ce15b987 INFO: 172.21.0.1:48410 - "POST /node/69407b38575bc060ce15b987 HTTP/1.0" 202 Accepted 2025-12-17 21:19:07,252 - lava_callback - INFO - Starting async job submit for node 69407b38575bc060ce15b987 2025-12-17 21:19:07,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:07,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b38575bc060ce15b987 HTTP/1.1" 502 150 2025-12-17 21:19:07,259 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b38575bc060ce15b987'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:07,259 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b38575bc060ce15b987 2025-12-17 21:19:07,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b38575bc060ce15b987 HTTP/1.1" 502 150 2025-12-17 21:19:07,260 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b38575bc060ce15b987'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:07,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078db575bc060ce15b58d 2025-12-17 21:19:07,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078db575bc060ce15b58d HTTP/1.1" 502 150 2025-12-17 21:19:07,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078db575bc060ce15b58d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:07,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2d575bc060ce15b971 2025-12-17 21:19:07,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2d575bc060ce15b971 HTTP/1.1" 502 150 2025-12-17 21:19:07,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2d575bc060ce15b971'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:07,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b25575bc060ce15b95c 2025-12-17 21:19:07,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b25575bc060ce15b95c HTTP/1.1" 502 150 2025-12-17 21:19:07,698 - lava_callback - ERROR - Error processing callback for node 69407b25575bc060ce15b95c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b25575bc060ce15b95c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b25575bc060ce15b95c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b25575bc060ce15b95c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:07,934 - lava_callback - INFO - Received callback for node 69407b3a575bc060ce15b98c INFO: 172.21.0.1:48420 - "POST /node/69407b3a575bc060ce15b98c HTTP/1.0" 202 Accepted 2025-12-17 21:19:07,941 - lava_callback - INFO - Starting async job submit for node 69407b3a575bc060ce15b98c 2025-12-17 21:19:07,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:07,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b3a575bc060ce15b98c HTTP/1.1" 502 150 2025-12-17 21:19:07,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b3a575bc060ce15b98c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:07,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b3a575bc060ce15b98c 2025-12-17 21:19:07,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b3a575bc060ce15b98c HTTP/1.1" 502 150 2025-12-17 21:19:07,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b3a575bc060ce15b98c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:08,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b26575bc060ce15b95f 2025-12-17 21:19:08,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b26575bc060ce15b95f HTTP/1.1" 502 150 2025-12-17 21:19:08,528 - lava_callback - ERROR - Error processing callback for node 69407b26575bc060ce15b95f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b26575bc060ce15b95f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b26575bc060ce15b95f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b26575bc060ce15b95f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:08,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b27575bc060ce15b962 2025-12-17 21:19:08,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b27575bc060ce15b962 HTTP/1.1" 502 150 2025-12-17 21:19:08,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b27575bc060ce15b962'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:09,260 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b38575bc060ce15b987 2025-12-17 21:19:09,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b38575bc060ce15b987 HTTP/1.1" 502 150 2025-12-17 21:19:09,262 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b38575bc060ce15b987'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:09,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b33575bc060ce15b97d 2025-12-17 21:19:09,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b33575bc060ce15b97d HTTP/1.1" 502 150 2025-12-17 21:19:09,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b33575bc060ce15b97d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:09,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b10575bc060ce15b933 2025-12-17 21:19:09,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b10575bc060ce15b933 HTTP/1.1" 502 150 2025-12-17 21:19:09,907 - lava_callback - ERROR - Error processing callback for node 69407b10575bc060ce15b933: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b10575bc060ce15b933 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b10575bc060ce15b933 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b10575bc060ce15b933 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:09,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b3a575bc060ce15b98c 2025-12-17 21:19:09,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b3a575bc060ce15b98c HTTP/1.1" 502 150 2025-12-17 21:19:09,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b3a575bc060ce15b98c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:10,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b29575bc060ce15b965 2025-12-17 21:19:10,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b29575bc060ce15b965 HTTP/1.1" 502 150 2025-12-17 21:19:10,435 - lava_callback - ERROR - Error processing callback for node 69407b29575bc060ce15b965: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b29575bc060ce15b965 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b29575bc060ce15b965 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b29575bc060ce15b965 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:10,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b34575bc060ce15b982 2025-12-17 21:19:10,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b34575bc060ce15b982 HTTP/1.1" 502 150 2025-12-17 21:19:10,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b34575bc060ce15b982'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:11,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2a575bc060ce15b968 2025-12-17 21:19:11,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2a575bc060ce15b968 HTTP/1.1" 502 150 2025-12-17 21:19:11,423 - lava_callback - ERROR - Error processing callback for node 69407b2a575bc060ce15b968: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b2a575bc060ce15b968 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b2a575bc060ce15b968 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b2a575bc060ce15b968 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:11,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078db575bc060ce15b58d 2025-12-17 21:19:11,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078db575bc060ce15b58d HTTP/1.1" 502 150 2025-12-17 21:19:11,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078db575bc060ce15b58d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:11,732 - lava_callback - INFO - Received callback for node 69407b45575bc060ce15b992 INFO: 172.21.0.1:49096 - "POST /node/69407b45575bc060ce15b992 HTTP/1.0" 202 Accepted 2025-12-17 21:19:11,738 - lava_callback - INFO - Starting async job submit for node 69407b45575bc060ce15b992 2025-12-17 21:19:11,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:11,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b45575bc060ce15b992 HTTP/1.1" 502 150 2025-12-17 21:19:11,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b45575bc060ce15b992'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:11,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b45575bc060ce15b992 2025-12-17 21:19:11,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b45575bc060ce15b992 HTTP/1.1" 502 150 2025-12-17 21:19:11,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b45575bc060ce15b992'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:12,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2b575bc060ce15b96b 2025-12-17 21:19:12,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2b575bc060ce15b96b HTTP/1.1" 502 150 2025-12-17 21:19:12,348 - lava_callback - ERROR - Error processing callback for node 69407b2b575bc060ce15b96b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b2b575bc060ce15b96b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b2b575bc060ce15b96b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b2b575bc060ce15b96b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:12,536 - lava_callback - INFO - Received callback for node 69407b47575bc060ce15b999 INFO: 172.21.0.1:49110 - "POST /node/69407b47575bc060ce15b999 HTTP/1.0" 202 Accepted 2025-12-17 21:19:12,542 - lava_callback - INFO - Starting async job submit for node 69407b47575bc060ce15b999 2025-12-17 21:19:12,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:12,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b47575bc060ce15b999 HTTP/1.1" 502 150 2025-12-17 21:19:12,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b47575bc060ce15b999'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:12,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b47575bc060ce15b999 2025-12-17 21:19:12,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b47575bc060ce15b999 HTTP/1.1" 502 150 2025-12-17 21:19:12,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b47575bc060ce15b999'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:12,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2c575bc060ce15b96e 2025-12-17 21:19:12,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2c575bc060ce15b96e HTTP/1.1" 502 150 2025-12-17 21:19:12,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2c575bc060ce15b96e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:13,262 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b38575bc060ce15b987 2025-12-17 21:19:13,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b38575bc060ce15b987 HTTP/1.1" 502 150 2025-12-17 21:19:13,263 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b38575bc060ce15b987'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:13,275 - lava_callback - INFO - Received callback for node 69407b48575bc060ce15b99d INFO: 172.21.0.1:49124 - "POST /node/69407b48575bc060ce15b99d HTTP/1.0" 202 Accepted 2025-12-17 21:19:13,286 - lava_callback - INFO - Starting async job submit for node 69407b48575bc060ce15b99d 2025-12-17 21:19:13,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:13,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b48575bc060ce15b99d HTTP/1.1" 502 150 2025-12-17 21:19:13,294 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b48575bc060ce15b99d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:13,294 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b48575bc060ce15b99d 2025-12-17 21:19:13,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b48575bc060ce15b99d HTTP/1.1" 502 150 2025-12-17 21:19:13,295 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b48575bc060ce15b99d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:13,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b45575bc060ce15b992 2025-12-17 21:19:13,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b45575bc060ce15b992 HTTP/1.1" 502 150 2025-12-17 21:19:13,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b45575bc060ce15b992'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:13,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b3a575bc060ce15b98c 2025-12-17 21:19:13,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b3a575bc060ce15b98c HTTP/1.1" 502 150 2025-12-17 21:19:13,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b3a575bc060ce15b98c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:14,074 - lava_callback - INFO - Received callback for node 69407b50575bc060ce15b9de INFO: 172.21.0.1:49132 - "POST /node/69407b50575bc060ce15b9de HTTP/1.0" 202 Accepted 2025-12-17 21:19:14,084 - lava_callback - INFO - Starting async job submit for node 69407b50575bc060ce15b9de 2025-12-17 21:19:14,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:14,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b50575bc060ce15b9de HTTP/1.1" 502 150 2025-12-17 21:19:14,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b50575bc060ce15b9de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:14,115 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b50575bc060ce15b9de 2025-12-17 21:19:14,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b50575bc060ce15b9de HTTP/1.1" 502 150 2025-12-17 21:19:14,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b50575bc060ce15b9de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:14,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b47575bc060ce15b999 2025-12-17 21:19:14,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b47575bc060ce15b999 HTTP/1.1" 502 150 2025-12-17 21:19:14,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b47575bc060ce15b999'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:14,840 - lava_callback - INFO - Received callback for node 69407b51575bc060ce15b9ee INFO: 172.21.0.1:49142 - "POST /node/69407b51575bc060ce15b9ee HTTP/1.0" 202 Accepted 2025-12-17 21:19:14,850 - lava_callback - INFO - Starting async job submit for node 69407b51575bc060ce15b9ee 2025-12-17 21:19:14,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:14,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b51575bc060ce15b9ee HTTP/1.1" 502 150 2025-12-17 21:19:14,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b51575bc060ce15b9ee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:14,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b51575bc060ce15b9ee 2025-12-17 21:19:14,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b51575bc060ce15b9ee HTTP/1.1" 502 150 2025-12-17 21:19:14,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b51575bc060ce15b9ee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49148 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:19:15,296 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b48575bc060ce15b99d 2025-12-17 21:19:15,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b48575bc060ce15b99d HTTP/1.1" 502 150 2025-12-17 21:19:15,297 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b48575bc060ce15b99d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:15,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2d575bc060ce15b971 2025-12-17 21:19:15,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2d575bc060ce15b971 HTTP/1.1" 502 150 2025-12-17 21:19:15,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b2d575bc060ce15b971'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:16,116 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b50575bc060ce15b9de INFO: 172.21.0.1:49158 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:19:16,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b51575bc060ce15b9ee 2025-12-17 21:19:16,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b51575bc060ce15b9ee HTTP/1.1" 502 150 2025-12-17 21:19:16,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b51575bc060ce15b9ee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:17,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b17575bc060ce15b945 2025-12-17 21:19:17,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b17575bc060ce15b945 HTTP/1.1" 502 150 2025-12-17 21:19:17,479 - lava_callback - ERROR - Error processing callback for node 69407b17575bc060ce15b945: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b17575bc060ce15b945 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b17575bc060ce15b945 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b17575bc060ce15b945 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:17,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b45575bc060ce15b992 2025-12-17 21:19:17,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b45575bc060ce15b992 HTTP/1.1" 502 150 2025-12-17 21:19:17,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b45575bc060ce15b992'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:17,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b33575bc060ce15b97d 2025-12-17 21:19:17,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b33575bc060ce15b97d HTTP/1.1" 502 150 2025-12-17 21:19:17,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b33575bc060ce15b97d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:18,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b47575bc060ce15b999 2025-12-17 21:19:18,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b47575bc060ce15b999 HTTP/1.1" 502 150 2025-12-17 21:19:18,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b47575bc060ce15b999'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:18,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b34575bc060ce15b982 2025-12-17 21:19:18,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b34575bc060ce15b982 HTTP/1.1" 502 150 2025-12-17 21:19:18,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b34575bc060ce15b982'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:19,297 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b48575bc060ce15b99d 2025-12-17 21:19:19,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b48575bc060ce15b99d HTTP/1.1" 502 150 2025-12-17 21:19:19,298 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b48575bc060ce15b99d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:19,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078db575bc060ce15b58d 2025-12-17 21:19:19,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078db575bc060ce15b58d HTTP/1.1" 502 150 2025-12-17 21:19:19,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078db575bc060ce15b58d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:20,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b51575bc060ce15b9ee 2025-12-17 21:19:20,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b51575bc060ce15b9ee HTTP/1.1" 502 150 2025-12-17 21:19:20,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b51575bc060ce15b9ee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:21,264 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b38575bc060ce15b987 2025-12-17 21:19:21,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b38575bc060ce15b987 HTTP/1.1" 502 150 2025-12-17 21:19:21,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b38575bc060ce15b987'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:21,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b22575bc060ce15b956 2025-12-17 21:19:21,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b22575bc060ce15b956 HTTP/1.1" 502 150 2025-12-17 21:19:21,655 - lava_callback - ERROR - Error processing callback for node 69407b22575bc060ce15b956: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b22575bc060ce15b956 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b22575bc060ce15b956 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b22575bc060ce15b956 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:21,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b3a575bc060ce15b98c 2025-12-17 21:19:21,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b3a575bc060ce15b98c HTTP/1.1" 502 150 2025-12-17 21:19:21,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b3a575bc060ce15b98c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:24,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b27575bc060ce15b962 2025-12-17 21:19:24,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b27575bc060ce15b962 HTTP/1.1" 502 150 2025-12-17 21:19:24,648 - lava_callback - ERROR - Error processing callback for node 69407b27575bc060ce15b962: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b27575bc060ce15b962 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b27575bc060ce15b962 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b27575bc060ce15b962 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:25,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b45575bc060ce15b992 2025-12-17 21:19:25,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b45575bc060ce15b992 HTTP/1.1" 502 150 2025-12-17 21:19:25,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b45575bc060ce15b992'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34500 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:19:26,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b47575bc060ce15b999 2025-12-17 21:19:26,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b47575bc060ce15b999 HTTP/1.1" 502 150 2025-12-17 21:19:26,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b47575bc060ce15b999'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:27,298 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b48575bc060ce15b99d 2025-12-17 21:19:27,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b48575bc060ce15b99d HTTP/1.1" 502 150 2025-12-17 21:19:27,300 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b48575bc060ce15b99d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:28,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2c575bc060ce15b96e 2025-12-17 21:19:28,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2c575bc060ce15b96e HTTP/1.1" 502 150 2025-12-17 21:19:28,743 - lava_callback - ERROR - Error processing callback for node 69407b2c575bc060ce15b96e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b2c575bc060ce15b96e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b2c575bc060ce15b96e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b2c575bc060ce15b96e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:28,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b51575bc060ce15b9ee 2025-12-17 21:19:28,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b51575bc060ce15b9ee HTTP/1.1" 502 150 2025-12-17 21:19:28,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b51575bc060ce15b9ee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:31,288 - lava_callback - INFO - Received callback for node 69407b65575bc060ce15ba29 INFO: 172.21.0.1:55770 - "POST /node/69407b65575bc060ce15ba29 HTTP/1.0" 202 Accepted 2025-12-17 21:19:31,294 - lava_callback - INFO - Starting async job submit for node 69407b65575bc060ce15ba29 2025-12-17 21:19:31,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:31,457 - lava_callback - INFO - Received callback for node 69407b53575bc060ce15b9f1 INFO: 172.21.0.1:55780 - "POST /node/69407b53575bc060ce15b9f1 HTTP/1.0" 202 Accepted 2025-12-17 21:19:31,463 - lava_callback - INFO - Starting async job submit for node 69407b53575bc060ce15b9f1 2025-12-17 21:19:31,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:31,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b53575bc060ce15b9f1 HTTP/1.1" 502 150 2025-12-17 21:19:31,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b53575bc060ce15b9f1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:31,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b53575bc060ce15b9f1 2025-12-17 21:19:31,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b53575bc060ce15b9f1 HTTP/1.1" 502 150 2025-12-17 21:19:31,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b53575bc060ce15b9f1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:31,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b2d575bc060ce15b971 2025-12-17 21:19:31,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b2d575bc060ce15b971 HTTP/1.1" 502 150 2025-12-17 21:19:31,483 - lava_callback - ERROR - Error processing callback for node 69407b2d575bc060ce15b971: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b2d575bc060ce15b971 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b2d575bc060ce15b971 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b2d575bc060ce15b971 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:31,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b50575bc060ce15b9de HTTP/1.1" 502 150 2025-12-17 21:19:31,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b50575bc060ce15b9de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:32,201 - lava_callback - INFO - Received callback for node 69407b54575bc060ce15b9f4 INFO: 172.21.0.1:55788 - "POST /node/69407b54575bc060ce15b9f4 HTTP/1.0" 202 Accepted 2025-12-17 21:19:32,207 - lava_callback - INFO - Starting async job submit for node 69407b54575bc060ce15b9f4 2025-12-17 21:19:32,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:32,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b54575bc060ce15b9f4 HTTP/1.1" 502 150 2025-12-17 21:19:32,214 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b54575bc060ce15b9f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:32,214 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b54575bc060ce15b9f4 2025-12-17 21:19:32,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b54575bc060ce15b9f4 HTTP/1.1" 502 150 2025-12-17 21:19:32,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b54575bc060ce15b9f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:32,950 - lava_callback - INFO - Received callback for node 69407b55575bc060ce15b9f7 INFO: 172.21.0.1:55796 - "POST /node/69407b55575bc060ce15b9f7 HTTP/1.0" 202 Accepted 2025-12-17 21:19:32,956 - lava_callback - INFO - Starting async job submit for node 69407b55575bc060ce15b9f7 2025-12-17 21:19:32,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:32,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b55575bc060ce15b9f7 HTTP/1.1" 502 150 2025-12-17 21:19:32,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b55575bc060ce15b9f7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:32,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b55575bc060ce15b9f7 2025-12-17 21:19:32,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b55575bc060ce15b9f7 HTTP/1.1" 502 150 2025-12-17 21:19:32,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b55575bc060ce15b9f7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:33,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b53575bc060ce15b9f1 2025-12-17 21:19:33,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b53575bc060ce15b9f1 HTTP/1.1" 502 150 2025-12-17 21:19:33,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b53575bc060ce15b9f1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:33,677 - lava_callback - INFO - Received callback for node 69407b56575bc060ce15b9fa INFO: 172.21.0.1:55810 - "POST /node/69407b56575bc060ce15b9fa HTTP/1.0" 202 Accepted 2025-12-17 21:19:33,682 - lava_callback - INFO - Starting async job submit for node 69407b56575bc060ce15b9fa 2025-12-17 21:19:33,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:33,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b56575bc060ce15b9fa HTTP/1.1" 502 150 2025-12-17 21:19:33,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b56575bc060ce15b9fa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:33,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b56575bc060ce15b9fa 2025-12-17 21:19:33,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b56575bc060ce15b9fa HTTP/1.1" 502 150 2025-12-17 21:19:33,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b56575bc060ce15b9fa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:33,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b33575bc060ce15b97d 2025-12-17 21:19:33,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b33575bc060ce15b97d HTTP/1.1" 502 150 2025-12-17 21:19:33,830 - lava_callback - ERROR - Error processing callback for node 69407b33575bc060ce15b97d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b33575bc060ce15b97d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b33575bc060ce15b97d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b33575bc060ce15b97d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:34,215 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b54575bc060ce15b9f4 2025-12-17 21:19:34,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b54575bc060ce15b9f4 HTTP/1.1" 502 150 2025-12-17 21:19:34,221 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b54575bc060ce15b9f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:34,423 - lava_callback - INFO - Received callback for node 69407b58575bc060ce15b9fd INFO: 172.21.0.1:55814 - "POST /node/69407b58575bc060ce15b9fd HTTP/1.0" 202 Accepted 2025-12-17 21:19:34,429 - lava_callback - INFO - Starting async job submit for node 69407b58575bc060ce15b9fd 2025-12-17 21:19:34,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:34,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b58575bc060ce15b9fd HTTP/1.1" 502 150 2025-12-17 21:19:34,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b58575bc060ce15b9fd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:34,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b58575bc060ce15b9fd 2025-12-17 21:19:34,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b58575bc060ce15b9fd HTTP/1.1" 502 150 2025-12-17 21:19:34,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b58575bc060ce15b9fd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:34,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b34575bc060ce15b982 2025-12-17 21:19:34,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b34575bc060ce15b982 HTTP/1.1" 502 150 2025-12-17 21:19:34,638 - lava_callback - ERROR - Error processing callback for node 69407b34575bc060ce15b982: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b34575bc060ce15b982 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b34575bc060ce15b982 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b34575bc060ce15b982 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:34,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b55575bc060ce15b9f7 2025-12-17 21:19:34,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b55575bc060ce15b9f7 HTTP/1.1" 502 150 2025-12-17 21:19:34,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b55575bc060ce15b9f7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:35,206 - lava_callback - INFO - Received callback for node 69407b59575bc060ce15ba01 INFO: 172.21.0.1:55820 - "POST /node/69407b59575bc060ce15ba01 HTTP/1.0" 202 Accepted 2025-12-17 21:19:35,212 - lava_callback - INFO - Starting async job submit for node 69407b59575bc060ce15ba01 2025-12-17 21:19:35,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:35,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078db575bc060ce15b58d 2025-12-17 21:19:35,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078db575bc060ce15b58d HTTP/1.1" 502 150 2025-12-17 21:19:35,465 - lava_callback - ERROR - Error processing callback for node 694078db575bc060ce15b58d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078db575bc060ce15b58d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078db575bc060ce15b58d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078db575bc060ce15b58d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:35,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b50575bc060ce15b9de 2025-12-17 21:19:35,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b50575bc060ce15b9de HTTP/1.1" 502 150 2025-12-17 21:19:35,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b50575bc060ce15b9de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:35,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b56575bc060ce15b9fa 2025-12-17 21:19:35,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b56575bc060ce15b9fa HTTP/1.1" 502 150 2025-12-17 21:19:35,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b56575bc060ce15b9fa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:35,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b32575bc060ce15b978 HTTP/1.1" 502 150 2025-12-17 21:19:35,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b32575bc060ce15b978'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:35,965 - lava_callback - INFO - Received callback for node 69407b5a575bc060ce15ba06 INFO: 172.21.0.1:55822 - "POST /node/69407b5a575bc060ce15ba06 HTTP/1.0" 202 Accepted 2025-12-17 21:19:35,971 - lava_callback - INFO - Starting async job submit for node 69407b5a575bc060ce15ba06 2025-12-17 21:19:35,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:35,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5a575bc060ce15ba06 HTTP/1.1" 502 150 2025-12-17 21:19:35,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5a575bc060ce15ba06'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:35,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5a575bc060ce15ba06 2025-12-17 21:19:35,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5a575bc060ce15ba06 HTTP/1.1" 502 150 2025-12-17 21:19:35,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5a575bc060ce15ba06'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:36,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b58575bc060ce15b9fd 2025-12-17 21:19:36,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b58575bc060ce15b9fd HTTP/1.1" 502 150 2025-12-17 21:19:36,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b58575bc060ce15b9fd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55836 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:19:36,713 - lava_callback - INFO - Received callback for node 69407b5b575bc060ce15ba0b INFO: 172.21.0.1:55846 - "POST /node/69407b5b575bc060ce15ba0b HTTP/1.0" 202 Accepted 2025-12-17 21:19:37,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b38575bc060ce15b987 2025-12-17 21:19:37,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b38575bc060ce15b987 HTTP/1.1" 502 150 2025-12-17 21:19:37,266 - lava_callback - ERROR - Error processing callback for node 69407b38575bc060ce15b987: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b38575bc060ce15b987 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b38575bc060ce15b987 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b38575bc060ce15b987 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:37,268 - lava_callback - INFO - Starting async job submit for node 69407b5b575bc060ce15ba0b 2025-12-17 21:19:37,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:37,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5b575bc060ce15ba0b HTTP/1.1" 502 150 2025-12-17 21:19:37,281 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5b575bc060ce15ba0b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:37,281 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5b575bc060ce15ba0b 2025-12-17 21:19:37,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5b575bc060ce15ba0b HTTP/1.1" 502 150 2025-12-17 21:19:37,282 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5b575bc060ce15ba0b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:37,463 - lava_callback - INFO - Received callback for node 69407b5d575bc060ce15ba0f INFO: 172.21.0.1:55860 - "POST /node/69407b5d575bc060ce15ba0f HTTP/1.0" 202 Accepted 2025-12-17 21:19:37,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b53575bc060ce15b9f1 2025-12-17 21:19:37,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b53575bc060ce15b9f1 HTTP/1.1" 502 150 2025-12-17 21:19:37,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b53575bc060ce15b9f1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:37,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b3a575bc060ce15b98c 2025-12-17 21:19:37,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b3a575bc060ce15b98c HTTP/1.1" 502 150 2025-12-17 21:19:37,955 - lava_callback - ERROR - Error processing callback for node 69407b3a575bc060ce15b98c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b3a575bc060ce15b98c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b3a575bc060ce15b98c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b3a575bc060ce15b98c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:37,957 - lava_callback - INFO - Starting async job submit for node 69407b5d575bc060ce15ba0f 2025-12-17 21:19:37,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:37,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5d575bc060ce15ba0f HTTP/1.1" 502 150 2025-12-17 21:19:37,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5d575bc060ce15ba0f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:37,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5d575bc060ce15ba0f 2025-12-17 21:19:37,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5d575bc060ce15ba0f HTTP/1.1" 502 150 2025-12-17 21:19:37,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5d575bc060ce15ba0f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:37,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5a575bc060ce15ba06 2025-12-17 21:19:37,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5a575bc060ce15ba06 HTTP/1.1" 502 150 2025-12-17 21:19:37,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5a575bc060ce15ba06'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:38,221 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b54575bc060ce15b9f4 2025-12-17 21:19:38,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b54575bc060ce15b9f4 HTTP/1.1" 502 150 2025-12-17 21:19:38,222 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b54575bc060ce15b9f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:38,247 - lava_callback - INFO - Received callback for node 69407b5e575bc060ce15ba12 INFO: 172.21.0.1:55868 - "POST /node/69407b5e575bc060ce15ba12 HTTP/1.0" 202 Accepted 2025-12-17 21:19:38,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b55575bc060ce15b9f7 2025-12-17 21:19:38,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b55575bc060ce15b9f7 HTTP/1.1" 502 150 2025-12-17 21:19:38,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b55575bc060ce15b9f7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:38,994 - lava_callback - INFO - Received callback for node 69407b5f575bc060ce15ba1a INFO: 172.21.0.1:55872 - "POST /node/69407b5f575bc060ce15ba1a HTTP/1.0" 202 Accepted 2025-12-17 21:19:39,282 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5b575bc060ce15ba0b 2025-12-17 21:19:39,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5b575bc060ce15ba0b HTTP/1.1" 502 150 2025-12-17 21:19:39,283 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5b575bc060ce15ba0b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:39,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b56575bc060ce15b9fa 2025-12-17 21:19:39,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b56575bc060ce15b9fa HTTP/1.1" 502 150 2025-12-17 21:19:39,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b56575bc060ce15b9fa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:39,749 - lava_callback - INFO - Received callback for node 69407b61575bc060ce15ba1d INFO: 172.21.0.1:51356 - "POST /node/69407b61575bc060ce15ba1d HTTP/1.0" 202 Accepted 2025-12-17 21:19:39,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b32575bc060ce15b978 2025-12-17 21:19:39,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b32575bc060ce15b978 HTTP/1.1" 502 150 2025-12-17 21:19:39,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b32575bc060ce15b978'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:39,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5d575bc060ce15ba0f 2025-12-17 21:19:39,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5d575bc060ce15ba0f HTTP/1.1" 502 150 2025-12-17 21:19:39,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5d575bc060ce15ba0f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:40,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b58575bc060ce15b9fd 2025-12-17 21:19:40,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b58575bc060ce15b9fd HTTP/1.1" 502 150 2025-12-17 21:19:40,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b58575bc060ce15b9fd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:40,483 - lava_callback - INFO - Received callback for node 69407b62575bc060ce15ba20 INFO: 172.21.0.1:51366 - "POST /node/69407b62575bc060ce15ba20 HTTP/1.0" 202 Accepted 2025-12-17 21:19:41,230 - lava_callback - INFO - Received callback for node 69407b63575bc060ce15ba23 INFO: 172.21.0.1:51372 - "POST /node/69407b63575bc060ce15ba23 HTTP/1.0" 202 Accepted 2025-12-17 21:19:41,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b45575bc060ce15b992 2025-12-17 21:19:41,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b45575bc060ce15b992 HTTP/1.1" 502 150 2025-12-17 21:19:41,757 - lava_callback - ERROR - Error processing callback for node 69407b45575bc060ce15b992: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b45575bc060ce15b992 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b45575bc060ce15b992 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b45575bc060ce15b992 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:41,759 - lava_callback - INFO - Starting async job submit for node 69407b5e575bc060ce15ba12 2025-12-17 21:19:41,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:41,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5e575bc060ce15ba12 HTTP/1.1" 502 150 2025-12-17 21:19:41,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5e575bc060ce15ba12'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:41,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5e575bc060ce15ba12 2025-12-17 21:19:41,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5e575bc060ce15ba12 HTTP/1.1" 502 150 2025-12-17 21:19:41,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5e575bc060ce15ba12'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:41,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5a575bc060ce15ba06 2025-12-17 21:19:41,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5a575bc060ce15ba06 HTTP/1.1" 502 150 2025-12-17 21:19:41,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5a575bc060ce15ba06'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:41,998 - lava_callback - INFO - Received callback for node 69407b64575bc060ce15ba26 INFO: 172.21.0.1:51388 - "POST /node/69407b64575bc060ce15ba26 HTTP/1.0" 202 Accepted 2025-12-17 21:19:42,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b47575bc060ce15b999 2025-12-17 21:19:42,742 - lava_callback - INFO - Received callback for node 69407b66575bc060ce15ba2c INFO: 172.21.0.1:51402 - "POST /node/69407b66575bc060ce15ba2c HTTP/1.0" 202 Accepted 2025-12-17 21:19:43,283 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5b575bc060ce15ba0b 2025-12-17 21:19:43,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5b575bc060ce15ba0b HTTP/1.1" 502 150 2025-12-17 21:19:43,285 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5b575bc060ce15ba0b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:43,300 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b48575bc060ce15b99d 2025-12-17 21:19:43,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b48575bc060ce15b99d HTTP/1.1" 502 150 2025-12-17 21:19:43,301 - lava_callback - ERROR - Error processing callback for node 69407b48575bc060ce15b99d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b48575bc060ce15b99d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b48575bc060ce15b99d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b48575bc060ce15b99d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:43,303 - lava_callback - INFO - Starting async job submit for node 69407b5f575bc060ce15ba1a 2025-12-17 21:19:43,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:43,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5f575bc060ce15ba1a HTTP/1.1" 502 150 2025-12-17 21:19:43,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5f575bc060ce15ba1a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:43,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5f575bc060ce15ba1a 2025-12-17 21:19:43,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5f575bc060ce15ba1a HTTP/1.1" 502 150 2025-12-17 21:19:43,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5f575bc060ce15ba1a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:43,501 - lava_callback - INFO - Received callback for node 69407b66575bc060ce15ba2f INFO: 172.21.0.1:51412 - "POST /node/69407b66575bc060ce15ba2f HTTP/1.0" 202 Accepted 2025-12-17 21:19:43,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b50575bc060ce15b9de 2025-12-17 21:19:43,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b50575bc060ce15b9de HTTP/1.1" 502 150 2025-12-17 21:19:43,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b50575bc060ce15b9de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:43,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5e575bc060ce15ba12 2025-12-17 21:19:43,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5e575bc060ce15ba12 HTTP/1.1" 502 150 2025-12-17 21:19:43,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5e575bc060ce15ba12'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:43,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5d575bc060ce15ba0f 2025-12-17 21:19:43,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5d575bc060ce15ba0f HTTP/1.1" 502 150 2025-12-17 21:19:43,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5d575bc060ce15ba0f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:44,258 - lava_callback - INFO - Received callback for node 69407b66575bc060ce15ba32 INFO: 172.21.0.1:51420 - "POST /node/69407b66575bc060ce15ba32 HTTP/1.0" 202 Accepted 2025-12-17 21:19:44,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b51575bc060ce15b9ee 2025-12-17 21:19:44,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b51575bc060ce15b9ee HTTP/1.1" 502 150 2025-12-17 21:19:44,865 - lava_callback - ERROR - Error processing callback for node 69407b51575bc060ce15b9ee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b51575bc060ce15b9ee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b51575bc060ce15b9ee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b51575bc060ce15b9ee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:44,867 - lava_callback - INFO - Starting async job submit for node 69407b61575bc060ce15ba1d 2025-12-17 21:19:44,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:44,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b61575bc060ce15ba1d HTTP/1.1" 502 150 2025-12-17 21:19:44,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b61575bc060ce15ba1d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:44,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b61575bc060ce15ba1d 2025-12-17 21:19:44,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b61575bc060ce15ba1d HTTP/1.1" 502 150 2025-12-17 21:19:44,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b61575bc060ce15ba1d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:45,023 - lava_callback - INFO - Received callback for node 69407b67575bc060ce15ba35 INFO: 172.21.0.1:51422 - "POST /node/69407b67575bc060ce15ba35 HTTP/1.0" 202 Accepted 2025-12-17 21:19:45,317 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5f575bc060ce15ba1a 2025-12-17 21:19:45,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5f575bc060ce15ba1a HTTP/1.1" 502 150 2025-12-17 21:19:45,318 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5f575bc060ce15ba1a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:45,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b53575bc060ce15b9f1 2025-12-17 21:19:45,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b53575bc060ce15b9f1 HTTP/1.1" 502 150 2025-12-17 21:19:45,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b53575bc060ce15b9f1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:45,773 - lava_callback - INFO - Received callback for node 69407b67575bc060ce15ba38 INFO: 172.21.0.1:51430 - "POST /node/69407b67575bc060ce15ba38 HTTP/1.0" 202 Accepted 2025-12-17 21:19:46,223 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b54575bc060ce15b9f4 2025-12-17 21:19:46,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b54575bc060ce15b9f4 HTTP/1.1" 502 150 2025-12-17 21:19:46,224 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b54575bc060ce15b9f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51442 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:19:46,517 - lava_callback - INFO - Received callback for node 69407b68575bc060ce15ba3b INFO: 172.21.0.1:51444 - "POST /node/69407b68575bc060ce15ba3b HTTP/1.0" 202 Accepted 2025-12-17 21:19:46,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b65575bc060ce15ba29 HTTP/1.1" 502 150 2025-12-17 21:19:46,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b65575bc060ce15ba29'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:46,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b65575bc060ce15ba29 2025-12-17 21:19:46,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b65575bc060ce15ba29 HTTP/1.1" 502 150 2025-12-17 21:19:46,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b65575bc060ce15ba29'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:46,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b61575bc060ce15ba1d 2025-12-17 21:19:46,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b61575bc060ce15ba1d HTTP/1.1" 502 150 2025-12-17 21:19:46,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b61575bc060ce15ba1d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:46,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b55575bc060ce15b9f7 2025-12-17 21:19:46,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b55575bc060ce15b9f7 HTTP/1.1" 502 150 2025-12-17 21:19:46,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b55575bc060ce15b9f7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:47,383 - lava_callback - INFO - Received callback for node 69407b6a575bc060ce15ba3e INFO: 172.21.0.1:51454 - "POST /node/69407b6a575bc060ce15ba3e HTTP/1.0" 202 Accepted 2025-12-17 21:19:47,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b56575bc060ce15b9fa 2025-12-17 21:19:47,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b56575bc060ce15b9fa HTTP/1.1" 502 150 2025-12-17 21:19:47,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b56575bc060ce15b9fa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:47,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5e575bc060ce15ba12 2025-12-17 21:19:47,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5e575bc060ce15ba12 HTTP/1.1" 502 150 2025-12-17 21:19:47,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5e575bc060ce15ba12'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:47,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b32575bc060ce15b978 2025-12-17 21:19:47,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b32575bc060ce15b978 HTTP/1.1" 502 150 2025-12-17 21:19:47,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b32575bc060ce15b978'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:48,336 - lava_callback - INFO - Received callback for node 69407b6f575bc060ce15ba48 INFO: 172.21.0.1:51458 - "POST /node/69407b6f575bc060ce15ba48 HTTP/1.0" 202 Accepted 2025-12-17 21:19:48,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b58575bc060ce15b9fd 2025-12-17 21:19:48,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b58575bc060ce15b9fd HTTP/1.1" 502 150 2025-12-17 21:19:48,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b58575bc060ce15b9fd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:48,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b65575bc060ce15ba29 2025-12-17 21:19:48,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b65575bc060ce15ba29 HTTP/1.1" 502 150 2025-12-17 21:19:48,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b65575bc060ce15ba29'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:49,072 - lava_callback - INFO - Received callback for node 69407b70575bc060ce15ba53 INFO: 172.21.0.1:51462 - "POST /node/69407b70575bc060ce15ba53 HTTP/1.0" 202 Accepted 2025-12-17 21:19:49,319 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5f575bc060ce15ba1a 2025-12-17 21:19:49,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5f575bc060ce15ba1a HTTP/1.1" 502 150 2025-12-17 21:19:49,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5f575bc060ce15ba1a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:49,805 - lava_callback - INFO - Received callback for node 69407b73575bc060ce15ba5f INFO: 172.21.0.1:56908 - "POST /node/69407b73575bc060ce15ba5f HTTP/1.0" 202 Accepted 2025-12-17 21:19:49,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5a575bc060ce15ba06 2025-12-17 21:19:49,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5a575bc060ce15ba06 HTTP/1.1" 502 150 2025-12-17 21:19:49,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5a575bc060ce15ba06'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:50,423 - lava_callback - INFO - Received callback for node 69407b71575bc060ce15ba59 INFO: 172.21.0.1:56914 - "POST /node/69407b71575bc060ce15ba59 HTTP/1.0" 202 Accepted 2025-12-17 21:19:50,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b59575bc060ce15ba01 HTTP/1.1" 502 150 2025-12-17 21:19:50,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b59575bc060ce15ba01'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:50,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b59575bc060ce15ba01 2025-12-17 21:19:50,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b59575bc060ce15ba01 HTTP/1.1" 502 150 2025-12-17 21:19:50,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b59575bc060ce15ba01'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:50,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b61575bc060ce15ba1d 2025-12-17 21:19:50,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b61575bc060ce15ba1d HTTP/1.1" 502 150 2025-12-17 21:19:50,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b61575bc060ce15ba1d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:51,048 - lava_callback - INFO - Received callback for node 69407b74575bc060ce15ba62 INFO: 172.21.0.1:56930 - "POST /node/69407b74575bc060ce15ba62 HTTP/1.0" 202 Accepted 2025-12-17 21:19:51,285 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5b575bc060ce15ba0b 2025-12-17 21:19:51,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5b575bc060ce15ba0b HTTP/1.1" 502 150 2025-12-17 21:19:51,286 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5b575bc060ce15ba0b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:51,684 - lava_callback - INFO - Received callback for node 69407b75575bc060ce15ba65 INFO: 172.21.0.1:56934 - "POST /node/69407b75575bc060ce15ba65 HTTP/1.0" 202 Accepted 2025-12-17 21:19:51,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5d575bc060ce15ba0f 2025-12-17 21:19:51,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5d575bc060ce15ba0f HTTP/1.1" 502 150 2025-12-17 21:19:51,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5d575bc060ce15ba0f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:52,478 - lava_callback - INFO - Received callback for node 69407b76575bc060ce15ba68 INFO: 172.21.0.1:56936 - "POST /node/69407b76575bc060ce15ba68 HTTP/1.0" 202 Accepted 2025-12-17 21:19:52,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b65575bc060ce15ba29 2025-12-17 21:19:52,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b65575bc060ce15ba29 HTTP/1.1" 502 150 2025-12-17 21:19:52,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b65575bc060ce15ba29'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:52,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b59575bc060ce15ba01 2025-12-17 21:19:52,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b59575bc060ce15ba01 HTTP/1.1" 502 150 2025-12-17 21:19:52,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b59575bc060ce15ba01'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:53,238 - lava_callback - INFO - Received callback for node 69407b78575bc060ce15ba6b INFO: 172.21.0.1:56944 - "POST /node/69407b78575bc060ce15ba6b HTTP/1.0" 202 Accepted 2025-12-17 21:19:55,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5e575bc060ce15ba12 2025-12-17 21:19:55,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5e575bc060ce15ba12 HTTP/1.1" 502 150 2025-12-17 21:19:55,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5e575bc060ce15ba12'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56960 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:19:56,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b59575bc060ce15ba01 2025-12-17 21:19:56,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b59575bc060ce15ba01 HTTP/1.1" 502 150 2025-12-17 21:19:56,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b59575bc060ce15ba01'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:57,320 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5f575bc060ce15ba1a 2025-12-17 21:19:57,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b47575bc060ce15b999 HTTP/1.1" 502 150 2025-12-17 21:19:57,872 - lava_callback - ERROR - Error processing callback for node 69407b47575bc060ce15b999: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b47575bc060ce15b999 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b47575bc060ce15b999 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b47575bc060ce15b999 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:57,874 - lava_callback - INFO - Starting async job submit for node 69407b62575bc060ce15ba20 2025-12-17 21:19:57,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:57,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b62575bc060ce15ba20 HTTP/1.1" 502 150 2025-12-17 21:19:57,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b62575bc060ce15ba20'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:57,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b62575bc060ce15ba20 2025-12-17 21:19:57,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b62575bc060ce15ba20 HTTP/1.1" 502 150 2025-12-17 21:19:57,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b62575bc060ce15ba20'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:58,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b61575bc060ce15ba1d 2025-12-17 21:19:58,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b61575bc060ce15ba1d HTTP/1.1" 502 150 2025-12-17 21:19:58,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b61575bc060ce15ba1d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:59,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b50575bc060ce15b9de 2025-12-17 21:19:59,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b50575bc060ce15b9de HTTP/1.1" 502 150 2025-12-17 21:19:59,509 - lava_callback - ERROR - Error processing callback for node 69407b50575bc060ce15b9de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b50575bc060ce15b9de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b50575bc060ce15b9de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b50575bc060ce15b9de (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:19:59,511 - lava_callback - INFO - Starting async job submit for node 69407b63575bc060ce15ba23 2025-12-17 21:19:59,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:19:59,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b63575bc060ce15ba23 HTTP/1.1" 502 150 2025-12-17 21:19:59,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b63575bc060ce15ba23'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:59,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b63575bc060ce15ba23 2025-12-17 21:19:59,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b63575bc060ce15ba23 HTTP/1.1" 502 150 2025-12-17 21:19:59,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b63575bc060ce15ba23'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:19:59,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b62575bc060ce15ba20 2025-12-17 21:19:59,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b62575bc060ce15ba20 HTTP/1.1" 502 150 2025-12-17 21:19:59,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b62575bc060ce15ba20'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:00,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b65575bc060ce15ba29 2025-12-17 21:20:00,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b65575bc060ce15ba29 HTTP/1.1" 502 150 2025-12-17 21:20:00,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b65575bc060ce15ba29'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38236 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:20:01,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b53575bc060ce15b9f1 2025-12-17 21:20:01,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b53575bc060ce15b9f1 HTTP/1.1" 502 150 2025-12-17 21:20:01,477 - lava_callback - ERROR - Error processing callback for node 69407b53575bc060ce15b9f1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b53575bc060ce15b9f1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b53575bc060ce15b9f1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b53575bc060ce15b9f1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:01,479 - lava_callback - INFO - Starting async job submit for node 69407b64575bc060ce15ba26 2025-12-17 21:20:01,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:01,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b64575bc060ce15ba26 HTTP/1.1" 502 150 2025-12-17 21:20:01,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b64575bc060ce15ba26'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:01,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b64575bc060ce15ba26 2025-12-17 21:20:01,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b64575bc060ce15ba26 HTTP/1.1" 502 150 2025-12-17 21:20:01,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b64575bc060ce15ba26'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:01,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b63575bc060ce15ba23 2025-12-17 21:20:01,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b63575bc060ce15ba23 HTTP/1.1" 502 150 2025-12-17 21:20:01,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b63575bc060ce15ba23'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:02,224 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b54575bc060ce15b9f4 2025-12-17 21:20:02,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b54575bc060ce15b9f4 HTTP/1.1" 502 150 2025-12-17 21:20:02,225 - lava_callback - ERROR - Error processing callback for node 69407b54575bc060ce15b9f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b54575bc060ce15b9f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b54575bc060ce15b9f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b54575bc060ce15b9f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:02,227 - lava_callback - INFO - Starting async job submit for node 69407b66575bc060ce15ba2c 2025-12-17 21:20:02,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:02,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba2c HTTP/1.1" 502 150 2025-12-17 21:20:02,233 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b66575bc060ce15ba2c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:02,233 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b66575bc060ce15ba2c 2025-12-17 21:20:02,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba2c HTTP/1.1" 502 150 2025-12-17 21:20:02,234 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b66575bc060ce15ba2c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:02,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b55575bc060ce15b9f7 2025-12-17 21:20:02,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b55575bc060ce15b9f7 HTTP/1.1" 502 150 2025-12-17 21:20:02,973 - lava_callback - ERROR - Error processing callback for node 69407b55575bc060ce15b9f7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b55575bc060ce15b9f7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b55575bc060ce15b9f7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b55575bc060ce15b9f7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:02,975 - lava_callback - INFO - Starting async job submit for node 69407b66575bc060ce15ba2f 2025-12-17 21:20:02,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:02,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba2f HTTP/1.1" 502 150 2025-12-17 21:20:02,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b66575bc060ce15ba2f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:02,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b66575bc060ce15ba2f 2025-12-17 21:20:02,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba2f HTTP/1.1" 502 150 2025-12-17 21:20:02,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b66575bc060ce15ba2f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:03,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b64575bc060ce15ba26 2025-12-17 21:20:03,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b64575bc060ce15ba26 HTTP/1.1" 502 150 2025-12-17 21:20:03,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b64575bc060ce15ba26'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:03,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b56575bc060ce15b9fa 2025-12-17 21:20:03,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b56575bc060ce15b9fa HTTP/1.1" 502 150 2025-12-17 21:20:03,696 - lava_callback - ERROR - Error processing callback for node 69407b56575bc060ce15b9fa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b56575bc060ce15b9fa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b56575bc060ce15b9fa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b56575bc060ce15b9fa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:03,698 - lava_callback - INFO - Starting async job submit for node 69407b66575bc060ce15ba32 2025-12-17 21:20:03,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:03,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba32 HTTP/1.1" 502 150 2025-12-17 21:20:03,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b66575bc060ce15ba32'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:03,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b66575bc060ce15ba32 2025-12-17 21:20:03,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba32 HTTP/1.1" 502 150 2025-12-17 21:20:03,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b66575bc060ce15ba32'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:03,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b32575bc060ce15b978 2025-12-17 21:20:03,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b32575bc060ce15b978 HTTP/1.1" 502 150 2025-12-17 21:20:03,860 - lava_callback - ERROR - Error processing callback for node 69407b32575bc060ce15b978: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b32575bc060ce15b978 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b32575bc060ce15b978 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b32575bc060ce15b978 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:03,862 - lava_callback - INFO - Starting async job submit for node 69407b67575bc060ce15ba35 2025-12-17 21:20:03,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:03,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b67575bc060ce15ba35 HTTP/1.1" 502 150 2025-12-17 21:20:03,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b67575bc060ce15ba35'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:03,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b67575bc060ce15ba35 2025-12-17 21:20:03,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b67575bc060ce15ba35 HTTP/1.1" 502 150 2025-12-17 21:20:03,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b67575bc060ce15ba35'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:03,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b62575bc060ce15ba20 2025-12-17 21:20:03,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b62575bc060ce15ba20 HTTP/1.1" 502 150 2025-12-17 21:20:03,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b62575bc060ce15ba20'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:04,234 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b66575bc060ce15ba2c 2025-12-17 21:20:04,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba2c HTTP/1.1" 502 150 2025-12-17 21:20:04,235 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b66575bc060ce15ba2c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:04,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b58575bc060ce15b9fd 2025-12-17 21:20:04,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b58575bc060ce15b9fd HTTP/1.1" 502 150 2025-12-17 21:20:04,466 - lava_callback - ERROR - Error processing callback for node 69407b58575bc060ce15b9fd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b58575bc060ce15b9fd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b58575bc060ce15b9fd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b58575bc060ce15b9fd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:04,467 - lava_callback - INFO - Starting async job submit for node 69407b67575bc060ce15ba38 2025-12-17 21:20:04,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:04,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b67575bc060ce15ba38 HTTP/1.1" 502 150 2025-12-17 21:20:04,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b67575bc060ce15ba38'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:04,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b67575bc060ce15ba38 2025-12-17 21:20:04,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b67575bc060ce15ba38 HTTP/1.1" 502 150 2025-12-17 21:20:04,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b67575bc060ce15ba38'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:04,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b59575bc060ce15ba01 2025-12-17 21:20:04,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b59575bc060ce15ba01 HTTP/1.1" 502 150 2025-12-17 21:20:04,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b59575bc060ce15ba01'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:04,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b66575bc060ce15ba2f 2025-12-17 21:20:04,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba2f HTTP/1.1" 502 150 2025-12-17 21:20:04,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b66575bc060ce15ba2f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:05,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b63575bc060ce15ba23 2025-12-17 21:20:05,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b63575bc060ce15ba23 HTTP/1.1" 502 150 2025-12-17 21:20:05,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b63575bc060ce15ba23'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:05,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b66575bc060ce15ba32 2025-12-17 21:20:05,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba32 HTTP/1.1" 502 150 2025-12-17 21:20:05,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b66575bc060ce15ba32'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:05,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b67575bc060ce15ba35 2025-12-17 21:20:05,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b67575bc060ce15ba35 HTTP/1.1" 502 150 2025-12-17 21:20:05,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b67575bc060ce15ba35'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:05,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5a575bc060ce15ba06 2025-12-17 21:20:05,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5a575bc060ce15ba06 HTTP/1.1" 502 150 2025-12-17 21:20:05,984 - lava_callback - ERROR - Error processing callback for node 69407b5a575bc060ce15ba06: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b5a575bc060ce15ba06 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b5a575bc060ce15ba06 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b5a575bc060ce15ba06 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:05,986 - lava_callback - INFO - Starting async job submit for node 69407b68575bc060ce15ba3b 2025-12-17 21:20:05,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:05,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b68575bc060ce15ba3b HTTP/1.1" 502 150 2025-12-17 21:20:05,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b68575bc060ce15ba3b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:05,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b68575bc060ce15ba3b 2025-12-17 21:20:05,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b68575bc060ce15ba3b HTTP/1.1" 502 150 2025-12-17 21:20:05,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b68575bc060ce15ba3b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:06,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b67575bc060ce15ba38 2025-12-17 21:20:06,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b67575bc060ce15ba38 HTTP/1.1" 502 150 2025-12-17 21:20:06,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b67575bc060ce15ba38'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38238 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:20:07,286 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5b575bc060ce15ba0b 2025-12-17 21:20:07,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5b575bc060ce15ba0b HTTP/1.1" 502 150 2025-12-17 21:20:07,288 - lava_callback - ERROR - Error processing callback for node 69407b5b575bc060ce15ba0b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b5b575bc060ce15ba0b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b5b575bc060ce15ba0b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b5b575bc060ce15ba0b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:07,290 - lava_callback - INFO - Starting async job submit for node 69407b6a575bc060ce15ba3e 2025-12-17 21:20:07,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:07,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b6a575bc060ce15ba3e HTTP/1.1" 502 150 2025-12-17 21:20:07,302 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b6a575bc060ce15ba3e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:07,302 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b6a575bc060ce15ba3e 2025-12-17 21:20:07,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b6a575bc060ce15ba3e HTTP/1.1" 502 150 2025-12-17 21:20:07,302 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b6a575bc060ce15ba3e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:07,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b64575bc060ce15ba26 2025-12-17 21:20:07,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b64575bc060ce15ba26 HTTP/1.1" 502 150 2025-12-17 21:20:07,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b64575bc060ce15ba26'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:07,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5d575bc060ce15ba0f 2025-12-17 21:20:07,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5d575bc060ce15ba0f HTTP/1.1" 502 150 2025-12-17 21:20:07,975 - lava_callback - ERROR - Error processing callback for node 69407b5d575bc060ce15ba0f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b5d575bc060ce15ba0f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b5d575bc060ce15ba0f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b5d575bc060ce15ba0f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:07,977 - lava_callback - INFO - Starting async job submit for node 69407b6f575bc060ce15ba48 2025-12-17 21:20:07,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:07,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b6f575bc060ce15ba48 HTTP/1.1" 502 150 2025-12-17 21:20:07,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b6f575bc060ce15ba48'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:07,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b6f575bc060ce15ba48 2025-12-17 21:20:07,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b6f575bc060ce15ba48 HTTP/1.1" 502 150 2025-12-17 21:20:07,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b6f575bc060ce15ba48'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:07,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b68575bc060ce15ba3b 2025-12-17 21:20:07,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b68575bc060ce15ba3b HTTP/1.1" 502 150 2025-12-17 21:20:07,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b68575bc060ce15ba3b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:08,235 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b66575bc060ce15ba2c 2025-12-17 21:20:08,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b66575bc060ce15ba2f 2025-12-17 21:20:08,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba2f HTTP/1.1" 502 150 2025-12-17 21:20:08,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b66575bc060ce15ba2f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:09,303 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b6a575bc060ce15ba3e 2025-12-17 21:20:09,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b6a575bc060ce15ba3e HTTP/1.1" 502 150 2025-12-17 21:20:09,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b6a575bc060ce15ba3e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:09,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b66575bc060ce15ba32 2025-12-17 21:20:09,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba32 HTTP/1.1" 502 150 2025-12-17 21:20:09,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b66575bc060ce15ba32'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:09,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b67575bc060ce15ba35 2025-12-17 21:20:09,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b67575bc060ce15ba35 HTTP/1.1" 502 150 2025-12-17 21:20:09,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b67575bc060ce15ba35'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:09,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b6f575bc060ce15ba48 2025-12-17 21:20:09,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b6f575bc060ce15ba48 HTTP/1.1" 502 150 2025-12-17 21:20:09,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b6f575bc060ce15ba48'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:10,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b67575bc060ce15ba38 2025-12-17 21:20:10,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b67575bc060ce15ba38 HTTP/1.1" 502 150 2025-12-17 21:20:10,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b67575bc060ce15ba38'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:11,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5e575bc060ce15ba12 2025-12-17 21:20:11,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5e575bc060ce15ba12 HTTP/1.1" 502 150 2025-12-17 21:20:11,771 - lava_callback - ERROR - Error processing callback for node 69407b5e575bc060ce15ba12: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b5e575bc060ce15ba12 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b5e575bc060ce15ba12 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b5e575bc060ce15ba12 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:11,774 - lava_callback - INFO - Starting async job submit for node 69407b70575bc060ce15ba53 2025-12-17 21:20:11,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:11,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b70575bc060ce15ba53 HTTP/1.1" 502 150 2025-12-17 21:20:11,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b70575bc060ce15ba53'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:11,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b70575bc060ce15ba53 2025-12-17 21:20:11,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b70575bc060ce15ba53 HTTP/1.1" 502 150 2025-12-17 21:20:11,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b70575bc060ce15ba53'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:11,842 - lava_callback - INFO - Received callback for node 69407b82575bc060ce15ba72 INFO: 172.21.0.1:47146 - "POST /node/69407b82575bc060ce15ba72 HTTP/1.0" 202 Accepted 2025-12-17 21:20:11,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b62575bc060ce15ba20 2025-12-17 21:20:11,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b62575bc060ce15ba20 HTTP/1.1" 502 150 2025-12-17 21:20:11,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b62575bc060ce15ba20'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:11,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b68575bc060ce15ba3b 2025-12-17 21:20:11,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b68575bc060ce15ba3b HTTP/1.1" 502 150 2025-12-17 21:20:11,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b68575bc060ce15ba3b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:12,574 - lava_callback - INFO - Received callback for node 69407b83575bc060ce15ba78 INFO: 172.21.0.1:47162 - "POST /node/69407b83575bc060ce15ba78 HTTP/1.0" 202 Accepted 2025-12-17 21:20:13,304 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b6a575bc060ce15ba3e 2025-12-17 21:20:13,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b6a575bc060ce15ba3e HTTP/1.1" 502 150 2025-12-17 21:20:13,305 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b6a575bc060ce15ba3e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:13,322 - lava_callback - INFO - Received callback for node 69407b84575bc060ce15ba7e INFO: 172.21.0.1:47166 - "POST /node/69407b84575bc060ce15ba7e HTTP/1.0" 202 Accepted 2025-12-17 21:20:13,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b63575bc060ce15ba23 2025-12-17 21:20:13,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b63575bc060ce15ba23 HTTP/1.1" 502 150 2025-12-17 21:20:13,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b63575bc060ce15ba23'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:13,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b70575bc060ce15ba53 2025-12-17 21:20:13,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b70575bc060ce15ba53 HTTP/1.1" 502 150 2025-12-17 21:20:13,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b70575bc060ce15ba53'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:13,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b6f575bc060ce15ba48 2025-12-17 21:20:13,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b6f575bc060ce15ba48 HTTP/1.1" 502 150 2025-12-17 21:20:13,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b6f575bc060ce15ba48'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:14,211 - lava_callback - INFO - Received callback for node 69407b86575bc060ce15ba85 INFO: 172.21.0.1:47168 - "POST /node/69407b86575bc060ce15ba85 HTTP/1.0" 202 Accepted 2025-12-17 21:20:14,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b61575bc060ce15ba1d 2025-12-17 21:20:14,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b61575bc060ce15ba1d HTTP/1.1" 502 150 2025-12-17 21:20:14,884 - lava_callback - ERROR - Error processing callback for node 69407b61575bc060ce15ba1d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b61575bc060ce15ba1d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b61575bc060ce15ba1d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b61575bc060ce15ba1d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:14,885 - lava_callback - INFO - Starting async job submit for node 69407b73575bc060ce15ba5f 2025-12-17 21:20:14,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:14,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b73575bc060ce15ba5f HTTP/1.1" 502 150 2025-12-17 21:20:14,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b73575bc060ce15ba5f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:14,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b73575bc060ce15ba5f 2025-12-17 21:20:14,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b73575bc060ce15ba5f HTTP/1.1" 502 150 2025-12-17 21:20:14,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b73575bc060ce15ba5f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:14,965 - lava_callback - INFO - Received callback for node 69407b87575bc060ce15ba8e INFO: 172.21.0.1:47182 - "POST /node/69407b87575bc060ce15ba8e HTTP/1.0" 202 Accepted INFO: 172.21.0.1:47194 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:20:15,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b64575bc060ce15ba26 2025-12-17 21:20:15,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b64575bc060ce15ba26 HTTP/1.1" 502 150 2025-12-17 21:20:15,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b64575bc060ce15ba26'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:15,764 - lava_callback - INFO - Received callback for node 69407b88575bc060ce15ba93 INFO: 172.21.0.1:47208 - "POST /node/69407b88575bc060ce15ba93 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:47210 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:20:16,578 - lava_callback - INFO - Received callback for node 69407b8d575bc060ce15bad9 INFO: 172.21.0.1:47218 - "POST /node/69407b8d575bc060ce15bad9 HTTP/1.0" 202 Accepted 2025-12-17 21:20:16,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b65575bc060ce15ba29 2025-12-17 21:20:16,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b65575bc060ce15ba29 HTTP/1.1" 502 150 2025-12-17 21:20:16,615 - lava_callback - ERROR - Error processing callback for node 69407b65575bc060ce15ba29: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b65575bc060ce15ba29 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b65575bc060ce15ba29 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b65575bc060ce15ba29 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:16,616 - lava_callback - INFO - Starting async job submit for node 69407b71575bc060ce15ba59 2025-12-17 21:20:16,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:16,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b71575bc060ce15ba59 HTTP/1.1" 502 150 2025-12-17 21:20:16,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b71575bc060ce15ba59'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:16,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b71575bc060ce15ba59 2025-12-17 21:20:16,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b71575bc060ce15ba59 HTTP/1.1" 502 150 2025-12-17 21:20:16,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b71575bc060ce15ba59'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:16,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b73575bc060ce15ba5f 2025-12-17 21:20:16,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b73575bc060ce15ba5f HTTP/1.1" 502 150 2025-12-17 21:20:16,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b73575bc060ce15ba5f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:16,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b66575bc060ce15ba2f 2025-12-17 21:20:16,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba2f HTTP/1.1" 502 150 2025-12-17 21:20:16,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b66575bc060ce15ba2f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:17,354 - lava_callback - INFO - Received callback for node 69407b8e575bc060ce15badc INFO: 172.21.0.1:47230 - "POST /node/69407b8e575bc060ce15badc HTTP/1.0" 202 Accepted 2025-12-17 21:20:17,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b66575bc060ce15ba32 2025-12-17 21:20:17,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba32 HTTP/1.1" 502 150 2025-12-17 21:20:17,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b66575bc060ce15ba32'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:17,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b70575bc060ce15ba53 2025-12-17 21:20:17,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b70575bc060ce15ba53 HTTP/1.1" 502 150 2025-12-17 21:20:17,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b70575bc060ce15ba53'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:17,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b67575bc060ce15ba35 2025-12-17 21:20:17,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b67575bc060ce15ba35 HTTP/1.1" 502 150 2025-12-17 21:20:17,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b67575bc060ce15ba35'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:18,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b67575bc060ce15ba38 2025-12-17 21:20:18,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b67575bc060ce15ba38 HTTP/1.1" 502 150 2025-12-17 21:20:18,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b67575bc060ce15ba38'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:18,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b71575bc060ce15ba59 2025-12-17 21:20:18,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b71575bc060ce15ba59 HTTP/1.1" 502 150 2025-12-17 21:20:18,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b71575bc060ce15ba59'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:19,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b68575bc060ce15ba3b 2025-12-17 21:20:19,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b68575bc060ce15ba3b HTTP/1.1" 502 150 2025-12-17 21:20:19,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b68575bc060ce15ba3b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:20,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b59575bc060ce15ba01 2025-12-17 21:20:20,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b59575bc060ce15ba01 HTTP/1.1" 502 150 2025-12-17 21:20:20,712 - lava_callback - ERROR - Error processing callback for node 69407b59575bc060ce15ba01: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b59575bc060ce15ba01 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b59575bc060ce15ba01 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b59575bc060ce15ba01 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:20,715 - lava_callback - INFO - Starting async job submit for node 69407b74575bc060ce15ba62 2025-12-17 21:20:20,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:20,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b74575bc060ce15ba62 HTTP/1.1" 502 150 2025-12-17 21:20:20,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b74575bc060ce15ba62'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:20,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b74575bc060ce15ba62 2025-12-17 21:20:20,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b74575bc060ce15ba62 HTTP/1.1" 502 150 2025-12-17 21:20:20,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b74575bc060ce15ba62'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:20,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b73575bc060ce15ba5f 2025-12-17 21:20:20,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b73575bc060ce15ba5f HTTP/1.1" 502 150 2025-12-17 21:20:20,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b73575bc060ce15ba5f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:21,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b6a575bc060ce15ba3e 2025-12-17 21:20:21,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b6a575bc060ce15ba3e HTTP/1.1" 502 150 2025-12-17 21:20:21,307 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b6a575bc060ce15ba3e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:21,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b6f575bc060ce15ba48 2025-12-17 21:20:21,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b6f575bc060ce15ba48 HTTP/1.1" 502 150 2025-12-17 21:20:21,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b6f575bc060ce15ba48'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:22,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b71575bc060ce15ba59 2025-12-17 21:20:22,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b71575bc060ce15ba59 HTTP/1.1" 502 150 2025-12-17 21:20:22,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b71575bc060ce15ba59'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:22,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b74575bc060ce15ba62 2025-12-17 21:20:22,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b74575bc060ce15ba62 HTTP/1.1" 502 150 2025-12-17 21:20:22,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b74575bc060ce15ba62'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:23,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba2c HTTP/1.1" 502 150 2025-12-17 21:20:23,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b66575bc060ce15ba2c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:24,440 - lava_callback - INFO - Received callback for node 69408c63575bc060ce15f57a INFO: 172.21.0.1:53452 - "POST /node/69408c63575bc060ce15f57a HTTP/1.0" 202 Accepted 2025-12-17 21:20:25,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b70575bc060ce15ba53 2025-12-17 21:20:25,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b70575bc060ce15ba53 HTTP/1.1" 502 150 2025-12-17 21:20:25,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b70575bc060ce15ba53'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53458 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:20:26,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b74575bc060ce15ba62 2025-12-17 21:20:26,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b74575bc060ce15ba62 HTTP/1.1" 502 150 2025-12-17 21:20:26,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b74575bc060ce15ba62'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:27,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b62575bc060ce15ba20 2025-12-17 21:20:27,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b62575bc060ce15ba20 HTTP/1.1" 502 150 2025-12-17 21:20:27,888 - lava_callback - ERROR - Error processing callback for node 69407b62575bc060ce15ba20: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b62575bc060ce15ba20 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b62575bc060ce15ba20 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b62575bc060ce15ba20 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:27,890 - lava_callback - INFO - Starting async job submit for node 69407b75575bc060ce15ba65 2025-12-17 21:20:27,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:27,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b75575bc060ce15ba65 HTTP/1.1" 502 150 2025-12-17 21:20:27,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b75575bc060ce15ba65'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:27,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b75575bc060ce15ba65 2025-12-17 21:20:27,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b75575bc060ce15ba65 HTTP/1.1" 502 150 2025-12-17 21:20:27,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b75575bc060ce15ba65'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:28,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5f575bc060ce15ba1a HTTP/1.1" 502 150 2025-12-17 21:20:28,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b5f575bc060ce15ba1a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:28,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b73575bc060ce15ba5f 2025-12-17 21:20:28,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b73575bc060ce15ba5f HTTP/1.1" 502 150 2025-12-17 21:20:28,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b73575bc060ce15ba5f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:29,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b63575bc060ce15ba23 2025-12-17 21:20:29,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b63575bc060ce15ba23 HTTP/1.1" 502 150 2025-12-17 21:20:29,526 - lava_callback - ERROR - Error processing callback for node 69407b63575bc060ce15ba23: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b63575bc060ce15ba23 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b63575bc060ce15ba23 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b63575bc060ce15ba23 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:29,528 - lava_callback - INFO - Starting async job submit for node 69407b76575bc060ce15ba68 2025-12-17 21:20:29,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:29,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b76575bc060ce15ba68 HTTP/1.1" 502 150 2025-12-17 21:20:29,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b76575bc060ce15ba68'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:29,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b76575bc060ce15ba68 2025-12-17 21:20:29,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b76575bc060ce15ba68 HTTP/1.1" 502 150 2025-12-17 21:20:29,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b76575bc060ce15ba68'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:29,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b75575bc060ce15ba65 2025-12-17 21:20:29,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b75575bc060ce15ba65 HTTP/1.1" 502 150 2025-12-17 21:20:29,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b75575bc060ce15ba65'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:30,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b71575bc060ce15ba59 2025-12-17 21:20:30,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b71575bc060ce15ba59 HTTP/1.1" 502 150 2025-12-17 21:20:30,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b71575bc060ce15ba59'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:31,342 - lava_callback - INFO - Received callback for node 69407ba4575bc060ce15bb0f INFO: 172.21.0.1:42118 - "POST /node/69407ba4575bc060ce15bb0f HTTP/1.0" 202 Accepted 2025-12-17 21:20:31,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b64575bc060ce15ba26 2025-12-17 21:20:31,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b64575bc060ce15ba26 HTTP/1.1" 502 150 2025-12-17 21:20:31,495 - lava_callback - ERROR - Error processing callback for node 69407b64575bc060ce15ba26: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b64575bc060ce15ba26 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b64575bc060ce15ba26 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b64575bc060ce15ba26 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:31,497 - lava_callback - INFO - Starting async job submit for node 69407b78575bc060ce15ba6b 2025-12-17 21:20:31,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:31,502 - lava_callback - INFO - Received callback for node 69407b94575bc060ce15badf INFO: 172.21.0.1:42132 - "POST /node/69407b94575bc060ce15badf HTTP/1.0" 202 Accepted 2025-12-17 21:20:31,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b78575bc060ce15ba6b HTTP/1.1" 502 150 2025-12-17 21:20:31,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b78575bc060ce15ba6b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:31,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b78575bc060ce15ba6b 2025-12-17 21:20:31,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b78575bc060ce15ba6b HTTP/1.1" 502 150 2025-12-17 21:20:31,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b78575bc060ce15ba6b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:31,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b76575bc060ce15ba68 2025-12-17 21:20:31,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b76575bc060ce15ba68 HTTP/1.1" 502 150 2025-12-17 21:20:31,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b76575bc060ce15ba68'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:31,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b66575bc060ce15ba2c 2025-12-17 21:20:31,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba2c HTTP/1.1" 502 150 2025-12-17 21:20:31,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b66575bc060ce15ba2c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:32,275 - lava_callback - INFO - Received callback for node 69407b99575bc060ce15baef INFO: 172.21.0.1:42136 - "POST /node/69407b99575bc060ce15baef HTTP/1.0" 202 Accepted 2025-12-17 21:20:32,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b66575bc060ce15ba2f 2025-12-17 21:20:33,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba2f HTTP/1.1" 502 150 2025-12-17 21:20:33,001 - lava_callback - ERROR - Error processing callback for node 69407b66575bc060ce15ba2f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b66575bc060ce15ba2f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b66575bc060ce15ba2f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b66575bc060ce15ba2f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:33,003 - lava_callback - INFO - Starting async job submit for node 69407b82575bc060ce15ba72 2025-12-17 21:20:33,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:33,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b82575bc060ce15ba72 HTTP/1.1" 502 150 2025-12-17 21:20:33,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b82575bc060ce15ba72'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:33,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b82575bc060ce15ba72 2025-12-17 21:20:33,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b82575bc060ce15ba72 HTTP/1.1" 502 150 2025-12-17 21:20:33,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b82575bc060ce15ba72'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:33,025 - lava_callback - INFO - Received callback for node 69407b9a575bc060ce15baf6 INFO: 172.21.0.1:42152 - "POST /node/69407b9a575bc060ce15baf6 HTTP/1.0" 202 Accepted 2025-12-17 21:20:33,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b78575bc060ce15ba6b 2025-12-17 21:20:33,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b78575bc060ce15ba6b HTTP/1.1" 502 150 2025-12-17 21:20:33,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b78575bc060ce15ba6b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:33,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b66575bc060ce15ba32 2025-12-17 21:20:33,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba32 HTTP/1.1" 502 150 2025-12-17 21:20:33,713 - lava_callback - ERROR - Error processing callback for node 69407b66575bc060ce15ba32: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b66575bc060ce15ba32 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b66575bc060ce15ba32 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b66575bc060ce15ba32 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:33,716 - lava_callback - INFO - Starting async job submit for node 69407b83575bc060ce15ba78 2025-12-17 21:20:33,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:33,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b83575bc060ce15ba78 HTTP/1.1" 502 150 2025-12-17 21:20:33,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b83575bc060ce15ba78'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:33,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b83575bc060ce15ba78 2025-12-17 21:20:33,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b83575bc060ce15ba78 HTTP/1.1" 502 150 2025-12-17 21:20:33,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b83575bc060ce15ba78'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:33,844 - lava_callback - INFO - Received callback for node 69407b9c575bc060ce15bafb INFO: 172.21.0.1:42156 - "POST /node/69407b9c575bc060ce15bafb HTTP/1.0" 202 Accepted 2025-12-17 21:20:33,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b67575bc060ce15ba35 2025-12-17 21:20:33,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b67575bc060ce15ba35 HTTP/1.1" 502 150 2025-12-17 21:20:33,878 - lava_callback - ERROR - Error processing callback for node 69407b67575bc060ce15ba35: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b67575bc060ce15ba35 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b67575bc060ce15ba35 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b67575bc060ce15ba35 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:33,881 - lava_callback - INFO - Starting async job submit for node 69407b84575bc060ce15ba7e 2025-12-17 21:20:33,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:33,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b84575bc060ce15ba7e HTTP/1.1" 502 150 2025-12-17 21:20:33,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b84575bc060ce15ba7e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:33,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b84575bc060ce15ba7e 2025-12-17 21:20:33,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b84575bc060ce15ba7e HTTP/1.1" 502 150 2025-12-17 21:20:33,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b84575bc060ce15ba7e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:33,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b75575bc060ce15ba65 2025-12-17 21:20:33,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b75575bc060ce15ba65 HTTP/1.1" 502 150 2025-12-17 21:20:33,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b75575bc060ce15ba65'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:34,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b67575bc060ce15ba38 2025-12-17 21:20:34,632 - lava_callback - INFO - Received callback for node 69407b9d575bc060ce15bb06 INFO: 172.21.0.1:42168 - "POST /node/69407b9d575bc060ce15bb06 HTTP/1.0" 202 Accepted 2025-12-17 21:20:34,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b74575bc060ce15ba62 2025-12-17 21:20:34,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b74575bc060ce15ba62 HTTP/1.1" 502 150 2025-12-17 21:20:34,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b74575bc060ce15ba62'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:35,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b82575bc060ce15ba72 2025-12-17 21:20:35,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b82575bc060ce15ba72 HTTP/1.1" 502 150 2025-12-17 21:20:35,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b82575bc060ce15ba72'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:35,372 - lava_callback - INFO - Received callback for node 69407ba1575bc060ce15bb09 INFO: 172.21.0.1:42174 - "POST /node/69407ba1575bc060ce15bb09 HTTP/1.0" 202 Accepted 2025-12-17 21:20:35,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b76575bc060ce15ba68 2025-12-17 21:20:35,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b76575bc060ce15ba68 HTTP/1.1" 502 150 2025-12-17 21:20:35,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b76575bc060ce15ba68'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:35,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b83575bc060ce15ba78 2025-12-17 21:20:35,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b83575bc060ce15ba78 HTTP/1.1" 502 150 2025-12-17 21:20:35,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b83575bc060ce15ba78'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:35,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b84575bc060ce15ba7e 2025-12-17 21:20:35,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b84575bc060ce15ba7e HTTP/1.1" 502 150 2025-12-17 21:20:35,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b84575bc060ce15ba7e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:35,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b68575bc060ce15ba3b 2025-12-17 21:20:35,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b68575bc060ce15ba3b HTTP/1.1" 502 150 2025-12-17 21:20:35,998 - lava_callback - ERROR - Error processing callback for node 69407b68575bc060ce15ba3b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b68575bc060ce15ba3b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b68575bc060ce15ba3b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b68575bc060ce15ba3b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:36,000 - lava_callback - INFO - Starting async job submit for node 69407b86575bc060ce15ba85 2025-12-17 21:20:36,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:36,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b86575bc060ce15ba85 HTTP/1.1" 502 150 2025-12-17 21:20:36,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b86575bc060ce15ba85'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:36,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b86575bc060ce15ba85 2025-12-17 21:20:36,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b86575bc060ce15ba85 HTTP/1.1" 502 150 2025-12-17 21:20:36,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b86575bc060ce15ba85'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42190 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:20:37,307 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b6a575bc060ce15ba3e 2025-12-17 21:20:37,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b6a575bc060ce15ba3e HTTP/1.1" 502 150 2025-12-17 21:20:37,309 - lava_callback - ERROR - Error processing callback for node 69407b6a575bc060ce15ba3e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b6a575bc060ce15ba3e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b6a575bc060ce15ba3e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b6a575bc060ce15ba3e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:37,311 - lava_callback - INFO - Starting async job submit for node 69407b87575bc060ce15ba8e 2025-12-17 21:20:37,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:37,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b87575bc060ce15ba8e HTTP/1.1" 502 150 2025-12-17 21:20:37,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b87575bc060ce15ba8e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:37,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b87575bc060ce15ba8e 2025-12-17 21:20:37,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b87575bc060ce15ba8e HTTP/1.1" 502 150 2025-12-17 21:20:37,338 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b87575bc060ce15ba8e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:37,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b78575bc060ce15ba6b 2025-12-17 21:20:37,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b78575bc060ce15ba6b HTTP/1.1" 502 150 2025-12-17 21:20:37,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b78575bc060ce15ba6b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:37,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b6f575bc060ce15ba48 2025-12-17 21:20:37,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b6f575bc060ce15ba48 HTTP/1.1" 502 150 2025-12-17 21:20:37,990 - lava_callback - ERROR - Error processing callback for node 69407b6f575bc060ce15ba48: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b6f575bc060ce15ba48 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b6f575bc060ce15ba48 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b6f575bc060ce15ba48 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:37,992 - lava_callback - INFO - Starting async job submit for node 69407b88575bc060ce15ba93 2025-12-17 21:20:37,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:38,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b86575bc060ce15ba85 2025-12-17 21:20:38,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b86575bc060ce15ba85 HTTP/1.1" 502 150 2025-12-17 21:20:38,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b86575bc060ce15ba85'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:38,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b88575bc060ce15ba93 HTTP/1.1" 502 150 2025-12-17 21:20:38,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b88575bc060ce15ba93'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:38,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b88575bc060ce15ba93 2025-12-17 21:20:38,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b88575bc060ce15ba93 HTTP/1.1" 502 150 2025-12-17 21:20:38,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b88575bc060ce15ba93'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:39,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b82575bc060ce15ba72 2025-12-17 21:20:39,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b87575bc060ce15ba8e 2025-12-17 21:20:39,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b87575bc060ce15ba8e HTTP/1.1" 502 150 2025-12-17 21:20:39,340 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b87575bc060ce15ba8e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:39,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b83575bc060ce15ba78 2025-12-17 21:20:39,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b83575bc060ce15ba78 HTTP/1.1" 502 150 2025-12-17 21:20:39,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b83575bc060ce15ba78'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:39,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b84575bc060ce15ba7e 2025-12-17 21:20:39,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b84575bc060ce15ba7e HTTP/1.1" 502 150 2025-12-17 21:20:39,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b84575bc060ce15ba7e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:40,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b88575bc060ce15ba93 2025-12-17 21:20:40,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b88575bc060ce15ba93 HTTP/1.1" 502 150 2025-12-17 21:20:40,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b88575bc060ce15ba93'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:41,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b70575bc060ce15ba53 2025-12-17 21:20:41,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b70575bc060ce15ba53 HTTP/1.1" 502 150 2025-12-17 21:20:41,793 - lava_callback - ERROR - Error processing callback for node 69407b70575bc060ce15ba53: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b70575bc060ce15ba53 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b70575bc060ce15ba53 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b70575bc060ce15ba53 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:41,795 - lava_callback - INFO - Starting async job submit for node 69407b8d575bc060ce15bad9 2025-12-17 21:20:41,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:41,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b8d575bc060ce15bad9 HTTP/1.1" 502 150 2025-12-17 21:20:41,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b8d575bc060ce15bad9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:41,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b8d575bc060ce15bad9 2025-12-17 21:20:41,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b8d575bc060ce15bad9 HTTP/1.1" 502 150 2025-12-17 21:20:41,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b8d575bc060ce15bad9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:41,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b75575bc060ce15ba65 2025-12-17 21:20:41,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b75575bc060ce15ba65 HTTP/1.1" 502 150 2025-12-17 21:20:41,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b75575bc060ce15ba65'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:42,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b86575bc060ce15ba85 2025-12-17 21:20:42,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b86575bc060ce15ba85 HTTP/1.1" 502 150 2025-12-17 21:20:42,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b86575bc060ce15ba85'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:43,340 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b87575bc060ce15ba8e 2025-12-17 21:20:43,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b87575bc060ce15ba8e HTTP/1.1" 502 150 2025-12-17 21:20:43,341 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b87575bc060ce15ba8e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:43,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b76575bc060ce15ba68 2025-12-17 21:20:43,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b76575bc060ce15ba68 HTTP/1.1" 502 150 2025-12-17 21:20:43,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b76575bc060ce15ba68'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:43,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b8d575bc060ce15bad9 2025-12-17 21:20:43,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b8d575bc060ce15bad9 HTTP/1.1" 502 150 2025-12-17 21:20:43,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b8d575bc060ce15bad9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:44,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b88575bc060ce15ba93 2025-12-17 21:20:44,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b88575bc060ce15ba93 HTTP/1.1" 502 150 2025-12-17 21:20:44,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b88575bc060ce15ba93'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:44,080 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b5f575bc060ce15ba1a 2025-12-17 21:20:44,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b5f575bc060ce15ba1a HTTP/1.1" 502 150 2025-12-17 21:20:44,081 - lava_callback - ERROR - Error processing callback for node 69407b5f575bc060ce15ba1a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b5f575bc060ce15ba1a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b5f575bc060ce15ba1a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b5f575bc060ce15ba1a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:44,084 - lava_callback - INFO - Starting async job submit for node 69407b8e575bc060ce15badc 2025-12-17 21:20:44,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:44,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b8e575bc060ce15badc HTTP/1.1" 502 150 2025-12-17 21:20:44,092 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b8e575bc060ce15badc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:44,092 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b8e575bc060ce15badc 2025-12-17 21:20:44,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b8e575bc060ce15badc HTTP/1.1" 502 150 2025-12-17 21:20:44,093 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b8e575bc060ce15badc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:44,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b73575bc060ce15ba5f 2025-12-17 21:20:44,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b73575bc060ce15ba5f HTTP/1.1" 502 150 2025-12-17 21:20:44,901 - lava_callback - ERROR - Error processing callback for node 69407b73575bc060ce15ba5f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b73575bc060ce15ba5f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b73575bc060ce15ba5f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b73575bc060ce15ba5f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:44,902 - lava_callback - INFO - Starting async job submit for node 69408c63575bc060ce15f57a 2025-12-17 21:20:44,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:44,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c63575bc060ce15f57a HTTP/1.1" 502 150 2025-12-17 21:20:44,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c63575bc060ce15f57a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:44,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c63575bc060ce15f57a 2025-12-17 21:20:44,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c63575bc060ce15f57a HTTP/1.1" 502 150 2025-12-17 21:20:44,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c63575bc060ce15f57a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:45,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b78575bc060ce15ba6b 2025-12-17 21:20:45,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b78575bc060ce15ba6b HTTP/1.1" 502 150 2025-12-17 21:20:45,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b78575bc060ce15ba6b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:46,093 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b8e575bc060ce15badc 2025-12-17 21:20:46,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b8e575bc060ce15badc HTTP/1.1" 502 150 2025-12-17 21:20:46,094 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b8e575bc060ce15badc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53166 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:20:46,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b71575bc060ce15ba59 2025-12-17 21:20:46,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b71575bc060ce15ba59 HTTP/1.1" 502 150 2025-12-17 21:20:46,629 - lava_callback - ERROR - Error processing callback for node 69407b71575bc060ce15ba59: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b71575bc060ce15ba59 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b71575bc060ce15ba59 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b71575bc060ce15ba59 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:46,631 - lava_callback - INFO - Starting async job submit for node 69407ba4575bc060ce15bb0f 2025-12-17 21:20:46,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:46,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba4575bc060ce15bb0f HTTP/1.1" 502 150 2025-12-17 21:20:46,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba4575bc060ce15bb0f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:46,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba4575bc060ce15bb0f 2025-12-17 21:20:46,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba4575bc060ce15bb0f HTTP/1.1" 502 150 2025-12-17 21:20:46,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba4575bc060ce15bb0f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:46,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c63575bc060ce15f57a 2025-12-17 21:20:46,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c63575bc060ce15f57a HTTP/1.1" 502 150 2025-12-17 21:20:46,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c63575bc060ce15f57a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:47,115 - lava_callback - INFO - Received callback for node 69407ba7575bc060ce15bb21 INFO: 172.21.0.1:53178 - "POST /node/69407ba7575bc060ce15bb21 HTTP/1.0" 202 Accepted 2025-12-17 21:20:47,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b66575bc060ce15ba2c 2025-12-17 21:20:47,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b66575bc060ce15ba2c HTTP/1.1" 502 150 2025-12-17 21:20:47,739 - lava_callback - ERROR - Error processing callback for node 69407b66575bc060ce15ba2c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b66575bc060ce15ba2c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b66575bc060ce15ba2c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b66575bc060ce15ba2c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:47,741 - lava_callback - INFO - Starting async job submit for node 69407b94575bc060ce15badf 2025-12-17 21:20:47,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:47,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b94575bc060ce15badf HTTP/1.1" 502 150 2025-12-17 21:20:47,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b94575bc060ce15badf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:47,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b94575bc060ce15badf 2025-12-17 21:20:47,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b94575bc060ce15badf HTTP/1.1" 502 150 2025-12-17 21:20:47,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b94575bc060ce15badf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:47,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b83575bc060ce15ba78 2025-12-17 21:20:47,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b83575bc060ce15ba78 HTTP/1.1" 502 150 2025-12-17 21:20:47,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b83575bc060ce15ba78'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:47,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b8d575bc060ce15bad9 2025-12-17 21:20:47,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b8d575bc060ce15bad9 HTTP/1.1" 502 150 2025-12-17 21:20:47,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b8d575bc060ce15bad9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:47,836 - lava_callback - INFO - Received callback for node 69407ba8575bc060ce15bb24 INFO: 172.21.0.1:53192 - "POST /node/69407ba8575bc060ce15bb24 HTTP/1.0" 202 Accepted 2025-12-17 21:20:47,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b84575bc060ce15ba7e 2025-12-17 21:20:47,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b84575bc060ce15ba7e HTTP/1.1" 502 150 2025-12-17 21:20:47,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b84575bc060ce15ba7e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:48,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba4575bc060ce15bb0f 2025-12-17 21:20:48,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba4575bc060ce15bb0f HTTP/1.1" 502 150 2025-12-17 21:20:48,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba4575bc060ce15bb0f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:49,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b94575bc060ce15badf 2025-12-17 21:20:49,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b67575bc060ce15ba38 HTTP/1.1" 502 150 2025-12-17 21:20:49,840 - lava_callback - ERROR - Error processing callback for node 69407b67575bc060ce15ba38: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b67575bc060ce15ba38 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b67575bc060ce15ba38 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b67575bc060ce15ba38 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:49,842 - lava_callback - INFO - Starting async job submit for node 69407b99575bc060ce15baef 2025-12-17 21:20:49,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:49,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b99575bc060ce15baef HTTP/1.1" 502 150 2025-12-17 21:20:49,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b99575bc060ce15baef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:49,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b99575bc060ce15baef 2025-12-17 21:20:49,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b99575bc060ce15baef HTTP/1.1" 502 150 2025-12-17 21:20:49,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b99575bc060ce15baef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:50,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b86575bc060ce15ba85 2025-12-17 21:20:50,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b86575bc060ce15ba85 HTTP/1.1" 502 150 2025-12-17 21:20:50,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b86575bc060ce15ba85'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:50,095 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b8e575bc060ce15badc 2025-12-17 21:20:50,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b8e575bc060ce15badc HTTP/1.1" 502 150 2025-12-17 21:20:50,096 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b8e575bc060ce15badc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:50,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b74575bc060ce15ba62 2025-12-17 21:20:50,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b74575bc060ce15ba62 HTTP/1.1" 502 150 2025-12-17 21:20:50,729 - lava_callback - ERROR - Error processing callback for node 69407b74575bc060ce15ba62: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b74575bc060ce15ba62 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b74575bc060ce15ba62 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b74575bc060ce15ba62 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:50,731 - lava_callback - INFO - Starting async job submit for node 69407b9a575bc060ce15baf6 2025-12-17 21:20:50,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:50,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9a575bc060ce15baf6 HTTP/1.1" 502 150 2025-12-17 21:20:50,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b9a575bc060ce15baf6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:50,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b9a575bc060ce15baf6 2025-12-17 21:20:50,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9a575bc060ce15baf6 HTTP/1.1" 502 150 2025-12-17 21:20:50,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b9a575bc060ce15baf6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:50,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c63575bc060ce15f57a 2025-12-17 21:20:50,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c63575bc060ce15f57a HTTP/1.1" 502 150 2025-12-17 21:20:50,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c63575bc060ce15f57a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:51,341 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b87575bc060ce15ba8e 2025-12-17 21:20:51,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b87575bc060ce15ba8e HTTP/1.1" 502 150 2025-12-17 21:20:51,343 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b87575bc060ce15ba8e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:51,586 - lava_callback - INFO - Received callback for node 69407ba3575bc060ce15bb0c INFO: 172.21.0.1:56768 - "POST /node/69407ba3575bc060ce15bb0c HTTP/1.0" 202 Accepted 2025-12-17 21:20:51,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b99575bc060ce15baef 2025-12-17 21:20:51,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b99575bc060ce15baef HTTP/1.1" 502 150 2025-12-17 21:20:51,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b99575bc060ce15baef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:52,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b88575bc060ce15ba93 2025-12-17 21:20:52,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b88575bc060ce15ba93 HTTP/1.1" 502 150 2025-12-17 21:20:52,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b88575bc060ce15ba93'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:52,354 - lava_callback - INFO - Received callback for node 69407ba5575bc060ce15bb15 INFO: 172.21.0.1:56776 - "POST /node/69407ba5575bc060ce15bb15 HTTP/1.0" 202 Accepted 2025-12-17 21:20:52,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba4575bc060ce15bb0f 2025-12-17 21:20:52,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba4575bc060ce15bb0f HTTP/1.1" 502 150 2025-12-17 21:20:52,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba4575bc060ce15bb0f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:52,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b9a575bc060ce15baf6 2025-12-17 21:20:52,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9a575bc060ce15baf6 HTTP/1.1" 502 150 2025-12-17 21:20:52,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b9a575bc060ce15baf6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:53,318 - lava_callback - INFO - Received callback for node 69407bac575bc060ce15bb39 INFO: 172.21.0.1:56784 - "POST /node/69407bac575bc060ce15bb39 HTTP/1.0" 202 Accepted 2025-12-17 21:20:54,136 - lava_callback - INFO - Received callback for node 69407ba5575bc060ce15bb18 INFO: 172.21.0.1:56788 - "POST /node/69407ba5575bc060ce15bb18 HTTP/1.0" 202 Accepted 2025-12-17 21:20:54,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b82575bc060ce15ba72 HTTP/1.1" 502 150 2025-12-17 21:20:54,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b82575bc060ce15ba72'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:54,886 - lava_callback - INFO - Received callback for node 69407ba6575bc060ce15bb1b INFO: 172.21.0.1:56804 - "POST /node/69407ba6575bc060ce15bb1b HTTP/1.0" 202 Accepted 2025-12-17 21:20:55,724 - lava_callback - INFO - Received callback for node 69407ba6575bc060ce15bb1e INFO: 172.21.0.1:56808 - "POST /node/69407ba6575bc060ce15bb1e HTTP/1.0" 202 Accepted 2025-12-17 21:20:55,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b8d575bc060ce15bad9 2025-12-17 21:20:55,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b8d575bc060ce15bad9 HTTP/1.1" 502 150 2025-12-17 21:20:55,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b8d575bc060ce15bad9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:55,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b99575bc060ce15baef 2025-12-17 21:20:55,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b99575bc060ce15baef HTTP/1.1" 502 150 2025-12-17 21:20:55,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b99575bc060ce15baef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:56,505 - lava_callback - INFO - Received callback for node 69407ba9575bc060ce15bb2b INFO: 172.21.0.1:56820 - "POST /node/69407ba9575bc060ce15bb2b HTTP/1.0" 202 Accepted INFO: 172.21.0.1:56828 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:20:56,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b9a575bc060ce15baf6 2025-12-17 21:20:56,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9a575bc060ce15baf6 HTTP/1.1" 502 150 2025-12-17 21:20:56,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b9a575bc060ce15baf6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:57,348 - lava_callback - INFO - Received callback for node 69407bab575bc060ce15bb32 INFO: 172.21.0.1:56842 - "POST /node/69407bab575bc060ce15bb32 HTTP/1.0" 202 Accepted 2025-12-17 21:20:57,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b75575bc060ce15ba65 2025-12-17 21:20:57,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b75575bc060ce15ba65 HTTP/1.1" 502 150 2025-12-17 21:20:57,903 - lava_callback - ERROR - Error processing callback for node 69407b75575bc060ce15ba65: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b75575bc060ce15ba65 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b75575bc060ce15ba65 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b75575bc060ce15ba65 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:57,905 - lava_callback - INFO - Starting async job submit for node 69407b9c575bc060ce15bafb 2025-12-17 21:20:57,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:57,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9c575bc060ce15bafb HTTP/1.1" 502 150 2025-12-17 21:20:57,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b9c575bc060ce15bafb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:57,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b9c575bc060ce15bafb 2025-12-17 21:20:57,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9c575bc060ce15bafb HTTP/1.1" 502 150 2025-12-17 21:20:57,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b9c575bc060ce15bafb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:58,096 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b8e575bc060ce15badc 2025-12-17 21:20:58,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b8e575bc060ce15badc HTTP/1.1" 502 150 2025-12-17 21:20:58,097 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b8e575bc060ce15badc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:58,113 - lava_callback - INFO - Received callback for node 69407bad575bc060ce15bb3e INFO: 172.21.0.1:56854 - "POST /node/69407bad575bc060ce15bb3e HTTP/1.0" 202 Accepted 2025-12-17 21:20:58,839 - lava_callback - INFO - Received callback for node 69407baf575bc060ce15bb46 INFO: 172.21.0.1:56864 - "POST /node/69407baf575bc060ce15bb46 HTTP/1.0" 202 Accepted 2025-12-17 21:20:58,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c63575bc060ce15f57a 2025-12-17 21:20:58,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c63575bc060ce15f57a HTTP/1.1" 502 150 2025-12-17 21:20:58,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c63575bc060ce15f57a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:59,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b76575bc060ce15ba68 2025-12-17 21:20:59,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b76575bc060ce15ba68 HTTP/1.1" 502 150 2025-12-17 21:20:59,568 - lava_callback - ERROR - Error processing callback for node 69407b76575bc060ce15ba68: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b76575bc060ce15ba68 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b76575bc060ce15ba68 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b76575bc060ce15ba68 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:20:59,570 - lava_callback - INFO - Starting async job submit for node 69407b9d575bc060ce15bb06 2025-12-17 21:20:59,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:20:59,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9d575bc060ce15bb06 HTTP/1.1" 502 150 2025-12-17 21:20:59,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b9d575bc060ce15bb06'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:59,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b9d575bc060ce15bb06 2025-12-17 21:20:59,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9d575bc060ce15bb06 HTTP/1.1" 502 150 2025-12-17 21:20:59,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b9d575bc060ce15bb06'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:20:59,624 - lava_callback - INFO - Received callback for node 69407bb1575bc060ce15bb4b INFO: 172.21.0.1:56872 - "POST /node/69407bb1575bc060ce15bb4b HTTP/1.0" 202 Accepted 2025-12-17 21:20:59,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b9c575bc060ce15bafb 2025-12-17 21:20:59,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9c575bc060ce15bafb HTTP/1.1" 502 150 2025-12-17 21:20:59,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b9c575bc060ce15bafb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:00,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba4575bc060ce15bb0f 2025-12-17 21:21:00,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba4575bc060ce15bb0f HTTP/1.1" 502 150 2025-12-17 21:21:00,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba4575bc060ce15bb0f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:00,696 - lava_callback - INFO - Received callback for node 69407bb2575bc060ce15bb4e INFO: 172.21.0.1:40252 - "POST /node/69407bb2575bc060ce15bb4e HTTP/1.0" 202 Accepted INFO: 172.21.0.1:40256 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:21:01,435 - lava_callback - INFO - Received callback for node 69407bb3575bc060ce15bb51 INFO: 172.21.0.1:40262 - "POST /node/69407bb3575bc060ce15bb51 HTTP/1.0" 202 Accepted 2025-12-17 21:21:01,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b78575bc060ce15ba6b 2025-12-17 21:21:01,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b78575bc060ce15ba6b HTTP/1.1" 502 150 2025-12-17 21:21:01,517 - lava_callback - ERROR - Error processing callback for node 69407b78575bc060ce15ba6b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b78575bc060ce15ba6b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b78575bc060ce15ba6b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b78575bc060ce15ba6b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:01,519 - lava_callback - INFO - Starting async job submit for node 69407ba1575bc060ce15bb09 2025-12-17 21:21:01,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:01,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba1575bc060ce15bb09 HTTP/1.1" 502 150 2025-12-17 21:21:01,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba1575bc060ce15bb09'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:01,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba1575bc060ce15bb09 2025-12-17 21:21:01,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba1575bc060ce15bb09 HTTP/1.1" 502 150 2025-12-17 21:21:01,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba1575bc060ce15bb09'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:01,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b9d575bc060ce15bb06 2025-12-17 21:21:01,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9d575bc060ce15bb06 HTTP/1.1" 502 150 2025-12-17 21:21:01,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b9d575bc060ce15bb06'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:02,167 - lava_callback - INFO - Received callback for node 69407bb4575bc060ce15bb54 INFO: 172.21.0.1:40264 - "POST /node/69407bb4575bc060ce15bb54 HTTP/1.0" 202 Accepted 2025-12-17 21:21:02,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b82575bc060ce15ba72 2025-12-17 21:21:02,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b82575bc060ce15ba72 HTTP/1.1" 502 150 2025-12-17 21:21:02,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b82575bc060ce15ba72'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:02,929 - lava_callback - INFO - Received callback for node 69407bb6575bc060ce15bb57 INFO: 172.21.0.1:40272 - "POST /node/69407bb6575bc060ce15bb57 HTTP/1.0" 202 Accepted 2025-12-17 21:21:03,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba1575bc060ce15bb09 2025-12-17 21:21:03,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba1575bc060ce15bb09 HTTP/1.1" 502 150 2025-12-17 21:21:03,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba1575bc060ce15bb09'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:03,664 - lava_callback - INFO - Received callback for node 69407bb7575bc060ce15bb5e INFO: 172.21.0.1:40284 - "POST /node/69407bb7575bc060ce15bb5e HTTP/1.0" 202 Accepted 2025-12-17 21:21:03,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b83575bc060ce15ba78 2025-12-17 21:21:03,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b83575bc060ce15ba78 HTTP/1.1" 502 150 2025-12-17 21:21:03,756 - lava_callback - ERROR - Error processing callback for node 69407b83575bc060ce15ba78: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b83575bc060ce15ba78 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b83575bc060ce15ba78 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b83575bc060ce15ba78 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:03,758 - lava_callback - INFO - Starting async job submit for node 69407ba7575bc060ce15bb21 2025-12-17 21:21:03,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:03,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba7575bc060ce15bb21 HTTP/1.1" 502 150 2025-12-17 21:21:03,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba7575bc060ce15bb21'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:03,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba7575bc060ce15bb21 2025-12-17 21:21:03,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba7575bc060ce15bb21 HTTP/1.1" 502 150 2025-12-17 21:21:03,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba7575bc060ce15bb21'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:03,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b99575bc060ce15baef 2025-12-17 21:21:03,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b99575bc060ce15baef HTTP/1.1" 502 150 2025-12-17 21:21:03,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b99575bc060ce15baef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:03,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b84575bc060ce15ba7e 2025-12-17 21:21:03,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b84575bc060ce15ba7e HTTP/1.1" 502 150 2025-12-17 21:21:03,895 - lava_callback - ERROR - Error processing callback for node 69407b84575bc060ce15ba7e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b84575bc060ce15ba7e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b84575bc060ce15ba7e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b84575bc060ce15ba7e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:03,896 - lava_callback - INFO - Starting async job submit for node 69407ba8575bc060ce15bb24 2025-12-17 21:21:03,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:03,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba8575bc060ce15bb24 HTTP/1.1" 502 150 2025-12-17 21:21:03,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba8575bc060ce15bb24'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:03,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba8575bc060ce15bb24 2025-12-17 21:21:03,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba8575bc060ce15bb24 HTTP/1.1" 502 150 2025-12-17 21:21:03,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba8575bc060ce15bb24'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:03,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b9c575bc060ce15bafb 2025-12-17 21:21:03,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9c575bc060ce15bafb HTTP/1.1" 502 150 2025-12-17 21:21:03,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b9c575bc060ce15bafb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:04,454 - lava_callback - INFO - Received callback for node 69407bb9575bc060ce15bb61 INFO: 172.21.0.1:40298 - "POST /node/69407bb9575bc060ce15bb61 HTTP/1.0" 202 Accepted 2025-12-17 21:21:04,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b9a575bc060ce15baf6 2025-12-17 21:21:04,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9a575bc060ce15baf6 HTTP/1.1" 502 150 2025-12-17 21:21:04,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b9a575bc060ce15baf6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:05,194 - lava_callback - INFO - Received callback for node 69407bba575bc060ce15bb64 INFO: 172.21.0.1:40312 - "POST /node/69407bba575bc060ce15bb64 HTTP/1.0" 202 Accepted 2025-12-17 21:21:05,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b9d575bc060ce15bb06 2025-12-17 21:21:05,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9d575bc060ce15bb06 HTTP/1.1" 502 150 2025-12-17 21:21:05,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b9d575bc060ce15bb06'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:05,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba7575bc060ce15bb21 2025-12-17 21:21:05,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba7575bc060ce15bb21 HTTP/1.1" 502 150 2025-12-17 21:21:05,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba7575bc060ce15bb21'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:05,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba8575bc060ce15bb24 2025-12-17 21:21:05,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba8575bc060ce15bb24 HTTP/1.1" 502 150 2025-12-17 21:21:05,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba8575bc060ce15bb24'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:05,938 - lava_callback - INFO - Received callback for node 69407bbb575bc060ce15bb67 INFO: 172.21.0.1:40320 - "POST /node/69407bbb575bc060ce15bb67 HTTP/1.0" 202 Accepted 2025-12-17 21:21:06,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b86575bc060ce15ba85 2025-12-17 21:21:06,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b86575bc060ce15ba85 HTTP/1.1" 502 150 2025-12-17 21:21:06,020 - lava_callback - ERROR - Error processing callback for node 69407b86575bc060ce15ba85: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b86575bc060ce15ba85 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b86575bc060ce15ba85 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b86575bc060ce15ba85 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:06,022 - lava_callback - INFO - Starting async job submit for node 69407ba3575bc060ce15bb0c 2025-12-17 21:21:06,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:06,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba3575bc060ce15bb0c HTTP/1.1" 502 150 2025-12-17 21:21:06,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba3575bc060ce15bb0c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:06,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba3575bc060ce15bb0c 2025-12-17 21:21:06,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba3575bc060ce15bb0c HTTP/1.1" 502 150 2025-12-17 21:21:06,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba3575bc060ce15bb0c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40322 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:21:06,669 - lava_callback - INFO - Received callback for node 69407bbc575bc060ce15bb6b INFO: 172.21.0.1:40326 - "POST /node/69407bbc575bc060ce15bb6b HTTP/1.0" 202 Accepted 2025-12-17 21:21:07,343 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b87575bc060ce15ba8e 2025-12-17 21:21:07,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b87575bc060ce15ba8e HTTP/1.1" 502 150 2025-12-17 21:21:07,345 - lava_callback - ERROR - Error processing callback for node 69407b87575bc060ce15ba8e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b87575bc060ce15ba8e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b87575bc060ce15ba8e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b87575bc060ce15ba8e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:07,347 - lava_callback - INFO - Starting async job submit for node 69407ba5575bc060ce15bb15 2025-12-17 21:21:07,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:07,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb15 HTTP/1.1" 502 150 2025-12-17 21:21:07,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba5575bc060ce15bb15'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:07,355 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba5575bc060ce15bb15 2025-12-17 21:21:07,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb15 HTTP/1.1" 502 150 2025-12-17 21:21:07,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba5575bc060ce15bb15'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:07,406 - lava_callback - INFO - Received callback for node 69407bbe575bc060ce15bb71 INFO: 172.21.0.1:40332 - "POST /node/69407bbe575bc060ce15bb71 HTTP/1.0" 202 Accepted 2025-12-17 21:21:07,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba1575bc060ce15bb09 2025-12-17 21:21:07,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba1575bc060ce15bb09 HTTP/1.1" 502 150 2025-12-17 21:21:07,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba1575bc060ce15bb09'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:08,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b88575bc060ce15ba93 2025-12-17 21:21:08,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b88575bc060ce15ba93 HTTP/1.1" 502 150 2025-12-17 21:21:08,027 - lava_callback - ERROR - Error processing callback for node 69407b88575bc060ce15ba93: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b88575bc060ce15ba93 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b88575bc060ce15ba93 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b88575bc060ce15ba93 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:08,030 - lava_callback - INFO - Starting async job submit for node 69407bac575bc060ce15bb39 2025-12-17 21:21:08,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:08,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba3575bc060ce15bb0c 2025-12-17 21:21:08,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba3575bc060ce15bb0c HTTP/1.1" 502 150 2025-12-17 21:21:08,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba3575bc060ce15bb0c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:08,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bac575bc060ce15bb39 HTTP/1.1" 502 150 2025-12-17 21:21:08,061 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bac575bc060ce15bb39'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:08,061 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bac575bc060ce15bb39 2025-12-17 21:21:08,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bac575bc060ce15bb39 HTTP/1.1" 502 150 2025-12-17 21:21:08,062 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bac575bc060ce15bb39'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:08,169 - lava_callback - INFO - Received callback for node 69407bbf575bc060ce15bb77 INFO: 172.21.0.1:40340 - "POST /node/69407bbf575bc060ce15bb77 HTTP/1.0" 202 Accepted 2025-12-17 21:21:08,919 - lava_callback - INFO - Received callback for node 69407bc0575bc060ce15bb7e INFO: 172.21.0.1:40346 - "POST /node/69407bc0575bc060ce15bb7e HTTP/1.0" 202 Accepted 2025-12-17 21:21:09,356 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba5575bc060ce15bb15 2025-12-17 21:21:09,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb15 HTTP/1.1" 502 150 2025-12-17 21:21:09,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba5575bc060ce15bb15'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:09,651 - lava_callback - INFO - Received callback for node 69407bc2575bc060ce15bb83 INFO: 172.21.0.1:40352 - "POST /node/69407bc2575bc060ce15bb83 HTTP/1.0" 202 Accepted 2025-12-17 21:21:09,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba7575bc060ce15bb21 2025-12-17 21:21:09,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba7575bc060ce15bb21 HTTP/1.1" 502 150 2025-12-17 21:21:09,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba7575bc060ce15bb21'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:09,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba8575bc060ce15bb24 2025-12-17 21:21:09,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba8575bc060ce15bb24 HTTP/1.1" 502 150 2025-12-17 21:21:09,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba8575bc060ce15bb24'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:10,062 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bac575bc060ce15bb39 2025-12-17 21:21:10,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bac575bc060ce15bb39 HTTP/1.1" 502 150 2025-12-17 21:21:10,064 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bac575bc060ce15bb39'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:10,390 - lava_callback - INFO - Received callback for node 69407bc3575bc060ce15bb88 INFO: 172.21.0.1:45520 - "POST /node/69407bc3575bc060ce15bb88 HTTP/1.0" 202 Accepted 2025-12-17 21:21:11,158 - lava_callback - INFO - Received callback for node 69407bc4575bc060ce15bb8b INFO: 172.21.0.1:45536 - "POST /node/69407bc4575bc060ce15bb8b HTTP/1.0" 202 Accepted 2025-12-17 21:21:11,358 - lava_callback - INFO - Received callback for node 69407ba5575bc060ce15bb12 INFO: 172.21.0.1:45550 - "POST /node/69407ba5575bc060ce15bb12 HTTP/1.0" 202 Accepted 2025-12-17 21:21:11,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b8d575bc060ce15bad9 2025-12-17 21:21:11,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b8d575bc060ce15bad9 HTTP/1.1" 502 150 2025-12-17 21:21:11,808 - lava_callback - ERROR - Error processing callback for node 69407b8d575bc060ce15bad9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b8d575bc060ce15bad9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b8d575bc060ce15bad9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b8d575bc060ce15bad9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:11,810 - lava_callback - INFO - Starting async job submit for node 69407ba5575bc060ce15bb18 2025-12-17 21:21:11,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:11,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb18 HTTP/1.1" 502 150 2025-12-17 21:21:11,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba5575bc060ce15bb18'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:11,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba5575bc060ce15bb18 2025-12-17 21:21:11,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb18 HTTP/1.1" 502 150 2025-12-17 21:21:11,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba5575bc060ce15bb18'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:11,887 - lava_callback - INFO - Received callback for node 69407bc6575bc060ce15bb8e INFO: 172.21.0.1:45566 - "POST /node/69407bc6575bc060ce15bb8e HTTP/1.0" 202 Accepted 2025-12-17 21:21:11,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b9c575bc060ce15bafb 2025-12-17 21:21:11,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9c575bc060ce15bafb HTTP/1.1" 502 150 2025-12-17 21:21:11,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b9c575bc060ce15bafb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:12,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba3575bc060ce15bb0c 2025-12-17 21:21:12,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba3575bc060ce15bb0c HTTP/1.1" 502 150 2025-12-17 21:21:12,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba3575bc060ce15bb0c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:12,634 - lava_callback - INFO - Received callback for node 69407bc7575bc060ce15bb95 INFO: 172.21.0.1:45568 - "POST /node/69407bc7575bc060ce15bb95 HTTP/1.0" 202 Accepted 2025-12-17 21:21:13,357 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba5575bc060ce15bb15 2025-12-17 21:21:13,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb15 HTTP/1.1" 502 150 2025-12-17 21:21:13,358 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba5575bc060ce15bb15'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:13,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b9d575bc060ce15bb06 2025-12-17 21:21:13,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9d575bc060ce15bb06 HTTP/1.1" 502 150 2025-12-17 21:21:13,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b9d575bc060ce15bb06'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:13,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba5575bc060ce15bb18 2025-12-17 21:21:13,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb18 HTTP/1.1" 502 150 2025-12-17 21:21:13,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba5575bc060ce15bb18'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:14,064 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bac575bc060ce15bb39 2025-12-17 21:21:14,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bac575bc060ce15bb39 HTTP/1.1" 502 150 2025-12-17 21:21:14,065 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bac575bc060ce15bb39'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:14,098 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b8e575bc060ce15badc 2025-12-17 21:21:14,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b8e575bc060ce15badc HTTP/1.1" 502 150 2025-12-17 21:21:14,099 - lava_callback - ERROR - Error processing callback for node 69407b8e575bc060ce15badc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b8e575bc060ce15badc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b8e575bc060ce15badc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b8e575bc060ce15badc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:14,101 - lava_callback - INFO - Starting async job submit for node 69407ba6575bc060ce15bb1b 2025-12-17 21:21:14,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:14,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba6575bc060ce15bb1b HTTP/1.1" 502 150 2025-12-17 21:21:14,107 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba6575bc060ce15bb1b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:14,107 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba6575bc060ce15bb1b 2025-12-17 21:21:14,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba6575bc060ce15bb1b HTTP/1.1" 502 150 2025-12-17 21:21:14,108 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba6575bc060ce15bb1b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:14,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c63575bc060ce15f57a 2025-12-17 21:21:14,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c63575bc060ce15f57a HTTP/1.1" 502 150 2025-12-17 21:21:14,935 - lava_callback - ERROR - Error processing callback for node 69408c63575bc060ce15f57a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c63575bc060ce15f57a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c63575bc060ce15f57a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c63575bc060ce15f57a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:14,938 - lava_callback - INFO - Starting async job submit for node 69407ba6575bc060ce15bb1e 2025-12-17 21:21:14,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:14,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba6575bc060ce15bb1e HTTP/1.1" 502 150 2025-12-17 21:21:14,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba6575bc060ce15bb1e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:14,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba6575bc060ce15bb1e 2025-12-17 21:21:14,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba6575bc060ce15bb1e HTTP/1.1" 502 150 2025-12-17 21:21:14,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba6575bc060ce15bb1e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45580 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:21:15,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba1575bc060ce15bb09 2025-12-17 21:21:15,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba1575bc060ce15bb09 HTTP/1.1" 502 150 2025-12-17 21:21:15,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba1575bc060ce15bb09'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:16,108 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba6575bc060ce15bb1b 2025-12-17 21:21:16,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba6575bc060ce15bb1b HTTP/1.1" 502 150 2025-12-17 21:21:16,109 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba6575bc060ce15bb1b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45592 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:21:16,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba4575bc060ce15bb0f 2025-12-17 21:21:16,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba4575bc060ce15bb0f HTTP/1.1" 502 150 2025-12-17 21:21:16,648 - lava_callback - ERROR - Error processing callback for node 69407ba4575bc060ce15bb0f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba4575bc060ce15bb0f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba4575bc060ce15bb0f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba4575bc060ce15bb0f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:16,650 - lava_callback - INFO - Starting async job submit for node 69407ba9575bc060ce15bb2b 2025-12-17 21:21:16,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:16,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba9575bc060ce15bb2b HTTP/1.1" 502 150 2025-12-17 21:21:16,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba9575bc060ce15bb2b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:16,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba9575bc060ce15bb2b 2025-12-17 21:21:16,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba9575bc060ce15bb2b HTTP/1.1" 502 150 2025-12-17 21:21:16,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba9575bc060ce15bb2b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:16,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba6575bc060ce15bb1e 2025-12-17 21:21:16,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba6575bc060ce15bb1e HTTP/1.1" 502 150 2025-12-17 21:21:16,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba6575bc060ce15bb1e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:17,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba7575bc060ce15bb21 2025-12-17 21:21:17,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba7575bc060ce15bb21 HTTP/1.1" 502 150 2025-12-17 21:21:17,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba7575bc060ce15bb21'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:17,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba5575bc060ce15bb18 2025-12-17 21:21:17,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb18 HTTP/1.1" 502 150 2025-12-17 21:21:17,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba5575bc060ce15bb18'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:17,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba8575bc060ce15bb24 2025-12-17 21:21:17,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba8575bc060ce15bb24 HTTP/1.1" 502 150 2025-12-17 21:21:17,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba8575bc060ce15bb24'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:18,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b82575bc060ce15ba72 2025-12-17 21:21:18,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b82575bc060ce15ba72 HTTP/1.1" 502 150 2025-12-17 21:21:18,451 - lava_callback - ERROR - Error processing callback for node 69407b82575bc060ce15ba72: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b82575bc060ce15ba72 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b82575bc060ce15ba72 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b82575bc060ce15ba72 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:18,453 - lava_callback - INFO - Starting async job submit for node 69407bab575bc060ce15bb32 2025-12-17 21:21:18,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:18,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bab575bc060ce15bb32 HTTP/1.1" 502 150 2025-12-17 21:21:18,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bab575bc060ce15bb32'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:18,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bab575bc060ce15bb32 2025-12-17 21:21:18,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bab575bc060ce15bb32 HTTP/1.1" 502 150 2025-12-17 21:21:18,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bab575bc060ce15bb32'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:18,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba9575bc060ce15bb2b 2025-12-17 21:21:18,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba9575bc060ce15bb2b HTTP/1.1" 502 150 2025-12-17 21:21:18,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba9575bc060ce15bb2b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:19,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b99575bc060ce15baef 2025-12-17 21:21:19,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b99575bc060ce15baef HTTP/1.1" 502 150 2025-12-17 21:21:19,856 - lava_callback - ERROR - Error processing callback for node 69407b99575bc060ce15baef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b99575bc060ce15baef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b99575bc060ce15baef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b99575bc060ce15baef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:19,858 - lava_callback - INFO - Starting async job submit for node 69407bad575bc060ce15bb3e 2025-12-17 21:21:19,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:19,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bad575bc060ce15bb3e HTTP/1.1" 502 150 2025-12-17 21:21:19,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bad575bc060ce15bb3e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:19,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bad575bc060ce15bb3e 2025-12-17 21:21:19,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bad575bc060ce15bb3e HTTP/1.1" 502 150 2025-12-17 21:21:19,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bad575bc060ce15bb3e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:20,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba3575bc060ce15bb0c 2025-12-17 21:21:20,110 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba6575bc060ce15bb1b 2025-12-17 21:21:20,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba6575bc060ce15bb1b HTTP/1.1" 502 150 2025-12-17 21:21:20,111 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba6575bc060ce15bb1b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:20,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bab575bc060ce15bb32 2025-12-17 21:21:20,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bab575bc060ce15bb32 HTTP/1.1" 502 150 2025-12-17 21:21:20,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bab575bc060ce15bb32'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:20,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b94575bc060ce15badf HTTP/1.1" 502 150 2025-12-17 21:21:20,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b94575bc060ce15badf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:20,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b9a575bc060ce15baf6 2025-12-17 21:21:20,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9a575bc060ce15baf6 HTTP/1.1" 502 150 2025-12-17 21:21:20,745 - lava_callback - ERROR - Error processing callback for node 69407b9a575bc060ce15baf6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b9a575bc060ce15baf6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b9a575bc060ce15baf6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b9a575bc060ce15baf6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:20,747 - lava_callback - INFO - Starting async job submit for node 69407baf575bc060ce15bb46 2025-12-17 21:21:20,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:20,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407baf575bc060ce15bb46 HTTP/1.1" 502 150 2025-12-17 21:21:20,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407baf575bc060ce15bb46'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:20,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407baf575bc060ce15bb46 2025-12-17 21:21:20,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407baf575bc060ce15bb46 HTTP/1.1" 502 150 2025-12-17 21:21:20,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407baf575bc060ce15bb46'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:20,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba6575bc060ce15bb1e 2025-12-17 21:21:20,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba6575bc060ce15bb1e HTTP/1.1" 502 150 2025-12-17 21:21:20,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba6575bc060ce15bb1e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:21,359 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba5575bc060ce15bb15 2025-12-17 21:21:21,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb15 HTTP/1.1" 502 150 2025-12-17 21:21:21,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba5575bc060ce15bb15'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:21,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bad575bc060ce15bb3e 2025-12-17 21:21:21,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bad575bc060ce15bb3e HTTP/1.1" 502 150 2025-12-17 21:21:21,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bad575bc060ce15bb3e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:22,065 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bac575bc060ce15bb39 2025-12-17 21:21:22,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bac575bc060ce15bb39 HTTP/1.1" 502 150 2025-12-17 21:21:22,067 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bac575bc060ce15bb39'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:22,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba9575bc060ce15bb2b 2025-12-17 21:21:22,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba9575bc060ce15bb2b HTTP/1.1" 502 150 2025-12-17 21:21:22,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba9575bc060ce15bb2b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:22,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407baf575bc060ce15bb46 2025-12-17 21:21:22,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407baf575bc060ce15bb46 HTTP/1.1" 502 150 2025-12-17 21:21:22,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407baf575bc060ce15bb46'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:24,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bab575bc060ce15bb32 2025-12-17 21:21:24,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bab575bc060ce15bb32 HTTP/1.1" 502 150 2025-12-17 21:21:24,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bab575bc060ce15bb32'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:24,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b94575bc060ce15badf 2025-12-17 21:21:24,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b94575bc060ce15badf HTTP/1.1" 502 150 2025-12-17 21:21:24,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b94575bc060ce15badf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:25,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba5575bc060ce15bb18 2025-12-17 21:21:25,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb18 HTTP/1.1" 502 150 2025-12-17 21:21:25,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba5575bc060ce15bb18'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:25,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bad575bc060ce15bb3e 2025-12-17 21:21:25,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bad575bc060ce15bb3e HTTP/1.1" 502 150 2025-12-17 21:21:25,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bad575bc060ce15bb3e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37430 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:21:26,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407baf575bc060ce15bb46 2025-12-17 21:21:26,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407baf575bc060ce15bb46 HTTP/1.1" 502 150 2025-12-17 21:21:26,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407baf575bc060ce15bb46'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:27,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b9c575bc060ce15bafb 2025-12-17 21:21:27,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9c575bc060ce15bafb HTTP/1.1" 502 150 2025-12-17 21:21:27,944 - lava_callback - ERROR - Error processing callback for node 69407b9c575bc060ce15bafb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b9c575bc060ce15bafb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b9c575bc060ce15bafb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b9c575bc060ce15bafb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:27,946 - lava_callback - INFO - Starting async job submit for node 69407bb1575bc060ce15bb4b 2025-12-17 21:21:27,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:27,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb1575bc060ce15bb4b HTTP/1.1" 502 150 2025-12-17 21:21:27,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb1575bc060ce15bb4b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:27,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb1575bc060ce15bb4b 2025-12-17 21:21:27,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb1575bc060ce15bb4b HTTP/1.1" 502 150 2025-12-17 21:21:27,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb1575bc060ce15bb4b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:28,111 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba6575bc060ce15bb1b 2025-12-17 21:21:28,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba6575bc060ce15bb1b HTTP/1.1" 502 150 2025-12-17 21:21:28,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba6575bc060ce15bb1b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:28,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba6575bc060ce15bb1e 2025-12-17 21:21:28,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba6575bc060ce15bb1e HTTP/1.1" 502 150 2025-12-17 21:21:28,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba6575bc060ce15bb1e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:29,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b9d575bc060ce15bb06 2025-12-17 21:21:29,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b9d575bc060ce15bb06 HTTP/1.1" 502 150 2025-12-17 21:21:29,598 - lava_callback - ERROR - Error processing callback for node 69407b9d575bc060ce15bb06: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b9d575bc060ce15bb06 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b9d575bc060ce15bb06 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b9d575bc060ce15bb06 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:29,600 - lava_callback - INFO - Starting async job submit for node 69407bb2575bc060ce15bb4e 2025-12-17 21:21:29,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:29,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb2575bc060ce15bb4e HTTP/1.1" 502 150 2025-12-17 21:21:29,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb2575bc060ce15bb4e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:29,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb2575bc060ce15bb4e 2025-12-17 21:21:29,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb2575bc060ce15bb4e HTTP/1.1" 502 150 2025-12-17 21:21:29,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb2575bc060ce15bb4e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:29,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb1575bc060ce15bb4b 2025-12-17 21:21:29,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb1575bc060ce15bb4b HTTP/1.1" 502 150 2025-12-17 21:21:29,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb1575bc060ce15bb4b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:30,503 - lava_callback - INFO - Received callback for node 69407bd4575bc060ce15bbea INFO: 172.21.0.1:48142 - "POST /node/69407bd4575bc060ce15bbea HTTP/1.0" 202 Accepted 2025-12-17 21:21:30,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba9575bc060ce15bb2b 2025-12-17 21:21:30,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba9575bc060ce15bb2b HTTP/1.1" 502 150 2025-12-17 21:21:30,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba9575bc060ce15bb2b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:31,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba1575bc060ce15bb09 2025-12-17 21:21:31,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb2575bc060ce15bb4e 2025-12-17 21:21:31,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb2575bc060ce15bb4e HTTP/1.1" 502 150 2025-12-17 21:21:31,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb2575bc060ce15bb4e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:32,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bab575bc060ce15bb32 2025-12-17 21:21:32,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bab575bc060ce15bb32 HTTP/1.1" 502 150 2025-12-17 21:21:32,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bab575bc060ce15bb32'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:32,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b94575bc060ce15badf 2025-12-17 21:21:32,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b94575bc060ce15badf HTTP/1.1" 502 150 2025-12-17 21:21:32,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b94575bc060ce15badf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:33,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba7575bc060ce15bb21 2025-12-17 21:21:33,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba7575bc060ce15bb21 HTTP/1.1" 502 150 2025-12-17 21:21:33,790 - lava_callback - ERROR - Error processing callback for node 69407ba7575bc060ce15bb21: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba7575bc060ce15bb21 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba7575bc060ce15bb21 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba7575bc060ce15bb21 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:33,792 - lava_callback - INFO - Starting async job submit for node 69407bb3575bc060ce15bb51 2025-12-17 21:21:33,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:33,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb3575bc060ce15bb51 HTTP/1.1" 502 150 2025-12-17 21:21:33,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb3575bc060ce15bb51'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:33,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb3575bc060ce15bb51 2025-12-17 21:21:33,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb3575bc060ce15bb51 HTTP/1.1" 502 150 2025-12-17 21:21:33,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb3575bc060ce15bb51'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:33,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bad575bc060ce15bb3e 2025-12-17 21:21:33,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bad575bc060ce15bb3e HTTP/1.1" 502 150 2025-12-17 21:21:33,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bad575bc060ce15bb3e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:33,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba8575bc060ce15bb24 2025-12-17 21:21:33,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba8575bc060ce15bb24 HTTP/1.1" 502 150 2025-12-17 21:21:33,912 - lava_callback - ERROR - Error processing callback for node 69407ba8575bc060ce15bb24: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba8575bc060ce15bb24 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba8575bc060ce15bb24 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba8575bc060ce15bb24 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:33,914 - lava_callback - INFO - Starting async job submit for node 69407bb4575bc060ce15bb54 2025-12-17 21:21:33,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:33,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb4575bc060ce15bb54 HTTP/1.1" 502 150 2025-12-17 21:21:33,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb4575bc060ce15bb54'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:33,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb4575bc060ce15bb54 2025-12-17 21:21:33,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb4575bc060ce15bb54 HTTP/1.1" 502 150 2025-12-17 21:21:33,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb4575bc060ce15bb54'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:33,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb1575bc060ce15bb4b 2025-12-17 21:21:33,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb1575bc060ce15bb4b HTTP/1.1" 502 150 2025-12-17 21:21:33,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb1575bc060ce15bb4b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:34,297 - lava_callback - INFO - Received callback for node 69407bc9575bc060ce15bbc7 INFO: 172.21.0.1:48152 - "POST /node/69407bc9575bc060ce15bbc7 HTTP/1.0" 202 Accepted 2025-12-17 21:21:34,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407baf575bc060ce15bb46 2025-12-17 21:21:34,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407baf575bc060ce15bb46 HTTP/1.1" 502 150 2025-12-17 21:21:34,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407baf575bc060ce15bb46'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:35,046 - lava_callback - INFO - Received callback for node 69407bca575bc060ce15bbcb INFO: 172.21.0.1:48158 - "POST /node/69407bca575bc060ce15bbcb HTTP/1.0" 202 Accepted 2025-12-17 21:21:35,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba3575bc060ce15bb0c HTTP/1.1" 502 150 2025-12-17 21:21:35,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba3575bc060ce15bb0c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:35,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb2575bc060ce15bb4e 2025-12-17 21:21:35,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb2575bc060ce15bb4e HTTP/1.1" 502 150 2025-12-17 21:21:35,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb2575bc060ce15bb4e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:35,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb3575bc060ce15bb51 2025-12-17 21:21:35,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb3575bc060ce15bb51 HTTP/1.1" 502 150 2025-12-17 21:21:35,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb3575bc060ce15bb51'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:35,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb4575bc060ce15bb54 2025-12-17 21:21:35,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb4575bc060ce15bb54 HTTP/1.1" 502 150 2025-12-17 21:21:35,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb4575bc060ce15bb54'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:35,970 - lava_callback - INFO - Received callback for node 69407bcc575bc060ce15bbce INFO: 172.21.0.1:48162 - "POST /node/69407bcc575bc060ce15bbce HTTP/1.0" 202 Accepted INFO: 172.21.0.1:48178 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:21:36,866 - lava_callback - INFO - Received callback for node 69407bcd575bc060ce15bbd1 INFO: 172.21.0.1:48192 - "POST /node/69407bcd575bc060ce15bbd1 HTTP/1.0" 202 Accepted 2025-12-17 21:21:37,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba5575bc060ce15bb15 2025-12-17 21:21:37,621 - lava_callback - INFO - Received callback for node 69407bce575bc060ce15bbd4 INFO: 172.21.0.1:48206 - "POST /node/69407bce575bc060ce15bbd4 HTTP/1.0" 202 Accepted 2025-12-17 21:21:38,067 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bac575bc060ce15bb39 2025-12-17 21:21:38,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bac575bc060ce15bb39 HTTP/1.1" 502 150 2025-12-17 21:21:38,069 - lava_callback - ERROR - Error processing callback for node 69407bac575bc060ce15bb39: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bac575bc060ce15bb39 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bac575bc060ce15bb39 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bac575bc060ce15bb39 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:38,071 - lava_callback - INFO - Starting async job submit for node 69407bb6575bc060ce15bb57 2025-12-17 21:21:38,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:38,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb6575bc060ce15bb57 HTTP/1.1" 502 150 2025-12-17 21:21:38,079 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb6575bc060ce15bb57'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:38,079 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb6575bc060ce15bb57 2025-12-17 21:21:38,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb6575bc060ce15bb57 HTTP/1.1" 502 150 2025-12-17 21:21:38,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb6575bc060ce15bb57'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:38,384 - lava_callback - INFO - Received callback for node 69407bd0575bc060ce15bbd7 INFO: 172.21.0.1:48208 - "POST /node/69407bd0575bc060ce15bbd7 HTTP/1.0" 202 Accepted 2025-12-17 21:21:39,138 - lava_callback - INFO - Received callback for node 69407bd1575bc060ce15bbdc INFO: 172.21.0.1:48222 - "POST /node/69407bd1575bc060ce15bbdc HTTP/1.0" 202 Accepted 2025-12-17 21:21:39,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb3575bc060ce15bb51 2025-12-17 21:21:39,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb3575bc060ce15bb51 HTTP/1.1" 502 150 2025-12-17 21:21:39,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb3575bc060ce15bb51'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:39,897 - lava_callback - INFO - Received callback for node 69407bd2575bc060ce15bbe1 INFO: 172.21.0.1:55064 - "POST /node/69407bd2575bc060ce15bbe1 HTTP/1.0" 202 Accepted 2025-12-17 21:21:39,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb4575bc060ce15bb54 2025-12-17 21:21:39,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb4575bc060ce15bb54 HTTP/1.1" 502 150 2025-12-17 21:21:39,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb4575bc060ce15bb54'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:40,080 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb6575bc060ce15bb57 2025-12-17 21:21:40,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb6575bc060ce15bb57 HTTP/1.1" 502 150 2025-12-17 21:21:40,081 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb6575bc060ce15bb57'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:40,661 - lava_callback - INFO - Received callback for node 69407bd3575bc060ce15bbe5 INFO: 172.21.0.1:55080 - "POST /node/69407bd3575bc060ce15bbe5 HTTP/1.0" 202 Accepted 2025-12-17 21:21:41,420 - lava_callback - INFO - Received callback for node 69407bd6575bc060ce15bbef INFO: 172.21.0.1:55082 - "POST /node/69407bd6575bc060ce15bbef HTTP/1.0" 202 Accepted 2025-12-17 21:21:41,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba5575bc060ce15bb18 2025-12-17 21:21:41,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb18 HTTP/1.1" 502 150 2025-12-17 21:21:41,825 - lava_callback - ERROR - Error processing callback for node 69407ba5575bc060ce15bb18: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba5575bc060ce15bb18 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba5575bc060ce15bb18 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba5575bc060ce15bb18 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:41,827 - lava_callback - INFO - Starting async job submit for node 69407bb7575bc060ce15bb5e 2025-12-17 21:21:41,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:41,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb7575bc060ce15bb5e HTTP/1.1" 502 150 2025-12-17 21:21:41,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb7575bc060ce15bb5e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:41,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb7575bc060ce15bb5e 2025-12-17 21:21:41,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb7575bc060ce15bb5e HTTP/1.1" 502 150 2025-12-17 21:21:41,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb7575bc060ce15bb5e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:41,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb1575bc060ce15bb4b 2025-12-17 21:21:41,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb1575bc060ce15bb4b HTTP/1.1" 502 150 2025-12-17 21:21:41,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb1575bc060ce15bb4b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:42,161 - lava_callback - INFO - Received callback for node 69407bd8575bc060ce15bbf3 INFO: 172.21.0.1:55098 - "POST /node/69407bd8575bc060ce15bbf3 HTTP/1.0" 202 Accepted 2025-12-17 21:21:42,928 - lava_callback - INFO - Received callback for node 69407bd9575bc060ce15bbf6 INFO: 172.21.0.1:55114 - "POST /node/69407bd9575bc060ce15bbf6 HTTP/1.0" 202 Accepted 2025-12-17 21:21:43,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb2575bc060ce15bb4e 2025-12-17 21:21:43,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb2575bc060ce15bb4e HTTP/1.1" 502 150 2025-12-17 21:21:43,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb2575bc060ce15bb4e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:43,681 - lava_callback - INFO - Received callback for node 69407bda575bc060ce15bbf9 INFO: 172.21.0.1:55120 - "POST /node/69407bda575bc060ce15bbf9 HTTP/1.0" 202 Accepted 2025-12-17 21:21:43,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb7575bc060ce15bb5e 2025-12-17 21:21:43,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb7575bc060ce15bb5e HTTP/1.1" 502 150 2025-12-17 21:21:43,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb7575bc060ce15bb5e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:44,082 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb6575bc060ce15bb57 2025-12-17 21:21:44,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb6575bc060ce15bb57 HTTP/1.1" 502 150 2025-12-17 21:21:44,083 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb6575bc060ce15bb57'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:44,113 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba6575bc060ce15bb1b 2025-12-17 21:21:44,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba6575bc060ce15bb1b HTTP/1.1" 502 150 2025-12-17 21:21:44,114 - lava_callback - ERROR - Error processing callback for node 69407ba6575bc060ce15bb1b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba6575bc060ce15bb1b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba6575bc060ce15bb1b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba6575bc060ce15bb1b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:44,117 - lava_callback - INFO - Starting async job submit for node 69407bb9575bc060ce15bb61 2025-12-17 21:21:44,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:44,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb9575bc060ce15bb61 HTTP/1.1" 502 150 2025-12-17 21:21:44,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb9575bc060ce15bb61'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:44,124 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb9575bc060ce15bb61 2025-12-17 21:21:44,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb9575bc060ce15bb61 HTTP/1.1" 502 150 2025-12-17 21:21:44,124 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb9575bc060ce15bb61'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:44,421 - lava_callback - INFO - Received callback for node 69407bdb575bc060ce15bc00 INFO: 172.21.0.1:55130 - "POST /node/69407bdb575bc060ce15bc00 HTTP/1.0" 202 Accepted 2025-12-17 21:21:44,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba6575bc060ce15bb1e 2025-12-17 21:21:44,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba6575bc060ce15bb1e HTTP/1.1" 502 150 2025-12-17 21:21:44,952 - lava_callback - ERROR - Error processing callback for node 69407ba6575bc060ce15bb1e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba6575bc060ce15bb1e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba6575bc060ce15bb1e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba6575bc060ce15bb1e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:44,954 - lava_callback - INFO - Starting async job submit for node 69407bba575bc060ce15bb64 2025-12-17 21:21:44,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:45,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bba575bc060ce15bb64 HTTP/1.1" 502 150 2025-12-17 21:21:45,131 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bba575bc060ce15bb64'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:45,131 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bba575bc060ce15bb64 2025-12-17 21:21:45,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bba575bc060ce15bb64 HTTP/1.1" 502 150 2025-12-17 21:21:45,132 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bba575bc060ce15bb64'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:45,225 - lava_callback - INFO - Received callback for node 69407bdf575bc060ce15bc09 INFO: 172.21.0.1:55146 - "POST /node/69407bdf575bc060ce15bc09 HTTP/1.0" 202 Accepted 2025-12-17 21:21:45,996 - lava_callback - INFO - Received callback for node 69407be0575bc060ce15bc0c INFO: 172.21.0.1:55162 - "POST /node/69407be0575bc060ce15bc0c HTTP/1.0" 202 Accepted 2025-12-17 21:21:46,125 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb9575bc060ce15bb61 INFO: 172.21.0.1:55174 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:21:46,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba9575bc060ce15bb2b 2025-12-17 21:21:46,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba9575bc060ce15bb2b HTTP/1.1" 502 150 2025-12-17 21:21:46,665 - lava_callback - ERROR - Error processing callback for node 69407ba9575bc060ce15bb2b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba9575bc060ce15bb2b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba9575bc060ce15bb2b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba9575bc060ce15bb2b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:46,666 - lava_callback - INFO - Starting async job submit for node 69407bbb575bc060ce15bb67 2025-12-17 21:21:46,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:46,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbb575bc060ce15bb67 HTTP/1.1" 502 150 2025-12-17 21:21:46,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbb575bc060ce15bb67'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:46,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbb575bc060ce15bb67 2025-12-17 21:21:46,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbb575bc060ce15bb67 HTTP/1.1" 502 150 2025-12-17 21:21:46,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbb575bc060ce15bb67'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:46,765 - lava_callback - INFO - Received callback for node 69407be2575bc060ce15bc0f INFO: 172.21.0.1:55178 - "POST /node/69407be2575bc060ce15bc0f HTTP/1.0" 202 Accepted 2025-12-17 21:21:46,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba1575bc060ce15bb09 HTTP/1.1" 502 150 2025-12-17 21:21:46,928 - lava_callback - ERROR - Error processing callback for node 69407ba1575bc060ce15bb09: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba1575bc060ce15bb09 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba1575bc060ce15bb09 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba1575bc060ce15bb09 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:46,930 - lava_callback - INFO - Starting async job submit for node 69407bbc575bc060ce15bb6b 2025-12-17 21:21:46,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:46,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbc575bc060ce15bb6b HTTP/1.1" 502 150 2025-12-17 21:21:46,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbc575bc060ce15bb6b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:46,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbc575bc060ce15bb6b 2025-12-17 21:21:46,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbc575bc060ce15bb6b HTTP/1.1" 502 150 2025-12-17 21:21:46,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbc575bc060ce15bb6b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:47,133 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bba575bc060ce15bb64 2025-12-17 21:21:47,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bba575bc060ce15bb64 HTTP/1.1" 502 150 2025-12-17 21:21:47,134 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bba575bc060ce15bb64'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:47,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb3575bc060ce15bb51 2025-12-17 21:21:47,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb3575bc060ce15bb51 HTTP/1.1" 502 150 2025-12-17 21:21:47,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb3575bc060ce15bb51'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:47,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb7575bc060ce15bb5e 2025-12-17 21:21:47,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb7575bc060ce15bb5e HTTP/1.1" 502 150 2025-12-17 21:21:47,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb7575bc060ce15bb5e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:47,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb4575bc060ce15bb54 2025-12-17 21:21:47,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb4575bc060ce15bb54 HTTP/1.1" 502 150 2025-12-17 21:21:47,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb4575bc060ce15bb54'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:48,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bab575bc060ce15bb32 2025-12-17 21:21:48,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bab575bc060ce15bb32 HTTP/1.1" 502 150 2025-12-17 21:21:48,470 - lava_callback - ERROR - Error processing callback for node 69407bab575bc060ce15bb32: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bab575bc060ce15bb32 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bab575bc060ce15bb32 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bab575bc060ce15bb32 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:48,472 - lava_callback - INFO - Starting async job submit for node 69407bbe575bc060ce15bb71 2025-12-17 21:21:48,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:48,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbe575bc060ce15bb71 HTTP/1.1" 502 150 2025-12-17 21:21:48,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbe575bc060ce15bb71'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:48,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbe575bc060ce15bb71 2025-12-17 21:21:48,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbe575bc060ce15bb71 HTTP/1.1" 502 150 2025-12-17 21:21:48,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbe575bc060ce15bb71'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:48,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b94575bc060ce15badf 2025-12-17 21:21:48,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b94575bc060ce15badf HTTP/1.1" 502 150 2025-12-17 21:21:48,564 - lava_callback - ERROR - Error processing callback for node 69407b94575bc060ce15badf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b94575bc060ce15badf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b94575bc060ce15badf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b94575bc060ce15badf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:48,566 - lava_callback - INFO - Starting async job submit for node 69407bbf575bc060ce15bb77 2025-12-17 21:21:48,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:48,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbf575bc060ce15bb77 HTTP/1.1" 502 150 2025-12-17 21:21:48,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbf575bc060ce15bb77'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:48,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbf575bc060ce15bb77 2025-12-17 21:21:48,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbf575bc060ce15bb77 HTTP/1.1" 502 150 2025-12-17 21:21:48,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbf575bc060ce15bb77'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:48,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbb575bc060ce15bb67 2025-12-17 21:21:48,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbb575bc060ce15bb67 HTTP/1.1" 502 150 2025-12-17 21:21:48,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbb575bc060ce15bb67'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:48,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbc575bc060ce15bb6b 2025-12-17 21:21:48,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbc575bc060ce15bb6b HTTP/1.1" 502 150 2025-12-17 21:21:48,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbc575bc060ce15bb6b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:49,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bad575bc060ce15bb3e 2025-12-17 21:21:49,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bad575bc060ce15bb3e HTTP/1.1" 502 150 2025-12-17 21:21:49,873 - lava_callback - ERROR - Error processing callback for node 69407bad575bc060ce15bb3e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bad575bc060ce15bb3e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bad575bc060ce15bb3e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bad575bc060ce15bb3e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:49,875 - lava_callback - INFO - Starting async job submit for node 69407bc0575bc060ce15bb7e 2025-12-17 21:21:49,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:49,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc0575bc060ce15bb7e HTTP/1.1" 502 150 2025-12-17 21:21:49,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc0575bc060ce15bb7e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:49,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc0575bc060ce15bb7e 2025-12-17 21:21:49,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc0575bc060ce15bb7e HTTP/1.1" 502 150 2025-12-17 21:21:49,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc0575bc060ce15bb7e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:50,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbe575bc060ce15bb71 2025-12-17 21:21:50,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbe575bc060ce15bb71 HTTP/1.1" 502 150 2025-12-17 21:21:50,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbe575bc060ce15bb71'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:50,517 - lava_callback - INFO - Received callback for node 69407bdc575bc060ce15bc03 INFO: 172.21.0.1:33582 - "POST /node/69407bdc575bc060ce15bc03 HTTP/1.0" 202 Accepted 2025-12-17 21:21:50,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbf575bc060ce15bb77 2025-12-17 21:21:50,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbf575bc060ce15bb77 HTTP/1.1" 502 150 2025-12-17 21:21:50,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbf575bc060ce15bb77'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:50,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407baf575bc060ce15bb46 2025-12-17 21:21:50,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407baf575bc060ce15bb46 HTTP/1.1" 502 150 2025-12-17 21:21:50,760 - lava_callback - ERROR - Error processing callback for node 69407baf575bc060ce15bb46: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407baf575bc060ce15bb46 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407baf575bc060ce15bb46 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407baf575bc060ce15bb46 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:50,762 - lava_callback - INFO - Starting async job submit for node 69407bc2575bc060ce15bb83 2025-12-17 21:21:50,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:50,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc2575bc060ce15bb83 HTTP/1.1" 502 150 2025-12-17 21:21:50,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc2575bc060ce15bb83'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:50,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc2575bc060ce15bb83 2025-12-17 21:21:50,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc2575bc060ce15bb83 HTTP/1.1" 502 150 2025-12-17 21:21:50,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc2575bc060ce15bb83'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:51,134 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bba575bc060ce15bb64 2025-12-17 21:21:51,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bba575bc060ce15bb64 HTTP/1.1" 502 150 2025-12-17 21:21:51,135 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bba575bc060ce15bb64'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:51,154 - lava_callback - INFO - Received callback for node 69407bde575bc060ce15bc06 INFO: 172.21.0.1:33590 - "POST /node/69407bde575bc060ce15bc06 HTTP/1.0" 202 Accepted 2025-12-17 21:21:51,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba3575bc060ce15bb0c 2025-12-17 21:21:51,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba3575bc060ce15bb0c HTTP/1.1" 502 150 2025-12-17 21:21:51,410 - lava_callback - ERROR - Error processing callback for node 69407ba3575bc060ce15bb0c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba3575bc060ce15bb0c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba3575bc060ce15bb0c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba3575bc060ce15bb0c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:51,412 - lava_callback - INFO - Starting async job submit for node 69407bc3575bc060ce15bb88 2025-12-17 21:21:51,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:51,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc3575bc060ce15bb88 HTTP/1.1" 502 150 2025-12-17 21:21:51,437 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc3575bc060ce15bb88'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:51,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc3575bc060ce15bb88 2025-12-17 21:21:51,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc3575bc060ce15bb88 HTTP/1.1" 502 150 2025-12-17 21:21:51,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc3575bc060ce15bb88'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:51,442 - lava_callback - INFO - Received callback for node 69407d46575bc060ce15c030 INFO: 172.21.0.1:33600 - "POST /node/69407d46575bc060ce15c030 HTTP/1.0" 202 Accepted 2025-12-17 21:21:51,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc0575bc060ce15bb7e 2025-12-17 21:21:51,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc0575bc060ce15bb7e HTTP/1.1" 502 150 2025-12-17 21:21:51,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc0575bc060ce15bb7e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:52,083 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb6575bc060ce15bb57 2025-12-17 21:21:52,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbb575bc060ce15bb67 2025-12-17 21:21:52,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbb575bc060ce15bb67 HTTP/1.1" 502 150 2025-12-17 21:21:52,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbb575bc060ce15bb67'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:52,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc2575bc060ce15bb83 2025-12-17 21:21:52,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc2575bc060ce15bb83 HTTP/1.1" 502 150 2025-12-17 21:21:52,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc2575bc060ce15bb83'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:52,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb15 HTTP/1.1" 502 150 2025-12-17 21:21:52,816 - lava_callback - ERROR - Error processing callback for node 69407ba5575bc060ce15bb15: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba5575bc060ce15bb15 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba5575bc060ce15bb15 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba5575bc060ce15bb15 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:52,818 - lava_callback - INFO - Starting async job submit for node 69407bc4575bc060ce15bb8b 2025-12-17 21:21:52,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:52,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc4575bc060ce15bb8b HTTP/1.1" 502 150 2025-12-17 21:21:52,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc4575bc060ce15bb8b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:52,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc4575bc060ce15bb8b 2025-12-17 21:21:52,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc4575bc060ce15bb8b HTTP/1.1" 502 150 2025-12-17 21:21:52,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc4575bc060ce15bb8b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:52,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbc575bc060ce15bb6b 2025-12-17 21:21:52,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbc575bc060ce15bb6b HTTP/1.1" 502 150 2025-12-17 21:21:52,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbc575bc060ce15bb6b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:53,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc3575bc060ce15bb88 2025-12-17 21:21:53,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc3575bc060ce15bb88 HTTP/1.1" 502 150 2025-12-17 21:21:53,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc3575bc060ce15bb88'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:54,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbe575bc060ce15bb71 2025-12-17 21:21:54,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbe575bc060ce15bb71 HTTP/1.1" 502 150 2025-12-17 21:21:54,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbe575bc060ce15bb71'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:54,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbf575bc060ce15bb77 2025-12-17 21:21:54,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbf575bc060ce15bb77 HTTP/1.1" 502 150 2025-12-17 21:21:54,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbf575bc060ce15bb77'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:54,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc4575bc060ce15bb8b 2025-12-17 21:21:54,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc4575bc060ce15bb8b HTTP/1.1" 502 150 2025-12-17 21:21:54,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc4575bc060ce15bb8b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:55,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb7575bc060ce15bb5e 2025-12-17 21:21:55,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb7575bc060ce15bb5e HTTP/1.1" 502 150 2025-12-17 21:21:55,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb7575bc060ce15bb5e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:55,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc0575bc060ce15bb7e 2025-12-17 21:21:55,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc0575bc060ce15bb7e HTTP/1.1" 502 150 2025-12-17 21:21:55,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc0575bc060ce15bb7e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33608 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:21:56,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc2575bc060ce15bb83 2025-12-17 21:21:56,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc2575bc060ce15bb83 HTTP/1.1" 502 150 2025-12-17 21:21:56,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc2575bc060ce15bb83'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:57,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc3575bc060ce15bb88 2025-12-17 21:21:57,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb1575bc060ce15bb4b 2025-12-17 21:21:57,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb1575bc060ce15bb4b HTTP/1.1" 502 150 2025-12-17 21:21:57,965 - lava_callback - ERROR - Error processing callback for node 69407bb1575bc060ce15bb4b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb1575bc060ce15bb4b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb1575bc060ce15bb4b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb1575bc060ce15bb4b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:57,967 - lava_callback - INFO - Starting async job submit for node 69407ba5575bc060ce15bb12 2025-12-17 21:21:57,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:57,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb12 HTTP/1.1" 502 150 2025-12-17 21:21:57,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba5575bc060ce15bb12'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:57,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba5575bc060ce15bb12 2025-12-17 21:21:57,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb12 HTTP/1.1" 502 150 2025-12-17 21:21:57,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba5575bc060ce15bb12'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:58,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc4575bc060ce15bb8b 2025-12-17 21:21:58,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc4575bc060ce15bb8b HTTP/1.1" 502 150 2025-12-17 21:21:58,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc4575bc060ce15bb8b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:59,136 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bba575bc060ce15bb64 2025-12-17 21:21:59,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bba575bc060ce15bb64 HTTP/1.1" 502 150 2025-12-17 21:21:59,137 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bba575bc060ce15bb64'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:59,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb2575bc060ce15bb4e 2025-12-17 21:21:59,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb2575bc060ce15bb4e HTTP/1.1" 502 150 2025-12-17 21:21:59,612 - lava_callback - ERROR - Error processing callback for node 69407bb2575bc060ce15bb4e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb2575bc060ce15bb4e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb2575bc060ce15bb4e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb2575bc060ce15bb4e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:21:59,614 - lava_callback - INFO - Starting async job submit for node 69407bc6575bc060ce15bb8e 2025-12-17 21:21:59,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:21:59,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc6575bc060ce15bb8e HTTP/1.1" 502 150 2025-12-17 21:21:59,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc6575bc060ce15bb8e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:59,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc6575bc060ce15bb8e 2025-12-17 21:21:59,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc6575bc060ce15bb8e HTTP/1.1" 502 150 2025-12-17 21:21:59,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc6575bc060ce15bb8e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:21:59,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba5575bc060ce15bb12 2025-12-17 21:21:59,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb12 HTTP/1.1" 502 150 2025-12-17 21:21:59,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba5575bc060ce15bb12'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:00,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbb575bc060ce15bb67 2025-12-17 21:22:00,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbb575bc060ce15bb67 HTTP/1.1" 502 150 2025-12-17 21:22:00,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbb575bc060ce15bb67'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:00,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbc575bc060ce15bb6b 2025-12-17 21:22:00,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbc575bc060ce15bb6b HTTP/1.1" 502 150 2025-12-17 21:22:00,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbc575bc060ce15bb6b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38474 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:22:01,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb9575bc060ce15bb61 HTTP/1.1" 502 150 2025-12-17 21:22:01,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb9575bc060ce15bb61'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:01,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc6575bc060ce15bb8e 2025-12-17 21:22:01,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc6575bc060ce15bb8e HTTP/1.1" 502 150 2025-12-17 21:22:01,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc6575bc060ce15bb8e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:02,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbe575bc060ce15bb71 2025-12-17 21:22:02,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbe575bc060ce15bb71 HTTP/1.1" 502 150 2025-12-17 21:22:02,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbe575bc060ce15bb71'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:02,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbf575bc060ce15bb77 2025-12-17 21:22:02,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbf575bc060ce15bb77 HTTP/1.1" 502 150 2025-12-17 21:22:02,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bbf575bc060ce15bb77'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:03,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb3575bc060ce15bb51 2025-12-17 21:22:03,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb3575bc060ce15bb51 HTTP/1.1" 502 150 2025-12-17 21:22:03,815 - lava_callback - ERROR - Error processing callback for node 69407bb3575bc060ce15bb51: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb3575bc060ce15bb51 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb3575bc060ce15bb51 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb3575bc060ce15bb51 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:03,817 - lava_callback - INFO - Starting async job submit for node 69407bc7575bc060ce15bb95 2025-12-17 21:22:03,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:03,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc7575bc060ce15bb95 HTTP/1.1" 502 150 2025-12-17 21:22:03,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc7575bc060ce15bb95'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:03,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc7575bc060ce15bb95 2025-12-17 21:22:03,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc7575bc060ce15bb95 HTTP/1.1" 502 150 2025-12-17 21:22:03,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc7575bc060ce15bb95'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:03,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc0575bc060ce15bb7e 2025-12-17 21:22:03,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc0575bc060ce15bb7e HTTP/1.1" 502 150 2025-12-17 21:22:03,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc0575bc060ce15bb7e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:03,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb4575bc060ce15bb54 2025-12-17 21:22:03,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb4575bc060ce15bb54 HTTP/1.1" 502 150 2025-12-17 21:22:03,927 - lava_callback - ERROR - Error processing callback for node 69407bb4575bc060ce15bb54: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb4575bc060ce15bb54 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb4575bc060ce15bb54 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb4575bc060ce15bb54 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:03,928 - lava_callback - INFO - Starting async job submit for node 69407bd4575bc060ce15bbea 2025-12-17 21:22:03,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:03,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd4575bc060ce15bbea HTTP/1.1" 502 150 2025-12-17 21:22:03,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd4575bc060ce15bbea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:03,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd4575bc060ce15bbea 2025-12-17 21:22:03,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd4575bc060ce15bbea HTTP/1.1" 502 150 2025-12-17 21:22:03,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd4575bc060ce15bbea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:03,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba5575bc060ce15bb12 2025-12-17 21:22:03,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb12 HTTP/1.1" 502 150 2025-12-17 21:22:03,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba5575bc060ce15bb12'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:04,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc2575bc060ce15bb83 2025-12-17 21:22:04,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc2575bc060ce15bb83 HTTP/1.1" 502 150 2025-12-17 21:22:04,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc2575bc060ce15bb83'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:05,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb9575bc060ce15bb61 2025-12-17 21:22:05,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb9575bc060ce15bb61 HTTP/1.1" 502 150 2025-12-17 21:22:05,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb9575bc060ce15bb61'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:05,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc6575bc060ce15bb8e 2025-12-17 21:22:05,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc6575bc060ce15bb8e HTTP/1.1" 502 150 2025-12-17 21:22:05,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc6575bc060ce15bb8e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:05,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc7575bc060ce15bb95 2025-12-17 21:22:05,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc7575bc060ce15bb95 HTTP/1.1" 502 150 2025-12-17 21:22:05,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc7575bc060ce15bb95'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:05,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd4575bc060ce15bbea 2025-12-17 21:22:05,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd4575bc060ce15bbea HTTP/1.1" 502 150 2025-12-17 21:22:05,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd4575bc060ce15bbea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38478 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:22:06,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc4575bc060ce15bb8b 2025-12-17 21:22:06,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc4575bc060ce15bb8b HTTP/1.1" 502 150 2025-12-17 21:22:06,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc4575bc060ce15bb8b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:07,288 - lava_callback - INFO - Received callback for node 69407bf3575bc060ce15bc2e INFO: 172.21.0.1:38484 - "POST /node/69407bf3575bc060ce15bc2e HTTP/1.0" 202 Accepted 2025-12-17 21:22:07,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb6575bc060ce15bb57 HTTP/1.1" 502 150 2025-12-17 21:22:07,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb6575bc060ce15bb57'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:09,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc7575bc060ce15bb95 2025-12-17 21:22:09,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc7575bc060ce15bb95 HTTP/1.1" 502 150 2025-12-17 21:22:09,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc7575bc060ce15bb95'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:09,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd4575bc060ce15bbea 2025-12-17 21:22:09,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd4575bc060ce15bbea HTTP/1.1" 502 150 2025-12-17 21:22:09,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd4575bc060ce15bbea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:11,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb7575bc060ce15bb5e 2025-12-17 21:22:11,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb7575bc060ce15bb5e HTTP/1.1" 502 150 2025-12-17 21:22:11,855 - lava_callback - ERROR - Error processing callback for node 69407bb7575bc060ce15bb5e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb7575bc060ce15bb5e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb7575bc060ce15bb5e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb7575bc060ce15bb5e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:11,857 - lava_callback - INFO - Starting async job submit for node 69407bc9575bc060ce15bbc7 2025-12-17 21:22:11,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:11,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc9575bc060ce15bbc7 HTTP/1.1" 502 150 2025-12-17 21:22:11,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc9575bc060ce15bbc7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:11,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc9575bc060ce15bbc7 2025-12-17 21:22:11,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc9575bc060ce15bbc7 HTTP/1.1" 502 150 2025-12-17 21:22:11,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc9575bc060ce15bbc7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:11,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba5575bc060ce15bb12 2025-12-17 21:22:11,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb12 HTTP/1.1" 502 150 2025-12-17 21:22:11,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ba5575bc060ce15bb12'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:12,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc3575bc060ce15bb88 HTTP/1.1" 502 150 2025-12-17 21:22:12,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc3575bc060ce15bb88'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:13,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb9575bc060ce15bb61 2025-12-17 21:22:13,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb9575bc060ce15bb61 HTTP/1.1" 502 150 2025-12-17 21:22:13,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bb9575bc060ce15bb61'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:13,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc6575bc060ce15bb8e 2025-12-17 21:22:13,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc6575bc060ce15bb8e HTTP/1.1" 502 150 2025-12-17 21:22:13,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc6575bc060ce15bb8e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:13,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc9575bc060ce15bbc7 2025-12-17 21:22:13,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc9575bc060ce15bbc7 HTTP/1.1" 502 150 2025-12-17 21:22:13,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc9575bc060ce15bbc7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:14,378 - lava_callback - INFO - Received callback for node 69407bf1575bc060ce15bc28 INFO: 172.21.0.1:36304 - "POST /node/69407bf1575bc060ce15bc28 HTTP/1.0" 202 Accepted 2025-12-17 21:22:15,137 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bba575bc060ce15bb64 2025-12-17 21:22:15,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bba575bc060ce15bb64 HTTP/1.1" 502 150 2025-12-17 21:22:15,138 - lava_callback - ERROR - Error processing callback for node 69407bba575bc060ce15bb64: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bba575bc060ce15bb64 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bba575bc060ce15bb64 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bba575bc060ce15bb64 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:15,140 - lava_callback - INFO - Starting async job submit for node 69407bca575bc060ce15bbcb 2025-12-17 21:22:15,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:15,160 - lava_callback - INFO - Received callback for node 69407bf2575bc060ce15bc2b INFO: 172.21.0.1:36314 - "POST /node/69407bf2575bc060ce15bc2b HTTP/1.0" 202 Accepted INFO: 172.21.0.1:36320 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:22:15,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bca575bc060ce15bbcb HTTP/1.1" 502 150 2025-12-17 21:22:15,171 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bca575bc060ce15bbcb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:15,171 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bca575bc060ce15bbcb 2025-12-17 21:22:15,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bca575bc060ce15bbcb HTTP/1.1" 502 150 2025-12-17 21:22:15,172 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bca575bc060ce15bbcb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:15,964 - lava_callback - INFO - Received callback for node 69407bf8575bc060ce15bc33 INFO: 172.21.0.1:36326 - "POST /node/69407bf8575bc060ce15bc33 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:36334 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:22:16,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbb575bc060ce15bb67 2025-12-17 21:22:16,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbb575bc060ce15bb67 HTTP/1.1" 502 150 2025-12-17 21:22:16,679 - lava_callback - ERROR - Error processing callback for node 69407bbb575bc060ce15bb67: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bbb575bc060ce15bb67 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bbb575bc060ce15bb67 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bbb575bc060ce15bb67 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:16,682 - lava_callback - INFO - Starting async job submit for node 69407bcc575bc060ce15bbce 2025-12-17 21:22:16,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:16,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bcc575bc060ce15bbce HTTP/1.1" 502 150 2025-12-17 21:22:16,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bcc575bc060ce15bbce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:16,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bcc575bc060ce15bbce 2025-12-17 21:22:16,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bcc575bc060ce15bbce HTTP/1.1" 502 150 2025-12-17 21:22:16,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bcc575bc060ce15bbce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:16,706 - lava_callback - INFO - Received callback for node 69407bfb575bc060ce15bc3a INFO: 172.21.0.1:36340 - "POST /node/69407bfb575bc060ce15bc3a HTTP/1.0" 202 Accepted 2025-12-17 21:22:16,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbc575bc060ce15bb6b 2025-12-17 21:22:16,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbc575bc060ce15bb6b HTTP/1.1" 502 150 2025-12-17 21:22:16,942 - lava_callback - ERROR - Error processing callback for node 69407bbc575bc060ce15bb6b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bbc575bc060ce15bb6b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bbc575bc060ce15bb6b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bbc575bc060ce15bb6b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:16,944 - lava_callback - INFO - Starting async job submit for node 69407bcd575bc060ce15bbd1 2025-12-17 21:22:16,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:16,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bcd575bc060ce15bbd1 HTTP/1.1" 502 150 2025-12-17 21:22:16,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bcd575bc060ce15bbd1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:16,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bcd575bc060ce15bbd1 2025-12-17 21:22:16,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bcd575bc060ce15bbd1 HTTP/1.1" 502 150 2025-12-17 21:22:16,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bcd575bc060ce15bbd1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:17,172 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bca575bc060ce15bbcb 2025-12-17 21:22:17,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bca575bc060ce15bbcb HTTP/1.1" 502 150 2025-12-17 21:22:17,173 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bca575bc060ce15bbcb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:17,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc7575bc060ce15bb95 2025-12-17 21:22:17,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc7575bc060ce15bb95 HTTP/1.1" 502 150 2025-12-17 21:22:17,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc7575bc060ce15bb95'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:17,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc9575bc060ce15bbc7 2025-12-17 21:22:17,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc9575bc060ce15bbc7 HTTP/1.1" 502 150 2025-12-17 21:22:17,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc9575bc060ce15bbc7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:17,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd4575bc060ce15bbea 2025-12-17 21:22:17,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd4575bc060ce15bbea HTTP/1.1" 502 150 2025-12-17 21:22:17,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd4575bc060ce15bbea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:18,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbe575bc060ce15bb71 2025-12-17 21:22:18,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbe575bc060ce15bb71 HTTP/1.1" 502 150 2025-12-17 21:22:18,486 - lava_callback - ERROR - Error processing callback for node 69407bbe575bc060ce15bb71: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bbe575bc060ce15bb71 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bbe575bc060ce15bb71 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bbe575bc060ce15bb71 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:18,488 - lava_callback - INFO - Starting async job submit for node 69407bce575bc060ce15bbd4 2025-12-17 21:22:18,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:18,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bce575bc060ce15bbd4 HTTP/1.1" 502 150 2025-12-17 21:22:18,495 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bce575bc060ce15bbd4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:18,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bce575bc060ce15bbd4 2025-12-17 21:22:18,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bce575bc060ce15bbd4 HTTP/1.1" 502 150 2025-12-17 21:22:18,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bce575bc060ce15bbd4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:18,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bbf575bc060ce15bb77 2025-12-17 21:22:18,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bbf575bc060ce15bb77 HTTP/1.1" 502 150 2025-12-17 21:22:18,578 - lava_callback - ERROR - Error processing callback for node 69407bbf575bc060ce15bb77: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bbf575bc060ce15bb77 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bbf575bc060ce15bb77 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bbf575bc060ce15bb77 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:18,580 - lava_callback - INFO - Starting async job submit for node 69407bd0575bc060ce15bbd7 2025-12-17 21:22:18,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:18,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd0575bc060ce15bbd7 HTTP/1.1" 502 150 2025-12-17 21:22:18,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd0575bc060ce15bbd7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:18,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd0575bc060ce15bbd7 2025-12-17 21:22:18,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd0575bc060ce15bbd7 HTTP/1.1" 502 150 2025-12-17 21:22:18,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd0575bc060ce15bbd7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:18,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bcc575bc060ce15bbce 2025-12-17 21:22:18,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bcc575bc060ce15bbce HTTP/1.1" 502 150 2025-12-17 21:22:18,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bcc575bc060ce15bbce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:18,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bcd575bc060ce15bbd1 2025-12-17 21:22:18,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bcd575bc060ce15bbd1 HTTP/1.1" 502 150 2025-12-17 21:22:18,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bcd575bc060ce15bbd1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:19,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc0575bc060ce15bb7e 2025-12-17 21:22:19,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc0575bc060ce15bb7e HTTP/1.1" 502 150 2025-12-17 21:22:19,888 - lava_callback - ERROR - Error processing callback for node 69407bc0575bc060ce15bb7e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc0575bc060ce15bb7e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc0575bc060ce15bb7e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc0575bc060ce15bb7e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:19,890 - lava_callback - INFO - Starting async job submit for node 69407bd1575bc060ce15bbdc 2025-12-17 21:22:19,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:19,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd1575bc060ce15bbdc HTTP/1.1" 502 150 2025-12-17 21:22:19,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd1575bc060ce15bbdc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:19,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd1575bc060ce15bbdc 2025-12-17 21:22:19,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd1575bc060ce15bbdc HTTP/1.1" 502 150 2025-12-17 21:22:19,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd1575bc060ce15bbdc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:20,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bce575bc060ce15bbd4 2025-12-17 21:22:20,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bce575bc060ce15bbd4 HTTP/1.1" 502 150 2025-12-17 21:22:20,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bce575bc060ce15bbd4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:20,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd0575bc060ce15bbd7 2025-12-17 21:22:20,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd0575bc060ce15bbd7 HTTP/1.1" 502 150 2025-12-17 21:22:20,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd0575bc060ce15bbd7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:20,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc2575bc060ce15bb83 2025-12-17 21:22:20,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc2575bc060ce15bb83 HTTP/1.1" 502 150 2025-12-17 21:22:20,775 - lava_callback - ERROR - Error processing callback for node 69407bc2575bc060ce15bb83: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc2575bc060ce15bb83 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc2575bc060ce15bb83 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc2575bc060ce15bb83 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:20,777 - lava_callback - INFO - Starting async job submit for node 69407bd2575bc060ce15bbe1 2025-12-17 21:22:20,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:20,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd2575bc060ce15bbe1 HTTP/1.1" 502 150 2025-12-17 21:22:20,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd2575bc060ce15bbe1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:20,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd2575bc060ce15bbe1 2025-12-17 21:22:20,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd2575bc060ce15bbe1 HTTP/1.1" 502 150 2025-12-17 21:22:20,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd2575bc060ce15bbe1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:20,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc3575bc060ce15bb88 2025-12-17 21:22:20,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc3575bc060ce15bb88 HTTP/1.1" 502 150 2025-12-17 21:22:20,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc3575bc060ce15bb88'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:21,173 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bca575bc060ce15bbcb 2025-12-17 21:22:21,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bca575bc060ce15bbcb HTTP/1.1" 502 150 2025-12-17 21:22:21,175 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bca575bc060ce15bbcb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:21,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd1575bc060ce15bbdc 2025-12-17 21:22:21,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd1575bc060ce15bbdc HTTP/1.1" 502 150 2025-12-17 21:22:21,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd1575bc060ce15bbdc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:22,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bcc575bc060ce15bbce 2025-12-17 21:22:22,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bcc575bc060ce15bbce HTTP/1.1" 502 150 2025-12-17 21:22:22,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bcc575bc060ce15bbce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:22,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd2575bc060ce15bbe1 2025-12-17 21:22:22,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd2575bc060ce15bbe1 HTTP/1.1" 502 150 2025-12-17 21:22:22,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd2575bc060ce15bbe1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:22,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc4575bc060ce15bb8b 2025-12-17 21:22:22,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc4575bc060ce15bb8b HTTP/1.1" 502 150 2025-12-17 21:22:22,830 - lava_callback - ERROR - Error processing callback for node 69407bc4575bc060ce15bb8b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc4575bc060ce15bb8b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc4575bc060ce15bb8b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc4575bc060ce15bb8b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:22,832 - lava_callback - INFO - Starting async job submit for node 69407bd3575bc060ce15bbe5 2025-12-17 21:22:22,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:22,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd3575bc060ce15bbe5 HTTP/1.1" 502 150 2025-12-17 21:22:22,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd3575bc060ce15bbe5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:22,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd3575bc060ce15bbe5 2025-12-17 21:22:22,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd3575bc060ce15bbe5 HTTP/1.1" 502 150 2025-12-17 21:22:22,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd3575bc060ce15bbe5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:22,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bcd575bc060ce15bbd1 2025-12-17 21:22:22,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bcd575bc060ce15bbd1 HTTP/1.1" 502 150 2025-12-17 21:22:22,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bcd575bc060ce15bbd1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:23,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb6575bc060ce15bb57 2025-12-17 21:22:24,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bce575bc060ce15bbd4 2025-12-17 21:22:24,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bce575bc060ce15bbd4 HTTP/1.1" 502 150 2025-12-17 21:22:24,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bce575bc060ce15bbd4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:24,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd0575bc060ce15bbd7 2025-12-17 21:22:24,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd0575bc060ce15bbd7 HTTP/1.1" 502 150 2025-12-17 21:22:24,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd0575bc060ce15bbd7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:24,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd3575bc060ce15bbe5 2025-12-17 21:22:24,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd3575bc060ce15bbe5 HTTP/1.1" 502 150 2025-12-17 21:22:24,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd3575bc060ce15bbe5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:25,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc9575bc060ce15bbc7 2025-12-17 21:22:25,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc9575bc060ce15bbc7 HTTP/1.1" 502 150 2025-12-17 21:22:25,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bc9575bc060ce15bbc7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:25,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd1575bc060ce15bbdc 2025-12-17 21:22:25,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd1575bc060ce15bbdc HTTP/1.1" 502 150 2025-12-17 21:22:25,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd1575bc060ce15bbdc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57076 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:22:26,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd2575bc060ce15bbe1 2025-12-17 21:22:26,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd2575bc060ce15bbe1 HTTP/1.1" 502 150 2025-12-17 21:22:26,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd2575bc060ce15bbe1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:27,194 - lava_callback - INFO - Received callback for node 69407c01575bc060ce15bc3f INFO: 172.21.0.1:57092 - "POST /node/69407c01575bc060ce15bc3f HTTP/1.0" 202 Accepted 2025-12-17 21:22:27,895 - lava_callback - INFO - Received callback for node 69407c03575bc060ce15bc44 INFO: 172.21.0.1:57094 - "POST /node/69407c03575bc060ce15bc44 HTTP/1.0" 202 Accepted 2025-12-17 21:22:27,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ba5575bc060ce15bb12 2025-12-17 21:22:27,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ba5575bc060ce15bb12 HTTP/1.1" 502 150 2025-12-17 21:22:27,983 - lava_callback - ERROR - Error processing callback for node 69407ba5575bc060ce15bb12: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba5575bc060ce15bb12 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba5575bc060ce15bb12 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ba5575bc060ce15bb12 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:27,984 - lava_callback - INFO - Starting async job submit for node 69407bd6575bc060ce15bbef 2025-12-17 21:22:27,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:27,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd6575bc060ce15bbef HTTP/1.1" 502 150 2025-12-17 21:22:27,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd6575bc060ce15bbef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:27,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd6575bc060ce15bbef 2025-12-17 21:22:27,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd6575bc060ce15bbef HTTP/1.1" 502 150 2025-12-17 21:22:27,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd6575bc060ce15bbef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:28,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd3575bc060ce15bbe5 2025-12-17 21:22:28,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd3575bc060ce15bbe5 HTTP/1.1" 502 150 2025-12-17 21:22:28,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd3575bc060ce15bbe5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:29,175 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bca575bc060ce15bbcb 2025-12-17 21:22:29,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bb9575bc060ce15bb61 2025-12-17 21:22:29,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb9575bc060ce15bb61 HTTP/1.1" 502 150 2025-12-17 21:22:29,527 - lava_callback - ERROR - Error processing callback for node 69407bb9575bc060ce15bb61: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb9575bc060ce15bb61 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb9575bc060ce15bb61 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb9575bc060ce15bb61 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:29,530 - lava_callback - INFO - Starting async job submit for node 69407bd8575bc060ce15bbf3 2025-12-17 21:22:29,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:29,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd8575bc060ce15bbf3 HTTP/1.1" 502 150 2025-12-17 21:22:29,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd8575bc060ce15bbf3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:29,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd8575bc060ce15bbf3 2025-12-17 21:22:29,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd8575bc060ce15bbf3 HTTP/1.1" 502 150 2025-12-17 21:22:29,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd8575bc060ce15bbf3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:29,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc6575bc060ce15bb8e 2025-12-17 21:22:29,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc6575bc060ce15bb8e HTTP/1.1" 502 150 2025-12-17 21:22:29,641 - lava_callback - ERROR - Error processing callback for node 69407bc6575bc060ce15bb8e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc6575bc060ce15bb8e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc6575bc060ce15bb8e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc6575bc060ce15bb8e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:29,644 - lava_callback - INFO - Starting async job submit for node 69407bd9575bc060ce15bbf6 2025-12-17 21:22:29,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:29,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd9575bc060ce15bbf6 HTTP/1.1" 502 150 2025-12-17 21:22:29,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd9575bc060ce15bbf6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:29,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd9575bc060ce15bbf6 2025-12-17 21:22:29,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd9575bc060ce15bbf6 HTTP/1.1" 502 150 2025-12-17 21:22:29,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd9575bc060ce15bbf6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:29,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd6575bc060ce15bbef 2025-12-17 21:22:29,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd6575bc060ce15bbef HTTP/1.1" 502 150 2025-12-17 21:22:29,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd6575bc060ce15bbef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:30,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bcc575bc060ce15bbce 2025-12-17 21:22:30,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bcc575bc060ce15bbce HTTP/1.1" 502 150 2025-12-17 21:22:30,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bcc575bc060ce15bbce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:30,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bcd575bc060ce15bbd1 2025-12-17 21:22:30,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bcd575bc060ce15bbd1 HTTP/1.1" 502 150 2025-12-17 21:22:30,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bcd575bc060ce15bbd1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:31,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd8575bc060ce15bbf3 2025-12-17 21:22:31,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd8575bc060ce15bbf3 HTTP/1.1" 502 150 2025-12-17 21:22:31,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd8575bc060ce15bbf3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:31,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd9575bc060ce15bbf6 2025-12-17 21:22:31,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd9575bc060ce15bbf6 HTTP/1.1" 502 150 2025-12-17 21:22:31,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd9575bc060ce15bbf6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:32,363 - lava_callback - INFO - Received callback for node 69407b12575bc060ce15b939 INFO: 172.21.0.1:47720 - "POST /node/69407b12575bc060ce15b939 HTTP/1.0" 202 Accepted 2025-12-17 21:22:32,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bce575bc060ce15bbd4 2025-12-17 21:22:32,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bce575bc060ce15bbd4 HTTP/1.1" 502 150 2025-12-17 21:22:32,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bce575bc060ce15bbd4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:32,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd0575bc060ce15bbd7 2025-12-17 21:22:32,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd0575bc060ce15bbd7 HTTP/1.1" 502 150 2025-12-17 21:22:32,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd0575bc060ce15bbd7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:33,348 - lava_callback - INFO - Received callback for node 69407d47575bc060ce15c033 INFO: 172.21.0.1:47730 - "POST /node/69407d47575bc060ce15c033 HTTP/1.0" 202 Accepted 2025-12-17 21:22:33,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc7575bc060ce15bb95 2025-12-17 21:22:33,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc7575bc060ce15bb95 HTTP/1.1" 502 150 2025-12-17 21:22:33,832 - lava_callback - ERROR - Error processing callback for node 69407bc7575bc060ce15bb95: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc7575bc060ce15bb95 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc7575bc060ce15bb95 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc7575bc060ce15bb95 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:33,835 - lava_callback - INFO - Starting async job submit for node 69407bda575bc060ce15bbf9 2025-12-17 21:22:33,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:33,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bda575bc060ce15bbf9 HTTP/1.1" 502 150 2025-12-17 21:22:33,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bda575bc060ce15bbf9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:33,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bda575bc060ce15bbf9 2025-12-17 21:22:33,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bda575bc060ce15bbf9 HTTP/1.1" 502 150 2025-12-17 21:22:33,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bda575bc060ce15bbf9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:33,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd1575bc060ce15bbdc 2025-12-17 21:22:33,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd1575bc060ce15bbdc HTTP/1.1" 502 150 2025-12-17 21:22:33,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd1575bc060ce15bbdc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:33,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd4575bc060ce15bbea 2025-12-17 21:22:33,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd4575bc060ce15bbea HTTP/1.1" 502 150 2025-12-17 21:22:33,945 - lava_callback - ERROR - Error processing callback for node 69407bd4575bc060ce15bbea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd4575bc060ce15bbea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd4575bc060ce15bbea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd4575bc060ce15bbea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:33,947 - lava_callback - INFO - Starting async job submit for node 69407bdb575bc060ce15bc00 2025-12-17 21:22:33,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:33,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdb575bc060ce15bc00 HTTP/1.1" 502 150 2025-12-17 21:22:33,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bdb575bc060ce15bc00'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:33,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bdb575bc060ce15bc00 2025-12-17 21:22:33,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdb575bc060ce15bc00 HTTP/1.1" 502 150 2025-12-17 21:22:33,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bdb575bc060ce15bc00'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:33,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd6575bc060ce15bbef 2025-12-17 21:22:33,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd6575bc060ce15bbef HTTP/1.1" 502 150 2025-12-17 21:22:33,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd6575bc060ce15bbef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:34,121 - lava_callback - INFO - Received callback for node 69408c07575bc060ce15f42b INFO: 172.21.0.1:47746 - "POST /node/69408c07575bc060ce15f42b HTTP/1.0" 202 Accepted 2025-12-17 21:22:34,609 - lava_callback - INFO - Received callback for node 69407c0b575bc060ce15bc8e INFO: 172.21.0.1:47756 - "POST /node/69407c0b575bc060ce15bc8e HTTP/1.0" 202 Accepted 2025-12-17 21:22:34,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd2575bc060ce15bbe1 2025-12-17 21:22:34,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd2575bc060ce15bbe1 HTTP/1.1" 502 150 2025-12-17 21:22:34,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd2575bc060ce15bbe1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:35,449 - lava_callback - INFO - Received callback for node 69407c0c575bc060ce15bc91 INFO: 172.21.0.1:47760 - "POST /node/69407c0c575bc060ce15bc91 HTTP/1.0" 202 Accepted 2025-12-17 21:22:35,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd8575bc060ce15bbf3 2025-12-17 21:22:35,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd8575bc060ce15bbf3 HTTP/1.1" 502 150 2025-12-17 21:22:35,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd8575bc060ce15bbf3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:35,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd9575bc060ce15bbf6 2025-12-17 21:22:35,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd9575bc060ce15bbf6 HTTP/1.1" 502 150 2025-12-17 21:22:35,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd9575bc060ce15bbf6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:35,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bda575bc060ce15bbf9 2025-12-17 21:22:35,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bda575bc060ce15bbf9 HTTP/1.1" 502 150 2025-12-17 21:22:35,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bda575bc060ce15bbf9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:35,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bdb575bc060ce15bc00 2025-12-17 21:22:35,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdb575bc060ce15bc00 HTTP/1.1" 502 150 2025-12-17 21:22:35,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bdb575bc060ce15bc00'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:36,240 - lava_callback - INFO - Received callback for node 69407c0e575bc060ce15bc97 INFO: 172.21.0.1:47776 - "POST /node/69407c0e575bc060ce15bc97 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:47782 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:22:36,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc3575bc060ce15bb88 2025-12-17 21:22:36,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc3575bc060ce15bb88 HTTP/1.1" 502 150 2025-12-17 21:22:36,787 - lava_callback - ERROR - Error processing callback for node 69407bc3575bc060ce15bb88: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc3575bc060ce15bb88 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc3575bc060ce15bb88 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc3575bc060ce15bb88 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:36,789 - lava_callback - INFO - Starting async job submit for node 69407bdf575bc060ce15bc09 2025-12-17 21:22:36,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:36,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd3575bc060ce15bbe5 2025-12-17 21:22:36,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd3575bc060ce15bbe5 HTTP/1.1" 502 150 2025-12-17 21:22:36,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd3575bc060ce15bbe5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:36,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdf575bc060ce15bc09 HTTP/1.1" 502 150 2025-12-17 21:22:36,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bdf575bc060ce15bc09'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:36,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bdf575bc060ce15bc09 2025-12-17 21:22:36,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdf575bc060ce15bc09 HTTP/1.1" 502 150 2025-12-17 21:22:36,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bdf575bc060ce15bc09'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:37,001 - lava_callback - INFO - Received callback for node 69407c0f575bc060ce15bc9b INFO: 172.21.0.1:47786 - "POST /node/69407c0f575bc060ce15bc9b HTTP/1.0" 202 Accepted 2025-12-17 21:22:37,764 - lava_callback - INFO - Received callback for node 69407c10575bc060ce15bca2 INFO: 172.21.0.1:47792 - "POST /node/69407c10575bc060ce15bca2 HTTP/1.0" 202 Accepted 2025-12-17 21:22:38,702 - lava_callback - INFO - Received callback for node 69407c11575bc060ce15bca7 INFO: 172.21.0.1:47804 - "POST /node/69407c11575bc060ce15bca7 HTTP/1.0" 202 Accepted 2025-12-17 21:22:38,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bb6575bc060ce15bb57 HTTP/1.1" 502 150 2025-12-17 21:22:38,896 - lava_callback - ERROR - Error processing callback for node 69407bb6575bc060ce15bb57: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb6575bc060ce15bb57 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb6575bc060ce15bb57 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bb6575bc060ce15bb57 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:38,898 - lava_callback - INFO - Starting async job submit for node 69407be0575bc060ce15bc0c 2025-12-17 21:22:38,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:38,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407be0575bc060ce15bc0c HTTP/1.1" 502 150 2025-12-17 21:22:38,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be0575bc060ce15bc0c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:38,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407be0575bc060ce15bc0c 2025-12-17 21:22:38,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407be0575bc060ce15bc0c HTTP/1.1" 502 150 2025-12-17 21:22:38,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be0575bc060ce15bc0c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:38,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bdf575bc060ce15bc09 2025-12-17 21:22:38,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdf575bc060ce15bc09 HTTP/1.1" 502 150 2025-12-17 21:22:38,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bdf575bc060ce15bc09'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:39,460 - lava_callback - INFO - Received callback for node 69407c13575bc060ce15bcae INFO: 172.21.0.1:47810 - "POST /node/69407c13575bc060ce15bcae HTTP/1.0" 202 Accepted 2025-12-17 21:22:39,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bda575bc060ce15bbf9 2025-12-17 21:22:39,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bda575bc060ce15bbf9 HTTP/1.1" 502 150 2025-12-17 21:22:39,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bda575bc060ce15bbf9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:39,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bdb575bc060ce15bc00 2025-12-17 21:22:39,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdb575bc060ce15bc00 HTTP/1.1" 502 150 2025-12-17 21:22:39,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bdb575bc060ce15bc00'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:40,193 - lava_callback - INFO - Received callback for node 69407c14575bc060ce15bcb1 INFO: 172.21.0.1:43976 - "POST /node/69407c14575bc060ce15bcb1 HTTP/1.0" 202 Accepted 2025-12-17 21:22:40,797 - lava_callback - INFO - Received callback for node 69407b14575bc060ce15b93c INFO: 172.21.0.1:43980 - "POST /node/69407b14575bc060ce15b93c HTTP/1.0" 202 Accepted 2025-12-17 21:22:40,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407be0575bc060ce15bc0c 2025-12-17 21:22:40,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407be0575bc060ce15bc0c HTTP/1.1" 502 150 2025-12-17 21:22:40,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be0575bc060ce15bc0c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:40,984 - lava_callback - INFO - Received callback for node 69407c15575bc060ce15bcb4 INFO: 172.21.0.1:43994 - "POST /node/69407c15575bc060ce15bcb4 HTTP/1.0" 202 Accepted 2025-12-17 21:22:41,756 - lava_callback - INFO - Received callback for node 69407c16575bc060ce15bcb7 INFO: 172.21.0.1:44008 - "POST /node/69407c16575bc060ce15bcb7 HTTP/1.0" 202 Accepted 2025-12-17 21:22:41,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bc9575bc060ce15bbc7 2025-12-17 21:22:41,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bc9575bc060ce15bbc7 HTTP/1.1" 502 150 2025-12-17 21:22:41,872 - lava_callback - ERROR - Error processing callback for node 69407bc9575bc060ce15bbc7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc9575bc060ce15bbc7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc9575bc060ce15bbc7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bc9575bc060ce15bbc7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:41,874 - lava_callback - INFO - Starting async job submit for node 69407be2575bc060ce15bc0f 2025-12-17 21:22:41,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:41,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd6575bc060ce15bbef 2025-12-17 21:22:41,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd6575bc060ce15bbef HTTP/1.1" 502 150 2025-12-17 21:22:41,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd6575bc060ce15bbef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:42,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407be2575bc060ce15bc0f HTTP/1.1" 502 150 2025-12-17 21:22:42,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be2575bc060ce15bc0f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:42,071 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407be2575bc060ce15bc0f 2025-12-17 21:22:42,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407be2575bc060ce15bc0f HTTP/1.1" 502 150 2025-12-17 21:22:42,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be2575bc060ce15bc0f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:42,526 - lava_callback - INFO - Received callback for node 69407c18575bc060ce15bcba INFO: 172.21.0.1:44024 - "POST /node/69407c18575bc060ce15bcba HTTP/1.0" 202 Accepted 2025-12-17 21:22:42,648 - lava_callback - INFO - Received callback for node 69407b15575bc060ce15b93f INFO: 172.21.0.1:44040 - "POST /node/69407b15575bc060ce15b93f HTTP/1.0" 202 Accepted 2025-12-17 21:22:42,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bdf575bc060ce15bc09 2025-12-17 21:22:42,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdf575bc060ce15bc09 HTTP/1.1" 502 150 2025-12-17 21:22:42,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bdf575bc060ce15bc09'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:43,357 - lava_callback - INFO - Received callback for node 69407c19575bc060ce15bcbd INFO: 172.21.0.1:44054 - "POST /node/69407c19575bc060ce15bcbd HTTP/1.0" 202 Accepted 2025-12-17 21:22:43,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd8575bc060ce15bbf3 2025-12-17 21:22:43,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd8575bc060ce15bbf3 HTTP/1.1" 502 150 2025-12-17 21:22:43,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd8575bc060ce15bbf3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:43,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd9575bc060ce15bbf6 2025-12-17 21:22:43,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd9575bc060ce15bbf6 HTTP/1.1" 502 150 2025-12-17 21:22:43,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bd9575bc060ce15bbf6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:44,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407be2575bc060ce15bc0f 2025-12-17 21:22:44,170 - lava_callback - INFO - Received callback for node 69407c1a575bc060ce15bcc0 INFO: 172.21.0.1:44058 - "POST /node/69407c1a575bc060ce15bcc0 HTTP/1.0" 202 Accepted 2025-12-17 21:22:44,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bca575bc060ce15bbcb HTTP/1.1" 502 150 2025-12-17 21:22:44,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bca575bc060ce15bbcb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:44,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407be0575bc060ce15bc0c 2025-12-17 21:22:44,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407be0575bc060ce15bc0c HTTP/1.1" 502 150 2025-12-17 21:22:44,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be0575bc060ce15bc0c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:44,945 - lava_callback - INFO - Received callback for node 69407c1b575bc060ce15bcc3 INFO: 172.21.0.1:44068 - "POST /node/69407c1b575bc060ce15bcc3 HTTP/1.0" 202 Accepted 2025-12-17 21:22:45,688 - lava_callback - INFO - Received callback for node 69407c1d575bc060ce15bcc6 INFO: 172.21.0.1:44080 - "POST /node/69407c1d575bc060ce15bcc6 HTTP/1.0" 202 Accepted 2025-12-17 21:22:46,472 - lava_callback - INFO - Received callback for node 69407c1e575bc060ce15bcc9 INFO: 172.21.0.1:44086 - "POST /node/69407c1e575bc060ce15bcc9 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:44094 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:22:46,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bcc575bc060ce15bbce 2025-12-17 21:22:46,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bcc575bc060ce15bbce HTTP/1.1" 502 150 2025-12-17 21:22:46,704 - lava_callback - ERROR - Error processing callback for node 69407bcc575bc060ce15bbce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bcc575bc060ce15bbce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bcc575bc060ce15bbce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bcc575bc060ce15bbce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:46,705 - lava_callback - INFO - Starting async job submit for node 69407bdc575bc060ce15bc03 2025-12-17 21:22:46,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:46,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdc575bc060ce15bc03 HTTP/1.1" 502 150 2025-12-17 21:22:46,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bdc575bc060ce15bc03'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:46,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bdc575bc060ce15bc03 2025-12-17 21:22:46,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdc575bc060ce15bc03 HTTP/1.1" 502 150 2025-12-17 21:22:46,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bdc575bc060ce15bc03'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:46,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bcd575bc060ce15bbd1 2025-12-17 21:22:46,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bcd575bc060ce15bbd1 HTTP/1.1" 502 150 2025-12-17 21:22:46,988 - lava_callback - ERROR - Error processing callback for node 69407bcd575bc060ce15bbd1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bcd575bc060ce15bbd1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bcd575bc060ce15bbd1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bcd575bc060ce15bbd1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:46,990 - lava_callback - INFO - Starting async job submit for node 69407bde575bc060ce15bc06 2025-12-17 21:22:46,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:46,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bde575bc060ce15bc06 HTTP/1.1" 502 150 2025-12-17 21:22:46,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bde575bc060ce15bc06'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:46,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bde575bc060ce15bc06 2025-12-17 21:22:46,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bde575bc060ce15bc06 HTTP/1.1" 502 150 2025-12-17 21:22:46,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bde575bc060ce15bc06'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:47,200 - lava_callback - INFO - Received callback for node 69407c1f575bc060ce15bccc INFO: 172.21.0.1:44102 - "POST /node/69407c1f575bc060ce15bccc HTTP/1.0" 202 Accepted 2025-12-17 21:22:47,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bda575bc060ce15bbf9 2025-12-17 21:22:47,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bda575bc060ce15bbf9 HTTP/1.1" 502 150 2025-12-17 21:22:47,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bda575bc060ce15bbf9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:47,947 - lava_callback - INFO - Received callback for node 69407c20575bc060ce15bccf INFO: 172.21.0.1:44110 - "POST /node/69407c20575bc060ce15bccf HTTP/1.0" 202 Accepted 2025-12-17 21:22:47,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bdb575bc060ce15bc00 2025-12-17 21:22:47,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdb575bc060ce15bc00 HTTP/1.1" 502 150 2025-12-17 21:22:47,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bdb575bc060ce15bc00'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:48,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bce575bc060ce15bbd4 2025-12-17 21:22:48,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bce575bc060ce15bbd4 HTTP/1.1" 502 150 2025-12-17 21:22:48,502 - lava_callback - ERROR - Error processing callback for node 69407bce575bc060ce15bbd4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bce575bc060ce15bbd4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bce575bc060ce15bbd4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bce575bc060ce15bbd4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:48,504 - lava_callback - INFO - Starting async job submit for node 69407d46575bc060ce15c030 2025-12-17 21:22:48,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:48,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd0575bc060ce15bbd7 2025-12-17 21:22:48,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd0575bc060ce15bbd7 HTTP/1.1" 502 150 2025-12-17 21:22:48,592 - lava_callback - ERROR - Error processing callback for node 69407bd0575bc060ce15bbd7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd0575bc060ce15bbd7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd0575bc060ce15bbd7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd0575bc060ce15bbd7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:48,595 - lava_callback - INFO - Starting async job submit for node 69407bf3575bc060ce15bc2e 2025-12-17 21:22:48,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:48,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf3575bc060ce15bc2e HTTP/1.1" 502 150 2025-12-17 21:22:48,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf3575bc060ce15bc2e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:48,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf3575bc060ce15bc2e 2025-12-17 21:22:48,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf3575bc060ce15bc2e HTTP/1.1" 502 150 2025-12-17 21:22:48,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf3575bc060ce15bc2e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:48,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d46575bc060ce15c030 HTTP/1.1" 502 150 2025-12-17 21:22:48,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d46575bc060ce15c030'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:48,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d46575bc060ce15c030 2025-12-17 21:22:48,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d46575bc060ce15c030 HTTP/1.1" 502 150 2025-12-17 21:22:48,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d46575bc060ce15c030'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:48,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bdc575bc060ce15bc03 2025-12-17 21:22:48,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdc575bc060ce15bc03 HTTP/1.1" 502 150 2025-12-17 21:22:48,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bdc575bc060ce15bc03'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:48,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bde575bc060ce15bc06 2025-12-17 21:22:49,185 - lava_callback - INFO - Received callback for node 69407c22575bc060ce15bcd5 INFO: 172.21.0.1:44112 - "POST /node/69407c22575bc060ce15bcd5 HTTP/1.0" 202 Accepted 2025-12-17 21:22:49,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd1575bc060ce15bbdc 2025-12-17 21:22:49,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd1575bc060ce15bbdc HTTP/1.1" 502 150 2025-12-17 21:22:49,903 - lava_callback - ERROR - Error processing callback for node 69407bd1575bc060ce15bbdc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd1575bc060ce15bbdc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd1575bc060ce15bbdc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd1575bc060ce15bbdc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:49,905 - lava_callback - INFO - Starting async job submit for node 69407bf1575bc060ce15bc28 2025-12-17 21:22:49,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:49,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf1575bc060ce15bc28 HTTP/1.1" 502 150 2025-12-17 21:22:49,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf1575bc060ce15bc28'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:49,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf1575bc060ce15bc28 2025-12-17 21:22:49,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf1575bc060ce15bc28 HTTP/1.1" 502 150 2025-12-17 21:22:49,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf1575bc060ce15bc28'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:49,963 - lava_callback - INFO - Received callback for node 69407c23575bc060ce15bcdb INFO: 172.21.0.1:46974 - "POST /node/69407c23575bc060ce15bcdb HTTP/1.0" 202 Accepted 2025-12-17 21:22:50,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf3575bc060ce15bc2e 2025-12-17 21:22:50,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf3575bc060ce15bc2e HTTP/1.1" 502 150 2025-12-17 21:22:50,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf3575bc060ce15bc2e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:50,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d46575bc060ce15c030 2025-12-17 21:22:50,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d46575bc060ce15c030 HTTP/1.1" 502 150 2025-12-17 21:22:50,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d46575bc060ce15c030'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:50,721 - lava_callback - INFO - Received callback for node 69407c25575bc060ce15bce0 INFO: 172.21.0.1:46988 - "POST /node/69407c25575bc060ce15bce0 HTTP/1.0" 202 Accepted 2025-12-17 21:22:50,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd2575bc060ce15bbe1 2025-12-17 21:22:50,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd2575bc060ce15bbe1 HTTP/1.1" 502 150 2025-12-17 21:22:50,796 - lava_callback - ERROR - Error processing callback for node 69407bd2575bc060ce15bbe1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd2575bc060ce15bbe1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd2575bc060ce15bbe1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd2575bc060ce15bbe1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:50,798 - lava_callback - INFO - Starting async job submit for node 69407bf2575bc060ce15bc2b 2025-12-17 21:22:50,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:50,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf2575bc060ce15bc2b HTTP/1.1" 502 150 2025-12-17 21:22:50,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf2575bc060ce15bc2b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:50,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf2575bc060ce15bc2b 2025-12-17 21:22:50,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf2575bc060ce15bc2b HTTP/1.1" 502 150 2025-12-17 21:22:50,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf2575bc060ce15bc2b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:50,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bdf575bc060ce15bc09 2025-12-17 21:22:50,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdf575bc060ce15bc09 HTTP/1.1" 502 150 2025-12-17 21:22:50,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bdf575bc060ce15bc09'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:51,480 - lava_callback - INFO - Received callback for node 69407c26575bc060ce15bce3 INFO: 172.21.0.1:46990 - "POST /node/69407c26575bc060ce15bce3 HTTP/1.0" 202 Accepted 2025-12-17 21:22:51,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf1575bc060ce15bc28 2025-12-17 21:22:51,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf1575bc060ce15bc28 HTTP/1.1" 502 150 2025-12-17 21:22:51,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf1575bc060ce15bc28'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:52,309 - lava_callback - INFO - Received callback for node 69407c27575bc060ce15bce6 INFO: 172.21.0.1:46992 - "POST /node/69407c27575bc060ce15bce6 HTTP/1.0" 202 Accepted 2025-12-17 21:22:52,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bdc575bc060ce15bc03 2025-12-17 21:22:52,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdc575bc060ce15bc03 HTTP/1.1" 502 150 2025-12-17 21:22:52,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bdc575bc060ce15bc03'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:52,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf2575bc060ce15bc2b 2025-12-17 21:22:52,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf2575bc060ce15bc2b HTTP/1.1" 502 150 2025-12-17 21:22:52,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf2575bc060ce15bc2b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:52,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd3575bc060ce15bbe5 2025-12-17 21:22:52,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd3575bc060ce15bbe5 HTTP/1.1" 502 150 2025-12-17 21:22:52,845 - lava_callback - ERROR - Error processing callback for node 69407bd3575bc060ce15bbe5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd3575bc060ce15bbe5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd3575bc060ce15bbe5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd3575bc060ce15bbe5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:52,847 - lava_callback - INFO - Starting async job submit for node 69407bf8575bc060ce15bc33 2025-12-17 21:22:52,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:52,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf8575bc060ce15bc33 HTTP/1.1" 502 150 2025-12-17 21:22:52,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf8575bc060ce15bc33'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:52,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf8575bc060ce15bc33 2025-12-17 21:22:52,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf8575bc060ce15bc33 HTTP/1.1" 502 150 2025-12-17 21:22:52,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf8575bc060ce15bc33'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:52,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407be0575bc060ce15bc0c 2025-12-17 21:22:52,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407be0575bc060ce15bc0c HTTP/1.1" 502 150 2025-12-17 21:22:52,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be0575bc060ce15bc0c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:53,055 - lava_callback - INFO - Received callback for node 69407c28575bc060ce15bce9 INFO: 172.21.0.1:47002 - "POST /node/69407c28575bc060ce15bce9 HTTP/1.0" 202 Accepted 2025-12-17 21:22:53,862 - lava_callback - INFO - Received callback for node 69407c29575bc060ce15bcec INFO: 172.21.0.1:47016 - "POST /node/69407c29575bc060ce15bcec HTTP/1.0" 202 Accepted 2025-12-17 21:22:54,622 - lava_callback - INFO - Received callback for node 69407c2a575bc060ce15bcf2 INFO: 172.21.0.1:47022 - "POST /node/69407c2a575bc060ce15bcf2 HTTP/1.0" 202 Accepted 2025-12-17 21:22:54,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf3575bc060ce15bc2e 2025-12-17 21:22:54,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf3575bc060ce15bc2e HTTP/1.1" 502 150 2025-12-17 21:22:54,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf3575bc060ce15bc2e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:54,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d46575bc060ce15c030 2025-12-17 21:22:54,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d46575bc060ce15c030 HTTP/1.1" 502 150 2025-12-17 21:22:54,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d46575bc060ce15c030'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:54,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf8575bc060ce15bc33 2025-12-17 21:22:54,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf8575bc060ce15bc33 HTTP/1.1" 502 150 2025-12-17 21:22:54,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf8575bc060ce15bc33'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:55,358 - lava_callback - INFO - Received callback for node 69407c2c575bc060ce15bcf5 INFO: 172.21.0.1:47038 - "POST /node/69407c2c575bc060ce15bcf5 HTTP/1.0" 202 Accepted 2025-12-17 21:22:55,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf1575bc060ce15bc28 2025-12-17 21:22:55,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf1575bc060ce15bc28 HTTP/1.1" 502 150 2025-12-17 21:22:55,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf1575bc060ce15bc28'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47048 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:22:56,802 - lava_callback - INFO - Received callback for node 69408c16575bc060ce15f454 INFO: 172.21.0.1:47052 - "POST /node/69408c16575bc060ce15f454 HTTP/1.0" 202 Accepted 2025-12-17 21:22:56,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf2575bc060ce15bc2b 2025-12-17 21:22:56,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf2575bc060ce15bc2b HTTP/1.1" 502 150 2025-12-17 21:22:56,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf2575bc060ce15bc2b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:57,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd6575bc060ce15bbef 2025-12-17 21:22:57,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd6575bc060ce15bbef HTTP/1.1" 502 150 2025-12-17 21:22:57,999 - lava_callback - ERROR - Error processing callback for node 69407bd6575bc060ce15bbef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd6575bc060ce15bbef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd6575bc060ce15bbef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd6575bc060ce15bbef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:58,002 - lava_callback - INFO - Starting async job submit for node 69407bfb575bc060ce15bc3a 2025-12-17 21:22:58,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:58,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bfb575bc060ce15bc3a HTTP/1.1" 502 150 2025-12-17 21:22:58,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bfb575bc060ce15bc3a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:58,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bfb575bc060ce15bc3a 2025-12-17 21:22:58,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bfb575bc060ce15bc3a HTTP/1.1" 502 150 2025-12-17 21:22:58,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bfb575bc060ce15bc3a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:58,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf8575bc060ce15bc33 2025-12-17 21:22:58,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf8575bc060ce15bc33 HTTP/1.1" 502 150 2025-12-17 21:22:58,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf8575bc060ce15bc33'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:59,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407be2575bc060ce15bc0f HTTP/1.1" 502 150 2025-12-17 21:22:59,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be2575bc060ce15bc0f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:59,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd8575bc060ce15bbf3 2025-12-17 21:22:59,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd8575bc060ce15bbf3 HTTP/1.1" 502 150 2025-12-17 21:22:59,551 - lava_callback - ERROR - Error processing callback for node 69407bd8575bc060ce15bbf3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd8575bc060ce15bbf3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd8575bc060ce15bbf3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd8575bc060ce15bbf3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:59,552 - lava_callback - INFO - Starting async job submit for node 69407c01575bc060ce15bc3f 2025-12-17 21:22:59,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:59,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c01575bc060ce15bc3f HTTP/1.1" 502 150 2025-12-17 21:22:59,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c01575bc060ce15bc3f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:59,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c01575bc060ce15bc3f 2025-12-17 21:22:59,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c01575bc060ce15bc3f HTTP/1.1" 502 150 2025-12-17 21:22:59,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c01575bc060ce15bc3f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:59,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bd9575bc060ce15bbf6 2025-12-17 21:22:59,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bd9575bc060ce15bbf6 HTTP/1.1" 502 150 2025-12-17 21:22:59,662 - lava_callback - ERROR - Error processing callback for node 69407bd9575bc060ce15bbf6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd9575bc060ce15bbf6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd9575bc060ce15bbf6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bd9575bc060ce15bbf6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:22:59,664 - lava_callback - INFO - Starting async job submit for node 69407c03575bc060ce15bc44 2025-12-17 21:22:59,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:22:59,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c03575bc060ce15bc44 HTTP/1.1" 502 150 2025-12-17 21:22:59,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c03575bc060ce15bc44'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:22:59,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c03575bc060ce15bc44 2025-12-17 21:22:59,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c03575bc060ce15bc44 HTTP/1.1" 502 150 2025-12-17 21:22:59,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c03575bc060ce15bc44'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:00,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bfb575bc060ce15bc3a 2025-12-17 21:23:00,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bca575bc060ce15bbcb 2025-12-17 21:23:00,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bca575bc060ce15bbcb HTTP/1.1" 502 150 2025-12-17 21:23:00,533 - lava_callback - ERROR - Error processing callback for node 69407bca575bc060ce15bbcb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bca575bc060ce15bbcb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bca575bc060ce15bbcb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bca575bc060ce15bbcb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:00,535 - lava_callback - INFO - Starting async job submit for node 69407b12575bc060ce15b939 2025-12-17 21:23:00,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:00,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b12575bc060ce15b939 HTTP/1.1" 502 150 2025-12-17 21:23:00,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b12575bc060ce15b939'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:00,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b12575bc060ce15b939 2025-12-17 21:23:00,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b12575bc060ce15b939 HTTP/1.1" 502 150 2025-12-17 21:23:00,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b12575bc060ce15b939'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:00,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bdc575bc060ce15bc03 2025-12-17 21:23:00,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdc575bc060ce15bc03 HTTP/1.1" 502 150 2025-12-17 21:23:00,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bdc575bc060ce15bc03'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48324 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:23:01,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c01575bc060ce15bc3f 2025-12-17 21:23:01,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c01575bc060ce15bc3f HTTP/1.1" 502 150 2025-12-17 21:23:01,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c01575bc060ce15bc3f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:01,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c03575bc060ce15bc44 2025-12-17 21:23:01,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c03575bc060ce15bc44 HTTP/1.1" 502 150 2025-12-17 21:23:01,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c03575bc060ce15bc44'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:02,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf3575bc060ce15bc2e 2025-12-17 21:23:02,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf3575bc060ce15bc2e HTTP/1.1" 502 150 2025-12-17 21:23:02,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf3575bc060ce15bc2e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:02,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b12575bc060ce15b939 2025-12-17 21:23:02,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b12575bc060ce15b939 HTTP/1.1" 502 150 2025-12-17 21:23:02,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b12575bc060ce15b939'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:02,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d46575bc060ce15c030 2025-12-17 21:23:02,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d46575bc060ce15c030 HTTP/1.1" 502 150 2025-12-17 21:23:02,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d46575bc060ce15c030'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:03,377 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407be2575bc060ce15bc0f 2025-12-17 21:23:03,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407be2575bc060ce15bc0f HTTP/1.1" 502 150 2025-12-17 21:23:03,378 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be2575bc060ce15bc0f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:03,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bda575bc060ce15bbf9 2025-12-17 21:23:03,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bda575bc060ce15bbf9 HTTP/1.1" 502 150 2025-12-17 21:23:03,850 - lava_callback - ERROR - Error processing callback for node 69407bda575bc060ce15bbf9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bda575bc060ce15bbf9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bda575bc060ce15bbf9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bda575bc060ce15bbf9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:03,852 - lava_callback - INFO - Starting async job submit for node 69407d47575bc060ce15c033 2025-12-17 21:23:03,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:03,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d47575bc060ce15c033 HTTP/1.1" 502 150 2025-12-17 21:23:03,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d47575bc060ce15c033'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:03,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d47575bc060ce15c033 2025-12-17 21:23:03,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d47575bc060ce15c033 HTTP/1.1" 502 150 2025-12-17 21:23:03,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d47575bc060ce15c033'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:03,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf1575bc060ce15bc28 2025-12-17 21:23:03,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf1575bc060ce15bc28 HTTP/1.1" 502 150 2025-12-17 21:23:03,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf1575bc060ce15bc28'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:03,946 - lava_callback - INFO - Received callback for node 69409356575bc060ce160d16 INFO: 172.21.0.1:48328 - "POST /node/69409356575bc060ce160d16 HTTP/1.0" 202 Accepted 2025-12-17 21:23:03,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bdb575bc060ce15bc00 2025-12-17 21:23:03,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdb575bc060ce15bc00 HTTP/1.1" 502 150 2025-12-17 21:23:03,960 - lava_callback - ERROR - Error processing callback for node 69407bdb575bc060ce15bc00: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bdb575bc060ce15bc00 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bdb575bc060ce15bc00 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bdb575bc060ce15bc00 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:03,962 - lava_callback - INFO - Starting async job submit for node 69408c07575bc060ce15f42b 2025-12-17 21:23:04,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:04,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c07575bc060ce15f42b HTTP/1.1" 502 150 2025-12-17 21:23:04,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c07575bc060ce15f42b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:04,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c07575bc060ce15f42b 2025-12-17 21:23:04,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c07575bc060ce15f42b HTTP/1.1" 502 150 2025-12-17 21:23:04,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c07575bc060ce15f42b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:04,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bde575bc060ce15bc06 HTTP/1.1" 502 150 2025-12-17 21:23:04,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bde575bc060ce15bc06'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:04,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf2575bc060ce15bc2b 2025-12-17 21:23:04,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf2575bc060ce15bc2b HTTP/1.1" 502 150 2025-12-17 21:23:04,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf2575bc060ce15bc2b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:05,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c01575bc060ce15bc3f 2025-12-17 21:23:05,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c01575bc060ce15bc3f HTTP/1.1" 502 150 2025-12-17 21:23:05,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c01575bc060ce15bc3f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:05,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c03575bc060ce15bc44 2025-12-17 21:23:05,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c03575bc060ce15bc44 HTTP/1.1" 502 150 2025-12-17 21:23:05,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c03575bc060ce15bc44'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:05,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d47575bc060ce15c033 2025-12-17 21:23:05,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d47575bc060ce15c033 HTTP/1.1" 502 150 2025-12-17 21:23:05,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d47575bc060ce15c033'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:06,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c07575bc060ce15f42b 2025-12-17 21:23:06,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c07575bc060ce15f42b HTTP/1.1" 502 150 2025-12-17 21:23:06,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c07575bc060ce15f42b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48330 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:23:06,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b12575bc060ce15b939 2025-12-17 21:23:06,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b12575bc060ce15b939 HTTP/1.1" 502 150 2025-12-17 21:23:06,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b12575bc060ce15b939'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:06,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf8575bc060ce15bc33 2025-12-17 21:23:06,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf8575bc060ce15bc33 HTTP/1.1" 502 150 2025-12-17 21:23:06,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bf8575bc060ce15bc33'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:06,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bdf575bc060ce15bc09 2025-12-17 21:23:06,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdf575bc060ce15bc09 HTTP/1.1" 502 150 2025-12-17 21:23:06,973 - lava_callback - ERROR - Error processing callback for node 69407bdf575bc060ce15bc09: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bdf575bc060ce15bc09 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bdf575bc060ce15bc09 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bdf575bc060ce15bc09 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:06,976 - lava_callback - INFO - Starting async job submit for node 69407c0b575bc060ce15bc8e 2025-12-17 21:23:06,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:06,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0b575bc060ce15bc8e HTTP/1.1" 502 150 2025-12-17 21:23:06,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0b575bc060ce15bc8e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:06,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0b575bc060ce15bc8e 2025-12-17 21:23:06,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0b575bc060ce15bc8e HTTP/1.1" 502 150 2025-12-17 21:23:06,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0b575bc060ce15bc8e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:08,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bde575bc060ce15bc06 2025-12-17 21:23:08,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bde575bc060ce15bc06 HTTP/1.1" 502 150 2025-12-17 21:23:08,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bde575bc060ce15bc06'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:08,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407be0575bc060ce15bc0c 2025-12-17 21:23:08,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407be0575bc060ce15bc0c HTTP/1.1" 502 150 2025-12-17 21:23:08,911 - lava_callback - ERROR - Error processing callback for node 69407be0575bc060ce15bc0c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407be0575bc060ce15bc0c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407be0575bc060ce15bc0c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407be0575bc060ce15bc0c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:08,913 - lava_callback - INFO - Starting async job submit for node 69407c0c575bc060ce15bc91 2025-12-17 21:23:08,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:08,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0c575bc060ce15bc91 HTTP/1.1" 502 150 2025-12-17 21:23:08,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0c575bc060ce15bc91'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:08,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0c575bc060ce15bc91 2025-12-17 21:23:08,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0c575bc060ce15bc91 HTTP/1.1" 502 150 2025-12-17 21:23:08,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0c575bc060ce15bc91'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:08,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0b575bc060ce15bc8e 2025-12-17 21:23:08,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0b575bc060ce15bc8e HTTP/1.1" 502 150 2025-12-17 21:23:08,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0b575bc060ce15bc8e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:09,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d47575bc060ce15c033 2025-12-17 21:23:09,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d47575bc060ce15c033 HTTP/1.1" 502 150 2025-12-17 21:23:09,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d47575bc060ce15c033'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:10,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c07575bc060ce15f42b 2025-12-17 21:23:10,577 - lava_callback - INFO - Received callback for node 69407c2d575bc060ce15bcfc INFO: 172.21.0.1:44732 - "POST /node/69407c2d575bc060ce15bcfc HTTP/1.0" 202 Accepted 2025-12-17 21:23:10,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0c575bc060ce15bc91 2025-12-17 21:23:10,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0c575bc060ce15bc91 HTTP/1.1" 502 150 2025-12-17 21:23:10,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0c575bc060ce15bc91'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:11,184 - lava_callback - INFO - Received callback for node 69407c2e575bc060ce15bcff INFO: 172.21.0.1:44742 - "POST /node/69407c2e575bc060ce15bcff HTTP/1.0" 202 Accepted 2025-12-17 21:23:11,378 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407be2575bc060ce15bc0f 2025-12-17 21:23:11,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407be2575bc060ce15bc0f HTTP/1.1" 502 150 2025-12-17 21:23:11,379 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407be2575bc060ce15bc0f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:11,799 - lava_callback - INFO - Received callback for node 69407c38575bc060ce15bd1a INFO: 172.21.0.1:44752 - "POST /node/69407c38575bc060ce15bd1a HTTP/1.0" 202 Accepted 2025-12-17 21:23:12,432 - lava_callback - INFO - Received callback for node 69407c3b575bc060ce15bd27 INFO: 172.21.0.1:44762 - "POST /node/69407c3b575bc060ce15bd27 HTTP/1.0" 202 Accepted 2025-12-17 21:23:12,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0b575bc060ce15bc8e 2025-12-17 21:23:12,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0b575bc060ce15bc8e HTTP/1.1" 502 150 2025-12-17 21:23:12,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0b575bc060ce15bc8e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:13,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c01575bc060ce15bc3f 2025-12-17 21:23:13,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c01575bc060ce15bc3f HTTP/1.1" 502 150 2025-12-17 21:23:13,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c01575bc060ce15bc3f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:13,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c03575bc060ce15bc44 2025-12-17 21:23:13,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c03575bc060ce15bc44 HTTP/1.1" 502 150 2025-12-17 21:23:13,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c03575bc060ce15bc44'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:14,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b12575bc060ce15b939 2025-12-17 21:23:14,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b12575bc060ce15b939 HTTP/1.1" 502 150 2025-12-17 21:23:14,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b12575bc060ce15b939'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:14,724 - lava_callback - INFO - Received callback for node 69408c32575bc060ce15f4a9 INFO: 172.21.0.1:44778 - "POST /node/69408c32575bc060ce15f4a9 HTTP/1.0" 202 Accepted 2025-12-17 21:23:14,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0c575bc060ce15bc91 2025-12-17 21:23:14,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0c575bc060ce15bc91 HTTP/1.1" 502 150 2025-12-17 21:23:14,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0c575bc060ce15bc91'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44792 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:23:15,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bfb575bc060ce15bc3a HTTP/1.1" 502 150 2025-12-17 21:23:15,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bfb575bc060ce15bc3a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:16,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bde575bc060ce15bc06 2025-12-17 21:23:16,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bde575bc060ce15bc06 HTTP/1.1" 502 150 2025-12-17 21:23:16,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bde575bc060ce15bc06'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44806 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:23:16,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bdc575bc060ce15bc03 2025-12-17 21:23:16,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bdc575bc060ce15bc03 HTTP/1.1" 502 150 2025-12-17 21:23:16,720 - lava_callback - ERROR - Error processing callback for node 69407bdc575bc060ce15bc03: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bdc575bc060ce15bc03 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bdc575bc060ce15bc03 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bdc575bc060ce15bc03 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:16,722 - lava_callback - INFO - Starting async job submit for node 69407c0e575bc060ce15bc97 2025-12-17 21:23:16,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:16,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0e575bc060ce15bc97 HTTP/1.1" 502 150 2025-12-17 21:23:16,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0e575bc060ce15bc97'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:16,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0e575bc060ce15bc97 2025-12-17 21:23:16,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0e575bc060ce15bc97 HTTP/1.1" 502 150 2025-12-17 21:23:16,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0e575bc060ce15bc97'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:16,960 - lava_callback - INFO - Received callback for node 69407c37575bc060ce15bd13 INFO: 172.21.0.1:44818 - "POST /node/69407c37575bc060ce15bd13 HTTP/1.0" 202 Accepted 2025-12-17 21:23:17,698 - lava_callback - INFO - Received callback for node 69407c3a575bc060ce15bd20 INFO: 172.21.0.1:44822 - "POST /node/69407c3a575bc060ce15bd20 HTTP/1.0" 202 Accepted 2025-12-17 21:23:17,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d47575bc060ce15c033 2025-12-17 21:23:17,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d47575bc060ce15c033 HTTP/1.1" 502 150 2025-12-17 21:23:17,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d47575bc060ce15c033'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:18,496 - lava_callback - INFO - Received callback for node 69407c3d575bc060ce15bd2b INFO: 172.21.0.1:44826 - "POST /node/69407c3d575bc060ce15bd2b HTTP/1.0" 202 Accepted 2025-12-17 21:23:18,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf3575bc060ce15bc2e 2025-12-17 21:23:18,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf3575bc060ce15bc2e HTTP/1.1" 502 150 2025-12-17 21:23:18,632 - lava_callback - ERROR - Error processing callback for node 69407bf3575bc060ce15bc2e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bf3575bc060ce15bc2e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bf3575bc060ce15bc2e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bf3575bc060ce15bc2e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:18,633 - lava_callback - INFO - Starting async job submit for node 69407c0f575bc060ce15bc9b 2025-12-17 21:23:18,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:18,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0f575bc060ce15bc9b HTTP/1.1" 502 150 2025-12-17 21:23:18,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0f575bc060ce15bc9b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:18,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0f575bc060ce15bc9b 2025-12-17 21:23:18,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0f575bc060ce15bc9b HTTP/1.1" 502 150 2025-12-17 21:23:18,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0f575bc060ce15bc9b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:18,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d46575bc060ce15c030 2025-12-17 21:23:18,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d46575bc060ce15c030 HTTP/1.1" 502 150 2025-12-17 21:23:18,705 - lava_callback - ERROR - Error processing callback for node 69407d46575bc060ce15c030: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d46575bc060ce15c030 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d46575bc060ce15c030 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d46575bc060ce15c030 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:18,707 - lava_callback - INFO - Starting async job submit for node 69407c10575bc060ce15bca2 2025-12-17 21:23:18,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:18,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c10575bc060ce15bca2 HTTP/1.1" 502 150 2025-12-17 21:23:18,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c10575bc060ce15bca2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:18,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c10575bc060ce15bca2 2025-12-17 21:23:18,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c10575bc060ce15bca2 HTTP/1.1" 502 150 2025-12-17 21:23:18,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c10575bc060ce15bca2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:18,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0e575bc060ce15bc97 2025-12-17 21:23:18,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0e575bc060ce15bc97 HTTP/1.1" 502 150 2025-12-17 21:23:18,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0e575bc060ce15bc97'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:19,266 - lava_callback - INFO - Received callback for node 69407c3e575bc060ce15bd2e INFO: 172.21.0.1:44830 - "POST /node/69407c3e575bc060ce15bd2e HTTP/1.0" 202 Accepted 2025-12-17 21:23:19,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bfb575bc060ce15bc3a 2025-12-17 21:23:19,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bfb575bc060ce15bc3a HTTP/1.1" 502 150 2025-12-17 21:23:19,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bfb575bc060ce15bc3a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:19,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf1575bc060ce15bc28 2025-12-17 21:23:19,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf1575bc060ce15bc28 HTTP/1.1" 502 150 2025-12-17 21:23:19,919 - lava_callback - ERROR - Error processing callback for node 69407bf1575bc060ce15bc28: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bf1575bc060ce15bc28 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bf1575bc060ce15bc28 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bf1575bc060ce15bc28 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:19,921 - lava_callback - INFO - Starting async job submit for node 69407c11575bc060ce15bca7 2025-12-17 21:23:19,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:19,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c11575bc060ce15bca7 HTTP/1.1" 502 150 2025-12-17 21:23:19,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c11575bc060ce15bca7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:19,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c11575bc060ce15bca7 2025-12-17 21:23:19,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c11575bc060ce15bca7 HTTP/1.1" 502 150 2025-12-17 21:23:19,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c11575bc060ce15bca7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:20,102 - lava_callback - INFO - Received callback for node 69407c3f575bc060ce15bd31 INFO: 172.21.0.1:51590 - "POST /node/69407c3f575bc060ce15bd31 HTTP/1.0" 202 Accepted 2025-12-17 21:23:20,124 - lava_callback - INFO - Received callback for node 69407b4f575bc060ce15b9db INFO: 172.21.0.1:51606 - "POST /node/69407b4f575bc060ce15b9db HTTP/1.0" 202 Accepted 2025-12-17 21:23:20,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0f575bc060ce15bc9b 2025-12-17 21:23:20,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0f575bc060ce15bc9b HTTP/1.1" 502 150 2025-12-17 21:23:20,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0f575bc060ce15bc9b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:20,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c10575bc060ce15bca2 2025-12-17 21:23:20,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c10575bc060ce15bca2 HTTP/1.1" 502 150 2025-12-17 21:23:20,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c10575bc060ce15bca2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:20,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf2575bc060ce15bc2b 2025-12-17 21:23:20,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf2575bc060ce15bc2b HTTP/1.1" 502 150 2025-12-17 21:23:20,814 - lava_callback - ERROR - Error processing callback for node 69407bf2575bc060ce15bc2b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bf2575bc060ce15bc2b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bf2575bc060ce15bc2b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bf2575bc060ce15bc2b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:20,817 - lava_callback - INFO - Starting async job submit for node 69407c13575bc060ce15bcae 2025-12-17 21:23:20,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:20,851 - lava_callback - INFO - Received callback for node 69407c41575bc060ce15bd34 INFO: 172.21.0.1:51608 - "POST /node/69407c41575bc060ce15bd34 HTTP/1.0" 202 Accepted 2025-12-17 21:23:20,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0b575bc060ce15bc8e 2025-12-17 21:23:20,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0b575bc060ce15bc8e HTTP/1.1" 502 150 2025-12-17 21:23:20,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0b575bc060ce15bc8e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:21,610 - lava_callback - INFO - Received callback for node 69407c45575bc060ce15bd6c INFO: 172.21.0.1:51614 - "POST /node/69407c45575bc060ce15bd6c HTTP/1.0" 202 Accepted 2025-12-17 21:23:21,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c11575bc060ce15bca7 2025-12-17 21:23:21,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c11575bc060ce15bca7 HTTP/1.1" 502 150 2025-12-17 21:23:21,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c11575bc060ce15bca7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:22,365 - lava_callback - INFO - Received callback for node 69407c46575bc060ce15bd76 INFO: 172.21.0.1:51622 - "POST /node/69407c46575bc060ce15bd76 HTTP/1.0" 202 Accepted 2025-12-17 21:23:22,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bf8575bc060ce15bc33 2025-12-17 21:23:22,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bf8575bc060ce15bc33 HTTP/1.1" 502 150 2025-12-17 21:23:22,860 - lava_callback - ERROR - Error processing callback for node 69407bf8575bc060ce15bc33: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bf8575bc060ce15bc33 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bf8575bc060ce15bc33 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bf8575bc060ce15bc33 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:22,862 - lava_callback - INFO - Starting async job submit for node 69407c14575bc060ce15bcb1 2025-12-17 21:23:22,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:22,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c14575bc060ce15bcb1 HTTP/1.1" 502 150 2025-12-17 21:23:22,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c14575bc060ce15bcb1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:22,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c14575bc060ce15bcb1 2025-12-17 21:23:22,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c14575bc060ce15bcb1 HTTP/1.1" 502 150 2025-12-17 21:23:22,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c14575bc060ce15bcb1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:22,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0e575bc060ce15bc97 2025-12-17 21:23:22,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0e575bc060ce15bc97 HTTP/1.1" 502 150 2025-12-17 21:23:22,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0e575bc060ce15bc97'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:22,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0c575bc060ce15bc91 2025-12-17 21:23:22,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0c575bc060ce15bc91 HTTP/1.1" 502 150 2025-12-17 21:23:22,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0c575bc060ce15bc91'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:23,111 - lava_callback - INFO - Received callback for node 69407c47575bc060ce15bd79 INFO: 172.21.0.1:51628 - "POST /node/69407c47575bc060ce15bd79 HTTP/1.0" 202 Accepted 2025-12-17 21:23:23,868 - lava_callback - INFO - Received callback for node 69407c49575bc060ce15bd7e INFO: 172.21.0.1:51644 - "POST /node/69407c49575bc060ce15bd7e HTTP/1.0" 202 Accepted 2025-12-17 21:23:24,642 - lava_callback - INFO - Received callback for node 69407c4a575bc060ce15bd85 INFO: 172.21.0.1:51650 - "POST /node/69407c4a575bc060ce15bd85 HTTP/1.0" 202 Accepted 2025-12-17 21:23:24,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0f575bc060ce15bc9b 2025-12-17 21:23:24,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0f575bc060ce15bc9b HTTP/1.1" 502 150 2025-12-17 21:23:24,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0f575bc060ce15bc9b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:24,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c10575bc060ce15bca2 2025-12-17 21:23:24,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c10575bc060ce15bca2 HTTP/1.1" 502 150 2025-12-17 21:23:24,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c10575bc060ce15bca2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:24,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c14575bc060ce15bcb1 2025-12-17 21:23:24,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c14575bc060ce15bcb1 HTTP/1.1" 502 150 2025-12-17 21:23:24,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c14575bc060ce15bcb1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:25,418 - lava_callback - INFO - Received callback for node 69407c4b575bc060ce15bd8c INFO: 172.21.0.1:51654 - "POST /node/69407c4b575bc060ce15bd8c HTTP/1.0" 202 Accepted 2025-12-17 21:23:25,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c07575bc060ce15f42b HTTP/1.1" 502 150 2025-12-17 21:23:25,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c07575bc060ce15f42b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:25,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c11575bc060ce15bca7 2025-12-17 21:23:25,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c11575bc060ce15bca7 HTTP/1.1" 502 150 2025-12-17 21:23:25,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c11575bc060ce15bca7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:26,204 - lava_callback - INFO - Received callback for node 69407c4d575bc060ce15bd91 INFO: 172.21.0.1:51666 - "POST /node/69407c4d575bc060ce15bd91 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:51682 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:23:27,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407be2575bc060ce15bc0f 2025-12-17 21:23:27,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bfb575bc060ce15bc3a 2025-12-17 21:23:27,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bfb575bc060ce15bc3a HTTP/1.1" 502 150 2025-12-17 21:23:27,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407bfb575bc060ce15bc3a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:28,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c14575bc060ce15bcb1 2025-12-17 21:23:28,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c14575bc060ce15bcb1 HTTP/1.1" 502 150 2025-12-17 21:23:28,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c14575bc060ce15bcb1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:29,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c01575bc060ce15bc3f 2025-12-17 21:23:29,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c01575bc060ce15bc3f HTTP/1.1" 502 150 2025-12-17 21:23:29,565 - lava_callback - ERROR - Error processing callback for node 69407c01575bc060ce15bc3f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c01575bc060ce15bc3f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c01575bc060ce15bc3f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c01575bc060ce15bc3f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:29,567 - lava_callback - INFO - Starting async job submit for node 69407b14575bc060ce15b93c 2025-12-17 21:23:29,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:29,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c03575bc060ce15bc44 2025-12-17 21:23:29,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c03575bc060ce15bc44 HTTP/1.1" 502 150 2025-12-17 21:23:29,677 - lava_callback - ERROR - Error processing callback for node 69407c03575bc060ce15bc44: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c03575bc060ce15bc44 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c03575bc060ce15bc44 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c03575bc060ce15bc44 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:29,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b14575bc060ce15b93c HTTP/1.1" 502 150 2025-12-17 21:23:29,679 - lava_callback - INFO - Starting async job submit for node 69407c15575bc060ce15bcb4 2025-12-17 21:23:29,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b14575bc060ce15b93c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:29,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:29,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b14575bc060ce15b93c 2025-12-17 21:23:29,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b14575bc060ce15b93c HTTP/1.1" 502 150 2025-12-17 21:23:29,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b14575bc060ce15b93c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:29,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c15575bc060ce15bcb4 HTTP/1.1" 502 150 2025-12-17 21:23:29,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c15575bc060ce15bcb4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:29,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c15575bc060ce15bcb4 2025-12-17 21:23:29,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c15575bc060ce15bcb4 HTTP/1.1" 502 150 2025-12-17 21:23:29,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c15575bc060ce15bcb4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:30,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b12575bc060ce15b939 2025-12-17 21:23:30,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b12575bc060ce15b939 HTTP/1.1" 502 150 2025-12-17 21:23:30,684 - lava_callback - ERROR - Error processing callback for node 69407b12575bc060ce15b939: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b12575bc060ce15b939 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b12575bc060ce15b939 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b12575bc060ce15b939 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:30,686 - lava_callback - INFO - Starting async job submit for node 69407c16575bc060ce15bcb7 2025-12-17 21:23:30,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:30,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c16575bc060ce15bcb7 HTTP/1.1" 502 150 2025-12-17 21:23:30,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c16575bc060ce15bcb7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:30,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c16575bc060ce15bcb7 2025-12-17 21:23:30,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c16575bc060ce15bcb7 HTTP/1.1" 502 150 2025-12-17 21:23:30,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c16575bc060ce15bcb7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:30,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0e575bc060ce15bc97 2025-12-17 21:23:30,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0e575bc060ce15bc97 HTTP/1.1" 502 150 2025-12-17 21:23:30,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0e575bc060ce15bc97'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:30,933 - lava_callback - INFO - Received callback for node 69407e35575bc060ce15c274 INFO: 172.21.0.1:48380 - "POST /node/69407e35575bc060ce15c274 HTTP/1.0" 202 Accepted 2025-12-17 21:23:31,321 - lava_callback - INFO - Received callback for node 69407d48575bc060ce15c036 INFO: 172.21.0.1:48390 - "POST /node/69407d48575bc060ce15c036 HTTP/1.0" 202 Accepted 2025-12-17 21:23:31,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b14575bc060ce15b93c 2025-12-17 21:23:31,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b14575bc060ce15b93c HTTP/1.1" 502 150 2025-12-17 21:23:31,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b14575bc060ce15b93c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:31,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c15575bc060ce15bcb4 2025-12-17 21:23:31,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c15575bc060ce15bcb4 HTTP/1.1" 502 150 2025-12-17 21:23:31,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c15575bc060ce15bcb4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:32,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bde575bc060ce15bc06 2025-12-17 21:23:32,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bde575bc060ce15bc06 HTTP/1.1" 502 150 2025-12-17 21:23:32,501 - lava_callback - ERROR - Error processing callback for node 69407bde575bc060ce15bc06: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bde575bc060ce15bc06 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bde575bc060ce15bc06 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bde575bc060ce15bc06 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:32,503 - lava_callback - INFO - Starting async job submit for node 69407c18575bc060ce15bcba 2025-12-17 21:23:32,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:32,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c18575bc060ce15bcba HTTP/1.1" 502 150 2025-12-17 21:23:32,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c18575bc060ce15bcba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:32,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c18575bc060ce15bcba 2025-12-17 21:23:32,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c18575bc060ce15bcba HTTP/1.1" 502 150 2025-12-17 21:23:32,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c18575bc060ce15bcba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:32,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0f575bc060ce15bc9b 2025-12-17 21:23:32,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0f575bc060ce15bc9b HTTP/1.1" 502 150 2025-12-17 21:23:32,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0f575bc060ce15bc9b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:32,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c16575bc060ce15bcb7 2025-12-17 21:23:32,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c16575bc060ce15bcb7 HTTP/1.1" 502 150 2025-12-17 21:23:32,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c16575bc060ce15bcb7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:32,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c10575bc060ce15bca2 2025-12-17 21:23:32,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c10575bc060ce15bca2 HTTP/1.1" 502 150 2025-12-17 21:23:32,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c10575bc060ce15bca2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:33,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c07575bc060ce15f42b 2025-12-17 21:23:33,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c07575bc060ce15f42b HTTP/1.1" 502 150 2025-12-17 21:23:33,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c07575bc060ce15f42b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:33,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d47575bc060ce15c033 2025-12-17 21:23:33,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d47575bc060ce15c033 HTTP/1.1" 502 150 2025-12-17 21:23:33,868 - lava_callback - ERROR - Error processing callback for node 69407d47575bc060ce15c033: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d47575bc060ce15c033 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d47575bc060ce15c033 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d47575bc060ce15c033 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:33,871 - lava_callback - INFO - Starting async job submit for node 69407b15575bc060ce15b93f 2025-12-17 21:23:33,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c11575bc060ce15bca7 2025-12-17 21:23:33,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c11575bc060ce15bca7 HTTP/1.1" 502 150 2025-12-17 21:23:33,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c11575bc060ce15bca7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:34,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:34,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b15575bc060ce15b93f HTTP/1.1" 502 150 2025-12-17 21:23:34,248 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b15575bc060ce15b93f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:34,249 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b15575bc060ce15b93f 2025-12-17 21:23:34,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b15575bc060ce15b93f HTTP/1.1" 502 150 2025-12-17 21:23:34,250 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b15575bc060ce15b93f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:34,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c18575bc060ce15bcba 2025-12-17 21:23:34,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c18575bc060ce15bcba HTTP/1.1" 502 150 2025-12-17 21:23:34,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c18575bc060ce15bcba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:35,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b14575bc060ce15b93c 2025-12-17 21:23:35,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b14575bc060ce15b93c HTTP/1.1" 502 150 2025-12-17 21:23:35,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b14575bc060ce15b93c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:35,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c15575bc060ce15bcb4 2025-12-17 21:23:35,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c15575bc060ce15bcb4 HTTP/1.1" 502 150 2025-12-17 21:23:35,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c15575bc060ce15bcb4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:36,250 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b15575bc060ce15b93f 2025-12-17 21:23:36,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c13575bc060ce15bcae HTTP/1.1" 502 150 2025-12-17 21:23:36,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c13575bc060ce15bcae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:36,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c13575bc060ce15bcae 2025-12-17 21:23:36,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c13575bc060ce15bcae HTTP/1.1" 502 150 2025-12-17 21:23:36,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c13575bc060ce15bcae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48404 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:23:36,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c16575bc060ce15bcb7 2025-12-17 21:23:36,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c16575bc060ce15bcb7 HTTP/1.1" 502 150 2025-12-17 21:23:36,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c16575bc060ce15bcb7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:36,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c14575bc060ce15bcb1 2025-12-17 21:23:36,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c14575bc060ce15bcb1 HTTP/1.1" 502 150 2025-12-17 21:23:36,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c14575bc060ce15bcb1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:36,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0b575bc060ce15bc8e 2025-12-17 21:23:36,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0b575bc060ce15bc8e HTTP/1.1" 502 150 2025-12-17 21:23:36,989 - lava_callback - ERROR - Error processing callback for node 69407c0b575bc060ce15bc8e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c0b575bc060ce15bc8e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c0b575bc060ce15bc8e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c0b575bc060ce15bc8e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:36,991 - lava_callback - INFO - Starting async job submit for node 69407c19575bc060ce15bcbd 2025-12-17 21:23:36,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:37,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c19575bc060ce15bcbd HTTP/1.1" 502 150 2025-12-17 21:23:37,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c19575bc060ce15bcbd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:37,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c19575bc060ce15bcbd 2025-12-17 21:23:37,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c19575bc060ce15bcbd HTTP/1.1" 502 150 2025-12-17 21:23:37,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c19575bc060ce15bcbd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:37,945 - lava_callback - INFO - Received callback for node 69407c4f575bc060ce15bd9e INFO: 172.21.0.1:48420 - "POST /node/69407c4f575bc060ce15bd9e HTTP/1.0" 202 Accepted 2025-12-17 21:23:38,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c13575bc060ce15bcae 2025-12-17 21:23:38,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c13575bc060ce15bcae HTTP/1.1" 502 150 2025-12-17 21:23:38,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c13575bc060ce15bcae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:38,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c18575bc060ce15bcba 2025-12-17 21:23:38,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c18575bc060ce15bcba HTTP/1.1" 502 150 2025-12-17 21:23:38,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c18575bc060ce15bcba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:38,734 - lava_callback - INFO - Received callback for node 69407c51575bc060ce15bda4 INFO: 172.21.0.1:48434 - "POST /node/69407c51575bc060ce15bda4 HTTP/1.0" 202 Accepted 2025-12-17 21:23:38,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0c575bc060ce15bc91 2025-12-17 21:23:38,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0c575bc060ce15bc91 HTTP/1.1" 502 150 2025-12-17 21:23:38,926 - lava_callback - ERROR - Error processing callback for node 69407c0c575bc060ce15bc91: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c0c575bc060ce15bc91 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c0c575bc060ce15bc91 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c0c575bc060ce15bc91 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:38,928 - lava_callback - INFO - Starting async job submit for node 69407c1a575bc060ce15bcc0 2025-12-17 21:23:38,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:38,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1a575bc060ce15bcc0 HTTP/1.1" 502 150 2025-12-17 21:23:38,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1a575bc060ce15bcc0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:38,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1a575bc060ce15bcc0 2025-12-17 21:23:38,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1a575bc060ce15bcc0 HTTP/1.1" 502 150 2025-12-17 21:23:38,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1a575bc060ce15bcc0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:39,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c19575bc060ce15bcbd 2025-12-17 21:23:39,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c19575bc060ce15bcbd HTTP/1.1" 502 150 2025-12-17 21:23:39,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c19575bc060ce15bcbd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:39,481 - lava_callback - INFO - Received callback for node 69407c54575bc060ce15bdaa INFO: 172.21.0.1:48436 - "POST /node/69407c54575bc060ce15bdaa HTTP/1.0" 202 Accepted 2025-12-17 21:23:40,245 - lava_callback - INFO - Received callback for node 69407c55575bc060ce15bdad INFO: 172.21.0.1:41328 - "POST /node/69407c55575bc060ce15bdad HTTP/1.0" 202 Accepted 2025-12-17 21:23:40,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1a575bc060ce15bcc0 2025-12-17 21:23:40,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1a575bc060ce15bcc0 HTTP/1.1" 502 150 2025-12-17 21:23:40,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1a575bc060ce15bcc0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:41,015 - lava_callback - INFO - Received callback for node 69407c57575bc060ce15bdb0 INFO: 172.21.0.1:41344 - "POST /node/69407c57575bc060ce15bdb0 HTTP/1.0" 202 Accepted 2025-12-17 21:23:41,787 - lava_callback - INFO - Received callback for node 69407c58575bc060ce15bdb3 INFO: 172.21.0.1:41346 - "POST /node/69407c58575bc060ce15bdb3 HTTP/1.0" 202 Accepted 2025-12-17 21:23:42,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c13575bc060ce15bcae 2025-12-17 21:23:42,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c13575bc060ce15bcae HTTP/1.1" 502 150 2025-12-17 21:23:42,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c13575bc060ce15bcae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:42,532 - lava_callback - INFO - Received callback for node 69407c59575bc060ce15bdb6 INFO: 172.21.0.1:41360 - "POST /node/69407c59575bc060ce15bdb6 HTTP/1.0" 202 Accepted 2025-12-17 21:23:42,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407be2575bc060ce15bc0f HTTP/1.1" 502 150 2025-12-17 21:23:42,640 - lava_callback - ERROR - Error processing callback for node 69407be2575bc060ce15bc0f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407be2575bc060ce15bc0f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407be2575bc060ce15bc0f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407be2575bc060ce15bc0f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:42,642 - lava_callback - INFO - Starting async job submit for node 69407c1b575bc060ce15bcc3 2025-12-17 21:23:42,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:42,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1b575bc060ce15bcc3 HTTP/1.1" 502 150 2025-12-17 21:23:42,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1b575bc060ce15bcc3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:42,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1b575bc060ce15bcc3 2025-12-17 21:23:42,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1b575bc060ce15bcc3 HTTP/1.1" 502 150 2025-12-17 21:23:42,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1b575bc060ce15bcc3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:43,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c19575bc060ce15bcbd 2025-12-17 21:23:43,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c19575bc060ce15bcbd HTTP/1.1" 502 150 2025-12-17 21:23:43,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c19575bc060ce15bcbd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:43,324 - lava_callback - INFO - Received callback for node 69407c5a575bc060ce15bdb9 INFO: 172.21.0.1:41362 - "POST /node/69407c5a575bc060ce15bdb9 HTTP/1.0" 202 Accepted 2025-12-17 21:23:43,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407bfb575bc060ce15bc3a 2025-12-17 21:23:43,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407bfb575bc060ce15bc3a HTTP/1.1" 502 150 2025-12-17 21:23:43,509 - lava_callback - ERROR - Error processing callback for node 69407bfb575bc060ce15bc3a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bfb575bc060ce15bc3a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bfb575bc060ce15bc3a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407bfb575bc060ce15bc3a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:43,511 - lava_callback - INFO - Starting async job submit for node 69407c1d575bc060ce15bcc6 2025-12-17 21:23:43,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:43,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1d575bc060ce15bcc6 HTTP/1.1" 502 150 2025-12-17 21:23:43,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1d575bc060ce15bcc6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:43,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1d575bc060ce15bcc6 2025-12-17 21:23:43,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1d575bc060ce15bcc6 HTTP/1.1" 502 150 2025-12-17 21:23:43,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1d575bc060ce15bcc6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:43,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b14575bc060ce15b93c 2025-12-17 21:23:43,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b14575bc060ce15b93c HTTP/1.1" 502 150 2025-12-17 21:23:43,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b14575bc060ce15b93c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:43,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c15575bc060ce15bcb4 2025-12-17 21:23:43,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c15575bc060ce15bcb4 HTTP/1.1" 502 150 2025-12-17 21:23:43,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c15575bc060ce15bcb4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:44,059 - lava_callback - INFO - Received callback for node 69407c5b575bc060ce15bdbc INFO: 172.21.0.1:41376 - "POST /node/69407c5b575bc060ce15bdbc HTTP/1.0" 202 Accepted 2025-12-17 21:23:44,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1b575bc060ce15bcc3 2025-12-17 21:23:44,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1b575bc060ce15bcc3 HTTP/1.1" 502 150 2025-12-17 21:23:44,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1b575bc060ce15bcc3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:44,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c16575bc060ce15bcb7 2025-12-17 21:23:44,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c16575bc060ce15bcb7 HTTP/1.1" 502 150 2025-12-17 21:23:44,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c16575bc060ce15bcb7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:44,829 - lava_callback - INFO - Received callback for node 69407c5d575bc060ce15bdc0 INFO: 172.21.0.1:41382 - "POST /node/69407c5d575bc060ce15bdc0 HTTP/1.0" 202 Accepted 2025-12-17 21:23:44,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1a575bc060ce15bcc0 2025-12-17 21:23:44,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1a575bc060ce15bcc0 HTTP/1.1" 502 150 2025-12-17 21:23:44,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1a575bc060ce15bcc0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:45,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1d575bc060ce15bcc6 2025-12-17 21:23:45,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1d575bc060ce15bcc6 HTTP/1.1" 502 150 2025-12-17 21:23:45,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1d575bc060ce15bcc6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:45,554 - lava_callback - INFO - Received callback for node 69407c5e575bc060ce15bdc7 INFO: 172.21.0.1:41386 - "POST /node/69407c5e575bc060ce15bdc7 HTTP/1.0" 202 Accepted 2025-12-17 21:23:46,357 - lava_callback - INFO - Received callback for node 69407c5f575bc060ce15bdcb INFO: 172.21.0.1:41390 - "POST /node/69407c5f575bc060ce15bdcb HTTP/1.0" 202 Accepted 2025-12-17 21:23:46,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c18575bc060ce15bcba 2025-12-17 21:23:46,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c18575bc060ce15bcba HTTP/1.1" 502 150 2025-12-17 21:23:46,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c18575bc060ce15bcba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41392 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:23:46,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0e575bc060ce15bc97 2025-12-17 21:23:46,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0e575bc060ce15bc97 HTTP/1.1" 502 150 2025-12-17 21:23:46,923 - lava_callback - ERROR - Error processing callback for node 69407c0e575bc060ce15bc97: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c0e575bc060ce15bc97 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c0e575bc060ce15bc97 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c0e575bc060ce15bc97 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:46,926 - lava_callback - INFO - Starting async job submit for node 69407c1e575bc060ce15bcc9 2025-12-17 21:23:46,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:46,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1e575bc060ce15bcc9 HTTP/1.1" 502 150 2025-12-17 21:23:46,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1e575bc060ce15bcc9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:46,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1e575bc060ce15bcc9 2025-12-17 21:23:46,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1e575bc060ce15bcc9 HTTP/1.1" 502 150 2025-12-17 21:23:46,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1e575bc060ce15bcc9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:47,136 - lava_callback - INFO - Received callback for node 69407c61575bc060ce15bdd0 INFO: 172.21.0.1:41396 - "POST /node/69407c61575bc060ce15bdd0 HTTP/1.0" 202 Accepted 2025-12-17 21:23:47,931 - lava_callback - INFO - Received callback for node 69407c62575bc060ce15bdd5 INFO: 172.21.0.1:41402 - "POST /node/69407c62575bc060ce15bdd5 HTTP/1.0" 202 Accepted 2025-12-17 21:23:48,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0f575bc060ce15bc9b 2025-12-17 21:23:48,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0f575bc060ce15bc9b HTTP/1.1" 502 150 2025-12-17 21:23:48,652 - lava_callback - ERROR - Error processing callback for node 69407c0f575bc060ce15bc9b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c0f575bc060ce15bc9b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c0f575bc060ce15bc9b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c0f575bc060ce15bc9b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:48,654 - lava_callback - INFO - Starting async job submit for node 69407c1f575bc060ce15bccc 2025-12-17 21:23:48,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:48,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1b575bc060ce15bcc3 2025-12-17 21:23:48,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1f575bc060ce15bccc HTTP/1.1" 502 150 2025-12-17 21:23:48,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1f575bc060ce15bccc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:48,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1f575bc060ce15bccc 2025-12-17 21:23:48,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1b575bc060ce15bcc3 HTTP/1.1" 502 150 2025-12-17 21:23:48,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1b575bc060ce15bcc3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:48,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1f575bc060ce15bccc HTTP/1.1" 502 150 2025-12-17 21:23:48,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1f575bc060ce15bccc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:48,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c10575bc060ce15bca2 2025-12-17 21:23:48,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c10575bc060ce15bca2 HTTP/1.1" 502 150 2025-12-17 21:23:48,719 - lava_callback - ERROR - Error processing callback for node 69407c10575bc060ce15bca2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c10575bc060ce15bca2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c10575bc060ce15bca2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c10575bc060ce15bca2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:48,721 - lava_callback - INFO - Starting async job submit for node 69407c20575bc060ce15bccf 2025-12-17 21:23:48,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:48,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c20575bc060ce15bccf HTTP/1.1" 502 150 2025-12-17 21:23:48,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c20575bc060ce15bccf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:48,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c20575bc060ce15bccf 2025-12-17 21:23:48,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c20575bc060ce15bccf HTTP/1.1" 502 150 2025-12-17 21:23:48,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c20575bc060ce15bccf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:48,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1e575bc060ce15bcc9 2025-12-17 21:23:48,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1e575bc060ce15bcc9 HTTP/1.1" 502 150 2025-12-17 21:23:48,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1e575bc060ce15bcc9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:49,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c07575bc060ce15f42b 2025-12-17 21:23:49,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c07575bc060ce15f42b HTTP/1.1" 502 150 2025-12-17 21:23:49,499 - lava_callback - ERROR - Error processing callback for node 69408c07575bc060ce15f42b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c07575bc060ce15f42b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c07575bc060ce15f42b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c07575bc060ce15f42b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:49,500 - lava_callback - INFO - Starting async job submit for node 69407c22575bc060ce15bcd5 2025-12-17 21:23:49,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:49,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1d575bc060ce15bcc6 2025-12-17 21:23:49,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1d575bc060ce15bcc6 HTTP/1.1" 502 150 2025-12-17 21:23:49,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1d575bc060ce15bcc6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:49,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c22575bc060ce15bcd5 HTTP/1.1" 502 150 2025-12-17 21:23:49,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c22575bc060ce15bcd5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:49,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c22575bc060ce15bcd5 2025-12-17 21:23:49,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c22575bc060ce15bcd5 HTTP/1.1" 502 150 2025-12-17 21:23:49,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c22575bc060ce15bcd5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:49,834 - lava_callback - INFO - Received callback for node 69407eb0575bc060ce15c33c INFO: 172.21.0.1:40184 - "POST /node/69407eb0575bc060ce15c33c HTTP/1.0" 202 Accepted 2025-12-17 21:23:49,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c11575bc060ce15bca7 2025-12-17 21:23:49,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c11575bc060ce15bca7 HTTP/1.1" 502 150 2025-12-17 21:23:49,958 - lava_callback - ERROR - Error processing callback for node 69407c11575bc060ce15bca7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c11575bc060ce15bca7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c11575bc060ce15bca7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c11575bc060ce15bca7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:49,960 - lava_callback - INFO - Starting async job submit for node 69407c23575bc060ce15bcdb 2025-12-17 21:23:49,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:49,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c23575bc060ce15bcdb HTTP/1.1" 502 150 2025-12-17 21:23:49,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c23575bc060ce15bcdb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:49,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c23575bc060ce15bcdb 2025-12-17 21:23:49,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c23575bc060ce15bcdb HTTP/1.1" 502 150 2025-12-17 21:23:49,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c23575bc060ce15bcdb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:50,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c13575bc060ce15bcae 2025-12-17 21:23:50,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c13575bc060ce15bcae HTTP/1.1" 502 150 2025-12-17 21:23:50,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c13575bc060ce15bcae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:50,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1f575bc060ce15bccc 2025-12-17 21:23:50,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1f575bc060ce15bccc HTTP/1.1" 502 150 2025-12-17 21:23:50,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1f575bc060ce15bccc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:50,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c20575bc060ce15bccf 2025-12-17 21:23:50,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c20575bc060ce15bccf HTTP/1.1" 502 150 2025-12-17 21:23:50,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c20575bc060ce15bccf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:51,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c19575bc060ce15bcbd 2025-12-17 21:23:51,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c19575bc060ce15bcbd HTTP/1.1" 502 150 2025-12-17 21:23:51,037 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c19575bc060ce15bcbd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:51,404 - lava_callback - INFO - Received callback for node 69407e36575bc060ce15c288 INFO: 172.21.0.1:40196 - "POST /node/69407e36575bc060ce15c288 HTTP/1.0" 202 Accepted 2025-12-17 21:23:51,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c22575bc060ce15bcd5 2025-12-17 21:23:51,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c22575bc060ce15bcd5 HTTP/1.1" 502 150 2025-12-17 21:23:51,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c22575bc060ce15bcd5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:51,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b15575bc060ce15b93f HTTP/1.1" 502 150 2025-12-17 21:23:51,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b15575bc060ce15b93f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:51,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c23575bc060ce15bcdb 2025-12-17 21:23:51,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c23575bc060ce15bcdb HTTP/1.1" 502 150 2025-12-17 21:23:51,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c23575bc060ce15bcdb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:52,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c14575bc060ce15bcb1 2025-12-17 21:23:52,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c14575bc060ce15bcb1 HTTP/1.1" 502 150 2025-12-17 21:23:52,902 - lava_callback - ERROR - Error processing callback for node 69407c14575bc060ce15bcb1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c14575bc060ce15bcb1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c14575bc060ce15bcb1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c14575bc060ce15bcb1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:52,904 - lava_callback - INFO - Starting async job submit for node 69407c25575bc060ce15bce0 2025-12-17 21:23:52,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:52,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c25575bc060ce15bce0 HTTP/1.1" 502 150 2025-12-17 21:23:52,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c25575bc060ce15bce0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:52,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c25575bc060ce15bce0 2025-12-17 21:23:52,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c25575bc060ce15bce0 HTTP/1.1" 502 150 2025-12-17 21:23:52,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c25575bc060ce15bce0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:52,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1a575bc060ce15bcc0 2025-12-17 21:23:52,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1a575bc060ce15bcc0 HTTP/1.1" 502 150 2025-12-17 21:23:52,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1a575bc060ce15bcc0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:52,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1e575bc060ce15bcc9 2025-12-17 21:23:52,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1e575bc060ce15bcc9 HTTP/1.1" 502 150 2025-12-17 21:23:52,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1e575bc060ce15bcc9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:54,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1f575bc060ce15bccc 2025-12-17 21:23:54,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1f575bc060ce15bccc HTTP/1.1" 502 150 2025-12-17 21:23:54,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1f575bc060ce15bccc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:54,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c20575bc060ce15bccf 2025-12-17 21:23:54,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c20575bc060ce15bccf HTTP/1.1" 502 150 2025-12-17 21:23:54,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c20575bc060ce15bccf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:54,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c25575bc060ce15bce0 2025-12-17 21:23:54,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c25575bc060ce15bce0 HTTP/1.1" 502 150 2025-12-17 21:23:54,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c25575bc060ce15bce0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:55,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c22575bc060ce15bcd5 2025-12-17 21:23:55,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c22575bc060ce15bcd5 HTTP/1.1" 502 150 2025-12-17 21:23:55,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c22575bc060ce15bcd5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:55,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b15575bc060ce15b93f 2025-12-17 21:23:55,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b15575bc060ce15b93f HTTP/1.1" 502 150 2025-12-17 21:23:55,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b15575bc060ce15b93f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:55,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c23575bc060ce15bcdb 2025-12-17 21:23:55,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c23575bc060ce15bcdb HTTP/1.1" 502 150 2025-12-17 21:23:55,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c23575bc060ce15bcdb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40200 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:23:56,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1b575bc060ce15bcc3 2025-12-17 21:23:56,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1b575bc060ce15bcc3 HTTP/1.1" 502 150 2025-12-17 21:23:56,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1b575bc060ce15bcc3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:57,237 - lava_callback - INFO - Received callback for node 69407c64575bc060ce15bde0 INFO: 172.21.0.1:40204 - "POST /node/69407c64575bc060ce15bde0 HTTP/1.0" 202 Accepted 2025-12-17 21:23:57,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1d575bc060ce15bcc6 2025-12-17 21:23:57,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1d575bc060ce15bcc6 HTTP/1.1" 502 150 2025-12-17 21:23:57,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1d575bc060ce15bcc6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:57,979 - lava_callback - INFO - Received callback for node 69407c65575bc060ce15bde7 INFO: 172.21.0.1:40210 - "POST /node/69407c65575bc060ce15bde7 HTTP/1.0" 202 Accepted 2025-12-17 21:23:58,763 - lava_callback - INFO - Received callback for node 69407c66575bc060ce15bdea INFO: 172.21.0.1:40216 - "POST /node/69407c66575bc060ce15bdea HTTP/1.0" 202 Accepted 2025-12-17 21:23:58,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c25575bc060ce15bce0 2025-12-17 21:23:58,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c25575bc060ce15bce0 HTTP/1.1" 502 150 2025-12-17 21:23:58,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c25575bc060ce15bce0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:59,508 - lava_callback - INFO - Received callback for node 69407c68575bc060ce15bded INFO: 172.21.0.1:40226 - "POST /node/69407c68575bc060ce15bded HTTP/1.0" 202 Accepted 2025-12-17 21:23:59,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b14575bc060ce15b93c 2025-12-17 21:23:59,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b14575bc060ce15b93c HTTP/1.1" 502 150 2025-12-17 21:23:59,691 - lava_callback - ERROR - Error processing callback for node 69407b14575bc060ce15b93c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b14575bc060ce15b93c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b14575bc060ce15b93c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b14575bc060ce15b93c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:59,693 - lava_callback - INFO - Starting async job submit for node 69407c26575bc060ce15bce3 2025-12-17 21:23:59,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:59,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c15575bc060ce15bcb4 2025-12-17 21:23:59,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c15575bc060ce15bcb4 HTTP/1.1" 502 150 2025-12-17 21:23:59,695 - lava_callback - ERROR - Error processing callback for node 69407c15575bc060ce15bcb4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c15575bc060ce15bcb4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c15575bc060ce15bcb4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c15575bc060ce15bcb4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:23:59,696 - lava_callback - INFO - Starting async job submit for node 69407c27575bc060ce15bce6 2025-12-17 21:23:59,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:23:59,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c27575bc060ce15bce6 HTTP/1.1" 502 150 2025-12-17 21:23:59,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c27575bc060ce15bce6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:59,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c27575bc060ce15bce6 2025-12-17 21:23:59,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c26575bc060ce15bce3 HTTP/1.1" 502 150 2025-12-17 21:23:59,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c26575bc060ce15bce3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:59,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c26575bc060ce15bce3 2025-12-17 21:23:59,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c27575bc060ce15bce6 HTTP/1.1" 502 150 2025-12-17 21:23:59,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c27575bc060ce15bce6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:23:59,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c26575bc060ce15bce3 HTTP/1.1" 502 150 2025-12-17 21:23:59,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c26575bc060ce15bce3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:00,260 - lava_callback - INFO - Received callback for node 69407c69575bc060ce15bdf0 INFO: 172.21.0.1:43836 - "POST /node/69407c69575bc060ce15bdf0 HTTP/1.0" 202 Accepted 2025-12-17 21:24:00,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c16575bc060ce15bcb7 2025-12-17 21:24:00,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c16575bc060ce15bcb7 HTTP/1.1" 502 150 2025-12-17 21:24:00,707 - lava_callback - ERROR - Error processing callback for node 69407c16575bc060ce15bcb7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c16575bc060ce15bcb7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c16575bc060ce15bcb7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c16575bc060ce15bcb7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:00,709 - lava_callback - INFO - Starting async job submit for node 69407c28575bc060ce15bce9 2025-12-17 21:24:00,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:00,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c28575bc060ce15bce9 HTTP/1.1" 502 150 2025-12-17 21:24:00,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c28575bc060ce15bce9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:00,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c28575bc060ce15bce9 2025-12-17 21:24:00,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c28575bc060ce15bce9 HTTP/1.1" 502 150 2025-12-17 21:24:00,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c28575bc060ce15bce9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:00,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1e575bc060ce15bcc9 2025-12-17 21:24:00,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1e575bc060ce15bcc9 HTTP/1.1" 502 150 2025-12-17 21:24:00,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1e575bc060ce15bcc9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:01,018 - lava_callback - INFO - Received callback for node 69407c6a575bc060ce15bdf3 INFO: 172.21.0.1:43838 - "POST /node/69407c6a575bc060ce15bdf3 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:43844 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:24:01,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c27575bc060ce15bce6 2025-12-17 21:24:01,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c26575bc060ce15bce3 2025-12-17 21:24:01,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c27575bc060ce15bce6 HTTP/1.1" 502 150 2025-12-17 21:24:01,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c27575bc060ce15bce6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:01,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c26575bc060ce15bce3 HTTP/1.1" 502 150 2025-12-17 21:24:01,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c26575bc060ce15bce3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:01,804 - lava_callback - INFO - Received callback for node 69407c6b575bc060ce15bdf6 INFO: 172.21.0.1:43852 - "POST /node/69407c6b575bc060ce15bdf6 HTTP/1.0" 202 Accepted 2025-12-17 21:24:02,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c18575bc060ce15bcba 2025-12-17 21:24:02,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c18575bc060ce15bcba HTTP/1.1" 502 150 2025-12-17 21:24:02,518 - lava_callback - ERROR - Error processing callback for node 69407c18575bc060ce15bcba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c18575bc060ce15bcba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c18575bc060ce15bcba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c18575bc060ce15bcba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:02,521 - lava_callback - INFO - Starting async job submit for node 69407c29575bc060ce15bcec 2025-12-17 21:24:02,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:02,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c29575bc060ce15bcec HTTP/1.1" 502 150 2025-12-17 21:24:02,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c29575bc060ce15bcec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:02,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c29575bc060ce15bcec 2025-12-17 21:24:02,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c29575bc060ce15bcec HTTP/1.1" 502 150 2025-12-17 21:24:02,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c29575bc060ce15bcec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:02,565 - lava_callback - INFO - Received callback for node 69407c6d575bc060ce15bdf9 INFO: 172.21.0.1:43868 - "POST /node/69407c6d575bc060ce15bdf9 HTTP/1.0" 202 Accepted 2025-12-17 21:24:02,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1f575bc060ce15bccc 2025-12-17 21:24:02,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1f575bc060ce15bccc HTTP/1.1" 502 150 2025-12-17 21:24:02,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c1f575bc060ce15bccc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:02,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c28575bc060ce15bce9 2025-12-17 21:24:02,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c28575bc060ce15bce9 HTTP/1.1" 502 150 2025-12-17 21:24:02,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c28575bc060ce15bce9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:02,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c20575bc060ce15bccf 2025-12-17 21:24:02,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c20575bc060ce15bccf HTTP/1.1" 502 150 2025-12-17 21:24:02,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c20575bc060ce15bccf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:03,492 - lava_callback - INFO - Received callback for node 69407c6e575bc060ce15bdfc INFO: 172.21.0.1:43874 - "POST /node/69407c6e575bc060ce15bdfc HTTP/1.0" 202 Accepted 2025-12-17 21:24:03,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c22575bc060ce15bcd5 2025-12-17 21:24:03,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c22575bc060ce15bcd5 HTTP/1.1" 502 150 2025-12-17 21:24:03,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c22575bc060ce15bcd5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:03,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b15575bc060ce15b93f 2025-12-17 21:24:03,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b15575bc060ce15b93f HTTP/1.1" 502 150 2025-12-17 21:24:03,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b15575bc060ce15b93f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:03,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c23575bc060ce15bcdb 2025-12-17 21:24:03,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c23575bc060ce15bcdb HTTP/1.1" 502 150 2025-12-17 21:24:03,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c23575bc060ce15bcdb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:04,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c29575bc060ce15bcec 2025-12-17 21:24:04,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c29575bc060ce15bcec HTTP/1.1" 502 150 2025-12-17 21:24:04,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c29575bc060ce15bcec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:04,640 - lava_callback - INFO - Received callback for node 69407c70575bc060ce15bdff INFO: 172.21.0.1:43878 - "POST /node/69407c70575bc060ce15bdff HTTP/1.0" 202 Accepted 2025-12-17 21:24:05,383 - lava_callback - INFO - Received callback for node 69407c71575bc060ce15be05 INFO: 172.21.0.1:43892 - "POST /node/69407c71575bc060ce15be05 HTTP/1.0" 202 Accepted 2025-12-17 21:24:05,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c27575bc060ce15bce6 2025-12-17 21:24:05,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c26575bc060ce15bce3 2025-12-17 21:24:05,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c27575bc060ce15bce6 HTTP/1.1" 502 150 2025-12-17 21:24:05,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c27575bc060ce15bce6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:05,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c26575bc060ce15bce3 HTTP/1.1" 502 150 2025-12-17 21:24:05,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c26575bc060ce15bce3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:06,178 - lava_callback - INFO - Received callback for node 69407c73575bc060ce15be0c INFO: 172.21.0.1:43902 - "POST /node/69407c73575bc060ce15be0c HTTP/1.0" 202 Accepted 2025-12-17 21:24:06,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c13575bc060ce15bcae 2025-12-17 21:24:06,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c13575bc060ce15bcae HTTP/1.1" 502 150 2025-12-17 21:24:06,507 - lava_callback - ERROR - Error processing callback for node 69407c13575bc060ce15bcae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c13575bc060ce15bcae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c13575bc060ce15bcae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c13575bc060ce15bcae (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:06,510 - lava_callback - INFO - Starting async job submit for node 69407c2a575bc060ce15bcf2 2025-12-17 21:24:06,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:43912 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:24:06,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2a575bc060ce15bcf2 HTTP/1.1" 502 150 2025-12-17 21:24:06,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2a575bc060ce15bcf2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:06,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2a575bc060ce15bcf2 2025-12-17 21:24:06,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2a575bc060ce15bcf2 HTTP/1.1" 502 150 2025-12-17 21:24:06,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2a575bc060ce15bcf2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:06,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c28575bc060ce15bce9 2025-12-17 21:24:06,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c28575bc060ce15bce9 HTTP/1.1" 502 150 2025-12-17 21:24:06,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c28575bc060ce15bce9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:06,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c25575bc060ce15bce0 2025-12-17 21:24:06,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c25575bc060ce15bce0 HTTP/1.1" 502 150 2025-12-17 21:24:06,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c25575bc060ce15bce0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:06,938 - lava_callback - INFO - Received callback for node 69407c74575bc060ce15be13 INFO: 172.21.0.1:43924 - "POST /node/69407c74575bc060ce15be13 HTTP/1.0" 202 Accepted 2025-12-17 21:24:07,037 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c19575bc060ce15bcbd 2025-12-17 21:24:07,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c19575bc060ce15bcbd HTTP/1.1" 502 150 2025-12-17 21:24:07,039 - lava_callback - ERROR - Error processing callback for node 69407c19575bc060ce15bcbd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c19575bc060ce15bcbd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c19575bc060ce15bcbd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c19575bc060ce15bcbd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:07,042 - lava_callback - INFO - Starting async job submit for node 69407c2c575bc060ce15bcf5 2025-12-17 21:24:07,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:07,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2c575bc060ce15bcf5 HTTP/1.1" 502 150 2025-12-17 21:24:07,051 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2c575bc060ce15bcf5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:07,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2c575bc060ce15bcf5 2025-12-17 21:24:07,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2c575bc060ce15bcf5 HTTP/1.1" 502 150 2025-12-17 21:24:07,052 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2c575bc060ce15bcf5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:07,695 - lava_callback - INFO - Received callback for node 69407c75575bc060ce15be18 INFO: 172.21.0.1:43928 - "POST /node/69407c75575bc060ce15be18 HTTP/1.0" 202 Accepted 2025-12-17 21:24:08,422 - lava_callback - INFO - Received callback for node 69407c77575bc060ce15be1f INFO: 172.21.0.1:43938 - "POST /node/69407c77575bc060ce15be1f HTTP/1.0" 202 Accepted 2025-12-17 21:24:08,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2a575bc060ce15bcf2 2025-12-17 21:24:08,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2a575bc060ce15bcf2 HTTP/1.1" 502 150 2025-12-17 21:24:08,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2a575bc060ce15bcf2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:08,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c29575bc060ce15bcec 2025-12-17 21:24:08,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c29575bc060ce15bcec HTTP/1.1" 502 150 2025-12-17 21:24:08,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c29575bc060ce15bcec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:08,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1a575bc060ce15bcc0 2025-12-17 21:24:08,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1a575bc060ce15bcc0 HTTP/1.1" 502 150 2025-12-17 21:24:08,942 - lava_callback - ERROR - Error processing callback for node 69407c1a575bc060ce15bcc0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c1a575bc060ce15bcc0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c1a575bc060ce15bcc0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c1a575bc060ce15bcc0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:08,944 - lava_callback - INFO - Starting async job submit for node 69408c16575bc060ce15f454 2025-12-17 21:24:09,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:09,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c16575bc060ce15f454 HTTP/1.1" 502 150 2025-12-17 21:24:09,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c16575bc060ce15f454'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:09,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c16575bc060ce15f454 2025-12-17 21:24:09,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c16575bc060ce15f454 HTTP/1.1" 502 150 2025-12-17 21:24:09,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c16575bc060ce15f454'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:09,052 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2c575bc060ce15bcf5 2025-12-17 21:24:09,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2c575bc060ce15bcf5 HTTP/1.1" 502 150 2025-12-17 21:24:09,053 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2c575bc060ce15bcf5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:11,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c16575bc060ce15f454 2025-12-17 21:24:11,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c16575bc060ce15f454 HTTP/1.1" 502 150 2025-12-17 21:24:11,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c16575bc060ce15f454'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:11,582 - lava_callback - INFO - Received callback for node 69407d48575bc060ce15c039 INFO: 172.21.0.1:36994 - "POST /node/69407d48575bc060ce15c039 HTTP/1.0" 202 Accepted 2025-12-17 21:24:11,642 - lava_callback - INFO - Received callback for node 69408c97575bc060ce15f627 INFO: 172.21.0.1:37006 - "POST /node/69408c97575bc060ce15f627 HTTP/1.0" 202 Accepted 2025-12-17 21:24:12,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2a575bc060ce15bcf2 2025-12-17 21:24:12,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2a575bc060ce15bcf2 HTTP/1.1" 502 150 2025-12-17 21:24:12,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2a575bc060ce15bcf2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:12,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1b575bc060ce15bcc3 2025-12-17 21:24:12,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1b575bc060ce15bcc3 HTTP/1.1" 502 150 2025-12-17 21:24:12,664 - lava_callback - ERROR - Error processing callback for node 69407c1b575bc060ce15bcc3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c1b575bc060ce15bcc3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c1b575bc060ce15bcc3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c1b575bc060ce15bcc3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:12,666 - lava_callback - INFO - Starting async job submit for node 69409356575bc060ce160d16 2025-12-17 21:24:12,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:12,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409356575bc060ce160d16 HTTP/1.1" 502 150 2025-12-17 21:24:12,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409356575bc060ce160d16'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:12,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409356575bc060ce160d16 2025-12-17 21:24:12,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409356575bc060ce160d16 HTTP/1.1" 502 150 2025-12-17 21:24:12,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409356575bc060ce160d16'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:13,054 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2c575bc060ce15bcf5 2025-12-17 21:24:13,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1d575bc060ce15bcc6 2025-12-17 21:24:13,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1d575bc060ce15bcc6 HTTP/1.1" 502 150 2025-12-17 21:24:13,524 - lava_callback - ERROR - Error processing callback for node 69407c1d575bc060ce15bcc6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c1d575bc060ce15bcc6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c1d575bc060ce15bcc6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c1d575bc060ce15bcc6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:13,526 - lava_callback - INFO - Starting async job submit for node 69407c2d575bc060ce15bcfc 2025-12-17 21:24:13,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:13,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2d575bc060ce15bcfc HTTP/1.1" 502 150 2025-12-17 21:24:13,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2d575bc060ce15bcfc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:13,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2d575bc060ce15bcfc 2025-12-17 21:24:13,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2d575bc060ce15bcfc HTTP/1.1" 502 150 2025-12-17 21:24:13,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2d575bc060ce15bcfc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:13,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c27575bc060ce15bce6 2025-12-17 21:24:13,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c26575bc060ce15bce3 2025-12-17 21:24:13,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c27575bc060ce15bce6 HTTP/1.1" 502 150 2025-12-17 21:24:13,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c27575bc060ce15bce6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:13,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c26575bc060ce15bce3 HTTP/1.1" 502 150 2025-12-17 21:24:13,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c26575bc060ce15bce3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:14,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409356575bc060ce160d16 2025-12-17 21:24:14,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409356575bc060ce160d16 HTTP/1.1" 502 150 2025-12-17 21:24:14,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409356575bc060ce160d16'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:14,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c28575bc060ce15bce9 2025-12-17 21:24:14,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c28575bc060ce15bce9 HTTP/1.1" 502 150 2025-12-17 21:24:14,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c28575bc060ce15bce9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:15,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c16575bc060ce15f454 2025-12-17 21:24:15,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c16575bc060ce15f454 HTTP/1.1" 502 150 2025-12-17 21:24:15,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c16575bc060ce15f454'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37010 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:24:15,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2d575bc060ce15bcfc 2025-12-17 21:24:15,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2d575bc060ce15bcfc HTTP/1.1" 502 150 2025-12-17 21:24:15,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2d575bc060ce15bcfc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:15,833 - lava_callback - INFO - Received callback for node 69408d41575bc060ce15f7f4 INFO: 172.21.0.1:37018 - "POST /node/69408d41575bc060ce15f7f4 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:37028 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:24:16,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c29575bc060ce15bcec 2025-12-17 21:24:16,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c29575bc060ce15bcec HTTP/1.1" 502 150 2025-12-17 21:24:16,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c29575bc060ce15bcec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:16,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1e575bc060ce15bcc9 2025-12-17 21:24:16,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1e575bc060ce15bcc9 HTTP/1.1" 502 150 2025-12-17 21:24:16,951 - lava_callback - ERROR - Error processing callback for node 69407c1e575bc060ce15bcc9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c1e575bc060ce15bcc9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c1e575bc060ce15bcc9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c1e575bc060ce15bcc9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:16,954 - lava_callback - INFO - Starting async job submit for node 69407c2e575bc060ce15bcff 2025-12-17 21:24:16,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:16,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2e575bc060ce15bcff HTTP/1.1" 502 150 2025-12-17 21:24:16,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2e575bc060ce15bcff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:16,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2e575bc060ce15bcff 2025-12-17 21:24:16,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2e575bc060ce15bcff HTTP/1.1" 502 150 2025-12-17 21:24:16,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2e575bc060ce15bcff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:17,115 - lava_callback - INFO - Received callback for node 69407c7c575bc060ce15be23 INFO: 172.21.0.1:37040 - "POST /node/69407c7c575bc060ce15be23 HTTP/1.0" 202 Accepted 2025-12-17 21:24:17,858 - lava_callback - INFO - Received callback for node 69407c7d575bc060ce15be26 INFO: 172.21.0.1:37042 - "POST /node/69407c7d575bc060ce15be26 HTTP/1.0" 202 Accepted 2025-12-17 21:24:18,579 - lava_callback - INFO - Received callback for node 69407c7f575bc060ce15be29 INFO: 172.21.0.1:37056 - "POST /node/69407c7f575bc060ce15be29 HTTP/1.0" 202 Accepted 2025-12-17 21:24:18,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c1f575bc060ce15bccc 2025-12-17 21:24:18,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c1f575bc060ce15bccc HTTP/1.1" 502 150 2025-12-17 21:24:18,668 - lava_callback - ERROR - Error processing callback for node 69407c1f575bc060ce15bccc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c1f575bc060ce15bccc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c1f575bc060ce15bccc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c1f575bc060ce15bccc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:18,670 - lava_callback - INFO - Starting async job submit for node 69407c38575bc060ce15bd1a 2025-12-17 21:24:18,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:18,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409356575bc060ce160d16 2025-12-17 21:24:18,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409356575bc060ce160d16 HTTP/1.1" 502 150 2025-12-17 21:24:18,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409356575bc060ce160d16'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:18,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c38575bc060ce15bd1a HTTP/1.1" 502 150 2025-12-17 21:24:18,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c38575bc060ce15bd1a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:18,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c38575bc060ce15bd1a 2025-12-17 21:24:18,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c38575bc060ce15bd1a HTTP/1.1" 502 150 2025-12-17 21:24:18,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c38575bc060ce15bd1a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:18,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c20575bc060ce15bccf 2025-12-17 21:24:18,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c20575bc060ce15bccf HTTP/1.1" 502 150 2025-12-17 21:24:18,759 - lava_callback - ERROR - Error processing callback for node 69407c20575bc060ce15bccf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c20575bc060ce15bccf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c20575bc060ce15bccf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c20575bc060ce15bccf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:18,761 - lava_callback - INFO - Starting async job submit for node 69407c3b575bc060ce15bd27 2025-12-17 21:24:18,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:18,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3b575bc060ce15bd27 HTTP/1.1" 502 150 2025-12-17 21:24:18,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3b575bc060ce15bd27'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:18,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3b575bc060ce15bd27 2025-12-17 21:24:18,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3b575bc060ce15bd27 HTTP/1.1" 502 150 2025-12-17 21:24:18,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3b575bc060ce15bd27'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:18,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2e575bc060ce15bcff 2025-12-17 21:24:18,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2e575bc060ce15bcff HTTP/1.1" 502 150 2025-12-17 21:24:18,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2e575bc060ce15bcff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:19,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c22575bc060ce15bcd5 2025-12-17 21:24:19,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c22575bc060ce15bcd5 HTTP/1.1" 502 150 2025-12-17 21:24:19,534 - lava_callback - ERROR - Error processing callback for node 69407c22575bc060ce15bcd5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c22575bc060ce15bcd5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c22575bc060ce15bcd5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c22575bc060ce15bcd5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:19,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2d575bc060ce15bcfc 2025-12-17 21:24:19,536 - lava_callback - INFO - Starting async job submit for node 69408c32575bc060ce15f4a9 2025-12-17 21:24:19,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2d575bc060ce15bcfc HTTP/1.1" 502 150 2025-12-17 21:24:19,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2d575bc060ce15bcfc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:19,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:19,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b15575bc060ce15b93f 2025-12-17 21:24:19,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b15575bc060ce15b93f HTTP/1.1" 502 150 2025-12-17 21:24:19,604 - lava_callback - ERROR - Error processing callback for node 69407b15575bc060ce15b93f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b15575bc060ce15b93f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b15575bc060ce15b93f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b15575bc060ce15b93f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:19,607 - lava_callback - INFO - Starting async job submit for node 69407c37575bc060ce15bd13 2025-12-17 21:24:19,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:19,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c32575bc060ce15f4a9 HTTP/1.1" 502 150 2025-12-17 21:24:19,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c32575bc060ce15f4a9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:19,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c32575bc060ce15f4a9 2025-12-17 21:24:19,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c32575bc060ce15f4a9 HTTP/1.1" 502 150 2025-12-17 21:24:19,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c32575bc060ce15f4a9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:19,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c37575bc060ce15bd13 HTTP/1.1" 502 150 2025-12-17 21:24:19,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c37575bc060ce15bd13'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:19,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c37575bc060ce15bd13 2025-12-17 21:24:19,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c37575bc060ce15bd13 HTTP/1.1" 502 150 2025-12-17 21:24:19,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c37575bc060ce15bd13'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:19,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c23575bc060ce15bcdb 2025-12-17 21:24:19,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c23575bc060ce15bcdb HTTP/1.1" 502 150 2025-12-17 21:24:19,999 - lava_callback - ERROR - Error processing callback for node 69407c23575bc060ce15bcdb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c23575bc060ce15bcdb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c23575bc060ce15bcdb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c23575bc060ce15bcdb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:20,001 - lava_callback - INFO - Starting async job submit for node 69407c3a575bc060ce15bd20 2025-12-17 21:24:20,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:20,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3a575bc060ce15bd20 HTTP/1.1" 502 150 2025-12-17 21:24:20,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3a575bc060ce15bd20'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:20,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3a575bc060ce15bd20 2025-12-17 21:24:20,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3a575bc060ce15bd20 HTTP/1.1" 502 150 2025-12-17 21:24:20,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3a575bc060ce15bd20'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:20,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2a575bc060ce15bcf2 2025-12-17 21:24:20,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2a575bc060ce15bcf2 HTTP/1.1" 502 150 2025-12-17 21:24:20,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2a575bc060ce15bcf2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:20,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c38575bc060ce15bd1a 2025-12-17 21:24:20,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c38575bc060ce15bd1a HTTP/1.1" 502 150 2025-12-17 21:24:20,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c38575bc060ce15bd1a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:20,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3b575bc060ce15bd27 2025-12-17 21:24:20,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3b575bc060ce15bd27 HTTP/1.1" 502 150 2025-12-17 21:24:20,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3b575bc060ce15bd27'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:21,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c32575bc060ce15f4a9 2025-12-17 21:24:21,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c32575bc060ce15f4a9 HTTP/1.1" 502 150 2025-12-17 21:24:21,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c32575bc060ce15f4a9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:21,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c37575bc060ce15bd13 2025-12-17 21:24:21,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c37575bc060ce15bd13 HTTP/1.1" 502 150 2025-12-17 21:24:21,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c37575bc060ce15bd13'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:22,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3a575bc060ce15bd20 2025-12-17 21:24:22,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3a575bc060ce15bd20 HTTP/1.1" 502 150 2025-12-17 21:24:22,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3a575bc060ce15bd20'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:22,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c25575bc060ce15bce0 2025-12-17 21:24:22,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c25575bc060ce15bce0 HTTP/1.1" 502 150 2025-12-17 21:24:22,917 - lava_callback - ERROR - Error processing callback for node 69407c25575bc060ce15bce0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c25575bc060ce15bce0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c25575bc060ce15bce0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c25575bc060ce15bce0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:22,919 - lava_callback - INFO - Starting async job submit for node 69407c3d575bc060ce15bd2b 2025-12-17 21:24:22,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:22,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3d575bc060ce15bd2b HTTP/1.1" 502 150 2025-12-17 21:24:22,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3d575bc060ce15bd2b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:22,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3d575bc060ce15bd2b 2025-12-17 21:24:22,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3d575bc060ce15bd2b HTTP/1.1" 502 150 2025-12-17 21:24:22,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3d575bc060ce15bd2b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:22,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2e575bc060ce15bcff 2025-12-17 21:24:22,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2e575bc060ce15bcff HTTP/1.1" 502 150 2025-12-17 21:24:22,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2e575bc060ce15bcff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:23,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c16575bc060ce15f454 2025-12-17 21:24:23,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c16575bc060ce15f454 HTTP/1.1" 502 150 2025-12-17 21:24:23,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c16575bc060ce15f454'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:24,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c38575bc060ce15bd1a 2025-12-17 21:24:24,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c38575bc060ce15bd1a HTTP/1.1" 502 150 2025-12-17 21:24:24,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c38575bc060ce15bd1a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:24,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3b575bc060ce15bd27 2025-12-17 21:24:24,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3b575bc060ce15bd27 HTTP/1.1" 502 150 2025-12-17 21:24:24,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3b575bc060ce15bd27'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:24,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3d575bc060ce15bd2b 2025-12-17 21:24:24,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3d575bc060ce15bd2b HTTP/1.1" 502 150 2025-12-17 21:24:24,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3d575bc060ce15bd2b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:25,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c32575bc060ce15f4a9 2025-12-17 21:24:25,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c32575bc060ce15f4a9 HTTP/1.1" 502 150 2025-12-17 21:24:25,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c32575bc060ce15f4a9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:25,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c37575bc060ce15bd13 2025-12-17 21:24:25,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c37575bc060ce15bd13 HTTP/1.1" 502 150 2025-12-17 21:24:25,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c37575bc060ce15bd13'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:26,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3a575bc060ce15bd20 2025-12-17 21:24:26,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3a575bc060ce15bd20 HTTP/1.1" 502 150 2025-12-17 21:24:26,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3a575bc060ce15bd20'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42354 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:24:26,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409356575bc060ce160d16 2025-12-17 21:24:26,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409356575bc060ce160d16 HTTP/1.1" 502 150 2025-12-17 21:24:26,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409356575bc060ce160d16'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:27,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2d575bc060ce15bcfc 2025-12-17 21:24:27,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2d575bc060ce15bcfc HTTP/1.1" 502 150 2025-12-17 21:24:27,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2d575bc060ce15bcfc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:28,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2c575bc060ce15bcf5 HTTP/1.1" 502 150 2025-12-17 21:24:28,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2c575bc060ce15bcf5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:28,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3d575bc060ce15bd2b 2025-12-17 21:24:28,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3d575bc060ce15bd2b HTTP/1.1" 502 150 2025-12-17 21:24:28,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3d575bc060ce15bd2b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:29,610 - lava_callback - INFO - Received callback for node 69407eb1575bc060ce15c34e INFO: 172.21.0.1:42364 - "POST /node/69407eb1575bc060ce15c34e HTTP/1.0" 202 Accepted 2025-12-17 21:24:29,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c27575bc060ce15bce6 2025-12-17 21:24:29,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c26575bc060ce15bce3 2025-12-17 21:24:29,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c27575bc060ce15bce6 HTTP/1.1" 502 150 2025-12-17 21:24:29,711 - lava_callback - ERROR - Error processing callback for node 69407c27575bc060ce15bce6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c27575bc060ce15bce6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c27575bc060ce15bce6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c27575bc060ce15bce6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:29,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c26575bc060ce15bce3 HTTP/1.1" 502 150 2025-12-17 21:24:29,713 - lava_callback - ERROR - Error processing callback for node 69407c26575bc060ce15bce3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c26575bc060ce15bce3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c26575bc060ce15bce3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c26575bc060ce15bce3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:29,713 - lava_callback - INFO - Starting async job submit for node 69407c3e575bc060ce15bd2e 2025-12-17 21:24:29,715 - lava_callback - INFO - Starting async job submit for node 69407c3f575bc060ce15bd31 2025-12-17 21:24:29,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:29,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:29,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3f575bc060ce15bd31 HTTP/1.1" 502 150 2025-12-17 21:24:29,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3f575bc060ce15bd31'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:29,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3f575bc060ce15bd31 2025-12-17 21:24:29,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3f575bc060ce15bd31 HTTP/1.1" 502 150 2025-12-17 21:24:29,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3f575bc060ce15bd31'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:29,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3e575bc060ce15bd2e HTTP/1.1" 502 150 2025-12-17 21:24:29,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3e575bc060ce15bd2e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:29,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3e575bc060ce15bd2e 2025-12-17 21:24:29,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3e575bc060ce15bd2e HTTP/1.1" 502 150 2025-12-17 21:24:29,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3e575bc060ce15bd2e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:29,882 - lava_callback - INFO - Received callback for node 69407eb1575bc060ce15c34b INFO: 172.21.0.1:57998 - "POST /node/69407eb1575bc060ce15c34b HTTP/1.0" 202 Accepted 2025-12-17 21:24:30,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c28575bc060ce15bce9 2025-12-17 21:24:30,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2e575bc060ce15bcff 2025-12-17 21:24:30,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2e575bc060ce15bcff HTTP/1.1" 502 150 2025-12-17 21:24:30,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2e575bc060ce15bcff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:31,535 - lava_callback - INFO - Received callback for node 69407e36575bc060ce15c291 INFO: 172.21.0.1:58010 - "POST /node/69407e36575bc060ce15c291 HTTP/1.0" 202 Accepted 2025-12-17 21:24:31,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3f575bc060ce15bd31 2025-12-17 21:24:31,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3f575bc060ce15bd31 HTTP/1.1" 502 150 2025-12-17 21:24:31,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3f575bc060ce15bd31'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:31,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3e575bc060ce15bd2e 2025-12-17 21:24:31,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3e575bc060ce15bd2e HTTP/1.1" 502 150 2025-12-17 21:24:31,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3e575bc060ce15bd2e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:32,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c29575bc060ce15bcec 2025-12-17 21:24:32,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c29575bc060ce15bcec HTTP/1.1" 502 150 2025-12-17 21:24:32,534 - lava_callback - ERROR - Error processing callback for node 69407c29575bc060ce15bcec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c29575bc060ce15bcec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c29575bc060ce15bcec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c29575bc060ce15bcec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:32,536 - lava_callback - INFO - Starting async job submit for node 69407b4f575bc060ce15b9db 2025-12-17 21:24:32,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:32,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b4f575bc060ce15b9db HTTP/1.1" 502 150 2025-12-17 21:24:32,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b4f575bc060ce15b9db'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:32,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b4f575bc060ce15b9db 2025-12-17 21:24:32,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b4f575bc060ce15b9db HTTP/1.1" 502 150 2025-12-17 21:24:32,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b4f575bc060ce15b9db'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:32,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c38575bc060ce15bd1a 2025-12-17 21:24:32,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c38575bc060ce15bd1a HTTP/1.1" 502 150 2025-12-17 21:24:32,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c38575bc060ce15bd1a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:32,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3b575bc060ce15bd27 2025-12-17 21:24:32,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3b575bc060ce15bd27 HTTP/1.1" 502 150 2025-12-17 21:24:32,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3b575bc060ce15bd27'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:33,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c32575bc060ce15f4a9 2025-12-17 21:24:33,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c32575bc060ce15f4a9 HTTP/1.1" 502 150 2025-12-17 21:24:33,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c32575bc060ce15f4a9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:33,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c37575bc060ce15bd13 2025-12-17 21:24:33,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c37575bc060ce15bd13 HTTP/1.1" 502 150 2025-12-17 21:24:33,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c37575bc060ce15bd13'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:34,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3a575bc060ce15bd20 2025-12-17 21:24:34,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3a575bc060ce15bd20 HTTP/1.1" 502 150 2025-12-17 21:24:34,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3a575bc060ce15bd20'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:34,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b4f575bc060ce15b9db 2025-12-17 21:24:34,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b4f575bc060ce15b9db HTTP/1.1" 502 150 2025-12-17 21:24:34,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b4f575bc060ce15b9db'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:35,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3f575bc060ce15bd31 2025-12-17 21:24:35,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3f575bc060ce15bd31 HTTP/1.1" 502 150 2025-12-17 21:24:35,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3f575bc060ce15bd31'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:35,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3e575bc060ce15bd2e 2025-12-17 21:24:35,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3e575bc060ce15bd2e HTTP/1.1" 502 150 2025-12-17 21:24:35,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3e575bc060ce15bd2e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:36,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2c575bc060ce15bcf5 2025-12-17 21:24:36,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2c575bc060ce15bcf5 HTTP/1.1" 502 150 2025-12-17 21:24:36,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c2c575bc060ce15bcf5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58012 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:24:36,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2a575bc060ce15bcf2 2025-12-17 21:24:36,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2a575bc060ce15bcf2 HTTP/1.1" 502 150 2025-12-17 21:24:36,527 - lava_callback - ERROR - Error processing callback for node 69407c2a575bc060ce15bcf2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c2a575bc060ce15bcf2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c2a575bc060ce15bcf2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c2a575bc060ce15bcf2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:36,529 - lava_callback - INFO - Starting async job submit for node 69407c41575bc060ce15bd34 2025-12-17 21:24:36,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:36,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c41575bc060ce15bd34 HTTP/1.1" 502 150 2025-12-17 21:24:36,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c41575bc060ce15bd34'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:36,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c41575bc060ce15bd34 2025-12-17 21:24:36,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c41575bc060ce15bd34 HTTP/1.1" 502 150 2025-12-17 21:24:36,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c41575bc060ce15bd34'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:36,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3d575bc060ce15bd2b 2025-12-17 21:24:36,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3d575bc060ce15bd2b HTTP/1.1" 502 150 2025-12-17 21:24:36,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3d575bc060ce15bd2b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:37,056 - lava_callback - INFO - Received callback for node 69407c89575bc060ce15be57 INFO: 172.21.0.1:58026 - "POST /node/69407c89575bc060ce15be57 HTTP/1.0" 202 Accepted 2025-12-17 21:24:37,818 - lava_callback - INFO - Received callback for node 69407c8b575bc060ce15be5e INFO: 172.21.0.1:58038 - "POST /node/69407c8b575bc060ce15be5e HTTP/1.0" 202 Accepted 2025-12-17 21:24:38,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c41575bc060ce15bd34 2025-12-17 21:24:38,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c41575bc060ce15bd34 HTTP/1.1" 502 150 2025-12-17 21:24:38,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c41575bc060ce15bd34'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:38,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b4f575bc060ce15b9db 2025-12-17 21:24:38,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b4f575bc060ce15b9db HTTP/1.1" 502 150 2025-12-17 21:24:38,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b4f575bc060ce15b9db'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:38,620 - lava_callback - INFO - Received callback for node 69407c91575bc060ce15be69 INFO: 172.21.0.1:58042 - "POST /node/69407c91575bc060ce15be69 HTTP/1.0" 202 Accepted 2025-12-17 21:24:39,028 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c16575bc060ce15f454 2025-12-17 21:24:39,353 - lava_callback - INFO - Received callback for node 69407c93575bc060ce15be6c INFO: 172.21.0.1:58056 - "POST /node/69407c93575bc060ce15be6c HTTP/1.0" 202 Accepted 2025-12-17 21:24:40,099 - lava_callback - INFO - Received callback for node 69407c94575bc060ce15be74 INFO: 172.21.0.1:35144 - "POST /node/69407c94575bc060ce15be74 HTTP/1.0" 202 Accepted 2025-12-17 21:24:40,838 - lava_callback - INFO - Received callback for node 69407c95575bc060ce15be77 INFO: 172.21.0.1:35152 - "POST /node/69407c95575bc060ce15be77 HTTP/1.0" 202 Accepted 2025-12-17 21:24:41,569 - lava_callback - INFO - Received callback for node 69407c97575bc060ce15be7a INFO: 172.21.0.1:35164 - "POST /node/69407c97575bc060ce15be7a HTTP/1.0" 202 Accepted 2025-12-17 21:24:42,316 - lava_callback - INFO - Received callback for node 69407c98575bc060ce15be81 INFO: 172.21.0.1:35180 - "POST /node/69407c98575bc060ce15be81 HTTP/1.0" 202 Accepted 2025-12-17 21:24:42,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c41575bc060ce15bd34 2025-12-17 21:24:42,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c41575bc060ce15bd34 HTTP/1.1" 502 150 2025-12-17 21:24:42,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c41575bc060ce15bd34'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:42,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409356575bc060ce160d16 2025-12-17 21:24:42,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409356575bc060ce160d16 HTTP/1.1" 502 150 2025-12-17 21:24:42,697 - lava_callback - ERROR - Error processing callback for node 69409356575bc060ce160d16: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409356575bc060ce160d16 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409356575bc060ce160d16 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409356575bc060ce160d16 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:42,698 - lava_callback - INFO - Starting async job submit for node 69407c45575bc060ce15bd6c 2025-12-17 21:24:42,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:42,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c45575bc060ce15bd6c HTTP/1.1" 502 150 2025-12-17 21:24:42,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c45575bc060ce15bd6c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:42,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c45575bc060ce15bd6c 2025-12-17 21:24:42,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c45575bc060ce15bd6c HTTP/1.1" 502 150 2025-12-17 21:24:42,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c45575bc060ce15bd6c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:43,067 - lava_callback - INFO - Received callback for node 69407c99575bc060ce15be86 INFO: 172.21.0.1:35196 - "POST /node/69407c99575bc060ce15be86 HTTP/1.0" 202 Accepted 2025-12-17 21:24:43,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2d575bc060ce15bcfc 2025-12-17 21:24:43,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2d575bc060ce15bcfc HTTP/1.1" 502 150 2025-12-17 21:24:43,551 - lava_callback - ERROR - Error processing callback for node 69407c2d575bc060ce15bcfc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c2d575bc060ce15bcfc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c2d575bc060ce15bcfc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c2d575bc060ce15bcfc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:43,553 - lava_callback - INFO - Starting async job submit for node 69407c46575bc060ce15bd76 2025-12-17 21:24:43,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:43,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c46575bc060ce15bd76 HTTP/1.1" 502 150 2025-12-17 21:24:43,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c46575bc060ce15bd76'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:43,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c46575bc060ce15bd76 2025-12-17 21:24:43,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c46575bc060ce15bd76 HTTP/1.1" 502 150 2025-12-17 21:24:43,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c46575bc060ce15bd76'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:43,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3f575bc060ce15bd31 2025-12-17 21:24:43,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3f575bc060ce15bd31 HTTP/1.1" 502 150 2025-12-17 21:24:43,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3f575bc060ce15bd31'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:43,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3e575bc060ce15bd2e 2025-12-17 21:24:43,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3e575bc060ce15bd2e HTTP/1.1" 502 150 2025-12-17 21:24:43,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c3e575bc060ce15bd2e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:44,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c45575bc060ce15bd6c 2025-12-17 21:24:44,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c45575bc060ce15bd6c HTTP/1.1" 502 150 2025-12-17 21:24:44,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c45575bc060ce15bd6c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:45,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c46575bc060ce15bd76 2025-12-17 21:24:45,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c46575bc060ce15bd76 HTTP/1.1" 502 150 2025-12-17 21:24:45,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c46575bc060ce15bd76'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:46,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c28575bc060ce15bce9 HTTP/1.1" 502 150 2025-12-17 21:24:46,128 - lava_callback - ERROR - Error processing callback for node 69407c28575bc060ce15bce9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c28575bc060ce15bce9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c28575bc060ce15bce9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c28575bc060ce15bce9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:46,130 - lava_callback - INFO - Starting async job submit for node 69407c47575bc060ce15bd79 2025-12-17 21:24:46,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:46,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c47575bc060ce15bd79 HTTP/1.1" 502 150 2025-12-17 21:24:46,163 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c47575bc060ce15bd79'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:46,164 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c47575bc060ce15bd79 2025-12-17 21:24:46,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c47575bc060ce15bd79 HTTP/1.1" 502 150 2025-12-17 21:24:46,164 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c47575bc060ce15bd79'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35198 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:24:46,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b4f575bc060ce15b9db 2025-12-17 21:24:46,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b4f575bc060ce15b9db HTTP/1.1" 502 150 2025-12-17 21:24:46,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b4f575bc060ce15b9db'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:46,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2e575bc060ce15bcff 2025-12-17 21:24:46,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2e575bc060ce15bcff HTTP/1.1" 502 150 2025-12-17 21:24:46,972 - lava_callback - ERROR - Error processing callback for node 69407c2e575bc060ce15bcff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c2e575bc060ce15bcff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c2e575bc060ce15bcff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c2e575bc060ce15bcff (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:46,973 - lava_callback - INFO - Starting async job submit for node 69407c49575bc060ce15bd7e 2025-12-17 21:24:46,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:46,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c49575bc060ce15bd7e HTTP/1.1" 502 150 2025-12-17 21:24:46,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c49575bc060ce15bd7e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:46,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c49575bc060ce15bd7e 2025-12-17 21:24:46,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c49575bc060ce15bd7e HTTP/1.1" 502 150 2025-12-17 21:24:46,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c49575bc060ce15bd7e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:48,164 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c47575bc060ce15bd79 2025-12-17 21:24:48,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c47575bc060ce15bd79 HTTP/1.1" 502 150 2025-12-17 21:24:48,166 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c47575bc060ce15bd79'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:48,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c38575bc060ce15bd1a 2025-12-17 21:24:48,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c38575bc060ce15bd1a HTTP/1.1" 502 150 2025-12-17 21:24:48,707 - lava_callback - ERROR - Error processing callback for node 69407c38575bc060ce15bd1a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c38575bc060ce15bd1a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c38575bc060ce15bd1a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c38575bc060ce15bd1a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:48,709 - lava_callback - INFO - Starting async job submit for node 69407c4a575bc060ce15bd85 2025-12-17 21:24:48,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:48,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4a575bc060ce15bd85 HTTP/1.1" 502 150 2025-12-17 21:24:48,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4a575bc060ce15bd85'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:48,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4a575bc060ce15bd85 2025-12-17 21:24:48,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4a575bc060ce15bd85 HTTP/1.1" 502 150 2025-12-17 21:24:48,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4a575bc060ce15bd85'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:48,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c45575bc060ce15bd6c 2025-12-17 21:24:48,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c45575bc060ce15bd6c HTTP/1.1" 502 150 2025-12-17 21:24:48,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c45575bc060ce15bd6c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:48,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3b575bc060ce15bd27 2025-12-17 21:24:48,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3b575bc060ce15bd27 HTTP/1.1" 502 150 2025-12-17 21:24:48,772 - lava_callback - ERROR - Error processing callback for node 69407c3b575bc060ce15bd27: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c3b575bc060ce15bd27 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c3b575bc060ce15bd27 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c3b575bc060ce15bd27 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:48,774 - lava_callback - INFO - Starting async job submit for node 69407c4b575bc060ce15bd8c 2025-12-17 21:24:48,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:48,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4b575bc060ce15bd8c HTTP/1.1" 502 150 2025-12-17 21:24:48,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4b575bc060ce15bd8c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:48,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4b575bc060ce15bd8c 2025-12-17 21:24:48,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4b575bc060ce15bd8c HTTP/1.1" 502 150 2025-12-17 21:24:48,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4b575bc060ce15bd8c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:48,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c49575bc060ce15bd7e 2025-12-17 21:24:48,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c49575bc060ce15bd7e HTTP/1.1" 502 150 2025-12-17 21:24:48,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c49575bc060ce15bd7e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:49,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c46575bc060ce15bd76 2025-12-17 21:24:49,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c46575bc060ce15bd76 HTTP/1.1" 502 150 2025-12-17 21:24:49,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c46575bc060ce15bd76'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:49,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c32575bc060ce15f4a9 2025-12-17 21:24:49,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c32575bc060ce15f4a9 HTTP/1.1" 502 150 2025-12-17 21:24:49,618 - lava_callback - ERROR - Error processing callback for node 69408c32575bc060ce15f4a9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c32575bc060ce15f4a9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c32575bc060ce15f4a9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c32575bc060ce15f4a9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:49,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c37575bc060ce15bd13 2025-12-17 21:24:49,620 - lava_callback - INFO - Starting async job submit for node 69407c4d575bc060ce15bd91 2025-12-17 21:24:49,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:49,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c37575bc060ce15bd13 HTTP/1.1" 502 150 2025-12-17 21:24:49,622 - lava_callback - ERROR - Error processing callback for node 69407c37575bc060ce15bd13: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c37575bc060ce15bd13 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c37575bc060ce15bd13 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c37575bc060ce15bd13 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:49,623 - lava_callback - INFO - Starting async job submit for node 69407e35575bc060ce15c274 2025-12-17 21:24:49,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:49,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4d575bc060ce15bd91 HTTP/1.1" 502 150 2025-12-17 21:24:49,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4d575bc060ce15bd91'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:49,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4d575bc060ce15bd91 2025-12-17 21:24:49,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4d575bc060ce15bd91 HTTP/1.1" 502 150 2025-12-17 21:24:49,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4d575bc060ce15bd91'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:49,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e35575bc060ce15c274 HTTP/1.1" 502 150 2025-12-17 21:24:49,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e35575bc060ce15c274'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:49,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e35575bc060ce15c274 2025-12-17 21:24:49,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e35575bc060ce15c274 HTTP/1.1" 502 150 2025-12-17 21:24:49,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e35575bc060ce15c274'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:50,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3a575bc060ce15bd20 2025-12-17 21:24:50,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c41575bc060ce15bd34 2025-12-17 21:24:50,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c41575bc060ce15bd34 HTTP/1.1" 502 150 2025-12-17 21:24:50,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c41575bc060ce15bd34'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:50,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4a575bc060ce15bd85 2025-12-17 21:24:50,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4a575bc060ce15bd85 HTTP/1.1" 502 150 2025-12-17 21:24:50,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4a575bc060ce15bd85'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:50,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4b575bc060ce15bd8c 2025-12-17 21:24:50,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4b575bc060ce15bd8c HTTP/1.1" 502 150 2025-12-17 21:24:50,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4b575bc060ce15bd8c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:51,563 - lava_callback - INFO - Received callback for node 69407d49575bc060ce15c03c INFO: 172.21.0.1:50112 - "POST /node/69407d49575bc060ce15c03c HTTP/1.0" 202 Accepted 2025-12-17 21:24:51,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4d575bc060ce15bd91 2025-12-17 21:24:51,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4d575bc060ce15bd91 HTTP/1.1" 502 150 2025-12-17 21:24:51,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4d575bc060ce15bd91'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:51,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e35575bc060ce15c274 2025-12-17 21:24:51,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e35575bc060ce15c274 HTTP/1.1" 502 150 2025-12-17 21:24:51,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e35575bc060ce15c274'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:52,168 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c47575bc060ce15bd79 2025-12-17 21:24:52,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c47575bc060ce15bd79 HTTP/1.1" 502 150 2025-12-17 21:24:52,170 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c47575bc060ce15bd79'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:52,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c2c575bc060ce15bcf5 2025-12-17 21:24:52,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c2c575bc060ce15bcf5 HTTP/1.1" 502 150 2025-12-17 21:24:52,467 - lava_callback - ERROR - Error processing callback for node 69407c2c575bc060ce15bcf5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c2c575bc060ce15bcf5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c2c575bc060ce15bcf5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c2c575bc060ce15bcf5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:52,470 - lava_callback - INFO - Starting async job submit for node 69407d48575bc060ce15c036 2025-12-17 21:24:52,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:52,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d48575bc060ce15c036 HTTP/1.1" 502 150 2025-12-17 21:24:52,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d48575bc060ce15c036'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:52,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d48575bc060ce15c036 2025-12-17 21:24:52,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d48575bc060ce15c036 HTTP/1.1" 502 150 2025-12-17 21:24:52,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d48575bc060ce15c036'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:52,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3d575bc060ce15bd2b 2025-12-17 21:24:52,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3d575bc060ce15bd2b HTTP/1.1" 502 150 2025-12-17 21:24:52,933 - lava_callback - ERROR - Error processing callback for node 69407c3d575bc060ce15bd2b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c3d575bc060ce15bd2b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c3d575bc060ce15bd2b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c3d575bc060ce15bd2b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:52,935 - lava_callback - INFO - Starting async job submit for node 69407c4f575bc060ce15bd9e 2025-12-17 21:24:52,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:52,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4f575bc060ce15bd9e HTTP/1.1" 502 150 2025-12-17 21:24:52,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4f575bc060ce15bd9e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:52,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4f575bc060ce15bd9e 2025-12-17 21:24:52,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4f575bc060ce15bd9e HTTP/1.1" 502 150 2025-12-17 21:24:52,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4f575bc060ce15bd9e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:52,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c49575bc060ce15bd7e 2025-12-17 21:24:52,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c49575bc060ce15bd7e HTTP/1.1" 502 150 2025-12-17 21:24:52,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c49575bc060ce15bd7e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:54,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c16575bc060ce15f454 HTTP/1.1" 502 150 2025-12-17 21:24:54,320 - lava_callback - ERROR - Error processing callback for node 69408c16575bc060ce15f454: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c16575bc060ce15f454 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c16575bc060ce15f454 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c16575bc060ce15f454 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:54,322 - lava_callback - INFO - Starting async job submit for node 69407c51575bc060ce15bda4 2025-12-17 21:24:54,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:54,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c51575bc060ce15bda4 HTTP/1.1" 502 150 2025-12-17 21:24:54,329 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c51575bc060ce15bda4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:54,329 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c51575bc060ce15bda4 2025-12-17 21:24:54,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c51575bc060ce15bda4 HTTP/1.1" 502 150 2025-12-17 21:24:54,330 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c51575bc060ce15bda4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:54,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d48575bc060ce15c036 2025-12-17 21:24:54,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d48575bc060ce15c036 HTTP/1.1" 502 150 2025-12-17 21:24:54,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d48575bc060ce15c036'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:54,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4a575bc060ce15bd85 2025-12-17 21:24:54,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4a575bc060ce15bd85 HTTP/1.1" 502 150 2025-12-17 21:24:54,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4a575bc060ce15bd85'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:54,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4b575bc060ce15bd8c 2025-12-17 21:24:54,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4b575bc060ce15bd8c HTTP/1.1" 502 150 2025-12-17 21:24:54,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4b575bc060ce15bd8c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:54,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4f575bc060ce15bd9e 2025-12-17 21:24:54,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4f575bc060ce15bd9e HTTP/1.1" 502 150 2025-12-17 21:24:54,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4f575bc060ce15bd9e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:55,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4d575bc060ce15bd91 2025-12-17 21:24:55,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4d575bc060ce15bd91 HTTP/1.1" 502 150 2025-12-17 21:24:55,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4d575bc060ce15bd91'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:55,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e35575bc060ce15c274 2025-12-17 21:24:55,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e35575bc060ce15c274 HTTP/1.1" 502 150 2025-12-17 21:24:55,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e35575bc060ce15c274'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:56,330 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c51575bc060ce15bda4 2025-12-17 21:24:56,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c51575bc060ce15bda4 HTTP/1.1" 502 150 2025-12-17 21:24:56,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c51575bc060ce15bda4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50124 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:24:56,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c45575bc060ce15bd6c 2025-12-17 21:24:56,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c45575bc060ce15bd6c HTTP/1.1" 502 150 2025-12-17 21:24:56,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c45575bc060ce15bd6c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:57,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c46575bc060ce15bd76 2025-12-17 21:24:57,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c46575bc060ce15bd76 HTTP/1.1" 502 150 2025-12-17 21:24:57,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c46575bc060ce15bd76'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:58,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d48575bc060ce15c036 2025-12-17 21:24:58,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d48575bc060ce15c036 HTTP/1.1" 502 150 2025-12-17 21:24:58,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d48575bc060ce15c036'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:58,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4f575bc060ce15bd9e 2025-12-17 21:24:58,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4f575bc060ce15bd9e HTTP/1.1" 502 150 2025-12-17 21:24:58,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4f575bc060ce15bd9e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:59,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3f575bc060ce15bd31 2025-12-17 21:24:59,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3f575bc060ce15bd31 HTTP/1.1" 502 150 2025-12-17 21:24:59,729 - lava_callback - ERROR - Error processing callback for node 69407c3f575bc060ce15bd31: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c3f575bc060ce15bd31 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c3f575bc060ce15bd31 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c3f575bc060ce15bd31 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:59,730 - lava_callback - INFO - Starting async job submit for node 69407c54575bc060ce15bdaa 2025-12-17 21:24:59,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:59,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c54575bc060ce15bdaa HTTP/1.1" 502 150 2025-12-17 21:24:59,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c54575bc060ce15bdaa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:59,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c54575bc060ce15bdaa 2025-12-17 21:24:59,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c54575bc060ce15bdaa HTTP/1.1" 502 150 2025-12-17 21:24:59,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c54575bc060ce15bdaa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:59,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c3e575bc060ce15bd2e 2025-12-17 21:24:59,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3e575bc060ce15bd2e HTTP/1.1" 502 150 2025-12-17 21:24:59,752 - lava_callback - ERROR - Error processing callback for node 69407c3e575bc060ce15bd2e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c3e575bc060ce15bd2e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c3e575bc060ce15bd2e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c3e575bc060ce15bd2e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:24:59,754 - lava_callback - INFO - Starting async job submit for node 69407c55575bc060ce15bdad 2025-12-17 21:24:59,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:24:59,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c55575bc060ce15bdad HTTP/1.1" 502 150 2025-12-17 21:24:59,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c55575bc060ce15bdad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:24:59,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c55575bc060ce15bdad 2025-12-17 21:24:59,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c55575bc060ce15bdad HTTP/1.1" 502 150 2025-12-17 21:24:59,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c55575bc060ce15bdad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:00,170 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c47575bc060ce15bd79 2025-12-17 21:25:00,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c47575bc060ce15bd79 HTTP/1.1" 502 150 2025-12-17 21:25:00,171 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c47575bc060ce15bd79'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:00,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c51575bc060ce15bda4 2025-12-17 21:25:00,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c51575bc060ce15bda4 HTTP/1.1" 502 150 2025-12-17 21:25:00,333 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c51575bc060ce15bda4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:00,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c49575bc060ce15bd7e 2025-12-17 21:25:00,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c49575bc060ce15bd7e HTTP/1.1" 502 150 2025-12-17 21:25:00,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c49575bc060ce15bd7e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53298 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:25:01,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c54575bc060ce15bdaa 2025-12-17 21:25:01,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c54575bc060ce15bdaa HTTP/1.1" 502 150 2025-12-17 21:25:01,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c54575bc060ce15bdaa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:01,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c55575bc060ce15bdad 2025-12-17 21:25:01,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c55575bc060ce15bdad HTTP/1.1" 502 150 2025-12-17 21:25:01,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c55575bc060ce15bdad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:02,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b4f575bc060ce15b9db 2025-12-17 21:25:02,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b4f575bc060ce15b9db HTTP/1.1" 502 150 2025-12-17 21:25:02,578 - lava_callback - ERROR - Error processing callback for node 69407b4f575bc060ce15b9db: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b4f575bc060ce15b9db (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b4f575bc060ce15b9db (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b4f575bc060ce15b9db (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:02,580 - lava_callback - INFO - Starting async job submit for node 69407c57575bc060ce15bdb0 2025-12-17 21:25:02,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:02,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c57575bc060ce15bdb0 HTTP/1.1" 502 150 2025-12-17 21:25:02,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c57575bc060ce15bdb0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:02,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c57575bc060ce15bdb0 2025-12-17 21:25:02,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c57575bc060ce15bdb0 HTTP/1.1" 502 150 2025-12-17 21:25:02,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c57575bc060ce15bdb0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:02,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4a575bc060ce15bd85 2025-12-17 21:25:02,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4a575bc060ce15bd85 HTTP/1.1" 502 150 2025-12-17 21:25:02,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4a575bc060ce15bd85'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:02,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4b575bc060ce15bd8c 2025-12-17 21:25:02,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4b575bc060ce15bd8c HTTP/1.1" 502 150 2025-12-17 21:25:02,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4b575bc060ce15bd8c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:03,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4d575bc060ce15bd91 2025-12-17 21:25:03,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4d575bc060ce15bd91 HTTP/1.1" 502 150 2025-12-17 21:25:03,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4d575bc060ce15bd91'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:03,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e35575bc060ce15c274 2025-12-17 21:25:03,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e35575bc060ce15c274 HTTP/1.1" 502 150 2025-12-17 21:25:03,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e35575bc060ce15c274'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:04,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c57575bc060ce15bdb0 2025-12-17 21:25:04,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c57575bc060ce15bdb0 HTTP/1.1" 502 150 2025-12-17 21:25:04,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c57575bc060ce15bdb0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:05,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c3a575bc060ce15bd20 HTTP/1.1" 502 150 2025-12-17 21:25:05,328 - lava_callback - ERROR - Error processing callback for node 69407c3a575bc060ce15bd20: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c3a575bc060ce15bd20 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c3a575bc060ce15bd20 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c3a575bc060ce15bd20 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:05,330 - lava_callback - INFO - Starting async job submit for node 69407c58575bc060ce15bdb3 2025-12-17 21:25:05,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:05,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c58575bc060ce15bdb3 HTTP/1.1" 502 150 2025-12-17 21:25:05,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c58575bc060ce15bdb3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:05,342 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c58575bc060ce15bdb3 2025-12-17 21:25:05,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c58575bc060ce15bdb3 HTTP/1.1" 502 150 2025-12-17 21:25:05,343 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c58575bc060ce15bdb3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:05,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c54575bc060ce15bdaa 2025-12-17 21:25:05,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c54575bc060ce15bdaa HTTP/1.1" 502 150 2025-12-17 21:25:05,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c54575bc060ce15bdaa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:05,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c55575bc060ce15bdad 2025-12-17 21:25:05,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c55575bc060ce15bdad HTTP/1.1" 502 150 2025-12-17 21:25:05,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c55575bc060ce15bdad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53304 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:25:06,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d48575bc060ce15c036 2025-12-17 21:25:06,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d48575bc060ce15c036 HTTP/1.1" 502 150 2025-12-17 21:25:06,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d48575bc060ce15c036'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:06,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c41575bc060ce15bd34 2025-12-17 21:25:06,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c41575bc060ce15bd34 HTTP/1.1" 502 150 2025-12-17 21:25:06,542 - lava_callback - ERROR - Error processing callback for node 69407c41575bc060ce15bd34: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c41575bc060ce15bd34 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c41575bc060ce15bd34 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c41575bc060ce15bd34 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:06,544 - lava_callback - INFO - Starting async job submit for node 69407c59575bc060ce15bdb6 2025-12-17 21:25:06,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:06,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c59575bc060ce15bdb6 HTTP/1.1" 502 150 2025-12-17 21:25:06,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c59575bc060ce15bdb6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:06,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c59575bc060ce15bdb6 2025-12-17 21:25:06,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c59575bc060ce15bdb6 HTTP/1.1" 502 150 2025-12-17 21:25:06,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c59575bc060ce15bdb6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:06,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4f575bc060ce15bd9e 2025-12-17 21:25:06,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4f575bc060ce15bd9e HTTP/1.1" 502 150 2025-12-17 21:25:06,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4f575bc060ce15bd9e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:07,343 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c58575bc060ce15bdb3 2025-12-17 21:25:07,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c58575bc060ce15bdb3 HTTP/1.1" 502 150 2025-12-17 21:25:07,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c58575bc060ce15bdb3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:08,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c51575bc060ce15bda4 2025-12-17 21:25:08,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c51575bc060ce15bda4 HTTP/1.1" 502 150 2025-12-17 21:25:08,334 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c51575bc060ce15bda4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:08,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c59575bc060ce15bdb6 2025-12-17 21:25:08,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c59575bc060ce15bdb6 HTTP/1.1" 502 150 2025-12-17 21:25:08,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c59575bc060ce15bdb6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:08,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c57575bc060ce15bdb0 2025-12-17 21:25:08,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c57575bc060ce15bdb0 HTTP/1.1" 502 150 2025-12-17 21:25:08,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c57575bc060ce15bdb0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:11,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c58575bc060ce15bdb3 2025-12-17 21:25:11,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c58575bc060ce15bdb3 HTTP/1.1" 502 150 2025-12-17 21:25:11,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c58575bc060ce15bdb3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:12,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c59575bc060ce15bdb6 2025-12-17 21:25:12,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c59575bc060ce15bdb6 HTTP/1.1" 502 150 2025-12-17 21:25:12,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c59575bc060ce15bdb6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:12,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c45575bc060ce15bd6c 2025-12-17 21:25:12,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c45575bc060ce15bd6c HTTP/1.1" 502 150 2025-12-17 21:25:12,734 - lava_callback - ERROR - Error processing callback for node 69407c45575bc060ce15bd6c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c45575bc060ce15bd6c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c45575bc060ce15bd6c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c45575bc060ce15bd6c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:12,736 - lava_callback - INFO - Starting async job submit for node 69407c5a575bc060ce15bdb9 2025-12-17 21:25:12,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:12,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5a575bc060ce15bdb9 HTTP/1.1" 502 150 2025-12-17 21:25:12,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5a575bc060ce15bdb9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:12,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5a575bc060ce15bdb9 2025-12-17 21:25:12,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5a575bc060ce15bdb9 HTTP/1.1" 502 150 2025-12-17 21:25:12,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5a575bc060ce15bdb9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:13,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c46575bc060ce15bd76 2025-12-17 21:25:13,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c46575bc060ce15bd76 HTTP/1.1" 502 150 2025-12-17 21:25:13,585 - lava_callback - ERROR - Error processing callback for node 69407c46575bc060ce15bd76: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c46575bc060ce15bd76 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c46575bc060ce15bd76 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c46575bc060ce15bd76 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:13,587 - lava_callback - INFO - Starting async job submit for node 69407c5b575bc060ce15bdbc 2025-12-17 21:25:13,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:13,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5b575bc060ce15bdbc HTTP/1.1" 502 150 2025-12-17 21:25:13,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5b575bc060ce15bdbc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:13,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5b575bc060ce15bdbc 2025-12-17 21:25:13,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5b575bc060ce15bdbc HTTP/1.1" 502 150 2025-12-17 21:25:13,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5b575bc060ce15bdbc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:13,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c54575bc060ce15bdaa 2025-12-17 21:25:13,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c54575bc060ce15bdaa HTTP/1.1" 502 150 2025-12-17 21:25:13,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c54575bc060ce15bdaa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:13,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c55575bc060ce15bdad 2025-12-17 21:25:13,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c55575bc060ce15bdad HTTP/1.1" 502 150 2025-12-17 21:25:13,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c55575bc060ce15bdad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:14,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5a575bc060ce15bdb9 2025-12-17 21:25:14,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5a575bc060ce15bdb9 HTTP/1.1" 502 150 2025-12-17 21:25:14,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5a575bc060ce15bdb9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57646 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:25:15,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5b575bc060ce15bdbc 2025-12-17 21:25:15,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5b575bc060ce15bdbc HTTP/1.1" 502 150 2025-12-17 21:25:15,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5b575bc060ce15bdbc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:16,171 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c47575bc060ce15bd79 2025-12-17 21:25:16,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c47575bc060ce15bd79 HTTP/1.1" 502 150 2025-12-17 21:25:16,173 - lava_callback - ERROR - Error processing callback for node 69407c47575bc060ce15bd79: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c47575bc060ce15bd79 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c47575bc060ce15bd79 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c47575bc060ce15bd79 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:16,174 - lava_callback - INFO - Starting async job submit for node 69407c5d575bc060ce15bdc0 2025-12-17 21:25:16,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:16,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5d575bc060ce15bdc0 HTTP/1.1" 502 150 2025-12-17 21:25:16,181 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5d575bc060ce15bdc0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:16,181 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5d575bc060ce15bdc0 2025-12-17 21:25:16,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5d575bc060ce15bdc0 HTTP/1.1" 502 150 2025-12-17 21:25:16,182 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5d575bc060ce15bdc0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57656 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:25:16,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c57575bc060ce15bdb0 2025-12-17 21:25:16,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c57575bc060ce15bdb0 HTTP/1.1" 502 150 2025-12-17 21:25:16,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c57575bc060ce15bdb0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:16,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c49575bc060ce15bd7e 2025-12-17 21:25:16,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c49575bc060ce15bd7e HTTP/1.1" 502 150 2025-12-17 21:25:16,987 - lava_callback - ERROR - Error processing callback for node 69407c49575bc060ce15bd7e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c49575bc060ce15bd7e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c49575bc060ce15bd7e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c49575bc060ce15bd7e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:16,989 - lava_callback - INFO - Starting async job submit for node 69407c5e575bc060ce15bdc7 2025-12-17 21:25:16,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:17,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5e575bc060ce15bdc7 HTTP/1.1" 502 150 2025-12-17 21:25:17,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5e575bc060ce15bdc7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:17,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5e575bc060ce15bdc7 2025-12-17 21:25:17,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5e575bc060ce15bdc7 HTTP/1.1" 502 150 2025-12-17 21:25:17,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5e575bc060ce15bdc7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:18,182 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5d575bc060ce15bdc0 2025-12-17 21:25:18,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5d575bc060ce15bdc0 HTTP/1.1" 502 150 2025-12-17 21:25:18,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5d575bc060ce15bdc0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:18,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4a575bc060ce15bd85 2025-12-17 21:25:18,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4a575bc060ce15bd85 HTTP/1.1" 502 150 2025-12-17 21:25:18,731 - lava_callback - ERROR - Error processing callback for node 69407c4a575bc060ce15bd85: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c4a575bc060ce15bd85 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c4a575bc060ce15bd85 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c4a575bc060ce15bd85 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:18,734 - lava_callback - INFO - Starting async job submit for node 69407c5f575bc060ce15bdcb 2025-12-17 21:25:18,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:18,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5f575bc060ce15bdcb HTTP/1.1" 502 150 2025-12-17 21:25:18,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5f575bc060ce15bdcb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:18,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5f575bc060ce15bdcb 2025-12-17 21:25:18,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5f575bc060ce15bdcb HTTP/1.1" 502 150 2025-12-17 21:25:18,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5f575bc060ce15bdcb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:18,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5a575bc060ce15bdb9 2025-12-17 21:25:18,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5a575bc060ce15bdb9 HTTP/1.1" 502 150 2025-12-17 21:25:18,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5a575bc060ce15bdb9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:18,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4b575bc060ce15bd8c 2025-12-17 21:25:18,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4b575bc060ce15bd8c HTTP/1.1" 502 150 2025-12-17 21:25:18,793 - lava_callback - ERROR - Error processing callback for node 69407c4b575bc060ce15bd8c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c4b575bc060ce15bd8c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c4b575bc060ce15bd8c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c4b575bc060ce15bd8c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:18,795 - lava_callback - INFO - Starting async job submit for node 69407c61575bc060ce15bdd0 2025-12-17 21:25:18,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:18,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c61575bc060ce15bdd0 HTTP/1.1" 502 150 2025-12-17 21:25:18,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c61575bc060ce15bdd0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:18,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c61575bc060ce15bdd0 2025-12-17 21:25:18,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c61575bc060ce15bdd0 HTTP/1.1" 502 150 2025-12-17 21:25:18,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c61575bc060ce15bdd0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:19,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5e575bc060ce15bdc7 2025-12-17 21:25:19,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5e575bc060ce15bdc7 HTTP/1.1" 502 150 2025-12-17 21:25:19,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5e575bc060ce15bdc7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:19,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c58575bc060ce15bdb3 2025-12-17 21:25:19,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c58575bc060ce15bdb3 HTTP/1.1" 502 150 2025-12-17 21:25:19,350 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c58575bc060ce15bdb3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:19,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5b575bc060ce15bdbc 2025-12-17 21:25:19,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5b575bc060ce15bdbc HTTP/1.1" 502 150 2025-12-17 21:25:19,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5b575bc060ce15bdbc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:19,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4d575bc060ce15bd91 2025-12-17 21:25:19,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4d575bc060ce15bd91 HTTP/1.1" 502 150 2025-12-17 21:25:19,668 - lava_callback - ERROR - Error processing callback for node 69407c4d575bc060ce15bd91: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c4d575bc060ce15bd91 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c4d575bc060ce15bd91 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c4d575bc060ce15bd91 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:19,670 - lava_callback - INFO - Starting async job submit for node 69407c62575bc060ce15bdd5 2025-12-17 21:25:19,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:19,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e35575bc060ce15c274 2025-12-17 21:25:19,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e35575bc060ce15c274 HTTP/1.1" 502 150 2025-12-17 21:25:19,676 - lava_callback - ERROR - Error processing callback for node 69407e35575bc060ce15c274: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e35575bc060ce15c274 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e35575bc060ce15c274 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e35575bc060ce15c274 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:19,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c62575bc060ce15bdd5 HTTP/1.1" 502 150 2025-12-17 21:25:19,678 - lava_callback - INFO - Starting async job submit for node 69407eb0575bc060ce15c33c 2025-12-17 21:25:19,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c62575bc060ce15bdd5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:19,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:19,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c62575bc060ce15bdd5 2025-12-17 21:25:19,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c62575bc060ce15bdd5 HTTP/1.1" 502 150 2025-12-17 21:25:19,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c62575bc060ce15bdd5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:19,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb0575bc060ce15c33c HTTP/1.1" 502 150 2025-12-17 21:25:19,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb0575bc060ce15c33c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:19,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb0575bc060ce15c33c 2025-12-17 21:25:19,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb0575bc060ce15c33c HTTP/1.1" 502 150 2025-12-17 21:25:19,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb0575bc060ce15c33c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:20,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c59575bc060ce15bdb6 2025-12-17 21:25:20,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c59575bc060ce15bdb6 HTTP/1.1" 502 150 2025-12-17 21:25:20,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c59575bc060ce15bdb6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:20,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5f575bc060ce15bdcb 2025-12-17 21:25:20,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5f575bc060ce15bdcb HTTP/1.1" 502 150 2025-12-17 21:25:20,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5f575bc060ce15bdcb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:20,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c61575bc060ce15bdd0 2025-12-17 21:25:20,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c61575bc060ce15bdd0 HTTP/1.1" 502 150 2025-12-17 21:25:20,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c61575bc060ce15bdd0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:21,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c62575bc060ce15bdd5 2025-12-17 21:25:21,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c62575bc060ce15bdd5 HTTP/1.1" 502 150 2025-12-17 21:25:21,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c62575bc060ce15bdd5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:21,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb0575bc060ce15c33c 2025-12-17 21:25:21,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb0575bc060ce15c33c HTTP/1.1" 502 150 2025-12-17 21:25:21,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb0575bc060ce15c33c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:22,183 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5d575bc060ce15bdc0 2025-12-17 21:25:22,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5d575bc060ce15bdc0 HTTP/1.1" 502 150 2025-12-17 21:25:22,184 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5d575bc060ce15bdc0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:22,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d48575bc060ce15c036 2025-12-17 21:25:22,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d48575bc060ce15c036 HTTP/1.1" 502 150 2025-12-17 21:25:22,512 - lava_callback - ERROR - Error processing callback for node 69407d48575bc060ce15c036: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d48575bc060ce15c036 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d48575bc060ce15c036 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d48575bc060ce15c036 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:22,513 - lava_callback - INFO - Starting async job submit for node 69407e36575bc060ce15c288 2025-12-17 21:25:22,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:22,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e36575bc060ce15c288 HTTP/1.1" 502 150 2025-12-17 21:25:22,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e36575bc060ce15c288'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:22,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e36575bc060ce15c288 2025-12-17 21:25:22,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e36575bc060ce15c288 HTTP/1.1" 502 150 2025-12-17 21:25:22,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e36575bc060ce15c288'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:22,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4f575bc060ce15bd9e 2025-12-17 21:25:22,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4f575bc060ce15bd9e HTTP/1.1" 502 150 2025-12-17 21:25:22,952 - lava_callback - ERROR - Error processing callback for node 69407c4f575bc060ce15bd9e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c4f575bc060ce15bd9e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c4f575bc060ce15bd9e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c4f575bc060ce15bd9e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:22,955 - lava_callback - INFO - Starting async job submit for node 69407c64575bc060ce15bde0 2025-12-17 21:25:22,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:22,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c64575bc060ce15bde0 HTTP/1.1" 502 150 2025-12-17 21:25:22,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c64575bc060ce15bde0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:22,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c64575bc060ce15bde0 2025-12-17 21:25:22,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c64575bc060ce15bde0 HTTP/1.1" 502 150 2025-12-17 21:25:22,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c64575bc060ce15bde0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:23,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5e575bc060ce15bdc7 2025-12-17 21:25:24,335 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c51575bc060ce15bda4 2025-12-17 21:25:24,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c51575bc060ce15bda4 HTTP/1.1" 502 150 2025-12-17 21:25:24,336 - lava_callback - ERROR - Error processing callback for node 69407c51575bc060ce15bda4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c51575bc060ce15bda4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c51575bc060ce15bda4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c51575bc060ce15bda4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:24,339 - lava_callback - INFO - Starting async job submit for node 69407c65575bc060ce15bde7 2025-12-17 21:25:24,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:24,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c65575bc060ce15bde7 HTTP/1.1" 502 150 2025-12-17 21:25:24,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c65575bc060ce15bde7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:24,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c65575bc060ce15bde7 2025-12-17 21:25:24,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c65575bc060ce15bde7 HTTP/1.1" 502 150 2025-12-17 21:25:24,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c65575bc060ce15bde7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:24,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e36575bc060ce15c288 2025-12-17 21:25:24,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e36575bc060ce15c288 HTTP/1.1" 502 150 2025-12-17 21:25:24,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e36575bc060ce15c288'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:24,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5f575bc060ce15bdcb 2025-12-17 21:25:24,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5f575bc060ce15bdcb HTTP/1.1" 502 150 2025-12-17 21:25:24,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5f575bc060ce15bdcb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:24,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c61575bc060ce15bdd0 2025-12-17 21:25:24,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c61575bc060ce15bdd0 HTTP/1.1" 502 150 2025-12-17 21:25:24,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c61575bc060ce15bdd0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:24,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c64575bc060ce15bde0 2025-12-17 21:25:24,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c64575bc060ce15bde0 HTTP/1.1" 502 150 2025-12-17 21:25:24,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c64575bc060ce15bde0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:25,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c62575bc060ce15bdd5 2025-12-17 21:25:25,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c62575bc060ce15bdd5 HTTP/1.1" 502 150 2025-12-17 21:25:25,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c62575bc060ce15bdd5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:25,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb0575bc060ce15c33c 2025-12-17 21:25:25,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb0575bc060ce15c33c HTTP/1.1" 502 150 2025-12-17 21:25:25,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb0575bc060ce15c33c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:26,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c65575bc060ce15bde7 2025-12-17 21:25:26,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c65575bc060ce15bde7 HTTP/1.1" 502 150 2025-12-17 21:25:26,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c65575bc060ce15bde7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55866 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:25:26,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5a575bc060ce15bdb9 2025-12-17 21:25:26,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5a575bc060ce15bdb9 HTTP/1.1" 502 150 2025-12-17 21:25:26,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5a575bc060ce15bdb9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:27,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5b575bc060ce15bdbc 2025-12-17 21:25:27,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5b575bc060ce15bdbc HTTP/1.1" 502 150 2025-12-17 21:25:27,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5b575bc060ce15bdbc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:28,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e36575bc060ce15c288 2025-12-17 21:25:28,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e36575bc060ce15c288 HTTP/1.1" 502 150 2025-12-17 21:25:28,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e36575bc060ce15c288'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:28,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c64575bc060ce15bde0 2025-12-17 21:25:28,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c64575bc060ce15bde0 HTTP/1.1" 502 150 2025-12-17 21:25:28,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c64575bc060ce15bde0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:29,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c54575bc060ce15bdaa 2025-12-17 21:25:29,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c54575bc060ce15bdaa HTTP/1.1" 502 150 2025-12-17 21:25:29,753 - lava_callback - ERROR - Error processing callback for node 69407c54575bc060ce15bdaa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c54575bc060ce15bdaa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c54575bc060ce15bdaa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c54575bc060ce15bdaa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:29,755 - lava_callback - INFO - Starting async job submit for node 69407c66575bc060ce15bdea 2025-12-17 21:25:29,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:29,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c66575bc060ce15bdea HTTP/1.1" 502 150 2025-12-17 21:25:29,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c66575bc060ce15bdea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:29,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c66575bc060ce15bdea 2025-12-17 21:25:29,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c66575bc060ce15bdea HTTP/1.1" 502 150 2025-12-17 21:25:29,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c66575bc060ce15bdea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:29,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c55575bc060ce15bdad 2025-12-17 21:25:29,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c55575bc060ce15bdad HTTP/1.1" 502 150 2025-12-17 21:25:29,767 - lava_callback - ERROR - Error processing callback for node 69407c55575bc060ce15bdad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c55575bc060ce15bdad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c55575bc060ce15bdad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c55575bc060ce15bdad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:29,769 - lava_callback - INFO - Starting async job submit for node 69407c68575bc060ce15bded 2025-12-17 21:25:29,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:29,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c68575bc060ce15bded HTTP/1.1" 502 150 2025-12-17 21:25:29,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c68575bc060ce15bded'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:29,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c68575bc060ce15bded 2025-12-17 21:25:29,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c68575bc060ce15bded HTTP/1.1" 502 150 2025-12-17 21:25:29,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c68575bc060ce15bded'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:30,185 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5d575bc060ce15bdc0 2025-12-17 21:25:30,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5d575bc060ce15bdc0 HTTP/1.1" 502 150 2025-12-17 21:25:30,186 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5d575bc060ce15bdc0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:30,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c65575bc060ce15bde7 2025-12-17 21:25:30,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c65575bc060ce15bde7 HTTP/1.1" 502 150 2025-12-17 21:25:30,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c65575bc060ce15bde7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:31,762 - lava_callback - INFO - Received callback for node 69407d49575bc060ce15c03f 2025-12-17 21:25:31,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c66575bc060ce15bdea INFO: 172.21.0.1:33188 - "POST /node/69407d49575bc060ce15c03f HTTP/1.0" 202 Accepted 2025-12-17 21:25:31,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c66575bc060ce15bdea HTTP/1.1" 502 150 2025-12-17 21:25:31,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c66575bc060ce15bdea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:31,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c68575bc060ce15bded 2025-12-17 21:25:31,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c68575bc060ce15bded HTTP/1.1" 502 150 2025-12-17 21:25:31,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c68575bc060ce15bded'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:32,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c57575bc060ce15bdb0 2025-12-17 21:25:32,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c57575bc060ce15bdb0 HTTP/1.1" 502 150 2025-12-17 21:25:32,593 - lava_callback - ERROR - Error processing callback for node 69407c57575bc060ce15bdb0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c57575bc060ce15bdb0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c57575bc060ce15bdb0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c57575bc060ce15bdb0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:32,595 - lava_callback - INFO - Starting async job submit for node 69407c69575bc060ce15bdf0 2025-12-17 21:25:32,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:32,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c69575bc060ce15bdf0 HTTP/1.1" 502 150 2025-12-17 21:25:32,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c69575bc060ce15bdf0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:32,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c69575bc060ce15bdf0 2025-12-17 21:25:32,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c69575bc060ce15bdf0 HTTP/1.1" 502 150 2025-12-17 21:25:32,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c69575bc060ce15bdf0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:32,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5f575bc060ce15bdcb 2025-12-17 21:25:32,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5f575bc060ce15bdcb HTTP/1.1" 502 150 2025-12-17 21:25:32,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5f575bc060ce15bdcb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:32,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c61575bc060ce15bdd0 2025-12-17 21:25:32,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c61575bc060ce15bdd0 HTTP/1.1" 502 150 2025-12-17 21:25:32,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c61575bc060ce15bdd0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:33,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c62575bc060ce15bdd5 2025-12-17 21:25:33,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c62575bc060ce15bdd5 HTTP/1.1" 502 150 2025-12-17 21:25:33,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c62575bc060ce15bdd5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:33,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb0575bc060ce15c33c 2025-12-17 21:25:33,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb0575bc060ce15c33c HTTP/1.1" 502 150 2025-12-17 21:25:33,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb0575bc060ce15c33c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:34,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c69575bc060ce15bdf0 2025-12-17 21:25:34,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c69575bc060ce15bdf0 HTTP/1.1" 502 150 2025-12-17 21:25:34,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c69575bc060ce15bdf0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:35,350 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c58575bc060ce15bdb3 2025-12-17 21:25:35,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c58575bc060ce15bdb3 HTTP/1.1" 502 150 2025-12-17 21:25:35,352 - lava_callback - ERROR - Error processing callback for node 69407c58575bc060ce15bdb3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c58575bc060ce15bdb3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c58575bc060ce15bdb3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c58575bc060ce15bdb3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:35,354 - lava_callback - INFO - Starting async job submit for node 69407c6a575bc060ce15bdf3 2025-12-17 21:25:35,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:35,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6a575bc060ce15bdf3 HTTP/1.1" 502 150 2025-12-17 21:25:35,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6a575bc060ce15bdf3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:35,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6a575bc060ce15bdf3 2025-12-17 21:25:35,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6a575bc060ce15bdf3 HTTP/1.1" 502 150 2025-12-17 21:25:35,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6a575bc060ce15bdf3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:35,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c66575bc060ce15bdea 2025-12-17 21:25:35,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c66575bc060ce15bdea HTTP/1.1" 502 150 2025-12-17 21:25:35,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c66575bc060ce15bdea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:35,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c68575bc060ce15bded 2025-12-17 21:25:35,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c68575bc060ce15bded HTTP/1.1" 502 150 2025-12-17 21:25:35,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c68575bc060ce15bded'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:36,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c59575bc060ce15bdb6 2025-12-17 21:25:36,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c59575bc060ce15bdb6 HTTP/1.1" 502 150 2025-12-17 21:25:36,558 - lava_callback - ERROR - Error processing callback for node 69407c59575bc060ce15bdb6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c59575bc060ce15bdb6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c59575bc060ce15bdb6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c59575bc060ce15bdb6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:36,560 - lava_callback - INFO - Starting async job submit for node 69407c6b575bc060ce15bdf6 2025-12-17 21:25:36,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:36,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6b575bc060ce15bdf6 HTTP/1.1" 502 150 2025-12-17 21:25:36,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6b575bc060ce15bdf6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:36,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6b575bc060ce15bdf6 2025-12-17 21:25:36,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6b575bc060ce15bdf6 HTTP/1.1" 502 150 2025-12-17 21:25:36,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6b575bc060ce15bdf6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:36,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e36575bc060ce15c288 2025-12-17 21:25:36,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e36575bc060ce15c288 HTTP/1.1" 502 150 2025-12-17 21:25:36,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e36575bc060ce15c288'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33190 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:25:36,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c64575bc060ce15bde0 2025-12-17 21:25:36,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c64575bc060ce15bde0 HTTP/1.1" 502 150 2025-12-17 21:25:36,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c64575bc060ce15bde0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:37,363 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6a575bc060ce15bdf3 2025-12-17 21:25:38,351 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c65575bc060ce15bde7 2025-12-17 21:25:38,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c65575bc060ce15bde7 HTTP/1.1" 502 150 2025-12-17 21:25:38,352 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c65575bc060ce15bde7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:38,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5e575bc060ce15bdc7 HTTP/1.1" 502 150 2025-12-17 21:25:38,352 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5e575bc060ce15bdc7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:38,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6b575bc060ce15bdf6 2025-12-17 21:25:38,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6b575bc060ce15bdf6 HTTP/1.1" 502 150 2025-12-17 21:25:38,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6b575bc060ce15bdf6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:38,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c69575bc060ce15bdf0 2025-12-17 21:25:38,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c69575bc060ce15bdf0 HTTP/1.1" 502 150 2025-12-17 21:25:38,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c69575bc060ce15bdf0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:41,647 - lava_callback - INFO - Received callback for node 69407cd4575bc060ce15bebc INFO: 172.21.0.1:52126 - "POST /node/69407cd4575bc060ce15bebc HTTP/1.0" 202 Accepted 2025-12-17 21:25:42,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6b575bc060ce15bdf6 2025-12-17 21:25:42,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5a575bc060ce15bdb9 2025-12-17 21:25:42,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5a575bc060ce15bdb9 HTTP/1.1" 502 150 2025-12-17 21:25:42,764 - lava_callback - ERROR - Error processing callback for node 69407c5a575bc060ce15bdb9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c5a575bc060ce15bdb9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c5a575bc060ce15bdb9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c5a575bc060ce15bdb9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:42,766 - lava_callback - INFO - Starting async job submit for node 69407c6d575bc060ce15bdf9 2025-12-17 21:25:42,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:42,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6d575bc060ce15bdf9 HTTP/1.1" 502 150 2025-12-17 21:25:42,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6d575bc060ce15bdf9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:42,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6d575bc060ce15bdf9 2025-12-17 21:25:42,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6d575bc060ce15bdf9 HTTP/1.1" 502 150 2025-12-17 21:25:42,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6d575bc060ce15bdf9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:43,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5b575bc060ce15bdbc 2025-12-17 21:25:43,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5b575bc060ce15bdbc HTTP/1.1" 502 150 2025-12-17 21:25:43,601 - lava_callback - ERROR - Error processing callback for node 69407c5b575bc060ce15bdbc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c5b575bc060ce15bdbc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c5b575bc060ce15bdbc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c5b575bc060ce15bdbc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:43,604 - lava_callback - INFO - Starting async job submit for node 69407c6e575bc060ce15bdfc 2025-12-17 21:25:43,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:43,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6e575bc060ce15bdfc HTTP/1.1" 502 150 2025-12-17 21:25:43,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6e575bc060ce15bdfc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:43,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6e575bc060ce15bdfc 2025-12-17 21:25:43,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6e575bc060ce15bdfc HTTP/1.1" 502 150 2025-12-17 21:25:43,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6e575bc060ce15bdfc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:43,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c66575bc060ce15bdea 2025-12-17 21:25:43,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c66575bc060ce15bdea HTTP/1.1" 502 150 2025-12-17 21:25:43,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c66575bc060ce15bdea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:43,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c68575bc060ce15bded 2025-12-17 21:25:43,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c68575bc060ce15bded HTTP/1.1" 502 150 2025-12-17 21:25:43,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c68575bc060ce15bded'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:44,164 - lava_callback - INFO - Received callback for node 694097e9575bc060ce161955 INFO: 172.21.0.1:52134 - "POST /node/694097e9575bc060ce161955 HTTP/1.0" 202 Accepted 2025-12-17 21:25:44,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6d575bc060ce15bdf9 2025-12-17 21:25:44,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6d575bc060ce15bdf9 HTTP/1.1" 502 150 2025-12-17 21:25:44,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6d575bc060ce15bdf9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:45,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6e575bc060ce15bdfc 2025-12-17 21:25:45,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6e575bc060ce15bdfc HTTP/1.1" 502 150 2025-12-17 21:25:45,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6e575bc060ce15bdfc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:46,186 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5d575bc060ce15bdc0 2025-12-17 21:25:46,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5d575bc060ce15bdc0 HTTP/1.1" 502 150 2025-12-17 21:25:46,188 - lava_callback - ERROR - Error processing callback for node 69407c5d575bc060ce15bdc0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c5d575bc060ce15bdc0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c5d575bc060ce15bdc0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c5d575bc060ce15bdc0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:46,189 - lava_callback - INFO - Starting async job submit for node 69407c70575bc060ce15bdff 2025-12-17 21:25:46,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:46,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c70575bc060ce15bdff HTTP/1.1" 502 150 2025-12-17 21:25:46,196 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c70575bc060ce15bdff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:46,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c70575bc060ce15bdff 2025-12-17 21:25:46,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c70575bc060ce15bdff HTTP/1.1" 502 150 2025-12-17 21:25:46,197 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c70575bc060ce15bdff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:46,353 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5e575bc060ce15bdc7 2025-12-17 21:25:46,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5e575bc060ce15bdc7 HTTP/1.1" 502 150 2025-12-17 21:25:46,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c5e575bc060ce15bdc7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52146 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:25:46,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c69575bc060ce15bdf0 2025-12-17 21:25:46,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c69575bc060ce15bdf0 HTTP/1.1" 502 150 2025-12-17 21:25:46,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c69575bc060ce15bdf0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:48,197 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c70575bc060ce15bdff 2025-12-17 21:25:48,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c70575bc060ce15bdff HTTP/1.1" 502 150 2025-12-17 21:25:48,199 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c70575bc060ce15bdff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:48,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5f575bc060ce15bdcb 2025-12-17 21:25:48,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5f575bc060ce15bdcb HTTP/1.1" 502 150 2025-12-17 21:25:48,750 - lava_callback - ERROR - Error processing callback for node 69407c5f575bc060ce15bdcb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c5f575bc060ce15bdcb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c5f575bc060ce15bdcb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c5f575bc060ce15bdcb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:48,753 - lava_callback - INFO - Starting async job submit for node 69407c71575bc060ce15be05 2025-12-17 21:25:48,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:48,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c71575bc060ce15be05 HTTP/1.1" 502 150 2025-12-17 21:25:48,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c71575bc060ce15be05'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:48,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c71575bc060ce15be05 2025-12-17 21:25:48,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c71575bc060ce15be05 HTTP/1.1" 502 150 2025-12-17 21:25:48,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c71575bc060ce15be05'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:48,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6d575bc060ce15bdf9 2025-12-17 21:25:48,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6d575bc060ce15bdf9 HTTP/1.1" 502 150 2025-12-17 21:25:48,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6d575bc060ce15bdf9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:48,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c61575bc060ce15bdd0 2025-12-17 21:25:48,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c61575bc060ce15bdd0 HTTP/1.1" 502 150 2025-12-17 21:25:48,808 - lava_callback - ERROR - Error processing callback for node 69407c61575bc060ce15bdd0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c61575bc060ce15bdd0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c61575bc060ce15bdd0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c61575bc060ce15bdd0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:48,809 - lava_callback - INFO - Starting async job submit for node 69407c73575bc060ce15be0c 2025-12-17 21:25:48,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:48,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c73575bc060ce15be0c HTTP/1.1" 502 150 2025-12-17 21:25:48,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c73575bc060ce15be0c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:48,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c73575bc060ce15be0c 2025-12-17 21:25:48,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c73575bc060ce15be0c HTTP/1.1" 502 150 2025-12-17 21:25:48,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c73575bc060ce15be0c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:49,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6e575bc060ce15bdfc 2025-12-17 21:25:49,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6e575bc060ce15bdfc HTTP/1.1" 502 150 2025-12-17 21:25:49,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6e575bc060ce15bdfc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:49,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c62575bc060ce15bdd5 2025-12-17 21:25:49,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c62575bc060ce15bdd5 HTTP/1.1" 502 150 2025-12-17 21:25:49,691 - lava_callback - ERROR - Error processing callback for node 69407c62575bc060ce15bdd5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c62575bc060ce15bdd5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c62575bc060ce15bdd5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c62575bc060ce15bdd5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:49,693 - lava_callback - INFO - Starting async job submit for node 69407c74575bc060ce15be13 2025-12-17 21:25:49,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:49,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb0575bc060ce15c33c 2025-12-17 21:25:49,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb0575bc060ce15c33c HTTP/1.1" 502 150 2025-12-17 21:25:49,698 - lava_callback - ERROR - Error processing callback for node 69407eb0575bc060ce15c33c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eb0575bc060ce15c33c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eb0575bc060ce15c33c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eb0575bc060ce15c33c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:49,699 - lava_callback - INFO - Starting async job submit for node 69407c75575bc060ce15be18 2025-12-17 21:25:49,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:49,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c74575bc060ce15be13 HTTP/1.1" 502 150 2025-12-17 21:25:49,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c74575bc060ce15be13'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:49,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c74575bc060ce15be13 2025-12-17 21:25:49,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c74575bc060ce15be13 HTTP/1.1" 502 150 2025-12-17 21:25:49,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c74575bc060ce15be13'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:49,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c75575bc060ce15be18 HTTP/1.1" 502 150 2025-12-17 21:25:49,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c75575bc060ce15be18'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:49,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c75575bc060ce15be18 2025-12-17 21:25:49,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c75575bc060ce15be18 HTTP/1.1" 502 150 2025-12-17 21:25:49,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c75575bc060ce15be18'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:50,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c71575bc060ce15be05 2025-12-17 21:25:50,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c71575bc060ce15be05 HTTP/1.1" 502 150 2025-12-17 21:25:50,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c71575bc060ce15be05'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:50,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c73575bc060ce15be0c 2025-12-17 21:25:50,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c73575bc060ce15be0c HTTP/1.1" 502 150 2025-12-17 21:25:50,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c73575bc060ce15be0c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:51,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c74575bc060ce15be13 2025-12-17 21:25:51,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c74575bc060ce15be13 HTTP/1.1" 502 150 2025-12-17 21:25:51,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c74575bc060ce15be13'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:51,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c75575bc060ce15be18 2025-12-17 21:25:51,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c75575bc060ce15be18 HTTP/1.1" 502 150 2025-12-17 21:25:51,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c75575bc060ce15be18'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:52,199 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c70575bc060ce15bdff 2025-12-17 21:25:52,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c70575bc060ce15bdff HTTP/1.1" 502 150 2025-12-17 21:25:52,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c70575bc060ce15bdff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:52,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e36575bc060ce15c288 2025-12-17 21:25:52,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e36575bc060ce15c288 HTTP/1.1" 502 150 2025-12-17 21:25:52,580 - lava_callback - ERROR - Error processing callback for node 69407e36575bc060ce15c288: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e36575bc060ce15c288 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e36575bc060ce15c288 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e36575bc060ce15c288 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:52,582 - lava_callback - INFO - Starting async job submit for node 69407c77575bc060ce15be1f 2025-12-17 21:25:52,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:52,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c77575bc060ce15be1f HTTP/1.1" 502 150 2025-12-17 21:25:52,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c77575bc060ce15be1f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:52,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c77575bc060ce15be1f 2025-12-17 21:25:52,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c77575bc060ce15be1f HTTP/1.1" 502 150 2025-12-17 21:25:52,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c77575bc060ce15be1f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:52,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6a575bc060ce15bdf3 HTTP/1.1" 502 150 2025-12-17 21:25:52,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6a575bc060ce15bdf3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:52,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c64575bc060ce15bde0 2025-12-17 21:25:52,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c64575bc060ce15bde0 HTTP/1.1" 502 150 2025-12-17 21:25:52,970 - lava_callback - ERROR - Error processing callback for node 69407c64575bc060ce15bde0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c64575bc060ce15bde0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c64575bc060ce15bde0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c64575bc060ce15bde0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:52,972 - lava_callback - INFO - Starting async job submit for node 69407d48575bc060ce15c039 2025-12-17 21:25:52,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:52,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d48575bc060ce15c039 HTTP/1.1" 502 150 2025-12-17 21:25:52,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d48575bc060ce15c039'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:52,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d48575bc060ce15c039 2025-12-17 21:25:52,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d48575bc060ce15c039 HTTP/1.1" 502 150 2025-12-17 21:25:52,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d48575bc060ce15c039'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:53,617 - lava_callback - INFO - Received callback for node 69408cf0575bc060ce15f6ef INFO: 172.21.0.1:58708 - "POST /node/69408cf0575bc060ce15f6ef HTTP/1.0" 202 Accepted 2025-12-17 21:25:54,352 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c65575bc060ce15bde7 2025-12-17 21:25:54,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c65575bc060ce15bde7 HTTP/1.1" 502 150 2025-12-17 21:25:54,354 - lava_callback - ERROR - Error processing callback for node 69407c65575bc060ce15bde7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c65575bc060ce15bde7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c65575bc060ce15bde7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c65575bc060ce15bde7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:54,357 - lava_callback - INFO - Starting async job submit for node 69408c97575bc060ce15f627 2025-12-17 21:25:54,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:54,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c97575bc060ce15f627 HTTP/1.1" 502 150 2025-12-17 21:25:54,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c97575bc060ce15f627'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:54,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c97575bc060ce15f627 2025-12-17 21:25:54,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c97575bc060ce15f627 HTTP/1.1" 502 150 2025-12-17 21:25:54,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c97575bc060ce15f627'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:54,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c77575bc060ce15be1f 2025-12-17 21:25:54,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c77575bc060ce15be1f HTTP/1.1" 502 150 2025-12-17 21:25:54,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c77575bc060ce15be1f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:54,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c71575bc060ce15be05 2025-12-17 21:25:54,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c71575bc060ce15be05 HTTP/1.1" 502 150 2025-12-17 21:25:54,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c71575bc060ce15be05'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:54,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c73575bc060ce15be0c 2025-12-17 21:25:54,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c73575bc060ce15be0c HTTP/1.1" 502 150 2025-12-17 21:25:54,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c73575bc060ce15be0c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:54,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d48575bc060ce15c039 2025-12-17 21:25:54,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d48575bc060ce15c039 HTTP/1.1" 502 150 2025-12-17 21:25:54,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d48575bc060ce15c039'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:55,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c74575bc060ce15be13 2025-12-17 21:25:55,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c74575bc060ce15be13 HTTP/1.1" 502 150 2025-12-17 21:25:55,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c74575bc060ce15be13'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:55,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c75575bc060ce15be18 2025-12-17 21:25:55,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c75575bc060ce15be18 HTTP/1.1" 502 150 2025-12-17 21:25:55,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c75575bc060ce15be18'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:56,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c97575bc060ce15f627 2025-12-17 21:25:56,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c97575bc060ce15f627 HTTP/1.1" 502 150 2025-12-17 21:25:56,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c97575bc060ce15f627'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58718 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:25:56,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6a575bc060ce15bdf3 2025-12-17 21:25:56,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6a575bc060ce15bdf3 HTTP/1.1" 502 150 2025-12-17 21:25:56,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6a575bc060ce15bdf3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:56,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6d575bc060ce15bdf9 2025-12-17 21:25:56,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6d575bc060ce15bdf9 HTTP/1.1" 502 150 2025-12-17 21:25:56,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6d575bc060ce15bdf9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:57,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6e575bc060ce15bdfc 2025-12-17 21:25:57,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6e575bc060ce15bdfc HTTP/1.1" 502 150 2025-12-17 21:25:57,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6e575bc060ce15bdfc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:58,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6b575bc060ce15bdf6 HTTP/1.1" 502 150 2025-12-17 21:25:58,064 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6b575bc060ce15bdf6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:58,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c77575bc060ce15be1f 2025-12-17 21:25:58,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c77575bc060ce15be1f HTTP/1.1" 502 150 2025-12-17 21:25:58,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c77575bc060ce15be1f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:58,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d48575bc060ce15c039 2025-12-17 21:25:58,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d48575bc060ce15c039 HTTP/1.1" 502 150 2025-12-17 21:25:58,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d48575bc060ce15c039'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:59,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c66575bc060ce15bdea 2025-12-17 21:25:59,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c66575bc060ce15bdea HTTP/1.1" 502 150 2025-12-17 21:25:59,774 - lava_callback - ERROR - Error processing callback for node 69407c66575bc060ce15bdea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c66575bc060ce15bdea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c66575bc060ce15bdea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c66575bc060ce15bdea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:59,776 - lava_callback - INFO - Starting async job submit for node 69408d41575bc060ce15f7f4 2025-12-17 21:25:59,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c68575bc060ce15bded 2025-12-17 21:25:59,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:59,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c68575bc060ce15bded HTTP/1.1" 502 150 2025-12-17 21:25:59,798 - lava_callback - ERROR - Error processing callback for node 69407c68575bc060ce15bded: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c68575bc060ce15bded (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c68575bc060ce15bded (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c68575bc060ce15bded (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:25:59,800 - lava_callback - INFO - Starting async job submit for node 69407c7c575bc060ce15be23 2025-12-17 21:25:59,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:25:59,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d41575bc060ce15f7f4 HTTP/1.1" 502 150 2025-12-17 21:25:59,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d41575bc060ce15f7f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:59,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d41575bc060ce15f7f4 2025-12-17 21:25:59,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d41575bc060ce15f7f4 HTTP/1.1" 502 150 2025-12-17 21:25:59,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d41575bc060ce15f7f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:59,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7c575bc060ce15be23 HTTP/1.1" 502 150 2025-12-17 21:25:59,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c7c575bc060ce15be23'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:25:59,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c7c575bc060ce15be23 2025-12-17 21:25:59,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7c575bc060ce15be23 HTTP/1.1" 502 150 2025-12-17 21:25:59,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c7c575bc060ce15be23'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:00,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c70575bc060ce15bdff 2025-12-17 21:26:00,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c70575bc060ce15bdff HTTP/1.1" 502 150 2025-12-17 21:26:00,201 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c70575bc060ce15bdff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:00,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c97575bc060ce15f627 2025-12-17 21:26:00,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c97575bc060ce15f627 HTTP/1.1" 502 150 2025-12-17 21:26:00,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c97575bc060ce15f627'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37794 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:26:01,684 - lava_callback - INFO - Received callback for node 69407cd6575bc060ce15bec3 INFO: 172.21.0.1:37798 - "POST /node/69407cd6575bc060ce15bec3 HTTP/1.0" 202 Accepted 2025-12-17 21:26:01,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d41575bc060ce15f7f4 2025-12-17 21:26:01,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d41575bc060ce15f7f4 HTTP/1.1" 502 150 2025-12-17 21:26:01,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d41575bc060ce15f7f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:01,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c7c575bc060ce15be23 2025-12-17 21:26:01,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7c575bc060ce15be23 HTTP/1.1" 502 150 2025-12-17 21:26:01,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c7c575bc060ce15be23'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:02,336 - lava_callback - INFO - Received callback for node 69407cd7575bc060ce15bec6 INFO: 172.21.0.1:37814 - "POST /node/69407cd7575bc060ce15bec6 HTTP/1.0" 202 Accepted 2025-12-17 21:26:02,354 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c5e575bc060ce15bdc7 2025-12-17 21:26:02,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c5e575bc060ce15bdc7 HTTP/1.1" 502 150 2025-12-17 21:26:02,355 - lava_callback - ERROR - Error processing callback for node 69407c5e575bc060ce15bdc7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c5e575bc060ce15bdc7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c5e575bc060ce15bdc7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c5e575bc060ce15bdc7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:02,357 - lava_callback - INFO - Starting async job submit for node 69407c7d575bc060ce15be26 2025-12-17 21:26:02,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:02,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7d575bc060ce15be26 HTTP/1.1" 502 150 2025-12-17 21:26:02,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c7d575bc060ce15be26'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:02,363 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c7d575bc060ce15be26 2025-12-17 21:26:02,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7d575bc060ce15be26 HTTP/1.1" 502 150 2025-12-17 21:26:02,364 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c7d575bc060ce15be26'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:02,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c69575bc060ce15bdf0 2025-12-17 21:26:02,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c69575bc060ce15bdf0 HTTP/1.1" 502 150 2025-12-17 21:26:02,607 - lava_callback - ERROR - Error processing callback for node 69407c69575bc060ce15bdf0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c69575bc060ce15bdf0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c69575bc060ce15bdf0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c69575bc060ce15bdf0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:02,609 - lava_callback - INFO - Starting async job submit for node 69407c7f575bc060ce15be29 2025-12-17 21:26:02,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:02,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7f575bc060ce15be29 HTTP/1.1" 502 150 2025-12-17 21:26:02,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c7f575bc060ce15be29'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:02,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c7f575bc060ce15be29 2025-12-17 21:26:02,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7f575bc060ce15be29 HTTP/1.1" 502 150 2025-12-17 21:26:02,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c7f575bc060ce15be29'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:02,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c71575bc060ce15be05 2025-12-17 21:26:02,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c71575bc060ce15be05 HTTP/1.1" 502 150 2025-12-17 21:26:02,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c71575bc060ce15be05'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:02,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c73575bc060ce15be0c 2025-12-17 21:26:02,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c73575bc060ce15be0c HTTP/1.1" 502 150 2025-12-17 21:26:02,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c73575bc060ce15be0c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:03,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c74575bc060ce15be13 2025-12-17 21:26:03,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c75575bc060ce15be18 2025-12-17 21:26:03,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c75575bc060ce15be18 HTTP/1.1" 502 150 2025-12-17 21:26:03,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c75575bc060ce15be18'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:04,364 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c7d575bc060ce15be26 2025-12-17 21:26:04,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7d575bc060ce15be26 HTTP/1.1" 502 150 2025-12-17 21:26:04,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c7d575bc060ce15be26'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:04,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c7f575bc060ce15be29 2025-12-17 21:26:04,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7f575bc060ce15be29 HTTP/1.1" 502 150 2025-12-17 21:26:04,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c7f575bc060ce15be29'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:04,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6a575bc060ce15bdf3 2025-12-17 21:26:04,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6a575bc060ce15bdf3 HTTP/1.1" 502 150 2025-12-17 21:26:04,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6a575bc060ce15bdf3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:05,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d41575bc060ce15f7f4 2025-12-17 21:26:05,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d41575bc060ce15f7f4 HTTP/1.1" 502 150 2025-12-17 21:26:05,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d41575bc060ce15f7f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:05,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c7c575bc060ce15be23 2025-12-17 21:26:05,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7c575bc060ce15be23 HTTP/1.1" 502 150 2025-12-17 21:26:05,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c7c575bc060ce15be23'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:06,065 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6b575bc060ce15bdf6 2025-12-17 21:26:06,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6b575bc060ce15bdf6 HTTP/1.1" 502 150 2025-12-17 21:26:06,066 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c6b575bc060ce15bdf6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37816 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:26:06,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c77575bc060ce15be1f 2025-12-17 21:26:06,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c77575bc060ce15be1f HTTP/1.1" 502 150 2025-12-17 21:26:06,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c77575bc060ce15be1f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:06,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d48575bc060ce15c039 2025-12-17 21:26:06,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d48575bc060ce15c039 HTTP/1.1" 502 150 2025-12-17 21:26:06,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d48575bc060ce15c039'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:07,667 - lava_callback - INFO - Received callback for node 69407cdd575bc060ce15becb INFO: 172.21.0.1:37820 - "POST /node/69407cdd575bc060ce15becb HTTP/1.0" 202 Accepted 2025-12-17 21:26:08,343 - lava_callback - INFO - Received callback for node 69407cde575bc060ce15bed2 INFO: 172.21.0.1:37836 - "POST /node/69407cde575bc060ce15bed2 HTTP/1.0" 202 Accepted 2025-12-17 21:26:08,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c7d575bc060ce15be26 2025-12-17 21:26:08,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7d575bc060ce15be26 HTTP/1.1" 502 150 2025-12-17 21:26:08,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c7d575bc060ce15be26'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:08,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c97575bc060ce15f627 2025-12-17 21:26:08,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c97575bc060ce15f627 HTTP/1.1" 502 150 2025-12-17 21:26:08,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c97575bc060ce15f627'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:08,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c7f575bc060ce15be29 2025-12-17 21:26:08,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7f575bc060ce15be29 HTTP/1.1" 502 150 2025-12-17 21:26:08,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c7f575bc060ce15be29'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:09,032 - lava_callback - INFO - Received callback for node 69407ce0575bc060ce15bee5 INFO: 172.21.0.1:37840 - "POST /node/69407ce0575bc060ce15bee5 HTTP/1.0" 202 Accepted 2025-12-17 21:26:09,729 - lava_callback - INFO - Received callback for node 69407ce1575bc060ce15bef8 INFO: 172.21.0.1:50996 - "POST /node/69407ce1575bc060ce15bef8 HTTP/1.0" 202 Accepted 2025-12-17 21:26:11,610 - lava_callback - INFO - Received callback for node 69407d4b575bc060ce15c045 INFO: 172.21.0.1:51004 - "POST /node/69407d4b575bc060ce15c045 HTTP/1.0" 202 Accepted 2025-12-17 21:26:12,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6d575bc060ce15bdf9 2025-12-17 21:26:12,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6d575bc060ce15bdf9 HTTP/1.1" 502 150 2025-12-17 21:26:12,801 - lava_callback - ERROR - Error processing callback for node 69407c6d575bc060ce15bdf9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c6d575bc060ce15bdf9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c6d575bc060ce15bdf9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c6d575bc060ce15bdf9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:12,803 - lava_callback - INFO - Starting async job submit for node 69407eb1575bc060ce15c34e 2025-12-17 21:26:12,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:12,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb1575bc060ce15c34e HTTP/1.1" 502 150 2025-12-17 21:26:12,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb1575bc060ce15c34e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:12,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb1575bc060ce15c34e 2025-12-17 21:26:12,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb1575bc060ce15c34e HTTP/1.1" 502 150 2025-12-17 21:26:12,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb1575bc060ce15c34e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:13,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6e575bc060ce15bdfc 2025-12-17 21:26:13,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6e575bc060ce15bdfc HTTP/1.1" 502 150 2025-12-17 21:26:13,628 - lava_callback - ERROR - Error processing callback for node 69407c6e575bc060ce15bdfc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c6e575bc060ce15bdfc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c6e575bc060ce15bdfc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c6e575bc060ce15bdfc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:13,632 - lava_callback - INFO - Starting async job submit for node 69407eb1575bc060ce15c34b 2025-12-17 21:26:13,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:13,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb1575bc060ce15c34b HTTP/1.1" 502 150 2025-12-17 21:26:13,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb1575bc060ce15c34b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:13,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb1575bc060ce15c34b 2025-12-17 21:26:13,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb1575bc060ce15c34b HTTP/1.1" 502 150 2025-12-17 21:26:13,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb1575bc060ce15c34b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:13,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d41575bc060ce15f7f4 2025-12-17 21:26:13,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c7c575bc060ce15be23 2025-12-17 21:26:13,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d41575bc060ce15f7f4 HTTP/1.1" 502 150 2025-12-17 21:26:13,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d41575bc060ce15f7f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:13,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7c575bc060ce15be23 HTTP/1.1" 502 150 2025-12-17 21:26:13,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c7c575bc060ce15be23'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:14,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb1575bc060ce15c34e 2025-12-17 21:26:14,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb1575bc060ce15c34e HTTP/1.1" 502 150 2025-12-17 21:26:14,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb1575bc060ce15c34e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51014 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:26:15,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb1575bc060ce15c34b 2025-12-17 21:26:15,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb1575bc060ce15c34b HTTP/1.1" 502 150 2025-12-17 21:26:15,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb1575bc060ce15c34b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:16,202 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c70575bc060ce15bdff 2025-12-17 21:26:16,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c70575bc060ce15bdff HTTP/1.1" 502 150 2025-12-17 21:26:16,203 - lava_callback - ERROR - Error processing callback for node 69407c70575bc060ce15bdff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c70575bc060ce15bdff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c70575bc060ce15bdff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c70575bc060ce15bdff (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:16,205 - lava_callback - INFO - Starting async job submit for node 69407e36575bc060ce15c291 2025-12-17 21:26:16,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:16,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e36575bc060ce15c291 HTTP/1.1" 502 150 2025-12-17 21:26:16,244 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e36575bc060ce15c291'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:16,244 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e36575bc060ce15c291 2025-12-17 21:26:16,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e36575bc060ce15c291 HTTP/1.1" 502 150 2025-12-17 21:26:16,245 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e36575bc060ce15c291'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:16,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c7d575bc060ce15be26 2025-12-17 21:26:16,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7d575bc060ce15be26 HTTP/1.1" 502 150 2025-12-17 21:26:16,371 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c7d575bc060ce15be26'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51030 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:26:16,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c7f575bc060ce15be29 2025-12-17 21:26:16,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7f575bc060ce15be29 HTTP/1.1" 502 150 2025-12-17 21:26:16,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c7f575bc060ce15be29'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:18,245 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e36575bc060ce15c291 2025-12-17 21:26:18,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e36575bc060ce15c291 HTTP/1.1" 502 150 2025-12-17 21:26:18,246 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e36575bc060ce15c291'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:18,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c71575bc060ce15be05 2025-12-17 21:26:18,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c71575bc060ce15be05 HTTP/1.1" 502 150 2025-12-17 21:26:18,789 - lava_callback - ERROR - Error processing callback for node 69407c71575bc060ce15be05: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c71575bc060ce15be05 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c71575bc060ce15be05 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c71575bc060ce15be05 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:18,791 - lava_callback - INFO - Starting async job submit for node 69407c89575bc060ce15be57 2025-12-17 21:26:18,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:18,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c89575bc060ce15be57 HTTP/1.1" 502 150 2025-12-17 21:26:18,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c89575bc060ce15be57'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:18,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c89575bc060ce15be57 2025-12-17 21:26:18,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c89575bc060ce15be57 HTTP/1.1" 502 150 2025-12-17 21:26:18,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c89575bc060ce15be57'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:18,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c73575bc060ce15be0c 2025-12-17 21:26:18,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c73575bc060ce15be0c HTTP/1.1" 502 150 2025-12-17 21:26:18,823 - lava_callback - ERROR - Error processing callback for node 69407c73575bc060ce15be0c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c73575bc060ce15be0c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c73575bc060ce15be0c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c73575bc060ce15be0c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:18,824 - lava_callback - INFO - Starting async job submit for node 69407c8b575bc060ce15be5e 2025-12-17 21:26:18,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:18,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb1575bc060ce15c34e 2025-12-17 21:26:18,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb1575bc060ce15c34e HTTP/1.1" 502 150 2025-12-17 21:26:18,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb1575bc060ce15c34e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:18,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c8b575bc060ce15be5e HTTP/1.1" 502 150 2025-12-17 21:26:18,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c8b575bc060ce15be5e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:18,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c8b575bc060ce15be5e 2025-12-17 21:26:18,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c8b575bc060ce15be5e HTTP/1.1" 502 150 2025-12-17 21:26:18,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c8b575bc060ce15be5e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:19,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c74575bc060ce15be13 HTTP/1.1" 502 150 2025-12-17 21:26:19,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c74575bc060ce15be13'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:19,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb1575bc060ce15c34b 2025-12-17 21:26:19,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb1575bc060ce15c34b HTTP/1.1" 502 150 2025-12-17 21:26:19,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb1575bc060ce15c34b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:19,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c75575bc060ce15be18 2025-12-17 21:26:19,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c75575bc060ce15be18 HTTP/1.1" 502 150 2025-12-17 21:26:19,733 - lava_callback - ERROR - Error processing callback for node 69407c75575bc060ce15be18: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c75575bc060ce15be18 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c75575bc060ce15be18 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c75575bc060ce15be18 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:19,735 - lava_callback - INFO - Starting async job submit for node 69407c91575bc060ce15be69 2025-12-17 21:26:19,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:19,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c91575bc060ce15be69 HTTP/1.1" 502 150 2025-12-17 21:26:19,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c91575bc060ce15be69'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:19,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c91575bc060ce15be69 2025-12-17 21:26:19,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c91575bc060ce15be69 HTTP/1.1" 502 150 2025-12-17 21:26:19,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c91575bc060ce15be69'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:20,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6a575bc060ce15bdf3 2025-12-17 21:26:20,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6a575bc060ce15bdf3 HTTP/1.1" 502 150 2025-12-17 21:26:20,693 - lava_callback - ERROR - Error processing callback for node 69407c6a575bc060ce15bdf3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c6a575bc060ce15bdf3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c6a575bc060ce15bdf3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c6a575bc060ce15bdf3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:20,695 - lava_callback - INFO - Starting async job submit for node 69407c93575bc060ce15be6c 2025-12-17 21:26:20,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:20,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c93575bc060ce15be6c HTTP/1.1" 502 150 2025-12-17 21:26:20,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c93575bc060ce15be6c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:20,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c93575bc060ce15be6c 2025-12-17 21:26:20,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c93575bc060ce15be6c HTTP/1.1" 502 150 2025-12-17 21:26:20,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c93575bc060ce15be6c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:20,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c89575bc060ce15be57 2025-12-17 21:26:20,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c89575bc060ce15be57 HTTP/1.1" 502 150 2025-12-17 21:26:20,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c89575bc060ce15be57'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:20,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c8b575bc060ce15be5e 2025-12-17 21:26:20,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c8b575bc060ce15be5e HTTP/1.1" 502 150 2025-12-17 21:26:20,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c8b575bc060ce15be5e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:21,657 - lava_callback - INFO - Received callback for node 69407ce8575bc060ce15befd INFO: 172.21.0.1:60120 - "POST /node/69407ce8575bc060ce15befd HTTP/1.0" 202 Accepted 2025-12-17 21:26:21,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c91575bc060ce15be69 2025-12-17 21:26:21,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c91575bc060ce15be69 HTTP/1.1" 502 150 2025-12-17 21:26:21,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c91575bc060ce15be69'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:22,066 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c6b575bc060ce15bdf6 2025-12-17 21:26:22,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c6b575bc060ce15bdf6 HTTP/1.1" 502 150 2025-12-17 21:26:22,067 - lava_callback - ERROR - Error processing callback for node 69407c6b575bc060ce15bdf6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c6b575bc060ce15bdf6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c6b575bc060ce15bdf6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c6b575bc060ce15bdf6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:22,069 - lava_callback - INFO - Starting async job submit for node 69407c94575bc060ce15be74 2025-12-17 21:26:22,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:22,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c94575bc060ce15be74 HTTP/1.1" 502 150 2025-12-17 21:26:22,076 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c94575bc060ce15be74'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:22,076 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c94575bc060ce15be74 2025-12-17 21:26:22,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c94575bc060ce15be74 HTTP/1.1" 502 150 2025-12-17 21:26:22,077 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c94575bc060ce15be74'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:22,246 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e36575bc060ce15c291 2025-12-17 21:26:22,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e36575bc060ce15c291 HTTP/1.1" 502 150 2025-12-17 21:26:22,248 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e36575bc060ce15c291'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:22,319 - lava_callback - INFO - Received callback for node 69407cea575bc060ce15bf06 INFO: 172.21.0.1:60126 - "POST /node/69407cea575bc060ce15bf06 HTTP/1.0" 202 Accepted 2025-12-17 21:26:22,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c77575bc060ce15be1f 2025-12-17 21:26:22,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c77575bc060ce15be1f HTTP/1.1" 502 150 2025-12-17 21:26:22,604 - lava_callback - ERROR - Error processing callback for node 69407c77575bc060ce15be1f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c77575bc060ce15be1f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c77575bc060ce15be1f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c77575bc060ce15be1f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:22,606 - lava_callback - INFO - Starting async job submit for node 69407c95575bc060ce15be77 2025-12-17 21:26:22,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:22,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c95575bc060ce15be77 HTTP/1.1" 502 150 2025-12-17 21:26:22,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c95575bc060ce15be77'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:22,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c95575bc060ce15be77 2025-12-17 21:26:22,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c95575bc060ce15be77 HTTP/1.1" 502 150 2025-12-17 21:26:22,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c95575bc060ce15be77'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:22,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c93575bc060ce15be6c 2025-12-17 21:26:22,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c93575bc060ce15be6c HTTP/1.1" 502 150 2025-12-17 21:26:22,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c93575bc060ce15be6c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:22,815 - lava_callback - INFO - Received callback for node 69407eb5575bc060ce15c352 INFO: 172.21.0.1:60128 - "POST /node/69407eb5575bc060ce15c352 HTTP/1.0" 202 Accepted 2025-12-17 21:26:22,985 - lava_callback - INFO - Received callback for node 69407ceb575bc060ce15bf0d INFO: 172.21.0.1:60138 - "POST /node/69407ceb575bc060ce15bf0d HTTP/1.0" 202 Accepted 2025-12-17 21:26:22,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d48575bc060ce15c039 2025-12-17 21:26:22,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d48575bc060ce15c039 HTTP/1.1" 502 150 2025-12-17 21:26:22,992 - lava_callback - ERROR - Error processing callback for node 69407d48575bc060ce15c039: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d48575bc060ce15c039 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d48575bc060ce15c039 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d48575bc060ce15c039 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:22,993 - lava_callback - INFO - Starting async job submit for node 69407c97575bc060ce15be7a 2025-12-17 21:26:22,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:23,627 - lava_callback - INFO - Received callback for node 69407cec575bc060ce15bf10 INFO: 172.21.0.1:60144 - "POST /node/69407cec575bc060ce15bf10 HTTP/1.0" 202 Accepted 2025-12-17 21:26:24,020 - lava_callback - INFO - Received callback for node 69407eb7575bc060ce15c35a INFO: 172.21.0.1:60158 - "POST /node/69407eb7575bc060ce15c35a HTTP/1.0" 202 Accepted 2025-12-17 21:26:24,077 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c94575bc060ce15be74 2025-12-17 21:26:24,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c94575bc060ce15be74 HTTP/1.1" 502 150 2025-12-17 21:26:24,078 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c94575bc060ce15be74'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:24,266 - lava_callback - INFO - Received callback for node 69407cee575bc060ce15bf13 INFO: 172.21.0.1:60172 - "POST /node/69407cee575bc060ce15bf13 HTTP/1.0" 202 Accepted 2025-12-17 21:26:24,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c97575bc060ce15f627 2025-12-17 21:26:24,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c97575bc060ce15f627 HTTP/1.1" 502 150 2025-12-17 21:26:24,461 - lava_callback - ERROR - Error processing callback for node 69408c97575bc060ce15f627: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c97575bc060ce15f627 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c97575bc060ce15f627 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c97575bc060ce15f627 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:24,463 - lava_callback - INFO - Starting async job submit for node 69407c98575bc060ce15be81 2025-12-17 21:26:24,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:24,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c98575bc060ce15be81 HTTP/1.1" 502 150 2025-12-17 21:26:24,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c98575bc060ce15be81'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:24,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c98575bc060ce15be81 2025-12-17 21:26:24,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c98575bc060ce15be81 HTTP/1.1" 502 150 2025-12-17 21:26:24,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c98575bc060ce15be81'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:24,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c95575bc060ce15be77 2025-12-17 21:26:24,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c95575bc060ce15be77 HTTP/1.1" 502 150 2025-12-17 21:26:24,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c95575bc060ce15be77'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:24,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c89575bc060ce15be57 2025-12-17 21:26:24,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c89575bc060ce15be57 HTTP/1.1" 502 150 2025-12-17 21:26:24,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c89575bc060ce15be57'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:24,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c8b575bc060ce15be5e 2025-12-17 21:26:24,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c8b575bc060ce15be5e HTTP/1.1" 502 150 2025-12-17 21:26:24,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c8b575bc060ce15be5e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:24,902 - lava_callback - INFO - Received callback for node 69407cef575bc060ce15bf16 INFO: 172.21.0.1:60184 - "POST /node/69407cef575bc060ce15bf16 HTTP/1.0" 202 Accepted 2025-12-17 21:26:25,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c91575bc060ce15be69 2025-12-17 21:26:25,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c91575bc060ce15be69 HTTP/1.1" 502 150 2025-12-17 21:26:25,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c91575bc060ce15be69'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:26,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c98575bc060ce15be81 2025-12-17 21:26:26,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c98575bc060ce15be81 HTTP/1.1" 502 150 2025-12-17 21:26:26,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c98575bc060ce15be81'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60200 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:26:26,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c93575bc060ce15be6c 2025-12-17 21:26:26,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c93575bc060ce15be6c HTTP/1.1" 502 150 2025-12-17 21:26:26,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c93575bc060ce15be6c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:26,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb1575bc060ce15c34e 2025-12-17 21:26:26,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb1575bc060ce15c34e HTTP/1.1" 502 150 2025-12-17 21:26:26,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb1575bc060ce15c34e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:27,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb1575bc060ce15c34b 2025-12-17 21:26:27,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb1575bc060ce15c34b HTTP/1.1" 502 150 2025-12-17 21:26:27,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb1575bc060ce15c34b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:27,653 - lava_callback - INFO - Received callback for node 69407cf1575bc060ce15bf1b INFO: 172.21.0.1:60206 - "POST /node/69407cf1575bc060ce15bf1b HTTP/1.0" 202 Accepted 2025-12-17 21:26:28,079 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c94575bc060ce15be74 2025-12-17 21:26:28,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c94575bc060ce15be74 HTTP/1.1" 502 150 2025-12-17 21:26:28,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c94575bc060ce15be74'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:28,349 - lava_callback - INFO - Received callback for node 69407cf2575bc060ce15bf22 INFO: 172.21.0.1:60212 - "POST /node/69407cf2575bc060ce15bf22 HTTP/1.0" 202 Accepted 2025-12-17 21:26:28,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c95575bc060ce15be77 2025-12-17 21:26:28,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c95575bc060ce15be77 HTTP/1.1" 502 150 2025-12-17 21:26:28,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c95575bc060ce15be77'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:29,017 - lava_callback - INFO - Received callback for node 69407cf3575bc060ce15bf27 INFO: 172.21.0.1:60214 - "POST /node/69407cf3575bc060ce15bf27 HTTP/1.0" 202 Accepted 2025-12-17 21:26:29,677 - lava_callback - INFO - Received callback for node 69407cf5575bc060ce15bf2a INFO: 172.21.0.1:60226 - "POST /node/69407cf5575bc060ce15bf2a HTTP/1.0" 202 Accepted 2025-12-17 21:26:29,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d41575bc060ce15f7f4 2025-12-17 21:26:29,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c7c575bc060ce15be23 2025-12-17 21:26:29,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d41575bc060ce15f7f4 HTTP/1.1" 502 150 2025-12-17 21:26:29,812 - lava_callback - ERROR - Error processing callback for node 69408d41575bc060ce15f7f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d41575bc060ce15f7f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d41575bc060ce15f7f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d41575bc060ce15f7f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:29,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7c575bc060ce15be23 HTTP/1.1" 502 150 2025-12-17 21:26:29,815 - lava_callback - ERROR - Error processing callback for node 69407c7c575bc060ce15be23: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c7c575bc060ce15be23 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c7c575bc060ce15be23 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c7c575bc060ce15be23 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:29,814 - lava_callback - INFO - Starting async job submit for node 69407c99575bc060ce15be86 2025-12-17 21:26:29,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:29,818 - lava_callback - INFO - Starting async job submit for node 69407d49575bc060ce15c03c 2025-12-17 21:26:29,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:29,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c99575bc060ce15be86 HTTP/1.1" 502 150 2025-12-17 21:26:29,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c99575bc060ce15be86'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:29,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c99575bc060ce15be86 2025-12-17 21:26:29,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c99575bc060ce15be86 HTTP/1.1" 502 150 2025-12-17 21:26:29,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c99575bc060ce15be86'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:29,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d49575bc060ce15c03c HTTP/1.1" 502 150 2025-12-17 21:26:29,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d49575bc060ce15c03c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:29,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d49575bc060ce15c03c 2025-12-17 21:26:29,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d49575bc060ce15c03c HTTP/1.1" 502 150 2025-12-17 21:26:29,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d49575bc060ce15c03c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:30,248 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e36575bc060ce15c291 2025-12-17 21:26:30,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e36575bc060ce15c291 HTTP/1.1" 502 150 2025-12-17 21:26:30,249 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e36575bc060ce15c291'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:30,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c98575bc060ce15be81 2025-12-17 21:26:30,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c98575bc060ce15be81 HTTP/1.1" 502 150 2025-12-17 21:26:30,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c98575bc060ce15be81'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:30,631 - lava_callback - INFO - Received callback for node 69407eba575bc060ce15c35f INFO: 172.21.0.1:56484 - "POST /node/69407eba575bc060ce15c35f HTTP/1.0" 202 Accepted 2025-12-17 21:26:31,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c99575bc060ce15be86 2025-12-17 21:26:31,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c99575bc060ce15be86 HTTP/1.1" 502 150 2025-12-17 21:26:31,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c99575bc060ce15be86'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:31,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d49575bc060ce15c03c 2025-12-17 21:26:31,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d49575bc060ce15c03c HTTP/1.1" 502 150 2025-12-17 21:26:31,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d49575bc060ce15c03c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:32,371 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c7d575bc060ce15be26 2025-12-17 21:26:32,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7d575bc060ce15be26 HTTP/1.1" 502 150 2025-12-17 21:26:32,372 - lava_callback - ERROR - Error processing callback for node 69407c7d575bc060ce15be26: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c7d575bc060ce15be26 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c7d575bc060ce15be26 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c7d575bc060ce15be26 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:32,374 - lava_callback - INFO - Starting async job submit for node 69407d49575bc060ce15c03f 2025-12-17 21:26:32,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:32,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d49575bc060ce15c03f HTTP/1.1" 502 150 2025-12-17 21:26:32,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d49575bc060ce15c03f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:32,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d49575bc060ce15c03f 2025-12-17 21:26:32,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d49575bc060ce15c03f HTTP/1.1" 502 150 2025-12-17 21:26:32,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d49575bc060ce15c03f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:32,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c7f575bc060ce15be29 2025-12-17 21:26:32,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c7f575bc060ce15be29 HTTP/1.1" 502 150 2025-12-17 21:26:32,622 - lava_callback - ERROR - Error processing callback for node 69407c7f575bc060ce15be29: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c7f575bc060ce15be29 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c7f575bc060ce15be29 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c7f575bc060ce15be29 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:32,624 - lava_callback - INFO - Starting async job submit for node 69407cd4575bc060ce15bebc 2025-12-17 21:26:32,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:32,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd4575bc060ce15bebc HTTP/1.1" 502 150 2025-12-17 21:26:32,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cd4575bc060ce15bebc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:32,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cd4575bc060ce15bebc 2025-12-17 21:26:32,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd4575bc060ce15bebc HTTP/1.1" 502 150 2025-12-17 21:26:32,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cd4575bc060ce15bebc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:32,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c89575bc060ce15be57 2025-12-17 21:26:32,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c89575bc060ce15be57 HTTP/1.1" 502 150 2025-12-17 21:26:32,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c89575bc060ce15be57'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:32,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c8b575bc060ce15be5e 2025-12-17 21:26:32,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c8b575bc060ce15be5e HTTP/1.1" 502 150 2025-12-17 21:26:32,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c8b575bc060ce15be5e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:33,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c91575bc060ce15be69 2025-12-17 21:26:33,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c91575bc060ce15be69 HTTP/1.1" 502 150 2025-12-17 21:26:33,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c91575bc060ce15be69'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:34,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d49575bc060ce15c03f 2025-12-17 21:26:34,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d49575bc060ce15c03f HTTP/1.1" 502 150 2025-12-17 21:26:34,385 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d49575bc060ce15c03f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:34,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cd4575bc060ce15bebc 2025-12-17 21:26:34,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd4575bc060ce15bebc HTTP/1.1" 502 150 2025-12-17 21:26:34,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cd4575bc060ce15bebc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:34,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c93575bc060ce15be6c 2025-12-17 21:26:34,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c93575bc060ce15be6c HTTP/1.1" 502 150 2025-12-17 21:26:34,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c93575bc060ce15be6c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:35,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c74575bc060ce15be13 2025-12-17 21:26:35,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c74575bc060ce15be13 HTTP/1.1" 502 150 2025-12-17 21:26:35,058 - lava_callback - ERROR - Error processing callback for node 69407c74575bc060ce15be13: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c74575bc060ce15be13 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c74575bc060ce15be13 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c74575bc060ce15be13 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:35,060 - lava_callback - INFO - Starting async job submit for node 694097e9575bc060ce161955 2025-12-17 21:26:35,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:35,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e9575bc060ce161955 HTTP/1.1" 502 150 2025-12-17 21:26:35,085 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e9575bc060ce161955'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:35,086 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e9575bc060ce161955 2025-12-17 21:26:35,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e9575bc060ce161955 HTTP/1.1" 502 150 2025-12-17 21:26:35,086 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e9575bc060ce161955'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:35,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c99575bc060ce15be86 2025-12-17 21:26:35,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d49575bc060ce15c03c 2025-12-17 21:26:35,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c99575bc060ce15be86 HTTP/1.1" 502 150 2025-12-17 21:26:35,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c99575bc060ce15be86'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:35,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d49575bc060ce15c03c HTTP/1.1" 502 150 2025-12-17 21:26:35,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d49575bc060ce15c03c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:36,080 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c94575bc060ce15be74 2025-12-17 21:26:36,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c94575bc060ce15be74 HTTP/1.1" 502 150 2025-12-17 21:26:36,081 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c94575bc060ce15be74'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56486 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:26:36,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c95575bc060ce15be77 2025-12-17 21:26:36,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c95575bc060ce15be77 HTTP/1.1" 502 150 2025-12-17 21:26:36,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c95575bc060ce15be77'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:37,086 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e9575bc060ce161955 2025-12-17 21:26:37,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e9575bc060ce161955 HTTP/1.1" 502 150 2025-12-17 21:26:37,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e9575bc060ce161955'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:38,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c97575bc060ce15be7a HTTP/1.1" 502 150 2025-12-17 21:26:38,256 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c97575bc060ce15be7a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:38,256 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c97575bc060ce15be7a 2025-12-17 21:26:38,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c97575bc060ce15be7a HTTP/1.1" 502 150 2025-12-17 21:26:38,257 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c97575bc060ce15be7a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:38,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d49575bc060ce15c03f 2025-12-17 21:26:38,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d49575bc060ce15c03f HTTP/1.1" 502 150 2025-12-17 21:26:38,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d49575bc060ce15c03f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:38,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c98575bc060ce15be81 2025-12-17 21:26:38,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c98575bc060ce15be81 HTTP/1.1" 502 150 2025-12-17 21:26:38,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c98575bc060ce15be81'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:38,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cd4575bc060ce15bebc 2025-12-17 21:26:38,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd4575bc060ce15bebc HTTP/1.1" 502 150 2025-12-17 21:26:38,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cd4575bc060ce15bebc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:40,258 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c97575bc060ce15be7a 2025-12-17 21:26:40,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c97575bc060ce15be7a HTTP/1.1" 502 150 2025-12-17 21:26:40,259 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c97575bc060ce15be7a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:41,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e9575bc060ce161955 2025-12-17 21:26:41,687 - lava_callback - INFO - Received callback for node 69407cfc575bc060ce15bf2f INFO: 172.21.0.1:58232 - "POST /node/69407cfc575bc060ce15bf2f HTTP/1.0" 202 Accepted 2025-12-17 21:26:42,327 - lava_callback - INFO - Received callback for node 69407cfe575bc060ce15bf38 INFO: 172.21.0.1:58240 - "POST /node/69407cfe575bc060ce15bf38 HTTP/1.0" 202 Accepted 2025-12-17 21:26:42,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb1575bc060ce15c34e 2025-12-17 21:26:42,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb1575bc060ce15c34e HTTP/1.1" 502 150 2025-12-17 21:26:42,856 - lava_callback - ERROR - Error processing callback for node 69407eb1575bc060ce15c34e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eb1575bc060ce15c34e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eb1575bc060ce15c34e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eb1575bc060ce15c34e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:42,857 - lava_callback - INFO - Starting async job submit for node 69408cf0575bc060ce15f6ef 2025-12-17 21:26:42,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:42,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf0575bc060ce15f6ef HTTP/1.1" 502 150 2025-12-17 21:26:42,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf0575bc060ce15f6ef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:42,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf0575bc060ce15f6ef 2025-12-17 21:26:42,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf0575bc060ce15f6ef HTTP/1.1" 502 150 2025-12-17 21:26:42,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf0575bc060ce15f6ef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:42,976 - lava_callback - INFO - Received callback for node 69407cff575bc060ce15bf40 INFO: 172.21.0.1:58254 - "POST /node/69407cff575bc060ce15bf40 HTTP/1.0" 202 Accepted 2025-12-17 21:26:43,626 - lava_callback - INFO - Received callback for node 69407d01575bc060ce15bf67 INFO: 172.21.0.1:58256 - "POST /node/69407d01575bc060ce15bf67 HTTP/1.0" 202 Accepted 2025-12-17 21:26:43,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb1575bc060ce15c34b 2025-12-17 21:26:43,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb1575bc060ce15c34b HTTP/1.1" 502 150 2025-12-17 21:26:43,651 - lava_callback - ERROR - Error processing callback for node 69407eb1575bc060ce15c34b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eb1575bc060ce15c34b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eb1575bc060ce15c34b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eb1575bc060ce15c34b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:43,653 - lava_callback - INFO - Starting async job submit for node 69407cd6575bc060ce15bec3 2025-12-17 21:26:43,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:43,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd6575bc060ce15bec3 HTTP/1.1" 502 150 2025-12-17 21:26:43,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cd6575bc060ce15bec3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:43,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cd6575bc060ce15bec3 2025-12-17 21:26:43,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd6575bc060ce15bec3 HTTP/1.1" 502 150 2025-12-17 21:26:43,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cd6575bc060ce15bec3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:43,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c99575bc060ce15be86 2025-12-17 21:26:43,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d49575bc060ce15c03c 2025-12-17 21:26:43,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c99575bc060ce15be86 HTTP/1.1" 502 150 2025-12-17 21:26:43,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c99575bc060ce15be86'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:43,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d49575bc060ce15c03c HTTP/1.1" 502 150 2025-12-17 21:26:43,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d49575bc060ce15c03c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:44,259 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c97575bc060ce15be7a 2025-12-17 21:26:44,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c97575bc060ce15be7a HTTP/1.1" 502 150 2025-12-17 21:26:44,260 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c97575bc060ce15be7a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:44,286 - lava_callback - INFO - Received callback for node 69407d02575bc060ce15bf6a INFO: 172.21.0.1:58258 - "POST /node/69407d02575bc060ce15bf6a HTTP/1.0" 202 Accepted 2025-12-17 21:26:44,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf0575bc060ce15f6ef 2025-12-17 21:26:44,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf0575bc060ce15f6ef HTTP/1.1" 502 150 2025-12-17 21:26:44,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf0575bc060ce15f6ef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:44,938 - lava_callback - INFO - Received callback for node 69407d04575bc060ce15bf6d INFO: 172.21.0.1:58264 - "POST /node/69407d04575bc060ce15bf6d HTTP/1.0" 202 Accepted 2025-12-17 21:26:45,601 - lava_callback - INFO - Received callback for node 69407d05575bc060ce15bf71 INFO: 172.21.0.1:58272 - "POST /node/69407d05575bc060ce15bf71 HTTP/1.0" 202 Accepted 2025-12-17 21:26:45,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cd6575bc060ce15bec3 2025-12-17 21:26:45,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd6575bc060ce15bec3 HTTP/1.1" 502 150 2025-12-17 21:26:45,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cd6575bc060ce15bec3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:46,245 - lava_callback - INFO - Received callback for node 69407d06575bc060ce15bf74 INFO: 172.21.0.1:58282 - "POST /node/69407d06575bc060ce15bf74 HTTP/1.0" 202 Accepted 2025-12-17 21:26:46,251 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e36575bc060ce15c291 2025-12-17 21:26:46,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e36575bc060ce15c291 HTTP/1.1" 502 150 2025-12-17 21:26:46,253 - lava_callback - ERROR - Error processing callback for node 69407e36575bc060ce15c291: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e36575bc060ce15c291 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e36575bc060ce15c291 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e36575bc060ce15c291 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:46,256 - lava_callback - INFO - Starting async job submit for node 69407cd7575bc060ce15bec6 2025-12-17 21:26:46,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:46,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd7575bc060ce15bec6 HTTP/1.1" 502 150 2025-12-17 21:26:46,281 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cd7575bc060ce15bec6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:46,282 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cd7575bc060ce15bec6 2025-12-17 21:26:46,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd7575bc060ce15bec6 HTTP/1.1" 502 150 2025-12-17 21:26:46,282 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cd7575bc060ce15bec6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:46,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d49575bc060ce15c03f 2025-12-17 21:26:46,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d49575bc060ce15c03f HTTP/1.1" 502 150 2025-12-17 21:26:46,388 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d49575bc060ce15c03f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58292 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:26:46,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cd4575bc060ce15bebc 2025-12-17 21:26:46,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd4575bc060ce15bebc HTTP/1.1" 502 150 2025-12-17 21:26:46,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cd4575bc060ce15bebc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:46,895 - lava_callback - INFO - Received callback for node 69407d09575bc060ce15bf7a INFO: 172.21.0.1:58306 - "POST /node/69407d09575bc060ce15bf7a HTTP/1.0" 202 Accepted 2025-12-17 21:26:48,283 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cd7575bc060ce15bec6 2025-12-17 21:26:48,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd7575bc060ce15bec6 HTTP/1.1" 502 150 2025-12-17 21:26:48,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cd7575bc060ce15bec6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:48,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c89575bc060ce15be57 2025-12-17 21:26:48,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c89575bc060ce15be57 HTTP/1.1" 502 150 2025-12-17 21:26:48,803 - lava_callback - ERROR - Error processing callback for node 69407c89575bc060ce15be57: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c89575bc060ce15be57 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c89575bc060ce15be57 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c89575bc060ce15be57 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:48,805 - lava_callback - INFO - Starting async job submit for node 69407cdd575bc060ce15becb 2025-12-17 21:26:48,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:48,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cdd575bc060ce15becb HTTP/1.1" 502 150 2025-12-17 21:26:48,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cdd575bc060ce15becb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:48,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cdd575bc060ce15becb 2025-12-17 21:26:48,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cdd575bc060ce15becb HTTP/1.1" 502 150 2025-12-17 21:26:48,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cdd575bc060ce15becb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:48,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c8b575bc060ce15be5e 2025-12-17 21:26:48,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c8b575bc060ce15be5e HTTP/1.1" 502 150 2025-12-17 21:26:48,860 - lava_callback - ERROR - Error processing callback for node 69407c8b575bc060ce15be5e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c8b575bc060ce15be5e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c8b575bc060ce15be5e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c8b575bc060ce15be5e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:48,862 - lava_callback - INFO - Starting async job submit for node 69407cde575bc060ce15bed2 2025-12-17 21:26:48,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:48,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cde575bc060ce15bed2 HTTP/1.1" 502 150 2025-12-17 21:26:48,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cde575bc060ce15bed2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:48,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cde575bc060ce15bed2 2025-12-17 21:26:48,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cde575bc060ce15bed2 HTTP/1.1" 502 150 2025-12-17 21:26:48,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cde575bc060ce15bed2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:48,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf0575bc060ce15f6ef 2025-12-17 21:26:48,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf0575bc060ce15f6ef HTTP/1.1" 502 150 2025-12-17 21:26:48,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf0575bc060ce15f6ef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:49,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cd6575bc060ce15bec3 2025-12-17 21:26:49,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd6575bc060ce15bec3 HTTP/1.1" 502 150 2025-12-17 21:26:49,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cd6575bc060ce15bec3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:49,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c91575bc060ce15be69 2025-12-17 21:26:49,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c91575bc060ce15be69 HTTP/1.1" 502 150 2025-12-17 21:26:49,749 - lava_callback - ERROR - Error processing callback for node 69407c91575bc060ce15be69: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c91575bc060ce15be69 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c91575bc060ce15be69 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c91575bc060ce15be69 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:49,751 - lava_callback - INFO - Starting async job submit for node 69407ce0575bc060ce15bee5 2025-12-17 21:26:49,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:49,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce0575bc060ce15bee5 HTTP/1.1" 502 150 2025-12-17 21:26:49,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ce0575bc060ce15bee5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:49,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ce0575bc060ce15bee5 2025-12-17 21:26:49,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce0575bc060ce15bee5 HTTP/1.1" 502 150 2025-12-17 21:26:49,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ce0575bc060ce15bee5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:50,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c93575bc060ce15be6c 2025-12-17 21:26:50,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c93575bc060ce15be6c HTTP/1.1" 502 150 2025-12-17 21:26:50,721 - lava_callback - ERROR - Error processing callback for node 69407c93575bc060ce15be6c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c93575bc060ce15be6c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c93575bc060ce15be6c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c93575bc060ce15be6c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:50,722 - lava_callback - INFO - Starting async job submit for node 69407ce1575bc060ce15bef8 2025-12-17 21:26:50,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:50,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce1575bc060ce15bef8 HTTP/1.1" 502 150 2025-12-17 21:26:50,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ce1575bc060ce15bef8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:50,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ce1575bc060ce15bef8 2025-12-17 21:26:50,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce1575bc060ce15bef8 HTTP/1.1" 502 150 2025-12-17 21:26:50,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ce1575bc060ce15bef8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:50,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cdd575bc060ce15becb 2025-12-17 21:26:50,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cdd575bc060ce15becb HTTP/1.1" 502 150 2025-12-17 21:26:50,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cdd575bc060ce15becb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:50,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cde575bc060ce15bed2 2025-12-17 21:26:50,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cde575bc060ce15bed2 HTTP/1.1" 502 150 2025-12-17 21:26:50,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cde575bc060ce15bed2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:51,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ce0575bc060ce15bee5 2025-12-17 21:26:51,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce0575bc060ce15bee5 HTTP/1.1" 502 150 2025-12-17 21:26:51,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ce0575bc060ce15bee5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:52,082 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c94575bc060ce15be74 2025-12-17 21:26:52,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c94575bc060ce15be74 HTTP/1.1" 502 150 2025-12-17 21:26:52,083 - lava_callback - ERROR - Error processing callback for node 69407c94575bc060ce15be74: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c94575bc060ce15be74 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c94575bc060ce15be74 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c94575bc060ce15be74 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:52,085 - lava_callback - INFO - Starting async job submit for node 69407d4b575bc060ce15c045 2025-12-17 21:26:52,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:52,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4b575bc060ce15c045 HTTP/1.1" 502 150 2025-12-17 21:26:52,116 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4b575bc060ce15c045'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:52,116 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4b575bc060ce15c045 2025-12-17 21:26:52,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4b575bc060ce15c045 HTTP/1.1" 502 150 2025-12-17 21:26:52,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4b575bc060ce15c045'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:52,138 - lava_callback - INFO - Received callback for node 69407c0a575bc060ce15bc8b INFO: 172.21.0.1:52698 - "POST /node/69407c0a575bc060ce15bc8b HTTP/1.0" 202 Accepted 2025-12-17 21:26:52,261 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c97575bc060ce15be7a 2025-12-17 21:26:52,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c97575bc060ce15be7a HTTP/1.1" 502 150 2025-12-17 21:26:52,263 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c97575bc060ce15be7a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:52,284 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cd7575bc060ce15bec6 2025-12-17 21:26:52,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd7575bc060ce15bec6 HTTP/1.1" 502 150 2025-12-17 21:26:52,285 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cd7575bc060ce15bec6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:52,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c95575bc060ce15be77 2025-12-17 21:26:52,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c95575bc060ce15be77 HTTP/1.1" 502 150 2025-12-17 21:26:52,623 - lava_callback - ERROR - Error processing callback for node 69407c95575bc060ce15be77: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c95575bc060ce15be77 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c95575bc060ce15be77 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c95575bc060ce15be77 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:52,625 - lava_callback - INFO - Starting async job submit for node 69407ce8575bc060ce15befd 2025-12-17 21:26:52,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:52,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce8575bc060ce15befd HTTP/1.1" 502 150 2025-12-17 21:26:52,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ce8575bc060ce15befd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:52,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ce8575bc060ce15befd 2025-12-17 21:26:52,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce8575bc060ce15befd HTTP/1.1" 502 150 2025-12-17 21:26:52,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ce8575bc060ce15befd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:52,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ce1575bc060ce15bef8 2025-12-17 21:26:52,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce1575bc060ce15bef8 HTTP/1.1" 502 150 2025-12-17 21:26:52,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ce1575bc060ce15bef8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:53,067 - lava_callback - INFO - Received callback for node 69407d4b575bc060ce15c048 INFO: 172.21.0.1:52706 - "POST /node/69407d4b575bc060ce15c048 HTTP/1.0" 202 Accepted 2025-12-17 21:26:54,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4b575bc060ce15c045 2025-12-17 21:26:54,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4b575bc060ce15c045 HTTP/1.1" 502 150 2025-12-17 21:26:54,118 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4b575bc060ce15c045'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:54,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c98575bc060ce15be81 2025-12-17 21:26:54,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c98575bc060ce15be81 HTTP/1.1" 502 150 2025-12-17 21:26:54,483 - lava_callback - ERROR - Error processing callback for node 69407c98575bc060ce15be81: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c98575bc060ce15be81 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c98575bc060ce15be81 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c98575bc060ce15be81 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:54,485 - lava_callback - INFO - Starting async job submit for node 69407cea575bc060ce15bf06 2025-12-17 21:26:54,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:54,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cea575bc060ce15bf06 HTTP/1.1" 502 150 2025-12-17 21:26:54,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cea575bc060ce15bf06'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:54,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cea575bc060ce15bf06 2025-12-17 21:26:54,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cea575bc060ce15bf06 HTTP/1.1" 502 150 2025-12-17 21:26:54,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cea575bc060ce15bf06'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:54,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ce8575bc060ce15befd 2025-12-17 21:26:54,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce8575bc060ce15befd HTTP/1.1" 502 150 2025-12-17 21:26:54,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ce8575bc060ce15befd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:54,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cdd575bc060ce15becb 2025-12-17 21:26:54,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cdd575bc060ce15becb HTTP/1.1" 502 150 2025-12-17 21:26:54,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cdd575bc060ce15becb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:54,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cde575bc060ce15bed2 2025-12-17 21:26:54,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cde575bc060ce15bed2 HTTP/1.1" 502 150 2025-12-17 21:26:54,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cde575bc060ce15bed2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:55,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ce0575bc060ce15bee5 2025-12-17 21:26:55,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce0575bc060ce15bee5 HTTP/1.1" 502 150 2025-12-17 21:26:55,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ce0575bc060ce15bee5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:56,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e9575bc060ce161955 HTTP/1.1" 502 150 2025-12-17 21:26:56,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e9575bc060ce161955'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52722 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:26:56,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cea575bc060ce15bf06 2025-12-17 21:26:56,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cea575bc060ce15bf06 HTTP/1.1" 502 150 2025-12-17 21:26:56,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cea575bc060ce15bf06'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:56,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ce1575bc060ce15bef8 2025-12-17 21:26:56,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce1575bc060ce15bef8 HTTP/1.1" 502 150 2025-12-17 21:26:56,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ce1575bc060ce15bef8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:56,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf0575bc060ce15f6ef 2025-12-17 21:26:56,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf0575bc060ce15f6ef HTTP/1.1" 502 150 2025-12-17 21:26:56,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf0575bc060ce15f6ef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:57,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cd6575bc060ce15bec3 2025-12-17 21:26:57,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd6575bc060ce15bec3 HTTP/1.1" 502 150 2025-12-17 21:26:57,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cd6575bc060ce15bec3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:58,119 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4b575bc060ce15c045 2025-12-17 21:26:58,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4b575bc060ce15c045 HTTP/1.1" 502 150 2025-12-17 21:26:58,120 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4b575bc060ce15c045'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:58,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ce8575bc060ce15befd 2025-12-17 21:26:58,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce8575bc060ce15befd HTTP/1.1" 502 150 2025-12-17 21:26:58,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ce8575bc060ce15befd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:59,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c99575bc060ce15be86 2025-12-17 21:26:59,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d49575bc060ce15c03c 2025-12-17 21:26:59,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c99575bc060ce15be86 HTTP/1.1" 502 150 2025-12-17 21:26:59,833 - lava_callback - ERROR - Error processing callback for node 69407c99575bc060ce15be86: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c99575bc060ce15be86 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c99575bc060ce15be86 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c99575bc060ce15be86 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:59,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d49575bc060ce15c03c HTTP/1.1" 502 150 2025-12-17 21:26:59,835 - lava_callback - ERROR - Error processing callback for node 69407d49575bc060ce15c03c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d49575bc060ce15c03c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d49575bc060ce15c03c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d49575bc060ce15c03c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:26:59,836 - lava_callback - INFO - Starting async job submit for node 69407eb5575bc060ce15c352 2025-12-17 21:26:59,843 - lava_callback - INFO - Starting async job submit for node 69407ceb575bc060ce15bf0d 2025-12-17 21:26:59,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:59,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:26:59,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ceb575bc060ce15bf0d HTTP/1.1" 502 150 2025-12-17 21:26:59,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ceb575bc060ce15bf0d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:59,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ceb575bc060ce15bf0d 2025-12-17 21:26:59,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ceb575bc060ce15bf0d HTTP/1.1" 502 150 2025-12-17 21:26:59,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ceb575bc060ce15bf0d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:59,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb5575bc060ce15c352 HTTP/1.1" 502 150 2025-12-17 21:26:59,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb5575bc060ce15c352'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:26:59,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb5575bc060ce15c352 2025-12-17 21:26:59,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb5575bc060ce15c352 HTTP/1.1" 502 150 2025-12-17 21:26:59,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb5575bc060ce15c352'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:00,285 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cd7575bc060ce15bec6 2025-12-17 21:27:00,304 - lava_callback - INFO - Received callback for node 69407c35575bc060ce15bd05 INFO: 172.21.0.1:34124 - "POST /node/69407c35575bc060ce15bd05 HTTP/1.0" 202 Accepted 2025-12-17 21:27:00,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cea575bc060ce15bf06 2025-12-17 21:27:00,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cea575bc060ce15bf06 HTTP/1.1" 502 150 2025-12-17 21:27:00,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cea575bc060ce15bf06'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34136 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:27:01,692 - lava_callback - INFO - Received callback for node 69407d16575bc060ce15bf95 INFO: 172.21.0.1:34138 - "POST /node/69407d16575bc060ce15bf95 HTTP/1.0" 202 Accepted 2025-12-17 21:27:01,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ceb575bc060ce15bf0d 2025-12-17 21:27:01,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb5575bc060ce15c352 2025-12-17 21:27:01,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ceb575bc060ce15bf0d HTTP/1.1" 502 150 2025-12-17 21:27:01,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ceb575bc060ce15bf0d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:01,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb5575bc060ce15c352 HTTP/1.1" 502 150 2025-12-17 21:27:01,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb5575bc060ce15c352'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:02,346 - lava_callback - INFO - Received callback for node 69407d12575bc060ce15bf82 INFO: 172.21.0.1:34142 - "POST /node/69407d12575bc060ce15bf82 HTTP/1.0" 202 Accepted 2025-12-17 21:27:02,388 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d49575bc060ce15c03f 2025-12-17 21:27:02,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d49575bc060ce15c03f HTTP/1.1" 502 150 2025-12-17 21:27:02,389 - lava_callback - ERROR - Error processing callback for node 69407d49575bc060ce15c03f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d49575bc060ce15c03f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d49575bc060ce15c03f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d49575bc060ce15c03f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:02,391 - lava_callback - INFO - Starting async job submit for node 69407cec575bc060ce15bf10 2025-12-17 21:27:02,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:02,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cec575bc060ce15bf10 HTTP/1.1" 502 150 2025-12-17 21:27:02,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cec575bc060ce15bf10'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:02,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cec575bc060ce15bf10 2025-12-17 21:27:02,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cec575bc060ce15bf10 HTTP/1.1" 502 150 2025-12-17 21:27:02,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cec575bc060ce15bf10'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:02,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cd4575bc060ce15bebc 2025-12-17 21:27:02,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd4575bc060ce15bebc HTTP/1.1" 502 150 2025-12-17 21:27:02,642 - lava_callback - ERROR - Error processing callback for node 69407cd4575bc060ce15bebc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cd4575bc060ce15bebc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cd4575bc060ce15bebc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cd4575bc060ce15bebc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:02,645 - lava_callback - INFO - Starting async job submit for node 69407eb7575bc060ce15c35a 2025-12-17 21:27:02,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:02,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb7575bc060ce15c35a HTTP/1.1" 502 150 2025-12-17 21:27:02,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb7575bc060ce15c35a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:02,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb7575bc060ce15c35a 2025-12-17 21:27:02,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb7575bc060ce15c35a HTTP/1.1" 502 150 2025-12-17 21:27:02,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb7575bc060ce15c35a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:02,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cdd575bc060ce15becb 2025-12-17 21:27:02,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cdd575bc060ce15becb HTTP/1.1" 502 150 2025-12-17 21:27:02,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cdd575bc060ce15becb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:02,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cde575bc060ce15bed2 2025-12-17 21:27:02,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cde575bc060ce15bed2 HTTP/1.1" 502 150 2025-12-17 21:27:02,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cde575bc060ce15bed2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:02,993 - lava_callback - INFO - Received callback for node 69407d14575bc060ce15bf89 INFO: 172.21.0.1:34152 - "POST /node/69407d14575bc060ce15bf89 HTTP/1.0" 202 Accepted 2025-12-17 21:27:03,632 - lava_callback - INFO - Received callback for node 69407d15575bc060ce15bf90 INFO: 172.21.0.1:34158 - "POST /node/69407d15575bc060ce15bf90 HTTP/1.0" 202 Accepted 2025-12-17 21:27:03,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ce0575bc060ce15bee5 2025-12-17 21:27:03,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce0575bc060ce15bee5 HTTP/1.1" 502 150 2025-12-17 21:27:03,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ce0575bc060ce15bee5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:04,274 - lava_callback - INFO - Received callback for node 69407d17575bc060ce15bf98 INFO: 172.21.0.1:34172 - "POST /node/69407d17575bc060ce15bf98 HTTP/1.0" 202 Accepted 2025-12-17 21:27:04,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cec575bc060ce15bf10 2025-12-17 21:27:04,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cec575bc060ce15bf10 HTTP/1.1" 502 150 2025-12-17 21:27:04,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cec575bc060ce15bf10'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:04,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e9575bc060ce161955 2025-12-17 21:27:04,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e9575bc060ce161955 HTTP/1.1" 502 150 2025-12-17 21:27:04,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e9575bc060ce161955'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:04,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb7575bc060ce15c35a 2025-12-17 21:27:04,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb7575bc060ce15c35a HTTP/1.1" 502 150 2025-12-17 21:27:04,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb7575bc060ce15c35a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:04,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ce1575bc060ce15bef8 2025-12-17 21:27:04,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce1575bc060ce15bef8 HTTP/1.1" 502 150 2025-12-17 21:27:04,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ce1575bc060ce15bef8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:04,919 - lava_callback - INFO - Received callback for node 69407d19575bc060ce15bf9b INFO: 172.21.0.1:34188 - "POST /node/69407d19575bc060ce15bf9b HTTP/1.0" 202 Accepted 2025-12-17 21:27:05,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ceb575bc060ce15bf0d 2025-12-17 21:27:05,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb5575bc060ce15c352 2025-12-17 21:27:05,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ceb575bc060ce15bf0d HTTP/1.1" 502 150 2025-12-17 21:27:05,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ceb575bc060ce15bf0d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:05,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb5575bc060ce15c352 HTTP/1.1" 502 150 2025-12-17 21:27:05,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb5575bc060ce15c352'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:06,120 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4b575bc060ce15c045 2025-12-17 21:27:06,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4b575bc060ce15c045 HTTP/1.1" 502 150 2025-12-17 21:27:06,121 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4b575bc060ce15c045'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34204 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:27:06,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ce8575bc060ce15befd 2025-12-17 21:27:06,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce8575bc060ce15befd HTTP/1.1" 502 150 2025-12-17 21:27:06,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ce8575bc060ce15befd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:08,263 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c97575bc060ce15be7a 2025-12-17 21:27:08,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c97575bc060ce15be7a HTTP/1.1" 502 150 2025-12-17 21:27:08,265 - lava_callback - ERROR - Error processing callback for node 69407c97575bc060ce15be7a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c97575bc060ce15be7a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c97575bc060ce15be7a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c97575bc060ce15be7a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:08,266 - lava_callback - INFO - Starting async job submit for node 69407cee575bc060ce15bf13 2025-12-17 21:27:08,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:08,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cee575bc060ce15bf13 HTTP/1.1" 502 150 2025-12-17 21:27:08,273 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cee575bc060ce15bf13'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:08,273 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cee575bc060ce15bf13 2025-12-17 21:27:08,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cee575bc060ce15bf13 HTTP/1.1" 502 150 2025-12-17 21:27:08,274 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cee575bc060ce15bf13'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:08,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cec575bc060ce15bf10 2025-12-17 21:27:08,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cec575bc060ce15bf10 HTTP/1.1" 502 150 2025-12-17 21:27:08,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cec575bc060ce15bf10'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:08,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cea575bc060ce15bf06 2025-12-17 21:27:08,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cea575bc060ce15bf06 HTTP/1.1" 502 150 2025-12-17 21:27:08,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cea575bc060ce15bf06'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:08,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb7575bc060ce15c35a 2025-12-17 21:27:08,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb7575bc060ce15c35a HTTP/1.1" 502 150 2025-12-17 21:27:08,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb7575bc060ce15c35a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:10,274 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cee575bc060ce15bf13 2025-12-17 21:27:10,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cee575bc060ce15bf13 HTTP/1.1" 502 150 2025-12-17 21:27:10,278 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cee575bc060ce15bf13'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:12,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf0575bc060ce15f6ef 2025-12-17 21:27:12,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf0575bc060ce15f6ef HTTP/1.1" 502 150 2025-12-17 21:27:12,933 - lava_callback - ERROR - Error processing callback for node 69408cf0575bc060ce15f6ef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf0575bc060ce15f6ef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf0575bc060ce15f6ef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf0575bc060ce15f6ef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:12,935 - lava_callback - INFO - Starting async job submit for node 69407cef575bc060ce15bf16 2025-12-17 21:27:12,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:12,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cef575bc060ce15bf16 HTTP/1.1" 502 150 2025-12-17 21:27:12,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cef575bc060ce15bf16'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:12,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cef575bc060ce15bf16 2025-12-17 21:27:12,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cef575bc060ce15bf16 HTTP/1.1" 502 150 2025-12-17 21:27:12,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cef575bc060ce15bf16'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:13,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cd6575bc060ce15bec3 2025-12-17 21:27:13,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd6575bc060ce15bec3 HTTP/1.1" 502 150 2025-12-17 21:27:13,666 - lava_callback - ERROR - Error processing callback for node 69407cd6575bc060ce15bec3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cd6575bc060ce15bec3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cd6575bc060ce15bec3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cd6575bc060ce15bec3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:13,668 - lava_callback - INFO - Starting async job submit for node 69407cf1575bc060ce15bf1b 2025-12-17 21:27:13,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:13,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf1575bc060ce15bf1b HTTP/1.1" 502 150 2025-12-17 21:27:13,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf1575bc060ce15bf1b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:13,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf1575bc060ce15bf1b 2025-12-17 21:27:13,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf1575bc060ce15bf1b HTTP/1.1" 502 150 2025-12-17 21:27:13,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf1575bc060ce15bf1b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:13,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ceb575bc060ce15bf0d 2025-12-17 21:27:13,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb5575bc060ce15c352 2025-12-17 21:27:13,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb5575bc060ce15c352 HTTP/1.1" 502 150 2025-12-17 21:27:13,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb5575bc060ce15c352'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:13,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ceb575bc060ce15bf0d HTTP/1.1" 502 150 2025-12-17 21:27:13,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ceb575bc060ce15bf0d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:14,279 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cee575bc060ce15bf13 2025-12-17 21:27:14,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cee575bc060ce15bf13 HTTP/1.1" 502 150 2025-12-17 21:27:14,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cee575bc060ce15bf13'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:14,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cef575bc060ce15bf16 2025-12-17 21:27:14,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cef575bc060ce15bf16 HTTP/1.1" 502 150 2025-12-17 21:27:14,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cef575bc060ce15bf16'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58386 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:27:15,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd7575bc060ce15bec6 HTTP/1.1" 502 150 2025-12-17 21:27:15,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cd7575bc060ce15bec6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:15,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf1575bc060ce15bf1b 2025-12-17 21:27:15,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf1575bc060ce15bf1b HTTP/1.1" 502 150 2025-12-17 21:27:15,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf1575bc060ce15bf1b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:16,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cec575bc060ce15bf10 2025-12-17 21:27:16,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cec575bc060ce15bf10 HTTP/1.1" 502 150 2025-12-17 21:27:16,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cec575bc060ce15bf10'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58396 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:27:16,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb7575bc060ce15c35a 2025-12-17 21:27:16,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb7575bc060ce15c35a HTTP/1.1" 502 150 2025-12-17 21:27:16,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eb7575bc060ce15c35a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:18,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cdd575bc060ce15becb 2025-12-17 21:27:18,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cdd575bc060ce15becb HTTP/1.1" 502 150 2025-12-17 21:27:18,818 - lava_callback - ERROR - Error processing callback for node 69407cdd575bc060ce15becb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cdd575bc060ce15becb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cdd575bc060ce15becb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cdd575bc060ce15becb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:18,820 - lava_callback - INFO - Starting async job submit for node 69407cf2575bc060ce15bf22 2025-12-17 21:27:18,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:18,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf2575bc060ce15bf22 HTTP/1.1" 502 150 2025-12-17 21:27:18,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf2575bc060ce15bf22'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:18,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf2575bc060ce15bf22 2025-12-17 21:27:18,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf2575bc060ce15bf22 HTTP/1.1" 502 150 2025-12-17 21:27:18,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf2575bc060ce15bf22'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:18,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cde575bc060ce15bed2 2025-12-17 21:27:18,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cde575bc060ce15bed2 HTTP/1.1" 502 150 2025-12-17 21:27:18,874 - lava_callback - ERROR - Error processing callback for node 69407cde575bc060ce15bed2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cde575bc060ce15bed2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cde575bc060ce15bed2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cde575bc060ce15bed2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:18,875 - lava_callback - INFO - Starting async job submit for node 69407cf3575bc060ce15bf27 2025-12-17 21:27:18,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:18,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf3575bc060ce15bf27 HTTP/1.1" 502 150 2025-12-17 21:27:18,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf3575bc060ce15bf27'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:18,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf3575bc060ce15bf27 2025-12-17 21:27:18,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf3575bc060ce15bf27 HTTP/1.1" 502 150 2025-12-17 21:27:18,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf3575bc060ce15bf27'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:18,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cef575bc060ce15bf16 2025-12-17 21:27:18,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cef575bc060ce15bf16 HTTP/1.1" 502 150 2025-12-17 21:27:18,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cef575bc060ce15bf16'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:19,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf1575bc060ce15bf1b 2025-12-17 21:27:19,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf1575bc060ce15bf1b HTTP/1.1" 502 150 2025-12-17 21:27:19,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf1575bc060ce15bf1b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:19,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ce0575bc060ce15bee5 2025-12-17 21:27:19,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce0575bc060ce15bee5 HTTP/1.1" 502 150 2025-12-17 21:27:19,764 - lava_callback - ERROR - Error processing callback for node 69407ce0575bc060ce15bee5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ce0575bc060ce15bee5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ce0575bc060ce15bee5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ce0575bc060ce15bee5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:19,766 - lava_callback - INFO - Starting async job submit for node 69407cf5575bc060ce15bf2a 2025-12-17 21:27:19,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:19,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf5575bc060ce15bf2a HTTP/1.1" 502 150 2025-12-17 21:27:19,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf5575bc060ce15bf2a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:19,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf5575bc060ce15bf2a 2025-12-17 21:27:19,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf5575bc060ce15bf2a HTTP/1.1" 502 150 2025-12-17 21:27:19,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf5575bc060ce15bf2a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:20,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e9575bc060ce161955 2025-12-17 21:27:20,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e9575bc060ce161955 HTTP/1.1" 502 150 2025-12-17 21:27:20,435 - lava_callback - ERROR - Error processing callback for node 694097e9575bc060ce161955: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097e9575bc060ce161955 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097e9575bc060ce161955 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097e9575bc060ce161955 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:20,437 - lava_callback - INFO - Starting async job submit for node 69407eba575bc060ce15c35f 2025-12-17 21:27:20,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:20,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eba575bc060ce15c35f HTTP/1.1" 502 150 2025-12-17 21:27:20,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eba575bc060ce15c35f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:20,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eba575bc060ce15c35f 2025-12-17 21:27:20,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eba575bc060ce15c35f HTTP/1.1" 502 150 2025-12-17 21:27:20,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eba575bc060ce15c35f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:20,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ce1575bc060ce15bef8 2025-12-17 21:27:20,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce1575bc060ce15bef8 HTTP/1.1" 502 150 2025-12-17 21:27:20,736 - lava_callback - ERROR - Error processing callback for node 69407ce1575bc060ce15bef8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ce1575bc060ce15bef8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ce1575bc060ce15bef8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ce1575bc060ce15bef8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:20,737 - lava_callback - INFO - Starting async job submit for node 69407cfc575bc060ce15bf2f 2025-12-17 21:27:20,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:20,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cfc575bc060ce15bf2f HTTP/1.1" 502 150 2025-12-17 21:27:20,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cfc575bc060ce15bf2f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:20,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cfc575bc060ce15bf2f 2025-12-17 21:27:20,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cfc575bc060ce15bf2f HTTP/1.1" 502 150 2025-12-17 21:27:20,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cfc575bc060ce15bf2f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:20,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf2575bc060ce15bf22 2025-12-17 21:27:20,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf2575bc060ce15bf22 HTTP/1.1" 502 150 2025-12-17 21:27:20,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf2575bc060ce15bf22'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:20,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf3575bc060ce15bf27 2025-12-17 21:27:20,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf3575bc060ce15bf27 HTTP/1.1" 502 150 2025-12-17 21:27:20,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf3575bc060ce15bf27'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:21,722 - lava_callback - INFO - Received callback for node 69407d24575bc060ce15bfa0 INFO: 172.21.0.1:39460 - "POST /node/69407d24575bc060ce15bfa0 HTTP/1.0" 202 Accepted 2025-12-17 21:27:21,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf5575bc060ce15bf2a 2025-12-17 21:27:21,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf5575bc060ce15bf2a HTTP/1.1" 502 150 2025-12-17 21:27:21,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf5575bc060ce15bf2a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:22,122 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4b575bc060ce15c045 2025-12-17 21:27:22,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4b575bc060ce15c045 HTTP/1.1" 502 150 2025-12-17 21:27:22,123 - lava_callback - ERROR - Error processing callback for node 69407d4b575bc060ce15c045: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4b575bc060ce15c045 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4b575bc060ce15c045 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4b575bc060ce15c045 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:22,125 - lava_callback - INFO - Starting async job submit for node 69407cfe575bc060ce15bf38 2025-12-17 21:27:22,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:22,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cfe575bc060ce15bf38 HTTP/1.1" 502 150 2025-12-17 21:27:22,132 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cfe575bc060ce15bf38'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:22,132 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cfe575bc060ce15bf38 2025-12-17 21:27:22,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cfe575bc060ce15bf38 HTTP/1.1" 502 150 2025-12-17 21:27:22,133 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cfe575bc060ce15bf38'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:22,280 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cee575bc060ce15bf13 2025-12-17 21:27:22,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cee575bc060ce15bf13 HTTP/1.1" 502 150 2025-12-17 21:27:22,282 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cee575bc060ce15bf13'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:22,366 - lava_callback - INFO - Received callback for node 69407d26575bc060ce15bfa9 INFO: 172.21.0.1:39472 - "POST /node/69407d26575bc060ce15bfa9 HTTP/1.0" 202 Accepted 2025-12-17 21:27:22,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eba575bc060ce15c35f 2025-12-17 21:27:22,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eba575bc060ce15c35f HTTP/1.1" 502 150 2025-12-17 21:27:22,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eba575bc060ce15c35f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:22,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ce8575bc060ce15befd 2025-12-17 21:27:22,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ce8575bc060ce15befd HTTP/1.1" 502 150 2025-12-17 21:27:22,638 - lava_callback - ERROR - Error processing callback for node 69407ce8575bc060ce15befd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ce8575bc060ce15befd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ce8575bc060ce15befd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ce8575bc060ce15befd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:22,640 - lava_callback - INFO - Starting async job submit for node 69407cff575bc060ce15bf40 2025-12-17 21:27:22,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:22,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cff575bc060ce15bf40 HTTP/1.1" 502 150 2025-12-17 21:27:22,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cff575bc060ce15bf40'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:22,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cff575bc060ce15bf40 2025-12-17 21:27:22,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cff575bc060ce15bf40 HTTP/1.1" 502 150 2025-12-17 21:27:22,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cff575bc060ce15bf40'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:22,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cfc575bc060ce15bf2f 2025-12-17 21:27:22,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cfc575bc060ce15bf2f HTTP/1.1" 502 150 2025-12-17 21:27:22,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cfc575bc060ce15bf2f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:23,017 - lava_callback - INFO - Received callback for node 69407d27575bc060ce15bfb0 INFO: 172.21.0.1:39488 - "POST /node/69407d27575bc060ce15bfb0 HTTP/1.0" 202 Accepted 2025-12-17 21:27:23,671 - lava_callback - INFO - Received callback for node 69407d29575bc060ce15bfb7 INFO: 172.21.0.1:39502 - "POST /node/69407d29575bc060ce15bfb7 HTTP/1.0" 202 Accepted 2025-12-17 21:27:24,133 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cfe575bc060ce15bf38 2025-12-17 21:27:24,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cfe575bc060ce15bf38 HTTP/1.1" 502 150 2025-12-17 21:27:24,134 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cfe575bc060ce15bf38'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:24,319 - lava_callback - INFO - Received callback for node 69407d2a575bc060ce15bfbe INFO: 172.21.0.1:39508 - "POST /node/69407d2a575bc060ce15bfbe HTTP/1.0" 202 Accepted 2025-12-17 21:27:24,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cea575bc060ce15bf06 2025-12-17 21:27:24,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cea575bc060ce15bf06 HTTP/1.1" 502 150 2025-12-17 21:27:24,518 - lava_callback - ERROR - Error processing callback for node 69407cea575bc060ce15bf06: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cea575bc060ce15bf06 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cea575bc060ce15bf06 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cea575bc060ce15bf06 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:24,520 - lava_callback - INFO - Starting async job submit for node 69407d01575bc060ce15bf67 2025-12-17 21:27:24,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:24,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d01575bc060ce15bf67 HTTP/1.1" 502 150 2025-12-17 21:27:24,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d01575bc060ce15bf67'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:24,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d01575bc060ce15bf67 2025-12-17 21:27:24,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d01575bc060ce15bf67 HTTP/1.1" 502 150 2025-12-17 21:27:24,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d01575bc060ce15bf67'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:24,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cff575bc060ce15bf40 2025-12-17 21:27:24,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cff575bc060ce15bf40 HTTP/1.1" 502 150 2025-12-17 21:27:24,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cff575bc060ce15bf40'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:24,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf2575bc060ce15bf22 2025-12-17 21:27:24,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf2575bc060ce15bf22 HTTP/1.1" 502 150 2025-12-17 21:27:24,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf2575bc060ce15bf22'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:24,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf3575bc060ce15bf27 2025-12-17 21:27:24,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf3575bc060ce15bf27 HTTP/1.1" 502 150 2025-12-17 21:27:24,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf3575bc060ce15bf27'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:24,989 - lava_callback - INFO - Received callback for node 69407d2b575bc060ce15bfc1 INFO: 172.21.0.1:39520 - "POST /node/69407d2b575bc060ce15bfc1 HTTP/1.0" 202 Accepted 2025-12-17 21:27:25,635 - lava_callback - INFO - Received callback for node 69407d2c575bc060ce15bfc4 INFO: 172.21.0.1:39524 - "POST /node/69407d2c575bc060ce15bfc4 HTTP/1.0" 202 Accepted 2025-12-17 21:27:25,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf5575bc060ce15bf2a 2025-12-17 21:27:25,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf5575bc060ce15bf2a HTTP/1.1" 502 150 2025-12-17 21:27:25,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf5575bc060ce15bf2a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:26,272 - lava_callback - INFO - Received callback for node 69407d2e575bc060ce15bfc7 INFO: 172.21.0.1:39528 - "POST /node/69407d2e575bc060ce15bfc7 HTTP/1.0" 202 Accepted 2025-12-17 21:27:26,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eba575bc060ce15c35f 2025-12-17 21:27:26,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eba575bc060ce15c35f HTTP/1.1" 502 150 2025-12-17 21:27:26,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eba575bc060ce15c35f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39540 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:27:26,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d01575bc060ce15bf67 2025-12-17 21:27:26,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d01575bc060ce15bf67 HTTP/1.1" 502 150 2025-12-17 21:27:26,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d01575bc060ce15bf67'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:26,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cfc575bc060ce15bf2f 2025-12-17 21:27:26,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cfc575bc060ce15bf2f HTTP/1.1" 502 150 2025-12-17 21:27:26,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cfc575bc060ce15bf2f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:26,910 - lava_callback - INFO - Received callback for node 69407d2f575bc060ce15bfca INFO: 172.21.0.1:39548 - "POST /node/69407d2f575bc060ce15bfca HTTP/1.0" 202 Accepted 2025-12-17 21:27:26,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cef575bc060ce15bf16 2025-12-17 21:27:26,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cef575bc060ce15bf16 HTTP/1.1" 502 150 2025-12-17 21:27:26,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cef575bc060ce15bf16'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:27,571 - lava_callback - INFO - Received callback for node 69407d30575bc060ce15bfcd INFO: 172.21.0.1:39552 - "POST /node/69407d30575bc060ce15bfcd HTTP/1.0" 202 Accepted 2025-12-17 21:27:27,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf1575bc060ce15bf1b 2025-12-17 21:27:27,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf1575bc060ce15bf1b HTTP/1.1" 502 150 2025-12-17 21:27:27,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf1575bc060ce15bf1b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:28,134 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cfe575bc060ce15bf38 2025-12-17 21:27:28,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cfe575bc060ce15bf38 HTTP/1.1" 502 150 2025-12-17 21:27:28,135 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cfe575bc060ce15bf38'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:28,210 - lava_callback - INFO - Received callback for node 69407d38575bc060ce15bfd2 INFO: 172.21.0.1:39558 - "POST /node/69407d38575bc060ce15bfd2 HTTP/1.0" 202 Accepted 2025-12-17 21:27:28,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cff575bc060ce15bf40 2025-12-17 21:27:28,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cff575bc060ce15bf40 HTTP/1.1" 502 150 2025-12-17 21:27:28,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cff575bc060ce15bf40'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:29,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb5575bc060ce15c352 2025-12-17 21:27:29,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ceb575bc060ce15bf0d 2025-12-17 21:27:29,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ceb575bc060ce15bf0d HTTP/1.1" 502 150 2025-12-17 21:27:29,870 - lava_callback - ERROR - Error processing callback for node 69407ceb575bc060ce15bf0d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ceb575bc060ce15bf0d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ceb575bc060ce15bf0d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ceb575bc060ce15bf0d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:29,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb5575bc060ce15c352 HTTP/1.1" 502 150 2025-12-17 21:27:29,872 - lava_callback - ERROR - Error processing callback for node 69407eb5575bc060ce15c352: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eb5575bc060ce15c352 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eb5575bc060ce15c352 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eb5575bc060ce15c352 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:29,872 - lava_callback - INFO - Starting async job submit for node 69407d02575bc060ce15bf6a 2025-12-17 21:27:29,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:29,875 - lava_callback - INFO - Starting async job submit for node 69407d04575bc060ce15bf6d 2025-12-17 21:27:29,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:29,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d02575bc060ce15bf6a HTTP/1.1" 502 150 2025-12-17 21:27:29,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d02575bc060ce15bf6a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:29,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d02575bc060ce15bf6a 2025-12-17 21:27:29,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d04575bc060ce15bf6d HTTP/1.1" 502 150 2025-12-17 21:27:29,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d04575bc060ce15bf6d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:29,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d04575bc060ce15bf6d 2025-12-17 21:27:29,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d02575bc060ce15bf6a HTTP/1.1" 502 150 2025-12-17 21:27:29,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d02575bc060ce15bf6a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:29,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d04575bc060ce15bf6d HTTP/1.1" 502 150 2025-12-17 21:27:29,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d04575bc060ce15bf6d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:30,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d01575bc060ce15bf67 2025-12-17 21:27:30,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d01575bc060ce15bf67 HTTP/1.1" 502 150 2025-12-17 21:27:30,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d01575bc060ce15bf67'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:31,459 - lava_callback - INFO - Received callback for node 69407d4c575bc060ce15c04b INFO: 172.21.0.1:34972 - "POST /node/69407d4c575bc060ce15c04b HTTP/1.0" 202 Accepted 2025-12-17 21:27:31,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cd7575bc060ce15bec6 2025-12-17 21:27:31,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cd7575bc060ce15bec6 HTTP/1.1" 502 150 2025-12-17 21:27:31,634 - lava_callback - ERROR - Error processing callback for node 69407cd7575bc060ce15bec6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cd7575bc060ce15bec6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cd7575bc060ce15bec6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cd7575bc060ce15bec6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:31,635 - lava_callback - INFO - Starting async job submit for node 69407d05575bc060ce15bf71 2025-12-17 21:27:31,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:31,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d05575bc060ce15bf71 HTTP/1.1" 502 150 2025-12-17 21:27:31,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d05575bc060ce15bf71'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:31,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d05575bc060ce15bf71 2025-12-17 21:27:31,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d05575bc060ce15bf71 HTTP/1.1" 502 150 2025-12-17 21:27:31,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d05575bc060ce15bf71'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:31,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d02575bc060ce15bf6a 2025-12-17 21:27:31,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d04575bc060ce15bf6d 2025-12-17 21:27:31,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d02575bc060ce15bf6a HTTP/1.1" 502 150 2025-12-17 21:27:31,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d02575bc060ce15bf6a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:31,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d04575bc060ce15bf6d HTTP/1.1" 502 150 2025-12-17 21:27:31,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d04575bc060ce15bf6d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:32,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cec575bc060ce15bf10 2025-12-17 21:27:32,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cec575bc060ce15bf10 HTTP/1.1" 502 150 2025-12-17 21:27:32,404 - lava_callback - ERROR - Error processing callback for node 69407cec575bc060ce15bf10: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cec575bc060ce15bf10 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cec575bc060ce15bf10 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cec575bc060ce15bf10 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:32,406 - lava_callback - INFO - Starting async job submit for node 69407d06575bc060ce15bf74 2025-12-17 21:27:32,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:32,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d06575bc060ce15bf74 HTTP/1.1" 502 150 2025-12-17 21:27:32,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d06575bc060ce15bf74'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:32,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d06575bc060ce15bf74 2025-12-17 21:27:32,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d06575bc060ce15bf74 HTTP/1.1" 502 150 2025-12-17 21:27:32,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d06575bc060ce15bf74'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:32,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eb7575bc060ce15c35a 2025-12-17 21:27:32,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eb7575bc060ce15c35a HTTP/1.1" 502 150 2025-12-17 21:27:32,692 - lava_callback - ERROR - Error processing callback for node 69407eb7575bc060ce15c35a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eb7575bc060ce15c35a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eb7575bc060ce15c35a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eb7575bc060ce15c35a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:32,693 - lava_callback - INFO - Starting async job submit for node 69407d09575bc060ce15bf7a 2025-12-17 21:27:32,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:32,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d09575bc060ce15bf7a HTTP/1.1" 502 150 2025-12-17 21:27:32,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d09575bc060ce15bf7a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:32,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d09575bc060ce15bf7a 2025-12-17 21:27:32,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d09575bc060ce15bf7a HTTP/1.1" 502 150 2025-12-17 21:27:32,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d09575bc060ce15bf7a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:32,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf2575bc060ce15bf22 2025-12-17 21:27:32,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf2575bc060ce15bf22 HTTP/1.1" 502 150 2025-12-17 21:27:32,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf2575bc060ce15bf22'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:32,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf3575bc060ce15bf27 2025-12-17 21:27:32,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf3575bc060ce15bf27 HTTP/1.1" 502 150 2025-12-17 21:27:32,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf3575bc060ce15bf27'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:33,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d05575bc060ce15bf71 2025-12-17 21:27:33,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d05575bc060ce15bf71 HTTP/1.1" 502 150 2025-12-17 21:27:33,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d05575bc060ce15bf71'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:33,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf5575bc060ce15bf2a 2025-12-17 21:27:33,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf5575bc060ce15bf2a HTTP/1.1" 502 150 2025-12-17 21:27:33,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cf5575bc060ce15bf2a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:34,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d06575bc060ce15bf74 2025-12-17 21:27:34,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d06575bc060ce15bf74 HTTP/1.1" 502 150 2025-12-17 21:27:34,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d06575bc060ce15bf74'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:34,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eba575bc060ce15c35f 2025-12-17 21:27:34,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eba575bc060ce15c35f HTTP/1.1" 502 150 2025-12-17 21:27:34,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eba575bc060ce15c35f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:34,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d09575bc060ce15bf7a 2025-12-17 21:27:34,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d09575bc060ce15bf7a HTTP/1.1" 502 150 2025-12-17 21:27:34,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d09575bc060ce15bf7a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:34,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cfc575bc060ce15bf2f 2025-12-17 21:27:34,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cfc575bc060ce15bf2f HTTP/1.1" 502 150 2025-12-17 21:27:34,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cfc575bc060ce15bf2f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:35,313 - lava_callback - INFO - Received callback for node 694078eb575bc060ce15b5fc INFO: 172.21.0.1:34976 - "POST /node/694078eb575bc060ce15b5fc HTTP/1.0" 202 Accepted 2025-12-17 21:27:35,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d02575bc060ce15bf6a 2025-12-17 21:27:35,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d04575bc060ce15bf6d 2025-12-17 21:27:35,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d02575bc060ce15bf6a HTTP/1.1" 502 150 2025-12-17 21:27:35,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d02575bc060ce15bf6a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:35,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d04575bc060ce15bf6d HTTP/1.1" 502 150 2025-12-17 21:27:35,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d04575bc060ce15bf6d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:36,136 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cfe575bc060ce15bf38 2025-12-17 21:27:36,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cfe575bc060ce15bf38 HTTP/1.1" 502 150 2025-12-17 21:27:36,142 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cfe575bc060ce15bf38'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34984 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:27:36,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cff575bc060ce15bf40 2025-12-17 21:27:36,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cff575bc060ce15bf40 HTTP/1.1" 502 150 2025-12-17 21:27:36,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407cff575bc060ce15bf40'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:37,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d05575bc060ce15bf71 2025-12-17 21:27:37,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d05575bc060ce15bf71 HTTP/1.1" 502 150 2025-12-17 21:27:37,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d05575bc060ce15bf71'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:38,282 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cee575bc060ce15bf13 2025-12-17 21:27:38,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cee575bc060ce15bf13 HTTP/1.1" 502 150 2025-12-17 21:27:38,283 - lava_callback - ERROR - Error processing callback for node 69407cee575bc060ce15bf13: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cee575bc060ce15bf13 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cee575bc060ce15bf13 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cee575bc060ce15bf13 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:38,285 - lava_callback - INFO - Starting async job submit for node 69407c0a575bc060ce15bc8b 2025-12-17 21:27:38,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:38,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0a575bc060ce15bc8b HTTP/1.1" 502 150 2025-12-17 21:27:38,318 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0a575bc060ce15bc8b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:38,318 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0a575bc060ce15bc8b 2025-12-17 21:27:38,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0a575bc060ce15bc8b HTTP/1.1" 502 150 2025-12-17 21:27:38,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0a575bc060ce15bc8b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:38,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d06575bc060ce15bf74 2025-12-17 21:27:38,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d06575bc060ce15bf74 HTTP/1.1" 502 150 2025-12-17 21:27:38,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d06575bc060ce15bf74'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:38,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d01575bc060ce15bf67 2025-12-17 21:27:38,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d01575bc060ce15bf67 HTTP/1.1" 502 150 2025-12-17 21:27:38,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d01575bc060ce15bf67'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:38,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d09575bc060ce15bf7a 2025-12-17 21:27:38,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d09575bc060ce15bf7a HTTP/1.1" 502 150 2025-12-17 21:27:38,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d09575bc060ce15bf7a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:40,319 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0a575bc060ce15bc8b 2025-12-17 21:27:40,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0a575bc060ce15bc8b HTTP/1.1" 502 150 2025-12-17 21:27:40,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0a575bc060ce15bc8b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:40,484 - lava_callback - INFO - Received callback for node 69407c36575bc060ce15bd0b INFO: 172.21.0.1:58998 - "POST /node/69407c36575bc060ce15bd0b HTTP/1.0" 202 Accepted 2025-12-17 21:27:41,726 - lava_callback - INFO - Received callback for node 69407d3a575bc060ce15bfd9 INFO: 172.21.0.1:59002 - "POST /node/69407d3a575bc060ce15bfd9 HTTP/1.0" 202 Accepted 2025-12-17 21:27:42,368 - lava_callback - INFO - Received callback for node 69407d3b575bc060ce15bfe1 INFO: 172.21.0.1:59018 - "POST /node/69407d3b575bc060ce15bfe1 HTTP/1.0" 202 Accepted 2025-12-17 21:27:42,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cef575bc060ce15bf16 2025-12-17 21:27:42,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cef575bc060ce15bf16 HTTP/1.1" 502 150 2025-12-17 21:27:42,948 - lava_callback - ERROR - Error processing callback for node 69407cef575bc060ce15bf16: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cef575bc060ce15bf16 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cef575bc060ce15bf16 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cef575bc060ce15bf16 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:42,950 - lava_callback - INFO - Starting async job submit for node 69407d4b575bc060ce15c048 2025-12-17 21:27:42,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:42,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4b575bc060ce15c048 HTTP/1.1" 502 150 2025-12-17 21:27:42,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4b575bc060ce15c048'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:42,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4b575bc060ce15c048 2025-12-17 21:27:42,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4b575bc060ce15c048 HTTP/1.1" 502 150 2025-12-17 21:27:42,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4b575bc060ce15c048'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:43,029 - lava_callback - INFO - Received callback for node 69407d3c575bc060ce15bfe9 INFO: 172.21.0.1:59034 - "POST /node/69407d3c575bc060ce15bfe9 HTTP/1.0" 202 Accepted 2025-12-17 21:27:43,701 - lava_callback - INFO - Received callback for node 69407d3f575bc060ce15c01c INFO: 172.21.0.1:59040 - "POST /node/69407d3f575bc060ce15c01c HTTP/1.0" 202 Accepted 2025-12-17 21:27:43,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf1575bc060ce15bf1b 2025-12-17 21:27:43,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf1575bc060ce15bf1b HTTP/1.1" 502 150 2025-12-17 21:27:43,708 - lava_callback - ERROR - Error processing callback for node 69407cf1575bc060ce15bf1b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cf1575bc060ce15bf1b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cf1575bc060ce15bf1b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cf1575bc060ce15bf1b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:43,709 - lava_callback - INFO - Starting async job submit for node 69407c35575bc060ce15bd05 2025-12-17 21:27:43,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:43,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c35575bc060ce15bd05 HTTP/1.1" 502 150 2025-12-17 21:27:43,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c35575bc060ce15bd05'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:43,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c35575bc060ce15bd05 2025-12-17 21:27:43,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c35575bc060ce15bd05 HTTP/1.1" 502 150 2025-12-17 21:27:43,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c35575bc060ce15bd05'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:43,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d02575bc060ce15bf6a 2025-12-17 21:27:43,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d04575bc060ce15bf6d 2025-12-17 21:27:43,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d02575bc060ce15bf6a HTTP/1.1" 502 150 2025-12-17 21:27:43,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d02575bc060ce15bf6a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:43,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d04575bc060ce15bf6d HTTP/1.1" 502 150 2025-12-17 21:27:43,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d04575bc060ce15bf6d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:44,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0a575bc060ce15bc8b 2025-12-17 21:27:44,372 - lava_callback - INFO - Received callback for node 69407d40575bc060ce15c021 INFO: 172.21.0.1:59042 - "POST /node/69407d40575bc060ce15c021 HTTP/1.0" 202 Accepted 2025-12-17 21:27:44,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4b575bc060ce15c048 2025-12-17 21:27:44,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4b575bc060ce15c048 HTTP/1.1" 502 150 2025-12-17 21:27:44,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4b575bc060ce15c048'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:45,021 - lava_callback - INFO - Received callback for node 69407d42575bc060ce15c024 INFO: 172.21.0.1:59046 - "POST /node/69407d42575bc060ce15c024 HTTP/1.0" 202 Accepted 2025-12-17 21:27:45,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d05575bc060ce15bf71 2025-12-17 21:27:45,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d05575bc060ce15bf71 HTTP/1.1" 502 150 2025-12-17 21:27:45,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d05575bc060ce15bf71'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:45,681 - lava_callback - INFO - Received callback for node 69407d43575bc060ce15c027 INFO: 172.21.0.1:59048 - "POST /node/69407d43575bc060ce15c027 HTTP/1.0" 202 Accepted 2025-12-17 21:27:45,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c35575bc060ce15bd05 2025-12-17 21:27:45,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c35575bc060ce15bd05 HTTP/1.1" 502 150 2025-12-17 21:27:45,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c35575bc060ce15bd05'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:46,332 - lava_callback - INFO - Received callback for node 69407d44575bc060ce15c02a INFO: 172.21.0.1:59054 - "POST /node/69407d44575bc060ce15c02a HTTP/1.0" 202 Accepted 2025-12-17 21:27:46,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d06575bc060ce15bf74 2025-12-17 21:27:46,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d06575bc060ce15bf74 HTTP/1.1" 502 150 2025-12-17 21:27:46,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d06575bc060ce15bf74'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59058 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:27:46,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d09575bc060ce15bf7a 2025-12-17 21:27:46,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d09575bc060ce15bf7a HTTP/1.1" 502 150 2025-12-17 21:27:46,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d09575bc060ce15bf7a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:46,976 - lava_callback - INFO - Received callback for node 69407d45575bc060ce15c02d INFO: 172.21.0.1:59062 - "POST /node/69407d45575bc060ce15c02d HTTP/1.0" 202 Accepted 2025-12-17 21:27:47,613 - lava_callback - INFO - Received callback for node 69407d4a575bc060ce15c042 INFO: 172.21.0.1:59072 - "POST /node/69407d4a575bc060ce15c042 HTTP/1.0" 202 Accepted 2025-12-17 21:27:48,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf2575bc060ce15bf22 2025-12-17 21:27:48,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf2575bc060ce15bf22 HTTP/1.1" 502 150 2025-12-17 21:27:48,850 - lava_callback - ERROR - Error processing callback for node 69407cf2575bc060ce15bf22: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cf2575bc060ce15bf22 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cf2575bc060ce15bf22 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cf2575bc060ce15bf22 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:48,852 - lava_callback - INFO - Starting async job submit for node 69407d16575bc060ce15bf95 2025-12-17 21:27:48,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:48,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d16575bc060ce15bf95 HTTP/1.1" 502 150 2025-12-17 21:27:48,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d16575bc060ce15bf95'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:48,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d16575bc060ce15bf95 2025-12-17 21:27:48,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d16575bc060ce15bf95 HTTP/1.1" 502 150 2025-12-17 21:27:48,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d16575bc060ce15bf95'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:48,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf3575bc060ce15bf27 2025-12-17 21:27:48,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf3575bc060ce15bf27 HTTP/1.1" 502 150 2025-12-17 21:27:48,913 - lava_callback - ERROR - Error processing callback for node 69407cf3575bc060ce15bf27: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cf3575bc060ce15bf27 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cf3575bc060ce15bf27 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cf3575bc060ce15bf27 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:48,915 - lava_callback - INFO - Starting async job submit for node 69407d12575bc060ce15bf82 2025-12-17 21:27:48,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:48,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d12575bc060ce15bf82 HTTP/1.1" 502 150 2025-12-17 21:27:48,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d12575bc060ce15bf82'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:48,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d12575bc060ce15bf82 2025-12-17 21:27:48,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d12575bc060ce15bf82 HTTP/1.1" 502 150 2025-12-17 21:27:48,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d12575bc060ce15bf82'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:48,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4b575bc060ce15c048 2025-12-17 21:27:48,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4b575bc060ce15c048 HTTP/1.1" 502 150 2025-12-17 21:27:48,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4b575bc060ce15c048'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:49,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c35575bc060ce15bd05 2025-12-17 21:27:49,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c35575bc060ce15bd05 HTTP/1.1" 502 150 2025-12-17 21:27:49,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c35575bc060ce15bd05'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:49,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cf5575bc060ce15bf2a 2025-12-17 21:27:49,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cf5575bc060ce15bf2a HTTP/1.1" 502 150 2025-12-17 21:27:49,780 - lava_callback - ERROR - Error processing callback for node 69407cf5575bc060ce15bf2a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cf5575bc060ce15bf2a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cf5575bc060ce15bf2a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cf5575bc060ce15bf2a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:49,783 - lava_callback - INFO - Starting async job submit for node 69407d14575bc060ce15bf89 2025-12-17 21:27:49,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:49,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d14575bc060ce15bf89 HTTP/1.1" 502 150 2025-12-17 21:27:49,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d14575bc060ce15bf89'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:49,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d14575bc060ce15bf89 2025-12-17 21:27:49,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d14575bc060ce15bf89 HTTP/1.1" 502 150 2025-12-17 21:27:49,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d14575bc060ce15bf89'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:50,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eba575bc060ce15c35f 2025-12-17 21:27:50,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eba575bc060ce15c35f HTTP/1.1" 502 150 2025-12-17 21:27:50,467 - lava_callback - ERROR - Error processing callback for node 69407eba575bc060ce15c35f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eba575bc060ce15c35f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eba575bc060ce15c35f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eba575bc060ce15c35f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:50,469 - lava_callback - INFO - Starting async job submit for node 69407d15575bc060ce15bf90 2025-12-17 21:27:50,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:50,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d15575bc060ce15bf90 HTTP/1.1" 502 150 2025-12-17 21:27:50,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d15575bc060ce15bf90'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:50,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d15575bc060ce15bf90 2025-12-17 21:27:50,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d15575bc060ce15bf90 HTTP/1.1" 502 150 2025-12-17 21:27:50,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d15575bc060ce15bf90'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:50,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cfc575bc060ce15bf2f 2025-12-17 21:27:50,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cfc575bc060ce15bf2f HTTP/1.1" 502 150 2025-12-17 21:27:50,753 - lava_callback - ERROR - Error processing callback for node 69407cfc575bc060ce15bf2f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cfc575bc060ce15bf2f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cfc575bc060ce15bf2f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cfc575bc060ce15bf2f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:50,755 - lava_callback - INFO - Starting async job submit for node 69407d17575bc060ce15bf98 2025-12-17 21:27:50,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:50,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d17575bc060ce15bf98 HTTP/1.1" 502 150 2025-12-17 21:27:50,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d17575bc060ce15bf98'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:50,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d17575bc060ce15bf98 2025-12-17 21:27:50,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d17575bc060ce15bf98 HTTP/1.1" 502 150 2025-12-17 21:27:50,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d17575bc060ce15bf98'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:50,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d16575bc060ce15bf95 2025-12-17 21:27:50,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d16575bc060ce15bf95 HTTP/1.1" 502 150 2025-12-17 21:27:50,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d16575bc060ce15bf95'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:50,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d12575bc060ce15bf82 2025-12-17 21:27:50,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d12575bc060ce15bf82 HTTP/1.1" 502 150 2025-12-17 21:27:50,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d12575bc060ce15bf82'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:51,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d14575bc060ce15bf89 2025-12-17 21:27:51,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d14575bc060ce15bf89 HTTP/1.1" 502 150 2025-12-17 21:27:51,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d14575bc060ce15bf89'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:52,142 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cfe575bc060ce15bf38 2025-12-17 21:27:52,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cfe575bc060ce15bf38 HTTP/1.1" 502 150 2025-12-17 21:27:52,152 - lava_callback - ERROR - Error processing callback for node 69407cfe575bc060ce15bf38: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cfe575bc060ce15bf38 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cfe575bc060ce15bf38 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cfe575bc060ce15bf38 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:52,154 - lava_callback - INFO - Starting async job submit for node 69407d19575bc060ce15bf9b 2025-12-17 21:27:52,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:52,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d19575bc060ce15bf9b HTTP/1.1" 502 150 2025-12-17 21:27:52,163 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d19575bc060ce15bf9b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:52,163 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d19575bc060ce15bf9b 2025-12-17 21:27:52,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d19575bc060ce15bf9b HTTP/1.1" 502 150 2025-12-17 21:27:52,164 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d19575bc060ce15bf9b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:52,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d15575bc060ce15bf90 2025-12-17 21:27:52,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d15575bc060ce15bf90 HTTP/1.1" 502 150 2025-12-17 21:27:52,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d15575bc060ce15bf90'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:52,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407cff575bc060ce15bf40 2025-12-17 21:27:52,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407cff575bc060ce15bf40 HTTP/1.1" 502 150 2025-12-17 21:27:52,673 - lava_callback - ERROR - Error processing callback for node 69407cff575bc060ce15bf40: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cff575bc060ce15bf40 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cff575bc060ce15bf40 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407cff575bc060ce15bf40 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:52,674 - lava_callback - INFO - Starting async job submit for node 69407d24575bc060ce15bfa0 2025-12-17 21:27:52,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:52,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d24575bc060ce15bfa0 HTTP/1.1" 502 150 2025-12-17 21:27:52,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d24575bc060ce15bfa0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:52,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d24575bc060ce15bfa0 2025-12-17 21:27:52,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d24575bc060ce15bfa0 HTTP/1.1" 502 150 2025-12-17 21:27:52,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d24575bc060ce15bfa0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:52,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d17575bc060ce15bf98 2025-12-17 21:27:52,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d17575bc060ce15bf98 HTTP/1.1" 502 150 2025-12-17 21:27:52,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d17575bc060ce15bf98'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:53,563 - lava_callback - INFO - Received callback for node 69407ee8575bc060ce15c384 INFO: 172.21.0.1:41454 - "POST /node/69407ee8575bc060ce15c384 HTTP/1.0" 202 Accepted 2025-12-17 21:27:54,165 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d19575bc060ce15bf9b 2025-12-17 21:27:54,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d19575bc060ce15bf9b HTTP/1.1" 502 150 2025-12-17 21:27:54,169 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d19575bc060ce15bf9b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:54,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d01575bc060ce15bf67 2025-12-17 21:27:54,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d01575bc060ce15bf67 HTTP/1.1" 502 150 2025-12-17 21:27:54,556 - lava_callback - ERROR - Error processing callback for node 69407d01575bc060ce15bf67: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d01575bc060ce15bf67 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d01575bc060ce15bf67 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d01575bc060ce15bf67 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:54,558 - lava_callback - INFO - Starting async job submit for node 69407d26575bc060ce15bfa9 2025-12-17 21:27:54,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:54,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d26575bc060ce15bfa9 HTTP/1.1" 502 150 2025-12-17 21:27:54,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d26575bc060ce15bfa9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:54,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d26575bc060ce15bfa9 2025-12-17 21:27:54,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d26575bc060ce15bfa9 HTTP/1.1" 502 150 2025-12-17 21:27:54,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d26575bc060ce15bfa9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:54,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d24575bc060ce15bfa0 2025-12-17 21:27:54,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d24575bc060ce15bfa0 HTTP/1.1" 502 150 2025-12-17 21:27:54,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d24575bc060ce15bfa0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:54,731 - lava_callback - INFO - Received callback for node 69407eea575bc060ce15c387 INFO: 172.21.0.1:41468 - "POST /node/69407eea575bc060ce15c387 HTTP/1.0" 202 Accepted 2025-12-17 21:27:54,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d16575bc060ce15bf95 2025-12-17 21:27:54,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d16575bc060ce15bf95 HTTP/1.1" 502 150 2025-12-17 21:27:54,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d16575bc060ce15bf95'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:54,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d12575bc060ce15bf82 2025-12-17 21:27:54,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d12575bc060ce15bf82 HTTP/1.1" 502 150 2025-12-17 21:27:54,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d12575bc060ce15bf82'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:55,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d14575bc060ce15bf89 2025-12-17 21:27:55,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d14575bc060ce15bf89 HTTP/1.1" 502 150 2025-12-17 21:27:55,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d14575bc060ce15bf89'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:56,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d15575bc060ce15bf90 2025-12-17 21:27:56,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d15575bc060ce15bf90 HTTP/1.1" 502 150 2025-12-17 21:27:56,501 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d15575bc060ce15bf90'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41484 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:27:56,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d26575bc060ce15bfa9 2025-12-17 21:27:56,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d26575bc060ce15bfa9 HTTP/1.1" 502 150 2025-12-17 21:27:56,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d26575bc060ce15bfa9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:56,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d17575bc060ce15bf98 2025-12-17 21:27:56,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d17575bc060ce15bf98 HTTP/1.1" 502 150 2025-12-17 21:27:56,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d17575bc060ce15bf98'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:56,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4b575bc060ce15c048 2025-12-17 21:27:56,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4b575bc060ce15c048 HTTP/1.1" 502 150 2025-12-17 21:27:56,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4b575bc060ce15c048'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:57,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c35575bc060ce15bd05 2025-12-17 21:27:57,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c35575bc060ce15bd05 HTTP/1.1" 502 150 2025-12-17 21:27:57,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c35575bc060ce15bd05'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:58,169 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d19575bc060ce15bf9b 2025-12-17 21:27:58,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d19575bc060ce15bf9b HTTP/1.1" 502 150 2025-12-17 21:27:58,170 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d19575bc060ce15bf9b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:58,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d24575bc060ce15bfa0 2025-12-17 21:27:58,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d24575bc060ce15bfa0 HTTP/1.1" 502 150 2025-12-17 21:27:58,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d24575bc060ce15bfa0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:59,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0a575bc060ce15bc8b HTTP/1.1" 502 150 2025-12-17 21:27:59,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0a575bc060ce15bc8b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:59,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d02575bc060ce15bf6a 2025-12-17 21:27:59,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d04575bc060ce15bf6d 2025-12-17 21:27:59,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d02575bc060ce15bf6a HTTP/1.1" 502 150 2025-12-17 21:27:59,889 - lava_callback - ERROR - Error processing callback for node 69407d02575bc060ce15bf6a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d02575bc060ce15bf6a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d02575bc060ce15bf6a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d02575bc060ce15bf6a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:59,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d04575bc060ce15bf6d HTTP/1.1" 502 150 2025-12-17 21:27:59,891 - lava_callback - INFO - Starting async job submit for node 69407d27575bc060ce15bfb0 2025-12-17 21:27:59,891 - lava_callback - ERROR - Error processing callback for node 69407d04575bc060ce15bf6d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d04575bc060ce15bf6d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d04575bc060ce15bf6d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d04575bc060ce15bf6d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:27:59,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:59,893 - lava_callback - INFO - Starting async job submit for node 69407d29575bc060ce15bfb7 2025-12-17 21:27:59,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:27:59,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d29575bc060ce15bfb7 HTTP/1.1" 502 150 2025-12-17 21:27:59,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d29575bc060ce15bfb7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:59,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d29575bc060ce15bfb7 2025-12-17 21:27:59,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d27575bc060ce15bfb0 HTTP/1.1" 502 150 2025-12-17 21:27:59,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d27575bc060ce15bfb0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:59,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d27575bc060ce15bfb0 2025-12-17 21:27:59,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d29575bc060ce15bfb7 HTTP/1.1" 502 150 2025-12-17 21:27:59,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d29575bc060ce15bfb7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:27:59,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d27575bc060ce15bfb0 HTTP/1.1" 502 150 2025-12-17 21:27:59,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d27575bc060ce15bfb0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:00,512 - lava_callback - INFO - Received callback for node 69407c4e575bc060ce15bd97 INFO: 172.21.0.1:55736 - "POST /node/69407c4e575bc060ce15bd97 HTTP/1.0" 202 Accepted 2025-12-17 21:28:00,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d26575bc060ce15bfa9 2025-12-17 21:28:00,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d26575bc060ce15bfa9 HTTP/1.1" 502 150 2025-12-17 21:28:00,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d26575bc060ce15bfa9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55740 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:28:01,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d05575bc060ce15bf71 2025-12-17 21:28:01,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d05575bc060ce15bf71 HTTP/1.1" 502 150 2025-12-17 21:28:01,650 - lava_callback - ERROR - Error processing callback for node 69407d05575bc060ce15bf71: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d05575bc060ce15bf71 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d05575bc060ce15bf71 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d05575bc060ce15bf71 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:01,653 - lava_callback - INFO - Starting async job submit for node 69407d2a575bc060ce15bfbe 2025-12-17 21:28:01,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:01,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2a575bc060ce15bfbe HTTP/1.1" 502 150 2025-12-17 21:28:01,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2a575bc060ce15bfbe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:01,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2a575bc060ce15bfbe 2025-12-17 21:28:01,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2a575bc060ce15bfbe HTTP/1.1" 502 150 2025-12-17 21:28:01,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2a575bc060ce15bfbe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:01,743 - lava_callback - INFO - Received callback for node 69407d4d575bc060ce15c052 INFO: 172.21.0.1:55750 - "POST /node/69407d4d575bc060ce15c052 HTTP/1.0" 202 Accepted 2025-12-17 21:28:01,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d29575bc060ce15bfb7 2025-12-17 21:28:01,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d27575bc060ce15bfb0 2025-12-17 21:28:01,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d29575bc060ce15bfb7 HTTP/1.1" 502 150 2025-12-17 21:28:01,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d29575bc060ce15bfb7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:01,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d27575bc060ce15bfb0 HTTP/1.1" 502 150 2025-12-17 21:28:01,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d27575bc060ce15bfb0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:02,387 - lava_callback - INFO - Received callback for node 69407d4e575bc060ce15c05a INFO: 172.21.0.1:55756 - "POST /node/69407d4e575bc060ce15c05a HTTP/1.0" 202 Accepted 2025-12-17 21:28:02,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d06575bc060ce15bf74 2025-12-17 21:28:02,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d06575bc060ce15bf74 HTTP/1.1" 502 150 2025-12-17 21:28:02,420 - lava_callback - ERROR - Error processing callback for node 69407d06575bc060ce15bf74: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d06575bc060ce15bf74 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d06575bc060ce15bf74 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d06575bc060ce15bf74 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:02,423 - lava_callback - INFO - Starting async job submit for node 69407d2b575bc060ce15bfc1 2025-12-17 21:28:02,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:02,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2b575bc060ce15bfc1 HTTP/1.1" 502 150 2025-12-17 21:28:02,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2b575bc060ce15bfc1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:02,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2b575bc060ce15bfc1 2025-12-17 21:28:02,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2b575bc060ce15bfc1 HTTP/1.1" 502 150 2025-12-17 21:28:02,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2b575bc060ce15bfc1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:02,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d09575bc060ce15bf7a 2025-12-17 21:28:02,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d09575bc060ce15bf7a HTTP/1.1" 502 150 2025-12-17 21:28:02,707 - lava_callback - ERROR - Error processing callback for node 69407d09575bc060ce15bf7a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d09575bc060ce15bf7a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d09575bc060ce15bf7a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d09575bc060ce15bf7a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:02,709 - lava_callback - INFO - Starting async job submit for node 69407d2c575bc060ce15bfc4 2025-12-17 21:28:02,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:02,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2c575bc060ce15bfc4 HTTP/1.1" 502 150 2025-12-17 21:28:02,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2c575bc060ce15bfc4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:02,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2c575bc060ce15bfc4 2025-12-17 21:28:02,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2c575bc060ce15bfc4 HTTP/1.1" 502 150 2025-12-17 21:28:02,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2c575bc060ce15bfc4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:02,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d16575bc060ce15bf95 2025-12-17 21:28:02,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d16575bc060ce15bf95 HTTP/1.1" 502 150 2025-12-17 21:28:02,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d16575bc060ce15bf95'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:02,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d12575bc060ce15bf82 2025-12-17 21:28:02,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d12575bc060ce15bf82 HTTP/1.1" 502 150 2025-12-17 21:28:02,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d12575bc060ce15bf82'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:03,043 - lava_callback - INFO - Received callback for node 69407d4f575bc060ce15c062 INFO: 172.21.0.1:55766 - "POST /node/69407d4f575bc060ce15c062 HTTP/1.0" 202 Accepted 2025-12-17 21:28:03,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2a575bc060ce15bfbe 2025-12-17 21:28:03,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2a575bc060ce15bfbe HTTP/1.1" 502 150 2025-12-17 21:28:03,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2a575bc060ce15bfbe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:03,695 - lava_callback - INFO - Received callback for node 69407d51575bc060ce15c069 INFO: 172.21.0.1:55780 - "POST /node/69407d51575bc060ce15c069 HTTP/1.0" 202 Accepted 2025-12-17 21:28:03,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d14575bc060ce15bf89 2025-12-17 21:28:03,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d14575bc060ce15bf89 HTTP/1.1" 502 150 2025-12-17 21:28:03,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d14575bc060ce15bf89'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:04,328 - lava_callback - INFO - Received callback for node 69407d52575bc060ce15c06c INFO: 172.21.0.1:55792 - "POST /node/69407d52575bc060ce15c06c HTTP/1.0" 202 Accepted 2025-12-17 21:28:04,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2b575bc060ce15bfc1 2025-12-17 21:28:04,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2b575bc060ce15bfc1 HTTP/1.1" 502 150 2025-12-17 21:28:04,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2b575bc060ce15bfc1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:04,501 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d15575bc060ce15bf90 2025-12-17 21:28:04,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d15575bc060ce15bf90 HTTP/1.1" 502 150 2025-12-17 21:28:04,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d15575bc060ce15bf90'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:04,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2c575bc060ce15bfc4 2025-12-17 21:28:04,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2c575bc060ce15bfc4 HTTP/1.1" 502 150 2025-12-17 21:28:04,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2c575bc060ce15bfc4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:04,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d17575bc060ce15bf98 2025-12-17 21:28:04,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d17575bc060ce15bf98 HTTP/1.1" 502 150 2025-12-17 21:28:04,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d17575bc060ce15bf98'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:04,991 - lava_callback - INFO - Received callback for node 69407d53575bc060ce15c06f INFO: 172.21.0.1:55808 - "POST /node/69407d53575bc060ce15c06f HTTP/1.0" 202 Accepted 2025-12-17 21:28:05,657 - lava_callback - INFO - Received callback for node 69407d54575bc060ce15c072 INFO: 172.21.0.1:55822 - "POST /node/69407d54575bc060ce15c072 HTTP/1.0" 202 Accepted 2025-12-17 21:28:05,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d29575bc060ce15bfb7 2025-12-17 21:28:05,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d27575bc060ce15bfb0 2025-12-17 21:28:05,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d29575bc060ce15bfb7 HTTP/1.1" 502 150 2025-12-17 21:28:05,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d29575bc060ce15bfb7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:05,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d27575bc060ce15bfb0 HTTP/1.1" 502 150 2025-12-17 21:28:05,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d27575bc060ce15bfb0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:06,170 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d19575bc060ce15bf9b 2025-12-17 21:28:06,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d19575bc060ce15bf9b HTTP/1.1" 502 150 2025-12-17 21:28:06,172 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d19575bc060ce15bf9b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:06,318 - lava_callback - INFO - Received callback for node 69407d60575bc060ce15c077 INFO: 172.21.0.1:55836 - "POST /node/69407d60575bc060ce15c077 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:55840 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:28:06,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d24575bc060ce15bfa0 2025-12-17 21:28:06,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d24575bc060ce15bfa0 HTTP/1.1" 502 150 2025-12-17 21:28:06,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d24575bc060ce15bfa0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:07,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0a575bc060ce15bc8b 2025-12-17 21:28:07,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0a575bc060ce15bc8b HTTP/1.1" 502 150 2025-12-17 21:28:07,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c0a575bc060ce15bc8b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:07,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2a575bc060ce15bfbe 2025-12-17 21:28:07,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2a575bc060ce15bfbe HTTP/1.1" 502 150 2025-12-17 21:28:07,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2a575bc060ce15bfbe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:08,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2b575bc060ce15bfc1 2025-12-17 21:28:08,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2b575bc060ce15bfc1 HTTP/1.1" 502 150 2025-12-17 21:28:08,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2b575bc060ce15bfc1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:08,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d26575bc060ce15bfa9 2025-12-17 21:28:08,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d26575bc060ce15bfa9 HTTP/1.1" 502 150 2025-12-17 21:28:08,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d26575bc060ce15bfa9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:08,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2c575bc060ce15bfc4 2025-12-17 21:28:08,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2c575bc060ce15bfc4 HTTP/1.1" 502 150 2025-12-17 21:28:08,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2c575bc060ce15bfc4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:11,599 - lava_callback - INFO - Received callback for node 69407d75575bc060ce15c0b5 INFO: 172.21.0.1:38834 - "POST /node/69407d75575bc060ce15c0b5 HTTP/1.0" 202 Accepted 2025-12-17 21:28:12,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4b575bc060ce15c048 2025-12-17 21:28:12,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4b575bc060ce15c048 HTTP/1.1" 502 150 2025-12-17 21:28:12,966 - lava_callback - ERROR - Error processing callback for node 69407d4b575bc060ce15c048: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4b575bc060ce15c048 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4b575bc060ce15c048 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4b575bc060ce15c048 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:12,968 - lava_callback - INFO - Starting async job submit for node 69407d2e575bc060ce15bfc7 2025-12-17 21:28:12,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:12,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2e575bc060ce15bfc7 HTTP/1.1" 502 150 2025-12-17 21:28:12,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2e575bc060ce15bfc7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:12,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2e575bc060ce15bfc7 2025-12-17 21:28:12,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2e575bc060ce15bfc7 HTTP/1.1" 502 150 2025-12-17 21:28:12,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2e575bc060ce15bfc7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:13,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c35575bc060ce15bd05 2025-12-17 21:28:13,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c35575bc060ce15bd05 HTTP/1.1" 502 150 2025-12-17 21:28:13,758 - lava_callback - ERROR - Error processing callback for node 69407c35575bc060ce15bd05: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c35575bc060ce15bd05 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c35575bc060ce15bd05 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c35575bc060ce15bd05 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:13,760 - lava_callback - INFO - Starting async job submit for node 69407d2f575bc060ce15bfca 2025-12-17 21:28:13,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:13,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2f575bc060ce15bfca HTTP/1.1" 502 150 2025-12-17 21:28:13,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2f575bc060ce15bfca'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:13,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2f575bc060ce15bfca 2025-12-17 21:28:13,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2f575bc060ce15bfca HTTP/1.1" 502 150 2025-12-17 21:28:13,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2f575bc060ce15bfca'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:13,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d29575bc060ce15bfb7 2025-12-17 21:28:13,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d29575bc060ce15bfb7 HTTP/1.1" 502 150 2025-12-17 21:28:13,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d29575bc060ce15bfb7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:13,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d27575bc060ce15bfb0 2025-12-17 21:28:13,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d27575bc060ce15bfb0 HTTP/1.1" 502 150 2025-12-17 21:28:13,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d27575bc060ce15bfb0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:14,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2e575bc060ce15bfc7 2025-12-17 21:28:14,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2e575bc060ce15bfc7 HTTP/1.1" 502 150 2025-12-17 21:28:14,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2e575bc060ce15bfc7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38836 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:28:15,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2a575bc060ce15bfbe 2025-12-17 21:28:15,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2a575bc060ce15bfbe HTTP/1.1" 502 150 2025-12-17 21:28:15,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2a575bc060ce15bfbe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:15,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2f575bc060ce15bfca 2025-12-17 21:28:15,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2f575bc060ce15bfca HTTP/1.1" 502 150 2025-12-17 21:28:15,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2f575bc060ce15bfca'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:16,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2b575bc060ce15bfc1 2025-12-17 21:28:16,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2b575bc060ce15bfc1 HTTP/1.1" 502 150 2025-12-17 21:28:16,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2b575bc060ce15bfc1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38844 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:28:16,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2c575bc060ce15bfc4 2025-12-17 21:28:16,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2c575bc060ce15bfc4 HTTP/1.1" 502 150 2025-12-17 21:28:16,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2c575bc060ce15bfc4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:17,642 - lava_callback - INFO - Received callback for node 69407d63575bc060ce15c085 INFO: 172.21.0.1:38848 - "POST /node/69407d63575bc060ce15c085 HTTP/1.0" 202 Accepted 2025-12-17 21:28:18,421 - lava_callback - INFO - Received callback for node 69407d64575bc060ce15c08c INFO: 172.21.0.1:38850 - "POST /node/69407d64575bc060ce15c08c HTTP/1.0" 202 Accepted 2025-12-17 21:28:18,556 - lava_callback - INFO - Received callback for node 69407eee575bc060ce15c39d INFO: 172.21.0.1:38858 - "POST /node/69407eee575bc060ce15c39d HTTP/1.0" 202 Accepted 2025-12-17 21:28:18,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d16575bc060ce15bf95 2025-12-17 21:28:18,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d16575bc060ce15bf95 HTTP/1.1" 502 150 2025-12-17 21:28:18,871 - lava_callback - ERROR - Error processing callback for node 69407d16575bc060ce15bf95: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d16575bc060ce15bf95 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d16575bc060ce15bf95 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d16575bc060ce15bf95 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:18,873 - lava_callback - INFO - Starting async job submit for node 69407d30575bc060ce15bfcd 2025-12-17 21:28:18,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:18,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d30575bc060ce15bfcd HTTP/1.1" 502 150 2025-12-17 21:28:18,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d30575bc060ce15bfcd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:18,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d30575bc060ce15bfcd 2025-12-17 21:28:18,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d30575bc060ce15bfcd HTTP/1.1" 502 150 2025-12-17 21:28:18,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d30575bc060ce15bfcd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:18,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d12575bc060ce15bf82 2025-12-17 21:28:18,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d12575bc060ce15bf82 HTTP/1.1" 502 150 2025-12-17 21:28:18,928 - lava_callback - ERROR - Error processing callback for node 69407d12575bc060ce15bf82: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d12575bc060ce15bf82 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d12575bc060ce15bf82 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d12575bc060ce15bf82 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:18,930 - lava_callback - INFO - Starting async job submit for node 69407d38575bc060ce15bfd2 2025-12-17 21:28:18,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:18,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d38575bc060ce15bfd2 HTTP/1.1" 502 150 2025-12-17 21:28:18,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d38575bc060ce15bfd2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:18,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d38575bc060ce15bfd2 2025-12-17 21:28:18,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d38575bc060ce15bfd2 HTTP/1.1" 502 150 2025-12-17 21:28:18,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d38575bc060ce15bfd2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:18,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2e575bc060ce15bfc7 2025-12-17 21:28:18,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2e575bc060ce15bfc7 HTTP/1.1" 502 150 2025-12-17 21:28:18,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2e575bc060ce15bfc7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:19,173 - lava_callback - INFO - Received callback for node 69407d65575bc060ce15c093 INFO: 172.21.0.1:38864 - "POST /node/69407d65575bc060ce15c093 HTTP/1.0" 202 Accepted 2025-12-17 21:28:19,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2f575bc060ce15bfca 2025-12-17 21:28:19,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2f575bc060ce15bfca HTTP/1.1" 502 150 2025-12-17 21:28:19,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2f575bc060ce15bfca'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:19,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d14575bc060ce15bf89 2025-12-17 21:28:19,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d14575bc060ce15bf89 HTTP/1.1" 502 150 2025-12-17 21:28:19,801 - lava_callback - ERROR - Error processing callback for node 69407d14575bc060ce15bf89: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d14575bc060ce15bf89 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d14575bc060ce15bf89 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d14575bc060ce15bf89 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:19,803 - lava_callback - INFO - Starting async job submit for node 69407d4c575bc060ce15c04b 2025-12-17 21:28:19,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:19,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4c575bc060ce15c04b HTTP/1.1" 502 150 2025-12-17 21:28:19,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4c575bc060ce15c04b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:19,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4c575bc060ce15c04b 2025-12-17 21:28:19,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4c575bc060ce15c04b HTTP/1.1" 502 150 2025-12-17 21:28:19,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4c575bc060ce15c04b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:19,945 - lava_callback - INFO - Received callback for node 69407d67575bc060ce15c09a INFO: 172.21.0.1:42672 - "POST /node/69407d67575bc060ce15c09a HTTP/1.0" 202 Accepted 2025-12-17 21:28:20,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d15575bc060ce15bf90 2025-12-17 21:28:20,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d15575bc060ce15bf90 HTTP/1.1" 502 150 2025-12-17 21:28:20,505 - lava_callback - ERROR - Error processing callback for node 69407d15575bc060ce15bf90: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d15575bc060ce15bf90 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d15575bc060ce15bf90 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d15575bc060ce15bf90 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:20,507 - lava_callback - INFO - Starting async job submit for node 694078eb575bc060ce15b5fc 2025-12-17 21:28:20,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:20,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078eb575bc060ce15b5fc HTTP/1.1" 502 150 2025-12-17 21:28:20,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078eb575bc060ce15b5fc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:20,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078eb575bc060ce15b5fc 2025-12-17 21:28:20,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078eb575bc060ce15b5fc HTTP/1.1" 502 150 2025-12-17 21:28:20,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078eb575bc060ce15b5fc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:20,766 - lava_callback - INFO - Received callback for node 69407d68575bc060ce15c09f INFO: 172.21.0.1:42688 - "POST /node/69407d68575bc060ce15c09f HTTP/1.0" 202 Accepted 2025-12-17 21:28:20,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d17575bc060ce15bf98 2025-12-17 21:28:20,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d17575bc060ce15bf98 HTTP/1.1" 502 150 2025-12-17 21:28:20,773 - lava_callback - ERROR - Error processing callback for node 69407d17575bc060ce15bf98: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d17575bc060ce15bf98 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d17575bc060ce15bf98 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d17575bc060ce15bf98 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:20,775 - lava_callback - INFO - Starting async job submit for node 69407c36575bc060ce15bd0b 2025-12-17 21:28:20,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:20,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d30575bc060ce15bfcd 2025-12-17 21:28:20,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d30575bc060ce15bfcd HTTP/1.1" 502 150 2025-12-17 21:28:20,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d30575bc060ce15bfcd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:20,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c36575bc060ce15bd0b HTTP/1.1" 502 150 2025-12-17 21:28:20,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c36575bc060ce15bd0b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:20,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c36575bc060ce15bd0b 2025-12-17 21:28:20,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c36575bc060ce15bd0b HTTP/1.1" 502 150 2025-12-17 21:28:20,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c36575bc060ce15bd0b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:20,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d38575bc060ce15bfd2 2025-12-17 21:28:20,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d38575bc060ce15bfd2 HTTP/1.1" 502 150 2025-12-17 21:28:20,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d38575bc060ce15bfd2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:21,537 - lava_callback - INFO - Received callback for node 69407d69575bc060ce15c0a2 INFO: 172.21.0.1:42696 - "POST /node/69407d69575bc060ce15c0a2 HTTP/1.0" 202 Accepted 2025-12-17 21:28:21,744 - lava_callback - INFO - Received callback for node 69407d61575bc060ce15c080 INFO: 172.21.0.1:42708 - "POST /node/69407d61575bc060ce15c080 HTTP/1.0" 202 Accepted 2025-12-17 21:28:21,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4c575bc060ce15c04b 2025-12-17 21:28:21,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4c575bc060ce15c04b HTTP/1.1" 502 150 2025-12-17 21:28:21,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4c575bc060ce15c04b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:22,172 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d19575bc060ce15bf9b 2025-12-17 21:28:22,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d19575bc060ce15bf9b HTTP/1.1" 502 150 2025-12-17 21:28:22,173 - lava_callback - ERROR - Error processing callback for node 69407d19575bc060ce15bf9b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d19575bc060ce15bf9b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d19575bc060ce15bf9b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d19575bc060ce15bf9b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:22,175 - lava_callback - INFO - Starting async job submit for node 69407d3a575bc060ce15bfd9 2025-12-17 21:28:22,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:22,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3a575bc060ce15bfd9 HTTP/1.1" 502 150 2025-12-17 21:28:22,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3a575bc060ce15bfd9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:22,183 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3a575bc060ce15bfd9 2025-12-17 21:28:22,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3a575bc060ce15bfd9 HTTP/1.1" 502 150 2025-12-17 21:28:22,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3a575bc060ce15bfd9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:22,319 - lava_callback - INFO - Received callback for node 69407d6b575bc060ce15c0a5 INFO: 172.21.0.1:42724 - "POST /node/69407d6b575bc060ce15c0a5 HTTP/1.0" 202 Accepted 2025-12-17 21:28:22,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078eb575bc060ce15b5fc 2025-12-17 21:28:22,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078eb575bc060ce15b5fc HTTP/1.1" 502 150 2025-12-17 21:28:22,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078eb575bc060ce15b5fc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:22,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d24575bc060ce15bfa0 2025-12-17 21:28:22,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d24575bc060ce15bfa0 HTTP/1.1" 502 150 2025-12-17 21:28:22,688 - lava_callback - ERROR - Error processing callback for node 69407d24575bc060ce15bfa0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d24575bc060ce15bfa0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d24575bc060ce15bfa0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d24575bc060ce15bfa0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:22,690 - lava_callback - INFO - Starting async job submit for node 69407d3b575bc060ce15bfe1 2025-12-17 21:28:22,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:22,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3b575bc060ce15bfe1 HTTP/1.1" 502 150 2025-12-17 21:28:22,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3b575bc060ce15bfe1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:22,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3b575bc060ce15bfe1 2025-12-17 21:28:22,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3b575bc060ce15bfe1 HTTP/1.1" 502 150 2025-12-17 21:28:22,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3b575bc060ce15bfe1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:22,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c36575bc060ce15bd0b 2025-12-17 21:28:22,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c36575bc060ce15bd0b HTTP/1.1" 502 150 2025-12-17 21:28:22,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c36575bc060ce15bd0b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:22,909 - lava_callback - INFO - Received callback for node 694079a1575bc060ce15b721 INFO: 172.21.0.1:42736 - "POST /node/694079a1575bc060ce15b721 HTTP/1.0" 202 Accepted 2025-12-17 21:28:23,110 - lava_callback - INFO - Received callback for node 69407d6c575bc060ce15c0a8 INFO: 172.21.0.1:42752 - "POST /node/69407d6c575bc060ce15c0a8 HTTP/1.0" 202 Accepted 2025-12-17 21:28:23,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c0a575bc060ce15bc8b 2025-12-17 21:28:23,896 - lava_callback - INFO - Received callback for node 69407d6d575bc060ce15c0ab INFO: 172.21.0.1:42764 - "POST /node/69407d6d575bc060ce15c0ab HTTP/1.0" 202 Accepted 2025-12-17 21:28:24,184 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3a575bc060ce15bfd9 2025-12-17 21:28:24,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d26575bc060ce15bfa9 2025-12-17 21:28:24,590 - lava_callback - INFO - Received callback for node 6940998d575bc060ce161d37 INFO: 172.21.0.1:42770 - "POST /node/6940998d575bc060ce161d37 HTTP/1.0" 202 Accepted 2025-12-17 21:28:24,659 - lava_callback - INFO - Received callback for node 69407d77575bc060ce15c0bf INFO: 172.21.0.1:42784 - "POST /node/69407d77575bc060ce15c0bf HTTP/1.0" 202 Accepted 2025-12-17 21:28:24,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3b575bc060ce15bfe1 2025-12-17 21:28:24,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d30575bc060ce15bfcd 2025-12-17 21:28:24,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d38575bc060ce15bfd2 2025-12-17 21:28:25,419 - lava_callback - INFO - Received callback for node 69407d78575bc060ce15c0c4 INFO: 172.21.0.1:42800 - "POST /node/69407d78575bc060ce15c0c4 HTTP/1.0" 202 Accepted 2025-12-17 21:28:25,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4c575bc060ce15c04b INFO: 172.21.0.1:42812 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:28:26,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078eb575bc060ce15b5fc 2025-12-17 21:28:26,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c36575bc060ce15bd0b 2025-12-17 21:28:26,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2e575bc060ce15bfc7 2025-12-17 21:28:27,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2f575bc060ce15bfca 2025-12-17 21:28:29,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d29575bc060ce15bfb7 2025-12-17 21:28:29,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d27575bc060ce15bfb0 2025-12-17 21:28:31,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2a575bc060ce15bfbe 2025-12-17 21:28:32,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2b575bc060ce15bfc1 2025-12-17 21:28:32,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2c575bc060ce15bfc4 INFO: 172.21.0.1:38174 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:28:37,368 - lava_callback - INFO - Received callback for node 69407d79575bc060ce15c0c9 INFO: 172.21.0.1:38190 - "POST /node/69407d79575bc060ce15c0c9 HTTP/1.0" 202 Accepted 2025-12-17 21:28:38,201 - lava_callback - INFO - Received callback for node 69407d79575bc060ce15c0ce INFO: 172.21.0.1:38198 - "POST /node/69407d79575bc060ce15c0ce HTTP/1.0" 202 Accepted 2025-12-17 21:28:38,940 - lava_callback - INFO - Received callback for node 69407d7b575bc060ce15c0d7 INFO: 172.21.0.1:38210 - "POST /node/69407d7b575bc060ce15c0d7 HTTP/1.0" 202 Accepted 2025-12-17 21:28:39,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c0a575bc060ce15bc8b HTTP/1.1" 502 150 2025-12-17 21:28:39,088 - lava_callback - ERROR - Error processing callback for node 69407c0a575bc060ce15bc8b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c0a575bc060ce15bc8b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c0a575bc060ce15bc8b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c0a575bc060ce15bc8b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:39,090 - lava_callback - INFO - Starting async job submit for node 69407d3c575bc060ce15bfe9 2025-12-17 21:28:39,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:39,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3c575bc060ce15bfe9 HTTP/1.1" 502 150 2025-12-17 21:28:39,116 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3c575bc060ce15bfe9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:39,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3c575bc060ce15bfe9 2025-12-17 21:28:39,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3c575bc060ce15bfe9 HTTP/1.1" 502 150 2025-12-17 21:28:39,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3c575bc060ce15bfe9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:39,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3a575bc060ce15bfd9 HTTP/1.1" 502 150 2025-12-17 21:28:39,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3a575bc060ce15bfd9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:39,758 - lava_callback - INFO - Received callback for node 69407d7b575bc060ce15c0ee INFO: 172.21.0.1:58030 - "POST /node/69407d7b575bc060ce15c0ee HTTP/1.0" 202 Accepted 2025-12-17 21:28:39,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d26575bc060ce15bfa9 HTTP/1.1" 502 150 2025-12-17 21:28:39,856 - lava_callback - ERROR - Error processing callback for node 69407d26575bc060ce15bfa9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d26575bc060ce15bfa9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d26575bc060ce15bfa9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d26575bc060ce15bfa9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:39,858 - lava_callback - INFO - Starting async job submit for node 69407d3f575bc060ce15c01c 2025-12-17 21:28:39,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:39,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3f575bc060ce15c01c HTTP/1.1" 502 150 2025-12-17 21:28:39,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3f575bc060ce15c01c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:39,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3f575bc060ce15c01c 2025-12-17 21:28:39,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3f575bc060ce15c01c HTTP/1.1" 502 150 2025-12-17 21:28:39,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3f575bc060ce15c01c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:40,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3b575bc060ce15bfe1 HTTP/1.1" 502 150 2025-12-17 21:28:40,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3b575bc060ce15bfe1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:40,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d30575bc060ce15bfcd HTTP/1.1" 502 150 2025-12-17 21:28:40,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d30575bc060ce15bfcd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:40,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d38575bc060ce15bfd2 HTTP/1.1" 502 150 2025-12-17 21:28:40,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d38575bc060ce15bfd2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:40,498 - lava_callback - INFO - Received callback for node 69407d7c575bc060ce15c0ff INFO: 172.21.0.1:58040 - "POST /node/69407d7c575bc060ce15c0ff HTTP/1.0" 202 Accepted 2025-12-17 21:28:41,118 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3c575bc060ce15bfe9 2025-12-17 21:28:41,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3c575bc060ce15bfe9 HTTP/1.1" 502 150 2025-12-17 21:28:41,119 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3c575bc060ce15bfe9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:41,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4c575bc060ce15c04b HTTP/1.1" 502 150 2025-12-17 21:28:41,136 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4c575bc060ce15c04b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:41,234 - lava_callback - INFO - Received callback for node 69407d7d575bc060ce15c10d INFO: 172.21.0.1:58046 - "POST /node/69407d7d575bc060ce15c10d HTTP/1.0" 202 Accepted 2025-12-17 21:28:41,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3f575bc060ce15c01c 2025-12-17 21:28:41,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3f575bc060ce15c01c HTTP/1.1" 502 150 2025-12-17 21:28:41,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3f575bc060ce15c01c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:41,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078eb575bc060ce15b5fc HTTP/1.1" 502 150 2025-12-17 21:28:41,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078eb575bc060ce15b5fc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:42,037 - lava_callback - INFO - Received callback for node 69407d7d575bc060ce15c115 INFO: 172.21.0.1:58050 - "POST /node/69407d7d575bc060ce15c115 HTTP/1.0" 202 Accepted 2025-12-17 21:28:42,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c36575bc060ce15bd0b HTTP/1.1" 502 150 2025-12-17 21:28:42,160 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c36575bc060ce15bd0b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:42,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2e575bc060ce15bfc7 HTTP/1.1" 502 150 2025-12-17 21:28:42,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2e575bc060ce15bfc7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:42,763 - lava_callback - INFO - Received callback for node 69407d80575bc060ce15c127 INFO: 172.21.0.1:58066 - "POST /node/69407d80575bc060ce15c127 HTTP/1.0" 202 Accepted 2025-12-17 21:28:43,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2f575bc060ce15bfca HTTP/1.1" 502 150 2025-12-17 21:28:43,184 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d2f575bc060ce15bfca'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:43,515 - lava_callback - INFO - Received callback for node 69407d89575bc060ce15c130 INFO: 172.21.0.1:58082 - "POST /node/69407d89575bc060ce15c130 HTTP/1.0" 202 Accepted 2025-12-17 21:28:43,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3a575bc060ce15bfd9 2025-12-17 21:28:43,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3a575bc060ce15bfd9 HTTP/1.1" 502 150 2025-12-17 21:28:43,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3a575bc060ce15bfd9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:44,113 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3b575bc060ce15bfe1 2025-12-17 21:28:44,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3b575bc060ce15bfe1 HTTP/1.1" 502 150 2025-12-17 21:28:44,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3b575bc060ce15bfe1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:44,254 - lava_callback - INFO - Received callback for node 69407d89575bc060ce15c133 INFO: 172.21.0.1:58084 - "POST /node/69407d89575bc060ce15c133 HTTP/1.0" 202 Accepted 2025-12-17 21:28:45,011 - lava_callback - INFO - Received callback for node 69407d8a575bc060ce15c136 INFO: 172.21.0.1:58096 - "POST /node/69407d8a575bc060ce15c136 HTTP/1.0" 202 Accepted 2025-12-17 21:28:45,119 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3c575bc060ce15bfe9 2025-12-17 21:28:45,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3c575bc060ce15bfe9 HTTP/1.1" 502 150 2025-12-17 21:28:45,120 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3c575bc060ce15bfe9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:45,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d27575bc060ce15bfb0 HTTP/1.1" 502 150 2025-12-17 21:28:45,232 - lava_callback - ERROR - Error processing callback for node 69407d27575bc060ce15bfb0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d27575bc060ce15bfb0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d27575bc060ce15bfb0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d27575bc060ce15bfb0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:45,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d29575bc060ce15bfb7 HTTP/1.1" 502 150 2025-12-17 21:28:45,234 - lava_callback - ERROR - Error processing callback for node 69407d29575bc060ce15bfb7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d29575bc060ce15bfb7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d29575bc060ce15bfb7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d29575bc060ce15bfb7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:45,235 - lava_callback - INFO - Starting async job submit for node 69407d40575bc060ce15c021 2025-12-17 21:28:45,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:45,237 - lava_callback - INFO - Starting async job submit for node 69407d42575bc060ce15c024 2025-12-17 21:28:45,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:45,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d42575bc060ce15c024 HTTP/1.1" 502 150 2025-12-17 21:28:45,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d42575bc060ce15c024'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:45,244 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d42575bc060ce15c024 2025-12-17 21:28:45,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d42575bc060ce15c024 HTTP/1.1" 502 150 2025-12-17 21:28:45,244 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d42575bc060ce15c024'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:45,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d40575bc060ce15c021 HTTP/1.1" 502 150 2025-12-17 21:28:45,279 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d40575bc060ce15c021'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:45,279 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d40575bc060ce15c021 2025-12-17 21:28:45,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d40575bc060ce15c021 HTTP/1.1" 502 150 2025-12-17 21:28:45,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d40575bc060ce15c021'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:45,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3f575bc060ce15c01c 2025-12-17 21:28:45,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3f575bc060ce15c01c HTTP/1.1" 502 150 2025-12-17 21:28:45,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3f575bc060ce15c01c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58098 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:28:47,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2a575bc060ce15bfbe HTTP/1.1" 502 150 2025-12-17 21:28:47,024 - lava_callback - ERROR - Error processing callback for node 69407d2a575bc060ce15bfbe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d2a575bc060ce15bfbe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d2a575bc060ce15bfbe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d2a575bc060ce15bfbe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:47,026 - lava_callback - INFO - Starting async job submit for node 69407d43575bc060ce15c027 2025-12-17 21:28:47,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:47,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d43575bc060ce15c027 HTTP/1.1" 502 150 2025-12-17 21:28:47,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d43575bc060ce15c027'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:47,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d43575bc060ce15c027 2025-12-17 21:28:47,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d43575bc060ce15c027 HTTP/1.1" 502 150 2025-12-17 21:28:47,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d43575bc060ce15c027'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:47,245 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d42575bc060ce15c024 2025-12-17 21:28:47,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d42575bc060ce15c024 HTTP/1.1" 502 150 2025-12-17 21:28:47,246 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d42575bc060ce15c024'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:47,280 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d40575bc060ce15c021 2025-12-17 21:28:47,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d40575bc060ce15c021 HTTP/1.1" 502 150 2025-12-17 21:28:47,281 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d40575bc060ce15c021'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:47,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2b575bc060ce15bfc1 HTTP/1.1" 502 150 2025-12-17 21:28:47,797 - lava_callback - ERROR - Error processing callback for node 69407d2b575bc060ce15bfc1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d2b575bc060ce15bfc1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d2b575bc060ce15bfc1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d2b575bc060ce15bfc1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:47,799 - lava_callback - INFO - Starting async job submit for node 69407d44575bc060ce15c02a 2025-12-17 21:28:47,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:47,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d44575bc060ce15c02a HTTP/1.1" 502 150 2025-12-17 21:28:47,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d44575bc060ce15c02a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:47,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d44575bc060ce15c02a 2025-12-17 21:28:47,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d44575bc060ce15c02a HTTP/1.1" 502 150 2025-12-17 21:28:47,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d44575bc060ce15c02a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:48,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2c575bc060ce15bfc4 HTTP/1.1" 502 150 2025-12-17 21:28:48,048 - lava_callback - ERROR - Error processing callback for node 69407d2c575bc060ce15bfc4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d2c575bc060ce15bfc4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d2c575bc060ce15bfc4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d2c575bc060ce15bfc4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:48,050 - lava_callback - INFO - Starting async job submit for node 69407d45575bc060ce15c02d 2025-12-17 21:28:48,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:48,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d45575bc060ce15c02d HTTP/1.1" 502 150 2025-12-17 21:28:48,057 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d45575bc060ce15c02d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:48,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d45575bc060ce15c02d 2025-12-17 21:28:48,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d45575bc060ce15c02d HTTP/1.1" 502 150 2025-12-17 21:28:48,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d45575bc060ce15c02d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:48,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d30575bc060ce15bfcd 2025-12-17 21:28:48,369 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d38575bc060ce15bfd2 2025-12-17 21:28:48,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d30575bc060ce15bfcd HTTP/1.1" 502 150 2025-12-17 21:28:48,370 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d30575bc060ce15bfcd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:48,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d38575bc060ce15bfd2 HTTP/1.1" 502 150 2025-12-17 21:28:48,370 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d38575bc060ce15bfd2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:49,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d43575bc060ce15c027 2025-12-17 21:28:49,136 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4c575bc060ce15c04b 2025-12-17 21:28:49,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4c575bc060ce15c04b HTTP/1.1" 502 150 2025-12-17 21:28:49,137 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4c575bc060ce15c04b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:49,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d44575bc060ce15c02a 2025-12-17 21:28:49,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d44575bc060ce15c02a HTTP/1.1" 502 150 2025-12-17 21:28:49,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d44575bc060ce15c02a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:49,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078eb575bc060ce15b5fc 2025-12-17 21:28:49,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078eb575bc060ce15b5fc HTTP/1.1" 502 150 2025-12-17 21:28:49,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694078eb575bc060ce15b5fc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:50,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d45575bc060ce15c02d 2025-12-17 21:28:50,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d45575bc060ce15c02d HTTP/1.1" 502 150 2025-12-17 21:28:50,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d45575bc060ce15c02d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:50,160 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c36575bc060ce15bd0b 2025-12-17 21:28:50,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c36575bc060ce15bd0b HTTP/1.1" 502 150 2025-12-17 21:28:50,162 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c36575bc060ce15bd0b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:51,246 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d42575bc060ce15c024 2025-12-17 21:28:51,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d42575bc060ce15c024 HTTP/1.1" 502 150 2025-12-17 21:28:51,248 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d42575bc060ce15c024'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:51,281 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d40575bc060ce15c021 2025-12-17 21:28:51,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d40575bc060ce15c021 HTTP/1.1" 502 150 2025-12-17 21:28:51,283 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d40575bc060ce15c021'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:51,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3a575bc060ce15bfd9 2025-12-17 21:28:51,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3a575bc060ce15bfd9 HTTP/1.1" 502 150 2025-12-17 21:28:51,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3a575bc060ce15bfd9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:51,961 - lava_callback - INFO - Received callback for node 69407d76575bc060ce15c0ba INFO: 172.21.0.1:36370 - "POST /node/69407d76575bc060ce15c0ba HTTP/1.0" 202 Accepted 2025-12-17 21:28:52,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3b575bc060ce15bfe1 2025-12-17 21:28:52,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3b575bc060ce15bfe1 HTTP/1.1" 502 150 2025-12-17 21:28:52,116 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3b575bc060ce15bfe1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:53,120 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3c575bc060ce15bfe9 2025-12-17 21:28:53,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3c575bc060ce15bfe9 HTTP/1.1" 502 150 2025-12-17 21:28:53,122 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3c575bc060ce15bfe9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:53,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d44575bc060ce15c02a 2025-12-17 21:28:53,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d44575bc060ce15c02a HTTP/1.1" 502 150 2025-12-17 21:28:53,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d44575bc060ce15c02a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:53,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3f575bc060ce15c01c 2025-12-17 21:28:53,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3f575bc060ce15c01c HTTP/1.1" 502 150 2025-12-17 21:28:53,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d3f575bc060ce15c01c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:54,059 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d45575bc060ce15c02d 2025-12-17 21:28:54,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d45575bc060ce15c02d HTTP/1.1" 502 150 2025-12-17 21:28:54,061 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d45575bc060ce15c02d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36386 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:28:58,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2e575bc060ce15bfc7 2025-12-17 21:28:58,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2e575bc060ce15bfc7 HTTP/1.1" 502 150 2025-12-17 21:28:58,419 - lava_callback - ERROR - Error processing callback for node 69407d2e575bc060ce15bfc7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d2e575bc060ce15bfc7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d2e575bc060ce15bfc7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d2e575bc060ce15bfc7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:58,421 - lava_callback - INFO - Starting async job submit for node 69407d4a575bc060ce15c042 2025-12-17 21:28:58,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:58,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4a575bc060ce15c042 HTTP/1.1" 502 150 2025-12-17 21:28:58,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4a575bc060ce15c042'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:58,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4a575bc060ce15c042 2025-12-17 21:28:58,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4a575bc060ce15c042 HTTP/1.1" 502 150 2025-12-17 21:28:58,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4a575bc060ce15c042'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:59,185 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d2f575bc060ce15bfca 2025-12-17 21:28:59,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d2f575bc060ce15bfca HTTP/1.1" 502 150 2025-12-17 21:28:59,186 - lava_callback - ERROR - Error processing callback for node 69407d2f575bc060ce15bfca: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d2f575bc060ce15bfca (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d2f575bc060ce15bfca (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d2f575bc060ce15bfca (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:28:59,188 - lava_callback - INFO - Starting async job submit for node 69407ee8575bc060ce15c384 2025-12-17 21:28:59,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:28:59,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ee8575bc060ce15c384 HTTP/1.1" 502 150 2025-12-17 21:28:59,226 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ee8575bc060ce15c384'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:59,226 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ee8575bc060ce15c384 2025-12-17 21:28:59,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ee8575bc060ce15c384 HTTP/1.1" 502 150 2025-12-17 21:28:59,226 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ee8575bc060ce15c384'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:59,248 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d42575bc060ce15c024 2025-12-17 21:28:59,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d42575bc060ce15c024 HTTP/1.1" 502 150 2025-12-17 21:28:59,249 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d42575bc060ce15c024'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:28:59,283 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d40575bc060ce15c021 2025-12-17 21:28:59,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d40575bc060ce15c021 HTTP/1.1" 502 150 2025-12-17 21:28:59,286 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d40575bc060ce15c021'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:00,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4a575bc060ce15c042 2025-12-17 21:29:00,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4a575bc060ce15c042 HTTP/1.1" 502 150 2025-12-17 21:29:00,437 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4a575bc060ce15c042'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:01,227 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ee8575bc060ce15c384 INFO: 172.21.0.1:48276 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:29:01,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d44575bc060ce15c02a 2025-12-17 21:29:01,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d44575bc060ce15c02a HTTP/1.1" 502 150 2025-12-17 21:29:01,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d44575bc060ce15c02a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:02,061 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d45575bc060ce15c02d 2025-12-17 21:29:02,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d45575bc060ce15c02d HTTP/1.1" 502 150 2025-12-17 21:29:02,062 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d45575bc060ce15c02d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:04,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d30575bc060ce15bfcd 2025-12-17 21:29:04,371 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d38575bc060ce15bfd2 2025-12-17 21:29:04,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d30575bc060ce15bfcd HTTP/1.1" 502 150 2025-12-17 21:29:04,372 - lava_callback - ERROR - Error processing callback for node 69407d30575bc060ce15bfcd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d30575bc060ce15bfcd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d30575bc060ce15bfcd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d30575bc060ce15bfcd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:04,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d38575bc060ce15bfd2 HTTP/1.1" 502 150 2025-12-17 21:29:04,374 - lava_callback - INFO - Starting async job submit for node 69407eea575bc060ce15c387 2025-12-17 21:29:04,374 - lava_callback - ERROR - Error processing callback for node 69407d38575bc060ce15bfd2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d38575bc060ce15bfd2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d38575bc060ce15bfd2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d38575bc060ce15bfd2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:04,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:04,393 - lava_callback - INFO - Starting async job submit for node 69407c4e575bc060ce15bd97 2025-12-17 21:29:04,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:04,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eea575bc060ce15c387 HTTP/1.1" 502 150 2025-12-17 21:29:04,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eea575bc060ce15c387'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:04,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eea575bc060ce15c387 2025-12-17 21:29:04,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eea575bc060ce15c387 HTTP/1.1" 502 150 2025-12-17 21:29:04,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eea575bc060ce15c387'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:04,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4e575bc060ce15bd97 HTTP/1.1" 502 150 2025-12-17 21:29:04,425 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4e575bc060ce15bd97'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:04,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4e575bc060ce15bd97 2025-12-17 21:29:04,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4e575bc060ce15bd97 HTTP/1.1" 502 150 2025-12-17 21:29:04,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4e575bc060ce15bd97'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:04,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d43575bc060ce15c027 HTTP/1.1" 502 150 2025-12-17 21:29:04,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d43575bc060ce15c027'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:04,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4a575bc060ce15c042 2025-12-17 21:29:04,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4a575bc060ce15c042 HTTP/1.1" 502 150 2025-12-17 21:29:04,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4a575bc060ce15c042'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:05,138 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4c575bc060ce15c04b 2025-12-17 21:29:05,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4c575bc060ce15c04b HTTP/1.1" 502 150 2025-12-17 21:29:05,139 - lava_callback - ERROR - Error processing callback for node 69407d4c575bc060ce15c04b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4c575bc060ce15c04b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4c575bc060ce15c04b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4c575bc060ce15c04b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:05,141 - lava_callback - INFO - Starting async job submit for node 69407d4d575bc060ce15c052 2025-12-17 21:29:05,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:05,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4d575bc060ce15c052 HTTP/1.1" 502 150 2025-12-17 21:29:05,152 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4d575bc060ce15c052'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:05,153 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4d575bc060ce15c052 2025-12-17 21:29:05,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4d575bc060ce15c052 HTTP/1.1" 502 150 2025-12-17 21:29:05,153 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4d575bc060ce15c052'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:05,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694078eb575bc060ce15b5fc 2025-12-17 21:29:05,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694078eb575bc060ce15b5fc HTTP/1.1" 502 150 2025-12-17 21:29:05,907 - lava_callback - ERROR - Error processing callback for node 694078eb575bc060ce15b5fc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078eb575bc060ce15b5fc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078eb575bc060ce15b5fc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694078eb575bc060ce15b5fc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:05,909 - lava_callback - INFO - Starting async job submit for node 69407d4e575bc060ce15c05a 2025-12-17 21:29:05,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:05,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4e575bc060ce15c05a HTTP/1.1" 502 150 2025-12-17 21:29:05,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4e575bc060ce15c05a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:05,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4e575bc060ce15c05a 2025-12-17 21:29:05,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4e575bc060ce15c05a HTTP/1.1" 502 150 2025-12-17 21:29:05,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4e575bc060ce15c05a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:06,162 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c36575bc060ce15bd0b 2025-12-17 21:29:06,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c36575bc060ce15bd0b HTTP/1.1" 502 150 2025-12-17 21:29:06,163 - lava_callback - ERROR - Error processing callback for node 69407c36575bc060ce15bd0b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c36575bc060ce15bd0b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c36575bc060ce15bd0b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c36575bc060ce15bd0b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:06,165 - lava_callback - INFO - Starting async job submit for node 69407d4f575bc060ce15c062 2025-12-17 21:29:06,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:06,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4f575bc060ce15c062 HTTP/1.1" 502 150 2025-12-17 21:29:06,197 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4f575bc060ce15c062'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:06,197 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4f575bc060ce15c062 2025-12-17 21:29:06,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4f575bc060ce15c062 HTTP/1.1" 502 150 2025-12-17 21:29:06,198 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4f575bc060ce15c062'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:06,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eea575bc060ce15c387 2025-12-17 21:29:06,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eea575bc060ce15c387 HTTP/1.1" 502 150 2025-12-17 21:29:06,425 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eea575bc060ce15c387'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:06,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4e575bc060ce15bd97 2025-12-17 21:29:06,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4e575bc060ce15bd97 HTTP/1.1" 502 150 2025-12-17 21:29:06,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4e575bc060ce15bd97'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48286 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:29:07,153 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4d575bc060ce15c052 2025-12-17 21:29:07,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4d575bc060ce15c052 HTTP/1.1" 502 150 2025-12-17 21:29:07,155 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4d575bc060ce15c052'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:07,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3a575bc060ce15bfd9 2025-12-17 21:29:07,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3a575bc060ce15bfd9 HTTP/1.1" 502 150 2025-12-17 21:29:07,606 - lava_callback - ERROR - Error processing callback for node 69407d3a575bc060ce15bfd9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d3a575bc060ce15bfd9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d3a575bc060ce15bfd9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d3a575bc060ce15bfd9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:07,608 - lava_callback - INFO - Starting async job submit for node 69407d51575bc060ce15c069 2025-12-17 21:29:07,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:07,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d51575bc060ce15c069 HTTP/1.1" 502 150 2025-12-17 21:29:07,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d51575bc060ce15c069'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:07,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d51575bc060ce15c069 2025-12-17 21:29:07,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d51575bc060ce15c069 HTTP/1.1" 502 150 2025-12-17 21:29:07,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d51575bc060ce15c069'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:07,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4e575bc060ce15c05a 2025-12-17 21:29:07,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4e575bc060ce15c05a HTTP/1.1" 502 150 2025-12-17 21:29:07,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4e575bc060ce15c05a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:08,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3b575bc060ce15bfe1 2025-12-17 21:29:08,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3b575bc060ce15bfe1 HTTP/1.1" 502 150 2025-12-17 21:29:08,118 - lava_callback - ERROR - Error processing callback for node 69407d3b575bc060ce15bfe1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d3b575bc060ce15bfe1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d3b575bc060ce15bfe1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d3b575bc060ce15bfe1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:08,120 - lava_callback - INFO - Starting async job submit for node 69407d52575bc060ce15c06c 2025-12-17 21:29:08,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:08,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d52575bc060ce15c06c HTTP/1.1" 502 150 2025-12-17 21:29:08,126 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d52575bc060ce15c06c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:08,127 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d52575bc060ce15c06c 2025-12-17 21:29:08,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d52575bc060ce15c06c HTTP/1.1" 502 150 2025-12-17 21:29:08,127 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d52575bc060ce15c06c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:08,198 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4f575bc060ce15c062 2025-12-17 21:29:08,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4f575bc060ce15c062 HTTP/1.1" 502 150 2025-12-17 21:29:08,199 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4f575bc060ce15c062'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:08,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d43575bc060ce15c027 2025-12-17 21:29:08,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d43575bc060ce15c027 HTTP/1.1" 502 150 2025-12-17 21:29:08,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d43575bc060ce15c027'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:09,122 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3c575bc060ce15bfe9 2025-12-17 21:29:09,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3c575bc060ce15bfe9 HTTP/1.1" 502 150 2025-12-17 21:29:09,123 - lava_callback - ERROR - Error processing callback for node 69407d3c575bc060ce15bfe9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d3c575bc060ce15bfe9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d3c575bc060ce15bfe9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d3c575bc060ce15bfe9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:09,126 - lava_callback - INFO - Starting async job submit for node 69407d53575bc060ce15c06f 2025-12-17 21:29:09,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:09,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d53575bc060ce15c06f HTTP/1.1" 502 150 2025-12-17 21:29:09,133 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d53575bc060ce15c06f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:09,133 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d53575bc060ce15c06f 2025-12-17 21:29:09,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d53575bc060ce15c06f HTTP/1.1" 502 150 2025-12-17 21:29:09,134 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d53575bc060ce15c06f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:09,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d51575bc060ce15c069 2025-12-17 21:29:09,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d51575bc060ce15c069 HTTP/1.1" 502 150 2025-12-17 21:29:09,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d51575bc060ce15c069'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:09,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d3f575bc060ce15c01c 2025-12-17 21:29:09,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d3f575bc060ce15c01c HTTP/1.1" 502 150 2025-12-17 21:29:09,871 - lava_callback - ERROR - Error processing callback for node 69407d3f575bc060ce15c01c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d3f575bc060ce15c01c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d3f575bc060ce15c01c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d3f575bc060ce15c01c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:09,873 - lava_callback - INFO - Starting async job submit for node 69407d54575bc060ce15c072 2025-12-17 21:29:09,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:09,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d54575bc060ce15c072 HTTP/1.1" 502 150 2025-12-17 21:29:09,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d54575bc060ce15c072'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:09,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d54575bc060ce15c072 2025-12-17 21:29:09,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d54575bc060ce15c072 HTTP/1.1" 502 150 2025-12-17 21:29:09,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d54575bc060ce15c072'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:10,128 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d52575bc060ce15c06c 2025-12-17 21:29:10,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d52575bc060ce15c06c HTTP/1.1" 502 150 2025-12-17 21:29:10,129 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d52575bc060ce15c06c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:10,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eea575bc060ce15c387 2025-12-17 21:29:10,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eea575bc060ce15c387 HTTP/1.1" 502 150 2025-12-17 21:29:10,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eea575bc060ce15c387'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:10,427 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4e575bc060ce15bd97 2025-12-17 21:29:10,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4e575bc060ce15bd97 HTTP/1.1" 502 150 2025-12-17 21:29:10,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4e575bc060ce15bd97'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:11,134 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d53575bc060ce15c06f 2025-12-17 21:29:11,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d53575bc060ce15c06f HTTP/1.1" 502 150 2025-12-17 21:29:11,135 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d53575bc060ce15c06f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:11,155 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4d575bc060ce15c052 2025-12-17 21:29:11,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4d575bc060ce15c052 HTTP/1.1" 502 150 2025-12-17 21:29:11,156 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4d575bc060ce15c052'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:11,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d54575bc060ce15c072 2025-12-17 21:29:11,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d54575bc060ce15c072 HTTP/1.1" 502 150 2025-12-17 21:29:11,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d54575bc060ce15c072'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:11,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4e575bc060ce15c05a 2025-12-17 21:29:11,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4e575bc060ce15c05a HTTP/1.1" 502 150 2025-12-17 21:29:11,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4e575bc060ce15c05a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:12,199 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4f575bc060ce15c062 2025-12-17 21:29:12,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4a575bc060ce15c042 2025-12-17 21:29:12,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4a575bc060ce15c042 HTTP/1.1" 502 150 2025-12-17 21:29:12,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4a575bc060ce15c042'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:12,542 - lava_callback - INFO - Received callback for node 69407d9e575bc060ce15c13b INFO: 172.21.0.1:60192 - "POST /node/69407d9e575bc060ce15c13b HTTP/1.0" 202 Accepted 2025-12-17 21:29:13,218 - lava_callback - INFO - Received callback for node 69407da0575bc060ce15c146 INFO: 172.21.0.1:60204 - "POST /node/69407da0575bc060ce15c146 HTTP/1.0" 202 Accepted 2025-12-17 21:29:13,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d51575bc060ce15c069 2025-12-17 21:29:13,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d51575bc060ce15c069 HTTP/1.1" 502 150 2025-12-17 21:29:13,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d51575bc060ce15c069'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:13,867 - lava_callback - INFO - Received callback for node 69407da2575bc060ce15c14d INFO: 172.21.0.1:60212 - "POST /node/69407da2575bc060ce15c14d HTTP/1.0" 202 Accepted 2025-12-17 21:29:14,129 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d52575bc060ce15c06c 2025-12-17 21:29:14,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d52575bc060ce15c06c HTTP/1.1" 502 150 2025-12-17 21:29:14,130 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d52575bc060ce15c06c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:14,550 - lava_callback - INFO - Received callback for node 69407da3575bc060ce15c154 INFO: 172.21.0.1:60228 - "POST /node/69407da3575bc060ce15c154 HTTP/1.0" 202 Accepted 2025-12-17 21:29:15,135 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d53575bc060ce15c06f INFO: 172.21.0.1:60244 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:29:15,222 - lava_callback - INFO - Received callback for node 69407da4575bc060ce15c157 INFO: 172.21.0.1:60256 - "POST /node/69407da4575bc060ce15c157 HTTP/1.0" 202 Accepted 2025-12-17 21:29:15,249 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d42575bc060ce15c024 2025-12-17 21:29:15,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d42575bc060ce15c024 HTTP/1.1" 502 150 2025-12-17 21:29:15,251 - lava_callback - ERROR - Error processing callback for node 69407d42575bc060ce15c024: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d42575bc060ce15c024 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d42575bc060ce15c024 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d42575bc060ce15c024 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:15,252 - lava_callback - INFO - Starting async job submit for node 69407d60575bc060ce15c077 2025-12-17 21:29:15,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:15,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d60575bc060ce15c077 HTTP/1.1" 502 150 2025-12-17 21:29:15,263 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d60575bc060ce15c077'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:15,263 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d60575bc060ce15c077 2025-12-17 21:29:15,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d60575bc060ce15c077 HTTP/1.1" 502 150 2025-12-17 21:29:15,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d60575bc060ce15c077'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:15,286 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d40575bc060ce15c021 2025-12-17 21:29:15,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d40575bc060ce15c021 HTTP/1.1" 502 150 2025-12-17 21:29:15,287 - lava_callback - ERROR - Error processing callback for node 69407d40575bc060ce15c021: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d40575bc060ce15c021 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d40575bc060ce15c021 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d40575bc060ce15c021 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:15,289 - lava_callback - INFO - Starting async job submit for node 69407d75575bc060ce15c0b5 2025-12-17 21:29:15,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:15,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d75575bc060ce15c0b5 HTTP/1.1" 502 150 2025-12-17 21:29:15,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d75575bc060ce15c0b5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:15,313 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d75575bc060ce15c0b5 2025-12-17 21:29:15,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d75575bc060ce15c0b5 HTTP/1.1" 502 150 2025-12-17 21:29:15,314 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d75575bc060ce15c0b5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:15,872 - lava_callback - INFO - Received callback for node 69407da6575bc060ce15c15a INFO: 172.21.0.1:60264 - "POST /node/69407da6575bc060ce15c15a HTTP/1.0" 202 Accepted 2025-12-17 21:29:15,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d54575bc060ce15c072 2025-12-17 21:29:15,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d54575bc060ce15c072 HTTP/1.1" 502 150 2025-12-17 21:29:15,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d54575bc060ce15c072'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:16,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d43575bc060ce15c027 2025-12-17 21:29:16,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d43575bc060ce15c027 HTTP/1.1" 502 150 2025-12-17 21:29:16,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d43575bc060ce15c027'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60278 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:29:16,531 - lava_callback - INFO - Received callback for node 69407da7575bc060ce15c15d INFO: 172.21.0.1:60284 - "POST /node/69407da7575bc060ce15c15d HTTP/1.0" 202 Accepted 2025-12-17 21:29:16,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ee8575bc060ce15c384 HTTP/1.1" 502 150 2025-12-17 21:29:16,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ee8575bc060ce15c384'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:17,264 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d60575bc060ce15c077 2025-12-17 21:29:17,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d60575bc060ce15c077 HTTP/1.1" 502 150 2025-12-17 21:29:17,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d60575bc060ce15c077'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:17,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d75575bc060ce15c0b5 2025-12-17 21:29:17,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d75575bc060ce15c0b5 HTTP/1.1" 502 150 2025-12-17 21:29:17,315 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d75575bc060ce15c0b5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:17,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d44575bc060ce15c02a 2025-12-17 21:29:17,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d44575bc060ce15c02a HTTP/1.1" 502 150 2025-12-17 21:29:17,820 - lava_callback - ERROR - Error processing callback for node 69407d44575bc060ce15c02a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d44575bc060ce15c02a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d44575bc060ce15c02a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d44575bc060ce15c02a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:17,821 - lava_callback - INFO - Starting async job submit for node 69407d63575bc060ce15c085 2025-12-17 21:29:17,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:17,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d63575bc060ce15c085 HTTP/1.1" 502 150 2025-12-17 21:29:17,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d63575bc060ce15c085'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:17,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d63575bc060ce15c085 2025-12-17 21:29:17,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d63575bc060ce15c085 HTTP/1.1" 502 150 2025-12-17 21:29:17,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d63575bc060ce15c085'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:18,062 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d45575bc060ce15c02d 2025-12-17 21:29:18,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d45575bc060ce15c02d HTTP/1.1" 502 150 2025-12-17 21:29:18,064 - lava_callback - ERROR - Error processing callback for node 69407d45575bc060ce15c02d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d45575bc060ce15c02d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d45575bc060ce15c02d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d45575bc060ce15c02d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:18,065 - lava_callback - INFO - Starting async job submit for node 69407d64575bc060ce15c08c 2025-12-17 21:29:18,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:18,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d64575bc060ce15c08c HTTP/1.1" 502 150 2025-12-17 21:29:18,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d64575bc060ce15c08c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:18,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d64575bc060ce15c08c 2025-12-17 21:29:18,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d64575bc060ce15c08c HTTP/1.1" 502 150 2025-12-17 21:29:18,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d64575bc060ce15c08c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:18,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eea575bc060ce15c387 2025-12-17 21:29:18,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eea575bc060ce15c387 HTTP/1.1" 502 150 2025-12-17 21:29:18,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4e575bc060ce15bd97 2025-12-17 21:29:18,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eea575bc060ce15c387'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:18,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4e575bc060ce15bd97 HTTP/1.1" 502 150 2025-12-17 21:29:18,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407c4e575bc060ce15bd97'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:19,156 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4d575bc060ce15c052 2025-12-17 21:29:19,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4d575bc060ce15c052 HTTP/1.1" 502 150 2025-12-17 21:29:19,157 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4d575bc060ce15c052'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:19,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d63575bc060ce15c085 2025-12-17 21:29:19,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d63575bc060ce15c085 HTTP/1.1" 502 150 2025-12-17 21:29:19,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d63575bc060ce15c085'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:19,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4e575bc060ce15c05a 2025-12-17 21:29:19,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4e575bc060ce15c05a HTTP/1.1" 502 150 2025-12-17 21:29:19,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4e575bc060ce15c05a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:20,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d64575bc060ce15c08c 2025-12-17 21:29:20,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d64575bc060ce15c08c HTTP/1.1" 502 150 2025-12-17 21:29:20,074 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d64575bc060ce15c08c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:20,608 - lava_callback - INFO - Received callback for node 69407b11575bc060ce15b936 INFO: 172.21.0.1:57864 - "POST /node/69407b11575bc060ce15b936 HTTP/1.0" 202 Accepted 2025-12-17 21:29:20,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ee8575bc060ce15c384 2025-12-17 21:29:20,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ee8575bc060ce15c384 HTTP/1.1" 502 150 2025-12-17 21:29:20,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ee8575bc060ce15c384'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:21,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d60575bc060ce15c077 2025-12-17 21:29:21,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d60575bc060ce15c077 HTTP/1.1" 502 150 2025-12-17 21:29:21,267 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d60575bc060ce15c077'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:21,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d75575bc060ce15c0b5 2025-12-17 21:29:21,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d75575bc060ce15c0b5 HTTP/1.1" 502 150 2025-12-17 21:29:21,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d75575bc060ce15c0b5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:21,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d51575bc060ce15c069 2025-12-17 21:29:21,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d51575bc060ce15c069 HTTP/1.1" 502 150 2025-12-17 21:29:21,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d51575bc060ce15c069'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:22,130 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d52575bc060ce15c06c 2025-12-17 21:29:22,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d52575bc060ce15c06c HTTP/1.1" 502 150 2025-12-17 21:29:22,131 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d52575bc060ce15c06c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:23,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d63575bc060ce15c085 2025-12-17 21:29:23,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d63575bc060ce15c085 HTTP/1.1" 502 150 2025-12-17 21:29:23,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d63575bc060ce15c085'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:23,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d54575bc060ce15c072 2025-12-17 21:29:23,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d54575bc060ce15c072 HTTP/1.1" 502 150 2025-12-17 21:29:23,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d54575bc060ce15c072'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:24,074 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d64575bc060ce15c08c 2025-12-17 21:29:24,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d64575bc060ce15c08c HTTP/1.1" 502 150 2025-12-17 21:29:24,076 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d64575bc060ce15c08c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57868 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:29:27,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4f575bc060ce15c062 HTTP/1.1" 502 150 2025-12-17 21:29:27,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4f575bc060ce15c062'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:28,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4a575bc060ce15c042 2025-12-17 21:29:28,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4a575bc060ce15c042 HTTP/1.1" 502 150 2025-12-17 21:29:28,441 - lava_callback - ERROR - Error processing callback for node 69407d4a575bc060ce15c042: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4a575bc060ce15c042 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4a575bc060ce15c042 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4a575bc060ce15c042 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:28,443 - lava_callback - INFO - Starting async job submit for node 69407eee575bc060ce15c39d 2025-12-17 21:29:28,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:28,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eee575bc060ce15c39d HTTP/1.1" 502 150 2025-12-17 21:29:28,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eee575bc060ce15c39d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:28,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eee575bc060ce15c39d 2025-12-17 21:29:28,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eee575bc060ce15c39d HTTP/1.1" 502 150 2025-12-17 21:29:28,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eee575bc060ce15c39d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:28,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ee8575bc060ce15c384 2025-12-17 21:29:28,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ee8575bc060ce15c384 HTTP/1.1" 502 150 2025-12-17 21:29:28,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ee8575bc060ce15c384'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:29,267 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d60575bc060ce15c077 2025-12-17 21:29:29,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d60575bc060ce15c077 HTTP/1.1" 502 150 2025-12-17 21:29:29,269 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d60575bc060ce15c077'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:29,317 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d75575bc060ce15c0b5 2025-12-17 21:29:29,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d75575bc060ce15c0b5 HTTP/1.1" 502 150 2025-12-17 21:29:29,318 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d75575bc060ce15c0b5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:30,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eee575bc060ce15c39d 2025-12-17 21:29:30,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eee575bc060ce15c39d HTTP/1.1" 502 150 2025-12-17 21:29:30,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eee575bc060ce15c39d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:30,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d53575bc060ce15c06f HTTP/1.1" 502 150 2025-12-17 21:29:30,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d53575bc060ce15c06f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:31,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d63575bc060ce15c085 2025-12-17 21:29:31,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d63575bc060ce15c085 HTTP/1.1" 502 150 2025-12-17 21:29:31,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d63575bc060ce15c085'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:32,076 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d64575bc060ce15c08c 2025-12-17 21:29:32,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d64575bc060ce15c08c HTTP/1.1" 502 150 2025-12-17 21:29:32,077 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d64575bc060ce15c08c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:32,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d43575bc060ce15c027 2025-12-17 21:29:32,435 - lava_callback - INFO - Received callback for node 69407da8575bc060ce15c160 INFO: 172.21.0.1:44248 - "POST /node/69407da8575bc060ce15c160 HTTP/1.0" 202 Accepted 2025-12-17 21:29:32,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d43575bc060ce15c027 HTTP/1.1" 502 150 2025-12-17 21:29:32,442 - lava_callback - ERROR - Error processing callback for node 69407d43575bc060ce15c027: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d43575bc060ce15c027 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d43575bc060ce15c027 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d43575bc060ce15c027 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:32,443 - lava_callback - INFO - Starting async job submit for node 69407d65575bc060ce15c093 2025-12-17 21:29:32,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:32,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d65575bc060ce15c093 HTTP/1.1" 502 150 2025-12-17 21:29:32,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d65575bc060ce15c093'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:32,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d65575bc060ce15c093 2025-12-17 21:29:32,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d65575bc060ce15c093 HTTP/1.1" 502 150 2025-12-17 21:29:32,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d65575bc060ce15c093'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:33,105 - lava_callback - INFO - Received callback for node 69407da9575bc060ce15c163 INFO: 172.21.0.1:44254 - "POST /node/69407da9575bc060ce15c163 HTTP/1.0" 202 Accepted 2025-12-17 21:29:34,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eea575bc060ce15c387 2025-12-17 21:29:34,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407c4e575bc060ce15bd97 2025-12-17 21:29:34,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eea575bc060ce15c387 HTTP/1.1" 502 150 2025-12-17 21:29:34,430 - lava_callback - ERROR - Error processing callback for node 69407eea575bc060ce15c387: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eea575bc060ce15c387 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eea575bc060ce15c387 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eea575bc060ce15c387 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:34,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407c4e575bc060ce15bd97 HTTP/1.1" 502 150 2025-12-17 21:29:34,432 - lava_callback - INFO - Starting async job submit for node 69407d67575bc060ce15c09a 2025-12-17 21:29:34,432 - lava_callback - ERROR - Error processing callback for node 69407c4e575bc060ce15bd97: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c4e575bc060ce15bd97 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c4e575bc060ce15bd97 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407c4e575bc060ce15bd97 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:34,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:34,434 - lava_callback - INFO - Starting async job submit for node 69407d68575bc060ce15c09f 2025-12-17 21:29:34,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:34,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d68575bc060ce15c09f HTTP/1.1" 502 150 2025-12-17 21:29:34,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d68575bc060ce15c09f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:34,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d68575bc060ce15c09f 2025-12-17 21:29:34,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d68575bc060ce15c09f HTTP/1.1" 502 150 2025-12-17 21:29:34,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d68575bc060ce15c09f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:34,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d67575bc060ce15c09a HTTP/1.1" 502 150 2025-12-17 21:29:34,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d67575bc060ce15c09a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:34,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d67575bc060ce15c09a 2025-12-17 21:29:34,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d67575bc060ce15c09a HTTP/1.1" 502 150 2025-12-17 21:29:34,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d67575bc060ce15c09a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:34,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d65575bc060ce15c093 2025-12-17 21:29:34,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d65575bc060ce15c093 HTTP/1.1" 502 150 2025-12-17 21:29:34,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d65575bc060ce15c093'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:34,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eee575bc060ce15c39d 2025-12-17 21:29:34,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eee575bc060ce15c39d HTTP/1.1" 502 150 2025-12-17 21:29:34,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eee575bc060ce15c39d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:35,157 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4d575bc060ce15c052 2025-12-17 21:29:35,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4d575bc060ce15c052 HTTP/1.1" 502 150 2025-12-17 21:29:35,159 - lava_callback - ERROR - Error processing callback for node 69407d4d575bc060ce15c052: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4d575bc060ce15c052 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4d575bc060ce15c052 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4d575bc060ce15c052 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:35,161 - lava_callback - INFO - Starting async job submit for node 69407d69575bc060ce15c0a2 2025-12-17 21:29:35,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:35,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d69575bc060ce15c0a2 HTTP/1.1" 502 150 2025-12-17 21:29:35,167 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d69575bc060ce15c0a2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:35,167 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d69575bc060ce15c0a2 2025-12-17 21:29:35,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d69575bc060ce15c0a2 HTTP/1.1" 502 150 2025-12-17 21:29:35,168 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d69575bc060ce15c0a2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:35,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4f575bc060ce15c062 2025-12-17 21:29:35,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4f575bc060ce15c062 HTTP/1.1" 502 150 2025-12-17 21:29:35,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d4f575bc060ce15c062'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:35,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4e575bc060ce15c05a 2025-12-17 21:29:35,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4e575bc060ce15c05a HTTP/1.1" 502 150 2025-12-17 21:29:35,921 - lava_callback - ERROR - Error processing callback for node 69407d4e575bc060ce15c05a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4e575bc060ce15c05a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4e575bc060ce15c05a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4e575bc060ce15c05a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:35,923 - lava_callback - INFO - Starting async job submit for node 69407d61575bc060ce15c080 2025-12-17 21:29:35,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:35,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d61575bc060ce15c080 HTTP/1.1" 502 150 2025-12-17 21:29:35,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d61575bc060ce15c080'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:35,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d61575bc060ce15c080 2025-12-17 21:29:35,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d61575bc060ce15c080 HTTP/1.1" 502 150 2025-12-17 21:29:35,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d61575bc060ce15c080'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:36,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d68575bc060ce15c09f 2025-12-17 21:29:36,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d68575bc060ce15c09f HTTP/1.1" 502 150 2025-12-17 21:29:36,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d68575bc060ce15c09f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:36,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d67575bc060ce15c09a 2025-12-17 21:29:36,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d67575bc060ce15c09a HTTP/1.1" 502 150 2025-12-17 21:29:36,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d67575bc060ce15c09a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44262 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:29:37,168 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d69575bc060ce15c0a2 2025-12-17 21:29:37,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d69575bc060ce15c0a2 HTTP/1.1" 502 150 2025-12-17 21:29:37,169 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d69575bc060ce15c0a2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:37,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d51575bc060ce15c069 2025-12-17 21:29:37,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d51575bc060ce15c069 HTTP/1.1" 502 150 2025-12-17 21:29:37,622 - lava_callback - ERROR - Error processing callback for node 69407d51575bc060ce15c069: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d51575bc060ce15c069 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d51575bc060ce15c069 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d51575bc060ce15c069 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:37,623 - lava_callback - INFO - Starting async job submit for node 69407d6b575bc060ce15c0a5 2025-12-17 21:29:37,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:37,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6b575bc060ce15c0a5 HTTP/1.1" 502 150 2025-12-17 21:29:37,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d6b575bc060ce15c0a5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:37,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d6b575bc060ce15c0a5 2025-12-17 21:29:37,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6b575bc060ce15c0a5 HTTP/1.1" 502 150 2025-12-17 21:29:37,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d6b575bc060ce15c0a5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:37,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d61575bc060ce15c080 2025-12-17 21:29:37,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d61575bc060ce15c080 HTTP/1.1" 502 150 2025-12-17 21:29:37,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d61575bc060ce15c080'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:38,132 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d52575bc060ce15c06c 2025-12-17 21:29:38,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d65575bc060ce15c093 2025-12-17 21:29:38,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d65575bc060ce15c093 HTTP/1.1" 502 150 2025-12-17 21:29:38,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d65575bc060ce15c093'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:38,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d53575bc060ce15c06f 2025-12-17 21:29:38,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d53575bc060ce15c06f HTTP/1.1" 502 150 2025-12-17 21:29:38,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d53575bc060ce15c06f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:39,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d6b575bc060ce15c0a5 2025-12-17 21:29:39,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6b575bc060ce15c0a5 HTTP/1.1" 502 150 2025-12-17 21:29:39,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d6b575bc060ce15c0a5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:39,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d54575bc060ce15c072 2025-12-17 21:29:39,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d54575bc060ce15c072 HTTP/1.1" 502 150 2025-12-17 21:29:39,885 - lava_callback - ERROR - Error processing callback for node 69407d54575bc060ce15c072: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d54575bc060ce15c072 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d54575bc060ce15c072 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d54575bc060ce15c072 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:39,887 - lava_callback - INFO - Starting async job submit for node 694079a1575bc060ce15b721 2025-12-17 21:29:39,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:39,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694079a1575bc060ce15b721 HTTP/1.1" 502 150 2025-12-17 21:29:39,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694079a1575bc060ce15b721'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:39,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694079a1575bc060ce15b721 2025-12-17 21:29:39,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694079a1575bc060ce15b721 HTTP/1.1" 502 150 2025-12-17 21:29:39,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694079a1575bc060ce15b721'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:40,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d68575bc060ce15c09f 2025-12-17 21:29:40,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d68575bc060ce15c09f HTTP/1.1" 502 150 2025-12-17 21:29:40,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d68575bc060ce15c09f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:40,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d67575bc060ce15c09a 2025-12-17 21:29:40,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d67575bc060ce15c09a HTTP/1.1" 502 150 2025-12-17 21:29:40,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d67575bc060ce15c09a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:41,169 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d69575bc060ce15c0a2 2025-12-17 21:29:41,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694079a1575bc060ce15b721 2025-12-17 21:29:41,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694079a1575bc060ce15b721 HTTP/1.1" 502 150 2025-12-17 21:29:41,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694079a1575bc060ce15b721'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:41,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d61575bc060ce15c080 2025-12-17 21:29:41,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d61575bc060ce15c080 HTTP/1.1" 502 150 2025-12-17 21:29:41,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d61575bc060ce15c080'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:42,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eee575bc060ce15c39d 2025-12-17 21:29:42,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eee575bc060ce15c39d HTTP/1.1" 502 150 2025-12-17 21:29:42,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407eee575bc060ce15c39d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:43,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d6b575bc060ce15c0a5 2025-12-17 21:29:43,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6b575bc060ce15c0a5 HTTP/1.1" 502 150 2025-12-17 21:29:43,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d6b575bc060ce15c0a5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:44,489 - lava_callback - INFO - Received callback for node 69407ef4575bc060ce15c3b9 INFO: 172.21.0.1:52284 - "POST /node/69407ef4575bc060ce15c3b9 HTTP/1.0" 202 Accepted 2025-12-17 21:29:44,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ee8575bc060ce15c384 2025-12-17 21:29:44,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ee8575bc060ce15c384 HTTP/1.1" 502 150 2025-12-17 21:29:44,725 - lava_callback - ERROR - Error processing callback for node 69407ee8575bc060ce15c384: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ee8575bc060ce15c384 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ee8575bc060ce15c384 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ee8575bc060ce15c384 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:44,726 - lava_callback - INFO - Starting async job submit for node 69407d6c575bc060ce15c0a8 2025-12-17 21:29:44,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:44,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6c575bc060ce15c0a8 HTTP/1.1" 502 150 2025-12-17 21:29:44,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d6c575bc060ce15c0a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:44,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d6c575bc060ce15c0a8 2025-12-17 21:29:44,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6c575bc060ce15c0a8 HTTP/1.1" 502 150 2025-12-17 21:29:44,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d6c575bc060ce15c0a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:45,269 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d60575bc060ce15c077 2025-12-17 21:29:45,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d60575bc060ce15c077 HTTP/1.1" 502 150 2025-12-17 21:29:45,270 - lava_callback - ERROR - Error processing callback for node 69407d60575bc060ce15c077: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d60575bc060ce15c077 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d60575bc060ce15c077 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d60575bc060ce15c077 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:45,272 - lava_callback - INFO - Starting async job submit for node 69407d6d575bc060ce15c0ab 2025-12-17 21:29:45,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:45,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6d575bc060ce15c0ab HTTP/1.1" 502 150 2025-12-17 21:29:45,279 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d6d575bc060ce15c0ab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:45,279 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d6d575bc060ce15c0ab 2025-12-17 21:29:45,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6d575bc060ce15c0ab HTTP/1.1" 502 150 2025-12-17 21:29:45,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d6d575bc060ce15c0ab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:45,319 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d75575bc060ce15c0b5 2025-12-17 21:29:45,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d75575bc060ce15c0b5 HTTP/1.1" 502 150 2025-12-17 21:29:45,320 - lava_callback - ERROR - Error processing callback for node 69407d75575bc060ce15c0b5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d75575bc060ce15c0b5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d75575bc060ce15c0b5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d75575bc060ce15c0b5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:45,323 - lava_callback - INFO - Starting async job submit for node 6940998d575bc060ce161d37 2025-12-17 21:29:45,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:45,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998d575bc060ce161d37 HTTP/1.1" 502 150 2025-12-17 21:29:45,373 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940998d575bc060ce161d37'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:45,373 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940998d575bc060ce161d37 2025-12-17 21:29:45,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998d575bc060ce161d37 HTTP/1.1" 502 150 2025-12-17 21:29:45,373 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940998d575bc060ce161d37'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:45,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694079a1575bc060ce15b721 2025-12-17 21:29:45,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694079a1575bc060ce15b721 HTTP/1.1" 502 150 2025-12-17 21:29:45,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694079a1575bc060ce15b721'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:46,412 - lava_callback - INFO - Received callback for node 69407ef1575bc060ce15c3b4 INFO: 172.21.0.1:52298 - "POST /node/69407ef1575bc060ce15c3b4 HTTP/1.0" 202 Accepted 2025-12-17 21:29:46,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d65575bc060ce15c093 2025-12-17 21:29:46,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d65575bc060ce15c093 HTTP/1.1" 502 150 2025-12-17 21:29:46,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d65575bc060ce15c093'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52308 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:29:46,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d6c575bc060ce15c0a8 2025-12-17 21:29:46,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6c575bc060ce15c0a8 HTTP/1.1" 502 150 2025-12-17 21:29:46,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d6c575bc060ce15c0a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:47,280 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d6d575bc060ce15c0ab 2025-12-17 21:29:47,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6d575bc060ce15c0ab HTTP/1.1" 502 150 2025-12-17 21:29:47,281 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d6d575bc060ce15c0ab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:47,374 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940998d575bc060ce161d37 2025-12-17 21:29:47,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998d575bc060ce161d37 HTTP/1.1" 502 150 2025-12-17 21:29:47,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940998d575bc060ce161d37'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:47,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d63575bc060ce15c085 2025-12-17 21:29:47,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d63575bc060ce15c085 HTTP/1.1" 502 150 2025-12-17 21:29:47,836 - lava_callback - ERROR - Error processing callback for node 69407d63575bc060ce15c085: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d63575bc060ce15c085 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d63575bc060ce15c085 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d63575bc060ce15c085 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:47,838 - lava_callback - INFO - Starting async job submit for node 69407d77575bc060ce15c0bf 2025-12-17 21:29:47,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:47,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d77575bc060ce15c0bf HTTP/1.1" 502 150 2025-12-17 21:29:47,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d77575bc060ce15c0bf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:47,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d77575bc060ce15c0bf 2025-12-17 21:29:47,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d77575bc060ce15c0bf HTTP/1.1" 502 150 2025-12-17 21:29:47,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d77575bc060ce15c0bf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:48,077 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d64575bc060ce15c08c 2025-12-17 21:29:48,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d64575bc060ce15c08c HTTP/1.1" 502 150 2025-12-17 21:29:48,079 - lava_callback - ERROR - Error processing callback for node 69407d64575bc060ce15c08c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d64575bc060ce15c08c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d64575bc060ce15c08c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d64575bc060ce15c08c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:48,080 - lava_callback - INFO - Starting async job submit for node 69407d78575bc060ce15c0c4 2025-12-17 21:29:48,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:48,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d78575bc060ce15c0c4 HTTP/1.1" 502 150 2025-12-17 21:29:48,106 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d78575bc060ce15c0c4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:48,106 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d78575bc060ce15c0c4 2025-12-17 21:29:48,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d78575bc060ce15c0c4 HTTP/1.1" 502 150 2025-12-17 21:29:48,107 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d78575bc060ce15c0c4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:48,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d68575bc060ce15c09f 2025-12-17 21:29:48,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d67575bc060ce15c09a 2025-12-17 21:29:48,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d68575bc060ce15c09f HTTP/1.1" 502 150 2025-12-17 21:29:48,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d68575bc060ce15c09f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:48,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d67575bc060ce15c09a HTTP/1.1" 502 150 2025-12-17 21:29:48,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d67575bc060ce15c09a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:49,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d77575bc060ce15c0bf 2025-12-17 21:29:49,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d77575bc060ce15c0bf HTTP/1.1" 502 150 2025-12-17 21:29:49,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d77575bc060ce15c0bf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:49,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d61575bc060ce15c080 2025-12-17 21:29:49,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d61575bc060ce15c080 HTTP/1.1" 502 150 2025-12-17 21:29:49,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d61575bc060ce15c080'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:50,084 - lava_callback - INFO - Received callback for node 69407ef7575bc060ce15c3bf INFO: 172.21.0.1:59818 - "POST /node/69407ef7575bc060ce15c3bf HTTP/1.0" 202 Accepted 2025-12-17 21:29:50,107 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d78575bc060ce15c0c4 2025-12-17 21:29:50,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d78575bc060ce15c0c4 HTTP/1.1" 502 150 2025-12-17 21:29:50,109 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d78575bc060ce15c0c4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:50,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d6c575bc060ce15c0a8 2025-12-17 21:29:50,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6c575bc060ce15c0a8 HTTP/1.1" 502 150 2025-12-17 21:29:50,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d6c575bc060ce15c0a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:51,281 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d6d575bc060ce15c0ab 2025-12-17 21:29:51,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6d575bc060ce15c0ab HTTP/1.1" 502 150 2025-12-17 21:29:51,283 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d6d575bc060ce15c0ab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:51,375 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940998d575bc060ce161d37 2025-12-17 21:29:51,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998d575bc060ce161d37 HTTP/1.1" 502 150 2025-12-17 21:29:51,377 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940998d575bc060ce161d37'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:51,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d4f575bc060ce15c062 2025-12-17 21:29:51,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d4f575bc060ce15c062 HTTP/1.1" 502 150 2025-12-17 21:29:51,475 - lava_callback - ERROR - Error processing callback for node 69407d4f575bc060ce15c062: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4f575bc060ce15c062 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4f575bc060ce15c062 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d4f575bc060ce15c062 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:51,477 - lava_callback - INFO - Starting async job submit for node 69407d79575bc060ce15c0c9 2025-12-17 21:29:51,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:51,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d79575bc060ce15c0c9 HTTP/1.1" 502 150 2025-12-17 21:29:51,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d79575bc060ce15c0c9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:51,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d79575bc060ce15c0c9 2025-12-17 21:29:51,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d79575bc060ce15c0c9 HTTP/1.1" 502 150 2025-12-17 21:29:51,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d79575bc060ce15c0c9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:51,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d6b575bc060ce15c0a5 2025-12-17 21:29:51,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6b575bc060ce15c0a5 HTTP/1.1" 502 150 2025-12-17 21:29:51,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d6b575bc060ce15c0a5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:53,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d79575bc060ce15c0c9 2025-12-17 21:29:53,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d79575bc060ce15c0c9 HTTP/1.1" 502 150 2025-12-17 21:29:53,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d79575bc060ce15c0c9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:53,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d52575bc060ce15c06c HTTP/1.1" 502 150 2025-12-17 21:29:53,584 - lava_callback - ERROR - Error processing callback for node 69407d52575bc060ce15c06c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d52575bc060ce15c06c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d52575bc060ce15c06c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d52575bc060ce15c06c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:53,586 - lava_callback - INFO - Starting async job submit for node 69407d79575bc060ce15c0ce 2025-12-17 21:29:53,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:53,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d79575bc060ce15c0ce HTTP/1.1" 502 150 2025-12-17 21:29:53,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d79575bc060ce15c0ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:53,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d79575bc060ce15c0ce 2025-12-17 21:29:53,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d79575bc060ce15c0ce HTTP/1.1" 502 150 2025-12-17 21:29:53,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d79575bc060ce15c0ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:53,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d77575bc060ce15c0bf 2025-12-17 21:29:53,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d77575bc060ce15c0bf HTTP/1.1" 502 150 2025-12-17 21:29:53,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d77575bc060ce15c0bf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:53,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694079a1575bc060ce15b721 2025-12-17 21:29:53,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694079a1575bc060ce15b721 HTTP/1.1" 502 150 2025-12-17 21:29:53,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694079a1575bc060ce15b721'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:54,109 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d78575bc060ce15c0c4 2025-12-17 21:29:54,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d78575bc060ce15c0c4 HTTP/1.1" 502 150 2025-12-17 21:29:54,110 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d78575bc060ce15c0c4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:54,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d53575bc060ce15c06f 2025-12-17 21:29:54,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d53575bc060ce15c06f HTTP/1.1" 502 150 2025-12-17 21:29:54,551 - lava_callback - ERROR - Error processing callback for node 69407d53575bc060ce15c06f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d53575bc060ce15c06f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d53575bc060ce15c06f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d53575bc060ce15c06f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:54,553 - lava_callback - INFO - Starting async job submit for node 69407d7b575bc060ce15c0d7 2025-12-17 21:29:54,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:54,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7b575bc060ce15c0d7 HTTP/1.1" 502 150 2025-12-17 21:29:54,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7b575bc060ce15c0d7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:54,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7b575bc060ce15c0d7 2025-12-17 21:29:54,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7b575bc060ce15c0d7 HTTP/1.1" 502 150 2025-12-17 21:29:54,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7b575bc060ce15c0d7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:55,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d79575bc060ce15c0ce 2025-12-17 21:29:55,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d79575bc060ce15c0ce HTTP/1.1" 502 150 2025-12-17 21:29:55,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d79575bc060ce15c0ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59822 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:29:56,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7b575bc060ce15c0d7 2025-12-17 21:29:56,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7b575bc060ce15c0d7 HTTP/1.1" 502 150 2025-12-17 21:29:56,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7b575bc060ce15c0d7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:56,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d69575bc060ce15c0a2 HTTP/1.1" 502 150 2025-12-17 21:29:56,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d69575bc060ce15c0a2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:57,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d79575bc060ce15c0c9 2025-12-17 21:29:57,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d79575bc060ce15c0c9 HTTP/1.1" 502 150 2025-12-17 21:29:57,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d79575bc060ce15c0c9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:58,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407eee575bc060ce15c39d 2025-12-17 21:29:58,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407eee575bc060ce15c39d HTTP/1.1" 502 150 2025-12-17 21:29:58,496 - lava_callback - ERROR - Error processing callback for node 69407eee575bc060ce15c39d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eee575bc060ce15c39d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eee575bc060ce15c39d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407eee575bc060ce15c39d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:29:58,498 - lava_callback - INFO - Starting async job submit for node 69407d7b575bc060ce15c0ee 2025-12-17 21:29:58,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:29:58,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7b575bc060ce15c0ee HTTP/1.1" 502 150 2025-12-17 21:29:58,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7b575bc060ce15c0ee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:58,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7b575bc060ce15c0ee 2025-12-17 21:29:58,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7b575bc060ce15c0ee HTTP/1.1" 502 150 2025-12-17 21:29:58,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7b575bc060ce15c0ee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:58,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d6c575bc060ce15c0a8 2025-12-17 21:29:58,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6c575bc060ce15c0a8 HTTP/1.1" 502 150 2025-12-17 21:29:58,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d6c575bc060ce15c0a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:59,283 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d6d575bc060ce15c0ab 2025-12-17 21:29:59,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6d575bc060ce15c0ab HTTP/1.1" 502 150 2025-12-17 21:29:59,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d6d575bc060ce15c0ab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:59,377 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940998d575bc060ce161d37 2025-12-17 21:29:59,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998d575bc060ce161d37 HTTP/1.1" 502 150 2025-12-17 21:29:59,378 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940998d575bc060ce161d37'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:29:59,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d79575bc060ce15c0ce 2025-12-17 21:29:59,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d79575bc060ce15c0ce HTTP/1.1" 502 150 2025-12-17 21:29:59,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d79575bc060ce15c0ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:00,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7b575bc060ce15c0ee 2025-12-17 21:30:00,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7b575bc060ce15c0ee HTTP/1.1" 502 150 2025-12-17 21:30:00,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7b575bc060ce15c0ee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:00,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7b575bc060ce15c0d7 2025-12-17 21:30:00,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7b575bc060ce15c0d7 HTTP/1.1" 502 150 2025-12-17 21:30:00,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7b575bc060ce15c0d7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56904 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:30:01,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d77575bc060ce15c0bf 2025-12-17 21:30:01,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d77575bc060ce15c0bf HTTP/1.1" 502 150 2025-12-17 21:30:01,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d77575bc060ce15c0bf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:02,110 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d78575bc060ce15c0c4 2025-12-17 21:30:02,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d78575bc060ce15c0c4 HTTP/1.1" 502 150 2025-12-17 21:30:02,111 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d78575bc060ce15c0c4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:02,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d65575bc060ce15c093 2025-12-17 21:30:02,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d65575bc060ce15c093 HTTP/1.1" 502 150 2025-12-17 21:30:02,457 - lava_callback - ERROR - Error processing callback for node 69407d65575bc060ce15c093: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d65575bc060ce15c093 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d65575bc060ce15c093 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d65575bc060ce15c093 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:02,459 - lava_callback - INFO - Starting async job submit for node 69407d7c575bc060ce15c0ff 2025-12-17 21:30:02,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:02,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7c575bc060ce15c0ff HTTP/1.1" 502 150 2025-12-17 21:30:02,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7c575bc060ce15c0ff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:02,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7c575bc060ce15c0ff 2025-12-17 21:30:02,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7c575bc060ce15c0ff HTTP/1.1" 502 150 2025-12-17 21:30:02,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7c575bc060ce15c0ff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:04,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d68575bc060ce15c09f 2025-12-17 21:30:04,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d67575bc060ce15c09a 2025-12-17 21:30:04,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d67575bc060ce15c09a HTTP/1.1" 502 150 2025-12-17 21:30:04,456 - lava_callback - ERROR - Error processing callback for node 69407d67575bc060ce15c09a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d67575bc060ce15c09a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d67575bc060ce15c09a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d67575bc060ce15c09a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:04,459 - lava_callback - INFO - Starting async job submit for node 69407d7d575bc060ce15c10d 2025-12-17 21:30:04,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:04,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7d575bc060ce15c10d HTTP/1.1" 502 150 2025-12-17 21:30:04,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7d575bc060ce15c10d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:04,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7d575bc060ce15c10d 2025-12-17 21:30:04,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7c575bc060ce15c0ff 2025-12-17 21:30:04,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7d575bc060ce15c10d HTTP/1.1" 502 150 2025-12-17 21:30:04,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7d575bc060ce15c10d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:04,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7c575bc060ce15c0ff HTTP/1.1" 502 150 2025-12-17 21:30:04,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7c575bc060ce15c0ff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:04,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7b575bc060ce15c0ee 2025-12-17 21:30:04,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7b575bc060ce15c0ee HTTP/1.1" 502 150 2025-12-17 21:30:04,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7b575bc060ce15c0ee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:04,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d69575bc060ce15c0a2 2025-12-17 21:30:04,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d69575bc060ce15c0a2 HTTP/1.1" 502 150 2025-12-17 21:30:04,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d69575bc060ce15c0a2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:05,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d79575bc060ce15c0c9 2025-12-17 21:30:05,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d79575bc060ce15c0c9 HTTP/1.1" 502 150 2025-12-17 21:30:05,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d79575bc060ce15c0c9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:05,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d61575bc060ce15c080 2025-12-17 21:30:05,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d61575bc060ce15c080 HTTP/1.1" 502 150 2025-12-17 21:30:05,936 - lava_callback - ERROR - Error processing callback for node 69407d61575bc060ce15c080: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d61575bc060ce15c080 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d61575bc060ce15c080 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d61575bc060ce15c080 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:05,937 - lava_callback - INFO - Starting async job submit for node 69407d7d575bc060ce15c115 2025-12-17 21:30:05,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:05,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7d575bc060ce15c115 HTTP/1.1" 502 150 2025-12-17 21:30:05,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7d575bc060ce15c115'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:05,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7d575bc060ce15c115 2025-12-17 21:30:05,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7d575bc060ce15c115 HTTP/1.1" 502 150 2025-12-17 21:30:05,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7d575bc060ce15c115'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:06,241 - lava_callback - INFO - Received callback for node 69407ef6575bc060ce15c3bc INFO: 172.21.0.1:56910 - "POST /node/69407ef6575bc060ce15c3bc HTTP/1.0" 202 Accepted 2025-12-17 21:30:06,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7d575bc060ce15c10d 2025-12-17 21:30:06,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7d575bc060ce15c10d HTTP/1.1" 502 150 2025-12-17 21:30:06,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7d575bc060ce15c10d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56922 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:30:07,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d79575bc060ce15c0ce 2025-12-17 21:30:07,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d79575bc060ce15c0ce HTTP/1.1" 502 150 2025-12-17 21:30:07,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d79575bc060ce15c0ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:07,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d6b575bc060ce15c0a5 2025-12-17 21:30:07,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6b575bc060ce15c0a5 HTTP/1.1" 502 150 2025-12-17 21:30:07,636 - lava_callback - ERROR - Error processing callback for node 69407d6b575bc060ce15c0a5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d6b575bc060ce15c0a5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d6b575bc060ce15c0a5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d6b575bc060ce15c0a5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:07,638 - lava_callback - INFO - Starting async job submit for node 69407d80575bc060ce15c127 2025-12-17 21:30:07,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:07,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d80575bc060ce15c127 HTTP/1.1" 502 150 2025-12-17 21:30:07,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d80575bc060ce15c127'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:07,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d80575bc060ce15c127 2025-12-17 21:30:07,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d80575bc060ce15c127 HTTP/1.1" 502 150 2025-12-17 21:30:07,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d80575bc060ce15c127'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:07,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7d575bc060ce15c115 2025-12-17 21:30:07,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7d575bc060ce15c115 HTTP/1.1" 502 150 2025-12-17 21:30:07,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7d575bc060ce15c115'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:08,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7c575bc060ce15c0ff 2025-12-17 21:30:08,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7c575bc060ce15c0ff HTTP/1.1" 502 150 2025-12-17 21:30:08,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7c575bc060ce15c0ff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:08,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7b575bc060ce15c0d7 2025-12-17 21:30:08,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7b575bc060ce15c0d7 HTTP/1.1" 502 150 2025-12-17 21:30:08,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7b575bc060ce15c0d7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:09,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d80575bc060ce15c127 2025-12-17 21:30:09,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d80575bc060ce15c127 HTTP/1.1" 502 150 2025-12-17 21:30:09,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d80575bc060ce15c127'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:09,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694079a1575bc060ce15b721 2025-12-17 21:30:09,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694079a1575bc060ce15b721 HTTP/1.1" 502 150 2025-12-17 21:30:09,931 - lava_callback - ERROR - Error processing callback for node 694079a1575bc060ce15b721: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694079a1575bc060ce15b721 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694079a1575bc060ce15b721 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694079a1575bc060ce15b721 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:09,933 - lava_callback - INFO - Starting async job submit for node 69407d89575bc060ce15c130 2025-12-17 21:30:09,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:09,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d89575bc060ce15c130 HTTP/1.1" 502 150 2025-12-17 21:30:09,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d89575bc060ce15c130'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:09,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d89575bc060ce15c130 2025-12-17 21:30:09,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d89575bc060ce15c130 HTTP/1.1" 502 150 2025-12-17 21:30:09,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d89575bc060ce15c130'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:10,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7d575bc060ce15c10d 2025-12-17 21:30:10,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7d575bc060ce15c10d HTTP/1.1" 502 150 2025-12-17 21:30:10,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7d575bc060ce15c10d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:11,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d89575bc060ce15c130 2025-12-17 21:30:11,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d89575bc060ce15c130 HTTP/1.1" 502 150 2025-12-17 21:30:11,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d89575bc060ce15c130'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:11,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7d575bc060ce15c115 2025-12-17 21:30:11,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7d575bc060ce15c115 HTTP/1.1" 502 150 2025-12-17 21:30:11,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7d575bc060ce15c115'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:12,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7b575bc060ce15c0ee 2025-12-17 21:30:12,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7b575bc060ce15c0ee HTTP/1.1" 502 150 2025-12-17 21:30:12,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7b575bc060ce15c0ee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:13,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d80575bc060ce15c127 2025-12-17 21:30:13,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d80575bc060ce15c127 HTTP/1.1" 502 150 2025-12-17 21:30:13,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d80575bc060ce15c127'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:14,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d6c575bc060ce15c0a8 2025-12-17 21:30:14,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6c575bc060ce15c0a8 HTTP/1.1" 502 150 2025-12-17 21:30:14,766 - lava_callback - ERROR - Error processing callback for node 69407d6c575bc060ce15c0a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d6c575bc060ce15c0a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d6c575bc060ce15c0a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d6c575bc060ce15c0a8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:14,767 - lava_callback - INFO - Starting async job submit for node 69407d89575bc060ce15c133 2025-12-17 21:30:14,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:14,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d89575bc060ce15c133 HTTP/1.1" 502 150 2025-12-17 21:30:14,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d89575bc060ce15c133'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:14,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d89575bc060ce15c133 2025-12-17 21:30:14,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d89575bc060ce15c133 HTTP/1.1" 502 150 2025-12-17 21:30:14,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d89575bc060ce15c133'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52850 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:30:15,285 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d6d575bc060ce15c0ab 2025-12-17 21:30:15,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d6d575bc060ce15c0ab HTTP/1.1" 502 150 2025-12-17 21:30:15,286 - lava_callback - ERROR - Error processing callback for node 69407d6d575bc060ce15c0ab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d6d575bc060ce15c0ab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d6d575bc060ce15c0ab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d6d575bc060ce15c0ab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:15,287 - lava_callback - INFO - Starting async job submit for node 69407d8a575bc060ce15c136 2025-12-17 21:30:15,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:15,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d8a575bc060ce15c136 HTTP/1.1" 502 150 2025-12-17 21:30:15,315 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d8a575bc060ce15c136'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:15,315 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d8a575bc060ce15c136 2025-12-17 21:30:15,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d8a575bc060ce15c136 HTTP/1.1" 502 150 2025-12-17 21:30:15,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d8a575bc060ce15c136'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:15,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940998d575bc060ce161d37 2025-12-17 21:30:15,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998d575bc060ce161d37 HTTP/1.1" 502 150 2025-12-17 21:30:15,380 - lava_callback - ERROR - Error processing callback for node 6940998d575bc060ce161d37: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940998d575bc060ce161d37 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940998d575bc060ce161d37 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940998d575bc060ce161d37 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:15,381 - lava_callback - INFO - Starting async job submit for node 69407d76575bc060ce15c0ba 2025-12-17 21:30:15,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:15,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d76575bc060ce15c0ba HTTP/1.1" 502 150 2025-12-17 21:30:15,390 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d76575bc060ce15c0ba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:15,390 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d76575bc060ce15c0ba 2025-12-17 21:30:15,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d76575bc060ce15c0ba HTTP/1.1" 502 150 2025-12-17 21:30:15,391 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d76575bc060ce15c0ba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:15,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d89575bc060ce15c130 2025-12-17 21:30:15,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d89575bc060ce15c130 HTTP/1.1" 502 150 2025-12-17 21:30:15,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d89575bc060ce15c130'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:16,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7c575bc060ce15c0ff 2025-12-17 21:30:16,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7c575bc060ce15c0ff HTTP/1.1" 502 150 2025-12-17 21:30:16,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7c575bc060ce15c0ff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52860 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:30:16,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d89575bc060ce15c133 2025-12-17 21:30:16,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d89575bc060ce15c133 HTTP/1.1" 502 150 2025-12-17 21:30:16,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d89575bc060ce15c133'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:17,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d8a575bc060ce15c136 2025-12-17 21:30:17,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d8a575bc060ce15c136 HTTP/1.1" 502 150 2025-12-17 21:30:17,318 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d8a575bc060ce15c136'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:17,391 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d76575bc060ce15c0ba 2025-12-17 21:30:17,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d76575bc060ce15c0ba HTTP/1.1" 502 150 2025-12-17 21:30:17,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d76575bc060ce15c0ba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:17,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d77575bc060ce15c0bf 2025-12-17 21:30:17,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d77575bc060ce15c0bf HTTP/1.1" 502 150 2025-12-17 21:30:17,852 - lava_callback - ERROR - Error processing callback for node 69407d77575bc060ce15c0bf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d77575bc060ce15c0bf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d77575bc060ce15c0bf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d77575bc060ce15c0bf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:17,854 - lava_callback - INFO - Starting async job submit for node 69407d9e575bc060ce15c13b 2025-12-17 21:30:17,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:17,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d9e575bc060ce15c13b HTTP/1.1" 502 150 2025-12-17 21:30:17,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d9e575bc060ce15c13b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:17,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d9e575bc060ce15c13b 2025-12-17 21:30:17,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d9e575bc060ce15c13b HTTP/1.1" 502 150 2025-12-17 21:30:17,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d9e575bc060ce15c13b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:18,112 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d78575bc060ce15c0c4 2025-12-17 21:30:18,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d78575bc060ce15c0c4 HTTP/1.1" 502 150 2025-12-17 21:30:18,113 - lava_callback - ERROR - Error processing callback for node 69407d78575bc060ce15c0c4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d78575bc060ce15c0c4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d78575bc060ce15c0c4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d78575bc060ce15c0c4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:18,115 - lava_callback - INFO - Starting async job submit for node 69407da0575bc060ce15c146 2025-12-17 21:30:18,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:18,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da0575bc060ce15c146 HTTP/1.1" 502 150 2025-12-17 21:30:18,122 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da0575bc060ce15c146'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:18,122 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da0575bc060ce15c146 2025-12-17 21:30:18,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da0575bc060ce15c146 HTTP/1.1" 502 150 2025-12-17 21:30:18,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da0575bc060ce15c146'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:18,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7d575bc060ce15c10d 2025-12-17 21:30:18,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7d575bc060ce15c10d HTTP/1.1" 502 150 2025-12-17 21:30:18,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7d575bc060ce15c10d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:19,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d9e575bc060ce15c13b 2025-12-17 21:30:19,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7d575bc060ce15c115 2025-12-17 21:30:19,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7d575bc060ce15c115 HTTP/1.1" 502 150 2025-12-17 21:30:19,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d7d575bc060ce15c115'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:20,123 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da0575bc060ce15c146 2025-12-17 21:30:20,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da0575bc060ce15c146 HTTP/1.1" 502 150 2025-12-17 21:30:20,124 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da0575bc060ce15c146'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:20,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d69575bc060ce15c0a2 2025-12-17 21:30:20,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d69575bc060ce15c0a2 HTTP/1.1" 502 150 2025-12-17 21:30:20,663 - lava_callback - ERROR - Error processing callback for node 69407d69575bc060ce15c0a2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d69575bc060ce15c0a2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d69575bc060ce15c0a2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d69575bc060ce15c0a2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:20,665 - lava_callback - INFO - Starting async job submit for node 69407da2575bc060ce15c14d 2025-12-17 21:30:20,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:20,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da2575bc060ce15c14d HTTP/1.1" 502 150 2025-12-17 21:30:20,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da2575bc060ce15c14d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:20,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da2575bc060ce15c14d 2025-12-17 21:30:20,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da2575bc060ce15c14d HTTP/1.1" 502 150 2025-12-17 21:30:20,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da2575bc060ce15c14d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:20,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d89575bc060ce15c133 2025-12-17 21:30:20,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d89575bc060ce15c133 HTTP/1.1" 502 150 2025-12-17 21:30:20,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d89575bc060ce15c133'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:21,318 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d8a575bc060ce15c136 2025-12-17 21:30:21,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d8a575bc060ce15c136 HTTP/1.1" 502 150 2025-12-17 21:30:21,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d8a575bc060ce15c136'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:21,393 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d76575bc060ce15c0ba 2025-12-17 21:30:21,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d76575bc060ce15c0ba HTTP/1.1" 502 150 2025-12-17 21:30:21,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d76575bc060ce15c0ba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:21,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d79575bc060ce15c0c9 2025-12-17 21:30:21,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d79575bc060ce15c0c9 HTTP/1.1" 502 150 2025-12-17 21:30:21,494 - lava_callback - ERROR - Error processing callback for node 69407d79575bc060ce15c0c9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d79575bc060ce15c0c9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d79575bc060ce15c0c9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d79575bc060ce15c0c9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:21,496 - lava_callback - INFO - Starting async job submit for node 69407da3575bc060ce15c154 2025-12-17 21:30:21,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:21,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da3575bc060ce15c154 HTTP/1.1" 502 150 2025-12-17 21:30:21,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da3575bc060ce15c154'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:21,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da3575bc060ce15c154 2025-12-17 21:30:21,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da3575bc060ce15c154 HTTP/1.1" 502 150 2025-12-17 21:30:21,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da3575bc060ce15c154'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:21,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d80575bc060ce15c127 2025-12-17 21:30:21,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d80575bc060ce15c127 HTTP/1.1" 502 150 2025-12-17 21:30:21,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d80575bc060ce15c127'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:22,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da2575bc060ce15c14d 2025-12-17 21:30:22,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da2575bc060ce15c14d HTTP/1.1" 502 150 2025-12-17 21:30:22,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da2575bc060ce15c14d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:23,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da3575bc060ce15c154 2025-12-17 21:30:23,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da3575bc060ce15c154 HTTP/1.1" 502 150 2025-12-17 21:30:23,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da3575bc060ce15c154'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:23,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d79575bc060ce15c0ce 2025-12-17 21:30:23,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d79575bc060ce15c0ce HTTP/1.1" 502 150 2025-12-17 21:30:23,600 - lava_callback - ERROR - Error processing callback for node 69407d79575bc060ce15c0ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d79575bc060ce15c0ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d79575bc060ce15c0ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d79575bc060ce15c0ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:23,601 - lava_callback - INFO - Starting async job submit for node 69407da4575bc060ce15c157 2025-12-17 21:30:23,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:23,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da4575bc060ce15c157 HTTP/1.1" 502 150 2025-12-17 21:30:23,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da4575bc060ce15c157'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:23,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da4575bc060ce15c157 2025-12-17 21:30:23,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da4575bc060ce15c157 HTTP/1.1" 502 150 2025-12-17 21:30:23,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da4575bc060ce15c157'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:23,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d89575bc060ce15c130 2025-12-17 21:30:23,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d89575bc060ce15c130 HTTP/1.1" 502 150 2025-12-17 21:30:23,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d89575bc060ce15c130'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:24,124 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da0575bc060ce15c146 2025-12-17 21:30:24,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da0575bc060ce15c146 HTTP/1.1" 502 150 2025-12-17 21:30:24,125 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da0575bc060ce15c146'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:24,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7b575bc060ce15c0d7 2025-12-17 21:30:24,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7b575bc060ce15c0d7 HTTP/1.1" 502 150 2025-12-17 21:30:24,568 - lava_callback - ERROR - Error processing callback for node 69407d7b575bc060ce15c0d7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d7b575bc060ce15c0d7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d7b575bc060ce15c0d7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d7b575bc060ce15c0d7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:24,569 - lava_callback - INFO - Starting async job submit for node 69407da6575bc060ce15c15a 2025-12-17 21:30:24,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:24,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da6575bc060ce15c15a HTTP/1.1" 502 150 2025-12-17 21:30:24,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da6575bc060ce15c15a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:24,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da6575bc060ce15c15a 2025-12-17 21:30:24,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da6575bc060ce15c15a HTTP/1.1" 502 150 2025-12-17 21:30:24,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da6575bc060ce15c15a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:25,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da4575bc060ce15c157 2025-12-17 21:30:25,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da4575bc060ce15c157 HTTP/1.1" 502 150 2025-12-17 21:30:25,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da4575bc060ce15c157'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49046 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:30:26,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da6575bc060ce15c15a 2025-12-17 21:30:26,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da6575bc060ce15c15a HTTP/1.1" 502 150 2025-12-17 21:30:26,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da6575bc060ce15c15a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:26,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da2575bc060ce15c14d 2025-12-17 21:30:26,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da2575bc060ce15c14d HTTP/1.1" 502 150 2025-12-17 21:30:26,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da2575bc060ce15c14d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:27,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da3575bc060ce15c154 2025-12-17 21:30:27,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da3575bc060ce15c154 HTTP/1.1" 502 150 2025-12-17 21:30:27,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da3575bc060ce15c154'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:28,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7b575bc060ce15c0ee 2025-12-17 21:30:28,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7b575bc060ce15c0ee HTTP/1.1" 502 150 2025-12-17 21:30:28,516 - lava_callback - ERROR - Error processing callback for node 69407d7b575bc060ce15c0ee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d7b575bc060ce15c0ee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d7b575bc060ce15c0ee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d7b575bc060ce15c0ee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:28,517 - lava_callback - INFO - Starting async job submit for node 69407da7575bc060ce15c15d 2025-12-17 21:30:28,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:28,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da7575bc060ce15c15d HTTP/1.1" 502 150 2025-12-17 21:30:28,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da7575bc060ce15c15d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:28,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da7575bc060ce15c15d 2025-12-17 21:30:28,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da7575bc060ce15c15d HTTP/1.1" 502 150 2025-12-17 21:30:28,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da7575bc060ce15c15d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:28,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d89575bc060ce15c133 2025-12-17 21:30:28,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d89575bc060ce15c133 HTTP/1.1" 502 150 2025-12-17 21:30:28,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d89575bc060ce15c133'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:29,319 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d8a575bc060ce15c136 2025-12-17 21:30:29,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d8a575bc060ce15c136 HTTP/1.1" 502 150 2025-12-17 21:30:29,321 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d8a575bc060ce15c136'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:29,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d76575bc060ce15c0ba 2025-12-17 21:30:29,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d76575bc060ce15c0ba HTTP/1.1" 502 150 2025-12-17 21:30:29,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d76575bc060ce15c0ba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:29,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da4575bc060ce15c157 2025-12-17 21:30:29,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da4575bc060ce15c157 HTTP/1.1" 502 150 2025-12-17 21:30:29,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da4575bc060ce15c157'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:30,199 - lava_callback - INFO - Received callback for node 69407ef7575bc060ce15c3c2 INFO: 172.21.0.1:60286 - "POST /node/69407ef7575bc060ce15c3c2 HTTP/1.0" 202 Accepted 2025-12-17 21:30:30,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da7575bc060ce15c15d 2025-12-17 21:30:30,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da6575bc060ce15c15a 2025-12-17 21:30:30,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da6575bc060ce15c15a HTTP/1.1" 502 150 2025-12-17 21:30:30,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da6575bc060ce15c15a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:31,723 - lava_callback - INFO - Received callback for node 69407ef8575bc060ce15c3c5 INFO: 172.21.0.1:60296 - "POST /node/69407ef8575bc060ce15c3c5 HTTP/1.0" 202 Accepted 2025-12-17 21:30:32,125 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da0575bc060ce15c146 2025-12-17 21:30:32,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da0575bc060ce15c146 HTTP/1.1" 502 150 2025-12-17 21:30:32,127 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da0575bc060ce15c146'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:32,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7c575bc060ce15c0ff 2025-12-17 21:30:32,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7c575bc060ce15c0ff HTTP/1.1" 502 150 2025-12-17 21:30:32,473 - lava_callback - ERROR - Error processing callback for node 69407d7c575bc060ce15c0ff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d7c575bc060ce15c0ff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d7c575bc060ce15c0ff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d7c575bc060ce15c0ff (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:32,475 - lava_callback - INFO - Starting async job submit for node 69407b11575bc060ce15b936 2025-12-17 21:30:32,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:32,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b11575bc060ce15b936 HTTP/1.1" 502 150 2025-12-17 21:30:32,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b11575bc060ce15b936'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:32,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b11575bc060ce15b936 2025-12-17 21:30:32,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b11575bc060ce15b936 HTTP/1.1" 502 150 2025-12-17 21:30:32,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b11575bc060ce15b936'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:32,538 - lava_callback - INFO - Received callback for node 69407dee575bc060ce15c1a2 INFO: 172.21.0.1:60308 - "POST /node/69407dee575bc060ce15c1a2 HTTP/1.0" 202 Accepted 2025-12-17 21:30:33,213 - lava_callback - INFO - Received callback for node 69407df1575bc060ce15c1ad INFO: 172.21.0.1:60324 - "POST /node/69407df1575bc060ce15c1ad HTTP/1.0" 202 Accepted 2025-12-17 21:30:33,869 - lava_callback - INFO - Received callback for node 69407df2575bc060ce15c1b4 INFO: 172.21.0.1:60336 - "POST /node/69407df2575bc060ce15c1b4 HTTP/1.0" 202 Accepted 2025-12-17 21:30:34,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7d575bc060ce15c10d 2025-12-17 21:30:34,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7d575bc060ce15c10d HTTP/1.1" 502 150 2025-12-17 21:30:34,474 - lava_callback - ERROR - Error processing callback for node 69407d7d575bc060ce15c10d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d7d575bc060ce15c10d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d7d575bc060ce15c10d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d7d575bc060ce15c10d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:34,476 - lava_callback - INFO - Starting async job submit for node 69407da8575bc060ce15c160 2025-12-17 21:30:34,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:34,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da8575bc060ce15c160 HTTP/1.1" 502 150 2025-12-17 21:30:34,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da8575bc060ce15c160'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:34,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da8575bc060ce15c160 2025-12-17 21:30:34,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da8575bc060ce15c160 HTTP/1.1" 502 150 2025-12-17 21:30:34,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da8575bc060ce15c160'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:34,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b11575bc060ce15b936 2025-12-17 21:30:34,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b11575bc060ce15b936 HTTP/1.1" 502 150 2025-12-17 21:30:34,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b11575bc060ce15b936'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:34,528 - lava_callback - INFO - Received callback for node 69407df3575bc060ce15c1bb INFO: 172.21.0.1:60338 - "POST /node/69407df3575bc060ce15c1bb HTTP/1.0" 202 Accepted 2025-12-17 21:30:34,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da2575bc060ce15c14d 2025-12-17 21:30:34,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da2575bc060ce15c14d HTTP/1.1" 502 150 2025-12-17 21:30:34,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da2575bc060ce15c14d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:35,252 - lava_callback - INFO - Received callback for node 69407df5575bc060ce15c1c0 INFO: 172.21.0.1:60348 - "POST /node/69407df5575bc060ce15c1c0 HTTP/1.0" 202 Accepted 2025-12-17 21:30:35,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d9e575bc060ce15c13b HTTP/1.1" 502 150 2025-12-17 21:30:35,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d68575bc060ce15c09f HTTP/1.1" 502 150 2025-12-17 21:30:35,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d9e575bc060ce15c13b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:35,313 - lava_callback - ERROR - Error processing callback for node 69407d68575bc060ce15c09f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d68575bc060ce15c09f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d68575bc060ce15c09f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d68575bc060ce15c09f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:35,316 - lava_callback - INFO - Starting async job submit for node 69407da9575bc060ce15c163 2025-12-17 21:30:35,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:35,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da9575bc060ce15c163 HTTP/1.1" 502 150 2025-12-17 21:30:35,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da9575bc060ce15c163'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:35,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da9575bc060ce15c163 2025-12-17 21:30:35,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da9575bc060ce15c163 HTTP/1.1" 502 150 2025-12-17 21:30:35,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da9575bc060ce15c163'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:35,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da3575bc060ce15c154 2025-12-17 21:30:35,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da3575bc060ce15c154 HTTP/1.1" 502 150 2025-12-17 21:30:35,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da3575bc060ce15c154'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:35,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d7d575bc060ce15c115 2025-12-17 21:30:35,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d7d575bc060ce15c115 HTTP/1.1" 502 150 2025-12-17 21:30:35,954 - lava_callback - ERROR - Error processing callback for node 69407d7d575bc060ce15c115: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d7d575bc060ce15c115 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d7d575bc060ce15c115 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d7d575bc060ce15c115 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:35,954 - lava_callback - INFO - Received callback for node 69407df6575bc060ce15c1d0 2025-12-17 21:30:35,956 - lava_callback - INFO - Starting async job submit for node 69407ef4575bc060ce15c3b9 INFO: 172.21.0.1:60350 - "POST /node/69407df6575bc060ce15c1d0 HTTP/1.0" 202 Accepted 2025-12-17 21:30:35,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:35,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef4575bc060ce15c3b9 HTTP/1.1" 502 150 2025-12-17 21:30:35,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef4575bc060ce15c3b9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:35,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef4575bc060ce15c3b9 2025-12-17 21:30:35,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef4575bc060ce15c3b9 HTTP/1.1" 502 150 2025-12-17 21:30:35,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef4575bc060ce15c3b9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:36,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da8575bc060ce15c160 2025-12-17 21:30:36,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da8575bc060ce15c160 HTTP/1.1" 502 150 2025-12-17 21:30:36,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da8575bc060ce15c160'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60352 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:30:37,350 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da9575bc060ce15c163 2025-12-17 21:30:37,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da9575bc060ce15c163 HTTP/1.1" 502 150 2025-12-17 21:30:37,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da9575bc060ce15c163'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:37,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da4575bc060ce15c157 2025-12-17 21:30:37,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da4575bc060ce15c157 HTTP/1.1" 502 150 2025-12-17 21:30:37,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da4575bc060ce15c157'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:37,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d80575bc060ce15c127 2025-12-17 21:30:37,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d80575bc060ce15c127 HTTP/1.1" 502 150 2025-12-17 21:30:37,673 - lava_callback - ERROR - Error processing callback for node 69407d80575bc060ce15c127: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d80575bc060ce15c127 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d80575bc060ce15c127 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d80575bc060ce15c127 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:37,675 - lava_callback - INFO - Starting async job submit for node 69407ef1575bc060ce15c3b4 2025-12-17 21:30:37,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:37,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef1575bc060ce15c3b4 HTTP/1.1" 502 150 2025-12-17 21:30:37,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef1575bc060ce15c3b4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:37,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef1575bc060ce15c3b4 2025-12-17 21:30:37,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef1575bc060ce15c3b4 HTTP/1.1" 502 150 2025-12-17 21:30:37,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef1575bc060ce15c3b4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:37,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef4575bc060ce15c3b9 2025-12-17 21:30:37,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef4575bc060ce15c3b9 HTTP/1.1" 502 150 2025-12-17 21:30:37,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef4575bc060ce15c3b9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:38,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b11575bc060ce15b936 2025-12-17 21:30:38,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b11575bc060ce15b936 HTTP/1.1" 502 150 2025-12-17 21:30:38,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b11575bc060ce15b936'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:38,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da6575bc060ce15c15a 2025-12-17 21:30:38,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da6575bc060ce15c15a HTTP/1.1" 502 150 2025-12-17 21:30:38,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da6575bc060ce15c15a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:39,313 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d9e575bc060ce15c13b 2025-12-17 21:30:39,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d9e575bc060ce15c13b HTTP/1.1" 502 150 2025-12-17 21:30:39,314 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d9e575bc060ce15c13b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:39,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef1575bc060ce15c3b4 2025-12-17 21:30:39,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef1575bc060ce15c3b4 HTTP/1.1" 502 150 2025-12-17 21:30:39,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef1575bc060ce15c3b4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:39,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d89575bc060ce15c130 2025-12-17 21:30:39,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d89575bc060ce15c130 HTTP/1.1" 502 150 2025-12-17 21:30:39,947 - lava_callback - ERROR - Error processing callback for node 69407d89575bc060ce15c130: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d89575bc060ce15c130 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d89575bc060ce15c130 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d89575bc060ce15c130 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:39,949 - lava_callback - INFO - Starting async job submit for node 69407ef7575bc060ce15c3bf 2025-12-17 21:30:39,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:39,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef7575bc060ce15c3bf HTTP/1.1" 502 150 2025-12-17 21:30:39,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef7575bc060ce15c3bf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:39,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef7575bc060ce15c3bf 2025-12-17 21:30:39,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef7575bc060ce15c3bf HTTP/1.1" 502 150 2025-12-17 21:30:39,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef7575bc060ce15c3bf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:40,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da8575bc060ce15c160 2025-12-17 21:30:40,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da8575bc060ce15c160 HTTP/1.1" 502 150 2025-12-17 21:30:40,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da8575bc060ce15c160'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:41,351 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da9575bc060ce15c163 2025-12-17 21:30:41,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da9575bc060ce15c163 HTTP/1.1" 502 150 2025-12-17 21:30:41,352 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da9575bc060ce15c163'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:41,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef7575bc060ce15c3bf 2025-12-17 21:30:41,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef7575bc060ce15c3bf HTTP/1.1" 502 150 2025-12-17 21:30:41,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef7575bc060ce15c3bf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:41,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef4575bc060ce15c3b9 2025-12-17 21:30:41,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef4575bc060ce15c3b9 HTTP/1.1" 502 150 2025-12-17 21:30:41,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef4575bc060ce15c3b9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:43,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef1575bc060ce15c3b4 2025-12-17 21:30:43,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef1575bc060ce15c3b4 HTTP/1.1" 502 150 2025-12-17 21:30:43,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef1575bc060ce15c3b4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:44,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d89575bc060ce15c133 2025-12-17 21:30:44,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d89575bc060ce15c133 HTTP/1.1" 502 150 2025-12-17 21:30:44,784 - lava_callback - ERROR - Error processing callback for node 69407d89575bc060ce15c133: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d89575bc060ce15c133 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d89575bc060ce15c133 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d89575bc060ce15c133 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:44,787 - lava_callback - INFO - Starting async job submit for node 69407ef6575bc060ce15c3bc 2025-12-17 21:30:44,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:44,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef6575bc060ce15c3bc HTTP/1.1" 502 150 2025-12-17 21:30:44,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef6575bc060ce15c3bc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:44,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef6575bc060ce15c3bc 2025-12-17 21:30:44,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef6575bc060ce15c3bc HTTP/1.1" 502 150 2025-12-17 21:30:44,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef6575bc060ce15c3bc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:45,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d8a575bc060ce15c136 2025-12-17 21:30:45,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d8a575bc060ce15c136 HTTP/1.1" 502 150 2025-12-17 21:30:45,322 - lava_callback - ERROR - Error processing callback for node 69407d8a575bc060ce15c136: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d8a575bc060ce15c136 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d8a575bc060ce15c136 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d8a575bc060ce15c136 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:45,324 - lava_callback - INFO - Starting async job submit for node 69407ef7575bc060ce15c3c2 2025-12-17 21:30:45,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:45,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef7575bc060ce15c3c2 HTTP/1.1" 502 150 2025-12-17 21:30:45,335 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef7575bc060ce15c3c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:45,336 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef7575bc060ce15c3c2 2025-12-17 21:30:45,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef7575bc060ce15c3c2 HTTP/1.1" 502 150 2025-12-17 21:30:45,336 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef7575bc060ce15c3c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:45,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d76575bc060ce15c0ba 2025-12-17 21:30:45,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d76575bc060ce15c0ba HTTP/1.1" 502 150 2025-12-17 21:30:45,397 - lava_callback - ERROR - Error processing callback for node 69407d76575bc060ce15c0ba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d76575bc060ce15c0ba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d76575bc060ce15c0ba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d76575bc060ce15c0ba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:45,399 - lava_callback - INFO - Starting async job submit for node 69407ef8575bc060ce15c3c5 2025-12-17 21:30:45,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:45,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef8575bc060ce15c3c5 HTTP/1.1" 502 150 2025-12-17 21:30:45,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef8575bc060ce15c3c5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:45,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef8575bc060ce15c3c5 2025-12-17 21:30:45,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef8575bc060ce15c3c5 HTTP/1.1" 502 150 2025-12-17 21:30:45,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef8575bc060ce15c3c5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:45,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da7575bc060ce15c15d HTTP/1.1" 502 150 2025-12-17 21:30:45,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da7575bc060ce15c15d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:45,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef7575bc060ce15c3bf 2025-12-17 21:30:45,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef7575bc060ce15c3bf HTTP/1.1" 502 150 2025-12-17 21:30:45,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef7575bc060ce15c3bf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:46,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b11575bc060ce15b936 INFO: 172.21.0.1:42236 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:30:46,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef6575bc060ce15c3bc 2025-12-17 21:30:46,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef6575bc060ce15c3bc HTTP/1.1" 502 150 2025-12-17 21:30:46,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef6575bc060ce15c3bc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:47,315 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d9e575bc060ce15c13b 2025-12-17 21:30:47,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d9e575bc060ce15c13b HTTP/1.1" 502 150 2025-12-17 21:30:47,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407d9e575bc060ce15c13b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:47,336 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef7575bc060ce15c3c2 2025-12-17 21:30:47,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef7575bc060ce15c3c2 HTTP/1.1" 502 150 2025-12-17 21:30:47,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef7575bc060ce15c3c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:47,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef8575bc060ce15c3c5 2025-12-17 21:30:47,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef8575bc060ce15c3c5 HTTP/1.1" 502 150 2025-12-17 21:30:47,410 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef8575bc060ce15c3c5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:48,127 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da0575bc060ce15c146 2025-12-17 21:30:48,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da0575bc060ce15c146 HTTP/1.1" 502 150 2025-12-17 21:30:48,128 - lava_callback - ERROR - Error processing callback for node 69407da0575bc060ce15c146: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da0575bc060ce15c146 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da0575bc060ce15c146 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da0575bc060ce15c146 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:48,130 - lava_callback - INFO - Starting async job submit for node 69407dee575bc060ce15c1a2 2025-12-17 21:30:48,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:48,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dee575bc060ce15c1a2 HTTP/1.1" 502 150 2025-12-17 21:30:48,137 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dee575bc060ce15c1a2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:48,137 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dee575bc060ce15c1a2 2025-12-17 21:30:48,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dee575bc060ce15c1a2 HTTP/1.1" 502 150 2025-12-17 21:30:48,138 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dee575bc060ce15c1a2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:48,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da8575bc060ce15c160 2025-12-17 21:30:48,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da8575bc060ce15c160 HTTP/1.1" 502 150 2025-12-17 21:30:48,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da8575bc060ce15c160'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:49,353 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da9575bc060ce15c163 2025-12-17 21:30:49,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da9575bc060ce15c163 HTTP/1.1" 502 150 2025-12-17 21:30:49,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da9575bc060ce15c163'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:49,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da7575bc060ce15c15d 2025-12-17 21:30:49,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da7575bc060ce15c15d HTTP/1.1" 502 150 2025-12-17 21:30:49,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da7575bc060ce15c15d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:49,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef4575bc060ce15c3b9 2025-12-17 21:30:49,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef4575bc060ce15c3b9 HTTP/1.1" 502 150 2025-12-17 21:30:49,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef4575bc060ce15c3b9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:50,138 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dee575bc060ce15c1a2 2025-12-17 21:30:50,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dee575bc060ce15c1a2 HTTP/1.1" 502 150 2025-12-17 21:30:50,140 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dee575bc060ce15c1a2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:50,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da2575bc060ce15c14d 2025-12-17 21:30:50,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da2575bc060ce15c14d HTTP/1.1" 502 150 2025-12-17 21:30:50,678 - lava_callback - ERROR - Error processing callback for node 69407da2575bc060ce15c14d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da2575bc060ce15c14d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da2575bc060ce15c14d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da2575bc060ce15c14d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:50,680 - lava_callback - INFO - Starting async job submit for node 69407df1575bc060ce15c1ad 2025-12-17 21:30:50,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:50,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df1575bc060ce15c1ad HTTP/1.1" 502 150 2025-12-17 21:30:50,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df1575bc060ce15c1ad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:50,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df1575bc060ce15c1ad 2025-12-17 21:30:50,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df1575bc060ce15c1ad HTTP/1.1" 502 150 2025-12-17 21:30:50,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df1575bc060ce15c1ad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:50,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef6575bc060ce15c3bc 2025-12-17 21:30:50,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef6575bc060ce15c3bc HTTP/1.1" 502 150 2025-12-17 21:30:50,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef6575bc060ce15c3bc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:51,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef7575bc060ce15c3c2 2025-12-17 21:30:51,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef7575bc060ce15c3c2 HTTP/1.1" 502 150 2025-12-17 21:30:51,339 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef7575bc060ce15c3c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:51,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef8575bc060ce15c3c5 2025-12-17 21:30:51,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef8575bc060ce15c3c5 HTTP/1.1" 502 150 2025-12-17 21:30:51,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef8575bc060ce15c3c5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:51,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da3575bc060ce15c154 2025-12-17 21:30:51,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da3575bc060ce15c154 HTTP/1.1" 502 150 2025-12-17 21:30:51,511 - lava_callback - ERROR - Error processing callback for node 69407da3575bc060ce15c154: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da3575bc060ce15c154 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da3575bc060ce15c154 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da3575bc060ce15c154 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:51,514 - lava_callback - INFO - Starting async job submit for node 69407df2575bc060ce15c1b4 2025-12-17 21:30:51,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:51,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df2575bc060ce15c1b4 HTTP/1.1" 502 150 2025-12-17 21:30:51,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df2575bc060ce15c1b4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:51,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df2575bc060ce15c1b4 2025-12-17 21:30:51,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df2575bc060ce15c1b4 HTTP/1.1" 502 150 2025-12-17 21:30:51,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df2575bc060ce15c1b4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:51,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef1575bc060ce15c3b4 2025-12-17 21:30:51,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef1575bc060ce15c3b4 HTTP/1.1" 502 150 2025-12-17 21:30:51,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef1575bc060ce15c3b4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:52,502 - lava_callback - INFO - Received callback for node 69407df9575bc060ce15c1ed INFO: 172.21.0.1:53186 - "POST /node/69407df9575bc060ce15c1ed HTTP/1.0" 202 Accepted 2025-12-17 21:30:52,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df1575bc060ce15c1ad 2025-12-17 21:30:52,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df1575bc060ce15c1ad HTTP/1.1" 502 150 2025-12-17 21:30:52,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df1575bc060ce15c1ad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:53,158 - lava_callback - INFO - Received callback for node 69407dfa575bc060ce15c1f4 INFO: 172.21.0.1:53198 - "POST /node/69407dfa575bc060ce15c1f4 HTTP/1.0" 202 Accepted 2025-12-17 21:30:53,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df2575bc060ce15c1b4 2025-12-17 21:30:53,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df2575bc060ce15c1b4 HTTP/1.1" 502 150 2025-12-17 21:30:53,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df2575bc060ce15c1b4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:53,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da4575bc060ce15c157 2025-12-17 21:30:53,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da4575bc060ce15c157 HTTP/1.1" 502 150 2025-12-17 21:30:53,616 - lava_callback - ERROR - Error processing callback for node 69407da4575bc060ce15c157: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da4575bc060ce15c157 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da4575bc060ce15c157 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da4575bc060ce15c157 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:53,618 - lava_callback - INFO - Starting async job submit for node 69407df3575bc060ce15c1bb 2025-12-17 21:30:53,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:53,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df3575bc060ce15c1bb HTTP/1.1" 502 150 2025-12-17 21:30:53,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df3575bc060ce15c1bb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:53,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df3575bc060ce15c1bb 2025-12-17 21:30:53,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df3575bc060ce15c1bb HTTP/1.1" 502 150 2025-12-17 21:30:53,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df3575bc060ce15c1bb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:53,826 - lava_callback - INFO - Received callback for node 69407dfb575bc060ce15c1f7 INFO: 172.21.0.1:53202 - "POST /node/69407dfb575bc060ce15c1f7 HTTP/1.0" 202 Accepted 2025-12-17 21:30:53,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef7575bc060ce15c3bf 2025-12-17 21:30:53,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef7575bc060ce15c3bf HTTP/1.1" 502 150 2025-12-17 21:30:53,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef7575bc060ce15c3bf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:54,140 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dee575bc060ce15c1a2 2025-12-17 21:30:54,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dee575bc060ce15c1a2 HTTP/1.1" 502 150 2025-12-17 21:30:54,141 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dee575bc060ce15c1a2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:54,524 - lava_callback - INFO - Received callback for node 69407dfd575bc060ce15c1fa INFO: 172.21.0.1:53214 - "POST /node/69407dfd575bc060ce15c1fa HTTP/1.0" 202 Accepted 2025-12-17 21:30:54,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da6575bc060ce15c15a 2025-12-17 21:30:54,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da6575bc060ce15c15a HTTP/1.1" 502 150 2025-12-17 21:30:54,582 - lava_callback - ERROR - Error processing callback for node 69407da6575bc060ce15c15a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da6575bc060ce15c15a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da6575bc060ce15c15a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da6575bc060ce15c15a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:30:54,584 - lava_callback - INFO - Starting async job submit for node 69407df5575bc060ce15c1c0 2025-12-17 21:30:54,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:30:54,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df5575bc060ce15c1c0 HTTP/1.1" 502 150 2025-12-17 21:30:54,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df5575bc060ce15c1c0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:54,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df5575bc060ce15c1c0 2025-12-17 21:30:54,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df5575bc060ce15c1c0 HTTP/1.1" 502 150 2025-12-17 21:30:54,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df5575bc060ce15c1c0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:55,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df3575bc060ce15c1bb 2025-12-17 21:30:55,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df3575bc060ce15c1bb HTTP/1.1" 502 150 2025-12-17 21:30:55,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df3575bc060ce15c1bb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53228 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:30:56,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df5575bc060ce15c1c0 2025-12-17 21:30:56,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df1575bc060ce15c1ad 2025-12-17 21:30:56,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df1575bc060ce15c1ad HTTP/1.1" 502 150 2025-12-17 21:30:56,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df1575bc060ce15c1ad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:57,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df2575bc060ce15c1b4 2025-12-17 21:30:57,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df2575bc060ce15c1b4 HTTP/1.1" 502 150 2025-12-17 21:30:57,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df2575bc060ce15c1b4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:57,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da7575bc060ce15c15d 2025-12-17 21:30:57,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da7575bc060ce15c15d HTTP/1.1" 502 150 2025-12-17 21:30:57,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407da7575bc060ce15c15d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:58,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef6575bc060ce15c3bc 2025-12-17 21:30:58,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef6575bc060ce15c3bc HTTP/1.1" 502 150 2025-12-17 21:30:58,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef6575bc060ce15c3bc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:59,339 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef7575bc060ce15c3c2 2025-12-17 21:30:59,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef7575bc060ce15c3c2 HTTP/1.1" 502 150 2025-12-17 21:30:59,341 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef7575bc060ce15c3c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:59,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef8575bc060ce15c3c5 2025-12-17 21:30:59,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef8575bc060ce15c3c5 HTTP/1.1" 502 150 2025-12-17 21:30:59,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ef8575bc060ce15c3c5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:30:59,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df3575bc060ce15c1bb 2025-12-17 21:30:59,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df3575bc060ce15c1bb HTTP/1.1" 502 150 2025-12-17 21:30:59,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df3575bc060ce15c1bb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41858 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:31:01,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b11575bc060ce15b936 HTTP/1.1" 502 150 2025-12-17 21:31:01,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407b11575bc060ce15b936'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:02,141 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dee575bc060ce15c1a2 2025-12-17 21:31:02,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dee575bc060ce15c1a2 HTTP/1.1" 502 150 2025-12-17 21:31:02,142 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dee575bc060ce15c1a2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:03,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407d9e575bc060ce15c13b 2025-12-17 21:31:03,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407d9e575bc060ce15c13b HTTP/1.1" 502 150 2025-12-17 21:31:03,317 - lava_callback - ERROR - Error processing callback for node 69407d9e575bc060ce15c13b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d9e575bc060ce15c13b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d9e575bc060ce15c13b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407d9e575bc060ce15c13b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:03,319 - lava_callback - INFO - Starting async job submit for node 69407df6575bc060ce15c1d0 2025-12-17 21:31:03,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:03,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df6575bc060ce15c1d0 HTTP/1.1" 502 150 2025-12-17 21:31:03,356 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df6575bc060ce15c1d0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:03,356 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df6575bc060ce15c1d0 2025-12-17 21:31:03,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df6575bc060ce15c1d0 HTTP/1.1" 502 150 2025-12-17 21:31:03,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df6575bc060ce15c1d0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:04,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da8575bc060ce15c160 2025-12-17 21:31:04,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da8575bc060ce15c160 HTTP/1.1" 502 150 2025-12-17 21:31:04,500 - lava_callback - ERROR - Error processing callback for node 69407da8575bc060ce15c160: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da8575bc060ce15c160 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da8575bc060ce15c160 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da8575bc060ce15c160 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:04,502 - lava_callback - INFO - Starting async job submit for node 69407df9575bc060ce15c1ed 2025-12-17 21:31:04,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:04,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df9575bc060ce15c1ed HTTP/1.1" 502 150 2025-12-17 21:31:04,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df9575bc060ce15c1ed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:04,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df9575bc060ce15c1ed 2025-12-17 21:31:04,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df9575bc060ce15c1ed HTTP/1.1" 502 150 2025-12-17 21:31:04,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df9575bc060ce15c1ed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:04,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df1575bc060ce15c1ad 2025-12-17 21:31:04,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df1575bc060ce15c1ad HTTP/1.1" 502 150 2025-12-17 21:31:04,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df1575bc060ce15c1ad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:05,354 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da9575bc060ce15c163 2025-12-17 21:31:05,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da9575bc060ce15c163 HTTP/1.1" 502 150 2025-12-17 21:31:05,356 - lava_callback - ERROR - Error processing callback for node 69407da9575bc060ce15c163: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da9575bc060ce15c163 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da9575bc060ce15c163 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da9575bc060ce15c163 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:05,357 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df6575bc060ce15c1d0 2025-12-17 21:31:05,358 - lava_callback - INFO - Starting async job submit for node 69407dfa575bc060ce15c1f4 2025-12-17 21:31:05,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:05,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df6575bc060ce15c1d0 HTTP/1.1" 502 150 2025-12-17 21:31:05,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df6575bc060ce15c1d0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:05,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfa575bc060ce15c1f4 HTTP/1.1" 502 150 2025-12-17 21:31:05,364 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dfa575bc060ce15c1f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:05,364 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dfa575bc060ce15c1f4 2025-12-17 21:31:05,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfa575bc060ce15c1f4 HTTP/1.1" 502 150 2025-12-17 21:31:05,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dfa575bc060ce15c1f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:05,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df2575bc060ce15c1b4 2025-12-17 21:31:05,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df2575bc060ce15c1b4 HTTP/1.1" 502 150 2025-12-17 21:31:05,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df2575bc060ce15c1b4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:05,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef4575bc060ce15c3b9 2025-12-17 21:31:05,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef4575bc060ce15c3b9 HTTP/1.1" 502 150 2025-12-17 21:31:06,000 - lava_callback - ERROR - Error processing callback for node 69407ef4575bc060ce15c3b9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef4575bc060ce15c3b9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef4575bc060ce15c3b9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef4575bc060ce15c3b9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:06,002 - lava_callback - INFO - Starting async job submit for node 69407dfb575bc060ce15c1f7 2025-12-17 21:31:06,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:06,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfb575bc060ce15c1f7 HTTP/1.1" 502 150 2025-12-17 21:31:06,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dfb575bc060ce15c1f7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:06,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dfb575bc060ce15c1f7 2025-12-17 21:31:06,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfb575bc060ce15c1f7 HTTP/1.1" 502 150 2025-12-17 21:31:06,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dfb575bc060ce15c1f7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:06,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df9575bc060ce15c1ed INFO: 172.21.0.1:41868 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:31:06,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df9575bc060ce15c1ed HTTP/1.1" 502 150 2025-12-17 21:31:06,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df9575bc060ce15c1ed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:07,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dfa575bc060ce15c1f4 2025-12-17 21:31:07,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df3575bc060ce15c1bb 2025-12-17 21:31:07,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df3575bc060ce15c1bb HTTP/1.1" 502 150 2025-12-17 21:31:07,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df3575bc060ce15c1bb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:07,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef1575bc060ce15c3b4 2025-12-17 21:31:07,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef1575bc060ce15c3b4 HTTP/1.1" 502 150 2025-12-17 21:31:07,714 - lava_callback - ERROR - Error processing callback for node 69407ef1575bc060ce15c3b4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef1575bc060ce15c3b4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef1575bc060ce15c3b4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef1575bc060ce15c3b4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:07,717 - lava_callback - INFO - Starting async job submit for node 69407dfd575bc060ce15c1fa 2025-12-17 21:31:07,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:07,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfd575bc060ce15c1fa HTTP/1.1" 502 150 2025-12-17 21:31:07,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dfd575bc060ce15c1fa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:07,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dfd575bc060ce15c1fa 2025-12-17 21:31:07,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfd575bc060ce15c1fa HTTP/1.1" 502 150 2025-12-17 21:31:07,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dfd575bc060ce15c1fa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:08,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dfb575bc060ce15c1f7 2025-12-17 21:31:08,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfb575bc060ce15c1f7 HTTP/1.1" 502 150 2025-12-17 21:31:08,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dfb575bc060ce15c1f7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:09,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df6575bc060ce15c1d0 2025-12-17 21:31:09,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df6575bc060ce15c1d0 HTTP/1.1" 502 150 2025-12-17 21:31:09,361 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df6575bc060ce15c1d0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:09,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dfd575bc060ce15c1fa 2025-12-17 21:31:09,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfd575bc060ce15c1fa HTTP/1.1" 502 150 2025-12-17 21:31:09,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dfd575bc060ce15c1fa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:09,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef7575bc060ce15c3bf 2025-12-17 21:31:09,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef7575bc060ce15c3bf HTTP/1.1" 502 150 2025-12-17 21:31:09,969 - lava_callback - ERROR - Error processing callback for node 69407ef7575bc060ce15c3bf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef7575bc060ce15c3bf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef7575bc060ce15c3bf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef7575bc060ce15c3bf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:10,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df9575bc060ce15c1ed 2025-12-17 21:31:10,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df9575bc060ce15c1ed HTTP/1.1" 502 150 2025-12-17 21:31:10,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df9575bc060ce15c1ed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:11,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df5575bc060ce15c1c0 HTTP/1.1" 502 150 2025-12-17 21:31:11,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df5575bc060ce15c1c0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:12,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dfb575bc060ce15c1f7 2025-12-17 21:31:12,559 - lava_callback - INFO - Received callback for node 69407e0d575bc060ce15c21d INFO: 172.21.0.1:37170 - "POST /node/69407e0d575bc060ce15c21d HTTP/1.0" 202 Accepted 2025-12-17 21:31:12,568 - lava_callback - INFO - Starting async job submit for node 69407e0d575bc060ce15c21d 2025-12-17 21:31:12,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:12,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0d575bc060ce15c21d HTTP/1.1" 502 150 2025-12-17 21:31:12,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0d575bc060ce15c21d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:12,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0d575bc060ce15c21d 2025-12-17 21:31:12,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0d575bc060ce15c21d HTTP/1.1" 502 150 2025-12-17 21:31:12,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0d575bc060ce15c21d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:13,244 - lava_callback - INFO - Received callback for node 69407e0e575bc060ce15c220 INFO: 172.21.0.1:37178 - "POST /node/69407e0e575bc060ce15c220 HTTP/1.0" 202 Accepted 2025-12-17 21:31:13,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dfd575bc060ce15c1fa 2025-12-17 21:31:13,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfd575bc060ce15c1fa HTTP/1.1" 502 150 2025-12-17 21:31:13,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dfd575bc060ce15c1fa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:13,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407da7575bc060ce15c15d 2025-12-17 21:31:13,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407da7575bc060ce15c15d HTTP/1.1" 502 150 2025-12-17 21:31:13,813 - lava_callback - ERROR - Error processing callback for node 69407da7575bc060ce15c15d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da7575bc060ce15c15d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da7575bc060ce15c15d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407da7575bc060ce15c15d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:13,814 - lava_callback - INFO - Starting async job submit for node 69407e0e575bc060ce15c220 2025-12-17 21:31:13,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:13,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0e575bc060ce15c220 HTTP/1.1" 502 150 2025-12-17 21:31:13,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0e575bc060ce15c220'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:13,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0e575bc060ce15c220 2025-12-17 21:31:13,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0e575bc060ce15c220 HTTP/1.1" 502 150 2025-12-17 21:31:13,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0e575bc060ce15c220'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:13,919 - lava_callback - INFO - Received callback for node 69407e0f575bc060ce15c223 INFO: 172.21.0.1:37184 - "POST /node/69407e0f575bc060ce15c223 HTTP/1.0" 202 Accepted 2025-12-17 21:31:14,259 - lava_callback - INFO - Received callback for node 6940a5bd1e3c24e956819d52 INFO: 172.21.0.1:37198 - "POST /node/6940a5bd1e3c24e956819d52 HTTP/1.0" 202 Accepted 2025-12-17 21:31:14,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0d575bc060ce15c21d 2025-12-17 21:31:14,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0d575bc060ce15c21d HTTP/1.1" 502 150 2025-12-17 21:31:14,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0d575bc060ce15c21d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:14,587 - lava_callback - INFO - Received callback for node 69407e10575bc060ce15c226 INFO: 172.21.0.1:37214 - "POST /node/69407e10575bc060ce15c226 HTTP/1.0" 202 Accepted 2025-12-17 21:31:14,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef6575bc060ce15c3bc 2025-12-17 21:31:14,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef6575bc060ce15c3bc HTTP/1.1" 502 150 2025-12-17 21:31:14,830 - lava_callback - ERROR - Error processing callback for node 69407ef6575bc060ce15c3bc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef6575bc060ce15c3bc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef6575bc060ce15c3bc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef6575bc060ce15c3bc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:14,832 - lava_callback - INFO - Starting async job submit for node 69407e0f575bc060ce15c223 2025-12-17 21:31:14,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:14,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0f575bc060ce15c223 HTTP/1.1" 502 150 2025-12-17 21:31:14,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0f575bc060ce15c223'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:14,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0f575bc060ce15c223 2025-12-17 21:31:14,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0f575bc060ce15c223 HTTP/1.1" 502 150 2025-12-17 21:31:14,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0f575bc060ce15c223'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37228 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:31:15,254 - lava_callback - INFO - Received callback for node 69407e12575bc060ce15c229 INFO: 172.21.0.1:37236 - "POST /node/69407e12575bc060ce15c229 HTTP/1.0" 202 Accepted 2025-12-17 21:31:15,341 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef7575bc060ce15c3c2 2025-12-17 21:31:15,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef7575bc060ce15c3c2 HTTP/1.1" 502 150 2025-12-17 21:31:15,342 - lava_callback - ERROR - Error processing callback for node 69407ef7575bc060ce15c3c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef7575bc060ce15c3c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef7575bc060ce15c3c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef7575bc060ce15c3c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:15,344 - lava_callback - INFO - Starting async job submit for node 6940a5bd1e3c24e956819d52 2025-12-17 21:31:15,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:15,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a5bd1e3c24e956819d52 HTTP/1.1" 502 150 2025-12-17 21:31:15,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a5bd1e3c24e956819d52'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:15,375 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a5bd1e3c24e956819d52 2025-12-17 21:31:15,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a5bd1e3c24e956819d52 HTTP/1.1" 502 150 2025-12-17 21:31:15,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a5bd1e3c24e956819d52'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:15,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ef8575bc060ce15c3c5 2025-12-17 21:31:15,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ef8575bc060ce15c3c5 HTTP/1.1" 502 150 2025-12-17 21:31:15,415 - lava_callback - ERROR - Error processing callback for node 69407ef8575bc060ce15c3c5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef8575bc060ce15c3c5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef8575bc060ce15c3c5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ef8575bc060ce15c3c5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:15,416 - lava_callback - INFO - Starting async job submit for node 69407e10575bc060ce15c226 2025-12-17 21:31:15,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:15,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e10575bc060ce15c226 HTTP/1.1" 502 150 2025-12-17 21:31:15,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e10575bc060ce15c226'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:15,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e10575bc060ce15c226 2025-12-17 21:31:15,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e10575bc060ce15c226 HTTP/1.1" 502 150 2025-12-17 21:31:15,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e10575bc060ce15c226'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:15,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0e575bc060ce15c220 2025-12-17 21:31:15,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0e575bc060ce15c220 HTTP/1.1" 502 150 2025-12-17 21:31:15,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0e575bc060ce15c220'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:15,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df5575bc060ce15c1c0 2025-12-17 21:31:15,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df5575bc060ce15c1c0 HTTP/1.1" 502 150 2025-12-17 21:31:15,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df5575bc060ce15c1c0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:15,927 - lava_callback - INFO - Received callback for node 69407e13575bc060ce15c22c INFO: 172.21.0.1:37240 - "POST /node/69407e13575bc060ce15c22c HTTP/1.0" 202 Accepted INFO: 172.21.0.1:37250 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:31:16,587 - lava_callback - INFO - Received callback for node 69407e14575bc060ce15c230 INFO: 172.21.0.1:37262 - "POST /node/69407e14575bc060ce15c230 HTTP/1.0" 202 Accepted 2025-12-17 21:31:16,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0f575bc060ce15c223 2025-12-17 21:31:16,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0f575bc060ce15c223 HTTP/1.1" 502 150 2025-12-17 21:31:16,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0f575bc060ce15c223'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:17,255 - lava_callback - INFO - Received callback for node 69407e16575bc060ce15c233 INFO: 172.21.0.1:37268 - "POST /node/69407e16575bc060ce15c233 HTTP/1.0" 202 Accepted 2025-12-17 21:31:17,361 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df6575bc060ce15c1d0 2025-12-17 21:31:17,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df6575bc060ce15c1d0 HTTP/1.1" 502 150 2025-12-17 21:31:17,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df6575bc060ce15c1d0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:17,376 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a5bd1e3c24e956819d52 2025-12-17 21:31:17,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a5bd1e3c24e956819d52 HTTP/1.1" 502 150 2025-12-17 21:31:17,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a5bd1e3c24e956819d52'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:17,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e10575bc060ce15c226 2025-12-17 21:31:17,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e10575bc060ce15c226 HTTP/1.1" 502 150 2025-12-17 21:31:17,425 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e10575bc060ce15c226'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:17,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407b11575bc060ce15b936 2025-12-17 21:31:17,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407b11575bc060ce15b936 HTTP/1.1" 502 150 2025-12-17 21:31:17,940 - lava_callback - ERROR - Error processing callback for node 69407b11575bc060ce15b936: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b11575bc060ce15b936 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b11575bc060ce15b936 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407b11575bc060ce15b936 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:17,942 - lava_callback - INFO - Starting async job submit for node 69407e12575bc060ce15c229 2025-12-17 21:31:17,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:17,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e12575bc060ce15c229 HTTP/1.1" 502 150 2025-12-17 21:31:17,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e12575bc060ce15c229'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:17,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e12575bc060ce15c229 2025-12-17 21:31:17,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e12575bc060ce15c229 HTTP/1.1" 502 150 2025-12-17 21:31:17,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e12575bc060ce15c229'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:18,143 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dee575bc060ce15c1a2 2025-12-17 21:31:18,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dee575bc060ce15c1a2 HTTP/1.1" 502 150 2025-12-17 21:31:18,144 - lava_callback - ERROR - Error processing callback for node 69407dee575bc060ce15c1a2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dee575bc060ce15c1a2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dee575bc060ce15c1a2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dee575bc060ce15c1a2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:18,146 - lava_callback - INFO - Starting async job submit for node 69407e13575bc060ce15c22c 2025-12-17 21:31:18,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:18,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e13575bc060ce15c22c HTTP/1.1" 502 150 2025-12-17 21:31:18,175 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e13575bc060ce15c22c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:18,175 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e13575bc060ce15c22c 2025-12-17 21:31:18,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e13575bc060ce15c22c HTTP/1.1" 502 150 2025-12-17 21:31:18,175 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e13575bc060ce15c22c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:18,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df9575bc060ce15c1ed 2025-12-17 21:31:18,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df9575bc060ce15c1ed HTTP/1.1" 502 150 2025-12-17 21:31:18,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df9575bc060ce15c1ed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:18,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0d575bc060ce15c21d 2025-12-17 21:31:18,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0d575bc060ce15c21d HTTP/1.1" 502 150 2025-12-17 21:31:18,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0d575bc060ce15c21d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:19,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0e575bc060ce15c220 2025-12-17 21:31:19,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0e575bc060ce15c220 HTTP/1.1" 502 150 2025-12-17 21:31:19,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0e575bc060ce15c220'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:19,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e12575bc060ce15c229 2025-12-17 21:31:19,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e12575bc060ce15c229 HTTP/1.1" 502 150 2025-12-17 21:31:19,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e12575bc060ce15c229'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:20,176 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e13575bc060ce15c22c 2025-12-17 21:31:20,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e13575bc060ce15c22c HTTP/1.1" 502 150 2025-12-17 21:31:20,180 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e13575bc060ce15c22c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:20,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df1575bc060ce15c1ad 2025-12-17 21:31:20,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df1575bc060ce15c1ad HTTP/1.1" 502 150 2025-12-17 21:31:20,695 - lava_callback - ERROR - Error processing callback for node 69407df1575bc060ce15c1ad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df1575bc060ce15c1ad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df1575bc060ce15c1ad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df1575bc060ce15c1ad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:20,698 - lava_callback - INFO - Starting async job submit for node 69407e14575bc060ce15c230 2025-12-17 21:31:20,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:20,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e14575bc060ce15c230 HTTP/1.1" 502 150 2025-12-17 21:31:20,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e14575bc060ce15c230'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:20,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e14575bc060ce15c230 2025-12-17 21:31:20,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e14575bc060ce15c230 HTTP/1.1" 502 150 2025-12-17 21:31:20,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e14575bc060ce15c230'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:20,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0f575bc060ce15c223 2025-12-17 21:31:20,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0f575bc060ce15c223 HTTP/1.1" 502 150 2025-12-17 21:31:20,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0f575bc060ce15c223'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:21,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a5bd1e3c24e956819d52 2025-12-17 21:31:21,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a5bd1e3c24e956819d52 HTTP/1.1" 502 150 2025-12-17 21:31:21,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a5bd1e3c24e956819d52'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:21,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e10575bc060ce15c226 2025-12-17 21:31:21,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e10575bc060ce15c226 HTTP/1.1" 502 150 2025-12-17 21:31:21,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e10575bc060ce15c226'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:21,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df2575bc060ce15c1b4 2025-12-17 21:31:21,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df2575bc060ce15c1b4 HTTP/1.1" 502 150 2025-12-17 21:31:21,542 - lava_callback - ERROR - Error processing callback for node 69407df2575bc060ce15c1b4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df2575bc060ce15c1b4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df2575bc060ce15c1b4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df2575bc060ce15c1b4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:21,543 - lava_callback - INFO - Starting async job submit for node 69407e16575bc060ce15c233 2025-12-17 21:31:21,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:21,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e16575bc060ce15c233 HTTP/1.1" 502 150 2025-12-17 21:31:21,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e16575bc060ce15c233'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:21,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e16575bc060ce15c233 2025-12-17 21:31:21,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e16575bc060ce15c233 HTTP/1.1" 502 150 2025-12-17 21:31:21,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e16575bc060ce15c233'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:21,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dfd575bc060ce15c1fa 2025-12-17 21:31:21,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfd575bc060ce15c1fa HTTP/1.1" 502 150 2025-12-17 21:31:21,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dfd575bc060ce15c1fa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:22,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfa575bc060ce15c1f4 HTTP/1.1" 502 150 2025-12-17 21:31:22,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dfa575bc060ce15c1f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:22,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e14575bc060ce15c230 2025-12-17 21:31:22,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e14575bc060ce15c230 HTTP/1.1" 502 150 2025-12-17 21:31:22,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e14575bc060ce15c230'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:23,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e16575bc060ce15c233 2025-12-17 21:31:23,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e16575bc060ce15c233 HTTP/1.1" 502 150 2025-12-17 21:31:23,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e16575bc060ce15c233'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:23,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df3575bc060ce15c1bb 2025-12-17 21:31:23,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df3575bc060ce15c1bb HTTP/1.1" 502 150 2025-12-17 21:31:23,653 - lava_callback - ERROR - Error processing callback for node 69407df3575bc060ce15c1bb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df3575bc060ce15c1bb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df3575bc060ce15c1bb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df3575bc060ce15c1bb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:23,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df5575bc060ce15c1c0 2025-12-17 21:31:23,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df5575bc060ce15c1c0 HTTP/1.1" 502 150 2025-12-17 21:31:23,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407df5575bc060ce15c1c0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:23,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e12575bc060ce15c229 2025-12-17 21:31:23,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e12575bc060ce15c229 HTTP/1.1" 502 150 2025-12-17 21:31:23,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e12575bc060ce15c229'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:24,181 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e13575bc060ce15c22c 2025-12-17 21:31:24,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e13575bc060ce15c22c HTTP/1.1" 502 150 2025-12-17 21:31:24,182 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e13575bc060ce15c22c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55622 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:31:26,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0d575bc060ce15c21d 2025-12-17 21:31:26,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0d575bc060ce15c21d HTTP/1.1" 502 150 2025-12-17 21:31:26,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0d575bc060ce15c21d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:26,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dfa575bc060ce15c1f4 2025-12-17 21:31:26,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfa575bc060ce15c1f4 HTTP/1.1" 502 150 2025-12-17 21:31:26,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dfa575bc060ce15c1f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:26,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e14575bc060ce15c230 2025-12-17 21:31:26,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e14575bc060ce15c230 HTTP/1.1" 502 150 2025-12-17 21:31:26,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e14575bc060ce15c230'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:27,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfb575bc060ce15c1f7 HTTP/1.1" 502 150 2025-12-17 21:31:27,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dfb575bc060ce15c1f7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:27,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e16575bc060ce15c233 2025-12-17 21:31:27,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e16575bc060ce15c233 HTTP/1.1" 502 150 2025-12-17 21:31:27,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e16575bc060ce15c233'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:27,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0e575bc060ce15c220 2025-12-17 21:31:27,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0e575bc060ce15c220 HTTP/1.1" 502 150 2025-12-17 21:31:27,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0e575bc060ce15c220'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:28,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0f575bc060ce15c223 2025-12-17 21:31:28,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0f575bc060ce15c223 HTTP/1.1" 502 150 2025-12-17 21:31:28,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e0f575bc060ce15c223'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:29,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a5bd1e3c24e956819d52 2025-12-17 21:31:29,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a5bd1e3c24e956819d52 HTTP/1.1" 502 150 2025-12-17 21:31:29,385 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a5bd1e3c24e956819d52'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:29,427 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e10575bc060ce15c226 2025-12-17 21:31:29,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e10575bc060ce15c226 HTTP/1.1" 502 150 2025-12-17 21:31:29,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e10575bc060ce15c226'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:31,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e12575bc060ce15c229 2025-12-17 21:31:31,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e12575bc060ce15c229 HTTP/1.1" 502 150 2025-12-17 21:31:31,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e12575bc060ce15c229'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:32,182 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e13575bc060ce15c22c 2025-12-17 21:31:32,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e13575bc060ce15c22c HTTP/1.1" 502 150 2025-12-17 21:31:32,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e13575bc060ce15c22c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:33,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df6575bc060ce15c1d0 2025-12-17 21:31:33,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df6575bc060ce15c1d0 HTTP/1.1" 502 150 2025-12-17 21:31:33,364 - lava_callback - ERROR - Error processing callback for node 69407df6575bc060ce15c1d0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df6575bc060ce15c1d0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df6575bc060ce15c1d0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df6575bc060ce15c1d0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:34,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df9575bc060ce15c1ed 2025-12-17 21:31:34,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df9575bc060ce15c1ed HTTP/1.1" 502 150 2025-12-17 21:31:34,516 - lava_callback - ERROR - Error processing callback for node 69407df9575bc060ce15c1ed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df9575bc060ce15c1ed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df9575bc060ce15c1ed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df9575bc060ce15c1ed (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:34,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dfa575bc060ce15c1f4 2025-12-17 21:31:34,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfa575bc060ce15c1f4 HTTP/1.1" 502 150 2025-12-17 21:31:34,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dfa575bc060ce15c1f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:34,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e14575bc060ce15c230 2025-12-17 21:31:34,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e14575bc060ce15c230 HTTP/1.1" 502 150 2025-12-17 21:31:34,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e14575bc060ce15c230'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:35,281 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dfb575bc060ce15c1f7 2025-12-17 21:31:35,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfb575bc060ce15c1f7 HTTP/1.1" 502 150 2025-12-17 21:31:35,289 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407dfb575bc060ce15c1f7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:31:35,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e16575bc060ce15c233 2025-12-17 21:31:35,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e16575bc060ce15c233 HTTP/1.1" 502 150 2025-12-17 21:31:35,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e16575bc060ce15c233'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45684 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:31:37,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dfd575bc060ce15c1fa 2025-12-17 21:31:37,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfd575bc060ce15c1fa HTTP/1.1" 502 150 2025-12-17 21:31:37,732 - lava_callback - ERROR - Error processing callback for node 69407dfd575bc060ce15c1fa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dfd575bc060ce15c1fa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dfd575bc060ce15c1fa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dfd575bc060ce15c1fa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:39,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407df5575bc060ce15c1c0 2025-12-17 21:31:39,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407df5575bc060ce15c1c0 HTTP/1.1" 502 150 2025-12-17 21:31:39,928 - lava_callback - ERROR - Error processing callback for node 69407df5575bc060ce15c1c0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df5575bc060ce15c1c0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df5575bc060ce15c1c0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407df5575bc060ce15c1c0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:42,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0d575bc060ce15c21d 2025-12-17 21:31:42,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0d575bc060ce15c21d HTTP/1.1" 502 150 2025-12-17 21:31:42,586 - lava_callback - ERROR - Error processing callback for node 69407e0d575bc060ce15c21d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e0d575bc060ce15c21d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e0d575bc060ce15c21d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e0d575bc060ce15c21d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:31:43,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0e575bc060ce15c220 2025-12-17 21:31:44,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e0f575bc060ce15c223 2025-12-17 21:31:45,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a5bd1e3c24e956819d52 2025-12-17 21:31:45,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e10575bc060ce15c226 INFO: 172.21.0.1:41440 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:31:47,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e12575bc060ce15c229 2025-12-17 21:31:48,184 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e13575bc060ce15c22c 2025-12-17 21:31:50,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dfa575bc060ce15c1f4 2025-12-17 21:31:50,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e14575bc060ce15c230 2025-12-17 21:31:51,290 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407dfb575bc060ce15c1f7 2025-12-17 21:31:51,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e16575bc060ce15c233 2025-12-17 21:31:52,589 - lava_callback - INFO - Received callback for node 69407e40575bc060ce15c2a8 INFO: 172.21.0.1:49404 - "POST /node/69407e40575bc060ce15c2a8 HTTP/1.0" 202 Accepted 2025-12-17 21:31:52,595 - lava_callback - INFO - Starting async job submit for node 69407e40575bc060ce15c2a8 2025-12-17 21:31:52,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:53,263 - lava_callback - INFO - Received callback for node 69407e42575bc060ce15c2b1 INFO: 172.21.0.1:49410 - "POST /node/69407e42575bc060ce15c2b1 HTTP/1.0" 202 Accepted 2025-12-17 21:31:53,269 - lava_callback - INFO - Starting async job submit for node 69407e42575bc060ce15c2b1 2025-12-17 21:31:53,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:53,920 - lava_callback - INFO - Received callback for node 69407e43575bc060ce15c2b4 INFO: 172.21.0.1:49418 - "POST /node/69407e43575bc060ce15c2b4 HTTP/1.0" 202 Accepted 2025-12-17 21:31:53,926 - lava_callback - INFO - Starting async job submit for node 69407e43575bc060ce15c2b4 2025-12-17 21:31:53,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:54,600 - lava_callback - INFO - Received callback for node 69407e45575bc060ce15c2b7 INFO: 172.21.0.1:49428 - "POST /node/69407e45575bc060ce15c2b7 HTTP/1.0" 202 Accepted 2025-12-17 21:31:54,606 - lava_callback - INFO - Starting async job submit for node 69407e45575bc060ce15c2b7 2025-12-17 21:31:54,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:31:55,297 - lava_callback - INFO - Received callback for node 69407e46575bc060ce15c2be INFO: 172.21.0.1:49438 - "POST /node/69407e46575bc060ce15c2be HTTP/1.0" 202 Accepted 2025-12-17 21:31:55,303 - lava_callback - INFO - Starting async job submit for node 69407e46575bc060ce15c2be 2025-12-17 21:31:55,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:49444 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:31:59,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0e575bc060ce15c220 HTTP/1.1" 502 150 2025-12-17 21:31:59,280 - lava_callback - ERROR - Error processing callback for node 69407e0e575bc060ce15c220: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e0e575bc060ce15c220 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e0e575bc060ce15c220 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e0e575bc060ce15c220 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:00,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e0f575bc060ce15c223 HTTP/1.1" 502 150 2025-12-17 21:32:00,304 - lava_callback - ERROR - Error processing callback for node 69407e0f575bc060ce15c223: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e0f575bc060ce15c223 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e0f575bc060ce15c223 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e0f575bc060ce15c223 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:00,364 - lava_callback - INFO - Received callback for node 69407f19575bc060ce15c3db INFO: 172.21.0.1:48182 - "POST /node/69407f19575bc060ce15c3db HTTP/1.0" 202 Accepted 2025-12-17 21:32:00,370 - lava_callback - INFO - Starting async job submit for node 69407f19575bc060ce15c3db 2025-12-17 21:32:00,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:32:00,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f19575bc060ce15c3db HTTP/1.1" 502 150 2025-12-17 21:32:00,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f19575bc060ce15c3db'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:00,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f19575bc060ce15c3db 2025-12-17 21:32:00,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f19575bc060ce15c3db HTTP/1.1" 502 150 2025-12-17 21:32:00,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f19575bc060ce15c3db'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:00,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a5bd1e3c24e956819d52 HTTP/1.1" 502 150 2025-12-17 21:32:00,817 - lava_callback - ERROR - Error processing callback for node 6940a5bd1e3c24e956819d52: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a5bd1e3c24e956819d52 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a5bd1e3c24e956819d52 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a5bd1e3c24e956819d52 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:00,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e10575bc060ce15c226 HTTP/1.1" 502 150 2025-12-17 21:32:00,819 - lava_callback - ERROR - Error processing callback for node 69407e10575bc060ce15c226: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e10575bc060ce15c226 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e10575bc060ce15c226 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e10575bc060ce15c226 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:48184 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:32:02,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f19575bc060ce15c3db 2025-12-17 21:32:02,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f19575bc060ce15c3db HTTP/1.1" 502 150 2025-12-17 21:32:02,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f19575bc060ce15c3db'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:03,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e12575bc060ce15c229 HTTP/1.1" 502 150 2025-12-17 21:32:03,380 - lava_callback - ERROR - Error processing callback for node 69407e12575bc060ce15c229: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e12575bc060ce15c229 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e12575bc060ce15c229 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e12575bc060ce15c229 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:03,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e13575bc060ce15c22c HTTP/1.1" 502 150 2025-12-17 21:32:03,632 - lava_callback - ERROR - Error processing callback for node 69407e13575bc060ce15c22c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e13575bc060ce15c22c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e13575bc060ce15c22c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e13575bc060ce15c22c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:05,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfa575bc060ce15c1f4 HTTP/1.1" 502 150 2025-12-17 21:32:05,936 - lava_callback - ERROR - Error processing callback for node 69407dfa575bc060ce15c1f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dfa575bc060ce15c1f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dfa575bc060ce15c1f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dfa575bc060ce15c1f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:06,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e14575bc060ce15c230 HTTP/1.1" 502 150 2025-12-17 21:32:06,196 - lava_callback - ERROR - Error processing callback for node 69407e14575bc060ce15c230: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e14575bc060ce15c230 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e14575bc060ce15c230 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e14575bc060ce15c230 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:06,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f19575bc060ce15c3db 2025-12-17 21:32:06,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f19575bc060ce15c3db HTTP/1.1" 502 150 2025-12-17 21:32:06,406 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f19575bc060ce15c3db'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48192 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:32:06,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407dfb575bc060ce15c1f7 HTTP/1.1" 502 150 2025-12-17 21:32:06,710 - lava_callback - ERROR - Error processing callback for node 69407dfb575bc060ce15c1f7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dfb575bc060ce15c1f7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dfb575bc060ce15c1f7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407dfb575bc060ce15c1f7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:06,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e16575bc060ce15c233 HTTP/1.1" 502 150 2025-12-17 21:32:06,960 - lava_callback - ERROR - Error processing callback for node 69407e16575bc060ce15c233: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e16575bc060ce15c233 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e16575bc060ce15c233 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e16575bc060ce15c233 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:07,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e40575bc060ce15c2a8 HTTP/1.1" 502 150 2025-12-17 21:32:07,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e40575bc060ce15c2a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:07,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e40575bc060ce15c2a8 2025-12-17 21:32:07,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e40575bc060ce15c2a8 HTTP/1.1" 502 150 2025-12-17 21:32:07,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e40575bc060ce15c2a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:08,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e42575bc060ce15c2b1 HTTP/1.1" 502 150 2025-12-17 21:32:08,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e42575bc060ce15c2b1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:08,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e42575bc060ce15c2b1 2025-12-17 21:32:08,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e42575bc060ce15c2b1 HTTP/1.1" 502 150 2025-12-17 21:32:08,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e42575bc060ce15c2b1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:09,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e43575bc060ce15c2b4 HTTP/1.1" 502 150 2025-12-17 21:32:09,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e43575bc060ce15c2b4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:09,264 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e43575bc060ce15c2b4 2025-12-17 21:32:09,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e43575bc060ce15c2b4 HTTP/1.1" 502 150 2025-12-17 21:32:09,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e43575bc060ce15c2b4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:09,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e40575bc060ce15c2a8 2025-12-17 21:32:09,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e40575bc060ce15c2a8 HTTP/1.1" 502 150 2025-12-17 21:32:09,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e40575bc060ce15c2a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:10,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e45575bc060ce15c2b7 HTTP/1.1" 502 150 2025-12-17 21:32:10,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e45575bc060ce15c2b7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:10,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e45575bc060ce15c2b7 2025-12-17 21:32:10,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e45575bc060ce15c2b7 HTTP/1.1" 502 150 2025-12-17 21:32:10,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e45575bc060ce15c2b7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:10,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e42575bc060ce15c2b1 2025-12-17 21:32:10,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e42575bc060ce15c2b1 HTTP/1.1" 502 150 2025-12-17 21:32:10,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e42575bc060ce15c2b1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:10,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e46575bc060ce15c2be HTTP/1.1" 502 150 2025-12-17 21:32:10,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e46575bc060ce15c2be'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:10,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e46575bc060ce15c2be 2025-12-17 21:32:10,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e46575bc060ce15c2be HTTP/1.1" 502 150 2025-12-17 21:32:10,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e46575bc060ce15c2be'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:11,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e43575bc060ce15c2b4 2025-12-17 21:32:11,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e43575bc060ce15c2b4 HTTP/1.1" 502 150 2025-12-17 21:32:11,267 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e43575bc060ce15c2b4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:12,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e45575bc060ce15c2b7 2025-12-17 21:32:12,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e45575bc060ce15c2b7 HTTP/1.1" 502 150 2025-12-17 21:32:12,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e45575bc060ce15c2b7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:12,571 - lava_callback - INFO - Received callback for node 69407e47575bc060ce15c2c6 INFO: 172.21.0.1:37072 - "POST /node/69407e47575bc060ce15c2c6 HTTP/1.0" 202 Accepted 2025-12-17 21:32:12,577 - lava_callback - INFO - Starting async job submit for node 69407e47575bc060ce15c2c6 2025-12-17 21:32:12,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:32:12,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e47575bc060ce15c2c6 HTTP/1.1" 502 150 2025-12-17 21:32:12,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e47575bc060ce15c2c6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:12,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e47575bc060ce15c2c6 2025-12-17 21:32:12,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e47575bc060ce15c2c6 HTTP/1.1" 502 150 2025-12-17 21:32:12,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e47575bc060ce15c2c6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:12,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e46575bc060ce15c2be 2025-12-17 21:32:12,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e46575bc060ce15c2be HTTP/1.1" 502 150 2025-12-17 21:32:12,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e46575bc060ce15c2be'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:13,232 - lava_callback - INFO - Received callback for node 69407e52575bc060ce15c2ce INFO: 172.21.0.1:37088 - "POST /node/69407e52575bc060ce15c2ce HTTP/1.0" 202 Accepted 2025-12-17 21:32:13,238 - lava_callback - INFO - Starting async job submit for node 69407e52575bc060ce15c2ce 2025-12-17 21:32:13,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:32:13,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e52575bc060ce15c2ce HTTP/1.1" 502 150 2025-12-17 21:32:13,262 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e52575bc060ce15c2ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:13,262 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e52575bc060ce15c2ce 2025-12-17 21:32:13,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e52575bc060ce15c2ce HTTP/1.1" 502 150 2025-12-17 21:32:13,263 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e52575bc060ce15c2ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:13,910 - lava_callback - INFO - Received callback for node 69407e53575bc060ce15c2d5 INFO: 172.21.0.1:37094 - "POST /node/69407e53575bc060ce15c2d5 HTTP/1.0" 202 Accepted 2025-12-17 21:32:13,916 - lava_callback - INFO - Starting async job submit for node 69407e53575bc060ce15c2d5 2025-12-17 21:32:13,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:32:13,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e53575bc060ce15c2d5 HTTP/1.1" 502 150 2025-12-17 21:32:13,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e53575bc060ce15c2d5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:13,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e53575bc060ce15c2d5 2025-12-17 21:32:13,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e53575bc060ce15c2d5 HTTP/1.1" 502 150 2025-12-17 21:32:13,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e53575bc060ce15c2d5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:13,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e40575bc060ce15c2a8 2025-12-17 21:32:13,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e40575bc060ce15c2a8 HTTP/1.1" 502 150 2025-12-17 21:32:13,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e40575bc060ce15c2a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:14,406 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f19575bc060ce15c3db 2025-12-17 21:32:14,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f19575bc060ce15c3db HTTP/1.1" 502 150 2025-12-17 21:32:14,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f19575bc060ce15c3db'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:14,450 - lava_callback - INFO - Received callback for node 69407f1b575bc060ce15c3de INFO: 172.21.0.1:37106 - "POST /node/69407f1b575bc060ce15c3de HTTP/1.0" 202 Accepted 2025-12-17 21:32:14,457 - lava_callback - INFO - Starting async job submit for node 69407f1b575bc060ce15c3de 2025-12-17 21:32:14,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:32:14,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f1b575bc060ce15c3de HTTP/1.1" 502 150 2025-12-17 21:32:14,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f1b575bc060ce15c3de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:14,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f1b575bc060ce15c3de 2025-12-17 21:32:14,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f1b575bc060ce15c3de HTTP/1.1" 502 150 2025-12-17 21:32:14,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f1b575bc060ce15c3de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:14,560 - lava_callback - INFO - Received callback for node 69407e55575bc060ce15c2d8 INFO: 172.21.0.1:37116 - "POST /node/69407e55575bc060ce15c2d8 HTTP/1.0" 202 Accepted 2025-12-17 21:32:14,565 - lava_callback - INFO - Starting async job submit for node 69407e55575bc060ce15c2d8 2025-12-17 21:32:14,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:32:14,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e47575bc060ce15c2c6 2025-12-17 21:32:14,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e47575bc060ce15c2c6 HTTP/1.1" 502 150 2025-12-17 21:32:14,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e47575bc060ce15c2c6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:14,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e55575bc060ce15c2d8 HTTP/1.1" 502 150 2025-12-17 21:32:14,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e55575bc060ce15c2d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:14,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e55575bc060ce15c2d8 2025-12-17 21:32:14,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e55575bc060ce15c2d8 HTTP/1.1" 502 150 2025-12-17 21:32:14,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e55575bc060ce15c2d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:14,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e42575bc060ce15c2b1 2025-12-17 21:32:14,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e42575bc060ce15c2b1 HTTP/1.1" 502 150 2025-12-17 21:32:14,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e42575bc060ce15c2b1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37122 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:32:15,263 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e52575bc060ce15c2ce 2025-12-17 21:32:15,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e52575bc060ce15c2ce HTTP/1.1" 502 150 2025-12-17 21:32:15,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e52575bc060ce15c2ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:15,267 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e43575bc060ce15c2b4 2025-12-17 21:32:15,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e43575bc060ce15c2b4 HTTP/1.1" 502 150 2025-12-17 21:32:15,268 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e43575bc060ce15c2b4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:15,625 - lava_callback - INFO - Received callback for node 69407f1e575bc060ce15c3e1 INFO: 172.21.0.1:37128 - "POST /node/69407f1e575bc060ce15c3e1 HTTP/1.0" 202 Accepted 2025-12-17 21:32:15,631 - lava_callback - INFO - Starting async job submit for node 69407f1e575bc060ce15c3e1 2025-12-17 21:32:15,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:32:15,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f1e575bc060ce15c3e1 HTTP/1.1" 502 150 2025-12-17 21:32:15,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f1e575bc060ce15c3e1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:15,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f1e575bc060ce15c3e1 2025-12-17 21:32:15,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f1e575bc060ce15c3e1 HTTP/1.1" 502 150 2025-12-17 21:32:15,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f1e575bc060ce15c3e1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:15,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e53575bc060ce15c2d5 2025-12-17 21:32:15,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e53575bc060ce15c2d5 HTTP/1.1" 502 150 2025-12-17 21:32:15,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e53575bc060ce15c2d5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:16,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e45575bc060ce15c2b7 2025-12-17 21:32:16,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e45575bc060ce15c2b7 HTTP/1.1" 502 150 2025-12-17 21:32:16,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e45575bc060ce15c2b7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:16,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f1b575bc060ce15c3de 2025-12-17 21:32:16,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f1b575bc060ce15c3de HTTP/1.1" 502 150 2025-12-17 21:32:16,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f1b575bc060ce15c3de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37138 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:32:16,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e55575bc060ce15c2d8 2025-12-17 21:32:16,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e55575bc060ce15c2d8 HTTP/1.1" 502 150 2025-12-17 21:32:16,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e55575bc060ce15c2d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:16,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e46575bc060ce15c2be 2025-12-17 21:32:16,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e46575bc060ce15c2be HTTP/1.1" 502 150 2025-12-17 21:32:16,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e46575bc060ce15c2be'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:17,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f1e575bc060ce15c3e1 2025-12-17 21:32:17,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f1e575bc060ce15c3e1 HTTP/1.1" 502 150 2025-12-17 21:32:17,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f1e575bc060ce15c3e1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:18,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e47575bc060ce15c2c6 2025-12-17 21:32:18,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e47575bc060ce15c2c6 HTTP/1.1" 502 150 2025-12-17 21:32:18,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e47575bc060ce15c2c6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:19,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e52575bc060ce15c2ce 2025-12-17 21:32:19,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e52575bc060ce15c2ce HTTP/1.1" 502 150 2025-12-17 21:32:19,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e52575bc060ce15c2ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:19,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e53575bc060ce15c2d5 2025-12-17 21:32:19,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e53575bc060ce15c2d5 HTTP/1.1" 502 150 2025-12-17 21:32:19,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e53575bc060ce15c2d5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:20,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f1b575bc060ce15c3de 2025-12-17 21:32:20,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f1b575bc060ce15c3de HTTP/1.1" 502 150 2025-12-17 21:32:20,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f1b575bc060ce15c3de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:20,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e55575bc060ce15c2d8 2025-12-17 21:32:20,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e55575bc060ce15c2d8 HTTP/1.1" 502 150 2025-12-17 21:32:20,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e55575bc060ce15c2d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:21,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f1e575bc060ce15c3e1 2025-12-17 21:32:21,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f1e575bc060ce15c3e1 HTTP/1.1" 502 150 2025-12-17 21:32:21,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f1e575bc060ce15c3e1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:21,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e40575bc060ce15c2a8 2025-12-17 21:32:21,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e40575bc060ce15c2a8 HTTP/1.1" 502 150 2025-12-17 21:32:21,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e40575bc060ce15c2a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:22,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e42575bc060ce15c2b1 2025-12-17 21:32:22,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e42575bc060ce15c2b1 HTTP/1.1" 502 150 2025-12-17 21:32:22,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e42575bc060ce15c2b1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:23,268 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e43575bc060ce15c2b4 2025-12-17 21:32:23,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e43575bc060ce15c2b4 HTTP/1.1" 502 150 2025-12-17 21:32:23,269 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e43575bc060ce15c2b4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:24,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e45575bc060ce15c2b7 2025-12-17 21:32:24,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e45575bc060ce15c2b7 HTTP/1.1" 502 150 2025-12-17 21:32:24,039 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e45575bc060ce15c2b7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:24,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e46575bc060ce15c2be 2025-12-17 21:32:24,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e46575bc060ce15c2be HTTP/1.1" 502 150 2025-12-17 21:32:24,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e46575bc060ce15c2be'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35920 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:32:26,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e47575bc060ce15c2c6 2025-12-17 21:32:26,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e47575bc060ce15c2c6 HTTP/1.1" 502 150 2025-12-17 21:32:26,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e47575bc060ce15c2c6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:27,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e52575bc060ce15c2ce 2025-12-17 21:32:27,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e52575bc060ce15c2ce HTTP/1.1" 502 150 2025-12-17 21:32:27,267 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e52575bc060ce15c2ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:27,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e53575bc060ce15c2d5 2025-12-17 21:32:27,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e53575bc060ce15c2d5 HTTP/1.1" 502 150 2025-12-17 21:32:27,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e53575bc060ce15c2d5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:28,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f1b575bc060ce15c3de 2025-12-17 21:32:28,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f1b575bc060ce15c3de HTTP/1.1" 502 150 2025-12-17 21:32:28,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f1b575bc060ce15c3de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:28,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e55575bc060ce15c2d8 2025-12-17 21:32:28,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e55575bc060ce15c2d8 HTTP/1.1" 502 150 2025-12-17 21:32:28,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e55575bc060ce15c2d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:29,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f1e575bc060ce15c3e1 2025-12-17 21:32:29,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f1e575bc060ce15c3e1 HTTP/1.1" 502 150 2025-12-17 21:32:29,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f1e575bc060ce15c3e1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:29,929 - lava_callback - INFO - Received callback for node 69407f36575bc060ce15c415 INFO: 172.21.0.1:55170 - "POST /node/69407f36575bc060ce15c415 HTTP/1.0" 202 Accepted 2025-12-17 21:32:29,935 - lava_callback - INFO - Starting async job submit for node 69407f36575bc060ce15c415 2025-12-17 21:32:29,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:32:29,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f36575bc060ce15c415 HTTP/1.1" 502 150 2025-12-17 21:32:29,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f36575bc060ce15c415'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:29,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f36575bc060ce15c415 2025-12-17 21:32:29,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f36575bc060ce15c415 HTTP/1.1" 502 150 2025-12-17 21:32:29,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f36575bc060ce15c415'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:30,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f19575bc060ce15c3db 2025-12-17 21:32:30,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f19575bc060ce15c3db HTTP/1.1" 502 150 2025-12-17 21:32:30,411 - lava_callback - ERROR - Error processing callback for node 69407f19575bc060ce15c3db: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f19575bc060ce15c3db (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f19575bc060ce15c3db (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f19575bc060ce15c3db (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:31,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f36575bc060ce15c415 2025-12-17 21:32:31,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f36575bc060ce15c415 HTTP/1.1" 502 150 2025-12-17 21:32:31,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f36575bc060ce15c415'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:35,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f36575bc060ce15c415 2025-12-17 21:32:35,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f36575bc060ce15c415 HTTP/1.1" 502 150 2025-12-17 21:32:35,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f36575bc060ce15c415'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55174 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:32:37,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e40575bc060ce15c2a8 2025-12-17 21:32:37,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e40575bc060ce15c2a8 HTTP/1.1" 502 150 2025-12-17 21:32:37,991 - lava_callback - ERROR - Error processing callback for node 69407e40575bc060ce15c2a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e40575bc060ce15c2a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e40575bc060ce15c2a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e40575bc060ce15c2a8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:38,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e42575bc060ce15c2b1 2025-12-17 21:32:38,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e42575bc060ce15c2b1 HTTP/1.1" 502 150 2025-12-17 21:32:38,760 - lava_callback - ERROR - Error processing callback for node 69407e42575bc060ce15c2b1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e42575bc060ce15c2b1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e42575bc060ce15c2b1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e42575bc060ce15c2b1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:39,269 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e43575bc060ce15c2b4 2025-12-17 21:32:39,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e43575bc060ce15c2b4 HTTP/1.1" 502 150 2025-12-17 21:32:39,271 - lava_callback - ERROR - Error processing callback for node 69407e43575bc060ce15c2b4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e43575bc060ce15c2b4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e43575bc060ce15c2b4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e43575bc060ce15c2b4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:40,040 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e45575bc060ce15c2b7 2025-12-17 21:32:40,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e45575bc060ce15c2b7 HTTP/1.1" 502 150 2025-12-17 21:32:40,041 - lava_callback - ERROR - Error processing callback for node 69407e45575bc060ce15c2b7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e45575bc060ce15c2b7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e45575bc060ce15c2b7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e45575bc060ce15c2b7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:40,678 - lava_callback - INFO - Received callback for node 6940c52e1e3c24e95681d696 INFO: 172.21.0.1:44676 - "POST /node/6940c52e1e3c24e95681d696 HTTP/1.0" 202 Accepted 2025-12-17 21:32:40,685 - lava_callback - INFO - Starting async job submit for node 6940c52e1e3c24e95681d696 2025-12-17 21:32:40,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:32:40,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c52e1e3c24e95681d696 HTTP/1.1" 502 150 2025-12-17 21:32:40,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c52e1e3c24e95681d696'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:40,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c52e1e3c24e95681d696 2025-12-17 21:32:40,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c52e1e3c24e95681d696 HTTP/1.1" 502 150 2025-12-17 21:32:40,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c52e1e3c24e95681d696'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:40,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e46575bc060ce15c2be 2025-12-17 21:32:40,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e46575bc060ce15c2be HTTP/1.1" 502 150 2025-12-17 21:32:40,807 - lava_callback - ERROR - Error processing callback for node 69407e46575bc060ce15c2be: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e46575bc060ce15c2be (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e46575bc060ce15c2be (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e46575bc060ce15c2be (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:42,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e47575bc060ce15c2c6 2025-12-17 21:32:42,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e47575bc060ce15c2c6 HTTP/1.1" 502 150 2025-12-17 21:32:42,589 - lava_callback - ERROR - Error processing callback for node 69407e47575bc060ce15c2c6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e47575bc060ce15c2c6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e47575bc060ce15c2c6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e47575bc060ce15c2c6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:42,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c52e1e3c24e95681d696 2025-12-17 21:32:42,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c52e1e3c24e95681d696 HTTP/1.1" 502 150 2025-12-17 21:32:42,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c52e1e3c24e95681d696'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:43,267 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e52575bc060ce15c2ce 2025-12-17 21:32:43,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e52575bc060ce15c2ce HTTP/1.1" 502 150 2025-12-17 21:32:43,268 - lava_callback - ERROR - Error processing callback for node 69407e52575bc060ce15c2ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e52575bc060ce15c2ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e52575bc060ce15c2ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e52575bc060ce15c2ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:43,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e53575bc060ce15c2d5 2025-12-17 21:32:43,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e53575bc060ce15c2d5 HTTP/1.1" 502 150 2025-12-17 21:32:43,929 - lava_callback - ERROR - Error processing callback for node 69407e53575bc060ce15c2d5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e53575bc060ce15c2d5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e53575bc060ce15c2d5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e53575bc060ce15c2d5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:43,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f36575bc060ce15c415 2025-12-17 21:32:43,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f36575bc060ce15c415 HTTP/1.1" 502 150 2025-12-17 21:32:43,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f36575bc060ce15c415'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:44,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f1b575bc060ce15c3de 2025-12-17 21:32:44,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f1b575bc060ce15c3de HTTP/1.1" 502 150 2025-12-17 21:32:44,488 - lava_callback - ERROR - Error processing callback for node 69407f1b575bc060ce15c3de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f1b575bc060ce15c3de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f1b575bc060ce15c3de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f1b575bc060ce15c3de (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:44,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e55575bc060ce15c2d8 2025-12-17 21:32:44,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e55575bc060ce15c2d8 HTTP/1.1" 502 150 2025-12-17 21:32:44,604 - lava_callback - ERROR - Error processing callback for node 69407e55575bc060ce15c2d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e55575bc060ce15c2d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e55575bc060ce15c2d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e55575bc060ce15c2d8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:32:45,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f1e575bc060ce15c3e1 2025-12-17 21:32:45,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f1e575bc060ce15c3e1 HTTP/1.1" 502 150 2025-12-17 21:32:45,661 - lava_callback - ERROR - Error processing callback for node 69407f1e575bc060ce15c3e1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f1e575bc060ce15c3e1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f1e575bc060ce15c3e1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f1e575bc060ce15c3e1 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:44692 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:32:46,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c52e1e3c24e95681d696 2025-12-17 21:32:46,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c52e1e3c24e95681d696 HTTP/1.1" 502 150 2025-12-17 21:32:46,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c52e1e3c24e95681d696'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:50,247 - lava_callback - INFO - Received callback for node 69407f36575bc060ce15c41a INFO: 172.21.0.1:41392 - "POST /node/69407f36575bc060ce15c41a HTTP/1.0" 202 Accepted 2025-12-17 21:32:50,253 - lava_callback - INFO - Starting async job submit for node 69407f36575bc060ce15c41a 2025-12-17 21:32:50,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:32:50,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f36575bc060ce15c41a HTTP/1.1" 502 150 2025-12-17 21:32:50,262 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f36575bc060ce15c41a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:50,263 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f36575bc060ce15c41a 2025-12-17 21:32:50,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f36575bc060ce15c41a HTTP/1.1" 502 150 2025-12-17 21:32:50,263 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f36575bc060ce15c41a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:51,760 - lava_callback - INFO - Received callback for node 69407f37575bc060ce15c41f INFO: 172.21.0.1:41398 - "POST /node/69407f37575bc060ce15c41f HTTP/1.0" 202 Accepted 2025-12-17 21:32:51,765 - lava_callback - INFO - Starting async job submit for node 69407f37575bc060ce15c41f 2025-12-17 21:32:51,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:32:52,264 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f36575bc060ce15c41a 2025-12-17 21:32:52,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f36575bc060ce15c41a HTTP/1.1" 502 150 2025-12-17 21:32:52,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f36575bc060ce15c41a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:54,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c52e1e3c24e95681d696 2025-12-17 21:32:54,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c52e1e3c24e95681d696 HTTP/1.1" 502 150 2025-12-17 21:32:54,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c52e1e3c24e95681d696'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:32:56,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f36575bc060ce15c41a 2025-12-17 21:32:56,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f36575bc060ce15c41a HTTP/1.1" 502 150 2025-12-17 21:32:56,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f36575bc060ce15c41a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41412 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:32:59,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f36575bc060ce15c415 2025-12-17 21:32:59,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f36575bc060ce15c415 HTTP/1.1" 502 150 2025-12-17 21:32:59,976 - lava_callback - ERROR - Error processing callback for node 69407f36575bc060ce15c415: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f36575bc060ce15c415 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f36575bc060ce15c415 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f36575bc060ce15c415 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:52150 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:33:04,267 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f36575bc060ce15c41a 2025-12-17 21:33:04,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f36575bc060ce15c41a HTTP/1.1" 502 150 2025-12-17 21:33:04,268 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f36575bc060ce15c41a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52158 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:33:07,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f37575bc060ce15c41f HTTP/1.1" 502 150 2025-12-17 21:33:07,120 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f37575bc060ce15c41f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:33:07,120 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f37575bc060ce15c41f 2025-12-17 21:33:07,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f37575bc060ce15c41f HTTP/1.1" 502 150 2025-12-17 21:33:07,121 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f37575bc060ce15c41f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:33:09,121 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f37575bc060ce15c41f 2025-12-17 21:33:09,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f37575bc060ce15c41f HTTP/1.1" 502 150 2025-12-17 21:33:09,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f37575bc060ce15c41f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:33:10,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c52e1e3c24e95681d696 2025-12-17 21:33:10,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c52e1e3c24e95681d696 HTTP/1.1" 502 150 2025-12-17 21:33:10,751 - lava_callback - ERROR - Error processing callback for node 6940c52e1e3c24e95681d696: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c52e1e3c24e95681d696 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c52e1e3c24e95681d696 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c52e1e3c24e95681d696 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:33:13,123 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f37575bc060ce15c41f INFO: 172.21.0.1:53580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53592 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:33:20,268 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f36575bc060ce15c41a 2025-12-17 21:33:20,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f36575bc060ce15c41a HTTP/1.1" 502 150 2025-12-17 21:33:20,270 - lava_callback - ERROR - Error processing callback for node 69407f36575bc060ce15c41a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f36575bc060ce15c41a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f36575bc060ce15c41a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f36575bc060ce15c41a (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:42582 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:33:28,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f37575bc060ce15c41f HTTP/1.1" 502 150 2025-12-17 21:33:28,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f37575bc060ce15c41f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:33:36,369 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f37575bc060ce15c41f 2025-12-17 21:33:36,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f37575bc060ce15c41f HTTP/1.1" 502 150 2025-12-17 21:33:36,370 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f37575bc060ce15c41f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33006 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:33:52,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f37575bc060ce15c41f 2025-12-17 21:33:52,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f37575bc060ce15c41f HTTP/1.1" 502 150 2025-12-17 21:33:52,371 - lava_callback - ERROR - Error processing callback for node 69407f37575bc060ce15c41f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f37575bc060ce15c41f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f37575bc060ce15c41f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f37575bc060ce15c41f (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:52056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43986 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:34:05,310 - lava_callback - INFO - Received callback for node 6940aa071e3c24e95681a369 INFO: 172.21.0.1:43998 - "POST /node/6940aa071e3c24e95681a369 HTTP/1.0" 202 Accepted 2025-12-17 21:34:05,318 - lava_callback - INFO - Starting async job submit for node 6940aa071e3c24e95681a369 2025-12-17 21:34:05,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:44004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44850 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:34:20,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940aa071e3c24e95681a369 HTTP/1.1" 502 150 2025-12-17 21:34:20,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940aa071e3c24e95681a369'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:34:20,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940aa071e3c24e95681a369 2025-12-17 21:34:20,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940aa071e3c24e95681a369 HTTP/1.1" 502 150 2025-12-17 21:34:20,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940aa071e3c24e95681a369'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:34:22,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940aa071e3c24e95681a369 2025-12-17 21:34:22,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940aa071e3c24e95681a369 HTTP/1.1" 502 150 2025-12-17 21:34:22,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940aa071e3c24e95681a369'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:34:23,131 - lava_callback - INFO - Received callback for node 69407f75575bc060ce15c47a INFO: 172.21.0.1:33876 - "POST /node/69407f75575bc060ce15c47a HTTP/1.0" 202 Accepted 2025-12-17 21:34:23,137 - lava_callback - INFO - Starting async job submit for node 69407f75575bc060ce15c47a 2025-12-17 21:34:23,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:34:23,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f75575bc060ce15c47a HTTP/1.1" 502 150 2025-12-17 21:34:23,160 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f75575bc060ce15c47a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:34:23,160 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f75575bc060ce15c47a 2025-12-17 21:34:23,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f75575bc060ce15c47a HTTP/1.1" 502 150 2025-12-17 21:34:23,160 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f75575bc060ce15c47a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:34:25,161 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f75575bc060ce15c47a 2025-12-17 21:34:25,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f75575bc060ce15c47a HTTP/1.1" 502 150 2025-12-17 21:34:25,162 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f75575bc060ce15c47a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33880 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:34:26,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940aa071e3c24e95681a369 2025-12-17 21:34:26,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940aa071e3c24e95681a369 HTTP/1.1" 502 150 2025-12-17 21:34:26,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940aa071e3c24e95681a369'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:34:28,981 - lava_callback - INFO - Received callback for node 69407f6d575bc060ce15c470 INFO: 172.21.0.1:33890 - "POST /node/69407f6d575bc060ce15c470 HTTP/1.0" 202 Accepted 2025-12-17 21:34:28,987 - lava_callback - INFO - Starting async job submit for node 69407f6d575bc060ce15c470 2025-12-17 21:34:29,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:34:29,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f6d575bc060ce15c470 HTTP/1.1" 502 150 2025-12-17 21:34:29,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f6d575bc060ce15c470'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:34:29,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f6d575bc060ce15c470 2025-12-17 21:34:29,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f6d575bc060ce15c470 HTTP/1.1" 502 150 2025-12-17 21:34:29,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f6d575bc060ce15c470'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:34:29,162 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f75575bc060ce15c47a 2025-12-17 21:34:29,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f75575bc060ce15c47a HTTP/1.1" 502 150 2025-12-17 21:34:29,164 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f75575bc060ce15c47a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:34:31,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f6d575bc060ce15c470 2025-12-17 21:34:31,926 - lava_callback - INFO - Received callback for node 69407f77575bc060ce15c47d INFO: 172.21.0.1:46632 - "POST /node/69407f77575bc060ce15c47d HTTP/1.0" 202 Accepted 2025-12-17 21:34:31,932 - lava_callback - INFO - Starting async job submit for node 69407f77575bc060ce15c47d 2025-12-17 21:34:31,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:34:31,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f77575bc060ce15c47d HTTP/1.1" 502 150 2025-12-17 21:34:31,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f77575bc060ce15c47d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:34:31,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f77575bc060ce15c47d 2025-12-17 21:34:31,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f77575bc060ce15c47d HTTP/1.1" 502 150 2025-12-17 21:34:31,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f77575bc060ce15c47d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:34:33,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f77575bc060ce15c47d 2025-12-17 21:34:33,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f77575bc060ce15c47d HTTP/1.1" 502 150 2025-12-17 21:34:33,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f77575bc060ce15c47d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:34:34,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940aa071e3c24e95681a369 2025-12-17 21:34:34,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940aa071e3c24e95681a369 HTTP/1.1" 502 150 2025-12-17 21:34:34,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940aa071e3c24e95681a369'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46648 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:34:37,164 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f75575bc060ce15c47a 2025-12-17 21:34:37,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f75575bc060ce15c47a HTTP/1.1" 502 150 2025-12-17 21:34:37,165 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f75575bc060ce15c47a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:34:37,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f77575bc060ce15c47d 2025-12-17 21:34:37,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f77575bc060ce15c47d HTTP/1.1" 502 150 2025-12-17 21:34:37,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f77575bc060ce15c47d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:34:45,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f77575bc060ce15c47d 2025-12-17 21:34:45,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f77575bc060ce15c47d HTTP/1.1" 502 150 2025-12-17 21:34:45,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f77575bc060ce15c47d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:34:46,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f6d575bc060ce15c470 HTTP/1.1" 502 150 2025-12-17 21:34:46,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f6d575bc060ce15c470'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39904 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:34:50,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f6d575bc060ce15c470 2025-12-17 21:34:50,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f6d575bc060ce15c470 HTTP/1.1" 502 150 2025-12-17 21:34:50,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f6d575bc060ce15c470'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:34:50,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940aa071e3c24e95681a369 2025-12-17 21:34:50,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940aa071e3c24e95681a369 HTTP/1.1" 502 150 2025-12-17 21:34:50,855 - lava_callback - ERROR - Error processing callback for node 6940aa071e3c24e95681a369: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940aa071e3c24e95681a369 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940aa071e3c24e95681a369 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940aa071e3c24e95681a369 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:34:53,165 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f75575bc060ce15c47a 2025-12-17 21:34:53,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f75575bc060ce15c47a HTTP/1.1" 502 150 2025-12-17 21:34:53,166 - lava_callback - ERROR - Error processing callback for node 69407f75575bc060ce15c47a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f75575bc060ce15c47a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f75575bc060ce15c47a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f75575bc060ce15c47a (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:43506 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:34:58,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f6d575bc060ce15c470 2025-12-17 21:34:58,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f6d575bc060ce15c470 HTTP/1.1" 502 150 2025-12-17 21:34:58,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f6d575bc060ce15c470'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35980 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:35:01,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f77575bc060ce15c47d 2025-12-17 21:35:01,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f77575bc060ce15c47d HTTP/1.1" 502 150 2025-12-17 21:35:01,979 - lava_callback - ERROR - Error processing callback for node 69407f77575bc060ce15c47d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f77575bc060ce15c47d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f77575bc060ce15c47d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f77575bc060ce15c47d (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:35994 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:35:14,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f6d575bc060ce15c470 2025-12-17 21:35:14,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f6d575bc060ce15c470 HTTP/1.1" 502 150 2025-12-17 21:35:14,461 - lava_callback - ERROR - Error processing callback for node 69407f6d575bc060ce15c470: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f6d575bc060ce15c470 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f6d575bc060ce15c470 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f6d575bc060ce15c470 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:52976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36760 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:35:34,392 - lava_callback - INFO - Received callback for node 69407f88575bc060ce15c492 INFO: 172.21.0.1:39684 - "POST /node/69407f88575bc060ce15c492 HTTP/1.0" 202 Accepted 2025-12-17 21:35:34,403 - lava_callback - INFO - Starting async job submit for node 69407f88575bc060ce15c492 2025-12-17 21:35:34,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:39688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47584 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:35:49,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f88575bc060ce15c492 HTTP/1.1" 502 150 2025-12-17 21:35:49,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f88575bc060ce15c492'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:35:49,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f88575bc060ce15c492 2025-12-17 21:35:49,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f88575bc060ce15c492 HTTP/1.1" 502 150 2025-12-17 21:35:49,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f88575bc060ce15c492'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:35:51,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f88575bc060ce15c492 2025-12-17 21:35:51,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f88575bc060ce15c492 HTTP/1.1" 502 150 2025-12-17 21:35:51,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f88575bc060ce15c492'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:35:53,759 - lava_callback - INFO - Received callback for node 69407f8a575bc060ce15c495 INFO: 172.21.0.1:45722 - "POST /node/69407f8a575bc060ce15c495 HTTP/1.0" 202 Accepted 2025-12-17 21:35:53,765 - lava_callback - INFO - Starting async job submit for node 69407f8a575bc060ce15c495 2025-12-17 21:35:53,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:35:53,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f8a575bc060ce15c495 HTTP/1.1" 502 150 2025-12-17 21:35:53,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f8a575bc060ce15c495'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:35:53,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f8a575bc060ce15c495 2025-12-17 21:35:53,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f8a575bc060ce15c495 HTTP/1.1" 502 150 2025-12-17 21:35:53,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f8a575bc060ce15c495'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:35:55,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f8a575bc060ce15c495 2025-12-17 21:35:55,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f8a575bc060ce15c495 HTTP/1.1" 502 150 2025-12-17 21:35:55,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f8a575bc060ce15c495'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:35:55,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f88575bc060ce15c492 2025-12-17 21:35:55,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f88575bc060ce15c492 HTTP/1.1" 502 150 2025-12-17 21:35:55,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f88575bc060ce15c492'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45734 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:35:59,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f8a575bc060ce15c495 2025-12-17 21:35:59,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f8a575bc060ce15c495 HTTP/1.1" 502 150 2025-12-17 21:35:59,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f8a575bc060ce15c495'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39342 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:36:03,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f88575bc060ce15c492 2025-12-17 21:36:03,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f88575bc060ce15c492 HTTP/1.1" 502 150 2025-12-17 21:36:03,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f88575bc060ce15c492'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39358 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:36:07,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f8a575bc060ce15c495 2025-12-17 21:36:07,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f8a575bc060ce15c495 HTTP/1.1" 502 150 2025-12-17 21:36:07,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f8a575bc060ce15c495'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56518 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:36:19,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f88575bc060ce15c492 2025-12-17 21:36:19,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f88575bc060ce15c492 HTTP/1.1" 502 150 2025-12-17 21:36:19,944 - lava_callback - ERROR - Error processing callback for node 69407f88575bc060ce15c492: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f88575bc060ce15c492 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f88575bc060ce15c492 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f88575bc060ce15c492 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:36:23,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f8a575bc060ce15c495 2025-12-17 21:36:23,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f8a575bc060ce15c495 HTTP/1.1" 502 150 2025-12-17 21:36:23,807 - lava_callback - ERROR - Error processing callback for node 69407f8a575bc060ce15c495: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f8a575bc060ce15c495 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f8a575bc060ce15c495 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f8a575bc060ce15c495 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:36222 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:36:26,870 - lava_callback - INFO - Received callback for node 6940812b575bc060ce15ca22 INFO: 172.21.0.1:36234 - "POST /node/6940812b575bc060ce15ca22 HTTP/1.0" 202 Accepted 2025-12-17 21:36:26,876 - lava_callback - INFO - Starting async job submit for node 6940812b575bc060ce15ca22 2025-12-17 21:36:26,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:36:26,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940812b575bc060ce15ca22 HTTP/1.1" 502 150 2025-12-17 21:36:26,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940812b575bc060ce15ca22'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:26,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940812b575bc060ce15ca22 2025-12-17 21:36:26,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940812b575bc060ce15ca22 HTTP/1.1" 502 150 2025-12-17 21:36:26,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940812b575bc060ce15ca22'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:28,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940812b575bc060ce15ca22 2025-12-17 21:36:28,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940812b575bc060ce15ca22 HTTP/1.1" 502 150 2025-12-17 21:36:28,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940812b575bc060ce15ca22'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:32,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940812b575bc060ce15ca22 2025-12-17 21:36:32,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940812b575bc060ce15ca22 HTTP/1.1" 502 150 2025-12-17 21:36:32,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940812b575bc060ce15ca22'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37468 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:36:40,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940812b575bc060ce15ca22 2025-12-17 21:36:40,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940812b575bc060ce15ca22 HTTP/1.1" 502 150 2025-12-17 21:36:40,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940812b575bc060ce15ca22'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41136 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:36:47,966 - lava_callback - INFO - Received callback for node 69407f5d575bc060ce15c432 INFO: 172.21.0.1:41152 - "POST /node/69407f5d575bc060ce15c432 HTTP/1.0" 202 Accepted 2025-12-17 21:36:47,972 - lava_callback - INFO - Starting async job submit for node 69407f5d575bc060ce15c432 2025-12-17 21:36:47,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:36:47,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f5d575bc060ce15c432 HTTP/1.1" 502 150 2025-12-17 21:36:47,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f5d575bc060ce15c432'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:47,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f5d575bc060ce15c432 2025-12-17 21:36:47,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f5d575bc060ce15c432 HTTP/1.1" 502 150 2025-12-17 21:36:47,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f5d575bc060ce15c432'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:48,512 - lava_callback - INFO - Received callback for node 6940ad311e3c24e95681a9db INFO: 172.21.0.1:41162 - "POST /node/6940ad311e3c24e95681a9db HTTP/1.0" 202 Accepted 2025-12-17 21:36:48,519 - lava_callback - INFO - Starting async job submit for node 6940ad311e3c24e95681a9db 2025-12-17 21:36:48,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:36:48,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ad311e3c24e95681a9db HTTP/1.1" 502 150 2025-12-17 21:36:48,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ad311e3c24e95681a9db'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:48,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ad311e3c24e95681a9db 2025-12-17 21:36:48,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ad311e3c24e95681a9db HTTP/1.1" 502 150 2025-12-17 21:36:48,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ad311e3c24e95681a9db'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:48,676 - lava_callback - INFO - Received callback for node 69407f5f575bc060ce15c437 INFO: 172.21.0.1:41170 - "POST /node/69407f5f575bc060ce15c437 HTTP/1.0" 202 Accepted 2025-12-17 21:36:48,682 - lava_callback - INFO - Starting async job submit for node 69407f5f575bc060ce15c437 2025-12-17 21:36:48,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:36:48,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f5f575bc060ce15c437 HTTP/1.1" 502 150 2025-12-17 21:36:48,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f5f575bc060ce15c437'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:48,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f5f575bc060ce15c437 2025-12-17 21:36:48,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f5f575bc060ce15c437 HTTP/1.1" 502 150 2025-12-17 21:36:48,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f5f575bc060ce15c437'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:49,325 - lava_callback - INFO - Received callback for node 69407f60575bc060ce15c43e INFO: 172.21.0.1:41180 - "POST /node/69407f60575bc060ce15c43e HTTP/1.0" 202 Accepted 2025-12-17 21:36:49,332 - lava_callback - INFO - Starting async job submit for node 69407f60575bc060ce15c43e 2025-12-17 21:36:49,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:36:49,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f60575bc060ce15c43e HTTP/1.1" 502 150 2025-12-17 21:36:49,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f60575bc060ce15c43e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:49,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f60575bc060ce15c43e 2025-12-17 21:36:49,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f60575bc060ce15c43e HTTP/1.1" 502 150 2025-12-17 21:36:49,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f60575bc060ce15c43e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:49,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f5d575bc060ce15c432 2025-12-17 21:36:49,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f5d575bc060ce15c432 HTTP/1.1" 502 150 2025-12-17 21:36:49,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f5d575bc060ce15c432'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:49,999 - lava_callback - INFO - Received callback for node 69407f61575bc060ce15c445 INFO: 172.21.0.1:58762 - "POST /node/69407f61575bc060ce15c445 HTTP/1.0" 202 Accepted 2025-12-17 21:36:50,006 - lava_callback - INFO - Starting async job submit for node 69407f61575bc060ce15c445 2025-12-17 21:36:50,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:36:50,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f61575bc060ce15c445 HTTP/1.1" 502 150 2025-12-17 21:36:50,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f61575bc060ce15c445'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:50,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f61575bc060ce15c445 2025-12-17 21:36:50,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f61575bc060ce15c445 HTTP/1.1" 502 150 2025-12-17 21:36:50,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f61575bc060ce15c445'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:50,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ad311e3c24e95681a9db 2025-12-17 21:36:50,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ad311e3c24e95681a9db HTTP/1.1" 502 150 2025-12-17 21:36:50,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ad311e3c24e95681a9db'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:50,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f5f575bc060ce15c437 2025-12-17 21:36:50,691 - lava_callback - INFO - Received callback for node 69407f62575bc060ce15c448 2025-12-17 21:36:50,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f5f575bc060ce15c437 HTTP/1.1" 502 150 2025-12-17 21:36:50,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f5f575bc060ce15c437'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58768 - "POST /node/69407f62575bc060ce15c448 HTTP/1.0" 202 Accepted 2025-12-17 21:36:50,700 - lava_callback - INFO - Starting async job submit for node 69407f62575bc060ce15c448 2025-12-17 21:36:50,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:36:50,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f62575bc060ce15c448 HTTP/1.1" 502 150 2025-12-17 21:36:50,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f62575bc060ce15c448'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:50,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f62575bc060ce15c448 2025-12-17 21:36:50,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f62575bc060ce15c448 HTTP/1.1" 502 150 2025-12-17 21:36:50,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f62575bc060ce15c448'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:51,346 - lava_callback - INFO - Received callback for node 69407f64575bc060ce15c44b 2025-12-17 21:36:51,352 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f60575bc060ce15c43e INFO: 172.21.0.1:58776 - "POST /node/69407f64575bc060ce15c44b HTTP/1.0" 202 Accepted 2025-12-17 21:36:51,356 - lava_callback - INFO - Starting async job submit for node 69407f64575bc060ce15c44b 2025-12-17 21:36:51,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f60575bc060ce15c43e HTTP/1.1" 502 150 2025-12-17 21:36:51,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f60575bc060ce15c43e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:51,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:36:51,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f64575bc060ce15c44b HTTP/1.1" 502 150 2025-12-17 21:36:51,367 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f64575bc060ce15c44b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:51,367 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f64575bc060ce15c44b 2025-12-17 21:36:51,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f64575bc060ce15c44b HTTP/1.1" 502 150 2025-12-17 21:36:51,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f64575bc060ce15c44b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:52,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f61575bc060ce15c445 2025-12-17 21:36:52,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f61575bc060ce15c445 HTTP/1.1" 502 150 2025-12-17 21:36:52,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f61575bc060ce15c445'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:52,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f62575bc060ce15c448 2025-12-17 21:36:52,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f62575bc060ce15c448 HTTP/1.1" 502 150 2025-12-17 21:36:52,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f62575bc060ce15c448'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:53,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f64575bc060ce15c44b 2025-12-17 21:36:53,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f64575bc060ce15c44b HTTP/1.1" 502 150 2025-12-17 21:36:53,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f64575bc060ce15c44b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:53,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f5d575bc060ce15c432 2025-12-17 21:36:53,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f5d575bc060ce15c432 HTTP/1.1" 502 150 2025-12-17 21:36:53,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f5d575bc060ce15c432'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:54,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ad311e3c24e95681a9db 2025-12-17 21:36:54,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ad311e3c24e95681a9db HTTP/1.1" 502 150 2025-12-17 21:36:54,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ad311e3c24e95681a9db'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:54,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f5f575bc060ce15c437 2025-12-17 21:36:54,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f5f575bc060ce15c437 HTTP/1.1" 502 150 2025-12-17 21:36:54,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f5f575bc060ce15c437'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:55,358 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f60575bc060ce15c43e 2025-12-17 21:36:55,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f60575bc060ce15c43e HTTP/1.1" 502 150 2025-12-17 21:36:55,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f60575bc060ce15c43e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:56,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f61575bc060ce15c445 2025-12-17 21:36:56,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f61575bc060ce15c445 HTTP/1.1" 502 150 2025-12-17 21:36:56,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f61575bc060ce15c445'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58778 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:36:56,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f62575bc060ce15c448 2025-12-17 21:36:56,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f62575bc060ce15c448 HTTP/1.1" 502 150 2025-12-17 21:36:56,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f62575bc060ce15c448'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:36:56,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940812b575bc060ce15ca22 2025-12-17 21:36:56,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940812b575bc060ce15ca22 HTTP/1.1" 502 150 2025-12-17 21:36:56,919 - lava_callback - ERROR - Error processing callback for node 6940812b575bc060ce15ca22: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940812b575bc060ce15ca22 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940812b575bc060ce15ca22 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940812b575bc060ce15ca22 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:36:57,369 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f64575bc060ce15c44b INFO: 172.21.0.1:44858 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:37:01,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f5d575bc060ce15c432 2025-12-17 21:37:01,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f5d575bc060ce15c432 HTTP/1.1" 502 150 2025-12-17 21:37:01,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f5d575bc060ce15c432'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:02,301 - lava_callback - INFO - Received callback for node 69407f69575bc060ce15c45e INFO: 172.21.0.1:44866 - "POST /node/69407f69575bc060ce15c45e HTTP/1.0" 202 Accepted 2025-12-17 21:37:02,307 - lava_callback - INFO - Starting async job submit for node 69407f69575bc060ce15c45e 2025-12-17 21:37:02,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:37:02,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f69575bc060ce15c45e HTTP/1.1" 502 150 2025-12-17 21:37:02,315 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f69575bc060ce15c45e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:02,315 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f69575bc060ce15c45e 2025-12-17 21:37:02,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f69575bc060ce15c45e HTTP/1.1" 502 150 2025-12-17 21:37:02,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f69575bc060ce15c45e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:02,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ad311e3c24e95681a9db 2025-12-17 21:37:02,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ad311e3c24e95681a9db HTTP/1.1" 502 150 2025-12-17 21:37:02,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ad311e3c24e95681a9db'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:02,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f5f575bc060ce15c437 2025-12-17 21:37:02,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f5f575bc060ce15c437 HTTP/1.1" 502 150 2025-12-17 21:37:02,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f5f575bc060ce15c437'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:02,945 - lava_callback - INFO - Received callback for node 69407f6d575bc060ce15c473 INFO: 172.21.0.1:44868 - "POST /node/69407f6d575bc060ce15c473 HTTP/1.0" 202 Accepted 2025-12-17 21:37:02,954 - lava_callback - INFO - Starting async job submit for node 69407f6d575bc060ce15c473 2025-12-17 21:37:02,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:37:02,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f6d575bc060ce15c473 HTTP/1.1" 502 150 2025-12-17 21:37:02,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f6d575bc060ce15c473'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:02,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f6d575bc060ce15c473 2025-12-17 21:37:02,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f6d575bc060ce15c473 HTTP/1.1" 502 150 2025-12-17 21:37:02,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f6d575bc060ce15c473'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:03,359 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f60575bc060ce15c43e 2025-12-17 21:37:04,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f61575bc060ce15c445 2025-12-17 21:37:04,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f61575bc060ce15c445 HTTP/1.1" 502 150 2025-12-17 21:37:04,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f61575bc060ce15c445'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:04,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f69575bc060ce15c45e 2025-12-17 21:37:04,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f69575bc060ce15c45e HTTP/1.1" 502 150 2025-12-17 21:37:04,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f69575bc060ce15c45e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:04,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f62575bc060ce15c448 2025-12-17 21:37:04,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f62575bc060ce15c448 HTTP/1.1" 502 150 2025-12-17 21:37:04,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f62575bc060ce15c448'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:04,949 - lava_callback - INFO - Received callback for node 6940812d575bc060ce15ca27 INFO: 172.21.0.1:44884 - "POST /node/6940812d575bc060ce15ca27 HTTP/1.0" 202 Accepted 2025-12-17 21:37:04,955 - lava_callback - INFO - Starting async job submit for node 6940812d575bc060ce15ca27 2025-12-17 21:37:04,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:37:04,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f6d575bc060ce15c473 2025-12-17 21:37:04,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f6d575bc060ce15c473 HTTP/1.1" 502 150 2025-12-17 21:37:04,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f6d575bc060ce15c473'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:05,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940812d575bc060ce15ca27 HTTP/1.1" 502 150 2025-12-17 21:37:05,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940812d575bc060ce15ca27'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:05,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940812d575bc060ce15ca27 2025-12-17 21:37:05,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940812d575bc060ce15ca27 HTTP/1.1" 502 150 2025-12-17 21:37:05,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940812d575bc060ce15ca27'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44890 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:37:07,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940812d575bc060ce15ca27 2025-12-17 21:37:07,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940812d575bc060ce15ca27 HTTP/1.1" 502 150 2025-12-17 21:37:07,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940812d575bc060ce15ca27'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:07,933 - lava_callback - INFO - Received callback for node 69407f78575bc060ce15c480 INFO: 172.21.0.1:44892 - "POST /node/69407f78575bc060ce15c480 HTTP/1.0" 202 Accepted 2025-12-17 21:37:07,938 - lava_callback - INFO - Starting async job submit for node 69407f78575bc060ce15c480 2025-12-17 21:37:07,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:37:07,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f78575bc060ce15c480 HTTP/1.1" 502 150 2025-12-17 21:37:07,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f78575bc060ce15c480'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:07,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f78575bc060ce15c480 2025-12-17 21:37:07,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f78575bc060ce15c480 HTTP/1.1" 502 150 2025-12-17 21:37:07,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f78575bc060ce15c480'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:08,317 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f69575bc060ce15c45e 2025-12-17 21:37:08,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f69575bc060ce15c45e HTTP/1.1" 502 150 2025-12-17 21:37:08,318 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f69575bc060ce15c45e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:08,603 - lava_callback - INFO - Received callback for node 69407f79575bc060ce15c483 INFO: 172.21.0.1:44902 - "POST /node/69407f79575bc060ce15c483 HTTP/1.0" 202 Accepted 2025-12-17 21:37:08,612 - lava_callback - INFO - Starting async job submit for node 69407f79575bc060ce15c483 2025-12-17 21:37:08,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:37:08,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f79575bc060ce15c483 HTTP/1.1" 502 150 2025-12-17 21:37:08,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f79575bc060ce15c483'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:08,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f79575bc060ce15c483 2025-12-17 21:37:08,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f79575bc060ce15c483 HTTP/1.1" 502 150 2025-12-17 21:37:08,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f79575bc060ce15c483'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:08,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f6d575bc060ce15c473 2025-12-17 21:37:08,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f6d575bc060ce15c473 HTTP/1.1" 502 150 2025-12-17 21:37:08,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f6d575bc060ce15c473'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:09,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f78575bc060ce15c480 2025-12-17 21:37:09,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f78575bc060ce15c480 HTTP/1.1" 502 150 2025-12-17 21:37:09,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f78575bc060ce15c480'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:10,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f79575bc060ce15c483 2025-12-17 21:37:10,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f79575bc060ce15c483 HTTP/1.1" 502 150 2025-12-17 21:37:10,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f79575bc060ce15c483'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:11,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940812d575bc060ce15ca27 2025-12-17 21:37:11,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940812d575bc060ce15ca27 HTTP/1.1" 502 150 2025-12-17 21:37:11,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940812d575bc060ce15ca27'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:12,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f64575bc060ce15c44b HTTP/1.1" 502 150 2025-12-17 21:37:12,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f64575bc060ce15c44b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:13,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f78575bc060ce15c480 2025-12-17 21:37:13,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f78575bc060ce15c480 HTTP/1.1" 502 150 2025-12-17 21:37:13,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f78575bc060ce15c480'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:14,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f79575bc060ce15c483 2025-12-17 21:37:14,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f79575bc060ce15c483 HTTP/1.1" 502 150 2025-12-17 21:37:14,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f79575bc060ce15c483'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34252 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:37:16,319 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f69575bc060ce15c45e 2025-12-17 21:37:16,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f69575bc060ce15c45e HTTP/1.1" 502 150 2025-12-17 21:37:16,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f69575bc060ce15c45e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34254 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:37:16,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f6d575bc060ce15c473 2025-12-17 21:37:16,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f6d575bc060ce15c473 HTTP/1.1" 502 150 2025-12-17 21:37:16,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f6d575bc060ce15c473'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:17,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f5d575bc060ce15c432 2025-12-17 21:37:17,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f5d575bc060ce15c432 HTTP/1.1" 502 150 2025-12-17 21:37:17,997 - lava_callback - ERROR - Error processing callback for node 69407f5d575bc060ce15c432: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f5d575bc060ce15c432 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f5d575bc060ce15c432 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f5d575bc060ce15c432 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:37:18,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ad311e3c24e95681a9db 2025-12-17 21:37:18,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ad311e3c24e95681a9db HTTP/1.1" 502 150 2025-12-17 21:37:18,553 - lava_callback - ERROR - Error processing callback for node 6940ad311e3c24e95681a9db: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ad311e3c24e95681a9db (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ad311e3c24e95681a9db (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ad311e3c24e95681a9db (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:37:18,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f5f575bc060ce15c437 2025-12-17 21:37:18,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f5f575bc060ce15c437 HTTP/1.1" 502 150 2025-12-17 21:37:18,702 - lava_callback - ERROR - Error processing callback for node 69407f5f575bc060ce15c437: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f5f575bc060ce15c437 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f5f575bc060ce15c437 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f5f575bc060ce15c437 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:37:18,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f60575bc060ce15c43e HTTP/1.1" 502 150 2025-12-17 21:37:18,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f60575bc060ce15c43e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:19,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940812d575bc060ce15ca27 2025-12-17 21:37:19,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940812d575bc060ce15ca27 HTTP/1.1" 502 150 2025-12-17 21:37:19,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940812d575bc060ce15ca27'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:20,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f61575bc060ce15c445 2025-12-17 21:37:20,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f61575bc060ce15c445 HTTP/1.1" 502 150 2025-12-17 21:37:20,021 - lava_callback - ERROR - Error processing callback for node 69407f61575bc060ce15c445: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f61575bc060ce15c445 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f61575bc060ce15c445 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f61575bc060ce15c445 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:37:20,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f64575bc060ce15c44b 2025-12-17 21:37:20,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f64575bc060ce15c44b HTTP/1.1" 502 150 2025-12-17 21:37:20,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f64575bc060ce15c44b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:20,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f62575bc060ce15c448 2025-12-17 21:37:20,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f62575bc060ce15c448 HTTP/1.1" 502 150 2025-12-17 21:37:20,720 - lava_callback - ERROR - Error processing callback for node 69407f62575bc060ce15c448: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f62575bc060ce15c448 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f62575bc060ce15c448 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f62575bc060ce15c448 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:37:21,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f78575bc060ce15c480 2025-12-17 21:37:21,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f78575bc060ce15c480 HTTP/1.1" 502 150 2025-12-17 21:37:21,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f78575bc060ce15c480'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:22,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f79575bc060ce15c483 2025-12-17 21:37:22,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f79575bc060ce15c483 HTTP/1.1" 502 150 2025-12-17 21:37:22,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f79575bc060ce15c483'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41992 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:37:32,320 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f69575bc060ce15c45e 2025-12-17 21:37:32,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f69575bc060ce15c45e HTTP/1.1" 502 150 2025-12-17 21:37:32,322 - lava_callback - ERROR - Error processing callback for node 69407f69575bc060ce15c45e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f69575bc060ce15c45e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f69575bc060ce15c45e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f69575bc060ce15c45e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:37:32,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f6d575bc060ce15c473 2025-12-17 21:37:32,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f6d575bc060ce15c473 HTTP/1.1" 502 150 2025-12-17 21:37:32,990 - lava_callback - ERROR - Error processing callback for node 69407f6d575bc060ce15c473: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f6d575bc060ce15c473 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f6d575bc060ce15c473 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f6d575bc060ce15c473 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:37:34,526 - lava_callback - INFO - Received callback for node 69408130575bc060ce15ca2e INFO: 172.21.0.1:47872 - "POST /node/69408130575bc060ce15ca2e HTTP/1.0" 202 Accepted 2025-12-17 21:37:34,532 - lava_callback - INFO - Starting async job submit for node 69408130575bc060ce15ca2e 2025-12-17 21:37:34,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:37:34,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408130575bc060ce15ca2e HTTP/1.1" 502 150 2025-12-17 21:37:34,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408130575bc060ce15ca2e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:34,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408130575bc060ce15ca2e 2025-12-17 21:37:34,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408130575bc060ce15ca2e HTTP/1.1" 502 150 2025-12-17 21:37:34,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408130575bc060ce15ca2e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:34,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f60575bc060ce15c43e 2025-12-17 21:37:34,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f60575bc060ce15c43e HTTP/1.1" 502 150 2025-12-17 21:37:34,770 - lava_callback - ERROR - Error processing callback for node 69407f60575bc060ce15c43e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f60575bc060ce15c43e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f60575bc060ce15c43e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f60575bc060ce15c43e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:37:35,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940812d575bc060ce15ca27 INFO: 172.21.0.1:47876 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:37:36,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408130575bc060ce15ca2e 2025-12-17 21:37:36,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408130575bc060ce15ca2e HTTP/1.1" 502 150 2025-12-17 21:37:36,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408130575bc060ce15ca2e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:36,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f64575bc060ce15c44b 2025-12-17 21:37:36,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f64575bc060ce15c44b HTTP/1.1" 502 150 2025-12-17 21:37:36,627 - lava_callback - ERROR - Error processing callback for node 69407f64575bc060ce15c44b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f64575bc060ce15c44b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f64575bc060ce15c44b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f64575bc060ce15c44b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:37:37,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f78575bc060ce15c480 2025-12-17 21:37:37,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f78575bc060ce15c480 HTTP/1.1" 502 150 2025-12-17 21:37:37,957 - lava_callback - ERROR - Error processing callback for node 69407f78575bc060ce15c480: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f78575bc060ce15c480 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f78575bc060ce15c480 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f78575bc060ce15c480 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:37:38,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f79575bc060ce15c483 2025-12-17 21:37:38,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f79575bc060ce15c483 HTTP/1.1" 502 150 2025-12-17 21:37:38,654 - lava_callback - ERROR - Error processing callback for node 69407f79575bc060ce15c483: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f79575bc060ce15c483 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f79575bc060ce15c483 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f79575bc060ce15c483 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:37:40,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408130575bc060ce15ca2e 2025-12-17 21:37:42,331 - lava_callback - INFO - Received callback for node 69407f90575bc060ce15c4a2 INFO: 172.21.0.1:56768 - "POST /node/69407f90575bc060ce15c4a2 HTTP/1.0" 202 Accepted 2025-12-17 21:37:42,337 - lava_callback - INFO - Starting async job submit for node 69407f90575bc060ce15c4a2 2025-12-17 21:37:42,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:37:42,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f90575bc060ce15c4a2 HTTP/1.1" 502 150 2025-12-17 21:37:42,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f90575bc060ce15c4a2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:42,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f90575bc060ce15c4a2 2025-12-17 21:37:42,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f90575bc060ce15c4a2 HTTP/1.1" 502 150 2025-12-17 21:37:42,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f90575bc060ce15c4a2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:42,986 - lava_callback - INFO - Received callback for node 69407f92575bc060ce15c4ab INFO: 172.21.0.1:56774 - "POST /node/69407f92575bc060ce15c4ab HTTP/1.0" 202 Accepted 2025-12-17 21:37:42,992 - lava_callback - INFO - Starting async job submit for node 69407f92575bc060ce15c4ab 2025-12-17 21:37:42,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:37:42,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f92575bc060ce15c4ab HTTP/1.1" 502 150 2025-12-17 21:37:42,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f92575bc060ce15c4ab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:42,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f92575bc060ce15c4ab 2025-12-17 21:37:42,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f92575bc060ce15c4ab HTTP/1.1" 502 150 2025-12-17 21:37:42,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f92575bc060ce15c4ab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:43,645 - lava_callback - INFO - Received callback for node 69407f94575bc060ce15c4b2 INFO: 172.21.0.1:56786 - "POST /node/69407f94575bc060ce15c4b2 HTTP/1.0" 202 Accepted 2025-12-17 21:37:43,650 - lava_callback - INFO - Starting async job submit for node 69407f94575bc060ce15c4b2 2025-12-17 21:37:43,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:37:43,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f94575bc060ce15c4b2 HTTP/1.1" 502 150 2025-12-17 21:37:43,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f94575bc060ce15c4b2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:43,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f94575bc060ce15c4b2 2025-12-17 21:37:43,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f94575bc060ce15c4b2 HTTP/1.1" 502 150 2025-12-17 21:37:43,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f94575bc060ce15c4b2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:44,314 - lava_callback - INFO - Received callback for node 69407f95575bc060ce15c4b8 INFO: 172.21.0.1:56794 - "POST /node/69407f95575bc060ce15c4b8 HTTP/1.0" 202 Accepted 2025-12-17 21:37:44,319 - lava_callback - INFO - Starting async job submit for node 69407f95575bc060ce15c4b8 2025-12-17 21:37:44,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:37:44,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f95575bc060ce15c4b8 HTTP/1.1" 502 150 2025-12-17 21:37:44,326 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f95575bc060ce15c4b8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:44,326 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f95575bc060ce15c4b8 2025-12-17 21:37:44,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f95575bc060ce15c4b8 HTTP/1.1" 502 150 2025-12-17 21:37:44,327 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f95575bc060ce15c4b8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:44,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f90575bc060ce15c4a2 2025-12-17 21:37:44,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f90575bc060ce15c4a2 HTTP/1.1" 502 150 2025-12-17 21:37:44,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f90575bc060ce15c4a2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:44,963 - lava_callback - INFO - Received callback for node 69407f96575bc060ce15c4bc INFO: 172.21.0.1:56796 - "POST /node/69407f96575bc060ce15c4bc HTTP/1.0" 202 Accepted 2025-12-17 21:37:44,968 - lava_callback - INFO - Starting async job submit for node 69407f96575bc060ce15c4bc 2025-12-17 21:37:44,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:37:44,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f92575bc060ce15c4ab 2025-12-17 21:37:45,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f92575bc060ce15c4ab HTTP/1.1" 502 150 2025-12-17 21:37:45,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f92575bc060ce15c4ab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:45,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f96575bc060ce15c4bc HTTP/1.1" 502 150 2025-12-17 21:37:45,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f96575bc060ce15c4bc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:45,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f96575bc060ce15c4bc 2025-12-17 21:37:45,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f96575bc060ce15c4bc HTTP/1.1" 502 150 2025-12-17 21:37:45,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f96575bc060ce15c4bc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:45,632 - lava_callback - INFO - Received callback for node 69407f98575bc060ce15c4c3 INFO: 172.21.0.1:56798 - "POST /node/69407f98575bc060ce15c4c3 HTTP/1.0" 202 Accepted 2025-12-17 21:37:45,638 - lava_callback - INFO - Starting async job submit for node 69407f98575bc060ce15c4c3 2025-12-17 21:37:45,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:37:45,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f94575bc060ce15c4b2 2025-12-17 21:37:45,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f94575bc060ce15c4b2 HTTP/1.1" 502 150 2025-12-17 21:37:45,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f94575bc060ce15c4b2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:45,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f98575bc060ce15c4c3 HTTP/1.1" 502 150 2025-12-17 21:37:45,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f98575bc060ce15c4c3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:45,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f98575bc060ce15c4c3 2025-12-17 21:37:45,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f98575bc060ce15c4c3 HTTP/1.1" 502 150 2025-12-17 21:37:45,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f98575bc060ce15c4c3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:46,284 - lava_callback - INFO - Received callback for node 69407f99575bc060ce15c4c6 INFO: 172.21.0.1:56800 - "POST /node/69407f99575bc060ce15c4c6 HTTP/1.0" 202 Accepted 2025-12-17 21:37:46,293 - lava_callback - INFO - Starting async job submit for node 69407f99575bc060ce15c4c6 2025-12-17 21:37:46,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:37:46,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f99575bc060ce15c4c6 HTTP/1.1" 502 150 2025-12-17 21:37:46,301 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f99575bc060ce15c4c6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:46,301 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f99575bc060ce15c4c6 2025-12-17 21:37:46,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f99575bc060ce15c4c6 HTTP/1.1" 502 150 2025-12-17 21:37:46,302 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f99575bc060ce15c4c6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:46,327 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f95575bc060ce15c4b8 2025-12-17 21:37:46,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f95575bc060ce15c4b8 HTTP/1.1" 502 150 2025-12-17 21:37:46,328 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f95575bc060ce15c4b8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56812 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:37:46,962 - lava_callback - INFO - Received callback for node 69407f9a575bc060ce15c4c9 INFO: 172.21.0.1:56818 - "POST /node/69407f9a575bc060ce15c4c9 HTTP/1.0" 202 Accepted 2025-12-17 21:37:46,973 - lava_callback - INFO - Starting async job submit for node 69407f9a575bc060ce15c4c9 2025-12-17 21:37:46,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:37:46,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f9a575bc060ce15c4c9 HTTP/1.1" 502 150 2025-12-17 21:37:46,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f9a575bc060ce15c4c9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:46,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f9a575bc060ce15c4c9 2025-12-17 21:37:46,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f9a575bc060ce15c4c9 HTTP/1.1" 502 150 2025-12-17 21:37:46,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f9a575bc060ce15c4c9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:47,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f96575bc060ce15c4bc 2025-12-17 21:37:47,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f96575bc060ce15c4bc HTTP/1.1" 502 150 2025-12-17 21:37:47,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f96575bc060ce15c4bc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:47,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f98575bc060ce15c4c3 2025-12-17 21:37:47,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f98575bc060ce15c4c3 HTTP/1.1" 502 150 2025-12-17 21:37:47,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f98575bc060ce15c4c3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:48,302 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f99575bc060ce15c4c6 2025-12-17 21:37:48,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f99575bc060ce15c4c6 HTTP/1.1" 502 150 2025-12-17 21:37:48,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f99575bc060ce15c4c6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:48,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f90575bc060ce15c4a2 2025-12-17 21:37:48,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f90575bc060ce15c4a2 HTTP/1.1" 502 150 2025-12-17 21:37:48,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f90575bc060ce15c4a2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:48,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f9a575bc060ce15c4c9 2025-12-17 21:37:48,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f9a575bc060ce15c4c9 HTTP/1.1" 502 150 2025-12-17 21:37:48,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f9a575bc060ce15c4c9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:49,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f92575bc060ce15c4ab 2025-12-17 21:37:49,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f92575bc060ce15c4ab HTTP/1.1" 502 150 2025-12-17 21:37:49,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f92575bc060ce15c4ab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:49,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f94575bc060ce15c4b2 2025-12-17 21:37:49,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f94575bc060ce15c4b2 HTTP/1.1" 502 150 2025-12-17 21:37:49,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f94575bc060ce15c4b2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:50,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940812d575bc060ce15ca27 HTTP/1.1" 502 150 2025-12-17 21:37:50,257 - lava_callback - ERROR - Error processing callback for node 6940812d575bc060ce15ca27: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940812d575bc060ce15ca27 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940812d575bc060ce15ca27 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940812d575bc060ce15ca27 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:37:50,328 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f95575bc060ce15c4b8 2025-12-17 21:37:50,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f95575bc060ce15c4b8 HTTP/1.1" 502 150 2025-12-17 21:37:50,329 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f95575bc060ce15c4b8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:51,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f96575bc060ce15c4bc 2025-12-17 21:37:51,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f96575bc060ce15c4bc HTTP/1.1" 502 150 2025-12-17 21:37:51,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f96575bc060ce15c4bc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:51,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f98575bc060ce15c4c3 2025-12-17 21:37:51,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f98575bc060ce15c4c3 HTTP/1.1" 502 150 2025-12-17 21:37:51,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f98575bc060ce15c4c3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:52,304 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f99575bc060ce15c4c6 2025-12-17 21:37:52,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f99575bc060ce15c4c6 HTTP/1.1" 502 150 2025-12-17 21:37:52,306 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f99575bc060ce15c4c6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:52,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f9a575bc060ce15c4c9 2025-12-17 21:37:52,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f9a575bc060ce15c4c9 HTTP/1.1" 502 150 2025-12-17 21:37:52,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f9a575bc060ce15c4c9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:55,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408130575bc060ce15ca2e HTTP/1.1" 502 150 2025-12-17 21:37:55,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408130575bc060ce15ca2e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:56,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f90575bc060ce15c4a2 2025-12-17 21:37:56,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f90575bc060ce15c4a2 HTTP/1.1" 502 150 2025-12-17 21:37:56,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f90575bc060ce15c4a2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39182 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:37:57,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f92575bc060ce15c4ab 2025-12-17 21:37:57,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f92575bc060ce15c4ab HTTP/1.1" 502 150 2025-12-17 21:37:57,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f92575bc060ce15c4ab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:57,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f94575bc060ce15c4b2 2025-12-17 21:37:57,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f94575bc060ce15c4b2 HTTP/1.1" 502 150 2025-12-17 21:37:57,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f94575bc060ce15c4b2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:58,329 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f95575bc060ce15c4b8 2025-12-17 21:37:58,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f95575bc060ce15c4b8 HTTP/1.1" 502 150 2025-12-17 21:37:58,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f95575bc060ce15c4b8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:59,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f96575bc060ce15c4bc 2025-12-17 21:37:59,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f96575bc060ce15c4bc HTTP/1.1" 502 150 2025-12-17 21:37:59,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f96575bc060ce15c4bc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:37:59,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f98575bc060ce15c4c3 2025-12-17 21:37:59,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f98575bc060ce15c4c3 HTTP/1.1" 502 150 2025-12-17 21:37:59,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f98575bc060ce15c4c3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:00,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f99575bc060ce15c4c6 2025-12-17 21:38:00,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f99575bc060ce15c4c6 HTTP/1.1" 502 150 2025-12-17 21:38:00,307 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f99575bc060ce15c4c6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:00,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f9a575bc060ce15c4c9 2025-12-17 21:38:00,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f9a575bc060ce15c4c9 HTTP/1.1" 502 150 2025-12-17 21:38:00,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f9a575bc060ce15c4c9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48450 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:38:02,318 - lava_callback - INFO - Received callback for node 69407fb0575bc060ce15c51c INFO: 172.21.0.1:48458 - "POST /node/69407fb0575bc060ce15c51c HTTP/1.0" 202 Accepted 2025-12-17 21:38:02,324 - lava_callback - INFO - Starting async job submit for node 69407fb0575bc060ce15c51c 2025-12-17 21:38:02,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:02,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb0575bc060ce15c51c HTTP/1.1" 502 150 2025-12-17 21:38:02,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb0575bc060ce15c51c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:02,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb0575bc060ce15c51c 2025-12-17 21:38:02,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb0575bc060ce15c51c HTTP/1.1" 502 150 2025-12-17 21:38:02,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb0575bc060ce15c51c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:03,021 - lava_callback - INFO - Received callback for node 69407fa4575bc060ce15c4ce INFO: 172.21.0.1:48464 - "POST /node/69407fa4575bc060ce15c4ce HTTP/1.0" 202 Accepted 2025-12-17 21:38:03,026 - lava_callback - INFO - Starting async job submit for node 69407fa4575bc060ce15c4ce 2025-12-17 21:38:03,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:03,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa4575bc060ce15c4ce HTTP/1.1" 502 150 2025-12-17 21:38:03,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fa4575bc060ce15c4ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:03,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fa4575bc060ce15c4ce 2025-12-17 21:38:03,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa4575bc060ce15c4ce HTTP/1.1" 502 150 2025-12-17 21:38:03,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fa4575bc060ce15c4ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:03,693 - lava_callback - INFO - Received callback for node 69407fa7575bc060ce15c4f9 INFO: 172.21.0.1:48478 - "POST /node/69407fa7575bc060ce15c4f9 HTTP/1.0" 202 Accepted 2025-12-17 21:38:03,699 - lava_callback - INFO - Starting async job submit for node 69407fa7575bc060ce15c4f9 2025-12-17 21:38:03,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:03,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa7575bc060ce15c4f9 HTTP/1.1" 502 150 2025-12-17 21:38:03,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fa7575bc060ce15c4f9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:03,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fa7575bc060ce15c4f9 2025-12-17 21:38:03,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa7575bc060ce15c4f9 HTTP/1.1" 502 150 2025-12-17 21:38:03,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fa7575bc060ce15c4f9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:03,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408130575bc060ce15ca2e 2025-12-17 21:38:03,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408130575bc060ce15ca2e HTTP/1.1" 502 150 2025-12-17 21:38:03,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408130575bc060ce15ca2e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:04,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb0575bc060ce15c51c 2025-12-17 21:38:04,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb0575bc060ce15c51c HTTP/1.1" 502 150 2025-12-17 21:38:04,333 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb0575bc060ce15c51c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:04,360 - lava_callback - INFO - Received callback for node 69407fa8575bc060ce15c500 INFO: 172.21.0.1:48488 - "POST /node/69407fa8575bc060ce15c500 HTTP/1.0" 202 Accepted 2025-12-17 21:38:04,366 - lava_callback - INFO - Starting async job submit for node 69407fa8575bc060ce15c500 2025-12-17 21:38:04,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:04,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa8575bc060ce15c500 HTTP/1.1" 502 150 2025-12-17 21:38:04,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fa8575bc060ce15c500'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:04,376 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fa8575bc060ce15c500 2025-12-17 21:38:04,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa8575bc060ce15c500 HTTP/1.1" 502 150 2025-12-17 21:38:04,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fa8575bc060ce15c500'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:05,018 - lava_callback - INFO - Received callback for node 69407faa575bc060ce15c507 INFO: 172.21.0.1:48496 - "POST /node/69407faa575bc060ce15c507 HTTP/1.0" 202 Accepted 2025-12-17 21:38:05,024 - lava_callback - INFO - Starting async job submit for node 69407faa575bc060ce15c507 2025-12-17 21:38:05,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:05,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407faa575bc060ce15c507 HTTP/1.1" 502 150 2025-12-17 21:38:05,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407faa575bc060ce15c507'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:05,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407faa575bc060ce15c507 2025-12-17 21:38:05,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407faa575bc060ce15c507 HTTP/1.1" 502 150 2025-12-17 21:38:05,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407faa575bc060ce15c507'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:05,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fa4575bc060ce15c4ce 2025-12-17 21:38:05,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa4575bc060ce15c4ce HTTP/1.1" 502 150 2025-12-17 21:38:05,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fa4575bc060ce15c4ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:05,682 - lava_callback - INFO - Received callback for node 69407fac575bc060ce15c513 INFO: 172.21.0.1:48510 - "POST /node/69407fac575bc060ce15c513 HTTP/1.0" 202 Accepted 2025-12-17 21:38:05,691 - lava_callback - INFO - Starting async job submit for node 69407fac575bc060ce15c513 2025-12-17 21:38:05,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:05,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fac575bc060ce15c513 HTTP/1.1" 502 150 2025-12-17 21:38:05,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fac575bc060ce15c513'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:05,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fac575bc060ce15c513 2025-12-17 21:38:05,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fac575bc060ce15c513 HTTP/1.1" 502 150 2025-12-17 21:38:05,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fac575bc060ce15c513'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:05,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fa7575bc060ce15c4f9 2025-12-17 21:38:05,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa7575bc060ce15c4f9 HTTP/1.1" 502 150 2025-12-17 21:38:05,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fa7575bc060ce15c4f9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:06,343 - lava_callback - INFO - Received callback for node 69407fae575bc060ce15c516 INFO: 172.21.0.1:48514 - "POST /node/69407fae575bc060ce15c516 HTTP/1.0" 202 Accepted 2025-12-17 21:38:06,349 - lava_callback - INFO - Starting async job submit for node 69407fae575bc060ce15c516 2025-12-17 21:38:06,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:06,377 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fa8575bc060ce15c500 2025-12-17 21:38:06,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fae575bc060ce15c516 HTTP/1.1" 502 150 2025-12-17 21:38:06,379 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fae575bc060ce15c516'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:06,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fae575bc060ce15c516 2025-12-17 21:38:06,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fae575bc060ce15c516 HTTP/1.1" 502 150 2025-12-17 21:38:06,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fae575bc060ce15c516'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48524 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:38:07,028 - lava_callback - INFO - Received callback for node 69407faf575bc060ce15c519 INFO: 172.21.0.1:48530 - "POST /node/69407faf575bc060ce15c519 HTTP/1.0" 202 Accepted 2025-12-17 21:38:07,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407faa575bc060ce15c507 2025-12-17 21:38:07,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407faa575bc060ce15c507 HTTP/1.1" 502 150 2025-12-17 21:38:07,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407faa575bc060ce15c507'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:07,666 - lava_callback - INFO - Received callback for node 69407fb2575bc060ce15c51f INFO: 172.21.0.1:48532 - "POST /node/69407fb2575bc060ce15c51f HTTP/1.0" 202 Accepted 2025-12-17 21:38:07,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fac575bc060ce15c513 2025-12-17 21:38:07,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fac575bc060ce15c513 HTTP/1.1" 502 150 2025-12-17 21:38:07,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fac575bc060ce15c513'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:07,817 - lava_callback - INFO - Received callback for node 69407fab575bc060ce15c50e INFO: 172.21.0.1:48542 - "POST /node/69407fab575bc060ce15c50e HTTP/1.0" 202 Accepted 2025-12-17 21:38:08,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb0575bc060ce15c51c 2025-12-17 21:38:08,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb0575bc060ce15c51c HTTP/1.1" 502 150 2025-12-17 21:38:08,338 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb0575bc060ce15c51c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:08,358 - lava_callback - INFO - Received callback for node 69407fb3575bc060ce15c522 INFO: 172.21.0.1:48556 - "POST /node/69407fb3575bc060ce15c522 HTTP/1.0" 202 Accepted 2025-12-17 21:38:08,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fae575bc060ce15c516 2025-12-17 21:38:08,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fae575bc060ce15c516 HTTP/1.1" 502 150 2025-12-17 21:38:08,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fae575bc060ce15c516'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:09,010 - lava_callback - INFO - Received callback for node 69407fb4575bc060ce15c525 INFO: 172.21.0.1:48570 - "POST /node/69407fb4575bc060ce15c525 HTTP/1.0" 202 Accepted 2025-12-17 21:38:09,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fa4575bc060ce15c4ce 2025-12-17 21:38:09,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa4575bc060ce15c4ce HTTP/1.1" 502 150 2025-12-17 21:38:09,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fa4575bc060ce15c4ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:09,698 - lava_callback - INFO - Received callback for node 69407fb5575bc060ce15c528 INFO: 172.21.0.1:49284 - "POST /node/69407fb5575bc060ce15c528 HTTP/1.0" 202 Accepted 2025-12-17 21:38:09,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fa7575bc060ce15c4f9 2025-12-17 21:38:09,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa7575bc060ce15c4f9 HTTP/1.1" 502 150 2025-12-17 21:38:09,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fa7575bc060ce15c4f9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:10,377 - lava_callback - INFO - Received callback for node 69407fb8575bc060ce15c531 INFO: 172.21.0.1:49290 - "POST /node/69407fb8575bc060ce15c531 HTTP/1.0" 202 Accepted 2025-12-17 21:38:11,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407faa575bc060ce15c507 2025-12-17 21:38:11,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407faa575bc060ce15c507 HTTP/1.1" 502 150 2025-12-17 21:38:11,037 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407faa575bc060ce15c507'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:11,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fac575bc060ce15c513 2025-12-17 21:38:11,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fac575bc060ce15c513 HTTP/1.1" 502 150 2025-12-17 21:38:11,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fac575bc060ce15c513'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:12,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f90575bc060ce15c4a2 2025-12-17 21:38:12,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f90575bc060ce15c4a2 HTTP/1.1" 502 150 2025-12-17 21:38:12,350 - lava_callback - ERROR - Error processing callback for node 69407f90575bc060ce15c4a2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f90575bc060ce15c4a2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f90575bc060ce15c4a2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f90575bc060ce15c4a2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:12,352 - lava_callback - INFO - Starting async job submit for node 69407faf575bc060ce15c519 2025-12-17 21:38:12,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:12,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407faf575bc060ce15c519 HTTP/1.1" 502 150 2025-12-17 21:38:12,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407faf575bc060ce15c519'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:12,359 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407faf575bc060ce15c519 2025-12-17 21:38:12,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407faf575bc060ce15c519 HTTP/1.1" 502 150 2025-12-17 21:38:12,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407faf575bc060ce15c519'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:12,381 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fae575bc060ce15c516 2025-12-17 21:38:12,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fae575bc060ce15c516 HTTP/1.1" 502 150 2025-12-17 21:38:12,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fae575bc060ce15c516'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:13,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f92575bc060ce15c4ab 2025-12-17 21:38:13,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f92575bc060ce15c4ab HTTP/1.1" 502 150 2025-12-17 21:38:13,005 - lava_callback - ERROR - Error processing callback for node 69407f92575bc060ce15c4ab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f92575bc060ce15c4ab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f92575bc060ce15c4ab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f92575bc060ce15c4ab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:13,007 - lava_callback - INFO - Starting async job submit for node 69407fb2575bc060ce15c51f 2025-12-17 21:38:13,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:13,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb2575bc060ce15c51f HTTP/1.1" 502 150 2025-12-17 21:38:13,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb2575bc060ce15c51f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:13,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb2575bc060ce15c51f 2025-12-17 21:38:13,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb2575bc060ce15c51f HTTP/1.1" 502 150 2025-12-17 21:38:13,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb2575bc060ce15c51f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:13,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f94575bc060ce15c4b2 2025-12-17 21:38:13,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f94575bc060ce15c4b2 HTTP/1.1" 502 150 2025-12-17 21:38:13,664 - lava_callback - ERROR - Error processing callback for node 69407f94575bc060ce15c4b2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f94575bc060ce15c4b2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f94575bc060ce15c4b2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f94575bc060ce15c4b2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:13,666 - lava_callback - INFO - Starting async job submit for node 69407fab575bc060ce15c50e 2025-12-17 21:38:13,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:13,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fab575bc060ce15c50e HTTP/1.1" 502 150 2025-12-17 21:38:13,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fab575bc060ce15c50e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:13,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fab575bc060ce15c50e 2025-12-17 21:38:13,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fab575bc060ce15c50e HTTP/1.1" 502 150 2025-12-17 21:38:13,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fab575bc060ce15c50e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:14,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f95575bc060ce15c4b8 2025-12-17 21:38:14,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f95575bc060ce15c4b8 HTTP/1.1" 502 150 2025-12-17 21:38:14,332 - lava_callback - ERROR - Error processing callback for node 69407f95575bc060ce15c4b8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f95575bc060ce15c4b8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f95575bc060ce15c4b8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f95575bc060ce15c4b8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:14,334 - lava_callback - INFO - Starting async job submit for node 69407fb3575bc060ce15c522 2025-12-17 21:38:14,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:14,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb3575bc060ce15c522 HTTP/1.1" 502 150 2025-12-17 21:38:14,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb3575bc060ce15c522'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:14,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb3575bc060ce15c522 2025-12-17 21:38:14,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb3575bc060ce15c522 HTTP/1.1" 502 150 2025-12-17 21:38:14,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb3575bc060ce15c522'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:14,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407faf575bc060ce15c519 2025-12-17 21:38:14,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407faf575bc060ce15c519 HTTP/1.1" 502 150 2025-12-17 21:38:14,361 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407faf575bc060ce15c519'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:15,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb2575bc060ce15c51f 2025-12-17 21:38:15,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb2575bc060ce15c51f HTTP/1.1" 502 150 2025-12-17 21:38:15,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb2575bc060ce15c51f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:15,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f96575bc060ce15c4bc 2025-12-17 21:38:15,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f96575bc060ce15c4bc HTTP/1.1" 502 150 2025-12-17 21:38:15,022 - lava_callback - ERROR - Error processing callback for node 69407f96575bc060ce15c4bc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f96575bc060ce15c4bc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f96575bc060ce15c4bc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f96575bc060ce15c4bc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:15,024 - lava_callback - INFO - Starting async job submit for node 69407fb4575bc060ce15c525 2025-12-17 21:38:15,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:15,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb4575bc060ce15c525 HTTP/1.1" 502 150 2025-12-17 21:38:15,055 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb4575bc060ce15c525'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:15,055 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb4575bc060ce15c525 2025-12-17 21:38:15,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb4575bc060ce15c525 HTTP/1.1" 502 150 2025-12-17 21:38:15,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb4575bc060ce15c525'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49292 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:38:15,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f98575bc060ce15c4c3 2025-12-17 21:38:15,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f98575bc060ce15c4c3 HTTP/1.1" 502 150 2025-12-17 21:38:15,673 - lava_callback - ERROR - Error processing callback for node 69407f98575bc060ce15c4c3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f98575bc060ce15c4c3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f98575bc060ce15c4c3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f98575bc060ce15c4c3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:15,674 - lava_callback - INFO - Starting async job submit for node 69407fb5575bc060ce15c528 2025-12-17 21:38:15,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:15,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fab575bc060ce15c50e 2025-12-17 21:38:15,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fab575bc060ce15c50e HTTP/1.1" 502 150 2025-12-17 21:38:15,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fab575bc060ce15c50e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:15,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb5575bc060ce15c528 HTTP/1.1" 502 150 2025-12-17 21:38:15,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb5575bc060ce15c528'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:15,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb5575bc060ce15c528 2025-12-17 21:38:15,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb5575bc060ce15c528 HTTP/1.1" 502 150 2025-12-17 21:38:15,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb5575bc060ce15c528'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:16,307 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f99575bc060ce15c4c6 2025-12-17 21:38:16,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f99575bc060ce15c4c6 HTTP/1.1" 502 150 2025-12-17 21:38:16,309 - lava_callback - ERROR - Error processing callback for node 69407f99575bc060ce15c4c6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f99575bc060ce15c4c6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f99575bc060ce15c4c6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f99575bc060ce15c4c6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:16,310 - lava_callback - INFO - Starting async job submit for node 69407fb8575bc060ce15c531 2025-12-17 21:38:16,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:16,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb8575bc060ce15c531 HTTP/1.1" 502 150 2025-12-17 21:38:16,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb8575bc060ce15c531'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:16,317 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb8575bc060ce15c531 2025-12-17 21:38:16,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb8575bc060ce15c531 HTTP/1.1" 502 150 2025-12-17 21:38:16,318 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb8575bc060ce15c531'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:16,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb0575bc060ce15c51c 2025-12-17 21:38:16,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb0575bc060ce15c51c HTTP/1.1" 502 150 2025-12-17 21:38:16,339 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb0575bc060ce15c51c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:16,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb3575bc060ce15c522 2025-12-17 21:38:16,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb3575bc060ce15c522 HTTP/1.1" 502 150 2025-12-17 21:38:16,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb3575bc060ce15c522'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49308 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:38:16,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f9a575bc060ce15c4c9 2025-12-17 21:38:16,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f9a575bc060ce15c4c9 HTTP/1.1" 502 150 2025-12-17 21:38:16,988 - lava_callback - ERROR - Error processing callback for node 69407f9a575bc060ce15c4c9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f9a575bc060ce15c4c9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f9a575bc060ce15c4c9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f9a575bc060ce15c4c9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:17,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fa4575bc060ce15c4ce 2025-12-17 21:38:17,056 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb4575bc060ce15c525 2025-12-17 21:38:17,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb4575bc060ce15c525 HTTP/1.1" 502 150 2025-12-17 21:38:17,057 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb4575bc060ce15c525'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:17,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb5575bc060ce15c528 2025-12-17 21:38:17,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb5575bc060ce15c528 HTTP/1.1" 502 150 2025-12-17 21:38:17,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb5575bc060ce15c528'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:17,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fa7575bc060ce15c4f9 2025-12-17 21:38:17,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa7575bc060ce15c4f9 HTTP/1.1" 502 150 2025-12-17 21:38:17,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fa7575bc060ce15c4f9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:18,318 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb8575bc060ce15c531 2025-12-17 21:38:18,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb8575bc060ce15c531 HTTP/1.1" 502 150 2025-12-17 21:38:18,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb8575bc060ce15c531'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:18,361 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407faf575bc060ce15c519 2025-12-17 21:38:18,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407faf575bc060ce15c519 HTTP/1.1" 502 150 2025-12-17 21:38:18,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407faf575bc060ce15c519'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:19,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb2575bc060ce15c51f 2025-12-17 21:38:19,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb2575bc060ce15c51f HTTP/1.1" 502 150 2025-12-17 21:38:19,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb2575bc060ce15c51f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:19,037 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407faa575bc060ce15c507 2025-12-17 21:38:19,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407faa575bc060ce15c507 HTTP/1.1" 502 150 2025-12-17 21:38:19,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407faa575bc060ce15c507'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:19,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fab575bc060ce15c50e 2025-12-17 21:38:19,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fab575bc060ce15c50e HTTP/1.1" 502 150 2025-12-17 21:38:19,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fab575bc060ce15c50e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:19,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fac575bc060ce15c513 2025-12-17 21:38:19,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fac575bc060ce15c513 HTTP/1.1" 502 150 2025-12-17 21:38:19,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fac575bc060ce15c513'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:19,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408130575bc060ce15ca2e 2025-12-17 21:38:19,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408130575bc060ce15ca2e HTTP/1.1" 502 150 2025-12-17 21:38:19,897 - lava_callback - ERROR - Error processing callback for node 69408130575bc060ce15ca2e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408130575bc060ce15ca2e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408130575bc060ce15ca2e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408130575bc060ce15ca2e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:20,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb3575bc060ce15c522 2025-12-17 21:38:20,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb3575bc060ce15c522 HTTP/1.1" 502 150 2025-12-17 21:38:20,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb3575bc060ce15c522'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:20,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fae575bc060ce15c516 2025-12-17 21:38:20,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fae575bc060ce15c516 HTTP/1.1" 502 150 2025-12-17 21:38:20,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fae575bc060ce15c516'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:21,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb4575bc060ce15c525 2025-12-17 21:38:21,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb4575bc060ce15c525 HTTP/1.1" 502 150 2025-12-17 21:38:21,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb4575bc060ce15c525'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:21,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb5575bc060ce15c528 2025-12-17 21:38:21,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb5575bc060ce15c528 HTTP/1.1" 502 150 2025-12-17 21:38:21,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb5575bc060ce15c528'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:21,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa8575bc060ce15c500 HTTP/1.1" 502 150 2025-12-17 21:38:21,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fa8575bc060ce15c500'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:22,320 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb8575bc060ce15c531 2025-12-17 21:38:22,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb8575bc060ce15c531 HTTP/1.1" 502 150 2025-12-17 21:38:22,321 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb8575bc060ce15c531'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:22,347 - lava_callback - INFO - Received callback for node 69407fbe575bc060ce15c54d INFO: 172.21.0.1:41832 - "POST /node/69407fbe575bc060ce15c54d HTTP/1.0" 202 Accepted 2025-12-17 21:38:22,353 - lava_callback - INFO - Starting async job submit for node 69407fbe575bc060ce15c54d 2025-12-17 21:38:22,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:22,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbe575bc060ce15c54d HTTP/1.1" 502 150 2025-12-17 21:38:22,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbe575bc060ce15c54d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:22,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbe575bc060ce15c54d 2025-12-17 21:38:22,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbe575bc060ce15c54d HTTP/1.1" 502 150 2025-12-17 21:38:22,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbe575bc060ce15c54d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:23,021 - lava_callback - INFO - Received callback for node 69407fba575bc060ce15c538 INFO: 172.21.0.1:41834 - "POST /node/69407fba575bc060ce15c538 HTTP/1.0" 202 Accepted 2025-12-17 21:38:23,027 - lava_callback - INFO - Starting async job submit for node 69407fba575bc060ce15c538 2025-12-17 21:38:23,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:23,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fba575bc060ce15c538 HTTP/1.1" 502 150 2025-12-17 21:38:23,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fba575bc060ce15c538'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:23,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fba575bc060ce15c538 2025-12-17 21:38:23,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fba575bc060ce15c538 HTTP/1.1" 502 150 2025-12-17 21:38:23,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fba575bc060ce15c538'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:23,693 - lava_callback - INFO - Received callback for node 69407fbb575bc060ce15c53f INFO: 172.21.0.1:41846 - "POST /node/69407fbb575bc060ce15c53f HTTP/1.0" 202 Accepted 2025-12-17 21:38:24,357 - lava_callback - INFO - Received callback for node 69407fbc575bc060ce15c546 INFO: 172.21.0.1:41856 - "POST /node/69407fbc575bc060ce15c546 HTTP/1.0" 202 Accepted 2025-12-17 21:38:24,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbe575bc060ce15c54d 2025-12-17 21:38:24,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbe575bc060ce15c54d HTTP/1.1" 502 150 2025-12-17 21:38:24,388 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbe575bc060ce15c54d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:25,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fba575bc060ce15c538 2025-12-17 21:38:25,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fba575bc060ce15c538 HTTP/1.1" 502 150 2025-12-17 21:38:25,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fba575bc060ce15c538'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:25,037 - lava_callback - INFO - Received callback for node 69407fbf575bc060ce15c554 INFO: 172.21.0.1:41858 - "POST /node/69407fbf575bc060ce15c554 HTTP/1.0" 202 Accepted 2025-12-17 21:38:25,683 - lava_callback - INFO - Received callback for node 69407fc0575bc060ce15c557 INFO: 172.21.0.1:41866 - "POST /node/69407fc0575bc060ce15c557 HTTP/1.0" 202 Accepted 2025-12-17 21:38:25,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fa8575bc060ce15c500 2025-12-17 21:38:25,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa8575bc060ce15c500 HTTP/1.1" 502 150 2025-12-17 21:38:25,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fa8575bc060ce15c500'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:26,360 - lava_callback - INFO - Received callback for node 69407fc2575bc060ce15c55a INFO: 172.21.0.1:41878 - "POST /node/69407fc2575bc060ce15c55a HTTP/1.0" 202 Accepted 2025-12-17 21:38:26,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407faf575bc060ce15c519 2025-12-17 21:38:26,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407faf575bc060ce15c519 HTTP/1.1" 502 150 2025-12-17 21:38:26,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407faf575bc060ce15c519'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41892 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:38:27,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb2575bc060ce15c51f 2025-12-17 21:38:27,047 - lava_callback - INFO - Received callback for node 69407fc3575bc060ce15c55d INFO: 172.21.0.1:41894 - "POST /node/69407fc3575bc060ce15c55d HTTP/1.0" 202 Accepted 2025-12-17 21:38:27,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fab575bc060ce15c50e 2025-12-17 21:38:27,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fab575bc060ce15c50e HTTP/1.1" 502 150 2025-12-17 21:38:27,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fab575bc060ce15c50e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:27,737 - lava_callback - INFO - Received callback for node 69407fc4575bc060ce15c560 INFO: 172.21.0.1:41902 - "POST /node/69407fc4575bc060ce15c560 HTTP/1.0" 202 Accepted 2025-12-17 21:38:28,351 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb3575bc060ce15c522 2025-12-17 21:38:28,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb3575bc060ce15c522 HTTP/1.1" 502 150 2025-12-17 21:38:28,356 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb3575bc060ce15c522'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:28,389 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbe575bc060ce15c54d 2025-12-17 21:38:28,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbe575bc060ce15c54d HTTP/1.1" 502 150 2025-12-17 21:38:28,390 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbe575bc060ce15c54d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:28,394 - lava_callback - INFO - Received callback for node 69407fc5575bc060ce15c563 INFO: 172.21.0.1:41912 - "POST /node/69407fc5575bc060ce15c563 HTTP/1.0" 202 Accepted 2025-12-17 21:38:29,037 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fba575bc060ce15c538 2025-12-17 21:38:29,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fba575bc060ce15c538 HTTP/1.1" 502 150 2025-12-17 21:38:29,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fba575bc060ce15c538'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:29,059 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb4575bc060ce15c525 2025-12-17 21:38:29,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb4575bc060ce15c525 HTTP/1.1" 502 150 2025-12-17 21:38:29,060 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb4575bc060ce15c525'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:29,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb5575bc060ce15c528 2025-12-17 21:38:29,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb5575bc060ce15c528 HTTP/1.1" 502 150 2025-12-17 21:38:29,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb5575bc060ce15c528'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:30,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb8575bc060ce15c531 2025-12-17 21:38:30,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb8575bc060ce15c531 HTTP/1.1" 502 150 2025-12-17 21:38:30,322 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb8575bc060ce15c531'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:32,339 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb0575bc060ce15c51c 2025-12-17 21:38:32,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa4575bc060ce15c4ce HTTP/1.1" 502 150 2025-12-17 21:38:32,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fa4575bc060ce15c4ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:33,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fa7575bc060ce15c4f9 2025-12-17 21:38:33,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa7575bc060ce15c4f9 HTTP/1.1" 502 150 2025-12-17 21:38:33,715 - lava_callback - ERROR - Error processing callback for node 69407fa7575bc060ce15c4f9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fa7575bc060ce15c4f9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fa7575bc060ce15c4f9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fa7575bc060ce15c4f9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:33,718 - lava_callback - INFO - Starting async job submit for node 69407fbb575bc060ce15c53f 2025-12-17 21:38:33,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:33,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbb575bc060ce15c53f HTTP/1.1" 502 150 2025-12-17 21:38:33,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbb575bc060ce15c53f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:33,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbb575bc060ce15c53f 2025-12-17 21:38:33,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbb575bc060ce15c53f HTTP/1.1" 502 150 2025-12-17 21:38:33,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbb575bc060ce15c53f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:33,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fa8575bc060ce15c500 2025-12-17 21:38:33,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa8575bc060ce15c500 HTTP/1.1" 502 150 2025-12-17 21:38:33,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fa8575bc060ce15c500'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:35,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407faa575bc060ce15c507 2025-12-17 21:38:35,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407faa575bc060ce15c507 HTTP/1.1" 502 150 2025-12-17 21:38:35,040 - lava_callback - ERROR - Error processing callback for node 69407faa575bc060ce15c507: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407faa575bc060ce15c507 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407faa575bc060ce15c507 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407faa575bc060ce15c507 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:35,041 - lava_callback - INFO - Starting async job submit for node 69407fbc575bc060ce15c546 2025-12-17 21:38:35,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:35,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbc575bc060ce15c546 HTTP/1.1" 502 150 2025-12-17 21:38:35,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbc575bc060ce15c546'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:35,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbc575bc060ce15c546 2025-12-17 21:38:35,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbc575bc060ce15c546 HTTP/1.1" 502 150 2025-12-17 21:38:35,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbc575bc060ce15c546'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:35,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fac575bc060ce15c513 2025-12-17 21:38:35,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fac575bc060ce15c513 HTTP/1.1" 502 150 2025-12-17 21:38:35,713 - lava_callback - ERROR - Error processing callback for node 69407fac575bc060ce15c513: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fac575bc060ce15c513 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fac575bc060ce15c513 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fac575bc060ce15c513 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:35,715 - lava_callback - INFO - Starting async job submit for node 69407fbf575bc060ce15c554 2025-12-17 21:38:35,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:35,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbb575bc060ce15c53f 2025-12-17 21:38:35,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbb575bc060ce15c53f HTTP/1.1" 502 150 2025-12-17 21:38:35,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbb575bc060ce15c53f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:35,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbf575bc060ce15c554 HTTP/1.1" 502 150 2025-12-17 21:38:35,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbf575bc060ce15c554'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:35,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbf575bc060ce15c554 2025-12-17 21:38:35,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbf575bc060ce15c554 HTTP/1.1" 502 150 2025-12-17 21:38:35,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbf575bc060ce15c554'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:36,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fae575bc060ce15c516 2025-12-17 21:38:36,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fae575bc060ce15c516 HTTP/1.1" 502 150 2025-12-17 21:38:36,385 - lava_callback - ERROR - Error processing callback for node 69407fae575bc060ce15c516: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fae575bc060ce15c516 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fae575bc060ce15c516 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fae575bc060ce15c516 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:36,387 - lava_callback - INFO - Starting async job submit for node 69407fc0575bc060ce15c557 2025-12-17 21:38:36,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:36,390 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbe575bc060ce15c54d 2025-12-17 21:38:36,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbe575bc060ce15c54d HTTP/1.1" 502 150 2025-12-17 21:38:36,391 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbe575bc060ce15c54d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:36,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc0575bc060ce15c557 HTTP/1.1" 502 150 2025-12-17 21:38:36,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc0575bc060ce15c557'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:36,395 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc0575bc060ce15c557 2025-12-17 21:38:36,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc0575bc060ce15c557 HTTP/1.1" 502 150 2025-12-17 21:38:36,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc0575bc060ce15c557'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40732 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:38:36,656 - lava_callback - INFO - Received callback for node 6940851d575bc060ce15da7d INFO: 172.21.0.1:40734 - "POST /node/6940851d575bc060ce15da7d HTTP/1.0" 202 Accepted 2025-12-17 21:38:37,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fba575bc060ce15c538 2025-12-17 21:38:37,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fba575bc060ce15c538 HTTP/1.1" 502 150 2025-12-17 21:38:37,040 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fba575bc060ce15c538'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:37,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbc575bc060ce15c546 2025-12-17 21:38:37,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbc575bc060ce15c546 HTTP/1.1" 502 150 2025-12-17 21:38:37,050 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbc575bc060ce15c546'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:37,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbf575bc060ce15c554 2025-12-17 21:38:37,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbf575bc060ce15c554 HTTP/1.1" 502 150 2025-12-17 21:38:37,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbf575bc060ce15c554'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:38,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc0575bc060ce15c557 2025-12-17 21:38:39,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbb575bc060ce15c53f 2025-12-17 21:38:39,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbb575bc060ce15c53f HTTP/1.1" 502 150 2025-12-17 21:38:39,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbb575bc060ce15c53f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:41,050 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbc575bc060ce15c546 2025-12-17 21:38:41,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbc575bc060ce15c546 HTTP/1.1" 502 150 2025-12-17 21:38:41,051 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbc575bc060ce15c546'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:41,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbf575bc060ce15c554 2025-12-17 21:38:41,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbf575bc060ce15c554 HTTP/1.1" 502 150 2025-12-17 21:38:41,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbf575bc060ce15c554'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:42,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407faf575bc060ce15c519 2025-12-17 21:38:42,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407faf575bc060ce15c519 HTTP/1.1" 502 150 2025-12-17 21:38:42,368 - lava_callback - ERROR - Error processing callback for node 69407faf575bc060ce15c519: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407faf575bc060ce15c519 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407faf575bc060ce15c519 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407faf575bc060ce15c519 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:42,370 - lava_callback - INFO - Starting async job submit for node 69407fc2575bc060ce15c55a 2025-12-17 21:38:42,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:42,374 - lava_callback - INFO - Received callback for node 69407fcc575bc060ce15c568 INFO: 172.21.0.1:44228 - "POST /node/69407fcc575bc060ce15c568 HTTP/1.0" 202 Accepted 2025-12-17 21:38:42,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc2575bc060ce15c55a HTTP/1.1" 502 150 2025-12-17 21:38:42,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc2575bc060ce15c55a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:42,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc2575bc060ce15c55a 2025-12-17 21:38:42,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc2575bc060ce15c55a HTTP/1.1" 502 150 2025-12-17 21:38:42,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc2575bc060ce15c55a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:42,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb2575bc060ce15c51f HTTP/1.1" 502 150 2025-12-17 21:38:42,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fb2575bc060ce15c51f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:43,041 - lava_callback - INFO - Received callback for node 69407fcf575bc060ce15c571 INFO: 172.21.0.1:44230 - "POST /node/69407fcf575bc060ce15c571 HTTP/1.0" 202 Accepted 2025-12-17 21:38:43,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fab575bc060ce15c50e 2025-12-17 21:38:43,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fab575bc060ce15c50e HTTP/1.1" 502 150 2025-12-17 21:38:43,686 - lava_callback - ERROR - Error processing callback for node 69407fab575bc060ce15c50e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fab575bc060ce15c50e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fab575bc060ce15c50e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fab575bc060ce15c50e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:43,688 - lava_callback - INFO - Starting async job submit for node 69407fc3575bc060ce15c55d 2025-12-17 21:38:43,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:43,718 - lava_callback - INFO - Received callback for node 69407fd0575bc060ce15c578 INFO: 172.21.0.1:44238 - "POST /node/69407fd0575bc060ce15c578 HTTP/1.0" 202 Accepted 2025-12-17 21:38:43,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc3575bc060ce15c55d HTTP/1.1" 502 150 2025-12-17 21:38:43,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc3575bc060ce15c55d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:43,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc3575bc060ce15c55d 2025-12-17 21:38:43,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc3575bc060ce15c55d HTTP/1.1" 502 150 2025-12-17 21:38:43,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc3575bc060ce15c55d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:44,356 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb3575bc060ce15c522 2025-12-17 21:38:44,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb3575bc060ce15c522 HTTP/1.1" 502 150 2025-12-17 21:38:44,358 - lava_callback - ERROR - Error processing callback for node 69407fb3575bc060ce15c522: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb3575bc060ce15c522 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb3575bc060ce15c522 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb3575bc060ce15c522 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:44,360 - lava_callback - INFO - Starting async job submit for node 69407fc4575bc060ce15c560 2025-12-17 21:38:44,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:44,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc4575bc060ce15c560 HTTP/1.1" 502 150 2025-12-17 21:38:44,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc4575bc060ce15c560'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:44,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc4575bc060ce15c560 2025-12-17 21:38:44,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc4575bc060ce15c560 HTTP/1.1" 502 150 2025-12-17 21:38:44,367 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc4575bc060ce15c560'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:44,376 - lava_callback - INFO - Received callback for node 69407fd1575bc060ce15c57f INFO: 172.21.0.1:44248 - "POST /node/69407fd1575bc060ce15c57f HTTP/1.0" 202 Accepted 2025-12-17 21:38:44,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc2575bc060ce15c55a 2025-12-17 21:38:44,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc2575bc060ce15c55a HTTP/1.1" 502 150 2025-12-17 21:38:44,389 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc2575bc060ce15c55a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:45,035 - lava_callback - INFO - Received callback for node 69407fd3575bc060ce15c586 INFO: 172.21.0.1:44264 - "POST /node/69407fd3575bc060ce15c586 HTTP/1.0" 202 Accepted 2025-12-17 21:38:45,060 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb4575bc060ce15c525 2025-12-17 21:38:45,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb4575bc060ce15c525 HTTP/1.1" 502 150 2025-12-17 21:38:45,063 - lava_callback - ERROR - Error processing callback for node 69407fb4575bc060ce15c525: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb4575bc060ce15c525 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb4575bc060ce15c525 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb4575bc060ce15c525 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:45,065 - lava_callback - INFO - Starting async job submit for node 69407fc5575bc060ce15c563 2025-12-17 21:38:45,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:45,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc5575bc060ce15c563 HTTP/1.1" 502 150 2025-12-17 21:38:45,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc5575bc060ce15c563'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:45,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc5575bc060ce15c563 2025-12-17 21:38:45,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc5575bc060ce15c563 HTTP/1.1" 502 150 2025-12-17 21:38:45,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc5575bc060ce15c563'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:45,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb5575bc060ce15c528 2025-12-17 21:38:45,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb5575bc060ce15c528 HTTP/1.1" 502 150 2025-12-17 21:38:45,709 - lava_callback - ERROR - Error processing callback for node 69407fb5575bc060ce15c528: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb5575bc060ce15c528 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb5575bc060ce15c528 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb5575bc060ce15c528 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:45,711 - lava_callback - INFO - Starting async job submit for node 6940851d575bc060ce15da7d 2025-12-17 21:38:45,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:45,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc3575bc060ce15c55d 2025-12-17 21:38:45,729 - lava_callback - INFO - Received callback for node 69407fd4575bc060ce15c58d INFO: 172.21.0.1:44274 - "POST /node/69407fd4575bc060ce15c58d HTTP/1.0" 202 Accepted 2025-12-17 21:38:45,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc3575bc060ce15c55d HTTP/1.1" 502 150 2025-12-17 21:38:45,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc3575bc060ce15c55d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:45,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940851d575bc060ce15da7d HTTP/1.1" 502 150 2025-12-17 21:38:45,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940851d575bc060ce15da7d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:45,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940851d575bc060ce15da7d 2025-12-17 21:38:45,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940851d575bc060ce15da7d HTTP/1.1" 502 150 2025-12-17 21:38:45,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940851d575bc060ce15da7d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:46,323 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb8575bc060ce15c531 2025-12-17 21:38:46,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb8575bc060ce15c531 HTTP/1.1" 502 150 2025-12-17 21:38:46,324 - lava_callback - ERROR - Error processing callback for node 69407fb8575bc060ce15c531: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb8575bc060ce15c531 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb8575bc060ce15c531 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb8575bc060ce15c531 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:46,326 - lava_callback - INFO - Starting async job submit for node 69407fcc575bc060ce15c568 2025-12-17 21:38:46,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:46,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fcc575bc060ce15c568 HTTP/1.1" 502 150 2025-12-17 21:38:46,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fcc575bc060ce15c568'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:46,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fcc575bc060ce15c568 2025-12-17 21:38:46,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fcc575bc060ce15c568 HTTP/1.1" 502 150 2025-12-17 21:38:46,333 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fcc575bc060ce15c568'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:46,367 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc4575bc060ce15c560 2025-12-17 21:38:46,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc4575bc060ce15c560 HTTP/1.1" 502 150 2025-12-17 21:38:46,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc4575bc060ce15c560'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:46,404 - lava_callback - INFO - Received callback for node 69407fd5575bc060ce15c590 INFO: 172.21.0.1:44290 - "POST /node/69407fd5575bc060ce15c590 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:44292 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:38:47,050 - lava_callback - INFO - Received callback for node 69407fd6575bc060ce15c593 INFO: 172.21.0.1:44304 - "POST /node/69407fd6575bc060ce15c593 HTTP/1.0" 202 Accepted 2025-12-17 21:38:47,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc5575bc060ce15c563 2025-12-17 21:38:47,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc5575bc060ce15c563 HTTP/1.1" 502 150 2025-12-17 21:38:47,074 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc5575bc060ce15c563'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:47,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb0575bc060ce15c51c HTTP/1.1" 502 150 2025-12-17 21:38:47,600 - lava_callback - ERROR - Error processing callback for node 69407fb0575bc060ce15c51c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb0575bc060ce15c51c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb0575bc060ce15c51c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb0575bc060ce15c51c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:47,603 - lava_callback - INFO - Starting async job submit for node 69407fcf575bc060ce15c571 2025-12-17 21:38:47,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:47,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fcf575bc060ce15c571 HTTP/1.1" 502 150 2025-12-17 21:38:47,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fcf575bc060ce15c571'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:47,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fcf575bc060ce15c571 2025-12-17 21:38:47,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fcf575bc060ce15c571 HTTP/1.1" 502 150 2025-12-17 21:38:47,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fcf575bc060ce15c571'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:47,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbb575bc060ce15c53f 2025-12-17 21:38:47,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbb575bc060ce15c53f HTTP/1.1" 502 150 2025-12-17 21:38:47,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbb575bc060ce15c53f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:47,735 - lava_callback - INFO - Received callback for node 69407fd8575bc060ce15c59a 2025-12-17 21:38:47,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940851d575bc060ce15da7d INFO: 172.21.0.1:44312 - "POST /node/69407fd8575bc060ce15c59a HTTP/1.0" 202 Accepted 2025-12-17 21:38:47,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940851d575bc060ce15da7d HTTP/1.1" 502 150 2025-12-17 21:38:47,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940851d575bc060ce15da7d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:48,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fcc575bc060ce15c568 2025-12-17 21:38:48,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fcc575bc060ce15c568 HTTP/1.1" 502 150 2025-12-17 21:38:48,335 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fcc575bc060ce15c568'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:48,389 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc2575bc060ce15c55a 2025-12-17 21:38:48,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc2575bc060ce15c55a HTTP/1.1" 502 150 2025-12-17 21:38:48,390 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc2575bc060ce15c55a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:48,417 - lava_callback - INFO - Received callback for node 69407fd9575bc060ce15c59d INFO: 172.21.0.1:44328 - "POST /node/69407fd9575bc060ce15c59d HTTP/1.0" 202 Accepted 2025-12-17 21:38:48,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fa4575bc060ce15c4ce 2025-12-17 21:38:48,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa4575bc060ce15c4ce HTTP/1.1" 502 150 2025-12-17 21:38:48,498 - lava_callback - ERROR - Error processing callback for node 69407fa4575bc060ce15c4ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fa4575bc060ce15c4ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fa4575bc060ce15c4ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fa4575bc060ce15c4ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:48,501 - lava_callback - INFO - Starting async job submit for node 69407fd0575bc060ce15c578 2025-12-17 21:38:48,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:48,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd0575bc060ce15c578 HTTP/1.1" 502 150 2025-12-17 21:38:48,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd0575bc060ce15c578'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:48,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd0575bc060ce15c578 2025-12-17 21:38:48,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd0575bc060ce15c578 HTTP/1.1" 502 150 2025-12-17 21:38:48,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd0575bc060ce15c578'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:49,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbc575bc060ce15c546 2025-12-17 21:38:49,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbc575bc060ce15c546 HTTP/1.1" 502 150 2025-12-17 21:38:49,053 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbc575bc060ce15c546'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:49,063 - lava_callback - INFO - Received callback for node 69407fda575bc060ce15c5a0 INFO: 172.21.0.1:44342 - "POST /node/69407fda575bc060ce15c5a0 HTTP/1.0" 202 Accepted 2025-12-17 21:38:49,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fcf575bc060ce15c571 2025-12-17 21:38:49,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fcf575bc060ce15c571 HTTP/1.1" 502 150 2025-12-17 21:38:49,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fcf575bc060ce15c571'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:49,721 - lava_callback - INFO - Received callback for node 69407fdb575bc060ce15c5a3 INFO: 172.21.0.1:40814 - "POST /node/69407fdb575bc060ce15c5a3 HTTP/1.0" 202 Accepted 2025-12-17 21:38:49,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbf575bc060ce15c554 2025-12-17 21:38:49,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbf575bc060ce15c554 HTTP/1.1" 502 150 2025-12-17 21:38:49,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fbf575bc060ce15c554'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:49,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc3575bc060ce15c55d 2025-12-17 21:38:49,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc3575bc060ce15c55d HTTP/1.1" 502 150 2025-12-17 21:38:49,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc3575bc060ce15c55d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:49,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fa8575bc060ce15c500 2025-12-17 21:38:49,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fa8575bc060ce15c500 HTTP/1.1" 502 150 2025-12-17 21:38:49,748 - lava_callback - ERROR - Error processing callback for node 69407fa8575bc060ce15c500: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fa8575bc060ce15c500 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fa8575bc060ce15c500 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fa8575bc060ce15c500 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:49,749 - lava_callback - INFO - Starting async job submit for node 69407fd1575bc060ce15c57f 2025-12-17 21:38:49,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:49,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd1575bc060ce15c57f HTTP/1.1" 502 150 2025-12-17 21:38:49,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd1575bc060ce15c57f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:49,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd1575bc060ce15c57f 2025-12-17 21:38:49,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd1575bc060ce15c57f HTTP/1.1" 502 150 2025-12-17 21:38:49,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd1575bc060ce15c57f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:50,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc4575bc060ce15c560 2025-12-17 21:38:50,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc4575bc060ce15c560 HTTP/1.1" 502 150 2025-12-17 21:38:50,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc4575bc060ce15c560'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:50,376 - lava_callback - INFO - Received callback for node 69407fe1575bc060ce15c5a8 INFO: 172.21.0.1:40816 - "POST /node/69407fe1575bc060ce15c5a8 HTTP/1.0" 202 Accepted 2025-12-17 21:38:50,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd0575bc060ce15c578 2025-12-17 21:38:50,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd0575bc060ce15c578 HTTP/1.1" 502 150 2025-12-17 21:38:50,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd0575bc060ce15c578'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:51,074 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc5575bc060ce15c563 2025-12-17 21:38:51,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc5575bc060ce15c563 HTTP/1.1" 502 150 2025-12-17 21:38:51,076 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc5575bc060ce15c563'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:51,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd1575bc060ce15c57f 2025-12-17 21:38:51,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd1575bc060ce15c57f HTTP/1.1" 502 150 2025-12-17 21:38:51,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd1575bc060ce15c57f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:51,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940851d575bc060ce15da7d 2025-12-17 21:38:51,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940851d575bc060ce15da7d HTTP/1.1" 502 150 2025-12-17 21:38:51,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940851d575bc060ce15da7d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:52,335 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fcc575bc060ce15c568 2025-12-17 21:38:52,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fcc575bc060ce15c568 HTTP/1.1" 502 150 2025-12-17 21:38:52,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fcc575bc060ce15c568'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:52,391 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbe575bc060ce15c54d 2025-12-17 21:38:52,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbe575bc060ce15c54d HTTP/1.1" 502 150 2025-12-17 21:38:52,393 - lava_callback - ERROR - Error processing callback for node 69407fbe575bc060ce15c54d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fbe575bc060ce15c54d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fbe575bc060ce15c54d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fbe575bc060ce15c54d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:52,395 - lava_callback - INFO - Starting async job submit for node 69407fd3575bc060ce15c586 2025-12-17 21:38:52,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:52,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd3575bc060ce15c586 HTTP/1.1" 502 150 2025-12-17 21:38:52,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd3575bc060ce15c586'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:52,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd3575bc060ce15c586 2025-12-17 21:38:52,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd3575bc060ce15c586 HTTP/1.1" 502 150 2025-12-17 21:38:52,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd3575bc060ce15c586'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:53,040 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fba575bc060ce15c538 2025-12-17 21:38:53,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fcf575bc060ce15c571 2025-12-17 21:38:53,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fcf575bc060ce15c571 HTTP/1.1" 502 150 2025-12-17 21:38:53,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fcf575bc060ce15c571'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:53,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc0575bc060ce15c557 HTTP/1.1" 502 150 2025-12-17 21:38:53,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc0575bc060ce15c557'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:54,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd3575bc060ce15c586 2025-12-17 21:38:54,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd3575bc060ce15c586 HTTP/1.1" 502 150 2025-12-17 21:38:54,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd3575bc060ce15c586'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:54,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd0575bc060ce15c578 2025-12-17 21:38:54,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd0575bc060ce15c578 HTTP/1.1" 502 150 2025-12-17 21:38:54,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd0575bc060ce15c578'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:55,433 - lava_callback - INFO - Received callback for node 69408d51575bc060ce15f811 INFO: 172.21.0.1:40830 - "POST /node/69408d51575bc060ce15f811 HTTP/1.0" 202 Accepted 2025-12-17 21:38:55,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd1575bc060ce15c57f 2025-12-17 21:38:55,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd1575bc060ce15c57f HTTP/1.1" 502 150 2025-12-17 21:38:55,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd1575bc060ce15c57f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:56,391 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc2575bc060ce15c55a 2025-12-17 21:38:56,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc2575bc060ce15c55a HTTP/1.1" 502 150 2025-12-17 21:38:56,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc2575bc060ce15c55a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40844 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:38:57,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc3575bc060ce15c55d 2025-12-17 21:38:57,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc3575bc060ce15c55d HTTP/1.1" 502 150 2025-12-17 21:38:57,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc3575bc060ce15c55d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:57,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc0575bc060ce15c557 2025-12-17 21:38:57,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc0575bc060ce15c557 HTTP/1.1" 502 150 2025-12-17 21:38:57,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc0575bc060ce15c557'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:58,369 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc4575bc060ce15c560 2025-12-17 21:38:58,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc4575bc060ce15c560 HTTP/1.1" 502 150 2025-12-17 21:38:58,371 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc4575bc060ce15c560'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:58,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd3575bc060ce15c586 2025-12-17 21:38:58,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd3575bc060ce15c586 HTTP/1.1" 502 150 2025-12-17 21:38:58,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd3575bc060ce15c586'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:58,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fb2575bc060ce15c51f 2025-12-17 21:38:58,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fb2575bc060ce15c51f HTTP/1.1" 502 150 2025-12-17 21:38:58,483 - lava_callback - ERROR - Error processing callback for node 69407fb2575bc060ce15c51f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb2575bc060ce15c51f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb2575bc060ce15c51f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fb2575bc060ce15c51f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:38:58,485 - lava_callback - INFO - Starting async job submit for node 69407fd4575bc060ce15c58d 2025-12-17 21:38:58,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:38:58,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd4575bc060ce15c58d HTTP/1.1" 502 150 2025-12-17 21:38:58,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd4575bc060ce15c58d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:58,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd4575bc060ce15c58d 2025-12-17 21:38:58,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd4575bc060ce15c58d HTTP/1.1" 502 150 2025-12-17 21:38:58,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd4575bc060ce15c58d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:59,076 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc5575bc060ce15c563 2025-12-17 21:38:59,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc5575bc060ce15c563 HTTP/1.1" 502 150 2025-12-17 21:38:59,077 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc5575bc060ce15c563'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:38:59,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940851d575bc060ce15da7d 2025-12-17 21:38:59,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940851d575bc060ce15da7d HTTP/1.1" 502 150 2025-12-17 21:38:59,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940851d575bc060ce15da7d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:00,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fcc575bc060ce15c568 2025-12-17 21:39:00,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fcc575bc060ce15c568 HTTP/1.1" 502 150 2025-12-17 21:39:00,338 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fcc575bc060ce15c568'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:00,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd4575bc060ce15c58d 2025-12-17 21:39:00,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd4575bc060ce15c58d HTTP/1.1" 502 150 2025-12-17 21:39:00,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd4575bc060ce15c58d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:01,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fcf575bc060ce15c571 2025-12-17 21:39:01,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fcf575bc060ce15c571 HTTP/1.1" 502 150 2025-12-17 21:39:01,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fcf575bc060ce15c571'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55020 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:39:01,660 - lava_callback - INFO - Received callback for node 694085fd575bc060ce15de96 INFO: 172.21.0.1:55036 - "POST /node/694085fd575bc060ce15de96 HTTP/1.0" 202 Accepted 2025-12-17 21:39:02,311 - lava_callback - INFO - Received callback for node 69407fe2575bc060ce15c5af INFO: 172.21.0.1:55042 - "POST /node/69407fe2575bc060ce15c5af HTTP/1.0" 202 Accepted 2025-12-17 21:39:02,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd0575bc060ce15c578 2025-12-17 21:39:02,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd0575bc060ce15c578 HTTP/1.1" 502 150 2025-12-17 21:39:02,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd0575bc060ce15c578'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:02,970 - lava_callback - INFO - Received callback for node 69407fe4575bc060ce15c5d3 INFO: 172.21.0.1:55058 - "POST /node/69407fe4575bc060ce15c5d3 HTTP/1.0" 202 Accepted 2025-12-17 21:39:03,619 - lava_callback - INFO - Received callback for node 69407fe6575bc060ce15c5e6 INFO: 172.21.0.1:55062 - "POST /node/69407fe6575bc060ce15c5e6 HTTP/1.0" 202 Accepted 2025-12-17 21:39:03,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbb575bc060ce15c53f 2025-12-17 21:39:03,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbb575bc060ce15c53f HTTP/1.1" 502 150 2025-12-17 21:39:03,731 - lava_callback - ERROR - Error processing callback for node 69407fbb575bc060ce15c53f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fbb575bc060ce15c53f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fbb575bc060ce15c53f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fbb575bc060ce15c53f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:03,733 - lava_callback - INFO - Starting async job submit for node 69407fd5575bc060ce15c590 2025-12-17 21:39:03,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:03,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd5575bc060ce15c590 HTTP/1.1" 502 150 2025-12-17 21:39:03,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd5575bc060ce15c590'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:03,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd5575bc060ce15c590 2025-12-17 21:39:03,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd5575bc060ce15c590 HTTP/1.1" 502 150 2025-12-17 21:39:03,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd5575bc060ce15c590'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:03,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd1575bc060ce15c57f 2025-12-17 21:39:03,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd1575bc060ce15c57f HTTP/1.1" 502 150 2025-12-17 21:39:03,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd1575bc060ce15c57f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:04,283 - lava_callback - INFO - Received callback for node 69407fe7575bc060ce15c5e9 INFO: 172.21.0.1:55076 - "POST /node/69407fe7575bc060ce15c5e9 HTTP/1.0" 202 Accepted 2025-12-17 21:39:04,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd4575bc060ce15c58d 2025-12-17 21:39:04,969 - lava_callback - INFO - Received callback for node 69407fe8575bc060ce15c5ec INFO: 172.21.0.1:55078 - "POST /node/69407fe8575bc060ce15c5ec HTTP/1.0" 202 Accepted 2025-12-17 21:39:05,053 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbc575bc060ce15c546 2025-12-17 21:39:05,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbc575bc060ce15c546 HTTP/1.1" 502 150 2025-12-17 21:39:05,054 - lava_callback - ERROR - Error processing callback for node 69407fbc575bc060ce15c546: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fbc575bc060ce15c546 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fbc575bc060ce15c546 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fbc575bc060ce15c546 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:05,057 - lava_callback - INFO - Starting async job submit for node 69407fd6575bc060ce15c593 2025-12-17 21:39:05,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:05,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd6575bc060ce15c593 HTTP/1.1" 502 150 2025-12-17 21:39:05,064 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd6575bc060ce15c593'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:05,064 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd6575bc060ce15c593 2025-12-17 21:39:05,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd6575bc060ce15c593 HTTP/1.1" 502 150 2025-12-17 21:39:05,066 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd6575bc060ce15c593'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:05,625 - lava_callback - INFO - Received callback for node 69407fe9575bc060ce15c5ef INFO: 172.21.0.1:55088 - "POST /node/69407fe9575bc060ce15c5ef HTTP/1.0" 202 Accepted 2025-12-17 21:39:05,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fbf575bc060ce15c554 2025-12-17 21:39:05,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fbf575bc060ce15c554 HTTP/1.1" 502 150 2025-12-17 21:39:05,735 - lava_callback - ERROR - Error processing callback for node 69407fbf575bc060ce15c554: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fbf575bc060ce15c554 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fbf575bc060ce15c554 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fbf575bc060ce15c554 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:05,737 - lava_callback - INFO - Starting async job submit for node 69407fd8575bc060ce15c59a 2025-12-17 21:39:05,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:05,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc0575bc060ce15c557 2025-12-17 21:39:05,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd5575bc060ce15c590 2025-12-17 21:39:05,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc0575bc060ce15c557 HTTP/1.1" 502 150 2025-12-17 21:39:05,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fc0575bc060ce15c557'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:05,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd5575bc060ce15c590 HTTP/1.1" 502 150 2025-12-17 21:39:05,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd5575bc060ce15c590'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:05,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd8575bc060ce15c59a HTTP/1.1" 502 150 2025-12-17 21:39:05,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd8575bc060ce15c59a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:05,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd8575bc060ce15c59a 2025-12-17 21:39:05,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd8575bc060ce15c59a HTTP/1.1" 502 150 2025-12-17 21:39:05,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd8575bc060ce15c59a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:06,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd3575bc060ce15c586 2025-12-17 21:39:06,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd3575bc060ce15c586 HTTP/1.1" 502 150 2025-12-17 21:39:06,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd3575bc060ce15c586'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55094 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:39:07,066 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd6575bc060ce15c593 2025-12-17 21:39:07,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd6575bc060ce15c593 HTTP/1.1" 502 150 2025-12-17 21:39:07,068 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd6575bc060ce15c593'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:07,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd8575bc060ce15c59a 2025-12-17 21:39:07,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd8575bc060ce15c59a HTTP/1.1" 502 150 2025-12-17 21:39:07,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd8575bc060ce15c59a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:08,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fba575bc060ce15c538 HTTP/1.1" 502 150 2025-12-17 21:39:08,336 - lava_callback - ERROR - Error processing callback for node 69407fba575bc060ce15c538: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fba575bc060ce15c538 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fba575bc060ce15c538 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fba575bc060ce15c538 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:08,339 - lava_callback - INFO - Starting async job submit for node 69407fd9575bc060ce15c59d 2025-12-17 21:39:08,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:08,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd9575bc060ce15c59d HTTP/1.1" 502 150 2025-12-17 21:39:08,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd9575bc060ce15c59d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:08,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd9575bc060ce15c59d 2025-12-17 21:39:08,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd9575bc060ce15c59d HTTP/1.1" 502 150 2025-12-17 21:39:08,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd9575bc060ce15c59d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:09,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd5575bc060ce15c590 2025-12-17 21:39:09,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd5575bc060ce15c590 HTTP/1.1" 502 150 2025-12-17 21:39:09,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd5575bc060ce15c590'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:10,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd9575bc060ce15c59d 2025-12-17 21:39:10,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd9575bc060ce15c59d HTTP/1.1" 502 150 2025-12-17 21:39:10,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd9575bc060ce15c59d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:11,068 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd6575bc060ce15c593 2025-12-17 21:39:11,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd6575bc060ce15c593 HTTP/1.1" 502 150 2025-12-17 21:39:11,069 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd6575bc060ce15c593'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:11,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd8575bc060ce15c59a 2025-12-17 21:39:11,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd8575bc060ce15c59a HTTP/1.1" 502 150 2025-12-17 21:39:11,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd8575bc060ce15c59a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:12,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc2575bc060ce15c55a 2025-12-17 21:39:12,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc2575bc060ce15c55a HTTP/1.1" 502 150 2025-12-17 21:39:12,394 - lava_callback - ERROR - Error processing callback for node 69407fc2575bc060ce15c55a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fc2575bc060ce15c55a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fc2575bc060ce15c55a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fc2575bc060ce15c55a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:12,396 - lava_callback - INFO - Starting async job submit for node 69407fda575bc060ce15c5a0 2025-12-17 21:39:12,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:12,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fda575bc060ce15c5a0 HTTP/1.1" 502 150 2025-12-17 21:39:12,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fda575bc060ce15c5a0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:12,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fda575bc060ce15c5a0 2025-12-17 21:39:12,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fda575bc060ce15c5a0 HTTP/1.1" 502 150 2025-12-17 21:39:12,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fda575bc060ce15c5a0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:13,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc3575bc060ce15c55d 2025-12-17 21:39:13,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc3575bc060ce15c55d HTTP/1.1" 502 150 2025-12-17 21:39:13,746 - lava_callback - ERROR - Error processing callback for node 69407fc3575bc060ce15c55d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fc3575bc060ce15c55d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fc3575bc060ce15c55d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fc3575bc060ce15c55d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:13,748 - lava_callback - INFO - Starting async job submit for node 69407fdb575bc060ce15c5a3 2025-12-17 21:39:13,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:13,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fdb575bc060ce15c5a3 HTTP/1.1" 502 150 2025-12-17 21:39:13,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fdb575bc060ce15c5a3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:13,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fdb575bc060ce15c5a3 2025-12-17 21:39:13,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fdb575bc060ce15c5a3 HTTP/1.1" 502 150 2025-12-17 21:39:13,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fdb575bc060ce15c5a3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:14,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd9575bc060ce15c59d 2025-12-17 21:39:14,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd9575bc060ce15c59d HTTP/1.1" 502 150 2025-12-17 21:39:14,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd9575bc060ce15c59d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:14,371 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc4575bc060ce15c560 2025-12-17 21:39:14,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc4575bc060ce15c560 HTTP/1.1" 502 150 2025-12-17 21:39:14,372 - lava_callback - ERROR - Error processing callback for node 69407fc4575bc060ce15c560: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fc4575bc060ce15c560 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fc4575bc060ce15c560 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fc4575bc060ce15c560 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:14,374 - lava_callback - INFO - Starting async job submit for node 69407fe1575bc060ce15c5a8 2025-12-17 21:39:14,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:14,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe1575bc060ce15c5a8 HTTP/1.1" 502 150 2025-12-17 21:39:14,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe1575bc060ce15c5a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:14,381 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe1575bc060ce15c5a8 2025-12-17 21:39:14,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe1575bc060ce15c5a8 HTTP/1.1" 502 150 2025-12-17 21:39:14,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe1575bc060ce15c5a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:14,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fda575bc060ce15c5a0 2025-12-17 21:39:14,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fda575bc060ce15c5a0 HTTP/1.1" 502 150 2025-12-17 21:39:14,405 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fda575bc060ce15c5a0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:15,078 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc5575bc060ce15c563 2025-12-17 21:39:15,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc5575bc060ce15c563 HTTP/1.1" 502 150 2025-12-17 21:39:15,079 - lava_callback - ERROR - Error processing callback for node 69407fc5575bc060ce15c563: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fc5575bc060ce15c563 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fc5575bc060ce15c563 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fc5575bc060ce15c563 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:15,081 - lava_callback - INFO - Starting async job submit for node 69408d51575bc060ce15f811 2025-12-17 21:39:15,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:39964 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:39:15,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d51575bc060ce15f811 HTTP/1.1" 502 150 2025-12-17 21:39:15,170 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d51575bc060ce15f811'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:15,170 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d51575bc060ce15f811 2025-12-17 21:39:15,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d51575bc060ce15f811 HTTP/1.1" 502 150 2025-12-17 21:39:15,171 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d51575bc060ce15f811'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:15,533 - lava_callback - INFO - Received callback for node 694086f5575bc060ce15e223 INFO: 172.21.0.1:39976 - "POST /node/694086f5575bc060ce15e223 HTTP/1.0" 202 Accepted 2025-12-17 21:39:15,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940851d575bc060ce15da7d 2025-12-17 21:39:15,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940851d575bc060ce15da7d HTTP/1.1" 502 150 2025-12-17 21:39:15,771 - lava_callback - ERROR - Error processing callback for node 6940851d575bc060ce15da7d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940851d575bc060ce15da7d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940851d575bc060ce15da7d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940851d575bc060ce15da7d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:15,773 - lava_callback - INFO - Starting async job submit for node 694085fd575bc060ce15de96 2025-12-17 21:39:15,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fdb575bc060ce15c5a3 2025-12-17 21:39:15,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:15,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fdb575bc060ce15c5a3 HTTP/1.1" 502 150 2025-12-17 21:39:15,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fdb575bc060ce15c5a3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:15,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085fd575bc060ce15de96 HTTP/1.1" 502 150 2025-12-17 21:39:15,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085fd575bc060ce15de96'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:15,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085fd575bc060ce15de96 2025-12-17 21:39:15,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085fd575bc060ce15de96 HTTP/1.1" 502 150 2025-12-17 21:39:15,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085fd575bc060ce15de96'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:16,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fcc575bc060ce15c568 2025-12-17 21:39:16,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fcc575bc060ce15c568 HTTP/1.1" 502 150 2025-12-17 21:39:16,340 - lava_callback - ERROR - Error processing callback for node 69407fcc575bc060ce15c568: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fcc575bc060ce15c568 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fcc575bc060ce15c568 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fcc575bc060ce15c568 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:16,342 - lava_callback - INFO - Starting async job submit for node 69407fe2575bc060ce15c5af 2025-12-17 21:39:16,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:16,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe2575bc060ce15c5af HTTP/1.1" 502 150 2025-12-17 21:39:16,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe2575bc060ce15c5af'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:16,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe2575bc060ce15c5af 2025-12-17 21:39:16,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe2575bc060ce15c5af HTTP/1.1" 502 150 2025-12-17 21:39:16,350 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe2575bc060ce15c5af'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:16,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe1575bc060ce15c5a8 2025-12-17 21:39:16,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe1575bc060ce15c5a8 HTTP/1.1" 502 150 2025-12-17 21:39:16,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe1575bc060ce15c5a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39990 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:39:17,171 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d51575bc060ce15f811 2025-12-17 21:39:17,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d51575bc060ce15f811 HTTP/1.1" 502 150 2025-12-17 21:39:17,172 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d51575bc060ce15f811'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:17,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fcf575bc060ce15c571 2025-12-17 21:39:17,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fcf575bc060ce15c571 HTTP/1.1" 502 150 2025-12-17 21:39:17,619 - lava_callback - ERROR - Error processing callback for node 69407fcf575bc060ce15c571: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fcf575bc060ce15c571 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fcf575bc060ce15c571 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fcf575bc060ce15c571 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:17,621 - lava_callback - INFO - Starting async job submit for node 69407fe4575bc060ce15c5d3 2025-12-17 21:39:17,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:17,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe4575bc060ce15c5d3 HTTP/1.1" 502 150 2025-12-17 21:39:17,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe4575bc060ce15c5d3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:17,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe4575bc060ce15c5d3 2025-12-17 21:39:17,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe4575bc060ce15c5d3 HTTP/1.1" 502 150 2025-12-17 21:39:17,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe4575bc060ce15c5d3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:17,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd5575bc060ce15c590 2025-12-17 21:39:17,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd5575bc060ce15c590 HTTP/1.1" 502 150 2025-12-17 21:39:17,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd5575bc060ce15c590'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:17,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085fd575bc060ce15de96 2025-12-17 21:39:17,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085fd575bc060ce15de96 HTTP/1.1" 502 150 2025-12-17 21:39:17,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085fd575bc060ce15de96'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:17,842 - lava_callback - INFO - Received callback for node 69407ff8575bc060ce15c601 INFO: 172.21.0.1:39996 - "POST /node/69407ff8575bc060ce15c601 HTTP/1.0" 202 Accepted 2025-12-17 21:39:18,350 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe2575bc060ce15c5af 2025-12-17 21:39:18,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe2575bc060ce15c5af HTTP/1.1" 502 150 2025-12-17 21:39:18,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe2575bc060ce15c5af'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:18,405 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fda575bc060ce15c5a0 2025-12-17 21:39:18,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fda575bc060ce15c5a0 HTTP/1.1" 502 150 2025-12-17 21:39:18,406 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fda575bc060ce15c5a0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:18,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd0575bc060ce15c578 2025-12-17 21:39:18,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd0575bc060ce15c578 HTTP/1.1" 502 150 2025-12-17 21:39:18,525 - lava_callback - ERROR - Error processing callback for node 69407fd0575bc060ce15c578: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd0575bc060ce15c578 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd0575bc060ce15c578 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd0575bc060ce15c578 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:18,527 - lava_callback - INFO - Starting async job submit for node 69407fe6575bc060ce15c5e6 2025-12-17 21:39:18,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:18,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe6575bc060ce15c5e6 HTTP/1.1" 502 150 2025-12-17 21:39:18,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe6575bc060ce15c5e6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:18,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe6575bc060ce15c5e6 2025-12-17 21:39:18,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe6575bc060ce15c5e6 HTTP/1.1" 502 150 2025-12-17 21:39:18,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe6575bc060ce15c5e6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40002 - "GET /node/69402598575bc060ce154107%5Cn HTTP/1.0" 405 Method Not Allowed 2025-12-17 21:39:19,069 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd6575bc060ce15c593 2025-12-17 21:39:19,071 - lava_callback - INFO - Received callback for node 69407ff9575bc060ce15c604 INFO: 172.21.0.1:40004 - "POST /node/69407ff9575bc060ce15c604 HTTP/1.0" 202 Accepted 2025-12-17 21:39:19,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe4575bc060ce15c5d3 2025-12-17 21:39:19,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe4575bc060ce15c5d3 HTTP/1.1" 502 150 2025-12-17 21:39:19,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe4575bc060ce15c5d3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:19,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd8575bc060ce15c59a 2025-12-17 21:39:19,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd8575bc060ce15c59a HTTP/1.1" 502 150 2025-12-17 21:39:19,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd8575bc060ce15c59a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:19,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd1575bc060ce15c57f 2025-12-17 21:39:19,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd1575bc060ce15c57f HTTP/1.1" 502 150 2025-12-17 21:39:19,765 - lava_callback - ERROR - Error processing callback for node 69407fd1575bc060ce15c57f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd1575bc060ce15c57f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd1575bc060ce15c57f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd1575bc060ce15c57f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:19,767 - lava_callback - INFO - Starting async job submit for node 69407fe7575bc060ce15c5e9 2025-12-17 21:39:19,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:19,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fdb575bc060ce15c5a3 2025-12-17 21:39:19,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fdb575bc060ce15c5a3 HTTP/1.1" 502 150 2025-12-17 21:39:19,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fdb575bc060ce15c5a3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:19,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe7575bc060ce15c5e9 HTTP/1.1" 502 150 2025-12-17 21:39:19,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe7575bc060ce15c5e9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:19,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe7575bc060ce15c5e9 2025-12-17 21:39:19,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe7575bc060ce15c5e9 HTTP/1.1" 502 150 2025-12-17 21:39:19,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe7575bc060ce15c5e9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:19,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd4575bc060ce15c58d HTTP/1.1" 502 150 2025-12-17 21:39:19,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd4575bc060ce15c58d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:20,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe1575bc060ce15c5a8 2025-12-17 21:39:20,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe6575bc060ce15c5e6 2025-12-17 21:39:20,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe6575bc060ce15c5e6 HTTP/1.1" 502 150 2025-12-17 21:39:20,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe6575bc060ce15c5e6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:21,172 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d51575bc060ce15f811 2025-12-17 21:39:21,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d51575bc060ce15f811 HTTP/1.1" 502 150 2025-12-17 21:39:21,173 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d51575bc060ce15f811'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:21,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fc0575bc060ce15c557 2025-12-17 21:39:21,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fc0575bc060ce15c557 HTTP/1.1" 502 150 2025-12-17 21:39:21,748 - lava_callback - ERROR - Error processing callback for node 69407fc0575bc060ce15c557: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fc0575bc060ce15c557 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fc0575bc060ce15c557 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fc0575bc060ce15c557 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:21,751 - lava_callback - INFO - Starting async job submit for node 69407fe8575bc060ce15c5ec 2025-12-17 21:39:21,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:21,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe8575bc060ce15c5ec HTTP/1.1" 502 150 2025-12-17 21:39:21,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe8575bc060ce15c5ec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:21,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe8575bc060ce15c5ec 2025-12-17 21:39:21,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe8575bc060ce15c5ec HTTP/1.1" 502 150 2025-12-17 21:39:21,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe8575bc060ce15c5ec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:21,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe7575bc060ce15c5e9 2025-12-17 21:39:21,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe7575bc060ce15c5e9 HTTP/1.1" 502 150 2025-12-17 21:39:21,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe7575bc060ce15c5e9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:21,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085fd575bc060ce15de96 2025-12-17 21:39:21,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085fd575bc060ce15de96 HTTP/1.1" 502 150 2025-12-17 21:39:21,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085fd575bc060ce15de96'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:22,351 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd9575bc060ce15c59d 2025-12-17 21:39:22,351 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe2575bc060ce15c5af 2025-12-17 21:39:22,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd9575bc060ce15c59d HTTP/1.1" 502 150 2025-12-17 21:39:22,352 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd9575bc060ce15c59d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:22,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe2575bc060ce15c5af HTTP/1.1" 502 150 2025-12-17 21:39:22,353 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe2575bc060ce15c5af'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:22,366 - lava_callback - INFO - Received callback for node 69407ff5575bc060ce15c5f4 INFO: 172.21.0.1:60914 - "POST /node/69407ff5575bc060ce15c5f4 HTTP/1.0" 202 Accepted 2025-12-17 21:39:22,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd3575bc060ce15c586 2025-12-17 21:39:22,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd3575bc060ce15c586 HTTP/1.1" 502 150 2025-12-17 21:39:22,428 - lava_callback - ERROR - Error processing callback for node 69407fd3575bc060ce15c586: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd3575bc060ce15c586 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd3575bc060ce15c586 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd3575bc060ce15c586 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:22,430 - lava_callback - INFO - Starting async job submit for node 69407fe9575bc060ce15c5ef 2025-12-17 21:39:22,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:22,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe9575bc060ce15c5ef HTTP/1.1" 502 150 2025-12-17 21:39:22,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe9575bc060ce15c5ef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:22,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe9575bc060ce15c5ef 2025-12-17 21:39:22,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe9575bc060ce15c5ef HTTP/1.1" 502 150 2025-12-17 21:39:22,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe9575bc060ce15c5ef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:23,017 - lava_callback - INFO - Received callback for node 69407ff6575bc060ce15c5fe INFO: 172.21.0.1:60926 - "POST /node/69407ff6575bc060ce15c5fe HTTP/1.0" 202 Accepted 2025-12-17 21:39:23,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe4575bc060ce15c5d3 2025-12-17 21:39:23,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe4575bc060ce15c5d3 HTTP/1.1" 502 150 2025-12-17 21:39:23,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe4575bc060ce15c5d3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:23,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe8575bc060ce15c5ec 2025-12-17 21:39:23,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe8575bc060ce15c5ec HTTP/1.1" 502 150 2025-12-17 21:39:23,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe8575bc060ce15c5ec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:24,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe9575bc060ce15c5ef 2025-12-17 21:39:24,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe9575bc060ce15c5ef HTTP/1.1" 502 150 2025-12-17 21:39:24,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe9575bc060ce15c5ef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:24,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe6575bc060ce15c5e6 2025-12-17 21:39:24,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe6575bc060ce15c5e6 HTTP/1.1" 502 150 2025-12-17 21:39:24,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe6575bc060ce15c5e6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:25,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe7575bc060ce15c5e9 2025-12-17 21:39:25,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe7575bc060ce15c5e9 HTTP/1.1" 502 150 2025-12-17 21:39:25,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe7575bc060ce15c5e9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:26,406 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fda575bc060ce15c5a0 2025-12-17 21:39:26,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fda575bc060ce15c5a0 HTTP/1.1" 502 150 2025-12-17 21:39:26,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fda575bc060ce15c5a0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60930 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:39:27,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe8575bc060ce15c5ec 2025-12-17 21:39:27,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe8575bc060ce15c5ec HTTP/1.1" 502 150 2025-12-17 21:39:27,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe8575bc060ce15c5ec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:27,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fdb575bc060ce15c5a3 2025-12-17 21:39:27,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fdb575bc060ce15c5a3 HTTP/1.1" 502 150 2025-12-17 21:39:27,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fdb575bc060ce15c5a3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:27,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd4575bc060ce15c58d 2025-12-17 21:39:27,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd4575bc060ce15c58d HTTP/1.1" 502 150 2025-12-17 21:39:27,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd4575bc060ce15c58d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:28,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe9575bc060ce15c5ef 2025-12-17 21:39:28,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe9575bc060ce15c5ef HTTP/1.1" 502 150 2025-12-17 21:39:28,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe9575bc060ce15c5ef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:29,174 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d51575bc060ce15f811 2025-12-17 21:39:29,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d51575bc060ce15f811 HTTP/1.1" 502 150 2025-12-17 21:39:29,175 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d51575bc060ce15f811'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:29,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085fd575bc060ce15de96 2025-12-17 21:39:29,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085fd575bc060ce15de96 HTTP/1.1" 502 150 2025-12-17 21:39:29,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085fd575bc060ce15de96'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:30,353 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe2575bc060ce15c5af 2025-12-17 21:39:30,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe2575bc060ce15c5af HTTP/1.1" 502 150 2025-12-17 21:39:30,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe2575bc060ce15c5af'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:31,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe4575bc060ce15c5d3 2025-12-17 21:39:31,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe4575bc060ce15c5d3 HTTP/1.1" 502 150 2025-12-17 21:39:31,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe4575bc060ce15c5d3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:32,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe6575bc060ce15c5e6 2025-12-17 21:39:32,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe6575bc060ce15c5e6 HTTP/1.1" 502 150 2025-12-17 21:39:32,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe6575bc060ce15c5e6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:32,917 - lava_callback - INFO - Received callback for node 6940800c575bc060ce15c668 INFO: 172.21.0.1:37460 - "POST /node/6940800c575bc060ce15c668 HTTP/1.0" 202 Accepted 2025-12-17 21:39:33,579 - lava_callback - INFO - Received callback for node 6940800e575bc060ce15c671 INFO: 172.21.0.1:37474 - "POST /node/6940800e575bc060ce15c671 HTTP/1.0" 202 Accepted 2025-12-17 21:39:33,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd5575bc060ce15c590 2025-12-17 21:39:33,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd5575bc060ce15c590 HTTP/1.1" 502 150 2025-12-17 21:39:33,751 - lava_callback - ERROR - Error processing callback for node 69407fd5575bc060ce15c590: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd5575bc060ce15c590 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd5575bc060ce15c590 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd5575bc060ce15c590 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:33,753 - lava_callback - INFO - Starting async job submit for node 694086f5575bc060ce15e223 2025-12-17 21:39:33,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:33,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f5575bc060ce15e223 HTTP/1.1" 502 150 2025-12-17 21:39:33,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086f5575bc060ce15e223'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:33,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086f5575bc060ce15e223 2025-12-17 21:39:33,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f5575bc060ce15e223 HTTP/1.1" 502 150 2025-12-17 21:39:33,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086f5575bc060ce15e223'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:33,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe7575bc060ce15c5e9 2025-12-17 21:39:33,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe7575bc060ce15c5e9 HTTP/1.1" 502 150 2025-12-17 21:39:33,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe7575bc060ce15c5e9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:34,247 - lava_callback - INFO - Received callback for node 69408010575bc060ce15c678 INFO: 172.21.0.1:37476 - "POST /node/69408010575bc060ce15c678 HTTP/1.0" 202 Accepted 2025-12-17 21:39:34,422 - lava_callback - INFO - Received callback for node 69408e78575bc060ce15fe31 INFO: 172.21.0.1:37492 - "POST /node/69408e78575bc060ce15fe31 HTTP/1.0" 202 Accepted 2025-12-17 21:39:34,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd6575bc060ce15c593 HTTP/1.1" 502 150 2025-12-17 21:39:34,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fd6575bc060ce15c593'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:34,914 - lava_callback - INFO - Received callback for node 69408011575bc060ce15c67b INFO: 172.21.0.1:37496 - "POST /node/69408011575bc060ce15c67b HTTP/1.0" 202 Accepted 2025-12-17 21:39:35,092 - lava_callback - INFO - Received callback for node 69408dec575bc060ce15fb0a INFO: 172.21.0.1:37506 - "POST /node/69408dec575bc060ce15fb0a HTTP/1.0" 202 Accepted 2025-12-17 21:39:35,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe1575bc060ce15c5a8 HTTP/1.1" 502 150 2025-12-17 21:39:35,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe1575bc060ce15c5a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:35,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd8575bc060ce15c59a 2025-12-17 21:39:35,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd8575bc060ce15c59a HTTP/1.1" 502 150 2025-12-17 21:39:35,758 - lava_callback - ERROR - Error processing callback for node 69407fd8575bc060ce15c59a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd8575bc060ce15c59a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd8575bc060ce15c59a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd8575bc060ce15c59a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:35,760 - lava_callback - INFO - Starting async job submit for node 69407ff8575bc060ce15c601 2025-12-17 21:39:35,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:35,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe8575bc060ce15c5ec 2025-12-17 21:39:35,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe8575bc060ce15c5ec HTTP/1.1" 502 150 2025-12-17 21:39:35,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe8575bc060ce15c5ec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:35,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff8575bc060ce15c601 HTTP/1.1" 502 150 2025-12-17 21:39:35,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff8575bc060ce15c601'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:35,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff8575bc060ce15c601 2025-12-17 21:39:35,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff8575bc060ce15c601 HTTP/1.1" 502 150 2025-12-17 21:39:35,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff8575bc060ce15c601'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:35,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086f5575bc060ce15e223 2025-12-17 21:39:35,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f5575bc060ce15e223 HTTP/1.1" 502 150 2025-12-17 21:39:35,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086f5575bc060ce15e223'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:36,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe9575bc060ce15c5ef 2025-12-17 21:39:36,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe9575bc060ce15c5ef HTTP/1.1" 502 150 2025-12-17 21:39:36,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe9575bc060ce15c5ef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37522 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:39:37,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff8575bc060ce15c601 2025-12-17 21:39:37,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff8575bc060ce15c601 HTTP/1.1" 502 150 2025-12-17 21:39:37,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff8575bc060ce15c601'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:38,352 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd9575bc060ce15c59d 2025-12-17 21:39:38,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd9575bc060ce15c59d HTTP/1.1" 502 150 2025-12-17 21:39:38,354 - lava_callback - ERROR - Error processing callback for node 69407fd9575bc060ce15c59d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd9575bc060ce15c59d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd9575bc060ce15c59d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd9575bc060ce15c59d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:38,355 - lava_callback - INFO - Starting async job submit for node 69407ff9575bc060ce15c604 2025-12-17 21:39:38,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:38,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff9575bc060ce15c604 HTTP/1.1" 502 150 2025-12-17 21:39:38,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff9575bc060ce15c604'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:38,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff9575bc060ce15c604 2025-12-17 21:39:38,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff9575bc060ce15c604 HTTP/1.1" 502 150 2025-12-17 21:39:38,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff9575bc060ce15c604'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:39,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086f5575bc060ce15e223 2025-12-17 21:39:39,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f5575bc060ce15e223 HTTP/1.1" 502 150 2025-12-17 21:39:39,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086f5575bc060ce15e223'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:40,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff9575bc060ce15c604 2025-12-17 21:39:40,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff9575bc060ce15c604 HTTP/1.1" 502 150 2025-12-17 21:39:40,367 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff9575bc060ce15c604'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:41,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff8575bc060ce15c601 2025-12-17 21:39:41,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff8575bc060ce15c601 HTTP/1.1" 502 150 2025-12-17 21:39:41,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff8575bc060ce15c601'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:42,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fda575bc060ce15c5a0 2025-12-17 21:39:42,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fda575bc060ce15c5a0 HTTP/1.1" 502 150 2025-12-17 21:39:42,409 - lava_callback - ERROR - Error processing callback for node 69407fda575bc060ce15c5a0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fda575bc060ce15c5a0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fda575bc060ce15c5a0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fda575bc060ce15c5a0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:42,412 - lava_callback - INFO - Starting async job submit for node 69407ff5575bc060ce15c5f4 2025-12-17 21:39:42,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:42,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff5575bc060ce15c5f4 HTTP/1.1" 502 150 2025-12-17 21:39:42,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff5575bc060ce15c5f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:42,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff5575bc060ce15c5f4 2025-12-17 21:39:42,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff5575bc060ce15c5f4 HTTP/1.1" 502 150 2025-12-17 21:39:42,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff5575bc060ce15c5f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:43,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe1575bc060ce15c5a8 2025-12-17 21:39:43,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe1575bc060ce15c5a8 HTTP/1.1" 502 150 2025-12-17 21:39:43,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407fe1575bc060ce15c5a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:43,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fdb575bc060ce15c5a3 2025-12-17 21:39:43,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fdb575bc060ce15c5a3 HTTP/1.1" 502 150 2025-12-17 21:39:43,800 - lava_callback - ERROR - Error processing callback for node 69407fdb575bc060ce15c5a3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fdb575bc060ce15c5a3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fdb575bc060ce15c5a3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fdb575bc060ce15c5a3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:43,802 - lava_callback - INFO - Starting async job submit for node 69407ff6575bc060ce15c5fe 2025-12-17 21:39:43,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:43,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff6575bc060ce15c5fe HTTP/1.1" 502 150 2025-12-17 21:39:43,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff6575bc060ce15c5fe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:43,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff6575bc060ce15c5fe 2025-12-17 21:39:43,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff6575bc060ce15c5fe HTTP/1.1" 502 150 2025-12-17 21:39:43,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff6575bc060ce15c5fe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:43,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd4575bc060ce15c58d 2025-12-17 21:39:43,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd4575bc060ce15c58d HTTP/1.1" 502 150 2025-12-17 21:39:43,858 - lava_callback - ERROR - Error processing callback for node 69407fd4575bc060ce15c58d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd4575bc060ce15c58d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd4575bc060ce15c58d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd4575bc060ce15c58d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:43,860 - lava_callback - INFO - Starting async job submit for node 6940800c575bc060ce15c668 2025-12-17 21:39:43,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:43,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940800c575bc060ce15c668 HTTP/1.1" 502 150 2025-12-17 21:39:43,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940800c575bc060ce15c668'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:43,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940800c575bc060ce15c668 2025-12-17 21:39:43,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940800c575bc060ce15c668 HTTP/1.1" 502 150 2025-12-17 21:39:43,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940800c575bc060ce15c668'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:44,367 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff9575bc060ce15c604 2025-12-17 21:39:44,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff9575bc060ce15c604 HTTP/1.1" 502 150 2025-12-17 21:39:44,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff9575bc060ce15c604'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:44,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff5575bc060ce15c5f4 2025-12-17 21:39:44,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff5575bc060ce15c5f4 HTTP/1.1" 502 150 2025-12-17 21:39:44,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff5575bc060ce15c5f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:45,175 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d51575bc060ce15f811 2025-12-17 21:39:45,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff6575bc060ce15c5fe 2025-12-17 21:39:45,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff6575bc060ce15c5fe HTTP/1.1" 502 150 2025-12-17 21:39:45,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff6575bc060ce15c5fe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:45,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085fd575bc060ce15de96 2025-12-17 21:39:45,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085fd575bc060ce15de96 HTTP/1.1" 502 150 2025-12-17 21:39:45,812 - lava_callback - ERROR - Error processing callback for node 694085fd575bc060ce15de96: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085fd575bc060ce15de96 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085fd575bc060ce15de96 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085fd575bc060ce15de96 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:45,814 - lava_callback - INFO - Starting async job submit for node 6940800e575bc060ce15c671 2025-12-17 21:39:45,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:45,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940800e575bc060ce15c671 HTTP/1.1" 502 150 2025-12-17 21:39:45,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940800e575bc060ce15c671'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:45,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940800e575bc060ce15c671 2025-12-17 21:39:45,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940800e575bc060ce15c671 HTTP/1.1" 502 150 2025-12-17 21:39:45,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940800e575bc060ce15c671'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:45,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940800c575bc060ce15c668 2025-12-17 21:39:45,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940800c575bc060ce15c668 HTTP/1.1" 502 150 2025-12-17 21:39:45,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940800c575bc060ce15c668'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:46,354 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe2575bc060ce15c5af INFO: 172.21.0.1:54864 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:39:47,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe4575bc060ce15c5d3 2025-12-17 21:39:47,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe4575bc060ce15c5d3 HTTP/1.1" 502 150 2025-12-17 21:39:47,667 - lava_callback - ERROR - Error processing callback for node 69407fe4575bc060ce15c5d3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe4575bc060ce15c5d3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe4575bc060ce15c5d3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe4575bc060ce15c5d3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:47,668 - lava_callback - INFO - Starting async job submit for node 69408010575bc060ce15c678 2025-12-17 21:39:47,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:47,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408010575bc060ce15c678 HTTP/1.1" 502 150 2025-12-17 21:39:47,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408010575bc060ce15c678'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:47,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408010575bc060ce15c678 2025-12-17 21:39:47,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408010575bc060ce15c678 HTTP/1.1" 502 150 2025-12-17 21:39:47,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408010575bc060ce15c678'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:47,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086f5575bc060ce15e223 2025-12-17 21:39:47,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f5575bc060ce15e223 HTTP/1.1" 502 150 2025-12-17 21:39:47,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086f5575bc060ce15e223'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:47,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940800e575bc060ce15c671 2025-12-17 21:39:47,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940800e575bc060ce15c671 HTTP/1.1" 502 150 2025-12-17 21:39:47,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940800e575bc060ce15c671'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:48,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff5575bc060ce15c5f4 2025-12-17 21:39:48,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff5575bc060ce15c5f4 HTTP/1.1" 502 150 2025-12-17 21:39:48,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff5575bc060ce15c5f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:48,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe6575bc060ce15c5e6 2025-12-17 21:39:48,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe6575bc060ce15c5e6 HTTP/1.1" 502 150 2025-12-17 21:39:48,541 - lava_callback - ERROR - Error processing callback for node 69407fe6575bc060ce15c5e6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe6575bc060ce15c5e6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe6575bc060ce15c5e6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe6575bc060ce15c5e6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:48,543 - lava_callback - INFO - Starting async job submit for node 69408e78575bc060ce15fe31 2025-12-17 21:39:48,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:48,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e78575bc060ce15fe31 HTTP/1.1" 502 150 2025-12-17 21:39:48,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e78575bc060ce15fe31'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:48,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e78575bc060ce15fe31 2025-12-17 21:39:48,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e78575bc060ce15fe31 HTTP/1.1" 502 150 2025-12-17 21:39:48,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e78575bc060ce15fe31'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:49,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408010575bc060ce15c678 2025-12-17 21:39:49,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408010575bc060ce15c678 HTTP/1.1" 502 150 2025-12-17 21:39:49,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408010575bc060ce15c678'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:49,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff8575bc060ce15c601 2025-12-17 21:39:49,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff8575bc060ce15c601 HTTP/1.1" 502 150 2025-12-17 21:39:49,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff8575bc060ce15c601'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:49,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe7575bc060ce15c5e9 2025-12-17 21:39:49,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe7575bc060ce15c5e9 HTTP/1.1" 502 150 2025-12-17 21:39:49,804 - lava_callback - ERROR - Error processing callback for node 69407fe7575bc060ce15c5e9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe7575bc060ce15c5e9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe7575bc060ce15c5e9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe7575bc060ce15c5e9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:49,806 - lava_callback - INFO - Starting async job submit for node 69408011575bc060ce15c67b 2025-12-17 21:39:49,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:49,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff6575bc060ce15c5fe 2025-12-17 21:39:49,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff6575bc060ce15c5fe HTTP/1.1" 502 150 2025-12-17 21:39:49,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff6575bc060ce15c5fe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:49,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408011575bc060ce15c67b HTTP/1.1" 502 150 2025-12-17 21:39:49,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408011575bc060ce15c67b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:49,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408011575bc060ce15c67b 2025-12-17 21:39:49,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408011575bc060ce15c67b HTTP/1.1" 502 150 2025-12-17 21:39:49,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408011575bc060ce15c67b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:49,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940800c575bc060ce15c668 2025-12-17 21:39:49,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940800c575bc060ce15c668 HTTP/1.1" 502 150 2025-12-17 21:39:49,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940800c575bc060ce15c668'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:50,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fd6575bc060ce15c593 2025-12-17 21:39:50,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fd6575bc060ce15c593 HTTP/1.1" 502 150 2025-12-17 21:39:50,454 - lava_callback - ERROR - Error processing callback for node 69407fd6575bc060ce15c593: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd6575bc060ce15c593 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd6575bc060ce15c593 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fd6575bc060ce15c593 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:50,455 - lava_callback - INFO - Starting async job submit for node 69408dec575bc060ce15fb0a 2025-12-17 21:39:50,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:50,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dec575bc060ce15fb0a HTTP/1.1" 502 150 2025-12-17 21:39:50,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dec575bc060ce15fb0a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:50,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dec575bc060ce15fb0a 2025-12-17 21:39:50,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dec575bc060ce15fb0a HTTP/1.1" 502 150 2025-12-17 21:39:50,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dec575bc060ce15fb0a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:50,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e78575bc060ce15fe31 2025-12-17 21:39:50,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e78575bc060ce15fe31 HTTP/1.1" 502 150 2025-12-17 21:39:50,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e78575bc060ce15fe31'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:51,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe8575bc060ce15c5ec 2025-12-17 21:39:51,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe8575bc060ce15c5ec HTTP/1.1" 502 150 2025-12-17 21:39:51,774 - lava_callback - ERROR - Error processing callback for node 69407fe8575bc060ce15c5ec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe8575bc060ce15c5ec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe8575bc060ce15c5ec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe8575bc060ce15c5ec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:51,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408011575bc060ce15c67b 2025-12-17 21:39:51,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408011575bc060ce15c67b HTTP/1.1" 502 150 2025-12-17 21:39:51,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408011575bc060ce15c67b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:51,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940800e575bc060ce15c671 2025-12-17 21:39:51,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940800e575bc060ce15c671 HTTP/1.1" 502 150 2025-12-17 21:39:51,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940800e575bc060ce15c671'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:52,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff9575bc060ce15c604 2025-12-17 21:39:52,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff9575bc060ce15c604 HTTP/1.1" 502 150 2025-12-17 21:39:52,370 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff9575bc060ce15c604'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:52,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe9575bc060ce15c5ef 2025-12-17 21:39:52,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe9575bc060ce15c5ef HTTP/1.1" 502 150 2025-12-17 21:39:52,442 - lava_callback - ERROR - Error processing callback for node 69407fe9575bc060ce15c5ef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe9575bc060ce15c5ef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe9575bc060ce15c5ef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe9575bc060ce15c5ef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:52,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dec575bc060ce15fb0a 2025-12-17 21:39:52,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dec575bc060ce15fb0a HTTP/1.1" 502 150 2025-12-17 21:39:52,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dec575bc060ce15fb0a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:52,910 - lava_callback - INFO - Received callback for node 6940801e575bc060ce15c684 INFO: 172.21.0.1:50846 - "POST /node/6940801e575bc060ce15c684 HTTP/1.0" 202 Accepted 2025-12-17 21:39:52,917 - lava_callback - INFO - Starting async job submit for node 6940801e575bc060ce15c684 2025-12-17 21:39:52,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:52,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940801e575bc060ce15c684 HTTP/1.1" 502 150 2025-12-17 21:39:52,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940801e575bc060ce15c684'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:52,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940801e575bc060ce15c684 2025-12-17 21:39:52,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940801e575bc060ce15c684 HTTP/1.1" 502 150 2025-12-17 21:39:52,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940801e575bc060ce15c684'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:53,554 - lava_callback - INFO - Received callback for node 69408020575bc060ce15c68a INFO: 172.21.0.1:50858 - "POST /node/69408020575bc060ce15c68a HTTP/1.0" 202 Accepted 2025-12-17 21:39:53,560 - lava_callback - INFO - Starting async job submit for node 69408020575bc060ce15c68a 2025-12-17 21:39:53,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:39:53,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408020575bc060ce15c68a HTTP/1.1" 502 150 2025-12-17 21:39:53,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408020575bc060ce15c68a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:53,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408020575bc060ce15c68a 2025-12-17 21:39:53,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408020575bc060ce15c68a HTTP/1.1" 502 150 2025-12-17 21:39:53,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408020575bc060ce15c68a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:53,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408010575bc060ce15c678 2025-12-17 21:39:53,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408010575bc060ce15c678 HTTP/1.1" 502 150 2025-12-17 21:39:53,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408010575bc060ce15c678'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:54,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e78575bc060ce15fe31 2025-12-17 21:39:54,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e78575bc060ce15fe31 HTTP/1.1" 502 150 2025-12-17 21:39:54,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e78575bc060ce15fe31'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:54,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940801e575bc060ce15c684 2025-12-17 21:39:54,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940801e575bc060ce15c684 HTTP/1.1" 502 150 2025-12-17 21:39:54,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940801e575bc060ce15c684'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:55,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408020575bc060ce15c68a 2025-12-17 21:39:55,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408020575bc060ce15c68a HTTP/1.1" 502 150 2025-12-17 21:39:55,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408020575bc060ce15c68a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:55,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408011575bc060ce15c67b 2025-12-17 21:39:55,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408011575bc060ce15c67b HTTP/1.1" 502 150 2025-12-17 21:39:55,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408011575bc060ce15c67b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:56,427 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff5575bc060ce15c5f4 2025-12-17 21:39:56,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff5575bc060ce15c5f4 HTTP/1.1" 502 150 2025-12-17 21:39:56,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff5575bc060ce15c5f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:56,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dec575bc060ce15fb0a 2025-12-17 21:39:56,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dec575bc060ce15fb0a HTTP/1.1" 502 150 2025-12-17 21:39:56,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dec575bc060ce15fb0a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50868 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:39:57,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff6575bc060ce15c5fe 2025-12-17 21:39:57,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff6575bc060ce15c5fe HTTP/1.1" 502 150 2025-12-17 21:39:57,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407ff6575bc060ce15c5fe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:57,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940800c575bc060ce15c668 2025-12-17 21:39:57,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940800c575bc060ce15c668 HTTP/1.1" 502 150 2025-12-17 21:39:57,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940800c575bc060ce15c668'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:58,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940801e575bc060ce15c684 2025-12-17 21:39:58,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940801e575bc060ce15c684 HTTP/1.1" 502 150 2025-12-17 21:39:58,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940801e575bc060ce15c684'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:59,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408020575bc060ce15c68a 2025-12-17 21:39:59,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408020575bc060ce15c68a HTTP/1.1" 502 150 2025-12-17 21:39:59,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408020575bc060ce15c68a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:39:59,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407fe1575bc060ce15c5a8 2025-12-17 21:39:59,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe1575bc060ce15c5a8 HTTP/1.1" 502 150 2025-12-17 21:39:59,731 - lava_callback - ERROR - Error processing callback for node 69407fe1575bc060ce15c5a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe1575bc060ce15c5a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe1575bc060ce15c5a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe1575bc060ce15c5a8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:39:59,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940800e575bc060ce15c671 2025-12-17 21:39:59,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940800e575bc060ce15c671 HTTP/1.1" 502 150 2025-12-17 21:39:59,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940800e575bc060ce15c671'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:00,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d51575bc060ce15f811 HTTP/1.1" 502 150 2025-12-17 21:40:00,560 - lava_callback - ERROR - Error processing callback for node 69408d51575bc060ce15f811: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d51575bc060ce15f811 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d51575bc060ce15f811 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d51575bc060ce15f811 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:55736 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:40:01,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408010575bc060ce15c678 2025-12-17 21:40:01,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407fe2575bc060ce15c5af HTTP/1.1" 502 150 2025-12-17 21:40:01,844 - lava_callback - ERROR - Error processing callback for node 69407fe2575bc060ce15c5af: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe2575bc060ce15c5af (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe2575bc060ce15c5af (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407fe2575bc060ce15c5af (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:02,392 - lava_callback - INFO - Received callback for node 6940802a575bc060ce15c6df INFO: 172.21.0.1:55738 - "POST /node/6940802a575bc060ce15c6df HTTP/1.0" 202 Accepted 2025-12-17 21:40:02,398 - lava_callback - INFO - Starting async job submit for node 6940802a575bc060ce15c6df 2025-12-17 21:40:02,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:02,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802a575bc060ce15c6df HTTP/1.1" 502 150 2025-12-17 21:40:02,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802a575bc060ce15c6df'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:02,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802a575bc060ce15c6df 2025-12-17 21:40:02,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802a575bc060ce15c6df HTTP/1.1" 502 150 2025-12-17 21:40:02,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802a575bc060ce15c6df'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:02,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e78575bc060ce15fe31 2025-12-17 21:40:02,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e78575bc060ce15fe31 HTTP/1.1" 502 150 2025-12-17 21:40:02,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e78575bc060ce15fe31'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:03,064 - lava_callback - INFO - Received callback for node 6940802c575bc060ce15c6e2 INFO: 172.21.0.1:55754 - "POST /node/6940802c575bc060ce15c6e2 HTTP/1.0" 202 Accepted 2025-12-17 21:40:03,070 - lava_callback - INFO - Starting async job submit for node 6940802c575bc060ce15c6e2 2025-12-17 21:40:03,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:03,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802c575bc060ce15c6e2 HTTP/1.1" 502 150 2025-12-17 21:40:03,097 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802c575bc060ce15c6e2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:03,097 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802c575bc060ce15c6e2 2025-12-17 21:40:03,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802c575bc060ce15c6e2 HTTP/1.1" 502 150 2025-12-17 21:40:03,098 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802c575bc060ce15c6e2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:03,717 - lava_callback - INFO - Received callback for node 6940802d575bc060ce15c6e5 INFO: 172.21.0.1:55760 - "POST /node/6940802d575bc060ce15c6e5 HTTP/1.0" 202 Accepted 2025-12-17 21:40:03,723 - lava_callback - INFO - Starting async job submit for node 6940802d575bc060ce15c6e5 2025-12-17 21:40:03,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:03,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802d575bc060ce15c6e5 HTTP/1.1" 502 150 2025-12-17 21:40:03,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802d575bc060ce15c6e5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:03,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802d575bc060ce15c6e5 2025-12-17 21:40:03,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802d575bc060ce15c6e5 HTTP/1.1" 502 150 2025-12-17 21:40:03,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802d575bc060ce15c6e5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:03,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086f5575bc060ce15e223 2025-12-17 21:40:03,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f5575bc060ce15e223 HTTP/1.1" 502 150 2025-12-17 21:40:03,784 - lava_callback - ERROR - Error processing callback for node 694086f5575bc060ce15e223: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086f5575bc060ce15e223 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086f5575bc060ce15e223 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086f5575bc060ce15e223 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:03,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408011575bc060ce15c67b 2025-12-17 21:40:03,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408011575bc060ce15c67b HTTP/1.1" 502 150 2025-12-17 21:40:03,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408011575bc060ce15c67b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:04,393 - lava_callback - INFO - Received callback for node 6940802e575bc060ce15c6e8 INFO: 172.21.0.1:55764 - "POST /node/6940802e575bc060ce15c6e8 HTTP/1.0" 202 Accepted 2025-12-17 21:40:04,399 - lava_callback - INFO - Starting async job submit for node 6940802e575bc060ce15c6e8 2025-12-17 21:40:04,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:04,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802e575bc060ce15c6e8 HTTP/1.1" 502 150 2025-12-17 21:40:04,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802e575bc060ce15c6e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:04,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802e575bc060ce15c6e8 2025-12-17 21:40:04,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802a575bc060ce15c6df 2025-12-17 21:40:04,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802e575bc060ce15c6e8 HTTP/1.1" 502 150 2025-12-17 21:40:04,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802e575bc060ce15c6e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:04,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802a575bc060ce15c6df HTTP/1.1" 502 150 2025-12-17 21:40:04,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802a575bc060ce15c6df'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:04,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dec575bc060ce15fb0a 2025-12-17 21:40:04,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dec575bc060ce15fb0a HTTP/1.1" 502 150 2025-12-17 21:40:04,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dec575bc060ce15fb0a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:05,081 - lava_callback - INFO - Received callback for node 6940802f575bc060ce15c6eb INFO: 172.21.0.1:55768 - "POST /node/6940802f575bc060ce15c6eb HTTP/1.0" 202 Accepted 2025-12-17 21:40:05,098 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802c575bc060ce15c6e2 2025-12-17 21:40:05,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802c575bc060ce15c6e2 HTTP/1.1" 502 150 2025-12-17 21:40:05,099 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802c575bc060ce15c6e2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:05,746 - lava_callback - INFO - Received callback for node 69408031575bc060ce15c6f0 INFO: 172.21.0.1:55780 - "POST /node/69408031575bc060ce15c6f0 HTTP/1.0" 202 Accepted 2025-12-17 21:40:05,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802d575bc060ce15c6e5 2025-12-17 21:40:05,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802d575bc060ce15c6e5 HTTP/1.1" 502 150 2025-12-17 21:40:05,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802d575bc060ce15c6e5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:05,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff8575bc060ce15c601 2025-12-17 21:40:05,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff8575bc060ce15c601 HTTP/1.1" 502 150 2025-12-17 21:40:05,780 - lava_callback - ERROR - Error processing callback for node 69407ff8575bc060ce15c601: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ff8575bc060ce15c601 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ff8575bc060ce15c601 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ff8575bc060ce15c601 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:05,782 - lava_callback - INFO - Starting async job submit for node 6940802f575bc060ce15c6eb 2025-12-17 21:40:05,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:05,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802f575bc060ce15c6eb HTTP/1.1" 502 150 2025-12-17 21:40:05,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802f575bc060ce15c6eb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:05,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802f575bc060ce15c6eb 2025-12-17 21:40:05,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802f575bc060ce15c6eb HTTP/1.1" 502 150 2025-12-17 21:40:05,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802f575bc060ce15c6eb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:06,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802e575bc060ce15c6e8 2025-12-17 21:40:06,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802e575bc060ce15c6e8 HTTP/1.1" 502 150 2025-12-17 21:40:06,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802e575bc060ce15c6e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55784 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:40:06,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940801e575bc060ce15c684 2025-12-17 21:40:06,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940801e575bc060ce15c684 HTTP/1.1" 502 150 2025-12-17 21:40:06,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940801e575bc060ce15c684'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:07,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408020575bc060ce15c68a 2025-12-17 21:40:07,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408020575bc060ce15c68a HTTP/1.1" 502 150 2025-12-17 21:40:07,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408020575bc060ce15c68a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:07,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802f575bc060ce15c6eb 2025-12-17 21:40:07,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802f575bc060ce15c6eb HTTP/1.1" 502 150 2025-12-17 21:40:07,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802f575bc060ce15c6eb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:08,282 - lava_callback - INFO - Received callback for node 69408027575bc060ce15c6d9 INFO: 172.21.0.1:55798 - "POST /node/69408027575bc060ce15c6d9 HTTP/1.0" 202 Accepted 2025-12-17 21:40:08,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff9575bc060ce15c604 2025-12-17 21:40:08,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff9575bc060ce15c604 HTTP/1.1" 502 150 2025-12-17 21:40:08,371 - lava_callback - ERROR - Error processing callback for node 69407ff9575bc060ce15c604: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ff9575bc060ce15c604 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ff9575bc060ce15c604 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ff9575bc060ce15c604 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:08,373 - lava_callback - INFO - Starting async job submit for node 69408031575bc060ce15c6f0 2025-12-17 21:40:08,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:08,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408031575bc060ce15c6f0 HTTP/1.1" 502 150 2025-12-17 21:40:08,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408031575bc060ce15c6f0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:08,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408031575bc060ce15c6f0 2025-12-17 21:40:08,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408031575bc060ce15c6f0 HTTP/1.1" 502 150 2025-12-17 21:40:08,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408031575bc060ce15c6f0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:08,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802a575bc060ce15c6df 2025-12-17 21:40:08,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802a575bc060ce15c6df HTTP/1.1" 502 150 2025-12-17 21:40:08,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802a575bc060ce15c6df'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:09,010 - lava_callback - INFO - Received callback for node 69408029575bc060ce15c6dc INFO: 172.21.0.1:55814 - "POST /node/69408029575bc060ce15c6dc HTTP/1.0" 202 Accepted 2025-12-17 21:40:09,100 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802c575bc060ce15c6e2 2025-12-17 21:40:09,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802c575bc060ce15c6e2 HTTP/1.1" 502 150 2025-12-17 21:40:09,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802c575bc060ce15c6e2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:09,665 - lava_callback - INFO - Received callback for node 69408032575bc060ce15c6f7 INFO: 172.21.0.1:55822 - "POST /node/69408032575bc060ce15c6f7 HTTP/1.0" 202 Accepted 2025-12-17 21:40:09,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802d575bc060ce15c6e5 2025-12-17 21:40:09,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802d575bc060ce15c6e5 HTTP/1.1" 502 150 2025-12-17 21:40:09,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802d575bc060ce15c6e5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:10,338 - lava_callback - INFO - Received callback for node 69408033575bc060ce15c701 INFO: 172.21.0.1:48512 - "POST /node/69408033575bc060ce15c701 HTTP/1.0" 202 Accepted 2025-12-17 21:40:10,381 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408031575bc060ce15c6f0 2025-12-17 21:40:10,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408031575bc060ce15c6f0 HTTP/1.1" 502 150 2025-12-17 21:40:10,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408031575bc060ce15c6f0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:10,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802e575bc060ce15c6e8 2025-12-17 21:40:10,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802e575bc060ce15c6e8 HTTP/1.1" 502 150 2025-12-17 21:40:10,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802e575bc060ce15c6e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:11,045 - lava_callback - INFO - Received callback for node 69408037575bc060ce15c714 INFO: 172.21.0.1:48520 - "POST /node/69408037575bc060ce15c714 HTTP/1.0" 202 Accepted 2025-12-17 21:40:11,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802f575bc060ce15c6eb 2025-12-17 21:40:12,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff5575bc060ce15c5f4 2025-12-17 21:40:12,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff5575bc060ce15c5f4 HTTP/1.1" 502 150 2025-12-17 21:40:12,430 - lava_callback - ERROR - Error processing callback for node 69407ff5575bc060ce15c5f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ff5575bc060ce15c5f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ff5575bc060ce15c5f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ff5575bc060ce15c5f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:12,432 - lava_callback - INFO - Starting async job submit for node 69408027575bc060ce15c6d9 2025-12-17 21:40:12,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:12,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408027575bc060ce15c6d9 HTTP/1.1" 502 150 2025-12-17 21:40:12,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408027575bc060ce15c6d9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:12,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408027575bc060ce15c6d9 2025-12-17 21:40:12,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408027575bc060ce15c6d9 HTTP/1.1" 502 150 2025-12-17 21:40:12,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408027575bc060ce15c6d9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:13,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407ff6575bc060ce15c5fe 2025-12-17 21:40:13,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407ff6575bc060ce15c5fe HTTP/1.1" 502 150 2025-12-17 21:40:13,815 - lava_callback - ERROR - Error processing callback for node 69407ff6575bc060ce15c5fe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ff6575bc060ce15c5fe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ff6575bc060ce15c5fe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407ff6575bc060ce15c5fe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:13,817 - lava_callback - INFO - Starting async job submit for node 69408029575bc060ce15c6dc 2025-12-17 21:40:13,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:13,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408029575bc060ce15c6dc HTTP/1.1" 502 150 2025-12-17 21:40:13,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408029575bc060ce15c6dc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:13,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408029575bc060ce15c6dc 2025-12-17 21:40:13,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408029575bc060ce15c6dc HTTP/1.1" 502 150 2025-12-17 21:40:13,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408029575bc060ce15c6dc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:13,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940800c575bc060ce15c668 2025-12-17 21:40:13,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940800c575bc060ce15c668 HTTP/1.1" 502 150 2025-12-17 21:40:13,876 - lava_callback - ERROR - Error processing callback for node 6940800c575bc060ce15c668: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940800c575bc060ce15c668 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940800c575bc060ce15c668 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940800c575bc060ce15c668 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:13,878 - lava_callback - INFO - Starting async job submit for node 69408032575bc060ce15c6f7 2025-12-17 21:40:13,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:13,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408032575bc060ce15c6f7 HTTP/1.1" 502 150 2025-12-17 21:40:13,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408032575bc060ce15c6f7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:13,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408032575bc060ce15c6f7 2025-12-17 21:40:13,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408032575bc060ce15c6f7 HTTP/1.1" 502 150 2025-12-17 21:40:13,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408032575bc060ce15c6f7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:14,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408031575bc060ce15c6f0 2025-12-17 21:40:14,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408031575bc060ce15c6f0 HTTP/1.1" 502 150 2025-12-17 21:40:14,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408031575bc060ce15c6f0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:14,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408027575bc060ce15c6d9 2025-12-17 21:40:14,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408027575bc060ce15c6d9 HTTP/1.1" 502 150 2025-12-17 21:40:14,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408027575bc060ce15c6d9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48530 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:40:15,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408029575bc060ce15c6dc 2025-12-17 21:40:15,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408029575bc060ce15c6dc HTTP/1.1" 502 150 2025-12-17 21:40:15,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408029575bc060ce15c6dc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:15,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940800e575bc060ce15c671 2025-12-17 21:40:15,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940800e575bc060ce15c671 HTTP/1.1" 502 150 2025-12-17 21:40:15,855 - lava_callback - ERROR - Error processing callback for node 6940800e575bc060ce15c671: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940800e575bc060ce15c671 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940800e575bc060ce15c671 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940800e575bc060ce15c671 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:15,857 - lava_callback - INFO - Starting async job submit for node 69408033575bc060ce15c701 2025-12-17 21:40:15,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:15,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408033575bc060ce15c701 HTTP/1.1" 502 150 2025-12-17 21:40:15,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408033575bc060ce15c701'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:15,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408033575bc060ce15c701 2025-12-17 21:40:15,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408033575bc060ce15c701 HTTP/1.1" 502 150 2025-12-17 21:40:15,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408033575bc060ce15c701'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:15,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408032575bc060ce15c6f7 2025-12-17 21:40:15,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408032575bc060ce15c6f7 HTTP/1.1" 502 150 2025-12-17 21:40:15,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408032575bc060ce15c6f7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:16,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802a575bc060ce15c6df 2025-12-17 21:40:16,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802a575bc060ce15c6df HTTP/1.1" 502 150 2025-12-17 21:40:16,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802a575bc060ce15c6df'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48532 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:40:16,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408010575bc060ce15c678 HTTP/1.1" 502 150 2025-12-17 21:40:16,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408010575bc060ce15c678'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:17,101 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802c575bc060ce15c6e2 2025-12-17 21:40:17,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802c575bc060ce15c6e2 HTTP/1.1" 502 150 2025-12-17 21:40:17,102 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802c575bc060ce15c6e2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:17,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802d575bc060ce15c6e5 2025-12-17 21:40:17,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802d575bc060ce15c6e5 HTTP/1.1" 502 150 2025-12-17 21:40:17,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802d575bc060ce15c6e5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:17,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408033575bc060ce15c701 2025-12-17 21:40:17,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408033575bc060ce15c701 HTTP/1.1" 502 150 2025-12-17 21:40:17,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408033575bc060ce15c701'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:18,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802e575bc060ce15c6e8 2025-12-17 21:40:18,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802e575bc060ce15c6e8 HTTP/1.1" 502 150 2025-12-17 21:40:18,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802e575bc060ce15c6e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:18,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408027575bc060ce15c6d9 2025-12-17 21:40:18,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408027575bc060ce15c6d9 HTTP/1.1" 502 150 2025-12-17 21:40:18,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408027575bc060ce15c6d9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:18,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e78575bc060ce15fe31 2025-12-17 21:40:18,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e78575bc060ce15fe31 HTTP/1.1" 502 150 2025-12-17 21:40:18,621 - lava_callback - ERROR - Error processing callback for node 69408e78575bc060ce15fe31: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e78575bc060ce15fe31 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e78575bc060ce15fe31 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e78575bc060ce15fe31 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:18,623 - lava_callback - INFO - Starting async job submit for node 69408037575bc060ce15c714 2025-12-17 21:40:18,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:18,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408037575bc060ce15c714 HTTP/1.1" 502 150 2025-12-17 21:40:18,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408037575bc060ce15c714'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:18,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408037575bc060ce15c714 2025-12-17 21:40:18,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408037575bc060ce15c714 HTTP/1.1" 502 150 2025-12-17 21:40:18,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408037575bc060ce15c714'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:19,147 - lava_callback - INFO - Received callback for node 694086f8575bc060ce15e226 INFO: 172.21.0.1:48542 - "POST /node/694086f8575bc060ce15e226 HTTP/1.0" 202 Accepted 2025-12-17 21:40:19,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408011575bc060ce15c67b 2025-12-17 21:40:19,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408011575bc060ce15c67b HTTP/1.1" 502 150 2025-12-17 21:40:19,819 - lava_callback - ERROR - Error processing callback for node 69408011575bc060ce15c67b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408011575bc060ce15c67b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408011575bc060ce15c67b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408011575bc060ce15c67b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:19,821 - lava_callback - INFO - Starting async job submit for node 694086f8575bc060ce15e226 2025-12-17 21:40:19,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:19,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408029575bc060ce15c6dc 2025-12-17 21:40:19,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408029575bc060ce15c6dc HTTP/1.1" 502 150 2025-12-17 21:40:19,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408029575bc060ce15c6dc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:19,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f8575bc060ce15e226 HTTP/1.1" 502 150 2025-12-17 21:40:19,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086f8575bc060ce15e226'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:19,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086f8575bc060ce15e226 2025-12-17 21:40:19,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f8575bc060ce15e226 HTTP/1.1" 502 150 2025-12-17 21:40:19,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086f8575bc060ce15e226'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:19,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408032575bc060ce15c6f7 2025-12-17 21:40:19,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408032575bc060ce15c6f7 HTTP/1.1" 502 150 2025-12-17 21:40:19,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408032575bc060ce15c6f7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:20,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dec575bc060ce15fb0a 2025-12-17 21:40:20,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dec575bc060ce15fb0a HTTP/1.1" 502 150 2025-12-17 21:40:20,532 - lava_callback - ERROR - Error processing callback for node 69408dec575bc060ce15fb0a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dec575bc060ce15fb0a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dec575bc060ce15fb0a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dec575bc060ce15fb0a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:20,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408037575bc060ce15c714 2025-12-17 21:40:20,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408037575bc060ce15c714 HTTP/1.1" 502 150 2025-12-17 21:40:20,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408037575bc060ce15c714'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:21,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408033575bc060ce15c701 2025-12-17 21:40:21,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408033575bc060ce15c701 HTTP/1.1" 502 150 2025-12-17 21:40:21,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408033575bc060ce15c701'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:21,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086f8575bc060ce15e226 2025-12-17 21:40:21,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f8575bc060ce15e226 HTTP/1.1" 502 150 2025-12-17 21:40:21,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086f8575bc060ce15e226'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:22,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408031575bc060ce15c6f0 2025-12-17 21:40:22,427 - lava_callback - INFO - Received callback for node 69408035575bc060ce15c70a INFO: 172.21.0.1:37036 - "POST /node/69408035575bc060ce15c70a HTTP/1.0" 202 Accepted 2025-12-17 21:40:22,434 - lava_callback - INFO - Starting async job submit for node 69408035575bc060ce15c70a 2025-12-17 21:40:22,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:22,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408035575bc060ce15c70a HTTP/1.1" 502 150 2025-12-17 21:40:22,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408035575bc060ce15c70a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:22,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408035575bc060ce15c70a 2025-12-17 21:40:22,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408035575bc060ce15c70a HTTP/1.1" 502 150 2025-12-17 21:40:22,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408035575bc060ce15c70a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:22,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940801e575bc060ce15c684 2025-12-17 21:40:22,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940801e575bc060ce15c684 HTTP/1.1" 502 150 2025-12-17 21:40:22,931 - lava_callback - ERROR - Error processing callback for node 6940801e575bc060ce15c684: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940801e575bc060ce15c684 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940801e575bc060ce15c684 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940801e575bc060ce15c684 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:23,107 - lava_callback - INFO - Received callback for node 6940803a575bc060ce15c71d INFO: 172.21.0.1:37038 - "POST /node/6940803a575bc060ce15c71d HTTP/1.0" 202 Accepted 2025-12-17 21:40:23,113 - lava_callback - INFO - Starting async job submit for node 6940803a575bc060ce15c71d 2025-12-17 21:40:23,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:23,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803a575bc060ce15c71d HTTP/1.1" 502 150 2025-12-17 21:40:23,121 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803a575bc060ce15c71d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:23,121 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803a575bc060ce15c71d 2025-12-17 21:40:23,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803a575bc060ce15c71d HTTP/1.1" 502 150 2025-12-17 21:40:23,122 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803a575bc060ce15c71d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:23,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408020575bc060ce15c68a 2025-12-17 21:40:23,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408020575bc060ce15c68a HTTP/1.1" 502 150 2025-12-17 21:40:23,587 - lava_callback - ERROR - Error processing callback for node 69408020575bc060ce15c68a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408020575bc060ce15c68a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408020575bc060ce15c68a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408020575bc060ce15c68a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:23,757 - lava_callback - INFO - Received callback for node 6940803c575bc060ce15c720 INFO: 172.21.0.1:37044 - "POST /node/6940803c575bc060ce15c720 HTTP/1.0" 202 Accepted 2025-12-17 21:40:23,763 - lava_callback - INFO - Starting async job submit for node 6940803c575bc060ce15c720 2025-12-17 21:40:23,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:23,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803c575bc060ce15c720 HTTP/1.1" 502 150 2025-12-17 21:40:23,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803c575bc060ce15c720'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:23,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803c575bc060ce15c720 2025-12-17 21:40:23,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803c575bc060ce15c720 HTTP/1.1" 502 150 2025-12-17 21:40:23,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803c575bc060ce15c720'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:24,417 - lava_callback - INFO - Received callback for node 6940803d575bc060ce15c723 INFO: 172.21.0.1:37046 - "POST /node/6940803d575bc060ce15c723 HTTP/1.0" 202 Accepted 2025-12-17 21:40:24,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408035575bc060ce15c70a 2025-12-17 21:40:24,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408035575bc060ce15c70a HTTP/1.1" 502 150 2025-12-17 21:40:24,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408035575bc060ce15c70a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:24,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408037575bc060ce15c714 2025-12-17 21:40:24,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408037575bc060ce15c714 HTTP/1.1" 502 150 2025-12-17 21:40:24,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408037575bc060ce15c714'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:25,071 - lava_callback - INFO - Received callback for node 6940803e575bc060ce15c726 INFO: 172.21.0.1:37054 - "POST /node/6940803e575bc060ce15c726 HTTP/1.0" 202 Accepted 2025-12-17 21:40:25,122 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803a575bc060ce15c71d 2025-12-17 21:40:25,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803a575bc060ce15c71d HTTP/1.1" 502 150 2025-12-17 21:40:25,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803a575bc060ce15c71d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:25,743 - lava_callback - INFO - Received callback for node 69408042575bc060ce15c72c INFO: 172.21.0.1:37062 - "POST /node/69408042575bc060ce15c72c HTTP/1.0" 202 Accepted 2025-12-17 21:40:25,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803c575bc060ce15c720 2025-12-17 21:40:25,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803c575bc060ce15c720 HTTP/1.1" 502 150 2025-12-17 21:40:25,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803c575bc060ce15c720'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:25,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086f8575bc060ce15e226 2025-12-17 21:40:25,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f8575bc060ce15e226 HTTP/1.1" 502 150 2025-12-17 21:40:25,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086f8575bc060ce15e226'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:26,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408027575bc060ce15c6d9 2025-12-17 21:40:26,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408027575bc060ce15c6d9 HTTP/1.1" 502 150 2025-12-17 21:40:26,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408027575bc060ce15c6d9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:26,444 - lava_callback - INFO - Received callback for node 69408044575bc060ce15c732 INFO: 172.21.0.1:37064 - "POST /node/69408044575bc060ce15c732 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:37070 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:40:27,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408029575bc060ce15c6dc 2025-12-17 21:40:27,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408029575bc060ce15c6dc HTTP/1.1" 502 150 2025-12-17 21:40:27,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408029575bc060ce15c6dc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:27,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408032575bc060ce15c6f7 2025-12-17 21:40:27,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408032575bc060ce15c6f7 HTTP/1.1" 502 150 2025-12-17 21:40:27,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408032575bc060ce15c6f7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:27,955 - lava_callback - INFO - Received callback for node 69408038575bc060ce15c717 INFO: 172.21.0.1:37086 - "POST /node/69408038575bc060ce15c717 HTTP/1.0" 202 Accepted 2025-12-17 21:40:28,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408035575bc060ce15c70a 2025-12-17 21:40:28,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408035575bc060ce15c70a HTTP/1.1" 502 150 2025-12-17 21:40:28,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408035575bc060ce15c70a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:28,641 - lava_callback - INFO - Received callback for node 69408039575bc060ce15c71a INFO: 172.21.0.1:37092 - "POST /node/69408039575bc060ce15c71a HTTP/1.0" 202 Accepted 2025-12-17 21:40:29,123 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803a575bc060ce15c71d 2025-12-17 21:40:29,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803a575bc060ce15c71d HTTP/1.1" 502 150 2025-12-17 21:40:29,124 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803a575bc060ce15c71d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:29,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803c575bc060ce15c720 2025-12-17 21:40:29,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803c575bc060ce15c720 HTTP/1.1" 502 150 2025-12-17 21:40:29,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803c575bc060ce15c720'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:29,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408033575bc060ce15c701 2025-12-17 21:40:29,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408033575bc060ce15c701 HTTP/1.1" 502 150 2025-12-17 21:40:29,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408033575bc060ce15c701'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:32,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802a575bc060ce15c6df 2025-12-17 21:40:32,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802a575bc060ce15c6df HTTP/1.1" 502 150 2025-12-17 21:40:32,418 - lava_callback - ERROR - Error processing callback for node 6940802a575bc060ce15c6df: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940802a575bc060ce15c6df (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940802a575bc060ce15c6df (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940802a575bc060ce15c6df (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:32,420 - lava_callback - INFO - Starting async job submit for node 6940803d575bc060ce15c723 2025-12-17 21:40:32,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:32,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803d575bc060ce15c723 HTTP/1.1" 502 150 2025-12-17 21:40:32,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803d575bc060ce15c723'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:32,427 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803d575bc060ce15c723 2025-12-17 21:40:32,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803d575bc060ce15c723 HTTP/1.1" 502 150 2025-12-17 21:40:32,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803d575bc060ce15c723'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:32,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408037575bc060ce15c714 2025-12-17 21:40:32,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408037575bc060ce15c714 HTTP/1.1" 502 150 2025-12-17 21:40:32,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408037575bc060ce15c714'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:32,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408010575bc060ce15c678 2025-12-17 21:40:32,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408010575bc060ce15c678 HTTP/1.1" 502 150 2025-12-17 21:40:32,949 - lava_callback - ERROR - Error processing callback for node 69408010575bc060ce15c678: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408010575bc060ce15c678 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408010575bc060ce15c678 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408010575bc060ce15c678 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:32,951 - lava_callback - INFO - Starting async job submit for node 6940803e575bc060ce15c726 2025-12-17 21:40:32,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:32,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803e575bc060ce15c726 HTTP/1.1" 502 150 2025-12-17 21:40:32,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803e575bc060ce15c726'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:32,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803e575bc060ce15c726 2025-12-17 21:40:32,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803e575bc060ce15c726 HTTP/1.1" 502 150 2025-12-17 21:40:32,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803e575bc060ce15c726'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:32,962 - lava_callback - INFO - Received callback for node 69408040575bc060ce15c729 INFO: 172.21.0.1:40684 - "POST /node/69408040575bc060ce15c729 HTTP/1.0" 202 Accepted 2025-12-17 21:40:33,103 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802c575bc060ce15c6e2 2025-12-17 21:40:33,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802c575bc060ce15c6e2 HTTP/1.1" 502 150 2025-12-17 21:40:33,104 - lava_callback - ERROR - Error processing callback for node 6940802c575bc060ce15c6e2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940802c575bc060ce15c6e2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940802c575bc060ce15c6e2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940802c575bc060ce15c6e2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:33,106 - lava_callback - INFO - Starting async job submit for node 69408042575bc060ce15c72c 2025-12-17 21:40:33,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:33,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408042575bc060ce15c72c HTTP/1.1" 502 150 2025-12-17 21:40:33,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408042575bc060ce15c72c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:33,113 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408042575bc060ce15c72c 2025-12-17 21:40:33,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408042575bc060ce15c72c HTTP/1.1" 502 150 2025-12-17 21:40:33,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408042575bc060ce15c72c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:33,608 - lava_callback - INFO - Received callback for node 69408043575bc060ce15c72f INFO: 172.21.0.1:40696 - "POST /node/69408043575bc060ce15c72f HTTP/1.0" 202 Accepted 2025-12-17 21:40:33,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802d575bc060ce15c6e5 2025-12-17 21:40:33,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802d575bc060ce15c6e5 HTTP/1.1" 502 150 2025-12-17 21:40:33,762 - lava_callback - ERROR - Error processing callback for node 6940802d575bc060ce15c6e5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940802d575bc060ce15c6e5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940802d575bc060ce15c6e5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940802d575bc060ce15c6e5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:33,764 - lava_callback - INFO - Starting async job submit for node 69408044575bc060ce15c732 2025-12-17 21:40:33,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:33,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408044575bc060ce15c732 HTTP/1.1" 502 150 2025-12-17 21:40:33,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408044575bc060ce15c732'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:33,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408044575bc060ce15c732 2025-12-17 21:40:33,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408044575bc060ce15c732 HTTP/1.1" 502 150 2025-12-17 21:40:33,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408044575bc060ce15c732'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:33,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086f8575bc060ce15e226 2025-12-17 21:40:33,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f8575bc060ce15e226 HTTP/1.1" 502 150 2025-12-17 21:40:33,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086f8575bc060ce15e226'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:34,290 - lava_callback - INFO - Received callback for node 69408046575bc060ce15c735 INFO: 172.21.0.1:40702 - "POST /node/69408046575bc060ce15c735 HTTP/1.0" 202 Accepted 2025-12-17 21:40:34,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802e575bc060ce15c6e8 2025-12-17 21:40:34,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802e575bc060ce15c6e8 HTTP/1.1" 502 150 2025-12-17 21:40:34,419 - lava_callback - ERROR - Error processing callback for node 6940802e575bc060ce15c6e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940802e575bc060ce15c6e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940802e575bc060ce15c6e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940802e575bc060ce15c6e8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:34,421 - lava_callback - INFO - Starting async job submit for node 69408038575bc060ce15c717 2025-12-17 21:40:34,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:34,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408038575bc060ce15c717 HTTP/1.1" 502 150 2025-12-17 21:40:34,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408038575bc060ce15c717'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:34,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408038575bc060ce15c717 2025-12-17 21:40:34,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803d575bc060ce15c723 2025-12-17 21:40:34,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408038575bc060ce15c717 HTTP/1.1" 502 150 2025-12-17 21:40:34,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408038575bc060ce15c717'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:34,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803d575bc060ce15c723 HTTP/1.1" 502 150 2025-12-17 21:40:34,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803d575bc060ce15c723'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:34,840 - lava_callback - INFO - Received callback for node 694086fa575bc060ce15e229 INFO: 172.21.0.1:40712 - "POST /node/694086fa575bc060ce15e229 HTTP/1.0" 202 Accepted 2025-12-17 21:40:34,945 - lava_callback - INFO - Received callback for node 69408047575bc060ce15c73c INFO: 172.21.0.1:40724 - "POST /node/69408047575bc060ce15c73c HTTP/1.0" 202 Accepted 2025-12-17 21:40:34,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803e575bc060ce15c726 2025-12-17 21:40:34,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803e575bc060ce15c726 HTTP/1.1" 502 150 2025-12-17 21:40:34,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803e575bc060ce15c726'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:35,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408042575bc060ce15c72c 2025-12-17 21:40:35,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408042575bc060ce15c72c HTTP/1.1" 502 150 2025-12-17 21:40:35,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408042575bc060ce15c72c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:35,641 - lava_callback - INFO - Received callback for node 69408048575bc060ce15c743 INFO: 172.21.0.1:40732 - "POST /node/69408048575bc060ce15c743 HTTP/1.0" 202 Accepted 2025-12-17 21:40:35,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408044575bc060ce15c732 2025-12-17 21:40:35,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408044575bc060ce15c732 HTTP/1.1" 502 150 2025-12-17 21:40:35,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408044575bc060ce15c732'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:36,322 - lava_callback - INFO - Received callback for node 6940804a575bc060ce15c74a INFO: 172.21.0.1:40734 - "POST /node/6940804a575bc060ce15c74a HTTP/1.0" 202 Accepted 2025-12-17 21:40:36,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408038575bc060ce15c717 2025-12-17 21:40:36,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408038575bc060ce15c717 HTTP/1.1" 502 150 2025-12-17 21:40:36,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408038575bc060ce15c717'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:36,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408035575bc060ce15c70a 2025-12-17 21:40:36,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408035575bc060ce15c70a HTTP/1.1" 502 150 2025-12-17 21:40:36,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408035575bc060ce15c70a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40748 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:40:36,977 - lava_callback - INFO - Received callback for node 6940804c575bc060ce15c757 INFO: 172.21.0.1:40752 - "POST /node/6940804c575bc060ce15c757 HTTP/1.0" 202 Accepted 2025-12-17 21:40:37,125 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803a575bc060ce15c71d 2025-12-17 21:40:37,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803a575bc060ce15c71d HTTP/1.1" 502 150 2025-12-17 21:40:37,126 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803a575bc060ce15c71d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:37,672 - lava_callback - INFO - Received callback for node 6940804d575bc060ce15c75e INFO: 172.21.0.1:40756 - "POST /node/6940804d575bc060ce15c75e HTTP/1.0" 202 Accepted 2025-12-17 21:40:37,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408031575bc060ce15c6f0 HTTP/1.1" 502 150 2025-12-17 21:40:37,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408031575bc060ce15c6f0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:37,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803c575bc060ce15c720 2025-12-17 21:40:37,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803c575bc060ce15c720 HTTP/1.1" 502 150 2025-12-17 21:40:37,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803c575bc060ce15c720'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:38,347 - lava_callback - INFO - Received callback for node 6940804f575bc060ce15c761 INFO: 172.21.0.1:40772 - "POST /node/6940804f575bc060ce15c761 HTTP/1.0" 202 Accepted 2025-12-17 21:40:38,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803d575bc060ce15c723 2025-12-17 21:40:38,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803d575bc060ce15c723 HTTP/1.1" 502 150 2025-12-17 21:40:38,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803d575bc060ce15c723'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:38,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803e575bc060ce15c726 2025-12-17 21:40:38,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803e575bc060ce15c726 HTTP/1.1" 502 150 2025-12-17 21:40:38,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803e575bc060ce15c726'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:39,115 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408042575bc060ce15c72c 2025-12-17 21:40:39,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408042575bc060ce15c72c HTTP/1.1" 502 150 2025-12-17 21:40:39,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408042575bc060ce15c72c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:39,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408044575bc060ce15c732 2025-12-17 21:40:39,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408044575bc060ce15c732 HTTP/1.1" 502 150 2025-12-17 21:40:39,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408044575bc060ce15c732'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:40,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408038575bc060ce15c717 2025-12-17 21:40:40,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408038575bc060ce15c717 HTTP/1.1" 502 150 2025-12-17 21:40:40,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408038575bc060ce15c717'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:42,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408027575bc060ce15c6d9 2025-12-17 21:40:42,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408027575bc060ce15c6d9 HTTP/1.1" 502 150 2025-12-17 21:40:42,445 - lava_callback - ERROR - Error processing callback for node 69408027575bc060ce15c6d9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408027575bc060ce15c6d9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408027575bc060ce15c6d9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408027575bc060ce15c6d9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:42,446 - lava_callback - INFO - Starting async job submit for node 69408039575bc060ce15c71a 2025-12-17 21:40:42,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:42,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408039575bc060ce15c71a HTTP/1.1" 502 150 2025-12-17 21:40:42,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408039575bc060ce15c71a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:42,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408039575bc060ce15c71a 2025-12-17 21:40:42,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408039575bc060ce15c71a HTTP/1.1" 502 150 2025-12-17 21:40:42,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408039575bc060ce15c71a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:42,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802f575bc060ce15c6eb HTTP/1.1" 502 150 2025-12-17 21:40:42,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802f575bc060ce15c6eb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:42,792 - lava_callback - INFO - Received callback for node 694087bb575bc060ce15e4b1 INFO: 172.21.0.1:49614 - "POST /node/694087bb575bc060ce15e4b1 HTTP/1.0" 202 Accepted 2025-12-17 21:40:43,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408029575bc060ce15c6dc 2025-12-17 21:40:43,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408029575bc060ce15c6dc HTTP/1.1" 502 150 2025-12-17 21:40:43,858 - lava_callback - ERROR - Error processing callback for node 69408029575bc060ce15c6dc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408029575bc060ce15c6dc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408029575bc060ce15c6dc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408029575bc060ce15c6dc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:43,860 - lava_callback - INFO - Starting async job submit for node 69408040575bc060ce15c729 2025-12-17 21:40:43,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:43,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408040575bc060ce15c729 HTTP/1.1" 502 150 2025-12-17 21:40:43,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408040575bc060ce15c729'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:43,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408040575bc060ce15c729 2025-12-17 21:40:43,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408040575bc060ce15c729 HTTP/1.1" 502 150 2025-12-17 21:40:43,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408040575bc060ce15c729'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:43,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408032575bc060ce15c6f7 2025-12-17 21:40:43,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408032575bc060ce15c6f7 HTTP/1.1" 502 150 2025-12-17 21:40:43,909 - lava_callback - ERROR - Error processing callback for node 69408032575bc060ce15c6f7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408032575bc060ce15c6f7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408032575bc060ce15c6f7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408032575bc060ce15c6f7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:43,910 - lava_callback - INFO - Starting async job submit for node 69408043575bc060ce15c72f 2025-12-17 21:40:43,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:43,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408043575bc060ce15c72f HTTP/1.1" 502 150 2025-12-17 21:40:43,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408043575bc060ce15c72f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:43,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408043575bc060ce15c72f 2025-12-17 21:40:43,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408043575bc060ce15c72f HTTP/1.1" 502 150 2025-12-17 21:40:43,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408043575bc060ce15c72f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:44,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408039575bc060ce15c71a 2025-12-17 21:40:44,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408039575bc060ce15c71a HTTP/1.1" 502 150 2025-12-17 21:40:44,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408039575bc060ce15c71a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:45,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408033575bc060ce15c701 2025-12-17 21:40:45,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408040575bc060ce15c729 2025-12-17 21:40:45,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408033575bc060ce15c701 HTTP/1.1" 502 150 2025-12-17 21:40:45,869 - lava_callback - ERROR - Error processing callback for node 69408033575bc060ce15c701: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408033575bc060ce15c701 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408033575bc060ce15c701 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408033575bc060ce15c701 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:45,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408040575bc060ce15c729 HTTP/1.1" 502 150 2025-12-17 21:40:45,872 - lava_callback - INFO - Starting async job submit for node 69408046575bc060ce15c735 2025-12-17 21:40:45,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408040575bc060ce15c729'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:45,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:45,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408046575bc060ce15c735 HTTP/1.1" 502 150 2025-12-17 21:40:45,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408046575bc060ce15c735'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:45,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408046575bc060ce15c735 2025-12-17 21:40:45,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408046575bc060ce15c735 HTTP/1.1" 502 150 2025-12-17 21:40:45,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408046575bc060ce15c735'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:45,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408043575bc060ce15c72f 2025-12-17 21:40:45,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408043575bc060ce15c72f HTTP/1.1" 502 150 2025-12-17 21:40:45,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408043575bc060ce15c72f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:46,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803d575bc060ce15c723 2025-12-17 21:40:46,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803d575bc060ce15c723 HTTP/1.1" 502 150 2025-12-17 21:40:46,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803d575bc060ce15c723'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49630 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:40:46,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803e575bc060ce15c726 2025-12-17 21:40:46,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803e575bc060ce15c726 HTTP/1.1" 502 150 2025-12-17 21:40:46,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940803e575bc060ce15c726'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:47,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408042575bc060ce15c72c 2025-12-17 21:40:47,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408042575bc060ce15c72c HTTP/1.1" 502 150 2025-12-17 21:40:47,118 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408042575bc060ce15c72c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:47,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408044575bc060ce15c732 2025-12-17 21:40:47,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408044575bc060ce15c732 HTTP/1.1" 502 150 2025-12-17 21:40:47,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408044575bc060ce15c732'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:47,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408046575bc060ce15c735 2025-12-17 21:40:47,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408046575bc060ce15c735 HTTP/1.1" 502 150 2025-12-17 21:40:47,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408046575bc060ce15c735'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:48,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408038575bc060ce15c717 2025-12-17 21:40:48,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408038575bc060ce15c717 HTTP/1.1" 502 150 2025-12-17 21:40:48,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408038575bc060ce15c717'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:48,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408039575bc060ce15c71a 2025-12-17 21:40:48,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408039575bc060ce15c71a HTTP/1.1" 502 150 2025-12-17 21:40:48,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408039575bc060ce15c71a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:48,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408037575bc060ce15c714 2025-12-17 21:40:48,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408037575bc060ce15c714 HTTP/1.1" 502 150 2025-12-17 21:40:48,636 - lava_callback - ERROR - Error processing callback for node 69408037575bc060ce15c714: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408037575bc060ce15c714 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408037575bc060ce15c714 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408037575bc060ce15c714 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:48,637 - lava_callback - INFO - Starting async job submit for node 694086fa575bc060ce15e229 2025-12-17 21:40:48,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:48,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fa575bc060ce15e229 HTTP/1.1" 502 150 2025-12-17 21:40:48,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086fa575bc060ce15e229'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:48,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086fa575bc060ce15e229 2025-12-17 21:40:48,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fa575bc060ce15e229 HTTP/1.1" 502 150 2025-12-17 21:40:48,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086fa575bc060ce15e229'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:49,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086f8575bc060ce15e226 2025-12-17 21:40:49,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f8575bc060ce15e226 HTTP/1.1" 502 150 2025-12-17 21:40:49,871 - lava_callback - ERROR - Error processing callback for node 694086f8575bc060ce15e226: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086f8575bc060ce15e226 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086f8575bc060ce15e226 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086f8575bc060ce15e226 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:49,873 - lava_callback - INFO - Starting async job submit for node 69408047575bc060ce15c73c 2025-12-17 21:40:49,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408040575bc060ce15c729 2025-12-17 21:40:49,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:49,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408040575bc060ce15c729 HTTP/1.1" 502 150 2025-12-17 21:40:49,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408040575bc060ce15c729'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:49,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408047575bc060ce15c73c HTTP/1.1" 502 150 2025-12-17 21:40:49,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408047575bc060ce15c73c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:49,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408047575bc060ce15c73c 2025-12-17 21:40:49,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408047575bc060ce15c73c HTTP/1.1" 502 150 2025-12-17 21:40:49,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408047575bc060ce15c73c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:49,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408043575bc060ce15c72f 2025-12-17 21:40:49,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408043575bc060ce15c72f HTTP/1.1" 502 150 2025-12-17 21:40:49,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408043575bc060ce15c72f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:50,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802f575bc060ce15c6eb 2025-12-17 21:40:50,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802f575bc060ce15c6eb HTTP/1.1" 502 150 2025-12-17 21:40:50,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940802f575bc060ce15c6eb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:50,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086fa575bc060ce15e229 2025-12-17 21:40:50,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fa575bc060ce15e229 HTTP/1.1" 502 150 2025-12-17 21:40:50,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086fa575bc060ce15e229'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:51,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408047575bc060ce15c73c 2025-12-17 21:40:51,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408046575bc060ce15c735 2025-12-17 21:40:51,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408047575bc060ce15c73c HTTP/1.1" 502 150 2025-12-17 21:40:51,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408046575bc060ce15c735 HTTP/1.1" 502 150 2025-12-17 21:40:51,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408047575bc060ce15c73c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:51,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408046575bc060ce15c735'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:52,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408035575bc060ce15c70a 2025-12-17 21:40:52,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408035575bc060ce15c70a HTTP/1.1" 502 150 2025-12-17 21:40:52,448 - lava_callback - ERROR - Error processing callback for node 69408035575bc060ce15c70a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408035575bc060ce15c70a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408035575bc060ce15c70a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408035575bc060ce15c70a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:52,451 - lava_callback - INFO - Starting async job submit for node 69408048575bc060ce15c743 2025-12-17 21:40:52,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:52,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408048575bc060ce15c743 HTTP/1.1" 502 150 2025-12-17 21:40:52,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408048575bc060ce15c743'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:52,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408048575bc060ce15c743 2025-12-17 21:40:52,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408048575bc060ce15c743 HTTP/1.1" 502 150 2025-12-17 21:40:52,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408048575bc060ce15c743'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:52,937 - lava_callback - INFO - Received callback for node 69408050575bc060ce15c764 INFO: 172.21.0.1:33732 - "POST /node/69408050575bc060ce15c764 HTTP/1.0" 202 Accepted 2025-12-17 21:40:53,126 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803a575bc060ce15c71d 2025-12-17 21:40:53,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803a575bc060ce15c71d HTTP/1.1" 502 150 2025-12-17 21:40:53,128 - lava_callback - ERROR - Error processing callback for node 6940803a575bc060ce15c71d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940803a575bc060ce15c71d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940803a575bc060ce15c71d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940803a575bc060ce15c71d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:53,130 - lava_callback - INFO - Starting async job submit for node 6940804a575bc060ce15c74a 2025-12-17 21:40:53,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:53,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804a575bc060ce15c74a HTTP/1.1" 502 150 2025-12-17 21:40:53,139 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804a575bc060ce15c74a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:53,139 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804a575bc060ce15c74a 2025-12-17 21:40:53,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804a575bc060ce15c74a HTTP/1.1" 502 150 2025-12-17 21:40:53,140 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804a575bc060ce15c74a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:53,595 - lava_callback - INFO - Received callback for node 69408051575bc060ce15c767 INFO: 172.21.0.1:33748 - "POST /node/69408051575bc060ce15c767 HTTP/1.0" 202 Accepted 2025-12-17 21:40:53,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408031575bc060ce15c6f0 2025-12-17 21:40:53,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408031575bc060ce15c6f0 HTTP/1.1" 502 150 2025-12-17 21:40:53,681 - lava_callback - ERROR - Error processing callback for node 69408031575bc060ce15c6f0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408031575bc060ce15c6f0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408031575bc060ce15c6f0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408031575bc060ce15c6f0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:53,683 - lava_callback - INFO - Starting async job submit for node 6940804c575bc060ce15c757 2025-12-17 21:40:53,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:53,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804c575bc060ce15c757 HTTP/1.1" 502 150 2025-12-17 21:40:53,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804c575bc060ce15c757'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:53,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804c575bc060ce15c757 2025-12-17 21:40:53,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804c575bc060ce15c757 HTTP/1.1" 502 150 2025-12-17 21:40:53,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804c575bc060ce15c757'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:53,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803c575bc060ce15c720 2025-12-17 21:40:53,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803c575bc060ce15c720 HTTP/1.1" 502 150 2025-12-17 21:40:53,795 - lava_callback - ERROR - Error processing callback for node 6940803c575bc060ce15c720: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940803c575bc060ce15c720 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940803c575bc060ce15c720 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940803c575bc060ce15c720 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:40:53,797 - lava_callback - INFO - Starting async job submit for node 6940804d575bc060ce15c75e 2025-12-17 21:40:53,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:40:53,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804d575bc060ce15c75e HTTP/1.1" 502 150 2025-12-17 21:40:53,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804d575bc060ce15c75e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:53,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804d575bc060ce15c75e 2025-12-17 21:40:53,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804d575bc060ce15c75e HTTP/1.1" 502 150 2025-12-17 21:40:53,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804d575bc060ce15c75e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:54,273 - lava_callback - INFO - Received callback for node 69408052575bc060ce15c76a INFO: 172.21.0.1:33750 - "POST /node/69408052575bc060ce15c76a HTTP/1.0" 202 Accepted 2025-12-17 21:40:54,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408048575bc060ce15c743 2025-12-17 21:40:54,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408048575bc060ce15c743 HTTP/1.1" 502 150 2025-12-17 21:40:54,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408048575bc060ce15c743'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:54,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086fa575bc060ce15e229 2025-12-17 21:40:54,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fa575bc060ce15e229 HTTP/1.1" 502 150 2025-12-17 21:40:54,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086fa575bc060ce15e229'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:54,929 - lava_callback - INFO - Received callback for node 69408054575bc060ce15c76d INFO: 172.21.0.1:33760 - "POST /node/69408054575bc060ce15c76d HTTP/1.0" 202 Accepted 2025-12-17 21:40:55,140 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804a575bc060ce15c74a 2025-12-17 21:40:55,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804a575bc060ce15c74a HTTP/1.1" 502 150 2025-12-17 21:40:55,141 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804a575bc060ce15c74a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:55,602 - lava_callback - INFO - Received callback for node 69408055575bc060ce15c770 INFO: 172.21.0.1:33762 - "POST /node/69408055575bc060ce15c770 HTTP/1.0" 202 Accepted 2025-12-17 21:40:55,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804c575bc060ce15c757 2025-12-17 21:40:55,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804c575bc060ce15c757 HTTP/1.1" 502 150 2025-12-17 21:40:55,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804c575bc060ce15c757'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:55,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804d575bc060ce15c75e 2025-12-17 21:40:55,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804d575bc060ce15c75e HTTP/1.1" 502 150 2025-12-17 21:40:55,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804d575bc060ce15c75e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:55,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408047575bc060ce15c73c 2025-12-17 21:40:55,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408047575bc060ce15c73c HTTP/1.1" 502 150 2025-12-17 21:40:55,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408047575bc060ce15c73c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:56,271 - lava_callback - INFO - Received callback for node 69408056575bc060ce15c773 INFO: 172.21.0.1:33770 - "POST /node/69408056575bc060ce15c773 HTTP/1.0" 202 Accepted 2025-12-17 21:40:56,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408039575bc060ce15c71a 2025-12-17 21:40:56,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408039575bc060ce15c71a HTTP/1.1" 502 150 2025-12-17 21:40:56,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408039575bc060ce15c71a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33784 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:40:56,950 - lava_callback - INFO - Received callback for node 69408057575bc060ce15c776 INFO: 172.21.0.1:33796 - "POST /node/69408057575bc060ce15c776 HTTP/1.0" 202 Accepted 2025-12-17 21:40:57,628 - lava_callback - INFO - Received callback for node 6940805e575bc060ce15c792 INFO: 172.21.0.1:33810 - "POST /node/6940805e575bc060ce15c792 HTTP/1.0" 202 Accepted 2025-12-17 21:40:57,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408040575bc060ce15c729 2025-12-17 21:40:57,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408040575bc060ce15c729 HTTP/1.1" 502 150 2025-12-17 21:40:57,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408040575bc060ce15c729'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:57,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408043575bc060ce15c72f 2025-12-17 21:40:57,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408043575bc060ce15c72f HTTP/1.1" 502 150 2025-12-17 21:40:57,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408043575bc060ce15c72f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:58,295 - lava_callback - INFO - Received callback for node 6940805f575bc060ce15c79f INFO: 172.21.0.1:33822 - "POST /node/6940805f575bc060ce15c79f HTTP/1.0" 202 Accepted 2025-12-17 21:40:58,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408048575bc060ce15c743 2025-12-17 21:40:58,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408048575bc060ce15c743 HTTP/1.1" 502 150 2025-12-17 21:40:58,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408048575bc060ce15c743'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:58,959 - lava_callback - INFO - Received callback for node 69408062575bc060ce15c7d5 INFO: 172.21.0.1:33830 - "POST /node/69408062575bc060ce15c7d5 HTTP/1.0" 202 Accepted 2025-12-17 21:40:59,141 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804a575bc060ce15c74a 2025-12-17 21:40:59,627 - lava_callback - INFO - Received callback for node 69408063575bc060ce15c7dc INFO: 172.21.0.1:33840 - "POST /node/69408063575bc060ce15c7dc HTTP/1.0" 202 Accepted 2025-12-17 21:40:59,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804c575bc060ce15c757 2025-12-17 21:40:59,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804c575bc060ce15c757 HTTP/1.1" 502 150 2025-12-17 21:40:59,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804c575bc060ce15c757'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:59,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804d575bc060ce15c75e 2025-12-17 21:40:59,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804d575bc060ce15c75e HTTP/1.1" 502 150 2025-12-17 21:40:59,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804d575bc060ce15c75e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:40:59,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408046575bc060ce15c735 2025-12-17 21:40:59,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408046575bc060ce15c735 HTTP/1.1" 502 150 2025-12-17 21:40:59,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408046575bc060ce15c735'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44492 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:41:02,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803d575bc060ce15c723 2025-12-17 21:41:02,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803d575bc060ce15c723 HTTP/1.1" 502 150 2025-12-17 21:41:02,433 - lava_callback - ERROR - Error processing callback for node 6940803d575bc060ce15c723: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940803d575bc060ce15c723 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940803d575bc060ce15c723 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940803d575bc060ce15c723 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:02,435 - lava_callback - INFO - Starting async job submit for node 6940804f575bc060ce15c761 2025-12-17 21:41:02,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:02,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804f575bc060ce15c761 HTTP/1.1" 502 150 2025-12-17 21:41:02,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804f575bc060ce15c761'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:02,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804f575bc060ce15c761 2025-12-17 21:41:02,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804f575bc060ce15c761 HTTP/1.1" 502 150 2025-12-17 21:41:02,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804f575bc060ce15c761'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:02,478 - lava_callback - INFO - Received callback for node 69408059575bc060ce15c77b INFO: 172.21.0.1:44502 - "POST /node/69408059575bc060ce15c77b HTTP/1.0" 202 Accepted 2025-12-17 21:41:02,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086fa575bc060ce15e229 2025-12-17 21:41:02,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fa575bc060ce15e229 HTTP/1.1" 502 150 2025-12-17 21:41:02,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086fa575bc060ce15e229'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:02,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940803e575bc060ce15c726 2025-12-17 21:41:02,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940803e575bc060ce15c726 HTTP/1.1" 502 150 2025-12-17 21:41:02,965 - lava_callback - ERROR - Error processing callback for node 6940803e575bc060ce15c726: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940803e575bc060ce15c726 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940803e575bc060ce15c726 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940803e575bc060ce15c726 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:02,968 - lava_callback - INFO - Starting async job submit for node 694087bb575bc060ce15e4b1 2025-12-17 21:41:02,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:03,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087bb575bc060ce15e4b1 HTTP/1.1" 502 150 2025-12-17 21:41:03,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087bb575bc060ce15e4b1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:03,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087bb575bc060ce15e4b1 2025-12-17 21:41:03,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087bb575bc060ce15e4b1 HTTP/1.1" 502 150 2025-12-17 21:41:03,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087bb575bc060ce15e4b1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:03,118 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408042575bc060ce15c72c 2025-12-17 21:41:03,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408042575bc060ce15c72c HTTP/1.1" 502 150 2025-12-17 21:41:03,121 - lava_callback - ERROR - Error processing callback for node 69408042575bc060ce15c72c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408042575bc060ce15c72c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408042575bc060ce15c72c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408042575bc060ce15c72c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:03,123 - lava_callback - INFO - Starting async job submit for node 69408050575bc060ce15c764 2025-12-17 21:41:03,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:03,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408050575bc060ce15c764 HTTP/1.1" 502 150 2025-12-17 21:41:03,130 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408050575bc060ce15c764'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:03,130 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408050575bc060ce15c764 2025-12-17 21:41:03,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408050575bc060ce15c764 HTTP/1.1" 502 150 2025-12-17 21:41:03,131 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408050575bc060ce15c764'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:03,134 - lava_callback - INFO - Received callback for node 6940805a575bc060ce15c784 INFO: 172.21.0.1:44504 - "POST /node/6940805a575bc060ce15c784 HTTP/1.0" 202 Accepted 2025-12-17 21:41:03,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408044575bc060ce15c732 2025-12-17 21:41:03,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408044575bc060ce15c732 HTTP/1.1" 502 150 2025-12-17 21:41:03,778 - lava_callback - ERROR - Error processing callback for node 69408044575bc060ce15c732: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408044575bc060ce15c732 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408044575bc060ce15c732 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408044575bc060ce15c732 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:03,780 - lava_callback - INFO - Starting async job submit for node 69408051575bc060ce15c767 2025-12-17 21:41:03,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:03,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408051575bc060ce15c767 HTTP/1.1" 502 150 2025-12-17 21:41:03,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408051575bc060ce15c767'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:03,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408051575bc060ce15c767 2025-12-17 21:41:03,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408051575bc060ce15c767 HTTP/1.1" 502 150 2025-12-17 21:41:03,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408051575bc060ce15c767'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:03,826 - lava_callback - INFO - Received callback for node 6940805c575bc060ce15c78b INFO: 172.21.0.1:44520 - "POST /node/6940805c575bc060ce15c78b HTTP/1.0" 202 Accepted 2025-12-17 21:41:03,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408047575bc060ce15c73c 2025-12-17 21:41:03,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408047575bc060ce15c73c HTTP/1.1" 502 150 2025-12-17 21:41:03,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408047575bc060ce15c73c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:04,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408038575bc060ce15c717 2025-12-17 21:41:04,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408038575bc060ce15c717 HTTP/1.1" 502 150 2025-12-17 21:41:04,435 - lava_callback - ERROR - Error processing callback for node 69408038575bc060ce15c717: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408038575bc060ce15c717 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408038575bc060ce15c717 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408038575bc060ce15c717 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:04,437 - lava_callback - INFO - Starting async job submit for node 69408052575bc060ce15c76a 2025-12-17 21:41:04,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:04,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804f575bc060ce15c761 2025-12-17 21:41:04,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804f575bc060ce15c761 HTTP/1.1" 502 150 2025-12-17 21:41:04,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804f575bc060ce15c761'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:04,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408052575bc060ce15c76a HTTP/1.1" 502 150 2025-12-17 21:41:04,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408052575bc060ce15c76a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:04,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408052575bc060ce15c76a 2025-12-17 21:41:04,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408052575bc060ce15c76a HTTP/1.1" 502 150 2025-12-17 21:41:04,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408052575bc060ce15c76a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:04,514 - lava_callback - INFO - Received callback for node 69408066575bc060ce15c7e2 INFO: 172.21.0.1:44524 - "POST /node/69408066575bc060ce15c7e2 HTTP/1.0" 202 Accepted 2025-12-17 21:41:05,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087bb575bc060ce15e4b1 2025-12-17 21:41:05,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087bb575bc060ce15e4b1 HTTP/1.1" 502 150 2025-12-17 21:41:05,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087bb575bc060ce15e4b1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:05,131 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408050575bc060ce15c764 2025-12-17 21:41:05,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408050575bc060ce15c764 HTTP/1.1" 502 150 2025-12-17 21:41:05,132 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408050575bc060ce15c764'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:05,198 - lava_callback - INFO - Received callback for node 69408067575bc060ce15c7e5 INFO: 172.21.0.1:44534 - "POST /node/69408067575bc060ce15c7e5 HTTP/1.0" 202 Accepted 2025-12-17 21:41:05,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408051575bc060ce15c767 2025-12-17 21:41:05,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408051575bc060ce15c767 HTTP/1.1" 502 150 2025-12-17 21:41:05,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408051575bc060ce15c767'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:05,882 - lava_callback - INFO - Received callback for node 69408069575bc060ce15c7e8 INFO: 172.21.0.1:44544 - "POST /node/69408069575bc060ce15c7e8 HTTP/1.0" 202 Accepted 2025-12-17 21:41:06,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408052575bc060ce15c76a 2025-12-17 21:41:06,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408052575bc060ce15c76a HTTP/1.1" 502 150 2025-12-17 21:41:06,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408052575bc060ce15c76a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:06,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408048575bc060ce15c743 2025-12-17 21:41:06,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408048575bc060ce15c743 HTTP/1.1" 502 150 2025-12-17 21:41:06,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408048575bc060ce15c743'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44554 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:41:06,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940802f575bc060ce15c6eb 2025-12-17 21:41:06,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940802f575bc060ce15c6eb HTTP/1.1" 502 150 2025-12-17 21:41:06,547 - lava_callback - ERROR - Error processing callback for node 6940802f575bc060ce15c6eb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940802f575bc060ce15c6eb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940802f575bc060ce15c6eb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940802f575bc060ce15c6eb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:06,549 - lava_callback - INFO - Starting async job submit for node 69408054575bc060ce15c76d 2025-12-17 21:41:06,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:06,550 - lava_callback - INFO - Received callback for node 6940806a575bc060ce15c7eb INFO: 172.21.0.1:44558 - "POST /node/6940806a575bc060ce15c7eb HTTP/1.0" 202 Accepted 2025-12-17 21:41:06,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408054575bc060ce15c76d HTTP/1.1" 502 150 2025-12-17 21:41:06,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408054575bc060ce15c76d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:06,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408054575bc060ce15c76d 2025-12-17 21:41:06,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408054575bc060ce15c76d HTTP/1.1" 502 150 2025-12-17 21:41:06,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408054575bc060ce15c76d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:07,230 - lava_callback - INFO - Received callback for node 6940806b575bc060ce15c7ee INFO: 172.21.0.1:44562 - "POST /node/6940806b575bc060ce15c7ee HTTP/1.0" 202 Accepted 2025-12-17 21:41:07,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804c575bc060ce15c757 2025-12-17 21:41:07,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804c575bc060ce15c757 HTTP/1.1" 502 150 2025-12-17 21:41:07,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804c575bc060ce15c757'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:07,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804d575bc060ce15c75e 2025-12-17 21:41:07,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804d575bc060ce15c75e HTTP/1.1" 502 150 2025-12-17 21:41:07,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804d575bc060ce15c75e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:07,908 - lava_callback - INFO - Received callback for node 6940806d575bc060ce15c7f3 INFO: 172.21.0.1:44572 - "POST /node/6940806d575bc060ce15c7f3 HTTP/1.0" 202 Accepted 2025-12-17 21:41:08,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804f575bc060ce15c761 2025-12-17 21:41:08,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804f575bc060ce15c761 HTTP/1.1" 502 150 2025-12-17 21:41:08,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804f575bc060ce15c761'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:08,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408054575bc060ce15c76d 2025-12-17 21:41:08,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408054575bc060ce15c76d HTTP/1.1" 502 150 2025-12-17 21:41:08,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408054575bc060ce15c76d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:09,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087bb575bc060ce15e4b1 2025-12-17 21:41:09,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087bb575bc060ce15e4b1 HTTP/1.1" 502 150 2025-12-17 21:41:09,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087bb575bc060ce15e4b1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:09,132 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408050575bc060ce15c764 2025-12-17 21:41:09,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408050575bc060ce15c764 HTTP/1.1" 502 150 2025-12-17 21:41:09,133 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408050575bc060ce15c764'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:09,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408051575bc060ce15c767 2025-12-17 21:41:09,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408051575bc060ce15c767 HTTP/1.1" 502 150 2025-12-17 21:41:09,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408051575bc060ce15c767'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:10,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408052575bc060ce15c76a 2025-12-17 21:41:10,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408052575bc060ce15c76a HTTP/1.1" 502 150 2025-12-17 21:41:10,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408052575bc060ce15c76a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:12,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408039575bc060ce15c71a 2025-12-17 21:41:12,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408039575bc060ce15c71a HTTP/1.1" 502 150 2025-12-17 21:41:12,464 - lava_callback - ERROR - Error processing callback for node 69408039575bc060ce15c71a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408039575bc060ce15c71a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408039575bc060ce15c71a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408039575bc060ce15c71a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:12,466 - lava_callback - INFO - Starting async job submit for node 69408055575bc060ce15c770 2025-12-17 21:41:12,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:12,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408055575bc060ce15c770 HTTP/1.1" 502 150 2025-12-17 21:41:12,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408055575bc060ce15c770'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:12,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408055575bc060ce15c770 2025-12-17 21:41:12,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408055575bc060ce15c770 HTTP/1.1" 502 150 2025-12-17 21:41:12,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408055575bc060ce15c770'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:12,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408054575bc060ce15c76d 2025-12-17 21:41:12,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408054575bc060ce15c76d HTTP/1.1" 502 150 2025-12-17 21:41:12,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408054575bc060ce15c76d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:12,965 - lava_callback - INFO - Received callback for node 69408065575bc060ce15c7df INFO: 172.21.0.1:47680 - "POST /node/69408065575bc060ce15c7df HTTP/1.0" 202 Accepted 2025-12-17 21:41:13,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408040575bc060ce15c729 2025-12-17 21:41:13,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408040575bc060ce15c729 HTTP/1.1" 502 150 2025-12-17 21:41:13,878 - lava_callback - ERROR - Error processing callback for node 69408040575bc060ce15c729: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408040575bc060ce15c729 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408040575bc060ce15c729 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408040575bc060ce15c729 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:13,880 - lava_callback - INFO - Starting async job submit for node 69408056575bc060ce15c773 2025-12-17 21:41:13,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:13,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408056575bc060ce15c773 HTTP/1.1" 502 150 2025-12-17 21:41:13,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408056575bc060ce15c773'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:13,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408056575bc060ce15c773 2025-12-17 21:41:13,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408056575bc060ce15c773 HTTP/1.1" 502 150 2025-12-17 21:41:13,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408056575bc060ce15c773'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:13,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408043575bc060ce15c72f 2025-12-17 21:41:13,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408043575bc060ce15c72f HTTP/1.1" 502 150 2025-12-17 21:41:13,924 - lava_callback - ERROR - Error processing callback for node 69408043575bc060ce15c72f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408043575bc060ce15c72f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408043575bc060ce15c72f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408043575bc060ce15c72f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:13,926 - lava_callback - INFO - Starting async job submit for node 69408057575bc060ce15c776 2025-12-17 21:41:13,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:13,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408057575bc060ce15c776 HTTP/1.1" 502 150 2025-12-17 21:41:13,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408057575bc060ce15c776'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:13,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408057575bc060ce15c776 2025-12-17 21:41:13,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408057575bc060ce15c776 HTTP/1.1" 502 150 2025-12-17 21:41:13,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408057575bc060ce15c776'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:14,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408055575bc060ce15c770 2025-12-17 21:41:14,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804a575bc060ce15c74a HTTP/1.1" 502 150 2025-12-17 21:41:14,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804a575bc060ce15c74a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47684 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:41:15,677 - lava_callback - INFO - Received callback for node 69408eb6575bc060ce15ff43 INFO: 172.21.0.1:47692 - "POST /node/69408eb6575bc060ce15ff43 HTTP/1.0" 202 Accepted 2025-12-17 21:41:15,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408046575bc060ce15c735 2025-12-17 21:41:15,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408046575bc060ce15c735 HTTP/1.1" 502 150 2025-12-17 21:41:15,886 - lava_callback - ERROR - Error processing callback for node 69408046575bc060ce15c735: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408046575bc060ce15c735 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408046575bc060ce15c735 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408046575bc060ce15c735 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:15,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408056575bc060ce15c773 2025-12-17 21:41:15,889 - lava_callback - INFO - Starting async job submit for node 6940805e575bc060ce15c792 2025-12-17 21:41:15,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:15,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408056575bc060ce15c773 HTTP/1.1" 502 150 2025-12-17 21:41:15,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408056575bc060ce15c773'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:15,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805e575bc060ce15c792 HTTP/1.1" 502 150 2025-12-17 21:41:15,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805e575bc060ce15c792'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:15,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805e575bc060ce15c792 2025-12-17 21:41:15,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805e575bc060ce15c792 HTTP/1.1" 502 150 2025-12-17 21:41:15,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805e575bc060ce15c792'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:15,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408057575bc060ce15c776 2025-12-17 21:41:15,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408057575bc060ce15c776 HTTP/1.1" 502 150 2025-12-17 21:41:15,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408057575bc060ce15c776'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:16,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804f575bc060ce15c761 2025-12-17 21:41:16,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804f575bc060ce15c761 HTTP/1.1" 502 150 2025-12-17 21:41:16,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804f575bc060ce15c761'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47698 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:41:17,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087bb575bc060ce15e4b1 2025-12-17 21:41:17,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087bb575bc060ce15e4b1 HTTP/1.1" 502 150 2025-12-17 21:41:17,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087bb575bc060ce15e4b1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:17,134 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408050575bc060ce15c764 2025-12-17 21:41:17,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408050575bc060ce15c764 HTTP/1.1" 502 150 2025-12-17 21:41:17,135 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408050575bc060ce15c764'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:17,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408051575bc060ce15c767 2025-12-17 21:41:17,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408051575bc060ce15c767 HTTP/1.1" 502 150 2025-12-17 21:41:17,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408051575bc060ce15c767'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:17,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805e575bc060ce15c792 2025-12-17 21:41:17,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805e575bc060ce15c792 HTTP/1.1" 502 150 2025-12-17 21:41:17,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805e575bc060ce15c792'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:18,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408052575bc060ce15c76a 2025-12-17 21:41:18,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408052575bc060ce15c76a HTTP/1.1" 502 150 2025-12-17 21:41:18,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408052575bc060ce15c76a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:18,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086fa575bc060ce15e229 2025-12-17 21:41:18,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fa575bc060ce15e229 HTTP/1.1" 502 150 2025-12-17 21:41:18,682 - lava_callback - ERROR - Error processing callback for node 694086fa575bc060ce15e229: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086fa575bc060ce15e229 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086fa575bc060ce15e229 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086fa575bc060ce15e229 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:18,684 - lava_callback - INFO - Starting async job submit for node 6940805f575bc060ce15c79f 2025-12-17 21:41:18,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:18,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805f575bc060ce15c79f HTTP/1.1" 502 150 2025-12-17 21:41:18,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805f575bc060ce15c79f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:18,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805f575bc060ce15c79f 2025-12-17 21:41:18,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805f575bc060ce15c79f HTTP/1.1" 502 150 2025-12-17 21:41:18,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805f575bc060ce15c79f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:19,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408047575bc060ce15c73c 2025-12-17 21:41:19,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408047575bc060ce15c73c HTTP/1.1" 502 150 2025-12-17 21:41:19,887 - lava_callback - ERROR - Error processing callback for node 69408047575bc060ce15c73c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408047575bc060ce15c73c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408047575bc060ce15c73c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408047575bc060ce15c73c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:19,889 - lava_callback - INFO - Starting async job submit for node 69408062575bc060ce15c7d5 2025-12-17 21:41:19,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:19,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408056575bc060ce15c773 2025-12-17 21:41:19,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408056575bc060ce15c773 HTTP/1.1" 502 150 2025-12-17 21:41:19,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408056575bc060ce15c773'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:19,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408062575bc060ce15c7d5 HTTP/1.1" 502 150 2025-12-17 21:41:19,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408062575bc060ce15c7d5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:19,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408062575bc060ce15c7d5 2025-12-17 21:41:19,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408062575bc060ce15c7d5 HTTP/1.1" 502 150 2025-12-17 21:41:19,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408062575bc060ce15c7d5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:19,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408057575bc060ce15c776 2025-12-17 21:41:19,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408057575bc060ce15c776 HTTP/1.1" 502 150 2025-12-17 21:41:19,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408057575bc060ce15c776'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:20,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408054575bc060ce15c76d 2025-12-17 21:41:20,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408054575bc060ce15c76d HTTP/1.1" 502 150 2025-12-17 21:41:20,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408054575bc060ce15c76d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:20,628 - lava_callback - INFO - Received callback for node 694080ab575bc060ce15c8cc INFO: 172.21.0.1:33222 - "POST /node/694080ab575bc060ce15c8cc HTTP/1.0" 202 Accepted 2025-12-17 21:41:20,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805f575bc060ce15c79f 2025-12-17 21:41:20,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805f575bc060ce15c79f HTTP/1.1" 502 150 2025-12-17 21:41:20,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805f575bc060ce15c79f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:21,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805e575bc060ce15c792 2025-12-17 21:41:21,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805e575bc060ce15c792 HTTP/1.1" 502 150 2025-12-17 21:41:21,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805e575bc060ce15c792'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:21,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408062575bc060ce15c7d5 2025-12-17 21:41:21,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408062575bc060ce15c7d5 HTTP/1.1" 502 150 2025-12-17 21:41:21,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408062575bc060ce15c7d5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:22,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408048575bc060ce15c743 2025-12-17 21:41:22,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408048575bc060ce15c743 HTTP/1.1" 502 150 2025-12-17 21:41:22,471 - lava_callback - ERROR - Error processing callback for node 69408048575bc060ce15c743: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408048575bc060ce15c743 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408048575bc060ce15c743 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408048575bc060ce15c743 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:22,473 - lava_callback - INFO - Starting async job submit for node 69408063575bc060ce15c7dc 2025-12-17 21:41:22,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:22,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408063575bc060ce15c7dc HTTP/1.1" 502 150 2025-12-17 21:41:22,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408063575bc060ce15c7dc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:22,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408063575bc060ce15c7dc 2025-12-17 21:41:22,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408063575bc060ce15c7dc HTTP/1.1" 502 150 2025-12-17 21:41:22,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408063575bc060ce15c7dc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:22,507 - lava_callback - INFO - Received callback for node 6940806e575bc060ce15c7fa INFO: 172.21.0.1:33226 - "POST /node/6940806e575bc060ce15c7fa HTTP/1.0" 202 Accepted 2025-12-17 21:41:22,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804a575bc060ce15c74a 2025-12-17 21:41:22,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804a575bc060ce15c74a HTTP/1.1" 502 150 2025-12-17 21:41:22,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940804a575bc060ce15c74a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:23,168 - lava_callback - INFO - Received callback for node 6940806f575bc060ce15c805 INFO: 172.21.0.1:33228 - "POST /node/6940806f575bc060ce15c805 HTTP/1.0" 202 Accepted 2025-12-17 21:41:23,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804c575bc060ce15c757 2025-12-17 21:41:23,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804c575bc060ce15c757 HTTP/1.1" 502 150 2025-12-17 21:41:23,698 - lava_callback - ERROR - Error processing callback for node 6940804c575bc060ce15c757: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940804c575bc060ce15c757 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940804c575bc060ce15c757 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940804c575bc060ce15c757 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:23,700 - lava_callback - INFO - Starting async job submit for node 69408059575bc060ce15c77b 2025-12-17 21:41:23,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:23,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408059575bc060ce15c77b HTTP/1.1" 502 150 2025-12-17 21:41:23,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408059575bc060ce15c77b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:23,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408059575bc060ce15c77b 2025-12-17 21:41:23,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408059575bc060ce15c77b HTTP/1.1" 502 150 2025-12-17 21:41:23,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408059575bc060ce15c77b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:23,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804d575bc060ce15c75e 2025-12-17 21:41:23,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804d575bc060ce15c75e HTTP/1.1" 502 150 2025-12-17 21:41:23,809 - lava_callback - ERROR - Error processing callback for node 6940804d575bc060ce15c75e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940804d575bc060ce15c75e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940804d575bc060ce15c75e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940804d575bc060ce15c75e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:23,811 - lava_callback - INFO - Starting async job submit for node 6940805a575bc060ce15c784 2025-12-17 21:41:23,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:23,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805a575bc060ce15c784 HTTP/1.1" 502 150 2025-12-17 21:41:23,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805a575bc060ce15c784'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:23,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805a575bc060ce15c784 2025-12-17 21:41:23,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805a575bc060ce15c784 HTTP/1.1" 502 150 2025-12-17 21:41:23,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805a575bc060ce15c784'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:23,837 - lava_callback - INFO - Received callback for node 69408071575bc060ce15c80f INFO: 172.21.0.1:33244 - "POST /node/69408071575bc060ce15c80f HTTP/1.0" 202 Accepted 2025-12-17 21:41:24,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408063575bc060ce15c7dc 2025-12-17 21:41:24,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408063575bc060ce15c7dc HTTP/1.1" 502 150 2025-12-17 21:41:24,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408063575bc060ce15c7dc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:24,514 - lava_callback - INFO - Received callback for node 69408072575bc060ce15c812 INFO: 172.21.0.1:33256 - "POST /node/69408072575bc060ce15c812 HTTP/1.0" 202 Accepted 2025-12-17 21:41:24,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805f575bc060ce15c79f 2025-12-17 21:41:24,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805f575bc060ce15c79f HTTP/1.1" 502 150 2025-12-17 21:41:24,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805f575bc060ce15c79f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:25,184 - lava_callback - INFO - Received callback for node 69408073575bc060ce15c816 INFO: 172.21.0.1:33260 - "POST /node/69408073575bc060ce15c816 HTTP/1.0" 202 Accepted 2025-12-17 21:41:25,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408059575bc060ce15c77b 2025-12-17 21:41:25,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805a575bc060ce15c784 2025-12-17 21:41:25,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805a575bc060ce15c784 HTTP/1.1" 502 150 2025-12-17 21:41:25,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805a575bc060ce15c784'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:25,861 - lava_callback - INFO - Received callback for node 69408074575bc060ce15c819 INFO: 172.21.0.1:33270 - "POST /node/69408074575bc060ce15c819 HTTP/1.0" 202 Accepted 2025-12-17 21:41:25,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408062575bc060ce15c7d5 2025-12-17 21:41:25,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408062575bc060ce15c7d5 HTTP/1.1" 502 150 2025-12-17 21:41:25,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408062575bc060ce15c7d5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:26,533 - lava_callback - INFO - Received callback for node 69408076575bc060ce15c81c INFO: 172.21.0.1:33272 - "POST /node/69408076575bc060ce15c81c HTTP/1.0" 202 Accepted INFO: 172.21.0.1:33282 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:41:27,197 - lava_callback - INFO - Received callback for node 69408077575bc060ce15c81f INFO: 172.21.0.1:33292 - "POST /node/69408077575bc060ce15c81f HTTP/1.0" 202 Accepted 2025-12-17 21:41:27,863 - lava_callback - INFO - Received callback for node 69408078575bc060ce15c826 INFO: 172.21.0.1:33304 - "POST /node/69408078575bc060ce15c826 HTTP/1.0" 202 Accepted 2025-12-17 21:41:27,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408056575bc060ce15c773 2025-12-17 21:41:27,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408056575bc060ce15c773 HTTP/1.1" 502 150 2025-12-17 21:41:27,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408056575bc060ce15c773'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:27,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408057575bc060ce15c776 2025-12-17 21:41:27,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408057575bc060ce15c776 HTTP/1.1" 502 150 2025-12-17 21:41:27,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408057575bc060ce15c776'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:28,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408063575bc060ce15c7dc 2025-12-17 21:41:28,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408063575bc060ce15c7dc HTTP/1.1" 502 150 2025-12-17 21:41:28,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408063575bc060ce15c7dc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:28,523 - lava_callback - INFO - Received callback for node 69408079575bc060ce15c82d INFO: 172.21.0.1:33320 - "POST /node/69408079575bc060ce15c82d HTTP/1.0" 202 Accepted 2025-12-17 21:41:29,197 - lava_callback - INFO - Received callback for node 6940807b575bc060ce15c830 INFO: 172.21.0.1:33326 - "POST /node/6940807b575bc060ce15c830 HTTP/1.0" 202 Accepted 2025-12-17 21:41:29,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805a575bc060ce15c784 2025-12-17 21:41:29,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805a575bc060ce15c784 HTTP/1.1" 502 150 2025-12-17 21:41:29,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805a575bc060ce15c784'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:29,857 - lava_callback - INFO - Received callback for node 6940807e575bc060ce15c836 INFO: 172.21.0.1:42196 - "POST /node/6940807e575bc060ce15c836 HTTP/1.0" 202 Accepted 2025-12-17 21:41:29,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408055575bc060ce15c770 HTTP/1.1" 502 150 2025-12-17 21:41:29,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408055575bc060ce15c770'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:29,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805e575bc060ce15c792 2025-12-17 21:41:29,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805e575bc060ce15c792 HTTP/1.1" 502 150 2025-12-17 21:41:29,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805e575bc060ce15c792'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:32,447 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804f575bc060ce15c761 2025-12-17 21:41:32,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804f575bc060ce15c761 HTTP/1.1" 502 150 2025-12-17 21:41:32,449 - lava_callback - ERROR - Error processing callback for node 6940804f575bc060ce15c761: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940804f575bc060ce15c761 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940804f575bc060ce15c761 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940804f575bc060ce15c761 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:32,451 - lava_callback - INFO - Starting async job submit for node 6940805c575bc060ce15c78b 2025-12-17 21:41:32,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:32,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805c575bc060ce15c78b HTTP/1.1" 502 150 2025-12-17 21:41:32,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805c575bc060ce15c78b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:32,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805c575bc060ce15c78b 2025-12-17 21:41:32,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805c575bc060ce15c78b HTTP/1.1" 502 150 2025-12-17 21:41:32,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805c575bc060ce15c78b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:32,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805f575bc060ce15c79f 2025-12-17 21:41:32,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805f575bc060ce15c79f HTTP/1.1" 502 150 2025-12-17 21:41:32,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805f575bc060ce15c79f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:32,987 - lava_callback - INFO - Received callback for node 6940807c575bc060ce15c833 INFO: 172.21.0.1:42204 - "POST /node/6940807c575bc060ce15c833 HTTP/1.0" 202 Accepted 2025-12-17 21:41:33,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087bb575bc060ce15e4b1 2025-12-17 21:41:33,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087bb575bc060ce15e4b1 HTTP/1.1" 502 150 2025-12-17 21:41:33,013 - lava_callback - ERROR - Error processing callback for node 694087bb575bc060ce15e4b1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087bb575bc060ce15e4b1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087bb575bc060ce15e4b1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087bb575bc060ce15e4b1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:33,014 - lava_callback - INFO - Starting async job submit for node 69408066575bc060ce15c7e2 2025-12-17 21:41:33,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:33,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408066575bc060ce15c7e2 HTTP/1.1" 502 150 2025-12-17 21:41:33,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408066575bc060ce15c7e2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:33,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408066575bc060ce15c7e2 2025-12-17 21:41:33,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408066575bc060ce15c7e2 HTTP/1.1" 502 150 2025-12-17 21:41:33,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408066575bc060ce15c7e2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:33,135 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408050575bc060ce15c764 2025-12-17 21:41:33,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408050575bc060ce15c764 HTTP/1.1" 502 150 2025-12-17 21:41:33,137 - lava_callback - ERROR - Error processing callback for node 69408050575bc060ce15c764: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408050575bc060ce15c764 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408050575bc060ce15c764 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408050575bc060ce15c764 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:33,138 - lava_callback - INFO - Starting async job submit for node 69408067575bc060ce15c7e5 2025-12-17 21:41:33,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:33,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408067575bc060ce15c7e5 HTTP/1.1" 502 150 2025-12-17 21:41:33,161 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408067575bc060ce15c7e5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:33,161 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408067575bc060ce15c7e5 2025-12-17 21:41:33,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408067575bc060ce15c7e5 HTTP/1.1" 502 150 2025-12-17 21:41:33,162 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408067575bc060ce15c7e5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:33,651 - lava_callback - INFO - Received callback for node 69408080575bc060ce15c839 INFO: 172.21.0.1:42208 - "POST /node/69408080575bc060ce15c839 HTTP/1.0" 202 Accepted 2025-12-17 21:41:33,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408051575bc060ce15c767 2025-12-17 21:41:33,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408051575bc060ce15c767 HTTP/1.1" 502 150 2025-12-17 21:41:33,793 - lava_callback - ERROR - Error processing callback for node 69408051575bc060ce15c767: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408051575bc060ce15c767 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408051575bc060ce15c767 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408051575bc060ce15c767 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:33,795 - lava_callback - INFO - Starting async job submit for node 69408069575bc060ce15c7e8 2025-12-17 21:41:33,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:33,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408069575bc060ce15c7e8 HTTP/1.1" 502 150 2025-12-17 21:41:33,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408069575bc060ce15c7e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:33,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408069575bc060ce15c7e8 2025-12-17 21:41:33,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408069575bc060ce15c7e8 HTTP/1.1" 502 150 2025-12-17 21:41:33,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408069575bc060ce15c7e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:33,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408055575bc060ce15c770 2025-12-17 21:41:33,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408055575bc060ce15c770 HTTP/1.1" 502 150 2025-12-17 21:41:33,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408055575bc060ce15c770'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:33,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408062575bc060ce15c7d5 2025-12-17 21:41:33,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408062575bc060ce15c7d5 HTTP/1.1" 502 150 2025-12-17 21:41:33,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408062575bc060ce15c7d5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:34,321 - lava_callback - INFO - Received callback for node 69408083575bc060ce15c841 INFO: 172.21.0.1:42212 - "POST /node/69408083575bc060ce15c841 HTTP/1.0" 202 Accepted 2025-12-17 21:41:34,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408052575bc060ce15c76a 2025-12-17 21:41:34,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408052575bc060ce15c76a HTTP/1.1" 502 150 2025-12-17 21:41:34,452 - lava_callback - ERROR - Error processing callback for node 69408052575bc060ce15c76a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408052575bc060ce15c76a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408052575bc060ce15c76a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408052575bc060ce15c76a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:34,454 - lava_callback - INFO - Starting async job submit for node 6940806a575bc060ce15c7eb 2025-12-17 21:41:34,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:34,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805c575bc060ce15c78b 2025-12-17 21:41:34,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805c575bc060ce15c78b HTTP/1.1" 502 150 2025-12-17 21:41:34,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805c575bc060ce15c78b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:34,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806a575bc060ce15c7eb HTTP/1.1" 502 150 2025-12-17 21:41:34,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806a575bc060ce15c7eb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:34,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806a575bc060ce15c7eb 2025-12-17 21:41:34,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806a575bc060ce15c7eb HTTP/1.1" 502 150 2025-12-17 21:41:34,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806a575bc060ce15c7eb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:35,022 - lava_callback - INFO - Received callback for node 69408085575bc060ce15c84f INFO: 172.21.0.1:42228 - "POST /node/69408085575bc060ce15c84f HTTP/1.0" 202 Accepted 2025-12-17 21:41:35,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408066575bc060ce15c7e2 2025-12-17 21:41:35,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408066575bc060ce15c7e2 HTTP/1.1" 502 150 2025-12-17 21:41:35,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408066575bc060ce15c7e2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:35,162 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408067575bc060ce15c7e5 2025-12-17 21:41:35,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408067575bc060ce15c7e5 HTTP/1.1" 502 150 2025-12-17 21:41:35,164 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408067575bc060ce15c7e5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:35,690 - lava_callback - INFO - Received callback for node 69408086575bc060ce15c852 INFO: 172.21.0.1:42234 - "POST /node/69408086575bc060ce15c852 HTTP/1.0" 202 Accepted 2025-12-17 21:41:35,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408069575bc060ce15c7e8 2025-12-17 21:41:35,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408069575bc060ce15c7e8 HTTP/1.1" 502 150 2025-12-17 21:41:35,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408069575bc060ce15c7e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:36,361 - lava_callback - INFO - Received callback for node 69408088575bc060ce15c855 INFO: 172.21.0.1:42250 - "POST /node/69408088575bc060ce15c855 HTTP/1.0" 202 Accepted 2025-12-17 21:41:36,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806a575bc060ce15c7eb 2025-12-17 21:41:36,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806a575bc060ce15c7eb HTTP/1.1" 502 150 2025-12-17 21:41:36,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806a575bc060ce15c7eb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:36,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408063575bc060ce15c7dc 2025-12-17 21:41:36,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408063575bc060ce15c7dc HTTP/1.1" 502 150 2025-12-17 21:41:36,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408063575bc060ce15c7dc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42260 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:41:36,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408054575bc060ce15c76d 2025-12-17 21:41:36,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408054575bc060ce15c76d HTTP/1.1" 502 150 2025-12-17 21:41:36,569 - lava_callback - ERROR - Error processing callback for node 69408054575bc060ce15c76d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408054575bc060ce15c76d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408054575bc060ce15c76d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408054575bc060ce15c76d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:36,571 - lava_callback - INFO - Starting async job submit for node 6940806b575bc060ce15c7ee 2025-12-17 21:41:36,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:36,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806b575bc060ce15c7ee HTTP/1.1" 502 150 2025-12-17 21:41:36,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806b575bc060ce15c7ee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:36,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806b575bc060ce15c7ee 2025-12-17 21:41:36,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806b575bc060ce15c7ee HTTP/1.1" 502 150 2025-12-17 21:41:36,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806b575bc060ce15c7ee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:37,037 - lava_callback - INFO - Received callback for node 69408089575bc060ce15c858 INFO: 172.21.0.1:42272 - "POST /node/69408089575bc060ce15c858 HTTP/1.0" 202 Accepted 2025-12-17 21:41:37,736 - lava_callback - INFO - Received callback for node 6940808b575bc060ce15c85b INFO: 172.21.0.1:42278 - "POST /node/6940808b575bc060ce15c85b HTTP/1.0" 202 Accepted 2025-12-17 21:41:37,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805a575bc060ce15c784 2025-12-17 21:41:37,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805a575bc060ce15c784 HTTP/1.1" 502 150 2025-12-17 21:41:37,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805a575bc060ce15c784'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:38,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805c575bc060ce15c78b 2025-12-17 21:41:38,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805c575bc060ce15c78b HTTP/1.1" 502 150 2025-12-17 21:41:38,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805c575bc060ce15c78b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:38,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940804a575bc060ce15c74a 2025-12-17 21:41:38,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940804a575bc060ce15c74a HTTP/1.1" 502 150 2025-12-17 21:41:38,553 - lava_callback - ERROR - Error processing callback for node 6940804a575bc060ce15c74a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940804a575bc060ce15c74a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940804a575bc060ce15c74a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940804a575bc060ce15c74a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:38,555 - lava_callback - INFO - Starting async job submit for node 6940806d575bc060ce15c7f3 2025-12-17 21:41:38,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:38,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806d575bc060ce15c7f3 HTTP/1.1" 502 150 2025-12-17 21:41:38,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806d575bc060ce15c7f3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:38,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806d575bc060ce15c7f3 2025-12-17 21:41:38,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806d575bc060ce15c7f3 HTTP/1.1" 502 150 2025-12-17 21:41:38,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806d575bc060ce15c7f3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:38,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806b575bc060ce15c7ee 2025-12-17 21:41:38,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806b575bc060ce15c7ee HTTP/1.1" 502 150 2025-12-17 21:41:38,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806b575bc060ce15c7ee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:39,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408066575bc060ce15c7e2 2025-12-17 21:41:39,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408066575bc060ce15c7e2 HTTP/1.1" 502 150 2025-12-17 21:41:39,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408066575bc060ce15c7e2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:39,164 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408067575bc060ce15c7e5 2025-12-17 21:41:39,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408067575bc060ce15c7e5 HTTP/1.1" 502 150 2025-12-17 21:41:39,165 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408067575bc060ce15c7e5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:39,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408069575bc060ce15c7e8 2025-12-17 21:41:39,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408069575bc060ce15c7e8 HTTP/1.1" 502 150 2025-12-17 21:41:39,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408069575bc060ce15c7e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:40,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806a575bc060ce15c7eb 2025-12-17 21:41:40,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806d575bc060ce15c7f3 2025-12-17 21:41:40,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806d575bc060ce15c7f3 HTTP/1.1" 502 150 2025-12-17 21:41:40,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806d575bc060ce15c7f3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:41,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408059575bc060ce15c77b HTTP/1.1" 502 150 2025-12-17 21:41:41,168 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408059575bc060ce15c77b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:41,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408055575bc060ce15c770 2025-12-17 21:41:41,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408055575bc060ce15c770 HTTP/1.1" 502 150 2025-12-17 21:41:41,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408055575bc060ce15c770'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:42,487 - lava_callback - INFO - Received callback for node 69408081575bc060ce15c83c INFO: 172.21.0.1:50738 - "POST /node/69408081575bc060ce15c83c HTTP/1.0" 202 Accepted 2025-12-17 21:41:42,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806b575bc060ce15c7ee 2025-12-17 21:41:42,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806b575bc060ce15c7ee HTTP/1.1" 502 150 2025-12-17 21:41:42,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806b575bc060ce15c7ee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:43,145 - lava_callback - INFO - Received callback for node 69408084575bc060ce15c848 INFO: 172.21.0.1:50742 - "POST /node/69408084575bc060ce15c848 HTTP/1.0" 202 Accepted 2025-12-17 21:41:43,783 - lava_callback - INFO - Received callback for node 69408095575bc060ce15c863 INFO: 172.21.0.1:50746 - "POST /node/69408095575bc060ce15c863 HTTP/1.0" 202 Accepted 2025-12-17 21:41:43,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408056575bc060ce15c773 2025-12-17 21:41:43,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408056575bc060ce15c773 HTTP/1.1" 502 150 2025-12-17 21:41:43,895 - lava_callback - ERROR - Error processing callback for node 69408056575bc060ce15c773: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408056575bc060ce15c773 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408056575bc060ce15c773 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408056575bc060ce15c773 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:43,897 - lava_callback - INFO - Starting async job submit for node 69408065575bc060ce15c7df 2025-12-17 21:41:43,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:43,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408065575bc060ce15c7df HTTP/1.1" 502 150 2025-12-17 21:41:43,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408065575bc060ce15c7df'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:43,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408065575bc060ce15c7df 2025-12-17 21:41:43,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408065575bc060ce15c7df HTTP/1.1" 502 150 2025-12-17 21:41:43,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408065575bc060ce15c7df'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:43,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408057575bc060ce15c776 2025-12-17 21:41:43,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408057575bc060ce15c776 HTTP/1.1" 502 150 2025-12-17 21:41:43,938 - lava_callback - ERROR - Error processing callback for node 69408057575bc060ce15c776: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408057575bc060ce15c776 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408057575bc060ce15c776 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408057575bc060ce15c776 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:43,940 - lava_callback - INFO - Starting async job submit for node 69408eb6575bc060ce15ff43 2025-12-17 21:41:44,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:44,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb6575bc060ce15ff43 HTTP/1.1" 502 150 2025-12-17 21:41:44,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb6575bc060ce15ff43'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:44,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb6575bc060ce15ff43 2025-12-17 21:41:44,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb6575bc060ce15ff43 HTTP/1.1" 502 150 2025-12-17 21:41:44,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb6575bc060ce15ff43'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:44,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806d575bc060ce15c7f3 2025-12-17 21:41:44,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806d575bc060ce15c7f3 HTTP/1.1" 502 150 2025-12-17 21:41:44,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806d575bc060ce15c7f3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:45,169 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408059575bc060ce15c77b 2025-12-17 21:41:45,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408065575bc060ce15c7df 2025-12-17 21:41:45,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408065575bc060ce15c7df HTTP/1.1" 502 150 2025-12-17 21:41:45,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408065575bc060ce15c7df'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:45,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805e575bc060ce15c792 2025-12-17 21:41:45,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805e575bc060ce15c792 HTTP/1.1" 502 150 2025-12-17 21:41:45,923 - lava_callback - ERROR - Error processing callback for node 6940805e575bc060ce15c792: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940805e575bc060ce15c792 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940805e575bc060ce15c792 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940805e575bc060ce15c792 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:45,926 - lava_callback - INFO - Starting async job submit for node 694080ab575bc060ce15c8cc 2025-12-17 21:41:45,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:45,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ab575bc060ce15c8cc HTTP/1.1" 502 150 2025-12-17 21:41:45,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ab575bc060ce15c8cc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:45,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ab575bc060ce15c8cc 2025-12-17 21:41:45,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ab575bc060ce15c8cc HTTP/1.1" 502 150 2025-12-17 21:41:45,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ab575bc060ce15c8cc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:46,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb6575bc060ce15ff43 2025-12-17 21:41:46,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb6575bc060ce15ff43 HTTP/1.1" 502 150 2025-12-17 21:41:46,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb6575bc060ce15ff43'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:46,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805c575bc060ce15c78b 2025-12-17 21:41:46,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805c575bc060ce15c78b HTTP/1.1" 502 150 2025-12-17 21:41:46,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940805c575bc060ce15c78b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50754 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:41:47,037 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408066575bc060ce15c7e2 2025-12-17 21:41:47,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408066575bc060ce15c7e2 HTTP/1.1" 502 150 2025-12-17 21:41:47,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408066575bc060ce15c7e2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:47,166 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408067575bc060ce15c7e5 2025-12-17 21:41:47,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408067575bc060ce15c7e5 HTTP/1.1" 502 150 2025-12-17 21:41:47,167 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408067575bc060ce15c7e5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:47,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408069575bc060ce15c7e8 2025-12-17 21:41:47,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408069575bc060ce15c7e8 HTTP/1.1" 502 150 2025-12-17 21:41:47,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408069575bc060ce15c7e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:47,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ab575bc060ce15c8cc 2025-12-17 21:41:47,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ab575bc060ce15c8cc HTTP/1.1" 502 150 2025-12-17 21:41:47,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ab575bc060ce15c8cc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:48,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805f575bc060ce15c79f 2025-12-17 21:41:48,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805f575bc060ce15c79f HTTP/1.1" 502 150 2025-12-17 21:41:48,698 - lava_callback - ERROR - Error processing callback for node 6940805f575bc060ce15c79f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940805f575bc060ce15c79f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940805f575bc060ce15c79f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940805f575bc060ce15c79f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:48,701 - lava_callback - INFO - Starting async job submit for node 6940806e575bc060ce15c7fa 2025-12-17 21:41:48,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:48,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806e575bc060ce15c7fa HTTP/1.1" 502 150 2025-12-17 21:41:48,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806e575bc060ce15c7fa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:48,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806e575bc060ce15c7fa 2025-12-17 21:41:48,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806e575bc060ce15c7fa HTTP/1.1" 502 150 2025-12-17 21:41:48,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806e575bc060ce15c7fa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:49,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408065575bc060ce15c7df 2025-12-17 21:41:49,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408065575bc060ce15c7df HTTP/1.1" 502 150 2025-12-17 21:41:49,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408065575bc060ce15c7df'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:49,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408062575bc060ce15c7d5 2025-12-17 21:41:49,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408062575bc060ce15c7d5 HTTP/1.1" 502 150 2025-12-17 21:41:49,923 - lava_callback - ERROR - Error processing callback for node 69408062575bc060ce15c7d5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408062575bc060ce15c7d5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408062575bc060ce15c7d5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408062575bc060ce15c7d5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:49,925 - lava_callback - INFO - Starting async job submit for node 6940806f575bc060ce15c805 2025-12-17 21:41:49,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:49,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806f575bc060ce15c805 HTTP/1.1" 502 150 2025-12-17 21:41:49,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806f575bc060ce15c805'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:49,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806f575bc060ce15c805 2025-12-17 21:41:49,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806f575bc060ce15c805 HTTP/1.1" 502 150 2025-12-17 21:41:49,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806f575bc060ce15c805'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:50,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb6575bc060ce15ff43 2025-12-17 21:41:50,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb6575bc060ce15ff43 HTTP/1.1" 502 150 2025-12-17 21:41:50,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb6575bc060ce15ff43'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:50,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806b575bc060ce15c7ee 2025-12-17 21:41:50,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806b575bc060ce15c7ee HTTP/1.1" 502 150 2025-12-17 21:41:50,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806b575bc060ce15c7ee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:50,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806e575bc060ce15c7fa 2025-12-17 21:41:50,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806e575bc060ce15c7fa HTTP/1.1" 502 150 2025-12-17 21:41:50,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806e575bc060ce15c7fa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:51,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806f575bc060ce15c805 2025-12-17 21:41:51,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806f575bc060ce15c805 HTTP/1.1" 502 150 2025-12-17 21:41:51,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806f575bc060ce15c805'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:51,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ab575bc060ce15c8cc 2025-12-17 21:41:51,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ab575bc060ce15c8cc HTTP/1.1" 502 150 2025-12-17 21:41:51,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ab575bc060ce15c8cc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:52,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408063575bc060ce15c7dc 2025-12-17 21:41:52,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806d575bc060ce15c7f3 2025-12-17 21:41:52,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806d575bc060ce15c7f3 HTTP/1.1" 502 150 2025-12-17 21:41:52,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806d575bc060ce15c7f3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:52,955 - lava_callback - INFO - Received callback for node 6940808c575bc060ce15c85e INFO: 172.21.0.1:35856 - "POST /node/6940808c575bc060ce15c85e HTTP/1.0" 202 Accepted 2025-12-17 21:41:53,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805a575bc060ce15c784 2025-12-17 21:41:53,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805a575bc060ce15c784 HTTP/1.1" 502 150 2025-12-17 21:41:53,825 - lava_callback - ERROR - Error processing callback for node 6940805a575bc060ce15c784: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940805a575bc060ce15c784 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940805a575bc060ce15c784 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940805a575bc060ce15c784 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:53,827 - lava_callback - INFO - Starting async job submit for node 69408071575bc060ce15c80f 2025-12-17 21:41:53,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:53,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408071575bc060ce15c80f HTTP/1.1" 502 150 2025-12-17 21:41:53,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408071575bc060ce15c80f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:53,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408071575bc060ce15c80f 2025-12-17 21:41:53,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408071575bc060ce15c80f HTTP/1.1" 502 150 2025-12-17 21:41:53,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408071575bc060ce15c80f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:54,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806e575bc060ce15c7fa 2025-12-17 21:41:54,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806e575bc060ce15c7fa HTTP/1.1" 502 150 2025-12-17 21:41:54,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806e575bc060ce15c7fa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:55,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806a575bc060ce15c7eb HTTP/1.1" 502 150 2025-12-17 21:41:55,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806a575bc060ce15c7eb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:55,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408071575bc060ce15c80f 2025-12-17 21:41:55,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408071575bc060ce15c80f HTTP/1.1" 502 150 2025-12-17 21:41:55,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408071575bc060ce15c80f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:55,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806f575bc060ce15c805 2025-12-17 21:41:55,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806f575bc060ce15c805 HTTP/1.1" 502 150 2025-12-17 21:41:55,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806f575bc060ce15c805'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35870 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:41:57,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408055575bc060ce15c770 2025-12-17 21:41:57,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408055575bc060ce15c770 HTTP/1.1" 502 150 2025-12-17 21:41:57,909 - lava_callback - ERROR - Error processing callback for node 69408055575bc060ce15c770: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408055575bc060ce15c770 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408055575bc060ce15c770 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408055575bc060ce15c770 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:41:57,912 - lava_callback - INFO - Starting async job submit for node 69408072575bc060ce15c812 2025-12-17 21:41:57,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:41:57,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408065575bc060ce15c7df 2025-12-17 21:41:57,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408065575bc060ce15c7df HTTP/1.1" 502 150 2025-12-17 21:41:57,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408065575bc060ce15c7df'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:57,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408072575bc060ce15c812 HTTP/1.1" 502 150 2025-12-17 21:41:57,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408072575bc060ce15c812'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:57,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408072575bc060ce15c812 2025-12-17 21:41:57,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408072575bc060ce15c812 HTTP/1.1" 502 150 2025-12-17 21:41:57,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408072575bc060ce15c812'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:58,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb6575bc060ce15ff43 2025-12-17 21:41:58,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb6575bc060ce15ff43 HTTP/1.1" 502 150 2025-12-17 21:41:58,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb6575bc060ce15ff43'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:59,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408071575bc060ce15c80f 2025-12-17 21:41:59,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408071575bc060ce15c80f HTTP/1.1" 502 150 2025-12-17 21:41:59,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408071575bc060ce15c80f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:59,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408072575bc060ce15c812 2025-12-17 21:41:59,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408072575bc060ce15c812 HTTP/1.1" 502 150 2025-12-17 21:41:59,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408072575bc060ce15c812'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:41:59,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ab575bc060ce15c8cc 2025-12-17 21:41:59,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ab575bc060ce15c8cc HTTP/1.1" 502 150 2025-12-17 21:41:59,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ab575bc060ce15c8cc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:00,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408059575bc060ce15c77b HTTP/1.1" 502 150 2025-12-17 21:42:00,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408059575bc060ce15c77b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42828 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:42:02,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940805c575bc060ce15c78b 2025-12-17 21:42:02,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940805c575bc060ce15c78b HTTP/1.1" 502 150 2025-12-17 21:42:02,464 - lava_callback - ERROR - Error processing callback for node 6940805c575bc060ce15c78b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940805c575bc060ce15c78b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940805c575bc060ce15c78b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940805c575bc060ce15c78b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:02,466 - lava_callback - INFO - Starting async job submit for node 69408073575bc060ce15c816 2025-12-17 21:42:02,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:02,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408073575bc060ce15c816 HTTP/1.1" 502 150 2025-12-17 21:42:02,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408073575bc060ce15c816'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:02,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408073575bc060ce15c816 2025-12-17 21:42:02,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408073575bc060ce15c816 HTTP/1.1" 502 150 2025-12-17 21:42:02,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408073575bc060ce15c816'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:02,535 - lava_callback - INFO - Received callback for node 69408096575bc060ce15c86b INFO: 172.21.0.1:42840 - "POST /node/69408096575bc060ce15c86b HTTP/1.0" 202 Accepted 2025-12-17 21:42:02,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806e575bc060ce15c7fa 2025-12-17 21:42:02,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806e575bc060ce15c7fa HTTP/1.1" 502 150 2025-12-17 21:42:02,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806e575bc060ce15c7fa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:03,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408066575bc060ce15c7e2 2025-12-17 21:42:03,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408066575bc060ce15c7e2 HTTP/1.1" 502 150 2025-12-17 21:42:03,040 - lava_callback - ERROR - Error processing callback for node 69408066575bc060ce15c7e2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408066575bc060ce15c7e2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408066575bc060ce15c7e2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408066575bc060ce15c7e2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:03,043 - lava_callback - INFO - Starting async job submit for node 69408074575bc060ce15c819 2025-12-17 21:42:03,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:03,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408074575bc060ce15c819 HTTP/1.1" 502 150 2025-12-17 21:42:03,050 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408074575bc060ce15c819'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:03,050 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408074575bc060ce15c819 2025-12-17 21:42:03,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408074575bc060ce15c819 HTTP/1.1" 502 150 2025-12-17 21:42:03,050 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408074575bc060ce15c819'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:03,167 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408067575bc060ce15c7e5 2025-12-17 21:42:03,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408067575bc060ce15c7e5 HTTP/1.1" 502 150 2025-12-17 21:42:03,170 - lava_callback - ERROR - Error processing callback for node 69408067575bc060ce15c7e5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408067575bc060ce15c7e5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408067575bc060ce15c7e5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408067575bc060ce15c7e5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:03,172 - lava_callback - INFO - Starting async job submit for node 69408076575bc060ce15c81c 2025-12-17 21:42:03,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:03,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408076575bc060ce15c81c HTTP/1.1" 502 150 2025-12-17 21:42:03,179 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408076575bc060ce15c81c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:03,179 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408076575bc060ce15c81c 2025-12-17 21:42:03,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408076575bc060ce15c81c HTTP/1.1" 502 150 2025-12-17 21:42:03,180 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408076575bc060ce15c81c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:03,217 - lava_callback - INFO - Received callback for node 69408098575bc060ce15c870 INFO: 172.21.0.1:42848 - "POST /node/69408098575bc060ce15c870 HTTP/1.0" 202 Accepted 2025-12-17 21:42:03,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806a575bc060ce15c7eb 2025-12-17 21:42:03,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806a575bc060ce15c7eb HTTP/1.1" 502 150 2025-12-17 21:42:03,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806a575bc060ce15c7eb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:03,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408069575bc060ce15c7e8 2025-12-17 21:42:03,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408069575bc060ce15c7e8 HTTP/1.1" 502 150 2025-12-17 21:42:03,821 - lava_callback - ERROR - Error processing callback for node 69408069575bc060ce15c7e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408069575bc060ce15c7e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408069575bc060ce15c7e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408069575bc060ce15c7e8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:03,823 - lava_callback - INFO - Starting async job submit for node 69408077575bc060ce15c81f 2025-12-17 21:42:03,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:03,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408077575bc060ce15c81f HTTP/1.1" 502 150 2025-12-17 21:42:03,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408077575bc060ce15c81f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:03,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408077575bc060ce15c81f 2025-12-17 21:42:03,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408077575bc060ce15c81f HTTP/1.1" 502 150 2025-12-17 21:42:03,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408077575bc060ce15c81f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:03,862 - lava_callback - INFO - Received callback for node 6940809e575bc060ce15c883 INFO: 172.21.0.1:42856 - "POST /node/6940809e575bc060ce15c883 HTTP/1.0" 202 Accepted 2025-12-17 21:42:03,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408072575bc060ce15c812 2025-12-17 21:42:03,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408072575bc060ce15c812 HTTP/1.1" 502 150 2025-12-17 21:42:03,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408072575bc060ce15c812'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:03,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806f575bc060ce15c805 2025-12-17 21:42:03,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806f575bc060ce15c805 HTTP/1.1" 502 150 2025-12-17 21:42:03,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940806f575bc060ce15c805'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:04,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408073575bc060ce15c816 2025-12-17 21:42:04,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408073575bc060ce15c816 HTTP/1.1" 502 150 2025-12-17 21:42:04,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408073575bc060ce15c816'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:04,520 - lava_callback - INFO - Received callback for node 694080a0575bc060ce15c8ad INFO: 172.21.0.1:42872 - "POST /node/694080a0575bc060ce15c8ad HTTP/1.0" 202 Accepted 2025-12-17 21:42:05,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408074575bc060ce15c819 2025-12-17 21:42:05,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408074575bc060ce15c819 HTTP/1.1" 502 150 2025-12-17 21:42:05,052 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408074575bc060ce15c819'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:05,180 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408076575bc060ce15c81c 2025-12-17 21:42:05,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408076575bc060ce15c81c HTTP/1.1" 502 150 2025-12-17 21:42:05,181 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408076575bc060ce15c81c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:05,207 - lava_callback - INFO - Received callback for node 694080a2575bc060ce15c8b0 INFO: 172.21.0.1:42878 - "POST /node/694080a2575bc060ce15c8b0 HTTP/1.0" 202 Accepted 2025-12-17 21:42:05,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408077575bc060ce15c81f 2025-12-17 21:42:05,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408077575bc060ce15c81f HTTP/1.1" 502 150 2025-12-17 21:42:05,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408077575bc060ce15c81f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:05,902 - lava_callback - INFO - Received callback for node 694080a3575bc060ce15c8b3 INFO: 172.21.0.1:42880 - "POST /node/694080a3575bc060ce15c8b3 HTTP/1.0" 202 Accepted 2025-12-17 21:42:06,103 - lava_callback - INFO - Received callback for node 694087c7575bc060ce15e4bd INFO: 172.21.0.1:42894 - "POST /node/694087c7575bc060ce15e4bd HTTP/1.0" 202 Accepted INFO: 172.21.0.1:42902 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:42:06,576 - lava_callback - INFO - Received callback for node 694080a4575bc060ce15c8b6 INFO: 172.21.0.1:42918 - "POST /node/694080a4575bc060ce15c8b6 HTTP/1.0" 202 Accepted 2025-12-17 21:42:06,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806b575bc060ce15c7ee 2025-12-17 21:42:06,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806b575bc060ce15c7ee HTTP/1.1" 502 150 2025-12-17 21:42:06,607 - lava_callback - ERROR - Error processing callback for node 6940806b575bc060ce15c7ee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940806b575bc060ce15c7ee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940806b575bc060ce15c7ee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940806b575bc060ce15c7ee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:06,609 - lava_callback - INFO - Starting async job submit for node 69408078575bc060ce15c826 2025-12-17 21:42:06,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:06,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408078575bc060ce15c826 HTTP/1.1" 502 150 2025-12-17 21:42:06,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408078575bc060ce15c826'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:06,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408078575bc060ce15c826 2025-12-17 21:42:06,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408078575bc060ce15c826 HTTP/1.1" 502 150 2025-12-17 21:42:06,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408078575bc060ce15c826'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:07,250 - lava_callback - INFO - Received callback for node 694080a5575bc060ce15c8b9 INFO: 172.21.0.1:42932 - "POST /node/694080a5575bc060ce15c8b9 HTTP/1.0" 202 Accepted 2025-12-17 21:42:07,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408063575bc060ce15c7dc HTTP/1.1" 502 150 2025-12-17 21:42:07,793 - lava_callback - ERROR - Error processing callback for node 69408063575bc060ce15c7dc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408063575bc060ce15c7dc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408063575bc060ce15c7dc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408063575bc060ce15c7dc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:07,795 - lava_callback - INFO - Starting async job submit for node 69408079575bc060ce15c82d 2025-12-17 21:42:07,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:07,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408079575bc060ce15c82d HTTP/1.1" 502 150 2025-12-17 21:42:07,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408079575bc060ce15c82d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:07,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408079575bc060ce15c82d 2025-12-17 21:42:07,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408079575bc060ce15c82d HTTP/1.1" 502 150 2025-12-17 21:42:07,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408079575bc060ce15c82d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:07,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408071575bc060ce15c80f 2025-12-17 21:42:07,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408071575bc060ce15c80f HTTP/1.1" 502 150 2025-12-17 21:42:07,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408071575bc060ce15c80f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:07,948 - lava_callback - INFO - Received callback for node 694080a7575bc060ce15c8bc INFO: 172.21.0.1:42934 - "POST /node/694080a7575bc060ce15c8bc HTTP/1.0" 202 Accepted 2025-12-17 21:42:08,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408073575bc060ce15c816 2025-12-17 21:42:08,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408073575bc060ce15c816 HTTP/1.1" 502 150 2025-12-17 21:42:08,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408073575bc060ce15c816'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:08,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806d575bc060ce15c7f3 2025-12-17 21:42:08,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806d575bc060ce15c7f3 HTTP/1.1" 502 150 2025-12-17 21:42:08,569 - lava_callback - ERROR - Error processing callback for node 6940806d575bc060ce15c7f3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940806d575bc060ce15c7f3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940806d575bc060ce15c7f3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940806d575bc060ce15c7f3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:08,571 - lava_callback - INFO - Starting async job submit for node 6940807b575bc060ce15c830 2025-12-17 21:42:08,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:08,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807b575bc060ce15c830 HTTP/1.1" 502 150 2025-12-17 21:42:08,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940807b575bc060ce15c830'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:08,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940807b575bc060ce15c830 2025-12-17 21:42:08,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807b575bc060ce15c830 HTTP/1.1" 502 150 2025-12-17 21:42:08,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940807b575bc060ce15c830'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:08,613 - lava_callback - INFO - Received callback for node 694080a8575bc060ce15c8bf INFO: 172.21.0.1:42950 - "POST /node/694080a8575bc060ce15c8bf HTTP/1.0" 202 Accepted 2025-12-17 21:42:08,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408059575bc060ce15c77b 2025-12-17 21:42:08,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408059575bc060ce15c77b HTTP/1.1" 502 150 2025-12-17 21:42:08,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408059575bc060ce15c77b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:08,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408078575bc060ce15c826 2025-12-17 21:42:08,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408078575bc060ce15c826 HTTP/1.1" 502 150 2025-12-17 21:42:08,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408078575bc060ce15c826'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:09,052 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408074575bc060ce15c819 2025-12-17 21:42:09,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408074575bc060ce15c819 HTTP/1.1" 502 150 2025-12-17 21:42:09,054 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408074575bc060ce15c819'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:09,182 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408076575bc060ce15c81c 2025-12-17 21:42:09,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408076575bc060ce15c81c HTTP/1.1" 502 150 2025-12-17 21:42:09,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408076575bc060ce15c81c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:09,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408079575bc060ce15c82d 2025-12-17 21:42:09,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408079575bc060ce15c82d HTTP/1.1" 502 150 2025-12-17 21:42:09,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408079575bc060ce15c82d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:09,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408077575bc060ce15c81f 2025-12-17 21:42:09,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408077575bc060ce15c81f HTTP/1.1" 502 150 2025-12-17 21:42:09,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408077575bc060ce15c81f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:10,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940807b575bc060ce15c830 2025-12-17 21:42:10,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807b575bc060ce15c830 HTTP/1.1" 502 150 2025-12-17 21:42:10,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940807b575bc060ce15c830'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:11,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408072575bc060ce15c812 2025-12-17 21:42:12,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408078575bc060ce15c826 2025-12-17 21:42:12,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408078575bc060ce15c826 HTTP/1.1" 502 150 2025-12-17 21:42:12,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408078575bc060ce15c826'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:13,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408079575bc060ce15c82d 2025-12-17 21:42:13,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408079575bc060ce15c82d HTTP/1.1" 502 150 2025-12-17 21:42:13,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408079575bc060ce15c82d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:13,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408065575bc060ce15c7df 2025-12-17 21:42:13,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408065575bc060ce15c7df HTTP/1.1" 502 150 2025-12-17 21:42:13,916 - lava_callback - ERROR - Error processing callback for node 69408065575bc060ce15c7df: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408065575bc060ce15c7df (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408065575bc060ce15c7df (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408065575bc060ce15c7df (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:13,918 - lava_callback - INFO - Starting async job submit for node 6940807e575bc060ce15c836 2025-12-17 21:42:13,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:13,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807e575bc060ce15c836 HTTP/1.1" 502 150 2025-12-17 21:42:13,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940807e575bc060ce15c836'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:13,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940807e575bc060ce15c836 2025-12-17 21:42:13,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807e575bc060ce15c836 HTTP/1.1" 502 150 2025-12-17 21:42:13,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940807e575bc060ce15c836'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:14,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb6575bc060ce15ff43 2025-12-17 21:42:14,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb6575bc060ce15ff43 HTTP/1.1" 502 150 2025-12-17 21:42:14,026 - lava_callback - ERROR - Error processing callback for node 69408eb6575bc060ce15ff43: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb6575bc060ce15ff43 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb6575bc060ce15ff43 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb6575bc060ce15ff43 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:14,028 - lava_callback - INFO - Starting async job submit for node 6940807c575bc060ce15c833 2025-12-17 21:42:14,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:14,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807c575bc060ce15c833 HTTP/1.1" 502 150 2025-12-17 21:42:14,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940807c575bc060ce15c833'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:14,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940807c575bc060ce15c833 2025-12-17 21:42:14,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807c575bc060ce15c833 HTTP/1.1" 502 150 2025-12-17 21:42:14,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940807c575bc060ce15c833'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:14,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940807b575bc060ce15c830 2025-12-17 21:42:14,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807b575bc060ce15c830 HTTP/1.1" 502 150 2025-12-17 21:42:14,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940807b575bc060ce15c830'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53112 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:42:15,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940807e575bc060ce15c836 2025-12-17 21:42:15,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807e575bc060ce15c836 HTTP/1.1" 502 150 2025-12-17 21:42:15,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940807e575bc060ce15c836'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:15,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ab575bc060ce15c8cc 2025-12-17 21:42:15,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ab575bc060ce15c8cc HTTP/1.1" 502 150 2025-12-17 21:42:15,976 - lava_callback - ERROR - Error processing callback for node 694080ab575bc060ce15c8cc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ab575bc060ce15c8cc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ab575bc060ce15c8cc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ab575bc060ce15c8cc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:15,978 - lava_callback - INFO - Starting async job submit for node 69408080575bc060ce15c839 2025-12-17 21:42:15,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:15,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408080575bc060ce15c839 HTTP/1.1" 502 150 2025-12-17 21:42:15,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408080575bc060ce15c839'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:15,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408080575bc060ce15c839 2025-12-17 21:42:15,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408080575bc060ce15c839 HTTP/1.1" 502 150 2025-12-17 21:42:15,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408080575bc060ce15c839'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:16,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940807c575bc060ce15c833 2025-12-17 21:42:16,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807c575bc060ce15c833 HTTP/1.1" 502 150 2025-12-17 21:42:16,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940807c575bc060ce15c833'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:16,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408073575bc060ce15c816 2025-12-17 21:42:16,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408073575bc060ce15c816 HTTP/1.1" 502 150 2025-12-17 21:42:16,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408073575bc060ce15c816'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53116 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:42:17,054 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408074575bc060ce15c819 2025-12-17 21:42:17,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408074575bc060ce15c819 HTTP/1.1" 502 150 2025-12-17 21:42:17,055 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408074575bc060ce15c819'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:17,183 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408076575bc060ce15c81c 2025-12-17 21:42:17,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408076575bc060ce15c81c HTTP/1.1" 502 150 2025-12-17 21:42:17,184 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408076575bc060ce15c81c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:17,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408077575bc060ce15c81f 2025-12-17 21:42:17,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408077575bc060ce15c81f HTTP/1.1" 502 150 2025-12-17 21:42:17,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408077575bc060ce15c81f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:17,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408080575bc060ce15c839 2025-12-17 21:42:17,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408080575bc060ce15c839 HTTP/1.1" 502 150 2025-12-17 21:42:17,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408080575bc060ce15c839'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:18,096 - lava_callback - INFO - Received callback for node 694080b3575bc060ce15c8df INFO: 172.21.0.1:53130 - "POST /node/694080b3575bc060ce15c8df HTTP/1.0" 202 Accepted 2025-12-17 21:42:18,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806e575bc060ce15c7fa 2025-12-17 21:42:18,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806e575bc060ce15c7fa HTTP/1.1" 502 150 2025-12-17 21:42:18,719 - lava_callback - ERROR - Error processing callback for node 6940806e575bc060ce15c7fa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940806e575bc060ce15c7fa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940806e575bc060ce15c7fa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940806e575bc060ce15c7fa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:18,721 - lava_callback - INFO - Starting async job submit for node 69408083575bc060ce15c841 2025-12-17 21:42:18,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:18,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408083575bc060ce15c841 HTTP/1.1" 502 150 2025-12-17 21:42:18,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408083575bc060ce15c841'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:18,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408083575bc060ce15c841 2025-12-17 21:42:18,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408083575bc060ce15c841 HTTP/1.1" 502 150 2025-12-17 21:42:18,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408083575bc060ce15c841'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:18,885 - lava_callback - INFO - Received callback for node 694080b5575bc060ce15c8e2 INFO: 172.21.0.1:53142 - "POST /node/694080b5575bc060ce15c8e2 HTTP/1.0" 202 Accepted 2025-12-17 21:42:19,645 - lava_callback - INFO - Received callback for node 694080b6575bc060ce15c8e5 INFO: 172.21.0.1:53144 - "POST /node/694080b6575bc060ce15c8e5 HTTP/1.0" 202 Accepted 2025-12-17 21:42:19,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806a575bc060ce15c7eb 2025-12-17 21:42:19,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806a575bc060ce15c7eb HTTP/1.1" 502 150 2025-12-17 21:42:19,769 - lava_callback - ERROR - Error processing callback for node 6940806a575bc060ce15c7eb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940806a575bc060ce15c7eb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940806a575bc060ce15c7eb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940806a575bc060ce15c7eb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:19,773 - lava_callback - INFO - Starting async job submit for node 69408085575bc060ce15c84f 2025-12-17 21:42:19,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:19,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408085575bc060ce15c84f HTTP/1.1" 502 150 2025-12-17 21:42:19,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408085575bc060ce15c84f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:19,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408085575bc060ce15c84f 2025-12-17 21:42:19,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408085575bc060ce15c84f HTTP/1.1" 502 150 2025-12-17 21:42:19,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408085575bc060ce15c84f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:19,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940807e575bc060ce15c836 2025-12-17 21:42:19,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807e575bc060ce15c836 HTTP/1.1" 502 150 2025-12-17 21:42:19,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940807e575bc060ce15c836'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:19,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940806f575bc060ce15c805 2025-12-17 21:42:19,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940806f575bc060ce15c805 HTTP/1.1" 502 150 2025-12-17 21:42:19,940 - lava_callback - ERROR - Error processing callback for node 6940806f575bc060ce15c805: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940806f575bc060ce15c805 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940806f575bc060ce15c805 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940806f575bc060ce15c805 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:19,943 - lava_callback - INFO - Starting async job submit for node 69408086575bc060ce15c852 2025-12-17 21:42:19,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:19,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408086575bc060ce15c852 HTTP/1.1" 502 150 2025-12-17 21:42:19,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408086575bc060ce15c852'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:19,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408086575bc060ce15c852 2025-12-17 21:42:19,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408086575bc060ce15c852 HTTP/1.1" 502 150 2025-12-17 21:42:19,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408086575bc060ce15c852'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:20,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940807c575bc060ce15c833 2025-12-17 21:42:20,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807c575bc060ce15c833 HTTP/1.1" 502 150 2025-12-17 21:42:20,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940807c575bc060ce15c833'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:20,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408078575bc060ce15c826 2025-12-17 21:42:20,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408078575bc060ce15c826 HTTP/1.1" 502 150 2025-12-17 21:42:20,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408078575bc060ce15c826'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:20,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408083575bc060ce15c841 2025-12-17 21:42:20,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408083575bc060ce15c841 HTTP/1.1" 502 150 2025-12-17 21:42:20,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408083575bc060ce15c841'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:21,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408085575bc060ce15c84f 2025-12-17 21:42:21,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408085575bc060ce15c84f HTTP/1.1" 502 150 2025-12-17 21:42:21,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408085575bc060ce15c84f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:21,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408079575bc060ce15c82d 2025-12-17 21:42:21,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408079575bc060ce15c82d HTTP/1.1" 502 150 2025-12-17 21:42:21,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408079575bc060ce15c82d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:21,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408086575bc060ce15c852 2025-12-17 21:42:21,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408086575bc060ce15c852 HTTP/1.1" 502 150 2025-12-17 21:42:21,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408086575bc060ce15c852'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:21,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408080575bc060ce15c839 2025-12-17 21:42:21,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408080575bc060ce15c839 HTTP/1.1" 502 150 2025-12-17 21:42:21,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408080575bc060ce15c839'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:22,553 - lava_callback - INFO - Received callback for node 694080aa575bc060ce15c8c4 INFO: 172.21.0.1:44856 - "POST /node/694080aa575bc060ce15c8c4 HTTP/1.0" 202 Accepted 2025-12-17 21:42:22,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940807b575bc060ce15c830 2025-12-17 21:42:22,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807b575bc060ce15c830 HTTP/1.1" 502 150 2025-12-17 21:42:22,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940807b575bc060ce15c830'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:23,207 - lava_callback - INFO - Received callback for node 694080ae575bc060ce15c8d3 INFO: 172.21.0.1:44866 - "POST /node/694080ae575bc060ce15c8d3 HTTP/1.0" 202 Accepted 2025-12-17 21:42:23,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408071575bc060ce15c80f 2025-12-17 21:42:23,897 - lava_callback - INFO - Received callback for node 694080af575bc060ce15c8d6 INFO: 172.21.0.1:44876 - "POST /node/694080af575bc060ce15c8d6 HTTP/1.0" 202 Accepted 2025-12-17 21:42:24,577 - lava_callback - INFO - Received callback for node 694080b2575bc060ce15c8dc INFO: 172.21.0.1:44884 - "POST /node/694080b2575bc060ce15c8dc HTTP/1.0" 202 Accepted 2025-12-17 21:42:24,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408059575bc060ce15c77b 2025-12-17 21:42:24,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408059575bc060ce15c77b HTTP/1.1" 502 150 2025-12-17 21:42:24,627 - lava_callback - ERROR - Error processing callback for node 69408059575bc060ce15c77b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408059575bc060ce15c77b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408059575bc060ce15c77b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408059575bc060ce15c77b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:24,629 - lava_callback - INFO - Starting async job submit for node 69408088575bc060ce15c855 2025-12-17 21:42:24,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:24,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408088575bc060ce15c855 HTTP/1.1" 502 150 2025-12-17 21:42:24,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408088575bc060ce15c855'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:24,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408088575bc060ce15c855 2025-12-17 21:42:24,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408088575bc060ce15c855 HTTP/1.1" 502 150 2025-12-17 21:42:24,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408088575bc060ce15c855'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:24,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408083575bc060ce15c841 2025-12-17 21:42:24,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408083575bc060ce15c841 HTTP/1.1" 502 150 2025-12-17 21:42:24,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408083575bc060ce15c841'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:25,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408085575bc060ce15c84f 2025-12-17 21:42:25,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408085575bc060ce15c84f HTTP/1.1" 502 150 2025-12-17 21:42:25,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408085575bc060ce15c84f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:25,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408086575bc060ce15c852 2025-12-17 21:42:25,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408086575bc060ce15c852 HTTP/1.1" 502 150 2025-12-17 21:42:25,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408086575bc060ce15c852'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44888 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:42:26,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408088575bc060ce15c855 2025-12-17 21:42:26,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408088575bc060ce15c855 HTTP/1.1" 502 150 2025-12-17 21:42:26,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408088575bc060ce15c855'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:27,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408072575bc060ce15c812 HTTP/1.1" 502 150 2025-12-17 21:42:27,248 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408072575bc060ce15c812'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:27,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940807e575bc060ce15c836 2025-12-17 21:42:27,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807e575bc060ce15c836 HTTP/1.1" 502 150 2025-12-17 21:42:27,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940807e575bc060ce15c836'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:28,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940807c575bc060ce15c833 2025-12-17 21:42:28,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807c575bc060ce15c833 HTTP/1.1" 502 150 2025-12-17 21:42:28,040 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940807c575bc060ce15c833'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:29,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408080575bc060ce15c839 2025-12-17 21:42:29,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408080575bc060ce15c839 HTTP/1.1" 502 150 2025-12-17 21:42:29,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408080575bc060ce15c839'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:30,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408088575bc060ce15c855 2025-12-17 21:42:30,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408088575bc060ce15c855 HTTP/1.1" 502 150 2025-12-17 21:42:30,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408088575bc060ce15c855'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:32,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408073575bc060ce15c816 2025-12-17 21:42:32,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408073575bc060ce15c816 HTTP/1.1" 502 150 2025-12-17 21:42:32,485 - lava_callback - ERROR - Error processing callback for node 69408073575bc060ce15c816: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408073575bc060ce15c816 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408073575bc060ce15c816 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408073575bc060ce15c816 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:32,487 - lava_callback - INFO - Starting async job submit for node 69408089575bc060ce15c858 2025-12-17 21:42:32,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:32,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408089575bc060ce15c858 HTTP/1.1" 502 150 2025-12-17 21:42:32,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408089575bc060ce15c858'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:32,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408089575bc060ce15c858 2025-12-17 21:42:32,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408089575bc060ce15c858 HTTP/1.1" 502 150 2025-12-17 21:42:32,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408089575bc060ce15c858'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:32,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408083575bc060ce15c841 2025-12-17 21:42:32,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408083575bc060ce15c841 HTTP/1.1" 502 150 2025-12-17 21:42:32,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408083575bc060ce15c841'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:33,055 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408074575bc060ce15c819 2025-12-17 21:42:33,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408074575bc060ce15c819 HTTP/1.1" 502 150 2025-12-17 21:42:33,062 - lava_callback - ERROR - Error processing callback for node 69408074575bc060ce15c819: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408074575bc060ce15c819 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408074575bc060ce15c819 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408074575bc060ce15c819 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:33,064 - lava_callback - INFO - Starting async job submit for node 6940808b575bc060ce15c85b 2025-12-17 21:42:33,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:33,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940808b575bc060ce15c85b HTTP/1.1" 502 150 2025-12-17 21:42:33,094 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940808b575bc060ce15c85b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:33,094 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940808b575bc060ce15c85b 2025-12-17 21:42:33,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940808b575bc060ce15c85b HTTP/1.1" 502 150 2025-12-17 21:42:33,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940808b575bc060ce15c85b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:33,184 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408076575bc060ce15c81c 2025-12-17 21:42:33,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408076575bc060ce15c81c HTTP/1.1" 502 150 2025-12-17 21:42:33,186 - lava_callback - ERROR - Error processing callback for node 69408076575bc060ce15c81c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408076575bc060ce15c81c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408076575bc060ce15c81c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408076575bc060ce15c81c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:33,188 - lava_callback - INFO - Starting async job submit for node 69408081575bc060ce15c83c 2025-12-17 21:42:33,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:33,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408081575bc060ce15c83c HTTP/1.1" 502 150 2025-12-17 21:42:33,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408081575bc060ce15c83c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:33,194 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408081575bc060ce15c83c 2025-12-17 21:42:33,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408081575bc060ce15c83c HTTP/1.1" 502 150 2025-12-17 21:42:33,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408081575bc060ce15c83c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:33,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408085575bc060ce15c84f 2025-12-17 21:42:33,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408085575bc060ce15c84f HTTP/1.1" 502 150 2025-12-17 21:42:33,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408085575bc060ce15c84f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:33,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408077575bc060ce15c81f 2025-12-17 21:42:33,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408077575bc060ce15c81f HTTP/1.1" 502 150 2025-12-17 21:42:33,839 - lava_callback - ERROR - Error processing callback for node 69408077575bc060ce15c81f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408077575bc060ce15c81f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408077575bc060ce15c81f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408077575bc060ce15c81f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:33,841 - lava_callback - INFO - Starting async job submit for node 69408084575bc060ce15c848 2025-12-17 21:42:33,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:33,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408084575bc060ce15c848 HTTP/1.1" 502 150 2025-12-17 21:42:33,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408084575bc060ce15c848'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:33,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408084575bc060ce15c848 2025-12-17 21:42:33,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408084575bc060ce15c848 HTTP/1.1" 502 150 2025-12-17 21:42:33,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408084575bc060ce15c848'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:33,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408086575bc060ce15c852 2025-12-17 21:42:33,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408086575bc060ce15c852 HTTP/1.1" 502 150 2025-12-17 21:42:33,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408086575bc060ce15c852'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:34,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408089575bc060ce15c858 2025-12-17 21:42:34,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408089575bc060ce15c858 HTTP/1.1" 502 150 2025-12-17 21:42:34,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408089575bc060ce15c858'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:35,095 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940808b575bc060ce15c85b 2025-12-17 21:42:35,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940808b575bc060ce15c85b HTTP/1.1" 502 150 2025-12-17 21:42:35,096 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940808b575bc060ce15c85b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:35,195 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408081575bc060ce15c83c 2025-12-17 21:42:35,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408081575bc060ce15c83c HTTP/1.1" 502 150 2025-12-17 21:42:35,196 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408081575bc060ce15c83c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:35,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408084575bc060ce15c848 2025-12-17 21:42:35,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408084575bc060ce15c848 HTTP/1.1" 502 150 2025-12-17 21:42:35,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408084575bc060ce15c848'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33280 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:42:36,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408078575bc060ce15c826 2025-12-17 21:42:36,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408078575bc060ce15c826 HTTP/1.1" 502 150 2025-12-17 21:42:36,649 - lava_callback - ERROR - Error processing callback for node 69408078575bc060ce15c826: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408078575bc060ce15c826 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408078575bc060ce15c826 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408078575bc060ce15c826 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:36,651 - lava_callback - INFO - Starting async job submit for node 69408095575bc060ce15c863 2025-12-17 21:42:36,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:36,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408095575bc060ce15c863 HTTP/1.1" 502 150 2025-12-17 21:42:36,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408095575bc060ce15c863'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:36,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408095575bc060ce15c863 2025-12-17 21:42:36,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408095575bc060ce15c863 HTTP/1.1" 502 150 2025-12-17 21:42:36,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408095575bc060ce15c863'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:37,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408079575bc060ce15c82d 2025-12-17 21:42:37,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408079575bc060ce15c82d HTTP/1.1" 502 150 2025-12-17 21:42:37,812 - lava_callback - ERROR - Error processing callback for node 69408079575bc060ce15c82d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408079575bc060ce15c82d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408079575bc060ce15c82d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408079575bc060ce15c82d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:37,814 - lava_callback - INFO - Starting async job submit for node 6940808c575bc060ce15c85e 2025-12-17 21:42:37,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:37,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940808c575bc060ce15c85e HTTP/1.1" 502 150 2025-12-17 21:42:37,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940808c575bc060ce15c85e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:37,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940808c575bc060ce15c85e 2025-12-17 21:42:37,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940808c575bc060ce15c85e HTTP/1.1" 502 150 2025-12-17 21:42:37,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940808c575bc060ce15c85e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:38,387 - lava_callback - INFO - Received callback for node 694080bf575bc060ce15c8f0 INFO: 172.21.0.1:33292 - "POST /node/694080bf575bc060ce15c8f0 HTTP/1.0" 202 Accepted 2025-12-17 21:42:38,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408089575bc060ce15c858 2025-12-17 21:42:38,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408089575bc060ce15c858 HTTP/1.1" 502 150 2025-12-17 21:42:38,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408089575bc060ce15c858'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:38,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940807b575bc060ce15c830 2025-12-17 21:42:38,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807b575bc060ce15c830 HTTP/1.1" 502 150 2025-12-17 21:42:38,584 - lava_callback - ERROR - Error processing callback for node 6940807b575bc060ce15c830: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940807b575bc060ce15c830 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940807b575bc060ce15c830 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940807b575bc060ce15c830 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:38,585 - lava_callback - INFO - Starting async job submit for node 69408096575bc060ce15c86b 2025-12-17 21:42:38,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:38,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408096575bc060ce15c86b HTTP/1.1" 502 150 2025-12-17 21:42:38,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408096575bc060ce15c86b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:38,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408096575bc060ce15c86b 2025-12-17 21:42:38,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408096575bc060ce15c86b HTTP/1.1" 502 150 2025-12-17 21:42:38,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408096575bc060ce15c86b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:38,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408088575bc060ce15c855 2025-12-17 21:42:38,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408088575bc060ce15c855 HTTP/1.1" 502 150 2025-12-17 21:42:38,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408088575bc060ce15c855'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:38,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408095575bc060ce15c863 2025-12-17 21:42:38,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408095575bc060ce15c863 HTTP/1.1" 502 150 2025-12-17 21:42:38,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408095575bc060ce15c863'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:39,097 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940808b575bc060ce15c85b 2025-12-17 21:42:39,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940808b575bc060ce15c85b HTTP/1.1" 502 150 2025-12-17 21:42:39,098 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940808b575bc060ce15c85b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:39,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408081575bc060ce15c83c 2025-12-17 21:42:39,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408081575bc060ce15c83c HTTP/1.1" 502 150 2025-12-17 21:42:39,198 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408081575bc060ce15c83c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:39,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408071575bc060ce15c80f HTTP/1.1" 502 150 2025-12-17 21:42:39,280 - lava_callback - ERROR - Error processing callback for node 69408071575bc060ce15c80f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408071575bc060ce15c80f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408071575bc060ce15c80f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408071575bc060ce15c80f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:39,283 - lava_callback - INFO - Starting async job submit for node 69408098575bc060ce15c870 2025-12-17 21:42:39,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:39,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408098575bc060ce15c870 HTTP/1.1" 502 150 2025-12-17 21:42:39,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408098575bc060ce15c870'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:39,313 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408098575bc060ce15c870 2025-12-17 21:42:39,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408098575bc060ce15c870 HTTP/1.1" 502 150 2025-12-17 21:42:39,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408098575bc060ce15c870'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:39,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940808c575bc060ce15c85e 2025-12-17 21:42:39,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940808c575bc060ce15c85e HTTP/1.1" 502 150 2025-12-17 21:42:39,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940808c575bc060ce15c85e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:39,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408084575bc060ce15c848 2025-12-17 21:42:39,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408084575bc060ce15c848 HTTP/1.1" 502 150 2025-12-17 21:42:39,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408084575bc060ce15c848'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:40,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408096575bc060ce15c86b 2025-12-17 21:42:40,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408096575bc060ce15c86b HTTP/1.1" 502 150 2025-12-17 21:42:40,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408096575bc060ce15c86b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:41,319 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408098575bc060ce15c870 2025-12-17 21:42:41,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408098575bc060ce15c870 HTTP/1.1" 502 150 2025-12-17 21:42:41,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408098575bc060ce15c870'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:42,214 - lava_callback - INFO - Received callback for node 694076dc575bc060ce15b1f9 INFO: 172.21.0.1:40006 - "POST /node/694076dc575bc060ce15b1f9 HTTP/1.0" 202 Accepted 2025-12-17 21:42:42,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408095575bc060ce15c863 2025-12-17 21:42:42,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408095575bc060ce15c863 HTTP/1.1" 502 150 2025-12-17 21:42:42,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408095575bc060ce15c863'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:43,249 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408072575bc060ce15c812 2025-12-17 21:42:43,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408072575bc060ce15c812 HTTP/1.1" 502 150 2025-12-17 21:42:43,250 - lava_callback - ERROR - Error processing callback for node 69408072575bc060ce15c812: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408072575bc060ce15c812 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408072575bc060ce15c812 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408072575bc060ce15c812 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:43,253 - lava_callback - INFO - Starting async job submit for node 6940809e575bc060ce15c883 2025-12-17 21:42:43,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:43,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940809e575bc060ce15c883 HTTP/1.1" 502 150 2025-12-17 21:42:43,261 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940809e575bc060ce15c883'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:43,261 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940809e575bc060ce15c883 2025-12-17 21:42:43,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940809e575bc060ce15c883 HTTP/1.1" 502 150 2025-12-17 21:42:43,262 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940809e575bc060ce15c883'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:43,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940808c575bc060ce15c85e 2025-12-17 21:42:43,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940808c575bc060ce15c85e HTTP/1.1" 502 150 2025-12-17 21:42:43,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940808c575bc060ce15c85e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:43,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940807e575bc060ce15c836 2025-12-17 21:42:43,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807e575bc060ce15c836 HTTP/1.1" 502 150 2025-12-17 21:42:43,932 - lava_callback - ERROR - Error processing callback for node 6940807e575bc060ce15c836: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940807e575bc060ce15c836 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940807e575bc060ce15c836 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940807e575bc060ce15c836 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:43,933 - lava_callback - INFO - Starting async job submit for node 694080a0575bc060ce15c8ad 2025-12-17 21:42:43,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:43,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a0575bc060ce15c8ad HTTP/1.1" 502 150 2025-12-17 21:42:43,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a0575bc060ce15c8ad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:43,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a0575bc060ce15c8ad 2025-12-17 21:42:43,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a0575bc060ce15c8ad HTTP/1.1" 502 150 2025-12-17 21:42:43,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a0575bc060ce15c8ad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:44,040 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940807c575bc060ce15c833 2025-12-17 21:42:44,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408096575bc060ce15c86b 2025-12-17 21:42:44,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408096575bc060ce15c86b HTTP/1.1" 502 150 2025-12-17 21:42:44,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408096575bc060ce15c86b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:45,262 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940809e575bc060ce15c883 2025-12-17 21:42:45,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940809e575bc060ce15c883 HTTP/1.1" 502 150 2025-12-17 21:42:45,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940809e575bc060ce15c883'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:45,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408098575bc060ce15c870 2025-12-17 21:42:45,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408098575bc060ce15c870 HTTP/1.1" 502 150 2025-12-17 21:42:45,322 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408098575bc060ce15c870'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:45,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a0575bc060ce15c8ad 2025-12-17 21:42:45,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a0575bc060ce15c8ad HTTP/1.1" 502 150 2025-12-17 21:42:45,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a0575bc060ce15c8ad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:45,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408080575bc060ce15c839 2025-12-17 21:42:45,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408080575bc060ce15c839 HTTP/1.1" 502 150 2025-12-17 21:42:45,998 - lava_callback - ERROR - Error processing callback for node 69408080575bc060ce15c839: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408080575bc060ce15c839 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408080575bc060ce15c839 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408080575bc060ce15c839 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:46,001 - lava_callback - INFO - Starting async job submit for node 694080a2575bc060ce15c8b0 2025-12-17 21:42:46,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:46,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a2575bc060ce15c8b0 HTTP/1.1" 502 150 2025-12-17 21:42:46,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a2575bc060ce15c8b0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:46,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a2575bc060ce15c8b0 2025-12-17 21:42:46,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a2575bc060ce15c8b0 HTTP/1.1" 502 150 2025-12-17 21:42:46,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a2575bc060ce15c8b0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40014 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:42:46,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408089575bc060ce15c858 2025-12-17 21:42:46,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408089575bc060ce15c858 HTTP/1.1" 502 150 2025-12-17 21:42:46,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408089575bc060ce15c858'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:47,098 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940808b575bc060ce15c85b 2025-12-17 21:42:47,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940808b575bc060ce15c85b HTTP/1.1" 502 150 2025-12-17 21:42:47,100 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940808b575bc060ce15c85b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:47,198 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408081575bc060ce15c83c 2025-12-17 21:42:47,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408081575bc060ce15c83c HTTP/1.1" 502 150 2025-12-17 21:42:47,199 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408081575bc060ce15c83c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:47,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408084575bc060ce15c848 2025-12-17 21:42:47,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408084575bc060ce15c848 HTTP/1.1" 502 150 2025-12-17 21:42:47,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408084575bc060ce15c848'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:48,028 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a2575bc060ce15c8b0 2025-12-17 21:42:48,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a2575bc060ce15c8b0 HTTP/1.1" 502 150 2025-12-17 21:42:48,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a2575bc060ce15c8b0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:48,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408083575bc060ce15c841 2025-12-17 21:42:48,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408083575bc060ce15c841 HTTP/1.1" 502 150 2025-12-17 21:42:48,736 - lava_callback - ERROR - Error processing callback for node 69408083575bc060ce15c841: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408083575bc060ce15c841 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408083575bc060ce15c841 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408083575bc060ce15c841 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:48,738 - lava_callback - INFO - Starting async job submit for node 694080a3575bc060ce15c8b3 2025-12-17 21:42:48,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:48,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a3575bc060ce15c8b3 HTTP/1.1" 502 150 2025-12-17 21:42:48,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a3575bc060ce15c8b3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:48,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a3575bc060ce15c8b3 2025-12-17 21:42:48,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a3575bc060ce15c8b3 HTTP/1.1" 502 150 2025-12-17 21:42:48,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a3575bc060ce15c8b3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:49,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940809e575bc060ce15c883 2025-12-17 21:42:49,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940809e575bc060ce15c883 HTTP/1.1" 502 150 2025-12-17 21:42:49,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940809e575bc060ce15c883'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:49,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408085575bc060ce15c84f 2025-12-17 21:42:49,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408085575bc060ce15c84f HTTP/1.1" 502 150 2025-12-17 21:42:49,786 - lava_callback - ERROR - Error processing callback for node 69408085575bc060ce15c84f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408085575bc060ce15c84f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408085575bc060ce15c84f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408085575bc060ce15c84f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:49,788 - lava_callback - INFO - Starting async job submit for node 694087c7575bc060ce15e4bd 2025-12-17 21:42:49,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:49,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087c7575bc060ce15e4bd HTTP/1.1" 502 150 2025-12-17 21:42:49,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087c7575bc060ce15e4bd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:49,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087c7575bc060ce15e4bd 2025-12-17 21:42:49,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087c7575bc060ce15e4bd HTTP/1.1" 502 150 2025-12-17 21:42:49,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087c7575bc060ce15e4bd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:49,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a0575bc060ce15c8ad 2025-12-17 21:42:49,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a0575bc060ce15c8ad HTTP/1.1" 502 150 2025-12-17 21:42:49,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a0575bc060ce15c8ad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:49,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408086575bc060ce15c852 2025-12-17 21:42:49,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408086575bc060ce15c852 HTTP/1.1" 502 150 2025-12-17 21:42:49,958 - lava_callback - ERROR - Error processing callback for node 69408086575bc060ce15c852: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408086575bc060ce15c852 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408086575bc060ce15c852 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408086575bc060ce15c852 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:49,960 - lava_callback - INFO - Starting async job submit for node 694080a4575bc060ce15c8b6 2025-12-17 21:42:49,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:49,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a4575bc060ce15c8b6 HTTP/1.1" 502 150 2025-12-17 21:42:49,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a4575bc060ce15c8b6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:49,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a4575bc060ce15c8b6 2025-12-17 21:42:49,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a4575bc060ce15c8b6 HTTP/1.1" 502 150 2025-12-17 21:42:49,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a4575bc060ce15c8b6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:50,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408095575bc060ce15c863 2025-12-17 21:42:50,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408095575bc060ce15c863 HTTP/1.1" 502 150 2025-12-17 21:42:50,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408095575bc060ce15c863'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:50,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a3575bc060ce15c8b3 2025-12-17 21:42:50,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a3575bc060ce15c8b3 HTTP/1.1" 502 150 2025-12-17 21:42:50,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a3575bc060ce15c8b3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:51,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087c7575bc060ce15e4bd 2025-12-17 21:42:51,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087c7575bc060ce15e4bd HTTP/1.1" 502 150 2025-12-17 21:42:51,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087c7575bc060ce15e4bd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:51,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940808c575bc060ce15c85e 2025-12-17 21:42:51,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940808c575bc060ce15c85e HTTP/1.1" 502 150 2025-12-17 21:42:51,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940808c575bc060ce15c85e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:51,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a4575bc060ce15c8b6 2025-12-17 21:42:51,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a4575bc060ce15c8b6 HTTP/1.1" 502 150 2025-12-17 21:42:51,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a4575bc060ce15c8b6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:52,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a2575bc060ce15c8b0 2025-12-17 21:42:52,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a2575bc060ce15c8b0 HTTP/1.1" 502 150 2025-12-17 21:42:52,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a2575bc060ce15c8b0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:52,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408096575bc060ce15c86b 2025-12-17 21:42:52,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408096575bc060ce15c86b HTTP/1.1" 502 150 2025-12-17 21:42:52,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408096575bc060ce15c86b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:53,322 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408098575bc060ce15c870 2025-12-17 21:42:53,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408098575bc060ce15c870 HTTP/1.1" 502 150 2025-12-17 21:42:53,323 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408098575bc060ce15c870'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:54,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408088575bc060ce15c855 2025-12-17 21:42:54,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408088575bc060ce15c855 HTTP/1.1" 502 150 2025-12-17 21:42:54,642 - lava_callback - ERROR - Error processing callback for node 69408088575bc060ce15c855: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408088575bc060ce15c855 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408088575bc060ce15c855 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408088575bc060ce15c855 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:54,643 - lava_callback - INFO - Starting async job submit for node 694080a5575bc060ce15c8b9 2025-12-17 21:42:54,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:54,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a5575bc060ce15c8b9 HTTP/1.1" 502 150 2025-12-17 21:42:54,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a5575bc060ce15c8b9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:54,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a5575bc060ce15c8b9 2025-12-17 21:42:54,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a5575bc060ce15c8b9 HTTP/1.1" 502 150 2025-12-17 21:42:54,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a5575bc060ce15c8b9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:54,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a3575bc060ce15c8b3 2025-12-17 21:42:54,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a3575bc060ce15c8b3 HTTP/1.1" 502 150 2025-12-17 21:42:54,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a3575bc060ce15c8b3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:55,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087c7575bc060ce15e4bd 2025-12-17 21:42:55,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087c7575bc060ce15e4bd HTTP/1.1" 502 150 2025-12-17 21:42:55,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087c7575bc060ce15e4bd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:55,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a4575bc060ce15c8b6 2025-12-17 21:42:55,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a4575bc060ce15c8b6 HTTP/1.1" 502 150 2025-12-17 21:42:55,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a4575bc060ce15c8b6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57492 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:42:56,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a5575bc060ce15c8b9 2025-12-17 21:42:56,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a5575bc060ce15c8b9 HTTP/1.1" 502 150 2025-12-17 21:42:56,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a5575bc060ce15c8b9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:57,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940809e575bc060ce15c883 2025-12-17 21:42:57,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940809e575bc060ce15c883 HTTP/1.1" 502 150 2025-12-17 21:42:57,267 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940809e575bc060ce15c883'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:57,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a0575bc060ce15c8ad 2025-12-17 21:42:57,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a0575bc060ce15c8ad HTTP/1.1" 502 150 2025-12-17 21:42:57,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a0575bc060ce15c8ad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:58,446 - lava_callback - INFO - Received callback for node 694080d8575bc060ce15c909 INFO: 172.21.0.1:57504 - "POST /node/694080d8575bc060ce15c909 HTTP/1.0" 202 Accepted 2025-12-17 21:42:59,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940807c575bc060ce15c833 HTTP/1.1" 502 150 2025-12-17 21:42:59,504 - lava_callback - ERROR - Error processing callback for node 6940807c575bc060ce15c833: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940807c575bc060ce15c833 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940807c575bc060ce15c833 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940807c575bc060ce15c833 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:42:59,506 - lava_callback - INFO - Starting async job submit for node 694080a7575bc060ce15c8bc 2025-12-17 21:42:59,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:42:59,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a7575bc060ce15c8bc HTTP/1.1" 502 150 2025-12-17 21:42:59,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a7575bc060ce15c8bc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:42:59,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a7575bc060ce15c8bc 2025-12-17 21:42:59,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a7575bc060ce15c8bc HTTP/1.1" 502 150 2025-12-17 21:42:59,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a7575bc060ce15c8bc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:00,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a2575bc060ce15c8b0 2025-12-17 21:43:00,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a2575bc060ce15c8b0 HTTP/1.1" 502 150 2025-12-17 21:43:00,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a2575bc060ce15c8b0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:00,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a5575bc060ce15c8b9 2025-12-17 21:43:00,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a5575bc060ce15c8b9 HTTP/1.1" 502 150 2025-12-17 21:43:00,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a5575bc060ce15c8b9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:01,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a7575bc060ce15c8bc 2025-12-17 21:43:01,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a7575bc060ce15c8bc HTTP/1.1" 502 150 2025-12-17 21:43:01,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a7575bc060ce15c8bc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58156 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:43:02,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408089575bc060ce15c858 2025-12-17 21:43:02,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408089575bc060ce15c858 HTTP/1.1" 502 150 2025-12-17 21:43:02,527 - lava_callback - ERROR - Error processing callback for node 69408089575bc060ce15c858: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408089575bc060ce15c858 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408089575bc060ce15c858 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408089575bc060ce15c858 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:02,529 - lava_callback - INFO - Starting async job submit for node 694080a8575bc060ce15c8bf 2025-12-17 21:43:02,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:02,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a8575bc060ce15c8bf HTTP/1.1" 502 150 2025-12-17 21:43:02,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a8575bc060ce15c8bf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:02,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a8575bc060ce15c8bf 2025-12-17 21:43:02,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a8575bc060ce15c8bf HTTP/1.1" 502 150 2025-12-17 21:43:02,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a8575bc060ce15c8bf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:02,570 - lava_callback - INFO - Received callback for node 694080dc575bc060ce15c91c INFO: 172.21.0.1:58172 - "POST /node/694080dc575bc060ce15c91c HTTP/1.0" 202 Accepted 2025-12-17 21:43:02,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a3575bc060ce15c8b3 2025-12-17 21:43:02,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a3575bc060ce15c8b3 HTTP/1.1" 502 150 2025-12-17 21:43:02,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a3575bc060ce15c8b3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:03,100 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940808b575bc060ce15c85b 2025-12-17 21:43:03,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940808b575bc060ce15c85b HTTP/1.1" 502 150 2025-12-17 21:43:03,101 - lava_callback - ERROR - Error processing callback for node 6940808b575bc060ce15c85b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940808b575bc060ce15c85b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940808b575bc060ce15c85b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940808b575bc060ce15c85b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:03,103 - lava_callback - INFO - Starting async job submit for node 694080b3575bc060ce15c8df 2025-12-17 21:43:03,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:03,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b3575bc060ce15c8df HTTP/1.1" 502 150 2025-12-17 21:43:03,109 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b3575bc060ce15c8df'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:03,110 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b3575bc060ce15c8df 2025-12-17 21:43:03,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b3575bc060ce15c8df HTTP/1.1" 502 150 2025-12-17 21:43:03,110 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b3575bc060ce15c8df'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:03,199 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408081575bc060ce15c83c 2025-12-17 21:43:03,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408081575bc060ce15c83c HTTP/1.1" 502 150 2025-12-17 21:43:03,201 - lava_callback - ERROR - Error processing callback for node 69408081575bc060ce15c83c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408081575bc060ce15c83c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408081575bc060ce15c83c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408081575bc060ce15c83c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:03,203 - lava_callback - INFO - Starting async job submit for node 694080b5575bc060ce15c8e2 2025-12-17 21:43:03,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:03,220 - lava_callback - INFO - Received callback for node 694080de575bc060ce15c946 INFO: 172.21.0.1:58182 - "POST /node/694080de575bc060ce15c946 HTTP/1.0" 202 Accepted 2025-12-17 21:43:03,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b5575bc060ce15c8e2 HTTP/1.1" 502 150 2025-12-17 21:43:03,234 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b5575bc060ce15c8e2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:03,234 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b5575bc060ce15c8e2 2025-12-17 21:43:03,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b5575bc060ce15c8e2 HTTP/1.1" 502 150 2025-12-17 21:43:03,235 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b5575bc060ce15c8e2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:03,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087c7575bc060ce15e4bd 2025-12-17 21:43:03,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087c7575bc060ce15e4bd HTTP/1.1" 502 150 2025-12-17 21:43:03,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087c7575bc060ce15e4bd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:03,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408084575bc060ce15c848 2025-12-17 21:43:03,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408084575bc060ce15c848 HTTP/1.1" 502 150 2025-12-17 21:43:03,886 - lava_callback - ERROR - Error processing callback for node 69408084575bc060ce15c848: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408084575bc060ce15c848 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408084575bc060ce15c848 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408084575bc060ce15c848 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:03,888 - lava_callback - INFO - Starting async job submit for node 694080b6575bc060ce15c8e5 2025-12-17 21:43:03,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:03,889 - lava_callback - INFO - Received callback for node 694080df575bc060ce15c949 INFO: 172.21.0.1:58196 - "POST /node/694080df575bc060ce15c949 HTTP/1.0" 202 Accepted 2025-12-17 21:43:03,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b6575bc060ce15c8e5 HTTP/1.1" 502 150 2025-12-17 21:43:03,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b6575bc060ce15c8e5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:03,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b6575bc060ce15c8e5 2025-12-17 21:43:03,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b6575bc060ce15c8e5 HTTP/1.1" 502 150 2025-12-17 21:43:03,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b6575bc060ce15c8e5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:03,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a4575bc060ce15c8b6 2025-12-17 21:43:03,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a4575bc060ce15c8b6 HTTP/1.1" 502 150 2025-12-17 21:43:03,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a4575bc060ce15c8b6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:04,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a8575bc060ce15c8bf 2025-12-17 21:43:04,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a8575bc060ce15c8bf HTTP/1.1" 502 150 2025-12-17 21:43:04,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a8575bc060ce15c8bf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:05,111 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b3575bc060ce15c8df 2025-12-17 21:43:05,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b3575bc060ce15c8df HTTP/1.1" 502 150 2025-12-17 21:43:05,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b3575bc060ce15c8df'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:05,235 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b5575bc060ce15c8e2 2025-12-17 21:43:05,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b5575bc060ce15c8e2 HTTP/1.1" 502 150 2025-12-17 21:43:05,237 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b5575bc060ce15c8e2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:05,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a7575bc060ce15c8bc 2025-12-17 21:43:05,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a7575bc060ce15c8bc HTTP/1.1" 502 150 2025-12-17 21:43:05,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a7575bc060ce15c8bc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:05,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b6575bc060ce15c8e5 2025-12-17 21:43:05,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b6575bc060ce15c8e5 HTTP/1.1" 502 150 2025-12-17 21:43:05,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b6575bc060ce15c8e5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58208 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:43:06,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408095575bc060ce15c863 2025-12-17 21:43:06,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408095575bc060ce15c863 HTTP/1.1" 502 150 2025-12-17 21:43:06,668 - lava_callback - ERROR - Error processing callback for node 69408095575bc060ce15c863: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408095575bc060ce15c863 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408095575bc060ce15c863 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408095575bc060ce15c863 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:06,670 - lava_callback - INFO - Starting async job submit for node 694080aa575bc060ce15c8c4 2025-12-17 21:43:06,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:06,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080aa575bc060ce15c8c4 HTTP/1.1" 502 150 2025-12-17 21:43:06,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080aa575bc060ce15c8c4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:06,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080aa575bc060ce15c8c4 2025-12-17 21:43:06,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080aa575bc060ce15c8c4 HTTP/1.1" 502 150 2025-12-17 21:43:06,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080aa575bc060ce15c8c4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:07,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940808c575bc060ce15c85e 2025-12-17 21:43:07,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940808c575bc060ce15c85e HTTP/1.1" 502 150 2025-12-17 21:43:07,828 - lava_callback - ERROR - Error processing callback for node 6940808c575bc060ce15c85e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940808c575bc060ce15c85e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940808c575bc060ce15c85e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940808c575bc060ce15c85e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:07,830 - lava_callback - INFO - Starting async job submit for node 694080ae575bc060ce15c8d3 2025-12-17 21:43:07,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:07,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ae575bc060ce15c8d3 HTTP/1.1" 502 150 2025-12-17 21:43:07,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ae575bc060ce15c8d3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:07,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ae575bc060ce15c8d3 2025-12-17 21:43:07,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ae575bc060ce15c8d3 HTTP/1.1" 502 150 2025-12-17 21:43:07,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ae575bc060ce15c8d3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:08,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a8575bc060ce15c8bf 2025-12-17 21:43:08,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a8575bc060ce15c8bf HTTP/1.1" 502 150 2025-12-17 21:43:08,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a8575bc060ce15c8bf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:08,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408096575bc060ce15c86b 2025-12-17 21:43:08,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408096575bc060ce15c86b HTTP/1.1" 502 150 2025-12-17 21:43:08,598 - lava_callback - ERROR - Error processing callback for node 69408096575bc060ce15c86b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408096575bc060ce15c86b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408096575bc060ce15c86b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408096575bc060ce15c86b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:08,600 - lava_callback - INFO - Starting async job submit for node 694080af575bc060ce15c8d6 2025-12-17 21:43:08,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:08,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080af575bc060ce15c8d6 HTTP/1.1" 502 150 2025-12-17 21:43:08,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080af575bc060ce15c8d6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:08,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080af575bc060ce15c8d6 2025-12-17 21:43:08,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080af575bc060ce15c8d6 HTTP/1.1" 502 150 2025-12-17 21:43:08,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080af575bc060ce15c8d6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:08,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a5575bc060ce15c8b9 2025-12-17 21:43:08,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a5575bc060ce15c8b9 HTTP/1.1" 502 150 2025-12-17 21:43:08,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a5575bc060ce15c8b9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:08,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080aa575bc060ce15c8c4 2025-12-17 21:43:08,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080aa575bc060ce15c8c4 HTTP/1.1" 502 150 2025-12-17 21:43:08,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080aa575bc060ce15c8c4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:09,112 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b3575bc060ce15c8df 2025-12-17 21:43:09,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b3575bc060ce15c8df HTTP/1.1" 502 150 2025-12-17 21:43:09,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b3575bc060ce15c8df'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:09,237 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b5575bc060ce15c8e2 2025-12-17 21:43:09,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b5575bc060ce15c8e2 HTTP/1.1" 502 150 2025-12-17 21:43:09,238 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b5575bc060ce15c8e2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:09,323 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408098575bc060ce15c870 2025-12-17 21:43:09,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408098575bc060ce15c870 HTTP/1.1" 502 150 2025-12-17 21:43:09,325 - lava_callback - ERROR - Error processing callback for node 69408098575bc060ce15c870: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408098575bc060ce15c870 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408098575bc060ce15c870 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408098575bc060ce15c870 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:09,326 - lava_callback - INFO - Starting async job submit for node 694080b2575bc060ce15c8dc 2025-12-17 21:43:09,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:09,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b2575bc060ce15c8dc HTTP/1.1" 502 150 2025-12-17 21:43:09,333 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b2575bc060ce15c8dc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:09,334 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b2575bc060ce15c8dc 2025-12-17 21:43:09,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b2575bc060ce15c8dc HTTP/1.1" 502 150 2025-12-17 21:43:09,334 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b2575bc060ce15c8dc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:09,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ae575bc060ce15c8d3 2025-12-17 21:43:09,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ae575bc060ce15c8d3 HTTP/1.1" 502 150 2025-12-17 21:43:09,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ae575bc060ce15c8d3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:09,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b6575bc060ce15c8e5 2025-12-17 21:43:09,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b6575bc060ce15c8e5 HTTP/1.1" 502 150 2025-12-17 21:43:09,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b6575bc060ce15c8e5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:10,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080af575bc060ce15c8d6 2025-12-17 21:43:11,335 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b2575bc060ce15c8dc 2025-12-17 21:43:11,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b2575bc060ce15c8dc HTTP/1.1" 502 150 2025-12-17 21:43:11,340 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b2575bc060ce15c8dc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:12,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080aa575bc060ce15c8c4 2025-12-17 21:43:12,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080aa575bc060ce15c8c4 HTTP/1.1" 502 150 2025-12-17 21:43:12,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080aa575bc060ce15c8c4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:13,038 - lava_callback - INFO - Received callback for node 694080ee575bc060ce15c967 INFO: 172.21.0.1:45828 - "POST /node/694080ee575bc060ce15c967 HTTP/1.0" 202 Accepted 2025-12-17 21:43:13,268 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940809e575bc060ce15c883 2025-12-17 21:43:13,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940809e575bc060ce15c883 HTTP/1.1" 502 150 2025-12-17 21:43:13,269 - lava_callback - ERROR - Error processing callback for node 6940809e575bc060ce15c883: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940809e575bc060ce15c883 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940809e575bc060ce15c883 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940809e575bc060ce15c883 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:13,271 - lava_callback - INFO - Starting async job submit for node 694080bf575bc060ce15c8f0 2025-12-17 21:43:13,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:13,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080bf575bc060ce15c8f0 HTTP/1.1" 502 150 2025-12-17 21:43:13,282 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080bf575bc060ce15c8f0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:13,282 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080bf575bc060ce15c8f0 2025-12-17 21:43:13,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080bf575bc060ce15c8f0 HTTP/1.1" 502 150 2025-12-17 21:43:13,283 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080bf575bc060ce15c8f0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:13,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a7575bc060ce15c8bc 2025-12-17 21:43:13,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a7575bc060ce15c8bc HTTP/1.1" 502 150 2025-12-17 21:43:13,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a7575bc060ce15c8bc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:13,697 - lava_callback - INFO - Received callback for node 694080ea575bc060ce15c961 INFO: 172.21.0.1:45830 - "POST /node/694080ea575bc060ce15c961 HTTP/1.0" 202 Accepted 2025-12-17 21:43:13,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ae575bc060ce15c8d3 2025-12-17 21:43:13,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ae575bc060ce15c8d3 HTTP/1.1" 502 150 2025-12-17 21:43:13,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ae575bc060ce15c8d3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:13,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a0575bc060ce15c8ad 2025-12-17 21:43:13,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a0575bc060ce15c8ad HTTP/1.1" 502 150 2025-12-17 21:43:13,953 - lava_callback - ERROR - Error processing callback for node 694080a0575bc060ce15c8ad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a0575bc060ce15c8ad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a0575bc060ce15c8ad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a0575bc060ce15c8ad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:13,955 - lava_callback - INFO - Starting async job submit for node 694076dc575bc060ce15b1f9 2025-12-17 21:43:14,369 - lava_callback - INFO - Received callback for node 694080ed575bc060ce15c964 INFO: 172.21.0.1:45840 - "POST /node/694080ed575bc060ce15c964 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:45854 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:43:15,288 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080bf575bc060ce15c8f0 2025-12-17 21:43:15,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080bf575bc060ce15c8f0 HTTP/1.1" 502 150 2025-12-17 21:43:15,309 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080bf575bc060ce15c8f0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:15,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b2575bc060ce15c8dc 2025-12-17 21:43:15,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b2575bc060ce15c8dc HTTP/1.1" 502 150 2025-12-17 21:43:15,367 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b2575bc060ce15c8dc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:15,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:15,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076dc575bc060ce15b1f9 HTTP/1.1" 502 150 2025-12-17 21:43:15,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076dc575bc060ce15b1f9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:15,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076dc575bc060ce15b1f9 2025-12-17 21:43:15,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076dc575bc060ce15b1f9 HTTP/1.1" 502 150 2025-12-17 21:43:15,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076dc575bc060ce15b1f9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:16,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a2575bc060ce15c8b0 INFO: 172.21.0.1:45856 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:43:16,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a8575bc060ce15c8bf 2025-12-17 21:43:16,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a8575bc060ce15c8bf HTTP/1.1" 502 150 2025-12-17 21:43:16,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080a8575bc060ce15c8bf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:17,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b3575bc060ce15c8df 2025-12-17 21:43:17,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b3575bc060ce15c8df HTTP/1.1" 502 150 2025-12-17 21:43:17,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b3575bc060ce15c8df'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:17,238 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b5575bc060ce15c8e2 2025-12-17 21:43:17,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b5575bc060ce15c8e2 HTTP/1.1" 502 150 2025-12-17 21:43:17,240 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b5575bc060ce15c8e2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:17,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076dc575bc060ce15b1f9 2025-12-17 21:43:17,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076dc575bc060ce15b1f9 HTTP/1.1" 502 150 2025-12-17 21:43:17,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076dc575bc060ce15b1f9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:17,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b6575bc060ce15c8e5 2025-12-17 21:43:17,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b6575bc060ce15c8e5 HTTP/1.1" 502 150 2025-12-17 21:43:17,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b6575bc060ce15c8e5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:17,913 - lava_callback - INFO - Received callback for node 694080e2575bc060ce15c94f INFO: 172.21.0.1:45866 - "POST /node/694080e2575bc060ce15c94f HTTP/1.0" 202 Accepted 2025-12-17 21:43:18,667 - lava_callback - INFO - Received callback for node 694080e3575bc060ce15c952 INFO: 172.21.0.1:45876 - "POST /node/694080e3575bc060ce15c952 HTTP/1.0" 202 Accepted 2025-12-17 21:43:18,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a3575bc060ce15c8b3 2025-12-17 21:43:18,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a3575bc060ce15c8b3 HTTP/1.1" 502 150 2025-12-17 21:43:18,751 - lava_callback - ERROR - Error processing callback for node 694080a3575bc060ce15c8b3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a3575bc060ce15c8b3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a3575bc060ce15c8b3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a3575bc060ce15c8b3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:18,753 - lava_callback - INFO - Starting async job submit for node 694080d8575bc060ce15c909 2025-12-17 21:43:18,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:18,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080d8575bc060ce15c909 HTTP/1.1" 502 150 2025-12-17 21:43:18,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080d8575bc060ce15c909'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:18,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080d8575bc060ce15c909 2025-12-17 21:43:18,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080d8575bc060ce15c909 HTTP/1.1" 502 150 2025-12-17 21:43:18,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080d8575bc060ce15c909'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:19,315 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080bf575bc060ce15c8f0 2025-12-17 21:43:19,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080bf575bc060ce15c8f0 HTTP/1.1" 502 150 2025-12-17 21:43:19,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080bf575bc060ce15c8f0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:19,449 - lava_callback - INFO - Received callback for node 694080e4575bc060ce15c955 INFO: 172.21.0.1:45886 - "POST /node/694080e4575bc060ce15c955 HTTP/1.0" 202 Accepted 2025-12-17 21:43:19,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087c7575bc060ce15e4bd 2025-12-17 21:43:19,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087c7575bc060ce15e4bd HTTP/1.1" 502 150 2025-12-17 21:43:19,820 - lava_callback - ERROR - Error processing callback for node 694087c7575bc060ce15e4bd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087c7575bc060ce15e4bd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087c7575bc060ce15e4bd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087c7575bc060ce15e4bd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:19,822 - lava_callback - INFO - Starting async job submit for node 694080dc575bc060ce15c91c 2025-12-17 21:43:19,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:19,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080dc575bc060ce15c91c HTTP/1.1" 502 150 2025-12-17 21:43:19,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080dc575bc060ce15c91c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:19,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080dc575bc060ce15c91c 2025-12-17 21:43:19,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080dc575bc060ce15c91c HTTP/1.1" 502 150 2025-12-17 21:43:19,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080dc575bc060ce15c91c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:19,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a4575bc060ce15c8b6 2025-12-17 21:43:19,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a4575bc060ce15c8b6 HTTP/1.1" 502 150 2025-12-17 21:43:19,973 - lava_callback - ERROR - Error processing callback for node 694080a4575bc060ce15c8b6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a4575bc060ce15c8b6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a4575bc060ce15c8b6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a4575bc060ce15c8b6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:19,975 - lava_callback - INFO - Starting async job submit for node 694080de575bc060ce15c946 2025-12-17 21:43:19,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:19,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080de575bc060ce15c946 HTTP/1.1" 502 150 2025-12-17 21:43:19,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080de575bc060ce15c946'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:19,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080de575bc060ce15c946 2025-12-17 21:43:19,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080de575bc060ce15c946 HTTP/1.1" 502 150 2025-12-17 21:43:19,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080de575bc060ce15c946'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:20,186 - lava_callback - INFO - Received callback for node 694080e6575bc060ce15c958 INFO: 172.21.0.1:32794 - "POST /node/694080e6575bc060ce15c958 HTTP/1.0" 202 Accepted 2025-12-17 21:43:20,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080aa575bc060ce15c8c4 2025-12-17 21:43:20,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080aa575bc060ce15c8c4 HTTP/1.1" 502 150 2025-12-17 21:43:20,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080aa575bc060ce15c8c4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:20,741 - lava_callback - INFO - Received callback for node 694080ac575bc060ce15c8d0 INFO: 172.21.0.1:32798 - "POST /node/694080ac575bc060ce15c8d0 HTTP/1.0" 202 Accepted 2025-12-17 21:43:20,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080d8575bc060ce15c909 2025-12-17 21:43:20,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080d8575bc060ce15c909 HTTP/1.1" 502 150 2025-12-17 21:43:20,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080d8575bc060ce15c909'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:20,932 - lava_callback - INFO - Received callback for node 694080e7575bc060ce15c95b INFO: 172.21.0.1:32810 - "POST /node/694080e7575bc060ce15c95b HTTP/1.0" 202 Accepted 2025-12-17 21:43:21,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076dc575bc060ce15b1f9 2025-12-17 21:43:21,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076dc575bc060ce15b1f9 HTTP/1.1" 502 150 2025-12-17 21:43:21,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076dc575bc060ce15b1f9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:21,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ae575bc060ce15c8d3 2025-12-17 21:43:21,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ae575bc060ce15c8d3 HTTP/1.1" 502 150 2025-12-17 21:43:21,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ae575bc060ce15c8d3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:21,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080dc575bc060ce15c91c 2025-12-17 21:43:21,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080dc575bc060ce15c91c HTTP/1.1" 502 150 2025-12-17 21:43:21,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080dc575bc060ce15c91c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:21,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080de575bc060ce15c946 2025-12-17 21:43:21,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080de575bc060ce15c946 HTTP/1.1" 502 150 2025-12-17 21:43:21,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080de575bc060ce15c946'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:23,372 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b2575bc060ce15c8dc 2025-12-17 21:43:23,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b2575bc060ce15c8dc HTTP/1.1" 502 150 2025-12-17 21:43:23,373 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080b2575bc060ce15c8dc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:24,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a5575bc060ce15c8b9 2025-12-17 21:43:24,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080d8575bc060ce15c909 2025-12-17 21:43:24,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080d8575bc060ce15c909 HTTP/1.1" 502 150 2025-12-17 21:43:24,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080d8575bc060ce15c909'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:25,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080dc575bc060ce15c91c 2025-12-17 21:43:25,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080dc575bc060ce15c91c HTTP/1.1" 502 150 2025-12-17 21:43:25,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080dc575bc060ce15c91c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:25,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080af575bc060ce15c8d6 HTTP/1.1" 502 150 2025-12-17 21:43:25,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080af575bc060ce15c8d6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:25,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080de575bc060ce15c946 2025-12-17 21:43:25,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080de575bc060ce15c946 HTTP/1.1" 502 150 2025-12-17 21:43:25,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080de575bc060ce15c946'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:32814 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:43:27,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080bf575bc060ce15c8f0 2025-12-17 21:43:27,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080bf575bc060ce15c8f0 HTTP/1.1" 502 150 2025-12-17 21:43:27,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080bf575bc060ce15c8f0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:29,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a7575bc060ce15c8bc 2025-12-17 21:43:29,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a7575bc060ce15c8bc HTTP/1.1" 502 150 2025-12-17 21:43:29,533 - lava_callback - ERROR - Error processing callback for node 694080a7575bc060ce15c8bc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a7575bc060ce15c8bc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a7575bc060ce15c8bc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a7575bc060ce15c8bc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:29,535 - lava_callback - INFO - Starting async job submit for node 694080df575bc060ce15c949 2025-12-17 21:43:29,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:29,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080df575bc060ce15c949 HTTP/1.1" 502 150 2025-12-17 21:43:29,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080df575bc060ce15c949'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:29,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080df575bc060ce15c949 2025-12-17 21:43:29,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080df575bc060ce15c949 HTTP/1.1" 502 150 2025-12-17 21:43:29,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080df575bc060ce15c949'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:29,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076dc575bc060ce15b1f9 2025-12-17 21:43:29,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076dc575bc060ce15b1f9 HTTP/1.1" 502 150 2025-12-17 21:43:29,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694076dc575bc060ce15b1f9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:29,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080af575bc060ce15c8d6 2025-12-17 21:43:29,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080af575bc060ce15c8d6 HTTP/1.1" 502 150 2025-12-17 21:43:29,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080af575bc060ce15c8d6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:31,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a2575bc060ce15c8b0 HTTP/1.1" 502 150 2025-12-17 21:43:31,504 - lava_callback - ERROR - Error processing callback for node 694080a2575bc060ce15c8b0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a2575bc060ce15c8b0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a2575bc060ce15c8b0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a2575bc060ce15c8b0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:31,506 - lava_callback - INFO - Starting async job submit for node 694080ee575bc060ce15c967 2025-12-17 21:43:31,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:31,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ee575bc060ce15c967 HTTP/1.1" 502 150 2025-12-17 21:43:31,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ee575bc060ce15c967'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:31,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ee575bc060ce15c967 2025-12-17 21:43:31,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ee575bc060ce15c967 HTTP/1.1" 502 150 2025-12-17 21:43:31,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ee575bc060ce15c967'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:31,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080df575bc060ce15c949 2025-12-17 21:43:31,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080df575bc060ce15c949 HTTP/1.1" 502 150 2025-12-17 21:43:31,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080df575bc060ce15c949'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:32,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080a8575bc060ce15c8bf 2025-12-17 21:43:32,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a8575bc060ce15c8bf HTTP/1.1" 502 150 2025-12-17 21:43:32,542 - lava_callback - ERROR - Error processing callback for node 694080a8575bc060ce15c8bf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a8575bc060ce15c8bf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a8575bc060ce15c8bf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a8575bc060ce15c8bf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:32,544 - lava_callback - INFO - Starting async job submit for node 694080ea575bc060ce15c961 2025-12-17 21:43:32,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:32,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ea575bc060ce15c961 HTTP/1.1" 502 150 2025-12-17 21:43:32,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ea575bc060ce15c961'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:32,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ea575bc060ce15c961 2025-12-17 21:43:32,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ea575bc060ce15c961 HTTP/1.1" 502 150 2025-12-17 21:43:32,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ea575bc060ce15c961'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:32,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080d8575bc060ce15c909 2025-12-17 21:43:32,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080d8575bc060ce15c909 HTTP/1.1" 502 150 2025-12-17 21:43:32,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080d8575bc060ce15c909'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:33,024 - lava_callback - INFO - Received callback for node 694080fd575bc060ce15c97e INFO: 172.21.0.1:57374 - "POST /node/694080fd575bc060ce15c97e HTTP/1.0" 202 Accepted 2025-12-17 21:43:33,116 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b3575bc060ce15c8df 2025-12-17 21:43:33,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b3575bc060ce15c8df HTTP/1.1" 502 150 2025-12-17 21:43:33,117 - lava_callback - ERROR - Error processing callback for node 694080b3575bc060ce15c8df: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080b3575bc060ce15c8df (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080b3575bc060ce15c8df (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080b3575bc060ce15c8df (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:33,119 - lava_callback - INFO - Starting async job submit for node 694080ed575bc060ce15c964 2025-12-17 21:43:33,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:33,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ed575bc060ce15c964 HTTP/1.1" 502 150 2025-12-17 21:43:33,125 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ed575bc060ce15c964'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:33,125 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ed575bc060ce15c964 2025-12-17 21:43:33,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ed575bc060ce15c964 HTTP/1.1" 502 150 2025-12-17 21:43:33,126 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ed575bc060ce15c964'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:33,240 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b5575bc060ce15c8e2 2025-12-17 21:43:33,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b5575bc060ce15c8e2 HTTP/1.1" 502 150 2025-12-17 21:43:33,241 - lava_callback - ERROR - Error processing callback for node 694080b5575bc060ce15c8e2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080b5575bc060ce15c8e2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080b5575bc060ce15c8e2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080b5575bc060ce15c8e2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:33,243 - lava_callback - INFO - Starting async job submit for node 694080e2575bc060ce15c94f 2025-12-17 21:43:33,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:33,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e2575bc060ce15c94f HTTP/1.1" 502 150 2025-12-17 21:43:33,249 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e2575bc060ce15c94f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:33,250 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e2575bc060ce15c94f 2025-12-17 21:43:33,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e2575bc060ce15c94f HTTP/1.1" 502 150 2025-12-17 21:43:33,250 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e2575bc060ce15c94f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:33,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ee575bc060ce15c967 2025-12-17 21:43:33,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ee575bc060ce15c967 HTTP/1.1" 502 150 2025-12-17 21:43:33,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ee575bc060ce15c967'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:33,667 - lava_callback - INFO - Received callback for node 694080fe575bc060ce15c983 INFO: 172.21.0.1:57390 - "POST /node/694080fe575bc060ce15c983 HTTP/1.0" 202 Accepted 2025-12-17 21:43:33,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080dc575bc060ce15c91c 2025-12-17 21:43:33,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080dc575bc060ce15c91c HTTP/1.1" 502 150 2025-12-17 21:43:33,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080dc575bc060ce15c91c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:33,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b6575bc060ce15c8e5 2025-12-17 21:43:33,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b6575bc060ce15c8e5 HTTP/1.1" 502 150 2025-12-17 21:43:33,905 - lava_callback - ERROR - Error processing callback for node 694080b6575bc060ce15c8e5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080b6575bc060ce15c8e5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080b6575bc060ce15c8e5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080b6575bc060ce15c8e5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:33,907 - lava_callback - INFO - Starting async job submit for node 694080e3575bc060ce15c952 2025-12-17 21:43:33,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:33,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e3575bc060ce15c952 HTTP/1.1" 502 150 2025-12-17 21:43:33,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e3575bc060ce15c952'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:33,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e3575bc060ce15c952 2025-12-17 21:43:33,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e3575bc060ce15c952 HTTP/1.1" 502 150 2025-12-17 21:43:33,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e3575bc060ce15c952'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:33,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080de575bc060ce15c946 2025-12-17 21:43:33,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080de575bc060ce15c946 HTTP/1.1" 502 150 2025-12-17 21:43:33,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080de575bc060ce15c946'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:34,318 - lava_callback - INFO - Received callback for node 694080ff575bc060ce15c98a INFO: 172.21.0.1:57396 - "POST /node/694080ff575bc060ce15c98a HTTP/1.0" 202 Accepted 2025-12-17 21:43:34,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ea575bc060ce15c961 2025-12-17 21:43:34,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ea575bc060ce15c961 HTTP/1.1" 502 150 2025-12-17 21:43:34,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ea575bc060ce15c961'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:34,980 - lava_callback - INFO - Received callback for node 69408101575bc060ce15c98d INFO: 172.21.0.1:57402 - "POST /node/69408101575bc060ce15c98d HTTP/1.0" 202 Accepted 2025-12-17 21:43:35,126 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ed575bc060ce15c964 2025-12-17 21:43:35,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ed575bc060ce15c964 HTTP/1.1" 502 150 2025-12-17 21:43:35,127 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ed575bc060ce15c964'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:35,250 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e2575bc060ce15c94f 2025-12-17 21:43:35,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e2575bc060ce15c94f HTTP/1.1" 502 150 2025-12-17 21:43:35,251 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e2575bc060ce15c94f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:35,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080df575bc060ce15c949 2025-12-17 21:43:35,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080df575bc060ce15c949 HTTP/1.1" 502 150 2025-12-17 21:43:35,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080df575bc060ce15c949'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:35,656 - lava_callback - INFO - Received callback for node 694080fb575bc060ce15c977 INFO: 172.21.0.1:57406 - "POST /node/694080fb575bc060ce15c977 HTTP/1.0" 202 Accepted 2025-12-17 21:43:35,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e3575bc060ce15c952 2025-12-17 21:43:35,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e3575bc060ce15c952 HTTP/1.1" 502 150 2025-12-17 21:43:35,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e3575bc060ce15c952'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57416 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:43:36,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080aa575bc060ce15c8c4 2025-12-17 21:43:36,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080aa575bc060ce15c8c4 HTTP/1.1" 502 150 2025-12-17 21:43:36,689 - lava_callback - ERROR - Error processing callback for node 694080aa575bc060ce15c8c4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080aa575bc060ce15c8c4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080aa575bc060ce15c8c4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080aa575bc060ce15c8c4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:36,691 - lava_callback - INFO - Starting async job submit for node 694080e4575bc060ce15c955 2025-12-17 21:43:36,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:36,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e4575bc060ce15c955 HTTP/1.1" 502 150 2025-12-17 21:43:36,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e4575bc060ce15c955'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:36,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e4575bc060ce15c955 2025-12-17 21:43:36,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e4575bc060ce15c955 HTTP/1.1" 502 150 2025-12-17 21:43:36,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e4575bc060ce15c955'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:37,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ee575bc060ce15c967 2025-12-17 21:43:37,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ee575bc060ce15c967 HTTP/1.1" 502 150 2025-12-17 21:43:37,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ee575bc060ce15c967'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:37,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ae575bc060ce15c8d3 2025-12-17 21:43:37,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ae575bc060ce15c8d3 HTTP/1.1" 502 150 2025-12-17 21:43:37,844 - lava_callback - ERROR - Error processing callback for node 694080ae575bc060ce15c8d3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ae575bc060ce15c8d3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ae575bc060ce15c8d3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ae575bc060ce15c8d3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:37,846 - lava_callback - INFO - Starting async job submit for node 694080e6575bc060ce15c958 2025-12-17 21:43:37,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:37,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e6575bc060ce15c958 HTTP/1.1" 502 150 2025-12-17 21:43:37,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e6575bc060ce15c958'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:37,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e6575bc060ce15c958 2025-12-17 21:43:37,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e6575bc060ce15c958 HTTP/1.1" 502 150 2025-12-17 21:43:37,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e6575bc060ce15c958'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:37,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080af575bc060ce15c8d6 2025-12-17 21:43:37,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080af575bc060ce15c8d6 HTTP/1.1" 502 150 2025-12-17 21:43:37,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080af575bc060ce15c8d6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:38,248 - lava_callback - INFO - Received callback for node 694080fb575bc060ce15c972 INFO: 172.21.0.1:57430 - "POST /node/694080fb575bc060ce15c972 HTTP/1.0" 202 Accepted 2025-12-17 21:43:38,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ea575bc060ce15c961 2025-12-17 21:43:38,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ea575bc060ce15c961 HTTP/1.1" 502 150 2025-12-17 21:43:38,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ea575bc060ce15c961'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:38,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e4575bc060ce15c955 2025-12-17 21:43:38,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e4575bc060ce15c955 HTTP/1.1" 502 150 2025-12-17 21:43:38,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e4575bc060ce15c955'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:39,128 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ed575bc060ce15c964 2025-12-17 21:43:39,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ed575bc060ce15c964 HTTP/1.1" 502 150 2025-12-17 21:43:39,129 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ed575bc060ce15c964'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:39,252 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e2575bc060ce15c94f 2025-12-17 21:43:39,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e2575bc060ce15c94f HTTP/1.1" 502 150 2025-12-17 21:43:39,253 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e2575bc060ce15c94f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:39,373 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080b2575bc060ce15c8dc 2025-12-17 21:43:39,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080b2575bc060ce15c8dc HTTP/1.1" 502 150 2025-12-17 21:43:39,375 - lava_callback - ERROR - Error processing callback for node 694080b2575bc060ce15c8dc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080b2575bc060ce15c8dc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080b2575bc060ce15c8dc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080b2575bc060ce15c8dc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:39,378 - lava_callback - INFO - Starting async job submit for node 694080ac575bc060ce15c8d0 2025-12-17 21:43:39,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:39,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ac575bc060ce15c8d0 HTTP/1.1" 502 150 2025-12-17 21:43:39,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ac575bc060ce15c8d0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:39,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ac575bc060ce15c8d0 2025-12-17 21:43:39,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ac575bc060ce15c8d0 HTTP/1.1" 502 150 2025-12-17 21:43:39,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ac575bc060ce15c8d0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:39,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e6575bc060ce15c958 2025-12-17 21:43:39,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e6575bc060ce15c958 HTTP/1.1" 502 150 2025-12-17 21:43:39,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e6575bc060ce15c958'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:39,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e3575bc060ce15c952 2025-12-17 21:43:39,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e3575bc060ce15c952 HTTP/1.1" 502 150 2025-12-17 21:43:39,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e3575bc060ce15c952'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:39,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080a5575bc060ce15c8b9 HTTP/1.1" 502 150 2025-12-17 21:43:39,952 - lava_callback - ERROR - Error processing callback for node 694080a5575bc060ce15c8b9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a5575bc060ce15c8b9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a5575bc060ce15c8b9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080a5575bc060ce15c8b9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:39,955 - lava_callback - INFO - Starting async job submit for node 694080e7575bc060ce15c95b 2025-12-17 21:43:39,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:39,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e7575bc060ce15c95b HTTP/1.1" 502 150 2025-12-17 21:43:39,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e7575bc060ce15c95b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:39,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e7575bc060ce15c95b 2025-12-17 21:43:39,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e7575bc060ce15c95b HTTP/1.1" 502 150 2025-12-17 21:43:39,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e7575bc060ce15c95b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:41,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ac575bc060ce15c8d0 2025-12-17 21:43:41,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ac575bc060ce15c8d0 HTTP/1.1" 502 150 2025-12-17 21:43:41,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ac575bc060ce15c8d0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:41,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e7575bc060ce15c95b 2025-12-17 21:43:41,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e7575bc060ce15c95b HTTP/1.1" 502 150 2025-12-17 21:43:41,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e7575bc060ce15c95b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:42,582 - lava_callback - INFO - Received callback for node 6940810d575bc060ce15c992 INFO: 172.21.0.1:35100 - "POST /node/6940810d575bc060ce15c992 HTTP/1.0" 202 Accepted 2025-12-17 21:43:42,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e4575bc060ce15c955 2025-12-17 21:43:42,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e4575bc060ce15c955 HTTP/1.1" 502 150 2025-12-17 21:43:42,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e4575bc060ce15c955'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:43,318 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080bf575bc060ce15c8f0 2025-12-17 21:43:43,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080bf575bc060ce15c8f0 HTTP/1.1" 502 150 2025-12-17 21:43:43,319 - lava_callback - ERROR - Error processing callback for node 694080bf575bc060ce15c8f0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080bf575bc060ce15c8f0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080bf575bc060ce15c8f0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080bf575bc060ce15c8f0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:43,321 - lava_callback - INFO - Starting async job submit for node 694080fd575bc060ce15c97e 2025-12-17 21:43:43,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:43,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fd575bc060ce15c97e HTTP/1.1" 502 150 2025-12-17 21:43:43,328 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fd575bc060ce15c97e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:43,328 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fd575bc060ce15c97e 2025-12-17 21:43:43,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fd575bc060ce15c97e HTTP/1.1" 502 150 2025-12-17 21:43:43,329 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fd575bc060ce15c97e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:43,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080df575bc060ce15c949 2025-12-17 21:43:43,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080df575bc060ce15c949 HTTP/1.1" 502 150 2025-12-17 21:43:43,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080df575bc060ce15c949'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:43,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e6575bc060ce15c958 2025-12-17 21:43:43,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e6575bc060ce15c958 HTTP/1.1" 502 150 2025-12-17 21:43:43,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e6575bc060ce15c958'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:45,329 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fd575bc060ce15c97e 2025-12-17 21:43:45,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fd575bc060ce15c97e HTTP/1.1" 502 150 2025-12-17 21:43:45,330 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fd575bc060ce15c97e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:45,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ac575bc060ce15c8d0 2025-12-17 21:43:45,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ac575bc060ce15c8d0 HTTP/1.1" 502 150 2025-12-17 21:43:45,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ac575bc060ce15c8d0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:45,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ee575bc060ce15c967 2025-12-17 21:43:45,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ee575bc060ce15c967 HTTP/1.1" 502 150 2025-12-17 21:43:45,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ee575bc060ce15c967'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:45,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694076dc575bc060ce15b1f9 2025-12-17 21:43:45,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694076dc575bc060ce15b1f9 HTTP/1.1" 502 150 2025-12-17 21:43:45,600 - lava_callback - ERROR - Error processing callback for node 694076dc575bc060ce15b1f9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076dc575bc060ce15b1f9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076dc575bc060ce15b1f9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694076dc575bc060ce15b1f9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:45,603 - lava_callback - INFO - Starting async job submit for node 694080fe575bc060ce15c983 2025-12-17 21:43:45,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:45,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fe575bc060ce15c983 HTTP/1.1" 502 150 2025-12-17 21:43:45,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fe575bc060ce15c983'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:45,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fe575bc060ce15c983 2025-12-17 21:43:45,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fe575bc060ce15c983 HTTP/1.1" 502 150 2025-12-17 21:43:45,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fe575bc060ce15c983'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:45,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e7575bc060ce15c95b 2025-12-17 21:43:45,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e7575bc060ce15c95b HTTP/1.1" 502 150 2025-12-17 21:43:45,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e7575bc060ce15c95b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35112 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:43:46,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ea575bc060ce15c961 2025-12-17 21:43:46,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ea575bc060ce15c961 HTTP/1.1" 502 150 2025-12-17 21:43:46,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ea575bc060ce15c961'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:47,129 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ed575bc060ce15c964 2025-12-17 21:43:47,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ed575bc060ce15c964 HTTP/1.1" 502 150 2025-12-17 21:43:47,130 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ed575bc060ce15c964'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:47,253 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e2575bc060ce15c94f 2025-12-17 21:43:47,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e2575bc060ce15c94f HTTP/1.1" 502 150 2025-12-17 21:43:47,254 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e2575bc060ce15c94f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:47,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fe575bc060ce15c983 2025-12-17 21:43:47,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fe575bc060ce15c983 HTTP/1.1" 502 150 2025-12-17 21:43:47,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fe575bc060ce15c983'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:47,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e3575bc060ce15c952 2025-12-17 21:43:47,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e3575bc060ce15c952 HTTP/1.1" 502 150 2025-12-17 21:43:47,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e3575bc060ce15c952'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:48,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080d8575bc060ce15c909 2025-12-17 21:43:48,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080d8575bc060ce15c909 HTTP/1.1" 502 150 2025-12-17 21:43:48,775 - lava_callback - ERROR - Error processing callback for node 694080d8575bc060ce15c909: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080d8575bc060ce15c909 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080d8575bc060ce15c909 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080d8575bc060ce15c909 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:48,778 - lava_callback - INFO - Starting async job submit for node 694080ff575bc060ce15c98a 2025-12-17 21:43:48,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:48,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ff575bc060ce15c98a HTTP/1.1" 502 150 2025-12-17 21:43:48,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ff575bc060ce15c98a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:48,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ff575bc060ce15c98a 2025-12-17 21:43:48,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ff575bc060ce15c98a HTTP/1.1" 502 150 2025-12-17 21:43:48,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ff575bc060ce15c98a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:49,330 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fd575bc060ce15c97e 2025-12-17 21:43:49,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fd575bc060ce15c97e HTTP/1.1" 502 150 2025-12-17 21:43:49,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fd575bc060ce15c97e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:49,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080dc575bc060ce15c91c 2025-12-17 21:43:49,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080dc575bc060ce15c91c HTTP/1.1" 502 150 2025-12-17 21:43:49,855 - lava_callback - ERROR - Error processing callback for node 694080dc575bc060ce15c91c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080dc575bc060ce15c91c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080dc575bc060ce15c91c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080dc575bc060ce15c91c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:49,857 - lava_callback - INFO - Starting async job submit for node 69408101575bc060ce15c98d 2025-12-17 21:43:49,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:49,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408101575bc060ce15c98d HTTP/1.1" 502 150 2025-12-17 21:43:49,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408101575bc060ce15c98d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:49,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408101575bc060ce15c98d 2025-12-17 21:43:49,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408101575bc060ce15c98d HTTP/1.1" 502 150 2025-12-17 21:43:49,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408101575bc060ce15c98d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:49,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080de575bc060ce15c946 2025-12-17 21:43:49,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080de575bc060ce15c946 HTTP/1.1" 502 150 2025-12-17 21:43:49,990 - lava_callback - ERROR - Error processing callback for node 694080de575bc060ce15c946: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080de575bc060ce15c946 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080de575bc060ce15c946 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080de575bc060ce15c946 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:49,991 - lava_callback - INFO - Starting async job submit for node 694080fb575bc060ce15c977 2025-12-17 21:43:49,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:49,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fb575bc060ce15c977 HTTP/1.1" 502 150 2025-12-17 21:43:49,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fb575bc060ce15c977'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:49,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fb575bc060ce15c977 2025-12-17 21:43:49,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fb575bc060ce15c977 HTTP/1.1" 502 150 2025-12-17 21:43:49,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fb575bc060ce15c977'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:50,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e4575bc060ce15c955 2025-12-17 21:43:50,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e4575bc060ce15c955 HTTP/1.1" 502 150 2025-12-17 21:43:50,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e4575bc060ce15c955'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:50,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ff575bc060ce15c98a 2025-12-17 21:43:50,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ff575bc060ce15c98a HTTP/1.1" 502 150 2025-12-17 21:43:50,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ff575bc060ce15c98a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:51,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fe575bc060ce15c983 2025-12-17 21:43:51,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fe575bc060ce15c983 HTTP/1.1" 502 150 2025-12-17 21:43:51,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fe575bc060ce15c983'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:51,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408101575bc060ce15c98d 2025-12-17 21:43:51,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408101575bc060ce15c98d HTTP/1.1" 502 150 2025-12-17 21:43:51,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408101575bc060ce15c98d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:51,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e6575bc060ce15c958 2025-12-17 21:43:51,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e6575bc060ce15c958 HTTP/1.1" 502 150 2025-12-17 21:43:51,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e6575bc060ce15c958'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:51,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fb575bc060ce15c977 2025-12-17 21:43:51,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fb575bc060ce15c977 HTTP/1.1" 502 150 2025-12-17 21:43:51,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fb575bc060ce15c977'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:53,100 - lava_callback - INFO - Received callback for node 69408111575bc060ce15c9a8 INFO: 172.21.0.1:32768 - "POST /node/69408111575bc060ce15c9a8 HTTP/1.0" 202 Accepted 2025-12-17 21:43:53,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ac575bc060ce15c8d0 2025-12-17 21:43:53,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ac575bc060ce15c8d0 HTTP/1.1" 502 150 2025-12-17 21:43:53,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ac575bc060ce15c8d0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:53,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080af575bc060ce15c8d6 2025-12-17 21:43:53,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080af575bc060ce15c8d6 HTTP/1.1" 502 150 2025-12-17 21:43:53,876 - lava_callback - ERROR - Error processing callback for node 694080af575bc060ce15c8d6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080af575bc060ce15c8d6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080af575bc060ce15c8d6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080af575bc060ce15c8d6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:53,878 - lava_callback - INFO - Starting async job submit for node 694080fb575bc060ce15c972 2025-12-17 21:43:53,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:53,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fb575bc060ce15c972 HTTP/1.1" 502 150 2025-12-17 21:43:53,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fb575bc060ce15c972'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:53,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fb575bc060ce15c972 2025-12-17 21:43:53,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fb575bc060ce15c972 HTTP/1.1" 502 150 2025-12-17 21:43:53,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fb575bc060ce15c972'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:53,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e7575bc060ce15c95b 2025-12-17 21:43:53,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e7575bc060ce15c95b HTTP/1.1" 502 150 2025-12-17 21:43:53,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e7575bc060ce15c95b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:54,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ff575bc060ce15c98a 2025-12-17 21:43:54,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ff575bc060ce15c98a HTTP/1.1" 502 150 2025-12-17 21:43:54,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ff575bc060ce15c98a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:55,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408101575bc060ce15c98d 2025-12-17 21:43:55,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408101575bc060ce15c98d HTTP/1.1" 502 150 2025-12-17 21:43:55,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408101575bc060ce15c98d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:55,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fb575bc060ce15c972 2025-12-17 21:43:55,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fb575bc060ce15c972 HTTP/1.1" 502 150 2025-12-17 21:43:55,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fb575bc060ce15c972'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:56,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fb575bc060ce15c977 2025-12-17 21:43:56,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fb575bc060ce15c977 HTTP/1.1" 502 150 2025-12-17 21:43:56,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fb575bc060ce15c977'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:32782 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:43:57,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fd575bc060ce15c97e 2025-12-17 21:43:57,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fd575bc060ce15c97e HTTP/1.1" 502 150 2025-12-17 21:43:57,333 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fd575bc060ce15c97e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:57,948 - lava_callback - INFO - Received callback for node 69408110575bc060ce15c9a3 INFO: 172.21.0.1:32794 - "POST /node/69408110575bc060ce15c9a3 HTTP/1.0" 202 Accepted 2025-12-17 21:43:58,730 - lava_callback - INFO - Received callback for node 69408114575bc060ce15c9ab INFO: 172.21.0.1:32802 - "POST /node/69408114575bc060ce15c9ab HTTP/1.0" 202 Accepted 2025-12-17 21:43:59,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080df575bc060ce15c949 2025-12-17 21:43:59,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080df575bc060ce15c949 HTTP/1.1" 502 150 2025-12-17 21:43:59,548 - lava_callback - ERROR - Error processing callback for node 694080df575bc060ce15c949: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080df575bc060ce15c949 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080df575bc060ce15c949 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080df575bc060ce15c949 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:43:59,550 - lava_callback - INFO - Starting async job submit for node 6940810d575bc060ce15c992 2025-12-17 21:43:59,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:43:59,563 - lava_callback - INFO - Received callback for node 69408115575bc060ce15c9ae 2025-12-17 21:43:59,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940810d575bc060ce15c992 HTTP/1.1" 502 150 INFO: 172.21.0.1:32810 - "POST /node/69408115575bc060ce15c9ae HTTP/1.0" 202 Accepted 2025-12-17 21:43:59,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940810d575bc060ce15c992'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:59,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940810d575bc060ce15c992 2025-12-17 21:43:59,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940810d575bc060ce15c992 HTTP/1.1" 502 150 2025-12-17 21:43:59,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940810d575bc060ce15c992'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:59,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fe575bc060ce15c983 2025-12-17 21:43:59,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fe575bc060ce15c983 HTTP/1.1" 502 150 2025-12-17 21:43:59,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fe575bc060ce15c983'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:43:59,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fb575bc060ce15c972 2025-12-17 21:43:59,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fb575bc060ce15c972 HTTP/1.1" 502 150 2025-12-17 21:43:59,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fb575bc060ce15c972'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:00,352 - lava_callback - INFO - Received callback for node 69408116575bc060ce15c9b5 INFO: 172.21.0.1:45188 - "POST /node/69408116575bc060ce15c9b5 HTTP/1.0" 202 Accepted 2025-12-17 21:44:01,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ee575bc060ce15c967 2025-12-17 21:44:01,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ee575bc060ce15c967 HTTP/1.1" 502 150 2025-12-17 21:44:01,519 - lava_callback - ERROR - Error processing callback for node 694080ee575bc060ce15c967: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ee575bc060ce15c967 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ee575bc060ce15c967 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ee575bc060ce15c967 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:01,521 - lava_callback - INFO - Starting async job submit for node 69408111575bc060ce15c9a8 2025-12-17 21:44:01,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:01,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408111575bc060ce15c9a8 HTTP/1.1" 502 150 2025-12-17 21:44:01,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408111575bc060ce15c9a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:01,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408111575bc060ce15c9a8 2025-12-17 21:44:01,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408111575bc060ce15c9a8 HTTP/1.1" 502 150 2025-12-17 21:44:01,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408111575bc060ce15c9a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:01,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940810d575bc060ce15c992 2025-12-17 21:44:01,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940810d575bc060ce15c992 HTTP/1.1" 502 150 2025-12-17 21:44:01,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940810d575bc060ce15c992'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45204 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:44:02,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ea575bc060ce15c961 2025-12-17 21:44:02,610 - lava_callback - INFO - Received callback for node 6940810e575bc060ce15c99c INFO: 172.21.0.1:45218 - "POST /node/6940810e575bc060ce15c99c HTTP/1.0" 202 Accepted 2025-12-17 21:44:02,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ff575bc060ce15c98a 2025-12-17 21:44:02,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ff575bc060ce15c98a HTTP/1.1" 502 150 2025-12-17 21:44:02,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080ff575bc060ce15c98a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:03,131 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ed575bc060ce15c964 2025-12-17 21:44:03,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ed575bc060ce15c964 HTTP/1.1" 502 150 2025-12-17 21:44:03,132 - lava_callback - ERROR - Error processing callback for node 694080ed575bc060ce15c964: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ed575bc060ce15c964 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ed575bc060ce15c964 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ed575bc060ce15c964 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:03,134 - lava_callback - INFO - Starting async job submit for node 69408110575bc060ce15c9a3 2025-12-17 21:44:03,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:03,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408110575bc060ce15c9a3 HTTP/1.1" 502 150 2025-12-17 21:44:03,148 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408110575bc060ce15c9a3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:03,148 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408110575bc060ce15c9a3 2025-12-17 21:44:03,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408110575bc060ce15c9a3 HTTP/1.1" 502 150 2025-12-17 21:44:03,149 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408110575bc060ce15c9a3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:03,254 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e2575bc060ce15c94f 2025-12-17 21:44:03,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e2575bc060ce15c94f HTTP/1.1" 502 150 2025-12-17 21:44:03,257 - lava_callback - ERROR - Error processing callback for node 694080e2575bc060ce15c94f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e2575bc060ce15c94f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e2575bc060ce15c94f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e2575bc060ce15c94f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:03,259 - lava_callback - INFO - Starting async job submit for node 69408114575bc060ce15c9ab 2025-12-17 21:44:03,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:03,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408114575bc060ce15c9ab HTTP/1.1" 502 150 2025-12-17 21:44:03,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408114575bc060ce15c9ab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:03,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408114575bc060ce15c9ab 2025-12-17 21:44:03,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408114575bc060ce15c9ab HTTP/1.1" 502 150 2025-12-17 21:44:03,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408114575bc060ce15c9ab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:03,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408111575bc060ce15c9a8 2025-12-17 21:44:03,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408111575bc060ce15c9a8 HTTP/1.1" 502 150 2025-12-17 21:44:03,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408111575bc060ce15c9a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:03,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408101575bc060ce15c98d 2025-12-17 21:44:03,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408101575bc060ce15c98d HTTP/1.1" 502 150 2025-12-17 21:44:03,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408101575bc060ce15c98d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:03,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e3575bc060ce15c952 2025-12-17 21:44:03,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e3575bc060ce15c952 HTTP/1.1" 502 150 2025-12-17 21:44:03,930 - lava_callback - ERROR - Error processing callback for node 694080e3575bc060ce15c952: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e3575bc060ce15c952 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e3575bc060ce15c952 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e3575bc060ce15c952 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:03,932 - lava_callback - INFO - Starting async job submit for node 69408115575bc060ce15c9ae 2025-12-17 21:44:03,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:03,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408115575bc060ce15c9ae HTTP/1.1" 502 150 2025-12-17 21:44:03,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408115575bc060ce15c9ae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:03,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408115575bc060ce15c9ae 2025-12-17 21:44:03,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408115575bc060ce15c9ae HTTP/1.1" 502 150 2025-12-17 21:44:03,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408115575bc060ce15c9ae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:04,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fb575bc060ce15c977 2025-12-17 21:44:04,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fb575bc060ce15c977 HTTP/1.1" 502 150 2025-12-17 21:44:04,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fb575bc060ce15c977'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:05,149 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408110575bc060ce15c9a3 2025-12-17 21:44:05,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408110575bc060ce15c9a3 HTTP/1.1" 502 150 2025-12-17 21:44:05,150 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408110575bc060ce15c9a3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:05,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408114575bc060ce15c9ab 2025-12-17 21:44:05,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408114575bc060ce15c9ab HTTP/1.1" 502 150 2025-12-17 21:44:05,267 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408114575bc060ce15c9ab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:05,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940810d575bc060ce15c992 2025-12-17 21:44:05,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940810d575bc060ce15c992 HTTP/1.1" 502 150 2025-12-17 21:44:05,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940810d575bc060ce15c992'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:05,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408115575bc060ce15c9ae 2025-12-17 21:44:05,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408115575bc060ce15c9ae HTTP/1.1" 502 150 2025-12-17 21:44:05,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408115575bc060ce15c9ae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45224 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:44:06,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e4575bc060ce15c955 2025-12-17 21:44:06,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e4575bc060ce15c955 HTTP/1.1" 502 150 2025-12-17 21:44:06,738 - lava_callback - ERROR - Error processing callback for node 694080e4575bc060ce15c955: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e4575bc060ce15c955 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e4575bc060ce15c955 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e4575bc060ce15c955 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:06,740 - lava_callback - INFO - Starting async job submit for node 69408116575bc060ce15c9b5 2025-12-17 21:44:06,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:06,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408116575bc060ce15c9b5 HTTP/1.1" 502 150 2025-12-17 21:44:06,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408116575bc060ce15c9b5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:06,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408116575bc060ce15c9b5 2025-12-17 21:44:06,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408116575bc060ce15c9b5 HTTP/1.1" 502 150 2025-12-17 21:44:06,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408116575bc060ce15c9b5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:07,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408111575bc060ce15c9a8 2025-12-17 21:44:07,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408111575bc060ce15c9a8 HTTP/1.1" 502 150 2025-12-17 21:44:07,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408111575bc060ce15c9a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:07,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e6575bc060ce15c958 2025-12-17 21:44:07,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e6575bc060ce15c958 HTTP/1.1" 502 150 2025-12-17 21:44:07,872 - lava_callback - ERROR - Error processing callback for node 694080e6575bc060ce15c958: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e6575bc060ce15c958 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e6575bc060ce15c958 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e6575bc060ce15c958 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:07,874 - lava_callback - INFO - Starting async job submit for node 6940810e575bc060ce15c99c 2025-12-17 21:44:07,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:07,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940810e575bc060ce15c99c HTTP/1.1" 502 150 2025-12-17 21:44:07,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940810e575bc060ce15c99c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:07,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940810e575bc060ce15c99c 2025-12-17 21:44:07,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940810e575bc060ce15c99c HTTP/1.1" 502 150 2025-12-17 21:44:07,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940810e575bc060ce15c99c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:07,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fb575bc060ce15c972 2025-12-17 21:44:07,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fb575bc060ce15c972 HTTP/1.1" 502 150 2025-12-17 21:44:07,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080fb575bc060ce15c972'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:08,063 - lava_callback - INFO - Received callback for node 69408117575bc060ce15c9ba INFO: 172.21.0.1:45228 - "POST /node/69408117575bc060ce15c9ba HTTP/1.0" 202 Accepted 2025-12-17 21:44:08,718 - lava_callback - INFO - Received callback for node 69408118575bc060ce15c9c1 INFO: 172.21.0.1:45244 - "POST /node/69408118575bc060ce15c9c1 HTTP/1.0" 202 Accepted 2025-12-17 21:44:08,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408116575bc060ce15c9b5 2025-12-17 21:44:08,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408116575bc060ce15c9b5 HTTP/1.1" 502 150 2025-12-17 21:44:08,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408116575bc060ce15c9b5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:09,150 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408110575bc060ce15c9a3 2025-12-17 21:44:09,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408110575bc060ce15c9a3 HTTP/1.1" 502 150 2025-12-17 21:44:09,152 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408110575bc060ce15c9a3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:09,268 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408114575bc060ce15c9ab 2025-12-17 21:44:09,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408114575bc060ce15c9ab HTTP/1.1" 502 150 2025-12-17 21:44:09,269 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408114575bc060ce15c9ab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:09,377 - lava_callback - INFO - Received callback for node 69408119575bc060ce15c9c6 INFO: 172.21.0.1:45252 - "POST /node/69408119575bc060ce15c9c6 HTTP/1.0" 202 Accepted 2025-12-17 21:44:09,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ac575bc060ce15c8d0 2025-12-17 21:44:09,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ac575bc060ce15c8d0 HTTP/1.1" 502 150 2025-12-17 21:44:09,487 - lava_callback - ERROR - Error processing callback for node 694080ac575bc060ce15c8d0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ac575bc060ce15c8d0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ac575bc060ce15c8d0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ac575bc060ce15c8d0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:09,489 - lava_callback - INFO - Starting async job submit for node 69408117575bc060ce15c9ba 2025-12-17 21:44:09,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:09,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408117575bc060ce15c9ba HTTP/1.1" 502 150 2025-12-17 21:44:09,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408117575bc060ce15c9ba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:09,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408117575bc060ce15c9ba 2025-12-17 21:44:09,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408117575bc060ce15c9ba HTTP/1.1" 502 150 2025-12-17 21:44:09,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408117575bc060ce15c9ba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:09,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940810e575bc060ce15c99c 2025-12-17 21:44:09,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940810e575bc060ce15c99c HTTP/1.1" 502 150 2025-12-17 21:44:09,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940810e575bc060ce15c99c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:09,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408115575bc060ce15c9ae 2025-12-17 21:44:09,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408115575bc060ce15c9ae HTTP/1.1" 502 150 2025-12-17 21:44:09,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408115575bc060ce15c9ae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:09,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e7575bc060ce15c95b 2025-12-17 21:44:09,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e7575bc060ce15c95b HTTP/1.1" 502 150 2025-12-17 21:44:09,975 - lava_callback - ERROR - Error processing callback for node 694080e7575bc060ce15c95b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e7575bc060ce15c95b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e7575bc060ce15c95b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e7575bc060ce15c95b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:09,977 - lava_callback - INFO - Starting async job submit for node 69408118575bc060ce15c9c1 2025-12-17 21:44:09,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:09,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408118575bc060ce15c9c1 HTTP/1.1" 502 150 2025-12-17 21:44:09,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408118575bc060ce15c9c1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:09,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408118575bc060ce15c9c1 2025-12-17 21:44:09,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408118575bc060ce15c9c1 HTTP/1.1" 502 150 2025-12-17 21:44:09,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408118575bc060ce15c9c1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:10,077 - lava_callback - INFO - Received callback for node 6940811b575bc060ce15c9f9 INFO: 172.21.0.1:55356 - "POST /node/6940811b575bc060ce15c9f9 HTTP/1.0" 202 Accepted 2025-12-17 21:44:10,785 - lava_callback - INFO - Received callback for node 6940811d575bc060ce15c9fc INFO: 172.21.0.1:55364 - "POST /node/6940811d575bc060ce15c9fc HTTP/1.0" 202 Accepted 2025-12-17 21:44:11,457 - lava_callback - INFO - Received callback for node 6940811e575bc060ce15c9ff INFO: 172.21.0.1:55380 - "POST /node/6940811e575bc060ce15c9ff HTTP/1.0" 202 Accepted 2025-12-17 21:44:11,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408117575bc060ce15c9ba 2025-12-17 21:44:11,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408117575bc060ce15c9ba HTTP/1.1" 502 150 2025-12-17 21:44:11,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408117575bc060ce15c9ba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:11,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408118575bc060ce15c9c1 2025-12-17 21:44:11,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408118575bc060ce15c9c1 HTTP/1.1" 502 150 2025-12-17 21:44:11,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408118575bc060ce15c9c1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:12,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408116575bc060ce15c9b5 2025-12-17 21:44:12,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408116575bc060ce15c9b5 HTTP/1.1" 502 150 2025-12-17 21:44:12,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408116575bc060ce15c9b5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:13,055 - lava_callback - INFO - Received callback for node 6940811f575bc060ce15ca02 INFO: 172.21.0.1:55394 - "POST /node/6940811f575bc060ce15ca02 HTTP/1.0" 202 Accepted 2025-12-17 21:44:13,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fd575bc060ce15c97e 2025-12-17 21:44:13,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fd575bc060ce15c97e HTTP/1.1" 502 150 2025-12-17 21:44:13,335 - lava_callback - ERROR - Error processing callback for node 694080fd575bc060ce15c97e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080fd575bc060ce15c97e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080fd575bc060ce15c97e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080fd575bc060ce15c97e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:13,338 - lava_callback - INFO - Starting async job submit for node 69408119575bc060ce15c9c6 2025-12-17 21:44:13,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:13,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408119575bc060ce15c9c6 HTTP/1.1" 502 150 2025-12-17 21:44:13,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408119575bc060ce15c9c6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:13,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408119575bc060ce15c9c6 2025-12-17 21:44:13,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408119575bc060ce15c9c6 HTTP/1.1" 502 150 2025-12-17 21:44:13,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408119575bc060ce15c9c6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:13,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940810d575bc060ce15c992 2025-12-17 21:44:13,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940810d575bc060ce15c992 HTTP/1.1" 502 150 2025-12-17 21:44:13,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940810d575bc060ce15c992'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:13,726 - lava_callback - INFO - Received callback for node 69408121575bc060ce15ca07 INFO: 172.21.0.1:55402 - "POST /node/69408121575bc060ce15ca07 HTTP/1.0" 202 Accepted 2025-12-17 21:44:13,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940810e575bc060ce15c99c 2025-12-17 21:44:13,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940810e575bc060ce15c99c HTTP/1.1" 502 150 2025-12-17 21:44:13,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940810e575bc060ce15c99c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:14,386 - lava_callback - INFO - Received callback for node 69408122575bc060ce15ca0a INFO: 172.21.0.1:55412 - "POST /node/69408122575bc060ce15ca0a HTTP/1.0" 202 Accepted INFO: 172.21.0.1:55414 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:44:15,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408119575bc060ce15c9c6 2025-12-17 21:44:15,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408119575bc060ce15c9c6 HTTP/1.1" 502 150 2025-12-17 21:44:15,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408119575bc060ce15c9c6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:15,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408117575bc060ce15c9ba 2025-12-17 21:44:15,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408117575bc060ce15c9ba HTTP/1.1" 502 150 2025-12-17 21:44:15,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408117575bc060ce15c9ba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:15,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408111575bc060ce15c9a8 2025-12-17 21:44:15,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408111575bc060ce15c9a8 HTTP/1.1" 502 150 2025-12-17 21:44:15,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408111575bc060ce15c9a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:15,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fe575bc060ce15c983 2025-12-17 21:44:15,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fe575bc060ce15c983 HTTP/1.1" 502 150 2025-12-17 21:44:15,637 - lava_callback - ERROR - Error processing callback for node 694080fe575bc060ce15c983: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080fe575bc060ce15c983 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080fe575bc060ce15c983 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080fe575bc060ce15c983 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:15,639 - lava_callback - INFO - Starting async job submit for node 6940811b575bc060ce15c9f9 2025-12-17 21:44:15,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:15,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811b575bc060ce15c9f9 HTTP/1.1" 502 150 2025-12-17 21:44:15,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811b575bc060ce15c9f9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:15,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811b575bc060ce15c9f9 2025-12-17 21:44:15,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811b575bc060ce15c9f9 HTTP/1.1" 502 150 2025-12-17 21:44:15,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811b575bc060ce15c9f9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:15,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408118575bc060ce15c9c1 2025-12-17 21:44:15,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408118575bc060ce15c9c1 HTTP/1.1" 502 150 2025-12-17 21:44:15,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408118575bc060ce15c9c1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55422 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:44:17,152 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408110575bc060ce15c9a3 2025-12-17 21:44:17,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408110575bc060ce15c9a3 HTTP/1.1" 502 150 2025-12-17 21:44:17,153 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408110575bc060ce15c9a3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:17,269 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408114575bc060ce15c9ab 2025-12-17 21:44:17,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408114575bc060ce15c9ab HTTP/1.1" 502 150 2025-12-17 21:44:17,270 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408114575bc060ce15c9ab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:17,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811b575bc060ce15c9f9 2025-12-17 21:44:17,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811b575bc060ce15c9f9 HTTP/1.1" 502 150 2025-12-17 21:44:17,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811b575bc060ce15c9f9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:17,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ea575bc060ce15c961 HTTP/1.1" 502 150 2025-12-17 21:44:17,840 - lava_callback - ERROR - Error processing callback for node 694080ea575bc060ce15c961: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ea575bc060ce15c961 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ea575bc060ce15c961 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ea575bc060ce15c961 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:17,842 - lava_callback - INFO - Starting async job submit for node 6940811d575bc060ce15c9fc 2025-12-17 21:44:17,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:17,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811d575bc060ce15c9fc HTTP/1.1" 502 150 2025-12-17 21:44:17,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811d575bc060ce15c9fc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:17,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811d575bc060ce15c9fc 2025-12-17 21:44:17,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811d575bc060ce15c9fc HTTP/1.1" 502 150 2025-12-17 21:44:17,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811d575bc060ce15c9fc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:17,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408115575bc060ce15c9ae 2025-12-17 21:44:17,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408115575bc060ce15c9ae HTTP/1.1" 502 150 2025-12-17 21:44:17,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408115575bc060ce15c9ae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:18,166 - lava_callback - INFO - Received callback for node 69408125575bc060ce15ca11 INFO: 172.21.0.1:55424 - "POST /node/69408125575bc060ce15ca11 HTTP/1.0" 202 Accepted 2025-12-17 21:44:18,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080ff575bc060ce15c98a 2025-12-17 21:44:18,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080ff575bc060ce15c98a HTTP/1.1" 502 150 2025-12-17 21:44:18,819 - lava_callback - ERROR - Error processing callback for node 694080ff575bc060ce15c98a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ff575bc060ce15c98a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ff575bc060ce15c98a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080ff575bc060ce15c98a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:18,821 - lava_callback - INFO - Starting async job submit for node 6940811e575bc060ce15c9ff 2025-12-17 21:44:18,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:18,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811e575bc060ce15c9ff HTTP/1.1" 502 150 2025-12-17 21:44:18,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811e575bc060ce15c9ff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:18,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811e575bc060ce15c9ff 2025-12-17 21:44:18,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811e575bc060ce15c9ff HTTP/1.1" 502 150 2025-12-17 21:44:18,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811e575bc060ce15c9ff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:18,974 - lava_callback - INFO - Received callback for node 69408126575bc060ce15ca14 INFO: 172.21.0.1:55432 - "POST /node/69408126575bc060ce15ca14 HTTP/1.0" 202 Accepted 2025-12-17 21:44:19,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408119575bc060ce15c9c6 2025-12-17 21:44:19,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811d575bc060ce15c9fc 2025-12-17 21:44:19,851 - lava_callback - INFO - Received callback for node 69408127575bc060ce15ca17 INFO: 172.21.0.1:46862 - "POST /node/69408127575bc060ce15ca17 HTTP/1.0" 202 Accepted 2025-12-17 21:44:19,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811d575bc060ce15c9fc HTTP/1.1" 502 150 2025-12-17 21:44:19,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811d575bc060ce15c9fc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:19,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408101575bc060ce15c98d 2025-12-17 21:44:19,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408101575bc060ce15c98d HTTP/1.1" 502 150 2025-12-17 21:44:19,875 - lava_callback - ERROR - Error processing callback for node 69408101575bc060ce15c98d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408101575bc060ce15c98d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408101575bc060ce15c98d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408101575bc060ce15c98d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:19,877 - lava_callback - INFO - Starting async job submit for node 6940811f575bc060ce15ca02 2025-12-17 21:44:19,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:19,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811f575bc060ce15ca02 HTTP/1.1" 502 150 2025-12-17 21:44:19,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811f575bc060ce15ca02'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:19,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811f575bc060ce15ca02 2025-12-17 21:44:19,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811f575bc060ce15ca02 HTTP/1.1" 502 150 2025-12-17 21:44:19,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811f575bc060ce15ca02'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:20,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fb575bc060ce15c977 2025-12-17 21:44:20,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fb575bc060ce15c977 HTTP/1.1" 502 150 2025-12-17 21:44:20,004 - lava_callback - ERROR - Error processing callback for node 694080fb575bc060ce15c977: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080fb575bc060ce15c977 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080fb575bc060ce15c977 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080fb575bc060ce15c977 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:20,006 - lava_callback - INFO - Starting async job submit for node 69408121575bc060ce15ca07 2025-12-17 21:44:20,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:20,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408121575bc060ce15ca07 HTTP/1.1" 502 150 2025-12-17 21:44:20,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408121575bc060ce15ca07'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:20,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408121575bc060ce15ca07 2025-12-17 21:44:20,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408121575bc060ce15ca07 HTTP/1.1" 502 150 2025-12-17 21:44:20,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408121575bc060ce15ca07'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:20,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408116575bc060ce15c9b5 2025-12-17 21:44:20,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408116575bc060ce15c9b5 HTTP/1.1" 502 150 2025-12-17 21:44:20,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408116575bc060ce15c9b5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:20,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811e575bc060ce15c9ff 2025-12-17 21:44:20,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811e575bc060ce15c9ff HTTP/1.1" 502 150 2025-12-17 21:44:20,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811e575bc060ce15c9ff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:21,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811b575bc060ce15c9f9 2025-12-17 21:44:21,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811b575bc060ce15c9f9 HTTP/1.1" 502 150 2025-12-17 21:44:21,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811b575bc060ce15c9f9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:21,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940810e575bc060ce15c99c 2025-12-17 21:44:21,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940810e575bc060ce15c99c HTTP/1.1" 502 150 2025-12-17 21:44:21,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940810e575bc060ce15c99c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:21,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811f575bc060ce15ca02 2025-12-17 21:44:21,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811f575bc060ce15ca02 HTTP/1.1" 502 150 2025-12-17 21:44:21,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811f575bc060ce15ca02'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:22,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408121575bc060ce15ca07 2025-12-17 21:44:22,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408121575bc060ce15ca07 HTTP/1.1" 502 150 2025-12-17 21:44:22,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408121575bc060ce15ca07'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:22,638 - lava_callback - INFO - Received callback for node 69408132575bc060ce15ca34 INFO: 172.21.0.1:46876 - "POST /node/69408132575bc060ce15ca34 HTTP/1.0" 202 Accepted 2025-12-17 21:44:23,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408117575bc060ce15c9ba 2025-12-17 21:44:23,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408117575bc060ce15c9ba HTTP/1.1" 502 150 2025-12-17 21:44:23,501 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408117575bc060ce15c9ba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:23,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811d575bc060ce15c9fc 2025-12-17 21:44:23,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811d575bc060ce15c9fc HTTP/1.1" 502 150 2025-12-17 21:44:23,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811d575bc060ce15c9fc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:23,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080fb575bc060ce15c972 2025-12-17 21:44:23,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080fb575bc060ce15c972 HTTP/1.1" 502 150 2025-12-17 21:44:23,914 - lava_callback - ERROR - Error processing callback for node 694080fb575bc060ce15c972: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080fb575bc060ce15c972 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080fb575bc060ce15c972 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080fb575bc060ce15c972 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:23,917 - lava_callback - INFO - Starting async job submit for node 69408122575bc060ce15ca0a 2025-12-17 21:44:23,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:23,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408122575bc060ce15ca0a HTTP/1.1" 502 150 2025-12-17 21:44:23,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408122575bc060ce15ca0a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:23,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408122575bc060ce15ca0a 2025-12-17 21:44:23,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408122575bc060ce15ca0a HTTP/1.1" 502 150 2025-12-17 21:44:23,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408122575bc060ce15ca0a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:23,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408118575bc060ce15c9c1 2025-12-17 21:44:23,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408118575bc060ce15c9c1 HTTP/1.1" 502 150 2025-12-17 21:44:23,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408118575bc060ce15c9c1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:24,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811e575bc060ce15c9ff 2025-12-17 21:44:24,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811e575bc060ce15c9ff HTTP/1.1" 502 150 2025-12-17 21:44:24,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811e575bc060ce15c9ff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:25,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811f575bc060ce15ca02 2025-12-17 21:44:25,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811f575bc060ce15ca02 HTTP/1.1" 502 150 2025-12-17 21:44:25,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811f575bc060ce15ca02'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:25,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408122575bc060ce15ca0a 2025-12-17 21:44:25,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408122575bc060ce15ca0a HTTP/1.1" 502 150 2025-12-17 21:44:25,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408122575bc060ce15ca0a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:26,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408121575bc060ce15ca07 2025-12-17 21:44:26,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408121575bc060ce15ca07 HTTP/1.1" 502 150 2025-12-17 21:44:26,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408121575bc060ce15ca07'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46882 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:44:27,920 - lava_callback - INFO - Received callback for node 69408134575bc060ce15ca37 INFO: 172.21.0.1:46890 - "POST /node/69408134575bc060ce15ca37 HTTP/1.0" 202 Accepted 2025-12-17 21:44:28,618 - lava_callback - INFO - Received callback for node 69408135575bc060ce15ca3a INFO: 172.21.0.1:46906 - "POST /node/69408135575bc060ce15ca3a HTTP/1.0" 202 Accepted 2025-12-17 21:44:29,285 - lava_callback - INFO - Received callback for node 69408136575bc060ce15ca3d INFO: 172.21.0.1:46908 - "POST /node/69408136575bc060ce15ca3d HTTP/1.0" 202 Accepted 2025-12-17 21:44:29,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940810d575bc060ce15c992 2025-12-17 21:44:29,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940810d575bc060ce15c992 HTTP/1.1" 502 150 2025-12-17 21:44:29,576 - lava_callback - ERROR - Error processing callback for node 6940810d575bc060ce15c992: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940810d575bc060ce15c992 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940810d575bc060ce15c992 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940810d575bc060ce15c992 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:29,578 - lava_callback - INFO - Starting async job submit for node 69408125575bc060ce15ca11 2025-12-17 21:44:29,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:29,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408125575bc060ce15ca11 HTTP/1.1" 502 150 2025-12-17 21:44:29,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408125575bc060ce15ca11'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:29,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408125575bc060ce15ca11 2025-12-17 21:44:29,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408125575bc060ce15ca11 HTTP/1.1" 502 150 2025-12-17 21:44:29,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408125575bc060ce15ca11'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:29,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811b575bc060ce15c9f9 2025-12-17 21:44:29,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811b575bc060ce15c9f9 HTTP/1.1" 502 150 2025-12-17 21:44:29,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811b575bc060ce15c9f9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:29,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408122575bc060ce15ca0a 2025-12-17 21:44:29,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408122575bc060ce15ca0a HTTP/1.1" 502 150 2025-12-17 21:44:29,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408122575bc060ce15ca0a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:31,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408111575bc060ce15c9a8 2025-12-17 21:44:31,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408111575bc060ce15c9a8 HTTP/1.1" 502 150 2025-12-17 21:44:31,534 - lava_callback - ERROR - Error processing callback for node 69408111575bc060ce15c9a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408111575bc060ce15c9a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408111575bc060ce15c9a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408111575bc060ce15c9a8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:31,536 - lava_callback - INFO - Starting async job submit for node 69408126575bc060ce15ca14 2025-12-17 21:44:31,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:31,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408126575bc060ce15ca14 HTTP/1.1" 502 150 2025-12-17 21:44:31,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408126575bc060ce15ca14'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:31,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408126575bc060ce15ca14 2025-12-17 21:44:31,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408126575bc060ce15ca14 HTTP/1.1" 502 150 2025-12-17 21:44:31,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408126575bc060ce15ca14'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:31,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408125575bc060ce15ca11 2025-12-17 21:44:31,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408125575bc060ce15ca11 HTTP/1.1" 502 150 2025-12-17 21:44:31,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408125575bc060ce15ca11'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:31,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811d575bc060ce15c9fc 2025-12-17 21:44:31,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811d575bc060ce15c9fc HTTP/1.1" 502 150 2025-12-17 21:44:31,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811d575bc060ce15c9fc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:32,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811e575bc060ce15c9ff 2025-12-17 21:44:32,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811e575bc060ce15c9ff HTTP/1.1" 502 150 2025-12-17 21:44:32,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811e575bc060ce15c9ff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:33,153 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408110575bc060ce15c9a3 2025-12-17 21:44:33,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408110575bc060ce15c9a3 HTTP/1.1" 502 150 2025-12-17 21:44:33,155 - lava_callback - ERROR - Error processing callback for node 69408110575bc060ce15c9a3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408110575bc060ce15c9a3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408110575bc060ce15c9a3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408110575bc060ce15c9a3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:33,157 - lava_callback - INFO - Starting async job submit for node 69408127575bc060ce15ca17 2025-12-17 21:44:33,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:33,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408127575bc060ce15ca17 HTTP/1.1" 502 150 2025-12-17 21:44:33,166 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408127575bc060ce15ca17'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:33,166 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408127575bc060ce15ca17 2025-12-17 21:44:33,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408127575bc060ce15ca17 HTTP/1.1" 502 150 2025-12-17 21:44:33,167 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408127575bc060ce15ca17'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:33,270 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408114575bc060ce15c9ab 2025-12-17 21:44:33,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408114575bc060ce15c9ab HTTP/1.1" 502 150 2025-12-17 21:44:33,272 - lava_callback - ERROR - Error processing callback for node 69408114575bc060ce15c9ab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408114575bc060ce15c9ab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408114575bc060ce15c9ab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408114575bc060ce15c9ab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:33,276 - lava_callback - INFO - Starting async job submit for node 69408132575bc060ce15ca34 2025-12-17 21:44:33,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:33,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408132575bc060ce15ca34 HTTP/1.1" 502 150 2025-12-17 21:44:33,289 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408132575bc060ce15ca34'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:33,289 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408132575bc060ce15ca34 2025-12-17 21:44:33,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408132575bc060ce15ca34 HTTP/1.1" 502 150 2025-12-17 21:44:33,291 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408132575bc060ce15ca34'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:33,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408126575bc060ce15ca14 2025-12-17 21:44:33,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408126575bc060ce15ca14 HTTP/1.1" 502 150 2025-12-17 21:44:33,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408126575bc060ce15ca14'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:33,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811f575bc060ce15ca02 2025-12-17 21:44:33,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811f575bc060ce15ca02 HTTP/1.1" 502 150 2025-12-17 21:44:33,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940811f575bc060ce15ca02'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:33,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408115575bc060ce15c9ae 2025-12-17 21:44:33,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408115575bc060ce15c9ae HTTP/1.1" 502 150 2025-12-17 21:44:33,971 - lava_callback - ERROR - Error processing callback for node 69408115575bc060ce15c9ae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408115575bc060ce15c9ae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408115575bc060ce15c9ae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408115575bc060ce15c9ae (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:33,973 - lava_callback - INFO - Starting async job submit for node 69408134575bc060ce15ca37 2025-12-17 21:44:33,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:33,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408134575bc060ce15ca37 HTTP/1.1" 502 150 2025-12-17 21:44:33,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408134575bc060ce15ca37'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:33,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408134575bc060ce15ca37 2025-12-17 21:44:33,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408134575bc060ce15ca37 HTTP/1.1" 502 150 2025-12-17 21:44:33,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408134575bc060ce15ca37'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:34,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408121575bc060ce15ca07 2025-12-17 21:44:34,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408119575bc060ce15c9c6 HTTP/1.1" 502 150 2025-12-17 21:44:34,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408119575bc060ce15c9c6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:35,167 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408127575bc060ce15ca17 2025-12-17 21:44:35,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408127575bc060ce15ca17 HTTP/1.1" 502 150 2025-12-17 21:44:35,168 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408127575bc060ce15ca17'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:35,291 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408132575bc060ce15ca34 2025-12-17 21:44:35,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408132575bc060ce15ca34 HTTP/1.1" 502 150 2025-12-17 21:44:35,292 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408132575bc060ce15ca34'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:35,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408125575bc060ce15ca11 2025-12-17 21:44:35,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408125575bc060ce15ca11 HTTP/1.1" 502 150 2025-12-17 21:44:35,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408125575bc060ce15ca11'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:35,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408134575bc060ce15ca37 2025-12-17 21:44:35,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408134575bc060ce15ca37 HTTP/1.1" 502 150 2025-12-17 21:44:35,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408134575bc060ce15ca37'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34166 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:44:36,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408116575bc060ce15c9b5 2025-12-17 21:44:36,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408116575bc060ce15c9b5 HTTP/1.1" 502 150 2025-12-17 21:44:36,753 - lava_callback - ERROR - Error processing callback for node 69408116575bc060ce15c9b5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408116575bc060ce15c9b5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408116575bc060ce15c9b5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408116575bc060ce15c9b5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:36,756 - lava_callback - INFO - Starting async job submit for node 69408135575bc060ce15ca3a 2025-12-17 21:44:36,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:36,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408135575bc060ce15ca3a HTTP/1.1" 502 150 2025-12-17 21:44:36,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408135575bc060ce15ca3a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:36,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408135575bc060ce15ca3a 2025-12-17 21:44:36,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408135575bc060ce15ca3a HTTP/1.1" 502 150 2025-12-17 21:44:36,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408135575bc060ce15ca3a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:37,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408126575bc060ce15ca14 2025-12-17 21:44:37,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408126575bc060ce15ca14 HTTP/1.1" 502 150 2025-12-17 21:44:37,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408126575bc060ce15ca14'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:37,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940810e575bc060ce15c99c 2025-12-17 21:44:37,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940810e575bc060ce15c99c HTTP/1.1" 502 150 2025-12-17 21:44:37,887 - lava_callback - ERROR - Error processing callback for node 6940810e575bc060ce15c99c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940810e575bc060ce15c99c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940810e575bc060ce15c99c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940810e575bc060ce15c99c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:37,889 - lava_callback - INFO - Starting async job submit for node 69408136575bc060ce15ca3d 2025-12-17 21:44:37,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:37,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408136575bc060ce15ca3d HTTP/1.1" 502 150 2025-12-17 21:44:37,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408136575bc060ce15ca3d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:37,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408136575bc060ce15ca3d 2025-12-17 21:44:37,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408136575bc060ce15ca3d HTTP/1.1" 502 150 2025-12-17 21:44:37,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408136575bc060ce15ca3d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:37,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408122575bc060ce15ca0a 2025-12-17 21:44:37,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408122575bc060ce15ca0a HTTP/1.1" 502 150 2025-12-17 21:44:37,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408122575bc060ce15ca0a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:38,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408135575bc060ce15ca3a 2025-12-17 21:44:38,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408135575bc060ce15ca3a HTTP/1.1" 502 150 2025-12-17 21:44:38,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408135575bc060ce15ca3a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:39,169 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408127575bc060ce15ca17 2025-12-17 21:44:39,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408127575bc060ce15ca17 HTTP/1.1" 502 150 2025-12-17 21:44:39,170 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408127575bc060ce15ca17'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:39,292 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408132575bc060ce15ca34 2025-12-17 21:44:39,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408132575bc060ce15ca34 HTTP/1.1" 502 150 2025-12-17 21:44:39,294 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408132575bc060ce15ca34'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:39,501 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408117575bc060ce15c9ba 2025-12-17 21:44:39,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408117575bc060ce15c9ba HTTP/1.1" 502 150 2025-12-17 21:44:39,503 - lava_callback - ERROR - Error processing callback for node 69408117575bc060ce15c9ba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408117575bc060ce15c9ba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408117575bc060ce15c9ba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408117575bc060ce15c9ba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:39,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408136575bc060ce15ca3d 2025-12-17 21:44:39,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408136575bc060ce15ca3d HTTP/1.1" 502 150 2025-12-17 21:44:39,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408136575bc060ce15ca3d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:39,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408134575bc060ce15ca37 2025-12-17 21:44:39,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408134575bc060ce15ca37 HTTP/1.1" 502 150 2025-12-17 21:44:39,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408134575bc060ce15ca37'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:39,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408118575bc060ce15c9c1 2025-12-17 21:44:39,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408118575bc060ce15c9c1 HTTP/1.1" 502 150 2025-12-17 21:44:39,989 - lava_callback - ERROR - Error processing callback for node 69408118575bc060ce15c9c1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408118575bc060ce15c9c1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408118575bc060ce15c9c1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408118575bc060ce15c9c1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:40,777 - lava_callback - INFO - Received callback for node 694080e1575bc060ce15c94c INFO: 172.21.0.1:42326 - "POST /node/694080e1575bc060ce15c94c HTTP/1.0" 202 Accepted 2025-12-17 21:44:40,783 - lava_callback - INFO - Starting async job submit for node 694080e1575bc060ce15c94c 2025-12-17 21:44:40,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:40,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e1575bc060ce15c94c HTTP/1.1" 502 150 2025-12-17 21:44:40,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e1575bc060ce15c94c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:40,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e1575bc060ce15c94c 2025-12-17 21:44:40,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e1575bc060ce15c94c HTTP/1.1" 502 150 2025-12-17 21:44:40,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e1575bc060ce15c94c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:42,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408119575bc060ce15c9c6 2025-12-17 21:44:42,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408135575bc060ce15ca3a 2025-12-17 21:44:42,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408135575bc060ce15ca3a HTTP/1.1" 502 150 2025-12-17 21:44:42,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408135575bc060ce15ca3a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:42,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e1575bc060ce15c94c 2025-12-17 21:44:42,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e1575bc060ce15c94c HTTP/1.1" 502 150 2025-12-17 21:44:42,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e1575bc060ce15c94c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:43,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408125575bc060ce15ca11 2025-12-17 21:44:43,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408125575bc060ce15ca11 HTTP/1.1" 502 150 2025-12-17 21:44:43,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408125575bc060ce15ca11'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:43,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408136575bc060ce15ca3d 2025-12-17 21:44:43,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408136575bc060ce15ca3d HTTP/1.1" 502 150 2025-12-17 21:44:43,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408136575bc060ce15ca3d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:45,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408126575bc060ce15ca14 2025-12-17 21:44:45,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408126575bc060ce15ca14 HTTP/1.1" 502 150 2025-12-17 21:44:45,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408126575bc060ce15ca14'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:45,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811b575bc060ce15c9f9 2025-12-17 21:44:45,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811b575bc060ce15c9f9 HTTP/1.1" 502 150 2025-12-17 21:44:45,653 - lava_callback - ERROR - Error processing callback for node 6940811b575bc060ce15c9f9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940811b575bc060ce15c9f9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940811b575bc060ce15c9f9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940811b575bc060ce15c9f9 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:42340 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:44:46,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e1575bc060ce15c94c 2025-12-17 21:44:46,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e1575bc060ce15c94c HTTP/1.1" 502 150 2025-12-17 21:44:46,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e1575bc060ce15c94c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:47,170 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408127575bc060ce15ca17 2025-12-17 21:44:47,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408127575bc060ce15ca17 HTTP/1.1" 502 150 2025-12-17 21:44:47,172 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408127575bc060ce15ca17'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:47,294 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408132575bc060ce15ca34 2025-12-17 21:44:47,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408132575bc060ce15ca34 HTTP/1.1" 502 150 2025-12-17 21:44:47,295 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408132575bc060ce15ca34'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:47,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811d575bc060ce15c9fc 2025-12-17 21:44:47,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811d575bc060ce15c9fc HTTP/1.1" 502 150 2025-12-17 21:44:47,867 - lava_callback - ERROR - Error processing callback for node 6940811d575bc060ce15c9fc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940811d575bc060ce15c9fc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940811d575bc060ce15c9fc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940811d575bc060ce15c9fc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:47,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408134575bc060ce15ca37 2025-12-17 21:44:47,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408134575bc060ce15ca37 HTTP/1.1" 502 150 2025-12-17 21:44:47,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408134575bc060ce15ca37'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:48,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811e575bc060ce15c9ff 2025-12-17 21:44:48,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811e575bc060ce15c9ff HTTP/1.1" 502 150 2025-12-17 21:44:48,836 - lava_callback - ERROR - Error processing callback for node 6940811e575bc060ce15c9ff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940811e575bc060ce15c9ff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940811e575bc060ce15c9ff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940811e575bc060ce15c9ff (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:49,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408121575bc060ce15ca07 HTTP/1.1" 502 150 2025-12-17 21:44:49,328 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408121575bc060ce15ca07'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:49,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940811f575bc060ce15ca02 2025-12-17 21:44:49,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940811f575bc060ce15ca02 HTTP/1.1" 502 150 2025-12-17 21:44:49,892 - lava_callback - ERROR - Error processing callback for node 6940811f575bc060ce15ca02: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940811f575bc060ce15ca02 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940811f575bc060ce15ca02 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940811f575bc060ce15ca02 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:50,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408135575bc060ce15ca3a 2025-12-17 21:44:50,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408135575bc060ce15ca3a HTTP/1.1" 502 150 2025-12-17 21:44:50,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408135575bc060ce15ca3a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:51,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408136575bc060ce15ca3d 2025-12-17 21:44:51,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408136575bc060ce15ca3d HTTP/1.1" 502 150 2025-12-17 21:44:51,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408136575bc060ce15ca3d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:53,114 - lava_callback - INFO - Received callback for node 6940814b575bc060ce15ca43 INFO: 172.21.0.1:37800 - "POST /node/6940814b575bc060ce15ca43 HTTP/1.0" 202 Accepted 2025-12-17 21:44:53,120 - lava_callback - INFO - Starting async job submit for node 6940814b575bc060ce15ca43 2025-12-17 21:44:53,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:53,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814b575bc060ce15ca43 HTTP/1.1" 502 150 2025-12-17 21:44:53,139 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814b575bc060ce15ca43'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:53,140 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814b575bc060ce15ca43 2025-12-17 21:44:53,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814b575bc060ce15ca43 HTTP/1.1" 502 150 2025-12-17 21:44:53,140 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814b575bc060ce15ca43'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:53,775 - lava_callback - INFO - Received callback for node 6940814d575bc060ce15ca4d INFO: 172.21.0.1:37814 - "POST /node/6940814d575bc060ce15ca4d HTTP/1.0" 202 Accepted 2025-12-17 21:44:53,781 - lava_callback - INFO - Starting async job submit for node 6940814d575bc060ce15ca4d 2025-12-17 21:44:53,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:53,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814d575bc060ce15ca4d HTTP/1.1" 502 150 2025-12-17 21:44:53,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814d575bc060ce15ca4d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:53,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814d575bc060ce15ca4d 2025-12-17 21:44:53,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814d575bc060ce15ca4d HTTP/1.1" 502 150 2025-12-17 21:44:53,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814d575bc060ce15ca4d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:53,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408122575bc060ce15ca0a 2025-12-17 21:44:53,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408122575bc060ce15ca0a HTTP/1.1" 502 150 2025-12-17 21:44:53,957 - lava_callback - ERROR - Error processing callback for node 69408122575bc060ce15ca0a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408122575bc060ce15ca0a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408122575bc060ce15ca0a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408122575bc060ce15ca0a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:54,451 - lava_callback - INFO - Received callback for node 6940814e575bc060ce15ca54 INFO: 172.21.0.1:37822 - "POST /node/6940814e575bc060ce15ca54 HTTP/1.0" 202 Accepted 2025-12-17 21:44:54,457 - lava_callback - INFO - Starting async job submit for node 6940814e575bc060ce15ca54 2025-12-17 21:44:54,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:54,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814e575bc060ce15ca54 HTTP/1.1" 502 150 2025-12-17 21:44:54,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814e575bc060ce15ca54'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:54,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814e575bc060ce15ca54 2025-12-17 21:44:54,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814e575bc060ce15ca54 HTTP/1.1" 502 150 2025-12-17 21:44:54,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814e575bc060ce15ca54'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:54,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e1575bc060ce15c94c 2025-12-17 21:44:54,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e1575bc060ce15c94c HTTP/1.1" 502 150 2025-12-17 21:44:54,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080e1575bc060ce15c94c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:55,101 - lava_callback - INFO - Received callback for node 6940814f575bc060ce15ca57 INFO: 172.21.0.1:37828 - "POST /node/6940814f575bc060ce15ca57 HTTP/1.0" 202 Accepted 2025-12-17 21:44:55,108 - lava_callback - INFO - Starting async job submit for node 6940814f575bc060ce15ca57 2025-12-17 21:44:55,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:55,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814f575bc060ce15ca57 HTTP/1.1" 502 150 2025-12-17 21:44:55,116 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814f575bc060ce15ca57'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:55,116 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814f575bc060ce15ca57 2025-12-17 21:44:55,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814f575bc060ce15ca57 HTTP/1.1" 502 150 2025-12-17 21:44:55,116 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814f575bc060ce15ca57'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:55,140 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814b575bc060ce15ca43 2025-12-17 21:44:55,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814b575bc060ce15ca43 HTTP/1.1" 502 150 2025-12-17 21:44:55,141 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814b575bc060ce15ca43'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:55,763 - lava_callback - INFO - Received callback for node 69408151575bc060ce15ca5a INFO: 172.21.0.1:37840 - "POST /node/69408151575bc060ce15ca5a HTTP/1.0" 202 Accepted 2025-12-17 21:44:55,769 - lava_callback - INFO - Starting async job submit for node 69408151575bc060ce15ca5a 2025-12-17 21:44:55,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:55,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408151575bc060ce15ca5a HTTP/1.1" 502 150 2025-12-17 21:44:55,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408151575bc060ce15ca5a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:55,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408151575bc060ce15ca5a 2025-12-17 21:44:55,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408151575bc060ce15ca5a HTTP/1.1" 502 150 2025-12-17 21:44:55,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408151575bc060ce15ca5a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:55,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814d575bc060ce15ca4d 2025-12-17 21:44:55,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814d575bc060ce15ca4d HTTP/1.1" 502 150 2025-12-17 21:44:55,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814d575bc060ce15ca4d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:56,447 - lava_callback - INFO - Received callback for node 69408152575bc060ce15ca5d INFO: 172.21.0.1:37852 - "POST /node/69408152575bc060ce15ca5d HTTP/1.0" 202 Accepted 2025-12-17 21:44:56,453 - lava_callback - INFO - Starting async job submit for node 69408152575bc060ce15ca5d 2025-12-17 21:44:56,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:44:56,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408152575bc060ce15ca5d HTTP/1.1" 502 150 2025-12-17 21:44:56,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408152575bc060ce15ca5d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:56,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408152575bc060ce15ca5d 2025-12-17 21:44:56,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408152575bc060ce15ca5d HTTP/1.1" 502 150 2025-12-17 21:44:56,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408152575bc060ce15ca5d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:56,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814e575bc060ce15ca54 2025-12-17 21:44:56,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814e575bc060ce15ca54 HTTP/1.1" 502 150 2025-12-17 21:44:56,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814e575bc060ce15ca54'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37868 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:44:57,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814f575bc060ce15ca57 2025-12-17 21:44:57,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814f575bc060ce15ca57 HTTP/1.1" 502 150 2025-12-17 21:44:57,118 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814f575bc060ce15ca57'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:57,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408151575bc060ce15ca5a 2025-12-17 21:44:57,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408151575bc060ce15ca5a HTTP/1.1" 502 150 2025-12-17 21:44:57,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408151575bc060ce15ca5a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:58,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408119575bc060ce15c9c6 HTTP/1.1" 502 150 2025-12-17 21:44:58,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408119575bc060ce15c9c6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:58,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408152575bc060ce15ca5d 2025-12-17 21:44:58,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408152575bc060ce15ca5d HTTP/1.1" 502 150 2025-12-17 21:44:58,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408152575bc060ce15ca5d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:59,141 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814b575bc060ce15ca43 2025-12-17 21:44:59,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814b575bc060ce15ca43 HTTP/1.1" 502 150 2025-12-17 21:44:59,143 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814b575bc060ce15ca43'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:44:59,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408125575bc060ce15ca11 2025-12-17 21:44:59,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408125575bc060ce15ca11 HTTP/1.1" 502 150 2025-12-17 21:44:59,590 - lava_callback - ERROR - Error processing callback for node 69408125575bc060ce15ca11: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408125575bc060ce15ca11 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408125575bc060ce15ca11 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408125575bc060ce15ca11 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:44:59,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814d575bc060ce15ca4d 2025-12-17 21:44:59,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814d575bc060ce15ca4d HTTP/1.1" 502 150 2025-12-17 21:44:59,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814d575bc060ce15ca4d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:00,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814e575bc060ce15ca54 2025-12-17 21:45:01,118 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814f575bc060ce15ca57 2025-12-17 21:45:01,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814f575bc060ce15ca57 HTTP/1.1" 502 150 2025-12-17 21:45:01,119 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814f575bc060ce15ca57'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:01,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408126575bc060ce15ca14 2025-12-17 21:45:01,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408126575bc060ce15ca14 HTTP/1.1" 502 150 2025-12-17 21:45:01,549 - lava_callback - ERROR - Error processing callback for node 69408126575bc060ce15ca14: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408126575bc060ce15ca14 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408126575bc060ce15ca14 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408126575bc060ce15ca14 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:50048 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:45:01,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408151575bc060ce15ca5a 2025-12-17 21:45:01,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408151575bc060ce15ca5a HTTP/1.1" 502 150 2025-12-17 21:45:01,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408151575bc060ce15ca5a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:02,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408152575bc060ce15ca5d 2025-12-17 21:45:02,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408152575bc060ce15ca5d HTTP/1.1" 502 150 2025-12-17 21:45:02,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408152575bc060ce15ca5d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:03,172 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408127575bc060ce15ca17 2025-12-17 21:45:03,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408127575bc060ce15ca17 HTTP/1.1" 502 150 2025-12-17 21:45:03,173 - lava_callback - ERROR - Error processing callback for node 69408127575bc060ce15ca17: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408127575bc060ce15ca17 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408127575bc060ce15ca17 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408127575bc060ce15ca17 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:03,295 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408132575bc060ce15ca34 2025-12-17 21:45:03,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408132575bc060ce15ca34 HTTP/1.1" 502 150 2025-12-17 21:45:03,297 - lava_callback - ERROR - Error processing callback for node 69408132575bc060ce15ca34: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408132575bc060ce15ca34 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408132575bc060ce15ca34 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408132575bc060ce15ca34 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:03,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408134575bc060ce15ca37 2025-12-17 21:45:03,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408134575bc060ce15ca37 HTTP/1.1" 502 150 2025-12-17 21:45:03,986 - lava_callback - ERROR - Error processing callback for node 69408134575bc060ce15ca37: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408134575bc060ce15ca37 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408134575bc060ce15ca37 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408134575bc060ce15ca37 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:05,329 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408121575bc060ce15ca07 2025-12-17 21:45:05,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408121575bc060ce15ca07 HTTP/1.1" 502 150 2025-12-17 21:45:05,330 - lava_callback - ERROR - Error processing callback for node 69408121575bc060ce15ca07: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408121575bc060ce15ca07 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408121575bc060ce15ca07 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408121575bc060ce15ca07 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:50056 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:45:06,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408135575bc060ce15ca3a 2025-12-17 21:45:06,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408135575bc060ce15ca3a HTTP/1.1" 502 150 2025-12-17 21:45:06,788 - lava_callback - ERROR - Error processing callback for node 69408135575bc060ce15ca3a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408135575bc060ce15ca3a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408135575bc060ce15ca3a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408135575bc060ce15ca3a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:07,143 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814b575bc060ce15ca43 2025-12-17 21:45:07,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814b575bc060ce15ca43 HTTP/1.1" 502 150 2025-12-17 21:45:07,144 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814b575bc060ce15ca43'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:07,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814d575bc060ce15ca4d 2025-12-17 21:45:07,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814d575bc060ce15ca4d HTTP/1.1" 502 150 2025-12-17 21:45:07,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814d575bc060ce15ca4d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:07,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408136575bc060ce15ca3d 2025-12-17 21:45:07,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408136575bc060ce15ca3d HTTP/1.1" 502 150 2025-12-17 21:45:07,906 - lava_callback - ERROR - Error processing callback for node 69408136575bc060ce15ca3d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408136575bc060ce15ca3d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408136575bc060ce15ca3d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408136575bc060ce15ca3d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:09,120 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814f575bc060ce15ca57 2025-12-17 21:45:09,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408151575bc060ce15ca5a 2025-12-17 21:45:09,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408151575bc060ce15ca5a HTTP/1.1" 502 150 2025-12-17 21:45:09,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408151575bc060ce15ca5a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:10,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408152575bc060ce15ca5d 2025-12-17 21:45:10,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408152575bc060ce15ca5d HTTP/1.1" 502 150 2025-12-17 21:45:10,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408152575bc060ce15ca5d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:10,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080e1575bc060ce15c94c 2025-12-17 21:45:10,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080e1575bc060ce15c94c HTTP/1.1" 502 150 2025-12-17 21:45:10,819 - lava_callback - ERROR - Error processing callback for node 694080e1575bc060ce15c94c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e1575bc060ce15c94c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e1575bc060ce15c94c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080e1575bc060ce15c94c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:13,103 - lava_callback - INFO - Received callback for node 69408161575bc060ce15caba INFO: 172.21.0.1:56356 - "POST /node/69408161575bc060ce15caba HTTP/1.0" 202 Accepted 2025-12-17 21:45:13,108 - lava_callback - INFO - Starting async job submit for node 69408161575bc060ce15caba 2025-12-17 21:45:13,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:13,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408161575bc060ce15caba HTTP/1.1" 502 150 2025-12-17 21:45:13,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408161575bc060ce15caba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:13,115 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408161575bc060ce15caba 2025-12-17 21:45:13,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408161575bc060ce15caba HTTP/1.1" 502 150 2025-12-17 21:45:13,116 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408161575bc060ce15caba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:13,785 - lava_callback - INFO - Received callback for node 69408163575bc060ce15cabe INFO: 172.21.0.1:56362 - "POST /node/69408163575bc060ce15cabe HTTP/1.0" 202 Accepted 2025-12-17 21:45:13,791 - lava_callback - INFO - Starting async job submit for node 69408163575bc060ce15cabe 2025-12-17 21:45:13,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:13,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408163575bc060ce15cabe HTTP/1.1" 502 150 2025-12-17 21:45:13,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408163575bc060ce15cabe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:13,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408163575bc060ce15cabe 2025-12-17 21:45:13,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408163575bc060ce15cabe HTTP/1.1" 502 150 2025-12-17 21:45:13,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408163575bc060ce15cabe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:14,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408119575bc060ce15c9c6 2025-12-17 21:45:14,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408119575bc060ce15c9c6 HTTP/1.1" 502 150 2025-12-17 21:45:14,034 - lava_callback - ERROR - Error processing callback for node 69408119575bc060ce15c9c6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408119575bc060ce15c9c6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408119575bc060ce15c9c6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408119575bc060ce15c9c6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:14,464 - lava_callback - INFO - Received callback for node 6940815b575bc060ce15caa0 INFO: 172.21.0.1:56374 - "POST /node/6940815b575bc060ce15caa0 HTTP/1.0" 202 Accepted 2025-12-17 21:45:14,470 - lava_callback - INFO - Starting async job submit for node 6940815b575bc060ce15caa0 2025-12-17 21:45:14,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:14,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815b575bc060ce15caa0 HTTP/1.1" 502 150 2025-12-17 21:45:14,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815b575bc060ce15caa0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:14,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815b575bc060ce15caa0 2025-12-17 21:45:14,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815b575bc060ce15caa0 HTTP/1.1" 502 150 2025-12-17 21:45:14,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815b575bc060ce15caa0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:15,116 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408161575bc060ce15caba 2025-12-17 21:45:15,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408161575bc060ce15caba HTTP/1.1" 502 150 2025-12-17 21:45:15,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408161575bc060ce15caba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:15,131 - lava_callback - INFO - Received callback for node 6940815c575bc060ce15caa3 INFO: 172.21.0.1:56388 - "POST /node/6940815c575bc060ce15caa3 HTTP/1.0" 202 Accepted 2025-12-17 21:45:15,138 - lava_callback - INFO - Starting async job submit for node 6940815c575bc060ce15caa3 2025-12-17 21:45:15,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:15,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815c575bc060ce15caa3 HTTP/1.1" 502 150 2025-12-17 21:45:15,148 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815c575bc060ce15caa3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:15,149 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815c575bc060ce15caa3 2025-12-17 21:45:15,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815c575bc060ce15caa3 HTTP/1.1" 502 150 2025-12-17 21:45:15,149 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815c575bc060ce15caa3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56392 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:45:15,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408163575bc060ce15cabe 2025-12-17 21:45:15,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408163575bc060ce15cabe HTTP/1.1" 502 150 2025-12-17 21:45:15,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408163575bc060ce15cabe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:15,874 - lava_callback - INFO - Received callback for node 6940815d575bc060ce15caa8 INFO: 172.21.0.1:56394 - "POST /node/6940815d575bc060ce15caa8 HTTP/1.0" 202 Accepted 2025-12-17 21:45:15,882 - lava_callback - INFO - Starting async job submit for node 6940815d575bc060ce15caa8 2025-12-17 21:45:15,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:15,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815d575bc060ce15caa8 HTTP/1.1" 502 150 2025-12-17 21:45:15,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815d575bc060ce15caa8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:15,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815d575bc060ce15caa8 2025-12-17 21:45:15,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815d575bc060ce15caa8 HTTP/1.1" 502 150 2025-12-17 21:45:15,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815d575bc060ce15caa8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:15,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814e575bc060ce15ca54 HTTP/1.1" 502 150 2025-12-17 21:45:15,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814e575bc060ce15ca54'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:16,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815b575bc060ce15caa0 2025-12-17 21:45:16,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815b575bc060ce15caa0 HTTP/1.1" 502 150 2025-12-17 21:45:16,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815b575bc060ce15caa0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56410 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:45:16,553 - lava_callback - INFO - Received callback for node 6940815f575bc060ce15caad INFO: 172.21.0.1:56426 - "POST /node/6940815f575bc060ce15caad HTTP/1.0" 202 Accepted 2025-12-17 21:45:16,559 - lava_callback - INFO - Starting async job submit for node 6940815f575bc060ce15caad 2025-12-17 21:45:16,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:16,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815f575bc060ce15caad HTTP/1.1" 502 150 2025-12-17 21:45:16,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815f575bc060ce15caad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:16,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815f575bc060ce15caad 2025-12-17 21:45:16,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815f575bc060ce15caad HTTP/1.1" 502 150 2025-12-17 21:45:16,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815f575bc060ce15caad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:17,150 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815c575bc060ce15caa3 2025-12-17 21:45:17,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815c575bc060ce15caa3 HTTP/1.1" 502 150 2025-12-17 21:45:17,151 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815c575bc060ce15caa3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:17,225 - lava_callback - INFO - Received callback for node 69408160575bc060ce15cab5 INFO: 172.21.0.1:56442 - "POST /node/69408160575bc060ce15cab5 HTTP/1.0" 202 Accepted 2025-12-17 21:45:17,231 - lava_callback - INFO - Starting async job submit for node 69408160575bc060ce15cab5 2025-12-17 21:45:17,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:17,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408160575bc060ce15cab5 HTTP/1.1" 502 150 2025-12-17 21:45:17,262 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408160575bc060ce15cab5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:17,262 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408160575bc060ce15cab5 2025-12-17 21:45:17,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408160575bc060ce15cab5 HTTP/1.1" 502 150 2025-12-17 21:45:17,263 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408160575bc060ce15cab5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:17,890 - lava_callback - INFO - Received callback for node 69408164575bc060ce15cac2 INFO: 172.21.0.1:56454 - "POST /node/69408164575bc060ce15cac2 HTTP/1.0" 202 Accepted 2025-12-17 21:45:17,896 - lava_callback - INFO - Starting async job submit for node 69408164575bc060ce15cac2 2025-12-17 21:45:17,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:17,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408164575bc060ce15cac2 HTTP/1.1" 502 150 2025-12-17 21:45:17,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408164575bc060ce15cac2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:17,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408164575bc060ce15cac2 2025-12-17 21:45:17,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408164575bc060ce15cac2 HTTP/1.1" 502 150 2025-12-17 21:45:17,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408164575bc060ce15cac2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:17,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815d575bc060ce15caa8 2025-12-17 21:45:17,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815d575bc060ce15caa8 HTTP/1.1" 502 150 2025-12-17 21:45:17,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815d575bc060ce15caa8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:18,561 - lava_callback - INFO - Received callback for node 69408168575bc060ce15cacb INFO: 172.21.0.1:56460 - "POST /node/69408168575bc060ce15cacb HTTP/1.0" 202 Accepted 2025-12-17 21:45:18,566 - lava_callback - INFO - Starting async job submit for node 69408168575bc060ce15cacb 2025-12-17 21:45:18,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:18,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815f575bc060ce15caad 2025-12-17 21:45:18,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815f575bc060ce15caad HTTP/1.1" 502 150 2025-12-17 21:45:18,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815f575bc060ce15caad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:18,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408168575bc060ce15cacb HTTP/1.1" 502 150 2025-12-17 21:45:18,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408168575bc060ce15cacb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:18,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408168575bc060ce15cacb 2025-12-17 21:45:18,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408168575bc060ce15cacb HTTP/1.1" 502 150 2025-12-17 21:45:18,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408168575bc060ce15cacb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:19,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408161575bc060ce15caba 2025-12-17 21:45:19,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408161575bc060ce15caba HTTP/1.1" 502 150 2025-12-17 21:45:19,119 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408161575bc060ce15caba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:19,263 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408160575bc060ce15cab5 2025-12-17 21:45:19,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408160575bc060ce15cab5 HTTP/1.1" 502 150 2025-12-17 21:45:19,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408160575bc060ce15cab5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:19,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408163575bc060ce15cabe 2025-12-17 21:45:19,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408163575bc060ce15cabe HTTP/1.1" 502 150 2025-12-17 21:45:19,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408163575bc060ce15cabe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:19,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408164575bc060ce15cac2 2025-12-17 21:45:19,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408164575bc060ce15cac2 HTTP/1.1" 502 150 2025-12-17 21:45:19,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408164575bc060ce15cac2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:20,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815b575bc060ce15caa0 2025-12-17 21:45:20,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408168575bc060ce15cacb 2025-12-17 21:45:20,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408168575bc060ce15cacb HTTP/1.1" 502 150 2025-12-17 21:45:20,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408168575bc060ce15cacb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:21,151 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815c575bc060ce15caa3 2025-12-17 21:45:21,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815c575bc060ce15caa3 HTTP/1.1" 502 150 2025-12-17 21:45:21,153 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815c575bc060ce15caa3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:21,482 - lava_callback - INFO - Received callback for node 69408123575bc060ce15ca0d INFO: 172.21.0.1:53028 - "POST /node/69408123575bc060ce15ca0d HTTP/1.0" 202 Accepted 2025-12-17 21:45:21,489 - lava_callback - INFO - Starting async job submit for node 69408123575bc060ce15ca0d 2025-12-17 21:45:21,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:21,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408123575bc060ce15ca0d HTTP/1.1" 502 150 2025-12-17 21:45:21,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408123575bc060ce15ca0d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:21,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408123575bc060ce15ca0d 2025-12-17 21:45:21,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408123575bc060ce15ca0d HTTP/1.1" 502 150 2025-12-17 21:45:21,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408123575bc060ce15ca0d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:21,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815d575bc060ce15caa8 2025-12-17 21:45:21,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815d575bc060ce15caa8 HTTP/1.1" 502 150 2025-12-17 21:45:21,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815d575bc060ce15caa8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:22,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815f575bc060ce15caad 2025-12-17 21:45:22,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815f575bc060ce15caad HTTP/1.1" 502 150 2025-12-17 21:45:22,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815f575bc060ce15caad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:23,144 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814b575bc060ce15ca43 2025-12-17 21:45:23,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814b575bc060ce15ca43 HTTP/1.1" 502 150 2025-12-17 21:45:23,146 - lava_callback - ERROR - Error processing callback for node 6940814b575bc060ce15ca43: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940814b575bc060ce15ca43 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940814b575bc060ce15ca43 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940814b575bc060ce15ca43 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:23,264 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408160575bc060ce15cab5 2025-12-17 21:45:23,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408160575bc060ce15cab5 HTTP/1.1" 502 150 2025-12-17 21:45:23,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408160575bc060ce15cab5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:23,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408123575bc060ce15ca0d 2025-12-17 21:45:23,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408123575bc060ce15ca0d HTTP/1.1" 502 150 2025-12-17 21:45:23,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408123575bc060ce15ca0d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:23,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814d575bc060ce15ca4d 2025-12-17 21:45:23,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814d575bc060ce15ca4d HTTP/1.1" 502 150 2025-12-17 21:45:23,815 - lava_callback - ERROR - Error processing callback for node 6940814d575bc060ce15ca4d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940814d575bc060ce15ca4d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940814d575bc060ce15ca4d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940814d575bc060ce15ca4d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:23,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408164575bc060ce15cac2 2025-12-17 21:45:23,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408164575bc060ce15cac2 HTTP/1.1" 502 150 2025-12-17 21:45:23,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408164575bc060ce15cac2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:23,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814e575bc060ce15ca54 2025-12-17 21:45:23,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814e575bc060ce15ca54 HTTP/1.1" 502 150 2025-12-17 21:45:23,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814e575bc060ce15ca54'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:24,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814f575bc060ce15ca57 HTTP/1.1" 502 150 2025-12-17 21:45:24,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940814f575bc060ce15ca57'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:24,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408168575bc060ce15cacb 2025-12-17 21:45:24,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408168575bc060ce15cacb HTTP/1.1" 502 150 2025-12-17 21:45:24,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408168575bc060ce15cacb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:25,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408151575bc060ce15ca5a 2025-12-17 21:45:25,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408151575bc060ce15ca5a HTTP/1.1" 502 150 2025-12-17 21:45:25,784 - lava_callback - ERROR - Error processing callback for node 69408151575bc060ce15ca5a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408151575bc060ce15ca5a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408151575bc060ce15ca5a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408151575bc060ce15ca5a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:26,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408152575bc060ce15ca5d 2025-12-17 21:45:26,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408152575bc060ce15ca5d HTTP/1.1" 502 150 2025-12-17 21:45:26,467 - lava_callback - ERROR - Error processing callback for node 69408152575bc060ce15ca5d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408152575bc060ce15ca5d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408152575bc060ce15ca5d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408152575bc060ce15ca5d (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:53034 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:45:27,119 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408161575bc060ce15caba 2025-12-17 21:45:27,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408161575bc060ce15caba HTTP/1.1" 502 150 2025-12-17 21:45:27,120 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408161575bc060ce15caba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:27,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408123575bc060ce15ca0d 2025-12-17 21:45:27,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408123575bc060ce15ca0d HTTP/1.1" 502 150 2025-12-17 21:45:27,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408123575bc060ce15ca0d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:27,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408163575bc060ce15cabe 2025-12-17 21:45:27,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408163575bc060ce15cabe HTTP/1.1" 502 150 2025-12-17 21:45:27,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408163575bc060ce15cabe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:29,153 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815c575bc060ce15caa3 2025-12-17 21:45:29,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815c575bc060ce15caa3 HTTP/1.1" 502 150 2025-12-17 21:45:29,154 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815c575bc060ce15caa3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:29,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815d575bc060ce15caa8 2025-12-17 21:45:29,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815d575bc060ce15caa8 HTTP/1.1" 502 150 2025-12-17 21:45:29,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815d575bc060ce15caa8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:30,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815f575bc060ce15caad 2025-12-17 21:45:30,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815f575bc060ce15caad HTTP/1.1" 502 150 2025-12-17 21:45:30,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815f575bc060ce15caad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:31,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408160575bc060ce15cab5 2025-12-17 21:45:31,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408160575bc060ce15cab5 HTTP/1.1" 502 150 2025-12-17 21:45:31,267 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408160575bc060ce15cab5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:31,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408164575bc060ce15cac2 2025-12-17 21:45:31,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408164575bc060ce15cac2 HTTP/1.1" 502 150 2025-12-17 21:45:31,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408164575bc060ce15cac2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:32,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408168575bc060ce15cacb 2025-12-17 21:45:32,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408168575bc060ce15cacb HTTP/1.1" 502 150 2025-12-17 21:45:32,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408168575bc060ce15cacb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:33,094 - lava_callback - INFO - Received callback for node 69408169575bc060ce15cace INFO: 172.21.0.1:48510 - "POST /node/69408169575bc060ce15cace HTTP/1.0" 202 Accepted 2025-12-17 21:45:33,100 - lava_callback - INFO - Starting async job submit for node 69408169575bc060ce15cace 2025-12-17 21:45:33,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:33,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408169575bc060ce15cace HTTP/1.1" 502 150 2025-12-17 21:45:33,106 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408169575bc060ce15cace'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:33,107 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408169575bc060ce15cace 2025-12-17 21:45:33,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408169575bc060ce15cace HTTP/1.1" 502 150 2025-12-17 21:45:33,108 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408169575bc060ce15cace'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:33,768 - lava_callback - INFO - Received callback for node 6940816a575bc060ce15cad1 INFO: 172.21.0.1:48526 - "POST /node/6940816a575bc060ce15cad1 HTTP/1.0" 202 Accepted 2025-12-17 21:45:33,774 - lava_callback - INFO - Starting async job submit for node 6940816a575bc060ce15cad1 2025-12-17 21:45:33,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:33,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816a575bc060ce15cad1 HTTP/1.1" 502 150 2025-12-17 21:45:33,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816a575bc060ce15cad1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:33,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940816a575bc060ce15cad1 2025-12-17 21:45:33,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816a575bc060ce15cad1 HTTP/1.1" 502 150 2025-12-17 21:45:33,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816a575bc060ce15cad1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:34,445 - lava_callback - INFO - Received callback for node 6940816b575bc060ce15cad4 INFO: 172.21.0.1:48530 - "POST /node/6940816b575bc060ce15cad4 HTTP/1.0" 202 Accepted 2025-12-17 21:45:34,452 - lava_callback - INFO - Starting async job submit for node 6940816b575bc060ce15cad4 2025-12-17 21:45:34,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:34,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816b575bc060ce15cad4 HTTP/1.1" 502 150 2025-12-17 21:45:34,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816b575bc060ce15cad4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:34,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940816b575bc060ce15cad4 2025-12-17 21:45:34,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816b575bc060ce15cad4 HTTP/1.1" 502 150 2025-12-17 21:45:34,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816b575bc060ce15cad4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:35,108 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408169575bc060ce15cace 2025-12-17 21:45:35,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408169575bc060ce15cace HTTP/1.1" 502 150 2025-12-17 21:45:35,109 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408169575bc060ce15cace'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:35,128 - lava_callback - INFO - Received callback for node 6940816f575bc060ce15cada INFO: 172.21.0.1:48544 - "POST /node/6940816f575bc060ce15cada HTTP/1.0" 202 Accepted 2025-12-17 21:45:35,134 - lava_callback - INFO - Starting async job submit for node 6940816f575bc060ce15cada 2025-12-17 21:45:35,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:35,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816f575bc060ce15cada HTTP/1.1" 502 150 2025-12-17 21:45:35,144 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816f575bc060ce15cada'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:35,144 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940816f575bc060ce15cada 2025-12-17 21:45:35,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816f575bc060ce15cada HTTP/1.1" 502 150 2025-12-17 21:45:35,144 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816f575bc060ce15cada'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:35,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408123575bc060ce15ca0d 2025-12-17 21:45:35,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408123575bc060ce15ca0d HTTP/1.1" 502 150 2025-12-17 21:45:35,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408123575bc060ce15ca0d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:35,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940816a575bc060ce15cad1 2025-12-17 21:45:35,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816a575bc060ce15cad1 HTTP/1.1" 502 150 2025-12-17 21:45:35,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816a575bc060ce15cad1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:35,852 - lava_callback - INFO - Received callback for node 69408170575bc060ce15cadd INFO: 172.21.0.1:48546 - "POST /node/69408170575bc060ce15cadd HTTP/1.0" 202 Accepted 2025-12-17 21:45:35,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815b575bc060ce15caa0 HTTP/1.1" 502 150 2025-12-17 21:45:35,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815b575bc060ce15caa0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:36,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940816b575bc060ce15cad4 2025-12-17 21:45:36,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816b575bc060ce15cad4 HTTP/1.1" 502 150 2025-12-17 21:45:36,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816b575bc060ce15cad4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:36,514 - lava_callback - INFO - Received callback for node 69408173575bc060ce15caec INFO: 172.21.0.1:48550 - "POST /node/69408173575bc060ce15caec HTTP/1.0" 202 Accepted INFO: 172.21.0.1:48552 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:45:37,145 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940816f575bc060ce15cada 2025-12-17 21:45:37,188 - lava_callback - INFO - Received callback for node 69408174575bc060ce15caf4 INFO: 172.21.0.1:48554 - "POST /node/69408174575bc060ce15caf4 HTTP/1.0" 202 Accepted 2025-12-17 21:45:37,887 - lava_callback - INFO - Received callback for node 69408176575bc060ce15cafb INFO: 172.21.0.1:48566 - "POST /node/69408176575bc060ce15cafb HTTP/1.0" 202 Accepted 2025-12-17 21:45:39,109 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408169575bc060ce15cace 2025-12-17 21:45:39,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408169575bc060ce15cace HTTP/1.1" 502 150 2025-12-17 21:45:39,110 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408169575bc060ce15cace'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:39,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940816a575bc060ce15cad1 2025-12-17 21:45:39,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816a575bc060ce15cad1 HTTP/1.1" 502 150 2025-12-17 21:45:39,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816a575bc060ce15cad1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:39,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814e575bc060ce15ca54 2025-12-17 21:45:39,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814e575bc060ce15ca54 HTTP/1.1" 502 150 2025-12-17 21:45:39,955 - lava_callback - ERROR - Error processing callback for node 6940814e575bc060ce15ca54: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940814e575bc060ce15ca54 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940814e575bc060ce15ca54 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940814e575bc060ce15ca54 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:39,957 - lava_callback - INFO - Starting async job submit for node 69408170575bc060ce15cadd 2025-12-17 21:45:39,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:39,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408170575bc060ce15cadd HTTP/1.1" 502 150 2025-12-17 21:45:39,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408170575bc060ce15cadd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:39,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408170575bc060ce15cadd 2025-12-17 21:45:39,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408170575bc060ce15cadd HTTP/1.1" 502 150 2025-12-17 21:45:39,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408170575bc060ce15cadd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:40,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940814f575bc060ce15ca57 2025-12-17 21:45:40,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940814f575bc060ce15ca57 HTTP/1.1" 502 150 2025-12-17 21:45:40,402 - lava_callback - ERROR - Error processing callback for node 6940814f575bc060ce15ca57: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940814f575bc060ce15ca57 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940814f575bc060ce15ca57 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940814f575bc060ce15ca57 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:40,404 - lava_callback - INFO - Starting async job submit for node 69408173575bc060ce15caec 2025-12-17 21:45:40,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:40,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408173575bc060ce15caec HTTP/1.1" 502 150 2025-12-17 21:45:40,410 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408173575bc060ce15caec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:40,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408173575bc060ce15caec 2025-12-17 21:45:40,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408173575bc060ce15caec HTTP/1.1" 502 150 2025-12-17 21:45:40,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408173575bc060ce15caec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:40,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940816b575bc060ce15cad4 2025-12-17 21:45:40,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816b575bc060ce15cad4 HTTP/1.1" 502 150 2025-12-17 21:45:40,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816b575bc060ce15cad4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:41,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408170575bc060ce15cadd 2025-12-17 21:45:41,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408170575bc060ce15cadd HTTP/1.1" 502 150 2025-12-17 21:45:41,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408170575bc060ce15cadd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:42,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408173575bc060ce15caec 2025-12-17 21:45:42,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408173575bc060ce15caec HTTP/1.1" 502 150 2025-12-17 21:45:42,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408173575bc060ce15caec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:42,710 - lava_callback - INFO - Received callback for node 69408171575bc060ce15cae2 INFO: 172.21.0.1:51260 - "POST /node/69408171575bc060ce15cae2 HTTP/1.0" 202 Accepted 2025-12-17 21:45:43,120 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408161575bc060ce15caba 2025-12-17 21:45:43,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408161575bc060ce15caba HTTP/1.1" 502 150 2025-12-17 21:45:43,122 - lava_callback - ERROR - Error processing callback for node 69408161575bc060ce15caba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408161575bc060ce15caba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408161575bc060ce15caba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408161575bc060ce15caba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:43,123 - lava_callback - INFO - Starting async job submit for node 69408174575bc060ce15caf4 2025-12-17 21:45:43,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:43,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408174575bc060ce15caf4 HTTP/1.1" 502 150 2025-12-17 21:45:43,150 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408174575bc060ce15caf4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:43,150 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408174575bc060ce15caf4 2025-12-17 21:45:43,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408174575bc060ce15caf4 HTTP/1.1" 502 150 2025-12-17 21:45:43,150 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408174575bc060ce15caf4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:43,372 - lava_callback - INFO - Received callback for node 69408177575bc060ce15cafe INFO: 172.21.0.1:51264 - "POST /node/69408177575bc060ce15cafe HTTP/1.0" 202 Accepted 2025-12-17 21:45:43,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408163575bc060ce15cabe 2025-12-17 21:45:43,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408163575bc060ce15cabe HTTP/1.1" 502 150 2025-12-17 21:45:43,809 - lava_callback - ERROR - Error processing callback for node 69408163575bc060ce15cabe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408163575bc060ce15cabe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408163575bc060ce15cabe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408163575bc060ce15cabe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:43,811 - lava_callback - INFO - Starting async job submit for node 69408176575bc060ce15cafb 2025-12-17 21:45:43,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:43,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408176575bc060ce15cafb HTTP/1.1" 502 150 2025-12-17 21:45:43,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408176575bc060ce15cafb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:43,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408176575bc060ce15cafb 2025-12-17 21:45:43,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408176575bc060ce15cafb HTTP/1.1" 502 150 2025-12-17 21:45:43,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408176575bc060ce15cafb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:43,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815b575bc060ce15caa0 2025-12-17 21:45:43,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815b575bc060ce15caa0 HTTP/1.1" 502 150 2025-12-17 21:45:43,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940815b575bc060ce15caa0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:44,016 - lava_callback - INFO - Received callback for node 69408178575bc060ce15cb01 INFO: 172.21.0.1:51278 - "POST /node/69408178575bc060ce15cb01 HTTP/1.0" 202 Accepted 2025-12-17 21:45:44,678 - lava_callback - INFO - Received callback for node 6940817a575bc060ce15cb0a INFO: 172.21.0.1:51288 - "POST /node/6940817a575bc060ce15cb0a HTTP/1.0" 202 Accepted 2025-12-17 21:45:45,151 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408174575bc060ce15caf4 2025-12-17 21:45:45,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408174575bc060ce15caf4 HTTP/1.1" 502 150 2025-12-17 21:45:45,152 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408174575bc060ce15caf4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:45,154 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815c575bc060ce15caa3 2025-12-17 21:45:45,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815c575bc060ce15caa3 HTTP/1.1" 502 150 2025-12-17 21:45:45,156 - lava_callback - ERROR - Error processing callback for node 6940815c575bc060ce15caa3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940815c575bc060ce15caa3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940815c575bc060ce15caa3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940815c575bc060ce15caa3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:45,158 - lava_callback - INFO - Starting async job submit for node 69408171575bc060ce15cae2 2025-12-17 21:45:45,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:45,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408171575bc060ce15cae2 HTTP/1.1" 502 150 2025-12-17 21:45:45,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408171575bc060ce15cae2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:45,192 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408171575bc060ce15cae2 2025-12-17 21:45:45,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408171575bc060ce15cae2 HTTP/1.1" 502 150 2025-12-17 21:45:45,193 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408171575bc060ce15cae2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:45,345 - lava_callback - INFO - Received callback for node 6940817b575bc060ce15cb0d INFO: 172.21.0.1:51294 - "POST /node/6940817b575bc060ce15cb0d HTTP/1.0" 202 Accepted 2025-12-17 21:45:45,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408176575bc060ce15cafb 2025-12-17 21:45:45,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408176575bc060ce15cafb HTTP/1.1" 502 150 2025-12-17 21:45:45,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408176575bc060ce15cafb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:45,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815d575bc060ce15caa8 2025-12-17 21:45:45,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815d575bc060ce15caa8 HTTP/1.1" 502 150 2025-12-17 21:45:45,914 - lava_callback - ERROR - Error processing callback for node 6940815d575bc060ce15caa8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940815d575bc060ce15caa8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940815d575bc060ce15caa8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940815d575bc060ce15caa8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:45,917 - lava_callback - INFO - Starting async job submit for node 69408177575bc060ce15cafe 2025-12-17 21:45:45,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:45,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408177575bc060ce15cafe HTTP/1.1" 502 150 2025-12-17 21:45:45,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408177575bc060ce15cafe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:45,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408177575bc060ce15cafe 2025-12-17 21:45:45,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408177575bc060ce15cafe HTTP/1.1" 502 150 2025-12-17 21:45:45,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408177575bc060ce15cafe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:45,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408170575bc060ce15cadd 2025-12-17 21:45:45,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408170575bc060ce15cadd HTTP/1.1" 502 150 2025-12-17 21:45:45,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408170575bc060ce15cadd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:46,016 - lava_callback - INFO - Received callback for node 6940817c575bc060ce15cb10 INFO: 172.21.0.1:51308 - "POST /node/6940817c575bc060ce15cb10 HTTP/1.0" 202 Accepted 2025-12-17 21:45:46,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408173575bc060ce15caec INFO: 172.21.0.1:51320 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:45:46,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815f575bc060ce15caad 2025-12-17 21:45:46,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815f575bc060ce15caad HTTP/1.1" 502 150 2025-12-17 21:45:46,578 - lava_callback - ERROR - Error processing callback for node 6940815f575bc060ce15caad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940815f575bc060ce15caad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940815f575bc060ce15caad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940815f575bc060ce15caad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:46,581 - lava_callback - INFO - Starting async job submit for node 69408178575bc060ce15cb01 2025-12-17 21:45:46,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:46,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408178575bc060ce15cb01 HTTP/1.1" 502 150 2025-12-17 21:45:46,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408178575bc060ce15cb01'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:46,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408178575bc060ce15cb01 2025-12-17 21:45:46,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408178575bc060ce15cb01 HTTP/1.1" 502 150 2025-12-17 21:45:46,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408178575bc060ce15cb01'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:46,701 - lava_callback - INFO - Received callback for node 6940817d575bc060ce15cb13 INFO: 172.21.0.1:51322 - "POST /node/6940817d575bc060ce15cb13 HTTP/1.0" 202 Accepted 2025-12-17 21:45:47,111 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408169575bc060ce15cace 2025-12-17 21:45:47,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408169575bc060ce15cace HTTP/1.1" 502 150 2025-12-17 21:45:47,118 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408169575bc060ce15cace'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:47,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408171575bc060ce15cae2 2025-12-17 21:45:47,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408171575bc060ce15cae2 HTTP/1.1" 502 150 2025-12-17 21:45:47,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408171575bc060ce15cae2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:47,267 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408160575bc060ce15cab5 2025-12-17 21:45:47,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408160575bc060ce15cab5 HTTP/1.1" 502 150 2025-12-17 21:45:47,269 - lava_callback - ERROR - Error processing callback for node 69408160575bc060ce15cab5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408160575bc060ce15cab5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408160575bc060ce15cab5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408160575bc060ce15cab5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:47,271 - lava_callback - INFO - Starting async job submit for node 6940817a575bc060ce15cb0a 2025-12-17 21:45:47,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:47,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817a575bc060ce15cb0a HTTP/1.1" 502 150 2025-12-17 21:45:47,279 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817a575bc060ce15cb0a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:47,279 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817a575bc060ce15cb0a 2025-12-17 21:45:47,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817a575bc060ce15cb0a HTTP/1.1" 502 150 2025-12-17 21:45:47,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817a575bc060ce15cb0a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:47,365 - lava_callback - INFO - Received callback for node 6940817f575bc060ce15cb16 INFO: 172.21.0.1:51338 - "POST /node/6940817f575bc060ce15cb16 HTTP/1.0" 202 Accepted 2025-12-17 21:45:47,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940816a575bc060ce15cad1 2025-12-17 21:45:47,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816a575bc060ce15cad1 HTTP/1.1" 502 150 2025-12-17 21:45:47,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816a575bc060ce15cad1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:47,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408164575bc060ce15cac2 2025-12-17 21:45:47,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408164575bc060ce15cac2 HTTP/1.1" 502 150 2025-12-17 21:45:47,909 - lava_callback - ERROR - Error processing callback for node 69408164575bc060ce15cac2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408164575bc060ce15cac2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408164575bc060ce15cac2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408164575bc060ce15cac2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:47,912 - lava_callback - INFO - Starting async job submit for node 6940817b575bc060ce15cb0d 2025-12-17 21:45:47,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:47,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817b575bc060ce15cb0d HTTP/1.1" 502 150 2025-12-17 21:45:47,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817b575bc060ce15cb0d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:47,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817b575bc060ce15cb0d 2025-12-17 21:45:47,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817b575bc060ce15cb0d HTTP/1.1" 502 150 2025-12-17 21:45:47,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817b575bc060ce15cb0d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:47,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408177575bc060ce15cafe 2025-12-17 21:45:47,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408177575bc060ce15cafe HTTP/1.1" 502 150 2025-12-17 21:45:47,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408177575bc060ce15cafe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:48,041 - lava_callback - INFO - Received callback for node 69408180575bc060ce15cb19 INFO: 172.21.0.1:51352 - "POST /node/69408180575bc060ce15cb19 HTTP/1.0" 202 Accepted 2025-12-17 21:45:48,064 - lava_callback - INFO - Received callback for node 69408181575bc060ce15cb1c INFO: 172.21.0.1:51368 - "POST /node/69408181575bc060ce15cb1c HTTP/1.0" 202 Accepted 2025-12-17 21:45:48,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940816b575bc060ce15cad4 2025-12-17 21:45:48,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816b575bc060ce15cad4 HTTP/1.1" 502 150 2025-12-17 21:45:48,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816b575bc060ce15cad4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:48,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408168575bc060ce15cacb 2025-12-17 21:45:48,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408168575bc060ce15cacb HTTP/1.1" 502 150 2025-12-17 21:45:48,580 - lava_callback - ERROR - Error processing callback for node 69408168575bc060ce15cacb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408168575bc060ce15cacb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408168575bc060ce15cacb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408168575bc060ce15cacb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:48,581 - lava_callback - INFO - Starting async job submit for node 6940817c575bc060ce15cb10 2025-12-17 21:45:48,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:48,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817c575bc060ce15cb10 HTTP/1.1" 502 150 2025-12-17 21:45:48,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817c575bc060ce15cb10'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:48,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817c575bc060ce15cb10 2025-12-17 21:45:48,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817c575bc060ce15cb10 HTTP/1.1" 502 150 2025-12-17 21:45:48,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817c575bc060ce15cb10'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:48,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408178575bc060ce15cb01 2025-12-17 21:45:48,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408178575bc060ce15cb01 HTTP/1.1" 502 150 2025-12-17 21:45:48,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408178575bc060ce15cb01'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:49,152 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408174575bc060ce15caf4 2025-12-17 21:45:49,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408174575bc060ce15caf4 HTTP/1.1" 502 150 2025-12-17 21:45:49,153 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408174575bc060ce15caf4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:49,280 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817a575bc060ce15cb0a 2025-12-17 21:45:49,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817a575bc060ce15cb0a HTTP/1.1" 502 150 2025-12-17 21:45:49,281 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817a575bc060ce15cb0a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:49,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408176575bc060ce15cafb 2025-12-17 21:45:49,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408176575bc060ce15cafb HTTP/1.1" 502 150 2025-12-17 21:45:49,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408176575bc060ce15cafb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:49,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817b575bc060ce15cb0d 2025-12-17 21:45:49,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817b575bc060ce15cb0d HTTP/1.1" 502 150 2025-12-17 21:45:49,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817b575bc060ce15cb0d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:50,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817c575bc060ce15cb10 2025-12-17 21:45:50,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817c575bc060ce15cb10 HTTP/1.1" 502 150 2025-12-17 21:45:50,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817c575bc060ce15cb10'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:51,194 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408171575bc060ce15cae2 2025-12-17 21:45:51,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408171575bc060ce15cae2 HTTP/1.1" 502 150 2025-12-17 21:45:51,196 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408171575bc060ce15cae2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:51,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408123575bc060ce15ca0d 2025-12-17 21:45:51,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408123575bc060ce15ca0d HTTP/1.1" 502 150 2025-12-17 21:45:51,527 - lava_callback - ERROR - Error processing callback for node 69408123575bc060ce15ca0d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408123575bc060ce15ca0d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408123575bc060ce15ca0d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408123575bc060ce15ca0d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:51,529 - lava_callback - INFO - Starting async job submit for node 6940817d575bc060ce15cb13 2025-12-17 21:45:51,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:51,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817d575bc060ce15cb13 HTTP/1.1" 502 150 2025-12-17 21:45:51,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817d575bc060ce15cb13'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:51,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817d575bc060ce15cb13 2025-12-17 21:45:51,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817d575bc060ce15cb13 HTTP/1.1" 502 150 2025-12-17 21:45:51,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817d575bc060ce15cb13'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:51,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408177575bc060ce15cafe 2025-12-17 21:45:51,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408177575bc060ce15cafe HTTP/1.1" 502 150 2025-12-17 21:45:51,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408177575bc060ce15cafe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:52,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816f575bc060ce15cada HTTP/1.1" 502 150 2025-12-17 21:45:52,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816f575bc060ce15cada'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:52,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408178575bc060ce15cb01 2025-12-17 21:45:52,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408178575bc060ce15cb01 HTTP/1.1" 502 150 2025-12-17 21:45:52,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408178575bc060ce15cb01'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:53,281 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817a575bc060ce15cb0a 2025-12-17 21:45:53,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817a575bc060ce15cb0a HTTP/1.1" 502 150 2025-12-17 21:45:53,282 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817a575bc060ce15cb0a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:53,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817d575bc060ce15cb13 2025-12-17 21:45:53,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817d575bc060ce15cb13 HTTP/1.1" 502 150 2025-12-17 21:45:53,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817d575bc060ce15cb13'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:53,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817b575bc060ce15cb0d 2025-12-17 21:45:53,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817b575bc060ce15cb0d HTTP/1.1" 502 150 2025-12-17 21:45:53,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817b575bc060ce15cb0d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:53,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408170575bc060ce15cadd 2025-12-17 21:45:53,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408170575bc060ce15cadd HTTP/1.1" 502 150 2025-12-17 21:45:53,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408170575bc060ce15cadd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:54,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817c575bc060ce15cb10 2025-12-17 21:45:54,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817c575bc060ce15cb10 HTTP/1.1" 502 150 2025-12-17 21:45:54,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817c575bc060ce15cb10'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49378 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:45:56,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940816f575bc060ce15cada 2025-12-17 21:45:56,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816f575bc060ce15cada HTTP/1.1" 502 150 2025-12-17 21:45:56,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816f575bc060ce15cada'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:57,154 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408174575bc060ce15caf4 2025-12-17 21:45:57,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408174575bc060ce15caf4 HTTP/1.1" 502 150 2025-12-17 21:45:57,157 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408174575bc060ce15caf4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:57,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817d575bc060ce15cb13 2025-12-17 21:45:57,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817d575bc060ce15cb13 HTTP/1.1" 502 150 2025-12-17 21:45:57,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817d575bc060ce15cb13'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:57,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408176575bc060ce15cafb 2025-12-17 21:45:57,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408176575bc060ce15cafb HTTP/1.1" 502 150 2025-12-17 21:45:57,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408176575bc060ce15cafb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:59,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408171575bc060ce15cae2 2025-12-17 21:45:59,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408171575bc060ce15cae2 HTTP/1.1" 502 150 2025-12-17 21:45:59,197 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408171575bc060ce15cae2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:59,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940815b575bc060ce15caa0 2025-12-17 21:45:59,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940815b575bc060ce15caa0 HTTP/1.1" 502 150 2025-12-17 21:45:59,927 - lava_callback - ERROR - Error processing callback for node 6940815b575bc060ce15caa0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940815b575bc060ce15caa0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940815b575bc060ce15caa0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940815b575bc060ce15caa0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:45:59,929 - lava_callback - INFO - Starting async job submit for node 6940817f575bc060ce15cb16 2025-12-17 21:45:59,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:45:59,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408177575bc060ce15cafe 2025-12-17 21:45:59,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408177575bc060ce15cafe HTTP/1.1" 502 150 2025-12-17 21:45:59,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408177575bc060ce15cafe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:59,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817f575bc060ce15cb16 HTTP/1.1" 502 150 2025-12-17 21:45:59,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817f575bc060ce15cb16'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:45:59,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817f575bc060ce15cb16 2025-12-17 21:45:59,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817f575bc060ce15cb16 HTTP/1.1" 502 150 2025-12-17 21:45:59,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817f575bc060ce15cb16'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:00,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408178575bc060ce15cb01 2025-12-17 21:46:00,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408178575bc060ce15cb01 HTTP/1.1" 502 150 2025-12-17 21:46:00,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408178575bc060ce15cb01'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:01,283 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817a575bc060ce15cb0a 2025-12-17 21:46:01,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817a575bc060ce15cb0a HTTP/1.1" 502 150 2025-12-17 21:46:01,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817a575bc060ce15cb0a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39142 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:46:01,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408173575bc060ce15caec HTTP/1.1" 502 150 2025-12-17 21:46:01,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408173575bc060ce15caec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:01,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817b575bc060ce15cb0d 2025-12-17 21:46:01,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817b575bc060ce15cb0d HTTP/1.1" 502 150 2025-12-17 21:46:01,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817b575bc060ce15cb0d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:01,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817f575bc060ce15cb16 2025-12-17 21:46:01,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817f575bc060ce15cb16 HTTP/1.1" 502 150 2025-12-17 21:46:01,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817f575bc060ce15cb16'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:02,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817c575bc060ce15cb10 2025-12-17 21:46:02,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817c575bc060ce15cb10 HTTP/1.1" 502 150 2025-12-17 21:46:02,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817c575bc060ce15cb10'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:03,118 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408169575bc060ce15cace 2025-12-17 21:46:03,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940816a575bc060ce15cad1 2025-12-17 21:46:03,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816a575bc060ce15cad1 HTTP/1.1" 502 150 2025-12-17 21:46:03,792 - lava_callback - ERROR - Error processing callback for node 6940816a575bc060ce15cad1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940816a575bc060ce15cad1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940816a575bc060ce15cad1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940816a575bc060ce15cad1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:03,794 - lava_callback - INFO - Starting async job submit for node 69408180575bc060ce15cb19 2025-12-17 21:46:03,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:46:03,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408180575bc060ce15cb19 HTTP/1.1" 502 150 2025-12-17 21:46:03,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408180575bc060ce15cb19'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:03,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408180575bc060ce15cb19 2025-12-17 21:46:03,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408180575bc060ce15cb19 HTTP/1.1" 502 150 2025-12-17 21:46:03,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408180575bc060ce15cb19'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:04,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940816b575bc060ce15cad4 2025-12-17 21:46:04,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816b575bc060ce15cad4 HTTP/1.1" 502 150 2025-12-17 21:46:04,488 - lava_callback - ERROR - Error processing callback for node 6940816b575bc060ce15cad4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940816b575bc060ce15cad4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940816b575bc060ce15cad4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940816b575bc060ce15cad4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:04,490 - lava_callback - INFO - Starting async job submit for node 69408181575bc060ce15cb1c 2025-12-17 21:46:04,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:46:04,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408181575bc060ce15cb1c HTTP/1.1" 502 150 2025-12-17 21:46:04,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408181575bc060ce15cb1c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:04,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408181575bc060ce15cb1c 2025-12-17 21:46:04,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408181575bc060ce15cb1c HTTP/1.1" 502 150 2025-12-17 21:46:04,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408181575bc060ce15cb1c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:04,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940816f575bc060ce15cada 2025-12-17 21:46:04,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816f575bc060ce15cada HTTP/1.1" 502 150 2025-12-17 21:46:04,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940816f575bc060ce15cada'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:05,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817d575bc060ce15cb13 2025-12-17 21:46:05,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817d575bc060ce15cb13 HTTP/1.1" 502 150 2025-12-17 21:46:05,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817d575bc060ce15cb13'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:05,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408180575bc060ce15cb19 2025-12-17 21:46:05,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408180575bc060ce15cb19 HTTP/1.1" 502 150 2025-12-17 21:46:05,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408180575bc060ce15cb19'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:05,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817f575bc060ce15cb16 2025-12-17 21:46:05,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817f575bc060ce15cb16 HTTP/1.1" 502 150 2025-12-17 21:46:05,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817f575bc060ce15cb16'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39158 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:46:06,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408181575bc060ce15cb1c 2025-12-17 21:46:06,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408181575bc060ce15cb1c HTTP/1.1" 502 150 2025-12-17 21:46:06,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408181575bc060ce15cb1c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:09,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408173575bc060ce15caec 2025-12-17 21:46:09,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408173575bc060ce15caec HTTP/1.1" 502 150 2025-12-17 21:46:09,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408173575bc060ce15caec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:09,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408180575bc060ce15cb19 2025-12-17 21:46:09,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408180575bc060ce15cb19 HTTP/1.1" 502 150 2025-12-17 21:46:09,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408180575bc060ce15cb19'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:09,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408170575bc060ce15cadd 2025-12-17 21:46:09,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408170575bc060ce15cadd HTTP/1.1" 502 150 2025-12-17 21:46:09,971 - lava_callback - ERROR - Error processing callback for node 69408170575bc060ce15cadd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408170575bc060ce15cadd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408170575bc060ce15cadd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408170575bc060ce15cadd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:10,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408181575bc060ce15cb1c 2025-12-17 21:46:10,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408181575bc060ce15cb1c HTTP/1.1" 502 150 2025-12-17 21:46:10,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408181575bc060ce15cb1c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:13,144 - lava_callback - INFO - Received callback for node 69408196575bc060ce15cb35 INFO: 172.21.0.1:48894 - "POST /node/69408196575bc060ce15cb35 HTTP/1.0" 202 Accepted 2025-12-17 21:46:13,153 - lava_callback - INFO - Starting async job submit for node 69408196575bc060ce15cb35 2025-12-17 21:46:13,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:46:13,157 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408174575bc060ce15caf4 2025-12-17 21:46:13,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408174575bc060ce15caf4 HTTP/1.1" 502 150 2025-12-17 21:46:13,158 - lava_callback - ERROR - Error processing callback for node 69408174575bc060ce15caf4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408174575bc060ce15caf4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408174575bc060ce15caf4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408174575bc060ce15caf4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:13,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408196575bc060ce15cb35 HTTP/1.1" 502 150 2025-12-17 21:46:13,162 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408196575bc060ce15cb35'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:13,162 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408196575bc060ce15cb35 2025-12-17 21:46:13,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408196575bc060ce15cb35 HTTP/1.1" 502 150 2025-12-17 21:46:13,162 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408196575bc060ce15cb35'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:13,810 - lava_callback - INFO - Received callback for node 6940819b575bc060ce15cb58 INFO: 172.21.0.1:48908 - "POST /node/6940819b575bc060ce15cb58 HTTP/1.0" 202 Accepted 2025-12-17 21:46:13,820 - lava_callback - INFO - Starting async job submit for node 6940819b575bc060ce15cb58 2025-12-17 21:46:13,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:46:13,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408176575bc060ce15cafb 2025-12-17 21:46:13,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408176575bc060ce15cafb HTTP/1.1" 502 150 2025-12-17 21:46:13,829 - lava_callback - ERROR - Error processing callback for node 69408176575bc060ce15cafb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408176575bc060ce15cafb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408176575bc060ce15cafb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408176575bc060ce15cafb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:13,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819b575bc060ce15cb58 HTTP/1.1" 502 150 2025-12-17 21:46:13,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819b575bc060ce15cb58'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:13,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819b575bc060ce15cb58 2025-12-17 21:46:13,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819b575bc060ce15cb58 HTTP/1.1" 502 150 2025-12-17 21:46:13,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819b575bc060ce15cb58'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:13,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817f575bc060ce15cb16 2025-12-17 21:46:13,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817f575bc060ce15cb16 HTTP/1.1" 502 150 2025-12-17 21:46:13,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940817f575bc060ce15cb16'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:14,497 - lava_callback - INFO - Received callback for node 6940819c575bc060ce15cb5c INFO: 172.21.0.1:48912 - "POST /node/6940819c575bc060ce15cb5c HTTP/1.0" 202 Accepted 2025-12-17 21:46:14,506 - lava_callback - INFO - Starting async job submit for node 6940819c575bc060ce15cb5c 2025-12-17 21:46:14,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:46:14,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819c575bc060ce15cb5c HTTP/1.1" 502 150 2025-12-17 21:46:14,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819c575bc060ce15cb5c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:14,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819c575bc060ce15cb5c 2025-12-17 21:46:14,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819c575bc060ce15cb5c HTTP/1.1" 502 150 2025-12-17 21:46:14,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819c575bc060ce15cb5c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:15,159 - lava_callback - INFO - Received callback for node 6940819d575bc060ce15cb63 INFO: 172.21.0.1:48916 - "POST /node/6940819d575bc060ce15cb63 HTTP/1.0" 202 Accepted 2025-12-17 21:46:15,165 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408196575bc060ce15cb35 INFO: 172.21.0.1:48926 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:46:15,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408196575bc060ce15cb35 HTTP/1.1" 502 150 2025-12-17 21:46:15,166 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408196575bc060ce15cb35'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:15,197 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408171575bc060ce15cae2 2025-12-17 21:46:15,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408171575bc060ce15cae2 HTTP/1.1" 502 150 2025-12-17 21:46:15,199 - lava_callback - ERROR - Error processing callback for node 69408171575bc060ce15cae2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408171575bc060ce15cae2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408171575bc060ce15cae2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408171575bc060ce15cae2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:15,201 - lava_callback - INFO - Starting async job submit for node 6940819d575bc060ce15cb63 2025-12-17 21:46:15,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:46:15,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819d575bc060ce15cb63 HTTP/1.1" 502 150 2025-12-17 21:46:15,210 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819d575bc060ce15cb63'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:15,210 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819d575bc060ce15cb63 2025-12-17 21:46:15,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819d575bc060ce15cb63 HTTP/1.1" 502 150 2025-12-17 21:46:15,211 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819d575bc060ce15cb63'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:15,843 - lava_callback - INFO - Received callback for node 6940819e575bc060ce15cb68 INFO: 172.21.0.1:48938 - "POST /node/6940819e575bc060ce15cb68 HTTP/1.0" 202 Accepted 2025-12-17 21:46:15,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819b575bc060ce15cb58 2025-12-17 21:46:15,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819b575bc060ce15cb58 HTTP/1.1" 502 150 2025-12-17 21:46:15,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819b575bc060ce15cb58'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:15,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408177575bc060ce15cafe 2025-12-17 21:46:15,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408177575bc060ce15cafe HTTP/1.1" 502 150 2025-12-17 21:46:15,933 - lava_callback - ERROR - Error processing callback for node 69408177575bc060ce15cafe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408177575bc060ce15cafe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408177575bc060ce15cafe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408177575bc060ce15cafe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:15,935 - lava_callback - INFO - Starting async job submit for node 6940819e575bc060ce15cb68 2025-12-17 21:46:15,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:46:15,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819e575bc060ce15cb68 HTTP/1.1" 502 150 2025-12-17 21:46:15,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819e575bc060ce15cb68'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:15,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819e575bc060ce15cb68 2025-12-17 21:46:15,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819e575bc060ce15cb68 HTTP/1.1" 502 150 2025-12-17 21:46:15,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819e575bc060ce15cb68'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48954 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:46:16,513 - lava_callback - INFO - Received callback for node 694081a0575bc060ce15cb6b INFO: 172.21.0.1:48960 - "POST /node/694081a0575bc060ce15cb6b HTTP/1.0" 202 Accepted 2025-12-17 21:46:16,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819c575bc060ce15cb5c 2025-12-17 21:46:16,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819c575bc060ce15cb5c HTTP/1.1" 502 150 2025-12-17 21:46:16,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819c575bc060ce15cb5c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:16,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408178575bc060ce15cb01 2025-12-17 21:46:16,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408178575bc060ce15cb01 HTTP/1.1" 502 150 2025-12-17 21:46:16,595 - lava_callback - ERROR - Error processing callback for node 69408178575bc060ce15cb01: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408178575bc060ce15cb01 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408178575bc060ce15cb01 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408178575bc060ce15cb01 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:16,596 - lava_callback - INFO - Starting async job submit for node 694081a0575bc060ce15cb6b 2025-12-17 21:46:16,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:46:16,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a0575bc060ce15cb6b HTTP/1.1" 502 150 2025-12-17 21:46:16,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a0575bc060ce15cb6b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:16,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a0575bc060ce15cb6b 2025-12-17 21:46:16,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a0575bc060ce15cb6b HTTP/1.1" 502 150 2025-12-17 21:46:16,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a0575bc060ce15cb6b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:17,183 - lava_callback - INFO - Received callback for node 694081a1575bc060ce15cb6e INFO: 172.21.0.1:48966 - "POST /node/694081a1575bc060ce15cb6e HTTP/1.0" 202 Accepted 2025-12-17 21:46:17,211 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819d575bc060ce15cb63 2025-12-17 21:46:17,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819d575bc060ce15cb63 HTTP/1.1" 502 150 2025-12-17 21:46:17,216 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819d575bc060ce15cb63'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:17,284 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817a575bc060ce15cb0a 2025-12-17 21:46:17,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817a575bc060ce15cb0a HTTP/1.1" 502 150 2025-12-17 21:46:17,285 - lava_callback - ERROR - Error processing callback for node 6940817a575bc060ce15cb0a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940817a575bc060ce15cb0a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940817a575bc060ce15cb0a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940817a575bc060ce15cb0a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:17,287 - lava_callback - INFO - Starting async job submit for node 694081a1575bc060ce15cb6e 2025-12-17 21:46:17,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:46:17,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a1575bc060ce15cb6e HTTP/1.1" 502 150 2025-12-17 21:46:17,293 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a1575bc060ce15cb6e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:17,293 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a1575bc060ce15cb6e 2025-12-17 21:46:17,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a1575bc060ce15cb6e HTTP/1.1" 502 150 2025-12-17 21:46:17,293 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a1575bc060ce15cb6e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:17,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408180575bc060ce15cb19 2025-12-17 21:46:17,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408180575bc060ce15cb19 HTTP/1.1" 502 150 2025-12-17 21:46:17,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408180575bc060ce15cb19'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:17,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817b575bc060ce15cb0d 2025-12-17 21:46:17,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817b575bc060ce15cb0d HTTP/1.1" 502 150 2025-12-17 21:46:17,930 - lava_callback - ERROR - Error processing callback for node 6940817b575bc060ce15cb0d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940817b575bc060ce15cb0d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940817b575bc060ce15cb0d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940817b575bc060ce15cb0d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:17,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819e575bc060ce15cb68 2025-12-17 21:46:17,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819e575bc060ce15cb68 HTTP/1.1" 502 150 2025-12-17 21:46:17,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819e575bc060ce15cb68'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:18,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408169575bc060ce15cace HTTP/1.1" 502 150 2025-12-17 21:46:18,416 - lava_callback - ERROR - Error processing callback for node 69408169575bc060ce15cace: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408169575bc060ce15cace (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408169575bc060ce15cace (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408169575bc060ce15cace (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:18,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408181575bc060ce15cb1c 2025-12-17 21:46:18,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408181575bc060ce15cb1c HTTP/1.1" 502 150 2025-12-17 21:46:18,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408181575bc060ce15cb1c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:18,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817c575bc060ce15cb10 2025-12-17 21:46:18,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817c575bc060ce15cb10 HTTP/1.1" 502 150 2025-12-17 21:46:18,596 - lava_callback - ERROR - Error processing callback for node 6940817c575bc060ce15cb10: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940817c575bc060ce15cb10 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940817c575bc060ce15cb10 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940817c575bc060ce15cb10 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:18,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a0575bc060ce15cb6b 2025-12-17 21:46:18,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a0575bc060ce15cb6b HTTP/1.1" 502 150 2025-12-17 21:46:18,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a0575bc060ce15cb6b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:19,167 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408196575bc060ce15cb35 2025-12-17 21:46:19,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408196575bc060ce15cb35 HTTP/1.1" 502 150 2025-12-17 21:46:19,168 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408196575bc060ce15cb35'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:19,294 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a1575bc060ce15cb6e 2025-12-17 21:46:19,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a1575bc060ce15cb6e HTTP/1.1" 502 150 2025-12-17 21:46:19,295 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a1575bc060ce15cb6e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:19,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819b575bc060ce15cb58 2025-12-17 21:46:19,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819b575bc060ce15cb58 HTTP/1.1" 502 150 2025-12-17 21:46:19,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819b575bc060ce15cb58'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:20,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819c575bc060ce15cb5c 2025-12-17 21:46:20,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819c575bc060ce15cb5c HTTP/1.1" 502 150 2025-12-17 21:46:20,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819c575bc060ce15cb5c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:20,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940816f575bc060ce15cada 2025-12-17 21:46:20,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940816f575bc060ce15cada HTTP/1.1" 502 150 2025-12-17 21:46:20,568 - lava_callback - ERROR - Error processing callback for node 6940816f575bc060ce15cada: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940816f575bc060ce15cada (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940816f575bc060ce15cada (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940816f575bc060ce15cada (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:21,217 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819d575bc060ce15cb63 2025-12-17 21:46:21,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819d575bc060ce15cb63 HTTP/1.1" 502 150 2025-12-17 21:46:21,218 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819d575bc060ce15cb63'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:21,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817d575bc060ce15cb13 2025-12-17 21:46:21,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817d575bc060ce15cb13 HTTP/1.1" 502 150 2025-12-17 21:46:21,542 - lava_callback - ERROR - Error processing callback for node 6940817d575bc060ce15cb13: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940817d575bc060ce15cb13 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940817d575bc060ce15cb13 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940817d575bc060ce15cb13 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:21,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819e575bc060ce15cb68 2025-12-17 21:46:21,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819e575bc060ce15cb68 HTTP/1.1" 502 150 2025-12-17 21:46:21,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819e575bc060ce15cb68'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:22,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a0575bc060ce15cb6b 2025-12-17 21:46:22,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a0575bc060ce15cb6b HTTP/1.1" 502 150 2025-12-17 21:46:22,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a0575bc060ce15cb6b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:22,751 - lava_callback - INFO - Received callback for node 694081ad575bc060ce15cb7c INFO: 172.21.0.1:39414 - "POST /node/694081ad575bc060ce15cb7c HTTP/1.0" 202 Accepted 2025-12-17 21:46:22,756 - lava_callback - INFO - Starting async job submit for node 694081ad575bc060ce15cb7c 2025-12-17 21:46:22,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:46:22,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ad575bc060ce15cb7c HTTP/1.1" 502 150 2025-12-17 21:46:22,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ad575bc060ce15cb7c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:22,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ad575bc060ce15cb7c 2025-12-17 21:46:22,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ad575bc060ce15cb7c HTTP/1.1" 502 150 2025-12-17 21:46:22,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ad575bc060ce15cb7c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:23,295 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a1575bc060ce15cb6e 2025-12-17 21:46:23,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a1575bc060ce15cb6e HTTP/1.1" 502 150 2025-12-17 21:46:23,296 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a1575bc060ce15cb6e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:24,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ad575bc060ce15cb7c 2025-12-17 21:46:24,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ad575bc060ce15cb7c HTTP/1.1" 502 150 2025-12-17 21:46:24,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ad575bc060ce15cb7c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:25,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408173575bc060ce15caec 2025-12-17 21:46:25,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408173575bc060ce15caec HTTP/1.1" 502 150 2025-12-17 21:46:25,780 - lava_callback - ERROR - Error processing callback for node 69408173575bc060ce15caec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408173575bc060ce15caec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408173575bc060ce15caec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408173575bc060ce15caec (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:39428 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:46:27,168 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408196575bc060ce15cb35 2025-12-17 21:46:27,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819b575bc060ce15cb58 2025-12-17 21:46:27,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819b575bc060ce15cb58 HTTP/1.1" 502 150 2025-12-17 21:46:27,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819b575bc060ce15cb58'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:28,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819c575bc060ce15cb5c 2025-12-17 21:46:28,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819c575bc060ce15cb5c HTTP/1.1" 502 150 2025-12-17 21:46:28,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819c575bc060ce15cb5c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:28,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ad575bc060ce15cb7c 2025-12-17 21:46:28,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ad575bc060ce15cb7c HTTP/1.1" 502 150 2025-12-17 21:46:28,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ad575bc060ce15cb7c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:29,218 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819d575bc060ce15cb63 2025-12-17 21:46:29,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940817f575bc060ce15cb16 2025-12-17 21:46:29,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940817f575bc060ce15cb16 HTTP/1.1" 502 150 2025-12-17 21:46:29,960 - lava_callback - ERROR - Error processing callback for node 6940817f575bc060ce15cb16: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940817f575bc060ce15cb16 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940817f575bc060ce15cb16 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940817f575bc060ce15cb16 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:29,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819e575bc060ce15cb68 2025-12-17 21:46:29,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819e575bc060ce15cb68 HTTP/1.1" 502 150 2025-12-17 21:46:29,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819e575bc060ce15cb68'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:30,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a0575bc060ce15cb6b 2025-12-17 21:46:30,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a0575bc060ce15cb6b HTTP/1.1" 502 150 2025-12-17 21:46:30,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a0575bc060ce15cb6b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:31,297 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a1575bc060ce15cb6e 2025-12-17 21:46:31,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a1575bc060ce15cb6e HTTP/1.1" 502 150 2025-12-17 21:46:31,298 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a1575bc060ce15cb6e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:31,720 - lava_callback - INFO - Received callback for node 694081a2575bc060ce15cb71 INFO: 172.21.0.1:40076 - "POST /node/694081a2575bc060ce15cb71 HTTP/1.0" 202 Accepted 2025-12-17 21:46:31,726 - lava_callback - INFO - Starting async job submit for node 694081a2575bc060ce15cb71 2025-12-17 21:46:31,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:46:31,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a2575bc060ce15cb71 HTTP/1.1" 502 150 2025-12-17 21:46:31,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a2575bc060ce15cb71'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:31,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a2575bc060ce15cb71 2025-12-17 21:46:31,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a2575bc060ce15cb71 HTTP/1.1" 502 150 2025-12-17 21:46:31,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a2575bc060ce15cb71'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:32,346 - lava_callback - INFO - Received callback for node 694081a5575bc060ce15cb74 INFO: 172.21.0.1:40080 - "POST /node/694081a5575bc060ce15cb74 HTTP/1.0" 202 Accepted 2025-12-17 21:46:32,352 - lava_callback - INFO - Starting async job submit for node 694081a5575bc060ce15cb74 2025-12-17 21:46:32,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:46:32,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a5575bc060ce15cb74 HTTP/1.1" 502 150 2025-12-17 21:46:32,374 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a5575bc060ce15cb74'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:32,374 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a5575bc060ce15cb74 2025-12-17 21:46:32,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a5575bc060ce15cb74 HTTP/1.1" 502 150 2025-12-17 21:46:32,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a5575bc060ce15cb74'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:32,948 - lava_callback - INFO - Received callback for node 694081a7575bc060ce15cb77 INFO: 172.21.0.1:40086 - "POST /node/694081a7575bc060ce15cb77 HTTP/1.0" 202 Accepted 2025-12-17 21:46:32,954 - lava_callback - INFO - Starting async job submit for node 694081a7575bc060ce15cb77 2025-12-17 21:46:32,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:46:32,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a7575bc060ce15cb77 HTTP/1.1" 502 150 2025-12-17 21:46:32,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a7575bc060ce15cb77'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:32,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a7575bc060ce15cb77 2025-12-17 21:46:32,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a7575bc060ce15cb77 HTTP/1.1" 502 150 2025-12-17 21:46:32,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a7575bc060ce15cb77'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:33,120 - lava_callback - INFO - Received callback for node 69407f05575bc060ce15c3ca INFO: 172.21.0.1:40102 - "POST /node/69407f05575bc060ce15c3ca HTTP/1.0" 202 Accepted 2025-12-17 21:46:33,128 - lava_callback - INFO - Starting async job submit for node 69407f05575bc060ce15c3ca 2025-12-17 21:46:33,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:46:33,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f05575bc060ce15c3ca HTTP/1.1" 502 150 2025-12-17 21:46:33,201 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f05575bc060ce15c3ca'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:33,201 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f05575bc060ce15c3ca 2025-12-17 21:46:33,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f05575bc060ce15c3ca HTTP/1.1" 502 150 2025-12-17 21:46:33,202 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f05575bc060ce15c3ca'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:33,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a2575bc060ce15cb71 2025-12-17 21:46:33,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a2575bc060ce15cb71 HTTP/1.1" 502 150 2025-12-17 21:46:33,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a2575bc060ce15cb71'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:33,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408180575bc060ce15cb19 2025-12-17 21:46:33,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408180575bc060ce15cb19 HTTP/1.1" 502 150 2025-12-17 21:46:33,807 - lava_callback - ERROR - Error processing callback for node 69408180575bc060ce15cb19: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408180575bc060ce15cb19 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408180575bc060ce15cb19 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408180575bc060ce15cb19 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:34,375 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a5575bc060ce15cb74 2025-12-17 21:46:34,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a5575bc060ce15cb74 HTTP/1.1" 502 150 2025-12-17 21:46:34,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a5575bc060ce15cb74'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:34,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408181575bc060ce15cb1c 2025-12-17 21:46:34,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408181575bc060ce15cb1c HTTP/1.1" 502 150 2025-12-17 21:46:34,525 - lava_callback - ERROR - Error processing callback for node 69408181575bc060ce15cb1c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408181575bc060ce15cb1c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408181575bc060ce15cb1c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408181575bc060ce15cb1c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:34,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a7575bc060ce15cb77 2025-12-17 21:46:34,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a7575bc060ce15cb77 HTTP/1.1" 502 150 2025-12-17 21:46:34,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a7575bc060ce15cb77'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:35,202 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f05575bc060ce15c3ca 2025-12-17 21:46:35,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f05575bc060ce15c3ca HTTP/1.1" 502 150 2025-12-17 21:46:35,203 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f05575bc060ce15c3ca'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40116 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:46:36,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ad575bc060ce15cb7c 2025-12-17 21:46:36,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ad575bc060ce15cb7c HTTP/1.1" 502 150 2025-12-17 21:46:36,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ad575bc060ce15cb7c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:37,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a2575bc060ce15cb71 2025-12-17 21:46:37,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a2575bc060ce15cb71 HTTP/1.1" 502 150 2025-12-17 21:46:37,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a2575bc060ce15cb71'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:38,377 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a5575bc060ce15cb74 2025-12-17 21:46:38,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a7575bc060ce15cb77 2025-12-17 21:46:38,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a7575bc060ce15cb77 HTTP/1.1" 502 150 2025-12-17 21:46:38,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a7575bc060ce15cb77'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:39,204 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f05575bc060ce15c3ca 2025-12-17 21:46:39,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f05575bc060ce15c3ca HTTP/1.1" 502 150 2025-12-17 21:46:39,205 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f05575bc060ce15c3ca'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:42,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408196575bc060ce15cb35 HTTP/1.1" 502 150 2025-12-17 21:46:42,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408196575bc060ce15cb35'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:42,772 - lava_callback - INFO - Received callback for node 694081c1575bc060ce15cb90 INFO: 172.21.0.1:39320 - "POST /node/694081c1575bc060ce15cb90 HTTP/1.0" 202 Accepted 2025-12-17 21:46:42,778 - lava_callback - INFO - Starting async job submit for node 694081c1575bc060ce15cb90 2025-12-17 21:46:42,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:46:42,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c1575bc060ce15cb90 HTTP/1.1" 502 150 2025-12-17 21:46:42,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c1575bc060ce15cb90'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:42,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c1575bc060ce15cb90 2025-12-17 21:46:42,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c1575bc060ce15cb90 HTTP/1.1" 502 150 2025-12-17 21:46:42,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c1575bc060ce15cb90'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:43,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819b575bc060ce15cb58 2025-12-17 21:46:43,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819b575bc060ce15cb58 HTTP/1.1" 502 150 2025-12-17 21:46:43,860 - lava_callback - ERROR - Error processing callback for node 6940819b575bc060ce15cb58: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940819b575bc060ce15cb58 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940819b575bc060ce15cb58 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940819b575bc060ce15cb58 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:44,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819c575bc060ce15cb5c 2025-12-17 21:46:44,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819c575bc060ce15cb5c HTTP/1.1" 502 150 2025-12-17 21:46:44,525 - lava_callback - ERROR - Error processing callback for node 6940819c575bc060ce15cb5c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940819c575bc060ce15cb5c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940819c575bc060ce15cb5c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940819c575bc060ce15cb5c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:44,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819d575bc060ce15cb63 HTTP/1.1" 502 150 2025-12-17 21:46:44,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940819d575bc060ce15cb63'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:44,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c1575bc060ce15cb90 2025-12-17 21:46:44,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c1575bc060ce15cb90 HTTP/1.1" 502 150 2025-12-17 21:46:44,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c1575bc060ce15cb90'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:45,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a2575bc060ce15cb71 2025-12-17 21:46:45,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a2575bc060ce15cb71 HTTP/1.1" 502 150 2025-12-17 21:46:45,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a2575bc060ce15cb71'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:45,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819e575bc060ce15cb68 2025-12-17 21:46:45,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819e575bc060ce15cb68 HTTP/1.1" 502 150 2025-12-17 21:46:45,970 - lava_callback - ERROR - Error processing callback for node 6940819e575bc060ce15cb68: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940819e575bc060ce15cb68 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940819e575bc060ce15cb68 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940819e575bc060ce15cb68 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:39336 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:46:46,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a0575bc060ce15cb6b 2025-12-17 21:46:46,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a0575bc060ce15cb6b HTTP/1.1" 502 150 2025-12-17 21:46:46,609 - lava_callback - ERROR - Error processing callback for node 694081a0575bc060ce15cb6b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081a0575bc060ce15cb6b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081a0575bc060ce15cb6b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081a0575bc060ce15cb6b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:46,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a7575bc060ce15cb77 2025-12-17 21:46:46,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a7575bc060ce15cb77 HTTP/1.1" 502 150 2025-12-17 21:46:46,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a7575bc060ce15cb77'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:47,205 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f05575bc060ce15c3ca 2025-12-17 21:46:47,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f05575bc060ce15c3ca HTTP/1.1" 502 150 2025-12-17 21:46:47,206 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f05575bc060ce15c3ca'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:47,298 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a1575bc060ce15cb6e 2025-12-17 21:46:47,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a1575bc060ce15cb6e HTTP/1.1" 502 150 2025-12-17 21:46:47,299 - lava_callback - ERROR - Error processing callback for node 694081a1575bc060ce15cb6e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081a1575bc060ce15cb6e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081a1575bc060ce15cb6e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081a1575bc060ce15cb6e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:48,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c1575bc060ce15cb90 2025-12-17 21:46:48,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c1575bc060ce15cb90 HTTP/1.1" 502 150 2025-12-17 21:46:48,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c1575bc060ce15cb90'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:52,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ad575bc060ce15cb7c 2025-12-17 21:46:52,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ad575bc060ce15cb7c HTTP/1.1" 502 150 2025-12-17 21:46:52,772 - lava_callback - ERROR - Error processing callback for node 694081ad575bc060ce15cb7c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ad575bc060ce15cb7c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ad575bc060ce15cb7c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ad575bc060ce15cb7c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:53,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a5575bc060ce15cb74 HTTP/1.1" 502 150 2025-12-17 21:46:53,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a5575bc060ce15cb74'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:56,507 - lava_callback - INFO - Received callback for node 69407f65575bc060ce15c452 INFO: 172.21.0.1:44218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44210 - "POST /node/69407f65575bc060ce15c452 HTTP/1.0" 202 Accepted 2025-12-17 21:46:56,557 - lava_callback - INFO - Starting async job submit for node 69407f65575bc060ce15c452 2025-12-17 21:46:56,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:46:56,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f65575bc060ce15c452 HTTP/1.1" 502 150 2025-12-17 21:46:56,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f65575bc060ce15c452'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:56,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f65575bc060ce15c452 2025-12-17 21:46:56,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f65575bc060ce15c452 HTTP/1.1" 502 150 2025-12-17 21:46:56,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f65575bc060ce15c452'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:56,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c1575bc060ce15cb90 2025-12-17 21:46:56,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c1575bc060ce15cb90 HTTP/1.1" 502 150 2025-12-17 21:46:56,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c1575bc060ce15cb90'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:46:58,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408196575bc060ce15cb35 2025-12-17 21:46:58,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408196575bc060ce15cb35 HTTP/1.1" 502 150 2025-12-17 21:46:58,482 - lava_callback - ERROR - Error processing callback for node 69408196575bc060ce15cb35: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408196575bc060ce15cb35 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408196575bc060ce15cb35 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408196575bc060ce15cb35 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:46:58,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f65575bc060ce15c452 2025-12-17 21:46:58,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f65575bc060ce15c452 HTTP/1.1" 502 150 2025-12-17 21:46:58,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f65575bc060ce15c452'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:00,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940819d575bc060ce15cb63 2025-12-17 21:47:00,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940819d575bc060ce15cb63 HTTP/1.1" 502 150 2025-12-17 21:47:00,529 - lava_callback - ERROR - Error processing callback for node 6940819d575bc060ce15cb63: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940819d575bc060ce15cb63 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940819d575bc060ce15cb63 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940819d575bc060ce15cb63 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:01,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a2575bc060ce15cb71 2025-12-17 21:47:01,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a2575bc060ce15cb71 HTTP/1.1" 502 150 2025-12-17 21:47:01,739 - lava_callback - ERROR - Error processing callback for node 694081a2575bc060ce15cb71: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081a2575bc060ce15cb71 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081a2575bc060ce15cb71 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081a2575bc060ce15cb71 (Caused by ResponseError('too many 502 error responses')) INFO: 172.21.0.1:54664 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:47:01,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a5575bc060ce15cb74 2025-12-17 21:47:01,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a5575bc060ce15cb74 HTTP/1.1" 502 150 2025-12-17 21:47:01,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081a5575bc060ce15cb74'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:02,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f65575bc060ce15c452 2025-12-17 21:47:02,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f65575bc060ce15c452 HTTP/1.1" 502 150 2025-12-17 21:47:02,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f65575bc060ce15c452'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:02,778 - lava_callback - INFO - Received callback for node 694081c3575bc060ce15cb99 INFO: 172.21.0.1:54676 - "POST /node/694081c3575bc060ce15cb99 HTTP/1.0" 202 Accepted 2025-12-17 21:47:02,783 - lava_callback - INFO - Starting async job submit for node 694081c3575bc060ce15cb99 2025-12-17 21:47:02,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:02,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c3575bc060ce15cb99 HTTP/1.1" 502 150 2025-12-17 21:47:02,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c3575bc060ce15cb99'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:02,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c3575bc060ce15cb99 2025-12-17 21:47:02,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c3575bc060ce15cb99 HTTP/1.1" 502 150 2025-12-17 21:47:02,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c3575bc060ce15cb99'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:02,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a7575bc060ce15cb77 2025-12-17 21:47:02,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a7575bc060ce15cb77 HTTP/1.1" 502 150 2025-12-17 21:47:02,989 - lava_callback - ERROR - Error processing callback for node 694081a7575bc060ce15cb77: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081a7575bc060ce15cb77 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081a7575bc060ce15cb77 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081a7575bc060ce15cb77 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:03,206 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f05575bc060ce15c3ca 2025-12-17 21:47:03,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f05575bc060ce15c3ca HTTP/1.1" 502 150 2025-12-17 21:47:03,208 - lava_callback - ERROR - Error processing callback for node 69407f05575bc060ce15c3ca: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f05575bc060ce15c3ca (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f05575bc060ce15c3ca (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f05575bc060ce15c3ca (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:03,452 - lava_callback - INFO - Received callback for node 694081c4575bc060ce15cba0 INFO: 172.21.0.1:54680 - "POST /node/694081c4575bc060ce15cba0 HTTP/1.0" 202 Accepted 2025-12-17 21:47:03,458 - lava_callback - INFO - Starting async job submit for node 694081c4575bc060ce15cba0 2025-12-17 21:47:03,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:03,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c4575bc060ce15cba0 HTTP/1.1" 502 150 2025-12-17 21:47:03,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c4575bc060ce15cba0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:03,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c4575bc060ce15cba0 2025-12-17 21:47:03,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c4575bc060ce15cba0 HTTP/1.1" 502 150 2025-12-17 21:47:03,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c4575bc060ce15cba0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:04,131 - lava_callback - INFO - Received callback for node 694081c6575bc060ce15cba7 INFO: 172.21.0.1:54686 - "POST /node/694081c6575bc060ce15cba7 HTTP/1.0" 202 Accepted 2025-12-17 21:47:04,137 - lava_callback - INFO - Starting async job submit for node 694081c6575bc060ce15cba7 2025-12-17 21:47:04,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:04,789 - lava_callback - INFO - Received callback for node 694081c7575bc060ce15cbac INFO: 172.21.0.1:54702 - "POST /node/694081c7575bc060ce15cbac HTTP/1.0" 202 Accepted 2025-12-17 21:47:04,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c3575bc060ce15cb99 2025-12-17 21:47:04,795 - lava_callback - INFO - Starting async job submit for node 694081c7575bc060ce15cbac 2025-12-17 21:47:04,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:04,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c3575bc060ce15cb99 HTTP/1.1" 502 150 2025-12-17 21:47:04,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c3575bc060ce15cb99'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:04,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c7575bc060ce15cbac HTTP/1.1" 502 150 2025-12-17 21:47:04,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c7575bc060ce15cbac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:04,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c7575bc060ce15cbac 2025-12-17 21:47:04,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c7575bc060ce15cbac HTTP/1.1" 502 150 2025-12-17 21:47:04,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c7575bc060ce15cbac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:05,462 - lava_callback - INFO - Received callback for node 694081c8575bc060ce15cbaf INFO: 172.21.0.1:54714 - "POST /node/694081c8575bc060ce15cbaf HTTP/1.0" 202 Accepted 2025-12-17 21:47:05,468 - lava_callback - INFO - Starting async job submit for node 694081c8575bc060ce15cbaf 2025-12-17 21:47:05,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c4575bc060ce15cba0 2025-12-17 21:47:05,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:05,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c4575bc060ce15cba0 HTTP/1.1" 502 150 2025-12-17 21:47:05,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c4575bc060ce15cba0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:05,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c8575bc060ce15cbaf HTTP/1.1" 502 150 2025-12-17 21:47:05,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c8575bc060ce15cbaf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:05,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c8575bc060ce15cbaf 2025-12-17 21:47:05,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c8575bc060ce15cbaf HTTP/1.1" 502 150 2025-12-17 21:47:05,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c8575bc060ce15cbaf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:06,137 - lava_callback - INFO - Received callback for node 694081c9575bc060ce15cbb2 INFO: 172.21.0.1:54716 - "POST /node/694081c9575bc060ce15cbb2 HTTP/1.0" 202 Accepted 2025-12-17 21:47:06,142 - lava_callback - INFO - Starting async job submit for node 694081c9575bc060ce15cbb2 2025-12-17 21:47:06,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:06,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c9575bc060ce15cbb2 HTTP/1.1" 502 150 2025-12-17 21:47:06,176 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c9575bc060ce15cbb2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:06,176 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c9575bc060ce15cbb2 2025-12-17 21:47:06,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c9575bc060ce15cbb2 HTTP/1.1" 502 150 2025-12-17 21:47:06,177 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c9575bc060ce15cbb2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54730 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:47:06,803 - lava_callback - INFO - Received callback for node 694081cb575bc060ce15cbb5 2025-12-17 21:47:06,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c7575bc060ce15cbac INFO: 172.21.0.1:54734 - "POST /node/694081cb575bc060ce15cbb5 HTTP/1.0" 202 Accepted 2025-12-17 21:47:06,809 - lava_callback - INFO - Starting async job submit for node 694081cb575bc060ce15cbb5 2025-12-17 21:47:06,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:06,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c7575bc060ce15cbac HTTP/1.1" 502 150 2025-12-17 21:47:06,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c7575bc060ce15cbac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:06,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081cb575bc060ce15cbb5 HTTP/1.1" 502 150 2025-12-17 21:47:06,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081cb575bc060ce15cbb5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:06,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081cb575bc060ce15cbb5 2025-12-17 21:47:06,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081cb575bc060ce15cbb5 HTTP/1.1" 502 150 2025-12-17 21:47:06,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081cb575bc060ce15cbb5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:07,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c8575bc060ce15cbaf 2025-12-17 21:47:07,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c8575bc060ce15cbaf HTTP/1.1" 502 150 2025-12-17 21:47:07,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c8575bc060ce15cbaf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:07,537 - lava_callback - INFO - Received callback for node 694081cc575bc060ce15cbb8 INFO: 172.21.0.1:54744 - "POST /node/694081cc575bc060ce15cbb8 HTTP/1.0" 202 Accepted 2025-12-17 21:47:07,543 - lava_callback - INFO - Starting async job submit for node 694081cc575bc060ce15cbb8 2025-12-17 21:47:07,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:07,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081cc575bc060ce15cbb8 HTTP/1.1" 502 150 2025-12-17 21:47:07,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081cc575bc060ce15cbb8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:07,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081cc575bc060ce15cbb8 2025-12-17 21:47:07,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081cc575bc060ce15cbb8 HTTP/1.1" 502 150 2025-12-17 21:47:07,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081cc575bc060ce15cbb8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:08,177 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c9575bc060ce15cbb2 2025-12-17 21:47:08,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c3575bc060ce15cb99 2025-12-17 21:47:08,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c3575bc060ce15cb99 HTTP/1.1" 502 150 2025-12-17 21:47:08,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c3575bc060ce15cb99'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:08,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081cb575bc060ce15cbb5 2025-12-17 21:47:08,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081cb575bc060ce15cbb5 HTTP/1.1" 502 150 2025-12-17 21:47:08,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081cb575bc060ce15cbb5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:09,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c4575bc060ce15cba0 2025-12-17 21:47:09,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c4575bc060ce15cba0 HTTP/1.1" 502 150 2025-12-17 21:47:09,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c4575bc060ce15cba0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:09,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081cc575bc060ce15cbb8 2025-12-17 21:47:09,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081cc575bc060ce15cbb8 HTTP/1.1" 502 150 2025-12-17 21:47:09,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081cc575bc060ce15cbb8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:10,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f65575bc060ce15c452 2025-12-17 21:47:10,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f65575bc060ce15c452 HTTP/1.1" 502 150 2025-12-17 21:47:10,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407f65575bc060ce15c452'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:10,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c7575bc060ce15cbac 2025-12-17 21:47:10,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c7575bc060ce15cbac HTTP/1.1" 502 150 2025-12-17 21:47:10,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c7575bc060ce15cbac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:11,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c8575bc060ce15cbaf 2025-12-17 21:47:11,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c8575bc060ce15cbaf HTTP/1.1" 502 150 2025-12-17 21:47:11,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c8575bc060ce15cbaf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:11,802 - lava_callback - INFO - Received callback for node 694081de575bc060ce15cc18 INFO: 172.21.0.1:53898 - "POST /node/694081de575bc060ce15cc18 HTTP/1.0" 202 Accepted 2025-12-17 21:47:11,807 - lava_callback - INFO - Starting async job submit for node 694081de575bc060ce15cc18 2025-12-17 21:47:11,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:11,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081de575bc060ce15cc18 HTTP/1.1" 502 150 2025-12-17 21:47:11,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081de575bc060ce15cc18'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:11,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081de575bc060ce15cc18 2025-12-17 21:47:11,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081de575bc060ce15cc18 HTTP/1.1" 502 150 2025-12-17 21:47:11,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081de575bc060ce15cc18'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:12,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c1575bc060ce15cb90 2025-12-17 21:47:12,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c1575bc060ce15cb90 HTTP/1.1" 502 150 2025-12-17 21:47:12,792 - lava_callback - ERROR - Error processing callback for node 694081c1575bc060ce15cb90: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c1575bc060ce15cb90 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c1575bc060ce15cb90 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c1575bc060ce15cb90 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:12,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081cb575bc060ce15cbb5 2025-12-17 21:47:12,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081cb575bc060ce15cbb5 HTTP/1.1" 502 150 2025-12-17 21:47:12,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081cb575bc060ce15cbb5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:13,157 - lava_callback - INFO - Received callback for node 694081d6575bc060ce15cbf5 INFO: 172.21.0.1:53902 - "POST /node/694081d6575bc060ce15cbf5 HTTP/1.0" 202 Accepted 2025-12-17 21:47:13,163 - lava_callback - INFO - Starting async job submit for node 694081d6575bc060ce15cbf5 2025-12-17 21:47:13,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:13,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081d6575bc060ce15cbf5 HTTP/1.1" 502 150 2025-12-17 21:47:13,188 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081d6575bc060ce15cbf5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:13,188 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081d6575bc060ce15cbf5 2025-12-17 21:47:13,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081d6575bc060ce15cbf5 HTTP/1.1" 502 150 2025-12-17 21:47:13,189 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081d6575bc060ce15cbf5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:13,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081cc575bc060ce15cbb8 2025-12-17 21:47:13,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081cc575bc060ce15cbb8 HTTP/1.1" 502 150 2025-12-17 21:47:13,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081cc575bc060ce15cbb8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:13,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081de575bc060ce15cc18 2025-12-17 21:47:13,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081de575bc060ce15cc18 HTTP/1.1" 502 150 2025-12-17 21:47:13,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081de575bc060ce15cc18'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:13,839 - lava_callback - INFO - Received callback for node 694081d9575bc060ce15cbfe INFO: 172.21.0.1:53904 - "POST /node/694081d9575bc060ce15cbfe HTTP/1.0" 202 Accepted 2025-12-17 21:47:13,844 - lava_callback - INFO - Starting async job submit for node 694081d9575bc060ce15cbfe 2025-12-17 21:47:13,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:13,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081d9575bc060ce15cbfe HTTP/1.1" 502 150 2025-12-17 21:47:13,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081d9575bc060ce15cbfe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:13,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081d9575bc060ce15cbfe 2025-12-17 21:47:13,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081d9575bc060ce15cbfe HTTP/1.1" 502 150 2025-12-17 21:47:13,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081d9575bc060ce15cbfe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:14,538 - lava_callback - INFO - Received callback for node 694081da575bc060ce15cc05 INFO: 172.21.0.1:53920 - "POST /node/694081da575bc060ce15cc05 HTTP/1.0" 202 Accepted 2025-12-17 21:47:14,545 - lava_callback - INFO - Starting async job submit for node 694081da575bc060ce15cc05 2025-12-17 21:47:14,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:14,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081da575bc060ce15cc05 HTTP/1.1" 502 150 2025-12-17 21:47:14,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081da575bc060ce15cc05'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:14,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081da575bc060ce15cc05 2025-12-17 21:47:14,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081da575bc060ce15cc05 HTTP/1.1" 502 150 2025-12-17 21:47:14,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081da575bc060ce15cc05'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53930 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:47:15,189 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081d6575bc060ce15cbf5 2025-12-17 21:47:15,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081d6575bc060ce15cbf5 HTTP/1.1" 502 150 2025-12-17 21:47:15,190 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081d6575bc060ce15cbf5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:15,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081d9575bc060ce15cbfe 2025-12-17 21:47:15,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081d9575bc060ce15cbfe HTTP/1.1" 502 150 2025-12-17 21:47:15,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081d9575bc060ce15cbfe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53942 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:47:16,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081da575bc060ce15cc05 2025-12-17 21:47:16,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081da575bc060ce15cc05 HTTP/1.1" 502 150 2025-12-17 21:47:16,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081da575bc060ce15cc05'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:16,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c3575bc060ce15cb99 2025-12-17 21:47:16,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c3575bc060ce15cb99 HTTP/1.1" 502 150 2025-12-17 21:47:16,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c3575bc060ce15cb99'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:17,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c4575bc060ce15cba0 2025-12-17 21:47:17,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c4575bc060ce15cba0 HTTP/1.1" 502 150 2025-12-17 21:47:17,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c4575bc060ce15cba0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:17,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081a5575bc060ce15cb74 2025-12-17 21:47:17,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081a5575bc060ce15cb74 HTTP/1.1" 502 150 2025-12-17 21:47:17,751 - lava_callback - ERROR - Error processing callback for node 694081a5575bc060ce15cb74: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081a5575bc060ce15cb74 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081a5575bc060ce15cb74 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081a5575bc060ce15cb74 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:17,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081de575bc060ce15cc18 2025-12-17 21:47:17,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081de575bc060ce15cc18 HTTP/1.1" 502 150 2025-12-17 21:47:17,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081de575bc060ce15cc18'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:18,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c7575bc060ce15cbac 2025-12-17 21:47:18,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c7575bc060ce15cbac HTTP/1.1" 502 150 2025-12-17 21:47:18,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c7575bc060ce15cbac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:19,191 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081d6575bc060ce15cbf5 2025-12-17 21:47:19,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c8575bc060ce15cbaf 2025-12-17 21:47:19,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c8575bc060ce15cbaf HTTP/1.1" 502 150 2025-12-17 21:47:19,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c8575bc060ce15cbaf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:19,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c6575bc060ce15cba7 HTTP/1.1" 502 150 2025-12-17 21:47:19,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c6575bc060ce15cba7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:19,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c6575bc060ce15cba7 2025-12-17 21:47:19,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c6575bc060ce15cba7 HTTP/1.1" 502 150 2025-12-17 21:47:19,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c6575bc060ce15cba7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:19,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081d9575bc060ce15cbfe 2025-12-17 21:47:19,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081d9575bc060ce15cbfe HTTP/1.1" 502 150 2025-12-17 21:47:19,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081d9575bc060ce15cbfe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:20,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081da575bc060ce15cc05 2025-12-17 21:47:20,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081da575bc060ce15cc05 HTTP/1.1" 502 150 2025-12-17 21:47:20,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081da575bc060ce15cc05'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:20,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081cb575bc060ce15cbb5 2025-12-17 21:47:20,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081cb575bc060ce15cbb5 HTTP/1.1" 502 150 2025-12-17 21:47:20,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081cb575bc060ce15cbb5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:21,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081cc575bc060ce15cbb8 2025-12-17 21:47:21,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081cc575bc060ce15cbb8 HTTP/1.1" 502 150 2025-12-17 21:47:21,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081cc575bc060ce15cbb8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:21,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c6575bc060ce15cba7 2025-12-17 21:47:21,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c6575bc060ce15cba7 HTTP/1.1" 502 150 2025-12-17 21:47:21,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c6575bc060ce15cba7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:22,813 - lava_callback - INFO - Received callback for node 694081e0575bc060ce15cc1e INFO: 172.21.0.1:52286 - "POST /node/694081e0575bc060ce15cc1e HTTP/1.0" 202 Accepted 2025-12-17 21:47:22,819 - lava_callback - INFO - Starting async job submit for node 694081e0575bc060ce15cc1e 2025-12-17 21:47:22,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:22,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e0575bc060ce15cc1e HTTP/1.1" 502 150 2025-12-17 21:47:22,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e0575bc060ce15cc1e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:22,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e0575bc060ce15cc1e 2025-12-17 21:47:22,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e0575bc060ce15cc1e HTTP/1.1" 502 150 2025-12-17 21:47:22,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e0575bc060ce15cc1e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:23,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c9575bc060ce15cbb2 HTTP/1.1" 502 150 2025-12-17 21:47:23,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c9575bc060ce15cbb2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:23,472 - lava_callback - INFO - Received callback for node 694081e2575bc060ce15cc21 INFO: 172.21.0.1:52288 - "POST /node/694081e2575bc060ce15cc21 HTTP/1.0" 202 Accepted 2025-12-17 21:47:23,478 - lava_callback - INFO - Starting async job submit for node 694081e2575bc060ce15cc21 2025-12-17 21:47:23,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:23,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e2575bc060ce15cc21 HTTP/1.1" 502 150 2025-12-17 21:47:23,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e2575bc060ce15cc21'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:23,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e2575bc060ce15cc21 2025-12-17 21:47:23,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e2575bc060ce15cc21 HTTP/1.1" 502 150 2025-12-17 21:47:23,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e2575bc060ce15cc21'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:24,163 - lava_callback - INFO - Received callback for node 694081e3575bc060ce15cc24 INFO: 172.21.0.1:52292 - "POST /node/694081e3575bc060ce15cc24 HTTP/1.0" 202 Accepted 2025-12-17 21:47:24,169 - lava_callback - INFO - Starting async job submit for node 694081e3575bc060ce15cc24 2025-12-17 21:47:24,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:24,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e3575bc060ce15cc24 HTTP/1.1" 502 150 2025-12-17 21:47:24,176 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e3575bc060ce15cc24'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:24,176 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e3575bc060ce15cc24 2025-12-17 21:47:24,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e3575bc060ce15cc24 HTTP/1.1" 502 150 2025-12-17 21:47:24,177 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e3575bc060ce15cc24'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:24,830 - lava_callback - INFO - Received callback for node 694081e4575bc060ce15cc2b INFO: 172.21.0.1:52296 - "POST /node/694081e4575bc060ce15cc2b HTTP/1.0" 202 Accepted 2025-12-17 21:47:24,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e0575bc060ce15cc1e 2025-12-17 21:47:24,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e0575bc060ce15cc1e HTTP/1.1" 502 150 2025-12-17 21:47:24,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e0575bc060ce15cc1e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:25,474 - lava_callback - INFO - Received callback for node 694081e6575bc060ce15cc2e INFO: 172.21.0.1:52310 - "POST /node/694081e6575bc060ce15cc2e HTTP/1.0" 202 Accepted 2025-12-17 21:47:25,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e2575bc060ce15cc21 2025-12-17 21:47:25,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e2575bc060ce15cc21 HTTP/1.1" 502 150 2025-12-17 21:47:25,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e2575bc060ce15cc21'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:25,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c6575bc060ce15cba7 2025-12-17 21:47:25,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c6575bc060ce15cba7 HTTP/1.1" 502 150 2025-12-17 21:47:25,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c6575bc060ce15cba7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:25,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081de575bc060ce15cc18 2025-12-17 21:47:25,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081de575bc060ce15cc18 HTTP/1.1" 502 150 2025-12-17 21:47:25,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081de575bc060ce15cc18'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:26,139 - lava_callback - INFO - Received callback for node 694081e7575bc060ce15cc31 INFO: 172.21.0.1:52316 - "POST /node/694081e7575bc060ce15cc31 HTTP/1.0" 202 Accepted 2025-12-17 21:47:26,177 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e3575bc060ce15cc24 2025-12-17 21:47:26,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e3575bc060ce15cc24 HTTP/1.1" 502 150 2025-12-17 21:47:26,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e3575bc060ce15cc24'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52328 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:47:26,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407f65575bc060ce15c452 2025-12-17 21:47:26,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407f65575bc060ce15c452 HTTP/1.1" 502 150 2025-12-17 21:47:26,593 - lava_callback - ERROR - Error processing callback for node 69407f65575bc060ce15c452: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f65575bc060ce15c452 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f65575bc060ce15c452 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407f65575bc060ce15c452 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:26,595 - lava_callback - INFO - Starting async job submit for node 694081e4575bc060ce15cc2b 2025-12-17 21:47:26,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:26,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e4575bc060ce15cc2b HTTP/1.1" 502 150 2025-12-17 21:47:26,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e4575bc060ce15cc2b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:26,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e4575bc060ce15cc2b 2025-12-17 21:47:26,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e4575bc060ce15cc2b HTTP/1.1" 502 150 2025-12-17 21:47:26,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e4575bc060ce15cc2b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:26,819 - lava_callback - INFO - Received callback for node 694081e8575bc060ce15cc34 INFO: 172.21.0.1:52340 - "POST /node/694081e8575bc060ce15cc34 HTTP/1.0" 202 Accepted 2025-12-17 21:47:27,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c9575bc060ce15cbb2 2025-12-17 21:47:27,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c9575bc060ce15cbb2 HTTP/1.1" 502 150 2025-12-17 21:47:27,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c9575bc060ce15cbb2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:27,499 - lava_callback - INFO - Received callback for node 694081e9575bc060ce15cc39 INFO: 172.21.0.1:52356 - "POST /node/694081e9575bc060ce15cc39 HTTP/1.0" 202 Accepted 2025-12-17 21:47:27,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081d9575bc060ce15cbfe 2025-12-17 21:47:27,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081d9575bc060ce15cbfe HTTP/1.1" 502 150 2025-12-17 21:47:27,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081d9575bc060ce15cbfe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:28,039 - lava_callback - INFO - Received callback for node 694081db575bc060ce15cc0a INFO: 172.21.0.1:52366 - "POST /node/694081db575bc060ce15cc0a HTTP/1.0" 202 Accepted 2025-12-17 21:47:28,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081da575bc060ce15cc05 2025-12-17 21:47:28,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081da575bc060ce15cc05 HTTP/1.1" 502 150 2025-12-17 21:47:28,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081da575bc060ce15cc05'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:28,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e4575bc060ce15cc2b 2025-12-17 21:47:28,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e4575bc060ce15cc2b HTTP/1.1" 502 150 2025-12-17 21:47:28,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e4575bc060ce15cc2b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:28,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e0575bc060ce15cc1e 2025-12-17 21:47:28,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e0575bc060ce15cc1e HTTP/1.1" 502 150 2025-12-17 21:47:28,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e0575bc060ce15cc1e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:29,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e2575bc060ce15cc21 2025-12-17 21:47:29,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e2575bc060ce15cc21 HTTP/1.1" 502 150 2025-12-17 21:47:29,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e2575bc060ce15cc21'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:30,178 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e3575bc060ce15cc24 2025-12-17 21:47:31,753 - lava_callback - INFO - Received callback for node 694081df575bc060ce15cc1b INFO: 172.21.0.1:56676 - "POST /node/694081df575bc060ce15cc1b HTTP/1.0" 202 Accepted 2025-12-17 21:47:32,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e4575bc060ce15cc2b 2025-12-17 21:47:32,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e4575bc060ce15cc2b HTTP/1.1" 502 150 2025-12-17 21:47:32,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e4575bc060ce15cc2b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:32,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c3575bc060ce15cb99 2025-12-17 21:47:32,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c3575bc060ce15cb99 HTTP/1.1" 502 150 2025-12-17 21:47:32,801 - lava_callback - ERROR - Error processing callback for node 694081c3575bc060ce15cb99: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c3575bc060ce15cb99 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c3575bc060ce15cb99 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c3575bc060ce15cb99 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:32,803 - lava_callback - INFO - Starting async job submit for node 694081e6575bc060ce15cc2e 2025-12-17 21:47:32,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:32,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e6575bc060ce15cc2e HTTP/1.1" 502 150 2025-12-17 21:47:32,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e6575bc060ce15cc2e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:32,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e6575bc060ce15cc2e 2025-12-17 21:47:32,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e6575bc060ce15cc2e HTTP/1.1" 502 150 2025-12-17 21:47:32,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e6575bc060ce15cc2e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:33,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c4575bc060ce15cba0 2025-12-17 21:47:33,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c4575bc060ce15cba0 HTTP/1.1" 502 150 2025-12-17 21:47:33,475 - lava_callback - ERROR - Error processing callback for node 694081c4575bc060ce15cba0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c4575bc060ce15cba0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c4575bc060ce15cba0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c4575bc060ce15cba0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:33,478 - lava_callback - INFO - Starting async job submit for node 694081e7575bc060ce15cc31 2025-12-17 21:47:33,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:33,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e7575bc060ce15cc31 HTTP/1.1" 502 150 2025-12-17 21:47:33,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e7575bc060ce15cc31'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:33,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e7575bc060ce15cc31 2025-12-17 21:47:33,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e7575bc060ce15cc31 HTTP/1.1" 502 150 2025-12-17 21:47:33,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e7575bc060ce15cc31'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:33,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c6575bc060ce15cba7 2025-12-17 21:47:33,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c6575bc060ce15cba7 HTTP/1.1" 502 150 2025-12-17 21:47:33,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c6575bc060ce15cba7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:34,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081d6575bc060ce15cbf5 HTTP/1.1" 502 150 2025-12-17 21:47:34,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081d6575bc060ce15cbf5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:34,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c7575bc060ce15cbac 2025-12-17 21:47:34,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c7575bc060ce15cbac HTTP/1.1" 502 150 2025-12-17 21:47:34,815 - lava_callback - ERROR - Error processing callback for node 694081c7575bc060ce15cbac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c7575bc060ce15cbac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c7575bc060ce15cbac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c7575bc060ce15cbac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:34,816 - lava_callback - INFO - Starting async job submit for node 694081e8575bc060ce15cc34 2025-12-17 21:47:34,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:34,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e8575bc060ce15cc34 HTTP/1.1" 502 150 2025-12-17 21:47:34,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e8575bc060ce15cc34'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:34,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e8575bc060ce15cc34 2025-12-17 21:47:34,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e6575bc060ce15cc2e 2025-12-17 21:47:34,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e6575bc060ce15cc2e HTTP/1.1" 502 150 2025-12-17 21:47:34,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e6575bc060ce15cc2e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:34,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e8575bc060ce15cc34 HTTP/1.1" 502 150 2025-12-17 21:47:34,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e8575bc060ce15cc34'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:35,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c9575bc060ce15cbb2 2025-12-17 21:47:35,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c9575bc060ce15cbb2 HTTP/1.1" 502 150 2025-12-17 21:47:35,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081c9575bc060ce15cbb2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:35,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c8575bc060ce15cbaf 2025-12-17 21:47:35,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c8575bc060ce15cbaf HTTP/1.1" 502 150 2025-12-17 21:47:35,482 - lava_callback - ERROR - Error processing callback for node 694081c8575bc060ce15cbaf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c8575bc060ce15cbaf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c8575bc060ce15cbaf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c8575bc060ce15cbaf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:35,484 - lava_callback - INFO - Starting async job submit for node 694081e9575bc060ce15cc39 2025-12-17 21:47:35,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:35,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e9575bc060ce15cc39 HTTP/1.1" 502 150 2025-12-17 21:47:35,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e9575bc060ce15cc39'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:35,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e9575bc060ce15cc39 2025-12-17 21:47:35,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e9575bc060ce15cc39 HTTP/1.1" 502 150 2025-12-17 21:47:35,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e9575bc060ce15cc39'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:35,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e7575bc060ce15cc31 2025-12-17 21:47:35,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e7575bc060ce15cc31 HTTP/1.1" 502 150 2025-12-17 21:47:35,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e7575bc060ce15cc31'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56680 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:47:36,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e8575bc060ce15cc34 2025-12-17 21:47:36,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e8575bc060ce15cc34 HTTP/1.1" 502 150 2025-12-17 21:47:36,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e8575bc060ce15cc34'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:36,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081cb575bc060ce15cbb5 2025-12-17 21:47:36,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081cb575bc060ce15cbb5 HTTP/1.1" 502 150 2025-12-17 21:47:36,832 - lava_callback - ERROR - Error processing callback for node 694081cb575bc060ce15cbb5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081cb575bc060ce15cbb5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081cb575bc060ce15cbb5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081cb575bc060ce15cbb5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:36,834 - lava_callback - INFO - Starting async job submit for node 694081db575bc060ce15cc0a 2025-12-17 21:47:36,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:36,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081db575bc060ce15cc0a HTTP/1.1" 502 150 2025-12-17 21:47:36,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081db575bc060ce15cc0a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:36,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081db575bc060ce15cc0a 2025-12-17 21:47:36,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081db575bc060ce15cc0a HTTP/1.1" 502 150 2025-12-17 21:47:36,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081db575bc060ce15cc0a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:36,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e0575bc060ce15cc1e 2025-12-17 21:47:36,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e0575bc060ce15cc1e HTTP/1.1" 502 150 2025-12-17 21:47:36,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e0575bc060ce15cc1e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:37,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e2575bc060ce15cc21 2025-12-17 21:47:37,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e2575bc060ce15cc21 HTTP/1.1" 502 150 2025-12-17 21:47:37,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e2575bc060ce15cc21'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:37,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e9575bc060ce15cc39 2025-12-17 21:47:37,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e9575bc060ce15cc39 HTTP/1.1" 502 150 2025-12-17 21:47:37,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e9575bc060ce15cc39'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:37,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081cc575bc060ce15cbb8 2025-12-17 21:47:37,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081cc575bc060ce15cbb8 HTTP/1.1" 502 150 2025-12-17 21:47:37,579 - lava_callback - ERROR - Error processing callback for node 694081cc575bc060ce15cbb8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081cc575bc060ce15cbb8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081cc575bc060ce15cbb8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081cc575bc060ce15cbb8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:37,581 - lava_callback - INFO - Starting async job submit for node 694081df575bc060ce15cc1b 2025-12-17 21:47:37,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:37,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081df575bc060ce15cc1b HTTP/1.1" 502 150 2025-12-17 21:47:37,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081df575bc060ce15cc1b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:37,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081df575bc060ce15cc1b 2025-12-17 21:47:37,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081df575bc060ce15cc1b HTTP/1.1" 502 150 2025-12-17 21:47:37,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081df575bc060ce15cc1b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:38,193 - lava_callback - INFO - Received callback for node 694081f1575bc060ce15cc65 INFO: 172.21.0.1:56696 - "POST /node/694081f1575bc060ce15cc65 HTTP/1.0" 202 Accepted 2025-12-17 21:47:38,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e6575bc060ce15cc2e 2025-12-17 21:47:38,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e6575bc060ce15cc2e HTTP/1.1" 502 150 2025-12-17 21:47:38,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e6575bc060ce15cc2e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:38,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081db575bc060ce15cc0a 2025-12-17 21:47:38,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081db575bc060ce15cc0a HTTP/1.1" 502 150 2025-12-17 21:47:38,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081db575bc060ce15cc0a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:39,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e7575bc060ce15cc31 2025-12-17 21:47:39,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e7575bc060ce15cc31 HTTP/1.1" 502 150 2025-12-17 21:47:39,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e7575bc060ce15cc31'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:39,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081df575bc060ce15cc1b 2025-12-17 21:47:39,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081df575bc060ce15cc1b HTTP/1.1" 502 150 2025-12-17 21:47:39,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081df575bc060ce15cc1b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:40,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e4575bc060ce15cc2b 2025-12-17 21:47:40,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e4575bc060ce15cc2b HTTP/1.1" 502 150 2025-12-17 21:47:40,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e4575bc060ce15cc2b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:40,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e8575bc060ce15cc34 2025-12-17 21:47:40,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e8575bc060ce15cc34 HTTP/1.1" 502 150 2025-12-17 21:47:40,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e8575bc060ce15cc34'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:41,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e9575bc060ce15cc39 2025-12-17 21:47:41,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e9575bc060ce15cc39 HTTP/1.1" 502 150 2025-12-17 21:47:41,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e9575bc060ce15cc39'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:41,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081de575bc060ce15cc18 2025-12-17 21:47:41,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081de575bc060ce15cc18 HTTP/1.1" 502 150 2025-12-17 21:47:41,820 - lava_callback - ERROR - Error processing callback for node 694081de575bc060ce15cc18: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081de575bc060ce15cc18 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081de575bc060ce15cc18 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081de575bc060ce15cc18 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:41,822 - lava_callback - INFO - Starting async job submit for node 694081f1575bc060ce15cc65 2025-12-17 21:47:41,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:41,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f1575bc060ce15cc65 HTTP/1.1" 502 150 2025-12-17 21:47:41,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f1575bc060ce15cc65'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:41,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f1575bc060ce15cc65 2025-12-17 21:47:41,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f1575bc060ce15cc65 HTTP/1.1" 502 150 2025-12-17 21:47:41,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f1575bc060ce15cc65'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:42,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081d6575bc060ce15cbf5 2025-12-17 21:47:42,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081d6575bc060ce15cbf5 HTTP/1.1" 502 150 2025-12-17 21:47:42,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081d6575bc060ce15cbf5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:42,806 - lava_callback - INFO - Received callback for node 694081ea575bc060ce15cc42 INFO: 172.21.0.1:56182 - "POST /node/694081ea575bc060ce15cc42 HTTP/1.0" 202 Accepted 2025-12-17 21:47:42,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081db575bc060ce15cc0a 2025-12-17 21:47:42,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081db575bc060ce15cc0a HTTP/1.1" 502 150 2025-12-17 21:47:42,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081db575bc060ce15cc0a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:43,478 - lava_callback - INFO - Received callback for node 694081ec575bc060ce15cc49 INFO: 172.21.0.1:56184 - "POST /node/694081ec575bc060ce15cc49 HTTP/1.0" 202 Accepted 2025-12-17 21:47:43,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081df575bc060ce15cc1b 2025-12-17 21:47:43,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081df575bc060ce15cc1b HTTP/1.1" 502 150 2025-12-17 21:47:43,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081df575bc060ce15cc1b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:43,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f1575bc060ce15cc65 2025-12-17 21:47:43,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f1575bc060ce15cc65 HTTP/1.1" 502 150 2025-12-17 21:47:43,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f1575bc060ce15cc65'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:43,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081d9575bc060ce15cbfe 2025-12-17 21:47:43,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081d9575bc060ce15cbfe HTTP/1.1" 502 150 2025-12-17 21:47:43,858 - lava_callback - ERROR - Error processing callback for node 694081d9575bc060ce15cbfe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081d9575bc060ce15cbfe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081d9575bc060ce15cbfe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081d9575bc060ce15cbfe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:43,860 - lava_callback - INFO - Starting async job submit for node 694081ea575bc060ce15cc42 2025-12-17 21:47:43,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:43,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ea575bc060ce15cc42 HTTP/1.1" 502 150 2025-12-17 21:47:43,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ea575bc060ce15cc42'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:43,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ea575bc060ce15cc42 2025-12-17 21:47:43,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ea575bc060ce15cc42 HTTP/1.1" 502 150 2025-12-17 21:47:43,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ea575bc060ce15cc42'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:44,147 - lava_callback - INFO - Received callback for node 694081ed575bc060ce15cc50 INFO: 172.21.0.1:56196 - "POST /node/694081ed575bc060ce15cc50 HTTP/1.0" 202 Accepted 2025-12-17 21:47:44,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081da575bc060ce15cc05 2025-12-17 21:47:44,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081da575bc060ce15cc05 HTTP/1.1" 502 150 2025-12-17 21:47:44,561 - lava_callback - ERROR - Error processing callback for node 694081da575bc060ce15cc05: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081da575bc060ce15cc05 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081da575bc060ce15cc05 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081da575bc060ce15cc05 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:44,563 - lava_callback - INFO - Starting async job submit for node 694081ec575bc060ce15cc49 2025-12-17 21:47:44,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:44,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ec575bc060ce15cc49 HTTP/1.1" 502 150 2025-12-17 21:47:44,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ec575bc060ce15cc49'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:44,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ec575bc060ce15cc49 2025-12-17 21:47:44,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ec575bc060ce15cc49 HTTP/1.1" 502 150 2025-12-17 21:47:44,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ec575bc060ce15cc49'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:44,829 - lava_callback - INFO - Received callback for node 694081ee575bc060ce15cc57 INFO: 172.21.0.1:56210 - "POST /node/694081ee575bc060ce15cc57 HTTP/1.0" 202 Accepted 2025-12-17 21:47:45,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e3575bc060ce15cc24 HTTP/1.1" 502 150 2025-12-17 21:47:45,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e3575bc060ce15cc24'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:45,494 - lava_callback - INFO - Received callback for node 694081f0575bc060ce15cc5e INFO: 172.21.0.1:56220 - "POST /node/694081f0575bc060ce15cc5e HTTP/1.0" 202 Accepted 2025-12-17 21:47:45,777 - lava_callback - INFO - Received callback for node 69407e3e575bc060ce15c2a1 INFO: 172.21.0.1:56236 - "POST /node/69407e3e575bc060ce15c2a1 HTTP/1.0" 202 Accepted 2025-12-17 21:47:45,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ea575bc060ce15cc42 2025-12-17 21:47:45,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ea575bc060ce15cc42 HTTP/1.1" 502 150 2025-12-17 21:47:45,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ea575bc060ce15cc42'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:46,156 - lava_callback - INFO - Received callback for node 694081f2575bc060ce15cc68 INFO: 172.21.0.1:56250 - "POST /node/694081f2575bc060ce15cc68 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:56252 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:47:46,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ec575bc060ce15cc49 2025-12-17 21:47:46,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ec575bc060ce15cc49 HTTP/1.1" 502 150 2025-12-17 21:47:46,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ec575bc060ce15cc49'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:46,810 - lava_callback - INFO - Received callback for node 694081f3575bc060ce15cc6b INFO: 172.21.0.1:56258 - "POST /node/694081f3575bc060ce15cc6b HTTP/1.0" 202 Accepted 2025-12-17 21:47:46,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e6575bc060ce15cc2e 2025-12-17 21:47:46,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e6575bc060ce15cc2e HTTP/1.1" 502 150 2025-12-17 21:47:46,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e6575bc060ce15cc2e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:47,490 - lava_callback - INFO - Received callback for node 694081f5575bc060ce15cc6e INFO: 172.21.0.1:56274 - "POST /node/694081f5575bc060ce15cc6e HTTP/1.0" 202 Accepted 2025-12-17 21:47:47,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e7575bc060ce15cc31 2025-12-17 21:47:47,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e7575bc060ce15cc31 HTTP/1.1" 502 150 2025-12-17 21:47:47,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e7575bc060ce15cc31'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:47,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f1575bc060ce15cc65 2025-12-17 21:47:47,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f1575bc060ce15cc65 HTTP/1.1" 502 150 2025-12-17 21:47:47,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f1575bc060ce15cc65'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:48,148 - lava_callback - INFO - Received callback for node 694081f6575bc060ce15cc71 INFO: 172.21.0.1:56284 - "POST /node/694081f6575bc060ce15cc71 HTTP/1.0" 202 Accepted 2025-12-17 21:47:48,802 - lava_callback - INFO - Received callback for node 694081f7575bc060ce15cc74 INFO: 172.21.0.1:56292 - "POST /node/694081f7575bc060ce15cc74 HTTP/1.0" 202 Accepted 2025-12-17 21:47:48,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e8575bc060ce15cc34 2025-12-17 21:47:48,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e8575bc060ce15cc34 HTTP/1.1" 502 150 2025-12-17 21:47:48,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e8575bc060ce15cc34'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:49,462 - lava_callback - INFO - Received callback for node 694081f8575bc060ce15cc77 INFO: 172.21.0.1:56300 - "POST /node/694081f8575bc060ce15cc77 HTTP/1.0" 202 Accepted 2025-12-17 21:47:49,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e9575bc060ce15cc39 2025-12-17 21:47:49,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e9575bc060ce15cc39 HTTP/1.1" 502 150 2025-12-17 21:47:49,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e9575bc060ce15cc39'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:49,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c6575bc060ce15cba7 2025-12-17 21:47:49,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c6575bc060ce15cba7 HTTP/1.1" 502 150 2025-12-17 21:47:49,612 - lava_callback - ERROR - Error processing callback for node 694081c6575bc060ce15cba7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c6575bc060ce15cba7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c6575bc060ce15cba7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c6575bc060ce15cba7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:49,614 - lava_callback - INFO - Starting async job submit for node 694081ed575bc060ce15cc50 2025-12-17 21:47:49,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:49,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ed575bc060ce15cc50 HTTP/1.1" 502 150 2025-12-17 21:47:49,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ed575bc060ce15cc50'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:49,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ed575bc060ce15cc50 2025-12-17 21:47:49,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ed575bc060ce15cc50 HTTP/1.1" 502 150 2025-12-17 21:47:49,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ed575bc060ce15cc50'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:49,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ea575bc060ce15cc42 2025-12-17 21:47:49,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ea575bc060ce15cc42 HTTP/1.1" 502 150 2025-12-17 21:47:49,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ea575bc060ce15cc42'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:50,126 - lava_callback - INFO - Received callback for node 694081fa575bc060ce15cc7a INFO: 172.21.0.1:43370 - "POST /node/694081fa575bc060ce15cc7a HTTP/1.0" 202 Accepted 2025-12-17 21:47:50,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ec575bc060ce15cc49 2025-12-17 21:47:50,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ec575bc060ce15cc49 HTTP/1.1" 502 150 2025-12-17 21:47:50,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ec575bc060ce15cc49'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:50,797 - lava_callback - INFO - Received callback for node 694081fb575bc060ce15cc7d INFO: 172.21.0.1:43376 - "POST /node/694081fb575bc060ce15cc7d HTTP/1.0" 202 Accepted 2025-12-17 21:47:50,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081db575bc060ce15cc0a 2025-12-17 21:47:50,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081db575bc060ce15cc0a HTTP/1.1" 502 150 2025-12-17 21:47:50,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081db575bc060ce15cc0a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:51,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081c9575bc060ce15cbb2 2025-12-17 21:47:51,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081c9575bc060ce15cbb2 HTTP/1.1" 502 150 2025-12-17 21:47:51,446 - lava_callback - ERROR - Error processing callback for node 694081c9575bc060ce15cbb2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c9575bc060ce15cbb2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c9575bc060ce15cbb2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081c9575bc060ce15cbb2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:51,448 - lava_callback - INFO - Starting async job submit for node 694081ee575bc060ce15cc57 2025-12-17 21:47:51,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:51,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ee575bc060ce15cc57 HTTP/1.1" 502 150 2025-12-17 21:47:51,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ee575bc060ce15cc57'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:51,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ee575bc060ce15cc57 2025-12-17 21:47:51,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ee575bc060ce15cc57 HTTP/1.1" 502 150 2025-12-17 21:47:51,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ee575bc060ce15cc57'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:51,469 - lava_callback - INFO - Received callback for node 694081fc575bc060ce15cc80 INFO: 172.21.0.1:43392 - "POST /node/694081fc575bc060ce15cc80 HTTP/1.0" 202 Accepted 2025-12-17 21:47:51,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081df575bc060ce15cc1b 2025-12-17 21:47:51,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081df575bc060ce15cc1b HTTP/1.1" 502 150 2025-12-17 21:47:51,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081df575bc060ce15cc1b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:51,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ed575bc060ce15cc50 2025-12-17 21:47:51,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ed575bc060ce15cc50 HTTP/1.1" 502 150 2025-12-17 21:47:51,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ed575bc060ce15cc50'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:52,124 - lava_callback - INFO - Received callback for node 694081fd575bc060ce15cc85 INFO: 172.21.0.1:43400 - "POST /node/694081fd575bc060ce15cc85 HTTP/1.0" 202 Accepted 2025-12-17 21:47:52,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e0575bc060ce15cc1e 2025-12-17 21:47:52,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e0575bc060ce15cc1e HTTP/1.1" 502 150 2025-12-17 21:47:52,865 - lava_callback - ERROR - Error processing callback for node 694081e0575bc060ce15cc1e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e0575bc060ce15cc1e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e0575bc060ce15cc1e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e0575bc060ce15cc1e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:52,868 - lava_callback - INFO - Starting async job submit for node 694081f0575bc060ce15cc5e 2025-12-17 21:47:52,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:52,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f0575bc060ce15cc5e HTTP/1.1" 502 150 2025-12-17 21:47:52,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f0575bc060ce15cc5e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:52,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f0575bc060ce15cc5e 2025-12-17 21:47:52,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f0575bc060ce15cc5e HTTP/1.1" 502 150 2025-12-17 21:47:52,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f0575bc060ce15cc5e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:53,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ee575bc060ce15cc57 2025-12-17 21:47:53,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e3575bc060ce15cc24 2025-12-17 21:47:53,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ee575bc060ce15cc57 HTTP/1.1" 502 150 2025-12-17 21:47:53,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ee575bc060ce15cc57'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:53,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e3575bc060ce15cc24 HTTP/1.1" 502 150 2025-12-17 21:47:53,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081e3575bc060ce15cc24'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:53,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e2575bc060ce15cc21 2025-12-17 21:47:53,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e2575bc060ce15cc21 HTTP/1.1" 502 150 2025-12-17 21:47:53,491 - lava_callback - ERROR - Error processing callback for node 694081e2575bc060ce15cc21: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e2575bc060ce15cc21 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e2575bc060ce15cc21 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e2575bc060ce15cc21 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:53,493 - lava_callback - INFO - Starting async job submit for node 69407e3e575bc060ce15c2a1 2025-12-17 21:47:53,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:53,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e3e575bc060ce15c2a1 HTTP/1.1" 502 150 2025-12-17 21:47:53,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e3e575bc060ce15c2a1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:53,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e3e575bc060ce15c2a1 2025-12-17 21:47:53,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e3e575bc060ce15c2a1 HTTP/1.1" 502 150 2025-12-17 21:47:53,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e3e575bc060ce15c2a1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:54,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f0575bc060ce15cc5e 2025-12-17 21:47:54,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f0575bc060ce15cc5e HTTP/1.1" 502 150 2025-12-17 21:47:54,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f0575bc060ce15cc5e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:55,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e3e575bc060ce15c2a1 2025-12-17 21:47:55,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e3e575bc060ce15c2a1 HTTP/1.1" 502 150 2025-12-17 21:47:55,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e3e575bc060ce15c2a1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:55,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ed575bc060ce15cc50 2025-12-17 21:47:55,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ed575bc060ce15cc50 HTTP/1.1" 502 150 2025-12-17 21:47:55,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ed575bc060ce15cc50'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:55,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f1575bc060ce15cc65 2025-12-17 21:47:55,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f1575bc060ce15cc65 HTTP/1.1" 502 150 2025-12-17 21:47:55,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f1575bc060ce15cc65'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43414 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:47:56,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e4575bc060ce15cc2b 2025-12-17 21:47:57,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ee575bc060ce15cc57 2025-12-17 21:47:57,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ee575bc060ce15cc57 HTTP/1.1" 502 150 2025-12-17 21:47:57,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ee575bc060ce15cc57'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:57,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ea575bc060ce15cc42 2025-12-17 21:47:57,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ea575bc060ce15cc42 HTTP/1.1" 502 150 2025-12-17 21:47:57,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ea575bc060ce15cc42'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:58,362 - lava_callback - INFO - Received callback for node 69408200575bc060ce15cca2 INFO: 172.21.0.1:43430 - "POST /node/69408200575bc060ce15cca2 HTTP/1.0" 202 Accepted 2025-12-17 21:47:58,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081d6575bc060ce15cbf5 2025-12-17 21:47:58,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081d6575bc060ce15cbf5 HTTP/1.1" 502 150 2025-12-17 21:47:58,452 - lava_callback - ERROR - Error processing callback for node 694081d6575bc060ce15cbf5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081d6575bc060ce15cbf5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081d6575bc060ce15cbf5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081d6575bc060ce15cbf5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:47:58,454 - lava_callback - INFO - Starting async job submit for node 694081f2575bc060ce15cc68 2025-12-17 21:47:58,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:47:58,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f2575bc060ce15cc68 HTTP/1.1" 502 150 2025-12-17 21:47:58,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f2575bc060ce15cc68'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:58,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f2575bc060ce15cc68 2025-12-17 21:47:58,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f2575bc060ce15cc68 HTTP/1.1" 502 150 2025-12-17 21:47:58,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f2575bc060ce15cc68'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:58,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ec575bc060ce15cc49 2025-12-17 21:47:58,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ec575bc060ce15cc49 HTTP/1.1" 502 150 2025-12-17 21:47:58,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ec575bc060ce15cc49'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:58,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f0575bc060ce15cc5e 2025-12-17 21:47:58,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f0575bc060ce15cc5e HTTP/1.1" 502 150 2025-12-17 21:47:58,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f0575bc060ce15cc5e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:59,237 - lava_callback - INFO - Received callback for node 69408203575bc060ce15ccac INFO: 172.21.0.1:43436 - "POST /node/69408203575bc060ce15ccac HTTP/1.0" 202 Accepted 2025-12-17 21:47:59,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e3e575bc060ce15c2a1 2025-12-17 21:47:59,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e3e575bc060ce15c2a1 HTTP/1.1" 502 150 2025-12-17 21:47:59,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e3e575bc060ce15c2a1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:47:59,975 - lava_callback - INFO - Received callback for node 69408206575bc060ce15ccb2 INFO: 172.21.0.1:46926 - "POST /node/69408206575bc060ce15ccb2 HTTP/1.0" 202 Accepted 2025-12-17 21:48:00,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f2575bc060ce15cc68 2025-12-17 21:48:00,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f2575bc060ce15cc68 HTTP/1.1" 502 150 2025-12-17 21:48:00,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f2575bc060ce15cc68'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:00,766 - lava_callback - INFO - Received callback for node 6940820a575bc060ce15ccbf INFO: 172.21.0.1:46940 - "POST /node/6940820a575bc060ce15ccbf HTTP/1.0" 202 Accepted 2025-12-17 21:48:01,590 - lava_callback - INFO - Received callback for node 6940820c575bc060ce15ccc5 INFO: 172.21.0.1:46948 - "POST /node/6940820c575bc060ce15ccc5 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:46956 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:48:02,396 - lava_callback - INFO - Received callback for node 6940820e575bc060ce15ccc8 INFO: 172.21.0.1:46958 - "POST /node/6940820e575bc060ce15ccc8 HTTP/1.0" 202 Accepted 2025-12-17 21:48:02,831 - lava_callback - INFO - Received callback for node 694081ff575bc060ce15cc8e 2025-12-17 21:48:02,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e6575bc060ce15cc2e INFO: 172.21.0.1:46962 - "POST /node/694081ff575bc060ce15cc8e HTTP/1.0" 202 Accepted 2025-12-17 21:48:02,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e6575bc060ce15cc2e HTTP/1.1" 502 150 2025-12-17 21:48:02,840 - lava_callback - ERROR - Error processing callback for node 694081e6575bc060ce15cc2e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e6575bc060ce15cc2e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e6575bc060ce15cc2e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e6575bc060ce15cc2e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:02,842 - lava_callback - INFO - Starting async job submit for node 694081f3575bc060ce15cc6b 2025-12-17 21:48:02,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:02,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f3575bc060ce15cc6b HTTP/1.1" 502 150 2025-12-17 21:48:02,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f3575bc060ce15cc6b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:02,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f3575bc060ce15cc6b 2025-12-17 21:48:02,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f3575bc060ce15cc6b HTTP/1.1" 502 150 2025-12-17 21:48:02,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f3575bc060ce15cc6b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:03,503 - lava_callback - INFO - Received callback for node 69408202575bc060ce15cca9 2025-12-17 21:48:03,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e7575bc060ce15cc31 INFO: 172.21.0.1:46970 - "POST /node/69408202575bc060ce15cca9 HTTP/1.0" 202 Accepted 2025-12-17 21:48:03,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e7575bc060ce15cc31 HTTP/1.1" 502 150 2025-12-17 21:48:03,514 - lava_callback - ERROR - Error processing callback for node 694081e7575bc060ce15cc31: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e7575bc060ce15cc31 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e7575bc060ce15cc31 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e7575bc060ce15cc31 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:03,517 - lava_callback - INFO - Starting async job submit for node 694081f5575bc060ce15cc6e 2025-12-17 21:48:03,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:03,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f5575bc060ce15cc6e HTTP/1.1" 502 150 2025-12-17 21:48:03,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f5575bc060ce15cc6e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:03,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f5575bc060ce15cc6e 2025-12-17 21:48:03,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f5575bc060ce15cc6e HTTP/1.1" 502 150 2025-12-17 21:48:03,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f5575bc060ce15cc6e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:03,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ed575bc060ce15cc50 2025-12-17 21:48:03,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ed575bc060ce15cc50 HTTP/1.1" 502 150 2025-12-17 21:48:03,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ed575bc060ce15cc50'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:04,186 - lava_callback - INFO - Received callback for node 69408204575bc060ce15ccaf INFO: 172.21.0.1:46976 - "POST /node/69408204575bc060ce15ccaf HTTP/1.0" 202 Accepted 2025-12-17 21:48:04,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f2575bc060ce15cc68 2025-12-17 21:48:04,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f2575bc060ce15cc68 HTTP/1.1" 502 150 2025-12-17 21:48:04,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f2575bc060ce15cc68'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:04,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e8575bc060ce15cc34 2025-12-17 21:48:04,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e8575bc060ce15cc34 HTTP/1.1" 502 150 2025-12-17 21:48:04,834 - lava_callback - ERROR - Error processing callback for node 694081e8575bc060ce15cc34: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e8575bc060ce15cc34 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e8575bc060ce15cc34 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e8575bc060ce15cc34 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:04,837 - lava_callback - INFO - Starting async job submit for node 694081f6575bc060ce15cc71 2025-12-17 21:48:04,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:04,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f3575bc060ce15cc6b 2025-12-17 21:48:04,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f6575bc060ce15cc71 HTTP/1.1" 502 150 2025-12-17 21:48:04,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f6575bc060ce15cc71'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:04,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f6575bc060ce15cc71 2025-12-17 21:48:04,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f3575bc060ce15cc6b HTTP/1.1" 502 150 2025-12-17 21:48:04,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f3575bc060ce15cc6b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:04,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f6575bc060ce15cc71 HTTP/1.1" 502 150 2025-12-17 21:48:04,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f6575bc060ce15cc71'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:04,862 - lava_callback - INFO - Received callback for node 69408207575bc060ce15ccb5 INFO: 172.21.0.1:46986 - "POST /node/69408207575bc060ce15ccb5 HTTP/1.0" 202 Accepted 2025-12-17 21:48:05,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ee575bc060ce15cc57 2025-12-17 21:48:05,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ee575bc060ce15cc57 HTTP/1.1" 502 150 2025-12-17 21:48:05,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ee575bc060ce15cc57'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:05,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e9575bc060ce15cc39 2025-12-17 21:48:05,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e9575bc060ce15cc39 HTTP/1.1" 502 150 2025-12-17 21:48:05,503 - lava_callback - ERROR - Error processing callback for node 694081e9575bc060ce15cc39: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e9575bc060ce15cc39 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e9575bc060ce15cc39 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e9575bc060ce15cc39 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:05,521 - lava_callback - INFO - Starting async job submit for node 694081f7575bc060ce15cc74 2025-12-17 21:48:05,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:05,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f7575bc060ce15cc74 HTTP/1.1" 502 150 2025-12-17 21:48:05,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f7575bc060ce15cc74'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:05,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f7575bc060ce15cc74 2025-12-17 21:48:05,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f7575bc060ce15cc74 HTTP/1.1" 502 150 2025-12-17 21:48:05,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f7575bc060ce15cc74'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:05,557 - lava_callback - INFO - Received callback for node 6940820b575bc060ce15ccc2 2025-12-17 21:48:05,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f5575bc060ce15cc6e INFO: 172.21.0.1:47000 - "POST /node/6940820b575bc060ce15ccc2 HTTP/1.0" 202 Accepted 2025-12-17 21:48:05,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f5575bc060ce15cc6e HTTP/1.1" 502 150 2025-12-17 21:48:05,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f5575bc060ce15cc6e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47002 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:48:06,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081db575bc060ce15cc0a 2025-12-17 21:48:06,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081db575bc060ce15cc0a HTTP/1.1" 502 150 2025-12-17 21:48:06,847 - lava_callback - ERROR - Error processing callback for node 694081db575bc060ce15cc0a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081db575bc060ce15cc0a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081db575bc060ce15cc0a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081db575bc060ce15cc0a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:06,849 - lava_callback - INFO - Starting async job submit for node 694081f8575bc060ce15cc77 2025-12-17 21:48:06,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:06,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f6575bc060ce15cc71 2025-12-17 21:48:06,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f6575bc060ce15cc71 HTTP/1.1" 502 150 2025-12-17 21:48:06,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f6575bc060ce15cc71'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:06,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f8575bc060ce15cc77 HTTP/1.1" 502 150 2025-12-17 21:48:06,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f8575bc060ce15cc77'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:06,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f8575bc060ce15cc77 2025-12-17 21:48:06,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f8575bc060ce15cc77 HTTP/1.1" 502 150 2025-12-17 21:48:06,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f8575bc060ce15cc77'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:06,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f0575bc060ce15cc5e 2025-12-17 21:48:06,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f0575bc060ce15cc5e HTTP/1.1" 502 150 2025-12-17 21:48:06,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f0575bc060ce15cc5e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:07,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f7575bc060ce15cc74 2025-12-17 21:48:07,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f7575bc060ce15cc74 HTTP/1.1" 502 150 2025-12-17 21:48:07,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f7575bc060ce15cc74'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:07,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e3e575bc060ce15c2a1 2025-12-17 21:48:07,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e3e575bc060ce15c2a1 HTTP/1.1" 502 150 2025-12-17 21:48:07,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69407e3e575bc060ce15c2a1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:07,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081df575bc060ce15cc1b 2025-12-17 21:48:07,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081df575bc060ce15cc1b HTTP/1.1" 502 150 2025-12-17 21:48:07,595 - lava_callback - ERROR - Error processing callback for node 694081df575bc060ce15cc1b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081df575bc060ce15cc1b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081df575bc060ce15cc1b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081df575bc060ce15cc1b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:07,597 - lava_callback - INFO - Starting async job submit for node 694081fa575bc060ce15cc7a 2025-12-17 21:48:07,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:07,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fa575bc060ce15cc7a HTTP/1.1" 502 150 2025-12-17 21:48:07,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fa575bc060ce15cc7a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:07,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fa575bc060ce15cc7a 2025-12-17 21:48:07,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fa575bc060ce15cc7a HTTP/1.1" 502 150 2025-12-17 21:48:07,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fa575bc060ce15cc7a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:08,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f3575bc060ce15cc6b 2025-12-17 21:48:08,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f3575bc060ce15cc6b HTTP/1.1" 502 150 2025-12-17 21:48:08,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f3575bc060ce15cc6b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:08,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f8575bc060ce15cc77 2025-12-17 21:48:08,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f8575bc060ce15cc77 HTTP/1.1" 502 150 2025-12-17 21:48:08,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f8575bc060ce15cc77'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:09,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081e3575bc060ce15cc24 2025-12-17 21:48:09,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e3575bc060ce15cc24 HTTP/1.1" 502 150 2025-12-17 21:48:09,459 - lava_callback - ERROR - Error processing callback for node 694081e3575bc060ce15cc24: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e3575bc060ce15cc24 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e3575bc060ce15cc24 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e3575bc060ce15cc24 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:09,461 - lava_callback - INFO - Starting async job submit for node 694081fb575bc060ce15cc7d 2025-12-17 21:48:09,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:09,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fb575bc060ce15cc7d HTTP/1.1" 502 150 2025-12-17 21:48:09,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fb575bc060ce15cc7d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:09,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fb575bc060ce15cc7d 2025-12-17 21:48:09,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fb575bc060ce15cc7d HTTP/1.1" 502 150 2025-12-17 21:48:09,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fb575bc060ce15cc7d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:09,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f5575bc060ce15cc6e 2025-12-17 21:48:09,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f5575bc060ce15cc6e HTTP/1.1" 502 150 2025-12-17 21:48:09,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f5575bc060ce15cc6e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:09,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fa575bc060ce15cc7a 2025-12-17 21:48:09,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fa575bc060ce15cc7a HTTP/1.1" 502 150 2025-12-17 21:48:09,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fa575bc060ce15cc7a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:10,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f6575bc060ce15cc71 2025-12-17 21:48:10,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f6575bc060ce15cc71 HTTP/1.1" 502 150 2025-12-17 21:48:10,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f6575bc060ce15cc71'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:11,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fb575bc060ce15cc7d 2025-12-17 21:48:11,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fb575bc060ce15cc7d HTTP/1.1" 502 150 2025-12-17 21:48:11,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fb575bc060ce15cc7d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:11,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f7575bc060ce15cc74 2025-12-17 21:48:11,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f7575bc060ce15cc74 HTTP/1.1" 502 150 2025-12-17 21:48:11,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f7575bc060ce15cc74'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:11,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f1575bc060ce15cc65 2025-12-17 21:48:11,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f1575bc060ce15cc65 HTTP/1.1" 502 150 2025-12-17 21:48:11,837 - lava_callback - ERROR - Error processing callback for node 694081f1575bc060ce15cc65: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f1575bc060ce15cc65 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f1575bc060ce15cc65 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f1575bc060ce15cc65 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:11,838 - lava_callback - INFO - Starting async job submit for node 694081fc575bc060ce15cc80 2025-12-17 21:48:11,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:11,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fc575bc060ce15cc80 HTTP/1.1" 502 150 2025-12-17 21:48:11,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fc575bc060ce15cc80'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:11,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fc575bc060ce15cc80 2025-12-17 21:48:11,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fc575bc060ce15cc80 HTTP/1.1" 502 150 2025-12-17 21:48:11,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fc575bc060ce15cc80'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:12,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081e4575bc060ce15cc2b HTTP/1.1" 502 150 2025-12-17 21:48:12,080 - lava_callback - ERROR - Error processing callback for node 694081e4575bc060ce15cc2b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e4575bc060ce15cc2b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e4575bc060ce15cc2b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081e4575bc060ce15cc2b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:12,082 - lava_callback - INFO - Starting async job submit for node 694081fd575bc060ce15cc85 2025-12-17 21:48:12,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:12,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fd575bc060ce15cc85 HTTP/1.1" 502 150 2025-12-17 21:48:12,110 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fd575bc060ce15cc85'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:12,110 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fd575bc060ce15cc85 2025-12-17 21:48:12,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fd575bc060ce15cc85 HTTP/1.1" 502 150 2025-12-17 21:48:12,110 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fd575bc060ce15cc85'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:12,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f2575bc060ce15cc68 2025-12-17 21:48:12,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f2575bc060ce15cc68 HTTP/1.1" 502 150 2025-12-17 21:48:12,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f2575bc060ce15cc68'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:12,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f8575bc060ce15cc77 2025-12-17 21:48:12,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f8575bc060ce15cc77 HTTP/1.1" 502 150 2025-12-17 21:48:12,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f8575bc060ce15cc77'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:13,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fa575bc060ce15cc7a 2025-12-17 21:48:13,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fc575bc060ce15cc80 2025-12-17 21:48:13,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fc575bc060ce15cc80 HTTP/1.1" 502 150 2025-12-17 21:48:13,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fc575bc060ce15cc80'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:13,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ea575bc060ce15cc42 2025-12-17 21:48:13,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ea575bc060ce15cc42 HTTP/1.1" 502 150 2025-12-17 21:48:13,874 - lava_callback - ERROR - Error processing callback for node 694081ea575bc060ce15cc42: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ea575bc060ce15cc42 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ea575bc060ce15cc42 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ea575bc060ce15cc42 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:13,877 - lava_callback - INFO - Starting async job submit for node 69408200575bc060ce15cca2 2025-12-17 21:48:13,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:13,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408200575bc060ce15cca2 HTTP/1.1" 502 150 2025-12-17 21:48:13,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408200575bc060ce15cca2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:13,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408200575bc060ce15cca2 2025-12-17 21:48:13,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408200575bc060ce15cca2 HTTP/1.1" 502 150 2025-12-17 21:48:13,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408200575bc060ce15cca2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:14,111 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fd575bc060ce15cc85 2025-12-17 21:48:14,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fd575bc060ce15cc85 HTTP/1.1" 502 150 2025-12-17 21:48:14,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fd575bc060ce15cc85'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:14,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ec575bc060ce15cc49 2025-12-17 21:48:14,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ec575bc060ce15cc49 HTTP/1.1" 502 150 2025-12-17 21:48:14,586 - lava_callback - ERROR - Error processing callback for node 694081ec575bc060ce15cc49: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ec575bc060ce15cc49 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ec575bc060ce15cc49 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ec575bc060ce15cc49 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:14,588 - lava_callback - INFO - Starting async job submit for node 69408203575bc060ce15ccac 2025-12-17 21:48:14,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:14,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408203575bc060ce15ccac HTTP/1.1" 502 150 2025-12-17 21:48:14,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408203575bc060ce15ccac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:14,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408203575bc060ce15ccac 2025-12-17 21:48:14,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408203575bc060ce15ccac HTTP/1.1" 502 150 2025-12-17 21:48:14,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408203575bc060ce15ccac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51394 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:48:15,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fb575bc060ce15cc7d 2025-12-17 21:48:15,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fb575bc060ce15cc7d HTTP/1.1" 502 150 2025-12-17 21:48:15,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fb575bc060ce15cc7d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:15,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408200575bc060ce15cca2 2025-12-17 21:48:15,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408200575bc060ce15cca2 HTTP/1.1" 502 150 2025-12-17 21:48:15,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408200575bc060ce15cca2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51402 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:48:16,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408203575bc060ce15ccac 2025-12-17 21:48:16,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408203575bc060ce15ccac HTTP/1.1" 502 150 2025-12-17 21:48:16,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408203575bc060ce15ccac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:16,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f3575bc060ce15cc6b 2025-12-17 21:48:16,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f3575bc060ce15cc6b HTTP/1.1" 502 150 2025-12-17 21:48:16,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f3575bc060ce15cc6b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:17,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f5575bc060ce15cc6e 2025-12-17 21:48:17,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f5575bc060ce15cc6e HTTP/1.1" 502 150 2025-12-17 21:48:17,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f5575bc060ce15cc6e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:17,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fc575bc060ce15cc80 2025-12-17 21:48:17,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fc575bc060ce15cc80 HTTP/1.1" 502 150 2025-12-17 21:48:17,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fc575bc060ce15cc80'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:17,966 - lava_callback - INFO - Received callback for node 6940c5311e3c24e95681d69c INFO: 172.21.0.1:51416 - "POST /node/6940c5311e3c24e95681d69c HTTP/1.0" 202 Accepted 2025-12-17 21:48:18,089 - lava_callback - INFO - Received callback for node 69408217575bc060ce15cd25 INFO: 172.21.0.1:51418 - "POST /node/69408217575bc060ce15cd25 HTTP/1.0" 202 Accepted 2025-12-17 21:48:18,112 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fd575bc060ce15cc85 2025-12-17 21:48:18,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fd575bc060ce15cc85 HTTP/1.1" 502 150 2025-12-17 21:48:18,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fd575bc060ce15cc85'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:18,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f6575bc060ce15cc71 2025-12-17 21:48:18,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f6575bc060ce15cc71 HTTP/1.1" 502 150 2025-12-17 21:48:18,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f6575bc060ce15cc71'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:18,990 - lava_callback - INFO - Received callback for node 69408218575bc060ce15cd28 INFO: 172.21.0.1:51426 - "POST /node/69408218575bc060ce15cd28 HTTP/1.0" 202 Accepted 2025-12-17 21:48:19,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f7575bc060ce15cc74 2025-12-17 21:48:19,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f7575bc060ce15cc74 HTTP/1.1" 502 150 2025-12-17 21:48:19,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f7575bc060ce15cc74'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:19,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ed575bc060ce15cc50 2025-12-17 21:48:19,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ed575bc060ce15cc50 HTTP/1.1" 502 150 2025-12-17 21:48:19,626 - lava_callback - ERROR - Error processing callback for node 694081ed575bc060ce15cc50: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ed575bc060ce15cc50 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ed575bc060ce15cc50 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ed575bc060ce15cc50 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:19,628 - lava_callback - INFO - Starting async job submit for node 69408206575bc060ce15ccb2 2025-12-17 21:48:19,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:19,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408206575bc060ce15ccb2 HTTP/1.1" 502 150 2025-12-17 21:48:19,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408206575bc060ce15ccb2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:19,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408206575bc060ce15ccb2 2025-12-17 21:48:19,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408206575bc060ce15ccb2 HTTP/1.1" 502 150 2025-12-17 21:48:19,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408206575bc060ce15ccb2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:19,843 - lava_callback - INFO - Received callback for node 6940821a575bc060ce15cd2b INFO: 172.21.0.1:33252 - "POST /node/6940821a575bc060ce15cd2b HTTP/1.0" 202 Accepted 2025-12-17 21:48:19,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408200575bc060ce15cca2 2025-12-17 21:48:19,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408200575bc060ce15cca2 HTTP/1.1" 502 150 2025-12-17 21:48:19,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408200575bc060ce15cca2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:20,565 - lava_callback - INFO - Received callback for node 6940821b575bc060ce15cd2e INFO: 172.21.0.1:33266 - "POST /node/6940821b575bc060ce15cd2e HTTP/1.0" 202 Accepted 2025-12-17 21:48:20,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408203575bc060ce15ccac 2025-12-17 21:48:20,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408203575bc060ce15ccac HTTP/1.1" 502 150 2025-12-17 21:48:20,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408203575bc060ce15ccac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:20,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f8575bc060ce15cc77 2025-12-17 21:48:20,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f8575bc060ce15cc77 HTTP/1.1" 502 150 2025-12-17 21:48:20,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081f8575bc060ce15cc77'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:21,374 - lava_callback - INFO - Received callback for node 6940821d575bc060ce15cd34 INFO: 172.21.0.1:33282 - "POST /node/6940821d575bc060ce15cd34 HTTP/1.0" 202 Accepted 2025-12-17 21:48:21,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ee575bc060ce15cc57 2025-12-17 21:48:21,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ee575bc060ce15cc57 HTTP/1.1" 502 150 2025-12-17 21:48:21,475 - lava_callback - ERROR - Error processing callback for node 694081ee575bc060ce15cc57: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ee575bc060ce15cc57 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ee575bc060ce15cc57 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ee575bc060ce15cc57 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:21,477 - lava_callback - INFO - Starting async job submit for node 6940820a575bc060ce15ccbf 2025-12-17 21:48:21,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:21,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820a575bc060ce15ccbf HTTP/1.1" 502 150 2025-12-17 21:48:21,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820a575bc060ce15ccbf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:21,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820a575bc060ce15ccbf 2025-12-17 21:48:21,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820a575bc060ce15ccbf HTTP/1.1" 502 150 2025-12-17 21:48:21,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820a575bc060ce15ccbf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:21,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408206575bc060ce15ccb2 2025-12-17 21:48:21,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408206575bc060ce15ccb2 HTTP/1.1" 502 150 2025-12-17 21:48:21,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408206575bc060ce15ccb2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:22,565 - lava_callback - INFO - Received callback for node 6940821f575bc060ce15cd37 INFO: 172.21.0.1:33298 - "POST /node/6940821f575bc060ce15cd37 HTTP/1.0" 202 Accepted 2025-12-17 21:48:22,821 - lava_callback - INFO - Received callback for node 69408211575bc060ce15ccd6 INFO: 172.21.0.1:33302 - "POST /node/69408211575bc060ce15ccd6 HTTP/1.0" 202 Accepted 2025-12-17 21:48:22,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f0575bc060ce15cc5e 2025-12-17 21:48:22,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f0575bc060ce15cc5e HTTP/1.1" 502 150 2025-12-17 21:48:22,887 - lava_callback - ERROR - Error processing callback for node 694081f0575bc060ce15cc5e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f0575bc060ce15cc5e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f0575bc060ce15cc5e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f0575bc060ce15cc5e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:22,889 - lava_callback - INFO - Starting async job submit for node 6940820c575bc060ce15ccc5 2025-12-17 21:48:22,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:22,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820c575bc060ce15ccc5 HTTP/1.1" 502 150 2025-12-17 21:48:22,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820c575bc060ce15ccc5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:22,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820c575bc060ce15ccc5 2025-12-17 21:48:22,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820c575bc060ce15ccc5 HTTP/1.1" 502 150 2025-12-17 21:48:22,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820c575bc060ce15ccc5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:23,444 - lava_callback - INFO - Received callback for node 69408221575bc060ce15ce99 INFO: 172.21.0.1:33308 - "POST /node/69408221575bc060ce15ce99 HTTP/1.0" 202 Accepted 2025-12-17 21:48:23,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fb575bc060ce15cc7d 2025-12-17 21:48:23,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fb575bc060ce15cc7d HTTP/1.1" 502 150 2025-12-17 21:48:23,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fb575bc060ce15cc7d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:23,497 - lava_callback - INFO - Received callback for node 6940821b575bc060ce15cd31 INFO: 172.21.0.1:33320 - "POST /node/6940821b575bc060ce15cd31 HTTP/1.0" 202 Accepted 2025-12-17 21:48:23,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820a575bc060ce15ccbf 2025-12-17 21:48:23,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820a575bc060ce15ccbf HTTP/1.1" 502 150 2025-12-17 21:48:23,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820a575bc060ce15ccbf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:23,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69407e3e575bc060ce15c2a1 2025-12-17 21:48:23,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69407e3e575bc060ce15c2a1 HTTP/1.1" 502 150 2025-12-17 21:48:23,544 - lava_callback - ERROR - Error processing callback for node 69407e3e575bc060ce15c2a1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e3e575bc060ce15c2a1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e3e575bc060ce15c2a1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69407e3e575bc060ce15c2a1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:23,546 - lava_callback - INFO - Starting async job submit for node 6940820e575bc060ce15ccc8 2025-12-17 21:48:23,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:23,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820e575bc060ce15ccc8 HTTP/1.1" 502 150 2025-12-17 21:48:23,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820e575bc060ce15ccc8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:23,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820e575bc060ce15ccc8 2025-12-17 21:48:23,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820e575bc060ce15ccc8 HTTP/1.1" 502 150 2025-12-17 21:48:23,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820e575bc060ce15ccc8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:24,177 - lava_callback - INFO - Received callback for node 69408226575bc060ce15ce9e INFO: 172.21.0.1:33326 - "POST /node/69408226575bc060ce15ce9e HTTP/1.0" 202 Accepted 2025-12-17 21:48:24,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820c575bc060ce15ccc5 2025-12-17 21:48:24,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820c575bc060ce15ccc5 HTTP/1.1" 502 150 2025-12-17 21:48:24,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820c575bc060ce15ccc5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:25,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820e575bc060ce15ccc8 2025-12-17 21:48:25,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820e575bc060ce15ccc8 HTTP/1.1" 502 150 2025-12-17 21:48:25,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820e575bc060ce15ccc8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:25,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408206575bc060ce15ccb2 2025-12-17 21:48:25,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408206575bc060ce15ccb2 HTTP/1.1" 502 150 2025-12-17 21:48:25,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408206575bc060ce15ccb2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:25,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fc575bc060ce15cc80 2025-12-17 21:48:25,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fc575bc060ce15cc80 HTTP/1.1" 502 150 2025-12-17 21:48:25,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fc575bc060ce15cc80'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:26,113 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fd575bc060ce15cc85 INFO: 172.21.0.1:33336 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:48:27,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820a575bc060ce15ccbf 2025-12-17 21:48:27,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820a575bc060ce15ccbf HTTP/1.1" 502 150 2025-12-17 21:48:27,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820a575bc060ce15ccbf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:27,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408200575bc060ce15cca2 2025-12-17 21:48:27,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408200575bc060ce15cca2 HTTP/1.1" 502 150 2025-12-17 21:48:27,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408200575bc060ce15cca2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:28,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f2575bc060ce15cc68 2025-12-17 21:48:28,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f2575bc060ce15cc68 HTTP/1.1" 502 150 2025-12-17 21:48:28,473 - lava_callback - ERROR - Error processing callback for node 694081f2575bc060ce15cc68: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f2575bc060ce15cc68 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f2575bc060ce15cc68 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f2575bc060ce15cc68 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:28,476 - lava_callback - INFO - Starting async job submit for node 694081ff575bc060ce15cc8e 2025-12-17 21:48:28,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:28,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ff575bc060ce15cc8e HTTP/1.1" 502 150 2025-12-17 21:48:28,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ff575bc060ce15cc8e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:28,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ff575bc060ce15cc8e 2025-12-17 21:48:28,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ff575bc060ce15cc8e HTTP/1.1" 502 150 2025-12-17 21:48:28,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ff575bc060ce15cc8e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:28,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408203575bc060ce15ccac 2025-12-17 21:48:28,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408203575bc060ce15ccac HTTP/1.1" 502 150 2025-12-17 21:48:28,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408203575bc060ce15ccac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:28,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820c575bc060ce15ccc5 2025-12-17 21:48:28,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820c575bc060ce15ccc5 HTTP/1.1" 502 150 2025-12-17 21:48:28,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820c575bc060ce15ccc5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:28,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fa575bc060ce15cc7a HTTP/1.1" 502 150 2025-12-17 21:48:28,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fa575bc060ce15cc7a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:29,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820e575bc060ce15ccc8 2025-12-17 21:48:29,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820e575bc060ce15ccc8 HTTP/1.1" 502 150 2025-12-17 21:48:29,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820e575bc060ce15ccc8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:30,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ff575bc060ce15cc8e 2025-12-17 21:48:30,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ff575bc060ce15cc8e HTTP/1.1" 502 150 2025-12-17 21:48:30,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ff575bc060ce15cc8e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:31,747 - lava_callback - INFO - Received callback for node 6940822c575bc060ce15ceb8 INFO: 172.21.0.1:45020 - "POST /node/6940822c575bc060ce15ceb8 HTTP/1.0" 202 Accepted 2025-12-17 21:48:32,764 - lava_callback - INFO - Received callback for node 69408480575bc060ce15d7d4 INFO: 172.21.0.1:45026 - "POST /node/69408480575bc060ce15d7d4 HTTP/1.0" 202 Accepted 2025-12-17 21:48:32,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f3575bc060ce15cc6b 2025-12-17 21:48:32,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f3575bc060ce15cc6b HTTP/1.1" 502 150 2025-12-17 21:48:32,857 - lava_callback - ERROR - Error processing callback for node 694081f3575bc060ce15cc6b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f3575bc060ce15cc6b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f3575bc060ce15cc6b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f3575bc060ce15cc6b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:32,859 - lava_callback - INFO - Starting async job submit for node 69408202575bc060ce15cca9 2025-12-17 21:48:32,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:32,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408202575bc060ce15cca9 HTTP/1.1" 502 150 2025-12-17 21:48:32,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408202575bc060ce15cca9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:32,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408202575bc060ce15cca9 2025-12-17 21:48:32,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408202575bc060ce15cca9 HTTP/1.1" 502 150 2025-12-17 21:48:32,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408202575bc060ce15cca9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:33,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f5575bc060ce15cc6e 2025-12-17 21:48:33,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408206575bc060ce15ccb2 2025-12-17 21:48:33,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408206575bc060ce15ccb2 HTTP/1.1" 502 150 2025-12-17 21:48:33,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408206575bc060ce15ccb2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:34,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ff575bc060ce15cc8e 2025-12-17 21:48:34,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ff575bc060ce15cc8e HTTP/1.1" 502 150 2025-12-17 21:48:34,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ff575bc060ce15cc8e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:34,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f6575bc060ce15cc71 2025-12-17 21:48:34,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f6575bc060ce15cc71 HTTP/1.1" 502 150 2025-12-17 21:48:34,858 - lava_callback - ERROR - Error processing callback for node 694081f6575bc060ce15cc71: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f6575bc060ce15cc71 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f6575bc060ce15cc71 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f6575bc060ce15cc71 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:34,860 - lava_callback - INFO - Starting async job submit for node 69408204575bc060ce15ccaf 2025-12-17 21:48:34,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:34,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408202575bc060ce15cca9 2025-12-17 21:48:34,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408202575bc060ce15cca9 HTTP/1.1" 502 150 2025-12-17 21:48:34,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408202575bc060ce15cca9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:34,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408204575bc060ce15ccaf HTTP/1.1" 502 150 2025-12-17 21:48:34,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408204575bc060ce15ccaf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:34,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408204575bc060ce15ccaf 2025-12-17 21:48:34,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408204575bc060ce15ccaf HTTP/1.1" 502 150 2025-12-17 21:48:34,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408204575bc060ce15ccaf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:35,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820a575bc060ce15ccbf 2025-12-17 21:48:35,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820a575bc060ce15ccbf HTTP/1.1" 502 150 2025-12-17 21:48:35,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820a575bc060ce15ccbf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:35,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f7575bc060ce15cc74 2025-12-17 21:48:35,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f7575bc060ce15cc74 HTTP/1.1" 502 150 2025-12-17 21:48:35,546 - lava_callback - ERROR - Error processing callback for node 694081f7575bc060ce15cc74: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f7575bc060ce15cc74 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f7575bc060ce15cc74 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f7575bc060ce15cc74 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:35,548 - lava_callback - INFO - Starting async job submit for node 69408207575bc060ce15ccb5 2025-12-17 21:48:35,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:35,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408207575bc060ce15ccb5 HTTP/1.1" 502 150 2025-12-17 21:48:35,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408207575bc060ce15ccb5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:35,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408207575bc060ce15ccb5 2025-12-17 21:48:35,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408207575bc060ce15ccb5 HTTP/1.1" 502 150 2025-12-17 21:48:35,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408207575bc060ce15ccb5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45036 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:48:36,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081f8575bc060ce15cc77 2025-12-17 21:48:36,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f8575bc060ce15cc77 HTTP/1.1" 502 150 2025-12-17 21:48:36,862 - lava_callback - ERROR - Error processing callback for node 694081f8575bc060ce15cc77: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f8575bc060ce15cc77 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f8575bc060ce15cc77 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f8575bc060ce15cc77 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:36,865 - lava_callback - INFO - Starting async job submit for node 6940820b575bc060ce15ccc2 2025-12-17 21:48:36,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:36,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408204575bc060ce15ccaf 2025-12-17 21:48:36,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408204575bc060ce15ccaf HTTP/1.1" 502 150 2025-12-17 21:48:36,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408204575bc060ce15ccaf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:36,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820b575bc060ce15ccc2 HTTP/1.1" 502 150 2025-12-17 21:48:36,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820b575bc060ce15ccc2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:36,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820b575bc060ce15ccc2 2025-12-17 21:48:36,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820b575bc060ce15ccc2 HTTP/1.1" 502 150 2025-12-17 21:48:36,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820b575bc060ce15ccc2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:36,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820c575bc060ce15ccc5 2025-12-17 21:48:36,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820c575bc060ce15ccc5 HTTP/1.1" 502 150 2025-12-17 21:48:36,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820c575bc060ce15ccc5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:36,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fa575bc060ce15cc7a 2025-12-17 21:48:36,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fa575bc060ce15cc7a HTTP/1.1" 502 150 2025-12-17 21:48:36,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fa575bc060ce15cc7a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:37,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408207575bc060ce15ccb5 2025-12-17 21:48:37,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408207575bc060ce15ccb5 HTTP/1.1" 502 150 2025-12-17 21:48:37,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408207575bc060ce15ccb5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:37,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820e575bc060ce15ccc8 2025-12-17 21:48:37,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820e575bc060ce15ccc8 HTTP/1.1" 502 150 2025-12-17 21:48:37,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820e575bc060ce15ccc8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:38,485 - lava_callback - INFO - Received callback for node 69408228575bc060ce15cea6 INFO: 172.21.0.1:45052 - "POST /node/69408228575bc060ce15cea6 HTTP/1.0" 202 Accepted 2025-12-17 21:48:38,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408202575bc060ce15cca9 2025-12-17 21:48:38,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408202575bc060ce15cca9 HTTP/1.1" 502 150 2025-12-17 21:48:38,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408202575bc060ce15cca9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:38,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820b575bc060ce15ccc2 2025-12-17 21:48:38,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820b575bc060ce15ccc2 HTTP/1.1" 502 150 2025-12-17 21:48:38,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820b575bc060ce15ccc2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:39,235 - lava_callback - INFO - Received callback for node 69408229575bc060ce15ceab INFO: 172.21.0.1:45054 - "POST /node/69408229575bc060ce15ceab HTTP/1.0" 202 Accepted 2025-12-17 21:48:39,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fb575bc060ce15cc7d 2025-12-17 21:48:39,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fb575bc060ce15cc7d HTTP/1.1" 502 150 2025-12-17 21:48:39,474 - lava_callback - ERROR - Error processing callback for node 694081fb575bc060ce15cc7d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081fb575bc060ce15cc7d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081fb575bc060ce15cc7d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081fb575bc060ce15cc7d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:39,476 - lava_callback - INFO - Starting async job submit for node 6940c5311e3c24e95681d69c 2025-12-17 21:48:39,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:39,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5311e3c24e95681d69c HTTP/1.1" 502 150 2025-12-17 21:48:39,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5311e3c24e95681d69c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:39,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5311e3c24e95681d69c 2025-12-17 21:48:39,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5311e3c24e95681d69c HTTP/1.1" 502 150 2025-12-17 21:48:39,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5311e3c24e95681d69c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:39,968 - lava_callback - INFO - Received callback for node 6940822b575bc060ce15ceb2 INFO: 172.21.0.1:50904 - "POST /node/6940822b575bc060ce15ceb2 HTTP/1.0" 202 Accepted 2025-12-17 21:48:40,702 - lava_callback - INFO - Received callback for node 69408236575bc060ce15cec8 INFO: 172.21.0.1:50914 - "POST /node/69408236575bc060ce15cec8 HTTP/1.0" 202 Accepted 2025-12-17 21:48:40,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408204575bc060ce15ccaf 2025-12-17 21:48:40,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408204575bc060ce15ccaf HTTP/1.1" 502 150 2025-12-17 21:48:40,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408204575bc060ce15ccaf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:41,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fd575bc060ce15cc85 HTTP/1.1" 502 150 2025-12-17 21:48:41,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081fd575bc060ce15cc85'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:41,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5311e3c24e95681d69c 2025-12-17 21:48:41,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5311e3c24e95681d69c HTTP/1.1" 502 150 2025-12-17 21:48:41,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5311e3c24e95681d69c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:41,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408207575bc060ce15ccb5 2025-12-17 21:48:41,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408207575bc060ce15ccb5 HTTP/1.1" 502 150 2025-12-17 21:48:41,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408207575bc060ce15ccb5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:41,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fc575bc060ce15cc80 2025-12-17 21:48:41,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fc575bc060ce15cc80 HTTP/1.1" 502 150 2025-12-17 21:48:41,852 - lava_callback - ERROR - Error processing callback for node 694081fc575bc060ce15cc80: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081fc575bc060ce15cc80 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081fc575bc060ce15cc80 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081fc575bc060ce15cc80 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:41,854 - lava_callback - INFO - Starting async job submit for node 69408217575bc060ce15cd25 2025-12-17 21:48:41,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:41,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408217575bc060ce15cd25 HTTP/1.1" 502 150 2025-12-17 21:48:41,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408217575bc060ce15cd25'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:41,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408217575bc060ce15cd25 2025-12-17 21:48:41,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408217575bc060ce15cd25 HTTP/1.1" 502 150 2025-12-17 21:48:41,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408217575bc060ce15cd25'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:42,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ff575bc060ce15cc8e 2025-12-17 21:48:42,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ff575bc060ce15cc8e HTTP/1.1" 502 150 2025-12-17 21:48:42,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081ff575bc060ce15cc8e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:42,851 - lava_callback - INFO - Received callback for node 69408232575bc060ce15cec2 INFO: 172.21.0.1:50924 - "POST /node/69408232575bc060ce15cec2 HTTP/1.0" 202 Accepted 2025-12-17 21:48:42,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820b575bc060ce15ccc2 2025-12-17 21:48:42,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820b575bc060ce15ccc2 HTTP/1.1" 502 150 2025-12-17 21:48:42,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820b575bc060ce15ccc2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:43,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408217575bc060ce15cd25 2025-12-17 21:48:43,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408217575bc060ce15cd25 HTTP/1.1" 502 150 2025-12-17 21:48:43,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408217575bc060ce15cd25'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:43,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408200575bc060ce15cca2 2025-12-17 21:48:43,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408200575bc060ce15cca2 HTTP/1.1" 502 150 2025-12-17 21:48:43,892 - lava_callback - ERROR - Error processing callback for node 69408200575bc060ce15cca2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408200575bc060ce15cca2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408200575bc060ce15cca2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408200575bc060ce15cca2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:43,894 - lava_callback - INFO - Starting async job submit for node 69408218575bc060ce15cd28 2025-12-17 21:48:43,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:43,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408218575bc060ce15cd28 HTTP/1.1" 502 150 2025-12-17 21:48:43,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408218575bc060ce15cd28'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:43,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408218575bc060ce15cd28 2025-12-17 21:48:43,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408218575bc060ce15cd28 HTTP/1.1" 502 150 2025-12-17 21:48:43,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408218575bc060ce15cd28'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:44,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408203575bc060ce15ccac 2025-12-17 21:48:45,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5311e3c24e95681d69c 2025-12-17 21:48:45,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5311e3c24e95681d69c HTTP/1.1" 502 150 2025-12-17 21:48:45,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5311e3c24e95681d69c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:45,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408218575bc060ce15cd28 2025-12-17 21:48:45,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408218575bc060ce15cd28 HTTP/1.1" 502 150 2025-12-17 21:48:45,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408218575bc060ce15cd28'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50930 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:48:46,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408202575bc060ce15cca9 2025-12-17 21:48:46,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408202575bc060ce15cca9 HTTP/1.1" 502 150 2025-12-17 21:48:46,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408202575bc060ce15cca9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:47,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408217575bc060ce15cd25 2025-12-17 21:48:47,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408217575bc060ce15cd25 HTTP/1.1" 502 150 2025-12-17 21:48:47,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408217575bc060ce15cd25'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:48,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408204575bc060ce15ccaf 2025-12-17 21:48:48,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408204575bc060ce15ccaf HTTP/1.1" 502 150 2025-12-17 21:48:48,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408204575bc060ce15ccaf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:48,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081f5575bc060ce15cc6e HTTP/1.1" 502 150 2025-12-17 21:48:48,948 - lava_callback - ERROR - Error processing callback for node 694081f5575bc060ce15cc6e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f5575bc060ce15cc6e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f5575bc060ce15cc6e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081f5575bc060ce15cc6e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:48,950 - lava_callback - INFO - Starting async job submit for node 6940821a575bc060ce15cd2b 2025-12-17 21:48:48,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:48,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821a575bc060ce15cd2b HTTP/1.1" 502 150 2025-12-17 21:48:48,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821a575bc060ce15cd2b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:48,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821a575bc060ce15cd2b 2025-12-17 21:48:48,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821a575bc060ce15cd2b HTTP/1.1" 502 150 2025-12-17 21:48:48,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821a575bc060ce15cd2b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:49,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408207575bc060ce15ccb5 2025-12-17 21:48:49,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408207575bc060ce15ccb5 HTTP/1.1" 502 150 2025-12-17 21:48:49,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408207575bc060ce15ccb5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:49,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408206575bc060ce15ccb2 2025-12-17 21:48:49,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408206575bc060ce15ccb2 HTTP/1.1" 502 150 2025-12-17 21:48:49,648 - lava_callback - ERROR - Error processing callback for node 69408206575bc060ce15ccb2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408206575bc060ce15ccb2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408206575bc060ce15ccb2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408206575bc060ce15ccb2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:49,649 - lava_callback - INFO - Starting async job submit for node 6940821b575bc060ce15cd2e 2025-12-17 21:48:49,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:49,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821b575bc060ce15cd2e HTTP/1.1" 502 150 2025-12-17 21:48:49,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821b575bc060ce15cd2e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:49,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821b575bc060ce15cd2e 2025-12-17 21:48:49,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821b575bc060ce15cd2e HTTP/1.1" 502 150 2025-12-17 21:48:49,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821b575bc060ce15cd2e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:49,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408218575bc060ce15cd28 2025-12-17 21:48:49,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408218575bc060ce15cd28 HTTP/1.1" 502 150 2025-12-17 21:48:49,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408218575bc060ce15cd28'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:50,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820b575bc060ce15ccc2 2025-12-17 21:48:50,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820b575bc060ce15ccc2 HTTP/1.1" 502 150 2025-12-17 21:48:50,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940820b575bc060ce15ccc2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:50,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821a575bc060ce15cd2b 2025-12-17 21:48:50,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821a575bc060ce15cd2b HTTP/1.1" 502 150 2025-12-17 21:48:50,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821a575bc060ce15cd2b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:51,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820a575bc060ce15ccbf 2025-12-17 21:48:51,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820a575bc060ce15ccbf HTTP/1.1" 502 150 2025-12-17 21:48:51,513 - lava_callback - ERROR - Error processing callback for node 6940820a575bc060ce15ccbf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940820a575bc060ce15ccbf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940820a575bc060ce15ccbf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940820a575bc060ce15ccbf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:51,515 - lava_callback - INFO - Starting async job submit for node 6940821d575bc060ce15cd34 2025-12-17 21:48:51,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:51,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821d575bc060ce15cd34 HTTP/1.1" 502 150 2025-12-17 21:48:51,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821d575bc060ce15cd34'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:51,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821d575bc060ce15cd34 2025-12-17 21:48:51,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821d575bc060ce15cd34 HTTP/1.1" 502 150 2025-12-17 21:48:51,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821d575bc060ce15cd34'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:51,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821b575bc060ce15cd2e 2025-12-17 21:48:51,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821b575bc060ce15cd2e HTTP/1.1" 502 150 2025-12-17 21:48:51,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821b575bc060ce15cd2e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:51,786 - lava_callback - INFO - Received callback for node 69408231575bc060ce15cebf INFO: 172.21.0.1:50862 - "POST /node/69408231575bc060ce15cebf HTTP/1.0" 202 Accepted 2025-12-17 21:48:52,413 - lava_callback - INFO - Received callback for node 69408234575bc060ce15cec5 INFO: 172.21.0.1:50868 - "POST /node/69408234575bc060ce15cec5 HTTP/1.0" 202 Accepted 2025-12-17 21:48:52,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820c575bc060ce15ccc5 2025-12-17 21:48:52,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820c575bc060ce15ccc5 HTTP/1.1" 502 150 2025-12-17 21:48:52,908 - lava_callback - ERROR - Error processing callback for node 6940820c575bc060ce15ccc5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940820c575bc060ce15ccc5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940820c575bc060ce15ccc5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940820c575bc060ce15ccc5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:52,909 - lava_callback - INFO - Starting async job submit for node 6940821f575bc060ce15cd37 2025-12-17 21:48:52,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:52,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821f575bc060ce15cd37 HTTP/1.1" 502 150 2025-12-17 21:48:52,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821f575bc060ce15cd37'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:52,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821f575bc060ce15cd37 2025-12-17 21:48:52,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821f575bc060ce15cd37 HTTP/1.1" 502 150 2025-12-17 21:48:52,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821f575bc060ce15cd37'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:52,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fa575bc060ce15cc7a 2025-12-17 21:48:52,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fa575bc060ce15cc7a HTTP/1.1" 502 150 2025-12-17 21:48:52,979 - lava_callback - ERROR - Error processing callback for node 694081fa575bc060ce15cc7a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081fa575bc060ce15cc7a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081fa575bc060ce15cc7a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081fa575bc060ce15cc7a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:52,981 - lava_callback - INFO - Starting async job submit for node 69408211575bc060ce15ccd6 2025-12-17 21:48:52,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:52,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408211575bc060ce15ccd6 HTTP/1.1" 502 150 2025-12-17 21:48:52,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408211575bc060ce15ccd6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:52,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408211575bc060ce15ccd6 2025-12-17 21:48:52,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408211575bc060ce15ccd6 HTTP/1.1" 502 150 2025-12-17 21:48:52,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408211575bc060ce15ccd6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:53,236 - lava_callback - INFO - Received callback for node 69408237575bc060ce15cecb INFO: 172.21.0.1:50870 - "POST /node/69408237575bc060ce15cecb HTTP/1.0" 202 Accepted 2025-12-17 21:48:53,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5311e3c24e95681d69c 2025-12-17 21:48:53,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5311e3c24e95681d69c HTTP/1.1" 502 150 2025-12-17 21:48:53,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5311e3c24e95681d69c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:53,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821d575bc060ce15cd34 2025-12-17 21:48:53,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821d575bc060ce15cd34 HTTP/1.1" 502 150 2025-12-17 21:48:53,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821d575bc060ce15cd34'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:53,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820e575bc060ce15ccc8 2025-12-17 21:48:53,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820e575bc060ce15ccc8 HTTP/1.1" 502 150 2025-12-17 21:48:53,567 - lava_callback - ERROR - Error processing callback for node 6940820e575bc060ce15ccc8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940820e575bc060ce15ccc8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940820e575bc060ce15ccc8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940820e575bc060ce15ccc8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:53,569 - lava_callback - INFO - Starting async job submit for node 69408221575bc060ce15ce99 2025-12-17 21:48:53,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:53,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408221575bc060ce15ce99 HTTP/1.1" 502 150 2025-12-17 21:48:53,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408221575bc060ce15ce99'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:53,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408221575bc060ce15ce99 2025-12-17 21:48:53,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408221575bc060ce15ce99 HTTP/1.1" 502 150 2025-12-17 21:48:53,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408221575bc060ce15ce99'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:54,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821f575bc060ce15cd37 2025-12-17 21:48:54,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821f575bc060ce15cd37 HTTP/1.1" 502 150 2025-12-17 21:48:54,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821f575bc060ce15cd37'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:54,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821a575bc060ce15cd2b 2025-12-17 21:48:54,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821a575bc060ce15cd2b HTTP/1.1" 502 150 2025-12-17 21:48:54,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821a575bc060ce15cd2b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:54,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408211575bc060ce15ccd6 2025-12-17 21:48:54,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408211575bc060ce15ccd6 HTTP/1.1" 502 150 2025-12-17 21:48:54,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408211575bc060ce15ccd6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:55,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408221575bc060ce15ce99 2025-12-17 21:48:55,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408221575bc060ce15ce99 HTTP/1.1" 502 150 2025-12-17 21:48:55,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408221575bc060ce15ce99'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:55,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821b575bc060ce15cd2e 2025-12-17 21:48:55,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821b575bc060ce15cd2e HTTP/1.1" 502 150 2025-12-17 21:48:55,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821b575bc060ce15cd2e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:55,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408217575bc060ce15cd25 2025-12-17 21:48:55,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408217575bc060ce15cd25 HTTP/1.1" 502 150 2025-12-17 21:48:55,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408217575bc060ce15cd25'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50876 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:48:57,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081fd575bc060ce15cc85 2025-12-17 21:48:57,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081fd575bc060ce15cc85 HTTP/1.1" 502 150 2025-12-17 21:48:57,523 - lava_callback - ERROR - Error processing callback for node 694081fd575bc060ce15cc85: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081fd575bc060ce15cc85 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081fd575bc060ce15cc85 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081fd575bc060ce15cc85 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:57,525 - lava_callback - INFO - Starting async job submit for node 6940821b575bc060ce15cd31 2025-12-17 21:48:57,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:57,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821b575bc060ce15cd31 HTTP/1.1" 502 150 2025-12-17 21:48:57,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821b575bc060ce15cd31'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:57,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821b575bc060ce15cd31 2025-12-17 21:48:57,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821b575bc060ce15cd31 HTTP/1.1" 502 150 2025-12-17 21:48:57,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821b575bc060ce15cd31'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:57,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821d575bc060ce15cd34 2025-12-17 21:48:57,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821d575bc060ce15cd34 HTTP/1.1" 502 150 2025-12-17 21:48:57,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821d575bc060ce15cd34'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:57,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408218575bc060ce15cd28 2025-12-17 21:48:57,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408218575bc060ce15cd28 HTTP/1.1" 502 150 2025-12-17 21:48:57,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408218575bc060ce15cd28'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:58,234 - lava_callback - INFO - Received callback for node 69408239575bc060ce15cece INFO: 172.21.0.1:50886 - "POST /node/69408239575bc060ce15cece HTTP/1.0" 202 Accepted 2025-12-17 21:48:58,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081ff575bc060ce15cc8e 2025-12-17 21:48:58,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081ff575bc060ce15cc8e HTTP/1.1" 502 150 2025-12-17 21:48:58,491 - lava_callback - ERROR - Error processing callback for node 694081ff575bc060ce15cc8e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ff575bc060ce15cc8e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ff575bc060ce15cc8e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081ff575bc060ce15cc8e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:58,493 - lava_callback - INFO - Starting async job submit for node 69408226575bc060ce15ce9e 2025-12-17 21:48:58,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:58,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408226575bc060ce15ce9e HTTP/1.1" 502 150 2025-12-17 21:48:58,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408226575bc060ce15ce9e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:58,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408226575bc060ce15ce9e 2025-12-17 21:48:58,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408226575bc060ce15ce9e HTTP/1.1" 502 150 2025-12-17 21:48:58,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408226575bc060ce15ce9e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:58,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821f575bc060ce15cd37 2025-12-17 21:48:58,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821f575bc060ce15cd37 HTTP/1.1" 502 150 2025-12-17 21:48:58,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821f575bc060ce15cd37'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:58,986 - lava_callback - INFO - Received callback for node 6940823f575bc060ce15ced1 INFO: 172.21.0.1:50892 - "POST /node/6940823f575bc060ce15ced1 HTTP/1.0" 202 Accepted 2025-12-17 21:48:58,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408211575bc060ce15ccd6 2025-12-17 21:48:58,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408211575bc060ce15ccd6 HTTP/1.1" 502 150 2025-12-17 21:48:58,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408211575bc060ce15ccd6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:59,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821b575bc060ce15cd31 2025-12-17 21:48:59,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821b575bc060ce15cd31 HTTP/1.1" 502 150 2025-12-17 21:48:59,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821b575bc060ce15cd31'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:59,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408221575bc060ce15ce99 2025-12-17 21:48:59,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408221575bc060ce15ce99 HTTP/1.1" 502 150 2025-12-17 21:48:59,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408221575bc060ce15ce99'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:59,717 - lava_callback - INFO - Received callback for node 69408242575bc060ce15ced6 INFO: 172.21.0.1:47124 - "POST /node/69408242575bc060ce15ced6 HTTP/1.0" 202 Accepted 2025-12-17 21:48:59,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408203575bc060ce15ccac HTTP/1.1" 502 150 2025-12-17 21:48:59,952 - lava_callback - ERROR - Error processing callback for node 69408203575bc060ce15ccac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408203575bc060ce15ccac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408203575bc060ce15ccac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408203575bc060ce15ccac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:48:59,954 - lava_callback - INFO - Starting async job submit for node 6940822c575bc060ce15ceb8 2025-12-17 21:48:59,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:48:59,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940822c575bc060ce15ceb8 HTTP/1.1" 502 150 2025-12-17 21:48:59,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940822c575bc060ce15ceb8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:48:59,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940822c575bc060ce15ceb8 2025-12-17 21:48:59,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940822c575bc060ce15ceb8 HTTP/1.1" 502 150 2025-12-17 21:48:59,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940822c575bc060ce15ceb8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:00,467 - lava_callback - INFO - Received callback for node 69408245575bc060ce15ced9 INFO: 172.21.0.1:47140 - "POST /node/69408245575bc060ce15ced9 HTTP/1.0" 202 Accepted 2025-12-17 21:49:00,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408226575bc060ce15ce9e 2025-12-17 21:49:00,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408226575bc060ce15ce9e HTTP/1.1" 502 150 2025-12-17 21:49:00,501 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408226575bc060ce15ce9e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:01,202 - lava_callback - INFO - Received callback for node 69408248575bc060ce15cedf INFO: 172.21.0.1:47146 - "POST /node/69408248575bc060ce15cedf HTTP/1.0" 202 Accepted INFO: 172.21.0.1:47162 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:49:01,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940822c575bc060ce15ceb8 2025-12-17 21:49:01,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940822c575bc060ce15ceb8 HTTP/1.1" 502 150 2025-12-17 21:49:01,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940822c575bc060ce15ceb8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:01,990 - lava_callback - INFO - Received callback for node 6940824a575bc060ce15cee2 INFO: 172.21.0.1:47170 - "POST /node/6940824a575bc060ce15cee2 HTTP/1.0" 202 Accepted 2025-12-17 21:49:02,747 - lava_callback - INFO - Received callback for node 6940824b575bc060ce15cee5 INFO: 172.21.0.1:47182 - "POST /node/6940824b575bc060ce15cee5 HTTP/1.0" 202 Accepted 2025-12-17 21:49:02,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408202575bc060ce15cca9 2025-12-17 21:49:02,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408202575bc060ce15cca9 HTTP/1.1" 502 150 2025-12-17 21:49:02,873 - lava_callback - ERROR - Error processing callback for node 69408202575bc060ce15cca9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408202575bc060ce15cca9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408202575bc060ce15cca9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408202575bc060ce15cca9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:02,874 - lava_callback - INFO - Starting async job submit for node 69408480575bc060ce15d7d4 2025-12-17 21:49:02,893 - lava_callback - INFO - Received callback for node 69408246575bc060ce15cedc 2025-12-17 21:49:02,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:47188 - "POST /node/69408246575bc060ce15cedc HTTP/1.0" 202 Accepted 2025-12-17 21:49:02,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408480575bc060ce15d7d4 HTTP/1.1" 502 150 2025-12-17 21:49:02,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408480575bc060ce15d7d4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:02,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408480575bc060ce15d7d4 2025-12-17 21:49:02,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408480575bc060ce15d7d4 HTTP/1.1" 502 150 2025-12-17 21:49:02,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408480575bc060ce15d7d4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:02,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821a575bc060ce15cd2b 2025-12-17 21:49:02,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821a575bc060ce15cd2b HTTP/1.1" 502 150 2025-12-17 21:49:02,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821a575bc060ce15cd2b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:03,483 - lava_callback - INFO - Received callback for node 6940824c575bc060ce15cee8 INFO: 172.21.0.1:47202 - "POST /node/6940824c575bc060ce15cee8 HTTP/1.0" 202 Accepted 2025-12-17 21:49:03,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821b575bc060ce15cd31 2025-12-17 21:49:03,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821b575bc060ce15cd31 HTTP/1.1" 502 150 2025-12-17 21:49:03,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821b575bc060ce15cd31'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:03,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821b575bc060ce15cd2e 2025-12-17 21:49:03,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821b575bc060ce15cd2e HTTP/1.1" 502 150 2025-12-17 21:49:03,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821b575bc060ce15cd2e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:04,229 - lava_callback - INFO - Received callback for node 6940824d575bc060ce15ceeb INFO: 172.21.0.1:47206 - "POST /node/6940824d575bc060ce15ceeb HTTP/1.0" 202 Accepted 2025-12-17 21:49:04,501 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408226575bc060ce15ce9e 2025-12-17 21:49:04,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408226575bc060ce15ce9e HTTP/1.1" 502 150 2025-12-17 21:49:04,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408226575bc060ce15ce9e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:04,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408204575bc060ce15ccaf 2025-12-17 21:49:04,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408204575bc060ce15ccaf HTTP/1.1" 502 150 2025-12-17 21:49:04,893 - lava_callback - ERROR - Error processing callback for node 69408204575bc060ce15ccaf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408204575bc060ce15ccaf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408204575bc060ce15ccaf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408204575bc060ce15ccaf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:04,895 - lava_callback - INFO - Starting async job submit for node 69408228575bc060ce15cea6 2025-12-17 21:49:04,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:04,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408480575bc060ce15d7d4 2025-12-17 21:49:04,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408480575bc060ce15d7d4 HTTP/1.1" 502 150 2025-12-17 21:49:04,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408480575bc060ce15d7d4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:04,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408228575bc060ce15cea6 HTTP/1.1" 502 150 2025-12-17 21:49:04,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408228575bc060ce15cea6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:04,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408228575bc060ce15cea6 2025-12-17 21:49:04,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408228575bc060ce15cea6 HTTP/1.1" 502 150 2025-12-17 21:49:04,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408228575bc060ce15cea6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:05,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821d575bc060ce15cd34 2025-12-17 21:49:05,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408207575bc060ce15ccb5 2025-12-17 21:49:05,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408207575bc060ce15ccb5 HTTP/1.1" 502 150 2025-12-17 21:49:05,565 - lava_callback - ERROR - Error processing callback for node 69408207575bc060ce15ccb5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408207575bc060ce15ccb5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408207575bc060ce15ccb5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408207575bc060ce15ccb5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:05,568 - lava_callback - INFO - Starting async job submit for node 69408229575bc060ce15ceab 2025-12-17 21:49:05,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:05,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408229575bc060ce15ceab HTTP/1.1" 502 150 2025-12-17 21:49:05,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408229575bc060ce15ceab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:05,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408229575bc060ce15ceab 2025-12-17 21:49:05,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408229575bc060ce15ceab HTTP/1.1" 502 150 2025-12-17 21:49:05,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408229575bc060ce15ceab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:05,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940822c575bc060ce15ceb8 2025-12-17 21:49:05,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940822c575bc060ce15ceb8 HTTP/1.1" 502 150 2025-12-17 21:49:05,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940822c575bc060ce15ceb8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47214 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:49:06,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940820b575bc060ce15ccc2 2025-12-17 21:49:06,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940820b575bc060ce15ccc2 HTTP/1.1" 502 150 2025-12-17 21:49:06,904 - lava_callback - ERROR - Error processing callback for node 6940820b575bc060ce15ccc2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940820b575bc060ce15ccc2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940820b575bc060ce15ccc2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940820b575bc060ce15ccc2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:06,905 - lava_callback - INFO - Starting async job submit for node 6940822b575bc060ce15ceb2 2025-12-17 21:49:06,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:06,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940822b575bc060ce15ceb2 HTTP/1.1" 502 150 2025-12-17 21:49:06,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940822b575bc060ce15ceb2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:06,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940822b575bc060ce15ceb2 2025-12-17 21:49:06,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940822b575bc060ce15ceb2 HTTP/1.1" 502 150 2025-12-17 21:49:06,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940822b575bc060ce15ceb2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:06,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821f575bc060ce15cd37 2025-12-17 21:49:06,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821f575bc060ce15cd37 HTTP/1.1" 502 150 2025-12-17 21:49:06,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821f575bc060ce15cd37'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:06,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408228575bc060ce15cea6 2025-12-17 21:49:06,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408228575bc060ce15cea6 HTTP/1.1" 502 150 2025-12-17 21:49:06,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408228575bc060ce15cea6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:06,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408211575bc060ce15ccd6 2025-12-17 21:49:06,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408211575bc060ce15ccd6 HTTP/1.1" 502 150 2025-12-17 21:49:06,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408211575bc060ce15ccd6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:07,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408221575bc060ce15ce99 2025-12-17 21:49:07,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408221575bc060ce15ce99 HTTP/1.1" 502 150 2025-12-17 21:49:07,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408221575bc060ce15ce99'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:07,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408229575bc060ce15ceab 2025-12-17 21:49:07,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408229575bc060ce15ceab HTTP/1.1" 502 150 2025-12-17 21:49:07,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408229575bc060ce15ceab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:08,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408480575bc060ce15d7d4 2025-12-17 21:49:08,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408480575bc060ce15d7d4 HTTP/1.1" 502 150 2025-12-17 21:49:08,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408480575bc060ce15d7d4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:08,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940822b575bc060ce15ceb2 2025-12-17 21:49:08,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940822b575bc060ce15ceb2 HTTP/1.1" 502 150 2025-12-17 21:49:08,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940822b575bc060ce15ceb2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:09,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5311e3c24e95681d69c 2025-12-17 21:49:09,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5311e3c24e95681d69c HTTP/1.1" 502 150 2025-12-17 21:49:09,535 - lava_callback - ERROR - Error processing callback for node 6940c5311e3c24e95681d69c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5311e3c24e95681d69c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5311e3c24e95681d69c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5311e3c24e95681d69c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:09,537 - lava_callback - INFO - Starting async job submit for node 69408236575bc060ce15cec8 2025-12-17 21:49:09,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:09,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408236575bc060ce15cec8 HTTP/1.1" 502 150 2025-12-17 21:49:09,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408236575bc060ce15cec8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:09,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408236575bc060ce15cec8 2025-12-17 21:49:09,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408236575bc060ce15cec8 HTTP/1.1" 502 150 2025-12-17 21:49:09,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408236575bc060ce15cec8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:10,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408228575bc060ce15cea6 2025-12-17 21:49:10,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408228575bc060ce15cea6 HTTP/1.1" 502 150 2025-12-17 21:49:10,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408228575bc060ce15cea6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:11,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821b575bc060ce15cd31 2025-12-17 21:49:11,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821b575bc060ce15cd31 HTTP/1.1" 502 150 2025-12-17 21:49:11,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821b575bc060ce15cd31'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:11,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408236575bc060ce15cec8 2025-12-17 21:49:11,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408236575bc060ce15cec8 HTTP/1.1" 502 150 2025-12-17 21:49:11,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408236575bc060ce15cec8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:11,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408229575bc060ce15ceab 2025-12-17 21:49:11,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408229575bc060ce15ceab HTTP/1.1" 502 150 2025-12-17 21:49:11,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408229575bc060ce15ceab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:11,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408217575bc060ce15cd25 2025-12-17 21:49:11,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408217575bc060ce15cd25 HTTP/1.1" 502 150 2025-12-17 21:49:11,869 - lava_callback - ERROR - Error processing callback for node 69408217575bc060ce15cd25: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408217575bc060ce15cd25 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408217575bc060ce15cd25 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408217575bc060ce15cd25 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:11,870 - lava_callback - INFO - Starting async job submit for node 69408232575bc060ce15cec2 2025-12-17 21:49:11,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:11,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408232575bc060ce15cec2 HTTP/1.1" 502 150 2025-12-17 21:49:11,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408232575bc060ce15cec2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:11,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408232575bc060ce15cec2 2025-12-17 21:49:11,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408232575bc060ce15cec2 HTTP/1.1" 502 150 2025-12-17 21:49:11,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408232575bc060ce15cec2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:12,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408226575bc060ce15ce9e 2025-12-17 21:49:12,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408226575bc060ce15ce9e HTTP/1.1" 502 150 2025-12-17 21:49:12,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408226575bc060ce15ce9e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:12,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940822b575bc060ce15ceb2 2025-12-17 21:49:12,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940822b575bc060ce15ceb2 HTTP/1.1" 502 150 2025-12-17 21:49:12,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940822b575bc060ce15ceb2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:13,230 - lava_callback - INFO - Received callback for node 69408251575bc060ce15cf22 INFO: 172.21.0.1:39284 - "POST /node/69408251575bc060ce15cf22 HTTP/1.0" 202 Accepted 2025-12-17 21:49:13,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408232575bc060ce15cec2 2025-12-17 21:49:13,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408232575bc060ce15cec2 HTTP/1.1" 502 150 2025-12-17 21:49:13,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408232575bc060ce15cec2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:13,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408218575bc060ce15cd28 2025-12-17 21:49:13,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408218575bc060ce15cd28 HTTP/1.1" 502 150 2025-12-17 21:49:13,913 - lava_callback - ERROR - Error processing callback for node 69408218575bc060ce15cd28: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408218575bc060ce15cd28 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408218575bc060ce15cd28 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408218575bc060ce15cd28 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:13,915 - lava_callback - INFO - Starting async job submit for node 69408231575bc060ce15cebf 2025-12-17 21:49:13,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:13,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408231575bc060ce15cebf HTTP/1.1" 502 150 2025-12-17 21:49:13,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408231575bc060ce15cebf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:13,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408231575bc060ce15cebf 2025-12-17 21:49:13,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408231575bc060ce15cebf HTTP/1.1" 502 150 2025-12-17 21:49:13,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408231575bc060ce15cebf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:13,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940822c575bc060ce15ceb8 2025-12-17 21:49:13,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940822c575bc060ce15ceb8 HTTP/1.1" 502 150 2025-12-17 21:49:13,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940822c575bc060ce15ceb8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39296 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:49:15,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408236575bc060ce15cec8 2025-12-17 21:49:15,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408236575bc060ce15cec8 HTTP/1.1" 502 150 2025-12-17 21:49:15,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408236575bc060ce15cec8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:15,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408231575bc060ce15cebf 2025-12-17 21:49:15,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408231575bc060ce15cebf HTTP/1.1" 502 150 2025-12-17 21:49:15,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408231575bc060ce15cebf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39312 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:49:16,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408480575bc060ce15d7d4 2025-12-17 21:49:16,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408480575bc060ce15d7d4 HTTP/1.1" 502 150 2025-12-17 21:49:16,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408480575bc060ce15d7d4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:17,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408232575bc060ce15cec2 2025-12-17 21:49:17,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408232575bc060ce15cec2 HTTP/1.1" 502 150 2025-12-17 21:49:17,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408232575bc060ce15cec2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:18,087 - lava_callback - INFO - Received callback for node 6940824f575bc060ce15cef8 INFO: 172.21.0.1:39328 - "POST /node/6940824f575bc060ce15cef8 HTTP/1.0" 202 Accepted 2025-12-17 21:49:18,844 - lava_callback - INFO - Received callback for node 69408254575bc060ce15cf2b INFO: 172.21.0.1:39334 - "POST /node/69408254575bc060ce15cf2b HTTP/1.0" 202 Accepted 2025-12-17 21:49:18,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408228575bc060ce15cea6 2025-12-17 21:49:18,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408228575bc060ce15cea6 HTTP/1.1" 502 150 2025-12-17 21:49:18,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408228575bc060ce15cea6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:18,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821a575bc060ce15cd2b 2025-12-17 21:49:18,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821a575bc060ce15cd2b HTTP/1.1" 502 150 2025-12-17 21:49:18,963 - lava_callback - ERROR - Error processing callback for node 6940821a575bc060ce15cd2b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940821a575bc060ce15cd2b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940821a575bc060ce15cd2b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940821a575bc060ce15cd2b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:18,966 - lava_callback - INFO - Starting async job submit for node 69408234575bc060ce15cec5 2025-12-17 21:49:18,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:18,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408234575bc060ce15cec5 HTTP/1.1" 502 150 2025-12-17 21:49:18,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408234575bc060ce15cec5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:18,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408234575bc060ce15cec5 2025-12-17 21:49:18,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408234575bc060ce15cec5 HTTP/1.1" 502 150 2025-12-17 21:49:18,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408234575bc060ce15cec5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:19,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408229575bc060ce15ceab 2025-12-17 21:49:19,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408229575bc060ce15ceab HTTP/1.1" 502 150 2025-12-17 21:49:19,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408229575bc060ce15ceab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:19,679 - lava_callback - INFO - Received callback for node 69408256575bc060ce15cf2e INFO: 172.21.0.1:39340 - "POST /node/69408256575bc060ce15cf2e HTTP/1.0" 202 Accepted 2025-12-17 21:49:19,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821b575bc060ce15cd2e 2025-12-17 21:49:19,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821b575bc060ce15cd2e HTTP/1.1" 502 150 2025-12-17 21:49:19,686 - lava_callback - ERROR - Error processing callback for node 6940821b575bc060ce15cd2e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940821b575bc060ce15cd2e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940821b575bc060ce15cd2e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940821b575bc060ce15cd2e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:19,688 - lava_callback - INFO - Starting async job submit for node 69408237575bc060ce15cecb 2025-12-17 21:49:19,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:19,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408237575bc060ce15cecb HTTP/1.1" 502 150 2025-12-17 21:49:19,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408237575bc060ce15cecb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:19,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408237575bc060ce15cecb 2025-12-17 21:49:19,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408237575bc060ce15cecb HTTP/1.1" 502 150 2025-12-17 21:49:19,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408237575bc060ce15cecb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:19,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408231575bc060ce15cebf 2025-12-17 21:49:19,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408231575bc060ce15cebf HTTP/1.1" 502 150 2025-12-17 21:49:19,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408231575bc060ce15cebf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:20,436 - lava_callback - INFO - Received callback for node 69408258575bc060ce15cf31 INFO: 172.21.0.1:40278 - "POST /node/69408258575bc060ce15cf31 HTTP/1.0" 202 Accepted 2025-12-17 21:49:20,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940822b575bc060ce15ceb2 2025-12-17 21:49:20,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940822b575bc060ce15ceb2 HTTP/1.1" 502 150 2025-12-17 21:49:20,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940822b575bc060ce15ceb2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:20,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821d575bc060ce15cd34 HTTP/1.1" 502 150 2025-12-17 21:49:20,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940821d575bc060ce15cd34'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:20,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408234575bc060ce15cec5 2025-12-17 21:49:20,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408234575bc060ce15cec5 HTTP/1.1" 502 150 2025-12-17 21:49:20,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408234575bc060ce15cec5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:21,220 - lava_callback - INFO - Received callback for node 69408259575bc060ce15cf34 INFO: 172.21.0.1:40292 - "POST /node/69408259575bc060ce15cf34 HTTP/1.0" 202 Accepted 2025-12-17 21:49:21,242 - lava_callback - INFO - Received callback for node 694081af575bc060ce15cb82 INFO: 172.21.0.1:40306 - "POST /node/694081af575bc060ce15cb82 HTTP/1.0" 202 Accepted 2025-12-17 21:49:21,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408237575bc060ce15cecb 2025-12-17 21:49:21,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408237575bc060ce15cecb HTTP/1.1" 502 150 2025-12-17 21:49:21,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408237575bc060ce15cecb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:22,095 - lava_callback - INFO - Received callback for node 6940825b575bc060ce15cf37 INFO: 172.21.0.1:40314 - "POST /node/6940825b575bc060ce15cf37 HTTP/1.0" 202 Accepted 2025-12-17 21:49:22,848 - lava_callback - INFO - Received callback for node 69408261575bc060ce15cf43 INFO: 172.21.0.1:40330 - "POST /node/69408261575bc060ce15cf43 HTTP/1.0" 202 Accepted 2025-12-17 21:49:22,858 - lava_callback - INFO - Received callback for node 6940825c575bc060ce15cf3a INFO: 172.21.0.1:40342 - "POST /node/6940825c575bc060ce15cf3a HTTP/1.0" 202 Accepted 2025-12-17 21:49:22,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821f575bc060ce15cd37 2025-12-17 21:49:22,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821f575bc060ce15cd37 HTTP/1.1" 502 150 2025-12-17 21:49:22,926 - lava_callback - ERROR - Error processing callback for node 6940821f575bc060ce15cd37: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940821f575bc060ce15cd37 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940821f575bc060ce15cd37 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940821f575bc060ce15cd37 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:22,928 - lava_callback - INFO - Starting async job submit for node 69408239575bc060ce15cece 2025-12-17 21:49:22,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:22,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408211575bc060ce15ccd6 2025-12-17 21:49:22,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408211575bc060ce15ccd6 HTTP/1.1" 502 150 2025-12-17 21:49:22,996 - lava_callback - ERROR - Error processing callback for node 69408211575bc060ce15ccd6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408211575bc060ce15ccd6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408211575bc060ce15ccd6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408211575bc060ce15ccd6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:22,997 - lava_callback - INFO - Starting async job submit for node 6940823f575bc060ce15ced1 2025-12-17 21:49:22,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:23,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940823f575bc060ce15ced1 HTTP/1.1" 502 150 2025-12-17 21:49:23,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940823f575bc060ce15ced1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:23,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940823f575bc060ce15ced1 2025-12-17 21:49:23,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940823f575bc060ce15ced1 HTTP/1.1" 502 150 2025-12-17 21:49:23,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940823f575bc060ce15ced1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:23,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408236575bc060ce15cec8 2025-12-17 21:49:23,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408236575bc060ce15cec8 HTTP/1.1" 502 150 2025-12-17 21:49:23,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408236575bc060ce15cec8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:23,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408221575bc060ce15ce99 2025-12-17 21:49:23,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408221575bc060ce15ce99 HTTP/1.1" 502 150 2025-12-17 21:49:23,587 - lava_callback - ERROR - Error processing callback for node 69408221575bc060ce15ce99: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408221575bc060ce15ce99 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408221575bc060ce15ce99 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408221575bc060ce15ce99 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:23,590 - lava_callback - INFO - Starting async job submit for node 69408242575bc060ce15ced6 2025-12-17 21:49:23,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:23,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408242575bc060ce15ced6 HTTP/1.1" 502 150 2025-12-17 21:49:23,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408242575bc060ce15ced6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:23,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408242575bc060ce15ced6 2025-12-17 21:49:23,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408242575bc060ce15ced6 HTTP/1.1" 502 150 2025-12-17 21:49:23,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408242575bc060ce15ced6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:23,609 - lava_callback - INFO - Received callback for node 6940825e575bc060ce15cf3d INFO: 172.21.0.1:40346 - "POST /node/6940825e575bc060ce15cf3d HTTP/1.0" 202 Accepted 2025-12-17 21:49:24,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408234575bc060ce15cec5 2025-12-17 21:49:25,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408234575bc060ce15cec5 HTTP/1.1" 502 150 2025-12-17 21:49:25,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408234575bc060ce15cec5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:25,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940823f575bc060ce15ced1 2025-12-17 21:49:25,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408242575bc060ce15ced6 2025-12-17 21:49:25,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408242575bc060ce15ced6 HTTP/1.1" 502 150 2025-12-17 21:49:25,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408242575bc060ce15ced6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:25,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408237575bc060ce15cecb 2025-12-17 21:49:25,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408237575bc060ce15cecb HTTP/1.1" 502 150 2025-12-17 21:49:25,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408237575bc060ce15cecb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:25,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408232575bc060ce15cec2 2025-12-17 21:49:25,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408232575bc060ce15cec2 HTTP/1.1" 502 150 2025-12-17 21:49:25,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408232575bc060ce15cec2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40348 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:49:26,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408239575bc060ce15cece HTTP/1.1" 502 150 2025-12-17 21:49:26,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408239575bc060ce15cece'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:26,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408239575bc060ce15cece 2025-12-17 21:49:26,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408239575bc060ce15cece HTTP/1.1" 502 150 2025-12-17 21:49:26,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408239575bc060ce15cece'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:27,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821b575bc060ce15cd31 2025-12-17 21:49:27,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821b575bc060ce15cd31 HTTP/1.1" 502 150 2025-12-17 21:49:27,543 - lava_callback - ERROR - Error processing callback for node 6940821b575bc060ce15cd31: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940821b575bc060ce15cd31 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940821b575bc060ce15cd31 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940821b575bc060ce15cd31 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:27,545 - lava_callback - INFO - Starting async job submit for node 69408245575bc060ce15ced9 2025-12-17 21:49:27,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:27,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408245575bc060ce15ced9 HTTP/1.1" 502 150 2025-12-17 21:49:27,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408245575bc060ce15ced9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:27,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408245575bc060ce15ced9 2025-12-17 21:49:27,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408245575bc060ce15ced9 HTTP/1.1" 502 150 2025-12-17 21:49:27,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408245575bc060ce15ced9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:27,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408231575bc060ce15cebf 2025-12-17 21:49:27,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408231575bc060ce15cebf HTTP/1.1" 502 150 2025-12-17 21:49:27,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408231575bc060ce15cebf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:28,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408226575bc060ce15ce9e 2025-12-17 21:49:28,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408226575bc060ce15ce9e HTTP/1.1" 502 150 2025-12-17 21:49:28,505 - lava_callback - ERROR - Error processing callback for node 69408226575bc060ce15ce9e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408226575bc060ce15ce9e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408226575bc060ce15ce9e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408226575bc060ce15ce9e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:28,507 - lava_callback - INFO - Starting async job submit for node 69408248575bc060ce15cedf 2025-12-17 21:49:28,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:28,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408248575bc060ce15cedf HTTP/1.1" 502 150 2025-12-17 21:49:28,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408248575bc060ce15cedf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:28,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408248575bc060ce15cedf 2025-12-17 21:49:28,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408248575bc060ce15cedf HTTP/1.1" 502 150 2025-12-17 21:49:28,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408248575bc060ce15cedf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:28,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408239575bc060ce15cece 2025-12-17 21:49:28,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408239575bc060ce15cece HTTP/1.1" 502 150 2025-12-17 21:49:28,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408239575bc060ce15cece'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:29,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408245575bc060ce15ced9 2025-12-17 21:49:29,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408245575bc060ce15ced9 HTTP/1.1" 502 150 2025-12-17 21:49:29,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408245575bc060ce15ced9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:29,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408242575bc060ce15ced6 2025-12-17 21:49:29,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408242575bc060ce15ced6 HTTP/1.1" 502 150 2025-12-17 21:49:29,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408242575bc060ce15ced6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:29,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940822c575bc060ce15ceb8 2025-12-17 21:49:29,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940822c575bc060ce15ceb8 HTTP/1.1" 502 150 2025-12-17 21:49:29,969 - lava_callback - ERROR - Error processing callback for node 6940822c575bc060ce15ceb8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940822c575bc060ce15ceb8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940822c575bc060ce15ceb8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940822c575bc060ce15ceb8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:29,970 - lava_callback - INFO - Starting async job submit for node 6940824a575bc060ce15cee2 2025-12-17 21:49:29,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:30,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824a575bc060ce15cee2 HTTP/1.1" 502 150 2025-12-17 21:49:30,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824a575bc060ce15cee2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:30,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824a575bc060ce15cee2 2025-12-17 21:49:30,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824a575bc060ce15cee2 HTTP/1.1" 502 150 2025-12-17 21:49:30,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824a575bc060ce15cee2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:30,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408248575bc060ce15cedf 2025-12-17 21:49:30,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408248575bc060ce15cedf HTTP/1.1" 502 150 2025-12-17 21:49:30,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408248575bc060ce15cedf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:32,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824a575bc060ce15cee2 2025-12-17 21:49:32,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408480575bc060ce15d7d4 2025-12-17 21:49:32,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408480575bc060ce15d7d4 HTTP/1.1" 502 150 2025-12-17 21:49:32,915 - lava_callback - ERROR - Error processing callback for node 69408480575bc060ce15d7d4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408480575bc060ce15d7d4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408480575bc060ce15d7d4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408480575bc060ce15d7d4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:32,917 - lava_callback - INFO - Starting async job submit for node 6940824b575bc060ce15cee5 2025-12-17 21:49:32,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:32,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824b575bc060ce15cee5 HTTP/1.1" 502 150 2025-12-17 21:49:32,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824b575bc060ce15cee5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:32,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824b575bc060ce15cee5 2025-12-17 21:49:32,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824b575bc060ce15cee5 HTTP/1.1" 502 150 2025-12-17 21:49:32,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824b575bc060ce15cee5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:32,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408239575bc060ce15cece 2025-12-17 21:49:32,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408239575bc060ce15cece HTTP/1.1" 502 150 2025-12-17 21:49:32,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408239575bc060ce15cece'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:33,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408234575bc060ce15cec5 2025-12-17 21:49:33,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408234575bc060ce15cec5 HTTP/1.1" 502 150 2025-12-17 21:49:33,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408234575bc060ce15cec5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:33,218 - lava_callback - INFO - Received callback for node 69408264575bc060ce15cf4f INFO: 172.21.0.1:54734 - "POST /node/69408264575bc060ce15cf4f HTTP/1.0" 202 Accepted 2025-12-17 21:49:33,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408245575bc060ce15ced9 2025-12-17 21:49:33,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408245575bc060ce15ced9 HTTP/1.1" 502 150 2025-12-17 21:49:33,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408245575bc060ce15ced9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:33,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408237575bc060ce15cecb 2025-12-17 21:49:33,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408237575bc060ce15cecb HTTP/1.1" 502 150 2025-12-17 21:49:33,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408237575bc060ce15cecb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:33,869 - lava_callback - INFO - Received callback for node 69408267575bc060ce15cf5f INFO: 172.21.0.1:54744 - "POST /node/69408267575bc060ce15cf5f HTTP/1.0" 202 Accepted 2025-12-17 21:49:34,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408248575bc060ce15cedf 2025-12-17 21:49:34,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408248575bc060ce15cedf HTTP/1.1" 502 150 2025-12-17 21:49:34,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408248575bc060ce15cedf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:34,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824b575bc060ce15cee5 2025-12-17 21:49:34,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824b575bc060ce15cee5 HTTP/1.1" 502 150 2025-12-17 21:49:34,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824b575bc060ce15cee5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:34,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408228575bc060ce15cea6 2025-12-17 21:49:34,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408228575bc060ce15cea6 HTTP/1.1" 502 150 2025-12-17 21:49:34,941 - lava_callback - ERROR - Error processing callback for node 69408228575bc060ce15cea6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408228575bc060ce15cea6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408228575bc060ce15cea6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408228575bc060ce15cea6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:34,943 - lava_callback - INFO - Starting async job submit for node 69408246575bc060ce15cedc 2025-12-17 21:49:34,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:34,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408246575bc060ce15cedc HTTP/1.1" 502 150 2025-12-17 21:49:34,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408246575bc060ce15cedc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:34,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408246575bc060ce15cedc 2025-12-17 21:49:34,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408246575bc060ce15cedc HTTP/1.1" 502 150 2025-12-17 21:49:34,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408246575bc060ce15cedc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:35,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408229575bc060ce15ceab 2025-12-17 21:49:35,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408229575bc060ce15ceab HTTP/1.1" 502 150 2025-12-17 21:49:35,608 - lava_callback - ERROR - Error processing callback for node 69408229575bc060ce15ceab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408229575bc060ce15ceab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408229575bc060ce15ceab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408229575bc060ce15ceab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:35,610 - lava_callback - INFO - Starting async job submit for node 6940824c575bc060ce15cee8 2025-12-17 21:49:35,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:35,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824c575bc060ce15cee8 HTTP/1.1" 502 150 2025-12-17 21:49:35,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824c575bc060ce15cee8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:35,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824c575bc060ce15cee8 2025-12-17 21:49:35,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824c575bc060ce15cee8 HTTP/1.1" 502 150 2025-12-17 21:49:35,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824c575bc060ce15cee8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54752 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:49:36,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940822b575bc060ce15ceb2 2025-12-17 21:49:36,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940822b575bc060ce15ceb2 HTTP/1.1" 502 150 2025-12-17 21:49:36,921 - lava_callback - ERROR - Error processing callback for node 6940822b575bc060ce15ceb2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940822b575bc060ce15ceb2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940822b575bc060ce15ceb2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940822b575bc060ce15ceb2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:36,924 - lava_callback - INFO - Starting async job submit for node 6940824d575bc060ce15ceeb 2025-12-17 21:49:36,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:36,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824d575bc060ce15ceeb HTTP/1.1" 502 150 2025-12-17 21:49:36,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824d575bc060ce15ceeb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:36,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824d575bc060ce15ceeb 2025-12-17 21:49:36,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824d575bc060ce15ceeb HTTP/1.1" 502 150 2025-12-17 21:49:36,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824d575bc060ce15ceeb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:36,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940821d575bc060ce15cd34 2025-12-17 21:49:36,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940821d575bc060ce15cd34 HTTP/1.1" 502 150 2025-12-17 21:49:36,946 - lava_callback - ERROR - Error processing callback for node 6940821d575bc060ce15cd34: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940821d575bc060ce15cd34 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940821d575bc060ce15cd34 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940821d575bc060ce15cd34 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:36,948 - lava_callback - INFO - Starting async job submit for node 69408251575bc060ce15cf22 2025-12-17 21:49:36,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:36,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408246575bc060ce15cedc 2025-12-17 21:49:36,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408246575bc060ce15cedc HTTP/1.1" 502 150 2025-12-17 21:49:36,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408246575bc060ce15cedc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:36,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408251575bc060ce15cf22 HTTP/1.1" 502 150 2025-12-17 21:49:36,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408251575bc060ce15cf22'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:36,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408251575bc060ce15cf22 2025-12-17 21:49:36,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408251575bc060ce15cf22 HTTP/1.1" 502 150 2025-12-17 21:49:36,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408251575bc060ce15cf22'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:37,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408242575bc060ce15ced6 2025-12-17 21:49:37,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408242575bc060ce15ced6 HTTP/1.1" 502 150 2025-12-17 21:49:37,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408242575bc060ce15ced6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:37,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824c575bc060ce15cee8 2025-12-17 21:49:37,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824c575bc060ce15cee8 HTTP/1.1" 502 150 2025-12-17 21:49:37,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824c575bc060ce15cee8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:38,626 - lava_callback - INFO - Received callback for node 6940826b575bc060ce15cf6f INFO: 172.21.0.1:54758 - "POST /node/6940826b575bc060ce15cf6f HTTP/1.0" 202 Accepted 2025-12-17 21:49:38,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824b575bc060ce15cee5 2025-12-17 21:49:38,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824b575bc060ce15cee5 HTTP/1.1" 502 150 2025-12-17 21:49:38,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824b575bc060ce15cee5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:38,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824d575bc060ce15ceeb 2025-12-17 21:49:38,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824d575bc060ce15ceeb HTTP/1.1" 502 150 2025-12-17 21:49:38,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824d575bc060ce15ceeb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:38,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408251575bc060ce15cf22 2025-12-17 21:49:38,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408251575bc060ce15cf22 HTTP/1.1" 502 150 2025-12-17 21:49:38,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408251575bc060ce15cf22'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:39,413 - lava_callback - INFO - Received callback for node 6940826d575bc060ce15cf76 INFO: 172.21.0.1:54764 - "POST /node/6940826d575bc060ce15cf76 HTTP/1.0" 202 Accepted 2025-12-17 21:49:39,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408236575bc060ce15cec8 2025-12-17 21:49:39,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408236575bc060ce15cec8 HTTP/1.1" 502 150 2025-12-17 21:49:39,552 - lava_callback - ERROR - Error processing callback for node 69408236575bc060ce15cec8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408236575bc060ce15cec8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408236575bc060ce15cec8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408236575bc060ce15cec8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:39,554 - lava_callback - INFO - Starting async job submit for node 6940824f575bc060ce15cef8 2025-12-17 21:49:39,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:39,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824f575bc060ce15cef8 HTTP/1.1" 502 150 2025-12-17 21:49:39,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824f575bc060ce15cef8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:39,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824f575bc060ce15cef8 2025-12-17 21:49:39,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824f575bc060ce15cef8 HTTP/1.1" 502 150 2025-12-17 21:49:39,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824f575bc060ce15cef8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:40,351 - lava_callback - INFO - Received callback for node 6940826e575bc060ce15cf79 INFO: 172.21.0.1:58894 - "POST /node/6940826e575bc060ce15cf79 HTTP/1.0" 202 Accepted 2025-12-17 21:49:40,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940823f575bc060ce15ced1 HTTP/1.1" 502 150 2025-12-17 21:49:40,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940823f575bc060ce15ced1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:40,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408239575bc060ce15cece 2025-12-17 21:49:40,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408239575bc060ce15cece HTTP/1.1" 502 150 2025-12-17 21:49:40,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408239575bc060ce15cece'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:40,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408246575bc060ce15cedc 2025-12-17 21:49:40,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408246575bc060ce15cedc HTTP/1.1" 502 150 2025-12-17 21:49:40,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408246575bc060ce15cedc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:41,477 - lava_callback - INFO - Received callback for node 694081b1575bc060ce15cb87 INFO: 172.21.0.1:58902 - "POST /node/694081b1575bc060ce15cb87 HTTP/1.0" 202 Accepted 2025-12-17 21:49:41,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824f575bc060ce15cef8 2025-12-17 21:49:41,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824f575bc060ce15cef8 HTTP/1.1" 502 150 2025-12-17 21:49:41,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824f575bc060ce15cef8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:41,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408245575bc060ce15ced9 2025-12-17 21:49:41,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408245575bc060ce15ced9 HTTP/1.1" 502 150 2025-12-17 21:49:41,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408245575bc060ce15ced9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:41,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824c575bc060ce15cee8 2025-12-17 21:49:41,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824c575bc060ce15cee8 HTTP/1.1" 502 150 2025-12-17 21:49:41,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824c575bc060ce15cee8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:41,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408232575bc060ce15cec2 2025-12-17 21:49:41,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408232575bc060ce15cec2 HTTP/1.1" 502 150 2025-12-17 21:49:41,889 - lava_callback - ERROR - Error processing callback for node 69408232575bc060ce15cec2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408232575bc060ce15cec2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408232575bc060ce15cec2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408232575bc060ce15cec2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:41,892 - lava_callback - INFO - Starting async job submit for node 69408254575bc060ce15cf2b 2025-12-17 21:49:41,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:41,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408254575bc060ce15cf2b HTTP/1.1" 502 150 2025-12-17 21:49:41,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408254575bc060ce15cf2b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:41,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408254575bc060ce15cf2b 2025-12-17 21:49:41,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408254575bc060ce15cf2b HTTP/1.1" 502 150 2025-12-17 21:49:41,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408254575bc060ce15cf2b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:42,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408248575bc060ce15cedf 2025-12-17 21:49:42,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408248575bc060ce15cedf HTTP/1.1" 502 150 2025-12-17 21:49:42,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408248575bc060ce15cedf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:42,922 - lava_callback - INFO - Received callback for node 69408262575bc060ce15cf47 INFO: 172.21.0.1:58910 - "POST /node/69408262575bc060ce15cf47 HTTP/1.0" 202 Accepted 2025-12-17 21:49:42,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824d575bc060ce15ceeb 2025-12-17 21:49:42,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824d575bc060ce15ceeb HTTP/1.1" 502 150 2025-12-17 21:49:42,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824d575bc060ce15ceeb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:42,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408251575bc060ce15cf22 2025-12-17 21:49:42,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408251575bc060ce15cf22 HTTP/1.1" 502 150 2025-12-17 21:49:42,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408251575bc060ce15cf22'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:43,578 - lava_callback - INFO - Received callback for node 69408266575bc060ce15cf58 INFO: 172.21.0.1:58920 - "POST /node/69408266575bc060ce15cf58 HTTP/1.0" 202 Accepted 2025-12-17 21:49:43,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408254575bc060ce15cf2b 2025-12-17 21:49:43,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408254575bc060ce15cf2b HTTP/1.1" 502 150 2025-12-17 21:49:43,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408254575bc060ce15cf2b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:43,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408231575bc060ce15cebf 2025-12-17 21:49:43,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408231575bc060ce15cebf HTTP/1.1" 502 150 2025-12-17 21:49:43,928 - lava_callback - ERROR - Error processing callback for node 69408231575bc060ce15cebf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408231575bc060ce15cebf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408231575bc060ce15cebf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408231575bc060ce15cebf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:43,930 - lava_callback - INFO - Starting async job submit for node 69408256575bc060ce15cf2e 2025-12-17 21:49:43,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:43,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408256575bc060ce15cf2e HTTP/1.1" 502 150 2025-12-17 21:49:43,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408256575bc060ce15cf2e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:43,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408256575bc060ce15cf2e 2025-12-17 21:49:43,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408256575bc060ce15cf2e HTTP/1.1" 502 150 2025-12-17 21:49:43,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408256575bc060ce15cf2e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:44,245 - lava_callback - INFO - Received callback for node 69408271575bc060ce15cf82 INFO: 172.21.0.1:58928 - "POST /node/69408271575bc060ce15cf82 HTTP/1.0" 202 Accepted 2025-12-17 21:49:44,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940823f575bc060ce15ced1 2025-12-17 21:49:44,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940823f575bc060ce15ced1 HTTP/1.1" 502 150 2025-12-17 21:49:44,405 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940823f575bc060ce15ced1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:44,917 - lava_callback - INFO - Received callback for node 69408273575bc060ce15cf89 INFO: 172.21.0.1:58934 - "POST /node/69408273575bc060ce15cf89 HTTP/1.0" 202 Accepted 2025-12-17 21:49:45,568 - lava_callback - INFO - Received callback for node 69408274575bc060ce15cf8c INFO: 172.21.0.1:58944 - "POST /node/69408274575bc060ce15cf8c HTTP/1.0" 202 Accepted 2025-12-17 21:49:45,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824f575bc060ce15cef8 2025-12-17 21:49:45,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824f575bc060ce15cef8 HTTP/1.1" 502 150 2025-12-17 21:49:45,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824f575bc060ce15cef8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:45,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408256575bc060ce15cf2e 2025-12-17 21:49:45,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408256575bc060ce15cf2e HTTP/1.1" 502 150 2025-12-17 21:49:45,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408256575bc060ce15cf2e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58950 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:49:46,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824b575bc060ce15cee5 2025-12-17 21:49:46,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824b575bc060ce15cee5 HTTP/1.1" 502 150 2025-12-17 21:49:46,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824b575bc060ce15cee5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:47,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824a575bc060ce15cee2 HTTP/1.1" 502 150 2025-12-17 21:49:47,312 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824a575bc060ce15cee2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:47,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408254575bc060ce15cf2b 2025-12-17 21:49:47,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408254575bc060ce15cf2b HTTP/1.1" 502 150 2025-12-17 21:49:47,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408254575bc060ce15cf2b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:48,213 - lava_callback - INFO - Received callback for node 6940826f575bc060ce15cf7c INFO: 172.21.0.1:58954 - "POST /node/6940826f575bc060ce15cf7c HTTP/1.0" 202 Accepted 2025-12-17 21:49:48,941 - lava_callback - INFO - Received callback for node 69408270575bc060ce15cf7f INFO: 172.21.0.1:58968 - "POST /node/69408270575bc060ce15cf7f HTTP/1.0" 202 Accepted 2025-12-17 21:49:48,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408246575bc060ce15cedc 2025-12-17 21:49:48,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408246575bc060ce15cedc HTTP/1.1" 502 150 2025-12-17 21:49:48,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408246575bc060ce15cedc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:49,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408234575bc060ce15cec5 2025-12-17 21:49:49,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408234575bc060ce15cec5 HTTP/1.1" 502 150 2025-12-17 21:49:49,002 - lava_callback - ERROR - Error processing callback for node 69408234575bc060ce15cec5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408234575bc060ce15cec5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408234575bc060ce15cec5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408234575bc060ce15cec5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:49,004 - lava_callback - INFO - Starting async job submit for node 69408258575bc060ce15cf31 2025-12-17 21:49:49,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:49,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408258575bc060ce15cf31 HTTP/1.1" 502 150 2025-12-17 21:49:49,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408258575bc060ce15cf31'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:49,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408258575bc060ce15cf31 2025-12-17 21:49:49,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408258575bc060ce15cf31 HTTP/1.1" 502 150 2025-12-17 21:49:49,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408258575bc060ce15cf31'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:49,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824c575bc060ce15cee8 2025-12-17 21:49:49,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824c575bc060ce15cee8 HTTP/1.1" 502 150 2025-12-17 21:49:49,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824c575bc060ce15cee8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:49,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408237575bc060ce15cecb 2025-12-17 21:49:49,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408237575bc060ce15cecb HTTP/1.1" 502 150 2025-12-17 21:49:49,705 - lava_callback - ERROR - Error processing callback for node 69408237575bc060ce15cecb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408237575bc060ce15cecb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408237575bc060ce15cecb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408237575bc060ce15cecb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:49,707 - lava_callback - INFO - Starting async job submit for node 69408259575bc060ce15cf34 2025-12-17 21:49:49,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:49,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408259575bc060ce15cf34 HTTP/1.1" 502 150 2025-12-17 21:49:49,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408259575bc060ce15cf34'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:49,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408259575bc060ce15cf34 2025-12-17 21:49:49,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408259575bc060ce15cf34 HTTP/1.1" 502 150 2025-12-17 21:49:49,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408259575bc060ce15cf34'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:49,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408256575bc060ce15cf2e 2025-12-17 21:49:49,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408256575bc060ce15cf2e HTTP/1.1" 502 150 2025-12-17 21:49:49,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408256575bc060ce15cf2e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:50,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824d575bc060ce15ceeb 2025-12-17 21:49:50,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824d575bc060ce15ceeb HTTP/1.1" 502 150 2025-12-17 21:49:50,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824d575bc060ce15ceeb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:50,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408251575bc060ce15cf22 2025-12-17 21:49:50,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408251575bc060ce15cf22 HTTP/1.1" 502 150 2025-12-17 21:49:50,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408251575bc060ce15cf22'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:51,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408258575bc060ce15cf31 2025-12-17 21:49:51,313 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824a575bc060ce15cee2 2025-12-17 21:49:51,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824a575bc060ce15cee2 HTTP/1.1" 502 150 2025-12-17 21:49:51,314 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824a575bc060ce15cee2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:51,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408259575bc060ce15cf34 2025-12-17 21:49:51,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408259575bc060ce15cf34 HTTP/1.1" 502 150 2025-12-17 21:49:51,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408259575bc060ce15cf34'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:52,405 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940823f575bc060ce15ced1 2025-12-17 21:49:52,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940823f575bc060ce15ced1 HTTP/1.1" 502 150 2025-12-17 21:49:52,407 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940823f575bc060ce15ced1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:53,265 - lava_callback - INFO - Received callback for node 69408276575bc060ce15cf8f INFO: 172.21.0.1:34950 - "POST /node/69408276575bc060ce15cf8f HTTP/1.0" 202 Accepted 2025-12-17 21:49:53,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824f575bc060ce15cef8 2025-12-17 21:49:53,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824f575bc060ce15cef8 HTTP/1.1" 502 150 2025-12-17 21:49:53,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824f575bc060ce15cef8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:53,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408242575bc060ce15ced6 2025-12-17 21:49:53,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408242575bc060ce15ced6 HTTP/1.1" 502 150 2025-12-17 21:49:53,603 - lava_callback - ERROR - Error processing callback for node 69408242575bc060ce15ced6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408242575bc060ce15ced6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408242575bc060ce15ced6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408242575bc060ce15ced6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:53,605 - lava_callback - INFO - Starting async job submit for node 694081af575bc060ce15cb82 2025-12-17 21:49:53,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:53,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081af575bc060ce15cb82 HTTP/1.1" 502 150 2025-12-17 21:49:53,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081af575bc060ce15cb82'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:53,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081af575bc060ce15cb82 2025-12-17 21:49:53,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081af575bc060ce15cb82 HTTP/1.1" 502 150 2025-12-17 21:49:53,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081af575bc060ce15cb82'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:53,913 - lava_callback - INFO - Received callback for node 69408277575bc060ce15cf94 INFO: 172.21.0.1:34966 - "POST /node/69408277575bc060ce15cf94 HTTP/1.0" 202 Accepted 2025-12-17 21:49:54,589 - lava_callback - INFO - Received callback for node 69408278575bc060ce15cf9b INFO: 172.21.0.1:34976 - "POST /node/69408278575bc060ce15cf9b HTTP/1.0" 202 Accepted 2025-12-17 21:49:55,281 - lava_callback - INFO - Received callback for node 6940827a575bc060ce15cfa2 INFO: 172.21.0.1:34984 - "POST /node/6940827a575bc060ce15cfa2 HTTP/1.0" 202 Accepted 2025-12-17 21:49:55,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081af575bc060ce15cb82 2025-12-17 21:49:55,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081af575bc060ce15cb82 HTTP/1.1" 502 150 2025-12-17 21:49:55,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081af575bc060ce15cb82'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:55,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408259575bc060ce15cf34 2025-12-17 21:49:55,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408259575bc060ce15cf34 HTTP/1.1" 502 150 2025-12-17 21:49:55,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408259575bc060ce15cf34'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:55,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408254575bc060ce15cf2b 2025-12-17 21:49:55,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408254575bc060ce15cf2b HTTP/1.1" 502 150 2025-12-17 21:49:55,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408254575bc060ce15cf2b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:55,955 - lava_callback - INFO - Received callback for node 6940827b575bc060ce15cfa9 INFO: 172.21.0.1:34998 - "POST /node/6940827b575bc060ce15cfa9 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:35012 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:49:56,636 - lava_callback - INFO - Received callback for node 6940827c575bc060ce15cfb0 INFO: 172.21.0.1:35020 - "POST /node/6940827c575bc060ce15cfb0 HTTP/1.0" 202 Accepted 2025-12-17 21:49:56,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408239575bc060ce15cece 2025-12-17 21:49:56,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408239575bc060ce15cece HTTP/1.1" 502 150 2025-12-17 21:49:56,947 - lava_callback - ERROR - Error processing callback for node 69408239575bc060ce15cece: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408239575bc060ce15cece (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408239575bc060ce15cece (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408239575bc060ce15cece (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:56,949 - lava_callback - INFO - Starting async job submit for node 6940825b575bc060ce15cf37 2025-12-17 21:49:56,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:56,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825b575bc060ce15cf37 HTTP/1.1" 502 150 2025-12-17 21:49:56,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940825b575bc060ce15cf37'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:56,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940825b575bc060ce15cf37 2025-12-17 21:49:56,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825b575bc060ce15cf37 HTTP/1.1" 502 150 2025-12-17 21:49:56,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940825b575bc060ce15cf37'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:57,314 - lava_callback - INFO - Received callback for node 6940827d575bc060ce15cfb7 INFO: 172.21.0.1:35036 - "POST /node/6940827d575bc060ce15cfb7 HTTP/1.0" 202 Accepted 2025-12-17 21:49:57,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408245575bc060ce15ced9 2025-12-17 21:49:57,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408245575bc060ce15ced9 HTTP/1.1" 502 150 2025-12-17 21:49:57,583 - lava_callback - ERROR - Error processing callback for node 69408245575bc060ce15ced9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408245575bc060ce15ced9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408245575bc060ce15ced9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408245575bc060ce15ced9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:57,586 - lava_callback - INFO - Starting async job submit for node 69408261575bc060ce15cf43 2025-12-17 21:49:57,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:57,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408261575bc060ce15cf43 HTTP/1.1" 502 150 2025-12-17 21:49:57,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408261575bc060ce15cf43'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:57,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408261575bc060ce15cf43 2025-12-17 21:49:57,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408261575bc060ce15cf43 HTTP/1.1" 502 150 2025-12-17 21:49:57,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408261575bc060ce15cf43'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:57,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408256575bc060ce15cf2e 2025-12-17 21:49:57,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408256575bc060ce15cf2e HTTP/1.1" 502 150 2025-12-17 21:49:57,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408256575bc060ce15cf2e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:57,994 - lava_callback - INFO - Received callback for node 6940827f575bc060ce15cfba INFO: 172.21.0.1:35044 - "POST /node/6940827f575bc060ce15cfba HTTP/1.0" 202 Accepted 2025-12-17 21:49:58,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408248575bc060ce15cedf 2025-12-17 21:49:58,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408248575bc060ce15cedf HTTP/1.1" 502 150 2025-12-17 21:49:58,521 - lava_callback - ERROR - Error processing callback for node 69408248575bc060ce15cedf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408248575bc060ce15cedf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408248575bc060ce15cedf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408248575bc060ce15cedf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:49:58,523 - lava_callback - INFO - Starting async job submit for node 6940825c575bc060ce15cf3a 2025-12-17 21:49:58,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:49:58,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825c575bc060ce15cf3a HTTP/1.1" 502 150 2025-12-17 21:49:58,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940825c575bc060ce15cf3a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:58,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940825c575bc060ce15cf3a 2025-12-17 21:49:58,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825c575bc060ce15cf3a HTTP/1.1" 502 150 2025-12-17 21:49:58,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940825c575bc060ce15cf3a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:58,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940825b575bc060ce15cf37 2025-12-17 21:49:58,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825b575bc060ce15cf37 HTTP/1.1" 502 150 2025-12-17 21:49:58,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940825b575bc060ce15cf37'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:59,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824a575bc060ce15cee2 2025-12-17 21:49:59,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824a575bc060ce15cee2 HTTP/1.1" 502 150 2025-12-17 21:49:59,315 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940824a575bc060ce15cee2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:59,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408261575bc060ce15cf43 2025-12-17 21:49:59,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408261575bc060ce15cf43 HTTP/1.1" 502 150 2025-12-17 21:49:59,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408261575bc060ce15cf43'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:49:59,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081af575bc060ce15cb82 2025-12-17 21:49:59,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081af575bc060ce15cb82 HTTP/1.1" 502 150 2025-12-17 21:49:59,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081af575bc060ce15cb82'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:00,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940825c575bc060ce15cf3a 2025-12-17 21:50:00,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825c575bc060ce15cf3a HTTP/1.1" 502 150 2025-12-17 21:50:00,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940825c575bc060ce15cf3a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53594 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:50:02,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824b575bc060ce15cee5 2025-12-17 21:50:02,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940825b575bc060ce15cf37 2025-12-17 21:50:02,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825b575bc060ce15cf37 HTTP/1.1" 502 150 2025-12-17 21:50:02,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940825b575bc060ce15cf37'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:03,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408261575bc060ce15cf43 2025-12-17 21:50:03,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408261575bc060ce15cf43 HTTP/1.1" 502 150 2025-12-17 21:50:03,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408261575bc060ce15cf43'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:03,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408259575bc060ce15cf34 2025-12-17 21:50:03,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408259575bc060ce15cf34 HTTP/1.1" 502 150 2025-12-17 21:50:03,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408259575bc060ce15cf34'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:04,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940825c575bc060ce15cf3a 2025-12-17 21:50:04,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825c575bc060ce15cf3a HTTP/1.1" 502 150 2025-12-17 21:50:04,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940825c575bc060ce15cf3a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:04,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408246575bc060ce15cedc 2025-12-17 21:50:04,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408246575bc060ce15cedc HTTP/1.1" 502 150 2025-12-17 21:50:04,957 - lava_callback - ERROR - Error processing callback for node 69408246575bc060ce15cedc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408246575bc060ce15cedc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408246575bc060ce15cedc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408246575bc060ce15cedc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:04,959 - lava_callback - INFO - Starting async job submit for node 6940825e575bc060ce15cf3d 2025-12-17 21:50:04,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:04,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825e575bc060ce15cf3d HTTP/1.1" 502 150 2025-12-17 21:50:04,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940825e575bc060ce15cf3d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:04,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940825e575bc060ce15cf3d 2025-12-17 21:50:04,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825e575bc060ce15cf3d HTTP/1.1" 502 150 2025-12-17 21:50:04,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940825e575bc060ce15cf3d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:05,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824c575bc060ce15cee8 2025-12-17 21:50:05,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824c575bc060ce15cee8 HTTP/1.1" 502 150 2025-12-17 21:50:05,624 - lava_callback - ERROR - Error processing callback for node 6940824c575bc060ce15cee8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940824c575bc060ce15cee8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940824c575bc060ce15cee8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940824c575bc060ce15cee8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:05,626 - lava_callback - INFO - Starting async job submit for node 69408264575bc060ce15cf4f 2025-12-17 21:50:05,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:05,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408264575bc060ce15cf4f HTTP/1.1" 502 150 2025-12-17 21:50:05,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408264575bc060ce15cf4f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:05,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408264575bc060ce15cf4f 2025-12-17 21:50:05,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408264575bc060ce15cf4f HTTP/1.1" 502 150 2025-12-17 21:50:05,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408264575bc060ce15cf4f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:06,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408258575bc060ce15cf31 HTTP/1.1" 502 150 2025-12-17 21:50:06,257 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408258575bc060ce15cf31'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53600 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:50:06,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824d575bc060ce15ceeb 2025-12-17 21:50:06,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824d575bc060ce15ceeb HTTP/1.1" 502 150 2025-12-17 21:50:06,941 - lava_callback - ERROR - Error processing callback for node 6940824d575bc060ce15ceeb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940824d575bc060ce15ceeb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940824d575bc060ce15ceeb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940824d575bc060ce15ceeb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:06,943 - lava_callback - INFO - Starting async job submit for node 69408267575bc060ce15cf5f 2025-12-17 21:50:06,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:06,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408267575bc060ce15cf5f HTTP/1.1" 502 150 2025-12-17 21:50:06,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408267575bc060ce15cf5f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:06,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408267575bc060ce15cf5f 2025-12-17 21:50:06,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408267575bc060ce15cf5f HTTP/1.1" 502 150 2025-12-17 21:50:06,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408267575bc060ce15cf5f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:06,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408251575bc060ce15cf22 2025-12-17 21:50:06,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408251575bc060ce15cf22 HTTP/1.1" 502 150 2025-12-17 21:50:06,960 - lava_callback - ERROR - Error processing callback for node 69408251575bc060ce15cf22: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408251575bc060ce15cf22 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408251575bc060ce15cf22 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408251575bc060ce15cf22 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:06,961 - lava_callback - INFO - Starting async job submit for node 6940826b575bc060ce15cf6f 2025-12-17 21:50:06,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:06,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940825e575bc060ce15cf3d 2025-12-17 21:50:06,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826b575bc060ce15cf6f HTTP/1.1" 502 150 2025-12-17 21:50:06,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826b575bc060ce15cf6f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:06,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826b575bc060ce15cf6f 2025-12-17 21:50:06,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825e575bc060ce15cf3d HTTP/1.1" 502 150 2025-12-17 21:50:06,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940825e575bc060ce15cf3d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:06,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826b575bc060ce15cf6f HTTP/1.1" 502 150 2025-12-17 21:50:06,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826b575bc060ce15cf6f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:07,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081af575bc060ce15cb82 2025-12-17 21:50:07,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081af575bc060ce15cb82 HTTP/1.1" 502 150 2025-12-17 21:50:07,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081af575bc060ce15cb82'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:07,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408264575bc060ce15cf4f 2025-12-17 21:50:07,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408264575bc060ce15cf4f HTTP/1.1" 502 150 2025-12-17 21:50:07,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408264575bc060ce15cf4f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:08,407 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940823f575bc060ce15ced1 2025-12-17 21:50:08,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940823f575bc060ce15ced1 HTTP/1.1" 502 150 2025-12-17 21:50:08,409 - lava_callback - ERROR - Error processing callback for node 6940823f575bc060ce15ced1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940823f575bc060ce15ced1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940823f575bc060ce15ced1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940823f575bc060ce15ced1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:08,410 - lava_callback - INFO - Starting async job submit for node 6940826d575bc060ce15cf76 2025-12-17 21:50:08,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:08,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826d575bc060ce15cf76 HTTP/1.1" 502 150 2025-12-17 21:50:08,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826d575bc060ce15cf76'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:08,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826d575bc060ce15cf76 2025-12-17 21:50:08,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826d575bc060ce15cf76 HTTP/1.1" 502 150 2025-12-17 21:50:08,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826d575bc060ce15cf76'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:08,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408267575bc060ce15cf5f 2025-12-17 21:50:08,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408267575bc060ce15cf5f HTTP/1.1" 502 150 2025-12-17 21:50:08,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408267575bc060ce15cf5f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:08,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826b575bc060ce15cf6f 2025-12-17 21:50:08,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826b575bc060ce15cf6f HTTP/1.1" 502 150 2025-12-17 21:50:08,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826b575bc060ce15cf6f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:09,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824f575bc060ce15cef8 2025-12-17 21:50:09,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824f575bc060ce15cef8 HTTP/1.1" 502 150 2025-12-17 21:50:09,578 - lava_callback - ERROR - Error processing callback for node 6940824f575bc060ce15cef8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940824f575bc060ce15cef8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940824f575bc060ce15cef8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940824f575bc060ce15cef8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:09,580 - lava_callback - INFO - Starting async job submit for node 6940826e575bc060ce15cf79 2025-12-17 21:50:09,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:09,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826e575bc060ce15cf79 HTTP/1.1" 502 150 2025-12-17 21:50:09,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826e575bc060ce15cf79'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:09,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826e575bc060ce15cf79 2025-12-17 21:50:09,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826e575bc060ce15cf79 HTTP/1.1" 502 150 2025-12-17 21:50:09,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826e575bc060ce15cf79'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:10,257 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408258575bc060ce15cf31 2025-12-17 21:50:10,427 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826d575bc060ce15cf76 2025-12-17 21:50:10,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826d575bc060ce15cf76 HTTP/1.1" 502 150 2025-12-17 21:50:10,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826d575bc060ce15cf76'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:10,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940825b575bc060ce15cf37 2025-12-17 21:50:10,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825b575bc060ce15cf37 HTTP/1.1" 502 150 2025-12-17 21:50:10,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940825b575bc060ce15cf37'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:10,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940825e575bc060ce15cf3d 2025-12-17 21:50:10,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825e575bc060ce15cf3d HTTP/1.1" 502 150 2025-12-17 21:50:10,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940825e575bc060ce15cf3d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:11,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826e575bc060ce15cf79 2025-12-17 21:50:11,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826e575bc060ce15cf79 HTTP/1.1" 502 150 2025-12-17 21:50:11,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826e575bc060ce15cf79'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:11,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408261575bc060ce15cf43 2025-12-17 21:50:11,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408261575bc060ce15cf43 HTTP/1.1" 502 150 2025-12-17 21:50:11,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408261575bc060ce15cf43'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:11,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408264575bc060ce15cf4f 2025-12-17 21:50:11,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408264575bc060ce15cf4f HTTP/1.1" 502 150 2025-12-17 21:50:11,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408264575bc060ce15cf4f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:11,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408254575bc060ce15cf2b 2025-12-17 21:50:11,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408254575bc060ce15cf2b HTTP/1.1" 502 150 2025-12-17 21:50:11,912 - lava_callback - ERROR - Error processing callback for node 69408254575bc060ce15cf2b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408254575bc060ce15cf2b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408254575bc060ce15cf2b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408254575bc060ce15cf2b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:11,913 - lava_callback - INFO - Starting async job submit for node 694081b1575bc060ce15cb87 2025-12-17 21:50:11,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:11,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081b1575bc060ce15cb87 HTTP/1.1" 502 150 2025-12-17 21:50:11,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081b1575bc060ce15cb87'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:11,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081b1575bc060ce15cb87 2025-12-17 21:50:11,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081b1575bc060ce15cb87 HTTP/1.1" 502 150 2025-12-17 21:50:11,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081b1575bc060ce15cb87'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:12,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940825c575bc060ce15cf3a 2025-12-17 21:50:12,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825c575bc060ce15cf3a HTTP/1.1" 502 150 2025-12-17 21:50:12,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940825c575bc060ce15cf3a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:12,745 - lava_callback - INFO - Received callback for node 69408481575bc060ce15d7da INFO: 172.21.0.1:51802 - "POST /node/69408481575bc060ce15d7da HTTP/1.0" 202 Accepted 2025-12-17 21:50:12,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408267575bc060ce15cf5f 2025-12-17 21:50:12,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408267575bc060ce15cf5f HTTP/1.1" 502 150 2025-12-17 21:50:12,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408267575bc060ce15cf5f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:12,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826b575bc060ce15cf6f 2025-12-17 21:50:12,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826b575bc060ce15cf6f HTTP/1.1" 502 150 2025-12-17 21:50:12,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826b575bc060ce15cf6f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:13,239 - lava_callback - INFO - Received callback for node 69408280575bc060ce15cfbd INFO: 172.21.0.1:51806 - "POST /node/69408280575bc060ce15cfbd HTTP/1.0" 202 Accepted 2025-12-17 21:50:13,903 - lava_callback - INFO - Received callback for node 69408282575bc060ce15cfc0 INFO: 172.21.0.1:51810 - "POST /node/69408282575bc060ce15cfc0 HTTP/1.0" 202 Accepted 2025-12-17 21:50:13,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408256575bc060ce15cf2e 2025-12-17 21:50:13,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408256575bc060ce15cf2e HTTP/1.1" 502 150 2025-12-17 21:50:13,944 - lava_callback - ERROR - Error processing callback for node 69408256575bc060ce15cf2e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408256575bc060ce15cf2e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408256575bc060ce15cf2e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408256575bc060ce15cf2e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:13,947 - lava_callback - INFO - Starting async job submit for node 69408262575bc060ce15cf47 2025-12-17 21:50:13,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:13,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081b1575bc060ce15cb87 2025-12-17 21:50:13,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081b1575bc060ce15cb87 HTTP/1.1" 502 150 2025-12-17 21:50:13,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081b1575bc060ce15cb87'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:13,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408262575bc060ce15cf47 HTTP/1.1" 502 150 2025-12-17 21:50:13,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408262575bc060ce15cf47'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:13,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408262575bc060ce15cf47 2025-12-17 21:50:13,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408262575bc060ce15cf47 HTTP/1.1" 502 150 2025-12-17 21:50:13,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408262575bc060ce15cf47'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:14,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826d575bc060ce15cf76 2025-12-17 21:50:14,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826d575bc060ce15cf76 HTTP/1.1" 502 150 2025-12-17 21:50:14,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826d575bc060ce15cf76'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:14,581 - lava_callback - INFO - Received callback for node 69408283575bc060ce15cfc3 INFO: 172.21.0.1:51820 - "POST /node/69408283575bc060ce15cfc3 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:51826 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:50:15,243 - lava_callback - INFO - Received callback for node 69408284575bc060ce15cfc6 INFO: 172.21.0.1:51830 - "POST /node/69408284575bc060ce15cfc6 HTTP/1.0" 202 Accepted 2025-12-17 21:50:15,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940824a575bc060ce15cee2 2025-12-17 21:50:15,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824a575bc060ce15cee2 HTTP/1.1" 502 150 2025-12-17 21:50:15,322 - lava_callback - ERROR - Error processing callback for node 6940824a575bc060ce15cee2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940824a575bc060ce15cee2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940824a575bc060ce15cee2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940824a575bc060ce15cee2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:15,324 - lava_callback - INFO - Starting async job submit for node 69408266575bc060ce15cf58 2025-12-17 21:50:15,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:15,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408266575bc060ce15cf58 HTTP/1.1" 502 150 2025-12-17 21:50:15,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408266575bc060ce15cf58'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:15,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408266575bc060ce15cf58 2025-12-17 21:50:15,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408266575bc060ce15cf58 HTTP/1.1" 502 150 2025-12-17 21:50:15,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408266575bc060ce15cf58'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:15,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826e575bc060ce15cf79 2025-12-17 21:50:15,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826e575bc060ce15cf79 HTTP/1.1" 502 150 2025-12-17 21:50:15,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826e575bc060ce15cf79'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:15,919 - lava_callback - INFO - Received callback for node 69408289575bc060ce15cfcb INFO: 172.21.0.1:51836 - "POST /node/69408289575bc060ce15cfcb HTTP/1.0" 202 Accepted 2025-12-17 21:50:15,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408262575bc060ce15cf47 2025-12-17 21:50:15,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408262575bc060ce15cf47 HTTP/1.1" 502 150 2025-12-17 21:50:15,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408262575bc060ce15cf47'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51842 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:50:16,577 - lava_callback - INFO - Received callback for node 6940828b575bc060ce15cfd2 INFO: 172.21.0.1:51856 - "POST /node/6940828b575bc060ce15cfd2 HTTP/1.0" 202 Accepted 2025-12-17 21:50:17,228 - lava_callback - INFO - Received callback for node 6940828c575bc060ce15cfd9 INFO: 172.21.0.1:51864 - "POST /node/6940828c575bc060ce15cfd9 HTTP/1.0" 202 Accepted 2025-12-17 21:50:17,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408266575bc060ce15cf58 2025-12-17 21:50:17,885 - lava_callback - INFO - Received callback for node 6940828e575bc060ce15cff2 INFO: 172.21.0.1:51876 - "POST /node/6940828e575bc060ce15cff2 HTTP/1.0" 202 Accepted 2025-12-17 21:50:17,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081b1575bc060ce15cb87 2025-12-17 21:50:17,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081b1575bc060ce15cb87 HTTP/1.1" 502 150 2025-12-17 21:50:17,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081b1575bc060ce15cb87'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:18,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940824b575bc060ce15cee5 HTTP/1.1" 502 150 2025-12-17 21:50:18,288 - lava_callback - ERROR - Error processing callback for node 6940824b575bc060ce15cee5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940824b575bc060ce15cee5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940824b575bc060ce15cee5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940824b575bc060ce15cee5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:18,290 - lava_callback - INFO - Starting async job submit for node 69408271575bc060ce15cf82 2025-12-17 21:50:18,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:18,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408271575bc060ce15cf82 HTTP/1.1" 502 150 2025-12-17 21:50:18,300 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408271575bc060ce15cf82'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:18,300 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408271575bc060ce15cf82 2025-12-17 21:50:18,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408271575bc060ce15cf82 HTTP/1.1" 502 150 2025-12-17 21:50:18,300 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408271575bc060ce15cf82'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:18,568 - lava_callback - INFO - Received callback for node 69408290575bc060ce15d01d INFO: 172.21.0.1:51884 - "POST /node/69408290575bc060ce15d01d HTTP/1.0" 202 Accepted 2025-12-17 21:50:18,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940825e575bc060ce15cf3d 2025-12-17 21:50:18,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825e575bc060ce15cf3d HTTP/1.1" 502 150 2025-12-17 21:50:18,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940825e575bc060ce15cf3d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:19,244 - lava_callback - INFO - Received callback for node 69408291575bc060ce15d020 INFO: 172.21.0.1:51896 - "POST /node/69408291575bc060ce15d020 HTTP/1.0" 202 Accepted 2025-12-17 21:50:19,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408264575bc060ce15cf4f 2025-12-17 21:50:19,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408264575bc060ce15cf4f HTTP/1.1" 502 150 2025-12-17 21:50:19,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408264575bc060ce15cf4f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:19,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408259575bc060ce15cf34 2025-12-17 21:50:19,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408259575bc060ce15cf34 HTTP/1.1" 502 150 2025-12-17 21:50:19,720 - lava_callback - ERROR - Error processing callback for node 69408259575bc060ce15cf34: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408259575bc060ce15cf34 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408259575bc060ce15cf34 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408259575bc060ce15cf34 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:19,723 - lava_callback - INFO - Starting async job submit for node 69408273575bc060ce15cf89 2025-12-17 21:50:19,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:19,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408273575bc060ce15cf89 HTTP/1.1" 502 150 2025-12-17 21:50:19,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408273575bc060ce15cf89'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:19,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408273575bc060ce15cf89 2025-12-17 21:50:19,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408273575bc060ce15cf89 HTTP/1.1" 502 150 2025-12-17 21:50:19,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408273575bc060ce15cf89'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:19,913 - lava_callback - INFO - Received callback for node 69408292575bc060ce15d023 INFO: 172.21.0.1:52424 - "POST /node/69408292575bc060ce15d023 HTTP/1.0" 202 Accepted 2025-12-17 21:50:19,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408262575bc060ce15cf47 2025-12-17 21:50:19,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408262575bc060ce15cf47 HTTP/1.1" 502 150 2025-12-17 21:50:19,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408262575bc060ce15cf47'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:20,301 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408271575bc060ce15cf82 2025-12-17 21:50:20,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408271575bc060ce15cf82 HTTP/1.1" 502 150 2025-12-17 21:50:20,302 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408271575bc060ce15cf82'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:20,582 - lava_callback - INFO - Received callback for node 69408294575bc060ce15d026 INFO: 172.21.0.1:52440 - "POST /node/69408294575bc060ce15d026 HTTP/1.0" 202 Accepted 2025-12-17 21:50:20,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408267575bc060ce15cf5f 2025-12-17 21:50:20,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408267575bc060ce15cf5f HTTP/1.1" 502 150 2025-12-17 21:50:20,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408267575bc060ce15cf5f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:20,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826b575bc060ce15cf6f 2025-12-17 21:50:20,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826b575bc060ce15cf6f HTTP/1.1" 502 150 2025-12-17 21:50:20,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826b575bc060ce15cf6f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:21,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408273575bc060ce15cf89 2025-12-17 21:50:21,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408273575bc060ce15cf89 HTTP/1.1" 502 150 2025-12-17 21:50:21,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408273575bc060ce15cf89'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:22,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826d575bc060ce15cf76 2025-12-17 21:50:22,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826d575bc060ce15cf76 HTTP/1.1" 502 150 2025-12-17 21:50:22,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826d575bc060ce15cf76'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:22,887 - lava_callback - INFO - Received callback for node 69408297575bc060ce15d091 INFO: 172.21.0.1:52454 - "POST /node/69408297575bc060ce15d091 HTTP/1.0" 202 Accepted 2025-12-17 21:50:23,549 - lava_callback - INFO - Received callback for node 69408299575bc060ce15d094 INFO: 172.21.0.1:52466 - "POST /node/69408299575bc060ce15d094 HTTP/1.0" 202 Accepted 2025-12-17 21:50:23,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826e575bc060ce15cf79 2025-12-17 21:50:23,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826e575bc060ce15cf79 HTTP/1.1" 502 150 2025-12-17 21:50:23,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826e575bc060ce15cf79'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:23,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081af575bc060ce15cb82 2025-12-17 21:50:23,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081af575bc060ce15cb82 HTTP/1.1" 502 150 2025-12-17 21:50:23,644 - lava_callback - ERROR - Error processing callback for node 694081af575bc060ce15cb82: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081af575bc060ce15cb82 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081af575bc060ce15cb82 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081af575bc060ce15cb82 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:23,646 - lava_callback - INFO - Starting async job submit for node 69408274575bc060ce15cf8c 2025-12-17 21:50:23,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:23,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408274575bc060ce15cf8c HTTP/1.1" 502 150 2025-12-17 21:50:23,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408274575bc060ce15cf8c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:23,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408274575bc060ce15cf8c 2025-12-17 21:50:23,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408274575bc060ce15cf8c HTTP/1.1" 502 150 2025-12-17 21:50:23,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408274575bc060ce15cf8c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:24,302 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408271575bc060ce15cf82 2025-12-17 21:50:25,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408274575bc060ce15cf8c 2025-12-17 21:50:25,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408274575bc060ce15cf8c HTTP/1.1" 502 150 2025-12-17 21:50:25,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408274575bc060ce15cf8c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:25,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408258575bc060ce15cf31 HTTP/1.1" 502 150 2025-12-17 21:50:25,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408258575bc060ce15cf31'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:25,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408273575bc060ce15cf89 2025-12-17 21:50:25,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408273575bc060ce15cf89 HTTP/1.1" 502 150 2025-12-17 21:50:25,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408273575bc060ce15cf89'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:25,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081b1575bc060ce15cb87 2025-12-17 21:50:25,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081b1575bc060ce15cb87 HTTP/1.1" 502 150 2025-12-17 21:50:25,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081b1575bc060ce15cb87'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52482 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:50:26,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940825b575bc060ce15cf37 2025-12-17 21:50:26,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825b575bc060ce15cf37 HTTP/1.1" 502 150 2025-12-17 21:50:26,964 - lava_callback - ERROR - Error processing callback for node 6940825b575bc060ce15cf37: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940825b575bc060ce15cf37 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940825b575bc060ce15cf37 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940825b575bc060ce15cf37 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:26,966 - lava_callback - INFO - Starting async job submit for node 6940826f575bc060ce15cf7c 2025-12-17 21:50:26,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:26,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826f575bc060ce15cf7c HTTP/1.1" 502 150 2025-12-17 21:50:26,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826f575bc060ce15cf7c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:26,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826f575bc060ce15cf7c 2025-12-17 21:50:26,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826f575bc060ce15cf7c HTTP/1.1" 502 150 2025-12-17 21:50:26,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826f575bc060ce15cf7c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:27,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408261575bc060ce15cf43 2025-12-17 21:50:27,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408261575bc060ce15cf43 HTTP/1.1" 502 150 2025-12-17 21:50:27,600 - lava_callback - ERROR - Error processing callback for node 69408261575bc060ce15cf43: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408261575bc060ce15cf43 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408261575bc060ce15cf43 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408261575bc060ce15cf43 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:27,602 - lava_callback - INFO - Starting async job submit for node 69408270575bc060ce15cf7f 2025-12-17 21:50:27,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:27,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408270575bc060ce15cf7f HTTP/1.1" 502 150 2025-12-17 21:50:27,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408270575bc060ce15cf7f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:27,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408270575bc060ce15cf7f 2025-12-17 21:50:27,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408270575bc060ce15cf7f HTTP/1.1" 502 150 2025-12-17 21:50:27,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408270575bc060ce15cf7f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:27,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408262575bc060ce15cf47 2025-12-17 21:50:27,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408262575bc060ce15cf47 HTTP/1.1" 502 150 2025-12-17 21:50:27,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408262575bc060ce15cf47'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:28,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940825c575bc060ce15cf3a 2025-12-17 21:50:28,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825c575bc060ce15cf3a HTTP/1.1" 502 150 2025-12-17 21:50:28,540 - lava_callback - ERROR - Error processing callback for node 6940825c575bc060ce15cf3a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940825c575bc060ce15cf3a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940825c575bc060ce15cf3a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940825c575bc060ce15cf3a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:28,542 - lava_callback - INFO - Starting async job submit for node 69408276575bc060ce15cf8f 2025-12-17 21:50:28,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:28,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408276575bc060ce15cf8f HTTP/1.1" 502 150 2025-12-17 21:50:28,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408276575bc060ce15cf8f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:28,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408276575bc060ce15cf8f 2025-12-17 21:50:28,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408276575bc060ce15cf8f HTTP/1.1" 502 150 2025-12-17 21:50:28,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408276575bc060ce15cf8f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:28,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826f575bc060ce15cf7c 2025-12-17 21:50:28,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826f575bc060ce15cf7c HTTP/1.1" 502 150 2025-12-17 21:50:28,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826f575bc060ce15cf7c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:29,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408270575bc060ce15cf7f 2025-12-17 21:50:29,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408270575bc060ce15cf7f HTTP/1.1" 502 150 2025-12-17 21:50:29,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408270575bc060ce15cf7f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:29,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408274575bc060ce15cf8c 2025-12-17 21:50:29,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408274575bc060ce15cf8c HTTP/1.1" 502 150 2025-12-17 21:50:29,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408274575bc060ce15cf8c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:30,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408276575bc060ce15cf8f 2025-12-17 21:50:30,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408276575bc060ce15cf8f HTTP/1.1" 502 150 2025-12-17 21:50:30,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408276575bc060ce15cf8f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:32,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408266575bc060ce15cf58 HTTP/1.1" 502 150 2025-12-17 21:50:32,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408266575bc060ce15cf58'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:32,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826f575bc060ce15cf7c 2025-12-17 21:50:32,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826f575bc060ce15cf7c HTTP/1.1" 502 150 2025-12-17 21:50:32,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826f575bc060ce15cf7c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:33,259 - lava_callback - INFO - Received callback for node 69408295575bc060ce15d08b INFO: 172.21.0.1:52004 - "POST /node/69408295575bc060ce15d08b HTTP/1.0" 202 Accepted 2025-12-17 21:50:33,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408270575bc060ce15cf7f 2025-12-17 21:50:33,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408270575bc060ce15cf7f HTTP/1.1" 502 150 2025-12-17 21:50:33,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408270575bc060ce15cf7f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:33,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408258575bc060ce15cf31 2025-12-17 21:50:33,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408258575bc060ce15cf31 HTTP/1.1" 502 150 2025-12-17 21:50:33,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408258575bc060ce15cf31'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:33,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408273575bc060ce15cf89 2025-12-17 21:50:33,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408273575bc060ce15cf89 HTTP/1.1" 502 150 2025-12-17 21:50:33,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408273575bc060ce15cf89'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:33,912 - lava_callback - INFO - Received callback for node 69408296575bc060ce15d08e INFO: 172.21.0.1:52012 - "POST /node/69408296575bc060ce15d08e HTTP/1.0" 202 Accepted 2025-12-17 21:50:34,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408276575bc060ce15cf8f 2025-12-17 21:50:34,591 - lava_callback - INFO - Received callback for node 6940829a575bc060ce15d097 INFO: 172.21.0.1:52018 - "POST /node/6940829a575bc060ce15d097 HTTP/1.0" 202 Accepted 2025-12-17 21:50:34,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408276575bc060ce15cf8f HTTP/1.1" 502 150 2025-12-17 21:50:34,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408276575bc060ce15cf8f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:34,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940825e575bc060ce15cf3d 2025-12-17 21:50:34,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940825e575bc060ce15cf3d HTTP/1.1" 502 150 2025-12-17 21:50:34,973 - lava_callback - ERROR - Error processing callback for node 6940825e575bc060ce15cf3d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940825e575bc060ce15cf3d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940825e575bc060ce15cf3d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940825e575bc060ce15cf3d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:34,975 - lava_callback - INFO - Starting async job submit for node 69408277575bc060ce15cf94 2025-12-17 21:50:34,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:35,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408277575bc060ce15cf94 HTTP/1.1" 502 150 2025-12-17 21:50:35,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408277575bc060ce15cf94'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:35,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408277575bc060ce15cf94 2025-12-17 21:50:35,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408277575bc060ce15cf94 HTTP/1.1" 502 150 2025-12-17 21:50:35,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408277575bc060ce15cf94'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:35,265 - lava_callback - INFO - Received callback for node 6940829c575bc060ce15d09a INFO: 172.21.0.1:52020 - "POST /node/6940829c575bc060ce15d09a HTTP/1.0" 202 Accepted 2025-12-17 21:50:35,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408264575bc060ce15cf4f 2025-12-17 21:50:35,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408264575bc060ce15cf4f HTTP/1.1" 502 150 2025-12-17 21:50:35,689 - lava_callback - ERROR - Error processing callback for node 69408264575bc060ce15cf4f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408264575bc060ce15cf4f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408264575bc060ce15cf4f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408264575bc060ce15cf4f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:35,692 - lava_callback - INFO - Starting async job submit for node 69408278575bc060ce15cf9b 2025-12-17 21:50:35,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:35,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408278575bc060ce15cf9b HTTP/1.1" 502 150 2025-12-17 21:50:35,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408278575bc060ce15cf9b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:35,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408278575bc060ce15cf9b 2025-12-17 21:50:35,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408278575bc060ce15cf9b HTTP/1.1" 502 150 2025-12-17 21:50:35,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408278575bc060ce15cf9b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:35,931 - lava_callback - INFO - Received callback for node 6940829d575bc060ce15d09f INFO: 172.21.0.1:52030 - "POST /node/6940829d575bc060ce15d09f HTTP/1.0" 202 Accepted INFO: 172.21.0.1:52032 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:50:36,608 - lava_callback - INFO - Received callback for node 6940829f575bc060ce15d0a8 INFO: 172.21.0.1:52038 - "POST /node/6940829f575bc060ce15d0a8 HTTP/1.0" 202 Accepted 2025-12-17 21:50:36,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408266575bc060ce15cf58 2025-12-17 21:50:36,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408266575bc060ce15cf58 HTTP/1.1" 502 150 2025-12-17 21:50:36,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408266575bc060ce15cf58'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:36,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408267575bc060ce15cf5f 2025-12-17 21:50:36,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408267575bc060ce15cf5f HTTP/1.1" 502 150 2025-12-17 21:50:36,957 - lava_callback - ERROR - Error processing callback for node 69408267575bc060ce15cf5f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408267575bc060ce15cf5f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408267575bc060ce15cf5f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408267575bc060ce15cf5f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:36,959 - lava_callback - INFO - Starting async job submit for node 6940827a575bc060ce15cfa2 2025-12-17 21:50:36,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:36,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827a575bc060ce15cfa2 HTTP/1.1" 502 150 2025-12-17 21:50:36,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827a575bc060ce15cfa2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:36,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827a575bc060ce15cfa2 2025-12-17 21:50:36,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827a575bc060ce15cfa2 HTTP/1.1" 502 150 2025-12-17 21:50:36,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827a575bc060ce15cfa2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:36,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826b575bc060ce15cf6f 2025-12-17 21:50:36,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826b575bc060ce15cf6f HTTP/1.1" 502 150 2025-12-17 21:50:36,974 - lava_callback - ERROR - Error processing callback for node 6940826b575bc060ce15cf6f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940826b575bc060ce15cf6f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940826b575bc060ce15cf6f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940826b575bc060ce15cf6f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:36,976 - lava_callback - INFO - Starting async job submit for node 6940827b575bc060ce15cfa9 2025-12-17 21:50:36,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:36,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827b575bc060ce15cfa9 HTTP/1.1" 502 150 2025-12-17 21:50:36,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827b575bc060ce15cfa9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:36,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827b575bc060ce15cfa9 2025-12-17 21:50:36,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827b575bc060ce15cfa9 HTTP/1.1" 502 150 2025-12-17 21:50:36,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827b575bc060ce15cfa9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:37,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408277575bc060ce15cf94 2025-12-17 21:50:37,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408277575bc060ce15cf94 HTTP/1.1" 502 150 2025-12-17 21:50:37,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408277575bc060ce15cf94'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:37,282 - lava_callback - INFO - Received callback for node 694082a3575bc060ce15d0bd INFO: 172.21.0.1:52052 - "POST /node/694082a3575bc060ce15d0bd HTTP/1.0" 202 Accepted 2025-12-17 21:50:37,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408274575bc060ce15cf8c 2025-12-17 21:50:37,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408274575bc060ce15cf8c HTTP/1.1" 502 150 2025-12-17 21:50:37,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408274575bc060ce15cf8c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:37,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408278575bc060ce15cf9b 2025-12-17 21:50:37,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408278575bc060ce15cf9b HTTP/1.1" 502 150 2025-12-17 21:50:37,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408278575bc060ce15cf9b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:38,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826d575bc060ce15cf76 2025-12-17 21:50:38,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826d575bc060ce15cf76 HTTP/1.1" 502 150 2025-12-17 21:50:38,432 - lava_callback - ERROR - Error processing callback for node 6940826d575bc060ce15cf76: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940826d575bc060ce15cf76 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940826d575bc060ce15cf76 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940826d575bc060ce15cf76 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:38,434 - lava_callback - INFO - Starting async job submit for node 6940827c575bc060ce15cfb0 2025-12-17 21:50:38,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:38,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827c575bc060ce15cfb0 HTTP/1.1" 502 150 2025-12-17 21:50:38,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827c575bc060ce15cfb0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:38,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827c575bc060ce15cfb0 2025-12-17 21:50:38,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827c575bc060ce15cfb0 HTTP/1.1" 502 150 2025-12-17 21:50:38,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827c575bc060ce15cfb0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:38,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827a575bc060ce15cfa2 2025-12-17 21:50:38,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827a575bc060ce15cfa2 HTTP/1.1" 502 150 2025-12-17 21:50:38,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827a575bc060ce15cfa2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:38,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827b575bc060ce15cfa9 2025-12-17 21:50:38,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827b575bc060ce15cfa9 HTTP/1.1" 502 150 2025-12-17 21:50:38,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827b575bc060ce15cfa9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:39,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826e575bc060ce15cf79 2025-12-17 21:50:39,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826e575bc060ce15cf79 HTTP/1.1" 502 150 2025-12-17 21:50:39,594 - lava_callback - ERROR - Error processing callback for node 6940826e575bc060ce15cf79: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940826e575bc060ce15cf79 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940826e575bc060ce15cf79 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940826e575bc060ce15cf79 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:39,596 - lava_callback - INFO - Starting async job submit for node 6940827d575bc060ce15cfb7 2025-12-17 21:50:39,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:39,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827d575bc060ce15cfb7 HTTP/1.1" 502 150 2025-12-17 21:50:39,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827d575bc060ce15cfb7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:39,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827d575bc060ce15cfb7 2025-12-17 21:50:39,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827d575bc060ce15cfb7 HTTP/1.1" 502 150 2025-12-17 21:50:39,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827d575bc060ce15cfb7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:39,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408271575bc060ce15cf82 HTTP/1.1" 502 150 2025-12-17 21:50:39,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408271575bc060ce15cf82'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:40,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827c575bc060ce15cfb0 2025-12-17 21:50:40,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827c575bc060ce15cfb0 HTTP/1.1" 502 150 2025-12-17 21:50:40,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827c575bc060ce15cfb0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:40,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826f575bc060ce15cf7c 2025-12-17 21:50:40,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826f575bc060ce15cf7c HTTP/1.1" 502 150 2025-12-17 21:50:40,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940826f575bc060ce15cf7c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:41,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408277575bc060ce15cf94 2025-12-17 21:50:41,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408277575bc060ce15cf94 HTTP/1.1" 502 150 2025-12-17 21:50:41,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408277575bc060ce15cf94'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:41,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827d575bc060ce15cfb7 2025-12-17 21:50:41,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827d575bc060ce15cfb7 HTTP/1.1" 502 150 2025-12-17 21:50:41,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827d575bc060ce15cfb7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:41,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408270575bc060ce15cf7f 2025-12-17 21:50:41,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408270575bc060ce15cf7f HTTP/1.1" 502 150 2025-12-17 21:50:41,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408270575bc060ce15cf7f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:41,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408278575bc060ce15cf9b 2025-12-17 21:50:41,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408278575bc060ce15cf9b HTTP/1.1" 502 150 2025-12-17 21:50:41,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408278575bc060ce15cf9b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:41,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081b1575bc060ce15cb87 2025-12-17 21:50:41,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081b1575bc060ce15cb87 HTTP/1.1" 502 150 2025-12-17 21:50:41,954 - lava_callback - ERROR - Error processing callback for node 694081b1575bc060ce15cb87: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081b1575bc060ce15cb87 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081b1575bc060ce15cb87 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081b1575bc060ce15cb87 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:41,956 - lava_callback - INFO - Starting async job submit for node 6940827f575bc060ce15cfba 2025-12-17 21:50:41,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:41,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827f575bc060ce15cfba HTTP/1.1" 502 150 2025-12-17 21:50:41,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827f575bc060ce15cfba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:41,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827f575bc060ce15cfba 2025-12-17 21:50:41,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827f575bc060ce15cfba HTTP/1.1" 502 150 2025-12-17 21:50:41,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827f575bc060ce15cfba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:42,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408276575bc060ce15cf8f 2025-12-17 21:50:42,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408276575bc060ce15cf8f HTTP/1.1" 502 150 2025-12-17 21:50:42,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408276575bc060ce15cf8f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:42,953 - lava_callback - INFO - Received callback for node 694082a0575bc060ce15d0af INFO: 172.21.0.1:40808 - "POST /node/694082a0575bc060ce15d0af HTTP/1.0" 202 Accepted 2025-12-17 21:50:42,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827a575bc060ce15cfa2 2025-12-17 21:50:42,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827a575bc060ce15cfa2 HTTP/1.1" 502 150 2025-12-17 21:50:42,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827a575bc060ce15cfa2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:42,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827b575bc060ce15cfa9 2025-12-17 21:50:42,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827b575bc060ce15cfa9 HTTP/1.1" 502 150 2025-12-17 21:50:42,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827b575bc060ce15cfa9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:43,627 - lava_callback - INFO - Received callback for node 694082a1575bc060ce15d0b6 INFO: 172.21.0.1:40816 - "POST /node/694082a1575bc060ce15d0b6 HTTP/1.0" 202 Accepted 2025-12-17 21:50:43,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408262575bc060ce15cf47 2025-12-17 21:50:43,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408262575bc060ce15cf47 HTTP/1.1" 502 150 2025-12-17 21:50:43,962 - lava_callback - ERROR - Error processing callback for node 69408262575bc060ce15cf47: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408262575bc060ce15cf47 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408262575bc060ce15cf47 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408262575bc060ce15cf47 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:43,964 - lava_callback - INFO - Starting async job submit for node 69408481575bc060ce15d7da 2025-12-17 21:50:43,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827f575bc060ce15cfba 2025-12-17 21:50:43,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:43,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827f575bc060ce15cfba HTTP/1.1" 502 150 2025-12-17 21:50:43,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827f575bc060ce15cfba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:43,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408481575bc060ce15d7da HTTP/1.1" 502 150 2025-12-17 21:50:43,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408481575bc060ce15d7da'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:43,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408481575bc060ce15d7da 2025-12-17 21:50:43,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408481575bc060ce15d7da HTTP/1.1" 502 150 2025-12-17 21:50:43,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408481575bc060ce15d7da'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:44,284 - lava_callback - INFO - Received callback for node 694082a4575bc060ce15d0c2 INFO: 172.21.0.1:40828 - "POST /node/694082a4575bc060ce15d0c2 HTTP/1.0" 202 Accepted 2025-12-17 21:50:44,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827c575bc060ce15cfb0 2025-12-17 21:50:44,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827c575bc060ce15cfb0 HTTP/1.1" 502 150 2025-12-17 21:50:44,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827c575bc060ce15cfb0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:44,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408266575bc060ce15cf58 2025-12-17 21:50:44,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408266575bc060ce15cf58 HTTP/1.1" 502 150 2025-12-17 21:50:44,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408266575bc060ce15cf58'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:44,956 - lava_callback - INFO - Received callback for node 694082a5575bc060ce15d0c5 INFO: 172.21.0.1:40840 - "POST /node/694082a5575bc060ce15d0c5 HTTP/1.0" 202 Accepted 2025-12-17 21:50:45,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827d575bc060ce15cfb7 2025-12-17 21:50:45,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827d575bc060ce15cfb7 HTTP/1.1" 502 150 2025-12-17 21:50:45,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827d575bc060ce15cfb7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:45,616 - lava_callback - INFO - Received callback for node 694082a6575bc060ce15d0c8 INFO: 172.21.0.1:40852 - "POST /node/694082a6575bc060ce15d0c8 HTTP/1.0" 202 Accepted 2025-12-17 21:50:45,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408481575bc060ce15d7da 2025-12-17 21:50:45,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408481575bc060ce15d7da HTTP/1.1" 502 150 2025-12-17 21:50:45,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408481575bc060ce15d7da'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:46,284 - lava_callback - INFO - Received callback for node 694082a7575bc060ce15d0cb INFO: 172.21.0.1:40864 - "POST /node/694082a7575bc060ce15d0cb HTTP/1.0" 202 Accepted INFO: 172.21.0.1:40878 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:50:46,970 - lava_callback - INFO - Received callback for node 694082a9575bc060ce15d0ce INFO: 172.21.0.1:40890 - "POST /node/694082a9575bc060ce15d0ce HTTP/1.0" 202 Accepted 2025-12-17 21:50:47,657 - lava_callback - INFO - Received callback for node 694082aa575bc060ce15d0d1 INFO: 172.21.0.1:40896 - "POST /node/694082aa575bc060ce15d0d1 HTTP/1.0" 202 Accepted 2025-12-17 21:50:47,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408271575bc060ce15cf82 2025-12-17 21:50:47,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408271575bc060ce15cf82 HTTP/1.1" 502 150 2025-12-17 21:50:47,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408271575bc060ce15cf82'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:47,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827f575bc060ce15cfba 2025-12-17 21:50:47,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827f575bc060ce15cfba HTTP/1.1" 502 150 2025-12-17 21:50:47,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827f575bc060ce15cfba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:48,326 - lava_callback - INFO - Received callback for node 694082ab575bc060ce15d0d4 INFO: 172.21.0.1:40902 - "POST /node/694082ab575bc060ce15d0d4 HTTP/1.0" 202 Accepted 2025-12-17 21:50:48,990 - lava_callback - INFO - Received callback for node 694082ac575bc060ce15d0d7 INFO: 172.21.0.1:40908 - "POST /node/694082ac575bc060ce15d0d7 HTTP/1.0" 202 Accepted 2025-12-17 21:50:49,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408277575bc060ce15cf94 2025-12-17 21:50:49,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408277575bc060ce15cf94 HTTP/1.1" 502 150 2025-12-17 21:50:49,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408277575bc060ce15cf94'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:49,669 - lava_callback - INFO - Received callback for node 694082ae575bc060ce15d0da INFO: 172.21.0.1:40918 - "POST /node/694082ae575bc060ce15d0da HTTP/1.0" 202 Accepted 2025-12-17 21:50:49,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408278575bc060ce15cf9b 2025-12-17 21:50:49,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408278575bc060ce15cf9b HTTP/1.1" 502 150 2025-12-17 21:50:49,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408278575bc060ce15cf9b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:49,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408258575bc060ce15cf31 2025-12-17 21:50:49,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408258575bc060ce15cf31 HTTP/1.1" 502 150 2025-12-17 21:50:49,719 - lava_callback - ERROR - Error processing callback for node 69408258575bc060ce15cf31: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408258575bc060ce15cf31 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408258575bc060ce15cf31 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408258575bc060ce15cf31 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:49,721 - lava_callback - INFO - Starting async job submit for node 69408280575bc060ce15cfbd 2025-12-17 21:50:49,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:49,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408280575bc060ce15cfbd HTTP/1.1" 502 150 2025-12-17 21:50:49,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408280575bc060ce15cfbd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:49,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408280575bc060ce15cfbd 2025-12-17 21:50:49,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408280575bc060ce15cfbd HTTP/1.1" 502 150 2025-12-17 21:50:49,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408280575bc060ce15cfbd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:49,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408273575bc060ce15cf89 2025-12-17 21:50:49,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408273575bc060ce15cf89 HTTP/1.1" 502 150 2025-12-17 21:50:49,758 - lava_callback - ERROR - Error processing callback for node 69408273575bc060ce15cf89: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408273575bc060ce15cf89 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408273575bc060ce15cf89 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408273575bc060ce15cf89 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:49,760 - lava_callback - INFO - Starting async job submit for node 69408282575bc060ce15cfc0 2025-12-17 21:50:49,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:49,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408282575bc060ce15cfc0 HTTP/1.1" 502 150 2025-12-17 21:50:49,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408282575bc060ce15cfc0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:49,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408282575bc060ce15cfc0 2025-12-17 21:50:49,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408282575bc060ce15cfc0 HTTP/1.1" 502 150 2025-12-17 21:50:49,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408282575bc060ce15cfc0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:49,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408481575bc060ce15d7da 2025-12-17 21:50:49,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408481575bc060ce15d7da HTTP/1.1" 502 150 2025-12-17 21:50:50,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408481575bc060ce15d7da'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:50,336 - lava_callback - INFO - Received callback for node 694082af575bc060ce15d0dd INFO: 172.21.0.1:45726 - "POST /node/694082af575bc060ce15d0dd HTTP/1.0" 202 Accepted 2025-12-17 21:50:50,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827a575bc060ce15cfa2 2025-12-17 21:50:50,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827a575bc060ce15cfa2 HTTP/1.1" 502 150 2025-12-17 21:50:50,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827a575bc060ce15cfa2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:50,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827b575bc060ce15cfa9 2025-12-17 21:50:50,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827b575bc060ce15cfa9 HTTP/1.1" 502 150 2025-12-17 21:50:50,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827b575bc060ce15cfa9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:51,007 - lava_callback - INFO - Received callback for node 694082b0575bc060ce15d0e0 INFO: 172.21.0.1:45734 - "POST /node/694082b0575bc060ce15d0e0 HTTP/1.0" 202 Accepted 2025-12-17 21:50:51,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408280575bc060ce15cfbd 2025-12-17 21:50:51,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408280575bc060ce15cfbd HTTP/1.1" 502 150 2025-12-17 21:50:51,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408280575bc060ce15cfbd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:51,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408282575bc060ce15cfc0 2025-12-17 21:50:51,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408282575bc060ce15cfc0 HTTP/1.1" 502 150 2025-12-17 21:50:51,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408282575bc060ce15cfc0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:52,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827c575bc060ce15cfb0 2025-12-17 21:50:52,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827c575bc060ce15cfb0 HTTP/1.1" 502 150 2025-12-17 21:50:52,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827c575bc060ce15cfb0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:53,310 - lava_callback - INFO - Received callback for node 694082b3575bc060ce15d0ec INFO: 172.21.0.1:45738 - "POST /node/694082b3575bc060ce15d0ec HTTP/1.0" 202 Accepted 2025-12-17 21:50:53,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827d575bc060ce15cfb7 2025-12-17 21:50:53,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827d575bc060ce15cfb7 HTTP/1.1" 502 150 2025-12-17 21:50:53,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827d575bc060ce15cfb7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:53,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408274575bc060ce15cf8c 2025-12-17 21:50:53,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408274575bc060ce15cf8c HTTP/1.1" 502 150 2025-12-17 21:50:53,665 - lava_callback - ERROR - Error processing callback for node 69408274575bc060ce15cf8c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408274575bc060ce15cf8c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408274575bc060ce15cf8c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408274575bc060ce15cf8c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:53,667 - lava_callback - INFO - Starting async job submit for node 69408283575bc060ce15cfc3 2025-12-17 21:50:53,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:53,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408283575bc060ce15cfc3 HTTP/1.1" 502 150 2025-12-17 21:50:53,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408283575bc060ce15cfc3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:53,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408283575bc060ce15cfc3 2025-12-17 21:50:53,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408283575bc060ce15cfc3 HTTP/1.1" 502 150 2025-12-17 21:50:53,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408283575bc060ce15cfc3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:53,985 - lava_callback - INFO - Received callback for node 694082b5575bc060ce15d0ef INFO: 172.21.0.1:45748 - "POST /node/694082b5575bc060ce15d0ef HTTP/1.0" 202 Accepted 2025-12-17 21:50:54,692 - lava_callback - INFO - Received callback for node 694082b6575bc060ce15d0f2 INFO: 172.21.0.1:45760 - "POST /node/694082b6575bc060ce15d0f2 HTTP/1.0" 202 Accepted 2025-12-17 21:50:55,358 - lava_callback - INFO - Received callback for node 694082bc575bc060ce15d0fe INFO: 172.21.0.1:45774 - "POST /node/694082bc575bc060ce15d0fe HTTP/1.0" 202 Accepted 2025-12-17 21:50:55,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408283575bc060ce15cfc3 2025-12-17 21:50:55,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408280575bc060ce15cfbd 2025-12-17 21:50:55,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408280575bc060ce15cfbd HTTP/1.1" 502 150 2025-12-17 21:50:55,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408280575bc060ce15cfbd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:55,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408282575bc060ce15cfc0 2025-12-17 21:50:55,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408282575bc060ce15cfc0 HTTP/1.1" 502 150 2025-12-17 21:50:55,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408282575bc060ce15cfc0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:55,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827f575bc060ce15cfba 2025-12-17 21:50:55,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827f575bc060ce15cfba HTTP/1.1" 502 150 2025-12-17 21:50:55,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940827f575bc060ce15cfba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45778 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:50:56,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940826f575bc060ce15cf7c 2025-12-17 21:50:56,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940826f575bc060ce15cf7c HTTP/1.1" 502 150 2025-12-17 21:50:56,982 - lava_callback - ERROR - Error processing callback for node 6940826f575bc060ce15cf7c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940826f575bc060ce15cf7c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940826f575bc060ce15cf7c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940826f575bc060ce15cf7c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:56,985 - lava_callback - INFO - Starting async job submit for node 69408284575bc060ce15cfc6 2025-12-17 21:50:56,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:57,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408284575bc060ce15cfc6 HTTP/1.1" 502 150 2025-12-17 21:50:57,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408284575bc060ce15cfc6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:57,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408284575bc060ce15cfc6 2025-12-17 21:50:57,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408284575bc060ce15cfc6 HTTP/1.1" 502 150 2025-12-17 21:50:57,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408284575bc060ce15cfc6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:57,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408270575bc060ce15cf7f 2025-12-17 21:50:57,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408270575bc060ce15cf7f HTTP/1.1" 502 150 2025-12-17 21:50:57,619 - lava_callback - ERROR - Error processing callback for node 69408270575bc060ce15cf7f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408270575bc060ce15cf7f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408270575bc060ce15cf7f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408270575bc060ce15cf7f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:57,621 - lava_callback - INFO - Starting async job submit for node 69408289575bc060ce15cfcb 2025-12-17 21:50:57,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:57,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408289575bc060ce15cfcb HTTP/1.1" 502 150 2025-12-17 21:50:57,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408289575bc060ce15cfcb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:57,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408289575bc060ce15cfcb 2025-12-17 21:50:57,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408289575bc060ce15cfcb HTTP/1.1" 502 150 2025-12-17 21:50:57,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408289575bc060ce15cfcb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:58,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408481575bc060ce15d7da 2025-12-17 21:50:58,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408481575bc060ce15d7da HTTP/1.1" 502 150 2025-12-17 21:50:58,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408481575bc060ce15d7da'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:58,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408276575bc060ce15cf8f 2025-12-17 21:50:58,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408276575bc060ce15cf8f HTTP/1.1" 502 150 2025-12-17 21:50:58,600 - lava_callback - ERROR - Error processing callback for node 69408276575bc060ce15cf8f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408276575bc060ce15cf8f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408276575bc060ce15cf8f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408276575bc060ce15cf8f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:50:58,602 - lava_callback - INFO - Starting async job submit for node 6940828b575bc060ce15cfd2 2025-12-17 21:50:58,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:50:58,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828b575bc060ce15cfd2 HTTP/1.1" 502 150 2025-12-17 21:50:58,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940828b575bc060ce15cfd2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:58,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940828b575bc060ce15cfd2 2025-12-17 21:50:58,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828b575bc060ce15cfd2 HTTP/1.1" 502 150 2025-12-17 21:50:58,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940828b575bc060ce15cfd2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:59,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408284575bc060ce15cfc6 2025-12-17 21:50:59,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408284575bc060ce15cfc6 HTTP/1.1" 502 150 2025-12-17 21:50:59,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408284575bc060ce15cfc6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:50:59,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408289575bc060ce15cfcb 2025-12-17 21:50:59,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408289575bc060ce15cfcb HTTP/1.1" 502 150 2025-12-17 21:50:59,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408289575bc060ce15cfcb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:00,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940828b575bc060ce15cfd2 2025-12-17 21:51:00,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828b575bc060ce15cfd2 HTTP/1.1" 502 150 2025-12-17 21:51:00,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940828b575bc060ce15cfd2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:00,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408266575bc060ce15cf58 2025-12-17 21:51:00,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408266575bc060ce15cf58 HTTP/1.1" 502 150 2025-12-17 21:51:00,628 - lava_callback - ERROR - Error processing callback for node 69408266575bc060ce15cf58: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408266575bc060ce15cf58 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408266575bc060ce15cf58 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408266575bc060ce15cf58 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:00,630 - lava_callback - INFO - Starting async job submit for node 6940828c575bc060ce15cfd9 2025-12-17 21:51:00,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:00,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828c575bc060ce15cfd9 HTTP/1.1" 502 150 2025-12-17 21:51:00,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940828c575bc060ce15cfd9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:00,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940828c575bc060ce15cfd9 2025-12-17 21:51:00,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828c575bc060ce15cfd9 HTTP/1.1" 502 150 2025-12-17 21:51:00,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940828c575bc060ce15cfd9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58228 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:51:02,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940828c575bc060ce15cfd9 2025-12-17 21:51:02,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828c575bc060ce15cfd9 HTTP/1.1" 502 150 2025-12-17 21:51:02,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940828c575bc060ce15cfd9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:02,947 - lava_callback - INFO - Received callback for node 694082b2575bc060ce15d0e7 INFO: 172.21.0.1:58230 - "POST /node/694082b2575bc060ce15d0e7 HTTP/1.0" 202 Accepted 2025-12-17 21:51:03,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408284575bc060ce15cfc6 2025-12-17 21:51:03,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408284575bc060ce15cfc6 HTTP/1.1" 502 150 2025-12-17 21:51:03,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408284575bc060ce15cfc6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:03,626 - lava_callback - INFO - Received callback for node 694082b7575bc060ce15d0f5 INFO: 172.21.0.1:58246 - "POST /node/694082b7575bc060ce15d0f5 HTTP/1.0" 202 Accepted 2025-12-17 21:51:03,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408289575bc060ce15cfcb 2025-12-17 21:51:03,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408289575bc060ce15cfcb HTTP/1.1" 502 150 2025-12-17 21:51:03,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408289575bc060ce15cfcb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:03,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408280575bc060ce15cfbd 2025-12-17 21:51:03,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408280575bc060ce15cfbd HTTP/1.1" 502 150 2025-12-17 21:51:03,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408280575bc060ce15cfbd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:03,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408282575bc060ce15cfc0 2025-12-17 21:51:03,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408282575bc060ce15cfc0 HTTP/1.1" 502 150 2025-12-17 21:51:03,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408282575bc060ce15cfc0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:03,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408271575bc060ce15cf82 2025-12-17 21:51:03,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408271575bc060ce15cf82 HTTP/1.1" 502 150 2025-12-17 21:51:03,795 - lava_callback - ERROR - Error processing callback for node 69408271575bc060ce15cf82: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408271575bc060ce15cf82 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408271575bc060ce15cf82 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408271575bc060ce15cf82 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:03,797 - lava_callback - INFO - Starting async job submit for node 6940828e575bc060ce15cff2 2025-12-17 21:51:03,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:03,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828e575bc060ce15cff2 HTTP/1.1" 502 150 2025-12-17 21:51:03,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940828e575bc060ce15cff2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:03,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940828e575bc060ce15cff2 2025-12-17 21:51:03,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828e575bc060ce15cff2 HTTP/1.1" 502 150 2025-12-17 21:51:03,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940828e575bc060ce15cff2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:04,299 - lava_callback - INFO - Received callback for node 694082b9575bc060ce15d0f8 INFO: 172.21.0.1:58252 - "POST /node/694082b9575bc060ce15d0f8 HTTP/1.0" 202 Accepted 2025-12-17 21:51:04,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940828b575bc060ce15cfd2 2025-12-17 21:51:04,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828b575bc060ce15cfd2 HTTP/1.1" 502 150 2025-12-17 21:51:04,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940828b575bc060ce15cfd2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:04,979 - lava_callback - INFO - Received callback for node 694082ba575bc060ce15d0fb INFO: 172.21.0.1:58256 - "POST /node/694082ba575bc060ce15d0fb HTTP/1.0" 202 Accepted 2025-12-17 21:51:05,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408277575bc060ce15cf94 2025-12-17 21:51:05,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408277575bc060ce15cf94 HTTP/1.1" 502 150 2025-12-17 21:51:05,016 - lava_callback - ERROR - Error processing callback for node 69408277575bc060ce15cf94: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408277575bc060ce15cf94 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408277575bc060ce15cf94 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408277575bc060ce15cf94 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:05,018 - lava_callback - INFO - Starting async job submit for node 69408290575bc060ce15d01d 2025-12-17 21:51:05,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:05,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408290575bc060ce15d01d HTTP/1.1" 502 150 2025-12-17 21:51:05,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408290575bc060ce15d01d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:05,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408290575bc060ce15d01d 2025-12-17 21:51:05,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408290575bc060ce15d01d HTTP/1.1" 502 150 2025-12-17 21:51:05,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408290575bc060ce15d01d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:05,654 - lava_callback - INFO - Received callback for node 694082bd575bc060ce15d101 INFO: 172.21.0.1:58264 - "POST /node/694082bd575bc060ce15d101 HTTP/1.0" 202 Accepted 2025-12-17 21:51:05,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408278575bc060ce15cf9b 2025-12-17 21:51:05,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408278575bc060ce15cf9b HTTP/1.1" 502 150 2025-12-17 21:51:05,716 - lava_callback - ERROR - Error processing callback for node 69408278575bc060ce15cf9b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408278575bc060ce15cf9b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408278575bc060ce15cf9b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408278575bc060ce15cf9b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:05,717 - lava_callback - INFO - Starting async job submit for node 69408291575bc060ce15d020 2025-12-17 21:51:05,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:05,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408291575bc060ce15d020 HTTP/1.1" 502 150 2025-12-17 21:51:05,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408291575bc060ce15d020'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:05,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408291575bc060ce15d020 2025-12-17 21:51:05,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408291575bc060ce15d020 HTTP/1.1" 502 150 2025-12-17 21:51:05,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408291575bc060ce15d020'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:05,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940828e575bc060ce15cff2 2025-12-17 21:51:05,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828e575bc060ce15cff2 HTTP/1.1" 502 150 2025-12-17 21:51:05,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940828e575bc060ce15cff2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:06,337 - lava_callback - INFO - Received callback for node 694082be575bc060ce15d104 INFO: 172.21.0.1:58268 - "POST /node/694082be575bc060ce15d104 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:58278 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:51:06,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940828c575bc060ce15cfd9 2025-12-17 21:51:06,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828c575bc060ce15cfd9 HTTP/1.1" 502 150 2025-12-17 21:51:06,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940828c575bc060ce15cfd9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:06,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827a575bc060ce15cfa2 2025-12-17 21:51:06,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827a575bc060ce15cfa2 HTTP/1.1" 502 150 2025-12-17 21:51:06,973 - lava_callback - ERROR - Error processing callback for node 6940827a575bc060ce15cfa2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940827a575bc060ce15cfa2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940827a575bc060ce15cfa2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940827a575bc060ce15cfa2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:06,975 - lava_callback - INFO - Starting async job submit for node 69408292575bc060ce15d023 2025-12-17 21:51:06,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:06,988 - lava_callback - INFO - Received callback for node 694082c0575bc060ce15d107 2025-12-17 21:51:06,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827b575bc060ce15cfa9 INFO: 172.21.0.1:58284 - "POST /node/694082c0575bc060ce15d107 HTTP/1.0" 202 Accepted 2025-12-17 21:51:06,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827b575bc060ce15cfa9 HTTP/1.1" 502 150 2025-12-17 21:51:06,995 - lava_callback - ERROR - Error processing callback for node 6940827b575bc060ce15cfa9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940827b575bc060ce15cfa9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940827b575bc060ce15cfa9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940827b575bc060ce15cfa9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:06,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408292575bc060ce15d023 HTTP/1.1" 502 150 2025-12-17 21:51:06,997 - lava_callback - INFO - Starting async job submit for node 69408294575bc060ce15d026 2025-12-17 21:51:06,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408292575bc060ce15d023'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:06,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:06,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408292575bc060ce15d023 2025-12-17 21:51:06,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408292575bc060ce15d023 HTTP/1.1" 502 150 2025-12-17 21:51:06,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408292575bc060ce15d023'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:07,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408294575bc060ce15d026 HTTP/1.1" 502 150 2025-12-17 21:51:07,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408294575bc060ce15d026'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:07,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408294575bc060ce15d026 2025-12-17 21:51:07,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408294575bc060ce15d026 HTTP/1.1" 502 150 2025-12-17 21:51:07,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408294575bc060ce15d026'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:07,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408290575bc060ce15d01d 2025-12-17 21:51:07,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408290575bc060ce15d01d HTTP/1.1" 502 150 2025-12-17 21:51:07,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408290575bc060ce15d01d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:07,652 - lava_callback - INFO - Received callback for node 694082c1575bc060ce15d10a INFO: 172.21.0.1:58294 - "POST /node/694082c1575bc060ce15d10a HTTP/1.0" 202 Accepted 2025-12-17 21:51:07,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408291575bc060ce15d020 2025-12-17 21:51:07,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408291575bc060ce15d020 HTTP/1.1" 502 150 2025-12-17 21:51:07,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408291575bc060ce15d020'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:08,332 - lava_callback - INFO - Received callback for node 694082c2575bc060ce15d10d INFO: 172.21.0.1:58300 - "POST /node/694082c2575bc060ce15d10d HTTP/1.0" 202 Accepted 2025-12-17 21:51:08,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827c575bc060ce15cfb0 2025-12-17 21:51:08,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827c575bc060ce15cfb0 HTTP/1.1" 502 150 2025-12-17 21:51:08,474 - lava_callback - ERROR - Error processing callback for node 6940827c575bc060ce15cfb0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940827c575bc060ce15cfb0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940827c575bc060ce15cfb0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940827c575bc060ce15cfb0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:08,476 - lava_callback - INFO - Starting async job submit for node 69408297575bc060ce15d091 2025-12-17 21:51:08,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:08,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408297575bc060ce15d091 HTTP/1.1" 502 150 2025-12-17 21:51:08,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408297575bc060ce15d091'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:08,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408297575bc060ce15d091 2025-12-17 21:51:08,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408297575bc060ce15d091 HTTP/1.1" 502 150 2025-12-17 21:51:08,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408297575bc060ce15d091'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:08,996 - lava_callback - INFO - Received callback for node 694082c4575bc060ce15d110 INFO: 172.21.0.1:58314 - "POST /node/694082c4575bc060ce15d110 HTTP/1.0" 202 Accepted 2025-12-17 21:51:09,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408292575bc060ce15d023 2025-12-17 21:51:09,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408292575bc060ce15d023 HTTP/1.1" 502 150 2025-12-17 21:51:09,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408292575bc060ce15d023'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:09,028 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408294575bc060ce15d026 2025-12-17 21:51:09,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408294575bc060ce15d026 HTTP/1.1" 502 150 2025-12-17 21:51:09,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408294575bc060ce15d026'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:09,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827d575bc060ce15cfb7 2025-12-17 21:51:09,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827d575bc060ce15cfb7 HTTP/1.1" 502 150 2025-12-17 21:51:09,612 - lava_callback - ERROR - Error processing callback for node 6940827d575bc060ce15cfb7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940827d575bc060ce15cfb7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940827d575bc060ce15cfb7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940827d575bc060ce15cfb7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:09,614 - lava_callback - INFO - Starting async job submit for node 69408299575bc060ce15d094 2025-12-17 21:51:09,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:09,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408299575bc060ce15d094 HTTP/1.1" 502 150 2025-12-17 21:51:09,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408299575bc060ce15d094'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:09,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408299575bc060ce15d094 2025-12-17 21:51:09,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408299575bc060ce15d094 HTTP/1.1" 502 150 2025-12-17 21:51:09,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408299575bc060ce15d094'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:09,664 - lava_callback - INFO - Received callback for node 694082c5575bc060ce15d113 INFO: 172.21.0.1:58326 - "POST /node/694082c5575bc060ce15d113 HTTP/1.0" 202 Accepted 2025-12-17 21:51:09,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940828e575bc060ce15cff2 2025-12-17 21:51:09,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828e575bc060ce15cff2 HTTP/1.1" 502 150 2025-12-17 21:51:09,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940828e575bc060ce15cff2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:10,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408297575bc060ce15d091 2025-12-17 21:51:11,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408284575bc060ce15cfc6 2025-12-17 21:51:11,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408284575bc060ce15cfc6 HTTP/1.1" 502 150 2025-12-17 21:51:11,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408284575bc060ce15cfc6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:11,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408283575bc060ce15cfc3 HTTP/1.1" 502 150 2025-12-17 21:51:11,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408283575bc060ce15cfc3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:11,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408290575bc060ce15d01d 2025-12-17 21:51:11,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408290575bc060ce15d01d HTTP/1.1" 502 150 2025-12-17 21:51:11,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408290575bc060ce15d01d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:11,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408299575bc060ce15d094 2025-12-17 21:51:11,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408299575bc060ce15d094 HTTP/1.1" 502 150 2025-12-17 21:51:11,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408299575bc060ce15d094'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:11,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408289575bc060ce15cfcb 2025-12-17 21:51:11,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408289575bc060ce15cfcb HTTP/1.1" 502 150 2025-12-17 21:51:11,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408289575bc060ce15cfcb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:11,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408291575bc060ce15d020 2025-12-17 21:51:11,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408291575bc060ce15d020 HTTP/1.1" 502 150 2025-12-17 21:51:11,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408291575bc060ce15d020'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:11,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940827f575bc060ce15cfba 2025-12-17 21:51:11,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940827f575bc060ce15cfba HTTP/1.1" 502 150 2025-12-17 21:51:11,997 - lava_callback - ERROR - Error processing callback for node 6940827f575bc060ce15cfba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940827f575bc060ce15cfba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940827f575bc060ce15cfba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940827f575bc060ce15cfba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:11,999 - lava_callback - INFO - Starting async job submit for node 69408295575bc060ce15d08b 2025-12-17 21:51:12,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:12,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408295575bc060ce15d08b HTTP/1.1" 502 150 2025-12-17 21:51:12,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408295575bc060ce15d08b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:12,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408295575bc060ce15d08b 2025-12-17 21:51:12,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408295575bc060ce15d08b HTTP/1.1" 502 150 2025-12-17 21:51:12,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408295575bc060ce15d08b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:12,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940828b575bc060ce15cfd2 2025-12-17 21:51:12,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828b575bc060ce15cfd2 HTTP/1.1" 502 150 2025-12-17 21:51:12,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940828b575bc060ce15cfd2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:13,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408292575bc060ce15d023 2025-12-17 21:51:13,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408292575bc060ce15d023 HTTP/1.1" 502 150 2025-12-17 21:51:13,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408292575bc060ce15d023'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:13,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408294575bc060ce15d026 2025-12-17 21:51:13,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408294575bc060ce15d026 HTTP/1.1" 502 150 2025-12-17 21:51:13,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408294575bc060ce15d026'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:13,269 - lava_callback - INFO - Received callback for node 694082c9575bc060ce15d12c INFO: 172.21.0.1:55692 - "POST /node/694082c9575bc060ce15d12c HTTP/1.0" 202 Accepted 2025-12-17 21:51:13,922 - lava_callback - INFO - Received callback for node 694082cd575bc060ce15d154 INFO: 172.21.0.1:55694 - "POST /node/694082cd575bc060ce15d154 HTTP/1.0" 202 Accepted 2025-12-17 21:51:14,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408481575bc060ce15d7da 2025-12-17 21:51:14,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408481575bc060ce15d7da HTTP/1.1" 502 150 2025-12-17 21:51:14,002 - lava_callback - ERROR - Error processing callback for node 69408481575bc060ce15d7da: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408481575bc060ce15d7da (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408481575bc060ce15d7da (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408481575bc060ce15d7da (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:14,004 - lava_callback - INFO - Starting async job submit for node 69408296575bc060ce15d08e 2025-12-17 21:51:14,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:14,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408295575bc060ce15d08b 2025-12-17 21:51:14,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408295575bc060ce15d08b HTTP/1.1" 502 150 2025-12-17 21:51:14,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408295575bc060ce15d08b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:14,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408296575bc060ce15d08e HTTP/1.1" 502 150 2025-12-17 21:51:14,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408296575bc060ce15d08e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:14,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408296575bc060ce15d08e 2025-12-17 21:51:14,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408296575bc060ce15d08e HTTP/1.1" 502 150 2025-12-17 21:51:14,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408296575bc060ce15d08e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:14,580 - lava_callback - INFO - Received callback for node 694082cf575bc060ce15d168 INFO: 172.21.0.1:55698 - "POST /node/694082cf575bc060ce15d168 HTTP/1.0" 202 Accepted 2025-12-17 21:51:14,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940828c575bc060ce15cfd9 2025-12-17 21:51:14,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828c575bc060ce15cfd9 HTTP/1.1" 502 150 2025-12-17 21:51:14,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940828c575bc060ce15cfd9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:15,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408283575bc060ce15cfc3 2025-12-17 21:51:15,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408283575bc060ce15cfc3 HTTP/1.1" 502 150 2025-12-17 21:51:15,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408283575bc060ce15cfc3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55704 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:51:15,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408299575bc060ce15d094 2025-12-17 21:51:15,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408299575bc060ce15d094 HTTP/1.1" 502 150 2025-12-17 21:51:15,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408299575bc060ce15d094'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:16,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408296575bc060ce15d08e INFO: 172.21.0.1:55720 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:51:17,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940828e575bc060ce15cff2 2025-12-17 21:51:17,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828e575bc060ce15cff2 HTTP/1.1" 502 150 2025-12-17 21:51:17,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940828e575bc060ce15cff2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:18,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408295575bc060ce15d08b 2025-12-17 21:51:18,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408295575bc060ce15d08b HTTP/1.1" 502 150 2025-12-17 21:51:18,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408295575bc060ce15d08b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:18,322 - lava_callback - INFO - Received callback for node 694082cb575bc060ce15d12f INFO: 172.21.0.1:55734 - "POST /node/694082cb575bc060ce15d12f HTTP/1.0" 202 Accepted 2025-12-17 21:51:19,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408290575bc060ce15d01d 2025-12-17 21:51:19,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408290575bc060ce15d01d HTTP/1.1" 502 150 2025-12-17 21:51:19,037 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408290575bc060ce15d01d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:19,141 - lava_callback - INFO - Received callback for node 694082cf575bc060ce15d165 INFO: 172.21.0.1:55750 - "POST /node/694082cf575bc060ce15d165 HTTP/1.0" 202 Accepted 2025-12-17 21:51:19,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408291575bc060ce15d020 2025-12-17 21:51:19,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408291575bc060ce15d020 HTTP/1.1" 502 150 2025-12-17 21:51:19,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408291575bc060ce15d020'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:19,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408280575bc060ce15cfbd 2025-12-17 21:51:19,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408280575bc060ce15cfbd HTTP/1.1" 502 150 2025-12-17 21:51:19,750 - lava_callback - ERROR - Error processing callback for node 69408280575bc060ce15cfbd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408280575bc060ce15cfbd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408280575bc060ce15cfbd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408280575bc060ce15cfbd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:19,752 - lava_callback - INFO - Starting async job submit for node 6940829a575bc060ce15d097 2025-12-17 21:51:19,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:19,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829a575bc060ce15d097 HTTP/1.1" 502 150 2025-12-17 21:51:19,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829a575bc060ce15d097'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:19,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829a575bc060ce15d097 2025-12-17 21:51:19,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829a575bc060ce15d097 HTTP/1.1" 502 150 2025-12-17 21:51:19,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829a575bc060ce15d097'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:19,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408282575bc060ce15cfc0 2025-12-17 21:51:19,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408282575bc060ce15cfc0 HTTP/1.1" 502 150 2025-12-17 21:51:19,794 - lava_callback - ERROR - Error processing callback for node 69408282575bc060ce15cfc0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408282575bc060ce15cfc0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408282575bc060ce15cfc0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408282575bc060ce15cfc0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:19,796 - lava_callback - INFO - Starting async job submit for node 6940829c575bc060ce15d09a 2025-12-17 21:51:19,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:19,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829c575bc060ce15d09a HTTP/1.1" 502 150 2025-12-17 21:51:19,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829c575bc060ce15d09a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:19,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829c575bc060ce15d09a 2025-12-17 21:51:19,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829c575bc060ce15d09a HTTP/1.1" 502 150 2025-12-17 21:51:19,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829c575bc060ce15d09a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:19,917 - lava_callback - INFO - Received callback for node 694082d1575bc060ce15d16b INFO: 172.21.0.1:36462 - "POST /node/694082d1575bc060ce15d16b HTTP/1.0" 202 Accepted 2025-12-17 21:51:20,679 - lava_callback - INFO - Received callback for node 694082d3575bc060ce15d171 INFO: 172.21.0.1:36470 - "POST /node/694082d3575bc060ce15d171 HTTP/1.0" 202 Accepted 2025-12-17 21:51:21,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408292575bc060ce15d023 2025-12-17 21:51:21,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408292575bc060ce15d023 HTTP/1.1" 502 150 2025-12-17 21:51:21,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408292575bc060ce15d023'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:21,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408294575bc060ce15d026 2025-12-17 21:51:21,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408294575bc060ce15d026 HTTP/1.1" 502 150 2025-12-17 21:51:21,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408294575bc060ce15d026'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:21,453 - lava_callback - INFO - Received callback for node 694082d5575bc060ce15d174 INFO: 172.21.0.1:36484 - "POST /node/694082d5575bc060ce15d174 HTTP/1.0" 202 Accepted 2025-12-17 21:51:21,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829a575bc060ce15d097 2025-12-17 21:51:21,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829a575bc060ce15d097 HTTP/1.1" 502 150 2025-12-17 21:51:21,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829a575bc060ce15d097'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:21,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829c575bc060ce15d09a 2025-12-17 21:51:21,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829c575bc060ce15d09a HTTP/1.1" 502 150 2025-12-17 21:51:21,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829c575bc060ce15d09a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:22,981 - lava_callback - INFO - Received callback for node 694082c6575bc060ce15d118 INFO: 172.21.0.1:36494 - "POST /node/694082c6575bc060ce15d118 HTTP/1.0" 202 Accepted 2025-12-17 21:51:23,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408283575bc060ce15cfc3 2025-12-17 21:51:23,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408283575bc060ce15cfc3 HTTP/1.1" 502 150 2025-12-17 21:51:23,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408283575bc060ce15cfc3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:23,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408299575bc060ce15d094 2025-12-17 21:51:23,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408299575bc060ce15d094 HTTP/1.1" 502 150 2025-12-17 21:51:23,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408299575bc060ce15d094'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:23,652 - lava_callback - INFO - Received callback for node 694082c8575bc060ce15d123 INFO: 172.21.0.1:36498 - "POST /node/694082c8575bc060ce15d123 HTTP/1.0" 202 Accepted 2025-12-17 21:51:25,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829a575bc060ce15d097 2025-12-17 21:51:25,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829a575bc060ce15d097 HTTP/1.1" 502 150 2025-12-17 21:51:25,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829a575bc060ce15d097'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:25,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829c575bc060ce15d09a 2025-12-17 21:51:25,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829c575bc060ce15d09a HTTP/1.1" 502 150 2025-12-17 21:51:25,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829c575bc060ce15d09a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:25,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408297575bc060ce15d091 HTTP/1.1" 502 150 2025-12-17 21:51:25,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408297575bc060ce15d091'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:26,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408295575bc060ce15d08b 2025-12-17 21:51:26,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408295575bc060ce15d08b HTTP/1.1" 502 150 2025-12-17 21:51:26,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408295575bc060ce15d08b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36514 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:51:27,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408284575bc060ce15cfc6 2025-12-17 21:51:27,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408284575bc060ce15cfc6 HTTP/1.1" 502 150 2025-12-17 21:51:27,014 - lava_callback - ERROR - Error processing callback for node 69408284575bc060ce15cfc6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408284575bc060ce15cfc6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408284575bc060ce15cfc6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408284575bc060ce15cfc6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:27,016 - lava_callback - INFO - Starting async job submit for node 6940829d575bc060ce15d09f 2025-12-17 21:51:27,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:27,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829d575bc060ce15d09f HTTP/1.1" 502 150 2025-12-17 21:51:27,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829d575bc060ce15d09f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:27,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829d575bc060ce15d09f 2025-12-17 21:51:27,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829d575bc060ce15d09f HTTP/1.1" 502 150 2025-12-17 21:51:27,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829d575bc060ce15d09f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:27,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408289575bc060ce15cfcb 2025-12-17 21:51:27,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408289575bc060ce15cfcb HTTP/1.1" 502 150 2025-12-17 21:51:27,635 - lava_callback - ERROR - Error processing callback for node 69408289575bc060ce15cfcb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408289575bc060ce15cfcb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408289575bc060ce15cfcb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408289575bc060ce15cfcb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:27,637 - lava_callback - INFO - Starting async job submit for node 6940829f575bc060ce15d0a8 2025-12-17 21:51:27,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:27,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829f575bc060ce15d0a8 HTTP/1.1" 502 150 2025-12-17 21:51:27,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829f575bc060ce15d0a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:27,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829f575bc060ce15d0a8 2025-12-17 21:51:27,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829f575bc060ce15d0a8 HTTP/1.1" 502 150 2025-12-17 21:51:27,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829f575bc060ce15d0a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:28,150 - lava_callback - INFO - Received callback for node 694082de575bc060ce15d197 INFO: 172.21.0.1:36530 - "POST /node/694082de575bc060ce15d197 HTTP/1.0" 202 Accepted 2025-12-17 21:51:28,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940828b575bc060ce15cfd2 2025-12-17 21:51:28,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828b575bc060ce15cfd2 HTTP/1.1" 502 150 2025-12-17 21:51:28,616 - lava_callback - ERROR - Error processing callback for node 6940828b575bc060ce15cfd2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940828b575bc060ce15cfd2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940828b575bc060ce15cfd2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940828b575bc060ce15cfd2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:28,619 - lava_callback - INFO - Starting async job submit for node 694082a3575bc060ce15d0bd 2025-12-17 21:51:28,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:28,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a3575bc060ce15d0bd HTTP/1.1" 502 150 2025-12-17 21:51:28,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a3575bc060ce15d0bd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:28,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a3575bc060ce15d0bd 2025-12-17 21:51:28,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a3575bc060ce15d0bd HTTP/1.1" 502 150 2025-12-17 21:51:28,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a3575bc060ce15d0bd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:29,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829d575bc060ce15d09f 2025-12-17 21:51:29,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829d575bc060ce15d09f HTTP/1.1" 502 150 2025-12-17 21:51:29,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829d575bc060ce15d09f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:29,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829f575bc060ce15d0a8 2025-12-17 21:51:29,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829f575bc060ce15d0a8 HTTP/1.1" 502 150 2025-12-17 21:51:29,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829f575bc060ce15d0a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:29,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408297575bc060ce15d091 2025-12-17 21:51:29,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408297575bc060ce15d091 HTTP/1.1" 502 150 2025-12-17 21:51:29,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408297575bc060ce15d091'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:30,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a3575bc060ce15d0bd 2025-12-17 21:51:30,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a3575bc060ce15d0bd HTTP/1.1" 502 150 2025-12-17 21:51:30,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a3575bc060ce15d0bd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:30,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940828c575bc060ce15cfd9 2025-12-17 21:51:30,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828c575bc060ce15cfd9 HTTP/1.1" 502 150 2025-12-17 21:51:30,642 - lava_callback - ERROR - Error processing callback for node 6940828c575bc060ce15cfd9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940828c575bc060ce15cfd9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940828c575bc060ce15cfd9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940828c575bc060ce15cfd9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:30,644 - lava_callback - INFO - Starting async job submit for node 694082a0575bc060ce15d0af 2025-12-17 21:51:30,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:30,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a0575bc060ce15d0af HTTP/1.1" 502 150 2025-12-17 21:51:30,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a0575bc060ce15d0af'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:30,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a0575bc060ce15d0af 2025-12-17 21:51:30,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a0575bc060ce15d0af HTTP/1.1" 502 150 2025-12-17 21:51:30,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a0575bc060ce15d0af'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:31,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408296575bc060ce15d08e HTTP/1.1" 502 150 2025-12-17 21:51:31,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408296575bc060ce15d08e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:32,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a0575bc060ce15d0af 2025-12-17 21:51:32,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a0575bc060ce15d0af HTTP/1.1" 502 150 2025-12-17 21:51:32,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a0575bc060ce15d0af'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:33,001 - lava_callback - INFO - Received callback for node 69408216575bc060ce15cd22 INFO: 172.21.0.1:55550 - "POST /node/69408216575bc060ce15cd22 HTTP/1.0" 202 Accepted 2025-12-17 21:51:33,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829d575bc060ce15d09f 2025-12-17 21:51:33,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829d575bc060ce15d09f HTTP/1.1" 502 150 2025-12-17 21:51:33,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829d575bc060ce15d09f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:33,317 - lava_callback - INFO - Received callback for node 694082d2575bc060ce15d16e INFO: 172.21.0.1:55554 - "POST /node/694082d2575bc060ce15d16e HTTP/1.0" 202 Accepted 2025-12-17 21:51:33,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829f575bc060ce15d0a8 2025-12-17 21:51:33,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829f575bc060ce15d0a8 HTTP/1.1" 502 150 2025-12-17 21:51:33,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829f575bc060ce15d0a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:33,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829a575bc060ce15d097 2025-12-17 21:51:33,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829a575bc060ce15d097 HTTP/1.1" 502 150 2025-12-17 21:51:33,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829a575bc060ce15d097'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:33,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940828e575bc060ce15cff2 2025-12-17 21:51:33,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940828e575bc060ce15cff2 HTTP/1.1" 502 150 2025-12-17 21:51:33,811 - lava_callback - ERROR - Error processing callback for node 6940828e575bc060ce15cff2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940828e575bc060ce15cff2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940828e575bc060ce15cff2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940828e575bc060ce15cff2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:33,813 - lava_callback - INFO - Starting async job submit for node 694082a1575bc060ce15d0b6 2025-12-17 21:51:33,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:33,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a1575bc060ce15d0b6 HTTP/1.1" 502 150 2025-12-17 21:51:33,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a1575bc060ce15d0b6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:33,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a1575bc060ce15d0b6 2025-12-17 21:51:33,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a1575bc060ce15d0b6 HTTP/1.1" 502 150 2025-12-17 21:51:33,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a1575bc060ce15d0b6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:33,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829c575bc060ce15d09a 2025-12-17 21:51:33,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829c575bc060ce15d09a HTTP/1.1" 502 150 2025-12-17 21:51:33,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829c575bc060ce15d09a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:33,980 - lava_callback - INFO - Received callback for node 694082d5575bc060ce15d177 INFO: 172.21.0.1:55568 - "POST /node/694082d5575bc060ce15d177 HTTP/1.0" 202 Accepted 2025-12-17 21:51:34,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a3575bc060ce15d0bd 2025-12-17 21:51:34,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a3575bc060ce15d0bd HTTP/1.1" 502 150 2025-12-17 21:51:34,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a3575bc060ce15d0bd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:34,678 - lava_callback - INFO - Received callback for node 694082d7575bc060ce15d17a INFO: 172.21.0.1:55570 - "POST /node/694082d7575bc060ce15d17a HTTP/1.0" 202 Accepted 2025-12-17 21:51:35,037 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408290575bc060ce15d01d 2025-12-17 21:51:35,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408290575bc060ce15d01d HTTP/1.1" 502 150 2025-12-17 21:51:35,039 - lava_callback - ERROR - Error processing callback for node 69408290575bc060ce15d01d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408290575bc060ce15d01d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408290575bc060ce15d01d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408290575bc060ce15d01d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:35,041 - lava_callback - INFO - Starting async job submit for node 694082a4575bc060ce15d0c2 2025-12-17 21:51:35,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:35,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a4575bc060ce15d0c2 HTTP/1.1" 502 150 2025-12-17 21:51:35,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a4575bc060ce15d0c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:35,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a4575bc060ce15d0c2 2025-12-17 21:51:35,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a4575bc060ce15d0c2 HTTP/1.1" 502 150 2025-12-17 21:51:35,074 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a4575bc060ce15d0c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:35,358 - lava_callback - INFO - Received callback for node 694082d8575bc060ce15d17d INFO: 172.21.0.1:55580 - "POST /node/694082d8575bc060ce15d17d HTTP/1.0" 202 Accepted 2025-12-17 21:51:35,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408296575bc060ce15d08e 2025-12-17 21:51:35,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408296575bc060ce15d08e HTTP/1.1" 502 150 2025-12-17 21:51:35,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408296575bc060ce15d08e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:35,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408291575bc060ce15d020 2025-12-17 21:51:35,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408291575bc060ce15d020 HTTP/1.1" 502 150 2025-12-17 21:51:35,733 - lava_callback - ERROR - Error processing callback for node 69408291575bc060ce15d020: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408291575bc060ce15d020 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408291575bc060ce15d020 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408291575bc060ce15d020 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:35,736 - lava_callback - INFO - Starting async job submit for node 694082a5575bc060ce15d0c5 2025-12-17 21:51:35,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:35,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a5575bc060ce15d0c5 HTTP/1.1" 502 150 2025-12-17 21:51:35,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a5575bc060ce15d0c5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:35,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a5575bc060ce15d0c5 2025-12-17 21:51:35,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a5575bc060ce15d0c5 HTTP/1.1" 502 150 2025-12-17 21:51:35,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a5575bc060ce15d0c5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:35,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a1575bc060ce15d0b6 2025-12-17 21:51:35,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a1575bc060ce15d0b6 HTTP/1.1" 502 150 2025-12-17 21:51:35,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a1575bc060ce15d0b6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:36,015 - lava_callback - INFO - Received callback for node 694082d9575bc060ce15d180 INFO: 172.21.0.1:55596 - "POST /node/694082d9575bc060ce15d180 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:55602 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:51:36,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a0575bc060ce15d0af 2025-12-17 21:51:36,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a0575bc060ce15d0af HTTP/1.1" 502 150 2025-12-17 21:51:36,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a0575bc060ce15d0af'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:36,682 - lava_callback - INFO - Received callback for node 694082db575bc060ce15d18d INFO: 172.21.0.1:55608 - "POST /node/694082db575bc060ce15d18d HTTP/1.0" 202 Accepted 2025-12-17 21:51:37,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408292575bc060ce15d023 2025-12-17 21:51:37,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408292575bc060ce15d023 HTTP/1.1" 502 150 2025-12-17 21:51:37,010 - lava_callback - ERROR - Error processing callback for node 69408292575bc060ce15d023: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408292575bc060ce15d023 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408292575bc060ce15d023 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408292575bc060ce15d023 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:37,012 - lava_callback - INFO - Starting async job submit for node 694082a6575bc060ce15d0c8 2025-12-17 21:51:37,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:37,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408294575bc060ce15d026 2025-12-17 21:51:37,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408294575bc060ce15d026 HTTP/1.1" 502 150 2025-12-17 21:51:37,033 - lava_callback - ERROR - Error processing callback for node 69408294575bc060ce15d026: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408294575bc060ce15d026 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408294575bc060ce15d026 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408294575bc060ce15d026 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:37,035 - lava_callback - INFO - Starting async job submit for node 694082a7575bc060ce15d0cb 2025-12-17 21:51:37,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:37,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a7575bc060ce15d0cb HTTP/1.1" 502 150 2025-12-17 21:51:37,041 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a7575bc060ce15d0cb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:37,041 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a7575bc060ce15d0cb 2025-12-17 21:51:37,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a6575bc060ce15d0c8 HTTP/1.1" 502 150 2025-12-17 21:51:37,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a6575bc060ce15d0c8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:37,042 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a6575bc060ce15d0c8 2025-12-17 21:51:37,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a7575bc060ce15d0cb HTTP/1.1" 502 150 2025-12-17 21:51:37,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a7575bc060ce15d0cb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:37,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a6575bc060ce15d0c8 HTTP/1.1" 502 150 2025-12-17 21:51:37,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a6575bc060ce15d0c8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:37,075 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a4575bc060ce15d0c2 2025-12-17 21:51:37,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a4575bc060ce15d0c2 HTTP/1.1" 502 150 2025-12-17 21:51:37,076 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a4575bc060ce15d0c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:37,353 - lava_callback - INFO - Received callback for node 694082e0575bc060ce15d1a1 INFO: 172.21.0.1:55616 - "POST /node/694082e0575bc060ce15d1a1 HTTP/1.0" 202 Accepted 2025-12-17 21:51:37,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a5575bc060ce15d0c5 2025-12-17 21:51:37,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a5575bc060ce15d0c5 HTTP/1.1" 502 150 2025-12-17 21:51:37,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a5575bc060ce15d0c5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:37,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408297575bc060ce15d091 2025-12-17 21:51:37,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408297575bc060ce15d091 HTTP/1.1" 502 150 2025-12-17 21:51:37,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408297575bc060ce15d091'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:38,822 - lava_callback - INFO - Received callback for node 694082dd575bc060ce15d194 INFO: 172.21.0.1:55620 - "POST /node/694082dd575bc060ce15d194 HTTP/1.0" 202 Accepted 2025-12-17 21:51:39,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408283575bc060ce15cfc3 2025-12-17 21:51:39,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408283575bc060ce15cfc3 HTTP/1.1" 502 150 2025-12-17 21:51:39,029 - lava_callback - ERROR - Error processing callback for node 69408283575bc060ce15cfc3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408283575bc060ce15cfc3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408283575bc060ce15cfc3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408283575bc060ce15cfc3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:39,031 - lava_callback - INFO - Starting async job submit for node 694082a9575bc060ce15d0ce 2025-12-17 21:51:39,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:39,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a9575bc060ce15d0ce HTTP/1.1" 502 150 2025-12-17 21:51:39,039 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a9575bc060ce15d0ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:39,039 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a9575bc060ce15d0ce 2025-12-17 21:51:39,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a9575bc060ce15d0ce HTTP/1.1" 502 150 2025-12-17 21:51:39,040 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a9575bc060ce15d0ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:39,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a7575bc060ce15d0cb 2025-12-17 21:51:39,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a6575bc060ce15d0c8 2025-12-17 21:51:39,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a7575bc060ce15d0cb HTTP/1.1" 502 150 2025-12-17 21:51:39,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a7575bc060ce15d0cb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:39,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a6575bc060ce15d0c8 HTTP/1.1" 502 150 2025-12-17 21:51:39,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a6575bc060ce15d0c8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:39,613 - lava_callback - INFO - Received callback for node 694082e0575bc060ce15d19e INFO: 172.21.0.1:55624 - "POST /node/694082e0575bc060ce15d19e HTTP/1.0" 202 Accepted 2025-12-17 21:51:39,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408299575bc060ce15d094 2025-12-17 21:51:39,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408299575bc060ce15d094 HTTP/1.1" 502 150 2025-12-17 21:51:39,627 - lava_callback - ERROR - Error processing callback for node 69408299575bc060ce15d094: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408299575bc060ce15d094 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408299575bc060ce15d094 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408299575bc060ce15d094 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:39,629 - lava_callback - INFO - Starting async job submit for node 694082aa575bc060ce15d0d1 2025-12-17 21:51:39,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:39,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082aa575bc060ce15d0d1 HTTP/1.1" 502 150 2025-12-17 21:51:39,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082aa575bc060ce15d0d1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:39,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082aa575bc060ce15d0d1 2025-12-17 21:51:39,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082aa575bc060ce15d0d1 HTTP/1.1" 502 150 2025-12-17 21:51:39,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082aa575bc060ce15d0d1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:39,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a1575bc060ce15d0b6 2025-12-17 21:51:39,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a1575bc060ce15d0b6 HTTP/1.1" 502 150 2025-12-17 21:51:39,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a1575bc060ce15d0b6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:40,366 - lava_callback - INFO - Received callback for node 694082e2575bc060ce15d1a4 INFO: 172.21.0.1:49036 - "POST /node/694082e2575bc060ce15d1a4 HTTP/1.0" 202 Accepted 2025-12-17 21:51:41,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829d575bc060ce15d09f 2025-12-17 21:51:41,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829d575bc060ce15d09f HTTP/1.1" 502 150 2025-12-17 21:51:41,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829d575bc060ce15d09f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:41,034 - lava_callback - INFO - Received callback for node 6940839a575bc060ce15d41e INFO: 172.21.0.1:49048 - "POST /node/6940839a575bc060ce15d41e HTTP/1.0" 202 Accepted 2025-12-17 21:51:41,040 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a9575bc060ce15d0ce 2025-12-17 21:51:41,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a9575bc060ce15d0ce HTTP/1.1" 502 150 2025-12-17 21:51:41,041 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a9575bc060ce15d0ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:41,076 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a4575bc060ce15d0c2 2025-12-17 21:51:41,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a4575bc060ce15d0c2 HTTP/1.1" 502 150 2025-12-17 21:51:41,077 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a4575bc060ce15d0c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:41,132 - lava_callback - INFO - Received callback for node 694082e3575bc060ce15d1a7 INFO: 172.21.0.1:49056 - "POST /node/694082e3575bc060ce15d1a7 HTTP/1.0" 202 Accepted 2025-12-17 21:51:41,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082aa575bc060ce15d0d1 2025-12-17 21:51:41,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082aa575bc060ce15d0d1 HTTP/1.1" 502 150 2025-12-17 21:51:41,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082aa575bc060ce15d0d1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:41,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829f575bc060ce15d0a8 2025-12-17 21:51:41,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829f575bc060ce15d0a8 HTTP/1.1" 502 150 2025-12-17 21:51:41,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940829f575bc060ce15d0a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:41,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a5575bc060ce15d0c5 2025-12-17 21:51:41,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a5575bc060ce15d0c5 HTTP/1.1" 502 150 2025-12-17 21:51:41,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a5575bc060ce15d0c5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:42,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408295575bc060ce15d08b 2025-12-17 21:51:42,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a3575bc060ce15d0bd 2025-12-17 21:51:42,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a3575bc060ce15d0bd HTTP/1.1" 502 150 2025-12-17 21:51:42,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a3575bc060ce15d0bd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:43,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a7575bc060ce15d0cb 2025-12-17 21:51:43,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a6575bc060ce15d0c8 2025-12-17 21:51:43,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a7575bc060ce15d0cb HTTP/1.1" 502 150 2025-12-17 21:51:43,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a7575bc060ce15d0cb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:43,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a6575bc060ce15d0c8 HTTP/1.1" 502 150 2025-12-17 21:51:43,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a6575bc060ce15d0c8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:43,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408296575bc060ce15d08e 2025-12-17 21:51:43,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408296575bc060ce15d08e HTTP/1.1" 502 150 2025-12-17 21:51:43,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408296575bc060ce15d08e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:44,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a0575bc060ce15d0af 2025-12-17 21:51:44,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a0575bc060ce15d0af HTTP/1.1" 502 150 2025-12-17 21:51:44,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a0575bc060ce15d0af'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:45,041 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a9575bc060ce15d0ce 2025-12-17 21:51:45,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a9575bc060ce15d0ce HTTP/1.1" 502 150 2025-12-17 21:51:45,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a9575bc060ce15d0ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:45,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082aa575bc060ce15d0d1 2025-12-17 21:51:45,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082aa575bc060ce15d0d1 HTTP/1.1" 502 150 2025-12-17 21:51:45,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082aa575bc060ce15d0d1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49074 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:51:47,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a1575bc060ce15d0b6 2025-12-17 21:51:47,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a1575bc060ce15d0b6 HTTP/1.1" 502 150 2025-12-17 21:51:47,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a1575bc060ce15d0b6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:49,077 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a4575bc060ce15d0c2 2025-12-17 21:51:49,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829a575bc060ce15d097 2025-12-17 21:51:49,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829a575bc060ce15d097 HTTP/1.1" 502 150 2025-12-17 21:51:49,769 - lava_callback - ERROR - Error processing callback for node 6940829a575bc060ce15d097: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940829a575bc060ce15d097 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940829a575bc060ce15d097 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940829a575bc060ce15d097 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:49,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a5575bc060ce15d0c5 2025-12-17 21:51:49,771 - lava_callback - INFO - Starting async job submit for node 694082ab575bc060ce15d0d4 2025-12-17 21:51:49,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a5575bc060ce15d0c5 HTTP/1.1" 502 150 2025-12-17 21:51:49,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a5575bc060ce15d0c5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:49,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:49,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ab575bc060ce15d0d4 HTTP/1.1" 502 150 2025-12-17 21:51:49,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ab575bc060ce15d0d4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:49,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ab575bc060ce15d0d4 2025-12-17 21:51:49,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ab575bc060ce15d0d4 HTTP/1.1" 502 150 2025-12-17 21:51:49,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ab575bc060ce15d0d4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:49,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829c575bc060ce15d09a 2025-12-17 21:51:49,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829c575bc060ce15d09a HTTP/1.1" 502 150 2025-12-17 21:51:49,833 - lava_callback - ERROR - Error processing callback for node 6940829c575bc060ce15d09a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940829c575bc060ce15d09a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940829c575bc060ce15d09a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940829c575bc060ce15d09a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:49,835 - lava_callback - INFO - Starting async job submit for node 694082ac575bc060ce15d0d7 2025-12-17 21:51:49,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:49,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ac575bc060ce15d0d7 HTTP/1.1" 502 150 2025-12-17 21:51:49,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ac575bc060ce15d0d7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:49,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ac575bc060ce15d0d7 2025-12-17 21:51:49,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ac575bc060ce15d0d7 HTTP/1.1" 502 150 2025-12-17 21:51:49,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ac575bc060ce15d0d7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:51,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a7575bc060ce15d0cb 2025-12-17 21:51:51,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a6575bc060ce15d0c8 2025-12-17 21:51:51,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a7575bc060ce15d0cb HTTP/1.1" 502 150 2025-12-17 21:51:51,047 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a7575bc060ce15d0cb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:51,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a6575bc060ce15d0c8 HTTP/1.1" 502 150 2025-12-17 21:51:51,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a6575bc060ce15d0c8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:51,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ab575bc060ce15d0d4 2025-12-17 21:51:51,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ab575bc060ce15d0d4 HTTP/1.1" 502 150 2025-12-17 21:51:51,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ab575bc060ce15d0d4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:51,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ac575bc060ce15d0d7 2025-12-17 21:51:51,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ac575bc060ce15d0d7 HTTP/1.1" 502 150 2025-12-17 21:51:51,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ac575bc060ce15d0d7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51896 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:51:53,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a9575bc060ce15d0ce 2025-12-17 21:51:53,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082aa575bc060ce15d0d1 2025-12-17 21:51:53,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082aa575bc060ce15d0d1 HTTP/1.1" 502 150 2025-12-17 21:51:53,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082aa575bc060ce15d0d1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:53,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408297575bc060ce15d091 2025-12-17 21:51:53,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408297575bc060ce15d091 HTTP/1.1" 502 150 2025-12-17 21:51:53,876 - lava_callback - ERROR - Error processing callback for node 69408297575bc060ce15d091: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408297575bc060ce15d091 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408297575bc060ce15d091 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408297575bc060ce15d091 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:53,878 - lava_callback - INFO - Starting async job submit for node 694082ae575bc060ce15d0da 2025-12-17 21:51:53,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:53,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ae575bc060ce15d0da HTTP/1.1" 502 150 2025-12-17 21:51:53,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ae575bc060ce15d0da'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:53,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ae575bc060ce15d0da 2025-12-17 21:51:53,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ae575bc060ce15d0da HTTP/1.1" 502 150 2025-12-17 21:51:53,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ae575bc060ce15d0da'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:55,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ab575bc060ce15d0d4 2025-12-17 21:51:55,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ab575bc060ce15d0d4 HTTP/1.1" 502 150 2025-12-17 21:51:55,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ab575bc060ce15d0d4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:55,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ac575bc060ce15d0d7 2025-12-17 21:51:55,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ac575bc060ce15d0d7 HTTP/1.1" 502 150 2025-12-17 21:51:55,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ac575bc060ce15d0d7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:55,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ae575bc060ce15d0da 2025-12-17 21:51:55,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ae575bc060ce15d0da HTTP/1.1" 502 150 2025-12-17 21:51:55,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ae575bc060ce15d0da'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51912 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:51:57,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829d575bc060ce15d09f 2025-12-17 21:51:57,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829d575bc060ce15d09f HTTP/1.1" 502 150 2025-12-17 21:51:57,031 - lava_callback - ERROR - Error processing callback for node 6940829d575bc060ce15d09f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940829d575bc060ce15d09f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940829d575bc060ce15d09f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940829d575bc060ce15d09f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:57,034 - lava_callback - INFO - Starting async job submit for node 694082af575bc060ce15d0dd 2025-12-17 21:51:57,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:57,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082af575bc060ce15d0dd HTTP/1.1" 502 150 2025-12-17 21:51:57,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082af575bc060ce15d0dd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:57,042 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082af575bc060ce15d0dd 2025-12-17 21:51:57,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082af575bc060ce15d0dd HTTP/1.1" 502 150 2025-12-17 21:51:57,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082af575bc060ce15d0dd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:57,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408295575bc060ce15d08b HTTP/1.1" 502 150 2025-12-17 21:51:57,360 - lava_callback - ERROR - Error processing callback for node 69408295575bc060ce15d08b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408295575bc060ce15d08b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408295575bc060ce15d08b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408295575bc060ce15d08b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:57,363 - lava_callback - INFO - Starting async job submit for node 694082b0575bc060ce15d0e0 2025-12-17 21:51:57,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:57,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b0575bc060ce15d0e0 HTTP/1.1" 502 150 2025-12-17 21:51:57,371 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b0575bc060ce15d0e0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:57,371 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b0575bc060ce15d0e0 2025-12-17 21:51:57,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b0575bc060ce15d0e0 HTTP/1.1" 502 150 2025-12-17 21:51:57,372 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b0575bc060ce15d0e0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:57,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940829f575bc060ce15d0a8 2025-12-17 21:51:57,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940829f575bc060ce15d0a8 HTTP/1.1" 502 150 2025-12-17 21:51:57,676 - lava_callback - ERROR - Error processing callback for node 6940829f575bc060ce15d0a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940829f575bc060ce15d0a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940829f575bc060ce15d0a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940829f575bc060ce15d0a8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:57,678 - lava_callback - INFO - Starting async job submit for node 694082b3575bc060ce15d0ec 2025-12-17 21:51:57,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:57,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b3575bc060ce15d0ec HTTP/1.1" 502 150 2025-12-17 21:51:57,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b3575bc060ce15d0ec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:57,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b3575bc060ce15d0ec 2025-12-17 21:51:57,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b3575bc060ce15d0ec HTTP/1.1" 502 150 2025-12-17 21:51:57,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b3575bc060ce15d0ec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:58,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a3575bc060ce15d0bd 2025-12-17 21:51:58,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a3575bc060ce15d0bd HTTP/1.1" 502 150 2025-12-17 21:51:58,636 - lava_callback - ERROR - Error processing callback for node 694082a3575bc060ce15d0bd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a3575bc060ce15d0bd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a3575bc060ce15d0bd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a3575bc060ce15d0bd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:58,637 - lava_callback - INFO - Starting async job submit for node 694082b5575bc060ce15d0ef 2025-12-17 21:51:58,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:58,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b5575bc060ce15d0ef HTTP/1.1" 502 150 2025-12-17 21:51:58,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b5575bc060ce15d0ef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:58,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b5575bc060ce15d0ef 2025-12-17 21:51:58,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b5575bc060ce15d0ef HTTP/1.1" 502 150 2025-12-17 21:51:58,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b5575bc060ce15d0ef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:59,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082af575bc060ce15d0dd 2025-12-17 21:51:59,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082af575bc060ce15d0dd HTTP/1.1" 502 150 2025-12-17 21:51:59,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082af575bc060ce15d0dd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:59,372 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b0575bc060ce15d0e0 2025-12-17 21:51:59,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b0575bc060ce15d0e0 HTTP/1.1" 502 150 2025-12-17 21:51:59,373 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b0575bc060ce15d0e0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:59,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408296575bc060ce15d08e 2025-12-17 21:51:59,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408296575bc060ce15d08e HTTP/1.1" 502 150 2025-12-17 21:51:59,509 - lava_callback - ERROR - Error processing callback for node 69408296575bc060ce15d08e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408296575bc060ce15d08e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408296575bc060ce15d08e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408296575bc060ce15d08e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:51:59,511 - lava_callback - INFO - Starting async job submit for node 694082b6575bc060ce15d0f2 2025-12-17 21:51:59,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:51:59,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b6575bc060ce15d0f2 HTTP/1.1" 502 150 2025-12-17 21:51:59,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b6575bc060ce15d0f2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:59,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b6575bc060ce15d0f2 2025-12-17 21:51:59,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b6575bc060ce15d0f2 HTTP/1.1" 502 150 2025-12-17 21:51:59,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b6575bc060ce15d0f2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:59,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b3575bc060ce15d0ec 2025-12-17 21:51:59,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b3575bc060ce15d0ec HTTP/1.1" 502 150 2025-12-17 21:51:59,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b3575bc060ce15d0ec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:51:59,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ae575bc060ce15d0da 2025-12-17 21:51:59,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ae575bc060ce15d0da HTTP/1.1" 502 150 2025-12-17 21:51:59,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ae575bc060ce15d0da'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:00,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b5575bc060ce15d0ef 2025-12-17 21:52:00,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b5575bc060ce15d0ef HTTP/1.1" 502 150 2025-12-17 21:52:00,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b5575bc060ce15d0ef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:00,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a0575bc060ce15d0af 2025-12-17 21:52:00,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a0575bc060ce15d0af HTTP/1.1" 502 150 2025-12-17 21:52:00,677 - lava_callback - ERROR - Error processing callback for node 694082a0575bc060ce15d0af: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a0575bc060ce15d0af (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a0575bc060ce15d0af (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a0575bc060ce15d0af (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:00,679 - lava_callback - INFO - Starting async job submit for node 694082bc575bc060ce15d0fe 2025-12-17 21:52:00,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:00,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082bc575bc060ce15d0fe HTTP/1.1" 502 150 2025-12-17 21:52:00,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082bc575bc060ce15d0fe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:00,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082bc575bc060ce15d0fe 2025-12-17 21:52:00,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082bc575bc060ce15d0fe HTTP/1.1" 502 150 2025-12-17 21:52:00,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082bc575bc060ce15d0fe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:01,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b6575bc060ce15d0f2 2025-12-17 21:52:01,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b6575bc060ce15d0f2 HTTP/1.1" 502 150 2025-12-17 21:52:01,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b6575bc060ce15d0f2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58202 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:52:02,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082bc575bc060ce15d0fe 2025-12-17 21:52:02,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082bc575bc060ce15d0fe HTTP/1.1" 502 150 2025-12-17 21:52:02,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082bc575bc060ce15d0fe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:03,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082af575bc060ce15d0dd 2025-12-17 21:52:03,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082af575bc060ce15d0dd HTTP/1.1" 502 150 2025-12-17 21:52:03,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082af575bc060ce15d0dd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:03,373 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b0575bc060ce15d0e0 2025-12-17 21:52:03,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b0575bc060ce15d0e0 HTTP/1.1" 502 150 2025-12-17 21:52:03,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b0575bc060ce15d0e0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:03,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b3575bc060ce15d0ec 2025-12-17 21:52:03,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b3575bc060ce15d0ec HTTP/1.1" 502 150 2025-12-17 21:52:03,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b3575bc060ce15d0ec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:03,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ab575bc060ce15d0d4 2025-12-17 21:52:03,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ab575bc060ce15d0d4 HTTP/1.1" 502 150 2025-12-17 21:52:03,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ab575bc060ce15d0d4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:03,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a1575bc060ce15d0b6 2025-12-17 21:52:03,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a1575bc060ce15d0b6 HTTP/1.1" 502 150 2025-12-17 21:52:03,834 - lava_callback - ERROR - Error processing callback for node 694082a1575bc060ce15d0b6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a1575bc060ce15d0b6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a1575bc060ce15d0b6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a1575bc060ce15d0b6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:03,836 - lava_callback - INFO - Starting async job submit for node 694082b2575bc060ce15d0e7 2025-12-17 21:52:03,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:03,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b2575bc060ce15d0e7 HTTP/1.1" 502 150 2025-12-17 21:52:03,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b2575bc060ce15d0e7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:03,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b2575bc060ce15d0e7 2025-12-17 21:52:03,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b2575bc060ce15d0e7 HTTP/1.1" 502 150 2025-12-17 21:52:03,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b2575bc060ce15d0e7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:03,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ac575bc060ce15d0d7 2025-12-17 21:52:03,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ac575bc060ce15d0d7 HTTP/1.1" 502 150 2025-12-17 21:52:03,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ac575bc060ce15d0d7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:04,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a4575bc060ce15d0c2 HTTP/1.1" 502 150 2025-12-17 21:52:04,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a4575bc060ce15d0c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:04,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b5575bc060ce15d0ef 2025-12-17 21:52:04,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b5575bc060ce15d0ef HTTP/1.1" 502 150 2025-12-17 21:52:04,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b5575bc060ce15d0ef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:05,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b6575bc060ce15d0f2 2025-12-17 21:52:05,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b6575bc060ce15d0f2 HTTP/1.1" 502 150 2025-12-17 21:52:05,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b6575bc060ce15d0f2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:05,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a5575bc060ce15d0c5 2025-12-17 21:52:05,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a5575bc060ce15d0c5 HTTP/1.1" 502 150 2025-12-17 21:52:05,774 - lava_callback - ERROR - Error processing callback for node 694082a5575bc060ce15d0c5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a5575bc060ce15d0c5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a5575bc060ce15d0c5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a5575bc060ce15d0c5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:05,775 - lava_callback - INFO - Starting async job submit for node 694082b7575bc060ce15d0f5 2025-12-17 21:52:05,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:05,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b7575bc060ce15d0f5 HTTP/1.1" 502 150 2025-12-17 21:52:05,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b7575bc060ce15d0f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:05,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b7575bc060ce15d0f5 2025-12-17 21:52:05,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b7575bc060ce15d0f5 HTTP/1.1" 502 150 2025-12-17 21:52:05,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b7575bc060ce15d0f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:05,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b2575bc060ce15d0e7 2025-12-17 21:52:05,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b2575bc060ce15d0e7 HTTP/1.1" 502 150 2025-12-17 21:52:05,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b2575bc060ce15d0e7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58204 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:52:06,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082bc575bc060ce15d0fe 2025-12-17 21:52:06,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082bc575bc060ce15d0fe HTTP/1.1" 502 150 2025-12-17 21:52:06,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082bc575bc060ce15d0fe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:07,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a7575bc060ce15d0cb 2025-12-17 21:52:07,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a6575bc060ce15d0c8 2025-12-17 21:52:07,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a6575bc060ce15d0c8 HTTP/1.1" 502 150 2025-12-17 21:52:07,049 - lava_callback - ERROR - Error processing callback for node 694082a6575bc060ce15d0c8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a6575bc060ce15d0c8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a6575bc060ce15d0c8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a6575bc060ce15d0c8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:07,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a7575bc060ce15d0cb HTTP/1.1" 502 150 2025-12-17 21:52:07,051 - lava_callback - ERROR - Error processing callback for node 694082a7575bc060ce15d0cb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a7575bc060ce15d0cb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a7575bc060ce15d0cb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a7575bc060ce15d0cb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:07,051 - lava_callback - INFO - Starting async job submit for node 694082b9575bc060ce15d0f8 2025-12-17 21:52:07,053 - lava_callback - INFO - Starting async job submit for node 694082ba575bc060ce15d0fb 2025-12-17 21:52:07,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:07,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:07,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b9575bc060ce15d0f8 HTTP/1.1" 502 150 2025-12-17 21:52:07,065 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b9575bc060ce15d0f8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:07,065 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b9575bc060ce15d0f8 2025-12-17 21:52:07,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b9575bc060ce15d0f8 HTTP/1.1" 502 150 2025-12-17 21:52:07,066 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b9575bc060ce15d0f8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:07,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ba575bc060ce15d0fb HTTP/1.1" 502 150 2025-12-17 21:52:07,066 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ba575bc060ce15d0fb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:07,067 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ba575bc060ce15d0fb 2025-12-17 21:52:07,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ba575bc060ce15d0fb HTTP/1.1" 502 150 2025-12-17 21:52:07,067 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ba575bc060ce15d0fb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:07,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b7575bc060ce15d0f5 2025-12-17 21:52:07,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b7575bc060ce15d0f5 HTTP/1.1" 502 150 2025-12-17 21:52:07,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b7575bc060ce15d0f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:07,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ae575bc060ce15d0da 2025-12-17 21:52:07,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ae575bc060ce15d0da HTTP/1.1" 502 150 2025-12-17 21:52:07,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ae575bc060ce15d0da'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:08,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a9575bc060ce15d0ce HTTP/1.1" 502 150 2025-12-17 21:52:08,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082a9575bc060ce15d0ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:09,067 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b9575bc060ce15d0f8 2025-12-17 21:52:09,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b9575bc060ce15d0f8 HTTP/1.1" 502 150 2025-12-17 21:52:09,068 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b9575bc060ce15d0f8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:09,068 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ba575bc060ce15d0fb 2025-12-17 21:52:09,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ba575bc060ce15d0fb HTTP/1.1" 502 150 2025-12-17 21:52:09,069 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ba575bc060ce15d0fb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:09,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082aa575bc060ce15d0d1 2025-12-17 21:52:09,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082aa575bc060ce15d0d1 HTTP/1.1" 502 150 2025-12-17 21:52:09,652 - lava_callback - ERROR - Error processing callback for node 694082aa575bc060ce15d0d1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082aa575bc060ce15d0d1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082aa575bc060ce15d0d1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082aa575bc060ce15d0d1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:09,654 - lava_callback - INFO - Starting async job submit for node 694082bd575bc060ce15d101 2025-12-17 21:52:09,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:09,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082bd575bc060ce15d101 HTTP/1.1" 502 150 2025-12-17 21:52:09,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082bd575bc060ce15d101'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:09,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082bd575bc060ce15d101 2025-12-17 21:52:09,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082bd575bc060ce15d101 HTTP/1.1" 502 150 2025-12-17 21:52:09,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082bd575bc060ce15d101'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:09,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b2575bc060ce15d0e7 2025-12-17 21:52:09,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b2575bc060ce15d0e7 HTTP/1.1" 502 150 2025-12-17 21:52:09,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b2575bc060ce15d0e7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:11,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082af575bc060ce15d0dd 2025-12-17 21:52:11,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082af575bc060ce15d0dd HTTP/1.1" 502 150 2025-12-17 21:52:11,047 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082af575bc060ce15d0dd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:11,375 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b0575bc060ce15d0e0 2025-12-17 21:52:11,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b0575bc060ce15d0e0 HTTP/1.1" 502 150 2025-12-17 21:52:11,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b0575bc060ce15d0e0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:11,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082bd575bc060ce15d101 2025-12-17 21:52:11,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082bd575bc060ce15d101 HTTP/1.1" 502 150 2025-12-17 21:52:11,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082bd575bc060ce15d101'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:11,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b3575bc060ce15d0ec 2025-12-17 21:52:11,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b3575bc060ce15d0ec HTTP/1.1" 502 150 2025-12-17 21:52:11,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b3575bc060ce15d0ec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:11,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b7575bc060ce15d0f5 2025-12-17 21:52:11,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b7575bc060ce15d0f5 HTTP/1.1" 502 150 2025-12-17 21:52:11,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b7575bc060ce15d0f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:12,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b5575bc060ce15d0ef 2025-12-17 21:52:12,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b5575bc060ce15d0ef HTTP/1.1" 502 150 2025-12-17 21:52:12,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b5575bc060ce15d0ef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:13,068 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b9575bc060ce15d0f8 2025-12-17 21:52:13,069 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ba575bc060ce15d0fb 2025-12-17 21:52:13,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b9575bc060ce15d0f8 HTTP/1.1" 502 150 2025-12-17 21:52:13,070 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b9575bc060ce15d0f8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:13,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ba575bc060ce15d0fb HTTP/1.1" 502 150 2025-12-17 21:52:13,070 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ba575bc060ce15d0fb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:13,348 - lava_callback - INFO - Received callback for node 69408303575bc060ce15d1ac INFO: 172.21.0.1:50096 - "POST /node/69408303575bc060ce15d1ac HTTP/1.0" 202 Accepted 2025-12-17 21:52:13,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b6575bc060ce15d0f2 2025-12-17 21:52:13,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b6575bc060ce15d0f2 HTTP/1.1" 502 150 2025-12-17 21:52:13,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b6575bc060ce15d0f2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:14,014 - lava_callback - INFO - Received callback for node 69408305575bc060ce15d1b7 INFO: 172.21.0.1:50098 - "POST /node/69408305575bc060ce15d1b7 HTTP/1.0" 202 Accepted 2025-12-17 21:52:14,668 - lava_callback - INFO - Received callback for node 69408307575bc060ce15d1bc INFO: 172.21.0.1:50108 - "POST /node/69408307575bc060ce15d1bc HTTP/1.0" 202 Accepted 2025-12-17 21:52:14,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082bc575bc060ce15d0fe 2025-12-17 21:52:14,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082bc575bc060ce15d0fe HTTP/1.1" 502 150 2025-12-17 21:52:14,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082bc575bc060ce15d0fe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50116 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:52:15,343 - lava_callback - INFO - Received callback for node 69408308575bc060ce15d1c4 INFO: 172.21.0.1:50120 - "POST /node/69408308575bc060ce15d1c4 HTTP/1.0" 202 Accepted 2025-12-17 21:52:15,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082bd575bc060ce15d101 2025-12-17 21:52:16,005 - lava_callback - INFO - Received callback for node 6940830a575bc060ce15d1d2 INFO: 172.21.0.1:50122 - "POST /node/6940830a575bc060ce15d1d2 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:50126 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:52:16,652 - lava_callback - INFO - Received callback for node 6940830c575bc060ce15d1ff INFO: 172.21.0.1:50128 - "POST /node/6940830c575bc060ce15d1ff HTTP/1.0" 202 Accepted 2025-12-17 21:52:17,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b2575bc060ce15d0e7 2025-12-17 21:52:17,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b2575bc060ce15d0e7 HTTP/1.1" 502 150 2025-12-17 21:52:17,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b2575bc060ce15d0e7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:19,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b7575bc060ce15d0f5 2025-12-17 21:52:19,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b7575bc060ce15d0f5 HTTP/1.1" 502 150 2025-12-17 21:52:19,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b7575bc060ce15d0f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:19,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ab575bc060ce15d0d4 2025-12-17 21:52:19,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ab575bc060ce15d0d4 HTTP/1.1" 502 150 2025-12-17 21:52:19,812 - lava_callback - ERROR - Error processing callback for node 694082ab575bc060ce15d0d4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082ab575bc060ce15d0d4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082ab575bc060ce15d0d4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082ab575bc060ce15d0d4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:19,814 - lava_callback - INFO - Starting async job submit for node 694082be575bc060ce15d104 2025-12-17 21:52:19,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:19,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082be575bc060ce15d104 HTTP/1.1" 502 150 2025-12-17 21:52:19,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082be575bc060ce15d104'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:19,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082be575bc060ce15d104 2025-12-17 21:52:19,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082be575bc060ce15d104 HTTP/1.1" 502 150 2025-12-17 21:52:19,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082be575bc060ce15d104'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:19,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ac575bc060ce15d0d7 2025-12-17 21:52:19,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ac575bc060ce15d0d7 HTTP/1.1" 502 150 2025-12-17 21:52:19,848 - lava_callback - ERROR - Error processing callback for node 694082ac575bc060ce15d0d7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082ac575bc060ce15d0d7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082ac575bc060ce15d0d7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082ac575bc060ce15d0d7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:19,850 - lava_callback - INFO - Starting async job submit for node 694082c0575bc060ce15d107 2025-12-17 21:52:19,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:19,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c0575bc060ce15d107 HTTP/1.1" 502 150 2025-12-17 21:52:19,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c0575bc060ce15d107'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:19,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c0575bc060ce15d107 2025-12-17 21:52:19,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c0575bc060ce15d107 HTTP/1.1" 502 150 2025-12-17 21:52:19,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c0575bc060ce15d107'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:20,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a4575bc060ce15d0c2 2025-12-17 21:52:20,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a4575bc060ce15d0c2 HTTP/1.1" 502 150 2025-12-17 21:52:20,534 - lava_callback - ERROR - Error processing callback for node 694082a4575bc060ce15d0c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a4575bc060ce15d0c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a4575bc060ce15d0c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a4575bc060ce15d0c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:20,535 - lava_callback - INFO - Starting async job submit for node 694082c1575bc060ce15d10a 2025-12-17 21:52:20,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:20,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c1575bc060ce15d10a HTTP/1.1" 502 150 2025-12-17 21:52:20,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c1575bc060ce15d10a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:20,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c1575bc060ce15d10a 2025-12-17 21:52:20,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c1575bc060ce15d10a HTTP/1.1" 502 150 2025-12-17 21:52:20,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c1575bc060ce15d10a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:21,070 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b9575bc060ce15d0f8 2025-12-17 21:52:21,071 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ba575bc060ce15d0fb 2025-12-17 21:52:21,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b9575bc060ce15d0f8 HTTP/1.1" 502 150 2025-12-17 21:52:21,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082b9575bc060ce15d0f8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:21,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ba575bc060ce15d0fb HTTP/1.1" 502 150 2025-12-17 21:52:21,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082ba575bc060ce15d0fb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:21,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082be575bc060ce15d104 2025-12-17 21:52:21,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082be575bc060ce15d104 HTTP/1.1" 502 150 2025-12-17 21:52:21,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082be575bc060ce15d104'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:21,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c0575bc060ce15d107 2025-12-17 21:52:21,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c0575bc060ce15d107 HTTP/1.1" 502 150 2025-12-17 21:52:21,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c0575bc060ce15d107'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:22,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c1575bc060ce15d10a 2025-12-17 21:52:22,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c1575bc060ce15d10a HTTP/1.1" 502 150 2025-12-17 21:52:22,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c1575bc060ce15d10a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:23,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ae575bc060ce15d0da 2025-12-17 21:52:23,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ae575bc060ce15d0da HTTP/1.1" 502 150 2025-12-17 21:52:23,890 - lava_callback - ERROR - Error processing callback for node 694082ae575bc060ce15d0da: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082ae575bc060ce15d0da (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082ae575bc060ce15d0da (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082ae575bc060ce15d0da (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:23,892 - lava_callback - INFO - Starting async job submit for node 694082c2575bc060ce15d10d 2025-12-17 21:52:23,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:23,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c2575bc060ce15d10d HTTP/1.1" 502 150 2025-12-17 21:52:23,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c2575bc060ce15d10d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:23,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c2575bc060ce15d10d 2025-12-17 21:52:23,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c2575bc060ce15d10d HTTP/1.1" 502 150 2025-12-17 21:52:23,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c2575bc060ce15d10d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:24,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082a9575bc060ce15d0ce 2025-12-17 21:52:24,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082a9575bc060ce15d0ce HTTP/1.1" 502 150 2025-12-17 21:52:24,370 - lava_callback - ERROR - Error processing callback for node 694082a9575bc060ce15d0ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a9575bc060ce15d0ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a9575bc060ce15d0ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082a9575bc060ce15d0ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:24,372 - lava_callback - INFO - Starting async job submit for node 694082c4575bc060ce15d110 2025-12-17 21:52:24,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:24,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c4575bc060ce15d110 HTTP/1.1" 502 150 2025-12-17 21:52:24,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c4575bc060ce15d110'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:24,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c4575bc060ce15d110 2025-12-17 21:52:24,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c4575bc060ce15d110 HTTP/1.1" 502 150 2025-12-17 21:52:24,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c4575bc060ce15d110'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:25,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082be575bc060ce15d104 2025-12-17 21:52:25,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082be575bc060ce15d104 HTTP/1.1" 502 150 2025-12-17 21:52:25,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082be575bc060ce15d104'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:25,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c0575bc060ce15d107 2025-12-17 21:52:25,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c0575bc060ce15d107 HTTP/1.1" 502 150 2025-12-17 21:52:25,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c0575bc060ce15d107'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:25,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c2575bc060ce15d10d 2025-12-17 21:52:25,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c2575bc060ce15d10d HTTP/1.1" 502 150 2025-12-17 21:52:25,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c2575bc060ce15d10d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:26,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c4575bc060ce15d110 2025-12-17 21:52:26,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c4575bc060ce15d110 HTTP/1.1" 502 150 2025-12-17 21:52:26,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c4575bc060ce15d110'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52570 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:52:26,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c1575bc060ce15d10a 2025-12-17 21:52:26,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c1575bc060ce15d10a HTTP/1.1" 502 150 2025-12-17 21:52:26,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c1575bc060ce15d10a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:27,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082af575bc060ce15d0dd 2025-12-17 21:52:27,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082af575bc060ce15d0dd HTTP/1.1" 502 150 2025-12-17 21:52:27,048 - lava_callback - ERROR - Error processing callback for node 694082af575bc060ce15d0dd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082af575bc060ce15d0dd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082af575bc060ce15d0dd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082af575bc060ce15d0dd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:27,050 - lava_callback - INFO - Starting async job submit for node 694082c5575bc060ce15d113 2025-12-17 21:52:27,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:27,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c5575bc060ce15d113 HTTP/1.1" 502 150 2025-12-17 21:52:27,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c5575bc060ce15d113'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:27,059 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c5575bc060ce15d113 2025-12-17 21:52:27,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c5575bc060ce15d113 HTTP/1.1" 502 150 2025-12-17 21:52:27,060 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c5575bc060ce15d113'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:27,376 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b0575bc060ce15d0e0 2025-12-17 21:52:27,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b0575bc060ce15d0e0 HTTP/1.1" 502 150 2025-12-17 21:52:27,378 - lava_callback - ERROR - Error processing callback for node 694082b0575bc060ce15d0e0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b0575bc060ce15d0e0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b0575bc060ce15d0e0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b0575bc060ce15d0e0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:27,379 - lava_callback - INFO - Starting async job submit for node 694082c9575bc060ce15d12c 2025-12-17 21:52:27,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:27,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c9575bc060ce15d12c HTTP/1.1" 502 150 2025-12-17 21:52:27,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c9575bc060ce15d12c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:27,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c9575bc060ce15d12c 2025-12-17 21:52:27,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c9575bc060ce15d12c HTTP/1.1" 502 150 2025-12-17 21:52:27,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c9575bc060ce15d12c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:27,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b3575bc060ce15d0ec 2025-12-17 21:52:27,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b3575bc060ce15d0ec HTTP/1.1" 502 150 2025-12-17 21:52:27,697 - lava_callback - ERROR - Error processing callback for node 694082b3575bc060ce15d0ec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b3575bc060ce15d0ec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b3575bc060ce15d0ec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b3575bc060ce15d0ec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:27,699 - lava_callback - INFO - Starting async job submit for node 694082cd575bc060ce15d154 2025-12-17 21:52:27,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:27,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cd575bc060ce15d154 HTTP/1.1" 502 150 2025-12-17 21:52:27,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cd575bc060ce15d154'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:27,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cd575bc060ce15d154 2025-12-17 21:52:27,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cd575bc060ce15d154 HTTP/1.1" 502 150 2025-12-17 21:52:27,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cd575bc060ce15d154'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:28,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b5575bc060ce15d0ef 2025-12-17 21:52:28,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b5575bc060ce15d0ef HTTP/1.1" 502 150 2025-12-17 21:52:28,651 - lava_callback - ERROR - Error processing callback for node 694082b5575bc060ce15d0ef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b5575bc060ce15d0ef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b5575bc060ce15d0ef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b5575bc060ce15d0ef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:28,653 - lava_callback - INFO - Starting async job submit for node 694082cf575bc060ce15d168 2025-12-17 21:52:28,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:28,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cf575bc060ce15d168 HTTP/1.1" 502 150 2025-12-17 21:52:28,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cf575bc060ce15d168'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:28,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cf575bc060ce15d168 2025-12-17 21:52:28,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cf575bc060ce15d168 HTTP/1.1" 502 150 2025-12-17 21:52:28,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cf575bc060ce15d168'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:29,060 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c5575bc060ce15d113 2025-12-17 21:52:29,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c5575bc060ce15d113 HTTP/1.1" 502 150 2025-12-17 21:52:29,062 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c5575bc060ce15d113'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:29,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c9575bc060ce15d12c 2025-12-17 21:52:29,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c9575bc060ce15d12c HTTP/1.1" 502 150 2025-12-17 21:52:29,389 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c9575bc060ce15d12c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:29,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b6575bc060ce15d0f2 2025-12-17 21:52:29,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b6575bc060ce15d0f2 HTTP/1.1" 502 150 2025-12-17 21:52:29,524 - lava_callback - ERROR - Error processing callback for node 694082b6575bc060ce15d0f2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b6575bc060ce15d0f2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b6575bc060ce15d0f2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b6575bc060ce15d0f2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:29,526 - lava_callback - INFO - Starting async job submit for node 694082cb575bc060ce15d12f 2025-12-17 21:52:29,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:29,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cb575bc060ce15d12f HTTP/1.1" 502 150 2025-12-17 21:52:29,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cb575bc060ce15d12f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:29,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cb575bc060ce15d12f 2025-12-17 21:52:29,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cb575bc060ce15d12f HTTP/1.1" 502 150 2025-12-17 21:52:29,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cb575bc060ce15d12f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:29,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cd575bc060ce15d154 2025-12-17 21:52:29,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cd575bc060ce15d154 HTTP/1.1" 502 150 2025-12-17 21:52:29,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cd575bc060ce15d154'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:29,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c2575bc060ce15d10d 2025-12-17 21:52:29,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c2575bc060ce15d10d HTTP/1.1" 502 150 2025-12-17 21:52:29,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c2575bc060ce15d10d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:30,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c4575bc060ce15d110 2025-12-17 21:52:30,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c4575bc060ce15d110 HTTP/1.1" 502 150 2025-12-17 21:52:30,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c4575bc060ce15d110'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:30,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cf575bc060ce15d168 2025-12-17 21:52:30,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cf575bc060ce15d168 HTTP/1.1" 502 150 2025-12-17 21:52:30,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cf575bc060ce15d168'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:30,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082bc575bc060ce15d0fe 2025-12-17 21:52:30,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082bc575bc060ce15d0fe HTTP/1.1" 502 150 2025-12-17 21:52:30,700 - lava_callback - ERROR - Error processing callback for node 694082bc575bc060ce15d0fe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082bc575bc060ce15d0fe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082bc575bc060ce15d0fe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082bc575bc060ce15d0fe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:30,703 - lava_callback - INFO - Starting async job submit for node 694082cf575bc060ce15d165 2025-12-17 21:52:30,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:30,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cf575bc060ce15d165 HTTP/1.1" 502 150 2025-12-17 21:52:30,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cf575bc060ce15d165'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:30,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cf575bc060ce15d165 2025-12-17 21:52:30,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cf575bc060ce15d165 HTTP/1.1" 502 150 2025-12-17 21:52:30,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cf575bc060ce15d165'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:31,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082bd575bc060ce15d101 HTTP/1.1" 502 150 2025-12-17 21:52:31,152 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082bd575bc060ce15d101'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:31,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cb575bc060ce15d12f 2025-12-17 21:52:31,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cb575bc060ce15d12f HTTP/1.1" 502 150 2025-12-17 21:52:31,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cb575bc060ce15d12f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:32,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cf575bc060ce15d165 2025-12-17 21:52:32,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cf575bc060ce15d165 HTTP/1.1" 502 150 2025-12-17 21:52:32,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cf575bc060ce15d165'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:33,062 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c5575bc060ce15d113 2025-12-17 21:52:33,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c5575bc060ce15d113 HTTP/1.1" 502 150 2025-12-17 21:52:33,063 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c5575bc060ce15d113'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:33,321 - lava_callback - INFO - Received callback for node 69408315575bc060ce15d20e INFO: 172.21.0.1:43182 - "POST /node/69408315575bc060ce15d20e HTTP/1.0" 202 Accepted 2025-12-17 21:52:33,389 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c9575bc060ce15d12c 2025-12-17 21:52:33,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c9575bc060ce15d12c HTTP/1.1" 502 150 2025-12-17 21:52:33,390 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c9575bc060ce15d12c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:33,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cd575bc060ce15d154 2025-12-17 21:52:33,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cd575bc060ce15d154 HTTP/1.1" 502 150 2025-12-17 21:52:33,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cd575bc060ce15d154'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:33,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082be575bc060ce15d104 2025-12-17 21:52:33,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082be575bc060ce15d104 HTTP/1.1" 502 150 2025-12-17 21:52:33,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082be575bc060ce15d104'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:33,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b2575bc060ce15d0e7 2025-12-17 21:52:33,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b2575bc060ce15d0e7 HTTP/1.1" 502 150 2025-12-17 21:52:33,848 - lava_callback - ERROR - Error processing callback for node 694082b2575bc060ce15d0e7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b2575bc060ce15d0e7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b2575bc060ce15d0e7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b2575bc060ce15d0e7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:33,849 - lava_callback - INFO - Starting async job submit for node 694082d1575bc060ce15d16b 2025-12-17 21:52:33,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:33,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c0575bc060ce15d107 2025-12-17 21:52:33,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c0575bc060ce15d107 HTTP/1.1" 502 150 2025-12-17 21:52:33,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c0575bc060ce15d107'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:33,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d1575bc060ce15d16b HTTP/1.1" 502 150 2025-12-17 21:52:33,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d1575bc060ce15d16b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:33,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d1575bc060ce15d16b 2025-12-17 21:52:33,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d1575bc060ce15d16b HTTP/1.1" 502 150 2025-12-17 21:52:33,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d1575bc060ce15d16b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:33,992 - lava_callback - INFO - Received callback for node 69408316575bc060ce15d215 INFO: 172.21.0.1:43194 - "POST /node/69408316575bc060ce15d215 HTTP/1.0" 202 Accepted 2025-12-17 21:52:34,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c1575bc060ce15d10a 2025-12-17 21:52:34,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c1575bc060ce15d10a HTTP/1.1" 502 150 2025-12-17 21:52:34,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c1575bc060ce15d10a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:34,655 - lava_callback - INFO - Received callback for node 69408317575bc060ce15d21a INFO: 172.21.0.1:43206 - "POST /node/69408317575bc060ce15d21a HTTP/1.0" 202 Accepted 2025-12-17 21:52:34,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cf575bc060ce15d168 2025-12-17 21:52:34,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cf575bc060ce15d168 HTTP/1.1" 502 150 2025-12-17 21:52:34,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cf575bc060ce15d168'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:35,330 - lava_callback - INFO - Received callback for node 69408318575bc060ce15d21d INFO: 172.21.0.1:43220 - "POST /node/69408318575bc060ce15d21d HTTP/1.0" 202 Accepted 2025-12-17 21:52:35,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cb575bc060ce15d12f 2025-12-17 21:52:35,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cb575bc060ce15d12f HTTP/1.1" 502 150 2025-12-17 21:52:35,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cb575bc060ce15d12f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:35,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b7575bc060ce15d0f5 2025-12-17 21:52:35,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b7575bc060ce15d0f5 HTTP/1.1" 502 150 2025-12-17 21:52:35,789 - lava_callback - ERROR - Error processing callback for node 694082b7575bc060ce15d0f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b7575bc060ce15d0f5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b7575bc060ce15d0f5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b7575bc060ce15d0f5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:35,790 - lava_callback - INFO - Starting async job submit for node 694082d3575bc060ce15d171 2025-12-17 21:52:35,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:35,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d3575bc060ce15d171 HTTP/1.1" 502 150 2025-12-17 21:52:35,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d3575bc060ce15d171'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:35,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d3575bc060ce15d171 2025-12-17 21:52:35,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d3575bc060ce15d171 HTTP/1.1" 502 150 2025-12-17 21:52:35,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d3575bc060ce15d171'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:35,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d1575bc060ce15d16b 2025-12-17 21:52:35,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d1575bc060ce15d16b HTTP/1.1" 502 150 2025-12-17 21:52:35,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d1575bc060ce15d16b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:36,037 - lava_callback - INFO - Received callback for node 6940831a575bc060ce15d220 INFO: 172.21.0.1:43232 - "POST /node/6940831a575bc060ce15d220 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:43242 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:52:36,696 - lava_callback - INFO - Received callback for node 6940831c575bc060ce15d223 INFO: 172.21.0.1:43244 - "POST /node/6940831c575bc060ce15d223 HTTP/1.0" 202 Accepted 2025-12-17 21:52:36,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cf575bc060ce15d165 2025-12-17 21:52:36,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cf575bc060ce15d165 HTTP/1.1" 502 150 2025-12-17 21:52:36,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cf575bc060ce15d165'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:37,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082b9575bc060ce15d0f8 2025-12-17 21:52:37,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082ba575bc060ce15d0fb 2025-12-17 21:52:37,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082b9575bc060ce15d0f8 HTTP/1.1" 502 150 2025-12-17 21:52:37,073 - lava_callback - ERROR - Error processing callback for node 694082b9575bc060ce15d0f8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b9575bc060ce15d0f8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b9575bc060ce15d0f8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082b9575bc060ce15d0f8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:37,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082ba575bc060ce15d0fb HTTP/1.1" 502 150 2025-12-17 21:52:37,075 - lava_callback - INFO - Starting async job submit for node 694082d5575bc060ce15d174 2025-12-17 21:52:37,076 - lava_callback - ERROR - Error processing callback for node 694082ba575bc060ce15d0fb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082ba575bc060ce15d0fb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082ba575bc060ce15d0fb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082ba575bc060ce15d0fb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:37,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:37,078 - lava_callback - INFO - Starting async job submit for node 694082c6575bc060ce15d118 2025-12-17 21:52:37,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:37,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d5575bc060ce15d174 HTTP/1.1" 502 150 2025-12-17 21:52:37,091 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d5575bc060ce15d174'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:37,091 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d5575bc060ce15d174 2025-12-17 21:52:37,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d5575bc060ce15d174 HTTP/1.1" 502 150 2025-12-17 21:52:37,091 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d5575bc060ce15d174'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:37,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c6575bc060ce15d118 HTTP/1.1" 502 150 2025-12-17 21:52:37,093 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c6575bc060ce15d118'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:37,093 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c6575bc060ce15d118 2025-12-17 21:52:37,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c6575bc060ce15d118 HTTP/1.1" 502 150 2025-12-17 21:52:37,093 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c6575bc060ce15d118'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:37,348 - lava_callback - INFO - Received callback for node 6940831d575bc060ce15d22b INFO: 172.21.0.1:43256 - "POST /node/6940831d575bc060ce15d22b HTTP/1.0" 202 Accepted 2025-12-17 21:52:37,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d3575bc060ce15d171 2025-12-17 21:52:37,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d3575bc060ce15d171 HTTP/1.1" 502 150 2025-12-17 21:52:37,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d3575bc060ce15d171'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:37,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c2575bc060ce15d10d 2025-12-17 21:52:37,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c2575bc060ce15d10d HTTP/1.1" 502 150 2025-12-17 21:52:37,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c2575bc060ce15d10d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43258 - "GET /login HTTP/1.0" 404 Not Found 2025-12-17 21:52:38,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c4575bc060ce15d110 2025-12-17 21:52:38,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c4575bc060ce15d110 HTTP/1.1" 502 150 2025-12-17 21:52:38,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c4575bc060ce15d110'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:39,092 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d5575bc060ce15d174 2025-12-17 21:52:39,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d5575bc060ce15d174 HTTP/1.1" 502 150 2025-12-17 21:52:39,093 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d5575bc060ce15d174'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:39,094 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c6575bc060ce15d118 2025-12-17 21:52:39,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c6575bc060ce15d118 HTTP/1.1" 502 150 2025-12-17 21:52:39,094 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c6575bc060ce15d118'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:39,153 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082bd575bc060ce15d101 2025-12-17 21:52:39,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082bd575bc060ce15d101 HTTP/1.1" 502 150 2025-12-17 21:52:39,153 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082bd575bc060ce15d101'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:39,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d1575bc060ce15d16b 2025-12-17 21:52:39,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d1575bc060ce15d16b HTTP/1.1" 502 150 2025-12-17 21:52:39,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d1575bc060ce15d16b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:41,063 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c5575bc060ce15d113 2025-12-17 21:52:41,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c5575bc060ce15d113 HTTP/1.1" 502 150 2025-12-17 21:52:41,065 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c5575bc060ce15d113'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:41,390 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c9575bc060ce15d12c 2025-12-17 21:52:41,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c9575bc060ce15d12c HTTP/1.1" 502 150 2025-12-17 21:52:41,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c9575bc060ce15d12c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:41,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cd575bc060ce15d154 2025-12-17 21:52:41,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cd575bc060ce15d154 HTTP/1.1" 502 150 2025-12-17 21:52:41,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cd575bc060ce15d154'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:41,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d3575bc060ce15d171 2025-12-17 21:52:41,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d3575bc060ce15d171 HTTP/1.1" 502 150 2025-12-17 21:52:41,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d3575bc060ce15d171'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:42,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cf575bc060ce15d168 2025-12-17 21:52:43,072 - lava_callback - INFO - Received callback for node 6940831e575bc060ce15d230 INFO: 172.21.0.1:47278 - "POST /node/6940831e575bc060ce15d230 HTTP/1.0" 202 Accepted 2025-12-17 21:52:43,093 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d5575bc060ce15d174 2025-12-17 21:52:43,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d5575bc060ce15d174 HTTP/1.1" 502 150 2025-12-17 21:52:43,094 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d5575bc060ce15d174'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:43,094 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c6575bc060ce15d118 2025-12-17 21:52:43,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c6575bc060ce15d118 HTTP/1.1" 502 150 2025-12-17 21:52:43,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c6575bc060ce15d118'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:43,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cb575bc060ce15d12f 2025-12-17 21:52:43,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cb575bc060ce15d12f HTTP/1.1" 502 150 2025-12-17 21:52:43,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cb575bc060ce15d12f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:43,763 - lava_callback - INFO - Received callback for node 69408320575bc060ce15d237 INFO: 172.21.0.1:47282 - "POST /node/69408320575bc060ce15d237 HTTP/1.0" 202 Accepted 2025-12-17 21:52:44,435 - lava_callback - INFO - Received callback for node 69408329575bc060ce15d24b INFO: 172.21.0.1:47294 - "POST /node/69408329575bc060ce15d24b HTTP/1.0" 202 Accepted 2025-12-17 21:52:44,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cf575bc060ce15d165 2025-12-17 21:52:44,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cf575bc060ce15d165 HTTP/1.1" 502 150 2025-12-17 21:52:44,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cf575bc060ce15d165'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47308 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:52:47,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d1575bc060ce15d16b 2025-12-17 21:52:47,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d1575bc060ce15d16b HTTP/1.1" 502 150 2025-12-17 21:52:47,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d1575bc060ce15d16b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:48,241 - lava_callback - INFO - Received callback for node 69408324575bc060ce15d240 INFO: 172.21.0.1:47310 - "POST /node/69408324575bc060ce15d240 HTTP/1.0" 202 Accepted 2025-12-17 21:52:48,903 - lava_callback - INFO - Received callback for node 69408325575bc060ce15d243 INFO: 172.21.0.1:47314 - "POST /node/69408325575bc060ce15d243 HTTP/1.0" 202 Accepted 2025-12-17 21:52:49,547 - lava_callback - INFO - Received callback for node 69408326575bc060ce15d246 INFO: 172.21.0.1:47326 - "POST /node/69408326575bc060ce15d246 HTTP/1.0" 202 Accepted 2025-12-17 21:52:49,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d3575bc060ce15d171 2025-12-17 21:52:49,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d3575bc060ce15d171 HTTP/1.1" 502 150 2025-12-17 21:52:49,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d3575bc060ce15d171'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:49,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082be575bc060ce15d104 2025-12-17 21:52:49,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082be575bc060ce15d104 HTTP/1.1" 502 150 2025-12-17 21:52:49,835 - lava_callback - ERROR - Error processing callback for node 694082be575bc060ce15d104: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082be575bc060ce15d104 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082be575bc060ce15d104 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082be575bc060ce15d104 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:49,836 - lava_callback - INFO - Starting async job submit for node 694082c8575bc060ce15d123 2025-12-17 21:52:49,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:49,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c8575bc060ce15d123 HTTP/1.1" 502 150 2025-12-17 21:52:49,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c8575bc060ce15d123'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:49,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c8575bc060ce15d123 2025-12-17 21:52:49,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c8575bc060ce15d123 HTTP/1.1" 502 150 2025-12-17 21:52:49,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c8575bc060ce15d123'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:49,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c0575bc060ce15d107 2025-12-17 21:52:49,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c0575bc060ce15d107 HTTP/1.1" 502 150 2025-12-17 21:52:49,862 - lava_callback - ERROR - Error processing callback for node 694082c0575bc060ce15d107: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c0575bc060ce15d107 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c0575bc060ce15d107 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c0575bc060ce15d107 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:49,864 - lava_callback - INFO - Starting async job submit for node 694082de575bc060ce15d197 2025-12-17 21:52:49,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:49,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082de575bc060ce15d197 HTTP/1.1" 502 150 2025-12-17 21:52:49,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082de575bc060ce15d197'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:49,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082de575bc060ce15d197 2025-12-17 21:52:49,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082de575bc060ce15d197 HTTP/1.1" 502 150 2025-12-17 21:52:49,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082de575bc060ce15d197'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:50,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c1575bc060ce15d10a 2025-12-17 21:52:50,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c1575bc060ce15d10a HTTP/1.1" 502 150 2025-12-17 21:52:50,554 - lava_callback - ERROR - Error processing callback for node 694082c1575bc060ce15d10a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c1575bc060ce15d10a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c1575bc060ce15d10a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c1575bc060ce15d10a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:50,556 - lava_callback - INFO - Starting async job submit for node 69408216575bc060ce15cd22 2025-12-17 21:52:50,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:50,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408216575bc060ce15cd22 HTTP/1.1" 502 150 2025-12-17 21:52:50,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408216575bc060ce15cd22'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:50,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408216575bc060ce15cd22 2025-12-17 21:52:50,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408216575bc060ce15cd22 HTTP/1.1" 502 150 2025-12-17 21:52:50,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408216575bc060ce15cd22'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:51,094 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d5575bc060ce15d174 2025-12-17 21:52:51,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d5575bc060ce15d174 HTTP/1.1" 502 150 2025-12-17 21:52:51,096 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d5575bc060ce15d174'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:51,096 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c6575bc060ce15d118 2025-12-17 21:52:51,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c6575bc060ce15d118 HTTP/1.1" 502 150 2025-12-17 21:52:51,097 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c6575bc060ce15d118'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:51,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c8575bc060ce15d123 2025-12-17 21:52:51,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c8575bc060ce15d123 HTTP/1.1" 502 150 2025-12-17 21:52:51,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c8575bc060ce15d123'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:51,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082de575bc060ce15d197 2025-12-17 21:52:51,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082de575bc060ce15d197 HTTP/1.1" 502 150 2025-12-17 21:52:51,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082de575bc060ce15d197'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:52,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408216575bc060ce15cd22 2025-12-17 21:52:52,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408216575bc060ce15cd22 HTTP/1.1" 502 150 2025-12-17 21:52:52,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408216575bc060ce15cd22'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:53,345 - lava_callback - INFO - Received callback for node 69408321575bc060ce15d23a INFO: 172.21.0.1:38204 - "POST /node/69408321575bc060ce15d23a HTTP/1.0" 202 Accepted 2025-12-17 21:52:53,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c2575bc060ce15d10d 2025-12-17 21:52:53,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c2575bc060ce15d10d HTTP/1.1" 502 150 2025-12-17 21:52:53,906 - lava_callback - ERROR - Error processing callback for node 694082c2575bc060ce15d10d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c2575bc060ce15d10d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c2575bc060ce15d10d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c2575bc060ce15d10d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:53,907 - lava_callback - INFO - Starting async job submit for node 694082d2575bc060ce15d16e 2025-12-17 21:52:53,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:53,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d2575bc060ce15d16e HTTP/1.1" 502 150 2025-12-17 21:52:53,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d2575bc060ce15d16e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:53,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d2575bc060ce15d16e 2025-12-17 21:52:53,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d2575bc060ce15d16e HTTP/1.1" 502 150 2025-12-17 21:52:53,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d2575bc060ce15d16e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:54,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c4575bc060ce15d110 2025-12-17 21:52:54,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c4575bc060ce15d110 HTTP/1.1" 502 150 2025-12-17 21:52:54,406 - lava_callback - ERROR - Error processing callback for node 694082c4575bc060ce15d110: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c4575bc060ce15d110 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c4575bc060ce15d110 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c4575bc060ce15d110 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:54,408 - lava_callback - INFO - Starting async job submit for node 694082d5575bc060ce15d177 2025-12-17 21:52:54,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:54,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d5575bc060ce15d177 HTTP/1.1" 502 150 2025-12-17 21:52:54,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d5575bc060ce15d177'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:54,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d5575bc060ce15d177 2025-12-17 21:52:54,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d5575bc060ce15d177 HTTP/1.1" 502 150 2025-12-17 21:52:54,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d5575bc060ce15d177'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:55,154 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082bd575bc060ce15d101 2025-12-17 21:52:55,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082bd575bc060ce15d101 HTTP/1.1" 502 150 2025-12-17 21:52:55,155 - lava_callback - ERROR - Error processing callback for node 694082bd575bc060ce15d101: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082bd575bc060ce15d101 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082bd575bc060ce15d101 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082bd575bc060ce15d101 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:55,158 - lava_callback - INFO - Starting async job submit for node 694082d7575bc060ce15d17a 2025-12-17 21:52:55,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:55,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d7575bc060ce15d17a HTTP/1.1" 502 150 2025-12-17 21:52:55,175 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d7575bc060ce15d17a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:55,175 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d7575bc060ce15d17a 2025-12-17 21:52:55,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d7575bc060ce15d17a HTTP/1.1" 502 150 2025-12-17 21:52:55,176 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d7575bc060ce15d17a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:55,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c8575bc060ce15d123 2025-12-17 21:52:55,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c8575bc060ce15d123 HTTP/1.1" 502 150 2025-12-17 21:52:55,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c8575bc060ce15d123'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:55,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082de575bc060ce15d197 2025-12-17 21:52:55,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082de575bc060ce15d197 HTTP/1.1" 502 150 2025-12-17 21:52:55,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082de575bc060ce15d197'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:55,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d2575bc060ce15d16e 2025-12-17 21:52:55,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d2575bc060ce15d16e HTTP/1.1" 502 150 2025-12-17 21:52:55,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d2575bc060ce15d16e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:56,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d5575bc060ce15d177 2025-12-17 21:52:56,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d5575bc060ce15d177 HTTP/1.1" 502 150 2025-12-17 21:52:56,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d5575bc060ce15d177'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38216 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:52:56,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408216575bc060ce15cd22 2025-12-17 21:52:56,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408216575bc060ce15cd22 HTTP/1.1" 502 150 2025-12-17 21:52:56,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408216575bc060ce15cd22'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:57,065 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c5575bc060ce15d113 2025-12-17 21:52:57,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c5575bc060ce15d113 HTTP/1.1" 502 150 2025-12-17 21:52:57,066 - lava_callback - ERROR - Error processing callback for node 694082c5575bc060ce15d113: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c5575bc060ce15d113 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c5575bc060ce15d113 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c5575bc060ce15d113 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:57,068 - lava_callback - INFO - Starting async job submit for node 694082d8575bc060ce15d17d 2025-12-17 21:52:57,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:57,176 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d7575bc060ce15d17a 2025-12-17 21:52:57,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d7575bc060ce15d17a HTTP/1.1" 502 150 2025-12-17 21:52:57,177 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d7575bc060ce15d17a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:57,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d8575bc060ce15d17d HTTP/1.1" 502 150 2025-12-17 21:52:57,245 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d8575bc060ce15d17d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:57,245 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d8575bc060ce15d17d 2025-12-17 21:52:57,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d8575bc060ce15d17d HTTP/1.1" 502 150 2025-12-17 21:52:57,246 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d8575bc060ce15d17d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:57,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c9575bc060ce15d12c 2025-12-17 21:52:57,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c9575bc060ce15d12c HTTP/1.1" 502 150 2025-12-17 21:52:57,394 - lava_callback - ERROR - Error processing callback for node 694082c9575bc060ce15d12c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c9575bc060ce15d12c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c9575bc060ce15d12c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c9575bc060ce15d12c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:57,397 - lava_callback - INFO - Starting async job submit for node 694082d9575bc060ce15d180 2025-12-17 21:52:57,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:57,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d9575bc060ce15d180 HTTP/1.1" 502 150 2025-12-17 21:52:57,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d9575bc060ce15d180'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:57,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d9575bc060ce15d180 2025-12-17 21:52:57,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d9575bc060ce15d180 HTTP/1.1" 502 150 2025-12-17 21:52:57,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d9575bc060ce15d180'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:57,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cd575bc060ce15d154 2025-12-17 21:52:57,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cd575bc060ce15d154 HTTP/1.1" 502 150 2025-12-17 21:52:57,712 - lava_callback - ERROR - Error processing callback for node 694082cd575bc060ce15d154: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082cd575bc060ce15d154 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082cd575bc060ce15d154 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082cd575bc060ce15d154 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:57,714 - lava_callback - INFO - Starting async job submit for node 694082db575bc060ce15d18d 2025-12-17 21:52:57,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:57,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082db575bc060ce15d18d HTTP/1.1" 502 150 2025-12-17 21:52:57,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082db575bc060ce15d18d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:57,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082db575bc060ce15d18d 2025-12-17 21:52:57,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082db575bc060ce15d18d HTTP/1.1" 502 150 2025-12-17 21:52:57,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082db575bc060ce15d18d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:58,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cf575bc060ce15d168 HTTP/1.1" 502 150 2025-12-17 21:52:58,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082cf575bc060ce15d168'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:59,246 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d8575bc060ce15d17d 2025-12-17 21:52:59,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d8575bc060ce15d17d HTTP/1.1" 502 150 2025-12-17 21:52:59,248 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d8575bc060ce15d17d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:59,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d9575bc060ce15d180 2025-12-17 21:52:59,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d9575bc060ce15d180 HTTP/1.1" 502 150 2025-12-17 21:52:59,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d9575bc060ce15d180'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:59,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cb575bc060ce15d12f 2025-12-17 21:52:59,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cb575bc060ce15d12f HTTP/1.1" 502 150 2025-12-17 21:52:59,541 - lava_callback - ERROR - Error processing callback for node 694082cb575bc060ce15d12f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082cb575bc060ce15d12f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082cb575bc060ce15d12f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082cb575bc060ce15d12f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:52:59,543 - lava_callback - INFO - Starting async job submit for node 694082e0575bc060ce15d1a1 2025-12-17 21:52:59,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:52:59,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e0575bc060ce15d1a1 HTTP/1.1" 502 150 2025-12-17 21:52:59,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e0575bc060ce15d1a1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:59,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e0575bc060ce15d1a1 2025-12-17 21:52:59,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e0575bc060ce15d1a1 HTTP/1.1" 502 150 2025-12-17 21:52:59,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e0575bc060ce15d1a1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:59,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082db575bc060ce15d18d 2025-12-17 21:52:59,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082db575bc060ce15d18d HTTP/1.1" 502 150 2025-12-17 21:52:59,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082db575bc060ce15d18d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:52:59,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d2575bc060ce15d16e 2025-12-17 21:52:59,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d2575bc060ce15d16e HTTP/1.1" 502 150 2025-12-17 21:52:59,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d2575bc060ce15d16e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:00,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d5575bc060ce15d177 2025-12-17 21:53:00,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d5575bc060ce15d177 HTTP/1.1" 502 150 2025-12-17 21:53:00,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d5575bc060ce15d177'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:00,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cf575bc060ce15d165 2025-12-17 21:53:00,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cf575bc060ce15d165 HTTP/1.1" 502 150 2025-12-17 21:53:00,726 - lava_callback - ERROR - Error processing callback for node 694082cf575bc060ce15d165: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082cf575bc060ce15d165 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082cf575bc060ce15d165 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082cf575bc060ce15d165 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:00,728 - lava_callback - INFO - Starting async job submit for node 694082dd575bc060ce15d194 2025-12-17 21:53:00,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:00,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082dd575bc060ce15d194 HTTP/1.1" 502 150 2025-12-17 21:53:00,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082dd575bc060ce15d194'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:00,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082dd575bc060ce15d194 2025-12-17 21:53:00,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082dd575bc060ce15d194 HTTP/1.1" 502 150 2025-12-17 21:53:00,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082dd575bc060ce15d194'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:01,178 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d7575bc060ce15d17a 2025-12-17 21:53:01,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d7575bc060ce15d17a HTTP/1.1" 502 150 2025-12-17 21:53:01,179 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d7575bc060ce15d17a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:01,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e0575bc060ce15d1a1 2025-12-17 21:53:01,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e0575bc060ce15d1a1 HTTP/1.1" 502 150 2025-12-17 21:53:01,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e0575bc060ce15d1a1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48824 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:53:02,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082dd575bc060ce15d194 2025-12-17 21:53:02,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082dd575bc060ce15d194 HTTP/1.1" 502 150 2025-12-17 21:53:02,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082dd575bc060ce15d194'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:03,067 - lava_callback - INFO - Received callback for node 6940832b575bc060ce15d252 INFO: 172.21.0.1:48836 - "POST /node/6940832b575bc060ce15d252 HTTP/1.0" 202 Accepted 2025-12-17 21:53:03,248 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d8575bc060ce15d17d 2025-12-17 21:53:03,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d8575bc060ce15d17d HTTP/1.1" 502 150 2025-12-17 21:53:03,249 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d8575bc060ce15d17d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:03,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d9575bc060ce15d180 2025-12-17 21:53:03,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d9575bc060ce15d180 HTTP/1.1" 502 150 2025-12-17 21:53:03,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d9575bc060ce15d180'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:03,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082db575bc060ce15d18d 2025-12-17 21:53:03,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082db575bc060ce15d18d HTTP/1.1" 502 150 2025-12-17 21:53:03,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082db575bc060ce15d18d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:03,738 - lava_callback - INFO - Received callback for node 6940832c575bc060ce15d259 INFO: 172.21.0.1:48838 - "POST /node/6940832c575bc060ce15d259 HTTP/1.0" 202 Accepted 2025-12-17 21:53:03,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c8575bc060ce15d123 2025-12-17 21:53:03,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c8575bc060ce15d123 HTTP/1.1" 502 150 2025-12-17 21:53:03,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082c8575bc060ce15d123'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:03,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d1575bc060ce15d16b 2025-12-17 21:53:03,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d1575bc060ce15d16b HTTP/1.1" 502 150 2025-12-17 21:53:03,871 - lava_callback - ERROR - Error processing callback for node 694082d1575bc060ce15d16b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d1575bc060ce15d16b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d1575bc060ce15d16b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d1575bc060ce15d16b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:03,874 - lava_callback - INFO - Starting async job submit for node 694082e0575bc060ce15d19e 2025-12-17 21:53:03,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:03,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e0575bc060ce15d19e HTTP/1.1" 502 150 2025-12-17 21:53:03,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e0575bc060ce15d19e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:03,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e0575bc060ce15d19e 2025-12-17 21:53:03,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e0575bc060ce15d19e HTTP/1.1" 502 150 2025-12-17 21:53:03,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e0575bc060ce15d19e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:03,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082de575bc060ce15d197 2025-12-17 21:53:03,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082de575bc060ce15d197 HTTP/1.1" 502 150 2025-12-17 21:53:03,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082de575bc060ce15d197'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:04,416 - lava_callback - INFO - Received callback for node 6940832d575bc060ce15d25c INFO: 172.21.0.1:48848 - "POST /node/6940832d575bc060ce15d25c HTTP/1.0" 202 Accepted 2025-12-17 21:53:04,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408216575bc060ce15cd22 2025-12-17 21:53:04,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408216575bc060ce15cd22 HTTP/1.1" 502 150 2025-12-17 21:53:04,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408216575bc060ce15cd22'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:05,069 - lava_callback - INFO - Received callback for node 6940832e575bc060ce15d25f INFO: 172.21.0.1:48858 - "POST /node/6940832e575bc060ce15d25f HTTP/1.0" 202 Accepted 2025-12-17 21:53:05,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e0575bc060ce15d1a1 2025-12-17 21:53:05,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e0575bc060ce15d1a1 HTTP/1.1" 502 150 2025-12-17 21:53:05,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e0575bc060ce15d1a1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:05,754 - lava_callback - INFO - Received callback for node 6940833d575bc060ce15d27d INFO: 172.21.0.1:48866 - "POST /node/6940833d575bc060ce15d27d HTTP/1.0" 202 Accepted 2025-12-17 21:53:05,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d3575bc060ce15d171 2025-12-17 21:53:05,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d3575bc060ce15d171 HTTP/1.1" 502 150 2025-12-17 21:53:05,803 - lava_callback - ERROR - Error processing callback for node 694082d3575bc060ce15d171: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d3575bc060ce15d171 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d3575bc060ce15d171 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d3575bc060ce15d171 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:05,806 - lava_callback - INFO - Starting async job submit for node 694082e2575bc060ce15d1a4 2025-12-17 21:53:05,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:05,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e0575bc060ce15d19e 2025-12-17 21:53:05,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e0575bc060ce15d19e HTTP/1.1" 502 150 2025-12-17 21:53:05,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e0575bc060ce15d19e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:05,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e2575bc060ce15d1a4 HTTP/1.1" 502 150 2025-12-17 21:53:05,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e2575bc060ce15d1a4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:05,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e2575bc060ce15d1a4 2025-12-17 21:53:06,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e2575bc060ce15d1a4 HTTP/1.1" 502 150 2025-12-17 21:53:06,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e2575bc060ce15d1a4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48874 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:53:06,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082dd575bc060ce15d194 2025-12-17 21:53:06,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082dd575bc060ce15d194 HTTP/1.1" 502 150 2025-12-17 21:53:06,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082dd575bc060ce15d194'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:07,096 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d5575bc060ce15d174 2025-12-17 21:53:07,097 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c6575bc060ce15d118 2025-12-17 21:53:07,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d5575bc060ce15d174 HTTP/1.1" 502 150 2025-12-17 21:53:07,098 - lava_callback - ERROR - Error processing callback for node 694082d5575bc060ce15d174: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d5575bc060ce15d174 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d5575bc060ce15d174 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d5575bc060ce15d174 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:07,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c6575bc060ce15d118 HTTP/1.1" 502 150 2025-12-17 21:53:07,100 - lava_callback - ERROR - Error processing callback for node 694082c6575bc060ce15d118: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c6575bc060ce15d118 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c6575bc060ce15d118 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c6575bc060ce15d118 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:07,100 - lava_callback - INFO - Starting async job submit for node 6940839a575bc060ce15d41e 2025-12-17 21:53:07,107 - lava_callback - INFO - Starting async job submit for node 694082e3575bc060ce15d1a7 2025-12-17 21:53:07,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:07,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:07,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e3575bc060ce15d1a7 HTTP/1.1" 502 150 2025-12-17 21:53:07,130 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e3575bc060ce15d1a7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:07,130 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e3575bc060ce15d1a7 2025-12-17 21:53:07,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e3575bc060ce15d1a7 HTTP/1.1" 502 150 2025-12-17 21:53:07,131 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e3575bc060ce15d1a7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:07,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839a575bc060ce15d41e HTTP/1.1" 502 150 2025-12-17 21:53:07,132 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839a575bc060ce15d41e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:07,132 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839a575bc060ce15d41e 2025-12-17 21:53:07,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839a575bc060ce15d41e HTTP/1.1" 502 150 2025-12-17 21:53:07,132 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839a575bc060ce15d41e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:07,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d2575bc060ce15d16e 2025-12-17 21:53:07,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d2575bc060ce15d16e HTTP/1.1" 502 150 2025-12-17 21:53:07,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d2575bc060ce15d16e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:08,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e2575bc060ce15d1a4 2025-12-17 21:53:08,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e2575bc060ce15d1a4 HTTP/1.1" 502 150 2025-12-17 21:53:08,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e2575bc060ce15d1a4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:08,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d5575bc060ce15d177 2025-12-17 21:53:08,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d5575bc060ce15d177 HTTP/1.1" 502 150 2025-12-17 21:53:08,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d5575bc060ce15d177'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:08,454 - lava_callback - INFO - Received callback for node 69408332575bc060ce15d264 INFO: 172.21.0.1:48884 - "POST /node/69408332575bc060ce15d264 HTTP/1.0" 202 Accepted 2025-12-17 21:53:09,115 - lava_callback - INFO - Received callback for node 69408333575bc060ce15d26b INFO: 172.21.0.1:48898 - "POST /node/69408333575bc060ce15d26b HTTP/1.0" 202 Accepted 2025-12-17 21:53:09,131 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e3575bc060ce15d1a7 2025-12-17 21:53:09,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e3575bc060ce15d1a7 HTTP/1.1" 502 150 2025-12-17 21:53:09,132 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e3575bc060ce15d1a7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:09,133 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839a575bc060ce15d41e 2025-12-17 21:53:09,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839a575bc060ce15d41e HTTP/1.1" 502 150 2025-12-17 21:53:09,133 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839a575bc060ce15d41e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:09,179 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d7575bc060ce15d17a 2025-12-17 21:53:09,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d7575bc060ce15d17a HTTP/1.1" 502 150 2025-12-17 21:53:09,181 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d7575bc060ce15d17a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:09,745 - lava_callback - INFO - Received callback for node 69408334575bc060ce15d270 INFO: 172.21.0.1:51612 - "POST /node/69408334575bc060ce15d270 HTTP/1.0" 202 Accepted 2025-12-17 21:53:09,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e0575bc060ce15d19e 2025-12-17 21:53:09,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e0575bc060ce15d19e HTTP/1.1" 502 150 2025-12-17 21:53:09,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e0575bc060ce15d19e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:10,410 - lava_callback - INFO - Received callback for node 69408336575bc060ce15d275 INFO: 172.21.0.1:51614 - "POST /node/69408336575bc060ce15d275 HTTP/1.0" 202 Accepted 2025-12-17 21:53:11,073 - lava_callback - INFO - Received callback for node 69408337575bc060ce15d278 INFO: 172.21.0.1:51630 - "POST /node/69408337575bc060ce15d278 HTTP/1.0" 202 Accepted 2025-12-17 21:53:11,249 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d8575bc060ce15d17d 2025-12-17 21:53:11,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d8575bc060ce15d17d HTTP/1.1" 502 150 2025-12-17 21:53:11,250 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d8575bc060ce15d17d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:11,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d9575bc060ce15d180 2025-12-17 21:53:11,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d9575bc060ce15d180 HTTP/1.1" 502 150 2025-12-17 21:53:11,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082d9575bc060ce15d180'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:11,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082db575bc060ce15d18d 2025-12-17 21:53:11,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082db575bc060ce15d18d HTTP/1.1" 502 150 2025-12-17 21:53:11,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082db575bc060ce15d18d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:12,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e2575bc060ce15d1a4 2025-12-17 21:53:12,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e2575bc060ce15d1a4 HTTP/1.1" 502 150 2025-12-17 21:53:12,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e2575bc060ce15d1a4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:13,133 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e3575bc060ce15d1a7 2025-12-17 21:53:13,134 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839a575bc060ce15d41e 2025-12-17 21:53:13,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e3575bc060ce15d1a7 HTTP/1.1" 502 150 2025-12-17 21:53:13,134 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e3575bc060ce15d1a7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:13,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839a575bc060ce15d41e HTTP/1.1" 502 150 2025-12-17 21:53:13,135 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839a575bc060ce15d41e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:13,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e0575bc060ce15d1a1 2025-12-17 21:53:13,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e0575bc060ce15d1a1 HTTP/1.1" 502 150 2025-12-17 21:53:13,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e0575bc060ce15d1a1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:14,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082cf575bc060ce15d168 2025-12-17 21:53:14,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082cf575bc060ce15d168 HTTP/1.1" 502 150 2025-12-17 21:53:14,039 - lava_callback - ERROR - Error processing callback for node 694082cf575bc060ce15d168: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082cf575bc060ce15d168 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082cf575bc060ce15d168 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082cf575bc060ce15d168 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:14,041 - lava_callback - INFO - Starting async job submit for node 69408303575bc060ce15d1ac 2025-12-17 21:53:14,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:14,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408303575bc060ce15d1ac HTTP/1.1" 502 150 2025-12-17 21:53:14,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408303575bc060ce15d1ac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:14,056 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408303575bc060ce15d1ac 2025-12-17 21:53:14,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408303575bc060ce15d1ac HTTP/1.1" 502 150 2025-12-17 21:53:14,057 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408303575bc060ce15d1ac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:14,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082dd575bc060ce15d194 2025-12-17 21:53:14,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082dd575bc060ce15d194 HTTP/1.1" 502 150 2025-12-17 21:53:14,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082dd575bc060ce15d194'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51640 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:53:16,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408303575bc060ce15d1ac 2025-12-17 21:53:16,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408303575bc060ce15d1ac HTTP/1.1" 502 150 2025-12-17 21:53:16,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408303575bc060ce15d1ac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51656 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:53:17,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e0575bc060ce15d19e 2025-12-17 21:53:17,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e0575bc060ce15d19e HTTP/1.1" 502 150 2025-12-17 21:53:17,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e0575bc060ce15d19e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:19,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082c8575bc060ce15d123 2025-12-17 21:53:19,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082c8575bc060ce15d123 HTTP/1.1" 502 150 2025-12-17 21:53:19,853 - lava_callback - ERROR - Error processing callback for node 694082c8575bc060ce15d123: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c8575bc060ce15d123 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c8575bc060ce15d123 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082c8575bc060ce15d123 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:19,855 - lava_callback - INFO - Starting async job submit for node 69408305575bc060ce15d1b7 2025-12-17 21:53:19,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:19,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408305575bc060ce15d1b7 HTTP/1.1" 502 150 2025-12-17 21:53:19,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408305575bc060ce15d1b7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:19,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408305575bc060ce15d1b7 2025-12-17 21:53:19,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408305575bc060ce15d1b7 HTTP/1.1" 502 150 2025-12-17 21:53:19,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408305575bc060ce15d1b7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:19,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082de575bc060ce15d197 2025-12-17 21:53:19,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082de575bc060ce15d197 HTTP/1.1" 502 150 2025-12-17 21:53:19,902 - lava_callback - ERROR - Error processing callback for node 694082de575bc060ce15d197: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082de575bc060ce15d197 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082de575bc060ce15d197 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082de575bc060ce15d197 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:19,904 - lava_callback - INFO - Starting async job submit for node 69408307575bc060ce15d1bc 2025-12-17 21:53:19,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:19,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408307575bc060ce15d1bc HTTP/1.1" 502 150 2025-12-17 21:53:19,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408307575bc060ce15d1bc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:19,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408307575bc060ce15d1bc 2025-12-17 21:53:19,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408307575bc060ce15d1bc HTTP/1.1" 502 150 2025-12-17 21:53:19,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408307575bc060ce15d1bc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:20,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e2575bc060ce15d1a4 2025-12-17 21:53:20,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408303575bc060ce15d1ac 2025-12-17 21:53:20,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408303575bc060ce15d1ac HTTP/1.1" 502 150 2025-12-17 21:53:20,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408303575bc060ce15d1ac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:20,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408216575bc060ce15cd22 2025-12-17 21:53:20,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408216575bc060ce15cd22 HTTP/1.1" 502 150 2025-12-17 21:53:20,594 - lava_callback - ERROR - Error processing callback for node 69408216575bc060ce15cd22: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408216575bc060ce15cd22 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408216575bc060ce15cd22 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408216575bc060ce15cd22 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:20,596 - lava_callback - INFO - Starting async job submit for node 69408308575bc060ce15d1c4 2025-12-17 21:53:20,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:20,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408308575bc060ce15d1c4 HTTP/1.1" 502 150 2025-12-17 21:53:20,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408308575bc060ce15d1c4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:20,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408308575bc060ce15d1c4 2025-12-17 21:53:20,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408308575bc060ce15d1c4 HTTP/1.1" 502 150 2025-12-17 21:53:20,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408308575bc060ce15d1c4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:21,135 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e3575bc060ce15d1a7 2025-12-17 21:53:21,135 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839a575bc060ce15d41e 2025-12-17 21:53:21,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e3575bc060ce15d1a7 HTTP/1.1" 502 150 2025-12-17 21:53:21,136 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e3575bc060ce15d1a7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:21,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839a575bc060ce15d41e HTTP/1.1" 502 150 2025-12-17 21:53:21,136 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839a575bc060ce15d41e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:21,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408305575bc060ce15d1b7 2025-12-17 21:53:21,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408305575bc060ce15d1b7 HTTP/1.1" 502 150 2025-12-17 21:53:21,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408305575bc060ce15d1b7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:21,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408307575bc060ce15d1bc 2025-12-17 21:53:21,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408307575bc060ce15d1bc HTTP/1.1" 502 150 2025-12-17 21:53:21,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408307575bc060ce15d1bc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:22,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408308575bc060ce15d1c4 2025-12-17 21:53:23,086 - lava_callback - INFO - Received callback for node 6940833f575bc060ce15d284 INFO: 172.21.0.1:44754 - "POST /node/6940833f575bc060ce15d284 HTTP/1.0" 202 Accepted 2025-12-17 21:53:23,760 - lava_callback - INFO - Received callback for node 69408340575bc060ce15d28b INFO: 172.21.0.1:44760 - "POST /node/69408340575bc060ce15d28b HTTP/1.0" 202 Accepted 2025-12-17 21:53:23,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d2575bc060ce15d16e 2025-12-17 21:53:23,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d2575bc060ce15d16e HTTP/1.1" 502 150 2025-12-17 21:53:23,920 - lava_callback - ERROR - Error processing callback for node 694082d2575bc060ce15d16e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d2575bc060ce15d16e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d2575bc060ce15d16e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d2575bc060ce15d16e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:23,922 - lava_callback - INFO - Starting async job submit for node 6940830a575bc060ce15d1d2 2025-12-17 21:53:23,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:23,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940830a575bc060ce15d1d2 HTTP/1.1" 502 150 2025-12-17 21:53:23,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830a575bc060ce15d1d2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:23,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940830a575bc060ce15d1d2 2025-12-17 21:53:23,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940830a575bc060ce15d1d2 HTTP/1.1" 502 150 2025-12-17 21:53:23,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830a575bc060ce15d1d2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:24,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d5575bc060ce15d177 2025-12-17 21:53:24,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d5575bc060ce15d177 HTTP/1.1" 502 150 2025-12-17 21:53:24,423 - lava_callback - ERROR - Error processing callback for node 694082d5575bc060ce15d177: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d5575bc060ce15d177 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d5575bc060ce15d177 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d5575bc060ce15d177 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:24,425 - lava_callback - INFO - Starting async job submit for node 6940830c575bc060ce15d1ff 2025-12-17 21:53:24,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:24,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940830c575bc060ce15d1ff HTTP/1.1" 502 150 2025-12-17 21:53:24,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830c575bc060ce15d1ff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:24,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940830c575bc060ce15d1ff 2025-12-17 21:53:24,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940830c575bc060ce15d1ff HTTP/1.1" 502 150 2025-12-17 21:53:24,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830c575bc060ce15d1ff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:24,437 - lava_callback - INFO - Received callback for node 69408341575bc060ce15d292 INFO: 172.21.0.1:44772 - "POST /node/69408341575bc060ce15d292 HTTP/1.0" 202 Accepted 2025-12-17 21:53:25,117 - lava_callback - INFO - Received callback for node 69408343575bc060ce15d297 INFO: 172.21.0.1:44778 - "POST /node/69408343575bc060ce15d297 HTTP/1.0" 202 Accepted 2025-12-17 21:53:25,181 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d7575bc060ce15d17a 2025-12-17 21:53:25,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d7575bc060ce15d17a HTTP/1.1" 502 150 2025-12-17 21:53:25,183 - lava_callback - ERROR - Error processing callback for node 694082d7575bc060ce15d17a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d7575bc060ce15d17a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d7575bc060ce15d17a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d7575bc060ce15d17a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:25,184 - lava_callback - INFO - Starting async job submit for node 69408315575bc060ce15d20e 2025-12-17 21:53:25,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:25,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408315575bc060ce15d20e HTTP/1.1" 502 150 2025-12-17 21:53:25,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408315575bc060ce15d20e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:25,192 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408315575bc060ce15d20e 2025-12-17 21:53:25,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408315575bc060ce15d20e HTTP/1.1" 502 150 2025-12-17 21:53:25,193 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408315575bc060ce15d20e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:25,771 - lava_callback - INFO - Received callback for node 69408344575bc060ce15d29a INFO: 172.21.0.1:44790 - "POST /node/69408344575bc060ce15d29a HTTP/1.0" 202 Accepted 2025-12-17 21:53:25,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408305575bc060ce15d1b7 2025-12-17 21:53:25,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408305575bc060ce15d1b7 HTTP/1.1" 502 150 2025-12-17 21:53:25,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408305575bc060ce15d1b7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:25,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408307575bc060ce15d1bc 2025-12-17 21:53:25,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408307575bc060ce15d1bc HTTP/1.1" 502 150 2025-12-17 21:53:25,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408307575bc060ce15d1bc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:25,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940830a575bc060ce15d1d2 2025-12-17 21:53:25,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940830a575bc060ce15d1d2 HTTP/1.1" 502 150 2025-12-17 21:53:25,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830a575bc060ce15d1d2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:26,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940830c575bc060ce15d1ff 2025-12-17 21:53:26,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940830c575bc060ce15d1ff HTTP/1.1" 502 150 2025-12-17 21:53:26,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830c575bc060ce15d1ff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:26,459 - lava_callback - INFO - Received callback for node 69408345575bc060ce15d29d INFO: 172.21.0.1:44806 - "POST /node/69408345575bc060ce15d29d HTTP/1.0" 202 Accepted INFO: 172.21.0.1:44818 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:53:27,146 - lava_callback - INFO - Received callback for node 69408346575bc060ce15d2a0 INFO: 172.21.0.1:44832 - "POST /node/69408346575bc060ce15d2a0 HTTP/1.0" 202 Accepted 2025-12-17 21:53:27,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408315575bc060ce15d20e 2025-12-17 21:53:27,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408315575bc060ce15d20e HTTP/1.1" 502 150 2025-12-17 21:53:27,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408315575bc060ce15d20e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:27,251 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d8575bc060ce15d17d 2025-12-17 21:53:27,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d8575bc060ce15d17d HTTP/1.1" 502 150 2025-12-17 21:53:27,252 - lava_callback - ERROR - Error processing callback for node 694082d8575bc060ce15d17d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d8575bc060ce15d17d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d8575bc060ce15d17d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d8575bc060ce15d17d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:27,253 - lava_callback - INFO - Starting async job submit for node 69408316575bc060ce15d215 2025-12-17 21:53:27,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:27,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408316575bc060ce15d215 HTTP/1.1" 502 150 2025-12-17 21:53:27,261 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408316575bc060ce15d215'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:27,261 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408316575bc060ce15d215 2025-12-17 21:53:27,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408316575bc060ce15d215 HTTP/1.1" 502 150 2025-12-17 21:53:27,262 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408316575bc060ce15d215'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:27,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082d9575bc060ce15d180 2025-12-17 21:53:27,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082d9575bc060ce15d180 HTTP/1.1" 502 150 2025-12-17 21:53:27,415 - lava_callback - ERROR - Error processing callback for node 694082d9575bc060ce15d180: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d9575bc060ce15d180 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d9575bc060ce15d180 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082d9575bc060ce15d180 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:27,417 - lava_callback - INFO - Starting async job submit for node 69408317575bc060ce15d21a 2025-12-17 21:53:27,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:27,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408317575bc060ce15d21a HTTP/1.1" 502 150 2025-12-17 21:53:27,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408317575bc060ce15d21a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:27,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408317575bc060ce15d21a 2025-12-17 21:53:27,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408317575bc060ce15d21a HTTP/1.1" 502 150 2025-12-17 21:53:27,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408317575bc060ce15d21a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:27,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082db575bc060ce15d18d 2025-12-17 21:53:27,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082db575bc060ce15d18d HTTP/1.1" 502 150 2025-12-17 21:53:27,730 - lava_callback - ERROR - Error processing callback for node 694082db575bc060ce15d18d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082db575bc060ce15d18d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082db575bc060ce15d18d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082db575bc060ce15d18d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:27,732 - lava_callback - INFO - Starting async job submit for node 69408318575bc060ce15d21d 2025-12-17 21:53:27,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:27,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408318575bc060ce15d21d HTTP/1.1" 502 150 2025-12-17 21:53:27,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408318575bc060ce15d21d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:27,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408318575bc060ce15d21d 2025-12-17 21:53:27,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408318575bc060ce15d21d HTTP/1.1" 502 150 2025-12-17 21:53:27,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408318575bc060ce15d21d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:27,829 - lava_callback - INFO - Received callback for node 69408351575bc060ce15d2d4 INFO: 172.21.0.1:44848 - "POST /node/69408351575bc060ce15d2d4 HTTP/1.0" 202 Accepted 2025-12-17 21:53:28,060 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408303575bc060ce15d1ac 2025-12-17 21:53:28,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408303575bc060ce15d1ac HTTP/1.1" 502 150 2025-12-17 21:53:28,065 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408303575bc060ce15d1ac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:29,262 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408316575bc060ce15d215 2025-12-17 21:53:29,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408316575bc060ce15d215 HTTP/1.1" 502 150 2025-12-17 21:53:29,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408316575bc060ce15d215'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:29,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408317575bc060ce15d21a 2025-12-17 21:53:29,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408317575bc060ce15d21a HTTP/1.1" 502 150 2025-12-17 21:53:29,425 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408317575bc060ce15d21a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:29,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e0575bc060ce15d1a1 2025-12-17 21:53:29,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e0575bc060ce15d1a1 HTTP/1.1" 502 150 2025-12-17 21:53:29,559 - lava_callback - ERROR - Error processing callback for node 694082e0575bc060ce15d1a1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082e0575bc060ce15d1a1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082e0575bc060ce15d1a1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082e0575bc060ce15d1a1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:29,560 - lava_callback - INFO - Starting async job submit for node 6940831a575bc060ce15d220 2025-12-17 21:53:29,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:29,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408318575bc060ce15d21d 2025-12-17 21:53:29,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408318575bc060ce15d21d HTTP/1.1" 502 150 2025-12-17 21:53:29,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408318575bc060ce15d21d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:29,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831a575bc060ce15d220 HTTP/1.1" 502 150 2025-12-17 21:53:29,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831a575bc060ce15d220'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:29,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831a575bc060ce15d220 2025-12-17 21:53:29,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831a575bc060ce15d220 HTTP/1.1" 502 150 2025-12-17 21:53:29,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831a575bc060ce15d220'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:29,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940830a575bc060ce15d1d2 2025-12-17 21:53:29,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940830a575bc060ce15d1d2 HTTP/1.1" 502 150 2025-12-17 21:53:29,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830a575bc060ce15d1d2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:30,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940830c575bc060ce15d1ff 2025-12-17 21:53:30,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940830c575bc060ce15d1ff HTTP/1.1" 502 150 2025-12-17 21:53:30,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830c575bc060ce15d1ff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:30,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082dd575bc060ce15d194 2025-12-17 21:53:30,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082dd575bc060ce15d194 HTTP/1.1" 502 150 2025-12-17 21:53:30,913 - lava_callback - ERROR - Error processing callback for node 694082dd575bc060ce15d194: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082dd575bc060ce15d194 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082dd575bc060ce15d194 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082dd575bc060ce15d194 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:30,916 - lava_callback - INFO - Starting async job submit for node 6940831c575bc060ce15d223 2025-12-17 21:53:30,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:30,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831c575bc060ce15d223 HTTP/1.1" 502 150 2025-12-17 21:53:30,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831c575bc060ce15d223'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:30,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831c575bc060ce15d223 2025-12-17 21:53:30,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831c575bc060ce15d223 HTTP/1.1" 502 150 2025-12-17 21:53:30,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831c575bc060ce15d223'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:31,194 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408315575bc060ce15d20e 2025-12-17 21:53:31,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831a575bc060ce15d220 2025-12-17 21:53:31,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831a575bc060ce15d220 HTTP/1.1" 502 150 2025-12-17 21:53:31,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831a575bc060ce15d220'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53054 - "GET /favicon.ico HTTP/1.0" 404 Not Found 2025-12-17 21:53:32,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831c575bc060ce15d223 2025-12-17 21:53:32,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831c575bc060ce15d223 HTTP/1.1" 502 150 2025-12-17 21:53:32,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831c575bc060ce15d223'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:33,156 - lava_callback - INFO - Received callback for node 6940839c575bc060ce15d421 INFO: 172.21.0.1:53056 - "POST /node/6940839c575bc060ce15d421 HTTP/1.0" 202 Accepted 2025-12-17 21:53:33,264 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408316575bc060ce15d215 2025-12-17 21:53:33,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408316575bc060ce15d215 HTTP/1.1" 502 150 2025-12-17 21:53:33,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408316575bc060ce15d215'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:33,383 - lava_callback - INFO - Received callback for node 69408356575bc060ce15d2ea INFO: 172.21.0.1:53070 - "POST /node/69408356575bc060ce15d2ea HTTP/1.0" 202 Accepted 2025-12-17 21:53:33,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408317575bc060ce15d21a 2025-12-17 21:53:33,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408317575bc060ce15d21a HTTP/1.1" 502 150 2025-12-17 21:53:33,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408317575bc060ce15d21a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:33,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408318575bc060ce15d21d 2025-12-17 21:53:33,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408318575bc060ce15d21d HTTP/1.1" 502 150 2025-12-17 21:53:33,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408318575bc060ce15d21d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:33,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408305575bc060ce15d1b7 2025-12-17 21:53:33,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408305575bc060ce15d1b7 HTTP/1.1" 502 150 2025-12-17 21:53:33,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408305575bc060ce15d1b7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:33,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e0575bc060ce15d19e 2025-12-17 21:53:33,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e0575bc060ce15d19e HTTP/1.1" 502 150 2025-12-17 21:53:33,891 - lava_callback - ERROR - Error processing callback for node 694082e0575bc060ce15d19e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082e0575bc060ce15d19e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082e0575bc060ce15d19e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082e0575bc060ce15d19e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:33,893 - lava_callback - INFO - Starting async job submit for node 6940831d575bc060ce15d22b 2025-12-17 21:53:33,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:33,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831d575bc060ce15d22b HTTP/1.1" 502 150 2025-12-17 21:53:33,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831d575bc060ce15d22b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:33,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831d575bc060ce15d22b 2025-12-17 21:53:33,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831d575bc060ce15d22b HTTP/1.1" 502 150 2025-12-17 21:53:33,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831d575bc060ce15d22b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:33,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408307575bc060ce15d1bc 2025-12-17 21:53:33,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408307575bc060ce15d1bc HTTP/1.1" 502 150 2025-12-17 21:53:33,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408307575bc060ce15d1bc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:35,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e2575bc060ce15d1a4 HTTP/1.1" 502 150 2025-12-17 21:53:35,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694082e2575bc060ce15d1a4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:35,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831a575bc060ce15d220 2025-12-17 21:53:35,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831a575bc060ce15d220 HTTP/1.1" 502 150 2025-12-17 21:53:35,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831a575bc060ce15d220'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:35,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831d575bc060ce15d22b 2025-12-17 21:53:35,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831d575bc060ce15d22b HTTP/1.1" 502 150 2025-12-17 21:53:35,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831d575bc060ce15d22b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53078 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:53:36,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831c575bc060ce15d223 2025-12-17 21:53:36,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831c575bc060ce15d223 HTTP/1.1" 502 150 2025-12-17 21:53:36,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831c575bc060ce15d223'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:37,136 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e3575bc060ce15d1a7 2025-12-17 21:53:37,137 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839a575bc060ce15d41e 2025-12-17 21:53:37,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e3575bc060ce15d1a7 HTTP/1.1" 502 150 2025-12-17 21:53:37,138 - lava_callback - ERROR - Error processing callback for node 694082e3575bc060ce15d1a7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082e3575bc060ce15d1a7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082e3575bc060ce15d1a7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082e3575bc060ce15d1a7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:37,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839a575bc060ce15d41e HTTP/1.1" 502 150 2025-12-17 21:53:37,141 - lava_callback - ERROR - Error processing callback for node 6940839a575bc060ce15d41e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940839a575bc060ce15d41e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940839a575bc060ce15d41e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940839a575bc060ce15d41e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:37,141 - lava_callback - INFO - Starting async job submit for node 6940831e575bc060ce15d230 2025-12-17 21:53:37,143 - lava_callback - INFO - Starting async job submit for node 69408320575bc060ce15d237 2025-12-17 21:53:37,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:37,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:37,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408320575bc060ce15d237 HTTP/1.1" 502 150 2025-12-17 21:53:37,152 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408320575bc060ce15d237'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:37,152 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408320575bc060ce15d237 2025-12-17 21:53:37,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408320575bc060ce15d237 HTTP/1.1" 502 150 2025-12-17 21:53:37,153 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408320575bc060ce15d237'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:37,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831e575bc060ce15d230 HTTP/1.1" 502 150 2025-12-17 21:53:37,153 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831e575bc060ce15d230'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:37,153 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831e575bc060ce15d230 2025-12-17 21:53:37,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831e575bc060ce15d230 HTTP/1.1" 502 150 2025-12-17 21:53:37,154 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831e575bc060ce15d230'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:37,307 - lava_callback - INFO - Received callback for node 69408f95575bc060ce1601b0 INFO: 172.21.0.1:53094 - "POST /node/69408f95575bc060ce1601b0 HTTP/1.0" 202 Accepted 2025-12-17 21:53:37,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940830a575bc060ce15d1d2 2025-12-17 21:53:37,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940830a575bc060ce15d1d2 HTTP/1.1" 502 150 2025-12-17 21:53:37,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830a575bc060ce15d1d2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:37,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408308575bc060ce15d1c4 HTTP/1.1" 502 150 2025-12-17 21:53:37,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408308575bc060ce15d1c4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:38,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940830c575bc060ce15d1ff 2025-12-17 21:53:38,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940830c575bc060ce15d1ff HTTP/1.1" 502 150 2025-12-17 21:53:38,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940830c575bc060ce15d1ff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:39,153 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408320575bc060ce15d237 2025-12-17 21:53:39,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408320575bc060ce15d237 HTTP/1.1" 502 150 2025-12-17 21:53:39,154 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831e575bc060ce15d230 2025-12-17 21:53:39,154 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408320575bc060ce15d237'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:39,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831e575bc060ce15d230 HTTP/1.1" 502 150 2025-12-17 21:53:39,155 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831e575bc060ce15d230'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:39,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831d575bc060ce15d22b 2025-12-17 21:53:39,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831d575bc060ce15d22b HTTP/1.1" 502 150 2025-12-17 21:53:39,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831d575bc060ce15d22b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:41,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408316575bc060ce15d215 2025-12-17 21:53:41,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408316575bc060ce15d215 HTTP/1.1" 502 150 2025-12-17 21:53:41,267 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408316575bc060ce15d215'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:41,427 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408317575bc060ce15d21a 2025-12-17 21:53:41,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408317575bc060ce15d21a HTTP/1.1" 502 150 2025-12-17 21:53:41,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408317575bc060ce15d21a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:41,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408318575bc060ce15d21d 2025-12-17 21:53:41,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408318575bc060ce15d21d HTTP/1.1" 502 150 2025-12-17 21:53:41,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408318575bc060ce15d21d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:41,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408308575bc060ce15d1c4 2025-12-17 21:53:41,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408308575bc060ce15d1c4 HTTP/1.1" 502 150 2025-12-17 21:53:41,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408308575bc060ce15d1c4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:43,085 - lava_callback - INFO - Received callback for node 69408353575bc060ce15d2dd INFO: 172.21.0.1:35166 - "POST /node/69408353575bc060ce15d2dd HTTP/1.0" 202 Accepted 2025-12-17 21:53:43,155 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408320575bc060ce15d237 2025-12-17 21:53:43,155 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831e575bc060ce15d230 2025-12-17 21:53:43,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408320575bc060ce15d237 HTTP/1.1" 502 150 2025-12-17 21:53:43,156 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408320575bc060ce15d237'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:43,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831e575bc060ce15d230 HTTP/1.1" 502 150 2025-12-17 21:53:43,157 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831e575bc060ce15d230'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:43,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831a575bc060ce15d220 2025-12-17 21:53:43,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831a575bc060ce15d220 HTTP/1.1" 502 150 2025-12-17 21:53:43,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831a575bc060ce15d220'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:43,765 - lava_callback - INFO - Received callback for node 69408354575bc060ce15d2e4 INFO: 172.21.0.1:35180 - "POST /node/69408354575bc060ce15d2e4 HTTP/1.0" 202 Accepted 2025-12-17 21:53:44,065 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408303575bc060ce15d1ac 2025-12-17 21:53:44,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408303575bc060ce15d1ac HTTP/1.1" 502 150 2025-12-17 21:53:44,067 - lava_callback - ERROR - Error processing callback for node 69408303575bc060ce15d1ac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408303575bc060ce15d1ac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408303575bc060ce15d1ac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408303575bc060ce15d1ac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:44,069 - lava_callback - INFO - Starting async job submit for node 69408329575bc060ce15d24b 2025-12-17 21:53:44,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:44,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408329575bc060ce15d24b HTTP/1.1" 502 150 2025-12-17 21:53:44,079 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408329575bc060ce15d24b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:44,080 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408329575bc060ce15d24b 2025-12-17 21:53:44,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408329575bc060ce15d24b HTTP/1.1" 502 150 2025-12-17 21:53:44,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408329575bc060ce15d24b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:44,460 - lava_callback - INFO - Received callback for node 69408358575bc060ce15d2ed INFO: 172.21.0.1:35196 - "POST /node/69408358575bc060ce15d2ed HTTP/1.0" 202 Accepted 2025-12-17 21:53:44,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831c575bc060ce15d223 2025-12-17 21:53:44,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831c575bc060ce15d223 HTTP/1.1" 502 150 2025-12-17 21:53:44,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831c575bc060ce15d223'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:45,108 - lava_callback - INFO - Received callback for node 69408359575bc060ce15d2f0 INFO: 172.21.0.1:35200 - "POST /node/69408359575bc060ce15d2f0 HTTP/1.0" 202 Accepted 2025-12-17 21:53:45,778 - lava_callback - INFO - Received callback for node 6940835b575bc060ce15d2f3 INFO: 172.21.0.1:35216 - "POST /node/6940835b575bc060ce15d2f3 HTTP/1.0" 202 Accepted 2025-12-17 21:53:46,081 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408329575bc060ce15d24b INFO: 172.21.0.1:35232 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:53:46,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408315575bc060ce15d20e HTTP/1.1" 502 150 2025-12-17 21:53:46,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408315575bc060ce15d20e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:47,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831d575bc060ce15d22b 2025-12-17 21:53:47,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831d575bc060ce15d22b HTTP/1.1" 502 150 2025-12-17 21:53:47,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831d575bc060ce15d22b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:49,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408305575bc060ce15d1b7 2025-12-17 21:53:49,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408305575bc060ce15d1b7 HTTP/1.1" 502 150 2025-12-17 21:53:49,873 - lava_callback - ERROR - Error processing callback for node 69408305575bc060ce15d1b7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408305575bc060ce15d1b7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408305575bc060ce15d1b7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408305575bc060ce15d1b7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:49,875 - lava_callback - INFO - Starting async job submit for node 69408324575bc060ce15d240 2025-12-17 21:53:49,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:49,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408307575bc060ce15d1bc 2025-12-17 21:53:49,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408307575bc060ce15d1bc HTTP/1.1" 502 150 2025-12-17 21:53:49,917 - lava_callback - ERROR - Error processing callback for node 69408307575bc060ce15d1bc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408307575bc060ce15d1bc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408307575bc060ce15d1bc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408307575bc060ce15d1bc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:49,920 - lava_callback - INFO - Starting async job submit for node 69408325575bc060ce15d243 2025-12-17 21:53:49,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:49,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408325575bc060ce15d243 HTTP/1.1" 502 150 2025-12-17 21:53:49,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408325575bc060ce15d243'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:49,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408325575bc060ce15d243 2025-12-17 21:53:49,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408325575bc060ce15d243 HTTP/1.1" 502 150 2025-12-17 21:53:49,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408325575bc060ce15d243'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:49,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408324575bc060ce15d240 HTTP/1.1" 502 150 2025-12-17 21:53:49,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408324575bc060ce15d240'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:49,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408324575bc060ce15d240 2025-12-17 21:53:49,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408324575bc060ce15d240 HTTP/1.1" 502 150 2025-12-17 21:53:49,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408324575bc060ce15d240'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:49,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408308575bc060ce15d1c4 2025-12-17 21:53:49,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408308575bc060ce15d1c4 HTTP/1.1" 502 150 2025-12-17 21:53:49,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408308575bc060ce15d1c4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:51,156 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408320575bc060ce15d237 2025-12-17 21:53:51,157 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831e575bc060ce15d230 2025-12-17 21:53:51,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408320575bc060ce15d237 HTTP/1.1" 502 150 2025-12-17 21:53:51,158 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408320575bc060ce15d237'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:51,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831e575bc060ce15d230 HTTP/1.1" 502 150 2025-12-17 21:53:51,158 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940831e575bc060ce15d230'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:51,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694082e2575bc060ce15d1a4 2025-12-17 21:53:51,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694082e2575bc060ce15d1a4 HTTP/1.1" 502 150 2025-12-17 21:53:51,410 - lava_callback - ERROR - Error processing callback for node 694082e2575bc060ce15d1a4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082e2575bc060ce15d1a4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082e2575bc060ce15d1a4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694082e2575bc060ce15d1a4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:51,412 - lava_callback - INFO - Starting async job submit for node 69408326575bc060ce15d246 2025-12-17 21:53:51,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:51,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408326575bc060ce15d246 HTTP/1.1" 502 150 2025-12-17 21:53:51,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408326575bc060ce15d246'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:51,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408326575bc060ce15d246 2025-12-17 21:53:51,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408326575bc060ce15d246 HTTP/1.1" 502 150 2025-12-17 21:53:51,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408326575bc060ce15d246'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:51,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408325575bc060ce15d243 2025-12-17 21:53:51,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408325575bc060ce15d243 HTTP/1.1" 502 150 2025-12-17 21:53:51,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408324575bc060ce15d240 2025-12-17 21:53:51,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408325575bc060ce15d243'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:51,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408324575bc060ce15d240 HTTP/1.1" 502 150 2025-12-17 21:53:51,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408324575bc060ce15d240'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:53,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408326575bc060ce15d246 2025-12-17 21:53:53,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408326575bc060ce15d246 HTTP/1.1" 502 150 2025-12-17 21:53:53,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408326575bc060ce15d246'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:53,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940830a575bc060ce15d1d2 2025-12-17 21:53:53,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940830a575bc060ce15d1d2 HTTP/1.1" 502 150 2025-12-17 21:53:53,936 - lava_callback - ERROR - Error processing callback for node 6940830a575bc060ce15d1d2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940830a575bc060ce15d1d2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940830a575bc060ce15d1d2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940830a575bc060ce15d1d2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:53,938 - lava_callback - INFO - Starting async job submit for node 69408321575bc060ce15d23a 2025-12-17 21:53:53,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:53,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408321575bc060ce15d23a HTTP/1.1" 502 150 2025-12-17 21:53:53,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408321575bc060ce15d23a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:53,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408321575bc060ce15d23a 2025-12-17 21:53:53,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408321575bc060ce15d23a HTTP/1.1" 502 150 2025-12-17 21:53:53,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408321575bc060ce15d23a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:54,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940830c575bc060ce15d1ff 2025-12-17 21:53:54,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940830c575bc060ce15d1ff HTTP/1.1" 502 150 2025-12-17 21:53:54,438 - lava_callback - ERROR - Error processing callback for node 6940830c575bc060ce15d1ff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940830c575bc060ce15d1ff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940830c575bc060ce15d1ff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940830c575bc060ce15d1ff (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:54,440 - lava_callback - INFO - Starting async job submit for node 6940832b575bc060ce15d252 2025-12-17 21:53:54,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:54,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832b575bc060ce15d252 HTTP/1.1" 502 150 2025-12-17 21:53:54,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832b575bc060ce15d252'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:54,447 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832b575bc060ce15d252 2025-12-17 21:53:54,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832b575bc060ce15d252 HTTP/1.1" 502 150 2025-12-17 21:53:54,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832b575bc060ce15d252'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:54,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408315575bc060ce15d20e 2025-12-17 21:53:54,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408315575bc060ce15d20e HTTP/1.1" 502 150 2025-12-17 21:53:54,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408315575bc060ce15d20e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:55,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408325575bc060ce15d243 2025-12-17 21:53:55,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408324575bc060ce15d240 2025-12-17 21:53:55,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408325575bc060ce15d243 HTTP/1.1" 502 150 2025-12-17 21:53:55,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408325575bc060ce15d243'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:55,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408324575bc060ce15d240 HTTP/1.1" 502 150 2025-12-17 21:53:55,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408324575bc060ce15d240'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:55,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408321575bc060ce15d23a 2025-12-17 21:53:55,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408321575bc060ce15d23a HTTP/1.1" 502 150 2025-12-17 21:53:55,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408321575bc060ce15d23a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:56,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832b575bc060ce15d252 2025-12-17 21:53:56,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832b575bc060ce15d252 HTTP/1.1" 502 150 2025-12-17 21:53:56,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832b575bc060ce15d252'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35538 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:53:57,267 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408316575bc060ce15d215 2025-12-17 21:53:57,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408316575bc060ce15d215 HTTP/1.1" 502 150 2025-12-17 21:53:57,269 - lava_callback - ERROR - Error processing callback for node 69408316575bc060ce15d215: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408316575bc060ce15d215 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408316575bc060ce15d215 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408316575bc060ce15d215 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:57,271 - lava_callback - INFO - Starting async job submit for node 6940832c575bc060ce15d259 2025-12-17 21:53:57,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:57,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832c575bc060ce15d259 HTTP/1.1" 502 150 2025-12-17 21:53:57,300 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832c575bc060ce15d259'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:57,301 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832c575bc060ce15d259 2025-12-17 21:53:57,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832c575bc060ce15d259 HTTP/1.1" 502 150 2025-12-17 21:53:57,301 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832c575bc060ce15d259'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:57,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408317575bc060ce15d21a 2025-12-17 21:53:57,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408317575bc060ce15d21a HTTP/1.1" 502 150 2025-12-17 21:53:57,429 - lava_callback - ERROR - Error processing callback for node 69408317575bc060ce15d21a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408317575bc060ce15d21a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408317575bc060ce15d21a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408317575bc060ce15d21a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:57,431 - lava_callback - INFO - Starting async job submit for node 6940832d575bc060ce15d25c 2025-12-17 21:53:57,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:57,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832d575bc060ce15d25c HTTP/1.1" 502 150 2025-12-17 21:53:57,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832d575bc060ce15d25c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:57,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832d575bc060ce15d25c 2025-12-17 21:53:57,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832d575bc060ce15d25c HTTP/1.1" 502 150 2025-12-17 21:53:57,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832d575bc060ce15d25c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:57,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408326575bc060ce15d246 2025-12-17 21:53:57,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408326575bc060ce15d246 HTTP/1.1" 502 150 2025-12-17 21:53:57,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408326575bc060ce15d246'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:57,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408318575bc060ce15d21d 2025-12-17 21:53:57,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408318575bc060ce15d21d HTTP/1.1" 502 150 2025-12-17 21:53:57,745 - lava_callback - ERROR - Error processing callback for node 69408318575bc060ce15d21d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408318575bc060ce15d21d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408318575bc060ce15d21d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408318575bc060ce15d21d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:57,747 - lava_callback - INFO - Starting async job submit for node 6940832e575bc060ce15d25f 2025-12-17 21:53:57,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:57,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832e575bc060ce15d25f HTTP/1.1" 502 150 2025-12-17 21:53:57,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832e575bc060ce15d25f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:57,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832e575bc060ce15d25f 2025-12-17 21:53:57,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832e575bc060ce15d25f HTTP/1.1" 502 150 2025-12-17 21:53:57,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832e575bc060ce15d25f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:59,301 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832c575bc060ce15d259 2025-12-17 21:53:59,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832c575bc060ce15d259 HTTP/1.1" 502 150 2025-12-17 21:53:59,303 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832c575bc060ce15d259'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:59,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832d575bc060ce15d25c 2025-12-17 21:53:59,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832d575bc060ce15d25c HTTP/1.1" 502 150 2025-12-17 21:53:59,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832d575bc060ce15d25c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:59,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831a575bc060ce15d220 2025-12-17 21:53:59,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831a575bc060ce15d220 HTTP/1.1" 502 150 2025-12-17 21:53:59,758 - lava_callback - ERROR - Error processing callback for node 6940831a575bc060ce15d220: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940831a575bc060ce15d220 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940831a575bc060ce15d220 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940831a575bc060ce15d220 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:53:59,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832e575bc060ce15d25f 2025-12-17 21:53:59,760 - lava_callback - INFO - Starting async job submit for node 6940833d575bc060ce15d27d 2025-12-17 21:53:59,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:53:59,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832e575bc060ce15d25f HTTP/1.1" 502 150 2025-12-17 21:53:59,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832e575bc060ce15d25f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:59,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940833d575bc060ce15d27d HTTP/1.1" 502 150 2025-12-17 21:53:59,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940833d575bc060ce15d27d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:59,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940833d575bc060ce15d27d 2025-12-17 21:53:59,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940833d575bc060ce15d27d HTTP/1.1" 502 150 2025-12-17 21:53:59,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940833d575bc060ce15d27d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:53:59,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408321575bc060ce15d23a 2025-12-17 21:53:59,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408321575bc060ce15d23a HTTP/1.1" 502 150 2025-12-17 21:53:59,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408321575bc060ce15d23a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:00,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832b575bc060ce15d252 2025-12-17 21:54:00,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832b575bc060ce15d252 HTTP/1.1" 502 150 2025-12-17 21:54:00,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832b575bc060ce15d252'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:00,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831c575bc060ce15d223 2025-12-17 21:54:00,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831c575bc060ce15d223 HTTP/1.1" 502 150 2025-12-17 21:54:00,937 - lava_callback - ERROR - Error processing callback for node 6940831c575bc060ce15d223: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940831c575bc060ce15d223 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940831c575bc060ce15d223 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940831c575bc060ce15d223 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:00,939 - lava_callback - INFO - Starting async job submit for node 69408332575bc060ce15d264 2025-12-17 21:54:00,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:00,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408332575bc060ce15d264 HTTP/1.1" 502 150 2025-12-17 21:54:00,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408332575bc060ce15d264'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:00,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408332575bc060ce15d264 2025-12-17 21:54:00,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408332575bc060ce15d264 HTTP/1.1" 502 150 2025-12-17 21:54:00,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408332575bc060ce15d264'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:01,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408329575bc060ce15d24b HTTP/1.1" 502 150 2025-12-17 21:54:01,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408329575bc060ce15d24b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:01,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940833d575bc060ce15d27d 2025-12-17 21:54:01,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940833d575bc060ce15d27d HTTP/1.1" 502 150 2025-12-17 21:54:01,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940833d575bc060ce15d27d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54344 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:54:02,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408332575bc060ce15d264 2025-12-17 21:54:02,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408332575bc060ce15d264 HTTP/1.1" 502 150 2025-12-17 21:54:02,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408332575bc060ce15d264'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:03,108 - lava_callback - INFO - Received callback for node 69408379575bc060ce15d367 INFO: 172.21.0.1:54348 - "POST /node/69408379575bc060ce15d367 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:54358 - "GET /wiki HTTP/1.0" 404 Not Found 2025-12-17 21:54:03,303 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832c575bc060ce15d259 2025-12-17 21:54:03,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832c575bc060ce15d259 HTTP/1.1" 502 150 2025-12-17 21:54:03,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832c575bc060ce15d259'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:03,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832d575bc060ce15d25c 2025-12-17 21:54:03,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832d575bc060ce15d25c HTTP/1.1" 502 150 2025-12-17 21:54:03,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832d575bc060ce15d25c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:03,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832e575bc060ce15d25f 2025-12-17 21:54:03,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832e575bc060ce15d25f HTTP/1.1" 502 150 2025-12-17 21:54:03,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832e575bc060ce15d25f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:03,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831d575bc060ce15d22b 2025-12-17 21:54:03,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831d575bc060ce15d22b HTTP/1.1" 502 150 2025-12-17 21:54:03,914 - lava_callback - ERROR - Error processing callback for node 6940831d575bc060ce15d22b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940831d575bc060ce15d22b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940831d575bc060ce15d22b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940831d575bc060ce15d22b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:03,917 - lava_callback - INFO - Starting async job submit for node 69408333575bc060ce15d26b 2025-12-17 21:54:03,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:03,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408333575bc060ce15d26b HTTP/1.1" 502 150 2025-12-17 21:54:03,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408333575bc060ce15d26b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:03,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408333575bc060ce15d26b 2025-12-17 21:54:03,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408333575bc060ce15d26b HTTP/1.1" 502 150 2025-12-17 21:54:03,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408333575bc060ce15d26b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:03,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408325575bc060ce15d243 2025-12-17 21:54:03,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408324575bc060ce15d240 2025-12-17 21:54:03,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408325575bc060ce15d243 HTTP/1.1" 502 150 2025-12-17 21:54:03,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408325575bc060ce15d243'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:03,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408324575bc060ce15d240 HTTP/1.1" 502 150 2025-12-17 21:54:03,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408324575bc060ce15d240'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:05,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408326575bc060ce15d246 2025-12-17 21:54:05,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408326575bc060ce15d246 HTTP/1.1" 502 150 2025-12-17 21:54:05,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408326575bc060ce15d246'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:05,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408329575bc060ce15d24b 2025-12-17 21:54:05,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408329575bc060ce15d24b HTTP/1.1" 502 150 2025-12-17 21:54:05,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408329575bc060ce15d24b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:05,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940833d575bc060ce15d27d 2025-12-17 21:54:05,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940833d575bc060ce15d27d HTTP/1.1" 502 150 2025-12-17 21:54:05,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940833d575bc060ce15d27d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:05,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408333575bc060ce15d26b 2025-12-17 21:54:05,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408333575bc060ce15d26b HTTP/1.1" 502 150 2025-12-17 21:54:05,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408333575bc060ce15d26b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:05,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408308575bc060ce15d1c4 2025-12-17 21:54:05,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408308575bc060ce15d1c4 HTTP/1.1" 502 150 2025-12-17 21:54:05,972 - lava_callback - ERROR - Error processing callback for node 69408308575bc060ce15d1c4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408308575bc060ce15d1c4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408308575bc060ce15d1c4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408308575bc060ce15d1c4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:05,974 - lava_callback - INFO - Starting async job submit for node 69408334575bc060ce15d270 2025-12-17 21:54:05,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:06,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408334575bc060ce15d270 HTTP/1.1" 502 150 2025-12-17 21:54:06,167 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408334575bc060ce15d270'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:06,168 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408334575bc060ce15d270 2025-12-17 21:54:06,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408334575bc060ce15d270 HTTP/1.1" 502 150 2025-12-17 21:54:06,168 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408334575bc060ce15d270'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54364 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:54:06,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408332575bc060ce15d264 2025-12-17 21:54:06,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408332575bc060ce15d264 HTTP/1.1" 502 150 2025-12-17 21:54:06,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408332575bc060ce15d264'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:07,158 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408320575bc060ce15d237 2025-12-17 21:54:07,158 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940831e575bc060ce15d230 2025-12-17 21:54:07,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408320575bc060ce15d237 HTTP/1.1" 502 150 2025-12-17 21:54:07,160 - lava_callback - ERROR - Error processing callback for node 69408320575bc060ce15d237: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408320575bc060ce15d237 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408320575bc060ce15d237 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408320575bc060ce15d237 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:07,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940831e575bc060ce15d230 HTTP/1.1" 502 150 2025-12-17 21:54:07,162 - lava_callback - ERROR - Error processing callback for node 6940831e575bc060ce15d230: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940831e575bc060ce15d230 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940831e575bc060ce15d230 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940831e575bc060ce15d230 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:07,162 - lava_callback - INFO - Starting async job submit for node 69408336575bc060ce15d275 2025-12-17 21:54:07,164 - lava_callback - INFO - Starting async job submit for node 69408337575bc060ce15d278 2025-12-17 21:54:07,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:07,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:07,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408336575bc060ce15d275 HTTP/1.1" 502 150 2025-12-17 21:54:07,171 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408336575bc060ce15d275'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:07,171 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408336575bc060ce15d275 2025-12-17 21:54:07,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408337575bc060ce15d278 HTTP/1.1" 502 150 2025-12-17 21:54:07,172 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408337575bc060ce15d278'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:07,172 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408337575bc060ce15d278 2025-12-17 21:54:07,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408336575bc060ce15d275 HTTP/1.1" 502 150 2025-12-17 21:54:07,172 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408336575bc060ce15d275'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:07,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408337575bc060ce15d278 HTTP/1.1" 502 150 2025-12-17 21:54:07,173 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408337575bc060ce15d278'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:07,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408321575bc060ce15d23a 2025-12-17 21:54:07,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408321575bc060ce15d23a HTTP/1.1" 502 150 2025-12-17 21:54:07,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408321575bc060ce15d23a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:08,169 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408334575bc060ce15d270 2025-12-17 21:54:08,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408334575bc060ce15d270 HTTP/1.1" 502 150 2025-12-17 21:54:08,170 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408334575bc060ce15d270'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:08,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832b575bc060ce15d252 2025-12-17 21:54:08,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832b575bc060ce15d252 HTTP/1.1" 502 150 2025-12-17 21:54:08,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832b575bc060ce15d252'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:08,508 - lava_callback - INFO - Received callback for node 6940836e575bc060ce15d359 INFO: 172.21.0.1:54378 - "POST /node/6940836e575bc060ce15d359 HTTP/1.0" 202 Accepted 2025-12-17 21:54:09,173 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408336575bc060ce15d275 2025-12-17 21:54:09,173 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408337575bc060ce15d278 2025-12-17 21:54:09,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408336575bc060ce15d275 HTTP/1.1" 502 150 2025-12-17 21:54:09,174 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408336575bc060ce15d275'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:09,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408337575bc060ce15d278 HTTP/1.1" 502 150 2025-12-17 21:54:09,174 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408337575bc060ce15d278'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:09,222 - lava_callback - INFO - Received callback for node 6940836f575bc060ce15d35e INFO: 172.21.0.1:54388 - "POST /node/6940836f575bc060ce15d35e HTTP/1.0" 202 Accepted 2025-12-17 21:54:09,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408333575bc060ce15d26b 2025-12-17 21:54:09,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408333575bc060ce15d26b HTTP/1.1" 502 150 2025-12-17 21:54:09,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408333575bc060ce15d26b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:10,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408315575bc060ce15d20e 2025-12-17 21:54:10,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408315575bc060ce15d20e HTTP/1.1" 502 150 2025-12-17 21:54:10,675 - lava_callback - ERROR - Error processing callback for node 69408315575bc060ce15d20e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408315575bc060ce15d20e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408315575bc060ce15d20e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408315575bc060ce15d20e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:10,677 - lava_callback - INFO - Starting async job submit for node 6940833f575bc060ce15d284 2025-12-17 21:54:10,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:10,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940833f575bc060ce15d284 HTTP/1.1" 502 150 2025-12-17 21:54:10,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940833f575bc060ce15d284'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:10,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940833f575bc060ce15d284 2025-12-17 21:54:10,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940833f575bc060ce15d284 HTTP/1.1" 502 150 2025-12-17 21:54:10,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940833f575bc060ce15d284'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:11,304 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832c575bc060ce15d259 2025-12-17 21:54:11,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832c575bc060ce15d259 HTTP/1.1" 502 150 2025-12-17 21:54:11,305 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832c575bc060ce15d259'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:11,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832d575bc060ce15d25c 2025-12-17 21:54:11,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832d575bc060ce15d25c HTTP/1.1" 502 150 2025-12-17 21:54:11,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832d575bc060ce15d25c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:11,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832e575bc060ce15d25f 2025-12-17 21:54:11,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832e575bc060ce15d25f HTTP/1.1" 502 150 2025-12-17 21:54:11,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940832e575bc060ce15d25f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:12,170 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408334575bc060ce15d270 2025-12-17 21:54:12,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408334575bc060ce15d270 HTTP/1.1" 502 150 2025-12-17 21:54:12,171 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408334575bc060ce15d270'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:12,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940833f575bc060ce15d284 2025-12-17 21:54:12,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940833f575bc060ce15d284 HTTP/1.1" 502 150 2025-12-17 21:54:12,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940833f575bc060ce15d284'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:13,174 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408336575bc060ce15d275 2025-12-17 21:54:13,175 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408337575bc060ce15d278 2025-12-17 21:54:13,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408336575bc060ce15d275 HTTP/1.1" 502 150 2025-12-17 21:54:13,176 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408336575bc060ce15d275'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:13,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408337575bc060ce15d278 HTTP/1.1" 502 150 2025-12-17 21:54:13,176 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408337575bc060ce15d278'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:13,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408329575bc060ce15d24b 2025-12-17 21:54:13,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408329575bc060ce15d24b HTTP/1.1" 502 150 2025-12-17 21:54:13,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408329575bc060ce15d24b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:13,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940833d575bc060ce15d27d 2025-12-17 21:54:13,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940833d575bc060ce15d27d HTTP/1.1" 502 150 2025-12-17 21:54:13,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940833d575bc060ce15d27d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:14,839 - lava_callback - INFO - Received callback for node 69408fad575bc060ce160201 INFO: 172.21.0.1:42080 - "POST /node/69408fad575bc060ce160201 HTTP/1.0" 202 Accepted 2025-12-17 21:54:14,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408332575bc060ce15d264 2025-12-17 21:54:14,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408332575bc060ce15d264 HTTP/1.1" 502 150 2025-12-17 21:54:14,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408332575bc060ce15d264'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42096 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:54:16,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940833f575bc060ce15d284 2025-12-17 21:54:16,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940833f575bc060ce15d284 HTTP/1.1" 502 150 2025-12-17 21:54:16,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940833f575bc060ce15d284'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:17,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408333575bc060ce15d26b 2025-12-17 21:54:17,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408333575bc060ce15d26b HTTP/1.1" 502 150 2025-12-17 21:54:17,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408333575bc060ce15d26b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:19,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408325575bc060ce15d243 2025-12-17 21:54:19,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408324575bc060ce15d240 2025-12-17 21:54:19,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408325575bc060ce15d243 HTTP/1.1" 502 150 2025-12-17 21:54:19,935 - lava_callback - ERROR - Error processing callback for node 69408325575bc060ce15d243: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408325575bc060ce15d243 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408325575bc060ce15d243 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408325575bc060ce15d243 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:19,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408324575bc060ce15d240 HTTP/1.1" 502 150 2025-12-17 21:54:19,938 - lava_callback - ERROR - Error processing callback for node 69408324575bc060ce15d240: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408324575bc060ce15d240 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408324575bc060ce15d240 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408324575bc060ce15d240 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:19,938 - lava_callback - INFO - Starting async job submit for node 69408340575bc060ce15d28b 2025-12-17 21:54:19,940 - lava_callback - INFO - Starting async job submit for node 69408341575bc060ce15d292 2025-12-17 21:54:19,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:19,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:19,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408340575bc060ce15d28b HTTP/1.1" 502 150 2025-12-17 21:54:19,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408340575bc060ce15d28b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:19,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408340575bc060ce15d28b 2025-12-17 21:54:19,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408340575bc060ce15d28b HTTP/1.1" 502 150 2025-12-17 21:54:19,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408340575bc060ce15d28b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:20,172 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408334575bc060ce15d270 2025-12-17 21:54:20,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408334575bc060ce15d270 HTTP/1.1" 502 150 2025-12-17 21:54:20,173 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408334575bc060ce15d270'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:21,176 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408336575bc060ce15d275 2025-12-17 21:54:21,176 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408337575bc060ce15d278 2025-12-17 21:54:21,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408336575bc060ce15d275 HTTP/1.1" 502 150 2025-12-17 21:54:21,177 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408336575bc060ce15d275'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:21,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408337575bc060ce15d278 HTTP/1.1" 502 150 2025-12-17 21:54:21,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408337575bc060ce15d278'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:21,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408326575bc060ce15d246 2025-12-17 21:54:21,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408326575bc060ce15d246 HTTP/1.1" 502 150 2025-12-17 21:54:21,452 - lava_callback - ERROR - Error processing callback for node 69408326575bc060ce15d246: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408326575bc060ce15d246 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408326575bc060ce15d246 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408326575bc060ce15d246 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:21,454 - lava_callback - INFO - Starting async job submit for node 69408343575bc060ce15d297 2025-12-17 21:54:21,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:21,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408343575bc060ce15d297 HTTP/1.1" 502 150 2025-12-17 21:54:21,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408343575bc060ce15d297'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:21,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408343575bc060ce15d297 2025-12-17 21:54:21,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408343575bc060ce15d297 HTTP/1.1" 502 150 2025-12-17 21:54:21,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408343575bc060ce15d297'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:21,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408340575bc060ce15d28b 2025-12-17 21:54:21,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408340575bc060ce15d28b HTTP/1.1" 502 150 2025-12-17 21:54:21,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408340575bc060ce15d28b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:23,149 - lava_callback - INFO - Received callback for node 6940837b575bc060ce15d36e INFO: 172.21.0.1:56348 - "POST /node/6940837b575bc060ce15d36e HTTP/1.0" 202 Accepted 2025-12-17 21:54:23,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408343575bc060ce15d297 2025-12-17 21:54:23,787 - lava_callback - INFO - Received callback for node 6940837c575bc060ce15d376 INFO: 172.21.0.1:56362 - "POST /node/6940837c575bc060ce15d376 HTTP/1.0" 202 Accepted 2025-12-17 21:54:23,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408321575bc060ce15d23a 2025-12-17 21:54:23,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408321575bc060ce15d23a HTTP/1.1" 502 150 2025-12-17 21:54:23,950 - lava_callback - ERROR - Error processing callback for node 69408321575bc060ce15d23a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408321575bc060ce15d23a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408321575bc060ce15d23a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408321575bc060ce15d23a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:23,953 - lava_callback - INFO - Starting async job submit for node 69408344575bc060ce15d29a 2025-12-17 21:54:23,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:23,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408344575bc060ce15d29a HTTP/1.1" 502 150 2025-12-17 21:54:23,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408344575bc060ce15d29a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:23,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408344575bc060ce15d29a 2025-12-17 21:54:23,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408344575bc060ce15d29a HTTP/1.1" 502 150 2025-12-17 21:54:23,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408344575bc060ce15d29a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:24,450 - lava_callback - INFO - Received callback for node 6940837e575bc060ce15d37e INFO: 172.21.0.1:56378 - "POST /node/6940837e575bc060ce15d37e HTTP/1.0" 202 Accepted 2025-12-17 21:54:24,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832b575bc060ce15d252 2025-12-17 21:54:24,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832b575bc060ce15d252 HTTP/1.1" 502 150 2025-12-17 21:54:24,459 - lava_callback - ERROR - Error processing callback for node 6940832b575bc060ce15d252: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940832b575bc060ce15d252 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940832b575bc060ce15d252 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940832b575bc060ce15d252 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:24,462 - lava_callback - INFO - Starting async job submit for node 69408345575bc060ce15d29d 2025-12-17 21:54:24,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:24,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408345575bc060ce15d29d HTTP/1.1" 502 150 2025-12-17 21:54:24,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408345575bc060ce15d29d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:24,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408345575bc060ce15d29d 2025-12-17 21:54:24,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408345575bc060ce15d29d HTTP/1.1" 502 150 2025-12-17 21:54:24,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408345575bc060ce15d29d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:24,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940833f575bc060ce15d284 2025-12-17 21:54:24,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940833f575bc060ce15d284 HTTP/1.1" 502 150 2025-12-17 21:54:24,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940833f575bc060ce15d284'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:25,107 - lava_callback - INFO - Received callback for node 6940837f575bc060ce15d383 INFO: 172.21.0.1:56382 - "POST /node/6940837f575bc060ce15d383 HTTP/1.0" 202 Accepted 2025-12-17 21:54:25,765 - lava_callback - INFO - Received callback for node 69408381575bc060ce15d386 INFO: 172.21.0.1:56386 - "POST /node/69408381575bc060ce15d386 HTTP/1.0" 202 Accepted 2025-12-17 21:54:25,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408340575bc060ce15d28b 2025-12-17 21:54:25,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408340575bc060ce15d28b HTTP/1.1" 502 150 2025-12-17 21:54:25,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408340575bc060ce15d28b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:25,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408344575bc060ce15d29a 2025-12-17 21:54:25,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408344575bc060ce15d29a HTTP/1.1" 502 150 2025-12-17 21:54:25,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408344575bc060ce15d29a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:26,426 - lava_callback - INFO - Received callback for node 69408382575bc060ce15d389 INFO: 172.21.0.1:56398 - "POST /node/69408382575bc060ce15d389 HTTP/1.0" 202 Accepted 2025-12-17 21:54:26,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408345575bc060ce15d29d 2025-12-17 21:54:26,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408345575bc060ce15d29d HTTP/1.1" 502 150 2025-12-17 21:54:26,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408345575bc060ce15d29d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56402 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:54:27,113 - lava_callback - INFO - Received callback for node 69408383575bc060ce15d38c INFO: 172.21.0.1:56414 - "POST /node/69408383575bc060ce15d38c HTTP/1.0" 202 Accepted 2025-12-17 21:54:27,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832c575bc060ce15d259 2025-12-17 21:54:27,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832c575bc060ce15d259 HTTP/1.1" 502 150 2025-12-17 21:54:27,307 - lava_callback - ERROR - Error processing callback for node 6940832c575bc060ce15d259: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940832c575bc060ce15d259 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940832c575bc060ce15d259 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940832c575bc060ce15d259 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:27,309 - lava_callback - INFO - Starting async job submit for node 69408346575bc060ce15d2a0 2025-12-17 21:54:27,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:27,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408346575bc060ce15d2a0 HTTP/1.1" 502 150 2025-12-17 21:54:27,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408346575bc060ce15d2a0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:27,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408346575bc060ce15d2a0 2025-12-17 21:54:27,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408346575bc060ce15d2a0 HTTP/1.1" 502 150 2025-12-17 21:54:27,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408346575bc060ce15d2a0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:27,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832d575bc060ce15d25c 2025-12-17 21:54:27,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832d575bc060ce15d25c HTTP/1.1" 502 150 2025-12-17 21:54:27,444 - lava_callback - ERROR - Error processing callback for node 6940832d575bc060ce15d25c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940832d575bc060ce15d25c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940832d575bc060ce15d25c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940832d575bc060ce15d25c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:27,445 - lava_callback - INFO - Starting async job submit for node 69408351575bc060ce15d2d4 2025-12-17 21:54:27,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:27,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408351575bc060ce15d2d4 HTTP/1.1" 502 150 2025-12-17 21:54:27,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408351575bc060ce15d2d4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:27,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408351575bc060ce15d2d4 2025-12-17 21:54:27,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408351575bc060ce15d2d4 HTTP/1.1" 502 150 2025-12-17 21:54:27,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408351575bc060ce15d2d4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:27,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940832e575bc060ce15d25f 2025-12-17 21:54:27,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940832e575bc060ce15d25f HTTP/1.1" 502 150 2025-12-17 21:54:27,771 - lava_callback - ERROR - Error processing callback for node 6940832e575bc060ce15d25f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940832e575bc060ce15d25f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940832e575bc060ce15d25f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940832e575bc060ce15d25f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:27,773 - lava_callback - INFO - Starting async job submit for node 6940839c575bc060ce15d421 2025-12-17 21:54:27,793 - lava_callback - INFO - Received callback for node 69408384575bc060ce15d38f INFO: 172.21.0.1:56416 - "POST /node/69408384575bc060ce15d38f HTTP/1.0" 202 Accepted 2025-12-17 21:54:27,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:27,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839c575bc060ce15d421 HTTP/1.1" 502 150 2025-12-17 21:54:27,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839c575bc060ce15d421'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:27,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839c575bc060ce15d421 2025-12-17 21:54:27,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839c575bc060ce15d421 HTTP/1.1" 502 150 2025-12-17 21:54:27,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839c575bc060ce15d421'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:28,273 - lava_callback - INFO - Received callback for node 69408386575bc060ce15d3a8 INFO: 172.21.0.1:56430 - "POST /node/69408386575bc060ce15d3a8 HTTP/1.0" 202 Accepted 2025-12-17 21:54:28,454 - lava_callback - INFO - Received callback for node 6940838a575bc060ce15d3e2 INFO: 172.21.0.1:56432 - "POST /node/6940838a575bc060ce15d3e2 HTTP/1.0" 202 Accepted 2025-12-17 21:54:28,902 - lava_callback - INFO - Received callback for node 69408388575bc060ce15d3dc INFO: 172.21.0.1:56444 - "POST /node/69408388575bc060ce15d3dc HTTP/1.0" 202 Accepted 2025-12-17 21:54:29,116 - lava_callback - INFO - Received callback for node 6940838c575bc060ce15d3e5 INFO: 172.21.0.1:56446 - "POST /node/6940838c575bc060ce15d3e5 HTTP/1.0" 202 Accepted 2025-12-17 21:54:29,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408346575bc060ce15d2a0 2025-12-17 21:54:29,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408351575bc060ce15d2d4 2025-12-17 21:54:29,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408351575bc060ce15d2d4 HTTP/1.1" 502 150 2025-12-17 21:54:29,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408351575bc060ce15d2d4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:29,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408329575bc060ce15d24b 2025-12-17 21:54:29,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408329575bc060ce15d24b HTTP/1.1" 502 150 2025-12-17 21:54:29,528 - lava_callback - ERROR - Error processing callback for node 69408329575bc060ce15d24b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408329575bc060ce15d24b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408329575bc060ce15d24b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408329575bc060ce15d24b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:29,530 - lava_callback - INFO - Starting async job submit for node 69408356575bc060ce15d2ea 2025-12-17 21:54:29,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:29,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408356575bc060ce15d2ea HTTP/1.1" 502 150 2025-12-17 21:54:29,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408356575bc060ce15d2ea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:29,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408356575bc060ce15d2ea 2025-12-17 21:54:29,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408356575bc060ce15d2ea HTTP/1.1" 502 150 2025-12-17 21:54:29,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408356575bc060ce15d2ea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:29,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940833d575bc060ce15d27d 2025-12-17 21:54:29,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940833d575bc060ce15d27d HTTP/1.1" 502 150 2025-12-17 21:54:29,775 - lava_callback - ERROR - Error processing callback for node 6940833d575bc060ce15d27d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940833d575bc060ce15d27d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940833d575bc060ce15d27d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940833d575bc060ce15d27d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:29,777 - lava_callback - INFO - Starting async job submit for node 69408f95575bc060ce1601b0 2025-12-17 21:54:29,798 - lava_callback - INFO - Received callback for node 6940838d575bc060ce15d3ea INFO: 172.21.0.1:48312 - "POST /node/6940838d575bc060ce15d3ea HTTP/1.0" 202 Accepted 2025-12-17 21:54:29,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839c575bc060ce15d421 2025-12-17 21:54:29,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839c575bc060ce15d421 HTTP/1.1" 502 150 2025-12-17 21:54:29,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839c575bc060ce15d421'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:29,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:29,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f95575bc060ce1601b0 HTTP/1.1" 502 150 2025-12-17 21:54:29,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f95575bc060ce1601b0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:29,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f95575bc060ce1601b0 2025-12-17 21:54:29,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f95575bc060ce1601b0 HTTP/1.1" 502 150 2025-12-17 21:54:29,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f95575bc060ce1601b0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:29,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408344575bc060ce15d29a 2025-12-17 21:54:29,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408344575bc060ce15d29a HTTP/1.1" 502 150 2025-12-17 21:54:29,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408344575bc060ce15d29a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:30,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408345575bc060ce15d29d 2025-12-17 21:54:30,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408345575bc060ce15d29d HTTP/1.1" 502 150 2025-12-17 21:54:30,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408345575bc060ce15d29d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:30,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408332575bc060ce15d264 2025-12-17 21:54:30,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408332575bc060ce15d264 HTTP/1.1" 502 150 2025-12-17 21:54:30,961 - lava_callback - ERROR - Error processing callback for node 69408332575bc060ce15d264: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408332575bc060ce15d264 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408332575bc060ce15d264 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408332575bc060ce15d264 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:30,963 - lava_callback - INFO - Starting async job submit for node 69408353575bc060ce15d2dd 2025-12-17 21:54:30,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:30,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408353575bc060ce15d2dd HTTP/1.1" 502 150 2025-12-17 21:54:30,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408353575bc060ce15d2dd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:30,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408353575bc060ce15d2dd 2025-12-17 21:54:30,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408353575bc060ce15d2dd HTTP/1.1" 502 150 2025-12-17 21:54:30,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408353575bc060ce15d2dd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:31,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408356575bc060ce15d2ea 2025-12-17 21:54:31,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408356575bc060ce15d2ea HTTP/1.1" 502 150 2025-12-17 21:54:31,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408356575bc060ce15d2ea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:31,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f95575bc060ce1601b0 2025-12-17 21:54:31,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f95575bc060ce1601b0 HTTP/1.1" 502 150 2025-12-17 21:54:31,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f95575bc060ce1601b0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:32,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408353575bc060ce15d2dd 2025-12-17 21:54:32,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408353575bc060ce15d2dd HTTP/1.1" 502 150 2025-12-17 21:54:32,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408353575bc060ce15d2dd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:33,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408351575bc060ce15d2d4 2025-12-17 21:54:33,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408351575bc060ce15d2d4 HTTP/1.1" 502 150 2025-12-17 21:54:33,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408351575bc060ce15d2d4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:33,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839c575bc060ce15d421 2025-12-17 21:54:33,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839c575bc060ce15d421 HTTP/1.1" 502 150 2025-12-17 21:54:33,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839c575bc060ce15d421'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:33,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408333575bc060ce15d26b 2025-12-17 21:54:33,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408333575bc060ce15d26b HTTP/1.1" 502 150 2025-12-17 21:54:33,938 - lava_callback - ERROR - Error processing callback for node 69408333575bc060ce15d26b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408333575bc060ce15d26b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408333575bc060ce15d26b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408333575bc060ce15d26b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:33,940 - lava_callback - INFO - Starting async job submit for node 69408354575bc060ce15d2e4 2025-12-17 21:54:33,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:33,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408340575bc060ce15d28b 2025-12-17 21:54:33,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408340575bc060ce15d28b HTTP/1.1" 502 150 2025-12-17 21:54:33,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408340575bc060ce15d28b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:33,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408354575bc060ce15d2e4 HTTP/1.1" 502 150 2025-12-17 21:54:33,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408354575bc060ce15d2e4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:33,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408354575bc060ce15d2e4 2025-12-17 21:54:33,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408354575bc060ce15d2e4 HTTP/1.1" 502 150 2025-12-17 21:54:33,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408354575bc060ce15d2e4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:35,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408356575bc060ce15d2ea 2025-12-17 21:54:35,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408356575bc060ce15d2ea HTTP/1.1" 502 150 2025-12-17 21:54:35,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408356575bc060ce15d2ea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:35,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f95575bc060ce1601b0 2025-12-17 21:54:35,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f95575bc060ce1601b0 HTTP/1.1" 502 150 2025-12-17 21:54:35,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f95575bc060ce1601b0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:35,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408354575bc060ce15d2e4 2025-12-17 21:54:35,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408354575bc060ce15d2e4 HTTP/1.1" 502 150 2025-12-17 21:54:35,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408354575bc060ce15d2e4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:36,173 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408334575bc060ce15d270 2025-12-17 21:54:36,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408334575bc060ce15d270 HTTP/1.1" 502 150 2025-12-17 21:54:36,174 - lava_callback - ERROR - Error processing callback for node 69408334575bc060ce15d270: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408334575bc060ce15d270 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408334575bc060ce15d270 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408334575bc060ce15d270 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:36,176 - lava_callback - INFO - Starting async job submit for node 69408358575bc060ce15d2ed 2025-12-17 21:54:36,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:36,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408358575bc060ce15d2ed HTTP/1.1" 502 150 2025-12-17 21:54:36,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408358575bc060ce15d2ed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:36,183 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408358575bc060ce15d2ed 2025-12-17 21:54:36,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408358575bc060ce15d2ed HTTP/1.1" 502 150 2025-12-17 21:54:36,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408358575bc060ce15d2ed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48324 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:54:36,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408353575bc060ce15d2dd 2025-12-17 21:54:36,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408353575bc060ce15d2dd HTTP/1.1" 502 150 2025-12-17 21:54:36,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408353575bc060ce15d2dd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:37,177 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408336575bc060ce15d275 2025-12-17 21:54:37,178 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408337575bc060ce15d278 2025-12-17 21:54:37,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408336575bc060ce15d275 HTTP/1.1" 502 150 2025-12-17 21:54:37,179 - lava_callback - ERROR - Error processing callback for node 69408336575bc060ce15d275: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408336575bc060ce15d275 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408336575bc060ce15d275 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408336575bc060ce15d275 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:37,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408337575bc060ce15d278 HTTP/1.1" 502 150 2025-12-17 21:54:37,181 - lava_callback - ERROR - Error processing callback for node 69408337575bc060ce15d278: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408337575bc060ce15d278 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408337575bc060ce15d278 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408337575bc060ce15d278 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:37,181 - lava_callback - INFO - Starting async job submit for node 69408359575bc060ce15d2f0 2025-12-17 21:54:37,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:37,184 - lava_callback - INFO - Starting async job submit for node 6940835b575bc060ce15d2f3 2025-12-17 21:54:37,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:37,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408359575bc060ce15d2f0 HTTP/1.1" 502 150 2025-12-17 21:54:37,191 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408359575bc060ce15d2f0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:37,191 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408359575bc060ce15d2f0 2025-12-17 21:54:37,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940835b575bc060ce15d2f3 HTTP/1.1" 502 150 2025-12-17 21:54:37,191 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940835b575bc060ce15d2f3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:37,191 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940835b575bc060ce15d2f3 2025-12-17 21:54:37,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408359575bc060ce15d2f0 HTTP/1.1" 502 150 2025-12-17 21:54:37,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408359575bc060ce15d2f0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:37,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940835b575bc060ce15d2f3 HTTP/1.1" 502 150 2025-12-17 21:54:37,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940835b575bc060ce15d2f3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:37,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408344575bc060ce15d29a 2025-12-17 21:54:37,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408344575bc060ce15d29a HTTP/1.1" 502 150 2025-12-17 21:54:37,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408344575bc060ce15d29a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:38,184 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408358575bc060ce15d2ed 2025-12-17 21:54:38,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408345575bc060ce15d29d 2025-12-17 21:54:38,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408345575bc060ce15d29d HTTP/1.1" 502 150 2025-12-17 21:54:38,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408345575bc060ce15d29d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:38,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408343575bc060ce15d297 HTTP/1.1" 502 150 2025-12-17 21:54:38,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408343575bc060ce15d297'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:39,192 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408359575bc060ce15d2f0 2025-12-17 21:54:39,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940835b575bc060ce15d2f3 2025-12-17 21:54:39,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408359575bc060ce15d2f0 HTTP/1.1" 502 150 2025-12-17 21:54:39,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408359575bc060ce15d2f0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:39,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940835b575bc060ce15d2f3 HTTP/1.1" 502 150 2025-12-17 21:54:39,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940835b575bc060ce15d2f3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:39,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408354575bc060ce15d2e4 2025-12-17 21:54:39,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408354575bc060ce15d2e4 HTTP/1.1" 502 150 2025-12-17 21:54:39,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408354575bc060ce15d2e4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:40,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940833f575bc060ce15d284 2025-12-17 21:54:40,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940833f575bc060ce15d284 HTTP/1.1" 502 150 2025-12-17 21:54:40,691 - lava_callback - ERROR - Error processing callback for node 6940833f575bc060ce15d284: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940833f575bc060ce15d284 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940833f575bc060ce15d284 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940833f575bc060ce15d284 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:40,693 - lava_callback - INFO - Starting async job submit for node 69408379575bc060ce15d367 2025-12-17 21:54:40,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:40,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408379575bc060ce15d367 HTTP/1.1" 502 150 2025-12-17 21:54:40,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408379575bc060ce15d367'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:40,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408379575bc060ce15d367 2025-12-17 21:54:40,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408379575bc060ce15d367 HTTP/1.1" 502 150 2025-12-17 21:54:40,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408379575bc060ce15d367'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:41,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408351575bc060ce15d2d4 2025-12-17 21:54:41,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839c575bc060ce15d421 2025-12-17 21:54:41,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839c575bc060ce15d421 HTTP/1.1" 502 150 2025-12-17 21:54:41,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839c575bc060ce15d421'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:42,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408379575bc060ce15d367 2025-12-17 21:54:42,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408379575bc060ce15d367 HTTP/1.1" 502 150 2025-12-17 21:54:42,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408379575bc060ce15d367'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:42,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408343575bc060ce15d297 2025-12-17 21:54:42,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408343575bc060ce15d297 HTTP/1.1" 502 150 2025-12-17 21:54:42,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408343575bc060ce15d297'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:43,171 - lava_callback - INFO - Received callback for node 6940838f575bc060ce15d3f5 INFO: 172.21.0.1:45200 - "POST /node/6940838f575bc060ce15d3f5 HTTP/1.0" 202 Accepted 2025-12-17 21:54:43,194 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408359575bc060ce15d2f0 2025-12-17 21:54:43,194 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940835b575bc060ce15d2f3 2025-12-17 21:54:43,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408359575bc060ce15d2f0 HTTP/1.1" 502 150 2025-12-17 21:54:43,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408359575bc060ce15d2f0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:43,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940835b575bc060ce15d2f3 HTTP/1.1" 502 150 2025-12-17 21:54:43,196 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940835b575bc060ce15d2f3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:43,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408356575bc060ce15d2ea 2025-12-17 21:54:43,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408356575bc060ce15d2ea HTTP/1.1" 502 150 2025-12-17 21:54:43,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408356575bc060ce15d2ea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:43,857 - lava_callback - INFO - Received callback for node 69408390575bc060ce15d3fc INFO: 172.21.0.1:45202 - "POST /node/69408390575bc060ce15d3fc HTTP/1.0" 202 Accepted 2025-12-17 21:54:43,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f95575bc060ce1601b0 2025-12-17 21:54:43,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f95575bc060ce1601b0 HTTP/1.1" 502 150 2025-12-17 21:54:43,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f95575bc060ce1601b0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:44,521 - lava_callback - INFO - Received callback for node 69408391575bc060ce15d403 INFO: 172.21.0.1:45206 - "POST /node/69408391575bc060ce15d403 HTTP/1.0" 202 Accepted 2025-12-17 21:54:44,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408346575bc060ce15d2a0 HTTP/1.1" 502 150 2025-12-17 21:54:44,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408346575bc060ce15d2a0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:44,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408353575bc060ce15d2dd 2025-12-17 21:54:44,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408353575bc060ce15d2dd HTTP/1.1" 502 150 2025-12-17 21:54:44,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408353575bc060ce15d2dd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:45,181 - lava_callback - INFO - Received callback for node 69408393575bc060ce15d408 INFO: 172.21.0.1:45214 - "POST /node/69408393575bc060ce15d408 HTTP/1.0" 202 Accepted 2025-12-17 21:54:45,853 - lava_callback - INFO - Received callback for node 69408394575bc060ce15d40f INFO: 172.21.0.1:45230 - "POST /node/69408394575bc060ce15d40f HTTP/1.0" 202 Accepted 2025-12-17 21:54:46,512 - lava_callback - INFO - Received callback for node 69408395575bc060ce15d412 INFO: 172.21.0.1:45234 - "POST /node/69408395575bc060ce15d412 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:45238 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:54:46,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408379575bc060ce15d367 2025-12-17 21:54:46,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408379575bc060ce15d367 HTTP/1.1" 502 150 2025-12-17 21:54:46,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408379575bc060ce15d367'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:47,180 - lava_callback - INFO - Received callback for node 69408398575bc060ce15d418 INFO: 172.21.0.1:45254 - "POST /node/69408398575bc060ce15d418 HTTP/1.0" 202 Accepted 2025-12-17 21:54:47,843 - lava_callback - INFO - Received callback for node 69408399575bc060ce15d41b INFO: 172.21.0.1:45262 - "POST /node/69408399575bc060ce15d41b HTTP/1.0" 202 Accepted 2025-12-17 21:54:47,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408354575bc060ce15d2e4 2025-12-17 21:54:47,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408354575bc060ce15d2e4 HTTP/1.1" 502 150 2025-12-17 21:54:47,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408354575bc060ce15d2e4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:48,287 - lava_callback - INFO - Received callback for node 69408396575bc060ce15d415 INFO: 172.21.0.1:45266 - "POST /node/69408396575bc060ce15d415 HTTP/1.0" 202 Accepted 2025-12-17 21:54:48,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408346575bc060ce15d2a0 2025-12-17 21:54:48,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408346575bc060ce15d2a0 HTTP/1.1" 502 150 2025-12-17 21:54:48,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408346575bc060ce15d2a0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:48,999 - lava_callback - INFO - Received callback for node 694083a0575bc060ce15d42a INFO: 172.21.0.1:45274 - "POST /node/694083a0575bc060ce15d42a HTTP/1.0" 202 Accepted 2025-12-17 21:54:49,686 - lava_callback - INFO - Received callback for node 694083a1575bc060ce15d42d INFO: 172.21.0.1:45286 - "POST /node/694083a1575bc060ce15d42d HTTP/1.0" 202 Accepted 2025-12-17 21:54:49,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408340575bc060ce15d28b 2025-12-17 21:54:49,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408340575bc060ce15d28b HTTP/1.1" 502 150 2025-12-17 21:54:49,955 - lava_callback - ERROR - Error processing callback for node 69408340575bc060ce15d28b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408340575bc060ce15d28b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408340575bc060ce15d28b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408340575bc060ce15d28b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:49,958 - lava_callback - INFO - Starting async job submit for node 6940836e575bc060ce15d359 2025-12-17 21:54:49,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:49,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940836e575bc060ce15d359 HTTP/1.1" 502 150 2025-12-17 21:54:49,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940836e575bc060ce15d359'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:49,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940836e575bc060ce15d359 2025-12-17 21:54:49,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940836e575bc060ce15d359 HTTP/1.1" 502 150 2025-12-17 21:54:49,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940836e575bc060ce15d359'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:50,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408341575bc060ce15d292 HTTP/1.1" 502 150 2025-12-17 21:54:50,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408341575bc060ce15d292'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:50,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408341575bc060ce15d292 2025-12-17 21:54:50,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408343575bc060ce15d297 2025-12-17 21:54:50,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408343575bc060ce15d297 HTTP/1.1" 502 150 2025-12-17 21:54:50,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408343575bc060ce15d297'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:51,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408359575bc060ce15d2f0 2025-12-17 21:54:51,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940835b575bc060ce15d2f3 2025-12-17 21:54:51,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408359575bc060ce15d2f0 HTTP/1.1" 502 150 2025-12-17 21:54:51,197 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408359575bc060ce15d2f0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:51,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940835b575bc060ce15d2f3 HTTP/1.1" 502 150 2025-12-17 21:54:51,197 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940835b575bc060ce15d2f3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:51,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940836e575bc060ce15d359 2025-12-17 21:54:51,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940836e575bc060ce15d359 HTTP/1.1" 502 150 2025-12-17 21:54:51,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940836e575bc060ce15d359'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:53,334 - lava_callback - INFO - Received callback for node 69408fdd575bc060ce160285 INFO: 172.21.0.1:55696 - "POST /node/69408fdd575bc060ce160285 HTTP/1.0" 202 Accepted 2025-12-17 21:54:53,429 - lava_callback - INFO - Received callback for node 694083a5575bc060ce15d444 INFO: 172.21.0.1:55708 - "POST /node/694083a5575bc060ce15d444 HTTP/1.0" 202 Accepted 2025-12-17 21:54:53,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408358575bc060ce15d2ed HTTP/1.1" 502 150 2025-12-17 21:54:53,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408358575bc060ce15d2ed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:53,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408344575bc060ce15d29a 2025-12-17 21:54:53,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408344575bc060ce15d29a HTTP/1.1" 502 150 2025-12-17 21:54:53,969 - lava_callback - ERROR - Error processing callback for node 69408344575bc060ce15d29a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408344575bc060ce15d29a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408344575bc060ce15d29a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408344575bc060ce15d29a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:53,971 - lava_callback - INFO - Starting async job submit for node 6940836f575bc060ce15d35e 2025-12-17 21:54:53,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:53,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940836f575bc060ce15d35e HTTP/1.1" 502 150 2025-12-17 21:54:53,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940836f575bc060ce15d35e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:53,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940836f575bc060ce15d35e 2025-12-17 21:54:53,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940836f575bc060ce15d35e HTTP/1.1" 502 150 2025-12-17 21:54:53,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940836f575bc060ce15d35e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:54,106 - lava_callback - INFO - Received callback for node 694083a8575bc060ce15d44d INFO: 172.21.0.1:55710 - "POST /node/694083a8575bc060ce15d44d HTTP/1.0" 202 Accepted 2025-12-17 21:54:54,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408345575bc060ce15d29d 2025-12-17 21:54:54,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408345575bc060ce15d29d HTTP/1.1" 502 150 2025-12-17 21:54:54,479 - lava_callback - ERROR - Error processing callback for node 69408345575bc060ce15d29d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408345575bc060ce15d29d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408345575bc060ce15d29d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408345575bc060ce15d29d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:54,482 - lava_callback - INFO - Starting async job submit for node 69408fad575bc060ce160201 2025-12-17 21:54:54,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:54,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fad575bc060ce160201 HTTP/1.1" 502 150 2025-12-17 21:54:54,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fad575bc060ce160201'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:54,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fad575bc060ce160201 2025-12-17 21:54:54,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fad575bc060ce160201 HTTP/1.1" 502 150 2025-12-17 21:54:54,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fad575bc060ce160201'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:54,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408379575bc060ce15d367 2025-12-17 21:54:54,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408379575bc060ce15d367 HTTP/1.1" 502 150 2025-12-17 21:54:54,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408379575bc060ce15d367'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:54,782 - lava_callback - INFO - Received callback for node 694083a9575bc060ce15d452 INFO: 172.21.0.1:55714 - "POST /node/694083a9575bc060ce15d452 HTTP/1.0" 202 Accepted 2025-12-17 21:54:55,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940836f575bc060ce15d35e 2025-12-17 21:54:55,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940836f575bc060ce15d35e HTTP/1.1" 502 150 2025-12-17 21:54:55,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940836f575bc060ce15d35e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:55,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940836e575bc060ce15d359 2025-12-17 21:54:55,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940836e575bc060ce15d359 HTTP/1.1" 502 150 2025-12-17 21:54:55,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940836e575bc060ce15d359'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55724 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:54:56,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fad575bc060ce160201 2025-12-17 21:54:56,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fad575bc060ce160201 HTTP/1.1" 502 150 2025-12-17 21:54:56,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fad575bc060ce160201'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:56,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408346575bc060ce15d2a0 2025-12-17 21:54:56,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408346575bc060ce15d2a0 HTTP/1.1" 502 150 2025-12-17 21:54:56,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408346575bc060ce15d2a0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:56,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408351575bc060ce15d2d4 HTTP/1.1" 502 150 2025-12-17 21:54:56,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408351575bc060ce15d2d4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:57,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408358575bc060ce15d2ed 2025-12-17 21:54:57,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408358575bc060ce15d2ed HTTP/1.1" 502 150 2025-12-17 21:54:57,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408358575bc060ce15d2ed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:57,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839c575bc060ce15d421 2025-12-17 21:54:57,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839c575bc060ce15d421 HTTP/1.1" 502 150 2025-12-17 21:54:57,859 - lava_callback - ERROR - Error processing callback for node 6940839c575bc060ce15d421: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940839c575bc060ce15d421 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940839c575bc060ce15d421 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940839c575bc060ce15d421 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:57,861 - lava_callback - INFO - Starting async job submit for node 6940837b575bc060ce15d36e 2025-12-17 21:54:57,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:57,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837b575bc060ce15d36e HTTP/1.1" 502 150 2025-12-17 21:54:57,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837b575bc060ce15d36e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:57,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837b575bc060ce15d36e 2025-12-17 21:54:57,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837b575bc060ce15d36e HTTP/1.1" 502 150 2025-12-17 21:54:57,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837b575bc060ce15d36e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:58,474 - lava_callback - INFO - Received callback for node 6940839d575bc060ce15d424 INFO: 172.21.0.1:55740 - "POST /node/6940839d575bc060ce15d424 HTTP/1.0" 202 Accepted 2025-12-17 21:54:58,593 - lava_callback - INFO - Received callback for node 69408fdf575bc060ce160288 INFO: 172.21.0.1:55744 - "POST /node/69408fdf575bc060ce160288 HTTP/1.0" 202 Accepted 2025-12-17 21:54:59,272 - lava_callback - INFO - Received callback for node 6940839e575bc060ce15d427 INFO: 172.21.0.1:55750 - "POST /node/6940839e575bc060ce15d427 HTTP/1.0" 202 Accepted 2025-12-17 21:54:59,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408356575bc060ce15d2ea 2025-12-17 21:54:59,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408356575bc060ce15d2ea HTTP/1.1" 502 150 2025-12-17 21:54:59,544 - lava_callback - ERROR - Error processing callback for node 69408356575bc060ce15d2ea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408356575bc060ce15d2ea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408356575bc060ce15d2ea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408356575bc060ce15d2ea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:59,546 - lava_callback - INFO - Starting async job submit for node 6940837c575bc060ce15d376 2025-12-17 21:54:59,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:59,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837c575bc060ce15d376 HTTP/1.1" 502 150 2025-12-17 21:54:59,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837c575bc060ce15d376'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:59,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837c575bc060ce15d376 2025-12-17 21:54:59,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837c575bc060ce15d376 HTTP/1.1" 502 150 2025-12-17 21:54:59,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837c575bc060ce15d376'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:59,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837b575bc060ce15d36e 2025-12-17 21:54:59,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837b575bc060ce15d36e HTTP/1.1" 502 150 2025-12-17 21:54:59,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837b575bc060ce15d36e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:59,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f95575bc060ce1601b0 2025-12-17 21:54:59,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f95575bc060ce1601b0 HTTP/1.1" 502 150 2025-12-17 21:54:59,883 - lava_callback - ERROR - Error processing callback for node 69408f95575bc060ce1601b0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f95575bc060ce1601b0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f95575bc060ce1601b0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f95575bc060ce1601b0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:54:59,885 - lava_callback - INFO - Starting async job submit for node 6940837e575bc060ce15d37e 2025-12-17 21:54:59,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:54:59,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837e575bc060ce15d37e HTTP/1.1" 502 150 2025-12-17 21:54:59,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837e575bc060ce15d37e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:59,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837e575bc060ce15d37e 2025-12-17 21:54:59,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837e575bc060ce15d37e HTTP/1.1" 502 150 2025-12-17 21:54:59,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837e575bc060ce15d37e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:54:59,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940836f575bc060ce15d35e 2025-12-17 21:54:59,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940836f575bc060ce15d35e HTTP/1.1" 502 150 2025-12-17 21:54:59,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940836f575bc060ce15d35e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:00,064 - lava_callback - INFO - Received callback for node 694083b2575bc060ce15d45e INFO: 172.21.0.1:50534 - "POST /node/694083b2575bc060ce15d45e HTTP/1.0" 202 Accepted 2025-12-17 21:55:00,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fad575bc060ce160201 2025-12-17 21:55:00,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fad575bc060ce160201 HTTP/1.1" 502 150 2025-12-17 21:55:00,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fad575bc060ce160201'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:00,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408353575bc060ce15d2dd 2025-12-17 21:55:00,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408353575bc060ce15d2dd HTTP/1.1" 502 150 2025-12-17 21:55:00,982 - lava_callback - ERROR - Error processing callback for node 69408353575bc060ce15d2dd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408353575bc060ce15d2dd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408353575bc060ce15d2dd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408353575bc060ce15d2dd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:00,984 - lava_callback - INFO - Starting async job submit for node 6940837f575bc060ce15d383 2025-12-17 21:55:00,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:00,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837f575bc060ce15d383 HTTP/1.1" 502 150 2025-12-17 21:55:00,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837f575bc060ce15d383'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:00,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837f575bc060ce15d383 2025-12-17 21:55:00,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837f575bc060ce15d383 HTTP/1.1" 502 150 2025-12-17 21:55:00,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837f575bc060ce15d383'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:01,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837c575bc060ce15d376 2025-12-17 21:55:01,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837c575bc060ce15d376 HTTP/1.1" 502 150 2025-12-17 21:55:01,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837c575bc060ce15d376'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50542 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:55:01,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837e575bc060ce15d37e 2025-12-17 21:55:01,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837e575bc060ce15d37e HTTP/1.1" 502 150 2025-12-17 21:55:01,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837e575bc060ce15d37e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:02,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837f575bc060ce15d383 2025-12-17 21:55:02,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837f575bc060ce15d383 HTTP/1.1" 502 150 2025-12-17 21:55:02,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837f575bc060ce15d383'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:03,149 - lava_callback - INFO - Received callback for node 694083a2575bc060ce15d434 INFO: 172.21.0.1:50554 - "POST /node/694083a2575bc060ce15d434 HTTP/1.0" 202 Accepted 2025-12-17 21:55:03,798 - lava_callback - INFO - Received callback for node 694083a4575bc060ce15d43d INFO: 172.21.0.1:50566 - "POST /node/694083a4575bc060ce15d43d HTTP/1.0" 202 Accepted 2025-12-17 21:55:03,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837b575bc060ce15d36e 2025-12-17 21:55:03,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837b575bc060ce15d36e HTTP/1.1" 502 150 2025-12-17 21:55:03,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837b575bc060ce15d36e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:03,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408354575bc060ce15d2e4 2025-12-17 21:55:03,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408354575bc060ce15d2e4 HTTP/1.1" 502 150 2025-12-17 21:55:03,975 - lava_callback - ERROR - Error processing callback for node 69408354575bc060ce15d2e4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408354575bc060ce15d2e4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408354575bc060ce15d2e4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408354575bc060ce15d2e4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:03,977 - lava_callback - INFO - Starting async job submit for node 69408381575bc060ce15d386 2025-12-17 21:55:03,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:03,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408381575bc060ce15d386 HTTP/1.1" 502 150 2025-12-17 21:55:03,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408381575bc060ce15d386'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:03,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408381575bc060ce15d386 2025-12-17 21:55:03,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408381575bc060ce15d386 HTTP/1.1" 502 150 2025-12-17 21:55:03,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408381575bc060ce15d386'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:03,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940836e575bc060ce15d359 2025-12-17 21:55:03,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940836e575bc060ce15d359 HTTP/1.1" 502 150 2025-12-17 21:55:03,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940836e575bc060ce15d359'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:04,457 - lava_callback - INFO - Received callback for node 694083af575bc060ce15d458 INFO: 172.21.0.1:50568 - "POST /node/694083af575bc060ce15d458 HTTP/1.0" 202 Accepted 2025-12-17 21:55:05,128 - lava_callback - INFO - Received callback for node 694083b1575bc060ce15d45b INFO: 172.21.0.1:50580 - "POST /node/694083b1575bc060ce15d45b HTTP/1.0" 202 Accepted 2025-12-17 21:55:05,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408358575bc060ce15d2ed 2025-12-17 21:55:05,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408358575bc060ce15d2ed HTTP/1.1" 502 150 2025-12-17 21:55:05,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408358575bc060ce15d2ed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:05,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837c575bc060ce15d376 2025-12-17 21:55:05,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837c575bc060ce15d376 HTTP/1.1" 502 150 2025-12-17 21:55:05,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837c575bc060ce15d376'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:05,825 - lava_callback - INFO - Received callback for node 694083b3575bc060ce15d461 INFO: 172.21.0.1:50592 - "POST /node/694083b3575bc060ce15d461 HTTP/1.0" 202 Accepted 2025-12-17 21:55:05,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837e575bc060ce15d37e 2025-12-17 21:55:05,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837e575bc060ce15d37e HTTP/1.1" 502 150 2025-12-17 21:55:05,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837e575bc060ce15d37e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:05,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408381575bc060ce15d386 2025-12-17 21:55:05,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408381575bc060ce15d386 HTTP/1.1" 502 150 2025-12-17 21:55:05,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408381575bc060ce15d386'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50600 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:55:06,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408343575bc060ce15d297 2025-12-17 21:55:06,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408343575bc060ce15d297 HTTP/1.1" 502 150 2025-12-17 21:55:06,909 - lava_callback - ERROR - Error processing callback for node 69408343575bc060ce15d297: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408343575bc060ce15d297 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408343575bc060ce15d297 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408343575bc060ce15d297 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:06,910 - lava_callback - INFO - Starting async job submit for node 69408382575bc060ce15d389 2025-12-17 21:55:06,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:06,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408382575bc060ce15d389 HTTP/1.1" 502 150 2025-12-17 21:55:06,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408382575bc060ce15d389'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:06,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408382575bc060ce15d389 2025-12-17 21:55:06,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408382575bc060ce15d389 HTTP/1.1" 502 150 2025-12-17 21:55:06,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408382575bc060ce15d389'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:06,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837f575bc060ce15d383 2025-12-17 21:55:06,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837f575bc060ce15d383 HTTP/1.1" 502 150 2025-12-17 21:55:06,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837f575bc060ce15d383'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:07,197 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408359575bc060ce15d2f0 2025-12-17 21:55:07,198 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940835b575bc060ce15d2f3 2025-12-17 21:55:07,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408359575bc060ce15d2f0 HTTP/1.1" 502 150 2025-12-17 21:55:07,199 - lava_callback - ERROR - Error processing callback for node 69408359575bc060ce15d2f0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408359575bc060ce15d2f0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408359575bc060ce15d2f0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408359575bc060ce15d2f0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:07,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940835b575bc060ce15d2f3 HTTP/1.1" 502 150 2025-12-17 21:55:07,201 - lava_callback - ERROR - Error processing callback for node 6940835b575bc060ce15d2f3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940835b575bc060ce15d2f3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940835b575bc060ce15d2f3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940835b575bc060ce15d2f3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:07,201 - lava_callback - INFO - Starting async job submit for node 69408383575bc060ce15d38c 2025-12-17 21:55:07,203 - lava_callback - INFO - Starting async job submit for node 69408384575bc060ce15d38f 2025-12-17 21:55:07,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:07,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:07,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408384575bc060ce15d38f HTTP/1.1" 502 150 2025-12-17 21:55:07,213 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408384575bc060ce15d38f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:07,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408383575bc060ce15d38c HTTP/1.1" 502 150 2025-12-17 21:55:07,213 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408384575bc060ce15d38f 2025-12-17 21:55:07,213 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408383575bc060ce15d38c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:07,214 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408383575bc060ce15d38c 2025-12-17 21:55:07,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408384575bc060ce15d38f HTTP/1.1" 502 150 2025-12-17 21:55:07,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408384575bc060ce15d38f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:07,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408383575bc060ce15d38c HTTP/1.1" 502 150 2025-12-17 21:55:07,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408383575bc060ce15d38c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:07,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940836f575bc060ce15d35e 2025-12-17 21:55:07,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940836f575bc060ce15d35e HTTP/1.1" 502 150 2025-12-17 21:55:07,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940836f575bc060ce15d35e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:08,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fad575bc060ce160201 2025-12-17 21:55:08,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fad575bc060ce160201 HTTP/1.1" 502 150 2025-12-17 21:55:08,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fad575bc060ce160201'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:08,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408382575bc060ce15d389 2025-12-17 21:55:08,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408382575bc060ce15d389 HTTP/1.1" 502 150 2025-12-17 21:55:08,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408382575bc060ce15d389'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:09,215 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408384575bc060ce15d38f 2025-12-17 21:55:09,215 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408383575bc060ce15d38c 2025-12-17 21:55:09,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408384575bc060ce15d38f HTTP/1.1" 502 150 2025-12-17 21:55:09,216 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408384575bc060ce15d38f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:09,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408383575bc060ce15d38c HTTP/1.1" 502 150 2025-12-17 21:55:09,216 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408383575bc060ce15d38c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:09,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408381575bc060ce15d386 2025-12-17 21:55:09,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408381575bc060ce15d386 HTTP/1.1" 502 150 2025-12-17 21:55:09,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408381575bc060ce15d386'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:10,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408379575bc060ce15d367 2025-12-17 21:55:10,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408379575bc060ce15d367 HTTP/1.1" 502 150 2025-12-17 21:55:10,728 - lava_callback - ERROR - Error processing callback for node 69408379575bc060ce15d367: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408379575bc060ce15d367 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408379575bc060ce15d367 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408379575bc060ce15d367 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:10,730 - lava_callback - INFO - Starting async job submit for node 69408386575bc060ce15d3a8 2025-12-17 21:55:10,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:10,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408386575bc060ce15d3a8 HTTP/1.1" 502 150 2025-12-17 21:55:10,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408386575bc060ce15d3a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:10,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408386575bc060ce15d3a8 2025-12-17 21:55:10,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408386575bc060ce15d3a8 HTTP/1.1" 502 150 2025-12-17 21:55:10,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408386575bc060ce15d3a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:11,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837b575bc060ce15d36e 2025-12-17 21:55:11,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837b575bc060ce15d36e HTTP/1.1" 502 150 2025-12-17 21:55:11,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837b575bc060ce15d36e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:12,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408386575bc060ce15d3a8 2025-12-17 21:55:12,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408386575bc060ce15d3a8 HTTP/1.1" 502 150 2025-12-17 21:55:12,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408386575bc060ce15d3a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:12,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408346575bc060ce15d2a0 2025-12-17 21:55:12,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408346575bc060ce15d2a0 HTTP/1.1" 502 150 2025-12-17 21:55:12,796 - lava_callback - ERROR - Error processing callback for node 69408346575bc060ce15d2a0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408346575bc060ce15d2a0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408346575bc060ce15d2a0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408346575bc060ce15d2a0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:12,798 - lava_callback - INFO - Starting async job submit for node 6940838a575bc060ce15d3e2 2025-12-17 21:55:12,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:12,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838a575bc060ce15d3e2 HTTP/1.1" 502 150 2025-12-17 21:55:12,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838a575bc060ce15d3e2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:12,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838a575bc060ce15d3e2 2025-12-17 21:55:12,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838a575bc060ce15d3e2 HTTP/1.1" 502 150 2025-12-17 21:55:12,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838a575bc060ce15d3e2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:12,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408351575bc060ce15d2d4 2025-12-17 21:55:12,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408351575bc060ce15d2d4 HTTP/1.1" 502 150 2025-12-17 21:55:12,817 - lava_callback - ERROR - Error processing callback for node 69408351575bc060ce15d2d4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408351575bc060ce15d2d4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408351575bc060ce15d2d4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408351575bc060ce15d2d4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:12,820 - lava_callback - INFO - Starting async job submit for node 69408388575bc060ce15d3dc 2025-12-17 21:55:12,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:12,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408388575bc060ce15d3dc HTTP/1.1" 502 150 2025-12-17 21:55:12,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408388575bc060ce15d3dc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:12,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408388575bc060ce15d3dc 2025-12-17 21:55:12,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408388575bc060ce15d3dc HTTP/1.1" 502 150 2025-12-17 21:55:12,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408388575bc060ce15d3dc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:12,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408382575bc060ce15d389 2025-12-17 21:55:12,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408382575bc060ce15d389 HTTP/1.1" 502 150 2025-12-17 21:55:12,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408382575bc060ce15d389'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:13,216 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408384575bc060ce15d38f 2025-12-17 21:55:13,217 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408383575bc060ce15d38c 2025-12-17 21:55:13,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408384575bc060ce15d38f HTTP/1.1" 502 150 2025-12-17 21:55:13,218 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408384575bc060ce15d38f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:13,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408383575bc060ce15d38c HTTP/1.1" 502 150 2025-12-17 21:55:13,218 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408383575bc060ce15d38c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:13,474 - lava_callback - INFO - Received callback for node 694083ae575bc060ce15d455 INFO: 172.21.0.1:56674 - "POST /node/694083ae575bc060ce15d455 HTTP/1.0" 202 Accepted 2025-12-17 21:55:13,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837c575bc060ce15d376 2025-12-17 21:55:13,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837c575bc060ce15d376 HTTP/1.1" 502 150 2025-12-17 21:55:13,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837c575bc060ce15d376'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:13,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837e575bc060ce15d37e 2025-12-17 21:55:13,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837e575bc060ce15d37e HTTP/1.1" 502 150 2025-12-17 21:55:13,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837e575bc060ce15d37e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:14,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838a575bc060ce15d3e2 2025-12-17 21:55:14,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838a575bc060ce15d3e2 HTTP/1.1" 502 150 2025-12-17 21:55:14,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838a575bc060ce15d3e2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:14,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408388575bc060ce15d3dc 2025-12-17 21:55:14,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408388575bc060ce15d3dc HTTP/1.1" 502 150 2025-12-17 21:55:14,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408388575bc060ce15d3dc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:14,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837f575bc060ce15d383 2025-12-17 21:55:14,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837f575bc060ce15d383 HTTP/1.1" 502 150 2025-12-17 21:55:14,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940837f575bc060ce15d383'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56692 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:55:16,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408386575bc060ce15d3a8 2025-12-17 21:55:17,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408381575bc060ce15d386 2025-12-17 21:55:17,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408381575bc060ce15d386 HTTP/1.1" 502 150 2025-12-17 21:55:17,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408381575bc060ce15d386'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:18,347 - lava_callback - INFO - Received callback for node 694083b5575bc060ce15d464 INFO: 172.21.0.1:56706 - "POST /node/694083b5575bc060ce15d464 HTTP/1.0" 202 Accepted 2025-12-17 21:55:18,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838a575bc060ce15d3e2 2025-12-17 21:55:18,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838a575bc060ce15d3e2 HTTP/1.1" 502 150 2025-12-17 21:55:18,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838a575bc060ce15d3e2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:18,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408388575bc060ce15d3dc 2025-12-17 21:55:18,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408388575bc060ce15d3dc HTTP/1.1" 502 150 2025-12-17 21:55:18,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408388575bc060ce15d3dc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:19,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940836e575bc060ce15d359 2025-12-17 21:55:19,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940836e575bc060ce15d359 HTTP/1.1" 502 150 2025-12-17 21:55:19,992 - lava_callback - ERROR - Error processing callback for node 6940836e575bc060ce15d359: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940836e575bc060ce15d359 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940836e575bc060ce15d359 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940836e575bc060ce15d359 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:19,994 - lava_callback - INFO - Starting async job submit for node 6940838c575bc060ce15d3e5 2025-12-17 21:55:19,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:20,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838c575bc060ce15d3e5 HTTP/1.1" 502 150 2025-12-17 21:55:20,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838c575bc060ce15d3e5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:20,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838c575bc060ce15d3e5 2025-12-17 21:55:20,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838c575bc060ce15d3e5 HTTP/1.1" 502 150 2025-12-17 21:55:20,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838c575bc060ce15d3e5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:20,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408382575bc060ce15d389 2025-12-17 21:55:20,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408382575bc060ce15d389 HTTP/1.1" 502 150 2025-12-17 21:55:20,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408382575bc060ce15d389'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:21,218 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408384575bc060ce15d38f 2025-12-17 21:55:21,218 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408383575bc060ce15d38c 2025-12-17 21:55:21,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408384575bc060ce15d38f HTTP/1.1" 502 150 2025-12-17 21:55:21,219 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408384575bc060ce15d38f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:21,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408341575bc060ce15d292 HTTP/1.1" 502 150 2025-12-17 21:55:21,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408341575bc060ce15d292'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:21,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408358575bc060ce15d2ed 2025-12-17 21:55:21,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408358575bc060ce15d2ed HTTP/1.1" 502 150 2025-12-17 21:55:21,493 - lava_callback - ERROR - Error processing callback for node 69408358575bc060ce15d2ed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408358575bc060ce15d2ed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408358575bc060ce15d2ed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408358575bc060ce15d2ed (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:21,495 - lava_callback - INFO - Starting async job submit for node 6940838d575bc060ce15d3ea 2025-12-17 21:55:21,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:21,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838d575bc060ce15d3ea HTTP/1.1" 502 150 2025-12-17 21:55:21,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838d575bc060ce15d3ea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:21,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838d575bc060ce15d3ea 2025-12-17 21:55:21,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838d575bc060ce15d3ea HTTP/1.1" 502 150 2025-12-17 21:55:21,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838d575bc060ce15d3ea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:22,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838c575bc060ce15d3e5 2025-12-17 21:55:22,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838c575bc060ce15d3e5 HTTP/1.1" 502 150 2025-12-17 21:55:22,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838c575bc060ce15d3e5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:23,181 - lava_callback - INFO - Received callback for node 694083b6575bc060ce15d46b INFO: 172.21.0.1:60804 - "POST /node/694083b6575bc060ce15d46b HTTP/1.0" 202 Accepted 2025-12-17 21:55:23,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408341575bc060ce15d292 2025-12-17 21:55:23,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838d575bc060ce15d3ea 2025-12-17 21:55:23,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838d575bc060ce15d3ea HTTP/1.1" 502 150 2025-12-17 21:55:23,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838d575bc060ce15d3ea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:23,864 - lava_callback - INFO - Received callback for node 694083b7575bc060ce15d472 INFO: 172.21.0.1:60820 - "POST /node/694083b7575bc060ce15d472 HTTP/1.0" 202 Accepted 2025-12-17 21:55:23,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940836f575bc060ce15d35e 2025-12-17 21:55:23,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940836f575bc060ce15d35e HTTP/1.1" 502 150 2025-12-17 21:55:23,987 - lava_callback - ERROR - Error processing callback for node 6940836f575bc060ce15d35e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940836f575bc060ce15d35e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940836f575bc060ce15d35e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940836f575bc060ce15d35e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:23,989 - lava_callback - INFO - Starting async job submit for node 6940838f575bc060ce15d3f5 2025-12-17 21:55:23,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:24,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838f575bc060ce15d3f5 HTTP/1.1" 502 150 2025-12-17 21:55:24,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838f575bc060ce15d3f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:24,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838f575bc060ce15d3f5 2025-12-17 21:55:24,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838f575bc060ce15d3f5 HTTP/1.1" 502 150 2025-12-17 21:55:24,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838f575bc060ce15d3f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:24,540 - lava_callback - INFO - Received callback for node 694083b9575bc060ce15d47a INFO: 172.21.0.1:60822 - "POST /node/694083b9575bc060ce15d47a HTTP/1.0" 202 Accepted 2025-12-17 21:55:24,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fad575bc060ce160201 2025-12-17 21:55:24,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fad575bc060ce160201 HTTP/1.1" 502 150 2025-12-17 21:55:24,602 - lava_callback - ERROR - Error processing callback for node 69408fad575bc060ce160201: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fad575bc060ce160201 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fad575bc060ce160201 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fad575bc060ce160201 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:24,604 - lava_callback - INFO - Starting async job submit for node 69408390575bc060ce15d3fc 2025-12-17 21:55:24,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:24,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408390575bc060ce15d3fc HTTP/1.1" 502 150 2025-12-17 21:55:24,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408390575bc060ce15d3fc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:24,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408390575bc060ce15d3fc 2025-12-17 21:55:24,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408390575bc060ce15d3fc HTTP/1.1" 502 150 2025-12-17 21:55:24,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408390575bc060ce15d3fc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:25,216 - lava_callback - INFO - Received callback for node 694083ba575bc060ce15d47f INFO: 172.21.0.1:60838 - "POST /node/694083ba575bc060ce15d47f HTTP/1.0" 202 Accepted 2025-12-17 21:55:25,889 - lava_callback - INFO - Received callback for node 694083bc575bc060ce15d482 INFO: 172.21.0.1:60846 - "POST /node/694083bc575bc060ce15d482 HTTP/1.0" 202 Accepted 2025-12-17 21:55:26,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838c575bc060ce15d3e5 2025-12-17 21:55:26,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838c575bc060ce15d3e5 HTTP/1.1" 502 150 2025-12-17 21:55:26,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838c575bc060ce15d3e5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:26,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838f575bc060ce15d3f5 2025-12-17 21:55:26,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838f575bc060ce15d3f5 HTTP/1.1" 502 150 2025-12-17 21:55:26,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838f575bc060ce15d3f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60862 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:55:26,556 - lava_callback - INFO - Received callback for node 694083bd575bc060ce15d485 INFO: 172.21.0.1:60872 - "POST /node/694083bd575bc060ce15d485 HTTP/1.0" 202 Accepted 2025-12-17 21:55:26,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408390575bc060ce15d3fc 2025-12-17 21:55:26,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408390575bc060ce15d3fc HTTP/1.1" 502 150 2025-12-17 21:55:26,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408390575bc060ce15d3fc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:26,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838a575bc060ce15d3e2 2025-12-17 21:55:26,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838a575bc060ce15d3e2 HTTP/1.1" 502 150 2025-12-17 21:55:26,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838a575bc060ce15d3e2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:26,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408388575bc060ce15d3dc 2025-12-17 21:55:26,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408388575bc060ce15d3dc HTTP/1.1" 502 150 2025-12-17 21:55:26,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408388575bc060ce15d3dc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:27,227 - lava_callback - INFO - Received callback for node 694083be575bc060ce15d488 INFO: 172.21.0.1:60884 - "POST /node/694083be575bc060ce15d488 HTTP/1.0" 202 Accepted 2025-12-17 21:55:27,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838d575bc060ce15d3ea 2025-12-17 21:55:27,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838d575bc060ce15d3ea HTTP/1.1" 502 150 2025-12-17 21:55:27,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838d575bc060ce15d3ea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:27,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837b575bc060ce15d36e 2025-12-17 21:55:27,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837b575bc060ce15d36e HTTP/1.1" 502 150 2025-12-17 21:55:27,875 - lava_callback - ERROR - Error processing callback for node 6940837b575bc060ce15d36e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940837b575bc060ce15d36e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940837b575bc060ce15d36e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940837b575bc060ce15d36e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:27,877 - lava_callback - INFO - Starting async job submit for node 69408391575bc060ce15d403 2025-12-17 21:55:27,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:27,878 - lava_callback - INFO - Received callback for node 694083bf575bc060ce15d48b INFO: 172.21.0.1:60892 - "POST /node/694083bf575bc060ce15d48b HTTP/1.0" 202 Accepted 2025-12-17 21:55:27,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408391575bc060ce15d403 HTTP/1.1" 502 150 2025-12-17 21:55:27,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408391575bc060ce15d403'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:27,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408391575bc060ce15d403 2025-12-17 21:55:27,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408391575bc060ce15d403 HTTP/1.1" 502 150 2025-12-17 21:55:27,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408391575bc060ce15d403'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:28,569 - lava_callback - INFO - Received callback for node 694083c0575bc060ce15d48e INFO: 172.21.0.1:60904 - "POST /node/694083c0575bc060ce15d48e HTTP/1.0" 202 Accepted 2025-12-17 21:55:29,234 - lava_callback - INFO - Received callback for node 694083c2575bc060ce15d491 INFO: 172.21.0.1:60920 - "POST /node/694083c2575bc060ce15d491 HTTP/1.0" 202 Accepted 2025-12-17 21:55:29,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837c575bc060ce15d376 2025-12-17 21:55:29,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837c575bc060ce15d376 HTTP/1.1" 502 150 2025-12-17 21:55:29,574 - lava_callback - ERROR - Error processing callback for node 6940837c575bc060ce15d376: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940837c575bc060ce15d376 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940837c575bc060ce15d376 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940837c575bc060ce15d376 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:29,577 - lava_callback - INFO - Starting async job submit for node 69408393575bc060ce15d408 2025-12-17 21:55:29,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:29,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408393575bc060ce15d408 HTTP/1.1" 502 150 2025-12-17 21:55:29,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408393575bc060ce15d408'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:29,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408393575bc060ce15d408 2025-12-17 21:55:29,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408393575bc060ce15d408 HTTP/1.1" 502 150 2025-12-17 21:55:29,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408393575bc060ce15d408'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:29,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408391575bc060ce15d403 2025-12-17 21:55:29,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408391575bc060ce15d403 HTTP/1.1" 502 150 2025-12-17 21:55:29,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408391575bc060ce15d403'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:29,893 - lava_callback - INFO - Received callback for node 694083c3575bc060ce15d494 INFO: 172.21.0.1:52728 - "POST /node/694083c3575bc060ce15d494 HTTP/1.0" 202 Accepted 2025-12-17 21:55:29,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837e575bc060ce15d37e 2025-12-17 21:55:29,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837e575bc060ce15d37e HTTP/1.1" 502 150 2025-12-17 21:55:29,900 - lava_callback - ERROR - Error processing callback for node 6940837e575bc060ce15d37e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940837e575bc060ce15d37e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940837e575bc060ce15d37e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940837e575bc060ce15d37e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:29,901 - lava_callback - INFO - Starting async job submit for node 69408394575bc060ce15d40f 2025-12-17 21:55:29,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:29,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408394575bc060ce15d40f HTTP/1.1" 502 150 2025-12-17 21:55:29,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408394575bc060ce15d40f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:29,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408394575bc060ce15d40f 2025-12-17 21:55:29,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408394575bc060ce15d40f HTTP/1.1" 502 150 2025-12-17 21:55:29,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408394575bc060ce15d40f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:30,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838f575bc060ce15d3f5 2025-12-17 21:55:30,557 - lava_callback - INFO - Received callback for node 694083c5575bc060ce15d4af INFO: 172.21.0.1:52744 - "POST /node/694083c5575bc060ce15d4af HTTP/1.0" 202 Accepted 2025-12-17 21:55:30,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408390575bc060ce15d3fc 2025-12-17 21:55:30,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408390575bc060ce15d3fc HTTP/1.1" 502 150 2025-12-17 21:55:30,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408390575bc060ce15d3fc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:31,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940837f575bc060ce15d383 2025-12-17 21:55:31,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940837f575bc060ce15d383 HTTP/1.1" 502 150 2025-12-17 21:55:31,001 - lava_callback - ERROR - Error processing callback for node 6940837f575bc060ce15d383: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940837f575bc060ce15d383 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940837f575bc060ce15d383 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940837f575bc060ce15d383 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:31,003 - lava_callback - INFO - Starting async job submit for node 69408395575bc060ce15d412 2025-12-17 21:55:31,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:31,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408395575bc060ce15d412 HTTP/1.1" 502 150 2025-12-17 21:55:31,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408395575bc060ce15d412'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:31,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408395575bc060ce15d412 2025-12-17 21:55:31,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408395575bc060ce15d412 HTTP/1.1" 502 150 2025-12-17 21:55:31,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408395575bc060ce15d412'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:31,240 - lava_callback - INFO - Received callback for node 694083c6575bc060ce15d4d2 INFO: 172.21.0.1:52758 - "POST /node/694083c6575bc060ce15d4d2 HTTP/1.0" 202 Accepted 2025-12-17 21:55:31,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408393575bc060ce15d408 2025-12-17 21:55:31,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408393575bc060ce15d408 HTTP/1.1" 502 150 2025-12-17 21:55:31,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408393575bc060ce15d408'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:31,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408394575bc060ce15d40f 2025-12-17 21:55:31,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408394575bc060ce15d40f HTTP/1.1" 502 150 2025-12-17 21:55:31,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408394575bc060ce15d40f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:31,925 - lava_callback - INFO - Received callback for node 694083c8575bc060ce15d4d5 INFO: 172.21.0.1:52760 - "POST /node/694083c8575bc060ce15d4d5 HTTP/1.0" 202 Accepted 2025-12-17 21:55:32,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408386575bc060ce15d3a8 HTTP/1.1" 502 150 2025-12-17 21:55:32,144 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408386575bc060ce15d3a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:32,595 - lava_callback - INFO - Received callback for node 694083c9575bc060ce15d4d8 INFO: 172.21.0.1:52770 - "POST /node/694083c9575bc060ce15d4d8 HTTP/1.0" 202 Accepted 2025-12-17 21:55:33,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408395575bc060ce15d412 2025-12-17 21:55:33,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408391575bc060ce15d403 2025-12-17 21:55:33,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408391575bc060ce15d403 HTTP/1.1" 502 150 2025-12-17 21:55:33,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408391575bc060ce15d403'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:33,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408381575bc060ce15d386 2025-12-17 21:55:33,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408381575bc060ce15d386 HTTP/1.1" 502 150 2025-12-17 21:55:33,994 - lava_callback - ERROR - Error processing callback for node 69408381575bc060ce15d386: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408381575bc060ce15d386 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408381575bc060ce15d386 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408381575bc060ce15d386 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:33,996 - lava_callback - INFO - Starting async job submit for node 69408398575bc060ce15d418 2025-12-17 21:55:33,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:34,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838c575bc060ce15d3e5 2025-12-17 21:55:34,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838c575bc060ce15d3e5 HTTP/1.1" 502 150 2025-12-17 21:55:34,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838c575bc060ce15d3e5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:34,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408398575bc060ce15d418 HTTP/1.1" 502 150 2025-12-17 21:55:34,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408398575bc060ce15d418'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:34,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408398575bc060ce15d418 2025-12-17 21:55:34,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408398575bc060ce15d418 HTTP/1.1" 502 150 2025-12-17 21:55:34,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408398575bc060ce15d418'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:35,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838d575bc060ce15d3ea 2025-12-17 21:55:35,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838d575bc060ce15d3ea HTTP/1.1" 502 150 2025-12-17 21:55:35,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838d575bc060ce15d3ea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:35,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408393575bc060ce15d408 2025-12-17 21:55:35,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408393575bc060ce15d408 HTTP/1.1" 502 150 2025-12-17 21:55:35,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408393575bc060ce15d408'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:35,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408394575bc060ce15d40f 2025-12-17 21:55:35,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408394575bc060ce15d40f HTTP/1.1" 502 150 2025-12-17 21:55:35,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408394575bc060ce15d40f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:36,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408398575bc060ce15d418 2025-12-17 21:55:36,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408398575bc060ce15d418 HTTP/1.1" 502 150 2025-12-17 21:55:36,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408398575bc060ce15d418'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52786 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:55:36,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408382575bc060ce15d389 2025-12-17 21:55:36,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408382575bc060ce15d389 HTTP/1.1" 502 150 2025-12-17 21:55:36,934 - lava_callback - ERROR - Error processing callback for node 69408382575bc060ce15d389: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408382575bc060ce15d389 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408382575bc060ce15d389 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408382575bc060ce15d389 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:36,936 - lava_callback - INFO - Starting async job submit for node 69408399575bc060ce15d41b 2025-12-17 21:55:36,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:36,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408399575bc060ce15d41b HTTP/1.1" 502 150 2025-12-17 21:55:36,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408399575bc060ce15d41b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:36,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408399575bc060ce15d41b 2025-12-17 21:55:36,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408399575bc060ce15d41b HTTP/1.1" 502 150 2025-12-17 21:55:36,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408399575bc060ce15d41b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:37,219 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408384575bc060ce15d38f 2025-12-17 21:55:37,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408384575bc060ce15d38f HTTP/1.1" 502 150 2025-12-17 21:55:37,221 - lava_callback - ERROR - Error processing callback for node 69408384575bc060ce15d38f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408384575bc060ce15d38f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408384575bc060ce15d38f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408384575bc060ce15d38f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:37,224 - lava_callback - INFO - Starting async job submit for node 69408396575bc060ce15d415 2025-12-17 21:55:37,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:37,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408396575bc060ce15d415 HTTP/1.1" 502 150 2025-12-17 21:55:37,233 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408396575bc060ce15d415'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:37,233 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408396575bc060ce15d415 2025-12-17 21:55:37,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408396575bc060ce15d415 HTTP/1.1" 502 150 2025-12-17 21:55:37,234 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408396575bc060ce15d415'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:38,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408390575bc060ce15d3fc 2025-12-17 21:55:38,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408390575bc060ce15d3fc HTTP/1.1" 502 150 2025-12-17 21:55:38,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408390575bc060ce15d3fc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:38,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408341575bc060ce15d292 HTTP/1.1" 502 150 2025-12-17 21:55:38,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408341575bc060ce15d292'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:38,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408399575bc060ce15d41b 2025-12-17 21:55:38,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408399575bc060ce15d41b HTTP/1.1" 502 150 2025-12-17 21:55:38,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408399575bc060ce15d41b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:39,234 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408396575bc060ce15d415 2025-12-17 21:55:39,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408396575bc060ce15d415 HTTP/1.1" 502 150 2025-12-17 21:55:39,235 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408396575bc060ce15d415'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:40,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408398575bc060ce15d418 2025-12-17 21:55:40,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408398575bc060ce15d418 HTTP/1.1" 502 150 2025-12-17 21:55:40,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408398575bc060ce15d418'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:40,145 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408386575bc060ce15d3a8 2025-12-17 21:55:40,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408386575bc060ce15d3a8 HTTP/1.1" 502 150 2025-12-17 21:55:40,146 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408386575bc060ce15d3a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:41,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408391575bc060ce15d403 2025-12-17 21:55:41,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408391575bc060ce15d403 HTTP/1.1" 502 150 2025-12-17 21:55:41,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408391575bc060ce15d403'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:42,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408341575bc060ce15d292 2025-12-17 21:55:42,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408341575bc060ce15d292 HTTP/1.1" 502 150 2025-12-17 21:55:42,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408341575bc060ce15d292'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:42,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838a575bc060ce15d3e2 2025-12-17 21:55:42,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838a575bc060ce15d3e2 HTTP/1.1" 502 150 2025-12-17 21:55:42,819 - lava_callback - ERROR - Error processing callback for node 6940838a575bc060ce15d3e2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940838a575bc060ce15d3e2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940838a575bc060ce15d3e2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940838a575bc060ce15d3e2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:42,821 - lava_callback - INFO - Starting async job submit for node 694083a0575bc060ce15d42a 2025-12-17 21:55:42,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:42,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a0575bc060ce15d42a HTTP/1.1" 502 150 2025-12-17 21:55:42,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a0575bc060ce15d42a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:42,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a0575bc060ce15d42a 2025-12-17 21:55:42,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a0575bc060ce15d42a HTTP/1.1" 502 150 2025-12-17 21:55:42,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a0575bc060ce15d42a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:42,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408388575bc060ce15d3dc 2025-12-17 21:55:42,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408388575bc060ce15d3dc HTTP/1.1" 502 150 2025-12-17 21:55:42,844 - lava_callback - ERROR - Error processing callback for node 69408388575bc060ce15d3dc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408388575bc060ce15d3dc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408388575bc060ce15d3dc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408388575bc060ce15d3dc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:42,846 - lava_callback - INFO - Starting async job submit for node 694083a1575bc060ce15d42d 2025-12-17 21:55:42,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:42,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a1575bc060ce15d42d HTTP/1.1" 502 150 2025-12-17 21:55:42,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a1575bc060ce15d42d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:42,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a1575bc060ce15d42d 2025-12-17 21:55:42,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a1575bc060ce15d42d HTTP/1.1" 502 150 2025-12-17 21:55:42,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a1575bc060ce15d42d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:42,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408399575bc060ce15d41b 2025-12-17 21:55:42,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408399575bc060ce15d41b HTTP/1.1" 502 150 2025-12-17 21:55:42,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408399575bc060ce15d41b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:43,195 - lava_callback - INFO - Received callback for node 694083ca575bc060ce15d4e1 INFO: 172.21.0.1:53662 - "POST /node/694083ca575bc060ce15d4e1 HTTP/1.0" 202 Accepted 2025-12-17 21:55:43,236 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408396575bc060ce15d415 2025-12-17 21:55:43,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408393575bc060ce15d408 2025-12-17 21:55:43,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408393575bc060ce15d408 HTTP/1.1" 502 150 2025-12-17 21:55:43,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408393575bc060ce15d408'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:43,846 - lava_callback - INFO - Received callback for node 694083cc575bc060ce15d4ec INFO: 172.21.0.1:53678 - "POST /node/694083cc575bc060ce15d4ec HTTP/1.0" 202 Accepted 2025-12-17 21:55:43,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408394575bc060ce15d40f 2025-12-17 21:55:43,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408394575bc060ce15d40f HTTP/1.1" 502 150 2025-12-17 21:55:43,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408394575bc060ce15d40f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:44,517 - lava_callback - INFO - Received callback for node 694083cd575bc060ce15d4f5 INFO: 172.21.0.1:53684 - "POST /node/694083cd575bc060ce15d4f5 HTTP/1.0" 202 Accepted 2025-12-17 21:55:44,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a0575bc060ce15d42a 2025-12-17 21:55:44,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a0575bc060ce15d42a HTTP/1.1" 502 150 2025-12-17 21:55:44,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a0575bc060ce15d42a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:44,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a1575bc060ce15d42d 2025-12-17 21:55:44,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a1575bc060ce15d42d HTTP/1.1" 502 150 2025-12-17 21:55:44,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a1575bc060ce15d42d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:45,202 - lava_callback - INFO - Received callback for node 694083ce575bc060ce15d4fc INFO: 172.21.0.1:53690 - "POST /node/694083ce575bc060ce15d4fc HTTP/1.0" 202 Accepted 2025-12-17 21:55:45,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838f575bc060ce15d3f5 HTTP/1.1" 502 150 2025-12-17 21:55:45,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838f575bc060ce15d3f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:45,870 - lava_callback - INFO - Received callback for node 694083cf575bc060ce15d503 INFO: 172.21.0.1:53698 - "POST /node/694083cf575bc060ce15d503 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:53702 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:55:46,532 - lava_callback - INFO - Received callback for node 694083d1575bc060ce15d506 INFO: 172.21.0.1:53704 - "POST /node/694083d1575bc060ce15d506 HTTP/1.0" 202 Accepted 2025-12-17 21:55:47,181 - lava_callback - INFO - Received callback for node 694083d2575bc060ce15d509 INFO: 172.21.0.1:53718 - "POST /node/694083d2575bc060ce15d509 HTTP/1.0" 202 Accepted 2025-12-17 21:55:47,842 - lava_callback - INFO - Received callback for node 694083d3575bc060ce15d50c INFO: 172.21.0.1:53734 - "POST /node/694083d3575bc060ce15d50c HTTP/1.0" 202 Accepted 2025-12-17 21:55:48,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408398575bc060ce15d418 2025-12-17 21:55:48,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408398575bc060ce15d418 HTTP/1.1" 502 150 2025-12-17 21:55:48,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408398575bc060ce15d418'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:48,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408395575bc060ce15d412 HTTP/1.1" 502 150 2025-12-17 21:55:48,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408395575bc060ce15d412'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:48,500 - lava_callback - INFO - Received callback for node 694083db575bc060ce15d518 INFO: 172.21.0.1:53750 - "POST /node/694083db575bc060ce15d518 HTTP/1.0" 202 Accepted 2025-12-17 21:55:48,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a0575bc060ce15d42a 2025-12-17 21:55:48,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a0575bc060ce15d42a HTTP/1.1" 502 150 2025-12-17 21:55:48,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a0575bc060ce15d42a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:48,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a1575bc060ce15d42d 2025-12-17 21:55:48,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a1575bc060ce15d42d HTTP/1.1" 502 150 2025-12-17 21:55:48,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a1575bc060ce15d42d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:49,171 - lava_callback - INFO - Received callback for node 694083dd575bc060ce15d51b INFO: 172.21.0.1:53756 - "POST /node/694083dd575bc060ce15d51b HTTP/1.0" 202 Accepted 2025-12-17 21:55:50,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838c575bc060ce15d3e5 2025-12-17 21:55:50,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838c575bc060ce15d3e5 HTTP/1.1" 502 150 2025-12-17 21:55:50,008 - lava_callback - ERROR - Error processing callback for node 6940838c575bc060ce15d3e5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940838c575bc060ce15d3e5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940838c575bc060ce15d3e5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940838c575bc060ce15d3e5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:50,010 - lava_callback - INFO - Starting async job submit for node 69408fdd575bc060ce160285 2025-12-17 21:55:50,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:50,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fdd575bc060ce160285 HTTP/1.1" 502 150 2025-12-17 21:55:50,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fdd575bc060ce160285'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:50,080 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fdd575bc060ce160285 2025-12-17 21:55:50,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fdd575bc060ce160285 HTTP/1.1" 502 150 2025-12-17 21:55:50,081 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fdd575bc060ce160285'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:50,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408341575bc060ce15d292 2025-12-17 21:55:50,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408399575bc060ce15d41b 2025-12-17 21:55:50,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408399575bc060ce15d41b HTTP/1.1" 502 150 2025-12-17 21:55:50,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408399575bc060ce15d41b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:51,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838d575bc060ce15d3ea 2025-12-17 21:55:51,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838d575bc060ce15d3ea HTTP/1.1" 502 150 2025-12-17 21:55:51,528 - lava_callback - ERROR - Error processing callback for node 6940838d575bc060ce15d3ea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940838d575bc060ce15d3ea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940838d575bc060ce15d3ea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940838d575bc060ce15d3ea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:51,532 - lava_callback - INFO - Starting async job submit for node 694083a5575bc060ce15d444 2025-12-17 21:55:51,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:51,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a5575bc060ce15d444 HTTP/1.1" 502 150 2025-12-17 21:55:51,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a5575bc060ce15d444'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:51,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a5575bc060ce15d444 2025-12-17 21:55:51,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a5575bc060ce15d444 HTTP/1.1" 502 150 2025-12-17 21:55:51,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a5575bc060ce15d444'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:51,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408383575bc060ce15d38c HTTP/1.1" 502 150 2025-12-17 21:55:51,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408383575bc060ce15d38c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:52,081 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fdd575bc060ce160285 2025-12-17 21:55:52,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fdd575bc060ce160285 HTTP/1.1" 502 150 2025-12-17 21:55:52,082 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fdd575bc060ce160285'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:52,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408395575bc060ce15d412 2025-12-17 21:55:52,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408395575bc060ce15d412 HTTP/1.1" 502 150 2025-12-17 21:55:52,274 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408395575bc060ce15d412'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:53,431 - lava_callback - INFO - Received callback for node 694083e3575bc060ce15d536 INFO: 172.21.0.1:58492 - "POST /node/694083e3575bc060ce15d536 HTTP/1.0" 202 Accepted 2025-12-17 21:55:53,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838f575bc060ce15d3f5 2025-12-17 21:55:53,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838f575bc060ce15d3f5 HTTP/1.1" 502 150 2025-12-17 21:55:53,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940838f575bc060ce15d3f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:53,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a5575bc060ce15d444 2025-12-17 21:55:53,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a5575bc060ce15d444 HTTP/1.1" 502 150 2025-12-17 21:55:53,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a5575bc060ce15d444'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:54,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408390575bc060ce15d3fc 2025-12-17 21:55:54,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408390575bc060ce15d3fc HTTP/1.1" 502 150 2025-12-17 21:55:54,617 - lava_callback - ERROR - Error processing callback for node 69408390575bc060ce15d3fc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408390575bc060ce15d3fc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408390575bc060ce15d3fc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408390575bc060ce15d3fc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:54,619 - lava_callback - INFO - Starting async job submit for node 694083a8575bc060ce15d44d 2025-12-17 21:55:54,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:54,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a8575bc060ce15d44d HTTP/1.1" 502 150 2025-12-17 21:55:54,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a8575bc060ce15d44d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:54,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a8575bc060ce15d44d 2025-12-17 21:55:54,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a8575bc060ce15d44d HTTP/1.1" 502 150 2025-12-17 21:55:54,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a8575bc060ce15d44d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:56,082 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fdd575bc060ce160285 2025-12-17 21:55:56,146 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408386575bc060ce15d3a8 2025-12-17 21:55:56,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408386575bc060ce15d3a8 HTTP/1.1" 502 150 2025-12-17 21:55:56,147 - lava_callback - ERROR - Error processing callback for node 69408386575bc060ce15d3a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408386575bc060ce15d3a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408386575bc060ce15d3a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408386575bc060ce15d3a8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:56,149 - lava_callback - INFO - Starting async job submit for node 694083a9575bc060ce15d452 2025-12-17 21:55:56,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:56,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a9575bc060ce15d452 HTTP/1.1" 502 150 2025-12-17 21:55:56,182 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a9575bc060ce15d452'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:56,182 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a9575bc060ce15d452 2025-12-17 21:55:56,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a9575bc060ce15d452 HTTP/1.1" 502 150 2025-12-17 21:55:56,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a9575bc060ce15d452'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58494 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:55:56,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a8575bc060ce15d44d 2025-12-17 21:55:56,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a8575bc060ce15d44d HTTP/1.1" 502 150 2025-12-17 21:55:56,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a8575bc060ce15d44d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:56,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a0575bc060ce15d42a 2025-12-17 21:55:56,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a0575bc060ce15d42a HTTP/1.1" 502 150 2025-12-17 21:55:56,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a0575bc060ce15d42a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:56,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a1575bc060ce15d42d 2025-12-17 21:55:56,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a1575bc060ce15d42d HTTP/1.1" 502 150 2025-12-17 21:55:56,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a1575bc060ce15d42d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:57,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a5575bc060ce15d444 2025-12-17 21:55:57,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a5575bc060ce15d444 HTTP/1.1" 502 150 2025-12-17 21:55:57,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a5575bc060ce15d444'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:57,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408391575bc060ce15d403 2025-12-17 21:55:57,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408391575bc060ce15d403 HTTP/1.1" 502 150 2025-12-17 21:55:57,895 - lava_callback - ERROR - Error processing callback for node 69408391575bc060ce15d403: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408391575bc060ce15d403 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408391575bc060ce15d403 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408391575bc060ce15d403 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:57,897 - lava_callback - INFO - Starting async job submit for node 6940839d575bc060ce15d424 2025-12-17 21:55:57,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:57,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839d575bc060ce15d424 HTTP/1.1" 502 150 2025-12-17 21:55:57,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839d575bc060ce15d424'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:57,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839d575bc060ce15d424 2025-12-17 21:55:57,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839d575bc060ce15d424 HTTP/1.1" 502 150 2025-12-17 21:55:57,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839d575bc060ce15d424'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:58,183 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a9575bc060ce15d452 2025-12-17 21:55:58,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a9575bc060ce15d452 HTTP/1.1" 502 150 2025-12-17 21:55:58,184 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a9575bc060ce15d452'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:58,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408396575bc060ce15d415 HTTP/1.1" 502 150 2025-12-17 21:55:58,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408396575bc060ce15d415'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:59,144 - lava_callback - INFO - Received callback for node 694083eb575bc060ce15d552 INFO: 172.21.0.1:58498 - "POST /node/694083eb575bc060ce15d552 HTTP/1.0" 202 Accepted 2025-12-17 21:55:59,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408393575bc060ce15d408 2025-12-17 21:55:59,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408393575bc060ce15d408 HTTP/1.1" 502 150 2025-12-17 21:55:59,599 - lava_callback - ERROR - Error processing callback for node 69408393575bc060ce15d408: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408393575bc060ce15d408 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408393575bc060ce15d408 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408393575bc060ce15d408 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:59,601 - lava_callback - INFO - Starting async job submit for node 69408fdf575bc060ce160288 2025-12-17 21:55:59,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:59,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fdf575bc060ce160288 HTTP/1.1" 502 150 2025-12-17 21:55:59,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fdf575bc060ce160288'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:59,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fdf575bc060ce160288 2025-12-17 21:55:59,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fdf575bc060ce160288 HTTP/1.1" 502 150 2025-12-17 21:55:59,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fdf575bc060ce160288'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:59,909 - lava_callback - INFO - Received callback for node 694083ed575bc060ce15d558 INFO: 172.21.0.1:51600 - "POST /node/694083ed575bc060ce15d558 HTTP/1.0" 202 Accepted 2025-12-17 21:55:59,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839d575bc060ce15d424 2025-12-17 21:55:59,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839d575bc060ce15d424 HTTP/1.1" 502 150 2025-12-17 21:55:59,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839d575bc060ce15d424'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:59,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408394575bc060ce15d40f 2025-12-17 21:55:59,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408394575bc060ce15d40f HTTP/1.1" 502 150 2025-12-17 21:55:59,924 - lava_callback - ERROR - Error processing callback for node 69408394575bc060ce15d40f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408394575bc060ce15d40f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408394575bc060ce15d40f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408394575bc060ce15d40f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:55:59,925 - lava_callback - INFO - Starting async job submit for node 6940839e575bc060ce15d427 2025-12-17 21:55:59,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:55:59,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839e575bc060ce15d427 HTTP/1.1" 502 150 2025-12-17 21:55:59,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839e575bc060ce15d427'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:55:59,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839e575bc060ce15d427 2025-12-17 21:55:59,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839e575bc060ce15d427 HTTP/1.1" 502 150 2025-12-17 21:55:59,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839e575bc060ce15d427'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:00,274 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408395575bc060ce15d412 2025-12-17 21:56:00,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408395575bc060ce15d412 HTTP/1.1" 502 150 2025-12-17 21:56:00,275 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408395575bc060ce15d412'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:00,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a8575bc060ce15d44d 2025-12-17 21:56:00,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a8575bc060ce15d44d HTTP/1.1" 502 150 2025-12-17 21:56:00,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a8575bc060ce15d44d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:01,581 - lava_callback - INFO - Received callback for node 69408454575bc060ce15d71c INFO: 172.21.0.1:51604 - "POST /node/69408454575bc060ce15d71c HTTP/1.0" 202 Accepted 2025-12-17 21:56:01,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fdf575bc060ce160288 2025-12-17 21:56:01,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fdf575bc060ce160288 HTTP/1.1" 502 150 2025-12-17 21:56:01,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fdf575bc060ce160288'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51618 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:56:01,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839e575bc060ce15d427 2025-12-17 21:56:01,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839e575bc060ce15d427 HTTP/1.1" 502 150 2025-12-17 21:56:01,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839e575bc060ce15d427'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:02,184 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a9575bc060ce15d452 2025-12-17 21:56:02,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a9575bc060ce15d452 HTTP/1.1" 502 150 2025-12-17 21:56:02,186 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a9575bc060ce15d452'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:03,209 - lava_callback - INFO - Received callback for node 694083de575bc060ce15d520 INFO: 172.21.0.1:51628 - "POST /node/694083de575bc060ce15d520 HTTP/1.0" 202 Accepted 2025-12-17 21:56:03,877 - lava_callback - INFO - Received callback for node 694083df575bc060ce15d527 INFO: 172.21.0.1:51638 - "POST /node/694083df575bc060ce15d527 HTTP/1.0" 202 Accepted 2025-12-17 21:56:03,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839d575bc060ce15d424 2025-12-17 21:56:03,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839d575bc060ce15d424 HTTP/1.1" 502 150 2025-12-17 21:56:03,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839d575bc060ce15d424'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:04,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408398575bc060ce15d418 2025-12-17 21:56:04,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408398575bc060ce15d418 HTTP/1.1" 502 150 2025-12-17 21:56:04,034 - lava_callback - ERROR - Error processing callback for node 69408398575bc060ce15d418: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408398575bc060ce15d418 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408398575bc060ce15d418 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408398575bc060ce15d418 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:04,036 - lava_callback - INFO - Starting async job submit for node 694083b2575bc060ce15d45e 2025-12-17 21:56:04,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:04,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b2575bc060ce15d45e HTTP/1.1" 502 150 2025-12-17 21:56:04,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b2575bc060ce15d45e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:04,071 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b2575bc060ce15d45e 2025-12-17 21:56:04,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b2575bc060ce15d45e HTTP/1.1" 502 150 2025-12-17 21:56:04,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b2575bc060ce15d45e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:04,528 - lava_callback - INFO - Received callback for node 694083e1575bc060ce15d52e INFO: 172.21.0.1:51650 - "POST /node/694083e1575bc060ce15d52e HTTP/1.0" 202 Accepted 2025-12-17 21:56:05,180 - lava_callback - INFO - Received callback for node 694083e2575bc060ce15d533 INFO: 172.21.0.1:51662 - "POST /node/694083e2575bc060ce15d533 HTTP/1.0" 202 Accepted 2025-12-17 21:56:05,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a5575bc060ce15d444 2025-12-17 21:56:05,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a5575bc060ce15d444 HTTP/1.1" 502 150 2025-12-17 21:56:05,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a5575bc060ce15d444'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:05,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fdf575bc060ce160288 2025-12-17 21:56:05,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fdf575bc060ce160288 HTTP/1.1" 502 150 2025-12-17 21:56:05,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fdf575bc060ce160288'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:05,871 - lava_callback - INFO - Received callback for node 694083e6575bc060ce15d53e INFO: 172.21.0.1:51672 - "POST /node/694083e6575bc060ce15d53e HTTP/1.0" 202 Accepted 2025-12-17 21:56:05,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839e575bc060ce15d427 2025-12-17 21:56:05,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839e575bc060ce15d427 HTTP/1.1" 502 150 2025-12-17 21:56:05,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839e575bc060ce15d427'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:06,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b2575bc060ce15d45e 2025-12-17 21:56:06,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b2575bc060ce15d45e HTTP/1.1" 502 150 2025-12-17 21:56:06,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b2575bc060ce15d45e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51686 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:56:06,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408396575bc060ce15d415 2025-12-17 21:56:06,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408396575bc060ce15d415 HTTP/1.1" 502 150 2025-12-17 21:56:06,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408396575bc060ce15d415'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:06,532 - lava_callback - INFO - Received callback for node 694083e8575bc060ce15d549 INFO: 172.21.0.1:51700 - "POST /node/694083e8575bc060ce15d549 HTTP/1.0" 202 Accepted 2025-12-17 21:56:06,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408399575bc060ce15d41b 2025-12-17 21:56:06,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408399575bc060ce15d41b HTTP/1.1" 502 150 2025-12-17 21:56:06,963 - lava_callback - ERROR - Error processing callback for node 69408399575bc060ce15d41b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408399575bc060ce15d41b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408399575bc060ce15d41b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408399575bc060ce15d41b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:06,965 - lava_callback - INFO - Starting async job submit for node 694083a2575bc060ce15d434 2025-12-17 21:56:06,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:06,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a2575bc060ce15d434 HTTP/1.1" 502 150 2025-12-17 21:56:06,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a2575bc060ce15d434'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:06,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a2575bc060ce15d434 2025-12-17 21:56:06,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a2575bc060ce15d434 HTTP/1.1" 502 150 2025-12-17 21:56:06,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a2575bc060ce15d434'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:07,230 - lava_callback - INFO - Received callback for node 694083e9575bc060ce15d54f INFO: 172.21.0.1:51706 - "POST /node/694083e9575bc060ce15d54f HTTP/1.0" 202 Accepted 2025-12-17 21:56:07,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408383575bc060ce15d38c 2025-12-17 21:56:07,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408383575bc060ce15d38c HTTP/1.1" 502 150 2025-12-17 21:56:07,858 - lava_callback - ERROR - Error processing callback for node 69408383575bc060ce15d38c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408383575bc060ce15d38c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408383575bc060ce15d38c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408383575bc060ce15d38c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:07,861 - lava_callback - INFO - Starting async job submit for node 694083a4575bc060ce15d43d 2025-12-17 21:56:07,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:07,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a4575bc060ce15d43d HTTP/1.1" 502 150 2025-12-17 21:56:07,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a4575bc060ce15d43d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:07,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a4575bc060ce15d43d 2025-12-17 21:56:07,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a4575bc060ce15d43d HTTP/1.1" 502 150 2025-12-17 21:56:07,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a4575bc060ce15d43d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:07,901 - lava_callback - INFO - Received callback for node 694083ec575bc060ce15d555 INFO: 172.21.0.1:51718 - "POST /node/694083ec575bc060ce15d555 HTTP/1.0" 202 Accepted 2025-12-17 21:56:08,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a8575bc060ce15d44d 2025-12-17 21:56:08,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a8575bc060ce15d44d HTTP/1.1" 502 150 2025-12-17 21:56:08,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a8575bc060ce15d44d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:08,812 - lava_callback - INFO - Received callback for node 694083e5575bc060ce15d539 INFO: 172.21.0.1:51728 - "POST /node/694083e5575bc060ce15d539 HTTP/1.0" 202 Accepted 2025-12-17 21:56:08,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a2575bc060ce15d434 2025-12-17 21:56:08,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a2575bc060ce15d434 HTTP/1.1" 502 150 2025-12-17 21:56:08,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a2575bc060ce15d434'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:09,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940838f575bc060ce15d3f5 2025-12-17 21:56:09,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940838f575bc060ce15d3f5 HTTP/1.1" 502 150 2025-12-17 21:56:09,464 - lava_callback - ERROR - Error processing callback for node 6940838f575bc060ce15d3f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940838f575bc060ce15d3f5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940838f575bc060ce15d3f5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940838f575bc060ce15d3f5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:09,465 - lava_callback - INFO - Starting async job submit for node 694083af575bc060ce15d458 2025-12-17 21:56:09,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:09,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083af575bc060ce15d458 HTTP/1.1" 502 150 2025-12-17 21:56:09,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083af575bc060ce15d458'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:09,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083af575bc060ce15d458 2025-12-17 21:56:09,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083af575bc060ce15d458 HTTP/1.1" 502 150 2025-12-17 21:56:09,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083af575bc060ce15d458'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:09,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a4575bc060ce15d43d 2025-12-17 21:56:09,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a4575bc060ce15d43d HTTP/1.1" 502 150 2025-12-17 21:56:09,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a4575bc060ce15d43d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:10,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b2575bc060ce15d45e 2025-12-17 21:56:10,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b2575bc060ce15d45e HTTP/1.1" 502 150 2025-12-17 21:56:10,074 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b2575bc060ce15d45e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:10,186 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a9575bc060ce15d452 2025-12-17 21:56:10,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a9575bc060ce15d452 HTTP/1.1" 502 150 2025-12-17 21:56:10,187 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a9575bc060ce15d452'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:11,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083af575bc060ce15d458 2025-12-17 21:56:11,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083af575bc060ce15d458 HTTP/1.1" 502 150 2025-12-17 21:56:11,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083af575bc060ce15d458'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:11,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fdd575bc060ce160285 HTTP/1.1" 502 150 2025-12-17 21:56:11,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fdd575bc060ce160285'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:11,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839d575bc060ce15d424 2025-12-17 21:56:11,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839d575bc060ce15d424 HTTP/1.1" 502 150 2025-12-17 21:56:11,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839d575bc060ce15d424'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:12,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a0575bc060ce15d42a 2025-12-17 21:56:12,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a0575bc060ce15d42a HTTP/1.1" 502 150 2025-12-17 21:56:12,846 - lava_callback - ERROR - Error processing callback for node 694083a0575bc060ce15d42a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a0575bc060ce15d42a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a0575bc060ce15d42a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a0575bc060ce15d42a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:12,848 - lava_callback - INFO - Starting async job submit for node 694083b1575bc060ce15d45b 2025-12-17 21:56:12,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:12,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b1575bc060ce15d45b HTTP/1.1" 502 150 2025-12-17 21:56:12,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b1575bc060ce15d45b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:12,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b1575bc060ce15d45b 2025-12-17 21:56:12,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b1575bc060ce15d45b HTTP/1.1" 502 150 2025-12-17 21:56:12,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b1575bc060ce15d45b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:12,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a1575bc060ce15d42d 2025-12-17 21:56:12,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a1575bc060ce15d42d HTTP/1.1" 502 150 2025-12-17 21:56:12,858 - lava_callback - ERROR - Error processing callback for node 694083a1575bc060ce15d42d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a1575bc060ce15d42d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a1575bc060ce15d42d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a1575bc060ce15d42d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:12,860 - lava_callback - INFO - Starting async job submit for node 694083b3575bc060ce15d461 2025-12-17 21:56:12,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:12,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b3575bc060ce15d461 HTTP/1.1" 502 150 2025-12-17 21:56:12,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b3575bc060ce15d461'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:12,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b3575bc060ce15d461 2025-12-17 21:56:12,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b3575bc060ce15d461 HTTP/1.1" 502 150 2025-12-17 21:56:12,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b3575bc060ce15d461'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:12,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a2575bc060ce15d434 2025-12-17 21:56:12,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a2575bc060ce15d434 HTTP/1.1" 502 150 2025-12-17 21:56:12,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a2575bc060ce15d434'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:13,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fdf575bc060ce160288 2025-12-17 21:56:13,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a4575bc060ce15d43d 2025-12-17 21:56:13,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a4575bc060ce15d43d HTTP/1.1" 502 150 2025-12-17 21:56:13,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a4575bc060ce15d43d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:13,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839e575bc060ce15d427 2025-12-17 21:56:13,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839e575bc060ce15d427 HTTP/1.1" 502 150 2025-12-17 21:56:13,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940839e575bc060ce15d427'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:14,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b1575bc060ce15d45b 2025-12-17 21:56:14,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b1575bc060ce15d45b HTTP/1.1" 502 150 2025-12-17 21:56:14,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b1575bc060ce15d45b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:14,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b3575bc060ce15d461 2025-12-17 21:56:14,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b3575bc060ce15d461 HTTP/1.1" 502 150 2025-12-17 21:56:14,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b3575bc060ce15d461'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58596 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:56:15,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083af575bc060ce15d458 2025-12-17 21:56:15,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083af575bc060ce15d458 HTTP/1.1" 502 150 2025-12-17 21:56:15,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083af575bc060ce15d458'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:16,275 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408395575bc060ce15d412 2025-12-17 21:56:16,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408395575bc060ce15d412 HTTP/1.1" 502 150 2025-12-17 21:56:16,277 - lava_callback - ERROR - Error processing callback for node 69408395575bc060ce15d412: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408395575bc060ce15d412 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408395575bc060ce15d412 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408395575bc060ce15d412 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:16,279 - lava_callback - INFO - Starting async job submit for node 694083ae575bc060ce15d455 2025-12-17 21:56:16,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:16,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ae575bc060ce15d455 HTTP/1.1" 502 150 2025-12-17 21:56:16,306 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ae575bc060ce15d455'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:16,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ae575bc060ce15d455 2025-12-17 21:56:16,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ae575bc060ce15d455 HTTP/1.1" 502 150 2025-12-17 21:56:16,307 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ae575bc060ce15d455'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58610 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:56:18,074 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b2575bc060ce15d45e 2025-12-17 21:56:18,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b2575bc060ce15d45e HTTP/1.1" 502 150 2025-12-17 21:56:18,076 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b2575bc060ce15d45e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:18,307 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ae575bc060ce15d455 2025-12-17 21:56:18,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ae575bc060ce15d455 HTTP/1.1" 502 150 2025-12-17 21:56:18,308 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ae575bc060ce15d455'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:18,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b1575bc060ce15d45b 2025-12-17 21:56:18,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b1575bc060ce15d45b HTTP/1.1" 502 150 2025-12-17 21:56:18,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b1575bc060ce15d45b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:18,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b3575bc060ce15d461 2025-12-17 21:56:18,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b3575bc060ce15d461 HTTP/1.1" 502 150 2025-12-17 21:56:18,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b3575bc060ce15d461'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:18,894 - lava_callback - INFO - Received callback for node 694083f3575bc060ce15d566 INFO: 172.21.0.1:58622 - "POST /node/694083f3575bc060ce15d566 HTTP/1.0" 202 Accepted 2025-12-17 21:56:19,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fdd575bc060ce160285 2025-12-17 21:56:19,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fdd575bc060ce160285 HTTP/1.1" 502 150 2025-12-17 21:56:19,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fdd575bc060ce160285'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:19,684 - lava_callback - INFO - Received callback for node 694083f6575bc060ce15d56c INFO: 172.21.0.1:58628 - "POST /node/694083f6575bc060ce15d56c HTTP/1.0" 202 Accepted 2025-12-17 21:56:20,468 - lava_callback - INFO - Received callback for node 694083fc575bc060ce15d57b INFO: 172.21.0.1:47506 - "POST /node/694083fc575bc060ce15d57b HTTP/1.0" 202 Accepted 2025-12-17 21:56:20,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a2575bc060ce15d434 2025-12-17 21:56:20,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a2575bc060ce15d434 HTTP/1.1" 502 150 2025-12-17 21:56:20,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a2575bc060ce15d434'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:21,206 - lava_callback - INFO - Received callback for node 694083fd575bc060ce15d57e INFO: 172.21.0.1:47512 - "POST /node/694083fd575bc060ce15d57e HTTP/1.0" 202 Accepted 2025-12-17 21:56:21,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408341575bc060ce15d292 HTTP/1.1" 502 150 2025-12-17 21:56:21,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408341575bc060ce15d292'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:21,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a5575bc060ce15d444 2025-12-17 21:56:21,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a5575bc060ce15d444 HTTP/1.1" 502 150 2025-12-17 21:56:21,555 - lava_callback - ERROR - Error processing callback for node 694083a5575bc060ce15d444: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a5575bc060ce15d444 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a5575bc060ce15d444 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a5575bc060ce15d444 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:21,557 - lava_callback - INFO - Starting async job submit for node 694083b5575bc060ce15d464 2025-12-17 21:56:21,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:21,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b5575bc060ce15d464 HTTP/1.1" 502 150 2025-12-17 21:56:21,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b5575bc060ce15d464'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:21,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b5575bc060ce15d464 2025-12-17 21:56:21,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b5575bc060ce15d464 HTTP/1.1" 502 150 2025-12-17 21:56:21,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b5575bc060ce15d464'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:21,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a4575bc060ce15d43d 2025-12-17 21:56:21,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a4575bc060ce15d43d HTTP/1.1" 502 150 2025-12-17 21:56:21,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083a4575bc060ce15d43d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:22,008 - lava_callback - INFO - Received callback for node 694083fe575bc060ce15d581 INFO: 172.21.0.1:47516 - "POST /node/694083fe575bc060ce15d581 HTTP/1.0" 202 Accepted 2025-12-17 21:56:22,309 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ae575bc060ce15d455 2025-12-17 21:56:22,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408396575bc060ce15d415 2025-12-17 21:56:22,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408396575bc060ce15d415 HTTP/1.1" 502 150 2025-12-17 21:56:22,515 - lava_callback - ERROR - Error processing callback for node 69408396575bc060ce15d415: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408396575bc060ce15d415 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408396575bc060ce15d415 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408396575bc060ce15d415 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:22,517 - lava_callback - INFO - Starting async job submit for node 694083b6575bc060ce15d46b 2025-12-17 21:56:22,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:22,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b6575bc060ce15d46b HTTP/1.1" 502 150 2025-12-17 21:56:22,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b6575bc060ce15d46b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:22,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b6575bc060ce15d46b 2025-12-17 21:56:22,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b6575bc060ce15d46b HTTP/1.1" 502 150 2025-12-17 21:56:22,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b6575bc060ce15d46b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:22,803 - lava_callback - INFO - Received callback for node 69408400575bc060ce15d584 INFO: 172.21.0.1:47528 - "POST /node/69408400575bc060ce15d584 HTTP/1.0" 202 Accepted 2025-12-17 21:56:23,193 - lava_callback - INFO - Received callback for node 694083ee575bc060ce15d55b INFO: 172.21.0.1:47544 - "POST /node/694083ee575bc060ce15d55b HTTP/1.0" 202 Accepted 2025-12-17 21:56:23,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083af575bc060ce15d458 2025-12-17 21:56:23,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083af575bc060ce15d458 HTTP/1.1" 502 150 2025-12-17 21:56:23,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083af575bc060ce15d458'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:23,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b5575bc060ce15d464 2025-12-17 21:56:23,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b5575bc060ce15d464 HTTP/1.1" 502 150 2025-12-17 21:56:23,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b5575bc060ce15d464'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:23,644 - lava_callback - INFO - Received callback for node 69408401575bc060ce15d587 INFO: 172.21.0.1:47560 - "POST /node/69408401575bc060ce15d587 HTTP/1.0" 202 Accepted 2025-12-17 21:56:23,852 - lava_callback - INFO - Received callback for node 694083f4575bc060ce15d569 INFO: 172.21.0.1:47572 - "POST /node/694083f4575bc060ce15d569 HTTP/1.0" 202 Accepted 2025-12-17 21:56:24,521 - lava_callback - INFO - Received callback for node 694083f7575bc060ce15d56f INFO: 172.21.0.1:47582 - "POST /node/694083f7575bc060ce15d56f HTTP/1.0" 202 Accepted 2025-12-17 21:56:24,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b6575bc060ce15d46b 2025-12-17 21:56:24,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b6575bc060ce15d46b HTTP/1.1" 502 150 2025-12-17 21:56:24,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b6575bc060ce15d46b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:24,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a8575bc060ce15d44d 2025-12-17 21:56:24,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a8575bc060ce15d44d HTTP/1.1" 502 150 2025-12-17 21:56:24,636 - lava_callback - ERROR - Error processing callback for node 694083a8575bc060ce15d44d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a8575bc060ce15d44d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a8575bc060ce15d44d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a8575bc060ce15d44d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:24,638 - lava_callback - INFO - Starting async job submit for node 694083b7575bc060ce15d472 2025-12-17 21:56:24,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:24,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b7575bc060ce15d472 HTTP/1.1" 502 150 2025-12-17 21:56:24,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b7575bc060ce15d472'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:24,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b7575bc060ce15d472 2025-12-17 21:56:24,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b7575bc060ce15d472 HTTP/1.1" 502 150 2025-12-17 21:56:24,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b7575bc060ce15d472'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:25,194 - lava_callback - INFO - Received callback for node 694083f8575bc060ce15d572 INFO: 172.21.0.1:47598 - "POST /node/694083f8575bc060ce15d572 HTTP/1.0" 202 Accepted 2025-12-17 21:56:25,873 - lava_callback - INFO - Received callback for node 694083f9575bc060ce15d575 INFO: 172.21.0.1:47610 - "POST /node/694083f9575bc060ce15d575 HTTP/1.0" 202 Accepted 2025-12-17 21:56:26,187 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a9575bc060ce15d452 2025-12-17 21:56:26,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a9575bc060ce15d452 HTTP/1.1" 502 150 2025-12-17 21:56:26,189 - lava_callback - ERROR - Error processing callback for node 694083a9575bc060ce15d452: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a9575bc060ce15d452 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a9575bc060ce15d452 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a9575bc060ce15d452 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:26,191 - lava_callback - INFO - Starting async job submit for node 694083b9575bc060ce15d47a 2025-12-17 21:56:26,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:26,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b9575bc060ce15d47a HTTP/1.1" 502 150 2025-12-17 21:56:26,197 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b9575bc060ce15d47a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:26,197 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b9575bc060ce15d47a 2025-12-17 21:56:26,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b9575bc060ce15d47a HTTP/1.1" 502 150 2025-12-17 21:56:26,198 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b9575bc060ce15d47a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47614 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:56:26,546 - lava_callback - INFO - Received callback for node 694083fb575bc060ce15d578 INFO: 172.21.0.1:47628 - "POST /node/694083fb575bc060ce15d578 HTTP/1.0" 202 Accepted 2025-12-17 21:56:26,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b7575bc060ce15d472 2025-12-17 21:56:26,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b7575bc060ce15d472 HTTP/1.1" 502 150 2025-12-17 21:56:26,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b7575bc060ce15d472'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:26,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b1575bc060ce15d45b 2025-12-17 21:56:26,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b1575bc060ce15d45b HTTP/1.1" 502 150 2025-12-17 21:56:26,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b1575bc060ce15d45b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:26,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b3575bc060ce15d461 2025-12-17 21:56:26,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b3575bc060ce15d461 HTTP/1.1" 502 150 2025-12-17 21:56:26,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b3575bc060ce15d461'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:27,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b5575bc060ce15d464 2025-12-17 21:56:27,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b5575bc060ce15d464 HTTP/1.1" 502 150 2025-12-17 21:56:27,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b5575bc060ce15d464'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:27,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839d575bc060ce15d424 2025-12-17 21:56:27,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839d575bc060ce15d424 HTTP/1.1" 502 150 2025-12-17 21:56:27,920 - lava_callback - ERROR - Error processing callback for node 6940839d575bc060ce15d424: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940839d575bc060ce15d424 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940839d575bc060ce15d424 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940839d575bc060ce15d424 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:27,922 - lava_callback - INFO - Starting async job submit for node 694083ba575bc060ce15d47f 2025-12-17 21:56:27,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:27,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ba575bc060ce15d47f HTTP/1.1" 502 150 2025-12-17 21:56:27,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ba575bc060ce15d47f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:27,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ba575bc060ce15d47f 2025-12-17 21:56:27,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ba575bc060ce15d47f HTTP/1.1" 502 150 2025-12-17 21:56:27,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ba575bc060ce15d47f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:28,198 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b9575bc060ce15d47a 2025-12-17 21:56:28,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b9575bc060ce15d47a HTTP/1.1" 502 150 2025-12-17 21:56:28,199 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b9575bc060ce15d47a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:28,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b6575bc060ce15d46b 2025-12-17 21:56:28,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b6575bc060ce15d46b HTTP/1.1" 502 150 2025-12-17 21:56:28,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b6575bc060ce15d46b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:28,583 - lava_callback - INFO - Received callback for node 69408403575bc060ce15d5a4 INFO: 172.21.0.1:47634 - "POST /node/69408403575bc060ce15d5a4 HTTP/1.0" 202 Accepted 2025-12-17 21:56:28,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fdf575bc060ce160288 HTTP/1.1" 502 150 2025-12-17 21:56:28,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fdf575bc060ce160288'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:29,251 - lava_callback - INFO - Received callback for node 69408404575bc060ce15d5c5 INFO: 172.21.0.1:47640 - "POST /node/69408404575bc060ce15d5c5 HTTP/1.0" 202 Accepted 2025-12-17 21:56:29,911 - lava_callback - INFO - Received callback for node 69408406575bc060ce15d5ca INFO: 172.21.0.1:57766 - "POST /node/69408406575bc060ce15d5ca HTTP/1.0" 202 Accepted 2025-12-17 21:56:29,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ba575bc060ce15d47f 2025-12-17 21:56:29,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ba575bc060ce15d47f HTTP/1.1" 502 150 2025-12-17 21:56:29,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ba575bc060ce15d47f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:29,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940839e575bc060ce15d427 2025-12-17 21:56:29,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940839e575bc060ce15d427 HTTP/1.1" 502 150 2025-12-17 21:56:29,938 - lava_callback - ERROR - Error processing callback for node 6940839e575bc060ce15d427: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940839e575bc060ce15d427 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940839e575bc060ce15d427 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940839e575bc060ce15d427 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:29,939 - lava_callback - INFO - Starting async job submit for node 694083bc575bc060ce15d482 2025-12-17 21:56:29,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:29,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bc575bc060ce15d482 HTTP/1.1" 502 150 2025-12-17 21:56:29,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083bc575bc060ce15d482'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:29,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083bc575bc060ce15d482 2025-12-17 21:56:29,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bc575bc060ce15d482 HTTP/1.1" 502 150 2025-12-17 21:56:29,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083bc575bc060ce15d482'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:30,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b7575bc060ce15d472 2025-12-17 21:56:30,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b7575bc060ce15d472 HTTP/1.1" 502 150 2025-12-17 21:56:30,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b7575bc060ce15d472'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:31,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083bc575bc060ce15d482 2025-12-17 21:56:31,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bc575bc060ce15d482 HTTP/1.1" 502 150 2025-12-17 21:56:31,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083bc575bc060ce15d482'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:32,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b9575bc060ce15d47a 2025-12-17 21:56:32,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b9575bc060ce15d47a HTTP/1.1" 502 150 2025-12-17 21:56:32,201 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b9575bc060ce15d47a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:33,440 - lava_callback - INFO - Received callback for node 6940840b575bc060ce15d5ea INFO: 172.21.0.1:57770 - "POST /node/6940840b575bc060ce15d5ea HTTP/1.0" 202 Accepted 2025-12-17 21:56:33,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ba575bc060ce15d47f 2025-12-17 21:56:33,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ba575bc060ce15d47f HTTP/1.1" 502 150 2025-12-17 21:56:33,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ba575bc060ce15d47f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:34,076 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b2575bc060ce15d45e 2025-12-17 21:56:34,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b2575bc060ce15d45e HTTP/1.1" 502 150 2025-12-17 21:56:34,077 - lava_callback - ERROR - Error processing callback for node 694083b2575bc060ce15d45e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b2575bc060ce15d45e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b2575bc060ce15d45e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b2575bc060ce15d45e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:34,079 - lava_callback - INFO - Starting async job submit for node 694083bd575bc060ce15d485 2025-12-17 21:56:34,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:34,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bd575bc060ce15d485 HTTP/1.1" 502 150 2025-12-17 21:56:34,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083bd575bc060ce15d485'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:34,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083bd575bc060ce15d485 2025-12-17 21:56:34,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bd575bc060ce15d485 HTTP/1.1" 502 150 2025-12-17 21:56:34,089 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083bd575bc060ce15d485'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:35,263 - lava_callback - INFO - Received callback for node 6940907e575bc060ce1605b3 INFO: 172.21.0.1:57784 - "POST /node/6940907e575bc060ce1605b3 HTTP/1.0" 202 Accepted 2025-12-17 21:56:35,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fdd575bc060ce160285 2025-12-17 21:56:35,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fdd575bc060ce160285 HTTP/1.1" 502 150 2025-12-17 21:56:35,571 - lava_callback - ERROR - Error processing callback for node 69408fdd575bc060ce160285: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fdd575bc060ce160285 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fdd575bc060ce160285 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fdd575bc060ce160285 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:35,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b5575bc060ce15d464 2025-12-17 21:56:35,574 - lava_callback - INFO - Starting async job submit for node 694083be575bc060ce15d488 2025-12-17 21:56:35,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b5575bc060ce15d464 HTTP/1.1" 502 150 2025-12-17 21:56:35,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b5575bc060ce15d464'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:35,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:35,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083be575bc060ce15d488 HTTP/1.1" 502 150 2025-12-17 21:56:35,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083be575bc060ce15d488'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:35,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083be575bc060ce15d488 2025-12-17 21:56:35,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083be575bc060ce15d488 HTTP/1.1" 502 150 2025-12-17 21:56:35,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083be575bc060ce15d488'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:35,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083bc575bc060ce15d482 2025-12-17 21:56:35,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bc575bc060ce15d482 HTTP/1.1" 502 150 2025-12-17 21:56:35,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083bc575bc060ce15d482'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:36,089 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083bd575bc060ce15d485 2025-12-17 21:56:36,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bd575bc060ce15d485 HTTP/1.1" 502 150 2025-12-17 21:56:36,090 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083bd575bc060ce15d485'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:36,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b6575bc060ce15d46b 2025-12-17 21:56:36,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b6575bc060ce15d46b HTTP/1.1" 502 150 2025-12-17 21:56:36,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b6575bc060ce15d46b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57798 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:56:36,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a2575bc060ce15d434 2025-12-17 21:56:36,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a2575bc060ce15d434 HTTP/1.1" 502 150 2025-12-17 21:56:36,978 - lava_callback - ERROR - Error processing callback for node 694083a2575bc060ce15d434: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a2575bc060ce15d434 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a2575bc060ce15d434 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a2575bc060ce15d434 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:36,980 - lava_callback - INFO - Starting async job submit for node 694083bf575bc060ce15d48b 2025-12-17 21:56:36,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:36,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bf575bc060ce15d48b HTTP/1.1" 502 150 2025-12-17 21:56:36,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083bf575bc060ce15d48b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:36,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083bf575bc060ce15d48b 2025-12-17 21:56:36,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bf575bc060ce15d48b HTTP/1.1" 502 150 2025-12-17 21:56:36,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083bf575bc060ce15d48b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:37,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408341575bc060ce15d292 2025-12-17 21:56:37,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083be575bc060ce15d488 2025-12-17 21:56:37,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083be575bc060ce15d488 HTTP/1.1" 502 150 2025-12-17 21:56:37,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083be575bc060ce15d488'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:37,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ae575bc060ce15d455 HTTP/1.1" 502 150 2025-12-17 21:56:37,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ae575bc060ce15d455'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:37,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083a4575bc060ce15d43d 2025-12-17 21:56:37,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083a4575bc060ce15d43d HTTP/1.1" 502 150 2025-12-17 21:56:37,886 - lava_callback - ERROR - Error processing callback for node 694083a4575bc060ce15d43d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a4575bc060ce15d43d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a4575bc060ce15d43d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083a4575bc060ce15d43d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:37,888 - lava_callback - INFO - Starting async job submit for node 694083c0575bc060ce15d48e 2025-12-17 21:56:37,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:37,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c0575bc060ce15d48e HTTP/1.1" 502 150 2025-12-17 21:56:37,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c0575bc060ce15d48e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:37,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c0575bc060ce15d48e 2025-12-17 21:56:37,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c0575bc060ce15d48e HTTP/1.1" 502 150 2025-12-17 21:56:37,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c0575bc060ce15d48e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:38,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b7575bc060ce15d472 2025-12-17 21:56:38,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b7575bc060ce15d472 HTTP/1.1" 502 150 2025-12-17 21:56:38,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b7575bc060ce15d472'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:38,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083bf575bc060ce15d48b 2025-12-17 21:56:38,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bf575bc060ce15d48b HTTP/1.1" 502 150 2025-12-17 21:56:38,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083bf575bc060ce15d48b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:39,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083af575bc060ce15d458 2025-12-17 21:56:39,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c0575bc060ce15d48e 2025-12-17 21:56:39,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c0575bc060ce15d48e HTTP/1.1" 502 150 2025-12-17 21:56:39,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c0575bc060ce15d48e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:40,091 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083bd575bc060ce15d485 2025-12-17 21:56:40,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bd575bc060ce15d485 HTTP/1.1" 502 150 2025-12-17 21:56:40,092 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083bd575bc060ce15d485'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:40,201 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b9575bc060ce15d47a 2025-12-17 21:56:40,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b9575bc060ce15d47a HTTP/1.1" 502 150 2025-12-17 21:56:40,202 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083b9575bc060ce15d47a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:41,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083be575bc060ce15d488 2025-12-17 21:56:41,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083be575bc060ce15d488 HTTP/1.1" 502 150 2025-12-17 21:56:41,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083be575bc060ce15d488'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:41,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ba575bc060ce15d47f 2025-12-17 21:56:41,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ba575bc060ce15d47f HTTP/1.1" 502 150 2025-12-17 21:56:41,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ba575bc060ce15d47f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:42,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b1575bc060ce15d45b 2025-12-17 21:56:42,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b1575bc060ce15d45b HTTP/1.1" 502 150 2025-12-17 21:56:42,868 - lava_callback - ERROR - Error processing callback for node 694083b1575bc060ce15d45b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b1575bc060ce15d45b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b1575bc060ce15d45b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b1575bc060ce15d45b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:42,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b3575bc060ce15d461 2025-12-17 21:56:42,870 - lava_callback - INFO - Starting async job submit for node 694083c2575bc060ce15d491 2025-12-17 21:56:42,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:42,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b3575bc060ce15d461 HTTP/1.1" 502 150 2025-12-17 21:56:42,871 - lava_callback - ERROR - Error processing callback for node 694083b3575bc060ce15d461: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b3575bc060ce15d461 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b3575bc060ce15d461 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b3575bc060ce15d461 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:42,873 - lava_callback - INFO - Starting async job submit for node 694083c3575bc060ce15d494 2025-12-17 21:56:42,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:42,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c3575bc060ce15d494 HTTP/1.1" 502 150 2025-12-17 21:56:42,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c3575bc060ce15d494'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:42,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c3575bc060ce15d494 2025-12-17 21:56:42,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c2575bc060ce15d491 HTTP/1.1" 502 150 2025-12-17 21:56:42,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c2575bc060ce15d491'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:42,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c2575bc060ce15d491 2025-12-17 21:56:42,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c3575bc060ce15d494 HTTP/1.1" 502 150 2025-12-17 21:56:42,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c3575bc060ce15d494'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:42,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c2575bc060ce15d491 HTTP/1.1" 502 150 2025-12-17 21:56:42,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c2575bc060ce15d491'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:42,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083bf575bc060ce15d48b 2025-12-17 21:56:43,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bf575bc060ce15d48b HTTP/1.1" 502 150 2025-12-17 21:56:43,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083bf575bc060ce15d48b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:43,249 - lava_callback - INFO - Received callback for node 69408407575bc060ce15d5d3 INFO: 172.21.0.1:59608 - "POST /node/69408407575bc060ce15d5d3 HTTP/1.0" 202 Accepted 2025-12-17 21:56:43,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c0575bc060ce15d48e 2025-12-17 21:56:43,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c0575bc060ce15d48e HTTP/1.1" 502 150 2025-12-17 21:56:43,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c0575bc060ce15d48e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:43,925 - lava_callback - INFO - Received callback for node 69408409575bc060ce15d5e2 INFO: 172.21.0.1:59624 - "POST /node/69408409575bc060ce15d5e2 HTTP/1.0" 202 Accepted 2025-12-17 21:56:43,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083bc575bc060ce15d482 2025-12-17 21:56:43,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bc575bc060ce15d482 HTTP/1.1" 502 150 2025-12-17 21:56:43,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083bc575bc060ce15d482'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:44,613 - lava_callback - INFO - Received callback for node 6940840a575bc060ce15d5e7 INFO: 172.21.0.1:59628 - "POST /node/6940840a575bc060ce15d5e7 HTTP/1.0" 202 Accepted 2025-12-17 21:56:44,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c3575bc060ce15d494 2025-12-17 21:56:44,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c2575bc060ce15d491 2025-12-17 21:56:44,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c3575bc060ce15d494 HTTP/1.1" 502 150 2025-12-17 21:56:44,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c3575bc060ce15d494'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:44,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c2575bc060ce15d491 HTTP/1.1" 502 150 2025-12-17 21:56:44,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c2575bc060ce15d491'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:44,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fdf575bc060ce160288 2025-12-17 21:56:44,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fdf575bc060ce160288 HTTP/1.1" 502 150 2025-12-17 21:56:44,981 - lava_callback - ERROR - Error processing callback for node 69408fdf575bc060ce160288: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fdf575bc060ce160288 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fdf575bc060ce160288 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fdf575bc060ce160288 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:44,983 - lava_callback - INFO - Starting async job submit for node 694083c5575bc060ce15d4af 2025-12-17 21:56:44,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:44,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c5575bc060ce15d4af HTTP/1.1" 502 150 2025-12-17 21:56:44,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c5575bc060ce15d4af'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:44,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c5575bc060ce15d4af 2025-12-17 21:56:44,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c5575bc060ce15d4af HTTP/1.1" 502 150 2025-12-17 21:56:44,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c5575bc060ce15d4af'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:45,297 - lava_callback - INFO - Received callback for node 6940840d575bc060ce15d5ed INFO: 172.21.0.1:59632 - "POST /node/6940840d575bc060ce15d5ed HTTP/1.0" 202 Accepted 2025-12-17 21:56:45,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ae575bc060ce15d455 2025-12-17 21:56:45,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ae575bc060ce15d455 HTTP/1.1" 502 150 2025-12-17 21:56:45,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ae575bc060ce15d455'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:45,967 - lava_callback - INFO - Received callback for node 6940840f575bc060ce15d5f2 INFO: 172.21.0.1:59636 - "POST /node/6940840f575bc060ce15d5f2 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:59638 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:56:46,634 - lava_callback - INFO - Received callback for node 69408410575bc060ce15d5f5 INFO: 172.21.0.1:59652 - "POST /node/69408410575bc060ce15d5f5 HTTP/1.0" 202 Accepted 2025-12-17 21:56:46,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c5575bc060ce15d4af 2025-12-17 21:56:46,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c5575bc060ce15d4af HTTP/1.1" 502 150 2025-12-17 21:56:46,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c5575bc060ce15d4af'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:47,324 - lava_callback - INFO - Received callback for node 69408411575bc060ce15d5f8 INFO: 172.21.0.1:59666 - "POST /node/69408411575bc060ce15d5f8 HTTP/1.0" 202 Accepted 2025-12-17 21:56:47,996 - lava_callback - INFO - Received callback for node 69408412575bc060ce15d5fb INFO: 172.21.0.1:59668 - "POST /node/69408412575bc060ce15d5fb HTTP/1.0" 202 Accepted 2025-12-17 21:56:48,092 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083bd575bc060ce15d485 2025-12-17 21:56:48,671 - lava_callback - INFO - Received callback for node 69408413575bc060ce15d5fe INFO: 172.21.0.1:59672 - "POST /node/69408413575bc060ce15d5fe HTTP/1.0" 202 Accepted 2025-12-17 21:56:48,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c3575bc060ce15d494 2025-12-17 21:56:48,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c2575bc060ce15d491 2025-12-17 21:56:48,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c3575bc060ce15d494 HTTP/1.1" 502 150 2025-12-17 21:56:48,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c3575bc060ce15d494'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:48,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c2575bc060ce15d491 HTTP/1.1" 502 150 2025-12-17 21:56:48,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c2575bc060ce15d491'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:49,317 - lava_callback - INFO - Received callback for node 69408415575bc060ce15d601 INFO: 172.21.0.1:59678 - "POST /node/69408415575bc060ce15d601 HTTP/1.0" 202 Accepted 2025-12-17 21:56:49,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083be575bc060ce15d488 2025-12-17 21:56:49,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083be575bc060ce15d488 HTTP/1.1" 502 150 2025-12-17 21:56:49,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083be575bc060ce15d488'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:49,972 - lava_callback - INFO - Received callback for node 69408416575bc060ce15d604 INFO: 172.21.0.1:35608 - "POST /node/69408416575bc060ce15d604 HTTP/1.0" 202 Accepted 2025-12-17 21:56:50,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c5575bc060ce15d4af 2025-12-17 21:56:50,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c5575bc060ce15d4af HTTP/1.1" 502 150 2025-12-17 21:56:50,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c5575bc060ce15d4af'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:51,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083bf575bc060ce15d48b 2025-12-17 21:56:51,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b5575bc060ce15d464 2025-12-17 21:56:51,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b5575bc060ce15d464 HTTP/1.1" 502 150 2025-12-17 21:56:51,577 - lava_callback - ERROR - Error processing callback for node 694083b5575bc060ce15d464: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b5575bc060ce15d464 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b5575bc060ce15d464 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b5575bc060ce15d464 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:51,579 - lava_callback - INFO - Starting async job submit for node 694083c6575bc060ce15d4d2 2025-12-17 21:56:51,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:51,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c6575bc060ce15d4d2 HTTP/1.1" 502 150 2025-12-17 21:56:51,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c6575bc060ce15d4d2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:51,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c6575bc060ce15d4d2 2025-12-17 21:56:51,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c6575bc060ce15d4d2 HTTP/1.1" 502 150 2025-12-17 21:56:51,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c6575bc060ce15d4d2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:51,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c0575bc060ce15d48e 2025-12-17 21:56:51,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c0575bc060ce15d48e HTTP/1.1" 502 150 2025-12-17 21:56:51,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c0575bc060ce15d48e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:52,252 - lava_callback - INFO - Received callback for node 6940841a575bc060ce15d611 INFO: 172.21.0.1:35624 - "POST /node/6940841a575bc060ce15d611 HTTP/1.0" 202 Accepted 2025-12-17 21:56:52,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b6575bc060ce15d46b 2025-12-17 21:56:52,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b6575bc060ce15d46b HTTP/1.1" 502 150 2025-12-17 21:56:52,534 - lava_callback - ERROR - Error processing callback for node 694083b6575bc060ce15d46b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b6575bc060ce15d46b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b6575bc060ce15d46b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b6575bc060ce15d46b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:52,536 - lava_callback - INFO - Starting async job submit for node 694083c8575bc060ce15d4d5 2025-12-17 21:56:52,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:52,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c8575bc060ce15d4d5 HTTP/1.1" 502 150 2025-12-17 21:56:52,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c8575bc060ce15d4d5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:52,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c8575bc060ce15d4d5 2025-12-17 21:56:52,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c8575bc060ce15d4d5 HTTP/1.1" 502 150 2025-12-17 21:56:52,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c8575bc060ce15d4d5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:52,888 - lava_callback - INFO - Received callback for node 6940841c575bc060ce15d61e INFO: 172.21.0.1:35632 - "POST /node/6940841c575bc060ce15d61e HTTP/1.0" 202 Accepted 2025-12-17 21:56:53,495 - lava_callback - INFO - Received callback for node 69408417575bc060ce15d607 INFO: 172.21.0.1:35634 - "POST /node/69408417575bc060ce15d607 HTTP/1.0" 202 Accepted 2025-12-17 21:56:53,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c6575bc060ce15d4d2 2025-12-17 21:56:53,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c6575bc060ce15d4d2 HTTP/1.1" 502 150 2025-12-17 21:56:53,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c6575bc060ce15d4d2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:54,169 - lava_callback - INFO - Received callback for node 69408419575bc060ce15d60a INFO: 172.21.0.1:35642 - "POST /node/69408419575bc060ce15d60a HTTP/1.0" 202 Accepted 2025-12-17 21:56:54,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c8575bc060ce15d4d5 2025-12-17 21:56:54,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c8575bc060ce15d4d5 HTTP/1.1" 502 150 2025-12-17 21:56:54,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c8575bc060ce15d4d5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:54,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b7575bc060ce15d472 2025-12-17 21:56:54,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b7575bc060ce15d472 HTTP/1.1" 502 150 2025-12-17 21:56:54,655 - lava_callback - ERROR - Error processing callback for node 694083b7575bc060ce15d472: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b7575bc060ce15d472 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b7575bc060ce15d472 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b7575bc060ce15d472 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:54,657 - lava_callback - INFO - Starting async job submit for node 694083c9575bc060ce15d4d8 2025-12-17 21:56:54,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:54,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c9575bc060ce15d4d8 HTTP/1.1" 502 150 2025-12-17 21:56:54,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c9575bc060ce15d4d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:54,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c9575bc060ce15d4d8 2025-12-17 21:56:54,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c9575bc060ce15d4d8 HTTP/1.1" 502 150 2025-12-17 21:56:54,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c9575bc060ce15d4d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:54,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083af575bc060ce15d458 HTTP/1.1" 502 150 2025-12-17 21:56:54,832 - lava_callback - ERROR - Error processing callback for node 694083af575bc060ce15d458: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083af575bc060ce15d458 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083af575bc060ce15d458 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083af575bc060ce15d458 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:54,834 - lava_callback - INFO - Starting async job submit for node 694083ca575bc060ce15d4e1 2025-12-17 21:56:54,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:54,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ca575bc060ce15d4e1 HTTP/1.1" 502 150 2025-12-17 21:56:54,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ca575bc060ce15d4e1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:54,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ca575bc060ce15d4e1 2025-12-17 21:56:54,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ca575bc060ce15d4e1 HTTP/1.1" 502 150 2025-12-17 21:56:54,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ca575bc060ce15d4e1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:54,841 - lava_callback - INFO - Received callback for node 6940841d575bc060ce15d623 INFO: 172.21.0.1:35658 - "POST /node/6940841d575bc060ce15d623 HTTP/1.0" 202 Accepted 2025-12-17 21:56:55,516 - lava_callback - INFO - Received callback for node 6940841f575bc060ce15d626 INFO: 172.21.0.1:35662 - "POST /node/6940841f575bc060ce15d626 HTTP/1.0" 202 Accepted 2025-12-17 21:56:56,203 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083b9575bc060ce15d47a 2025-12-17 21:56:56,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083b9575bc060ce15d47a HTTP/1.1" 502 150 2025-12-17 21:56:56,204 - lava_callback - ERROR - Error processing callback for node 694083b9575bc060ce15d47a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b9575bc060ce15d47a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b9575bc060ce15d47a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083b9575bc060ce15d47a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:56,206 - lava_callback - INFO - Starting async job submit for node 694083cc575bc060ce15d4ec 2025-12-17 21:56:56,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:56,207 - lava_callback - INFO - Received callback for node 69408420575bc060ce15d629 INFO: 172.21.0.1:35664 - "POST /node/69408420575bc060ce15d629 HTTP/1.0" 202 Accepted 2025-12-17 21:56:56,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cc575bc060ce15d4ec HTTP/1.1" 502 150 2025-12-17 21:56:56,217 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083cc575bc060ce15d4ec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:56,217 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083cc575bc060ce15d4ec 2025-12-17 21:56:56,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cc575bc060ce15d4ec HTTP/1.1" 502 150 2025-12-17 21:56:56,218 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083cc575bc060ce15d4ec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35668 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:56:56,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c9575bc060ce15d4d8 2025-12-17 21:56:56,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c9575bc060ce15d4d8 HTTP/1.1" 502 150 2025-12-17 21:56:56,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c9575bc060ce15d4d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:56,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ca575bc060ce15d4e1 2025-12-17 21:56:56,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ca575bc060ce15d4e1 HTTP/1.1" 502 150 2025-12-17 21:56:56,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ca575bc060ce15d4e1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:56,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c3575bc060ce15d494 2025-12-17 21:56:56,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c2575bc060ce15d491 2025-12-17 21:56:56,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c3575bc060ce15d494 HTTP/1.1" 502 150 2025-12-17 21:56:56,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c3575bc060ce15d494'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:56,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c2575bc060ce15d491 HTTP/1.1" 502 150 2025-12-17 21:56:56,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c2575bc060ce15d491'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:57,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c6575bc060ce15d4d2 2025-12-17 21:56:57,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c6575bc060ce15d4d2 HTTP/1.1" 502 150 2025-12-17 21:56:57,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c6575bc060ce15d4d2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:57,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ba575bc060ce15d47f 2025-12-17 21:56:57,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ba575bc060ce15d47f HTTP/1.1" 502 150 2025-12-17 21:56:57,935 - lava_callback - ERROR - Error processing callback for node 694083ba575bc060ce15d47f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ba575bc060ce15d47f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ba575bc060ce15d47f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ba575bc060ce15d47f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:57,937 - lava_callback - INFO - Starting async job submit for node 694083cd575bc060ce15d4f5 2025-12-17 21:56:57,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:56:57,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cd575bc060ce15d4f5 HTTP/1.1" 502 150 2025-12-17 21:56:57,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083cd575bc060ce15d4f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:57,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083cd575bc060ce15d4f5 2025-12-17 21:56:57,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cd575bc060ce15d4f5 HTTP/1.1" 502 150 2025-12-17 21:56:57,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083cd575bc060ce15d4f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:58,218 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083cc575bc060ce15d4ec 2025-12-17 21:56:58,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cc575bc060ce15d4ec HTTP/1.1" 502 150 2025-12-17 21:56:58,219 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083cc575bc060ce15d4ec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:58,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c8575bc060ce15d4d5 2025-12-17 21:56:58,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c8575bc060ce15d4d5 HTTP/1.1" 502 150 2025-12-17 21:56:58,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c8575bc060ce15d4d5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:58,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c5575bc060ce15d4af 2025-12-17 21:56:58,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c5575bc060ce15d4af HTTP/1.1" 502 150 2025-12-17 21:56:58,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c5575bc060ce15d4af'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:59,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083cd575bc060ce15d4f5 2025-12-17 21:56:59,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cd575bc060ce15d4f5 HTTP/1.1" 502 150 2025-12-17 21:56:59,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083cd575bc060ce15d4f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:56:59,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083bc575bc060ce15d482 2025-12-17 21:56:59,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bc575bc060ce15d482 HTTP/1.1" 502 150 2025-12-17 21:56:59,976 - lava_callback - ERROR - Error processing callback for node 694083bc575bc060ce15d482: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083bc575bc060ce15d482 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083bc575bc060ce15d482 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083bc575bc060ce15d482 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:56:59,978 - lava_callback - INFO - Starting async job submit for node 694083ce575bc060ce15d4fc 2025-12-17 21:56:59,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:00,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ce575bc060ce15d4fc HTTP/1.1" 502 150 2025-12-17 21:57:00,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ce575bc060ce15d4fc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:00,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ce575bc060ce15d4fc 2025-12-17 21:57:00,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ce575bc060ce15d4fc HTTP/1.1" 502 150 2025-12-17 21:57:00,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ce575bc060ce15d4fc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:00,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c9575bc060ce15d4d8 2025-12-17 21:57:00,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c9575bc060ce15d4d8 HTTP/1.1" 502 150 2025-12-17 21:57:00,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c9575bc060ce15d4d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:00,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ca575bc060ce15d4e1 2025-12-17 21:57:00,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ca575bc060ce15d4e1 HTTP/1.1" 502 150 2025-12-17 21:57:00,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ca575bc060ce15d4e1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:01,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ae575bc060ce15d455 2025-12-17 21:57:01,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ae575bc060ce15d455 HTTP/1.1" 502 150 2025-12-17 21:57:01,687 - lava_callback - ERROR - Error processing callback for node 694083ae575bc060ce15d455: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ae575bc060ce15d455 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ae575bc060ce15d455 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ae575bc060ce15d455 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:01,689 - lava_callback - INFO - Starting async job submit for node 694083cf575bc060ce15d503 2025-12-17 21:57:01,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:01,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cf575bc060ce15d503 HTTP/1.1" 502 150 2025-12-17 21:57:01,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083cf575bc060ce15d503'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:01,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083cf575bc060ce15d503 2025-12-17 21:57:01,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cf575bc060ce15d503 HTTP/1.1" 502 150 2025-12-17 21:57:01,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083cf575bc060ce15d503'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39126 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:57:02,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ce575bc060ce15d4fc 2025-12-17 21:57:02,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ce575bc060ce15d4fc HTTP/1.1" 502 150 2025-12-17 21:57:02,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ce575bc060ce15d4fc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:02,219 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083cc575bc060ce15d4ec 2025-12-17 21:57:02,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cc575bc060ce15d4ec HTTP/1.1" 502 150 2025-12-17 21:57:02,220 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083cc575bc060ce15d4ec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:03,230 - lava_callback - INFO - Received callback for node 69408421575bc060ce15d62c INFO: 172.21.0.1:39136 - "POST /node/69408421575bc060ce15d62c HTTP/1.0" 202 Accepted 2025-12-17 21:57:03,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bd575bc060ce15d485 HTTP/1.1" 502 150 2025-12-17 21:57:03,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083bd575bc060ce15d485'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:03,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083cf575bc060ce15d503 2025-12-17 21:57:03,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cf575bc060ce15d503 HTTP/1.1" 502 150 2025-12-17 21:57:03,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083cf575bc060ce15d503'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:03,909 - lava_callback - INFO - Received callback for node 69408423575bc060ce15d62f INFO: 172.21.0.1:39148 - "POST /node/69408423575bc060ce15d62f HTTP/1.0" 202 Accepted 2025-12-17 21:57:03,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083cd575bc060ce15d4f5 2025-12-17 21:57:03,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cd575bc060ce15d4f5 HTTP/1.1" 502 150 2025-12-17 21:57:03,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083cd575bc060ce15d4f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:04,554 - lava_callback - INFO - Received callback for node 69408424575bc060ce15d632 INFO: 172.21.0.1:39154 - "POST /node/69408424575bc060ce15d632 HTTP/1.0" 202 Accepted 2025-12-17 21:57:05,243 - lava_callback - INFO - Received callback for node 69408426575bc060ce15d638 INFO: 172.21.0.1:39168 - "POST /node/69408426575bc060ce15d638 HTTP/1.0" 202 Accepted 2025-12-17 21:57:05,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c6575bc060ce15d4d2 2025-12-17 21:57:05,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083be575bc060ce15d488 2025-12-17 21:57:05,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083be575bc060ce15d488 HTTP/1.1" 502 150 2025-12-17 21:57:05,616 - lava_callback - ERROR - Error processing callback for node 694083be575bc060ce15d488: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083be575bc060ce15d488 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083be575bc060ce15d488 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083be575bc060ce15d488 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:05,618 - lava_callback - INFO - Starting async job submit for node 694083d1575bc060ce15d506 2025-12-17 21:57:05,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:05,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d1575bc060ce15d506 HTTP/1.1" 502 150 2025-12-17 21:57:05,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d1575bc060ce15d506'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:05,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083d1575bc060ce15d506 2025-12-17 21:57:05,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d1575bc060ce15d506 HTTP/1.1" 502 150 2025-12-17 21:57:05,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d1575bc060ce15d506'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:05,896 - lava_callback - INFO - Received callback for node 69408429575bc060ce15d63e INFO: 172.21.0.1:39170 - "POST /node/69408429575bc060ce15d63e HTTP/1.0" 202 Accepted 2025-12-17 21:57:06,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ce575bc060ce15d4fc 2025-12-17 21:57:06,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ce575bc060ce15d4fc HTTP/1.1" 502 150 2025-12-17 21:57:06,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ce575bc060ce15d4fc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:06,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bf575bc060ce15d48b HTTP/1.1" 502 150 2025-12-17 21:57:06,352 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083bf575bc060ce15d48b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39186 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:57:06,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c8575bc060ce15d4d5 2025-12-17 21:57:06,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c8575bc060ce15d4d5 HTTP/1.1" 502 150 2025-12-17 21:57:06,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c8575bc060ce15d4d5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:06,548 - lava_callback - INFO - Received callback for node 6940842b575bc060ce15d644 INFO: 172.21.0.1:39194 - "POST /node/6940842b575bc060ce15d644 HTTP/1.0" 202 Accepted 2025-12-17 21:57:07,224 - lava_callback - INFO - Received callback for node 6940842e575bc060ce15d64a INFO: 172.21.0.1:39208 - "POST /node/6940842e575bc060ce15d64a HTTP/1.0" 202 Accepted 2025-12-17 21:57:07,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083d1575bc060ce15d506 2025-12-17 21:57:07,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d1575bc060ce15d506 HTTP/1.1" 502 150 2025-12-17 21:57:07,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d1575bc060ce15d506'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:07,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083cf575bc060ce15d503 2025-12-17 21:57:07,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cf575bc060ce15d503 HTTP/1.1" 502 150 2025-12-17 21:57:07,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083cf575bc060ce15d503'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:07,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c0575bc060ce15d48e 2025-12-17 21:57:07,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c0575bc060ce15d48e HTTP/1.1" 502 150 2025-12-17 21:57:07,906 - lava_callback - ERROR - Error processing callback for node 694083c0575bc060ce15d48e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c0575bc060ce15d48e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c0575bc060ce15d48e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c0575bc060ce15d48e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:07,908 - lava_callback - INFO - Starting async job submit for node 694083d2575bc060ce15d509 2025-12-17 21:57:07,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:07,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d2575bc060ce15d509 HTTP/1.1" 502 150 2025-12-17 21:57:07,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d2575bc060ce15d509'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:07,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083d2575bc060ce15d509 2025-12-17 21:57:07,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d2575bc060ce15d509 HTTP/1.1" 502 150 2025-12-17 21:57:07,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d2575bc060ce15d509'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:08,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408341575bc060ce15d292 HTTP/1.1" 502 150 2025-12-17 21:57:08,400 - lava_callback - ERROR - Error processing callback for node 69408341575bc060ce15d292: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408341575bc060ce15d292 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408341575bc060ce15d292 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408341575bc060ce15d292 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:08,402 - lava_callback - INFO - Starting async job submit for node 694083d3575bc060ce15d50c 2025-12-17 21:57:08,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:08,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d3575bc060ce15d50c HTTP/1.1" 502 150 2025-12-17 21:57:08,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d3575bc060ce15d50c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:08,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083d3575bc060ce15d50c 2025-12-17 21:57:08,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d3575bc060ce15d50c HTTP/1.1" 502 150 2025-12-17 21:57:08,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d3575bc060ce15d50c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:08,534 - lava_callback - INFO - Received callback for node 6940842f575bc060ce15d64f INFO: 172.21.0.1:39220 - "POST /node/6940842f575bc060ce15d64f HTTP/1.0" 202 Accepted 2025-12-17 21:57:08,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c9575bc060ce15d4d8 2025-12-17 21:57:08,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c9575bc060ce15d4d8 HTTP/1.1" 502 150 2025-12-17 21:57:08,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c9575bc060ce15d4d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:08,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ca575bc060ce15d4e1 2025-12-17 21:57:08,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ca575bc060ce15d4e1 HTTP/1.1" 502 150 2025-12-17 21:57:08,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ca575bc060ce15d4e1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:09,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083d2575bc060ce15d509 2025-12-17 21:57:09,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d2575bc060ce15d509 HTTP/1.1" 502 150 2025-12-17 21:57:09,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d2575bc060ce15d509'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:10,221 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083cc575bc060ce15d4ec 2025-12-17 21:57:10,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cc575bc060ce15d4ec HTTP/1.1" 502 150 2025-12-17 21:57:10,224 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083cc575bc060ce15d4ec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:10,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083d3575bc060ce15d50c 2025-12-17 21:57:10,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d3575bc060ce15d50c HTTP/1.1" 502 150 2025-12-17 21:57:10,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d3575bc060ce15d50c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:11,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083d1575bc060ce15d506 2025-12-17 21:57:11,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d1575bc060ce15d506 HTTP/1.1" 502 150 2025-12-17 21:57:11,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d1575bc060ce15d506'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:11,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083cd575bc060ce15d4f5 2025-12-17 21:57:11,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cd575bc060ce15d4f5 HTTP/1.1" 502 150 2025-12-17 21:57:11,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083cd575bc060ce15d4f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:12,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c3575bc060ce15d494 2025-12-17 21:57:12,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c3575bc060ce15d494 HTTP/1.1" 502 150 2025-12-17 21:57:12,886 - lava_callback - ERROR - Error processing callback for node 694083c3575bc060ce15d494: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c3575bc060ce15d494 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c3575bc060ce15d494 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c3575bc060ce15d494 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:12,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c2575bc060ce15d491 2025-12-17 21:57:12,890 - lava_callback - INFO - Starting async job submit for node 694083db575bc060ce15d518 2025-12-17 21:57:12,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c2575bc060ce15d491 HTTP/1.1" 502 150 2025-12-17 21:57:12,890 - lava_callback - ERROR - Error processing callback for node 694083c2575bc060ce15d491: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c2575bc060ce15d491 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c2575bc060ce15d491 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c2575bc060ce15d491 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:12,893 - lava_callback - INFO - Starting async job submit for node 694083dd575bc060ce15d51b 2025-12-17 21:57:12,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:12,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:12,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083db575bc060ce15d518 HTTP/1.1" 502 150 2025-12-17 21:57:12,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083db575bc060ce15d518'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:12,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083db575bc060ce15d518 2025-12-17 21:57:12,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083dd575bc060ce15d51b HTTP/1.1" 502 150 2025-12-17 21:57:12,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083db575bc060ce15d518 HTTP/1.1" 502 150 2025-12-17 21:57:12,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083db575bc060ce15d518'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:12,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083dd575bc060ce15d51b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:12,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083dd575bc060ce15d51b 2025-12-17 21:57:12,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083dd575bc060ce15d51b HTTP/1.1" 502 150 2025-12-17 21:57:12,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083dd575bc060ce15d51b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:13,453 - lava_callback - INFO - Received callback for node 69408425575bc060ce15d635 INFO: 172.21.0.1:49736 - "POST /node/69408425575bc060ce15d635 HTTP/1.0" 202 Accepted 2025-12-17 21:57:13,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083d2575bc060ce15d509 2025-12-17 21:57:13,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d2575bc060ce15d509 HTTP/1.1" 502 150 2025-12-17 21:57:13,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d2575bc060ce15d509'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:14,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ce575bc060ce15d4fc 2025-12-17 21:57:14,113 - lava_callback - INFO - Received callback for node 69408428575bc060ce15d63b INFO: 172.21.0.1:49750 - "POST /node/69408428575bc060ce15d63b HTTP/1.0" 202 Accepted 2025-12-17 21:57:14,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083d3575bc060ce15d50c 2025-12-17 21:57:14,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d3575bc060ce15d50c HTTP/1.1" 502 150 2025-12-17 21:57:14,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d3575bc060ce15d50c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:14,776 - lava_callback - INFO - Received callback for node 6940842a575bc060ce15d641 INFO: 172.21.0.1:49752 - "POST /node/6940842a575bc060ce15d641 HTTP/1.0" 202 Accepted 2025-12-17 21:57:14,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083db575bc060ce15d518 2025-12-17 21:57:14,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083db575bc060ce15d518 HTTP/1.1" 502 150 2025-12-17 21:57:14,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083db575bc060ce15d518'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:14,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083dd575bc060ce15d51b 2025-12-17 21:57:14,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083dd575bc060ce15d51b HTTP/1.1" 502 150 2025-12-17 21:57:14,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083dd575bc060ce15d51b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:14,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c5575bc060ce15d4af 2025-12-17 21:57:14,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c5575bc060ce15d4af HTTP/1.1" 502 150 2025-12-17 21:57:14,998 - lava_callback - ERROR - Error processing callback for node 694083c5575bc060ce15d4af: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c5575bc060ce15d4af (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c5575bc060ce15d4af (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c5575bc060ce15d4af (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:15,000 - lava_callback - INFO - Starting async job submit for node 694083e3575bc060ce15d536 2025-12-17 21:57:15,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:15,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e3575bc060ce15d536 HTTP/1.1" 502 150 2025-12-17 21:57:15,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e3575bc060ce15d536'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:15,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e3575bc060ce15d536 2025-12-17 21:57:15,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e3575bc060ce15d536 HTTP/1.1" 502 150 2025-12-17 21:57:15,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e3575bc060ce15d536'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49768 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:57:15,436 - lava_callback - INFO - Received callback for node 6940842d575bc060ce15d647 INFO: 172.21.0.1:49784 - "POST /node/6940842d575bc060ce15d647 HTTP/1.0" 202 Accepted 2025-12-17 21:57:15,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083cf575bc060ce15d503 2025-12-17 21:57:15,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cf575bc060ce15d503 HTTP/1.1" 502 150 2025-12-17 21:57:15,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083cf575bc060ce15d503'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:16,121 - lava_callback - INFO - Received callback for node 69408434575bc060ce15d66b INFO: 172.21.0.1:49792 - "POST /node/69408434575bc060ce15d66b HTTP/1.0" 202 Accepted INFO: 172.21.0.1:49798 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:57:16,763 - lava_callback - INFO - Received callback for node 69408437575bc060ce15d675 INFO: 172.21.0.1:49810 - "POST /node/69408437575bc060ce15d675 HTTP/1.0" 202 Accepted 2025-12-17 21:57:17,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e3575bc060ce15d536 2025-12-17 21:57:18,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083db575bc060ce15d518 2025-12-17 21:57:18,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083dd575bc060ce15d51b 2025-12-17 21:57:18,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083db575bc060ce15d518 HTTP/1.1" 502 150 2025-12-17 21:57:18,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083dd575bc060ce15d51b HTTP/1.1" 502 150 2025-12-17 21:57:18,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083dd575bc060ce15d51b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:18,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083db575bc060ce15d518'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:19,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083bd575bc060ce15d485 2025-12-17 21:57:19,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bd575bc060ce15d485 HTTP/1.1" 502 150 2025-12-17 21:57:19,538 - lava_callback - ERROR - Error processing callback for node 694083bd575bc060ce15d485: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083bd575bc060ce15d485 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083bd575bc060ce15d485 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083bd575bc060ce15d485 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:19,541 - lava_callback - INFO - Starting async job submit for node 694083eb575bc060ce15d552 2025-12-17 21:57:19,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:19,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083eb575bc060ce15d552 HTTP/1.1" 502 150 2025-12-17 21:57:19,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083eb575bc060ce15d552'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:19,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083eb575bc060ce15d552 2025-12-17 21:57:19,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083eb575bc060ce15d552 HTTP/1.1" 502 150 2025-12-17 21:57:19,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083eb575bc060ce15d552'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:19,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083d1575bc060ce15d506 2025-12-17 21:57:19,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d1575bc060ce15d506 HTTP/1.1" 502 150 2025-12-17 21:57:19,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d1575bc060ce15d506'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:20,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c6575bc060ce15d4d2 HTTP/1.1" 502 150 2025-12-17 21:57:20,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083c6575bc060ce15d4d2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:21,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083eb575bc060ce15d552 2025-12-17 21:57:21,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083eb575bc060ce15d552 HTTP/1.1" 502 150 2025-12-17 21:57:21,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083eb575bc060ce15d552'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:21,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083d2575bc060ce15d509 2025-12-17 21:57:21,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d2575bc060ce15d509 HTTP/1.1" 502 150 2025-12-17 21:57:21,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d2575bc060ce15d509'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:22,352 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083bf575bc060ce15d48b 2025-12-17 21:57:22,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083bf575bc060ce15d48b HTTP/1.1" 502 150 2025-12-17 21:57:22,354 - lava_callback - ERROR - Error processing callback for node 694083bf575bc060ce15d48b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083bf575bc060ce15d48b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083bf575bc060ce15d48b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083bf575bc060ce15d48b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:22,355 - lava_callback - INFO - Starting async job submit for node 694083ed575bc060ce15d558 2025-12-17 21:57:22,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:22,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ed575bc060ce15d558 HTTP/1.1" 502 150 2025-12-17 21:57:22,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ed575bc060ce15d558'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:22,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ed575bc060ce15d558 2025-12-17 21:57:22,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ed575bc060ce15d558 HTTP/1.1" 502 150 2025-12-17 21:57:22,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ed575bc060ce15d558'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:22,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083d3575bc060ce15d50c 2025-12-17 21:57:22,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d3575bc060ce15d50c HTTP/1.1" 502 150 2025-12-17 21:57:22,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083d3575bc060ce15d50c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:22,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c8575bc060ce15d4d5 2025-12-17 21:57:22,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c8575bc060ce15d4d5 HTTP/1.1" 502 150 2025-12-17 21:57:22,549 - lava_callback - ERROR - Error processing callback for node 694083c8575bc060ce15d4d5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c8575bc060ce15d4d5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c8575bc060ce15d4d5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c8575bc060ce15d4d5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:22,550 - lava_callback - INFO - Starting async job submit for node 69408454575bc060ce15d71c 2025-12-17 21:57:22,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:22,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408454575bc060ce15d71c HTTP/1.1" 502 150 2025-12-17 21:57:22,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408454575bc060ce15d71c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:22,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408454575bc060ce15d71c 2025-12-17 21:57:22,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408454575bc060ce15d71c HTTP/1.1" 502 150 2025-12-17 21:57:22,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408454575bc060ce15d71c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:23,221 - lava_callback - INFO - Received callback for node 69408430575bc060ce15d656 INFO: 172.21.0.1:42258 - "POST /node/69408430575bc060ce15d656 HTTP/1.0" 202 Accepted 2025-12-17 21:57:23,893 - lava_callback - INFO - Received callback for node 69408432575bc060ce15d65d INFO: 172.21.0.1:42272 - "POST /node/69408432575bc060ce15d65d HTTP/1.0" 202 Accepted 2025-12-17 21:57:24,363 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ed575bc060ce15d558 2025-12-17 21:57:24,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ed575bc060ce15d558 HTTP/1.1" 502 150 2025-12-17 21:57:24,364 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ed575bc060ce15d558'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:24,569 - lava_callback - INFO - Received callback for node 69408433575bc060ce15d664 INFO: 172.21.0.1:42278 - "POST /node/69408433575bc060ce15d664 HTTP/1.0" 202 Accepted 2025-12-17 21:57:24,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408454575bc060ce15d71c 2025-12-17 21:57:24,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408454575bc060ce15d71c HTTP/1.1" 502 150 2025-12-17 21:57:24,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408454575bc060ce15d71c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:24,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c9575bc060ce15d4d8 2025-12-17 21:57:24,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c9575bc060ce15d4d8 HTTP/1.1" 502 150 2025-12-17 21:57:24,677 - lava_callback - ERROR - Error processing callback for node 694083c9575bc060ce15d4d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c9575bc060ce15d4d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c9575bc060ce15d4d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c9575bc060ce15d4d8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:24,679 - lava_callback - INFO - Starting async job submit for node 694083de575bc060ce15d520 2025-12-17 21:57:24,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:24,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083de575bc060ce15d520 HTTP/1.1" 502 150 2025-12-17 21:57:24,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083de575bc060ce15d520'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:24,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083de575bc060ce15d520 2025-12-17 21:57:24,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083de575bc060ce15d520 HTTP/1.1" 502 150 2025-12-17 21:57:24,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083de575bc060ce15d520'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:24,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ca575bc060ce15d4e1 2025-12-17 21:57:24,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ca575bc060ce15d4e1 HTTP/1.1" 502 150 2025-12-17 21:57:24,846 - lava_callback - ERROR - Error processing callback for node 694083ca575bc060ce15d4e1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ca575bc060ce15d4e1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ca575bc060ce15d4e1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ca575bc060ce15d4e1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:24,848 - lava_callback - INFO - Starting async job submit for node 694083df575bc060ce15d527 2025-12-17 21:57:24,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:24,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083df575bc060ce15d527 HTTP/1.1" 502 150 2025-12-17 21:57:24,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083df575bc060ce15d527'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:24,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083df575bc060ce15d527 2025-12-17 21:57:24,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083df575bc060ce15d527 HTTP/1.1" 502 150 2025-12-17 21:57:24,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083df575bc060ce15d527'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:25,243 - lava_callback - INFO - Received callback for node 69408436575bc060ce15d672 INFO: 172.21.0.1:42294 - "POST /node/69408436575bc060ce15d672 HTTP/1.0" 202 Accepted 2025-12-17 21:57:25,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083eb575bc060ce15d552 2025-12-17 21:57:25,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083eb575bc060ce15d552 HTTP/1.1" 502 150 2025-12-17 21:57:25,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083eb575bc060ce15d552'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:26,224 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083cc575bc060ce15d4ec 2025-12-17 21:57:26,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cc575bc060ce15d4ec HTTP/1.1" 502 150 2025-12-17 21:57:26,225 - lava_callback - ERROR - Error processing callback for node 694083cc575bc060ce15d4ec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083cc575bc060ce15d4ec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083cc575bc060ce15d4ec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083cc575bc060ce15d4ec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:26,227 - lava_callback - INFO - Starting async job submit for node 694083e1575bc060ce15d52e 2025-12-17 21:57:26,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:26,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e1575bc060ce15d52e HTTP/1.1" 502 150 2025-12-17 21:57:26,238 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e1575bc060ce15d52e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:26,238 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e1575bc060ce15d52e 2025-12-17 21:57:26,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e1575bc060ce15d52e HTTP/1.1" 502 150 2025-12-17 21:57:26,239 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e1575bc060ce15d52e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42298 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:57:26,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083de575bc060ce15d520 2025-12-17 21:57:26,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083de575bc060ce15d520 HTTP/1.1" 502 150 2025-12-17 21:57:26,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083de575bc060ce15d520'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:26,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083df575bc060ce15d527 2025-12-17 21:57:26,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083df575bc060ce15d527 HTTP/1.1" 502 150 2025-12-17 21:57:26,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083df575bc060ce15d527'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:26,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083dd575bc060ce15d51b 2025-12-17 21:57:26,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083db575bc060ce15d518 2025-12-17 21:57:26,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083dd575bc060ce15d51b HTTP/1.1" 502 150 2025-12-17 21:57:26,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083dd575bc060ce15d51b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:26,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083db575bc060ce15d518 HTTP/1.1" 502 150 2025-12-17 21:57:26,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083db575bc060ce15d518'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:27,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083cd575bc060ce15d4f5 2025-12-17 21:57:27,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cd575bc060ce15d4f5 HTTP/1.1" 502 150 2025-12-17 21:57:27,951 - lava_callback - ERROR - Error processing callback for node 694083cd575bc060ce15d4f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083cd575bc060ce15d4f5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083cd575bc060ce15d4f5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083cd575bc060ce15d4f5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:27,953 - lava_callback - INFO - Starting async job submit for node 694083e2575bc060ce15d533 2025-12-17 21:57:27,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:27,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e2575bc060ce15d533 HTTP/1.1" 502 150 2025-12-17 21:57:27,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e2575bc060ce15d533'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:27,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e2575bc060ce15d533 2025-12-17 21:57:27,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e2575bc060ce15d533 HTTP/1.1" 502 150 2025-12-17 21:57:27,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e2575bc060ce15d533'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:28,239 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e1575bc060ce15d52e 2025-12-17 21:57:28,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e1575bc060ce15d52e HTTP/1.1" 502 150 2025-12-17 21:57:28,240 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e1575bc060ce15d52e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:28,364 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ed575bc060ce15d558 2025-12-17 21:57:28,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ed575bc060ce15d558 HTTP/1.1" 502 150 2025-12-17 21:57:28,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ed575bc060ce15d558'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:28,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408454575bc060ce15d71c 2025-12-17 21:57:28,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408454575bc060ce15d71c HTTP/1.1" 502 150 2025-12-17 21:57:28,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408454575bc060ce15d71c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:29,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ce575bc060ce15d4fc HTTP/1.1" 502 150 2025-12-17 21:57:29,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ce575bc060ce15d4fc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:29,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e2575bc060ce15d533 2025-12-17 21:57:29,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e2575bc060ce15d533 HTTP/1.1" 502 150 2025-12-17 21:57:29,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e2575bc060ce15d533'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:30,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083de575bc060ce15d520 2025-12-17 21:57:30,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083de575bc060ce15d520 HTTP/1.1" 502 150 2025-12-17 21:57:30,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083de575bc060ce15d520'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:30,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083df575bc060ce15d527 2025-12-17 21:57:30,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083df575bc060ce15d527 HTTP/1.1" 502 150 2025-12-17 21:57:30,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083df575bc060ce15d527'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:31,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083cf575bc060ce15d503 2025-12-17 21:57:31,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083cf575bc060ce15d503 HTTP/1.1" 502 150 2025-12-17 21:57:31,708 - lava_callback - ERROR - Error processing callback for node 694083cf575bc060ce15d503: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083cf575bc060ce15d503 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083cf575bc060ce15d503 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083cf575bc060ce15d503 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:31,710 - lava_callback - INFO - Starting async job submit for node 694083e6575bc060ce15d53e 2025-12-17 21:57:31,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:31,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e6575bc060ce15d53e HTTP/1.1" 502 150 2025-12-17 21:57:31,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e6575bc060ce15d53e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:31,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e6575bc060ce15d53e 2025-12-17 21:57:31,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e6575bc060ce15d53e HTTP/1.1" 502 150 2025-12-17 21:57:31,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e6575bc060ce15d53e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:32,241 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e1575bc060ce15d52e 2025-12-17 21:57:32,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e1575bc060ce15d52e HTTP/1.1" 502 150 2025-12-17 21:57:32,242 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e1575bc060ce15d52e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:32,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e3575bc060ce15d536 HTTP/1.1" 502 150 2025-12-17 21:57:32,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e3575bc060ce15d536'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:33,479 - lava_callback - INFO - Received callback for node 69408439575bc060ce15d678 INFO: 172.21.0.1:35350 - "POST /node/69408439575bc060ce15d678 HTTP/1.0" 202 Accepted 2025-12-17 21:57:33,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083eb575bc060ce15d552 2025-12-17 21:57:33,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083eb575bc060ce15d552 HTTP/1.1" 502 150 2025-12-17 21:57:33,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083eb575bc060ce15d552'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:33,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e6575bc060ce15d53e 2025-12-17 21:57:33,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e6575bc060ce15d53e HTTP/1.1" 502 150 2025-12-17 21:57:33,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e6575bc060ce15d53e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:33,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e2575bc060ce15d533 2025-12-17 21:57:33,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e2575bc060ce15d533 HTTP/1.1" 502 150 2025-12-17 21:57:33,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e2575bc060ce15d533'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:34,127 - lava_callback - INFO - Received callback for node 6940843a575bc060ce15d67b INFO: 172.21.0.1:35358 - "POST /node/6940843a575bc060ce15d67b HTTP/1.0" 202 Accepted 2025-12-17 21:57:34,787 - lava_callback - INFO - Received callback for node 6940843b575bc060ce15d67e INFO: 172.21.0.1:35364 - "POST /node/6940843b575bc060ce15d67e HTTP/1.0" 202 Accepted 2025-12-17 21:57:35,457 - lava_callback - INFO - Received callback for node 6940843c575bc060ce15d681 INFO: 172.21.0.1:35378 - "POST /node/6940843c575bc060ce15d681 HTTP/1.0" 202 Accepted 2025-12-17 21:57:35,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083d1575bc060ce15d506 2025-12-17 21:57:35,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d1575bc060ce15d506 HTTP/1.1" 502 150 2025-12-17 21:57:35,636 - lava_callback - ERROR - Error processing callback for node 694083d1575bc060ce15d506: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083d1575bc060ce15d506 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083d1575bc060ce15d506 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083d1575bc060ce15d506 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:35,637 - lava_callback - INFO - Starting async job submit for node 694083e8575bc060ce15d549 2025-12-17 21:57:35,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:35,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e8575bc060ce15d549 HTTP/1.1" 502 150 2025-12-17 21:57:35,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e8575bc060ce15d549'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:35,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e8575bc060ce15d549 2025-12-17 21:57:35,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e8575bc060ce15d549 HTTP/1.1" 502 150 2025-12-17 21:57:35,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e8575bc060ce15d549'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:36,151 - lava_callback - INFO - Received callback for node 6940843e575bc060ce15d688 INFO: 172.21.0.1:35380 - "POST /node/6940843e575bc060ce15d688 HTTP/1.0" 202 Accepted 2025-12-17 21:57:36,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ed575bc060ce15d558 2025-12-17 21:57:36,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ed575bc060ce15d558 HTTP/1.1" 502 150 2025-12-17 21:57:36,367 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ed575bc060ce15d558'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:36,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e3575bc060ce15d536 2025-12-17 21:57:36,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e3575bc060ce15d536 HTTP/1.1" 502 150 2025-12-17 21:57:36,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e3575bc060ce15d536'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35390 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:57:36,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408454575bc060ce15d71c 2025-12-17 21:57:36,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408454575bc060ce15d71c HTTP/1.1" 502 150 2025-12-17 21:57:36,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408454575bc060ce15d71c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:36,804 - lava_callback - INFO - Received callback for node 6940843f575bc060ce15d690 INFO: 172.21.0.1:35396 - "POST /node/6940843f575bc060ce15d690 HTTP/1.0" 202 Accepted 2025-12-17 21:57:36,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083c6575bc060ce15d4d2 2025-12-17 21:57:36,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083c6575bc060ce15d4d2 HTTP/1.1" 502 150 2025-12-17 21:57:36,946 - lava_callback - ERROR - Error processing callback for node 694083c6575bc060ce15d4d2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c6575bc060ce15d4d2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c6575bc060ce15d4d2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083c6575bc060ce15d4d2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:36,948 - lava_callback - INFO - Starting async job submit for node 694083e9575bc060ce15d54f 2025-12-17 21:57:36,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:36,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e9575bc060ce15d54f HTTP/1.1" 502 150 2025-12-17 21:57:36,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e9575bc060ce15d54f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:36,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e9575bc060ce15d54f 2025-12-17 21:57:36,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e9575bc060ce15d54f HTTP/1.1" 502 150 2025-12-17 21:57:36,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e9575bc060ce15d54f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:37,479 - lava_callback - INFO - Received callback for node 69408442575bc060ce15d6c7 INFO: 172.21.0.1:35410 - "POST /node/69408442575bc060ce15d6c7 HTTP/1.0" 202 Accepted 2025-12-17 21:57:37,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e8575bc060ce15d549 2025-12-17 21:57:37,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e8575bc060ce15d549 HTTP/1.1" 502 150 2025-12-17 21:57:37,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e8575bc060ce15d549'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:37,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e6575bc060ce15d53e 2025-12-17 21:57:37,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e6575bc060ce15d53e HTTP/1.1" 502 150 2025-12-17 21:57:37,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e6575bc060ce15d53e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:37,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083d2575bc060ce15d509 2025-12-17 21:57:37,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d2575bc060ce15d509 HTTP/1.1" 502 150 2025-12-17 21:57:37,926 - lava_callback - ERROR - Error processing callback for node 694083d2575bc060ce15d509: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083d2575bc060ce15d509 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083d2575bc060ce15d509 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083d2575bc060ce15d509 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:37,927 - lava_callback - INFO - Starting async job submit for node 694083ec575bc060ce15d555 2025-12-17 21:57:37,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:37,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ec575bc060ce15d555 HTTP/1.1" 502 150 2025-12-17 21:57:37,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ec575bc060ce15d555'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:37,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ec575bc060ce15d555 2025-12-17 21:57:37,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ec575bc060ce15d555 HTTP/1.1" 502 150 2025-12-17 21:57:37,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ec575bc060ce15d555'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:38,145 - lava_callback - INFO - Received callback for node 69408447575bc060ce15d6f8 INFO: 172.21.0.1:35422 - "POST /node/69408447575bc060ce15d6f8 HTTP/1.0" 202 Accepted 2025-12-17 21:57:38,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083d3575bc060ce15d50c 2025-12-17 21:57:38,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083d3575bc060ce15d50c HTTP/1.1" 502 150 2025-12-17 21:57:38,428 - lava_callback - ERROR - Error processing callback for node 694083d3575bc060ce15d50c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083d3575bc060ce15d50c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083d3575bc060ce15d50c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083d3575bc060ce15d50c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:38,429 - lava_callback - INFO - Starting async job submit for node 694083e5575bc060ce15d539 2025-12-17 21:57:38,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:38,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e5575bc060ce15d539 HTTP/1.1" 502 150 2025-12-17 21:57:38,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e5575bc060ce15d539'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:38,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e5575bc060ce15d539 2025-12-17 21:57:38,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e5575bc060ce15d539 HTTP/1.1" 502 150 2025-12-17 21:57:38,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e5575bc060ce15d539'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:38,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083de575bc060ce15d520 2025-12-17 21:57:38,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083de575bc060ce15d520 HTTP/1.1" 502 150 2025-12-17 21:57:38,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083de575bc060ce15d520'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:38,787 - lava_callback - INFO - Received callback for node 69408449575bc060ce15d705 INFO: 172.21.0.1:35434 - "POST /node/69408449575bc060ce15d705 HTTP/1.0" 202 Accepted 2025-12-17 21:57:38,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083df575bc060ce15d527 2025-12-17 21:57:38,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083df575bc060ce15d527 HTTP/1.1" 502 150 2025-12-17 21:57:38,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083df575bc060ce15d527'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:38,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e9575bc060ce15d54f 2025-12-17 21:57:38,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e9575bc060ce15d54f HTTP/1.1" 502 150 2025-12-17 21:57:38,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e9575bc060ce15d54f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:39,456 - lava_callback - INFO - Received callback for node 6940844a575bc060ce15d70a INFO: 172.21.0.1:35448 - "POST /node/6940844a575bc060ce15d70a HTTP/1.0" 202 Accepted 2025-12-17 21:57:39,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ec575bc060ce15d555 2025-12-17 21:57:39,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ec575bc060ce15d555 HTTP/1.1" 502 150 2025-12-17 21:57:39,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ec575bc060ce15d555'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:40,138 - lava_callback - INFO - Received callback for node 6940844c575bc060ce15d70d INFO: 172.21.0.1:59142 - "POST /node/6940844c575bc060ce15d70d HTTP/1.0" 202 Accepted 2025-12-17 21:57:40,242 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e1575bc060ce15d52e 2025-12-17 21:57:40,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e5575bc060ce15d539 2025-12-17 21:57:40,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e5575bc060ce15d539 HTTP/1.1" 502 150 2025-12-17 21:57:40,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e5575bc060ce15d539'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:41,451 - lava_callback - INFO - Received callback for node 69408215575bc060ce15cd1f INFO: 172.21.0.1:59154 - "POST /node/69408215575bc060ce15cd1f HTTP/1.0" 202 Accepted 2025-12-17 21:57:41,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e8575bc060ce15d549 2025-12-17 21:57:41,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e8575bc060ce15d549 HTTP/1.1" 502 150 2025-12-17 21:57:41,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e8575bc060ce15d549'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:41,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e2575bc060ce15d533 2025-12-17 21:57:41,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e2575bc060ce15d533 HTTP/1.1" 502 150 2025-12-17 21:57:41,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e2575bc060ce15d533'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:42,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083dd575bc060ce15d51b 2025-12-17 21:57:42,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083db575bc060ce15d518 2025-12-17 21:57:42,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083dd575bc060ce15d51b HTTP/1.1" 502 150 2025-12-17 21:57:42,924 - lava_callback - ERROR - Error processing callback for node 694083dd575bc060ce15d51b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083dd575bc060ce15d51b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083dd575bc060ce15d51b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083dd575bc060ce15d51b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:42,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083db575bc060ce15d518 HTTP/1.1" 502 150 2025-12-17 21:57:42,926 - lava_callback - INFO - Starting async job submit for node 694083f3575bc060ce15d566 2025-12-17 21:57:42,927 - lava_callback - ERROR - Error processing callback for node 694083db575bc060ce15d518: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083db575bc060ce15d518 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083db575bc060ce15d518 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083db575bc060ce15d518 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:42,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:42,930 - lava_callback - INFO - Starting async job submit for node 694083f6575bc060ce15d56c 2025-12-17 21:57:42,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:42,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f6575bc060ce15d56c HTTP/1.1" 502 150 2025-12-17 21:57:42,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f6575bc060ce15d56c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:42,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f6575bc060ce15d56c 2025-12-17 21:57:42,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f6575bc060ce15d56c HTTP/1.1" 502 150 2025-12-17 21:57:42,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f6575bc060ce15d56c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:42,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f3575bc060ce15d566 HTTP/1.1" 502 150 2025-12-17 21:57:42,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f3575bc060ce15d566'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:42,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f3575bc060ce15d566 2025-12-17 21:57:42,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f3575bc060ce15d566 HTTP/1.1" 502 150 2025-12-17 21:57:42,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f3575bc060ce15d566'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:42,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e9575bc060ce15d54f 2025-12-17 21:57:42,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e9575bc060ce15d54f HTTP/1.1" 502 150 2025-12-17 21:57:42,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e9575bc060ce15d54f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:43,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ec575bc060ce15d555 2025-12-17 21:57:43,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ec575bc060ce15d555 HTTP/1.1" 502 150 2025-12-17 21:57:43,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ec575bc060ce15d555'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:44,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e5575bc060ce15d539 2025-12-17 21:57:44,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e3575bc060ce15d536 2025-12-17 21:57:44,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e5575bc060ce15d539 HTTP/1.1" 502 150 2025-12-17 21:57:44,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e5575bc060ce15d539'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:44,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e3575bc060ce15d536 HTTP/1.1" 502 150 2025-12-17 21:57:44,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e3575bc060ce15d536'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:44,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f6575bc060ce15d56c 2025-12-17 21:57:44,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f6575bc060ce15d56c HTTP/1.1" 502 150 2025-12-17 21:57:44,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f6575bc060ce15d56c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:44,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f3575bc060ce15d566 2025-12-17 21:57:44,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f3575bc060ce15d566 HTTP/1.1" 502 150 2025-12-17 21:57:44,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f3575bc060ce15d566'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:45,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ce575bc060ce15d4fc 2025-12-17 21:57:45,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ce575bc060ce15d4fc HTTP/1.1" 502 150 2025-12-17 21:57:45,394 - lava_callback - ERROR - Error processing callback for node 694083ce575bc060ce15d4fc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ce575bc060ce15d4fc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ce575bc060ce15d4fc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ce575bc060ce15d4fc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:45,397 - lava_callback - INFO - Starting async job submit for node 694083fc575bc060ce15d57b 2025-12-17 21:57:45,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:45,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fc575bc060ce15d57b HTTP/1.1" 502 150 2025-12-17 21:57:45,406 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fc575bc060ce15d57b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:45,406 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fc575bc060ce15d57b 2025-12-17 21:57:45,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fc575bc060ce15d57b HTTP/1.1" 502 150 2025-12-17 21:57:45,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fc575bc060ce15d57b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:45,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e6575bc060ce15d53e 2025-12-17 21:57:45,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e6575bc060ce15d53e HTTP/1.1" 502 150 2025-12-17 21:57:45,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e6575bc060ce15d53e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59160 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:57:47,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fc575bc060ce15d57b 2025-12-17 21:57:47,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fc575bc060ce15d57b HTTP/1.1" 502 150 2025-12-17 21:57:47,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fc575bc060ce15d57b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:48,424 - lava_callback - INFO - Received callback for node 6940844e575bc060ce15d713 INFO: 172.21.0.1:59174 - "POST /node/6940844e575bc060ce15d713 HTTP/1.0" 202 Accepted 2025-12-17 21:57:48,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f6575bc060ce15d56c 2025-12-17 21:57:48,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f6575bc060ce15d56c HTTP/1.1" 502 150 2025-12-17 21:57:48,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f6575bc060ce15d56c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:48,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f3575bc060ce15d566 2025-12-17 21:57:48,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f3575bc060ce15d566 HTTP/1.1" 502 150 2025-12-17 21:57:48,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f3575bc060ce15d566'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:49,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083eb575bc060ce15d552 2025-12-17 21:57:49,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083eb575bc060ce15d552 HTTP/1.1" 502 150 2025-12-17 21:57:49,556 - lava_callback - ERROR - Error processing callback for node 694083eb575bc060ce15d552: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083eb575bc060ce15d552 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083eb575bc060ce15d552 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083eb575bc060ce15d552 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:49,558 - lava_callback - INFO - Starting async job submit for node 694083fd575bc060ce15d57e 2025-12-17 21:57:49,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:49,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fd575bc060ce15d57e HTTP/1.1" 502 150 2025-12-17 21:57:49,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fd575bc060ce15d57e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:49,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fd575bc060ce15d57e 2025-12-17 21:57:49,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fd575bc060ce15d57e HTTP/1.1" 502 150 2025-12-17 21:57:49,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fd575bc060ce15d57e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:49,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e8575bc060ce15d549 2025-12-17 21:57:49,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e8575bc060ce15d549 HTTP/1.1" 502 150 2025-12-17 21:57:49,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e8575bc060ce15d549'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:50,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e9575bc060ce15d54f 2025-12-17 21:57:50,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e9575bc060ce15d54f HTTP/1.1" 502 150 2025-12-17 21:57:50,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e9575bc060ce15d54f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:51,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fc575bc060ce15d57b 2025-12-17 21:57:51,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fc575bc060ce15d57b HTTP/1.1" 502 150 2025-12-17 21:57:51,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fc575bc060ce15d57b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:51,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fd575bc060ce15d57e 2025-12-17 21:57:51,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fd575bc060ce15d57e HTTP/1.1" 502 150 2025-12-17 21:57:51,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fd575bc060ce15d57e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:51,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ec575bc060ce15d555 2025-12-17 21:57:51,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ec575bc060ce15d555 HTTP/1.1" 502 150 2025-12-17 21:57:51,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ec575bc060ce15d555'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:52,367 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ed575bc060ce15d558 2025-12-17 21:57:52,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ed575bc060ce15d558 HTTP/1.1" 502 150 2025-12-17 21:57:52,369 - lava_callback - ERROR - Error processing callback for node 694083ed575bc060ce15d558: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ed575bc060ce15d558 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ed575bc060ce15d558 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ed575bc060ce15d558 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:52,372 - lava_callback - INFO - Starting async job submit for node 694083fe575bc060ce15d581 2025-12-17 21:57:52,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:52,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fe575bc060ce15d581 HTTP/1.1" 502 150 2025-12-17 21:57:52,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fe575bc060ce15d581'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:52,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fe575bc060ce15d581 2025-12-17 21:57:52,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fe575bc060ce15d581 HTTP/1.1" 502 150 2025-12-17 21:57:52,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fe575bc060ce15d581'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:52,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e5575bc060ce15d539 2025-12-17 21:57:52,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e5575bc060ce15d539 HTTP/1.1" 502 150 2025-12-17 21:57:52,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e5575bc060ce15d539'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:52,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408454575bc060ce15d71c 2025-12-17 21:57:52,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408454575bc060ce15d71c HTTP/1.1" 502 150 2025-12-17 21:57:52,600 - lava_callback - ERROR - Error processing callback for node 69408454575bc060ce15d71c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408454575bc060ce15d71c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408454575bc060ce15d71c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408454575bc060ce15d71c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:52,602 - lava_callback - INFO - Starting async job submit for node 69408400575bc060ce15d584 2025-12-17 21:57:52,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:52,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408400575bc060ce15d584 HTTP/1.1" 502 150 2025-12-17 21:57:52,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408400575bc060ce15d584'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:52,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408400575bc060ce15d584 2025-12-17 21:57:52,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408400575bc060ce15d584 HTTP/1.1" 502 150 2025-12-17 21:57:52,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408400575bc060ce15d584'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:53,390 - lava_callback - INFO - Received callback for node 69408456575bc060ce15d723 INFO: 172.21.0.1:57504 - "POST /node/69408456575bc060ce15d723 HTTP/1.0" 202 Accepted 2025-12-17 21:57:53,521 - lava_callback - INFO - Received callback for node 6940844d575bc060ce15d710 INFO: 172.21.0.1:57510 - "POST /node/6940844d575bc060ce15d710 HTTP/1.0" 202 Accepted 2025-12-17 21:57:54,381 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fe575bc060ce15d581 2025-12-17 21:57:54,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fe575bc060ce15d581 HTTP/1.1" 502 150 2025-12-17 21:57:54,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fe575bc060ce15d581'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:54,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408400575bc060ce15d584 2025-12-17 21:57:54,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408400575bc060ce15d584 HTTP/1.1" 502 150 2025-12-17 21:57:54,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408400575bc060ce15d584'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:54,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083de575bc060ce15d520 2025-12-17 21:57:54,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083de575bc060ce15d520 HTTP/1.1" 502 150 2025-12-17 21:57:54,700 - lava_callback - ERROR - Error processing callback for node 694083de575bc060ce15d520: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083de575bc060ce15d520 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083de575bc060ce15d520 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083de575bc060ce15d520 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:54,702 - lava_callback - INFO - Starting async job submit for node 694083ee575bc060ce15d55b 2025-12-17 21:57:54,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:54,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ee575bc060ce15d55b HTTP/1.1" 502 150 2025-12-17 21:57:54,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ee575bc060ce15d55b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:54,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ee575bc060ce15d55b 2025-12-17 21:57:54,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ee575bc060ce15d55b HTTP/1.1" 502 150 2025-12-17 21:57:54,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ee575bc060ce15d55b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:54,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083df575bc060ce15d527 2025-12-17 21:57:54,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083df575bc060ce15d527 HTTP/1.1" 502 150 2025-12-17 21:57:54,874 - lava_callback - ERROR - Error processing callback for node 694083df575bc060ce15d527: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083df575bc060ce15d527 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083df575bc060ce15d527 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083df575bc060ce15d527 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:54,876 - lava_callback - INFO - Starting async job submit for node 69408401575bc060ce15d587 2025-12-17 21:57:54,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:54,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408401575bc060ce15d587 HTTP/1.1" 502 150 2025-12-17 21:57:54,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408401575bc060ce15d587'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:54,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408401575bc060ce15d587 2025-12-17 21:57:54,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408401575bc060ce15d587 HTTP/1.1" 502 150 2025-12-17 21:57:54,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408401575bc060ce15d587'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:55,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e1575bc060ce15d52e HTTP/1.1" 502 150 2025-12-17 21:57:55,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083e1575bc060ce15d52e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:55,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fd575bc060ce15d57e 2025-12-17 21:57:55,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fd575bc060ce15d57e HTTP/1.1" 502 150 2025-12-17 21:57:55,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fd575bc060ce15d57e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57522 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:57:56,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ee575bc060ce15d55b 2025-12-17 21:57:56,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ee575bc060ce15d55b HTTP/1.1" 502 150 2025-12-17 21:57:56,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ee575bc060ce15d55b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:56,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408401575bc060ce15d587 2025-12-17 21:57:56,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408401575bc060ce15d587 HTTP/1.1" 502 150 2025-12-17 21:57:56,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408401575bc060ce15d587'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:56,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f6575bc060ce15d56c 2025-12-17 21:57:56,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f6575bc060ce15d56c HTTP/1.1" 502 150 2025-12-17 21:57:56,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f3575bc060ce15d566 2025-12-17 21:57:56,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f6575bc060ce15d56c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:56,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f3575bc060ce15d566 HTTP/1.1" 502 150 2025-12-17 21:57:56,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f3575bc060ce15d566'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:57,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e2575bc060ce15d533 2025-12-17 21:57:57,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e2575bc060ce15d533 HTTP/1.1" 502 150 2025-12-17 21:57:57,966 - lava_callback - ERROR - Error processing callback for node 694083e2575bc060ce15d533: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e2575bc060ce15d533 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e2575bc060ce15d533 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e2575bc060ce15d533 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:57:57,968 - lava_callback - INFO - Starting async job submit for node 694083f4575bc060ce15d569 2025-12-17 21:57:57,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:57:57,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f4575bc060ce15d569 HTTP/1.1" 502 150 2025-12-17 21:57:57,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f4575bc060ce15d569'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:57,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f4575bc060ce15d569 2025-12-17 21:57:57,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f4575bc060ce15d569 HTTP/1.1" 502 150 2025-12-17 21:57:57,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f4575bc060ce15d569'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:58,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fe575bc060ce15d581 2025-12-17 21:57:58,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408400575bc060ce15d584 2025-12-17 21:57:58,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408400575bc060ce15d584 HTTP/1.1" 502 150 2025-12-17 21:57:58,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408400575bc060ce15d584'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:59,250 - lava_callback - INFO - Received callback for node 6940845d575bc060ce15d740 INFO: 172.21.0.1:57524 - "POST /node/6940845d575bc060ce15d740 HTTP/1.0" 202 Accepted 2025-12-17 21:57:59,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fc575bc060ce15d57b 2025-12-17 21:57:59,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fc575bc060ce15d57b HTTP/1.1" 502 150 2025-12-17 21:57:59,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fc575bc060ce15d57b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:57:59,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f4575bc060ce15d569 2025-12-17 21:57:59,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f4575bc060ce15d569 HTTP/1.1" 502 150 2025-12-17 21:57:59,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f4575bc060ce15d569'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:00,023 - lava_callback - INFO - Received callback for node 6940845e575bc060ce15d743 INFO: 172.21.0.1:43758 - "POST /node/6940845e575bc060ce15d743 HTTP/1.0" 202 Accepted 2025-12-17 21:58:00,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e3575bc060ce15d536 2025-12-17 21:58:00,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e3575bc060ce15d536 HTTP/1.1" 502 150 2025-12-17 21:58:00,469 - lava_callback - ERROR - Error processing callback for node 694083e3575bc060ce15d536: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e3575bc060ce15d536 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e3575bc060ce15d536 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e3575bc060ce15d536 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:00,471 - lava_callback - INFO - Starting async job submit for node 694083f7575bc060ce15d56f 2025-12-17 21:58:00,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:00,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f7575bc060ce15d56f HTTP/1.1" 502 150 2025-12-17 21:58:00,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f7575bc060ce15d56f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:00,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f7575bc060ce15d56f 2025-12-17 21:58:00,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f7575bc060ce15d56f HTTP/1.1" 502 150 2025-12-17 21:58:00,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f7575bc060ce15d56f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:00,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ee575bc060ce15d55b 2025-12-17 21:58:00,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ee575bc060ce15d55b HTTP/1.1" 502 150 2025-12-17 21:58:00,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ee575bc060ce15d55b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:00,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408401575bc060ce15d587 2025-12-17 21:58:00,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408401575bc060ce15d587 HTTP/1.1" 502 150 2025-12-17 21:58:00,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408401575bc060ce15d587'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:00,955 - lava_callback - INFO - Received callback for node 6940845f575bc060ce15d746 INFO: 172.21.0.1:43770 - "POST /node/6940845f575bc060ce15d746 HTTP/1.0" 202 Accepted 2025-12-17 21:58:01,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e6575bc060ce15d53e 2025-12-17 21:58:01,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e6575bc060ce15d53e HTTP/1.1" 502 150 2025-12-17 21:58:01,729 - lava_callback - ERROR - Error processing callback for node 694083e6575bc060ce15d53e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e6575bc060ce15d53e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e6575bc060ce15d53e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e6575bc060ce15d53e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:01,731 - lava_callback - INFO - Starting async job submit for node 694083f8575bc060ce15d572 2025-12-17 21:58:01,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:01,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f8575bc060ce15d572 HTTP/1.1" 502 150 2025-12-17 21:58:01,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f8575bc060ce15d572'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:01,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f8575bc060ce15d572 2025-12-17 21:58:01,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f8575bc060ce15d572 HTTP/1.1" 502 150 2025-12-17 21:58:01,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f8575bc060ce15d572'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:01,764 - lava_callback - INFO - Received callback for node 69408460575bc060ce15d749 INFO: 172.21.0.1:43772 - "POST /node/69408460575bc060ce15d749 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:43774 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:58:02,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f7575bc060ce15d56f 2025-12-17 21:58:02,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f7575bc060ce15d56f HTTP/1.1" 502 150 2025-12-17 21:58:02,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f7575bc060ce15d56f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:02,529 - lava_callback - INFO - Received callback for node 69408462575bc060ce15d74c INFO: 172.21.0.1:43788 - "POST /node/69408462575bc060ce15d74c HTTP/1.0" 202 Accepted 2025-12-17 21:58:03,289 - lava_callback - INFO - Received callback for node 69408463575bc060ce15d74f INFO: 172.21.0.1:43798 - "POST /node/69408463575bc060ce15d74f HTTP/1.0" 202 Accepted 2025-12-17 21:58:03,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fd575bc060ce15d57e 2025-12-17 21:58:03,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fd575bc060ce15d57e HTTP/1.1" 502 150 2025-12-17 21:58:03,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fd575bc060ce15d57e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:03,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f8575bc060ce15d572 2025-12-17 21:58:03,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f8575bc060ce15d572 HTTP/1.1" 502 150 2025-12-17 21:58:03,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f8575bc060ce15d572'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:03,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f4575bc060ce15d569 2025-12-17 21:58:03,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f4575bc060ce15d569 HTTP/1.1" 502 150 2025-12-17 21:58:03,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f4575bc060ce15d569'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:04,103 - lava_callback - INFO - Received callback for node 69408464575bc060ce15d752 INFO: 172.21.0.1:43812 - "POST /node/69408464575bc060ce15d752 HTTP/1.0" 202 Accepted 2025-12-17 21:58:04,847 - lava_callback - INFO - Received callback for node 69408465575bc060ce15d755 INFO: 172.21.0.1:43818 - "POST /node/69408465575bc060ce15d755 HTTP/1.0" 202 Accepted 2025-12-17 21:58:05,644 - lava_callback - INFO - Received callback for node 6940846c575bc060ce15d76a INFO: 172.21.0.1:43828 - "POST /node/6940846c575bc060ce15d76a HTTP/1.0" 202 Accepted 2025-12-17 21:58:05,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e8575bc060ce15d549 2025-12-17 21:58:05,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e8575bc060ce15d549 HTTP/1.1" 502 150 2025-12-17 21:58:05,653 - lava_callback - ERROR - Error processing callback for node 694083e8575bc060ce15d549: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e8575bc060ce15d549 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e8575bc060ce15d549 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e8575bc060ce15d549 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:05,655 - lava_callback - INFO - Starting async job submit for node 694083f9575bc060ce15d575 2025-12-17 21:58:05,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:05,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f9575bc060ce15d575 HTTP/1.1" 502 150 2025-12-17 21:58:05,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f9575bc060ce15d575'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:05,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f9575bc060ce15d575 2025-12-17 21:58:05,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f9575bc060ce15d575 HTTP/1.1" 502 150 2025-12-17 21:58:05,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f9575bc060ce15d575'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:06,409 - lava_callback - INFO - Received callback for node 6940846d575bc060ce15d772 INFO: 172.21.0.1:43840 - "POST /node/6940846d575bc060ce15d772 HTTP/1.0" 202 Accepted 2025-12-17 21:58:06,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f7575bc060ce15d56f INFO: 172.21.0.1:43850 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:58:06,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408400575bc060ce15d584 2025-12-17 21:58:06,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408400575bc060ce15d584 HTTP/1.1" 502 150 2025-12-17 21:58:06,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408400575bc060ce15d584'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:06,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e9575bc060ce15d54f 2025-12-17 21:58:06,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e9575bc060ce15d54f HTTP/1.1" 502 150 2025-12-17 21:58:06,963 - lava_callback - ERROR - Error processing callback for node 694083e9575bc060ce15d54f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e9575bc060ce15d54f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e9575bc060ce15d54f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e9575bc060ce15d54f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:06,966 - lava_callback - INFO - Starting async job submit for node 694083fb575bc060ce15d578 2025-12-17 21:58:06,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:06,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fb575bc060ce15d578 HTTP/1.1" 502 150 2025-12-17 21:58:06,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fb575bc060ce15d578'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:06,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fb575bc060ce15d578 2025-12-17 21:58:06,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fb575bc060ce15d578 HTTP/1.1" 502 150 2025-12-17 21:58:06,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fb575bc060ce15d578'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:07,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f9575bc060ce15d575 2025-12-17 21:58:07,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f9575bc060ce15d575 HTTP/1.1" 502 150 2025-12-17 21:58:07,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f9575bc060ce15d575'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:07,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f8575bc060ce15d572 2025-12-17 21:58:07,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f8575bc060ce15d572 HTTP/1.1" 502 150 2025-12-17 21:58:07,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f8575bc060ce15d572'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:07,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ec575bc060ce15d555 2025-12-17 21:58:07,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ec575bc060ce15d555 HTTP/1.1" 502 150 2025-12-17 21:58:07,940 - lava_callback - ERROR - Error processing callback for node 694083ec575bc060ce15d555: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ec575bc060ce15d555 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ec575bc060ce15d555 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ec575bc060ce15d555 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:07,942 - lava_callback - INFO - Starting async job submit for node 69408403575bc060ce15d5a4 2025-12-17 21:58:07,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:07,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408403575bc060ce15d5a4 HTTP/1.1" 502 150 2025-12-17 21:58:07,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408403575bc060ce15d5a4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:07,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408403575bc060ce15d5a4 2025-12-17 21:58:07,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408403575bc060ce15d5a4 HTTP/1.1" 502 150 2025-12-17 21:58:07,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408403575bc060ce15d5a4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:08,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e5575bc060ce15d539 2025-12-17 21:58:08,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e5575bc060ce15d539 HTTP/1.1" 502 150 2025-12-17 21:58:08,471 - lava_callback - ERROR - Error processing callback for node 694083e5575bc060ce15d539: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e5575bc060ce15d539 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e5575bc060ce15d539 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e5575bc060ce15d539 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:08,473 - lava_callback - INFO - Starting async job submit for node 69408404575bc060ce15d5c5 2025-12-17 21:58:08,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:08,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408404575bc060ce15d5c5 HTTP/1.1" 502 150 2025-12-17 21:58:08,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408404575bc060ce15d5c5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:08,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408404575bc060ce15d5c5 2025-12-17 21:58:08,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408404575bc060ce15d5c5 HTTP/1.1" 502 150 2025-12-17 21:58:08,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408404575bc060ce15d5c5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:08,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ee575bc060ce15d55b 2025-12-17 21:58:08,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ee575bc060ce15d55b HTTP/1.1" 502 150 2025-12-17 21:58:08,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083ee575bc060ce15d55b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:08,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408401575bc060ce15d587 2025-12-17 21:58:08,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408401575bc060ce15d587 HTTP/1.1" 502 150 2025-12-17 21:58:08,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408401575bc060ce15d587'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:08,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fb575bc060ce15d578 2025-12-17 21:58:08,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fb575bc060ce15d578 HTTP/1.1" 502 150 2025-12-17 21:58:08,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fb575bc060ce15d578'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:09,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408403575bc060ce15d5a4 2025-12-17 21:58:09,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408403575bc060ce15d5a4 HTTP/1.1" 502 150 2025-12-17 21:58:09,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408403575bc060ce15d5a4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:10,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408404575bc060ce15d5c5 2025-12-17 21:58:10,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408404575bc060ce15d5c5 HTTP/1.1" 502 150 2025-12-17 21:58:10,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408404575bc060ce15d5c5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:11,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083e1575bc060ce15d52e 2025-12-17 21:58:11,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083e1575bc060ce15d52e HTTP/1.1" 502 150 2025-12-17 21:58:11,506 - lava_callback - ERROR - Error processing callback for node 694083e1575bc060ce15d52e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e1575bc060ce15d52e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e1575bc060ce15d52e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083e1575bc060ce15d52e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:11,508 - lava_callback - INFO - Starting async job submit for node 69408406575bc060ce15d5ca 2025-12-17 21:58:11,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:11,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408406575bc060ce15d5ca HTTP/1.1" 502 150 2025-12-17 21:58:11,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408406575bc060ce15d5ca'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:11,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408406575bc060ce15d5ca 2025-12-17 21:58:11,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408406575bc060ce15d5ca HTTP/1.1" 502 150 2025-12-17 21:58:11,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408406575bc060ce15d5ca'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:11,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f9575bc060ce15d575 2025-12-17 21:58:11,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f9575bc060ce15d575 HTTP/1.1" 502 150 2025-12-17 21:58:11,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f9575bc060ce15d575'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:11,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f4575bc060ce15d569 2025-12-17 21:58:11,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f4575bc060ce15d569 HTTP/1.1" 502 150 2025-12-17 21:58:11,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f4575bc060ce15d569'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:12,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f6575bc060ce15d56c 2025-12-17 21:58:12,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f3575bc060ce15d566 2025-12-17 21:58:12,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f6575bc060ce15d56c HTTP/1.1" 502 150 2025-12-17 21:58:12,944 - lava_callback - ERROR - Error processing callback for node 694083f6575bc060ce15d56c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f6575bc060ce15d56c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f6575bc060ce15d56c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f6575bc060ce15d56c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:12,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f3575bc060ce15d566 HTTP/1.1" 502 150 2025-12-17 21:58:12,947 - lava_callback - INFO - Starting async job submit for node 6940840b575bc060ce15d5ea 2025-12-17 21:58:12,948 - lava_callback - ERROR - Error processing callback for node 694083f3575bc060ce15d566: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f3575bc060ce15d566 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f3575bc060ce15d566 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f3575bc060ce15d566 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:12,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:12,951 - lava_callback - INFO - Starting async job submit for node 6940907e575bc060ce1605b3 2025-12-17 21:58:12,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fb575bc060ce15d578 2025-12-17 21:58:12,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fb575bc060ce15d578 HTTP/1.1" 502 150 2025-12-17 21:58:12,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fb575bc060ce15d578'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:13,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840b575bc060ce15d5ea HTTP/1.1" 502 150 2025-12-17 21:58:13,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840b575bc060ce15d5ea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:13,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840b575bc060ce15d5ea 2025-12-17 21:58:13,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840b575bc060ce15d5ea HTTP/1.1" 502 150 2025-12-17 21:58:13,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840b575bc060ce15d5ea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:13,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:13,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907e575bc060ce1605b3 HTTP/1.1" 502 150 2025-12-17 21:58:13,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940907e575bc060ce1605b3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:13,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940907e575bc060ce1605b3 2025-12-17 21:58:13,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907e575bc060ce1605b3 HTTP/1.1" 502 150 2025-12-17 21:58:13,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940907e575bc060ce1605b3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:13,504 - lava_callback - INFO - Received callback for node 69408466575bc060ce15d758 INFO: 172.21.0.1:43502 - "POST /node/69408466575bc060ce15d758 HTTP/1.0" 202 Accepted 2025-12-17 21:58:13,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408406575bc060ce15d5ca 2025-12-17 21:58:13,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408406575bc060ce15d5ca HTTP/1.1" 502 150 2025-12-17 21:58:13,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408406575bc060ce15d5ca'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:13,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fe575bc060ce15d581 HTTP/1.1" 502 150 2025-12-17 21:58:13,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fe575bc060ce15d581'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:13,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408403575bc060ce15d5a4 2025-12-17 21:58:13,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408403575bc060ce15d5a4 HTTP/1.1" 502 150 2025-12-17 21:58:13,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408403575bc060ce15d5a4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:14,163 - lava_callback - INFO - Received callback for node 69408468575bc060ce15d75b INFO: 172.21.0.1:43508 - "POST /node/69408468575bc060ce15d75b HTTP/1.0" 202 Accepted 2025-12-17 21:58:14,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408404575bc060ce15d5c5 2025-12-17 21:58:14,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408404575bc060ce15d5c5 HTTP/1.1" 502 150 2025-12-17 21:58:14,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408404575bc060ce15d5c5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:14,814 - lava_callback - INFO - Received callback for node 6940846a575bc060ce15d760 INFO: 172.21.0.1:43514 - "POST /node/6940846a575bc060ce15d760 HTTP/1.0" 202 Accepted 2025-12-17 21:58:15,028 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840b575bc060ce15d5ea 2025-12-17 21:58:15,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840b575bc060ce15d5ea HTTP/1.1" 502 150 2025-12-17 21:58:15,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840b575bc060ce15d5ea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:15,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940907e575bc060ce1605b3 2025-12-17 21:58:15,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907e575bc060ce1605b3 HTTP/1.1" 502 150 2025-12-17 21:58:15,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940907e575bc060ce1605b3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43526 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:58:15,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fc575bc060ce15d57b 2025-12-17 21:58:15,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fc575bc060ce15d57b HTTP/1.1" 502 150 2025-12-17 21:58:15,415 - lava_callback - ERROR - Error processing callback for node 694083fc575bc060ce15d57b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083fc575bc060ce15d57b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083fc575bc060ce15d57b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083fc575bc060ce15d57b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:15,417 - lava_callback - INFO - Starting async job submit for node 69408407575bc060ce15d5d3 2025-12-17 21:58:15,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:15,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408407575bc060ce15d5d3 HTTP/1.1" 502 150 2025-12-17 21:58:15,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408407575bc060ce15d5d3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:15,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408407575bc060ce15d5d3 2025-12-17 21:58:15,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408407575bc060ce15d5d3 HTTP/1.1" 502 150 2025-12-17 21:58:15,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408407575bc060ce15d5d3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:15,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f8575bc060ce15d572 2025-12-17 21:58:15,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f8575bc060ce15d572 HTTP/1.1" 502 150 2025-12-17 21:58:15,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f8575bc060ce15d572'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43538 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:58:17,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408407575bc060ce15d5d3 2025-12-17 21:58:17,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408407575bc060ce15d5d3 HTTP/1.1" 502 150 2025-12-17 21:58:17,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408407575bc060ce15d5d3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:17,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408406575bc060ce15d5ca 2025-12-17 21:58:17,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408406575bc060ce15d5ca HTTP/1.1" 502 150 2025-12-17 21:58:17,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408406575bc060ce15d5ca'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:18,597 - lava_callback - INFO - Received callback for node 6940846e575bc060ce15d778 INFO: 172.21.0.1:43552 - "POST /node/6940846e575bc060ce15d778 HTTP/1.0" 202 Accepted 2025-12-17 21:58:19,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840b575bc060ce15d5ea 2025-12-17 21:58:19,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940907e575bc060ce1605b3 2025-12-17 21:58:19,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907e575bc060ce1605b3 HTTP/1.1" 502 150 2025-12-17 21:58:19,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940907e575bc060ce1605b3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:19,332 - lava_callback - INFO - Received callback for node 6940846f575bc060ce15d77b INFO: 172.21.0.1:43556 - "POST /node/6940846f575bc060ce15d77b HTTP/1.0" 202 Accepted 2025-12-17 21:58:19,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fd575bc060ce15d57e 2025-12-17 21:58:19,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fd575bc060ce15d57e HTTP/1.1" 502 150 2025-12-17 21:58:19,576 - lava_callback - ERROR - Error processing callback for node 694083fd575bc060ce15d57e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083fd575bc060ce15d57e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083fd575bc060ce15d57e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083fd575bc060ce15d57e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:19,578 - lava_callback - INFO - Starting async job submit for node 69408409575bc060ce15d5e2 2025-12-17 21:58:19,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:19,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408409575bc060ce15d5e2 HTTP/1.1" 502 150 2025-12-17 21:58:19,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408409575bc060ce15d5e2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:19,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408409575bc060ce15d5e2 2025-12-17 21:58:19,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408409575bc060ce15d5e2 HTTP/1.1" 502 150 2025-12-17 21:58:19,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408409575bc060ce15d5e2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:19,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f9575bc060ce15d575 2025-12-17 21:58:19,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f9575bc060ce15d575 HTTP/1.1" 502 150 2025-12-17 21:58:19,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f9575bc060ce15d575'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:20,073 - lava_callback - INFO - Received callback for node 69408471575bc060ce15d77e INFO: 172.21.0.1:34792 - "POST /node/69408471575bc060ce15d77e HTTP/1.0" 202 Accepted 2025-12-17 21:58:20,911 - lava_callback - INFO - Received callback for node 69408472575bc060ce15d781 INFO: 172.21.0.1:34806 - "POST /node/69408472575bc060ce15d781 HTTP/1.0" 202 Accepted 2025-12-17 21:58:20,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fb575bc060ce15d578 2025-12-17 21:58:20,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fb575bc060ce15d578 HTTP/1.1" 502 150 2025-12-17 21:58:20,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fb575bc060ce15d578'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:21,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408407575bc060ce15d5d3 2025-12-17 21:58:21,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408407575bc060ce15d5d3 HTTP/1.1" 502 150 2025-12-17 21:58:21,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408407575bc060ce15d5d3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:21,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408409575bc060ce15d5e2 2025-12-17 21:58:21,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408409575bc060ce15d5e2 HTTP/1.1" 502 150 2025-12-17 21:58:21,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408409575bc060ce15d5e2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:21,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fe575bc060ce15d581 2025-12-17 21:58:21,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fe575bc060ce15d581 HTTP/1.1" 502 150 2025-12-17 21:58:21,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083fe575bc060ce15d581'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:21,746 - lava_callback - INFO - Received callback for node 69408473575bc060ce15d784 INFO: 172.21.0.1:34808 - "POST /node/69408473575bc060ce15d784 HTTP/1.0" 202 Accepted 2025-12-17 21:58:21,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f7575bc060ce15d56f HTTP/1.1" 502 150 2025-12-17 21:58:21,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f7575bc060ce15d56f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:21,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408403575bc060ce15d5a4 2025-12-17 21:58:21,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408403575bc060ce15d5a4 HTTP/1.1" 502 150 2025-12-17 21:58:21,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408403575bc060ce15d5a4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:22,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408404575bc060ce15d5c5 2025-12-17 21:58:22,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408404575bc060ce15d5c5 HTTP/1.1" 502 150 2025-12-17 21:58:22,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408404575bc060ce15d5c5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:22,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408400575bc060ce15d584 2025-12-17 21:58:22,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408400575bc060ce15d584 HTTP/1.1" 502 150 2025-12-17 21:58:22,637 - lava_callback - ERROR - Error processing callback for node 69408400575bc060ce15d584: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408400575bc060ce15d584 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408400575bc060ce15d584 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408400575bc060ce15d584 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:22,638 - lava_callback - INFO - Starting async job submit for node 6940840a575bc060ce15d5e7 2025-12-17 21:58:22,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:22,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840a575bc060ce15d5e7 HTTP/1.1" 502 150 2025-12-17 21:58:22,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840a575bc060ce15d5e7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:22,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840a575bc060ce15d5e7 2025-12-17 21:58:22,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840a575bc060ce15d5e7 HTTP/1.1" 502 150 2025-12-17 21:58:22,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840a575bc060ce15d5e7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:22,828 - lava_callback - INFO - Received callback for node 69408475575bc060ce15d787 INFO: 172.21.0.1:34818 - "POST /node/69408475575bc060ce15d787 HTTP/1.0" 202 Accepted 2025-12-17 21:58:23,609 - lava_callback - INFO - Received callback for node 69408476575bc060ce15d78a INFO: 172.21.0.1:34824 - "POST /node/69408476575bc060ce15d78a HTTP/1.0" 202 Accepted 2025-12-17 21:58:24,413 - lava_callback - INFO - Received callback for node 69408477575bc060ce15d78d INFO: 172.21.0.1:34830 - "POST /node/69408477575bc060ce15d78d HTTP/1.0" 202 Accepted 2025-12-17 21:58:24,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840a575bc060ce15d5e7 2025-12-17 21:58:24,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840a575bc060ce15d5e7 HTTP/1.1" 502 150 2025-12-17 21:58:24,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840a575bc060ce15d5e7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:24,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083ee575bc060ce15d55b 2025-12-17 21:58:24,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083ee575bc060ce15d55b HTTP/1.1" 502 150 2025-12-17 21:58:24,737 - lava_callback - ERROR - Error processing callback for node 694083ee575bc060ce15d55b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ee575bc060ce15d55b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ee575bc060ce15d55b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083ee575bc060ce15d55b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:24,740 - lava_callback - INFO - Starting async job submit for node 6940840d575bc060ce15d5ed 2025-12-17 21:58:24,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:24,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840d575bc060ce15d5ed HTTP/1.1" 502 150 2025-12-17 21:58:24,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840d575bc060ce15d5ed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:24,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840d575bc060ce15d5ed 2025-12-17 21:58:24,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840d575bc060ce15d5ed HTTP/1.1" 502 150 2025-12-17 21:58:24,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840d575bc060ce15d5ed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:24,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408401575bc060ce15d587 2025-12-17 21:58:24,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408401575bc060ce15d587 HTTP/1.1" 502 150 2025-12-17 21:58:24,889 - lava_callback - ERROR - Error processing callback for node 69408401575bc060ce15d587: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408401575bc060ce15d587 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408401575bc060ce15d587 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408401575bc060ce15d587 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:24,891 - lava_callback - INFO - Starting async job submit for node 6940840f575bc060ce15d5f2 2025-12-17 21:58:24,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:24,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840f575bc060ce15d5f2 HTTP/1.1" 502 150 2025-12-17 21:58:24,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840f575bc060ce15d5f2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:24,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840f575bc060ce15d5f2 2025-12-17 21:58:24,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840f575bc060ce15d5f2 HTTP/1.1" 502 150 2025-12-17 21:58:24,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840f575bc060ce15d5f2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:25,154 - lava_callback - INFO - Received callback for node 69408479575bc060ce15d790 INFO: 172.21.0.1:34832 - "POST /node/69408479575bc060ce15d790 HTTP/1.0" 202 Accepted 2025-12-17 21:58:25,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408406575bc060ce15d5ca 2025-12-17 21:58:25,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408406575bc060ce15d5ca HTTP/1.1" 502 150 2025-12-17 21:58:25,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408406575bc060ce15d5ca'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:25,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408409575bc060ce15d5e2 2025-12-17 21:58:25,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408409575bc060ce15d5e2 HTTP/1.1" 502 150 2025-12-17 21:58:25,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408409575bc060ce15d5e2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:25,969 - lava_callback - INFO - Received callback for node 6940847a575bc060ce15d793 INFO: 172.21.0.1:34840 - "POST /node/6940847a575bc060ce15d793 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:34856 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:58:26,706 - lava_callback - INFO - Received callback for node 6940847b575bc060ce15d796 INFO: 172.21.0.1:34870 - "POST /node/6940847b575bc060ce15d796 HTTP/1.0" 202 Accepted 2025-12-17 21:58:26,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840d575bc060ce15d5ed 2025-12-17 21:58:26,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840d575bc060ce15d5ed HTTP/1.1" 502 150 2025-12-17 21:58:26,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840d575bc060ce15d5ed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:26,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840f575bc060ce15d5f2 2025-12-17 21:58:26,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840f575bc060ce15d5f2 HTTP/1.1" 502 150 2025-12-17 21:58:26,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840f575bc060ce15d5f2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:27,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940907e575bc060ce1605b3 2025-12-17 21:58:27,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907e575bc060ce1605b3 HTTP/1.1" 502 150 2025-12-17 21:58:27,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940907e575bc060ce1605b3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:27,450 - lava_callback - INFO - Received callback for node 6940847c575bc060ce15d799 INFO: 172.21.0.1:34886 - "POST /node/6940847c575bc060ce15d799 HTTP/1.0" 202 Accepted 2025-12-17 21:58:27,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f4575bc060ce15d569 2025-12-17 21:58:27,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f4575bc060ce15d569 HTTP/1.1" 502 150 2025-12-17 21:58:27,984 - lava_callback - ERROR - Error processing callback for node 694083f4575bc060ce15d569: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f4575bc060ce15d569 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f4575bc060ce15d569 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f4575bc060ce15d569 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:27,985 - lava_callback - INFO - Starting async job submit for node 69408410575bc060ce15d5f5 2025-12-17 21:58:27,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:28,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408410575bc060ce15d5f5 HTTP/1.1" 502 150 2025-12-17 21:58:28,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408410575bc060ce15d5f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:28,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408410575bc060ce15d5f5 2025-12-17 21:58:28,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408410575bc060ce15d5f5 HTTP/1.1" 502 150 2025-12-17 21:58:28,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408410575bc060ce15d5f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:28,203 - lava_callback - INFO - Received callback for node 6940847e575bc060ce15d79d INFO: 172.21.0.1:34888 - "POST /node/6940847e575bc060ce15d79d HTTP/1.0" 202 Accepted 2025-12-17 21:58:28,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840a575bc060ce15d5e7 2025-12-17 21:58:28,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840a575bc060ce15d5e7 HTTP/1.1" 502 150 2025-12-17 21:58:28,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840a575bc060ce15d5e7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:28,955 - lava_callback - INFO - Received callback for node 69408480575bc060ce15d7c4 INFO: 172.21.0.1:34892 - "POST /node/69408480575bc060ce15d7c4 HTTP/1.0" 202 Accepted 2025-12-17 21:58:29,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408407575bc060ce15d5d3 2025-12-17 21:58:29,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408407575bc060ce15d5d3 HTTP/1.1" 502 150 2025-12-17 21:58:29,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408407575bc060ce15d5d3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:29,708 - lava_callback - INFO - Received callback for node 69408482575bc060ce15d7e2 INFO: 172.21.0.1:45566 - "POST /node/69408482575bc060ce15d7e2 HTTP/1.0" 202 Accepted 2025-12-17 21:58:29,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f7575bc060ce15d56f 2025-12-17 21:58:29,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f7575bc060ce15d56f HTTP/1.1" 502 150 2025-12-17 21:58:29,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694083f7575bc060ce15d56f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:30,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408410575bc060ce15d5f5 2025-12-17 21:58:30,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408410575bc060ce15d5f5 HTTP/1.1" 502 150 2025-12-17 21:58:30,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408410575bc060ce15d5f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:30,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840d575bc060ce15d5ed 2025-12-17 21:58:30,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840d575bc060ce15d5ed HTTP/1.1" 502 150 2025-12-17 21:58:30,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840d575bc060ce15d5ed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:30,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840f575bc060ce15d5f2 2025-12-17 21:58:30,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840f575bc060ce15d5f2 HTTP/1.1" 502 150 2025-12-17 21:58:30,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840f575bc060ce15d5f2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:31,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f8575bc060ce15d572 2025-12-17 21:58:31,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f8575bc060ce15d572 HTTP/1.1" 502 150 2025-12-17 21:58:31,745 - lava_callback - ERROR - Error processing callback for node 694083f8575bc060ce15d572: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f8575bc060ce15d572 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f8575bc060ce15d572 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f8575bc060ce15d572 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:31,747 - lava_callback - INFO - Starting async job submit for node 69408411575bc060ce15d5f8 2025-12-17 21:58:31,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:31,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408411575bc060ce15d5f8 HTTP/1.1" 502 150 2025-12-17 21:58:31,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408411575bc060ce15d5f8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:31,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408411575bc060ce15d5f8 2025-12-17 21:58:31,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408411575bc060ce15d5f8 HTTP/1.1" 502 150 2025-12-17 21:58:31,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408411575bc060ce15d5f8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:33,511 - lava_callback - INFO - Received callback for node 69408482575bc060ce15d7e7 INFO: 172.21.0.1:45580 - "POST /node/69408482575bc060ce15d7e7 HTTP/1.0" 202 Accepted 2025-12-17 21:58:33,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408409575bc060ce15d5e2 2025-12-17 21:58:33,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408409575bc060ce15d5e2 HTTP/1.1" 502 150 2025-12-17 21:58:33,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408409575bc060ce15d5e2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:33,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408411575bc060ce15d5f8 2025-12-17 21:58:33,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408411575bc060ce15d5f8 HTTP/1.1" 502 150 2025-12-17 21:58:33,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408411575bc060ce15d5f8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:34,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408410575bc060ce15d5f5 2025-12-17 21:58:34,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408410575bc060ce15d5f5 HTTP/1.1" 502 150 2025-12-17 21:58:34,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408410575bc060ce15d5f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:34,158 - lava_callback - INFO - Received callback for node 69408485575bc060ce15d7f4 INFO: 172.21.0.1:45586 - "POST /node/69408485575bc060ce15d7f4 HTTP/1.0" 202 Accepted 2025-12-17 21:58:34,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840b575bc060ce15d5ea HTTP/1.1" 502 150 2025-12-17 21:58:34,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840b575bc060ce15d5ea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:34,837 - lava_callback - INFO - Received callback for node 69408487575bc060ce15d7fb INFO: 172.21.0.1:45596 - "POST /node/69408487575bc060ce15d7fb HTTP/1.0" 202 Accepted 2025-12-17 21:58:35,485 - lava_callback - INFO - Received callback for node 69408488575bc060ce15d802 INFO: 172.21.0.1:45612 - "POST /node/69408488575bc060ce15d802 HTTP/1.0" 202 Accepted 2025-12-17 21:58:35,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f9575bc060ce15d575 2025-12-17 21:58:35,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f9575bc060ce15d575 HTTP/1.1" 502 150 2025-12-17 21:58:35,669 - lava_callback - ERROR - Error processing callback for node 694083f9575bc060ce15d575: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f9575bc060ce15d575 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f9575bc060ce15d575 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f9575bc060ce15d575 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:35,671 - lava_callback - INFO - Starting async job submit for node 69408412575bc060ce15d5fb 2025-12-17 21:58:35,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:35,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408412575bc060ce15d5fb HTTP/1.1" 502 150 2025-12-17 21:58:35,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408412575bc060ce15d5fb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:35,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408412575bc060ce15d5fb 2025-12-17 21:58:35,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408412575bc060ce15d5fb HTTP/1.1" 502 150 2025-12-17 21:58:35,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408412575bc060ce15d5fb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45614 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:58:36,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840a575bc060ce15d5e7 2025-12-17 21:58:36,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840a575bc060ce15d5e7 HTTP/1.1" 502 150 2025-12-17 21:58:36,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840a575bc060ce15d5e7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:36,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fb575bc060ce15d578 2025-12-17 21:58:36,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fb575bc060ce15d578 HTTP/1.1" 502 150 2025-12-17 21:58:36,997 - lava_callback - ERROR - Error processing callback for node 694083fb575bc060ce15d578: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083fb575bc060ce15d578 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083fb575bc060ce15d578 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083fb575bc060ce15d578 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:36,999 - lava_callback - INFO - Starting async job submit for node 69408413575bc060ce15d5fe 2025-12-17 21:58:37,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:37,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408413575bc060ce15d5fe HTTP/1.1" 502 150 2025-12-17 21:58:37,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408413575bc060ce15d5fe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:37,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408413575bc060ce15d5fe 2025-12-17 21:58:37,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408413575bc060ce15d5fe HTTP/1.1" 502 150 2025-12-17 21:58:37,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408413575bc060ce15d5fe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:37,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408412575bc060ce15d5fb 2025-12-17 21:58:37,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408412575bc060ce15d5fb HTTP/1.1" 502 150 2025-12-17 21:58:37,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408412575bc060ce15d5fb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:37,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083fe575bc060ce15d581 2025-12-17 21:58:37,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083fe575bc060ce15d581 HTTP/1.1" 502 150 2025-12-17 21:58:37,683 - lava_callback - ERROR - Error processing callback for node 694083fe575bc060ce15d581: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083fe575bc060ce15d581 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083fe575bc060ce15d581 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083fe575bc060ce15d581 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:37,684 - lava_callback - INFO - Starting async job submit for node 69408415575bc060ce15d601 2025-12-17 21:58:37,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:37,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408415575bc060ce15d601 HTTP/1.1" 502 150 2025-12-17 21:58:37,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408415575bc060ce15d601'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:37,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408415575bc060ce15d601 2025-12-17 21:58:37,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408415575bc060ce15d601 HTTP/1.1" 502 150 2025-12-17 21:58:37,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408415575bc060ce15d601'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:37,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408411575bc060ce15d5f8 2025-12-17 21:58:37,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408411575bc060ce15d5f8 HTTP/1.1" 502 150 2025-12-17 21:58:37,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408411575bc060ce15d5f8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:37,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408403575bc060ce15d5a4 2025-12-17 21:58:37,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408403575bc060ce15d5a4 HTTP/1.1" 502 150 2025-12-17 21:58:37,956 - lava_callback - ERROR - Error processing callback for node 69408403575bc060ce15d5a4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408403575bc060ce15d5a4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408403575bc060ce15d5a4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408403575bc060ce15d5a4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:37,957 - lava_callback - INFO - Starting async job submit for node 69408416575bc060ce15d604 2025-12-17 21:58:37,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:37,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408416575bc060ce15d604 HTTP/1.1" 502 150 2025-12-17 21:58:37,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408416575bc060ce15d604'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:37,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408416575bc060ce15d604 2025-12-17 21:58:37,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408416575bc060ce15d604 HTTP/1.1" 502 150 2025-12-17 21:58:37,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408416575bc060ce15d604'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:38,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408404575bc060ce15d5c5 2025-12-17 21:58:38,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408404575bc060ce15d5c5 HTTP/1.1" 502 150 2025-12-17 21:58:38,489 - lava_callback - ERROR - Error processing callback for node 69408404575bc060ce15d5c5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408404575bc060ce15d5c5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408404575bc060ce15d5c5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408404575bc060ce15d5c5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:38,490 - lava_callback - INFO - Starting async job submit for node 6940841a575bc060ce15d611 2025-12-17 21:58:38,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:38,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841a575bc060ce15d611 HTTP/1.1" 502 150 2025-12-17 21:58:38,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841a575bc060ce15d611'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:38,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841a575bc060ce15d611 2025-12-17 21:58:38,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841a575bc060ce15d611 HTTP/1.1" 502 150 2025-12-17 21:58:38,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841a575bc060ce15d611'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:38,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840d575bc060ce15d5ed 2025-12-17 21:58:38,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840d575bc060ce15d5ed HTTP/1.1" 502 150 2025-12-17 21:58:38,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840d575bc060ce15d5ed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:38,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840f575bc060ce15d5f2 2025-12-17 21:58:38,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840f575bc060ce15d5f2 HTTP/1.1" 502 150 2025-12-17 21:58:38,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840f575bc060ce15d5f2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:39,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408413575bc060ce15d5fe 2025-12-17 21:58:39,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408413575bc060ce15d5fe HTTP/1.1" 502 150 2025-12-17 21:58:39,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408413575bc060ce15d5fe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:39,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408415575bc060ce15d601 2025-12-17 21:58:39,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408415575bc060ce15d601 HTTP/1.1" 502 150 2025-12-17 21:58:39,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408415575bc060ce15d601'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:39,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408416575bc060ce15d604 2025-12-17 21:58:39,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408416575bc060ce15d604 HTTP/1.1" 502 150 2025-12-17 21:58:39,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408416575bc060ce15d604'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:40,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841a575bc060ce15d611 2025-12-17 21:58:40,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841a575bc060ce15d611 HTTP/1.1" 502 150 2025-12-17 21:58:40,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841a575bc060ce15d611'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:41,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408406575bc060ce15d5ca 2025-12-17 21:58:41,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408406575bc060ce15d5ca HTTP/1.1" 502 150 2025-12-17 21:58:41,542 - lava_callback - ERROR - Error processing callback for node 69408406575bc060ce15d5ca: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408406575bc060ce15d5ca (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408406575bc060ce15d5ca (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408406575bc060ce15d5ca (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:41,544 - lava_callback - INFO - Starting async job submit for node 6940841c575bc060ce15d61e 2025-12-17 21:58:41,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:41,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841c575bc060ce15d61e HTTP/1.1" 502 150 2025-12-17 21:58:41,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841c575bc060ce15d61e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:41,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841c575bc060ce15d61e 2025-12-17 21:58:41,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841c575bc060ce15d61e HTTP/1.1" 502 150 2025-12-17 21:58:41,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841c575bc060ce15d61e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:41,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408412575bc060ce15d5fb 2025-12-17 21:58:41,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408412575bc060ce15d5fb HTTP/1.1" 502 150 2025-12-17 21:58:41,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408412575bc060ce15d5fb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:42,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408410575bc060ce15d5f5 2025-12-17 21:58:42,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408410575bc060ce15d5f5 HTTP/1.1" 502 150 2025-12-17 21:58:42,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408410575bc060ce15d5f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:42,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840b575bc060ce15d5ea 2025-12-17 21:58:42,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840b575bc060ce15d5ea HTTP/1.1" 502 150 2025-12-17 21:58:42,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940840b575bc060ce15d5ea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:43,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408413575bc060ce15d5fe 2025-12-17 21:58:43,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408413575bc060ce15d5fe HTTP/1.1" 502 150 2025-12-17 21:58:43,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408413575bc060ce15d5fe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:43,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940907e575bc060ce1605b3 2025-12-17 21:58:43,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907e575bc060ce1605b3 HTTP/1.1" 502 150 2025-12-17 21:58:43,049 - lava_callback - ERROR - Error processing callback for node 6940907e575bc060ce1605b3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940907e575bc060ce1605b3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940907e575bc060ce1605b3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940907e575bc060ce1605b3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:43,051 - lava_callback - INFO - Starting async job submit for node 69408417575bc060ce15d607 2025-12-17 21:58:43,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:43,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408417575bc060ce15d607 HTTP/1.1" 502 150 2025-12-17 21:58:43,069 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408417575bc060ce15d607'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:43,069 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408417575bc060ce15d607 2025-12-17 21:58:43,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408417575bc060ce15d607 HTTP/1.1" 502 150 2025-12-17 21:58:43,070 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408417575bc060ce15d607'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:43,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841c575bc060ce15d61e 2025-12-17 21:58:43,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841c575bc060ce15d61e HTTP/1.1" 502 150 2025-12-17 21:58:43,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841c575bc060ce15d61e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:43,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408415575bc060ce15d601 2025-12-17 21:58:43,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408415575bc060ce15d601 HTTP/1.1" 502 150 2025-12-17 21:58:43,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408415575bc060ce15d601'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:43,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408416575bc060ce15d604 2025-12-17 21:58:43,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408416575bc060ce15d604 HTTP/1.1" 502 150 2025-12-17 21:58:43,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408416575bc060ce15d604'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:44,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841a575bc060ce15d611 2025-12-17 21:58:44,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841a575bc060ce15d611 HTTP/1.1" 502 150 2025-12-17 21:58:44,501 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841a575bc060ce15d611'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:45,070 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408417575bc060ce15d607 2025-12-17 21:58:45,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408407575bc060ce15d5d3 2025-12-17 21:58:45,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408407575bc060ce15d5d3 HTTP/1.1" 502 150 2025-12-17 21:58:45,436 - lava_callback - ERROR - Error processing callback for node 69408407575bc060ce15d5d3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408407575bc060ce15d5d3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408407575bc060ce15d5d3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408407575bc060ce15d5d3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:45,438 - lava_callback - INFO - Starting async job submit for node 69408419575bc060ce15d60a 2025-12-17 21:58:45,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:45,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408419575bc060ce15d60a HTTP/1.1" 502 150 2025-12-17 21:58:45,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408419575bc060ce15d60a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:45,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408419575bc060ce15d60a 2025-12-17 21:58:45,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408419575bc060ce15d60a HTTP/1.1" 502 150 2025-12-17 21:58:45,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408419575bc060ce15d60a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:45,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408411575bc060ce15d5f8 2025-12-17 21:58:45,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408411575bc060ce15d5f8 HTTP/1.1" 502 150 2025-12-17 21:58:45,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408411575bc060ce15d5f8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:45,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694083f7575bc060ce15d56f 2025-12-17 21:58:45,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694083f7575bc060ce15d56f HTTP/1.1" 502 150 2025-12-17 21:58:45,875 - lava_callback - ERROR - Error processing callback for node 694083f7575bc060ce15d56f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f7575bc060ce15d56f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f7575bc060ce15d56f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694083f7575bc060ce15d56f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:45,877 - lava_callback - INFO - Starting async job submit for node 6940841d575bc060ce15d623 2025-12-17 21:58:45,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:45,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841d575bc060ce15d623 HTTP/1.1" 502 150 2025-12-17 21:58:45,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841d575bc060ce15d623'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:45,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841d575bc060ce15d623 2025-12-17 21:58:45,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841d575bc060ce15d623 HTTP/1.1" 502 150 2025-12-17 21:58:45,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841d575bc060ce15d623'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58616 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:58:47,447 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408419575bc060ce15d60a 2025-12-17 21:58:47,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408419575bc060ce15d60a HTTP/1.1" 502 150 2025-12-17 21:58:47,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408419575bc060ce15d60a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:47,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841c575bc060ce15d61e 2025-12-17 21:58:47,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841c575bc060ce15d61e HTTP/1.1" 502 150 2025-12-17 21:58:47,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841c575bc060ce15d61e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:47,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841d575bc060ce15d623 2025-12-17 21:58:47,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841d575bc060ce15d623 HTTP/1.1" 502 150 2025-12-17 21:58:47,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841d575bc060ce15d623'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:49,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408409575bc060ce15d5e2 2025-12-17 21:58:49,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408409575bc060ce15d5e2 HTTP/1.1" 502 150 2025-12-17 21:58:49,613 - lava_callback - ERROR - Error processing callback for node 69408409575bc060ce15d5e2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408409575bc060ce15d5e2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408409575bc060ce15d5e2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408409575bc060ce15d5e2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:49,617 - lava_callback - INFO - Starting async job submit for node 6940841f575bc060ce15d626 2025-12-17 21:58:49,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:49,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841f575bc060ce15d626 HTTP/1.1" 502 150 2025-12-17 21:58:49,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841f575bc060ce15d626'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:49,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841f575bc060ce15d626 2025-12-17 21:58:49,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841f575bc060ce15d626 HTTP/1.1" 502 150 2025-12-17 21:58:49,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841f575bc060ce15d626'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:49,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408412575bc060ce15d5fb 2025-12-17 21:58:49,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408412575bc060ce15d5fb HTTP/1.1" 502 150 2025-12-17 21:58:49,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408412575bc060ce15d5fb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:51,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408413575bc060ce15d5fe 2025-12-17 21:58:51,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408413575bc060ce15d5fe HTTP/1.1" 502 150 2025-12-17 21:58:51,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408413575bc060ce15d5fe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:51,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408419575bc060ce15d60a 2025-12-17 21:58:51,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408419575bc060ce15d60a HTTP/1.1" 502 150 2025-12-17 21:58:51,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408419575bc060ce15d60a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:51,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841f575bc060ce15d626 2025-12-17 21:58:51,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841f575bc060ce15d626 HTTP/1.1" 502 150 2025-12-17 21:58:51,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841f575bc060ce15d626'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:51,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408415575bc060ce15d601 2025-12-17 21:58:51,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408415575bc060ce15d601 HTTP/1.1" 502 150 2025-12-17 21:58:51,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408415575bc060ce15d601'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:51,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841d575bc060ce15d623 2025-12-17 21:58:51,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841d575bc060ce15d623 HTTP/1.1" 502 150 2025-12-17 21:58:51,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841d575bc060ce15d623'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:51,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408416575bc060ce15d604 2025-12-17 21:58:51,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408416575bc060ce15d604 HTTP/1.1" 502 150 2025-12-17 21:58:51,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408416575bc060ce15d604'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:52,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841a575bc060ce15d611 2025-12-17 21:58:52,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841a575bc060ce15d611 HTTP/1.1" 502 150 2025-12-17 21:58:52,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841a575bc060ce15d611'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:52,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840a575bc060ce15d5e7 2025-12-17 21:58:52,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840a575bc060ce15d5e7 HTTP/1.1" 502 150 2025-12-17 21:58:52,651 - lava_callback - ERROR - Error processing callback for node 6940840a575bc060ce15d5e7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940840a575bc060ce15d5e7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940840a575bc060ce15d5e7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940840a575bc060ce15d5e7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:52,653 - lava_callback - INFO - Starting async job submit for node 69408420575bc060ce15d629 2025-12-17 21:58:52,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:52,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408420575bc060ce15d629 HTTP/1.1" 502 150 2025-12-17 21:58:52,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408420575bc060ce15d629'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:52,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408420575bc060ce15d629 2025-12-17 21:58:52,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408420575bc060ce15d629 HTTP/1.1" 502 150 2025-12-17 21:58:52,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408420575bc060ce15d629'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:53,580 - lava_callback - INFO - Received callback for node 69408489575bc060ce15d805 INFO: 172.21.0.1:34740 - "POST /node/69408489575bc060ce15d805 HTTP/1.0" 202 Accepted 2025-12-17 21:58:54,247 - lava_callback - INFO - Received callback for node 6940848b575bc060ce15d808 INFO: 172.21.0.1:34752 - "POST /node/6940848b575bc060ce15d808 HTTP/1.0" 202 Accepted 2025-12-17 21:58:54,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408420575bc060ce15d629 2025-12-17 21:58:54,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408420575bc060ce15d629 HTTP/1.1" 502 150 2025-12-17 21:58:54,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408420575bc060ce15d629'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:54,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840d575bc060ce15d5ed 2025-12-17 21:58:54,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840d575bc060ce15d5ed HTTP/1.1" 502 150 2025-12-17 21:58:54,764 - lava_callback - ERROR - Error processing callback for node 6940840d575bc060ce15d5ed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940840d575bc060ce15d5ed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940840d575bc060ce15d5ed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940840d575bc060ce15d5ed (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:54,766 - lava_callback - INFO - Starting async job submit for node 69408421575bc060ce15d62c 2025-12-17 21:58:54,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:54,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408421575bc060ce15d62c HTTP/1.1" 502 150 2025-12-17 21:58:54,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408421575bc060ce15d62c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:54,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408421575bc060ce15d62c 2025-12-17 21:58:54,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408421575bc060ce15d62c HTTP/1.1" 502 150 2025-12-17 21:58:54,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408421575bc060ce15d62c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:54,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840f575bc060ce15d5f2 2025-12-17 21:58:54,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840f575bc060ce15d5f2 HTTP/1.1" 502 150 2025-12-17 21:58:54,911 - lava_callback - ERROR - Error processing callback for node 6940840f575bc060ce15d5f2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940840f575bc060ce15d5f2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940840f575bc060ce15d5f2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940840f575bc060ce15d5f2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:54,914 - lava_callback - INFO - Starting async job submit for node 69408423575bc060ce15d62f 2025-12-17 21:58:54,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:54,916 - lava_callback - INFO - Received callback for node 6940848c575bc060ce15d80b INFO: 172.21.0.1:34754 - "POST /node/6940848c575bc060ce15d80b HTTP/1.0" 202 Accepted 2025-12-17 21:58:54,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408423575bc060ce15d62f HTTP/1.1" 502 150 2025-12-17 21:58:54,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408423575bc060ce15d62f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:54,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408423575bc060ce15d62f 2025-12-17 21:58:54,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408423575bc060ce15d62f HTTP/1.1" 502 150 2025-12-17 21:58:54,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408423575bc060ce15d62f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:55,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841c575bc060ce15d61e 2025-12-17 21:58:55,560 - lava_callback - INFO - Received callback for node 6940848d575bc060ce15d80e 2025-12-17 21:58:55,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841c575bc060ce15d61e HTTP/1.1" 502 150 INFO: 172.21.0.1:34768 - "POST /node/6940848d575bc060ce15d80e HTTP/1.0" 202 Accepted 2025-12-17 21:58:55,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841c575bc060ce15d61e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:55,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841f575bc060ce15d626 2025-12-17 21:58:55,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841f575bc060ce15d626 HTTP/1.1" 502 150 2025-12-17 21:58:55,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841f575bc060ce15d626'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:56,255 - lava_callback - INFO - Received callback for node 6940848e575bc060ce15d811 INFO: 172.21.0.1:34782 - "POST /node/6940848e575bc060ce15d811 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:34784 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:58:56,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408421575bc060ce15d62c 2025-12-17 21:58:56,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408421575bc060ce15d62c HTTP/1.1" 502 150 2025-12-17 21:58:56,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408421575bc060ce15d62c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:56,904 - lava_callback - INFO - Received callback for node 6940848f575bc060ce15d814 INFO: 172.21.0.1:34790 - "POST /node/6940848f575bc060ce15d814 HTTP/1.0" 202 Accepted 2025-12-17 21:58:56,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408423575bc060ce15d62f 2025-12-17 21:58:56,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408423575bc060ce15d62f HTTP/1.1" 502 150 2025-12-17 21:58:56,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408423575bc060ce15d62f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:57,559 - lava_callback - INFO - Received callback for node 69408491575bc060ce15d817 INFO: 172.21.0.1:34802 - "POST /node/69408491575bc060ce15d817 HTTP/1.0" 202 Accepted 2025-12-17 21:58:58,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408410575bc060ce15d5f5 2025-12-17 21:58:58,239 - lava_callback - INFO - Received callback for node 69408495575bc060ce15d82a INFO: 172.21.0.1:34804 - "POST /node/69408495575bc060ce15d82a HTTP/1.0" 202 Accepted 2025-12-17 21:58:58,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940840b575bc060ce15d5ea 2025-12-17 21:58:58,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940840b575bc060ce15d5ea HTTP/1.1" 502 150 2025-12-17 21:58:58,419 - lava_callback - ERROR - Error processing callback for node 6940840b575bc060ce15d5ea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940840b575bc060ce15d5ea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940840b575bc060ce15d5ea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940840b575bc060ce15d5ea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:58:58,421 - lava_callback - INFO - Starting async job submit for node 69408424575bc060ce15d632 2025-12-17 21:58:58,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:58:58,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408424575bc060ce15d632 HTTP/1.1" 502 150 2025-12-17 21:58:58,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408424575bc060ce15d632'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:58,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408424575bc060ce15d632 2025-12-17 21:58:58,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408424575bc060ce15d632 HTTP/1.1" 502 150 2025-12-17 21:58:58,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408424575bc060ce15d632'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:58,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408420575bc060ce15d629 2025-12-17 21:58:58,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408420575bc060ce15d629 HTTP/1.1" 502 150 2025-12-17 21:58:58,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408420575bc060ce15d629'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:58,914 - lava_callback - INFO - Received callback for node 69408497575bc060ce15d831 INFO: 172.21.0.1:34820 - "POST /node/69408497575bc060ce15d831 HTTP/1.0" 202 Accepted 2025-12-17 21:58:59,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408419575bc060ce15d60a 2025-12-17 21:58:59,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408419575bc060ce15d60a HTTP/1.1" 502 150 2025-12-17 21:58:59,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408419575bc060ce15d60a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:58:59,552 - lava_callback - INFO - Received callback for node 69408498575bc060ce15d834 INFO: 172.21.0.1:34830 - "POST /node/69408498575bc060ce15d834 HTTP/1.0" 202 Accepted 2025-12-17 21:58:59,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841d575bc060ce15d623 2025-12-17 21:58:59,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841d575bc060ce15d623 HTTP/1.1" 502 150 2025-12-17 21:58:59,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841d575bc060ce15d623'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:00,236 - lava_callback - INFO - Received callback for node 69408499575bc060ce15d837 INFO: 172.21.0.1:58834 - "POST /node/69408499575bc060ce15d837 HTTP/1.0" 202 Accepted 2025-12-17 21:59:00,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408424575bc060ce15d632 2025-12-17 21:59:00,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408424575bc060ce15d632 HTTP/1.1" 502 150 2025-12-17 21:59:00,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408424575bc060ce15d632'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:00,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408417575bc060ce15d607 HTTP/1.1" 502 150 2025-12-17 21:59:00,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408417575bc060ce15d607'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:00,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408421575bc060ce15d62c 2025-12-17 21:59:00,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408421575bc060ce15d62c HTTP/1.1" 502 150 2025-12-17 21:59:00,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408421575bc060ce15d62c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:00,892 - lava_callback - INFO - Received callback for node 6940849a575bc060ce15d83a INFO: 172.21.0.1:58838 - "POST /node/6940849a575bc060ce15d83a HTTP/1.0" 202 Accepted 2025-12-17 21:59:00,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408423575bc060ce15d62f 2025-12-17 21:59:00,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408423575bc060ce15d62f HTTP/1.1" 502 150 2025-12-17 21:59:00,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408423575bc060ce15d62f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:01,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408411575bc060ce15d5f8 2025-12-17 21:59:01,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408411575bc060ce15d5f8 HTTP/1.1" 502 150 2025-12-17 21:59:01,762 - lava_callback - ERROR - Error processing callback for node 69408411575bc060ce15d5f8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408411575bc060ce15d5f8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408411575bc060ce15d5f8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408411575bc060ce15d5f8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:01,764 - lava_callback - INFO - Starting async job submit for node 69408426575bc060ce15d638 2025-12-17 21:59:01,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:01,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408426575bc060ce15d638 HTTP/1.1" 502 150 2025-12-17 21:59:01,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408426575bc060ce15d638'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:01,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408426575bc060ce15d638 2025-12-17 21:59:01,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408426575bc060ce15d638 HTTP/1.1" 502 150 2025-12-17 21:59:01,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408426575bc060ce15d638'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58848 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:59:02,118 - lava_callback - INFO - Received callback for node 69408269575bc060ce15cf6c INFO: 172.21.0.1:58850 - "POST /node/69408269575bc060ce15cf6c HTTP/1.0" 202 Accepted 2025-12-17 21:59:03,321 - lava_callback - INFO - Received callback for node 69408492575bc060ce15d81a INFO: 172.21.0.1:58862 - "POST /node/69408492575bc060ce15d81a HTTP/1.0" 202 Accepted 2025-12-17 21:59:03,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841f575bc060ce15d626 2025-12-17 21:59:03,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841f575bc060ce15d626 HTTP/1.1" 502 150 2025-12-17 21:59:03,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940841f575bc060ce15d626'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:03,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408426575bc060ce15d638 2025-12-17 21:59:03,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408426575bc060ce15d638 HTTP/1.1" 502 150 2025-12-17 21:59:03,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408426575bc060ce15d638'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:03,960 - lava_callback - INFO - Received callback for node 69408494575bc060ce15d823 INFO: 172.21.0.1:58868 - "POST /node/69408494575bc060ce15d823 HTTP/1.0" 202 Accepted 2025-12-17 21:59:04,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408424575bc060ce15d632 2025-12-17 21:59:04,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408424575bc060ce15d632 HTTP/1.1" 502 150 2025-12-17 21:59:04,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408424575bc060ce15d632'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:04,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408417575bc060ce15d607 2025-12-17 21:59:04,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408417575bc060ce15d607 HTTP/1.1" 502 150 2025-12-17 21:59:04,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408417575bc060ce15d607'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:04,623 - lava_callback - INFO - Received callback for node 694084a2575bc060ce15d843 INFO: 172.21.0.1:58870 - "POST /node/694084a2575bc060ce15d843 HTTP/1.0" 202 Accepted 2025-12-17 21:59:05,264 - lava_callback - INFO - Received callback for node 694084a4575bc060ce15d846 INFO: 172.21.0.1:58872 - "POST /node/694084a4575bc060ce15d846 HTTP/1.0" 202 Accepted 2025-12-17 21:59:05,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408412575bc060ce15d5fb 2025-12-17 21:59:05,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408412575bc060ce15d5fb HTTP/1.1" 502 150 2025-12-17 21:59:05,683 - lava_callback - ERROR - Error processing callback for node 69408412575bc060ce15d5fb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408412575bc060ce15d5fb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408412575bc060ce15d5fb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408412575bc060ce15d5fb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:05,686 - lava_callback - INFO - Starting async job submit for node 69408429575bc060ce15d63e 2025-12-17 21:59:05,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:05,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408429575bc060ce15d63e HTTP/1.1" 502 150 2025-12-17 21:59:05,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408429575bc060ce15d63e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:05,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408429575bc060ce15d63e 2025-12-17 21:59:05,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408429575bc060ce15d63e HTTP/1.1" 502 150 2025-12-17 21:59:05,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408429575bc060ce15d63e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58874 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:59:06,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408420575bc060ce15d629 2025-12-17 21:59:06,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408420575bc060ce15d629 HTTP/1.1" 502 150 2025-12-17 21:59:06,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408420575bc060ce15d629'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:07,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408413575bc060ce15d5fe 2025-12-17 21:59:07,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408413575bc060ce15d5fe HTTP/1.1" 502 150 2025-12-17 21:59:07,012 - lava_callback - ERROR - Error processing callback for node 69408413575bc060ce15d5fe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408413575bc060ce15d5fe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408413575bc060ce15d5fe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408413575bc060ce15d5fe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:07,014 - lava_callback - INFO - Starting async job submit for node 6940842b575bc060ce15d644 2025-12-17 21:59:07,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:07,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842b575bc060ce15d644 HTTP/1.1" 502 150 2025-12-17 21:59:07,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842b575bc060ce15d644'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:07,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842b575bc060ce15d644 2025-12-17 21:59:07,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842b575bc060ce15d644 HTTP/1.1" 502 150 2025-12-17 21:59:07,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842b575bc060ce15d644'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:07,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408415575bc060ce15d601 2025-12-17 21:59:07,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408415575bc060ce15d601 HTTP/1.1" 502 150 2025-12-17 21:59:07,698 - lava_callback - ERROR - Error processing callback for node 69408415575bc060ce15d601: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408415575bc060ce15d601 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408415575bc060ce15d601 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408415575bc060ce15d601 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:07,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408429575bc060ce15d63e 2025-12-17 21:59:07,700 - lava_callback - INFO - Starting async job submit for node 6940842e575bc060ce15d64a 2025-12-17 21:59:07,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:07,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408429575bc060ce15d63e HTTP/1.1" 502 150 2025-12-17 21:59:07,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408429575bc060ce15d63e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:07,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842e575bc060ce15d64a HTTP/1.1" 502 150 2025-12-17 21:59:07,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842e575bc060ce15d64a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:07,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842e575bc060ce15d64a 2025-12-17 21:59:07,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842e575bc060ce15d64a HTTP/1.1" 502 150 2025-12-17 21:59:07,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842e575bc060ce15d64a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:07,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408426575bc060ce15d638 2025-12-17 21:59:07,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408426575bc060ce15d638 HTTP/1.1" 502 150 2025-12-17 21:59:07,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408426575bc060ce15d638'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:07,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408416575bc060ce15d604 2025-12-17 21:59:07,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408416575bc060ce15d604 HTTP/1.1" 502 150 2025-12-17 21:59:07,989 - lava_callback - ERROR - Error processing callback for node 69408416575bc060ce15d604: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408416575bc060ce15d604 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408416575bc060ce15d604 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408416575bc060ce15d604 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:07,991 - lava_callback - INFO - Starting async job submit for node 6940842f575bc060ce15d64f 2025-12-17 21:59:07,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:07,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842f575bc060ce15d64f HTTP/1.1" 502 150 2025-12-17 21:59:07,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842f575bc060ce15d64f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:07,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842f575bc060ce15d64f 2025-12-17 21:59:07,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842f575bc060ce15d64f HTTP/1.1" 502 150 2025-12-17 21:59:07,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842f575bc060ce15d64f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:08,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841a575bc060ce15d611 2025-12-17 21:59:08,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841a575bc060ce15d611 HTTP/1.1" 502 150 2025-12-17 21:59:08,505 - lava_callback - ERROR - Error processing callback for node 6940841a575bc060ce15d611: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940841a575bc060ce15d611 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940841a575bc060ce15d611 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940841a575bc060ce15d611 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:08,507 - lava_callback - INFO - Starting async job submit for node 69408425575bc060ce15d635 2025-12-17 21:59:08,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:08,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408425575bc060ce15d635 HTTP/1.1" 502 150 2025-12-17 21:59:08,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408425575bc060ce15d635'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:08,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408425575bc060ce15d635 2025-12-17 21:59:08,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408425575bc060ce15d635 HTTP/1.1" 502 150 2025-12-17 21:59:08,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408425575bc060ce15d635'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:08,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408421575bc060ce15d62c 2025-12-17 21:59:08,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408421575bc060ce15d62c HTTP/1.1" 502 150 2025-12-17 21:59:08,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408421575bc060ce15d62c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:08,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408423575bc060ce15d62f 2025-12-17 21:59:08,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408423575bc060ce15d62f HTTP/1.1" 502 150 2025-12-17 21:59:08,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408423575bc060ce15d62f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:09,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842b575bc060ce15d644 2025-12-17 21:59:09,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842b575bc060ce15d644 HTTP/1.1" 502 150 2025-12-17 21:59:09,047 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842b575bc060ce15d644'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:09,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842e575bc060ce15d64a 2025-12-17 21:59:09,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842e575bc060ce15d64a HTTP/1.1" 502 150 2025-12-17 21:59:09,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842e575bc060ce15d64a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:09,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842f575bc060ce15d64f 2025-12-17 21:59:10,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842f575bc060ce15d64f HTTP/1.1" 502 150 2025-12-17 21:59:10,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842f575bc060ce15d64f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:10,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408425575bc060ce15d635 2025-12-17 21:59:10,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408425575bc060ce15d635 HTTP/1.1" 502 150 2025-12-17 21:59:10,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408425575bc060ce15d635'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:11,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841c575bc060ce15d61e 2025-12-17 21:59:11,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841c575bc060ce15d61e HTTP/1.1" 502 150 2025-12-17 21:59:11,596 - lava_callback - ERROR - Error processing callback for node 6940841c575bc060ce15d61e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940841c575bc060ce15d61e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940841c575bc060ce15d61e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940841c575bc060ce15d61e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:11,597 - lava_callback - INFO - Starting async job submit for node 69408428575bc060ce15d63b 2025-12-17 21:59:11,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:11,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408428575bc060ce15d63b HTTP/1.1" 502 150 2025-12-17 21:59:11,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408428575bc060ce15d63b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:11,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408428575bc060ce15d63b 2025-12-17 21:59:11,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408428575bc060ce15d63b HTTP/1.1" 502 150 2025-12-17 21:59:11,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408428575bc060ce15d63b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:11,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408429575bc060ce15d63e 2025-12-17 21:59:11,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408429575bc060ce15d63e HTTP/1.1" 502 150 2025-12-17 21:59:11,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408429575bc060ce15d63e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:12,344 - lava_callback - INFO - Received callback for node 694084aa575bc060ce15d863 INFO: 172.21.0.1:38904 - "POST /node/694084aa575bc060ce15d863 HTTP/1.0" 202 Accepted 2025-12-17 21:59:12,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408424575bc060ce15d632 2025-12-17 21:59:12,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408424575bc060ce15d632 HTTP/1.1" 502 150 2025-12-17 21:59:12,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408424575bc060ce15d632'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:12,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408417575bc060ce15d607 2025-12-17 21:59:12,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408417575bc060ce15d607 HTTP/1.1" 502 150 2025-12-17 21:59:12,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408417575bc060ce15d607'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:13,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842b575bc060ce15d644 2025-12-17 21:59:13,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842b575bc060ce15d644 HTTP/1.1" 502 150 2025-12-17 21:59:13,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842b575bc060ce15d644'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:13,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408410575bc060ce15d5f5 HTTP/1.1" 502 150 2025-12-17 21:59:13,328 - lava_callback - ERROR - Error processing callback for node 69408410575bc060ce15d5f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408410575bc060ce15d5f5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408410575bc060ce15d5f5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408410575bc060ce15d5f5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:13,330 - lava_callback - INFO - Starting async job submit for node 6940842a575bc060ce15d641 2025-12-17 21:59:13,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:13,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842a575bc060ce15d641 HTTP/1.1" 502 150 2025-12-17 21:59:13,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842a575bc060ce15d641'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:13,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842a575bc060ce15d641 2025-12-17 21:59:13,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842a575bc060ce15d641 HTTP/1.1" 502 150 2025-12-17 21:59:13,338 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842a575bc060ce15d641'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:13,527 - lava_callback - INFO - Received callback for node 694084a0575bc060ce15d83d INFO: 172.21.0.1:38908 - "POST /node/694084a0575bc060ce15d83d HTTP/1.0" 202 Accepted 2025-12-17 21:59:13,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408428575bc060ce15d63b 2025-12-17 21:59:13,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408428575bc060ce15d63b HTTP/1.1" 502 150 2025-12-17 21:59:13,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408428575bc060ce15d63b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:13,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842e575bc060ce15d64a 2025-12-17 21:59:13,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842e575bc060ce15d64a HTTP/1.1" 502 150 2025-12-17 21:59:13,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842e575bc060ce15d64a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:14,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842f575bc060ce15d64f 2025-12-17 21:59:14,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842f575bc060ce15d64f HTTP/1.1" 502 150 2025-12-17 21:59:14,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842f575bc060ce15d64f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:14,199 - lava_callback - INFO - Received callback for node 694084a1575bc060ce15d840 INFO: 172.21.0.1:38916 - "POST /node/694084a1575bc060ce15d840 HTTP/1.0" 202 Accepted 2025-12-17 21:59:14,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408425575bc060ce15d635 2025-12-17 21:59:14,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408425575bc060ce15d635 HTTP/1.1" 502 150 2025-12-17 21:59:14,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408425575bc060ce15d635'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:14,874 - lava_callback - INFO - Received callback for node 694084a5575bc060ce15d849 INFO: 172.21.0.1:38918 - "POST /node/694084a5575bc060ce15d849 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:38930 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 21:59:15,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842a575bc060ce15d641 2025-12-17 21:59:15,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842a575bc060ce15d641 HTTP/1.1" 502 150 2025-12-17 21:59:15,339 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842a575bc060ce15d641'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:15,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408419575bc060ce15d60a 2025-12-17 21:59:15,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408419575bc060ce15d60a HTTP/1.1" 502 150 2025-12-17 21:59:15,453 - lava_callback - ERROR - Error processing callback for node 69408419575bc060ce15d60a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408419575bc060ce15d60a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408419575bc060ce15d60a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408419575bc060ce15d60a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:15,454 - lava_callback - INFO - Starting async job submit for node 6940842d575bc060ce15d647 2025-12-17 21:59:15,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:15,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842d575bc060ce15d647 HTTP/1.1" 502 150 2025-12-17 21:59:15,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842d575bc060ce15d647'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:15,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842d575bc060ce15d647 2025-12-17 21:59:15,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842d575bc060ce15d647 HTTP/1.1" 502 150 2025-12-17 21:59:15,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842d575bc060ce15d647'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:15,537 - lava_callback - INFO - Received callback for node 694084a6575bc060ce15d84e INFO: 172.21.0.1:38934 - "POST /node/694084a6575bc060ce15d84e HTTP/1.0" 202 Accepted 2025-12-17 21:59:15,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408426575bc060ce15d638 2025-12-17 21:59:15,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408426575bc060ce15d638 HTTP/1.1" 502 150 2025-12-17 21:59:15,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408426575bc060ce15d638'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:15,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841d575bc060ce15d623 2025-12-17 21:59:15,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841d575bc060ce15d623 HTTP/1.1" 502 150 2025-12-17 21:59:15,892 - lava_callback - ERROR - Error processing callback for node 6940841d575bc060ce15d623: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940841d575bc060ce15d623 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940841d575bc060ce15d623 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940841d575bc060ce15d623 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:15,894 - lava_callback - INFO - Starting async job submit for node 69408434575bc060ce15d66b 2025-12-17 21:59:15,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:15,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408434575bc060ce15d66b HTTP/1.1" 502 150 2025-12-17 21:59:15,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408434575bc060ce15d66b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:15,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408434575bc060ce15d66b 2025-12-17 21:59:15,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408434575bc060ce15d66b HTTP/1.1" 502 150 2025-12-17 21:59:15,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408434575bc060ce15d66b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:16,202 - lava_callback - INFO - Received callback for node 694084a7575bc060ce15d855 INFO: 172.21.0.1:38936 - "POST /node/694084a7575bc060ce15d855 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:38952 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:59:16,875 - lava_callback - INFO - Received callback for node 694084a9575bc060ce15d85c INFO: 172.21.0.1:38960 - "POST /node/694084a9575bc060ce15d85c HTTP/1.0" 202 Accepted 2025-12-17 21:59:17,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842d575bc060ce15d647 2025-12-17 21:59:17,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842d575bc060ce15d647 HTTP/1.1" 502 150 2025-12-17 21:59:17,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842d575bc060ce15d647'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:17,560 - lava_callback - INFO - Received callback for node 694084ac575bc060ce15d86a INFO: 172.21.0.1:38964 - "POST /node/694084ac575bc060ce15d86a HTTP/1.0" 202 Accepted 2025-12-17 21:59:17,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408428575bc060ce15d63b 2025-12-17 21:59:17,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408428575bc060ce15d63b HTTP/1.1" 502 150 2025-12-17 21:59:17,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408428575bc060ce15d63b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:17,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408434575bc060ce15d66b 2025-12-17 21:59:17,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408434575bc060ce15d66b HTTP/1.1" 502 150 2025-12-17 21:59:17,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408434575bc060ce15d66b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:19,035 - lava_callback - INFO - Received callback for node 694084af575bc060ce15d878 INFO: 172.21.0.1:38978 - "POST /node/694084af575bc060ce15d878 HTTP/1.0" 202 Accepted 2025-12-17 21:59:19,339 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842a575bc060ce15d641 2025-12-17 21:59:19,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842a575bc060ce15d641 HTTP/1.1" 502 150 2025-12-17 21:59:19,341 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842a575bc060ce15d641'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:19,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940841f575bc060ce15d626 2025-12-17 21:59:19,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940841f575bc060ce15d626 HTTP/1.1" 502 150 2025-12-17 21:59:19,634 - lava_callback - ERROR - Error processing callback for node 6940841f575bc060ce15d626: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940841f575bc060ce15d626 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940841f575bc060ce15d626 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940841f575bc060ce15d626 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:19,636 - lava_callback - INFO - Starting async job submit for node 69408437575bc060ce15d675 2025-12-17 21:59:19,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:19,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408437575bc060ce15d675 HTTP/1.1" 502 150 2025-12-17 21:59:19,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408437575bc060ce15d675'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:19,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408437575bc060ce15d675 2025-12-17 21:59:19,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408437575bc060ce15d675 HTTP/1.1" 502 150 2025-12-17 21:59:19,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408437575bc060ce15d675'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:19,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408429575bc060ce15d63e 2025-12-17 21:59:19,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408429575bc060ce15d63e HTTP/1.1" 502 150 2025-12-17 21:59:19,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408429575bc060ce15d63e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:19,823 - lava_callback - INFO - Received callback for node 694084b0575bc060ce15d87f INFO: 172.21.0.1:48572 - "POST /node/694084b0575bc060ce15d87f HTTP/1.0" 202 Accepted 2025-12-17 21:59:20,598 - lava_callback - INFO - Received callback for node 694084b4575bc060ce15d88f INFO: 172.21.0.1:48578 - "POST /node/694084b4575bc060ce15d88f HTTP/1.0" 202 Accepted 2025-12-17 21:59:21,049 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842b575bc060ce15d644 2025-12-17 21:59:21,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842b575bc060ce15d644 HTTP/1.1" 502 150 2025-12-17 21:59:21,051 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842b575bc060ce15d644'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:21,396 - lava_callback - INFO - Received callback for node 694084b5575bc060ce15d894 INFO: 172.21.0.1:48584 - "POST /node/694084b5575bc060ce15d894 HTTP/1.0" 202 Accepted 2025-12-17 21:59:21,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842d575bc060ce15d647 2025-12-17 21:59:21,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842d575bc060ce15d647 HTTP/1.1" 502 150 2025-12-17 21:59:21,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842d575bc060ce15d647'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:21,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408437575bc060ce15d675 2025-12-17 21:59:21,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408437575bc060ce15d675 HTTP/1.1" 502 150 2025-12-17 21:59:21,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408437575bc060ce15d675'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:21,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842e575bc060ce15d64a 2025-12-17 21:59:21,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842e575bc060ce15d64a HTTP/1.1" 502 150 2025-12-17 21:59:21,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842e575bc060ce15d64a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:21,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408434575bc060ce15d66b 2025-12-17 21:59:21,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408434575bc060ce15d66b HTTP/1.1" 502 150 2025-12-17 21:59:21,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408434575bc060ce15d66b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:22,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842f575bc060ce15d64f 2025-12-17 21:59:22,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842f575bc060ce15d64f HTTP/1.1" 502 150 2025-12-17 21:59:22,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842f575bc060ce15d64f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:22,230 - lava_callback - INFO - Received callback for node 694084b7575bc060ce15d897 INFO: 172.21.0.1:48590 - "POST /node/694084b7575bc060ce15d897 HTTP/1.0" 202 Accepted 2025-12-17 21:59:22,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408425575bc060ce15d635 2025-12-17 21:59:22,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408425575bc060ce15d635 HTTP/1.1" 502 150 2025-12-17 21:59:22,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408425575bc060ce15d635'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:22,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408420575bc060ce15d629 2025-12-17 21:59:22,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408420575bc060ce15d629 HTTP/1.1" 502 150 2025-12-17 21:59:22,669 - lava_callback - ERROR - Error processing callback for node 69408420575bc060ce15d629: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408420575bc060ce15d629 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408420575bc060ce15d629 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408420575bc060ce15d629 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:22,671 - lava_callback - INFO - Starting async job submit for node 69408430575bc060ce15d656 2025-12-17 21:59:22,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:22,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408430575bc060ce15d656 HTTP/1.1" 502 150 2025-12-17 21:59:22,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408430575bc060ce15d656'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:22,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408430575bc060ce15d656 2025-12-17 21:59:22,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408430575bc060ce15d656 HTTP/1.1" 502 150 2025-12-17 21:59:22,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408430575bc060ce15d656'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:23,012 - lava_callback - INFO - Received callback for node 694084b8575bc060ce15d89a INFO: 172.21.0.1:48602 - "POST /node/694084b8575bc060ce15d89a HTTP/1.0" 202 Accepted 2025-12-17 21:59:23,758 - lava_callback - INFO - Received callback for node 694084b9575bc060ce15d89d INFO: 172.21.0.1:48618 - "POST /node/694084b9575bc060ce15d89d HTTP/1.0" 202 Accepted 2025-12-17 21:59:24,595 - lava_callback - INFO - Received callback for node 694084bb575bc060ce15d8a0 INFO: 172.21.0.1:48628 - "POST /node/694084bb575bc060ce15d8a0 HTTP/1.0" 202 Accepted 2025-12-17 21:59:24,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408430575bc060ce15d656 2025-12-17 21:59:24,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408421575bc060ce15d62c 2025-12-17 21:59:24,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408421575bc060ce15d62c HTTP/1.1" 502 150 2025-12-17 21:59:24,806 - lava_callback - ERROR - Error processing callback for node 69408421575bc060ce15d62c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408421575bc060ce15d62c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408421575bc060ce15d62c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408421575bc060ce15d62c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:24,808 - lava_callback - INFO - Starting async job submit for node 69408432575bc060ce15d65d 2025-12-17 21:59:24,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:24,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408432575bc060ce15d65d HTTP/1.1" 502 150 2025-12-17 21:59:24,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408432575bc060ce15d65d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:24,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408432575bc060ce15d65d 2025-12-17 21:59:24,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408432575bc060ce15d65d HTTP/1.1" 502 150 2025-12-17 21:59:24,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408432575bc060ce15d65d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:24,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408423575bc060ce15d62f 2025-12-17 21:59:24,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408423575bc060ce15d62f HTTP/1.1" 502 150 2025-12-17 21:59:24,933 - lava_callback - ERROR - Error processing callback for node 69408423575bc060ce15d62f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408423575bc060ce15d62f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408423575bc060ce15d62f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408423575bc060ce15d62f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:24,935 - lava_callback - INFO - Starting async job submit for node 69408433575bc060ce15d664 2025-12-17 21:59:24,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:24,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408433575bc060ce15d664 HTTP/1.1" 502 150 2025-12-17 21:59:24,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408433575bc060ce15d664'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:24,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408433575bc060ce15d664 2025-12-17 21:59:24,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408433575bc060ce15d664 HTTP/1.1" 502 150 2025-12-17 21:59:24,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408433575bc060ce15d664'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:25,391 - lava_callback - INFO - Received callback for node 694084bc575bc060ce15d8a7 INFO: 172.21.0.1:48632 - "POST /node/694084bc575bc060ce15d8a7 HTTP/1.0" 202 Accepted 2025-12-17 21:59:25,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408428575bc060ce15d63b 2025-12-17 21:59:25,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408428575bc060ce15d63b HTTP/1.1" 502 150 2025-12-17 21:59:25,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408428575bc060ce15d63b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:25,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408437575bc060ce15d675 2025-12-17 21:59:25,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408437575bc060ce15d675 HTTP/1.1" 502 150 2025-12-17 21:59:25,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408437575bc060ce15d675'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48646 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:59:26,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408432575bc060ce15d65d 2025-12-17 21:59:26,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408432575bc060ce15d65d HTTP/1.1" 502 150 2025-12-17 21:59:26,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408432575bc060ce15d65d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:26,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408433575bc060ce15d664 2025-12-17 21:59:26,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408433575bc060ce15d664 HTTP/1.1" 502 150 2025-12-17 21:59:26,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408433575bc060ce15d664'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:27,341 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842a575bc060ce15d641 2025-12-17 21:59:28,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408424575bc060ce15d632 2025-12-17 21:59:28,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408424575bc060ce15d632 HTTP/1.1" 502 150 2025-12-17 21:59:28,435 - lava_callback - ERROR - Error processing callback for node 69408424575bc060ce15d632: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408424575bc060ce15d632 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408424575bc060ce15d632 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408424575bc060ce15d632 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:28,437 - lava_callback - INFO - Starting async job submit for node 69408436575bc060ce15d672 2025-12-17 21:59:28,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:28,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408436575bc060ce15d672 HTTP/1.1" 502 150 2025-12-17 21:59:28,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408436575bc060ce15d672'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:28,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408436575bc060ce15d672 2025-12-17 21:59:28,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408436575bc060ce15d672 HTTP/1.1" 502 150 2025-12-17 21:59:28,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408436575bc060ce15d672'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:28,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408417575bc060ce15d607 2025-12-17 21:59:28,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408417575bc060ce15d607 HTTP/1.1" 502 150 2025-12-17 21:59:28,532 - lava_callback - ERROR - Error processing callback for node 69408417575bc060ce15d607: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408417575bc060ce15d607 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408417575bc060ce15d607 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408417575bc060ce15d607 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:28,534 - lava_callback - INFO - Starting async job submit for node 69408439575bc060ce15d678 2025-12-17 21:59:28,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:28,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408439575bc060ce15d678 HTTP/1.1" 502 150 2025-12-17 21:59:28,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408439575bc060ce15d678'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:28,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408439575bc060ce15d678 2025-12-17 21:59:28,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408439575bc060ce15d678 HTTP/1.1" 502 150 2025-12-17 21:59:28,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408439575bc060ce15d678'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:29,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842d575bc060ce15d647 2025-12-17 21:59:29,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842d575bc060ce15d647 HTTP/1.1" 502 150 2025-12-17 21:59:29,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842d575bc060ce15d647'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:29,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408434575bc060ce15d66b 2025-12-17 21:59:29,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408434575bc060ce15d66b HTTP/1.1" 502 150 2025-12-17 21:59:29,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408434575bc060ce15d66b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:30,447 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408436575bc060ce15d672 2025-12-17 21:59:30,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408436575bc060ce15d672 HTTP/1.1" 502 150 2025-12-17 21:59:30,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408436575bc060ce15d672'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:30,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408439575bc060ce15d678 2025-12-17 21:59:30,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408439575bc060ce15d678 HTTP/1.1" 502 150 2025-12-17 21:59:30,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408439575bc060ce15d678'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:30,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408432575bc060ce15d65d 2025-12-17 21:59:30,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408432575bc060ce15d65d HTTP/1.1" 502 150 2025-12-17 21:59:30,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408432575bc060ce15d65d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:30,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408433575bc060ce15d664 2025-12-17 21:59:30,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408433575bc060ce15d664 HTTP/1.1" 502 150 2025-12-17 21:59:30,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408433575bc060ce15d664'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:31,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408426575bc060ce15d638 2025-12-17 21:59:31,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408426575bc060ce15d638 HTTP/1.1" 502 150 2025-12-17 21:59:31,778 - lava_callback - ERROR - Error processing callback for node 69408426575bc060ce15d638: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408426575bc060ce15d638 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408426575bc060ce15d638 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408426575bc060ce15d638 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:31,779 - lava_callback - INFO - Starting async job submit for node 6940843a575bc060ce15d67b 2025-12-17 21:59:31,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:31,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843a575bc060ce15d67b HTTP/1.1" 502 150 2025-12-17 21:59:31,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843a575bc060ce15d67b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:31,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843a575bc060ce15d67b 2025-12-17 21:59:31,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843a575bc060ce15d67b HTTP/1.1" 502 150 2025-12-17 21:59:31,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843a575bc060ce15d67b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:33,556 - lava_callback - INFO - Received callback for node 694084c2575bc060ce15d8f6 INFO: 172.21.0.1:54914 - "POST /node/694084c2575bc060ce15d8f6 HTTP/1.0" 202 Accepted 2025-12-17 21:59:33,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408437575bc060ce15d675 2025-12-17 21:59:33,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408437575bc060ce15d675 HTTP/1.1" 502 150 2025-12-17 21:59:33,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408437575bc060ce15d675'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:33,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843a575bc060ce15d67b 2025-12-17 21:59:33,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843a575bc060ce15d67b HTTP/1.1" 502 150 2025-12-17 21:59:33,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843a575bc060ce15d67b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:34,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408436575bc060ce15d672 2025-12-17 21:59:34,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408436575bc060ce15d672 HTTP/1.1" 502 150 2025-12-17 21:59:34,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408436575bc060ce15d672'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:34,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408439575bc060ce15d678 2025-12-17 21:59:34,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408439575bc060ce15d678 HTTP/1.1" 502 150 2025-12-17 21:59:34,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408439575bc060ce15d678'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:35,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408429575bc060ce15d63e 2025-12-17 21:59:35,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408429575bc060ce15d63e HTTP/1.1" 502 150 2025-12-17 21:59:35,711 - lava_callback - ERROR - Error processing callback for node 69408429575bc060ce15d63e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408429575bc060ce15d63e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408429575bc060ce15d63e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408429575bc060ce15d63e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:35,712 - lava_callback - INFO - Starting async job submit for node 6940843b575bc060ce15d67e 2025-12-17 21:59:35,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:35,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843b575bc060ce15d67e HTTP/1.1" 502 150 2025-12-17 21:59:35,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843b575bc060ce15d67e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:35,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843b575bc060ce15d67e 2025-12-17 21:59:35,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843b575bc060ce15d67e HTTP/1.1" 502 150 2025-12-17 21:59:35,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843b575bc060ce15d67e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54918 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:59:37,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842b575bc060ce15d644 2025-12-17 21:59:37,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842b575bc060ce15d644 HTTP/1.1" 502 150 2025-12-17 21:59:37,052 - lava_callback - ERROR - Error processing callback for node 6940842b575bc060ce15d644: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940842b575bc060ce15d644 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940842b575bc060ce15d644 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940842b575bc060ce15d644 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:37,055 - lava_callback - INFO - Starting async job submit for node 6940843c575bc060ce15d681 2025-12-17 21:59:37,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:37,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843c575bc060ce15d681 HTTP/1.1" 502 150 2025-12-17 21:59:37,066 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843c575bc060ce15d681'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:37,066 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843c575bc060ce15d681 2025-12-17 21:59:37,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843c575bc060ce15d681 HTTP/1.1" 502 150 2025-12-17 21:59:37,067 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843c575bc060ce15d681'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:37,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843b575bc060ce15d67e 2025-12-17 21:59:37,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843b575bc060ce15d67e HTTP/1.1" 502 150 2025-12-17 21:59:37,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843b575bc060ce15d67e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:37,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842e575bc060ce15d64a 2025-12-17 21:59:37,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842e575bc060ce15d64a HTTP/1.1" 502 150 2025-12-17 21:59:37,739 - lava_callback - ERROR - Error processing callback for node 6940842e575bc060ce15d64a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940842e575bc060ce15d64a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940842e575bc060ce15d64a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940842e575bc060ce15d64a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:37,741 - lava_callback - INFO - Starting async job submit for node 6940843e575bc060ce15d688 2025-12-17 21:59:37,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:37,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843e575bc060ce15d688 HTTP/1.1" 502 150 2025-12-17 21:59:37,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843e575bc060ce15d688'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:37,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843e575bc060ce15d688 2025-12-17 21:59:37,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843e575bc060ce15d688 HTTP/1.1" 502 150 2025-12-17 21:59:37,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843e575bc060ce15d688'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:37,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843a575bc060ce15d67b 2025-12-17 21:59:37,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843a575bc060ce15d67b HTTP/1.1" 502 150 2025-12-17 21:59:37,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843a575bc060ce15d67b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:38,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842f575bc060ce15d64f 2025-12-17 21:59:38,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408425575bc060ce15d635 2025-12-17 21:59:38,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408425575bc060ce15d635 HTTP/1.1" 502 150 2025-12-17 21:59:38,523 - lava_callback - ERROR - Error processing callback for node 69408425575bc060ce15d635: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408425575bc060ce15d635 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408425575bc060ce15d635 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408425575bc060ce15d635 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:38,525 - lava_callback - INFO - Starting async job submit for node 6940843f575bc060ce15d690 2025-12-17 21:59:38,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:38,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843f575bc060ce15d690 HTTP/1.1" 502 150 2025-12-17 21:59:38,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843f575bc060ce15d690'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:38,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843f575bc060ce15d690 2025-12-17 21:59:38,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843f575bc060ce15d690 HTTP/1.1" 502 150 2025-12-17 21:59:38,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843f575bc060ce15d690'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:38,717 - lava_callback - INFO - Received callback for node 694084be575bc060ce15d8df INFO: 172.21.0.1:54930 - "POST /node/694084be575bc060ce15d8df HTTP/1.0" 202 Accepted 2025-12-17 21:59:38,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408432575bc060ce15d65d 2025-12-17 21:59:38,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408432575bc060ce15d65d HTTP/1.1" 502 150 2025-12-17 21:59:38,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408432575bc060ce15d65d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:38,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408433575bc060ce15d664 2025-12-17 21:59:38,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408433575bc060ce15d664 HTTP/1.1" 502 150 2025-12-17 21:59:38,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408433575bc060ce15d664'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:39,067 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843c575bc060ce15d681 2025-12-17 21:59:39,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843c575bc060ce15d681 HTTP/1.1" 502 150 2025-12-17 21:59:39,068 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843c575bc060ce15d681'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:39,455 - lava_callback - INFO - Received callback for node 694084c0575bc060ce15d8ec INFO: 172.21.0.1:54946 - "POST /node/694084c0575bc060ce15d8ec HTTP/1.0" 202 Accepted 2025-12-17 21:59:39,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843e575bc060ce15d688 2025-12-17 21:59:39,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843e575bc060ce15d688 HTTP/1.1" 502 150 2025-12-17 21:59:39,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843e575bc060ce15d688'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:39,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408430575bc060ce15d656 HTTP/1.1" 502 150 2025-12-17 21:59:39,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408430575bc060ce15d656'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:40,341 - lava_callback - INFO - Received callback for node 694084c1575bc060ce15d8ef INFO: 172.21.0.1:60828 - "POST /node/694084c1575bc060ce15d8ef HTTP/1.0" 202 Accepted 2025-12-17 21:59:40,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843f575bc060ce15d690 2025-12-17 21:59:40,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843f575bc060ce15d690 HTTP/1.1" 502 150 2025-12-17 21:59:40,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843f575bc060ce15d690'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:41,083 - lava_callback - INFO - Received callback for node 694084c2575bc060ce15d8f2 INFO: 172.21.0.1:60842 - "POST /node/694084c2575bc060ce15d8f2 HTTP/1.0" 202 Accepted 2025-12-17 21:59:41,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408428575bc060ce15d63b 2025-12-17 21:59:41,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408428575bc060ce15d63b HTTP/1.1" 502 150 2025-12-17 21:59:41,617 - lava_callback - ERROR - Error processing callback for node 69408428575bc060ce15d63b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408428575bc060ce15d63b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408428575bc060ce15d63b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408428575bc060ce15d63b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:41,619 - lava_callback - INFO - Starting async job submit for node 69408442575bc060ce15d6c7 2025-12-17 21:59:41,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:41,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408442575bc060ce15d6c7 HTTP/1.1" 502 150 2025-12-17 21:59:41,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408442575bc060ce15d6c7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:41,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408442575bc060ce15d6c7 2025-12-17 21:59:41,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408442575bc060ce15d6c7 HTTP/1.1" 502 150 2025-12-17 21:59:41,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408442575bc060ce15d6c7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:41,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843b575bc060ce15d67e 2025-12-17 21:59:41,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843b575bc060ce15d67e HTTP/1.1" 502 150 2025-12-17 21:59:41,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843b575bc060ce15d67e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:41,878 - lava_callback - INFO - Received callback for node 694084cb575bc060ce15d91d INFO: 172.21.0.1:60846 - "POST /node/694084cb575bc060ce15d91d HTTP/1.0" 202 Accepted 2025-12-17 21:59:42,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408436575bc060ce15d672 2025-12-17 21:59:42,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408436575bc060ce15d672 HTTP/1.1" 502 150 2025-12-17 21:59:42,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408436575bc060ce15d672'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:42,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408439575bc060ce15d678 2025-12-17 21:59:42,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408439575bc060ce15d678 HTTP/1.1" 502 150 2025-12-17 21:59:42,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408439575bc060ce15d678'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:42,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842a575bc060ce15d641 HTTP/1.1" 502 150 2025-12-17 21:59:42,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940842a575bc060ce15d641'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:43,068 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843c575bc060ce15d681 2025-12-17 21:59:43,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843c575bc060ce15d681 HTTP/1.1" 502 150 2025-12-17 21:59:43,070 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843c575bc060ce15d681'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:43,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408442575bc060ce15d6c7 2025-12-17 21:59:43,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408442575bc060ce15d6c7 HTTP/1.1" 502 150 2025-12-17 21:59:43,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408442575bc060ce15d6c7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:43,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843e575bc060ce15d688 2025-12-17 21:59:43,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843e575bc060ce15d688 HTTP/1.1" 502 150 2025-12-17 21:59:43,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843e575bc060ce15d688'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:43,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408430575bc060ce15d656 2025-12-17 21:59:43,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408430575bc060ce15d656 HTTP/1.1" 502 150 2025-12-17 21:59:43,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408430575bc060ce15d656'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:44,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843f575bc060ce15d690 2025-12-17 21:59:44,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843f575bc060ce15d690 HTTP/1.1" 502 150 2025-12-17 21:59:44,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843f575bc060ce15d690'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:45,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842d575bc060ce15d647 2025-12-17 21:59:45,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842d575bc060ce15d647 HTTP/1.1" 502 150 2025-12-17 21:59:45,473 - lava_callback - ERROR - Error processing callback for node 6940842d575bc060ce15d647: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940842d575bc060ce15d647 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940842d575bc060ce15d647 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940842d575bc060ce15d647 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:45,474 - lava_callback - INFO - Starting async job submit for node 69408447575bc060ce15d6f8 2025-12-17 21:59:45,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:45,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408447575bc060ce15d6f8 HTTP/1.1" 502 150 2025-12-17 21:59:45,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408447575bc060ce15d6f8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:45,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408447575bc060ce15d6f8 2025-12-17 21:59:45,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408447575bc060ce15d6f8 HTTP/1.1" 502 150 2025-12-17 21:59:45,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408447575bc060ce15d6f8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:45,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843a575bc060ce15d67b 2025-12-17 21:59:45,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843a575bc060ce15d67b HTTP/1.1" 502 150 2025-12-17 21:59:45,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843a575bc060ce15d67b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:45,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408434575bc060ce15d66b 2025-12-17 21:59:45,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408434575bc060ce15d66b HTTP/1.1" 502 150 2025-12-17 21:59:45,912 - lava_callback - ERROR - Error processing callback for node 69408434575bc060ce15d66b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408434575bc060ce15d66b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408434575bc060ce15d66b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408434575bc060ce15d66b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:45,914 - lava_callback - INFO - Starting async job submit for node 69408449575bc060ce15d705 2025-12-17 21:59:45,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:45,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408449575bc060ce15d705 HTTP/1.1" 502 150 2025-12-17 21:59:45,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408449575bc060ce15d705'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:45,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408449575bc060ce15d705 2025-12-17 21:59:45,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408449575bc060ce15d705 HTTP/1.1" 502 150 2025-12-17 21:59:45,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408449575bc060ce15d705'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60858 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:59:47,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408447575bc060ce15d6f8 2025-12-17 21:59:47,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408447575bc060ce15d6f8 HTTP/1.1" 502 150 2025-12-17 21:59:47,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408447575bc060ce15d6f8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:47,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408442575bc060ce15d6c7 2025-12-17 21:59:47,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408442575bc060ce15d6c7 HTTP/1.1" 502 150 2025-12-17 21:59:47,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408442575bc060ce15d6c7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:47,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408449575bc060ce15d705 2025-12-17 21:59:47,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408449575bc060ce15d705 HTTP/1.1" 502 150 2025-12-17 21:59:47,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408449575bc060ce15d705'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:49,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408437575bc060ce15d675 2025-12-17 21:59:49,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408437575bc060ce15d675 HTTP/1.1" 502 150 2025-12-17 21:59:49,680 - lava_callback - ERROR - Error processing callback for node 69408437575bc060ce15d675: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408437575bc060ce15d675 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408437575bc060ce15d675 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408437575bc060ce15d675 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:49,682 - lava_callback - INFO - Starting async job submit for node 6940844a575bc060ce15d70a 2025-12-17 21:59:49,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:49,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844a575bc060ce15d70a HTTP/1.1" 502 150 2025-12-17 21:59:49,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844a575bc060ce15d70a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:49,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844a575bc060ce15d70a 2025-12-17 21:59:49,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844a575bc060ce15d70a HTTP/1.1" 502 150 2025-12-17 21:59:49,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844a575bc060ce15d70a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:49,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843b575bc060ce15d67e 2025-12-17 21:59:49,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843b575bc060ce15d67e HTTP/1.1" 502 150 2025-12-17 21:59:49,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843b575bc060ce15d67e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:51,070 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843c575bc060ce15d681 2025-12-17 21:59:51,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408447575bc060ce15d6f8 2025-12-17 21:59:51,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408447575bc060ce15d6f8 HTTP/1.1" 502 150 2025-12-17 21:59:51,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408447575bc060ce15d6f8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:51,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844a575bc060ce15d70a 2025-12-17 21:59:51,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844a575bc060ce15d70a HTTP/1.1" 502 150 2025-12-17 21:59:51,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844a575bc060ce15d70a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:51,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843e575bc060ce15d688 2025-12-17 21:59:51,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843e575bc060ce15d688 HTTP/1.1" 502 150 2025-12-17 21:59:51,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843e575bc060ce15d688'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:51,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408449575bc060ce15d705 2025-12-17 21:59:51,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408449575bc060ce15d705 HTTP/1.1" 502 150 2025-12-17 21:59:51,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408449575bc060ce15d705'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:51,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408430575bc060ce15d656 2025-12-17 21:59:51,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408430575bc060ce15d656 HTTP/1.1" 502 150 2025-12-17 21:59:51,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408430575bc060ce15d656'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:52,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843f575bc060ce15d690 2025-12-17 21:59:52,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843f575bc060ce15d690 HTTP/1.1" 502 150 2025-12-17 21:59:52,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843f575bc060ce15d690'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:53,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842f575bc060ce15d64f HTTP/1.1" 502 150 2025-12-17 21:59:53,264 - lava_callback - ERROR - Error processing callback for node 6940842f575bc060ce15d64f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940842f575bc060ce15d64f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940842f575bc060ce15d64f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940842f575bc060ce15d64f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:53,266 - lava_callback - INFO - Starting async job submit for node 6940844c575bc060ce15d70d 2025-12-17 21:59:53,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:53,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844c575bc060ce15d70d HTTP/1.1" 502 150 2025-12-17 21:59:53,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844c575bc060ce15d70d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:53,273 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844c575bc060ce15d70d 2025-12-17 21:59:53,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844c575bc060ce15d70d HTTP/1.1" 502 150 2025-12-17 21:59:53,273 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844c575bc060ce15d70d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:53,604 - lava_callback - INFO - Received callback for node 694084c4575bc060ce15d90e INFO: 172.21.0.1:58396 - "POST /node/694084c4575bc060ce15d90e HTTP/1.0" 202 Accepted 2025-12-17 21:59:54,262 - lava_callback - INFO - Received callback for node 694084c6575bc060ce15d911 INFO: 172.21.0.1:58402 - "POST /node/694084c6575bc060ce15d911 HTTP/1.0" 202 Accepted 2025-12-17 21:59:54,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408432575bc060ce15d65d 2025-12-17 21:59:54,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408432575bc060ce15d65d HTTP/1.1" 502 150 2025-12-17 21:59:54,821 - lava_callback - ERROR - Error processing callback for node 69408432575bc060ce15d65d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408432575bc060ce15d65d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408432575bc060ce15d65d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408432575bc060ce15d65d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:54,823 - lava_callback - INFO - Starting async job submit for node 69408215575bc060ce15cd1f 2025-12-17 21:59:54,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:54,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408215575bc060ce15cd1f HTTP/1.1" 502 150 2025-12-17 21:59:54,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408215575bc060ce15cd1f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:54,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408215575bc060ce15cd1f 2025-12-17 21:59:54,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408215575bc060ce15cd1f HTTP/1.1" 502 150 2025-12-17 21:59:54,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408215575bc060ce15cd1f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:54,917 - lava_callback - INFO - Received callback for node 694084c7575bc060ce15d914 INFO: 172.21.0.1:58414 - "POST /node/694084c7575bc060ce15d914 HTTP/1.0" 202 Accepted 2025-12-17 21:59:54,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408433575bc060ce15d664 2025-12-17 21:59:54,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408433575bc060ce15d664 HTTP/1.1" 502 150 2025-12-17 21:59:54,954 - lava_callback - ERROR - Error processing callback for node 69408433575bc060ce15d664: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408433575bc060ce15d664 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408433575bc060ce15d664 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408433575bc060ce15d664 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:54,956 - lava_callback - INFO - Starting async job submit for node 6940844e575bc060ce15d713 2025-12-17 21:59:54,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:54,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844e575bc060ce15d713 HTTP/1.1" 502 150 2025-12-17 21:59:54,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844e575bc060ce15d713'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:54,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844e575bc060ce15d713 2025-12-17 21:59:54,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844e575bc060ce15d713 HTTP/1.1" 502 150 2025-12-17 21:59:54,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844e575bc060ce15d713'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:55,274 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844c575bc060ce15d70d 2025-12-17 21:59:55,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844c575bc060ce15d70d HTTP/1.1" 502 150 2025-12-17 21:59:55,275 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844c575bc060ce15d70d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:55,565 - lava_callback - INFO - Received callback for node 694084c8575bc060ce15d917 INFO: 172.21.0.1:58424 - "POST /node/694084c8575bc060ce15d917 HTTP/1.0" 202 Accepted 2025-12-17 21:59:55,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408442575bc060ce15d6c7 2025-12-17 21:59:55,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408442575bc060ce15d6c7 HTTP/1.1" 502 150 2025-12-17 21:59:55,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408442575bc060ce15d6c7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:55,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844a575bc060ce15d70a 2025-12-17 21:59:55,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844a575bc060ce15d70a HTTP/1.1" 502 150 2025-12-17 21:59:55,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844a575bc060ce15d70a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:56,229 - lava_callback - INFO - Received callback for node 694084c9575bc060ce15d91a INFO: 172.21.0.1:58426 - "POST /node/694084c9575bc060ce15d91a HTTP/1.0" 202 Accepted INFO: 172.21.0.1:58432 - "GET / HTTP/1.0" 200 OK 2025-12-17 21:59:56,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408215575bc060ce15cd1f 2025-12-17 21:59:56,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408215575bc060ce15cd1f HTTP/1.1" 502 150 2025-12-17 21:59:56,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408215575bc060ce15cd1f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:56,876 - lava_callback - INFO - Received callback for node 694084cc575bc060ce15d920 INFO: 172.21.0.1:58440 - "POST /node/694084cc575bc060ce15d920 HTTP/1.0" 202 Accepted 2025-12-17 21:59:56,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844e575bc060ce15d713 2025-12-17 21:59:56,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844e575bc060ce15d713 HTTP/1.1" 502 150 2025-12-17 21:59:56,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844e575bc060ce15d713'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:58,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408436575bc060ce15d672 2025-12-17 21:59:58,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408436575bc060ce15d672 HTTP/1.1" 502 150 2025-12-17 21:59:58,453 - lava_callback - ERROR - Error processing callback for node 69408436575bc060ce15d672: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408436575bc060ce15d672 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408436575bc060ce15d672 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408436575bc060ce15d672 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:58,455 - lava_callback - INFO - Starting async job submit for node 69408456575bc060ce15d723 2025-12-17 21:59:58,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:58,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408456575bc060ce15d723 HTTP/1.1" 502 150 2025-12-17 21:59:58,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408456575bc060ce15d723'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:58,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408456575bc060ce15d723 2025-12-17 21:59:58,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408456575bc060ce15d723 HTTP/1.1" 502 150 2025-12-17 21:59:58,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408456575bc060ce15d723'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:58,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408439575bc060ce15d678 2025-12-17 21:59:58,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408439575bc060ce15d678 HTTP/1.1" 502 150 2025-12-17 21:59:58,558 - lava_callback - ERROR - Error processing callback for node 69408439575bc060ce15d678: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408439575bc060ce15d678 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408439575bc060ce15d678 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408439575bc060ce15d678 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:58,560 - lava_callback - INFO - Starting async job submit for node 6940844d575bc060ce15d710 2025-12-17 21:59:58,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:58,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844d575bc060ce15d710 HTTP/1.1" 502 150 2025-12-17 21:59:58,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844d575bc060ce15d710'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:58,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844d575bc060ce15d710 2025-12-17 21:59:58,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844d575bc060ce15d710 HTTP/1.1" 502 150 2025-12-17 21:59:58,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844d575bc060ce15d710'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:58,639 - lava_callback - INFO - Received callback for node 694084ce575bc060ce15d923 INFO: 172.21.0.1:58442 - "POST /node/694084ce575bc060ce15d923 HTTP/1.0" 202 Accepted 2025-12-17 21:59:58,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940842a575bc060ce15d641 2025-12-17 21:59:58,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940842a575bc060ce15d641 HTTP/1.1" 502 150 2025-12-17 21:59:58,773 - lava_callback - ERROR - Error processing callback for node 6940842a575bc060ce15d641: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940842a575bc060ce15d641 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940842a575bc060ce15d641 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940842a575bc060ce15d641 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 21:59:58,775 - lava_callback - INFO - Starting async job submit for node 6940845d575bc060ce15d740 2025-12-17 21:59:58,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 21:59:58,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845d575bc060ce15d740 HTTP/1.1" 502 150 2025-12-17 21:59:58,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845d575bc060ce15d740'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:58,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845d575bc060ce15d740 2025-12-17 21:59:58,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845d575bc060ce15d740 HTTP/1.1" 502 150 2025-12-17 21:59:58,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845d575bc060ce15d740'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:59,275 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844c575bc060ce15d70d 2025-12-17 21:59:59,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844c575bc060ce15d70d HTTP/1.1" 502 150 2025-12-17 21:59:59,276 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844c575bc060ce15d70d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:59,399 - lava_callback - INFO - Received callback for node 694084cf575bc060ce15d926 INFO: 172.21.0.1:58454 - "POST /node/694084cf575bc060ce15d926 HTTP/1.0" 202 Accepted 2025-12-17 21:59:59,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408447575bc060ce15d6f8 2025-12-17 21:59:59,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408447575bc060ce15d6f8 HTTP/1.1" 502 150 2025-12-17 21:59:59,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408447575bc060ce15d6f8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 21:59:59,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408449575bc060ce15d705 2025-12-17 21:59:59,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408449575bc060ce15d705 HTTP/1.1" 502 150 2025-12-17 21:59:59,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408449575bc060ce15d705'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:00,144 - lava_callback - INFO - Received callback for node 694084d0575bc060ce15d92d INFO: 172.21.0.1:42244 - "POST /node/694084d0575bc060ce15d92d HTTP/1.0" 202 Accepted 2025-12-17 22:00:00,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408456575bc060ce15d723 2025-12-17 22:00:00,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408456575bc060ce15d723 HTTP/1.1" 502 150 2025-12-17 22:00:00,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408456575bc060ce15d723'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:00,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844d575bc060ce15d710 2025-12-17 22:00:00,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844d575bc060ce15d710 HTTP/1.1" 502 150 2025-12-17 22:00:00,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844d575bc060ce15d710'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:00,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845d575bc060ce15d740 2025-12-17 22:00:00,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845d575bc060ce15d740 HTTP/1.1" 502 150 2025-12-17 22:00:00,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845d575bc060ce15d740'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:00,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408215575bc060ce15cd1f 2025-12-17 22:00:00,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408215575bc060ce15cd1f HTTP/1.1" 502 150 2025-12-17 22:00:00,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408215575bc060ce15cd1f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:00,913 - lava_callback - INFO - Received callback for node 694084d1575bc060ce15d934 INFO: 172.21.0.1:42252 - "POST /node/694084d1575bc060ce15d934 HTTP/1.0" 202 Accepted 2025-12-17 22:00:00,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844e575bc060ce15d713 2025-12-17 22:00:00,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844e575bc060ce15d713 HTTP/1.1" 502 150 2025-12-17 22:00:00,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844e575bc060ce15d713'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:01,708 - lava_callback - INFO - Received callback for node 694084d3575bc060ce15d93d INFO: 172.21.0.1:42262 - "POST /node/694084d3575bc060ce15d93d HTTP/1.0" 202 Accepted 2025-12-17 22:00:01,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843a575bc060ce15d67b 2025-12-17 22:00:01,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843a575bc060ce15d67b HTTP/1.1" 502 150 2025-12-17 22:00:01,799 - lava_callback - ERROR - Error processing callback for node 6940843a575bc060ce15d67b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940843a575bc060ce15d67b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940843a575bc060ce15d67b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940843a575bc060ce15d67b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:01,802 - lava_callback - INFO - Starting async job submit for node 6940845e575bc060ce15d743 2025-12-17 22:00:01,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:01,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845e575bc060ce15d743 HTTP/1.1" 502 150 2025-12-17 22:00:01,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845e575bc060ce15d743'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:01,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845e575bc060ce15d743 2025-12-17 22:00:01,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845e575bc060ce15d743 HTTP/1.1" 502 150 2025-12-17 22:00:01,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845e575bc060ce15d743'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42274 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:00:02,479 - lava_callback - INFO - Received callback for node 694084d4575bc060ce15d940 INFO: 172.21.0.1:42284 - "POST /node/694084d4575bc060ce15d940 HTTP/1.0" 202 Accepted 2025-12-17 22:00:03,228 - lava_callback - INFO - Received callback for node 694084d5575bc060ce15d943 INFO: 172.21.0.1:42300 - "POST /node/694084d5575bc060ce15d943 HTTP/1.0" 202 Accepted 2025-12-17 22:00:03,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844a575bc060ce15d70a 2025-12-17 22:00:03,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844a575bc060ce15d70a HTTP/1.1" 502 150 2025-12-17 22:00:03,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844a575bc060ce15d70a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:03,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845e575bc060ce15d743 2025-12-17 22:00:03,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845e575bc060ce15d743 HTTP/1.1" 502 150 2025-12-17 22:00:03,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845e575bc060ce15d743'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:03,988 - lava_callback - INFO - Received callback for node 694084d7575bc060ce15d94a INFO: 172.21.0.1:42304 - "POST /node/694084d7575bc060ce15d94a HTTP/1.0" 202 Accepted 2025-12-17 22:00:04,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408456575bc060ce15d723 2025-12-17 22:00:04,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408456575bc060ce15d723 HTTP/1.1" 502 150 2025-12-17 22:00:04,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408456575bc060ce15d723'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:04,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844d575bc060ce15d710 2025-12-17 22:00:04,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844d575bc060ce15d710 HTTP/1.1" 502 150 2025-12-17 22:00:04,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844d575bc060ce15d710'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:04,746 - lava_callback - INFO - Received callback for node 694084d8575bc060ce15d94d INFO: 172.21.0.1:42306 - "POST /node/694084d8575bc060ce15d94d HTTP/1.0" 202 Accepted 2025-12-17 22:00:04,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845d575bc060ce15d740 2025-12-17 22:00:04,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845d575bc060ce15d740 HTTP/1.1" 502 150 2025-12-17 22:00:04,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845d575bc060ce15d740'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:05,491 - lava_callback - INFO - Received callback for node 694084d9575bc060ce15d950 INFO: 172.21.0.1:42314 - "POST /node/694084d9575bc060ce15d950 HTTP/1.0" 202 Accepted 2025-12-17 22:00:05,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843b575bc060ce15d67e 2025-12-17 22:00:05,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843b575bc060ce15d67e HTTP/1.1" 502 150 2025-12-17 22:00:05,725 - lava_callback - ERROR - Error processing callback for node 6940843b575bc060ce15d67e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940843b575bc060ce15d67e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940843b575bc060ce15d67e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940843b575bc060ce15d67e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:05,726 - lava_callback - INFO - Starting async job submit for node 6940845f575bc060ce15d746 2025-12-17 22:00:05,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:05,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845f575bc060ce15d746 HTTP/1.1" 502 150 2025-12-17 22:00:05,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845f575bc060ce15d746'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:05,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845f575bc060ce15d746 2025-12-17 22:00:05,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845f575bc060ce15d746 HTTP/1.1" 502 150 2025-12-17 22:00:05,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845f575bc060ce15d746'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:06,250 - lava_callback - INFO - Received callback for node 694084db575bc060ce15d953 INFO: 172.21.0.1:42320 - "POST /node/694084db575bc060ce15d953 HTTP/1.0" 202 Accepted 2025-12-17 22:00:06,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843c575bc060ce15d681 HTTP/1.1" 502 150 2025-12-17 22:00:06,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940843c575bc060ce15d681'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42324 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:00:07,000 - lava_callback - INFO - Received callback for node 694084dc575bc060ce15d956 INFO: 172.21.0.1:42340 - "POST /node/694084dc575bc060ce15d956 HTTP/1.0" 202 Accepted 2025-12-17 22:00:07,276 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844c575bc060ce15d70d 2025-12-17 22:00:07,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844c575bc060ce15d70d HTTP/1.1" 502 150 2025-12-17 22:00:07,278 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844c575bc060ce15d70d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:07,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845f575bc060ce15d746 2025-12-17 22:00:07,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845f575bc060ce15d746 HTTP/1.1" 502 150 2025-12-17 22:00:07,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845f575bc060ce15d746'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:07,737 - lava_callback - INFO - Received callback for node 694084dd575bc060ce15d959 INFO: 172.21.0.1:42354 - "POST /node/694084dd575bc060ce15d959 HTTP/1.0" 202 Accepted 2025-12-17 22:00:07,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843e575bc060ce15d688 2025-12-17 22:00:07,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843e575bc060ce15d688 HTTP/1.1" 502 150 2025-12-17 22:00:07,759 - lava_callback - ERROR - Error processing callback for node 6940843e575bc060ce15d688: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940843e575bc060ce15d688 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940843e575bc060ce15d688 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940843e575bc060ce15d688 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:07,760 - lava_callback - INFO - Starting async job submit for node 69408460575bc060ce15d749 2025-12-17 22:00:07,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:07,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408460575bc060ce15d749 HTTP/1.1" 502 150 2025-12-17 22:00:07,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408460575bc060ce15d749'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:07,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408460575bc060ce15d749 2025-12-17 22:00:07,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408460575bc060ce15d749 HTTP/1.1" 502 150 2025-12-17 22:00:07,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408460575bc060ce15d749'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:07,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845e575bc060ce15d743 2025-12-17 22:00:07,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845e575bc060ce15d743 HTTP/1.1" 502 150 2025-12-17 22:00:07,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845e575bc060ce15d743'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:07,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408430575bc060ce15d656 2025-12-17 22:00:07,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408430575bc060ce15d656 HTTP/1.1" 502 150 2025-12-17 22:00:07,957 - lava_callback - ERROR - Error processing callback for node 69408430575bc060ce15d656: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408430575bc060ce15d656 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408430575bc060ce15d656 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408430575bc060ce15d656 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:07,958 - lava_callback - INFO - Starting async job submit for node 69408462575bc060ce15d74c 2025-12-17 22:00:07,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:07,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408462575bc060ce15d74c HTTP/1.1" 502 150 2025-12-17 22:00:07,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408462575bc060ce15d74c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:07,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408462575bc060ce15d74c 2025-12-17 22:00:07,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408462575bc060ce15d74c HTTP/1.1" 502 150 2025-12-17 22:00:07,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408462575bc060ce15d74c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:08,480 - lava_callback - INFO - Received callback for node 694084de575bc060ce15d95c INFO: 172.21.0.1:42364 - "POST /node/694084de575bc060ce15d95c HTTP/1.0" 202 Accepted 2025-12-17 22:00:08,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843f575bc060ce15d690 2025-12-17 22:00:08,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843f575bc060ce15d690 HTTP/1.1" 502 150 2025-12-17 22:00:08,561 - lava_callback - ERROR - Error processing callback for node 6940843f575bc060ce15d690: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940843f575bc060ce15d690 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940843f575bc060ce15d690 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940843f575bc060ce15d690 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:08,563 - lava_callback - INFO - Starting async job submit for node 69408463575bc060ce15d74f 2025-12-17 22:00:08,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:08,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408463575bc060ce15d74f HTTP/1.1" 502 150 2025-12-17 22:00:08,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408463575bc060ce15d74f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:08,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408463575bc060ce15d74f 2025-12-17 22:00:08,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408463575bc060ce15d74f HTTP/1.1" 502 150 2025-12-17 22:00:08,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408463575bc060ce15d74f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:08,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408215575bc060ce15cd1f 2025-12-17 22:00:08,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408215575bc060ce15cd1f HTTP/1.1" 502 150 2025-12-17 22:00:08,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408215575bc060ce15cd1f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:08,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844e575bc060ce15d713 2025-12-17 22:00:08,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844e575bc060ce15d713 HTTP/1.1" 502 150 2025-12-17 22:00:08,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844e575bc060ce15d713'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:09,221 - lava_callback - INFO - Received callback for node 694084e0575bc060ce15d95f INFO: 172.21.0.1:42380 - "POST /node/694084e0575bc060ce15d95f HTTP/1.0" 202 Accepted 2025-12-17 22:00:09,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408460575bc060ce15d749 2025-12-17 22:00:09,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408460575bc060ce15d749 HTTP/1.1" 502 150 2025-12-17 22:00:09,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408460575bc060ce15d749'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:09,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408462575bc060ce15d74c 2025-12-17 22:00:09,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408462575bc060ce15d74c HTTP/1.1" 502 150 2025-12-17 22:00:09,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408462575bc060ce15d74c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:09,987 - lava_callback - INFO - Received callback for node 694084e0575bc060ce15d962 INFO: 172.21.0.1:38590 - "POST /node/694084e0575bc060ce15d962 HTTP/1.0" 202 Accepted 2025-12-17 22:00:10,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408463575bc060ce15d74f 2025-12-17 22:00:10,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408463575bc060ce15d74f HTTP/1.1" 502 150 2025-12-17 22:00:10,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408463575bc060ce15d74f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:10,699 - lava_callback - INFO - Received callback for node 694084e0575bc060ce15d965 INFO: 172.21.0.1:38602 - "POST /node/694084e0575bc060ce15d965 HTTP/1.0" 202 Accepted 2025-12-17 22:00:11,417 - lava_callback - INFO - Received callback for node 694084e1575bc060ce15d968 INFO: 172.21.0.1:38616 - "POST /node/694084e1575bc060ce15d968 HTTP/1.0" 202 Accepted 2025-12-17 22:00:11,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408442575bc060ce15d6c7 2025-12-17 22:00:11,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408442575bc060ce15d6c7 HTTP/1.1" 502 150 2025-12-17 22:00:11,633 - lava_callback - ERROR - Error processing callback for node 69408442575bc060ce15d6c7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408442575bc060ce15d6c7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408442575bc060ce15d6c7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408442575bc060ce15d6c7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:11,634 - lava_callback - INFO - Starting async job submit for node 69408464575bc060ce15d752 2025-12-17 22:00:11,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:11,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408464575bc060ce15d752 HTTP/1.1" 502 150 2025-12-17 22:00:11,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408464575bc060ce15d752'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:11,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408464575bc060ce15d752 2025-12-17 22:00:11,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408464575bc060ce15d752 HTTP/1.1" 502 150 2025-12-17 22:00:11,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408464575bc060ce15d752'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:11,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845f575bc060ce15d746 2025-12-17 22:00:11,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845f575bc060ce15d746 HTTP/1.1" 502 150 2025-12-17 22:00:11,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845f575bc060ce15d746'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:12,255 - lava_callback - INFO - Received callback for node 694084e1575bc060ce15d96b INFO: 172.21.0.1:38630 - "POST /node/694084e1575bc060ce15d96b HTTP/1.0" 202 Accepted 2025-12-17 22:00:12,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408456575bc060ce15d723 2025-12-17 22:00:12,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408456575bc060ce15d723 HTTP/1.1" 502 150 2025-12-17 22:00:12,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408456575bc060ce15d723'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:12,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844d575bc060ce15d710 2025-12-17 22:00:12,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844d575bc060ce15d710 HTTP/1.1" 502 150 2025-12-17 22:00:12,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940844d575bc060ce15d710'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:12,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845d575bc060ce15d740 2025-12-17 22:00:12,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845d575bc060ce15d740 HTTP/1.1" 502 150 2025-12-17 22:00:12,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845d575bc060ce15d740'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:13,569 - lava_callback - INFO - Received callback for node 694084e2575bc060ce15d970 INFO: 172.21.0.1:38646 - "POST /node/694084e2575bc060ce15d970 HTTP/1.0" 202 Accepted 2025-12-17 22:00:13,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408464575bc060ce15d752 2025-12-17 22:00:13,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408464575bc060ce15d752 HTTP/1.1" 502 150 2025-12-17 22:00:13,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408464575bc060ce15d752'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:13,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408460575bc060ce15d749 2025-12-17 22:00:13,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408460575bc060ce15d749 HTTP/1.1" 502 150 2025-12-17 22:00:13,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408460575bc060ce15d749'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:13,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408462575bc060ce15d74c 2025-12-17 22:00:13,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408462575bc060ce15d74c HTTP/1.1" 502 150 2025-12-17 22:00:13,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408462575bc060ce15d74c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:14,234 - lava_callback - INFO - Received callback for node 694084e5575bc060ce15d97f INFO: 172.21.0.1:38648 - "POST /node/694084e5575bc060ce15d97f HTTP/1.0" 202 Accepted 2025-12-17 22:00:14,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408463575bc060ce15d74f 2025-12-17 22:00:14,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408463575bc060ce15d74f HTTP/1.1" 502 150 2025-12-17 22:00:14,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408463575bc060ce15d74f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:14,905 - lava_callback - INFO - Received callback for node 694084e6575bc060ce15d986 INFO: 172.21.0.1:38662 - "POST /node/694084e6575bc060ce15d986 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:38668 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:00:15,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408447575bc060ce15d6f8 2025-12-17 22:00:15,555 - lava_callback - INFO - Received callback for node 694084e7575bc060ce15d989 INFO: 172.21.0.1:38674 - "POST /node/694084e7575bc060ce15d989 HTTP/1.0" 202 Accepted 2025-12-17 22:00:15,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845e575bc060ce15d743 2025-12-17 22:00:15,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845e575bc060ce15d743 HTTP/1.1" 502 150 2025-12-17 22:00:15,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845e575bc060ce15d743'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:15,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408449575bc060ce15d705 2025-12-17 22:00:15,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408449575bc060ce15d705 HTTP/1.1" 502 150 2025-12-17 22:00:15,927 - lava_callback - ERROR - Error processing callback for node 69408449575bc060ce15d705: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408449575bc060ce15d705 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408449575bc060ce15d705 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408449575bc060ce15d705 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:15,929 - lava_callback - INFO - Starting async job submit for node 69408465575bc060ce15d755 2025-12-17 22:00:15,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:15,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408465575bc060ce15d755 HTTP/1.1" 502 150 2025-12-17 22:00:15,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408465575bc060ce15d755'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:15,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408465575bc060ce15d755 2025-12-17 22:00:15,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408465575bc060ce15d755 HTTP/1.1" 502 150 2025-12-17 22:00:15,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408465575bc060ce15d755'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:16,221 - lava_callback - INFO - Received callback for node 694084e9575bc060ce15d98c INFO: 172.21.0.1:38690 - "POST /node/694084e9575bc060ce15d98c HTTP/1.0" 202 Accepted INFO: 172.21.0.1:38702 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:00:16,882 - lava_callback - INFO - Received callback for node 694084ea575bc060ce15d98f INFO: 172.21.0.1:38716 - "POST /node/694084ea575bc060ce15d98f HTTP/1.0" 202 Accepted 2025-12-17 22:00:17,553 - lava_callback - INFO - Received callback for node 694084eb575bc060ce15d992 INFO: 172.21.0.1:38718 - "POST /node/694084eb575bc060ce15d992 HTTP/1.0" 202 Accepted 2025-12-17 22:00:17,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408464575bc060ce15d752 2025-12-17 22:00:17,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408464575bc060ce15d752 HTTP/1.1" 502 150 2025-12-17 22:00:17,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408464575bc060ce15d752'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:17,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408465575bc060ce15d755 2025-12-17 22:00:17,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408465575bc060ce15d755 HTTP/1.1" 502 150 2025-12-17 22:00:17,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408465575bc060ce15d755'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:19,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844a575bc060ce15d70a 2025-12-17 22:00:19,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844a575bc060ce15d70a HTTP/1.1" 502 150 2025-12-17 22:00:19,714 - lava_callback - ERROR - Error processing callback for node 6940844a575bc060ce15d70a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940844a575bc060ce15d70a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940844a575bc060ce15d70a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940844a575bc060ce15d70a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:19,716 - lava_callback - INFO - Starting async job submit for node 6940846c575bc060ce15d76a 2025-12-17 22:00:19,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:19,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846c575bc060ce15d76a HTTP/1.1" 502 150 2025-12-17 22:00:19,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846c575bc060ce15d76a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:19,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846c575bc060ce15d76a 2025-12-17 22:00:19,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846c575bc060ce15d76a HTTP/1.1" 502 150 2025-12-17 22:00:19,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846c575bc060ce15d76a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:19,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845f575bc060ce15d746 2025-12-17 22:00:19,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845f575bc060ce15d746 HTTP/1.1" 502 150 2025-12-17 22:00:19,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845f575bc060ce15d746'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:21,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846c575bc060ce15d76a 2025-12-17 22:00:21,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846c575bc060ce15d76a HTTP/1.1" 502 150 2025-12-17 22:00:21,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846c575bc060ce15d76a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:21,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408460575bc060ce15d749 2025-12-17 22:00:21,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408460575bc060ce15d749 HTTP/1.1" 502 150 2025-12-17 22:00:21,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408460575bc060ce15d749'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:21,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408465575bc060ce15d755 2025-12-17 22:00:21,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408465575bc060ce15d755 HTTP/1.1" 502 150 2025-12-17 22:00:21,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408465575bc060ce15d755'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:21,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408462575bc060ce15d74c 2025-12-17 22:00:21,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408462575bc060ce15d74c HTTP/1.1" 502 150 2025-12-17 22:00:21,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408462575bc060ce15d74c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:22,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940843c575bc060ce15d681 2025-12-17 22:00:22,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940843c575bc060ce15d681 HTTP/1.1" 502 150 2025-12-17 22:00:22,322 - lava_callback - ERROR - Error processing callback for node 6940843c575bc060ce15d681: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940843c575bc060ce15d681 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940843c575bc060ce15d681 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940843c575bc060ce15d681 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:22,324 - lava_callback - INFO - Starting async job submit for node 6940846d575bc060ce15d772 2025-12-17 22:00:22,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:22,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846d575bc060ce15d772 HTTP/1.1" 502 150 2025-12-17 22:00:22,353 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846d575bc060ce15d772'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:22,353 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846d575bc060ce15d772 2025-12-17 22:00:22,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846d575bc060ce15d772 HTTP/1.1" 502 150 2025-12-17 22:00:22,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846d575bc060ce15d772'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:22,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408463575bc060ce15d74f 2025-12-17 22:00:22,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408463575bc060ce15d74f HTTP/1.1" 502 150 2025-12-17 22:00:22,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408463575bc060ce15d74f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:23,278 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844c575bc060ce15d70d 2025-12-17 22:00:23,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844c575bc060ce15d70d HTTP/1.1" 502 150 2025-12-17 22:00:23,279 - lava_callback - ERROR - Error processing callback for node 6940844c575bc060ce15d70d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940844c575bc060ce15d70d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940844c575bc060ce15d70d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940844c575bc060ce15d70d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:23,281 - lava_callback - INFO - Starting async job submit for node 69408466575bc060ce15d758 2025-12-17 22:00:23,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:23,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408466575bc060ce15d758 HTTP/1.1" 502 150 2025-12-17 22:00:23,292 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408466575bc060ce15d758'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:23,292 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408466575bc060ce15d758 2025-12-17 22:00:23,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408466575bc060ce15d758 HTTP/1.1" 502 150 2025-12-17 22:00:23,293 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408466575bc060ce15d758'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:23,373 - lava_callback - INFO - Received callback for node 694084f4575bc060ce15d9ae INFO: 172.21.0.1:39710 - "POST /node/694084f4575bc060ce15d9ae HTTP/1.0" 202 Accepted 2025-12-17 22:00:24,354 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846d575bc060ce15d772 2025-12-17 22:00:24,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846d575bc060ce15d772 HTTP/1.1" 502 150 2025-12-17 22:00:24,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846d575bc060ce15d772'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:24,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408215575bc060ce15cd1f 2025-12-17 22:00:24,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408215575bc060ce15cd1f HTTP/1.1" 502 150 2025-12-17 22:00:24,872 - lava_callback - ERROR - Error processing callback for node 69408215575bc060ce15cd1f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408215575bc060ce15cd1f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408215575bc060ce15cd1f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408215575bc060ce15cd1f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:24,874 - lava_callback - INFO - Starting async job submit for node 69408468575bc060ce15d75b 2025-12-17 22:00:24,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:24,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408468575bc060ce15d75b HTTP/1.1" 502 150 2025-12-17 22:00:24,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408468575bc060ce15d75b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:24,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408468575bc060ce15d75b 2025-12-17 22:00:24,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408468575bc060ce15d75b HTTP/1.1" 502 150 2025-12-17 22:00:24,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408468575bc060ce15d75b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:24,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844e575bc060ce15d713 2025-12-17 22:00:24,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844e575bc060ce15d713 HTTP/1.1" 502 150 2025-12-17 22:00:24,981 - lava_callback - ERROR - Error processing callback for node 6940844e575bc060ce15d713: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940844e575bc060ce15d713 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940844e575bc060ce15d713 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940844e575bc060ce15d713 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:24,983 - lava_callback - INFO - Starting async job submit for node 6940846a575bc060ce15d760 2025-12-17 22:00:24,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:25,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846a575bc060ce15d760 HTTP/1.1" 502 150 2025-12-17 22:00:25,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846a575bc060ce15d760'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:25,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846a575bc060ce15d760 2025-12-17 22:00:25,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846a575bc060ce15d760 HTTP/1.1" 502 150 2025-12-17 22:00:25,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846a575bc060ce15d760'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:25,293 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408466575bc060ce15d758 2025-12-17 22:00:25,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408466575bc060ce15d758 HTTP/1.1" 502 150 2025-12-17 22:00:25,296 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408466575bc060ce15d758'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:25,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408464575bc060ce15d752 2025-12-17 22:00:25,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408464575bc060ce15d752 HTTP/1.1" 502 150 2025-12-17 22:00:25,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408464575bc060ce15d752'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:25,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846c575bc060ce15d76a 2025-12-17 22:00:25,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846c575bc060ce15d76a HTTP/1.1" 502 150 2025-12-17 22:00:25,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846c575bc060ce15d76a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39722 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:00:26,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408468575bc060ce15d75b 2025-12-17 22:00:26,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408468575bc060ce15d75b HTTP/1.1" 502 150 2025-12-17 22:00:26,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408468575bc060ce15d75b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:27,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846a575bc060ce15d760 2025-12-17 22:00:27,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846a575bc060ce15d760 HTTP/1.1" 502 150 2025-12-17 22:00:27,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846a575bc060ce15d760'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:28,356 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846d575bc060ce15d772 2025-12-17 22:00:28,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846d575bc060ce15d772 HTTP/1.1" 502 150 2025-12-17 22:00:28,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846d575bc060ce15d772'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:28,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408456575bc060ce15d723 2025-12-17 22:00:28,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408456575bc060ce15d723 HTTP/1.1" 502 150 2025-12-17 22:00:28,493 - lava_callback - ERROR - Error processing callback for node 69408456575bc060ce15d723: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408456575bc060ce15d723 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408456575bc060ce15d723 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408456575bc060ce15d723 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:28,496 - lava_callback - INFO - Starting async job submit for node 6940846e575bc060ce15d778 2025-12-17 22:00:28,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:28,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846e575bc060ce15d778 HTTP/1.1" 502 150 2025-12-17 22:00:28,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846e575bc060ce15d778'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:28,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846e575bc060ce15d778 2025-12-17 22:00:28,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846e575bc060ce15d778 HTTP/1.1" 502 150 2025-12-17 22:00:28,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846e575bc060ce15d778'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:28,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940844d575bc060ce15d710 2025-12-17 22:00:28,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940844d575bc060ce15d710 HTTP/1.1" 502 150 2025-12-17 22:00:28,575 - lava_callback - ERROR - Error processing callback for node 6940844d575bc060ce15d710: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940844d575bc060ce15d710 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940844d575bc060ce15d710 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940844d575bc060ce15d710 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:28,576 - lava_callback - INFO - Starting async job submit for node 6940846f575bc060ce15d77b 2025-12-17 22:00:28,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:28,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846f575bc060ce15d77b HTTP/1.1" 502 150 2025-12-17 22:00:28,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846f575bc060ce15d77b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:28,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846f575bc060ce15d77b 2025-12-17 22:00:28,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846f575bc060ce15d77b HTTP/1.1" 502 150 2025-12-17 22:00:28,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846f575bc060ce15d77b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:28,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845d575bc060ce15d740 2025-12-17 22:00:28,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845d575bc060ce15d740 HTTP/1.1" 502 150 2025-12-17 22:00:28,788 - lava_callback - ERROR - Error processing callback for node 6940845d575bc060ce15d740: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940845d575bc060ce15d740 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940845d575bc060ce15d740 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940845d575bc060ce15d740 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:28,791 - lava_callback - INFO - Starting async job submit for node 69408471575bc060ce15d77e 2025-12-17 22:00:28,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:28,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408471575bc060ce15d77e HTTP/1.1" 502 150 2025-12-17 22:00:28,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408471575bc060ce15d77e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:28,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408471575bc060ce15d77e 2025-12-17 22:00:28,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408471575bc060ce15d77e HTTP/1.1" 502 150 2025-12-17 22:00:28,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408471575bc060ce15d77e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:28,942 - lava_callback - INFO - Received callback for node 694084f2575bc060ce15d9a8 INFO: 172.21.0.1:39726 - "POST /node/694084f2575bc060ce15d9a8 HTTP/1.0" 202 Accepted 2025-12-17 22:00:29,296 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408466575bc060ce15d758 2025-12-17 22:00:29,631 - lava_callback - INFO - Received callback for node 694084f3575bc060ce15d9ab INFO: 172.21.0.1:39732 - "POST /node/694084f3575bc060ce15d9ab HTTP/1.0" 202 Accepted 2025-12-17 22:00:29,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408465575bc060ce15d755 2025-12-17 22:00:29,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408465575bc060ce15d755 HTTP/1.1" 502 150 2025-12-17 22:00:29,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408465575bc060ce15d755'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:30,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846e575bc060ce15d778 2025-12-17 22:00:30,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846f575bc060ce15d77b 2025-12-17 22:00:30,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846f575bc060ce15d77b HTTP/1.1" 502 150 2025-12-17 22:00:30,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846f575bc060ce15d77b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:30,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408471575bc060ce15d77e 2025-12-17 22:00:30,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408471575bc060ce15d77e HTTP/1.1" 502 150 2025-12-17 22:00:30,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408471575bc060ce15d77e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:30,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408468575bc060ce15d75b 2025-12-17 22:00:30,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408468575bc060ce15d75b HTTP/1.1" 502 150 2025-12-17 22:00:30,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408468575bc060ce15d75b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:30,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408447575bc060ce15d6f8 HTTP/1.1" 502 150 2025-12-17 22:00:30,900 - lava_callback - ERROR - Error processing callback for node 69408447575bc060ce15d6f8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408447575bc060ce15d6f8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408447575bc060ce15d6f8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408447575bc060ce15d6f8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:30,902 - lava_callback - INFO - Starting async job submit for node 69408472575bc060ce15d781 2025-12-17 22:00:30,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:30,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408472575bc060ce15d781 HTTP/1.1" 502 150 2025-12-17 22:00:30,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408472575bc060ce15d781'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:30,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408472575bc060ce15d781 2025-12-17 22:00:30,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408472575bc060ce15d781 HTTP/1.1" 502 150 2025-12-17 22:00:30,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408472575bc060ce15d781'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:31,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846a575bc060ce15d760 2025-12-17 22:00:31,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846a575bc060ce15d760 HTTP/1.1" 502 150 2025-12-17 22:00:31,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846a575bc060ce15d760'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:31,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845e575bc060ce15d743 2025-12-17 22:00:31,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845e575bc060ce15d743 HTTP/1.1" 502 150 2025-12-17 22:00:31,829 - lava_callback - ERROR - Error processing callback for node 6940845e575bc060ce15d743: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940845e575bc060ce15d743 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940845e575bc060ce15d743 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940845e575bc060ce15d743 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:31,831 - lava_callback - INFO - Starting async job submit for node 69408473575bc060ce15d784 2025-12-17 22:00:31,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:31,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408473575bc060ce15d784 HTTP/1.1" 502 150 2025-12-17 22:00:31,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408473575bc060ce15d784'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:31,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408473575bc060ce15d784 2025-12-17 22:00:31,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408473575bc060ce15d784 HTTP/1.1" 502 150 2025-12-17 22:00:31,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408473575bc060ce15d784'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:32,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408472575bc060ce15d781 2025-12-17 22:00:32,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408472575bc060ce15d781 HTTP/1.1" 502 150 2025-12-17 22:00:32,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408472575bc060ce15d781'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:33,591 - lava_callback - INFO - Received callback for node 694084ed575bc060ce15d99c INFO: 172.21.0.1:43648 - "POST /node/694084ed575bc060ce15d99c HTTP/1.0" 202 Accepted 2025-12-17 22:00:33,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846c575bc060ce15d76a 2025-12-17 22:00:33,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846c575bc060ce15d76a HTTP/1.1" 502 150 2025-12-17 22:00:33,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846c575bc060ce15d76a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:33,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408473575bc060ce15d784 2025-12-17 22:00:33,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408473575bc060ce15d784 HTTP/1.1" 502 150 2025-12-17 22:00:33,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408473575bc060ce15d784'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:34,250 - lava_callback - INFO - Received callback for node 694084ee575bc060ce15d99f INFO: 172.21.0.1:43662 - "POST /node/694084ee575bc060ce15d99f HTTP/1.0" 202 Accepted 2025-12-17 22:00:34,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846f575bc060ce15d77b 2025-12-17 22:00:34,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846f575bc060ce15d77b HTTP/1.1" 502 150 2025-12-17 22:00:34,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846f575bc060ce15d77b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:34,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408471575bc060ce15d77e 2025-12-17 22:00:34,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408471575bc060ce15d77e HTTP/1.1" 502 150 2025-12-17 22:00:34,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408471575bc060ce15d77e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:34,894 - lava_callback - INFO - Received callback for node 694084ef575bc060ce15d9a2 INFO: 172.21.0.1:43676 - "POST /node/694084ef575bc060ce15d9a2 HTTP/1.0" 202 Accepted 2025-12-17 22:00:35,551 - lava_callback - INFO - Received callback for node 694084f0575bc060ce15d9a5 INFO: 172.21.0.1:43682 - "POST /node/694084f0575bc060ce15d9a5 HTTP/1.0" 202 Accepted 2025-12-17 22:00:35,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845f575bc060ce15d746 2025-12-17 22:00:35,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845f575bc060ce15d746 HTTP/1.1" 502 150 2025-12-17 22:00:35,739 - lava_callback - ERROR - Error processing callback for node 6940845f575bc060ce15d746: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940845f575bc060ce15d746 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940845f575bc060ce15d746 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940845f575bc060ce15d746 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:35,741 - lava_callback - INFO - Starting async job submit for node 69408475575bc060ce15d787 2025-12-17 22:00:35,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:35,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408475575bc060ce15d787 HTTP/1.1" 502 150 2025-12-17 22:00:35,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408475575bc060ce15d787'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:35,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408475575bc060ce15d787 2025-12-17 22:00:35,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408475575bc060ce15d787 HTTP/1.1" 502 150 2025-12-17 22:00:35,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408475575bc060ce15d787'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:36,357 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846d575bc060ce15d772 2025-12-17 22:00:36,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846d575bc060ce15d772 HTTP/1.1" 502 150 2025-12-17 22:00:36,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846d575bc060ce15d772'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43692 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:00:36,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408472575bc060ce15d781 2025-12-17 22:00:36,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408472575bc060ce15d781 HTTP/1.1" 502 150 2025-12-17 22:00:36,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408472575bc060ce15d781'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:37,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408475575bc060ce15d787 2025-12-17 22:00:37,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408475575bc060ce15d787 HTTP/1.1" 502 150 2025-12-17 22:00:37,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408475575bc060ce15d787'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:37,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408460575bc060ce15d749 2025-12-17 22:00:37,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408460575bc060ce15d749 HTTP/1.1" 502 150 2025-12-17 22:00:37,813 - lava_callback - ERROR - Error processing callback for node 69408460575bc060ce15d749: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408460575bc060ce15d749 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408460575bc060ce15d749 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408460575bc060ce15d749 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:37,815 - lava_callback - INFO - Starting async job submit for node 69408476575bc060ce15d78a 2025-12-17 22:00:37,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:37,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408476575bc060ce15d78a HTTP/1.1" 502 150 2025-12-17 22:00:37,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408476575bc060ce15d78a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:37,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408476575bc060ce15d78a 2025-12-17 22:00:37,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408476575bc060ce15d78a HTTP/1.1" 502 150 2025-12-17 22:00:37,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408476575bc060ce15d78a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:37,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408473575bc060ce15d784 2025-12-17 22:00:37,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408473575bc060ce15d784 HTTP/1.1" 502 150 2025-12-17 22:00:37,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408473575bc060ce15d784'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:37,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408462575bc060ce15d74c 2025-12-17 22:00:37,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408462575bc060ce15d74c HTTP/1.1" 502 150 2025-12-17 22:00:37,971 - lava_callback - ERROR - Error processing callback for node 69408462575bc060ce15d74c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408462575bc060ce15d74c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408462575bc060ce15d74c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408462575bc060ce15d74c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:37,973 - lava_callback - INFO - Starting async job submit for node 69408477575bc060ce15d78d 2025-12-17 22:00:37,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:37,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408477575bc060ce15d78d HTTP/1.1" 502 150 2025-12-17 22:00:37,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408477575bc060ce15d78d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:37,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408477575bc060ce15d78d 2025-12-17 22:00:37,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408477575bc060ce15d78d HTTP/1.1" 502 150 2025-12-17 22:00:37,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408477575bc060ce15d78d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:38,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408463575bc060ce15d74f 2025-12-17 22:00:38,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408463575bc060ce15d74f HTTP/1.1" 502 150 2025-12-17 22:00:38,578 - lava_callback - ERROR - Error processing callback for node 69408463575bc060ce15d74f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408463575bc060ce15d74f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408463575bc060ce15d74f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408463575bc060ce15d74f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:38,580 - lava_callback - INFO - Starting async job submit for node 69408479575bc060ce15d790 2025-12-17 22:00:38,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:38,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408479575bc060ce15d790 HTTP/1.1" 502 150 2025-12-17 22:00:38,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408479575bc060ce15d790'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:38,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408479575bc060ce15d790 2025-12-17 22:00:38,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408479575bc060ce15d790 HTTP/1.1" 502 150 2025-12-17 22:00:38,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408479575bc060ce15d790'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:38,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408468575bc060ce15d75b 2025-12-17 22:00:38,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408468575bc060ce15d75b HTTP/1.1" 502 150 2025-12-17 22:00:38,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408468575bc060ce15d75b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:39,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846a575bc060ce15d760 2025-12-17 22:00:39,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846a575bc060ce15d760 HTTP/1.1" 502 150 2025-12-17 22:00:39,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846a575bc060ce15d760'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:39,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408476575bc060ce15d78a 2025-12-17 22:00:39,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408476575bc060ce15d78a HTTP/1.1" 502 150 2025-12-17 22:00:39,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408476575bc060ce15d78a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:39,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408477575bc060ce15d78d 2025-12-17 22:00:39,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408477575bc060ce15d78d HTTP/1.1" 502 150 2025-12-17 22:00:39,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408477575bc060ce15d78d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:40,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408479575bc060ce15d790 2025-12-17 22:00:40,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408479575bc060ce15d790 HTTP/1.1" 502 150 2025-12-17 22:00:40,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408479575bc060ce15d790'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:41,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408464575bc060ce15d752 2025-12-17 22:00:41,713 - lava_callback - INFO - Received callback for node 69408457575bc060ce15d728 INFO: 172.21.0.1:42658 - "POST /node/69408457575bc060ce15d728 HTTP/1.0" 202 Accepted 2025-12-17 22:00:41,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408475575bc060ce15d787 2025-12-17 22:00:41,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408475575bc060ce15d787 HTTP/1.1" 502 150 2025-12-17 22:00:41,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408475575bc060ce15d787'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:42,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846f575bc060ce15d77b 2025-12-17 22:00:42,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846f575bc060ce15d77b HTTP/1.1" 502 150 2025-12-17 22:00:42,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846f575bc060ce15d77b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:42,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408471575bc060ce15d77e 2025-12-17 22:00:42,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408471575bc060ce15d77e HTTP/1.1" 502 150 2025-12-17 22:00:42,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408471575bc060ce15d77e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:43,393 - lava_callback - INFO - Received callback for node 694084f6575bc060ce15d9b3 INFO: 172.21.0.1:42670 - "POST /node/694084f6575bc060ce15d9b3 HTTP/1.0" 202 Accepted 2025-12-17 22:00:43,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408476575bc060ce15d78a 2025-12-17 22:00:43,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408476575bc060ce15d78a HTTP/1.1" 502 150 2025-12-17 22:00:43,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408476575bc060ce15d78a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:43,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408477575bc060ce15d78d 2025-12-17 22:00:43,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408477575bc060ce15d78d HTTP/1.1" 502 150 2025-12-17 22:00:43,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408477575bc060ce15d78d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:44,029 - lava_callback - INFO - Received callback for node 694084f7575bc060ce15d9ba INFO: 172.21.0.1:42672 - "POST /node/694084f7575bc060ce15d9ba HTTP/1.0" 202 Accepted 2025-12-17 22:00:44,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408479575bc060ce15d790 2025-12-17 22:00:44,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408479575bc060ce15d790 HTTP/1.1" 502 150 2025-12-17 22:00:44,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408479575bc060ce15d790'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:44,679 - lava_callback - INFO - Received callback for node 694084f9575bc060ce15d9bd INFO: 172.21.0.1:42686 - "POST /node/694084f9575bc060ce15d9bd HTTP/1.0" 202 Accepted 2025-12-17 22:00:44,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408466575bc060ce15d758 HTTP/1.1" 502 150 2025-12-17 22:00:44,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408466575bc060ce15d758'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:44,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408472575bc060ce15d781 2025-12-17 22:00:44,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408472575bc060ce15d781 HTTP/1.1" 502 150 2025-12-17 22:00:44,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408472575bc060ce15d781'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:45,324 - lava_callback - INFO - Received callback for node 694084fa575bc060ce15d9c0 INFO: 172.21.0.1:42700 - "POST /node/694084fa575bc060ce15d9c0 HTTP/1.0" 202 Accepted 2025-12-17 22:00:45,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408473575bc060ce15d784 2025-12-17 22:00:45,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408473575bc060ce15d784 HTTP/1.1" 502 150 2025-12-17 22:00:45,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408473575bc060ce15d784'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:45,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408465575bc060ce15d755 2025-12-17 22:00:45,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408465575bc060ce15d755 HTTP/1.1" 502 150 2025-12-17 22:00:45,945 - lava_callback - ERROR - Error processing callback for node 69408465575bc060ce15d755: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408465575bc060ce15d755 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408465575bc060ce15d755 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408465575bc060ce15d755 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:45,946 - lava_callback - INFO - Starting async job submit for node 6940847a575bc060ce15d793 2025-12-17 22:00:45,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:45,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847a575bc060ce15d793 HTTP/1.1" 502 150 2025-12-17 22:00:45,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847a575bc060ce15d793'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:45,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847a575bc060ce15d793 2025-12-17 22:00:45,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847a575bc060ce15d793 HTTP/1.1" 502 150 2025-12-17 22:00:45,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847a575bc060ce15d793'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:45,985 - lava_callback - INFO - Received callback for node 694084fc575bc060ce15d9e1 INFO: 172.21.0.1:42714 - "POST /node/694084fc575bc060ce15d9e1 HTTP/1.0" 202 Accepted 2025-12-17 22:00:46,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846e575bc060ce15d778 HTTP/1.1" 502 150 2025-12-17 22:00:46,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846e575bc060ce15d778'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42730 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:00:46,617 - lava_callback - INFO - Received callback for node 69408500575bc060ce15da26 INFO: 172.21.0.1:42736 - "POST /node/69408500575bc060ce15da26 HTTP/1.0" 202 Accepted 2025-12-17 22:00:47,259 - lava_callback - INFO - Received callback for node 69408501575bc060ce15da29 INFO: 172.21.0.1:42744 - "POST /node/69408501575bc060ce15da29 HTTP/1.0" 202 Accepted 2025-12-17 22:00:47,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847a575bc060ce15d793 2025-12-17 22:00:47,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847a575bc060ce15d793 HTTP/1.1" 502 150 2025-12-17 22:00:47,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847a575bc060ce15d793'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:48,532 - lava_callback - INFO - Received callback for node 694084ff575bc060ce15da18 INFO: 172.21.0.1:42754 - "POST /node/694084ff575bc060ce15da18 HTTP/1.0" 202 Accepted 2025-12-17 22:00:49,189 - lava_callback - INFO - Received callback for node 6940850d575bc060ce15da51 INFO: 172.21.0.1:42758 - "POST /node/6940850d575bc060ce15da51 HTTP/1.0" 202 Accepted 2025-12-17 22:00:49,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846c575bc060ce15d76a 2025-12-17 22:00:49,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846c575bc060ce15d76a HTTP/1.1" 502 150 2025-12-17 22:00:49,728 - lava_callback - ERROR - Error processing callback for node 6940846c575bc060ce15d76a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940846c575bc060ce15d76a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940846c575bc060ce15d76a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940846c575bc060ce15d76a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:49,732 - lava_callback - INFO - Starting async job submit for node 6940847b575bc060ce15d796 2025-12-17 22:00:49,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:49,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847b575bc060ce15d796 HTTP/1.1" 502 150 2025-12-17 22:00:49,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847b575bc060ce15d796'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:49,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847b575bc060ce15d796 2025-12-17 22:00:49,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847b575bc060ce15d796 HTTP/1.1" 502 150 2025-12-17 22:00:49,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847b575bc060ce15d796'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:49,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408475575bc060ce15d787 2025-12-17 22:00:49,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408475575bc060ce15d787 HTTP/1.1" 502 150 2025-12-17 22:00:49,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408475575bc060ce15d787'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:49,873 - lava_callback - INFO - Received callback for node 6940850e575bc060ce15da58 INFO: 172.21.0.1:39142 - "POST /node/6940850e575bc060ce15da58 HTTP/1.0" 202 Accepted 2025-12-17 22:00:50,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846e575bc060ce15d778 2025-12-17 22:00:50,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846e575bc060ce15d778 HTTP/1.1" 502 150 2025-12-17 22:00:50,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846e575bc060ce15d778'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:51,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847b575bc060ce15d796 2025-12-17 22:00:51,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847b575bc060ce15d796 HTTP/1.1" 502 150 2025-12-17 22:00:51,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847b575bc060ce15d796'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:51,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408476575bc060ce15d78a 2025-12-17 22:00:51,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408476575bc060ce15d78a HTTP/1.1" 502 150 2025-12-17 22:00:51,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408476575bc060ce15d78a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:51,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847a575bc060ce15d793 2025-12-17 22:00:51,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847a575bc060ce15d793 HTTP/1.1" 502 150 2025-12-17 22:00:51,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847a575bc060ce15d793'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:51,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408477575bc060ce15d78d 2025-12-17 22:00:51,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408477575bc060ce15d78d HTTP/1.1" 502 150 2025-12-17 22:00:51,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408477575bc060ce15d78d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:52,359 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846d575bc060ce15d772 2025-12-17 22:00:52,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846d575bc060ce15d772 HTTP/1.1" 502 150 2025-12-17 22:00:52,362 - lava_callback - ERROR - Error processing callback for node 6940846d575bc060ce15d772: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940846d575bc060ce15d772 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940846d575bc060ce15d772 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940846d575bc060ce15d772 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:52,364 - lava_callback - INFO - Starting async job submit for node 6940847c575bc060ce15d799 2025-12-17 22:00:52,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:52,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847c575bc060ce15d799 HTTP/1.1" 502 150 2025-12-17 22:00:52,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847c575bc060ce15d799'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:52,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847c575bc060ce15d799 2025-12-17 22:00:52,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847c575bc060ce15d799 HTTP/1.1" 502 150 2025-12-17 22:00:52,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847c575bc060ce15d799'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:52,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408479575bc060ce15d790 2025-12-17 22:00:52,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408479575bc060ce15d790 HTTP/1.1" 502 150 2025-12-17 22:00:52,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408479575bc060ce15d790'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:52,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408466575bc060ce15d758 2025-12-17 22:00:52,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408466575bc060ce15d758 HTTP/1.1" 502 150 2025-12-17 22:00:52,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408466575bc060ce15d758'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:53,639 - lava_callback - INFO - Received callback for node 69408503575bc060ce15da30 INFO: 172.21.0.1:39152 - "POST /node/69408503575bc060ce15da30 HTTP/1.0" 202 Accepted 2025-12-17 22:00:54,293 - lava_callback - INFO - Received callback for node 69408505575bc060ce15da33 INFO: 172.21.0.1:39168 - "POST /node/69408505575bc060ce15da33 HTTP/1.0" 202 Accepted 2025-12-17 22:00:54,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847c575bc060ce15d799 2025-12-17 22:00:54,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847c575bc060ce15d799 HTTP/1.1" 502 150 2025-12-17 22:00:54,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847c575bc060ce15d799'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:54,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408468575bc060ce15d75b 2025-12-17 22:00:54,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408468575bc060ce15d75b HTTP/1.1" 502 150 2025-12-17 22:00:54,887 - lava_callback - ERROR - Error processing callback for node 69408468575bc060ce15d75b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408468575bc060ce15d75b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408468575bc060ce15d75b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408468575bc060ce15d75b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:54,890 - lava_callback - INFO - Starting async job submit for node 6940847e575bc060ce15d79d 2025-12-17 22:00:54,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:54,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847e575bc060ce15d79d HTTP/1.1" 502 150 2025-12-17 22:00:54,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847e575bc060ce15d79d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:54,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847e575bc060ce15d79d 2025-12-17 22:00:54,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847e575bc060ce15d79d HTTP/1.1" 502 150 2025-12-17 22:00:54,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847e575bc060ce15d79d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:54,942 - lava_callback - INFO - Received callback for node 69408506575bc060ce15da36 INFO: 172.21.0.1:39182 - "POST /node/69408506575bc060ce15da36 HTTP/1.0" 202 Accepted 2025-12-17 22:00:55,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846a575bc060ce15d760 2025-12-17 22:00:55,623 - lava_callback - INFO - Received callback for node 69408508575bc060ce15da39 INFO: 172.21.0.1:39196 - "POST /node/69408508575bc060ce15da39 HTTP/1.0" 202 Accepted 2025-12-17 22:00:55,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847b575bc060ce15d796 2025-12-17 22:00:55,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847b575bc060ce15d796 HTTP/1.1" 502 150 2025-12-17 22:00:55,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847b575bc060ce15d796'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:56,304 - lava_callback - INFO - Received callback for node 69408509575bc060ce15da3c INFO: 172.21.0.1:39206 - "POST /node/69408509575bc060ce15da3c HTTP/1.0" 202 Accepted INFO: 172.21.0.1:39210 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:00:56,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847e575bc060ce15d79d 2025-12-17 22:00:56,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847e575bc060ce15d79d HTTP/1.1" 502 150 2025-12-17 22:00:56,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847e575bc060ce15d79d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:56,983 - lava_callback - INFO - Received callback for node 6940850a575bc060ce15da43 INFO: 172.21.0.1:39224 - "POST /node/6940850a575bc060ce15da43 HTTP/1.0" 202 Accepted 2025-12-17 22:00:57,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408464575bc060ce15d752 HTTP/1.1" 502 150 2025-12-17 22:00:57,010 - lava_callback - ERROR - Error processing callback for node 69408464575bc060ce15d752: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408464575bc060ce15d752 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408464575bc060ce15d752 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408464575bc060ce15d752 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:57,012 - lava_callback - INFO - Starting async job submit for node 69408480575bc060ce15d7c4 2025-12-17 22:00:57,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:57,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408480575bc060ce15d7c4 HTTP/1.1" 502 150 2025-12-17 22:00:57,205 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408480575bc060ce15d7c4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:57,205 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408480575bc060ce15d7c4 2025-12-17 22:00:57,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408480575bc060ce15d7c4 HTTP/1.1" 502 150 2025-12-17 22:00:57,206 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408480575bc060ce15d7c4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:57,666 - lava_callback - INFO - Received callback for node 6940850c575bc060ce15da4c INFO: 172.21.0.1:39234 - "POST /node/6940850c575bc060ce15da4c HTTP/1.0" 202 Accepted 2025-12-17 22:00:58,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846e575bc060ce15d778 2025-12-17 22:00:58,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846e575bc060ce15d778 HTTP/1.1" 502 150 2025-12-17 22:00:58,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940846e575bc060ce15d778'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:58,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847c575bc060ce15d799 2025-12-17 22:00:58,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847c575bc060ce15d799 HTTP/1.1" 502 150 2025-12-17 22:00:58,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847c575bc060ce15d799'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:58,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846f575bc060ce15d77b 2025-12-17 22:00:58,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846f575bc060ce15d77b HTTP/1.1" 502 150 2025-12-17 22:00:58,593 - lava_callback - ERROR - Error processing callback for node 6940846f575bc060ce15d77b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940846f575bc060ce15d77b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940846f575bc060ce15d77b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940846f575bc060ce15d77b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:58,595 - lava_callback - INFO - Starting async job submit for node 69408482575bc060ce15d7e2 2025-12-17 22:00:58,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:58,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408482575bc060ce15d7e2 HTTP/1.1" 502 150 2025-12-17 22:00:58,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408482575bc060ce15d7e2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:58,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408482575bc060ce15d7e2 2025-12-17 22:00:58,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408482575bc060ce15d7e2 HTTP/1.1" 502 150 2025-12-17 22:00:58,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408482575bc060ce15d7e2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:58,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408471575bc060ce15d77e 2025-12-17 22:00:58,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408471575bc060ce15d77e HTTP/1.1" 502 150 2025-12-17 22:00:58,807 - lava_callback - ERROR - Error processing callback for node 69408471575bc060ce15d77e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408471575bc060ce15d77e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408471575bc060ce15d77e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408471575bc060ce15d77e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:00:58,808 - lava_callback - INFO - Starting async job submit for node 69408482575bc060ce15d7e7 2025-12-17 22:00:58,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:00:58,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408482575bc060ce15d7e7 HTTP/1.1" 502 150 2025-12-17 22:00:58,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408482575bc060ce15d7e7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:58,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408482575bc060ce15d7e7 2025-12-17 22:00:58,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408482575bc060ce15d7e7 HTTP/1.1" 502 150 2025-12-17 22:00:58,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408482575bc060ce15d7e7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:59,206 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408480575bc060ce15d7c4 2025-12-17 22:00:59,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408480575bc060ce15d7c4 HTTP/1.1" 502 150 2025-12-17 22:00:59,207 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408480575bc060ce15d7c4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:00:59,236 - lava_callback - INFO - Received callback for node 69408512575bc060ce15da64 INFO: 172.21.0.1:39250 - "POST /node/69408512575bc060ce15da64 HTTP/1.0" 202 Accepted 2025-12-17 22:00:59,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847a575bc060ce15d793 2025-12-17 22:00:59,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847a575bc060ce15d793 HTTP/1.1" 502 150 2025-12-17 22:00:59,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847a575bc060ce15d793'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:00,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408482575bc060ce15d7e2 2025-12-17 22:01:00,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408482575bc060ce15d7e2 HTTP/1.1" 502 150 2025-12-17 22:01:00,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408482575bc060ce15d7e2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:00,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408482575bc060ce15d7e7 2025-12-17 22:01:00,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408482575bc060ce15d7e7 HTTP/1.1" 502 150 2025-12-17 22:01:00,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408482575bc060ce15d7e7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:00,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408472575bc060ce15d781 2025-12-17 22:01:00,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408472575bc060ce15d781 HTTP/1.1" 502 150 2025-12-17 22:01:00,916 - lava_callback - ERROR - Error processing callback for node 69408472575bc060ce15d781: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408472575bc060ce15d781 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408472575bc060ce15d781 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408472575bc060ce15d781 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:00,918 - lava_callback - INFO - Starting async job submit for node 69408485575bc060ce15d7f4 2025-12-17 22:01:00,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:00,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847e575bc060ce15d79d 2025-12-17 22:01:00,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847e575bc060ce15d79d HTTP/1.1" 502 150 2025-12-17 22:01:00,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847e575bc060ce15d79d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:00,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408485575bc060ce15d7f4 HTTP/1.1" 502 150 2025-12-17 22:01:00,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408485575bc060ce15d7f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:00,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408485575bc060ce15d7f4 2025-12-17 22:01:00,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408485575bc060ce15d7f4 HTTP/1.1" 502 150 2025-12-17 22:01:00,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408485575bc060ce15d7f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:01,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408473575bc060ce15d784 2025-12-17 22:01:01,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408473575bc060ce15d784 HTTP/1.1" 502 150 2025-12-17 22:01:01,843 - lava_callback - ERROR - Error processing callback for node 69408473575bc060ce15d784: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408473575bc060ce15d784 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408473575bc060ce15d784 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408473575bc060ce15d784 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:01,845 - lava_callback - INFO - Starting async job submit for node 69408487575bc060ce15d7fb 2025-12-17 22:01:01,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:01,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408487575bc060ce15d7fb HTTP/1.1" 502 150 2025-12-17 22:01:01,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408487575bc060ce15d7fb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:01,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408487575bc060ce15d7fb 2025-12-17 22:01:01,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408487575bc060ce15d7fb HTTP/1.1" 502 150 2025-12-17 22:01:01,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408487575bc060ce15d7fb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52614 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:01:02,011 - lava_callback - INFO - Received callback for node 69408459575bc060ce15d72f INFO: 172.21.0.1:52628 - "POST /node/69408459575bc060ce15d72f HTTP/1.0" 202 Accepted 2025-12-17 22:01:02,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408485575bc060ce15d7f4 2025-12-17 22:01:02,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408485575bc060ce15d7f4 HTTP/1.1" 502 150 2025-12-17 22:01:02,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408485575bc060ce15d7f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:03,208 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408480575bc060ce15d7c4 2025-12-17 22:01:03,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408480575bc060ce15d7c4 HTTP/1.1" 502 150 2025-12-17 22:01:03,209 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408480575bc060ce15d7c4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:03,431 - lava_callback - INFO - Received callback for node 6940850f575bc060ce15da5d INFO: 172.21.0.1:52632 - "POST /node/6940850f575bc060ce15da5d HTTP/1.0" 202 Accepted 2025-12-17 22:01:03,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847b575bc060ce15d796 2025-12-17 22:01:03,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847b575bc060ce15d796 HTTP/1.1" 502 150 2025-12-17 22:01:03,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847b575bc060ce15d796'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:03,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408487575bc060ce15d7fb 2025-12-17 22:01:03,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408487575bc060ce15d7fb HTTP/1.1" 502 150 2025-12-17 22:01:03,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408487575bc060ce15d7fb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:04,078 - lava_callback - INFO - Received callback for node 69408513575bc060ce15da67 INFO: 172.21.0.1:52648 - "POST /node/69408513575bc060ce15da67 HTTP/1.0" 202 Accepted 2025-12-17 22:01:04,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408482575bc060ce15d7e2 2025-12-17 22:01:04,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408482575bc060ce15d7e2 HTTP/1.1" 502 150 2025-12-17 22:01:04,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408482575bc060ce15d7e2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:04,738 - lava_callback - INFO - Received callback for node 69408514575bc060ce15da6a INFO: 172.21.0.1:52652 - "POST /node/69408514575bc060ce15da6a HTTP/1.0" 202 Accepted 2025-12-17 22:01:04,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408482575bc060ce15d7e7 2025-12-17 22:01:04,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408482575bc060ce15d7e7 HTTP/1.1" 502 150 2025-12-17 22:01:04,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408482575bc060ce15d7e7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:05,406 - lava_callback - INFO - Received callback for node 69408515575bc060ce15da71 INFO: 172.21.0.1:52654 - "POST /node/69408515575bc060ce15da71 HTTP/1.0" 202 Accepted 2025-12-17 22:01:05,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408475575bc060ce15d787 2025-12-17 22:01:05,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408475575bc060ce15d787 HTTP/1.1" 502 150 2025-12-17 22:01:05,755 - lava_callback - ERROR - Error processing callback for node 69408475575bc060ce15d787: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408475575bc060ce15d787 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408475575bc060ce15d787 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408475575bc060ce15d787 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:05,757 - lava_callback - INFO - Starting async job submit for node 69408488575bc060ce15d802 2025-12-17 22:01:05,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:05,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408488575bc060ce15d802 HTTP/1.1" 502 150 2025-12-17 22:01:05,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408488575bc060ce15d802'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:05,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408488575bc060ce15d802 2025-12-17 22:01:05,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408488575bc060ce15d802 HTTP/1.1" 502 150 2025-12-17 22:01:05,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408488575bc060ce15d802'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:06,053 - lava_callback - INFO - Received callback for node 69408516575bc060ce15da74 INFO: 172.21.0.1:52660 - "POST /node/69408516575bc060ce15da74 HTTP/1.0" 202 Accepted 2025-12-17 22:01:06,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847c575bc060ce15d799 2025-12-17 22:01:06,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847c575bc060ce15d799 HTTP/1.1" 502 150 2025-12-17 22:01:06,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847c575bc060ce15d799'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52664 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:01:06,716 - lava_callback - INFO - Received callback for node 69408518575bc060ce15da77 INFO: 172.21.0.1:52666 - "POST /node/69408518575bc060ce15da77 HTTP/1.0" 202 Accepted 2025-12-17 22:01:06,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408485575bc060ce15d7f4 2025-12-17 22:01:06,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408485575bc060ce15d7f4 HTTP/1.1" 502 150 2025-12-17 22:01:06,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408485575bc060ce15d7f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:07,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408488575bc060ce15d802 2025-12-17 22:01:07,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408488575bc060ce15d802 HTTP/1.1" 502 150 2025-12-17 22:01:07,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408488575bc060ce15d802'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:07,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408476575bc060ce15d78a 2025-12-17 22:01:07,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408476575bc060ce15d78a HTTP/1.1" 502 150 2025-12-17 22:01:07,833 - lava_callback - ERROR - Error processing callback for node 69408476575bc060ce15d78a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408476575bc060ce15d78a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408476575bc060ce15d78a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408476575bc060ce15d78a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:07,835 - lava_callback - INFO - Starting async job submit for node 69408489575bc060ce15d805 2025-12-17 22:01:07,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:07,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408489575bc060ce15d805 HTTP/1.1" 502 150 2025-12-17 22:01:07,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408489575bc060ce15d805'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:07,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408489575bc060ce15d805 2025-12-17 22:01:07,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408489575bc060ce15d805 HTTP/1.1" 502 150 2025-12-17 22:01:07,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408489575bc060ce15d805'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:07,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408487575bc060ce15d7fb 2025-12-17 22:01:07,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408487575bc060ce15d7fb HTTP/1.1" 502 150 2025-12-17 22:01:07,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408487575bc060ce15d7fb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:07,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408477575bc060ce15d78d 2025-12-17 22:01:07,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408477575bc060ce15d78d HTTP/1.1" 502 150 2025-12-17 22:01:07,987 - lava_callback - ERROR - Error processing callback for node 69408477575bc060ce15d78d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408477575bc060ce15d78d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408477575bc060ce15d78d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408477575bc060ce15d78d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:07,989 - lava_callback - INFO - Starting async job submit for node 6940848b575bc060ce15d808 2025-12-17 22:01:07,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:08,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408479575bc060ce15d790 2025-12-17 22:01:08,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408479575bc060ce15d790 HTTP/1.1" 502 150 2025-12-17 22:01:08,596 - lava_callback - ERROR - Error processing callback for node 69408479575bc060ce15d790: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408479575bc060ce15d790 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408479575bc060ce15d790 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408479575bc060ce15d790 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:08,598 - lava_callback - INFO - Starting async job submit for node 6940848c575bc060ce15d80b 2025-12-17 22:01:08,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:08,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408466575bc060ce15d758 2025-12-17 22:01:08,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408466575bc060ce15d758 HTTP/1.1" 502 150 2025-12-17 22:01:08,723 - lava_callback - ERROR - Error processing callback for node 69408466575bc060ce15d758: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408466575bc060ce15d758 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408466575bc060ce15d758 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408466575bc060ce15d758 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:08,725 - lava_callback - INFO - Starting async job submit for node 6940848d575bc060ce15d80e 2025-12-17 22:01:08,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:08,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848d575bc060ce15d80e HTTP/1.1" 502 150 2025-12-17 22:01:08,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848d575bc060ce15d80e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:08,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848d575bc060ce15d80e 2025-12-17 22:01:08,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848d575bc060ce15d80e HTTP/1.1" 502 150 2025-12-17 22:01:08,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848d575bc060ce15d80e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:08,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847e575bc060ce15d79d 2025-12-17 22:01:08,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847e575bc060ce15d79d HTTP/1.1" 502 150 2025-12-17 22:01:08,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940847e575bc060ce15d79d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:09,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408489575bc060ce15d805 2025-12-17 22:01:09,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408489575bc060ce15d805 HTTP/1.1" 502 150 2025-12-17 22:01:09,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408489575bc060ce15d805'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:10,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846a575bc060ce15d760 HTTP/1.1" 502 150 2025-12-17 22:01:10,320 - lava_callback - ERROR - Error processing callback for node 6940846a575bc060ce15d760: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940846a575bc060ce15d760 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940846a575bc060ce15d760 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940846a575bc060ce15d760 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:10,322 - lava_callback - INFO - Starting async job submit for node 6940848e575bc060ce15d811 2025-12-17 22:01:10,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:10,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848e575bc060ce15d811 HTTP/1.1" 502 150 2025-12-17 22:01:10,329 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848e575bc060ce15d811'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:10,329 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848e575bc060ce15d811 2025-12-17 22:01:10,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848e575bc060ce15d811 HTTP/1.1" 502 150 2025-12-17 22:01:10,330 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848e575bc060ce15d811'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:10,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848d575bc060ce15d80e 2025-12-17 22:01:10,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848d575bc060ce15d80e HTTP/1.1" 502 150 2025-12-17 22:01:10,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848d575bc060ce15d80e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:11,209 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408480575bc060ce15d7c4 2025-12-17 22:01:11,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408480575bc060ce15d7c4 HTTP/1.1" 502 150 2025-12-17 22:01:11,210 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408480575bc060ce15d7c4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:11,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408488575bc060ce15d802 2025-12-17 22:01:11,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408488575bc060ce15d802 HTTP/1.1" 502 150 2025-12-17 22:01:11,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408488575bc060ce15d802'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:12,330 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848e575bc060ce15d811 2025-12-17 22:01:12,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848e575bc060ce15d811 HTTP/1.1" 502 150 2025-12-17 22:01:12,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848e575bc060ce15d811'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:12,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848c575bc060ce15d80b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:12,604 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("HTTPSConnection(host='staging.kernelci.org', port=9000): Failed to resolve 'staging.kernelci.org' ([Errno -5] No address associated with hostname)")': /latest/node/6940848c575bc060ce15d80b 2025-12-17 22:01:12,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2025-12-17 22:01:12,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408482575bc060ce15d7e2 2025-12-17 22:01:12,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408482575bc060ce15d7e2 HTTP/1.1" 502 150 2025-12-17 22:01:12,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408482575bc060ce15d7e2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:12,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848c575bc060ce15d80b HTTP/1.1" 502 150 2025-12-17 22:01:12,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848c575bc060ce15d80b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:12,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408482575bc060ce15d7e7 2025-12-17 22:01:12,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408482575bc060ce15d7e7 HTTP/1.1" 502 150 2025-12-17 22:01:12,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408482575bc060ce15d7e7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:13,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408489575bc060ce15d805 2025-12-17 22:01:13,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408489575bc060ce15d805 HTTP/1.1" 502 150 2025-12-17 22:01:13,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408489575bc060ce15d805'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:14,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940846e575bc060ce15d778 2025-12-17 22:01:14,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940846e575bc060ce15d778 HTTP/1.1" 502 150 2025-12-17 22:01:14,007 - lava_callback - ERROR - Error processing callback for node 6940846e575bc060ce15d778: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940846e575bc060ce15d778 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940846e575bc060ce15d778 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940846e575bc060ce15d778 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:14,010 - lava_callback - INFO - Starting async job submit for node 6940848f575bc060ce15d814 2025-12-17 22:01:14,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:14,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848f575bc060ce15d814 HTTP/1.1" 502 150 2025-12-17 22:01:14,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848f575bc060ce15d814'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:14,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848f575bc060ce15d814 2025-12-17 22:01:14,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848f575bc060ce15d814 HTTP/1.1" 502 150 2025-12-17 22:01:14,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848f575bc060ce15d814'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:14,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848c575bc060ce15d80b 2025-12-17 22:01:14,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848c575bc060ce15d80b HTTP/1.1" 502 150 2025-12-17 22:01:14,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848c575bc060ce15d80b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:14,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848d575bc060ce15d80e 2025-12-17 22:01:14,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848d575bc060ce15d80e HTTP/1.1" 502 150 2025-12-17 22:01:14,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848d575bc060ce15d80e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:14,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408485575bc060ce15d7f4 2025-12-17 22:01:14,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408485575bc060ce15d7f4 HTTP/1.1" 502 150 2025-12-17 22:01:14,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408485575bc060ce15d7f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36370 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:01:15,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408487575bc060ce15d7fb 2025-12-17 22:01:15,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408487575bc060ce15d7fb HTTP/1.1" 502 150 2025-12-17 22:01:15,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408487575bc060ce15d7fb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:15,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847a575bc060ce15d793 2025-12-17 22:01:15,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847a575bc060ce15d793 HTTP/1.1" 502 150 2025-12-17 22:01:15,959 - lava_callback - ERROR - Error processing callback for node 6940847a575bc060ce15d793: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940847a575bc060ce15d793 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940847a575bc060ce15d793 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940847a575bc060ce15d793 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:15,961 - lava_callback - INFO - Starting async job submit for node 69408491575bc060ce15d817 2025-12-17 22:01:15,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:15,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408491575bc060ce15d817 HTTP/1.1" 502 150 2025-12-17 22:01:15,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408491575bc060ce15d817'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:15,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408491575bc060ce15d817 2025-12-17 22:01:15,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408491575bc060ce15d817 HTTP/1.1" 502 150 2025-12-17 22:01:15,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408491575bc060ce15d817'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:16,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848f575bc060ce15d814 2025-12-17 22:01:16,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848f575bc060ce15d814 HTTP/1.1" 502 150 2025-12-17 22:01:16,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848f575bc060ce15d814'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:16,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848e575bc060ce15d811 2025-12-17 22:01:16,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848e575bc060ce15d811 HTTP/1.1" 502 150 2025-12-17 22:01:16,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848e575bc060ce15d811'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36386 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:01:17,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408491575bc060ce15d817 2025-12-17 22:01:17,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408491575bc060ce15d817 HTTP/1.1" 502 150 2025-12-17 22:01:17,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408491575bc060ce15d817'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:18,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848c575bc060ce15d80b 2025-12-17 22:01:18,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848c575bc060ce15d80b HTTP/1.1" 502 150 2025-12-17 22:01:18,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848c575bc060ce15d80b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:19,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847b575bc060ce15d796 2025-12-17 22:01:19,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847b575bc060ce15d796 HTTP/1.1" 502 150 2025-12-17 22:01:19,748 - lava_callback - ERROR - Error processing callback for node 6940847b575bc060ce15d796: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940847b575bc060ce15d796 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940847b575bc060ce15d796 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940847b575bc060ce15d796 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:19,750 - lava_callback - INFO - Starting async job submit for node 69408495575bc060ce15d82a 2025-12-17 22:01:19,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:19,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408495575bc060ce15d82a HTTP/1.1" 502 150 2025-12-17 22:01:19,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408495575bc060ce15d82a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:19,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408495575bc060ce15d82a 2025-12-17 22:01:19,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408495575bc060ce15d82a HTTP/1.1" 502 150 2025-12-17 22:01:19,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408495575bc060ce15d82a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:19,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408488575bc060ce15d802 2025-12-17 22:01:19,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408488575bc060ce15d802 HTTP/1.1" 502 150 2025-12-17 22:01:19,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408488575bc060ce15d802'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:20,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848f575bc060ce15d814 2025-12-17 22:01:20,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848f575bc060ce15d814 HTTP/1.1" 502 150 2025-12-17 22:01:20,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848f575bc060ce15d814'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:21,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408495575bc060ce15d82a 2025-12-17 22:01:21,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408489575bc060ce15d805 2025-12-17 22:01:21,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408489575bc060ce15d805 HTTP/1.1" 502 150 2025-12-17 22:01:21,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408489575bc060ce15d805'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:21,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408491575bc060ce15d817 2025-12-17 22:01:21,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408491575bc060ce15d817 HTTP/1.1" 502 150 2025-12-17 22:01:21,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408491575bc060ce15d817'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:22,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847c575bc060ce15d799 2025-12-17 22:01:22,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847c575bc060ce15d799 HTTP/1.1" 502 150 2025-12-17 22:01:22,404 - lava_callback - ERROR - Error processing callback for node 6940847c575bc060ce15d799: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940847c575bc060ce15d799 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940847c575bc060ce15d799 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940847c575bc060ce15d799 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:22,405 - lava_callback - INFO - Starting async job submit for node 69408497575bc060ce15d831 2025-12-17 22:01:22,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:22,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408497575bc060ce15d831 HTTP/1.1" 502 150 2025-12-17 22:01:22,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408497575bc060ce15d831'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:22,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408497575bc060ce15d831 2025-12-17 22:01:22,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408497575bc060ce15d831 HTTP/1.1" 502 150 2025-12-17 22:01:22,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408497575bc060ce15d831'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:22,726 - lava_callback - INFO - Received callback for node 69408e7d575bc060ce15fe3a INFO: 172.21.0.1:39434 - "POST /node/69408e7d575bc060ce15fe3a HTTP/1.0" 202 Accepted 2025-12-17 22:01:22,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848d575bc060ce15d80e 2025-12-17 22:01:22,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848d575bc060ce15d80e HTTP/1.1" 502 150 2025-12-17 22:01:22,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848d575bc060ce15d80e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:23,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848b575bc060ce15d808 HTTP/1.1" 502 150 2025-12-17 22:01:23,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848b575bc060ce15d808'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:23,376 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848b575bc060ce15d808 2025-12-17 22:01:23,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848b575bc060ce15d808 HTTP/1.1" 502 150 2025-12-17 22:01:23,377 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848b575bc060ce15d808'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:23,445 - lava_callback - INFO - Received callback for node 6940851e575bc060ce15da82 INFO: 172.21.0.1:39436 - "POST /node/6940851e575bc060ce15da82 HTTP/1.0" 202 Accepted 2025-12-17 22:01:24,108 - lava_callback - INFO - Received callback for node 69408523575bc060ce15da8b INFO: 172.21.0.1:39440 - "POST /node/69408523575bc060ce15da8b HTTP/1.0" 202 Accepted 2025-12-17 22:01:24,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848e575bc060ce15d811 2025-12-17 22:01:24,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848e575bc060ce15d811 HTTP/1.1" 502 150 2025-12-17 22:01:24,334 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848e575bc060ce15d811'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:24,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408497575bc060ce15d831 2025-12-17 22:01:24,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408497575bc060ce15d831 HTTP/1.1" 502 150 2025-12-17 22:01:24,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408497575bc060ce15d831'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:24,756 - lava_callback - INFO - Received callback for node 69408524575bc060ce15da90 INFO: 172.21.0.1:39448 - "POST /node/69408524575bc060ce15da90 HTTP/1.0" 202 Accepted 2025-12-17 22:01:24,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940847e575bc060ce15d79d 2025-12-17 22:01:24,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940847e575bc060ce15d79d HTTP/1.1" 502 150 2025-12-17 22:01:24,927 - lava_callback - ERROR - Error processing callback for node 6940847e575bc060ce15d79d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940847e575bc060ce15d79d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940847e575bc060ce15d79d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940847e575bc060ce15d79d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:24,928 - lava_callback - INFO - Starting async job submit for node 69408498575bc060ce15d834 2025-12-17 22:01:24,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:24,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408498575bc060ce15d834 HTTP/1.1" 502 150 2025-12-17 22:01:24,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408498575bc060ce15d834'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:24,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408498575bc060ce15d834 2025-12-17 22:01:24,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408498575bc060ce15d834 HTTP/1.1" 502 150 2025-12-17 22:01:24,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408498575bc060ce15d834'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:25,378 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848b575bc060ce15d808 2025-12-17 22:01:25,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848b575bc060ce15d808 HTTP/1.1" 502 150 2025-12-17 22:01:25,379 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848b575bc060ce15d808'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:25,416 - lava_callback - INFO - Received callback for node 69408525575bc060ce15da93 INFO: 172.21.0.1:39458 - "POST /node/69408525575bc060ce15da93 HTTP/1.0" 202 Accepted 2025-12-17 22:01:26,068 - lava_callback - INFO - Received callback for node 69408526575bc060ce15da96 INFO: 172.21.0.1:39466 - "POST /node/69408526575bc060ce15da96 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:39476 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:01:26,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848c575bc060ce15d80b 2025-12-17 22:01:26,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848c575bc060ce15d80b HTTP/1.1" 502 150 2025-12-17 22:01:26,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848c575bc060ce15d80b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:26,731 - lava_callback - INFO - Received callback for node 69408528575bc060ce15da9d INFO: 172.21.0.1:39484 - "POST /node/69408528575bc060ce15da9d HTTP/1.0" 202 Accepted 2025-12-17 22:01:26,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408498575bc060ce15d834 2025-12-17 22:01:26,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408498575bc060ce15d834 HTTP/1.1" 502 150 2025-12-17 22:01:26,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408498575bc060ce15d834'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:27,210 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408480575bc060ce15d7c4 2025-12-17 22:01:27,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408480575bc060ce15d7c4 HTTP/1.1" 502 150 2025-12-17 22:01:27,212 - lava_callback - ERROR - Error processing callback for node 69408480575bc060ce15d7c4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408480575bc060ce15d7c4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408480575bc060ce15d7c4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408480575bc060ce15d7c4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:27,214 - lava_callback - INFO - Starting async job submit for node 69408499575bc060ce15d837 2025-12-17 22:01:27,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:27,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408499575bc060ce15d837 HTTP/1.1" 502 150 2025-12-17 22:01:27,246 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408499575bc060ce15d837'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:27,246 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408499575bc060ce15d837 2025-12-17 22:01:27,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408499575bc060ce15d837 HTTP/1.1" 502 150 2025-12-17 22:01:27,247 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408499575bc060ce15d837'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:27,381 - lava_callback - INFO - Received callback for node 69408529575bc060ce15daa0 INFO: 172.21.0.1:39498 - "POST /node/69408529575bc060ce15daa0 HTTP/1.0" 202 Accepted 2025-12-17 22:01:28,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848f575bc060ce15d814 2025-12-17 22:01:28,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848f575bc060ce15d814 HTTP/1.1" 502 150 2025-12-17 22:01:28,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848f575bc060ce15d814'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:28,022 - lava_callback - INFO - Received callback for node 6940852a575bc060ce15daa3 INFO: 172.21.0.1:39502 - "POST /node/6940852a575bc060ce15daa3 HTTP/1.0" 202 Accepted 2025-12-17 22:01:28,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408497575bc060ce15d831 2025-12-17 22:01:28,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408497575bc060ce15d831 HTTP/1.1" 502 150 2025-12-17 22:01:28,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408497575bc060ce15d831'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:28,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408482575bc060ce15d7e2 2025-12-17 22:01:28,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408482575bc060ce15d7e2 HTTP/1.1" 502 150 2025-12-17 22:01:28,611 - lava_callback - ERROR - Error processing callback for node 69408482575bc060ce15d7e2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408482575bc060ce15d7e2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408482575bc060ce15d7e2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408482575bc060ce15d7e2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:28,613 - lava_callback - INFO - Starting async job submit for node 6940849a575bc060ce15d83a 2025-12-17 22:01:28,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:28,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940849a575bc060ce15d83a HTTP/1.1" 502 150 2025-12-17 22:01:28,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940849a575bc060ce15d83a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:28,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940849a575bc060ce15d83a 2025-12-17 22:01:28,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940849a575bc060ce15d83a HTTP/1.1" 502 150 2025-12-17 22:01:28,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940849a575bc060ce15d83a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:28,670 - lava_callback - INFO - Received callback for node 6940852b575bc060ce15daa6 INFO: 172.21.0.1:39516 - "POST /node/6940852b575bc060ce15daa6 HTTP/1.0" 202 Accepted 2025-12-17 22:01:28,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408482575bc060ce15d7e7 2025-12-17 22:01:28,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408482575bc060ce15d7e7 HTTP/1.1" 502 150 2025-12-17 22:01:28,824 - lava_callback - ERROR - Error processing callback for node 69408482575bc060ce15d7e7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408482575bc060ce15d7e7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408482575bc060ce15d7e7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408482575bc060ce15d7e7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:28,826 - lava_callback - INFO - Starting async job submit for node 69408269575bc060ce15cf6c 2025-12-17 22:01:29,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:29,247 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408499575bc060ce15d837 2025-12-17 22:01:29,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408269575bc060ce15cf6c HTTP/1.1" 502 150 2025-12-17 22:01:29,248 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408269575bc060ce15cf6c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:29,248 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408269575bc060ce15cf6c 2025-12-17 22:01:29,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408499575bc060ce15d837 HTTP/1.1" 502 150 2025-12-17 22:01:29,248 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408499575bc060ce15d837'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:29,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408269575bc060ce15cf6c HTTP/1.1" 502 150 2025-12-17 22:01:29,249 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408269575bc060ce15cf6c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:29,319 - lava_callback - INFO - Received callback for node 6940852d575bc060ce15daa9 INFO: 172.21.0.1:39530 - "POST /node/6940852d575bc060ce15daa9 HTTP/1.0" 202 Accepted 2025-12-17 22:01:29,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848b575bc060ce15d808 2025-12-17 22:01:29,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848b575bc060ce15d808 HTTP/1.1" 502 150 2025-12-17 22:01:29,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848b575bc060ce15d808'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:29,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408491575bc060ce15d817 2025-12-17 22:01:29,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408491575bc060ce15d817 HTTP/1.1" 502 150 2025-12-17 22:01:29,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408491575bc060ce15d817'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:29,979 - lava_callback - INFO - Received callback for node 6940852e575bc060ce15daac INFO: 172.21.0.1:47636 - "POST /node/6940852e575bc060ce15daac HTTP/1.0" 202 Accepted 2025-12-17 22:01:30,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940849a575bc060ce15d83a 2025-12-17 22:01:30,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940849a575bc060ce15d83a HTTP/1.1" 502 150 2025-12-17 22:01:30,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940849a575bc060ce15d83a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:30,644 - lava_callback - INFO - Received callback for node 69408532575bc060ce15dab2 INFO: 172.21.0.1:47648 - "POST /node/69408532575bc060ce15dab2 HTTP/1.0" 202 Accepted 2025-12-17 22:01:30,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408485575bc060ce15d7f4 2025-12-17 22:01:30,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408485575bc060ce15d7f4 HTTP/1.1" 502 150 2025-12-17 22:01:30,933 - lava_callback - ERROR - Error processing callback for node 69408485575bc060ce15d7f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408485575bc060ce15d7f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408485575bc060ce15d7f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408485575bc060ce15d7f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:30,936 - lava_callback - INFO - Starting async job submit for node 69408492575bc060ce15d81a 2025-12-17 22:01:30,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:30,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408498575bc060ce15d834 2025-12-17 22:01:30,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408498575bc060ce15d834 HTTP/1.1" 502 150 2025-12-17 22:01:30,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408498575bc060ce15d834'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:30,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408492575bc060ce15d81a HTTP/1.1" 502 150 2025-12-17 22:01:30,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408492575bc060ce15d81a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:30,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408492575bc060ce15d81a 2025-12-17 22:01:30,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408492575bc060ce15d81a HTTP/1.1" 502 150 2025-12-17 22:01:30,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408492575bc060ce15d81a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:31,249 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408269575bc060ce15cf6c 2025-12-17 22:01:31,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408269575bc060ce15cf6c HTTP/1.1" 502 150 2025-12-17 22:01:31,250 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408269575bc060ce15cf6c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:31,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408487575bc060ce15d7fb 2025-12-17 22:01:31,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408487575bc060ce15d7fb HTTP/1.1" 502 150 2025-12-17 22:01:31,858 - lava_callback - ERROR - Error processing callback for node 69408487575bc060ce15d7fb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408487575bc060ce15d7fb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408487575bc060ce15d7fb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408487575bc060ce15d7fb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:31,860 - lava_callback - INFO - Starting async job submit for node 69408494575bc060ce15d823 2025-12-17 22:01:31,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:31,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408494575bc060ce15d823 HTTP/1.1" 502 150 2025-12-17 22:01:31,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408494575bc060ce15d823'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:31,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408494575bc060ce15d823 2025-12-17 22:01:31,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408494575bc060ce15d823 HTTP/1.1" 502 150 2025-12-17 22:01:31,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408494575bc060ce15d823'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:32,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408492575bc060ce15d81a 2025-12-17 22:01:32,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408492575bc060ce15d81a HTTP/1.1" 502 150 2025-12-17 22:01:32,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408492575bc060ce15d81a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:33,249 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408499575bc060ce15d837 2025-12-17 22:01:33,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408499575bc060ce15d837 HTTP/1.1" 502 150 2025-12-17 22:01:33,250 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408499575bc060ce15d837'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:33,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408494575bc060ce15d823 2025-12-17 22:01:33,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408494575bc060ce15d823 HTTP/1.1" 502 150 2025-12-17 22:01:33,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408494575bc060ce15d823'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:34,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940849a575bc060ce15d83a 2025-12-17 22:01:35,250 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408269575bc060ce15cf6c 2025-12-17 22:01:35,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408488575bc060ce15d802 2025-12-17 22:01:35,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408488575bc060ce15d802 HTTP/1.1" 502 150 2025-12-17 22:01:35,774 - lava_callback - ERROR - Error processing callback for node 69408488575bc060ce15d802: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408488575bc060ce15d802 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408488575bc060ce15d802 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408488575bc060ce15d802 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:35,777 - lava_callback - INFO - Starting async job submit for node 694084a2575bc060ce15d843 2025-12-17 22:01:35,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:35,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a2575bc060ce15d843 HTTP/1.1" 502 150 2025-12-17 22:01:35,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a2575bc060ce15d843'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:35,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a2575bc060ce15d843 2025-12-17 22:01:35,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a2575bc060ce15d843 HTTP/1.1" 502 150 2025-12-17 22:01:35,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a2575bc060ce15d843'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:36,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408497575bc060ce15d831 2025-12-17 22:01:36,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408497575bc060ce15d831 HTTP/1.1" 502 150 2025-12-17 22:01:36,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408497575bc060ce15d831'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47652 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:01:36,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408492575bc060ce15d81a 2025-12-17 22:01:36,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408492575bc060ce15d81a HTTP/1.1" 502 150 2025-12-17 22:01:36,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408492575bc060ce15d81a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:37,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408495575bc060ce15d82a HTTP/1.1" 502 150 2025-12-17 22:01:37,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408495575bc060ce15d82a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:37,381 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848b575bc060ce15d808 2025-12-17 22:01:37,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848b575bc060ce15d808 HTTP/1.1" 502 150 2025-12-17 22:01:37,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940848b575bc060ce15d808'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:37,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a2575bc060ce15d843 2025-12-17 22:01:37,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a2575bc060ce15d843 HTTP/1.1" 502 150 2025-12-17 22:01:37,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a2575bc060ce15d843'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:37,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408489575bc060ce15d805 2025-12-17 22:01:37,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408489575bc060ce15d805 HTTP/1.1" 502 150 2025-12-17 22:01:37,849 - lava_callback - ERROR - Error processing callback for node 69408489575bc060ce15d805: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408489575bc060ce15d805 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408489575bc060ce15d805 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408489575bc060ce15d805 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:37,851 - lava_callback - INFO - Starting async job submit for node 694084a4575bc060ce15d846 2025-12-17 22:01:37,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:37,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a4575bc060ce15d846 HTTP/1.1" 502 150 2025-12-17 22:01:37,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a4575bc060ce15d846'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:37,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a4575bc060ce15d846 2025-12-17 22:01:37,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a4575bc060ce15d846 HTTP/1.1" 502 150 2025-12-17 22:01:37,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a4575bc060ce15d846'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:37,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408494575bc060ce15d823 2025-12-17 22:01:37,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408494575bc060ce15d823 HTTP/1.1" 502 150 2025-12-17 22:01:37,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408494575bc060ce15d823'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:38,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848d575bc060ce15d80e 2025-12-17 22:01:38,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848d575bc060ce15d80e HTTP/1.1" 502 150 2025-12-17 22:01:38,739 - lava_callback - ERROR - Error processing callback for node 6940848d575bc060ce15d80e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940848d575bc060ce15d80e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940848d575bc060ce15d80e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940848d575bc060ce15d80e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:38,741 - lava_callback - INFO - Starting async job submit for node 694084aa575bc060ce15d863 2025-12-17 22:01:38,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:38,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084aa575bc060ce15d863 HTTP/1.1" 502 150 2025-12-17 22:01:38,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084aa575bc060ce15d863'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:38,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084aa575bc060ce15d863 2025-12-17 22:01:38,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084aa575bc060ce15d863 HTTP/1.1" 502 150 2025-12-17 22:01:38,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084aa575bc060ce15d863'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:38,912 - lava_callback - INFO - Received callback for node 6940853e575bc060ce15db36 INFO: 172.21.0.1:47660 - "POST /node/6940853e575bc060ce15db36 HTTP/1.0" 202 Accepted 2025-12-17 22:01:38,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408498575bc060ce15d834 2025-12-17 22:01:38,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408498575bc060ce15d834 HTTP/1.1" 502 150 2025-12-17 22:01:38,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408498575bc060ce15d834'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:39,686 - lava_callback - INFO - Received callback for node 6940853f575bc060ce15db39 INFO: 172.21.0.1:47668 - "POST /node/6940853f575bc060ce15db39 HTTP/1.0" 202 Accepted 2025-12-17 22:01:39,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a4575bc060ce15d846 2025-12-17 22:01:39,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a4575bc060ce15d846 HTTP/1.1" 502 150 2025-12-17 22:01:39,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a4575bc060ce15d846'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:40,334 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848e575bc060ce15d811 2025-12-17 22:01:40,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848e575bc060ce15d811 HTTP/1.1" 502 150 2025-12-17 22:01:40,336 - lava_callback - ERROR - Error processing callback for node 6940848e575bc060ce15d811: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940848e575bc060ce15d811 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940848e575bc060ce15d811 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940848e575bc060ce15d811 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:40,338 - lava_callback - INFO - Starting async job submit for node 694084a0575bc060ce15d83d 2025-12-17 22:01:40,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:40,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a0575bc060ce15d83d HTTP/1.1" 502 150 2025-12-17 22:01:40,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a0575bc060ce15d83d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:40,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a0575bc060ce15d83d 2025-12-17 22:01:40,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a0575bc060ce15d83d HTTP/1.1" 502 150 2025-12-17 22:01:40,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a0575bc060ce15d83d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:40,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084aa575bc060ce15d863 2025-12-17 22:01:40,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084aa575bc060ce15d863 HTTP/1.1" 502 150 2025-12-17 22:01:40,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084aa575bc060ce15d863'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:41,201 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408495575bc060ce15d82a 2025-12-17 22:01:41,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408495575bc060ce15d82a HTTP/1.1" 502 150 2025-12-17 22:01:41,202 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408495575bc060ce15d82a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:41,250 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408499575bc060ce15d837 2025-12-17 22:01:41,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408499575bc060ce15d837 HTTP/1.1" 502 150 2025-12-17 22:01:41,251 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408499575bc060ce15d837'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:41,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a2575bc060ce15d843 2025-12-17 22:01:41,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a2575bc060ce15d843 HTTP/1.1" 502 150 2025-12-17 22:01:41,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a2575bc060ce15d843'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:42,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a0575bc060ce15d83d 2025-12-17 22:01:42,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a0575bc060ce15d83d HTTP/1.1" 502 150 2025-12-17 22:01:42,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a0575bc060ce15d83d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:42,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848c575bc060ce15d80b 2025-12-17 22:01:42,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848c575bc060ce15d80b HTTP/1.1" 502 150 2025-12-17 22:01:42,616 - lava_callback - ERROR - Error processing callback for node 6940848c575bc060ce15d80b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940848c575bc060ce15d80b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 1 more time] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940848c575bc060ce15d80b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940848c575bc060ce15d80b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:42,618 - lava_callback - INFO - Starting async job submit for node 694084a1575bc060ce15d840 2025-12-17 22:01:42,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:42,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a1575bc060ce15d840 HTTP/1.1" 502 150 2025-12-17 22:01:42,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a1575bc060ce15d840'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:42,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a1575bc060ce15d840 2025-12-17 22:01:42,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a1575bc060ce15d840 HTTP/1.1" 502 150 2025-12-17 22:01:42,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a1575bc060ce15d840'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:43,449 - lava_callback - INFO - Received callback for node 69408533575bc060ce15dab6 INFO: 172.21.0.1:58128 - "POST /node/69408533575bc060ce15dab6 HTTP/1.0" 202 Accepted 2025-12-17 22:01:43,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a4575bc060ce15d846 2025-12-17 22:01:43,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a4575bc060ce15d846 HTTP/1.1" 502 150 2025-12-17 22:01:43,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a4575bc060ce15d846'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:44,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848f575bc060ce15d814 2025-12-17 22:01:44,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848f575bc060ce15d814 HTTP/1.1" 502 150 2025-12-17 22:01:44,023 - lava_callback - ERROR - Error processing callback for node 6940848f575bc060ce15d814: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940848f575bc060ce15d814 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940848f575bc060ce15d814 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940848f575bc060ce15d814 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:44,025 - lava_callback - INFO - Starting async job submit for node 694084a5575bc060ce15d849 2025-12-17 22:01:44,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:44,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a5575bc060ce15d849 HTTP/1.1" 502 150 2025-12-17 22:01:44,055 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a5575bc060ce15d849'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:44,055 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a5575bc060ce15d849 2025-12-17 22:01:44,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a5575bc060ce15d849 HTTP/1.1" 502 150 2025-12-17 22:01:44,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a5575bc060ce15d849'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:44,094 - lava_callback - INFO - Received callback for node 69408534575bc060ce15dabd INFO: 172.21.0.1:58136 - "POST /node/69408534575bc060ce15dabd HTTP/1.0" 202 Accepted 2025-12-17 22:01:44,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a1575bc060ce15d840 2025-12-17 22:01:44,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a1575bc060ce15d840 HTTP/1.1" 502 150 2025-12-17 22:01:44,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a1575bc060ce15d840'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:44,735 - lava_callback - INFO - Received callback for node 69408536575bc060ce15dac5 INFO: 172.21.0.1:58148 - "POST /node/69408536575bc060ce15dac5 HTTP/1.0" 202 Accepted 2025-12-17 22:01:44,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084aa575bc060ce15d863 2025-12-17 22:01:44,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084aa575bc060ce15d863 HTTP/1.1" 502 150 2025-12-17 22:01:44,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084aa575bc060ce15d863'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:44,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408492575bc060ce15d81a 2025-12-17 22:01:44,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408492575bc060ce15d81a HTTP/1.1" 502 150 2025-12-17 22:01:44,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408492575bc060ce15d81a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:45,416 - lava_callback - INFO - Received callback for node 69408537575bc060ce15daca INFO: 172.21.0.1:58164 - "POST /node/69408537575bc060ce15daca HTTP/1.0" 202 Accepted 2025-12-17 22:01:45,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408494575bc060ce15d823 2025-12-17 22:01:45,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408494575bc060ce15d823 HTTP/1.1" 502 150 2025-12-17 22:01:45,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408494575bc060ce15d823'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:45,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408491575bc060ce15d817 2025-12-17 22:01:45,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408491575bc060ce15d817 HTTP/1.1" 502 150 2025-12-17 22:01:45,980 - lava_callback - ERROR - Error processing callback for node 69408491575bc060ce15d817: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408491575bc060ce15d817 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408491575bc060ce15d817 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408491575bc060ce15d817 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:45,982 - lava_callback - INFO - Starting async job submit for node 694084a6575bc060ce15d84e 2025-12-17 22:01:45,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:45,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a6575bc060ce15d84e HTTP/1.1" 502 150 2025-12-17 22:01:45,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a6575bc060ce15d84e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:45,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a6575bc060ce15d84e 2025-12-17 22:01:45,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a6575bc060ce15d84e HTTP/1.1" 502 150 2025-12-17 22:01:45,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a6575bc060ce15d84e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:46,056 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a5575bc060ce15d849 2025-12-17 22:01:46,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a5575bc060ce15d849 HTTP/1.1" 502 150 2025-12-17 22:01:46,057 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a5575bc060ce15d849'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:46,064 - lava_callback - INFO - Received callback for node 69408538575bc060ce15dad1 INFO: 172.21.0.1:58170 - "POST /node/69408538575bc060ce15dad1 HTTP/1.0" 202 Accepted 2025-12-17 22:01:46,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a0575bc060ce15d83d 2025-12-17 22:01:46,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a0575bc060ce15d83d HTTP/1.1" 502 150 2025-12-17 22:01:46,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a0575bc060ce15d83d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58178 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:01:47,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a6575bc060ce15d84e 2025-12-17 22:01:47,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a6575bc060ce15d84e HTTP/1.1" 502 150 2025-12-17 22:01:47,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a6575bc060ce15d84e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:48,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a1575bc060ce15d840 2025-12-17 22:01:48,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a1575bc060ce15d840 HTTP/1.1" 502 150 2025-12-17 22:01:48,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a1575bc060ce15d840'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:49,202 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408495575bc060ce15d82a 2025-12-17 22:01:49,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408495575bc060ce15d82a HTTP/1.1" 502 150 2025-12-17 22:01:49,203 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408495575bc060ce15d82a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:49,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a2575bc060ce15d843 2025-12-17 22:01:49,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a2575bc060ce15d843 HTTP/1.1" 502 150 2025-12-17 22:01:49,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a2575bc060ce15d843'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:50,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940849a575bc060ce15d83a HTTP/1.1" 502 150 2025-12-17 22:01:50,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940849a575bc060ce15d83a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:50,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a5575bc060ce15d849 2025-12-17 22:01:50,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a5575bc060ce15d849 HTTP/1.1" 502 150 2025-12-17 22:01:50,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a5575bc060ce15d849'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:50,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408269575bc060ce15cf6c HTTP/1.1" 502 150 2025-12-17 22:01:50,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408269575bc060ce15cf6c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:51,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a4575bc060ce15d846 2025-12-17 22:01:51,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a4575bc060ce15d846 HTTP/1.1" 502 150 2025-12-17 22:01:51,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a4575bc060ce15d846'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:51,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a6575bc060ce15d84e 2025-12-17 22:01:51,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a6575bc060ce15d84e HTTP/1.1" 502 150 2025-12-17 22:01:51,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a6575bc060ce15d84e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:52,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408497575bc060ce15d831 2025-12-17 22:01:52,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408497575bc060ce15d831 HTTP/1.1" 502 150 2025-12-17 22:01:52,417 - lava_callback - ERROR - Error processing callback for node 69408497575bc060ce15d831: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408497575bc060ce15d831 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408497575bc060ce15d831 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408497575bc060ce15d831 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:52,419 - lava_callback - INFO - Starting async job submit for node 694084a7575bc060ce15d855 2025-12-17 22:01:52,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:52,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a7575bc060ce15d855 HTTP/1.1" 502 150 2025-12-17 22:01:52,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a7575bc060ce15d855'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:52,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a7575bc060ce15d855 2025-12-17 22:01:52,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a7575bc060ce15d855 HTTP/1.1" 502 150 2025-12-17 22:01:52,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a7575bc060ce15d855'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:52,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084aa575bc060ce15d863 2025-12-17 22:01:52,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084aa575bc060ce15d863 HTTP/1.1" 502 150 2025-12-17 22:01:52,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084aa575bc060ce15d863'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:53,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940848b575bc060ce15d808 2025-12-17 22:01:53,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940848b575bc060ce15d808 HTTP/1.1" 502 150 2025-12-17 22:01:53,384 - lava_callback - ERROR - Error processing callback for node 6940848b575bc060ce15d808: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940848b575bc060ce15d808 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940848b575bc060ce15d808 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940848b575bc060ce15d808 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:53,386 - lava_callback - INFO - Starting async job submit for node 694084a9575bc060ce15d85c 2025-12-17 22:01:53,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:53,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a9575bc060ce15d85c HTTP/1.1" 502 150 2025-12-17 22:01:53,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a9575bc060ce15d85c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:53,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a9575bc060ce15d85c 2025-12-17 22:01:53,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a9575bc060ce15d85c HTTP/1.1" 502 150 2025-12-17 22:01:53,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a9575bc060ce15d85c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:53,684 - lava_callback - INFO - Received callback for node 69408541575bc060ce15db3c INFO: 172.21.0.1:50266 - "POST /node/69408541575bc060ce15db3c HTTP/1.0" 202 Accepted 2025-12-17 22:01:54,341 - lava_callback - INFO - Received callback for node 69408543575bc060ce15db3f INFO: 172.21.0.1:50276 - "POST /node/69408543575bc060ce15db3f HTTP/1.0" 202 Accepted 2025-12-17 22:01:54,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a0575bc060ce15d83d 2025-12-17 22:01:54,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a0575bc060ce15d83d HTTP/1.1" 502 150 2025-12-17 22:01:54,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a0575bc060ce15d83d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:54,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a7575bc060ce15d855 2025-12-17 22:01:54,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a7575bc060ce15d855 HTTP/1.1" 502 150 2025-12-17 22:01:54,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a7575bc060ce15d855'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:54,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408498575bc060ce15d834 2025-12-17 22:01:54,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408498575bc060ce15d834 HTTP/1.1" 502 150 2025-12-17 22:01:54,948 - lava_callback - ERROR - Error processing callback for node 69408498575bc060ce15d834: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408498575bc060ce15d834 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408498575bc060ce15d834 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408498575bc060ce15d834 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:54,950 - lava_callback - INFO - Starting async job submit for node 694084ac575bc060ce15d86a 2025-12-17 22:01:54,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:54,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ac575bc060ce15d86a HTTP/1.1" 502 150 2025-12-17 22:01:54,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ac575bc060ce15d86a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:54,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ac575bc060ce15d86a 2025-12-17 22:01:54,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ac575bc060ce15d86a HTTP/1.1" 502 150 2025-12-17 22:01:54,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ac575bc060ce15d86a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:54,988 - lava_callback - INFO - Received callback for node 69408544575bc060ce15db42 INFO: 172.21.0.1:50280 - "POST /node/69408544575bc060ce15db42 HTTP/1.0" 202 Accepted 2025-12-17 22:01:55,395 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a9575bc060ce15d85c 2025-12-17 22:01:55,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a9575bc060ce15d85c HTTP/1.1" 502 150 2025-12-17 22:01:55,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a9575bc060ce15d85c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:55,649 - lava_callback - INFO - Received callback for node 69408546575bc060ce15db45 INFO: 172.21.0.1:50290 - "POST /node/69408546575bc060ce15db45 HTTP/1.0" 202 Accepted 2025-12-17 22:01:56,294 - lava_callback - INFO - Received callback for node 69408547575bc060ce15db4b INFO: 172.21.0.1:50294 - "POST /node/69408547575bc060ce15db4b HTTP/1.0" 202 Accepted INFO: 172.21.0.1:50302 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:01:56,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a1575bc060ce15d840 2025-12-17 22:01:56,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a1575bc060ce15d840 HTTP/1.1" 502 150 2025-12-17 22:01:56,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a1575bc060ce15d840'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:56,952 - lava_callback - INFO - Received callback for node 69408548575bc060ce15db55 2025-12-17 22:01:56,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ac575bc060ce15d86a INFO: 172.21.0.1:50310 - "POST /node/69408548575bc060ce15db55 HTTP/1.0" 202 Accepted 2025-12-17 22:01:56,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ac575bc060ce15d86a HTTP/1.1" 502 150 2025-12-17 22:01:56,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ac575bc060ce15d86a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:57,252 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408499575bc060ce15d837 2025-12-17 22:01:57,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408499575bc060ce15d837 HTTP/1.1" 502 150 2025-12-17 22:01:57,253 - lava_callback - ERROR - Error processing callback for node 69408499575bc060ce15d837: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408499575bc060ce15d837 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408499575bc060ce15d837 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408499575bc060ce15d837 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:01:57,256 - lava_callback - INFO - Starting async job submit for node 694084af575bc060ce15d878 2025-12-17 22:01:57,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:01:57,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084af575bc060ce15d878 HTTP/1.1" 502 150 2025-12-17 22:01:57,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084af575bc060ce15d878'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:57,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084af575bc060ce15d878 2025-12-17 22:01:57,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084af575bc060ce15d878 HTTP/1.1" 502 150 2025-12-17 22:01:57,273 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084af575bc060ce15d878'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:57,639 - lava_callback - INFO - Received callback for node 6940854a575bc060ce15db5c INFO: 172.21.0.1:50316 - "POST /node/6940854a575bc060ce15db5c HTTP/1.0" 202 Accepted 2025-12-17 22:01:58,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940849a575bc060ce15d83a 2025-12-17 22:01:58,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940849a575bc060ce15d83a HTTP/1.1" 502 150 2025-12-17 22:01:58,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940849a575bc060ce15d83a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:58,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a5575bc060ce15d849 2025-12-17 22:01:58,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a5575bc060ce15d849 HTTP/1.1" 502 150 2025-12-17 22:01:58,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a5575bc060ce15d849'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:58,301 - lava_callback - INFO - Received callback for node 6940854b575bc060ce15db5f INFO: 172.21.0.1:50324 - "POST /node/6940854b575bc060ce15db5f HTTP/1.0" 202 Accepted 2025-12-17 22:01:58,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a7575bc060ce15d855 2025-12-17 22:01:58,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a7575bc060ce15d855 HTTP/1.1" 502 150 2025-12-17 22:01:58,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a7575bc060ce15d855'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:58,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408269575bc060ce15cf6c 2025-12-17 22:01:58,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408269575bc060ce15cf6c HTTP/1.1" 502 150 2025-12-17 22:01:58,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408269575bc060ce15cf6c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:58,813 - lava_callback - INFO - Received callback for node 6940854e575bc060ce15db65 INFO: 172.21.0.1:50338 - "POST /node/6940854e575bc060ce15db65 HTTP/1.0" 202 Accepted 2025-12-17 22:01:59,273 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084af575bc060ce15d878 2025-12-17 22:01:59,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084af575bc060ce15d878 HTTP/1.1" 502 150 2025-12-17 22:01:59,274 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084af575bc060ce15d878'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:59,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a9575bc060ce15d85c 2025-12-17 22:01:59,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a9575bc060ce15d85c HTTP/1.1" 502 150 2025-12-17 22:01:59,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a9575bc060ce15d85c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:01:59,757 - lava_callback - INFO - Received callback for node 6940854f575bc060ce15db68 INFO: 172.21.0.1:55520 - "POST /node/6940854f575bc060ce15db68 HTTP/1.0" 202 Accepted 2025-12-17 22:01:59,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a6575bc060ce15d84e 2025-12-17 22:02:00,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a6575bc060ce15d84e HTTP/1.1" 502 150 2025-12-17 22:02:00,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a6575bc060ce15d84e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:00,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ac575bc060ce15d86a 2025-12-17 22:02:00,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ac575bc060ce15d86a HTTP/1.1" 502 150 2025-12-17 22:02:00,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ac575bc060ce15d86a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:00,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408492575bc060ce15d81a 2025-12-17 22:02:00,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408492575bc060ce15d81a HTTP/1.1" 502 150 2025-12-17 22:02:00,972 - lava_callback - ERROR - Error processing callback for node 69408492575bc060ce15d81a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408492575bc060ce15d81a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408492575bc060ce15d81a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408492575bc060ce15d81a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:00,974 - lava_callback - INFO - Starting async job submit for node 694084b0575bc060ce15d87f 2025-12-17 22:02:00,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:00,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b0575bc060ce15d87f HTTP/1.1" 502 150 2025-12-17 22:02:00,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b0575bc060ce15d87f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:00,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b0575bc060ce15d87f 2025-12-17 22:02:00,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b0575bc060ce15d87f HTTP/1.1" 502 150 2025-12-17 22:02:00,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b0575bc060ce15d87f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:01,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408494575bc060ce15d823 2025-12-17 22:02:01,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408494575bc060ce15d823 HTTP/1.1" 502 150 2025-12-17 22:02:01,879 - lava_callback - ERROR - Error processing callback for node 69408494575bc060ce15d823: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408494575bc060ce15d823 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408494575bc060ce15d823 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408494575bc060ce15d823 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:01,882 - lava_callback - INFO - Starting async job submit for node 694084b4575bc060ce15d88f 2025-12-17 22:02:01,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:01,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b4575bc060ce15d88f HTTP/1.1" 502 150 2025-12-17 22:02:01,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b4575bc060ce15d88f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:01,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b4575bc060ce15d88f 2025-12-17 22:02:01,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b4575bc060ce15d88f HTTP/1.1" 502 150 2025-12-17 22:02:01,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b4575bc060ce15d88f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55532 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:02:02,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b0575bc060ce15d87f 2025-12-17 22:02:02,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b0575bc060ce15d87f HTTP/1.1" 502 150 2025-12-17 22:02:02,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b0575bc060ce15d87f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:03,275 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084af575bc060ce15d878 2025-12-17 22:02:03,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084af575bc060ce15d878 HTTP/1.1" 502 150 2025-12-17 22:02:03,276 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084af575bc060ce15d878'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:03,454 - lava_callback - INFO - Received callback for node 6940854c575bc060ce15db62 INFO: 172.21.0.1:55544 - "POST /node/6940854c575bc060ce15db62 HTTP/1.0" 202 Accepted 2025-12-17 22:02:03,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b4575bc060ce15d88f 2025-12-17 22:02:03,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b4575bc060ce15d88f HTTP/1.1" 502 150 2025-12-17 22:02:03,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b4575bc060ce15d88f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:05,203 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408495575bc060ce15d82a 2025-12-17 22:02:05,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408495575bc060ce15d82a HTTP/1.1" 502 150 2025-12-17 22:02:05,205 - lava_callback - ERROR - Error processing callback for node 69408495575bc060ce15d82a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408495575bc060ce15d82a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408495575bc060ce15d82a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408495575bc060ce15d82a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:05,207 - lava_callback - INFO - Starting async job submit for node 694084b5575bc060ce15d894 2025-12-17 22:02:05,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:05,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b5575bc060ce15d894 HTTP/1.1" 502 150 2025-12-17 22:02:05,214 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b5575bc060ce15d894'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:05,214 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b5575bc060ce15d894 2025-12-17 22:02:05,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b5575bc060ce15d894 HTTP/1.1" 502 150 2025-12-17 22:02:05,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b5575bc060ce15d894'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:05,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a2575bc060ce15d843 2025-12-17 22:02:05,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a2575bc060ce15d843 HTTP/1.1" 502 150 2025-12-17 22:02:05,792 - lava_callback - ERROR - Error processing callback for node 694084a2575bc060ce15d843: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a2575bc060ce15d843 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a2575bc060ce15d843 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a2575bc060ce15d843 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:05,794 - lava_callback - INFO - Starting async job submit for node 694084b7575bc060ce15d897 2025-12-17 22:02:05,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:05,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b7575bc060ce15d897 HTTP/1.1" 502 150 2025-12-17 22:02:05,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b7575bc060ce15d897'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:05,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b7575bc060ce15d897 2025-12-17 22:02:05,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b7575bc060ce15d897 HTTP/1.1" 502 150 2025-12-17 22:02:05,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b7575bc060ce15d897'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:06,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a7575bc060ce15d855 2025-12-17 22:02:06,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a7575bc060ce15d855 HTTP/1.1" 502 150 2025-12-17 22:02:06,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a7575bc060ce15d855'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55556 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:02:06,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b0575bc060ce15d87f 2025-12-17 22:02:06,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b0575bc060ce15d87f HTTP/1.1" 502 150 2025-12-17 22:02:06,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b0575bc060ce15d87f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:07,215 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b5575bc060ce15d894 2025-12-17 22:02:07,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b5575bc060ce15d894 HTTP/1.1" 502 150 2025-12-17 22:02:07,216 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b5575bc060ce15d894'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:07,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a9575bc060ce15d85c 2025-12-17 22:02:07,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a9575bc060ce15d85c HTTP/1.1" 502 150 2025-12-17 22:02:07,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084a9575bc060ce15d85c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:07,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b7575bc060ce15d897 2025-12-17 22:02:07,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b7575bc060ce15d897 HTTP/1.1" 502 150 2025-12-17 22:02:07,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b7575bc060ce15d897'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:07,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a4575bc060ce15d846 2025-12-17 22:02:07,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a4575bc060ce15d846 HTTP/1.1" 502 150 2025-12-17 22:02:07,864 - lava_callback - ERROR - Error processing callback for node 694084a4575bc060ce15d846: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a4575bc060ce15d846 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a4575bc060ce15d846 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a4575bc060ce15d846 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:07,866 - lava_callback - INFO - Starting async job submit for node 694084b8575bc060ce15d89a 2025-12-17 22:02:07,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:07,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b8575bc060ce15d89a HTTP/1.1" 502 150 2025-12-17 22:02:07,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b8575bc060ce15d89a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:07,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b8575bc060ce15d89a 2025-12-17 22:02:07,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b8575bc060ce15d89a HTTP/1.1" 502 150 2025-12-17 22:02:07,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b8575bc060ce15d89a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:07,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b4575bc060ce15d88f 2025-12-17 22:02:07,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b4575bc060ce15d88f HTTP/1.1" 502 150 2025-12-17 22:02:07,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b4575bc060ce15d88f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:08,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084aa575bc060ce15d863 2025-12-17 22:02:08,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084aa575bc060ce15d863 HTTP/1.1" 502 150 2025-12-17 22:02:08,755 - lava_callback - ERROR - Error processing callback for node 694084aa575bc060ce15d863: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084aa575bc060ce15d863 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084aa575bc060ce15d863 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084aa575bc060ce15d863 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:08,756 - lava_callback - INFO - Starting async job submit for node 694084b9575bc060ce15d89d 2025-12-17 22:02:08,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:08,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b9575bc060ce15d89d HTTP/1.1" 502 150 2025-12-17 22:02:08,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b9575bc060ce15d89d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:08,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b9575bc060ce15d89d 2025-12-17 22:02:08,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b9575bc060ce15d89d HTTP/1.1" 502 150 2025-12-17 22:02:08,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b9575bc060ce15d89d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:08,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ac575bc060ce15d86a 2025-12-17 22:02:08,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ac575bc060ce15d86a HTTP/1.1" 502 150 2025-12-17 22:02:08,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ac575bc060ce15d86a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:09,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b8575bc060ce15d89a 2025-12-17 22:02:09,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b8575bc060ce15d89a HTTP/1.1" 502 150 2025-12-17 22:02:09,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b8575bc060ce15d89a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:10,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a0575bc060ce15d83d 2025-12-17 22:02:10,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a0575bc060ce15d83d HTTP/1.1" 502 150 2025-12-17 22:02:10,351 - lava_callback - ERROR - Error processing callback for node 694084a0575bc060ce15d83d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a0575bc060ce15d83d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a0575bc060ce15d83d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a0575bc060ce15d83d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:10,353 - lava_callback - INFO - Starting async job submit for node 694084bb575bc060ce15d8a0 2025-12-17 22:02:10,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:10,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084bb575bc060ce15d8a0 HTTP/1.1" 502 150 2025-12-17 22:02:10,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084bb575bc060ce15d8a0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:10,359 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084bb575bc060ce15d8a0 2025-12-17 22:02:10,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084bb575bc060ce15d8a0 HTTP/1.1" 502 150 2025-12-17 22:02:10,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084bb575bc060ce15d8a0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:10,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b9575bc060ce15d89d 2025-12-17 22:02:10,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b9575bc060ce15d89d HTTP/1.1" 502 150 2025-12-17 22:02:10,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b9575bc060ce15d89d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:11,216 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b5575bc060ce15d894 2025-12-17 22:02:11,276 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084af575bc060ce15d878 2025-12-17 22:02:11,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084af575bc060ce15d878 HTTP/1.1" 502 150 2025-12-17 22:02:11,277 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084af575bc060ce15d878'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:11,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b7575bc060ce15d897 2025-12-17 22:02:11,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b7575bc060ce15d897 HTTP/1.1" 502 150 2025-12-17 22:02:11,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b7575bc060ce15d897'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:12,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084bb575bc060ce15d8a0 2025-12-17 22:02:12,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084bb575bc060ce15d8a0 HTTP/1.1" 502 150 2025-12-17 22:02:12,361 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084bb575bc060ce15d8a0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:12,498 - lava_callback - INFO - Received callback for node 6940855f575bc060ce15db92 INFO: 172.21.0.1:39942 - "POST /node/6940855f575bc060ce15db92 HTTP/1.0" 202 Accepted 2025-12-17 22:02:12,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a1575bc060ce15d840 2025-12-17 22:02:12,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a1575bc060ce15d840 HTTP/1.1" 502 150 2025-12-17 22:02:12,644 - lava_callback - ERROR - Error processing callback for node 694084a1575bc060ce15d840: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a1575bc060ce15d840 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a1575bc060ce15d840 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a1575bc060ce15d840 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:12,646 - lava_callback - INFO - Starting async job submit for node 694084bc575bc060ce15d8a7 2025-12-17 22:02:12,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:12,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084bc575bc060ce15d8a7 HTTP/1.1" 502 150 2025-12-17 22:02:12,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084bc575bc060ce15d8a7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:12,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084bc575bc060ce15d8a7 2025-12-17 22:02:12,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084bc575bc060ce15d8a7 HTTP/1.1" 502 150 2025-12-17 22:02:12,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084bc575bc060ce15d8a7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:13,107 - lava_callback - INFO - Received callback for node 69408561575bc060ce15db95 INFO: 172.21.0.1:39952 - "POST /node/69408561575bc060ce15db95 HTTP/1.0" 202 Accepted 2025-12-17 22:02:13,658 - lava_callback - INFO - Received callback for node 69408551575bc060ce15db6b INFO: 172.21.0.1:39958 - "POST /node/69408551575bc060ce15db6b HTTP/1.0" 202 Accepted 2025-12-17 22:02:13,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b8575bc060ce15d89a 2025-12-17 22:02:13,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b8575bc060ce15d89a HTTP/1.1" 502 150 2025-12-17 22:02:13,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b8575bc060ce15d89a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:14,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940849a575bc060ce15d83a 2025-12-17 22:02:14,060 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a5575bc060ce15d849 2025-12-17 22:02:14,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a5575bc060ce15d849 HTTP/1.1" 502 150 2025-12-17 22:02:14,061 - lava_callback - ERROR - Error processing callback for node 694084a5575bc060ce15d849: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a5575bc060ce15d849 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a5575bc060ce15d849 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a5575bc060ce15d849 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:14,063 - lava_callback - INFO - Starting async job submit for node 694084c2575bc060ce15d8f6 2025-12-17 22:02:14,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:14,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c2575bc060ce15d8f6 HTTP/1.1" 502 150 2025-12-17 22:02:14,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c2575bc060ce15d8f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:14,071 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c2575bc060ce15d8f6 2025-12-17 22:02:14,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c2575bc060ce15d8f6 HTTP/1.1" 502 150 2025-12-17 22:02:14,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c2575bc060ce15d8f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:14,338 - lava_callback - INFO - Received callback for node 69408552575bc060ce15db6e INFO: 172.21.0.1:39966 - "POST /node/69408552575bc060ce15db6e HTTP/1.0" 202 Accepted 2025-12-17 22:02:14,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408269575bc060ce15cf6c 2025-12-17 22:02:14,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408269575bc060ce15cf6c HTTP/1.1" 502 150 2025-12-17 22:02:14,515 - lava_callback - ERROR - Error processing callback for node 69408269575bc060ce15cf6c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408269575bc060ce15cf6c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408269575bc060ce15cf6c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408269575bc060ce15cf6c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:14,517 - lava_callback - INFO - Starting async job submit for node 694084be575bc060ce15d8df 2025-12-17 22:02:14,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:14,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084be575bc060ce15d8df HTTP/1.1" 502 150 2025-12-17 22:02:14,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084be575bc060ce15d8df'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:14,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084be575bc060ce15d8df 2025-12-17 22:02:14,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084be575bc060ce15d8df HTTP/1.1" 502 150 2025-12-17 22:02:14,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084be575bc060ce15d8df'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:14,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084bc575bc060ce15d8a7 2025-12-17 22:02:14,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084bc575bc060ce15d8a7 HTTP/1.1" 502 150 2025-12-17 22:02:14,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084bc575bc060ce15d8a7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:14,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b9575bc060ce15d89d 2025-12-17 22:02:14,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b9575bc060ce15d89d HTTP/1.1" 502 150 2025-12-17 22:02:14,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b9575bc060ce15d89d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:14,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b0575bc060ce15d87f 2025-12-17 22:02:14,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b0575bc060ce15d87f HTTP/1.1" 502 150 2025-12-17 22:02:14,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b0575bc060ce15d87f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:15,015 - lava_callback - INFO - Received callback for node 69408554575bc060ce15db71 INFO: 172.21.0.1:39974 - "POST /node/69408554575bc060ce15db71 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:39980 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:02:15,682 - lava_callback - INFO - Received callback for node 69408555575bc060ce15db74 INFO: 172.21.0.1:39982 - "POST /node/69408555575bc060ce15db74 HTTP/1.0" 202 Accepted 2025-12-17 22:02:15,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b4575bc060ce15d88f 2025-12-17 22:02:15,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b4575bc060ce15d88f HTTP/1.1" 502 150 2025-12-17 22:02:15,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b4575bc060ce15d88f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:16,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a6575bc060ce15d84e 2025-12-17 22:02:16,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a6575bc060ce15d84e HTTP/1.1" 502 150 2025-12-17 22:02:16,002 - lava_callback - ERROR - Error processing callback for node 694084a6575bc060ce15d84e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a6575bc060ce15d84e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a6575bc060ce15d84e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a6575bc060ce15d84e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:16,004 - lava_callback - INFO - Starting async job submit for node 694084c0575bc060ce15d8ec 2025-12-17 22:02:16,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:16,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c0575bc060ce15d8ec HTTP/1.1" 502 150 2025-12-17 22:02:16,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c0575bc060ce15d8ec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:16,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c0575bc060ce15d8ec 2025-12-17 22:02:16,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c0575bc060ce15d8ec HTTP/1.1" 502 150 2025-12-17 22:02:16,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c0575bc060ce15d8ec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:16,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c2575bc060ce15d8f6 2025-12-17 22:02:16,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c2575bc060ce15d8f6 HTTP/1.1" 502 150 2025-12-17 22:02:16,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c2575bc060ce15d8f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:16,355 - lava_callback - INFO - Received callback for node 69408556575bc060ce15db77 INFO: 172.21.0.1:39996 - "POST /node/69408556575bc060ce15db77 HTTP/1.0" 202 Accepted 2025-12-17 22:02:16,361 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084bb575bc060ce15d8a0 2025-12-17 22:02:16,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084bb575bc060ce15d8a0 HTTP/1.1" 502 150 2025-12-17 22:02:16,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084bb575bc060ce15d8a0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40006 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:02:16,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084be575bc060ce15d8df 2025-12-17 22:02:16,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084be575bc060ce15d8df HTTP/1.1" 502 150 2025-12-17 22:02:16,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084be575bc060ce15d8df'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:17,040 - lava_callback - INFO - Received callback for node 69408557575bc060ce15db7a INFO: 172.21.0.1:40020 - "POST /node/69408557575bc060ce15db7a HTTP/1.0" 202 Accepted 2025-12-17 22:02:17,696 - lava_callback - INFO - Received callback for node 69408559575bc060ce15db7d INFO: 172.21.0.1:40034 - "POST /node/69408559575bc060ce15db7d HTTP/1.0" 202 Accepted 2025-12-17 22:02:18,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c0575bc060ce15d8ec 2025-12-17 22:02:18,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c0575bc060ce15d8ec HTTP/1.1" 502 150 2025-12-17 22:02:18,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c0575bc060ce15d8ec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:18,394 - lava_callback - INFO - Received callback for node 6940855b575bc060ce15db87 INFO: 172.21.0.1:40036 - "POST /node/6940855b575bc060ce15db87 HTTP/1.0" 202 Accepted 2025-12-17 22:02:18,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084bc575bc060ce15d8a7 2025-12-17 22:02:18,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084bc575bc060ce15d8a7 HTTP/1.1" 502 150 2025-12-17 22:02:18,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084bc575bc060ce15d8a7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:19,032 - lava_callback - INFO - Received callback for node 6940855a575bc060ce15db82 INFO: 172.21.0.1:40038 - "POST /node/6940855a575bc060ce15db82 HTTP/1.0" 202 Accepted 2025-12-17 22:02:19,076 - lava_callback - INFO - Received callback for node 6940855c575bc060ce15db8a INFO: 172.21.0.1:40044 - "POST /node/6940855c575bc060ce15db8a HTTP/1.0" 202 Accepted 2025-12-17 22:02:19,768 - lava_callback - INFO - Received callback for node 6940855e575bc060ce15db8f INFO: 172.21.0.1:45730 - "POST /node/6940855e575bc060ce15db8f HTTP/1.0" 202 Accepted 2025-12-17 22:02:19,775 - lava_callback - INFO - Received callback for node 69408569575bc060ce15dba4 INFO: 172.21.0.1:45736 - "POST /node/69408569575bc060ce15dba4 HTTP/1.0" 202 Accepted 2025-12-17 22:02:19,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b7575bc060ce15d897 2025-12-17 22:02:19,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b7575bc060ce15d897 HTTP/1.1" 502 150 2025-12-17 22:02:19,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b7575bc060ce15d897'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:20,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c2575bc060ce15d8f6 2025-12-17 22:02:20,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c2575bc060ce15d8f6 HTTP/1.1" 502 150 2025-12-17 22:02:20,074 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c2575bc060ce15d8f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:20,515 - lava_callback - INFO - Received callback for node 69408569575bc060ce15dba7 INFO: 172.21.0.1:45740 - "POST /node/69408569575bc060ce15dba7 HTTP/1.0" 202 Accepted 2025-12-17 22:02:20,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084be575bc060ce15d8df 2025-12-17 22:02:20,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084be575bc060ce15d8df HTTP/1.1" 502 150 2025-12-17 22:02:20,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084be575bc060ce15d8df'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:21,266 - lava_callback - INFO - Received callback for node 6940856a575bc060ce15dbaa INFO: 172.21.0.1:45750 - "POST /node/6940856a575bc060ce15dbaa HTTP/1.0" 202 Accepted 2025-12-17 22:02:21,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b8575bc060ce15d89a 2025-12-17 22:02:21,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b8575bc060ce15d89a HTTP/1.1" 502 150 2025-12-17 22:02:21,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b8575bc060ce15d89a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:22,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c0575bc060ce15d8ec 2025-12-17 22:02:22,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c0575bc060ce15d8ec HTTP/1.1" 502 150 2025-12-17 22:02:22,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c0575bc060ce15d8ec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:22,039 - lava_callback - INFO - Received callback for node 6940856c575bc060ce15dbad INFO: 172.21.0.1:45760 - "POST /node/6940856c575bc060ce15dbad HTTP/1.0" 202 Accepted 2025-12-17 22:02:22,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a7575bc060ce15d855 2025-12-17 22:02:22,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a7575bc060ce15d855 HTTP/1.1" 502 150 2025-12-17 22:02:22,449 - lava_callback - ERROR - Error processing callback for node 694084a7575bc060ce15d855: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a7575bc060ce15d855 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a7575bc060ce15d855 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a7575bc060ce15d855 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:22,451 - lava_callback - INFO - Starting async job submit for node 694084c1575bc060ce15d8ef 2025-12-17 22:02:22,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:22,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c1575bc060ce15d8ef HTTP/1.1" 502 150 2025-12-17 22:02:22,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c1575bc060ce15d8ef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:22,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c1575bc060ce15d8ef 2025-12-17 22:02:22,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c1575bc060ce15d8ef HTTP/1.1" 502 150 2025-12-17 22:02:22,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c1575bc060ce15d8ef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:22,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b9575bc060ce15d89d 2025-12-17 22:02:22,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b9575bc060ce15d89d HTTP/1.1" 502 150 2025-12-17 22:02:22,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b9575bc060ce15d89d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:22,802 - lava_callback - INFO - Received callback for node 6940856d575bc060ce15dbb0 INFO: 172.21.0.1:45766 - "POST /node/6940856d575bc060ce15dbb0 HTTP/1.0" 202 Accepted 2025-12-17 22:02:23,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084a9575bc060ce15d85c 2025-12-17 22:02:23,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084a9575bc060ce15d85c HTTP/1.1" 502 150 2025-12-17 22:02:23,401 - lava_callback - ERROR - Error processing callback for node 694084a9575bc060ce15d85c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a9575bc060ce15d85c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a9575bc060ce15d85c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084a9575bc060ce15d85c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:23,403 - lava_callback - INFO - Starting async job submit for node 694084c2575bc060ce15d8f2 2025-12-17 22:02:23,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:23,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c2575bc060ce15d8f2 HTTP/1.1" 502 150 2025-12-17 22:02:23,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c2575bc060ce15d8f2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:23,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c2575bc060ce15d8f2 2025-12-17 22:02:23,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c2575bc060ce15d8f2 HTTP/1.1" 502 150 2025-12-17 22:02:23,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c2575bc060ce15d8f2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:23,485 - lava_callback - INFO - Received callback for node 69408564575bc060ce15db9b INFO: 172.21.0.1:45772 - "POST /node/69408564575bc060ce15db9b HTTP/1.0" 202 Accepted 2025-12-17 22:02:24,132 - lava_callback - INFO - Received callback for node 69408565575bc060ce15db9e INFO: 172.21.0.1:45774 - "POST /node/69408565575bc060ce15db9e HTTP/1.0" 202 Accepted 2025-12-17 22:02:24,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084bb575bc060ce15d8a0 2025-12-17 22:02:24,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084bb575bc060ce15d8a0 HTTP/1.1" 502 150 2025-12-17 22:02:24,364 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084bb575bc060ce15d8a0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:24,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c1575bc060ce15d8ef 2025-12-17 22:02:24,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c1575bc060ce15d8ef HTTP/1.1" 502 150 2025-12-17 22:02:24,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c1575bc060ce15d8ef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:24,780 - lava_callback - INFO - Received callback for node 69408567575bc060ce15dba1 INFO: 172.21.0.1:45786 - "POST /node/69408567575bc060ce15dba1 HTTP/1.0" 202 Accepted 2025-12-17 22:02:24,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ac575bc060ce15d86a 2025-12-17 22:02:24,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ac575bc060ce15d86a HTTP/1.1" 502 150 2025-12-17 22:02:24,969 - lava_callback - ERROR - Error processing callback for node 694084ac575bc060ce15d86a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ac575bc060ce15d86a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ac575bc060ce15d86a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ac575bc060ce15d86a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:24,971 - lava_callback - INFO - Starting async job submit for node 694084cb575bc060ce15d91d 2025-12-17 22:02:24,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:24,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cb575bc060ce15d91d HTTP/1.1" 502 150 2025-12-17 22:02:24,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084cb575bc060ce15d91d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:24,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084cb575bc060ce15d91d 2025-12-17 22:02:24,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cb575bc060ce15d91d HTTP/1.1" 502 150 2025-12-17 22:02:24,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084cb575bc060ce15d91d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:25,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c2575bc060ce15d8f2 2025-12-17 22:02:25,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c2575bc060ce15d8f2 HTTP/1.1" 502 150 2025-12-17 22:02:25,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c2575bc060ce15d8f2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:26,063 - lava_callback - INFO - Received callback for node 6940bb491e3c24e95681b5f4 INFO: 172.21.0.1:45802 - "POST /node/6940bb491e3c24e95681b5f4 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:45816 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:02:26,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b5575bc060ce15d894 HTTP/1.1" 502 150 2025-12-17 22:02:26,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b5575bc060ce15d894'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:26,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084bc575bc060ce15d8a7 2025-12-17 22:02:26,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084bc575bc060ce15d8a7 HTTP/1.1" 502 150 2025-12-17 22:02:26,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084bc575bc060ce15d8a7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:26,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084cb575bc060ce15d91d 2025-12-17 22:02:26,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cb575bc060ce15d91d HTTP/1.1" 502 150 2025-12-17 22:02:26,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084cb575bc060ce15d91d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:27,278 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084af575bc060ce15d878 2025-12-17 22:02:28,074 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c2575bc060ce15d8f6 2025-12-17 22:02:28,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c2575bc060ce15d8f6 HTTP/1.1" 502 150 2025-12-17 22:02:28,075 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c2575bc060ce15d8f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:28,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c1575bc060ce15d8ef 2025-12-17 22:02:28,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c1575bc060ce15d8ef HTTP/1.1" 502 150 2025-12-17 22:02:28,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c1575bc060ce15d8ef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:28,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084be575bc060ce15d8df 2025-12-17 22:02:28,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084be575bc060ce15d8df HTTP/1.1" 502 150 2025-12-17 22:02:28,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084be575bc060ce15d8df'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:29,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c2575bc060ce15d8f2 2025-12-17 22:02:29,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c2575bc060ce15d8f2 HTTP/1.1" 502 150 2025-12-17 22:02:29,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c2575bc060ce15d8f2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:29,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940849a575bc060ce15d83a HTTP/1.1" 502 150 2025-12-17 22:02:29,424 - lava_callback - ERROR - Error processing callback for node 6940849a575bc060ce15d83a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940849a575bc060ce15d83a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940849a575bc060ce15d83a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940849a575bc060ce15d83a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:29,426 - lava_callback - INFO - Starting async job submit for node 694084c4575bc060ce15d90e 2025-12-17 22:02:29,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:29,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c4575bc060ce15d90e HTTP/1.1" 502 150 2025-12-17 22:02:29,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c4575bc060ce15d90e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:29,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c4575bc060ce15d90e 2025-12-17 22:02:29,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c4575bc060ce15d90e HTTP/1.1" 502 150 2025-12-17 22:02:29,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c4575bc060ce15d90e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:30,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c0575bc060ce15d8ec 2025-12-17 22:02:30,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c0575bc060ce15d8ec HTTP/1.1" 502 150 2025-12-17 22:02:30,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c0575bc060ce15d8ec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:30,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b0575bc060ce15d87f 2025-12-17 22:02:30,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084cb575bc060ce15d91d 2025-12-17 22:02:30,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b0575bc060ce15d87f HTTP/1.1" 502 150 2025-12-17 22:02:30,987 - lava_callback - ERROR - Error processing callback for node 694084b0575bc060ce15d87f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b0575bc060ce15d87f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b0575bc060ce15d87f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b0575bc060ce15d87f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:30,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cb575bc060ce15d91d HTTP/1.1" 502 150 2025-12-17 22:02:30,989 - lava_callback - INFO - Starting async job submit for node 694084c6575bc060ce15d911 2025-12-17 22:02:30,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084cb575bc060ce15d91d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:30,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:30,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c6575bc060ce15d911 HTTP/1.1" 502 150 2025-12-17 22:02:30,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c6575bc060ce15d911'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:30,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c6575bc060ce15d911 2025-12-17 22:02:30,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c6575bc060ce15d911 HTTP/1.1" 502 150 2025-12-17 22:02:30,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c6575bc060ce15d911'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:31,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c4575bc060ce15d90e 2025-12-17 22:02:31,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c4575bc060ce15d90e HTTP/1.1" 502 150 2025-12-17 22:02:31,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c4575bc060ce15d90e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:31,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b4575bc060ce15d88f 2025-12-17 22:02:31,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b4575bc060ce15d88f HTTP/1.1" 502 150 2025-12-17 22:02:31,897 - lava_callback - ERROR - Error processing callback for node 694084b4575bc060ce15d88f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b4575bc060ce15d88f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b4575bc060ce15d88f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b4575bc060ce15d88f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:31,899 - lava_callback - INFO - Starting async job submit for node 694084c7575bc060ce15d914 2025-12-17 22:02:31,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:31,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c7575bc060ce15d914 HTTP/1.1" 502 150 2025-12-17 22:02:31,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c7575bc060ce15d914'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:31,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c7575bc060ce15d914 2025-12-17 22:02:31,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c7575bc060ce15d914 HTTP/1.1" 502 150 2025-12-17 22:02:31,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c7575bc060ce15d914'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:32,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c6575bc060ce15d911 2025-12-17 22:02:32,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c6575bc060ce15d911 HTTP/1.1" 502 150 2025-12-17 22:02:32,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c6575bc060ce15d911'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:33,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c7575bc060ce15d914 2025-12-17 22:02:33,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c7575bc060ce15d914 HTTP/1.1" 502 150 2025-12-17 22:02:33,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c7575bc060ce15d914'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:34,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b5575bc060ce15d894 2025-12-17 22:02:34,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b5575bc060ce15d894 HTTP/1.1" 502 150 2025-12-17 22:02:34,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084b5575bc060ce15d894'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:35,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c4575bc060ce15d90e 2025-12-17 22:02:35,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c4575bc060ce15d90e HTTP/1.1" 502 150 2025-12-17 22:02:35,437 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c4575bc060ce15d90e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:35,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b7575bc060ce15d897 2025-12-17 22:02:35,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b7575bc060ce15d897 HTTP/1.1" 502 150 2025-12-17 22:02:35,838 - lava_callback - ERROR - Error processing callback for node 694084b7575bc060ce15d897: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b7575bc060ce15d897 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b7575bc060ce15d897 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b7575bc060ce15d897 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:35,840 - lava_callback - INFO - Starting async job submit for node 694084c8575bc060ce15d917 2025-12-17 22:02:35,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:35,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c8575bc060ce15d917 HTTP/1.1" 502 150 2025-12-17 22:02:35,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c8575bc060ce15d917'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:35,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c8575bc060ce15d917 2025-12-17 22:02:35,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c8575bc060ce15d917 HTTP/1.1" 502 150 2025-12-17 22:02:35,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c8575bc060ce15d917'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:36,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c1575bc060ce15d8ef 2025-12-17 22:02:36,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c1575bc060ce15d8ef HTTP/1.1" 502 150 2025-12-17 22:02:36,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c1575bc060ce15d8ef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58050 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:02:36,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c6575bc060ce15d911 2025-12-17 22:02:37,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c2575bc060ce15d8f2 2025-12-17 22:02:37,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c2575bc060ce15d8f2 HTTP/1.1" 502 150 2025-12-17 22:02:37,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c2575bc060ce15d8f2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:37,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c8575bc060ce15d917 2025-12-17 22:02:37,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c8575bc060ce15d917 HTTP/1.1" 502 150 2025-12-17 22:02:37,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c8575bc060ce15d917'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:37,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b8575bc060ce15d89a 2025-12-17 22:02:37,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b8575bc060ce15d89a HTTP/1.1" 502 150 2025-12-17 22:02:37,885 - lava_callback - ERROR - Error processing callback for node 694084b8575bc060ce15d89a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b8575bc060ce15d89a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b8575bc060ce15d89a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b8575bc060ce15d89a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:37,888 - lava_callback - INFO - Starting async job submit for node 694084c9575bc060ce15d91a 2025-12-17 22:02:37,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:37,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c9575bc060ce15d91a HTTP/1.1" 502 150 2025-12-17 22:02:37,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c9575bc060ce15d91a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:37,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c9575bc060ce15d91a 2025-12-17 22:02:37,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c9575bc060ce15d91a HTTP/1.1" 502 150 2025-12-17 22:02:37,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c9575bc060ce15d91a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:37,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c7575bc060ce15d914 2025-12-17 22:02:37,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c7575bc060ce15d914 HTTP/1.1" 502 150 2025-12-17 22:02:37,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c7575bc060ce15d914'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:38,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b9575bc060ce15d89d 2025-12-17 22:02:38,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b9575bc060ce15d89d HTTP/1.1" 502 150 2025-12-17 22:02:38,794 - lava_callback - ERROR - Error processing callback for node 694084b9575bc060ce15d89d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b9575bc060ce15d89d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b9575bc060ce15d89d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b9575bc060ce15d89d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:38,796 - lava_callback - INFO - Starting async job submit for node 694084cc575bc060ce15d920 2025-12-17 22:02:38,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:38,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cc575bc060ce15d920 HTTP/1.1" 502 150 2025-12-17 22:02:38,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084cc575bc060ce15d920'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:38,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084cc575bc060ce15d920 2025-12-17 22:02:38,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cc575bc060ce15d920 HTTP/1.1" 502 150 2025-12-17 22:02:38,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084cc575bc060ce15d920'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:38,949 - lava_callback - INFO - Received callback for node 6940856f575bc060ce15dbb7 INFO: 172.21.0.1:58052 - "POST /node/6940856f575bc060ce15dbb7 HTTP/1.0" 202 Accepted 2025-12-17 22:02:38,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084cb575bc060ce15d91d 2025-12-17 22:02:38,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cb575bc060ce15d91d HTTP/1.1" 502 150 2025-12-17 22:02:38,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084cb575bc060ce15d91d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:39,748 - lava_callback - INFO - Received callback for node 69408570575bc060ce15dbc0 INFO: 172.21.0.1:42326 - "POST /node/69408570575bc060ce15dbc0 HTTP/1.0" 202 Accepted 2025-12-17 22:02:39,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c9575bc060ce15d91a 2025-12-17 22:02:39,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c9575bc060ce15d91a HTTP/1.1" 502 150 2025-12-17 22:02:39,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c9575bc060ce15d91a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:40,364 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084bb575bc060ce15d8a0 2025-12-17 22:02:40,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084bb575bc060ce15d8a0 HTTP/1.1" 502 150 2025-12-17 22:02:40,365 - lava_callback - ERROR - Error processing callback for node 694084bb575bc060ce15d8a0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084bb575bc060ce15d8a0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084bb575bc060ce15d8a0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084bb575bc060ce15d8a0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:40,367 - lava_callback - INFO - Starting async job submit for node 694084ce575bc060ce15d923 2025-12-17 22:02:40,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:40,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ce575bc060ce15d923 HTTP/1.1" 502 150 2025-12-17 22:02:40,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ce575bc060ce15d923'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:40,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ce575bc060ce15d923 2025-12-17 22:02:40,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ce575bc060ce15d923 HTTP/1.1" 502 150 2025-12-17 22:02:40,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ce575bc060ce15d923'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:40,563 - lava_callback - INFO - Received callback for node 69408572575bc060ce15dbc7 INFO: 172.21.0.1:42338 - "POST /node/69408572575bc060ce15dbc7 HTTP/1.0" 202 Accepted 2025-12-17 22:02:40,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084cc575bc060ce15d920 2025-12-17 22:02:40,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cc575bc060ce15d920 HTTP/1.1" 502 150 2025-12-17 22:02:40,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084cc575bc060ce15d920'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:41,318 - lava_callback - INFO - Received callback for node 69408575575bc060ce15dbd2 INFO: 172.21.0.1:42342 - "POST /node/69408575575bc060ce15dbd2 HTTP/1.0" 202 Accepted 2025-12-17 22:02:41,555 - lava_callback - INFO - Received callback for node 69408f4a575bc060ce1600f1 INFO: 172.21.0.1:42352 - "POST /node/69408f4a575bc060ce1600f1 HTTP/1.0" 202 Accepted 2025-12-17 22:02:41,649 - lava_callback - INFO - Received callback for node 6940853a575bc060ce15dad4 INFO: 172.21.0.1:42366 - "POST /node/6940853a575bc060ce15dad4 HTTP/1.0" 202 Accepted 2025-12-17 22:02:41,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c8575bc060ce15d917 2025-12-17 22:02:41,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c8575bc060ce15d917 HTTP/1.1" 502 150 2025-12-17 22:02:41,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c8575bc060ce15d917'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:42,059 - lava_callback - INFO - Received callback for node 69408577575bc060ce15dbd7 INFO: 172.21.0.1:42380 - "POST /node/69408577575bc060ce15dbd7 HTTP/1.0" 202 Accepted 2025-12-17 22:02:42,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ce575bc060ce15d923 2025-12-17 22:02:42,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ce575bc060ce15d923 HTTP/1.1" 502 150 2025-12-17 22:02:42,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ce575bc060ce15d923'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:42,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084bc575bc060ce15d8a7 2025-12-17 22:02:42,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084bc575bc060ce15d8a7 HTTP/1.1" 502 150 2025-12-17 22:02:42,688 - lava_callback - ERROR - Error processing callback for node 694084bc575bc060ce15d8a7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084bc575bc060ce15d8a7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084bc575bc060ce15d8a7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084bc575bc060ce15d8a7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:42,690 - lava_callback - INFO - Starting async job submit for node 694084cf575bc060ce15d926 2025-12-17 22:02:42,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:42,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cf575bc060ce15d926 HTTP/1.1" 502 150 2025-12-17 22:02:42,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084cf575bc060ce15d926'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:42,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084cf575bc060ce15d926 2025-12-17 22:02:42,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cf575bc060ce15d926 HTTP/1.1" 502 150 2025-12-17 22:02:42,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084cf575bc060ce15d926'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:42,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084af575bc060ce15d878 HTTP/1.1" 502 150 2025-12-17 22:02:42,736 - lava_callback - ERROR - Error processing callback for node 694084af575bc060ce15d878: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084af575bc060ce15d878 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084af575bc060ce15d878 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084af575bc060ce15d878 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:42,739 - lava_callback - INFO - Starting async job submit for node 694084d0575bc060ce15d92d 2025-12-17 22:02:42,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:42,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d0575bc060ce15d92d HTTP/1.1" 502 150 2025-12-17 22:02:42,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d0575bc060ce15d92d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:42,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d0575bc060ce15d92d 2025-12-17 22:02:42,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d0575bc060ce15d92d HTTP/1.1" 502 150 2025-12-17 22:02:42,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d0575bc060ce15d92d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:43,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c4575bc060ce15d90e 2025-12-17 22:02:43,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c4575bc060ce15d90e HTTP/1.1" 502 150 2025-12-17 22:02:43,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c4575bc060ce15d90e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:43,494 - lava_callback - INFO - Received callback for node 69408572575bc060ce15dbca INFO: 172.21.0.1:42382 - "POST /node/69408572575bc060ce15dbca HTTP/1.0" 202 Accepted 2025-12-17 22:02:43,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c9575bc060ce15d91a 2025-12-17 22:02:43,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c9575bc060ce15d91a HTTP/1.1" 502 150 2025-12-17 22:02:43,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c9575bc060ce15d91a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:44,076 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c2575bc060ce15d8f6 2025-12-17 22:02:44,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c2575bc060ce15d8f6 HTTP/1.1" 502 150 2025-12-17 22:02:44,077 - lava_callback - ERROR - Error processing callback for node 694084c2575bc060ce15d8f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c2575bc060ce15d8f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c2575bc060ce15d8f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c2575bc060ce15d8f6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:44,079 - lava_callback - INFO - Starting async job submit for node 694084d1575bc060ce15d934 2025-12-17 22:02:44,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:44,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d1575bc060ce15d934 HTTP/1.1" 502 150 2025-12-17 22:02:44,086 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d1575bc060ce15d934'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:44,086 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d1575bc060ce15d934 2025-12-17 22:02:44,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d1575bc060ce15d934 HTTP/1.1" 502 150 2025-12-17 22:02:44,087 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d1575bc060ce15d934'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:44,139 - lava_callback - INFO - Received callback for node 69408574575bc060ce15dbcf INFO: 172.21.0.1:42388 - "POST /node/69408574575bc060ce15dbcf HTTP/1.0" 202 Accepted 2025-12-17 22:02:44,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084be575bc060ce15d8df 2025-12-17 22:02:44,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084be575bc060ce15d8df HTTP/1.1" 502 150 2025-12-17 22:02:44,531 - lava_callback - ERROR - Error processing callback for node 694084be575bc060ce15d8df: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084be575bc060ce15d8df (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084be575bc060ce15d8df (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084be575bc060ce15d8df (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:44,532 - lava_callback - INFO - Starting async job submit for node 694084d3575bc060ce15d93d 2025-12-17 22:02:44,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:44,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d3575bc060ce15d93d HTTP/1.1" 502 150 2025-12-17 22:02:44,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d3575bc060ce15d93d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:44,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d3575bc060ce15d93d 2025-12-17 22:02:44,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d3575bc060ce15d93d HTTP/1.1" 502 150 2025-12-17 22:02:44,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d3575bc060ce15d93d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:44,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084cf575bc060ce15d926 2025-12-17 22:02:44,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cf575bc060ce15d926 HTTP/1.1" 502 150 2025-12-17 22:02:44,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084cf575bc060ce15d926'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:44,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d0575bc060ce15d92d 2025-12-17 22:02:44,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d0575bc060ce15d92d HTTP/1.1" 502 150 2025-12-17 22:02:44,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d0575bc060ce15d92d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:44,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084cc575bc060ce15d920 2025-12-17 22:02:44,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cc575bc060ce15d920 HTTP/1.1" 502 150 2025-12-17 22:02:44,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084cc575bc060ce15d920'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:45,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c7575bc060ce15d914 2025-12-17 22:02:45,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c7575bc060ce15d914 HTTP/1.1" 502 150 2025-12-17 22:02:45,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c7575bc060ce15d914'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:46,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c0575bc060ce15d8ec 2025-12-17 22:02:46,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c0575bc060ce15d8ec HTTP/1.1" 502 150 2025-12-17 22:02:46,027 - lava_callback - ERROR - Error processing callback for node 694084c0575bc060ce15d8ec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c0575bc060ce15d8ec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c0575bc060ce15d8ec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c0575bc060ce15d8ec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:46,029 - lava_callback - INFO - Starting async job submit for node 694084d4575bc060ce15d940 2025-12-17 22:02:46,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:46,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d4575bc060ce15d940 HTTP/1.1" 502 150 2025-12-17 22:02:46,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d4575bc060ce15d940'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:46,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d4575bc060ce15d940 2025-12-17 22:02:46,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d4575bc060ce15d940 HTTP/1.1" 502 150 2025-12-17 22:02:46,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d4575bc060ce15d940'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:46,087 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d1575bc060ce15d934 2025-12-17 22:02:46,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d1575bc060ce15d934 HTTP/1.1" 502 150 2025-12-17 22:02:46,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d1575bc060ce15d934'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:46,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ce575bc060ce15d923 2025-12-17 22:02:46,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ce575bc060ce15d923 HTTP/1.1" 502 150 2025-12-17 22:02:46,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ce575bc060ce15d923'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42390 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:02:46,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d3575bc060ce15d93d 2025-12-17 22:02:46,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d3575bc060ce15d93d HTTP/1.1" 502 150 2025-12-17 22:02:46,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d3575bc060ce15d93d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:48,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d4575bc060ce15d940 2025-12-17 22:02:48,667 - lava_callback - INFO - Received callback for node 6940857a575bc060ce15dc0d INFO: 172.21.0.1:42406 - "POST /node/6940857a575bc060ce15dc0d HTTP/1.0" 202 Accepted 2025-12-17 22:02:48,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084cf575bc060ce15d926 2025-12-17 22:02:48,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cf575bc060ce15d926 HTTP/1.1" 502 150 2025-12-17 22:02:48,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084cf575bc060ce15d926'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:48,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d0575bc060ce15d92d 2025-12-17 22:02:48,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d0575bc060ce15d92d HTTP/1.1" 502 150 2025-12-17 22:02:48,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d0575bc060ce15d92d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:49,322 - lava_callback - INFO - Received callback for node 6940857c575bc060ce15dc3b INFO: 172.21.0.1:42408 - "POST /node/6940857c575bc060ce15dc3b HTTP/1.0" 202 Accepted 2025-12-17 22:02:49,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c8575bc060ce15d917 2025-12-17 22:02:49,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c8575bc060ce15d917 HTTP/1.1" 502 150 2025-12-17 22:02:49,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c8575bc060ce15d917'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:50,089 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d1575bc060ce15d934 2025-12-17 22:02:50,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d1575bc060ce15d934 HTTP/1.1" 502 150 2025-12-17 22:02:50,090 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d1575bc060ce15d934'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:50,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d3575bc060ce15d93d 2025-12-17 22:02:50,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d3575bc060ce15d93d HTTP/1.1" 502 150 2025-12-17 22:02:50,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d3575bc060ce15d93d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:50,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084b5575bc060ce15d894 2025-12-17 22:02:50,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084b5575bc060ce15d894 HTTP/1.1" 502 150 2025-12-17 22:02:50,612 - lava_callback - ERROR - Error processing callback for node 694084b5575bc060ce15d894: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b5575bc060ce15d894 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b5575bc060ce15d894 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084b5575bc060ce15d894 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:50,615 - lava_callback - INFO - Starting async job submit for node 694084d5575bc060ce15d943 2025-12-17 22:02:50,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:50,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d5575bc060ce15d943 HTTP/1.1" 502 150 2025-12-17 22:02:50,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d5575bc060ce15d943'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:50,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d5575bc060ce15d943 2025-12-17 22:02:50,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d5575bc060ce15d943 HTTP/1.1" 502 150 2025-12-17 22:02:50,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d5575bc060ce15d943'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:51,299 - lava_callback - INFO - Received callback for node 6940912f575bc060ce1607b2 INFO: 172.21.0.1:51226 - "POST /node/6940912f575bc060ce1607b2 HTTP/1.0" 202 Accepted 2025-12-17 22:02:51,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c9575bc060ce15d91a 2025-12-17 22:02:51,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c9575bc060ce15d91a HTTP/1.1" 502 150 2025-12-17 22:02:51,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c9575bc060ce15d91a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:52,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c1575bc060ce15d8ef 2025-12-17 22:02:52,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c6575bc060ce15d911 HTTP/1.1" 502 150 2025-12-17 22:02:52,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c6575bc060ce15d911'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:52,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c1575bc060ce15d8ef HTTP/1.1" 502 150 2025-12-17 22:02:52,465 - lava_callback - ERROR - Error processing callback for node 694084c1575bc060ce15d8ef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c1575bc060ce15d8ef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c1575bc060ce15d8ef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c1575bc060ce15d8ef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:52,467 - lava_callback - INFO - Starting async job submit for node 694084d7575bc060ce15d94a 2025-12-17 22:02:52,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:52,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d7575bc060ce15d94a HTTP/1.1" 502 150 2025-12-17 22:02:52,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d7575bc060ce15d94a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:52,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d7575bc060ce15d94a 2025-12-17 22:02:52,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d7575bc060ce15d94a HTTP/1.1" 502 150 2025-12-17 22:02:52,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d7575bc060ce15d94a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:52,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d5575bc060ce15d943 2025-12-17 22:02:52,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d5575bc060ce15d943 HTTP/1.1" 502 150 2025-12-17 22:02:52,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d5575bc060ce15d943'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:52,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084cc575bc060ce15d920 2025-12-17 22:02:52,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cc575bc060ce15d920 HTTP/1.1" 502 150 2025-12-17 22:02:52,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084cc575bc060ce15d920'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:53,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c2575bc060ce15d8f2 2025-12-17 22:02:53,708 - lava_callback - INFO - Received callback for node 6940857e575bc060ce15dc4b INFO: 172.21.0.1:51232 - "POST /node/6940857e575bc060ce15dc4b HTTP/1.0" 202 Accepted 2025-12-17 22:02:54,354 - lava_callback - INFO - Received callback for node 69408580575bc060ce15dc4e INFO: 172.21.0.1:51234 - "POST /node/69408580575bc060ce15dc4e HTTP/1.0" 202 Accepted 2025-12-17 22:02:54,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ce575bc060ce15d923 2025-12-17 22:02:54,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ce575bc060ce15d923 HTTP/1.1" 502 150 2025-12-17 22:02:54,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ce575bc060ce15d923'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:54,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d7575bc060ce15d94a 2025-12-17 22:02:54,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d7575bc060ce15d94a HTTP/1.1" 502 150 2025-12-17 22:02:54,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d7575bc060ce15d94a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:54,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084cb575bc060ce15d91d 2025-12-17 22:02:54,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cb575bc060ce15d91d HTTP/1.1" 502 150 2025-12-17 22:02:54,994 - lava_callback - ERROR - Error processing callback for node 694084cb575bc060ce15d91d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084cb575bc060ce15d91d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084cb575bc060ce15d91d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084cb575bc060ce15d91d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:54,995 - lava_callback - INFO - Starting async job submit for node 694084d8575bc060ce15d94d 2025-12-17 22:02:54,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:55,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d8575bc060ce15d94d HTTP/1.1" 502 150 2025-12-17 22:02:55,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d8575bc060ce15d94d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:55,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d8575bc060ce15d94d 2025-12-17 22:02:55,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d8575bc060ce15d94d HTTP/1.1" 502 150 2025-12-17 22:02:55,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d8575bc060ce15d94d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:55,010 - lava_callback - INFO - Received callback for node 69408581575bc060ce15dc51 INFO: 172.21.0.1:51244 - "POST /node/69408581575bc060ce15dc51 HTTP/1.0" 202 Accepted 2025-12-17 22:02:55,673 - lava_callback - INFO - Received callback for node 69408583575bc060ce15dc5a INFO: 172.21.0.1:51256 - "POST /node/69408583575bc060ce15dc5a HTTP/1.0" 202 Accepted 2025-12-17 22:02:56,368 - lava_callback - INFO - Received callback for node 69408584575bc060ce15dc63 INFO: 172.21.0.1:51258 - "POST /node/69408584575bc060ce15dc63 HTTP/1.0" 202 Accepted 2025-12-17 22:02:56,389 - lava_callback - INFO - Received callback for node 6942e2f2e43d2b76e9023925 INFO: 172.21.0.1:51266 - "POST /node/6942e2f2e43d2b76e9023925 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:51276 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:02:56,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d5575bc060ce15d943 2025-12-17 22:02:56,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d5575bc060ce15d943 HTTP/1.1" 502 150 2025-12-17 22:02:56,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d5575bc060ce15d943'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:56,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084cf575bc060ce15d926 2025-12-17 22:02:56,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cf575bc060ce15d926 HTTP/1.1" 502 150 2025-12-17 22:02:56,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084cf575bc060ce15d926'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:56,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d0575bc060ce15d92d 2025-12-17 22:02:56,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d0575bc060ce15d92d HTTP/1.1" 502 150 2025-12-17 22:02:56,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d0575bc060ce15d92d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:57,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d8575bc060ce15d94d 2025-12-17 22:02:57,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d8575bc060ce15d94d HTTP/1.1" 502 150 2025-12-17 22:02:57,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d8575bc060ce15d94d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:57,040 - lava_callback - INFO - Received callback for node 69408585575bc060ce15dc6a INFO: 172.21.0.1:51278 - "POST /node/69408585575bc060ce15dc6a HTTP/1.0" 202 Accepted 2025-12-17 22:02:58,090 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d1575bc060ce15d934 2025-12-17 22:02:58,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d1575bc060ce15d934 HTTP/1.1" 502 150 2025-12-17 22:02:58,091 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d1575bc060ce15d934'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:58,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d7575bc060ce15d94a 2025-12-17 22:02:58,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d7575bc060ce15d94a HTTP/1.1" 502 150 2025-12-17 22:02:58,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d7575bc060ce15d94a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:58,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d3575bc060ce15d93d 2025-12-17 22:02:58,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d3575bc060ce15d93d HTTP/1.1" 502 150 2025-12-17 22:02:58,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d3575bc060ce15d93d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:59,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c4575bc060ce15d90e 2025-12-17 22:02:59,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c4575bc060ce15d90e HTTP/1.1" 502 150 2025-12-17 22:02:59,441 - lava_callback - ERROR - Error processing callback for node 694084c4575bc060ce15d90e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c4575bc060ce15d90e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c4575bc060ce15d90e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c4575bc060ce15d90e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:02:59,443 - lava_callback - INFO - Starting async job submit for node 694084d9575bc060ce15d950 2025-12-17 22:02:59,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:02:59,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d9575bc060ce15d950 HTTP/1.1" 502 150 2025-12-17 22:02:59,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d9575bc060ce15d950'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:59,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d9575bc060ce15d950 2025-12-17 22:02:59,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d9575bc060ce15d950 HTTP/1.1" 502 150 2025-12-17 22:02:59,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d9575bc060ce15d950'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:02:59,456 - lava_callback - INFO - Received callback for node 69408588575bc060ce15dc72 INFO: 172.21.0.1:51288 - "POST /node/69408588575bc060ce15dc72 HTTP/1.0" 202 Accepted 2025-12-17 22:02:59,849 - lava_callback - INFO - Received callback for node 69408f13575bc060ce160064 INFO: 172.21.0.1:48430 - "POST /node/69408f13575bc060ce160064 HTTP/1.0" 202 Accepted 2025-12-17 22:03:00,405 - lava_callback - INFO - Received callback for node 69408589575bc060ce15dc75 INFO: 172.21.0.1:48446 - "POST /node/69408589575bc060ce15dc75 HTTP/1.0" 202 Accepted 2025-12-17 22:03:00,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c6575bc060ce15d911 2025-12-17 22:03:00,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c6575bc060ce15d911 HTTP/1.1" 502 150 2025-12-17 22:03:00,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084c6575bc060ce15d911'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:01,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d8575bc060ce15d94d 2025-12-17 22:03:01,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d8575bc060ce15d94d HTTP/1.1" 502 150 2025-12-17 22:03:01,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d8575bc060ce15d94d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:01,153 - lava_callback - INFO - Received callback for node 6940858b575bc060ce15dc78 INFO: 172.21.0.1:48462 - "POST /node/6940858b575bc060ce15dc78 HTTP/1.0" 202 Accepted 2025-12-17 22:03:01,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d9575bc060ce15d950 2025-12-17 22:03:01,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d9575bc060ce15d950 HTTP/1.1" 502 150 2025-12-17 22:03:01,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d9575bc060ce15d950'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:01,716 - lava_callback - INFO - Received callback for node 6940853c575bc060ce15db10 INFO: 172.21.0.1:48464 - "POST /node/6940853c575bc060ce15db10 HTTP/1.0" 202 Accepted 2025-12-17 22:03:01,897 - lava_callback - INFO - Received callback for node 6940858c575bc060ce15dc7f INFO: 172.21.0.1:48472 - "POST /node/6940858c575bc060ce15dc7f HTTP/1.0" 202 Accepted 2025-12-17 22:03:01,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c7575bc060ce15d914 2025-12-17 22:03:01,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c7575bc060ce15d914 HTTP/1.1" 502 150 2025-12-17 22:03:01,913 - lava_callback - ERROR - Error processing callback for node 694084c7575bc060ce15d914: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c7575bc060ce15d914 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c7575bc060ce15d914 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c7575bc060ce15d914 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:01,914 - lava_callback - INFO - Starting async job submit for node 694084db575bc060ce15d953 2025-12-17 22:03:01,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:01,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084db575bc060ce15d953 HTTP/1.1" 502 150 2025-12-17 22:03:01,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084db575bc060ce15d953'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:01,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084db575bc060ce15d953 2025-12-17 22:03:01,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084db575bc060ce15d953 HTTP/1.1" 502 150 2025-12-17 22:03:01,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084db575bc060ce15d953'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48486 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:03:02,645 - lava_callback - INFO - Received callback for node 6940858d575bc060ce15dc82 INFO: 172.21.0.1:48500 - "POST /node/6940858d575bc060ce15dc82 HTTP/1.0" 202 Accepted 2025-12-17 22:03:03,398 - lava_callback - INFO - Received callback for node 6940858f575bc060ce15dc8a INFO: 172.21.0.1:48506 - "POST /node/6940858f575bc060ce15dc8a HTTP/1.0" 202 Accepted 2025-12-17 22:03:03,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d4575bc060ce15d940 HTTP/1.1" 502 150 2025-12-17 22:03:03,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d4575bc060ce15d940'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:03,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084db575bc060ce15d953 2025-12-17 22:03:03,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084db575bc060ce15d953 HTTP/1.1" 502 150 2025-12-17 22:03:03,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084db575bc060ce15d953'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:04,142 - lava_callback - INFO - Received callback for node 69408590575bc060ce15dc8d INFO: 172.21.0.1:48516 - "POST /node/69408590575bc060ce15dc8d HTTP/1.0" 202 Accepted 2025-12-17 22:03:04,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d5575bc060ce15d943 2025-12-17 22:03:04,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d5575bc060ce15d943 HTTP/1.1" 502 150 2025-12-17 22:03:04,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d5575bc060ce15d943'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:04,914 - lava_callback - INFO - Received callback for node 69408591575bc060ce15dc94 INFO: 172.21.0.1:48518 - "POST /node/69408591575bc060ce15dc94 HTTP/1.0" 202 Accepted 2025-12-17 22:03:05,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d9575bc060ce15d950 2025-12-17 22:03:05,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d9575bc060ce15d950 HTTP/1.1" 502 150 2025-12-17 22:03:05,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d9575bc060ce15d950'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:05,654 - lava_callback - INFO - Received callback for node 69408593575bc060ce15dc97 INFO: 172.21.0.1:48520 - "POST /node/69408593575bc060ce15dc97 HTTP/1.0" 202 Accepted 2025-12-17 22:03:05,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c8575bc060ce15d917 2025-12-17 22:03:05,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c8575bc060ce15d917 HTTP/1.1" 502 150 2025-12-17 22:03:05,854 - lava_callback - ERROR - Error processing callback for node 694084c8575bc060ce15d917: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c8575bc060ce15d917 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c8575bc060ce15d917 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c8575bc060ce15d917 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:05,855 - lava_callback - INFO - Starting async job submit for node 694084dc575bc060ce15d956 2025-12-17 22:03:05,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:05,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084dc575bc060ce15d956 HTTP/1.1" 502 150 2025-12-17 22:03:05,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084dc575bc060ce15d956'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:05,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084dc575bc060ce15d956 2025-12-17 22:03:05,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084dc575bc060ce15d956 HTTP/1.1" 502 150 2025-12-17 22:03:05,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084dc575bc060ce15d956'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:06,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d7575bc060ce15d94a 2025-12-17 22:03:06,478 - lava_callback - INFO - Received callback for node 69408594575bc060ce15dc9a INFO: 172.21.0.1:48536 - "POST /node/69408594575bc060ce15dc9a HTTP/1.0" 202 Accepted INFO: 172.21.0.1:48546 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:03:07,226 - lava_callback - INFO - Received callback for node 69408595575bc060ce15dc9d INFO: 172.21.0.1:48548 - "POST /node/69408595575bc060ce15dc9d HTTP/1.0" 202 Accepted 2025-12-17 22:03:07,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d4575bc060ce15d940 2025-12-17 22:03:07,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d4575bc060ce15d940 HTTP/1.1" 502 150 2025-12-17 22:03:07,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d4575bc060ce15d940'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:07,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084dc575bc060ce15d956 2025-12-17 22:03:07,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084dc575bc060ce15d956 HTTP/1.1" 502 150 2025-12-17 22:03:07,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084dc575bc060ce15d956'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:07,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c9575bc060ce15d91a 2025-12-17 22:03:07,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c9575bc060ce15d91a HTTP/1.1" 502 150 2025-12-17 22:03:07,903 - lava_callback - ERROR - Error processing callback for node 694084c9575bc060ce15d91a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c9575bc060ce15d91a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c9575bc060ce15d91a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c9575bc060ce15d91a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:07,905 - lava_callback - INFO - Starting async job submit for node 694084dd575bc060ce15d959 2025-12-17 22:03:07,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:07,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084dd575bc060ce15d959 HTTP/1.1" 502 150 2025-12-17 22:03:07,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084dd575bc060ce15d959'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:07,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084dd575bc060ce15d959 2025-12-17 22:03:07,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084dd575bc060ce15d959 HTTP/1.1" 502 150 2025-12-17 22:03:07,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084dd575bc060ce15d959'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:07,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084db575bc060ce15d953 2025-12-17 22:03:07,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084db575bc060ce15d953 HTTP/1.1" 502 150 2025-12-17 22:03:07,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084db575bc060ce15d953'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:07,965 - lava_callback - INFO - Received callback for node 69408596575bc060ce15dca0 INFO: 172.21.0.1:48562 - "POST /node/69408596575bc060ce15dca0 HTTP/1.0" 202 Accepted 2025-12-17 22:03:08,729 - lava_callback - INFO - Received callback for node 69408598575bc060ce15dca7 INFO: 172.21.0.1:48566 - "POST /node/69408598575bc060ce15dca7 HTTP/1.0" 202 Accepted 2025-12-17 22:03:08,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084cc575bc060ce15d920 2025-12-17 22:03:08,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cc575bc060ce15d920 HTTP/1.1" 502 150 2025-12-17 22:03:08,809 - lava_callback - ERROR - Error processing callback for node 694084cc575bc060ce15d920: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084cc575bc060ce15d920 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084cc575bc060ce15d920 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084cc575bc060ce15d920 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:08,813 - lava_callback - INFO - Starting async job submit for node 694084de575bc060ce15d95c 2025-12-17 22:03:08,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:08,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084de575bc060ce15d95c HTTP/1.1" 502 150 2025-12-17 22:03:08,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084de575bc060ce15d95c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:08,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084de575bc060ce15d95c 2025-12-17 22:03:08,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084de575bc060ce15d95c HTTP/1.1" 502 150 2025-12-17 22:03:08,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084de575bc060ce15d95c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:08,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c2575bc060ce15d8f2 HTTP/1.1" 502 150 2025-12-17 22:03:08,848 - lava_callback - ERROR - Error processing callback for node 694084c2575bc060ce15d8f2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c2575bc060ce15d8f2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c2575bc060ce15d8f2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c2575bc060ce15d8f2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:08,852 - lava_callback - INFO - Starting async job submit for node 694084e0575bc060ce15d95f 2025-12-17 22:03:08,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:08,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d95f HTTP/1.1" 502 150 2025-12-17 22:03:08,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e0575bc060ce15d95f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:08,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e0575bc060ce15d95f 2025-12-17 22:03:08,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d95f HTTP/1.1" 502 150 2025-12-17 22:03:08,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e0575bc060ce15d95f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:09,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d8575bc060ce15d94d 2025-12-17 22:03:09,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d8575bc060ce15d94d HTTP/1.1" 502 150 2025-12-17 22:03:09,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d8575bc060ce15d94d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:09,478 - lava_callback - INFO - Received callback for node 6940859a575bc060ce15dcae INFO: 172.21.0.1:48580 - "POST /node/6940859a575bc060ce15dcae HTTP/1.0" 202 Accepted 2025-12-17 22:03:09,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084dd575bc060ce15d959 2025-12-17 22:03:09,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084dd575bc060ce15d959 HTTP/1.1" 502 150 2025-12-17 22:03:09,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084dd575bc060ce15d959'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:10,236 - lava_callback - INFO - Received callback for node 6940859b575bc060ce15dcb5 INFO: 172.21.0.1:44294 - "POST /node/6940859b575bc060ce15dcb5 HTTP/1.0" 202 Accepted 2025-12-17 22:03:10,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ce575bc060ce15d923 2025-12-17 22:03:10,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ce575bc060ce15d923 HTTP/1.1" 502 150 2025-12-17 22:03:10,404 - lava_callback - ERROR - Error processing callback for node 694084ce575bc060ce15d923: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ce575bc060ce15d923 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ce575bc060ce15d923 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ce575bc060ce15d923 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:10,406 - lava_callback - INFO - Starting async job submit for node 694084e0575bc060ce15d962 2025-12-17 22:03:10,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:10,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d962 HTTP/1.1" 502 150 2025-12-17 22:03:10,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e0575bc060ce15d962'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:10,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e0575bc060ce15d962 2025-12-17 22:03:10,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d962 HTTP/1.1" 502 150 2025-12-17 22:03:10,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e0575bc060ce15d962'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:10,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084de575bc060ce15d95c 2025-12-17 22:03:10,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084de575bc060ce15d95c HTTP/1.1" 502 150 2025-12-17 22:03:10,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084de575bc060ce15d95c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:10,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e0575bc060ce15d95f 2025-12-17 22:03:10,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d95f HTTP/1.1" 502 150 2025-12-17 22:03:10,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e0575bc060ce15d95f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:11,028 - lava_callback - INFO - Received callback for node 6940859c575bc060ce15dcbc INFO: 172.21.0.1:44296 - "POST /node/6940859c575bc060ce15dcbc HTTP/1.0" 202 Accepted 2025-12-17 22:03:11,846 - lava_callback - INFO - Received callback for node 6940859d575bc060ce15dcc1 INFO: 172.21.0.1:44300 - "POST /node/6940859d575bc060ce15dcc1 HTTP/1.0" 202 Accepted 2025-12-17 22:03:11,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084dc575bc060ce15d956 2025-12-17 22:03:11,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084dc575bc060ce15d956 HTTP/1.1" 502 150 2025-12-17 22:03:11,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084dc575bc060ce15d956'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:12,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e0575bc060ce15d962 2025-12-17 22:03:12,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d962 HTTP/1.1" 502 150 2025-12-17 22:03:12,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e0575bc060ce15d962'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:12,581 - lava_callback - INFO - Received callback for node 6940859f575bc060ce15dcc4 INFO: 172.21.0.1:44310 - "POST /node/6940859f575bc060ce15dcc4 HTTP/1.0" 202 Accepted 2025-12-17 22:03:12,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084cf575bc060ce15d926 2025-12-17 22:03:12,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084cf575bc060ce15d926 HTTP/1.1" 502 150 2025-12-17 22:03:12,703 - lava_callback - ERROR - Error processing callback for node 694084cf575bc060ce15d926: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084cf575bc060ce15d926 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084cf575bc060ce15d926 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084cf575bc060ce15d926 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:12,705 - lava_callback - INFO - Starting async job submit for node 694084e0575bc060ce15d965 2025-12-17 22:03:12,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:12,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d965 HTTP/1.1" 502 150 2025-12-17 22:03:12,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e0575bc060ce15d965'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:12,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e0575bc060ce15d965 2025-12-17 22:03:12,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d965 HTTP/1.1" 502 150 2025-12-17 22:03:12,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e0575bc060ce15d965'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:12,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d0575bc060ce15d92d 2025-12-17 22:03:12,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d0575bc060ce15d92d HTTP/1.1" 502 150 2025-12-17 22:03:12,775 - lava_callback - ERROR - Error processing callback for node 694084d0575bc060ce15d92d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d0575bc060ce15d92d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d0575bc060ce15d92d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d0575bc060ce15d92d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:12,777 - lava_callback - INFO - Starting async job submit for node 694084e1575bc060ce15d968 2025-12-17 22:03:12,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:12,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e1575bc060ce15d968 HTTP/1.1" 502 150 2025-12-17 22:03:12,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e1575bc060ce15d968'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:12,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e1575bc060ce15d968 2025-12-17 22:03:12,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e1575bc060ce15d968 HTTP/1.1" 502 150 2025-12-17 22:03:12,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e1575bc060ce15d968'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:13,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d9575bc060ce15d950 2025-12-17 22:03:13,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d9575bc060ce15d950 HTTP/1.1" 502 150 2025-12-17 22:03:13,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d9575bc060ce15d950'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:13,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084dd575bc060ce15d959 2025-12-17 22:03:13,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084dd575bc060ce15d959 HTTP/1.1" 502 150 2025-12-17 22:03:13,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084dd575bc060ce15d959'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:14,092 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d1575bc060ce15d934 2025-12-17 22:03:14,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d3575bc060ce15d93d 2025-12-17 22:03:14,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d3575bc060ce15d93d HTTP/1.1" 502 150 2025-12-17 22:03:14,546 - lava_callback - ERROR - Error processing callback for node 694084d3575bc060ce15d93d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d3575bc060ce15d93d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d3575bc060ce15d93d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d3575bc060ce15d93d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:14,548 - lava_callback - INFO - Starting async job submit for node 694084e1575bc060ce15d96b 2025-12-17 22:03:14,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:14,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e1575bc060ce15d96b HTTP/1.1" 502 150 2025-12-17 22:03:14,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e1575bc060ce15d96b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:14,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e1575bc060ce15d96b 2025-12-17 22:03:14,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e1575bc060ce15d96b HTTP/1.1" 502 150 2025-12-17 22:03:14,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e1575bc060ce15d96b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:14,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e0575bc060ce15d965 2025-12-17 22:03:14,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d965 HTTP/1.1" 502 150 2025-12-17 22:03:14,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e0575bc060ce15d965'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:14,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e1575bc060ce15d968 2025-12-17 22:03:14,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e1575bc060ce15d968 HTTP/1.1" 502 150 2025-12-17 22:03:14,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e1575bc060ce15d968'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:14,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084de575bc060ce15d95c 2025-12-17 22:03:14,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084de575bc060ce15d95c HTTP/1.1" 502 150 2025-12-17 22:03:14,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084de575bc060ce15d95c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:14,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e0575bc060ce15d95f 2025-12-17 22:03:14,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d95f HTTP/1.1" 502 150 2025-12-17 22:03:14,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e0575bc060ce15d95f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44320 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:03:15,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d4575bc060ce15d940 2025-12-17 22:03:15,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d4575bc060ce15d940 HTTP/1.1" 502 150 2025-12-17 22:03:15,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d4575bc060ce15d940'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:15,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084db575bc060ce15d953 2025-12-17 22:03:15,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084db575bc060ce15d953 HTTP/1.1" 502 150 2025-12-17 22:03:15,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084db575bc060ce15d953'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:16,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e0575bc060ce15d962 2025-12-17 22:03:16,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d962 HTTP/1.1" 502 150 2025-12-17 22:03:16,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e0575bc060ce15d962'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:16,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084c6575bc060ce15d911 2025-12-17 22:03:16,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084c6575bc060ce15d911 HTTP/1.1" 502 150 2025-12-17 22:03:16,467 - lava_callback - ERROR - Error processing callback for node 694084c6575bc060ce15d911: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c6575bc060ce15d911 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c6575bc060ce15d911 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084c6575bc060ce15d911 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:16,469 - lava_callback - INFO - Starting async job submit for node 694084e2575bc060ce15d970 2025-12-17 22:03:16,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:16,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e2575bc060ce15d970 HTTP/1.1" 502 150 2025-12-17 22:03:16,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e2575bc060ce15d970'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:16,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e2575bc060ce15d970 2025-12-17 22:03:16,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e2575bc060ce15d970 HTTP/1.1" 502 150 2025-12-17 22:03:16,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e2575bc060ce15d970'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44322 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:03:16,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e1575bc060ce15d96b 2025-12-17 22:03:16,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e1575bc060ce15d96b HTTP/1.1" 502 150 2025-12-17 22:03:16,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e1575bc060ce15d96b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:18,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e2575bc060ce15d970 2025-12-17 22:03:18,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e2575bc060ce15d970 HTTP/1.1" 502 150 2025-12-17 22:03:18,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e2575bc060ce15d970'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:18,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e0575bc060ce15d965 2025-12-17 22:03:18,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d965 HTTP/1.1" 502 150 2025-12-17 22:03:18,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e0575bc060ce15d965'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:18,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e1575bc060ce15d968 2025-12-17 22:03:18,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e1575bc060ce15d968 HTTP/1.1" 502 150 2025-12-17 22:03:18,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e1575bc060ce15d968'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:19,197 - lava_callback - INFO - Received callback for node 694085a1575bc060ce15dcca INFO: 172.21.0.1:44336 - "POST /node/694085a1575bc060ce15dcca HTTP/1.0" 202 Accepted 2025-12-17 22:03:19,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084dc575bc060ce15d956 2025-12-17 22:03:19,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084dc575bc060ce15d956 HTTP/1.1" 502 150 2025-12-17 22:03:19,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084dc575bc060ce15d956'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:19,936 - lava_callback - INFO - Received callback for node 694085a3575bc060ce15dccd INFO: 172.21.0.1:39762 - "POST /node/694085a3575bc060ce15dccd HTTP/1.0" 202 Accepted 2025-12-17 22:03:20,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e1575bc060ce15d96b 2025-12-17 22:03:20,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e1575bc060ce15d96b HTTP/1.1" 502 150 2025-12-17 22:03:20,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e1575bc060ce15d96b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:20,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d5575bc060ce15d943 2025-12-17 22:03:20,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d5575bc060ce15d943 HTTP/1.1" 502 150 2025-12-17 22:03:20,640 - lava_callback - ERROR - Error processing callback for node 694084d5575bc060ce15d943: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d5575bc060ce15d943 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d5575bc060ce15d943 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d5575bc060ce15d943 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:20,642 - lava_callback - INFO - Starting async job submit for node 694084e5575bc060ce15d97f 2025-12-17 22:03:20,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:20,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e5575bc060ce15d97f HTTP/1.1" 502 150 2025-12-17 22:03:20,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e5575bc060ce15d97f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:20,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e5575bc060ce15d97f 2025-12-17 22:03:20,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e5575bc060ce15d97f HTTP/1.1" 502 150 2025-12-17 22:03:20,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e5575bc060ce15d97f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:20,699 - lava_callback - INFO - Received callback for node 694085a4575bc060ce15dcd4 INFO: 172.21.0.1:39778 - "POST /node/694085a4575bc060ce15dcd4 HTTP/1.0" 202 Accepted 2025-12-17 22:03:21,433 - lava_callback - INFO - Received callback for node 694085a4575bc060ce15dcd7 INFO: 172.21.0.1:39780 - "POST /node/694085a4575bc060ce15dcd7 HTTP/1.0" 202 Accepted 2025-12-17 22:03:21,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d7575bc060ce15d94a HTTP/1.1" 502 150 2025-12-17 22:03:21,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084d7575bc060ce15d94a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:21,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084dd575bc060ce15d959 2025-12-17 22:03:21,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084dd575bc060ce15d959 HTTP/1.1" 502 150 2025-12-17 22:03:21,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084dd575bc060ce15d959'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:22,193 - lava_callback - INFO - Received callback for node 694085a5575bc060ce15dcda INFO: 172.21.0.1:39786 - "POST /node/694085a5575bc060ce15dcda HTTP/1.0" 202 Accepted 2025-12-17 22:03:22,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e2575bc060ce15d970 2025-12-17 22:03:22,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e2575bc060ce15d970 HTTP/1.1" 502 150 2025-12-17 22:03:22,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e2575bc060ce15d970'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:22,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e5575bc060ce15d97f 2025-12-17 22:03:22,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e5575bc060ce15d97f HTTP/1.1" 502 150 2025-12-17 22:03:22,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e5575bc060ce15d97f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:22,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084de575bc060ce15d95c 2025-12-17 22:03:22,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084de575bc060ce15d95c HTTP/1.1" 502 150 2025-12-17 22:03:22,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084de575bc060ce15d95c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:22,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e0575bc060ce15d95f 2025-12-17 22:03:22,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d95f HTTP/1.1" 502 150 2025-12-17 22:03:22,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e0575bc060ce15d95f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:24,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e0575bc060ce15d962 2025-12-17 22:03:24,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d962 HTTP/1.1" 502 150 2025-12-17 22:03:24,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e0575bc060ce15d962'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:25,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d8575bc060ce15d94d INFO: 172.21.0.1:39798 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:03:26,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e5575bc060ce15d97f 2025-12-17 22:03:26,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e5575bc060ce15d97f HTTP/1.1" 502 150 2025-12-17 22:03:26,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e5575bc060ce15d97f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:26,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e0575bc060ce15d965 2025-12-17 22:03:26,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d965 HTTP/1.1" 502 150 2025-12-17 22:03:26,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e0575bc060ce15d965'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:26,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e1575bc060ce15d968 2025-12-17 22:03:26,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e1575bc060ce15d968 HTTP/1.1" 502 150 2025-12-17 22:03:26,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e1575bc060ce15d968'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:28,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e1575bc060ce15d96b 2025-12-17 22:03:28,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e1575bc060ce15d96b HTTP/1.1" 502 150 2025-12-17 22:03:28,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e1575bc060ce15d96b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:29,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d9575bc060ce15d950 2025-12-17 22:03:29,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d9575bc060ce15d950 HTTP/1.1" 502 150 2025-12-17 22:03:29,458 - lava_callback - ERROR - Error processing callback for node 694084d9575bc060ce15d950: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d9575bc060ce15d950 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d9575bc060ce15d950 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d9575bc060ce15d950 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:29,460 - lava_callback - INFO - Starting async job submit for node 694084e6575bc060ce15d986 2025-12-17 22:03:29,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:29,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e6575bc060ce15d986 HTTP/1.1" 502 150 2025-12-17 22:03:29,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e6575bc060ce15d986'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:29,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e6575bc060ce15d986 2025-12-17 22:03:29,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e6575bc060ce15d986 HTTP/1.1" 502 150 2025-12-17 22:03:29,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e6575bc060ce15d986'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:29,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d1575bc060ce15d934 HTTP/1.1" 502 150 2025-12-17 22:03:29,585 - lava_callback - ERROR - Error processing callback for node 694084d1575bc060ce15d934: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d1575bc060ce15d934 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d1575bc060ce15d934 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d1575bc060ce15d934 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:29,588 - lava_callback - INFO - Starting async job submit for node 694084e7575bc060ce15d989 2025-12-17 22:03:29,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:29,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e7575bc060ce15d989 HTTP/1.1" 502 150 2025-12-17 22:03:29,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e7575bc060ce15d989'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:29,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e7575bc060ce15d989 2025-12-17 22:03:29,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e7575bc060ce15d989 HTTP/1.1" 502 150 2025-12-17 22:03:29,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e7575bc060ce15d989'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:30,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e2575bc060ce15d970 2025-12-17 22:03:30,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e2575bc060ce15d970 HTTP/1.1" 502 150 2025-12-17 22:03:30,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e2575bc060ce15d970'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:31,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e6575bc060ce15d986 2025-12-17 22:03:31,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e6575bc060ce15d986 HTTP/1.1" 502 150 2025-12-17 22:03:31,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e6575bc060ce15d986'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:31,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d4575bc060ce15d940 2025-12-17 22:03:31,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d4575bc060ce15d940 HTTP/1.1" 502 150 2025-12-17 22:03:31,476 - lava_callback - ERROR - Error processing callback for node 694084d4575bc060ce15d940: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d4575bc060ce15d940 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d4575bc060ce15d940 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d4575bc060ce15d940 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:31,478 - lava_callback - INFO - Starting async job submit for node 694084e9575bc060ce15d98c 2025-12-17 22:03:31,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:31,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e9575bc060ce15d98c HTTP/1.1" 502 150 2025-12-17 22:03:31,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e9575bc060ce15d98c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:31,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e9575bc060ce15d98c 2025-12-17 22:03:31,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e9575bc060ce15d98c HTTP/1.1" 502 150 2025-12-17 22:03:31,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e9575bc060ce15d98c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:31,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e7575bc060ce15d989 2025-12-17 22:03:31,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e7575bc060ce15d989 HTTP/1.1" 502 150 2025-12-17 22:03:31,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e7575bc060ce15d989'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:31,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084db575bc060ce15d953 2025-12-17 22:03:31,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084db575bc060ce15d953 HTTP/1.1" 502 150 2025-12-17 22:03:31,938 - lava_callback - ERROR - Error processing callback for node 694084db575bc060ce15d953: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084db575bc060ce15d953 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084db575bc060ce15d953 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084db575bc060ce15d953 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:31,940 - lava_callback - INFO - Starting async job submit for node 694084ea575bc060ce15d98f 2025-12-17 22:03:31,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:31,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ea575bc060ce15d98f HTTP/1.1" 502 150 2025-12-17 22:03:31,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ea575bc060ce15d98f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:31,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ea575bc060ce15d98f 2025-12-17 22:03:31,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ea575bc060ce15d98f HTTP/1.1" 502 150 2025-12-17 22:03:31,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ea575bc060ce15d98f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:32,574 - lava_callback - INFO - Received callback for node 694085a6575bc060ce15dcdd INFO: 172.21.0.1:52188 - "POST /node/694085a6575bc060ce15dcdd HTTP/1.0" 202 Accepted 2025-12-17 22:03:33,193 - lava_callback - INFO - Received callback for node 694085af575bc060ce15dcfd INFO: 172.21.0.1:52194 - "POST /node/694085af575bc060ce15dcfd HTTP/1.0" 202 Accepted 2025-12-17 22:03:33,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e9575bc060ce15d98c 2025-12-17 22:03:33,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e9575bc060ce15d98c HTTP/1.1" 502 150 2025-12-17 22:03:33,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e9575bc060ce15d98c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:33,719 - lava_callback - INFO - Received callback for node 694085a7575bc060ce15dce0 INFO: 172.21.0.1:52208 - "POST /node/694085a7575bc060ce15dce0 HTTP/1.0" 202 Accepted 2025-12-17 22:03:33,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ea575bc060ce15d98f 2025-12-17 22:03:33,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ea575bc060ce15d98f HTTP/1.1" 502 150 2025-12-17 22:03:33,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ea575bc060ce15d98f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:34,429 - lava_callback - INFO - Received callback for node 694085a9575bc060ce15dce3 INFO: 172.21.0.1:52212 - "POST /node/694085a9575bc060ce15dce3 HTTP/1.0" 202 Accepted 2025-12-17 22:03:34,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e5575bc060ce15d97f 2025-12-17 22:03:34,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e5575bc060ce15d97f HTTP/1.1" 502 150 2025-12-17 22:03:34,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e5575bc060ce15d97f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:35,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e6575bc060ce15d986 2025-12-17 22:03:35,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e6575bc060ce15d986 HTTP/1.1" 502 150 2025-12-17 22:03:35,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e6575bc060ce15d986'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:35,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e7575bc060ce15d989 2025-12-17 22:03:35,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e7575bc060ce15d989 HTTP/1.1" 502 150 2025-12-17 22:03:35,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e7575bc060ce15d989'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:35,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084dc575bc060ce15d956 2025-12-17 22:03:35,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084dc575bc060ce15d956 HTTP/1.1" 502 150 2025-12-17 22:03:35,890 - lava_callback - ERROR - Error processing callback for node 694084dc575bc060ce15d956: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084dc575bc060ce15d956 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084dc575bc060ce15d956 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084dc575bc060ce15d956 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:35,892 - lava_callback - INFO - Starting async job submit for node 694084eb575bc060ce15d992 2025-12-17 22:03:35,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:35,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084eb575bc060ce15d992 HTTP/1.1" 502 150 2025-12-17 22:03:35,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084eb575bc060ce15d992'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:35,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084eb575bc060ce15d992 2025-12-17 22:03:35,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084eb575bc060ce15d992 HTTP/1.1" 502 150 2025-12-17 22:03:35,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084eb575bc060ce15d992'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52216 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:03:37,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e9575bc060ce15d98c 2025-12-17 22:03:37,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e9575bc060ce15d98c HTTP/1.1" 502 150 2025-12-17 22:03:37,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e9575bc060ce15d98c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:37,756 - lava_callback - INFO - Received callback for node 6942e380e43d2b76e9023a67 INFO: 172.21.0.1:52230 - "POST /node/6942e380e43d2b76e9023a67 HTTP/1.0" 202 Accepted 2025-12-17 22:03:37,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084d7575bc060ce15d94a 2025-12-17 22:03:37,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d7575bc060ce15d94a HTTP/1.1" 502 150 2025-12-17 22:03:37,905 - lava_callback - ERROR - Error processing callback for node 694084d7575bc060ce15d94a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d7575bc060ce15d94a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d7575bc060ce15d94a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d7575bc060ce15d94a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:37,907 - lava_callback - INFO - Starting async job submit for node 694084f4575bc060ce15d9ae 2025-12-17 22:03:37,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:37,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f4575bc060ce15d9ae HTTP/1.1" 502 150 2025-12-17 22:03:37,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f4575bc060ce15d9ae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:37,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f4575bc060ce15d9ae 2025-12-17 22:03:37,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f4575bc060ce15d9ae HTTP/1.1" 502 150 2025-12-17 22:03:37,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f4575bc060ce15d9ae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:37,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084dd575bc060ce15d959 2025-12-17 22:03:37,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084dd575bc060ce15d959 HTTP/1.1" 502 150 2025-12-17 22:03:37,920 - lava_callback - ERROR - Error processing callback for node 694084dd575bc060ce15d959: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084dd575bc060ce15d959 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084dd575bc060ce15d959 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084dd575bc060ce15d959 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:37,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084eb575bc060ce15d992 2025-12-17 22:03:37,922 - lava_callback - INFO - Starting async job submit for node 694084f2575bc060ce15d9a8 2025-12-17 22:03:37,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:37,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084eb575bc060ce15d992 HTTP/1.1" 502 150 2025-12-17 22:03:37,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084eb575bc060ce15d992'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:37,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f2575bc060ce15d9a8 HTTP/1.1" 502 150 2025-12-17 22:03:37,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f2575bc060ce15d9a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:37,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f2575bc060ce15d9a8 2025-12-17 22:03:37,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f2575bc060ce15d9a8 HTTP/1.1" 502 150 2025-12-17 22:03:37,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f2575bc060ce15d9a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:37,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ea575bc060ce15d98f 2025-12-17 22:03:37,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ea575bc060ce15d98f HTTP/1.1" 502 150 2025-12-17 22:03:37,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ea575bc060ce15d98f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:38,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084de575bc060ce15d95c 2025-12-17 22:03:38,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084de575bc060ce15d95c HTTP/1.1" 502 150 2025-12-17 22:03:38,859 - lava_callback - ERROR - Error processing callback for node 694084de575bc060ce15d95c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084de575bc060ce15d95c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084de575bc060ce15d95c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084de575bc060ce15d95c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:38,861 - lava_callback - INFO - Starting async job submit for node 694084f3575bc060ce15d9ab 2025-12-17 22:03:38,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:38,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f3575bc060ce15d9ab HTTP/1.1" 502 150 2025-12-17 22:03:38,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f3575bc060ce15d9ab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:38,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f3575bc060ce15d9ab 2025-12-17 22:03:38,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f3575bc060ce15d9ab HTTP/1.1" 502 150 2025-12-17 22:03:38,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f3575bc060ce15d9ab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:38,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e0575bc060ce15d95f 2025-12-17 22:03:38,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d95f HTTP/1.1" 502 150 2025-12-17 22:03:38,886 - lava_callback - ERROR - Error processing callback for node 694084e0575bc060ce15d95f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e0575bc060ce15d95f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e0575bc060ce15d95f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e0575bc060ce15d95f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:38,888 - lava_callback - INFO - Starting async job submit for node 694084ed575bc060ce15d99c 2025-12-17 22:03:38,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:38,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ed575bc060ce15d99c HTTP/1.1" 502 150 2025-12-17 22:03:38,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ed575bc060ce15d99c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:38,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ed575bc060ce15d99c 2025-12-17 22:03:38,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ed575bc060ce15d99c HTTP/1.1" 502 150 2025-12-17 22:03:38,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ed575bc060ce15d99c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:38,912 - lava_callback - INFO - Received callback for node 694085ad575bc060ce15dcf5 INFO: 172.21.0.1:52242 - "POST /node/694085ad575bc060ce15dcf5 HTTP/1.0" 202 Accepted 2025-12-17 22:03:39,669 - lava_callback - INFO - Received callback for node 694085ae575bc060ce15dcfa INFO: 172.21.0.1:52250 - "POST /node/694085ae575bc060ce15dcfa HTTP/1.0" 202 Accepted 2025-12-17 22:03:39,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f4575bc060ce15d9ae 2025-12-17 22:03:39,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f4575bc060ce15d9ae HTTP/1.1" 502 150 2025-12-17 22:03:39,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f4575bc060ce15d9ae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:39,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f2575bc060ce15d9a8 2025-12-17 22:03:39,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f2575bc060ce15d9a8 HTTP/1.1" 502 150 2025-12-17 22:03:39,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f2575bc060ce15d9a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:40,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084d8575bc060ce15d94d HTTP/1.1" 502 150 2025-12-17 22:03:40,336 - lava_callback - ERROR - Error processing callback for node 694084d8575bc060ce15d94d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d8575bc060ce15d94d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d8575bc060ce15d94d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084d8575bc060ce15d94d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:40,338 - lava_callback - INFO - Starting async job submit for node 694084ee575bc060ce15d99f 2025-12-17 22:03:40,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:40,420 - lava_callback - INFO - Received callback for node 694085b2575bc060ce15dd05 INFO: 172.21.0.1:38114 - "POST /node/694085b2575bc060ce15dd05 HTTP/1.0" 202 Accepted 2025-12-17 22:03:40,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e0575bc060ce15d962 2025-12-17 22:03:40,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d962 HTTP/1.1" 502 150 2025-12-17 22:03:40,427 - lava_callback - ERROR - Error processing callback for node 694084e0575bc060ce15d962: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e0575bc060ce15d962 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e0575bc060ce15d962 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e0575bc060ce15d962 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:40,433 - lava_callback - INFO - Starting async job submit for node 694084ef575bc060ce15d9a2 2025-12-17 22:03:40,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:40,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ef575bc060ce15d9a2 HTTP/1.1" 502 150 2025-12-17 22:03:40,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ef575bc060ce15d9a2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:40,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ef575bc060ce15d9a2 2025-12-17 22:03:40,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ef575bc060ce15d9a2 HTTP/1.1" 502 150 2025-12-17 22:03:40,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ef575bc060ce15d9a2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:40,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f3575bc060ce15d9ab 2025-12-17 22:03:40,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f3575bc060ce15d9ab HTTP/1.1" 502 150 2025-12-17 22:03:40,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f3575bc060ce15d9ab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:40,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ed575bc060ce15d99c 2025-12-17 22:03:40,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ed575bc060ce15d99c HTTP/1.1" 502 150 2025-12-17 22:03:40,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ed575bc060ce15d99c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:41,178 - lava_callback - INFO - Received callback for node 694085b3575bc060ce15dd0c INFO: 172.21.0.1:38122 - "POST /node/694085b3575bc060ce15dd0c HTTP/1.0" 202 Accepted 2025-12-17 22:03:41,592 - lava_callback - INFO - Received callback for node 694084ae575bc060ce15d873 INFO: 172.21.0.1:38124 - "POST /node/694084ae575bc060ce15d873 HTTP/1.0" 202 Accepted 2025-12-17 22:03:41,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084eb575bc060ce15d992 2025-12-17 22:03:41,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084eb575bc060ce15d992 HTTP/1.1" 502 150 2025-12-17 22:03:41,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084eb575bc060ce15d992'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:41,926 - lava_callback - INFO - Received callback for node 694085b5575bc060ce15dd0f INFO: 172.21.0.1:38138 - "POST /node/694085b5575bc060ce15dd0f HTTP/1.0" 202 Accepted 2025-12-17 22:03:42,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ef575bc060ce15d9a2 2025-12-17 22:03:42,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ef575bc060ce15d9a2 HTTP/1.1" 502 150 2025-12-17 22:03:42,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ef575bc060ce15d9a2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:42,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e0575bc060ce15d965 2025-12-17 22:03:42,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e0575bc060ce15d965 HTTP/1.1" 502 150 2025-12-17 22:03:42,718 - lava_callback - ERROR - Error processing callback for node 694084e0575bc060ce15d965: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e0575bc060ce15d965 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e0575bc060ce15d965 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e0575bc060ce15d965 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:42,721 - lava_callback - INFO - Starting async job submit for node 694084f0575bc060ce15d9a5 2025-12-17 22:03:42,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:42,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f0575bc060ce15d9a5 HTTP/1.1" 502 150 2025-12-17 22:03:42,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f0575bc060ce15d9a5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:42,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f0575bc060ce15d9a5 2025-12-17 22:03:42,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f0575bc060ce15d9a5 HTTP/1.1" 502 150 2025-12-17 22:03:42,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f0575bc060ce15d9a5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:42,736 - lava_callback - INFO - Received callback for node 694085b6575bc060ce15dd12 INFO: 172.21.0.1:38142 - "POST /node/694085b6575bc060ce15dd12 HTTP/1.0" 202 Accepted 2025-12-17 22:03:42,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e1575bc060ce15d968 2025-12-17 22:03:42,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e1575bc060ce15d968 HTTP/1.1" 502 150 2025-12-17 22:03:42,814 - lava_callback - ERROR - Error processing callback for node 694084e1575bc060ce15d968: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e1575bc060ce15d968 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e1575bc060ce15d968 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e1575bc060ce15d968 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:42,815 - lava_callback - INFO - Starting async job submit for node 69408457575bc060ce15d728 2025-12-17 22:03:42,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:42,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408457575bc060ce15d728 HTTP/1.1" 502 150 2025-12-17 22:03:42,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408457575bc060ce15d728'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:42,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408457575bc060ce15d728 2025-12-17 22:03:42,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408457575bc060ce15d728 HTTP/1.1" 502 150 2025-12-17 22:03:42,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408457575bc060ce15d728'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:43,467 - lava_callback - INFO - Received callback for node 694085b7575bc060ce15dd15 INFO: 172.21.0.1:38150 - "POST /node/694085b7575bc060ce15dd15 HTTP/1.0" 202 Accepted 2025-12-17 22:03:43,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e6575bc060ce15d986 2025-12-17 22:03:43,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e6575bc060ce15d986 HTTP/1.1" 502 150 2025-12-17 22:03:43,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e6575bc060ce15d986'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:43,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e7575bc060ce15d989 2025-12-17 22:03:43,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e7575bc060ce15d989 HTTP/1.1" 502 150 2025-12-17 22:03:43,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e7575bc060ce15d989'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:43,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f4575bc060ce15d9ae 2025-12-17 22:03:43,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f4575bc060ce15d9ae HTTP/1.1" 502 150 2025-12-17 22:03:43,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f4575bc060ce15d9ae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:43,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f2575bc060ce15d9a8 2025-12-17 22:03:43,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f2575bc060ce15d9a8 HTTP/1.1" 502 150 2025-12-17 22:03:43,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f2575bc060ce15d9a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:44,313 - lava_callback - INFO - Received callback for node 694085b9575bc060ce15dd49 INFO: 172.21.0.1:38164 - "POST /node/694085b9575bc060ce15dd49 HTTP/1.0" 202 Accepted 2025-12-17 22:03:44,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e1575bc060ce15d96b 2025-12-17 22:03:44,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e1575bc060ce15d96b HTTP/1.1" 502 150 2025-12-17 22:03:44,564 - lava_callback - ERROR - Error processing callback for node 694084e1575bc060ce15d96b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e1575bc060ce15d96b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e1575bc060ce15d96b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e1575bc060ce15d96b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:44,565 - lava_callback - INFO - Starting async job submit for node 694084f6575bc060ce15d9b3 2025-12-17 22:03:44,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:44,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f6575bc060ce15d9b3 HTTP/1.1" 502 150 2025-12-17 22:03:44,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f6575bc060ce15d9b3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:44,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f6575bc060ce15d9b3 2025-12-17 22:03:44,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f6575bc060ce15d9b3 HTTP/1.1" 502 150 2025-12-17 22:03:44,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f6575bc060ce15d9b3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:44,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f0575bc060ce15d9a5 2025-12-17 22:03:44,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f0575bc060ce15d9a5 HTTP/1.1" 502 150 2025-12-17 22:03:44,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f0575bc060ce15d9a5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:44,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f3575bc060ce15d9ab 2025-12-17 22:03:44,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f3575bc060ce15d9ab HTTP/1.1" 502 150 2025-12-17 22:03:44,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f3575bc060ce15d9ab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:44,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ed575bc060ce15d99c 2025-12-17 22:03:44,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ed575bc060ce15d99c HTTP/1.1" 502 150 2025-12-17 22:03:44,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ed575bc060ce15d99c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:44,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408457575bc060ce15d728 2025-12-17 22:03:44,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408457575bc060ce15d728 HTTP/1.1" 502 150 2025-12-17 22:03:44,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408457575bc060ce15d728'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:45,065 - lava_callback - INFO - Received callback for node 694085bb575bc060ce15dd70 INFO: 172.21.0.1:38172 - "POST /node/694085bb575bc060ce15dd70 HTTP/1.0" 202 Accepted 2025-12-17 22:03:45,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e9575bc060ce15d98c 2025-12-17 22:03:45,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e9575bc060ce15d98c HTTP/1.1" 502 150 2025-12-17 22:03:45,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084e9575bc060ce15d98c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:45,802 - lava_callback - INFO - Received callback for node 694085bd575bc060ce15dd78 INFO: 172.21.0.1:38186 - "POST /node/694085bd575bc060ce15dd78 HTTP/1.0" 202 Accepted 2025-12-17 22:03:45,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ea575bc060ce15d98f 2025-12-17 22:03:45,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ea575bc060ce15d98f HTTP/1.1" 502 150 2025-12-17 22:03:45,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ea575bc060ce15d98f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:46,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ef575bc060ce15d9a2 2025-12-17 22:03:46,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ef575bc060ce15d9a2 HTTP/1.1" 502 150 2025-12-17 22:03:46,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ef575bc060ce15d9a2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38192 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:03:46,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e2575bc060ce15d970 2025-12-17 22:03:46,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e2575bc060ce15d970 HTTP/1.1" 502 150 2025-12-17 22:03:46,509 - lava_callback - ERROR - Error processing callback for node 694084e2575bc060ce15d970: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e2575bc060ce15d970 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e2575bc060ce15d970 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e2575bc060ce15d970 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:46,512 - lava_callback - INFO - Starting async job submit for node 694084f7575bc060ce15d9ba 2025-12-17 22:03:46,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:46,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f7575bc060ce15d9ba HTTP/1.1" 502 150 2025-12-17 22:03:46,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f7575bc060ce15d9ba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:46,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f7575bc060ce15d9ba 2025-12-17 22:03:46,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f7575bc060ce15d9ba HTTP/1.1" 502 150 2025-12-17 22:03:46,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f7575bc060ce15d9ba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:46,550 - lava_callback - INFO - Received callback for node 694085be575bc060ce15dd7d INFO: 172.21.0.1:38196 - "POST /node/694085be575bc060ce15dd7d HTTP/1.0" 202 Accepted 2025-12-17 22:03:46,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f6575bc060ce15d9b3 2025-12-17 22:03:46,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f6575bc060ce15d9b3 HTTP/1.1" 502 150 2025-12-17 22:03:46,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f6575bc060ce15d9b3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:47,294 - lava_callback - INFO - Received callback for node 694085bf575bc060ce15dd88 INFO: 172.21.0.1:38210 - "POST /node/694085bf575bc060ce15dd88 HTTP/1.0" 202 Accepted 2025-12-17 22:03:48,029 - lava_callback - INFO - Received callback for node 694085c1575bc060ce15dd8f INFO: 172.21.0.1:38222 - "POST /node/694085c1575bc060ce15dd8f HTTP/1.0" 202 Accepted 2025-12-17 22:03:48,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f7575bc060ce15d9ba 2025-12-17 22:03:48,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f7575bc060ce15d9ba HTTP/1.1" 502 150 2025-12-17 22:03:48,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f7575bc060ce15d9ba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:48,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f0575bc060ce15d9a5 2025-12-17 22:03:48,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f0575bc060ce15d9a5 HTTP/1.1" 502 150 2025-12-17 22:03:48,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f0575bc060ce15d9a5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:48,788 - lava_callback - INFO - Received callback for node 694085c2575bc060ce15dd96 INFO: 172.21.0.1:38234 - "POST /node/694085c2575bc060ce15dd96 HTTP/1.0" 202 Accepted 2025-12-17 22:03:48,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408457575bc060ce15d728 2025-12-17 22:03:48,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408457575bc060ce15d728 HTTP/1.1" 502 150 2025-12-17 22:03:48,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408457575bc060ce15d728'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:49,552 - lava_callback - INFO - Received callback for node 694085c4575bc060ce15dd9b INFO: 172.21.0.1:38248 - "POST /node/694085c4575bc060ce15dd9b HTTP/1.0" 202 Accepted 2025-12-17 22:03:49,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084eb575bc060ce15d992 2025-12-17 22:03:49,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084eb575bc060ce15d992 HTTP/1.1" 502 150 2025-12-17 22:03:49,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084eb575bc060ce15d992'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:50,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f6575bc060ce15d9b3 2025-12-17 22:03:50,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f6575bc060ce15d9b3 HTTP/1.1" 502 150 2025-12-17 22:03:50,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f6575bc060ce15d9b3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:50,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e5575bc060ce15d97f 2025-12-17 22:03:50,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e5575bc060ce15d97f HTTP/1.1" 502 150 2025-12-17 22:03:50,661 - lava_callback - ERROR - Error processing callback for node 694084e5575bc060ce15d97f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e5575bc060ce15d97f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e5575bc060ce15d97f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e5575bc060ce15d97f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:50,663 - lava_callback - INFO - Starting async job submit for node 694084f9575bc060ce15d9bd 2025-12-17 22:03:50,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:50,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f9575bc060ce15d9bd HTTP/1.1" 502 150 2025-12-17 22:03:50,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f9575bc060ce15d9bd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:50,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f9575bc060ce15d9bd 2025-12-17 22:03:50,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f9575bc060ce15d9bd HTTP/1.1" 502 150 2025-12-17 22:03:50,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f9575bc060ce15d9bd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:51,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f4575bc060ce15d9ae 2025-12-17 22:03:51,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f2575bc060ce15d9a8 2025-12-17 22:03:51,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f2575bc060ce15d9a8 HTTP/1.1" 502 150 2025-12-17 22:03:51,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f2575bc060ce15d9a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:52,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f7575bc060ce15d9ba 2025-12-17 22:03:52,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f7575bc060ce15d9ba HTTP/1.1" 502 150 2025-12-17 22:03:52,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f7575bc060ce15d9ba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:52,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f9575bc060ce15d9bd 2025-12-17 22:03:52,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f9575bc060ce15d9bd HTTP/1.1" 502 150 2025-12-17 22:03:52,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f9575bc060ce15d9bd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:52,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f3575bc060ce15d9ab 2025-12-17 22:03:52,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f3575bc060ce15d9ab HTTP/1.1" 502 150 2025-12-17 22:03:52,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f3575bc060ce15d9ab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:52,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ed575bc060ce15d99c 2025-12-17 22:03:52,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ed575bc060ce15d99c HTTP/1.1" 502 150 2025-12-17 22:03:52,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ed575bc060ce15d99c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:53,750 - lava_callback - INFO - Received callback for node 694085c4575bc060ce15dd9e INFO: 172.21.0.1:53138 - "POST /node/694085c4575bc060ce15dd9e HTTP/1.0" 202 Accepted 2025-12-17 22:03:54,403 - lava_callback - INFO - Received callback for node 694085c6575bc060ce15dda1 INFO: 172.21.0.1:53146 - "POST /node/694085c6575bc060ce15dda1 HTTP/1.0" 202 Accepted 2025-12-17 22:03:54,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ef575bc060ce15d9a2 2025-12-17 22:03:54,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ef575bc060ce15d9a2 HTTP/1.1" 502 150 2025-12-17 22:03:54,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ef575bc060ce15d9a2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53160 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:03:56,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f9575bc060ce15d9bd 2025-12-17 22:03:56,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f9575bc060ce15d9bd HTTP/1.1" 502 150 2025-12-17 22:03:56,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f9575bc060ce15d9bd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:56,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f0575bc060ce15d9a5 2025-12-17 22:03:56,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f0575bc060ce15d9a5 HTTP/1.1" 502 150 2025-12-17 22:03:56,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f0575bc060ce15d9a5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:56,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408457575bc060ce15d728 2025-12-17 22:03:56,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408457575bc060ce15d728 HTTP/1.1" 502 150 2025-12-17 22:03:56,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408457575bc060ce15d728'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:58,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f6575bc060ce15d9b3 2025-12-17 22:03:58,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f6575bc060ce15d9b3 HTTP/1.1" 502 150 2025-12-17 22:03:58,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f6575bc060ce15d9b3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:59,267 - lava_callback - INFO - Received callback for node 694085c8575bc060ce15dda4 INFO: 172.21.0.1:53176 - "POST /node/694085c8575bc060ce15dda4 HTTP/1.0" 202 Accepted 2025-12-17 22:03:59,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e6575bc060ce15d986 2025-12-17 22:03:59,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e7575bc060ce15d989 2025-12-17 22:03:59,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e7575bc060ce15d989 HTTP/1.1" 502 150 2025-12-17 22:03:59,605 - lava_callback - ERROR - Error processing callback for node 694084e7575bc060ce15d989: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e7575bc060ce15d989 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e7575bc060ce15d989 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e7575bc060ce15d989 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:03:59,606 - lava_callback - INFO - Starting async job submit for node 694084fa575bc060ce15d9c0 2025-12-17 22:03:59,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:03:59,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084fa575bc060ce15d9c0 HTTP/1.1" 502 150 2025-12-17 22:03:59,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084fa575bc060ce15d9c0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:03:59,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084fa575bc060ce15d9c0 2025-12-17 22:03:59,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084fa575bc060ce15d9c0 HTTP/1.1" 502 150 2025-12-17 22:03:59,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084fa575bc060ce15d9c0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:00,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f7575bc060ce15d9ba 2025-12-17 22:04:00,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f7575bc060ce15d9ba HTTP/1.1" 502 150 2025-12-17 22:04:00,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f7575bc060ce15d9ba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:01,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084e9575bc060ce15d98c 2025-12-17 22:04:01,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e9575bc060ce15d98c HTTP/1.1" 502 150 2025-12-17 22:04:01,495 - lava_callback - ERROR - Error processing callback for node 694084e9575bc060ce15d98c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e9575bc060ce15d98c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e9575bc060ce15d98c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e9575bc060ce15d98c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:01,497 - lava_callback - INFO - Starting async job submit for node 694084fc575bc060ce15d9e1 2025-12-17 22:04:01,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:01,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084fc575bc060ce15d9e1 HTTP/1.1" 502 150 2025-12-17 22:04:01,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084fc575bc060ce15d9e1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:01,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084fc575bc060ce15d9e1 2025-12-17 22:04:01,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084fc575bc060ce15d9e1 HTTP/1.1" 502 150 2025-12-17 22:04:01,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084fc575bc060ce15d9e1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:01,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084fa575bc060ce15d9c0 2025-12-17 22:04:01,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084fa575bc060ce15d9c0 HTTP/1.1" 502 150 2025-12-17 22:04:01,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084fa575bc060ce15d9c0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:01,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ea575bc060ce15d98f 2025-12-17 22:04:01,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ea575bc060ce15d98f HTTP/1.1" 502 150 2025-12-17 22:04:01,954 - lava_callback - ERROR - Error processing callback for node 694084ea575bc060ce15d98f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ea575bc060ce15d98f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ea575bc060ce15d98f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ea575bc060ce15d98f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:01,956 - lava_callback - INFO - Starting async job submit for node 69408500575bc060ce15da26 2025-12-17 22:04:01,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:01,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408500575bc060ce15da26 HTTP/1.1" 502 150 2025-12-17 22:04:01,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408500575bc060ce15da26'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:01,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408500575bc060ce15da26 2025-12-17 22:04:01,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408500575bc060ce15da26 HTTP/1.1" 502 150 2025-12-17 22:04:01,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408500575bc060ce15da26'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60518 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:04:03,087 - lava_callback - INFO - Received callback for node 6940c5331e3c24e95681d6a4 INFO: 172.21.0.1:60532 - "POST /node/6940c5331e3c24e95681d6a4 HTTP/1.0" 202 Accepted 2025-12-17 22:04:03,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084fc575bc060ce15d9e1 2025-12-17 22:04:03,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084fc575bc060ce15d9e1 HTTP/1.1" 502 150 2025-12-17 22:04:03,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084fc575bc060ce15d9e1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:03,582 - lava_callback - INFO - Received callback for node 694085d0575bc060ce15ddb9 INFO: 172.21.0.1:60542 - "POST /node/694085d0575bc060ce15ddb9 HTTP/1.0" 202 Accepted 2025-12-17 22:04:03,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408500575bc060ce15da26 2025-12-17 22:04:03,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408500575bc060ce15da26 HTTP/1.1" 502 150 2025-12-17 22:04:03,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408500575bc060ce15da26'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:04,237 - lava_callback - INFO - Received callback for node 694085d2575bc060ce15ddbe INFO: 172.21.0.1:60556 - "POST /node/694085d2575bc060ce15ddbe HTTP/1.0" 202 Accepted 2025-12-17 22:04:04,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f9575bc060ce15d9bd 2025-12-17 22:04:04,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f9575bc060ce15d9bd HTTP/1.1" 502 150 2025-12-17 22:04:04,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f9575bc060ce15d9bd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:05,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084fa575bc060ce15d9c0 2025-12-17 22:04:05,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084fa575bc060ce15d9c0 HTTP/1.1" 502 150 2025-12-17 22:04:05,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084fa575bc060ce15d9c0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:05,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084eb575bc060ce15d992 2025-12-17 22:04:05,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084eb575bc060ce15d992 HTTP/1.1" 502 150 2025-12-17 22:04:05,928 - lava_callback - ERROR - Error processing callback for node 694084eb575bc060ce15d992: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084eb575bc060ce15d992 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084eb575bc060ce15d992 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084eb575bc060ce15d992 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:05,930 - lava_callback - INFO - Starting async job submit for node 69408501575bc060ce15da29 2025-12-17 22:04:05,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:05,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408501575bc060ce15da29 HTTP/1.1" 502 150 2025-12-17 22:04:05,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408501575bc060ce15da29'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:05,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408501575bc060ce15da29 2025-12-17 22:04:05,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408501575bc060ce15da29 HTTP/1.1" 502 150 2025-12-17 22:04:05,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408501575bc060ce15da29'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60560 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:04:07,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f4575bc060ce15d9ae HTTP/1.1" 502 150 2025-12-17 22:04:07,216 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084f4575bc060ce15d9ae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:07,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084fc575bc060ce15d9e1 2025-12-17 22:04:07,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084fc575bc060ce15d9e1 HTTP/1.1" 502 150 2025-12-17 22:04:07,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084fc575bc060ce15d9e1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:07,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f2575bc060ce15d9a8 2025-12-17 22:04:07,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f2575bc060ce15d9a8 HTTP/1.1" 502 150 2025-12-17 22:04:07,934 - lava_callback - ERROR - Error processing callback for node 694084f2575bc060ce15d9a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f2575bc060ce15d9a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f2575bc060ce15d9a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f2575bc060ce15d9a8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:07,936 - lava_callback - INFO - Starting async job submit for node 694084ff575bc060ce15da18 2025-12-17 22:04:07,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:07,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408501575bc060ce15da29 2025-12-17 22:04:07,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408501575bc060ce15da29 HTTP/1.1" 502 150 2025-12-17 22:04:07,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408501575bc060ce15da29'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:07,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ff575bc060ce15da18 HTTP/1.1" 502 150 2025-12-17 22:04:07,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ff575bc060ce15da18'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:07,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ff575bc060ce15da18 2025-12-17 22:04:07,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ff575bc060ce15da18 HTTP/1.1" 502 150 2025-12-17 22:04:07,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ff575bc060ce15da18'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:07,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408500575bc060ce15da26 2025-12-17 22:04:07,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408500575bc060ce15da26 HTTP/1.1" 502 150 2025-12-17 22:04:07,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408500575bc060ce15da26'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:08,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f3575bc060ce15d9ab 2025-12-17 22:04:08,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f3575bc060ce15d9ab HTTP/1.1" 502 150 2025-12-17 22:04:08,877 - lava_callback - ERROR - Error processing callback for node 694084f3575bc060ce15d9ab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f3575bc060ce15d9ab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f3575bc060ce15d9ab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f3575bc060ce15d9ab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:08,879 - lava_callback - INFO - Starting async job submit for node 6940850d575bc060ce15da51 2025-12-17 22:04:08,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:08,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850d575bc060ce15da51 HTTP/1.1" 502 150 2025-12-17 22:04:08,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850d575bc060ce15da51'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:08,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850d575bc060ce15da51 2025-12-17 22:04:08,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850d575bc060ce15da51 HTTP/1.1" 502 150 2025-12-17 22:04:08,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850d575bc060ce15da51'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:08,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ed575bc060ce15d99c 2025-12-17 22:04:08,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ed575bc060ce15d99c HTTP/1.1" 502 150 2025-12-17 22:04:08,910 - lava_callback - ERROR - Error processing callback for node 694084ed575bc060ce15d99c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ed575bc060ce15d99c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ed575bc060ce15d99c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ed575bc060ce15d99c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:08,912 - lava_callback - INFO - Starting async job submit for node 6940850e575bc060ce15da58 2025-12-17 22:04:08,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:08,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850e575bc060ce15da58 HTTP/1.1" 502 150 2025-12-17 22:04:08,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850e575bc060ce15da58'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:08,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850e575bc060ce15da58 2025-12-17 22:04:08,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850e575bc060ce15da58 HTTP/1.1" 502 150 2025-12-17 22:04:08,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850e575bc060ce15da58'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:09,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ff575bc060ce15da18 2025-12-17 22:04:09,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ff575bc060ce15da18 HTTP/1.1" 502 150 2025-12-17 22:04:09,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ff575bc060ce15da18'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:10,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ef575bc060ce15d9a2 2025-12-17 22:04:10,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ef575bc060ce15d9a2 HTTP/1.1" 502 150 2025-12-17 22:04:10,457 - lava_callback - ERROR - Error processing callback for node 694084ef575bc060ce15d9a2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ef575bc060ce15d9a2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ef575bc060ce15d9a2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ef575bc060ce15d9a2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:10,459 - lava_callback - INFO - Starting async job submit for node 69408503575bc060ce15da30 2025-12-17 22:04:10,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:10,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408503575bc060ce15da30 HTTP/1.1" 502 150 2025-12-17 22:04:10,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408503575bc060ce15da30'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:10,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408503575bc060ce15da30 2025-12-17 22:04:10,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408503575bc060ce15da30 HTTP/1.1" 502 150 2025-12-17 22:04:10,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408503575bc060ce15da30'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:10,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850d575bc060ce15da51 2025-12-17 22:04:10,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850d575bc060ce15da51 HTTP/1.1" 502 150 2025-12-17 22:04:10,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850d575bc060ce15da51'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:10,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850e575bc060ce15da58 2025-12-17 22:04:10,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850e575bc060ce15da58 HTTP/1.1" 502 150 2025-12-17 22:04:10,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850e575bc060ce15da58'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:11,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ee575bc060ce15d99f HTTP/1.1" 502 150 2025-12-17 22:04:11,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ee575bc060ce15d99f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:11,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ee575bc060ce15d99f 2025-12-17 22:04:11,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408501575bc060ce15da29 2025-12-17 22:04:11,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408501575bc060ce15da29 HTTP/1.1" 502 150 2025-12-17 22:04:11,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408501575bc060ce15da29'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:12,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408503575bc060ce15da30 2025-12-17 22:04:12,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408503575bc060ce15da30 HTTP/1.1" 502 150 2025-12-17 22:04:12,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408503575bc060ce15da30'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:12,624 - lava_callback - INFO - Received callback for node 694085cc575bc060ce15ddad INFO: 172.21.0.1:39408 - "POST /node/694085cc575bc060ce15ddad HTTP/1.0" 202 Accepted 2025-12-17 22:04:12,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f0575bc060ce15d9a5 2025-12-17 22:04:12,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f0575bc060ce15d9a5 HTTP/1.1" 502 150 2025-12-17 22:04:12,736 - lava_callback - ERROR - Error processing callback for node 694084f0575bc060ce15d9a5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f0575bc060ce15d9a5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f0575bc060ce15d9a5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f0575bc060ce15d9a5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:12,739 - lava_callback - INFO - Starting async job submit for node 69408505575bc060ce15da33 2025-12-17 22:04:12,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:12,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408505575bc060ce15da33 HTTP/1.1" 502 150 2025-12-17 22:04:12,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408505575bc060ce15da33'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:12,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408505575bc060ce15da33 2025-12-17 22:04:12,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408505575bc060ce15da33 HTTP/1.1" 502 150 2025-12-17 22:04:12,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408505575bc060ce15da33'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:12,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408457575bc060ce15d728 2025-12-17 22:04:12,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408457575bc060ce15d728 HTTP/1.1" 502 150 2025-12-17 22:04:12,946 - lava_callback - ERROR - Error processing callback for node 69408457575bc060ce15d728: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408457575bc060ce15d728 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408457575bc060ce15d728 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408457575bc060ce15d728 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:12,948 - lava_callback - INFO - Starting async job submit for node 69408506575bc060ce15da36 2025-12-17 22:04:12,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:12,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408506575bc060ce15da36 HTTP/1.1" 502 150 2025-12-17 22:04:12,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408506575bc060ce15da36'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:12,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408506575bc060ce15da36 2025-12-17 22:04:12,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408506575bc060ce15da36 HTTP/1.1" 502 150 2025-12-17 22:04:12,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408506575bc060ce15da36'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:13,252 - lava_callback - INFO - Received callback for node 694085d0575bc060ce15ddb6 INFO: 172.21.0.1:39420 - "POST /node/694085d0575bc060ce15ddb6 HTTP/1.0" 202 Accepted 2025-12-17 22:04:13,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084fa575bc060ce15d9c0 2025-12-17 22:04:13,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084fa575bc060ce15d9c0 HTTP/1.1" 502 150 2025-12-17 22:04:13,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084fa575bc060ce15d9c0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:13,728 - lava_callback - INFO - Received callback for node 694085c9575bc060ce15dda7 INFO: 172.21.0.1:39434 - "POST /node/694085c9575bc060ce15dda7 HTTP/1.0" 202 Accepted 2025-12-17 22:04:13,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ff575bc060ce15da18 2025-12-17 22:04:13,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ff575bc060ce15da18 HTTP/1.1" 502 150 2025-12-17 22:04:13,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ff575bc060ce15da18'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:14,386 - lava_callback - INFO - Received callback for node 694085ca575bc060ce15ddaa INFO: 172.21.0.1:39438 - "POST /node/694085ca575bc060ce15ddaa HTTP/1.0" 202 Accepted 2025-12-17 22:04:14,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f6575bc060ce15d9b3 2025-12-17 22:04:14,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f6575bc060ce15d9b3 HTTP/1.1" 502 150 2025-12-17 22:04:14,592 - lava_callback - ERROR - Error processing callback for node 694084f6575bc060ce15d9b3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f6575bc060ce15d9b3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f6575bc060ce15d9b3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f6575bc060ce15d9b3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:14,594 - lava_callback - INFO - Starting async job submit for node 69408508575bc060ce15da39 2025-12-17 22:04:14,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:14,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408508575bc060ce15da39 HTTP/1.1" 502 150 2025-12-17 22:04:14,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408508575bc060ce15da39'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:14,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408508575bc060ce15da39 2025-12-17 22:04:14,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408508575bc060ce15da39 HTTP/1.1" 502 150 2025-12-17 22:04:14,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408508575bc060ce15da39'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:14,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408505575bc060ce15da33 2025-12-17 22:04:14,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408505575bc060ce15da33 HTTP/1.1" 502 150 2025-12-17 22:04:14,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408505575bc060ce15da33'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:14,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850d575bc060ce15da51 2025-12-17 22:04:14,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850d575bc060ce15da51 HTTP/1.1" 502 150 2025-12-17 22:04:14,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850d575bc060ce15da51'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:14,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084e6575bc060ce15d986 HTTP/1.1" 502 150 2025-12-17 22:04:14,896 - lava_callback - ERROR - Error processing callback for node 694084e6575bc060ce15d986: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e6575bc060ce15d986 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e6575bc060ce15d986 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084e6575bc060ce15d986 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:14,898 - lava_callback - INFO - Starting async job submit for node 69408509575bc060ce15da3c 2025-12-17 22:04:14,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:14,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408509575bc060ce15da3c HTTP/1.1" 502 150 2025-12-17 22:04:14,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408509575bc060ce15da3c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:14,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408509575bc060ce15da3c 2025-12-17 22:04:14,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408509575bc060ce15da3c HTTP/1.1" 502 150 2025-12-17 22:04:14,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408509575bc060ce15da3c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:14,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850e575bc060ce15da58 2025-12-17 22:04:14,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850e575bc060ce15da58 HTTP/1.1" 502 150 2025-12-17 22:04:14,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850e575bc060ce15da58'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:14,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408506575bc060ce15da36 2025-12-17 22:04:14,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408506575bc060ce15da36 HTTP/1.1" 502 150 2025-12-17 22:04:14,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408506575bc060ce15da36'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39450 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:04:15,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084fc575bc060ce15d9e1 2025-12-17 22:04:15,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084fc575bc060ce15d9e1 HTTP/1.1" 502 150 2025-12-17 22:04:15,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084fc575bc060ce15d9e1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:15,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408500575bc060ce15da26 2025-12-17 22:04:15,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408500575bc060ce15da26 HTTP/1.1" 502 150 2025-12-17 22:04:15,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408500575bc060ce15da26'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:16,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408503575bc060ce15da30 2025-12-17 22:04:16,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408503575bc060ce15da30 HTTP/1.1" 502 150 2025-12-17 22:04:16,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408503575bc060ce15da30'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39454 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:04:16,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f7575bc060ce15d9ba 2025-12-17 22:04:16,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f7575bc060ce15d9ba HTTP/1.1" 502 150 2025-12-17 22:04:16,525 - lava_callback - ERROR - Error processing callback for node 694084f7575bc060ce15d9ba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f7575bc060ce15d9ba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f7575bc060ce15d9ba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f7575bc060ce15d9ba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:16,527 - lava_callback - INFO - Starting async job submit for node 6940850a575bc060ce15da43 2025-12-17 22:04:16,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:16,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850a575bc060ce15da43 HTTP/1.1" 502 150 2025-12-17 22:04:16,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850a575bc060ce15da43'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:16,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850a575bc060ce15da43 2025-12-17 22:04:16,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850a575bc060ce15da43 HTTP/1.1" 502 150 2025-12-17 22:04:16,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850a575bc060ce15da43'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:16,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408508575bc060ce15da39 2025-12-17 22:04:16,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408508575bc060ce15da39 HTTP/1.1" 502 150 2025-12-17 22:04:16,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408508575bc060ce15da39'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:16,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408509575bc060ce15da3c 2025-12-17 22:04:16,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408509575bc060ce15da3c HTTP/1.1" 502 150 2025-12-17 22:04:16,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408509575bc060ce15da3c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:18,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850a575bc060ce15da43 2025-12-17 22:04:18,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850a575bc060ce15da43 HTTP/1.1" 502 150 2025-12-17 22:04:18,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850a575bc060ce15da43'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:18,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408505575bc060ce15da33 2025-12-17 22:04:18,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408505575bc060ce15da33 HTTP/1.1" 502 150 2025-12-17 22:04:18,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408505575bc060ce15da33'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:18,793 - lava_callback - INFO - Received callback for node 694085cf575bc060ce15ddb3 INFO: 172.21.0.1:39460 - "POST /node/694085cf575bc060ce15ddb3 HTTP/1.0" 202 Accepted 2025-12-17 22:04:18,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408506575bc060ce15da36 2025-12-17 22:04:18,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408506575bc060ce15da36 HTTP/1.1" 502 150 2025-12-17 22:04:18,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408506575bc060ce15da36'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:19,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408501575bc060ce15da29 2025-12-17 22:04:19,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408501575bc060ce15da29 HTTP/1.1" 502 150 2025-12-17 22:04:19,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408501575bc060ce15da29'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:20,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408508575bc060ce15da39 2025-12-17 22:04:20,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408508575bc060ce15da39 HTTP/1.1" 502 150 2025-12-17 22:04:20,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408508575bc060ce15da39'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:20,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f9575bc060ce15d9bd 2025-12-17 22:04:20,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f9575bc060ce15d9bd HTTP/1.1" 502 150 2025-12-17 22:04:20,678 - lava_callback - ERROR - Error processing callback for node 694084f9575bc060ce15d9bd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f9575bc060ce15d9bd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f9575bc060ce15d9bd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f9575bc060ce15d9bd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:20,680 - lava_callback - INFO - Starting async job submit for node 6940850c575bc060ce15da4c 2025-12-17 22:04:20,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:20,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850c575bc060ce15da4c HTTP/1.1" 502 150 2025-12-17 22:04:20,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850c575bc060ce15da4c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:20,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850c575bc060ce15da4c 2025-12-17 22:04:20,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850c575bc060ce15da4c HTTP/1.1" 502 150 2025-12-17 22:04:20,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850c575bc060ce15da4c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:20,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408509575bc060ce15da3c 2025-12-17 22:04:20,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408509575bc060ce15da3c HTTP/1.1" 502 150 2025-12-17 22:04:20,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408509575bc060ce15da3c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:21,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ff575bc060ce15da18 2025-12-17 22:04:21,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ff575bc060ce15da18 HTTP/1.1" 502 150 2025-12-17 22:04:21,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ff575bc060ce15da18'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:22,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850a575bc060ce15da43 2025-12-17 22:04:22,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850a575bc060ce15da43 HTTP/1.1" 502 150 2025-12-17 22:04:22,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850a575bc060ce15da43'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:22,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850c575bc060ce15da4c 2025-12-17 22:04:22,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850c575bc060ce15da4c HTTP/1.1" 502 150 2025-12-17 22:04:22,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850c575bc060ce15da4c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:22,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850d575bc060ce15da51 2025-12-17 22:04:22,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850d575bc060ce15da51 HTTP/1.1" 502 150 2025-12-17 22:04:22,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850d575bc060ce15da51'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:22,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850e575bc060ce15da58 2025-12-17 22:04:22,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850e575bc060ce15da58 HTTP/1.1" 502 150 2025-12-17 22:04:22,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850e575bc060ce15da58'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:23,217 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084f4575bc060ce15d9ae 2025-12-17 22:04:23,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084f4575bc060ce15d9ae HTTP/1.1" 502 150 2025-12-17 22:04:23,224 - lava_callback - ERROR - Error processing callback for node 694084f4575bc060ce15d9ae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f4575bc060ce15d9ae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f4575bc060ce15d9ae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084f4575bc060ce15d9ae (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:23,226 - lava_callback - INFO - Starting async job submit for node 69408512575bc060ce15da64 2025-12-17 22:04:23,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:23,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408512575bc060ce15da64 HTTP/1.1" 502 150 2025-12-17 22:04:23,232 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408512575bc060ce15da64'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:23,232 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408512575bc060ce15da64 2025-12-17 22:04:23,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408512575bc060ce15da64 HTTP/1.1" 502 150 2025-12-17 22:04:23,233 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408512575bc060ce15da64'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:23,585 - lava_callback - INFO - Received callback for node 694085d3575bc060ce15ddc5 INFO: 172.21.0.1:44300 - "POST /node/694085d3575bc060ce15ddc5 HTTP/1.0" 202 Accepted 2025-12-17 22:04:24,229 - lava_callback - INFO - Received callback for node 694085d5575bc060ce15ddcc INFO: 172.21.0.1:44308 - "POST /node/694085d5575bc060ce15ddcc HTTP/1.0" 202 Accepted 2025-12-17 22:04:24,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408503575bc060ce15da30 2025-12-17 22:04:24,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408503575bc060ce15da30 HTTP/1.1" 502 150 2025-12-17 22:04:24,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408503575bc060ce15da30'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:24,856 - lava_callback - INFO - Received callback for node 694085d6575bc060ce15ddd3 INFO: 172.21.0.1:44314 - "POST /node/694085d6575bc060ce15ddd3 HTTP/1.0" 202 Accepted 2025-12-17 22:04:25,233 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408512575bc060ce15da64 2025-12-17 22:04:25,485 - lava_callback - INFO - Received callback for node 694085d7575bc060ce15ddda INFO: 172.21.0.1:44328 - "POST /node/694085d7575bc060ce15ddda HTTP/1.0" 202 Accepted 2025-12-17 22:04:26,144 - lava_callback - INFO - Received callback for node 694085d9575bc060ce15dde1 INFO: 172.21.0.1:44338 - "POST /node/694085d9575bc060ce15dde1 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:44348 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:04:26,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850c575bc060ce15da4c 2025-12-17 22:04:26,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850c575bc060ce15da4c HTTP/1.1" 502 150 2025-12-17 22:04:26,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850c575bc060ce15da4c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:26,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408505575bc060ce15da33 2025-12-17 22:04:26,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408505575bc060ce15da33 HTTP/1.1" 502 150 2025-12-17 22:04:26,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408505575bc060ce15da33'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:26,788 - lava_callback - INFO - Received callback for node 694085da575bc060ce15dde4 INFO: 172.21.0.1:44350 - "POST /node/694085da575bc060ce15dde4 HTTP/1.0" 202 Accepted 2025-12-17 22:04:26,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408506575bc060ce15da36 2025-12-17 22:04:26,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408506575bc060ce15da36 HTTP/1.1" 502 150 2025-12-17 22:04:26,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408506575bc060ce15da36'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:28,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408508575bc060ce15da39 2025-12-17 22:04:28,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408508575bc060ce15da39 HTTP/1.1" 502 150 2025-12-17 22:04:28,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408508575bc060ce15da39'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:28,691 - lava_callback - INFO - Received callback for node 694085db575bc060ce15dde7 INFO: 172.21.0.1:44352 - "POST /node/694085db575bc060ce15dde7 HTTP/1.0" 202 Accepted 2025-12-17 22:04:28,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408509575bc060ce15da3c 2025-12-17 22:04:28,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408509575bc060ce15da3c HTTP/1.1" 502 150 2025-12-17 22:04:28,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408509575bc060ce15da3c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:29,347 - lava_callback - INFO - Received callback for node 694085dc575bc060ce15ddea INFO: 172.21.0.1:44368 - "POST /node/694085dc575bc060ce15ddea HTTP/1.0" 202 Accepted 2025-12-17 22:04:29,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084fa575bc060ce15d9c0 2025-12-17 22:04:29,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084fa575bc060ce15d9c0 HTTP/1.1" 502 150 2025-12-17 22:04:29,621 - lava_callback - ERROR - Error processing callback for node 694084fa575bc060ce15d9c0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084fa575bc060ce15d9c0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084fa575bc060ce15d9c0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084fa575bc060ce15d9c0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:29,624 - lava_callback - INFO - Starting async job submit for node 69408459575bc060ce15d72f 2025-12-17 22:04:29,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:29,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408459575bc060ce15d72f HTTP/1.1" 502 150 2025-12-17 22:04:29,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408459575bc060ce15d72f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:29,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408459575bc060ce15d72f 2025-12-17 22:04:29,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408459575bc060ce15d72f HTTP/1.1" 502 150 2025-12-17 22:04:29,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408459575bc060ce15d72f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:29,993 - lava_callback - INFO - Received callback for node 694085dd575bc060ce15ddf1 INFO: 172.21.0.1:57272 - "POST /node/694085dd575bc060ce15ddf1 HTTP/1.0" 202 Accepted 2025-12-17 22:04:30,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850a575bc060ce15da43 2025-12-17 22:04:30,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850a575bc060ce15da43 HTTP/1.1" 502 150 2025-12-17 22:04:30,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850a575bc060ce15da43'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:30,671 - lava_callback - INFO - Received callback for node 694085df575bc060ce15ddf4 INFO: 172.21.0.1:57280 - "POST /node/694085df575bc060ce15ddf4 HTTP/1.0" 202 Accepted 2025-12-17 22:04:31,339 - lava_callback - INFO - Received callback for node 694085e9575bc060ce15de0f INFO: 172.21.0.1:57290 - "POST /node/694085e9575bc060ce15de0f HTTP/1.0" 202 Accepted 2025-12-17 22:04:31,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084fc575bc060ce15d9e1 2025-12-17 22:04:31,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084fc575bc060ce15d9e1 HTTP/1.1" 502 150 2025-12-17 22:04:31,513 - lava_callback - ERROR - Error processing callback for node 694084fc575bc060ce15d9e1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084fc575bc060ce15d9e1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084fc575bc060ce15d9e1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084fc575bc060ce15d9e1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:31,515 - lava_callback - INFO - Starting async job submit for node 6940850f575bc060ce15da5d 2025-12-17 22:04:31,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:31,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850f575bc060ce15da5d HTTP/1.1" 502 150 2025-12-17 22:04:31,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850f575bc060ce15da5d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:31,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850f575bc060ce15da5d 2025-12-17 22:04:31,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850f575bc060ce15da5d HTTP/1.1" 502 150 2025-12-17 22:04:31,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850f575bc060ce15da5d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:31,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408459575bc060ce15d72f 2025-12-17 22:04:31,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408459575bc060ce15d72f HTTP/1.1" 502 150 2025-12-17 22:04:31,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408459575bc060ce15d72f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:31,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408500575bc060ce15da26 2025-12-17 22:04:31,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408500575bc060ce15da26 HTTP/1.1" 502 150 2025-12-17 22:04:31,976 - lava_callback - ERROR - Error processing callback for node 69408500575bc060ce15da26: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408500575bc060ce15da26 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408500575bc060ce15da26 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408500575bc060ce15da26 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:31,979 - lava_callback - INFO - Starting async job submit for node 69408513575bc060ce15da67 2025-12-17 22:04:31,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:33,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850f575bc060ce15da5d 2025-12-17 22:04:33,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850f575bc060ce15da5d HTTP/1.1" 502 150 2025-12-17 22:04:33,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850f575bc060ce15da5d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:34,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850c575bc060ce15da4c 2025-12-17 22:04:34,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850c575bc060ce15da4c HTTP/1.1" 502 150 2025-12-17 22:04:34,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850c575bc060ce15da4c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:35,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408459575bc060ce15d72f 2025-12-17 22:04:35,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408459575bc060ce15d72f HTTP/1.1" 502 150 2025-12-17 22:04:35,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408459575bc060ce15d72f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:35,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408501575bc060ce15da29 2025-12-17 22:04:35,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408501575bc060ce15da29 HTTP/1.1" 502 150 2025-12-17 22:04:35,945 - lava_callback - ERROR - Error processing callback for node 69408501575bc060ce15da29: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408501575bc060ce15da29 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408501575bc060ce15da29 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408501575bc060ce15da29 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:35,946 - lava_callback - INFO - Starting async job submit for node 69408514575bc060ce15da6a 2025-12-17 22:04:35,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:35,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408514575bc060ce15da6a HTTP/1.1" 502 150 2025-12-17 22:04:35,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408514575bc060ce15da6a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:35,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408514575bc060ce15da6a 2025-12-17 22:04:35,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408514575bc060ce15da6a HTTP/1.1" 502 150 2025-12-17 22:04:35,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408514575bc060ce15da6a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57292 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:04:37,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850f575bc060ce15da5d 2025-12-17 22:04:37,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850f575bc060ce15da5d HTTP/1.1" 502 150 2025-12-17 22:04:37,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850f575bc060ce15da5d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:37,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ff575bc060ce15da18 2025-12-17 22:04:37,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ff575bc060ce15da18 HTTP/1.1" 502 150 2025-12-17 22:04:37,971 - lava_callback - ERROR - Error processing callback for node 694084ff575bc060ce15da18: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ff575bc060ce15da18 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ff575bc060ce15da18 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ff575bc060ce15da18 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:37,973 - lava_callback - INFO - Starting async job submit for node 69408515575bc060ce15da71 2025-12-17 22:04:37,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:37,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408515575bc060ce15da71 HTTP/1.1" 502 150 2025-12-17 22:04:37,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408515575bc060ce15da71'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:37,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408515575bc060ce15da71 2025-12-17 22:04:37,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408515575bc060ce15da71 HTTP/1.1" 502 150 2025-12-17 22:04:37,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408515575bc060ce15da71'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:37,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408514575bc060ce15da6a 2025-12-17 22:04:37,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408514575bc060ce15da6a HTTP/1.1" 502 150 2025-12-17 22:04:37,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408514575bc060ce15da6a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:38,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850d575bc060ce15da51 2025-12-17 22:04:38,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850d575bc060ce15da51 HTTP/1.1" 502 150 2025-12-17 22:04:38,893 - lava_callback - ERROR - Error processing callback for node 6940850d575bc060ce15da51: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940850d575bc060ce15da51 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940850d575bc060ce15da51 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940850d575bc060ce15da51 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:38,894 - lava_callback - INFO - Starting async job submit for node 69408516575bc060ce15da74 2025-12-17 22:04:38,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:38,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408516575bc060ce15da74 HTTP/1.1" 502 150 2025-12-17 22:04:38,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408516575bc060ce15da74'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:38,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408516575bc060ce15da74 2025-12-17 22:04:38,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408516575bc060ce15da74 HTTP/1.1" 502 150 2025-12-17 22:04:38,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408516575bc060ce15da74'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:38,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850e575bc060ce15da58 2025-12-17 22:04:38,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850e575bc060ce15da58 HTTP/1.1" 502 150 2025-12-17 22:04:38,946 - lava_callback - ERROR - Error processing callback for node 6940850e575bc060ce15da58: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940850e575bc060ce15da58 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940850e575bc060ce15da58 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940850e575bc060ce15da58 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:38,948 - lava_callback - INFO - Starting async job submit for node 69408518575bc060ce15da77 2025-12-17 22:04:38,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:38,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408518575bc060ce15da77 HTTP/1.1" 502 150 2025-12-17 22:04:38,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408518575bc060ce15da77'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:38,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408518575bc060ce15da77 2025-12-17 22:04:38,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408518575bc060ce15da77 HTTP/1.1" 502 150 2025-12-17 22:04:38,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408518575bc060ce15da77'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:38,971 - lava_callback - INFO - Received callback for node 694085e5575bc060ce15ddf7 INFO: 172.21.0.1:57302 - "POST /node/694085e5575bc060ce15ddf7 HTTP/1.0" 202 Accepted 2025-12-17 22:04:39,784 - lava_callback - INFO - Received callback for node 694085e6575bc060ce15ddfc INFO: 172.21.0.1:46618 - "POST /node/694085e6575bc060ce15ddfc HTTP/1.0" 202 Accepted 2025-12-17 22:04:39,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408515575bc060ce15da71 2025-12-17 22:04:39,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408515575bc060ce15da71 HTTP/1.1" 502 150 2025-12-17 22:04:39,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408515575bc060ce15da71'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:40,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408503575bc060ce15da30 2025-12-17 22:04:40,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408512575bc060ce15da64 HTTP/1.1" 502 150 2025-12-17 22:04:40,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408512575bc060ce15da64'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:40,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408503575bc060ce15da30 HTTP/1.1" 502 150 2025-12-17 22:04:40,497 - lava_callback - ERROR - Error processing callback for node 69408503575bc060ce15da30: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408503575bc060ce15da30 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408503575bc060ce15da30 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408503575bc060ce15da30 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:40,499 - lava_callback - INFO - Starting async job submit for node 69408e7d575bc060ce15fe3a 2025-12-17 22:04:40,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:40,532 - lava_callback - INFO - Received callback for node 694085e7575bc060ce15de03 INFO: 172.21.0.1:46628 - "POST /node/694085e7575bc060ce15de03 HTTP/1.0" 202 Accepted 2025-12-17 22:04:40,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7d575bc060ce15fe3a HTTP/1.1" 502 150 2025-12-17 22:04:40,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7d575bc060ce15fe3a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:40,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7d575bc060ce15fe3a 2025-12-17 22:04:40,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7d575bc060ce15fe3a HTTP/1.1" 502 150 2025-12-17 22:04:40,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7d575bc060ce15fe3a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:40,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408516575bc060ce15da74 2025-12-17 22:04:40,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408516575bc060ce15da74 HTTP/1.1" 502 150 2025-12-17 22:04:40,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408516575bc060ce15da74'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:40,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408518575bc060ce15da77 2025-12-17 22:04:40,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408518575bc060ce15da77 HTTP/1.1" 502 150 2025-12-17 22:04:40,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408518575bc060ce15da77'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:41,338 - lava_callback - INFO - Received callback for node 694085e9575bc060ce15de0a INFO: 172.21.0.1:46636 - "POST /node/694085e9575bc060ce15de0a HTTP/1.0" 202 Accepted 2025-12-17 22:04:41,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ee575bc060ce15d99f HTTP/1.1" 502 150 2025-12-17 22:04:41,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ee575bc060ce15d99f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:41,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408514575bc060ce15da6a 2025-12-17 22:04:41,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408514575bc060ce15da6a HTTP/1.1" 502 150 2025-12-17 22:04:41,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408514575bc060ce15da6a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:42,082 - lava_callback - INFO - Received callback for node 694085f2575bc060ce15de33 INFO: 172.21.0.1:46646 - "POST /node/694085f2575bc060ce15de33 HTTP/1.0" 202 Accepted 2025-12-17 22:04:42,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7d575bc060ce15fe3a 2025-12-17 22:04:42,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7d575bc060ce15fe3a HTTP/1.1" 502 150 2025-12-17 22:04:42,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7d575bc060ce15fe3a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:42,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408505575bc060ce15da33 2025-12-17 22:04:42,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408505575bc060ce15da33 HTTP/1.1" 502 150 2025-12-17 22:04:42,758 - lava_callback - ERROR - Error processing callback for node 69408505575bc060ce15da33: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408505575bc060ce15da33 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408505575bc060ce15da33 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408505575bc060ce15da33 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:42,759 - lava_callback - INFO - Starting async job submit for node 6940851e575bc060ce15da82 2025-12-17 22:04:42,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:42,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940851e575bc060ce15da82 HTTP/1.1" 502 150 2025-12-17 22:04:42,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940851e575bc060ce15da82'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:42,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940851e575bc060ce15da82 2025-12-17 22:04:42,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940851e575bc060ce15da82 HTTP/1.1" 502 150 2025-12-17 22:04:42,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940851e575bc060ce15da82'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:42,863 - lava_callback - INFO - Received callback for node 694085f2575bc060ce15de36 INFO: 172.21.0.1:46648 - "POST /node/694085f2575bc060ce15de36 HTTP/1.0" 202 Accepted 2025-12-17 22:04:42,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408506575bc060ce15da36 2025-12-17 22:04:42,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408506575bc060ce15da36 HTTP/1.1" 502 150 2025-12-17 22:04:42,968 - lava_callback - ERROR - Error processing callback for node 69408506575bc060ce15da36: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408506575bc060ce15da36 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408506575bc060ce15da36 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408506575bc060ce15da36 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:42,969 - lava_callback - INFO - Starting async job submit for node 69408523575bc060ce15da8b 2025-12-17 22:04:42,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:42,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408523575bc060ce15da8b HTTP/1.1" 502 150 2025-12-17 22:04:42,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408523575bc060ce15da8b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:42,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408523575bc060ce15da8b 2025-12-17 22:04:42,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408523575bc060ce15da8b HTTP/1.1" 502 150 2025-12-17 22:04:42,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408523575bc060ce15da8b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:43,643 - lava_callback - INFO - Received callback for node 694085eb575bc060ce15de16 INFO: 172.21.0.1:46660 - "POST /node/694085eb575bc060ce15de16 HTTP/1.0" 202 Accepted 2025-12-17 22:04:43,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ee575bc060ce15d99f 2025-12-17 22:04:43,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408459575bc060ce15d72f 2025-12-17 22:04:43,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408459575bc060ce15d72f HTTP/1.1" 502 150 2025-12-17 22:04:43,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408459575bc060ce15d72f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:43,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408515575bc060ce15da71 2025-12-17 22:04:43,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408515575bc060ce15da71 HTTP/1.1" 502 150 2025-12-17 22:04:43,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408515575bc060ce15da71'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:44,292 - lava_callback - INFO - Received callback for node 694085ee575bc060ce15de26 INFO: 172.21.0.1:46666 - "POST /node/694085ee575bc060ce15de26 HTTP/1.0" 202 Accepted 2025-12-17 22:04:44,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408512575bc060ce15da64 2025-12-17 22:04:44,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408512575bc060ce15da64 HTTP/1.1" 502 150 2025-12-17 22:04:44,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408512575bc060ce15da64'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:44,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408508575bc060ce15da39 2025-12-17 22:04:44,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408508575bc060ce15da39 HTTP/1.1" 502 150 2025-12-17 22:04:44,607 - lava_callback - ERROR - Error processing callback for node 69408508575bc060ce15da39: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408508575bc060ce15da39 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408508575bc060ce15da39 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408508575bc060ce15da39 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:44,610 - lava_callback - INFO - Starting async job submit for node 69408524575bc060ce15da90 2025-12-17 22:04:44,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:44,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408524575bc060ce15da90 HTTP/1.1" 502 150 2025-12-17 22:04:44,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408524575bc060ce15da90'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:44,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408524575bc060ce15da90 2025-12-17 22:04:44,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408524575bc060ce15da90 HTTP/1.1" 502 150 2025-12-17 22:04:44,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408524575bc060ce15da90'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:44,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940851e575bc060ce15da82 2025-12-17 22:04:44,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940851e575bc060ce15da82 HTTP/1.1" 502 150 2025-12-17 22:04:44,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940851e575bc060ce15da82'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:44,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408516575bc060ce15da74 2025-12-17 22:04:44,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408516575bc060ce15da74 HTTP/1.1" 502 150 2025-12-17 22:04:44,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408516575bc060ce15da74'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:44,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408509575bc060ce15da3c 2025-12-17 22:04:44,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408509575bc060ce15da3c HTTP/1.1" 502 150 2025-12-17 22:04:44,914 - lava_callback - ERROR - Error processing callback for node 69408509575bc060ce15da3c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408509575bc060ce15da3c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408509575bc060ce15da3c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408509575bc060ce15da3c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:44,916 - lava_callback - INFO - Starting async job submit for node 69408525575bc060ce15da93 2025-12-17 22:04:44,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:44,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408525575bc060ce15da93 HTTP/1.1" 502 150 2025-12-17 22:04:44,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408525575bc060ce15da93'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:44,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408525575bc060ce15da93 2025-12-17 22:04:44,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408525575bc060ce15da93 HTTP/1.1" 502 150 2025-12-17 22:04:44,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408525575bc060ce15da93'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:44,930 - lava_callback - INFO - Received callback for node 694085f0575bc060ce15de2d INFO: 172.21.0.1:46678 - "POST /node/694085f0575bc060ce15de2d HTTP/1.0" 202 Accepted 2025-12-17 22:04:44,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408518575bc060ce15da77 2025-12-17 22:04:44,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408518575bc060ce15da77 HTTP/1.1" 502 150 2025-12-17 22:04:44,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408518575bc060ce15da77'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:44,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408523575bc060ce15da8b 2025-12-17 22:04:44,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408523575bc060ce15da8b HTTP/1.1" 502 150 2025-12-17 22:04:44,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408523575bc060ce15da8b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:45,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850f575bc060ce15da5d 2025-12-17 22:04:45,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850f575bc060ce15da5d HTTP/1.1" 502 150 2025-12-17 22:04:45,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940850f575bc060ce15da5d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:45,568 - lava_callback - INFO - Received callback for node 694085f1575bc060ce15de30 INFO: 172.21.0.1:46680 - "POST /node/694085f1575bc060ce15de30 HTTP/1.0" 202 Accepted 2025-12-17 22:04:46,209 - lava_callback - INFO - Received callback for node 694085f4575bc060ce15de3c INFO: 172.21.0.1:46690 - "POST /node/694085f4575bc060ce15de3c HTTP/1.0" 202 Accepted INFO: 172.21.0.1:46698 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:04:46,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850a575bc060ce15da43 2025-12-17 22:04:46,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850a575bc060ce15da43 HTTP/1.1" 502 150 2025-12-17 22:04:46,539 - lava_callback - ERROR - Error processing callback for node 6940850a575bc060ce15da43: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940850a575bc060ce15da43 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940850a575bc060ce15da43 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940850a575bc060ce15da43 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:46,541 - lava_callback - INFO - Starting async job submit for node 69408526575bc060ce15da96 2025-12-17 22:04:46,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:46,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7d575bc060ce15fe3a 2025-12-17 22:04:46,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7d575bc060ce15fe3a HTTP/1.1" 502 150 2025-12-17 22:04:46,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7d575bc060ce15fe3a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:46,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408526575bc060ce15da96 HTTP/1.1" 502 150 2025-12-17 22:04:46,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408526575bc060ce15da96'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:46,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408526575bc060ce15da96 2025-12-17 22:04:46,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408526575bc060ce15da96 HTTP/1.1" 502 150 2025-12-17 22:04:46,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408526575bc060ce15da96'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:46,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408524575bc060ce15da90 2025-12-17 22:04:46,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408524575bc060ce15da90 HTTP/1.1" 502 150 2025-12-17 22:04:46,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408524575bc060ce15da90'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:46,858 - lava_callback - INFO - Received callback for node 694085f7575bc060ce15de54 INFO: 172.21.0.1:46702 - "POST /node/694085f7575bc060ce15de54 HTTP/1.0" 202 Accepted 2025-12-17 22:04:46,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408525575bc060ce15da93 2025-12-17 22:04:46,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408525575bc060ce15da93 HTTP/1.1" 502 150 2025-12-17 22:04:46,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408525575bc060ce15da93'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:47,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408513575bc060ce15da67 HTTP/1.1" 502 150 2025-12-17 22:04:47,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408513575bc060ce15da67'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:47,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408513575bc060ce15da67 2025-12-17 22:04:47,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408513575bc060ce15da67 HTTP/1.1" 502 150 2025-12-17 22:04:47,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408513575bc060ce15da67'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:47,517 - lava_callback - INFO - Received callback for node 694085f9575bc060ce15de83 INFO: 172.21.0.1:46718 - "POST /node/694085f9575bc060ce15de83 HTTP/1.0" 202 Accepted 2025-12-17 22:04:48,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408526575bc060ce15da96 2025-12-17 22:04:48,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408526575bc060ce15da96 HTTP/1.1" 502 150 2025-12-17 22:04:48,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408526575bc060ce15da96'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:48,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940851e575bc060ce15da82 2025-12-17 22:04:48,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940851e575bc060ce15da82 HTTP/1.1" 502 150 2025-12-17 22:04:48,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940851e575bc060ce15da82'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:48,843 - lava_callback - INFO - Received callback for node 694085f3575bc060ce15de39 INFO: 172.21.0.1:46734 - "POST /node/694085f3575bc060ce15de39 HTTP/1.0" 202 Accepted 2025-12-17 22:04:48,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408523575bc060ce15da8b 2025-12-17 22:04:48,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408523575bc060ce15da8b HTTP/1.1" 502 150 2025-12-17 22:04:48,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408523575bc060ce15da8b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:49,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408513575bc060ce15da67 2025-12-17 22:04:49,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408513575bc060ce15da67 HTTP/1.1" 502 150 2025-12-17 22:04:49,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408513575bc060ce15da67'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:49,539 - lava_callback - INFO - Received callback for node 694085f5575bc060ce15de3f INFO: 172.21.0.1:46736 - "POST /node/694085f5575bc060ce15de3f HTTP/1.0" 202 Accepted 2025-12-17 22:04:49,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408514575bc060ce15da6a 2025-12-17 22:04:49,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408514575bc060ce15da6a HTTP/1.1" 502 150 2025-12-17 22:04:49,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408514575bc060ce15da6a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:50,217 - lava_callback - INFO - Received callback for node 694085fd575bc060ce15de99 INFO: 172.21.0.1:50134 - "POST /node/694085fd575bc060ce15de99 HTTP/1.0" 202 Accepted 2025-12-17 22:04:50,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408524575bc060ce15da90 2025-12-17 22:04:50,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408524575bc060ce15da90 HTTP/1.1" 502 150 2025-12-17 22:04:50,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408524575bc060ce15da90'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:50,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850c575bc060ce15da4c 2025-12-17 22:04:50,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850c575bc060ce15da4c HTTP/1.1" 502 150 2025-12-17 22:04:50,693 - lava_callback - ERROR - Error processing callback for node 6940850c575bc060ce15da4c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940850c575bc060ce15da4c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940850c575bc060ce15da4c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940850c575bc060ce15da4c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:50,694 - lava_callback - INFO - Starting async job submit for node 69408528575bc060ce15da9d 2025-12-17 22:04:50,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:50,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408528575bc060ce15da9d HTTP/1.1" 502 150 2025-12-17 22:04:50,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408528575bc060ce15da9d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:50,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408528575bc060ce15da9d 2025-12-17 22:04:50,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408528575bc060ce15da9d HTTP/1.1" 502 150 2025-12-17 22:04:50,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408528575bc060ce15da9d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:50,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408525575bc060ce15da93 2025-12-17 22:04:50,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408525575bc060ce15da93 HTTP/1.1" 502 150 2025-12-17 22:04:50,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408525575bc060ce15da93'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:51,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408515575bc060ce15da71 2025-12-17 22:04:52,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408512575bc060ce15da64 2025-12-17 22:04:52,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408512575bc060ce15da64 HTTP/1.1" 502 150 2025-12-17 22:04:52,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408512575bc060ce15da64'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:52,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408526575bc060ce15da96 2025-12-17 22:04:52,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408526575bc060ce15da96 HTTP/1.1" 502 150 2025-12-17 22:04:52,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408526575bc060ce15da96'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:52,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408528575bc060ce15da9d 2025-12-17 22:04:52,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408528575bc060ce15da9d HTTP/1.1" 502 150 2025-12-17 22:04:52,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408528575bc060ce15da9d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:52,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408516575bc060ce15da74 2025-12-17 22:04:52,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408516575bc060ce15da74 HTTP/1.1" 502 150 2025-12-17 22:04:52,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408516575bc060ce15da74'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:52,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408518575bc060ce15da77 2025-12-17 22:04:52,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408518575bc060ce15da77 HTTP/1.1" 502 150 2025-12-17 22:04:52,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408518575bc060ce15da77'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:53,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408513575bc060ce15da67 2025-12-17 22:04:53,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408513575bc060ce15da67 HTTP/1.1" 502 150 2025-12-17 22:04:53,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408513575bc060ce15da67'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:54,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7d575bc060ce15fe3a 2025-12-17 22:04:54,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7d575bc060ce15fe3a HTTP/1.1" 502 150 2025-12-17 22:04:54,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7d575bc060ce15fe3a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50140 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:04:56,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408528575bc060ce15da9d 2025-12-17 22:04:56,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408528575bc060ce15da9d HTTP/1.1" 502 150 2025-12-17 22:04:56,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408528575bc060ce15da9d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:56,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940851e575bc060ce15da82 2025-12-17 22:04:56,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940851e575bc060ce15da82 HTTP/1.1" 502 150 2025-12-17 22:04:56,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940851e575bc060ce15da82'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:56,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408523575bc060ce15da8b 2025-12-17 22:04:56,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408523575bc060ce15da8b HTTP/1.1" 502 150 2025-12-17 22:04:56,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408523575bc060ce15da8b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:58,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408524575bc060ce15da90 2025-12-17 22:04:58,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408524575bc060ce15da90 HTTP/1.1" 502 150 2025-12-17 22:04:58,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408524575bc060ce15da90'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:58,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408525575bc060ce15da93 2025-12-17 22:04:58,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408525575bc060ce15da93 HTTP/1.1" 502 150 2025-12-17 22:04:58,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408525575bc060ce15da93'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:59,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408459575bc060ce15d72f 2025-12-17 22:04:59,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408459575bc060ce15d72f HTTP/1.1" 502 150 2025-12-17 22:04:59,835 - lava_callback - ERROR - Error processing callback for node 69408459575bc060ce15d72f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408459575bc060ce15d72f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408459575bc060ce15d72f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408459575bc060ce15d72f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:04:59,838 - lava_callback - INFO - Starting async job submit for node 69408529575bc060ce15daa0 2025-12-17 22:04:59,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:04:59,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408529575bc060ce15daa0 HTTP/1.1" 502 150 2025-12-17 22:04:59,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408529575bc060ce15daa0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:04:59,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408529575bc060ce15daa0 2025-12-17 22:04:59,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408529575bc060ce15daa0 HTTP/1.1" 502 150 2025-12-17 22:04:59,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408529575bc060ce15daa0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:00,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408526575bc060ce15da96 2025-12-17 22:05:00,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408526575bc060ce15da96 HTTP/1.1" 502 150 2025-12-17 22:05:00,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408526575bc060ce15da96'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:01,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408513575bc060ce15da67 2025-12-17 22:05:01,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408513575bc060ce15da67 HTTP/1.1" 502 150 2025-12-17 22:05:01,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408513575bc060ce15da67'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:01,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940850f575bc060ce15da5d 2025-12-17 22:05:01,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940850f575bc060ce15da5d HTTP/1.1" 502 150 2025-12-17 22:05:01,530 - lava_callback - ERROR - Error processing callback for node 6940850f575bc060ce15da5d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940850f575bc060ce15da5d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940850f575bc060ce15da5d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940850f575bc060ce15da5d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:01,533 - lava_callback - INFO - Starting async job submit for node 6940852a575bc060ce15daa3 2025-12-17 22:05:01,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:01,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852a575bc060ce15daa3 HTTP/1.1" 502 150 2025-12-17 22:05:01,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852a575bc060ce15daa3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:01,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852a575bc060ce15daa3 2025-12-17 22:05:01,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852a575bc060ce15daa3 HTTP/1.1" 502 150 2025-12-17 22:05:01,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852a575bc060ce15daa3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:01,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408529575bc060ce15daa0 2025-12-17 22:05:01,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408529575bc060ce15daa0 HTTP/1.1" 502 150 2025-12-17 22:05:01,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408529575bc060ce15daa0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46790 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:05:03,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852a575bc060ce15daa3 2025-12-17 22:05:03,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852a575bc060ce15daa3 HTTP/1.1" 502 150 2025-12-17 22:05:03,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852a575bc060ce15daa3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:03,619 - lava_callback - INFO - Received callback for node 694085ff575bc060ce15de9e INFO: 172.21.0.1:46806 - "POST /node/694085ff575bc060ce15de9e HTTP/1.0" 202 Accepted 2025-12-17 22:05:04,283 - lava_callback - INFO - Received callback for node 69408600575bc060ce15dea4 INFO: 172.21.0.1:46814 - "POST /node/69408600575bc060ce15dea4 HTTP/1.0" 202 Accepted 2025-12-17 22:05:04,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408528575bc060ce15da9d 2025-12-17 22:05:04,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408528575bc060ce15da9d HTTP/1.1" 502 150 2025-12-17 22:05:04,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408528575bc060ce15da9d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:04,933 - lava_callback - INFO - Received callback for node 69408602575bc060ce15dea7 INFO: 172.21.0.1:46816 - "POST /node/69408602575bc060ce15dea7 HTTP/1.0" 202 Accepted 2025-12-17 22:05:05,587 - lava_callback - INFO - Received callback for node 69408603575bc060ce15deaa INFO: 172.21.0.1:46826 - "POST /node/69408603575bc060ce15deaa HTTP/1.0" 202 Accepted 2025-12-17 22:05:05,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408529575bc060ce15daa0 2025-12-17 22:05:05,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408529575bc060ce15daa0 HTTP/1.1" 502 150 2025-12-17 22:05:05,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408529575bc060ce15daa0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:05,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408514575bc060ce15da6a 2025-12-17 22:05:05,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408514575bc060ce15da6a HTTP/1.1" 502 150 2025-12-17 22:05:05,987 - lava_callback - ERROR - Error processing callback for node 69408514575bc060ce15da6a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408514575bc060ce15da6a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408514575bc060ce15da6a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408514575bc060ce15da6a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:05,989 - lava_callback - INFO - Starting async job submit for node 6940852b575bc060ce15daa6 2025-12-17 22:05:05,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:06,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852b575bc060ce15daa6 HTTP/1.1" 502 150 2025-12-17 22:05:06,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852b575bc060ce15daa6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:06,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852b575bc060ce15daa6 2025-12-17 22:05:06,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852b575bc060ce15daa6 HTTP/1.1" 502 150 2025-12-17 22:05:06,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852b575bc060ce15daa6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:06,004 - lava_callback - INFO - Received callback for node 6940bf611e3c24e95681c645 INFO: 172.21.0.1:46828 - "POST /node/6940bf611e3c24e95681c645 HTTP/1.0" 202 Accepted 2025-12-17 22:05:06,220 - lava_callback - INFO - Received callback for node 69408604575bc060ce15deb0 INFO: 172.21.0.1:46834 - "POST /node/69408604575bc060ce15deb0 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:46842 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:05:06,864 - lava_callback - INFO - Received callback for node 69408605575bc060ce15deb3 INFO: 172.21.0.1:46850 - "POST /node/69408605575bc060ce15deb3 HTTP/1.0" 202 Accepted 2025-12-17 22:05:07,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408515575bc060ce15da71 HTTP/1.1" 502 150 2025-12-17 22:05:07,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408515575bc060ce15da71'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:07,508 - lava_callback - INFO - Received callback for node 69408606575bc060ce15deb6 INFO: 172.21.0.1:46858 - "POST /node/69408606575bc060ce15deb6 HTTP/1.0" 202 Accepted 2025-12-17 22:05:07,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852a575bc060ce15daa3 2025-12-17 22:05:07,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852a575bc060ce15daa3 HTTP/1.1" 502 150 2025-12-17 22:05:07,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852a575bc060ce15daa3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:08,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852b575bc060ce15daa6 2025-12-17 22:05:08,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852b575bc060ce15daa6 HTTP/1.1" 502 150 2025-12-17 22:05:08,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852b575bc060ce15daa6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:08,145 - lava_callback - INFO - Received callback for node 69408608575bc060ce15deb9 INFO: 172.21.0.1:46866 - "POST /node/69408608575bc060ce15deb9 HTTP/1.0" 202 Accepted 2025-12-17 22:05:08,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408512575bc060ce15da64 2025-12-17 22:05:08,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408512575bc060ce15da64 HTTP/1.1" 502 150 2025-12-17 22:05:08,501 - lava_callback - ERROR - Error processing callback for node 69408512575bc060ce15da64: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408512575bc060ce15da64 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408512575bc060ce15da64 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408512575bc060ce15da64 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:08,503 - lava_callback - INFO - Starting async job submit for node 6940852d575bc060ce15daa9 2025-12-17 22:05:08,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:08,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852d575bc060ce15daa9 HTTP/1.1" 502 150 2025-12-17 22:05:08,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852d575bc060ce15daa9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:08,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852d575bc060ce15daa9 2025-12-17 22:05:08,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852d575bc060ce15daa9 HTTP/1.1" 502 150 2025-12-17 22:05:08,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852d575bc060ce15daa9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:08,799 - lava_callback - INFO - Received callback for node 69408609575bc060ce15debc INFO: 172.21.0.1:46872 - "POST /node/69408609575bc060ce15debc HTTP/1.0" 202 Accepted 2025-12-17 22:05:08,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408516575bc060ce15da74 2025-12-17 22:05:08,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408516575bc060ce15da74 HTTP/1.1" 502 150 2025-12-17 22:05:08,911 - lava_callback - ERROR - Error processing callback for node 69408516575bc060ce15da74: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408516575bc060ce15da74 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408516575bc060ce15da74 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408516575bc060ce15da74 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:08,913 - lava_callback - INFO - Starting async job submit for node 6940852e575bc060ce15daac 2025-12-17 22:05:08,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:08,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852e575bc060ce15daac HTTP/1.1" 502 150 2025-12-17 22:05:08,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852e575bc060ce15daac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:08,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852e575bc060ce15daac 2025-12-17 22:05:08,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852e575bc060ce15daac HTTP/1.1" 502 150 2025-12-17 22:05:08,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852e575bc060ce15daac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:08,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408518575bc060ce15da77 2025-12-17 22:05:08,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408518575bc060ce15da77 HTTP/1.1" 502 150 2025-12-17 22:05:08,960 - lava_callback - ERROR - Error processing callback for node 69408518575bc060ce15da77: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408518575bc060ce15da77 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408518575bc060ce15da77 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408518575bc060ce15da77 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:08,962 - lava_callback - INFO - Starting async job submit for node 69408532575bc060ce15dab2 2025-12-17 22:05:08,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:08,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408532575bc060ce15dab2 HTTP/1.1" 502 150 2025-12-17 22:05:08,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408532575bc060ce15dab2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:08,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408532575bc060ce15dab2 2025-12-17 22:05:08,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408532575bc060ce15dab2 HTTP/1.1" 502 150 2025-12-17 22:05:08,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408532575bc060ce15dab2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:10,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852d575bc060ce15daa9 2025-12-17 22:05:10,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852d575bc060ce15daa9 HTTP/1.1" 502 150 2025-12-17 22:05:10,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852d575bc060ce15daa9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:10,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7d575bc060ce15fe3a 2025-12-17 22:05:10,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7d575bc060ce15fe3a HTTP/1.1" 502 150 2025-12-17 22:05:10,547 - lava_callback - ERROR - Error processing callback for node 69408e7d575bc060ce15fe3a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e7d575bc060ce15fe3a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e7d575bc060ce15fe3a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e7d575bc060ce15fe3a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:10,549 - lava_callback - INFO - Starting async job submit for node 6940853e575bc060ce15db36 2025-12-17 22:05:10,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:10,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853e575bc060ce15db36 HTTP/1.1" 502 150 2025-12-17 22:05:10,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853e575bc060ce15db36'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:10,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853e575bc060ce15db36 2025-12-17 22:05:10,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853e575bc060ce15db36 HTTP/1.1" 502 150 2025-12-17 22:05:10,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853e575bc060ce15db36'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:10,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852e575bc060ce15daac 2025-12-17 22:05:10,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852e575bc060ce15daac HTTP/1.1" 502 150 2025-12-17 22:05:10,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852e575bc060ce15daac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:10,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408532575bc060ce15dab2 2025-12-17 22:05:10,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408532575bc060ce15dab2 HTTP/1.1" 502 150 2025-12-17 22:05:10,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408532575bc060ce15dab2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:12,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852b575bc060ce15daa6 2025-12-17 22:05:12,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852b575bc060ce15daa6 HTTP/1.1" 502 150 2025-12-17 22:05:12,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852b575bc060ce15daa6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:12,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853e575bc060ce15db36 2025-12-17 22:05:12,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853e575bc060ce15db36 HTTP/1.1" 502 150 2025-12-17 22:05:12,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853e575bc060ce15db36'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:12,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940851e575bc060ce15da82 2025-12-17 22:05:12,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940851e575bc060ce15da82 HTTP/1.1" 502 150 2025-12-17 22:05:12,773 - lava_callback - ERROR - Error processing callback for node 6940851e575bc060ce15da82: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940851e575bc060ce15da82 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940851e575bc060ce15da82 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940851e575bc060ce15da82 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:12,775 - lava_callback - INFO - Starting async job submit for node 6940853f575bc060ce15db39 2025-12-17 22:05:12,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:12,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853f575bc060ce15db39 HTTP/1.1" 502 150 2025-12-17 22:05:12,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853f575bc060ce15db39'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:12,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853f575bc060ce15db39 2025-12-17 22:05:12,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853f575bc060ce15db39 HTTP/1.1" 502 150 2025-12-17 22:05:12,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853f575bc060ce15db39'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:12,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408523575bc060ce15da8b 2025-12-17 22:05:12,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408523575bc060ce15da8b HTTP/1.1" 502 150 2025-12-17 22:05:12,981 - lava_callback - ERROR - Error processing callback for node 69408523575bc060ce15da8b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408523575bc060ce15da8b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408523575bc060ce15da8b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408523575bc060ce15da8b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:12,983 - lava_callback - INFO - Starting async job submit for node 69408533575bc060ce15dab6 2025-12-17 22:05:12,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:12,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408533575bc060ce15dab6 HTTP/1.1" 502 150 2025-12-17 22:05:12,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408533575bc060ce15dab6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:12,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408533575bc060ce15dab6 2025-12-17 22:05:12,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408533575bc060ce15dab6 HTTP/1.1" 502 150 2025-12-17 22:05:12,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408533575bc060ce15dab6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:13,812 - lava_callback - INFO - Received callback for node 69408614575bc060ce15dedc INFO: 172.21.0.1:37910 - "POST /node/69408614575bc060ce15dedc HTTP/1.0" 202 Accepted 2025-12-17 22:05:13,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408529575bc060ce15daa0 2025-12-17 22:05:13,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408529575bc060ce15daa0 HTTP/1.1" 502 150 2025-12-17 22:05:13,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408529575bc060ce15daa0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:14,492 - lava_callback - INFO - Received callback for node 69408617575bc060ce15dee2 INFO: 172.21.0.1:37922 - "POST /node/69408617575bc060ce15dee2 HTTP/1.0" 202 Accepted 2025-12-17 22:05:14,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852d575bc060ce15daa9 2025-12-17 22:05:14,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852d575bc060ce15daa9 HTTP/1.1" 502 150 2025-12-17 22:05:14,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852d575bc060ce15daa9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:14,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ee575bc060ce15d99f HTTP/1.1" 502 150 2025-12-17 22:05:14,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ee575bc060ce15d99f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:14,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408524575bc060ce15da90 2025-12-17 22:05:14,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408524575bc060ce15da90 HTTP/1.1" 502 150 2025-12-17 22:05:14,626 - lava_callback - ERROR - Error processing callback for node 69408524575bc060ce15da90: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408524575bc060ce15da90 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408524575bc060ce15da90 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408524575bc060ce15da90 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:14,628 - lava_callback - INFO - Starting async job submit for node 69408534575bc060ce15dabd 2025-12-17 22:05:14,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:14,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408534575bc060ce15dabd HTTP/1.1" 502 150 2025-12-17 22:05:14,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408534575bc060ce15dabd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:14,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408534575bc060ce15dabd 2025-12-17 22:05:14,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408534575bc060ce15dabd HTTP/1.1" 502 150 2025-12-17 22:05:14,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408534575bc060ce15dabd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:14,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853f575bc060ce15db39 2025-12-17 22:05:14,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853f575bc060ce15db39 HTTP/1.1" 502 150 2025-12-17 22:05:14,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853f575bc060ce15db39'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:14,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408525575bc060ce15da93 2025-12-17 22:05:14,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408525575bc060ce15da93 HTTP/1.1" 502 150 2025-12-17 22:05:14,932 - lava_callback - ERROR - Error processing callback for node 69408525575bc060ce15da93: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408525575bc060ce15da93 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408525575bc060ce15da93 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408525575bc060ce15da93 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:14,934 - lava_callback - INFO - Starting async job submit for node 69408536575bc060ce15dac5 2025-12-17 22:05:14,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:14,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408536575bc060ce15dac5 HTTP/1.1" 502 150 2025-12-17 22:05:14,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408536575bc060ce15dac5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:14,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408536575bc060ce15dac5 2025-12-17 22:05:14,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408536575bc060ce15dac5 HTTP/1.1" 502 150 2025-12-17 22:05:14,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408536575bc060ce15dac5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:14,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852e575bc060ce15daac 2025-12-17 22:05:14,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852e575bc060ce15daac HTTP/1.1" 502 150 2025-12-17 22:05:14,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852e575bc060ce15daac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:14,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408532575bc060ce15dab2 2025-12-17 22:05:14,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408532575bc060ce15dab2 HTTP/1.1" 502 150 2025-12-17 22:05:14,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408532575bc060ce15dab2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:14,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408533575bc060ce15dab6 2025-12-17 22:05:14,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408533575bc060ce15dab6 HTTP/1.1" 502 150 2025-12-17 22:05:14,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408533575bc060ce15dab6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:15,150 - lava_callback - INFO - Received callback for node 69408618575bc060ce15dee5 INFO: 172.21.0.1:37928 - "POST /node/69408618575bc060ce15dee5 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:37940 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:05:15,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852a575bc060ce15daa3 2025-12-17 22:05:15,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852a575bc060ce15daa3 HTTP/1.1" 502 150 2025-12-17 22:05:15,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852a575bc060ce15daa3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37956 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:05:16,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853e575bc060ce15db36 2025-12-17 22:05:16,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853e575bc060ce15db36 HTTP/1.1" 502 150 2025-12-17 22:05:16,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853e575bc060ce15db36'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:16,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408526575bc060ce15da96 2025-12-17 22:05:16,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408526575bc060ce15da96 HTTP/1.1" 502 150 2025-12-17 22:05:16,577 - lava_callback - ERROR - Error processing callback for node 69408526575bc060ce15da96: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408526575bc060ce15da96 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408526575bc060ce15da96 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408526575bc060ce15da96 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:16,578 - lava_callback - INFO - Starting async job submit for node 69408537575bc060ce15daca 2025-12-17 22:05:16,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:16,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408537575bc060ce15daca HTTP/1.1" 502 150 2025-12-17 22:05:16,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408537575bc060ce15daca'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:16,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408537575bc060ce15daca 2025-12-17 22:05:16,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408537575bc060ce15daca HTTP/1.1" 502 150 2025-12-17 22:05:16,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408537575bc060ce15daca'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:16,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408534575bc060ce15dabd 2025-12-17 22:05:16,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408534575bc060ce15dabd HTTP/1.1" 502 150 2025-12-17 22:05:16,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408534575bc060ce15dabd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:16,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408536575bc060ce15dac5 2025-12-17 22:05:16,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408536575bc060ce15dac5 HTTP/1.1" 502 150 2025-12-17 22:05:16,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408536575bc060ce15dac5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:17,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408513575bc060ce15da67 2025-12-17 22:05:17,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408513575bc060ce15da67 HTTP/1.1" 502 150 2025-12-17 22:05:17,416 - lava_callback - ERROR - Error processing callback for node 69408513575bc060ce15da67: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408513575bc060ce15da67 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408513575bc060ce15da67 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408513575bc060ce15da67 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:17,418 - lava_callback - INFO - Starting async job submit for node 69408538575bc060ce15dad1 2025-12-17 22:05:17,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:17,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408538575bc060ce15dad1 HTTP/1.1" 502 150 2025-12-17 22:05:17,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408538575bc060ce15dad1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:17,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408538575bc060ce15dad1 2025-12-17 22:05:17,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408538575bc060ce15dad1 HTTP/1.1" 502 150 2025-12-17 22:05:17,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408538575bc060ce15dad1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:18,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ee575bc060ce15d99f 2025-12-17 22:05:18,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408537575bc060ce15daca 2025-12-17 22:05:18,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408537575bc060ce15daca HTTP/1.1" 502 150 2025-12-17 22:05:18,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408537575bc060ce15daca'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:18,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853f575bc060ce15db39 2025-12-17 22:05:18,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853f575bc060ce15db39 HTTP/1.1" 502 150 2025-12-17 22:05:18,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853f575bc060ce15db39'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:18,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408533575bc060ce15dab6 2025-12-17 22:05:18,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408533575bc060ce15dab6 HTTP/1.1" 502 150 2025-12-17 22:05:18,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408533575bc060ce15dab6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:19,055 - lava_callback - INFO - Received callback for node 6940860a575bc060ce15debf INFO: 172.21.0.1:37968 - "POST /node/6940860a575bc060ce15debf HTTP/1.0" 202 Accepted 2025-12-17 22:05:19,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408538575bc060ce15dad1 2025-12-17 22:05:19,866 - lava_callback - INFO - Received callback for node 6940860b575bc060ce15dec2 INFO: 172.21.0.1:40064 - "POST /node/6940860b575bc060ce15dec2 HTTP/1.0" 202 Accepted 2025-12-17 22:05:20,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852b575bc060ce15daa6 2025-12-17 22:05:20,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408534575bc060ce15dabd 2025-12-17 22:05:20,647 - lava_callback - INFO - Received callback for node 69408611575bc060ce15dece INFO: 172.21.0.1:40068 - "POST /node/69408611575bc060ce15dece HTTP/1.0" 202 Accepted 2025-12-17 22:05:20,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408528575bc060ce15da9d 2025-12-17 22:05:20,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408536575bc060ce15dac5 2025-12-17 22:05:21,470 - lava_callback - INFO - Received callback for node 69408612575bc060ce15ded5 INFO: 172.21.0.1:40074 - "POST /node/69408612575bc060ce15ded5 HTTP/1.0" 202 Accepted 2025-12-17 22:05:22,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852d575bc060ce15daa9 2025-12-17 22:05:22,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408537575bc060ce15daca 2025-12-17 22:05:22,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852e575bc060ce15daac 2025-12-17 22:05:22,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408532575bc060ce15dab2 2025-12-17 22:05:23,376 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408515575bc060ce15da71 2025-12-17 22:05:24,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853e575bc060ce15db36 INFO: 172.21.0.1:40080 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:05:26,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853f575bc060ce15db39 2025-12-17 22:05:26,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408533575bc060ce15dab6 2025-12-17 22:05:28,632 - lava_callback - INFO - Received callback for node 69408616575bc060ce15dedf INFO: 172.21.0.1:40090 - "POST /node/69408616575bc060ce15dedf HTTP/1.0" 202 Accepted 2025-12-17 22:05:29,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408529575bc060ce15daa0 2025-12-17 22:05:31,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852a575bc060ce15daa3 2025-12-17 22:05:32,678 - lava_callback - INFO - Received callback for node 69408625575bc060ce15df12 INFO: 172.21.0.1:37484 - "POST /node/69408625575bc060ce15df12 HTTP/1.0" 202 Accepted 2025-12-17 22:05:33,804 - lava_callback - INFO - Received callback for node 6940861a575bc060ce15dee8 INFO: 172.21.0.1:37492 - "POST /node/6940861a575bc060ce15dee8 HTTP/1.0" 202 Accepted 2025-12-17 22:05:34,467 - lava_callback - INFO - Received callback for node 6940861b575bc060ce15deeb INFO: 172.21.0.1:37496 - "POST /node/6940861b575bc060ce15deeb HTTP/1.0" 202 Accepted 2025-12-17 22:05:34,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408538575bc060ce15dad1 HTTP/1.1" 502 150 2025-12-17 22:05:34,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408538575bc060ce15dad1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:35,107 - lava_callback - INFO - Received callback for node 6940861c575bc060ce15deee INFO: 172.21.0.1:37500 - "POST /node/6940861c575bc060ce15deee HTTP/1.0" 202 Accepted 2025-12-17 22:05:35,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852b575bc060ce15daa6 HTTP/1.1" 502 150 2025-12-17 22:05:35,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852b575bc060ce15daa6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:35,761 - lava_callback - INFO - Received callback for node 6940861e575bc060ce15def1 INFO: 172.21.0.1:37510 - "POST /node/6940861e575bc060ce15def1 HTTP/1.0" 202 Accepted 2025-12-17 22:05:36,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408528575bc060ce15da9d HTTP/1.1" 502 150 2025-12-17 22:05:36,048 - lava_callback - ERROR - Error processing callback for node 69408528575bc060ce15da9d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408528575bc060ce15da9d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408528575bc060ce15da9d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408528575bc060ce15da9d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:36,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408534575bc060ce15dabd HTTP/1.1" 502 150 2025-12-17 22:05:36,051 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408534575bc060ce15dabd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:36,051 - lava_callback - INFO - Starting async job submit for node 69408541575bc060ce15db3c 2025-12-17 22:05:36,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:36,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408541575bc060ce15db3c HTTP/1.1" 502 150 2025-12-17 22:05:36,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408541575bc060ce15db3c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:36,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408541575bc060ce15db3c 2025-12-17 22:05:36,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408541575bc060ce15db3c HTTP/1.1" 502 150 2025-12-17 22:05:36,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408541575bc060ce15db3c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:36,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408536575bc060ce15dac5 HTTP/1.1" 502 150 2025-12-17 22:05:36,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408536575bc060ce15dac5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:36,434 - lava_callback - INFO - Received callback for node 6940861f575bc060ce15def4 INFO: 172.21.0.1:37512 - "POST /node/6940861f575bc060ce15def4 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:37520 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:05:37,093 - lava_callback - INFO - Received callback for node 69408620575bc060ce15def7 INFO: 172.21.0.1:37524 - "POST /node/69408620575bc060ce15def7 HTTP/1.0" 202 Accepted 2025-12-17 22:05:37,739 - lava_callback - INFO - Received callback for node 69408621575bc060ce15defa INFO: 172.21.0.1:37526 - "POST /node/69408621575bc060ce15defa HTTP/1.0" 202 Accepted 2025-12-17 22:05:37,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852d575bc060ce15daa9 HTTP/1.1" 502 150 2025-12-17 22:05:37,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852d575bc060ce15daa9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:37,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408537575bc060ce15daca HTTP/1.1" 502 150 2025-12-17 22:05:37,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408537575bc060ce15daca'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:38,059 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408541575bc060ce15db3c 2025-12-17 22:05:38,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408541575bc060ce15db3c HTTP/1.1" 502 150 2025-12-17 22:05:38,060 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408541575bc060ce15db3c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:38,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852e575bc060ce15daac HTTP/1.1" 502 150 2025-12-17 22:05:38,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940852e575bc060ce15daac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:38,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408532575bc060ce15dab2 HTTP/1.1" 502 150 2025-12-17 22:05:38,356 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408532575bc060ce15dab2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:38,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408538575bc060ce15dad1 2025-12-17 22:05:38,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408538575bc060ce15dad1 HTTP/1.1" 502 150 2025-12-17 22:05:38,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408538575bc060ce15dad1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:38,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408515575bc060ce15da71 HTTP/1.1" 502 150 2025-12-17 22:05:38,864 - lava_callback - ERROR - Error processing callback for node 69408515575bc060ce15da71: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408515575bc060ce15da71 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408515575bc060ce15da71 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408515575bc060ce15da71 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:38,866 - lava_callback - INFO - Starting async job submit for node 69408543575bc060ce15db3f 2025-12-17 22:05:38,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:38,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408543575bc060ce15db3f HTTP/1.1" 502 150 2025-12-17 22:05:38,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408543575bc060ce15db3f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:38,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408543575bc060ce15db3f 2025-12-17 22:05:38,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408543575bc060ce15db3f HTTP/1.1" 502 150 2025-12-17 22:05:38,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408543575bc060ce15db3f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:38,973 - lava_callback - INFO - Received callback for node 69408627575bc060ce15df1b INFO: 172.21.0.1:37534 - "POST /node/69408627575bc060ce15df1b HTTP/1.0" 202 Accepted 2025-12-17 22:05:39,713 - lava_callback - INFO - Received callback for node 69408628575bc060ce15df1e INFO: 172.21.0.1:39868 - "POST /node/69408628575bc060ce15df1e HTTP/1.0" 202 Accepted 2025-12-17 22:05:39,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853e575bc060ce15db36 HTTP/1.1" 502 150 2025-12-17 22:05:39,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853e575bc060ce15db36'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:40,520 - lava_callback - INFO - Received callback for node 6940862a575bc060ce15df21 INFO: 172.21.0.1:39870 - "POST /node/6940862a575bc060ce15df21 HTTP/1.0" 202 Accepted 2025-12-17 22:05:40,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408543575bc060ce15db3f 2025-12-17 22:05:40,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408543575bc060ce15db3f HTTP/1.1" 502 150 2025-12-17 22:05:40,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408543575bc060ce15db3f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:41,294 - lava_callback - INFO - Received callback for node 6940862b575bc060ce15df24 INFO: 172.21.0.1:39878 - "POST /node/6940862b575bc060ce15df24 HTTP/1.0" 202 Accepted 2025-12-17 22:05:42,060 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408541575bc060ce15db3c 2025-12-17 22:05:42,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408541575bc060ce15db3c HTTP/1.1" 502 150 2025-12-17 22:05:42,061 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408541575bc060ce15db3c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:42,082 - lava_callback - INFO - Received callback for node 6940862c575bc060ce15df27 INFO: 172.21.0.1:39894 - "POST /node/6940862c575bc060ce15df27 HTTP/1.0" 202 Accepted 2025-12-17 22:05:42,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853f575bc060ce15db39 HTTP/1.1" 502 150 2025-12-17 22:05:42,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853f575bc060ce15db39'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:42,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408533575bc060ce15dab6 HTTP/1.1" 502 150 2025-12-17 22:05:42,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408533575bc060ce15dab6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:42,831 - lava_callback - INFO - Received callback for node 69408631575bc060ce15df30 INFO: 172.21.0.1:39904 - "POST /node/69408631575bc060ce15df30 HTTP/1.0" 202 Accepted 2025-12-17 22:05:43,581 - lava_callback - INFO - Received callback for node 69408633575bc060ce15df36 INFO: 172.21.0.1:39908 - "POST /node/69408633575bc060ce15df36 HTTP/1.0" 202 Accepted 2025-12-17 22:05:43,630 - lava_callback - INFO - Received callback for node 6940862d575bc060ce15df2a INFO: 172.21.0.1:39920 - "POST /node/6940862d575bc060ce15df2a HTTP/1.0" 202 Accepted 2025-12-17 22:05:44,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408534575bc060ce15dabd 2025-12-17 22:05:44,284 - lava_callback - INFO - Received callback for node 6940862f575bc060ce15df2d INFO: 172.21.0.1:39926 - "POST /node/6940862f575bc060ce15df2d HTTP/1.0" 202 Accepted 2025-12-17 22:05:44,304 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408536575bc060ce15dac5 2025-12-17 22:05:44,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408536575bc060ce15dac5 HTTP/1.1" 502 150 2025-12-17 22:05:44,305 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408536575bc060ce15dac5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:44,327 - lava_callback - INFO - Received callback for node 69408637575bc060ce15df76 INFO: 172.21.0.1:39928 - "POST /node/69408637575bc060ce15df76 HTTP/1.0" 202 Accepted 2025-12-17 22:05:44,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408543575bc060ce15db3f 2025-12-17 22:05:44,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408543575bc060ce15db3f HTTP/1.1" 502 150 2025-12-17 22:05:44,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408543575bc060ce15db3f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:44,931 - lava_callback - INFO - Received callback for node 69408631575bc060ce15df33 INFO: 172.21.0.1:39932 - "POST /node/69408631575bc060ce15df33 HTTP/1.0" 202 Accepted 2025-12-17 22:05:45,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408529575bc060ce15daa0 HTTP/1.1" 502 150 2025-12-17 22:05:45,264 - lava_callback - ERROR - Error processing callback for node 69408529575bc060ce15daa0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408529575bc060ce15daa0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408529575bc060ce15daa0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408529575bc060ce15daa0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:45,267 - lava_callback - INFO - Starting async job submit for node 69408544575bc060ce15db42 2025-12-17 22:05:45,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:45,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408544575bc060ce15db42 HTTP/1.1" 502 150 2025-12-17 22:05:45,274 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408544575bc060ce15db42'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:45,274 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408544575bc060ce15db42 2025-12-17 22:05:45,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408544575bc060ce15db42 HTTP/1.1" 502 150 2025-12-17 22:05:45,275 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408544575bc060ce15db42'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:45,601 - lava_callback - INFO - Received callback for node 69408634575bc060ce15df39 INFO: 172.21.0.1:39934 - "POST /node/69408634575bc060ce15df39 HTTP/1.0" 202 Accepted 2025-12-17 22:05:45,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408537575bc060ce15daca 2025-12-17 22:05:45,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408537575bc060ce15daca HTTP/1.1" 502 150 2025-12-17 22:05:45,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408537575bc060ce15daca'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39936 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:05:46,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408538575bc060ce15dad1 2025-12-17 22:05:46,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408538575bc060ce15dad1 HTTP/1.1" 502 150 2025-12-17 22:05:46,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408538575bc060ce15dad1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:46,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852a575bc060ce15daa3 HTTP/1.1" 502 150 2025-12-17 22:05:46,804 - lava_callback - ERROR - Error processing callback for node 6940852a575bc060ce15daa3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940852a575bc060ce15daa3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940852a575bc060ce15daa3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940852a575bc060ce15daa3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:46,806 - lava_callback - INFO - Starting async job submit for node 69408546575bc060ce15db45 2025-12-17 22:05:46,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:46,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408546575bc060ce15db45 HTTP/1.1" 502 150 2025-12-17 22:05:46,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408546575bc060ce15db45'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:46,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408546575bc060ce15db45 2025-12-17 22:05:46,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408546575bc060ce15db45 HTTP/1.1" 502 150 2025-12-17 22:05:46,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408546575bc060ce15db45'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:47,278 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408544575bc060ce15db42 2025-12-17 22:05:47,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408544575bc060ce15db42 HTTP/1.1" 502 150 2025-12-17 22:05:47,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408544575bc060ce15db42'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:48,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408546575bc060ce15db45 2025-12-17 22:05:48,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408546575bc060ce15db45 HTTP/1.1" 502 150 2025-12-17 22:05:48,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408546575bc060ce15db45'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:49,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ee575bc060ce15d99f HTTP/1.1" 502 150 2025-12-17 22:05:49,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ee575bc060ce15d99f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:50,062 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408541575bc060ce15db3c 2025-12-17 22:05:50,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408541575bc060ce15db3c HTTP/1.1" 502 150 2025-12-17 22:05:50,063 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408541575bc060ce15db3c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:51,280 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408544575bc060ce15db42 2025-12-17 22:05:51,280 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852b575bc060ce15daa6 2025-12-17 22:05:51,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408544575bc060ce15db42 HTTP/1.1" 502 150 2025-12-17 22:05:51,281 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408544575bc060ce15db42'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:51,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852b575bc060ce15daa6 HTTP/1.1" 502 150 2025-12-17 22:05:51,282 - lava_callback - ERROR - Error processing callback for node 6940852b575bc060ce15daa6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940852b575bc060ce15daa6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940852b575bc060ce15daa6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940852b575bc060ce15daa6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:51,283 - lava_callback - INFO - Starting async job submit for node 69408547575bc060ce15db4b 2025-12-17 22:05:51,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:51,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408547575bc060ce15db4b HTTP/1.1" 502 150 2025-12-17 22:05:51,294 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408547575bc060ce15db4b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:51,294 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408547575bc060ce15db4b 2025-12-17 22:05:51,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408547575bc060ce15db4b HTTP/1.1" 502 150 2025-12-17 22:05:51,295 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408547575bc060ce15db4b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:52,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408546575bc060ce15db45 2025-12-17 22:05:52,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408546575bc060ce15db45 HTTP/1.1" 502 150 2025-12-17 22:05:52,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408546575bc060ce15db45'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:52,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408543575bc060ce15db3f 2025-12-17 22:05:52,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408543575bc060ce15db3f HTTP/1.1" 502 150 2025-12-17 22:05:52,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408543575bc060ce15db3f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:53,295 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408547575bc060ce15db4b 2025-12-17 22:05:53,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408547575bc060ce15db4b HTTP/1.1" 502 150 2025-12-17 22:05:53,296 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408547575bc060ce15db4b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:53,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852d575bc060ce15daa9 2025-12-17 22:05:53,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852d575bc060ce15daa9 HTTP/1.1" 502 150 2025-12-17 22:05:53,842 - lava_callback - ERROR - Error processing callback for node 6940852d575bc060ce15daa9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940852d575bc060ce15daa9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940852d575bc060ce15daa9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940852d575bc060ce15daa9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:53,843 - lava_callback - INFO - Starting async job submit for node 69408548575bc060ce15db55 2025-12-17 22:05:53,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:53,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408548575bc060ce15db55 HTTP/1.1" 502 150 2025-12-17 22:05:53,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408548575bc060ce15db55'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:53,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408548575bc060ce15db55 2025-12-17 22:05:53,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408548575bc060ce15db55 HTTP/1.1" 502 150 2025-12-17 22:05:53,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408548575bc060ce15db55'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:54,354 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940852e575bc060ce15daac 2025-12-17 22:05:54,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940852e575bc060ce15daac HTTP/1.1" 502 150 2025-12-17 22:05:54,356 - lava_callback - ERROR - Error processing callback for node 6940852e575bc060ce15daac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940852e575bc060ce15daac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940852e575bc060ce15daac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940852e575bc060ce15daac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:54,356 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408532575bc060ce15dab2 2025-12-17 22:05:54,358 - lava_callback - INFO - Starting async job submit for node 6940854a575bc060ce15db5c 2025-12-17 22:05:54,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:54,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408532575bc060ce15dab2 HTTP/1.1" 502 150 2025-12-17 22:05:54,359 - lava_callback - ERROR - Error processing callback for node 69408532575bc060ce15dab2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408532575bc060ce15dab2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408532575bc060ce15dab2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408532575bc060ce15dab2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:54,361 - lava_callback - INFO - Starting async job submit for node 6940854b575bc060ce15db5f 2025-12-17 22:05:54,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:54,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854a575bc060ce15db5c HTTP/1.1" 502 150 2025-12-17 22:05:54,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854a575bc060ce15db5c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:54,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854a575bc060ce15db5c 2025-12-17 22:05:54,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854a575bc060ce15db5c HTTP/1.1" 502 150 2025-12-17 22:05:54,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854a575bc060ce15db5c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:54,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854b575bc060ce15db5f HTTP/1.1" 502 150 2025-12-17 22:05:54,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854b575bc060ce15db5f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:54,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854b575bc060ce15db5f 2025-12-17 22:05:54,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854b575bc060ce15db5f HTTP/1.1" 502 150 2025-12-17 22:05:54,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854b575bc060ce15db5f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:55,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408548575bc060ce15db55 2025-12-17 22:05:55,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408548575bc060ce15db55 HTTP/1.1" 502 150 2025-12-17 22:05:55,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408548575bc060ce15db55'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:55,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853e575bc060ce15db36 2025-12-17 22:05:55,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853e575bc060ce15db36 HTTP/1.1" 502 150 2025-12-17 22:05:55,890 - lava_callback - ERROR - Error processing callback for node 6940853e575bc060ce15db36: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940853e575bc060ce15db36 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940853e575bc060ce15db36 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940853e575bc060ce15db36 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:55,892 - lava_callback - INFO - Starting async job submit for node 6940854e575bc060ce15db65 2025-12-17 22:05:55,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:55,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854e575bc060ce15db65 HTTP/1.1" 502 150 2025-12-17 22:05:55,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854e575bc060ce15db65'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:55,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854e575bc060ce15db65 2025-12-17 22:05:55,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854e575bc060ce15db65 HTTP/1.1" 502 150 2025-12-17 22:05:55,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854e575bc060ce15db65'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:56,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854a575bc060ce15db5c 2025-12-17 22:05:56,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854a575bc060ce15db5c HTTP/1.1" 502 150 2025-12-17 22:05:56,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854a575bc060ce15db5c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:56,381 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854b575bc060ce15db5f 2025-12-17 22:05:56,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854b575bc060ce15db5f HTTP/1.1" 502 150 2025-12-17 22:05:56,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854b575bc060ce15db5f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53296 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:05:57,297 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408547575bc060ce15db4b 2025-12-17 22:05:57,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ee575bc060ce15d99f 2025-12-17 22:05:57,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854e575bc060ce15db65 2025-12-17 22:05:57,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854e575bc060ce15db65 HTTP/1.1" 502 150 2025-12-17 22:05:57,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854e575bc060ce15db65'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:58,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853f575bc060ce15db39 2025-12-17 22:05:58,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853f575bc060ce15db39 HTTP/1.1" 502 150 2025-12-17 22:05:58,194 - lava_callback - ERROR - Error processing callback for node 6940853f575bc060ce15db39: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940853f575bc060ce15db39 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940853f575bc060ce15db39 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940853f575bc060ce15db39 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:58,196 - lava_callback - INFO - Starting async job submit for node 6940854f575bc060ce15db68 2025-12-17 22:05:58,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:58,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854f575bc060ce15db68 HTTP/1.1" 502 150 2025-12-17 22:05:58,202 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854f575bc060ce15db68'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:58,202 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854f575bc060ce15db68 2025-12-17 22:05:58,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854f575bc060ce15db68 HTTP/1.1" 502 150 2025-12-17 22:05:58,203 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854f575bc060ce15db68'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:58,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408533575bc060ce15dab6 2025-12-17 22:05:58,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408533575bc060ce15dab6 HTTP/1.1" 502 150 2025-12-17 22:05:58,450 - lava_callback - ERROR - Error processing callback for node 69408533575bc060ce15dab6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408533575bc060ce15dab6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408533575bc060ce15dab6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408533575bc060ce15dab6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:05:58,452 - lava_callback - INFO - Starting async job submit for node 6940854c575bc060ce15db62 2025-12-17 22:05:58,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:05:58,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854c575bc060ce15db62 HTTP/1.1" 502 150 2025-12-17 22:05:58,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854c575bc060ce15db62'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:58,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854c575bc060ce15db62 2025-12-17 22:05:58,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854c575bc060ce15db62 HTTP/1.1" 502 150 2025-12-17 22:05:58,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854c575bc060ce15db62'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:59,281 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408544575bc060ce15db42 2025-12-17 22:05:59,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408544575bc060ce15db42 HTTP/1.1" 502 150 2025-12-17 22:05:59,283 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408544575bc060ce15db42'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:59,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408534575bc060ce15dabd HTTP/1.1" 502 150 2025-12-17 22:05:59,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408534575bc060ce15dabd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:05:59,389 - lava_callback - INFO - Received callback for node 69408638575bc060ce15df81 INFO: 172.21.0.1:53302 - "POST /node/69408638575bc060ce15df81 HTTP/1.0" 202 Accepted 2025-12-17 22:05:59,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408548575bc060ce15db55 2025-12-17 22:05:59,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408548575bc060ce15db55 HTTP/1.1" 502 150 2025-12-17 22:05:59,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408548575bc060ce15db55'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:00,128 - lava_callback - INFO - Received callback for node 69408639575bc060ce15df86 INFO: 172.21.0.1:48266 - "POST /node/69408639575bc060ce15df86 HTTP/1.0" 202 Accepted 2025-12-17 22:06:00,203 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854f575bc060ce15db68 2025-12-17 22:06:00,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408536575bc060ce15dac5 2025-12-17 22:06:00,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408536575bc060ce15dac5 HTTP/1.1" 502 150 2025-12-17 22:06:00,307 - lava_callback - ERROR - Error processing callback for node 69408536575bc060ce15dac5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408536575bc060ce15dac5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408536575bc060ce15dac5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408536575bc060ce15dac5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:00,309 - lava_callback - INFO - Starting async job submit for node 6940855f575bc060ce15db92 2025-12-17 22:06:00,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:00,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855f575bc060ce15db92 HTTP/1.1" 502 150 2025-12-17 22:06:00,341 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855f575bc060ce15db92'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:00,341 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855f575bc060ce15db92 2025-12-17 22:06:00,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855f575bc060ce15db92 HTTP/1.1" 502 150 2025-12-17 22:06:00,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855f575bc060ce15db92'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:00,369 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854a575bc060ce15db5c 2025-12-17 22:06:00,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854a575bc060ce15db5c HTTP/1.1" 502 150 2025-12-17 22:06:00,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854a575bc060ce15db5c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:00,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854b575bc060ce15db5f 2025-12-17 22:06:00,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854b575bc060ce15db5f HTTP/1.1" 502 150 2025-12-17 22:06:00,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854b575bc060ce15db5f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:00,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854c575bc060ce15db62 2025-12-17 22:06:00,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854c575bc060ce15db62 HTTP/1.1" 502 150 2025-12-17 22:06:00,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854c575bc060ce15db62'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:00,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408546575bc060ce15db45 2025-12-17 22:06:00,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408546575bc060ce15db45 HTTP/1.1" 502 150 2025-12-17 22:06:00,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408546575bc060ce15db45'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:00,942 - lava_callback - INFO - Received callback for node 6940863d575bc060ce15df8c INFO: 172.21.0.1:48274 - "POST /node/6940863d575bc060ce15df8c HTTP/1.0" 202 Accepted 2025-12-17 22:06:01,701 - lava_callback - INFO - Received callback for node 6940863e575bc060ce15df8f INFO: 172.21.0.1:48282 - "POST /node/6940863e575bc060ce15df8f HTTP/1.0" 202 Accepted 2025-12-17 22:06:01,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408537575bc060ce15daca 2025-12-17 22:06:01,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408537575bc060ce15daca HTTP/1.1" 502 150 2025-12-17 22:06:01,844 - lava_callback - ERROR - Error processing callback for node 69408537575bc060ce15daca: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408537575bc060ce15daca (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408537575bc060ce15daca (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408537575bc060ce15daca (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:01,846 - lava_callback - INFO - Starting async job submit for node 69408561575bc060ce15db95 2025-12-17 22:06:01,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:01,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408561575bc060ce15db95 HTTP/1.1" 502 150 2025-12-17 22:06:01,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408561575bc060ce15db95'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:01,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408561575bc060ce15db95 2025-12-17 22:06:01,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408561575bc060ce15db95 HTTP/1.1" 502 150 2025-12-17 22:06:01,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408561575bc060ce15db95'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:01,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854e575bc060ce15db65 2025-12-17 22:06:01,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854e575bc060ce15db65 HTTP/1.1" 502 150 2025-12-17 22:06:01,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854e575bc060ce15db65'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48298 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:06:02,342 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855f575bc060ce15db92 2025-12-17 22:06:02,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855f575bc060ce15db92 HTTP/1.1" 502 150 2025-12-17 22:06:02,343 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855f575bc060ce15db92'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:02,436 - lava_callback - INFO - Received callback for node 6940863f575bc060ce15df95 INFO: 172.21.0.1:48310 - "POST /node/6940863f575bc060ce15df95 HTTP/1.0" 202 Accepted 2025-12-17 22:06:02,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408538575bc060ce15dad1 2025-12-17 22:06:02,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408538575bc060ce15dad1 HTTP/1.1" 502 150 2025-12-17 22:06:02,780 - lava_callback - ERROR - Error processing callback for node 69408538575bc060ce15dad1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408538575bc060ce15dad1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408538575bc060ce15dad1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408538575bc060ce15dad1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:02,782 - lava_callback - INFO - Starting async job submit for node 69408551575bc060ce15db6b 2025-12-17 22:06:02,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:02,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408551575bc060ce15db6b HTTP/1.1" 502 150 2025-12-17 22:06:02,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408551575bc060ce15db6b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:02,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408551575bc060ce15db6b 2025-12-17 22:06:02,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408551575bc060ce15db6b HTTP/1.1" 502 150 2025-12-17 22:06:02,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408551575bc060ce15db6b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:03,191 - lava_callback - INFO - Received callback for node 69408641575bc060ce15df9c INFO: 172.21.0.1:48316 - "POST /node/69408641575bc060ce15df9c HTTP/1.0" 202 Accepted 2025-12-17 22:06:03,658 - lava_callback - INFO - Received callback for node 69408647575bc060ce15dfae INFO: 172.21.0.1:48320 - "POST /node/69408647575bc060ce15dfae HTTP/1.0" 202 Accepted 2025-12-17 22:06:03,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408561575bc060ce15db95 2025-12-17 22:06:03,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408561575bc060ce15db95 HTTP/1.1" 502 150 2025-12-17 22:06:03,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408561575bc060ce15db95'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:04,023 - lava_callback - INFO - Received callback for node 69408642575bc060ce15df9f INFO: 172.21.0.1:48336 - "POST /node/69408642575bc060ce15df9f HTTP/1.0" 202 Accepted 2025-12-17 22:06:04,306 - lava_callback - INFO - Received callback for node 6940864a575bc060ce15dfb3 INFO: 172.21.0.1:48352 - "POST /node/6940864a575bc060ce15dfb3 HTTP/1.0" 202 Accepted 2025-12-17 22:06:04,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854c575bc060ce15db62 2025-12-17 22:06:04,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854c575bc060ce15db62 HTTP/1.1" 502 150 2025-12-17 22:06:04,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854c575bc060ce15db62'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:04,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408551575bc060ce15db6b 2025-12-17 22:06:04,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408551575bc060ce15db6b HTTP/1.1" 502 150 2025-12-17 22:06:04,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408551575bc060ce15db6b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:04,865 - lava_callback - INFO - Received callback for node 69408643575bc060ce15dfa2 INFO: 172.21.0.1:48364 - "POST /node/69408643575bc060ce15dfa2 HTTP/1.0" 202 Accepted 2025-12-17 22:06:05,603 - lava_callback - INFO - Received callback for node 69408644575bc060ce15dfa5 INFO: 172.21.0.1:48376 - "POST /node/69408644575bc060ce15dfa5 HTTP/1.0" 202 Accepted 2025-12-17 22:06:06,063 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408541575bc060ce15db3c 2025-12-17 22:06:06,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408541575bc060ce15db3c HTTP/1.1" 502 150 2025-12-17 22:06:06,064 - lava_callback - ERROR - Error processing callback for node 69408541575bc060ce15db3c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408541575bc060ce15db3c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408541575bc060ce15db3c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408541575bc060ce15db3c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:06,066 - lava_callback - INFO - Starting async job submit for node 69408552575bc060ce15db6e 2025-12-17 22:06:06,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:06,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408552575bc060ce15db6e HTTP/1.1" 502 150 2025-12-17 22:06:06,074 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408552575bc060ce15db6e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:06,074 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408552575bc060ce15db6e 2025-12-17 22:06:06,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408552575bc060ce15db6e HTTP/1.1" 502 150 2025-12-17 22:06:06,075 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408552575bc060ce15db6e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:06,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855f575bc060ce15db92 2025-12-17 22:06:06,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855f575bc060ce15db92 HTTP/1.1" 502 150 2025-12-17 22:06:06,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855f575bc060ce15db92'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:06,357 - lava_callback - INFO - Received callback for node 69408645575bc060ce15dfa8 INFO: 172.21.0.1:48388 - "POST /node/69408645575bc060ce15dfa8 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:48394 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:06:07,090 - lava_callback - INFO - Received callback for node 69408647575bc060ce15dfab INFO: 172.21.0.1:48406 - "POST /node/69408647575bc060ce15dfab HTTP/1.0" 202 Accepted 2025-12-17 22:06:07,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408561575bc060ce15db95 2025-12-17 22:06:07,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408548575bc060ce15db55 2025-12-17 22:06:07,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408561575bc060ce15db95 HTTP/1.1" 502 150 2025-12-17 22:06:07,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408561575bc060ce15db95'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:07,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408548575bc060ce15db55 HTTP/1.1" 502 150 2025-12-17 22:06:07,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408548575bc060ce15db55'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:07,872 - lava_callback - INFO - Received callback for node 6940864d575bc060ce15dfbe INFO: 172.21.0.1:48414 - "POST /node/6940864d575bc060ce15dfbe HTTP/1.0" 202 Accepted 2025-12-17 22:06:08,075 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408552575bc060ce15db6e 2025-12-17 22:06:08,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408552575bc060ce15db6e HTTP/1.1" 502 150 2025-12-17 22:06:08,076 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408552575bc060ce15db6e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:08,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854a575bc060ce15db5c 2025-12-17 22:06:08,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854a575bc060ce15db5c HTTP/1.1" 502 150 2025-12-17 22:06:08,371 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854a575bc060ce15db5c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:08,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854b575bc060ce15db5f 2025-12-17 22:06:08,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854b575bc060ce15db5f HTTP/1.1" 502 150 2025-12-17 22:06:08,385 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854b575bc060ce15db5f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:08,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408551575bc060ce15db6b 2025-12-17 22:06:08,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408551575bc060ce15db6b HTTP/1.1" 502 150 2025-12-17 22:06:08,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408551575bc060ce15db6b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:08,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408543575bc060ce15db3f 2025-12-17 22:06:08,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408543575bc060ce15db3f HTTP/1.1" 502 150 2025-12-17 22:06:08,905 - lava_callback - ERROR - Error processing callback for node 69408543575bc060ce15db3f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408543575bc060ce15db3f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408543575bc060ce15db3f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408543575bc060ce15db3f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:08,906 - lava_callback - INFO - Starting async job submit for node 69408554575bc060ce15db71 2025-12-17 22:06:08,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:08,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408554575bc060ce15db71 HTTP/1.1" 502 150 2025-12-17 22:06:08,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408554575bc060ce15db71'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:08,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408554575bc060ce15db71 2025-12-17 22:06:08,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408554575bc060ce15db71 HTTP/1.1" 502 150 2025-12-17 22:06:08,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408554575bc060ce15db71'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:09,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854e575bc060ce15db65 2025-12-17 22:06:09,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854e575bc060ce15db65 HTTP/1.1" 502 150 2025-12-17 22:06:09,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854e575bc060ce15db65'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:10,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408554575bc060ce15db71 2025-12-17 22:06:10,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408554575bc060ce15db71 HTTP/1.1" 502 150 2025-12-17 22:06:10,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408554575bc060ce15db71'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:12,076 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408552575bc060ce15db6e 2025-12-17 22:06:12,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408552575bc060ce15db6e HTTP/1.1" 502 150 2025-12-17 22:06:12,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408552575bc060ce15db6e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:12,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854c575bc060ce15db62 2025-12-17 22:06:12,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854c575bc060ce15db62 HTTP/1.1" 502 150 2025-12-17 22:06:12,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854c575bc060ce15db62'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:12,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408547575bc060ce15db4b HTTP/1.1" 502 150 2025-12-17 22:06:12,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408547575bc060ce15db4b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:14,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855f575bc060ce15db92 2025-12-17 22:06:14,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855f575bc060ce15db92 HTTP/1.1" 502 150 2025-12-17 22:06:14,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855f575bc060ce15db92'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:14,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408554575bc060ce15db71 2025-12-17 22:06:14,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408554575bc060ce15db71 HTTP/1.1" 502 150 2025-12-17 22:06:14,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408554575bc060ce15db71'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48432 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:06:15,283 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408544575bc060ce15db42 2025-12-17 22:06:15,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408544575bc060ce15db42 HTTP/1.1" 502 150 2025-12-17 22:06:15,284 - lava_callback - ERROR - Error processing callback for node 69408544575bc060ce15db42: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408544575bc060ce15db42 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408544575bc060ce15db42 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408544575bc060ce15db42 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:15,286 - lava_callback - INFO - Starting async job submit for node 69408555575bc060ce15db74 2025-12-17 22:06:15,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:15,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408555575bc060ce15db74 HTTP/1.1" 502 150 2025-12-17 22:06:15,292 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408555575bc060ce15db74'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:15,292 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408555575bc060ce15db74 2025-12-17 22:06:15,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408555575bc060ce15db74 HTTP/1.1" 502 150 2025-12-17 22:06:15,293 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408555575bc060ce15db74'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:15,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408534575bc060ce15dabd 2025-12-17 22:06:15,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408534575bc060ce15dabd HTTP/1.1" 502 150 2025-12-17 22:06:15,345 - lava_callback - ERROR - Error processing callback for node 69408534575bc060ce15dabd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408534575bc060ce15dabd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408534575bc060ce15dabd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408534575bc060ce15dabd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:15,347 - lava_callback - INFO - Starting async job submit for node 69408556575bc060ce15db77 2025-12-17 22:06:15,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:15,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408556575bc060ce15db77 HTTP/1.1" 502 150 2025-12-17 22:06:15,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408556575bc060ce15db77'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:15,354 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408556575bc060ce15db77 2025-12-17 22:06:15,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408556575bc060ce15db77 HTTP/1.1" 502 150 2025-12-17 22:06:15,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408556575bc060ce15db77'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:15,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854f575bc060ce15db68 HTTP/1.1" 502 150 2025-12-17 22:06:15,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854f575bc060ce15db68'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:15,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408561575bc060ce15db95 2025-12-17 22:06:15,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408561575bc060ce15db95 HTTP/1.1" 502 150 2025-12-17 22:06:15,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408561575bc060ce15db95'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48440 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:06:16,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408551575bc060ce15db6b 2025-12-17 22:06:16,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408551575bc060ce15db6b HTTP/1.1" 502 150 2025-12-17 22:06:16,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408551575bc060ce15db6b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:16,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408546575bc060ce15db45 2025-12-17 22:06:16,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408546575bc060ce15db45 HTTP/1.1" 502 150 2025-12-17 22:06:16,821 - lava_callback - ERROR - Error processing callback for node 69408546575bc060ce15db45: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408546575bc060ce15db45 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408546575bc060ce15db45 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408546575bc060ce15db45 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:16,823 - lava_callback - INFO - Starting async job submit for node 69408557575bc060ce15db7a 2025-12-17 22:06:16,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:16,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408557575bc060ce15db7a HTTP/1.1" 502 150 2025-12-17 22:06:16,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408557575bc060ce15db7a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:16,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408557575bc060ce15db7a 2025-12-17 22:06:16,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408557575bc060ce15db7a HTTP/1.1" 502 150 2025-12-17 22:06:16,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408557575bc060ce15db7a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:17,293 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408555575bc060ce15db74 2025-12-17 22:06:17,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408555575bc060ce15db74 HTTP/1.1" 502 150 2025-12-17 22:06:17,294 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408555575bc060ce15db74'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:17,355 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408556575bc060ce15db77 2025-12-17 22:06:17,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408556575bc060ce15db77 HTTP/1.1" 502 150 2025-12-17 22:06:17,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408556575bc060ce15db77'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:18,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408557575bc060ce15db7a 2025-12-17 22:06:18,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408557575bc060ce15db7a HTTP/1.1" 502 150 2025-12-17 22:06:18,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408557575bc060ce15db7a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:18,983 - lava_callback - INFO - Received callback for node 69408658575bc060ce15dfda INFO: 172.21.0.1:48448 - "POST /node/69408658575bc060ce15dfda HTTP/1.0" 202 Accepted 2025-12-17 22:06:19,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854f575bc060ce15db68 2025-12-17 22:06:19,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854f575bc060ce15db68 HTTP/1.1" 502 150 2025-12-17 22:06:19,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854f575bc060ce15db68'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:19,731 - lava_callback - INFO - Received callback for node 69408659575bc060ce15dfdd INFO: 172.21.0.1:42836 - "POST /node/69408659575bc060ce15dfdd HTTP/1.0" 202 Accepted 2025-12-17 22:06:20,080 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408552575bc060ce15db6e 2025-12-17 22:06:20,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408552575bc060ce15db6e HTTP/1.1" 502 150 2025-12-17 22:06:20,081 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408552575bc060ce15db6e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:20,492 - lava_callback - INFO - Received callback for node 6940865a575bc060ce15dfe0 INFO: 172.21.0.1:42850 - "POST /node/6940865a575bc060ce15dfe0 HTTP/1.0" 202 Accepted 2025-12-17 22:06:20,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408547575bc060ce15db4b 2025-12-17 22:06:20,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408547575bc060ce15db4b HTTP/1.1" 502 150 2025-12-17 22:06:20,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408547575bc060ce15db4b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:21,244 - lava_callback - INFO - Received callback for node 6940865c575bc060ce15dfe3 INFO: 172.21.0.1:42864 - "POST /node/6940865c575bc060ce15dfe3 HTTP/1.0" 202 Accepted 2025-12-17 22:06:21,294 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408555575bc060ce15db74 2025-12-17 22:06:21,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408555575bc060ce15db74 HTTP/1.1" 502 150 2025-12-17 22:06:21,296 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408555575bc060ce15db74'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:21,299 - lava_callback - INFO - Received callback for node 694081dd575bc060ce15cc11 INFO: 172.21.0.1:42878 - "POST /node/694081dd575bc060ce15cc11 HTTP/1.0" 202 Accepted 2025-12-17 22:06:21,357 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408556575bc060ce15db77 2025-12-17 22:06:21,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408556575bc060ce15db77 HTTP/1.1" 502 150 2025-12-17 22:06:21,358 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408556575bc060ce15db77'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:22,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408557575bc060ce15db7a 2025-12-17 22:06:22,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408557575bc060ce15db7a HTTP/1.1" 502 150 2025-12-17 22:06:22,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408557575bc060ce15db7a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:22,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408554575bc060ce15db71 2025-12-17 22:06:22,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408554575bc060ce15db71 HTTP/1.1" 502 150 2025-12-17 22:06:22,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408554575bc060ce15db71'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:23,673 - lava_callback - INFO - Received callback for node 6940864e575bc060ce15dfc3 INFO: 172.21.0.1:42894 - "POST /node/6940864e575bc060ce15dfc3 HTTP/1.0" 202 Accepted 2025-12-17 22:06:23,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408548575bc060ce15db55 2025-12-17 22:06:23,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408548575bc060ce15db55 HTTP/1.1" 502 150 2025-12-17 22:06:23,858 - lava_callback - ERROR - Error processing callback for node 69408548575bc060ce15db55: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408548575bc060ce15db55 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408548575bc060ce15db55 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408548575bc060ce15db55 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:23,861 - lava_callback - INFO - Starting async job submit for node 69408559575bc060ce15db7d 2025-12-17 22:06:23,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:23,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408559575bc060ce15db7d HTTP/1.1" 502 150 2025-12-17 22:06:23,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408559575bc060ce15db7d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:23,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408559575bc060ce15db7d 2025-12-17 22:06:23,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408559575bc060ce15db7d HTTP/1.1" 502 150 2025-12-17 22:06:23,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408559575bc060ce15db7d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:24,323 - lava_callback - INFO - Received callback for node 6940864f575bc060ce15dfc8 INFO: 172.21.0.1:42898 - "POST /node/6940864f575bc060ce15dfc8 HTTP/1.0" 202 Accepted 2025-12-17 22:06:24,371 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854a575bc060ce15db5c 2025-12-17 22:06:24,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854a575bc060ce15db5c HTTP/1.1" 502 150 2025-12-17 22:06:24,372 - lava_callback - ERROR - Error processing callback for node 6940854a575bc060ce15db5c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940854a575bc060ce15db5c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940854a575bc060ce15db5c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940854a575bc060ce15db5c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:24,374 - lava_callback - INFO - Starting async job submit for node 6940855b575bc060ce15db87 2025-12-17 22:06:24,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:24,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854b575bc060ce15db5f 2025-12-17 22:06:24,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854b575bc060ce15db5f HTTP/1.1" 502 150 2025-12-17 22:06:24,386 - lava_callback - ERROR - Error processing callback for node 6940854b575bc060ce15db5f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940854b575bc060ce15db5f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940854b575bc060ce15db5f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940854b575bc060ce15db5f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:24,388 - lava_callback - INFO - Starting async job submit for node 6940855a575bc060ce15db82 2025-12-17 22:06:24,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:24,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855a575bc060ce15db82 HTTP/1.1" 502 150 2025-12-17 22:06:24,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855a575bc060ce15db82'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:24,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855a575bc060ce15db82 2025-12-17 22:06:24,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855a575bc060ce15db82 HTTP/1.1" 502 150 2025-12-17 22:06:24,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855a575bc060ce15db82'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:24,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855b575bc060ce15db87 HTTP/1.1" 502 150 2025-12-17 22:06:24,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855b575bc060ce15db87'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:24,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855b575bc060ce15db87 2025-12-17 22:06:24,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855b575bc060ce15db87 HTTP/1.1" 502 150 2025-12-17 22:06:24,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855b575bc060ce15db87'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:24,988 - lava_callback - INFO - Received callback for node 69408650575bc060ce15dfcb INFO: 172.21.0.1:42912 - "POST /node/69408650575bc060ce15dfcb HTTP/1.0" 202 Accepted 2025-12-17 22:06:25,652 - lava_callback - INFO - Received callback for node 69408652575bc060ce15dfce INFO: 172.21.0.1:42924 - "POST /node/69408652575bc060ce15dfce HTTP/1.0" 202 Accepted 2025-12-17 22:06:25,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408559575bc060ce15db7d 2025-12-17 22:06:25,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408559575bc060ce15db7d HTTP/1.1" 502 150 2025-12-17 22:06:25,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408559575bc060ce15db7d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:25,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854e575bc060ce15db65 2025-12-17 22:06:25,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854e575bc060ce15db65 HTTP/1.1" 502 150 2025-12-17 22:06:25,905 - lava_callback - ERROR - Error processing callback for node 6940854e575bc060ce15db65: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940854e575bc060ce15db65 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940854e575bc060ce15db65 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940854e575bc060ce15db65 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:25,907 - lava_callback - INFO - Starting async job submit for node 6940855c575bc060ce15db8a 2025-12-17 22:06:25,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:25,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855c575bc060ce15db8a HTTP/1.1" 502 150 2025-12-17 22:06:25,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855c575bc060ce15db8a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:25,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855c575bc060ce15db8a 2025-12-17 22:06:25,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855c575bc060ce15db8a HTTP/1.1" 502 150 2025-12-17 22:06:25,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855c575bc060ce15db8a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:26,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855a575bc060ce15db82 2025-12-17 22:06:26,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855a575bc060ce15db82 HTTP/1.1" 502 150 2025-12-17 22:06:26,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855a575bc060ce15db82'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:26,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855b575bc060ce15db87 2025-12-17 22:06:26,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855b575bc060ce15db87 HTTP/1.1" 502 150 2025-12-17 22:06:26,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855b575bc060ce15db87'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42940 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:06:27,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854f575bc060ce15db68 2025-12-17 22:06:27,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854f575bc060ce15db68 HTTP/1.1" 502 150 2025-12-17 22:06:27,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940854f575bc060ce15db68'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:27,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855c575bc060ce15db8a 2025-12-17 22:06:27,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855c575bc060ce15db8a HTTP/1.1" 502 150 2025-12-17 22:06:27,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855c575bc060ce15db8a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:28,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ee575bc060ce15d99f HTTP/1.1" 502 150 2025-12-17 22:06:28,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ee575bc060ce15d99f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:28,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854c575bc060ce15db62 2025-12-17 22:06:28,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854c575bc060ce15db62 HTTP/1.1" 502 150 2025-12-17 22:06:28,465 - lava_callback - ERROR - Error processing callback for node 6940854c575bc060ce15db62: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940854c575bc060ce15db62 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940854c575bc060ce15db62 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940854c575bc060ce15db62 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:28,467 - lava_callback - INFO - Starting async job submit for node 6940855e575bc060ce15db8f 2025-12-17 22:06:28,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:28,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855e575bc060ce15db8f HTTP/1.1" 502 150 2025-12-17 22:06:28,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855e575bc060ce15db8f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:28,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855e575bc060ce15db8f 2025-12-17 22:06:28,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855e575bc060ce15db8f HTTP/1.1" 502 150 2025-12-17 22:06:28,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855e575bc060ce15db8f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:28,760 - lava_callback - INFO - Received callback for node 69408655575bc060ce15dfd4 INFO: 172.21.0.1:42948 - "POST /node/69408655575bc060ce15dfd4 HTTP/1.0" 202 Accepted 2025-12-17 22:06:29,296 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408555575bc060ce15db74 2025-12-17 22:06:29,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408555575bc060ce15db74 HTTP/1.1" 502 150 2025-12-17 22:06:29,297 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408555575bc060ce15db74'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:29,358 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408556575bc060ce15db77 2025-12-17 22:06:29,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408556575bc060ce15db77 HTTP/1.1" 502 150 2025-12-17 22:06:29,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408556575bc060ce15db77'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:29,426 - lava_callback - INFO - Received callback for node 69408656575bc060ce15dfd7 INFO: 172.21.0.1:42952 - "POST /node/69408656575bc060ce15dfd7 HTTP/1.0" 202 Accepted 2025-12-17 22:06:29,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408559575bc060ce15db7d 2025-12-17 22:06:29,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408559575bc060ce15db7d HTTP/1.1" 502 150 2025-12-17 22:06:29,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408559575bc060ce15db7d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:30,081 - lava_callback - INFO - Received callback for node 69408663575bc060ce15e008 INFO: 172.21.0.1:48656 - "POST /node/69408663575bc060ce15e008 HTTP/1.0" 202 Accepted 2025-12-17 22:06:30,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855f575bc060ce15db92 2025-12-17 22:06:30,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855f575bc060ce15db92 HTTP/1.1" 502 150 2025-12-17 22:06:30,348 - lava_callback - ERROR - Error processing callback for node 6940855f575bc060ce15db92: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940855f575bc060ce15db92 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940855f575bc060ce15db92 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940855f575bc060ce15db92 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:30,350 - lava_callback - INFO - Starting async job submit for node 69408569575bc060ce15dba4 2025-12-17 22:06:30,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:30,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408569575bc060ce15dba4 HTTP/1.1" 502 150 2025-12-17 22:06:30,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408569575bc060ce15dba4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:30,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408569575bc060ce15dba4 2025-12-17 22:06:30,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408569575bc060ce15dba4 HTTP/1.1" 502 150 2025-12-17 22:06:30,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408569575bc060ce15dba4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:30,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855a575bc060ce15db82 2025-12-17 22:06:30,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855a575bc060ce15db82 HTTP/1.1" 502 150 2025-12-17 22:06:30,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855a575bc060ce15db82'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:30,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855b575bc060ce15db87 2025-12-17 22:06:30,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855b575bc060ce15db87 HTTP/1.1" 502 150 2025-12-17 22:06:30,405 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855b575bc060ce15db87'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:30,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855e575bc060ce15db8f 2025-12-17 22:06:30,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855e575bc060ce15db8f HTTP/1.1" 502 150 2025-12-17 22:06:30,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855e575bc060ce15db8f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:30,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408557575bc060ce15db7a 2025-12-17 22:06:30,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408557575bc060ce15db7a HTTP/1.1" 502 150 2025-12-17 22:06:30,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408557575bc060ce15db7a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:31,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408561575bc060ce15db95 2025-12-17 22:06:31,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408561575bc060ce15db95 HTTP/1.1" 502 150 2025-12-17 22:06:31,859 - lava_callback - ERROR - Error processing callback for node 69408561575bc060ce15db95: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408561575bc060ce15db95 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408561575bc060ce15db95 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408561575bc060ce15db95 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:31,861 - lava_callback - INFO - Starting async job submit for node 69408569575bc060ce15dba7 2025-12-17 22:06:31,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:31,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408569575bc060ce15dba7 HTTP/1.1" 502 150 2025-12-17 22:06:31,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408569575bc060ce15dba7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:31,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408569575bc060ce15dba7 2025-12-17 22:06:31,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408569575bc060ce15dba7 HTTP/1.1" 502 150 2025-12-17 22:06:31,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408569575bc060ce15dba7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:31,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855c575bc060ce15db8a 2025-12-17 22:06:31,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855c575bc060ce15db8a HTTP/1.1" 502 150 2025-12-17 22:06:31,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855c575bc060ce15db8a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:32,363 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408569575bc060ce15dba4 2025-12-17 22:06:32,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408569575bc060ce15dba4 HTTP/1.1" 502 150 2025-12-17 22:06:32,364 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408569575bc060ce15dba4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:32,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408551575bc060ce15db6b 2025-12-17 22:06:32,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408551575bc060ce15db6b HTTP/1.1" 502 150 2025-12-17 22:06:32,802 - lava_callback - ERROR - Error processing callback for node 69408551575bc060ce15db6b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408551575bc060ce15db6b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408551575bc060ce15db6b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408551575bc060ce15db6b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:32,803 - lava_callback - INFO - Starting async job submit for node 6940856a575bc060ce15dbaa 2025-12-17 22:06:32,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:32,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856a575bc060ce15dbaa HTTP/1.1" 502 150 2025-12-17 22:06:32,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856a575bc060ce15dbaa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:32,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856a575bc060ce15dbaa 2025-12-17 22:06:32,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856a575bc060ce15dbaa HTTP/1.1" 502 150 2025-12-17 22:06:32,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856a575bc060ce15dbaa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:33,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408569575bc060ce15dba7 2025-12-17 22:06:33,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408569575bc060ce15dba7 HTTP/1.1" 502 150 2025-12-17 22:06:33,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408569575bc060ce15dba7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:34,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855e575bc060ce15db8f 2025-12-17 22:06:34,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855e575bc060ce15db8f HTTP/1.1" 502 150 2025-12-17 22:06:34,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855e575bc060ce15db8f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:34,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856a575bc060ce15dbaa 2025-12-17 22:06:34,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856a575bc060ce15dbaa HTTP/1.1" 502 150 2025-12-17 22:06:34,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856a575bc060ce15dbaa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:36,081 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408552575bc060ce15db6e 2025-12-17 22:06:36,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408552575bc060ce15db6e HTTP/1.1" 502 150 2025-12-17 22:06:36,083 - lava_callback - ERROR - Error processing callback for node 69408552575bc060ce15db6e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408552575bc060ce15db6e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408552575bc060ce15db6e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408552575bc060ce15db6e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:36,085 - lava_callback - INFO - Starting async job submit for node 6940856c575bc060ce15dbad 2025-12-17 22:06:36,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:36,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856c575bc060ce15dbad HTTP/1.1" 502 150 2025-12-17 22:06:36,091 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856c575bc060ce15dbad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:36,091 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856c575bc060ce15dbad 2025-12-17 22:06:36,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856c575bc060ce15dbad HTTP/1.1" 502 150 2025-12-17 22:06:36,092 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856c575bc060ce15dbad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:36,364 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408569575bc060ce15dba4 2025-12-17 22:06:36,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408569575bc060ce15dba4 HTTP/1.1" 502 150 2025-12-17 22:06:36,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408569575bc060ce15dba4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48670 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:06:36,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408547575bc060ce15db4b 2025-12-17 22:06:36,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408547575bc060ce15db4b HTTP/1.1" 502 150 2025-12-17 22:06:36,659 - lava_callback - ERROR - Error processing callback for node 69408547575bc060ce15db4b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408547575bc060ce15db4b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408547575bc060ce15db4b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408547575bc060ce15db4b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:36,661 - lava_callback - INFO - Starting async job submit for node 6940856d575bc060ce15dbb0 2025-12-17 22:06:36,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:36,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856d575bc060ce15dbb0 HTTP/1.1" 502 150 2025-12-17 22:06:36,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856d575bc060ce15dbb0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:36,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856d575bc060ce15dbb0 2025-12-17 22:06:36,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856d575bc060ce15dbb0 HTTP/1.1" 502 150 2025-12-17 22:06:36,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856d575bc060ce15dbb0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:37,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408569575bc060ce15dba7 2025-12-17 22:06:37,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408559575bc060ce15db7d 2025-12-17 22:06:37,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408559575bc060ce15db7d HTTP/1.1" 502 150 2025-12-17 22:06:37,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408559575bc060ce15db7d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:38,092 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856c575bc060ce15dbad 2025-12-17 22:06:38,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856c575bc060ce15dbad HTTP/1.1" 502 150 2025-12-17 22:06:38,093 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856c575bc060ce15dbad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:38,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855a575bc060ce15db82 2025-12-17 22:06:38,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855a575bc060ce15db82 HTTP/1.1" 502 150 2025-12-17 22:06:38,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855a575bc060ce15db82'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:38,405 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855b575bc060ce15db87 2025-12-17 22:06:38,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855b575bc060ce15db87 HTTP/1.1" 502 150 2025-12-17 22:06:38,406 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855b575bc060ce15db87'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:38,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856d575bc060ce15dbb0 2025-12-17 22:06:38,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856d575bc060ce15dbb0 HTTP/1.1" 502 150 2025-12-17 22:06:38,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856d575bc060ce15dbb0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:38,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856a575bc060ce15dbaa 2025-12-17 22:06:38,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856a575bc060ce15dbaa HTTP/1.1" 502 150 2025-12-17 22:06:38,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856a575bc060ce15dbaa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:38,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408554575bc060ce15db71 2025-12-17 22:06:38,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408554575bc060ce15db71 HTTP/1.1" 502 150 2025-12-17 22:06:38,919 - lava_callback - ERROR - Error processing callback for node 69408554575bc060ce15db71: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408554575bc060ce15db71 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408554575bc060ce15db71 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408554575bc060ce15db71 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:38,921 - lava_callback - INFO - Starting async job submit for node 69408564575bc060ce15db9b 2025-12-17 22:06:38,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:38,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408564575bc060ce15db9b HTTP/1.1" 502 150 2025-12-17 22:06:38,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408564575bc060ce15db9b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:38,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408564575bc060ce15db9b 2025-12-17 22:06:38,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408564575bc060ce15db9b HTTP/1.1" 502 150 2025-12-17 22:06:38,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408564575bc060ce15db9b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:39,137 - lava_callback - INFO - Received callback for node 6940865d575bc060ce15dfe6 INFO: 172.21.0.1:48674 - "POST /node/6940865d575bc060ce15dfe6 HTTP/1.0" 202 Accepted 2025-12-17 22:06:39,869 - lava_callback - INFO - Received callback for node 6940865e575bc060ce15dfeb INFO: 172.21.0.1:41460 - "POST /node/6940865e575bc060ce15dfeb HTTP/1.0" 202 Accepted 2025-12-17 22:06:39,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855c575bc060ce15db8a 2025-12-17 22:06:39,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855c575bc060ce15db8a HTTP/1.1" 502 150 2025-12-17 22:06:39,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855c575bc060ce15db8a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:40,631 - lava_callback - INFO - Received callback for node 69408660575bc060ce15dff5 INFO: 172.21.0.1:41466 - "POST /node/69408660575bc060ce15dff5 HTTP/1.0" 202 Accepted 2025-12-17 22:06:40,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408564575bc060ce15db9b 2025-12-17 22:06:40,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408564575bc060ce15db9b HTTP/1.1" 502 150 2025-12-17 22:06:40,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408564575bc060ce15db9b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:41,364 - lava_callback - INFO - Received callback for node 69408661575bc060ce15dffc INFO: 172.21.0.1:41474 - "POST /node/69408661575bc060ce15dffc HTTP/1.0" 202 Accepted 2025-12-17 22:06:42,093 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856c575bc060ce15dbad 2025-12-17 22:06:42,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856c575bc060ce15dbad HTTP/1.1" 502 150 2025-12-17 22:06:42,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856c575bc060ce15dbad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:42,151 - lava_callback - INFO - Received callback for node 69408663575bc060ce15e003 INFO: 172.21.0.1:41482 - "POST /node/69408663575bc060ce15e003 HTTP/1.0" 202 Accepted 2025-12-17 22:06:42,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855e575bc060ce15db8f 2025-12-17 22:06:42,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855e575bc060ce15db8f HTTP/1.1" 502 150 2025-12-17 22:06:42,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940855e575bc060ce15db8f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:42,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856d575bc060ce15dbb0 2025-12-17 22:06:42,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856d575bc060ce15dbb0 HTTP/1.1" 502 150 2025-12-17 22:06:42,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856d575bc060ce15dbb0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:43,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940854f575bc060ce15db68 2025-12-17 22:06:43,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940854f575bc060ce15db68 HTTP/1.1" 502 150 2025-12-17 22:06:43,477 - lava_callback - ERROR - Error processing callback for node 6940854f575bc060ce15db68: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940854f575bc060ce15db68 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940854f575bc060ce15db68 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940854f575bc060ce15db68 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:43,479 - lava_callback - INFO - Starting async job submit for node 69408565575bc060ce15db9e 2025-12-17 22:06:43,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:43,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408565575bc060ce15db9e HTTP/1.1" 502 150 2025-12-17 22:06:43,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408565575bc060ce15db9e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:43,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408565575bc060ce15db9e 2025-12-17 22:06:43,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408565575bc060ce15db9e HTTP/1.1" 502 150 2025-12-17 22:06:43,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408565575bc060ce15db9e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:43,687 - lava_callback - INFO - Received callback for node 69408666575bc060ce15e015 INFO: 172.21.0.1:41486 - "POST /node/69408666575bc060ce15e015 HTTP/1.0" 202 Accepted 2025-12-17 22:06:44,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ee575bc060ce15d99f 2025-12-17 22:06:44,354 - lava_callback - INFO - Received callback for node 69408667575bc060ce15e018 INFO: 172.21.0.1:41498 - "POST /node/69408667575bc060ce15e018 HTTP/1.0" 202 Accepted 2025-12-17 22:06:44,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408569575bc060ce15dba4 2025-12-17 22:06:44,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408569575bc060ce15dba4 HTTP/1.1" 502 150 2025-12-17 22:06:44,367 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408569575bc060ce15dba4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:44,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408564575bc060ce15db9b 2025-12-17 22:06:44,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408564575bc060ce15db9b HTTP/1.1" 502 150 2025-12-17 22:06:44,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408564575bc060ce15db9b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:44,991 - lava_callback - INFO - Received callback for node 69408668575bc060ce15e01b INFO: 172.21.0.1:41506 - "POST /node/69408668575bc060ce15e01b HTTP/1.0" 202 Accepted 2025-12-17 22:06:45,297 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408555575bc060ce15db74 2025-12-17 22:06:45,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408555575bc060ce15db74 HTTP/1.1" 502 150 2025-12-17 22:06:45,299 - lava_callback - ERROR - Error processing callback for node 69408555575bc060ce15db74: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408555575bc060ce15db74 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408555575bc060ce15db74 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408555575bc060ce15db74 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:45,301 - lava_callback - INFO - Starting async job submit for node 69408567575bc060ce15dba1 2025-12-17 22:06:45,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:45,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408567575bc060ce15dba1 HTTP/1.1" 502 150 2025-12-17 22:06:45,308 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408567575bc060ce15dba1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:45,309 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408567575bc060ce15dba1 2025-12-17 22:06:45,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408567575bc060ce15dba1 HTTP/1.1" 502 150 2025-12-17 22:06:45,309 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408567575bc060ce15dba1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:45,359 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408556575bc060ce15db77 2025-12-17 22:06:45,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408556575bc060ce15db77 HTTP/1.1" 502 150 2025-12-17 22:06:45,360 - lava_callback - ERROR - Error processing callback for node 69408556575bc060ce15db77: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408556575bc060ce15db77 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408556575bc060ce15db77 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408556575bc060ce15db77 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:45,362 - lava_callback - INFO - Starting async job submit for node 6940bb491e3c24e95681b5f4 2025-12-17 22:06:45,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:45,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940bb491e3c24e95681b5f4 HTTP/1.1" 502 150 2025-12-17 22:06:45,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940bb491e3c24e95681b5f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:45,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940bb491e3c24e95681b5f4 2025-12-17 22:06:45,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940bb491e3c24e95681b5f4 HTTP/1.1" 502 150 2025-12-17 22:06:45,410 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940bb491e3c24e95681b5f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:45,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408565575bc060ce15db9e 2025-12-17 22:06:45,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408565575bc060ce15db9e HTTP/1.1" 502 150 2025-12-17 22:06:45,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408565575bc060ce15db9e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:45,656 - lava_callback - INFO - Received callback for node 6940866a575bc060ce15e022 INFO: 172.21.0.1:41514 - "POST /node/6940866a575bc060ce15e022 HTTP/1.0" 202 Accepted 2025-12-17 22:06:46,301 - lava_callback - INFO - Received callback for node 6940866b575bc060ce15e025 INFO: 172.21.0.1:41516 - "POST /node/6940866b575bc060ce15e025 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:41530 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:06:46,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856a575bc060ce15dbaa 2025-12-17 22:06:46,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856a575bc060ce15dbaa HTTP/1.1" 502 150 2025-12-17 22:06:46,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856a575bc060ce15dbaa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:46,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408557575bc060ce15db7a 2025-12-17 22:06:46,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408557575bc060ce15db7a HTTP/1.1" 502 150 2025-12-17 22:06:46,846 - lava_callback - ERROR - Error processing callback for node 69408557575bc060ce15db7a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408557575bc060ce15db7a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408557575bc060ce15db7a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408557575bc060ce15db7a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:46,848 - lava_callback - INFO - Starting async job submit for node 6940856f575bc060ce15dbb7 2025-12-17 22:06:46,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:46,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856f575bc060ce15dbb7 HTTP/1.1" 502 150 2025-12-17 22:06:46,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856f575bc060ce15dbb7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:46,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856f575bc060ce15dbb7 2025-12-17 22:06:46,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856f575bc060ce15dbb7 HTTP/1.1" 502 150 2025-12-17 22:06:46,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856f575bc060ce15dbb7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:46,944 - lava_callback - INFO - Received callback for node 6940866d575bc060ce15e028 INFO: 172.21.0.1:41532 - "POST /node/6940866d575bc060ce15e028 HTTP/1.0" 202 Accepted 2025-12-17 22:06:47,310 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408567575bc060ce15dba1 2025-12-17 22:06:47,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408567575bc060ce15dba1 HTTP/1.1" 502 150 2025-12-17 22:06:47,311 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408567575bc060ce15dba1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:47,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940bb491e3c24e95681b5f4 2025-12-17 22:06:47,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940bb491e3c24e95681b5f4 HTTP/1.1" 502 150 2025-12-17 22:06:47,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940bb491e3c24e95681b5f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:47,598 - lava_callback - INFO - Received callback for node 6940866e575bc060ce15e02b INFO: 172.21.0.1:41534 - "POST /node/6940866e575bc060ce15e02b HTTP/1.0" 202 Accepted 2025-12-17 22:06:48,230 - lava_callback - INFO - Received callback for node 6940866f575bc060ce15e02e INFO: 172.21.0.1:41540 - "POST /node/6940866f575bc060ce15e02e HTTP/1.0" 202 Accepted 2025-12-17 22:06:48,751 - lava_callback - INFO - Received callback for node 69408665575bc060ce15e012 INFO: 172.21.0.1:41552 - "POST /node/69408665575bc060ce15e012 HTTP/1.0" 202 Accepted 2025-12-17 22:06:48,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856f575bc060ce15dbb7 2025-12-17 22:06:48,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856f575bc060ce15dbb7 HTTP/1.1" 502 150 2025-12-17 22:06:48,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856f575bc060ce15dbb7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:48,888 - lava_callback - INFO - Received callback for node 69408670575bc060ce15e031 INFO: 172.21.0.1:41560 - "POST /node/69408670575bc060ce15e031 HTTP/1.0" 202 Accepted 2025-12-17 22:06:49,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408565575bc060ce15db9e 2025-12-17 22:06:49,546 - lava_callback - INFO - Received callback for node 69408672575bc060ce15e035 INFO: 172.21.0.1:41568 - "POST /node/69408672575bc060ce15e035 HTTP/1.0" 202 Accepted 2025-12-17 22:06:50,095 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856c575bc060ce15dbad 2025-12-17 22:06:50,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856c575bc060ce15dbad HTTP/1.1" 502 150 2025-12-17 22:06:50,096 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856c575bc060ce15dbad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:50,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856d575bc060ce15dbb0 2025-12-17 22:06:50,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856d575bc060ce15dbb0 HTTP/1.1" 502 150 2025-12-17 22:06:50,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856d575bc060ce15dbb0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:51,311 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408567575bc060ce15dba1 2025-12-17 22:06:51,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408567575bc060ce15dba1 HTTP/1.1" 502 150 2025-12-17 22:06:51,312 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408567575bc060ce15dba1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:51,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940bb491e3c24e95681b5f4 2025-12-17 22:06:51,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940bb491e3c24e95681b5f4 HTTP/1.1" 502 150 2025-12-17 22:06:51,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940bb491e3c24e95681b5f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:52,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856f575bc060ce15dbb7 2025-12-17 22:06:52,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856f575bc060ce15dbb7 HTTP/1.1" 502 150 2025-12-17 22:06:52,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856f575bc060ce15dbb7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:52,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408564575bc060ce15db9b 2025-12-17 22:06:52,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408564575bc060ce15db9b HTTP/1.1" 502 150 2025-12-17 22:06:52,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408564575bc060ce15db9b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:53,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408569575bc060ce15dba7 HTTP/1.1" 502 150 2025-12-17 22:06:53,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408569575bc060ce15dba7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:53,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408559575bc060ce15db7d 2025-12-17 22:06:53,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408559575bc060ce15db7d HTTP/1.1" 502 150 2025-12-17 22:06:53,899 - lava_callback - ERROR - Error processing callback for node 69408559575bc060ce15db7d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408559575bc060ce15db7d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408559575bc060ce15db7d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408559575bc060ce15db7d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:53,901 - lava_callback - INFO - Starting async job submit for node 69408570575bc060ce15dbc0 2025-12-17 22:06:53,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:53,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408570575bc060ce15dbc0 HTTP/1.1" 502 150 2025-12-17 22:06:53,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408570575bc060ce15dbc0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:53,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408570575bc060ce15dbc0 2025-12-17 22:06:53,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408570575bc060ce15dbc0 HTTP/1.1" 502 150 2025-12-17 22:06:53,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408570575bc060ce15dbc0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:54,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855a575bc060ce15db82 2025-12-17 22:06:54,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855a575bc060ce15db82 HTTP/1.1" 502 150 2025-12-17 22:06:54,403 - lava_callback - ERROR - Error processing callback for node 6940855a575bc060ce15db82: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940855a575bc060ce15db82 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940855a575bc060ce15db82 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940855a575bc060ce15db82 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:54,405 - lava_callback - INFO - Starting async job submit for node 69408572575bc060ce15dbc7 2025-12-17 22:06:54,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:54,406 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855b575bc060ce15db87 2025-12-17 22:06:54,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855b575bc060ce15db87 HTTP/1.1" 502 150 2025-12-17 22:06:54,407 - lava_callback - ERROR - Error processing callback for node 6940855b575bc060ce15db87: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940855b575bc060ce15db87 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940855b575bc060ce15db87 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940855b575bc060ce15db87 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:54,409 - lava_callback - INFO - Starting async job submit for node 69408575575bc060ce15dbd2 2025-12-17 22:06:54,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:54,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408572575bc060ce15dbc7 HTTP/1.1" 502 150 2025-12-17 22:06:54,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408572575bc060ce15dbc7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:54,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408572575bc060ce15dbc7 2025-12-17 22:06:54,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408572575bc060ce15dbc7 HTTP/1.1" 502 150 2025-12-17 22:06:54,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408572575bc060ce15dbc7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:54,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408575575bc060ce15dbd2 HTTP/1.1" 502 150 2025-12-17 22:06:54,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408575575bc060ce15dbd2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:54,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408575575bc060ce15dbd2 2025-12-17 22:06:54,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408575575bc060ce15dbd2 HTTP/1.1" 502 150 2025-12-17 22:06:54,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408575575bc060ce15dbd2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:55,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408570575bc060ce15dbc0 2025-12-17 22:06:55,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408570575bc060ce15dbc0 HTTP/1.1" 502 150 2025-12-17 22:06:55,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408570575bc060ce15dbc0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:55,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855c575bc060ce15db8a 2025-12-17 22:06:55,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855c575bc060ce15db8a HTTP/1.1" 502 150 2025-12-17 22:06:55,924 - lava_callback - ERROR - Error processing callback for node 6940855c575bc060ce15db8a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940855c575bc060ce15db8a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940855c575bc060ce15db8a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940855c575bc060ce15db8a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:55,926 - lava_callback - INFO - Starting async job submit for node 69408f4a575bc060ce1600f1 2025-12-17 22:06:55,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:55,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4a575bc060ce1600f1 HTTP/1.1" 502 150 2025-12-17 22:06:55,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4a575bc060ce1600f1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:55,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4a575bc060ce1600f1 2025-12-17 22:06:55,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4a575bc060ce1600f1 HTTP/1.1" 502 150 2025-12-17 22:06:55,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4a575bc060ce1600f1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:56,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408572575bc060ce15dbc7 2025-12-17 22:06:56,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408572575bc060ce15dbc7 HTTP/1.1" 502 150 2025-12-17 22:06:56,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408572575bc060ce15dbc7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:56,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408575575bc060ce15dbd2 2025-12-17 22:06:56,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408575575bc060ce15dbd2 HTTP/1.1" 502 150 2025-12-17 22:06:56,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408575575bc060ce15dbd2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36178 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:06:57,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4a575bc060ce1600f1 2025-12-17 22:06:57,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4a575bc060ce1600f1 HTTP/1.1" 502 150 2025-12-17 22:06:57,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4a575bc060ce1600f1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:58,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940855e575bc060ce15db8f 2025-12-17 22:06:58,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940855e575bc060ce15db8f HTTP/1.1" 502 150 2025-12-17 22:06:58,480 - lava_callback - ERROR - Error processing callback for node 6940855e575bc060ce15db8f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940855e575bc060ce15db8f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940855e575bc060ce15db8f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940855e575bc060ce15db8f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:06:58,482 - lava_callback - INFO - Starting async job submit for node 6940853a575bc060ce15dad4 2025-12-17 22:06:58,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:06:58,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853a575bc060ce15dad4 HTTP/1.1" 502 150 2025-12-17 22:06:58,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853a575bc060ce15dad4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:58,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853a575bc060ce15dad4 2025-12-17 22:06:58,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853a575bc060ce15dad4 HTTP/1.1" 502 150 2025-12-17 22:06:58,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853a575bc060ce15dad4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:59,164 - lava_callback - INFO - Received callback for node 69408676575bc060ce15e064 INFO: 172.21.0.1:36192 - "POST /node/69408676575bc060ce15e064 HTTP/1.0" 202 Accepted 2025-12-17 22:06:59,313 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408567575bc060ce15dba1 2025-12-17 22:06:59,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408567575bc060ce15dba1 HTTP/1.1" 502 150 2025-12-17 22:06:59,314 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408567575bc060ce15dba1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:59,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940bb491e3c24e95681b5f4 2025-12-17 22:06:59,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940bb491e3c24e95681b5f4 HTTP/1.1" 502 150 2025-12-17 22:06:59,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940bb491e3c24e95681b5f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:06:59,910 - lava_callback - INFO - Received callback for node 69408678575bc060ce15e06a 2025-12-17 22:06:59,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408570575bc060ce15dbc0 INFO: 172.21.0.1:44266 - "POST /node/69408678575bc060ce15e06a HTTP/1.0" 202 Accepted 2025-12-17 22:06:59,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408570575bc060ce15dbc0 HTTP/1.1" 502 150 2025-12-17 22:06:59,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408570575bc060ce15dbc0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:00,367 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408569575bc060ce15dba4 2025-12-17 22:07:00,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408569575bc060ce15dba4 HTTP/1.1" 502 150 2025-12-17 22:07:00,368 - lava_callback - ERROR - Error processing callback for node 69408569575bc060ce15dba4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408569575bc060ce15dba4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408569575bc060ce15dba4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408569575bc060ce15dba4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:00,371 - lava_callback - INFO - Starting async job submit for node 69408577575bc060ce15dbd7 2025-12-17 22:07:00,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:00,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408577575bc060ce15dbd7 HTTP/1.1" 502 150 2025-12-17 22:07:00,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408577575bc060ce15dbd7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:00,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408577575bc060ce15dbd7 2025-12-17 22:07:00,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408577575bc060ce15dbd7 HTTP/1.1" 502 150 2025-12-17 22:07:00,388 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408577575bc060ce15dbd7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:00,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408572575bc060ce15dbc7 2025-12-17 22:07:00,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408572575bc060ce15dbc7 HTTP/1.1" 502 150 2025-12-17 22:07:00,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408572575bc060ce15dbc7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:00,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408575575bc060ce15dbd2 2025-12-17 22:07:00,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408575575bc060ce15dbd2 HTTP/1.1" 502 150 2025-12-17 22:07:00,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408575575bc060ce15dbd2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:00,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853a575bc060ce15dad4 2025-12-17 22:07:00,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853a575bc060ce15dad4 HTTP/1.1" 502 150 2025-12-17 22:07:00,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853a575bc060ce15dad4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:00,646 - lava_callback - INFO - Received callback for node 6940867b575bc060ce15e070 INFO: 172.21.0.1:44272 - "POST /node/6940867b575bc060ce15e070 HTTP/1.0" 202 Accepted 2025-12-17 22:07:00,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856f575bc060ce15dbb7 2025-12-17 22:07:00,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856f575bc060ce15dbb7 HTTP/1.1" 502 150 2025-12-17 22:07:00,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940856f575bc060ce15dbb7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:01,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408569575bc060ce15dba7 2025-12-17 22:07:01,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408569575bc060ce15dba7 HTTP/1.1" 502 150 2025-12-17 22:07:01,364 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408569575bc060ce15dba7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:01,804 - lava_callback - INFO - Received callback for node 6940867f575bc060ce15e076 INFO: 172.21.0.1:44276 - "POST /node/6940867f575bc060ce15e076 HTTP/1.0" 202 Accepted 2025-12-17 22:07:01,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4a575bc060ce1600f1 2025-12-17 22:07:01,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4a575bc060ce1600f1 HTTP/1.1" 502 150 2025-12-17 22:07:01,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4a575bc060ce1600f1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44280 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:07:02,388 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408577575bc060ce15dbd7 2025-12-17 22:07:02,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408577575bc060ce15dbd7 HTTP/1.1" 502 150 2025-12-17 22:07:02,389 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408577575bc060ce15dbd7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:02,598 - lava_callback - INFO - Received callback for node 69408680575bc060ce15e079 INFO: 172.21.0.1:44288 - "POST /node/69408680575bc060ce15e079 HTTP/1.0" 202 Accepted 2025-12-17 22:07:02,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856a575bc060ce15dbaa 2025-12-17 22:07:02,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856a575bc060ce15dbaa HTTP/1.1" 502 150 2025-12-17 22:07:02,822 - lava_callback - ERROR - Error processing callback for node 6940856a575bc060ce15dbaa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940856a575bc060ce15dbaa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940856a575bc060ce15dbaa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940856a575bc060ce15dbaa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:02,823 - lava_callback - INFO - Starting async job submit for node 69408572575bc060ce15dbca 2025-12-17 22:07:02,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:02,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408572575bc060ce15dbca HTTP/1.1" 502 150 2025-12-17 22:07:02,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408572575bc060ce15dbca'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:02,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408572575bc060ce15dbca 2025-12-17 22:07:02,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408572575bc060ce15dbca HTTP/1.1" 502 150 2025-12-17 22:07:02,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408572575bc060ce15dbca'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:03,393 - lava_callback - INFO - Received callback for node 69408681575bc060ce15e07c INFO: 172.21.0.1:44292 - "POST /node/69408681575bc060ce15e07c HTTP/1.0" 202 Accepted 2025-12-17 22:07:03,687 - lava_callback - INFO - Received callback for node 69408674575bc060ce15e05d INFO: 172.21.0.1:44306 - "POST /node/69408674575bc060ce15e05d HTTP/1.0" 202 Accepted 2025-12-17 22:07:04,135 - lava_callback - INFO - Received callback for node 69408682575bc060ce15e07f INFO: 172.21.0.1:44322 - "POST /node/69408682575bc060ce15e07f HTTP/1.0" 202 Accepted 2025-12-17 22:07:04,324 - lava_callback - INFO - Received callback for node 69408676575bc060ce15e067 INFO: 172.21.0.1:44334 - "POST /node/69408676575bc060ce15e067 HTTP/1.0" 202 Accepted 2025-12-17 22:07:04,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853a575bc060ce15dad4 2025-12-17 22:07:04,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408572575bc060ce15dbca 2025-12-17 22:07:04,867 - lava_callback - INFO - Received callback for node 69408260575bc060ce15cf40 INFO: 172.21.0.1:44348 - "POST /node/69408260575bc060ce15cf40 HTTP/1.0" 202 Accepted 2025-12-17 22:07:04,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408565575bc060ce15db9e HTTP/1.1" 502 150 2025-12-17 22:07:04,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408565575bc060ce15db9e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:04,969 - lava_callback - INFO - Received callback for node 69408679575bc060ce15e06d INFO: 172.21.0.1:44364 - "POST /node/69408679575bc060ce15e06d HTTP/1.0" 202 Accepted 2025-12-17 22:07:05,625 - lava_callback - INFO - Received callback for node 6940867c575bc060ce15e073 INFO: 172.21.0.1:44370 - "POST /node/6940867c575bc060ce15e073 HTTP/1.0" 202 Accepted 2025-12-17 22:07:06,096 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856c575bc060ce15dbad 2025-12-17 22:07:06,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856c575bc060ce15dbad HTTP/1.1" 502 150 2025-12-17 22:07:06,098 - lava_callback - ERROR - Error processing callback for node 6940856c575bc060ce15dbad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940856c575bc060ce15dbad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940856c575bc060ce15dbad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940856c575bc060ce15dbad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:06,101 - lava_callback - INFO - Starting async job submit for node 69408574575bc060ce15dbcf 2025-12-17 22:07:06,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:06,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408574575bc060ce15dbcf HTTP/1.1" 502 150 2025-12-17 22:07:06,108 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408574575bc060ce15dbcf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:06,108 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408574575bc060ce15dbcf 2025-12-17 22:07:06,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408574575bc060ce15dbcf HTTP/1.1" 502 150 2025-12-17 22:07:06,109 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408574575bc060ce15dbcf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:06,266 - lava_callback - INFO - Received callback for node 69408685575bc060ce15e085 INFO: 172.21.0.1:44382 - "POST /node/69408685575bc060ce15e085 HTTP/1.0" 202 Accepted 2025-12-17 22:07:06,390 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408577575bc060ce15dbd7 2025-12-17 22:07:06,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408577575bc060ce15dbd7 HTTP/1.1" 502 150 2025-12-17 22:07:06,391 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408577575bc060ce15dbd7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44396 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:07:06,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856d575bc060ce15dbb0 2025-12-17 22:07:06,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856d575bc060ce15dbb0 HTTP/1.1" 502 150 2025-12-17 22:07:06,680 - lava_callback - ERROR - Error processing callback for node 6940856d575bc060ce15dbb0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940856d575bc060ce15dbb0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940856d575bc060ce15dbb0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940856d575bc060ce15dbb0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:06,684 - lava_callback - INFO - Starting async job submit for node 6940857a575bc060ce15dc0d 2025-12-17 22:07:06,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:06,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857a575bc060ce15dc0d HTTP/1.1" 502 150 2025-12-17 22:07:06,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940857a575bc060ce15dc0d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:06,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940857a575bc060ce15dc0d 2025-12-17 22:07:06,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857a575bc060ce15dc0d HTTP/1.1" 502 150 2025-12-17 22:07:06,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940857a575bc060ce15dc0d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:07,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408570575bc060ce15dbc0 2025-12-17 22:07:07,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408570575bc060ce15dbc0 HTTP/1.1" 502 150 2025-12-17 22:07:07,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408570575bc060ce15dbc0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:08,109 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408574575bc060ce15dbcf 2025-12-17 22:07:08,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408574575bc060ce15dbcf HTTP/1.1" 502 150 2025-12-17 22:07:08,110 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408574575bc060ce15dbcf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:08,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408572575bc060ce15dbc7 2025-12-17 22:07:08,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408572575bc060ce15dbc7 HTTP/1.1" 502 150 2025-12-17 22:07:08,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408572575bc060ce15dbc7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:08,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408575575bc060ce15dbd2 2025-12-17 22:07:08,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408575575bc060ce15dbd2 HTTP/1.1" 502 150 2025-12-17 22:07:08,425 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408575575bc060ce15dbd2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:08,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940857a575bc060ce15dc0d 2025-12-17 22:07:08,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857a575bc060ce15dc0d HTTP/1.1" 502 150 2025-12-17 22:07:08,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940857a575bc060ce15dc0d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:08,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408564575bc060ce15db9b 2025-12-17 22:07:08,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408564575bc060ce15db9b HTTP/1.1" 502 150 2025-12-17 22:07:08,935 - lava_callback - ERROR - Error processing callback for node 69408564575bc060ce15db9b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408564575bc060ce15db9b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408564575bc060ce15db9b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408564575bc060ce15db9b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:08,937 - lava_callback - INFO - Starting async job submit for node 6940857c575bc060ce15dc3b 2025-12-17 22:07:08,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:08,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857c575bc060ce15dc3b HTTP/1.1" 502 150 2025-12-17 22:07:08,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940857c575bc060ce15dc3b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:08,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940857c575bc060ce15dc3b 2025-12-17 22:07:08,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857c575bc060ce15dc3b HTTP/1.1" 502 150 2025-12-17 22:07:08,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940857c575bc060ce15dc3b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:09,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4a575bc060ce1600f1 2025-12-17 22:07:09,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4a575bc060ce1600f1 HTTP/1.1" 502 150 2025-12-17 22:07:09,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4a575bc060ce1600f1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:10,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940857c575bc060ce15dc3b 2025-12-17 22:07:10,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857c575bc060ce15dc3b HTTP/1.1" 502 150 2025-12-17 22:07:10,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940857c575bc060ce15dc3b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:12,111 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408574575bc060ce15dbcf 2025-12-17 22:07:12,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408574575bc060ce15dbcf HTTP/1.1" 502 150 2025-12-17 22:07:12,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408574575bc060ce15dbcf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:12,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940857a575bc060ce15dc0d 2025-12-17 22:07:12,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857a575bc060ce15dc0d HTTP/1.1" 502 150 2025-12-17 22:07:12,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940857a575bc060ce15dc0d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:12,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408565575bc060ce15db9e 2025-12-17 22:07:12,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408565575bc060ce15db9e HTTP/1.1" 502 150 2025-12-17 22:07:12,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408565575bc060ce15db9e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:14,391 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408577575bc060ce15dbd7 2025-12-17 22:07:14,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408577575bc060ce15dbd7 HTTP/1.1" 502 150 2025-12-17 22:07:14,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408577575bc060ce15dbd7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:14,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ee575bc060ce15d99f HTTP/1.1" 502 150 2025-12-17 22:07:14,864 - lava_callback - ERROR - Error processing callback for node 694084ee575bc060ce15d99f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ee575bc060ce15d99f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ee575bc060ce15d99f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ee575bc060ce15d99f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:14,866 - lava_callback - INFO - Starting async job submit for node 6940912f575bc060ce1607b2 2025-12-17 22:07:14,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:14,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912f575bc060ce1607b2 HTTP/1.1" 502 150 2025-12-17 22:07:14,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940912f575bc060ce1607b2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:14,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940912f575bc060ce1607b2 2025-12-17 22:07:14,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912f575bc060ce1607b2 HTTP/1.1" 502 150 2025-12-17 22:07:14,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940912f575bc060ce1607b2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:14,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940857c575bc060ce15dc3b 2025-12-17 22:07:14,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857c575bc060ce15dc3b HTTP/1.1" 502 150 2025-12-17 22:07:14,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940857c575bc060ce15dc3b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60358 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:07:15,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408567575bc060ce15dba1 2025-12-17 22:07:15,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940bb491e3c24e95681b5f4 2025-12-17 22:07:15,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940bb491e3c24e95681b5f4 HTTP/1.1" 502 150 2025-12-17 22:07:15,418 - lava_callback - ERROR - Error processing callback for node 6940bb491e3c24e95681b5f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940bb491e3c24e95681b5f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940bb491e3c24e95681b5f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940bb491e3c24e95681b5f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:15,420 - lava_callback - INFO - Starting async job submit for node 6940857e575bc060ce15dc4b 2025-12-17 22:07:15,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:15,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857e575bc060ce15dc4b HTTP/1.1" 502 150 2025-12-17 22:07:15,437 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940857e575bc060ce15dc4b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:15,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940857e575bc060ce15dc4b 2025-12-17 22:07:15,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857e575bc060ce15dc4b HTTP/1.1" 502 150 2025-12-17 22:07:15,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940857e575bc060ce15dc4b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60366 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:07:16,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940856f575bc060ce15dbb7 2025-12-17 22:07:16,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940856f575bc060ce15dbb7 HTTP/1.1" 502 150 2025-12-17 22:07:16,861 - lava_callback - ERROR - Error processing callback for node 6940856f575bc060ce15dbb7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940856f575bc060ce15dbb7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940856f575bc060ce15dbb7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940856f575bc060ce15dbb7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:16,863 - lava_callback - INFO - Starting async job submit for node 69408580575bc060ce15dc4e 2025-12-17 22:07:16,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:16,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408580575bc060ce15dc4e HTTP/1.1" 502 150 2025-12-17 22:07:16,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408580575bc060ce15dc4e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:16,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408580575bc060ce15dc4e 2025-12-17 22:07:16,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408580575bc060ce15dc4e HTTP/1.1" 502 150 2025-12-17 22:07:16,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408580575bc060ce15dc4e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:16,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940912f575bc060ce1607b2 2025-12-17 22:07:16,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912f575bc060ce1607b2 HTTP/1.1" 502 150 2025-12-17 22:07:16,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940912f575bc060ce1607b2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:17,364 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408569575bc060ce15dba7 2025-12-17 22:07:17,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408569575bc060ce15dba7 HTTP/1.1" 502 150 2025-12-17 22:07:17,366 - lava_callback - ERROR - Error processing callback for node 69408569575bc060ce15dba7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408569575bc060ce15dba7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408569575bc060ce15dba7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408569575bc060ce15dba7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:17,368 - lava_callback - INFO - Starting async job submit for node 69408581575bc060ce15dc51 2025-12-17 22:07:17,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:17,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408581575bc060ce15dc51 HTTP/1.1" 502 150 2025-12-17 22:07:17,377 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408581575bc060ce15dc51'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:17,377 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408581575bc060ce15dc51 2025-12-17 22:07:17,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408581575bc060ce15dc51 HTTP/1.1" 502 150 2025-12-17 22:07:17,377 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408581575bc060ce15dc51'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:17,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940857e575bc060ce15dc4b 2025-12-17 22:07:17,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857e575bc060ce15dc4b HTTP/1.1" 502 150 2025-12-17 22:07:17,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940857e575bc060ce15dc4b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:18,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408580575bc060ce15dc4e 2025-12-17 22:07:18,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408580575bc060ce15dc4e HTTP/1.1" 502 150 2025-12-17 22:07:18,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408580575bc060ce15dc4e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:18,991 - lava_callback - INFO - Received callback for node 6940868c575bc060ce15e098 INFO: 172.21.0.1:60382 - "POST /node/6940868c575bc060ce15e098 HTTP/1.0" 202 Accepted 2025-12-17 22:07:19,378 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408581575bc060ce15dc51 2025-12-17 22:07:19,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408581575bc060ce15dc51 HTTP/1.1" 502 150 2025-12-17 22:07:19,379 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408581575bc060ce15dc51'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:19,765 - lava_callback - INFO - Received callback for node 6940868d575bc060ce15e09b INFO: 172.21.0.1:45666 - "POST /node/6940868d575bc060ce15e09b HTTP/1.0" 202 Accepted 2025-12-17 22:07:19,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853a575bc060ce15dad4 HTTP/1.1" 502 150 2025-12-17 22:07:19,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853a575bc060ce15dad4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:20,112 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408574575bc060ce15dbcf 2025-12-17 22:07:20,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408574575bc060ce15dbcf HTTP/1.1" 502 150 2025-12-17 22:07:20,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408574575bc060ce15dbcf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:20,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408572575bc060ce15dbca HTTP/1.1" 502 150 2025-12-17 22:07:20,240 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408572575bc060ce15dbca'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:20,530 - lava_callback - INFO - Received callback for node 6940868f575bc060ce15e09e INFO: 172.21.0.1:45680 - "POST /node/6940868f575bc060ce15e09e HTTP/1.0" 202 Accepted 2025-12-17 22:07:20,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940857a575bc060ce15dc0d 2025-12-17 22:07:20,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857a575bc060ce15dc0d HTTP/1.1" 502 150 2025-12-17 22:07:20,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940857a575bc060ce15dc0d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:20,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940912f575bc060ce1607b2 2025-12-17 22:07:20,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912f575bc060ce1607b2 HTTP/1.1" 502 150 2025-12-17 22:07:20,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940912f575bc060ce1607b2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:21,267 - lava_callback - INFO - Received callback for node 69408690575bc060ce15e0a1 INFO: 172.21.0.1:45686 - "POST /node/69408690575bc060ce15e0a1 HTTP/1.0" 202 Accepted 2025-12-17 22:07:21,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940857e575bc060ce15dc4b 2025-12-17 22:07:21,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857e575bc060ce15dc4b HTTP/1.1" 502 150 2025-12-17 22:07:21,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940857e575bc060ce15dc4b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:21,806 - lava_callback - INFO - Received callback for node 694085a0575bc060ce15dcc7 INFO: 172.21.0.1:45696 - "POST /node/694085a0575bc060ce15dcc7 HTTP/1.0" 202 Accepted 2025-12-17 22:07:22,020 - lava_callback - INFO - Received callback for node 69408691575bc060ce15e0a4 INFO: 172.21.0.1:45712 - "POST /node/69408691575bc060ce15e0a4 HTTP/1.0" 202 Accepted 2025-12-17 22:07:22,788 - lava_callback - INFO - Received callback for node 69408696575bc060ce15e0a7 INFO: 172.21.0.1:45718 - "POST /node/69408696575bc060ce15e0a7 HTTP/1.0" 202 Accepted 2025-12-17 22:07:22,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408580575bc060ce15dc4e 2025-12-17 22:07:22,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408580575bc060ce15dc4e HTTP/1.1" 502 150 2025-12-17 22:07:22,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408580575bc060ce15dc4e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:22,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940857c575bc060ce15dc3b 2025-12-17 22:07:22,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857c575bc060ce15dc3b HTTP/1.1" 502 150 2025-12-17 22:07:22,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940857c575bc060ce15dc3b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:23,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408581575bc060ce15dc51 2025-12-17 22:07:23,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408581575bc060ce15dc51 HTTP/1.1" 502 150 2025-12-17 22:07:23,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408581575bc060ce15dc51'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:23,563 - lava_callback - INFO - Received callback for node 69408698575bc060ce15e0aa INFO: 172.21.0.1:45732 - "POST /node/69408698575bc060ce15e0aa HTTP/1.0" 202 Accepted 2025-12-17 22:07:23,701 - lava_callback - INFO - Received callback for node 69408687575bc060ce15e090 INFO: 172.21.0.1:45740 - "POST /node/69408687575bc060ce15e090 HTTP/1.0" 202 Accepted 2025-12-17 22:07:23,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408570575bc060ce15dbc0 2025-12-17 22:07:23,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408570575bc060ce15dbc0 HTTP/1.1" 502 150 2025-12-17 22:07:23,924 - lava_callback - ERROR - Error processing callback for node 69408570575bc060ce15dbc0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408570575bc060ce15dbc0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408570575bc060ce15dbc0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408570575bc060ce15dbc0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:23,926 - lava_callback - INFO - Starting async job submit for node 69408583575bc060ce15dc5a 2025-12-17 22:07:23,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:23,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408583575bc060ce15dc5a HTTP/1.1" 502 150 2025-12-17 22:07:23,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408583575bc060ce15dc5a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:23,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408583575bc060ce15dc5a 2025-12-17 22:07:23,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408583575bc060ce15dc5a HTTP/1.1" 502 150 2025-12-17 22:07:23,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408583575bc060ce15dc5a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:24,240 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408572575bc060ce15dbca 2025-12-17 22:07:24,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408572575bc060ce15dbca HTTP/1.1" 502 150 2025-12-17 22:07:24,242 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408572575bc060ce15dbca'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:24,351 - lava_callback - INFO - Received callback for node 69408689575bc060ce15e095 INFO: 172.21.0.1:45742 - "POST /node/69408689575bc060ce15e095 HTTP/1.0" 202 Accepted 2025-12-17 22:07:24,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408572575bc060ce15dbc7 2025-12-17 22:07:24,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408572575bc060ce15dbc7 HTTP/1.1" 502 150 2025-12-17 22:07:24,423 - lava_callback - ERROR - Error processing callback for node 69408572575bc060ce15dbc7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408572575bc060ce15dbc7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408572575bc060ce15dbc7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408572575bc060ce15dbc7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:24,425 - lava_callback - INFO - Starting async job submit for node 69408584575bc060ce15dc63 2025-12-17 22:07:24,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:24,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408575575bc060ce15dbd2 2025-12-17 22:07:24,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408575575bc060ce15dbd2 HTTP/1.1" 502 150 2025-12-17 22:07:24,427 - lava_callback - ERROR - Error processing callback for node 69408575575bc060ce15dbd2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408575575bc060ce15dbd2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408575575bc060ce15dbd2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408575575bc060ce15dbd2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:24,429 - lava_callback - INFO - Starting async job submit for node 6942e2f2e43d2b76e9023925 2025-12-17 22:07:24,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:24,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408584575bc060ce15dc63 HTTP/1.1" 502 150 2025-12-17 22:07:24,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408584575bc060ce15dc63'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:24,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408584575bc060ce15dc63 2025-12-17 22:07:24,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f2e43d2b76e9023925 HTTP/1.1" 502 150 2025-12-17 22:07:24,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e2f2e43d2b76e9023925'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:24,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e2f2e43d2b76e9023925 2025-12-17 22:07:24,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408584575bc060ce15dc63 HTTP/1.1" 502 150 2025-12-17 22:07:24,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408584575bc060ce15dc63'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:24,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f2e43d2b76e9023925 HTTP/1.1" 502 150 2025-12-17 22:07:24,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e2f2e43d2b76e9023925'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:24,998 - lava_callback - INFO - Received callback for node 69408698575bc060ce15e0ad INFO: 172.21.0.1:45746 - "POST /node/69408698575bc060ce15e0ad HTTP/1.0" 202 Accepted 2025-12-17 22:07:25,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408583575bc060ce15dc5a 2025-12-17 22:07:25,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408583575bc060ce15dc5a HTTP/1.1" 502 150 2025-12-17 22:07:25,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408583575bc060ce15dc5a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:25,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4a575bc060ce1600f1 2025-12-17 22:07:25,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4a575bc060ce1600f1 HTTP/1.1" 502 150 2025-12-17 22:07:25,974 - lava_callback - ERROR - Error processing callback for node 69408f4a575bc060ce1600f1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f4a575bc060ce1600f1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f4a575bc060ce1600f1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f4a575bc060ce1600f1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:25,976 - lava_callback - INFO - Starting async job submit for node 69408585575bc060ce15dc6a 2025-12-17 22:07:25,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:25,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408585575bc060ce15dc6a HTTP/1.1" 502 150 2025-12-17 22:07:25,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408585575bc060ce15dc6a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:25,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408585575bc060ce15dc6a 2025-12-17 22:07:25,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408585575bc060ce15dc6a HTTP/1.1" 502 150 2025-12-17 22:07:25,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408585575bc060ce15dc6a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:26,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408584575bc060ce15dc63 2025-12-17 22:07:26,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e2f2e43d2b76e9023925 2025-12-17 22:07:26,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408584575bc060ce15dc63 HTTP/1.1" 502 150 2025-12-17 22:07:26,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408584575bc060ce15dc63'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:26,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f2e43d2b76e9023925 HTTP/1.1" 502 150 2025-12-17 22:07:26,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e2f2e43d2b76e9023925'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45758 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:07:27,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408585575bc060ce15dc6a 2025-12-17 22:07:27,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853a575bc060ce15dad4 2025-12-17 22:07:27,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408585575bc060ce15dc6a HTTP/1.1" 502 150 2025-12-17 22:07:27,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408585575bc060ce15dc6a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:27,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853a575bc060ce15dad4 HTTP/1.1" 502 150 2025-12-17 22:07:27,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853a575bc060ce15dad4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:28,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940912f575bc060ce1607b2 2025-12-17 22:07:28,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912f575bc060ce1607b2 HTTP/1.1" 502 150 2025-12-17 22:07:28,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940912f575bc060ce1607b2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:28,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408565575bc060ce15db9e 2025-12-17 22:07:28,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408565575bc060ce15db9e HTTP/1.1" 502 150 2025-12-17 22:07:28,883 - lava_callback - ERROR - Error processing callback for node 69408565575bc060ce15db9e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408565575bc060ce15db9e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408565575bc060ce15db9e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408565575bc060ce15db9e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:28,884 - lava_callback - INFO - Starting async job submit for node 69408588575bc060ce15dc72 2025-12-17 22:07:28,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:28,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408588575bc060ce15dc72 HTTP/1.1" 502 150 2025-12-17 22:07:28,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408588575bc060ce15dc72'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:28,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408588575bc060ce15dc72 2025-12-17 22:07:28,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408588575bc060ce15dc72 HTTP/1.1" 502 150 2025-12-17 22:07:28,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408588575bc060ce15dc72'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:29,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940857e575bc060ce15dc4b 2025-12-17 22:07:29,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857e575bc060ce15dc4b HTTP/1.1" 502 150 2025-12-17 22:07:29,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940857e575bc060ce15dc4b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:29,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408583575bc060ce15dc5a 2025-12-17 22:07:29,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408583575bc060ce15dc5a HTTP/1.1" 502 150 2025-12-17 22:07:29,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408583575bc060ce15dc5a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:30,393 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408577575bc060ce15dbd7 2025-12-17 22:07:30,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408584575bc060ce15dc63 2025-12-17 22:07:30,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e2f2e43d2b76e9023925 2025-12-17 22:07:30,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408584575bc060ce15dc63 HTTP/1.1" 502 150 2025-12-17 22:07:30,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408584575bc060ce15dc63'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:30,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f2e43d2b76e9023925 HTTP/1.1" 502 150 2025-12-17 22:07:30,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e2f2e43d2b76e9023925'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:30,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408567575bc060ce15dba1 HTTP/1.1" 502 150 2025-12-17 22:07:30,736 - lava_callback - ERROR - Error processing callback for node 69408567575bc060ce15dba1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408567575bc060ce15dba1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408567575bc060ce15dba1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408567575bc060ce15dba1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:30,738 - lava_callback - INFO - Starting async job submit for node 69408f13575bc060ce160064 2025-12-17 22:07:30,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:30,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f13575bc060ce160064 HTTP/1.1" 502 150 2025-12-17 22:07:30,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f13575bc060ce160064'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:30,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f13575bc060ce160064 2025-12-17 22:07:30,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f13575bc060ce160064 HTTP/1.1" 502 150 2025-12-17 22:07:30,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f13575bc060ce160064'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:30,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408580575bc060ce15dc4e 2025-12-17 22:07:30,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408580575bc060ce15dc4e HTTP/1.1" 502 150 2025-12-17 22:07:30,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408580575bc060ce15dc4e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:30,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408588575bc060ce15dc72 2025-12-17 22:07:30,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408588575bc060ce15dc72 HTTP/1.1" 502 150 2025-12-17 22:07:30,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408588575bc060ce15dc72'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:31,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408581575bc060ce15dc51 2025-12-17 22:07:31,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408581575bc060ce15dc51 HTTP/1.1" 502 150 2025-12-17 22:07:31,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408581575bc060ce15dc51'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:31,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408585575bc060ce15dc6a 2025-12-17 22:07:31,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408585575bc060ce15dc6a HTTP/1.1" 502 150 2025-12-17 22:07:31,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408585575bc060ce15dc6a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:32,242 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408572575bc060ce15dbca 2025-12-17 22:07:32,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408572575bc060ce15dbca HTTP/1.1" 502 150 2025-12-17 22:07:32,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408572575bc060ce15dbca'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:32,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f13575bc060ce160064 2025-12-17 22:07:32,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f13575bc060ce160064 HTTP/1.1" 502 150 2025-12-17 22:07:32,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f13575bc060ce160064'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:34,056 - lava_callback - INFO - Received callback for node 6940845b575bc060ce15d73d INFO: 172.21.0.1:54214 - "POST /node/6940845b575bc060ce15d73d HTTP/1.0" 202 Accepted 2025-12-17 22:07:34,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408588575bc060ce15dc72 2025-12-17 22:07:34,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408588575bc060ce15dc72 HTTP/1.1" 502 150 2025-12-17 22:07:34,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408588575bc060ce15dc72'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:36,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408574575bc060ce15dbcf 2025-12-17 22:07:36,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408574575bc060ce15dbcf HTTP/1.1" 502 150 2025-12-17 22:07:36,115 - lava_callback - ERROR - Error processing callback for node 69408574575bc060ce15dbcf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408574575bc060ce15dbcf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408574575bc060ce15dbcf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408574575bc060ce15dbcf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:36,117 - lava_callback - INFO - Starting async job submit for node 69408589575bc060ce15dc75 2025-12-17 22:07:36,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:36,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408589575bc060ce15dc75 HTTP/1.1" 502 150 2025-12-17 22:07:36,124 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408589575bc060ce15dc75'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:36,124 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408589575bc060ce15dc75 2025-12-17 22:07:36,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408589575bc060ce15dc75 HTTP/1.1" 502 150 2025-12-17 22:07:36,125 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408589575bc060ce15dc75'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54230 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:07:36,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940857a575bc060ce15dc0d 2025-12-17 22:07:36,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857a575bc060ce15dc0d HTTP/1.1" 502 150 2025-12-17 22:07:36,722 - lava_callback - ERROR - Error processing callback for node 6940857a575bc060ce15dc0d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940857a575bc060ce15dc0d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940857a575bc060ce15dc0d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940857a575bc060ce15dc0d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:36,724 - lava_callback - INFO - Starting async job submit for node 6940858b575bc060ce15dc78 2025-12-17 22:07:36,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:36,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858b575bc060ce15dc78 HTTP/1.1" 502 150 2025-12-17 22:07:36,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858b575bc060ce15dc78'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:36,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858b575bc060ce15dc78 2025-12-17 22:07:36,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858b575bc060ce15dc78 HTTP/1.1" 502 150 2025-12-17 22:07:36,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858b575bc060ce15dc78'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:36,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f13575bc060ce160064 2025-12-17 22:07:36,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f13575bc060ce160064 HTTP/1.1" 502 150 2025-12-17 22:07:36,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f13575bc060ce160064'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:37,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408583575bc060ce15dc5a 2025-12-17 22:07:37,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408583575bc060ce15dc5a HTTP/1.1" 502 150 2025-12-17 22:07:37,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408583575bc060ce15dc5a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:38,125 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408589575bc060ce15dc75 2025-12-17 22:07:38,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408589575bc060ce15dc75 HTTP/1.1" 502 150 2025-12-17 22:07:38,126 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408589575bc060ce15dc75'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:38,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408584575bc060ce15dc63 2025-12-17 22:07:38,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e2f2e43d2b76e9023925 2025-12-17 22:07:38,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408584575bc060ce15dc63 HTTP/1.1" 502 150 2025-12-17 22:07:38,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408584575bc060ce15dc63'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:38,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f2e43d2b76e9023925 HTTP/1.1" 502 150 2025-12-17 22:07:38,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e2f2e43d2b76e9023925'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:38,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858b575bc060ce15dc78 2025-12-17 22:07:38,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858b575bc060ce15dc78 HTTP/1.1" 502 150 2025-12-17 22:07:38,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858b575bc060ce15dc78'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:38,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940857c575bc060ce15dc3b 2025-12-17 22:07:38,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857c575bc060ce15dc3b HTTP/1.1" 502 150 2025-12-17 22:07:38,955 - lava_callback - ERROR - Error processing callback for node 6940857c575bc060ce15dc3b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940857c575bc060ce15dc3b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940857c575bc060ce15dc3b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940857c575bc060ce15dc3b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:38,957 - lava_callback - INFO - Starting async job submit for node 6940853c575bc060ce15db10 2025-12-17 22:07:38,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:39,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853c575bc060ce15db10 HTTP/1.1" 502 150 2025-12-17 22:07:39,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853c575bc060ce15db10'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:39,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853c575bc060ce15db10 2025-12-17 22:07:39,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853c575bc060ce15db10 HTTP/1.1" 502 150 2025-12-17 22:07:39,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853c575bc060ce15db10'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:39,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408585575bc060ce15dc6a 2025-12-17 22:07:39,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408585575bc060ce15dc6a HTTP/1.1" 502 150 2025-12-17 22:07:39,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408585575bc060ce15dc6a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:41,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853c575bc060ce15db10 2025-12-17 22:07:42,126 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408589575bc060ce15dc75 2025-12-17 22:07:42,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408589575bc060ce15dc75 HTTP/1.1" 502 150 2025-12-17 22:07:42,128 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408589575bc060ce15dc75'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:42,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858b575bc060ce15dc78 2025-12-17 22:07:42,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858b575bc060ce15dc78 HTTP/1.1" 502 150 2025-12-17 22:07:42,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858b575bc060ce15dc78'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:42,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408588575bc060ce15dc72 2025-12-17 22:07:42,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408588575bc060ce15dc72 HTTP/1.1" 502 150 2025-12-17 22:07:42,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408588575bc060ce15dc72'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:43,737 - lava_callback - INFO - Received callback for node 6940869b575bc060ce15e0b2 INFO: 172.21.0.1:60458 - "POST /node/6940869b575bc060ce15e0b2 HTTP/1.0" 202 Accepted 2025-12-17 22:07:43,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853a575bc060ce15dad4 2025-12-17 22:07:43,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853a575bc060ce15dad4 HTTP/1.1" 502 150 2025-12-17 22:07:43,993 - lava_callback - ERROR - Error processing callback for node 6940853a575bc060ce15dad4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940853a575bc060ce15dad4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940853a575bc060ce15dad4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940853a575bc060ce15dad4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:43,995 - lava_callback - INFO - Starting async job submit for node 6940858c575bc060ce15dc7f 2025-12-17 22:07:43,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:44,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858c575bc060ce15dc7f HTTP/1.1" 502 150 2025-12-17 22:07:44,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858c575bc060ce15dc7f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:44,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858c575bc060ce15dc7f 2025-12-17 22:07:44,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858c575bc060ce15dc7f HTTP/1.1" 502 150 2025-12-17 22:07:44,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858c575bc060ce15dc7f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:44,392 - lava_callback - INFO - Received callback for node 6940869d575bc060ce15e0bb INFO: 172.21.0.1:60474 - "POST /node/6940869d575bc060ce15e0bb HTTP/1.0" 202 Accepted 2025-12-17 22:07:44,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f13575bc060ce160064 2025-12-17 22:07:44,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f13575bc060ce160064 HTTP/1.1" 502 150 2025-12-17 22:07:44,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f13575bc060ce160064'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:44,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940912f575bc060ce1607b2 2025-12-17 22:07:44,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912f575bc060ce1607b2 HTTP/1.1" 502 150 2025-12-17 22:07:44,881 - lava_callback - ERROR - Error processing callback for node 6940912f575bc060ce1607b2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940912f575bc060ce1607b2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940912f575bc060ce1607b2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940912f575bc060ce1607b2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:44,883 - lava_callback - INFO - Starting async job submit for node 6940858d575bc060ce15dc82 2025-12-17 22:07:44,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:44,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858d575bc060ce15dc82 HTTP/1.1" 502 150 2025-12-17 22:07:44,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858d575bc060ce15dc82'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:44,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858d575bc060ce15dc82 2025-12-17 22:07:44,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858d575bc060ce15dc82 HTTP/1.1" 502 150 2025-12-17 22:07:44,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858d575bc060ce15dc82'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:45,048 - lava_callback - INFO - Received callback for node 6940869e575bc060ce15e0be INFO: 172.21.0.1:60484 - "POST /node/6940869e575bc060ce15e0be HTTP/1.0" 202 Accepted 2025-12-17 22:07:45,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940857e575bc060ce15dc4b 2025-12-17 22:07:45,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940857e575bc060ce15dc4b HTTP/1.1" 502 150 2025-12-17 22:07:45,448 - lava_callback - ERROR - Error processing callback for node 6940857e575bc060ce15dc4b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940857e575bc060ce15dc4b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940857e575bc060ce15dc4b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940857e575bc060ce15dc4b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:45,450 - lava_callback - INFO - Starting async job submit for node 6940858f575bc060ce15dc8a 2025-12-17 22:07:45,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:45,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858f575bc060ce15dc8a HTTP/1.1" 502 150 2025-12-17 22:07:45,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858f575bc060ce15dc8a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:45,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858f575bc060ce15dc8a 2025-12-17 22:07:45,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858f575bc060ce15dc8a HTTP/1.1" 502 150 2025-12-17 22:07:45,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858f575bc060ce15dc8a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:45,684 - lava_callback - INFO - Received callback for node 6940869f575bc060ce15e0c1 INFO: 172.21.0.1:60492 - "POST /node/6940869f575bc060ce15e0c1 HTTP/1.0" 202 Accepted 2025-12-17 22:07:45,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408577575bc060ce15dbd7 HTTP/1.1" 502 150 2025-12-17 22:07:45,840 - lava_callback - ERROR - Error processing callback for node 69408577575bc060ce15dbd7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408577575bc060ce15dbd7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408577575bc060ce15dbd7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408577575bc060ce15dbd7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:45,842 - lava_callback - INFO - Starting async job submit for node 69408590575bc060ce15dc8d 2025-12-17 22:07:45,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:45,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408590575bc060ce15dc8d HTTP/1.1" 502 150 2025-12-17 22:07:45,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408590575bc060ce15dc8d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:45,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408590575bc060ce15dc8d 2025-12-17 22:07:45,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408590575bc060ce15dc8d HTTP/1.1" 502 150 2025-12-17 22:07:45,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408590575bc060ce15dc8d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:46,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858c575bc060ce15dc7f 2025-12-17 22:07:46,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858c575bc060ce15dc7f HTTP/1.1" 502 150 2025-12-17 22:07:46,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858c575bc060ce15dc7f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:46,245 - lava_callback - INFO - Received callback for node 6940c48b1e3c24e95681d4ea INFO: 172.21.0.1:60500 - "POST /node/6940c48b1e3c24e95681d4ea HTTP/1.0" 202 Accepted 2025-12-17 22:07:46,334 - lava_callback - INFO - Received callback for node 694086a1575bc060ce15e0c8 INFO: 172.21.0.1:60514 - "POST /node/694086a1575bc060ce15e0c8 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:60518 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:07:46,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408580575bc060ce15dc4e 2025-12-17 22:07:46,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408580575bc060ce15dc4e HTTP/1.1" 502 150 2025-12-17 22:07:46,878 - lava_callback - ERROR - Error processing callback for node 69408580575bc060ce15dc4e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408580575bc060ce15dc4e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408580575bc060ce15dc4e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408580575bc060ce15dc4e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:46,880 - lava_callback - INFO - Starting async job submit for node 69408591575bc060ce15dc94 2025-12-17 22:07:46,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:46,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408591575bc060ce15dc94 HTTP/1.1" 502 150 2025-12-17 22:07:46,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408591575bc060ce15dc94'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:46,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408591575bc060ce15dc94 2025-12-17 22:07:46,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408591575bc060ce15dc94 HTTP/1.1" 502 150 2025-12-17 22:07:46,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408591575bc060ce15dc94'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:46,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858d575bc060ce15dc82 2025-12-17 22:07:46,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858d575bc060ce15dc82 HTTP/1.1" 502 150 2025-12-17 22:07:46,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858d575bc060ce15dc82'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:46,985 - lava_callback - INFO - Received callback for node 694086a2575bc060ce15e0cb INFO: 172.21.0.1:60520 - "POST /node/694086a2575bc060ce15e0cb HTTP/1.0" 202 Accepted 2025-12-17 22:07:47,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408581575bc060ce15dc51 2025-12-17 22:07:47,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408581575bc060ce15dc51 HTTP/1.1" 502 150 2025-12-17 22:07:47,383 - lava_callback - ERROR - Error processing callback for node 69408581575bc060ce15dc51: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408581575bc060ce15dc51 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408581575bc060ce15dc51 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408581575bc060ce15dc51 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:47,385 - lava_callback - INFO - Starting async job submit for node 69408593575bc060ce15dc97 2025-12-17 22:07:47,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:47,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408593575bc060ce15dc97 HTTP/1.1" 502 150 2025-12-17 22:07:47,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408593575bc060ce15dc97'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:47,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408593575bc060ce15dc97 2025-12-17 22:07:47,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408593575bc060ce15dc97 HTTP/1.1" 502 150 2025-12-17 22:07:47,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408593575bc060ce15dc97'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:47,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858f575bc060ce15dc8a 2025-12-17 22:07:47,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858f575bc060ce15dc8a HTTP/1.1" 502 150 2025-12-17 22:07:47,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858f575bc060ce15dc8a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:47,635 - lava_callback - INFO - Received callback for node 694086a3575bc060ce15e0ce INFO: 172.21.0.1:60532 - "POST /node/694086a3575bc060ce15e0ce HTTP/1.0" 202 Accepted 2025-12-17 22:07:47,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408590575bc060ce15dc8d 2025-12-17 22:07:47,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408590575bc060ce15dc8d HTTP/1.1" 502 150 2025-12-17 22:07:47,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408590575bc060ce15dc8d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:48,243 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408572575bc060ce15dbca 2025-12-17 22:07:48,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408572575bc060ce15dbca HTTP/1.1" 502 150 2025-12-17 22:07:48,245 - lava_callback - ERROR - Error processing callback for node 69408572575bc060ce15dbca: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408572575bc060ce15dbca (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408572575bc060ce15dbca (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408572575bc060ce15dbca (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:48,247 - lava_callback - INFO - Starting async job submit for node 69408594575bc060ce15dc9a 2025-12-17 22:07:48,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:48,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408594575bc060ce15dc9a HTTP/1.1" 502 150 2025-12-17 22:07:48,259 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408594575bc060ce15dc9a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:48,259 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408594575bc060ce15dc9a 2025-12-17 22:07:48,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408594575bc060ce15dc9a HTTP/1.1" 502 150 2025-12-17 22:07:48,260 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408594575bc060ce15dc9a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:48,283 - lava_callback - INFO - Received callback for node 694086a4575bc060ce15e0d1 INFO: 172.21.0.1:60536 - "POST /node/694086a4575bc060ce15e0d1 HTTP/1.0" 202 Accepted 2025-12-17 22:07:48,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408591575bc060ce15dc94 2025-12-17 22:07:48,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408591575bc060ce15dc94 HTTP/1.1" 502 150 2025-12-17 22:07:48,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408591575bc060ce15dc94'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:48,920 - lava_callback - INFO - Received callback for node 694086a6575bc060ce15e0d8 INFO: 172.21.0.1:60540 - "POST /node/694086a6575bc060ce15e0d8 HTTP/1.0" 202 Accepted 2025-12-17 22:07:49,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408593575bc060ce15dc97 2025-12-17 22:07:49,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408593575bc060ce15dc97 HTTP/1.1" 502 150 2025-12-17 22:07:49,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408593575bc060ce15dc97'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:49,579 - lava_callback - INFO - Received callback for node 694086a7575bc060ce15e0db INFO: 172.21.0.1:60548 - "POST /node/694086a7575bc060ce15e0db HTTP/1.0" 202 Accepted 2025-12-17 22:07:50,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858c575bc060ce15dc7f 2025-12-17 22:07:50,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858c575bc060ce15dc7f HTTP/1.1" 502 150 2025-12-17 22:07:50,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858c575bc060ce15dc7f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:50,128 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408589575bc060ce15dc75 2025-12-17 22:07:50,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408589575bc060ce15dc75 HTTP/1.1" 502 150 2025-12-17 22:07:50,129 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408589575bc060ce15dc75'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:50,230 - lava_callback - INFO - Received callback for node 694086a8575bc060ce15e0de INFO: 172.21.0.1:58376 - "POST /node/694086a8575bc060ce15e0de HTTP/1.0" 202 Accepted 2025-12-17 22:07:50,260 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408594575bc060ce15dc9a 2025-12-17 22:07:50,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408594575bc060ce15dc9a HTTP/1.1" 502 150 2025-12-17 22:07:50,261 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408594575bc060ce15dc9a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:50,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858b575bc060ce15dc78 2025-12-17 22:07:50,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858b575bc060ce15dc78 HTTP/1.1" 502 150 2025-12-17 22:07:50,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858b575bc060ce15dc78'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:50,886 - lava_callback - INFO - Received callback for node 694086a9575bc060ce15e0e1 INFO: 172.21.0.1:58386 - "POST /node/694086a9575bc060ce15e0e1 HTTP/1.0" 202 Accepted 2025-12-17 22:07:50,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858d575bc060ce15dc82 2025-12-17 22:07:50,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858d575bc060ce15dc82 HTTP/1.1" 502 150 2025-12-17 22:07:50,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858d575bc060ce15dc82'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:51,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858f575bc060ce15dc8a 2025-12-17 22:07:51,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858f575bc060ce15dc8a HTTP/1.1" 502 150 2025-12-17 22:07:51,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858f575bc060ce15dc8a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:51,525 - lava_callback - INFO - Received callback for node 694086ab575bc060ce15e0e4 INFO: 172.21.0.1:58398 - "POST /node/694086ab575bc060ce15e0e4 HTTP/1.0" 202 Accepted 2025-12-17 22:07:51,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408590575bc060ce15dc8d 2025-12-17 22:07:51,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408590575bc060ce15dc8d HTTP/1.1" 502 150 2025-12-17 22:07:51,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408590575bc060ce15dc8d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:52,193 - lava_callback - INFO - Received callback for node 694086ac575bc060ce15e0e7 INFO: 172.21.0.1:58414 - "POST /node/694086ac575bc060ce15e0e7 HTTP/1.0" 202 Accepted 2025-12-17 22:07:52,854 - lava_callback - INFO - Received callback for node 694086ad575bc060ce15e0ea INFO: 172.21.0.1:58422 - "POST /node/694086ad575bc060ce15e0ea HTTP/1.0" 202 Accepted 2025-12-17 22:07:52,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408591575bc060ce15dc94 2025-12-17 22:07:52,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408591575bc060ce15dc94 HTTP/1.1" 502 150 2025-12-17 22:07:52,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408591575bc060ce15dc94'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:53,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408593575bc060ce15dc97 2025-12-17 22:07:53,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408593575bc060ce15dc97 HTTP/1.1" 502 150 2025-12-17 22:07:53,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408593575bc060ce15dc97'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:53,493 - lava_callback - INFO - Received callback for node 694086ae575bc060ce15e0ed INFO: 172.21.0.1:58430 - "POST /node/694086ae575bc060ce15e0ed HTTP/1.0" 202 Accepted 2025-12-17 22:07:53,824 - lava_callback - INFO - Received callback for node 69408481575bc060ce15d7de INFO: 172.21.0.1:58436 - "POST /node/69408481575bc060ce15d7de HTTP/1.0" 202 Accepted 2025-12-17 22:07:53,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408583575bc060ce15dc5a 2025-12-17 22:07:53,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408583575bc060ce15dc5a HTTP/1.1" 502 150 2025-12-17 22:07:53,963 - lava_callback - ERROR - Error processing callback for node 69408583575bc060ce15dc5a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408583575bc060ce15dc5a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408583575bc060ce15dc5a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408583575bc060ce15dc5a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:53,965 - lava_callback - INFO - Starting async job submit for node 69408595575bc060ce15dc9d 2025-12-17 22:07:53,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:54,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408595575bc060ce15dc9d HTTP/1.1" 502 150 2025-12-17 22:07:54,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408595575bc060ce15dc9d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:54,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408595575bc060ce15dc9d 2025-12-17 22:07:54,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408595575bc060ce15dc9d HTTP/1.1" 502 150 2025-12-17 22:07:54,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408595575bc060ce15dc9d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:54,262 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408594575bc060ce15dc9a 2025-12-17 22:07:54,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408594575bc060ce15dc9a HTTP/1.1" 502 150 2025-12-17 22:07:54,263 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408594575bc060ce15dc9a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:54,447 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408584575bc060ce15dc63 2025-12-17 22:07:54,447 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e2f2e43d2b76e9023925 2025-12-17 22:07:54,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408584575bc060ce15dc63 HTTP/1.1" 502 150 2025-12-17 22:07:54,448 - lava_callback - ERROR - Error processing callback for node 69408584575bc060ce15dc63: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408584575bc060ce15dc63 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408584575bc060ce15dc63 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408584575bc060ce15dc63 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:54,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f2e43d2b76e9023925 HTTP/1.1" 502 150 2025-12-17 22:07:54,450 - lava_callback - ERROR - Error processing callback for node 6942e2f2e43d2b76e9023925: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e2f2e43d2b76e9023925 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e2f2e43d2b76e9023925 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e2f2e43d2b76e9023925 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:54,450 - lava_callback - INFO - Starting async job submit for node 69408596575bc060ce15dca0 2025-12-17 22:07:54,452 - lava_callback - INFO - Starting async job submit for node 69408598575bc060ce15dca7 2025-12-17 22:07:54,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:54,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:54,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408596575bc060ce15dca0 HTTP/1.1" 502 150 2025-12-17 22:07:54,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408596575bc060ce15dca0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:54,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408596575bc060ce15dca0 2025-12-17 22:07:54,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408596575bc060ce15dca0 HTTP/1.1" 502 150 2025-12-17 22:07:54,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408596575bc060ce15dca0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:55,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408585575bc060ce15dc6a 2025-12-17 22:07:55,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408585575bc060ce15dc6a HTTP/1.1" 502 150 2025-12-17 22:07:55,990 - lava_callback - ERROR - Error processing callback for node 69408585575bc060ce15dc6a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408585575bc060ce15dc6a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408585575bc060ce15dc6a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408585575bc060ce15dc6a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:55,992 - lava_callback - INFO - Starting async job submit for node 6940859a575bc060ce15dcae 2025-12-17 22:07:55,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:55,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859a575bc060ce15dcae HTTP/1.1" 502 150 2025-12-17 22:07:55,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859a575bc060ce15dcae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:55,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859a575bc060ce15dcae 2025-12-17 22:07:56,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408595575bc060ce15dc9d 2025-12-17 22:07:56,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408595575bc060ce15dc9d HTTP/1.1" 502 150 2025-12-17 22:07:56,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408595575bc060ce15dc9d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:56,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853c575bc060ce15db10 HTTP/1.1" 502 150 2025-12-17 22:07:56,336 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853c575bc060ce15db10'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:56,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408596575bc060ce15dca0 2025-12-17 22:07:56,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408596575bc060ce15dca0 HTTP/1.1" 502 150 2025-12-17 22:07:56,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408596575bc060ce15dca0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58450 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:07:58,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858c575bc060ce15dc7f 2025-12-17 22:07:58,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858c575bc060ce15dc7f HTTP/1.1" 502 150 2025-12-17 22:07:58,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858c575bc060ce15dc7f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:58,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858d575bc060ce15dc82 2025-12-17 22:07:58,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858d575bc060ce15dc82 HTTP/1.1" 502 150 2025-12-17 22:07:58,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858d575bc060ce15dc82'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:58,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408588575bc060ce15dc72 2025-12-17 22:07:58,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408588575bc060ce15dc72 HTTP/1.1" 502 150 2025-12-17 22:07:58,900 - lava_callback - ERROR - Error processing callback for node 69408588575bc060ce15dc72: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408588575bc060ce15dc72 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408588575bc060ce15dc72 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408588575bc060ce15dc72 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:07:58,902 - lava_callback - INFO - Starting async job submit for node 6940859b575bc060ce15dcb5 2025-12-17 22:07:58,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:07:58,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859b575bc060ce15dcb5 HTTP/1.1" 502 150 2025-12-17 22:07:58,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859b575bc060ce15dcb5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:58,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859b575bc060ce15dcb5 2025-12-17 22:07:58,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859b575bc060ce15dcb5 HTTP/1.1" 502 150 2025-12-17 22:07:58,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859b575bc060ce15dcb5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:59,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858f575bc060ce15dc8a 2025-12-17 22:07:59,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858f575bc060ce15dc8a HTTP/1.1" 502 150 2025-12-17 22:07:59,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940858f575bc060ce15dc8a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:07:59,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408590575bc060ce15dc8d 2025-12-17 22:07:59,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408590575bc060ce15dc8d HTTP/1.1" 502 150 2025-12-17 22:07:59,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408590575bc060ce15dc8d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:00,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408595575bc060ce15dc9d 2025-12-17 22:08:00,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408595575bc060ce15dc9d HTTP/1.1" 502 150 2025-12-17 22:08:00,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408595575bc060ce15dc9d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:00,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853c575bc060ce15db10 2025-12-17 22:08:00,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853c575bc060ce15db10 HTTP/1.1" 502 150 2025-12-17 22:08:00,338 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853c575bc060ce15db10'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:00,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408596575bc060ce15dca0 2025-12-17 22:08:00,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408596575bc060ce15dca0 HTTP/1.1" 502 150 2025-12-17 22:08:00,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408596575bc060ce15dca0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:00,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f13575bc060ce160064 2025-12-17 22:08:00,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f13575bc060ce160064 HTTP/1.1" 502 150 2025-12-17 22:08:00,809 - lava_callback - ERROR - Error processing callback for node 69408f13575bc060ce160064: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f13575bc060ce160064 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f13575bc060ce160064 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f13575bc060ce160064 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:00,811 - lava_callback - INFO - Starting async job submit for node 6940859c575bc060ce15dcbc 2025-12-17 22:08:00,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:00,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859c575bc060ce15dcbc HTTP/1.1" 502 150 2025-12-17 22:08:00,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859c575bc060ce15dcbc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:00,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859c575bc060ce15dcbc 2025-12-17 22:08:00,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859c575bc060ce15dcbc HTTP/1.1" 502 150 2025-12-17 22:08:00,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859c575bc060ce15dcbc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:00,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408591575bc060ce15dc94 2025-12-17 22:08:00,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408591575bc060ce15dc94 HTTP/1.1" 502 150 2025-12-17 22:08:00,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408591575bc060ce15dc94'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:00,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859b575bc060ce15dcb5 2025-12-17 22:08:00,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859b575bc060ce15dcb5 HTTP/1.1" 502 150 2025-12-17 22:08:00,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859b575bc060ce15dcb5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:01,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408593575bc060ce15dc97 2025-12-17 22:08:01,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408593575bc060ce15dc97 HTTP/1.1" 502 150 2025-12-17 22:08:01,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408593575bc060ce15dc97'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57724 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:08:02,263 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408594575bc060ce15dc9a 2025-12-17 22:08:02,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408594575bc060ce15dc9a HTTP/1.1" 502 150 2025-12-17 22:08:02,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408594575bc060ce15dc9a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:02,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859c575bc060ce15dcbc 2025-12-17 22:08:02,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859c575bc060ce15dcbc HTTP/1.1" 502 150 2025-12-17 22:08:02,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859c575bc060ce15dcbc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:03,762 - lava_callback - INFO - Received callback for node 694086b1575bc060ce15e121 INFO: 172.21.0.1:57734 - "POST /node/694086b1575bc060ce15e121 HTTP/1.0" 202 Accepted 2025-12-17 22:08:04,420 - lava_callback - INFO - Received callback for node 694086b2575bc060ce15e132 INFO: 172.21.0.1:57744 - "POST /node/694086b2575bc060ce15e132 HTTP/1.0" 202 Accepted 2025-12-17 22:08:04,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859b575bc060ce15dcb5 2025-12-17 22:08:04,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859b575bc060ce15dcb5 HTTP/1.1" 502 150 2025-12-17 22:08:04,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859b575bc060ce15dcb5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:05,087 - lava_callback - INFO - Received callback for node 694086b4575bc060ce15e139 INFO: 172.21.0.1:57750 - "POST /node/694086b4575bc060ce15e139 HTTP/1.0" 202 Accepted 2025-12-17 22:08:05,734 - lava_callback - INFO - Received callback for node 694086b5575bc060ce15e140 INFO: 172.21.0.1:57764 - "POST /node/694086b5575bc060ce15e140 HTTP/1.0" 202 Accepted 2025-12-17 22:08:06,129 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408589575bc060ce15dc75 2025-12-17 22:08:06,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408589575bc060ce15dc75 HTTP/1.1" 502 150 2025-12-17 22:08:06,131 - lava_callback - ERROR - Error processing callback for node 69408589575bc060ce15dc75: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408589575bc060ce15dc75 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408589575bc060ce15dc75 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408589575bc060ce15dc75 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:06,132 - lava_callback - INFO - Starting async job submit for node 6940859d575bc060ce15dcc1 2025-12-17 22:08:06,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:06,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859d575bc060ce15dcc1 HTTP/1.1" 502 150 2025-12-17 22:08:06,140 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859d575bc060ce15dcc1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:06,140 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859d575bc060ce15dcc1 2025-12-17 22:08:06,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859d575bc060ce15dcc1 HTTP/1.1" 502 150 2025-12-17 22:08:06,148 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859d575bc060ce15dcc1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:06,372 - lava_callback - INFO - Received callback for node 694086b6575bc060ce15e147 INFO: 172.21.0.1:57772 - "POST /node/694086b6575bc060ce15e147 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:57782 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:08:06,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858b575bc060ce15dc78 2025-12-17 22:08:06,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858b575bc060ce15dc78 HTTP/1.1" 502 150 2025-12-17 22:08:06,753 - lava_callback - ERROR - Error processing callback for node 6940858b575bc060ce15dc78: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940858b575bc060ce15dc78 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940858b575bc060ce15dc78 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940858b575bc060ce15dc78 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:06,755 - lava_callback - INFO - Starting async job submit for node 6940859f575bc060ce15dcc4 2025-12-17 22:08:06,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:06,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859f575bc060ce15dcc4 HTTP/1.1" 502 150 2025-12-17 22:08:06,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859f575bc060ce15dcc4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:06,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859f575bc060ce15dcc4 2025-12-17 22:08:06,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859f575bc060ce15dcc4 HTTP/1.1" 502 150 2025-12-17 22:08:06,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859f575bc060ce15dcc4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:06,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859c575bc060ce15dcbc 2025-12-17 22:08:06,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859c575bc060ce15dcbc HTTP/1.1" 502 150 2025-12-17 22:08:06,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859c575bc060ce15dcbc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:07,011 - lava_callback - INFO - Received callback for node 694086b7575bc060ce15e14a INFO: 172.21.0.1:57786 - "POST /node/694086b7575bc060ce15e14a HTTP/1.0" 202 Accepted 2025-12-17 22:08:07,648 - lava_callback - INFO - Received callback for node 694086b9575bc060ce15e151 INFO: 172.21.0.1:57794 - "POST /node/694086b9575bc060ce15e151 HTTP/1.0" 202 Accepted 2025-12-17 22:08:08,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408595575bc060ce15dc9d 2025-12-17 22:08:08,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408595575bc060ce15dc9d HTTP/1.1" 502 150 2025-12-17 22:08:08,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408595575bc060ce15dc9d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:08,148 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859d575bc060ce15dcc1 2025-12-17 22:08:08,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859d575bc060ce15dcc1 HTTP/1.1" 502 150 2025-12-17 22:08:08,149 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859d575bc060ce15dcc1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:08,309 - lava_callback - INFO - Received callback for node 694086ba575bc060ce15e154 INFO: 172.21.0.1:57802 - "POST /node/694086ba575bc060ce15e154 HTTP/1.0" 202 Accepted 2025-12-17 22:08:08,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853c575bc060ce15db10 2025-12-17 22:08:08,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853c575bc060ce15db10 HTTP/1.1" 502 150 2025-12-17 22:08:08,339 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940853c575bc060ce15db10'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:08,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408596575bc060ce15dca0 2025-12-17 22:08:08,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408596575bc060ce15dca0 HTTP/1.1" 502 150 2025-12-17 22:08:08,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408596575bc060ce15dca0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:08,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859f575bc060ce15dcc4 2025-12-17 22:08:08,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859f575bc060ce15dcc4 HTTP/1.1" 502 150 2025-12-17 22:08:08,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859f575bc060ce15dcc4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:08,961 - lava_callback - INFO - Received callback for node 694086bb575bc060ce15e157 INFO: 172.21.0.1:57816 - "POST /node/694086bb575bc060ce15e157 HTTP/1.0" 202 Accepted 2025-12-17 22:08:09,603 - lava_callback - INFO - Received callback for node 694086bd575bc060ce15e15a INFO: 172.21.0.1:57828 - "POST /node/694086bd575bc060ce15e15a HTTP/1.0" 202 Accepted 2025-12-17 22:08:10,264 - lava_callback - INFO - Received callback for node 694086be575bc060ce15e161 INFO: 172.21.0.1:58910 - "POST /node/694086be575bc060ce15e161 HTTP/1.0" 202 Accepted 2025-12-17 22:08:10,915 - lava_callback - INFO - Received callback for node 694086bf575bc060ce15e164 INFO: 172.21.0.1:58912 - "POST /node/694086bf575bc060ce15e164 HTTP/1.0" 202 Accepted 2025-12-17 22:08:11,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859a575bc060ce15dcae HTTP/1.1" 502 150 2025-12-17 22:08:11,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859a575bc060ce15dcae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:11,546 - lava_callback - INFO - Received callback for node 694086c0575bc060ce15e167 INFO: 172.21.0.1:58924 - "POST /node/694086c0575bc060ce15e167 HTTP/1.0" 202 Accepted 2025-12-17 22:08:12,149 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859d575bc060ce15dcc1 2025-12-17 22:08:12,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859d575bc060ce15dcc1 HTTP/1.1" 502 150 2025-12-17 22:08:12,150 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859d575bc060ce15dcc1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:12,205 - lava_callback - INFO - Received callback for node 694086c1575bc060ce15e16a INFO: 172.21.0.1:58940 - "POST /node/694086c1575bc060ce15e16a HTTP/1.0" 202 Accepted 2025-12-17 22:08:12,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859f575bc060ce15dcc4 2025-12-17 22:08:12,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859f575bc060ce15dcc4 HTTP/1.1" 502 150 2025-12-17 22:08:12,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859f575bc060ce15dcc4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:12,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859b575bc060ce15dcb5 2025-12-17 22:08:12,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859b575bc060ce15dcb5 HTTP/1.1" 502 150 2025-12-17 22:08:12,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859b575bc060ce15dcb5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:13,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859a575bc060ce15dcae 2025-12-17 22:08:13,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859a575bc060ce15dcae HTTP/1.1" 502 150 2025-12-17 22:08:13,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859a575bc060ce15dcae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:14,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858c575bc060ce15dc7f 2025-12-17 22:08:14,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858c575bc060ce15dc7f HTTP/1.1" 502 150 2025-12-17 22:08:14,008 - lava_callback - ERROR - Error processing callback for node 6940858c575bc060ce15dc7f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940858c575bc060ce15dc7f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940858c575bc060ce15dc7f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940858c575bc060ce15dc7f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:14,009 - lava_callback - INFO - Starting async job submit for node 694085a1575bc060ce15dcca 2025-12-17 22:08:14,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:14,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a1575bc060ce15dcca HTTP/1.1" 502 150 2025-12-17 22:08:14,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a1575bc060ce15dcca'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:14,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a1575bc060ce15dcca 2025-12-17 22:08:14,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a1575bc060ce15dcca HTTP/1.1" 502 150 2025-12-17 22:08:14,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a1575bc060ce15dcca'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:14,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859c575bc060ce15dcbc 2025-12-17 22:08:14,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859c575bc060ce15dcbc HTTP/1.1" 502 150 2025-12-17 22:08:14,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859c575bc060ce15dcbc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:14,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858d575bc060ce15dc82 2025-12-17 22:08:14,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858d575bc060ce15dc82 HTTP/1.1" 502 150 2025-12-17 22:08:14,899 - lava_callback - ERROR - Error processing callback for node 6940858d575bc060ce15dc82: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940858d575bc060ce15dc82 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940858d575bc060ce15dc82 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940858d575bc060ce15dc82 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:14,901 - lava_callback - INFO - Starting async job submit for node 694085a3575bc060ce15dccd 2025-12-17 22:08:14,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:14,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a3575bc060ce15dccd HTTP/1.1" 502 150 2025-12-17 22:08:14,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a3575bc060ce15dccd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:14,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a3575bc060ce15dccd 2025-12-17 22:08:14,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a3575bc060ce15dccd HTTP/1.1" 502 150 2025-12-17 22:08:14,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a3575bc060ce15dccd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58946 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:08:15,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940858f575bc060ce15dc8a 2025-12-17 22:08:15,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940858f575bc060ce15dc8a HTTP/1.1" 502 150 2025-12-17 22:08:15,468 - lava_callback - ERROR - Error processing callback for node 6940858f575bc060ce15dc8a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940858f575bc060ce15dc8a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940858f575bc060ce15dc8a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940858f575bc060ce15dc8a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:15,470 - lava_callback - INFO - Starting async job submit for node 694085a4575bc060ce15dcd4 2025-12-17 22:08:15,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:15,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a4575bc060ce15dcd4 HTTP/1.1" 502 150 2025-12-17 22:08:15,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a4575bc060ce15dcd4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:15,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a4575bc060ce15dcd4 2025-12-17 22:08:15,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a4575bc060ce15dcd4 HTTP/1.1" 502 150 2025-12-17 22:08:15,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a4575bc060ce15dcd4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:15,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408590575bc060ce15dc8d 2025-12-17 22:08:15,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408590575bc060ce15dc8d HTTP/1.1" 502 150 2025-12-17 22:08:15,857 - lava_callback - ERROR - Error processing callback for node 69408590575bc060ce15dc8d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408590575bc060ce15dc8d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408590575bc060ce15dc8d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408590575bc060ce15dc8d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:15,858 - lava_callback - INFO - Starting async job submit for node 694085a4575bc060ce15dcd7 2025-12-17 22:08:15,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:15,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a4575bc060ce15dcd7 HTTP/1.1" 502 150 2025-12-17 22:08:15,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a4575bc060ce15dcd7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:15,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a4575bc060ce15dcd7 2025-12-17 22:08:15,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a4575bc060ce15dcd7 HTTP/1.1" 502 150 2025-12-17 22:08:15,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a4575bc060ce15dcd7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:16,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a1575bc060ce15dcca 2025-12-17 22:08:16,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a1575bc060ce15dcca HTTP/1.1" 502 150 2025-12-17 22:08:16,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a1575bc060ce15dcca'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58962 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:08:16,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408591575bc060ce15dc94 2025-12-17 22:08:16,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408591575bc060ce15dc94 HTTP/1.1" 502 150 2025-12-17 22:08:16,893 - lava_callback - ERROR - Error processing callback for node 69408591575bc060ce15dc94: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408591575bc060ce15dc94 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408591575bc060ce15dc94 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408591575bc060ce15dc94 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:16,895 - lava_callback - INFO - Starting async job submit for node 694085a5575bc060ce15dcda 2025-12-17 22:08:16,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:16,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a5575bc060ce15dcda HTTP/1.1" 502 150 2025-12-17 22:08:16,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a5575bc060ce15dcda'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:16,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a5575bc060ce15dcda 2025-12-17 22:08:16,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a5575bc060ce15dcda HTTP/1.1" 502 150 2025-12-17 22:08:16,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a5575bc060ce15dcda'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:16,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a3575bc060ce15dccd 2025-12-17 22:08:16,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a3575bc060ce15dccd HTTP/1.1" 502 150 2025-12-17 22:08:16,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a3575bc060ce15dccd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:17,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408593575bc060ce15dc97 2025-12-17 22:08:17,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408593575bc060ce15dc97 HTTP/1.1" 502 150 2025-12-17 22:08:17,421 - lava_callback - ERROR - Error processing callback for node 69408593575bc060ce15dc97: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408593575bc060ce15dc97 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408593575bc060ce15dc97 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408593575bc060ce15dc97 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:17,423 - lava_callback - INFO - Starting async job submit for node 694085a6575bc060ce15dcdd 2025-12-17 22:08:17,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:17,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a6575bc060ce15dcdd HTTP/1.1" 502 150 2025-12-17 22:08:17,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a6575bc060ce15dcdd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:17,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a6575bc060ce15dcdd 2025-12-17 22:08:17,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a6575bc060ce15dcdd HTTP/1.1" 502 150 2025-12-17 22:08:17,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a6575bc060ce15dcdd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:17,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859a575bc060ce15dcae 2025-12-17 22:08:17,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859a575bc060ce15dcae HTTP/1.1" 502 150 2025-12-17 22:08:17,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859a575bc060ce15dcae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:17,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a4575bc060ce15dcd4 2025-12-17 22:08:17,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a4575bc060ce15dcd4 HTTP/1.1" 502 150 2025-12-17 22:08:17,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a4575bc060ce15dcd4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:17,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a4575bc060ce15dcd7 2025-12-17 22:08:17,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a4575bc060ce15dcd7 HTTP/1.1" 502 150 2025-12-17 22:08:17,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a4575bc060ce15dcd7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:18,264 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408594575bc060ce15dc9a 2025-12-17 22:08:18,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a5575bc060ce15dcda 2025-12-17 22:08:18,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a5575bc060ce15dcda HTTP/1.1" 502 150 2025-12-17 22:08:18,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a5575bc060ce15dcda'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:18,947 - lava_callback - INFO - Received callback for node 694086ca575bc060ce15e18d INFO: 172.21.0.1:58974 - "POST /node/694086ca575bc060ce15e18d HTTP/1.0" 202 Accepted 2025-12-17 22:08:19,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a6575bc060ce15dcdd 2025-12-17 22:08:19,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a6575bc060ce15dcdd HTTP/1.1" 502 150 2025-12-17 22:08:19,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a6575bc060ce15dcdd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:19,733 - lava_callback - INFO - Received callback for node 694086cc575bc060ce15e193 INFO: 172.21.0.1:46660 - "POST /node/694086cc575bc060ce15e193 HTTP/1.0" 202 Accepted 2025-12-17 22:08:20,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a1575bc060ce15dcca 2025-12-17 22:08:20,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a1575bc060ce15dcca HTTP/1.1" 502 150 2025-12-17 22:08:20,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a1575bc060ce15dcca'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:20,151 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859d575bc060ce15dcc1 2025-12-17 22:08:20,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859d575bc060ce15dcc1 HTTP/1.1" 502 150 2025-12-17 22:08:20,152 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859d575bc060ce15dcc1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:20,488 - lava_callback - INFO - Received callback for node 694086cd575bc060ce15e19a INFO: 172.21.0.1:46666 - "POST /node/694086cd575bc060ce15e19a HTTP/1.0" 202 Accepted 2025-12-17 22:08:20,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859f575bc060ce15dcc4 2025-12-17 22:08:20,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859f575bc060ce15dcc4 HTTP/1.1" 502 150 2025-12-17 22:08:20,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859f575bc060ce15dcc4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:20,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a3575bc060ce15dccd 2025-12-17 22:08:20,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a3575bc060ce15dccd HTTP/1.1" 502 150 2025-12-17 22:08:20,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a3575bc060ce15dccd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:21,240 - lava_callback - INFO - Received callback for node 694086cf575bc060ce15e19d INFO: 172.21.0.1:46668 - "POST /node/694086cf575bc060ce15e19d HTTP/1.0" 202 Accepted 2025-12-17 22:08:21,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a4575bc060ce15dcd4 2025-12-17 22:08:21,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a4575bc060ce15dcd4 HTTP/1.1" 502 150 2025-12-17 22:08:21,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a4575bc060ce15dcd4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:21,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a4575bc060ce15dcd7 2025-12-17 22:08:21,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a4575bc060ce15dcd7 HTTP/1.1" 502 150 2025-12-17 22:08:21,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a4575bc060ce15dcd7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:22,093 - lava_callback - INFO - Received callback for node 694086d0575bc060ce15e1a0 INFO: 172.21.0.1:46680 - "POST /node/694086d0575bc060ce15e1a0 HTTP/1.0" 202 Accepted 2025-12-17 22:08:22,885 - lava_callback - INFO - Received callback for node 694086d1575bc060ce15e1a3 INFO: 172.21.0.1:46694 - "POST /node/694086d1575bc060ce15e1a3 HTTP/1.0" 202 Accepted 2025-12-17 22:08:22,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a5575bc060ce15dcda 2025-12-17 22:08:22,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a5575bc060ce15dcda HTTP/1.1" 502 150 2025-12-17 22:08:22,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a5575bc060ce15dcda'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:23,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a6575bc060ce15dcdd 2025-12-17 22:08:23,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a6575bc060ce15dcdd HTTP/1.1" 502 150 2025-12-17 22:08:23,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a6575bc060ce15dcdd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:23,647 - lava_callback - INFO - Received callback for node 694086d2575bc060ce15e1a6 INFO: 172.21.0.1:46698 - "POST /node/694086d2575bc060ce15e1a6 HTTP/1.0" 202 Accepted 2025-12-17 22:08:23,762 - lava_callback - INFO - Received callback for node 694086c5575bc060ce15e174 INFO: 172.21.0.1:46706 - "POST /node/694086c5575bc060ce15e174 HTTP/1.0" 202 Accepted 2025-12-17 22:08:24,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408595575bc060ce15dc9d 2025-12-17 22:08:24,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408595575bc060ce15dc9d HTTP/1.1" 502 150 2025-12-17 22:08:24,020 - lava_callback - ERROR - Error processing callback for node 69408595575bc060ce15dc9d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408595575bc060ce15dc9d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408595575bc060ce15dc9d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408595575bc060ce15dc9d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:24,022 - lava_callback - INFO - Starting async job submit for node 694085af575bc060ce15dcfd 2025-12-17 22:08:24,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:24,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085af575bc060ce15dcfd HTTP/1.1" 502 150 2025-12-17 22:08:24,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085af575bc060ce15dcfd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:24,028 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085af575bc060ce15dcfd 2025-12-17 22:08:24,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085af575bc060ce15dcfd HTTP/1.1" 502 150 2025-12-17 22:08:24,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085af575bc060ce15dcfd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:24,339 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940853c575bc060ce15db10 2025-12-17 22:08:24,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940853c575bc060ce15db10 HTTP/1.1" 502 150 2025-12-17 22:08:24,341 - lava_callback - ERROR - Error processing callback for node 6940853c575bc060ce15db10: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940853c575bc060ce15db10 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940853c575bc060ce15db10 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940853c575bc060ce15db10 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:24,342 - lava_callback - INFO - Starting async job submit for node 694085a7575bc060ce15dce0 2025-12-17 22:08:24,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:24,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a7575bc060ce15dce0 HTTP/1.1" 502 150 2025-12-17 22:08:24,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a7575bc060ce15dce0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:24,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a7575bc060ce15dce0 2025-12-17 22:08:24,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a7575bc060ce15dce0 HTTP/1.1" 502 150 2025-12-17 22:08:24,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a7575bc060ce15dce0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:24,388 - lava_callback - INFO - Received callback for node 694086c6575bc060ce15e17b INFO: 172.21.0.1:46710 - "POST /node/694086c6575bc060ce15e17b HTTP/1.0" 202 Accepted 2025-12-17 22:08:24,408 - lava_callback - INFO - Received callback for node 694086d4575bc060ce15e1a9 INFO: 172.21.0.1:46718 - "POST /node/694086d4575bc060ce15e1a9 HTTP/1.0" 202 Accepted 2025-12-17 22:08:24,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408596575bc060ce15dca0 2025-12-17 22:08:24,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408596575bc060ce15dca0 HTTP/1.1" 502 150 2025-12-17 22:08:24,470 - lava_callback - ERROR - Error processing callback for node 69408596575bc060ce15dca0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408596575bc060ce15dca0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408596575bc060ce15dca0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408596575bc060ce15dca0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:24,472 - lava_callback - INFO - Starting async job submit for node 694085a9575bc060ce15dce3 2025-12-17 22:08:24,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:24,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a9575bc060ce15dce3 HTTP/1.1" 502 150 2025-12-17 22:08:24,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a9575bc060ce15dce3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:24,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a9575bc060ce15dce3 2025-12-17 22:08:24,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a9575bc060ce15dce3 HTTP/1.1" 502 150 2025-12-17 22:08:24,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a9575bc060ce15dce3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:25,032 - lava_callback - INFO - Received callback for node 694086d6575bc060ce15e1af INFO: 172.21.0.1:46724 - "POST /node/694086d6575bc060ce15e1af HTTP/1.0" 202 Accepted 2025-12-17 22:08:25,192 - lava_callback - INFO - Received callback for node 694086d5575bc060ce15e1ac INFO: 172.21.0.1:46726 - "POST /node/694086d5575bc060ce15e1ac HTTP/1.0" 202 Accepted 2025-12-17 22:08:25,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408598575bc060ce15dca7 HTTP/1.1" 502 150 2025-12-17 22:08:25,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408598575bc060ce15dca7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:25,264 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408598575bc060ce15dca7 2025-12-17 22:08:25,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859a575bc060ce15dcae 2025-12-17 22:08:25,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859a575bc060ce15dcae HTTP/1.1" 502 150 2025-12-17 22:08:25,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940859a575bc060ce15dcae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:26,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085af575bc060ce15dcfd 2025-12-17 22:08:26,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085af575bc060ce15dcfd HTTP/1.1" 502 150 2025-12-17 22:08:26,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085af575bc060ce15dcfd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:26,355 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a7575bc060ce15dce0 2025-12-17 22:08:26,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a7575bc060ce15dce0 HTTP/1.1" 502 150 2025-12-17 22:08:26,356 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a7575bc060ce15dce0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:26,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a9575bc060ce15dce3 2025-12-17 22:08:26,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a9575bc060ce15dce3 HTTP/1.1" 502 150 2025-12-17 22:08:26,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a9575bc060ce15dce3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46738 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:08:28,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a1575bc060ce15dcca 2025-12-17 22:08:28,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a1575bc060ce15dcca HTTP/1.1" 502 150 2025-12-17 22:08:28,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a1575bc060ce15dcca'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:28,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a3575bc060ce15dccd 2025-12-17 22:08:28,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a3575bc060ce15dccd HTTP/1.1" 502 150 2025-12-17 22:08:28,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a3575bc060ce15dccd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:28,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859b575bc060ce15dcb5 2025-12-17 22:08:28,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859b575bc060ce15dcb5 HTTP/1.1" 502 150 2025-12-17 22:08:28,914 - lava_callback - ERROR - Error processing callback for node 6940859b575bc060ce15dcb5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940859b575bc060ce15dcb5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940859b575bc060ce15dcb5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940859b575bc060ce15dcb5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:28,916 - lava_callback - INFO - Starting async job submit for node 6942e380e43d2b76e9023a67 2025-12-17 22:08:28,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:28,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e380e43d2b76e9023a67 HTTP/1.1" 502 150 2025-12-17 22:08:28,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e380e43d2b76e9023a67'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:28,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e380e43d2b76e9023a67 2025-12-17 22:08:28,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e380e43d2b76e9023a67 HTTP/1.1" 502 150 2025-12-17 22:08:28,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e380e43d2b76e9023a67'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:29,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a4575bc060ce15dcd4 2025-12-17 22:08:29,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a4575bc060ce15dcd7 2025-12-17 22:08:29,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a4575bc060ce15dcd7 HTTP/1.1" 502 150 2025-12-17 22:08:29,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a4575bc060ce15dcd7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:30,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085af575bc060ce15dcfd 2025-12-17 22:08:30,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085af575bc060ce15dcfd HTTP/1.1" 502 150 2025-12-17 22:08:30,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085af575bc060ce15dcfd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:30,356 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a7575bc060ce15dce0 2025-12-17 22:08:30,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a7575bc060ce15dce0 HTTP/1.1" 502 150 2025-12-17 22:08:30,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a7575bc060ce15dce0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:30,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a9575bc060ce15dce3 2025-12-17 22:08:30,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a9575bc060ce15dce3 HTTP/1.1" 502 150 2025-12-17 22:08:30,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a9575bc060ce15dce3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:30,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859c575bc060ce15dcbc 2025-12-17 22:08:30,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859c575bc060ce15dcbc HTTP/1.1" 502 150 2025-12-17 22:08:30,833 - lava_callback - ERROR - Error processing callback for node 6940859c575bc060ce15dcbc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940859c575bc060ce15dcbc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940859c575bc060ce15dcbc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940859c575bc060ce15dcbc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:30,836 - lava_callback - INFO - Starting async job submit for node 694085ad575bc060ce15dcf5 2025-12-17 22:08:30,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:30,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ad575bc060ce15dcf5 HTTP/1.1" 502 150 2025-12-17 22:08:30,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ad575bc060ce15dcf5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:30,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ad575bc060ce15dcf5 2025-12-17 22:08:30,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ad575bc060ce15dcf5 HTTP/1.1" 502 150 2025-12-17 22:08:30,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ad575bc060ce15dcf5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:30,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a5575bc060ce15dcda 2025-12-17 22:08:30,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a5575bc060ce15dcda HTTP/1.1" 502 150 2025-12-17 22:08:30,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a5575bc060ce15dcda'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:30,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e380e43d2b76e9023a67 2025-12-17 22:08:30,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e380e43d2b76e9023a67 HTTP/1.1" 502 150 2025-12-17 22:08:30,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e380e43d2b76e9023a67'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:31,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a6575bc060ce15dcdd 2025-12-17 22:08:31,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a6575bc060ce15dcdd HTTP/1.1" 502 150 2025-12-17 22:08:31,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a6575bc060ce15dcdd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:32,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ad575bc060ce15dcf5 2025-12-17 22:08:32,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ad575bc060ce15dcf5 HTTP/1.1" 502 150 2025-12-17 22:08:32,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ad575bc060ce15dcf5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:33,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408594575bc060ce15dc9a HTTP/1.1" 502 150 2025-12-17 22:08:33,716 - lava_callback - ERROR - Error processing callback for node 69408594575bc060ce15dc9a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408594575bc060ce15dc9a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408594575bc060ce15dc9a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408594575bc060ce15dc9a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:33,718 - lava_callback - INFO - Starting async job submit for node 694085ae575bc060ce15dcfa 2025-12-17 22:08:33,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:33,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ae575bc060ce15dcfa HTTP/1.1" 502 150 2025-12-17 22:08:33,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ae575bc060ce15dcfa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:33,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ae575bc060ce15dcfa 2025-12-17 22:08:33,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ae575bc060ce15dcfa HTTP/1.1" 502 150 2025-12-17 22:08:33,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ae575bc060ce15dcfa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:34,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e380e43d2b76e9023a67 2025-12-17 22:08:34,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e380e43d2b76e9023a67 HTTP/1.1" 502 150 2025-12-17 22:08:34,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e380e43d2b76e9023a67'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:35,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ae575bc060ce15dcfa 2025-12-17 22:08:35,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ae575bc060ce15dcfa HTTP/1.1" 502 150 2025-12-17 22:08:35,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ae575bc060ce15dcfa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:36,152 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859d575bc060ce15dcc1 2025-12-17 22:08:36,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859d575bc060ce15dcc1 HTTP/1.1" 502 150 2025-12-17 22:08:36,153 - lava_callback - ERROR - Error processing callback for node 6940859d575bc060ce15dcc1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940859d575bc060ce15dcc1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940859d575bc060ce15dcc1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940859d575bc060ce15dcc1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:36,155 - lava_callback - INFO - Starting async job submit for node 694085b2575bc060ce15dd05 2025-12-17 22:08:36,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:36,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b2575bc060ce15dd05 HTTP/1.1" 502 150 2025-12-17 22:08:36,161 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b2575bc060ce15dd05'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:36,161 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b2575bc060ce15dd05 2025-12-17 22:08:36,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b2575bc060ce15dd05 HTTP/1.1" 502 150 2025-12-17 22:08:36,162 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b2575bc060ce15dd05'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43912 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:08:36,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859f575bc060ce15dcc4 2025-12-17 22:08:36,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859f575bc060ce15dcc4 HTTP/1.1" 502 150 2025-12-17 22:08:36,773 - lava_callback - ERROR - Error processing callback for node 6940859f575bc060ce15dcc4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940859f575bc060ce15dcc4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940859f575bc060ce15dcc4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940859f575bc060ce15dcc4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:36,775 - lava_callback - INFO - Starting async job submit for node 694085b3575bc060ce15dd0c 2025-12-17 22:08:36,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:36,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b3575bc060ce15dd0c HTTP/1.1" 502 150 2025-12-17 22:08:36,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b3575bc060ce15dd0c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:36,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b3575bc060ce15dd0c 2025-12-17 22:08:36,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b3575bc060ce15dd0c HTTP/1.1" 502 150 2025-12-17 22:08:36,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b3575bc060ce15dd0c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:36,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ad575bc060ce15dcf5 2025-12-17 22:08:36,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ad575bc060ce15dcf5 HTTP/1.1" 502 150 2025-12-17 22:08:36,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ad575bc060ce15dcf5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:38,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085af575bc060ce15dcfd 2025-12-17 22:08:38,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085af575bc060ce15dcfd HTTP/1.1" 502 150 2025-12-17 22:08:38,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085af575bc060ce15dcfd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:38,162 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b2575bc060ce15dd05 2025-12-17 22:08:38,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b2575bc060ce15dd05 HTTP/1.1" 502 150 2025-12-17 22:08:38,164 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b2575bc060ce15dd05'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:38,358 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a7575bc060ce15dce0 2025-12-17 22:08:38,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a7575bc060ce15dce0 HTTP/1.1" 502 150 2025-12-17 22:08:38,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a7575bc060ce15dce0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:38,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a9575bc060ce15dce3 2025-12-17 22:08:38,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a9575bc060ce15dce3 HTTP/1.1" 502 150 2025-12-17 22:08:38,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a9575bc060ce15dce3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:38,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b3575bc060ce15dd0c 2025-12-17 22:08:38,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b3575bc060ce15dd0c HTTP/1.1" 502 150 2025-12-17 22:08:38,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b3575bc060ce15dd0c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:39,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ae575bc060ce15dcfa 2025-12-17 22:08:39,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ae575bc060ce15dcfa HTTP/1.1" 502 150 2025-12-17 22:08:39,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ae575bc060ce15dcfa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:41,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940859a575bc060ce15dcae 2025-12-17 22:08:41,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940859a575bc060ce15dcae HTTP/1.1" 502 150 2025-12-17 22:08:41,446 - lava_callback - ERROR - Error processing callback for node 6940859a575bc060ce15dcae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940859a575bc060ce15dcae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940859a575bc060ce15dcae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940859a575bc060ce15dcae (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:41,448 - lava_callback - INFO - Starting async job submit for node 694084ae575bc060ce15d873 2025-12-17 22:08:41,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:41,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ae575bc060ce15d873 HTTP/1.1" 502 150 2025-12-17 22:08:41,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ae575bc060ce15d873'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:41,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ae575bc060ce15d873 2025-12-17 22:08:41,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ae575bc060ce15d873 HTTP/1.1" 502 150 2025-12-17 22:08:41,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ae575bc060ce15d873'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:42,164 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b2575bc060ce15dd05 2025-12-17 22:08:42,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b2575bc060ce15dd05 HTTP/1.1" 502 150 2025-12-17 22:08:42,169 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b2575bc060ce15dd05'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:42,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b3575bc060ce15dd0c 2025-12-17 22:08:42,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b3575bc060ce15dd0c HTTP/1.1" 502 150 2025-12-17 22:08:42,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b3575bc060ce15dd0c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:42,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e380e43d2b76e9023a67 2025-12-17 22:08:42,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e380e43d2b76e9023a67 HTTP/1.1" 502 150 2025-12-17 22:08:42,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e380e43d2b76e9023a67'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:43,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ae575bc060ce15d873 2025-12-17 22:08:43,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ae575bc060ce15d873 HTTP/1.1" 502 150 2025-12-17 22:08:43,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ae575bc060ce15d873'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:43,769 - lava_callback - INFO - Received callback for node 694086d8575bc060ce15e1b6 INFO: 172.21.0.1:33968 - "POST /node/694086d8575bc060ce15e1b6 HTTP/1.0" 202 Accepted 2025-12-17 22:08:44,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a1575bc060ce15dcca 2025-12-17 22:08:44,437 - lava_callback - INFO - Received callback for node 694086da575bc060ce15e1bf INFO: 172.21.0.1:33982 - "POST /node/694086da575bc060ce15e1bf HTTP/1.0" 202 Accepted 2025-12-17 22:08:44,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ad575bc060ce15dcf5 2025-12-17 22:08:44,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ad575bc060ce15dcf5 HTTP/1.1" 502 150 2025-12-17 22:08:44,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ad575bc060ce15dcf5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:44,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a3575bc060ce15dccd 2025-12-17 22:08:44,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a3575bc060ce15dccd HTTP/1.1" 502 150 2025-12-17 22:08:44,912 - lava_callback - ERROR - Error processing callback for node 694085a3575bc060ce15dccd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a3575bc060ce15dccd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a3575bc060ce15dccd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a3575bc060ce15dccd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:44,915 - lava_callback - INFO - Starting async job submit for node 694085b5575bc060ce15dd0f 2025-12-17 22:08:44,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:44,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b5575bc060ce15dd0f HTTP/1.1" 502 150 2025-12-17 22:08:44,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b5575bc060ce15dd0f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:44,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b5575bc060ce15dd0f 2025-12-17 22:08:44,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b5575bc060ce15dd0f HTTP/1.1" 502 150 2025-12-17 22:08:44,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b5575bc060ce15dd0f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:44,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a4575bc060ce15dcd4 HTTP/1.1" 502 150 2025-12-17 22:08:44,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a4575bc060ce15dcd4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:45,077 - lava_callback - INFO - Received callback for node 694086dc575bc060ce15e1c2 INFO: 172.21.0.1:33986 - "POST /node/694086dc575bc060ce15e1c2 HTTP/1.0" 202 Accepted 2025-12-17 22:08:45,724 - lava_callback - INFO - Received callback for node 694086de575bc060ce15e1c5 INFO: 172.21.0.1:33992 - "POST /node/694086de575bc060ce15e1c5 HTTP/1.0" 202 Accepted 2025-12-17 22:08:45,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a4575bc060ce15dcd7 2025-12-17 22:08:45,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a4575bc060ce15dcd7 HTTP/1.1" 502 150 2025-12-17 22:08:45,882 - lava_callback - ERROR - Error processing callback for node 694085a4575bc060ce15dcd7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a4575bc060ce15dcd7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a4575bc060ce15dcd7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a4575bc060ce15dcd7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:45,884 - lava_callback - INFO - Starting async job submit for node 694085b6575bc060ce15dd12 2025-12-17 22:08:45,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:45,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b6575bc060ce15dd12 HTTP/1.1" 502 150 2025-12-17 22:08:45,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b6575bc060ce15dd12'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:45,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b6575bc060ce15dd12 2025-12-17 22:08:45,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b6575bc060ce15dd12 HTTP/1.1" 502 150 2025-12-17 22:08:45,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b6575bc060ce15dd12'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:46,359 - lava_callback - INFO - Received callback for node 694086e0575bc060ce15e1c8 INFO: 172.21.0.1:33994 - "POST /node/694086e0575bc060ce15e1c8 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:33998 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:08:46,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a5575bc060ce15dcda 2025-12-17 22:08:46,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a5575bc060ce15dcda HTTP/1.1" 502 150 2025-12-17 22:08:46,908 - lava_callback - ERROR - Error processing callback for node 694085a5575bc060ce15dcda: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a5575bc060ce15dcda (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a5575bc060ce15dcda (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a5575bc060ce15dcda (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:46,910 - lava_callback - INFO - Starting async job submit for node 694085b7575bc060ce15dd15 2025-12-17 22:08:46,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:46,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b7575bc060ce15dd15 HTTP/1.1" 502 150 2025-12-17 22:08:46,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b7575bc060ce15dd15'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:46,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b7575bc060ce15dd15 2025-12-17 22:08:46,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b7575bc060ce15dd15 HTTP/1.1" 502 150 2025-12-17 22:08:46,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b7575bc060ce15dd15'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:46,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b5575bc060ce15dd0f 2025-12-17 22:08:46,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b5575bc060ce15dd0f HTTP/1.1" 502 150 2025-12-17 22:08:46,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b5575bc060ce15dd0f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:47,007 - lava_callback - INFO - Received callback for node 694086e2575bc060ce15e1cb INFO: 172.21.0.1:34010 - "POST /node/694086e2575bc060ce15e1cb HTTP/1.0" 202 Accepted 2025-12-17 22:08:47,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a6575bc060ce15dcdd 2025-12-17 22:08:47,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a6575bc060ce15dcdd HTTP/1.1" 502 150 2025-12-17 22:08:47,438 - lava_callback - ERROR - Error processing callback for node 694085a6575bc060ce15dcdd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a6575bc060ce15dcdd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a6575bc060ce15dcdd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a6575bc060ce15dcdd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:47,440 - lava_callback - INFO - Starting async job submit for node 694085b9575bc060ce15dd49 2025-12-17 22:08:47,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:47,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b9575bc060ce15dd49 HTTP/1.1" 502 150 2025-12-17 22:08:47,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b9575bc060ce15dd49'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:47,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b9575bc060ce15dd49 2025-12-17 22:08:47,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b9575bc060ce15dd49 HTTP/1.1" 502 150 2025-12-17 22:08:47,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b9575bc060ce15dd49'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:47,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ae575bc060ce15d873 2025-12-17 22:08:47,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ae575bc060ce15d873 HTTP/1.1" 502 150 2025-12-17 22:08:47,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ae575bc060ce15d873'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:47,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ae575bc060ce15dcfa 2025-12-17 22:08:47,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ae575bc060ce15dcfa HTTP/1.1" 502 150 2025-12-17 22:08:47,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ae575bc060ce15dcfa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:47,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b6575bc060ce15dd12 2025-12-17 22:08:47,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b6575bc060ce15dd12 HTTP/1.1" 502 150 2025-12-17 22:08:47,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b6575bc060ce15dd12'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:48,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b7575bc060ce15dd15 2025-12-17 22:08:48,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b7575bc060ce15dd15 HTTP/1.1" 502 150 2025-12-17 22:08:48,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b7575bc060ce15dd15'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:49,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b9575bc060ce15dd49 2025-12-17 22:08:49,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b9575bc060ce15dd49 HTTP/1.1" 502 150 2025-12-17 22:08:49,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b9575bc060ce15dd49'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:50,170 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b2575bc060ce15dd05 2025-12-17 22:08:50,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b2575bc060ce15dd05 HTTP/1.1" 502 150 2025-12-17 22:08:50,171 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b2575bc060ce15dd05'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:50,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b3575bc060ce15dd0c 2025-12-17 22:08:50,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b3575bc060ce15dd0c HTTP/1.1" 502 150 2025-12-17 22:08:50,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b3575bc060ce15dd0c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:50,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b5575bc060ce15dd0f 2025-12-17 22:08:50,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b5575bc060ce15dd0f HTTP/1.1" 502 150 2025-12-17 22:08:50,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b5575bc060ce15dd0f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:51,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b6575bc060ce15dd12 2025-12-17 22:08:51,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b6575bc060ce15dd12 HTTP/1.1" 502 150 2025-12-17 22:08:51,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b6575bc060ce15dd12'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:52,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b7575bc060ce15dd15 2025-12-17 22:08:52,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b7575bc060ce15dd15 HTTP/1.1" 502 150 2025-12-17 22:08:52,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b7575bc060ce15dd15'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:53,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b9575bc060ce15dd49 2025-12-17 22:08:53,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b9575bc060ce15dd49 HTTP/1.1" 502 150 2025-12-17 22:08:53,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b9575bc060ce15dd49'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:53,979 - lava_callback - INFO - Received callback for node 694086e6575bc060ce15e1d1 INFO: 172.21.0.1:58168 - "POST /node/694086e6575bc060ce15e1d1 HTTP/1.0" 202 Accepted 2025-12-17 22:08:54,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085af575bc060ce15dcfd 2025-12-17 22:08:54,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085af575bc060ce15dcfd HTTP/1.1" 502 150 2025-12-17 22:08:54,035 - lava_callback - ERROR - Error processing callback for node 694085af575bc060ce15dcfd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085af575bc060ce15dcfd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085af575bc060ce15dcfd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085af575bc060ce15dcfd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:54,036 - lava_callback - INFO - Starting async job submit for node 694085bb575bc060ce15dd70 2025-12-17 22:08:54,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:54,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bb575bc060ce15dd70 HTTP/1.1" 502 150 2025-12-17 22:08:54,052 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085bb575bc060ce15dd70'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:54,052 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085bb575bc060ce15dd70 2025-12-17 22:08:54,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bb575bc060ce15dd70 HTTP/1.1" 502 150 2025-12-17 22:08:54,053 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085bb575bc060ce15dd70'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:54,359 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a7575bc060ce15dce0 2025-12-17 22:08:54,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a7575bc060ce15dce0 HTTP/1.1" 502 150 2025-12-17 22:08:54,361 - lava_callback - ERROR - Error processing callback for node 694085a7575bc060ce15dce0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a7575bc060ce15dce0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a7575bc060ce15dce0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a7575bc060ce15dce0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:54,362 - lava_callback - INFO - Starting async job submit for node 694085bd575bc060ce15dd78 2025-12-17 22:08:54,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:54,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bd575bc060ce15dd78 HTTP/1.1" 502 150 2025-12-17 22:08:54,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085bd575bc060ce15dd78'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:54,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085bd575bc060ce15dd78 2025-12-17 22:08:54,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bd575bc060ce15dd78 HTTP/1.1" 502 150 2025-12-17 22:08:54,370 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085bd575bc060ce15dd78'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:54,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a9575bc060ce15dce3 2025-12-17 22:08:54,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a9575bc060ce15dce3 HTTP/1.1" 502 150 2025-12-17 22:08:54,485 - lava_callback - ERROR - Error processing callback for node 694085a9575bc060ce15dce3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a9575bc060ce15dce3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a9575bc060ce15dce3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a9575bc060ce15dce3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:54,486 - lava_callback - INFO - Starting async job submit for node 694085be575bc060ce15dd7d 2025-12-17 22:08:54,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:54,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085be575bc060ce15dd7d HTTP/1.1" 502 150 2025-12-17 22:08:54,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085be575bc060ce15dd7d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:54,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085be575bc060ce15dd7d 2025-12-17 22:08:54,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085be575bc060ce15dd7d HTTP/1.1" 502 150 2025-12-17 22:08:54,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085be575bc060ce15dd7d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:54,644 - lava_callback - INFO - Received callback for node 694086ea575bc060ce15e1d9 INFO: 172.21.0.1:58174 - "POST /node/694086ea575bc060ce15e1d9 HTTP/1.0" 202 Accepted 2025-12-17 22:08:55,317 - lava_callback - INFO - Received callback for node 694086ec575bc060ce15e1dc INFO: 172.21.0.1:58184 - "POST /node/694086ec575bc060ce15e1dc HTTP/1.0" 202 Accepted 2025-12-17 22:08:55,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ae575bc060ce15d873 2025-12-17 22:08:55,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ae575bc060ce15d873 HTTP/1.1" 502 150 2025-12-17 22:08:55,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694084ae575bc060ce15d873'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:55,980 - lava_callback - INFO - Received callback for node 694086ef575bc060ce15e21b INFO: 172.21.0.1:58198 - "POST /node/694086ef575bc060ce15e21b HTTP/1.0" 202 Accepted 2025-12-17 22:08:55,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408598575bc060ce15dca7 HTTP/1.1" 502 150 2025-12-17 22:08:55,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408598575bc060ce15dca7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:56,053 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085bb575bc060ce15dd70 2025-12-17 22:08:56,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bb575bc060ce15dd70 HTTP/1.1" 502 150 2025-12-17 22:08:56,054 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085bb575bc060ce15dd70'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:56,372 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085bd575bc060ce15dd78 2025-12-17 22:08:56,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bd575bc060ce15dd78 HTTP/1.1" 502 150 2025-12-17 22:08:56,374 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085bd575bc060ce15dd78'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:56,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085be575bc060ce15dd7d 2025-12-17 22:08:56,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085be575bc060ce15dd7d HTTP/1.1" 502 150 2025-12-17 22:08:56,495 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085be575bc060ce15dd7d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58210 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:08:56,658 - lava_callback - INFO - Received callback for node 694086f1575bc060ce15e220 INFO: 172.21.0.1:58216 - "POST /node/694086f1575bc060ce15e220 HTTP/1.0" 202 Accepted 2025-12-17 22:08:57,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408598575bc060ce15dca7 2025-12-17 22:08:58,567 - lava_callback - INFO - Received callback for node 69409080575bc060ce1605b6 INFO: 172.21.0.1:58230 - "POST /node/69409080575bc060ce1605b6 HTTP/1.0" 202 Accepted 2025-12-17 22:08:58,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b5575bc060ce15dd0f 2025-12-17 22:08:58,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e380e43d2b76e9023a67 2025-12-17 22:08:58,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b5575bc060ce15dd0f HTTP/1.1" 502 150 2025-12-17 22:08:58,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b5575bc060ce15dd0f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:58,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e380e43d2b76e9023a67 HTTP/1.1" 502 150 2025-12-17 22:08:58,931 - lava_callback - ERROR - Error processing callback for node 6942e380e43d2b76e9023a67: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e380e43d2b76e9023a67 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e380e43d2b76e9023a67 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e380e43d2b76e9023a67 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:58,933 - lava_callback - INFO - Starting async job submit for node 694085bf575bc060ce15dd88 2025-12-17 22:08:58,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:58,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bf575bc060ce15dd88 HTTP/1.1" 502 150 2025-12-17 22:08:58,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085bf575bc060ce15dd88'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:58,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085bf575bc060ce15dd88 2025-12-17 22:08:58,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bf575bc060ce15dd88 HTTP/1.1" 502 150 2025-12-17 22:08:58,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085bf575bc060ce15dd88'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:59,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a1575bc060ce15dcca HTTP/1.1" 502 150 2025-12-17 22:08:59,312 - lava_callback - ERROR - Error processing callback for node 694085a1575bc060ce15dcca: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a1575bc060ce15dcca (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a1575bc060ce15dcca (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a1575bc060ce15dcca (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:08:59,315 - lava_callback - INFO - Starting async job submit for node 694085c1575bc060ce15dd8f 2025-12-17 22:08:59,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:08:59,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c1575bc060ce15dd8f HTTP/1.1" 502 150 2025-12-17 22:08:59,322 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c1575bc060ce15dd8f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:59,322 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c1575bc060ce15dd8f 2025-12-17 22:08:59,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c1575bc060ce15dd8f HTTP/1.1" 502 150 2025-12-17 22:08:59,323 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c1575bc060ce15dd8f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:08:59,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b6575bc060ce15dd12 2025-12-17 22:08:59,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b6575bc060ce15dd12 HTTP/1.1" 502 150 2025-12-17 22:08:59,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b6575bc060ce15dd12'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:00,055 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085bb575bc060ce15dd70 2025-12-17 22:09:00,374 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085bd575bc060ce15dd78 2025-12-17 22:09:00,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bd575bc060ce15dd78 HTTP/1.1" 502 150 2025-12-17 22:09:00,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085bd575bc060ce15dd78'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:00,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085be575bc060ce15dd7d 2025-12-17 22:09:00,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085be575bc060ce15dd7d HTTP/1.1" 502 150 2025-12-17 22:09:00,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085be575bc060ce15dd7d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:00,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ad575bc060ce15dcf5 2025-12-17 22:09:00,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ad575bc060ce15dcf5 HTTP/1.1" 502 150 2025-12-17 22:09:00,852 - lava_callback - ERROR - Error processing callback for node 694085ad575bc060ce15dcf5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085ad575bc060ce15dcf5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085ad575bc060ce15dcf5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085ad575bc060ce15dcf5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:00,854 - lava_callback - INFO - Starting async job submit for node 694085c2575bc060ce15dd96 2025-12-17 22:09:00,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:00,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c2575bc060ce15dd96 HTTP/1.1" 502 150 2025-12-17 22:09:00,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c2575bc060ce15dd96'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:00,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c2575bc060ce15dd96 2025-12-17 22:09:00,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c2575bc060ce15dd96 HTTP/1.1" 502 150 2025-12-17 22:09:00,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c2575bc060ce15dd96'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:00,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b7575bc060ce15dd15 2025-12-17 22:09:00,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b7575bc060ce15dd15 HTTP/1.1" 502 150 2025-12-17 22:09:00,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b7575bc060ce15dd15'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:00,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085bf575bc060ce15dd88 2025-12-17 22:09:00,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bf575bc060ce15dd88 HTTP/1.1" 502 150 2025-12-17 22:09:00,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085bf575bc060ce15dd88'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:00,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a4575bc060ce15dcd4 2025-12-17 22:09:00,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a4575bc060ce15dcd4 HTTP/1.1" 502 150 2025-12-17 22:09:00,978 - lava_callback - ERROR - Error processing callback for node 694085a4575bc060ce15dcd4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a4575bc060ce15dcd4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a4575bc060ce15dcd4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a4575bc060ce15dcd4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:00,979 - lava_callback - INFO - Starting async job submit for node 694085c4575bc060ce15dd9b 2025-12-17 22:09:00,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:00,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c4575bc060ce15dd9b HTTP/1.1" 502 150 2025-12-17 22:09:00,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c4575bc060ce15dd9b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:00,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c4575bc060ce15dd9b 2025-12-17 22:09:00,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c4575bc060ce15dd9b HTTP/1.1" 502 150 2025-12-17 22:09:00,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c4575bc060ce15dd9b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:01,323 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c1575bc060ce15dd8f 2025-12-17 22:09:01,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c1575bc060ce15dd8f HTTP/1.1" 502 150 2025-12-17 22:09:01,324 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c1575bc060ce15dd8f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:01,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b9575bc060ce15dd49 2025-12-17 22:09:01,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b9575bc060ce15dd49 HTTP/1.1" 502 150 2025-12-17 22:09:01,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085b9575bc060ce15dd49'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57226 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:09:02,110 - lava_callback - INFO - Received callback for node 694085cd575bc060ce15ddb0 INFO: 172.21.0.1:57234 - "POST /node/694085cd575bc060ce15ddb0 HTTP/1.0" 202 Accepted 2025-12-17 22:09:02,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c2575bc060ce15dd96 2025-12-17 22:09:02,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c2575bc060ce15dd96 HTTP/1.1" 502 150 2025-12-17 22:09:02,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c2575bc060ce15dd96'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:02,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c4575bc060ce15dd9b 2025-12-17 22:09:02,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c4575bc060ce15dd9b HTTP/1.1" 502 150 2025-12-17 22:09:02,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c4575bc060ce15dd9b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:03,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ae575bc060ce15dcfa 2025-12-17 22:09:03,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ae575bc060ce15dcfa HTTP/1.1" 502 150 2025-12-17 22:09:03,732 - lava_callback - ERROR - Error processing callback for node 694085ae575bc060ce15dcfa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085ae575bc060ce15dcfa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085ae575bc060ce15dcfa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085ae575bc060ce15dcfa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:03,734 - lava_callback - INFO - Starting async job submit for node 694085c4575bc060ce15dd9e 2025-12-17 22:09:03,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:03,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c4575bc060ce15dd9e HTTP/1.1" 502 150 2025-12-17 22:09:03,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c4575bc060ce15dd9e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:03,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c4575bc060ce15dd9e 2025-12-17 22:09:03,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c4575bc060ce15dd9e HTTP/1.1" 502 150 2025-12-17 22:09:03,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c4575bc060ce15dd9e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:03,808 - lava_callback - INFO - Received callback for node 694086fb575bc060ce15e22c INFO: 172.21.0.1:57236 - "POST /node/694086fb575bc060ce15e22c HTTP/1.0" 202 Accepted 2025-12-17 22:09:04,444 - lava_callback - INFO - Received callback for node 694086fd575bc060ce15e22f INFO: 172.21.0.1:57250 - "POST /node/694086fd575bc060ce15e22f HTTP/1.0" 202 Accepted 2025-12-17 22:09:04,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085bf575bc060ce15dd88 2025-12-17 22:09:04,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bf575bc060ce15dd88 HTTP/1.1" 502 150 2025-12-17 22:09:04,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085bf575bc060ce15dd88'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:05,325 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c1575bc060ce15dd8f 2025-12-17 22:09:05,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c1575bc060ce15dd8f HTTP/1.1" 502 150 2025-12-17 22:09:05,326 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c1575bc060ce15dd8f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:05,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c4575bc060ce15dd9e 2025-12-17 22:09:05,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c4575bc060ce15dd9e HTTP/1.1" 502 150 2025-12-17 22:09:05,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c4575bc060ce15dd9e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:06,171 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b2575bc060ce15dd05 2025-12-17 22:09:06,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b2575bc060ce15dd05 HTTP/1.1" 502 150 2025-12-17 22:09:06,173 - lava_callback - ERROR - Error processing callback for node 694085b2575bc060ce15dd05: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b2575bc060ce15dd05 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b2575bc060ce15dd05 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b2575bc060ce15dd05 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:06,175 - lava_callback - INFO - Starting async job submit for node 694085c6575bc060ce15dda1 2025-12-17 22:09:06,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:06,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c6575bc060ce15dda1 HTTP/1.1" 502 150 2025-12-17 22:09:06,181 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c6575bc060ce15dda1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:06,181 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c6575bc060ce15dda1 2025-12-17 22:09:06,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c6575bc060ce15dda1 HTTP/1.1" 502 150 2025-12-17 22:09:06,182 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c6575bc060ce15dda1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57262 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:09:06,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b3575bc060ce15dd0c 2025-12-17 22:09:06,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b3575bc060ce15dd0c HTTP/1.1" 502 150 2025-12-17 22:09:06,792 - lava_callback - ERROR - Error processing callback for node 694085b3575bc060ce15dd0c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b3575bc060ce15dd0c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b3575bc060ce15dd0c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b3575bc060ce15dd0c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:06,794 - lava_callback - INFO - Starting async job submit for node 694085c8575bc060ce15dda4 2025-12-17 22:09:06,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:06,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c8575bc060ce15dda4 HTTP/1.1" 502 150 2025-12-17 22:09:06,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c8575bc060ce15dda4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:06,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c8575bc060ce15dda4 2025-12-17 22:09:06,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c8575bc060ce15dda4 HTTP/1.1" 502 150 2025-12-17 22:09:06,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c8575bc060ce15dda4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:06,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c2575bc060ce15dd96 2025-12-17 22:09:06,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c2575bc060ce15dd96 HTTP/1.1" 502 150 2025-12-17 22:09:06,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c2575bc060ce15dd96'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:06,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c4575bc060ce15dd9b 2025-12-17 22:09:06,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c4575bc060ce15dd9b HTTP/1.1" 502 150 2025-12-17 22:09:06,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c4575bc060ce15dd9b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:08,182 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c6575bc060ce15dda1 2025-12-17 22:09:08,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c6575bc060ce15dda1 HTTP/1.1" 502 150 2025-12-17 22:09:08,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c6575bc060ce15dda1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:08,376 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085bd575bc060ce15dd78 2025-12-17 22:09:08,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bd575bc060ce15dd78 HTTP/1.1" 502 150 2025-12-17 22:09:08,377 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085bd575bc060ce15dd78'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:08,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085be575bc060ce15dd7d 2025-12-17 22:09:08,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085be575bc060ce15dd7d HTTP/1.1" 502 150 2025-12-17 22:09:08,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085be575bc060ce15dd7d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:08,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c8575bc060ce15dda4 2025-12-17 22:09:08,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c8575bc060ce15dda4 HTTP/1.1" 502 150 2025-12-17 22:09:08,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c8575bc060ce15dda4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:09,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c4575bc060ce15dd9e 2025-12-17 22:09:09,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c4575bc060ce15dd9e HTTP/1.1" 502 150 2025-12-17 22:09:09,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c4575bc060ce15dd9e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:11,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694084ae575bc060ce15d873 2025-12-17 22:09:12,183 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c6575bc060ce15dda1 2025-12-17 22:09:12,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c6575bc060ce15dda1 HTTP/1.1" 502 150 2025-12-17 22:09:12,185 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c6575bc060ce15dda1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:12,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c8575bc060ce15dda4 2025-12-17 22:09:12,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c8575bc060ce15dda4 HTTP/1.1" 502 150 2025-12-17 22:09:12,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c8575bc060ce15dda4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:12,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085bf575bc060ce15dd88 2025-12-17 22:09:12,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bf575bc060ce15dd88 HTTP/1.1" 502 150 2025-12-17 22:09:12,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085bf575bc060ce15dd88'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:13,326 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c1575bc060ce15dd8f 2025-12-17 22:09:13,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c1575bc060ce15dd8f HTTP/1.1" 502 150 2025-12-17 22:09:13,327 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c1575bc060ce15dd8f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:14,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c2575bc060ce15dd96 2025-12-17 22:09:14,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c2575bc060ce15dd96 HTTP/1.1" 502 150 2025-12-17 22:09:14,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c2575bc060ce15dd96'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:14,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b5575bc060ce15dd0f 2025-12-17 22:09:14,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b5575bc060ce15dd0f HTTP/1.1" 502 150 2025-12-17 22:09:14,932 - lava_callback - ERROR - Error processing callback for node 694085b5575bc060ce15dd0f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b5575bc060ce15dd0f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b5575bc060ce15dd0f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b5575bc060ce15dd0f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:14,934 - lava_callback - INFO - Starting async job submit for node 6940c5331e3c24e95681d6a4 2025-12-17 22:09:14,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c4575bc060ce15dd9b 2025-12-17 22:09:14,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:15,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c4575bc060ce15dd9b HTTP/1.1" 502 150 2025-12-17 22:09:15,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c4575bc060ce15dd9b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:15,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5331e3c24e95681d6a4 HTTP/1.1" 502 150 2025-12-17 22:09:15,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5331e3c24e95681d6a4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:15,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5331e3c24e95681d6a4 2025-12-17 22:09:15,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5331e3c24e95681d6a4 HTTP/1.1" 502 150 2025-12-17 22:09:15,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5331e3c24e95681d6a4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38106 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:09:15,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bb575bc060ce15dd70 HTTP/1.1" 502 150 2025-12-17 22:09:15,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085bb575bc060ce15dd70'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:15,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b6575bc060ce15dd12 2025-12-17 22:09:15,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b6575bc060ce15dd12 HTTP/1.1" 502 150 2025-12-17 22:09:15,898 - lava_callback - ERROR - Error processing callback for node 694085b6575bc060ce15dd12: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b6575bc060ce15dd12 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b6575bc060ce15dd12 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b6575bc060ce15dd12 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:15,901 - lava_callback - INFO - Starting async job submit for node 694085d0575bc060ce15ddb9 2025-12-17 22:09:15,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:15,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d0575bc060ce15ddb9 HTTP/1.1" 502 150 2025-12-17 22:09:15,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d0575bc060ce15ddb9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:15,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d0575bc060ce15ddb9 2025-12-17 22:09:15,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d0575bc060ce15ddb9 HTTP/1.1" 502 150 2025-12-17 22:09:15,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d0575bc060ce15ddb9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38112 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:09:16,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b7575bc060ce15dd15 2025-12-17 22:09:16,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b7575bc060ce15dd15 HTTP/1.1" 502 150 2025-12-17 22:09:16,923 - lava_callback - ERROR - Error processing callback for node 694085b7575bc060ce15dd15: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b7575bc060ce15dd15 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b7575bc060ce15dd15 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b7575bc060ce15dd15 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:16,925 - lava_callback - INFO - Starting async job submit for node 694085d2575bc060ce15ddbe 2025-12-17 22:09:16,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:16,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d2575bc060ce15ddbe HTTP/1.1" 502 150 2025-12-17 22:09:16,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d2575bc060ce15ddbe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:16,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d2575bc060ce15ddbe 2025-12-17 22:09:16,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d2575bc060ce15ddbe HTTP/1.1" 502 150 2025-12-17 22:09:16,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d2575bc060ce15ddbe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:17,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5331e3c24e95681d6a4 2025-12-17 22:09:17,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5331e3c24e95681d6a4 HTTP/1.1" 502 150 2025-12-17 22:09:17,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5331e3c24e95681d6a4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:17,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085b9575bc060ce15dd49 2025-12-17 22:09:17,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085b9575bc060ce15dd49 HTTP/1.1" 502 150 2025-12-17 22:09:17,464 - lava_callback - ERROR - Error processing callback for node 694085b9575bc060ce15dd49: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b9575bc060ce15dd49 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b9575bc060ce15dd49 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085b9575bc060ce15dd49 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:17,466 - lava_callback - INFO - Starting async job submit for node 694085cc575bc060ce15ddad 2025-12-17 22:09:17,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:17,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cc575bc060ce15ddad HTTP/1.1" 502 150 2025-12-17 22:09:17,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085cc575bc060ce15ddad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:17,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085cc575bc060ce15ddad 2025-12-17 22:09:17,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cc575bc060ce15ddad HTTP/1.1" 502 150 2025-12-17 22:09:17,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085cc575bc060ce15ddad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:17,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c4575bc060ce15dd9e 2025-12-17 22:09:17,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c4575bc060ce15dd9e HTTP/1.1" 502 150 2025-12-17 22:09:17,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c4575bc060ce15dd9e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:17,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d0575bc060ce15ddb9 2025-12-17 22:09:17,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d0575bc060ce15ddb9 HTTP/1.1" 502 150 2025-12-17 22:09:17,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d0575bc060ce15ddb9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:18,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d2575bc060ce15ddbe 2025-12-17 22:09:18,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d2575bc060ce15ddbe HTTP/1.1" 502 150 2025-12-17 22:09:18,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d2575bc060ce15ddbe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:19,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085cc575bc060ce15ddad 2025-12-17 22:09:19,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cc575bc060ce15ddad HTTP/1.1" 502 150 2025-12-17 22:09:19,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085cc575bc060ce15ddad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:19,623 - lava_callback - INFO - Received callback for node 69408706575bc060ce15e244 INFO: 172.21.0.1:38122 - "POST /node/69408706575bc060ce15e244 HTTP/1.0" 202 Accepted 2025-12-17 22:09:20,185 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c6575bc060ce15dda1 2025-12-17 22:09:20,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c6575bc060ce15dda1 HTTP/1.1" 502 150 2025-12-17 22:09:20,188 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c6575bc060ce15dda1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:20,394 - lava_callback - INFO - Received callback for node 69408708575bc060ce15e24a INFO: 172.21.0.1:57386 - "POST /node/69408708575bc060ce15e24a HTTP/1.0" 202 Accepted 2025-12-17 22:09:20,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c8575bc060ce15dda4 2025-12-17 22:09:20,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c8575bc060ce15dda4 HTTP/1.1" 502 150 2025-12-17 22:09:20,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c8575bc060ce15dda4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:21,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5331e3c24e95681d6a4 2025-12-17 22:09:21,181 - lava_callback - INFO - Received callback for node 6940870b575bc060ce15e250 INFO: 172.21.0.1:57402 - "POST /node/6940870b575bc060ce15e250 HTTP/1.0" 202 Accepted 2025-12-17 22:09:21,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d0575bc060ce15ddb9 2025-12-17 22:09:21,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d0575bc060ce15ddb9 HTTP/1.1" 502 150 2025-12-17 22:09:21,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d0575bc060ce15ddb9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:22,236 - lava_callback - INFO - Received callback for node 6940870d575bc060ce15e253 INFO: 172.21.0.1:57418 - "POST /node/6940870d575bc060ce15e253 HTTP/1.0" 202 Accepted 2025-12-17 22:09:22,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d2575bc060ce15ddbe 2025-12-17 22:09:22,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d2575bc060ce15ddbe HTTP/1.1" 502 150 2025-12-17 22:09:22,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d2575bc060ce15ddbe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:23,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085bb575bc060ce15dd70 2025-12-17 22:09:23,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bb575bc060ce15dd70 HTTP/1.1" 502 150 2025-12-17 22:09:23,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085bb575bc060ce15dd70'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:23,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085cc575bc060ce15ddad 2025-12-17 22:09:23,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cc575bc060ce15ddad HTTP/1.1" 502 150 2025-12-17 22:09:23,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085cc575bc060ce15ddad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:23,796 - lava_callback - INFO - Received callback for node 694086ff575bc060ce15e238 INFO: 172.21.0.1:57430 - "POST /node/694086ff575bc060ce15e238 HTTP/1.0" 202 Accepted 2025-12-17 22:09:24,377 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085bd575bc060ce15dd78 2025-12-17 22:09:24,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bd575bc060ce15dd78 HTTP/1.1" 502 150 2025-12-17 22:09:24,378 - lava_callback - ERROR - Error processing callback for node 694085bd575bc060ce15dd78: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085bd575bc060ce15dd78 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085bd575bc060ce15dd78 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085bd575bc060ce15dd78 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:24,380 - lava_callback - INFO - Starting async job submit for node 694085d0575bc060ce15ddb6 2025-12-17 22:09:24,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:24,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d0575bc060ce15ddb6 HTTP/1.1" 502 150 2025-12-17 22:09:24,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d0575bc060ce15ddb6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:24,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d0575bc060ce15ddb6 2025-12-17 22:09:24,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d0575bc060ce15ddb6 HTTP/1.1" 502 150 2025-12-17 22:09:24,388 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d0575bc060ce15ddb6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:24,445 - lava_callback - INFO - Received callback for node 69408701575bc060ce15e23b INFO: 172.21.0.1:57434 - "POST /node/69408701575bc060ce15e23b HTTP/1.0" 202 Accepted 2025-12-17 22:09:24,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085be575bc060ce15dd7d 2025-12-17 22:09:24,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085be575bc060ce15dd7d HTTP/1.1" 502 150 2025-12-17 22:09:24,499 - lava_callback - ERROR - Error processing callback for node 694085be575bc060ce15dd7d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085be575bc060ce15dd7d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085be575bc060ce15dd7d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085be575bc060ce15dd7d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:24,501 - lava_callback - INFO - Starting async job submit for node 694085c9575bc060ce15dda7 2025-12-17 22:09:24,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:24,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c9575bc060ce15dda7 HTTP/1.1" 502 150 2025-12-17 22:09:24,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c9575bc060ce15dda7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:24,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c9575bc060ce15dda7 2025-12-17 22:09:24,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c9575bc060ce15dda7 HTTP/1.1" 502 150 2025-12-17 22:09:24,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c9575bc060ce15dda7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:25,092 - lava_callback - INFO - Received callback for node 69408703575bc060ce15e23e INFO: 172.21.0.1:57446 - "POST /node/69408703575bc060ce15e23e HTTP/1.0" 202 Accepted 2025-12-17 22:09:25,747 - lava_callback - INFO - Received callback for node 69408704575bc060ce15e241 INFO: 172.21.0.1:57452 - "POST /node/69408704575bc060ce15e241 HTTP/1.0" 202 Accepted 2025-12-17 22:09:26,377 - lava_callback - INFO - Received callback for node 69408707575bc060ce15e247 INFO: 172.21.0.1:57458 - "POST /node/69408707575bc060ce15e247 HTTP/1.0" 202 Accepted 2025-12-17 22:09:26,388 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d0575bc060ce15ddb6 INFO: 172.21.0.1:57462 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:09:26,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c9575bc060ce15dda7 2025-12-17 22:09:26,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c9575bc060ce15dda7 HTTP/1.1" 502 150 2025-12-17 22:09:26,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c9575bc060ce15dda7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:26,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694084ae575bc060ce15d873 HTTP/1.1" 502 150 2025-12-17 22:09:26,960 - lava_callback - ERROR - Error processing callback for node 694084ae575bc060ce15d873: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ae575bc060ce15d873 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ae575bc060ce15d873 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694084ae575bc060ce15d873 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:26,962 - lava_callback - INFO - Starting async job submit for node 694085ca575bc060ce15ddaa 2025-12-17 22:09:26,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:26,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ca575bc060ce15ddaa HTTP/1.1" 502 150 2025-12-17 22:09:26,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ca575bc060ce15ddaa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:26,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ca575bc060ce15ddaa 2025-12-17 22:09:26,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ca575bc060ce15ddaa HTTP/1.1" 502 150 2025-12-17 22:09:26,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ca575bc060ce15ddaa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:27,013 - lava_callback - INFO - Received callback for node 6940870a575bc060ce15e24d INFO: 172.21.0.1:57464 - "POST /node/6940870a575bc060ce15e24d HTTP/1.0" 202 Accepted 2025-12-17 22:09:27,656 - lava_callback - INFO - Received callback for node 6940870e575bc060ce15e256 INFO: 172.21.0.1:57478 - "POST /node/6940870e575bc060ce15e256 HTTP/1.0" 202 Accepted 2025-12-17 22:09:28,284 - lava_callback - INFO - Received callback for node 6940870f575bc060ce15e259 INFO: 172.21.0.1:57488 - "POST /node/6940870f575bc060ce15e259 HTTP/1.0" 202 Accepted 2025-12-17 22:09:28,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408598575bc060ce15dca7 HTTP/1.1" 502 150 2025-12-17 22:09:28,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408598575bc060ce15dca7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:28,913 - lava_callback - INFO - Received callback for node 69408710575bc060ce15e25c INFO: 172.21.0.1:57490 - "POST /node/69408710575bc060ce15e25c HTTP/1.0" 202 Accepted 2025-12-17 22:09:28,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085bf575bc060ce15dd88 2025-12-17 22:09:28,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bf575bc060ce15dd88 HTTP/1.1" 502 150 2025-12-17 22:09:28,953 - lava_callback - ERROR - Error processing callback for node 694085bf575bc060ce15dd88: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085bf575bc060ce15dd88 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085bf575bc060ce15dd88 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085bf575bc060ce15dd88 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:28,954 - lava_callback - INFO - Starting async job submit for node 694085cf575bc060ce15ddb3 2025-12-17 22:09:28,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:28,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cf575bc060ce15ddb3 HTTP/1.1" 502 150 2025-12-17 22:09:28,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085cf575bc060ce15ddb3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:28,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085cf575bc060ce15ddb3 2025-12-17 22:09:28,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cf575bc060ce15ddb3 HTTP/1.1" 502 150 2025-12-17 22:09:28,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085cf575bc060ce15ddb3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:28,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ca575bc060ce15ddaa 2025-12-17 22:09:28,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ca575bc060ce15ddaa HTTP/1.1" 502 150 2025-12-17 22:09:28,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ca575bc060ce15ddaa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:29,328 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c1575bc060ce15dd8f 2025-12-17 22:09:29,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c1575bc060ce15dd8f HTTP/1.1" 502 150 2025-12-17 22:09:29,329 - lava_callback - ERROR - Error processing callback for node 694085c1575bc060ce15dd8f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c1575bc060ce15dd8f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c1575bc060ce15dd8f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c1575bc060ce15dd8f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:29,330 - lava_callback - INFO - Starting async job submit for node 694085d3575bc060ce15ddc5 2025-12-17 22:09:29,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:29,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d3575bc060ce15ddc5 HTTP/1.1" 502 150 2025-12-17 22:09:29,338 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d3575bc060ce15ddc5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:29,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d3575bc060ce15ddc5 2025-12-17 22:09:29,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d3575bc060ce15ddc5 HTTP/1.1" 502 150 2025-12-17 22:09:29,339 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d3575bc060ce15ddc5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:29,553 - lava_callback - INFO - Received callback for node 69408712575bc060ce15e25f INFO: 172.21.0.1:57498 - "POST /node/69408712575bc060ce15e25f HTTP/1.0" 202 Accepted 2025-12-17 22:09:29,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d0575bc060ce15ddb9 2025-12-17 22:09:29,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d0575bc060ce15ddb9 HTTP/1.1" 502 150 2025-12-17 22:09:29,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d0575bc060ce15ddb9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:30,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c9575bc060ce15dda7 2025-12-17 22:09:30,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c9575bc060ce15dda7 HTTP/1.1" 502 150 2025-12-17 22:09:30,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c9575bc060ce15dda7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:30,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c2575bc060ce15dd96 2025-12-17 22:09:30,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c2575bc060ce15dd96 HTTP/1.1" 502 150 2025-12-17 22:09:30,891 - lava_callback - ERROR - Error processing callback for node 694085c2575bc060ce15dd96: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c2575bc060ce15dd96 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c2575bc060ce15dd96 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c2575bc060ce15dd96 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:30,893 - lava_callback - INFO - Starting async job submit for node 694085d5575bc060ce15ddcc 2025-12-17 22:09:30,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:30,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d5575bc060ce15ddcc HTTP/1.1" 502 150 2025-12-17 22:09:30,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d5575bc060ce15ddcc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:30,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d5575bc060ce15ddcc 2025-12-17 22:09:30,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d5575bc060ce15ddcc HTTP/1.1" 502 150 2025-12-17 22:09:30,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d5575bc060ce15ddcc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:30,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d2575bc060ce15ddbe 2025-12-17 22:09:30,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d2575bc060ce15ddbe HTTP/1.1" 502 150 2025-12-17 22:09:30,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d2575bc060ce15ddbe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:30,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085cf575bc060ce15ddb3 2025-12-17 22:09:30,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cf575bc060ce15ddb3 HTTP/1.1" 502 150 2025-12-17 22:09:30,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085cf575bc060ce15ddb3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:31,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c4575bc060ce15dd9b 2025-12-17 22:09:31,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c4575bc060ce15dd9b HTTP/1.1" 502 150 2025-12-17 22:09:31,001 - lava_callback - ERROR - Error processing callback for node 694085c4575bc060ce15dd9b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c4575bc060ce15dd9b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c4575bc060ce15dd9b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c4575bc060ce15dd9b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:31,003 - lava_callback - INFO - Starting async job submit for node 694085d6575bc060ce15ddd3 2025-12-17 22:09:31,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:31,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d6575bc060ce15ddd3 HTTP/1.1" 502 150 2025-12-17 22:09:31,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d6575bc060ce15ddd3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:31,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d6575bc060ce15ddd3 2025-12-17 22:09:31,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d6575bc060ce15ddd3 HTTP/1.1" 502 150 2025-12-17 22:09:31,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d6575bc060ce15ddd3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:31,339 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d3575bc060ce15ddc5 2025-12-17 22:09:31,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d3575bc060ce15ddc5 HTTP/1.1" 502 150 2025-12-17 22:09:31,340 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d3575bc060ce15ddc5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:31,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085cc575bc060ce15ddad 2025-12-17 22:09:31,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cc575bc060ce15ddad HTTP/1.1" 502 150 2025-12-17 22:09:31,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085cc575bc060ce15ddad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:32,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408598575bc060ce15dca7 2025-12-17 22:09:32,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d5575bc060ce15ddcc 2025-12-17 22:09:32,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ca575bc060ce15ddaa 2025-12-17 22:09:32,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ca575bc060ce15ddaa HTTP/1.1" 502 150 2025-12-17 22:09:32,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ca575bc060ce15ddaa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:33,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d6575bc060ce15ddd3 2025-12-17 22:09:33,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d6575bc060ce15ddd3 HTTP/1.1" 502 150 2025-12-17 22:09:33,037 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d6575bc060ce15ddd3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:33,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c4575bc060ce15dd9e 2025-12-17 22:09:33,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c4575bc060ce15dd9e HTTP/1.1" 502 150 2025-12-17 22:09:33,751 - lava_callback - ERROR - Error processing callback for node 694085c4575bc060ce15dd9e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c4575bc060ce15dd9e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c4575bc060ce15dd9e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c4575bc060ce15dd9e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:33,753 - lava_callback - INFO - Starting async job submit for node 694085d7575bc060ce15ddda 2025-12-17 22:09:33,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:33,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d7575bc060ce15ddda HTTP/1.1" 502 150 2025-12-17 22:09:33,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d7575bc060ce15ddda'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:33,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d7575bc060ce15ddda 2025-12-17 22:09:33,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d7575bc060ce15ddda HTTP/1.1" 502 150 2025-12-17 22:09:33,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d7575bc060ce15ddda'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:34,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085cf575bc060ce15ddb3 2025-12-17 22:09:34,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cf575bc060ce15ddb3 HTTP/1.1" 502 150 2025-12-17 22:09:34,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085cf575bc060ce15ddb3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:35,341 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d3575bc060ce15ddc5 2025-12-17 22:09:35,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d3575bc060ce15ddc5 HTTP/1.1" 502 150 2025-12-17 22:09:35,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d3575bc060ce15ddc5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:35,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d7575bc060ce15ddda 2025-12-17 22:09:35,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d7575bc060ce15ddda HTTP/1.1" 502 150 2025-12-17 22:09:35,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d7575bc060ce15ddda'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:36,188 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c6575bc060ce15dda1 2025-12-17 22:09:36,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c6575bc060ce15dda1 HTTP/1.1" 502 150 2025-12-17 22:09:36,189 - lava_callback - ERROR - Error processing callback for node 694085c6575bc060ce15dda1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c6575bc060ce15dda1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c6575bc060ce15dda1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c6575bc060ce15dda1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:36,192 - lava_callback - INFO - Starting async job submit for node 694085d9575bc060ce15dde1 2025-12-17 22:09:36,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:36,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d9575bc060ce15dde1 HTTP/1.1" 502 150 2025-12-17 22:09:36,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d9575bc060ce15dde1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:36,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d9575bc060ce15dde1 2025-12-17 22:09:36,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d9575bc060ce15dde1 HTTP/1.1" 502 150 2025-12-17 22:09:36,201 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d9575bc060ce15dde1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:36,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5331e3c24e95681d6a4 HTTP/1.1" 502 150 2025-12-17 22:09:36,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5331e3c24e95681d6a4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53022 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:09:36,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c8575bc060ce15dda4 2025-12-17 22:09:36,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c8575bc060ce15dda4 HTTP/1.1" 502 150 2025-12-17 22:09:36,816 - lava_callback - ERROR - Error processing callback for node 694085c8575bc060ce15dda4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c8575bc060ce15dda4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c8575bc060ce15dda4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c8575bc060ce15dda4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:36,819 - lava_callback - INFO - Starting async job submit for node 694085da575bc060ce15dde4 2025-12-17 22:09:36,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:36,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085da575bc060ce15dde4 HTTP/1.1" 502 150 2025-12-17 22:09:36,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085da575bc060ce15dde4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:36,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085da575bc060ce15dde4 2025-12-17 22:09:36,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085da575bc060ce15dde4 HTTP/1.1" 502 150 2025-12-17 22:09:36,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085da575bc060ce15dde4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:37,037 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d6575bc060ce15ddd3 2025-12-17 22:09:38,201 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d9575bc060ce15dde1 2025-12-17 22:09:38,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d9575bc060ce15dde1 HTTP/1.1" 502 150 2025-12-17 22:09:38,204 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d9575bc060ce15dde1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:38,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c9575bc060ce15dda7 2025-12-17 22:09:38,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c9575bc060ce15dda7 HTTP/1.1" 502 150 2025-12-17 22:09:38,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085c9575bc060ce15dda7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:38,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085da575bc060ce15dde4 2025-12-17 22:09:38,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085da575bc060ce15dde4 HTTP/1.1" 502 150 2025-12-17 22:09:38,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085da575bc060ce15dde4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:39,078 - lava_callback - INFO - Received callback for node 6940871a575bc060ce15e27e INFO: 172.21.0.1:53024 - "POST /node/6940871a575bc060ce15e27e HTTP/1.0" 202 Accepted 2025-12-17 22:09:39,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085bb575bc060ce15dd70 2025-12-17 22:09:39,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085bb575bc060ce15dd70 HTTP/1.1" 502 150 2025-12-17 22:09:39,443 - lava_callback - ERROR - Error processing callback for node 694085bb575bc060ce15dd70: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085bb575bc060ce15dd70 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085bb575bc060ce15dd70 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085bb575bc060ce15dd70 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:39,445 - lava_callback - INFO - Starting async job submit for node 694085db575bc060ce15dde7 2025-12-17 22:09:39,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:39,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085db575bc060ce15dde7 HTTP/1.1" 502 150 2025-12-17 22:09:39,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085db575bc060ce15dde7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:39,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085db575bc060ce15dde7 2025-12-17 22:09:39,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085db575bc060ce15dde7 HTTP/1.1" 502 150 2025-12-17 22:09:39,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085db575bc060ce15dde7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:39,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d7575bc060ce15ddda 2025-12-17 22:09:39,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d7575bc060ce15ddda HTTP/1.1" 502 150 2025-12-17 22:09:39,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d7575bc060ce15ddda'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:39,889 - lava_callback - INFO - Received callback for node 6940871e575bc060ce15e288 INFO: 172.21.0.1:38418 - "POST /node/6940871e575bc060ce15e288 HTTP/1.0" 202 Accepted 2025-12-17 22:09:40,719 - lava_callback - INFO - Received callback for node 69408721575bc060ce15e28e INFO: 172.21.0.1:38428 - "POST /node/69408721575bc060ce15e28e HTTP/1.0" 202 Accepted 2025-12-17 22:09:40,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ca575bc060ce15ddaa 2025-12-17 22:09:40,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ca575bc060ce15ddaa HTTP/1.1" 502 150 2025-12-17 22:09:40,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ca575bc060ce15ddaa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:41,444 - lava_callback - INFO - Received callback for node 69408722575bc060ce15e291 INFO: 172.21.0.1:38438 - "POST /node/69408722575bc060ce15e291 HTTP/1.0" 202 Accepted 2025-12-17 22:09:41,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085db575bc060ce15dde7 2025-12-17 22:09:41,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085db575bc060ce15dde7 HTTP/1.1" 502 150 2025-12-17 22:09:41,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085db575bc060ce15dde7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:41,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d0575bc060ce15ddb6 HTTP/1.1" 502 150 2025-12-17 22:09:41,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d0575bc060ce15ddb6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:42,204 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d9575bc060ce15dde1 2025-12-17 22:09:42,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d9575bc060ce15dde1 HTTP/1.1" 502 150 2025-12-17 22:09:42,206 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d9575bc060ce15dde1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:42,214 - lava_callback - INFO - Received callback for node 69408723575bc060ce15e294 INFO: 172.21.0.1:38452 - "POST /node/69408723575bc060ce15e294 HTTP/1.0" 202 Accepted 2025-12-17 22:09:42,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085da575bc060ce15dde4 2025-12-17 22:09:42,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085da575bc060ce15dde4 HTTP/1.1" 502 150 2025-12-17 22:09:42,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085da575bc060ce15dde4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:42,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085cf575bc060ce15ddb3 2025-12-17 22:09:42,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cf575bc060ce15ddb3 HTTP/1.1" 502 150 2025-12-17 22:09:42,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085cf575bc060ce15ddb3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:43,031 - lava_callback - INFO - Received callback for node 69408725575bc060ce15e297 INFO: 172.21.0.1:38464 - "POST /node/69408725575bc060ce15e297 HTTP/1.0" 202 Accepted 2025-12-17 22:09:43,342 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d3575bc060ce15ddc5 2025-12-17 22:09:43,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d3575bc060ce15ddc5 HTTP/1.1" 502 150 2025-12-17 22:09:43,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d3575bc060ce15ddc5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:43,823 - lava_callback - INFO - Received callback for node 69408713575bc060ce15e266 INFO: 172.21.0.1:38478 - "POST /node/69408713575bc060ce15e266 HTTP/1.0" 202 Accepted 2025-12-17 22:09:44,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5331e3c24e95681d6a4 2025-12-17 22:09:44,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5331e3c24e95681d6a4 HTTP/1.1" 502 150 2025-12-17 22:09:44,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5331e3c24e95681d6a4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:44,485 - lava_callback - INFO - Received callback for node 69408714575bc060ce15e26d INFO: 172.21.0.1:38482 - "POST /node/69408714575bc060ce15e26d HTTP/1.0" 202 Accepted 2025-12-17 22:09:45,120 - lava_callback - INFO - Received callback for node 69408716575bc060ce15e270 INFO: 172.21.0.1:38486 - "POST /node/69408716575bc060ce15e270 HTTP/1.0" 202 Accepted 2025-12-17 22:09:45,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085db575bc060ce15dde7 2025-12-17 22:09:45,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085db575bc060ce15dde7 HTTP/1.1" 502 150 2025-12-17 22:09:45,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085db575bc060ce15dde7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:45,754 - lava_callback - INFO - Received callback for node 69408718575bc060ce15e27b INFO: 172.21.0.1:38502 - "POST /node/69408718575bc060ce15e27b HTTP/1.0" 202 Accepted 2025-12-17 22:09:45,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d0575bc060ce15ddb6 2025-12-17 22:09:45,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d0575bc060ce15ddb6 HTTP/1.1" 502 150 2025-12-17 22:09:45,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d0575bc060ce15ddb6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:45,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d0575bc060ce15ddb9 2025-12-17 22:09:45,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d0575bc060ce15ddb9 HTTP/1.1" 502 150 2025-12-17 22:09:45,916 - lava_callback - ERROR - Error processing callback for node 694085d0575bc060ce15ddb9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d0575bc060ce15ddb9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d0575bc060ce15ddb9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d0575bc060ce15ddb9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:45,918 - lava_callback - INFO - Starting async job submit for node 694085dc575bc060ce15ddea 2025-12-17 22:09:45,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:45,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085dc575bc060ce15ddea HTTP/1.1" 502 150 2025-12-17 22:09:45,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085dc575bc060ce15ddea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:45,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085dc575bc060ce15ddea 2025-12-17 22:09:45,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085dc575bc060ce15ddea HTTP/1.1" 502 150 2025-12-17 22:09:45,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085dc575bc060ce15ddea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:46,420 - lava_callback - INFO - Received callback for node 6940871b575bc060ce15e285 INFO: 172.21.0.1:38504 - "POST /node/6940871b575bc060ce15e285 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:38508 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:09:46,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d2575bc060ce15ddbe 2025-12-17 22:09:46,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d2575bc060ce15ddbe HTTP/1.1" 502 150 2025-12-17 22:09:46,939 - lava_callback - ERROR - Error processing callback for node 694085d2575bc060ce15ddbe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d2575bc060ce15ddbe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d2575bc060ce15ddbe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d2575bc060ce15ddbe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:46,941 - lava_callback - INFO - Starting async job submit for node 694085dd575bc060ce15ddf1 2025-12-17 22:09:46,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:46,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085dd575bc060ce15ddf1 HTTP/1.1" 502 150 2025-12-17 22:09:46,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085dd575bc060ce15ddf1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:46,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085dd575bc060ce15ddf1 2025-12-17 22:09:46,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085dd575bc060ce15ddf1 HTTP/1.1" 502 150 2025-12-17 22:09:46,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085dd575bc060ce15ddf1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:47,092 - lava_callback - INFO - Received callback for node 6940871e575bc060ce15e28b INFO: 172.21.0.1:38518 - "POST /node/6940871e575bc060ce15e28b HTTP/1.0" 202 Accepted 2025-12-17 22:09:47,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085cc575bc060ce15ddad 2025-12-17 22:09:47,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d7575bc060ce15ddda 2025-12-17 22:09:47,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d7575bc060ce15ddda HTTP/1.1" 502 150 2025-12-17 22:09:47,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d7575bc060ce15ddda'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:47,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085dc575bc060ce15ddea 2025-12-17 22:09:47,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085dc575bc060ce15ddea HTTP/1.1" 502 150 2025-12-17 22:09:47,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085dc575bc060ce15ddea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:48,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d5575bc060ce15ddcc HTTP/1.1" 502 150 2025-12-17 22:09:48,208 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d5575bc060ce15ddcc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:48,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085dd575bc060ce15ddf1 2025-12-17 22:09:48,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085dd575bc060ce15ddf1 HTTP/1.1" 502 150 2025-12-17 22:09:48,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085dd575bc060ce15ddf1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:50,206 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d9575bc060ce15dde1 2025-12-17 22:09:50,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d9575bc060ce15dde1 HTTP/1.1" 502 150 2025-12-17 22:09:50,208 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d9575bc060ce15dde1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:50,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085da575bc060ce15dde4 2025-12-17 22:09:50,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085da575bc060ce15dde4 HTTP/1.1" 502 150 2025-12-17 22:09:50,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085da575bc060ce15dde4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:51,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085dc575bc060ce15ddea 2025-12-17 22:09:51,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085dc575bc060ce15ddea HTTP/1.1" 502 150 2025-12-17 22:09:51,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085dc575bc060ce15ddea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:52,209 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d5575bc060ce15ddcc 2025-12-17 22:09:52,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d5575bc060ce15ddcc HTTP/1.1" 502 150 2025-12-17 22:09:52,210 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d5575bc060ce15ddcc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:52,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d6575bc060ce15ddd3 HTTP/1.1" 502 150 2025-12-17 22:09:52,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d6575bc060ce15ddd3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:52,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085dd575bc060ce15ddf1 2025-12-17 22:09:52,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085dd575bc060ce15ddf1 HTTP/1.1" 502 150 2025-12-17 22:09:52,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085dd575bc060ce15ddf1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:53,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085db575bc060ce15dde7 2025-12-17 22:09:53,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085db575bc060ce15dde7 HTTP/1.1" 502 150 2025-12-17 22:09:53,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085db575bc060ce15dde7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:53,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d0575bc060ce15ddb6 2025-12-17 22:09:53,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d0575bc060ce15ddb6 HTTP/1.1" 502 150 2025-12-17 22:09:53,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d0575bc060ce15ddb6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:53,988 - lava_callback - INFO - Received callback for node 69408728575bc060ce15e2a5 INFO: 172.21.0.1:50562 - "POST /node/69408728575bc060ce15e2a5 HTTP/1.0" 202 Accepted 2025-12-17 22:09:54,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085c9575bc060ce15dda7 2025-12-17 22:09:54,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085c9575bc060ce15dda7 HTTP/1.1" 502 150 2025-12-17 22:09:54,541 - lava_callback - ERROR - Error processing callback for node 694085c9575bc060ce15dda7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c9575bc060ce15dda7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c9575bc060ce15dda7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085c9575bc060ce15dda7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:54,543 - lava_callback - INFO - Starting async job submit for node 694085df575bc060ce15ddf4 2025-12-17 22:09:54,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:54,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085df575bc060ce15ddf4 HTTP/1.1" 502 150 2025-12-17 22:09:54,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085df575bc060ce15ddf4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:54,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085df575bc060ce15ddf4 2025-12-17 22:09:54,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085df575bc060ce15ddf4 HTTP/1.1" 502 150 2025-12-17 22:09:54,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085df575bc060ce15ddf4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:54,657 - lava_callback - INFO - Received callback for node 6940872c575bc060ce15e2d6 INFO: 172.21.0.1:50564 - "POST /node/6940872c575bc060ce15e2d6 HTTP/1.0" 202 Accepted 2025-12-17 22:09:55,361 - lava_callback - INFO - Received callback for node 6940872e575bc060ce15e2e4 INFO: 172.21.0.1:50568 - "POST /node/6940872e575bc060ce15e2e4 HTTP/1.0" 202 Accepted 2025-12-17 22:09:56,025 - lava_callback - INFO - Received callback for node 69408730575bc060ce15e2e7 INFO: 172.21.0.1:50572 - "POST /node/69408730575bc060ce15e2e7 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:50582 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:09:56,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085df575bc060ce15ddf4 2025-12-17 22:09:56,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085df575bc060ce15ddf4 HTTP/1.1" 502 150 2025-12-17 22:09:56,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085df575bc060ce15ddf4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:56,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ca575bc060ce15ddaa 2025-12-17 22:09:56,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ca575bc060ce15ddaa HTTP/1.1" 502 150 2025-12-17 22:09:56,975 - lava_callback - ERROR - Error processing callback for node 694085ca575bc060ce15ddaa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085ca575bc060ce15ddaa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085ca575bc060ce15ddaa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085ca575bc060ce15ddaa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:56,977 - lava_callback - INFO - Starting async job submit for node 694085e9575bc060ce15de0f 2025-12-17 22:09:56,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:56,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e9575bc060ce15de0f HTTP/1.1" 502 150 2025-12-17 22:09:56,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e9575bc060ce15de0f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:56,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e9575bc060ce15de0f 2025-12-17 22:09:56,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e9575bc060ce15de0f HTTP/1.1" 502 150 2025-12-17 22:09:56,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e9575bc060ce15de0f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:58,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085cf575bc060ce15ddb3 2025-12-17 22:09:58,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cf575bc060ce15ddb3 HTTP/1.1" 502 150 2025-12-17 22:09:58,969 - lava_callback - ERROR - Error processing callback for node 694085cf575bc060ce15ddb3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085cf575bc060ce15ddb3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085cf575bc060ce15ddb3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085cf575bc060ce15ddb3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:09:58,970 - lava_callback - INFO - Starting async job submit for node 694085e5575bc060ce15ddf7 2025-12-17 22:09:58,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:09:58,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e9575bc060ce15de0f 2025-12-17 22:09:58,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e5575bc060ce15ddf7 HTTP/1.1" 502 150 2025-12-17 22:09:58,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e5575bc060ce15ddf7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:58,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e5575bc060ce15ddf7 2025-12-17 22:09:58,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e9575bc060ce15de0f HTTP/1.1" 502 150 2025-12-17 22:09:58,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e9575bc060ce15de0f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:58,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e5575bc060ce15ddf7 HTTP/1.1" 502 150 2025-12-17 22:09:58,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e5575bc060ce15ddf7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:09:59,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d3575bc060ce15ddc5 2025-12-17 22:09:59,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085dc575bc060ce15ddea 2025-12-17 22:09:59,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085dc575bc060ce15ddea HTTP/1.1" 502 150 2025-12-17 22:09:59,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085dc575bc060ce15ddea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:00,210 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d5575bc060ce15ddcc 2025-12-17 22:10:00,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d5575bc060ce15ddcc HTTP/1.1" 502 150 2025-12-17 22:10:00,211 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d5575bc060ce15ddcc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:00,305 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d6575bc060ce15ddd3 2025-12-17 22:10:00,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d6575bc060ce15ddd3 HTTP/1.1" 502 150 2025-12-17 22:10:00,306 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085d6575bc060ce15ddd3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:00,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5331e3c24e95681d6a4 2025-12-17 22:10:00,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5331e3c24e95681d6a4 HTTP/1.1" 502 150 2025-12-17 22:10:00,445 - lava_callback - ERROR - Error processing callback for node 6940c5331e3c24e95681d6a4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5331e3c24e95681d6a4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5331e3c24e95681d6a4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5331e3c24e95681d6a4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:00,447 - lava_callback - INFO - Starting async job submit for node 694085e6575bc060ce15ddfc 2025-12-17 22:10:00,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:00,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e6575bc060ce15ddfc HTTP/1.1" 502 150 2025-12-17 22:10:00,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e6575bc060ce15ddfc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:00,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e6575bc060ce15ddfc 2025-12-17 22:10:00,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e6575bc060ce15ddfc HTTP/1.1" 502 150 2025-12-17 22:10:00,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e6575bc060ce15ddfc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:00,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085df575bc060ce15ddf4 2025-12-17 22:10:00,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085df575bc060ce15ddf4 HTTP/1.1" 502 150 2025-12-17 22:10:00,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085df575bc060ce15ddf4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:00,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085dd575bc060ce15ddf1 2025-12-17 22:10:00,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085dd575bc060ce15ddf1 HTTP/1.1" 502 150 2025-12-17 22:10:00,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085dd575bc060ce15ddf1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:00,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e5575bc060ce15ddf7 2025-12-17 22:10:00,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e5575bc060ce15ddf7 HTTP/1.1" 502 150 2025-12-17 22:10:00,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e5575bc060ce15ddf7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47346 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:10:02,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e6575bc060ce15ddfc 2025-12-17 22:10:02,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e6575bc060ce15ddfc HTTP/1.1" 502 150 2025-12-17 22:10:02,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e6575bc060ce15ddfc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:02,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cc575bc060ce15ddad HTTP/1.1" 502 150 2025-12-17 22:10:02,800 - lava_callback - ERROR - Error processing callback for node 694085cc575bc060ce15ddad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085cc575bc060ce15ddad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085cc575bc060ce15ddad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085cc575bc060ce15ddad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:02,802 - lava_callback - INFO - Starting async job submit for node 694085e7575bc060ce15de03 2025-12-17 22:10:02,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:02,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e7575bc060ce15de03 HTTP/1.1" 502 150 2025-12-17 22:10:02,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e7575bc060ce15de03'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:02,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e7575bc060ce15de03 2025-12-17 22:10:02,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e7575bc060ce15de03 HTTP/1.1" 502 150 2025-12-17 22:10:02,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e7575bc060ce15de03'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:02,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e9575bc060ce15de0f 2025-12-17 22:10:02,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e9575bc060ce15de0f HTTP/1.1" 502 150 2025-12-17 22:10:02,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e9575bc060ce15de0f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:03,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408598575bc060ce15dca7 HTTP/1.1" 502 150 2025-12-17 22:10:03,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408598575bc060ce15dca7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:03,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d7575bc060ce15ddda 2025-12-17 22:10:03,825 - lava_callback - INFO - Received callback for node 69408733575bc060ce15e2ed INFO: 172.21.0.1:47352 - "POST /node/69408733575bc060ce15e2ed HTTP/1.0" 202 Accepted 2025-12-17 22:10:04,467 - lava_callback - INFO - Received callback for node 69408735575bc060ce15e2f0 INFO: 172.21.0.1:47362 - "POST /node/69408735575bc060ce15e2f0 HTTP/1.0" 202 Accepted 2025-12-17 22:10:04,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e7575bc060ce15de03 2025-12-17 22:10:04,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e7575bc060ce15de03 HTTP/1.1" 502 150 2025-12-17 22:10:04,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e7575bc060ce15de03'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:04,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e5575bc060ce15ddf7 2025-12-17 22:10:04,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e5575bc060ce15ddf7 HTTP/1.1" 502 150 2025-12-17 22:10:04,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e5575bc060ce15ddf7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:05,155 - lava_callback - INFO - Received callback for node 69408736575bc060ce15e2f3 INFO: 172.21.0.1:47370 - "POST /node/69408736575bc060ce15e2f3 HTTP/1.0" 202 Accepted 2025-12-17 22:10:05,800 - lava_callback - INFO - Received callback for node 69408739575bc060ce15e2fd INFO: 172.21.0.1:47384 - "POST /node/69408739575bc060ce15e2fd HTTP/1.0" 202 Accepted 2025-12-17 22:10:06,208 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d9575bc060ce15dde1 2025-12-17 22:10:06,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d9575bc060ce15dde1 HTTP/1.1" 502 150 2025-12-17 22:10:06,210 - lava_callback - ERROR - Error processing callback for node 694085d9575bc060ce15dde1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d9575bc060ce15dde1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d9575bc060ce15dde1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d9575bc060ce15dde1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:06,212 - lava_callback - INFO - Starting async job submit for node 694085e9575bc060ce15de0a 2025-12-17 22:10:06,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:06,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e9575bc060ce15de0a HTTP/1.1" 502 150 2025-12-17 22:10:06,228 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e9575bc060ce15de0a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:06,228 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e9575bc060ce15de0a 2025-12-17 22:10:06,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e9575bc060ce15de0a HTTP/1.1" 502 150 2025-12-17 22:10:06,229 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e9575bc060ce15de0a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:06,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e6575bc060ce15ddfc 2025-12-17 22:10:06,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e6575bc060ce15ddfc HTTP/1.1" 502 150 2025-12-17 22:10:06,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e6575bc060ce15ddfc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:06,466 - lava_callback - INFO - Received callback for node 6940873a575bc060ce15e300 INFO: 172.21.0.1:47394 - "POST /node/6940873a575bc060ce15e300 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:47406 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:10:06,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085da575bc060ce15dde4 2025-12-17 22:10:06,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085da575bc060ce15dde4 HTTP/1.1" 502 150 2025-12-17 22:10:06,843 - lava_callback - ERROR - Error processing callback for node 694085da575bc060ce15dde4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085da575bc060ce15dde4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085da575bc060ce15dde4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085da575bc060ce15dde4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:06,845 - lava_callback - INFO - Starting async job submit for node 694085f2575bc060ce15de33 2025-12-17 22:10:06,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:06,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f2575bc060ce15de33 HTTP/1.1" 502 150 2025-12-17 22:10:06,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f2575bc060ce15de33'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:06,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f2575bc060ce15de33 2025-12-17 22:10:06,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f2575bc060ce15de33 HTTP/1.1" 502 150 2025-12-17 22:10:06,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f2575bc060ce15de33'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:08,229 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e9575bc060ce15de0a 2025-12-17 22:10:08,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e9575bc060ce15de0a HTTP/1.1" 502 150 2025-12-17 22:10:08,230 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e9575bc060ce15de0a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:08,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085df575bc060ce15ddf4 2025-12-17 22:10:08,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085df575bc060ce15ddf4 HTTP/1.1" 502 150 2025-12-17 22:10:08,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085df575bc060ce15ddf4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:08,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e7575bc060ce15de03 2025-12-17 22:10:08,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e7575bc060ce15de03 HTTP/1.1" 502 150 2025-12-17 22:10:08,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e7575bc060ce15de03'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:08,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f2575bc060ce15de33 2025-12-17 22:10:08,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f2575bc060ce15de33 HTTP/1.1" 502 150 2025-12-17 22:10:08,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f2575bc060ce15de33'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:09,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085db575bc060ce15dde7 2025-12-17 22:10:09,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085db575bc060ce15dde7 HTTP/1.1" 502 150 2025-12-17 22:10:09,458 - lava_callback - ERROR - Error processing callback for node 694085db575bc060ce15dde7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085db575bc060ce15dde7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085db575bc060ce15dde7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085db575bc060ce15dde7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:09,461 - lava_callback - INFO - Starting async job submit for node 694085f2575bc060ce15de36 2025-12-17 22:10:09,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:09,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f2575bc060ce15de36 HTTP/1.1" 502 150 2025-12-17 22:10:09,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f2575bc060ce15de36'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:09,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f2575bc060ce15de36 2025-12-17 22:10:09,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f2575bc060ce15de36 HTTP/1.1" 502 150 2025-12-17 22:10:09,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f2575bc060ce15de36'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:09,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d0575bc060ce15ddb6 2025-12-17 22:10:09,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d0575bc060ce15ddb6 HTTP/1.1" 502 150 2025-12-17 22:10:09,813 - lava_callback - ERROR - Error processing callback for node 694085d0575bc060ce15ddb6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d0575bc060ce15ddb6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d0575bc060ce15ddb6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d0575bc060ce15ddb6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:09,815 - lava_callback - INFO - Starting async job submit for node 694085eb575bc060ce15de16 2025-12-17 22:10:09,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:09,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085eb575bc060ce15de16 HTTP/1.1" 502 150 2025-12-17 22:10:09,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085eb575bc060ce15de16'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:09,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085eb575bc060ce15de16 2025-12-17 22:10:09,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085eb575bc060ce15de16 HTTP/1.1" 502 150 2025-12-17 22:10:09,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085eb575bc060ce15de16'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:10,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e9575bc060ce15de0f 2025-12-17 22:10:10,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e9575bc060ce15de0f HTTP/1.1" 502 150 2025-12-17 22:10:10,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e9575bc060ce15de0f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:11,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f2575bc060ce15de36 2025-12-17 22:10:11,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f2575bc060ce15de36 HTTP/1.1" 502 150 2025-12-17 22:10:11,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f2575bc060ce15de36'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:11,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408598575bc060ce15dca7 2025-12-17 22:10:11,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085eb575bc060ce15de16 2025-12-17 22:10:11,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085eb575bc060ce15de16 HTTP/1.1" 502 150 2025-12-17 22:10:11,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085eb575bc060ce15de16'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:12,231 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e9575bc060ce15de0a 2025-12-17 22:10:12,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e9575bc060ce15de0a HTTP/1.1" 502 150 2025-12-17 22:10:12,232 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e9575bc060ce15de0a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:12,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f2575bc060ce15de33 2025-12-17 22:10:12,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f2575bc060ce15de33 HTTP/1.1" 502 150 2025-12-17 22:10:12,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f2575bc060ce15de33'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:12,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e5575bc060ce15ddf7 2025-12-17 22:10:12,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e5575bc060ce15ddf7 HTTP/1.1" 502 150 2025-12-17 22:10:12,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e5575bc060ce15ddf7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:13,987 - lava_callback - INFO - Received callback for node 69408732575bc060ce15e2ea INFO: 172.21.0.1:49572 - "POST /node/69408732575bc060ce15e2ea HTTP/1.0" 202 Accepted 2025-12-17 22:10:14,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e6575bc060ce15ddfc 2025-12-17 22:10:14,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e6575bc060ce15ddfc HTTP/1.1" 502 150 2025-12-17 22:10:14,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e6575bc060ce15ddfc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:14,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d3575bc060ce15ddc5 HTTP/1.1" 502 150 2025-12-17 22:10:14,833 - lava_callback - ERROR - Error processing callback for node 694085d3575bc060ce15ddc5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d3575bc060ce15ddc5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d3575bc060ce15ddc5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d3575bc060ce15ddc5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:14,834 - lava_callback - INFO - Starting async job submit for node 694085ee575bc060ce15de26 2025-12-17 22:10:14,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:14,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ee575bc060ce15de26 HTTP/1.1" 502 150 2025-12-17 22:10:14,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ee575bc060ce15de26'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:14,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ee575bc060ce15de26 2025-12-17 22:10:14,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ee575bc060ce15de26 HTTP/1.1" 502 150 2025-12-17 22:10:14,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ee575bc060ce15de26'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49574 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:10:15,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f2575bc060ce15de36 2025-12-17 22:10:15,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f2575bc060ce15de36 HTTP/1.1" 502 150 2025-12-17 22:10:15,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f2575bc060ce15de36'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:15,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085eb575bc060ce15de16 2025-12-17 22:10:15,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085eb575bc060ce15de16 HTTP/1.1" 502 150 2025-12-17 22:10:15,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085eb575bc060ce15de16'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:15,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085dc575bc060ce15ddea 2025-12-17 22:10:15,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085dc575bc060ce15ddea HTTP/1.1" 502 150 2025-12-17 22:10:15,961 - lava_callback - ERROR - Error processing callback for node 694085dc575bc060ce15ddea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085dc575bc060ce15ddea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085dc575bc060ce15ddea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085dc575bc060ce15ddea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:15,963 - lava_callback - INFO - Starting async job submit for node 694085f0575bc060ce15de2d 2025-12-17 22:10:15,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:15,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f0575bc060ce15de2d HTTP/1.1" 502 150 2025-12-17 22:10:15,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f0575bc060ce15de2d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:15,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f0575bc060ce15de2d 2025-12-17 22:10:15,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f0575bc060ce15de2d HTTP/1.1" 502 150 2025-12-17 22:10:15,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f0575bc060ce15de2d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:16,212 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d5575bc060ce15ddcc 2025-12-17 22:10:16,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d5575bc060ce15ddcc HTTP/1.1" 502 150 2025-12-17 22:10:16,213 - lava_callback - ERROR - Error processing callback for node 694085d5575bc060ce15ddcc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d5575bc060ce15ddcc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d5575bc060ce15ddcc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d5575bc060ce15ddcc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:16,216 - lava_callback - INFO - Starting async job submit for node 694085f1575bc060ce15de30 2025-12-17 22:10:16,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:16,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f1575bc060ce15de30 HTTP/1.1" 502 150 2025-12-17 22:10:16,224 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f1575bc060ce15de30'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:16,224 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f1575bc060ce15de30 2025-12-17 22:10:16,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f1575bc060ce15de30 HTTP/1.1" 502 150 2025-12-17 22:10:16,225 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f1575bc060ce15de30'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:16,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085d6575bc060ce15ddd3 2025-12-17 22:10:16,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d6575bc060ce15ddd3 HTTP/1.1" 502 150 2025-12-17 22:10:16,313 - lava_callback - ERROR - Error processing callback for node 694085d6575bc060ce15ddd3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d6575bc060ce15ddd3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d6575bc060ce15ddd3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d6575bc060ce15ddd3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:16,314 - lava_callback - INFO - Starting async job submit for node 694085f4575bc060ce15de3c 2025-12-17 22:10:16,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:16,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f4575bc060ce15de3c HTTP/1.1" 502 150 2025-12-17 22:10:16,322 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f4575bc060ce15de3c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:16,322 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f4575bc060ce15de3c 2025-12-17 22:10:16,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f4575bc060ce15de3c HTTP/1.1" 502 150 2025-12-17 22:10:16,322 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f4575bc060ce15de3c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49586 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:10:16,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e7575bc060ce15de03 2025-12-17 22:10:16,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e7575bc060ce15de03 HTTP/1.1" 502 150 2025-12-17 22:10:16,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e7575bc060ce15de03'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:16,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ee575bc060ce15de26 2025-12-17 22:10:16,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ee575bc060ce15de26 HTTP/1.1" 502 150 2025-12-17 22:10:16,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ee575bc060ce15de26'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:16,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085dd575bc060ce15ddf1 2025-12-17 22:10:16,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085dd575bc060ce15ddf1 HTTP/1.1" 502 150 2025-12-17 22:10:16,956 - lava_callback - ERROR - Error processing callback for node 694085dd575bc060ce15ddf1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085dd575bc060ce15ddf1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085dd575bc060ce15ddf1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085dd575bc060ce15ddf1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:16,958 - lava_callback - INFO - Starting async job submit for node 694085f7575bc060ce15de54 2025-12-17 22:10:16,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:16,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f7575bc060ce15de54 HTTP/1.1" 502 150 2025-12-17 22:10:16,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f7575bc060ce15de54'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:16,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f7575bc060ce15de54 2025-12-17 22:10:16,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f7575bc060ce15de54 HTTP/1.1" 502 150 2025-12-17 22:10:16,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f7575bc060ce15de54'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:17,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f0575bc060ce15de2d 2025-12-17 22:10:17,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f0575bc060ce15de2d HTTP/1.1" 502 150 2025-12-17 22:10:17,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f0575bc060ce15de2d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:18,225 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f1575bc060ce15de30 2025-12-17 22:10:18,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f1575bc060ce15de30 HTTP/1.1" 502 150 2025-12-17 22:10:18,226 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f1575bc060ce15de30'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:18,323 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f4575bc060ce15de3c 2025-12-17 22:10:18,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f4575bc060ce15de3c HTTP/1.1" 502 150 2025-12-17 22:10:18,324 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f4575bc060ce15de3c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:18,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f7575bc060ce15de54 2025-12-17 22:10:18,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f7575bc060ce15de54 HTTP/1.1" 502 150 2025-12-17 22:10:18,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f7575bc060ce15de54'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:18,995 - lava_callback - INFO - Received callback for node 69408738575bc060ce15e2f6 INFO: 172.21.0.1:49588 - "POST /node/69408738575bc060ce15e2f6 HTTP/1.0" 202 Accepted 2025-12-17 22:10:19,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085d7575bc060ce15ddda HTTP/1.1" 502 150 2025-12-17 22:10:19,184 - lava_callback - ERROR - Error processing callback for node 694085d7575bc060ce15ddda: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d7575bc060ce15ddda (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d7575bc060ce15ddda (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085d7575bc060ce15ddda (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:19,186 - lava_callback - INFO - Starting async job submit for node 694085f9575bc060ce15de83 2025-12-17 22:10:19,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:19,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f9575bc060ce15de83 HTTP/1.1" 502 150 2025-12-17 22:10:19,193 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f9575bc060ce15de83'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:19,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f9575bc060ce15de83 2025-12-17 22:10:19,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f9575bc060ce15de83 HTTP/1.1" 502 150 2025-12-17 22:10:19,193 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f9575bc060ce15de83'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:20,232 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e9575bc060ce15de0a 2025-12-17 22:10:20,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e9575bc060ce15de0a HTTP/1.1" 502 150 2025-12-17 22:10:20,233 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085e9575bc060ce15de0a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:20,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ee575bc060ce15de26 2025-12-17 22:10:20,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ee575bc060ce15de26 HTTP/1.1" 502 150 2025-12-17 22:10:20,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ee575bc060ce15de26'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:20,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f2575bc060ce15de33 2025-12-17 22:10:20,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f2575bc060ce15de33 HTTP/1.1" 502 150 2025-12-17 22:10:20,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f2575bc060ce15de33'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:21,194 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f9575bc060ce15de83 2025-12-17 22:10:21,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f9575bc060ce15de83 HTTP/1.1" 502 150 2025-12-17 22:10:21,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f9575bc060ce15de83'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:21,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f0575bc060ce15de2d 2025-12-17 22:10:21,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f0575bc060ce15de2d HTTP/1.1" 502 150 2025-12-17 22:10:21,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f0575bc060ce15de2d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:22,227 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f1575bc060ce15de30 2025-12-17 22:10:22,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f1575bc060ce15de30 HTTP/1.1" 502 150 2025-12-17 22:10:22,228 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f1575bc060ce15de30'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:22,325 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f4575bc060ce15de3c 2025-12-17 22:10:22,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f4575bc060ce15de3c HTTP/1.1" 502 150 2025-12-17 22:10:22,328 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f4575bc060ce15de3c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:22,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f7575bc060ce15de54 2025-12-17 22:10:22,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f7575bc060ce15de54 HTTP/1.1" 502 150 2025-12-17 22:10:22,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f7575bc060ce15de54'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:23,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f2575bc060ce15de36 2025-12-17 22:10:23,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f2575bc060ce15de36 HTTP/1.1" 502 150 2025-12-17 22:10:23,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f2575bc060ce15de36'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:23,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085eb575bc060ce15de16 2025-12-17 22:10:23,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085eb575bc060ce15de16 HTTP/1.1" 502 150 2025-12-17 22:10:23,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085eb575bc060ce15de16'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:23,860 - lava_callback - INFO - Received callback for node 6940873c575bc060ce15e307 INFO: 172.21.0.1:59638 - "POST /node/6940873c575bc060ce15e307 HTTP/1.0" 202 Accepted 2025-12-17 22:10:24,494 - lava_callback - INFO - Received callback for node 6940873e575bc060ce15e310 INFO: 172.21.0.1:59652 - "POST /node/6940873e575bc060ce15e310 HTTP/1.0" 202 Accepted 2025-12-17 22:10:24,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085df575bc060ce15ddf4 2025-12-17 22:10:24,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085df575bc060ce15ddf4 HTTP/1.1" 502 150 2025-12-17 22:10:24,557 - lava_callback - ERROR - Error processing callback for node 694085df575bc060ce15ddf4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085df575bc060ce15ddf4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085df575bc060ce15ddf4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085df575bc060ce15ddf4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:24,559 - lava_callback - INFO - Starting async job submit for node 694085f3575bc060ce15de39 2025-12-17 22:10:24,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:24,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f3575bc060ce15de39 HTTP/1.1" 502 150 2025-12-17 22:10:24,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f3575bc060ce15de39'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:24,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f3575bc060ce15de39 2025-12-17 22:10:24,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f3575bc060ce15de39 HTTP/1.1" 502 150 2025-12-17 22:10:24,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f3575bc060ce15de39'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:25,162 - lava_callback - INFO - Received callback for node 69408741575bc060ce15e319 INFO: 172.21.0.1:59660 - "POST /node/69408741575bc060ce15e319 HTTP/1.0" 202 Accepted 2025-12-17 22:10:25,195 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f9575bc060ce15de83 2025-12-17 22:10:25,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f9575bc060ce15de83 HTTP/1.1" 502 150 2025-12-17 22:10:25,196 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f9575bc060ce15de83'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:25,800 - lava_callback - INFO - Received callback for node 69408742575bc060ce15e31c INFO: 172.21.0.1:59666 - "POST /node/69408742575bc060ce15e31c HTTP/1.0" 202 Accepted 2025-12-17 22:10:26,232 - lava_callback - INFO - Received callback for node 6940c6651e3c24e95681d8d8 INFO: 172.21.0.1:59668 - "POST /node/6940c6651e3c24e95681d8d8 HTTP/1.0" 202 Accepted 2025-12-17 22:10:26,457 - lava_callback - INFO - Received callback for node 69408744575bc060ce15e31f INFO: 172.21.0.1:59680 - "POST /node/69408744575bc060ce15e31f HTTP/1.0" 202 Accepted INFO: 172.21.0.1:59686 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:10:26,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f3575bc060ce15de39 2025-12-17 22:10:26,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f3575bc060ce15de39 HTTP/1.1" 502 150 2025-12-17 22:10:26,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f3575bc060ce15de39'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:26,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e9575bc060ce15de0f 2025-12-17 22:10:26,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e9575bc060ce15de0f HTTP/1.1" 502 150 2025-12-17 22:10:26,990 - lava_callback - ERROR - Error processing callback for node 694085e9575bc060ce15de0f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085e9575bc060ce15de0f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085e9575bc060ce15de0f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085e9575bc060ce15de0f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:26,992 - lava_callback - INFO - Starting async job submit for node 694085f5575bc060ce15de3f 2025-12-17 22:10:26,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:26,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f5575bc060ce15de3f HTTP/1.1" 502 150 2025-12-17 22:10:26,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f5575bc060ce15de3f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:26,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f5575bc060ce15de3f 2025-12-17 22:10:26,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f5575bc060ce15de3f HTTP/1.1" 502 150 2025-12-17 22:10:26,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f5575bc060ce15de3f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:27,116 - lava_callback - INFO - Received callback for node 69408749575bc060ce15e325 INFO: 172.21.0.1:59700 - "POST /node/69408749575bc060ce15e325 HTTP/1.0" 202 Accepted 2025-12-17 22:10:28,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ee575bc060ce15de26 2025-12-17 22:10:28,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ee575bc060ce15de26 HTTP/1.1" 502 150 2025-12-17 22:10:28,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ee575bc060ce15de26'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:28,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e5575bc060ce15ddf7 2025-12-17 22:10:28,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e5575bc060ce15ddf7 HTTP/1.1" 502 150 2025-12-17 22:10:28,991 - lava_callback - ERROR - Error processing callback for node 694085e5575bc060ce15ddf7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085e5575bc060ce15ddf7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085e5575bc060ce15ddf7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085e5575bc060ce15ddf7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:28,993 - lava_callback - INFO - Starting async job submit for node 694085fd575bc060ce15de99 2025-12-17 22:10:28,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:28,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f5575bc060ce15de3f 2025-12-17 22:10:29,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f5575bc060ce15de3f HTTP/1.1" 502 150 2025-12-17 22:10:29,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f5575bc060ce15de3f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:29,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085fd575bc060ce15de99 HTTP/1.1" 502 150 2025-12-17 22:10:29,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085fd575bc060ce15de99'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:29,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085fd575bc060ce15de99 2025-12-17 22:10:29,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085fd575bc060ce15de99 HTTP/1.1" 502 150 2025-12-17 22:10:29,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085fd575bc060ce15de99'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:29,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f0575bc060ce15de2d 2025-12-17 22:10:29,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f0575bc060ce15de2d HTTP/1.1" 502 150 2025-12-17 22:10:29,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f0575bc060ce15de2d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:30,228 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f1575bc060ce15de30 2025-12-17 22:10:30,328 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f4575bc060ce15de3c 2025-12-17 22:10:30,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f4575bc060ce15de3c HTTP/1.1" 502 150 2025-12-17 22:10:30,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f4575bc060ce15de3c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:30,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e6575bc060ce15ddfc 2025-12-17 22:10:30,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e6575bc060ce15ddfc HTTP/1.1" 502 150 2025-12-17 22:10:30,460 - lava_callback - ERROR - Error processing callback for node 694085e6575bc060ce15ddfc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085e6575bc060ce15ddfc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085e6575bc060ce15ddfc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085e6575bc060ce15ddfc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:30,462 - lava_callback - INFO - Starting async job submit for node 694085ff575bc060ce15de9e 2025-12-17 22:10:30,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:30,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ff575bc060ce15de9e HTTP/1.1" 502 150 2025-12-17 22:10:30,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ff575bc060ce15de9e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:30,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ff575bc060ce15de9e 2025-12-17 22:10:30,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ff575bc060ce15de9e HTTP/1.1" 502 150 2025-12-17 22:10:30,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ff575bc060ce15de9e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:30,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f3575bc060ce15de39 2025-12-17 22:10:30,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f3575bc060ce15de39 HTTP/1.1" 502 150 2025-12-17 22:10:30,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f3575bc060ce15de39'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:30,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f7575bc060ce15de54 2025-12-17 22:10:30,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f7575bc060ce15de54 HTTP/1.1" 502 150 2025-12-17 22:10:30,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f7575bc060ce15de54'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:31,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085fd575bc060ce15de99 2025-12-17 22:10:31,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085fd575bc060ce15de99 HTTP/1.1" 502 150 2025-12-17 22:10:31,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085fd575bc060ce15de99'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:32,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ff575bc060ce15de9e 2025-12-17 22:10:32,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ff575bc060ce15de9e HTTP/1.1" 502 150 2025-12-17 22:10:32,495 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ff575bc060ce15de9e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:32,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e7575bc060ce15de03 2025-12-17 22:10:32,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e7575bc060ce15de03 HTTP/1.1" 502 150 2025-12-17 22:10:32,816 - lava_callback - ERROR - Error processing callback for node 694085e7575bc060ce15de03: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085e7575bc060ce15de03 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085e7575bc060ce15de03 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085e7575bc060ce15de03 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:32,818 - lava_callback - INFO - Starting async job submit for node 69408600575bc060ce15dea4 2025-12-17 22:10:32,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:32,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408600575bc060ce15dea4 HTTP/1.1" 502 150 2025-12-17 22:10:32,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408600575bc060ce15dea4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:32,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408600575bc060ce15dea4 2025-12-17 22:10:32,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408600575bc060ce15dea4 HTTP/1.1" 502 150 2025-12-17 22:10:32,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408600575bc060ce15dea4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:33,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f5575bc060ce15de3f 2025-12-17 22:10:33,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f5575bc060ce15de3f HTTP/1.1" 502 150 2025-12-17 22:10:33,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f5575bc060ce15de3f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:33,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f9575bc060ce15de83 2025-12-17 22:10:33,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f9575bc060ce15de83 HTTP/1.1" 502 150 2025-12-17 22:10:33,198 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f9575bc060ce15de83'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:34,314 - lava_callback - INFO - Received callback for node 6940874b575bc060ce15e328 INFO: 172.21.0.1:34834 - "POST /node/6940874b575bc060ce15e328 HTTP/1.0" 202 Accepted 2025-12-17 22:10:34,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408600575bc060ce15dea4 2025-12-17 22:10:34,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408600575bc060ce15dea4 HTTP/1.1" 502 150 2025-12-17 22:10:34,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408600575bc060ce15dea4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:35,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085fd575bc060ce15de99 2025-12-17 22:10:35,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085fd575bc060ce15de99 HTTP/1.1" 502 150 2025-12-17 22:10:35,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085fd575bc060ce15de99'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:35,004 - lava_callback - INFO - Received callback for node 6940874e575bc060ce15e32b INFO: 172.21.0.1:34840 - "POST /node/6940874e575bc060ce15e32b HTTP/1.0" 202 Accepted 2025-12-17 22:10:35,729 - lava_callback - INFO - Received callback for node 69408750575bc060ce15e332 INFO: 172.21.0.1:34844 - "POST /node/69408750575bc060ce15e332 HTTP/1.0" 202 Accepted 2025-12-17 22:10:36,233 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085e9575bc060ce15de0a 2025-12-17 22:10:36,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085e9575bc060ce15de0a HTTP/1.1" 502 150 2025-12-17 22:10:36,235 - lava_callback - ERROR - Error processing callback for node 694085e9575bc060ce15de0a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085e9575bc060ce15de0a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085e9575bc060ce15de0a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085e9575bc060ce15de0a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:36,238 - lava_callback - INFO - Starting async job submit for node 69408602575bc060ce15dea7 2025-12-17 22:10:36,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:36,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408602575bc060ce15dea7 HTTP/1.1" 502 150 2025-12-17 22:10:36,247 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408602575bc060ce15dea7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:36,247 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408602575bc060ce15dea7 2025-12-17 22:10:36,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408602575bc060ce15dea7 HTTP/1.1" 502 150 2025-12-17 22:10:36,248 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408602575bc060ce15dea7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:36,401 - lava_callback - INFO - Received callback for node 69408752575bc060ce15e335 INFO: 172.21.0.1:34854 - "POST /node/69408752575bc060ce15e335 HTTP/1.0" 202 Accepted 2025-12-17 22:10:36,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ff575bc060ce15de9e 2025-12-17 22:10:36,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ff575bc060ce15de9e HTTP/1.1" 502 150 2025-12-17 22:10:36,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ff575bc060ce15de9e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34858 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:10:36,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f2575bc060ce15de33 2025-12-17 22:10:36,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f2575bc060ce15de33 HTTP/1.1" 502 150 2025-12-17 22:10:36,858 - lava_callback - ERROR - Error processing callback for node 694085f2575bc060ce15de33: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f2575bc060ce15de33 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f2575bc060ce15de33 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f2575bc060ce15de33 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:36,860 - lava_callback - INFO - Starting async job submit for node 69408603575bc060ce15deaa 2025-12-17 22:10:36,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:36,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408603575bc060ce15deaa HTTP/1.1" 502 150 2025-12-17 22:10:36,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408603575bc060ce15deaa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:36,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408603575bc060ce15deaa 2025-12-17 22:10:36,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408603575bc060ce15deaa HTTP/1.1" 502 150 2025-12-17 22:10:36,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408603575bc060ce15deaa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:37,084 - lava_callback - INFO - Received callback for node 69408756575bc060ce15e340 INFO: 172.21.0.1:34864 - "POST /node/69408756575bc060ce15e340 HTTP/1.0" 202 Accepted 2025-12-17 22:10:38,248 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408602575bc060ce15dea7 2025-12-17 22:10:38,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408602575bc060ce15dea7 HTTP/1.1" 502 150 2025-12-17 22:10:38,250 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408602575bc060ce15dea7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:38,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f3575bc060ce15de39 2025-12-17 22:10:38,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f3575bc060ce15de39 HTTP/1.1" 502 150 2025-12-17 22:10:38,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f3575bc060ce15de39'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:38,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408600575bc060ce15dea4 2025-12-17 22:10:38,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408600575bc060ce15dea4 HTTP/1.1" 502 150 2025-12-17 22:10:38,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408600575bc060ce15dea4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:38,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408603575bc060ce15deaa 2025-12-17 22:10:38,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408603575bc060ce15deaa HTTP/1.1" 502 150 2025-12-17 22:10:38,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408603575bc060ce15deaa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:39,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f2575bc060ce15de36 2025-12-17 22:10:39,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085eb575bc060ce15de16 2025-12-17 22:10:39,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085eb575bc060ce15de16 HTTP/1.1" 502 150 2025-12-17 22:10:39,851 - lava_callback - ERROR - Error processing callback for node 694085eb575bc060ce15de16: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085eb575bc060ce15de16 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085eb575bc060ce15de16 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085eb575bc060ce15de16 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:39,852 - lava_callback - INFO - Starting async job submit for node 6940bf611e3c24e95681c645 2025-12-17 22:10:39,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:39,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940bf611e3c24e95681c645 HTTP/1.1" 502 150 2025-12-17 22:10:39,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940bf611e3c24e95681c645'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:39,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940bf611e3c24e95681c645 2025-12-17 22:10:39,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940bf611e3c24e95681c645 HTTP/1.1" 502 150 2025-12-17 22:10:39,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940bf611e3c24e95681c645'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:41,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f5575bc060ce15de3f 2025-12-17 22:10:41,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f5575bc060ce15de3f HTTP/1.1" 502 150 2025-12-17 22:10:41,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f5575bc060ce15de3f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:41,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940bf611e3c24e95681c645 2025-12-17 22:10:41,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940bf611e3c24e95681c645 HTTP/1.1" 502 150 2025-12-17 22:10:41,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940bf611e3c24e95681c645'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:42,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408598575bc060ce15dca7 HTTP/1.1" 502 150 2025-12-17 22:10:42,224 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408598575bc060ce15dca7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:42,250 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408602575bc060ce15dea7 2025-12-17 22:10:42,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408602575bc060ce15dea7 HTTP/1.1" 502 150 2025-12-17 22:10:42,251 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408602575bc060ce15dea7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:42,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408603575bc060ce15deaa 2025-12-17 22:10:42,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408603575bc060ce15deaa HTTP/1.1" 502 150 2025-12-17 22:10:42,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408603575bc060ce15deaa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:43,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085fd575bc060ce15de99 2025-12-17 22:10:43,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085fd575bc060ce15de99 HTTP/1.1" 502 150 2025-12-17 22:10:43,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085fd575bc060ce15de99'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:43,850 - lava_callback - INFO - Received callback for node 69408758575bc060ce15e343 INFO: 172.21.0.1:57312 - "POST /node/69408758575bc060ce15e343 HTTP/1.0" 202 Accepted 2025-12-17 22:10:44,476 - lava_callback - INFO - Received callback for node 6940875b575bc060ce15e346 INFO: 172.21.0.1:57316 - "POST /node/6940875b575bc060ce15e346 HTTP/1.0" 202 Accepted 2025-12-17 22:10:44,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ff575bc060ce15de9e 2025-12-17 22:10:44,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ee575bc060ce15de26 2025-12-17 22:10:44,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ee575bc060ce15de26 HTTP/1.1" 502 150 2025-12-17 22:10:44,851 - lava_callback - ERROR - Error processing callback for node 694085ee575bc060ce15de26: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085ee575bc060ce15de26 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085ee575bc060ce15de26 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085ee575bc060ce15de26 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:44,853 - lava_callback - INFO - Starting async job submit for node 69408604575bc060ce15deb0 2025-12-17 22:10:44,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:44,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408604575bc060ce15deb0 HTTP/1.1" 502 150 2025-12-17 22:10:44,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408604575bc060ce15deb0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:44,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408604575bc060ce15deb0 2025-12-17 22:10:44,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408604575bc060ce15deb0 HTTP/1.1" 502 150 2025-12-17 22:10:44,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408604575bc060ce15deb0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:45,120 - lava_callback - INFO - Received callback for node 6940875d575bc060ce15e349 INFO: 172.21.0.1:57326 - "POST /node/6940875d575bc060ce15e349 HTTP/1.0" 202 Accepted 2025-12-17 22:10:45,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f1575bc060ce15de30 HTTP/1.1" 502 150 2025-12-17 22:10:45,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085f1575bc060ce15de30'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:45,772 - lava_callback - INFO - Received callback for node 69408760575bc060ce15e34f INFO: 172.21.0.1:57334 - "POST /node/69408760575bc060ce15e34f HTTP/1.0" 202 Accepted 2025-12-17 22:10:45,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940bf611e3c24e95681c645 2025-12-17 22:10:45,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940bf611e3c24e95681c645 HTTP/1.1" 502 150 2025-12-17 22:10:45,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940bf611e3c24e95681c645'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:45,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f0575bc060ce15de2d 2025-12-17 22:10:45,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f0575bc060ce15de2d HTTP/1.1" 502 150 2025-12-17 22:10:45,988 - lava_callback - ERROR - Error processing callback for node 694085f0575bc060ce15de2d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f0575bc060ce15de2d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f0575bc060ce15de2d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f0575bc060ce15de2d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:45,991 - lava_callback - INFO - Starting async job submit for node 69408605575bc060ce15deb3 2025-12-17 22:10:45,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:45,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408605575bc060ce15deb3 HTTP/1.1" 502 150 2025-12-17 22:10:45,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408605575bc060ce15deb3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:45,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408605575bc060ce15deb3 2025-12-17 22:10:45,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408605575bc060ce15deb3 HTTP/1.1" 502 150 2025-12-17 22:10:45,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408605575bc060ce15deb3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:46,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f4575bc060ce15de3c 2025-12-17 22:10:46,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f4575bc060ce15de3c HTTP/1.1" 502 150 2025-12-17 22:10:46,333 - lava_callback - ERROR - Error processing callback for node 694085f4575bc060ce15de3c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f4575bc060ce15de3c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f4575bc060ce15de3c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f4575bc060ce15de3c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:46,336 - lava_callback - INFO - Starting async job submit for node 69408606575bc060ce15deb6 2025-12-17 22:10:46,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:46,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408606575bc060ce15deb6 HTTP/1.1" 502 150 2025-12-17 22:10:46,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408606575bc060ce15deb6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:46,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408606575bc060ce15deb6 2025-12-17 22:10:46,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408606575bc060ce15deb6 HTTP/1.1" 502 150 2025-12-17 22:10:46,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408606575bc060ce15deb6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57336 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:10:46,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408600575bc060ce15dea4 2025-12-17 22:10:46,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408600575bc060ce15dea4 HTTP/1.1" 502 150 2025-12-17 22:10:46,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408600575bc060ce15dea4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:46,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408604575bc060ce15deb0 2025-12-17 22:10:46,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408604575bc060ce15deb0 HTTP/1.1" 502 150 2025-12-17 22:10:46,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408604575bc060ce15deb0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:46,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f7575bc060ce15de54 2025-12-17 22:10:46,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f7575bc060ce15de54 HTTP/1.1" 502 150 2025-12-17 22:10:46,972 - lava_callback - ERROR - Error processing callback for node 694085f7575bc060ce15de54: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f7575bc060ce15de54 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f7575bc060ce15de54 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f7575bc060ce15de54 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:46,975 - lava_callback - INFO - Starting async job submit for node 69408608575bc060ce15deb9 2025-12-17 22:10:46,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:46,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408608575bc060ce15deb9 HTTP/1.1" 502 150 2025-12-17 22:10:46,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408608575bc060ce15deb9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:46,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408608575bc060ce15deb9 2025-12-17 22:10:46,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408608575bc060ce15deb9 HTTP/1.1" 502 150 2025-12-17 22:10:46,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408608575bc060ce15deb9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:48,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408605575bc060ce15deb3 2025-12-17 22:10:48,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408605575bc060ce15deb3 HTTP/1.1" 502 150 2025-12-17 22:10:48,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408605575bc060ce15deb3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:48,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408606575bc060ce15deb6 2025-12-17 22:10:48,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408606575bc060ce15deb6 HTTP/1.1" 502 150 2025-12-17 22:10:48,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408606575bc060ce15deb6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:48,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408608575bc060ce15deb9 2025-12-17 22:10:48,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408608575bc060ce15deb9 HTTP/1.1" 502 150 2025-12-17 22:10:48,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408608575bc060ce15deb9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:49,198 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f9575bc060ce15de83 2025-12-17 22:10:49,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f9575bc060ce15de83 HTTP/1.1" 502 150 2025-12-17 22:10:49,199 - lava_callback - ERROR - Error processing callback for node 694085f9575bc060ce15de83: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f9575bc060ce15de83 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f9575bc060ce15de83 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f9575bc060ce15de83 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:49,201 - lava_callback - INFO - Starting async job submit for node 69408609575bc060ce15debc 2025-12-17 22:10:49,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:49,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408609575bc060ce15debc HTTP/1.1" 502 150 2025-12-17 22:10:49,208 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408609575bc060ce15debc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:49,208 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408609575bc060ce15debc 2025-12-17 22:10:49,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408609575bc060ce15debc HTTP/1.1" 502 150 2025-12-17 22:10:49,209 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408609575bc060ce15debc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:50,251 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408602575bc060ce15dea7 2025-12-17 22:10:50,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408602575bc060ce15dea7 HTTP/1.1" 502 150 2025-12-17 22:10:50,252 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408602575bc060ce15dea7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:50,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408603575bc060ce15deaa 2025-12-17 22:10:50,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408603575bc060ce15deaa HTTP/1.1" 502 150 2025-12-17 22:10:50,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408603575bc060ce15deaa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:50,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408604575bc060ce15deb0 2025-12-17 22:10:50,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408604575bc060ce15deb0 HTTP/1.1" 502 150 2025-12-17 22:10:50,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408604575bc060ce15deb0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:51,209 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408609575bc060ce15debc 2025-12-17 22:10:52,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408605575bc060ce15deb3 2025-12-17 22:10:52,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408605575bc060ce15deb3 HTTP/1.1" 502 150 2025-12-17 22:10:52,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408605575bc060ce15deb3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:52,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408606575bc060ce15deb6 2025-12-17 22:10:52,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408606575bc060ce15deb6 HTTP/1.1" 502 150 2025-12-17 22:10:52,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408606575bc060ce15deb6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:52,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408608575bc060ce15deb9 2025-12-17 22:10:52,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408608575bc060ce15deb9 HTTP/1.1" 502 150 2025-12-17 22:10:52,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408608575bc060ce15deb9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:53,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940bf611e3c24e95681c645 2025-12-17 22:10:53,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940bf611e3c24e95681c645 HTTP/1.1" 502 150 2025-12-17 22:10:53,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940bf611e3c24e95681c645'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:54,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f3575bc060ce15de39 2025-12-17 22:10:54,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f3575bc060ce15de39 HTTP/1.1" 502 150 2025-12-17 22:10:54,572 - lava_callback - ERROR - Error processing callback for node 694085f3575bc060ce15de39: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f3575bc060ce15de39 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f3575bc060ce15de39 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f3575bc060ce15de39 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:54,574 - lava_callback - INFO - Starting async job submit for node 69408614575bc060ce15dedc 2025-12-17 22:10:54,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:54,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408614575bc060ce15dedc HTTP/1.1" 502 150 2025-12-17 22:10:54,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408614575bc060ce15dedc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:54,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408614575bc060ce15dedc 2025-12-17 22:10:54,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408614575bc060ce15dedc HTTP/1.1" 502 150 2025-12-17 22:10:54,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408614575bc060ce15dedc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:54,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f2575bc060ce15de36 HTTP/1.1" 502 150 2025-12-17 22:10:54,768 - lava_callback - ERROR - Error processing callback for node 694085f2575bc060ce15de36: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f2575bc060ce15de36 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f2575bc060ce15de36 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f2575bc060ce15de36 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:54,770 - lava_callback - INFO - Starting async job submit for node 69408617575bc060ce15dee2 2025-12-17 22:10:54,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:54,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408617575bc060ce15dee2 HTTP/1.1" 502 150 2025-12-17 22:10:54,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408617575bc060ce15dee2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:54,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408617575bc060ce15dee2 2025-12-17 22:10:54,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408617575bc060ce15dee2 HTTP/1.1" 502 150 2025-12-17 22:10:54,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408617575bc060ce15dee2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60216 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:10:56,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408614575bc060ce15dedc 2025-12-17 22:10:56,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408614575bc060ce15dedc HTTP/1.1" 502 150 2025-12-17 22:10:56,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408614575bc060ce15dedc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:56,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408617575bc060ce15dee2 2025-12-17 22:10:56,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408617575bc060ce15dee2 HTTP/1.1" 502 150 2025-12-17 22:10:56,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408617575bc060ce15dee2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:57,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f5575bc060ce15de3f 2025-12-17 22:10:57,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f5575bc060ce15de3f HTTP/1.1" 502 150 2025-12-17 22:10:57,004 - lava_callback - ERROR - Error processing callback for node 694085f5575bc060ce15de3f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f5575bc060ce15de3f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f5575bc060ce15de3f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f5575bc060ce15de3f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:57,006 - lava_callback - INFO - Starting async job submit for node 69408618575bc060ce15dee5 2025-12-17 22:10:57,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:57,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408618575bc060ce15dee5 HTTP/1.1" 502 150 2025-12-17 22:10:57,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408618575bc060ce15dee5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:57,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408618575bc060ce15dee5 2025-12-17 22:10:57,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408618575bc060ce15dee5 HTTP/1.1" 502 150 2025-12-17 22:10:57,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408618575bc060ce15dee5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:58,224 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408598575bc060ce15dca7 2025-12-17 22:10:58,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408604575bc060ce15deb0 2025-12-17 22:10:58,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408604575bc060ce15deb0 HTTP/1.1" 502 150 2025-12-17 22:10:58,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408604575bc060ce15deb0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:59,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085fd575bc060ce15de99 2025-12-17 22:10:59,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085fd575bc060ce15de99 HTTP/1.1" 502 150 2025-12-17 22:10:59,006 - lava_callback - ERROR - Error processing callback for node 694085fd575bc060ce15de99: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085fd575bc060ce15de99 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085fd575bc060ce15de99 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085fd575bc060ce15de99 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:10:59,008 - lava_callback - INFO - Starting async job submit for node 6940860a575bc060ce15debf 2025-12-17 22:10:59,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:10:59,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940860a575bc060ce15debf HTTP/1.1" 502 150 2025-12-17 22:10:59,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940860a575bc060ce15debf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:59,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940860a575bc060ce15debf 2025-12-17 22:10:59,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940860a575bc060ce15debf HTTP/1.1" 502 150 2025-12-17 22:10:59,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940860a575bc060ce15debf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:59,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408618575bc060ce15dee5 2025-12-17 22:10:59,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408618575bc060ce15dee5 HTTP/1.1" 502 150 2025-12-17 22:10:59,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408618575bc060ce15dee5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:10:59,043 - lava_callback - INFO - Received callback for node 6940875f575bc060ce15e34c INFO: 172.21.0.1:60222 - "POST /node/6940875f575bc060ce15e34c HTTP/1.0" 202 Accepted 2025-12-17 22:10:59,811 - lava_callback - INFO - Received callback for node 69408762575bc060ce15e352 INFO: 172.21.0.1:47750 - "POST /node/69408762575bc060ce15e352 HTTP/1.0" 202 Accepted 2025-12-17 22:10:59,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ff575bc060ce15de9e HTTP/1.1" 502 150 2025-12-17 22:10:59,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085ff575bc060ce15de9e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:00,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408605575bc060ce15deb3 2025-12-17 22:11:00,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408605575bc060ce15deb3 HTTP/1.1" 502 150 2025-12-17 22:11:00,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408605575bc060ce15deb3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:00,351 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408606575bc060ce15deb6 2025-12-17 22:11:00,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408606575bc060ce15deb6 HTTP/1.1" 502 150 2025-12-17 22:11:00,352 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408606575bc060ce15deb6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:00,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408614575bc060ce15dedc 2025-12-17 22:11:00,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408614575bc060ce15dedc HTTP/1.1" 502 150 2025-12-17 22:11:00,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408614575bc060ce15dedc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:00,639 - lava_callback - INFO - Received callback for node 69408766575bc060ce15e364 INFO: 172.21.0.1:47752 - "POST /node/69408766575bc060ce15e364 HTTP/1.0" 202 Accepted 2025-12-17 22:11:00,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408617575bc060ce15dee2 2025-12-17 22:11:00,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408617575bc060ce15dee2 HTTP/1.1" 502 150 2025-12-17 22:11:00,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408617575bc060ce15dee2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:00,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408608575bc060ce15deb9 2025-12-17 22:11:00,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408608575bc060ce15deb9 HTTP/1.1" 502 150 2025-12-17 22:11:00,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408608575bc060ce15deb9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:01,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940860a575bc060ce15debf 2025-12-17 22:11:01,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940860a575bc060ce15debf HTTP/1.1" 502 150 2025-12-17 22:11:01,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940860a575bc060ce15debf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:01,453 - lava_callback - INFO - Received callback for node 69408767575bc060ce15e369 INFO: 172.21.0.1:47754 - "POST /node/69408767575bc060ce15e369 HTTP/1.0" 202 Accepted 2025-12-17 22:11:01,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085f1575bc060ce15de30 2025-12-17 22:11:01,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085f1575bc060ce15de30 HTTP/1.1" 502 150 2025-12-17 22:11:01,555 - lava_callback - ERROR - Error processing callback for node 694085f1575bc060ce15de30: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f1575bc060ce15de30 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f1575bc060ce15de30 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085f1575bc060ce15de30 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:01,557 - lava_callback - INFO - Starting async job submit for node 6940860b575bc060ce15dec2 2025-12-17 22:11:01,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:01,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940860b575bc060ce15dec2 HTTP/1.1" 502 150 2025-12-17 22:11:01,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940860b575bc060ce15dec2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:01,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940860b575bc060ce15dec2 2025-12-17 22:11:01,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940860b575bc060ce15dec2 HTTP/1.1" 502 150 2025-12-17 22:11:01,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940860b575bc060ce15dec2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47756 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:11:02,266 - lava_callback - INFO - Received callback for node 6940876c575bc060ce15e3c5 INFO: 172.21.0.1:47760 - "POST /node/6940876c575bc060ce15e3c5 HTTP/1.0" 202 Accepted 2025-12-17 22:11:02,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408600575bc060ce15dea4 2025-12-17 22:11:02,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408600575bc060ce15dea4 HTTP/1.1" 502 150 2025-12-17 22:11:02,837 - lava_callback - ERROR - Error processing callback for node 69408600575bc060ce15dea4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408600575bc060ce15dea4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408600575bc060ce15dea4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408600575bc060ce15dea4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:02,839 - lava_callback - INFO - Starting async job submit for node 69408611575bc060ce15dece 2025-12-17 22:11:02,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:02,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408611575bc060ce15dece HTTP/1.1" 502 150 2025-12-17 22:11:02,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408611575bc060ce15dece'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:02,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408611575bc060ce15dece 2025-12-17 22:11:02,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408611575bc060ce15dece HTTP/1.1" 502 150 2025-12-17 22:11:02,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408611575bc060ce15dece'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:03,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408618575bc060ce15dee5 2025-12-17 22:11:03,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408618575bc060ce15dee5 HTTP/1.1" 502 150 2025-12-17 22:11:03,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408618575bc060ce15dee5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:03,078 - lava_callback - INFO - Received callback for node 69408770575bc060ce15e3cb INFO: 172.21.0.1:47762 - "POST /node/69408770575bc060ce15e3cb HTTP/1.0" 202 Accepted 2025-12-17 22:11:03,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940860b575bc060ce15dec2 2025-12-17 22:11:03,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940860b575bc060ce15dec2 HTTP/1.1" 502 150 2025-12-17 22:11:03,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940860b575bc060ce15dec2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:03,879 - lava_callback - INFO - Received callback for node 69408763575bc060ce15e357 INFO: 172.21.0.1:47772 - "POST /node/69408763575bc060ce15e357 HTTP/1.0" 202 Accepted 2025-12-17 22:11:03,959 - lava_callback - INFO - Received callback for node 69408771575bc060ce15e3ce INFO: 172.21.0.1:47778 - "POST /node/69408771575bc060ce15e3ce HTTP/1.0" 202 Accepted 2025-12-17 22:11:04,537 - lava_callback - INFO - Received callback for node 69408769575bc060ce15e38d INFO: 172.21.0.1:47794 - "POST /node/69408769575bc060ce15e38d HTTP/1.0" 202 Accepted 2025-12-17 22:11:04,712 - lava_callback - INFO - Received callback for node 69408772575bc060ce15e3d1 INFO: 172.21.0.1:47808 - "POST /node/69408772575bc060ce15e3d1 HTTP/1.0" 202 Accepted 2025-12-17 22:11:04,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408611575bc060ce15dece 2025-12-17 22:11:04,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408611575bc060ce15dece HTTP/1.1" 502 150 2025-12-17 22:11:04,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408611575bc060ce15dece'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:05,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940860a575bc060ce15debf 2025-12-17 22:11:05,442 - lava_callback - INFO - Received callback for node 69408774575bc060ce15e3d4 INFO: 172.21.0.1:47820 - "POST /node/69408774575bc060ce15e3d4 HTTP/1.0" 202 Accepted 2025-12-17 22:11:06,252 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408602575bc060ce15dea7 2025-12-17 22:11:06,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408602575bc060ce15dea7 HTTP/1.1" 502 150 2025-12-17 22:11:06,254 - lava_callback - ERROR - Error processing callback for node 69408602575bc060ce15dea7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408602575bc060ce15dea7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408602575bc060ce15dea7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408602575bc060ce15dea7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:06,256 - lava_callback - INFO - Starting async job submit for node 69408612575bc060ce15ded5 2025-12-17 22:11:06,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:06,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408612575bc060ce15ded5 HTTP/1.1" 502 150 2025-12-17 22:11:06,262 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408612575bc060ce15ded5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:06,262 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408612575bc060ce15ded5 2025-12-17 22:11:06,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408612575bc060ce15ded5 HTTP/1.1" 502 150 2025-12-17 22:11:06,263 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408612575bc060ce15ded5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47830 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:11:06,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408609575bc060ce15debc HTTP/1.1" 502 150 2025-12-17 22:11:06,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408609575bc060ce15debc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:06,608 - lava_callback - INFO - Received callback for node 69409094575bc060ce1605ed INFO: 172.21.0.1:47838 - "POST /node/69409094575bc060ce1605ed HTTP/1.0" 202 Accepted 2025-12-17 22:11:06,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408603575bc060ce15deaa 2025-12-17 22:11:06,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408603575bc060ce15deaa HTTP/1.1" 502 150 2025-12-17 22:11:06,876 - lava_callback - ERROR - Error processing callback for node 69408603575bc060ce15deaa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408603575bc060ce15deaa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408603575bc060ce15deaa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408603575bc060ce15deaa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:06,878 - lava_callback - INFO - Starting async job submit for node 69408616575bc060ce15dedf 2025-12-17 22:11:06,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:06,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408616575bc060ce15dedf HTTP/1.1" 502 150 2025-12-17 22:11:06,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408616575bc060ce15dedf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:06,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408616575bc060ce15dedf 2025-12-17 22:11:06,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408616575bc060ce15dedf HTTP/1.1" 502 150 2025-12-17 22:11:06,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408616575bc060ce15dedf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:07,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940860b575bc060ce15dec2 2025-12-17 22:11:07,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940860b575bc060ce15dec2 HTTP/1.1" 502 150 2025-12-17 22:11:07,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940860b575bc060ce15dec2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:08,263 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408612575bc060ce15ded5 2025-12-17 22:11:08,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408612575bc060ce15ded5 HTTP/1.1" 502 150 2025-12-17 22:11:08,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408612575bc060ce15ded5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:08,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408614575bc060ce15dedc 2025-12-17 22:11:08,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408614575bc060ce15dedc HTTP/1.1" 502 150 2025-12-17 22:11:08,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408614575bc060ce15dedc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:08,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408617575bc060ce15dee2 2025-12-17 22:11:08,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408617575bc060ce15dee2 HTTP/1.1" 502 150 2025-12-17 22:11:08,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408617575bc060ce15dee2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:08,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408611575bc060ce15dece 2025-12-17 22:11:08,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408611575bc060ce15dece HTTP/1.1" 502 150 2025-12-17 22:11:08,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408611575bc060ce15dece'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:08,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408616575bc060ce15dedf 2025-12-17 22:11:08,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408616575bc060ce15dedf HTTP/1.1" 502 150 2025-12-17 22:11:08,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408616575bc060ce15dedf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:09,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940bf611e3c24e95681c645 2025-12-17 22:11:09,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940bf611e3c24e95681c645 HTTP/1.1" 502 150 2025-12-17 22:11:09,884 - lava_callback - ERROR - Error processing callback for node 6940bf611e3c24e95681c645: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940bf611e3c24e95681c645 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940bf611e3c24e95681c645 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940bf611e3c24e95681c645 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:09,886 - lava_callback - INFO - Starting async job submit for node 69408625575bc060ce15df12 2025-12-17 22:11:09,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:09,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408625575bc060ce15df12 HTTP/1.1" 502 150 2025-12-17 22:11:09,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408625575bc060ce15df12'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:09,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408625575bc060ce15df12 2025-12-17 22:11:09,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408625575bc060ce15df12 HTTP/1.1" 502 150 2025-12-17 22:11:09,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408625575bc060ce15df12'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:10,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408609575bc060ce15debc 2025-12-17 22:11:11,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408618575bc060ce15dee5 2025-12-17 22:11:11,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408618575bc060ce15dee5 HTTP/1.1" 502 150 2025-12-17 22:11:11,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408618575bc060ce15dee5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:11,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408625575bc060ce15df12 2025-12-17 22:11:11,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408625575bc060ce15df12 HTTP/1.1" 502 150 2025-12-17 22:11:11,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408625575bc060ce15df12'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:12,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408612575bc060ce15ded5 2025-12-17 22:11:12,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408612575bc060ce15ded5 HTTP/1.1" 502 150 2025-12-17 22:11:12,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408612575bc060ce15ded5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:12,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408616575bc060ce15dedf 2025-12-17 22:11:12,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408616575bc060ce15dedf HTTP/1.1" 502 150 2025-12-17 22:11:12,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408616575bc060ce15dedf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:14,063 - lava_callback - INFO - Received callback for node 6940876d575bc060ce15e3c8 INFO: 172.21.0.1:36778 - "POST /node/6940876d575bc060ce15e3c8 HTTP/1.0" 202 Accepted 2025-12-17 22:11:14,732 - lava_callback - INFO - Received callback for node 69408775575bc060ce15e3d7 INFO: 172.21.0.1:36784 - "POST /node/69408775575bc060ce15e3d7 HTTP/1.0" 202 Accepted 2025-12-17 22:11:14,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408604575bc060ce15deb0 2025-12-17 22:11:14,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408604575bc060ce15deb0 HTTP/1.1" 502 150 2025-12-17 22:11:14,883 - lava_callback - ERROR - Error processing callback for node 69408604575bc060ce15deb0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408604575bc060ce15deb0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408604575bc060ce15deb0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408604575bc060ce15deb0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:14,884 - lava_callback - INFO - Starting async job submit for node 6940861a575bc060ce15dee8 2025-12-17 22:11:14,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:14,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861a575bc060ce15dee8 HTTP/1.1" 502 150 2025-12-17 22:11:14,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861a575bc060ce15dee8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:14,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861a575bc060ce15dee8 2025-12-17 22:11:14,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861a575bc060ce15dee8 HTTP/1.1" 502 150 2025-12-17 22:11:14,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861a575bc060ce15dee8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36794 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:11:15,407 - lava_callback - INFO - Received callback for node 69408776575bc060ce15e3de INFO: 172.21.0.1:36806 - "POST /node/69408776575bc060ce15e3de HTTP/1.0" 202 Accepted 2025-12-17 22:11:15,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940860b575bc060ce15dec2 2025-12-17 22:11:15,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940860b575bc060ce15dec2 HTTP/1.1" 502 150 2025-12-17 22:11:15,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940860b575bc060ce15dec2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:15,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085ff575bc060ce15de9e 2025-12-17 22:11:15,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085ff575bc060ce15de9e HTTP/1.1" 502 150 2025-12-17 22:11:15,890 - lava_callback - ERROR - Error processing callback for node 694085ff575bc060ce15de9e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085ff575bc060ce15de9e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085ff575bc060ce15de9e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085ff575bc060ce15de9e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:15,892 - lava_callback - INFO - Starting async job submit for node 6940861b575bc060ce15deeb 2025-12-17 22:11:15,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:15,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861b575bc060ce15deeb HTTP/1.1" 502 150 2025-12-17 22:11:15,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861b575bc060ce15deeb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:15,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861b575bc060ce15deeb 2025-12-17 22:11:15,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861b575bc060ce15deeb HTTP/1.1" 502 150 2025-12-17 22:11:15,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861b575bc060ce15deeb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:15,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408625575bc060ce15df12 2025-12-17 22:11:15,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408625575bc060ce15df12 HTTP/1.1" 502 150 2025-12-17 22:11:15,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408625575bc060ce15df12'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:16,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408605575bc060ce15deb3 2025-12-17 22:11:16,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408605575bc060ce15deb3 HTTP/1.1" 502 150 2025-12-17 22:11:16,005 - lava_callback - ERROR - Error processing callback for node 69408605575bc060ce15deb3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408605575bc060ce15deb3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408605575bc060ce15deb3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408605575bc060ce15deb3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:16,007 - lava_callback - INFO - Starting async job submit for node 6940861c575bc060ce15deee 2025-12-17 22:11:16,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:16,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861c575bc060ce15deee HTTP/1.1" 502 150 2025-12-17 22:11:16,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861c575bc060ce15deee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:16,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861c575bc060ce15deee 2025-12-17 22:11:16,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861c575bc060ce15deee HTTP/1.1" 502 150 2025-12-17 22:11:16,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861c575bc060ce15deee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:16,069 - lava_callback - INFO - Received callback for node 69408777575bc060ce15e3e3 INFO: 172.21.0.1:36814 - "POST /node/69408777575bc060ce15e3e3 HTTP/1.0" 202 Accepted 2025-12-17 22:11:16,352 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408606575bc060ce15deb6 2025-12-17 22:11:16,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408606575bc060ce15deb6 HTTP/1.1" 502 150 2025-12-17 22:11:16,354 - lava_callback - ERROR - Error processing callback for node 69408606575bc060ce15deb6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408606575bc060ce15deb6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408606575bc060ce15deb6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408606575bc060ce15deb6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:16,356 - lava_callback - INFO - Starting async job submit for node 6940861e575bc060ce15def1 2025-12-17 22:11:16,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:16,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861e575bc060ce15def1 HTTP/1.1" 502 150 2025-12-17 22:11:16,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861e575bc060ce15def1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:16,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861e575bc060ce15def1 2025-12-17 22:11:16,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861e575bc060ce15def1 HTTP/1.1" 502 150 2025-12-17 22:11:16,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861e575bc060ce15def1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36818 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:11:16,773 - lava_callback - INFO - Received callback for node 69408779575bc060ce15e3e8 INFO: 172.21.0.1:36822 - "POST /node/69408779575bc060ce15e3e8 HTTP/1.0" 202 Accepted 2025-12-17 22:11:16,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408611575bc060ce15dece 2025-12-17 22:11:16,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408611575bc060ce15dece HTTP/1.1" 502 150 2025-12-17 22:11:16,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408611575bc060ce15dece'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:16,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861a575bc060ce15dee8 2025-12-17 22:11:16,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861a575bc060ce15dee8 HTTP/1.1" 502 150 2025-12-17 22:11:16,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861a575bc060ce15dee8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:16,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408608575bc060ce15deb9 2025-12-17 22:11:16,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408608575bc060ce15deb9 HTTP/1.1" 502 150 2025-12-17 22:11:16,990 - lava_callback - ERROR - Error processing callback for node 69408608575bc060ce15deb9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408608575bc060ce15deb9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408608575bc060ce15deb9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408608575bc060ce15deb9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:16,992 - lava_callback - INFO - Starting async job submit for node 6940861f575bc060ce15def4 2025-12-17 22:11:16,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:16,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861f575bc060ce15def4 HTTP/1.1" 502 150 2025-12-17 22:11:16,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861f575bc060ce15def4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:16,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861f575bc060ce15def4 2025-12-17 22:11:16,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861f575bc060ce15def4 HTTP/1.1" 502 150 2025-12-17 22:11:16,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861f575bc060ce15def4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:17,456 - lava_callback - INFO - Received callback for node 6940877a575bc060ce15e3eb INFO: 172.21.0.1:36834 - "POST /node/6940877a575bc060ce15e3eb HTTP/1.0" 202 Accepted 2025-12-17 22:11:17,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861b575bc060ce15deeb 2025-12-17 22:11:17,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861b575bc060ce15deeb HTTP/1.1" 502 150 2025-12-17 22:11:17,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861b575bc060ce15deeb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:18,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861c575bc060ce15deee 2025-12-17 22:11:18,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861c575bc060ce15deee HTTP/1.1" 502 150 2025-12-17 22:11:18,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861c575bc060ce15deee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:18,130 - lava_callback - INFO - Received callback for node 6940877b575bc060ce15e3ee INFO: 172.21.0.1:36848 - "POST /node/6940877b575bc060ce15e3ee HTTP/1.0" 202 Accepted 2025-12-17 22:11:18,363 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861e575bc060ce15def1 2025-12-17 22:11:18,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861e575bc060ce15def1 HTTP/1.1" 502 150 2025-12-17 22:11:18,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861e575bc060ce15def1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:18,813 - lava_callback - INFO - Received callback for node 6940877d575bc060ce15e3f1 INFO: 172.21.0.1:36852 - "POST /node/6940877d575bc060ce15e3f1 HTTP/1.0" 202 Accepted 2025-12-17 22:11:19,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861f575bc060ce15def4 2025-12-17 22:11:19,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861f575bc060ce15def4 HTTP/1.1" 502 150 2025-12-17 22:11:19,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861f575bc060ce15def4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:20,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408612575bc060ce15ded5 2025-12-17 22:11:20,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408612575bc060ce15ded5 HTTP/1.1" 502 150 2025-12-17 22:11:20,268 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408612575bc060ce15ded5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:20,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940860a575bc060ce15debf HTTP/1.1" 502 150 2025-12-17 22:11:20,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940860a575bc060ce15debf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:20,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408616575bc060ce15dedf 2025-12-17 22:11:20,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408616575bc060ce15dedf HTTP/1.1" 502 150 2025-12-17 22:11:20,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408616575bc060ce15dedf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:20,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861a575bc060ce15dee8 2025-12-17 22:11:20,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861a575bc060ce15dee8 HTTP/1.1" 502 150 2025-12-17 22:11:20,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861a575bc060ce15dee8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:21,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861b575bc060ce15deeb 2025-12-17 22:11:21,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861b575bc060ce15deeb HTTP/1.1" 502 150 2025-12-17 22:11:21,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861b575bc060ce15deeb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:22,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861c575bc060ce15deee 2025-12-17 22:11:22,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861e575bc060ce15def1 2025-12-17 22:11:22,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861e575bc060ce15def1 HTTP/1.1" 502 150 2025-12-17 22:11:22,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861e575bc060ce15def1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:23,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861f575bc060ce15def4 2025-12-17 22:11:23,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861f575bc060ce15def4 HTTP/1.1" 502 150 2025-12-17 22:11:23,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861f575bc060ce15def4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:23,885 - lava_callback - INFO - Received callback for node 69408781575bc060ce15e3f7 INFO: 172.21.0.1:50918 - "POST /node/69408781575bc060ce15e3f7 HTTP/1.0" 202 Accepted 2025-12-17 22:11:23,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408625575bc060ce15df12 2025-12-17 22:11:23,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408625575bc060ce15df12 HTTP/1.1" 502 150 2025-12-17 22:11:23,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408625575bc060ce15df12'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:24,525 - lava_callback - INFO - Received callback for node 69408784575bc060ce15e3fa INFO: 172.21.0.1:50928 - "POST /node/69408784575bc060ce15e3fa HTTP/1.0" 202 Accepted 2025-12-17 22:11:24,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408614575bc060ce15dedc 2025-12-17 22:11:24,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408614575bc060ce15dedc HTTP/1.1" 502 150 2025-12-17 22:11:24,590 - lava_callback - ERROR - Error processing callback for node 69408614575bc060ce15dedc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408614575bc060ce15dedc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408614575bc060ce15dedc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408614575bc060ce15dedc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:24,593 - lava_callback - INFO - Starting async job submit for node 69408620575bc060ce15def7 2025-12-17 22:11:24,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:24,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408620575bc060ce15def7 HTTP/1.1" 502 150 2025-12-17 22:11:24,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408620575bc060ce15def7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:24,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408620575bc060ce15def7 2025-12-17 22:11:24,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408620575bc060ce15def7 HTTP/1.1" 502 150 2025-12-17 22:11:24,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408620575bc060ce15def7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:24,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408617575bc060ce15dee2 2025-12-17 22:11:24,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408617575bc060ce15dee2 HTTP/1.1" 502 150 2025-12-17 22:11:24,800 - lava_callback - ERROR - Error processing callback for node 69408617575bc060ce15dee2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408617575bc060ce15dee2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408617575bc060ce15dee2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408617575bc060ce15dee2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:24,803 - lava_callback - INFO - Starting async job submit for node 69408621575bc060ce15defa 2025-12-17 22:11:24,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:24,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408621575bc060ce15defa HTTP/1.1" 502 150 2025-12-17 22:11:24,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408621575bc060ce15defa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:24,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408621575bc060ce15defa 2025-12-17 22:11:24,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408621575bc060ce15defa HTTP/1.1" 502 150 2025-12-17 22:11:24,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408621575bc060ce15defa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:25,175 - lava_callback - INFO - Received callback for node 69408787575bc060ce15e3fd INFO: 172.21.0.1:50944 - "POST /node/69408787575bc060ce15e3fd HTTP/1.0" 202 Accepted 2025-12-17 22:11:25,837 - lava_callback - INFO - Received callback for node 69408789575bc060ce15e400 INFO: 172.21.0.1:50956 - "POST /node/69408789575bc060ce15e400 HTTP/1.0" 202 Accepted 2025-12-17 22:11:26,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408609575bc060ce15debc HTTP/1.1" 502 150 2025-12-17 22:11:26,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408609575bc060ce15debc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50966 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:11:26,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408620575bc060ce15def7 2025-12-17 22:11:26,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408620575bc060ce15def7 HTTP/1.1" 502 150 2025-12-17 22:11:26,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408620575bc060ce15def7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:26,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408621575bc060ce15defa 2025-12-17 22:11:26,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408621575bc060ce15defa HTTP/1.1" 502 150 2025-12-17 22:11:26,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408621575bc060ce15defa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:27,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408618575bc060ce15dee5 2025-12-17 22:11:27,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408618575bc060ce15dee5 HTTP/1.1" 502 150 2025-12-17 22:11:27,029 - lava_callback - ERROR - Error processing callback for node 69408618575bc060ce15dee5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408618575bc060ce15dee5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408618575bc060ce15dee5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408618575bc060ce15dee5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:27,030 - lava_callback - INFO - Starting async job submit for node 69408627575bc060ce15df1b 2025-12-17 22:11:27,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:27,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408627575bc060ce15df1b HTTP/1.1" 502 150 2025-12-17 22:11:27,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408627575bc060ce15df1b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:27,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408627575bc060ce15df1b 2025-12-17 22:11:27,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408627575bc060ce15df1b HTTP/1.1" 502 150 2025-12-17 22:11:27,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408627575bc060ce15df1b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:28,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940860a575bc060ce15debf 2025-12-17 22:11:28,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940860a575bc060ce15debf HTTP/1.1" 502 150 2025-12-17 22:11:28,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940860a575bc060ce15debf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:28,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861a575bc060ce15dee8 2025-12-17 22:11:28,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861a575bc060ce15dee8 HTTP/1.1" 502 150 2025-12-17 22:11:28,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861a575bc060ce15dee8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:29,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408627575bc060ce15df1b 2025-12-17 22:11:29,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408627575bc060ce15df1b HTTP/1.1" 502 150 2025-12-17 22:11:29,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408627575bc060ce15df1b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:29,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408598575bc060ce15dca7 HTTP/1.1" 502 150 2025-12-17 22:11:29,072 - lava_callback - ERROR - Error processing callback for node 69408598575bc060ce15dca7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408598575bc060ce15dca7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408598575bc060ce15dca7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408598575bc060ce15dca7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:29,074 - lava_callback - INFO - Starting async job submit for node 69408628575bc060ce15df1e 2025-12-17 22:11:29,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:29,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408628575bc060ce15df1e HTTP/1.1" 502 150 2025-12-17 22:11:29,084 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408628575bc060ce15df1e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:29,084 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408628575bc060ce15df1e 2025-12-17 22:11:29,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408628575bc060ce15df1e HTTP/1.1" 502 150 2025-12-17 22:11:29,085 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408628575bc060ce15df1e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:29,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861b575bc060ce15deeb 2025-12-17 22:11:29,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861b575bc060ce15deeb HTTP/1.1" 502 150 2025-12-17 22:11:29,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861b575bc060ce15deeb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:30,367 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861e575bc060ce15def1 2025-12-17 22:11:30,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861e575bc060ce15def1 HTTP/1.1" 502 150 2025-12-17 22:11:30,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861e575bc060ce15def1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:30,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408620575bc060ce15def7 2025-12-17 22:11:30,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408620575bc060ce15def7 HTTP/1.1" 502 150 2025-12-17 22:11:30,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408620575bc060ce15def7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:30,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408621575bc060ce15defa 2025-12-17 22:11:30,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408621575bc060ce15defa HTTP/1.1" 502 150 2025-12-17 22:11:30,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408621575bc060ce15defa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:31,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861f575bc060ce15def4 2025-12-17 22:11:31,085 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408628575bc060ce15df1e 2025-12-17 22:11:31,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408628575bc060ce15df1e HTTP/1.1" 502 150 2025-12-17 22:11:31,086 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408628575bc060ce15df1e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:31,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940860b575bc060ce15dec2 2025-12-17 22:11:31,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940860b575bc060ce15dec2 HTTP/1.1" 502 150 2025-12-17 22:11:31,582 - lava_callback - ERROR - Error processing callback for node 6940860b575bc060ce15dec2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940860b575bc060ce15dec2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940860b575bc060ce15dec2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940860b575bc060ce15dec2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:31,584 - lava_callback - INFO - Starting async job submit for node 6940862a575bc060ce15df21 2025-12-17 22:11:31,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:31,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862a575bc060ce15df21 HTTP/1.1" 502 150 2025-12-17 22:11:31,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862a575bc060ce15df21'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:31,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862a575bc060ce15df21 2025-12-17 22:11:31,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862a575bc060ce15df21 HTTP/1.1" 502 150 2025-12-17 22:11:31,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862a575bc060ce15df21'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:32,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408611575bc060ce15dece 2025-12-17 22:11:32,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408611575bc060ce15dece HTTP/1.1" 502 150 2025-12-17 22:11:32,858 - lava_callback - ERROR - Error processing callback for node 69408611575bc060ce15dece: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408611575bc060ce15dece (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408611575bc060ce15dece (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408611575bc060ce15dece (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:32,860 - lava_callback - INFO - Starting async job submit for node 6940862b575bc060ce15df24 2025-12-17 22:11:32,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:32,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862b575bc060ce15df24 HTTP/1.1" 502 150 2025-12-17 22:11:32,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862b575bc060ce15df24'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:32,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862b575bc060ce15df24 2025-12-17 22:11:32,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862b575bc060ce15df24 HTTP/1.1" 502 150 2025-12-17 22:11:32,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862b575bc060ce15df24'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:33,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408627575bc060ce15df1b 2025-12-17 22:11:33,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408627575bc060ce15df1b HTTP/1.1" 502 150 2025-12-17 22:11:33,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408627575bc060ce15df1b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:33,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862a575bc060ce15df21 2025-12-17 22:11:33,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862a575bc060ce15df21 HTTP/1.1" 502 150 2025-12-17 22:11:33,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862a575bc060ce15df21'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:34,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408609575bc060ce15debc 2025-12-17 22:11:34,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408609575bc060ce15debc HTTP/1.1" 502 150 2025-12-17 22:11:34,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408609575bc060ce15debc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:34,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862b575bc060ce15df24 2025-12-17 22:11:34,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862b575bc060ce15df24 HTTP/1.1" 502 150 2025-12-17 22:11:34,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862b575bc060ce15df24'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:35,087 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408628575bc060ce15df1e 2025-12-17 22:11:35,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408628575bc060ce15df1e HTTP/1.1" 502 150 2025-12-17 22:11:35,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408628575bc060ce15df1e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:36,268 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408612575bc060ce15ded5 2025-12-17 22:11:36,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408612575bc060ce15ded5 HTTP/1.1" 502 150 2025-12-17 22:11:36,273 - lava_callback - ERROR - Error processing callback for node 69408612575bc060ce15ded5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408612575bc060ce15ded5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408612575bc060ce15ded5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408612575bc060ce15ded5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:36,276 - lava_callback - INFO - Starting async job submit for node 6940862c575bc060ce15df27 2025-12-17 22:11:36,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:36,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862c575bc060ce15df27 HTTP/1.1" 502 150 2025-12-17 22:11:36,282 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862c575bc060ce15df27'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:36,283 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862c575bc060ce15df27 2025-12-17 22:11:36,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862c575bc060ce15df27 HTTP/1.1" 502 150 2025-12-17 22:11:36,283 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862c575bc060ce15df27'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39138 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:11:36,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408616575bc060ce15dedf 2025-12-17 22:11:36,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408616575bc060ce15dedf HTTP/1.1" 502 150 2025-12-17 22:11:36,891 - lava_callback - ERROR - Error processing callback for node 69408616575bc060ce15dedf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408616575bc060ce15dedf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408616575bc060ce15dedf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408616575bc060ce15dedf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:36,893 - lava_callback - INFO - Starting async job submit for node 69408631575bc060ce15df30 2025-12-17 22:11:36,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:36,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408631575bc060ce15df30 HTTP/1.1" 502 150 2025-12-17 22:11:36,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408631575bc060ce15df30'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:36,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408631575bc060ce15df30 2025-12-17 22:11:36,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408631575bc060ce15df30 HTTP/1.1" 502 150 2025-12-17 22:11:36,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408631575bc060ce15df30'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:37,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861c575bc060ce15deee HTTP/1.1" 502 150 2025-12-17 22:11:37,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861c575bc060ce15deee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:37,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862a575bc060ce15df21 2025-12-17 22:11:37,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862a575bc060ce15df21 HTTP/1.1" 502 150 2025-12-17 22:11:37,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862a575bc060ce15df21'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:38,284 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862c575bc060ce15df27 2025-12-17 22:11:38,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862c575bc060ce15df27 HTTP/1.1" 502 150 2025-12-17 22:11:38,285 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862c575bc060ce15df27'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:38,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408620575bc060ce15def7 2025-12-17 22:11:38,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408620575bc060ce15def7 HTTP/1.1" 502 150 2025-12-17 22:11:38,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408620575bc060ce15def7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:38,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408621575bc060ce15defa 2025-12-17 22:11:38,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408621575bc060ce15defa HTTP/1.1" 502 150 2025-12-17 22:11:38,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408621575bc060ce15defa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:38,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862b575bc060ce15df24 2025-12-17 22:11:38,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862b575bc060ce15df24 HTTP/1.1" 502 150 2025-12-17 22:11:38,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862b575bc060ce15df24'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:38,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408631575bc060ce15df30 2025-12-17 22:11:38,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408631575bc060ce15df30 HTTP/1.1" 502 150 2025-12-17 22:11:38,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408631575bc060ce15df30'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:39,196 - lava_callback - INFO - Received callback for node 6940879a575bc060ce15e41e INFO: 172.21.0.1:39142 - "POST /node/6940879a575bc060ce15e41e HTTP/1.0" 202 Accepted 2025-12-17 22:11:39,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408625575bc060ce15df12 2025-12-17 22:11:39,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408625575bc060ce15df12 HTTP/1.1" 502 150 2025-12-17 22:11:39,921 - lava_callback - ERROR - Error processing callback for node 69408625575bc060ce15df12: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408625575bc060ce15df12 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408625575bc060ce15df12 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408625575bc060ce15df12 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:39,923 - lava_callback - INFO - Starting async job submit for node 69408633575bc060ce15df36 2025-12-17 22:11:39,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:39,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408633575bc060ce15df36 HTTP/1.1" 502 150 2025-12-17 22:11:39,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408633575bc060ce15df36'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:39,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408633575bc060ce15df36 2025-12-17 22:11:39,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408633575bc060ce15df36 HTTP/1.1" 502 150 2025-12-17 22:11:39,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408633575bc060ce15df36'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:41,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408627575bc060ce15df1b 2025-12-17 22:11:41,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408627575bc060ce15df1b HTTP/1.1" 502 150 2025-12-17 22:11:41,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408627575bc060ce15df1b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:41,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408633575bc060ce15df36 2025-12-17 22:11:41,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408633575bc060ce15df36 HTTP/1.1" 502 150 2025-12-17 22:11:41,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408633575bc060ce15df36'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:42,285 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862c575bc060ce15df27 2025-12-17 22:11:42,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862c575bc060ce15df27 HTTP/1.1" 502 150 2025-12-17 22:11:42,286 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862c575bc060ce15df27'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:42,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408631575bc060ce15df30 2025-12-17 22:11:42,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408631575bc060ce15df30 HTTP/1.1" 502 150 2025-12-17 22:11:42,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408631575bc060ce15df30'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:43,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408628575bc060ce15df1e 2025-12-17 22:11:43,918 - lava_callback - INFO - Received callback for node 6940878b575bc060ce15e405 INFO: 172.21.0.1:44160 - "POST /node/6940878b575bc060ce15e405 HTTP/1.0" 202 Accepted 2025-12-17 22:11:44,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940860a575bc060ce15debf 2025-12-17 22:11:44,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940860a575bc060ce15debf HTTP/1.1" 502 150 2025-12-17 22:11:44,371 - lava_callback - ERROR - Error processing callback for node 6940860a575bc060ce15debf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940860a575bc060ce15debf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940860a575bc060ce15debf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940860a575bc060ce15debf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:44,373 - lava_callback - INFO - Starting async job submit for node 6940862d575bc060ce15df2a 2025-12-17 22:11:44,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:44,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862d575bc060ce15df2a HTTP/1.1" 502 150 2025-12-17 22:11:44,379 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862d575bc060ce15df2a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:44,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862d575bc060ce15df2a 2025-12-17 22:11:44,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862d575bc060ce15df2a HTTP/1.1" 502 150 2025-12-17 22:11:44,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862d575bc060ce15df2a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:44,590 - lava_callback - INFO - Received callback for node 6940878d575bc060ce15e408 INFO: 172.21.0.1:44172 - "POST /node/6940878d575bc060ce15e408 HTTP/1.0" 202 Accepted 2025-12-17 22:11:44,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861a575bc060ce15dee8 2025-12-17 22:11:44,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861a575bc060ce15dee8 HTTP/1.1" 502 150 2025-12-17 22:11:44,897 - lava_callback - ERROR - Error processing callback for node 6940861a575bc060ce15dee8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940861a575bc060ce15dee8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940861a575bc060ce15dee8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940861a575bc060ce15dee8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:44,898 - lava_callback - INFO - Starting async job submit for node 6940862f575bc060ce15df2d 2025-12-17 22:11:44,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:44,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862f575bc060ce15df2d HTTP/1.1" 502 150 2025-12-17 22:11:44,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862f575bc060ce15df2d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:44,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862f575bc060ce15df2d 2025-12-17 22:11:44,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862f575bc060ce15df2d HTTP/1.1" 502 150 2025-12-17 22:11:44,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862f575bc060ce15df2d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:45,236 - lava_callback - INFO - Received callback for node 6940878f575bc060ce15e40b INFO: 172.21.0.1:44182 - "POST /node/6940878f575bc060ce15e40b HTTP/1.0" 202 Accepted 2025-12-17 22:11:45,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861c575bc060ce15deee 2025-12-17 22:11:45,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861c575bc060ce15deee HTTP/1.1" 502 150 2025-12-17 22:11:45,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861c575bc060ce15deee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:45,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862a575bc060ce15df21 2025-12-17 22:11:45,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862a575bc060ce15df21 HTTP/1.1" 502 150 2025-12-17 22:11:45,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862a575bc060ce15df21'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:45,863 - lava_callback - INFO - Received callback for node 69408791575bc060ce15e40e INFO: 172.21.0.1:44196 - "POST /node/69408791575bc060ce15e40e HTTP/1.0" 202 Accepted 2025-12-17 22:11:45,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861b575bc060ce15deeb 2025-12-17 22:11:45,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861b575bc060ce15deeb HTTP/1.1" 502 150 2025-12-17 22:11:45,915 - lava_callback - ERROR - Error processing callback for node 6940861b575bc060ce15deeb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940861b575bc060ce15deeb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940861b575bc060ce15deeb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940861b575bc060ce15deeb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:45,917 - lava_callback - INFO - Starting async job submit for node 69408637575bc060ce15df76 2025-12-17 22:11:45,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:45,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408633575bc060ce15df36 2025-12-17 22:11:45,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408633575bc060ce15df36 HTTP/1.1" 502 150 2025-12-17 22:11:45,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408633575bc060ce15df36'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:45,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408637575bc060ce15df76 HTTP/1.1" 502 150 2025-12-17 22:11:45,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408637575bc060ce15df76'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:45,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408637575bc060ce15df76 2025-12-17 22:11:45,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408637575bc060ce15df76 HTTP/1.1" 502 150 2025-12-17 22:11:45,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408637575bc060ce15df76'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:46,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861e575bc060ce15def1 2025-12-17 22:11:46,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861e575bc060ce15def1 HTTP/1.1" 502 150 2025-12-17 22:11:46,369 - lava_callback - ERROR - Error processing callback for node 6940861e575bc060ce15def1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940861e575bc060ce15def1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940861e575bc060ce15def1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940861e575bc060ce15def1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:46,371 - lava_callback - INFO - Starting async job submit for node 69408631575bc060ce15df33 2025-12-17 22:11:46,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:46,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408631575bc060ce15df33 HTTP/1.1" 502 150 2025-12-17 22:11:46,378 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408631575bc060ce15df33'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:46,378 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408631575bc060ce15df33 2025-12-17 22:11:46,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408631575bc060ce15df33 HTTP/1.1" 502 150 2025-12-17 22:11:46,379 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408631575bc060ce15df33'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:46,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862d575bc060ce15df2a 2025-12-17 22:11:46,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862d575bc060ce15df2a HTTP/1.1" 502 150 2025-12-17 22:11:46,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862d575bc060ce15df2a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:46,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861f575bc060ce15def4 HTTP/1.1" 502 150 2025-12-17 22:11:46,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940861f575bc060ce15def4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44208 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:11:46,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862b575bc060ce15df24 2025-12-17 22:11:46,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862b575bc060ce15df24 HTTP/1.1" 502 150 2025-12-17 22:11:46,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862b575bc060ce15df24'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:46,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862f575bc060ce15df2d 2025-12-17 22:11:46,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862f575bc060ce15df2d HTTP/1.1" 502 150 2025-12-17 22:11:46,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862f575bc060ce15df2d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:47,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408637575bc060ce15df76 2025-12-17 22:11:47,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408637575bc060ce15df76 HTTP/1.1" 502 150 2025-12-17 22:11:47,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408637575bc060ce15df76'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:48,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408631575bc060ce15df33 2025-12-17 22:11:48,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408631575bc060ce15df33 HTTP/1.1" 502 150 2025-12-17 22:11:48,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408631575bc060ce15df33'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:50,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408609575bc060ce15debc 2025-12-17 22:11:50,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408609575bc060ce15debc HTTP/1.1" 502 150 2025-12-17 22:11:50,008 - lava_callback - ERROR - Error processing callback for node 69408609575bc060ce15debc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408609575bc060ce15debc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408609575bc060ce15debc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408609575bc060ce15debc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:50,010 - lava_callback - INFO - Starting async job submit for node 69408634575bc060ce15df39 2025-12-17 22:11:50,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:50,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408634575bc060ce15df39 HTTP/1.1" 502 150 2025-12-17 22:11:50,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408634575bc060ce15df39'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:50,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408634575bc060ce15df39 2025-12-17 22:11:50,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408634575bc060ce15df39 HTTP/1.1" 502 150 2025-12-17 22:11:50,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408634575bc060ce15df39'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:50,286 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862c575bc060ce15df27 2025-12-17 22:11:50,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862c575bc060ce15df27 HTTP/1.1" 502 150 2025-12-17 22:11:50,288 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862c575bc060ce15df27'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:50,381 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862d575bc060ce15df2a 2025-12-17 22:11:50,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862d575bc060ce15df2a HTTP/1.1" 502 150 2025-12-17 22:11:50,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862d575bc060ce15df2a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:50,780 - lava_callback - INFO - Received callback for node 69409139575bc060ce1607c1 INFO: 172.21.0.1:45200 - "POST /node/69409139575bc060ce1607c1 HTTP/1.0" 202 Accepted 2025-12-17 22:11:50,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408631575bc060ce15df30 2025-12-17 22:11:50,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408631575bc060ce15df30 HTTP/1.1" 502 150 2025-12-17 22:11:50,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408631575bc060ce15df30'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:50,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862f575bc060ce15df2d 2025-12-17 22:11:50,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862f575bc060ce15df2d HTTP/1.1" 502 150 2025-12-17 22:11:50,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862f575bc060ce15df2d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:51,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408637575bc060ce15df76 2025-12-17 22:11:51,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408637575bc060ce15df76 HTTP/1.1" 502 150 2025-12-17 22:11:51,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408637575bc060ce15df76'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:52,028 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408634575bc060ce15df39 2025-12-17 22:11:52,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408634575bc060ce15df39 HTTP/1.1" 502 150 2025-12-17 22:11:52,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408634575bc060ce15df39'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:52,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408631575bc060ce15df33 2025-12-17 22:11:52,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408631575bc060ce15df33 HTTP/1.1" 502 150 2025-12-17 22:11:52,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408631575bc060ce15df33'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:53,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408633575bc060ce15df36 2025-12-17 22:11:53,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408633575bc060ce15df36 HTTP/1.1" 502 150 2025-12-17 22:11:53,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408633575bc060ce15df36'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:54,068 - lava_callback - INFO - Received callback for node 69408795575bc060ce15e414 INFO: 172.21.0.1:45212 - "POST /node/69408795575bc060ce15e414 HTTP/1.0" 202 Accepted 2025-12-17 22:11:54,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408620575bc060ce15def7 2025-12-17 22:11:54,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408620575bc060ce15def7 HTTP/1.1" 502 150 2025-12-17 22:11:54,608 - lava_callback - ERROR - Error processing callback for node 69408620575bc060ce15def7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408620575bc060ce15def7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408620575bc060ce15def7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408620575bc060ce15def7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:54,610 - lava_callback - INFO - Starting async job submit for node 69408638575bc060ce15df81 2025-12-17 22:11:54,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:54,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408638575bc060ce15df81 HTTP/1.1" 502 150 2025-12-17 22:11:54,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408638575bc060ce15df81'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:54,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408638575bc060ce15df81 2025-12-17 22:11:54,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408638575bc060ce15df81 HTTP/1.1" 502 150 2025-12-17 22:11:54,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408638575bc060ce15df81'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:54,730 - lava_callback - INFO - Received callback for node 69408798575bc060ce15e41b INFO: 172.21.0.1:45224 - "POST /node/69408798575bc060ce15e41b HTTP/1.0" 202 Accepted 2025-12-17 22:11:54,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408621575bc060ce15defa 2025-12-17 22:11:54,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408621575bc060ce15defa HTTP/1.1" 502 150 2025-12-17 22:11:54,817 - lava_callback - ERROR - Error processing callback for node 69408621575bc060ce15defa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408621575bc060ce15defa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408621575bc060ce15defa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408621575bc060ce15defa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:11:54,819 - lava_callback - INFO - Starting async job submit for node 69408639575bc060ce15df86 2025-12-17 22:11:54,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:11:54,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408639575bc060ce15df86 HTTP/1.1" 502 150 2025-12-17 22:11:54,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408639575bc060ce15df86'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:54,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408639575bc060ce15df86 2025-12-17 22:11:54,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408639575bc060ce15df86 HTTP/1.1" 502 150 2025-12-17 22:11:54,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408639575bc060ce15df86'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:55,396 - lava_callback - INFO - Received callback for node 6940879c575bc060ce15e424 INFO: 172.21.0.1:45230 - "POST /node/6940879c575bc060ce15e424 HTTP/1.0" 202 Accepted 2025-12-17 22:11:56,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408634575bc060ce15df39 2025-12-17 22:11:56,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408634575bc060ce15df39 HTTP/1.1" 502 150 2025-12-17 22:11:56,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408634575bc060ce15df39'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:56,069 - lava_callback - INFO - Received callback for node 694087a1575bc060ce15e437 INFO: 172.21.0.1:45242 - "POST /node/694087a1575bc060ce15e437 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:45254 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:11:56,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408638575bc060ce15df81 2025-12-17 22:11:56,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408638575bc060ce15df81 HTTP/1.1" 502 150 2025-12-17 22:11:56,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408638575bc060ce15df81'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:56,748 - lava_callback - INFO - Received callback for node 694087a4575bc060ce15e43a INFO: 172.21.0.1:45256 - "POST /node/694087a4575bc060ce15e43a HTTP/1.0" 202 Accepted 2025-12-17 22:11:56,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408639575bc060ce15df86 2025-12-17 22:11:56,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408639575bc060ce15df86 HTTP/1.1" 502 150 2025-12-17 22:11:56,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408639575bc060ce15df86'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:57,050 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408627575bc060ce15df1b 2025-12-17 22:11:57,428 - lava_callback - INFO - Received callback for node 694087a6575bc060ce15e441 INFO: 172.21.0.1:45270 - "POST /node/694087a6575bc060ce15e441 HTTP/1.0" 202 Accepted 2025-12-17 22:11:58,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862d575bc060ce15df2a 2025-12-17 22:11:58,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862d575bc060ce15df2a HTTP/1.1" 502 150 2025-12-17 22:11:58,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862d575bc060ce15df2a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:58,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408628575bc060ce15df1e HTTP/1.1" 502 150 2025-12-17 22:11:58,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408628575bc060ce15df1e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:58,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862f575bc060ce15df2d 2025-12-17 22:11:58,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862f575bc060ce15df2d HTTP/1.1" 502 150 2025-12-17 22:11:58,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940862f575bc060ce15df2d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:11:59,414 - lava_callback - INFO - Received callback for node 6940879b575bc060ce15e421 INFO: 172.21.0.1:45282 - "POST /node/6940879b575bc060ce15e421 HTTP/1.0" 202 Accepted 2025-12-17 22:11:59,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408637575bc060ce15df76 2025-12-17 22:12:00,381 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408631575bc060ce15df33 2025-12-17 22:12:00,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408631575bc060ce15df33 HTTP/1.1" 502 150 2025-12-17 22:12:00,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408631575bc060ce15df33'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:00,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408638575bc060ce15df81 2025-12-17 22:12:00,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408638575bc060ce15df81 HTTP/1.1" 502 150 2025-12-17 22:12:00,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408638575bc060ce15df81'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:00,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408639575bc060ce15df86 2025-12-17 22:12:00,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408639575bc060ce15df86 HTTP/1.1" 502 150 2025-12-17 22:12:00,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408639575bc060ce15df86'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:01,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861c575bc060ce15deee 2025-12-17 22:12:01,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861c575bc060ce15deee HTTP/1.1" 502 150 2025-12-17 22:12:01,267 - lava_callback - ERROR - Error processing callback for node 6940861c575bc060ce15deee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940861c575bc060ce15deee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940861c575bc060ce15deee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940861c575bc060ce15deee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:01,269 - lava_callback - INFO - Starting async job submit for node 6940863d575bc060ce15df8c 2025-12-17 22:12:01,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:01,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863d575bc060ce15df8c HTTP/1.1" 502 150 2025-12-17 22:12:01,277 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863d575bc060ce15df8c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:01,277 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940863d575bc060ce15df8c 2025-12-17 22:12:01,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863d575bc060ce15df8c HTTP/1.1" 502 150 2025-12-17 22:12:01,278 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863d575bc060ce15df8c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:01,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862a575bc060ce15df21 2025-12-17 22:12:01,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862a575bc060ce15df21 HTTP/1.1" 502 150 2025-12-17 22:12:01,597 - lava_callback - ERROR - Error processing callback for node 6940862a575bc060ce15df21: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940862a575bc060ce15df21 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940862a575bc060ce15df21 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940862a575bc060ce15df21 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:01,599 - lava_callback - INFO - Starting async job submit for node 6940863e575bc060ce15df8f 2025-12-17 22:12:01,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:01,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863e575bc060ce15df8f HTTP/1.1" 502 150 2025-12-17 22:12:01,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863e575bc060ce15df8f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:01,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940863e575bc060ce15df8f 2025-12-17 22:12:01,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863e575bc060ce15df8f HTTP/1.1" 502 150 2025-12-17 22:12:01,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863e575bc060ce15df8f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:32904 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:12:02,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940861f575bc060ce15def4 2025-12-17 22:12:02,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940861f575bc060ce15def4 HTTP/1.1" 502 150 2025-12-17 22:12:02,481 - lava_callback - ERROR - Error processing callback for node 6940861f575bc060ce15def4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940861f575bc060ce15def4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940861f575bc060ce15def4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940861f575bc060ce15def4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:02,483 - lava_callback - INFO - Starting async job submit for node 6940863f575bc060ce15df95 2025-12-17 22:12:02,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:02,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863f575bc060ce15df95 HTTP/1.1" 502 150 2025-12-17 22:12:02,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863f575bc060ce15df95'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:02,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940863f575bc060ce15df95 2025-12-17 22:12:02,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863f575bc060ce15df95 HTTP/1.1" 502 150 2025-12-17 22:12:02,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863f575bc060ce15df95'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:02,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862b575bc060ce15df24 2025-12-17 22:12:02,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862b575bc060ce15df24 HTTP/1.1" 502 150 2025-12-17 22:12:02,875 - lava_callback - ERROR - Error processing callback for node 6940862b575bc060ce15df24: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940862b575bc060ce15df24 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940862b575bc060ce15df24 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940862b575bc060ce15df24 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:02,876 - lava_callback - INFO - Starting async job submit for node 69408641575bc060ce15df9c 2025-12-17 22:12:02,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:02,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408641575bc060ce15df9c HTTP/1.1" 502 150 2025-12-17 22:12:02,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408641575bc060ce15df9c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:02,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408641575bc060ce15df9c 2025-12-17 22:12:02,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408641575bc060ce15df9c HTTP/1.1" 502 150 2025-12-17 22:12:02,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408641575bc060ce15df9c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:03,278 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940863d575bc060ce15df8c 2025-12-17 22:12:03,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863d575bc060ce15df8c HTTP/1.1" 502 150 2025-12-17 22:12:03,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863d575bc060ce15df8c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:03,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940863e575bc060ce15df8f 2025-12-17 22:12:03,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863e575bc060ce15df8f HTTP/1.1" 502 150 2025-12-17 22:12:03,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863e575bc060ce15df8f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:03,939 - lava_callback - INFO - Received callback for node 6940879f575bc060ce15e42e INFO: 172.21.0.1:32912 - "POST /node/6940879f575bc060ce15e42e HTTP/1.0" 202 Accepted 2025-12-17 22:12:04,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408634575bc060ce15df39 2025-12-17 22:12:04,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408634575bc060ce15df39 HTTP/1.1" 502 150 2025-12-17 22:12:04,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408634575bc060ce15df39'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:04,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940863f575bc060ce15df95 2025-12-17 22:12:04,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863f575bc060ce15df95 HTTP/1.1" 502 150 2025-12-17 22:12:04,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863f575bc060ce15df95'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:04,577 - lava_callback - INFO - Received callback for node 694087a9575bc060ce15e484 INFO: 172.21.0.1:32918 - "POST /node/694087a9575bc060ce15e484 HTTP/1.0" 202 Accepted 2025-12-17 22:12:04,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408641575bc060ce15df9c 2025-12-17 22:12:04,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408641575bc060ce15df9c HTTP/1.1" 502 150 2025-12-17 22:12:04,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408641575bc060ce15df9c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:05,239 - lava_callback - INFO - Received callback for node 694087ab575bc060ce15e487 INFO: 172.21.0.1:32920 - "POST /node/694087ab575bc060ce15e487 HTTP/1.0" 202 Accepted 2025-12-17 22:12:05,880 - lava_callback - INFO - Received callback for node 694087ac575bc060ce15e48a INFO: 172.21.0.1:32922 - "POST /node/694087ac575bc060ce15e48a HTTP/1.0" 202 Accepted 2025-12-17 22:12:06,288 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862c575bc060ce15df27 2025-12-17 22:12:06,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862c575bc060ce15df27 HTTP/1.1" 502 150 2025-12-17 22:12:06,289 - lava_callback - ERROR - Error processing callback for node 6940862c575bc060ce15df27: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940862c575bc060ce15df27 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940862c575bc060ce15df27 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940862c575bc060ce15df27 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:06,291 - lava_callback - INFO - Starting async job submit for node 69408647575bc060ce15dfae 2025-12-17 22:12:06,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:06,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408647575bc060ce15dfae HTTP/1.1" 502 150 2025-12-17 22:12:06,297 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408647575bc060ce15dfae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:06,297 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408647575bc060ce15dfae 2025-12-17 22:12:06,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408647575bc060ce15dfae HTTP/1.1" 502 150 2025-12-17 22:12:06,298 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408647575bc060ce15dfae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:06,526 - lava_callback - INFO - Received callback for node 694087ae575bc060ce15e490 INFO: 172.21.0.1:32926 - "POST /node/694087ae575bc060ce15e490 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:32930 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:12:06,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408631575bc060ce15df30 2025-12-17 22:12:06,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408631575bc060ce15df30 HTTP/1.1" 502 150 2025-12-17 22:12:06,906 - lava_callback - ERROR - Error processing callback for node 69408631575bc060ce15df30: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408631575bc060ce15df30 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408631575bc060ce15df30 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408631575bc060ce15df30 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:06,908 - lava_callback - INFO - Starting async job submit for node 69408642575bc060ce15df9f 2025-12-17 22:12:06,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:06,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408642575bc060ce15df9f HTTP/1.1" 502 150 2025-12-17 22:12:06,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408642575bc060ce15df9f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:06,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408642575bc060ce15df9f 2025-12-17 22:12:06,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408642575bc060ce15df9f HTTP/1.1" 502 150 2025-12-17 22:12:06,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408642575bc060ce15df9f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:07,280 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940863d575bc060ce15df8c 2025-12-17 22:12:07,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863d575bc060ce15df8c HTTP/1.1" 502 150 2025-12-17 22:12:07,283 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863d575bc060ce15df8c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:07,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940863e575bc060ce15df8f 2025-12-17 22:12:07,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863e575bc060ce15df8f HTTP/1.1" 502 150 2025-12-17 22:12:07,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863e575bc060ce15df8f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:08,298 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408647575bc060ce15dfae 2025-12-17 22:12:08,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408647575bc060ce15dfae HTTP/1.1" 502 150 2025-12-17 22:12:08,300 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408647575bc060ce15dfae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:08,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940863f575bc060ce15df95 2025-12-17 22:12:08,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863f575bc060ce15df95 HTTP/1.1" 502 150 2025-12-17 22:12:08,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863f575bc060ce15df95'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:08,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408638575bc060ce15df81 2025-12-17 22:12:08,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408638575bc060ce15df81 HTTP/1.1" 502 150 2025-12-17 22:12:08,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408638575bc060ce15df81'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:08,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408639575bc060ce15df86 2025-12-17 22:12:08,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408639575bc060ce15df86 HTTP/1.1" 502 150 2025-12-17 22:12:08,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408639575bc060ce15df86'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:08,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408641575bc060ce15df9c 2025-12-17 22:12:08,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408641575bc060ce15df9c HTTP/1.1" 502 150 2025-12-17 22:12:08,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408641575bc060ce15df9c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:08,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408642575bc060ce15df9f 2025-12-17 22:12:08,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408642575bc060ce15df9f HTTP/1.1" 502 150 2025-12-17 22:12:08,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408642575bc060ce15df9f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:09,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408633575bc060ce15df36 2025-12-17 22:12:09,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408633575bc060ce15df36 HTTP/1.1" 502 150 2025-12-17 22:12:09,940 - lava_callback - ERROR - Error processing callback for node 69408633575bc060ce15df36: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408633575bc060ce15df36 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408633575bc060ce15df36 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408633575bc060ce15df36 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:09,942 - lava_callback - INFO - Starting async job submit for node 6940864a575bc060ce15dfb3 2025-12-17 22:12:09,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:09,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864a575bc060ce15dfb3 HTTP/1.1" 502 150 2025-12-17 22:12:09,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864a575bc060ce15dfb3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:09,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864a575bc060ce15dfb3 2025-12-17 22:12:09,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864a575bc060ce15dfb3 HTTP/1.1" 502 150 2025-12-17 22:12:09,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864a575bc060ce15dfb3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:11,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864a575bc060ce15dfb3 2025-12-17 22:12:11,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864a575bc060ce15dfb3 HTTP/1.1" 502 150 2025-12-17 22:12:11,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864a575bc060ce15dfb3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:12,300 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408647575bc060ce15dfae 2025-12-17 22:12:12,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408647575bc060ce15dfae HTTP/1.1" 502 150 2025-12-17 22:12:12,301 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408647575bc060ce15dfae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:12,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408627575bc060ce15df1b HTTP/1.1" 502 150 2025-12-17 22:12:12,336 - lava_callback - ERROR - Error processing callback for node 69408627575bc060ce15df1b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408627575bc060ce15df1b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408627575bc060ce15df1b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408627575bc060ce15df1b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:12,338 - lava_callback - INFO - Starting async job submit for node 69408643575bc060ce15dfa2 2025-12-17 22:12:12,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:12,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408643575bc060ce15dfa2 HTTP/1.1" 502 150 2025-12-17 22:12:12,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408643575bc060ce15dfa2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:12,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408643575bc060ce15dfa2 2025-12-17 22:12:12,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408643575bc060ce15dfa2 HTTP/1.1" 502 150 2025-12-17 22:12:12,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408643575bc060ce15dfa2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:12,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408642575bc060ce15df9f 2025-12-17 22:12:12,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408642575bc060ce15df9f HTTP/1.1" 502 150 2025-12-17 22:12:12,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408642575bc060ce15df9f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:14,081 - lava_callback - INFO - Received callback for node 694087ad575bc060ce15e48d INFO: 172.21.0.1:44038 - "POST /node/694087ad575bc060ce15e48d HTTP/1.0" 202 Accepted 2025-12-17 22:12:14,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408643575bc060ce15dfa2 2025-12-17 22:12:14,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408643575bc060ce15dfa2 HTTP/1.1" 502 150 2025-12-17 22:12:14,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408643575bc060ce15dfa2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:14,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862d575bc060ce15df2a 2025-12-17 22:12:14,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862d575bc060ce15df2a HTTP/1.1" 502 150 2025-12-17 22:12:14,384 - lava_callback - ERROR - Error processing callback for node 6940862d575bc060ce15df2a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940862d575bc060ce15df2a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940862d575bc060ce15df2a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940862d575bc060ce15df2a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:14,386 - lava_callback - INFO - Starting async job submit for node 69408644575bc060ce15dfa5 2025-12-17 22:12:14,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:14,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408644575bc060ce15dfa5 HTTP/1.1" 502 150 2025-12-17 22:12:14,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408644575bc060ce15dfa5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:14,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408644575bc060ce15dfa5 2025-12-17 22:12:14,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408644575bc060ce15dfa5 HTTP/1.1" 502 150 2025-12-17 22:12:14,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408644575bc060ce15dfa5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:14,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408628575bc060ce15df1e 2025-12-17 22:12:14,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408628575bc060ce15df1e HTTP/1.1" 502 150 2025-12-17 22:12:14,518 - lava_callback - ERROR - Error processing callback for node 69408628575bc060ce15df1e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408628575bc060ce15df1e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408628575bc060ce15df1e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408628575bc060ce15df1e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:14,519 - lava_callback - INFO - Starting async job submit for node 69408645575bc060ce15dfa8 2025-12-17 22:12:14,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:14,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408645575bc060ce15dfa8 HTTP/1.1" 502 150 2025-12-17 22:12:14,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408645575bc060ce15dfa8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:14,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408645575bc060ce15dfa8 2025-12-17 22:12:14,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408645575bc060ce15dfa8 HTTP/1.1" 502 150 2025-12-17 22:12:14,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408645575bc060ce15dfa8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:14,745 - lava_callback - INFO - Received callback for node 694087b1575bc060ce15e496 INFO: 172.21.0.1:44042 - "POST /node/694087b1575bc060ce15e496 HTTP/1.0" 202 Accepted 2025-12-17 22:12:14,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940862f575bc060ce15df2d 2025-12-17 22:12:14,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940862f575bc060ce15df2d HTTP/1.1" 502 150 2025-12-17 22:12:14,933 - lava_callback - ERROR - Error processing callback for node 6940862f575bc060ce15df2d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940862f575bc060ce15df2d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940862f575bc060ce15df2d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940862f575bc060ce15df2d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:14,934 - lava_callback - INFO - Starting async job submit for node 69408647575bc060ce15dfab 2025-12-17 22:12:14,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:14,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408647575bc060ce15dfab HTTP/1.1" 502 150 2025-12-17 22:12:14,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408647575bc060ce15dfab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:14,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408647575bc060ce15dfab 2025-12-17 22:12:14,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408647575bc060ce15dfab HTTP/1.1" 502 150 2025-12-17 22:12:14,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408647575bc060ce15dfab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44052 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:12:15,283 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940863d575bc060ce15df8c 2025-12-17 22:12:15,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863d575bc060ce15df8c HTTP/1.1" 502 150 2025-12-17 22:12:15,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863d575bc060ce15df8c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:15,403 - lava_callback - INFO - Received callback for node 694087b5575bc060ce15e4ab INFO: 172.21.0.1:44054 - "POST /node/694087b5575bc060ce15e4ab HTTP/1.0" 202 Accepted 2025-12-17 22:12:15,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408637575bc060ce15df76 HTTP/1.1" 502 150 2025-12-17 22:12:15,410 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408637575bc060ce15df76'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:15,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940863e575bc060ce15df8f 2025-12-17 22:12:15,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863e575bc060ce15df8f HTTP/1.1" 502 150 2025-12-17 22:12:15,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863e575bc060ce15df8f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:15,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864a575bc060ce15dfb3 2025-12-17 22:12:15,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864a575bc060ce15dfb3 HTTP/1.1" 502 150 2025-12-17 22:12:15,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864a575bc060ce15dfb3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:16,048 - lava_callback - INFO - Received callback for node 694087bc575bc060ce15e4b4 INFO: 172.21.0.1:44056 - "POST /node/694087bc575bc060ce15e4b4 HTTP/1.0" 202 Accepted 2025-12-17 22:12:16,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408631575bc060ce15df33 2025-12-17 22:12:16,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408631575bc060ce15df33 HTTP/1.1" 502 150 2025-12-17 22:12:16,384 - lava_callback - ERROR - Error processing callback for node 69408631575bc060ce15df33: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408631575bc060ce15df33 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408631575bc060ce15df33 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408631575bc060ce15df33 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:16,386 - lava_callback - INFO - Starting async job submit for node 6940864d575bc060ce15dfbe 2025-12-17 22:12:16,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:16,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864d575bc060ce15dfbe HTTP/1.1" 502 150 2025-12-17 22:12:16,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864d575bc060ce15dfbe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:16,393 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864d575bc060ce15dfbe 2025-12-17 22:12:16,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864d575bc060ce15dfbe HTTP/1.1" 502 150 2025-12-17 22:12:16,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864d575bc060ce15dfbe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:16,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408644575bc060ce15dfa5 2025-12-17 22:12:16,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408644575bc060ce15dfa5 HTTP/1.1" 502 150 2025-12-17 22:12:16,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408644575bc060ce15dfa5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:16,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940863f575bc060ce15df95 2025-12-17 22:12:16,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863f575bc060ce15df95 HTTP/1.1" 502 150 2025-12-17 22:12:16,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940863f575bc060ce15df95'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44060 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:12:16,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408645575bc060ce15dfa8 2025-12-17 22:12:16,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408645575bc060ce15dfa8 HTTP/1.1" 502 150 2025-12-17 22:12:16,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408645575bc060ce15dfa8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:16,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408641575bc060ce15df9c 2025-12-17 22:12:16,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408641575bc060ce15df9c HTTP/1.1" 502 150 2025-12-17 22:12:16,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408641575bc060ce15df9c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:16,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408647575bc060ce15dfab 2025-12-17 22:12:16,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408647575bc060ce15dfab HTTP/1.1" 502 150 2025-12-17 22:12:16,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408647575bc060ce15dfab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:18,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408643575bc060ce15dfa2 2025-12-17 22:12:18,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408643575bc060ce15dfa2 HTTP/1.1" 502 150 2025-12-17 22:12:18,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408643575bc060ce15dfa2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:18,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864d575bc060ce15dfbe 2025-12-17 22:12:18,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864d575bc060ce15dfbe HTTP/1.1" 502 150 2025-12-17 22:12:18,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864d575bc060ce15dfbe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:19,538 - lava_callback - INFO - Received callback for node 694087b0575bc060ce15e493 INFO: 172.21.0.1:44074 - "POST /node/694087b0575bc060ce15e493 HTTP/1.0" 202 Accepted 2025-12-17 22:12:20,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408634575bc060ce15df39 2025-12-17 22:12:20,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408634575bc060ce15df39 HTTP/1.1" 502 150 2025-12-17 22:12:20,033 - lava_callback - ERROR - Error processing callback for node 69408634575bc060ce15df39: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408634575bc060ce15df39 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408634575bc060ce15df39 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408634575bc060ce15df39 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:20,035 - lava_callback - INFO - Starting async job submit for node 69408658575bc060ce15dfda 2025-12-17 22:12:20,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:20,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408658575bc060ce15dfda HTTP/1.1" 502 150 2025-12-17 22:12:20,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408658575bc060ce15dfda'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:20,042 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408658575bc060ce15dfda 2025-12-17 22:12:20,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408658575bc060ce15dfda HTTP/1.1" 502 150 2025-12-17 22:12:20,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408658575bc060ce15dfda'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:20,301 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408647575bc060ce15dfae 2025-12-17 22:12:20,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408647575bc060ce15dfae HTTP/1.1" 502 150 2025-12-17 22:12:20,303 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408647575bc060ce15dfae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:20,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408644575bc060ce15dfa5 2025-12-17 22:12:20,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408644575bc060ce15dfa5 HTTP/1.1" 502 150 2025-12-17 22:12:20,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408644575bc060ce15dfa5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:20,430 - lava_callback - INFO - Received callback for node 694087b2575bc060ce15e49b INFO: 172.21.0.1:53336 - "POST /node/694087b2575bc060ce15e49b HTTP/1.0" 202 Accepted 2025-12-17 22:12:20,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408645575bc060ce15dfa8 2025-12-17 22:12:20,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408645575bc060ce15dfa8 HTTP/1.1" 502 150 2025-12-17 22:12:20,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408645575bc060ce15dfa8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:20,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408642575bc060ce15df9f 2025-12-17 22:12:20,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408642575bc060ce15df9f HTTP/1.1" 502 150 2025-12-17 22:12:20,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408642575bc060ce15df9f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:20,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408647575bc060ce15dfab 2025-12-17 22:12:20,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408647575bc060ce15dfab HTTP/1.1" 502 150 2025-12-17 22:12:20,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408647575bc060ce15dfab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:21,287 - lava_callback - INFO - Received callback for node 694087b4575bc060ce15e4a6 INFO: 172.21.0.1:53350 - "POST /node/694087b4575bc060ce15e4a6 HTTP/1.0" 202 Accepted 2025-12-17 22:12:22,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408658575bc060ce15dfda 2025-12-17 22:12:22,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408658575bc060ce15dfda HTTP/1.1" 502 150 2025-12-17 22:12:22,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408658575bc060ce15dfda'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:22,395 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864d575bc060ce15dfbe 2025-12-17 22:12:22,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864d575bc060ce15dfbe HTTP/1.1" 502 150 2025-12-17 22:12:22,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864d575bc060ce15dfbe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:23,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864a575bc060ce15dfb3 2025-12-17 22:12:23,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864a575bc060ce15dfb3 HTTP/1.1" 502 150 2025-12-17 22:12:23,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864a575bc060ce15dfb3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:24,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408638575bc060ce15df81 2025-12-17 22:12:24,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408638575bc060ce15df81 HTTP/1.1" 502 150 2025-12-17 22:12:24,624 - lava_callback - ERROR - Error processing callback for node 69408638575bc060ce15df81: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408638575bc060ce15df81 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408638575bc060ce15df81 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408638575bc060ce15df81 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:24,626 - lava_callback - INFO - Starting async job submit for node 69408659575bc060ce15dfdd 2025-12-17 22:12:24,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:24,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408659575bc060ce15dfdd HTTP/1.1" 502 150 2025-12-17 22:12:24,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408659575bc060ce15dfdd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:24,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408659575bc060ce15dfdd 2025-12-17 22:12:24,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408659575bc060ce15dfdd HTTP/1.1" 502 150 2025-12-17 22:12:24,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408659575bc060ce15dfdd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:24,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408639575bc060ce15df86 2025-12-17 22:12:24,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408639575bc060ce15df86 HTTP/1.1" 502 150 2025-12-17 22:12:24,832 - lava_callback - ERROR - Error processing callback for node 69408639575bc060ce15df86: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408639575bc060ce15df86 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408639575bc060ce15df86 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408639575bc060ce15df86 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:24,834 - lava_callback - INFO - Starting async job submit for node 6940865a575bc060ce15dfe0 2025-12-17 22:12:24,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:24,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865a575bc060ce15dfe0 HTTP/1.1" 502 150 2025-12-17 22:12:24,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865a575bc060ce15dfe0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:24,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865a575bc060ce15dfe0 2025-12-17 22:12:24,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865a575bc060ce15dfe0 HTTP/1.1" 502 150 2025-12-17 22:12:24,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865a575bc060ce15dfe0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:26,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408658575bc060ce15dfda 2025-12-17 22:12:26,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408658575bc060ce15dfda HTTP/1.1" 502 150 2025-12-17 22:12:26,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408658575bc060ce15dfda'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:26,351 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408643575bc060ce15dfa2 2025-12-17 22:12:26,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408643575bc060ce15dfa2 HTTP/1.1" 502 150 2025-12-17 22:12:26,352 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408643575bc060ce15dfa2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53356 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:12:26,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408659575bc060ce15dfdd 2025-12-17 22:12:26,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408659575bc060ce15dfdd HTTP/1.1" 502 150 2025-12-17 22:12:26,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408659575bc060ce15dfdd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:26,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865a575bc060ce15dfe0 2025-12-17 22:12:26,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865a575bc060ce15dfe0 HTTP/1.1" 502 150 2025-12-17 22:12:26,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865a575bc060ce15dfe0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:28,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408644575bc060ce15dfa5 2025-12-17 22:12:28,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408644575bc060ce15dfa5 HTTP/1.1" 502 150 2025-12-17 22:12:28,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408644575bc060ce15dfa5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:28,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408645575bc060ce15dfa8 2025-12-17 22:12:28,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408645575bc060ce15dfa8 HTTP/1.1" 502 150 2025-12-17 22:12:28,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408645575bc060ce15dfa8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:28,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408647575bc060ce15dfab 2025-12-17 22:12:28,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408647575bc060ce15dfab HTTP/1.1" 502 150 2025-12-17 22:12:28,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408647575bc060ce15dfab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:30,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864d575bc060ce15dfbe 2025-12-17 22:12:30,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864d575bc060ce15dfbe HTTP/1.1" 502 150 2025-12-17 22:12:30,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864d575bc060ce15dfbe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:30,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408659575bc060ce15dfdd 2025-12-17 22:12:30,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408659575bc060ce15dfdd HTTP/1.1" 502 150 2025-12-17 22:12:30,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408659575bc060ce15dfdd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:30,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865a575bc060ce15dfe0 2025-12-17 22:12:30,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865a575bc060ce15dfe0 HTTP/1.1" 502 150 2025-12-17 22:12:30,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865a575bc060ce15dfe0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:31,284 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940863d575bc060ce15df8c 2025-12-17 22:12:31,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863d575bc060ce15df8c HTTP/1.1" 502 150 2025-12-17 22:12:31,286 - lava_callback - ERROR - Error processing callback for node 6940863d575bc060ce15df8c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940863d575bc060ce15df8c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940863d575bc060ce15df8c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940863d575bc060ce15df8c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:31,288 - lava_callback - INFO - Starting async job submit for node 6940865c575bc060ce15dfe3 2025-12-17 22:12:31,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:31,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865c575bc060ce15dfe3 HTTP/1.1" 502 150 2025-12-17 22:12:31,295 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865c575bc060ce15dfe3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:31,295 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865c575bc060ce15dfe3 2025-12-17 22:12:31,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865c575bc060ce15dfe3 HTTP/1.1" 502 150 2025-12-17 22:12:31,295 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865c575bc060ce15dfe3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:31,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408637575bc060ce15df76 2025-12-17 22:12:31,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408637575bc060ce15df76 HTTP/1.1" 502 150 2025-12-17 22:12:31,411 - lava_callback - ERROR - Error processing callback for node 69408637575bc060ce15df76: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408637575bc060ce15df76 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408637575bc060ce15df76 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408637575bc060ce15df76 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:31,413 - lava_callback - INFO - Starting async job submit for node 694081dd575bc060ce15cc11 2025-12-17 22:12:31,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:31,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081dd575bc060ce15cc11 HTTP/1.1" 502 150 2025-12-17 22:12:31,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081dd575bc060ce15cc11'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:31,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081dd575bc060ce15cc11 2025-12-17 22:12:31,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081dd575bc060ce15cc11 HTTP/1.1" 502 150 2025-12-17 22:12:31,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081dd575bc060ce15cc11'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:31,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940863e575bc060ce15df8f 2025-12-17 22:12:31,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863e575bc060ce15df8f HTTP/1.1" 502 150 2025-12-17 22:12:31,635 - lava_callback - ERROR - Error processing callback for node 6940863e575bc060ce15df8f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940863e575bc060ce15df8f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940863e575bc060ce15df8f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940863e575bc060ce15df8f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:31,637 - lava_callback - INFO - Starting async job submit for node 6940864e575bc060ce15dfc3 2025-12-17 22:12:31,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:31,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864e575bc060ce15dfc3 HTTP/1.1" 502 150 2025-12-17 22:12:31,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864e575bc060ce15dfc3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:31,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864e575bc060ce15dfc3 2025-12-17 22:12:31,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864e575bc060ce15dfc3 HTTP/1.1" 502 150 2025-12-17 22:12:31,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864e575bc060ce15dfc3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:32,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940863f575bc060ce15df95 2025-12-17 22:12:32,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940863f575bc060ce15df95 HTTP/1.1" 502 150 2025-12-17 22:12:32,506 - lava_callback - ERROR - Error processing callback for node 6940863f575bc060ce15df95: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940863f575bc060ce15df95 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940863f575bc060ce15df95 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940863f575bc060ce15df95 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:32,508 - lava_callback - INFO - Starting async job submit for node 6940864f575bc060ce15dfc8 2025-12-17 22:12:32,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:32,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864f575bc060ce15dfc8 HTTP/1.1" 502 150 2025-12-17 22:12:32,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864f575bc060ce15dfc8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:32,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864f575bc060ce15dfc8 2025-12-17 22:12:32,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864f575bc060ce15dfc8 HTTP/1.1" 502 150 2025-12-17 22:12:32,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864f575bc060ce15dfc8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:32,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408641575bc060ce15df9c 2025-12-17 22:12:32,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408641575bc060ce15df9c HTTP/1.1" 502 150 2025-12-17 22:12:32,892 - lava_callback - ERROR - Error processing callback for node 69408641575bc060ce15df9c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408641575bc060ce15df9c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408641575bc060ce15df9c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408641575bc060ce15df9c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:32,894 - lava_callback - INFO - Starting async job submit for node 69408650575bc060ce15dfcb 2025-12-17 22:12:32,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:32,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408650575bc060ce15dfcb HTTP/1.1" 502 150 2025-12-17 22:12:32,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408650575bc060ce15dfcb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:32,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408650575bc060ce15dfcb 2025-12-17 22:12:32,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408650575bc060ce15dfcb HTTP/1.1" 502 150 2025-12-17 22:12:32,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408650575bc060ce15dfcb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:33,296 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865c575bc060ce15dfe3 2025-12-17 22:12:33,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865c575bc060ce15dfe3 HTTP/1.1" 502 150 2025-12-17 22:12:33,297 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865c575bc060ce15dfe3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:33,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081dd575bc060ce15cc11 2025-12-17 22:12:33,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081dd575bc060ce15cc11 HTTP/1.1" 502 150 2025-12-17 22:12:33,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081dd575bc060ce15cc11'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:33,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864e575bc060ce15dfc3 2025-12-17 22:12:33,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864e575bc060ce15dfc3 HTTP/1.1" 502 150 2025-12-17 22:12:33,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864e575bc060ce15dfc3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:34,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408658575bc060ce15dfda 2025-12-17 22:12:34,101 - lava_callback - INFO - Received callback for node 694087be575bc060ce15e4b7 INFO: 172.21.0.1:44360 - "POST /node/694087be575bc060ce15e4b7 HTTP/1.0" 202 Accepted 2025-12-17 22:12:34,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864f575bc060ce15dfc8 2025-12-17 22:12:34,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864f575bc060ce15dfc8 HTTP/1.1" 502 150 2025-12-17 22:12:34,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864f575bc060ce15dfc8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:34,793 - lava_callback - INFO - Received callback for node 694087ca575bc060ce15e4c9 INFO: 172.21.0.1:44364 - "POST /node/694087ca575bc060ce15e4c9 HTTP/1.0" 202 Accepted 2025-12-17 22:12:34,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408650575bc060ce15dfcb 2025-12-17 22:12:34,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408650575bc060ce15dfcb HTTP/1.1" 502 150 2025-12-17 22:12:34,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408650575bc060ce15dfcb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:36,303 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408647575bc060ce15dfae 2025-12-17 22:12:36,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408647575bc060ce15dfae HTTP/1.1" 502 150 2025-12-17 22:12:36,305 - lava_callback - ERROR - Error processing callback for node 69408647575bc060ce15dfae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408647575bc060ce15dfae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408647575bc060ce15dfae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408647575bc060ce15dfae (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:36,307 - lava_callback - INFO - Starting async job submit for node 69408652575bc060ce15dfce 2025-12-17 22:12:36,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:36,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408652575bc060ce15dfce HTTP/1.1" 502 150 2025-12-17 22:12:36,315 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408652575bc060ce15dfce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:36,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408652575bc060ce15dfce 2025-12-17 22:12:36,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408652575bc060ce15dfce HTTP/1.1" 502 150 2025-12-17 22:12:36,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408652575bc060ce15dfce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44372 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:12:36,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408642575bc060ce15df9f 2025-12-17 22:12:36,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408642575bc060ce15df9f HTTP/1.1" 502 150 2025-12-17 22:12:36,922 - lava_callback - ERROR - Error processing callback for node 69408642575bc060ce15df9f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408642575bc060ce15df9f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408642575bc060ce15df9f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408642575bc060ce15df9f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:36,925 - lava_callback - INFO - Starting async job submit for node 69408655575bc060ce15dfd4 2025-12-17 22:12:36,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:36,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408655575bc060ce15dfd4 HTTP/1.1" 502 150 2025-12-17 22:12:36,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408655575bc060ce15dfd4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:36,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408655575bc060ce15dfd4 2025-12-17 22:12:36,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408655575bc060ce15dfd4 HTTP/1.1" 502 150 2025-12-17 22:12:36,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408655575bc060ce15dfd4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:37,297 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865c575bc060ce15dfe3 2025-12-17 22:12:37,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865c575bc060ce15dfe3 HTTP/1.1" 502 150 2025-12-17 22:12:37,298 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865c575bc060ce15dfe3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:37,447 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081dd575bc060ce15cc11 2025-12-17 22:12:37,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081dd575bc060ce15cc11 HTTP/1.1" 502 150 2025-12-17 22:12:37,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081dd575bc060ce15cc11'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:37,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864e575bc060ce15dfc3 2025-12-17 22:12:37,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864e575bc060ce15dfc3 HTTP/1.1" 502 150 2025-12-17 22:12:37,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864e575bc060ce15dfc3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:38,317 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408652575bc060ce15dfce 2025-12-17 22:12:38,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408652575bc060ce15dfce HTTP/1.1" 502 150 2025-12-17 22:12:38,318 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408652575bc060ce15dfce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:38,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864f575bc060ce15dfc8 2025-12-17 22:12:38,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864f575bc060ce15dfc8 HTTP/1.1" 502 150 2025-12-17 22:12:38,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864f575bc060ce15dfc8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:38,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408659575bc060ce15dfdd 2025-12-17 22:12:38,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408659575bc060ce15dfdd HTTP/1.1" 502 150 2025-12-17 22:12:38,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408659575bc060ce15dfdd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:38,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865a575bc060ce15dfe0 2025-12-17 22:12:38,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865a575bc060ce15dfe0 HTTP/1.1" 502 150 2025-12-17 22:12:38,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865a575bc060ce15dfe0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:38,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408650575bc060ce15dfcb 2025-12-17 22:12:38,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408650575bc060ce15dfcb HTTP/1.1" 502 150 2025-12-17 22:12:38,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408650575bc060ce15dfcb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:38,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408655575bc060ce15dfd4 2025-12-17 22:12:38,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408655575bc060ce15dfd4 HTTP/1.1" 502 150 2025-12-17 22:12:38,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408655575bc060ce15dfd4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:39,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864a575bc060ce15dfb3 2025-12-17 22:12:39,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864a575bc060ce15dfb3 HTTP/1.1" 502 150 2025-12-17 22:12:39,956 - lava_callback - ERROR - Error processing callback for node 6940864a575bc060ce15dfb3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940864a575bc060ce15dfb3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940864a575bc060ce15dfb3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940864a575bc060ce15dfb3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:39,958 - lava_callback - INFO - Starting async job submit for node 69408656575bc060ce15dfd7 2025-12-17 22:12:39,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:39,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408656575bc060ce15dfd7 HTTP/1.1" 502 150 2025-12-17 22:12:39,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408656575bc060ce15dfd7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:39,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408656575bc060ce15dfd7 2025-12-17 22:12:39,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408656575bc060ce15dfd7 HTTP/1.1" 502 150 2025-12-17 22:12:39,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408656575bc060ce15dfd7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:41,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408656575bc060ce15dfd7 2025-12-17 22:12:41,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408656575bc060ce15dfd7 HTTP/1.1" 502 150 2025-12-17 22:12:41,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408656575bc060ce15dfd7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:42,318 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408652575bc060ce15dfce 2025-12-17 22:12:42,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408652575bc060ce15dfce HTTP/1.1" 502 150 2025-12-17 22:12:42,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408652575bc060ce15dfce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:42,352 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408643575bc060ce15dfa2 2025-12-17 22:12:42,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408643575bc060ce15dfa2 HTTP/1.1" 502 150 2025-12-17 22:12:42,354 - lava_callback - ERROR - Error processing callback for node 69408643575bc060ce15dfa2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408643575bc060ce15dfa2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408643575bc060ce15dfa2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408643575bc060ce15dfa2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:42,356 - lava_callback - INFO - Starting async job submit for node 69408663575bc060ce15e008 2025-12-17 22:12:42,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:42,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408663575bc060ce15e008 HTTP/1.1" 502 150 2025-12-17 22:12:42,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408663575bc060ce15e008'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:42,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408663575bc060ce15e008 2025-12-17 22:12:42,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408663575bc060ce15e008 HTTP/1.1" 502 150 2025-12-17 22:12:42,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408663575bc060ce15e008'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:42,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408655575bc060ce15dfd4 2025-12-17 22:12:42,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408655575bc060ce15dfd4 HTTP/1.1" 502 150 2025-12-17 22:12:42,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408655575bc060ce15dfd4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:43,958 - lava_callback - INFO - Received callback for node 694087cf575bc060ce15e4d7 INFO: 172.21.0.1:38320 - "POST /node/694087cf575bc060ce15e4d7 HTTP/1.0" 202 Accepted 2025-12-17 22:12:44,363 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408663575bc060ce15e008 2025-12-17 22:12:44,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408663575bc060ce15e008 HTTP/1.1" 502 150 2025-12-17 22:12:44,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408663575bc060ce15e008'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:44,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408644575bc060ce15dfa5 2025-12-17 22:12:44,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408644575bc060ce15dfa5 HTTP/1.1" 502 150 2025-12-17 22:12:44,421 - lava_callback - ERROR - Error processing callback for node 69408644575bc060ce15dfa5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408644575bc060ce15dfa5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408644575bc060ce15dfa5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408644575bc060ce15dfa5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:44,423 - lava_callback - INFO - Starting async job submit for node 6940865d575bc060ce15dfe6 2025-12-17 22:12:44,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:44,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865d575bc060ce15dfe6 HTTP/1.1" 502 150 2025-12-17 22:12:44,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865d575bc060ce15dfe6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:44,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865d575bc060ce15dfe6 2025-12-17 22:12:44,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865d575bc060ce15dfe6 HTTP/1.1" 502 150 2025-12-17 22:12:44,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865d575bc060ce15dfe6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:44,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408645575bc060ce15dfa8 2025-12-17 22:12:44,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408645575bc060ce15dfa8 HTTP/1.1" 502 150 2025-12-17 22:12:44,537 - lava_callback - ERROR - Error processing callback for node 69408645575bc060ce15dfa8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408645575bc060ce15dfa8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408645575bc060ce15dfa8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408645575bc060ce15dfa8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:44,539 - lava_callback - INFO - Starting async job submit for node 6940865e575bc060ce15dfeb 2025-12-17 22:12:44,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:44,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865e575bc060ce15dfeb HTTP/1.1" 502 150 2025-12-17 22:12:44,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865e575bc060ce15dfeb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:44,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865e575bc060ce15dfeb 2025-12-17 22:12:44,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865e575bc060ce15dfeb HTTP/1.1" 502 150 2025-12-17 22:12:44,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865e575bc060ce15dfeb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:44,607 - lava_callback - INFO - Received callback for node 694087d2575bc060ce15e4da INFO: 172.21.0.1:38326 - "POST /node/694087d2575bc060ce15e4da HTTP/1.0" 202 Accepted 2025-12-17 22:12:44,892 - lava_callback - INFO - Received callback for node 69409174575bc060ce16084f INFO: 172.21.0.1:38330 - "POST /node/69409174575bc060ce16084f HTTP/1.0" 202 Accepted 2025-12-17 22:12:44,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408647575bc060ce15dfab 2025-12-17 22:12:44,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408647575bc060ce15dfab HTTP/1.1" 502 150 2025-12-17 22:12:44,946 - lava_callback - ERROR - Error processing callback for node 69408647575bc060ce15dfab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408647575bc060ce15dfab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408647575bc060ce15dfab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408647575bc060ce15dfab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:44,948 - lava_callback - INFO - Starting async job submit for node 69408660575bc060ce15dff5 2025-12-17 22:12:44,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:44,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408660575bc060ce15dff5 HTTP/1.1" 502 150 2025-12-17 22:12:44,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408660575bc060ce15dff5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:44,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408660575bc060ce15dff5 2025-12-17 22:12:44,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408660575bc060ce15dff5 HTTP/1.1" 502 150 2025-12-17 22:12:44,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408660575bc060ce15dff5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:45,254 - lava_callback - INFO - Received callback for node 694087da575bc060ce15e4e0 INFO: 172.21.0.1:38334 - "POST /node/694087da575bc060ce15e4e0 HTTP/1.0" 202 Accepted 2025-12-17 22:12:45,298 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865c575bc060ce15dfe3 2025-12-17 22:12:45,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865c575bc060ce15dfe3 HTTP/1.1" 502 150 2025-12-17 22:12:45,299 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865c575bc060ce15dfe3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:45,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081dd575bc060ce15cc11 2025-12-17 22:12:45,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081dd575bc060ce15cc11 HTTP/1.1" 502 150 2025-12-17 22:12:45,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694081dd575bc060ce15cc11'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:45,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864e575bc060ce15dfc3 2025-12-17 22:12:45,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864e575bc060ce15dfc3 HTTP/1.1" 502 150 2025-12-17 22:12:45,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864e575bc060ce15dfc3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:45,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408656575bc060ce15dfd7 2025-12-17 22:12:45,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408656575bc060ce15dfd7 HTTP/1.1" 502 150 2025-12-17 22:12:45,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408656575bc060ce15dfd7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:46,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864d575bc060ce15dfbe 2025-12-17 22:12:46,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864d575bc060ce15dfbe HTTP/1.1" 502 150 2025-12-17 22:12:46,400 - lava_callback - ERROR - Error processing callback for node 6940864d575bc060ce15dfbe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940864d575bc060ce15dfbe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940864d575bc060ce15dfbe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940864d575bc060ce15dfbe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:46,402 - lava_callback - INFO - Starting async job submit for node 69408661575bc060ce15dffc 2025-12-17 22:12:46,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:46,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408661575bc060ce15dffc HTTP/1.1" 502 150 2025-12-17 22:12:46,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408661575bc060ce15dffc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:46,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408661575bc060ce15dffc 2025-12-17 22:12:46,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408661575bc060ce15dffc HTTP/1.1" 502 150 2025-12-17 22:12:46,410 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408661575bc060ce15dffc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:46,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865d575bc060ce15dfe6 2025-12-17 22:12:46,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865d575bc060ce15dfe6 HTTP/1.1" 502 150 2025-12-17 22:12:46,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865d575bc060ce15dfe6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38348 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:12:46,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864f575bc060ce15dfc8 2025-12-17 22:12:46,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864f575bc060ce15dfc8 HTTP/1.1" 502 150 2025-12-17 22:12:46,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940864f575bc060ce15dfc8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:46,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865e575bc060ce15dfeb 2025-12-17 22:12:46,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865e575bc060ce15dfeb HTTP/1.1" 502 150 2025-12-17 22:12:46,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865e575bc060ce15dfeb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:46,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408650575bc060ce15dfcb 2025-12-17 22:12:46,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408650575bc060ce15dfcb HTTP/1.1" 502 150 2025-12-17 22:12:46,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408650575bc060ce15dfcb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:46,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408660575bc060ce15dff5 2025-12-17 22:12:46,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408660575bc060ce15dff5 HTTP/1.1" 502 150 2025-12-17 22:12:46,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408660575bc060ce15dff5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:48,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408663575bc060ce15e008 2025-12-17 22:12:48,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408663575bc060ce15e008 HTTP/1.1" 502 150 2025-12-17 22:12:48,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408663575bc060ce15e008'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:48,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408661575bc060ce15dffc 2025-12-17 22:12:48,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408661575bc060ce15dffc HTTP/1.1" 502 150 2025-12-17 22:12:48,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408661575bc060ce15dffc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:49,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408658575bc060ce15dfda HTTP/1.1" 502 150 2025-12-17 22:12:49,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408658575bc060ce15dfda'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:50,320 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408652575bc060ce15dfce 2025-12-17 22:12:50,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408652575bc060ce15dfce HTTP/1.1" 502 150 2025-12-17 22:12:50,321 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408652575bc060ce15dfce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:50,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865d575bc060ce15dfe6 2025-12-17 22:12:50,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865d575bc060ce15dfe6 HTTP/1.1" 502 150 2025-12-17 22:12:50,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865d575bc060ce15dfe6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:50,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865e575bc060ce15dfeb 2025-12-17 22:12:50,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865e575bc060ce15dfeb HTTP/1.1" 502 150 2025-12-17 22:12:50,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865e575bc060ce15dfeb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:50,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408660575bc060ce15dff5 2025-12-17 22:12:50,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408660575bc060ce15dff5 HTTP/1.1" 502 150 2025-12-17 22:12:50,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408660575bc060ce15dff5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:50,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408655575bc060ce15dfd4 2025-12-17 22:12:50,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408655575bc060ce15dfd4 HTTP/1.1" 502 150 2025-12-17 22:12:50,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408655575bc060ce15dfd4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:52,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408661575bc060ce15dffc 2025-12-17 22:12:53,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408656575bc060ce15dfd7 2025-12-17 22:12:53,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408656575bc060ce15dfd7 HTTP/1.1" 502 150 2025-12-17 22:12:53,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408656575bc060ce15dfd7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:54,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408659575bc060ce15dfdd 2025-12-17 22:12:54,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408659575bc060ce15dfdd HTTP/1.1" 502 150 2025-12-17 22:12:54,652 - lava_callback - ERROR - Error processing callback for node 69408659575bc060ce15dfdd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408659575bc060ce15dfdd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408659575bc060ce15dfdd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408659575bc060ce15dfdd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:54,654 - lava_callback - INFO - Starting async job submit for node 69408663575bc060ce15e003 2025-12-17 22:12:54,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:54,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408663575bc060ce15e003 HTTP/1.1" 502 150 2025-12-17 22:12:54,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408663575bc060ce15e003'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:54,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408663575bc060ce15e003 2025-12-17 22:12:54,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408663575bc060ce15e003 HTTP/1.1" 502 150 2025-12-17 22:12:54,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408663575bc060ce15e003'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:54,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865a575bc060ce15dfe0 2025-12-17 22:12:54,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865a575bc060ce15dfe0 HTTP/1.1" 502 150 2025-12-17 22:12:54,850 - lava_callback - ERROR - Error processing callback for node 6940865a575bc060ce15dfe0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940865a575bc060ce15dfe0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940865a575bc060ce15dfe0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940865a575bc060ce15dfe0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:12:54,851 - lava_callback - INFO - Starting async job submit for node 69408666575bc060ce15e015 2025-12-17 22:12:54,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:12:54,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408666575bc060ce15e015 HTTP/1.1" 502 150 2025-12-17 22:12:54,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408666575bc060ce15e015'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:54,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408666575bc060ce15e015 2025-12-17 22:12:54,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408666575bc060ce15e015 HTTP/1.1" 502 150 2025-12-17 22:12:54,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408666575bc060ce15e015'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:56,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408663575bc060ce15e008 2025-12-17 22:12:56,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408663575bc060ce15e008 HTTP/1.1" 502 150 2025-12-17 22:12:56,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408663575bc060ce15e008'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36988 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:12:56,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408663575bc060ce15e003 2025-12-17 22:12:56,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408663575bc060ce15e003 HTTP/1.1" 502 150 2025-12-17 22:12:56,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408663575bc060ce15e003'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:56,828 - lava_callback - INFO - Received callback for node 6940927c575bc060ce160acb INFO: 172.21.0.1:36998 - "POST /node/6940927c575bc060ce160acb HTTP/1.0" 202 Accepted 2025-12-17 22:12:56,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408666575bc060ce15e015 2025-12-17 22:12:56,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408666575bc060ce15e015 HTTP/1.1" 502 150 2025-12-17 22:12:56,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408666575bc060ce15e015'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:58,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865d575bc060ce15dfe6 2025-12-17 22:12:58,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865d575bc060ce15dfe6 HTTP/1.1" 502 150 2025-12-17 22:12:58,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865d575bc060ce15dfe6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:58,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865e575bc060ce15dfeb 2025-12-17 22:12:58,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865e575bc060ce15dfeb HTTP/1.1" 502 150 2025-12-17 22:12:58,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940865e575bc060ce15dfeb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:12:58,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408660575bc060ce15dff5 2025-12-17 22:12:58,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408660575bc060ce15dff5 HTTP/1.1" 502 150 2025-12-17 22:12:58,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408660575bc060ce15dff5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:00,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408663575bc060ce15e003 2025-12-17 22:13:00,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408666575bc060ce15e015 2025-12-17 22:13:00,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408666575bc060ce15e015 HTTP/1.1" 502 150 2025-12-17 22:13:00,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408666575bc060ce15e015'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:01,299 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865c575bc060ce15dfe3 2025-12-17 22:13:01,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865c575bc060ce15dfe3 HTTP/1.1" 502 150 2025-12-17 22:13:01,301 - lava_callback - ERROR - Error processing callback for node 6940865c575bc060ce15dfe3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940865c575bc060ce15dfe3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940865c575bc060ce15dfe3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940865c575bc060ce15dfe3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:01,303 - lava_callback - INFO - Starting async job submit for node 69408667575bc060ce15e018 2025-12-17 22:13:01,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:01,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408667575bc060ce15e018 HTTP/1.1" 502 150 2025-12-17 22:13:01,321 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408667575bc060ce15e018'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:01,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408667575bc060ce15e018 2025-12-17 22:13:01,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408667575bc060ce15e018 HTTP/1.1" 502 150 2025-12-17 22:13:01,322 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408667575bc060ce15e018'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:01,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694081dd575bc060ce15cc11 2025-12-17 22:13:01,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694081dd575bc060ce15cc11 HTTP/1.1" 502 150 2025-12-17 22:13:01,453 - lava_callback - ERROR - Error processing callback for node 694081dd575bc060ce15cc11: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081dd575bc060ce15cc11 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081dd575bc060ce15cc11 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694081dd575bc060ce15cc11 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:01,454 - lava_callback - INFO - Starting async job submit for node 69408668575bc060ce15e01b 2025-12-17 22:13:01,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:01,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408668575bc060ce15e01b HTTP/1.1" 502 150 2025-12-17 22:13:01,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408668575bc060ce15e01b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:01,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408668575bc060ce15e01b 2025-12-17 22:13:01,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408668575bc060ce15e01b HTTP/1.1" 502 150 2025-12-17 22:13:01,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408668575bc060ce15e01b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:01,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864e575bc060ce15dfc3 2025-12-17 22:13:01,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864e575bc060ce15dfc3 HTTP/1.1" 502 150 2025-12-17 22:13:01,651 - lava_callback - ERROR - Error processing callback for node 6940864e575bc060ce15dfc3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940864e575bc060ce15dfc3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940864e575bc060ce15dfc3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940864e575bc060ce15dfc3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:01,654 - lava_callback - INFO - Starting async job submit for node 6940866a575bc060ce15e022 2025-12-17 22:13:01,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:01,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866a575bc060ce15e022 HTTP/1.1" 502 150 2025-12-17 22:13:01,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866a575bc060ce15e022'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:01,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866a575bc060ce15e022 2025-12-17 22:13:01,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866a575bc060ce15e022 HTTP/1.1" 502 150 2025-12-17 22:13:01,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866a575bc060ce15e022'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58528 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:13:02,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940864f575bc060ce15dfc8 2025-12-17 22:13:02,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940864f575bc060ce15dfc8 HTTP/1.1" 502 150 2025-12-17 22:13:02,521 - lava_callback - ERROR - Error processing callback for node 6940864f575bc060ce15dfc8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940864f575bc060ce15dfc8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940864f575bc060ce15dfc8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940864f575bc060ce15dfc8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:02,523 - lava_callback - INFO - Starting async job submit for node 6940866b575bc060ce15e025 2025-12-17 22:13:02,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:02,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866b575bc060ce15e025 HTTP/1.1" 502 150 2025-12-17 22:13:02,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866b575bc060ce15e025'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:02,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866b575bc060ce15e025 2025-12-17 22:13:02,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866b575bc060ce15e025 HTTP/1.1" 502 150 2025-12-17 22:13:02,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866b575bc060ce15e025'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:02,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408650575bc060ce15dfcb 2025-12-17 22:13:02,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408650575bc060ce15dfcb HTTP/1.1" 502 150 2025-12-17 22:13:02,908 - lava_callback - ERROR - Error processing callback for node 69408650575bc060ce15dfcb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408650575bc060ce15dfcb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408650575bc060ce15dfcb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408650575bc060ce15dfcb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:02,909 - lava_callback - INFO - Starting async job submit for node 6940866d575bc060ce15e028 2025-12-17 22:13:02,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:02,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866d575bc060ce15e028 HTTP/1.1" 502 150 2025-12-17 22:13:02,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866d575bc060ce15e028'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:02,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866d575bc060ce15e028 2025-12-17 22:13:02,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866d575bc060ce15e028 HTTP/1.1" 502 150 2025-12-17 22:13:02,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866d575bc060ce15e028'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:03,322 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408667575bc060ce15e018 2025-12-17 22:13:03,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408668575bc060ce15e01b 2025-12-17 22:13:03,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408668575bc060ce15e01b HTTP/1.1" 502 150 2025-12-17 22:13:03,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408668575bc060ce15e01b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:03,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866a575bc060ce15e022 2025-12-17 22:13:03,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866a575bc060ce15e022 HTTP/1.1" 502 150 2025-12-17 22:13:03,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866a575bc060ce15e022'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:03,987 - lava_callback - INFO - Received callback for node 694087e0575bc060ce15e4e7 INFO: 172.21.0.1:58530 - "POST /node/694087e0575bc060ce15e4e7 HTTP/1.0" 202 Accepted 2025-12-17 22:13:04,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866b575bc060ce15e025 2025-12-17 22:13:04,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866b575bc060ce15e025 HTTP/1.1" 502 150 2025-12-17 22:13:04,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866b575bc060ce15e025'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:04,620 - lava_callback - INFO - Received callback for node 694087e2575bc060ce15e4ea INFO: 172.21.0.1:58534 - "POST /node/694087e2575bc060ce15e4ea HTTP/1.0" 202 Accepted 2025-12-17 22:13:04,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866d575bc060ce15e028 2025-12-17 22:13:04,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866d575bc060ce15e028 HTTP/1.1" 502 150 2025-12-17 22:13:04,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866d575bc060ce15e028'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:05,280 - lava_callback - INFO - Received callback for node 694087e6575bc060ce15e51d INFO: 172.21.0.1:58544 - "POST /node/694087e6575bc060ce15e51d HTTP/1.0" 202 Accepted 2025-12-17 22:13:05,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408658575bc060ce15dfda 2025-12-17 22:13:05,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408658575bc060ce15dfda HTTP/1.1" 502 150 2025-12-17 22:13:05,462 - lava_callback - ERROR - Error processing callback for node 69408658575bc060ce15dfda: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408658575bc060ce15dfda (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408658575bc060ce15dfda (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408658575bc060ce15dfda (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:05,464 - lava_callback - INFO - Starting async job submit for node 6940866e575bc060ce15e02b 2025-12-17 22:13:05,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:05,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866e575bc060ce15e02b HTTP/1.1" 502 150 2025-12-17 22:13:05,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866e575bc060ce15e02b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:05,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866e575bc060ce15e02b 2025-12-17 22:13:05,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866e575bc060ce15e02b HTTP/1.1" 502 150 2025-12-17 22:13:05,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866e575bc060ce15e02b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:05,895 - lava_callback - INFO - Received callback for node 6940cb75460bc8e265111be3 INFO: 172.21.0.1:58560 - "POST /node/6940cb75460bc8e265111be3 HTTP/1.0" 202 Accepted 2025-12-17 22:13:05,934 - lava_callback - INFO - Received callback for node 694087e9575bc060ce15e54a INFO: 172.21.0.1:58568 - "POST /node/694087e9575bc060ce15e54a HTTP/1.0" 202 Accepted 2025-12-17 22:13:06,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408652575bc060ce15dfce 2025-12-17 22:13:06,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408652575bc060ce15dfce HTTP/1.1" 502 150 2025-12-17 22:13:06,322 - lava_callback - ERROR - Error processing callback for node 69408652575bc060ce15dfce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408652575bc060ce15dfce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408652575bc060ce15dfce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408652575bc060ce15dfce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:06,324 - lava_callback - INFO - Starting async job submit for node 6940866f575bc060ce15e02e 2025-12-17 22:13:06,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:06,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866f575bc060ce15e02e HTTP/1.1" 502 150 2025-12-17 22:13:06,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866f575bc060ce15e02e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:06,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866f575bc060ce15e02e 2025-12-17 22:13:06,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866f575bc060ce15e02e HTTP/1.1" 502 150 2025-12-17 22:13:06,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866f575bc060ce15e02e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58572 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:13:06,572 - lava_callback - INFO - Received callback for node 694087eb575bc060ce15e54d INFO: 172.21.0.1:58578 - "POST /node/694087eb575bc060ce15e54d HTTP/1.0" 202 Accepted 2025-12-17 22:13:06,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408655575bc060ce15dfd4 2025-12-17 22:13:06,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408655575bc060ce15dfd4 HTTP/1.1" 502 150 2025-12-17 22:13:06,967 - lava_callback - ERROR - Error processing callback for node 69408655575bc060ce15dfd4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408655575bc060ce15dfd4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408655575bc060ce15dfd4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408655575bc060ce15dfd4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:06,969 - lava_callback - INFO - Starting async job submit for node 69408665575bc060ce15e012 2025-12-17 22:13:06,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:06,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408665575bc060ce15e012 HTTP/1.1" 502 150 2025-12-17 22:13:06,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408665575bc060ce15e012'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:06,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408665575bc060ce15e012 2025-12-17 22:13:06,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408665575bc060ce15e012 HTTP/1.1" 502 150 2025-12-17 22:13:06,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408665575bc060ce15e012'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:07,220 - lava_callback - INFO - Received callback for node 694087ed575bc060ce15e550 INFO: 172.21.0.1:58590 - "POST /node/694087ed575bc060ce15e550 HTTP/1.0" 202 Accepted 2025-12-17 22:13:07,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408668575bc060ce15e01b 2025-12-17 22:13:07,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408668575bc060ce15e01b HTTP/1.1" 502 150 2025-12-17 22:13:07,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408668575bc060ce15e01b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:07,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866e575bc060ce15e02b 2025-12-17 22:13:07,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866e575bc060ce15e02b HTTP/1.1" 502 150 2025-12-17 22:13:07,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866e575bc060ce15e02b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:07,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866a575bc060ce15e022 2025-12-17 22:13:07,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866a575bc060ce15e022 HTTP/1.1" 502 150 2025-12-17 22:13:07,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866a575bc060ce15e022'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:07,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408661575bc060ce15dffc HTTP/1.1" 502 150 2025-12-17 22:13:07,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408661575bc060ce15dffc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:08,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866f575bc060ce15e02e 2025-12-17 22:13:08,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866f575bc060ce15e02e HTTP/1.1" 502 150 2025-12-17 22:13:08,340 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866f575bc060ce15e02e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:08,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866b575bc060ce15e025 2025-12-17 22:13:08,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866b575bc060ce15e025 HTTP/1.1" 502 150 2025-12-17 22:13:08,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866b575bc060ce15e025'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:08,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408666575bc060ce15e015 2025-12-17 22:13:08,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408666575bc060ce15e015 HTTP/1.1" 502 150 2025-12-17 22:13:08,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408666575bc060ce15e015'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:08,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866d575bc060ce15e028 2025-12-17 22:13:08,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866d575bc060ce15e028 HTTP/1.1" 502 150 2025-12-17 22:13:08,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866d575bc060ce15e028'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:08,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408665575bc060ce15e012 2025-12-17 22:13:09,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408665575bc060ce15e012 HTTP/1.1" 502 150 2025-12-17 22:13:09,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408665575bc060ce15e012'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:09,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408656575bc060ce15dfd7 2025-12-17 22:13:09,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408656575bc060ce15dfd7 HTTP/1.1" 502 150 2025-12-17 22:13:09,972 - lava_callback - ERROR - Error processing callback for node 69408656575bc060ce15dfd7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408656575bc060ce15dfd7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408656575bc060ce15dfd7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408656575bc060ce15dfd7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:09,973 - lava_callback - INFO - Starting async job submit for node 69408670575bc060ce15e031 2025-12-17 22:13:09,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:09,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408670575bc060ce15e031 HTTP/1.1" 502 150 2025-12-17 22:13:09,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408670575bc060ce15e031'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:09,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408670575bc060ce15e031 2025-12-17 22:13:09,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408670575bc060ce15e031 HTTP/1.1" 502 150 2025-12-17 22:13:09,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408670575bc060ce15e031'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:11,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866e575bc060ce15e02b 2025-12-17 22:13:11,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866e575bc060ce15e02b HTTP/1.1" 502 150 2025-12-17 22:13:11,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866e575bc060ce15e02b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:11,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408670575bc060ce15e031 2025-12-17 22:13:11,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408670575bc060ce15e031 HTTP/1.1" 502 150 2025-12-17 22:13:11,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408670575bc060ce15e031'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:12,341 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866f575bc060ce15e02e 2025-12-17 22:13:12,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866f575bc060ce15e02e HTTP/1.1" 502 150 2025-12-17 22:13:12,343 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866f575bc060ce15e02e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:12,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408663575bc060ce15e008 2025-12-17 22:13:12,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408663575bc060ce15e008 HTTP/1.1" 502 150 2025-12-17 22:13:12,369 - lava_callback - ERROR - Error processing callback for node 69408663575bc060ce15e008: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408663575bc060ce15e008 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408663575bc060ce15e008 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408663575bc060ce15e008 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:12,371 - lava_callback - INFO - Starting async job submit for node 69408672575bc060ce15e035 2025-12-17 22:13:12,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:12,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408672575bc060ce15e035 HTTP/1.1" 502 150 2025-12-17 22:13:12,378 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408672575bc060ce15e035'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:12,378 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408672575bc060ce15e035 2025-12-17 22:13:12,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408672575bc060ce15e035 HTTP/1.1" 502 150 2025-12-17 22:13:12,378 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408672575bc060ce15e035'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:13,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408665575bc060ce15e012 2025-12-17 22:13:13,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408665575bc060ce15e012 HTTP/1.1" 502 150 2025-12-17 22:13:13,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408665575bc060ce15e012'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:14,127 - lava_callback - INFO - Received callback for node 694087ef575bc060ce15e555 INFO: 172.21.0.1:52626 - "POST /node/694087ef575bc060ce15e555 HTTP/1.0" 202 Accepted 2025-12-17 22:13:14,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408672575bc060ce15e035 2025-12-17 22:13:14,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408672575bc060ce15e035 HTTP/1.1" 502 150 2025-12-17 22:13:14,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408672575bc060ce15e035'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:14,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865d575bc060ce15dfe6 2025-12-17 22:13:14,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865d575bc060ce15dfe6 HTTP/1.1" 502 150 2025-12-17 22:13:14,436 - lava_callback - ERROR - Error processing callback for node 6940865d575bc060ce15dfe6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940865d575bc060ce15dfe6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940865d575bc060ce15dfe6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940865d575bc060ce15dfe6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:14,438 - lava_callback - INFO - Starting async job submit for node 69408676575bc060ce15e064 2025-12-17 22:13:14,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:14,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408676575bc060ce15e064 HTTP/1.1" 502 150 2025-12-17 22:13:14,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408676575bc060ce15e064'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:14,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408676575bc060ce15e064 2025-12-17 22:13:14,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408676575bc060ce15e064 HTTP/1.1" 502 150 2025-12-17 22:13:14,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408676575bc060ce15e064'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:14,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940865e575bc060ce15dfeb 2025-12-17 22:13:14,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940865e575bc060ce15dfeb HTTP/1.1" 502 150 2025-12-17 22:13:14,576 - lava_callback - ERROR - Error processing callback for node 6940865e575bc060ce15dfeb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940865e575bc060ce15dfeb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940865e575bc060ce15dfeb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940865e575bc060ce15dfeb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:14,578 - lava_callback - INFO - Starting async job submit for node 69408678575bc060ce15e06a 2025-12-17 22:13:14,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:14,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408678575bc060ce15e06a HTTP/1.1" 502 150 2025-12-17 22:13:14,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408678575bc060ce15e06a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:14,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408678575bc060ce15e06a 2025-12-17 22:13:14,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408678575bc060ce15e06a HTTP/1.1" 502 150 2025-12-17 22:13:14,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408678575bc060ce15e06a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:14,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408660575bc060ce15dff5 2025-12-17 22:13:14,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408660575bc060ce15dff5 HTTP/1.1" 502 150 2025-12-17 22:13:14,969 - lava_callback - ERROR - Error processing callback for node 69408660575bc060ce15dff5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408660575bc060ce15dff5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408660575bc060ce15dff5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408660575bc060ce15dff5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:14,971 - lava_callback - INFO - Starting async job submit for node 6940867b575bc060ce15e070 2025-12-17 22:13:14,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:14,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867b575bc060ce15e070 HTTP/1.1" 502 150 2025-12-17 22:13:14,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940867b575bc060ce15e070'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:14,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940867b575bc060ce15e070 2025-12-17 22:13:14,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867b575bc060ce15e070 HTTP/1.1" 502 150 2025-12-17 22:13:14,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940867b575bc060ce15e070'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52642 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:13:15,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408668575bc060ce15e01b 2025-12-17 22:13:15,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408668575bc060ce15e01b HTTP/1.1" 502 150 2025-12-17 22:13:15,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408668575bc060ce15e01b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:15,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866a575bc060ce15e022 2025-12-17 22:13:15,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866a575bc060ce15e022 HTTP/1.1" 502 150 2025-12-17 22:13:15,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866a575bc060ce15e022'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:15,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408661575bc060ce15dffc 2025-12-17 22:13:15,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408661575bc060ce15dffc HTTP/1.1" 502 150 2025-12-17 22:13:15,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408661575bc060ce15dffc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:15,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408670575bc060ce15e031 2025-12-17 22:13:15,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408670575bc060ce15e031 HTTP/1.1" 502 150 2025-12-17 22:13:15,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408670575bc060ce15e031'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:16,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408663575bc060ce15e003 HTTP/1.1" 502 150 2025-12-17 22:13:16,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408663575bc060ce15e003'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:16,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408676575bc060ce15e064 2025-12-17 22:13:16,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408676575bc060ce15e064 HTTP/1.1" 502 150 2025-12-17 22:13:16,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408676575bc060ce15e064'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52656 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:13:16,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866b575bc060ce15e025 2025-12-17 22:13:16,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866b575bc060ce15e025 HTTP/1.1" 502 150 2025-12-17 22:13:16,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866b575bc060ce15e025'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:16,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408678575bc060ce15e06a 2025-12-17 22:13:16,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408678575bc060ce15e06a HTTP/1.1" 502 150 2025-12-17 22:13:16,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408678575bc060ce15e06a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:16,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866d575bc060ce15e028 2025-12-17 22:13:16,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866d575bc060ce15e028 HTTP/1.1" 502 150 2025-12-17 22:13:16,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866d575bc060ce15e028'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:16,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940867b575bc060ce15e070 2025-12-17 22:13:16,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867b575bc060ce15e070 HTTP/1.1" 502 150 2025-12-17 22:13:16,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940867b575bc060ce15e070'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:18,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408672575bc060ce15e035 2025-12-17 22:13:18,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408672575bc060ce15e035 HTTP/1.1" 502 150 2025-12-17 22:13:18,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408672575bc060ce15e035'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:18,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408667575bc060ce15e018 HTTP/1.1" 502 150 2025-12-17 22:13:18,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408667575bc060ce15e018'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:19,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866e575bc060ce15e02b 2025-12-17 22:13:19,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866e575bc060ce15e02b HTTP/1.1" 502 150 2025-12-17 22:13:19,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866e575bc060ce15e02b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:20,343 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866f575bc060ce15e02e 2025-12-17 22:13:20,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866f575bc060ce15e02e HTTP/1.1" 502 150 2025-12-17 22:13:20,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940866f575bc060ce15e02e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:20,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408676575bc060ce15e064 2025-12-17 22:13:20,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408676575bc060ce15e064 HTTP/1.1" 502 150 2025-12-17 22:13:20,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408676575bc060ce15e064'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:20,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408678575bc060ce15e06a 2025-12-17 22:13:20,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408678575bc060ce15e06a HTTP/1.1" 502 150 2025-12-17 22:13:20,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408678575bc060ce15e06a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:20,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940867b575bc060ce15e070 2025-12-17 22:13:20,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867b575bc060ce15e070 HTTP/1.1" 502 150 2025-12-17 22:13:20,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940867b575bc060ce15e070'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:21,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408665575bc060ce15e012 2025-12-17 22:13:21,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408665575bc060ce15e012 HTTP/1.1" 502 150 2025-12-17 22:13:21,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408665575bc060ce15e012'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:22,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408667575bc060ce15e018 2025-12-17 22:13:22,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408667575bc060ce15e018 HTTP/1.1" 502 150 2025-12-17 22:13:22,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408667575bc060ce15e018'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:23,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408670575bc060ce15e031 2025-12-17 22:13:23,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408670575bc060ce15e031 HTTP/1.1" 502 150 2025-12-17 22:13:23,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408670575bc060ce15e031'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:24,002 - lava_callback - INFO - Received callback for node 694087f1575bc060ce15e55a INFO: 172.21.0.1:35890 - "POST /node/694087f1575bc060ce15e55a HTTP/1.0" 202 Accepted 2025-12-17 22:13:24,081 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408663575bc060ce15e003 2025-12-17 22:13:24,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408663575bc060ce15e003 HTTP/1.1" 502 150 2025-12-17 22:13:24,081 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408663575bc060ce15e003'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:24,658 - lava_callback - INFO - Received callback for node 694087f3575bc060ce15e55d INFO: 172.21.0.1:35898 - "POST /node/694087f3575bc060ce15e55d HTTP/1.0" 202 Accepted 2025-12-17 22:13:24,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408666575bc060ce15e015 2025-12-17 22:13:24,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408666575bc060ce15e015 HTTP/1.1" 502 150 2025-12-17 22:13:24,890 - lava_callback - ERROR - Error processing callback for node 69408666575bc060ce15e015: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408666575bc060ce15e015 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408666575bc060ce15e015 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408666575bc060ce15e015 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:24,892 - lava_callback - INFO - Starting async job submit for node 6940867f575bc060ce15e076 2025-12-17 22:13:24,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:24,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867f575bc060ce15e076 HTTP/1.1" 502 150 2025-12-17 22:13:24,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940867f575bc060ce15e076'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:24,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940867f575bc060ce15e076 2025-12-17 22:13:24,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867f575bc060ce15e076 HTTP/1.1" 502 150 2025-12-17 22:13:24,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940867f575bc060ce15e076'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:25,321 - lava_callback - INFO - Received callback for node 694087f7575bc060ce15e563 INFO: 172.21.0.1:35900 - "POST /node/694087f7575bc060ce15e563 HTTP/1.0" 202 Accepted 2025-12-17 22:13:25,973 - lava_callback - INFO - Received callback for node 694087f9575bc060ce15e56a INFO: 172.21.0.1:35910 - "POST /node/694087f9575bc060ce15e56a HTTP/1.0" 202 Accepted 2025-12-17 22:13:26,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408672575bc060ce15e035 2025-12-17 22:13:26,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408672575bc060ce15e035 HTTP/1.1" 502 150 2025-12-17 22:13:26,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408672575bc060ce15e035'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35912 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:13:26,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940867f575bc060ce15e076 2025-12-17 22:13:26,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867f575bc060ce15e076 HTTP/1.1" 502 150 2025-12-17 22:13:26,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940867f575bc060ce15e076'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:28,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408676575bc060ce15e064 2025-12-17 22:13:28,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408676575bc060ce15e064 HTTP/1.1" 502 150 2025-12-17 22:13:28,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408676575bc060ce15e064'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:28,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408678575bc060ce15e06a 2025-12-17 22:13:28,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408678575bc060ce15e06a HTTP/1.1" 502 150 2025-12-17 22:13:28,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408678575bc060ce15e06a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:28,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940867b575bc060ce15e070 2025-12-17 22:13:28,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867b575bc060ce15e070 HTTP/1.1" 502 150 2025-12-17 22:13:28,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940867b575bc060ce15e070'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:30,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408667575bc060ce15e018 2025-12-17 22:13:30,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408667575bc060ce15e018 HTTP/1.1" 502 150 2025-12-17 22:13:30,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408667575bc060ce15e018'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:30,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940867f575bc060ce15e076 2025-12-17 22:13:30,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867f575bc060ce15e076 HTTP/1.1" 502 150 2025-12-17 22:13:30,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940867f575bc060ce15e076'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:31,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408668575bc060ce15e01b 2025-12-17 22:13:31,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408668575bc060ce15e01b HTTP/1.1" 502 150 2025-12-17 22:13:31,472 - lava_callback - ERROR - Error processing callback for node 69408668575bc060ce15e01b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408668575bc060ce15e01b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408668575bc060ce15e01b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408668575bc060ce15e01b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:31,473 - lava_callback - INFO - Starting async job submit for node 69408680575bc060ce15e079 2025-12-17 22:13:31,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:31,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408680575bc060ce15e079 HTTP/1.1" 502 150 2025-12-17 22:13:31,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408680575bc060ce15e079'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:31,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408680575bc060ce15e079 2025-12-17 22:13:31,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408680575bc060ce15e079 HTTP/1.1" 502 150 2025-12-17 22:13:31,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408680575bc060ce15e079'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:31,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866a575bc060ce15e022 2025-12-17 22:13:31,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866a575bc060ce15e022 HTTP/1.1" 502 150 2025-12-17 22:13:31,692 - lava_callback - ERROR - Error processing callback for node 6940866a575bc060ce15e022: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940866a575bc060ce15e022 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940866a575bc060ce15e022 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940866a575bc060ce15e022 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:31,694 - lava_callback - INFO - Starting async job submit for node 69408681575bc060ce15e07c 2025-12-17 22:13:31,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:31,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408681575bc060ce15e07c HTTP/1.1" 502 150 2025-12-17 22:13:31,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408681575bc060ce15e07c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:31,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408681575bc060ce15e07c 2025-12-17 22:13:31,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408681575bc060ce15e07c HTTP/1.1" 502 150 2025-12-17 22:13:31,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408681575bc060ce15e07c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:31,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408661575bc060ce15dffc 2025-12-17 22:13:31,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408661575bc060ce15dffc HTTP/1.1" 502 150 2025-12-17 22:13:31,892 - lava_callback - ERROR - Error processing callback for node 69408661575bc060ce15dffc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408661575bc060ce15dffc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408661575bc060ce15dffc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408661575bc060ce15dffc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:31,893 - lava_callback - INFO - Starting async job submit for node 69408674575bc060ce15e05d 2025-12-17 22:13:31,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:31,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408674575bc060ce15e05d HTTP/1.1" 502 150 2025-12-17 22:13:31,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408674575bc060ce15e05d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:31,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408674575bc060ce15e05d 2025-12-17 22:13:31,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408674575bc060ce15e05d HTTP/1.1" 502 150 2025-12-17 22:13:31,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408674575bc060ce15e05d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:32,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866b575bc060ce15e025 2025-12-17 22:13:32,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866b575bc060ce15e025 HTTP/1.1" 502 150 2025-12-17 22:13:32,550 - lava_callback - ERROR - Error processing callback for node 6940866b575bc060ce15e025: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940866b575bc060ce15e025 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940866b575bc060ce15e025 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940866b575bc060ce15e025 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:32,551 - lava_callback - INFO - Starting async job submit for node 69408682575bc060ce15e07f 2025-12-17 22:13:32,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:32,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408682575bc060ce15e07f HTTP/1.1" 502 150 2025-12-17 22:13:32,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408682575bc060ce15e07f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:32,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408682575bc060ce15e07f 2025-12-17 22:13:32,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408682575bc060ce15e07f HTTP/1.1" 502 150 2025-12-17 22:13:32,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408682575bc060ce15e07f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:32,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866d575bc060ce15e028 2025-12-17 22:13:32,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866d575bc060ce15e028 HTTP/1.1" 502 150 2025-12-17 22:13:32,922 - lava_callback - ERROR - Error processing callback for node 6940866d575bc060ce15e028: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940866d575bc060ce15e028 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940866d575bc060ce15e028 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940866d575bc060ce15e028 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:32,924 - lava_callback - INFO - Starting async job submit for node 69408676575bc060ce15e067 2025-12-17 22:13:32,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:32,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408676575bc060ce15e067 HTTP/1.1" 502 150 2025-12-17 22:13:32,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408676575bc060ce15e067'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:32,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408676575bc060ce15e067 2025-12-17 22:13:32,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408676575bc060ce15e067 HTTP/1.1" 502 150 2025-12-17 22:13:32,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408676575bc060ce15e067'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:33,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408680575bc060ce15e079 2025-12-17 22:13:33,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408680575bc060ce15e079 HTTP/1.1" 502 150 2025-12-17 22:13:33,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408680575bc060ce15e079'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:33,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408681575bc060ce15e07c 2025-12-17 22:13:33,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408681575bc060ce15e07c HTTP/1.1" 502 150 2025-12-17 22:13:33,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408681575bc060ce15e07c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:33,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408674575bc060ce15e05d 2025-12-17 22:13:33,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408674575bc060ce15e05d HTTP/1.1" 502 150 2025-12-17 22:13:33,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408674575bc060ce15e05d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:34,116 - lava_callback - INFO - Received callback for node 694087fa575bc060ce15e571 INFO: 172.21.0.1:55566 - "POST /node/694087fa575bc060ce15e571 HTTP/1.0" 202 Accepted 2025-12-17 22:13:34,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408682575bc060ce15e07f 2025-12-17 22:13:34,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408682575bc060ce15e07f HTTP/1.1" 502 150 2025-12-17 22:13:34,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408682575bc060ce15e07f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:34,782 - lava_callback - INFO - Received callback for node 694087fc575bc060ce15e574 INFO: 172.21.0.1:55576 - "POST /node/694087fc575bc060ce15e574 HTTP/1.0" 202 Accepted 2025-12-17 22:13:34,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408676575bc060ce15e067 2025-12-17 22:13:34,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408676575bc060ce15e067 HTTP/1.1" 502 150 2025-12-17 22:13:34,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408676575bc060ce15e067'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:35,452 - lava_callback - INFO - Received callback for node 694087fd575bc060ce15e577 INFO: 172.21.0.1:55588 - "POST /node/694087fd575bc060ce15e577 HTTP/1.0" 202 Accepted 2025-12-17 22:13:35,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866e575bc060ce15e02b 2025-12-17 22:13:35,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866e575bc060ce15e02b HTTP/1.1" 502 150 2025-12-17 22:13:35,476 - lava_callback - ERROR - Error processing callback for node 6940866e575bc060ce15e02b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940866e575bc060ce15e02b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940866e575bc060ce15e02b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940866e575bc060ce15e02b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:35,478 - lava_callback - INFO - Starting async job submit for node 69408260575bc060ce15cf40 2025-12-17 22:13:35,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:35,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408260575bc060ce15cf40 HTTP/1.1" 502 150 2025-12-17 22:13:35,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408260575bc060ce15cf40'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:35,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408260575bc060ce15cf40 2025-12-17 22:13:35,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408260575bc060ce15cf40 HTTP/1.1" 502 150 2025-12-17 22:13:35,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408260575bc060ce15cf40'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:36,116 - lava_callback - INFO - Received callback for node 694087fe575bc060ce15e57a INFO: 172.21.0.1:55596 - "POST /node/694087fe575bc060ce15e57a HTTP/1.0" 202 Accepted 2025-12-17 22:13:36,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940866f575bc060ce15e02e 2025-12-17 22:13:36,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940866f575bc060ce15e02e HTTP/1.1" 502 150 2025-12-17 22:13:36,346 - lava_callback - ERROR - Error processing callback for node 6940866f575bc060ce15e02e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940866f575bc060ce15e02e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940866f575bc060ce15e02e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940866f575bc060ce15e02e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:36,347 - lava_callback - INFO - Starting async job submit for node 69408679575bc060ce15e06d 2025-12-17 22:13:36,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:36,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408679575bc060ce15e06d HTTP/1.1" 502 150 2025-12-17 22:13:36,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408679575bc060ce15e06d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:36,354 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408679575bc060ce15e06d 2025-12-17 22:13:36,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408679575bc060ce15e06d HTTP/1.1" 502 150 2025-12-17 22:13:36,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408679575bc060ce15e06d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55606 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:13:36,780 - lava_callback - INFO - Received callback for node 69408800575bc060ce15e57d INFO: 172.21.0.1:55608 - "POST /node/69408800575bc060ce15e57d HTTP/1.0" 202 Accepted 2025-12-17 22:13:37,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408665575bc060ce15e012 2025-12-17 22:13:37,463 - lava_callback - INFO - Received callback for node 69408801575bc060ce15e580 INFO: 172.21.0.1:55624 - "POST /node/69408801575bc060ce15e580 HTTP/1.0" 202 Accepted 2025-12-17 22:13:37,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408680575bc060ce15e079 2025-12-17 22:13:37,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408680575bc060ce15e079 HTTP/1.1" 502 150 2025-12-17 22:13:37,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408680575bc060ce15e079'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:37,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408260575bc060ce15cf40 2025-12-17 22:13:37,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408260575bc060ce15cf40 HTTP/1.1" 502 150 2025-12-17 22:13:37,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408260575bc060ce15cf40'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:37,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408681575bc060ce15e07c 2025-12-17 22:13:37,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408681575bc060ce15e07c HTTP/1.1" 502 150 2025-12-17 22:13:37,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408681575bc060ce15e07c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:37,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408674575bc060ce15e05d 2025-12-17 22:13:37,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408674575bc060ce15e05d HTTP/1.1" 502 150 2025-12-17 22:13:37,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408674575bc060ce15e05d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:38,152 - lava_callback - INFO - Received callback for node 69408802575bc060ce15e585 INFO: 172.21.0.1:55636 - "POST /node/69408802575bc060ce15e585 HTTP/1.0" 202 Accepted 2025-12-17 22:13:38,355 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408679575bc060ce15e06d 2025-12-17 22:13:38,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408679575bc060ce15e06d HTTP/1.1" 502 150 2025-12-17 22:13:38,356 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408679575bc060ce15e06d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:38,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408682575bc060ce15e07f 2025-12-17 22:13:38,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408682575bc060ce15e07f HTTP/1.1" 502 150 2025-12-17 22:13:38,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408682575bc060ce15e07f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:38,818 - lava_callback - INFO - Received callback for node 69408804575bc060ce15e58c INFO: 172.21.0.1:55650 - "POST /node/69408804575bc060ce15e58c HTTP/1.0" 202 Accepted 2025-12-17 22:13:38,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940867f575bc060ce15e076 2025-12-17 22:13:38,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867f575bc060ce15e076 HTTP/1.1" 502 150 2025-12-17 22:13:38,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940867f575bc060ce15e076'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:38,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408676575bc060ce15e067 2025-12-17 22:13:38,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408676575bc060ce15e067 HTTP/1.1" 502 150 2025-12-17 22:13:38,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408676575bc060ce15e067'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:39,483 - lava_callback - INFO - Received callback for node 69408805575bc060ce15e593 INFO: 172.21.0.1:55664 - "POST /node/69408805575bc060ce15e593 HTTP/1.0" 202 Accepted 2025-12-17 22:13:39,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408670575bc060ce15e031 2025-12-17 22:13:39,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408670575bc060ce15e031 HTTP/1.1" 502 150 2025-12-17 22:13:39,986 - lava_callback - ERROR - Error processing callback for node 69408670575bc060ce15e031: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408670575bc060ce15e031 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408670575bc060ce15e031 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408670575bc060ce15e031 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:39,990 - lava_callback - INFO - Starting async job submit for node 6940867c575bc060ce15e073 2025-12-17 22:13:39,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:40,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867c575bc060ce15e073 HTTP/1.1" 502 150 2025-12-17 22:13:40,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940867c575bc060ce15e073'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:40,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940867c575bc060ce15e073 2025-12-17 22:13:40,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867c575bc060ce15e073 HTTP/1.1" 502 150 2025-12-17 22:13:40,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940867c575bc060ce15e073'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:40,082 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408663575bc060ce15e003 2025-12-17 22:13:40,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408663575bc060ce15e003 HTTP/1.1" 502 150 2025-12-17 22:13:40,083 - lava_callback - ERROR - Error processing callback for node 69408663575bc060ce15e003: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408663575bc060ce15e003 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408663575bc060ce15e003 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408663575bc060ce15e003 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:40,086 - lava_callback - INFO - Starting async job submit for node 69408685575bc060ce15e085 2025-12-17 22:13:40,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:40,153 - lava_callback - INFO - Received callback for node 69408806575bc060ce15e59a INFO: 172.21.0.1:56056 - "POST /node/69408806575bc060ce15e59a HTTP/1.0" 202 Accepted 2025-12-17 22:13:40,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408685575bc060ce15e085 HTTP/1.1" 502 150 2025-12-17 22:13:40,172 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408685575bc060ce15e085'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:40,172 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408685575bc060ce15e085 2025-12-17 22:13:40,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408685575bc060ce15e085 HTTP/1.1" 502 150 2025-12-17 22:13:40,173 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408685575bc060ce15e085'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:40,825 - lava_callback - INFO - Received callback for node 69408808575bc060ce15e59d INFO: 172.21.0.1:56068 - "POST /node/69408808575bc060ce15e59d HTTP/1.0" 202 Accepted 2025-12-17 22:13:41,507 - lava_callback - INFO - Received callback for node 69408809575bc060ce15e5a5 INFO: 172.21.0.1:56084 - "POST /node/69408809575bc060ce15e5a5 HTTP/1.0" 202 Accepted 2025-12-17 22:13:41,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408260575bc060ce15cf40 2025-12-17 22:13:41,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408260575bc060ce15cf40 HTTP/1.1" 502 150 2025-12-17 22:13:41,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408260575bc060ce15cf40'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:42,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940867c575bc060ce15e073 2025-12-17 22:13:42,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867c575bc060ce15e073 HTTP/1.1" 502 150 2025-12-17 22:13:42,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940867c575bc060ce15e073'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:42,161 - lava_callback - INFO - Received callback for node 6940880a575bc060ce15e5a8 INFO: 172.21.0.1:56090 - "POST /node/6940880a575bc060ce15e5a8 HTTP/1.0" 202 Accepted 2025-12-17 22:13:42,173 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408685575bc060ce15e085 2025-12-17 22:13:42,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408685575bc060ce15e085 HTTP/1.1" 502 150 2025-12-17 22:13:42,174 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408685575bc060ce15e085'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:42,357 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408679575bc060ce15e06d 2025-12-17 22:13:42,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408679575bc060ce15e06d HTTP/1.1" 502 150 2025-12-17 22:13:42,358 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408679575bc060ce15e06d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:42,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408672575bc060ce15e035 2025-12-17 22:13:42,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408672575bc060ce15e035 HTTP/1.1" 502 150 2025-12-17 22:13:42,384 - lava_callback - ERROR - Error processing callback for node 69408672575bc060ce15e035: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408672575bc060ce15e035 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408672575bc060ce15e035 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408672575bc060ce15e035 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:42,386 - lava_callback - INFO - Starting async job submit for node 6940868c575bc060ce15e098 2025-12-17 22:13:42,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:42,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868c575bc060ce15e098 HTTP/1.1" 502 150 2025-12-17 22:13:42,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940868c575bc060ce15e098'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:42,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940868c575bc060ce15e098 2025-12-17 22:13:42,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868c575bc060ce15e098 HTTP/1.1" 502 150 2025-12-17 22:13:42,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940868c575bc060ce15e098'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:42,833 - lava_callback - INFO - Received callback for node 6940880c575bc060ce15e5ab INFO: 172.21.0.1:56104 - "POST /node/6940880c575bc060ce15e5ab HTTP/1.0" 202 Accepted 2025-12-17 22:13:44,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940868c575bc060ce15e098 2025-12-17 22:13:44,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408676575bc060ce15e064 2025-12-17 22:13:44,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408676575bc060ce15e064 HTTP/1.1" 502 150 2025-12-17 22:13:44,458 - lava_callback - ERROR - Error processing callback for node 69408676575bc060ce15e064: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408676575bc060ce15e064 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408676575bc060ce15e064 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408676575bc060ce15e064 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:44,460 - lava_callback - INFO - Starting async job submit for node 6940868d575bc060ce15e09b 2025-12-17 22:13:44,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:44,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868d575bc060ce15e09b HTTP/1.1" 502 150 2025-12-17 22:13:44,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940868d575bc060ce15e09b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:44,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940868d575bc060ce15e09b 2025-12-17 22:13:44,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868d575bc060ce15e09b HTTP/1.1" 502 150 2025-12-17 22:13:44,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940868d575bc060ce15e09b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:44,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408678575bc060ce15e06a 2025-12-17 22:13:44,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408678575bc060ce15e06a HTTP/1.1" 502 150 2025-12-17 22:13:44,592 - lava_callback - ERROR - Error processing callback for node 69408678575bc060ce15e06a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408678575bc060ce15e06a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408678575bc060ce15e06a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408678575bc060ce15e06a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:44,594 - lava_callback - INFO - Starting async job submit for node 6940868f575bc060ce15e09e 2025-12-17 22:13:44,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:44,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868f575bc060ce15e09e HTTP/1.1" 502 150 2025-12-17 22:13:44,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940868f575bc060ce15e09e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:44,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940868f575bc060ce15e09e 2025-12-17 22:13:44,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868f575bc060ce15e09e HTTP/1.1" 502 150 2025-12-17 22:13:44,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940868f575bc060ce15e09e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:44,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940867b575bc060ce15e070 2025-12-17 22:13:44,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867b575bc060ce15e070 HTTP/1.1" 502 150 2025-12-17 22:13:44,987 - lava_callback - ERROR - Error processing callback for node 6940867b575bc060ce15e070: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940867b575bc060ce15e070 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940867b575bc060ce15e070 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940867b575bc060ce15e070 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:44,989 - lava_callback - INFO - Starting async job submit for node 69408690575bc060ce15e0a1 2025-12-17 22:13:44,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:44,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408690575bc060ce15e0a1 HTTP/1.1" 502 150 2025-12-17 22:13:44,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408690575bc060ce15e0a1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:44,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408690575bc060ce15e0a1 2025-12-17 22:13:44,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408690575bc060ce15e0a1 HTTP/1.1" 502 150 2025-12-17 22:13:44,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408690575bc060ce15e0a1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:45,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408680575bc060ce15e079 2025-12-17 22:13:45,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408680575bc060ce15e079 HTTP/1.1" 502 150 2025-12-17 22:13:45,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408680575bc060ce15e079'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:45,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408681575bc060ce15e07c 2025-12-17 22:13:45,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408681575bc060ce15e07c HTTP/1.1" 502 150 2025-12-17 22:13:45,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408681575bc060ce15e07c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:45,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408674575bc060ce15e05d 2025-12-17 22:13:45,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408674575bc060ce15e05d HTTP/1.1" 502 150 2025-12-17 22:13:45,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408674575bc060ce15e05d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:46,028 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940867c575bc060ce15e073 2025-12-17 22:13:46,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867c575bc060ce15e073 HTTP/1.1" 502 150 2025-12-17 22:13:46,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940867c575bc060ce15e073'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:46,174 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408685575bc060ce15e085 2025-12-17 22:13:46,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408685575bc060ce15e085 HTTP/1.1" 502 150 2025-12-17 22:13:46,176 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408685575bc060ce15e085'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:46,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940868d575bc060ce15e09b 2025-12-17 22:13:46,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868d575bc060ce15e09b HTTP/1.1" 502 150 2025-12-17 22:13:46,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940868d575bc060ce15e09b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56106 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:13:46,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408682575bc060ce15e07f 2025-12-17 22:13:46,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408682575bc060ce15e07f HTTP/1.1" 502 150 2025-12-17 22:13:46,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408682575bc060ce15e07f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:46,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940868f575bc060ce15e09e 2025-12-17 22:13:46,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868f575bc060ce15e09e HTTP/1.1" 502 150 2025-12-17 22:13:46,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940868f575bc060ce15e09e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:46,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408667575bc060ce15e018 2025-12-17 22:13:46,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408667575bc060ce15e018 HTTP/1.1" 502 150 2025-12-17 22:13:46,654 - lava_callback - ERROR - Error processing callback for node 69408667575bc060ce15e018: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408667575bc060ce15e018 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408667575bc060ce15e018 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408667575bc060ce15e018 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:46,655 - lava_callback - INFO - Starting async job submit for node 694085a0575bc060ce15dcc7 2025-12-17 22:13:46,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:46,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a0575bc060ce15dcc7 HTTP/1.1" 502 150 2025-12-17 22:13:46,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a0575bc060ce15dcc7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:46,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a0575bc060ce15dcc7 2025-12-17 22:13:46,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a0575bc060ce15dcc7 HTTP/1.1" 502 150 2025-12-17 22:13:46,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a0575bc060ce15dcc7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:46,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408676575bc060ce15e067 2025-12-17 22:13:46,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408676575bc060ce15e067 HTTP/1.1" 502 150 2025-12-17 22:13:46,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408676575bc060ce15e067'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:46,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408690575bc060ce15e0a1 2025-12-17 22:13:46,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408690575bc060ce15e0a1 HTTP/1.1" 502 150 2025-12-17 22:13:46,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408690575bc060ce15e0a1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:48,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a0575bc060ce15dcc7 2025-12-17 22:13:48,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a0575bc060ce15dcc7 HTTP/1.1" 502 150 2025-12-17 22:13:48,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a0575bc060ce15dcc7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:49,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408260575bc060ce15cf40 2025-12-17 22:13:49,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408260575bc060ce15cf40 HTTP/1.1" 502 150 2025-12-17 22:13:49,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408260575bc060ce15cf40'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:50,358 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408679575bc060ce15e06d 2025-12-17 22:13:50,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408679575bc060ce15e06d HTTP/1.1" 502 150 2025-12-17 22:13:50,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408679575bc060ce15e06d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:50,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940868d575bc060ce15e09b 2025-12-17 22:13:50,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868d575bc060ce15e09b HTTP/1.1" 502 150 2025-12-17 22:13:50,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940868d575bc060ce15e09b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:50,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940868f575bc060ce15e09e 2025-12-17 22:13:50,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868f575bc060ce15e09e HTTP/1.1" 502 150 2025-12-17 22:13:50,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940868f575bc060ce15e09e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:50,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408690575bc060ce15e0a1 2025-12-17 22:13:51,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408690575bc060ce15e0a1 HTTP/1.1" 502 150 2025-12-17 22:13:51,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408690575bc060ce15e0a1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:52,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408665575bc060ce15e012 HTTP/1.1" 502 150 2025-12-17 22:13:52,432 - lava_callback - ERROR - Error processing callback for node 69408665575bc060ce15e012: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408665575bc060ce15e012 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408665575bc060ce15e012 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408665575bc060ce15e012 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:52,434 - lava_callback - INFO - Starting async job submit for node 69408691575bc060ce15e0a4 2025-12-17 22:13:52,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:52,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408691575bc060ce15e0a4 HTTP/1.1" 502 150 2025-12-17 22:13:52,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408691575bc060ce15e0a4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:52,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408691575bc060ce15e0a4 2025-12-17 22:13:52,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408691575bc060ce15e0a4 HTTP/1.1" 502 150 2025-12-17 22:13:52,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408691575bc060ce15e0a4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:52,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a0575bc060ce15dcc7 2025-12-17 22:13:52,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a0575bc060ce15dcc7 HTTP/1.1" 502 150 2025-12-17 22:13:52,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a0575bc060ce15dcc7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:54,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940867c575bc060ce15e073 2025-12-17 22:13:54,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867c575bc060ce15e073 HTTP/1.1" 502 150 2025-12-17 22:13:54,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940867c575bc060ce15e073'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:54,132 - lava_callback - INFO - Received callback for node 6940880d575bc060ce15e5b2 INFO: 172.21.0.1:50154 - "POST /node/6940880d575bc060ce15e5b2 HTTP/1.0" 202 Accepted 2025-12-17 22:13:54,176 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408685575bc060ce15e085 2025-12-17 22:13:54,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408685575bc060ce15e085 HTTP/1.1" 502 150 2025-12-17 22:13:54,177 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408685575bc060ce15e085'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:54,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408691575bc060ce15e0a4 2025-12-17 22:13:54,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408691575bc060ce15e0a4 HTTP/1.1" 502 150 2025-12-17 22:13:54,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408691575bc060ce15e0a4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:54,794 - lava_callback - INFO - Received callback for node 6940880e575bc060ce15e5b5 INFO: 172.21.0.1:50158 - "POST /node/6940880e575bc060ce15e5b5 HTTP/1.0" 202 Accepted 2025-12-17 22:13:54,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940867f575bc060ce15e076 2025-12-17 22:13:54,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867f575bc060ce15e076 HTTP/1.1" 502 150 2025-12-17 22:13:54,910 - lava_callback - ERROR - Error processing callback for node 6940867f575bc060ce15e076: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940867f575bc060ce15e076 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940867f575bc060ce15e076 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940867f575bc060ce15e076 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:13:54,913 - lava_callback - INFO - Starting async job submit for node 69408696575bc060ce15e0a7 2025-12-17 22:13:54,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:13:54,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408696575bc060ce15e0a7 HTTP/1.1" 502 150 2025-12-17 22:13:54,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408696575bc060ce15e0a7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:54,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408696575bc060ce15e0a7 2025-12-17 22:13:54,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408696575bc060ce15e0a7 HTTP/1.1" 502 150 2025-12-17 22:13:54,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408696575bc060ce15e0a7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:55,464 - lava_callback - INFO - Received callback for node 69408810575bc060ce15e5b8 INFO: 172.21.0.1:50160 - "POST /node/69408810575bc060ce15e5b8 HTTP/1.0" 202 Accepted 2025-12-17 22:13:56,118 - lava_callback - INFO - Received callback for node 69408811575bc060ce15e5bb INFO: 172.21.0.1:50164 - "POST /node/69408811575bc060ce15e5bb HTTP/1.0" 202 Accepted INFO: 172.21.0.1:50170 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:13:56,779 - lava_callback - INFO - Received callback for node 69408812575bc060ce15e5be INFO: 172.21.0.1:50182 - "POST /node/69408812575bc060ce15e5be HTTP/1.0" 202 Accepted 2025-12-17 22:13:56,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408696575bc060ce15e0a7 2025-12-17 22:13:56,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408696575bc060ce15e0a7 HTTP/1.1" 502 150 2025-12-17 22:13:56,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408696575bc060ce15e0a7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:57,440 - lava_callback - INFO - Received callback for node 69408813575bc060ce15e5c1 INFO: 172.21.0.1:50194 - "POST /node/69408813575bc060ce15e5c1 HTTP/1.0" 202 Accepted 2025-12-17 22:13:58,097 - lava_callback - INFO - Received callback for node 69408815575bc060ce15e5c4 INFO: 172.21.0.1:50202 - "POST /node/69408815575bc060ce15e5c4 HTTP/1.0" 202 Accepted 2025-12-17 22:13:58,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408691575bc060ce15e0a4 2025-12-17 22:13:58,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408691575bc060ce15e0a4 HTTP/1.1" 502 150 2025-12-17 22:13:58,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408691575bc060ce15e0a4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:58,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940868d575bc060ce15e09b 2025-12-17 22:13:58,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868d575bc060ce15e09b HTTP/1.1" 502 150 2025-12-17 22:13:58,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940868d575bc060ce15e09b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:58,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940868f575bc060ce15e09e 2025-12-17 22:13:58,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868f575bc060ce15e09e HTTP/1.1" 502 150 2025-12-17 22:13:58,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940868f575bc060ce15e09e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:58,761 - lava_callback - INFO - Received callback for node 6940881a575bc060ce15e5d5 INFO: 172.21.0.1:50210 - "POST /node/6940881a575bc060ce15e5d5 HTTP/1.0" 202 Accepted 2025-12-17 22:13:59,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408690575bc060ce15e0a1 2025-12-17 22:13:59,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408690575bc060ce15e0a1 HTTP/1.1" 502 150 2025-12-17 22:13:59,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408690575bc060ce15e0a1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:13:59,400 - lava_callback - INFO - Received callback for node 6940881b575bc060ce15e5dc INFO: 172.21.0.1:50220 - "POST /node/6940881b575bc060ce15e5dc HTTP/1.0" 202 Accepted 2025-12-17 22:13:59,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868c575bc060ce15e098 HTTP/1.1" 502 150 2025-12-17 22:13:59,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940868c575bc060ce15e098'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:00,050 - lava_callback - INFO - Received callback for node 6940881d575bc060ce15e5df INFO: 172.21.0.1:51920 - "POST /node/6940881d575bc060ce15e5df HTTP/1.0" 202 Accepted 2025-12-17 22:14:00,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a0575bc060ce15dcc7 2025-12-17 22:14:00,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a0575bc060ce15dcc7 HTTP/1.1" 502 150 2025-12-17 22:14:00,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085a0575bc060ce15dcc7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:00,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408696575bc060ce15e0a7 2025-12-17 22:14:00,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408696575bc060ce15e0a7 HTTP/1.1" 502 150 2025-12-17 22:14:00,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408696575bc060ce15e0a7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:01,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408680575bc060ce15e079 2025-12-17 22:14:01,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408680575bc060ce15e079 HTTP/1.1" 502 150 2025-12-17 22:14:01,487 - lava_callback - ERROR - Error processing callback for node 69408680575bc060ce15e079: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408680575bc060ce15e079 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408680575bc060ce15e079 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408680575bc060ce15e079 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:01,489 - lava_callback - INFO - Starting async job submit for node 69408698575bc060ce15e0aa 2025-12-17 22:14:01,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:01,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408698575bc060ce15e0aa HTTP/1.1" 502 150 2025-12-17 22:14:01,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408698575bc060ce15e0aa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:01,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408698575bc060ce15e0aa 2025-12-17 22:14:01,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408698575bc060ce15e0aa HTTP/1.1" 502 150 2025-12-17 22:14:01,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408698575bc060ce15e0aa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:01,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408681575bc060ce15e07c 2025-12-17 22:14:01,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408681575bc060ce15e07c HTTP/1.1" 502 150 2025-12-17 22:14:01,708 - lava_callback - ERROR - Error processing callback for node 69408681575bc060ce15e07c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408681575bc060ce15e07c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408681575bc060ce15e07c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408681575bc060ce15e07c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:01,710 - lava_callback - INFO - Starting async job submit for node 69408687575bc060ce15e090 2025-12-17 22:14:01,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:01,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408687575bc060ce15e090 HTTP/1.1" 502 150 2025-12-17 22:14:01,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408687575bc060ce15e090'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:01,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408687575bc060ce15e090 2025-12-17 22:14:01,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408687575bc060ce15e090 HTTP/1.1" 502 150 2025-12-17 22:14:01,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408687575bc060ce15e090'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:01,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408674575bc060ce15e05d 2025-12-17 22:14:01,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408674575bc060ce15e05d HTTP/1.1" 502 150 2025-12-17 22:14:01,908 - lava_callback - ERROR - Error processing callback for node 69408674575bc060ce15e05d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408674575bc060ce15e05d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408674575bc060ce15e05d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408674575bc060ce15e05d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:01,910 - lava_callback - INFO - Starting async job submit for node 69408689575bc060ce15e095 2025-12-17 22:14:01,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:01,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408689575bc060ce15e095 HTTP/1.1" 502 150 2025-12-17 22:14:01,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408689575bc060ce15e095'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:01,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408689575bc060ce15e095 2025-12-17 22:14:01,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408689575bc060ce15e095 HTTP/1.1" 502 150 2025-12-17 22:14:01,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408689575bc060ce15e095'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51932 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:14:02,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408682575bc060ce15e07f 2025-12-17 22:14:02,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408682575bc060ce15e07f HTTP/1.1" 502 150 2025-12-17 22:14:02,565 - lava_callback - ERROR - Error processing callback for node 69408682575bc060ce15e07f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408682575bc060ce15e07f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408682575bc060ce15e07f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408682575bc060ce15e07f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:02,568 - lava_callback - INFO - Starting async job submit for node 69408698575bc060ce15e0ad 2025-12-17 22:14:02,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:02,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408698575bc060ce15e0ad HTTP/1.1" 502 150 2025-12-17 22:14:02,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408698575bc060ce15e0ad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:02,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408698575bc060ce15e0ad 2025-12-17 22:14:02,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408698575bc060ce15e0ad HTTP/1.1" 502 150 2025-12-17 22:14:02,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408698575bc060ce15e0ad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:02,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408676575bc060ce15e067 2025-12-17 22:14:02,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408676575bc060ce15e067 HTTP/1.1" 502 150 2025-12-17 22:14:02,939 - lava_callback - ERROR - Error processing callback for node 69408676575bc060ce15e067: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408676575bc060ce15e067 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408676575bc060ce15e067 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408676575bc060ce15e067 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:02,942 - lava_callback - INFO - Starting async job submit for node 6940845b575bc060ce15d73d 2025-12-17 22:14:03,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408698575bc060ce15e0aa 2025-12-17 22:14:03,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:03,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845b575bc060ce15d73d HTTP/1.1" 502 150 2025-12-17 22:14:03,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845b575bc060ce15d73d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:03,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845b575bc060ce15d73d 2025-12-17 22:14:03,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845b575bc060ce15d73d HTTP/1.1" 502 150 2025-12-17 22:14:03,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845b575bc060ce15d73d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:03,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408687575bc060ce15e090 2025-12-17 22:14:03,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408687575bc060ce15e090 HTTP/1.1" 502 150 2025-12-17 22:14:03,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408687575bc060ce15e090'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:03,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940868c575bc060ce15e098 2025-12-17 22:14:03,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868c575bc060ce15e098 HTTP/1.1" 502 150 2025-12-17 22:14:03,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940868c575bc060ce15e098'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:03,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408689575bc060ce15e095 2025-12-17 22:14:03,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408689575bc060ce15e095 HTTP/1.1" 502 150 2025-12-17 22:14:03,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408689575bc060ce15e095'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:04,054 - lava_callback - INFO - Received callback for node 69408822575bc060ce15e5e8 INFO: 172.21.0.1:51940 - "POST /node/69408822575bc060ce15e5e8 HTTP/1.0" 202 Accepted 2025-12-17 22:14:04,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408698575bc060ce15e0ad 2025-12-17 22:14:04,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408698575bc060ce15e0ad HTTP/1.1" 502 150 2025-12-17 22:14:04,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408698575bc060ce15e0ad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:04,705 - lava_callback - INFO - Received callback for node 69408826575bc060ce15e636 INFO: 172.21.0.1:51942 - "POST /node/69408826575bc060ce15e636 HTTP/1.0" 202 Accepted 2025-12-17 22:14:05,352 - lava_callback - INFO - Received callback for node 69408828575bc060ce15e639 INFO: 172.21.0.1:51952 - "POST /node/69408828575bc060ce15e639 HTTP/1.0" 202 Accepted 2025-12-17 22:14:05,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408260575bc060ce15cf40 2025-12-17 22:14:05,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408260575bc060ce15cf40 HTTP/1.1" 502 150 2025-12-17 22:14:05,535 - lava_callback - ERROR - Error processing callback for node 69408260575bc060ce15cf40: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408260575bc060ce15cf40 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408260575bc060ce15cf40 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408260575bc060ce15cf40 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:05,537 - lava_callback - INFO - Starting async job submit for node 6940869b575bc060ce15e0b2 2025-12-17 22:14:05,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:05,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869b575bc060ce15e0b2 HTTP/1.1" 502 150 2025-12-17 22:14:05,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869b575bc060ce15e0b2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:05,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869b575bc060ce15e0b2 2025-12-17 22:14:05,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869b575bc060ce15e0b2 HTTP/1.1" 502 150 2025-12-17 22:14:05,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869b575bc060ce15e0b2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:05,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845b575bc060ce15d73d 2025-12-17 22:14:05,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845b575bc060ce15d73d HTTP/1.1" 502 150 2025-12-17 22:14:05,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845b575bc060ce15d73d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:06,023 - lava_callback - INFO - Received callback for node 6940882a575bc060ce15e63c INFO: 172.21.0.1:51966 - "POST /node/6940882a575bc060ce15e63c HTTP/1.0" 202 Accepted 2025-12-17 22:14:06,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408679575bc060ce15e06d 2025-12-17 22:14:06,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408679575bc060ce15e06d HTTP/1.1" 502 150 2025-12-17 22:14:06,361 - lava_callback - ERROR - Error processing callback for node 69408679575bc060ce15e06d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408679575bc060ce15e06d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408679575bc060ce15e06d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408679575bc060ce15e06d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:06,363 - lava_callback - INFO - Starting async job submit for node 6940869d575bc060ce15e0bb 2025-12-17 22:14:06,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:06,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869d575bc060ce15e0bb HTTP/1.1" 502 150 2025-12-17 22:14:06,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869d575bc060ce15e0bb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:06,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869d575bc060ce15e0bb 2025-12-17 22:14:06,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869d575bc060ce15e0bb HTTP/1.1" 502 150 2025-12-17 22:14:06,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869d575bc060ce15e0bb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:06,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408691575bc060ce15e0a4 2025-12-17 22:14:06,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408691575bc060ce15e0a4 HTTP/1.1" 502 150 2025-12-17 22:14:06,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408691575bc060ce15e0a4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51976 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:14:07,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869b575bc060ce15e0b2 2025-12-17 22:14:07,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869b575bc060ce15e0b2 HTTP/1.1" 502 150 2025-12-17 22:14:07,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869b575bc060ce15e0b2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:07,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408687575bc060ce15e090 2025-12-17 22:14:07,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408687575bc060ce15e090 HTTP/1.1" 502 150 2025-12-17 22:14:07,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408687575bc060ce15e090'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:07,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408689575bc060ce15e095 2025-12-17 22:14:07,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408689575bc060ce15e095 HTTP/1.1" 502 150 2025-12-17 22:14:07,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408689575bc060ce15e095'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:08,393 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869d575bc060ce15e0bb 2025-12-17 22:14:08,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869d575bc060ce15e0bb HTTP/1.1" 502 150 2025-12-17 22:14:08,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869d575bc060ce15e0bb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:08,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408698575bc060ce15e0ad 2025-12-17 22:14:08,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408698575bc060ce15e0ad HTTP/1.1" 502 150 2025-12-17 22:14:08,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408698575bc060ce15e0ad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:08,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408696575bc060ce15e0a7 2025-12-17 22:14:08,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408696575bc060ce15e0a7 HTTP/1.1" 502 150 2025-12-17 22:14:08,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408696575bc060ce15e0a7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:09,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845b575bc060ce15d73d 2025-12-17 22:14:09,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845b575bc060ce15d73d HTTP/1.1" 502 150 2025-12-17 22:14:09,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845b575bc060ce15d73d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:10,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940867c575bc060ce15e073 2025-12-17 22:14:10,177 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408685575bc060ce15e085 2025-12-17 22:14:10,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408685575bc060ce15e085 HTTP/1.1" 502 150 2025-12-17 22:14:10,179 - lava_callback - ERROR - Error processing callback for node 69408685575bc060ce15e085: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408685575bc060ce15e085 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408685575bc060ce15e085 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408685575bc060ce15e085 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:10,181 - lava_callback - INFO - Starting async job submit for node 6940869e575bc060ce15e0be 2025-12-17 22:14:10,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:10,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869e575bc060ce15e0be HTTP/1.1" 502 150 2025-12-17 22:14:10,187 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869e575bc060ce15e0be'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:10,188 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869e575bc060ce15e0be 2025-12-17 22:14:10,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869e575bc060ce15e0be HTTP/1.1" 502 150 2025-12-17 22:14:10,188 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869e575bc060ce15e0be'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:11,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869b575bc060ce15e0b2 2025-12-17 22:14:11,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869b575bc060ce15e0b2 HTTP/1.1" 502 150 2025-12-17 22:14:11,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869b575bc060ce15e0b2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:11,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940868c575bc060ce15e098 2025-12-17 22:14:11,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868c575bc060ce15e098 HTTP/1.1" 502 150 2025-12-17 22:14:11,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940868c575bc060ce15e098'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:12,188 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869e575bc060ce15e0be 2025-12-17 22:14:12,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869e575bc060ce15e0be HTTP/1.1" 502 150 2025-12-17 22:14:12,190 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869e575bc060ce15e0be'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:12,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869d575bc060ce15e0bb 2025-12-17 22:14:12,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869d575bc060ce15e0bb HTTP/1.1" 502 150 2025-12-17 22:14:12,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869d575bc060ce15e0bb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:14,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940868d575bc060ce15e09b 2025-12-17 22:14:14,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940868f575bc060ce15e09e 2025-12-17 22:14:14,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868f575bc060ce15e09e HTTP/1.1" 502 150 2025-12-17 22:14:14,611 - lava_callback - ERROR - Error processing callback for node 6940868f575bc060ce15e09e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940868f575bc060ce15e09e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940868f575bc060ce15e09e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940868f575bc060ce15e09e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:14,612 - lava_callback - INFO - Starting async job submit for node 6940869f575bc060ce15e0c1 2025-12-17 22:14:14,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:14,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869f575bc060ce15e0c1 HTTP/1.1" 502 150 2025-12-17 22:14:14,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869f575bc060ce15e0c1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:14,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869f575bc060ce15e0c1 2025-12-17 22:14:14,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869f575bc060ce15e0c1 HTTP/1.1" 502 150 2025-12-17 22:14:14,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869f575bc060ce15e0c1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:15,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408690575bc060ce15e0a1 2025-12-17 22:14:15,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408690575bc060ce15e0a1 HTTP/1.1" 502 150 2025-12-17 22:14:15,003 - lava_callback - ERROR - Error processing callback for node 69408690575bc060ce15e0a1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408690575bc060ce15e0a1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408690575bc060ce15e0a1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408690575bc060ce15e0a1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:15,006 - lava_callback - INFO - Starting async job submit for node 6940c48b1e3c24e95681d4ea 2025-12-17 22:14:15,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:15,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c48b1e3c24e95681d4ea HTTP/1.1" 502 150 2025-12-17 22:14:15,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c48b1e3c24e95681d4ea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:15,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c48b1e3c24e95681d4ea 2025-12-17 22:14:15,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c48b1e3c24e95681d4ea HTTP/1.1" 502 150 2025-12-17 22:14:15,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c48b1e3c24e95681d4ea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39286 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:14:15,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408687575bc060ce15e090 2025-12-17 22:14:15,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408687575bc060ce15e090 HTTP/1.1" 502 150 2025-12-17 22:14:15,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408687575bc060ce15e090'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:15,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408689575bc060ce15e095 2025-12-17 22:14:15,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408689575bc060ce15e095 HTTP/1.1" 502 150 2025-12-17 22:14:15,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408689575bc060ce15e095'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:16,190 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869e575bc060ce15e0be 2025-12-17 22:14:16,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869e575bc060ce15e0be HTTP/1.1" 502 150 2025-12-17 22:14:16,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869e575bc060ce15e0be'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39294 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:14:16,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408698575bc060ce15e0ad 2025-12-17 22:14:16,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408698575bc060ce15e0ad HTTP/1.1" 502 150 2025-12-17 22:14:16,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408698575bc060ce15e0ad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:16,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869f575bc060ce15e0c1 2025-12-17 22:14:16,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869f575bc060ce15e0c1 HTTP/1.1" 502 150 2025-12-17 22:14:16,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869f575bc060ce15e0c1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:16,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085a0575bc060ce15dcc7 2025-12-17 22:14:16,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085a0575bc060ce15dcc7 HTTP/1.1" 502 150 2025-12-17 22:14:16,695 - lava_callback - ERROR - Error processing callback for node 694085a0575bc060ce15dcc7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a0575bc060ce15dcc7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a0575bc060ce15dcc7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085a0575bc060ce15dcc7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:16,698 - lava_callback - INFO - Starting async job submit for node 694086a1575bc060ce15e0c8 2025-12-17 22:14:16,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:16,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a1575bc060ce15e0c8 HTTP/1.1" 502 150 2025-12-17 22:14:16,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a1575bc060ce15e0c8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:16,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a1575bc060ce15e0c8 2025-12-17 22:14:16,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a1575bc060ce15e0c8 HTTP/1.1" 502 150 2025-12-17 22:14:16,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a1575bc060ce15e0c8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:17,049 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c48b1e3c24e95681d4ea 2025-12-17 22:14:17,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c48b1e3c24e95681d4ea HTTP/1.1" 502 150 2025-12-17 22:14:17,050 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c48b1e3c24e95681d4ea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:17,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845b575bc060ce15d73d 2025-12-17 22:14:17,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845b575bc060ce15d73d HTTP/1.1" 502 150 2025-12-17 22:14:17,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845b575bc060ce15d73d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:18,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a1575bc060ce15e0c8 2025-12-17 22:14:18,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a1575bc060ce15e0c8 HTTP/1.1" 502 150 2025-12-17 22:14:18,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a1575bc060ce15e0c8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:18,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408698575bc060ce15e0aa HTTP/1.1" 502 150 2025-12-17 22:14:18,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408698575bc060ce15e0aa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:19,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869b575bc060ce15e0b2 2025-12-17 22:14:19,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869b575bc060ce15e0b2 HTTP/1.1" 502 150 2025-12-17 22:14:19,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869b575bc060ce15e0b2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:20,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869d575bc060ce15e0bb 2025-12-17 22:14:20,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869d575bc060ce15e0bb HTTP/1.1" 502 150 2025-12-17 22:14:20,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869d575bc060ce15e0bb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:20,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869f575bc060ce15e0c1 2025-12-17 22:14:20,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869f575bc060ce15e0c1 HTTP/1.1" 502 150 2025-12-17 22:14:20,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869f575bc060ce15e0c1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:21,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c48b1e3c24e95681d4ea 2025-12-17 22:14:21,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c48b1e3c24e95681d4ea HTTP/1.1" 502 150 2025-12-17 22:14:21,052 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c48b1e3c24e95681d4ea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:22,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408691575bc060ce15e0a4 2025-12-17 22:14:22,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408691575bc060ce15e0a4 HTTP/1.1" 502 150 2025-12-17 22:14:22,447 - lava_callback - ERROR - Error processing callback for node 69408691575bc060ce15e0a4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408691575bc060ce15e0a4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408691575bc060ce15e0a4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408691575bc060ce15e0a4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:22,449 - lava_callback - INFO - Starting async job submit for node 694086a2575bc060ce15e0cb 2025-12-17 22:14:22,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:22,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a2575bc060ce15e0cb HTTP/1.1" 502 150 2025-12-17 22:14:22,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a2575bc060ce15e0cb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:22,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a2575bc060ce15e0cb 2025-12-17 22:14:22,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a2575bc060ce15e0cb HTTP/1.1" 502 150 2025-12-17 22:14:22,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a2575bc060ce15e0cb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:22,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a1575bc060ce15e0c8 2025-12-17 22:14:22,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a1575bc060ce15e0c8 HTTP/1.1" 502 150 2025-12-17 22:14:22,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a1575bc060ce15e0c8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:22,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408698575bc060ce15e0aa 2025-12-17 22:14:22,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408698575bc060ce15e0aa HTTP/1.1" 502 150 2025-12-17 22:14:22,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408698575bc060ce15e0aa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:24,053 - lava_callback - INFO - Received callback for node 6940882c575bc060ce15e647 INFO: 172.21.0.1:36454 - "POST /node/6940882c575bc060ce15e647 HTTP/1.0" 202 Accepted 2025-12-17 22:14:24,192 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869e575bc060ce15e0be 2025-12-17 22:14:24,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869e575bc060ce15e0be HTTP/1.1" 502 150 2025-12-17 22:14:24,193 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869e575bc060ce15e0be'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:24,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a2575bc060ce15e0cb 2025-12-17 22:14:24,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a2575bc060ce15e0cb HTTP/1.1" 502 150 2025-12-17 22:14:24,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a2575bc060ce15e0cb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:24,714 - lava_callback - INFO - Received callback for node 6940882e575bc060ce15e64c INFO: 172.21.0.1:36460 - "POST /node/6940882e575bc060ce15e64c HTTP/1.0" 202 Accepted 2025-12-17 22:14:24,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408696575bc060ce15e0a7 2025-12-17 22:14:24,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408696575bc060ce15e0a7 HTTP/1.1" 502 150 2025-12-17 22:14:24,927 - lava_callback - ERROR - Error processing callback for node 69408696575bc060ce15e0a7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408696575bc060ce15e0a7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408696575bc060ce15e0a7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408696575bc060ce15e0a7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:24,929 - lava_callback - INFO - Starting async job submit for node 694086a3575bc060ce15e0ce 2025-12-17 22:14:24,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:24,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a3575bc060ce15e0ce HTTP/1.1" 502 150 2025-12-17 22:14:24,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a3575bc060ce15e0ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:24,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a3575bc060ce15e0ce 2025-12-17 22:14:24,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a3575bc060ce15e0ce HTTP/1.1" 502 150 2025-12-17 22:14:24,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a3575bc060ce15e0ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:25,361 - lava_callback - INFO - Received callback for node 69408832575bc060ce15e652 INFO: 172.21.0.1:36472 - "POST /node/69408832575bc060ce15e652 HTTP/1.0" 202 Accepted 2025-12-17 22:14:25,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940867c575bc060ce15e073 HTTP/1.1" 502 150 2025-12-17 22:14:25,456 - lava_callback - ERROR - Error processing callback for node 6940867c575bc060ce15e073: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940867c575bc060ce15e073 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940867c575bc060ce15e073 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940867c575bc060ce15e073 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:25,458 - lava_callback - INFO - Starting async job submit for node 694086a4575bc060ce15e0d1 2025-12-17 22:14:25,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:25,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a4575bc060ce15e0d1 HTTP/1.1" 502 150 2025-12-17 22:14:25,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a4575bc060ce15e0d1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:25,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a4575bc060ce15e0d1 2025-12-17 22:14:25,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a4575bc060ce15e0d1 HTTP/1.1" 502 150 2025-12-17 22:14:25,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a4575bc060ce15e0d1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:26,024 - lava_callback - INFO - Received callback for node 69408834575bc060ce15e655 INFO: 172.21.0.1:36482 - "POST /node/69408834575bc060ce15e655 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:36484 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:14:26,666 - lava_callback - INFO - Received callback for node 69408836575bc060ce15e658 INFO: 172.21.0.1:36498 - "POST /node/69408836575bc060ce15e658 HTTP/1.0" 202 Accepted 2025-12-17 22:14:26,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a3575bc060ce15e0ce 2025-12-17 22:14:26,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a3575bc060ce15e0ce HTTP/1.1" 502 150 2025-12-17 22:14:26,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a3575bc060ce15e0ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:27,312 - lava_callback - INFO - Received callback for node 69408838575bc060ce15e65b INFO: 172.21.0.1:36504 - "POST /node/69408838575bc060ce15e65b HTTP/1.0" 202 Accepted 2025-12-17 22:14:27,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a4575bc060ce15e0d1 2025-12-17 22:14:27,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a4575bc060ce15e0d1 HTTP/1.1" 502 150 2025-12-17 22:14:27,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a4575bc060ce15e0d1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:27,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940868c575bc060ce15e098 2025-12-17 22:14:27,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868c575bc060ce15e098 HTTP/1.1" 502 150 2025-12-17 22:14:27,864 - lava_callback - ERROR - Error processing callback for node 6940868c575bc060ce15e098: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940868c575bc060ce15e098 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940868c575bc060ce15e098 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940868c575bc060ce15e098 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:27,866 - lava_callback - INFO - Starting async job submit for node 694086a6575bc060ce15e0d8 2025-12-17 22:14:27,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:27,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a6575bc060ce15e0d8 HTTP/1.1" 502 150 2025-12-17 22:14:27,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a6575bc060ce15e0d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:27,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a6575bc060ce15e0d8 2025-12-17 22:14:27,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a6575bc060ce15e0d8 HTTP/1.1" 502 150 2025-12-17 22:14:27,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a6575bc060ce15e0d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:28,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a2575bc060ce15e0cb 2025-12-17 22:14:28,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a2575bc060ce15e0cb HTTP/1.1" 502 150 2025-12-17 22:14:28,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a2575bc060ce15e0cb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:28,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869f575bc060ce15e0c1 2025-12-17 22:14:28,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869f575bc060ce15e0c1 HTTP/1.1" 502 150 2025-12-17 22:14:28,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940869f575bc060ce15e0c1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:29,052 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c48b1e3c24e95681d4ea 2025-12-17 22:14:29,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940868d575bc060ce15e09b HTTP/1.1" 502 150 2025-12-17 22:14:29,809 - lava_callback - ERROR - Error processing callback for node 6940868d575bc060ce15e09b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940868d575bc060ce15e09b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940868d575bc060ce15e09b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940868d575bc060ce15e09b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:29,810 - lava_callback - INFO - Starting async job submit for node 694086a7575bc060ce15e0db 2025-12-17 22:14:29,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:29,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a7575bc060ce15e0db HTTP/1.1" 502 150 2025-12-17 22:14:29,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a7575bc060ce15e0db'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:29,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a7575bc060ce15e0db 2025-12-17 22:14:29,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a7575bc060ce15e0db HTTP/1.1" 502 150 2025-12-17 22:14:29,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a7575bc060ce15e0db'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:29,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a6575bc060ce15e0d8 2025-12-17 22:14:29,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a6575bc060ce15e0d8 HTTP/1.1" 502 150 2025-12-17 22:14:29,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a6575bc060ce15e0d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:30,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a1575bc060ce15e0c8 2025-12-17 22:14:30,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a1575bc060ce15e0c8 HTTP/1.1" 502 150 2025-12-17 22:14:30,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a1575bc060ce15e0c8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:30,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408698575bc060ce15e0aa 2025-12-17 22:14:30,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408698575bc060ce15e0aa HTTP/1.1" 502 150 2025-12-17 22:14:30,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408698575bc060ce15e0aa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:30,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a3575bc060ce15e0ce 2025-12-17 22:14:30,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a3575bc060ce15e0ce HTTP/1.1" 502 150 2025-12-17 22:14:30,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a3575bc060ce15e0ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:31,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a4575bc060ce15e0d1 2025-12-17 22:14:31,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a4575bc060ce15e0d1 HTTP/1.1" 502 150 2025-12-17 22:14:31,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a4575bc060ce15e0d1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:31,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408687575bc060ce15e090 2025-12-17 22:14:31,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408687575bc060ce15e090 HTTP/1.1" 502 150 2025-12-17 22:14:31,724 - lava_callback - ERROR - Error processing callback for node 69408687575bc060ce15e090: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408687575bc060ce15e090 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408687575bc060ce15e090 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408687575bc060ce15e090 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:31,726 - lava_callback - INFO - Starting async job submit for node 694086a8575bc060ce15e0de 2025-12-17 22:14:31,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:31,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a8575bc060ce15e0de HTTP/1.1" 502 150 2025-12-17 22:14:31,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a8575bc060ce15e0de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:31,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a8575bc060ce15e0de 2025-12-17 22:14:31,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a8575bc060ce15e0de HTTP/1.1" 502 150 2025-12-17 22:14:31,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a8575bc060ce15e0de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:31,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a7575bc060ce15e0db 2025-12-17 22:14:31,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a7575bc060ce15e0db HTTP/1.1" 502 150 2025-12-17 22:14:31,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a7575bc060ce15e0db'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:31,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408689575bc060ce15e095 2025-12-17 22:14:31,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408689575bc060ce15e095 HTTP/1.1" 502 150 2025-12-17 22:14:31,923 - lava_callback - ERROR - Error processing callback for node 69408689575bc060ce15e095: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408689575bc060ce15e095 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408689575bc060ce15e095 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408689575bc060ce15e095 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:31,925 - lava_callback - INFO - Starting async job submit for node 694086a9575bc060ce15e0e1 2025-12-17 22:14:31,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:31,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a9575bc060ce15e0e1 HTTP/1.1" 502 150 2025-12-17 22:14:31,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a9575bc060ce15e0e1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:31,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a9575bc060ce15e0e1 2025-12-17 22:14:31,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a9575bc060ce15e0e1 HTTP/1.1" 502 150 2025-12-17 22:14:31,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a9575bc060ce15e0e1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:32,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408698575bc060ce15e0ad 2025-12-17 22:14:32,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408698575bc060ce15e0ad HTTP/1.1" 502 150 2025-12-17 22:14:32,588 - lava_callback - ERROR - Error processing callback for node 69408698575bc060ce15e0ad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408698575bc060ce15e0ad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408698575bc060ce15e0ad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408698575bc060ce15e0ad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:32,590 - lava_callback - INFO - Starting async job submit for node 694086ab575bc060ce15e0e4 2025-12-17 22:14:32,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:32,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ab575bc060ce15e0e4 HTTP/1.1" 502 150 2025-12-17 22:14:32,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ab575bc060ce15e0e4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:32,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ab575bc060ce15e0e4 2025-12-17 22:14:32,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ab575bc060ce15e0e4 HTTP/1.1" 502 150 2025-12-17 22:14:32,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ab575bc060ce15e0e4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:33,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845b575bc060ce15d73d 2025-12-17 22:14:33,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845b575bc060ce15d73d HTTP/1.1" 502 150 2025-12-17 22:14:33,625 - lava_callback - ERROR - Error processing callback for node 6940845b575bc060ce15d73d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940845b575bc060ce15d73d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940845b575bc060ce15d73d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940845b575bc060ce15d73d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:33,627 - lava_callback - INFO - Starting async job submit for node 694086ac575bc060ce15e0e7 2025-12-17 22:14:33,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:33,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ac575bc060ce15e0e7 HTTP/1.1" 502 150 2025-12-17 22:14:33,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ac575bc060ce15e0e7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:33,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ac575bc060ce15e0e7 2025-12-17 22:14:33,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ac575bc060ce15e0e7 HTTP/1.1" 502 150 2025-12-17 22:14:33,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ac575bc060ce15e0e7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:33,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a8575bc060ce15e0de 2025-12-17 22:14:33,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a8575bc060ce15e0de HTTP/1.1" 502 150 2025-12-17 22:14:33,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a8575bc060ce15e0de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:33,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a6575bc060ce15e0d8 2025-12-17 22:14:33,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a6575bc060ce15e0d8 HTTP/1.1" 502 150 2025-12-17 22:14:33,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a6575bc060ce15e0d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:33,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a9575bc060ce15e0e1 2025-12-17 22:14:33,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a9575bc060ce15e0e1 HTTP/1.1" 502 150 2025-12-17 22:14:33,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a9575bc060ce15e0e1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:34,180 - lava_callback - INFO - Received callback for node 6940883a575bc060ce15e65e INFO: 172.21.0.1:43512 - "POST /node/6940883a575bc060ce15e65e HTTP/1.0" 202 Accepted 2025-12-17 22:14:34,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ab575bc060ce15e0e4 2025-12-17 22:14:34,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ab575bc060ce15e0e4 HTTP/1.1" 502 150 2025-12-17 22:14:34,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ab575bc060ce15e0e4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:34,838 - lava_callback - INFO - Received callback for node 6940883d575bc060ce15e661 INFO: 172.21.0.1:43522 - "POST /node/6940883d575bc060ce15e661 HTTP/1.0" 202 Accepted 2025-12-17 22:14:35,516 - lava_callback - INFO - Received callback for node 6940883e575bc060ce15e664 INFO: 172.21.0.1:43536 - "POST /node/6940883e575bc060ce15e664 HTTP/1.0" 202 Accepted 2025-12-17 22:14:35,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869b575bc060ce15e0b2 2025-12-17 22:14:35,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869b575bc060ce15e0b2 HTTP/1.1" 502 150 2025-12-17 22:14:35,551 - lava_callback - ERROR - Error processing callback for node 6940869b575bc060ce15e0b2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940869b575bc060ce15e0b2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940869b575bc060ce15e0b2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940869b575bc060ce15e0b2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:35,553 - lava_callback - INFO - Starting async job submit for node 694086ad575bc060ce15e0ea 2025-12-17 22:14:35,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:35,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ad575bc060ce15e0ea HTTP/1.1" 502 150 2025-12-17 22:14:35,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ad575bc060ce15e0ea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:35,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ad575bc060ce15e0ea 2025-12-17 22:14:35,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ad575bc060ce15e0ea HTTP/1.1" 502 150 2025-12-17 22:14:35,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ad575bc060ce15e0ea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:35,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ac575bc060ce15e0e7 2025-12-17 22:14:35,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ac575bc060ce15e0e7 HTTP/1.1" 502 150 2025-12-17 22:14:35,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ac575bc060ce15e0e7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:35,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a7575bc060ce15e0db 2025-12-17 22:14:35,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a7575bc060ce15e0db HTTP/1.1" 502 150 2025-12-17 22:14:35,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a7575bc060ce15e0db'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:36,201 - lava_callback - INFO - Received callback for node 6940883f575bc060ce15e66b INFO: 172.21.0.1:43538 - "POST /node/6940883f575bc060ce15e66b HTTP/1.0" 202 Accepted 2025-12-17 22:14:36,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869d575bc060ce15e0bb 2025-12-17 22:14:36,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869d575bc060ce15e0bb HTTP/1.1" 502 150 2025-12-17 22:14:36,399 - lava_callback - ERROR - Error processing callback for node 6940869d575bc060ce15e0bb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940869d575bc060ce15e0bb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940869d575bc060ce15e0bb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940869d575bc060ce15e0bb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:36,401 - lava_callback - INFO - Starting async job submit for node 694086ae575bc060ce15e0ed 2025-12-17 22:14:36,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:36,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ae575bc060ce15e0ed HTTP/1.1" 502 150 2025-12-17 22:14:36,407 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ae575bc060ce15e0ed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:36,407 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ae575bc060ce15e0ed 2025-12-17 22:14:36,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ae575bc060ce15e0ed HTTP/1.1" 502 150 2025-12-17 22:14:36,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ae575bc060ce15e0ed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:36,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a2575bc060ce15e0cb 2025-12-17 22:14:36,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a2575bc060ce15e0cb HTTP/1.1" 502 150 2025-12-17 22:14:36,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a2575bc060ce15e0cb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43540 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:14:36,861 - lava_callback - INFO - Received callback for node 69408843575bc060ce15e679 INFO: 172.21.0.1:43554 - "POST /node/69408843575bc060ce15e679 HTTP/1.0" 202 Accepted 2025-12-17 22:14:37,520 - lava_callback - INFO - Received callback for node 69408844575bc060ce15e67c INFO: 172.21.0.1:43570 - "POST /node/69408844575bc060ce15e67c HTTP/1.0" 202 Accepted 2025-12-17 22:14:37,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ad575bc060ce15e0ea 2025-12-17 22:14:37,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ad575bc060ce15e0ea HTTP/1.1" 502 150 2025-12-17 22:14:37,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ad575bc060ce15e0ea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:37,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a8575bc060ce15e0de 2025-12-17 22:14:37,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a8575bc060ce15e0de HTTP/1.1" 502 150 2025-12-17 22:14:37,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a8575bc060ce15e0de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:37,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a9575bc060ce15e0e1 2025-12-17 22:14:37,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a9575bc060ce15e0e1 HTTP/1.1" 502 150 2025-12-17 22:14:37,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a9575bc060ce15e0e1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:38,197 - lava_callback - INFO - Received callback for node 69408846575bc060ce15e67f INFO: 172.21.0.1:43584 - "POST /node/69408846575bc060ce15e67f HTTP/1.0" 202 Accepted 2025-12-17 22:14:38,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ae575bc060ce15e0ed 2025-12-17 22:14:38,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ae575bc060ce15e0ed HTTP/1.1" 502 150 2025-12-17 22:14:38,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ae575bc060ce15e0ed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:38,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ab575bc060ce15e0e4 2025-12-17 22:14:38,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ab575bc060ce15e0e4 HTTP/1.1" 502 150 2025-12-17 22:14:38,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ab575bc060ce15e0e4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:38,858 - lava_callback - INFO - Received callback for node 69408847575bc060ce15e682 INFO: 172.21.0.1:43596 - "POST /node/69408847575bc060ce15e682 HTTP/1.0" 202 Accepted 2025-12-17 22:14:38,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a3575bc060ce15e0ce 2025-12-17 22:14:38,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a3575bc060ce15e0ce HTTP/1.1" 502 150 2025-12-17 22:14:38,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a3575bc060ce15e0ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:39,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a4575bc060ce15e0d1 2025-12-17 22:14:39,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a4575bc060ce15e0d1 HTTP/1.1" 502 150 2025-12-17 22:14:39,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a4575bc060ce15e0d1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:39,514 - lava_callback - INFO - Received callback for node 69408848575bc060ce15e687 INFO: 172.21.0.1:43604 - "POST /node/69408848575bc060ce15e687 HTTP/1.0" 202 Accepted 2025-12-17 22:14:39,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ac575bc060ce15e0e7 2025-12-17 22:14:39,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ac575bc060ce15e0e7 HTTP/1.1" 502 150 2025-12-17 22:14:39,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ac575bc060ce15e0e7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:40,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869e575bc060ce15e0be 2025-12-17 22:14:41,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ad575bc060ce15e0ea 2025-12-17 22:14:41,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ad575bc060ce15e0ea HTTP/1.1" 502 150 2025-12-17 22:14:41,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ad575bc060ce15e0ea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:41,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a6575bc060ce15e0d8 2025-12-17 22:14:41,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a6575bc060ce15e0d8 HTTP/1.1" 502 150 2025-12-17 22:14:41,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a6575bc060ce15e0d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:42,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ae575bc060ce15e0ed 2025-12-17 22:14:42,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ae575bc060ce15e0ed HTTP/1.1" 502 150 2025-12-17 22:14:42,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ae575bc060ce15e0ed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:43,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a7575bc060ce15e0db 2025-12-17 22:14:43,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a7575bc060ce15e0db HTTP/1.1" 502 150 2025-12-17 22:14:43,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a7575bc060ce15e0db'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:44,047 - lava_callback - INFO - Received callback for node 6940884e575bc060ce15e693 INFO: 172.21.0.1:50880 - "POST /node/6940884e575bc060ce15e693 HTTP/1.0" 202 Accepted 2025-12-17 22:14:44,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c48b1e3c24e95681d4ea HTTP/1.1" 502 150 2025-12-17 22:14:44,405 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c48b1e3c24e95681d4ea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:44,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940869f575bc060ce15e0c1 2025-12-17 22:14:44,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869f575bc060ce15e0c1 HTTP/1.1" 502 150 2025-12-17 22:14:44,629 - lava_callback - ERROR - Error processing callback for node 6940869f575bc060ce15e0c1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940869f575bc060ce15e0c1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940869f575bc060ce15e0c1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940869f575bc060ce15e0c1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:44,632 - lava_callback - INFO - Starting async job submit for node 69408481575bc060ce15d7de 2025-12-17 22:14:44,708 - lava_callback - INFO - Received callback for node 69408851575bc060ce15e696 INFO: 172.21.0.1:50890 - "POST /node/69408851575bc060ce15e696 HTTP/1.0" 202 Accepted 2025-12-17 22:14:44,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:44,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408481575bc060ce15d7de HTTP/1.1" 502 150 2025-12-17 22:14:44,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408481575bc060ce15d7de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:44,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408481575bc060ce15d7de 2025-12-17 22:14:44,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408481575bc060ce15d7de HTTP/1.1" 502 150 2025-12-17 22:14:44,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408481575bc060ce15d7de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:45,345 - lava_callback - INFO - Received callback for node 69408853575bc060ce15e69b INFO: 172.21.0.1:50896 - "POST /node/69408853575bc060ce15e69b HTTP/1.0" 202 Accepted 2025-12-17 22:14:45,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a8575bc060ce15e0de 2025-12-17 22:14:45,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a8575bc060ce15e0de HTTP/1.1" 502 150 2025-12-17 22:14:45,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a8575bc060ce15e0de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:45,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a9575bc060ce15e0e1 2025-12-17 22:14:45,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a9575bc060ce15e0e1 HTTP/1.1" 502 150 2025-12-17 22:14:45,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086a9575bc060ce15e0e1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50902 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:14:46,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ab575bc060ce15e0e4 2025-12-17 22:14:46,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ab575bc060ce15e0e4 HTTP/1.1" 502 150 2025-12-17 22:14:46,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ab575bc060ce15e0e4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:46,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a1575bc060ce15e0c8 2025-12-17 22:14:46,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a1575bc060ce15e0c8 HTTP/1.1" 502 150 2025-12-17 22:14:46,716 - lava_callback - ERROR - Error processing callback for node 694086a1575bc060ce15e0c8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a1575bc060ce15e0c8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a1575bc060ce15e0c8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a1575bc060ce15e0c8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:46,717 - lava_callback - INFO - Starting async job submit for node 694086b1575bc060ce15e121 2025-12-17 22:14:46,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:46,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b1575bc060ce15e121 HTTP/1.1" 502 150 2025-12-17 22:14:46,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b1575bc060ce15e121'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:46,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b1575bc060ce15e121 2025-12-17 22:14:46,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b1575bc060ce15e121 HTTP/1.1" 502 150 2025-12-17 22:14:46,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b1575bc060ce15e121'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:46,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408698575bc060ce15e0aa 2025-12-17 22:14:46,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408698575bc060ce15e0aa HTTP/1.1" 502 150 2025-12-17 22:14:46,804 - lava_callback - ERROR - Error processing callback for node 69408698575bc060ce15e0aa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408698575bc060ce15e0aa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408698575bc060ce15e0aa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408698575bc060ce15e0aa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:46,806 - lava_callback - INFO - Starting async job submit for node 694086b2575bc060ce15e132 2025-12-17 22:14:46,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:46,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b2575bc060ce15e132 HTTP/1.1" 502 150 2025-12-17 22:14:46,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b2575bc060ce15e132'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:46,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b2575bc060ce15e132 2025-12-17 22:14:46,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b2575bc060ce15e132 HTTP/1.1" 502 150 2025-12-17 22:14:46,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b2575bc060ce15e132'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:46,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408481575bc060ce15d7de 2025-12-17 22:14:46,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408481575bc060ce15d7de HTTP/1.1" 502 150 2025-12-17 22:14:46,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408481575bc060ce15d7de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:47,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ac575bc060ce15e0e7 2025-12-17 22:14:47,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ac575bc060ce15e0e7 HTTP/1.1" 502 150 2025-12-17 22:14:47,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ac575bc060ce15e0e7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:48,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b1575bc060ce15e121 2025-12-17 22:14:48,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b1575bc060ce15e121 HTTP/1.1" 502 150 2025-12-17 22:14:48,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b1575bc060ce15e121'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:48,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b2575bc060ce15e132 2025-12-17 22:14:48,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b2575bc060ce15e132 HTTP/1.1" 502 150 2025-12-17 22:14:48,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b2575bc060ce15e132'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:49,197 - lava_callback - INFO - Received callback for node 694091b9575bc060ce16091d INFO: 172.21.0.1:50906 - "POST /node/694091b9575bc060ce16091d HTTP/1.0" 202 Accepted 2025-12-17 22:14:49,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ad575bc060ce15e0ea 2025-12-17 22:14:49,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ad575bc060ce15e0ea HTTP/1.1" 502 150 2025-12-17 22:14:49,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ad575bc060ce15e0ea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:50,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ae575bc060ce15e0ed 2025-12-17 22:14:50,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ae575bc060ce15e0ed HTTP/1.1" 502 150 2025-12-17 22:14:50,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ae575bc060ce15e0ed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:50,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408481575bc060ce15d7de 2025-12-17 22:14:50,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408481575bc060ce15d7de HTTP/1.1" 502 150 2025-12-17 22:14:50,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408481575bc060ce15d7de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:52,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a2575bc060ce15e0cb 2025-12-17 22:14:52,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a2575bc060ce15e0cb HTTP/1.1" 502 150 2025-12-17 22:14:52,470 - lava_callback - ERROR - Error processing callback for node 694086a2575bc060ce15e0cb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a2575bc060ce15e0cb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a2575bc060ce15e0cb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a2575bc060ce15e0cb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:52,473 - lava_callback - INFO - Starting async job submit for node 694086b4575bc060ce15e139 2025-12-17 22:14:52,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:52,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b4575bc060ce15e139 HTTP/1.1" 502 150 2025-12-17 22:14:52,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b4575bc060ce15e139'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:52,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b4575bc060ce15e139 2025-12-17 22:14:52,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b4575bc060ce15e139 HTTP/1.1" 502 150 2025-12-17 22:14:52,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b4575bc060ce15e139'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:52,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b1575bc060ce15e121 2025-12-17 22:14:52,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b1575bc060ce15e121 HTTP/1.1" 502 150 2025-12-17 22:14:52,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b1575bc060ce15e121'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:52,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b2575bc060ce15e132 2025-12-17 22:14:52,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b2575bc060ce15e132 HTTP/1.1" 502 150 2025-12-17 22:14:52,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b2575bc060ce15e132'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:54,147 - lava_callback - INFO - Received callback for node 69408855575bc060ce15e6a5 INFO: 172.21.0.1:55068 - "POST /node/69408855575bc060ce15e6a5 HTTP/1.0" 202 Accepted 2025-12-17 22:14:54,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b4575bc060ce15e139 2025-12-17 22:14:54,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b4575bc060ce15e139 HTTP/1.1" 502 150 2025-12-17 22:14:54,495 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b4575bc060ce15e139'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:54,833 - lava_callback - INFO - Received callback for node 69408857575bc060ce15e6ab INFO: 172.21.0.1:55080 - "POST /node/69408857575bc060ce15e6ab HTTP/1.0" 202 Accepted 2025-12-17 22:14:54,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a3575bc060ce15e0ce 2025-12-17 22:14:54,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a3575bc060ce15e0ce HTTP/1.1" 502 150 2025-12-17 22:14:54,963 - lava_callback - ERROR - Error processing callback for node 694086a3575bc060ce15e0ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a3575bc060ce15e0ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a3575bc060ce15e0ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a3575bc060ce15e0ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:54,965 - lava_callback - INFO - Starting async job submit for node 694086b5575bc060ce15e140 2025-12-17 22:14:54,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:54,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b5575bc060ce15e140 HTTP/1.1" 502 150 2025-12-17 22:14:54,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b5575bc060ce15e140'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:54,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b5575bc060ce15e140 2025-12-17 22:14:54,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b5575bc060ce15e140 HTTP/1.1" 502 150 2025-12-17 22:14:54,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b5575bc060ce15e140'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:55,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a4575bc060ce15e0d1 2025-12-17 22:14:55,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a4575bc060ce15e0d1 HTTP/1.1" 502 150 2025-12-17 22:14:55,471 - lava_callback - ERROR - Error processing callback for node 694086a4575bc060ce15e0d1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a4575bc060ce15e0d1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a4575bc060ce15e0d1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a4575bc060ce15e0d1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:55,473 - lava_callback - INFO - Starting async job submit for node 694086b6575bc060ce15e147 2025-12-17 22:14:55,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:55,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b6575bc060ce15e147 HTTP/1.1" 502 150 2025-12-17 22:14:55,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b6575bc060ce15e147'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:55,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b6575bc060ce15e147 2025-12-17 22:14:55,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b6575bc060ce15e147 HTTP/1.1" 502 150 2025-12-17 22:14:55,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b6575bc060ce15e147'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:55,506 - lava_callback - INFO - Received callback for node 69408859575bc060ce15e6ae INFO: 172.21.0.1:55092 - "POST /node/69408859575bc060ce15e6ae HTTP/1.0" 202 Accepted 2025-12-17 22:14:55,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940869e575bc060ce15e0be HTTP/1.1" 502 150 2025-12-17 22:14:55,664 - lava_callback - ERROR - Error processing callback for node 6940869e575bc060ce15e0be: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940869e575bc060ce15e0be (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940869e575bc060ce15e0be (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940869e575bc060ce15e0be (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:55,666 - lava_callback - INFO - Starting async job submit for node 694086b7575bc060ce15e14a 2025-12-17 22:14:55,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:55,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b7575bc060ce15e14a HTTP/1.1" 502 150 2025-12-17 22:14:55,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b7575bc060ce15e14a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:55,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b7575bc060ce15e14a 2025-12-17 22:14:55,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b7575bc060ce15e14a HTTP/1.1" 502 150 2025-12-17 22:14:55,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b7575bc060ce15e14a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:56,182 - lava_callback - INFO - Received callback for node 6940884a575bc060ce15e68a INFO: 172.21.0.1:55098 - "POST /node/6940884a575bc060ce15e68a HTTP/1.0" 202 Accepted INFO: 172.21.0.1:55112 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:14:56,849 - lava_callback - INFO - Received callback for node 6940884b575bc060ce15e68d INFO: 172.21.0.1:55122 - "POST /node/6940884b575bc060ce15e68d HTTP/1.0" 202 Accepted 2025-12-17 22:14:56,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b5575bc060ce15e140 2025-12-17 22:14:56,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b5575bc060ce15e140 HTTP/1.1" 502 150 2025-12-17 22:14:56,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b5575bc060ce15e140'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:57,503 - lava_callback - INFO - Received callback for node 6940884c575bc060ce15e690 2025-12-17 22:14:57,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b6575bc060ce15e147 INFO: 172.21.0.1:55138 - "POST /node/6940884c575bc060ce15e690 HTTP/1.0" 202 Accepted 2025-12-17 22:14:57,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b6575bc060ce15e147 HTTP/1.1" 502 150 2025-12-17 22:14:57,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b6575bc060ce15e147'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:57,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b7575bc060ce15e14a 2025-12-17 22:14:57,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b7575bc060ce15e14a HTTP/1.1" 502 150 2025-12-17 22:14:57,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b7575bc060ce15e14a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:57,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a6575bc060ce15e0d8 2025-12-17 22:14:57,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a6575bc060ce15e0d8 HTTP/1.1" 502 150 2025-12-17 22:14:57,883 - lava_callback - ERROR - Error processing callback for node 694086a6575bc060ce15e0d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a6575bc060ce15e0d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a6575bc060ce15e0d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a6575bc060ce15e0d8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:57,884 - lava_callback - INFO - Starting async job submit for node 694086b9575bc060ce15e151 2025-12-17 22:14:57,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:57,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b9575bc060ce15e151 HTTP/1.1" 502 150 2025-12-17 22:14:57,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b9575bc060ce15e151'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:57,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b9575bc060ce15e151 2025-12-17 22:14:57,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b9575bc060ce15e151 HTTP/1.1" 502 150 2025-12-17 22:14:57,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b9575bc060ce15e151'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:58,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b4575bc060ce15e139 2025-12-17 22:14:58,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b4575bc060ce15e139 HTTP/1.1" 502 150 2025-12-17 22:14:58,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b4575bc060ce15e139'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:58,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408481575bc060ce15d7de 2025-12-17 22:14:58,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408481575bc060ce15d7de HTTP/1.1" 502 150 2025-12-17 22:14:58,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408481575bc060ce15d7de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:59,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a7575bc060ce15e0db 2025-12-17 22:14:59,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a7575bc060ce15e0db HTTP/1.1" 502 150 2025-12-17 22:14:59,824 - lava_callback - ERROR - Error processing callback for node 694086a7575bc060ce15e0db: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a7575bc060ce15e0db (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a7575bc060ce15e0db (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a7575bc060ce15e0db (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:14:59,825 - lava_callback - INFO - Starting async job submit for node 694086ba575bc060ce15e154 2025-12-17 22:14:59,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:14:59,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ba575bc060ce15e154 HTTP/1.1" 502 150 2025-12-17 22:14:59,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ba575bc060ce15e154'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:59,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ba575bc060ce15e154 2025-12-17 22:14:59,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ba575bc060ce15e154 HTTP/1.1" 502 150 2025-12-17 22:14:59,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ba575bc060ce15e154'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:14:59,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b9575bc060ce15e151 2025-12-17 22:14:59,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b9575bc060ce15e151 HTTP/1.1" 502 150 2025-12-17 22:14:59,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b9575bc060ce15e151'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:00,405 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c48b1e3c24e95681d4ea 2025-12-17 22:15:00,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c48b1e3c24e95681d4ea HTTP/1.1" 502 150 2025-12-17 22:15:00,406 - lava_callback - ERROR - Error processing callback for node 6940c48b1e3c24e95681d4ea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c48b1e3c24e95681d4ea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c48b1e3c24e95681d4ea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c48b1e3c24e95681d4ea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:00,408 - lava_callback - INFO - Starting async job submit for node 694086bb575bc060ce15e157 2025-12-17 22:15:00,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:00,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bb575bc060ce15e157 HTTP/1.1" 502 150 2025-12-17 22:15:00,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086bb575bc060ce15e157'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:00,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086bb575bc060ce15e157 2025-12-17 22:15:00,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bb575bc060ce15e157 HTTP/1.1" 502 150 2025-12-17 22:15:00,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086bb575bc060ce15e157'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:00,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b1575bc060ce15e121 2025-12-17 22:15:00,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b1575bc060ce15e121 HTTP/1.1" 502 150 2025-12-17 22:15:00,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b1575bc060ce15e121'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:00,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b2575bc060ce15e132 2025-12-17 22:15:00,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b2575bc060ce15e132 HTTP/1.1" 502 150 2025-12-17 22:15:00,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b2575bc060ce15e132'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:00,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b5575bc060ce15e140 2025-12-17 22:15:00,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b5575bc060ce15e140 HTTP/1.1" 502 150 2025-12-17 22:15:00,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b5575bc060ce15e140'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:01,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b6575bc060ce15e147 2025-12-17 22:15:01,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b6575bc060ce15e147 HTTP/1.1" 502 150 2025-12-17 22:15:01,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b6575bc060ce15e147'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:01,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b7575bc060ce15e14a 2025-12-17 22:15:01,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b7575bc060ce15e14a HTTP/1.1" 502 150 2025-12-17 22:15:01,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b7575bc060ce15e14a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:01,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a8575bc060ce15e0de 2025-12-17 22:15:01,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a8575bc060ce15e0de HTTP/1.1" 502 150 2025-12-17 22:15:01,763 - lava_callback - ERROR - Error processing callback for node 694086a8575bc060ce15e0de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a8575bc060ce15e0de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a8575bc060ce15e0de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a8575bc060ce15e0de (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:01,766 - lava_callback - INFO - Starting async job submit for node 694086bd575bc060ce15e15a 2025-12-17 22:15:01,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:01,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bd575bc060ce15e15a HTTP/1.1" 502 150 2025-12-17 22:15:01,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086bd575bc060ce15e15a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:01,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086bd575bc060ce15e15a 2025-12-17 22:15:01,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bd575bc060ce15e15a HTTP/1.1" 502 150 2025-12-17 22:15:01,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086bd575bc060ce15e15a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:01,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ba575bc060ce15e154 2025-12-17 22:15:01,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ba575bc060ce15e154 HTTP/1.1" 502 150 2025-12-17 22:15:01,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ba575bc060ce15e154'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:01,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086a9575bc060ce15e0e1 2025-12-17 22:15:01,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086a9575bc060ce15e0e1 HTTP/1.1" 502 150 2025-12-17 22:15:01,943 - lava_callback - ERROR - Error processing callback for node 694086a9575bc060ce15e0e1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a9575bc060ce15e0e1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a9575bc060ce15e0e1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086a9575bc060ce15e0e1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:01,945 - lava_callback - INFO - Starting async job submit for node 694086be575bc060ce15e161 2025-12-17 22:15:01,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:01,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086be575bc060ce15e161 HTTP/1.1" 502 150 2025-12-17 22:15:01,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086be575bc060ce15e161'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:01,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086be575bc060ce15e161 2025-12-17 22:15:01,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086be575bc060ce15e161 HTTP/1.1" 502 150 2025-12-17 22:15:01,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086be575bc060ce15e161'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59698 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:15:02,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086bb575bc060ce15e157 2025-12-17 22:15:02,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ab575bc060ce15e0e4 2025-12-17 22:15:02,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ab575bc060ce15e0e4 HTTP/1.1" 502 150 2025-12-17 22:15:02,606 - lava_callback - ERROR - Error processing callback for node 694086ab575bc060ce15e0e4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ab575bc060ce15e0e4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ab575bc060ce15e0e4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ab575bc060ce15e0e4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:02,608 - lava_callback - INFO - Starting async job submit for node 694086bf575bc060ce15e164 2025-12-17 22:15:02,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:02,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bf575bc060ce15e164 HTTP/1.1" 502 150 2025-12-17 22:15:02,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086bf575bc060ce15e164'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:02,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086bf575bc060ce15e164 2025-12-17 22:15:02,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bf575bc060ce15e164 HTTP/1.1" 502 150 2025-12-17 22:15:02,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086bf575bc060ce15e164'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:03,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ac575bc060ce15e0e7 2025-12-17 22:15:03,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ac575bc060ce15e0e7 HTTP/1.1" 502 150 2025-12-17 22:15:03,648 - lava_callback - ERROR - Error processing callback for node 694086ac575bc060ce15e0e7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ac575bc060ce15e0e7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ac575bc060ce15e0e7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ac575bc060ce15e0e7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:03,650 - lava_callback - INFO - Starting async job submit for node 694086c0575bc060ce15e167 2025-12-17 22:15:03,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:03,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c0575bc060ce15e167 HTTP/1.1" 502 150 2025-12-17 22:15:03,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c0575bc060ce15e167'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:03,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c0575bc060ce15e167 2025-12-17 22:15:03,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c0575bc060ce15e167 HTTP/1.1" 502 150 2025-12-17 22:15:03,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c0575bc060ce15e167'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:03,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086bd575bc060ce15e15a 2025-12-17 22:15:03,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bd575bc060ce15e15a HTTP/1.1" 502 150 2025-12-17 22:15:03,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086bd575bc060ce15e15a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:03,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b9575bc060ce15e151 2025-12-17 22:15:03,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b9575bc060ce15e151 HTTP/1.1" 502 150 2025-12-17 22:15:03,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b9575bc060ce15e151'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:03,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086be575bc060ce15e161 2025-12-17 22:15:03,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086be575bc060ce15e161 HTTP/1.1" 502 150 2025-12-17 22:15:03,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086be575bc060ce15e161'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:04,101 - lava_callback - INFO - Received callback for node 6940885b575bc060ce15e6b1 INFO: 172.21.0.1:59706 - "POST /node/6940885b575bc060ce15e6b1 HTTP/1.0" 202 Accepted 2025-12-17 22:15:04,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086bf575bc060ce15e164 2025-12-17 22:15:04,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bf575bc060ce15e164 HTTP/1.1" 502 150 2025-12-17 22:15:04,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086bf575bc060ce15e164'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:04,759 - lava_callback - INFO - Received callback for node 6940885d575bc060ce15e6b4 INFO: 172.21.0.1:59710 - "POST /node/6940885d575bc060ce15e6b4 HTTP/1.0" 202 Accepted 2025-12-17 22:15:05,405 - lava_callback - INFO - Received callback for node 69408861575bc060ce15e6c1 INFO: 172.21.0.1:59714 - "POST /node/69408861575bc060ce15e6c1 HTTP/1.0" 202 Accepted 2025-12-17 22:15:05,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ad575bc060ce15e0ea 2025-12-17 22:15:05,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ad575bc060ce15e0ea HTTP/1.1" 502 150 2025-12-17 22:15:05,566 - lava_callback - ERROR - Error processing callback for node 694086ad575bc060ce15e0ea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ad575bc060ce15e0ea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ad575bc060ce15e0ea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ad575bc060ce15e0ea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:05,568 - lava_callback - INFO - Starting async job submit for node 694086c1575bc060ce15e16a 2025-12-17 22:15:05,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:05,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c1575bc060ce15e16a HTTP/1.1" 502 150 2025-12-17 22:15:05,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c1575bc060ce15e16a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:05,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c1575bc060ce15e16a 2025-12-17 22:15:05,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c1575bc060ce15e16a HTTP/1.1" 502 150 2025-12-17 22:15:05,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c1575bc060ce15e16a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:05,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c0575bc060ce15e167 2025-12-17 22:15:05,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c0575bc060ce15e167 HTTP/1.1" 502 150 2025-12-17 22:15:05,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c0575bc060ce15e167'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:05,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ba575bc060ce15e154 2025-12-17 22:15:05,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ba575bc060ce15e154 HTTP/1.1" 502 150 2025-12-17 22:15:05,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ba575bc060ce15e154'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:06,033 - lava_callback - INFO - Received callback for node 69408864575bc060ce15e6f6 INFO: 172.21.0.1:59726 - "POST /node/69408864575bc060ce15e6f6 HTTP/1.0" 202 Accepted 2025-12-17 22:15:06,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ae575bc060ce15e0ed 2025-12-17 22:15:06,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b4575bc060ce15e139 2025-12-17 22:15:06,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b4575bc060ce15e139 HTTP/1.1" 502 150 2025-12-17 22:15:06,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b4575bc060ce15e139'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59732 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:15:07,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c1575bc060ce15e16a 2025-12-17 22:15:07,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c1575bc060ce15e16a HTTP/1.1" 502 150 2025-12-17 22:15:07,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c1575bc060ce15e16a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:07,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086bd575bc060ce15e15a 2025-12-17 22:15:07,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bd575bc060ce15e15a HTTP/1.1" 502 150 2025-12-17 22:15:07,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086bd575bc060ce15e15a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:07,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086be575bc060ce15e161 2025-12-17 22:15:07,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086be575bc060ce15e161 HTTP/1.1" 502 150 2025-12-17 22:15:07,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086be575bc060ce15e161'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:08,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086bf575bc060ce15e164 2025-12-17 22:15:08,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bf575bc060ce15e164 HTTP/1.1" 502 150 2025-12-17 22:15:08,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086bf575bc060ce15e164'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:08,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b5575bc060ce15e140 2025-12-17 22:15:08,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b5575bc060ce15e140 HTTP/1.1" 502 150 2025-12-17 22:15:08,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b5575bc060ce15e140'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:09,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b6575bc060ce15e147 2025-12-17 22:15:09,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b6575bc060ce15e147 HTTP/1.1" 502 150 2025-12-17 22:15:09,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b6575bc060ce15e147'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:09,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c0575bc060ce15e167 2025-12-17 22:15:09,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c0575bc060ce15e167 HTTP/1.1" 502 150 2025-12-17 22:15:09,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c0575bc060ce15e167'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:09,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b7575bc060ce15e14a 2025-12-17 22:15:09,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b7575bc060ce15e14a HTTP/1.1" 502 150 2025-12-17 22:15:09,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b7575bc060ce15e14a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:11,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c1575bc060ce15e16a 2025-12-17 22:15:11,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c1575bc060ce15e16a HTTP/1.1" 502 150 2025-12-17 22:15:11,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c1575bc060ce15e16a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:11,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b9575bc060ce15e151 2025-12-17 22:15:11,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b9575bc060ce15e151 HTTP/1.1" 502 150 2025-12-17 22:15:11,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086b9575bc060ce15e151'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:13,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ba575bc060ce15e154 2025-12-17 22:15:13,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ba575bc060ce15e154 HTTP/1.1" 502 150 2025-12-17 22:15:13,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ba575bc060ce15e154'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:14,190 - lava_callback - INFO - Received callback for node 6940885f575bc060ce15e6bb INFO: 172.21.0.1:41960 - "POST /node/6940885f575bc060ce15e6bb HTTP/1.0" 202 Accepted 2025-12-17 22:15:14,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408481575bc060ce15d7de INFO: 172.21.0.1:41970 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:15:15,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086bd575bc060ce15e15a 2025-12-17 22:15:15,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bd575bc060ce15e15a HTTP/1.1" 502 150 2025-12-17 22:15:15,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086bd575bc060ce15e15a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:15,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086be575bc060ce15e161 2025-12-17 22:15:15,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086be575bc060ce15e161 HTTP/1.1" 502 150 2025-12-17 22:15:15,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086be575bc060ce15e161'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41972 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:15:16,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086bf575bc060ce15e164 2025-12-17 22:15:16,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bf575bc060ce15e164 HTTP/1.1" 502 150 2025-12-17 22:15:16,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086bf575bc060ce15e164'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:16,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b1575bc060ce15e121 2025-12-17 22:15:16,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b1575bc060ce15e121 HTTP/1.1" 502 150 2025-12-17 22:15:16,733 - lava_callback - ERROR - Error processing callback for node 694086b1575bc060ce15e121: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b1575bc060ce15e121 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b1575bc060ce15e121 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b1575bc060ce15e121 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:16,736 - lava_callback - INFO - Starting async job submit for node 694086ca575bc060ce15e18d 2025-12-17 22:15:16,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:16,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ca575bc060ce15e18d HTTP/1.1" 502 150 2025-12-17 22:15:16,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ca575bc060ce15e18d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:16,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ca575bc060ce15e18d 2025-12-17 22:15:16,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ca575bc060ce15e18d HTTP/1.1" 502 150 2025-12-17 22:15:16,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ca575bc060ce15e18d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:16,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b2575bc060ce15e132 2025-12-17 22:15:16,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b2575bc060ce15e132 HTTP/1.1" 502 150 2025-12-17 22:15:16,821 - lava_callback - ERROR - Error processing callback for node 694086b2575bc060ce15e132: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b2575bc060ce15e132 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b2575bc060ce15e132 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b2575bc060ce15e132 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:16,824 - lava_callback - INFO - Starting async job submit for node 694086cc575bc060ce15e193 2025-12-17 22:15:16,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:16,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cc575bc060ce15e193 HTTP/1.1" 502 150 2025-12-17 22:15:16,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cc575bc060ce15e193'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:16,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cc575bc060ce15e193 2025-12-17 22:15:16,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cc575bc060ce15e193 HTTP/1.1" 502 150 2025-12-17 22:15:16,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cc575bc060ce15e193'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:17,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c0575bc060ce15e167 2025-12-17 22:15:17,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c0575bc060ce15e167 HTTP/1.1" 502 150 2025-12-17 22:15:17,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c0575bc060ce15e167'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:17,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bb575bc060ce15e157 HTTP/1.1" 502 150 2025-12-17 22:15:17,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086bb575bc060ce15e157'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:18,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ca575bc060ce15e18d 2025-12-17 22:15:18,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ca575bc060ce15e18d HTTP/1.1" 502 150 2025-12-17 22:15:18,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ca575bc060ce15e18d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:18,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cc575bc060ce15e193 2025-12-17 22:15:18,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cc575bc060ce15e193 HTTP/1.1" 502 150 2025-12-17 22:15:18,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cc575bc060ce15e193'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:19,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c1575bc060ce15e16a 2025-12-17 22:15:19,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c1575bc060ce15e16a HTTP/1.1" 502 150 2025-12-17 22:15:19,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c1575bc060ce15e16a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:21,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086bb575bc060ce15e157 2025-12-17 22:15:21,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bb575bc060ce15e157 HTTP/1.1" 502 150 2025-12-17 22:15:21,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086bb575bc060ce15e157'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:21,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ae575bc060ce15e0ed HTTP/1.1" 502 150 2025-12-17 22:15:21,776 - lava_callback - ERROR - Error processing callback for node 694086ae575bc060ce15e0ed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ae575bc060ce15e0ed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ae575bc060ce15e0ed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ae575bc060ce15e0ed (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:21,778 - lava_callback - INFO - Starting async job submit for node 694086cd575bc060ce15e19a 2025-12-17 22:15:21,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:21,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cd575bc060ce15e19a HTTP/1.1" 502 150 2025-12-17 22:15:21,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cd575bc060ce15e19a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:21,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cd575bc060ce15e19a 2025-12-17 22:15:21,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cd575bc060ce15e19a HTTP/1.1" 502 150 2025-12-17 22:15:21,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cd575bc060ce15e19a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:22,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b4575bc060ce15e139 2025-12-17 22:15:22,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b4575bc060ce15e139 HTTP/1.1" 502 150 2025-12-17 22:15:22,499 - lava_callback - ERROR - Error processing callback for node 694086b4575bc060ce15e139: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b4575bc060ce15e139 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b4575bc060ce15e139 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b4575bc060ce15e139 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:22,501 - lava_callback - INFO - Starting async job submit for node 694086cf575bc060ce15e19d 2025-12-17 22:15:22,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:22,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cf575bc060ce15e19d HTTP/1.1" 502 150 2025-12-17 22:15:22,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cf575bc060ce15e19d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:22,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cf575bc060ce15e19d 2025-12-17 22:15:22,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cf575bc060ce15e19d HTTP/1.1" 502 150 2025-12-17 22:15:22,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cf575bc060ce15e19d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:22,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ca575bc060ce15e18d 2025-12-17 22:15:22,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ca575bc060ce15e18d HTTP/1.1" 502 150 2025-12-17 22:15:22,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ca575bc060ce15e18d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:22,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cc575bc060ce15e193 2025-12-17 22:15:22,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cc575bc060ce15e193 HTTP/1.1" 502 150 2025-12-17 22:15:22,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cc575bc060ce15e193'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:23,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cd575bc060ce15e19a 2025-12-17 22:15:23,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cd575bc060ce15e19a HTTP/1.1" 502 150 2025-12-17 22:15:23,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cd575bc060ce15e19a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:24,101 - lava_callback - INFO - Received callback for node 6940886a575bc060ce15e703 INFO: 172.21.0.1:41642 - "POST /node/6940886a575bc060ce15e703 HTTP/1.0" 202 Accepted 2025-12-17 22:15:24,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cf575bc060ce15e19d 2025-12-17 22:15:24,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cf575bc060ce15e19d HTTP/1.1" 502 150 2025-12-17 22:15:24,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cf575bc060ce15e19d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:24,735 - lava_callback - INFO - Received callback for node 6940886d575bc060ce15e706 INFO: 172.21.0.1:41656 - "POST /node/6940886d575bc060ce15e706 HTTP/1.0" 202 Accepted 2025-12-17 22:15:24,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b5575bc060ce15e140 2025-12-17 22:15:24,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b5575bc060ce15e140 HTTP/1.1" 502 150 2025-12-17 22:15:24,980 - lava_callback - ERROR - Error processing callback for node 694086b5575bc060ce15e140: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b5575bc060ce15e140 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b5575bc060ce15e140 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b5575bc060ce15e140 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:24,982 - lava_callback - INFO - Starting async job submit for node 694086d0575bc060ce15e1a0 2025-12-17 22:15:24,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:24,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d0575bc060ce15e1a0 HTTP/1.1" 502 150 2025-12-17 22:15:24,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d0575bc060ce15e1a0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:24,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d0575bc060ce15e1a0 2025-12-17 22:15:24,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d0575bc060ce15e1a0 HTTP/1.1" 502 150 2025-12-17 22:15:24,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d0575bc060ce15e1a0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:25,368 - lava_callback - INFO - Received callback for node 69408870575bc060ce15e70e INFO: 172.21.0.1:41660 - "POST /node/69408870575bc060ce15e70e HTTP/1.0" 202 Accepted 2025-12-17 22:15:25,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b6575bc060ce15e147 2025-12-17 22:15:25,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b6575bc060ce15e147 HTTP/1.1" 502 150 2025-12-17 22:15:25,514 - lava_callback - ERROR - Error processing callback for node 694086b6575bc060ce15e147: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b6575bc060ce15e147 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b6575bc060ce15e147 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b6575bc060ce15e147 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:25,517 - lava_callback - INFO - Starting async job submit for node 694086d1575bc060ce15e1a3 2025-12-17 22:15:25,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:25,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d1575bc060ce15e1a3 HTTP/1.1" 502 150 2025-12-17 22:15:25,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d1575bc060ce15e1a3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:25,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d1575bc060ce15e1a3 2025-12-17 22:15:25,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d1575bc060ce15e1a3 HTTP/1.1" 502 150 2025-12-17 22:15:25,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d1575bc060ce15e1a3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:25,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b7575bc060ce15e14a 2025-12-17 22:15:25,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b7575bc060ce15e14a HTTP/1.1" 502 150 2025-12-17 22:15:25,678 - lava_callback - ERROR - Error processing callback for node 694086b7575bc060ce15e14a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b7575bc060ce15e14a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b7575bc060ce15e14a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b7575bc060ce15e14a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:25,681 - lava_callback - INFO - Starting async job submit for node 694086d2575bc060ce15e1a6 2025-12-17 22:15:25,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:25,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d2575bc060ce15e1a6 HTTP/1.1" 502 150 2025-12-17 22:15:25,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d2575bc060ce15e1a6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:25,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d2575bc060ce15e1a6 2025-12-17 22:15:25,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d2575bc060ce15e1a6 HTTP/1.1" 502 150 2025-12-17 22:15:25,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d2575bc060ce15e1a6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:26,000 - lava_callback - INFO - Received callback for node 69408872575bc060ce15e711 INFO: 172.21.0.1:41662 - "POST /node/69408872575bc060ce15e711 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:41674 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:15:26,658 - lava_callback - INFO - Received callback for node 69408876575bc060ce15e71b INFO: 172.21.0.1:41680 - "POST /node/69408876575bc060ce15e71b HTTP/1.0" 202 Accepted 2025-12-17 22:15:26,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d0575bc060ce15e1a0 2025-12-17 22:15:26,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d0575bc060ce15e1a0 HTTP/1.1" 502 150 2025-12-17 22:15:26,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d0575bc060ce15e1a0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:27,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d1575bc060ce15e1a3 2025-12-17 22:15:27,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d1575bc060ce15e1a3 HTTP/1.1" 502 150 2025-12-17 22:15:27,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d1575bc060ce15e1a3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:27,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d2575bc060ce15e1a6 2025-12-17 22:15:27,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d2575bc060ce15e1a6 HTTP/1.1" 502 150 2025-12-17 22:15:27,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d2575bc060ce15e1a6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:27,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cd575bc060ce15e19a 2025-12-17 22:15:27,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cd575bc060ce15e19a HTTP/1.1" 502 150 2025-12-17 22:15:27,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cd575bc060ce15e19a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:27,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086b9575bc060ce15e151 2025-12-17 22:15:27,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086b9575bc060ce15e151 HTTP/1.1" 502 150 2025-12-17 22:15:27,905 - lava_callback - ERROR - Error processing callback for node 694086b9575bc060ce15e151: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b9575bc060ce15e151 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b9575bc060ce15e151 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086b9575bc060ce15e151 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:27,906 - lava_callback - INFO - Starting async job submit for node 694086c5575bc060ce15e174 2025-12-17 22:15:27,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:27,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c5575bc060ce15e174 HTTP/1.1" 502 150 2025-12-17 22:15:27,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c5575bc060ce15e174'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:27,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c5575bc060ce15e174 2025-12-17 22:15:27,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c5575bc060ce15e174 HTTP/1.1" 502 150 2025-12-17 22:15:27,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c5575bc060ce15e174'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:28,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cf575bc060ce15e19d 2025-12-17 22:15:28,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cf575bc060ce15e19d HTTP/1.1" 502 150 2025-12-17 22:15:28,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cf575bc060ce15e19d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:29,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086bb575bc060ce15e157 2025-12-17 22:15:29,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bb575bc060ce15e157 HTTP/1.1" 502 150 2025-12-17 22:15:29,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086bb575bc060ce15e157'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:29,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ba575bc060ce15e154 2025-12-17 22:15:29,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ba575bc060ce15e154 HTTP/1.1" 502 150 2025-12-17 22:15:29,838 - lava_callback - ERROR - Error processing callback for node 694086ba575bc060ce15e154: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ba575bc060ce15e154 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ba575bc060ce15e154 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ba575bc060ce15e154 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:29,840 - lava_callback - INFO - Starting async job submit for node 694086c6575bc060ce15e17b 2025-12-17 22:15:29,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:29,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c6575bc060ce15e17b HTTP/1.1" 502 150 2025-12-17 22:15:29,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c6575bc060ce15e17b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:29,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c6575bc060ce15e17b 2025-12-17 22:15:29,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c6575bc060ce15e17b HTTP/1.1" 502 150 2025-12-17 22:15:29,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c6575bc060ce15e17b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:29,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c5575bc060ce15e174 2025-12-17 22:15:29,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c5575bc060ce15e174 HTTP/1.1" 502 150 2025-12-17 22:15:29,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c5575bc060ce15e174'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:30,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408481575bc060ce15d7de HTTP/1.1" 502 150 2025-12-17 22:15:30,227 - lava_callback - ERROR - Error processing callback for node 69408481575bc060ce15d7de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408481575bc060ce15d7de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408481575bc060ce15d7de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408481575bc060ce15d7de (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:30,229 - lava_callback - INFO - Starting async job submit for node 694086d4575bc060ce15e1a9 2025-12-17 22:15:30,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:30,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d4575bc060ce15e1a9 HTTP/1.1" 502 150 2025-12-17 22:15:30,236 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d4575bc060ce15e1a9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:30,236 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d4575bc060ce15e1a9 2025-12-17 22:15:30,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d4575bc060ce15e1a9 HTTP/1.1" 502 150 2025-12-17 22:15:30,237 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d4575bc060ce15e1a9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:30,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ca575bc060ce15e18d 2025-12-17 22:15:30,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ca575bc060ce15e18d HTTP/1.1" 502 150 2025-12-17 22:15:30,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ca575bc060ce15e18d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:30,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cc575bc060ce15e193 2025-12-17 22:15:30,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cc575bc060ce15e193 HTTP/1.1" 502 150 2025-12-17 22:15:30,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cc575bc060ce15e193'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:30,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d0575bc060ce15e1a0 2025-12-17 22:15:31,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d0575bc060ce15e1a0 HTTP/1.1" 502 150 2025-12-17 22:15:31,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d0575bc060ce15e1a0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:31,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d1575bc060ce15e1a3 2025-12-17 22:15:31,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d1575bc060ce15e1a3 HTTP/1.1" 502 150 2025-12-17 22:15:31,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d1575bc060ce15e1a3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:31,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d2575bc060ce15e1a6 2025-12-17 22:15:31,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d2575bc060ce15e1a6 HTTP/1.1" 502 150 2025-12-17 22:15:31,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d2575bc060ce15e1a6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:31,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086bd575bc060ce15e15a 2025-12-17 22:15:31,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bd575bc060ce15e15a HTTP/1.1" 502 150 2025-12-17 22:15:31,804 - lava_callback - ERROR - Error processing callback for node 694086bd575bc060ce15e15a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086bd575bc060ce15e15a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086bd575bc060ce15e15a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086bd575bc060ce15e15a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:31,806 - lava_callback - INFO - Starting async job submit for node 694086d6575bc060ce15e1af 2025-12-17 22:15:31,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:31,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d6575bc060ce15e1af HTTP/1.1" 502 150 2025-12-17 22:15:31,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d6575bc060ce15e1af'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:31,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d6575bc060ce15e1af 2025-12-17 22:15:31,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d6575bc060ce15e1af HTTP/1.1" 502 150 2025-12-17 22:15:31,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d6575bc060ce15e1af'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:31,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c6575bc060ce15e17b 2025-12-17 22:15:31,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c6575bc060ce15e17b HTTP/1.1" 502 150 2025-12-17 22:15:31,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c6575bc060ce15e17b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:31,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086be575bc060ce15e161 2025-12-17 22:15:31,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086be575bc060ce15e161 HTTP/1.1" 502 150 2025-12-17 22:15:31,966 - lava_callback - ERROR - Error processing callback for node 694086be575bc060ce15e161: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086be575bc060ce15e161 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086be575bc060ce15e161 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086be575bc060ce15e161 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:31,968 - lava_callback - INFO - Starting async job submit for node 694086d5575bc060ce15e1ac 2025-12-17 22:15:31,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:31,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d5575bc060ce15e1ac HTTP/1.1" 502 150 2025-12-17 22:15:31,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d5575bc060ce15e1ac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:31,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d5575bc060ce15e1ac 2025-12-17 22:15:31,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d5575bc060ce15e1ac HTTP/1.1" 502 150 2025-12-17 22:15:31,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d5575bc060ce15e1ac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:32,237 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d4575bc060ce15e1a9 2025-12-17 22:15:32,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086bf575bc060ce15e164 2025-12-17 22:15:32,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bf575bc060ce15e164 HTTP/1.1" 502 150 2025-12-17 22:15:32,625 - lava_callback - ERROR - Error processing callback for node 694086bf575bc060ce15e164: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086bf575bc060ce15e164 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086bf575bc060ce15e164 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086bf575bc060ce15e164 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:32,627 - lava_callback - INFO - Starting async job submit for node 694086d8575bc060ce15e1b6 2025-12-17 22:15:32,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:32,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d8575bc060ce15e1b6 HTTP/1.1" 502 150 2025-12-17 22:15:32,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d8575bc060ce15e1b6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:32,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d8575bc060ce15e1b6 2025-12-17 22:15:32,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d8575bc060ce15e1b6 HTTP/1.1" 502 150 2025-12-17 22:15:32,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d8575bc060ce15e1b6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:33,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c0575bc060ce15e167 2025-12-17 22:15:33,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c0575bc060ce15e167 HTTP/1.1" 502 150 2025-12-17 22:15:33,662 - lava_callback - ERROR - Error processing callback for node 694086c0575bc060ce15e167: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086c0575bc060ce15e167 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086c0575bc060ce15e167 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086c0575bc060ce15e167 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:33,664 - lava_callback - INFO - Starting async job submit for node 694086da575bc060ce15e1bf 2025-12-17 22:15:33,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:33,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086da575bc060ce15e1bf HTTP/1.1" 502 150 2025-12-17 22:15:33,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086da575bc060ce15e1bf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:33,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086da575bc060ce15e1bf 2025-12-17 22:15:33,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086da575bc060ce15e1bf HTTP/1.1" 502 150 2025-12-17 22:15:33,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086da575bc060ce15e1bf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:33,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d6575bc060ce15e1af 2025-12-17 22:15:33,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d6575bc060ce15e1af HTTP/1.1" 502 150 2025-12-17 22:15:33,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d6575bc060ce15e1af'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:33,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c5575bc060ce15e174 2025-12-17 22:15:33,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c5575bc060ce15e174 HTTP/1.1" 502 150 2025-12-17 22:15:33,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c5575bc060ce15e174'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:33,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d5575bc060ce15e1ac 2025-12-17 22:15:33,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d5575bc060ce15e1ac HTTP/1.1" 502 150 2025-12-17 22:15:33,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d5575bc060ce15e1ac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:34,413 - lava_callback - INFO - Received callback for node 69408874575bc060ce15e714 INFO: 172.21.0.1:39960 - "POST /node/69408874575bc060ce15e714 HTTP/1.0" 202 Accepted 2025-12-17 22:15:34,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d8575bc060ce15e1b6 2025-12-17 22:15:34,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d8575bc060ce15e1b6 HTTP/1.1" 502 150 2025-12-17 22:15:34,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d8575bc060ce15e1b6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:35,070 - lava_callback - INFO - Received callback for node 69408878575bc060ce15e71e INFO: 172.21.0.1:39972 - "POST /node/69408878575bc060ce15e71e HTTP/1.0" 202 Accepted 2025-12-17 22:15:35,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c1575bc060ce15e16a 2025-12-17 22:15:35,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c1575bc060ce15e16a HTTP/1.1" 502 150 2025-12-17 22:15:35,583 - lava_callback - ERROR - Error processing callback for node 694086c1575bc060ce15e16a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086c1575bc060ce15e16a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086c1575bc060ce15e16a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086c1575bc060ce15e16a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:35,585 - lava_callback - INFO - Starting async job submit for node 694086dc575bc060ce15e1c2 2025-12-17 22:15:35,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:35,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086dc575bc060ce15e1c2 HTTP/1.1" 502 150 2025-12-17 22:15:35,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086dc575bc060ce15e1c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:35,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086dc575bc060ce15e1c2 2025-12-17 22:15:35,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086dc575bc060ce15e1c2 HTTP/1.1" 502 150 2025-12-17 22:15:35,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086dc575bc060ce15e1c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:35,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086da575bc060ce15e1bf 2025-12-17 22:15:35,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086da575bc060ce15e1bf HTTP/1.1" 502 150 2025-12-17 22:15:35,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086da575bc060ce15e1bf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:35,762 - lava_callback - INFO - Received callback for node 6940887a575bc060ce15e721 INFO: 172.21.0.1:39988 - "POST /node/6940887a575bc060ce15e721 HTTP/1.0" 202 Accepted 2025-12-17 22:15:35,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cd575bc060ce15e19a 2025-12-17 22:15:35,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cd575bc060ce15e19a HTTP/1.1" 502 150 2025-12-17 22:15:35,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cd575bc060ce15e19a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:35,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c6575bc060ce15e17b 2025-12-17 22:15:35,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c6575bc060ce15e17b HTTP/1.1" 502 150 2025-12-17 22:15:35,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c6575bc060ce15e17b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:36,430 - lava_callback - INFO - Received callback for node 6940887b575bc060ce15e724 INFO: 172.21.0.1:39996 - "POST /node/6940887b575bc060ce15e724 HTTP/1.0" 202 Accepted 2025-12-17 22:15:36,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cf575bc060ce15e19d 2025-12-17 22:15:36,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cf575bc060ce15e19d HTTP/1.1" 502 150 2025-12-17 22:15:36,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cf575bc060ce15e19d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40000 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:15:37,075 - lava_callback - INFO - Received callback for node 6940887c575bc060ce15e72b INFO: 172.21.0.1:40010 - "POST /node/6940887c575bc060ce15e72b HTTP/1.0" 202 Accepted 2025-12-17 22:15:37,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086dc575bc060ce15e1c2 2025-12-17 22:15:37,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086dc575bc060ce15e1c2 HTTP/1.1" 502 150 2025-12-17 22:15:37,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086dc575bc060ce15e1c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:37,729 - lava_callback - INFO - Received callback for node 6940887e575bc060ce15e72e INFO: 172.21.0.1:40016 - "POST /node/6940887e575bc060ce15e72e HTTP/1.0" 202 Accepted 2025-12-17 22:15:37,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d6575bc060ce15e1af 2025-12-17 22:15:37,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d6575bc060ce15e1af HTTP/1.1" 502 150 2025-12-17 22:15:37,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d6575bc060ce15e1af'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:37,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d5575bc060ce15e1ac 2025-12-17 22:15:37,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d5575bc060ce15e1ac HTTP/1.1" 502 150 2025-12-17 22:15:37,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d5575bc060ce15e1ac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:38,382 - lava_callback - INFO - Received callback for node 69408880575bc060ce15e731 INFO: 172.21.0.1:40026 - "POST /node/69408880575bc060ce15e731 HTTP/1.0" 202 Accepted 2025-12-17 22:15:38,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d8575bc060ce15e1b6 2025-12-17 22:15:38,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d8575bc060ce15e1b6 HTTP/1.1" 502 150 2025-12-17 22:15:38,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d8575bc060ce15e1b6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:39,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d0575bc060ce15e1a0 2025-12-17 22:15:39,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d0575bc060ce15e1a0 HTTP/1.1" 502 150 2025-12-17 22:15:39,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d0575bc060ce15e1a0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:39,045 - lava_callback - INFO - Received callback for node 69408882575bc060ce15e739 INFO: 172.21.0.1:40028 - "POST /node/69408882575bc060ce15e739 HTTP/1.0" 202 Accepted 2025-12-17 22:15:39,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d1575bc060ce15e1a3 2025-12-17 22:15:39,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d1575bc060ce15e1a3 HTTP/1.1" 502 150 2025-12-17 22:15:39,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d1575bc060ce15e1a3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:39,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086da575bc060ce15e1bf 2025-12-17 22:15:39,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086da575bc060ce15e1bf HTTP/1.1" 502 150 2025-12-17 22:15:39,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086da575bc060ce15e1bf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:39,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d2575bc060ce15e1a6 2025-12-17 22:15:39,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d2575bc060ce15e1a6 HTTP/1.1" 502 150 2025-12-17 22:15:39,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d2575bc060ce15e1a6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:39,723 - lava_callback - INFO - Received callback for node 69408884575bc060ce15e73e INFO: 172.21.0.1:43720 - "POST /node/69408884575bc060ce15e73e HTTP/1.0" 202 Accepted 2025-12-17 22:15:41,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086dc575bc060ce15e1c2 2025-12-17 22:15:41,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c5575bc060ce15e174 2025-12-17 22:15:41,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c5575bc060ce15e174 HTTP/1.1" 502 150 2025-12-17 22:15:41,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c5575bc060ce15e174'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:43,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c6575bc060ce15e17b 2025-12-17 22:15:43,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c6575bc060ce15e17b HTTP/1.1" 502 150 2025-12-17 22:15:43,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c6575bc060ce15e17b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:45,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086bb575bc060ce15e157 2025-12-17 22:15:45,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086bb575bc060ce15e157 HTTP/1.1" 502 150 2025-12-17 22:15:45,685 - lava_callback - ERROR - Error processing callback for node 694086bb575bc060ce15e157: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086bb575bc060ce15e157 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086bb575bc060ce15e157 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086bb575bc060ce15e157 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:45,687 - lava_callback - INFO - Starting async job submit for node 694086de575bc060ce15e1c5 2025-12-17 22:15:45,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:45,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086de575bc060ce15e1c5 HTTP/1.1" 502 150 2025-12-17 22:15:45,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086de575bc060ce15e1c5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:45,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086de575bc060ce15e1c5 2025-12-17 22:15:45,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086de575bc060ce15e1c5 HTTP/1.1" 502 150 2025-12-17 22:15:45,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086de575bc060ce15e1c5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:45,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d6575bc060ce15e1af 2025-12-17 22:15:45,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d6575bc060ce15e1af HTTP/1.1" 502 150 2025-12-17 22:15:45,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d6575bc060ce15e1af'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:45,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d5575bc060ce15e1ac 2025-12-17 22:15:45,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d5575bc060ce15e1ac HTTP/1.1" 502 150 2025-12-17 22:15:45,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d5575bc060ce15e1ac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:46,352 - lava_callback - INFO - Received callback for node 6940ce69460bc8e265111f3d INFO: 172.21.0.1:43724 - "POST /node/6940ce69460bc8e265111f3d HTTP/1.0" 202 Accepted INFO: 172.21.0.1:43734 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:15:46,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d8575bc060ce15e1b6 2025-12-17 22:15:46,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d8575bc060ce15e1b6 HTTP/1.1" 502 150 2025-12-17 22:15:46,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d8575bc060ce15e1b6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:46,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ca575bc060ce15e18d 2025-12-17 22:15:46,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ca575bc060ce15e18d HTTP/1.1" 502 150 2025-12-17 22:15:46,789 - lava_callback - ERROR - Error processing callback for node 694086ca575bc060ce15e18d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ca575bc060ce15e18d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ca575bc060ce15e18d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ca575bc060ce15e18d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:46,791 - lava_callback - INFO - Starting async job submit for node 694086e0575bc060ce15e1c8 2025-12-17 22:15:46,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:46,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e0575bc060ce15e1c8 HTTP/1.1" 502 150 2025-12-17 22:15:46,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086e0575bc060ce15e1c8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:46,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086e0575bc060ce15e1c8 2025-12-17 22:15:46,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e0575bc060ce15e1c8 HTTP/1.1" 502 150 2025-12-17 22:15:46,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086e0575bc060ce15e1c8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:46,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cc575bc060ce15e193 2025-12-17 22:15:46,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cc575bc060ce15e193 HTTP/1.1" 502 150 2025-12-17 22:15:46,839 - lava_callback - ERROR - Error processing callback for node 694086cc575bc060ce15e193: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086cc575bc060ce15e193 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086cc575bc060ce15e193 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086cc575bc060ce15e193 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:46,841 - lava_callback - INFO - Starting async job submit for node 694086e2575bc060ce15e1cb 2025-12-17 22:15:46,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:46,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e2575bc060ce15e1cb HTTP/1.1" 502 150 2025-12-17 22:15:46,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086e2575bc060ce15e1cb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:46,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086e2575bc060ce15e1cb 2025-12-17 22:15:46,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e2575bc060ce15e1cb HTTP/1.1" 502 150 2025-12-17 22:15:46,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086e2575bc060ce15e1cb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:47,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d4575bc060ce15e1a9 HTTP/1.1" 502 150 2025-12-17 22:15:47,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d4575bc060ce15e1a9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:47,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086da575bc060ce15e1bf 2025-12-17 22:15:47,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086da575bc060ce15e1bf HTTP/1.1" 502 150 2025-12-17 22:15:47,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086da575bc060ce15e1bf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:47,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086de575bc060ce15e1c5 2025-12-17 22:15:47,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086de575bc060ce15e1c5 HTTP/1.1" 502 150 2025-12-17 22:15:47,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086de575bc060ce15e1c5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:48,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086e0575bc060ce15e1c8 2025-12-17 22:15:48,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e0575bc060ce15e1c8 HTTP/1.1" 502 150 2025-12-17 22:15:48,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086e0575bc060ce15e1c8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:48,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086e2575bc060ce15e1cb 2025-12-17 22:15:48,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e2575bc060ce15e1cb HTTP/1.1" 502 150 2025-12-17 22:15:48,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086e2575bc060ce15e1cb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:51,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d4575bc060ce15e1a9 2025-12-17 22:15:51,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d4575bc060ce15e1a9 HTTP/1.1" 502 150 2025-12-17 22:15:51,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d4575bc060ce15e1a9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:51,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086de575bc060ce15e1c5 2025-12-17 22:15:51,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086de575bc060ce15e1c5 HTTP/1.1" 502 150 2025-12-17 22:15:51,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086de575bc060ce15e1c5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:51,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cd575bc060ce15e19a 2025-12-17 22:15:51,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cd575bc060ce15e19a HTTP/1.1" 502 150 2025-12-17 22:15:51,790 - lava_callback - ERROR - Error processing callback for node 694086cd575bc060ce15e19a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086cd575bc060ce15e19a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086cd575bc060ce15e19a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086cd575bc060ce15e19a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:51,792 - lava_callback - INFO - Starting async job submit for node 694086e6575bc060ce15e1d1 2025-12-17 22:15:51,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:51,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e6575bc060ce15e1d1 HTTP/1.1" 502 150 2025-12-17 22:15:51,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086e6575bc060ce15e1d1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:51,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086e6575bc060ce15e1d1 2025-12-17 22:15:51,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e6575bc060ce15e1d1 HTTP/1.1" 502 150 2025-12-17 22:15:51,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086e6575bc060ce15e1d1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:52,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cf575bc060ce15e19d 2025-12-17 22:15:52,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cf575bc060ce15e19d HTTP/1.1" 502 150 2025-12-17 22:15:52,519 - lava_callback - ERROR - Error processing callback for node 694086cf575bc060ce15e19d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086cf575bc060ce15e19d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086cf575bc060ce15e19d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086cf575bc060ce15e19d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:52,522 - lava_callback - INFO - Starting async job submit for node 694086ea575bc060ce15e1d9 2025-12-17 22:15:52,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:52,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ea575bc060ce15e1d9 HTTP/1.1" 502 150 2025-12-17 22:15:52,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ea575bc060ce15e1d9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:52,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ea575bc060ce15e1d9 2025-12-17 22:15:52,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ea575bc060ce15e1d9 HTTP/1.1" 502 150 2025-12-17 22:15:52,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ea575bc060ce15e1d9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:52,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086e0575bc060ce15e1c8 2025-12-17 22:15:52,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e0575bc060ce15e1c8 HTTP/1.1" 502 150 2025-12-17 22:15:52,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086e0575bc060ce15e1c8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:52,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086e2575bc060ce15e1cb 2025-12-17 22:15:52,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e2575bc060ce15e1cb HTTP/1.1" 502 150 2025-12-17 22:15:52,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086e2575bc060ce15e1cb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:53,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086e6575bc060ce15e1d1 2025-12-17 22:15:53,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e6575bc060ce15e1d1 HTTP/1.1" 502 150 2025-12-17 22:15:53,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086e6575bc060ce15e1d1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:54,223 - lava_callback - INFO - Received callback for node 69408887575bc060ce15e741 INFO: 172.21.0.1:60406 - "POST /node/69408887575bc060ce15e741 HTTP/1.0" 202 Accepted 2025-12-17 22:15:54,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ea575bc060ce15e1d9 2025-12-17 22:15:54,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ea575bc060ce15e1d9 HTTP/1.1" 502 150 2025-12-17 22:15:54,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ea575bc060ce15e1d9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:54,871 - lava_callback - INFO - Received callback for node 6940888d575bc060ce15e744 INFO: 172.21.0.1:60412 - "POST /node/6940888d575bc060ce15e744 HTTP/1.0" 202 Accepted 2025-12-17 22:15:55,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d0575bc060ce15e1a0 2025-12-17 22:15:55,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d0575bc060ce15e1a0 HTTP/1.1" 502 150 2025-12-17 22:15:55,003 - lava_callback - ERROR - Error processing callback for node 694086d0575bc060ce15e1a0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d0575bc060ce15e1a0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d0575bc060ce15e1a0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d0575bc060ce15e1a0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:55,004 - lava_callback - INFO - Starting async job submit for node 694086ec575bc060ce15e1dc 2025-12-17 22:15:55,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:55,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ec575bc060ce15e1dc HTTP/1.1" 502 150 2025-12-17 22:15:55,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ec575bc060ce15e1dc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:55,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ec575bc060ce15e1dc 2025-12-17 22:15:55,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ec575bc060ce15e1dc HTTP/1.1" 502 150 2025-12-17 22:15:55,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ec575bc060ce15e1dc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:55,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d1575bc060ce15e1a3 2025-12-17 22:15:55,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d1575bc060ce15e1a3 HTTP/1.1" 502 150 2025-12-17 22:15:55,537 - lava_callback - ERROR - Error processing callback for node 694086d1575bc060ce15e1a3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d1575bc060ce15e1a3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d1575bc060ce15e1a3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d1575bc060ce15e1a3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:55,539 - lava_callback - INFO - Starting async job submit for node 694086ef575bc060ce15e21b 2025-12-17 22:15:55,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:55,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ef575bc060ce15e21b HTTP/1.1" 502 150 2025-12-17 22:15:55,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ef575bc060ce15e21b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:55,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ef575bc060ce15e21b 2025-12-17 22:15:55,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ef575bc060ce15e21b HTTP/1.1" 502 150 2025-12-17 22:15:55,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ef575bc060ce15e21b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:55,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d2575bc060ce15e1a6 2025-12-17 22:15:55,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d2575bc060ce15e1a6 HTTP/1.1" 502 150 2025-12-17 22:15:55,699 - lava_callback - ERROR - Error processing callback for node 694086d2575bc060ce15e1a6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d2575bc060ce15e1a6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d2575bc060ce15e1a6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d2575bc060ce15e1a6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:55,700 - lava_callback - INFO - Starting async job submit for node 694086f1575bc060ce15e220 2025-12-17 22:15:55,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:55,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f1575bc060ce15e220 HTTP/1.1" 502 150 2025-12-17 22:15:55,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086f1575bc060ce15e220'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:55,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086f1575bc060ce15e220 2025-12-17 22:15:55,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f1575bc060ce15e220 HTTP/1.1" 502 150 2025-12-17 22:15:55,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086f1575bc060ce15e220'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60426 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:15:56,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086dc575bc060ce15e1c2 HTTP/1.1" 502 150 2025-12-17 22:15:56,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086dc575bc060ce15e1c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:57,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ec575bc060ce15e1dc 2025-12-17 22:15:57,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ec575bc060ce15e1dc HTTP/1.1" 502 150 2025-12-17 22:15:57,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ec575bc060ce15e1dc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:57,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ef575bc060ce15e21b 2025-12-17 22:15:57,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ef575bc060ce15e21b HTTP/1.1" 502 150 2025-12-17 22:15:57,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ef575bc060ce15e21b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:57,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086f1575bc060ce15e220 2025-12-17 22:15:57,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f1575bc060ce15e220 HTTP/1.1" 502 150 2025-12-17 22:15:57,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086f1575bc060ce15e220'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:57,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086e6575bc060ce15e1d1 2025-12-17 22:15:57,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e6575bc060ce15e1d1 HTTP/1.1" 502 150 2025-12-17 22:15:57,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086e6575bc060ce15e1d1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:57,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c5575bc060ce15e174 2025-12-17 22:15:57,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c5575bc060ce15e174 HTTP/1.1" 502 150 2025-12-17 22:15:57,923 - lava_callback - ERROR - Error processing callback for node 694086c5575bc060ce15e174: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086c5575bc060ce15e174 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086c5575bc060ce15e174 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086c5575bc060ce15e174 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:57,925 - lava_callback - INFO - Starting async job submit for node 69409080575bc060ce1605b6 2025-12-17 22:15:57,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:58,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ea575bc060ce15e1d9 2025-12-17 22:15:58,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ea575bc060ce15e1d9 HTTP/1.1" 502 150 2025-12-17 22:15:58,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ea575bc060ce15e1d9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:59,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d4575bc060ce15e1a9 2025-12-17 22:15:59,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d4575bc060ce15e1a9 HTTP/1.1" 502 150 2025-12-17 22:15:59,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086d4575bc060ce15e1a9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:59,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086de575bc060ce15e1c5 2025-12-17 22:15:59,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086de575bc060ce15e1c5 HTTP/1.1" 502 150 2025-12-17 22:15:59,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086de575bc060ce15e1c5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:59,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c6575bc060ce15e17b 2025-12-17 22:15:59,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c6575bc060ce15e17b HTTP/1.1" 502 150 2025-12-17 22:15:59,853 - lava_callback - ERROR - Error processing callback for node 694086c6575bc060ce15e17b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086c6575bc060ce15e17b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086c6575bc060ce15e17b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086c6575bc060ce15e17b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:15:59,855 - lava_callback - INFO - Starting async job submit for node 694085cd575bc060ce15ddb0 2025-12-17 22:15:59,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:15:59,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cd575bc060ce15ddb0 HTTP/1.1" 502 150 2025-12-17 22:15:59,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085cd575bc060ce15ddb0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:15:59,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085cd575bc060ce15ddb0 2025-12-17 22:15:59,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cd575bc060ce15ddb0 HTTP/1.1" 502 150 2025-12-17 22:15:59,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085cd575bc060ce15ddb0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:00,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086e0575bc060ce15e1c8 2025-12-17 22:16:00,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e0575bc060ce15e1c8 HTTP/1.1" 502 150 2025-12-17 22:16:00,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086e0575bc060ce15e1c8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:00,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086e2575bc060ce15e1cb 2025-12-17 22:16:00,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e2575bc060ce15e1cb HTTP/1.1" 502 150 2025-12-17 22:16:00,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086e2575bc060ce15e1cb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:01,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ec575bc060ce15e1dc 2025-12-17 22:16:01,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ec575bc060ce15e1dc HTTP/1.1" 502 150 2025-12-17 22:16:01,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ec575bc060ce15e1dc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:01,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ef575bc060ce15e21b 2025-12-17 22:16:01,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ef575bc060ce15e21b HTTP/1.1" 502 150 2025-12-17 22:16:01,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ef575bc060ce15e21b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:01,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086f1575bc060ce15e220 2025-12-17 22:16:01,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f1575bc060ce15e220 HTTP/1.1" 502 150 2025-12-17 22:16:01,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086f1575bc060ce15e220'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:01,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d6575bc060ce15e1af 2025-12-17 22:16:01,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d6575bc060ce15e1af HTTP/1.1" 502 150 2025-12-17 22:16:01,819 - lava_callback - ERROR - Error processing callback for node 694086d6575bc060ce15e1af: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d6575bc060ce15e1af (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d6575bc060ce15e1af (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d6575bc060ce15e1af (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:01,821 - lava_callback - INFO - Starting async job submit for node 694086fb575bc060ce15e22c 2025-12-17 22:16:01,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:01,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fb575bc060ce15e22c HTTP/1.1" 502 150 2025-12-17 22:16:01,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086fb575bc060ce15e22c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:01,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086fb575bc060ce15e22c 2025-12-17 22:16:01,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fb575bc060ce15e22c HTTP/1.1" 502 150 2025-12-17 22:16:01,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086fb575bc060ce15e22c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:01,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d5575bc060ce15e1ac 2025-12-17 22:16:01,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d5575bc060ce15e1ac HTTP/1.1" 502 150 2025-12-17 22:16:01,980 - lava_callback - ERROR - Error processing callback for node 694086d5575bc060ce15e1ac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d5575bc060ce15e1ac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d5575bc060ce15e1ac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d5575bc060ce15e1ac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:01,982 - lava_callback - INFO - Starting async job submit for node 694086fd575bc060ce15e22f 2025-12-17 22:16:01,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:01,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fd575bc060ce15e22f HTTP/1.1" 502 150 2025-12-17 22:16:01,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086fd575bc060ce15e22f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:01,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086fd575bc060ce15e22f 2025-12-17 22:16:01,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fd575bc060ce15e22f HTTP/1.1" 502 150 2025-12-17 22:16:01,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086fd575bc060ce15e22f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:01,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085cd575bc060ce15ddb0 2025-12-17 22:16:01,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cd575bc060ce15ddb0 HTTP/1.1" 502 150 2025-12-17 22:16:01,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085cd575bc060ce15ddb0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41450 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:16:02,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d8575bc060ce15e1b6 2025-12-17 22:16:02,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d8575bc060ce15e1b6 HTTP/1.1" 502 150 2025-12-17 22:16:02,640 - lava_callback - ERROR - Error processing callback for node 694086d8575bc060ce15e1b6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d8575bc060ce15e1b6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d8575bc060ce15e1b6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d8575bc060ce15e1b6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:02,642 - lava_callback - INFO - Starting async job submit for node 69408706575bc060ce15e244 2025-12-17 22:16:02,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:02,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408706575bc060ce15e244 HTTP/1.1" 502 150 2025-12-17 22:16:02,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408706575bc060ce15e244'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:02,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408706575bc060ce15e244 2025-12-17 22:16:02,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408706575bc060ce15e244 HTTP/1.1" 502 150 2025-12-17 22:16:02,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408706575bc060ce15e244'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:03,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086da575bc060ce15e1bf 2025-12-17 22:16:03,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086da575bc060ce15e1bf HTTP/1.1" 502 150 2025-12-17 22:16:03,677 - lava_callback - ERROR - Error processing callback for node 694086da575bc060ce15e1bf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086da575bc060ce15e1bf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086da575bc060ce15e1bf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086da575bc060ce15e1bf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:03,679 - lava_callback - INFO - Starting async job submit for node 69408708575bc060ce15e24a 2025-12-17 22:16:03,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:03,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408708575bc060ce15e24a HTTP/1.1" 502 150 2025-12-17 22:16:03,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408708575bc060ce15e24a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:03,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408708575bc060ce15e24a 2025-12-17 22:16:03,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408708575bc060ce15e24a HTTP/1.1" 502 150 2025-12-17 22:16:03,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408708575bc060ce15e24a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:03,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086fb575bc060ce15e22c 2025-12-17 22:16:03,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fb575bc060ce15e22c HTTP/1.1" 502 150 2025-12-17 22:16:03,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086fb575bc060ce15e22c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:03,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086fd575bc060ce15e22f 2025-12-17 22:16:03,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fd575bc060ce15e22f HTTP/1.1" 502 150 2025-12-17 22:16:03,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086fd575bc060ce15e22f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:04,150 - lava_callback - INFO - Received callback for node 6940888e575bc060ce15e749 INFO: 172.21.0.1:41452 - "POST /node/6940888e575bc060ce15e749 HTTP/1.0" 202 Accepted 2025-12-17 22:16:04,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408706575bc060ce15e244 2025-12-17 22:16:04,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408706575bc060ce15e244 HTTP/1.1" 502 150 2025-12-17 22:16:04,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408706575bc060ce15e244'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:04,793 - lava_callback - INFO - Received callback for node 69408891575bc060ce15e750 INFO: 172.21.0.1:41456 - "POST /node/69408891575bc060ce15e750 HTTP/1.0" 202 Accepted 2025-12-17 22:16:04,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086dc575bc060ce15e1c2 2025-12-17 22:16:04,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086dc575bc060ce15e1c2 HTTP/1.1" 502 150 2025-12-17 22:16:04,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086dc575bc060ce15e1c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:05,433 - lava_callback - INFO - Received callback for node 69408893575bc060ce15e753 INFO: 172.21.0.1:41464 - "POST /node/69408893575bc060ce15e753 HTTP/1.0" 202 Accepted 2025-12-17 22:16:05,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408708575bc060ce15e24a 2025-12-17 22:16:05,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408708575bc060ce15e24a HTTP/1.1" 502 150 2025-12-17 22:16:05,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408708575bc060ce15e24a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:05,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086e6575bc060ce15e1d1 2025-12-17 22:16:05,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e6575bc060ce15e1d1 HTTP/1.1" 502 150 2025-12-17 22:16:05,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086e6575bc060ce15e1d1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:05,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085cd575bc060ce15ddb0 2025-12-17 22:16:05,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cd575bc060ce15ddb0 HTTP/1.1" 502 150 2025-12-17 22:16:05,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085cd575bc060ce15ddb0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:06,075 - lava_callback - INFO - Received callback for node 69408895575bc060ce15e756 INFO: 172.21.0.1:41470 - "POST /node/69408895575bc060ce15e756 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:41480 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:16:06,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ea575bc060ce15e1d9 2025-12-17 22:16:06,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ea575bc060ce15e1d9 HTTP/1.1" 502 150 2025-12-17 22:16:06,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ea575bc060ce15e1d9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:06,708 - lava_callback - INFO - Received callback for node 69408897575bc060ce15e759 INFO: 172.21.0.1:41482 - "POST /node/69408897575bc060ce15e759 HTTP/1.0" 202 Accepted 2025-12-17 22:16:07,345 - lava_callback - INFO - Received callback for node 69408899575bc060ce15e75c INFO: 172.21.0.1:41488 - "POST /node/69408899575bc060ce15e75c HTTP/1.0" 202 Accepted 2025-12-17 22:16:07,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086fb575bc060ce15e22c 2025-12-17 22:16:07,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086fd575bc060ce15e22f 2025-12-17 22:16:07,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fd575bc060ce15e22f HTTP/1.1" 502 150 2025-12-17 22:16:07,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086fd575bc060ce15e22f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:07,993 - lava_callback - INFO - Received callback for node 6940889b575bc060ce15e75f INFO: 172.21.0.1:41494 - "POST /node/6940889b575bc060ce15e75f HTTP/1.0" 202 Accepted 2025-12-17 22:16:08,629 - lava_callback - INFO - Received callback for node 6940889c575bc060ce15e762 INFO: 172.21.0.1:41500 - "POST /node/6940889c575bc060ce15e762 HTTP/1.0" 202 Accepted 2025-12-17 22:16:08,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408706575bc060ce15e244 2025-12-17 22:16:08,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408706575bc060ce15e244 HTTP/1.1" 502 150 2025-12-17 22:16:08,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408706575bc060ce15e244'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:09,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ec575bc060ce15e1dc 2025-12-17 22:16:09,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ec575bc060ce15e1dc HTTP/1.1" 502 150 2025-12-17 22:16:09,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ec575bc060ce15e1dc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:09,280 - lava_callback - INFO - Received callback for node 6940889e575bc060ce15e765 INFO: 172.21.0.1:41514 - "POST /node/6940889e575bc060ce15e765 HTTP/1.0" 202 Accepted 2025-12-17 22:16:09,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ef575bc060ce15e21b 2025-12-17 22:16:09,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ef575bc060ce15e21b HTTP/1.1" 502 150 2025-12-17 22:16:09,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ef575bc060ce15e21b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:09,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408708575bc060ce15e24a 2025-12-17 22:16:09,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408708575bc060ce15e24a HTTP/1.1" 502 150 2025-12-17 22:16:09,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408708575bc060ce15e24a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:09,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086f1575bc060ce15e220 2025-12-17 22:16:09,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f1575bc060ce15e220 HTTP/1.1" 502 150 2025-12-17 22:16:09,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086f1575bc060ce15e220'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:13,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409080575bc060ce1605b6 HTTP/1.1" 502 150 2025-12-17 22:16:13,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409080575bc060ce1605b6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:13,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409080575bc060ce1605b6 2025-12-17 22:16:13,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409080575bc060ce1605b6 HTTP/1.1" 502 150 2025-12-17 22:16:13,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409080575bc060ce1605b6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:13,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085cd575bc060ce15ddb0 2025-12-17 22:16:14,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cd575bc060ce15ddb0 HTTP/1.1" 502 150 2025-12-17 22:16:14,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694085cd575bc060ce15ddb0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:14,196 - lava_callback - INFO - Received callback for node 694088a5575bc060ce15e7bb INFO: 172.21.0.1:56264 - "POST /node/694088a5575bc060ce15e7bb HTTP/1.0" 202 Accepted 2025-12-17 22:16:14,849 - lava_callback - INFO - Received callback for node 694088a8575bc060ce15e7c2 INFO: 172.21.0.1:56276 - "POST /node/694088a8575bc060ce15e7c2 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:56292 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:16:15,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409080575bc060ce1605b6 2025-12-17 22:16:15,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409080575bc060ce1605b6 HTTP/1.1" 502 150 2025-12-17 22:16:15,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409080575bc060ce1605b6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:15,517 - lava_callback - INFO - Received callback for node 694088aa575bc060ce15e7c5 INFO: 172.21.0.1:56294 - "POST /node/694088aa575bc060ce15e7c5 HTTP/1.0" 202 Accepted 2025-12-17 22:16:15,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086d4575bc060ce15e1a9 2025-12-17 22:16:15,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086d4575bc060ce15e1a9 HTTP/1.1" 502 150 2025-12-17 22:16:15,637 - lava_callback - ERROR - Error processing callback for node 694086d4575bc060ce15e1a9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d4575bc060ce15e1a9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d4575bc060ce15e1a9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086d4575bc060ce15e1a9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:15,639 - lava_callback - INFO - Starting async job submit for node 6940870b575bc060ce15e250 2025-12-17 22:16:15,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:15,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870b575bc060ce15e250 HTTP/1.1" 502 150 2025-12-17 22:16:15,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870b575bc060ce15e250'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:15,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870b575bc060ce15e250 2025-12-17 22:16:15,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870b575bc060ce15e250 HTTP/1.1" 502 150 2025-12-17 22:16:15,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870b575bc060ce15e250'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:15,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086de575bc060ce15e1c5 2025-12-17 22:16:15,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086de575bc060ce15e1c5 HTTP/1.1" 502 150 2025-12-17 22:16:15,718 - lava_callback - ERROR - Error processing callback for node 694086de575bc060ce15e1c5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086de575bc060ce15e1c5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086de575bc060ce15e1c5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086de575bc060ce15e1c5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:15,719 - lava_callback - INFO - Starting async job submit for node 6940870d575bc060ce15e253 2025-12-17 22:16:15,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:15,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870d575bc060ce15e253 HTTP/1.1" 502 150 2025-12-17 22:16:15,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870d575bc060ce15e253'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:15,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870d575bc060ce15e253 2025-12-17 22:16:15,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870d575bc060ce15e253 HTTP/1.1" 502 150 2025-12-17 22:16:15,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870d575bc060ce15e253'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:15,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086fd575bc060ce15e22f 2025-12-17 22:16:15,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fd575bc060ce15e22f HTTP/1.1" 502 150 2025-12-17 22:16:15,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086fd575bc060ce15e22f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:16,191 - lava_callback - INFO - Received callback for node 694088ac575bc060ce15e7c9 INFO: 172.21.0.1:56302 - "POST /node/694088ac575bc060ce15e7c9 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:56312 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:16:16,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408706575bc060ce15e244 2025-12-17 22:16:16,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408706575bc060ce15e244 HTTP/1.1" 502 150 2025-12-17 22:16:16,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408706575bc060ce15e244'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:16,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086e0575bc060ce15e1c8 2025-12-17 22:16:16,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e0575bc060ce15e1c8 HTTP/1.1" 502 150 2025-12-17 22:16:16,810 - lava_callback - ERROR - Error processing callback for node 694086e0575bc060ce15e1c8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086e0575bc060ce15e1c8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086e0575bc060ce15e1c8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086e0575bc060ce15e1c8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:16,812 - lava_callback - INFO - Starting async job submit for node 694086ff575bc060ce15e238 2025-12-17 22:16:16,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:16,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ff575bc060ce15e238 HTTP/1.1" 502 150 2025-12-17 22:16:16,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ff575bc060ce15e238'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:16,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ff575bc060ce15e238 2025-12-17 22:16:16,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ff575bc060ce15e238 HTTP/1.1" 502 150 2025-12-17 22:16:16,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ff575bc060ce15e238'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:16,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086e2575bc060ce15e1cb 2025-12-17 22:16:16,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e2575bc060ce15e1cb HTTP/1.1" 502 150 2025-12-17 22:16:16,852 - lava_callback - ERROR - Error processing callback for node 694086e2575bc060ce15e1cb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086e2575bc060ce15e1cb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086e2575bc060ce15e1cb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086e2575bc060ce15e1cb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:16,854 - lava_callback - INFO - Starting async job submit for node 69408701575bc060ce15e23b 2025-12-17 22:16:16,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:16,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408701575bc060ce15e23b HTTP/1.1" 502 150 2025-12-17 22:16:16,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408701575bc060ce15e23b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:16,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408701575bc060ce15e23b 2025-12-17 22:16:16,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408701575bc060ce15e23b HTTP/1.1" 502 150 2025-12-17 22:16:16,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408701575bc060ce15e23b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:17,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870b575bc060ce15e250 2025-12-17 22:16:17,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870b575bc060ce15e250 HTTP/1.1" 502 150 2025-12-17 22:16:17,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870b575bc060ce15e250'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:17,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408708575bc060ce15e24a 2025-12-17 22:16:17,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408708575bc060ce15e24a HTTP/1.1" 502 150 2025-12-17 22:16:17,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408708575bc060ce15e24a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:17,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870d575bc060ce15e253 2025-12-17 22:16:17,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870d575bc060ce15e253 HTTP/1.1" 502 150 2025-12-17 22:16:17,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870d575bc060ce15e253'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:18,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ff575bc060ce15e238 2025-12-17 22:16:18,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ff575bc060ce15e238 HTTP/1.1" 502 150 2025-12-17 22:16:18,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ff575bc060ce15e238'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:18,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408701575bc060ce15e23b 2025-12-17 22:16:18,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408701575bc060ce15e23b HTTP/1.1" 502 150 2025-12-17 22:16:18,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408701575bc060ce15e23b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:19,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409080575bc060ce1605b6 2025-12-17 22:16:19,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409080575bc060ce1605b6 HTTP/1.1" 502 150 2025-12-17 22:16:19,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409080575bc060ce1605b6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:20,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086dc575bc060ce15e1c2 2025-12-17 22:16:20,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086dc575bc060ce15e1c2 HTTP/1.1" 502 150 2025-12-17 22:16:20,851 - lava_callback - ERROR - Error processing callback for node 694086dc575bc060ce15e1c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086dc575bc060ce15e1c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086dc575bc060ce15e1c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086dc575bc060ce15e1c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:20,853 - lava_callback - INFO - Starting async job submit for node 69408703575bc060ce15e23e 2025-12-17 22:16:20,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:20,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408703575bc060ce15e23e HTTP/1.1" 502 150 2025-12-17 22:16:20,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408703575bc060ce15e23e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:20,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408703575bc060ce15e23e 2025-12-17 22:16:20,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408703575bc060ce15e23e HTTP/1.1" 502 150 2025-12-17 22:16:20,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408703575bc060ce15e23e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:21,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870b575bc060ce15e250 2025-12-17 22:16:21,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870b575bc060ce15e250 HTTP/1.1" 502 150 2025-12-17 22:16:21,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870b575bc060ce15e250'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:21,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870d575bc060ce15e253 2025-12-17 22:16:21,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870d575bc060ce15e253 HTTP/1.1" 502 150 2025-12-17 22:16:21,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870d575bc060ce15e253'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:21,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086e6575bc060ce15e1d1 2025-12-17 22:16:21,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086e6575bc060ce15e1d1 HTTP/1.1" 502 150 2025-12-17 22:16:21,805 - lava_callback - ERROR - Error processing callback for node 694086e6575bc060ce15e1d1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086e6575bc060ce15e1d1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086e6575bc060ce15e1d1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086e6575bc060ce15e1d1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:21,807 - lava_callback - INFO - Starting async job submit for node 69408704575bc060ce15e241 2025-12-17 22:16:21,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:21,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408704575bc060ce15e241 HTTP/1.1" 502 150 2025-12-17 22:16:21,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408704575bc060ce15e241'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:21,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408704575bc060ce15e241 2025-12-17 22:16:21,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408704575bc060ce15e241 HTTP/1.1" 502 150 2025-12-17 22:16:21,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408704575bc060ce15e241'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:22,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ea575bc060ce15e1d9 2025-12-17 22:16:22,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ea575bc060ce15e1d9 HTTP/1.1" 502 150 2025-12-17 22:16:22,542 - lava_callback - ERROR - Error processing callback for node 694086ea575bc060ce15e1d9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ea575bc060ce15e1d9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ea575bc060ce15e1d9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ea575bc060ce15e1d9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:22,544 - lava_callback - INFO - Starting async job submit for node 69408707575bc060ce15e247 2025-12-17 22:16:22,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:22,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408707575bc060ce15e247 HTTP/1.1" 502 150 2025-12-17 22:16:22,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408707575bc060ce15e247'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:22,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408707575bc060ce15e247 2025-12-17 22:16:22,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408707575bc060ce15e247 HTTP/1.1" 502 150 2025-12-17 22:16:22,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408707575bc060ce15e247'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:22,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ff575bc060ce15e238 2025-12-17 22:16:22,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ff575bc060ce15e238 HTTP/1.1" 502 150 2025-12-17 22:16:22,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ff575bc060ce15e238'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:22,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408701575bc060ce15e23b 2025-12-17 22:16:22,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408701575bc060ce15e23b HTTP/1.1" 502 150 2025-12-17 22:16:22,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408701575bc060ce15e23b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:22,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408703575bc060ce15e23e 2025-12-17 22:16:22,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408703575bc060ce15e23e HTTP/1.1" 502 150 2025-12-17 22:16:22,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408703575bc060ce15e23e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:23,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fb575bc060ce15e22c HTTP/1.1" 502 150 2025-12-17 22:16:23,220 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086fb575bc060ce15e22c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:23,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408704575bc060ce15e241 2025-12-17 22:16:23,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408704575bc060ce15e241 HTTP/1.1" 502 150 2025-12-17 22:16:23,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408704575bc060ce15e241'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:24,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408707575bc060ce15e247 2025-12-17 22:16:25,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ec575bc060ce15e1dc 2025-12-17 22:16:25,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ec575bc060ce15e1dc HTTP/1.1" 502 150 2025-12-17 22:16:25,020 - lava_callback - ERROR - Error processing callback for node 694086ec575bc060ce15e1dc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ec575bc060ce15e1dc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ec575bc060ce15e1dc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ec575bc060ce15e1dc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:25,023 - lava_callback - INFO - Starting async job submit for node 6940870a575bc060ce15e24d 2025-12-17 22:16:25,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:25,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870a575bc060ce15e24d HTTP/1.1" 502 150 2025-12-17 22:16:25,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870a575bc060ce15e24d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:25,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870a575bc060ce15e24d 2025-12-17 22:16:25,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870a575bc060ce15e24d HTTP/1.1" 502 150 2025-12-17 22:16:25,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870a575bc060ce15e24d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:25,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ef575bc060ce15e21b 2025-12-17 22:16:25,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ef575bc060ce15e21b HTTP/1.1" 502 150 2025-12-17 22:16:25,553 - lava_callback - ERROR - Error processing callback for node 694086ef575bc060ce15e21b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ef575bc060ce15e21b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ef575bc060ce15e21b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ef575bc060ce15e21b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:25,555 - lava_callback - INFO - Starting async job submit for node 6940870e575bc060ce15e256 2025-12-17 22:16:25,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:25,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870e575bc060ce15e256 HTTP/1.1" 502 150 2025-12-17 22:16:25,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870e575bc060ce15e256'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:25,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870e575bc060ce15e256 2025-12-17 22:16:25,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870e575bc060ce15e256 HTTP/1.1" 502 150 2025-12-17 22:16:25,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870e575bc060ce15e256'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:25,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086f1575bc060ce15e220 2025-12-17 22:16:25,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086f1575bc060ce15e220 HTTP/1.1" 502 150 2025-12-17 22:16:25,713 - lava_callback - ERROR - Error processing callback for node 694086f1575bc060ce15e220: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086f1575bc060ce15e220 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086f1575bc060ce15e220 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086f1575bc060ce15e220 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:25,715 - lava_callback - INFO - Starting async job submit for node 6940870f575bc060ce15e259 2025-12-17 22:16:25,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:25,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870f575bc060ce15e259 HTTP/1.1" 502 150 2025-12-17 22:16:25,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870f575bc060ce15e259'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:25,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870f575bc060ce15e259 2025-12-17 22:16:25,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870f575bc060ce15e259 HTTP/1.1" 502 150 2025-12-17 22:16:25,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870f575bc060ce15e259'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33744 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:16:26,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408703575bc060ce15e23e 2025-12-17 22:16:26,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408703575bc060ce15e23e HTTP/1.1" 502 150 2025-12-17 22:16:26,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408703575bc060ce15e23e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:27,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870a575bc060ce15e24d 2025-12-17 22:16:27,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870a575bc060ce15e24d HTTP/1.1" 502 150 2025-12-17 22:16:27,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870a575bc060ce15e24d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:27,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409080575bc060ce1605b6 2025-12-17 22:16:27,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409080575bc060ce1605b6 HTTP/1.1" 502 150 2025-12-17 22:16:27,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409080575bc060ce1605b6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:27,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870e575bc060ce15e256 2025-12-17 22:16:27,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870e575bc060ce15e256 HTTP/1.1" 502 150 2025-12-17 22:16:27,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870e575bc060ce15e256'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:27,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870f575bc060ce15e259 2025-12-17 22:16:27,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870f575bc060ce15e259 HTTP/1.1" 502 150 2025-12-17 22:16:27,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870f575bc060ce15e259'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:27,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408704575bc060ce15e241 2025-12-17 22:16:27,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408704575bc060ce15e241 HTTP/1.1" 502 150 2025-12-17 22:16:27,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408704575bc060ce15e241'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:29,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870b575bc060ce15e250 2025-12-17 22:16:29,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870b575bc060ce15e250 HTTP/1.1" 502 150 2025-12-17 22:16:29,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870b575bc060ce15e250'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:29,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870d575bc060ce15e253 2025-12-17 22:16:29,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870d575bc060ce15e253 HTTP/1.1" 502 150 2025-12-17 22:16:29,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870d575bc060ce15e253'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:30,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694085cd575bc060ce15ddb0 2025-12-17 22:16:30,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694085cd575bc060ce15ddb0 HTTP/1.1" 502 150 2025-12-17 22:16:30,004 - lava_callback - ERROR - Error processing callback for node 694085cd575bc060ce15ddb0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085cd575bc060ce15ddb0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085cd575bc060ce15ddb0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694085cd575bc060ce15ddb0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:30,005 - lava_callback - INFO - Starting async job submit for node 69408710575bc060ce15e25c 2025-12-17 22:16:30,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:30,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408710575bc060ce15e25c HTTP/1.1" 502 150 2025-12-17 22:16:30,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408710575bc060ce15e25c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:30,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408710575bc060ce15e25c 2025-12-17 22:16:30,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408710575bc060ce15e25c HTTP/1.1" 502 150 2025-12-17 22:16:30,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408710575bc060ce15e25c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:30,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ff575bc060ce15e238 2025-12-17 22:16:30,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ff575bc060ce15e238 HTTP/1.1" 502 150 2025-12-17 22:16:30,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086ff575bc060ce15e238'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:30,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408701575bc060ce15e23b 2025-12-17 22:16:30,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408701575bc060ce15e23b HTTP/1.1" 502 150 2025-12-17 22:16:30,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408701575bc060ce15e23b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:31,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870a575bc060ce15e24d 2025-12-17 22:16:31,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870a575bc060ce15e24d HTTP/1.1" 502 150 2025-12-17 22:16:31,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870a575bc060ce15e24d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:31,221 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086fb575bc060ce15e22c 2025-12-17 22:16:31,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fb575bc060ce15e22c HTTP/1.1" 502 150 2025-12-17 22:16:31,222 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086fb575bc060ce15e22c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:31,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870e575bc060ce15e256 2025-12-17 22:16:31,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870e575bc060ce15e256 HTTP/1.1" 502 150 2025-12-17 22:16:31,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870e575bc060ce15e256'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:31,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870f575bc060ce15e259 2025-12-17 22:16:31,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870f575bc060ce15e259 HTTP/1.1" 502 150 2025-12-17 22:16:31,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870f575bc060ce15e259'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:31,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086fd575bc060ce15e22f 2025-12-17 22:16:31,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fd575bc060ce15e22f HTTP/1.1" 502 150 2025-12-17 22:16:31,995 - lava_callback - ERROR - Error processing callback for node 694086fd575bc060ce15e22f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086fd575bc060ce15e22f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086fd575bc060ce15e22f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086fd575bc060ce15e22f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:31,998 - lava_callback - INFO - Starting async job submit for node 69408712575bc060ce15e25f 2025-12-17 22:16:31,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:32,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408712575bc060ce15e25f HTTP/1.1" 502 150 2025-12-17 22:16:32,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408712575bc060ce15e25f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:32,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408712575bc060ce15e25f 2025-12-17 22:16:32,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408712575bc060ce15e25f HTTP/1.1" 502 150 2025-12-17 22:16:32,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408712575bc060ce15e25f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:32,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408710575bc060ce15e25c 2025-12-17 22:16:32,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408710575bc060ce15e25c HTTP/1.1" 502 150 2025-12-17 22:16:32,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408710575bc060ce15e25c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:32,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408706575bc060ce15e244 2025-12-17 22:16:32,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408706575bc060ce15e244 HTTP/1.1" 502 150 2025-12-17 22:16:32,656 - lava_callback - ERROR - Error processing callback for node 69408706575bc060ce15e244: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408706575bc060ce15e244 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408706575bc060ce15e244 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408706575bc060ce15e244 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:32,658 - lava_callback - INFO - Starting async job submit for node 6940871a575bc060ce15e27e 2025-12-17 22:16:32,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:32,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871a575bc060ce15e27e HTTP/1.1" 502 150 2025-12-17 22:16:32,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871a575bc060ce15e27e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:32,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871a575bc060ce15e27e 2025-12-17 22:16:32,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871a575bc060ce15e27e HTTP/1.1" 502 150 2025-12-17 22:16:32,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871a575bc060ce15e27e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:33,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408708575bc060ce15e24a 2025-12-17 22:16:33,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408708575bc060ce15e24a HTTP/1.1" 502 150 2025-12-17 22:16:33,693 - lava_callback - ERROR - Error processing callback for node 69408708575bc060ce15e24a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408708575bc060ce15e24a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408708575bc060ce15e24a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408708575bc060ce15e24a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:33,695 - lava_callback - INFO - Starting async job submit for node 6940871e575bc060ce15e288 2025-12-17 22:16:33,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:33,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871e575bc060ce15e288 HTTP/1.1" 502 150 2025-12-17 22:16:33,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871e575bc060ce15e288'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:33,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871e575bc060ce15e288 2025-12-17 22:16:33,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871e575bc060ce15e288 HTTP/1.1" 502 150 2025-12-17 22:16:33,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871e575bc060ce15e288'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:34,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408712575bc060ce15e25f 2025-12-17 22:16:34,260 - lava_callback - INFO - Received callback for node 694088b2575bc060ce15e7cc INFO: 172.21.0.1:58324 - "POST /node/694088b2575bc060ce15e7cc HTTP/1.0" 202 Accepted 2025-12-17 22:16:34,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871a575bc060ce15e27e 2025-12-17 22:16:34,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871a575bc060ce15e27e HTTP/1.1" 502 150 2025-12-17 22:16:34,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871a575bc060ce15e27e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:34,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408703575bc060ce15e23e 2025-12-17 22:16:34,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408703575bc060ce15e23e HTTP/1.1" 502 150 2025-12-17 22:16:34,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408703575bc060ce15e23e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:34,911 - lava_callback - INFO - Received callback for node 694088b4575bc060ce15e7d3 INFO: 172.21.0.1:58328 - "POST /node/694088b4575bc060ce15e7d3 HTTP/1.0" 202 Accepted 2025-12-17 22:16:35,574 - lava_callback - INFO - Received callback for node 694088b6575bc060ce15e7d6 INFO: 172.21.0.1:58334 - "POST /node/694088b6575bc060ce15e7d6 HTTP/1.0" 202 Accepted 2025-12-17 22:16:35,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871e575bc060ce15e288 2025-12-17 22:16:35,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871e575bc060ce15e288 HTTP/1.1" 502 150 2025-12-17 22:16:35,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871e575bc060ce15e288'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:35,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408704575bc060ce15e241 2025-12-17 22:16:35,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408704575bc060ce15e241 HTTP/1.1" 502 150 2025-12-17 22:16:35,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408704575bc060ce15e241'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:36,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408710575bc060ce15e25c 2025-12-17 22:16:36,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408710575bc060ce15e25c HTTP/1.1" 502 150 2025-12-17 22:16:36,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408710575bc060ce15e25c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:36,226 - lava_callback - INFO - Received callback for node 694088b8575bc060ce15e7df INFO: 172.21.0.1:58350 - "POST /node/694088b8575bc060ce15e7df HTTP/1.0" 202 Accepted INFO: 172.21.0.1:58352 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:16:36,870 - lava_callback - INFO - Received callback for node 694088ba575bc060ce15e7e8 INFO: 172.21.0.1:58354 - "POST /node/694088ba575bc060ce15e7e8 HTTP/1.0" 202 Accepted 2025-12-17 22:16:37,522 - lava_callback - INFO - Received callback for node 694088bb575bc060ce15e7eb INFO: 172.21.0.1:58368 - "POST /node/694088bb575bc060ce15e7eb HTTP/1.0" 202 Accepted 2025-12-17 22:16:38,180 - lava_callback - INFO - Received callback for node 694088bc575bc060ce15e7ee INFO: 172.21.0.1:58384 - "POST /node/694088bc575bc060ce15e7ee HTTP/1.0" 202 Accepted 2025-12-17 22:16:38,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871a575bc060ce15e27e 2025-12-17 22:16:38,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871a575bc060ce15e27e HTTP/1.1" 502 150 2025-12-17 22:16:38,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871a575bc060ce15e27e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:38,853 - lava_callback - INFO - Received callback for node 694088be575bc060ce15e7f5 INFO: 172.21.0.1:58396 - "POST /node/694088be575bc060ce15e7f5 HTTP/1.0" 202 Accepted 2025-12-17 22:16:39,042 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870a575bc060ce15e24d 2025-12-17 22:16:39,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870a575bc060ce15e24d HTTP/1.1" 502 150 2025-12-17 22:16:39,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870a575bc060ce15e24d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:39,506 - lava_callback - INFO - Received callback for node 694088bf575bc060ce15e7f8 INFO: 172.21.0.1:58404 - "POST /node/694088bf575bc060ce15e7f8 HTTP/1.0" 202 Accepted 2025-12-17 22:16:39,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870e575bc060ce15e256 2025-12-17 22:16:39,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870e575bc060ce15e256 HTTP/1.1" 502 150 2025-12-17 22:16:39,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870e575bc060ce15e256'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:39,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871e575bc060ce15e288 2025-12-17 22:16:39,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871e575bc060ce15e288 HTTP/1.1" 502 150 2025-12-17 22:16:39,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871e575bc060ce15e288'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:39,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870f575bc060ce15e259 2025-12-17 22:16:39,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870f575bc060ce15e259 HTTP/1.1" 502 150 2025-12-17 22:16:39,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940870f575bc060ce15e259'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:39,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408707575bc060ce15e247 HTTP/1.1" 502 150 2025-12-17 22:16:39,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408707575bc060ce15e247'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:40,180 - lava_callback - INFO - Received callback for node 694088c0575bc060ce15e7fb INFO: 172.21.0.1:44920 - "POST /node/694088c0575bc060ce15e7fb HTTP/1.0" 202 Accepted 2025-12-17 22:16:43,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409080575bc060ce1605b6 2025-12-17 22:16:43,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409080575bc060ce1605b6 HTTP/1.1" 502 150 2025-12-17 22:16:43,495 - lava_callback - ERROR - Error processing callback for node 69409080575bc060ce1605b6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409080575bc060ce1605b6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409080575bc060ce1605b6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409080575bc060ce1605b6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:43,497 - lava_callback - INFO - Starting async job submit for node 69408721575bc060ce15e28e 2025-12-17 22:16:43,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:43,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408721575bc060ce15e28e HTTP/1.1" 502 150 2025-12-17 22:16:43,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408721575bc060ce15e28e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:43,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408721575bc060ce15e28e 2025-12-17 22:16:43,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408721575bc060ce15e28e HTTP/1.1" 502 150 2025-12-17 22:16:43,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408721575bc060ce15e28e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:43,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408707575bc060ce15e247 2025-12-17 22:16:43,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408707575bc060ce15e247 HTTP/1.1" 502 150 2025-12-17 22:16:43,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408707575bc060ce15e247'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:44,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408710575bc060ce15e25c 2025-12-17 22:16:44,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408710575bc060ce15e25c HTTP/1.1" 502 150 2025-12-17 22:16:44,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408710575bc060ce15e25c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:45,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408721575bc060ce15e28e 2025-12-17 22:16:45,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408721575bc060ce15e28e HTTP/1.1" 502 150 2025-12-17 22:16:45,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408721575bc060ce15e28e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:45,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870b575bc060ce15e250 2025-12-17 22:16:45,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870b575bc060ce15e250 HTTP/1.1" 502 150 2025-12-17 22:16:45,656 - lava_callback - ERROR - Error processing callback for node 6940870b575bc060ce15e250: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940870b575bc060ce15e250 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940870b575bc060ce15e250 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940870b575bc060ce15e250 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:45,658 - lava_callback - INFO - Starting async job submit for node 69408722575bc060ce15e291 2025-12-17 22:16:45,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:45,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408722575bc060ce15e291 HTTP/1.1" 502 150 2025-12-17 22:16:45,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408722575bc060ce15e291'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:45,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408722575bc060ce15e291 2025-12-17 22:16:45,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408722575bc060ce15e291 HTTP/1.1" 502 150 2025-12-17 22:16:45,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408722575bc060ce15e291'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:45,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870d575bc060ce15e253 2025-12-17 22:16:45,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870d575bc060ce15e253 HTTP/1.1" 502 150 2025-12-17 22:16:45,732 - lava_callback - ERROR - Error processing callback for node 6940870d575bc060ce15e253: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940870d575bc060ce15e253 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940870d575bc060ce15e253 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940870d575bc060ce15e253 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:45,734 - lava_callback - INFO - Starting async job submit for node 69408723575bc060ce15e294 2025-12-17 22:16:45,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:45,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408723575bc060ce15e294 HTTP/1.1" 502 150 2025-12-17 22:16:45,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408723575bc060ce15e294'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:45,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408723575bc060ce15e294 2025-12-17 22:16:45,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408723575bc060ce15e294 HTTP/1.1" 502 150 2025-12-17 22:16:45,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408723575bc060ce15e294'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44932 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:16:46,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871a575bc060ce15e27e 2025-12-17 22:16:46,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871a575bc060ce15e27e HTTP/1.1" 502 150 2025-12-17 22:16:46,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871a575bc060ce15e27e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:46,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086ff575bc060ce15e238 2025-12-17 22:16:46,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086ff575bc060ce15e238 HTTP/1.1" 502 150 2025-12-17 22:16:46,833 - lava_callback - ERROR - Error processing callback for node 694086ff575bc060ce15e238: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ff575bc060ce15e238 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ff575bc060ce15e238 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086ff575bc060ce15e238 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:46,835 - lava_callback - INFO - Starting async job submit for node 69408725575bc060ce15e297 2025-12-17 22:16:46,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:46,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408725575bc060ce15e297 HTTP/1.1" 502 150 2025-12-17 22:16:46,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408725575bc060ce15e297'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:46,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408725575bc060ce15e297 2025-12-17 22:16:46,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408725575bc060ce15e297 HTTP/1.1" 502 150 2025-12-17 22:16:46,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408725575bc060ce15e297'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:46,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408701575bc060ce15e23b 2025-12-17 22:16:46,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408701575bc060ce15e23b HTTP/1.1" 502 150 2025-12-17 22:16:46,867 - lava_callback - ERROR - Error processing callback for node 69408701575bc060ce15e23b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408701575bc060ce15e23b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408701575bc060ce15e23b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408701575bc060ce15e23b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:46,869 - lava_callback - INFO - Starting async job submit for node 69408713575bc060ce15e266 2025-12-17 22:16:46,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:46,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408713575bc060ce15e266 HTTP/1.1" 502 150 2025-12-17 22:16:46,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408713575bc060ce15e266'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:46,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408713575bc060ce15e266 2025-12-17 22:16:46,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408713575bc060ce15e266 HTTP/1.1" 502 150 2025-12-17 22:16:46,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408713575bc060ce15e266'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:47,222 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086fb575bc060ce15e22c 2025-12-17 22:16:47,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086fb575bc060ce15e22c HTTP/1.1" 502 150 2025-12-17 22:16:47,223 - lava_callback - ERROR - Error processing callback for node 694086fb575bc060ce15e22c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086fb575bc060ce15e22c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086fb575bc060ce15e22c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086fb575bc060ce15e22c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:47,225 - lava_callback - INFO - Starting async job submit for node 69408714575bc060ce15e26d 2025-12-17 22:16:47,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:47,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408714575bc060ce15e26d HTTP/1.1" 502 150 2025-12-17 22:16:47,255 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408714575bc060ce15e26d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:47,256 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408714575bc060ce15e26d 2025-12-17 22:16:47,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408714575bc060ce15e26d HTTP/1.1" 502 150 2025-12-17 22:16:47,256 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408714575bc060ce15e26d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:47,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408722575bc060ce15e291 2025-12-17 22:16:47,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408722575bc060ce15e291 HTTP/1.1" 502 150 2025-12-17 22:16:47,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408722575bc060ce15e291'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:47,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871e575bc060ce15e288 2025-12-17 22:16:47,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871e575bc060ce15e288 HTTP/1.1" 502 150 2025-12-17 22:16:47,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871e575bc060ce15e288'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:47,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408723575bc060ce15e294 2025-12-17 22:16:47,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408723575bc060ce15e294 HTTP/1.1" 502 150 2025-12-17 22:16:47,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408723575bc060ce15e294'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:48,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408725575bc060ce15e297 2025-12-17 22:16:48,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408725575bc060ce15e297 HTTP/1.1" 502 150 2025-12-17 22:16:48,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408725575bc060ce15e297'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:48,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408713575bc060ce15e266 2025-12-17 22:16:48,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408713575bc060ce15e266 HTTP/1.1" 502 150 2025-12-17 22:16:48,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408713575bc060ce15e266'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:49,256 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408714575bc060ce15e26d 2025-12-17 22:16:49,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408714575bc060ce15e26d HTTP/1.1" 502 150 2025-12-17 22:16:49,258 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408714575bc060ce15e26d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:49,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408712575bc060ce15e25f HTTP/1.1" 502 150 2025-12-17 22:16:49,328 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408712575bc060ce15e25f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:49,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408721575bc060ce15e28e 2025-12-17 22:16:49,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408721575bc060ce15e28e HTTP/1.1" 502 150 2025-12-17 22:16:49,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408721575bc060ce15e28e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:50,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408703575bc060ce15e23e 2025-12-17 22:16:50,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408703575bc060ce15e23e HTTP/1.1" 502 150 2025-12-17 22:16:50,889 - lava_callback - ERROR - Error processing callback for node 69408703575bc060ce15e23e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408703575bc060ce15e23e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408703575bc060ce15e23e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408703575bc060ce15e23e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:50,891 - lava_callback - INFO - Starting async job submit for node 69408716575bc060ce15e270 2025-12-17 22:16:50,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:50,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408716575bc060ce15e270 HTTP/1.1" 502 150 2025-12-17 22:16:50,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408716575bc060ce15e270'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:50,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408716575bc060ce15e270 2025-12-17 22:16:50,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408716575bc060ce15e270 HTTP/1.1" 502 150 2025-12-17 22:16:50,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408716575bc060ce15e270'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:51,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408722575bc060ce15e291 2025-12-17 22:16:51,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408722575bc060ce15e291 HTTP/1.1" 502 150 2025-12-17 22:16:51,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408722575bc060ce15e291'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:51,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408723575bc060ce15e294 2025-12-17 22:16:51,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408723575bc060ce15e294 HTTP/1.1" 502 150 2025-12-17 22:16:51,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408723575bc060ce15e294'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:51,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408704575bc060ce15e241 2025-12-17 22:16:51,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408704575bc060ce15e241 HTTP/1.1" 502 150 2025-12-17 22:16:51,842 - lava_callback - ERROR - Error processing callback for node 69408704575bc060ce15e241: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408704575bc060ce15e241 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408704575bc060ce15e241 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408704575bc060ce15e241 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:51,844 - lava_callback - INFO - Starting async job submit for node 69408718575bc060ce15e27b 2025-12-17 22:16:51,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:51,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408718575bc060ce15e27b HTTP/1.1" 502 150 2025-12-17 22:16:51,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408718575bc060ce15e27b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:51,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408718575bc060ce15e27b 2025-12-17 22:16:51,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408718575bc060ce15e27b HTTP/1.1" 502 150 2025-12-17 22:16:51,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408718575bc060ce15e27b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:51,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408707575bc060ce15e247 2025-12-17 22:16:51,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408707575bc060ce15e247 HTTP/1.1" 502 150 2025-12-17 22:16:51,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408707575bc060ce15e247'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:52,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408725575bc060ce15e297 2025-12-17 22:16:52,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408725575bc060ce15e297 HTTP/1.1" 502 150 2025-12-17 22:16:52,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408725575bc060ce15e297'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:52,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408716575bc060ce15e270 2025-12-17 22:16:52,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408716575bc060ce15e270 HTTP/1.1" 502 150 2025-12-17 22:16:52,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408716575bc060ce15e270'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:52,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408713575bc060ce15e266 2025-12-17 22:16:52,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408713575bc060ce15e266 HTTP/1.1" 502 150 2025-12-17 22:16:52,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408713575bc060ce15e266'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:53,258 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408714575bc060ce15e26d 2025-12-17 22:16:53,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408714575bc060ce15e26d HTTP/1.1" 502 150 2025-12-17 22:16:53,259 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408714575bc060ce15e26d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:53,328 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408712575bc060ce15e25f 2025-12-17 22:16:53,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408712575bc060ce15e25f HTTP/1.1" 502 150 2025-12-17 22:16:53,329 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408712575bc060ce15e25f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:53,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408718575bc060ce15e27b 2025-12-17 22:16:53,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408718575bc060ce15e27b HTTP/1.1" 502 150 2025-12-17 22:16:53,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408718575bc060ce15e27b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:54,217 - lava_callback - INFO - Received callback for node 694088c2575bc060ce15e7fe INFO: 172.21.0.1:44920 - "POST /node/694088c2575bc060ce15e7fe HTTP/1.0" 202 Accepted 2025-12-17 22:16:54,859 - lava_callback - INFO - Received callback for node 694088c3575bc060ce15e805 INFO: 172.21.0.1:44924 - "POST /node/694088c3575bc060ce15e805 HTTP/1.0" 202 Accepted 2025-12-17 22:16:55,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870a575bc060ce15e24d 2025-12-17 22:16:55,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870a575bc060ce15e24d HTTP/1.1" 502 150 2025-12-17 22:16:55,044 - lava_callback - ERROR - Error processing callback for node 6940870a575bc060ce15e24d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940870a575bc060ce15e24d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940870a575bc060ce15e24d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940870a575bc060ce15e24d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:55,046 - lava_callback - INFO - Starting async job submit for node 6940871b575bc060ce15e285 2025-12-17 22:16:55,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:55,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871b575bc060ce15e285 HTTP/1.1" 502 150 2025-12-17 22:16:55,057 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871b575bc060ce15e285'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:55,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871b575bc060ce15e285 2025-12-17 22:16:55,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871b575bc060ce15e285 HTTP/1.1" 502 150 2025-12-17 22:16:55,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871b575bc060ce15e285'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:55,502 - lava_callback - INFO - Received callback for node 694088c5575bc060ce15e808 INFO: 172.21.0.1:44940 - "POST /node/694088c5575bc060ce15e808 HTTP/1.0" 202 Accepted 2025-12-17 22:16:55,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870e575bc060ce15e256 2025-12-17 22:16:55,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870e575bc060ce15e256 HTTP/1.1" 502 150 2025-12-17 22:16:55,574 - lava_callback - ERROR - Error processing callback for node 6940870e575bc060ce15e256: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940870e575bc060ce15e256 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940870e575bc060ce15e256 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940870e575bc060ce15e256 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:55,576 - lava_callback - INFO - Starting async job submit for node 6940871e575bc060ce15e28b 2025-12-17 22:16:55,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:55,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871e575bc060ce15e28b HTTP/1.1" 502 150 2025-12-17 22:16:55,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871e575bc060ce15e28b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:55,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871e575bc060ce15e28b 2025-12-17 22:16:55,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871e575bc060ce15e28b HTTP/1.1" 502 150 2025-12-17 22:16:55,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871e575bc060ce15e28b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:55,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940870f575bc060ce15e259 2025-12-17 22:16:55,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940870f575bc060ce15e259 HTTP/1.1" 502 150 2025-12-17 22:16:55,729 - lava_callback - ERROR - Error processing callback for node 6940870f575bc060ce15e259: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940870f575bc060ce15e259 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940870f575bc060ce15e259 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940870f575bc060ce15e259 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:16:55,732 - lava_callback - INFO - Starting async job submit for node 69408728575bc060ce15e2a5 2025-12-17 22:16:55,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:16:55,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408728575bc060ce15e2a5 HTTP/1.1" 502 150 2025-12-17 22:16:55,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408728575bc060ce15e2a5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:55,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408728575bc060ce15e2a5 2025-12-17 22:16:55,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408728575bc060ce15e2a5 HTTP/1.1" 502 150 2025-12-17 22:16:55,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408728575bc060ce15e2a5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:56,176 - lava_callback - INFO - Received callback for node 694088c7575bc060ce15e80b INFO: 172.21.0.1:44948 - "POST /node/694088c7575bc060ce15e80b HTTP/1.0" 202 Accepted INFO: 172.21.0.1:44952 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:16:56,827 - lava_callback - INFO - Received callback for node 694088cc575bc060ce15e81a INFO: 172.21.0.1:44954 - "POST /node/694088cc575bc060ce15e81a HTTP/1.0" 202 Accepted 2025-12-17 22:16:56,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408716575bc060ce15e270 2025-12-17 22:16:56,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408716575bc060ce15e270 HTTP/1.1" 502 150 2025-12-17 22:16:56,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408716575bc060ce15e270'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:57,059 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871b575bc060ce15e285 2025-12-17 22:16:57,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871b575bc060ce15e285 HTTP/1.1" 502 150 2025-12-17 22:16:57,060 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871b575bc060ce15e285'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:57,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408721575bc060ce15e28e 2025-12-17 22:16:57,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408721575bc060ce15e28e HTTP/1.1" 502 150 2025-12-17 22:16:57,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408721575bc060ce15e28e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:57,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871e575bc060ce15e28b 2025-12-17 22:16:57,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871e575bc060ce15e28b HTTP/1.1" 502 150 2025-12-17 22:16:57,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871e575bc060ce15e28b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:57,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408728575bc060ce15e2a5 2025-12-17 22:16:57,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408728575bc060ce15e2a5 HTTP/1.1" 502 150 2025-12-17 22:16:57,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408728575bc060ce15e2a5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:57,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408718575bc060ce15e27b 2025-12-17 22:16:57,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408718575bc060ce15e27b HTTP/1.1" 502 150 2025-12-17 22:16:57,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408718575bc060ce15e27b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:59,510 - lava_callback - INFO - Received callback for node 694088cb575bc060ce15e815 INFO: 172.21.0.1:44960 - "POST /node/694088cb575bc060ce15e815 HTTP/1.0" 202 Accepted 2025-12-17 22:16:59,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408722575bc060ce15e291 2025-12-17 22:16:59,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408722575bc060ce15e291 HTTP/1.1" 502 150 2025-12-17 22:16:59,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408722575bc060ce15e291'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:16:59,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408723575bc060ce15e294 2025-12-17 22:16:59,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408723575bc060ce15e294 HTTP/1.1" 502 150 2025-12-17 22:16:59,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408723575bc060ce15e294'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:00,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408710575bc060ce15e25c 2025-12-17 22:17:00,301 - lava_callback - INFO - Received callback for node 694088ce575bc060ce15e825 INFO: 172.21.0.1:53776 - "POST /node/694088ce575bc060ce15e825 HTTP/1.0" 202 Accepted 2025-12-17 22:17:00,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408725575bc060ce15e297 2025-12-17 22:17:00,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408725575bc060ce15e297 HTTP/1.1" 502 150 2025-12-17 22:17:00,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408725575bc060ce15e297'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:00,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408713575bc060ce15e266 2025-12-17 22:17:00,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408713575bc060ce15e266 HTTP/1.1" 502 150 2025-12-17 22:17:00,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408713575bc060ce15e266'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:01,060 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871b575bc060ce15e285 2025-12-17 22:17:01,105 - lava_callback - INFO - Received callback for node 694088d0575bc060ce15e828 INFO: 172.21.0.1:53788 - "POST /node/694088d0575bc060ce15e828 HTTP/1.0" 202 Accepted 2025-12-17 22:17:01,259 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408714575bc060ce15e26d 2025-12-17 22:17:01,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408714575bc060ce15e26d HTTP/1.1" 502 150 2025-12-17 22:17:01,260 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408714575bc060ce15e26d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:01,330 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408712575bc060ce15e25f 2025-12-17 22:17:01,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408712575bc060ce15e25f HTTP/1.1" 502 150 2025-12-17 22:17:01,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408712575bc060ce15e25f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:01,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871e575bc060ce15e28b 2025-12-17 22:17:01,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871e575bc060ce15e28b HTTP/1.1" 502 150 2025-12-17 22:17:01,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871e575bc060ce15e28b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:01,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408728575bc060ce15e2a5 2025-12-17 22:17:01,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408728575bc060ce15e2a5 HTTP/1.1" 502 150 2025-12-17 22:17:01,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408728575bc060ce15e2a5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:01,937 - lava_callback - INFO - Received callback for node 694088d1575bc060ce15e82b INFO: 172.21.0.1:53804 - "POST /node/694088d1575bc060ce15e82b HTTP/1.0" 202 Accepted INFO: 172.21.0.1:53808 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:17:02,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871a575bc060ce15e27e 2025-12-17 22:17:02,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871a575bc060ce15e27e HTTP/1.1" 502 150 2025-12-17 22:17:02,670 - lava_callback - ERROR - Error processing callback for node 6940871a575bc060ce15e27e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940871a575bc060ce15e27e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940871a575bc060ce15e27e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940871a575bc060ce15e27e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:02,672 - lava_callback - INFO - Starting async job submit for node 6940872c575bc060ce15e2d6 2025-12-17 22:17:02,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:02,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940872c575bc060ce15e2d6 HTTP/1.1" 502 150 2025-12-17 22:17:02,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940872c575bc060ce15e2d6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:02,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940872c575bc060ce15e2d6 2025-12-17 22:17:02,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940872c575bc060ce15e2d6 HTTP/1.1" 502 150 2025-12-17 22:17:02,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940872c575bc060ce15e2d6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:02,719 - lava_callback - INFO - Received callback for node 694088d2575bc060ce15e82e INFO: 172.21.0.1:53820 - "POST /node/694088d2575bc060ce15e82e HTTP/1.0" 202 Accepted 2025-12-17 22:17:03,509 - lava_callback - INFO - Received callback for node 694088d3575bc060ce15e831 INFO: 172.21.0.1:53826 - "POST /node/694088d3575bc060ce15e831 HTTP/1.0" 202 Accepted 2025-12-17 22:17:03,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871e575bc060ce15e288 2025-12-17 22:17:03,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871e575bc060ce15e288 HTTP/1.1" 502 150 2025-12-17 22:17:03,709 - lava_callback - ERROR - Error processing callback for node 6940871e575bc060ce15e288: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940871e575bc060ce15e288 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940871e575bc060ce15e288 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940871e575bc060ce15e288 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:03,711 - lava_callback - INFO - Starting async job submit for node 6940872e575bc060ce15e2e4 2025-12-17 22:17:03,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:03,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940872e575bc060ce15e2e4 HTTP/1.1" 502 150 2025-12-17 22:17:03,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940872e575bc060ce15e2e4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:03,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940872e575bc060ce15e2e4 2025-12-17 22:17:03,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940872e575bc060ce15e2e4 HTTP/1.1" 502 150 2025-12-17 22:17:03,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940872e575bc060ce15e2e4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:04,191 - lava_callback - INFO - Received callback for node 694088db575bc060ce15e843 INFO: 172.21.0.1:53840 - "POST /node/694088db575bc060ce15e843 HTTP/1.0" 202 Accepted 2025-12-17 22:17:04,297 - lava_callback - INFO - Received callback for node 694088d4575bc060ce15e834 INFO: 172.21.0.1:53846 - "POST /node/694088d4575bc060ce15e834 HTTP/1.0" 202 Accepted 2025-12-17 22:17:04,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940872c575bc060ce15e2d6 2025-12-17 22:17:04,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940872c575bc060ce15e2d6 HTTP/1.1" 502 150 2025-12-17 22:17:04,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940872c575bc060ce15e2d6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:04,822 - lava_callback - INFO - Received callback for node 694088dd575bc060ce15e850 INFO: 172.21.0.1:53858 - "POST /node/694088dd575bc060ce15e850 HTTP/1.0" 202 Accepted 2025-12-17 22:17:04,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408716575bc060ce15e270 2025-12-17 22:17:04,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408716575bc060ce15e270 HTTP/1.1" 502 150 2025-12-17 22:17:04,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408716575bc060ce15e270'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:05,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940872e575bc060ce15e2e4 2025-12-17 22:17:05,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940872e575bc060ce15e2e4 HTTP/1.1" 502 150 2025-12-17 22:17:05,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940872e575bc060ce15e2e4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:05,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408718575bc060ce15e27b 2025-12-17 22:17:05,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408718575bc060ce15e27b HTTP/1.1" 502 150 2025-12-17 22:17:05,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408718575bc060ce15e27b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53862 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:17:07,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408707575bc060ce15e247 2025-12-17 22:17:07,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408707575bc060ce15e247 HTTP/1.1" 502 150 2025-12-17 22:17:07,860 - lava_callback - ERROR - Error processing callback for node 69408707575bc060ce15e247: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408707575bc060ce15e247 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408707575bc060ce15e247 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408707575bc060ce15e247 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:07,863 - lava_callback - INFO - Starting async job submit for node 69408730575bc060ce15e2e7 2025-12-17 22:17:07,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:07,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408730575bc060ce15e2e7 HTTP/1.1" 502 150 2025-12-17 22:17:07,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408730575bc060ce15e2e7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:07,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408730575bc060ce15e2e7 2025-12-17 22:17:07,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408730575bc060ce15e2e7 HTTP/1.1" 502 150 2025-12-17 22:17:07,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408730575bc060ce15e2e7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:08,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940872c575bc060ce15e2d6 2025-12-17 22:17:08,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940872c575bc060ce15e2d6 HTTP/1.1" 502 150 2025-12-17 22:17:08,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940872c575bc060ce15e2d6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:09,408 - lava_callback - INFO - Received callback for node 694088d9575bc060ce15e83d INFO: 172.21.0.1:53868 - "POST /node/694088d9575bc060ce15e83d HTTP/1.0" 202 Accepted 2025-12-17 22:17:09,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871e575bc060ce15e28b 2025-12-17 22:17:09,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871e575bc060ce15e28b HTTP/1.1" 502 150 2025-12-17 22:17:09,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871e575bc060ce15e28b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:09,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940872e575bc060ce15e2e4 2025-12-17 22:17:09,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940872e575bc060ce15e2e4 HTTP/1.1" 502 150 2025-12-17 22:17:09,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940872e575bc060ce15e2e4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:09,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408728575bc060ce15e2a5 2025-12-17 22:17:09,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408728575bc060ce15e2a5 HTTP/1.1" 502 150 2025-12-17 22:17:09,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408728575bc060ce15e2a5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:09,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408730575bc060ce15e2e7 2025-12-17 22:17:09,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408730575bc060ce15e2e7 HTTP/1.1" 502 150 2025-12-17 22:17:09,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408730575bc060ce15e2e7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:10,127 - lava_callback - INFO - Received callback for node 694088da575bc060ce15e840 INFO: 172.21.0.1:35896 - "POST /node/694088da575bc060ce15e840 HTTP/1.0" 202 Accepted 2025-12-17 22:17:13,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408721575bc060ce15e28e 2025-12-17 22:17:13,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408721575bc060ce15e28e HTTP/1.1" 502 150 2025-12-17 22:17:13,511 - lava_callback - ERROR - Error processing callback for node 69408721575bc060ce15e28e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408721575bc060ce15e28e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408721575bc060ce15e28e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408721575bc060ce15e28e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:13,513 - lava_callback - INFO - Starting async job submit for node 69408733575bc060ce15e2ed 2025-12-17 22:17:13,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:13,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408733575bc060ce15e2ed HTTP/1.1" 502 150 2025-12-17 22:17:13,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408733575bc060ce15e2ed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:13,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408733575bc060ce15e2ed 2025-12-17 22:17:13,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408733575bc060ce15e2ed HTTP/1.1" 502 150 2025-12-17 22:17:13,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408733575bc060ce15e2ed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:13,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408730575bc060ce15e2e7 2025-12-17 22:17:13,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408730575bc060ce15e2e7 HTTP/1.1" 502 150 2025-12-17 22:17:13,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408730575bc060ce15e2e7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:14,225 - lava_callback - INFO - Received callback for node 694088d5575bc060ce15e837 INFO: 172.21.0.1:35898 - "POST /node/694088d5575bc060ce15e837 HTTP/1.0" 202 Accepted 2025-12-17 22:17:14,236 - lava_callback - INFO - Received callback for node 694086c8575bc060ce15e186 INFO: 172.21.0.1:35904 - "POST /node/694086c8575bc060ce15e186 HTTP/1.0" 202 Accepted 2025-12-17 22:17:14,898 - lava_callback - INFO - Received callback for node 694088d7575bc060ce15e83a INFO: 172.21.0.1:35918 - "POST /node/694088d7575bc060ce15e83a HTTP/1.0" 202 Accepted INFO: 172.21.0.1:35934 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:17:15,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408710575bc060ce15e25c HTTP/1.1" 502 150 2025-12-17 22:17:15,440 - lava_callback - ERROR - Error processing callback for node 69408710575bc060ce15e25c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408710575bc060ce15e25c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408710575bc060ce15e25c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408710575bc060ce15e25c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:15,442 - lava_callback - INFO - Starting async job submit for node 69408735575bc060ce15e2f0 2025-12-17 22:17:15,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:15,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408735575bc060ce15e2f0 HTTP/1.1" 502 150 2025-12-17 22:17:15,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408735575bc060ce15e2f0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:15,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408735575bc060ce15e2f0 2025-12-17 22:17:15,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408735575bc060ce15e2f0 HTTP/1.1" 502 150 2025-12-17 22:17:15,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408735575bc060ce15e2f0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:15,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408733575bc060ce15e2ed 2025-12-17 22:17:15,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408733575bc060ce15e2ed HTTP/1.1" 502 150 2025-12-17 22:17:15,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408733575bc060ce15e2ed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:15,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408722575bc060ce15e291 2025-12-17 22:17:15,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408722575bc060ce15e291 HTTP/1.1" 502 150 2025-12-17 22:17:15,671 - lava_callback - ERROR - Error processing callback for node 69408722575bc060ce15e291: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408722575bc060ce15e291 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408722575bc060ce15e291 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408722575bc060ce15e291 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:15,673 - lava_callback - INFO - Starting async job submit for node 69408736575bc060ce15e2f3 2025-12-17 22:17:15,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:15,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408736575bc060ce15e2f3 HTTP/1.1" 502 150 2025-12-17 22:17:15,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408736575bc060ce15e2f3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:15,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408736575bc060ce15e2f3 2025-12-17 22:17:15,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408736575bc060ce15e2f3 HTTP/1.1" 502 150 2025-12-17 22:17:15,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408736575bc060ce15e2f3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:15,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408723575bc060ce15e294 2025-12-17 22:17:15,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408723575bc060ce15e294 HTTP/1.1" 502 150 2025-12-17 22:17:15,752 - lava_callback - ERROR - Error processing callback for node 69408723575bc060ce15e294: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408723575bc060ce15e294 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408723575bc060ce15e294 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408723575bc060ce15e294 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:15,754 - lava_callback - INFO - Starting async job submit for node 69408739575bc060ce15e2fd 2025-12-17 22:17:15,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:15,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408739575bc060ce15e2fd HTTP/1.1" 502 150 2025-12-17 22:17:15,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408739575bc060ce15e2fd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:15,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408739575bc060ce15e2fd 2025-12-17 22:17:15,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408739575bc060ce15e2fd HTTP/1.1" 502 150 2025-12-17 22:17:15,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408739575bc060ce15e2fd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:16,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871b575bc060ce15e285 HTTP/1.1" 502 150 2025-12-17 22:17:16,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871b575bc060ce15e285'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35940 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:17:16,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940872c575bc060ce15e2d6 2025-12-17 22:17:16,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940872c575bc060ce15e2d6 HTTP/1.1" 502 150 2025-12-17 22:17:16,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940872c575bc060ce15e2d6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:16,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408725575bc060ce15e297 2025-12-17 22:17:16,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408725575bc060ce15e297 HTTP/1.1" 502 150 2025-12-17 22:17:16,850 - lava_callback - ERROR - Error processing callback for node 69408725575bc060ce15e297: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408725575bc060ce15e297 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408725575bc060ce15e297 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408725575bc060ce15e297 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:16,852 - lava_callback - INFO - Starting async job submit for node 6940873a575bc060ce15e300 2025-12-17 22:17:16,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:16,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873a575bc060ce15e300 HTTP/1.1" 502 150 2025-12-17 22:17:16,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940873a575bc060ce15e300'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:16,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940873a575bc060ce15e300 2025-12-17 22:17:16,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873a575bc060ce15e300 HTTP/1.1" 502 150 2025-12-17 22:17:16,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940873a575bc060ce15e300'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:16,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408713575bc060ce15e266 2025-12-17 22:17:16,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408713575bc060ce15e266 HTTP/1.1" 502 150 2025-12-17 22:17:16,906 - lava_callback - ERROR - Error processing callback for node 69408713575bc060ce15e266: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408713575bc060ce15e266 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408713575bc060ce15e266 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408713575bc060ce15e266 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:16,908 - lava_callback - INFO - Starting async job submit for node 69408732575bc060ce15e2ea 2025-12-17 22:17:16,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:16,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408732575bc060ce15e2ea HTTP/1.1" 502 150 2025-12-17 22:17:16,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408732575bc060ce15e2ea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:16,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408732575bc060ce15e2ea 2025-12-17 22:17:16,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408732575bc060ce15e2ea HTTP/1.1" 502 150 2025-12-17 22:17:16,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408732575bc060ce15e2ea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:17,260 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408714575bc060ce15e26d 2025-12-17 22:17:17,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408714575bc060ce15e26d HTTP/1.1" 502 150 2025-12-17 22:17:17,262 - lava_callback - ERROR - Error processing callback for node 69408714575bc060ce15e26d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408714575bc060ce15e26d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408714575bc060ce15e26d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408714575bc060ce15e26d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:17,264 - lava_callback - INFO - Starting async job submit for node 69408738575bc060ce15e2f6 2025-12-17 22:17:17,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:17,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408738575bc060ce15e2f6 HTTP/1.1" 502 150 2025-12-17 22:17:17,271 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408738575bc060ce15e2f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:17,271 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408738575bc060ce15e2f6 2025-12-17 22:17:17,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408738575bc060ce15e2f6 HTTP/1.1" 502 150 2025-12-17 22:17:17,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408738575bc060ce15e2f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:17,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408712575bc060ce15e25f 2025-12-17 22:17:17,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408712575bc060ce15e25f HTTP/1.1" 502 150 2025-12-17 22:17:17,332 - lava_callback - ERROR - Error processing callback for node 69408712575bc060ce15e25f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408712575bc060ce15e25f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408712575bc060ce15e25f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408712575bc060ce15e25f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:17,334 - lava_callback - INFO - Starting async job submit for node 6940873c575bc060ce15e307 2025-12-17 22:17:17,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:17,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873c575bc060ce15e307 HTTP/1.1" 502 150 2025-12-17 22:17:17,341 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940873c575bc060ce15e307'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:17,341 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940873c575bc060ce15e307 2025-12-17 22:17:17,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873c575bc060ce15e307 HTTP/1.1" 502 150 2025-12-17 22:17:17,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940873c575bc060ce15e307'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:17,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408735575bc060ce15e2f0 2025-12-17 22:17:17,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408735575bc060ce15e2f0 HTTP/1.1" 502 150 2025-12-17 22:17:17,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408735575bc060ce15e2f0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:17,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408736575bc060ce15e2f3 2025-12-17 22:17:17,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408736575bc060ce15e2f3 HTTP/1.1" 502 150 2025-12-17 22:17:17,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408736575bc060ce15e2f3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:17,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940872e575bc060ce15e2e4 2025-12-17 22:17:17,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940872e575bc060ce15e2e4 HTTP/1.1" 502 150 2025-12-17 22:17:17,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940872e575bc060ce15e2e4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:17,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408739575bc060ce15e2fd 2025-12-17 22:17:17,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408739575bc060ce15e2fd HTTP/1.1" 502 150 2025-12-17 22:17:17,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408739575bc060ce15e2fd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:18,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940873a575bc060ce15e300 2025-12-17 22:17:18,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873a575bc060ce15e300 HTTP/1.1" 502 150 2025-12-17 22:17:18,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940873a575bc060ce15e300'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:18,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408732575bc060ce15e2ea 2025-12-17 22:17:18,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408732575bc060ce15e2ea HTTP/1.1" 502 150 2025-12-17 22:17:18,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408732575bc060ce15e2ea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:19,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408738575bc060ce15e2f6 2025-12-17 22:17:19,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408738575bc060ce15e2f6 HTTP/1.1" 502 150 2025-12-17 22:17:19,274 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408738575bc060ce15e2f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:19,342 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940873c575bc060ce15e307 2025-12-17 22:17:19,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873c575bc060ce15e307 HTTP/1.1" 502 150 2025-12-17 22:17:19,343 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940873c575bc060ce15e307'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:19,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408733575bc060ce15e2ed 2025-12-17 22:17:19,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408733575bc060ce15e2ed HTTP/1.1" 502 150 2025-12-17 22:17:19,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408733575bc060ce15e2ed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:19,809 - lava_callback - INFO - Received callback for node 694088e5575bc060ce15e893 INFO: 172.21.0.1:36772 - "POST /node/694088e5575bc060ce15e893 HTTP/1.0" 202 Accepted 2025-12-17 22:17:20,574 - lava_callback - INFO - Received callback for node 694088ec575bc060ce15e8a9 INFO: 172.21.0.1:36786 - "POST /node/694088ec575bc060ce15e8a9 HTTP/1.0" 202 Accepted 2025-12-17 22:17:20,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408716575bc060ce15e270 2025-12-17 22:17:20,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408716575bc060ce15e270 HTTP/1.1" 502 150 2025-12-17 22:17:20,905 - lava_callback - ERROR - Error processing callback for node 69408716575bc060ce15e270: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408716575bc060ce15e270 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408716575bc060ce15e270 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408716575bc060ce15e270 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:20,907 - lava_callback - INFO - Starting async job submit for node 6940873e575bc060ce15e310 2025-12-17 22:17:20,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:20,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873e575bc060ce15e310 HTTP/1.1" 502 150 2025-12-17 22:17:20,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940873e575bc060ce15e310'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:20,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940873e575bc060ce15e310 2025-12-17 22:17:20,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873e575bc060ce15e310 HTTP/1.1" 502 150 2025-12-17 22:17:20,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940873e575bc060ce15e310'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:21,331 - lava_callback - INFO - Received callback for node 694088ef575bc060ce15e8af INFO: 172.21.0.1:36788 - "POST /node/694088ef575bc060ce15e8af HTTP/1.0" 202 Accepted 2025-12-17 22:17:21,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408735575bc060ce15e2f0 2025-12-17 22:17:21,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408735575bc060ce15e2f0 HTTP/1.1" 502 150 2025-12-17 22:17:21,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408735575bc060ce15e2f0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:21,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408736575bc060ce15e2f3 2025-12-17 22:17:21,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408736575bc060ce15e2f3 HTTP/1.1" 502 150 2025-12-17 22:17:21,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408736575bc060ce15e2f3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:21,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408739575bc060ce15e2fd 2025-12-17 22:17:21,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408739575bc060ce15e2fd HTTP/1.1" 502 150 2025-12-17 22:17:21,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408739575bc060ce15e2fd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:21,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408718575bc060ce15e27b 2025-12-17 22:17:21,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408718575bc060ce15e27b HTTP/1.1" 502 150 2025-12-17 22:17:21,864 - lava_callback - ERROR - Error processing callback for node 69408718575bc060ce15e27b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408718575bc060ce15e27b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408718575bc060ce15e27b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408718575bc060ce15e27b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:21,866 - lava_callback - INFO - Starting async job submit for node 69408741575bc060ce15e319 2025-12-17 22:17:21,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:21,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408730575bc060ce15e2e7 2025-12-17 22:17:21,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408730575bc060ce15e2e7 HTTP/1.1" 502 150 2025-12-17 22:17:21,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408730575bc060ce15e2e7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:21,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408741575bc060ce15e319 HTTP/1.1" 502 150 2025-12-17 22:17:21,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408741575bc060ce15e319'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:21,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408741575bc060ce15e319 2025-12-17 22:17:21,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408741575bc060ce15e319 HTTP/1.1" 502 150 2025-12-17 22:17:21,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408741575bc060ce15e319'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:22,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940873a575bc060ce15e300 2025-12-17 22:17:22,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873a575bc060ce15e300 HTTP/1.1" 502 150 2025-12-17 22:17:22,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940873a575bc060ce15e300'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:22,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408732575bc060ce15e2ea 2025-12-17 22:17:22,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408732575bc060ce15e2ea HTTP/1.1" 502 150 2025-12-17 22:17:22,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408732575bc060ce15e2ea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:22,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940873e575bc060ce15e310 2025-12-17 22:17:22,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873e575bc060ce15e310 HTTP/1.1" 502 150 2025-12-17 22:17:22,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940873e575bc060ce15e310'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:23,274 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408738575bc060ce15e2f6 2025-12-17 22:17:23,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408738575bc060ce15e2f6 HTTP/1.1" 502 150 2025-12-17 22:17:23,275 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408738575bc060ce15e2f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:23,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940873c575bc060ce15e307 2025-12-17 22:17:23,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873c575bc060ce15e307 HTTP/1.1" 502 150 2025-12-17 22:17:23,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940873c575bc060ce15e307'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:23,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408741575bc060ce15e319 2025-12-17 22:17:23,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408741575bc060ce15e319 HTTP/1.1" 502 150 2025-12-17 22:17:23,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408741575bc060ce15e319'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:24,205 - lava_callback - INFO - Received callback for node 694088e2575bc060ce15e88b INFO: 172.21.0.1:36792 - "POST /node/694088e2575bc060ce15e88b HTTP/1.0" 202 Accepted 2025-12-17 22:17:24,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871b575bc060ce15e285 2025-12-17 22:17:24,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871b575bc060ce15e285 HTTP/1.1" 502 150 2025-12-17 22:17:24,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940871b575bc060ce15e285'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:24,831 - lava_callback - INFO - Received callback for node 694088df575bc060ce15e87d INFO: 172.21.0.1:36798 - "POST /node/694088df575bc060ce15e87d HTTP/1.0" 202 Accepted 2025-12-17 22:17:25,462 - lava_callback - INFO - Received callback for node 694088e1575bc060ce15e884 INFO: 172.21.0.1:36812 - "POST /node/694088e1575bc060ce15e884 HTTP/1.0" 202 Accepted 2025-12-17 22:17:25,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871e575bc060ce15e28b 2025-12-17 22:17:25,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871e575bc060ce15e28b HTTP/1.1" 502 150 2025-12-17 22:17:25,592 - lava_callback - ERROR - Error processing callback for node 6940871e575bc060ce15e28b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940871e575bc060ce15e28b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940871e575bc060ce15e28b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940871e575bc060ce15e28b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:25,593 - lava_callback - INFO - Starting async job submit for node 69408742575bc060ce15e31c 2025-12-17 22:17:25,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:25,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408742575bc060ce15e31c HTTP/1.1" 502 150 2025-12-17 22:17:25,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408742575bc060ce15e31c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:25,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408742575bc060ce15e31c 2025-12-17 22:17:25,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408742575bc060ce15e31c HTTP/1.1" 502 150 2025-12-17 22:17:25,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408742575bc060ce15e31c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:25,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408728575bc060ce15e2a5 2025-12-17 22:17:25,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408728575bc060ce15e2a5 HTTP/1.1" 502 150 2025-12-17 22:17:25,749 - lava_callback - ERROR - Error processing callback for node 69408728575bc060ce15e2a5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408728575bc060ce15e2a5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408728575bc060ce15e2a5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408728575bc060ce15e2a5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:25,751 - lava_callback - INFO - Starting async job submit for node 6940c6651e3c24e95681d8d8 2025-12-17 22:17:25,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:25,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c6651e3c24e95681d8d8 HTTP/1.1" 502 150 2025-12-17 22:17:25,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c6651e3c24e95681d8d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:25,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c6651e3c24e95681d8d8 2025-12-17 22:17:25,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c6651e3c24e95681d8d8 HTTP/1.1" 502 150 2025-12-17 22:17:25,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c6651e3c24e95681d8d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:26,125 - lava_callback - INFO - Received callback for node 694088e3575bc060ce15e890 INFO: 172.21.0.1:36816 - "POST /node/694088e3575bc060ce15e890 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:36820 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:17:26,767 - lava_callback - INFO - Received callback for node 694088e6575bc060ce15e896 INFO: 172.21.0.1:36828 - "POST /node/694088e6575bc060ce15e896 HTTP/1.0" 202 Accepted 2025-12-17 22:17:26,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940873e575bc060ce15e310 2025-12-17 22:17:27,428 - lava_callback - INFO - Received callback for node 694088e7575bc060ce15e899 INFO: 172.21.0.1:36844 - "POST /node/694088e7575bc060ce15e899 HTTP/1.0" 202 Accepted 2025-12-17 22:17:27,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408733575bc060ce15e2ed 2025-12-17 22:17:27,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408742575bc060ce15e31c 2025-12-17 22:17:27,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408742575bc060ce15e31c HTTP/1.1" 502 150 2025-12-17 22:17:27,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408742575bc060ce15e31c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:27,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c6651e3c24e95681d8d8 2025-12-17 22:17:27,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c6651e3c24e95681d8d8 HTTP/1.1" 502 150 2025-12-17 22:17:27,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c6651e3c24e95681d8d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:27,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408741575bc060ce15e319 2025-12-17 22:17:27,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408741575bc060ce15e319 HTTP/1.1" 502 150 2025-12-17 22:17:27,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408741575bc060ce15e319'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:28,092 - lava_callback - INFO - Received callback for node 694088e8575bc060ce15e89c INFO: 172.21.0.1:36856 - "POST /node/694088e8575bc060ce15e89c HTTP/1.0" 202 Accepted 2025-12-17 22:17:28,728 - lava_callback - INFO - Received callback for node 694088ea575bc060ce15e8a3 INFO: 172.21.0.1:36858 - "POST /node/694088ea575bc060ce15e8a3 HTTP/1.0" 202 Accepted 2025-12-17 22:17:29,368 - lava_callback - INFO - Received callback for node 694088eb575bc060ce15e8a6 INFO: 172.21.0.1:36866 - "POST /node/694088eb575bc060ce15e8a6 HTTP/1.0" 202 Accepted 2025-12-17 22:17:29,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408735575bc060ce15e2f0 2025-12-17 22:17:29,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408735575bc060ce15e2f0 HTTP/1.1" 502 150 2025-12-17 22:17:29,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408735575bc060ce15e2f0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:29,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408736575bc060ce15e2f3 2025-12-17 22:17:29,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408736575bc060ce15e2f3 HTTP/1.1" 502 150 2025-12-17 22:17:29,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408736575bc060ce15e2f3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:29,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408739575bc060ce15e2fd 2025-12-17 22:17:29,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408739575bc060ce15e2fd HTTP/1.1" 502 150 2025-12-17 22:17:29,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408739575bc060ce15e2fd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:30,016 - lava_callback - INFO - Received callback for node 694088ed575bc060ce15e8ac INFO: 172.21.0.1:43208 - "POST /node/694088ed575bc060ce15e8ac HTTP/1.0" 202 Accepted 2025-12-17 22:17:30,646 - lava_callback - INFO - Received callback for node 694088f0575bc060ce15e8b2 INFO: 172.21.0.1:43214 - "POST /node/694088f0575bc060ce15e8b2 HTTP/1.0" 202 Accepted 2025-12-17 22:17:30,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940873a575bc060ce15e300 2025-12-17 22:17:30,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873a575bc060ce15e300 HTTP/1.1" 502 150 2025-12-17 22:17:30,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940873a575bc060ce15e300'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:30,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408732575bc060ce15e2ea 2025-12-17 22:17:30,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408732575bc060ce15e2ea HTTP/1.1" 502 150 2025-12-17 22:17:30,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408732575bc060ce15e2ea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:31,275 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408738575bc060ce15e2f6 2025-12-17 22:17:31,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408738575bc060ce15e2f6 HTTP/1.1" 502 150 2025-12-17 22:17:31,277 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408738575bc060ce15e2f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:31,288 - lava_callback - INFO - Received callback for node 694088f2575bc060ce15e8ba INFO: 172.21.0.1:43228 - "POST /node/694088f2575bc060ce15e8ba HTTP/1.0" 202 Accepted 2025-12-17 22:17:31,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940873c575bc060ce15e307 2025-12-17 22:17:31,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873c575bc060ce15e307 HTTP/1.1" 502 150 2025-12-17 22:17:31,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940873c575bc060ce15e307'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:31,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408742575bc060ce15e31c 2025-12-17 22:17:31,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408742575bc060ce15e31c HTTP/1.1" 502 150 2025-12-17 22:17:31,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408742575bc060ce15e31c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:31,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c6651e3c24e95681d8d8 2025-12-17 22:17:31,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c6651e3c24e95681d8d8 HTTP/1.1" 502 150 2025-12-17 22:17:31,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c6651e3c24e95681d8d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:32,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940872c575bc060ce15e2d6 2025-12-17 22:17:32,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940872c575bc060ce15e2d6 HTTP/1.1" 502 150 2025-12-17 22:17:32,685 - lava_callback - ERROR - Error processing callback for node 6940872c575bc060ce15e2d6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940872c575bc060ce15e2d6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940872c575bc060ce15e2d6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940872c575bc060ce15e2d6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:32,688 - lava_callback - INFO - Starting async job submit for node 69408744575bc060ce15e31f 2025-12-17 22:17:32,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:32,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408744575bc060ce15e31f HTTP/1.1" 502 150 2025-12-17 22:17:32,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408744575bc060ce15e31f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:32,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408744575bc060ce15e31f 2025-12-17 22:17:32,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408744575bc060ce15e31f HTTP/1.1" 502 150 2025-12-17 22:17:32,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408744575bc060ce15e31f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:33,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940872e575bc060ce15e2e4 2025-12-17 22:17:33,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940872e575bc060ce15e2e4 HTTP/1.1" 502 150 2025-12-17 22:17:33,736 - lava_callback - ERROR - Error processing callback for node 6940872e575bc060ce15e2e4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940872e575bc060ce15e2e4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940872e575bc060ce15e2e4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940872e575bc060ce15e2e4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:33,738 - lava_callback - INFO - Starting async job submit for node 69408749575bc060ce15e325 2025-12-17 22:17:33,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:33,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408749575bc060ce15e325 HTTP/1.1" 502 150 2025-12-17 22:17:33,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408749575bc060ce15e325'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:33,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408749575bc060ce15e325 2025-12-17 22:17:33,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408749575bc060ce15e325 HTTP/1.1" 502 150 2025-12-17 22:17:33,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408749575bc060ce15e325'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:34,293 - lava_callback - INFO - Received callback for node 694088f5575bc060ce15e8cc INFO: 172.21.0.1:43238 - "POST /node/694088f5575bc060ce15e8cc HTTP/1.0" 202 Accepted 2025-12-17 22:17:34,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408744575bc060ce15e31f 2025-12-17 22:17:34,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408744575bc060ce15e31f HTTP/1.1" 502 150 2025-12-17 22:17:34,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408744575bc060ce15e31f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:34,966 - lava_callback - INFO - Received callback for node 694088f9575bc060ce15e8d6 INFO: 172.21.0.1:43246 - "POST /node/694088f9575bc060ce15e8d6 HTTP/1.0" 202 Accepted 2025-12-17 22:17:35,638 - lava_callback - INFO - Received callback for node 694088fd575bc060ce15e8df INFO: 172.21.0.1:43254 - "POST /node/694088fd575bc060ce15e8df HTTP/1.0" 202 Accepted 2025-12-17 22:17:35,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408749575bc060ce15e325 2025-12-17 22:17:35,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408749575bc060ce15e325 HTTP/1.1" 502 150 2025-12-17 22:17:35,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408749575bc060ce15e325'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:35,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408741575bc060ce15e319 2025-12-17 22:17:35,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408741575bc060ce15e319 HTTP/1.1" 502 150 2025-12-17 22:17:35,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408741575bc060ce15e319'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43266 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:17:37,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408730575bc060ce15e2e7 2025-12-17 22:17:37,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408730575bc060ce15e2e7 HTTP/1.1" 502 150 2025-12-17 22:17:37,880 - lava_callback - ERROR - Error processing callback for node 69408730575bc060ce15e2e7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408730575bc060ce15e2e7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408730575bc060ce15e2e7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408730575bc060ce15e2e7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:37,882 - lava_callback - INFO - Starting async job submit for node 6940874b575bc060ce15e328 2025-12-17 22:17:37,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:37,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940874b575bc060ce15e328 HTTP/1.1" 502 150 2025-12-17 22:17:37,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940874b575bc060ce15e328'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:37,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940874b575bc060ce15e328 2025-12-17 22:17:37,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940874b575bc060ce15e328 HTTP/1.1" 502 150 2025-12-17 22:17:37,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940874b575bc060ce15e328'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:38,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408744575bc060ce15e31f 2025-12-17 22:17:38,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408744575bc060ce15e31f HTTP/1.1" 502 150 2025-12-17 22:17:38,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408744575bc060ce15e31f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:39,490 - lava_callback - INFO - Received callback for node 694088f1575bc060ce15e8b5 INFO: 172.21.0.1:43278 - "POST /node/694088f1575bc060ce15e8b5 HTTP/1.0" 202 Accepted 2025-12-17 22:17:39,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408742575bc060ce15e31c 2025-12-17 22:17:39,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408749575bc060ce15e325 2025-12-17 22:17:39,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408749575bc060ce15e325 HTTP/1.1" 502 150 2025-12-17 22:17:39,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408749575bc060ce15e325'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:39,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c6651e3c24e95681d8d8 2025-12-17 22:17:39,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c6651e3c24e95681d8d8 HTTP/1.1" 502 150 2025-12-17 22:17:39,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c6651e3c24e95681d8d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:39,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940874b575bc060ce15e328 2025-12-17 22:17:39,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940874b575bc060ce15e328 HTTP/1.1" 502 150 2025-12-17 22:17:39,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940874b575bc060ce15e328'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:40,246 - lava_callback - INFO - Received callback for node 694088fc575bc060ce15e8dc INFO: 172.21.0.1:40612 - "POST /node/694088fc575bc060ce15e8dc HTTP/1.0" 202 Accepted 2025-12-17 22:17:40,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940871b575bc060ce15e285 2025-12-17 22:17:40,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940871b575bc060ce15e285 HTTP/1.1" 502 150 2025-12-17 22:17:40,467 - lava_callback - ERROR - Error processing callback for node 6940871b575bc060ce15e285: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940871b575bc060ce15e285 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940871b575bc060ce15e285 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940871b575bc060ce15e285 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:40,469 - lava_callback - INFO - Starting async job submit for node 6940874e575bc060ce15e32b 2025-12-17 22:17:40,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:40,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940874e575bc060ce15e32b HTTP/1.1" 502 150 2025-12-17 22:17:40,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940874e575bc060ce15e32b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:40,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940874e575bc060ce15e32b 2025-12-17 22:17:40,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940874e575bc060ce15e32b HTTP/1.1" 502 150 2025-12-17 22:17:40,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940874e575bc060ce15e32b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:42,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873e575bc060ce15e310 HTTP/1.1" 502 150 2025-12-17 22:17:42,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940873e575bc060ce15e310'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:42,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940874e575bc060ce15e32b 2025-12-17 22:17:42,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940874e575bc060ce15e32b HTTP/1.1" 502 150 2025-12-17 22:17:42,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940874e575bc060ce15e32b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:42,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408733575bc060ce15e2ed HTTP/1.1" 502 150 2025-12-17 22:17:42,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408733575bc060ce15e2ed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:43,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940874b575bc060ce15e328 2025-12-17 22:17:43,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940874b575bc060ce15e328 HTTP/1.1" 502 150 2025-12-17 22:17:43,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940874b575bc060ce15e328'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:44,234 - lava_callback - INFO - Received callback for node 694088f4575bc060ce15e8c1 INFO: 172.21.0.1:40618 - "POST /node/694088f4575bc060ce15e8c1 HTTP/1.0" 202 Accepted 2025-12-17 22:17:44,878 - lava_callback - INFO - Received callback for node 694088f7575bc060ce15e8cf INFO: 172.21.0.1:40632 - "POST /node/694088f7575bc060ce15e8cf HTTP/1.0" 202 Accepted 2025-12-17 22:17:45,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408735575bc060ce15e2f0 2025-12-17 22:17:45,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408735575bc060ce15e2f0 HTTP/1.1" 502 150 2025-12-17 22:17:45,491 - lava_callback - ERROR - Error processing callback for node 69408735575bc060ce15e2f0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408735575bc060ce15e2f0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408735575bc060ce15e2f0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408735575bc060ce15e2f0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:45,493 - lava_callback - INFO - Starting async job submit for node 69408750575bc060ce15e332 2025-12-17 22:17:45,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:45,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408750575bc060ce15e332 HTTP/1.1" 502 150 2025-12-17 22:17:45,501 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408750575bc060ce15e332'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:45,501 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408750575bc060ce15e332 2025-12-17 22:17:45,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408750575bc060ce15e332 HTTP/1.1" 502 150 2025-12-17 22:17:45,501 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408750575bc060ce15e332'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:45,533 - lava_callback - INFO - Received callback for node 694088fa575bc060ce15e8d9 INFO: 172.21.0.1:40640 - "POST /node/694088fa575bc060ce15e8d9 HTTP/1.0" 202 Accepted 2025-12-17 22:17:45,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408736575bc060ce15e2f3 2025-12-17 22:17:45,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408736575bc060ce15e2f3 HTTP/1.1" 502 150 2025-12-17 22:17:45,687 - lava_callback - ERROR - Error processing callback for node 69408736575bc060ce15e2f3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408736575bc060ce15e2f3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408736575bc060ce15e2f3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408736575bc060ce15e2f3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:45,688 - lava_callback - INFO - Starting async job submit for node 69408752575bc060ce15e335 2025-12-17 22:17:45,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:45,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408752575bc060ce15e335 HTTP/1.1" 502 150 2025-12-17 22:17:45,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408752575bc060ce15e335'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:45,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408752575bc060ce15e335 2025-12-17 22:17:45,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408752575bc060ce15e335 HTTP/1.1" 502 150 2025-12-17 22:17:45,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408752575bc060ce15e335'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:45,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408739575bc060ce15e2fd 2025-12-17 22:17:45,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408739575bc060ce15e2fd HTTP/1.1" 502 150 2025-12-17 22:17:45,765 - lava_callback - ERROR - Error processing callback for node 69408739575bc060ce15e2fd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408739575bc060ce15e2fd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408739575bc060ce15e2fd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408739575bc060ce15e2fd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:45,767 - lava_callback - INFO - Starting async job submit for node 69408756575bc060ce15e340 2025-12-17 22:17:45,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:45,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408756575bc060ce15e340 HTTP/1.1" 502 150 2025-12-17 22:17:45,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408756575bc060ce15e340'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:45,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408756575bc060ce15e340 2025-12-17 22:17:45,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408756575bc060ce15e340 HTTP/1.1" 502 150 2025-12-17 22:17:45,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408756575bc060ce15e340'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:46,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940874e575bc060ce15e32b 2025-12-17 22:17:46,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940874e575bc060ce15e32b HTTP/1.1" 502 150 2025-12-17 22:17:46,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940874e575bc060ce15e32b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40654 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:17:46,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408744575bc060ce15e31f 2025-12-17 22:17:46,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408744575bc060ce15e31f HTTP/1.1" 502 150 2025-12-17 22:17:46,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408744575bc060ce15e31f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:46,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940873a575bc060ce15e300 2025-12-17 22:17:46,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873a575bc060ce15e300 HTTP/1.1" 502 150 2025-12-17 22:17:46,869 - lava_callback - ERROR - Error processing callback for node 6940873a575bc060ce15e300: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940873a575bc060ce15e300 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940873a575bc060ce15e300 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940873a575bc060ce15e300 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:46,871 - lava_callback - INFO - Starting async job submit for node 69408758575bc060ce15e343 2025-12-17 22:17:46,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:46,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408758575bc060ce15e343 HTTP/1.1" 502 150 2025-12-17 22:17:46,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408758575bc060ce15e343'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:46,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408758575bc060ce15e343 2025-12-17 22:17:46,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408758575bc060ce15e343 HTTP/1.1" 502 150 2025-12-17 22:17:46,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408758575bc060ce15e343'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:46,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408732575bc060ce15e2ea 2025-12-17 22:17:46,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408732575bc060ce15e2ea HTTP/1.1" 502 150 2025-12-17 22:17:46,919 - lava_callback - ERROR - Error processing callback for node 69408732575bc060ce15e2ea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408732575bc060ce15e2ea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408732575bc060ce15e2ea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408732575bc060ce15e2ea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:46,920 - lava_callback - INFO - Starting async job submit for node 6940875b575bc060ce15e346 2025-12-17 22:17:46,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:46,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875b575bc060ce15e346 HTTP/1.1" 502 150 2025-12-17 22:17:46,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940875b575bc060ce15e346'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:46,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940875b575bc060ce15e346 2025-12-17 22:17:46,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875b575bc060ce15e346 HTTP/1.1" 502 150 2025-12-17 22:17:46,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940875b575bc060ce15e346'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:47,277 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408738575bc060ce15e2f6 2025-12-17 22:17:47,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408738575bc060ce15e2f6 HTTP/1.1" 502 150 2025-12-17 22:17:47,278 - lava_callback - ERROR - Error processing callback for node 69408738575bc060ce15e2f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408738575bc060ce15e2f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408738575bc060ce15e2f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408738575bc060ce15e2f6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:47,280 - lava_callback - INFO - Starting async job submit for node 6940875d575bc060ce15e349 2025-12-17 22:17:47,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:47,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875d575bc060ce15e349 HTTP/1.1" 502 150 2025-12-17 22:17:47,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940875d575bc060ce15e349'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:47,313 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940875d575bc060ce15e349 2025-12-17 22:17:47,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875d575bc060ce15e349 HTTP/1.1" 502 150 2025-12-17 22:17:47,314 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940875d575bc060ce15e349'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:47,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940873c575bc060ce15e307 2025-12-17 22:17:47,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873c575bc060ce15e307 HTTP/1.1" 502 150 2025-12-17 22:17:47,347 - lava_callback - ERROR - Error processing callback for node 6940873c575bc060ce15e307: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940873c575bc060ce15e307 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940873c575bc060ce15e307 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940873c575bc060ce15e307 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:47,349 - lava_callback - INFO - Starting async job submit for node 69408760575bc060ce15e34f 2025-12-17 22:17:47,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:47,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408760575bc060ce15e34f HTTP/1.1" 502 150 2025-12-17 22:17:47,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408760575bc060ce15e34f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:47,355 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408760575bc060ce15e34f 2025-12-17 22:17:47,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408760575bc060ce15e34f HTTP/1.1" 502 150 2025-12-17 22:17:47,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408760575bc060ce15e34f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:47,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408750575bc060ce15e332 2025-12-17 22:17:47,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408750575bc060ce15e332 HTTP/1.1" 502 150 2025-12-17 22:17:47,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408750575bc060ce15e332'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:47,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408752575bc060ce15e335 2025-12-17 22:17:47,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408752575bc060ce15e335 HTTP/1.1" 502 150 2025-12-17 22:17:47,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408752575bc060ce15e335'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:47,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408749575bc060ce15e325 2025-12-17 22:17:47,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408749575bc060ce15e325 HTTP/1.1" 502 150 2025-12-17 22:17:47,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408749575bc060ce15e325'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:47,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408756575bc060ce15e340 2025-12-17 22:17:47,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408756575bc060ce15e340 HTTP/1.1" 502 150 2025-12-17 22:17:47,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408756575bc060ce15e340'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:48,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408758575bc060ce15e343 2025-12-17 22:17:48,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408758575bc060ce15e343 HTTP/1.1" 502 150 2025-12-17 22:17:48,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408758575bc060ce15e343'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:48,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940875b575bc060ce15e346 2025-12-17 22:17:48,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875b575bc060ce15e346 HTTP/1.1" 502 150 2025-12-17 22:17:48,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940875b575bc060ce15e346'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:49,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940875d575bc060ce15e349 2025-12-17 22:17:49,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875d575bc060ce15e349 HTTP/1.1" 502 150 2025-12-17 22:17:49,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940875d575bc060ce15e349'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:49,356 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408760575bc060ce15e34f 2025-12-17 22:17:49,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408760575bc060ce15e34f HTTP/1.1" 502 150 2025-12-17 22:17:49,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408760575bc060ce15e34f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:50,320 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940873e575bc060ce15e310 2025-12-17 22:17:50,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873e575bc060ce15e310 HTTP/1.1" 502 150 2025-12-17 22:17:50,322 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940873e575bc060ce15e310'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:51,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408750575bc060ce15e332 2025-12-17 22:17:51,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408750575bc060ce15e332 HTTP/1.1" 502 150 2025-12-17 22:17:51,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408750575bc060ce15e332'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:51,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408752575bc060ce15e335 2025-12-17 22:17:51,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408752575bc060ce15e335 HTTP/1.1" 502 150 2025-12-17 22:17:51,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408752575bc060ce15e335'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:51,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408756575bc060ce15e340 2025-12-17 22:17:51,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408756575bc060ce15e340 HTTP/1.1" 502 150 2025-12-17 22:17:51,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408756575bc060ce15e340'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:51,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408741575bc060ce15e319 2025-12-17 22:17:51,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408741575bc060ce15e319 HTTP/1.1" 502 150 2025-12-17 22:17:51,891 - lava_callback - ERROR - Error processing callback for node 69408741575bc060ce15e319: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408741575bc060ce15e319 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408741575bc060ce15e319 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408741575bc060ce15e319 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:51,893 - lava_callback - INFO - Starting async job submit for node 6940875f575bc060ce15e34c 2025-12-17 22:17:51,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:51,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940874b575bc060ce15e328 2025-12-17 22:17:51,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940874b575bc060ce15e328 HTTP/1.1" 502 150 2025-12-17 22:17:51,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940874b575bc060ce15e328'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:51,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875f575bc060ce15e34c HTTP/1.1" 502 150 2025-12-17 22:17:51,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940875f575bc060ce15e34c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:51,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940875f575bc060ce15e34c 2025-12-17 22:17:51,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875f575bc060ce15e34c HTTP/1.1" 502 150 2025-12-17 22:17:51,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940875f575bc060ce15e34c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:52,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408758575bc060ce15e343 2025-12-17 22:17:52,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408758575bc060ce15e343 HTTP/1.1" 502 150 2025-12-17 22:17:52,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408758575bc060ce15e343'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:52,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940875b575bc060ce15e346 2025-12-17 22:17:52,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875b575bc060ce15e346 HTTP/1.1" 502 150 2025-12-17 22:17:52,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940875b575bc060ce15e346'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:53,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940875d575bc060ce15e349 2025-12-17 22:17:53,357 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408760575bc060ce15e34f 2025-12-17 22:17:53,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940875f575bc060ce15e34c 2025-12-17 22:17:53,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875f575bc060ce15e34c HTTP/1.1" 502 150 2025-12-17 22:17:53,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940875f575bc060ce15e34c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:54,326 - lava_callback - INFO - Received callback for node 694088fe575bc060ce15e8e2 INFO: 172.21.0.1:57280 - "POST /node/694088fe575bc060ce15e8e2 HTTP/1.0" 202 Accepted 2025-12-17 22:17:54,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940874e575bc060ce15e32b 2025-12-17 22:17:54,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940874e575bc060ce15e32b HTTP/1.1" 502 150 2025-12-17 22:17:54,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940874e575bc060ce15e32b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:55,000 - lava_callback - INFO - Received callback for node 694088ff575bc060ce15e8e5 INFO: 172.21.0.1:57288 - "POST /node/694088ff575bc060ce15e8e5 HTTP/1.0" 202 Accepted 2025-12-17 22:17:55,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408742575bc060ce15e31c HTTP/1.1" 502 150 2025-12-17 22:17:55,120 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408742575bc060ce15e31c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:55,672 - lava_callback - INFO - Received callback for node 69408901575bc060ce15e8e8 INFO: 172.21.0.1:57302 - "POST /node/69408901575bc060ce15e8e8 HTTP/1.0" 202 Accepted 2025-12-17 22:17:55,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c6651e3c24e95681d8d8 2025-12-17 22:17:55,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c6651e3c24e95681d8d8 HTTP/1.1" 502 150 2025-12-17 22:17:55,790 - lava_callback - ERROR - Error processing callback for node 6940c6651e3c24e95681d8d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c6651e3c24e95681d8d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c6651e3c24e95681d8d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c6651e3c24e95681d8d8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:55,792 - lava_callback - INFO - Starting async job submit for node 69408762575bc060ce15e352 2025-12-17 22:17:55,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:55,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408762575bc060ce15e352 HTTP/1.1" 502 150 2025-12-17 22:17:55,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408762575bc060ce15e352'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:55,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408762575bc060ce15e352 2025-12-17 22:17:55,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408762575bc060ce15e352 HTTP/1.1" 502 150 2025-12-17 22:17:55,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408762575bc060ce15e352'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:56,385 - lava_callback - INFO - Received callback for node 69408906575bc060ce15e8ef INFO: 172.21.0.1:57308 - "POST /node/69408906575bc060ce15e8ef HTTP/1.0" 202 Accepted INFO: 172.21.0.1:57320 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:17:57,065 - lava_callback - INFO - Received callback for node 69408907575bc060ce15e8f4 INFO: 172.21.0.1:57326 - "POST /node/69408907575bc060ce15e8f4 HTTP/1.0" 202 Accepted 2025-12-17 22:17:57,730 - lava_callback - INFO - Received callback for node 69408909575bc060ce15e8fb INFO: 172.21.0.1:57338 - "POST /node/69408909575bc060ce15e8fb HTTP/1.0" 202 Accepted 2025-12-17 22:17:57,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408762575bc060ce15e352 2025-12-17 22:17:57,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408762575bc060ce15e352 HTTP/1.1" 502 150 2025-12-17 22:17:57,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408762575bc060ce15e352'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:57,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940875f575bc060ce15e34c 2025-12-17 22:17:57,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875f575bc060ce15e34c HTTP/1.1" 502 150 2025-12-17 22:17:57,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940875f575bc060ce15e34c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:58,405 - lava_callback - INFO - Received callback for node 6940890a575bc060ce15e902 INFO: 172.21.0.1:57346 - "POST /node/6940890a575bc060ce15e902 HTTP/1.0" 202 Accepted 2025-12-17 22:17:58,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408733575bc060ce15e2ed 2025-12-17 22:17:58,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408733575bc060ce15e2ed HTTP/1.1" 502 150 2025-12-17 22:17:58,834 - lava_callback - ERROR - Error processing callback for node 69408733575bc060ce15e2ed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408733575bc060ce15e2ed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408733575bc060ce15e2ed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408733575bc060ce15e2ed (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:17:58,835 - lava_callback - INFO - Starting async job submit for node 69408766575bc060ce15e364 2025-12-17 22:17:58,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:17:58,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408766575bc060ce15e364 HTTP/1.1" 502 150 2025-12-17 22:17:58,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408766575bc060ce15e364'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:58,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408766575bc060ce15e364 2025-12-17 22:17:58,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408766575bc060ce15e364 HTTP/1.1" 502 150 2025-12-17 22:17:58,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408766575bc060ce15e364'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:59,082 - lava_callback - INFO - Received callback for node 6940890b575bc060ce15e909 INFO: 172.21.0.1:57360 - "POST /node/6940890b575bc060ce15e909 HTTP/1.0" 202 Accepted 2025-12-17 22:17:59,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408750575bc060ce15e332 2025-12-17 22:17:59,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408750575bc060ce15e332 HTTP/1.1" 502 150 2025-12-17 22:17:59,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408750575bc060ce15e332'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:59,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408752575bc060ce15e335 2025-12-17 22:17:59,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408752575bc060ce15e335 HTTP/1.1" 502 150 2025-12-17 22:17:59,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408752575bc060ce15e335'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:17:59,763 - lava_callback - INFO - Received callback for node 6940890d575bc060ce15e910 INFO: 172.21.0.1:50902 - "POST /node/6940890d575bc060ce15e910 HTTP/1.0" 202 Accepted 2025-12-17 22:17:59,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408756575bc060ce15e340 2025-12-17 22:17:59,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408756575bc060ce15e340 HTTP/1.1" 502 150 2025-12-17 22:17:59,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408756575bc060ce15e340'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:00,405 - lava_callback - INFO - Received callback for node 6940890e575bc060ce15e913 INFO: 172.21.0.1:50914 - "POST /node/6940890e575bc060ce15e913 HTTP/1.0" 202 Accepted 2025-12-17 22:18:00,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408766575bc060ce15e364 2025-12-17 22:18:00,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408766575bc060ce15e364 HTTP/1.1" 502 150 2025-12-17 22:18:00,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408766575bc060ce15e364'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:00,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408758575bc060ce15e343 2025-12-17 22:18:00,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408758575bc060ce15e343 HTTP/1.1" 502 150 2025-12-17 22:18:00,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408758575bc060ce15e343'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:00,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940875b575bc060ce15e346 2025-12-17 22:18:00,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875b575bc060ce15e346 HTTP/1.1" 502 150 2025-12-17 22:18:00,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940875b575bc060ce15e346'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:01,080 - lava_callback - INFO - Received callback for node 69408910575bc060ce15e919 INFO: 172.21.0.1:50928 - "POST /node/69408910575bc060ce15e919 HTTP/1.0" 202 Accepted 2025-12-17 22:18:01,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408762575bc060ce15e352 2025-12-17 22:18:01,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408762575bc060ce15e352 HTTP/1.1" 502 150 2025-12-17 22:18:01,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408762575bc060ce15e352'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50940 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:18:02,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408744575bc060ce15e31f 2025-12-17 22:18:02,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408744575bc060ce15e31f HTTP/1.1" 502 150 2025-12-17 22:18:02,733 - lava_callback - ERROR - Error processing callback for node 69408744575bc060ce15e31f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408744575bc060ce15e31f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408744575bc060ce15e31f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408744575bc060ce15e31f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:02,735 - lava_callback - INFO - Starting async job submit for node 69408767575bc060ce15e369 2025-12-17 22:18:02,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:02,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408767575bc060ce15e369 HTTP/1.1" 502 150 2025-12-17 22:18:02,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408767575bc060ce15e369'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:02,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408767575bc060ce15e369 2025-12-17 22:18:02,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408767575bc060ce15e369 HTTP/1.1" 502 150 2025-12-17 22:18:02,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408767575bc060ce15e369'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:03,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408749575bc060ce15e325 2025-12-17 22:18:03,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408749575bc060ce15e325 HTTP/1.1" 502 150 2025-12-17 22:18:03,776 - lava_callback - ERROR - Error processing callback for node 69408749575bc060ce15e325: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408749575bc060ce15e325 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408749575bc060ce15e325 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408749575bc060ce15e325 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:03,777 - lava_callback - INFO - Starting async job submit for node 6940876c575bc060ce15e3c5 2025-12-17 22:18:03,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:03,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940876c575bc060ce15e3c5 HTTP/1.1" 502 150 2025-12-17 22:18:03,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940876c575bc060ce15e3c5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:03,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940876c575bc060ce15e3c5 2025-12-17 22:18:03,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940876c575bc060ce15e3c5 HTTP/1.1" 502 150 2025-12-17 22:18:03,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940876c575bc060ce15e3c5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:04,262 - lava_callback - INFO - Received callback for node 69408914575bc060ce15e91f INFO: 172.21.0.1:50956 - "POST /node/69408914575bc060ce15e91f HTTP/1.0" 202 Accepted 2025-12-17 22:18:04,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408767575bc060ce15e369 2025-12-17 22:18:04,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408767575bc060ce15e369 HTTP/1.1" 502 150 2025-12-17 22:18:04,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408767575bc060ce15e369'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:04,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408766575bc060ce15e364 2025-12-17 22:18:04,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408766575bc060ce15e364 HTTP/1.1" 502 150 2025-12-17 22:18:04,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408766575bc060ce15e364'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:04,913 - lava_callback - INFO - Received callback for node 69408917575bc060ce15e922 INFO: 172.21.0.1:50960 - "POST /node/69408917575bc060ce15e922 HTTP/1.0" 202 Accepted 2025-12-17 22:18:05,553 - lava_callback - INFO - Received callback for node 69408919575bc060ce15e925 INFO: 172.21.0.1:50970 - "POST /node/69408919575bc060ce15e925 HTTP/1.0" 202 Accepted 2025-12-17 22:18:05,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940876c575bc060ce15e3c5 2025-12-17 22:18:05,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940876c575bc060ce15e3c5 HTTP/1.1" 502 150 2025-12-17 22:18:05,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940876c575bc060ce15e3c5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:05,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940875f575bc060ce15e34c 2025-12-17 22:18:05,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875f575bc060ce15e34c HTTP/1.1" 502 150 2025-12-17 22:18:05,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940875f575bc060ce15e34c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:06,324 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940873e575bc060ce15e310 2025-12-17 22:18:06,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940873e575bc060ce15e310 HTTP/1.1" 502 150 2025-12-17 22:18:06,333 - lava_callback - ERROR - Error processing callback for node 6940873e575bc060ce15e310: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940873e575bc060ce15e310 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940873e575bc060ce15e310 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940873e575bc060ce15e310 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:06,336 - lava_callback - INFO - Starting async job submit for node 69408770575bc060ce15e3cb 2025-12-17 22:18:06,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:06,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408770575bc060ce15e3cb HTTP/1.1" 502 150 2025-12-17 22:18:06,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408770575bc060ce15e3cb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:06,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408770575bc060ce15e3cb 2025-12-17 22:18:06,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408770575bc060ce15e3cb HTTP/1.1" 502 150 2025-12-17 22:18:06,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408770575bc060ce15e3cb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50982 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:18:07,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940874b575bc060ce15e328 2025-12-17 22:18:07,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940874b575bc060ce15e328 HTTP/1.1" 502 150 2025-12-17 22:18:07,902 - lava_callback - ERROR - Error processing callback for node 6940874b575bc060ce15e328: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940874b575bc060ce15e328 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940874b575bc060ce15e328 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940874b575bc060ce15e328 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:07,904 - lava_callback - INFO - Starting async job submit for node 69408763575bc060ce15e357 2025-12-17 22:18:07,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:07,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408763575bc060ce15e357 HTTP/1.1" 502 150 2025-12-17 22:18:07,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408763575bc060ce15e357'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:07,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408763575bc060ce15e357 2025-12-17 22:18:07,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408763575bc060ce15e357 HTTP/1.1" 502 150 2025-12-17 22:18:07,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408763575bc060ce15e357'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:08,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408770575bc060ce15e3cb 2025-12-17 22:18:08,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408770575bc060ce15e3cb HTTP/1.1" 502 150 2025-12-17 22:18:08,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408770575bc060ce15e3cb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:08,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875d575bc060ce15e349 HTTP/1.1" 502 150 2025-12-17 22:18:08,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940875d575bc060ce15e349'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:08,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408760575bc060ce15e34f HTTP/1.1" 502 150 2025-12-17 22:18:08,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408760575bc060ce15e34f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:08,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408767575bc060ce15e369 2025-12-17 22:18:08,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408767575bc060ce15e369 HTTP/1.1" 502 150 2025-12-17 22:18:08,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408767575bc060ce15e369'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:09,573 - lava_callback - INFO - Received callback for node 6940890f575bc060ce15e916 INFO: 172.21.0.1:50986 - "POST /node/6940890f575bc060ce15e916 HTTP/1.0" 202 Accepted 2025-12-17 22:18:09,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940876c575bc060ce15e3c5 2025-12-17 22:18:09,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940876c575bc060ce15e3c5 HTTP/1.1" 502 150 2025-12-17 22:18:09,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940876c575bc060ce15e3c5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:09,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408762575bc060ce15e352 2025-12-17 22:18:09,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408762575bc060ce15e352 HTTP/1.1" 502 150 2025-12-17 22:18:09,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408762575bc060ce15e352'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:09,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408763575bc060ce15e357 2025-12-17 22:18:09,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408763575bc060ce15e357 HTTP/1.1" 502 150 2025-12-17 22:18:09,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408763575bc060ce15e357'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:10,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940874e575bc060ce15e32b 2025-12-17 22:18:10,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940874e575bc060ce15e32b HTTP/1.1" 502 150 2025-12-17 22:18:10,483 - lava_callback - ERROR - Error processing callback for node 6940874e575bc060ce15e32b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940874e575bc060ce15e32b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940874e575bc060ce15e32b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940874e575bc060ce15e32b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:10,485 - lava_callback - INFO - Starting async job submit for node 69408771575bc060ce15e3ce 2025-12-17 22:18:10,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:10,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408771575bc060ce15e3ce HTTP/1.1" 502 150 2025-12-17 22:18:10,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408771575bc060ce15e3ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:10,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408771575bc060ce15e3ce 2025-12-17 22:18:10,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408771575bc060ce15e3ce HTTP/1.1" 502 150 2025-12-17 22:18:10,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408771575bc060ce15e3ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:11,120 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408742575bc060ce15e31c 2025-12-17 22:18:11,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408742575bc060ce15e31c HTTP/1.1" 502 150 2025-12-17 22:18:11,122 - lava_callback - ERROR - Error processing callback for node 69408742575bc060ce15e31c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408742575bc060ce15e31c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408742575bc060ce15e31c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408742575bc060ce15e31c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:11,124 - lava_callback - INFO - Starting async job submit for node 69408769575bc060ce15e38d 2025-12-17 22:18:11,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:11,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408769575bc060ce15e38d HTTP/1.1" 502 150 2025-12-17 22:18:11,138 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408769575bc060ce15e38d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:11,138 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408769575bc060ce15e38d 2025-12-17 22:18:11,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408769575bc060ce15e38d HTTP/1.1" 502 150 2025-12-17 22:18:11,139 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408769575bc060ce15e38d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:12,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408770575bc060ce15e3cb 2025-12-17 22:18:12,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408770575bc060ce15e3cb HTTP/1.1" 502 150 2025-12-17 22:18:12,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408770575bc060ce15e3cb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:12,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408771575bc060ce15e3ce 2025-12-17 22:18:12,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408771575bc060ce15e3ce HTTP/1.1" 502 150 2025-12-17 22:18:12,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408771575bc060ce15e3ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:12,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408766575bc060ce15e364 2025-12-17 22:18:12,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408766575bc060ce15e364 HTTP/1.1" 502 150 2025-12-17 22:18:12,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408766575bc060ce15e364'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:13,139 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408769575bc060ce15e38d 2025-12-17 22:18:13,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408769575bc060ce15e38d HTTP/1.1" 502 150 2025-12-17 22:18:13,142 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408769575bc060ce15e38d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:13,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408763575bc060ce15e357 2025-12-17 22:18:13,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408763575bc060ce15e357 HTTP/1.1" 502 150 2025-12-17 22:18:13,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408763575bc060ce15e357'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:14,251 - lava_callback - INFO - Received callback for node 69408912575bc060ce15e91c INFO: 172.21.0.1:36720 - "POST /node/69408912575bc060ce15e91c HTTP/1.0" 202 Accepted 2025-12-17 22:18:14,895 - lava_callback - INFO - Received callback for node 69408922575bc060ce15e97c INFO: 172.21.0.1:36728 - "POST /node/69408922575bc060ce15e97c HTTP/1.0" 202 Accepted INFO: 172.21.0.1:36744 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:18:15,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408750575bc060ce15e332 2025-12-17 22:18:15,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408750575bc060ce15e332 HTTP/1.1" 502 150 2025-12-17 22:18:15,508 - lava_callback - ERROR - Error processing callback for node 69408750575bc060ce15e332: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408750575bc060ce15e332 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408750575bc060ce15e332 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408750575bc060ce15e332 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:15,509 - lava_callback - INFO - Starting async job submit for node 69408772575bc060ce15e3d1 2025-12-17 22:18:15,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:15,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408772575bc060ce15e3d1 HTTP/1.1" 502 150 2025-12-17 22:18:15,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408772575bc060ce15e3d1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:15,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408772575bc060ce15e3d1 2025-12-17 22:18:15,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408772575bc060ce15e3d1 HTTP/1.1" 502 150 2025-12-17 22:18:15,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408772575bc060ce15e3d1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:15,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408752575bc060ce15e335 2025-12-17 22:18:15,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408752575bc060ce15e335 HTTP/1.1" 502 150 2025-12-17 22:18:15,702 - lava_callback - ERROR - Error processing callback for node 69408752575bc060ce15e335: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408752575bc060ce15e335 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408752575bc060ce15e335 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408752575bc060ce15e335 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:15,706 - lava_callback - INFO - Starting async job submit for node 69408774575bc060ce15e3d4 2025-12-17 22:18:15,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:15,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408774575bc060ce15e3d4 HTTP/1.1" 502 150 2025-12-17 22:18:15,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408774575bc060ce15e3d4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:15,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408774575bc060ce15e3d4 2025-12-17 22:18:15,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408774575bc060ce15e3d4 HTTP/1.1" 502 150 2025-12-17 22:18:15,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408774575bc060ce15e3d4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:15,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408756575bc060ce15e340 2025-12-17 22:18:15,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408756575bc060ce15e340 HTTP/1.1" 502 150 2025-12-17 22:18:15,779 - lava_callback - ERROR - Error processing callback for node 69408756575bc060ce15e340: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408756575bc060ce15e340 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408756575bc060ce15e340 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408756575bc060ce15e340 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:15,781 - lava_callback - INFO - Starting async job submit for node 69409094575bc060ce1605ed 2025-12-17 22:18:15,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:15,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409094575bc060ce1605ed HTTP/1.1" 502 150 2025-12-17 22:18:15,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409094575bc060ce1605ed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:15,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409094575bc060ce1605ed 2025-12-17 22:18:15,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409094575bc060ce1605ed HTTP/1.1" 502 150 2025-12-17 22:18:15,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409094575bc060ce1605ed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:16,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408771575bc060ce15e3ce 2025-12-17 22:18:16,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408771575bc060ce15e3ce HTTP/1.1" 502 150 2025-12-17 22:18:16,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408771575bc060ce15e3ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36756 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:18:16,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940875d575bc060ce15e349 2025-12-17 22:18:16,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408760575bc060ce15e34f 2025-12-17 22:18:16,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875d575bc060ce15e349 HTTP/1.1" 502 150 2025-12-17 22:18:16,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940875d575bc060ce15e349'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:16,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408760575bc060ce15e34f HTTP/1.1" 502 150 2025-12-17 22:18:16,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408760575bc060ce15e34f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:16,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408767575bc060ce15e369 2025-12-17 22:18:16,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408767575bc060ce15e369 HTTP/1.1" 502 150 2025-12-17 22:18:16,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408767575bc060ce15e369'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:16,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408758575bc060ce15e343 2025-12-17 22:18:16,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408758575bc060ce15e343 HTTP/1.1" 502 150 2025-12-17 22:18:16,885 - lava_callback - ERROR - Error processing callback for node 69408758575bc060ce15e343: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408758575bc060ce15e343 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408758575bc060ce15e343 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408758575bc060ce15e343 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:16,887 - lava_callback - INFO - Starting async job submit for node 6940876d575bc060ce15e3c8 2025-12-17 22:18:16,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:16,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940876d575bc060ce15e3c8 HTTP/1.1" 502 150 2025-12-17 22:18:16,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940876d575bc060ce15e3c8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:16,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940876d575bc060ce15e3c8 2025-12-17 22:18:16,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940876d575bc060ce15e3c8 HTTP/1.1" 502 150 2025-12-17 22:18:16,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940876d575bc060ce15e3c8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:16,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940875b575bc060ce15e346 2025-12-17 22:18:16,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875b575bc060ce15e346 HTTP/1.1" 502 150 2025-12-17 22:18:16,942 - lava_callback - ERROR - Error processing callback for node 6940875b575bc060ce15e346: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940875b575bc060ce15e346 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940875b575bc060ce15e346 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940875b575bc060ce15e346 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:16,943 - lava_callback - INFO - Starting async job submit for node 69408775575bc060ce15e3d7 2025-12-17 22:18:16,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:16,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408775575bc060ce15e3d7 HTTP/1.1" 502 150 2025-12-17 22:18:16,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408775575bc060ce15e3d7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:16,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408775575bc060ce15e3d7 2025-12-17 22:18:16,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408775575bc060ce15e3d7 HTTP/1.1" 502 150 2025-12-17 22:18:16,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408775575bc060ce15e3d7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:17,142 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408769575bc060ce15e38d 2025-12-17 22:18:17,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408769575bc060ce15e38d HTTP/1.1" 502 150 2025-12-17 22:18:17,143 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408769575bc060ce15e38d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:17,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408772575bc060ce15e3d1 2025-12-17 22:18:17,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408772575bc060ce15e3d1 HTTP/1.1" 502 150 2025-12-17 22:18:17,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408772575bc060ce15e3d1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:17,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408774575bc060ce15e3d4 2025-12-17 22:18:17,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408774575bc060ce15e3d4 HTTP/1.1" 502 150 2025-12-17 22:18:17,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408774575bc060ce15e3d4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:17,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940876c575bc060ce15e3c5 2025-12-17 22:18:17,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940876c575bc060ce15e3c5 HTTP/1.1" 502 150 2025-12-17 22:18:17,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940876c575bc060ce15e3c5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:17,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409094575bc060ce1605ed 2025-12-17 22:18:17,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409094575bc060ce1605ed HTTP/1.1" 502 150 2025-12-17 22:18:17,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409094575bc060ce1605ed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:18,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940876d575bc060ce15e3c8 2025-12-17 22:18:18,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940876d575bc060ce15e3c8 HTTP/1.1" 502 150 2025-12-17 22:18:18,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940876d575bc060ce15e3c8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:18,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408775575bc060ce15e3d7 2025-12-17 22:18:18,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408775575bc060ce15e3d7 HTTP/1.1" 502 150 2025-12-17 22:18:18,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408775575bc060ce15e3d7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:20,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408770575bc060ce15e3cb 2025-12-17 22:18:20,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408770575bc060ce15e3cb HTTP/1.1" 502 150 2025-12-17 22:18:20,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408770575bc060ce15e3cb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:21,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408772575bc060ce15e3d1 2025-12-17 22:18:21,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408772575bc060ce15e3d1 HTTP/1.1" 502 150 2025-12-17 22:18:21,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408772575bc060ce15e3d1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:21,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408774575bc060ce15e3d4 2025-12-17 22:18:21,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408774575bc060ce15e3d4 HTTP/1.1" 502 150 2025-12-17 22:18:21,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408774575bc060ce15e3d4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:21,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409094575bc060ce1605ed 2025-12-17 22:18:21,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409094575bc060ce1605ed HTTP/1.1" 502 150 2025-12-17 22:18:21,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409094575bc060ce1605ed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:21,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940875f575bc060ce15e34c 2025-12-17 22:18:21,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875f575bc060ce15e34c HTTP/1.1" 502 150 2025-12-17 22:18:21,908 - lava_callback - ERROR - Error processing callback for node 6940875f575bc060ce15e34c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940875f575bc060ce15e34c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940875f575bc060ce15e34c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940875f575bc060ce15e34c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:21,910 - lava_callback - INFO - Starting async job submit for node 69408776575bc060ce15e3de 2025-12-17 22:18:21,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:21,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408776575bc060ce15e3de HTTP/1.1" 502 150 2025-12-17 22:18:21,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408776575bc060ce15e3de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:21,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408776575bc060ce15e3de 2025-12-17 22:18:21,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408776575bc060ce15e3de HTTP/1.1" 502 150 2025-12-17 22:18:21,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408776575bc060ce15e3de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:21,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408763575bc060ce15e357 2025-12-17 22:18:21,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408763575bc060ce15e357 HTTP/1.1" 502 150 2025-12-17 22:18:21,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408763575bc060ce15e357'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:22,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940876d575bc060ce15e3c8 2025-12-17 22:18:22,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940876d575bc060ce15e3c8 HTTP/1.1" 502 150 2025-12-17 22:18:22,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940876d575bc060ce15e3c8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:22,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408775575bc060ce15e3d7 2025-12-17 22:18:22,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408775575bc060ce15e3d7 HTTP/1.1" 502 150 2025-12-17 22:18:22,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408775575bc060ce15e3d7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:23,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408776575bc060ce15e3de 2025-12-17 22:18:23,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408776575bc060ce15e3de HTTP/1.1" 502 150 2025-12-17 22:18:23,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408776575bc060ce15e3de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:24,262 - lava_callback - INFO - Received callback for node 6940891b575bc060ce15e93a INFO: 172.21.0.1:42212 - "POST /node/6940891b575bc060ce15e93a HTTP/1.0" 202 Accepted 2025-12-17 22:18:24,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408771575bc060ce15e3ce 2025-12-17 22:18:24,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408771575bc060ce15e3ce HTTP/1.1" 502 150 2025-12-17 22:18:24,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408771575bc060ce15e3ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:24,916 - lava_callback - INFO - Received callback for node 6940891e575bc060ce15e976 INFO: 172.21.0.1:42224 - "POST /node/6940891e575bc060ce15e976 HTTP/1.0" 202 Accepted 2025-12-17 22:18:25,144 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408769575bc060ce15e38d 2025-12-17 22:18:25,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408769575bc060ce15e38d HTTP/1.1" 502 150 2025-12-17 22:18:25,145 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408769575bc060ce15e38d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:25,588 - lava_callback - INFO - Received callback for node 69408920575bc060ce15e979 INFO: 172.21.0.1:42228 - "POST /node/69408920575bc060ce15e979 HTTP/1.0" 202 Accepted 2025-12-17 22:18:25,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408762575bc060ce15e352 2025-12-17 22:18:25,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408762575bc060ce15e352 HTTP/1.1" 502 150 2025-12-17 22:18:25,811 - lava_callback - ERROR - Error processing callback for node 69408762575bc060ce15e352: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408762575bc060ce15e352 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408762575bc060ce15e352 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408762575bc060ce15e352 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:25,813 - lava_callback - INFO - Starting async job submit for node 69408777575bc060ce15e3e3 2025-12-17 22:18:25,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:25,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408777575bc060ce15e3e3 HTTP/1.1" 502 150 2025-12-17 22:18:25,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408777575bc060ce15e3e3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:25,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408777575bc060ce15e3e3 2025-12-17 22:18:25,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408777575bc060ce15e3e3 HTTP/1.1" 502 150 2025-12-17 22:18:25,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408777575bc060ce15e3e3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:25,951 - lava_callback - INFO - Received callback for node 6940d3d1460bc8e2651126dc INFO: 172.21.0.1:42230 - "POST /node/6940d3d1460bc8e2651126dc HTTP/1.0" 202 Accepted 2025-12-17 22:18:26,236 - lava_callback - INFO - Received callback for node 6940892a575bc060ce15e995 INFO: 172.21.0.1:42236 - "POST /node/6940892a575bc060ce15e995 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:42244 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:18:27,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408777575bc060ce15e3e3 2025-12-17 22:18:27,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408777575bc060ce15e3e3 HTTP/1.1" 502 150 2025-12-17 22:18:27,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408777575bc060ce15e3e3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:27,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408776575bc060ce15e3de 2025-12-17 22:18:27,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408776575bc060ce15e3de HTTP/1.1" 502 150 2025-12-17 22:18:27,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408776575bc060ce15e3de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:28,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408766575bc060ce15e364 2025-12-17 22:18:28,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408766575bc060ce15e364 HTTP/1.1" 502 150 2025-12-17 22:18:28,871 - lava_callback - ERROR - Error processing callback for node 69408766575bc060ce15e364: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408766575bc060ce15e364 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408766575bc060ce15e364 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408766575bc060ce15e364 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:28,873 - lava_callback - INFO - Starting async job submit for node 69408779575bc060ce15e3e8 2025-12-17 22:18:28,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:28,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408779575bc060ce15e3e8 HTTP/1.1" 502 150 2025-12-17 22:18:28,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408779575bc060ce15e3e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:28,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408779575bc060ce15e3e8 2025-12-17 22:18:28,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408779575bc060ce15e3e8 HTTP/1.1" 502 150 2025-12-17 22:18:28,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408779575bc060ce15e3e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:29,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408772575bc060ce15e3d1 2025-12-17 22:18:29,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408772575bc060ce15e3d1 HTTP/1.1" 502 150 2025-12-17 22:18:29,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408772575bc060ce15e3d1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:29,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408774575bc060ce15e3d4 2025-12-17 22:18:29,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408774575bc060ce15e3d4 HTTP/1.1" 502 150 2025-12-17 22:18:29,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408774575bc060ce15e3d4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:29,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409094575bc060ce1605ed 2025-12-17 22:18:29,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409094575bc060ce1605ed HTTP/1.1" 502 150 2025-12-17 22:18:29,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409094575bc060ce1605ed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:30,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408779575bc060ce15e3e8 2025-12-17 22:18:30,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940876d575bc060ce15e3c8 2025-12-17 22:18:30,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940876d575bc060ce15e3c8 HTTP/1.1" 502 150 2025-12-17 22:18:30,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940876d575bc060ce15e3c8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:30,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408775575bc060ce15e3d7 2025-12-17 22:18:30,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408775575bc060ce15e3d7 HTTP/1.1" 502 150 2025-12-17 22:18:30,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408775575bc060ce15e3d7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:31,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408777575bc060ce15e3e3 2025-12-17 22:18:31,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408777575bc060ce15e3e3 HTTP/1.1" 502 150 2025-12-17 22:18:31,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408777575bc060ce15e3e3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:32,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940875d575bc060ce15e349 2025-12-17 22:18:32,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408760575bc060ce15e34f 2025-12-17 22:18:32,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940875d575bc060ce15e349 HTTP/1.1" 502 150 2025-12-17 22:18:32,692 - lava_callback - ERROR - Error processing callback for node 6940875d575bc060ce15e349: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940875d575bc060ce15e349 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940875d575bc060ce15e349 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940875d575bc060ce15e349 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:32,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408760575bc060ce15e34f HTTP/1.1" 502 150 2025-12-17 22:18:32,694 - lava_callback - ERROR - Error processing callback for node 69408760575bc060ce15e34f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408760575bc060ce15e34f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408760575bc060ce15e34f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408760575bc060ce15e34f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:32,693 - lava_callback - INFO - Starting async job submit for node 6940877a575bc060ce15e3eb 2025-12-17 22:18:32,695 - lava_callback - INFO - Starting async job submit for node 6940877b575bc060ce15e3ee 2025-12-17 22:18:32,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:32,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:32,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877b575bc060ce15e3ee HTTP/1.1" 502 150 2025-12-17 22:18:32,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940877b575bc060ce15e3ee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:32,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940877b575bc060ce15e3ee 2025-12-17 22:18:32,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877a575bc060ce15e3eb HTTP/1.1" 502 150 2025-12-17 22:18:32,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940877a575bc060ce15e3eb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:32,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940877a575bc060ce15e3eb 2025-12-17 22:18:32,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877b575bc060ce15e3ee HTTP/1.1" 502 150 2025-12-17 22:18:32,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940877b575bc060ce15e3ee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:32,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877a575bc060ce15e3eb HTTP/1.1" 502 150 2025-12-17 22:18:32,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940877a575bc060ce15e3eb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:32,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408767575bc060ce15e369 2025-12-17 22:18:32,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408767575bc060ce15e369 HTTP/1.1" 502 150 2025-12-17 22:18:32,752 - lava_callback - ERROR - Error processing callback for node 69408767575bc060ce15e369: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408767575bc060ce15e369 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408767575bc060ce15e369 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408767575bc060ce15e369 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:32,753 - lava_callback - INFO - Starting async job submit for node 6940877d575bc060ce15e3f1 2025-12-17 22:18:32,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:32,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877d575bc060ce15e3f1 HTTP/1.1" 502 150 2025-12-17 22:18:32,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940877d575bc060ce15e3f1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:32,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940877d575bc060ce15e3f1 2025-12-17 22:18:32,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877d575bc060ce15e3f1 HTTP/1.1" 502 150 2025-12-17 22:18:32,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940877d575bc060ce15e3f1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:33,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940876c575bc060ce15e3c5 2025-12-17 22:18:33,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940876c575bc060ce15e3c5 HTTP/1.1" 502 150 2025-12-17 22:18:33,795 - lava_callback - ERROR - Error processing callback for node 6940876c575bc060ce15e3c5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940876c575bc060ce15e3c5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940876c575bc060ce15e3c5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940876c575bc060ce15e3c5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:33,798 - lava_callback - INFO - Starting async job submit for node 69408781575bc060ce15e3f7 2025-12-17 22:18:33,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:33,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408781575bc060ce15e3f7 HTTP/1.1" 502 150 2025-12-17 22:18:33,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408781575bc060ce15e3f7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:33,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408781575bc060ce15e3f7 2025-12-17 22:18:33,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408781575bc060ce15e3f7 HTTP/1.1" 502 150 2025-12-17 22:18:33,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408781575bc060ce15e3f7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:34,271 - lava_callback - INFO - Received callback for node 69408925575bc060ce15e988 INFO: 172.21.0.1:38024 - "POST /node/69408925575bc060ce15e988 HTTP/1.0" 202 Accepted 2025-12-17 22:18:34,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940877b575bc060ce15e3ee 2025-12-17 22:18:34,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940877a575bc060ce15e3eb 2025-12-17 22:18:34,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877b575bc060ce15e3ee HTTP/1.1" 502 150 2025-12-17 22:18:34,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940877b575bc060ce15e3ee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:34,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877a575bc060ce15e3eb HTTP/1.1" 502 150 2025-12-17 22:18:34,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940877a575bc060ce15e3eb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:34,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940877d575bc060ce15e3f1 2025-12-17 22:18:34,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877d575bc060ce15e3f1 HTTP/1.1" 502 150 2025-12-17 22:18:34,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940877d575bc060ce15e3f1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:35,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408781575bc060ce15e3f7 2025-12-17 22:18:35,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408781575bc060ce15e3f7 HTTP/1.1" 502 150 2025-12-17 22:18:35,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408781575bc060ce15e3f7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:35,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408776575bc060ce15e3de 2025-12-17 22:18:35,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408776575bc060ce15e3de HTTP/1.1" 502 150 2025-12-17 22:18:35,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408776575bc060ce15e3de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:36,351 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408770575bc060ce15e3cb 2025-12-17 22:18:36,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408770575bc060ce15e3cb HTTP/1.1" 502 150 2025-12-17 22:18:36,352 - lava_callback - ERROR - Error processing callback for node 69408770575bc060ce15e3cb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408770575bc060ce15e3cb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408770575bc060ce15e3cb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408770575bc060ce15e3cb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:36,354 - lava_callback - INFO - Starting async job submit for node 69408784575bc060ce15e3fa 2025-12-17 22:18:36,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:36,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408784575bc060ce15e3fa HTTP/1.1" 502 150 2025-12-17 22:18:36,361 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408784575bc060ce15e3fa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:36,361 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408784575bc060ce15e3fa 2025-12-17 22:18:36,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408784575bc060ce15e3fa HTTP/1.1" 502 150 2025-12-17 22:18:36,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408784575bc060ce15e3fa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38038 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:18:37,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408763575bc060ce15e357 2025-12-17 22:18:37,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408763575bc060ce15e357 HTTP/1.1" 502 150 2025-12-17 22:18:37,939 - lava_callback - ERROR - Error processing callback for node 69408763575bc060ce15e357: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408763575bc060ce15e357 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408763575bc060ce15e357 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408763575bc060ce15e357 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:37,941 - lava_callback - INFO - Starting async job submit for node 69408787575bc060ce15e3fd 2025-12-17 22:18:37,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:37,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408787575bc060ce15e3fd HTTP/1.1" 502 150 2025-12-17 22:18:37,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408787575bc060ce15e3fd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:37,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408787575bc060ce15e3fd 2025-12-17 22:18:37,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408787575bc060ce15e3fd HTTP/1.1" 502 150 2025-12-17 22:18:37,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408787575bc060ce15e3fd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:38,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408784575bc060ce15e3fa 2025-12-17 22:18:38,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408784575bc060ce15e3fa HTTP/1.1" 502 150 2025-12-17 22:18:38,364 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408784575bc060ce15e3fa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:38,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940877b575bc060ce15e3ee 2025-12-17 22:18:38,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940877a575bc060ce15e3eb 2025-12-17 22:18:38,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877b575bc060ce15e3ee HTTP/1.1" 502 150 2025-12-17 22:18:38,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940877b575bc060ce15e3ee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:38,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877a575bc060ce15e3eb HTTP/1.1" 502 150 2025-12-17 22:18:38,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940877a575bc060ce15e3eb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:38,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940877d575bc060ce15e3f1 2025-12-17 22:18:38,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877d575bc060ce15e3f1 HTTP/1.1" 502 150 2025-12-17 22:18:38,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940877d575bc060ce15e3f1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:39,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408781575bc060ce15e3f7 2025-12-17 22:18:39,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408781575bc060ce15e3f7 HTTP/1.1" 502 150 2025-12-17 22:18:39,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408781575bc060ce15e3f7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:39,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408777575bc060ce15e3e3 2025-12-17 22:18:39,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408777575bc060ce15e3e3 HTTP/1.1" 502 150 2025-12-17 22:18:39,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408777575bc060ce15e3e3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:39,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408787575bc060ce15e3fd 2025-12-17 22:18:39,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408787575bc060ce15e3fd HTTP/1.1" 502 150 2025-12-17 22:18:39,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408787575bc060ce15e3fd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:40,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408771575bc060ce15e3ce 2025-12-17 22:18:40,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408771575bc060ce15e3ce HTTP/1.1" 502 150 2025-12-17 22:18:40,503 - lava_callback - ERROR - Error processing callback for node 69408771575bc060ce15e3ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408771575bc060ce15e3ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408771575bc060ce15e3ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408771575bc060ce15e3ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:40,505 - lava_callback - INFO - Starting async job submit for node 69408789575bc060ce15e400 2025-12-17 22:18:40,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:40,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408789575bc060ce15e400 HTTP/1.1" 502 150 2025-12-17 22:18:40,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408789575bc060ce15e400'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:40,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408789575bc060ce15e400 2025-12-17 22:18:40,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408789575bc060ce15e400 HTTP/1.1" 502 150 2025-12-17 22:18:40,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408789575bc060ce15e400'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:41,145 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408769575bc060ce15e38d 2025-12-17 22:18:42,364 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408784575bc060ce15e3fa 2025-12-17 22:18:42,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408784575bc060ce15e3fa HTTP/1.1" 502 150 2025-12-17 22:18:42,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408784575bc060ce15e3fa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:42,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408789575bc060ce15e400 2025-12-17 22:18:42,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408789575bc060ce15e400 HTTP/1.1" 502 150 2025-12-17 22:18:42,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408789575bc060ce15e400'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:43,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408787575bc060ce15e3fd 2025-12-17 22:18:43,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408787575bc060ce15e3fd HTTP/1.1" 502 150 2025-12-17 22:18:43,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408787575bc060ce15e3fd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:44,262 - lava_callback - INFO - Received callback for node 69408930575bc060ce15e99a INFO: 172.21.0.1:55910 - "POST /node/69408930575bc060ce15e99a HTTP/1.0" 202 Accepted 2025-12-17 22:18:44,895 - lava_callback - INFO - Received callback for node 69408933575bc060ce15e99d INFO: 172.21.0.1:55920 - "POST /node/69408933575bc060ce15e99d HTTP/1.0" 202 Accepted 2025-12-17 22:18:45,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408772575bc060ce15e3d1 2025-12-17 22:18:45,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408772575bc060ce15e3d1 HTTP/1.1" 502 150 2025-12-17 22:18:45,523 - lava_callback - ERROR - Error processing callback for node 69408772575bc060ce15e3d1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408772575bc060ce15e3d1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408772575bc060ce15e3d1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408772575bc060ce15e3d1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:45,525 - lava_callback - INFO - Starting async job submit for node 6940879a575bc060ce15e41e 2025-12-17 22:18:45,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:45,530 - lava_callback - INFO - Received callback for node 69408935575bc060ce15e9a0 INFO: 172.21.0.1:55936 - "POST /node/69408935575bc060ce15e9a0 HTTP/1.0" 202 Accepted 2025-12-17 22:18:45,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879a575bc060ce15e41e HTTP/1.1" 502 150 2025-12-17 22:18:45,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879a575bc060ce15e41e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:45,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879a575bc060ce15e41e 2025-12-17 22:18:45,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879a575bc060ce15e41e HTTP/1.1" 502 150 2025-12-17 22:18:45,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879a575bc060ce15e41e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:45,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408774575bc060ce15e3d4 2025-12-17 22:18:45,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408774575bc060ce15e3d4 HTTP/1.1" 502 150 2025-12-17 22:18:45,728 - lava_callback - ERROR - Error processing callback for node 69408774575bc060ce15e3d4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408774575bc060ce15e3d4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408774575bc060ce15e3d4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408774575bc060ce15e3d4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:45,729 - lava_callback - INFO - Starting async job submit for node 6940878b575bc060ce15e405 2025-12-17 22:18:45,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:45,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878b575bc060ce15e405 HTTP/1.1" 502 150 2025-12-17 22:18:45,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940878b575bc060ce15e405'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:45,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940878b575bc060ce15e405 2025-12-17 22:18:45,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878b575bc060ce15e405 HTTP/1.1" 502 150 2025-12-17 22:18:45,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940878b575bc060ce15e405'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:45,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409094575bc060ce1605ed 2025-12-17 22:18:45,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409094575bc060ce1605ed HTTP/1.1" 502 150 2025-12-17 22:18:45,858 - lava_callback - ERROR - Error processing callback for node 69409094575bc060ce1605ed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409094575bc060ce1605ed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409094575bc060ce1605ed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409094575bc060ce1605ed (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:45,860 - lava_callback - INFO - Starting async job submit for node 6940878d575bc060ce15e408 2025-12-17 22:18:45,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:45,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878d575bc060ce15e408 HTTP/1.1" 502 150 2025-12-17 22:18:45,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940878d575bc060ce15e408'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:45,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940878d575bc060ce15e408 2025-12-17 22:18:45,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878d575bc060ce15e408 HTTP/1.1" 502 150 2025-12-17 22:18:45,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940878d575bc060ce15e408'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:46,160 - lava_callback - INFO - Received callback for node 6940893b575bc060ce15e9a9 INFO: 172.21.0.1:55940 - "POST /node/6940893b575bc060ce15e9a9 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:55952 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:18:46,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408789575bc060ce15e400 2025-12-17 22:18:46,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408789575bc060ce15e400 HTTP/1.1" 502 150 2025-12-17 22:18:46,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408789575bc060ce15e400'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:46,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940877b575bc060ce15e3ee 2025-12-17 22:18:46,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940877a575bc060ce15e3eb 2025-12-17 22:18:46,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877b575bc060ce15e3ee HTTP/1.1" 502 150 2025-12-17 22:18:46,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940877b575bc060ce15e3ee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:46,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877a575bc060ce15e3eb HTTP/1.1" 502 150 2025-12-17 22:18:46,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940877a575bc060ce15e3eb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:46,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940877d575bc060ce15e3f1 2025-12-17 22:18:46,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877d575bc060ce15e3f1 HTTP/1.1" 502 150 2025-12-17 22:18:46,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940877d575bc060ce15e3f1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:46,791 - lava_callback - INFO - Received callback for node 6940893d575bc060ce15e9ac INFO: 172.21.0.1:55962 - "POST /node/6940893d575bc060ce15e9ac HTTP/1.0" 202 Accepted 2025-12-17 22:18:46,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940876d575bc060ce15e3c8 2025-12-17 22:18:46,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940876d575bc060ce15e3c8 HTTP/1.1" 502 150 2025-12-17 22:18:46,902 - lava_callback - ERROR - Error processing callback for node 6940876d575bc060ce15e3c8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940876d575bc060ce15e3c8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940876d575bc060ce15e3c8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940876d575bc060ce15e3c8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:46,903 - lava_callback - INFO - Starting async job submit for node 6940878f575bc060ce15e40b 2025-12-17 22:18:46,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:46,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878f575bc060ce15e40b HTTP/1.1" 502 150 2025-12-17 22:18:46,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940878f575bc060ce15e40b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:46,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940878f575bc060ce15e40b 2025-12-17 22:18:46,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878f575bc060ce15e40b HTTP/1.1" 502 150 2025-12-17 22:18:46,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940878f575bc060ce15e40b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:46,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408775575bc060ce15e3d7 2025-12-17 22:18:46,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408775575bc060ce15e3d7 HTTP/1.1" 502 150 2025-12-17 22:18:46,974 - lava_callback - ERROR - Error processing callback for node 69408775575bc060ce15e3d7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408775575bc060ce15e3d7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408775575bc060ce15e3d7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408775575bc060ce15e3d7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:46,976 - lava_callback - INFO - Starting async job submit for node 69408791575bc060ce15e40e 2025-12-17 22:18:46,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:46,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408791575bc060ce15e40e HTTP/1.1" 502 150 2025-12-17 22:18:46,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408791575bc060ce15e40e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:46,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408791575bc060ce15e40e 2025-12-17 22:18:46,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408791575bc060ce15e40e HTTP/1.1" 502 150 2025-12-17 22:18:46,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408791575bc060ce15e40e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:47,439 - lava_callback - INFO - Received callback for node 69408940575bc060ce15e9af INFO: 172.21.0.1:55972 - "POST /node/69408940575bc060ce15e9af HTTP/1.0" 202 Accepted 2025-12-17 22:18:47,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879a575bc060ce15e41e 2025-12-17 22:18:47,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879a575bc060ce15e41e HTTP/1.1" 502 150 2025-12-17 22:18:47,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879a575bc060ce15e41e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:47,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940878b575bc060ce15e405 2025-12-17 22:18:47,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878b575bc060ce15e405 HTTP/1.1" 502 150 2025-12-17 22:18:47,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940878b575bc060ce15e405'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:47,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408781575bc060ce15e3f7 2025-12-17 22:18:47,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408781575bc060ce15e3f7 HTTP/1.1" 502 150 2025-12-17 22:18:47,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408781575bc060ce15e3f7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:47,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940878d575bc060ce15e408 2025-12-17 22:18:47,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878d575bc060ce15e408 HTTP/1.1" 502 150 2025-12-17 22:18:47,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940878d575bc060ce15e408'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:48,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940878f575bc060ce15e40b 2025-12-17 22:18:48,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878f575bc060ce15e40b HTTP/1.1" 502 150 2025-12-17 22:18:48,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940878f575bc060ce15e40b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:48,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408791575bc060ce15e40e 2025-12-17 22:18:48,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408791575bc060ce15e40e HTTP/1.1" 502 150 2025-12-17 22:18:48,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408791575bc060ce15e40e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:50,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408784575bc060ce15e3fa 2025-12-17 22:18:50,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408784575bc060ce15e3fa HTTP/1.1" 502 150 2025-12-17 22:18:50,367 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408784575bc060ce15e3fa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:51,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879a575bc060ce15e41e 2025-12-17 22:18:51,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879a575bc060ce15e41e HTTP/1.1" 502 150 2025-12-17 22:18:51,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879a575bc060ce15e41e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:51,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940878b575bc060ce15e405 2025-12-17 22:18:51,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878b575bc060ce15e405 HTTP/1.1" 502 150 2025-12-17 22:18:51,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940878b575bc060ce15e405'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:51,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940878d575bc060ce15e408 2025-12-17 22:18:51,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878d575bc060ce15e408 HTTP/1.1" 502 150 2025-12-17 22:18:51,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940878d575bc060ce15e408'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:51,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408776575bc060ce15e3de 2025-12-17 22:18:51,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408776575bc060ce15e3de HTTP/1.1" 502 150 2025-12-17 22:18:51,930 - lava_callback - ERROR - Error processing callback for node 69408776575bc060ce15e3de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408776575bc060ce15e3de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408776575bc060ce15e3de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408776575bc060ce15e3de (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:51,933 - lava_callback - INFO - Starting async job submit for node 69409139575bc060ce1607c1 2025-12-17 22:18:51,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408787575bc060ce15e3fd 2025-12-17 22:18:51,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408787575bc060ce15e3fd HTTP/1.1" 502 150 2025-12-17 22:18:51,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408787575bc060ce15e3fd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:52,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:52,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409139575bc060ce1607c1 HTTP/1.1" 502 150 2025-12-17 22:18:52,054 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409139575bc060ce1607c1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:52,054 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409139575bc060ce1607c1 2025-12-17 22:18:52,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409139575bc060ce1607c1 HTTP/1.1" 502 150 2025-12-17 22:18:52,055 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409139575bc060ce1607c1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:52,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940878f575bc060ce15e40b 2025-12-17 22:18:52,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878f575bc060ce15e40b HTTP/1.1" 502 150 2025-12-17 22:18:52,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940878f575bc060ce15e40b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:52,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408791575bc060ce15e40e 2025-12-17 22:18:52,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408791575bc060ce15e40e HTTP/1.1" 502 150 2025-12-17 22:18:52,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408791575bc060ce15e40e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:54,055 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409139575bc060ce1607c1 2025-12-17 22:18:54,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409139575bc060ce1607c1 HTTP/1.1" 502 150 2025-12-17 22:18:54,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409139575bc060ce1607c1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:54,280 - lava_callback - INFO - Received callback for node 6940894c575bc060ce15e9d0 INFO: 172.21.0.1:49726 - "POST /node/6940894c575bc060ce15e9d0 HTTP/1.0" 202 Accepted 2025-12-17 22:18:54,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408789575bc060ce15e400 2025-12-17 22:18:54,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408789575bc060ce15e400 HTTP/1.1" 502 150 2025-12-17 22:18:54,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408789575bc060ce15e400'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:55,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408777575bc060ce15e3e3 2025-12-17 22:18:55,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408777575bc060ce15e3e3 HTTP/1.1" 502 150 2025-12-17 22:18:55,836 - lava_callback - ERROR - Error processing callback for node 69408777575bc060ce15e3e3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408777575bc060ce15e3e3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408777575bc060ce15e3e3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408777575bc060ce15e3e3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:55,839 - lava_callback - INFO - Starting async job submit for node 69408795575bc060ce15e414 2025-12-17 22:18:55,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:55,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408795575bc060ce15e414 HTTP/1.1" 502 150 2025-12-17 22:18:55,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408795575bc060ce15e414'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:55,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408795575bc060ce15e414 2025-12-17 22:18:55,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408795575bc060ce15e414 HTTP/1.1" 502 150 2025-12-17 22:18:55,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408795575bc060ce15e414'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49734 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:18:56,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408769575bc060ce15e38d HTTP/1.1" 502 150 2025-12-17 22:18:56,560 - lava_callback - ERROR - Error processing callback for node 69408769575bc060ce15e38d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408769575bc060ce15e38d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408769575bc060ce15e38d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408769575bc060ce15e38d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:18:56,563 - lava_callback - INFO - Starting async job submit for node 69408798575bc060ce15e41b 2025-12-17 22:18:56,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:18:56,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408798575bc060ce15e41b HTTP/1.1" 502 150 2025-12-17 22:18:56,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408798575bc060ce15e41b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:56,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408798575bc060ce15e41b 2025-12-17 22:18:56,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408798575bc060ce15e41b HTTP/1.1" 502 150 2025-12-17 22:18:56,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408798575bc060ce15e41b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:57,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408795575bc060ce15e414 2025-12-17 22:18:57,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408795575bc060ce15e414 HTTP/1.1" 502 150 2025-12-17 22:18:57,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408795575bc060ce15e414'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:58,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409139575bc060ce1607c1 2025-12-17 22:18:58,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408798575bc060ce15e41b 2025-12-17 22:18:58,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408798575bc060ce15e41b HTTP/1.1" 502 150 2025-12-17 22:18:58,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408798575bc060ce15e41b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:59,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879a575bc060ce15e41e 2025-12-17 22:18:59,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879a575bc060ce15e41e HTTP/1.1" 502 150 2025-12-17 22:18:59,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879a575bc060ce15e41e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:59,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940878b575bc060ce15e405 2025-12-17 22:18:59,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878b575bc060ce15e405 HTTP/1.1" 502 150 2025-12-17 22:18:59,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940878b575bc060ce15e405'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:18:59,763 - lava_callback - INFO - Received callback for node 69408942575bc060ce15e9b2 INFO: 172.21.0.1:34484 - "POST /node/69408942575bc060ce15e9b2 HTTP/1.0" 202 Accepted 2025-12-17 22:18:59,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940878d575bc060ce15e408 2025-12-17 22:18:59,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878d575bc060ce15e408 HTTP/1.1" 502 150 2025-12-17 22:18:59,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940878d575bc060ce15e408'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:00,605 - lava_callback - INFO - Received callback for node 69408946575bc060ce15e9c0 INFO: 172.21.0.1:34494 - "POST /node/69408946575bc060ce15e9c0 HTTP/1.0" 202 Accepted 2025-12-17 22:19:00,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940878f575bc060ce15e40b 2025-12-17 22:19:00,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878f575bc060ce15e40b HTTP/1.1" 502 150 2025-12-17 22:19:00,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940878f575bc060ce15e40b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:00,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408791575bc060ce15e40e 2025-12-17 22:19:00,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408791575bc060ce15e40e HTTP/1.1" 502 150 2025-12-17 22:19:00,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408791575bc060ce15e40e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:01,362 - lava_callback - INFO - Received callback for node 69408947575bc060ce15e9c3 INFO: 172.21.0.1:34500 - "POST /node/69408947575bc060ce15e9c3 HTTP/1.0" 202 Accepted 2025-12-17 22:19:01,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408779575bc060ce15e3e8 HTTP/1.1" 502 150 2025-12-17 22:19:01,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408779575bc060ce15e3e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:01,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408795575bc060ce15e414 2025-12-17 22:19:01,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408795575bc060ce15e414 HTTP/1.1" 502 150 2025-12-17 22:19:01,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408795575bc060ce15e414'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34502 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:19:02,511 - lava_callback - INFO - Received callback for node 69408718575bc060ce15e273 INFO: 172.21.0.1:34510 - "POST /node/69408718575bc060ce15e273 HTTP/1.0" 202 Accepted 2025-12-17 22:19:02,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408798575bc060ce15e41b 2025-12-17 22:19:02,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408798575bc060ce15e41b HTTP/1.1" 502 150 2025-12-17 22:19:02,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408798575bc060ce15e41b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:02,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940877b575bc060ce15e3ee 2025-12-17 22:19:02,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940877a575bc060ce15e3eb 2025-12-17 22:19:02,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877b575bc060ce15e3ee HTTP/1.1" 502 150 2025-12-17 22:19:02,710 - lava_callback - ERROR - Error processing callback for node 6940877b575bc060ce15e3ee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940877b575bc060ce15e3ee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940877b575bc060ce15e3ee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940877b575bc060ce15e3ee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:02,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877a575bc060ce15e3eb HTTP/1.1" 502 150 2025-12-17 22:19:02,713 - lava_callback - ERROR - Error processing callback for node 6940877a575bc060ce15e3eb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940877a575bc060ce15e3eb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940877a575bc060ce15e3eb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940877a575bc060ce15e3eb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:02,713 - lava_callback - INFO - Starting async job submit for node 6940879c575bc060ce15e424 2025-12-17 22:19:02,715 - lava_callback - INFO - Starting async job submit for node 694087a1575bc060ce15e437 2025-12-17 22:19:02,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:02,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:02,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a1575bc060ce15e437 HTTP/1.1" 502 150 2025-12-17 22:19:02,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a1575bc060ce15e437'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:02,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879c575bc060ce15e424 HTTP/1.1" 502 150 2025-12-17 22:19:02,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a1575bc060ce15e437 2025-12-17 22:19:02,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879c575bc060ce15e424'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:02,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879c575bc060ce15e424 2025-12-17 22:19:02,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a1575bc060ce15e437 HTTP/1.1" 502 150 2025-12-17 22:19:02,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a1575bc060ce15e437'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:02,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879c575bc060ce15e424 HTTP/1.1" 502 150 2025-12-17 22:19:02,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879c575bc060ce15e424'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:02,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940877d575bc060ce15e3f1 2025-12-17 22:19:02,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940877d575bc060ce15e3f1 HTTP/1.1" 502 150 2025-12-17 22:19:02,771 - lava_callback - ERROR - Error processing callback for node 6940877d575bc060ce15e3f1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940877d575bc060ce15e3f1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940877d575bc060ce15e3f1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940877d575bc060ce15e3f1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:02,773 - lava_callback - INFO - Starting async job submit for node 694087a4575bc060ce15e43a 2025-12-17 22:19:02,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:02,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a4575bc060ce15e43a HTTP/1.1" 502 150 2025-12-17 22:19:02,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a4575bc060ce15e43a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:02,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a4575bc060ce15e43a 2025-12-17 22:19:02,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a4575bc060ce15e43a HTTP/1.1" 502 150 2025-12-17 22:19:02,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a4575bc060ce15e43a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:03,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408781575bc060ce15e3f7 2025-12-17 22:19:03,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408781575bc060ce15e3f7 HTTP/1.1" 502 150 2025-12-17 22:19:03,824 - lava_callback - ERROR - Error processing callback for node 69408781575bc060ce15e3f7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408781575bc060ce15e3f7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408781575bc060ce15e3f7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408781575bc060ce15e3f7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:03,826 - lava_callback - INFO - Starting async job submit for node 694087a6575bc060ce15e441 2025-12-17 22:19:03,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:03,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a6575bc060ce15e441 HTTP/1.1" 502 150 2025-12-17 22:19:03,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a6575bc060ce15e441'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:03,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a6575bc060ce15e441 2025-12-17 22:19:03,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a6575bc060ce15e441 HTTP/1.1" 502 150 2025-12-17 22:19:03,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a6575bc060ce15e441'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:04,295 - lava_callback - INFO - Received callback for node 69408943575bc060ce15e9b8 INFO: 172.21.0.1:34512 - "POST /node/69408943575bc060ce15e9b8 HTTP/1.0" 202 Accepted 2025-12-17 22:19:04,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a1575bc060ce15e437 2025-12-17 22:19:04,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879c575bc060ce15e424 2025-12-17 22:19:04,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a1575bc060ce15e437 HTTP/1.1" 502 150 2025-12-17 22:19:04,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a1575bc060ce15e437'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:04,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879c575bc060ce15e424 HTTP/1.1" 502 150 2025-12-17 22:19:04,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879c575bc060ce15e424'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:04,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a4575bc060ce15e43a 2025-12-17 22:19:04,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a4575bc060ce15e43a HTTP/1.1" 502 150 2025-12-17 22:19:04,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a4575bc060ce15e43a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:04,929 - lava_callback - INFO - Received callback for node 69408948575bc060ce15e9c6 INFO: 172.21.0.1:34520 - "POST /node/69408948575bc060ce15e9c6 HTTP/1.0" 202 Accepted 2025-12-17 22:19:05,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408779575bc060ce15e3e8 2025-12-17 22:19:05,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408779575bc060ce15e3e8 HTTP/1.1" 502 150 2025-12-17 22:19:05,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408779575bc060ce15e3e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:05,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a6575bc060ce15e441 2025-12-17 22:19:05,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a6575bc060ce15e441 HTTP/1.1" 502 150 2025-12-17 22:19:05,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a6575bc060ce15e441'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:06,367 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408784575bc060ce15e3fa 2025-12-17 22:19:06,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408784575bc060ce15e3fa HTTP/1.1" 502 150 2025-12-17 22:19:06,368 - lava_callback - ERROR - Error processing callback for node 69408784575bc060ce15e3fa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408784575bc060ce15e3fa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408784575bc060ce15e3fa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408784575bc060ce15e3fa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:06,371 - lava_callback - INFO - Starting async job submit for node 6940879b575bc060ce15e421 2025-12-17 22:19:06,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:06,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879b575bc060ce15e421 HTTP/1.1" 502 150 2025-12-17 22:19:06,379 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879b575bc060ce15e421'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:06,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879b575bc060ce15e421 2025-12-17 22:19:06,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879b575bc060ce15e421 HTTP/1.1" 502 150 2025-12-17 22:19:06,379 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879b575bc060ce15e421'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34524 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:19:07,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408787575bc060ce15e3fd 2025-12-17 22:19:07,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408787575bc060ce15e3fd HTTP/1.1" 502 150 2025-12-17 22:19:07,971 - lava_callback - ERROR - Error processing callback for node 69408787575bc060ce15e3fd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408787575bc060ce15e3fd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408787575bc060ce15e3fd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408787575bc060ce15e3fd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:07,973 - lava_callback - INFO - Starting async job submit for node 6940879f575bc060ce15e42e 2025-12-17 22:19:07,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:07,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879f575bc060ce15e42e HTTP/1.1" 502 150 2025-12-17 22:19:07,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879f575bc060ce15e42e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:07,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879f575bc060ce15e42e 2025-12-17 22:19:07,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879f575bc060ce15e42e HTTP/1.1" 502 150 2025-12-17 22:19:07,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879f575bc060ce15e42e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:08,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879b575bc060ce15e421 2025-12-17 22:19:08,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879b575bc060ce15e421 HTTP/1.1" 502 150 2025-12-17 22:19:08,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879b575bc060ce15e421'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:08,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a1575bc060ce15e437 2025-12-17 22:19:08,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879c575bc060ce15e424 2025-12-17 22:19:08,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a1575bc060ce15e437 HTTP/1.1" 502 150 2025-12-17 22:19:08,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a1575bc060ce15e437'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:08,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879c575bc060ce15e424 HTTP/1.1" 502 150 2025-12-17 22:19:08,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879c575bc060ce15e424'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:08,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a4575bc060ce15e43a 2025-12-17 22:19:08,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a4575bc060ce15e43a HTTP/1.1" 502 150 2025-12-17 22:19:08,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a4575bc060ce15e43a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:09,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a6575bc060ce15e441 2025-12-17 22:19:09,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a6575bc060ce15e441 HTTP/1.1" 502 150 2025-12-17 22:19:09,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a6575bc060ce15e441'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:09,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408795575bc060ce15e414 2025-12-17 22:19:09,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408795575bc060ce15e414 HTTP/1.1" 502 150 2025-12-17 22:19:09,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408795575bc060ce15e414'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:09,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879f575bc060ce15e42e 2025-12-17 22:19:09,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879f575bc060ce15e42e HTTP/1.1" 502 150 2025-12-17 22:19:09,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879f575bc060ce15e42e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:10,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408789575bc060ce15e400 2025-12-17 22:19:10,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408789575bc060ce15e400 HTTP/1.1" 502 150 2025-12-17 22:19:10,528 - lava_callback - ERROR - Error processing callback for node 69408789575bc060ce15e400: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408789575bc060ce15e400 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408789575bc060ce15e400 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408789575bc060ce15e400 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:10,530 - lava_callback - INFO - Starting async job submit for node 694087a9575bc060ce15e484 2025-12-17 22:19:10,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:10,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a9575bc060ce15e484 HTTP/1.1" 502 150 2025-12-17 22:19:10,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a9575bc060ce15e484'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:10,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a9575bc060ce15e484 2025-12-17 22:19:10,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a9575bc060ce15e484 HTTP/1.1" 502 150 2025-12-17 22:19:10,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a9575bc060ce15e484'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:10,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408798575bc060ce15e41b 2025-12-17 22:19:10,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408798575bc060ce15e41b HTTP/1.1" 502 150 2025-12-17 22:19:10,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408798575bc060ce15e41b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:12,381 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879b575bc060ce15e421 2025-12-17 22:19:12,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879b575bc060ce15e421 HTTP/1.1" 502 150 2025-12-17 22:19:12,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879b575bc060ce15e421'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:12,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a9575bc060ce15e484 2025-12-17 22:19:12,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a9575bc060ce15e484 HTTP/1.1" 502 150 2025-12-17 22:19:12,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a9575bc060ce15e484'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:13,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409139575bc060ce1607c1 HTTP/1.1" 502 150 2025-12-17 22:19:13,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409139575bc060ce1607c1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:13,611 - lava_callback - INFO - Received callback for node 69408952575bc060ce15e9d9 INFO: 172.21.0.1:53172 - "POST /node/69408952575bc060ce15e9d9 HTTP/1.0" 202 Accepted 2025-12-17 22:19:13,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408779575bc060ce15e3e8 2025-12-17 22:19:13,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408779575bc060ce15e3e8 HTTP/1.1" 502 150 2025-12-17 22:19:13,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408779575bc060ce15e3e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:13,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879f575bc060ce15e42e 2025-12-17 22:19:13,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879f575bc060ce15e42e HTTP/1.1" 502 150 2025-12-17 22:19:13,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879f575bc060ce15e42e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:14,220 - lava_callback - INFO - Received callback for node 69408954575bc060ce15e9dc INFO: 172.21.0.1:53186 - "POST /node/69408954575bc060ce15e9dc HTTP/1.0" 202 Accepted 2025-12-17 22:19:14,305 - lava_callback - INFO - Received callback for node 69408951575bc060ce15e9d6 INFO: 172.21.0.1:53198 - "POST /node/69408951575bc060ce15e9d6 HTTP/1.0" 202 Accepted 2025-12-17 22:19:14,970 - lava_callback - INFO - Received callback for node 6940894f575bc060ce15e9d3 INFO: 172.21.0.1:53212 - "POST /node/6940894f575bc060ce15e9d3 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:53222 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:19:15,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879a575bc060ce15e41e 2025-12-17 22:19:15,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879a575bc060ce15e41e HTTP/1.1" 502 150 2025-12-17 22:19:15,545 - lava_callback - ERROR - Error processing callback for node 6940879a575bc060ce15e41e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940879a575bc060ce15e41e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940879a575bc060ce15e41e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940879a575bc060ce15e41e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:15,547 - lava_callback - INFO - Starting async job submit for node 694087ab575bc060ce15e487 2025-12-17 22:19:15,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:15,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ab575bc060ce15e487 HTTP/1.1" 502 150 2025-12-17 22:19:15,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ab575bc060ce15e487'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:15,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ab575bc060ce15e487 2025-12-17 22:19:15,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ab575bc060ce15e487 HTTP/1.1" 502 150 2025-12-17 22:19:15,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ab575bc060ce15e487'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:15,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940878b575bc060ce15e405 2025-12-17 22:19:15,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878b575bc060ce15e405 HTTP/1.1" 502 150 2025-12-17 22:19:15,743 - lava_callback - ERROR - Error processing callback for node 6940878b575bc060ce15e405: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940878b575bc060ce15e405 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940878b575bc060ce15e405 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940878b575bc060ce15e405 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:15,745 - lava_callback - INFO - Starting async job submit for node 694087ac575bc060ce15e48a 2025-12-17 22:19:15,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:15,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ac575bc060ce15e48a HTTP/1.1" 502 150 2025-12-17 22:19:15,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ac575bc060ce15e48a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:15,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ac575bc060ce15e48a 2025-12-17 22:19:15,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ac575bc060ce15e48a HTTP/1.1" 502 150 2025-12-17 22:19:15,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ac575bc060ce15e48a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:15,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940878d575bc060ce15e408 2025-12-17 22:19:15,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878d575bc060ce15e408 HTTP/1.1" 502 150 2025-12-17 22:19:15,873 - lava_callback - ERROR - Error processing callback for node 6940878d575bc060ce15e408: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940878d575bc060ce15e408 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940878d575bc060ce15e408 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940878d575bc060ce15e408 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:15,875 - lava_callback - INFO - Starting async job submit for node 694087ae575bc060ce15e490 2025-12-17 22:19:15,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:15,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ae575bc060ce15e490 HTTP/1.1" 502 150 2025-12-17 22:19:15,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ae575bc060ce15e490'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:15,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ae575bc060ce15e490 2025-12-17 22:19:15,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ae575bc060ce15e490 HTTP/1.1" 502 150 2025-12-17 22:19:15,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ae575bc060ce15e490'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53234 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:19:16,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a9575bc060ce15e484 2025-12-17 22:19:16,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a9575bc060ce15e484 HTTP/1.1" 502 150 2025-12-17 22:19:16,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a9575bc060ce15e484'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:16,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a1575bc060ce15e437 2025-12-17 22:19:16,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879c575bc060ce15e424 2025-12-17 22:19:16,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a1575bc060ce15e437 HTTP/1.1" 502 150 2025-12-17 22:19:16,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a1575bc060ce15e437'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:16,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879c575bc060ce15e424 HTTP/1.1" 502 150 2025-12-17 22:19:16,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879c575bc060ce15e424'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:16,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a4575bc060ce15e43a 2025-12-17 22:19:16,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a4575bc060ce15e43a HTTP/1.1" 502 150 2025-12-17 22:19:16,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a4575bc060ce15e43a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:16,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940878f575bc060ce15e40b 2025-12-17 22:19:16,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940878f575bc060ce15e40b HTTP/1.1" 502 150 2025-12-17 22:19:16,918 - lava_callback - ERROR - Error processing callback for node 6940878f575bc060ce15e40b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940878f575bc060ce15e40b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940878f575bc060ce15e40b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940878f575bc060ce15e40b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:16,919 - lava_callback - INFO - Starting async job submit for node 694087ad575bc060ce15e48d 2025-12-17 22:19:16,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:16,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ad575bc060ce15e48d HTTP/1.1" 502 150 2025-12-17 22:19:16,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ad575bc060ce15e48d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:16,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ad575bc060ce15e48d 2025-12-17 22:19:16,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ad575bc060ce15e48d HTTP/1.1" 502 150 2025-12-17 22:19:16,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ad575bc060ce15e48d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:16,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408791575bc060ce15e40e 2025-12-17 22:19:16,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408791575bc060ce15e40e HTTP/1.1" 502 150 2025-12-17 22:19:16,991 - lava_callback - ERROR - Error processing callback for node 69408791575bc060ce15e40e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408791575bc060ce15e40e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408791575bc060ce15e40e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408791575bc060ce15e40e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:16,993 - lava_callback - INFO - Starting async job submit for node 694087b1575bc060ce15e496 2025-12-17 22:19:16,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:16,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b1575bc060ce15e496 HTTP/1.1" 502 150 2025-12-17 22:19:16,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b1575bc060ce15e496'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:16,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b1575bc060ce15e496 2025-12-17 22:19:16,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b1575bc060ce15e496 HTTP/1.1" 502 150 2025-12-17 22:19:16,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b1575bc060ce15e496'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:17,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ab575bc060ce15e487 2025-12-17 22:19:17,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ab575bc060ce15e487 HTTP/1.1" 502 150 2025-12-17 22:19:17,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ab575bc060ce15e487'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:17,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ac575bc060ce15e48a 2025-12-17 22:19:17,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ac575bc060ce15e48a HTTP/1.1" 502 150 2025-12-17 22:19:17,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ac575bc060ce15e48a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:17,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a6575bc060ce15e441 2025-12-17 22:19:17,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a6575bc060ce15e441 HTTP/1.1" 502 150 2025-12-17 22:19:17,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a6575bc060ce15e441'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:17,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ae575bc060ce15e490 2025-12-17 22:19:17,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ae575bc060ce15e490 HTTP/1.1" 502 150 2025-12-17 22:19:17,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ae575bc060ce15e490'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:18,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ad575bc060ce15e48d 2025-12-17 22:19:18,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ad575bc060ce15e48d HTTP/1.1" 502 150 2025-12-17 22:19:18,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ad575bc060ce15e48d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:18,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b1575bc060ce15e496 2025-12-17 22:19:19,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b1575bc060ce15e496 HTTP/1.1" 502 150 2025-12-17 22:19:19,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b1575bc060ce15e496'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:19,968 - lava_callback - INFO - Received callback for node 69408958575bc060ce15e9e8 INFO: 172.21.0.1:44586 - "POST /node/69408958575bc060ce15e9e8 HTTP/1.0" 202 Accepted 2025-12-17 22:19:20,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879b575bc060ce15e421 2025-12-17 22:19:20,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879b575bc060ce15e421 HTTP/1.1" 502 150 2025-12-17 22:19:20,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879b575bc060ce15e421'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:20,752 - lava_callback - INFO - Received callback for node 6940895a575bc060ce15ea16 INFO: 172.21.0.1:44592 - "POST /node/6940895a575bc060ce15ea16 HTTP/1.0" 202 Accepted 2025-12-17 22:19:21,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409139575bc060ce1607c1 2025-12-17 22:19:21,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409139575bc060ce1607c1 HTTP/1.1" 502 150 2025-12-17 22:19:21,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409139575bc060ce1607c1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:21,513 - lava_callback - INFO - Received callback for node 6940895c575bc060ce15ea4b INFO: 172.21.0.1:44594 - "POST /node/6940895c575bc060ce15ea4b HTTP/1.0" 202 Accepted 2025-12-17 22:19:21,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ab575bc060ce15e487 2025-12-17 22:19:21,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ab575bc060ce15e487 HTTP/1.1" 502 150 2025-12-17 22:19:21,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ab575bc060ce15e487'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:21,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ac575bc060ce15e48a 2025-12-17 22:19:21,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ac575bc060ce15e48a HTTP/1.1" 502 150 2025-12-17 22:19:21,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ac575bc060ce15e48a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:21,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ae575bc060ce15e490 2025-12-17 22:19:21,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ae575bc060ce15e490 HTTP/1.1" 502 150 2025-12-17 22:19:21,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ae575bc060ce15e490'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:21,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879f575bc060ce15e42e 2025-12-17 22:19:22,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879f575bc060ce15e42e HTTP/1.1" 502 150 2025-12-17 22:19:22,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879f575bc060ce15e42e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:22,245 - lava_callback - INFO - Received callback for node 6940895e575bc060ce15ea58 INFO: 172.21.0.1:44596 - "POST /node/6940895e575bc060ce15ea58 HTTP/1.0" 202 Accepted 2025-12-17 22:19:22,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ad575bc060ce15e48d 2025-12-17 22:19:22,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ad575bc060ce15e48d HTTP/1.1" 502 150 2025-12-17 22:19:22,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ad575bc060ce15e48d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:22,981 - lava_callback - INFO - Received callback for node 6940895f575bc060ce15ea5b INFO: 172.21.0.1:44600 - "POST /node/6940895f575bc060ce15ea5b HTTP/1.0" 202 Accepted 2025-12-17 22:19:23,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b1575bc060ce15e496 2025-12-17 22:19:23,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b1575bc060ce15e496 HTTP/1.1" 502 150 2025-12-17 22:19:23,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b1575bc060ce15e496'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:23,808 - lava_callback - INFO - Received callback for node 69408960575bc060ce15ea5e INFO: 172.21.0.1:44602 - "POST /node/69408960575bc060ce15ea5e HTTP/1.0" 202 Accepted 2025-12-17 22:19:24,293 - lava_callback - INFO - Received callback for node 69408964575bc060ce15ea67 INFO: 172.21.0.1:44618 - "POST /node/69408964575bc060ce15ea67 HTTP/1.0" 202 Accepted 2025-12-17 22:19:24,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a9575bc060ce15e484 2025-12-17 22:19:24,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a9575bc060ce15e484 HTTP/1.1" 502 150 2025-12-17 22:19:24,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087a9575bc060ce15e484'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:24,591 - lava_callback - INFO - Received callback for node 69408962575bc060ce15ea61 INFO: 172.21.0.1:44634 - "POST /node/69408962575bc060ce15ea61 HTTP/1.0" 202 Accepted 2025-12-17 22:19:24,942 - lava_callback - INFO - Received callback for node 69408966575bc060ce15ea6a INFO: 172.21.0.1:44642 - "POST /node/69408966575bc060ce15ea6a HTTP/1.0" 202 Accepted 2025-12-17 22:19:25,341 - lava_callback - INFO - Received callback for node 69408963575bc060ce15ea64 INFO: 172.21.0.1:44644 - "POST /node/69408963575bc060ce15ea64 HTTP/1.0" 202 Accepted 2025-12-17 22:19:25,589 - lava_callback - INFO - Received callback for node 69408969575bc060ce15ea70 INFO: 172.21.0.1:44656 - "POST /node/69408969575bc060ce15ea70 HTTP/1.0" 202 Accepted 2025-12-17 22:19:25,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408795575bc060ce15e414 2025-12-17 22:19:25,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408795575bc060ce15e414 HTTP/1.1" 502 150 2025-12-17 22:19:25,856 - lava_callback - ERROR - Error processing callback for node 69408795575bc060ce15e414: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408795575bc060ce15e414 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408795575bc060ce15e414 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408795575bc060ce15e414 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:25,858 - lava_callback - INFO - Starting async job submit for node 694087b5575bc060ce15e4ab 2025-12-17 22:19:25,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:25,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b5575bc060ce15e4ab HTTP/1.1" 502 150 2025-12-17 22:19:25,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b5575bc060ce15e4ab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:25,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b5575bc060ce15e4ab 2025-12-17 22:19:25,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b5575bc060ce15e4ab HTTP/1.1" 502 150 2025-12-17 22:19:25,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b5575bc060ce15e4ab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:26,364 - lava_callback - INFO - Received callback for node 69408968575bc060ce15ea6d INFO: 172.21.0.1:44670 - "POST /node/69408968575bc060ce15ea6d HTTP/1.0" 202 Accepted INFO: 172.21.0.1:44676 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:19:26,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408798575bc060ce15e41b 2025-12-17 22:19:26,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408798575bc060ce15e41b HTTP/1.1" 502 150 2025-12-17 22:19:26,617 - lava_callback - ERROR - Error processing callback for node 69408798575bc060ce15e41b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408798575bc060ce15e41b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408798575bc060ce15e41b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408798575bc060ce15e41b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:26,619 - lava_callback - INFO - Starting async job submit for node 694087bc575bc060ce15e4b4 2025-12-17 22:19:26,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:26,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087bc575bc060ce15e4b4 HTTP/1.1" 502 150 2025-12-17 22:19:26,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087bc575bc060ce15e4b4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:26,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087bc575bc060ce15e4b4 2025-12-17 22:19:26,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087bc575bc060ce15e4b4 HTTP/1.1" 502 150 2025-12-17 22:19:26,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087bc575bc060ce15e4b4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:27,134 - lava_callback - INFO - Received callback for node 6940896c575bc060ce15ea79 INFO: 172.21.0.1:44692 - "POST /node/6940896c575bc060ce15ea79 HTTP/1.0" 202 Accepted 2025-12-17 22:19:27,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b5575bc060ce15e4ab 2025-12-17 22:19:27,896 - lava_callback - INFO - Received callback for node 6940896d575bc060ce15ea7f INFO: 172.21.0.1:44698 - "POST /node/6940896d575bc060ce15ea7f HTTP/1.0" 202 Accepted 2025-12-17 22:19:27,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b5575bc060ce15e4ab HTTP/1.1" 502 150 2025-12-17 22:19:27,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b5575bc060ce15e4ab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:28,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087bc575bc060ce15e4b4 2025-12-17 22:19:28,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087bc575bc060ce15e4b4 HTTP/1.1" 502 150 2025-12-17 22:19:28,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087bc575bc060ce15e4b4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:29,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ab575bc060ce15e487 2025-12-17 22:19:29,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ab575bc060ce15e487 HTTP/1.1" 502 150 2025-12-17 22:19:29,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ab575bc060ce15e487'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:29,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408779575bc060ce15e3e8 2025-12-17 22:19:29,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408779575bc060ce15e3e8 HTTP/1.1" 502 150 2025-12-17 22:19:29,689 - lava_callback - ERROR - Error processing callback for node 69408779575bc060ce15e3e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408779575bc060ce15e3e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408779575bc060ce15e3e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408779575bc060ce15e3e8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:29,690 - lava_callback - INFO - Starting async job submit for node 694087b0575bc060ce15e493 2025-12-17 22:19:29,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:29,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b0575bc060ce15e493 HTTP/1.1" 502 150 2025-12-17 22:19:29,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b0575bc060ce15e493'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:29,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b0575bc060ce15e493 2025-12-17 22:19:29,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b0575bc060ce15e493 HTTP/1.1" 502 150 2025-12-17 22:19:29,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b0575bc060ce15e493'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:29,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ac575bc060ce15e48a 2025-12-17 22:19:29,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ac575bc060ce15e48a HTTP/1.1" 502 150 2025-12-17 22:19:29,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ac575bc060ce15e48a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:29,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ae575bc060ce15e490 2025-12-17 22:19:29,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ae575bc060ce15e490 HTTP/1.1" 502 150 2025-12-17 22:19:29,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ae575bc060ce15e490'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:30,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ad575bc060ce15e48d 2025-12-17 22:19:30,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ad575bc060ce15e48d HTTP/1.1" 502 150 2025-12-17 22:19:30,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ad575bc060ce15e48d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:31,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b1575bc060ce15e496 2025-12-17 22:19:31,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b1575bc060ce15e496 HTTP/1.1" 502 150 2025-12-17 22:19:31,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b1575bc060ce15e496'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:31,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b0575bc060ce15e493 2025-12-17 22:19:31,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b0575bc060ce15e493 HTTP/1.1" 502 150 2025-12-17 22:19:31,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b0575bc060ce15e493'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:31,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b5575bc060ce15e4ab 2025-12-17 22:19:31,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b5575bc060ce15e4ab HTTP/1.1" 502 150 2025-12-17 22:19:31,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b5575bc060ce15e4ab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:32,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087bc575bc060ce15e4b4 2025-12-17 22:19:32,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087bc575bc060ce15e4b4 HTTP/1.1" 502 150 2025-12-17 22:19:32,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087bc575bc060ce15e4b4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:32,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a1575bc060ce15e437 2025-12-17 22:19:32,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879c575bc060ce15e424 2025-12-17 22:19:32,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a1575bc060ce15e437 HTTP/1.1" 502 150 2025-12-17 22:19:32,730 - lava_callback - ERROR - Error processing callback for node 694087a1575bc060ce15e437: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087a1575bc060ce15e437 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087a1575bc060ce15e437 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087a1575bc060ce15e437 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:32,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879c575bc060ce15e424 HTTP/1.1" 502 150 2025-12-17 22:19:32,732 - lava_callback - INFO - Starting async job submit for node 694087b2575bc060ce15e49b 2025-12-17 22:19:32,732 - lava_callback - ERROR - Error processing callback for node 6940879c575bc060ce15e424: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940879c575bc060ce15e424 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940879c575bc060ce15e424 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940879c575bc060ce15e424 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:32,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:32,735 - lava_callback - INFO - Starting async job submit for node 694087b4575bc060ce15e4a6 2025-12-17 22:19:32,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:32,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b4575bc060ce15e4a6 HTTP/1.1" 502 150 2025-12-17 22:19:32,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b4575bc060ce15e4a6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:32,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b4575bc060ce15e4a6 2025-12-17 22:19:32,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b2575bc060ce15e49b HTTP/1.1" 502 150 2025-12-17 22:19:32,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b2575bc060ce15e49b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:32,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b2575bc060ce15e49b 2025-12-17 22:19:32,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b4575bc060ce15e4a6 HTTP/1.1" 502 150 2025-12-17 22:19:32,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b4575bc060ce15e4a6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:32,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b2575bc060ce15e49b HTTP/1.1" 502 150 2025-12-17 22:19:32,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b2575bc060ce15e49b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:32,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a4575bc060ce15e43a 2025-12-17 22:19:32,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a4575bc060ce15e43a HTTP/1.1" 502 150 2025-12-17 22:19:32,787 - lava_callback - ERROR - Error processing callback for node 694087a4575bc060ce15e43a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087a4575bc060ce15e43a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087a4575bc060ce15e43a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087a4575bc060ce15e43a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:32,789 - lava_callback - INFO - Starting async job submit for node 694087be575bc060ce15e4b7 2025-12-17 22:19:32,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:32,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087be575bc060ce15e4b7 HTTP/1.1" 502 150 2025-12-17 22:19:32,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087be575bc060ce15e4b7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:32,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087be575bc060ce15e4b7 2025-12-17 22:19:32,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087be575bc060ce15e4b7 HTTP/1.1" 502 150 2025-12-17 22:19:32,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087be575bc060ce15e4b7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:33,375 - lava_callback - INFO - Received callback for node 69426b011b61f1c6005f5896 INFO: 172.21.0.1:45520 - "POST /node/69426b011b61f1c6005f5896 HTTP/1.0" 202 Accepted 2025-12-17 22:19:33,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a6575bc060ce15e441 2025-12-17 22:19:33,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a6575bc060ce15e441 HTTP/1.1" 502 150 2025-12-17 22:19:33,839 - lava_callback - ERROR - Error processing callback for node 694087a6575bc060ce15e441: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087a6575bc060ce15e441 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087a6575bc060ce15e441 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087a6575bc060ce15e441 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:33,840 - lava_callback - INFO - Starting async job submit for node 694087ca575bc060ce15e4c9 2025-12-17 22:19:33,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:33,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ca575bc060ce15e4c9 HTTP/1.1" 502 150 2025-12-17 22:19:33,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ca575bc060ce15e4c9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:33,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ca575bc060ce15e4c9 2025-12-17 22:19:33,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ca575bc060ce15e4c9 HTTP/1.1" 502 150 2025-12-17 22:19:33,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ca575bc060ce15e4c9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:34,317 - lava_callback - INFO - Received callback for node 6940896e575bc060ce15ea85 INFO: 172.21.0.1:45526 - "POST /node/6940896e575bc060ce15ea85 HTTP/1.0" 202 Accepted 2025-12-17 22:19:34,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b4575bc060ce15e4a6 2025-12-17 22:19:34,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b2575bc060ce15e49b 2025-12-17 22:19:34,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b2575bc060ce15e49b HTTP/1.1" 502 150 2025-12-17 22:19:34,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b2575bc060ce15e49b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:34,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087be575bc060ce15e4b7 2025-12-17 22:19:34,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087be575bc060ce15e4b7 HTTP/1.1" 502 150 2025-12-17 22:19:34,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087be575bc060ce15e4b7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:34,992 - lava_callback - INFO - Received callback for node 69408971575bc060ce15ea8e INFO: 172.21.0.1:45528 - "POST /node/69408971575bc060ce15ea8e HTTP/1.0" 202 Accepted 2025-12-17 22:19:35,668 - lava_callback - INFO - Received callback for node 69408973575bc060ce15ea91 INFO: 172.21.0.1:45542 - "POST /node/69408973575bc060ce15ea91 HTTP/1.0" 202 Accepted 2025-12-17 22:19:35,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b0575bc060ce15e493 2025-12-17 22:19:35,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b0575bc060ce15e493 HTTP/1.1" 502 150 2025-12-17 22:19:35,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b0575bc060ce15e493'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:35,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ca575bc060ce15e4c9 2025-12-17 22:19:35,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ca575bc060ce15e4c9 HTTP/1.1" 502 150 2025-12-17 22:19:35,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ca575bc060ce15e4c9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:36,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879b575bc060ce15e421 2025-12-17 22:19:36,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879b575bc060ce15e421 HTTP/1.1" 502 150 2025-12-17 22:19:36,385 - lava_callback - ERROR - Error processing callback for node 6940879b575bc060ce15e421: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940879b575bc060ce15e421 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940879b575bc060ce15e421 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940879b575bc060ce15e421 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:36,387 - lava_callback - INFO - Starting async job submit for node 694087cf575bc060ce15e4d7 2025-12-17 22:19:36,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:36,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087cf575bc060ce15e4d7 HTTP/1.1" 502 150 2025-12-17 22:19:36,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087cf575bc060ce15e4d7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:36,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087cf575bc060ce15e4d7 2025-12-17 22:19:36,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087cf575bc060ce15e4d7 HTTP/1.1" 502 150 2025-12-17 22:19:36,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087cf575bc060ce15e4d7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45556 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:19:37,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409139575bc060ce1607c1 2025-12-17 22:19:37,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409139575bc060ce1607c1 HTTP/1.1" 502 150 2025-12-17 22:19:37,459 - lava_callback - ERROR - Error processing callback for node 69409139575bc060ce1607c1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409139575bc060ce1607c1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409139575bc060ce1607c1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409139575bc060ce1607c1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:37,461 - lava_callback - INFO - Starting async job submit for node 694087d2575bc060ce15e4da 2025-12-17 22:19:37,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:37,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087d2575bc060ce15e4da HTTP/1.1" 502 150 2025-12-17 22:19:37,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087d2575bc060ce15e4da'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:37,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087d2575bc060ce15e4da 2025-12-17 22:19:37,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087d2575bc060ce15e4da HTTP/1.1" 502 150 2025-12-17 22:19:37,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087d2575bc060ce15e4da'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:38,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879f575bc060ce15e42e 2025-12-17 22:19:38,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087cf575bc060ce15e4d7 2025-12-17 22:19:38,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087cf575bc060ce15e4d7 HTTP/1.1" 502 150 2025-12-17 22:19:38,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087cf575bc060ce15e4d7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:38,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b2575bc060ce15e49b 2025-12-17 22:19:38,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b2575bc060ce15e49b HTTP/1.1" 502 150 2025-12-17 22:19:38,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b2575bc060ce15e49b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:38,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087be575bc060ce15e4b7 2025-12-17 22:19:38,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087be575bc060ce15e4b7 HTTP/1.1" 502 150 2025-12-17 22:19:38,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087be575bc060ce15e4b7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:39,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087d2575bc060ce15e4da 2025-12-17 22:19:39,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087d2575bc060ce15e4da HTTP/1.1" 502 150 2025-12-17 22:19:39,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087d2575bc060ce15e4da'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:39,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ca575bc060ce15e4c9 2025-12-17 22:19:39,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ca575bc060ce15e4c9 HTTP/1.1" 502 150 2025-12-17 22:19:39,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ca575bc060ce15e4c9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:39,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b5575bc060ce15e4ab 2025-12-17 22:19:39,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b5575bc060ce15e4ab HTTP/1.1" 502 150 2025-12-17 22:19:39,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b5575bc060ce15e4ab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:40,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087a9575bc060ce15e484 2025-12-17 22:19:40,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087a9575bc060ce15e484 HTTP/1.1" 502 150 2025-12-17 22:19:40,546 - lava_callback - ERROR - Error processing callback for node 694087a9575bc060ce15e484: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087a9575bc060ce15e484 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087a9575bc060ce15e484 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087a9575bc060ce15e484 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:40,549 - lava_callback - INFO - Starting async job submit for node 69409174575bc060ce16084f 2025-12-17 22:19:40,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087bc575bc060ce15e4b4 2025-12-17 22:19:40,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:40,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087bc575bc060ce15e4b4 HTTP/1.1" 502 150 2025-12-17 22:19:40,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087bc575bc060ce15e4b4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:40,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409174575bc060ce16084f HTTP/1.1" 502 150 2025-12-17 22:19:40,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409174575bc060ce16084f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:40,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409174575bc060ce16084f 2025-12-17 22:19:40,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409174575bc060ce16084f HTTP/1.1" 502 150 2025-12-17 22:19:40,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409174575bc060ce16084f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:42,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087cf575bc060ce15e4d7 2025-12-17 22:19:42,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087cf575bc060ce15e4d7 HTTP/1.1" 502 150 2025-12-17 22:19:42,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087cf575bc060ce15e4d7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:42,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409174575bc060ce16084f 2025-12-17 22:19:42,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409174575bc060ce16084f HTTP/1.1" 502 150 2025-12-17 22:19:42,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409174575bc060ce16084f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:43,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087d2575bc060ce15e4da 2025-12-17 22:19:43,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087d2575bc060ce15e4da HTTP/1.1" 502 150 2025-12-17 22:19:43,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087d2575bc060ce15e4da'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:43,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b0575bc060ce15e493 2025-12-17 22:19:43,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b0575bc060ce15e493 HTTP/1.1" 502 150 2025-12-17 22:19:43,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b0575bc060ce15e493'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:44,766 - lava_callback - INFO - Received callback for node 6940c5361e3c24e95681d6a7 INFO: 172.21.0.1:59834 - "POST /node/6940c5361e3c24e95681d6a7 HTTP/1.0" 202 Accepted 2025-12-17 22:19:45,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ab575bc060ce15e487 2025-12-17 22:19:45,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ab575bc060ce15e487 HTTP/1.1" 502 150 2025-12-17 22:19:45,560 - lava_callback - ERROR - Error processing callback for node 694087ab575bc060ce15e487: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ab575bc060ce15e487 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ab575bc060ce15e487 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ab575bc060ce15e487 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:45,561 - lava_callback - INFO - Starting async job submit for node 694087da575bc060ce15e4e0 2025-12-17 22:19:45,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:45,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087da575bc060ce15e4e0 HTTP/1.1" 502 150 2025-12-17 22:19:45,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087da575bc060ce15e4e0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:45,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087da575bc060ce15e4e0 2025-12-17 22:19:45,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087da575bc060ce15e4e0 HTTP/1.1" 502 150 2025-12-17 22:19:45,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087da575bc060ce15e4e0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:45,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ac575bc060ce15e48a 2025-12-17 22:19:45,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ac575bc060ce15e48a HTTP/1.1" 502 150 2025-12-17 22:19:45,761 - lava_callback - ERROR - Error processing callback for node 694087ac575bc060ce15e48a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ac575bc060ce15e48a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ac575bc060ce15e48a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ac575bc060ce15e48a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:45,763 - lava_callback - INFO - Starting async job submit for node 6940927c575bc060ce160acb 2025-12-17 22:19:45,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:45,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940927c575bc060ce160acb HTTP/1.1" 502 150 2025-12-17 22:19:45,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940927c575bc060ce160acb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:45,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940927c575bc060ce160acb 2025-12-17 22:19:45,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940927c575bc060ce160acb HTTP/1.1" 502 150 2025-12-17 22:19:45,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940927c575bc060ce160acb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:45,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ae575bc060ce15e490 2025-12-17 22:19:45,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ae575bc060ce15e490 HTTP/1.1" 502 150 2025-12-17 22:19:45,908 - lava_callback - ERROR - Error processing callback for node 694087ae575bc060ce15e490: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ae575bc060ce15e490 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ae575bc060ce15e490 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ae575bc060ce15e490 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:45,910 - lava_callback - INFO - Starting async job submit for node 694087e0575bc060ce15e4e7 2025-12-17 22:19:45,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:45,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e0575bc060ce15e4e7 HTTP/1.1" 502 150 2025-12-17 22:19:45,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e0575bc060ce15e4e7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:45,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e0575bc060ce15e4e7 2025-12-17 22:19:45,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e0575bc060ce15e4e7 HTTP/1.1" 502 150 2025-12-17 22:19:45,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e0575bc060ce15e4e7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59840 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:19:46,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409174575bc060ce16084f 2025-12-17 22:19:46,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409174575bc060ce16084f HTTP/1.1" 502 150 2025-12-17 22:19:46,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409174575bc060ce16084f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:46,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b2575bc060ce15e49b 2025-12-17 22:19:46,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b2575bc060ce15e49b HTTP/1.1" 502 150 2025-12-17 22:19:46,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b2575bc060ce15e49b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:46,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087be575bc060ce15e4b7 2025-12-17 22:19:46,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087be575bc060ce15e4b7 HTTP/1.1" 502 150 2025-12-17 22:19:46,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087be575bc060ce15e4b7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:46,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ad575bc060ce15e48d 2025-12-17 22:19:46,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ad575bc060ce15e48d HTTP/1.1" 502 150 2025-12-17 22:19:46,941 - lava_callback - ERROR - Error processing callback for node 694087ad575bc060ce15e48d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ad575bc060ce15e48d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ad575bc060ce15e48d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ad575bc060ce15e48d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:46,943 - lava_callback - INFO - Starting async job submit for node 694087e2575bc060ce15e4ea 2025-12-17 22:19:46,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:46,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e2575bc060ce15e4ea HTTP/1.1" 502 150 2025-12-17 22:19:46,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e2575bc060ce15e4ea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:46,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e2575bc060ce15e4ea 2025-12-17 22:19:46,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e2575bc060ce15e4ea HTTP/1.1" 502 150 2025-12-17 22:19:46,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e2575bc060ce15e4ea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:47,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b1575bc060ce15e496 2025-12-17 22:19:47,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b1575bc060ce15e496 HTTP/1.1" 502 150 2025-12-17 22:19:47,004 - lava_callback - ERROR - Error processing callback for node 694087b1575bc060ce15e496: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b1575bc060ce15e496 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b1575bc060ce15e496 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b1575bc060ce15e496 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:47,006 - lava_callback - INFO - Starting async job submit for node 694087e6575bc060ce15e51d 2025-12-17 22:19:47,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:47,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e6575bc060ce15e51d HTTP/1.1" 502 150 2025-12-17 22:19:47,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e6575bc060ce15e51d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:47,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e6575bc060ce15e51d 2025-12-17 22:19:47,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e6575bc060ce15e51d HTTP/1.1" 502 150 2025-12-17 22:19:47,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e6575bc060ce15e51d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:47,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087da575bc060ce15e4e0 2025-12-17 22:19:47,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087da575bc060ce15e4e0 HTTP/1.1" 502 150 2025-12-17 22:19:47,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087da575bc060ce15e4e0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:47,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940927c575bc060ce160acb 2025-12-17 22:19:47,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940927c575bc060ce160acb HTTP/1.1" 502 150 2025-12-17 22:19:47,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940927c575bc060ce160acb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:47,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ca575bc060ce15e4c9 2025-12-17 22:19:47,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ca575bc060ce15e4c9 HTTP/1.1" 502 150 2025-12-17 22:19:47,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ca575bc060ce15e4c9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:47,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e0575bc060ce15e4e7 2025-12-17 22:19:47,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e0575bc060ce15e4e7 HTTP/1.1" 502 150 2025-12-17 22:19:47,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e0575bc060ce15e4e7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:48,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e2575bc060ce15e4ea 2025-12-17 22:19:48,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e2575bc060ce15e4ea HTTP/1.1" 502 150 2025-12-17 22:19:48,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e2575bc060ce15e4ea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:49,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e6575bc060ce15e51d 2025-12-17 22:19:49,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e6575bc060ce15e51d HTTP/1.1" 502 150 2025-12-17 22:19:49,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e6575bc060ce15e51d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:50,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087cf575bc060ce15e4d7 2025-12-17 22:19:50,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087cf575bc060ce15e4d7 HTTP/1.1" 502 150 2025-12-17 22:19:50,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087cf575bc060ce15e4d7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:51,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087d2575bc060ce15e4da 2025-12-17 22:19:51,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087d2575bc060ce15e4da HTTP/1.1" 502 150 2025-12-17 22:19:51,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087d2575bc060ce15e4da'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:51,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087da575bc060ce15e4e0 2025-12-17 22:19:51,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087da575bc060ce15e4e0 HTTP/1.1" 502 150 2025-12-17 22:19:51,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087da575bc060ce15e4e0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:51,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940927c575bc060ce160acb 2025-12-17 22:19:51,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940927c575bc060ce160acb HTTP/1.1" 502 150 2025-12-17 22:19:51,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940927c575bc060ce160acb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:51,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e0575bc060ce15e4e7 2025-12-17 22:19:51,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e0575bc060ce15e4e7 HTTP/1.1" 502 150 2025-12-17 22:19:51,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e0575bc060ce15e4e7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:52,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e2575bc060ce15e4ea 2025-12-17 22:19:52,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e2575bc060ce15e4ea HTTP/1.1" 502 150 2025-12-17 22:19:52,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e2575bc060ce15e4ea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:53,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e6575bc060ce15e51d 2025-12-17 22:19:53,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e6575bc060ce15e51d HTTP/1.1" 502 150 2025-12-17 22:19:53,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e6575bc060ce15e51d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:53,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879f575bc060ce15e42e HTTP/1.1" 502 150 2025-12-17 22:19:53,393 - lava_callback - ERROR - Error processing callback for node 6940879f575bc060ce15e42e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940879f575bc060ce15e42e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940879f575bc060ce15e42e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940879f575bc060ce15e42e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:53,396 - lava_callback - INFO - Starting async job submit for node 6940cb75460bc8e265111be3 2025-12-17 22:19:53,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:53,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940cb75460bc8e265111be3 HTTP/1.1" 502 150 2025-12-17 22:19:53,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940cb75460bc8e265111be3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:53,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940cb75460bc8e265111be3 2025-12-17 22:19:53,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940cb75460bc8e265111be3 HTTP/1.1" 502 150 2025-12-17 22:19:53,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940cb75460bc8e265111be3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:54,299 - lava_callback - INFO - Received callback for node 6940897b575bc060ce15ea9c INFO: 172.21.0.1:39334 - "POST /node/6940897b575bc060ce15ea9c HTTP/1.0" 202 Accepted 2025-12-17 22:19:54,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409174575bc060ce16084f 2025-12-17 22:19:54,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409174575bc060ce16084f HTTP/1.1" 502 150 2025-12-17 22:19:54,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409174575bc060ce16084f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:54,957 - lava_callback - INFO - Received callback for node 6940897d575bc060ce15ea9f INFO: 172.21.0.1:39338 - "POST /node/6940897d575bc060ce15ea9f HTTP/1.0" 202 Accepted 2025-12-17 22:19:55,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940cb75460bc8e265111be3 2025-12-17 22:19:55,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940cb75460bc8e265111be3 HTTP/1.1" 502 150 2025-12-17 22:19:55,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940cb75460bc8e265111be3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:55,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b5575bc060ce15e4ab 2025-12-17 22:19:55,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b5575bc060ce15e4ab HTTP/1.1" 502 150 2025-12-17 22:19:55,912 - lava_callback - ERROR - Error processing callback for node 694087b5575bc060ce15e4ab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b5575bc060ce15e4ab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b5575bc060ce15e4ab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b5575bc060ce15e4ab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:55,914 - lava_callback - INFO - Starting async job submit for node 694087e9575bc060ce15e54a 2025-12-17 22:19:55,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:55,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e9575bc060ce15e54a HTTP/1.1" 502 150 2025-12-17 22:19:55,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e9575bc060ce15e54a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:55,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e9575bc060ce15e54a 2025-12-17 22:19:55,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e9575bc060ce15e54a HTTP/1.1" 502 150 2025-12-17 22:19:55,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e9575bc060ce15e54a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39346 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:19:56,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087bc575bc060ce15e4b4 2025-12-17 22:19:56,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087bc575bc060ce15e4b4 HTTP/1.1" 502 150 2025-12-17 22:19:56,669 - lava_callback - ERROR - Error processing callback for node 694087bc575bc060ce15e4b4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087bc575bc060ce15e4b4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087bc575bc060ce15e4b4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087bc575bc060ce15e4b4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:56,671 - lava_callback - INFO - Starting async job submit for node 694087eb575bc060ce15e54d 2025-12-17 22:19:56,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:56,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087eb575bc060ce15e54d HTTP/1.1" 502 150 2025-12-17 22:19:56,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087eb575bc060ce15e54d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:56,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087eb575bc060ce15e54d 2025-12-17 22:19:56,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087eb575bc060ce15e54d HTTP/1.1" 502 150 2025-12-17 22:19:56,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087eb575bc060ce15e54d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:57,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e9575bc060ce15e54a 2025-12-17 22:19:57,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e9575bc060ce15e54a HTTP/1.1" 502 150 2025-12-17 22:19:57,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e9575bc060ce15e54a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:58,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087eb575bc060ce15e54d 2025-12-17 22:19:58,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087eb575bc060ce15e54d HTTP/1.1" 502 150 2025-12-17 22:19:58,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087eb575bc060ce15e54d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:59,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940cb75460bc8e265111be3 2025-12-17 22:19:59,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940cb75460bc8e265111be3 HTTP/1.1" 502 150 2025-12-17 22:19:59,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940cb75460bc8e265111be3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:59,518 - lava_callback - INFO - Received callback for node 69408980575bc060ce15eaa8 INFO: 172.21.0.1:39354 - "POST /node/69408980575bc060ce15eaa8 HTTP/1.0" 202 Accepted 2025-12-17 22:19:59,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087da575bc060ce15e4e0 2025-12-17 22:19:59,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087da575bc060ce15e4e0 HTTP/1.1" 502 150 2025-12-17 22:19:59,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087da575bc060ce15e4e0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:59,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b0575bc060ce15e493 2025-12-17 22:19:59,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b0575bc060ce15e493 HTTP/1.1" 502 150 2025-12-17 22:19:59,730 - lava_callback - ERROR - Error processing callback for node 694087b0575bc060ce15e493: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b0575bc060ce15e493 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b0575bc060ce15e493 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b0575bc060ce15e493 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:19:59,732 - lava_callback - INFO - Starting async job submit for node 694087ed575bc060ce15e550 2025-12-17 22:19:59,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:19:59,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ed575bc060ce15e550 HTTP/1.1" 502 150 2025-12-17 22:19:59,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ed575bc060ce15e550'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:59,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ed575bc060ce15e550 2025-12-17 22:19:59,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ed575bc060ce15e550 HTTP/1.1" 502 150 2025-12-17 22:19:59,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ed575bc060ce15e550'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:59,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940927c575bc060ce160acb 2025-12-17 22:19:59,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940927c575bc060ce160acb HTTP/1.1" 502 150 2025-12-17 22:19:59,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940927c575bc060ce160acb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:19:59,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e0575bc060ce15e4e7 2025-12-17 22:19:59,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e0575bc060ce15e4e7 HTTP/1.1" 502 150 2025-12-17 22:19:59,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e0575bc060ce15e4e7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:00,311 - lava_callback - INFO - Received callback for node 69408981575bc060ce15eaaf INFO: 172.21.0.1:46298 - "POST /node/69408981575bc060ce15eaaf HTTP/1.0" 202 Accepted 2025-12-17 22:20:00,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e2575bc060ce15e4ea 2025-12-17 22:20:00,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e2575bc060ce15e4ea HTTP/1.1" 502 150 2025-12-17 22:20:00,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e2575bc060ce15e4ea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:01,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e6575bc060ce15e51d 2025-12-17 22:20:01,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e6575bc060ce15e51d HTTP/1.1" 502 150 2025-12-17 22:20:01,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e6575bc060ce15e51d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:01,114 - lava_callback - INFO - Received callback for node 69408983575bc060ce15eab4 INFO: 172.21.0.1:46314 - "POST /node/69408983575bc060ce15eab4 HTTP/1.0" 202 Accepted 2025-12-17 22:20:01,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ed575bc060ce15e550 2025-12-17 22:20:01,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ed575bc060ce15e550 HTTP/1.1" 502 150 2025-12-17 22:20:01,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ed575bc060ce15e550'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:01,832 - lava_callback - INFO - Received callback for node 69408984575bc060ce15eab7 INFO: 172.21.0.1:46320 - "POST /node/69408984575bc060ce15eab7 HTTP/1.0" 202 Accepted 2025-12-17 22:20:01,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e9575bc060ce15e54a 2025-12-17 22:20:01,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e9575bc060ce15e54a HTTP/1.1" 502 150 2025-12-17 22:20:01,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e9575bc060ce15e54a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46326 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:20:02,645 - lava_callback - INFO - Received callback for node 69408985575bc060ce15eaba INFO: 172.21.0.1:46330 - "POST /node/69408985575bc060ce15eaba HTTP/1.0" 202 Accepted 2025-12-17 22:20:02,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087eb575bc060ce15e54d 2025-12-17 22:20:02,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087eb575bc060ce15e54d HTTP/1.1" 502 150 2025-12-17 22:20:02,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087eb575bc060ce15e54d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:02,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b2575bc060ce15e49b 2025-12-17 22:20:02,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b2575bc060ce15e49b HTTP/1.1" 502 150 2025-12-17 22:20:02,771 - lava_callback - ERROR - Error processing callback for node 694087b2575bc060ce15e49b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b2575bc060ce15e49b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b2575bc060ce15e49b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b2575bc060ce15e49b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:02,773 - lava_callback - INFO - Starting async job submit for node 694087ef575bc060ce15e555 2025-12-17 22:20:02,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:02,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ef575bc060ce15e555 HTTP/1.1" 502 150 2025-12-17 22:20:02,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ef575bc060ce15e555'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:02,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ef575bc060ce15e555 2025-12-17 22:20:02,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ef575bc060ce15e555 HTTP/1.1" 502 150 2025-12-17 22:20:02,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ef575bc060ce15e555'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:02,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087be575bc060ce15e4b7 2025-12-17 22:20:02,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087be575bc060ce15e4b7 HTTP/1.1" 502 150 2025-12-17 22:20:02,799 - lava_callback - ERROR - Error processing callback for node 694087be575bc060ce15e4b7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087be575bc060ce15e4b7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087be575bc060ce15e4b7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087be575bc060ce15e4b7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:02,801 - lava_callback - INFO - Starting async job submit for node 694087f1575bc060ce15e55a 2025-12-17 22:20:02,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:02,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f1575bc060ce15e55a HTTP/1.1" 502 150 2025-12-17 22:20:02,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f1575bc060ce15e55a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:02,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f1575bc060ce15e55a 2025-12-17 22:20:02,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f1575bc060ce15e55a HTTP/1.1" 502 150 2025-12-17 22:20:02,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f1575bc060ce15e55a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:03,657 - lava_callback - INFO - Received callback for node 69408986575bc060ce15eac2 INFO: 172.21.0.1:46342 - "POST /node/69408986575bc060ce15eac2 HTTP/1.0" 202 Accepted 2025-12-17 22:20:03,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ca575bc060ce15e4c9 2025-12-17 22:20:03,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ca575bc060ce15e4c9 HTTP/1.1" 502 150 2025-12-17 22:20:03,863 - lava_callback - ERROR - Error processing callback for node 694087ca575bc060ce15e4c9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ca575bc060ce15e4c9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ca575bc060ce15e4c9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ca575bc060ce15e4c9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:03,865 - lava_callback - INFO - Starting async job submit for node 694087f3575bc060ce15e55d 2025-12-17 22:20:03,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:03,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f3575bc060ce15e55d HTTP/1.1" 502 150 2025-12-17 22:20:03,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f3575bc060ce15e55d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:03,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f3575bc060ce15e55d 2025-12-17 22:20:03,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f3575bc060ce15e55d HTTP/1.1" 502 150 2025-12-17 22:20:03,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f3575bc060ce15e55d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:04,436 - lava_callback - INFO - Received callback for node 69408988575bc060ce15eac5 INFO: 172.21.0.1:46344 - "POST /node/69408988575bc060ce15eac5 HTTP/1.0" 202 Accepted 2025-12-17 22:20:04,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ef575bc060ce15e555 2025-12-17 22:20:04,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f1575bc060ce15e55a 2025-12-17 22:20:04,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f1575bc060ce15e55a HTTP/1.1" 502 150 2025-12-17 22:20:04,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f1575bc060ce15e55a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:05,176 - lava_callback - INFO - Received callback for node 69408989575bc060ce15eac8 INFO: 172.21.0.1:46360 - "POST /node/69408989575bc060ce15eac8 HTTP/1.0" 202 Accepted 2025-12-17 22:20:05,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b4575bc060ce15e4a6 HTTP/1.1" 502 150 2025-12-17 22:20:05,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b4575bc060ce15e4a6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:05,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ed575bc060ce15e550 2025-12-17 22:20:05,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ed575bc060ce15e550 HTTP/1.1" 502 150 2025-12-17 22:20:05,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ed575bc060ce15e550'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:05,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f3575bc060ce15e55d 2025-12-17 22:20:05,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f3575bc060ce15e55d HTTP/1.1" 502 150 2025-12-17 22:20:05,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f3575bc060ce15e55d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:05,913 - lava_callback - INFO - Received callback for node 6940898a575bc060ce15eacf INFO: 172.21.0.1:46362 - "POST /node/6940898a575bc060ce15eacf HTTP/1.0" 202 Accepted 2025-12-17 22:20:06,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087cf575bc060ce15e4d7 2025-12-17 22:20:06,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087cf575bc060ce15e4d7 HTTP/1.1" 502 150 2025-12-17 22:20:06,402 - lava_callback - ERROR - Error processing callback for node 694087cf575bc060ce15e4d7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087cf575bc060ce15e4d7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087cf575bc060ce15e4d7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087cf575bc060ce15e4d7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:06,404 - lava_callback - INFO - Starting async job submit for node 694087f7575bc060ce15e563 2025-12-17 22:20:06,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:06,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f7575bc060ce15e563 HTTP/1.1" 502 150 2025-12-17 22:20:06,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f7575bc060ce15e563'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:06,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f7575bc060ce15e563 2025-12-17 22:20:06,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f7575bc060ce15e563 HTTP/1.1" 502 150 2025-12-17 22:20:06,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f7575bc060ce15e563'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46372 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:20:06,674 - lava_callback - INFO - Received callback for node 6940898b575bc060ce15ead2 INFO: 172.21.0.1:46386 - "POST /node/6940898b575bc060ce15ead2 HTTP/1.0" 202 Accepted 2025-12-17 22:20:07,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940cb75460bc8e265111be3 2025-12-17 22:20:07,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940cb75460bc8e265111be3 HTTP/1.1" 502 150 2025-12-17 22:20:07,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940cb75460bc8e265111be3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:07,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087d2575bc060ce15e4da 2025-12-17 22:20:07,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087d2575bc060ce15e4da HTTP/1.1" 502 150 2025-12-17 22:20:07,484 - lava_callback - ERROR - Error processing callback for node 694087d2575bc060ce15e4da: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087d2575bc060ce15e4da (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087d2575bc060ce15e4da (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087d2575bc060ce15e4da (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:07,486 - lava_callback - INFO - Starting async job submit for node 694087f9575bc060ce15e56a 2025-12-17 22:20:07,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:07,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f9575bc060ce15e56a HTTP/1.1" 502 150 2025-12-17 22:20:07,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f9575bc060ce15e56a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:07,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f9575bc060ce15e56a 2025-12-17 22:20:07,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f9575bc060ce15e56a HTTP/1.1" 502 150 2025-12-17 22:20:07,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f9575bc060ce15e56a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:07,642 - lava_callback - INFO - Received callback for node 6940898d575bc060ce15ead5 INFO: 172.21.0.1:46398 - "POST /node/6940898d575bc060ce15ead5 HTTP/1.0" 202 Accepted 2025-12-17 22:20:08,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f7575bc060ce15e563 2025-12-17 22:20:08,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f7575bc060ce15e563 HTTP/1.1" 502 150 2025-12-17 22:20:08,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f7575bc060ce15e563'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:08,418 - lava_callback - INFO - Received callback for node 6940898e575bc060ce15ead8 INFO: 172.21.0.1:46404 - "POST /node/6940898e575bc060ce15ead8 HTTP/1.0" 202 Accepted 2025-12-17 22:20:08,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f1575bc060ce15e55a 2025-12-17 22:20:08,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f1575bc060ce15e55a HTTP/1.1" 502 150 2025-12-17 22:20:08,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f1575bc060ce15e55a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:09,158 - lava_callback - INFO - Received callback for node 6940898f575bc060ce15eadb INFO: 172.21.0.1:46412 - "POST /node/6940898f575bc060ce15eadb HTTP/1.0" 202 Accepted 2025-12-17 22:20:09,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b4575bc060ce15e4a6 2025-12-17 22:20:09,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b4575bc060ce15e4a6 HTTP/1.1" 502 150 2025-12-17 22:20:09,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b4575bc060ce15e4a6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:09,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f9575bc060ce15e56a 2025-12-17 22:20:09,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f9575bc060ce15e56a HTTP/1.1" 502 150 2025-12-17 22:20:09,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f9575bc060ce15e56a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:09,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f3575bc060ce15e55d 2025-12-17 22:20:09,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f3575bc060ce15e55d HTTP/1.1" 502 150 2025-12-17 22:20:09,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f3575bc060ce15e55d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:09,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e9575bc060ce15e54a 2025-12-17 22:20:09,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e9575bc060ce15e54a HTTP/1.1" 502 150 2025-12-17 22:20:09,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087e9575bc060ce15e54a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:09,976 - lava_callback - INFO - Received callback for node 69408990575bc060ce15eade INFO: 172.21.0.1:43842 - "POST /node/69408990575bc060ce15eade HTTP/1.0" 202 Accepted 2025-12-17 22:20:10,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087eb575bc060ce15e54d 2025-12-17 22:20:10,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087eb575bc060ce15e54d HTTP/1.1" 502 150 2025-12-17 22:20:10,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087eb575bc060ce15e54d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:10,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409174575bc060ce16084f 2025-12-17 22:20:10,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409174575bc060ce16084f HTTP/1.1" 502 150 2025-12-17 22:20:10,706 - lava_callback - ERROR - Error processing callback for node 69409174575bc060ce16084f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409174575bc060ce16084f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409174575bc060ce16084f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409174575bc060ce16084f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:10,708 - lava_callback - INFO - Starting async job submit for node 694087fa575bc060ce15e571 2025-12-17 22:20:10,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:10,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fa575bc060ce15e571 HTTP/1.1" 502 150 2025-12-17 22:20:10,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fa575bc060ce15e571'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:10,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fa575bc060ce15e571 2025-12-17 22:20:10,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fa575bc060ce15e571 HTTP/1.1" 502 150 2025-12-17 22:20:10,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fa575bc060ce15e571'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:10,746 - lava_callback - INFO - Received callback for node 69408992575bc060ce15eae1 INFO: 172.21.0.1:43850 - "POST /node/69408992575bc060ce15eae1 HTTP/1.0" 202 Accepted 2025-12-17 22:20:11,476 - lava_callback - INFO - Received callback for node 69408993575bc060ce15eae5 INFO: 172.21.0.1:43866 - "POST /node/69408993575bc060ce15eae5 HTTP/1.0" 202 Accepted 2025-12-17 22:20:12,215 - lava_callback - INFO - Received callback for node 69408994575bc060ce15eaeb INFO: 172.21.0.1:43880 - "POST /node/69408994575bc060ce15eaeb HTTP/1.0" 202 Accepted 2025-12-17 22:20:12,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f7575bc060ce15e563 2025-12-17 22:20:12,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f7575bc060ce15e563 HTTP/1.1" 502 150 2025-12-17 22:20:12,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f7575bc060ce15e563'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:12,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fa575bc060ce15e571 2025-12-17 22:20:12,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fa575bc060ce15e571 HTTP/1.1" 502 150 2025-12-17 22:20:12,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fa575bc060ce15e571'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:13,154 - lava_callback - INFO - Received callback for node 69408995575bc060ce15eaf0 INFO: 172.21.0.1:43884 - "POST /node/69408995575bc060ce15eaf0 HTTP/1.0" 202 Accepted 2025-12-17 22:20:13,405 - lava_callback - INFO - Received callback for node 694271b81b61f1c6005f71fd INFO: 172.21.0.1:43888 - "POST /node/694271b81b61f1c6005f71fd HTTP/1.0" 202 Accepted 2025-12-17 22:20:13,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f9575bc060ce15e56a 2025-12-17 22:20:13,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f9575bc060ce15e56a HTTP/1.1" 502 150 2025-12-17 22:20:13,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f9575bc060ce15e56a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:13,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ed575bc060ce15e550 2025-12-17 22:20:13,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ed575bc060ce15e550 HTTP/1.1" 502 150 2025-12-17 22:20:13,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ed575bc060ce15e550'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:13,896 - lava_callback - INFO - Received callback for node 69408997575bc060ce15eaf3 INFO: 172.21.0.1:43892 - "POST /node/69408997575bc060ce15eaf3 HTTP/1.0" 202 Accepted 2025-12-17 22:20:14,853 - lava_callback - INFO - Received callback for node 69408998575bc060ce15eb12 INFO: 172.21.0.1:43904 - "POST /node/69408998575bc060ce15eb12 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:43912 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:20:15,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087da575bc060ce15e4e0 2025-12-17 22:20:15,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087da575bc060ce15e4e0 HTTP/1.1" 502 150 2025-12-17 22:20:15,575 - lava_callback - ERROR - Error processing callback for node 694087da575bc060ce15e4e0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087da575bc060ce15e4e0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087da575bc060ce15e4e0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087da575bc060ce15e4e0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:15,577 - lava_callback - INFO - Starting async job submit for node 694087fc575bc060ce15e574 2025-12-17 22:20:15,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:15,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fc575bc060ce15e574 HTTP/1.1" 502 150 2025-12-17 22:20:15,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fc575bc060ce15e574'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:15,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fc575bc060ce15e574 2025-12-17 22:20:15,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fc575bc060ce15e574 HTTP/1.1" 502 150 2025-12-17 22:20:15,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fc575bc060ce15e574'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:15,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940927c575bc060ce160acb 2025-12-17 22:20:15,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940927c575bc060ce160acb HTTP/1.1" 502 150 2025-12-17 22:20:15,796 - lava_callback - ERROR - Error processing callback for node 6940927c575bc060ce160acb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940927c575bc060ce160acb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940927c575bc060ce160acb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940927c575bc060ce160acb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:15,798 - lava_callback - INFO - Starting async job submit for node 694087fd575bc060ce15e577 2025-12-17 22:20:15,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:15,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fd575bc060ce15e577 HTTP/1.1" 502 150 2025-12-17 22:20:15,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fd575bc060ce15e577'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:15,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fd575bc060ce15e577 2025-12-17 22:20:15,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fd575bc060ce15e577 HTTP/1.1" 502 150 2025-12-17 22:20:15,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fd575bc060ce15e577'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:15,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e0575bc060ce15e4e7 2025-12-17 22:20:15,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e0575bc060ce15e4e7 HTTP/1.1" 502 150 2025-12-17 22:20:15,922 - lava_callback - ERROR - Error processing callback for node 694087e0575bc060ce15e4e7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087e0575bc060ce15e4e7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087e0575bc060ce15e4e7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087e0575bc060ce15e4e7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:15,924 - lava_callback - INFO - Starting async job submit for node 694087fe575bc060ce15e57a 2025-12-17 22:20:15,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:15,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fe575bc060ce15e57a HTTP/1.1" 502 150 2025-12-17 22:20:15,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fe575bc060ce15e57a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:15,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fe575bc060ce15e57a 2025-12-17 22:20:15,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fe575bc060ce15e57a HTTP/1.1" 502 150 2025-12-17 22:20:15,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fe575bc060ce15e57a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:15,971 - lava_callback - INFO - Received callback for node 6940899b575bc060ce15eb49 INFO: 172.21.0.1:43928 - "POST /node/6940899b575bc060ce15eb49 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:43932 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:20:16,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fa575bc060ce15e571 2025-12-17 22:20:16,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fa575bc060ce15e571 HTTP/1.1" 502 150 2025-12-17 22:20:16,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fa575bc060ce15e571'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:16,782 - lava_callback - INFO - Received callback for node 6940899c575bc060ce15eb5c INFO: 172.21.0.1:43936 - "POST /node/6940899c575bc060ce15eb5c HTTP/1.0" 202 Accepted 2025-12-17 22:20:16,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f1575bc060ce15e55a 2025-12-17 22:20:16,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f1575bc060ce15e55a HTTP/1.1" 502 150 2025-12-17 22:20:16,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f1575bc060ce15e55a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:16,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e2575bc060ce15e4ea 2025-12-17 22:20:16,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e2575bc060ce15e4ea HTTP/1.1" 502 150 2025-12-17 22:20:16,960 - lava_callback - ERROR - Error processing callback for node 694087e2575bc060ce15e4ea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087e2575bc060ce15e4ea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087e2575bc060ce15e4ea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087e2575bc060ce15e4ea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:16,961 - lava_callback - INFO - Starting async job submit for node 69408800575bc060ce15e57d 2025-12-17 22:20:16,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:16,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408800575bc060ce15e57d HTTP/1.1" 502 150 2025-12-17 22:20:16,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408800575bc060ce15e57d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:16,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408800575bc060ce15e57d 2025-12-17 22:20:16,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408800575bc060ce15e57d HTTP/1.1" 502 150 2025-12-17 22:20:16,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408800575bc060ce15e57d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:17,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e6575bc060ce15e51d 2025-12-17 22:20:17,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e6575bc060ce15e51d HTTP/1.1" 502 150 2025-12-17 22:20:17,018 - lava_callback - ERROR - Error processing callback for node 694087e6575bc060ce15e51d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087e6575bc060ce15e51d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087e6575bc060ce15e51d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087e6575bc060ce15e51d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:17,020 - lava_callback - INFO - Starting async job submit for node 69408801575bc060ce15e580 2025-12-17 22:20:17,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:17,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408801575bc060ce15e580 HTTP/1.1" 502 150 2025-12-17 22:20:17,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408801575bc060ce15e580'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:17,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408801575bc060ce15e580 2025-12-17 22:20:17,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408801575bc060ce15e580 HTTP/1.1" 502 150 2025-12-17 22:20:17,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408801575bc060ce15e580'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:17,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b4575bc060ce15e4a6 2025-12-17 22:20:17,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b4575bc060ce15e4a6 HTTP/1.1" 502 150 2025-12-17 22:20:17,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087b4575bc060ce15e4a6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:17,517 - lava_callback - INFO - Received callback for node 6940899e575bc060ce15eb5f INFO: 172.21.0.1:43948 - "POST /node/6940899e575bc060ce15eb5f HTTP/1.0" 202 Accepted 2025-12-17 22:20:17,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fc575bc060ce15e574 2025-12-17 22:20:17,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fc575bc060ce15e574 HTTP/1.1" 502 150 2025-12-17 22:20:17,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fc575bc060ce15e574'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:17,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fd575bc060ce15e577 2025-12-17 22:20:17,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fd575bc060ce15e577 HTTP/1.1" 502 150 2025-12-17 22:20:17,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fd575bc060ce15e577'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:17,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f3575bc060ce15e55d 2025-12-17 22:20:17,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f3575bc060ce15e55d HTTP/1.1" 502 150 2025-12-17 22:20:17,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f3575bc060ce15e55d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:17,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fe575bc060ce15e57a 2025-12-17 22:20:17,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fe575bc060ce15e57a HTTP/1.1" 502 150 2025-12-17 22:20:17,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fe575bc060ce15e57a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:18,248 - lava_callback - INFO - Received callback for node 6940899f575bc060ce15eb62 INFO: 172.21.0.1:43956 - "POST /node/6940899f575bc060ce15eb62 HTTP/1.0" 202 Accepted 2025-12-17 22:20:18,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408800575bc060ce15e57d 2025-12-17 22:20:18,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408800575bc060ce15e57d HTTP/1.1" 502 150 2025-12-17 22:20:18,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408800575bc060ce15e57d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:18,986 - lava_callback - INFO - Received callback for node 694089a0575bc060ce15eb65 INFO: 172.21.0.1:43958 - "POST /node/694089a0575bc060ce15eb65 HTTP/1.0" 202 Accepted 2025-12-17 22:20:19,028 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408801575bc060ce15e580 2025-12-17 22:20:19,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408801575bc060ce15e580 HTTP/1.1" 502 150 2025-12-17 22:20:19,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408801575bc060ce15e580'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:19,782 - lava_callback - INFO - Received callback for node 694089a2575bc060ce15eb68 INFO: 172.21.0.1:60972 - "POST /node/694089a2575bc060ce15eb68 HTTP/1.0" 202 Accepted 2025-12-17 22:20:20,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ef575bc060ce15e555 HTTP/1.1" 502 150 2025-12-17 22:20:20,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ef575bc060ce15e555'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:20,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f7575bc060ce15e563 2025-12-17 22:20:20,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f7575bc060ce15e563 HTTP/1.1" 502 150 2025-12-17 22:20:20,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f7575bc060ce15e563'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:20,515 - lava_callback - INFO - Received callback for node 694089a3575bc060ce15eb70 INFO: 172.21.0.1:60988 - "POST /node/694089a3575bc060ce15eb70 HTTP/1.0" 202 Accepted 2025-12-17 22:20:21,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f9575bc060ce15e56a 2025-12-17 22:20:21,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f9575bc060ce15e56a HTTP/1.1" 502 150 2025-12-17 22:20:21,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087f9575bc060ce15e56a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:21,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fc575bc060ce15e574 2025-12-17 22:20:21,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fc575bc060ce15e574 HTTP/1.1" 502 150 2025-12-17 22:20:21,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fc575bc060ce15e574'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:21,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fd575bc060ce15e577 2025-12-17 22:20:21,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fd575bc060ce15e577 HTTP/1.1" 502 150 2025-12-17 22:20:21,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fd575bc060ce15e577'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:21,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fe575bc060ce15e57a 2025-12-17 22:20:21,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fe575bc060ce15e57a HTTP/1.1" 502 150 2025-12-17 22:20:21,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fe575bc060ce15e57a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:22,082 - lava_callback - INFO - Received callback for node 694089a4575bc060ce15eb73 INFO: 172.21.0.1:60992 - "POST /node/694089a4575bc060ce15eb73 HTTP/1.0" 202 Accepted 2025-12-17 22:20:22,836 - lava_callback - INFO - Received callback for node 694089a5575bc060ce15eb76 INFO: 172.21.0.1:32772 - "POST /node/694089a5575bc060ce15eb76 HTTP/1.0" 202 Accepted 2025-12-17 22:20:22,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408800575bc060ce15e57d 2025-12-17 22:20:23,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408801575bc060ce15e580 2025-12-17 22:20:23,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408801575bc060ce15e580 HTTP/1.1" 502 150 2025-12-17 22:20:23,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408801575bc060ce15e580'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:23,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940cb75460bc8e265111be3 2025-12-17 22:20:23,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940cb75460bc8e265111be3 HTTP/1.1" 502 150 2025-12-17 22:20:23,452 - lava_callback - ERROR - Error processing callback for node 6940cb75460bc8e265111be3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940cb75460bc8e265111be3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940cb75460bc8e265111be3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940cb75460bc8e265111be3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:23,454 - lava_callback - INFO - Starting async job submit for node 69408802575bc060ce15e585 2025-12-17 22:20:23,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:23,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408802575bc060ce15e585 HTTP/1.1" 502 150 2025-12-17 22:20:23,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408802575bc060ce15e585'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:23,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408802575bc060ce15e585 2025-12-17 22:20:23,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408802575bc060ce15e585 HTTP/1.1" 502 150 2025-12-17 22:20:23,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408802575bc060ce15e585'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:24,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ef575bc060ce15e555 2025-12-17 22:20:24,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ef575bc060ce15e555 HTTP/1.1" 502 150 2025-12-17 22:20:24,274 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ef575bc060ce15e555'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:24,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fa575bc060ce15e571 2025-12-17 22:20:24,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fa575bc060ce15e571 HTTP/1.1" 502 150 2025-12-17 22:20:24,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fa575bc060ce15e571'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:25,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408802575bc060ce15e585 2025-12-17 22:20:25,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408802575bc060ce15e585 HTTP/1.1" 502 150 2025-12-17 22:20:25,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408802575bc060ce15e585'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:25,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087e9575bc060ce15e54a 2025-12-17 22:20:25,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087e9575bc060ce15e54a HTTP/1.1" 502 150 2025-12-17 22:20:25,930 - lava_callback - ERROR - Error processing callback for node 694087e9575bc060ce15e54a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087e9575bc060ce15e54a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087e9575bc060ce15e54a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087e9575bc060ce15e54a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:25,932 - lava_callback - INFO - Starting async job submit for node 69408804575bc060ce15e58c 2025-12-17 22:20:25,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:25,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408804575bc060ce15e58c HTTP/1.1" 502 150 2025-12-17 22:20:25,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408804575bc060ce15e58c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:25,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408804575bc060ce15e58c 2025-12-17 22:20:25,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408804575bc060ce15e58c HTTP/1.1" 502 150 2025-12-17 22:20:25,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408804575bc060ce15e58c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:32782 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:20:26,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087eb575bc060ce15e54d 2025-12-17 22:20:26,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087eb575bc060ce15e54d HTTP/1.1" 502 150 2025-12-17 22:20:26,684 - lava_callback - ERROR - Error processing callback for node 694087eb575bc060ce15e54d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087eb575bc060ce15e54d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087eb575bc060ce15e54d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087eb575bc060ce15e54d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:26,686 - lava_callback - INFO - Starting async job submit for node 69408805575bc060ce15e593 2025-12-17 22:20:26,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:26,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408805575bc060ce15e593 HTTP/1.1" 502 150 2025-12-17 22:20:26,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408805575bc060ce15e593'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:26,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408805575bc060ce15e593 2025-12-17 22:20:26,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408805575bc060ce15e593 HTTP/1.1" 502 150 2025-12-17 22:20:26,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408805575bc060ce15e593'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:27,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408804575bc060ce15e58c 2025-12-17 22:20:27,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408804575bc060ce15e58c HTTP/1.1" 502 150 2025-12-17 22:20:27,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408804575bc060ce15e58c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:28,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408805575bc060ce15e593 2025-12-17 22:20:28,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408805575bc060ce15e593 HTTP/1.1" 502 150 2025-12-17 22:20:28,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408805575bc060ce15e593'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:29,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408802575bc060ce15e585 2025-12-17 22:20:29,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408802575bc060ce15e585 HTTP/1.1" 502 150 2025-12-17 22:20:29,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408802575bc060ce15e585'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:29,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fc575bc060ce15e574 2025-12-17 22:20:29,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fc575bc060ce15e574 HTTP/1.1" 502 150 2025-12-17 22:20:29,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fc575bc060ce15e574'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:29,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ed575bc060ce15e550 2025-12-17 22:20:29,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ed575bc060ce15e550 HTTP/1.1" 502 150 2025-12-17 22:20:29,772 - lava_callback - ERROR - Error processing callback for node 694087ed575bc060ce15e550: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ed575bc060ce15e550 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ed575bc060ce15e550 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ed575bc060ce15e550 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:29,774 - lava_callback - INFO - Starting async job submit for node 69408806575bc060ce15e59a 2025-12-17 22:20:29,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:29,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408806575bc060ce15e59a HTTP/1.1" 502 150 2025-12-17 22:20:29,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408806575bc060ce15e59a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:29,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408806575bc060ce15e59a 2025-12-17 22:20:29,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408806575bc060ce15e59a HTTP/1.1" 502 150 2025-12-17 22:20:29,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408806575bc060ce15e59a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:29,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fd575bc060ce15e577 2025-12-17 22:20:29,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fd575bc060ce15e577 HTTP/1.1" 502 150 2025-12-17 22:20:29,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fd575bc060ce15e577'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:29,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fe575bc060ce15e57a 2025-12-17 22:20:29,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fe575bc060ce15e57a HTTP/1.1" 502 150 2025-12-17 22:20:29,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087fe575bc060ce15e57a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:31,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408801575bc060ce15e580 2025-12-17 22:20:31,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408806575bc060ce15e59a 2025-12-17 22:20:31,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408806575bc060ce15e59a HTTP/1.1" 502 150 2025-12-17 22:20:31,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408806575bc060ce15e59a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:31,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408804575bc060ce15e58c 2025-12-17 22:20:31,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408804575bc060ce15e58c HTTP/1.1" 502 150 2025-12-17 22:20:31,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408804575bc060ce15e58c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:32,274 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ef575bc060ce15e555 2025-12-17 22:20:32,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ef575bc060ce15e555 HTTP/1.1" 502 150 2025-12-17 22:20:32,275 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694087ef575bc060ce15e555'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:32,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408805575bc060ce15e593 2025-12-17 22:20:32,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408805575bc060ce15e593 HTTP/1.1" 502 150 2025-12-17 22:20:32,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408805575bc060ce15e593'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:32,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f1575bc060ce15e55a 2025-12-17 22:20:32,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f1575bc060ce15e55a HTTP/1.1" 502 150 2025-12-17 22:20:32,814 - lava_callback - ERROR - Error processing callback for node 694087f1575bc060ce15e55a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087f1575bc060ce15e55a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087f1575bc060ce15e55a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087f1575bc060ce15e55a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:32,816 - lava_callback - INFO - Starting async job submit for node 69408808575bc060ce15e59d 2025-12-17 22:20:32,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:32,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408808575bc060ce15e59d HTTP/1.1" 502 150 2025-12-17 22:20:32,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408808575bc060ce15e59d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:32,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408808575bc060ce15e59d 2025-12-17 22:20:32,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408808575bc060ce15e59d HTTP/1.1" 502 150 2025-12-17 22:20:32,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408808575bc060ce15e59d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:33,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087b4575bc060ce15e4a6 2025-12-17 22:20:33,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087b4575bc060ce15e4a6 HTTP/1.1" 502 150 2025-12-17 22:20:33,433 - lava_callback - ERROR - Error processing callback for node 694087b4575bc060ce15e4a6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b4575bc060ce15e4a6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b4575bc060ce15e4a6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087b4575bc060ce15e4a6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:33,435 - lava_callback - INFO - Starting async job submit for node 69408809575bc060ce15e5a5 2025-12-17 22:20:33,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:33,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408809575bc060ce15e5a5 HTTP/1.1" 502 150 2025-12-17 22:20:33,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408809575bc060ce15e5a5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:33,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408809575bc060ce15e5a5 2025-12-17 22:20:33,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408809575bc060ce15e5a5 HTTP/1.1" 502 150 2025-12-17 22:20:33,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408809575bc060ce15e5a5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:33,472 - lava_callback - INFO - Received callback for node 694089a7575bc060ce15eb80 INFO: 172.21.0.1:43156 - "POST /node/694089a7575bc060ce15eb80 HTTP/1.0" 202 Accepted 2025-12-17 22:20:33,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f3575bc060ce15e55d 2025-12-17 22:20:33,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f3575bc060ce15e55d HTTP/1.1" 502 150 2025-12-17 22:20:33,901 - lava_callback - ERROR - Error processing callback for node 694087f3575bc060ce15e55d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087f3575bc060ce15e55d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087f3575bc060ce15e55d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087f3575bc060ce15e55d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:33,903 - lava_callback - INFO - Starting async job submit for node 6940880a575bc060ce15e5a8 2025-12-17 22:20:33,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:33,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880a575bc060ce15e5a8 HTTP/1.1" 502 150 2025-12-17 22:20:33,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880a575bc060ce15e5a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:33,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880a575bc060ce15e5a8 2025-12-17 22:20:33,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880a575bc060ce15e5a8 HTTP/1.1" 502 150 2025-12-17 22:20:33,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880a575bc060ce15e5a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:34,379 - lava_callback - INFO - Received callback for node 694089ab575bc060ce15eb90 INFO: 172.21.0.1:43160 - "POST /node/694089ab575bc060ce15eb90 HTTP/1.0" 202 Accepted 2025-12-17 22:20:34,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408808575bc060ce15e59d 2025-12-17 22:20:34,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408808575bc060ce15e59d HTTP/1.1" 502 150 2025-12-17 22:20:34,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408808575bc060ce15e59d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:35,042 - lava_callback - INFO - Received callback for node 694089ae575bc060ce15eb98 INFO: 172.21.0.1:43168 - "POST /node/694089ae575bc060ce15eb98 HTTP/1.0" 202 Accepted 2025-12-17 22:20:35,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408809575bc060ce15e5a5 2025-12-17 22:20:35,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408809575bc060ce15e5a5 HTTP/1.1" 502 150 2025-12-17 22:20:35,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408809575bc060ce15e5a5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:35,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408806575bc060ce15e59a 2025-12-17 22:20:35,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408806575bc060ce15e59a HTTP/1.1" 502 150 2025-12-17 22:20:35,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408806575bc060ce15e59a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:35,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880a575bc060ce15e5a8 2025-12-17 22:20:35,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880a575bc060ce15e5a8 HTTP/1.1" 502 150 2025-12-17 22:20:35,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880a575bc060ce15e5a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:36,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f7575bc060ce15e563 2025-12-17 22:20:36,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f7575bc060ce15e563 HTTP/1.1" 502 150 2025-12-17 22:20:36,418 - lava_callback - ERROR - Error processing callback for node 694087f7575bc060ce15e563: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087f7575bc060ce15e563 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087f7575bc060ce15e563 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087f7575bc060ce15e563 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:36,420 - lava_callback - INFO - Starting async job submit for node 6940880c575bc060ce15e5ab 2025-12-17 22:20:36,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:36,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880c575bc060ce15e5ab HTTP/1.1" 502 150 2025-12-17 22:20:36,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880c575bc060ce15e5ab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:36,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880c575bc060ce15e5ab 2025-12-17 22:20:36,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880c575bc060ce15e5ab HTTP/1.1" 502 150 2025-12-17 22:20:36,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880c575bc060ce15e5ab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43182 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:20:37,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408802575bc060ce15e585 2025-12-17 22:20:37,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087f9575bc060ce15e56a 2025-12-17 22:20:37,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087f9575bc060ce15e56a HTTP/1.1" 502 150 2025-12-17 22:20:37,498 - lava_callback - ERROR - Error processing callback for node 694087f9575bc060ce15e56a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087f9575bc060ce15e56a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087f9575bc060ce15e56a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087f9575bc060ce15e56a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:37,500 - lava_callback - INFO - Starting async job submit for node 6940880d575bc060ce15e5b2 2025-12-17 22:20:37,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:37,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880d575bc060ce15e5b2 HTTP/1.1" 502 150 2025-12-17 22:20:37,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880d575bc060ce15e5b2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:37,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880d575bc060ce15e5b2 2025-12-17 22:20:37,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880d575bc060ce15e5b2 HTTP/1.1" 502 150 2025-12-17 22:20:37,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880d575bc060ce15e5b2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:38,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408800575bc060ce15e57d HTTP/1.1" 502 150 2025-12-17 22:20:38,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408800575bc060ce15e57d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:38,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880c575bc060ce15e5ab 2025-12-17 22:20:38,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880c575bc060ce15e5ab HTTP/1.1" 502 150 2025-12-17 22:20:38,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880c575bc060ce15e5ab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:38,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408808575bc060ce15e59d 2025-12-17 22:20:38,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408808575bc060ce15e59d HTTP/1.1" 502 150 2025-12-17 22:20:38,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408808575bc060ce15e59d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:39,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408809575bc060ce15e5a5 2025-12-17 22:20:39,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408809575bc060ce15e5a5 HTTP/1.1" 502 150 2025-12-17 22:20:39,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408809575bc060ce15e5a5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:39,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880d575bc060ce15e5b2 2025-12-17 22:20:39,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880d575bc060ce15e5b2 HTTP/1.1" 502 150 2025-12-17 22:20:39,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880d575bc060ce15e5b2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:39,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880a575bc060ce15e5a8 2025-12-17 22:20:39,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880a575bc060ce15e5a8 HTTP/1.1" 502 150 2025-12-17 22:20:39,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880a575bc060ce15e5a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:39,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408804575bc060ce15e58c 2025-12-17 22:20:39,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408804575bc060ce15e58c HTTP/1.1" 502 150 2025-12-17 22:20:39,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408804575bc060ce15e58c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:40,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408805575bc060ce15e593 2025-12-17 22:20:40,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408805575bc060ce15e593 HTTP/1.1" 502 150 2025-12-17 22:20:40,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408805575bc060ce15e593'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:40,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fa575bc060ce15e571 2025-12-17 22:20:40,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fa575bc060ce15e571 HTTP/1.1" 502 150 2025-12-17 22:20:40,721 - lava_callback - ERROR - Error processing callback for node 694087fa575bc060ce15e571: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087fa575bc060ce15e571 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087fa575bc060ce15e571 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087fa575bc060ce15e571 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:40,723 - lava_callback - INFO - Starting async job submit for node 6940880e575bc060ce15e5b5 2025-12-17 22:20:40,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:40,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880e575bc060ce15e5b5 HTTP/1.1" 502 150 2025-12-17 22:20:40,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880e575bc060ce15e5b5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:40,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880e575bc060ce15e5b5 2025-12-17 22:20:40,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880e575bc060ce15e5b5 HTTP/1.1" 502 150 2025-12-17 22:20:40,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880e575bc060ce15e5b5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:42,179 - lava_callback - INFO - Received callback for node 694089a7575bc060ce15eb7b INFO: 172.21.0.1:59248 - "POST /node/694089a7575bc060ce15eb7b HTTP/1.0" 202 Accepted 2025-12-17 22:20:42,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880c575bc060ce15e5ab 2025-12-17 22:20:42,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880c575bc060ce15e5ab HTTP/1.1" 502 150 2025-12-17 22:20:42,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880c575bc060ce15e5ab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:42,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880e575bc060ce15e5b5 2025-12-17 22:20:42,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880e575bc060ce15e5b5 HTTP/1.1" 502 150 2025-12-17 22:20:42,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880e575bc060ce15e5b5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:43,006 - lava_callback - INFO - Received callback for node 694089aa575bc060ce15eb8b INFO: 172.21.0.1:59262 - "POST /node/694089aa575bc060ce15eb8b HTTP/1.0" 202 Accepted 2025-12-17 22:20:43,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880d575bc060ce15e5b2 2025-12-17 22:20:43,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880d575bc060ce15e5b2 HTTP/1.1" 502 150 2025-12-17 22:20:43,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880d575bc060ce15e5b2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:43,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408806575bc060ce15e59a 2025-12-17 22:20:43,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408806575bc060ce15e59a HTTP/1.1" 502 150 2025-12-17 22:20:43,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408806575bc060ce15e59a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:44,386 - lava_callback - INFO - Received callback for node 694089b5575bc060ce15eba8 INFO: 172.21.0.1:59274 - "POST /node/694089b5575bc060ce15eba8 HTTP/1.0" 202 Accepted 2025-12-17 22:20:45,048 - lava_callback - INFO - Received callback for node 694089b6575bc060ce15ebab INFO: 172.21.0.1:59290 - "POST /node/694089b6575bc060ce15ebab HTTP/1.0" 202 Accepted 2025-12-17 22:20:45,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fc575bc060ce15e574 2025-12-17 22:20:45,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fc575bc060ce15e574 HTTP/1.1" 502 150 2025-12-17 22:20:45,590 - lava_callback - ERROR - Error processing callback for node 694087fc575bc060ce15e574: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087fc575bc060ce15e574 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087fc575bc060ce15e574 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087fc575bc060ce15e574 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:45,592 - lava_callback - INFO - Starting async job submit for node 69408810575bc060ce15e5b8 2025-12-17 22:20:45,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:45,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408810575bc060ce15e5b8 HTTP/1.1" 502 150 2025-12-17 22:20:45,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408810575bc060ce15e5b8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:45,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408810575bc060ce15e5b8 2025-12-17 22:20:45,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408810575bc060ce15e5b8 HTTP/1.1" 502 150 2025-12-17 22:20:45,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408810575bc060ce15e5b8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:45,698 - lava_callback - INFO - Received callback for node 694089b8575bc060ce15ebae INFO: 172.21.0.1:59294 - "POST /node/694089b8575bc060ce15ebae HTTP/1.0" 202 Accepted 2025-12-17 22:20:45,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fd575bc060ce15e577 2025-12-17 22:20:45,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fd575bc060ce15e577 HTTP/1.1" 502 150 2025-12-17 22:20:45,812 - lava_callback - ERROR - Error processing callback for node 694087fd575bc060ce15e577: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087fd575bc060ce15e577 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087fd575bc060ce15e577 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087fd575bc060ce15e577 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:45,815 - lava_callback - INFO - Starting async job submit for node 69408811575bc060ce15e5bb 2025-12-17 22:20:45,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:45,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408811575bc060ce15e5bb HTTP/1.1" 502 150 2025-12-17 22:20:45,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408811575bc060ce15e5bb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:45,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408811575bc060ce15e5bb 2025-12-17 22:20:45,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408811575bc060ce15e5bb HTTP/1.1" 502 150 2025-12-17 22:20:45,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408811575bc060ce15e5bb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:45,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087fe575bc060ce15e57a 2025-12-17 22:20:45,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087fe575bc060ce15e57a HTTP/1.1" 502 150 2025-12-17 22:20:45,937 - lava_callback - ERROR - Error processing callback for node 694087fe575bc060ce15e57a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087fe575bc060ce15e57a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087fe575bc060ce15e57a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087fe575bc060ce15e57a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:45,940 - lava_callback - INFO - Starting async job submit for node 69408812575bc060ce15e5be 2025-12-17 22:20:45,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:45,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408812575bc060ce15e5be HTTP/1.1" 502 150 2025-12-17 22:20:45,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408812575bc060ce15e5be'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:45,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408812575bc060ce15e5be 2025-12-17 22:20:45,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408812575bc060ce15e5be HTTP/1.1" 502 150 2025-12-17 22:20:45,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408812575bc060ce15e5be'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:46,336 - lava_callback - INFO - Received callback for node 694089b9575bc060ce15ebb1 INFO: 172.21.0.1:59310 - "POST /node/694089b9575bc060ce15ebb1 HTTP/1.0" 202 Accepted 2025-12-17 22:20:46,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408801575bc060ce15e580 HTTP/1.1" 502 150 2025-12-17 22:20:46,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408801575bc060ce15e580'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:46,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408800575bc060ce15e57d 2025-12-17 22:20:46,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408800575bc060ce15e57d HTTP/1.1" 502 150 2025-12-17 22:20:46,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408800575bc060ce15e57d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59320 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:20:46,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880e575bc060ce15e5b5 2025-12-17 22:20:46,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880e575bc060ce15e5b5 HTTP/1.1" 502 150 2025-12-17 22:20:46,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880e575bc060ce15e5b5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:46,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408808575bc060ce15e59d 2025-12-17 22:20:46,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408808575bc060ce15e59d HTTP/1.1" 502 150 2025-12-17 22:20:46,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408808575bc060ce15e59d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:46,969 - lava_callback - INFO - Received callback for node 694089ba575bc060ce15ebb4 INFO: 172.21.0.1:59336 - "POST /node/694089ba575bc060ce15ebb4 HTTP/1.0" 202 Accepted 2025-12-17 22:20:47,447 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408809575bc060ce15e5a5 2025-12-17 22:20:47,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408809575bc060ce15e5a5 HTTP/1.1" 502 150 2025-12-17 22:20:47,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408809575bc060ce15e5a5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:47,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408810575bc060ce15e5b8 2025-12-17 22:20:47,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408810575bc060ce15e5b8 HTTP/1.1" 502 150 2025-12-17 22:20:47,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408810575bc060ce15e5b8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:47,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408811575bc060ce15e5bb 2025-12-17 22:20:47,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408811575bc060ce15e5bb HTTP/1.1" 502 150 2025-12-17 22:20:47,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408811575bc060ce15e5bb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:47,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880a575bc060ce15e5a8 2025-12-17 22:20:47,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880a575bc060ce15e5a8 HTTP/1.1" 502 150 2025-12-17 22:20:47,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880a575bc060ce15e5a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:47,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408812575bc060ce15e5be 2025-12-17 22:20:47,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408812575bc060ce15e5be HTTP/1.1" 502 150 2025-12-17 22:20:47,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408812575bc060ce15e5be'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:48,275 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694087ef575bc060ce15e555 2025-12-17 22:20:48,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694087ef575bc060ce15e555 HTTP/1.1" 502 150 2025-12-17 22:20:48,277 - lava_callback - ERROR - Error processing callback for node 694087ef575bc060ce15e555: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ef575bc060ce15e555 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ef575bc060ce15e555 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694087ef575bc060ce15e555 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:48,279 - lava_callback - INFO - Starting async job submit for node 69408813575bc060ce15e5c1 2025-12-17 22:20:48,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:48,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408813575bc060ce15e5c1 HTTP/1.1" 502 150 2025-12-17 22:20:48,289 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408813575bc060ce15e5c1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:48,289 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408813575bc060ce15e5c1 2025-12-17 22:20:48,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408813575bc060ce15e5c1 HTTP/1.1" 502 150 2025-12-17 22:20:48,290 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408813575bc060ce15e5c1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:49,231 - lava_callback - INFO - Received callback for node 694089b2575bc060ce15eba2 INFO: 172.21.0.1:59342 - "POST /node/694089b2575bc060ce15eba2 HTTP/1.0" 202 Accepted 2025-12-17 22:20:49,894 - lava_callback - INFO - Received callback for node 694089b3575bc060ce15eba5 INFO: 172.21.0.1:38610 - "POST /node/694089b3575bc060ce15eba5 HTTP/1.0" 202 Accepted 2025-12-17 22:20:50,290 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408813575bc060ce15e5c1 2025-12-17 22:20:50,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880c575bc060ce15e5ab 2025-12-17 22:20:50,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880c575bc060ce15e5ab HTTP/1.1" 502 150 2025-12-17 22:20:50,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880c575bc060ce15e5ab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:50,547 - lava_callback - INFO - Received callback for node 694089bd575bc060ce15ebc2 INFO: 172.21.0.1:38616 - "POST /node/694089bd575bc060ce15ebc2 HTTP/1.0" 202 Accepted 2025-12-17 22:20:51,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880d575bc060ce15e5b2 2025-12-17 22:20:51,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880d575bc060ce15e5b2 HTTP/1.1" 502 150 2025-12-17 22:20:51,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880d575bc060ce15e5b2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:51,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408810575bc060ce15e5b8 2025-12-17 22:20:51,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408810575bc060ce15e5b8 HTTP/1.1" 502 150 2025-12-17 22:20:51,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408810575bc060ce15e5b8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:51,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408811575bc060ce15e5bb 2025-12-17 22:20:51,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408811575bc060ce15e5bb HTTP/1.1" 502 150 2025-12-17 22:20:51,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408811575bc060ce15e5bb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:51,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408812575bc060ce15e5be 2025-12-17 22:20:51,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408812575bc060ce15e5be HTTP/1.1" 502 150 2025-12-17 22:20:51,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408812575bc060ce15e5be'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:52,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408802575bc060ce15e585 HTTP/1.1" 502 150 2025-12-17 22:20:52,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408802575bc060ce15e585'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:54,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880e575bc060ce15e5b5 2025-12-17 22:20:54,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880e575bc060ce15e5b5 HTTP/1.1" 502 150 2025-12-17 22:20:54,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940880e575bc060ce15e5b5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:55,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408804575bc060ce15e58c 2025-12-17 22:20:55,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408804575bc060ce15e58c HTTP/1.1" 502 150 2025-12-17 22:20:55,947 - lava_callback - ERROR - Error processing callback for node 69408804575bc060ce15e58c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408804575bc060ce15e58c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408804575bc060ce15e58c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408804575bc060ce15e58c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:55,949 - lava_callback - INFO - Starting async job submit for node 69408815575bc060ce15e5c4 2025-12-17 22:20:55,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:55,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408815575bc060ce15e5c4 HTTP/1.1" 502 150 2025-12-17 22:20:55,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408815575bc060ce15e5c4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:55,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408815575bc060ce15e5c4 2025-12-17 22:20:55,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408815575bc060ce15e5c4 HTTP/1.1" 502 150 2025-12-17 22:20:55,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408815575bc060ce15e5c4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38622 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:20:56,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408805575bc060ce15e593 2025-12-17 22:20:56,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408805575bc060ce15e593 HTTP/1.1" 502 150 2025-12-17 22:20:56,703 - lava_callback - ERROR - Error processing callback for node 69408805575bc060ce15e593: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408805575bc060ce15e593 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408805575bc060ce15e593 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408805575bc060ce15e593 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:56,705 - lava_callback - INFO - Starting async job submit for node 6940881a575bc060ce15e5d5 2025-12-17 22:20:56,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:56,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881a575bc060ce15e5d5 HTTP/1.1" 502 150 2025-12-17 22:20:56,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881a575bc060ce15e5d5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:56,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881a575bc060ce15e5d5 2025-12-17 22:20:56,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881a575bc060ce15e5d5 HTTP/1.1" 502 150 2025-12-17 22:20:56,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881a575bc060ce15e5d5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:57,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408815575bc060ce15e5c4 2025-12-17 22:20:57,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408815575bc060ce15e5c4 HTTP/1.1" 502 150 2025-12-17 22:20:57,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408815575bc060ce15e5c4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:58,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881a575bc060ce15e5d5 2025-12-17 22:20:58,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881a575bc060ce15e5d5 HTTP/1.1" 502 150 2025-12-17 22:20:58,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881a575bc060ce15e5d5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:59,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408810575bc060ce15e5b8 2025-12-17 22:20:59,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408810575bc060ce15e5b8 HTTP/1.1" 502 150 2025-12-17 22:20:59,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408810575bc060ce15e5b8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:59,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408806575bc060ce15e59a 2025-12-17 22:20:59,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408806575bc060ce15e59a HTTP/1.1" 502 150 2025-12-17 22:20:59,787 - lava_callback - ERROR - Error processing callback for node 69408806575bc060ce15e59a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408806575bc060ce15e59a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408806575bc060ce15e59a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408806575bc060ce15e59a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:20:59,789 - lava_callback - INFO - Starting async job submit for node 6940881b575bc060ce15e5dc 2025-12-17 22:20:59,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:20:59,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881b575bc060ce15e5dc HTTP/1.1" 502 150 2025-12-17 22:20:59,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881b575bc060ce15e5dc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:59,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881b575bc060ce15e5dc 2025-12-17 22:20:59,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881b575bc060ce15e5dc HTTP/1.1" 502 150 2025-12-17 22:20:59,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881b575bc060ce15e5dc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:59,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408811575bc060ce15e5bb 2025-12-17 22:20:59,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408811575bc060ce15e5bb HTTP/1.1" 502 150 2025-12-17 22:20:59,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408811575bc060ce15e5bb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:20:59,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408812575bc060ce15e5be 2025-12-17 22:20:59,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408812575bc060ce15e5be HTTP/1.1" 502 150 2025-12-17 22:20:59,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408812575bc060ce15e5be'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:01,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881b575bc060ce15e5dc 2025-12-17 22:21:01,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881b575bc060ce15e5dc HTTP/1.1" 502 150 2025-12-17 22:21:01,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881b575bc060ce15e5dc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:01,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408815575bc060ce15e5c4 2025-12-17 22:21:01,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408815575bc060ce15e5c4 HTTP/1.1" 502 150 2025-12-17 22:21:01,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408815575bc060ce15e5c4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53560 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:21:02,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408801575bc060ce15e580 2025-12-17 22:21:02,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408801575bc060ce15e580 HTTP/1.1" 502 150 2025-12-17 22:21:02,386 - lava_callback - ERROR - Error processing callback for node 69408801575bc060ce15e580: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408801575bc060ce15e580 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408801575bc060ce15e580 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408801575bc060ce15e580 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:02,389 - lava_callback - INFO - Starting async job submit for node 6940881d575bc060ce15e5df 2025-12-17 22:21:02,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:02,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881d575bc060ce15e5df HTTP/1.1" 502 150 2025-12-17 22:21:02,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881d575bc060ce15e5df'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:02,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881d575bc060ce15e5df 2025-12-17 22:21:02,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881d575bc060ce15e5df HTTP/1.1" 502 150 2025-12-17 22:21:02,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881d575bc060ce15e5df'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:02,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408800575bc060ce15e57d 2025-12-17 22:21:02,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408800575bc060ce15e57d HTTP/1.1" 502 150 2025-12-17 22:21:02,451 - lava_callback - ERROR - Error processing callback for node 69408800575bc060ce15e57d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408800575bc060ce15e57d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408800575bc060ce15e57d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408800575bc060ce15e57d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:02,454 - lava_callback - INFO - Starting async job submit for node 69408822575bc060ce15e5e8 2025-12-17 22:21:02,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:02,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408822575bc060ce15e5e8 HTTP/1.1" 502 150 2025-12-17 22:21:02,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408822575bc060ce15e5e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:02,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408822575bc060ce15e5e8 2025-12-17 22:21:02,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408822575bc060ce15e5e8 HTTP/1.1" 502 150 2025-12-17 22:21:02,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408822575bc060ce15e5e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:02,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881a575bc060ce15e5d5 2025-12-17 22:21:02,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881a575bc060ce15e5d5 HTTP/1.1" 502 150 2025-12-17 22:21:02,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881a575bc060ce15e5d5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:02,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408808575bc060ce15e59d 2025-12-17 22:21:02,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408808575bc060ce15e59d HTTP/1.1" 502 150 2025-12-17 22:21:02,835 - lava_callback - ERROR - Error processing callback for node 69408808575bc060ce15e59d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408808575bc060ce15e59d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408808575bc060ce15e59d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408808575bc060ce15e59d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:02,842 - lava_callback - INFO - Starting async job submit for node 69408826575bc060ce15e636 2025-12-17 22:21:02,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:02,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408826575bc060ce15e636 HTTP/1.1" 502 150 2025-12-17 22:21:02,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408826575bc060ce15e636'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:02,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408826575bc060ce15e636 2025-12-17 22:21:02,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408826575bc060ce15e636 HTTP/1.1" 502 150 2025-12-17 22:21:02,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408826575bc060ce15e636'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:03,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408809575bc060ce15e5a5 2025-12-17 22:21:03,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408809575bc060ce15e5a5 HTTP/1.1" 502 150 2025-12-17 22:21:03,450 - lava_callback - ERROR - Error processing callback for node 69408809575bc060ce15e5a5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408809575bc060ce15e5a5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408809575bc060ce15e5a5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408809575bc060ce15e5a5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:03,452 - lava_callback - INFO - Starting async job submit for node 69408828575bc060ce15e639 2025-12-17 22:21:03,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:03,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408828575bc060ce15e639 HTTP/1.1" 502 150 2025-12-17 22:21:03,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408828575bc060ce15e639'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:03,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408828575bc060ce15e639 2025-12-17 22:21:03,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408828575bc060ce15e639 HTTP/1.1" 502 150 2025-12-17 22:21:03,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408828575bc060ce15e639'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:03,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880a575bc060ce15e5a8 2025-12-17 22:21:03,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880a575bc060ce15e5a8 HTTP/1.1" 502 150 2025-12-17 22:21:03,919 - lava_callback - ERROR - Error processing callback for node 6940880a575bc060ce15e5a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940880a575bc060ce15e5a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940880a575bc060ce15e5a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940880a575bc060ce15e5a8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:03,921 - lava_callback - INFO - Starting async job submit for node 6940882a575bc060ce15e63c 2025-12-17 22:21:03,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:03,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882a575bc060ce15e63c HTTP/1.1" 502 150 2025-12-17 22:21:03,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940882a575bc060ce15e63c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:03,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940882a575bc060ce15e63c 2025-12-17 22:21:03,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882a575bc060ce15e63c HTTP/1.1" 502 150 2025-12-17 22:21:03,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940882a575bc060ce15e63c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:04,398 - lava_callback - INFO - Received callback for node 694089bb575bc060ce15ebbb 2025-12-17 22:21:04,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881d575bc060ce15e5df INFO: 172.21.0.1:53566 - "POST /node/694089bb575bc060ce15ebbb HTTP/1.0" 202 Accepted 2025-12-17 22:21:04,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881d575bc060ce15e5df HTTP/1.1" 502 150 2025-12-17 22:21:04,405 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881d575bc060ce15e5df'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:04,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408822575bc060ce15e5e8 2025-12-17 22:21:04,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408822575bc060ce15e5e8 HTTP/1.1" 502 150 2025-12-17 22:21:04,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408822575bc060ce15e5e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:04,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408826575bc060ce15e636 2025-12-17 22:21:04,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408826575bc060ce15e636 HTTP/1.1" 502 150 2025-12-17 22:21:04,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408826575bc060ce15e636'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:05,034 - lava_callback - INFO - Received callback for node 694089be575bc060ce15ebc5 INFO: 172.21.0.1:53572 - "POST /node/694089be575bc060ce15ebc5 HTTP/1.0" 202 Accepted 2025-12-17 22:21:05,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408828575bc060ce15e639 2025-12-17 22:21:05,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408828575bc060ce15e639 HTTP/1.1" 502 150 2025-12-17 22:21:05,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408828575bc060ce15e639'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:05,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408813575bc060ce15e5c1 HTTP/1.1" 502 150 2025-12-17 22:21:05,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408813575bc060ce15e5c1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:05,695 - lava_callback - INFO - Received callback for node 694089c0575bc060ce15ebc8 INFO: 172.21.0.1:53582 - "POST /node/694089c0575bc060ce15ebc8 HTTP/1.0" 202 Accepted 2025-12-17 22:21:05,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881b575bc060ce15e5dc 2025-12-17 22:21:05,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881b575bc060ce15e5dc HTTP/1.1" 502 150 2025-12-17 22:21:05,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881b575bc060ce15e5dc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:05,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940882a575bc060ce15e63c 2025-12-17 22:21:05,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882a575bc060ce15e63c HTTP/1.1" 502 150 2025-12-17 22:21:05,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940882a575bc060ce15e63c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:06,133 - lava_callback - INFO - Received callback for node 6940de35460bc8e2651133a4 INFO: 172.21.0.1:53586 - "POST /node/6940de35460bc8e2651133a4 HTTP/1.0" 202 Accepted 2025-12-17 22:21:06,328 - lava_callback - INFO - Received callback for node 694089c2575bc060ce15ebcb INFO: 172.21.0.1:53596 - "POST /node/694089c2575bc060ce15ebcb HTTP/1.0" 202 Accepted 2025-12-17 22:21:06,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880c575bc060ce15e5ab 2025-12-17 22:21:06,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880c575bc060ce15e5ab HTTP/1.1" 502 150 2025-12-17 22:21:06,465 - lava_callback - ERROR - Error processing callback for node 6940880c575bc060ce15e5ab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940880c575bc060ce15e5ab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940880c575bc060ce15e5ab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940880c575bc060ce15e5ab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:06,467 - lava_callback - INFO - Starting async job submit for node 6940882c575bc060ce15e647 2025-12-17 22:21:06,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:06,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882c575bc060ce15e647 HTTP/1.1" 502 150 2025-12-17 22:21:06,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940882c575bc060ce15e647'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:06,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940882c575bc060ce15e647 2025-12-17 22:21:06,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882c575bc060ce15e647 HTTP/1.1" 502 150 2025-12-17 22:21:06,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940882c575bc060ce15e647'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53598 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:21:06,969 - lava_callback - INFO - Received callback for node 694089c6575bc060ce15ebd4 INFO: 172.21.0.1:53610 - "POST /node/694089c6575bc060ce15ebd4 HTTP/1.0" 202 Accepted 2025-12-17 22:21:07,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880d575bc060ce15e5b2 2025-12-17 22:21:07,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880d575bc060ce15e5b2 HTTP/1.1" 502 150 2025-12-17 22:21:07,515 - lava_callback - ERROR - Error processing callback for node 6940880d575bc060ce15e5b2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940880d575bc060ce15e5b2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940880d575bc060ce15e5b2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940880d575bc060ce15e5b2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:07,517 - lava_callback - INFO - Starting async job submit for node 6940882e575bc060ce15e64c 2025-12-17 22:21:07,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:07,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882e575bc060ce15e64c HTTP/1.1" 502 150 2025-12-17 22:21:07,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940882e575bc060ce15e64c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:07,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940882e575bc060ce15e64c 2025-12-17 22:21:07,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882e575bc060ce15e64c HTTP/1.1" 502 150 2025-12-17 22:21:07,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940882e575bc060ce15e64c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:07,621 - lava_callback - INFO - Received callback for node 694089c8575bc060ce15ebd7 INFO: 172.21.0.1:53620 - "POST /node/694089c8575bc060ce15ebd7 HTTP/1.0" 202 Accepted 2025-12-17 22:21:08,252 - lava_callback - INFO - Received callback for node 694089c9575bc060ce15ebda INFO: 172.21.0.1:53630 - "POST /node/694089c9575bc060ce15ebda HTTP/1.0" 202 Accepted 2025-12-17 22:21:08,406 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881d575bc060ce15e5df 2025-12-17 22:21:08,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408822575bc060ce15e5e8 2025-12-17 22:21:08,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408822575bc060ce15e5e8 HTTP/1.1" 502 150 2025-12-17 22:21:08,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408822575bc060ce15e5e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:08,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940882c575bc060ce15e647 2025-12-17 22:21:08,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882c575bc060ce15e647 HTTP/1.1" 502 150 2025-12-17 22:21:08,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940882c575bc060ce15e647'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:08,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408802575bc060ce15e585 2025-12-17 22:21:08,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408802575bc060ce15e585 HTTP/1.1" 502 150 2025-12-17 22:21:08,786 - lava_callback - ERROR - Error processing callback for node 69408802575bc060ce15e585: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408802575bc060ce15e585 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408802575bc060ce15e585 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408802575bc060ce15e585 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:08,788 - lava_callback - INFO - Starting async job submit for node 69408832575bc060ce15e652 2025-12-17 22:21:08,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:08,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408832575bc060ce15e652 HTTP/1.1" 502 150 2025-12-17 22:21:08,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408832575bc060ce15e652'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:08,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408832575bc060ce15e652 2025-12-17 22:21:08,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408832575bc060ce15e652 HTTP/1.1" 502 150 2025-12-17 22:21:08,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408832575bc060ce15e652'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:08,883 - lava_callback - INFO - Received callback for node 694089ca575bc060ce15ebdd INFO: 172.21.0.1:53642 - "POST /node/694089ca575bc060ce15ebdd HTTP/1.0" 202 Accepted 2025-12-17 22:21:08,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408826575bc060ce15e636 2025-12-17 22:21:08,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408826575bc060ce15e636 HTTP/1.1" 502 150 2025-12-17 22:21:08,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408826575bc060ce15e636'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:09,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408828575bc060ce15e639 2025-12-17 22:21:09,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408828575bc060ce15e639 HTTP/1.1" 502 150 2025-12-17 22:21:09,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408828575bc060ce15e639'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:09,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940882e575bc060ce15e64c 2025-12-17 22:21:09,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882e575bc060ce15e64c HTTP/1.1" 502 150 2025-12-17 22:21:09,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940882e575bc060ce15e64c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:09,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408813575bc060ce15e5c1 2025-12-17 22:21:09,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408813575bc060ce15e5c1 HTTP/1.1" 502 150 2025-12-17 22:21:09,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408813575bc060ce15e5c1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:09,587 - lava_callback - INFO - Received callback for node 694089c4575bc060ce15ebce INFO: 172.21.0.1:53658 - "POST /node/694089c4575bc060ce15ebce HTTP/1.0" 202 Accepted 2025-12-17 22:21:09,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940882a575bc060ce15e63c 2025-12-17 22:21:09,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882a575bc060ce15e63c HTTP/1.1" 502 150 2025-12-17 22:21:09,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940882a575bc060ce15e63c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:09,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408815575bc060ce15e5c4 2025-12-17 22:21:09,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408815575bc060ce15e5c4 HTTP/1.1" 502 150 2025-12-17 22:21:09,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408815575bc060ce15e5c4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:10,262 - lava_callback - INFO - Received callback for node 694089c5575bc060ce15ebd1 INFO: 172.21.0.1:60982 - "POST /node/694089c5575bc060ce15ebd1 HTTP/1.0" 202 Accepted 2025-12-17 22:21:10,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881a575bc060ce15e5d5 2025-12-17 22:21:10,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881a575bc060ce15e5d5 HTTP/1.1" 502 150 2025-12-17 22:21:10,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881a575bc060ce15e5d5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:10,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940880e575bc060ce15e5b5 2025-12-17 22:21:10,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940880e575bc060ce15e5b5 HTTP/1.1" 502 150 2025-12-17 22:21:10,761 - lava_callback - ERROR - Error processing callback for node 6940880e575bc060ce15e5b5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940880e575bc060ce15e5b5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940880e575bc060ce15e5b5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940880e575bc060ce15e5b5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:10,763 - lava_callback - INFO - Starting async job submit for node 69408834575bc060ce15e655 2025-12-17 22:21:10,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:10,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408834575bc060ce15e655 HTTP/1.1" 502 150 2025-12-17 22:21:10,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408834575bc060ce15e655'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:10,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408834575bc060ce15e655 2025-12-17 22:21:10,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408834575bc060ce15e655 HTTP/1.1" 502 150 2025-12-17 22:21:10,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408834575bc060ce15e655'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:10,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408832575bc060ce15e652 2025-12-17 22:21:10,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408832575bc060ce15e652 HTTP/1.1" 502 150 2025-12-17 22:21:10,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408832575bc060ce15e652'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:12,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940882c575bc060ce15e647 2025-12-17 22:21:12,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882c575bc060ce15e647 HTTP/1.1" 502 150 2025-12-17 22:21:12,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940882c575bc060ce15e647'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:12,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408834575bc060ce15e655 2025-12-17 22:21:12,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408834575bc060ce15e655 HTTP/1.1" 502 150 2025-12-17 22:21:12,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408834575bc060ce15e655'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:13,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940882e575bc060ce15e64c 2025-12-17 22:21:13,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882e575bc060ce15e64c HTTP/1.1" 502 150 2025-12-17 22:21:13,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940882e575bc060ce15e64c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:13,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881b575bc060ce15e5dc 2025-12-17 22:21:13,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881b575bc060ce15e5dc HTTP/1.1" 502 150 2025-12-17 22:21:13,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881b575bc060ce15e5dc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:14,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408832575bc060ce15e652 2025-12-17 22:21:14,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408832575bc060ce15e652 HTTP/1.1" 502 150 2025-12-17 22:21:14,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408832575bc060ce15e652'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60990 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:21:15,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408810575bc060ce15e5b8 2025-12-17 22:21:15,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408810575bc060ce15e5b8 HTTP/1.1" 502 150 2025-12-17 22:21:15,627 - lava_callback - ERROR - Error processing callback for node 69408810575bc060ce15e5b8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408810575bc060ce15e5b8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408810575bc060ce15e5b8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408810575bc060ce15e5b8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:15,629 - lava_callback - INFO - Starting async job submit for node 69408836575bc060ce15e658 2025-12-17 22:21:15,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:15,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408836575bc060ce15e658 HTTP/1.1" 502 150 2025-12-17 22:21:15,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408836575bc060ce15e658'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:15,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408836575bc060ce15e658 2025-12-17 22:21:15,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408836575bc060ce15e658 HTTP/1.1" 502 150 2025-12-17 22:21:15,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408836575bc060ce15e658'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:15,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408811575bc060ce15e5bb 2025-12-17 22:21:15,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408811575bc060ce15e5bb HTTP/1.1" 502 150 2025-12-17 22:21:15,840 - lava_callback - ERROR - Error processing callback for node 69408811575bc060ce15e5bb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408811575bc060ce15e5bb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408811575bc060ce15e5bb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408811575bc060ce15e5bb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:15,842 - lava_callback - INFO - Starting async job submit for node 69408838575bc060ce15e65b 2025-12-17 22:21:15,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:15,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408838575bc060ce15e65b HTTP/1.1" 502 150 2025-12-17 22:21:15,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408838575bc060ce15e65b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:15,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408838575bc060ce15e65b 2025-12-17 22:21:15,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408838575bc060ce15e65b HTTP/1.1" 502 150 2025-12-17 22:21:15,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408838575bc060ce15e65b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:15,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408812575bc060ce15e5be 2025-12-17 22:21:15,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408812575bc060ce15e5be HTTP/1.1" 502 150 2025-12-17 22:21:15,965 - lava_callback - ERROR - Error processing callback for node 69408812575bc060ce15e5be: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408812575bc060ce15e5be (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408812575bc060ce15e5be (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408812575bc060ce15e5be (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:15,967 - lava_callback - INFO - Starting async job submit for node 6940883a575bc060ce15e65e 2025-12-17 22:21:15,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:15,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883a575bc060ce15e65e HTTP/1.1" 502 150 2025-12-17 22:21:15,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883a575bc060ce15e65e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:15,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883a575bc060ce15e65e 2025-12-17 22:21:15,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883a575bc060ce15e65e HTTP/1.1" 502 150 2025-12-17 22:21:15,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883a575bc060ce15e65e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:16,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408822575bc060ce15e5e8 INFO: 172.21.0.1:60998 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:21:16,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408834575bc060ce15e655 2025-12-17 22:21:16,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408834575bc060ce15e655 HTTP/1.1" 502 150 2025-12-17 22:21:16,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408834575bc060ce15e655'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:16,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408826575bc060ce15e636 2025-12-17 22:21:16,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408826575bc060ce15e636 HTTP/1.1" 502 150 2025-12-17 22:21:16,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408826575bc060ce15e636'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:17,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408828575bc060ce15e639 2025-12-17 22:21:17,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408828575bc060ce15e639 HTTP/1.1" 502 150 2025-12-17 22:21:17,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408828575bc060ce15e639'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:17,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408813575bc060ce15e5c1 2025-12-17 22:21:17,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408813575bc060ce15e5c1 HTTP/1.1" 502 150 2025-12-17 22:21:17,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408813575bc060ce15e5c1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:17,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408836575bc060ce15e658 2025-12-17 22:21:17,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408836575bc060ce15e658 HTTP/1.1" 502 150 2025-12-17 22:21:17,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408836575bc060ce15e658'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:17,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408838575bc060ce15e65b 2025-12-17 22:21:17,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408838575bc060ce15e65b HTTP/1.1" 502 150 2025-12-17 22:21:17,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408838575bc060ce15e65b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:17,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940882a575bc060ce15e63c 2025-12-17 22:21:17,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882a575bc060ce15e63c HTTP/1.1" 502 150 2025-12-17 22:21:17,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940882a575bc060ce15e63c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:17,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883a575bc060ce15e65e 2025-12-17 22:21:17,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883a575bc060ce15e65e HTTP/1.1" 502 150 2025-12-17 22:21:17,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883a575bc060ce15e65e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:20,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940882c575bc060ce15e647 2025-12-17 22:21:20,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882c575bc060ce15e647 HTTP/1.1" 502 150 2025-12-17 22:21:20,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940882c575bc060ce15e647'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:21,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940882e575bc060ce15e64c 2025-12-17 22:21:21,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882e575bc060ce15e64c HTTP/1.1" 502 150 2025-12-17 22:21:21,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940882e575bc060ce15e64c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:21,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408836575bc060ce15e658 2025-12-17 22:21:21,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408836575bc060ce15e658 HTTP/1.1" 502 150 2025-12-17 22:21:21,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408836575bc060ce15e658'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:21,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408838575bc060ce15e65b 2025-12-17 22:21:21,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408838575bc060ce15e65b HTTP/1.1" 502 150 2025-12-17 22:21:21,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408838575bc060ce15e65b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:21,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883a575bc060ce15e65e 2025-12-17 22:21:21,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883a575bc060ce15e65e HTTP/1.1" 502 150 2025-12-17 22:21:21,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883a575bc060ce15e65e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:22,773 - lava_callback - INFO - Received callback for node 6940845a575bc060ce15d736 INFO: 172.21.0.1:39342 - "POST /node/6940845a575bc060ce15d736 HTTP/1.0" 202 Accepted 2025-12-17 22:21:22,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408832575bc060ce15e652 2025-12-17 22:21:22,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408832575bc060ce15e652 HTTP/1.1" 502 150 2025-12-17 22:21:22,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408832575bc060ce15e652'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:23,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881d575bc060ce15e5df HTTP/1.1" 502 150 2025-12-17 22:21:23,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881d575bc060ce15e5df'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:24,408 - lava_callback - INFO - Received callback for node 694089d0575bc060ce15ebe4 INFO: 172.21.0.1:39354 - "POST /node/694089d0575bc060ce15ebe4 HTTP/1.0" 202 Accepted 2025-12-17 22:21:24,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408834575bc060ce15e655 2025-12-17 22:21:24,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408834575bc060ce15e655 HTTP/1.1" 502 150 2025-12-17 22:21:24,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408834575bc060ce15e655'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:25,038 - lava_callback - INFO - Received callback for node 694089d1575bc060ce15ebed INFO: 172.21.0.1:39360 - "POST /node/694089d1575bc060ce15ebed HTTP/1.0" 202 Accepted 2025-12-17 22:21:25,695 - lava_callback - INFO - Received callback for node 694089d3575bc060ce15ebf0 INFO: 172.21.0.1:39368 - "POST /node/694089d3575bc060ce15ebf0 HTTP/1.0" 202 Accepted 2025-12-17 22:21:25,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408815575bc060ce15e5c4 2025-12-17 22:21:25,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408815575bc060ce15e5c4 HTTP/1.1" 502 150 2025-12-17 22:21:25,965 - lava_callback - ERROR - Error processing callback for node 69408815575bc060ce15e5c4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408815575bc060ce15e5c4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408815575bc060ce15e5c4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408815575bc060ce15e5c4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:25,966 - lava_callback - INFO - Starting async job submit for node 6940883d575bc060ce15e661 2025-12-17 22:21:25,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:25,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883d575bc060ce15e661 HTTP/1.1" 502 150 2025-12-17 22:21:25,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883d575bc060ce15e661'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:25,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883d575bc060ce15e661 2025-12-17 22:21:25,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883d575bc060ce15e661 HTTP/1.1" 502 150 2025-12-17 22:21:25,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883d575bc060ce15e661'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:26,349 - lava_callback - INFO - Received callback for node 694089d6575bc060ce15ebf6 INFO: 172.21.0.1:39376 - "POST /node/694089d6575bc060ce15ebf6 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:39382 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:21:26,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881a575bc060ce15e5d5 2025-12-17 22:21:26,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881a575bc060ce15e5d5 HTTP/1.1" 502 150 2025-12-17 22:21:26,719 - lava_callback - ERROR - Error processing callback for node 6940881a575bc060ce15e5d5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940881a575bc060ce15e5d5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940881a575bc060ce15e5d5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940881a575bc060ce15e5d5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:26,721 - lava_callback - INFO - Starting async job submit for node 6940883e575bc060ce15e664 2025-12-17 22:21:26,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:26,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883e575bc060ce15e664 HTTP/1.1" 502 150 2025-12-17 22:21:26,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883e575bc060ce15e664'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:26,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883e575bc060ce15e664 2025-12-17 22:21:26,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883e575bc060ce15e664 HTTP/1.1" 502 150 2025-12-17 22:21:26,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883e575bc060ce15e664'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:26,990 - lava_callback - INFO - Received callback for node 694089d9575bc060ce15ec31 INFO: 172.21.0.1:39390 - "POST /node/694089d9575bc060ce15ec31 HTTP/1.0" 202 Accepted 2025-12-17 22:21:27,623 - lava_callback - INFO - Received callback for node 694089db575bc060ce15ec40 INFO: 172.21.0.1:39404 - "POST /node/694089db575bc060ce15ec40 HTTP/1.0" 202 Accepted 2025-12-17 22:21:27,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883d575bc060ce15e661 2025-12-17 22:21:27,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883d575bc060ce15e661 HTTP/1.1" 502 150 2025-12-17 22:21:27,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883d575bc060ce15e661'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:28,290 - lava_callback - INFO - Received callback for node 694089dc575bc060ce15ec43 INFO: 172.21.0.1:39410 - "POST /node/694089dc575bc060ce15ec43 HTTP/1.0" 202 Accepted 2025-12-17 22:21:28,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883e575bc060ce15e664 2025-12-17 22:21:28,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883e575bc060ce15e664 HTTP/1.1" 502 150 2025-12-17 22:21:28,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883e575bc060ce15e664'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:28,950 - lava_callback - INFO - Received callback for node 694089dd575bc060ce15ec46 INFO: 172.21.0.1:39418 - "POST /node/694089dd575bc060ce15ec46 HTTP/1.0" 202 Accepted 2025-12-17 22:21:29,593 - lava_callback - INFO - Received callback for node 694089df575bc060ce15ec49 INFO: 172.21.0.1:39422 - "POST /node/694089df575bc060ce15ec49 HTTP/1.0" 202 Accepted 2025-12-17 22:21:29,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408836575bc060ce15e658 2025-12-17 22:21:29,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408836575bc060ce15e658 HTTP/1.1" 502 150 2025-12-17 22:21:29,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408836575bc060ce15e658'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:29,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881b575bc060ce15e5dc 2025-12-17 22:21:29,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881b575bc060ce15e5dc HTTP/1.1" 502 150 2025-12-17 22:21:29,807 - lava_callback - ERROR - Error processing callback for node 6940881b575bc060ce15e5dc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940881b575bc060ce15e5dc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940881b575bc060ce15e5dc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940881b575bc060ce15e5dc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:29,809 - lava_callback - INFO - Starting async job submit for node 6940883f575bc060ce15e66b 2025-12-17 22:21:29,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:29,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883f575bc060ce15e66b HTTP/1.1" 502 150 2025-12-17 22:21:29,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883f575bc060ce15e66b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:29,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883f575bc060ce15e66b 2025-12-17 22:21:29,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883f575bc060ce15e66b HTTP/1.1" 502 150 2025-12-17 22:21:29,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883f575bc060ce15e66b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:29,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408838575bc060ce15e65b 2025-12-17 22:21:29,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408838575bc060ce15e65b HTTP/1.1" 502 150 2025-12-17 22:21:29,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408838575bc060ce15e65b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:29,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883a575bc060ce15e65e 2025-12-17 22:21:29,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883a575bc060ce15e65e HTTP/1.1" 502 150 2025-12-17 22:21:29,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883a575bc060ce15e65e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:30,229 - lava_callback - INFO - Received callback for node 694089e3575bc060ce15ec4e INFO: 172.21.0.1:49084 - "POST /node/694089e3575bc060ce15ec4e HTTP/1.0" 202 Accepted 2025-12-17 22:21:31,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881d575bc060ce15e5df 2025-12-17 22:21:31,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881d575bc060ce15e5df HTTP/1.1" 502 150 2025-12-17 22:21:31,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940881d575bc060ce15e5df'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:31,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883f575bc060ce15e66b 2025-12-17 22:21:31,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883f575bc060ce15e66b HTTP/1.1" 502 150 2025-12-17 22:21:31,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883f575bc060ce15e66b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:31,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408822575bc060ce15e5e8 HTTP/1.1" 502 150 2025-12-17 22:21:31,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408822575bc060ce15e5e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:31,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883d575bc060ce15e661 2025-12-17 22:21:31,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883d575bc060ce15e661 HTTP/1.1" 502 150 2025-12-17 22:21:31,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883d575bc060ce15e661'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:32,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883e575bc060ce15e664 2025-12-17 22:21:32,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883e575bc060ce15e664 HTTP/1.1" 502 150 2025-12-17 22:21:32,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883e575bc060ce15e664'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:32,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408826575bc060ce15e636 2025-12-17 22:21:32,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408826575bc060ce15e636 HTTP/1.1" 502 150 2025-12-17 22:21:32,916 - lava_callback - ERROR - Error processing callback for node 69408826575bc060ce15e636: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408826575bc060ce15e636 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408826575bc060ce15e636 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408826575bc060ce15e636 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:32,918 - lava_callback - INFO - Starting async job submit for node 69408843575bc060ce15e679 2025-12-17 22:21:32,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:32,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408843575bc060ce15e679 HTTP/1.1" 502 150 2025-12-17 22:21:32,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408843575bc060ce15e679'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:32,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408843575bc060ce15e679 2025-12-17 22:21:32,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408843575bc060ce15e679 HTTP/1.1" 502 150 2025-12-17 22:21:32,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408843575bc060ce15e679'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:33,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408828575bc060ce15e639 2025-12-17 22:21:33,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408828575bc060ce15e639 HTTP/1.1" 502 150 2025-12-17 22:21:33,467 - lava_callback - ERROR - Error processing callback for node 69408828575bc060ce15e639: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408828575bc060ce15e639 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408828575bc060ce15e639 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408828575bc060ce15e639 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:33,469 - lava_callback - INFO - Starting async job submit for node 69408844575bc060ce15e67c 2025-12-17 22:21:33,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:33,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408844575bc060ce15e67c HTTP/1.1" 502 150 2025-12-17 22:21:33,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408844575bc060ce15e67c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:33,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408844575bc060ce15e67c 2025-12-17 22:21:33,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408844575bc060ce15e67c HTTP/1.1" 502 150 2025-12-17 22:21:33,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408844575bc060ce15e67c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:33,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408813575bc060ce15e5c1 2025-12-17 22:21:33,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408813575bc060ce15e5c1 HTTP/1.1" 502 150 2025-12-17 22:21:33,588 - lava_callback - ERROR - Error processing callback for node 69408813575bc060ce15e5c1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408813575bc060ce15e5c1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408813575bc060ce15e5c1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408813575bc060ce15e5c1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:33,591 - lava_callback - INFO - Starting async job submit for node 69408846575bc060ce15e67f 2025-12-17 22:21:33,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:33,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408846575bc060ce15e67f HTTP/1.1" 502 150 2025-12-17 22:21:33,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408846575bc060ce15e67f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:33,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408846575bc060ce15e67f 2025-12-17 22:21:33,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408846575bc060ce15e67f HTTP/1.1" 502 150 2025-12-17 22:21:33,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408846575bc060ce15e67f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:33,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940882a575bc060ce15e63c 2025-12-17 22:21:33,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882a575bc060ce15e63c HTTP/1.1" 502 150 2025-12-17 22:21:33,934 - lava_callback - ERROR - Error processing callback for node 6940882a575bc060ce15e63c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940882a575bc060ce15e63c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940882a575bc060ce15e63c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940882a575bc060ce15e63c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:33,937 - lava_callback - INFO - Starting async job submit for node 69408847575bc060ce15e682 2025-12-17 22:21:33,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:33,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408847575bc060ce15e682 HTTP/1.1" 502 150 2025-12-17 22:21:33,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408847575bc060ce15e682'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:33,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408847575bc060ce15e682 2025-12-17 22:21:33,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408847575bc060ce15e682 HTTP/1.1" 502 150 2025-12-17 22:21:33,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408847575bc060ce15e682'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:34,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408843575bc060ce15e679 2025-12-17 22:21:35,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408844575bc060ce15e67c 2025-12-17 22:21:35,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408844575bc060ce15e67c HTTP/1.1" 502 150 2025-12-17 22:21:35,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408844575bc060ce15e67c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:35,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408846575bc060ce15e67f 2025-12-17 22:21:35,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408846575bc060ce15e67f HTTP/1.1" 502 150 2025-12-17 22:21:35,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408846575bc060ce15e67f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:35,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883f575bc060ce15e66b 2025-12-17 22:21:35,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883f575bc060ce15e66b HTTP/1.1" 502 150 2025-12-17 22:21:35,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883f575bc060ce15e66b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:35,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408847575bc060ce15e682 2025-12-17 22:21:35,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408847575bc060ce15e682 HTTP/1.1" 502 150 2025-12-17 22:21:35,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408847575bc060ce15e682'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:36,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940882c575bc060ce15e647 2025-12-17 22:21:36,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882c575bc060ce15e647 HTTP/1.1" 502 150 2025-12-17 22:21:36,483 - lava_callback - ERROR - Error processing callback for node 6940882c575bc060ce15e647: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940882c575bc060ce15e647 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940882c575bc060ce15e647 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940882c575bc060ce15e647 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:36,485 - lava_callback - INFO - Starting async job submit for node 69408848575bc060ce15e687 2025-12-17 22:21:36,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:36,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408848575bc060ce15e687 HTTP/1.1" 502 150 2025-12-17 22:21:36,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408848575bc060ce15e687'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:36,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408848575bc060ce15e687 2025-12-17 22:21:36,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408848575bc060ce15e687 HTTP/1.1" 502 150 2025-12-17 22:21:36,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408848575bc060ce15e687'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49094 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:21:37,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940882e575bc060ce15e64c 2025-12-17 22:21:37,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940882e575bc060ce15e64c HTTP/1.1" 502 150 2025-12-17 22:21:37,530 - lava_callback - ERROR - Error processing callback for node 6940882e575bc060ce15e64c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940882e575bc060ce15e64c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940882e575bc060ce15e64c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940882e575bc060ce15e64c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:37,532 - lava_callback - INFO - Starting async job submit for node 6940884e575bc060ce15e693 2025-12-17 22:21:37,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:37,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884e575bc060ce15e693 HTTP/1.1" 502 150 2025-12-17 22:21:37,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884e575bc060ce15e693'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:37,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884e575bc060ce15e693 2025-12-17 22:21:37,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884e575bc060ce15e693 HTTP/1.1" 502 150 2025-12-17 22:21:37,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884e575bc060ce15e693'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:38,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408848575bc060ce15e687 2025-12-17 22:21:38,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408848575bc060ce15e687 HTTP/1.1" 502 150 2025-12-17 22:21:38,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408848575bc060ce15e687'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:38,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408832575bc060ce15e652 2025-12-17 22:21:38,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408832575bc060ce15e652 HTTP/1.1" 502 150 2025-12-17 22:21:38,826 - lava_callback - ERROR - Error processing callback for node 69408832575bc060ce15e652: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408832575bc060ce15e652 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408832575bc060ce15e652 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408832575bc060ce15e652 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:38,828 - lava_callback - INFO - Starting async job submit for node 69408851575bc060ce15e696 2025-12-17 22:21:38,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:38,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408851575bc060ce15e696 HTTP/1.1" 502 150 2025-12-17 22:21:38,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408851575bc060ce15e696'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:38,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408851575bc060ce15e696 2025-12-17 22:21:38,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408851575bc060ce15e696 HTTP/1.1" 502 150 2025-12-17 22:21:38,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408851575bc060ce15e696'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:39,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408844575bc060ce15e67c 2025-12-17 22:21:39,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408844575bc060ce15e67c HTTP/1.1" 502 150 2025-12-17 22:21:39,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408844575bc060ce15e67c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:39,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884e575bc060ce15e693 2025-12-17 22:21:39,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884e575bc060ce15e693 HTTP/1.1" 502 150 2025-12-17 22:21:39,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884e575bc060ce15e693'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:39,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408846575bc060ce15e67f 2025-12-17 22:21:39,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408846575bc060ce15e67f HTTP/1.1" 502 150 2025-12-17 22:21:39,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408846575bc060ce15e67f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:39,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408847575bc060ce15e682 2025-12-17 22:21:39,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408847575bc060ce15e682 HTTP/1.1" 502 150 2025-12-17 22:21:39,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408847575bc060ce15e682'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:39,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883d575bc060ce15e661 2025-12-17 22:21:39,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883d575bc060ce15e661 HTTP/1.1" 502 150 2025-12-17 22:21:39,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883d575bc060ce15e661'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:40,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883e575bc060ce15e664 2025-12-17 22:21:40,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883e575bc060ce15e664 HTTP/1.1" 502 150 2025-12-17 22:21:40,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883e575bc060ce15e664'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:40,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408834575bc060ce15e655 2025-12-17 22:21:40,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408834575bc060ce15e655 HTTP/1.1" 502 150 2025-12-17 22:21:40,779 - lava_callback - ERROR - Error processing callback for node 69408834575bc060ce15e655: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408834575bc060ce15e655 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408834575bc060ce15e655 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408834575bc060ce15e655 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:40,781 - lava_callback - INFO - Starting async job submit for node 69408853575bc060ce15e69b 2025-12-17 22:21:40,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:40,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408853575bc060ce15e69b HTTP/1.1" 502 150 2025-12-17 22:21:40,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408853575bc060ce15e69b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:40,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408853575bc060ce15e69b 2025-12-17 22:21:40,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408853575bc060ce15e69b HTTP/1.1" 502 150 2025-12-17 22:21:40,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408853575bc060ce15e69b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:40,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408851575bc060ce15e696 2025-12-17 22:21:40,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408851575bc060ce15e696 HTTP/1.1" 502 150 2025-12-17 22:21:40,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408851575bc060ce15e696'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:42,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408848575bc060ce15e687 2025-12-17 22:21:42,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408853575bc060ce15e69b 2025-12-17 22:21:42,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408853575bc060ce15e69b HTTP/1.1" 502 150 2025-12-17 22:21:42,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408853575bc060ce15e69b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:43,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884e575bc060ce15e693 2025-12-17 22:21:43,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884e575bc060ce15e693 HTTP/1.1" 502 150 2025-12-17 22:21:43,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884e575bc060ce15e693'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:43,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883f575bc060ce15e66b 2025-12-17 22:21:43,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883f575bc060ce15e66b HTTP/1.1" 502 150 2025-12-17 22:21:43,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940883f575bc060ce15e66b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:44,409 - lava_callback - INFO - Received callback for node 694089e4575bc060ce15ec55 INFO: 172.21.0.1:33428 - "POST /node/694089e4575bc060ce15ec55 HTTP/1.0" 202 Accepted 2025-12-17 22:21:44,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408851575bc060ce15e696 2025-12-17 22:21:44,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408851575bc060ce15e696 HTTP/1.1" 502 150 2025-12-17 22:21:44,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408851575bc060ce15e696'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:45,057 - lava_callback - INFO - Received callback for node 694089e5575bc060ce15ec58 INFO: 172.21.0.1:33440 - "POST /node/694089e5575bc060ce15ec58 HTTP/1.0" 202 Accepted 2025-12-17 22:21:45,064 - lava_callback - INFO - Received callback for node 69408587575bc060ce15dc6f INFO: 172.21.0.1:33446 - "POST /node/69408587575bc060ce15dc6f HTTP/1.0" 202 Accepted 2025-12-17 22:21:45,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408836575bc060ce15e658 2025-12-17 22:21:45,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408836575bc060ce15e658 HTTP/1.1" 502 150 2025-12-17 22:21:45,642 - lava_callback - ERROR - Error processing callback for node 69408836575bc060ce15e658: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408836575bc060ce15e658 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408836575bc060ce15e658 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408836575bc060ce15e658 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:45,644 - lava_callback - INFO - Starting async job submit for node 694091b9575bc060ce16091d 2025-12-17 22:21:45,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:45,707 - lava_callback - INFO - Received callback for node 694089e6575bc060ce15ec5b INFO: 172.21.0.1:33454 - "POST /node/694089e6575bc060ce15ec5b HTTP/1.0" 202 Accepted 2025-12-17 22:21:45,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b9575bc060ce16091d HTTP/1.1" 502 150 2025-12-17 22:21:45,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b9575bc060ce16091d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:45,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b9575bc060ce16091d 2025-12-17 22:21:45,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b9575bc060ce16091d HTTP/1.1" 502 150 2025-12-17 22:21:45,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b9575bc060ce16091d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:45,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408838575bc060ce15e65b 2025-12-17 22:21:45,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408838575bc060ce15e65b HTTP/1.1" 502 150 2025-12-17 22:21:45,878 - lava_callback - ERROR - Error processing callback for node 69408838575bc060ce15e65b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408838575bc060ce15e65b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408838575bc060ce15e65b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408838575bc060ce15e65b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:45,880 - lava_callback - INFO - Starting async job submit for node 69408855575bc060ce15e6a5 2025-12-17 22:21:45,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:45,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408855575bc060ce15e6a5 HTTP/1.1" 502 150 2025-12-17 22:21:45,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408855575bc060ce15e6a5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:45,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408855575bc060ce15e6a5 2025-12-17 22:21:45,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408855575bc060ce15e6a5 HTTP/1.1" 502 150 2025-12-17 22:21:45,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408855575bc060ce15e6a5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:45,916 - lava_callback - INFO - Received callback for node 694089f7575bc060ce15ec83 INFO: 172.21.0.1:33462 - "POST /node/694089f7575bc060ce15ec83 HTTP/1.0" 202 Accepted 2025-12-17 22:21:45,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883a575bc060ce15e65e 2025-12-17 22:21:45,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883a575bc060ce15e65e HTTP/1.1" 502 150 2025-12-17 22:21:45,978 - lava_callback - ERROR - Error processing callback for node 6940883a575bc060ce15e65e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940883a575bc060ce15e65e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940883a575bc060ce15e65e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940883a575bc060ce15e65e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:45,980 - lava_callback - INFO - Starting async job submit for node 69408857575bc060ce15e6ab 2025-12-17 22:21:45,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:46,341 - lava_callback - INFO - Received callback for node 694089e8575bc060ce15ec5e INFO: 172.21.0.1:33468 - "POST /node/694089e8575bc060ce15ec5e HTTP/1.0" 202 Accepted INFO: 172.21.0.1:33482 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:21:46,690 - lava_callback - INFO - Received callback for node 694089f8575bc060ce15ec88 INFO: 172.21.0.1:33492 - "POST /node/694089f8575bc060ce15ec88 HTTP/1.0" 202 Accepted 2025-12-17 22:21:46,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408853575bc060ce15e69b 2025-12-17 22:21:46,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408853575bc060ce15e69b HTTP/1.1" 502 150 2025-12-17 22:21:46,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408853575bc060ce15e69b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:46,987 - lava_callback - INFO - Received callback for node 694089e9575bc060ce15ec61 INFO: 172.21.0.1:33508 - "POST /node/694089e9575bc060ce15ec61 HTTP/1.0" 202 Accepted 2025-12-17 22:21:47,456 - lava_callback - INFO - Received callback for node 694089f9575bc060ce15ec8f INFO: 172.21.0.1:33522 - "POST /node/694089f9575bc060ce15ec8f HTTP/1.0" 202 Accepted 2025-12-17 22:21:47,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408844575bc060ce15e67c 2025-12-17 22:21:47,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408844575bc060ce15e67c HTTP/1.1" 502 150 2025-12-17 22:21:47,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408844575bc060ce15e67c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:47,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408846575bc060ce15e67f 2025-12-17 22:21:47,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408846575bc060ce15e67f HTTP/1.1" 502 150 2025-12-17 22:21:47,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408846575bc060ce15e67f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:47,647 - lava_callback - INFO - Received callback for node 694089ea575bc060ce15ec64 INFO: 172.21.0.1:33534 - "POST /node/694089ea575bc060ce15ec64 HTTP/1.0" 202 Accepted 2025-12-17 22:21:47,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b9575bc060ce16091d 2025-12-17 22:21:47,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b9575bc060ce16091d HTTP/1.1" 502 150 2025-12-17 22:21:47,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b9575bc060ce16091d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:47,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940881d575bc060ce15e5df 2025-12-17 22:21:47,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940881d575bc060ce15e5df HTTP/1.1" 502 150 2025-12-17 22:21:47,763 - lava_callback - ERROR - Error processing callback for node 6940881d575bc060ce15e5df: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940881d575bc060ce15e5df (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940881d575bc060ce15e5df (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940881d575bc060ce15e5df (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:47,765 - lava_callback - INFO - Starting async job submit for node 69408859575bc060ce15e6ae 2025-12-17 22:21:47,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:47,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408859575bc060ce15e6ae HTTP/1.1" 502 150 2025-12-17 22:21:47,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408859575bc060ce15e6ae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:47,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408859575bc060ce15e6ae 2025-12-17 22:21:47,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408859575bc060ce15e6ae HTTP/1.1" 502 150 2025-12-17 22:21:47,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408859575bc060ce15e6ae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:47,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408855575bc060ce15e6a5 2025-12-17 22:21:47,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408855575bc060ce15e6a5 HTTP/1.1" 502 150 2025-12-17 22:21:47,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408855575bc060ce15e6a5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:47,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408847575bc060ce15e682 2025-12-17 22:21:47,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408847575bc060ce15e682 HTTP/1.1" 502 150 2025-12-17 22:21:47,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408847575bc060ce15e682'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:47,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408822575bc060ce15e5e8 2025-12-17 22:21:47,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408822575bc060ce15e5e8 HTTP/1.1" 502 150 2025-12-17 22:21:47,954 - lava_callback - ERROR - Error processing callback for node 69408822575bc060ce15e5e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408822575bc060ce15e5e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408822575bc060ce15e5e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408822575bc060ce15e5e8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:47,957 - lava_callback - INFO - Starting async job submit for node 6940884a575bc060ce15e68a 2025-12-17 22:21:47,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:47,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884a575bc060ce15e68a HTTP/1.1" 502 150 2025-12-17 22:21:47,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884a575bc060ce15e68a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:47,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884a575bc060ce15e68a 2025-12-17 22:21:47,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884a575bc060ce15e68a HTTP/1.1" 502 150 2025-12-17 22:21:47,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884a575bc060ce15e68a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:48,290 - lava_callback - INFO - Received callback for node 694089eb575bc060ce15ec67 INFO: 172.21.0.1:33538 - "POST /node/694089eb575bc060ce15ec67 HTTP/1.0" 202 Accepted 2025-12-17 22:21:48,934 - lava_callback - INFO - Received callback for node 694089ed575bc060ce15ec6a INFO: 172.21.0.1:33540 - "POST /node/694089ed575bc060ce15ec6a HTTP/1.0" 202 Accepted 2025-12-17 22:21:49,576 - lava_callback - INFO - Received callback for node 694089ee575bc060ce15ec6d INFO: 172.21.0.1:33554 - "POST /node/694089ee575bc060ce15ec6d HTTP/1.0" 202 Accepted 2025-12-17 22:21:49,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408859575bc060ce15e6ae 2025-12-17 22:21:49,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408859575bc060ce15e6ae HTTP/1.1" 502 150 2025-12-17 22:21:49,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408859575bc060ce15e6ae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:49,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884a575bc060ce15e68a 2025-12-17 22:21:49,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884a575bc060ce15e68a HTTP/1.1" 502 150 2025-12-17 22:21:49,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884a575bc060ce15e68a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:50,226 - lava_callback - INFO - Received callback for node 694089ef575bc060ce15ec70 INFO: 172.21.0.1:39800 - "POST /node/694089ef575bc060ce15ec70 HTTP/1.0" 202 Accepted 2025-12-17 22:21:50,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408843575bc060ce15e679 HTTP/1.1" 502 150 2025-12-17 22:21:50,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408843575bc060ce15e679'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:50,862 - lava_callback - INFO - Received callback for node 694089f3575bc060ce15ec7a INFO: 172.21.0.1:39812 - "POST /node/694089f3575bc060ce15ec7a HTTP/1.0" 202 Accepted 2025-12-17 22:21:51,500 - lava_callback - INFO - Received callback for node 694089f4575bc060ce15ec7d INFO: 172.21.0.1:39820 - "POST /node/694089f4575bc060ce15ec7d HTTP/1.0" 202 Accepted 2025-12-17 22:21:51,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884e575bc060ce15e693 2025-12-17 22:21:51,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884e575bc060ce15e693 HTTP/1.1" 502 150 2025-12-17 22:21:51,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884e575bc060ce15e693'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:51,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b9575bc060ce16091d 2025-12-17 22:21:51,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b9575bc060ce16091d HTTP/1.1" 502 150 2025-12-17 22:21:51,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b9575bc060ce16091d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:51,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408855575bc060ce15e6a5 2025-12-17 22:21:51,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408855575bc060ce15e6a5 HTTP/1.1" 502 150 2025-12-17 22:21:51,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408855575bc060ce15e6a5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:52,141 - lava_callback - INFO - Received callback for node 694089f5575bc060ce15ec80 INFO: 172.21.0.1:39828 - "POST /node/694089f5575bc060ce15ec80 HTTP/1.0" 202 Accepted 2025-12-17 22:21:52,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408851575bc060ce15e696 2025-12-17 22:21:52,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408851575bc060ce15e696 HTTP/1.1" 502 150 2025-12-17 22:21:52,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408851575bc060ce15e696'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:52,899 - lava_callback - INFO - Received callback for node 694090f4575bc060ce16070b INFO: 172.21.0.1:39838 - "POST /node/694090f4575bc060ce16070b HTTP/1.0" 202 Accepted 2025-12-17 22:21:53,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408859575bc060ce15e6ae 2025-12-17 22:21:53,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408859575bc060ce15e6ae HTTP/1.1" 502 150 2025-12-17 22:21:53,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408859575bc060ce15e6ae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:53,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884a575bc060ce15e68a 2025-12-17 22:21:53,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884a575bc060ce15e68a HTTP/1.1" 502 150 2025-12-17 22:21:53,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884a575bc060ce15e68a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:54,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408843575bc060ce15e679 2025-12-17 22:21:54,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408843575bc060ce15e679 HTTP/1.1" 502 150 2025-12-17 22:21:54,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408843575bc060ce15e679'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:54,431 - lava_callback - INFO - Received callback for node 694089fd575bc060ce15ec9b INFO: 172.21.0.1:39846 - "POST /node/694089fd575bc060ce15ec9b HTTP/1.0" 202 Accepted 2025-12-17 22:21:54,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408853575bc060ce15e69b 2025-12-17 22:21:54,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408853575bc060ce15e69b HTTP/1.1" 502 150 2025-12-17 22:21:54,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408853575bc060ce15e69b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:55,111 - lava_callback - INFO - Received callback for node 694089ff575bc060ce15ec9e INFO: 172.21.0.1:39850 - "POST /node/694089ff575bc060ce15ec9e HTTP/1.0" 202 Accepted 2025-12-17 22:21:55,793 - lava_callback - INFO - Received callback for node 69408a00575bc060ce15eca1 INFO: 172.21.0.1:39862 - "POST /node/69408a00575bc060ce15eca1 HTTP/1.0" 202 Accepted 2025-12-17 22:21:55,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883d575bc060ce15e661 INFO: 172.21.0.1:39878 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:21:56,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883e575bc060ce15e664 2025-12-17 22:21:56,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883e575bc060ce15e664 HTTP/1.1" 502 150 2025-12-17 22:21:56,734 - lava_callback - ERROR - Error processing callback for node 6940883e575bc060ce15e664: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940883e575bc060ce15e664 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940883e575bc060ce15e664 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940883e575bc060ce15e664 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:56,737 - lava_callback - INFO - Starting async job submit for node 6940884b575bc060ce15e68d 2025-12-17 22:21:56,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:56,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884b575bc060ce15e68d HTTP/1.1" 502 150 2025-12-17 22:21:56,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884b575bc060ce15e68d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:56,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884b575bc060ce15e68d 2025-12-17 22:21:56,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884b575bc060ce15e68d HTTP/1.1" 502 150 2025-12-17 22:21:56,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884b575bc060ce15e68d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:57,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408848575bc060ce15e687 HTTP/1.1" 502 150 2025-12-17 22:21:57,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408848575bc060ce15e687'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:58,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884b575bc060ce15e68d 2025-12-17 22:21:58,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884b575bc060ce15e68d HTTP/1.1" 502 150 2025-12-17 22:21:58,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884b575bc060ce15e68d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:59,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b9575bc060ce16091d 2025-12-17 22:21:59,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b9575bc060ce16091d HTTP/1.1" 502 150 2025-12-17 22:21:59,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b9575bc060ce16091d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:59,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940883f575bc060ce15e66b 2025-12-17 22:21:59,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883f575bc060ce15e66b HTTP/1.1" 502 150 2025-12-17 22:21:59,835 - lava_callback - ERROR - Error processing callback for node 6940883f575bc060ce15e66b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940883f575bc060ce15e66b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940883f575bc060ce15e66b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940883f575bc060ce15e66b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:21:59,839 - lava_callback - INFO - Starting async job submit for node 6940884c575bc060ce15e690 2025-12-17 22:21:59,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:21:59,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884c575bc060ce15e690 HTTP/1.1" 502 150 2025-12-17 22:21:59,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884c575bc060ce15e690'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:59,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884c575bc060ce15e690 2025-12-17 22:21:59,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884c575bc060ce15e690 HTTP/1.1" 502 150 2025-12-17 22:21:59,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884c575bc060ce15e690'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:21:59,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408855575bc060ce15e6a5 2025-12-17 22:21:59,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408855575bc060ce15e6a5 HTTP/1.1" 502 150 2025-12-17 22:21:59,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408855575bc060ce15e6a5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:01,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408857575bc060ce15e6ab HTTP/1.1" 502 150 2025-12-17 22:22:01,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408857575bc060ce15e6ab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:01,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408857575bc060ce15e6ab 2025-12-17 22:22:01,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408857575bc060ce15e6ab HTTP/1.1" 502 150 2025-12-17 22:22:01,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408857575bc060ce15e6ab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:01,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408859575bc060ce15e6ae 2025-12-17 22:22:01,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408859575bc060ce15e6ae HTTP/1.1" 502 150 2025-12-17 22:22:01,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408859575bc060ce15e6ae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:01,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884c575bc060ce15e690 2025-12-17 22:22:01,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884c575bc060ce15e690 HTTP/1.1" 502 150 2025-12-17 22:22:01,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884c575bc060ce15e690'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:01,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884a575bc060ce15e68a 2025-12-17 22:22:01,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884a575bc060ce15e68a HTTP/1.1" 502 150 2025-12-17 22:22:01,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884a575bc060ce15e68a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38006 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:22:02,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408843575bc060ce15e679 2025-12-17 22:22:02,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408843575bc060ce15e679 HTTP/1.1" 502 150 2025-12-17 22:22:02,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408843575bc060ce15e679'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:02,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884b575bc060ce15e68d 2025-12-17 22:22:02,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884b575bc060ce15e68d HTTP/1.1" 502 150 2025-12-17 22:22:02,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884b575bc060ce15e68d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:03,393 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408857575bc060ce15e6ab 2025-12-17 22:22:03,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408857575bc060ce15e6ab HTTP/1.1" 502 150 2025-12-17 22:22:03,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408857575bc060ce15e6ab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:03,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408844575bc060ce15e67c 2025-12-17 22:22:03,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408844575bc060ce15e67c HTTP/1.1" 502 150 2025-12-17 22:22:03,482 - lava_callback - ERROR - Error processing callback for node 69408844575bc060ce15e67c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408844575bc060ce15e67c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408844575bc060ce15e67c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408844575bc060ce15e67c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:03,484 - lava_callback - INFO - Starting async job submit for node 6940885b575bc060ce15e6b1 2025-12-17 22:22:03,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:03,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885b575bc060ce15e6b1 HTTP/1.1" 502 150 2025-12-17 22:22:03,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940885b575bc060ce15e6b1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:03,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940885b575bc060ce15e6b1 2025-12-17 22:22:03,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885b575bc060ce15e6b1 HTTP/1.1" 502 150 2025-12-17 22:22:03,501 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940885b575bc060ce15e6b1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:03,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408846575bc060ce15e67f 2025-12-17 22:22:03,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408846575bc060ce15e67f HTTP/1.1" 502 150 2025-12-17 22:22:03,610 - lava_callback - ERROR - Error processing callback for node 69408846575bc060ce15e67f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408846575bc060ce15e67f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408846575bc060ce15e67f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408846575bc060ce15e67f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:03,612 - lava_callback - INFO - Starting async job submit for node 6940885d575bc060ce15e6b4 2025-12-17 22:22:03,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:03,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885d575bc060ce15e6b4 HTTP/1.1" 502 150 2025-12-17 22:22:03,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940885d575bc060ce15e6b4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:03,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940885d575bc060ce15e6b4 2025-12-17 22:22:03,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885d575bc060ce15e6b4 HTTP/1.1" 502 150 2025-12-17 22:22:03,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940885d575bc060ce15e6b4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:03,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408847575bc060ce15e682 2025-12-17 22:22:03,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408847575bc060ce15e682 HTTP/1.1" 502 150 2025-12-17 22:22:03,950 - lava_callback - ERROR - Error processing callback for node 69408847575bc060ce15e682: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408847575bc060ce15e682 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408847575bc060ce15e682 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408847575bc060ce15e682 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:03,952 - lava_callback - INFO - Starting async job submit for node 69408861575bc060ce15e6c1 2025-12-17 22:22:03,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:03,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408861575bc060ce15e6c1 HTTP/1.1" 502 150 2025-12-17 22:22:03,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408861575bc060ce15e6c1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:03,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408861575bc060ce15e6c1 2025-12-17 22:22:03,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408861575bc060ce15e6c1 HTTP/1.1" 502 150 2025-12-17 22:22:03,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408861575bc060ce15e6c1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:05,501 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940885b575bc060ce15e6b1 2025-12-17 22:22:05,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885b575bc060ce15e6b1 HTTP/1.1" 502 150 2025-12-17 22:22:05,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940885b575bc060ce15e6b1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:05,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940885d575bc060ce15e6b4 2025-12-17 22:22:05,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885d575bc060ce15e6b4 HTTP/1.1" 502 150 2025-12-17 22:22:05,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940885d575bc060ce15e6b4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:05,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408848575bc060ce15e687 2025-12-17 22:22:05,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408848575bc060ce15e687 HTTP/1.1" 502 150 2025-12-17 22:22:05,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408848575bc060ce15e687'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:05,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884c575bc060ce15e690 2025-12-17 22:22:05,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884c575bc060ce15e690 HTTP/1.1" 502 150 2025-12-17 22:22:05,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884c575bc060ce15e690'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:05,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408861575bc060ce15e6c1 2025-12-17 22:22:05,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408861575bc060ce15e6c1 HTTP/1.1" 502 150 2025-12-17 22:22:05,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408861575bc060ce15e6c1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38010 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:22:07,395 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408857575bc060ce15e6ab 2025-12-17 22:22:07,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408857575bc060ce15e6ab HTTP/1.1" 502 150 2025-12-17 22:22:07,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408857575bc060ce15e6ab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:07,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884e575bc060ce15e693 2025-12-17 22:22:07,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884e575bc060ce15e693 HTTP/1.1" 502 150 2025-12-17 22:22:07,554 - lava_callback - ERROR - Error processing callback for node 6940884e575bc060ce15e693: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940884e575bc060ce15e693 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940884e575bc060ce15e693 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940884e575bc060ce15e693 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:07,556 - lava_callback - INFO - Starting async job submit for node 69408864575bc060ce15e6f6 2025-12-17 22:22:07,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:07,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408864575bc060ce15e6f6 HTTP/1.1" 502 150 2025-12-17 22:22:07,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408864575bc060ce15e6f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:07,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408864575bc060ce15e6f6 2025-12-17 22:22:07,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408864575bc060ce15e6f6 HTTP/1.1" 502 150 2025-12-17 22:22:07,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408864575bc060ce15e6f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:08,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408851575bc060ce15e696 2025-12-17 22:22:08,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408851575bc060ce15e696 HTTP/1.1" 502 150 2025-12-17 22:22:08,841 - lava_callback - ERROR - Error processing callback for node 69408851575bc060ce15e696: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408851575bc060ce15e696 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408851575bc060ce15e696 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408851575bc060ce15e696 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:08,842 - lava_callback - INFO - Starting async job submit for node 6940885f575bc060ce15e6bb 2025-12-17 22:22:08,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:08,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885f575bc060ce15e6bb HTTP/1.1" 502 150 2025-12-17 22:22:08,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940885f575bc060ce15e6bb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:08,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940885f575bc060ce15e6bb 2025-12-17 22:22:08,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885f575bc060ce15e6bb HTTP/1.1" 502 150 2025-12-17 22:22:08,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940885f575bc060ce15e6bb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:09,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940885b575bc060ce15e6b1 2025-12-17 22:22:09,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885b575bc060ce15e6b1 HTTP/1.1" 502 150 2025-12-17 22:22:09,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940885b575bc060ce15e6b1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:09,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408864575bc060ce15e6f6 2025-12-17 22:22:09,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408864575bc060ce15e6f6 HTTP/1.1" 502 150 2025-12-17 22:22:09,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408864575bc060ce15e6f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:09,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940885d575bc060ce15e6b4 2025-12-17 22:22:09,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885d575bc060ce15e6b4 HTTP/1.1" 502 150 2025-12-17 22:22:09,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940885d575bc060ce15e6b4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:09,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408861575bc060ce15e6c1 2025-12-17 22:22:09,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408861575bc060ce15e6c1 HTTP/1.1" 502 150 2025-12-17 22:22:09,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408861575bc060ce15e6c1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:10,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884b575bc060ce15e68d 2025-12-17 22:22:10,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884b575bc060ce15e68d HTTP/1.1" 502 150 2025-12-17 22:22:10,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884b575bc060ce15e68d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:10,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408853575bc060ce15e69b 2025-12-17 22:22:10,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408853575bc060ce15e69b HTTP/1.1" 502 150 2025-12-17 22:22:10,797 - lava_callback - ERROR - Error processing callback for node 69408853575bc060ce15e69b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408853575bc060ce15e69b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408853575bc060ce15e69b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408853575bc060ce15e69b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:10,799 - lava_callback - INFO - Starting async job submit for node 6940886a575bc060ce15e703 2025-12-17 22:22:10,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:10,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940886a575bc060ce15e703 HTTP/1.1" 502 150 2025-12-17 22:22:10,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940886a575bc060ce15e703'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:10,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940886a575bc060ce15e703 2025-12-17 22:22:10,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940886a575bc060ce15e703 HTTP/1.1" 502 150 2025-12-17 22:22:10,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940886a575bc060ce15e703'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:10,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940885f575bc060ce15e6bb 2025-12-17 22:22:10,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885f575bc060ce15e6bb HTTP/1.1" 502 150 2025-12-17 22:22:10,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940885f575bc060ce15e6bb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:11,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940883d575bc060ce15e661 HTTP/1.1" 502 150 2025-12-17 22:22:11,376 - lava_callback - ERROR - Error processing callback for node 6940883d575bc060ce15e661: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940883d575bc060ce15e661 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940883d575bc060ce15e661 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940883d575bc060ce15e661 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:11,378 - lava_callback - INFO - Starting async job submit for node 6940886d575bc060ce15e706 2025-12-17 22:22:11,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:11,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940886d575bc060ce15e706 HTTP/1.1" 502 150 2025-12-17 22:22:11,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940886d575bc060ce15e706'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:11,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940886d575bc060ce15e706 2025-12-17 22:22:11,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940886d575bc060ce15e706 HTTP/1.1" 502 150 2025-12-17 22:22:11,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940886d575bc060ce15e706'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:12,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940886a575bc060ce15e703 2025-12-17 22:22:13,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940886d575bc060ce15e706 2025-12-17 22:22:13,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940886d575bc060ce15e706 HTTP/1.1" 502 150 2025-12-17 22:22:13,388 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940886d575bc060ce15e706'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:13,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408864575bc060ce15e6f6 2025-12-17 22:22:13,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408864575bc060ce15e6f6 HTTP/1.1" 502 150 2025-12-17 22:22:13,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408864575bc060ce15e6f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:13,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884c575bc060ce15e690 2025-12-17 22:22:13,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884c575bc060ce15e690 HTTP/1.1" 502 150 2025-12-17 22:22:13,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940884c575bc060ce15e690'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:14,405 - lava_callback - INFO - Received callback for node 69408a02575bc060ce15eca4 INFO: 172.21.0.1:38828 - "POST /node/69408a02575bc060ce15eca4 HTTP/1.0" 202 Accepted 2025-12-17 22:22:14,567 - lava_callback - INFO - Received callback for node 69408955575bc060ce15e9df INFO: 172.21.0.1:38832 - "POST /node/69408955575bc060ce15e9df HTTP/1.0" 202 Accepted 2025-12-17 22:22:14,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940885f575bc060ce15e6bb 2025-12-17 22:22:14,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885f575bc060ce15e6bb HTTP/1.1" 502 150 2025-12-17 22:22:14,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940885f575bc060ce15e6bb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:15,061 - lava_callback - INFO - Received callback for node 69408a03575bc060ce15eca7 INFO: 172.21.0.1:38840 - "POST /node/69408a03575bc060ce15eca7 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:38852 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:22:15,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408857575bc060ce15e6ab 2025-12-17 22:22:15,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408857575bc060ce15e6ab HTTP/1.1" 502 150 2025-12-17 22:22:15,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408857575bc060ce15e6ab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:15,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b9575bc060ce16091d 2025-12-17 22:22:15,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b9575bc060ce16091d HTTP/1.1" 502 150 2025-12-17 22:22:15,723 - lava_callback - ERROR - Error processing callback for node 694091b9575bc060ce16091d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091b9575bc060ce16091d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091b9575bc060ce16091d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091b9575bc060ce16091d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:15,725 - lava_callback - INFO - Starting async job submit for node 69408870575bc060ce15e70e 2025-12-17 22:22:15,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:15,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408870575bc060ce15e70e HTTP/1.1" 502 150 2025-12-17 22:22:15,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408870575bc060ce15e70e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:15,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408870575bc060ce15e70e 2025-12-17 22:22:15,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408870575bc060ce15e70e HTTP/1.1" 502 150 2025-12-17 22:22:15,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408870575bc060ce15e70e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:15,742 - lava_callback - INFO - Received callback for node 69408a04575bc060ce15ecaa INFO: 172.21.0.1:38858 - "POST /node/69408a04575bc060ce15ecaa HTTP/1.0" 202 Accepted 2025-12-17 22:22:15,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408855575bc060ce15e6a5 2025-12-17 22:22:15,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408855575bc060ce15e6a5 HTTP/1.1" 502 150 2025-12-17 22:22:15,893 - lava_callback - ERROR - Error processing callback for node 69408855575bc060ce15e6a5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408855575bc060ce15e6a5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408855575bc060ce15e6a5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408855575bc060ce15e6a5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:15,895 - lava_callback - INFO - Starting async job submit for node 69408872575bc060ce15e711 2025-12-17 22:22:15,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:15,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408872575bc060ce15e711 HTTP/1.1" 502 150 2025-12-17 22:22:15,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408872575bc060ce15e711'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:15,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408872575bc060ce15e711 2025-12-17 22:22:15,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408872575bc060ce15e711 HTTP/1.1" 502 150 2025-12-17 22:22:15,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408872575bc060ce15e711'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:16,421 - lava_callback - INFO - Received callback for node 69408a09575bc060ce15ecad INFO: 172.21.0.1:38870 - "POST /node/69408a09575bc060ce15ecad HTTP/1.0" 202 Accepted INFO: 172.21.0.1:38876 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:22:17,099 - lava_callback - INFO - Received callback for node 69408a0b575bc060ce15ecb0 INFO: 172.21.0.1:38886 - "POST /node/69408a0b575bc060ce15ecb0 HTTP/1.0" 202 Accepted 2025-12-17 22:22:17,388 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940886d575bc060ce15e706 2025-12-17 22:22:17,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940886d575bc060ce15e706 HTTP/1.1" 502 150 2025-12-17 22:22:17,390 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940886d575bc060ce15e706'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:17,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940885b575bc060ce15e6b1 2025-12-17 22:22:17,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885b575bc060ce15e6b1 HTTP/1.1" 502 150 2025-12-17 22:22:17,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940885b575bc060ce15e6b1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:17,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940885d575bc060ce15e6b4 2025-12-17 22:22:17,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885d575bc060ce15e6b4 HTTP/1.1" 502 150 2025-12-17 22:22:17,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940885d575bc060ce15e6b4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:17,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408870575bc060ce15e70e 2025-12-17 22:22:17,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408870575bc060ce15e70e HTTP/1.1" 502 150 2025-12-17 22:22:17,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408870575bc060ce15e70e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:17,759 - lava_callback - INFO - Received callback for node 69408a0c575bc060ce15ecb6 INFO: 172.21.0.1:38894 - "POST /node/69408a0c575bc060ce15ecb6 HTTP/1.0" 202 Accepted 2025-12-17 22:22:17,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408859575bc060ce15e6ae 2025-12-17 22:22:17,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408859575bc060ce15e6ae HTTP/1.1" 502 150 2025-12-17 22:22:17,799 - lava_callback - ERROR - Error processing callback for node 69408859575bc060ce15e6ae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408859575bc060ce15e6ae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408859575bc060ce15e6ae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408859575bc060ce15e6ae (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:17,801 - lava_callback - INFO - Starting async job submit for node 69408876575bc060ce15e71b 2025-12-17 22:22:17,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:17,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408876575bc060ce15e71b HTTP/1.1" 502 150 2025-12-17 22:22:17,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408876575bc060ce15e71b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:17,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408876575bc060ce15e71b 2025-12-17 22:22:17,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408876575bc060ce15e71b HTTP/1.1" 502 150 2025-12-17 22:22:17,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408876575bc060ce15e71b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:17,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408872575bc060ce15e711 2025-12-17 22:22:17,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408872575bc060ce15e711 HTTP/1.1" 502 150 2025-12-17 22:22:17,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408872575bc060ce15e711'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:17,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884a575bc060ce15e68a 2025-12-17 22:22:17,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884a575bc060ce15e68a HTTP/1.1" 502 150 2025-12-17 22:22:17,975 - lava_callback - ERROR - Error processing callback for node 6940884a575bc060ce15e68a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940884a575bc060ce15e68a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940884a575bc060ce15e68a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940884a575bc060ce15e68a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:17,977 - lava_callback - INFO - Starting async job submit for node 69408874575bc060ce15e714 2025-12-17 22:22:17,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:17,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408861575bc060ce15e6c1 2025-12-17 22:22:17,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408861575bc060ce15e6c1 HTTP/1.1" 502 150 2025-12-17 22:22:17,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408861575bc060ce15e6c1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:18,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408874575bc060ce15e714 HTTP/1.1" 502 150 2025-12-17 22:22:18,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408874575bc060ce15e714'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:18,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408874575bc060ce15e714 2025-12-17 22:22:18,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408874575bc060ce15e714 HTTP/1.1" 502 150 2025-12-17 22:22:18,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408874575bc060ce15e714'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:18,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408843575bc060ce15e679 2025-12-17 22:22:18,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408843575bc060ce15e679 HTTP/1.1" 502 150 2025-12-17 22:22:18,389 - lava_callback - ERROR - Error processing callback for node 69408843575bc060ce15e679: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408843575bc060ce15e679 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408843575bc060ce15e679 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408843575bc060ce15e679 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:18,391 - lava_callback - INFO - Starting async job submit for node 69408878575bc060ce15e71e 2025-12-17 22:22:18,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:18,401 - lava_callback - INFO - Received callback for node 69408a0d575bc060ce15ecbe 2025-12-17 22:22:18,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408878575bc060ce15e71e HTTP/1.1" 502 150 INFO: 172.21.0.1:38910 - "POST /node/69408a0d575bc060ce15ecbe HTTP/1.0" 202 Accepted 2025-12-17 22:22:18,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408878575bc060ce15e71e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:18,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408878575bc060ce15e71e 2025-12-17 22:22:18,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408878575bc060ce15e71e HTTP/1.1" 502 150 2025-12-17 22:22:18,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408878575bc060ce15e71e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:19,059 - lava_callback - INFO - Received callback for node 69408a0f575bc060ce15ecc3 INFO: 172.21.0.1:38926 - "POST /node/69408a0f575bc060ce15ecc3 HTTP/1.0" 202 Accepted 2025-12-17 22:22:19,712 - lava_callback - INFO - Received callback for node 69408a10575bc060ce15ecc6 INFO: 172.21.0.1:37318 - "POST /node/69408a10575bc060ce15ecc6 HTTP/1.0" 202 Accepted 2025-12-17 22:22:19,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408876575bc060ce15e71b 2025-12-17 22:22:19,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408876575bc060ce15e71b HTTP/1.1" 502 150 2025-12-17 22:22:19,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408876575bc060ce15e71b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:20,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408874575bc060ce15e714 2025-12-17 22:22:20,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408874575bc060ce15e714 HTTP/1.1" 502 150 2025-12-17 22:22:20,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408874575bc060ce15e714'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:20,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408878575bc060ce15e71e 2025-12-17 22:22:20,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408878575bc060ce15e71e HTTP/1.1" 502 150 2025-12-17 22:22:20,410 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408878575bc060ce15e71e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:21,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408864575bc060ce15e6f6 2025-12-17 22:22:21,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408864575bc060ce15e6f6 HTTP/1.1" 502 150 2025-12-17 22:22:21,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408864575bc060ce15e6f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:21,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408870575bc060ce15e70e 2025-12-17 22:22:21,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408870575bc060ce15e70e HTTP/1.1" 502 150 2025-12-17 22:22:21,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408870575bc060ce15e70e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:21,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408848575bc060ce15e687 2025-12-17 22:22:21,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408848575bc060ce15e687 HTTP/1.1" 502 150 2025-12-17 22:22:21,812 - lava_callback - ERROR - Error processing callback for node 69408848575bc060ce15e687: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408848575bc060ce15e687 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408848575bc060ce15e687 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408848575bc060ce15e687 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:21,814 - lava_callback - INFO - Starting async job submit for node 6940887a575bc060ce15e721 2025-12-17 22:22:21,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:21,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887a575bc060ce15e721 HTTP/1.1" 502 150 2025-12-17 22:22:21,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887a575bc060ce15e721'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:21,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887a575bc060ce15e721 2025-12-17 22:22:21,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887a575bc060ce15e721 HTTP/1.1" 502 150 2025-12-17 22:22:21,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887a575bc060ce15e721'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:21,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408872575bc060ce15e711 2025-12-17 22:22:21,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408872575bc060ce15e711 HTTP/1.1" 502 150 2025-12-17 22:22:21,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408872575bc060ce15e711'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:22,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940885f575bc060ce15e6bb 2025-12-17 22:22:23,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408876575bc060ce15e71b 2025-12-17 22:22:23,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408876575bc060ce15e71b HTTP/1.1" 502 150 2025-12-17 22:22:23,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408876575bc060ce15e71b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:23,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887a575bc060ce15e721 2025-12-17 22:22:23,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887a575bc060ce15e721 HTTP/1.1" 502 150 2025-12-17 22:22:23,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887a575bc060ce15e721'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:24,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408874575bc060ce15e714 2025-12-17 22:22:24,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408874575bc060ce15e714 HTTP/1.1" 502 150 2025-12-17 22:22:24,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408874575bc060ce15e714'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:24,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408878575bc060ce15e71e 2025-12-17 22:22:24,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408878575bc060ce15e71e HTTP/1.1" 502 150 2025-12-17 22:22:24,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408878575bc060ce15e71e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:24,441 - lava_callback - INFO - Received callback for node 69408a11575bc060ce15ecc9 INFO: 172.21.0.1:37326 - "POST /node/69408a11575bc060ce15ecc9 HTTP/1.0" 202 Accepted 2025-12-17 22:22:25,070 - lava_callback - INFO - Received callback for node 69408a14575bc060ce15eccc INFO: 172.21.0.1:37342 - "POST /node/69408a14575bc060ce15eccc HTTP/1.0" 202 Accepted 2025-12-17 22:22:25,390 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940886d575bc060ce15e706 2025-12-17 22:22:25,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940886d575bc060ce15e706 HTTP/1.1" 502 150 2025-12-17 22:22:25,391 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940886d575bc060ce15e706'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:25,701 - lava_callback - INFO - Received callback for node 69408a18575bc060ce15ed6e INFO: 172.21.0.1:37344 - "POST /node/69408a18575bc060ce15ed6e HTTP/1.0" 202 Accepted 2025-12-17 22:22:26,332 - lava_callback - INFO - Received callback for node 69408a1a575bc060ce15ed71 INFO: 172.21.0.1:37348 - "POST /node/69408a1a575bc060ce15ed71 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:37350 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:22:26,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884b575bc060ce15e68d 2025-12-17 22:22:26,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884b575bc060ce15e68d HTTP/1.1" 502 150 2025-12-17 22:22:26,750 - lava_callback - ERROR - Error processing callback for node 6940884b575bc060ce15e68d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940884b575bc060ce15e68d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940884b575bc060ce15e68d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940884b575bc060ce15e68d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:26,753 - lava_callback - INFO - Starting async job submit for node 6940887b575bc060ce15e724 2025-12-17 22:22:26,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:26,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887b575bc060ce15e724 HTTP/1.1" 502 150 2025-12-17 22:22:26,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887b575bc060ce15e724'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:26,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887b575bc060ce15e724 2025-12-17 22:22:26,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887b575bc060ce15e724 HTTP/1.1" 502 150 2025-12-17 22:22:26,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887b575bc060ce15e724'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:26,974 - lava_callback - INFO - Received callback for node 69408a1c575bc060ce15ed74 INFO: 172.21.0.1:37354 - "POST /node/69408a1c575bc060ce15ed74 HTTP/1.0" 202 Accepted 2025-12-17 22:22:27,622 - lava_callback - INFO - Received callback for node 69408a1e575bc060ce15ed77 INFO: 172.21.0.1:37364 - "POST /node/69408a1e575bc060ce15ed77 HTTP/1.0" 202 Accepted 2025-12-17 22:22:27,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887a575bc060ce15e721 2025-12-17 22:22:27,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887a575bc060ce15e721 HTTP/1.1" 502 150 2025-12-17 22:22:27,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887a575bc060ce15e721'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:28,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940886a575bc060ce15e703 HTTP/1.1" 502 150 2025-12-17 22:22:28,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940886a575bc060ce15e703'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:28,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887b575bc060ce15e724 2025-12-17 22:22:28,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887b575bc060ce15e724 HTTP/1.1" 502 150 2025-12-17 22:22:28,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887b575bc060ce15e724'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:29,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408870575bc060ce15e70e 2025-12-17 22:22:29,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408870575bc060ce15e70e HTTP/1.1" 502 150 2025-12-17 22:22:29,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408870575bc060ce15e70e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:29,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940884c575bc060ce15e690 2025-12-17 22:22:29,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940884c575bc060ce15e690 HTTP/1.1" 502 150 2025-12-17 22:22:29,857 - lava_callback - ERROR - Error processing callback for node 6940884c575bc060ce15e690: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940884c575bc060ce15e690 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940884c575bc060ce15e690 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940884c575bc060ce15e690 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:29,859 - lava_callback - INFO - Starting async job submit for node 6940887c575bc060ce15e72b 2025-12-17 22:22:29,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:29,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887c575bc060ce15e72b HTTP/1.1" 502 150 2025-12-17 22:22:29,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887c575bc060ce15e72b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:29,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887c575bc060ce15e72b 2025-12-17 22:22:29,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887c575bc060ce15e72b HTTP/1.1" 502 150 2025-12-17 22:22:29,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887c575bc060ce15e72b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:29,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408872575bc060ce15e711 2025-12-17 22:22:29,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408872575bc060ce15e711 HTTP/1.1" 502 150 2025-12-17 22:22:29,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408872575bc060ce15e711'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:31,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408857575bc060ce15e6ab 2025-12-17 22:22:31,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408857575bc060ce15e6ab HTTP/1.1" 502 150 2025-12-17 22:22:31,399 - lava_callback - ERROR - Error processing callback for node 69408857575bc060ce15e6ab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408857575bc060ce15e6ab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408857575bc060ce15e6ab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408857575bc060ce15e6ab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:31,401 - lava_callback - INFO - Starting async job submit for node 6940887e575bc060ce15e72e 2025-12-17 22:22:31,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:31,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887e575bc060ce15e72e HTTP/1.1" 502 150 2025-12-17 22:22:31,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887e575bc060ce15e72e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:31,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887e575bc060ce15e72e 2025-12-17 22:22:31,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887e575bc060ce15e72e HTTP/1.1" 502 150 2025-12-17 22:22:31,410 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887e575bc060ce15e72e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:31,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408876575bc060ce15e71b 2025-12-17 22:22:31,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408876575bc060ce15e71b HTTP/1.1" 502 150 2025-12-17 22:22:31,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408876575bc060ce15e71b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:31,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887c575bc060ce15e72b 2025-12-17 22:22:31,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887c575bc060ce15e72b HTTP/1.1" 502 150 2025-12-17 22:22:31,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887c575bc060ce15e72b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:32,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408874575bc060ce15e714 2025-12-17 22:22:32,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408874575bc060ce15e714 HTTP/1.1" 502 150 2025-12-17 22:22:32,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408874575bc060ce15e714'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:32,273 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940886a575bc060ce15e703 2025-12-17 22:22:32,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940886a575bc060ce15e703 HTTP/1.1" 502 150 2025-12-17 22:22:32,274 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940886a575bc060ce15e703'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:32,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408878575bc060ce15e71e 2025-12-17 22:22:32,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408878575bc060ce15e71e HTTP/1.1" 502 150 2025-12-17 22:22:32,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408878575bc060ce15e71e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:32,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887b575bc060ce15e724 2025-12-17 22:22:32,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887b575bc060ce15e724 HTTP/1.1" 502 150 2025-12-17 22:22:32,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887b575bc060ce15e724'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:33,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887e575bc060ce15e72e 2025-12-17 22:22:33,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940885b575bc060ce15e6b1 2025-12-17 22:22:33,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885b575bc060ce15e6b1 HTTP/1.1" 502 150 2025-12-17 22:22:33,507 - lava_callback - ERROR - Error processing callback for node 6940885b575bc060ce15e6b1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940885b575bc060ce15e6b1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940885b575bc060ce15e6b1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940885b575bc060ce15e6b1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:33,509 - lava_callback - INFO - Starting async job submit for node 69408880575bc060ce15e731 2025-12-17 22:22:33,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:33,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408880575bc060ce15e731 HTTP/1.1" 502 150 2025-12-17 22:22:33,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408880575bc060ce15e731'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:33,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408880575bc060ce15e731 2025-12-17 22:22:33,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408880575bc060ce15e731 HTTP/1.1" 502 150 2025-12-17 22:22:33,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408880575bc060ce15e731'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:33,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940885d575bc060ce15e6b4 2025-12-17 22:22:33,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885d575bc060ce15e6b4 HTTP/1.1" 502 150 2025-12-17 22:22:33,625 - lava_callback - ERROR - Error processing callback for node 6940885d575bc060ce15e6b4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940885d575bc060ce15e6b4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940885d575bc060ce15e6b4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940885d575bc060ce15e6b4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:33,627 - lava_callback - INFO - Starting async job submit for node 69408882575bc060ce15e739 2025-12-17 22:22:33,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:33,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408882575bc060ce15e739 HTTP/1.1" 502 150 2025-12-17 22:22:33,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408882575bc060ce15e739'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:33,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408882575bc060ce15e739 2025-12-17 22:22:33,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408882575bc060ce15e739 HTTP/1.1" 502 150 2025-12-17 22:22:33,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408882575bc060ce15e739'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:33,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408861575bc060ce15e6c1 2025-12-17 22:22:33,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408861575bc060ce15e6c1 HTTP/1.1" 502 150 2025-12-17 22:22:33,981 - lava_callback - ERROR - Error processing callback for node 69408861575bc060ce15e6c1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408861575bc060ce15e6c1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408861575bc060ce15e6c1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408861575bc060ce15e6c1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:33,983 - lava_callback - INFO - Starting async job submit for node 69408884575bc060ce15e73e 2025-12-17 22:22:33,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:33,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408884575bc060ce15e73e HTTP/1.1" 502 150 2025-12-17 22:22:33,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408884575bc060ce15e73e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:33,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408884575bc060ce15e73e 2025-12-17 22:22:33,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408884575bc060ce15e73e HTTP/1.1" 502 150 2025-12-17 22:22:33,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408884575bc060ce15e73e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:35,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408880575bc060ce15e731 2025-12-17 22:22:35,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408880575bc060ce15e731 HTTP/1.1" 502 150 2025-12-17 22:22:35,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408880575bc060ce15e731'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:35,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408882575bc060ce15e739 2025-12-17 22:22:35,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408882575bc060ce15e739 HTTP/1.1" 502 150 2025-12-17 22:22:35,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408882575bc060ce15e739'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:35,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887a575bc060ce15e721 2025-12-17 22:22:35,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887a575bc060ce15e721 HTTP/1.1" 502 150 2025-12-17 22:22:35,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887a575bc060ce15e721'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:35,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887c575bc060ce15e72b 2025-12-17 22:22:35,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887c575bc060ce15e72b HTTP/1.1" 502 150 2025-12-17 22:22:35,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887c575bc060ce15e72b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:35,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408884575bc060ce15e73e 2025-12-17 22:22:35,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408884575bc060ce15e73e HTTP/1.1" 502 150 2025-12-17 22:22:35,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408884575bc060ce15e73e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36494 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:22:37,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408864575bc060ce15e6f6 2025-12-17 22:22:37,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408864575bc060ce15e6f6 HTTP/1.1" 502 150 2025-12-17 22:22:37,591 - lava_callback - ERROR - Error processing callback for node 69408864575bc060ce15e6f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408864575bc060ce15e6f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408864575bc060ce15e6f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408864575bc060ce15e6f6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:37,593 - lava_callback - INFO - Starting async job submit for node 6940ce69460bc8e265111f3d 2025-12-17 22:22:37,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:37,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ce69460bc8e265111f3d HTTP/1.1" 502 150 2025-12-17 22:22:37,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ce69460bc8e265111f3d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:37,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ce69460bc8e265111f3d 2025-12-17 22:22:37,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ce69460bc8e265111f3d HTTP/1.1" 502 150 2025-12-17 22:22:37,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ce69460bc8e265111f3d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:38,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885f575bc060ce15e6bb HTTP/1.1" 502 150 2025-12-17 22:22:38,256 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940885f575bc060ce15e6bb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:39,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408880575bc060ce15e731 2025-12-17 22:22:39,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408880575bc060ce15e731 HTTP/1.1" 502 150 2025-12-17 22:22:39,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408880575bc060ce15e731'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:39,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ce69460bc8e265111f3d 2025-12-17 22:22:39,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ce69460bc8e265111f3d HTTP/1.1" 502 150 2025-12-17 22:22:39,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ce69460bc8e265111f3d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:39,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408882575bc060ce15e739 2025-12-17 22:22:39,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408882575bc060ce15e739 HTTP/1.1" 502 150 2025-12-17 22:22:39,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408882575bc060ce15e739'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:39,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408884575bc060ce15e73e 2025-12-17 22:22:39,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408884575bc060ce15e73e HTTP/1.1" 502 150 2025-12-17 22:22:39,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408884575bc060ce15e73e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:40,274 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940886a575bc060ce15e703 2025-12-17 22:22:40,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940886a575bc060ce15e703 HTTP/1.1" 502 150 2025-12-17 22:22:40,275 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940886a575bc060ce15e703'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:40,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887b575bc060ce15e724 2025-12-17 22:22:40,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887b575bc060ce15e724 HTTP/1.1" 502 150 2025-12-17 22:22:40,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887b575bc060ce15e724'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:41,391 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940886d575bc060ce15e706 2025-12-17 22:22:41,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940886d575bc060ce15e706 HTTP/1.1" 502 150 2025-12-17 22:22:41,392 - lava_callback - ERROR - Error processing callback for node 6940886d575bc060ce15e706: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940886d575bc060ce15e706 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940886d575bc060ce15e706 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940886d575bc060ce15e706 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:41,394 - lava_callback - INFO - Starting async job submit for node 69408887575bc060ce15e741 2025-12-17 22:22:41,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:41,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408887575bc060ce15e741 HTTP/1.1" 502 150 2025-12-17 22:22:41,407 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408887575bc060ce15e741'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:41,407 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408887575bc060ce15e741 2025-12-17 22:22:41,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408887575bc060ce15e741 HTTP/1.1" 502 150 2025-12-17 22:22:41,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408887575bc060ce15e741'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:43,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408887575bc060ce15e741 2025-12-17 22:22:43,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408887575bc060ce15e741 HTTP/1.1" 502 150 2025-12-17 22:22:43,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408887575bc060ce15e741'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:43,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ce69460bc8e265111f3d 2025-12-17 22:22:43,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ce69460bc8e265111f3d HTTP/1.1" 502 150 2025-12-17 22:22:43,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ce69460bc8e265111f3d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:43,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887c575bc060ce15e72b 2025-12-17 22:22:43,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887c575bc060ce15e72b HTTP/1.1" 502 150 2025-12-17 22:22:43,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887c575bc060ce15e72b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:44,457 - lava_callback - INFO - Received callback for node 69408a20575bc060ce15ed7c INFO: 172.21.0.1:50904 - "POST /node/69408a20575bc060ce15ed7c HTTP/1.0" 202 Accepted 2025-12-17 22:22:45,089 - lava_callback - INFO - Received callback for node 69408a22575bc060ce15ed83 INFO: 172.21.0.1:50918 - "POST /node/69408a22575bc060ce15ed83 HTTP/1.0" 202 Accepted 2025-12-17 22:22:45,722 - lava_callback - INFO - Received callback for node 69408a25575bc060ce15ed86 INFO: 172.21.0.1:50926 - "POST /node/69408a25575bc060ce15ed86 HTTP/1.0" 202 Accepted 2025-12-17 22:22:45,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408870575bc060ce15e70e 2025-12-17 22:22:45,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408870575bc060ce15e70e HTTP/1.1" 502 150 2025-12-17 22:22:45,751 - lava_callback - ERROR - Error processing callback for node 69408870575bc060ce15e70e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408870575bc060ce15e70e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408870575bc060ce15e70e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408870575bc060ce15e70e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:45,752 - lava_callback - INFO - Starting async job submit for node 6940888d575bc060ce15e744 2025-12-17 22:22:45,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:45,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940888d575bc060ce15e744 HTTP/1.1" 502 150 2025-12-17 22:22:45,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940888d575bc060ce15e744'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:45,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940888d575bc060ce15e744 2025-12-17 22:22:45,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940888d575bc060ce15e744 HTTP/1.1" 502 150 2025-12-17 22:22:45,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940888d575bc060ce15e744'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:45,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408872575bc060ce15e711 2025-12-17 22:22:45,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408872575bc060ce15e711 HTTP/1.1" 502 150 2025-12-17 22:22:45,908 - lava_callback - ERROR - Error processing callback for node 69408872575bc060ce15e711: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408872575bc060ce15e711 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408872575bc060ce15e711 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408872575bc060ce15e711 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:45,911 - lava_callback - INFO - Starting async job submit for node 6940888e575bc060ce15e749 2025-12-17 22:22:45,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:45,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940888e575bc060ce15e749 HTTP/1.1" 502 150 2025-12-17 22:22:45,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940888e575bc060ce15e749'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:45,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940888e575bc060ce15e749 2025-12-17 22:22:45,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940888e575bc060ce15e749 HTTP/1.1" 502 150 2025-12-17 22:22:45,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940888e575bc060ce15e749'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:46,352 - lava_callback - INFO - Received callback for node 69408a26575bc060ce15ed8e INFO: 172.21.0.1:50928 - "POST /node/69408a26575bc060ce15ed8e HTTP/1.0" 202 Accepted INFO: 172.21.0.1:50942 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:22:47,036 - lava_callback - INFO - Received callback for node 69408a28575bc060ce15ed91 INFO: 172.21.0.1:50952 - "POST /node/69408a28575bc060ce15ed91 HTTP/1.0" 202 Accepted 2025-12-17 22:22:47,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408887575bc060ce15e741 2025-12-17 22:22:47,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408887575bc060ce15e741 HTTP/1.1" 502 150 2025-12-17 22:22:47,410 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408887575bc060ce15e741'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:47,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408880575bc060ce15e731 2025-12-17 22:22:47,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408880575bc060ce15e731 HTTP/1.1" 502 150 2025-12-17 22:22:47,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408880575bc060ce15e731'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:47,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408882575bc060ce15e739 2025-12-17 22:22:47,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408882575bc060ce15e739 HTTP/1.1" 502 150 2025-12-17 22:22:47,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408882575bc060ce15e739'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:47,690 - lava_callback - INFO - Received callback for node 69408a2a575bc060ce15ed94 INFO: 172.21.0.1:50958 - "POST /node/69408a2a575bc060ce15ed94 HTTP/1.0" 202 Accepted 2025-12-17 22:22:47,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940888d575bc060ce15e744 2025-12-17 22:22:47,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940888d575bc060ce15e744 HTTP/1.1" 502 150 2025-12-17 22:22:47,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940888d575bc060ce15e744'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:47,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408876575bc060ce15e71b 2025-12-17 22:22:47,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408876575bc060ce15e71b HTTP/1.1" 502 150 2025-12-17 22:22:47,815 - lava_callback - ERROR - Error processing callback for node 69408876575bc060ce15e71b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408876575bc060ce15e71b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408876575bc060ce15e71b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408876575bc060ce15e71b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:47,817 - lava_callback - INFO - Starting async job submit for node 69408891575bc060ce15e750 2025-12-17 22:22:47,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:47,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408891575bc060ce15e750 HTTP/1.1" 502 150 2025-12-17 22:22:47,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408891575bc060ce15e750'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:47,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408891575bc060ce15e750 2025-12-17 22:22:47,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408891575bc060ce15e750 HTTP/1.1" 502 150 2025-12-17 22:22:47,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408891575bc060ce15e750'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:47,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940888e575bc060ce15e749 2025-12-17 22:22:47,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940888e575bc060ce15e749 HTTP/1.1" 502 150 2025-12-17 22:22:47,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940888e575bc060ce15e749'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:47,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408884575bc060ce15e73e 2025-12-17 22:22:47,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408884575bc060ce15e73e HTTP/1.1" 502 150 2025-12-17 22:22:47,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408884575bc060ce15e73e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:48,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408874575bc060ce15e714 2025-12-17 22:22:48,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408874575bc060ce15e714 HTTP/1.1" 502 150 2025-12-17 22:22:48,011 - lava_callback - ERROR - Error processing callback for node 69408874575bc060ce15e714: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408874575bc060ce15e714 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408874575bc060ce15e714 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408874575bc060ce15e714 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:48,013 - lava_callback - INFO - Starting async job submit for node 69408893575bc060ce15e753 2025-12-17 22:22:48,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:48,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408893575bc060ce15e753 HTTP/1.1" 502 150 2025-12-17 22:22:48,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408893575bc060ce15e753'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:48,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408893575bc060ce15e753 2025-12-17 22:22:48,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408893575bc060ce15e753 HTTP/1.1" 502 150 2025-12-17 22:22:48,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408893575bc060ce15e753'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:48,357 - lava_callback - INFO - Received callback for node 69408a2c575bc060ce15ed97 INFO: 172.21.0.1:50974 - "POST /node/69408a2c575bc060ce15ed97 HTTP/1.0" 202 Accepted 2025-12-17 22:22:48,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408878575bc060ce15e71e 2025-12-17 22:22:48,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408878575bc060ce15e71e HTTP/1.1" 502 150 2025-12-17 22:22:48,415 - lava_callback - ERROR - Error processing callback for node 69408878575bc060ce15e71e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408878575bc060ce15e71e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408878575bc060ce15e71e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408878575bc060ce15e71e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:48,417 - lava_callback - INFO - Starting async job submit for node 69408895575bc060ce15e756 2025-12-17 22:22:48,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:48,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408895575bc060ce15e756 HTTP/1.1" 502 150 2025-12-17 22:22:48,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408895575bc060ce15e756'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:48,427 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408895575bc060ce15e756 2025-12-17 22:22:48,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408895575bc060ce15e756 HTTP/1.1" 502 150 2025-12-17 22:22:48,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408895575bc060ce15e756'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:48,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887e575bc060ce15e72e HTTP/1.1" 502 150 2025-12-17 22:22:48,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887e575bc060ce15e72e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:49,015 - lava_callback - INFO - Received callback for node 69408a32575bc060ce15ed9a INFO: 172.21.0.1:50978 - "POST /node/69408a32575bc060ce15ed9a HTTP/1.0" 202 Accepted 2025-12-17 22:22:49,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408891575bc060ce15e750 2025-12-17 22:22:49,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408891575bc060ce15e750 HTTP/1.1" 502 150 2025-12-17 22:22:49,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408891575bc060ce15e750'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:50,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408893575bc060ce15e753 2025-12-17 22:22:50,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408893575bc060ce15e753 HTTP/1.1" 502 150 2025-12-17 22:22:50,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408893575bc060ce15e753'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:50,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408895575bc060ce15e756 2025-12-17 22:22:50,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408895575bc060ce15e756 HTTP/1.1" 502 150 2025-12-17 22:22:50,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408895575bc060ce15e756'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:51,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ce69460bc8e265111f3d 2025-12-17 22:22:51,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ce69460bc8e265111f3d HTTP/1.1" 502 150 2025-12-17 22:22:51,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ce69460bc8e265111f3d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:51,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940888d575bc060ce15e744 2025-12-17 22:22:51,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940888d575bc060ce15e744 HTTP/1.1" 502 150 2025-12-17 22:22:51,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940888d575bc060ce15e744'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:51,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887a575bc060ce15e721 2025-12-17 22:22:51,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887a575bc060ce15e721 HTTP/1.1" 502 150 2025-12-17 22:22:51,828 - lava_callback - ERROR - Error processing callback for node 6940887a575bc060ce15e721: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940887a575bc060ce15e721 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940887a575bc060ce15e721 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940887a575bc060ce15e721 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:51,829 - lava_callback - INFO - Starting async job submit for node 69408897575bc060ce15e759 2025-12-17 22:22:51,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:51,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408897575bc060ce15e759 HTTP/1.1" 502 150 2025-12-17 22:22:51,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408897575bc060ce15e759'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:51,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408897575bc060ce15e759 2025-12-17 22:22:51,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408897575bc060ce15e759 HTTP/1.1" 502 150 2025-12-17 22:22:51,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408897575bc060ce15e759'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:51,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940888e575bc060ce15e749 2025-12-17 22:22:51,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940888e575bc060ce15e749 HTTP/1.1" 502 150 2025-12-17 22:22:51,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940888e575bc060ce15e749'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:52,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887e575bc060ce15e72e 2025-12-17 22:22:52,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887e575bc060ce15e72e HTTP/1.1" 502 150 2025-12-17 22:22:52,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887e575bc060ce15e72e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:53,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408891575bc060ce15e750 2025-12-17 22:22:53,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408891575bc060ce15e750 HTTP/1.1" 502 150 2025-12-17 22:22:53,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408891575bc060ce15e750'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:53,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408897575bc060ce15e759 2025-12-17 22:22:53,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408897575bc060ce15e759 HTTP/1.1" 502 150 2025-12-17 22:22:53,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408897575bc060ce15e759'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:54,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408893575bc060ce15e753 2025-12-17 22:22:54,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408893575bc060ce15e753 HTTP/1.1" 502 150 2025-12-17 22:22:54,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408893575bc060ce15e753'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:54,256 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940885f575bc060ce15e6bb 2025-12-17 22:22:54,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940885f575bc060ce15e6bb HTTP/1.1" 502 150 2025-12-17 22:22:54,258 - lava_callback - ERROR - Error processing callback for node 6940885f575bc060ce15e6bb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940885f575bc060ce15e6bb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940885f575bc060ce15e6bb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940885f575bc060ce15e6bb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:54,260 - lava_callback - INFO - Starting async job submit for node 69408899575bc060ce15e75c 2025-12-17 22:22:54,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:54,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408899575bc060ce15e75c HTTP/1.1" 502 150 2025-12-17 22:22:54,291 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408899575bc060ce15e75c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:54,292 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408899575bc060ce15e75c 2025-12-17 22:22:54,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408899575bc060ce15e75c HTTP/1.1" 502 150 2025-12-17 22:22:54,292 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408899575bc060ce15e75c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:54,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408895575bc060ce15e756 2025-12-17 22:22:54,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408895575bc060ce15e756 HTTP/1.1" 502 150 2025-12-17 22:22:54,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408895575bc060ce15e756'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:55,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408887575bc060ce15e741 2025-12-17 22:22:55,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408887575bc060ce15e741 HTTP/1.1" 502 150 2025-12-17 22:22:55,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408887575bc060ce15e741'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:56,276 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940886a575bc060ce15e703 2025-12-17 22:22:56,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940886a575bc060ce15e703 HTTP/1.1" 502 150 2025-12-17 22:22:56,277 - lava_callback - ERROR - Error processing callback for node 6940886a575bc060ce15e703: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940886a575bc060ce15e703 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940886a575bc060ce15e703 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940886a575bc060ce15e703 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:56,279 - lava_callback - INFO - Starting async job submit for node 6940889b575bc060ce15e75f 2025-12-17 22:22:56,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:56,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889b575bc060ce15e75f HTTP/1.1" 502 150 2025-12-17 22:22:56,289 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940889b575bc060ce15e75f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:56,289 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940889b575bc060ce15e75f 2025-12-17 22:22:56,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889b575bc060ce15e75f HTTP/1.1" 502 150 2025-12-17 22:22:56,290 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940889b575bc060ce15e75f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:56,292 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408899575bc060ce15e75c 2025-12-17 22:22:56,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408899575bc060ce15e75c HTTP/1.1" 502 150 2025-12-17 22:22:56,293 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408899575bc060ce15e75c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47444 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:22:56,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887b575bc060ce15e724 2025-12-17 22:22:56,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887b575bc060ce15e724 HTTP/1.1" 502 150 2025-12-17 22:22:56,770 - lava_callback - ERROR - Error processing callback for node 6940887b575bc060ce15e724: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940887b575bc060ce15e724 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940887b575bc060ce15e724 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940887b575bc060ce15e724 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:56,772 - lava_callback - INFO - Starting async job submit for node 6940889c575bc060ce15e762 2025-12-17 22:22:56,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:56,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889c575bc060ce15e762 HTTP/1.1" 502 150 2025-12-17 22:22:56,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940889c575bc060ce15e762'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:56,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940889c575bc060ce15e762 2025-12-17 22:22:56,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889c575bc060ce15e762 HTTP/1.1" 502 150 2025-12-17 22:22:56,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940889c575bc060ce15e762'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:57,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408897575bc060ce15e759 2025-12-17 22:22:57,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408897575bc060ce15e759 HTTP/1.1" 502 150 2025-12-17 22:22:57,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408897575bc060ce15e759'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:58,290 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940889b575bc060ce15e75f 2025-12-17 22:22:58,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889b575bc060ce15e75f HTTP/1.1" 502 150 2025-12-17 22:22:58,291 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940889b575bc060ce15e75f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:58,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940889c575bc060ce15e762 2025-12-17 22:22:58,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889c575bc060ce15e762 HTTP/1.1" 502 150 2025-12-17 22:22:58,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940889c575bc060ce15e762'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:59,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940888d575bc060ce15e744 2025-12-17 22:22:59,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940888d575bc060ce15e744 HTTP/1.1" 502 150 2025-12-17 22:22:59,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940888d575bc060ce15e744'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:59,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887c575bc060ce15e72b 2025-12-17 22:22:59,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887c575bc060ce15e72b HTTP/1.1" 502 150 2025-12-17 22:22:59,897 - lava_callback - ERROR - Error processing callback for node 6940887c575bc060ce15e72b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940887c575bc060ce15e72b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940887c575bc060ce15e72b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940887c575bc060ce15e72b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:22:59,898 - lava_callback - INFO - Starting async job submit for node 6940889e575bc060ce15e765 2025-12-17 22:22:59,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:22:59,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889e575bc060ce15e765 HTTP/1.1" 502 150 2025-12-17 22:22:59,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940889e575bc060ce15e765'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:59,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940889e575bc060ce15e765 2025-12-17 22:22:59,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889e575bc060ce15e765 HTTP/1.1" 502 150 2025-12-17 22:22:59,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940889e575bc060ce15e765'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:22:59,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940888e575bc060ce15e749 2025-12-17 22:22:59,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940888e575bc060ce15e749 HTTP/1.1" 502 150 2025-12-17 22:22:59,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940888e575bc060ce15e749'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:00,294 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408899575bc060ce15e75c 2025-12-17 22:23:00,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887e575bc060ce15e72e 2025-12-17 22:23:00,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887e575bc060ce15e72e HTTP/1.1" 502 150 2025-12-17 22:23:00,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940887e575bc060ce15e72e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:01,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408891575bc060ce15e750 2025-12-17 22:23:01,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408891575bc060ce15e750 HTTP/1.1" 502 150 2025-12-17 22:23:01,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408891575bc060ce15e750'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:01,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940889e575bc060ce15e765 2025-12-17 22:23:01,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889e575bc060ce15e765 HTTP/1.1" 502 150 2025-12-17 22:23:01,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940889e575bc060ce15e765'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:02,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408893575bc060ce15e753 2025-12-17 22:23:02,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408893575bc060ce15e753 HTTP/1.1" 502 150 2025-12-17 22:23:02,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408893575bc060ce15e753'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60322 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:23:02,292 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940889b575bc060ce15e75f 2025-12-17 22:23:02,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889b575bc060ce15e75f HTTP/1.1" 502 150 2025-12-17 22:23:02,293 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940889b575bc060ce15e75f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:02,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408895575bc060ce15e756 2025-12-17 22:23:02,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408895575bc060ce15e756 HTTP/1.1" 502 150 2025-12-17 22:23:02,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408895575bc060ce15e756'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:02,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940889c575bc060ce15e762 2025-12-17 22:23:02,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889c575bc060ce15e762 HTTP/1.1" 502 150 2025-12-17 22:23:02,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940889c575bc060ce15e762'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:03,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408880575bc060ce15e731 2025-12-17 22:23:03,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408880575bc060ce15e731 HTTP/1.1" 502 150 2025-12-17 22:23:03,540 - lava_callback - ERROR - Error processing callback for node 69408880575bc060ce15e731: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408880575bc060ce15e731 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408880575bc060ce15e731 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408880575bc060ce15e731 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:03,542 - lava_callback - INFO - Starting async job submit for node 694088a5575bc060ce15e7bb 2025-12-17 22:23:03,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:03,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088a5575bc060ce15e7bb HTTP/1.1" 502 150 2025-12-17 22:23:03,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088a5575bc060ce15e7bb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:03,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088a5575bc060ce15e7bb 2025-12-17 22:23:03,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088a5575bc060ce15e7bb HTTP/1.1" 502 150 2025-12-17 22:23:03,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088a5575bc060ce15e7bb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:03,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408882575bc060ce15e739 2025-12-17 22:23:03,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408882575bc060ce15e739 HTTP/1.1" 502 150 2025-12-17 22:23:03,665 - lava_callback - ERROR - Error processing callback for node 69408882575bc060ce15e739: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408882575bc060ce15e739 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408882575bc060ce15e739 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408882575bc060ce15e739 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:03,667 - lava_callback - INFO - Starting async job submit for node 694088a8575bc060ce15e7c2 2025-12-17 22:23:03,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:03,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088a8575bc060ce15e7c2 HTTP/1.1" 502 150 2025-12-17 22:23:03,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088a8575bc060ce15e7c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:03,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088a8575bc060ce15e7c2 2025-12-17 22:23:03,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088a8575bc060ce15e7c2 HTTP/1.1" 502 150 2025-12-17 22:23:03,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088a8575bc060ce15e7c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:03,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408884575bc060ce15e73e 2025-12-17 22:23:03,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408884575bc060ce15e73e HTTP/1.1" 502 150 2025-12-17 22:23:03,999 - lava_callback - ERROR - Error processing callback for node 69408884575bc060ce15e73e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408884575bc060ce15e73e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408884575bc060ce15e73e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408884575bc060ce15e73e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:04,001 - lava_callback - INFO - Starting async job submit for node 694088aa575bc060ce15e7c5 2025-12-17 22:23:04,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:04,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088aa575bc060ce15e7c5 HTTP/1.1" 502 150 2025-12-17 22:23:04,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088aa575bc060ce15e7c5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:04,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088aa575bc060ce15e7c5 2025-12-17 22:23:04,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088aa575bc060ce15e7c5 HTTP/1.1" 502 150 2025-12-17 22:23:04,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088aa575bc060ce15e7c5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:04,486 - lava_callback - INFO - Received callback for node 69408a34575bc060ce15eda3 INFO: 172.21.0.1:60324 - "POST /node/69408a34575bc060ce15eda3 HTTP/1.0" 202 Accepted 2025-12-17 22:23:05,123 - lava_callback - INFO - Received callback for node 69408a36575bc060ce15edaa INFO: 172.21.0.1:60340 - "POST /node/69408a36575bc060ce15edaa HTTP/1.0" 202 Accepted 2025-12-17 22:23:05,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088a5575bc060ce15e7bb 2025-12-17 22:23:05,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088a5575bc060ce15e7bb HTTP/1.1" 502 150 2025-12-17 22:23:05,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088a5575bc060ce15e7bb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:05,756 - lava_callback - INFO - Received callback for node 69408a38575bc060ce15edad INFO: 172.21.0.1:60352 - "POST /node/69408a38575bc060ce15edad HTTP/1.0" 202 Accepted 2025-12-17 22:23:05,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408897575bc060ce15e759 2025-12-17 22:23:05,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408897575bc060ce15e759 HTTP/1.1" 502 150 2025-12-17 22:23:05,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408897575bc060ce15e759'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:05,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088a8575bc060ce15e7c2 2025-12-17 22:23:05,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088a8575bc060ce15e7c2 HTTP/1.1" 502 150 2025-12-17 22:23:05,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088a8575bc060ce15e7c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:05,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940889e575bc060ce15e765 2025-12-17 22:23:05,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889e575bc060ce15e765 HTTP/1.1" 502 150 2025-12-17 22:23:05,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940889e575bc060ce15e765'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:06,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088aa575bc060ce15e7c5 2025-12-17 22:23:06,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088aa575bc060ce15e7c5 HTTP/1.1" 502 150 2025-12-17 22:23:06,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088aa575bc060ce15e7c5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:06,392 - lava_callback - INFO - Received callback for node 69408a3b575bc060ce15edb0 INFO: 172.21.0.1:60364 - "POST /node/69408a3b575bc060ce15edb0 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:60378 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:23:07,041 - lava_callback - INFO - Received callback for node 69408a3d575bc060ce15edb3 INFO: 172.21.0.1:60386 - "POST /node/69408a3d575bc060ce15edb3 HTTP/1.0" 202 Accepted 2025-12-17 22:23:07,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ce69460bc8e265111f3d 2025-12-17 22:23:07,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ce69460bc8e265111f3d HTTP/1.1" 502 150 2025-12-17 22:23:07,635 - lava_callback - ERROR - Error processing callback for node 6940ce69460bc8e265111f3d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ce69460bc8e265111f3d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ce69460bc8e265111f3d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ce69460bc8e265111f3d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:07,637 - lava_callback - INFO - Starting async job submit for node 694088ac575bc060ce15e7c9 2025-12-17 22:23:07,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:07,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ac575bc060ce15e7c9 HTTP/1.1" 502 150 2025-12-17 22:23:07,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ac575bc060ce15e7c9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:07,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ac575bc060ce15e7c9 2025-12-17 22:23:07,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ac575bc060ce15e7c9 HTTP/1.1" 502 150 2025-12-17 22:23:07,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ac575bc060ce15e7c9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:07,703 - lava_callback - INFO - Received callback for node 69408a3f575bc060ce15edb6 INFO: 172.21.0.1:60396 - "POST /node/69408a3f575bc060ce15edb6 HTTP/1.0" 202 Accepted 2025-12-17 22:23:08,370 - lava_callback - INFO - Received callback for node 69408a42575bc060ce15edbc INFO: 172.21.0.1:60404 - "POST /node/69408a42575bc060ce15edbc HTTP/1.0" 202 Accepted 2025-12-17 22:23:09,008 - lava_callback - INFO - Received callback for node 69408a44575bc060ce15edbf INFO: 172.21.0.1:60416 - "POST /node/69408a44575bc060ce15edbf HTTP/1.0" 202 Accepted 2025-12-17 22:23:09,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088a5575bc060ce15e7bb 2025-12-17 22:23:09,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088a5575bc060ce15e7bb HTTP/1.1" 502 150 2025-12-17 22:23:09,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088a5575bc060ce15e7bb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:09,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ac575bc060ce15e7c9 2025-12-17 22:23:09,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ac575bc060ce15e7c9 HTTP/1.1" 502 150 2025-12-17 22:23:09,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ac575bc060ce15e7c9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:09,669 - lava_callback - INFO - Received callback for node 69408a46575bc060ce15edc2 INFO: 172.21.0.1:60422 - "POST /node/69408a46575bc060ce15edc2 HTTP/1.0" 202 Accepted 2025-12-17 22:23:09,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088a8575bc060ce15e7c2 2025-12-17 22:23:09,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088a8575bc060ce15e7c2 HTTP/1.1" 502 150 2025-12-17 22:23:09,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088a8575bc060ce15e7c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:10,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088aa575bc060ce15e7c5 2025-12-17 22:23:10,293 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940889b575bc060ce15e75f 2025-12-17 22:23:10,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889b575bc060ce15e75f HTTP/1.1" 502 150 2025-12-17 22:23:10,295 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940889b575bc060ce15e75f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:10,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940889c575bc060ce15e762 2025-12-17 22:23:10,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889c575bc060ce15e762 HTTP/1.1" 502 150 2025-12-17 22:23:10,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940889c575bc060ce15e762'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:11,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408887575bc060ce15e741 2025-12-17 22:23:11,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408887575bc060ce15e741 HTTP/1.1" 502 150 2025-12-17 22:23:11,414 - lava_callback - ERROR - Error processing callback for node 69408887575bc060ce15e741: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408887575bc060ce15e741 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408887575bc060ce15e741 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408887575bc060ce15e741 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:11,415 - lava_callback - INFO - Starting async job submit for node 694088b2575bc060ce15e7cc 2025-12-17 22:23:11,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:11,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b2575bc060ce15e7cc HTTP/1.1" 502 150 2025-12-17 22:23:11,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b2575bc060ce15e7cc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:11,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b2575bc060ce15e7cc 2025-12-17 22:23:11,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b2575bc060ce15e7cc HTTP/1.1" 502 150 2025-12-17 22:23:11,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b2575bc060ce15e7cc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:13,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b2575bc060ce15e7cc 2025-12-17 22:23:13,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b2575bc060ce15e7cc HTTP/1.1" 502 150 2025-12-17 22:23:13,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b2575bc060ce15e7cc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:13,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ac575bc060ce15e7c9 2025-12-17 22:23:13,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ac575bc060ce15e7c9 HTTP/1.1" 502 150 2025-12-17 22:23:13,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ac575bc060ce15e7c9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:13,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940889e575bc060ce15e765 2025-12-17 22:23:13,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889e575bc060ce15e765 HTTP/1.1" 502 150 2025-12-17 22:23:13,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940889e575bc060ce15e765'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38578 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:23:15,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408899575bc060ce15e75c HTTP/1.1" 502 150 2025-12-17 22:23:15,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408899575bc060ce15e75c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:15,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940888d575bc060ce15e744 2025-12-17 22:23:15,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940888d575bc060ce15e744 HTTP/1.1" 502 150 2025-12-17 22:23:15,768 - lava_callback - ERROR - Error processing callback for node 6940888d575bc060ce15e744: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940888d575bc060ce15e744 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940888d575bc060ce15e744 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940888d575bc060ce15e744 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:15,770 - lava_callback - INFO - Starting async job submit for node 694088b4575bc060ce15e7d3 2025-12-17 22:23:15,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:15,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b4575bc060ce15e7d3 HTTP/1.1" 502 150 2025-12-17 22:23:15,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b4575bc060ce15e7d3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:15,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b4575bc060ce15e7d3 2025-12-17 22:23:15,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b4575bc060ce15e7d3 HTTP/1.1" 502 150 2025-12-17 22:23:15,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b4575bc060ce15e7d3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:15,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940888e575bc060ce15e749 2025-12-17 22:23:15,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940888e575bc060ce15e749 HTTP/1.1" 502 150 2025-12-17 22:23:15,927 - lava_callback - ERROR - Error processing callback for node 6940888e575bc060ce15e749: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940888e575bc060ce15e749 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940888e575bc060ce15e749 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940888e575bc060ce15e749 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:15,928 - lava_callback - INFO - Starting async job submit for node 694088b6575bc060ce15e7d6 2025-12-17 22:23:15,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:15,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b6575bc060ce15e7d6 HTTP/1.1" 502 150 2025-12-17 22:23:15,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b6575bc060ce15e7d6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:15,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b6575bc060ce15e7d6 2025-12-17 22:23:15,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b6575bc060ce15e7d6 HTTP/1.1" 502 150 2025-12-17 22:23:15,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b6575bc060ce15e7d6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38582 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:23:16,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940887e575bc060ce15e72e 2025-12-17 22:23:16,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940887e575bc060ce15e72e HTTP/1.1" 502 150 2025-12-17 22:23:16,758 - lava_callback - ERROR - Error processing callback for node 6940887e575bc060ce15e72e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940887e575bc060ce15e72e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940887e575bc060ce15e72e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940887e575bc060ce15e72e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:16,760 - lava_callback - INFO - Starting async job submit for node 694088b8575bc060ce15e7df 2025-12-17 22:23:16,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:16,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b8575bc060ce15e7df HTTP/1.1" 502 150 2025-12-17 22:23:16,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b8575bc060ce15e7df'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:16,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b8575bc060ce15e7df 2025-12-17 22:23:16,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b8575bc060ce15e7df HTTP/1.1" 502 150 2025-12-17 22:23:16,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b8575bc060ce15e7df'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:17,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b2575bc060ce15e7cc 2025-12-17 22:23:17,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b2575bc060ce15e7cc HTTP/1.1" 502 150 2025-12-17 22:23:17,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b2575bc060ce15e7cc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:17,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088a5575bc060ce15e7bb 2025-12-17 22:23:17,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088a5575bc060ce15e7bb HTTP/1.1" 502 150 2025-12-17 22:23:17,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088a5575bc060ce15e7bb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:17,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b4575bc060ce15e7d3 2025-12-17 22:23:17,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b4575bc060ce15e7d3 HTTP/1.1" 502 150 2025-12-17 22:23:17,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b4575bc060ce15e7d3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:17,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408891575bc060ce15e750 2025-12-17 22:23:17,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408891575bc060ce15e750 HTTP/1.1" 502 150 2025-12-17 22:23:17,831 - lava_callback - ERROR - Error processing callback for node 69408891575bc060ce15e750: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408891575bc060ce15e750 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408891575bc060ce15e750 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408891575bc060ce15e750 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:17,833 - lava_callback - INFO - Starting async job submit for node 694088ba575bc060ce15e7e8 2025-12-17 22:23:17,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:17,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ba575bc060ce15e7e8 HTTP/1.1" 502 150 2025-12-17 22:23:17,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ba575bc060ce15e7e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:17,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ba575bc060ce15e7e8 2025-12-17 22:23:17,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ba575bc060ce15e7e8 HTTP/1.1" 502 150 2025-12-17 22:23:17,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ba575bc060ce15e7e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:17,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088a8575bc060ce15e7c2 2025-12-17 22:23:17,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088a8575bc060ce15e7c2 HTTP/1.1" 502 150 2025-12-17 22:23:17,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088a8575bc060ce15e7c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:17,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b6575bc060ce15e7d6 2025-12-17 22:23:17,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b6575bc060ce15e7d6 HTTP/1.1" 502 150 2025-12-17 22:23:17,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b6575bc060ce15e7d6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:18,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408893575bc060ce15e753 2025-12-17 22:23:18,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408893575bc060ce15e753 HTTP/1.1" 502 150 2025-12-17 22:23:18,028 - lava_callback - ERROR - Error processing callback for node 69408893575bc060ce15e753: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408893575bc060ce15e753 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408893575bc060ce15e753 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408893575bc060ce15e753 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:18,030 - lava_callback - INFO - Starting async job submit for node 694088bb575bc060ce15e7eb 2025-12-17 22:23:18,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:18,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bb575bc060ce15e7eb HTTP/1.1" 502 150 2025-12-17 22:23:18,037 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088bb575bc060ce15e7eb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:18,037 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088bb575bc060ce15e7eb 2025-12-17 22:23:18,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bb575bc060ce15e7eb HTTP/1.1" 502 150 2025-12-17 22:23:18,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088bb575bc060ce15e7eb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:18,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408895575bc060ce15e756 2025-12-17 22:23:18,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408895575bc060ce15e756 HTTP/1.1" 502 150 2025-12-17 22:23:18,435 - lava_callback - ERROR - Error processing callback for node 69408895575bc060ce15e756: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408895575bc060ce15e756 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408895575bc060ce15e756 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408895575bc060ce15e756 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:18,437 - lava_callback - INFO - Starting async job submit for node 694088bc575bc060ce15e7ee 2025-12-17 22:23:18,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:18,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bc575bc060ce15e7ee HTTP/1.1" 502 150 2025-12-17 22:23:18,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088bc575bc060ce15e7ee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:18,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088bc575bc060ce15e7ee 2025-12-17 22:23:18,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bc575bc060ce15e7ee HTTP/1.1" 502 150 2025-12-17 22:23:18,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088bc575bc060ce15e7ee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:18,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b8575bc060ce15e7df 2025-12-17 22:23:18,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b8575bc060ce15e7df HTTP/1.1" 502 150 2025-12-17 22:23:18,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b8575bc060ce15e7df'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:19,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ba575bc060ce15e7e8 2025-12-17 22:23:19,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ba575bc060ce15e7e8 HTTP/1.1" 502 150 2025-12-17 22:23:19,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ba575bc060ce15e7e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:20,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088bb575bc060ce15e7eb 2025-12-17 22:23:20,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bb575bc060ce15e7eb HTTP/1.1" 502 150 2025-12-17 22:23:20,039 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088bb575bc060ce15e7eb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:20,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088bc575bc060ce15e7ee 2025-12-17 22:23:20,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bc575bc060ce15e7ee HTTP/1.1" 502 150 2025-12-17 22:23:20,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088bc575bc060ce15e7ee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:21,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ac575bc060ce15e7c9 2025-12-17 22:23:21,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ac575bc060ce15e7c9 HTTP/1.1" 502 150 2025-12-17 22:23:21,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ac575bc060ce15e7c9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:21,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b4575bc060ce15e7d3 2025-12-17 22:23:21,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b4575bc060ce15e7d3 HTTP/1.1" 502 150 2025-12-17 22:23:21,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b4575bc060ce15e7d3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:21,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408897575bc060ce15e759 2025-12-17 22:23:21,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408897575bc060ce15e759 HTTP/1.1" 502 150 2025-12-17 22:23:21,844 - lava_callback - ERROR - Error processing callback for node 69408897575bc060ce15e759: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408897575bc060ce15e759 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408897575bc060ce15e759 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408897575bc060ce15e759 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:21,846 - lava_callback - INFO - Starting async job submit for node 694088be575bc060ce15e7f5 2025-12-17 22:23:21,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:21,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088be575bc060ce15e7f5 HTTP/1.1" 502 150 2025-12-17 22:23:21,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088be575bc060ce15e7f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:21,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088be575bc060ce15e7f5 2025-12-17 22:23:21,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088be575bc060ce15e7f5 HTTP/1.1" 502 150 2025-12-17 22:23:21,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088be575bc060ce15e7f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:21,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b6575bc060ce15e7d6 2025-12-17 22:23:21,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b6575bc060ce15e7d6 HTTP/1.1" 502 150 2025-12-17 22:23:21,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b6575bc060ce15e7d6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:22,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b8575bc060ce15e7df 2025-12-17 22:23:22,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b8575bc060ce15e7df HTTP/1.1" 502 150 2025-12-17 22:23:22,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b8575bc060ce15e7df'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:23,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408899575bc060ce15e75c 2025-12-17 22:23:23,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408899575bc060ce15e75c HTTP/1.1" 502 150 2025-12-17 22:23:23,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408899575bc060ce15e75c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:23,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ba575bc060ce15e7e8 2025-12-17 22:23:23,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ba575bc060ce15e7e8 HTTP/1.1" 502 150 2025-12-17 22:23:23,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ba575bc060ce15e7e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:23,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088be575bc060ce15e7f5 2025-12-17 22:23:23,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088be575bc060ce15e7f5 HTTP/1.1" 502 150 2025-12-17 22:23:23,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088be575bc060ce15e7f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:24,040 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088bb575bc060ce15e7eb 2025-12-17 22:23:24,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bb575bc060ce15e7eb HTTP/1.1" 502 150 2025-12-17 22:23:24,041 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088bb575bc060ce15e7eb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:24,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088bc575bc060ce15e7ee 2025-12-17 22:23:24,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bc575bc060ce15e7ee HTTP/1.1" 502 150 2025-12-17 22:23:24,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088bc575bc060ce15e7ee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:24,502 - lava_callback - INFO - Received callback for node 69408a48575bc060ce15edcd INFO: 172.21.0.1:34134 - "POST /node/69408a48575bc060ce15edcd HTTP/1.0" 202 Accepted 2025-12-17 22:23:25,154 - lava_callback - INFO - Received callback for node 69408a4a575bc060ce15edd2 INFO: 172.21.0.1:34140 - "POST /node/69408a4a575bc060ce15edd2 HTTP/1.0" 202 Accepted 2025-12-17 22:23:25,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088aa575bc060ce15e7c5 HTTP/1.1" 502 150 2025-12-17 22:23:25,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088aa575bc060ce15e7c5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:25,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b2575bc060ce15e7cc 2025-12-17 22:23:25,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b2575bc060ce15e7cc HTTP/1.1" 502 150 2025-12-17 22:23:25,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b2575bc060ce15e7cc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:25,814 - lava_callback - INFO - Received callback for node 69408a4c575bc060ce15edd5 INFO: 172.21.0.1:34142 - "POST /node/69408a4c575bc060ce15edd5 HTTP/1.0" 202 Accepted 2025-12-17 22:23:26,295 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940889b575bc060ce15e75f 2025-12-17 22:23:26,465 - lava_callback - INFO - Received callback for node 69408a4e575bc060ce15edd8 INFO: 172.21.0.1:34158 - "POST /node/69408a4e575bc060ce15edd8 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:34160 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:23:26,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940889c575bc060ce15e762 2025-12-17 22:23:26,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889c575bc060ce15e762 HTTP/1.1" 502 150 2025-12-17 22:23:26,803 - lava_callback - ERROR - Error processing callback for node 6940889c575bc060ce15e762: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940889c575bc060ce15e762 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940889c575bc060ce15e762 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940889c575bc060ce15e762 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:26,806 - lava_callback - INFO - Starting async job submit for node 694088bf575bc060ce15e7f8 2025-12-17 22:23:26,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:26,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bf575bc060ce15e7f8 HTTP/1.1" 502 150 2025-12-17 22:23:26,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088bf575bc060ce15e7f8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:26,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088bf575bc060ce15e7f8 2025-12-17 22:23:26,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bf575bc060ce15e7f8 HTTP/1.1" 502 150 2025-12-17 22:23:26,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088bf575bc060ce15e7f8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:27,122 - lava_callback - INFO - Received callback for node 69408a50575bc060ce15eddb INFO: 172.21.0.1:34162 - "POST /node/69408a50575bc060ce15eddb HTTP/1.0" 202 Accepted 2025-12-17 22:23:27,752 - lava_callback - INFO - Received callback for node 69408a52575bc060ce15ede2 INFO: 172.21.0.1:34174 - "POST /node/69408a52575bc060ce15ede2 HTTP/1.0" 202 Accepted 2025-12-17 22:23:27,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088be575bc060ce15e7f5 2025-12-17 22:23:27,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088be575bc060ce15e7f5 HTTP/1.1" 502 150 2025-12-17 22:23:27,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088be575bc060ce15e7f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:28,405 - lava_callback - INFO - Received callback for node 69408a55575bc060ce15ee1f INFO: 172.21.0.1:34178 - "POST /node/69408a55575bc060ce15ee1f HTTP/1.0" 202 Accepted 2025-12-17 22:23:28,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088bf575bc060ce15e7f8 2025-12-17 22:23:28,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bf575bc060ce15e7f8 HTTP/1.1" 502 150 2025-12-17 22:23:28,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088bf575bc060ce15e7f8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:29,049 - lava_callback - INFO - Received callback for node 69408a57575bc060ce15ee22 INFO: 172.21.0.1:34194 - "POST /node/69408a57575bc060ce15ee22 HTTP/1.0" 202 Accepted 2025-12-17 22:23:29,689 - lava_callback - INFO - Received callback for node 69408a5a575bc060ce15ee25 INFO: 172.21.0.1:34198 - "POST /node/69408a5a575bc060ce15ee25 HTTP/1.0" 202 Accepted 2025-12-17 22:23:29,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b4575bc060ce15e7d3 2025-12-17 22:23:29,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b4575bc060ce15e7d3 HTTP/1.1" 502 150 2025-12-17 22:23:29,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b4575bc060ce15e7d3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:29,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940889e575bc060ce15e765 2025-12-17 22:23:29,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889e575bc060ce15e765 HTTP/1.1" 502 150 2025-12-17 22:23:29,911 - lava_callback - ERROR - Error processing callback for node 6940889e575bc060ce15e765: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940889e575bc060ce15e765 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940889e575bc060ce15e765 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940889e575bc060ce15e765 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:29,913 - lava_callback - INFO - Starting async job submit for node 694088c0575bc060ce15e7fb 2025-12-17 22:23:29,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:29,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c0575bc060ce15e7fb HTTP/1.1" 502 150 2025-12-17 22:23:29,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c0575bc060ce15e7fb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:29,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c0575bc060ce15e7fb 2025-12-17 22:23:29,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c0575bc060ce15e7fb HTTP/1.1" 502 150 2025-12-17 22:23:29,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c0575bc060ce15e7fb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:29,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b6575bc060ce15e7d6 2025-12-17 22:23:29,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b6575bc060ce15e7d6 HTTP/1.1" 502 150 2025-12-17 22:23:29,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b6575bc060ce15e7d6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:30,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b8575bc060ce15e7df 2025-12-17 22:23:30,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b8575bc060ce15e7df HTTP/1.1" 502 150 2025-12-17 22:23:30,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088b8575bc060ce15e7df'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:31,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ba575bc060ce15e7e8 2025-12-17 22:23:31,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ba575bc060ce15e7e8 HTTP/1.1" 502 150 2025-12-17 22:23:31,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ba575bc060ce15e7e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:31,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c0575bc060ce15e7fb 2025-12-17 22:23:31,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c0575bc060ce15e7fb HTTP/1.1" 502 150 2025-12-17 22:23:31,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c0575bc060ce15e7fb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:32,041 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088bb575bc060ce15e7eb 2025-12-17 22:23:32,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bb575bc060ce15e7eb HTTP/1.1" 502 150 2025-12-17 22:23:32,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088bb575bc060ce15e7eb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:32,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088bc575bc060ce15e7ee 2025-12-17 22:23:32,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bc575bc060ce15e7ee HTTP/1.1" 502 150 2025-12-17 22:23:32,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088bc575bc060ce15e7ee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:32,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088bf575bc060ce15e7f8 2025-12-17 22:23:32,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bf575bc060ce15e7f8 HTTP/1.1" 502 150 2025-12-17 22:23:32,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088bf575bc060ce15e7f8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:33,361 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088aa575bc060ce15e7c5 2025-12-17 22:23:33,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088aa575bc060ce15e7c5 HTTP/1.1" 502 150 2025-12-17 22:23:33,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088aa575bc060ce15e7c5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:33,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088a5575bc060ce15e7bb 2025-12-17 22:23:33,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088a5575bc060ce15e7bb HTTP/1.1" 502 150 2025-12-17 22:23:33,562 - lava_callback - ERROR - Error processing callback for node 694088a5575bc060ce15e7bb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088a5575bc060ce15e7bb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088a5575bc060ce15e7bb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088a5575bc060ce15e7bb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:33,564 - lava_callback - INFO - Starting async job submit for node 694088c2575bc060ce15e7fe 2025-12-17 22:23:33,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:33,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c2575bc060ce15e7fe HTTP/1.1" 502 150 2025-12-17 22:23:33,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c2575bc060ce15e7fe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:33,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c2575bc060ce15e7fe 2025-12-17 22:23:33,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c2575bc060ce15e7fe HTTP/1.1" 502 150 2025-12-17 22:23:33,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c2575bc060ce15e7fe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:33,613 - lava_callback - INFO - Received callback for node 6940945d575bc060ce161000 INFO: 172.21.0.1:45352 - "POST /node/6940945d575bc060ce161000 HTTP/1.0" 202 Accepted 2025-12-17 22:23:33,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088a8575bc060ce15e7c2 2025-12-17 22:23:33,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088a8575bc060ce15e7c2 HTTP/1.1" 502 150 2025-12-17 22:23:33,848 - lava_callback - ERROR - Error processing callback for node 694088a8575bc060ce15e7c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088a8575bc060ce15e7c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088a8575bc060ce15e7c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088a8575bc060ce15e7c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:33,850 - lava_callback - INFO - Starting async job submit for node 694088c3575bc060ce15e805 2025-12-17 22:23:33,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:33,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c3575bc060ce15e805 HTTP/1.1" 502 150 2025-12-17 22:23:33,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c3575bc060ce15e805'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:33,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c3575bc060ce15e805 2025-12-17 22:23:33,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c3575bc060ce15e805 HTTP/1.1" 502 150 2025-12-17 22:23:33,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c3575bc060ce15e805'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:35,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c2575bc060ce15e7fe 2025-12-17 22:23:35,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c2575bc060ce15e7fe HTTP/1.1" 502 150 2025-12-17 22:23:35,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c2575bc060ce15e7fe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:35,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088be575bc060ce15e7f5 2025-12-17 22:23:35,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088be575bc060ce15e7f5 HTTP/1.1" 502 150 2025-12-17 22:23:35,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088be575bc060ce15e7f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:35,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c3575bc060ce15e805 2025-12-17 22:23:35,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c3575bc060ce15e805 HTTP/1.1" 502 150 2025-12-17 22:23:35,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c3575bc060ce15e805'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:35,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c0575bc060ce15e7fb 2025-12-17 22:23:35,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c0575bc060ce15e7fb HTTP/1.1" 502 150 2025-12-17 22:23:35,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c0575bc060ce15e7fb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45358 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:23:37,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ac575bc060ce15e7c9 2025-12-17 22:23:39,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c2575bc060ce15e7fe 2025-12-17 22:23:39,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c2575bc060ce15e7fe HTTP/1.1" 502 150 2025-12-17 22:23:39,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c2575bc060ce15e7fe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:39,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408899575bc060ce15e75c 2025-12-17 22:23:39,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408899575bc060ce15e75c HTTP/1.1" 502 150 2025-12-17 22:23:39,635 - lava_callback - ERROR - Error processing callback for node 69408899575bc060ce15e75c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408899575bc060ce15e75c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408899575bc060ce15e75c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408899575bc060ce15e75c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:39,638 - lava_callback - INFO - Starting async job submit for node 694088c5575bc060ce15e808 2025-12-17 22:23:39,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:39,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c5575bc060ce15e808 HTTP/1.1" 502 150 2025-12-17 22:23:39,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c5575bc060ce15e808'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:39,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c5575bc060ce15e808 2025-12-17 22:23:39,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c5575bc060ce15e808 HTTP/1.1" 502 150 2025-12-17 22:23:39,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c5575bc060ce15e808'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:39,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c3575bc060ce15e805 2025-12-17 22:23:39,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c3575bc060ce15e805 HTTP/1.1" 502 150 2025-12-17 22:23:39,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c3575bc060ce15e805'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:40,711 - lava_callback - INFO - Received callback for node 6940937b575bc060ce160d99 INFO: 172.21.0.1:49826 - "POST /node/6940937b575bc060ce160d99 HTTP/1.0" 202 Accepted 2025-12-17 22:23:40,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088bf575bc060ce15e7f8 2025-12-17 22:23:40,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bf575bc060ce15e7f8 HTTP/1.1" 502 150 2025-12-17 22:23:40,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088bf575bc060ce15e7f8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:41,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b2575bc060ce15e7cc 2025-12-17 22:23:41,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b2575bc060ce15e7cc HTTP/1.1" 502 150 2025-12-17 22:23:41,456 - lava_callback - ERROR - Error processing callback for node 694088b2575bc060ce15e7cc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088b2575bc060ce15e7cc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088b2575bc060ce15e7cc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088b2575bc060ce15e7cc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:41,458 - lava_callback - INFO - Starting async job submit for node 694088c7575bc060ce15e80b 2025-12-17 22:23:41,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:41,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c7575bc060ce15e80b HTTP/1.1" 502 150 2025-12-17 22:23:41,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c7575bc060ce15e80b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:41,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c7575bc060ce15e80b 2025-12-17 22:23:41,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c7575bc060ce15e80b HTTP/1.1" 502 150 2025-12-17 22:23:41,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c7575bc060ce15e80b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:41,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940889b575bc060ce15e75f HTTP/1.1" 502 150 2025-12-17 22:23:41,748 - lava_callback - ERROR - Error processing callback for node 6940889b575bc060ce15e75f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940889b575bc060ce15e75f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940889b575bc060ce15e75f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940889b575bc060ce15e75f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:41,750 - lava_callback - INFO - Starting async job submit for node 694088cc575bc060ce15e81a 2025-12-17 22:23:41,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:41,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088cc575bc060ce15e81a HTTP/1.1" 502 150 2025-12-17 22:23:41,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088cc575bc060ce15e81a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:41,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088cc575bc060ce15e81a 2025-12-17 22:23:41,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088cc575bc060ce15e81a HTTP/1.1" 502 150 2025-12-17 22:23:41,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088cc575bc060ce15e81a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:41,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c5575bc060ce15e808 2025-12-17 22:23:41,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c5575bc060ce15e808 HTTP/1.1" 502 150 2025-12-17 22:23:41,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c5575bc060ce15e808'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:43,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c7575bc060ce15e80b 2025-12-17 22:23:43,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c7575bc060ce15e80b HTTP/1.1" 502 150 2025-12-17 22:23:43,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c7575bc060ce15e80b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:43,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088cc575bc060ce15e81a 2025-12-17 22:23:43,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088cc575bc060ce15e81a HTTP/1.1" 502 150 2025-12-17 22:23:43,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088cc575bc060ce15e81a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:43,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c0575bc060ce15e7fb 2025-12-17 22:23:43,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c0575bc060ce15e7fb HTTP/1.1" 502 150 2025-12-17 22:23:43,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c0575bc060ce15e7fb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:44,530 - lava_callback - INFO - Received callback for node 69408a5f575bc060ce15ee39 INFO: 172.21.0.1:49838 - "POST /node/69408a5f575bc060ce15ee39 HTTP/1.0" 202 Accepted 2025-12-17 22:23:45,176 - lava_callback - INFO - Received callback for node 69408a61575bc060ce15ee41 INFO: 172.21.0.1:49840 - "POST /node/69408a61575bc060ce15ee41 HTTP/1.0" 202 Accepted 2025-12-17 22:23:45,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b4575bc060ce15e7d3 2025-12-17 22:23:45,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b4575bc060ce15e7d3 HTTP/1.1" 502 150 2025-12-17 22:23:45,782 - lava_callback - ERROR - Error processing callback for node 694088b4575bc060ce15e7d3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088b4575bc060ce15e7d3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088b4575bc060ce15e7d3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088b4575bc060ce15e7d3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:45,784 - lava_callback - INFO - Starting async job submit for node 694088cb575bc060ce15e815 2025-12-17 22:23:45,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:45,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088cb575bc060ce15e815 HTTP/1.1" 502 150 2025-12-17 22:23:45,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088cb575bc060ce15e815'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:45,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088cb575bc060ce15e815 2025-12-17 22:23:45,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088cb575bc060ce15e815 HTTP/1.1" 502 150 2025-12-17 22:23:45,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088cb575bc060ce15e815'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:45,830 - lava_callback - INFO - Received callback for node 69408a63575bc060ce15ee44 INFO: 172.21.0.1:49852 - "POST /node/69408a63575bc060ce15ee44 HTTP/1.0" 202 Accepted 2025-12-17 22:23:45,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c5575bc060ce15e808 2025-12-17 22:23:45,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c5575bc060ce15e808 HTTP/1.1" 502 150 2025-12-17 22:23:45,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c5575bc060ce15e808'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:45,932 - lava_callback - INFO - Received callback for node 6940e125460bc8e26511373c INFO: 172.21.0.1:49854 - "POST /node/6940e125460bc8e26511373c HTTP/1.0" 202 Accepted 2025-12-17 22:23:45,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b6575bc060ce15e7d6 2025-12-17 22:23:45,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b6575bc060ce15e7d6 HTTP/1.1" 502 150 2025-12-17 22:23:45,943 - lava_callback - ERROR - Error processing callback for node 694088b6575bc060ce15e7d6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088b6575bc060ce15e7d6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088b6575bc060ce15e7d6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088b6575bc060ce15e7d6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:45,945 - lava_callback - INFO - Starting async job submit for node 694088ce575bc060ce15e825 2025-12-17 22:23:45,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:45,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ce575bc060ce15e825 HTTP/1.1" 502 150 2025-12-17 22:23:45,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ce575bc060ce15e825'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:45,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ce575bc060ce15e825 2025-12-17 22:23:45,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ce575bc060ce15e825 HTTP/1.1" 502 150 2025-12-17 22:23:45,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ce575bc060ce15e825'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49858 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:23:46,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088b8575bc060ce15e7df 2025-12-17 22:23:46,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088b8575bc060ce15e7df HTTP/1.1" 502 150 2025-12-17 22:23:46,773 - lava_callback - ERROR - Error processing callback for node 694088b8575bc060ce15e7df: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088b8575bc060ce15e7df (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088b8575bc060ce15e7df (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088b8575bc060ce15e7df (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:46,775 - lava_callback - INFO - Starting async job submit for node 694088d0575bc060ce15e828 2025-12-17 22:23:46,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:46,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d0575bc060ce15e828 HTTP/1.1" 502 150 2025-12-17 22:23:46,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d0575bc060ce15e828'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:46,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d0575bc060ce15e828 2025-12-17 22:23:46,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d0575bc060ce15e828 HTTP/1.1" 502 150 2025-12-17 22:23:46,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d0575bc060ce15e828'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:47,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c7575bc060ce15e80b 2025-12-17 22:23:47,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c7575bc060ce15e80b HTTP/1.1" 502 150 2025-12-17 22:23:47,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c7575bc060ce15e80b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:47,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c2575bc060ce15e7fe 2025-12-17 22:23:47,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c2575bc060ce15e7fe HTTP/1.1" 502 150 2025-12-17 22:23:47,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c2575bc060ce15e7fe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:47,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088cc575bc060ce15e81a 2025-12-17 22:23:47,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088cc575bc060ce15e81a HTTP/1.1" 502 150 2025-12-17 22:23:47,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088cc575bc060ce15e81a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:47,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088cb575bc060ce15e815 2025-12-17 22:23:47,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088cb575bc060ce15e815 HTTP/1.1" 502 150 2025-12-17 22:23:47,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088cb575bc060ce15e815'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:47,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ba575bc060ce15e7e8 2025-12-17 22:23:47,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ba575bc060ce15e7e8 HTTP/1.1" 502 150 2025-12-17 22:23:47,846 - lava_callback - ERROR - Error processing callback for node 694088ba575bc060ce15e7e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ba575bc060ce15e7e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ba575bc060ce15e7e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ba575bc060ce15e7e8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:47,848 - lava_callback - INFO - Starting async job submit for node 694088d1575bc060ce15e82b 2025-12-17 22:23:47,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:47,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d1575bc060ce15e82b HTTP/1.1" 502 150 2025-12-17 22:23:47,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d1575bc060ce15e82b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:47,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d1575bc060ce15e82b 2025-12-17 22:23:47,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d1575bc060ce15e82b HTTP/1.1" 502 150 2025-12-17 22:23:47,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d1575bc060ce15e82b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:47,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c3575bc060ce15e805 2025-12-17 22:23:47,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c3575bc060ce15e805 HTTP/1.1" 502 150 2025-12-17 22:23:47,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c3575bc060ce15e805'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:47,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ce575bc060ce15e825 2025-12-17 22:23:47,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ce575bc060ce15e825 HTTP/1.1" 502 150 2025-12-17 22:23:47,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ce575bc060ce15e825'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:48,042 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088bb575bc060ce15e7eb 2025-12-17 22:23:48,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bb575bc060ce15e7eb HTTP/1.1" 502 150 2025-12-17 22:23:48,044 - lava_callback - ERROR - Error processing callback for node 694088bb575bc060ce15e7eb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088bb575bc060ce15e7eb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088bb575bc060ce15e7eb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088bb575bc060ce15e7eb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:48,046 - lava_callback - INFO - Starting async job submit for node 694088d2575bc060ce15e82e 2025-12-17 22:23:48,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:48,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d2575bc060ce15e82e HTTP/1.1" 502 150 2025-12-17 22:23:48,054 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d2575bc060ce15e82e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:48,054 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d2575bc060ce15e82e 2025-12-17 22:23:48,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d2575bc060ce15e82e HTTP/1.1" 502 150 2025-12-17 22:23:48,055 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d2575bc060ce15e82e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:48,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088bc575bc060ce15e7ee 2025-12-17 22:23:48,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bc575bc060ce15e7ee HTTP/1.1" 502 150 2025-12-17 22:23:48,453 - lava_callback - ERROR - Error processing callback for node 694088bc575bc060ce15e7ee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088bc575bc060ce15e7ee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088bc575bc060ce15e7ee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088bc575bc060ce15e7ee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:48,454 - lava_callback - INFO - Starting async job submit for node 694088d3575bc060ce15e831 2025-12-17 22:23:48,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:48,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d3575bc060ce15e831 HTTP/1.1" 502 150 2025-12-17 22:23:48,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d3575bc060ce15e831'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:48,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d3575bc060ce15e831 2025-12-17 22:23:48,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d3575bc060ce15e831 HTTP/1.1" 502 150 2025-12-17 22:23:48,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d3575bc060ce15e831'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:48,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d0575bc060ce15e828 2025-12-17 22:23:48,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d0575bc060ce15e828 HTTP/1.1" 502 150 2025-12-17 22:23:48,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d0575bc060ce15e828'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:49,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088aa575bc060ce15e7c5 2025-12-17 22:23:49,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088aa575bc060ce15e7c5 HTTP/1.1" 502 150 2025-12-17 22:23:49,363 - lava_callback - ERROR - Error processing callback for node 694088aa575bc060ce15e7c5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088aa575bc060ce15e7c5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088aa575bc060ce15e7c5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088aa575bc060ce15e7c5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:49,365 - lava_callback - INFO - Starting async job submit for node 694088db575bc060ce15e843 2025-12-17 22:23:49,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:49,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088db575bc060ce15e843 HTTP/1.1" 502 150 2025-12-17 22:23:49,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088db575bc060ce15e843'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:49,376 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088db575bc060ce15e843 2025-12-17 22:23:49,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088db575bc060ce15e843 HTTP/1.1" 502 150 2025-12-17 22:23:49,377 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088db575bc060ce15e843'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:49,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d1575bc060ce15e82b 2025-12-17 22:23:49,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d1575bc060ce15e82b HTTP/1.1" 502 150 2025-12-17 22:23:49,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d1575bc060ce15e82b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:50,055 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d2575bc060ce15e82e 2025-12-17 22:23:50,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d2575bc060ce15e82e HTTP/1.1" 502 150 2025-12-17 22:23:50,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d2575bc060ce15e82e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:50,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d3575bc060ce15e831 2025-12-17 22:23:50,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d3575bc060ce15e831 HTTP/1.1" 502 150 2025-12-17 22:23:50,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d3575bc060ce15e831'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:51,377 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088db575bc060ce15e843 2025-12-17 22:23:51,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088cb575bc060ce15e815 2025-12-17 22:23:51,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088cb575bc060ce15e815 HTTP/1.1" 502 150 2025-12-17 22:23:51,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088cb575bc060ce15e815'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:51,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088be575bc060ce15e7f5 2025-12-17 22:23:51,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088be575bc060ce15e7f5 HTTP/1.1" 502 150 2025-12-17 22:23:51,859 - lava_callback - ERROR - Error processing callback for node 694088be575bc060ce15e7f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088be575bc060ce15e7f5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088be575bc060ce15e7f5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088be575bc060ce15e7f5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:51,861 - lava_callback - INFO - Starting async job submit for node 694088d4575bc060ce15e834 2025-12-17 22:23:51,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:51,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d4575bc060ce15e834 HTTP/1.1" 502 150 2025-12-17 22:23:51,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d4575bc060ce15e834'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:51,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d4575bc060ce15e834 2025-12-17 22:23:51,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d4575bc060ce15e834 HTTP/1.1" 502 150 2025-12-17 22:23:51,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d4575bc060ce15e834'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:51,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ce575bc060ce15e825 2025-12-17 22:23:51,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ce575bc060ce15e825 HTTP/1.1" 502 150 2025-12-17 22:23:51,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ce575bc060ce15e825'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:52,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d0575bc060ce15e828 2025-12-17 22:23:52,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d0575bc060ce15e828 HTTP/1.1" 502 150 2025-12-17 22:23:52,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d0575bc060ce15e828'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:53,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ac575bc060ce15e7c9 HTTP/1.1" 502 150 2025-12-17 22:23:53,008 - lava_callback - ERROR - Error processing callback for node 694088ac575bc060ce15e7c9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ac575bc060ce15e7c9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ac575bc060ce15e7c9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ac575bc060ce15e7c9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:53,011 - lava_callback - INFO - Starting async job submit for node 694088dd575bc060ce15e850 2025-12-17 22:23:53,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:53,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088dd575bc060ce15e850 HTTP/1.1" 502 150 2025-12-17 22:23:53,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088dd575bc060ce15e850'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:53,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088dd575bc060ce15e850 2025-12-17 22:23:53,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088dd575bc060ce15e850 HTTP/1.1" 502 150 2025-12-17 22:23:53,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088dd575bc060ce15e850'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:53,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c5575bc060ce15e808 2025-12-17 22:23:53,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c5575bc060ce15e808 HTTP/1.1" 502 150 2025-12-17 22:23:53,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c5575bc060ce15e808'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:53,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d1575bc060ce15e82b 2025-12-17 22:23:53,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d1575bc060ce15e82b HTTP/1.1" 502 150 2025-12-17 22:23:53,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d1575bc060ce15e82b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:53,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d4575bc060ce15e834 2025-12-17 22:23:53,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d4575bc060ce15e834 HTTP/1.1" 502 150 2025-12-17 22:23:53,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d4575bc060ce15e834'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:54,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d2575bc060ce15e82e 2025-12-17 22:23:54,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d2575bc060ce15e82e HTTP/1.1" 502 150 2025-12-17 22:23:54,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d2575bc060ce15e82e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:54,451 - lava_callback - INFO - Received callback for node 69408a67575bc060ce15ee4e INFO: 172.21.0.1:43672 - "POST /node/69408a67575bc060ce15ee4e HTTP/1.0" 202 Accepted 2025-12-17 22:23:54,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d3575bc060ce15e831 2025-12-17 22:23:54,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d3575bc060ce15e831 HTTP/1.1" 502 150 2025-12-17 22:23:54,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d3575bc060ce15e831'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:55,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088dd575bc060ce15e850 2025-12-17 22:23:55,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088dd575bc060ce15e850 HTTP/1.1" 502 150 2025-12-17 22:23:55,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088dd575bc060ce15e850'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:55,112 - lava_callback - INFO - Received callback for node 69408a6a575bc060ce15ee51 INFO: 172.21.0.1:43684 - "POST /node/69408a6a575bc060ce15ee51 HTTP/1.0" 202 Accepted 2025-12-17 22:23:55,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c7575bc060ce15e80b 2025-12-17 22:23:55,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c7575bc060ce15e80b HTTP/1.1" 502 150 2025-12-17 22:23:55,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088c7575bc060ce15e80b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:55,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088cc575bc060ce15e81a 2025-12-17 22:23:55,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088cc575bc060ce15e81a HTTP/1.1" 502 150 2025-12-17 22:23:55,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088cc575bc060ce15e81a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:55,838 - lava_callback - INFO - Received callback for node 69408a6c575bc060ce15ee54 INFO: 172.21.0.1:43688 - "POST /node/69408a6c575bc060ce15ee54 HTTP/1.0" 202 Accepted 2025-12-17 22:23:56,492 - lava_callback - INFO - Received callback for node 69408a6e575bc060ce15ee57 INFO: 172.21.0.1:43692 - "POST /node/69408a6e575bc060ce15ee57 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:43696 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:23:56,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088bf575bc060ce15e7f8 2025-12-17 22:23:56,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088bf575bc060ce15e7f8 HTTP/1.1" 502 150 2025-12-17 22:23:56,821 - lava_callback - ERROR - Error processing callback for node 694088bf575bc060ce15e7f8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088bf575bc060ce15e7f8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088bf575bc060ce15e7f8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088bf575bc060ce15e7f8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:56,823 - lava_callback - INFO - Starting async job submit for node 694088d9575bc060ce15e83d 2025-12-17 22:23:56,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:56,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d9575bc060ce15e83d HTTP/1.1" 502 150 2025-12-17 22:23:56,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d9575bc060ce15e83d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:56,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d9575bc060ce15e83d 2025-12-17 22:23:56,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d9575bc060ce15e83d HTTP/1.1" 502 150 2025-12-17 22:23:56,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d9575bc060ce15e83d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:57,138 - lava_callback - INFO - Received callback for node 69408a70575bc060ce15ee60 INFO: 172.21.0.1:43712 - "POST /node/69408a70575bc060ce15ee60 HTTP/1.0" 202 Accepted 2025-12-17 22:23:57,791 - lava_callback - INFO - Received callback for node 69408a72575bc060ce15ee69 INFO: 172.21.0.1:43716 - "POST /node/69408a72575bc060ce15ee69 HTTP/1.0" 202 Accepted 2025-12-17 22:23:57,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d4575bc060ce15e834 2025-12-17 22:23:57,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d4575bc060ce15e834 HTTP/1.1" 502 150 2025-12-17 22:23:57,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d4575bc060ce15e834'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:58,464 - lava_callback - INFO - Received callback for node 69408a74575bc060ce15ee6c INFO: 172.21.0.1:43726 - "POST /node/69408a74575bc060ce15ee6c HTTP/1.0" 202 Accepted 2025-12-17 22:23:58,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d9575bc060ce15e83d 2025-12-17 22:23:58,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d9575bc060ce15e83d HTTP/1.1" 502 150 2025-12-17 22:23:58,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d9575bc060ce15e83d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:59,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088dd575bc060ce15e850 2025-12-17 22:23:59,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088dd575bc060ce15e850 HTTP/1.1" 502 150 2025-12-17 22:23:59,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088dd575bc060ce15e850'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:59,130 - lava_callback - INFO - Received callback for node 69408a77575bc060ce15ee6f INFO: 172.21.0.1:43736 - "POST /node/69408a77575bc060ce15ee6f HTTP/1.0" 202 Accepted 2025-12-17 22:23:59,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088cb575bc060ce15e815 2025-12-17 22:23:59,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088cb575bc060ce15e815 HTTP/1.1" 502 150 2025-12-17 22:23:59,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088cb575bc060ce15e815'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:59,805 - lava_callback - INFO - Received callback for node 69408a79575bc060ce15ee74 INFO: 172.21.0.1:47268 - "POST /node/69408a79575bc060ce15ee74 HTTP/1.0" 202 Accepted 2025-12-17 22:23:59,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c0575bc060ce15e7fb 2025-12-17 22:23:59,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c0575bc060ce15e7fb HTTP/1.1" 502 150 2025-12-17 22:23:59,926 - lava_callback - ERROR - Error processing callback for node 694088c0575bc060ce15e7fb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088c0575bc060ce15e7fb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088c0575bc060ce15e7fb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088c0575bc060ce15e7fb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:23:59,929 - lava_callback - INFO - Starting async job submit for node 694088da575bc060ce15e840 2025-12-17 22:23:59,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:23:59,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088da575bc060ce15e840 HTTP/1.1" 502 150 2025-12-17 22:23:59,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088da575bc060ce15e840'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:59,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088da575bc060ce15e840 2025-12-17 22:23:59,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088da575bc060ce15e840 HTTP/1.1" 502 150 2025-12-17 22:23:59,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088da575bc060ce15e840'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:23:59,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ce575bc060ce15e825 2025-12-17 22:23:59,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ce575bc060ce15e825 HTTP/1.1" 502 150 2025-12-17 22:23:59,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ce575bc060ce15e825'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:00,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d0575bc060ce15e828 2025-12-17 22:24:00,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d0575bc060ce15e828 HTTP/1.1" 502 150 2025-12-17 22:24:00,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d0575bc060ce15e828'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:01,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d1575bc060ce15e82b 2025-12-17 22:24:01,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d1575bc060ce15e82b HTTP/1.1" 502 150 2025-12-17 22:24:01,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d1575bc060ce15e82b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:01,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088da575bc060ce15e840 2025-12-17 22:24:01,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088da575bc060ce15e840 HTTP/1.1" 502 150 2025-12-17 22:24:01,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088da575bc060ce15e840'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:02,060 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d2575bc060ce15e82e 2025-12-17 22:24:02,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d2575bc060ce15e82e HTTP/1.1" 502 150 2025-12-17 22:24:02,061 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d2575bc060ce15e82e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47270 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:24:02,374 - lava_callback - INFO - Received callback for node 69408be6575bc060ce15f398 INFO: 172.21.0.1:47272 - "POST /node/69408be6575bc060ce15f398 HTTP/1.0" 202 Accepted 2025-12-17 22:24:02,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d3575bc060ce15e831 2025-12-17 22:24:02,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d3575bc060ce15e831 HTTP/1.1" 502 150 2025-12-17 22:24:02,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d3575bc060ce15e831'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:02,768 - lava_callback - INFO - Received callback for node 69408a7d575bc060ce15ee7a INFO: 172.21.0.1:47288 - "POST /node/69408a7d575bc060ce15ee7a HTTP/1.0" 202 Accepted 2025-12-17 22:24:02,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d9575bc060ce15e83d 2025-12-17 22:24:02,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d9575bc060ce15e83d HTTP/1.1" 502 150 2025-12-17 22:24:02,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d9575bc060ce15e83d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:03,566 - lava_callback - INFO - Received callback for node 69408a7e575bc060ce15ee7d INFO: 172.21.0.1:47302 - "POST /node/69408a7e575bc060ce15ee7d HTTP/1.0" 202 Accepted 2025-12-17 22:24:03,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c2575bc060ce15e7fe 2025-12-17 22:24:03,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c2575bc060ce15e7fe HTTP/1.1" 502 150 2025-12-17 22:24:03,603 - lava_callback - ERROR - Error processing callback for node 694088c2575bc060ce15e7fe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088c2575bc060ce15e7fe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088c2575bc060ce15e7fe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088c2575bc060ce15e7fe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:03,604 - lava_callback - INFO - Starting async job submit for node 694088d5575bc060ce15e837 2025-12-17 22:24:03,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:03,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d5575bc060ce15e837 HTTP/1.1" 502 150 2025-12-17 22:24:03,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d5575bc060ce15e837'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:03,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d5575bc060ce15e837 2025-12-17 22:24:03,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d5575bc060ce15e837 HTTP/1.1" 502 150 2025-12-17 22:24:03,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d5575bc060ce15e837'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:03,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c3575bc060ce15e805 2025-12-17 22:24:03,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c3575bc060ce15e805 HTTP/1.1" 502 150 2025-12-17 22:24:03,863 - lava_callback - ERROR - Error processing callback for node 694088c3575bc060ce15e805: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088c3575bc060ce15e805 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088c3575bc060ce15e805 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088c3575bc060ce15e805 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:03,866 - lava_callback - INFO - Starting async job submit for node 694086c8575bc060ce15e186 2025-12-17 22:24:03,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:03,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c8575bc060ce15e186 HTTP/1.1" 502 150 2025-12-17 22:24:03,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c8575bc060ce15e186'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:03,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c8575bc060ce15e186 2025-12-17 22:24:03,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c8575bc060ce15e186 HTTP/1.1" 502 150 2025-12-17 22:24:03,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c8575bc060ce15e186'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:04,338 - lava_callback - INFO - Received callback for node 69408a80575bc060ce15ee80 INFO: 172.21.0.1:47316 - "POST /node/69408a80575bc060ce15ee80 HTTP/1.0" 202 Accepted 2025-12-17 22:24:04,549 - lava_callback - INFO - Received callback for node 69408a80575bc060ce15ee83 INFO: 172.21.0.1:47326 - "POST /node/69408a80575bc060ce15ee83 HTTP/1.0" 202 Accepted 2025-12-17 22:24:05,221 - lava_callback - INFO - Received callback for node 69408a83575bc060ce15ee86 INFO: 172.21.0.1:47338 - "POST /node/69408a83575bc060ce15ee86 HTTP/1.0" 202 Accepted 2025-12-17 22:24:05,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d5575bc060ce15e837 2025-12-17 22:24:05,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d5575bc060ce15e837 HTTP/1.1" 502 150 2025-12-17 22:24:05,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d5575bc060ce15e837'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:05,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d4575bc060ce15e834 2025-12-17 22:24:05,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d4575bc060ce15e834 HTTP/1.1" 502 150 2025-12-17 22:24:05,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d4575bc060ce15e834'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:05,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c8575bc060ce15e186 2025-12-17 22:24:05,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c8575bc060ce15e186 HTTP/1.1" 502 150 2025-12-17 22:24:05,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c8575bc060ce15e186'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:05,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088da575bc060ce15e840 2025-12-17 22:24:05,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088da575bc060ce15e840 HTTP/1.1" 502 150 2025-12-17 22:24:05,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088da575bc060ce15e840'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47348 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:24:06,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088db575bc060ce15e843 HTTP/1.1" 502 150 2025-12-17 22:24:06,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088db575bc060ce15e843'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:07,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088dd575bc060ce15e850 2025-12-17 22:24:07,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088dd575bc060ce15e850 HTTP/1.1" 502 150 2025-12-17 22:24:07,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088dd575bc060ce15e850'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:09,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d5575bc060ce15e837 2025-12-17 22:24:09,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d5575bc060ce15e837 HTTP/1.1" 502 150 2025-12-17 22:24:09,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d5575bc060ce15e837'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:09,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c5575bc060ce15e808 2025-12-17 22:24:09,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c5575bc060ce15e808 HTTP/1.1" 502 150 2025-12-17 22:24:09,841 - lava_callback - ERROR - Error processing callback for node 694088c5575bc060ce15e808: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088c5575bc060ce15e808 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088c5575bc060ce15e808 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088c5575bc060ce15e808 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:09,843 - lava_callback - INFO - Starting async job submit for node 694088d7575bc060ce15e83a 2025-12-17 22:24:09,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:09,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d7575bc060ce15e83a HTTP/1.1" 502 150 2025-12-17 22:24:09,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d7575bc060ce15e83a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:09,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d7575bc060ce15e83a 2025-12-17 22:24:09,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d7575bc060ce15e83a HTTP/1.1" 502 150 2025-12-17 22:24:09,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d7575bc060ce15e83a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:09,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c8575bc060ce15e186 2025-12-17 22:24:09,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c8575bc060ce15e186 HTTP/1.1" 502 150 2025-12-17 22:24:09,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c8575bc060ce15e186'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:10,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088db575bc060ce15e843 2025-12-17 22:24:10,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088db575bc060ce15e843 HTTP/1.1" 502 150 2025-12-17 22:24:10,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088db575bc060ce15e843'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:10,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d9575bc060ce15e83d 2025-12-17 22:24:10,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d9575bc060ce15e83d HTTP/1.1" 502 150 2025-12-17 22:24:10,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d9575bc060ce15e83d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:11,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088c7575bc060ce15e80b 2025-12-17 22:24:11,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088c7575bc060ce15e80b HTTP/1.1" 502 150 2025-12-17 22:24:11,475 - lava_callback - ERROR - Error processing callback for node 694088c7575bc060ce15e80b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088c7575bc060ce15e80b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088c7575bc060ce15e80b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088c7575bc060ce15e80b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:11,478 - lava_callback - INFO - Starting async job submit for node 694088e5575bc060ce15e893 2025-12-17 22:24:11,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:11,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e5575bc060ce15e893 HTTP/1.1" 502 150 2025-12-17 22:24:11,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e5575bc060ce15e893'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:11,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e5575bc060ce15e893 2025-12-17 22:24:11,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e5575bc060ce15e893 HTTP/1.1" 502 150 2025-12-17 22:24:11,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e5575bc060ce15e893'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:11,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088cc575bc060ce15e81a 2025-12-17 22:24:11,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088cc575bc060ce15e81a HTTP/1.1" 502 150 2025-12-17 22:24:11,766 - lava_callback - ERROR - Error processing callback for node 694088cc575bc060ce15e81a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088cc575bc060ce15e81a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088cc575bc060ce15e81a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088cc575bc060ce15e81a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:11,768 - lava_callback - INFO - Starting async job submit for node 694088ec575bc060ce15e8a9 2025-12-17 22:24:11,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:11,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d7575bc060ce15e83a 2025-12-17 22:24:11,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d7575bc060ce15e83a HTTP/1.1" 502 150 2025-12-17 22:24:11,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d7575bc060ce15e83a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:11,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ec575bc060ce15e8a9 HTTP/1.1" 502 150 2025-12-17 22:24:11,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ec575bc060ce15e8a9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:11,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ec575bc060ce15e8a9 2025-12-17 22:24:11,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ec575bc060ce15e8a9 HTTP/1.1" 502 150 2025-12-17 22:24:11,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ec575bc060ce15e8a9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:13,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e5575bc060ce15e893 2025-12-17 22:24:13,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e5575bc060ce15e893 HTTP/1.1" 502 150 2025-12-17 22:24:13,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e5575bc060ce15e893'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:13,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088da575bc060ce15e840 2025-12-17 22:24:13,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088da575bc060ce15e840 HTTP/1.1" 502 150 2025-12-17 22:24:13,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088da575bc060ce15e840'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:13,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ec575bc060ce15e8a9 2025-12-17 22:24:13,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ec575bc060ce15e8a9 HTTP/1.1" 502 150 2025-12-17 22:24:13,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ec575bc060ce15e8a9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:14,410 - lava_callback - INFO - Received callback for node 69408be8575bc060ce15f39b INFO: 172.21.0.1:46156 - "POST /node/69408be8575bc060ce15f39b HTTP/1.0" 202 Accepted 2025-12-17 22:24:14,448 - lava_callback - INFO - Received callback for node 69408a7b575bc060ce15ee77 INFO: 172.21.0.1:46166 - "POST /node/69408a7b575bc060ce15ee77 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:46176 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:24:15,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088cb575bc060ce15e815 2025-12-17 22:24:15,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088cb575bc060ce15e815 HTTP/1.1" 502 150 2025-12-17 22:24:15,797 - lava_callback - ERROR - Error processing callback for node 694088cb575bc060ce15e815: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088cb575bc060ce15e815 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088cb575bc060ce15e815 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088cb575bc060ce15e815 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:15,799 - lava_callback - INFO - Starting async job submit for node 694088ef575bc060ce15e8af 2025-12-17 22:24:15,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:15,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ef575bc060ce15e8af HTTP/1.1" 502 150 2025-12-17 22:24:15,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ef575bc060ce15e8af'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:15,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ef575bc060ce15e8af 2025-12-17 22:24:15,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ef575bc060ce15e8af HTTP/1.1" 502 150 2025-12-17 22:24:15,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ef575bc060ce15e8af'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:15,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d7575bc060ce15e83a 2025-12-17 22:24:15,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d7575bc060ce15e83a HTTP/1.1" 502 150 2025-12-17 22:24:15,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d7575bc060ce15e83a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:15,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ce575bc060ce15e825 2025-12-17 22:24:15,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ce575bc060ce15e825 HTTP/1.1" 502 150 2025-12-17 22:24:15,960 - lava_callback - ERROR - Error processing callback for node 694088ce575bc060ce15e825: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ce575bc060ce15e825 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ce575bc060ce15e825 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ce575bc060ce15e825 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:15,962 - lava_callback - INFO - Starting async job submit for node 694088e2575bc060ce15e88b 2025-12-17 22:24:15,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:15,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e2575bc060ce15e88b HTTP/1.1" 502 150 2025-12-17 22:24:15,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e2575bc060ce15e88b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:15,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e2575bc060ce15e88b 2025-12-17 22:24:15,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e2575bc060ce15e88b HTTP/1.1" 502 150 2025-12-17 22:24:15,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e2575bc060ce15e88b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46180 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:24:16,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d0575bc060ce15e828 2025-12-17 22:24:16,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d0575bc060ce15e828 HTTP/1.1" 502 150 2025-12-17 22:24:16,788 - lava_callback - ERROR - Error processing callback for node 694088d0575bc060ce15e828: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d0575bc060ce15e828 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d0575bc060ce15e828 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d0575bc060ce15e828 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:16,790 - lava_callback - INFO - Starting async job submit for node 694088df575bc060ce15e87d 2025-12-17 22:24:16,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:16,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088df575bc060ce15e87d HTTP/1.1" 502 150 2025-12-17 22:24:16,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088df575bc060ce15e87d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:16,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088df575bc060ce15e87d 2025-12-17 22:24:16,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088df575bc060ce15e87d HTTP/1.1" 502 150 2025-12-17 22:24:16,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088df575bc060ce15e87d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:17,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e5575bc060ce15e893 2025-12-17 22:24:17,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d5575bc060ce15e837 2025-12-17 22:24:17,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d5575bc060ce15e837 HTTP/1.1" 502 150 2025-12-17 22:24:17,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d5575bc060ce15e837'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:17,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ef575bc060ce15e8af 2025-12-17 22:24:17,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ef575bc060ce15e8af HTTP/1.1" 502 150 2025-12-17 22:24:17,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ef575bc060ce15e8af'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:17,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d1575bc060ce15e82b 2025-12-17 22:24:17,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d1575bc060ce15e82b HTTP/1.1" 502 150 2025-12-17 22:24:17,861 - lava_callback - ERROR - Error processing callback for node 694088d1575bc060ce15e82b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d1575bc060ce15e82b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d1575bc060ce15e82b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d1575bc060ce15e82b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:17,863 - lava_callback - INFO - Starting async job submit for node 694088e1575bc060ce15e884 2025-12-17 22:24:17,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:17,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e1575bc060ce15e884 HTTP/1.1" 502 150 2025-12-17 22:24:17,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e1575bc060ce15e884'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:17,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e1575bc060ce15e884 2025-12-17 22:24:17,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e1575bc060ce15e884 HTTP/1.1" 502 150 2025-12-17 22:24:17,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e1575bc060ce15e884'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:17,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c8575bc060ce15e186 2025-12-17 22:24:17,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c8575bc060ce15e186 HTTP/1.1" 502 150 2025-12-17 22:24:17,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086c8575bc060ce15e186'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:17,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ec575bc060ce15e8a9 2025-12-17 22:24:17,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ec575bc060ce15e8a9 HTTP/1.1" 502 150 2025-12-17 22:24:17,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ec575bc060ce15e8a9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:17,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e2575bc060ce15e88b 2025-12-17 22:24:17,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e2575bc060ce15e88b HTTP/1.1" 502 150 2025-12-17 22:24:17,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e2575bc060ce15e88b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:18,061 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d2575bc060ce15e82e 2025-12-17 22:24:18,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d3575bc060ce15e831 2025-12-17 22:24:18,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d3575bc060ce15e831 HTTP/1.1" 502 150 2025-12-17 22:24:18,658 - lava_callback - ERROR - Error processing callback for node 694088d3575bc060ce15e831: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d3575bc060ce15e831 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d3575bc060ce15e831 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d3575bc060ce15e831 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:18,660 - lava_callback - INFO - Starting async job submit for node 694088e3575bc060ce15e890 2025-12-17 22:24:18,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:18,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e3575bc060ce15e890 HTTP/1.1" 502 150 2025-12-17 22:24:18,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e3575bc060ce15e890'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:18,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e3575bc060ce15e890 2025-12-17 22:24:18,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e3575bc060ce15e890 HTTP/1.1" 502 150 2025-12-17 22:24:18,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e3575bc060ce15e890'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:18,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088df575bc060ce15e87d 2025-12-17 22:24:18,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088df575bc060ce15e87d HTTP/1.1" 502 150 2025-12-17 22:24:18,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088df575bc060ce15e87d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:18,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088db575bc060ce15e843 2025-12-17 22:24:18,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088db575bc060ce15e843 HTTP/1.1" 502 150 2025-12-17 22:24:18,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088db575bc060ce15e843'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:19,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e1575bc060ce15e884 2025-12-17 22:24:19,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e1575bc060ce15e884 HTTP/1.1" 502 150 2025-12-17 22:24:19,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e1575bc060ce15e884'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:20,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e3575bc060ce15e890 2025-12-17 22:24:20,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e3575bc060ce15e890 HTTP/1.1" 502 150 2025-12-17 22:24:20,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e3575bc060ce15e890'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:21,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ef575bc060ce15e8af 2025-12-17 22:24:21,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ef575bc060ce15e8af HTTP/1.1" 502 150 2025-12-17 22:24:21,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ef575bc060ce15e8af'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:21,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d4575bc060ce15e834 2025-12-17 22:24:21,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d4575bc060ce15e834 HTTP/1.1" 502 150 2025-12-17 22:24:21,874 - lava_callback - ERROR - Error processing callback for node 694088d4575bc060ce15e834: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d4575bc060ce15e834 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d4575bc060ce15e834 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d4575bc060ce15e834 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:21,876 - lava_callback - INFO - Starting async job submit for node 694088e6575bc060ce15e896 2025-12-17 22:24:21,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:21,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e6575bc060ce15e896 HTTP/1.1" 502 150 2025-12-17 22:24:21,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e6575bc060ce15e896'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:21,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e6575bc060ce15e896 2025-12-17 22:24:21,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e6575bc060ce15e896 HTTP/1.1" 502 150 2025-12-17 22:24:21,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e6575bc060ce15e896'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:21,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e2575bc060ce15e88b 2025-12-17 22:24:21,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e2575bc060ce15e88b HTTP/1.1" 502 150 2025-12-17 22:24:21,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e2575bc060ce15e88b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:22,267 - lava_callback - INFO - Received callback for node 69408a41575bc060ce15edb9 INFO: 172.21.0.1:49816 - "POST /node/69408a41575bc060ce15edb9 HTTP/1.0" 202 Accepted 2025-12-17 22:24:22,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088df575bc060ce15e87d 2025-12-17 22:24:22,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088df575bc060ce15e87d HTTP/1.1" 502 150 2025-12-17 22:24:22,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088df575bc060ce15e87d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:23,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088dd575bc060ce15e850 2025-12-17 22:24:23,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088dd575bc060ce15e850 HTTP/1.1" 502 150 2025-12-17 22:24:23,025 - lava_callback - ERROR - Error processing callback for node 694088dd575bc060ce15e850: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088dd575bc060ce15e850 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088dd575bc060ce15e850 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088dd575bc060ce15e850 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:23,027 - lava_callback - INFO - Starting async job submit for node 694088e7575bc060ce15e899 2025-12-17 22:24:23,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:23,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e7575bc060ce15e899 HTTP/1.1" 502 150 2025-12-17 22:24:23,047 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e7575bc060ce15e899'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:23,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e7575bc060ce15e899 2025-12-17 22:24:23,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e7575bc060ce15e899 HTTP/1.1" 502 150 2025-12-17 22:24:23,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e7575bc060ce15e899'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:23,170 - lava_callback - INFO - Received callback for node 69408a85575bc060ce15ee98 INFO: 172.21.0.1:49824 - "POST /node/69408a85575bc060ce15ee98 HTTP/1.0" 202 Accepted 2025-12-17 22:24:23,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d7575bc060ce15e83a 2025-12-17 22:24:23,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d7575bc060ce15e83a HTTP/1.1" 502 150 2025-12-17 22:24:23,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088d7575bc060ce15e83a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:23,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e1575bc060ce15e884 2025-12-17 22:24:23,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e1575bc060ce15e884 HTTP/1.1" 502 150 2025-12-17 22:24:23,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e1575bc060ce15e884'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:23,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e6575bc060ce15e896 2025-12-17 22:24:23,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e6575bc060ce15e896 HTTP/1.1" 502 150 2025-12-17 22:24:23,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e6575bc060ce15e896'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:24,046 - lava_callback - INFO - Received callback for node 69408a8e575bc060ce15eead INFO: 172.21.0.1:49828 - "POST /node/69408a8e575bc060ce15eead HTTP/1.0" 202 Accepted 2025-12-17 22:24:24,542 - lava_callback - INFO - Received callback for node 69408a84575bc060ce15ee91 INFO: 172.21.0.1:49838 - "POST /node/69408a84575bc060ce15ee91 HTTP/1.0" 202 Accepted 2025-12-17 22:24:24,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e3575bc060ce15e890 2025-12-17 22:24:24,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e3575bc060ce15e890 HTTP/1.1" 502 150 2025-12-17 22:24:24,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e3575bc060ce15e890'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:24,875 - lava_callback - INFO - Received callback for node 69408a8f575bc060ce15eeb0 INFO: 172.21.0.1:49844 - "POST /node/69408a8f575bc060ce15eeb0 HTTP/1.0" 202 Accepted 2025-12-17 22:24:25,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e7575bc060ce15e899 2025-12-17 22:24:25,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e7575bc060ce15e899 HTTP/1.1" 502 150 2025-12-17 22:24:25,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e7575bc060ce15e899'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:25,183 - lava_callback - INFO - Received callback for node 69408a87575bc060ce15ee9f INFO: 172.21.0.1:49856 - "POST /node/69408a87575bc060ce15ee9f HTTP/1.0" 202 Accepted 2025-12-17 22:24:25,839 - lava_callback - INFO - Received callback for node 69408a88575bc060ce15eea2 INFO: 172.21.0.1:49860 - "POST /node/69408a88575bc060ce15eea2 HTTP/1.0" 202 Accepted 2025-12-17 22:24:25,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ec575bc060ce15e8a9 2025-12-17 22:24:25,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ec575bc060ce15e8a9 HTTP/1.1" 502 150 2025-12-17 22:24:25,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ec575bc060ce15e8a9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:26,500 - lava_callback - INFO - Received callback for node 69408a96575bc060ce15eee7 INFO: 172.21.0.1:49862 - "POST /node/69408a96575bc060ce15eee7 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:49864 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:24:26,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d9575bc060ce15e83d 2025-12-17 22:24:26,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d9575bc060ce15e83d HTTP/1.1" 502 150 2025-12-17 22:24:26,838 - lava_callback - ERROR - Error processing callback for node 694088d9575bc060ce15e83d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d9575bc060ce15e83d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d9575bc060ce15e83d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d9575bc060ce15e83d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:26,840 - lava_callback - INFO - Starting async job submit for node 694088e8575bc060ce15e89c 2025-12-17 22:24:26,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:26,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e8575bc060ce15e89c HTTP/1.1" 502 150 2025-12-17 22:24:26,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e8575bc060ce15e89c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:26,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e8575bc060ce15e89c 2025-12-17 22:24:26,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e8575bc060ce15e89c HTTP/1.1" 502 150 2025-12-17 22:24:26,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e8575bc060ce15e89c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:27,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e6575bc060ce15e896 2025-12-17 22:24:27,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e6575bc060ce15e896 HTTP/1.1" 502 150 2025-12-17 22:24:27,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e6575bc060ce15e896'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:28,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e8575bc060ce15e89c 2025-12-17 22:24:28,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e8575bc060ce15e89c HTTP/1.1" 502 150 2025-12-17 22:24:28,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e8575bc060ce15e89c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:29,049 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e7575bc060ce15e899 2025-12-17 22:24:29,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e7575bc060ce15e899 HTTP/1.1" 502 150 2025-12-17 22:24:29,051 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e7575bc060ce15e899'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:29,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ef575bc060ce15e8af 2025-12-17 22:24:29,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ef575bc060ce15e8af HTTP/1.1" 502 150 2025-12-17 22:24:29,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ef575bc060ce15e8af'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:29,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088da575bc060ce15e840 2025-12-17 22:24:29,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088da575bc060ce15e840 HTTP/1.1" 502 150 2025-12-17 22:24:29,949 - lava_callback - ERROR - Error processing callback for node 694088da575bc060ce15e840: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088da575bc060ce15e840 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088da575bc060ce15e840 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088da575bc060ce15e840 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:29,951 - lava_callback - INFO - Starting async job submit for node 694088ea575bc060ce15e8a3 2025-12-17 22:24:29,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:29,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ea575bc060ce15e8a3 HTTP/1.1" 502 150 2025-12-17 22:24:29,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ea575bc060ce15e8a3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:29,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ea575bc060ce15e8a3 2025-12-17 22:24:29,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ea575bc060ce15e8a3 HTTP/1.1" 502 150 2025-12-17 22:24:29,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ea575bc060ce15e8a3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:29,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e2575bc060ce15e88b 2025-12-17 22:24:29,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e2575bc060ce15e88b HTTP/1.1" 502 150 2025-12-17 22:24:29,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e2575bc060ce15e88b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:30,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088df575bc060ce15e87d 2025-12-17 22:24:31,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e1575bc060ce15e884 2025-12-17 22:24:31,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e1575bc060ce15e884 HTTP/1.1" 502 150 2025-12-17 22:24:31,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e1575bc060ce15e884'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:31,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ea575bc060ce15e8a3 2025-12-17 22:24:31,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ea575bc060ce15e8a3 HTTP/1.1" 502 150 2025-12-17 22:24:31,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ea575bc060ce15e8a3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:32,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e3575bc060ce15e890 2025-12-17 22:24:32,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e3575bc060ce15e890 HTTP/1.1" 502 150 2025-12-17 22:24:32,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e3575bc060ce15e890'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:32,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e8575bc060ce15e89c 2025-12-17 22:24:32,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e8575bc060ce15e89c HTTP/1.1" 502 150 2025-12-17 22:24:32,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e8575bc060ce15e89c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:32,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e5575bc060ce15e893 HTTP/1.1" 502 150 2025-12-17 22:24:32,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e5575bc060ce15e893'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:33,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d2575bc060ce15e82e HTTP/1.1" 502 150 2025-12-17 22:24:33,456 - lava_callback - ERROR - Error processing callback for node 694088d2575bc060ce15e82e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d2575bc060ce15e82e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d2575bc060ce15e82e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d2575bc060ce15e82e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:33,459 - lava_callback - INFO - Starting async job submit for node 694088eb575bc060ce15e8a6 2025-12-17 22:24:33,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:33,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088eb575bc060ce15e8a6 HTTP/1.1" 502 150 2025-12-17 22:24:33,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088eb575bc060ce15e8a6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:33,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088eb575bc060ce15e8a6 2025-12-17 22:24:33,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088eb575bc060ce15e8a6 HTTP/1.1" 502 150 2025-12-17 22:24:33,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088eb575bc060ce15e8a6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:33,599 - lava_callback - INFO - Received callback for node 69408a8d575bc060ce15eeaa INFO: 172.21.0.1:36858 - "POST /node/69408a8d575bc060ce15eeaa HTTP/1.0" 202 Accepted 2025-12-17 22:24:33,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d5575bc060ce15e837 2025-12-17 22:24:33,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d5575bc060ce15e837 HTTP/1.1" 502 150 2025-12-17 22:24:33,617 - lava_callback - ERROR - Error processing callback for node 694088d5575bc060ce15e837: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d5575bc060ce15e837 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d5575bc060ce15e837 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d5575bc060ce15e837 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:33,620 - lava_callback - INFO - Starting async job submit for node 694088ed575bc060ce15e8ac 2025-12-17 22:24:33,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:33,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ed575bc060ce15e8ac HTTP/1.1" 502 150 2025-12-17 22:24:33,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ed575bc060ce15e8ac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:33,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ed575bc060ce15e8ac 2025-12-17 22:24:33,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ed575bc060ce15e8ac HTTP/1.1" 502 150 2025-12-17 22:24:33,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ed575bc060ce15e8ac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:33,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086c8575bc060ce15e186 2025-12-17 22:24:33,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086c8575bc060ce15e186 HTTP/1.1" 502 150 2025-12-17 22:24:33,944 - lava_callback - ERROR - Error processing callback for node 694086c8575bc060ce15e186: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086c8575bc060ce15e186 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086c8575bc060ce15e186 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086c8575bc060ce15e186 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:33,946 - lava_callback - INFO - Starting async job submit for node 694088f0575bc060ce15e8b2 2025-12-17 22:24:33,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:33,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f0575bc060ce15e8b2 HTTP/1.1" 502 150 2025-12-17 22:24:33,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f0575bc060ce15e8b2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:33,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f0575bc060ce15e8b2 2025-12-17 22:24:33,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f0575bc060ce15e8b2 HTTP/1.1" 502 150 2025-12-17 22:24:33,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f0575bc060ce15e8b2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:34,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088db575bc060ce15e843 2025-12-17 22:24:34,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088db575bc060ce15e843 HTTP/1.1" 502 150 2025-12-17 22:24:34,838 - lava_callback - ERROR - Error processing callback for node 694088db575bc060ce15e843: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088db575bc060ce15e843 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088db575bc060ce15e843 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088db575bc060ce15e843 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:34,840 - lava_callback - INFO - Starting async job submit for node 694088f2575bc060ce15e8ba 2025-12-17 22:24:34,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:34,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f2575bc060ce15e8ba HTTP/1.1" 502 150 2025-12-17 22:24:34,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f2575bc060ce15e8ba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:34,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f2575bc060ce15e8ba 2025-12-17 22:24:34,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f2575bc060ce15e8ba HTTP/1.1" 502 150 2025-12-17 22:24:34,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f2575bc060ce15e8ba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:35,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088eb575bc060ce15e8a6 2025-12-17 22:24:35,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088eb575bc060ce15e8a6 HTTP/1.1" 502 150 2025-12-17 22:24:35,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088eb575bc060ce15e8a6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:35,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ed575bc060ce15e8ac 2025-12-17 22:24:35,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ed575bc060ce15e8ac HTTP/1.1" 502 150 2025-12-17 22:24:35,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ed575bc060ce15e8ac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:35,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e6575bc060ce15e896 2025-12-17 22:24:35,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e6575bc060ce15e896 HTTP/1.1" 502 150 2025-12-17 22:24:35,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e6575bc060ce15e896'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:35,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f0575bc060ce15e8b2 2025-12-17 22:24:35,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f0575bc060ce15e8b2 HTTP/1.1" 502 150 2025-12-17 22:24:35,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f0575bc060ce15e8b2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:35,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ea575bc060ce15e8a3 2025-12-17 22:24:35,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ea575bc060ce15e8a3 HTTP/1.1" 502 150 2025-12-17 22:24:35,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ea575bc060ce15e8a3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36872 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:24:36,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f2575bc060ce15e8ba 2025-12-17 22:24:36,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f2575bc060ce15e8ba HTTP/1.1" 502 150 2025-12-17 22:24:36,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f2575bc060ce15e8ba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:37,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e7575bc060ce15e899 2025-12-17 22:24:37,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e7575bc060ce15e899 HTTP/1.1" 502 150 2025-12-17 22:24:37,052 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e7575bc060ce15e899'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:39,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088eb575bc060ce15e8a6 2025-12-17 22:24:39,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088eb575bc060ce15e8a6 HTTP/1.1" 502 150 2025-12-17 22:24:39,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088eb575bc060ce15e8a6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:39,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ed575bc060ce15e8ac 2025-12-17 22:24:39,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ed575bc060ce15e8ac HTTP/1.1" 502 150 2025-12-17 22:24:39,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ed575bc060ce15e8ac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:39,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088d7575bc060ce15e83a 2025-12-17 22:24:39,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088d7575bc060ce15e83a HTTP/1.1" 502 150 2025-12-17 22:24:39,864 - lava_callback - ERROR - Error processing callback for node 694088d7575bc060ce15e83a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d7575bc060ce15e83a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d7575bc060ce15e83a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088d7575bc060ce15e83a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:39,866 - lava_callback - INFO - Starting async job submit for node 694088f5575bc060ce15e8cc 2025-12-17 22:24:39,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:39,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f5575bc060ce15e8cc HTTP/1.1" 502 150 2025-12-17 22:24:39,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f5575bc060ce15e8cc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:39,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f5575bc060ce15e8cc 2025-12-17 22:24:39,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f5575bc060ce15e8cc HTTP/1.1" 502 150 2025-12-17 22:24:39,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f5575bc060ce15e8cc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:39,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f0575bc060ce15e8b2 2025-12-17 22:24:39,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f0575bc060ce15e8b2 HTTP/1.1" 502 150 2025-12-17 22:24:39,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f0575bc060ce15e8b2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:40,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f2575bc060ce15e8ba 2025-12-17 22:24:40,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f2575bc060ce15e8ba HTTP/1.1" 502 150 2025-12-17 22:24:40,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f2575bc060ce15e8ba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:40,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e8575bc060ce15e89c 2025-12-17 22:24:40,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e8575bc060ce15e89c HTTP/1.1" 502 150 2025-12-17 22:24:40,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e8575bc060ce15e89c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:40,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e5575bc060ce15e893 2025-12-17 22:24:40,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e5575bc060ce15e893 HTTP/1.1" 502 150 2025-12-17 22:24:40,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088e5575bc060ce15e893'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:41,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f5575bc060ce15e8cc 2025-12-17 22:24:41,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f5575bc060ce15e8cc HTTP/1.1" 502 150 2025-12-17 22:24:41,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f5575bc060ce15e8cc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:41,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ec575bc060ce15e8a9 2025-12-17 22:24:41,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ec575bc060ce15e8a9 HTTP/1.1" 502 150 2025-12-17 22:24:41,967 - lava_callback - ERROR - Error processing callback for node 694088ec575bc060ce15e8a9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ec575bc060ce15e8a9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ec575bc060ce15e8a9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ec575bc060ce15e8a9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:41,969 - lava_callback - INFO - Starting async job submit for node 694088f9575bc060ce15e8d6 2025-12-17 22:24:41,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:41,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f9575bc060ce15e8d6 HTTP/1.1" 502 150 2025-12-17 22:24:41,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f9575bc060ce15e8d6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:41,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f9575bc060ce15e8d6 2025-12-17 22:24:41,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f9575bc060ce15e8d6 HTTP/1.1" 502 150 2025-12-17 22:24:41,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f9575bc060ce15e8d6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:43,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ea575bc060ce15e8a3 2025-12-17 22:24:43,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ea575bc060ce15e8a3 HTTP/1.1" 502 150 2025-12-17 22:24:43,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ea575bc060ce15e8a3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:43,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f9575bc060ce15e8d6 2025-12-17 22:24:43,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f9575bc060ce15e8d6 HTTP/1.1" 502 150 2025-12-17 22:24:43,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f9575bc060ce15e8d6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:44,565 - lava_callback - INFO - Received callback for node 69408a99575bc060ce15eef3 INFO: 172.21.0.1:44306 - "POST /node/69408a99575bc060ce15eef3 HTTP/1.0" 202 Accepted 2025-12-17 22:24:45,215 - lava_callback - INFO - Received callback for node 69408a9b575bc060ce15eefd INFO: 172.21.0.1:44322 - "POST /node/69408a9b575bc060ce15eefd HTTP/1.0" 202 Accepted 2025-12-17 22:24:45,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ef575bc060ce15e8af 2025-12-17 22:24:45,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ef575bc060ce15e8af HTTP/1.1" 502 150 2025-12-17 22:24:45,836 - lava_callback - ERROR - Error processing callback for node 694088ef575bc060ce15e8af: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ef575bc060ce15e8af (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ef575bc060ce15e8af (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ef575bc060ce15e8af (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:45,839 - lava_callback - INFO - Starting async job submit for node 694088fd575bc060ce15e8df 2025-12-17 22:24:45,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:45,852 - lava_callback - INFO - Received callback for node 69408a9d575bc060ce15ef00 INFO: 172.21.0.1:44326 - "POST /node/69408a9d575bc060ce15ef00 HTTP/1.0" 202 Accepted 2025-12-17 22:24:45,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f5575bc060ce15e8cc 2025-12-17 22:24:45,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f5575bc060ce15e8cc HTTP/1.1" 502 150 2025-12-17 22:24:45,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f5575bc060ce15e8cc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:45,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e2575bc060ce15e88b 2025-12-17 22:24:45,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e2575bc060ce15e88b HTTP/1.1" 502 150 2025-12-17 22:24:45,974 - lava_callback - ERROR - Error processing callback for node 694088e2575bc060ce15e88b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e2575bc060ce15e88b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e2575bc060ce15e88b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e2575bc060ce15e88b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:45,976 - lava_callback - INFO - Starting async job submit for node 694088f1575bc060ce15e8b5 2025-12-17 22:24:45,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:45,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f1575bc060ce15e8b5 HTTP/1.1" 502 150 2025-12-17 22:24:45,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f1575bc060ce15e8b5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:45,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f1575bc060ce15e8b5 2025-12-17 22:24:45,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f1575bc060ce15e8b5 HTTP/1.1" 502 150 2025-12-17 22:24:45,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f1575bc060ce15e8b5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:46,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fd575bc060ce15e8df HTTP/1.1" 502 150 2025-12-17 22:24:46,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fd575bc060ce15e8df'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:46,028 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fd575bc060ce15e8df 2025-12-17 22:24:46,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fd575bc060ce15e8df HTTP/1.1" 502 150 2025-12-17 22:24:46,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fd575bc060ce15e8df'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:46,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088df575bc060ce15e87d HTTP/1.1" 502 150 2025-12-17 22:24:46,256 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088df575bc060ce15e87d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:46,484 - lava_callback - INFO - Received callback for node 69408a9f575bc060ce15ef04 INFO: 172.21.0.1:44334 - "POST /node/69408a9f575bc060ce15ef04 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:44342 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:24:47,132 - lava_callback - INFO - Received callback for node 69408aa2575bc060ce15ef0d INFO: 172.21.0.1:44356 - "POST /node/69408aa2575bc060ce15ef0d HTTP/1.0" 202 Accepted 2025-12-17 22:24:47,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088eb575bc060ce15e8a6 2025-12-17 22:24:47,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088eb575bc060ce15e8a6 HTTP/1.1" 502 150 2025-12-17 22:24:47,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088eb575bc060ce15e8a6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:47,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ed575bc060ce15e8ac 2025-12-17 22:24:47,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ed575bc060ce15e8ac HTTP/1.1" 502 150 2025-12-17 22:24:47,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ed575bc060ce15e8ac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:47,789 - lava_callback - INFO - Received callback for node 69408aa4575bc060ce15ef10 INFO: 172.21.0.1:44368 - "POST /node/69408aa4575bc060ce15ef10 HTTP/1.0" 202 Accepted 2025-12-17 22:24:47,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e1575bc060ce15e884 2025-12-17 22:24:47,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e1575bc060ce15e884 HTTP/1.1" 502 150 2025-12-17 22:24:47,875 - lava_callback - ERROR - Error processing callback for node 694088e1575bc060ce15e884: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e1575bc060ce15e884 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e1575bc060ce15e884 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e1575bc060ce15e884 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:47,877 - lava_callback - INFO - Starting async job submit for node 694088fc575bc060ce15e8dc 2025-12-17 22:24:47,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:47,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fc575bc060ce15e8dc HTTP/1.1" 502 150 2025-12-17 22:24:47,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fc575bc060ce15e8dc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:47,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fc575bc060ce15e8dc 2025-12-17 22:24:47,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fc575bc060ce15e8dc HTTP/1.1" 502 150 2025-12-17 22:24:47,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fc575bc060ce15e8dc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:47,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f0575bc060ce15e8b2 2025-12-17 22:24:47,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f0575bc060ce15e8b2 HTTP/1.1" 502 150 2025-12-17 22:24:47,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f0575bc060ce15e8b2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:47,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f9575bc060ce15e8d6 2025-12-17 22:24:47,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f9575bc060ce15e8d6 HTTP/1.1" 502 150 2025-12-17 22:24:47,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f9575bc060ce15e8d6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:47,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f1575bc060ce15e8b5 2025-12-17 22:24:47,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f1575bc060ce15e8b5 HTTP/1.1" 502 150 2025-12-17 22:24:47,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f1575bc060ce15e8b5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:48,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fd575bc060ce15e8df 2025-12-17 22:24:48,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fd575bc060ce15e8df HTTP/1.1" 502 150 2025-12-17 22:24:48,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fd575bc060ce15e8df'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:48,439 - lava_callback - INFO - Received callback for node 69408aa6575bc060ce15ef13 INFO: 172.21.0.1:44380 - "POST /node/69408aa6575bc060ce15ef13 HTTP/1.0" 202 Accepted 2025-12-17 22:24:48,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e3575bc060ce15e890 2025-12-17 22:24:48,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e3575bc060ce15e890 HTTP/1.1" 502 150 2025-12-17 22:24:48,674 - lava_callback - ERROR - Error processing callback for node 694088e3575bc060ce15e890: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e3575bc060ce15e890 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e3575bc060ce15e890 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e3575bc060ce15e890 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:48,676 - lava_callback - INFO - Starting async job submit for node 694088f4575bc060ce15e8c1 2025-12-17 22:24:48,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:48,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f4575bc060ce15e8c1 HTTP/1.1" 502 150 2025-12-17 22:24:48,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f4575bc060ce15e8c1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:48,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f4575bc060ce15e8c1 2025-12-17 22:24:48,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f4575bc060ce15e8c1 HTTP/1.1" 502 150 2025-12-17 22:24:48,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f4575bc060ce15e8c1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:48,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f2575bc060ce15e8ba 2025-12-17 22:24:48,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f2575bc060ce15e8ba HTTP/1.1" 502 150 2025-12-17 22:24:48,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f2575bc060ce15e8ba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:49,083 - lava_callback - INFO - Received callback for node 69408aa8575bc060ce15ef16 INFO: 172.21.0.1:44382 - "POST /node/69408aa8575bc060ce15ef16 HTTP/1.0" 202 Accepted 2025-12-17 22:24:49,721 - lava_callback - INFO - Received callback for node 69408aaa575bc060ce15ef19 INFO: 172.21.0.1:36342 - "POST /node/69408aaa575bc060ce15ef19 HTTP/1.0" 202 Accepted 2025-12-17 22:24:49,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fc575bc060ce15e8dc 2025-12-17 22:24:49,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fc575bc060ce15e8dc HTTP/1.1" 502 150 2025-12-17 22:24:49,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fc575bc060ce15e8dc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:50,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f4575bc060ce15e8c1 2025-12-17 22:24:50,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f4575bc060ce15e8c1 HTTP/1.1" 502 150 2025-12-17 22:24:50,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f4575bc060ce15e8c1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:51,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e6575bc060ce15e896 2025-12-17 22:24:51,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e6575bc060ce15e896 HTTP/1.1" 502 150 2025-12-17 22:24:51,894 - lava_callback - ERROR - Error processing callback for node 694088e6575bc060ce15e896: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e6575bc060ce15e896 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e6575bc060ce15e896 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e6575bc060ce15e896 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:51,896 - lava_callback - INFO - Starting async job submit for node 694088f7575bc060ce15e8cf 2025-12-17 22:24:51,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:51,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f7575bc060ce15e8cf HTTP/1.1" 502 150 2025-12-17 22:24:51,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f7575bc060ce15e8cf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:51,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f7575bc060ce15e8cf 2025-12-17 22:24:51,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f7575bc060ce15e8cf HTTP/1.1" 502 150 2025-12-17 22:24:51,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f7575bc060ce15e8cf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:51,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f1575bc060ce15e8b5 2025-12-17 22:24:51,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f1575bc060ce15e8b5 HTTP/1.1" 502 150 2025-12-17 22:24:51,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f1575bc060ce15e8b5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:52,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fd575bc060ce15e8df 2025-12-17 22:24:52,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fd575bc060ce15e8df HTTP/1.1" 502 150 2025-12-17 22:24:52,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fd575bc060ce15e8df'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:53,052 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e7575bc060ce15e899 2025-12-17 22:24:53,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e7575bc060ce15e899 HTTP/1.1" 502 150 2025-12-17 22:24:53,054 - lava_callback - ERROR - Error processing callback for node 694088e7575bc060ce15e899: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e7575bc060ce15e899 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e7575bc060ce15e899 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e7575bc060ce15e899 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:53,058 - lava_callback - INFO - Starting async job submit for node 694088fa575bc060ce15e8d9 2025-12-17 22:24:53,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:53,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fa575bc060ce15e8d9 HTTP/1.1" 502 150 2025-12-17 22:24:53,070 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fa575bc060ce15e8d9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:53,070 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fa575bc060ce15e8d9 2025-12-17 22:24:53,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fa575bc060ce15e8d9 HTTP/1.1" 502 150 2025-12-17 22:24:53,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fa575bc060ce15e8d9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:53,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f5575bc060ce15e8cc 2025-12-17 22:24:53,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f5575bc060ce15e8cc HTTP/1.1" 502 150 2025-12-17 22:24:53,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f5575bc060ce15e8cc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:53,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fc575bc060ce15e8dc 2025-12-17 22:24:53,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fc575bc060ce15e8dc HTTP/1.1" 502 150 2025-12-17 22:24:53,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fc575bc060ce15e8dc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:53,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f7575bc060ce15e8cf 2025-12-17 22:24:53,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f7575bc060ce15e8cf HTTP/1.1" 502 150 2025-12-17 22:24:53,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f7575bc060ce15e8cf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:54,495 - lava_callback - INFO - Received callback for node 69408ab0575bc060ce15ef30 INFO: 172.21.0.1:36350 - "POST /node/69408ab0575bc060ce15ef30 HTTP/1.0" 202 Accepted 2025-12-17 22:24:54,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f4575bc060ce15e8c1 2025-12-17 22:24:54,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f4575bc060ce15e8c1 HTTP/1.1" 502 150 2025-12-17 22:24:54,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f4575bc060ce15e8c1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:55,071 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fa575bc060ce15e8d9 2025-12-17 22:24:55,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fa575bc060ce15e8d9 HTTP/1.1" 502 150 2025-12-17 22:24:55,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fa575bc060ce15e8d9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:55,144 - lava_callback - INFO - Received callback for node 69408ab3575bc060ce15ef3a INFO: 172.21.0.1:36352 - "POST /node/69408ab3575bc060ce15ef3a HTTP/1.0" 202 Accepted 2025-12-17 22:24:55,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f9575bc060ce15e8d6 2025-12-17 22:24:55,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f9575bc060ce15e8d6 HTTP/1.1" 502 150 2025-12-17 22:24:55,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f9575bc060ce15e8d6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36356 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:24:56,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e8575bc060ce15e89c 2025-12-17 22:24:56,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e8575bc060ce15e89c HTTP/1.1" 502 150 2025-12-17 22:24:56,856 - lava_callback - ERROR - Error processing callback for node 694088e8575bc060ce15e89c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e8575bc060ce15e89c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e8575bc060ce15e89c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e8575bc060ce15e89c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:56,858 - lava_callback - INFO - Starting async job submit for node 694088fe575bc060ce15e8e2 2025-12-17 22:24:56,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:56,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fe575bc060ce15e8e2 HTTP/1.1" 502 150 2025-12-17 22:24:56,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fe575bc060ce15e8e2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:56,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fe575bc060ce15e8e2 2025-12-17 22:24:56,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fe575bc060ce15e8e2 HTTP/1.1" 502 150 2025-12-17 22:24:56,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fe575bc060ce15e8e2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:56,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088e5575bc060ce15e893 2025-12-17 22:24:56,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088e5575bc060ce15e893 HTTP/1.1" 502 150 2025-12-17 22:24:56,947 - lava_callback - ERROR - Error processing callback for node 694088e5575bc060ce15e893: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e5575bc060ce15e893 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e5575bc060ce15e893 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088e5575bc060ce15e893 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:56,948 - lava_callback - INFO - Starting async job submit for node 694088ff575bc060ce15e8e5 2025-12-17 22:24:56,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:56,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ff575bc060ce15e8e5 HTTP/1.1" 502 150 2025-12-17 22:24:56,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ff575bc060ce15e8e5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:56,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ff575bc060ce15e8e5 2025-12-17 22:24:56,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ff575bc060ce15e8e5 HTTP/1.1" 502 150 2025-12-17 22:24:56,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ff575bc060ce15e8e5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:57,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f7575bc060ce15e8cf 2025-12-17 22:24:57,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f7575bc060ce15e8cf HTTP/1.1" 502 150 2025-12-17 22:24:57,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f7575bc060ce15e8cf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:58,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fe575bc060ce15e8e2 2025-12-17 22:24:58,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fe575bc060ce15e8e2 HTTP/1.1" 502 150 2025-12-17 22:24:58,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fe575bc060ce15e8e2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:58,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ff575bc060ce15e8e5 2025-12-17 22:24:58,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ff575bc060ce15e8e5 HTTP/1.1" 502 150 2025-12-17 22:24:58,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ff575bc060ce15e8e5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:59,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fa575bc060ce15e8d9 2025-12-17 22:24:59,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fa575bc060ce15e8d9 HTTP/1.1" 502 150 2025-12-17 22:24:59,074 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fa575bc060ce15e8d9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:59,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ea575bc060ce15e8a3 2025-12-17 22:24:59,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ea575bc060ce15e8a3 HTTP/1.1" 502 150 2025-12-17 22:24:59,964 - lava_callback - ERROR - Error processing callback for node 694088ea575bc060ce15e8a3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ea575bc060ce15e8a3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ea575bc060ce15e8a3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ea575bc060ce15e8a3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:24:59,966 - lava_callback - INFO - Starting async job submit for node 69408901575bc060ce15e8e8 2025-12-17 22:24:59,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:24:59,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408901575bc060ce15e8e8 HTTP/1.1" 502 150 2025-12-17 22:24:59,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408901575bc060ce15e8e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:59,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408901575bc060ce15e8e8 2025-12-17 22:24:59,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408901575bc060ce15e8e8 HTTP/1.1" 502 150 2025-12-17 22:24:59,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408901575bc060ce15e8e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:24:59,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f1575bc060ce15e8b5 2025-12-17 22:24:59,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f1575bc060ce15e8b5 HTTP/1.1" 502 150 2025-12-17 22:24:59,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f1575bc060ce15e8b5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:00,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fd575bc060ce15e8df 2025-12-17 22:25:00,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fd575bc060ce15e8df HTTP/1.1" 502 150 2025-12-17 22:25:00,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fd575bc060ce15e8df'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:01,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fc575bc060ce15e8dc 2025-12-17 22:25:01,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fc575bc060ce15e8dc HTTP/1.1" 502 150 2025-12-17 22:25:01,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fc575bc060ce15e8dc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:01,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408901575bc060ce15e8e8 2025-12-17 22:25:01,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408901575bc060ce15e8e8 HTTP/1.1" 502 150 2025-12-17 22:25:01,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408901575bc060ce15e8e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:02,257 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088df575bc060ce15e87d 2025-12-17 22:25:02,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088df575bc060ce15e87d HTTP/1.1" 502 150 2025-12-17 22:25:02,258 - lava_callback - ERROR - Error processing callback for node 694088df575bc060ce15e87d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088df575bc060ce15e87d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088df575bc060ce15e87d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088df575bc060ce15e87d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:02,260 - lava_callback - INFO - Starting async job submit for node 69408906575bc060ce15e8ef 2025-12-17 22:25:02,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:02,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408906575bc060ce15e8ef HTTP/1.1" 502 150 2025-12-17 22:25:02,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408906575bc060ce15e8ef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:02,267 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408906575bc060ce15e8ef 2025-12-17 22:25:02,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408906575bc060ce15e8ef HTTP/1.1" 502 150 2025-12-17 22:25:02,267 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408906575bc060ce15e8ef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53512 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:25:02,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f4575bc060ce15e8c1 2025-12-17 22:25:02,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f4575bc060ce15e8c1 HTTP/1.1" 502 150 2025-12-17 22:25:02,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f4575bc060ce15e8c1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:02,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fe575bc060ce15e8e2 2025-12-17 22:25:02,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fe575bc060ce15e8e2 HTTP/1.1" 502 150 2025-12-17 22:25:02,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fe575bc060ce15e8e2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:02,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ff575bc060ce15e8e5 2025-12-17 22:25:02,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ff575bc060ce15e8e5 HTTP/1.1" 502 150 2025-12-17 22:25:02,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ff575bc060ce15e8e5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:03,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088eb575bc060ce15e8a6 2025-12-17 22:25:03,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088eb575bc060ce15e8a6 HTTP/1.1" 502 150 2025-12-17 22:25:03,475 - lava_callback - ERROR - Error processing callback for node 694088eb575bc060ce15e8a6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088eb575bc060ce15e8a6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088eb575bc060ce15e8a6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088eb575bc060ce15e8a6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:03,477 - lava_callback - INFO - Starting async job submit for node 69408907575bc060ce15e8f4 2025-12-17 22:25:03,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:03,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408907575bc060ce15e8f4 HTTP/1.1" 502 150 2025-12-17 22:25:03,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408907575bc060ce15e8f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:03,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408907575bc060ce15e8f4 2025-12-17 22:25:03,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408907575bc060ce15e8f4 HTTP/1.1" 502 150 2025-12-17 22:25:03,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408907575bc060ce15e8f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:03,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ed575bc060ce15e8ac 2025-12-17 22:25:03,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ed575bc060ce15e8ac HTTP/1.1" 502 150 2025-12-17 22:25:03,635 - lava_callback - ERROR - Error processing callback for node 694088ed575bc060ce15e8ac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ed575bc060ce15e8ac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ed575bc060ce15e8ac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ed575bc060ce15e8ac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:03,636 - lava_callback - INFO - Starting async job submit for node 69408909575bc060ce15e8fb 2025-12-17 22:25:03,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:03,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408909575bc060ce15e8fb HTTP/1.1" 502 150 2025-12-17 22:25:03,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408909575bc060ce15e8fb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:03,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408909575bc060ce15e8fb 2025-12-17 22:25:03,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408909575bc060ce15e8fb HTTP/1.1" 502 150 2025-12-17 22:25:03,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408909575bc060ce15e8fb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:03,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f0575bc060ce15e8b2 2025-12-17 22:25:03,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f0575bc060ce15e8b2 HTTP/1.1" 502 150 2025-12-17 22:25:03,960 - lava_callback - ERROR - Error processing callback for node 694088f0575bc060ce15e8b2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f0575bc060ce15e8b2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f0575bc060ce15e8b2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f0575bc060ce15e8b2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:03,962 - lava_callback - INFO - Starting async job submit for node 6940890a575bc060ce15e902 2025-12-17 22:25:03,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:03,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890a575bc060ce15e902 HTTP/1.1" 502 150 2025-12-17 22:25:03,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890a575bc060ce15e902'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:03,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890a575bc060ce15e902 2025-12-17 22:25:03,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890a575bc060ce15e902 HTTP/1.1" 502 150 2025-12-17 22:25:03,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890a575bc060ce15e902'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:04,268 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408906575bc060ce15e8ef 2025-12-17 22:25:04,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408906575bc060ce15e8ef HTTP/1.1" 502 150 2025-12-17 22:25:04,269 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408906575bc060ce15e8ef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:04,613 - lava_callback - INFO - Received callback for node 69408aac575bc060ce15ef1c INFO: 172.21.0.1:53518 - "POST /node/69408aac575bc060ce15ef1c HTTP/1.0" 202 Accepted 2025-12-17 22:25:04,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f2575bc060ce15e8ba 2025-12-17 22:25:04,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f2575bc060ce15e8ba HTTP/1.1" 502 150 2025-12-17 22:25:04,855 - lava_callback - ERROR - Error processing callback for node 694088f2575bc060ce15e8ba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f2575bc060ce15e8ba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f2575bc060ce15e8ba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f2575bc060ce15e8ba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:04,858 - lava_callback - INFO - Starting async job submit for node 6940890b575bc060ce15e909 2025-12-17 22:25:04,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:04,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890b575bc060ce15e909 HTTP/1.1" 502 150 2025-12-17 22:25:04,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890b575bc060ce15e909'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:04,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890b575bc060ce15e909 2025-12-17 22:25:04,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890b575bc060ce15e909 HTTP/1.1" 502 150 2025-12-17 22:25:04,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890b575bc060ce15e909'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:05,282 - lava_callback - INFO - Received callback for node 69408abc575bc060ce15ef4d INFO: 172.21.0.1:53520 - "POST /node/69408abc575bc060ce15ef4d HTTP/1.0" 202 Accepted 2025-12-17 22:25:05,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408907575bc060ce15e8f4 2025-12-17 22:25:05,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408907575bc060ce15e8f4 HTTP/1.1" 502 150 2025-12-17 22:25:05,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408907575bc060ce15e8f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:05,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408909575bc060ce15e8fb 2025-12-17 22:25:05,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408909575bc060ce15e8fb HTTP/1.1" 502 150 2025-12-17 22:25:05,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408909575bc060ce15e8fb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:05,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f7575bc060ce15e8cf 2025-12-17 22:25:05,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f7575bc060ce15e8cf HTTP/1.1" 502 150 2025-12-17 22:25:05,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088f7575bc060ce15e8cf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:05,923 - lava_callback - INFO - Received callback for node 69408abe575bc060ce15ef50 INFO: 172.21.0.1:53532 - "POST /node/69408abe575bc060ce15ef50 HTTP/1.0" 202 Accepted 2025-12-17 22:25:05,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890a575bc060ce15e902 2025-12-17 22:25:05,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890a575bc060ce15e902 HTTP/1.1" 502 150 2025-12-17 22:25:05,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890a575bc060ce15e902'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:05,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408901575bc060ce15e8e8 2025-12-17 22:25:05,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408901575bc060ce15e8e8 HTTP/1.1" 502 150 2025-12-17 22:25:05,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408901575bc060ce15e8e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53534 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:25:06,580 - lava_callback - INFO - Received callback for node 69408abf575bc060ce15ef55 INFO: 172.21.0.1:53540 - "POST /node/69408abf575bc060ce15ef55 HTTP/1.0" 202 Accepted 2025-12-17 22:25:06,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890b575bc060ce15e909 2025-12-17 22:25:06,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890b575bc060ce15e909 HTTP/1.1" 502 150 2025-12-17 22:25:06,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890b575bc060ce15e909'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:07,074 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fa575bc060ce15e8d9 2025-12-17 22:25:07,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fa575bc060ce15e8d9 HTTP/1.1" 502 150 2025-12-17 22:25:07,075 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fa575bc060ce15e8d9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:08,269 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408906575bc060ce15e8ef 2025-12-17 22:25:08,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408906575bc060ce15e8ef HTTP/1.1" 502 150 2025-12-17 22:25:08,270 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408906575bc060ce15e8ef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:09,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408907575bc060ce15e8f4 2025-12-17 22:25:09,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408907575bc060ce15e8f4 HTTP/1.1" 502 150 2025-12-17 22:25:09,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408907575bc060ce15e8f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:09,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408909575bc060ce15e8fb 2025-12-17 22:25:09,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408909575bc060ce15e8fb HTTP/1.1" 502 150 2025-12-17 22:25:09,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408909575bc060ce15e8fb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:09,697 - lava_callback - INFO - Received callback for node 69408abb575bc060ce15ef4a INFO: 172.21.0.1:34096 - "POST /node/69408abb575bc060ce15ef4a HTTP/1.0" 202 Accepted 2025-12-17 22:25:09,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f5575bc060ce15e8cc 2025-12-17 22:25:09,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f5575bc060ce15e8cc HTTP/1.1" 502 150 2025-12-17 22:25:09,880 - lava_callback - ERROR - Error processing callback for node 694088f5575bc060ce15e8cc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f5575bc060ce15e8cc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f5575bc060ce15e8cc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f5575bc060ce15e8cc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:09,882 - lava_callback - INFO - Starting async job submit for node 6940890d575bc060ce15e910 2025-12-17 22:25:09,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:09,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890d575bc060ce15e910 HTTP/1.1" 502 150 2025-12-17 22:25:09,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890d575bc060ce15e910'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:09,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890d575bc060ce15e910 2025-12-17 22:25:09,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890d575bc060ce15e910 HTTP/1.1" 502 150 2025-12-17 22:25:09,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890d575bc060ce15e910'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:09,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890a575bc060ce15e902 2025-12-17 22:25:09,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890a575bc060ce15e902 HTTP/1.1" 502 150 2025-12-17 22:25:09,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890a575bc060ce15e902'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:10,398 - lava_callback - INFO - Received callback for node 69408ac2575bc060ce15ef63 INFO: 172.21.0.1:34102 - "POST /node/69408ac2575bc060ce15ef63 HTTP/1.0" 202 Accepted 2025-12-17 22:25:10,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fe575bc060ce15e8e2 2025-12-17 22:25:10,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fe575bc060ce15e8e2 HTTP/1.1" 502 150 2025-12-17 22:25:10,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088fe575bc060ce15e8e2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:10,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890b575bc060ce15e909 2025-12-17 22:25:10,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890b575bc060ce15e909 HTTP/1.1" 502 150 2025-12-17 22:25:10,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890b575bc060ce15e909'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:10,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ff575bc060ce15e8e5 2025-12-17 22:25:10,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ff575bc060ce15e8e5 HTTP/1.1" 502 150 2025-12-17 22:25:10,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694088ff575bc060ce15e8e5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:11,061 - lava_callback - INFO - Received callback for node 69408ac3575bc060ce15ef6a INFO: 172.21.0.1:34104 - "POST /node/69408ac3575bc060ce15ef6a HTTP/1.0" 202 Accepted 2025-12-17 22:25:11,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890d575bc060ce15e910 2025-12-17 22:25:11,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890d575bc060ce15e910 HTTP/1.1" 502 150 2025-12-17 22:25:11,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890d575bc060ce15e910'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:11,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f9575bc060ce15e8d6 2025-12-17 22:25:11,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f9575bc060ce15e8d6 HTTP/1.1" 502 150 2025-12-17 22:25:11,983 - lava_callback - ERROR - Error processing callback for node 694088f9575bc060ce15e8d6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f9575bc060ce15e8d6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f9575bc060ce15e8d6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f9575bc060ce15e8d6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:11,985 - lava_callback - INFO - Starting async job submit for node 6940890e575bc060ce15e913 2025-12-17 22:25:11,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:11,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890e575bc060ce15e913 HTTP/1.1" 502 150 2025-12-17 22:25:11,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890e575bc060ce15e913'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:11,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890e575bc060ce15e913 2025-12-17 22:25:11,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890e575bc060ce15e913 HTTP/1.1" 502 150 2025-12-17 22:25:11,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890e575bc060ce15e913'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:13,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408901575bc060ce15e8e8 2025-12-17 22:25:13,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408901575bc060ce15e8e8 HTTP/1.1" 502 150 2025-12-17 22:25:13,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408901575bc060ce15e8e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:13,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890e575bc060ce15e913 2025-12-17 22:25:13,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890e575bc060ce15e913 HTTP/1.1" 502 150 2025-12-17 22:25:13,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890e575bc060ce15e913'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:14,484 - lava_callback - INFO - Received callback for node 69408ab5575bc060ce15ef41 INFO: 172.21.0.1:34106 - "POST /node/69408ab5575bc060ce15ef41 HTTP/1.0" 202 Accepted 2025-12-17 22:25:15,143 - lava_callback - INFO - Received callback for node 69408ab7575bc060ce15ef44 INFO: 172.21.0.1:34108 - "POST /node/69408ab7575bc060ce15ef44 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:34116 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:25:15,808 - lava_callback - INFO - Received callback for node 69408aba575bc060ce15ef47 INFO: 172.21.0.1:34122 - "POST /node/69408aba575bc060ce15ef47 HTTP/1.0" 202 Accepted 2025-12-17 22:25:15,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890d575bc060ce15e910 2025-12-17 22:25:15,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890d575bc060ce15e910 HTTP/1.1" 502 150 2025-12-17 22:25:15,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890d575bc060ce15e910'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:15,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f1575bc060ce15e8b5 2025-12-17 22:25:15,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f1575bc060ce15e8b5 HTTP/1.1" 502 150 2025-12-17 22:25:15,989 - lava_callback - ERROR - Error processing callback for node 694088f1575bc060ce15e8b5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f1575bc060ce15e8b5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f1575bc060ce15e8b5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f1575bc060ce15e8b5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:15,991 - lava_callback - INFO - Starting async job submit for node 69408910575bc060ce15e919 2025-12-17 22:25:15,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:16,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408910575bc060ce15e919 HTTP/1.1" 502 150 2025-12-17 22:25:16,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408910575bc060ce15e919'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:16,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408910575bc060ce15e919 2025-12-17 22:25:16,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408910575bc060ce15e919 HTTP/1.1" 502 150 2025-12-17 22:25:16,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408910575bc060ce15e919'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:16,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fd575bc060ce15e8df 2025-12-17 22:25:16,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fd575bc060ce15e8df HTTP/1.1" 502 150 2025-12-17 22:25:16,035 - lava_callback - ERROR - Error processing callback for node 694088fd575bc060ce15e8df: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088fd575bc060ce15e8df (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088fd575bc060ce15e8df (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088fd575bc060ce15e8df (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:16,037 - lava_callback - INFO - Starting async job submit for node 69408914575bc060ce15e91f 2025-12-17 22:25:16,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:16,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408914575bc060ce15e91f HTTP/1.1" 502 150 2025-12-17 22:25:16,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408914575bc060ce15e91f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:16,071 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408914575bc060ce15e91f 2025-12-17 22:25:16,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408914575bc060ce15e91f HTTP/1.1" 502 150 2025-12-17 22:25:16,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408914575bc060ce15e91f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:16,271 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408906575bc060ce15e8ef 2025-12-17 22:25:16,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408906575bc060ce15e8ef HTTP/1.1" 502 150 2025-12-17 22:25:16,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408906575bc060ce15e8ef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34126 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:25:17,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408907575bc060ce15e8f4 2025-12-17 22:25:17,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408907575bc060ce15e8f4 HTTP/1.1" 502 150 2025-12-17 22:25:17,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408907575bc060ce15e8f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:17,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408909575bc060ce15e8fb 2025-12-17 22:25:17,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408909575bc060ce15e8fb HTTP/1.1" 502 150 2025-12-17 22:25:17,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408909575bc060ce15e8fb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:17,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fc575bc060ce15e8dc 2025-12-17 22:25:17,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fc575bc060ce15e8dc HTTP/1.1" 502 150 2025-12-17 22:25:17,897 - lava_callback - ERROR - Error processing callback for node 694088fc575bc060ce15e8dc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088fc575bc060ce15e8dc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088fc575bc060ce15e8dc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088fc575bc060ce15e8dc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:17,900 - lava_callback - INFO - Starting async job submit for node 69408917575bc060ce15e922 2025-12-17 22:25:17,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:17,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408917575bc060ce15e922 HTTP/1.1" 502 150 2025-12-17 22:25:17,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408917575bc060ce15e922'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:17,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408917575bc060ce15e922 2025-12-17 22:25:17,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408917575bc060ce15e922 HTTP/1.1" 502 150 2025-12-17 22:25:17,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408917575bc060ce15e922'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:17,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890a575bc060ce15e902 2025-12-17 22:25:17,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890a575bc060ce15e902 HTTP/1.1" 502 150 2025-12-17 22:25:17,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890a575bc060ce15e902'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:17,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890e575bc060ce15e913 2025-12-17 22:25:18,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890e575bc060ce15e913 HTTP/1.1" 502 150 2025-12-17 22:25:18,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890e575bc060ce15e913'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:18,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408910575bc060ce15e919 2025-12-17 22:25:18,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408910575bc060ce15e919 HTTP/1.1" 502 150 2025-12-17 22:25:18,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408910575bc060ce15e919'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:18,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408914575bc060ce15e91f 2025-12-17 22:25:18,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408914575bc060ce15e91f HTTP/1.1" 502 150 2025-12-17 22:25:18,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408914575bc060ce15e91f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:18,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f4575bc060ce15e8c1 2025-12-17 22:25:18,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f4575bc060ce15e8c1 HTTP/1.1" 502 150 2025-12-17 22:25:18,688 - lava_callback - ERROR - Error processing callback for node 694088f4575bc060ce15e8c1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f4575bc060ce15e8c1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f4575bc060ce15e8c1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f4575bc060ce15e8c1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:18,690 - lava_callback - INFO - Starting async job submit for node 69408919575bc060ce15e925 2025-12-17 22:25:18,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:18,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408919575bc060ce15e925 HTTP/1.1" 502 150 2025-12-17 22:25:18,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408919575bc060ce15e925'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:18,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408919575bc060ce15e925 2025-12-17 22:25:18,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408919575bc060ce15e925 HTTP/1.1" 502 150 2025-12-17 22:25:18,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408919575bc060ce15e925'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:18,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890b575bc060ce15e909 2025-12-17 22:25:18,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890b575bc060ce15e909 HTTP/1.1" 502 150 2025-12-17 22:25:18,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890b575bc060ce15e909'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:19,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408917575bc060ce15e922 2025-12-17 22:25:20,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408919575bc060ce15e925 2025-12-17 22:25:21,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088f7575bc060ce15e8cf 2025-12-17 22:25:22,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408910575bc060ce15e919 2025-12-17 22:25:22,074 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408914575bc060ce15e91f 2025-12-17 22:25:23,076 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fa575bc060ce15e8d9 2025-12-17 22:25:23,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890d575bc060ce15e910 2025-12-17 22:25:24,605 - lava_callback - INFO - Received callback for node 69408ac0575bc060ce15ef5c INFO: 172.21.0.1:54810 - "POST /node/69408ac0575bc060ce15ef5c HTTP/1.0" 202 Accepted 2025-12-17 22:25:25,254 - lava_callback - INFO - Received callback for node 69408ac4575bc060ce15ef6d INFO: 172.21.0.1:54818 - "POST /node/69408ac4575bc060ce15ef6d HTTP/1.0" 202 Accepted 2025-12-17 22:25:25,915 - lava_callback - INFO - Received callback for node 69408ac5575bc060ce15ef70 INFO: 172.21.0.1:54828 - "POST /node/69408ac5575bc060ce15ef70 HTTP/1.0" 202 Accepted 2025-12-17 22:25:26,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890e575bc060ce15e913 INFO: 172.21.0.1:54832 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:25:26,565 - lava_callback - INFO - Received callback for node 69408ac7575bc060ce15ef73 INFO: 172.21.0.1:54846 - "POST /node/69408ac7575bc060ce15ef73 HTTP/1.0" 202 Accepted 2025-12-17 22:25:26,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088fe575bc060ce15e8e2 2025-12-17 22:25:26,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694088ff575bc060ce15e8e5 2025-12-17 22:25:27,205 - lava_callback - INFO - Received callback for node 69408ac8575bc060ce15ef76 INFO: 172.21.0.1:54852 - "POST /node/69408ac8575bc060ce15ef76 HTTP/1.0" 202 Accepted 2025-12-17 22:25:27,852 - lava_callback - INFO - Received callback for node 69408ac9575bc060ce15ef79 INFO: 172.21.0.1:54868 - "POST /node/69408ac9575bc060ce15ef79 HTTP/1.0" 202 Accepted 2025-12-17 22:25:28,494 - lava_callback - INFO - Received callback for node 69408aca575bc060ce15ef7c INFO: 172.21.0.1:54870 - "POST /node/69408aca575bc060ce15ef7c HTTP/1.0" 202 Accepted 2025-12-17 22:25:29,156 - lava_callback - INFO - Received callback for node 69408acc575bc060ce15ef7f INFO: 172.21.0.1:54878 - "POST /node/69408acc575bc060ce15ef7f HTTP/1.0" 202 Accepted 2025-12-17 22:25:29,794 - lava_callback - INFO - Received callback for node 69408acd575bc060ce15ef82 INFO: 172.21.0.1:54366 - "POST /node/69408acd575bc060ce15ef82 HTTP/1.0" 202 Accepted 2025-12-17 22:25:29,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408901575bc060ce15e8e8 2025-12-17 22:25:30,450 - lava_callback - INFO - Received callback for node 69408acf575bc060ce15efba INFO: 172.21.0.1:54382 - "POST /node/69408acf575bc060ce15efba HTTP/1.0" 202 Accepted 2025-12-17 22:25:31,091 - lava_callback - INFO - Received callback for node 69408ad1575bc060ce15efd5 INFO: 172.21.0.1:54398 - "POST /node/69408ad1575bc060ce15efd5 HTTP/1.0" 202 Accepted 2025-12-17 22:25:31,754 - lava_callback - INFO - Received callback for node 69408ad2575bc060ce15efd8 INFO: 172.21.0.1:54404 - "POST /node/69408ad2575bc060ce15efd8 HTTP/1.0" 202 Accepted 2025-12-17 22:25:32,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408906575bc060ce15e8ef 2025-12-17 22:25:33,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408907575bc060ce15e8f4 2025-12-17 22:25:33,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408909575bc060ce15e8fb 2025-12-17 22:25:33,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890a575bc060ce15e902 2025-12-17 22:25:34,240 - lava_callback - INFO - Received callback for node 69408fd6575bc060ce160272 INFO: 172.21.0.1:54412 - "POST /node/69408fd6575bc060ce160272 HTTP/1.0" 202 Accepted 2025-12-17 22:25:34,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890b575bc060ce15e909 2025-12-17 22:25:35,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408917575bc060ce15e922 HTTP/1.1" 502 150 2025-12-17 22:25:35,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408917575bc060ce15e922'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:36,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408919575bc060ce15e925 HTTP/1.1" 502 150 2025-12-17 22:25:36,176 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408919575bc060ce15e925'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54414 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:25:37,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408914575bc060ce15e91f HTTP/1.1" 502 150 2025-12-17 22:25:37,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408914575bc060ce15e91f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:37,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408910575bc060ce15e919 HTTP/1.1" 502 150 2025-12-17 22:25:37,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408910575bc060ce15e919'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:38,429 - lava_callback - INFO - Received callback for node 69408926575bc060ce15e98f INFO: 172.21.0.1:54416 - "POST /node/69408926575bc060ce15e98f HTTP/1.0" 202 Accepted 2025-12-17 22:25:38,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fa575bc060ce15e8d9 HTTP/1.1" 502 150 2025-12-17 22:25:38,480 - lava_callback - ERROR - Error processing callback for node 694088fa575bc060ce15e8d9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088fa575bc060ce15e8d9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088fa575bc060ce15e8d9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088fa575bc060ce15e8d9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:38,482 - lava_callback - INFO - Starting async job submit for node 6940890f575bc060ce15e916 2025-12-17 22:25:38,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:38,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890f575bc060ce15e916 HTTP/1.1" 502 150 2025-12-17 22:25:38,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890f575bc060ce15e916'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:38,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890f575bc060ce15e916 2025-12-17 22:25:38,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890f575bc060ce15e916 HTTP/1.1" 502 150 2025-12-17 22:25:38,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890f575bc060ce15e916'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:38,797 - lava_callback - INFO - Received callback for node 694098c0575bc060ce161b4f INFO: 172.21.0.1:54426 - "POST /node/694098c0575bc060ce161b4f HTTP/1.0" 202 Accepted 2025-12-17 22:25:39,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890d575bc060ce15e910 HTTP/1.1" 502 150 2025-12-17 22:25:39,248 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890d575bc060ce15e910'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:39,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408917575bc060ce15e922 2025-12-17 22:25:39,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408917575bc060ce15e922 HTTP/1.1" 502 150 2025-12-17 22:25:39,410 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408917575bc060ce15e922'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:40,176 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408919575bc060ce15e925 2025-12-17 22:25:40,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408919575bc060ce15e925 HTTP/1.1" 502 150 2025-12-17 22:25:40,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408919575bc060ce15e925'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:40,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890f575bc060ce15e916 2025-12-17 22:25:40,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890f575bc060ce15e916 HTTP/1.1" 502 150 2025-12-17 22:25:40,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890f575bc060ce15e916'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:41,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890e575bc060ce15e913 HTTP/1.1" 502 150 2025-12-17 22:25:41,296 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890e575bc060ce15e913'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:42,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088fe575bc060ce15e8e2 HTTP/1.1" 502 150 2025-12-17 22:25:42,328 - lava_callback - ERROR - Error processing callback for node 694088fe575bc060ce15e8e2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088fe575bc060ce15e8e2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088fe575bc060ce15e8e2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088fe575bc060ce15e8e2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:42,331 - lava_callback - INFO - Starting async job submit for node 69408912575bc060ce15e91c 2025-12-17 22:25:42,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088ff575bc060ce15e8e5 HTTP/1.1" 502 150 2025-12-17 22:25:42,332 - lava_callback - ERROR - Error processing callback for node 694088ff575bc060ce15e8e5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ff575bc060ce15e8e5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ff575bc060ce15e8e5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088ff575bc060ce15e8e5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:42,335 - lava_callback - INFO - Starting async job submit for node 69408922575bc060ce15e97c 2025-12-17 22:25:42,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:42,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:42,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408922575bc060ce15e97c HTTP/1.1" 502 150 2025-12-17 22:25:42,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408922575bc060ce15e97c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:42,351 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408922575bc060ce15e97c 2025-12-17 22:25:42,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408922575bc060ce15e97c HTTP/1.1" 502 150 2025-12-17 22:25:42,352 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408922575bc060ce15e97c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:42,816 - lava_callback - INFO - Received callback for node 69408ad4575bc060ce15efdf INFO: 172.21.0.1:33676 - "POST /node/69408ad4575bc060ce15efdf HTTP/1.0" 202 Accepted 2025-12-17 22:25:44,352 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408922575bc060ce15e97c 2025-12-17 22:25:44,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408922575bc060ce15e97c HTTP/1.1" 502 150 2025-12-17 22:25:44,353 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408922575bc060ce15e97c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:44,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890f575bc060ce15e916 2025-12-17 22:25:44,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890f575bc060ce15e916 HTTP/1.1" 502 150 2025-12-17 22:25:44,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890f575bc060ce15e916'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:44,630 - lava_callback - INFO - Received callback for node 69408ad5575bc060ce15efe6 INFO: 172.21.0.1:33682 - "POST /node/69408ad5575bc060ce15efe6 HTTP/1.0" 202 Accepted 2025-12-17 22:25:45,291 - lava_callback - INFO - Received callback for node 69408ad6575bc060ce15efed INFO: 172.21.0.1:33698 - "POST /node/69408ad6575bc060ce15efed HTTP/1.0" 202 Accepted 2025-12-17 22:25:45,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408901575bc060ce15e8e8 HTTP/1.1" 502 150 2025-12-17 22:25:45,392 - lava_callback - ERROR - Error processing callback for node 69408901575bc060ce15e8e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408901575bc060ce15e8e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408901575bc060ce15e8e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408901575bc060ce15e8e8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:45,395 - lava_callback - INFO - Starting async job submit for node 6940891b575bc060ce15e93a 2025-12-17 22:25:45,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:45,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940891b575bc060ce15e93a HTTP/1.1" 502 150 2025-12-17 22:25:45,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940891b575bc060ce15e93a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:45,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940891b575bc060ce15e93a 2025-12-17 22:25:45,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940891b575bc060ce15e93a HTTP/1.1" 502 150 2025-12-17 22:25:45,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940891b575bc060ce15e93a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:45,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408914575bc060ce15e91f 2025-12-17 22:25:45,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408910575bc060ce15e919 2025-12-17 22:25:45,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408914575bc060ce15e91f HTTP/1.1" 502 150 2025-12-17 22:25:45,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408914575bc060ce15e91f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:45,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408910575bc060ce15e919 HTTP/1.1" 502 150 2025-12-17 22:25:45,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408910575bc060ce15e919'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:45,943 - lava_callback - INFO - Received callback for node 69408ada575bc060ce15f001 INFO: 172.21.0.1:33710 - "POST /node/69408ada575bc060ce15f001 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:33714 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:25:46,576 - lava_callback - INFO - Received callback for node 69408adc575bc060ce15f004 INFO: 172.21.0.1:33730 - "POST /node/69408adc575bc060ce15f004 HTTP/1.0" 202 Accepted 2025-12-17 22:25:47,224 - lava_callback - INFO - Received callback for node 69408ade575bc060ce15f007 INFO: 172.21.0.1:33734 - "POST /node/69408ade575bc060ce15f007 HTTP/1.0" 202 Accepted 2025-12-17 22:25:47,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408917575bc060ce15e922 2025-12-17 22:25:47,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408917575bc060ce15e922 HTTP/1.1" 502 150 2025-12-17 22:25:47,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408917575bc060ce15e922'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:47,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940891b575bc060ce15e93a 2025-12-17 22:25:47,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940891b575bc060ce15e93a HTTP/1.1" 502 150 2025-12-17 22:25:47,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940891b575bc060ce15e93a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:47,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408906575bc060ce15e8ef HTTP/1.1" 502 150 2025-12-17 22:25:47,696 - lava_callback - ERROR - Error processing callback for node 69408906575bc060ce15e8ef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408906575bc060ce15e8ef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408906575bc060ce15e8ef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408906575bc060ce15e8ef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:47,699 - lava_callback - INFO - Starting async job submit for node 6940891e575bc060ce15e976 2025-12-17 22:25:47,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:47,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940891e575bc060ce15e976 HTTP/1.1" 502 150 2025-12-17 22:25:47,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940891e575bc060ce15e976'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:47,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940891e575bc060ce15e976 2025-12-17 22:25:47,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940891e575bc060ce15e976 HTTP/1.1" 502 150 2025-12-17 22:25:47,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940891e575bc060ce15e976'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:47,889 - lava_callback - INFO - Received callback for node 69408ae0575bc060ce15f00a INFO: 172.21.0.1:33738 - "POST /node/69408ae0575bc060ce15f00a HTTP/1.0" 202 Accepted 2025-12-17 22:25:48,178 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408919575bc060ce15e925 2025-12-17 22:25:48,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408919575bc060ce15e925 HTTP/1.1" 502 150 2025-12-17 22:25:48,179 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408919575bc060ce15e925'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:48,354 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408922575bc060ce15e97c 2025-12-17 22:25:48,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408922575bc060ce15e97c HTTP/1.1" 502 150 2025-12-17 22:25:48,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408922575bc060ce15e97c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:48,535 - lava_callback - INFO - Received callback for node 69408ae2575bc060ce15f00d INFO: 172.21.0.1:33748 - "POST /node/69408ae2575bc060ce15f00d HTTP/1.0" 202 Accepted 2025-12-17 22:25:48,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408909575bc060ce15e8fb HTTP/1.1" 502 150 2025-12-17 22:25:48,976 - lava_callback - ERROR - Error processing callback for node 69408909575bc060ce15e8fb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408909575bc060ce15e8fb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408909575bc060ce15e8fb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408909575bc060ce15e8fb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:48,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408907575bc060ce15e8f4 HTTP/1.1" 502 150 2025-12-17 22:25:48,978 - lava_callback - ERROR - Error processing callback for node 69408907575bc060ce15e8f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408907575bc060ce15e8f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408907575bc060ce15e8f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408907575bc060ce15e8f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:48,978 - lava_callback - INFO - Starting async job submit for node 69408920575bc060ce15e979 2025-12-17 22:25:48,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:48,981 - lava_callback - INFO - Starting async job submit for node 6940d3d1460bc8e2651126dc 2025-12-17 22:25:49,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:49,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408920575bc060ce15e979 HTTP/1.1" 502 150 2025-12-17 22:25:49,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408920575bc060ce15e979'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:49,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408920575bc060ce15e979 2025-12-17 22:25:49,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940d3d1460bc8e2651126dc HTTP/1.1" 502 150 2025-12-17 22:25:49,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940d3d1460bc8e2651126dc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:49,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940d3d1460bc8e2651126dc 2025-12-17 22:25:49,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408920575bc060ce15e979 HTTP/1.1" 502 150 2025-12-17 22:25:49,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408920575bc060ce15e979'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:49,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940d3d1460bc8e2651126dc HTTP/1.1" 502 150 2025-12-17 22:25:49,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940d3d1460bc8e2651126dc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:49,203 - lava_callback - INFO - Received callback for node 69408ae4575bc060ce15f010 INFO: 172.21.0.1:33754 - "POST /node/69408ae4575bc060ce15f010 HTTP/1.0" 202 Accepted 2025-12-17 22:25:49,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890a575bc060ce15e902 HTTP/1.1" 502 150 2025-12-17 22:25:49,232 - lava_callback - ERROR - Error processing callback for node 6940890a575bc060ce15e902: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940890a575bc060ce15e902 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940890a575bc060ce15e902 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940890a575bc060ce15e902 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:49,234 - lava_callback - INFO - Starting async job submit for node 6940892a575bc060ce15e995 2025-12-17 22:25:49,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:49,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940892a575bc060ce15e995 HTTP/1.1" 502 150 2025-12-17 22:25:49,242 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940892a575bc060ce15e995'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:49,242 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940892a575bc060ce15e995 2025-12-17 22:25:49,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940892a575bc060ce15e995 HTTP/1.1" 502 150 2025-12-17 22:25:49,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940892a575bc060ce15e995'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:49,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940891e575bc060ce15e976 2025-12-17 22:25:49,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940891e575bc060ce15e976 HTTP/1.1" 502 150 2025-12-17 22:25:49,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940891e575bc060ce15e976'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:49,834 - lava_callback - INFO - Received callback for node 69408ae6575bc060ce15f013 INFO: 172.21.0.1:37910 - "POST /node/69408ae6575bc060ce15f013 HTTP/1.0" 202 Accepted 2025-12-17 22:25:50,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890b575bc060ce15e909 HTTP/1.1" 502 150 2025-12-17 22:25:50,256 - lava_callback - ERROR - Error processing callback for node 6940890b575bc060ce15e909: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940890b575bc060ce15e909 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940890b575bc060ce15e909 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940890b575bc060ce15e909 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:50,258 - lava_callback - INFO - Starting async job submit for node 69408925575bc060ce15e988 2025-12-17 22:25:50,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:50,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408925575bc060ce15e988 HTTP/1.1" 502 150 2025-12-17 22:25:50,271 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408925575bc060ce15e988'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:50,271 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408925575bc060ce15e988 2025-12-17 22:25:50,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408925575bc060ce15e988 HTTP/1.1" 502 150 2025-12-17 22:25:50,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408925575bc060ce15e988'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:50,445 - lava_callback - INFO - Received callback for node 6940924b575bc060ce160a69 INFO: 172.21.0.1:37920 - "POST /node/6940924b575bc060ce160a69 HTTP/1.0" 202 Accepted 2025-12-17 22:25:51,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408920575bc060ce15e979 2025-12-17 22:25:51,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940d3d1460bc8e2651126dc 2025-12-17 22:25:51,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408920575bc060ce15e979 HTTP/1.1" 502 150 2025-12-17 22:25:51,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408920575bc060ce15e979'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:51,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940d3d1460bc8e2651126dc HTTP/1.1" 502 150 2025-12-17 22:25:51,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940d3d1460bc8e2651126dc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:51,243 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940892a575bc060ce15e995 2025-12-17 22:25:51,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940892a575bc060ce15e995 HTTP/1.1" 502 150 2025-12-17 22:25:51,244 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940892a575bc060ce15e995'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:51,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940891b575bc060ce15e93a 2025-12-17 22:25:51,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940891b575bc060ce15e93a HTTP/1.1" 502 150 2025-12-17 22:25:51,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940891b575bc060ce15e93a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:52,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408925575bc060ce15e988 2025-12-17 22:25:52,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890f575bc060ce15e916 2025-12-17 22:25:52,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890f575bc060ce15e916 HTTP/1.1" 502 150 2025-12-17 22:25:52,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940890f575bc060ce15e916'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:52,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694088f7575bc060ce15e8cf HTTP/1.1" 502 150 2025-12-17 22:25:52,560 - lava_callback - ERROR - Error processing callback for node 694088f7575bc060ce15e8cf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f7575bc060ce15e8cf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f7575bc060ce15e8cf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694088f7575bc060ce15e8cf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:52,562 - lava_callback - INFO - Starting async job submit for node 69408930575bc060ce15e99a 2025-12-17 22:25:52,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:52,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408930575bc060ce15e99a HTTP/1.1" 502 150 2025-12-17 22:25:52,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408930575bc060ce15e99a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:52,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408930575bc060ce15e99a 2025-12-17 22:25:52,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408930575bc060ce15e99a HTTP/1.1" 502 150 2025-12-17 22:25:52,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408930575bc060ce15e99a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:53,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940891e575bc060ce15e976 2025-12-17 22:25:53,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940891e575bc060ce15e976 HTTP/1.1" 502 150 2025-12-17 22:25:53,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940891e575bc060ce15e976'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:54,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408930575bc060ce15e99a 2025-12-17 22:25:54,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408930575bc060ce15e99a HTTP/1.1" 502 150 2025-12-17 22:25:54,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408930575bc060ce15e99a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:55,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408920575bc060ce15e979 2025-12-17 22:25:55,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940d3d1460bc8e2651126dc 2025-12-17 22:25:55,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408920575bc060ce15e979 HTTP/1.1" 502 150 2025-12-17 22:25:55,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408920575bc060ce15e979'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:55,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940d3d1460bc8e2651126dc HTTP/1.1" 502 150 2025-12-17 22:25:55,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940d3d1460bc8e2651126dc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:55,244 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940892a575bc060ce15e995 2025-12-17 22:25:55,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940892a575bc060ce15e995 HTTP/1.1" 502 150 2025-12-17 22:25:55,245 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940892a575bc060ce15e995'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:55,249 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890d575bc060ce15e910 2025-12-17 22:25:55,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890d575bc060ce15e910 HTTP/1.1" 502 150 2025-12-17 22:25:55,249 - lava_callback - ERROR - Error processing callback for node 6940890d575bc060ce15e910: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940890d575bc060ce15e910 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940890d575bc060ce15e910 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940890d575bc060ce15e910 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:55,251 - lava_callback - INFO - Starting async job submit for node 69408933575bc060ce15e99d 2025-12-17 22:25:55,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:55,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408933575bc060ce15e99d HTTP/1.1" 502 150 2025-12-17 22:25:55,289 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408933575bc060ce15e99d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:55,289 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408933575bc060ce15e99d 2025-12-17 22:25:55,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408933575bc060ce15e99d HTTP/1.1" 502 150 2025-12-17 22:25:55,289 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408933575bc060ce15e99d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:56,355 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408922575bc060ce15e97c 2025-12-17 22:25:56,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408922575bc060ce15e97c HTTP/1.1" 502 150 2025-12-17 22:25:56,356 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408922575bc060ce15e97c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37930 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:25:57,290 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408933575bc060ce15e99d 2025-12-17 22:25:57,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408933575bc060ce15e99d HTTP/1.1" 502 150 2025-12-17 22:25:57,291 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408933575bc060ce15e99d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:57,296 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890e575bc060ce15e913 2025-12-17 22:25:57,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890e575bc060ce15e913 HTTP/1.1" 502 150 2025-12-17 22:25:57,297 - lava_callback - ERROR - Error processing callback for node 6940890e575bc060ce15e913: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940890e575bc060ce15e913 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940890e575bc060ce15e913 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940890e575bc060ce15e913 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:25:57,299 - lava_callback - INFO - Starting async job submit for node 69408935575bc060ce15e9a0 2025-12-17 22:25:57,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:25:57,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408935575bc060ce15e9a0 HTTP/1.1" 502 150 2025-12-17 22:25:57,306 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408935575bc060ce15e9a0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:57,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408935575bc060ce15e9a0 2025-12-17 22:25:57,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408935575bc060ce15e9a0 HTTP/1.1" 502 150 2025-12-17 22:25:57,307 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408935575bc060ce15e9a0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:58,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408930575bc060ce15e99a 2025-12-17 22:25:58,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408930575bc060ce15e99a HTTP/1.1" 502 150 2025-12-17 22:25:58,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408930575bc060ce15e99a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:59,307 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408935575bc060ce15e9a0 2025-12-17 22:25:59,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408935575bc060ce15e9a0 HTTP/1.1" 502 150 2025-12-17 22:25:59,308 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408935575bc060ce15e9a0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:25:59,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940891b575bc060ce15e93a 2025-12-17 22:25:59,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940891b575bc060ce15e93a HTTP/1.1" 502 150 2025-12-17 22:25:59,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940891b575bc060ce15e93a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:01,291 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408933575bc060ce15e99d 2025-12-17 22:26:01,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408914575bc060ce15e91f 2025-12-17 22:26:01,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408910575bc060ce15e919 2025-12-17 22:26:01,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408914575bc060ce15e91f HTTP/1.1" 502 150 2025-12-17 22:26:01,460 - lava_callback - ERROR - Error processing callback for node 69408914575bc060ce15e91f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408914575bc060ce15e91f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408914575bc060ce15e91f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408914575bc060ce15e91f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:01,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408910575bc060ce15e919 HTTP/1.1" 502 150 2025-12-17 22:26:01,462 - lava_callback - ERROR - Error processing callback for node 69408910575bc060ce15e919: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408910575bc060ce15e919 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408910575bc060ce15e919 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408910575bc060ce15e919 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:01,461 - lava_callback - INFO - Starting async job submit for node 6940893b575bc060ce15e9a9 2025-12-17 22:26:01,464 - lava_callback - INFO - Starting async job submit for node 6940893d575bc060ce15e9ac 2025-12-17 22:26:01,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:01,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:01,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940893b575bc060ce15e9a9 HTTP/1.1" 502 150 2025-12-17 22:26:01,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940893b575bc060ce15e9a9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:01,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940893b575bc060ce15e9a9 2025-12-17 22:26:01,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940893b575bc060ce15e9a9 HTTP/1.1" 502 150 2025-12-17 22:26:01,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940893b575bc060ce15e9a9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:01,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940893d575bc060ce15e9ac HTTP/1.1" 502 150 2025-12-17 22:26:01,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940893d575bc060ce15e9ac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:01,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940893d575bc060ce15e9ac 2025-12-17 22:26:01,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940893d575bc060ce15e9ac HTTP/1.1" 502 150 2025-12-17 22:26:01,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940893d575bc060ce15e9ac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:01,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940891e575bc060ce15e976 2025-12-17 22:26:01,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940891e575bc060ce15e976 HTTP/1.1" 502 150 2025-12-17 22:26:01,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940891e575bc060ce15e976'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43536 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:26:03,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408920575bc060ce15e979 2025-12-17 22:26:03,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940d3d1460bc8e2651126dc 2025-12-17 22:26:03,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940d3d1460bc8e2651126dc HTTP/1.1" 502 150 2025-12-17 22:26:03,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940d3d1460bc8e2651126dc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:03,246 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940892a575bc060ce15e995 2025-12-17 22:26:03,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940892a575bc060ce15e995 HTTP/1.1" 502 150 2025-12-17 22:26:03,247 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940892a575bc060ce15e995'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:03,309 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408935575bc060ce15e9a0 2025-12-17 22:26:03,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408935575bc060ce15e9a0 HTTP/1.1" 502 150 2025-12-17 22:26:03,310 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408935575bc060ce15e9a0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:03,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408917575bc060ce15e922 2025-12-17 22:26:03,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408917575bc060ce15e922 HTTP/1.1" 502 150 2025-12-17 22:26:03,418 - lava_callback - ERROR - Error processing callback for node 69408917575bc060ce15e922: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408917575bc060ce15e922 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408917575bc060ce15e922 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408917575bc060ce15e922 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:03,420 - lava_callback - INFO - Starting async job submit for node 69408940575bc060ce15e9af 2025-12-17 22:26:03,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:03,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408940575bc060ce15e9af HTTP/1.1" 502 150 2025-12-17 22:26:03,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408940575bc060ce15e9af'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:03,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408940575bc060ce15e9af 2025-12-17 22:26:03,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408940575bc060ce15e9af HTTP/1.1" 502 150 2025-12-17 22:26:03,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408940575bc060ce15e9af'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:03,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940893b575bc060ce15e9a9 2025-12-17 22:26:03,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940893b575bc060ce15e9a9 HTTP/1.1" 502 150 2025-12-17 22:26:03,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940893b575bc060ce15e9a9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:03,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940893d575bc060ce15e9ac 2025-12-17 22:26:03,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940893d575bc060ce15e9ac HTTP/1.1" 502 150 2025-12-17 22:26:03,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940893d575bc060ce15e9ac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:04,179 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408919575bc060ce15e925 2025-12-17 22:26:04,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408919575bc060ce15e925 HTTP/1.1" 502 150 2025-12-17 22:26:04,181 - lava_callback - ERROR - Error processing callback for node 69408919575bc060ce15e925: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408919575bc060ce15e925 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408919575bc060ce15e925 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408919575bc060ce15e925 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:04,183 - lava_callback - INFO - Starting async job submit for node 6940894c575bc060ce15e9d0 2025-12-17 22:26:04,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:04,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940894c575bc060ce15e9d0 HTTP/1.1" 502 150 2025-12-17 22:26:04,190 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940894c575bc060ce15e9d0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:04,190 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940894c575bc060ce15e9d0 2025-12-17 22:26:04,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940894c575bc060ce15e9d0 HTTP/1.1" 502 150 2025-12-17 22:26:04,191 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940894c575bc060ce15e9d0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:04,621 - lava_callback - INFO - Received callback for node 69408ae8575bc060ce15f018 INFO: 172.21.0.1:43538 - "POST /node/69408ae8575bc060ce15f018 HTTP/1.0" 202 Accepted 2025-12-17 22:26:05,271 - lava_callback - INFO - Received callback for node 69408aea575bc060ce15f01f INFO: 172.21.0.1:43546 - "POST /node/69408aea575bc060ce15f01f HTTP/1.0" 202 Accepted 2025-12-17 22:26:05,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408940575bc060ce15e9af 2025-12-17 22:26:05,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408940575bc060ce15e9af HTTP/1.1" 502 150 2025-12-17 22:26:05,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408940575bc060ce15e9af'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:05,915 - lava_callback - INFO - Received callback for node 69408aec575bc060ce15f024 INFO: 172.21.0.1:43560 - "POST /node/69408aec575bc060ce15f024 HTTP/1.0" 202 Accepted 2025-12-17 22:26:06,192 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940894c575bc060ce15e9d0 2025-12-17 22:26:06,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940894c575bc060ce15e9d0 HTTP/1.1" 502 150 2025-12-17 22:26:06,193 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940894c575bc060ce15e9d0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43572 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:26:06,565 - lava_callback - INFO - Received callback for node 69408aee575bc060ce15f027 INFO: 172.21.0.1:43574 - "POST /node/69408aee575bc060ce15f027 HTTP/1.0" 202 Accepted 2025-12-17 22:26:06,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408930575bc060ce15e99a 2025-12-17 22:26:06,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408930575bc060ce15e99a HTTP/1.1" 502 150 2025-12-17 22:26:06,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408930575bc060ce15e99a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:07,227 - lava_callback - INFO - Received callback for node 69408af0575bc060ce15f02a INFO: 172.21.0.1:43576 - "POST /node/69408af0575bc060ce15f02a HTTP/1.0" 202 Accepted 2025-12-17 22:26:07,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940893b575bc060ce15e9a9 2025-12-17 22:26:07,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940893b575bc060ce15e9a9 HTTP/1.1" 502 150 2025-12-17 22:26:07,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940893b575bc060ce15e9a9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:07,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940893d575bc060ce15e9ac 2025-12-17 22:26:07,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940893d575bc060ce15e9ac HTTP/1.1" 502 150 2025-12-17 22:26:07,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940893d575bc060ce15e9ac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:07,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408925575bc060ce15e988 HTTP/1.1" 502 150 2025-12-17 22:26:07,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408925575bc060ce15e988'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:07,894 - lava_callback - INFO - Received callback for node 69408af2575bc060ce15f02d INFO: 172.21.0.1:43582 - "POST /node/69408af2575bc060ce15f02d HTTP/1.0" 202 Accepted 2025-12-17 22:26:08,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940890f575bc060ce15e916 2025-12-17 22:26:08,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940890f575bc060ce15e916 HTTP/1.1" 502 150 2025-12-17 22:26:08,506 - lava_callback - ERROR - Error processing callback for node 6940890f575bc060ce15e916: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940890f575bc060ce15e916 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940890f575bc060ce15e916 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940890f575bc060ce15e916 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:08,508 - lava_callback - INFO - Starting async job submit for node 69408942575bc060ce15e9b2 2025-12-17 22:26:08,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:08,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408942575bc060ce15e9b2 HTTP/1.1" 502 150 2025-12-17 22:26:08,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408942575bc060ce15e9b2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:08,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408942575bc060ce15e9b2 2025-12-17 22:26:08,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408942575bc060ce15e9b2 HTTP/1.1" 502 150 2025-12-17 22:26:08,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408942575bc060ce15e9b2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:08,541 - lava_callback - INFO - Received callback for node 69408af4575bc060ce15f030 INFO: 172.21.0.1:43596 - "POST /node/69408af4575bc060ce15f030 HTTP/1.0" 202 Accepted 2025-12-17 22:26:09,207 - lava_callback - INFO - Received callback for node 69408af5575bc060ce15f033 INFO: 172.21.0.1:43610 - "POST /node/69408af5575bc060ce15f033 HTTP/1.0" 202 Accepted 2025-12-17 22:26:09,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408940575bc060ce15e9af 2025-12-17 22:26:09,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408940575bc060ce15e9af HTTP/1.1" 502 150 2025-12-17 22:26:09,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408940575bc060ce15e9af'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:09,856 - lava_callback - INFO - Received callback for node 69408af8575bc060ce15f036 INFO: 172.21.0.1:56618 - "POST /node/69408af8575bc060ce15f036 HTTP/1.0" 202 Accepted 2025-12-17 22:26:10,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940894c575bc060ce15e9d0 2025-12-17 22:26:10,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940894c575bc060ce15e9d0 HTTP/1.1" 502 150 2025-12-17 22:26:10,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940894c575bc060ce15e9d0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:10,482 - lava_callback - INFO - Received callback for node 69408afa575bc060ce15f039 INFO: 172.21.0.1:56624 - "POST /node/69408afa575bc060ce15f039 HTTP/1.0" 202 Accepted 2025-12-17 22:26:10,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408942575bc060ce15e9b2 2025-12-17 22:26:10,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408942575bc060ce15e9b2 HTTP/1.1" 502 150 2025-12-17 22:26:10,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408942575bc060ce15e9b2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:11,310 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408935575bc060ce15e9a0 2025-12-17 22:26:11,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408935575bc060ce15e9a0 HTTP/1.1" 502 150 2025-12-17 22:26:11,311 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408935575bc060ce15e9a0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:11,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408925575bc060ce15e988 2025-12-17 22:26:11,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408925575bc060ce15e988 HTTP/1.1" 502 150 2025-12-17 22:26:11,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408925575bc060ce15e988'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:12,356 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408922575bc060ce15e97c 2025-12-17 22:26:12,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408922575bc060ce15e97c HTTP/1.1" 502 150 2025-12-17 22:26:12,358 - lava_callback - ERROR - Error processing callback for node 69408922575bc060ce15e97c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408922575bc060ce15e97c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408922575bc060ce15e97c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408922575bc060ce15e97c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:12,360 - lava_callback - INFO - Starting async job submit for node 69408946575bc060ce15e9c0 2025-12-17 22:26:12,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:12,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408946575bc060ce15e9c0 HTTP/1.1" 502 150 2025-12-17 22:26:12,389 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408946575bc060ce15e9c0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:12,390 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408946575bc060ce15e9c0 2025-12-17 22:26:12,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408946575bc060ce15e9c0 HTTP/1.1" 502 150 2025-12-17 22:26:12,390 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408946575bc060ce15e9c0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:13,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408912575bc060ce15e91c HTTP/1.1" 502 150 2025-12-17 22:26:13,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408912575bc060ce15e91c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:13,042 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408912575bc060ce15e91c 2025-12-17 22:26:14,390 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408946575bc060ce15e9c0 2025-12-17 22:26:14,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408946575bc060ce15e9c0 HTTP/1.1" 502 150 2025-12-17 22:26:14,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408946575bc060ce15e9c0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:14,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408942575bc060ce15e9b2 2025-12-17 22:26:14,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408942575bc060ce15e9b2 HTTP/1.1" 502 150 2025-12-17 22:26:14,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408942575bc060ce15e9b2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56634 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:26:15,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940891b575bc060ce15e93a 2025-12-17 22:26:15,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940891b575bc060ce15e93a HTTP/1.1" 502 150 2025-12-17 22:26:15,423 - lava_callback - ERROR - Error processing callback for node 6940891b575bc060ce15e93a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940891b575bc060ce15e93a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940891b575bc060ce15e93a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940891b575bc060ce15e93a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:15,424 - lava_callback - INFO - Starting async job submit for node 69408947575bc060ce15e9c3 2025-12-17 22:26:15,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:15,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408947575bc060ce15e9c3 HTTP/1.1" 502 150 2025-12-17 22:26:15,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408947575bc060ce15e9c3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:15,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408947575bc060ce15e9c3 2025-12-17 22:26:15,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408947575bc060ce15e9c3 HTTP/1.1" 502 150 2025-12-17 22:26:15,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408947575bc060ce15e9c3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:15,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940893b575bc060ce15e9a9 2025-12-17 22:26:15,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940893b575bc060ce15e9a9 HTTP/1.1" 502 150 2025-12-17 22:26:15,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940893b575bc060ce15e9a9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:15,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940893d575bc060ce15e9ac 2025-12-17 22:26:15,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940893d575bc060ce15e9ac HTTP/1.1" 502 150 2025-12-17 22:26:15,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940893d575bc060ce15e9ac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:16,078 - lava_callback - INFO - Received callback for node 69409253575bc060ce160a75 INFO: 172.21.0.1:56636 - "POST /node/69409253575bc060ce160a75 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:56646 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:26:16,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408933575bc060ce15e99d HTTP/1.1" 502 150 2025-12-17 22:26:16,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408933575bc060ce15e99d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:17,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408947575bc060ce15e9c3 2025-12-17 22:26:17,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408940575bc060ce15e9af 2025-12-17 22:26:17,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408947575bc060ce15e9c3 HTTP/1.1" 502 150 2025-12-17 22:26:17,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408947575bc060ce15e9c3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:17,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408940575bc060ce15e9af HTTP/1.1" 502 150 2025-12-17 22:26:17,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408940575bc060ce15e9af'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:17,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940891e575bc060ce15e976 2025-12-17 22:26:17,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940891e575bc060ce15e976 HTTP/1.1" 502 150 2025-12-17 22:26:17,712 - lava_callback - ERROR - Error processing callback for node 6940891e575bc060ce15e976: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940891e575bc060ce15e976 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940891e575bc060ce15e976 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940891e575bc060ce15e976 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:17,714 - lava_callback - INFO - Starting async job submit for node 69408718575bc060ce15e273 2025-12-17 22:26:17,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:17,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408718575bc060ce15e273 HTTP/1.1" 502 150 2025-12-17 22:26:17,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408718575bc060ce15e273'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:17,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408718575bc060ce15e273 2025-12-17 22:26:17,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408718575bc060ce15e273 HTTP/1.1" 502 150 2025-12-17 22:26:17,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408718575bc060ce15e273'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:18,195 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940894c575bc060ce15e9d0 2025-12-17 22:26:18,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940894c575bc060ce15e9d0 HTTP/1.1" 502 150 2025-12-17 22:26:18,204 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940894c575bc060ce15e9d0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:18,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408946575bc060ce15e9c0 2025-12-17 22:26:18,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408946575bc060ce15e9c0 HTTP/1.1" 502 150 2025-12-17 22:26:18,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408946575bc060ce15e9c0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:18,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408920575bc060ce15e979 HTTP/1.1" 502 150 2025-12-17 22:26:18,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408920575bc060ce15e979'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:19,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940d3d1460bc8e2651126dc 2025-12-17 22:26:19,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940d3d1460bc8e2651126dc HTTP/1.1" 502 150 2025-12-17 22:26:19,017 - lava_callback - ERROR - Error processing callback for node 6940d3d1460bc8e2651126dc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940d3d1460bc8e2651126dc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940d3d1460bc8e2651126dc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940d3d1460bc8e2651126dc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:19,020 - lava_callback - INFO - Starting async job submit for node 69408943575bc060ce15e9b8 2025-12-17 22:26:19,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:19,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408943575bc060ce15e9b8 HTTP/1.1" 502 150 2025-12-17 22:26:19,052 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408943575bc060ce15e9b8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:19,052 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408943575bc060ce15e9b8 2025-12-17 22:26:19,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408943575bc060ce15e9b8 HTTP/1.1" 502 150 2025-12-17 22:26:19,053 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408943575bc060ce15e9b8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:19,247 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940892a575bc060ce15e995 2025-12-17 22:26:19,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940892a575bc060ce15e995 HTTP/1.1" 502 150 2025-12-17 22:26:19,248 - lava_callback - ERROR - Error processing callback for node 6940892a575bc060ce15e995: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940892a575bc060ce15e995 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940892a575bc060ce15e995 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940892a575bc060ce15e995 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:19,251 - lava_callback - INFO - Starting async job submit for node 69408948575bc060ce15e9c6 2025-12-17 22:26:19,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:19,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408948575bc060ce15e9c6 HTTP/1.1" 502 150 2025-12-17 22:26:19,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408948575bc060ce15e9c6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:19,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408948575bc060ce15e9c6 2025-12-17 22:26:19,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408948575bc060ce15e9c6 HTTP/1.1" 502 150 2025-12-17 22:26:19,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408948575bc060ce15e9c6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:19,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408925575bc060ce15e988 2025-12-17 22:26:19,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408925575bc060ce15e988 HTTP/1.1" 502 150 2025-12-17 22:26:19,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408925575bc060ce15e988'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:19,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408718575bc060ce15e273 2025-12-17 22:26:19,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408718575bc060ce15e273 HTTP/1.1" 502 150 2025-12-17 22:26:19,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408718575bc060ce15e273'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:21,053 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408943575bc060ce15e9b8 2025-12-17 22:26:21,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408943575bc060ce15e9b8 HTTP/1.1" 502 150 2025-12-17 22:26:21,054 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408943575bc060ce15e9b8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:21,267 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408948575bc060ce15e9c6 2025-12-17 22:26:21,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408948575bc060ce15e9c6 HTTP/1.1" 502 150 2025-12-17 22:26:21,268 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408948575bc060ce15e9c6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:21,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408947575bc060ce15e9c3 2025-12-17 22:26:21,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408947575bc060ce15e9c3 HTTP/1.1" 502 150 2025-12-17 22:26:21,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408947575bc060ce15e9c3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:22,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408942575bc060ce15e9b2 2025-12-17 22:26:22,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408942575bc060ce15e9b2 HTTP/1.1" 502 150 2025-12-17 22:26:22,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408942575bc060ce15e9b2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:22,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408930575bc060ce15e99a 2025-12-17 22:26:22,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408930575bc060ce15e99a HTTP/1.1" 502 150 2025-12-17 22:26:22,575 - lava_callback - ERROR - Error processing callback for node 69408930575bc060ce15e99a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408930575bc060ce15e99a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408930575bc060ce15e99a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408930575bc060ce15e99a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:22,577 - lava_callback - INFO - Starting async job submit for node 69408952575bc060ce15e9d9 2025-12-17 22:26:22,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:22,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408952575bc060ce15e9d9 HTTP/1.1" 502 150 2025-12-17 22:26:22,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408952575bc060ce15e9d9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:22,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408952575bc060ce15e9d9 2025-12-17 22:26:22,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408952575bc060ce15e9d9 HTTP/1.1" 502 150 2025-12-17 22:26:22,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408952575bc060ce15e9d9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:22,595 - lava_callback - INFO - Received callback for node 694086cb575bc060ce15e190 INFO: 172.21.0.1:35802 - "POST /node/694086cb575bc060ce15e190 HTTP/1.0" 202 Accepted 2025-12-17 22:26:23,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408718575bc060ce15e273 2025-12-17 22:26:23,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408718575bc060ce15e273 HTTP/1.1" 502 150 2025-12-17 22:26:23,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408718575bc060ce15e273'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:24,033 - lava_callback - INFO - Received callback for node 69408be9575bc060ce15f39e INFO: 172.21.0.1:35806 - "POST /node/69408be9575bc060ce15f39e HTTP/1.0" 202 Accepted 2025-12-17 22:26:24,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408952575bc060ce15e9d9 2025-12-17 22:26:24,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408952575bc060ce15e9d9 HTTP/1.1" 502 150 2025-12-17 22:26:24,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408952575bc060ce15e9d9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:24,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408933575bc060ce15e99d 2025-12-17 22:26:24,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408933575bc060ce15e99d HTTP/1.1" 502 150 2025-12-17 22:26:24,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408933575bc060ce15e99d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:24,640 - lava_callback - INFO - Received callback for node 69408afc575bc060ce15f03c INFO: 172.21.0.1:35808 - "POST /node/69408afc575bc060ce15f03c HTTP/1.0" 202 Accepted 2025-12-17 22:26:25,054 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408943575bc060ce15e9b8 2025-12-17 22:26:25,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408943575bc060ce15e9b8 HTTP/1.1" 502 150 2025-12-17 22:26:25,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408943575bc060ce15e9b8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:25,064 - lava_callback - INFO - Received callback for node 6940879d575bc060ce15e427 INFO: 172.21.0.1:35822 - "POST /node/6940879d575bc060ce15e427 HTTP/1.0" 202 Accepted 2025-12-17 22:26:25,268 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408948575bc060ce15e9c6 2025-12-17 22:26:25,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408948575bc060ce15e9c6 HTTP/1.1" 502 150 2025-12-17 22:26:25,269 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408948575bc060ce15e9c6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:25,284 - lava_callback - INFO - Received callback for node 69408afe575bc060ce15f045 INFO: 172.21.0.1:35826 - "POST /node/69408afe575bc060ce15f045 HTTP/1.0" 202 Accepted 2025-12-17 22:26:25,929 - lava_callback - INFO - Received callback for node 69408b02575bc060ce15f04e INFO: 172.21.0.1:35832 - "POST /node/69408b02575bc060ce15f04e HTTP/1.0" 202 Accepted 2025-12-17 22:26:26,393 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408946575bc060ce15e9c0 2025-12-17 22:26:26,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408946575bc060ce15e9c0 HTTP/1.1" 502 150 2025-12-17 22:26:26,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408946575bc060ce15e9c0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35834 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:26:26,575 - lava_callback - INFO - Received callback for node 69408b06575bc060ce15f054 INFO: 172.21.0.1:35842 - "POST /node/69408b06575bc060ce15f054 HTTP/1.0" 202 Accepted 2025-12-17 22:26:27,220 - lava_callback - INFO - Received callback for node 69408b08575bc060ce15f057 INFO: 172.21.0.1:35858 - "POST /node/69408b08575bc060ce15f057 HTTP/1.0" 202 Accepted 2025-12-17 22:26:27,311 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408935575bc060ce15e9a0 2025-12-17 22:26:27,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408935575bc060ce15e9a0 HTTP/1.1" 502 150 2025-12-17 22:26:27,313 - lava_callback - ERROR - Error processing callback for node 69408935575bc060ce15e9a0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408935575bc060ce15e9a0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408935575bc060ce15e9a0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408935575bc060ce15e9a0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:27,315 - lava_callback - INFO - Starting async job submit for node 69408954575bc060ce15e9dc 2025-12-17 22:26:27,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:27,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408954575bc060ce15e9dc HTTP/1.1" 502 150 2025-12-17 22:26:27,321 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408954575bc060ce15e9dc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:27,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408954575bc060ce15e9dc 2025-12-17 22:26:27,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408954575bc060ce15e9dc HTTP/1.1" 502 150 2025-12-17 22:26:27,322 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408954575bc060ce15e9dc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:27,862 - lava_callback - INFO - Received callback for node 69408b0a575bc060ce15f05a INFO: 172.21.0.1:35862 - "POST /node/69408b0a575bc060ce15f05a HTTP/1.0" 202 Accepted 2025-12-17 22:26:28,504 - lava_callback - INFO - Received callback for node 69408b0d575bc060ce15f07c INFO: 172.21.0.1:35870 - "POST /node/69408b0d575bc060ce15f07c HTTP/1.0" 202 Accepted 2025-12-17 22:26:28,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408952575bc060ce15e9d9 2025-12-17 22:26:28,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408952575bc060ce15e9d9 HTTP/1.1" 502 150 2025-12-17 22:26:28,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408952575bc060ce15e9d9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:29,322 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408954575bc060ce15e9dc 2025-12-17 22:26:29,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408954575bc060ce15e9dc HTTP/1.1" 502 150 2025-12-17 22:26:29,323 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408954575bc060ce15e9dc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:29,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408947575bc060ce15e9c3 2025-12-17 22:26:29,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408947575bc060ce15e9c3 HTTP/1.1" 502 150 2025-12-17 22:26:29,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408947575bc060ce15e9c3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:30,450 - lava_callback - INFO - Received callback for node 6940e7cf460bc8e2651142e7 INFO: 172.21.0.1:46496 - "POST /node/6940e7cf460bc8e2651142e7 HTTP/1.0" 202 Accepted 2025-12-17 22:26:31,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940893b575bc060ce15e9a9 2025-12-17 22:26:31,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940893d575bc060ce15e9ac 2025-12-17 22:26:31,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940893b575bc060ce15e9a9 HTTP/1.1" 502 150 2025-12-17 22:26:31,481 - lava_callback - ERROR - Error processing callback for node 6940893b575bc060ce15e9a9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940893b575bc060ce15e9a9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940893b575bc060ce15e9a9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940893b575bc060ce15e9a9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:31,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940893d575bc060ce15e9ac HTTP/1.1" 502 150 2025-12-17 22:26:31,483 - lava_callback - ERROR - Error processing callback for node 6940893d575bc060ce15e9ac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940893d575bc060ce15e9ac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940893d575bc060ce15e9ac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940893d575bc060ce15e9ac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:31,485 - lava_callback - INFO - Starting async job submit for node 69408951575bc060ce15e9d6 2025-12-17 22:26:31,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:31,487 - lava_callback - INFO - Starting async job submit for node 6940894f575bc060ce15e9d3 2025-12-17 22:26:31,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:31,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940894f575bc060ce15e9d3 HTTP/1.1" 502 150 2025-12-17 22:26:31,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940894f575bc060ce15e9d3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:31,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940894f575bc060ce15e9d3 2025-12-17 22:26:31,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408951575bc060ce15e9d6 HTTP/1.1" 502 150 2025-12-17 22:26:31,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408951575bc060ce15e9d6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:31,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408951575bc060ce15e9d6 2025-12-17 22:26:31,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940894f575bc060ce15e9d3 HTTP/1.1" 502 150 2025-12-17 22:26:31,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940894f575bc060ce15e9d3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:31,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408951575bc060ce15e9d6 HTTP/1.1" 502 150 2025-12-17 22:26:31,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408951575bc060ce15e9d6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:31,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408718575bc060ce15e273 2025-12-17 22:26:31,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408718575bc060ce15e273 HTTP/1.1" 502 150 2025-12-17 22:26:31,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408718575bc060ce15e273'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:33,056 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408943575bc060ce15e9b8 2025-12-17 22:26:33,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408943575bc060ce15e9b8 HTTP/1.1" 502 150 2025-12-17 22:26:33,057 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408943575bc060ce15e9b8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:33,269 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408948575bc060ce15e9c6 2025-12-17 22:26:33,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408948575bc060ce15e9c6 HTTP/1.1" 502 150 2025-12-17 22:26:33,270 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408948575bc060ce15e9c6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:33,323 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408954575bc060ce15e9dc 2025-12-17 22:26:33,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408954575bc060ce15e9dc HTTP/1.1" 502 150 2025-12-17 22:26:33,324 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408954575bc060ce15e9dc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:33,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408940575bc060ce15e9af 2025-12-17 22:26:33,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408940575bc060ce15e9af HTTP/1.1" 502 150 2025-12-17 22:26:33,435 - lava_callback - ERROR - Error processing callback for node 69408940575bc060ce15e9af: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408940575bc060ce15e9af (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408940575bc060ce15e9af (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408940575bc060ce15e9af (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:33,437 - lava_callback - INFO - Starting async job submit for node 69408958575bc060ce15e9e8 2025-12-17 22:26:33,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:33,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408958575bc060ce15e9e8 HTTP/1.1" 502 150 2025-12-17 22:26:33,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408958575bc060ce15e9e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:33,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408958575bc060ce15e9e8 2025-12-17 22:26:33,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408958575bc060ce15e9e8 HTTP/1.1" 502 150 2025-12-17 22:26:33,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408958575bc060ce15e9e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:33,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940894f575bc060ce15e9d3 2025-12-17 22:26:33,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408951575bc060ce15e9d6 2025-12-17 22:26:33,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940894f575bc060ce15e9d3 HTTP/1.1" 502 150 2025-12-17 22:26:33,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940894f575bc060ce15e9d3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:33,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408951575bc060ce15e9d6 HTTP/1.1" 502 150 2025-12-17 22:26:33,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408951575bc060ce15e9d6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:34,204 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940894c575bc060ce15e9d0 2025-12-17 22:26:34,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940894c575bc060ce15e9d0 HTTP/1.1" 502 150 2025-12-17 22:26:34,205 - lava_callback - ERROR - Error processing callback for node 6940894c575bc060ce15e9d0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940894c575bc060ce15e9d0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940894c575bc060ce15e9d0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940894c575bc060ce15e9d0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:34,207 - lava_callback - INFO - Starting async job submit for node 6940895a575bc060ce15ea16 2025-12-17 22:26:34,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:34,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895a575bc060ce15ea16 HTTP/1.1" 502 150 2025-12-17 22:26:34,217 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895a575bc060ce15ea16'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:34,217 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895a575bc060ce15ea16 2025-12-17 22:26:34,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895a575bc060ce15ea16 HTTP/1.1" 502 150 2025-12-17 22:26:34,218 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895a575bc060ce15ea16'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:34,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408920575bc060ce15e979 2025-12-17 22:26:34,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408920575bc060ce15e979 HTTP/1.1" 502 150 2025-12-17 22:26:34,421 - lava_callback - ERROR - Error processing callback for node 69408920575bc060ce15e979: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408920575bc060ce15e979 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408920575bc060ce15e979 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408920575bc060ce15e979 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:34,423 - lava_callback - INFO - Starting async job submit for node 6940895c575bc060ce15ea4b 2025-12-17 22:26:34,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:34,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895c575bc060ce15ea4b HTTP/1.1" 502 150 2025-12-17 22:26:34,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895c575bc060ce15ea4b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:34,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895c575bc060ce15ea4b 2025-12-17 22:26:34,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895c575bc060ce15ea4b HTTP/1.1" 502 150 2025-12-17 22:26:34,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895c575bc060ce15ea4b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:35,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408958575bc060ce15e9e8 2025-12-17 22:26:35,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408958575bc060ce15e9e8 HTTP/1.1" 502 150 2025-12-17 22:26:35,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408958575bc060ce15e9e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:35,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408925575bc060ce15e988 2025-12-17 22:26:35,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408925575bc060ce15e988 HTTP/1.1" 502 150 2025-12-17 22:26:35,669 - lava_callback - ERROR - Error processing callback for node 69408925575bc060ce15e988: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408925575bc060ce15e988 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408925575bc060ce15e988 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408925575bc060ce15e988 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:35,671 - lava_callback - INFO - Starting async job submit for node 6940895e575bc060ce15ea58 2025-12-17 22:26:35,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:35,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895e575bc060ce15ea58 HTTP/1.1" 502 150 2025-12-17 22:26:35,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895e575bc060ce15ea58'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:35,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895e575bc060ce15ea58 2025-12-17 22:26:35,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895e575bc060ce15ea58 HTTP/1.1" 502 150 2025-12-17 22:26:35,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895e575bc060ce15ea58'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:36,218 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895a575bc060ce15ea16 2025-12-17 22:26:36,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895a575bc060ce15ea16 HTTP/1.1" 502 150 2025-12-17 22:26:36,220 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895a575bc060ce15ea16'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:36,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895c575bc060ce15ea4b 2025-12-17 22:26:36,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895c575bc060ce15ea4b HTTP/1.1" 502 150 2025-12-17 22:26:36,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895c575bc060ce15ea4b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46498 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:26:36,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408952575bc060ce15e9d9 2025-12-17 22:26:36,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408952575bc060ce15e9d9 HTTP/1.1" 502 150 2025-12-17 22:26:36,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408952575bc060ce15e9d9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:37,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940894f575bc060ce15e9d3 2025-12-17 22:26:37,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408951575bc060ce15e9d6 2025-12-17 22:26:37,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940894f575bc060ce15e9d3 HTTP/1.1" 502 150 2025-12-17 22:26:37,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940894f575bc060ce15e9d3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:37,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408951575bc060ce15e9d6 HTTP/1.1" 502 150 2025-12-17 22:26:37,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408951575bc060ce15e9d6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:37,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895e575bc060ce15ea58 2025-12-17 22:26:37,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895e575bc060ce15ea58 HTTP/1.1" 502 150 2025-12-17 22:26:37,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895e575bc060ce15ea58'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:38,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408942575bc060ce15e9b2 2025-12-17 22:26:38,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408942575bc060ce15e9b2 HTTP/1.1" 502 150 2025-12-17 22:26:38,521 - lava_callback - ERROR - Error processing callback for node 69408942575bc060ce15e9b2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408942575bc060ce15e9b2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408942575bc060ce15e9b2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408942575bc060ce15e9b2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:38,523 - lava_callback - INFO - Starting async job submit for node 6940895f575bc060ce15ea5b 2025-12-17 22:26:38,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:38,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895f575bc060ce15ea5b HTTP/1.1" 502 150 2025-12-17 22:26:38,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895f575bc060ce15ea5b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:38,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895f575bc060ce15ea5b 2025-12-17 22:26:38,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895f575bc060ce15ea5b HTTP/1.1" 502 150 2025-12-17 22:26:38,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895f575bc060ce15ea5b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:39,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408958575bc060ce15e9e8 2025-12-17 22:26:39,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408958575bc060ce15e9e8 HTTP/1.1" 502 150 2025-12-17 22:26:39,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408958575bc060ce15e9e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:40,220 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895a575bc060ce15ea16 2025-12-17 22:26:40,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895a575bc060ce15ea16 HTTP/1.1" 502 150 2025-12-17 22:26:40,221 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895a575bc060ce15ea16'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:40,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895c575bc060ce15ea4b 2025-12-17 22:26:40,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895c575bc060ce15ea4b HTTP/1.1" 502 150 2025-12-17 22:26:40,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895c575bc060ce15ea4b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:40,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895f575bc060ce15ea5b 2025-12-17 22:26:40,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895f575bc060ce15ea5b HTTP/1.1" 502 150 2025-12-17 22:26:40,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895f575bc060ce15ea5b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:40,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408933575bc060ce15e99d 2025-12-17 22:26:40,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408933575bc060ce15e99d HTTP/1.1" 502 150 2025-12-17 22:26:40,634 - lava_callback - ERROR - Error processing callback for node 69408933575bc060ce15e99d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408933575bc060ce15e99d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408933575bc060ce15e99d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408933575bc060ce15e99d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:40,636 - lava_callback - INFO - Starting async job submit for node 69408960575bc060ce15ea5e 2025-12-17 22:26:40,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:40,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408960575bc060ce15ea5e HTTP/1.1" 502 150 2025-12-17 22:26:40,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408960575bc060ce15ea5e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:40,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408960575bc060ce15ea5e 2025-12-17 22:26:40,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408960575bc060ce15ea5e HTTP/1.1" 502 150 2025-12-17 22:26:40,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408960575bc060ce15ea5e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:41,324 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408954575bc060ce15e9dc 2025-12-17 22:26:41,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408954575bc060ce15e9dc HTTP/1.1" 502 150 2025-12-17 22:26:41,326 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408954575bc060ce15e9dc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:41,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895e575bc060ce15ea58 2025-12-17 22:26:41,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895e575bc060ce15ea58 HTTP/1.1" 502 150 2025-12-17 22:26:41,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895e575bc060ce15ea58'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:42,395 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408946575bc060ce15e9c0 2025-12-17 22:26:42,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408946575bc060ce15e9c0 HTTP/1.1" 502 150 2025-12-17 22:26:42,396 - lava_callback - ERROR - Error processing callback for node 69408946575bc060ce15e9c0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408946575bc060ce15e9c0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408946575bc060ce15e9c0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408946575bc060ce15e9c0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:42,398 - lava_callback - INFO - Starting async job submit for node 69408964575bc060ce15ea67 2025-12-17 22:26:42,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:42,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408964575bc060ce15ea67 HTTP/1.1" 502 150 2025-12-17 22:26:42,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408964575bc060ce15ea67'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:42,405 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408964575bc060ce15ea67 2025-12-17 22:26:42,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408964575bc060ce15ea67 HTTP/1.1" 502 150 2025-12-17 22:26:42,405 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408964575bc060ce15ea67'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:42,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408960575bc060ce15ea5e 2025-12-17 22:26:42,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408960575bc060ce15ea5e HTTP/1.1" 502 150 2025-12-17 22:26:42,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408960575bc060ce15ea5e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:43,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408912575bc060ce15e91c HTTP/1.1" 502 150 2025-12-17 22:26:43,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408912575bc060ce15e91c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:44,405 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408964575bc060ce15ea67 2025-12-17 22:26:44,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895f575bc060ce15ea5b 2025-12-17 22:26:44,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895f575bc060ce15ea5b HTTP/1.1" 502 150 2025-12-17 22:26:44,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895f575bc060ce15ea5b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:44,653 - lava_callback - INFO - Received callback for node 69408b10575bc060ce15f0a5 INFO: 172.21.0.1:39052 - "POST /node/69408b10575bc060ce15f0a5 HTTP/1.0" 202 Accepted 2025-12-17 22:26:45,297 - lava_callback - INFO - Received callback for node 69408b14575bc060ce15f0b6 INFO: 172.21.0.1:39054 - "POST /node/69408b14575bc060ce15f0b6 HTTP/1.0" 202 Accepted 2025-12-17 22:26:45,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408947575bc060ce15e9c3 2025-12-17 22:26:45,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408947575bc060ce15e9c3 HTTP/1.1" 502 150 2025-12-17 22:26:45,439 - lava_callback - ERROR - Error processing callback for node 69408947575bc060ce15e9c3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408947575bc060ce15e9c3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408947575bc060ce15e9c3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408947575bc060ce15e9c3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:45,441 - lava_callback - INFO - Starting async job submit for node 69408962575bc060ce15ea61 2025-12-17 22:26:45,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:45,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408962575bc060ce15ea61 HTTP/1.1" 502 150 2025-12-17 22:26:45,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408962575bc060ce15ea61'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:45,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408962575bc060ce15ea61 2025-12-17 22:26:45,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408962575bc060ce15ea61 HTTP/1.1" 502 150 2025-12-17 22:26:45,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408962575bc060ce15ea61'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:45,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940894f575bc060ce15e9d3 2025-12-17 22:26:45,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408951575bc060ce15e9d6 2025-12-17 22:26:45,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940894f575bc060ce15e9d3 HTTP/1.1" 502 150 2025-12-17 22:26:45,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940894f575bc060ce15e9d3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:45,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408951575bc060ce15e9d6 HTTP/1.1" 502 150 2025-12-17 22:26:45,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408951575bc060ce15e9d6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:45,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408912575bc060ce15e91c 2025-12-17 22:26:45,956 - lava_callback - INFO - Received callback for node 69408b17575bc060ce15f0c4 INFO: 172.21.0.1:39062 - "POST /node/69408b17575bc060ce15f0c4 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:39074 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:26:46,604 - lava_callback - INFO - Received callback for node 69408b19575bc060ce15f0c7 INFO: 172.21.0.1:39076 - "POST /node/69408b19575bc060ce15f0c7 HTTP/1.0" 202 Accepted 2025-12-17 22:26:46,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408960575bc060ce15ea5e 2025-12-17 22:26:46,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408960575bc060ce15ea5e HTTP/1.1" 502 150 2025-12-17 22:26:46,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408960575bc060ce15ea5e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:47,239 - lava_callback - INFO - Received callback for node 69408b1b575bc060ce15f0ca INFO: 172.21.0.1:39084 - "POST /node/69408b1b575bc060ce15f0ca HTTP/1.0" 202 Accepted 2025-12-17 22:26:47,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408962575bc060ce15ea61 2025-12-17 22:26:47,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408962575bc060ce15ea61 HTTP/1.1" 502 150 2025-12-17 22:26:47,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408962575bc060ce15ea61'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:47,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408958575bc060ce15e9e8 2025-12-17 22:26:47,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408958575bc060ce15e9e8 HTTP/1.1" 502 150 2025-12-17 22:26:47,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408958575bc060ce15e9e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:47,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408718575bc060ce15e273 2025-12-17 22:26:47,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408718575bc060ce15e273 HTTP/1.1" 502 150 2025-12-17 22:26:47,776 - lava_callback - ERROR - Error processing callback for node 69408718575bc060ce15e273: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408718575bc060ce15e273 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408718575bc060ce15e273 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408718575bc060ce15e273 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:47,778 - lava_callback - INFO - Starting async job submit for node 69408966575bc060ce15ea6a 2025-12-17 22:26:47,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:47,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408966575bc060ce15ea6a HTTP/1.1" 502 150 2025-12-17 22:26:47,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408966575bc060ce15ea6a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:47,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408966575bc060ce15ea6a 2025-12-17 22:26:47,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408966575bc060ce15ea6a HTTP/1.1" 502 150 2025-12-17 22:26:47,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408966575bc060ce15ea6a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:47,882 - lava_callback - INFO - Received callback for node 69408b1d575bc060ce15f0cd INFO: 172.21.0.1:39100 - "POST /node/69408b1d575bc060ce15f0cd HTTP/1.0" 202 Accepted 2025-12-17 22:26:48,222 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895a575bc060ce15ea16 2025-12-17 22:26:48,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895a575bc060ce15ea16 HTTP/1.1" 502 150 2025-12-17 22:26:48,227 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895a575bc060ce15ea16'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:48,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895c575bc060ce15ea4b 2025-12-17 22:26:48,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895c575bc060ce15ea4b HTTP/1.1" 502 150 2025-12-17 22:26:48,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895c575bc060ce15ea4b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:48,534 - lava_callback - INFO - Received callback for node 69408b1f575bc060ce15f0d0 INFO: 172.21.0.1:39110 - "POST /node/69408b1f575bc060ce15f0d0 HTTP/1.0" 202 Accepted 2025-12-17 22:26:49,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408943575bc060ce15e9b8 2025-12-17 22:26:49,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408943575bc060ce15e9b8 HTTP/1.1" 502 150 2025-12-17 22:26:49,059 - lava_callback - ERROR - Error processing callback for node 69408943575bc060ce15e9b8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408943575bc060ce15e9b8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408943575bc060ce15e9b8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408943575bc060ce15e9b8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:49,061 - lava_callback - INFO - Starting async job submit for node 69408963575bc060ce15ea64 2025-12-17 22:26:49,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:49,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408963575bc060ce15ea64 HTTP/1.1" 502 150 2025-12-17 22:26:49,069 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408963575bc060ce15ea64'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:49,069 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408963575bc060ce15ea64 2025-12-17 22:26:49,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408963575bc060ce15ea64 HTTP/1.1" 502 150 2025-12-17 22:26:49,070 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408963575bc060ce15ea64'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:49,181 - lava_callback - INFO - Received callback for node 69408b21575bc060ce15f0d3 INFO: 172.21.0.1:39120 - "POST /node/69408b21575bc060ce15f0d3 HTTP/1.0" 202 Accepted 2025-12-17 22:26:49,270 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408948575bc060ce15e9c6 2025-12-17 22:26:49,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408948575bc060ce15e9c6 HTTP/1.1" 502 150 2025-12-17 22:26:49,272 - lava_callback - ERROR - Error processing callback for node 69408948575bc060ce15e9c6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408948575bc060ce15e9c6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408948575bc060ce15e9c6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408948575bc060ce15e9c6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:49,273 - lava_callback - INFO - Starting async job submit for node 69408969575bc060ce15ea70 2025-12-17 22:26:49,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:49,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408969575bc060ce15ea70 HTTP/1.1" 502 150 2025-12-17 22:26:49,283 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408969575bc060ce15ea70'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:49,283 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408969575bc060ce15ea70 2025-12-17 22:26:49,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408969575bc060ce15ea70 HTTP/1.1" 502 150 2025-12-17 22:26:49,283 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408969575bc060ce15ea70'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:49,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895e575bc060ce15ea58 2025-12-17 22:26:49,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895e575bc060ce15ea58 HTTP/1.1" 502 150 2025-12-17 22:26:49,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895e575bc060ce15ea58'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:49,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408966575bc060ce15ea6a 2025-12-17 22:26:49,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408966575bc060ce15ea6a HTTP/1.1" 502 150 2025-12-17 22:26:49,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408966575bc060ce15ea6a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:51,070 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408963575bc060ce15ea64 2025-12-17 22:26:51,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408963575bc060ce15ea64 HTTP/1.1" 502 150 2025-12-17 22:26:51,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408963575bc060ce15ea64'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:51,284 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408969575bc060ce15ea70 2025-12-17 22:26:51,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408969575bc060ce15ea70 HTTP/1.1" 502 150 2025-12-17 22:26:51,285 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408969575bc060ce15ea70'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:51,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408962575bc060ce15ea61 2025-12-17 22:26:51,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408962575bc060ce15ea61 HTTP/1.1" 502 150 2025-12-17 22:26:51,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408962575bc060ce15ea61'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:52,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895f575bc060ce15ea5b 2025-12-17 22:26:52,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895f575bc060ce15ea5b HTTP/1.1" 502 150 2025-12-17 22:26:52,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940895f575bc060ce15ea5b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:52,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408952575bc060ce15e9d9 2025-12-17 22:26:52,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408952575bc060ce15e9d9 HTTP/1.1" 502 150 2025-12-17 22:26:52,591 - lava_callback - ERROR - Error processing callback for node 69408952575bc060ce15e9d9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408952575bc060ce15e9d9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408952575bc060ce15e9d9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408952575bc060ce15e9d9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:52,593 - lava_callback - INFO - Starting async job submit for node 69408968575bc060ce15ea6d 2025-12-17 22:26:52,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:52,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408968575bc060ce15ea6d HTTP/1.1" 502 150 2025-12-17 22:26:52,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408968575bc060ce15ea6d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:52,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408968575bc060ce15ea6d 2025-12-17 22:26:52,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408968575bc060ce15ea6d HTTP/1.1" 502 150 2025-12-17 22:26:52,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408968575bc060ce15ea6d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:53,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408966575bc060ce15ea6a 2025-12-17 22:26:53,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408966575bc060ce15ea6a HTTP/1.1" 502 150 2025-12-17 22:26:53,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408966575bc060ce15ea6a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:54,542 - lava_callback - INFO - Received callback for node 69408b2c575bc060ce15f0f0 INFO: 172.21.0.1:47468 - "POST /node/69408b2c575bc060ce15f0f0 HTTP/1.0" 202 Accepted 2025-12-17 22:26:54,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408968575bc060ce15ea6d 2025-12-17 22:26:54,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408968575bc060ce15ea6d HTTP/1.1" 502 150 2025-12-17 22:26:54,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408968575bc060ce15ea6d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:54,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408960575bc060ce15ea5e 2025-12-17 22:26:54,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408960575bc060ce15ea5e HTTP/1.1" 502 150 2025-12-17 22:26:54,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408960575bc060ce15ea5e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:55,071 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408963575bc060ce15ea64 2025-12-17 22:26:55,285 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408969575bc060ce15ea70 2025-12-17 22:26:55,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408969575bc060ce15ea70 HTTP/1.1" 502 150 2025-12-17 22:26:55,288 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408969575bc060ce15ea70'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47474 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:26:57,326 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408954575bc060ce15e9dc 2025-12-17 22:26:57,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408954575bc060ce15e9dc HTTP/1.1" 502 150 2025-12-17 22:26:57,328 - lava_callback - ERROR - Error processing callback for node 69408954575bc060ce15e9dc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408954575bc060ce15e9dc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408954575bc060ce15e9dc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408954575bc060ce15e9dc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:26:57,330 - lava_callback - INFO - Starting async job submit for node 6940896c575bc060ce15ea79 2025-12-17 22:26:57,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:26:57,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896c575bc060ce15ea79 HTTP/1.1" 502 150 2025-12-17 22:26:57,339 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940896c575bc060ce15ea79'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:57,339 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940896c575bc060ce15ea79 2025-12-17 22:26:57,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896c575bc060ce15ea79 HTTP/1.1" 502 150 2025-12-17 22:26:57,339 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940896c575bc060ce15ea79'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:58,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408968575bc060ce15ea6d 2025-12-17 22:26:58,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408968575bc060ce15ea6d HTTP/1.1" 502 150 2025-12-17 22:26:58,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408968575bc060ce15ea6d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:59,340 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940896c575bc060ce15ea79 2025-12-17 22:26:59,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896c575bc060ce15ea79 HTTP/1.1" 502 150 2025-12-17 22:26:59,341 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940896c575bc060ce15ea79'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:59,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408962575bc060ce15ea61 2025-12-17 22:26:59,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408962575bc060ce15ea61 HTTP/1.1" 502 150 2025-12-17 22:26:59,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408962575bc060ce15ea61'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:26:59,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408964575bc060ce15ea67 HTTP/1.1" 502 150 2025-12-17 22:26:59,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408964575bc060ce15ea67'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:01,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940894f575bc060ce15e9d3 2025-12-17 22:27:01,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408951575bc060ce15e9d6 2025-12-17 22:27:01,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940894f575bc060ce15e9d3 HTTP/1.1" 502 150 2025-12-17 22:27:01,501 - lava_callback - ERROR - Error processing callback for node 6940894f575bc060ce15e9d3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940894f575bc060ce15e9d3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940894f575bc060ce15e9d3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940894f575bc060ce15e9d3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:01,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408951575bc060ce15e9d6 HTTP/1.1" 502 150 2025-12-17 22:27:01,504 - lava_callback - INFO - Starting async job submit for node 6940896d575bc060ce15ea7f 2025-12-17 22:27:01,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:01,504 - lava_callback - ERROR - Error processing callback for node 69408951575bc060ce15e9d6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408951575bc060ce15e9d6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408951575bc060ce15e9d6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408951575bc060ce15e9d6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:01,508 - lava_callback - INFO - Starting async job submit for node 69426b011b61f1c6005f5896 2025-12-17 22:27:01,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:01,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896d575bc060ce15ea7f HTTP/1.1" 502 150 2025-12-17 22:27:01,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940896d575bc060ce15ea7f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:01,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940896d575bc060ce15ea7f 2025-12-17 22:27:01,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896d575bc060ce15ea7f HTTP/1.1" 502 150 2025-12-17 22:27:01,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940896d575bc060ce15ea7f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:01,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69426b011b61f1c6005f5896 HTTP/1.1" 502 150 2025-12-17 22:27:01,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69426b011b61f1c6005f5896'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:01,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69426b011b61f1c6005f5896 2025-12-17 22:27:01,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69426b011b61f1c6005f5896 HTTP/1.1" 502 150 2025-12-17 22:27:01,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69426b011b61f1c6005f5896'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:01,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408966575bc060ce15ea6a 2025-12-17 22:27:01,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408966575bc060ce15ea6a HTTP/1.1" 502 150 2025-12-17 22:27:01,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408966575bc060ce15ea6a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36526 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:27:03,288 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408969575bc060ce15ea70 2025-12-17 22:27:03,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408969575bc060ce15ea70 HTTP/1.1" 502 150 2025-12-17 22:27:03,289 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408969575bc060ce15ea70'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:03,341 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940896c575bc060ce15ea79 2025-12-17 22:27:03,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896c575bc060ce15ea79 HTTP/1.1" 502 150 2025-12-17 22:27:03,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940896c575bc060ce15ea79'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:03,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408958575bc060ce15e9e8 2025-12-17 22:27:03,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408958575bc060ce15e9e8 HTTP/1.1" 502 150 2025-12-17 22:27:03,474 - lava_callback - ERROR - Error processing callback for node 69408958575bc060ce15e9e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408958575bc060ce15e9e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408958575bc060ce15e9e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408958575bc060ce15e9e8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:03,476 - lava_callback - INFO - Starting async job submit for node 6940896e575bc060ce15ea85 2025-12-17 22:27:03,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:03,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896e575bc060ce15ea85 HTTP/1.1" 502 150 2025-12-17 22:27:03,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940896e575bc060ce15ea85'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:03,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940896e575bc060ce15ea85 2025-12-17 22:27:03,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896e575bc060ce15ea85 HTTP/1.1" 502 150 2025-12-17 22:27:03,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940896e575bc060ce15ea85'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:03,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940896d575bc060ce15ea7f 2025-12-17 22:27:03,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896d575bc060ce15ea7f HTTP/1.1" 502 150 2025-12-17 22:27:03,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940896d575bc060ce15ea7f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:03,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69426b011b61f1c6005f5896 2025-12-17 22:27:03,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69426b011b61f1c6005f5896 HTTP/1.1" 502 150 2025-12-17 22:27:03,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69426b011b61f1c6005f5896'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:03,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408964575bc060ce15ea67 2025-12-17 22:27:03,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408964575bc060ce15ea67 HTTP/1.1" 502 150 2025-12-17 22:27:03,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408964575bc060ce15ea67'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:04,228 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895a575bc060ce15ea16 2025-12-17 22:27:04,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895c575bc060ce15ea4b 2025-12-17 22:27:04,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895c575bc060ce15ea4b HTTP/1.1" 502 150 2025-12-17 22:27:04,438 - lava_callback - ERROR - Error processing callback for node 6940895c575bc060ce15ea4b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940895c575bc060ce15ea4b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940895c575bc060ce15ea4b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940895c575bc060ce15ea4b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:04,440 - lava_callback - INFO - Starting async job submit for node 69408971575bc060ce15ea8e 2025-12-17 22:27:04,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:04,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408971575bc060ce15ea8e HTTP/1.1" 502 150 2025-12-17 22:27:04,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408971575bc060ce15ea8e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:04,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408971575bc060ce15ea8e 2025-12-17 22:27:04,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408971575bc060ce15ea8e HTTP/1.1" 502 150 2025-12-17 22:27:04,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408971575bc060ce15ea8e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:04,668 - lava_callback - INFO - Received callback for node 69408b27575bc060ce15f0e5 INFO: 172.21.0.1:36530 - "POST /node/69408b27575bc060ce15f0e5 HTTP/1.0" 202 Accepted 2025-12-17 22:27:05,320 - lava_callback - INFO - Received callback for node 69408b2a575bc060ce15f0ed INFO: 172.21.0.1:36542 - "POST /node/69408b2a575bc060ce15f0ed HTTP/1.0" 202 Accepted 2025-12-17 22:27:05,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940896e575bc060ce15ea85 2025-12-17 22:27:05,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896e575bc060ce15ea85 HTTP/1.1" 502 150 2025-12-17 22:27:05,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940896e575bc060ce15ea85'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:05,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895e575bc060ce15ea58 2025-12-17 22:27:05,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895e575bc060ce15ea58 HTTP/1.1" 502 150 2025-12-17 22:27:05,686 - lava_callback - ERROR - Error processing callback for node 6940895e575bc060ce15ea58: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940895e575bc060ce15ea58 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940895e575bc060ce15ea58 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940895e575bc060ce15ea58 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:05,688 - lava_callback - INFO - Starting async job submit for node 69408973575bc060ce15ea91 2025-12-17 22:27:05,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:05,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408973575bc060ce15ea91 HTTP/1.1" 502 150 2025-12-17 22:27:05,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408973575bc060ce15ea91'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:05,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408973575bc060ce15ea91 2025-12-17 22:27:05,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408973575bc060ce15ea91 HTTP/1.1" 502 150 2025-12-17 22:27:05,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408973575bc060ce15ea91'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:05,983 - lava_callback - INFO - Received callback for node 69408b2f575bc060ce15f0fb INFO: 172.21.0.1:36546 - "POST /node/69408b2f575bc060ce15f0fb HTTP/1.0" 202 Accepted 2025-12-17 22:27:06,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408971575bc060ce15ea8e 2025-12-17 22:27:06,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408971575bc060ce15ea8e HTTP/1.1" 502 150 2025-12-17 22:27:06,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408971575bc060ce15ea8e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36554 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:27:06,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408968575bc060ce15ea6d 2025-12-17 22:27:06,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408968575bc060ce15ea6d HTTP/1.1" 502 150 2025-12-17 22:27:06,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408968575bc060ce15ea6d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:06,642 - lava_callback - INFO - Received callback for node 69408b33575bc060ce15f101 INFO: 172.21.0.1:36566 - "POST /node/69408b33575bc060ce15f101 HTTP/1.0" 202 Accepted 2025-12-17 22:27:07,288 - lava_callback - INFO - Received callback for node 69408b37575bc060ce15f107 INFO: 172.21.0.1:36582 - "POST /node/69408b37575bc060ce15f107 HTTP/1.0" 202 Accepted 2025-12-17 22:27:07,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940896d575bc060ce15ea7f 2025-12-17 22:27:07,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896d575bc060ce15ea7f HTTP/1.1" 502 150 2025-12-17 22:27:07,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940896d575bc060ce15ea7f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:07,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69426b011b61f1c6005f5896 2025-12-17 22:27:07,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69426b011b61f1c6005f5896 HTTP/1.1" 502 150 2025-12-17 22:27:07,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69426b011b61f1c6005f5896'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:07,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408973575bc060ce15ea91 2025-12-17 22:27:07,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408973575bc060ce15ea91 HTTP/1.1" 502 150 2025-12-17 22:27:07,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408973575bc060ce15ea91'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:08,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940895f575bc060ce15ea5b 2025-12-17 22:27:08,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895f575bc060ce15ea5b HTTP/1.1" 502 150 2025-12-17 22:27:08,536 - lava_callback - ERROR - Error processing callback for node 6940895f575bc060ce15ea5b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940895f575bc060ce15ea5b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940895f575bc060ce15ea5b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940895f575bc060ce15ea5b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:08,538 - lava_callback - INFO - Starting async job submit for node 6940c5361e3c24e95681d6a7 2025-12-17 22:27:08,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:08,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5361e3c24e95681d6a7 HTTP/1.1" 502 150 2025-12-17 22:27:08,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5361e3c24e95681d6a7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:08,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5361e3c24e95681d6a7 2025-12-17 22:27:08,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5361e3c24e95681d6a7 HTTP/1.1" 502 150 2025-12-17 22:27:08,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5361e3c24e95681d6a7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:09,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940896e575bc060ce15ea85 2025-12-17 22:27:09,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896e575bc060ce15ea85 HTTP/1.1" 502 150 2025-12-17 22:27:09,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940896e575bc060ce15ea85'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:10,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408963575bc060ce15ea64 HTTP/1.1" 502 150 2025-12-17 22:27:10,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408963575bc060ce15ea64'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:10,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408971575bc060ce15ea8e 2025-12-17 22:27:10,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408971575bc060ce15ea8e HTTP/1.1" 502 150 2025-12-17 22:27:10,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408971575bc060ce15ea8e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:10,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5361e3c24e95681d6a7 2025-12-17 22:27:10,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5361e3c24e95681d6a7 HTTP/1.1" 502 150 2025-12-17 22:27:10,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5361e3c24e95681d6a7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:10,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408960575bc060ce15ea5e 2025-12-17 22:27:10,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408960575bc060ce15ea5e HTTP/1.1" 502 150 2025-12-17 22:27:10,657 - lava_callback - ERROR - Error processing callback for node 69408960575bc060ce15ea5e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408960575bc060ce15ea5e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408960575bc060ce15ea5e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408960575bc060ce15ea5e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:10,659 - lava_callback - INFO - Starting async job submit for node 6940897b575bc060ce15ea9c 2025-12-17 22:27:10,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:10,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940897b575bc060ce15ea9c HTTP/1.1" 502 150 2025-12-17 22:27:10,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940897b575bc060ce15ea9c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:10,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940897b575bc060ce15ea9c 2025-12-17 22:27:10,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940897b575bc060ce15ea9c HTTP/1.1" 502 150 2025-12-17 22:27:10,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940897b575bc060ce15ea9c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:11,342 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940896c575bc060ce15ea79 2025-12-17 22:27:11,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896c575bc060ce15ea79 HTTP/1.1" 502 150 2025-12-17 22:27:11,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940896c575bc060ce15ea79'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:11,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408973575bc060ce15ea91 2025-12-17 22:27:11,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408973575bc060ce15ea91 HTTP/1.1" 502 150 2025-12-17 22:27:11,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408973575bc060ce15ea91'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:11,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408964575bc060ce15ea67 2025-12-17 22:27:11,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408964575bc060ce15ea67 HTTP/1.1" 502 150 2025-12-17 22:27:11,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408964575bc060ce15ea67'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:12,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940897b575bc060ce15ea9c 2025-12-17 22:27:12,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940897b575bc060ce15ea9c HTTP/1.1" 502 150 2025-12-17 22:27:12,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940897b575bc060ce15ea9c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:14,391 - lava_callback - INFO - Received callback for node 69409228575bc060ce160a0c INFO: 172.21.0.1:54388 - "POST /node/69409228575bc060ce160a0c HTTP/1.0" 202 Accepted 2025-12-17 22:27:14,549 - lava_callback - INFO - Received callback for node 69408b31575bc060ce15f0fe INFO: 172.21.0.1:54400 - "POST /node/69408b31575bc060ce15f0fe HTTP/1.0" 202 Accepted 2025-12-17 22:27:14,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5361e3c24e95681d6a7 2025-12-17 22:27:14,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5361e3c24e95681d6a7 HTTP/1.1" 502 150 2025-12-17 22:27:14,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5361e3c24e95681d6a7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54404 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:27:15,216 - lava_callback - INFO - Received callback for node 69408b35575bc060ce15f104 INFO: 172.21.0.1:54408 - "POST /node/69408b35575bc060ce15f104 HTTP/1.0" 202 Accepted 2025-12-17 22:27:15,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408962575bc060ce15ea61 2025-12-17 22:27:15,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408962575bc060ce15ea61 HTTP/1.1" 502 150 2025-12-17 22:27:15,462 - lava_callback - ERROR - Error processing callback for node 69408962575bc060ce15ea61: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408962575bc060ce15ea61 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408962575bc060ce15ea61 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408962575bc060ce15ea61 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:15,464 - lava_callback - INFO - Starting async job submit for node 6940897d575bc060ce15ea9f 2025-12-17 22:27:15,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:15,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940897d575bc060ce15ea9f HTTP/1.1" 502 150 2025-12-17 22:27:15,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940897d575bc060ce15ea9f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:15,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940897d575bc060ce15ea9f 2025-12-17 22:27:15,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940897d575bc060ce15ea9f HTTP/1.1" 502 150 2025-12-17 22:27:15,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940897d575bc060ce15ea9f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:15,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940896d575bc060ce15ea7f 2025-12-17 22:27:15,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896d575bc060ce15ea7f HTTP/1.1" 502 150 2025-12-17 22:27:15,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940896d575bc060ce15ea7f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:15,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69426b011b61f1c6005f5896 2025-12-17 22:27:15,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69426b011b61f1c6005f5896 HTTP/1.1" 502 150 2025-12-17 22:27:15,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69426b011b61f1c6005f5896'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54412 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:27:16,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408912575bc060ce15e91c HTTP/1.1" 502 150 2025-12-17 22:27:16,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408912575bc060ce15e91c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:16,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940897b575bc060ce15ea9c 2025-12-17 22:27:16,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940897b575bc060ce15ea9c HTTP/1.1" 502 150 2025-12-17 22:27:16,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940897b575bc060ce15ea9c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:17,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940897d575bc060ce15ea9f 2025-12-17 22:27:17,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940897d575bc060ce15ea9f HTTP/1.1" 502 150 2025-12-17 22:27:17,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940897d575bc060ce15ea9f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:17,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940896e575bc060ce15ea85 2025-12-17 22:27:17,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896e575bc060ce15ea85 HTTP/1.1" 502 150 2025-12-17 22:27:17,495 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940896e575bc060ce15ea85'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:17,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408966575bc060ce15ea6a 2025-12-17 22:27:17,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408966575bc060ce15ea6a HTTP/1.1" 502 150 2025-12-17 22:27:17,813 - lava_callback - ERROR - Error processing callback for node 69408966575bc060ce15ea6a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408966575bc060ce15ea6a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408966575bc060ce15ea6a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408966575bc060ce15ea6a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:17,815 - lava_callback - INFO - Starting async job submit for node 69408980575bc060ce15eaa8 2025-12-17 22:27:17,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:17,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408980575bc060ce15eaa8 HTTP/1.1" 502 150 2025-12-17 22:27:17,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408980575bc060ce15eaa8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:17,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408980575bc060ce15eaa8 2025-12-17 22:27:17,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408980575bc060ce15eaa8 HTTP/1.1" 502 150 2025-12-17 22:27:17,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408980575bc060ce15eaa8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:18,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408963575bc060ce15ea64 2025-12-17 22:27:18,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408963575bc060ce15ea64 HTTP/1.1" 502 150 2025-12-17 22:27:18,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408963575bc060ce15ea64'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:18,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408971575bc060ce15ea8e 2025-12-17 22:27:18,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408971575bc060ce15ea8e HTTP/1.1" 502 150 2025-12-17 22:27:18,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408971575bc060ce15ea8e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:19,290 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408969575bc060ce15ea70 2025-12-17 22:27:19,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408969575bc060ce15ea70 HTTP/1.1" 502 150 2025-12-17 22:27:19,291 - lava_callback - ERROR - Error processing callback for node 69408969575bc060ce15ea70: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408969575bc060ce15ea70 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408969575bc060ce15ea70 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408969575bc060ce15ea70 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:19,293 - lava_callback - INFO - Starting async job submit for node 69408981575bc060ce15eaaf 2025-12-17 22:27:19,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:19,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408981575bc060ce15eaaf HTTP/1.1" 502 150 2025-12-17 22:27:19,299 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408981575bc060ce15eaaf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:19,299 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408981575bc060ce15eaaf 2025-12-17 22:27:19,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408981575bc060ce15eaaf HTTP/1.1" 502 150 2025-12-17 22:27:19,300 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408981575bc060ce15eaaf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:19,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940895a575bc060ce15ea16 HTTP/1.1" 502 150 2025-12-17 22:27:19,607 - lava_callback - ERROR - Error processing callback for node 6940895a575bc060ce15ea16: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940895a575bc060ce15ea16 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940895a575bc060ce15ea16 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940895a575bc060ce15ea16 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:19,609 - lava_callback - INFO - Starting async job submit for node 69408983575bc060ce15eab4 2025-12-17 22:27:19,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:19,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408983575bc060ce15eab4 HTTP/1.1" 502 150 2025-12-17 22:27:19,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408983575bc060ce15eab4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:19,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408983575bc060ce15eab4 2025-12-17 22:27:19,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408983575bc060ce15eab4 HTTP/1.1" 502 150 2025-12-17 22:27:19,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408983575bc060ce15eab4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:19,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408973575bc060ce15ea91 2025-12-17 22:27:19,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408973575bc060ce15ea91 HTTP/1.1" 502 150 2025-12-17 22:27:19,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408973575bc060ce15ea91'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:19,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408980575bc060ce15eaa8 2025-12-17 22:27:19,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408980575bc060ce15eaa8 HTTP/1.1" 502 150 2025-12-17 22:27:19,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408980575bc060ce15eaa8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:20,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408912575bc060ce15e91c 2025-12-17 22:27:21,300 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408981575bc060ce15eaaf 2025-12-17 22:27:21,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940897d575bc060ce15ea9f 2025-12-17 22:27:21,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940897d575bc060ce15ea9f HTTP/1.1" 502 150 2025-12-17 22:27:21,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940897d575bc060ce15ea9f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:21,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408983575bc060ce15eab4 2025-12-17 22:27:21,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408983575bc060ce15eab4 HTTP/1.1" 502 150 2025-12-17 22:27:21,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408983575bc060ce15eab4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:22,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408968575bc060ce15ea6d 2025-12-17 22:27:22,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408968575bc060ce15ea6d HTTP/1.1" 502 150 2025-12-17 22:27:22,607 - lava_callback - ERROR - Error processing callback for node 69408968575bc060ce15ea6d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408968575bc060ce15ea6d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408968575bc060ce15ea6d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408968575bc060ce15ea6d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:22,609 - lava_callback - INFO - Starting async job submit for node 69408984575bc060ce15eab7 2025-12-17 22:27:22,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:22,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408984575bc060ce15eab7 HTTP/1.1" 502 150 2025-12-17 22:27:22,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408984575bc060ce15eab7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:22,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408984575bc060ce15eab7 2025-12-17 22:27:22,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408984575bc060ce15eab7 HTTP/1.1" 502 150 2025-12-17 22:27:22,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408984575bc060ce15eab7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:22,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5361e3c24e95681d6a7 2025-12-17 22:27:22,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5361e3c24e95681d6a7 HTTP/1.1" 502 150 2025-12-17 22:27:22,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5361e3c24e95681d6a7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:23,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408980575bc060ce15eaa8 2025-12-17 22:27:23,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408980575bc060ce15eaa8 HTTP/1.1" 502 150 2025-12-17 22:27:23,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408980575bc060ce15eaa8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:24,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408984575bc060ce15eab7 2025-12-17 22:27:24,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408984575bc060ce15eab7 HTTP/1.1" 502 150 2025-12-17 22:27:24,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408984575bc060ce15eab7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:24,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940897b575bc060ce15ea9c 2025-12-17 22:27:24,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940897b575bc060ce15ea9c HTTP/1.1" 502 150 2025-12-17 22:27:24,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940897b575bc060ce15ea9c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:24,696 - lava_callback - INFO - Received callback for node 69408b39575bc060ce15f10e INFO: 172.21.0.1:38844 - "POST /node/69408b39575bc060ce15f10e HTTP/1.0" 202 Accepted 2025-12-17 22:27:25,363 - lava_callback - INFO - Received callback for node 69408b3b575bc060ce15f115 INFO: 172.21.0.1:38846 - "POST /node/69408b3b575bc060ce15f115 HTTP/1.0" 202 Accepted 2025-12-17 22:27:25,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408983575bc060ce15eab4 2025-12-17 22:27:25,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408983575bc060ce15eab4 HTTP/1.1" 502 150 2025-12-17 22:27:25,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408983575bc060ce15eab4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:26,000 - lava_callback - INFO - Received callback for node 69408b3d575bc060ce15f11e INFO: 172.21.0.1:38860 - "POST /node/69408b3d575bc060ce15f11e HTTP/1.0" 202 Accepted INFO: 172.21.0.1:38868 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:27:26,631 - lava_callback - INFO - Received callback for node 69408b3f575bc060ce15f121 INFO: 172.21.0.1:38884 - "POST /node/69408b3f575bc060ce15f121 HTTP/1.0" 202 Accepted 2025-12-17 22:27:27,277 - lava_callback - INFO - Received callback for node 69408b41575bc060ce15f124 INFO: 172.21.0.1:38900 - "POST /node/69408b41575bc060ce15f124 HTTP/1.0" 202 Accepted 2025-12-17 22:27:27,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940896c575bc060ce15ea79 2025-12-17 22:27:27,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896c575bc060ce15ea79 HTTP/1.1" 502 150 2025-12-17 22:27:27,345 - lava_callback - ERROR - Error processing callback for node 6940896c575bc060ce15ea79: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940896c575bc060ce15ea79 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940896c575bc060ce15ea79 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940896c575bc060ce15ea79 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:27,348 - lava_callback - INFO - Starting async job submit for node 69408985575bc060ce15eaba 2025-12-17 22:27:27,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:27,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408985575bc060ce15eaba HTTP/1.1" 502 150 2025-12-17 22:27:27,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408985575bc060ce15eaba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:27,355 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408985575bc060ce15eaba 2025-12-17 22:27:27,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408985575bc060ce15eaba HTTP/1.1" 502 150 2025-12-17 22:27:27,356 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408985575bc060ce15eaba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:27,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408964575bc060ce15ea67 2025-12-17 22:27:27,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408964575bc060ce15ea67 HTTP/1.1" 502 150 2025-12-17 22:27:27,893 - lava_callback - ERROR - Error processing callback for node 69408964575bc060ce15ea67: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408964575bc060ce15ea67 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408964575bc060ce15ea67 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408964575bc060ce15ea67 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:27,894 - lava_callback - INFO - Starting async job submit for node 69408986575bc060ce15eac2 2025-12-17 22:27:27,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:27,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408986575bc060ce15eac2 HTTP/1.1" 502 150 2025-12-17 22:27:27,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408986575bc060ce15eac2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:27,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408986575bc060ce15eac2 2025-12-17 22:27:27,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408986575bc060ce15eac2 HTTP/1.1" 502 150 2025-12-17 22:27:27,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408986575bc060ce15eac2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:27,924 - lava_callback - INFO - Received callback for node 69408b43575bc060ce15f127 INFO: 172.21.0.1:38906 - "POST /node/69408b43575bc060ce15f127 HTTP/1.0" 202 Accepted 2025-12-17 22:27:28,571 - lava_callback - INFO - Received callback for node 69408b45575bc060ce15f12a INFO: 172.21.0.1:38920 - "POST /node/69408b45575bc060ce15f12a HTTP/1.0" 202 Accepted 2025-12-17 22:27:28,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408984575bc060ce15eab7 2025-12-17 22:27:28,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408984575bc060ce15eab7 HTTP/1.1" 502 150 2025-12-17 22:27:28,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408984575bc060ce15eab7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:29,220 - lava_callback - INFO - Received callback for node 69408b47575bc060ce15f12d INFO: 172.21.0.1:38928 - "POST /node/69408b47575bc060ce15f12d HTTP/1.0" 202 Accepted 2025-12-17 22:27:29,356 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408985575bc060ce15eaba 2025-12-17 22:27:29,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408985575bc060ce15eaba HTTP/1.1" 502 150 2025-12-17 22:27:29,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408985575bc060ce15eaba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:29,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940897d575bc060ce15ea9f 2025-12-17 22:27:29,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940897d575bc060ce15ea9f HTTP/1.1" 502 150 2025-12-17 22:27:29,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940897d575bc060ce15ea9f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:29,856 - lava_callback - INFO - Received callback for node 69408b49575bc060ce15f130 INFO: 172.21.0.1:49614 - "POST /node/69408b49575bc060ce15f130 HTTP/1.0" 202 Accepted 2025-12-17 22:27:29,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408986575bc060ce15eac2 2025-12-17 22:27:29,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408986575bc060ce15eac2 HTTP/1.1" 502 150 2025-12-17 22:27:29,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408986575bc060ce15eac2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:31,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940896d575bc060ce15ea7f 2025-12-17 22:27:31,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896d575bc060ce15ea7f HTTP/1.1" 502 150 2025-12-17 22:27:31,524 - lava_callback - ERROR - Error processing callback for node 6940896d575bc060ce15ea7f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940896d575bc060ce15ea7f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940896d575bc060ce15ea7f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940896d575bc060ce15ea7f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:31,526 - lava_callback - INFO - Starting async job submit for node 69408988575bc060ce15eac5 2025-12-17 22:27:31,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:31,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69426b011b61f1c6005f5896 2025-12-17 22:27:31,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69426b011b61f1c6005f5896 HTTP/1.1" 502 150 2025-12-17 22:27:31,528 - lava_callback - ERROR - Error processing callback for node 69426b011b61f1c6005f5896: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69426b011b61f1c6005f5896 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69426b011b61f1c6005f5896 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69426b011b61f1c6005f5896 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:31,530 - lava_callback - INFO - Starting async job submit for node 69408989575bc060ce15eac8 2025-12-17 22:27:31,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:31,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408988575bc060ce15eac5 HTTP/1.1" 502 150 2025-12-17 22:27:31,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408988575bc060ce15eac5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:31,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408988575bc060ce15eac5 2025-12-17 22:27:31,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408988575bc060ce15eac5 HTTP/1.1" 502 150 2025-12-17 22:27:31,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408988575bc060ce15eac5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:31,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408989575bc060ce15eac8 HTTP/1.1" 502 150 2025-12-17 22:27:31,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408989575bc060ce15eac8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:31,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408989575bc060ce15eac8 2025-12-17 22:27:31,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408989575bc060ce15eac8 HTTP/1.1" 502 150 2025-12-17 22:27:31,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408989575bc060ce15eac8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:31,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408980575bc060ce15eaa8 2025-12-17 22:27:31,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408980575bc060ce15eaa8 HTTP/1.1" 502 150 2025-12-17 22:27:31,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408980575bc060ce15eaa8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:33,358 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408985575bc060ce15eaba 2025-12-17 22:27:33,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408985575bc060ce15eaba HTTP/1.1" 502 150 2025-12-17 22:27:33,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408985575bc060ce15eaba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:33,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940896e575bc060ce15ea85 2025-12-17 22:27:33,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940896e575bc060ce15ea85 HTTP/1.1" 502 150 2025-12-17 22:27:33,497 - lava_callback - ERROR - Error processing callback for node 6940896e575bc060ce15ea85: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940896e575bc060ce15ea85 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940896e575bc060ce15ea85 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940896e575bc060ce15ea85 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:33,500 - lava_callback - INFO - Starting async job submit for node 6940898a575bc060ce15eacf 2025-12-17 22:27:33,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:33,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898a575bc060ce15eacf HTTP/1.1" 502 150 2025-12-17 22:27:33,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898a575bc060ce15eacf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:33,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898a575bc060ce15eacf 2025-12-17 22:27:33,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898a575bc060ce15eacf HTTP/1.1" 502 150 2025-12-17 22:27:33,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898a575bc060ce15eacf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:33,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408988575bc060ce15eac5 2025-12-17 22:27:33,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408988575bc060ce15eac5 HTTP/1.1" 502 150 2025-12-17 22:27:33,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408988575bc060ce15eac5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:33,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408989575bc060ce15eac8 2025-12-17 22:27:33,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408989575bc060ce15eac8 HTTP/1.1" 502 150 2025-12-17 22:27:33,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408989575bc060ce15eac8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:33,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408983575bc060ce15eab4 2025-12-17 22:27:33,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408983575bc060ce15eab4 HTTP/1.1" 502 150 2025-12-17 22:27:33,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408983575bc060ce15eab4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:33,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408986575bc060ce15eac2 2025-12-17 22:27:33,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408986575bc060ce15eac2 HTTP/1.1" 502 150 2025-12-17 22:27:33,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408986575bc060ce15eac2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:34,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408963575bc060ce15ea64 2025-12-17 22:27:34,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408963575bc060ce15ea64 HTTP/1.1" 502 150 2025-12-17 22:27:34,387 - lava_callback - ERROR - Error processing callback for node 69408963575bc060ce15ea64: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408963575bc060ce15ea64 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408963575bc060ce15ea64 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408963575bc060ce15ea64 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:34,390 - lava_callback - INFO - Starting async job submit for node 6940898b575bc060ce15ead2 2025-12-17 22:27:34,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:34,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898b575bc060ce15ead2 HTTP/1.1" 502 150 2025-12-17 22:27:34,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898b575bc060ce15ead2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:34,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898b575bc060ce15ead2 2025-12-17 22:27:34,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898b575bc060ce15ead2 HTTP/1.1" 502 150 2025-12-17 22:27:34,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898b575bc060ce15ead2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:34,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408971575bc060ce15ea8e 2025-12-17 22:27:34,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408971575bc060ce15ea8e HTTP/1.1" 502 150 2025-12-17 22:27:34,458 - lava_callback - ERROR - Error processing callback for node 69408971575bc060ce15ea8e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408971575bc060ce15ea8e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408971575bc060ce15ea8e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408971575bc060ce15ea8e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:34,461 - lava_callback - INFO - Starting async job submit for node 6940898d575bc060ce15ead5 2025-12-17 22:27:34,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:34,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898d575bc060ce15ead5 HTTP/1.1" 502 150 2025-12-17 22:27:34,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898d575bc060ce15ead5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:34,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898d575bc060ce15ead5 2025-12-17 22:27:34,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898d575bc060ce15ead5 HTTP/1.1" 502 150 2025-12-17 22:27:34,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898d575bc060ce15ead5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:34,531 - lava_callback - INFO - Received callback for node 69408b51575bc060ce15f193 INFO: 172.21.0.1:49630 - "POST /node/69408b51575bc060ce15f193 HTTP/1.0" 202 Accepted 2025-12-17 22:27:35,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898a575bc060ce15eacf 2025-12-17 22:27:35,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898a575bc060ce15eacf HTTP/1.1" 502 150 2025-12-17 22:27:35,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898a575bc060ce15eacf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:35,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408973575bc060ce15ea91 2025-12-17 22:27:35,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408973575bc060ce15ea91 HTTP/1.1" 502 150 2025-12-17 22:27:35,707 - lava_callback - ERROR - Error processing callback for node 69408973575bc060ce15ea91: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408973575bc060ce15ea91 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408973575bc060ce15ea91 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408973575bc060ce15ea91 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:35,709 - lava_callback - INFO - Starting async job submit for node 6940898e575bc060ce15ead8 2025-12-17 22:27:35,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:35,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898e575bc060ce15ead8 HTTP/1.1" 502 150 2025-12-17 22:27:35,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898e575bc060ce15ead8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:35,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898e575bc060ce15ead8 2025-12-17 22:27:35,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898e575bc060ce15ead8 HTTP/1.1" 502 150 2025-12-17 22:27:35,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898e575bc060ce15ead8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:36,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898b575bc060ce15ead2 2025-12-17 22:27:36,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898b575bc060ce15ead2 HTTP/1.1" 502 150 2025-12-17 22:27:36,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898b575bc060ce15ead2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:36,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898d575bc060ce15ead5 2025-12-17 22:27:36,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898d575bc060ce15ead5 HTTP/1.1" 502 150 2025-12-17 22:27:36,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898d575bc060ce15ead5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49644 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:27:36,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408984575bc060ce15eab7 2025-12-17 22:27:36,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408984575bc060ce15eab7 HTTP/1.1" 502 150 2025-12-17 22:27:36,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408984575bc060ce15eab7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:36,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408981575bc060ce15eaaf HTTP/1.1" 502 150 2025-12-17 22:27:36,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408981575bc060ce15eaaf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:37,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408988575bc060ce15eac5 2025-12-17 22:27:37,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408988575bc060ce15eac5 HTTP/1.1" 502 150 2025-12-17 22:27:37,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408988575bc060ce15eac5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:37,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408989575bc060ce15eac8 2025-12-17 22:27:37,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408989575bc060ce15eac8 HTTP/1.1" 502 150 2025-12-17 22:27:37,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408989575bc060ce15eac8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:37,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898e575bc060ce15ead8 2025-12-17 22:27:37,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898e575bc060ce15ead8 HTTP/1.1" 502 150 2025-12-17 22:27:37,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898e575bc060ce15ead8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:38,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5361e3c24e95681d6a7 2025-12-17 22:27:38,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5361e3c24e95681d6a7 HTTP/1.1" 502 150 2025-12-17 22:27:38,646 - lava_callback - ERROR - Error processing callback for node 6940c5361e3c24e95681d6a7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5361e3c24e95681d6a7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5361e3c24e95681d6a7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5361e3c24e95681d6a7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:38,648 - lava_callback - INFO - Starting async job submit for node 6940898f575bc060ce15eadb 2025-12-17 22:27:38,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:38,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898f575bc060ce15eadb HTTP/1.1" 502 150 2025-12-17 22:27:38,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898f575bc060ce15eadb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:38,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898f575bc060ce15eadb 2025-12-17 22:27:38,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898f575bc060ce15eadb HTTP/1.1" 502 150 2025-12-17 22:27:38,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898f575bc060ce15eadb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:39,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898a575bc060ce15eacf 2025-12-17 22:27:39,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898a575bc060ce15eacf HTTP/1.1" 502 150 2025-12-17 22:27:39,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898a575bc060ce15eacf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:40,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898b575bc060ce15ead2 2025-12-17 22:27:40,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898b575bc060ce15ead2 HTTP/1.1" 502 150 2025-12-17 22:27:40,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898b575bc060ce15ead2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:40,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898d575bc060ce15ead5 2025-12-17 22:27:40,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898d575bc060ce15ead5 HTTP/1.1" 502 150 2025-12-17 22:27:40,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898d575bc060ce15ead5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:40,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898f575bc060ce15eadb 2025-12-17 22:27:40,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898f575bc060ce15eadb HTTP/1.1" 502 150 2025-12-17 22:27:40,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898f575bc060ce15eadb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:40,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940897b575bc060ce15ea9c 2025-12-17 22:27:40,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940897b575bc060ce15ea9c HTTP/1.1" 502 150 2025-12-17 22:27:40,682 - lava_callback - ERROR - Error processing callback for node 6940897b575bc060ce15ea9c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940897b575bc060ce15ea9c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940897b575bc060ce15ea9c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940897b575bc060ce15ea9c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:40,684 - lava_callback - INFO - Starting async job submit for node 69408990575bc060ce15eade 2025-12-17 22:27:40,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:40,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408990575bc060ce15eade HTTP/1.1" 502 150 2025-12-17 22:27:40,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408990575bc060ce15eade'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:40,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408990575bc060ce15eade 2025-12-17 22:27:40,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408990575bc060ce15eade HTTP/1.1" 502 150 2025-12-17 22:27:40,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408990575bc060ce15eade'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:40,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408981575bc060ce15eaaf 2025-12-17 22:27:40,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408981575bc060ce15eaaf HTTP/1.1" 502 150 2025-12-17 22:27:40,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408981575bc060ce15eaaf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:41,359 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408985575bc060ce15eaba 2025-12-17 22:27:41,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408985575bc060ce15eaba HTTP/1.1" 502 150 2025-12-17 22:27:41,361 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408985575bc060ce15eaba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:41,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898e575bc060ce15ead8 2025-12-17 22:27:41,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898e575bc060ce15ead8 HTTP/1.1" 502 150 2025-12-17 22:27:41,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898e575bc060ce15ead8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:41,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408986575bc060ce15eac2 2025-12-17 22:27:41,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408986575bc060ce15eac2 HTTP/1.1" 502 150 2025-12-17 22:27:41,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408986575bc060ce15eac2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:42,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408990575bc060ce15eade 2025-12-17 22:27:42,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408990575bc060ce15eade HTTP/1.1" 502 150 2025-12-17 22:27:42,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408990575bc060ce15eade'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:44,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898f575bc060ce15eadb 2025-12-17 22:27:44,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898f575bc060ce15eadb HTTP/1.1" 502 150 2025-12-17 22:27:44,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898f575bc060ce15eadb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:44,688 - lava_callback - INFO - Received callback for node 69408b4d575bc060ce15f17a INFO: 172.21.0.1:55990 - "POST /node/69408b4d575bc060ce15f17a HTTP/1.0" 202 Accepted 2025-12-17 22:27:45,335 - lava_callback - INFO - Received callback for node 69408b4f575bc060ce15f18a INFO: 172.21.0.1:56000 - "POST /node/69408b4f575bc060ce15f18a HTTP/1.0" 202 Accepted 2025-12-17 22:27:45,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940897d575bc060ce15ea9f 2025-12-17 22:27:45,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408988575bc060ce15eac5 2025-12-17 22:27:45,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408989575bc060ce15eac8 2025-12-17 22:27:45,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408988575bc060ce15eac5 HTTP/1.1" 502 150 2025-12-17 22:27:45,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408988575bc060ce15eac5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:45,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408989575bc060ce15eac8 HTTP/1.1" 502 150 2025-12-17 22:27:45,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408989575bc060ce15eac8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:45,993 - lava_callback - INFO - Received callback for node 69408b53575bc060ce15f19c INFO: 172.21.0.1:56012 - "POST /node/69408b53575bc060ce15f19c HTTP/1.0" 202 Accepted INFO: 172.21.0.1:56014 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:27:46,627 - lava_callback - INFO - Received callback for node 69408b57575bc060ce15f1a4 INFO: 172.21.0.1:56026 - "POST /node/69408b57575bc060ce15f1a4 HTTP/1.0" 202 Accepted 2025-12-17 22:27:46,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408990575bc060ce15eade 2025-12-17 22:27:46,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408990575bc060ce15eade HTTP/1.1" 502 150 2025-12-17 22:27:46,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408990575bc060ce15eade'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:47,276 - lava_callback - INFO - Received callback for node 69408b59575bc060ce15f1a7 INFO: 172.21.0.1:56038 - "POST /node/69408b59575bc060ce15f1a7 HTTP/1.0" 202 Accepted 2025-12-17 22:27:47,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898a575bc060ce15eacf 2025-12-17 22:27:47,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898a575bc060ce15eacf HTTP/1.1" 502 150 2025-12-17 22:27:47,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898a575bc060ce15eacf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:47,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408980575bc060ce15eaa8 2025-12-17 22:27:47,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408980575bc060ce15eaa8 HTTP/1.1" 502 150 2025-12-17 22:27:47,856 - lava_callback - ERROR - Error processing callback for node 69408980575bc060ce15eaa8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408980575bc060ce15eaa8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408980575bc060ce15eaa8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408980575bc060ce15eaa8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:47,858 - lava_callback - INFO - Starting async job submit for node 69408992575bc060ce15eae1 2025-12-17 22:27:47,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:47,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408992575bc060ce15eae1 HTTP/1.1" 502 150 2025-12-17 22:27:47,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408992575bc060ce15eae1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:47,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408992575bc060ce15eae1 2025-12-17 22:27:47,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408992575bc060ce15eae1 HTTP/1.1" 502 150 2025-12-17 22:27:47,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408992575bc060ce15eae1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:48,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898b575bc060ce15ead2 2025-12-17 22:27:48,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898b575bc060ce15ead2 HTTP/1.1" 502 150 2025-12-17 22:27:48,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898b575bc060ce15ead2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:48,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898d575bc060ce15ead5 2025-12-17 22:27:48,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898d575bc060ce15ead5 HTTP/1.1" 502 150 2025-12-17 22:27:48,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898d575bc060ce15ead5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:48,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408981575bc060ce15eaaf 2025-12-17 22:27:48,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408981575bc060ce15eaaf HTTP/1.1" 502 150 2025-12-17 22:27:48,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408981575bc060ce15eaaf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:49,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408983575bc060ce15eab4 2025-12-17 22:27:49,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408983575bc060ce15eab4 HTTP/1.1" 502 150 2025-12-17 22:27:49,625 - lava_callback - ERROR - Error processing callback for node 69408983575bc060ce15eab4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408983575bc060ce15eab4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408983575bc060ce15eab4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408983575bc060ce15eab4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:49,627 - lava_callback - INFO - Starting async job submit for node 69408993575bc060ce15eae5 2025-12-17 22:27:49,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:49,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408993575bc060ce15eae5 HTTP/1.1" 502 150 2025-12-17 22:27:49,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408993575bc060ce15eae5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:49,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408993575bc060ce15eae5 2025-12-17 22:27:49,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408993575bc060ce15eae5 HTTP/1.1" 502 150 2025-12-17 22:27:49,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408993575bc060ce15eae5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:49,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898e575bc060ce15ead8 2025-12-17 22:27:49,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898e575bc060ce15ead8 HTTP/1.1" 502 150 2025-12-17 22:27:49,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898e575bc060ce15ead8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:49,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408992575bc060ce15eae1 2025-12-17 22:27:49,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408992575bc060ce15eae1 HTTP/1.1" 502 150 2025-12-17 22:27:49,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408992575bc060ce15eae1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:51,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408912575bc060ce15e91c HTTP/1.1" 502 150 2025-12-17 22:27:51,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408912575bc060ce15e91c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:51,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408993575bc060ce15eae5 2025-12-17 22:27:51,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408993575bc060ce15eae5 HTTP/1.1" 502 150 2025-12-17 22:27:51,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408993575bc060ce15eae5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:52,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408984575bc060ce15eab7 2025-12-17 22:27:52,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408984575bc060ce15eab7 HTTP/1.1" 502 150 2025-12-17 22:27:52,622 - lava_callback - ERROR - Error processing callback for node 69408984575bc060ce15eab7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408984575bc060ce15eab7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408984575bc060ce15eab7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408984575bc060ce15eab7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:52,624 - lava_callback - INFO - Starting async job submit for node 69408994575bc060ce15eaeb 2025-12-17 22:27:52,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:52,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408994575bc060ce15eaeb HTTP/1.1" 502 150 2025-12-17 22:27:52,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408994575bc060ce15eaeb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:52,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408994575bc060ce15eaeb 2025-12-17 22:27:52,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408994575bc060ce15eaeb HTTP/1.1" 502 150 2025-12-17 22:27:52,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408994575bc060ce15eaeb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:52,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898f575bc060ce15eadb 2025-12-17 22:27:52,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898f575bc060ce15eadb HTTP/1.1" 502 150 2025-12-17 22:27:52,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940898f575bc060ce15eadb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:53,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408992575bc060ce15eae1 2025-12-17 22:27:53,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408992575bc060ce15eae1 HTTP/1.1" 502 150 2025-12-17 22:27:53,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408992575bc060ce15eae1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:54,564 - lava_callback - INFO - Received callback for node 69408b56575bc060ce15f1a1 INFO: 172.21.0.1:40136 - "POST /node/69408b56575bc060ce15f1a1 HTTP/1.0" 202 Accepted 2025-12-17 22:27:54,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408994575bc060ce15eaeb 2025-12-17 22:27:54,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408994575bc060ce15eaeb HTTP/1.1" 502 150 2025-12-17 22:27:54,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408994575bc060ce15eaeb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:54,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408990575bc060ce15eade 2025-12-17 22:27:54,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408990575bc060ce15eade HTTP/1.1" 502 150 2025-12-17 22:27:54,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408990575bc060ce15eade'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:55,006 - lava_callback - INFO - Received callback for node 69409287575bc060ce160adf INFO: 172.21.0.1:40150 - "POST /node/69409287575bc060ce160adf HTTP/1.0" 202 Accepted 2025-12-17 22:27:55,260 - lava_callback - INFO - Received callback for node 69408b5b575bc060ce15f1aa INFO: 172.21.0.1:40154 - "POST /node/69408b5b575bc060ce15f1aa HTTP/1.0" 202 Accepted 2025-12-17 22:27:55,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408993575bc060ce15eae5 2025-12-17 22:27:55,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408993575bc060ce15eae5 HTTP/1.1" 502 150 2025-12-17 22:27:55,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408993575bc060ce15eae5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:55,932 - lava_callback - INFO - Received callback for node 69408b5c575bc060ce15f1ad INFO: 172.21.0.1:40166 - "POST /node/69408b5c575bc060ce15f1ad HTTP/1.0" 202 Accepted INFO: 172.21.0.1:40170 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:27:56,606 - lava_callback - INFO - Received callback for node 69408b5d575bc060ce15f1b0 INFO: 172.21.0.1:40180 - "POST /node/69408b5d575bc060ce15f1b0 HTTP/1.0" 202 Accepted 2025-12-17 22:27:57,281 - lava_callback - INFO - Received callback for node 69408b5f575bc060ce15f1b3 INFO: 172.21.0.1:40188 - "POST /node/69408b5f575bc060ce15f1b3 HTTP/1.0" 202 Accepted 2025-12-17 22:27:57,361 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408985575bc060ce15eaba 2025-12-17 22:27:57,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408985575bc060ce15eaba HTTP/1.1" 502 150 2025-12-17 22:27:57,363 - lava_callback - ERROR - Error processing callback for node 69408985575bc060ce15eaba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408985575bc060ce15eaba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408985575bc060ce15eaba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408985575bc060ce15eaba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:57,364 - lava_callback - INFO - Starting async job submit for node 69408995575bc060ce15eaf0 2025-12-17 22:27:57,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:57,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408995575bc060ce15eaf0 HTTP/1.1" 502 150 2025-12-17 22:27:57,372 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408995575bc060ce15eaf0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:57,373 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408995575bc060ce15eaf0 2025-12-17 22:27:57,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408995575bc060ce15eaf0 HTTP/1.1" 502 150 2025-12-17 22:27:57,373 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408995575bc060ce15eaf0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:57,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408986575bc060ce15eac2 2025-12-17 22:27:57,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408986575bc060ce15eac2 HTTP/1.1" 502 150 2025-12-17 22:27:57,908 - lava_callback - ERROR - Error processing callback for node 69408986575bc060ce15eac2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408986575bc060ce15eac2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408986575bc060ce15eac2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408986575bc060ce15eac2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:27:57,910 - lava_callback - INFO - Starting async job submit for node 694271b81b61f1c6005f71fd 2025-12-17 22:27:57,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:27:57,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694271b81b61f1c6005f71fd HTTP/1.1" 502 150 2025-12-17 22:27:57,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694271b81b61f1c6005f71fd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:57,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694271b81b61f1c6005f71fd 2025-12-17 22:27:57,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694271b81b61f1c6005f71fd HTTP/1.1" 502 150 2025-12-17 22:27:57,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694271b81b61f1c6005f71fd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:57,942 - lava_callback - INFO - Received callback for node 69408b60575bc060ce15f1b8 INFO: 172.21.0.1:40194 - "POST /node/69408b60575bc060ce15f1b8 HTTP/1.0" 202 Accepted 2025-12-17 22:27:58,622 - lava_callback - INFO - Received callback for node 69408b61575bc060ce15f1bf INFO: 172.21.0.1:40204 - "POST /node/69408b61575bc060ce15f1bf HTTP/1.0" 202 Accepted 2025-12-17 22:27:58,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408994575bc060ce15eaeb 2025-12-17 22:27:58,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408994575bc060ce15eaeb HTTP/1.1" 502 150 2025-12-17 22:27:58,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408994575bc060ce15eaeb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:59,280 - lava_callback - INFO - Received callback for node 69408b63575bc060ce15f1c6 INFO: 172.21.0.1:40212 - "POST /node/69408b63575bc060ce15f1c6 HTTP/1.0" 202 Accepted 2025-12-17 22:27:59,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408912575bc060ce15e91c 2025-12-17 22:27:59,374 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408995575bc060ce15eaf0 2025-12-17 22:27:59,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408995575bc060ce15eaf0 HTTP/1.1" 502 150 2025-12-17 22:27:59,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408995575bc060ce15eaf0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:59,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694271b81b61f1c6005f71fd 2025-12-17 22:27:59,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694271b81b61f1c6005f71fd HTTP/1.1" 502 150 2025-12-17 22:27:59,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694271b81b61f1c6005f71fd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:27:59,935 - lava_callback - INFO - Received callback for node 69408b64575bc060ce15f1cb INFO: 172.21.0.1:51354 - "POST /node/69408b64575bc060ce15f1cb HTTP/1.0" 202 Accepted 2025-12-17 22:28:00,619 - lava_callback - INFO - Received callback for node 69408b65575bc060ce15f1d2 INFO: 172.21.0.1:51362 - "POST /node/69408b65575bc060ce15f1d2 HTTP/1.0" 202 Accepted 2025-12-17 22:28:00,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940897d575bc060ce15ea9f HTTP/1.1" 502 150 2025-12-17 22:28:00,816 - lava_callback - ERROR - Error processing callback for node 6940897d575bc060ce15ea9f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940897d575bc060ce15ea9f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940897d575bc060ce15ea9f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940897d575bc060ce15ea9f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:00,818 - lava_callback - INFO - Starting async job submit for node 69408997575bc060ce15eaf3 2025-12-17 22:28:00,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:00,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408997575bc060ce15eaf3 HTTP/1.1" 502 150 2025-12-17 22:28:00,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408997575bc060ce15eaf3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:00,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408997575bc060ce15eaf3 2025-12-17 22:28:00,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408997575bc060ce15eaf3 HTTP/1.1" 502 150 2025-12-17 22:28:00,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408997575bc060ce15eaf3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:01,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408988575bc060ce15eac5 2025-12-17 22:28:01,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408989575bc060ce15eac8 2025-12-17 22:28:01,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408988575bc060ce15eac5 HTTP/1.1" 502 150 2025-12-17 22:28:01,542 - lava_callback - ERROR - Error processing callback for node 69408988575bc060ce15eac5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408988575bc060ce15eac5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408988575bc060ce15eac5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408988575bc060ce15eac5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:01,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408989575bc060ce15eac8 HTTP/1.1" 502 150 2025-12-17 22:28:01,544 - lava_callback - ERROR - Error processing callback for node 69408989575bc060ce15eac8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408989575bc060ce15eac8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408989575bc060ce15eac8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408989575bc060ce15eac8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:01,545 - lava_callback - INFO - Starting async job submit for node 69408998575bc060ce15eb12 2025-12-17 22:28:01,546 - lava_callback - INFO - Starting async job submit for node 6940899b575bc060ce15eb49 2025-12-17 22:28:01,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:01,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:01,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408998575bc060ce15eb12 HTTP/1.1" 502 150 2025-12-17 22:28:01,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408998575bc060ce15eb12'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:01,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408998575bc060ce15eb12 2025-12-17 22:28:01,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408998575bc060ce15eb12 HTTP/1.1" 502 150 2025-12-17 22:28:01,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408998575bc060ce15eb12'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:01,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899b575bc060ce15eb49 HTTP/1.1" 502 150 2025-12-17 22:28:01,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899b575bc060ce15eb49'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:01,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899b575bc060ce15eb49 2025-12-17 22:28:01,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899b575bc060ce15eb49 HTTP/1.1" 502 150 2025-12-17 22:28:01,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899b575bc060ce15eb49'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:01,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408992575bc060ce15eae1 2025-12-17 22:28:01,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408992575bc060ce15eae1 HTTP/1.1" 502 150 2025-12-17 22:28:01,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408992575bc060ce15eae1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51370 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:28:02,458 - lava_callback - INFO - Received callback for node 69408b69575bc060ce15f1d8 INFO: 172.21.0.1:51384 - "POST /node/69408b69575bc060ce15f1d8 HTTP/1.0" 202 Accepted 2025-12-17 22:28:02,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408997575bc060ce15eaf3 2025-12-17 22:28:02,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408997575bc060ce15eaf3 HTTP/1.1" 502 150 2025-12-17 22:28:02,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408997575bc060ce15eaf3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:03,249 - lava_callback - INFO - Received callback for node 69408b6b575bc060ce15f1db INFO: 172.21.0.1:51390 - "POST /node/69408b6b575bc060ce15f1db HTTP/1.0" 202 Accepted 2025-12-17 22:28:03,375 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408995575bc060ce15eaf0 2025-12-17 22:28:03,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408995575bc060ce15eaf0 HTTP/1.1" 502 150 2025-12-17 22:28:03,377 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408995575bc060ce15eaf0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:03,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898a575bc060ce15eacf 2025-12-17 22:28:03,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898a575bc060ce15eacf HTTP/1.1" 502 150 2025-12-17 22:28:03,521 - lava_callback - ERROR - Error processing callback for node 6940898a575bc060ce15eacf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940898a575bc060ce15eacf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940898a575bc060ce15eacf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940898a575bc060ce15eacf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:03,524 - lava_callback - INFO - Starting async job submit for node 6940899c575bc060ce15eb5c 2025-12-17 22:28:03,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:03,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899c575bc060ce15eb5c HTTP/1.1" 502 150 2025-12-17 22:28:03,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899c575bc060ce15eb5c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:03,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899c575bc060ce15eb5c 2025-12-17 22:28:03,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899c575bc060ce15eb5c HTTP/1.1" 502 150 2025-12-17 22:28:03,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899c575bc060ce15eb5c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:03,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408998575bc060ce15eb12 2025-12-17 22:28:03,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408998575bc060ce15eb12 HTTP/1.1" 502 150 2025-12-17 22:28:03,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408998575bc060ce15eb12'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:03,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899b575bc060ce15eb49 2025-12-17 22:28:03,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899b575bc060ce15eb49 HTTP/1.1" 502 150 2025-12-17 22:28:03,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899b575bc060ce15eb49'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:03,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408993575bc060ce15eae5 2025-12-17 22:28:03,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408993575bc060ce15eae5 HTTP/1.1" 502 150 2025-12-17 22:28:03,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408993575bc060ce15eae5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:03,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694271b81b61f1c6005f71fd 2025-12-17 22:28:03,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694271b81b61f1c6005f71fd HTTP/1.1" 502 150 2025-12-17 22:28:03,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694271b81b61f1c6005f71fd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:04,405 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898b575bc060ce15ead2 2025-12-17 22:28:04,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898b575bc060ce15ead2 HTTP/1.1" 502 150 2025-12-17 22:28:04,406 - lava_callback - ERROR - Error processing callback for node 6940898b575bc060ce15ead2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940898b575bc060ce15ead2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940898b575bc060ce15ead2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940898b575bc060ce15ead2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:04,408 - lava_callback - INFO - Starting async job submit for node 6940899e575bc060ce15eb5f 2025-12-17 22:28:04,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:04,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899e575bc060ce15eb5f HTTP/1.1" 502 150 2025-12-17 22:28:04,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899e575bc060ce15eb5f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:04,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899e575bc060ce15eb5f 2025-12-17 22:28:04,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899e575bc060ce15eb5f HTTP/1.1" 502 150 2025-12-17 22:28:04,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899e575bc060ce15eb5f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:04,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898d575bc060ce15ead5 2025-12-17 22:28:04,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898d575bc060ce15ead5 HTTP/1.1" 502 150 2025-12-17 22:28:04,473 - lava_callback - ERROR - Error processing callback for node 6940898d575bc060ce15ead5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940898d575bc060ce15ead5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940898d575bc060ce15ead5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940898d575bc060ce15ead5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:04,475 - lava_callback - INFO - Starting async job submit for node 6940899f575bc060ce15eb62 2025-12-17 22:28:04,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:04,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899f575bc060ce15eb62 HTTP/1.1" 502 150 2025-12-17 22:28:04,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899f575bc060ce15eb62'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:04,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899f575bc060ce15eb62 2025-12-17 22:28:04,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899f575bc060ce15eb62 HTTP/1.1" 502 150 2025-12-17 22:28:04,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899f575bc060ce15eb62'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:04,721 - lava_callback - INFO - Received callback for node 69408b67575bc060ce15f1d5 INFO: 172.21.0.1:51392 - "POST /node/69408b67575bc060ce15f1d5 HTTP/1.0" 202 Accepted 2025-12-17 22:28:04,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408981575bc060ce15eaaf 2025-12-17 22:28:04,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408981575bc060ce15eaaf HTTP/1.1" 502 150 2025-12-17 22:28:04,757 - lava_callback - ERROR - Error processing callback for node 69408981575bc060ce15eaaf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408981575bc060ce15eaaf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408981575bc060ce15eaaf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408981575bc060ce15eaaf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:04,760 - lava_callback - INFO - Starting async job submit for node 694089a0575bc060ce15eb65 2025-12-17 22:28:04,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:04,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a0575bc060ce15eb65 HTTP/1.1" 502 150 2025-12-17 22:28:04,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a0575bc060ce15eb65'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:04,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a0575bc060ce15eb65 2025-12-17 22:28:04,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a0575bc060ce15eb65 HTTP/1.1" 502 150 2025-12-17 22:28:04,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a0575bc060ce15eb65'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:05,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899c575bc060ce15eb5c 2025-12-17 22:28:05,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899c575bc060ce15eb5c HTTP/1.1" 502 150 2025-12-17 22:28:05,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899c575bc060ce15eb5c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:05,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898e575bc060ce15ead8 2025-12-17 22:28:05,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898e575bc060ce15ead8 HTTP/1.1" 502 150 2025-12-17 22:28:05,735 - lava_callback - ERROR - Error processing callback for node 6940898e575bc060ce15ead8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940898e575bc060ce15ead8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940898e575bc060ce15ead8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940898e575bc060ce15ead8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:05,738 - lava_callback - INFO - Starting async job submit for node 694089a2575bc060ce15eb68 2025-12-17 22:28:05,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:05,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a2575bc060ce15eb68 HTTP/1.1" 502 150 2025-12-17 22:28:05,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a2575bc060ce15eb68'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:05,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a2575bc060ce15eb68 2025-12-17 22:28:05,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a2575bc060ce15eb68 HTTP/1.1" 502 150 2025-12-17 22:28:05,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a2575bc060ce15eb68'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:06,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899e575bc060ce15eb5f 2025-12-17 22:28:06,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899e575bc060ce15eb5f HTTP/1.1" 502 150 2025-12-17 22:28:06,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899e575bc060ce15eb5f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:06,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899f575bc060ce15eb62 2025-12-17 22:28:06,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899f575bc060ce15eb62 HTTP/1.1" 502 150 2025-12-17 22:28:06,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899f575bc060ce15eb62'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51396 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:28:06,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408994575bc060ce15eaeb 2025-12-17 22:28:06,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408994575bc060ce15eaeb HTTP/1.1" 502 150 2025-12-17 22:28:06,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408994575bc060ce15eaeb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:06,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a0575bc060ce15eb65 2025-12-17 22:28:06,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a0575bc060ce15eb65 HTTP/1.1" 502 150 2025-12-17 22:28:06,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a0575bc060ce15eb65'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:06,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408997575bc060ce15eaf3 2025-12-17 22:28:06,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408997575bc060ce15eaf3 HTTP/1.1" 502 150 2025-12-17 22:28:06,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408997575bc060ce15eaf3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:07,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408998575bc060ce15eb12 2025-12-17 22:28:07,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408998575bc060ce15eb12 HTTP/1.1" 502 150 2025-12-17 22:28:07,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408998575bc060ce15eb12'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:07,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899b575bc060ce15eb49 2025-12-17 22:28:07,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899b575bc060ce15eb49 HTTP/1.1" 502 150 2025-12-17 22:28:07,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899b575bc060ce15eb49'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:07,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a2575bc060ce15eb68 2025-12-17 22:28:07,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a2575bc060ce15eb68 HTTP/1.1" 502 150 2025-12-17 22:28:07,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a2575bc060ce15eb68'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:08,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940898f575bc060ce15eadb 2025-12-17 22:28:08,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940898f575bc060ce15eadb HTTP/1.1" 502 150 2025-12-17 22:28:08,661 - lava_callback - ERROR - Error processing callback for node 6940898f575bc060ce15eadb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940898f575bc060ce15eadb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940898f575bc060ce15eadb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940898f575bc060ce15eadb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:08,662 - lava_callback - INFO - Starting async job submit for node 694089a3575bc060ce15eb70 2025-12-17 22:28:08,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:08,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a3575bc060ce15eb70 HTTP/1.1" 502 150 2025-12-17 22:28:08,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a3575bc060ce15eb70'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:08,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a3575bc060ce15eb70 2025-12-17 22:28:08,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a3575bc060ce15eb70 HTTP/1.1" 502 150 2025-12-17 22:28:08,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a3575bc060ce15eb70'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:09,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899c575bc060ce15eb5c 2025-12-17 22:28:09,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899c575bc060ce15eb5c HTTP/1.1" 502 150 2025-12-17 22:28:09,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899c575bc060ce15eb5c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:10,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899e575bc060ce15eb5f 2025-12-17 22:28:10,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899e575bc060ce15eb5f HTTP/1.1" 502 150 2025-12-17 22:28:10,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899e575bc060ce15eb5f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:10,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899f575bc060ce15eb62 2025-12-17 22:28:10,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899f575bc060ce15eb62 HTTP/1.1" 502 150 2025-12-17 22:28:10,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899f575bc060ce15eb62'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:10,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a3575bc060ce15eb70 2025-12-17 22:28:10,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a3575bc060ce15eb70 HTTP/1.1" 502 150 2025-12-17 22:28:10,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a3575bc060ce15eb70'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:10,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408990575bc060ce15eade 2025-12-17 22:28:10,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408990575bc060ce15eade HTTP/1.1" 502 150 2025-12-17 22:28:10,719 - lava_callback - ERROR - Error processing callback for node 69408990575bc060ce15eade: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408990575bc060ce15eade (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408990575bc060ce15eade (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408990575bc060ce15eade (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:10,721 - lava_callback - INFO - Starting async job submit for node 694089a4575bc060ce15eb73 2025-12-17 22:28:10,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:10,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a4575bc060ce15eb73 HTTP/1.1" 502 150 2025-12-17 22:28:10,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a4575bc060ce15eb73'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:10,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a4575bc060ce15eb73 2025-12-17 22:28:10,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a4575bc060ce15eb73 HTTP/1.1" 502 150 2025-12-17 22:28:10,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a4575bc060ce15eb73'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:10,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a0575bc060ce15eb65 2025-12-17 22:28:10,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a0575bc060ce15eb65 HTTP/1.1" 502 150 2025-12-17 22:28:10,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a0575bc060ce15eb65'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:11,377 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408995575bc060ce15eaf0 2025-12-17 22:28:11,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a2575bc060ce15eb68 2025-12-17 22:28:11,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a2575bc060ce15eb68 HTTP/1.1" 502 150 2025-12-17 22:28:11,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a2575bc060ce15eb68'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:11,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694271b81b61f1c6005f71fd 2025-12-17 22:28:11,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694271b81b61f1c6005f71fd HTTP/1.1" 502 150 2025-12-17 22:28:11,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694271b81b61f1c6005f71fd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:12,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a4575bc060ce15eb73 2025-12-17 22:28:12,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a4575bc060ce15eb73 HTTP/1.1" 502 150 2025-12-17 22:28:12,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a4575bc060ce15eb73'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:14,568 - lava_callback - INFO - Received callback for node 69408b6c575bc060ce15f1de INFO: 172.21.0.1:60412 - "POST /node/69408b6c575bc060ce15f1de HTTP/1.0" 202 Accepted 2025-12-17 22:28:14,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a3575bc060ce15eb70 2025-12-17 22:28:14,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a3575bc060ce15eb70 HTTP/1.1" 502 150 2025-12-17 22:28:14,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a3575bc060ce15eb70'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:14,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408997575bc060ce15eaf3 2025-12-17 22:28:14,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408997575bc060ce15eaf3 HTTP/1.1" 502 150 2025-12-17 22:28:14,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408997575bc060ce15eaf3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60420 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:28:15,252 - lava_callback - INFO - Received callback for node 69408b71575bc060ce15f1e7 INFO: 172.21.0.1:60426 - "POST /node/69408b71575bc060ce15f1e7 HTTP/1.0" 202 Accepted 2025-12-17 22:28:15,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408998575bc060ce15eb12 2025-12-17 22:28:15,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408998575bc060ce15eb12 HTTP/1.1" 502 150 2025-12-17 22:28:15,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408998575bc060ce15eb12'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:15,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899b575bc060ce15eb49 2025-12-17 22:28:15,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899b575bc060ce15eb49 HTTP/1.1" 502 150 2025-12-17 22:28:15,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899b575bc060ce15eb49'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:15,907 - lava_callback - INFO - Received callback for node 69408b72575bc060ce15f1ea INFO: 172.21.0.1:60442 - "POST /node/69408b72575bc060ce15f1ea HTTP/1.0" 202 Accepted INFO: 172.21.0.1:60446 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:28:16,589 - lava_callback - INFO - Received callback for node 69408b73575bc060ce15f1ed INFO: 172.21.0.1:60460 - "POST /node/69408b73575bc060ce15f1ed HTTP/1.0" 202 Accepted 2025-12-17 22:28:16,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a4575bc060ce15eb73 2025-12-17 22:28:16,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a4575bc060ce15eb73 HTTP/1.1" 502 150 2025-12-17 22:28:16,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a4575bc060ce15eb73'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:17,277 - lava_callback - INFO - Received callback for node 69408b75575bc060ce15f1f4 INFO: 172.21.0.1:60466 - "POST /node/69408b75575bc060ce15f1f4 HTTP/1.0" 202 Accepted 2025-12-17 22:28:17,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899c575bc060ce15eb5c 2025-12-17 22:28:17,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899c575bc060ce15eb5c HTTP/1.1" 502 150 2025-12-17 22:28:17,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899c575bc060ce15eb5c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:17,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408992575bc060ce15eae1 2025-12-17 22:28:17,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408992575bc060ce15eae1 HTTP/1.1" 502 150 2025-12-17 22:28:17,873 - lava_callback - ERROR - Error processing callback for node 69408992575bc060ce15eae1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408992575bc060ce15eae1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408992575bc060ce15eae1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408992575bc060ce15eae1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:17,875 - lava_callback - INFO - Starting async job submit for node 694089a5575bc060ce15eb76 2025-12-17 22:28:17,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:17,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a5575bc060ce15eb76 HTTP/1.1" 502 150 2025-12-17 22:28:17,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a5575bc060ce15eb76'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:17,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a5575bc060ce15eb76 2025-12-17 22:28:17,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a5575bc060ce15eb76 HTTP/1.1" 502 150 2025-12-17 22:28:17,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a5575bc060ce15eb76'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:17,964 - lava_callback - INFO - Received callback for node 69408b76575bc060ce15f1fb INFO: 172.21.0.1:60476 - "POST /node/69408b76575bc060ce15f1fb HTTP/1.0" 202 Accepted 2025-12-17 22:28:18,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899e575bc060ce15eb5f 2025-12-17 22:28:18,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899e575bc060ce15eb5f HTTP/1.1" 502 150 2025-12-17 22:28:18,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899e575bc060ce15eb5f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:18,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899f575bc060ce15eb62 2025-12-17 22:28:18,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899f575bc060ce15eb62 HTTP/1.1" 502 150 2025-12-17 22:28:18,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940899f575bc060ce15eb62'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:18,645 - lava_callback - INFO - Received callback for node 69408b77575bc060ce15f202 INFO: 172.21.0.1:60492 - "POST /node/69408b77575bc060ce15f202 HTTP/1.0" 202 Accepted 2025-12-17 22:28:18,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a0575bc060ce15eb65 2025-12-17 22:28:18,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a0575bc060ce15eb65 HTTP/1.1" 502 150 2025-12-17 22:28:18,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a0575bc060ce15eb65'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:19,314 - lava_callback - INFO - Received callback for node 69408b79575bc060ce15f207 INFO: 172.21.0.1:60496 - "POST /node/69408b79575bc060ce15f207 HTTP/1.0" 202 Accepted 2025-12-17 22:28:19,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408993575bc060ce15eae5 2025-12-17 22:28:19,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408993575bc060ce15eae5 HTTP/1.1" 502 150 2025-12-17 22:28:19,641 - lava_callback - ERROR - Error processing callback for node 69408993575bc060ce15eae5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408993575bc060ce15eae5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408993575bc060ce15eae5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408993575bc060ce15eae5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:19,642 - lava_callback - INFO - Starting async job submit for node 694089a7575bc060ce15eb80 2025-12-17 22:28:19,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:19,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a7575bc060ce15eb80 HTTP/1.1" 502 150 2025-12-17 22:28:19,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a7575bc060ce15eb80'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:19,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a7575bc060ce15eb80 2025-12-17 22:28:19,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a7575bc060ce15eb80 HTTP/1.1" 502 150 2025-12-17 22:28:19,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a7575bc060ce15eb80'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:19,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a2575bc060ce15eb68 2025-12-17 22:28:19,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a2575bc060ce15eb68 HTTP/1.1" 502 150 2025-12-17 22:28:19,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a2575bc060ce15eb68'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:19,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a5575bc060ce15eb76 2025-12-17 22:28:19,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a5575bc060ce15eb76 HTTP/1.1" 502 150 2025-12-17 22:28:19,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a5575bc060ce15eb76'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:20,000 - lava_callback - INFO - Received callback for node 69408b7a575bc060ce15f20e INFO: 172.21.0.1:43182 - "POST /node/69408b7a575bc060ce15f20e HTTP/1.0" 202 Accepted 2025-12-17 22:28:20,643 - lava_callback - INFO - Received callback for node 69408b7b575bc060ce15f213 INFO: 172.21.0.1:43184 - "POST /node/69408b7b575bc060ce15f213 HTTP/1.0" 202 Accepted 2025-12-17 22:28:21,314 - lava_callback - INFO - Received callback for node 69408b7d575bc060ce15f21d INFO: 172.21.0.1:43196 - "POST /node/69408b7d575bc060ce15f21d HTTP/1.0" 202 Accepted 2025-12-17 22:28:21,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a7575bc060ce15eb80 2025-12-17 22:28:21,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a7575bc060ce15eb80 HTTP/1.1" 502 150 2025-12-17 22:28:21,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a7575bc060ce15eb80'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:22,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408994575bc060ce15eaeb 2025-12-17 22:28:22,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408994575bc060ce15eaeb HTTP/1.1" 502 150 2025-12-17 22:28:22,638 - lava_callback - ERROR - Error processing callback for node 69408994575bc060ce15eaeb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408994575bc060ce15eaeb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408994575bc060ce15eaeb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408994575bc060ce15eaeb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:22,641 - lava_callback - INFO - Starting async job submit for node 694089ab575bc060ce15eb90 2025-12-17 22:28:22,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:22,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ab575bc060ce15eb90 HTTP/1.1" 502 150 2025-12-17 22:28:22,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ab575bc060ce15eb90'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:22,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ab575bc060ce15eb90 2025-12-17 22:28:22,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ab575bc060ce15eb90 HTTP/1.1" 502 150 2025-12-17 22:28:22,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ab575bc060ce15eb90'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:22,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a3575bc060ce15eb70 2025-12-17 22:28:22,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a3575bc060ce15eb70 HTTP/1.1" 502 150 2025-12-17 22:28:22,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a3575bc060ce15eb70'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:23,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a5575bc060ce15eb76 2025-12-17 22:28:23,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a5575bc060ce15eb76 HTTP/1.1" 502 150 2025-12-17 22:28:23,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a5575bc060ce15eb76'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:24,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ab575bc060ce15eb90 2025-12-17 22:28:24,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ab575bc060ce15eb90 HTTP/1.1" 502 150 2025-12-17 22:28:24,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ab575bc060ce15eb90'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:24,716 - lava_callback - INFO - Received callback for node 69408b82575bc060ce15f22e INFO: 172.21.0.1:43206 - "POST /node/69408b82575bc060ce15f22e HTTP/1.0" 202 Accepted 2025-12-17 22:28:24,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a4575bc060ce15eb73 2025-12-17 22:28:24,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a4575bc060ce15eb73 HTTP/1.1" 502 150 2025-12-17 22:28:24,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a4575bc060ce15eb73'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:25,362 - lava_callback - INFO - Received callback for node 69408b86575bc060ce15f234 INFO: 172.21.0.1:43214 - "POST /node/69408b86575bc060ce15f234 HTTP/1.0" 202 Accepted 2025-12-17 22:28:25,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a7575bc060ce15eb80 2025-12-17 22:28:25,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a7575bc060ce15eb80 HTTP/1.1" 502 150 2025-12-17 22:28:25,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a7575bc060ce15eb80'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43226 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:28:26,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408995575bc060ce15eaf0 HTTP/1.1" 502 150 2025-12-17 22:28:26,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408995575bc060ce15eaf0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:27,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694271b81b61f1c6005f71fd 2025-12-17 22:28:27,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694271b81b61f1c6005f71fd HTTP/1.1" 502 150 2025-12-17 22:28:27,930 - lava_callback - ERROR - Error processing callback for node 694271b81b61f1c6005f71fd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694271b81b61f1c6005f71fd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694271b81b61f1c6005f71fd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694271b81b61f1c6005f71fd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:27,932 - lava_callback - INFO - Starting async job submit for node 694089ae575bc060ce15eb98 2025-12-17 22:28:27,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:27,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ae575bc060ce15eb98 HTTP/1.1" 502 150 2025-12-17 22:28:27,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ae575bc060ce15eb98'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:27,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ae575bc060ce15eb98 2025-12-17 22:28:27,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ae575bc060ce15eb98 HTTP/1.1" 502 150 2025-12-17 22:28:27,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ae575bc060ce15eb98'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:28,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ab575bc060ce15eb90 2025-12-17 22:28:28,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ab575bc060ce15eb90 HTTP/1.1" 502 150 2025-12-17 22:28:28,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ab575bc060ce15eb90'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:29,218 - lava_callback - INFO - Received callback for node 69408b7f575bc060ce15f225 INFO: 172.21.0.1:43236 - "POST /node/69408b7f575bc060ce15f225 HTTP/1.0" 202 Accepted 2025-12-17 22:28:29,864 - lava_callback - INFO - Received callback for node 69408b80575bc060ce15f228 INFO: 172.21.0.1:56542 - "POST /node/69408b80575bc060ce15f228 HTTP/1.0" 202 Accepted 2025-12-17 22:28:29,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ae575bc060ce15eb98 2025-12-17 22:28:29,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ae575bc060ce15eb98 HTTP/1.1" 502 150 2025-12-17 22:28:29,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ae575bc060ce15eb98'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:30,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408912575bc060ce15e91c HTTP/1.1" 502 150 2025-12-17 22:28:30,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408912575bc060ce15e91c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:30,536 - lava_callback - INFO - Received callback for node 69408b81575bc060ce15f22b INFO: 172.21.0.1:56554 - "POST /node/69408b81575bc060ce15f22b HTTP/1.0" 202 Accepted 2025-12-17 22:28:30,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408997575bc060ce15eaf3 2025-12-17 22:28:30,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408997575bc060ce15eaf3 HTTP/1.1" 502 150 2025-12-17 22:28:30,833 - lava_callback - ERROR - Error processing callback for node 69408997575bc060ce15eaf3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408997575bc060ce15eaf3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408997575bc060ce15eaf3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408997575bc060ce15eaf3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:30,835 - lava_callback - INFO - Starting async job submit for node 694089a7575bc060ce15eb7b 2025-12-17 22:28:30,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:30,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a7575bc060ce15eb7b HTTP/1.1" 502 150 2025-12-17 22:28:30,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a7575bc060ce15eb7b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:30,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a7575bc060ce15eb7b 2025-12-17 22:28:30,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a7575bc060ce15eb7b HTTP/1.1" 502 150 2025-12-17 22:28:30,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a7575bc060ce15eb7b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:31,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408998575bc060ce15eb12 2025-12-17 22:28:31,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408998575bc060ce15eb12 HTTP/1.1" 502 150 2025-12-17 22:28:31,560 - lava_callback - ERROR - Error processing callback for node 69408998575bc060ce15eb12: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408998575bc060ce15eb12 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408998575bc060ce15eb12 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408998575bc060ce15eb12 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:31,562 - lava_callback - INFO - Starting async job submit for node 694089aa575bc060ce15eb8b 2025-12-17 22:28:31,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:31,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089aa575bc060ce15eb8b HTTP/1.1" 502 150 2025-12-17 22:28:31,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089aa575bc060ce15eb8b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:31,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089aa575bc060ce15eb8b 2025-12-17 22:28:31,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089aa575bc060ce15eb8b HTTP/1.1" 502 150 2025-12-17 22:28:31,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089aa575bc060ce15eb8b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:31,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899b575bc060ce15eb49 2025-12-17 22:28:31,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899b575bc060ce15eb49 HTTP/1.1" 502 150 2025-12-17 22:28:31,581 - lava_callback - ERROR - Error processing callback for node 6940899b575bc060ce15eb49: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940899b575bc060ce15eb49 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940899b575bc060ce15eb49 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940899b575bc060ce15eb49 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:31,582 - lava_callback - INFO - Starting async job submit for node 694089b5575bc060ce15eba8 2025-12-17 22:28:31,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:31,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b5575bc060ce15eba8 HTTP/1.1" 502 150 2025-12-17 22:28:31,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b5575bc060ce15eba8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:31,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b5575bc060ce15eba8 2025-12-17 22:28:31,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b5575bc060ce15eba8 HTTP/1.1" 502 150 2025-12-17 22:28:31,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b5575bc060ce15eba8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:31,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a5575bc060ce15eb76 2025-12-17 22:28:31,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a5575bc060ce15eb76 HTTP/1.1" 502 150 2025-12-17 22:28:31,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a5575bc060ce15eb76'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:32,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a7575bc060ce15eb7b 2025-12-17 22:28:32,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a7575bc060ce15eb7b HTTP/1.1" 502 150 2025-12-17 22:28:32,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a7575bc060ce15eb7b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:33,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899c575bc060ce15eb5c 2025-12-17 22:28:33,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899c575bc060ce15eb5c HTTP/1.1" 502 150 2025-12-17 22:28:33,541 - lava_callback - ERROR - Error processing callback for node 6940899c575bc060ce15eb5c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940899c575bc060ce15eb5c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940899c575bc060ce15eb5c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940899c575bc060ce15eb5c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:33,543 - lava_callback - INFO - Starting async job submit for node 694089b6575bc060ce15ebab 2025-12-17 22:28:33,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:33,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b6575bc060ce15ebab HTTP/1.1" 502 150 2025-12-17 22:28:33,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b6575bc060ce15ebab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:33,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b6575bc060ce15ebab 2025-12-17 22:28:33,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b6575bc060ce15ebab HTTP/1.1" 502 150 2025-12-17 22:28:33,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b6575bc060ce15ebab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:33,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089aa575bc060ce15eb8b 2025-12-17 22:28:33,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089aa575bc060ce15eb8b HTTP/1.1" 502 150 2025-12-17 22:28:33,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089aa575bc060ce15eb8b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:33,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b5575bc060ce15eba8 2025-12-17 22:28:33,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b5575bc060ce15eba8 HTTP/1.1" 502 150 2025-12-17 22:28:33,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b5575bc060ce15eba8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:33,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a7575bc060ce15eb80 2025-12-17 22:28:33,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a7575bc060ce15eb80 HTTP/1.1" 502 150 2025-12-17 22:28:33,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a7575bc060ce15eb80'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:33,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ae575bc060ce15eb98 2025-12-17 22:28:33,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ae575bc060ce15eb98 HTTP/1.1" 502 150 2025-12-17 22:28:33,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ae575bc060ce15eb98'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:34,409 - lava_callback - INFO - Received callback for node 69408c24575bc060ce15f477 INFO: 172.21.0.1:56564 - "POST /node/69408c24575bc060ce15f477 HTTP/1.0" 202 Accepted 2025-12-17 22:28:34,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899e575bc060ce15eb5f 2025-12-17 22:28:34,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899e575bc060ce15eb5f HTTP/1.1" 502 150 2025-12-17 22:28:34,424 - lava_callback - ERROR - Error processing callback for node 6940899e575bc060ce15eb5f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940899e575bc060ce15eb5f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940899e575bc060ce15eb5f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940899e575bc060ce15eb5f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:34,427 - lava_callback - INFO - Starting async job submit for node 694089b8575bc060ce15ebae 2025-12-17 22:28:34,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:34,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b8575bc060ce15ebae HTTP/1.1" 502 150 2025-12-17 22:28:34,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b8575bc060ce15ebae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:34,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b8575bc060ce15ebae 2025-12-17 22:28:34,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b8575bc060ce15ebae HTTP/1.1" 502 150 2025-12-17 22:28:34,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b8575bc060ce15ebae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:34,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940899f575bc060ce15eb62 2025-12-17 22:28:34,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940899f575bc060ce15eb62 HTTP/1.1" 502 150 2025-12-17 22:28:34,488 - lava_callback - ERROR - Error processing callback for node 6940899f575bc060ce15eb62: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940899f575bc060ce15eb62 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940899f575bc060ce15eb62 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940899f575bc060ce15eb62 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:34,489 - lava_callback - INFO - Starting async job submit for node 694089b9575bc060ce15ebb1 2025-12-17 22:28:34,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:34,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b9575bc060ce15ebb1 HTTP/1.1" 502 150 2025-12-17 22:28:34,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b9575bc060ce15ebb1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:34,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b9575bc060ce15ebb1 2025-12-17 22:28:34,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b9575bc060ce15ebb1 HTTP/1.1" 502 150 2025-12-17 22:28:34,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b9575bc060ce15ebb1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:34,556 - lava_callback - INFO - Received callback for node 69408b7e575bc060ce15f222 INFO: 172.21.0.1:56572 - "POST /node/69408b7e575bc060ce15f222 HTTP/1.0" 202 Accepted 2025-12-17 22:28:34,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a0575bc060ce15eb65 2025-12-17 22:28:34,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a0575bc060ce15eb65 HTTP/1.1" 502 150 2025-12-17 22:28:34,774 - lava_callback - ERROR - Error processing callback for node 694089a0575bc060ce15eb65: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a0575bc060ce15eb65 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a0575bc060ce15eb65 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a0575bc060ce15eb65 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:34,777 - lava_callback - INFO - Starting async job submit for node 694089ba575bc060ce15ebb4 2025-12-17 22:28:34,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:34,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ba575bc060ce15ebb4 HTTP/1.1" 502 150 2025-12-17 22:28:34,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ba575bc060ce15ebb4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:34,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ba575bc060ce15ebb4 2025-12-17 22:28:34,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ba575bc060ce15ebb4 HTTP/1.1" 502 150 2025-12-17 22:28:34,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ba575bc060ce15ebb4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:35,227 - lava_callback - INFO - Received callback for node 69408b85575bc060ce15f231 INFO: 172.21.0.1:56584 - "POST /node/69408b85575bc060ce15f231 HTTP/1.0" 202 Accepted 2025-12-17 22:28:35,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b6575bc060ce15ebab 2025-12-17 22:28:35,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b6575bc060ce15ebab HTTP/1.1" 502 150 2025-12-17 22:28:35,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b6575bc060ce15ebab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:35,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a2575bc060ce15eb68 2025-12-17 22:28:35,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a2575bc060ce15eb68 HTTP/1.1" 502 150 2025-12-17 22:28:35,752 - lava_callback - ERROR - Error processing callback for node 694089a2575bc060ce15eb68: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a2575bc060ce15eb68 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a2575bc060ce15eb68 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a2575bc060ce15eb68 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:35,754 - lava_callback - INFO - Starting async job submit for node 694089b2575bc060ce15eba2 2025-12-17 22:28:35,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:35,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b2575bc060ce15eba2 HTTP/1.1" 502 150 2025-12-17 22:28:35,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b2575bc060ce15eba2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:35,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b2575bc060ce15eba2 2025-12-17 22:28:35,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b2575bc060ce15eba2 HTTP/1.1" 502 150 2025-12-17 22:28:35,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b2575bc060ce15eba2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:35,888 - lava_callback - INFO - Received callback for node 69408b8a575bc060ce15f277 INFO: 172.21.0.1:56596 - "POST /node/69408b8a575bc060ce15f277 HTTP/1.0" 202 Accepted 2025-12-17 22:28:36,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b8575bc060ce15ebae 2025-12-17 22:28:36,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b8575bc060ce15ebae HTTP/1.1" 502 150 2025-12-17 22:28:36,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b8575bc060ce15ebae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:36,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b9575bc060ce15ebb1 2025-12-17 22:28:36,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b9575bc060ce15ebb1 HTTP/1.1" 502 150 2025-12-17 22:28:36,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b9575bc060ce15ebb1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56602 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:28:36,540 - lava_callback - INFO - Received callback for node 69408b8c575bc060ce15f298 INFO: 172.21.0.1:56604 - "POST /node/69408b8c575bc060ce15f298 HTTP/1.0" 202 Accepted 2025-12-17 22:28:36,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ab575bc060ce15eb90 2025-12-17 22:28:36,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ab575bc060ce15eb90 HTTP/1.1" 502 150 2025-12-17 22:28:36,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ab575bc060ce15eb90'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:36,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ba575bc060ce15ebb4 2025-12-17 22:28:36,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ba575bc060ce15ebb4 HTTP/1.1" 502 150 2025-12-17 22:28:36,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ba575bc060ce15ebb4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:36,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a7575bc060ce15eb7b 2025-12-17 22:28:36,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a7575bc060ce15eb7b HTTP/1.1" 502 150 2025-12-17 22:28:36,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a7575bc060ce15eb7b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:37,205 - lava_callback - INFO - Received callback for node 69408b91575bc060ce15f2a3 INFO: 172.21.0.1:56618 - "POST /node/69408b91575bc060ce15f2a3 HTTP/1.0" 202 Accepted 2025-12-17 22:28:37,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089aa575bc060ce15eb8b 2025-12-17 22:28:37,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089aa575bc060ce15eb8b HTTP/1.1" 502 150 2025-12-17 22:28:37,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089aa575bc060ce15eb8b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:37,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b5575bc060ce15eba8 2025-12-17 22:28:37,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b5575bc060ce15eba8 HTTP/1.1" 502 150 2025-12-17 22:28:37,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b5575bc060ce15eba8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:37,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b2575bc060ce15eba2 2025-12-17 22:28:37,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b2575bc060ce15eba2 HTTP/1.1" 502 150 2025-12-17 22:28:37,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b2575bc060ce15eba2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:38,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a3575bc060ce15eb70 2025-12-17 22:28:38,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a3575bc060ce15eb70 HTTP/1.1" 502 150 2025-12-17 22:28:38,675 - lava_callback - ERROR - Error processing callback for node 694089a3575bc060ce15eb70: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a3575bc060ce15eb70 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a3575bc060ce15eb70 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a3575bc060ce15eb70 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:38,677 - lava_callback - INFO - Starting async job submit for node 694089b3575bc060ce15eba5 2025-12-17 22:28:38,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:38,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b3575bc060ce15eba5 HTTP/1.1" 502 150 2025-12-17 22:28:38,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b3575bc060ce15eba5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:38,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b3575bc060ce15eba5 2025-12-17 22:28:38,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b3575bc060ce15eba5 HTTP/1.1" 502 150 2025-12-17 22:28:38,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b3575bc060ce15eba5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:39,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b6575bc060ce15ebab 2025-12-17 22:28:40,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b8575bc060ce15ebae 2025-12-17 22:28:40,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b8575bc060ce15ebae HTTP/1.1" 502 150 2025-12-17 22:28:40,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b8575bc060ce15ebae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:40,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b9575bc060ce15ebb1 2025-12-17 22:28:40,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b9575bc060ce15ebb1 HTTP/1.1" 502 150 2025-12-17 22:28:40,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b9575bc060ce15ebb1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:40,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b3575bc060ce15eba5 2025-12-17 22:28:40,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b3575bc060ce15eba5 HTTP/1.1" 502 150 2025-12-17 22:28:40,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b3575bc060ce15eba5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:40,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a4575bc060ce15eb73 2025-12-17 22:28:40,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a4575bc060ce15eb73 HTTP/1.1" 502 150 2025-12-17 22:28:40,735 - lava_callback - ERROR - Error processing callback for node 694089a4575bc060ce15eb73: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a4575bc060ce15eb73 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a4575bc060ce15eb73 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a4575bc060ce15eb73 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:40,738 - lava_callback - INFO - Starting async job submit for node 694089bd575bc060ce15ebc2 2025-12-17 22:28:40,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:40,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089bd575bc060ce15ebc2 HTTP/1.1" 502 150 2025-12-17 22:28:40,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089bd575bc060ce15ebc2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:40,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089bd575bc060ce15ebc2 2025-12-17 22:28:40,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089bd575bc060ce15ebc2 HTTP/1.1" 502 150 2025-12-17 22:28:40,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089bd575bc060ce15ebc2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:40,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ba575bc060ce15ebb4 2025-12-17 22:28:40,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ba575bc060ce15ebb4 HTTP/1.1" 502 150 2025-12-17 22:28:40,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ba575bc060ce15ebb4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:41,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b2575bc060ce15eba2 2025-12-17 22:28:41,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b2575bc060ce15eba2 HTTP/1.1" 502 150 2025-12-17 22:28:41,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b2575bc060ce15eba2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:41,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ae575bc060ce15eb98 2025-12-17 22:28:41,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ae575bc060ce15eb98 HTTP/1.1" 502 150 2025-12-17 22:28:41,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ae575bc060ce15eb98'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:42,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408995575bc060ce15eaf0 2025-12-17 22:28:42,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408995575bc060ce15eaf0 HTTP/1.1" 502 150 2025-12-17 22:28:42,674 - lava_callback - ERROR - Error processing callback for node 69408995575bc060ce15eaf0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408995575bc060ce15eaf0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408995575bc060ce15eaf0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408995575bc060ce15eaf0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:42,677 - lava_callback - INFO - Starting async job submit for node 694089bb575bc060ce15ebbb 2025-12-17 22:28:42,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:42,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089bb575bc060ce15ebbb HTTP/1.1" 502 150 2025-12-17 22:28:42,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089bb575bc060ce15ebbb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:42,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089bb575bc060ce15ebbb 2025-12-17 22:28:42,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089bb575bc060ce15ebbb HTTP/1.1" 502 150 2025-12-17 22:28:42,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089bb575bc060ce15ebbb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:42,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089bd575bc060ce15ebc2 2025-12-17 22:28:42,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089bd575bc060ce15ebc2 HTTP/1.1" 502 150 2025-12-17 22:28:42,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089bd575bc060ce15ebc2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:43,160 - lava_callback - INFO - Received callback for node 69408956575bc060ce15e9e2 INFO: 172.21.0.1:49660 - "POST /node/69408956575bc060ce15e9e2 HTTP/1.0" 202 Accepted 2025-12-17 22:28:44,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b3575bc060ce15eba5 2025-12-17 22:28:44,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b3575bc060ce15eba5 HTTP/1.1" 502 150 2025-12-17 22:28:44,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b3575bc060ce15eba5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:44,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089bb575bc060ce15ebbb 2025-12-17 22:28:44,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089bb575bc060ce15ebbb HTTP/1.1" 502 150 2025-12-17 22:28:44,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089bb575bc060ce15ebbb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:44,764 - lava_callback - INFO - Received callback for node 69408b8e575bc060ce15f2a0 INFO: 172.21.0.1:49662 - "POST /node/69408b8e575bc060ce15f2a0 HTTP/1.0" 202 Accepted 2025-12-17 22:28:44,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a7575bc060ce15eb7b 2025-12-17 22:28:44,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a7575bc060ce15eb7b HTTP/1.1" 502 150 2025-12-17 22:28:44,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089a7575bc060ce15eb7b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:45,412 - lava_callback - INFO - Received callback for node 69408b92575bc060ce15f2a6 INFO: 172.21.0.1:49670 - "POST /node/69408b92575bc060ce15f2a6 HTTP/1.0" 202 Accepted 2025-12-17 22:28:45,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089aa575bc060ce15eb8b 2025-12-17 22:28:45,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089aa575bc060ce15eb8b HTTP/1.1" 502 150 2025-12-17 22:28:45,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089aa575bc060ce15eb8b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:45,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b5575bc060ce15eba8 2025-12-17 22:28:45,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b5575bc060ce15eba8 HTTP/1.1" 502 150 2025-12-17 22:28:45,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b5575bc060ce15eba8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:46,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408912575bc060ce15e91c INFO: 172.21.0.1:49680 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:28:46,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089bd575bc060ce15ebc2 2025-12-17 22:28:46,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089bd575bc060ce15ebc2 HTTP/1.1" 502 150 2025-12-17 22:28:46,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089bd575bc060ce15ebc2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:47,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a5575bc060ce15eb76 2025-12-17 22:28:47,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a5575bc060ce15eb76 HTTP/1.1" 502 150 2025-12-17 22:28:47,912 - lava_callback - ERROR - Error processing callback for node 694089a5575bc060ce15eb76: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a5575bc060ce15eb76 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a5575bc060ce15eb76 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a5575bc060ce15eb76 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:47,914 - lava_callback - INFO - Starting async job submit for node 694089be575bc060ce15ebc5 2025-12-17 22:28:47,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:47,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089be575bc060ce15ebc5 HTTP/1.1" 502 150 2025-12-17 22:28:47,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089be575bc060ce15ebc5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:47,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089be575bc060ce15ebc5 2025-12-17 22:28:47,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089be575bc060ce15ebc5 HTTP/1.1" 502 150 2025-12-17 22:28:47,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089be575bc060ce15ebc5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:48,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b8575bc060ce15ebae 2025-12-17 22:28:48,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b8575bc060ce15ebae HTTP/1.1" 502 150 2025-12-17 22:28:48,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b8575bc060ce15ebae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:48,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b9575bc060ce15ebb1 2025-12-17 22:28:48,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b9575bc060ce15ebb1 HTTP/1.1" 502 150 2025-12-17 22:28:48,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b9575bc060ce15ebb1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:48,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089bb575bc060ce15ebbb 2025-12-17 22:28:48,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089bb575bc060ce15ebbb HTTP/1.1" 502 150 2025-12-17 22:28:48,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089bb575bc060ce15ebbb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:48,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ba575bc060ce15ebb4 2025-12-17 22:28:48,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ba575bc060ce15ebb4 HTTP/1.1" 502 150 2025-12-17 22:28:48,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ba575bc060ce15ebb4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:49,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a7575bc060ce15eb80 2025-12-17 22:28:49,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a7575bc060ce15eb80 HTTP/1.1" 502 150 2025-12-17 22:28:49,676 - lava_callback - ERROR - Error processing callback for node 694089a7575bc060ce15eb80: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a7575bc060ce15eb80 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a7575bc060ce15eb80 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a7575bc060ce15eb80 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:49,678 - lava_callback - INFO - Starting async job submit for node 694089c0575bc060ce15ebc8 2025-12-17 22:28:49,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:49,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c0575bc060ce15ebc8 HTTP/1.1" 502 150 2025-12-17 22:28:49,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c0575bc060ce15ebc8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:49,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c0575bc060ce15ebc8 2025-12-17 22:28:49,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c0575bc060ce15ebc8 HTTP/1.1" 502 150 2025-12-17 22:28:49,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c0575bc060ce15ebc8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:49,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b2575bc060ce15eba2 2025-12-17 22:28:49,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b2575bc060ce15eba2 HTTP/1.1" 502 150 2025-12-17 22:28:49,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b2575bc060ce15eba2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:49,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089be575bc060ce15ebc5 2025-12-17 22:28:49,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089be575bc060ce15ebc5 HTTP/1.1" 502 150 2025-12-17 22:28:49,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089be575bc060ce15ebc5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:51,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c0575bc060ce15ebc8 2025-12-17 22:28:52,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ab575bc060ce15eb90 2025-12-17 22:28:52,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ab575bc060ce15eb90 HTTP/1.1" 502 150 2025-12-17 22:28:52,660 - lava_callback - ERROR - Error processing callback for node 694089ab575bc060ce15eb90: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ab575bc060ce15eb90 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ab575bc060ce15eb90 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ab575bc060ce15eb90 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:52,662 - lava_callback - INFO - Starting async job submit for node 6940de35460bc8e2651133a4 2025-12-17 22:28:52,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:52,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940de35460bc8e2651133a4 HTTP/1.1" 502 150 2025-12-17 22:28:52,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b3575bc060ce15eba5 2025-12-17 22:28:52,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940de35460bc8e2651133a4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:52,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940de35460bc8e2651133a4 2025-12-17 22:28:52,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b3575bc060ce15eba5 HTTP/1.1" 502 150 2025-12-17 22:28:52,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b3575bc060ce15eba5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:52,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940de35460bc8e2651133a4 HTTP/1.1" 502 150 2025-12-17 22:28:52,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940de35460bc8e2651133a4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:53,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089be575bc060ce15ebc5 2025-12-17 22:28:53,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089be575bc060ce15ebc5 HTTP/1.1" 502 150 2025-12-17 22:28:53,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089be575bc060ce15ebc5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:54,575 - lava_callback - INFO - Received callback for node 69408b95575bc060ce15f2a9 INFO: 172.21.0.1:39916 - "POST /node/69408b95575bc060ce15f2a9 HTTP/1.0" 202 Accepted 2025-12-17 22:28:54,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940de35460bc8e2651133a4 2025-12-17 22:28:54,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940de35460bc8e2651133a4 HTTP/1.1" 502 150 2025-12-17 22:28:54,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940de35460bc8e2651133a4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:54,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089bd575bc060ce15ebc2 2025-12-17 22:28:54,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089bd575bc060ce15ebc2 HTTP/1.1" 502 150 2025-12-17 22:28:54,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089bd575bc060ce15ebc2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:54,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b6575bc060ce15ebab HTTP/1.1" 502 150 2025-12-17 22:28:54,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b6575bc060ce15ebab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:55,228 - lava_callback - INFO - Received callback for node 69408b9e575bc060ce15f2b9 INFO: 172.21.0.1:39922 - "POST /node/69408b9e575bc060ce15f2b9 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:39928 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:28:56,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089bb575bc060ce15ebbb 2025-12-17 22:28:56,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089bb575bc060ce15ebbb HTTP/1.1" 502 150 2025-12-17 22:28:56,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089bb575bc060ce15ebbb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:57,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ae575bc060ce15eb98 2025-12-17 22:28:57,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ae575bc060ce15eb98 HTTP/1.1" 502 150 2025-12-17 22:28:57,947 - lava_callback - ERROR - Error processing callback for node 694089ae575bc060ce15eb98: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ae575bc060ce15eb98 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ae575bc060ce15eb98 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ae575bc060ce15eb98 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:28:57,949 - lava_callback - INFO - Starting async job submit for node 694089c2575bc060ce15ebcb 2025-12-17 22:28:57,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:28:57,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c2575bc060ce15ebcb HTTP/1.1" 502 150 2025-12-17 22:28:57,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c2575bc060ce15ebcb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:57,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c2575bc060ce15ebcb 2025-12-17 22:28:57,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c2575bc060ce15ebcb HTTP/1.1" 502 150 2025-12-17 22:28:57,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c2575bc060ce15ebcb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:58,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940de35460bc8e2651133a4 2025-12-17 22:28:58,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940de35460bc8e2651133a4 HTTP/1.1" 502 150 2025-12-17 22:28:58,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940de35460bc8e2651133a4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:28:59,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c2575bc060ce15ebcb 2025-12-17 22:28:59,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c2575bc060ce15ebcb HTTP/1.1" 502 150 2025-12-17 22:28:59,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c2575bc060ce15ebcb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:00,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089a7575bc060ce15eb7b 2025-12-17 22:29:00,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089a7575bc060ce15eb7b HTTP/1.1" 502 150 2025-12-17 22:29:00,849 - lava_callback - ERROR - Error processing callback for node 694089a7575bc060ce15eb7b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a7575bc060ce15eb7b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a7575bc060ce15eb7b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089a7575bc060ce15eb7b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:00,850 - lava_callback - INFO - Starting async job submit for node 694089c6575bc060ce15ebd4 2025-12-17 22:29:00,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:00,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c6575bc060ce15ebd4 HTTP/1.1" 502 150 2025-12-17 22:29:00,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c6575bc060ce15ebd4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:00,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c6575bc060ce15ebd4 2025-12-17 22:29:00,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c6575bc060ce15ebd4 HTTP/1.1" 502 150 2025-12-17 22:29:00,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c6575bc060ce15ebd4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:01,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089aa575bc060ce15eb8b 2025-12-17 22:29:01,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089aa575bc060ce15eb8b HTTP/1.1" 502 150 2025-12-17 22:29:01,582 - lava_callback - ERROR - Error processing callback for node 694089aa575bc060ce15eb8b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089aa575bc060ce15eb8b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089aa575bc060ce15eb8b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089aa575bc060ce15eb8b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:01,584 - lava_callback - INFO - Starting async job submit for node 694089c8575bc060ce15ebd7 2025-12-17 22:29:01,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:01,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c8575bc060ce15ebd7 HTTP/1.1" 502 150 2025-12-17 22:29:01,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c8575bc060ce15ebd7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:01,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c8575bc060ce15ebd7 2025-12-17 22:29:01,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c8575bc060ce15ebd7 HTTP/1.1" 502 150 2025-12-17 22:29:01,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c8575bc060ce15ebd7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:01,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b5575bc060ce15eba8 2025-12-17 22:29:01,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b5575bc060ce15eba8 HTTP/1.1" 502 150 2025-12-17 22:29:01,593 - lava_callback - ERROR - Error processing callback for node 694089b5575bc060ce15eba8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b5575bc060ce15eba8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b5575bc060ce15eba8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b5575bc060ce15eba8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:01,595 - lava_callback - INFO - Starting async job submit for node 694089c9575bc060ce15ebda 2025-12-17 22:29:01,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:01,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c9575bc060ce15ebda HTTP/1.1" 502 150 2025-12-17 22:29:01,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c9575bc060ce15ebda'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:01,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c9575bc060ce15ebda 2025-12-17 22:29:01,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c9575bc060ce15ebda HTTP/1.1" 502 150 2025-12-17 22:29:01,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c9575bc060ce15ebda'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:01,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089be575bc060ce15ebc5 2025-12-17 22:29:01,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089be575bc060ce15ebc5 HTTP/1.1" 502 150 2025-12-17 22:29:01,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089be575bc060ce15ebc5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59920 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:29:02,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b6575bc060ce15ebab 2025-12-17 22:29:02,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b6575bc060ce15ebab HTTP/1.1" 502 150 2025-12-17 22:29:02,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089b6575bc060ce15ebab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:02,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c6575bc060ce15ebd4 2025-12-17 22:29:02,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c6575bc060ce15ebd4 HTTP/1.1" 502 150 2025-12-17 22:29:02,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c6575bc060ce15ebd4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:03,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c8575bc060ce15ebd7 2025-12-17 22:29:03,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c8575bc060ce15ebd7 HTTP/1.1" 502 150 2025-12-17 22:29:03,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c8575bc060ce15ebd7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:03,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c9575bc060ce15ebda 2025-12-17 22:29:03,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c9575bc060ce15ebda HTTP/1.1" 502 150 2025-12-17 22:29:03,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c9575bc060ce15ebda'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:03,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c2575bc060ce15ebcb 2025-12-17 22:29:03,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c2575bc060ce15ebcb HTTP/1.1" 502 150 2025-12-17 22:29:03,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c2575bc060ce15ebcb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:04,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b8575bc060ce15ebae 2025-12-17 22:29:04,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b8575bc060ce15ebae HTTP/1.1" 502 150 2025-12-17 22:29:04,454 - lava_callback - ERROR - Error processing callback for node 694089b8575bc060ce15ebae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b8575bc060ce15ebae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b8575bc060ce15ebae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b8575bc060ce15ebae (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:04,456 - lava_callback - INFO - Starting async job submit for node 694089ca575bc060ce15ebdd 2025-12-17 22:29:04,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:04,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ca575bc060ce15ebdd HTTP/1.1" 502 150 2025-12-17 22:29:04,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ca575bc060ce15ebdd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:04,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ca575bc060ce15ebdd 2025-12-17 22:29:04,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ca575bc060ce15ebdd HTTP/1.1" 502 150 2025-12-17 22:29:04,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ca575bc060ce15ebdd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:04,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b9575bc060ce15ebb1 2025-12-17 22:29:04,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b9575bc060ce15ebb1 HTTP/1.1" 502 150 2025-12-17 22:29:04,507 - lava_callback - ERROR - Error processing callback for node 694089b9575bc060ce15ebb1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b9575bc060ce15ebb1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b9575bc060ce15ebb1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b9575bc060ce15ebb1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:04,508 - lava_callback - INFO - Starting async job submit for node 694089c4575bc060ce15ebce 2025-12-17 22:29:04,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:04,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c4575bc060ce15ebce HTTP/1.1" 502 150 2025-12-17 22:29:04,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c4575bc060ce15ebce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:04,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c4575bc060ce15ebce 2025-12-17 22:29:04,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c4575bc060ce15ebce HTTP/1.1" 502 150 2025-12-17 22:29:04,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c4575bc060ce15ebce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:04,783 - lava_callback - INFO - Received callback for node 69408ba2575bc060ce15f2c1 INFO: 172.21.0.1:59924 - "POST /node/69408ba2575bc060ce15f2c1 HTTP/1.0" 202 Accepted 2025-12-17 22:29:04,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ba575bc060ce15ebb4 2025-12-17 22:29:04,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ba575bc060ce15ebb4 HTTP/1.1" 502 150 2025-12-17 22:29:04,794 - lava_callback - ERROR - Error processing callback for node 694089ba575bc060ce15ebb4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ba575bc060ce15ebb4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ba575bc060ce15ebb4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ba575bc060ce15ebb4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:04,796 - lava_callback - INFO - Starting async job submit for node 694089c5575bc060ce15ebd1 2025-12-17 22:29:04,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:04,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c5575bc060ce15ebd1 HTTP/1.1" 502 150 2025-12-17 22:29:04,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c5575bc060ce15ebd1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:04,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c5575bc060ce15ebd1 2025-12-17 22:29:04,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c5575bc060ce15ebd1 HTTP/1.1" 502 150 2025-12-17 22:29:04,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c5575bc060ce15ebd1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:05,441 - lava_callback - INFO - Received callback for node 69408b9b575bc060ce15f2ac INFO: 172.21.0.1:59938 - "POST /node/69408b9b575bc060ce15f2ac HTTP/1.0" 202 Accepted 2025-12-17 22:29:05,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b2575bc060ce15eba2 2025-12-17 22:29:05,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b2575bc060ce15eba2 HTTP/1.1" 502 150 2025-12-17 22:29:05,768 - lava_callback - ERROR - Error processing callback for node 694089b2575bc060ce15eba2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b2575bc060ce15eba2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b2575bc060ce15eba2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b2575bc060ce15eba2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:05,770 - lava_callback - INFO - Starting async job submit for node 6940845a575bc060ce15d736 2025-12-17 22:29:06,100 - lava_callback - INFO - Received callback for node 69408ba0575bc060ce15f2be INFO: 172.21.0.1:59952 - "POST /node/69408ba0575bc060ce15f2be HTTP/1.0" 202 Accepted 2025-12-17 22:29:06,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:06,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845a575bc060ce15d736 HTTP/1.1" 502 150 2025-12-17 22:29:06,212 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845a575bc060ce15d736'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:06,212 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845a575bc060ce15d736 2025-12-17 22:29:06,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845a575bc060ce15d736 HTTP/1.1" 502 150 2025-12-17 22:29:06,213 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845a575bc060ce15d736'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:06,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ca575bc060ce15ebdd 2025-12-17 22:29:06,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ca575bc060ce15ebdd HTTP/1.1" 502 150 2025-12-17 22:29:06,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ca575bc060ce15ebdd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59954 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:29:06,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c4575bc060ce15ebce 2025-12-17 22:29:06,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c4575bc060ce15ebce HTTP/1.1" 502 150 2025-12-17 22:29:06,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c4575bc060ce15ebce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:06,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940de35460bc8e2651133a4 2025-12-17 22:29:06,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940de35460bc8e2651133a4 HTTP/1.1" 502 150 2025-12-17 22:29:06,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940de35460bc8e2651133a4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:06,748 - lava_callback - INFO - Received callback for node 69408ba3575bc060ce15f2c4 INFO: 172.21.0.1:59968 - "POST /node/69408ba3575bc060ce15f2c4 HTTP/1.0" 202 Accepted 2025-12-17 22:29:06,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c5575bc060ce15ebd1 2025-12-17 22:29:06,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c5575bc060ce15ebd1 HTTP/1.1" 502 150 2025-12-17 22:29:06,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c5575bc060ce15ebd1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:06,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c6575bc060ce15ebd4 2025-12-17 22:29:06,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c6575bc060ce15ebd4 HTTP/1.1" 502 150 2025-12-17 22:29:06,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c6575bc060ce15ebd4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:07,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c0575bc060ce15ebc8 HTTP/1.1" 502 150 2025-12-17 22:29:07,120 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c0575bc060ce15ebc8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:07,389 - lava_callback - INFO - Received callback for node 69408ba5575bc060ce15f2c7 INFO: 172.21.0.1:59970 - "POST /node/69408ba5575bc060ce15f2c7 HTTP/1.0" 202 Accepted 2025-12-17 22:29:07,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c8575bc060ce15ebd7 2025-12-17 22:29:07,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c8575bc060ce15ebd7 HTTP/1.1" 502 150 2025-12-17 22:29:07,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c8575bc060ce15ebd7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:07,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c9575bc060ce15ebda 2025-12-17 22:29:07,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c9575bc060ce15ebda HTTP/1.1" 502 150 2025-12-17 22:29:07,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c9575bc060ce15ebda'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:08,023 - lava_callback - INFO - Received callback for node 69408baa575bc060ce15f2d0 INFO: 172.21.0.1:59974 - "POST /node/69408baa575bc060ce15f2d0 HTTP/1.0" 202 Accepted 2025-12-17 22:29:08,213 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845a575bc060ce15d736 2025-12-17 22:29:08,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845a575bc060ce15d736 HTTP/1.1" 502 150 2025-12-17 22:29:08,214 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845a575bc060ce15d736'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:08,673 - lava_callback - INFO - Received callback for node 69408bac575bc060ce15f2d3 INFO: 172.21.0.1:59986 - "POST /node/69408bac575bc060ce15f2d3 HTTP/1.0" 202 Accepted 2025-12-17 22:29:08,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b3575bc060ce15eba5 2025-12-17 22:29:08,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b3575bc060ce15eba5 HTTP/1.1" 502 150 2025-12-17 22:29:08,692 - lava_callback - ERROR - Error processing callback for node 694089b3575bc060ce15eba5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b3575bc060ce15eba5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b3575bc060ce15eba5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b3575bc060ce15eba5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:08,694 - lava_callback - INFO - Starting async job submit for node 694089d0575bc060ce15ebe4 2025-12-17 22:29:08,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:08,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d0575bc060ce15ebe4 HTTP/1.1" 502 150 2025-12-17 22:29:08,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d0575bc060ce15ebe4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:08,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d0575bc060ce15ebe4 2025-12-17 22:29:08,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d0575bc060ce15ebe4 HTTP/1.1" 502 150 2025-12-17 22:29:08,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d0575bc060ce15ebe4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:09,315 - lava_callback - INFO - Received callback for node 69408bae575bc060ce15f2d6 INFO: 172.21.0.1:59992 - "POST /node/69408bae575bc060ce15f2d6 HTTP/1.0" 202 Accepted 2025-12-17 22:29:10,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ca575bc060ce15ebdd 2025-12-17 22:29:10,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ca575bc060ce15ebdd HTTP/1.1" 502 150 2025-12-17 22:29:10,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ca575bc060ce15ebdd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:10,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c4575bc060ce15ebce 2025-12-17 22:29:10,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c4575bc060ce15ebce HTTP/1.1" 502 150 2025-12-17 22:29:10,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c4575bc060ce15ebce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:10,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d0575bc060ce15ebe4 2025-12-17 22:29:10,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d0575bc060ce15ebe4 HTTP/1.1" 502 150 2025-12-17 22:29:10,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d0575bc060ce15ebe4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:10,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089bd575bc060ce15ebc2 2025-12-17 22:29:10,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089bd575bc060ce15ebc2 HTTP/1.1" 502 150 2025-12-17 22:29:10,754 - lava_callback - ERROR - Error processing callback for node 694089bd575bc060ce15ebc2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089bd575bc060ce15ebc2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089bd575bc060ce15ebc2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089bd575bc060ce15ebc2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:10,756 - lava_callback - INFO - Starting async job submit for node 694089d1575bc060ce15ebed 2025-12-17 22:29:10,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:10,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d1575bc060ce15ebed HTTP/1.1" 502 150 2025-12-17 22:29:10,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d1575bc060ce15ebed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:10,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d1575bc060ce15ebed 2025-12-17 22:29:10,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d1575bc060ce15ebed HTTP/1.1" 502 150 2025-12-17 22:29:10,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d1575bc060ce15ebed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:10,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c5575bc060ce15ebd1 2025-12-17 22:29:10,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c5575bc060ce15ebd1 HTTP/1.1" 502 150 2025-12-17 22:29:10,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c5575bc060ce15ebd1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:11,121 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c0575bc060ce15ebc8 2025-12-17 22:29:11,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c0575bc060ce15ebc8 HTTP/1.1" 502 150 2025-12-17 22:29:11,122 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c0575bc060ce15ebc8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:11,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c2575bc060ce15ebcb 2025-12-17 22:29:11,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c2575bc060ce15ebcb HTTP/1.1" 502 150 2025-12-17 22:29:11,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c2575bc060ce15ebcb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:12,215 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845a575bc060ce15d736 2025-12-17 22:29:12,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845a575bc060ce15d736 HTTP/1.1" 502 150 2025-12-17 22:29:12,216 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845a575bc060ce15d736'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:12,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089bb575bc060ce15ebbb 2025-12-17 22:29:12,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089bb575bc060ce15ebbb HTTP/1.1" 502 150 2025-12-17 22:29:12,695 - lava_callback - ERROR - Error processing callback for node 694089bb575bc060ce15ebbb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089bb575bc060ce15ebbb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089bb575bc060ce15ebbb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089bb575bc060ce15ebbb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:12,697 - lava_callback - INFO - Starting async job submit for node 694089d3575bc060ce15ebf0 2025-12-17 22:29:12,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:12,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d3575bc060ce15ebf0 HTTP/1.1" 502 150 2025-12-17 22:29:12,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d3575bc060ce15ebf0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:12,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d3575bc060ce15ebf0 2025-12-17 22:29:12,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d3575bc060ce15ebf0 HTTP/1.1" 502 150 2025-12-17 22:29:12,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d3575bc060ce15ebf0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:12,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d1575bc060ce15ebed 2025-12-17 22:29:12,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d1575bc060ce15ebed HTTP/1.1" 502 150 2025-12-17 22:29:12,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d1575bc060ce15ebed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:14,583 - lava_callback - INFO - Received callback for node 69408ba7575bc060ce15f2ca INFO: 172.21.0.1:55256 - "POST /node/69408ba7575bc060ce15f2ca HTTP/1.0" 202 Accepted 2025-12-17 22:29:14,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d0575bc060ce15ebe4 2025-12-17 22:29:14,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d0575bc060ce15ebe4 HTTP/1.1" 502 150 2025-12-17 22:29:14,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d0575bc060ce15ebe4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:14,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d3575bc060ce15ebf0 2025-12-17 22:29:14,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d3575bc060ce15ebf0 HTTP/1.1" 502 150 2025-12-17 22:29:14,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d3575bc060ce15ebf0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:14,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c6575bc060ce15ebd4 2025-12-17 22:29:14,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c6575bc060ce15ebd4 HTTP/1.1" 502 150 2025-12-17 22:29:14,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c6575bc060ce15ebd4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55268 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:29:15,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c8575bc060ce15ebd7 2025-12-17 22:29:15,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c8575bc060ce15ebd7 HTTP/1.1" 502 150 2025-12-17 22:29:15,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c8575bc060ce15ebd7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:15,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c9575bc060ce15ebda 2025-12-17 22:29:15,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c9575bc060ce15ebda HTTP/1.1" 502 150 2025-12-17 22:29:15,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c9575bc060ce15ebda'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55270 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:29:16,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d1575bc060ce15ebed 2025-12-17 22:29:16,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d1575bc060ce15ebed HTTP/1.1" 502 150 2025-12-17 22:29:16,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d1575bc060ce15ebed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:16,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408912575bc060ce15e91c HTTP/1.1" 502 150 2025-12-17 22:29:16,850 - lava_callback - ERROR - Error processing callback for node 69408912575bc060ce15e91c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408912575bc060ce15e91c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408912575bc060ce15e91c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408912575bc060ce15e91c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:16,853 - lava_callback - INFO - Starting async job submit for node 694089d6575bc060ce15ebf6 2025-12-17 22:29:16,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:16,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d6575bc060ce15ebf6 HTTP/1.1" 502 150 2025-12-17 22:29:16,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d6575bc060ce15ebf6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:16,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d6575bc060ce15ebf6 2025-12-17 22:29:16,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d6575bc060ce15ebf6 HTTP/1.1" 502 150 2025-12-17 22:29:16,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d6575bc060ce15ebf6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:17,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089be575bc060ce15ebc5 2025-12-17 22:29:17,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089be575bc060ce15ebc5 HTTP/1.1" 502 150 2025-12-17 22:29:17,927 - lava_callback - ERROR - Error processing callback for node 694089be575bc060ce15ebc5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089be575bc060ce15ebc5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089be575bc060ce15ebc5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089be575bc060ce15ebc5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:17,930 - lava_callback - INFO - Starting async job submit for node 694089d9575bc060ce15ec31 2025-12-17 22:29:17,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:17,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d9575bc060ce15ec31 HTTP/1.1" 502 150 2025-12-17 22:29:17,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d9575bc060ce15ec31'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:17,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d9575bc060ce15ec31 2025-12-17 22:29:17,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d9575bc060ce15ec31 HTTP/1.1" 502 150 2025-12-17 22:29:17,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d9575bc060ce15ec31'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:18,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ca575bc060ce15ebdd 2025-12-17 22:29:18,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ca575bc060ce15ebdd HTTP/1.1" 502 150 2025-12-17 22:29:18,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ca575bc060ce15ebdd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:18,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c4575bc060ce15ebce 2025-12-17 22:29:18,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c4575bc060ce15ebce HTTP/1.1" 502 150 2025-12-17 22:29:18,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c4575bc060ce15ebce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:18,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d3575bc060ce15ebf0 2025-12-17 22:29:18,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d3575bc060ce15ebf0 HTTP/1.1" 502 150 2025-12-17 22:29:18,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d3575bc060ce15ebf0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:18,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c5575bc060ce15ebd1 2025-12-17 22:29:18,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c5575bc060ce15ebd1 HTTP/1.1" 502 150 2025-12-17 22:29:18,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c5575bc060ce15ebd1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:18,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089b6575bc060ce15ebab 2025-12-17 22:29:18,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089b6575bc060ce15ebab HTTP/1.1" 502 150 2025-12-17 22:29:18,835 - lava_callback - ERROR - Error processing callback for node 694089b6575bc060ce15ebab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b6575bc060ce15ebab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b6575bc060ce15ebab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089b6575bc060ce15ebab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:18,837 - lava_callback - INFO - Starting async job submit for node 694089db575bc060ce15ec40 2025-12-17 22:29:18,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:18,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d6575bc060ce15ebf6 2025-12-17 22:29:18,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d6575bc060ce15ebf6 HTTP/1.1" 502 150 2025-12-17 22:29:18,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d6575bc060ce15ebf6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:18,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089db575bc060ce15ec40 HTTP/1.1" 502 150 2025-12-17 22:29:18,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089db575bc060ce15ec40'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:18,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089db575bc060ce15ec40 2025-12-17 22:29:18,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089db575bc060ce15ec40 HTTP/1.1" 502 150 2025-12-17 22:29:18,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089db575bc060ce15ec40'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:19,122 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c0575bc060ce15ebc8 2025-12-17 22:29:19,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c0575bc060ce15ebc8 HTTP/1.1" 502 150 2025-12-17 22:29:19,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089c0575bc060ce15ebc8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:19,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d9575bc060ce15ec31 2025-12-17 22:29:19,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d9575bc060ce15ec31 HTTP/1.1" 502 150 2025-12-17 22:29:19,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d9575bc060ce15ec31'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:20,216 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845a575bc060ce15d736 2025-12-17 22:29:20,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845a575bc060ce15d736 HTTP/1.1" 502 150 2025-12-17 22:29:20,218 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940845a575bc060ce15d736'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:20,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089db575bc060ce15ec40 2025-12-17 22:29:20,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089db575bc060ce15ec40 HTTP/1.1" 502 150 2025-12-17 22:29:20,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089db575bc060ce15ec40'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:22,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940de35460bc8e2651133a4 2025-12-17 22:29:22,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940de35460bc8e2651133a4 HTTP/1.1" 502 150 2025-12-17 22:29:22,697 - lava_callback - ERROR - Error processing callback for node 6940de35460bc8e2651133a4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940de35460bc8e2651133a4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940de35460bc8e2651133a4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940de35460bc8e2651133a4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:22,700 - lava_callback - INFO - Starting async job submit for node 694089dc575bc060ce15ec43 2025-12-17 22:29:22,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:22,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d0575bc060ce15ebe4 2025-12-17 22:29:22,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d0575bc060ce15ebe4 HTTP/1.1" 502 150 2025-12-17 22:29:22,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d0575bc060ce15ebe4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:22,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089dc575bc060ce15ec43 HTTP/1.1" 502 150 2025-12-17 22:29:22,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089dc575bc060ce15ec43'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:22,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089dc575bc060ce15ec43 2025-12-17 22:29:22,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089dc575bc060ce15ec43 HTTP/1.1" 502 150 2025-12-17 22:29:22,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089dc575bc060ce15ec43'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:22,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d6575bc060ce15ebf6 2025-12-17 22:29:22,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d6575bc060ce15ebf6 HTTP/1.1" 502 150 2025-12-17 22:29:22,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d6575bc060ce15ebf6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:23,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d9575bc060ce15ec31 2025-12-17 22:29:23,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d9575bc060ce15ec31 HTTP/1.1" 502 150 2025-12-17 22:29:23,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d9575bc060ce15ec31'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:24,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089dc575bc060ce15ec43 2025-12-17 22:29:24,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089dc575bc060ce15ec43 HTTP/1.1" 502 150 2025-12-17 22:29:24,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089dc575bc060ce15ec43'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:24,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d1575bc060ce15ebed 2025-12-17 22:29:24,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d1575bc060ce15ebed HTTP/1.1" 502 150 2025-12-17 22:29:24,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d1575bc060ce15ebed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:24,778 - lava_callback - INFO - Received callback for node 69408bb0575bc060ce15f2dd INFO: 172.21.0.1:59552 - "POST /node/69408bb0575bc060ce15f2dd HTTP/1.0" 202 Accepted 2025-12-17 22:29:24,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089db575bc060ce15ec40 2025-12-17 22:29:24,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089db575bc060ce15ec40 HTTP/1.1" 502 150 2025-12-17 22:29:24,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089db575bc060ce15ec40'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:25,422 - lava_callback - INFO - Received callback for node 69408bb6575bc060ce15f2ea INFO: 172.21.0.1:59554 - "POST /node/69408bb6575bc060ce15f2ea HTTP/1.0" 202 Accepted 2025-12-17 22:29:26,085 - lava_callback - INFO - Received callback for node 69408bb8575bc060ce15f2ed INFO: 172.21.0.1:59568 - "POST /node/69408bb8575bc060ce15f2ed HTTP/1.0" 202 Accepted INFO: 172.21.0.1:59584 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:29:26,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d3575bc060ce15ebf0 2025-12-17 22:29:26,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d3575bc060ce15ebf0 HTTP/1.1" 502 150 2025-12-17 22:29:26,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d3575bc060ce15ebf0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:26,721 - lava_callback - INFO - Received callback for node 69408bba575bc060ce15f2f0 INFO: 172.21.0.1:59598 - "POST /node/69408bba575bc060ce15f2f0 HTTP/1.0" 202 Accepted 2025-12-17 22:29:27,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c2575bc060ce15ebcb 2025-12-17 22:29:27,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c2575bc060ce15ebcb HTTP/1.1" 502 150 2025-12-17 22:29:27,962 - lava_callback - ERROR - Error processing callback for node 694089c2575bc060ce15ebcb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c2575bc060ce15ebcb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c2575bc060ce15ebcb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c2575bc060ce15ebcb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:27,965 - lava_callback - INFO - Starting async job submit for node 694089dd575bc060ce15ec46 2025-12-17 22:29:27,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:27,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089dd575bc060ce15ec46 HTTP/1.1" 502 150 2025-12-17 22:29:27,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089dd575bc060ce15ec46'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:27,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089dd575bc060ce15ec46 2025-12-17 22:29:27,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089dd575bc060ce15ec46 HTTP/1.1" 502 150 2025-12-17 22:29:27,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089dd575bc060ce15ec46'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:28,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089dc575bc060ce15ec43 2025-12-17 22:29:28,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089dc575bc060ce15ec43 HTTP/1.1" 502 150 2025-12-17 22:29:28,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089dc575bc060ce15ec43'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:29,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089dd575bc060ce15ec46 2025-12-17 22:29:29,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089dd575bc060ce15ec46 HTTP/1.1" 502 150 2025-12-17 22:29:29,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089dd575bc060ce15ec46'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:30,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d6575bc060ce15ebf6 2025-12-17 22:29:30,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d6575bc060ce15ebf6 HTTP/1.1" 502 150 2025-12-17 22:29:30,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d6575bc060ce15ebf6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:30,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c6575bc060ce15ebd4 2025-12-17 22:29:30,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c6575bc060ce15ebd4 HTTP/1.1" 502 150 2025-12-17 22:29:30,885 - lava_callback - ERROR - Error processing callback for node 694089c6575bc060ce15ebd4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c6575bc060ce15ebd4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c6575bc060ce15ebd4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c6575bc060ce15ebd4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:30,887 - lava_callback - INFO - Starting async job submit for node 694089df575bc060ce15ec49 2025-12-17 22:29:30,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:30,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089df575bc060ce15ec49 HTTP/1.1" 502 150 2025-12-17 22:29:30,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089df575bc060ce15ec49'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:30,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089df575bc060ce15ec49 2025-12-17 22:29:30,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089df575bc060ce15ec49 HTTP/1.1" 502 150 2025-12-17 22:29:30,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089df575bc060ce15ec49'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:31,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c8575bc060ce15ebd7 2025-12-17 22:29:31,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c8575bc060ce15ebd7 HTTP/1.1" 502 150 2025-12-17 22:29:31,600 - lava_callback - ERROR - Error processing callback for node 694089c8575bc060ce15ebd7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c8575bc060ce15ebd7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c8575bc060ce15ebd7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c8575bc060ce15ebd7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:31,602 - lava_callback - INFO - Starting async job submit for node 694089e3575bc060ce15ec4e 2025-12-17 22:29:31,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:31,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c9575bc060ce15ebda 2025-12-17 22:29:31,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c9575bc060ce15ebda HTTP/1.1" 502 150 2025-12-17 22:29:31,618 - lava_callback - ERROR - Error processing callback for node 694089c9575bc060ce15ebda: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c9575bc060ce15ebda (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c9575bc060ce15ebda (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c9575bc060ce15ebda (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:31,620 - lava_callback - INFO - Starting async job submit for node 694089e4575bc060ce15ec55 2025-12-17 22:29:31,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:31,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e4575bc060ce15ec55 HTTP/1.1" 502 150 2025-12-17 22:29:31,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e4575bc060ce15ec55'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:31,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e4575bc060ce15ec55 2025-12-17 22:29:31,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e4575bc060ce15ec55 HTTP/1.1" 502 150 2025-12-17 22:29:31,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e4575bc060ce15ec55'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:31,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e3575bc060ce15ec4e HTTP/1.1" 502 150 2025-12-17 22:29:31,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e3575bc060ce15ec4e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:31,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e3575bc060ce15ec4e 2025-12-17 22:29:31,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e3575bc060ce15ec4e HTTP/1.1" 502 150 2025-12-17 22:29:31,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e3575bc060ce15ec4e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:31,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d9575bc060ce15ec31 2025-12-17 22:29:31,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d9575bc060ce15ec31 HTTP/1.1" 502 150 2025-12-17 22:29:31,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089d9575bc060ce15ec31'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:32,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089db575bc060ce15ec40 2025-12-17 22:29:32,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089db575bc060ce15ec40 HTTP/1.1" 502 150 2025-12-17 22:29:32,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089db575bc060ce15ec40'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:32,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089df575bc060ce15ec49 2025-12-17 22:29:32,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089df575bc060ce15ec49 HTTP/1.1" 502 150 2025-12-17 22:29:32,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089df575bc060ce15ec49'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:33,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e4575bc060ce15ec55 2025-12-17 22:29:33,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e4575bc060ce15ec55 HTTP/1.1" 502 150 2025-12-17 22:29:33,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e4575bc060ce15ec55'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:33,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e3575bc060ce15ec4e 2025-12-17 22:29:33,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e3575bc060ce15ec4e HTTP/1.1" 502 150 2025-12-17 22:29:33,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e3575bc060ce15ec4e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:33,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089dd575bc060ce15ec46 2025-12-17 22:29:33,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089dd575bc060ce15ec46 HTTP/1.1" 502 150 2025-12-17 22:29:33,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089dd575bc060ce15ec46'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:34,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ca575bc060ce15ebdd 2025-12-17 22:29:34,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ca575bc060ce15ebdd HTTP/1.1" 502 150 2025-12-17 22:29:34,472 - lava_callback - ERROR - Error processing callback for node 694089ca575bc060ce15ebdd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ca575bc060ce15ebdd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ca575bc060ce15ebdd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ca575bc060ce15ebdd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:34,474 - lava_callback - INFO - Starting async job submit for node 694089e5575bc060ce15ec58 2025-12-17 22:29:34,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:34,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e5575bc060ce15ec58 HTTP/1.1" 502 150 2025-12-17 22:29:34,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e5575bc060ce15ec58'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:34,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e5575bc060ce15ec58 2025-12-17 22:29:34,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e5575bc060ce15ec58 HTTP/1.1" 502 150 2025-12-17 22:29:34,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e5575bc060ce15ec58'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:34,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c4575bc060ce15ebce 2025-12-17 22:29:34,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c4575bc060ce15ebce HTTP/1.1" 502 150 2025-12-17 22:29:34,530 - lava_callback - ERROR - Error processing callback for node 694089c4575bc060ce15ebce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c4575bc060ce15ebce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c4575bc060ce15ebce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c4575bc060ce15ebce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:34,531 - lava_callback - INFO - Starting async job submit for node 69408587575bc060ce15dc6f 2025-12-17 22:29:34,595 - lava_callback - INFO - Received callback for node 69408bbd575bc060ce15f2f3 INFO: 172.21.0.1:45014 - "POST /node/69408bbd575bc060ce15f2f3 HTTP/1.0" 202 Accepted 2025-12-17 22:29:34,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:34,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408587575bc060ce15dc6f HTTP/1.1" 502 150 2025-12-17 22:29:34,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408587575bc060ce15dc6f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:34,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408587575bc060ce15dc6f 2025-12-17 22:29:34,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408587575bc060ce15dc6f HTTP/1.1" 502 150 2025-12-17 22:29:34,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408587575bc060ce15dc6f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:34,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c5575bc060ce15ebd1 2025-12-17 22:29:34,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c5575bc060ce15ebd1 HTTP/1.1" 502 150 2025-12-17 22:29:34,811 - lava_callback - ERROR - Error processing callback for node 694089c5575bc060ce15ebd1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c5575bc060ce15ebd1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c5575bc060ce15ebd1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c5575bc060ce15ebd1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:34,813 - lava_callback - INFO - Starting async job submit for node 694089e6575bc060ce15ec5b 2025-12-17 22:29:34,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:34,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e6575bc060ce15ec5b HTTP/1.1" 502 150 2025-12-17 22:29:34,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e6575bc060ce15ec5b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:34,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e6575bc060ce15ec5b 2025-12-17 22:29:34,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e6575bc060ce15ec5b HTTP/1.1" 502 150 2025-12-17 22:29:34,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e6575bc060ce15ec5b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:35,124 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089c0575bc060ce15ebc8 2025-12-17 22:29:35,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089c0575bc060ce15ebc8 HTTP/1.1" 502 150 2025-12-17 22:29:35,125 - lava_callback - ERROR - Error processing callback for node 694089c0575bc060ce15ebc8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c0575bc060ce15ebc8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c0575bc060ce15ebc8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089c0575bc060ce15ebc8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:35,127 - lava_callback - INFO - Starting async job submit for node 694089f7575bc060ce15ec83 2025-12-17 22:29:35,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:35,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f7575bc060ce15ec83 HTTP/1.1" 502 150 2025-12-17 22:29:35,133 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f7575bc060ce15ec83'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:35,133 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f7575bc060ce15ec83 2025-12-17 22:29:35,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f7575bc060ce15ec83 HTTP/1.1" 502 150 2025-12-17 22:29:35,134 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f7575bc060ce15ec83'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:35,250 - lava_callback - INFO - Received callback for node 69408bbf575bc060ce15f2f6 INFO: 172.21.0.1:45022 - "POST /node/69408bbf575bc060ce15f2f6 HTTP/1.0" 202 Accepted 2025-12-17 22:29:35,935 - lava_callback - INFO - Received callback for node 69408bc1575bc060ce15f2f9 INFO: 172.21.0.1:45026 - "POST /node/69408bc1575bc060ce15f2f9 HTTP/1.0" 202 Accepted 2025-12-17 22:29:36,218 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940845a575bc060ce15d736 2025-12-17 22:29:36,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940845a575bc060ce15d736 HTTP/1.1" 502 150 2025-12-17 22:29:36,220 - lava_callback - ERROR - Error processing callback for node 6940845a575bc060ce15d736: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940845a575bc060ce15d736 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940845a575bc060ce15d736 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940845a575bc060ce15d736 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:36,221 - lava_callback - INFO - Starting async job submit for node 694089e8575bc060ce15ec5e 2025-12-17 22:29:36,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:36,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e8575bc060ce15ec5e HTTP/1.1" 502 150 2025-12-17 22:29:36,228 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e8575bc060ce15ec5e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:36,228 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e8575bc060ce15ec5e 2025-12-17 22:29:36,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e8575bc060ce15ec5e HTTP/1.1" 502 150 2025-12-17 22:29:36,229 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e8575bc060ce15ec5e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:36,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e5575bc060ce15ec58 2025-12-17 22:29:36,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e5575bc060ce15ec58 HTTP/1.1" 502 150 2025-12-17 22:29:36,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e5575bc060ce15ec58'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45040 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:29:36,578 - lava_callback - INFO - Received callback for node 69408bc5575bc060ce15f307 INFO: 172.21.0.1:45056 - "POST /node/69408bc5575bc060ce15f307 HTTP/1.0" 202 Accepted 2025-12-17 22:29:36,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408587575bc060ce15dc6f 2025-12-17 22:29:36,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408587575bc060ce15dc6f HTTP/1.1" 502 150 2025-12-17 22:29:36,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408587575bc060ce15dc6f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:36,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089dc575bc060ce15ec43 2025-12-17 22:29:36,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089dc575bc060ce15ec43 HTTP/1.1" 502 150 2025-12-17 22:29:36,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089dc575bc060ce15ec43'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:36,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e6575bc060ce15ec5b 2025-12-17 22:29:36,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e6575bc060ce15ec5b HTTP/1.1" 502 150 2025-12-17 22:29:36,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e6575bc060ce15ec5b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:36,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089df575bc060ce15ec49 2025-12-17 22:29:36,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089df575bc060ce15ec49 HTTP/1.1" 502 150 2025-12-17 22:29:36,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089df575bc060ce15ec49'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:37,134 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f7575bc060ce15ec83 2025-12-17 22:29:37,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f7575bc060ce15ec83 HTTP/1.1" 502 150 2025-12-17 22:29:37,136 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f7575bc060ce15ec83'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:37,232 - lava_callback - INFO - Received callback for node 69408bc9575bc060ce15f33f INFO: 172.21.0.1:45068 - "POST /node/69408bc9575bc060ce15f33f HTTP/1.0" 202 Accepted 2025-12-17 22:29:37,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e4575bc060ce15ec55 2025-12-17 22:29:37,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e4575bc060ce15ec55 HTTP/1.1" 502 150 2025-12-17 22:29:37,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e4575bc060ce15ec55'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:37,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e3575bc060ce15ec4e 2025-12-17 22:29:37,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e3575bc060ce15ec4e HTTP/1.1" 502 150 2025-12-17 22:29:37,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e3575bc060ce15ec4e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:37,909 - lava_callback - INFO - Received callback for node 69408bcb575bc060ce15f34d INFO: 172.21.0.1:45076 - "POST /node/69408bcb575bc060ce15f34d HTTP/1.0" 202 Accepted 2025-12-17 22:29:38,229 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e8575bc060ce15ec5e 2025-12-17 22:29:38,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e8575bc060ce15ec5e HTTP/1.1" 502 150 2025-12-17 22:29:38,231 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e8575bc060ce15ec5e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:38,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d0575bc060ce15ebe4 2025-12-17 22:29:38,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d0575bc060ce15ebe4 HTTP/1.1" 502 150 2025-12-17 22:29:38,706 - lava_callback - ERROR - Error processing callback for node 694089d0575bc060ce15ebe4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089d0575bc060ce15ebe4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089d0575bc060ce15ebe4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089d0575bc060ce15ebe4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:38,708 - lava_callback - INFO - Starting async job submit for node 694089f8575bc060ce15ec88 2025-12-17 22:29:38,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:38,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f8575bc060ce15ec88 HTTP/1.1" 502 150 2025-12-17 22:29:38,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f8575bc060ce15ec88'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:38,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f8575bc060ce15ec88 2025-12-17 22:29:38,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f8575bc060ce15ec88 HTTP/1.1" 502 150 2025-12-17 22:29:38,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f8575bc060ce15ec88'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:40,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e5575bc060ce15ec58 2025-12-17 22:29:40,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e5575bc060ce15ec58 HTTP/1.1" 502 150 2025-12-17 22:29:40,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e5575bc060ce15ec58'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:40,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408587575bc060ce15dc6f 2025-12-17 22:29:40,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408587575bc060ce15dc6f HTTP/1.1" 502 150 2025-12-17 22:29:40,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408587575bc060ce15dc6f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:40,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f8575bc060ce15ec88 2025-12-17 22:29:40,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f8575bc060ce15ec88 HTTP/1.1" 502 150 2025-12-17 22:29:40,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f8575bc060ce15ec88'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:40,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d1575bc060ce15ebed 2025-12-17 22:29:40,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d1575bc060ce15ebed HTTP/1.1" 502 150 2025-12-17 22:29:40,779 - lava_callback - ERROR - Error processing callback for node 694089d1575bc060ce15ebed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089d1575bc060ce15ebed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089d1575bc060ce15ebed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089d1575bc060ce15ebed (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:40,781 - lava_callback - INFO - Starting async job submit for node 694089e9575bc060ce15ec61 2025-12-17 22:29:40,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:40,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e9575bc060ce15ec61 HTTP/1.1" 502 150 2025-12-17 22:29:40,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e9575bc060ce15ec61'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:40,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e9575bc060ce15ec61 2025-12-17 22:29:40,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e9575bc060ce15ec61 HTTP/1.1" 502 150 2025-12-17 22:29:40,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e9575bc060ce15ec61'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:40,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e6575bc060ce15ec5b 2025-12-17 22:29:40,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e6575bc060ce15ec5b HTTP/1.1" 502 150 2025-12-17 22:29:40,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e6575bc060ce15ec5b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:41,136 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f7575bc060ce15ec83 2025-12-17 22:29:41,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f7575bc060ce15ec83 HTTP/1.1" 502 150 2025-12-17 22:29:41,137 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f7575bc060ce15ec83'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:41,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089dd575bc060ce15ec46 2025-12-17 22:29:41,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089dd575bc060ce15ec46 HTTP/1.1" 502 150 2025-12-17 22:29:41,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089dd575bc060ce15ec46'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:42,231 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e8575bc060ce15ec5e 2025-12-17 22:29:42,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e8575bc060ce15ec5e HTTP/1.1" 502 150 2025-12-17 22:29:42,232 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e8575bc060ce15ec5e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:42,602 - lava_callback - INFO - Received callback for node 6940ee14460bc8e265114da8 INFO: 172.21.0.1:49796 - "POST /node/6940ee14460bc8e265114da8 HTTP/1.0" 202 Accepted 2025-12-17 22:29:42,658 - lava_callback - INFO - Received callback for node 69408c26575bc060ce15f47a INFO: 172.21.0.1:49802 - "POST /node/69408c26575bc060ce15f47a HTTP/1.0" 202 Accepted 2025-12-17 22:29:42,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d3575bc060ce15ebf0 2025-12-17 22:29:42,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d3575bc060ce15ebf0 HTTP/1.1" 502 150 2025-12-17 22:29:42,711 - lava_callback - ERROR - Error processing callback for node 694089d3575bc060ce15ebf0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089d3575bc060ce15ebf0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089d3575bc060ce15ebf0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089d3575bc060ce15ebf0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:42,713 - lava_callback - INFO - Starting async job submit for node 694089f9575bc060ce15ec8f 2025-12-17 22:29:42,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:42,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f9575bc060ce15ec8f HTTP/1.1" 502 150 2025-12-17 22:29:42,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f9575bc060ce15ec8f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:42,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f9575bc060ce15ec8f 2025-12-17 22:29:42,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f9575bc060ce15ec8f HTTP/1.1" 502 150 2025-12-17 22:29:42,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f9575bc060ce15ec8f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:42,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e9575bc060ce15ec61 2025-12-17 22:29:42,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e9575bc060ce15ec61 HTTP/1.1" 502 150 2025-12-17 22:29:42,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e9575bc060ce15ec61'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:44,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f8575bc060ce15ec88 2025-12-17 22:29:44,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f8575bc060ce15ec88 HTTP/1.1" 502 150 2025-12-17 22:29:44,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f8575bc060ce15ec88'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:44,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f9575bc060ce15ec8f 2025-12-17 22:29:44,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f9575bc060ce15ec8f HTTP/1.1" 502 150 2025-12-17 22:29:44,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f9575bc060ce15ec8f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:44,813 - lava_callback - INFO - Received callback for node 69408bc3575bc060ce15f2fe INFO: 172.21.0.1:49808 - "POST /node/69408bc3575bc060ce15f2fe HTTP/1.0" 202 Accepted 2025-12-17 22:29:44,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089df575bc060ce15ec49 2025-12-17 22:29:44,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089df575bc060ce15ec49 HTTP/1.1" 502 150 2025-12-17 22:29:44,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089df575bc060ce15ec49'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:45,460 - lava_callback - INFO - Received callback for node 69408bcd575bc060ce15f354 INFO: 172.21.0.1:49822 - "POST /node/69408bcd575bc060ce15f354 HTTP/1.0" 202 Accepted 2025-12-17 22:29:45,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e4575bc060ce15ec55 2025-12-17 22:29:45,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e4575bc060ce15ec55 HTTP/1.1" 502 150 2025-12-17 22:29:45,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e4575bc060ce15ec55'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:45,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e3575bc060ce15ec4e 2025-12-17 22:29:45,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e3575bc060ce15ec4e HTTP/1.1" 502 150 2025-12-17 22:29:45,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e3575bc060ce15ec4e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:46,103 - lava_callback - INFO - Received callback for node 69408bcf575bc060ce15f357 INFO: 172.21.0.1:49828 - "POST /node/69408bcf575bc060ce15f357 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:49834 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:29:46,744 - lava_callback - INFO - Received callback for node 69408bd1575bc060ce15f35a INFO: 172.21.0.1:49838 - "POST /node/69408bd1575bc060ce15f35a HTTP/1.0" 202 Accepted 2025-12-17 22:29:46,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e9575bc060ce15ec61 2025-12-17 22:29:46,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e9575bc060ce15ec61 HTTP/1.1" 502 150 2025-12-17 22:29:46,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e9575bc060ce15ec61'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:46,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d6575bc060ce15ebf6 2025-12-17 22:29:46,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d6575bc060ce15ebf6 HTTP/1.1" 502 150 2025-12-17 22:29:46,868 - lava_callback - ERROR - Error processing callback for node 694089d6575bc060ce15ebf6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089d6575bc060ce15ebf6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089d6575bc060ce15ebf6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089d6575bc060ce15ebf6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:46,871 - lava_callback - INFO - Starting async job submit for node 694089ea575bc060ce15ec64 2025-12-17 22:29:46,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:46,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ea575bc060ce15ec64 HTTP/1.1" 502 150 2025-12-17 22:29:46,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ea575bc060ce15ec64'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:46,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ea575bc060ce15ec64 2025-12-17 22:29:46,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ea575bc060ce15ec64 HTTP/1.1" 502 150 2025-12-17 22:29:46,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ea575bc060ce15ec64'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:47,409 - lava_callback - INFO - Received callback for node 69408bd3575bc060ce15f35d INFO: 172.21.0.1:49852 - "POST /node/69408bd3575bc060ce15f35d HTTP/1.0" 202 Accepted 2025-12-17 22:29:47,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089d9575bc060ce15ec31 2025-12-17 22:29:47,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089d9575bc060ce15ec31 HTTP/1.1" 502 150 2025-12-17 22:29:47,949 - lava_callback - ERROR - Error processing callback for node 694089d9575bc060ce15ec31: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089d9575bc060ce15ec31 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089d9575bc060ce15ec31 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089d9575bc060ce15ec31 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:47,951 - lava_callback - INFO - Starting async job submit for node 694089eb575bc060ce15ec67 2025-12-17 22:29:47,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:47,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089eb575bc060ce15ec67 HTTP/1.1" 502 150 2025-12-17 22:29:47,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089eb575bc060ce15ec67'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:47,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089eb575bc060ce15ec67 2025-12-17 22:29:47,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089eb575bc060ce15ec67 HTTP/1.1" 502 150 2025-12-17 22:29:47,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089eb575bc060ce15ec67'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:48,080 - lava_callback - INFO - Received callback for node 69408bd4575bc060ce15f360 INFO: 172.21.0.1:49868 - "POST /node/69408bd4575bc060ce15f360 HTTP/1.0" 202 Accepted 2025-12-17 22:29:48,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e5575bc060ce15ec58 2025-12-17 22:29:48,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e5575bc060ce15ec58 HTTP/1.1" 502 150 2025-12-17 22:29:48,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e5575bc060ce15ec58'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:48,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408587575bc060ce15dc6f 2025-12-17 22:29:48,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408587575bc060ce15dc6f HTTP/1.1" 502 150 2025-12-17 22:29:48,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408587575bc060ce15dc6f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:48,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f9575bc060ce15ec8f 2025-12-17 22:29:48,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f9575bc060ce15ec8f HTTP/1.1" 502 150 2025-12-17 22:29:48,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f9575bc060ce15ec8f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:48,729 - lava_callback - INFO - Received callback for node 69408bd5575bc060ce15f363 INFO: 172.21.0.1:49884 - "POST /node/69408bd5575bc060ce15f363 HTTP/1.0" 202 Accepted 2025-12-17 22:29:48,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e6575bc060ce15ec5b 2025-12-17 22:29:48,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e6575bc060ce15ec5b HTTP/1.1" 502 150 2025-12-17 22:29:48,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e6575bc060ce15ec5b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:48,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089db575bc060ce15ec40 2025-12-17 22:29:48,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089db575bc060ce15ec40 HTTP/1.1" 502 150 2025-12-17 22:29:48,873 - lava_callback - ERROR - Error processing callback for node 694089db575bc060ce15ec40: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089db575bc060ce15ec40 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089db575bc060ce15ec40 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089db575bc060ce15ec40 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:48,875 - lava_callback - INFO - Starting async job submit for node 694089ed575bc060ce15ec6a 2025-12-17 22:29:48,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:48,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ed575bc060ce15ec6a HTTP/1.1" 502 150 2025-12-17 22:29:48,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ed575bc060ce15ec6a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:48,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ed575bc060ce15ec6a 2025-12-17 22:29:48,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ed575bc060ce15ec6a HTTP/1.1" 502 150 2025-12-17 22:29:48,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ed575bc060ce15ec6a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:48,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ea575bc060ce15ec64 2025-12-17 22:29:48,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ea575bc060ce15ec64 HTTP/1.1" 502 150 2025-12-17 22:29:48,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ea575bc060ce15ec64'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:49,137 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f7575bc060ce15ec83 2025-12-17 22:29:49,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f7575bc060ce15ec83 HTTP/1.1" 502 150 2025-12-17 22:29:49,138 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f7575bc060ce15ec83'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:49,316 - lava_callback - INFO - Received callback for node 69409289575bc060ce160ae7 INFO: 172.21.0.1:49886 - "POST /node/69409289575bc060ce160ae7 HTTP/1.0" 202 Accepted 2025-12-17 22:29:49,376 - lava_callback - INFO - Received callback for node 69408bd6575bc060ce15f366 INFO: 172.21.0.1:49894 - "POST /node/69408bd6575bc060ce15f366 HTTP/1.0" 202 Accepted 2025-12-17 22:29:49,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089eb575bc060ce15ec67 2025-12-17 22:29:49,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089eb575bc060ce15ec67 HTTP/1.1" 502 150 2025-12-17 22:29:49,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089eb575bc060ce15ec67'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:50,232 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e8575bc060ce15ec5e 2025-12-17 22:29:50,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e8575bc060ce15ec5e HTTP/1.1" 502 150 2025-12-17 22:29:50,233 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e8575bc060ce15ec5e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:50,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ed575bc060ce15ec6a 2025-12-17 22:29:50,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ed575bc060ce15ec6a HTTP/1.1" 502 150 2025-12-17 22:29:50,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ed575bc060ce15ec6a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:52,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089dc575bc060ce15ec43 2025-12-17 22:29:52,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089dc575bc060ce15ec43 HTTP/1.1" 502 150 2025-12-17 22:29:52,715 - lava_callback - ERROR - Error processing callback for node 694089dc575bc060ce15ec43: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089dc575bc060ce15ec43 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089dc575bc060ce15ec43 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089dc575bc060ce15ec43 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:52,717 - lava_callback - INFO - Starting async job submit for node 694089ee575bc060ce15ec6d 2025-12-17 22:29:52,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:52,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f8575bc060ce15ec88 2025-12-17 22:29:52,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f8575bc060ce15ec88 HTTP/1.1" 502 150 2025-12-17 22:29:52,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f8575bc060ce15ec88'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:52,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ee575bc060ce15ec6d HTTP/1.1" 502 150 2025-12-17 22:29:52,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ee575bc060ce15ec6d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:52,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ee575bc060ce15ec6d 2025-12-17 22:29:52,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ee575bc060ce15ec6d HTTP/1.1" 502 150 2025-12-17 22:29:52,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ee575bc060ce15ec6d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:52,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ea575bc060ce15ec64 2025-12-17 22:29:52,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ea575bc060ce15ec64 HTTP/1.1" 502 150 2025-12-17 22:29:52,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ea575bc060ce15ec64'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:53,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089eb575bc060ce15ec67 2025-12-17 22:29:53,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089eb575bc060ce15ec67 HTTP/1.1" 502 150 2025-12-17 22:29:53,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089eb575bc060ce15ec67'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:54,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ee575bc060ce15ec6d 2025-12-17 22:29:54,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ee575bc060ce15ec6d HTTP/1.1" 502 150 2025-12-17 22:29:54,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ee575bc060ce15ec6d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:54,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e9575bc060ce15ec61 2025-12-17 22:29:54,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e9575bc060ce15ec61 HTTP/1.1" 502 150 2025-12-17 22:29:54,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089e9575bc060ce15ec61'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:54,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ed575bc060ce15ec6a 2025-12-17 22:29:54,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ed575bc060ce15ec6a HTTP/1.1" 502 150 2025-12-17 22:29:54,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ed575bc060ce15ec6a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59620 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:29:56,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f9575bc060ce15ec8f 2025-12-17 22:29:56,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f9575bc060ce15ec8f HTTP/1.1" 502 150 2025-12-17 22:29:56,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f9575bc060ce15ec8f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:57,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089dd575bc060ce15ec46 2025-12-17 22:29:57,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089dd575bc060ce15ec46 HTTP/1.1" 502 150 2025-12-17 22:29:57,978 - lava_callback - ERROR - Error processing callback for node 694089dd575bc060ce15ec46: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089dd575bc060ce15ec46 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089dd575bc060ce15ec46 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089dd575bc060ce15ec46 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:29:57,980 - lava_callback - INFO - Starting async job submit for node 694089ef575bc060ce15ec70 2025-12-17 22:29:57,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:29:57,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ef575bc060ce15ec70 HTTP/1.1" 502 150 2025-12-17 22:29:57,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ef575bc060ce15ec70'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:57,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ef575bc060ce15ec70 2025-12-17 22:29:57,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ef575bc060ce15ec70 HTTP/1.1" 502 150 2025-12-17 22:29:57,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ef575bc060ce15ec70'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:58,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ee575bc060ce15ec6d 2025-12-17 22:29:58,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ee575bc060ce15ec6d HTTP/1.1" 502 150 2025-12-17 22:29:58,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ee575bc060ce15ec6d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:29:59,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ef575bc060ce15ec70 2025-12-17 22:29:59,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ef575bc060ce15ec70 HTTP/1.1" 502 150 2025-12-17 22:29:59,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ef575bc060ce15ec70'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:00,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089df575bc060ce15ec49 2025-12-17 22:30:00,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089df575bc060ce15ec49 HTTP/1.1" 502 150 2025-12-17 22:30:00,900 - lava_callback - ERROR - Error processing callback for node 694089df575bc060ce15ec49: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089df575bc060ce15ec49 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089df575bc060ce15ec49 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089df575bc060ce15ec49 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:00,903 - lava_callback - INFO - Starting async job submit for node 694089f3575bc060ce15ec7a 2025-12-17 22:30:00,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:00,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ea575bc060ce15ec64 2025-12-17 22:30:00,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f3575bc060ce15ec7a HTTP/1.1" 502 150 2025-12-17 22:30:00,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f3575bc060ce15ec7a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:00,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f3575bc060ce15ec7a 2025-12-17 22:30:00,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f3575bc060ce15ec7a HTTP/1.1" 502 150 2025-12-17 22:30:00,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f3575bc060ce15ec7a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:00,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ea575bc060ce15ec64 HTTP/1.1" 502 150 2025-12-17 22:30:00,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ea575bc060ce15ec64'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:01,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e4575bc060ce15ec55 2025-12-17 22:30:01,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e4575bc060ce15ec55 HTTP/1.1" 502 150 2025-12-17 22:30:01,633 - lava_callback - ERROR - Error processing callback for node 694089e4575bc060ce15ec55: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e4575bc060ce15ec55 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e4575bc060ce15ec55 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e4575bc060ce15ec55 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:01,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e3575bc060ce15ec4e 2025-12-17 22:30:01,636 - lava_callback - INFO - Starting async job submit for node 694089f4575bc060ce15ec7d 2025-12-17 22:30:01,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:01,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e3575bc060ce15ec4e HTTP/1.1" 502 150 2025-12-17 22:30:01,637 - lava_callback - ERROR - Error processing callback for node 694089e3575bc060ce15ec4e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e3575bc060ce15ec4e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e3575bc060ce15ec4e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e3575bc060ce15ec4e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:01,639 - lava_callback - INFO - Starting async job submit for node 694089f5575bc060ce15ec80 2025-12-17 22:30:01,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:01,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f4575bc060ce15ec7d HTTP/1.1" 502 150 2025-12-17 22:30:01,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f4575bc060ce15ec7d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:01,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f4575bc060ce15ec7d 2025-12-17 22:30:01,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f4575bc060ce15ec7d HTTP/1.1" 502 150 2025-12-17 22:30:01,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f4575bc060ce15ec7d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:01,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f5575bc060ce15ec80 HTTP/1.1" 502 150 2025-12-17 22:30:01,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f5575bc060ce15ec80'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:01,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f5575bc060ce15ec80 2025-12-17 22:30:01,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f5575bc060ce15ec80 HTTP/1.1" 502 150 2025-12-17 22:30:01,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f5575bc060ce15ec80'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:01,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089eb575bc060ce15ec67 2025-12-17 22:30:01,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089eb575bc060ce15ec67 HTTP/1.1" 502 150 2025-12-17 22:30:01,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089eb575bc060ce15ec67'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39428 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:30:02,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ed575bc060ce15ec6a 2025-12-17 22:30:02,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ed575bc060ce15ec6a HTTP/1.1" 502 150 2025-12-17 22:30:02,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ed575bc060ce15ec6a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:02,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f3575bc060ce15ec7a 2025-12-17 22:30:02,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f3575bc060ce15ec7a HTTP/1.1" 502 150 2025-12-17 22:30:02,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f3575bc060ce15ec7a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:03,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f4575bc060ce15ec7d 2025-12-17 22:30:03,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f4575bc060ce15ec7d HTTP/1.1" 502 150 2025-12-17 22:30:03,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f4575bc060ce15ec7d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:03,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f5575bc060ce15ec80 2025-12-17 22:30:03,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f5575bc060ce15ec80 HTTP/1.1" 502 150 2025-12-17 22:30:03,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f5575bc060ce15ec80'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:03,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ef575bc060ce15ec70 2025-12-17 22:30:03,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ef575bc060ce15ec70 HTTP/1.1" 502 150 2025-12-17 22:30:03,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ef575bc060ce15ec70'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:04,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e5575bc060ce15ec58 2025-12-17 22:30:04,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e5575bc060ce15ec58 HTTP/1.1" 502 150 2025-12-17 22:30:04,490 - lava_callback - ERROR - Error processing callback for node 694089e5575bc060ce15ec58: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e5575bc060ce15ec58 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e5575bc060ce15ec58 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e5575bc060ce15ec58 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:04,493 - lava_callback - INFO - Starting async job submit for node 694090f4575bc060ce16070b 2025-12-17 22:30:04,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:04,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f4575bc060ce16070b HTTP/1.1" 502 150 2025-12-17 22:30:04,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f4575bc060ce16070b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:04,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f4575bc060ce16070b 2025-12-17 22:30:04,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f4575bc060ce16070b HTTP/1.1" 502 150 2025-12-17 22:30:04,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f4575bc060ce16070b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:04,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408587575bc060ce15dc6f 2025-12-17 22:30:04,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408587575bc060ce15dc6f HTTP/1.1" 502 150 2025-12-17 22:30:04,663 - lava_callback - ERROR - Error processing callback for node 69408587575bc060ce15dc6f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408587575bc060ce15dc6f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408587575bc060ce15dc6f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408587575bc060ce15dc6f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:04,665 - lava_callback - INFO - Starting async job submit for node 694089fd575bc060ce15ec9b 2025-12-17 22:30:04,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:04,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089fd575bc060ce15ec9b HTTP/1.1" 502 150 2025-12-17 22:30:04,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089fd575bc060ce15ec9b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:04,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089fd575bc060ce15ec9b 2025-12-17 22:30:04,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089fd575bc060ce15ec9b HTTP/1.1" 502 150 2025-12-17 22:30:04,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089fd575bc060ce15ec9b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:04,822 - lava_callback - INFO - Received callback for node 69408bd8575bc060ce15f36f INFO: 172.21.0.1:39438 - "POST /node/69408bd8575bc060ce15f36f HTTP/1.0" 202 Accepted 2025-12-17 22:30:04,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e6575bc060ce15ec5b 2025-12-17 22:30:04,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e6575bc060ce15ec5b HTTP/1.1" 502 150 2025-12-17 22:30:04,855 - lava_callback - ERROR - Error processing callback for node 694089e6575bc060ce15ec5b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e6575bc060ce15ec5b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e6575bc060ce15ec5b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e6575bc060ce15ec5b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:04,857 - lava_callback - INFO - Starting async job submit for node 694089ff575bc060ce15ec9e 2025-12-17 22:30:04,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:04,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ff575bc060ce15ec9e HTTP/1.1" 502 150 2025-12-17 22:30:04,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ff575bc060ce15ec9e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:04,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ff575bc060ce15ec9e 2025-12-17 22:30:04,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ff575bc060ce15ec9e HTTP/1.1" 502 150 2025-12-17 22:30:04,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ff575bc060ce15ec9e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:05,139 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f7575bc060ce15ec83 2025-12-17 22:30:05,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f7575bc060ce15ec83 HTTP/1.1" 502 150 2025-12-17 22:30:05,140 - lava_callback - ERROR - Error processing callback for node 694089f7575bc060ce15ec83: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f7575bc060ce15ec83 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f7575bc060ce15ec83 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f7575bc060ce15ec83 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:05,142 - lava_callback - INFO - Starting async job submit for node 69408a00575bc060ce15eca1 2025-12-17 22:30:05,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:05,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a00575bc060ce15eca1 HTTP/1.1" 502 150 2025-12-17 22:30:05,173 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a00575bc060ce15eca1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:05,173 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a00575bc060ce15eca1 2025-12-17 22:30:05,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a00575bc060ce15eca1 HTTP/1.1" 502 150 2025-12-17 22:30:05,174 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a00575bc060ce15eca1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:05,481 - lava_callback - INFO - Received callback for node 69408bd9575bc060ce15f376 INFO: 172.21.0.1:39442 - "POST /node/69408bd9575bc060ce15f376 HTTP/1.0" 202 Accepted 2025-12-17 22:30:06,120 - lava_callback - INFO - Received callback for node 69408bda575bc060ce15f379 INFO: 172.21.0.1:39444 - "POST /node/69408bda575bc060ce15f379 HTTP/1.0" 202 Accepted 2025-12-17 22:30:06,234 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e8575bc060ce15ec5e 2025-12-17 22:30:06,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f4575bc060ce16070b 2025-12-17 22:30:06,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f4575bc060ce16070b HTTP/1.1" 502 150 2025-12-17 22:30:06,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f4575bc060ce16070b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39452 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:30:06,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089fd575bc060ce15ec9b 2025-12-17 22:30:06,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089fd575bc060ce15ec9b HTTP/1.1" 502 150 2025-12-17 22:30:06,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089fd575bc060ce15ec9b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:06,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ee575bc060ce15ec6d 2025-12-17 22:30:06,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ee575bc060ce15ec6d HTTP/1.1" 502 150 2025-12-17 22:30:06,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ee575bc060ce15ec6d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:06,758 - lava_callback - INFO - Received callback for node 69408bdb575bc060ce15f37c INFO: 172.21.0.1:39468 - "POST /node/69408bdb575bc060ce15f37c HTTP/1.0" 202 Accepted 2025-12-17 22:30:06,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ff575bc060ce15ec9e 2025-12-17 22:30:06,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ff575bc060ce15ec9e HTTP/1.1" 502 150 2025-12-17 22:30:06,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ff575bc060ce15ec9e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:06,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f3575bc060ce15ec7a 2025-12-17 22:30:06,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f3575bc060ce15ec7a HTTP/1.1" 502 150 2025-12-17 22:30:06,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f3575bc060ce15ec7a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:07,174 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a00575bc060ce15eca1 2025-12-17 22:30:07,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a00575bc060ce15eca1 HTTP/1.1" 502 150 2025-12-17 22:30:07,177 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a00575bc060ce15eca1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:07,404 - lava_callback - INFO - Received callback for node 69408bdd575bc060ce15f37f INFO: 172.21.0.1:39470 - "POST /node/69408bdd575bc060ce15f37f HTTP/1.0" 202 Accepted 2025-12-17 22:30:07,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f4575bc060ce15ec7d 2025-12-17 22:30:07,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f5575bc060ce15ec80 2025-12-17 22:30:07,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f4575bc060ce15ec7d HTTP/1.1" 502 150 2025-12-17 22:30:07,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f4575bc060ce15ec7d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:07,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f5575bc060ce15ec80 HTTP/1.1" 502 150 2025-12-17 22:30:07,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f5575bc060ce15ec80'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:08,054 - lava_callback - INFO - Received callback for node 69408bde575bc060ce15f382 INFO: 172.21.0.1:39474 - "POST /node/69408bde575bc060ce15f382 HTTP/1.0" 202 Accepted 2025-12-17 22:30:08,697 - lava_callback - INFO - Received callback for node 69408be0575bc060ce15f389 INFO: 172.21.0.1:39480 - "POST /node/69408be0575bc060ce15f389 HTTP/1.0" 202 Accepted 2025-12-17 22:30:08,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f8575bc060ce15ec88 2025-12-17 22:30:08,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f8575bc060ce15ec88 HTTP/1.1" 502 150 2025-12-17 22:30:08,720 - lava_callback - ERROR - Error processing callback for node 694089f8575bc060ce15ec88: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f8575bc060ce15ec88 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f8575bc060ce15ec88 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f8575bc060ce15ec88 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:08,723 - lava_callback - INFO - Starting async job submit for node 69408a02575bc060ce15eca4 2025-12-17 22:30:08,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:08,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a02575bc060ce15eca4 HTTP/1.1" 502 150 2025-12-17 22:30:08,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a02575bc060ce15eca4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:08,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a02575bc060ce15eca4 2025-12-17 22:30:08,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a02575bc060ce15eca4 HTTP/1.1" 502 150 2025-12-17 22:30:08,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a02575bc060ce15eca4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:09,347 - lava_callback - INFO - Received callback for node 69408be1575bc060ce15f38c INFO: 172.21.0.1:39484 - "POST /node/69408be1575bc060ce15f38c HTTP/1.0" 202 Accepted 2025-12-17 22:30:10,016 - lava_callback - INFO - Received callback for node 69408be3575bc060ce15f38f INFO: 172.21.0.1:37742 - "POST /node/69408be3575bc060ce15f38f HTTP/1.0" 202 Accepted 2025-12-17 22:30:10,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f4575bc060ce16070b 2025-12-17 22:30:10,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f4575bc060ce16070b HTTP/1.1" 502 150 2025-12-17 22:30:10,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f4575bc060ce16070b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:10,640 - lava_callback - INFO - Received callback for node 69408be4575bc060ce15f392 INFO: 172.21.0.1:37756 - "POST /node/69408be4575bc060ce15f392 HTTP/1.0" 202 Accepted 2025-12-17 22:30:10,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089fd575bc060ce15ec9b 2025-12-17 22:30:10,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089fd575bc060ce15ec9b HTTP/1.1" 502 150 2025-12-17 22:30:10,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089fd575bc060ce15ec9b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:10,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a02575bc060ce15eca4 2025-12-17 22:30:10,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a02575bc060ce15eca4 HTTP/1.1" 502 150 2025-12-17 22:30:10,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a02575bc060ce15eca4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:10,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089e9575bc060ce15ec61 2025-12-17 22:30:10,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e9575bc060ce15ec61 HTTP/1.1" 502 150 2025-12-17 22:30:10,795 - lava_callback - ERROR - Error processing callback for node 694089e9575bc060ce15ec61: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e9575bc060ce15ec61 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e9575bc060ce15ec61 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e9575bc060ce15ec61 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:10,798 - lava_callback - INFO - Starting async job submit for node 69408955575bc060ce15e9df 2025-12-17 22:30:10,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ff575bc060ce15ec9e 2025-12-17 22:30:10,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ff575bc060ce15ec9e HTTP/1.1" 502 150 2025-12-17 22:30:10,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ff575bc060ce15ec9e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:10,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:10,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408955575bc060ce15e9df HTTP/1.1" 502 150 2025-12-17 22:30:10,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408955575bc060ce15e9df'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:10,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408955575bc060ce15e9df 2025-12-17 22:30:10,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408955575bc060ce15e9df HTTP/1.1" 502 150 2025-12-17 22:30:10,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408955575bc060ce15e9df'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:11,177 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a00575bc060ce15eca1 2025-12-17 22:30:11,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a00575bc060ce15eca1 HTTP/1.1" 502 150 2025-12-17 22:30:11,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a00575bc060ce15eca1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:11,282 - lava_callback - INFO - Received callback for node 69408be5575bc060ce15f395 INFO: 172.21.0.1:37758 - "POST /node/69408be5575bc060ce15f395 HTTP/1.0" 202 Accepted 2025-12-17 22:30:11,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ef575bc060ce15ec70 2025-12-17 22:30:11,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ef575bc060ce15ec70 HTTP/1.1" 502 150 2025-12-17 22:30:11,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ef575bc060ce15ec70'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:12,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f9575bc060ce15ec8f 2025-12-17 22:30:12,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f9575bc060ce15ec8f HTTP/1.1" 502 150 2025-12-17 22:30:12,727 - lava_callback - ERROR - Error processing callback for node 694089f9575bc060ce15ec8f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f9575bc060ce15ec8f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f9575bc060ce15ec8f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f9575bc060ce15ec8f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:12,728 - lava_callback - INFO - Starting async job submit for node 69408a03575bc060ce15eca7 2025-12-17 22:30:12,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:12,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a03575bc060ce15eca7 HTTP/1.1" 502 150 2025-12-17 22:30:12,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a03575bc060ce15eca7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:12,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a03575bc060ce15eca7 2025-12-17 22:30:12,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a03575bc060ce15eca7 HTTP/1.1" 502 150 2025-12-17 22:30:12,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a03575bc060ce15eca7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:12,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408955575bc060ce15e9df 2025-12-17 22:30:12,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408955575bc060ce15e9df HTTP/1.1" 502 150 2025-12-17 22:30:12,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408955575bc060ce15e9df'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:14,693 - lava_callback - INFO - Received callback for node 69408bee575bc060ce15f3b0 INFO: 172.21.0.1:37774 - "POST /node/69408bee575bc060ce15f3b0 HTTP/1.0" 202 Accepted 2025-12-17 22:30:14,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a02575bc060ce15eca4 2025-12-17 22:30:14,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a02575bc060ce15eca4 HTTP/1.1" 502 150 2025-12-17 22:30:14,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a02575bc060ce15eca4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:14,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a03575bc060ce15eca7 2025-12-17 22:30:14,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a03575bc060ce15eca7 HTTP/1.1" 502 150 2025-12-17 22:30:14,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a03575bc060ce15eca7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:14,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f3575bc060ce15ec7a 2025-12-17 22:30:14,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f3575bc060ce15ec7a HTTP/1.1" 502 150 2025-12-17 22:30:14,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f3575bc060ce15ec7a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37788 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:30:15,368 - lava_callback - INFO - Received callback for node 69408bf1575bc060ce15f3bf INFO: 172.21.0.1:37790 - "POST /node/69408bf1575bc060ce15f3bf HTTP/1.0" 202 Accepted 2025-12-17 22:30:15,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f4575bc060ce15ec7d 2025-12-17 22:30:15,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f5575bc060ce15ec80 2025-12-17 22:30:15,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f4575bc060ce15ec7d HTTP/1.1" 502 150 2025-12-17 22:30:15,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f4575bc060ce15ec7d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:15,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f5575bc060ce15ec80 HTTP/1.1" 502 150 2025-12-17 22:30:15,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089f5575bc060ce15ec80'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:16,067 - lava_callback - INFO - Received callback for node 69408bf2575bc060ce15f3c2 INFO: 172.21.0.1:37798 - "POST /node/69408bf2575bc060ce15f3c2 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:37804 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:30:16,780 - lava_callback - INFO - Received callback for node 69408bf3575bc060ce15f3c5 INFO: 172.21.0.1:37812 - "POST /node/69408bf3575bc060ce15f3c5 HTTP/1.0" 202 Accepted 2025-12-17 22:30:16,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ea575bc060ce15ec64 2025-12-17 22:30:16,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ea575bc060ce15ec64 HTTP/1.1" 502 150 2025-12-17 22:30:16,918 - lava_callback - ERROR - Error processing callback for node 694089ea575bc060ce15ec64: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ea575bc060ce15ec64 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ea575bc060ce15ec64 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ea575bc060ce15ec64 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:16,919 - lava_callback - INFO - Starting async job submit for node 69408a04575bc060ce15ecaa 2025-12-17 22:30:16,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:16,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408955575bc060ce15e9df 2025-12-17 22:30:16,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408955575bc060ce15e9df HTTP/1.1" 502 150 2025-12-17 22:30:16,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408955575bc060ce15e9df'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:16,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a04575bc060ce15ecaa HTTP/1.1" 502 150 2025-12-17 22:30:16,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a04575bc060ce15ecaa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:16,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a04575bc060ce15ecaa 2025-12-17 22:30:16,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a04575bc060ce15ecaa HTTP/1.1" 502 150 2025-12-17 22:30:16,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a04575bc060ce15ecaa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:17,462 - lava_callback - INFO - Received callback for node 69408bf5575bc060ce15f3c8 INFO: 172.21.0.1:37820 - "POST /node/69408bf5575bc060ce15f3c8 HTTP/1.0" 202 Accepted 2025-12-17 22:30:17,901 - lava_callback - INFO - Received callback for node 694092fd575bc060ce160bf7 INFO: 172.21.0.1:37832 - "POST /node/694092fd575bc060ce160bf7 HTTP/1.0" 202 Accepted 2025-12-17 22:30:17,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089eb575bc060ce15ec67 2025-12-17 22:30:17,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089eb575bc060ce15ec67 HTTP/1.1" 502 150 2025-12-17 22:30:17,972 - lava_callback - ERROR - Error processing callback for node 694089eb575bc060ce15ec67: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089eb575bc060ce15ec67 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089eb575bc060ce15ec67 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089eb575bc060ce15ec67 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:17,973 - lava_callback - INFO - Starting async job submit for node 69408a09575bc060ce15ecad 2025-12-17 22:30:17,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:17,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a09575bc060ce15ecad HTTP/1.1" 502 150 2025-12-17 22:30:17,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a09575bc060ce15ecad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:17,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a09575bc060ce15ecad 2025-12-17 22:30:17,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a09575bc060ce15ecad HTTP/1.1" 502 150 2025-12-17 22:30:17,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a09575bc060ce15ecad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:18,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f4575bc060ce16070b 2025-12-17 22:30:18,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f4575bc060ce16070b HTTP/1.1" 502 150 2025-12-17 22:30:18,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f4575bc060ce16070b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:18,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089fd575bc060ce15ec9b 2025-12-17 22:30:18,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089fd575bc060ce15ec9b HTTP/1.1" 502 150 2025-12-17 22:30:18,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089fd575bc060ce15ec9b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:18,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a03575bc060ce15eca7 2025-12-17 22:30:18,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a03575bc060ce15eca7 HTTP/1.1" 502 150 2025-12-17 22:30:18,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a03575bc060ce15eca7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:18,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ff575bc060ce15ec9e 2025-12-17 22:30:18,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ff575bc060ce15ec9e HTTP/1.1" 502 150 2025-12-17 22:30:18,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694089ff575bc060ce15ec9e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:18,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ed575bc060ce15ec6a 2025-12-17 22:30:18,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ed575bc060ce15ec6a HTTP/1.1" 502 150 2025-12-17 22:30:18,897 - lava_callback - ERROR - Error processing callback for node 694089ed575bc060ce15ec6a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ed575bc060ce15ec6a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ed575bc060ce15ec6a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ed575bc060ce15ec6a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:18,899 - lava_callback - INFO - Starting async job submit for node 69408a0b575bc060ce15ecb0 2025-12-17 22:30:18,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:18,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0b575bc060ce15ecb0 HTTP/1.1" 502 150 2025-12-17 22:30:18,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0b575bc060ce15ecb0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:18,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0b575bc060ce15ecb0 2025-12-17 22:30:18,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0b575bc060ce15ecb0 HTTP/1.1" 502 150 2025-12-17 22:30:18,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0b575bc060ce15ecb0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:18,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a04575bc060ce15ecaa 2025-12-17 22:30:18,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a04575bc060ce15ecaa HTTP/1.1" 502 150 2025-12-17 22:30:18,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a04575bc060ce15ecaa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:19,179 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a00575bc060ce15eca1 2025-12-17 22:30:19,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a00575bc060ce15eca1 HTTP/1.1" 502 150 2025-12-17 22:30:19,180 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a00575bc060ce15eca1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:19,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a09575bc060ce15ecad 2025-12-17 22:30:19,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a09575bc060ce15ecad HTTP/1.1" 502 150 2025-12-17 22:30:19,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a09575bc060ce15ecad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:20,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0b575bc060ce15ecb0 2025-12-17 22:30:20,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0b575bc060ce15ecb0 HTTP/1.1" 502 150 2025-12-17 22:30:20,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0b575bc060ce15ecb0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:21,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089e8575bc060ce15ec5e HTTP/1.1" 502 150 2025-12-17 22:30:21,616 - lava_callback - ERROR - Error processing callback for node 694089e8575bc060ce15ec5e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e8575bc060ce15ec5e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e8575bc060ce15ec5e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089e8575bc060ce15ec5e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:21,618 - lava_callback - INFO - Starting async job submit for node 69408a0c575bc060ce15ecb6 2025-12-17 22:30:21,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:21,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0c575bc060ce15ecb6 HTTP/1.1" 502 150 2025-12-17 22:30:21,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0c575bc060ce15ecb6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:21,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0c575bc060ce15ecb6 2025-12-17 22:30:21,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0c575bc060ce15ecb6 HTTP/1.1" 502 150 2025-12-17 22:30:21,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0c575bc060ce15ecb6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:22,717 - lava_callback - INFO - Received callback for node 69408bea575bc060ce15f3a1 INFO: 172.21.0.1:33036 - "POST /node/69408bea575bc060ce15f3a1 HTTP/1.0" 202 Accepted 2025-12-17 22:30:22,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a02575bc060ce15eca4 2025-12-17 22:30:22,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a02575bc060ce15eca4 HTTP/1.1" 502 150 2025-12-17 22:30:22,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a02575bc060ce15eca4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:22,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ee575bc060ce15ec6d 2025-12-17 22:30:22,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ee575bc060ce15ec6d HTTP/1.1" 502 150 2025-12-17 22:30:22,746 - lava_callback - ERROR - Error processing callback for node 694089ee575bc060ce15ec6d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ee575bc060ce15ec6d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ee575bc060ce15ec6d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ee575bc060ce15ec6d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:22,748 - lava_callback - INFO - Starting async job submit for node 69408a0d575bc060ce15ecbe 2025-12-17 22:30:22,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:22,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0d575bc060ce15ecbe HTTP/1.1" 502 150 2025-12-17 22:30:22,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0d575bc060ce15ecbe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:22,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0d575bc060ce15ecbe 2025-12-17 22:30:22,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0d575bc060ce15ecbe HTTP/1.1" 502 150 2025-12-17 22:30:22,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0d575bc060ce15ecbe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:22,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a04575bc060ce15ecaa 2025-12-17 22:30:22,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a04575bc060ce15ecaa HTTP/1.1" 502 150 2025-12-17 22:30:22,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a04575bc060ce15ecaa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:23,491 - lava_callback - INFO - Received callback for node 69408beb575bc060ce15f3a4 INFO: 172.21.0.1:33046 - "POST /node/69408beb575bc060ce15f3a4 HTTP/1.0" 202 Accepted 2025-12-17 22:30:23,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0c575bc060ce15ecb6 2025-12-17 22:30:23,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a09575bc060ce15ecad 2025-12-17 22:30:23,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a09575bc060ce15ecad HTTP/1.1" 502 150 2025-12-17 22:30:23,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a09575bc060ce15ecad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:24,285 - lava_callback - INFO - Received callback for node 69408bed575bc060ce15f3a9 INFO: 172.21.0.1:33060 - "POST /node/69408bed575bc060ce15f3a9 HTTP/1.0" 202 Accepted 2025-12-17 22:30:24,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0d575bc060ce15ecbe 2025-12-17 22:30:24,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0d575bc060ce15ecbe HTTP/1.1" 502 150 2025-12-17 22:30:24,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0d575bc060ce15ecbe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:24,819 - lava_callback - INFO - Received callback for node 69408bf7575bc060ce15f3ce INFO: 172.21.0.1:33072 - "POST /node/69408bf7575bc060ce15f3ce HTTP/1.0" 202 Accepted 2025-12-17 22:30:24,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408955575bc060ce15e9df 2025-12-17 22:30:24,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408955575bc060ce15e9df HTTP/1.1" 502 150 2025-12-17 22:30:24,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408955575bc060ce15e9df'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:24,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0b575bc060ce15ecb0 2025-12-17 22:30:24,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0b575bc060ce15ecb0 HTTP/1.1" 502 150 2025-12-17 22:30:24,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0b575bc060ce15ecb0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:25,468 - lava_callback - INFO - Received callback for node 69408bfa575bc060ce15f3d1 INFO: 172.21.0.1:33080 - "POST /node/69408bfa575bc060ce15f3d1 HTTP/1.0" 202 Accepted 2025-12-17 22:30:26,126 - lava_callback - INFO - Received callback for node 69408bfc575bc060ce15f3d4 INFO: 172.21.0.1:33086 - "POST /node/69408bfc575bc060ce15f3d4 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:33096 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:30:26,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a03575bc060ce15eca7 2025-12-17 22:30:26,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a03575bc060ce15eca7 HTTP/1.1" 502 150 2025-12-17 22:30:26,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a03575bc060ce15eca7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:26,772 - lava_callback - INFO - Received callback for node 69408bfe575bc060ce15f3d7 INFO: 172.21.0.1:33102 - "POST /node/69408bfe575bc060ce15f3d7 HTTP/1.0" 202 Accepted 2025-12-17 22:30:27,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ef575bc060ce15ec70 2025-12-17 22:30:27,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ef575bc060ce15ec70 HTTP/1.1" 502 150 2025-12-17 22:30:27,996 - lava_callback - ERROR - Error processing callback for node 694089ef575bc060ce15ec70: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ef575bc060ce15ec70 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ef575bc060ce15ec70 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ef575bc060ce15ec70 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:27,999 - lava_callback - INFO - Starting async job submit for node 69408a0f575bc060ce15ecc3 2025-12-17 22:30:28,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:28,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0f575bc060ce15ecc3 HTTP/1.1" 502 150 2025-12-17 22:30:28,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0f575bc060ce15ecc3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:28,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0f575bc060ce15ecc3 2025-12-17 22:30:28,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0f575bc060ce15ecc3 HTTP/1.1" 502 150 2025-12-17 22:30:28,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0f575bc060ce15ecc3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:28,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0d575bc060ce15ecbe 2025-12-17 22:30:28,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0d575bc060ce15ecbe HTTP/1.1" 502 150 2025-12-17 22:30:28,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0d575bc060ce15ecbe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:30,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0f575bc060ce15ecc3 2025-12-17 22:30:30,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0f575bc060ce15ecc3 HTTP/1.1" 502 150 2025-12-17 22:30:30,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0f575bc060ce15ecc3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:30,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f3575bc060ce15ec7a 2025-12-17 22:30:30,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f3575bc060ce15ec7a HTTP/1.1" 502 150 2025-12-17 22:30:30,917 - lava_callback - ERROR - Error processing callback for node 694089f3575bc060ce15ec7a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f3575bc060ce15ec7a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f3575bc060ce15ec7a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f3575bc060ce15ec7a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:30,918 - lava_callback - INFO - Starting async job submit for node 69408a10575bc060ce15ecc6 2025-12-17 22:30:30,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:30,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a04575bc060ce15ecaa 2025-12-17 22:30:30,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a10575bc060ce15ecc6 HTTP/1.1" 502 150 2025-12-17 22:30:30,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a10575bc060ce15ecc6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:30,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a10575bc060ce15ecc6 2025-12-17 22:30:30,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a04575bc060ce15ecaa HTTP/1.1" 502 150 2025-12-17 22:30:30,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a04575bc060ce15ecaa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:30,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a10575bc060ce15ecc6 HTTP/1.1" 502 150 2025-12-17 22:30:30,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a10575bc060ce15ecc6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:31,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f4575bc060ce15ec7d 2025-12-17 22:30:31,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089f5575bc060ce15ec80 2025-12-17 22:30:31,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f4575bc060ce15ec7d HTTP/1.1" 502 150 2025-12-17 22:30:31,652 - lava_callback - ERROR - Error processing callback for node 694089f4575bc060ce15ec7d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f4575bc060ce15ec7d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f4575bc060ce15ec7d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f4575bc060ce15ec7d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:31,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089f5575bc060ce15ec80 HTTP/1.1" 502 150 2025-12-17 22:30:31,655 - lava_callback - INFO - Starting async job submit for node 69408a11575bc060ce15ecc9 2025-12-17 22:30:31,655 - lava_callback - ERROR - Error processing callback for node 694089f5575bc060ce15ec80: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f5575bc060ce15ec80 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f5575bc060ce15ec80 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089f5575bc060ce15ec80 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:31,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:31,658 - lava_callback - INFO - Starting async job submit for node 69408a14575bc060ce15eccc 2025-12-17 22:30:31,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:31,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a11575bc060ce15ecc9 HTTP/1.1" 502 150 2025-12-17 22:30:31,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a11575bc060ce15ecc9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:31,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a11575bc060ce15ecc9 2025-12-17 22:30:31,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a11575bc060ce15ecc9 HTTP/1.1" 502 150 2025-12-17 22:30:31,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a11575bc060ce15ecc9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:31,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a14575bc060ce15eccc HTTP/1.1" 502 150 2025-12-17 22:30:31,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a14575bc060ce15eccc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:31,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a14575bc060ce15eccc 2025-12-17 22:30:31,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a14575bc060ce15eccc HTTP/1.1" 502 150 2025-12-17 22:30:31,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a14575bc060ce15eccc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:31,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a09575bc060ce15ecad 2025-12-17 22:30:31,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a09575bc060ce15ecad HTTP/1.1" 502 150 2025-12-17 22:30:31,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a09575bc060ce15ecad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:32,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0b575bc060ce15ecb0 2025-12-17 22:30:32,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a10575bc060ce15ecc6 2025-12-17 22:30:32,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a10575bc060ce15ecc6 HTTP/1.1" 502 150 2025-12-17 22:30:32,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a10575bc060ce15ecc6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:33,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a11575bc060ce15ecc9 2025-12-17 22:30:33,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a11575bc060ce15ecc9 HTTP/1.1" 502 150 2025-12-17 22:30:33,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a11575bc060ce15ecc9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:33,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a14575bc060ce15eccc 2025-12-17 22:30:33,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a14575bc060ce15eccc HTTP/1.1" 502 150 2025-12-17 22:30:33,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a14575bc060ce15eccc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:34,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0f575bc060ce15ecc3 2025-12-17 22:30:34,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f4575bc060ce16070b 2025-12-17 22:30:34,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f4575bc060ce16070b HTTP/1.1" 502 150 2025-12-17 22:30:34,558 - lava_callback - ERROR - Error processing callback for node 694090f4575bc060ce16070b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f4575bc060ce16070b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f4575bc060ce16070b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f4575bc060ce16070b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:34,560 - lava_callback - INFO - Starting async job submit for node 69408a18575bc060ce15ed6e 2025-12-17 22:30:34,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:34,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a18575bc060ce15ed6e HTTP/1.1" 502 150 2025-12-17 22:30:34,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a18575bc060ce15ed6e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:34,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a18575bc060ce15ed6e 2025-12-17 22:30:34,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a18575bc060ce15ed6e HTTP/1.1" 502 150 2025-12-17 22:30:34,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a18575bc060ce15ed6e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:34,625 - lava_callback - INFO - Received callback for node 69408bf6575bc060ce15f3cb INFO: 172.21.0.1:53722 - "POST /node/69408bf6575bc060ce15f3cb HTTP/1.0" 202 Accepted 2025-12-17 22:30:34,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089fd575bc060ce15ec9b 2025-12-17 22:30:34,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089fd575bc060ce15ec9b HTTP/1.1" 502 150 2025-12-17 22:30:34,678 - lava_callback - ERROR - Error processing callback for node 694089fd575bc060ce15ec9b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089fd575bc060ce15ec9b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089fd575bc060ce15ec9b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089fd575bc060ce15ec9b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:34,680 - lava_callback - INFO - Starting async job submit for node 69408a1a575bc060ce15ed71 2025-12-17 22:30:34,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:34,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1a575bc060ce15ed71 HTTP/1.1" 502 150 2025-12-17 22:30:34,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a1a575bc060ce15ed71'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:34,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a1a575bc060ce15ed71 2025-12-17 22:30:34,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1a575bc060ce15ed71 HTTP/1.1" 502 150 2025-12-17 22:30:34,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a1a575bc060ce15ed71'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:34,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694089ff575bc060ce15ec9e 2025-12-17 22:30:34,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694089ff575bc060ce15ec9e HTTP/1.1" 502 150 2025-12-17 22:30:34,891 - lava_callback - ERROR - Error processing callback for node 694089ff575bc060ce15ec9e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ff575bc060ce15ec9e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ff575bc060ce15ec9e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694089ff575bc060ce15ec9e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:34,894 - lava_callback - INFO - Starting async job submit for node 69408a1c575bc060ce15ed74 2025-12-17 22:30:34,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:34,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1c575bc060ce15ed74 HTTP/1.1" 502 150 2025-12-17 22:30:34,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a1c575bc060ce15ed74'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:34,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a1c575bc060ce15ed74 2025-12-17 22:30:34,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1c575bc060ce15ed74 HTTP/1.1" 502 150 2025-12-17 22:30:34,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a1c575bc060ce15ed74'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:35,180 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a00575bc060ce15eca1 2025-12-17 22:30:35,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a00575bc060ce15eca1 HTTP/1.1" 502 150 2025-12-17 22:30:35,181 - lava_callback - ERROR - Error processing callback for node 69408a00575bc060ce15eca1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a00575bc060ce15eca1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a00575bc060ce15eca1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a00575bc060ce15eca1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:35,183 - lava_callback - INFO - Starting async job submit for node 69408a1e575bc060ce15ed77 2025-12-17 22:30:35,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:35,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1e575bc060ce15ed77 HTTP/1.1" 502 150 2025-12-17 22:30:35,190 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a1e575bc060ce15ed77'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:35,190 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a1e575bc060ce15ed77 2025-12-17 22:30:35,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1e575bc060ce15ed77 HTTP/1.1" 502 150 2025-12-17 22:30:35,190 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a1e575bc060ce15ed77'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53736 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:30:36,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a18575bc060ce15ed6e 2025-12-17 22:30:36,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a18575bc060ce15ed6e HTTP/1.1" 502 150 2025-12-17 22:30:36,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a18575bc060ce15ed6e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:36,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a1a575bc060ce15ed71 2025-12-17 22:30:36,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1a575bc060ce15ed71 HTTP/1.1" 502 150 2025-12-17 22:30:36,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a1a575bc060ce15ed71'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:36,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0d575bc060ce15ecbe 2025-12-17 22:30:36,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0d575bc060ce15ecbe HTTP/1.1" 502 150 2025-12-17 22:30:36,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0d575bc060ce15ecbe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:36,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a1c575bc060ce15ed74 2025-12-17 22:30:36,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1c575bc060ce15ed74 HTTP/1.1" 502 150 2025-12-17 22:30:36,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a1c575bc060ce15ed74'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:36,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a10575bc060ce15ecc6 2025-12-17 22:30:36,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a10575bc060ce15ecc6 HTTP/1.1" 502 150 2025-12-17 22:30:36,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a10575bc060ce15ecc6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:37,191 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a1e575bc060ce15ed77 2025-12-17 22:30:37,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1e575bc060ce15ed77 HTTP/1.1" 502 150 2025-12-17 22:30:37,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a1e575bc060ce15ed77'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:37,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a11575bc060ce15ecc9 2025-12-17 22:30:37,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a11575bc060ce15ecc9 HTTP/1.1" 502 150 2025-12-17 22:30:37,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a11575bc060ce15ecc9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:37,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a14575bc060ce15eccc 2025-12-17 22:30:37,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a14575bc060ce15eccc HTTP/1.1" 502 150 2025-12-17 22:30:37,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a14575bc060ce15eccc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:38,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a02575bc060ce15eca4 2025-12-17 22:30:38,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a02575bc060ce15eca4 HTTP/1.1" 502 150 2025-12-17 22:30:38,739 - lava_callback - ERROR - Error processing callback for node 69408a02575bc060ce15eca4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a02575bc060ce15eca4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a02575bc060ce15eca4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a02575bc060ce15eca4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:38,741 - lava_callback - INFO - Starting async job submit for node 69408a20575bc060ce15ed7c 2025-12-17 22:30:38,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:38,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a20575bc060ce15ed7c HTTP/1.1" 502 150 2025-12-17 22:30:38,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a20575bc060ce15ed7c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:38,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a20575bc060ce15ed7c 2025-12-17 22:30:38,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a20575bc060ce15ed7c HTTP/1.1" 502 150 2025-12-17 22:30:38,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a20575bc060ce15ed7c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:39,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0c575bc060ce15ecb6 HTTP/1.1" 502 150 2025-12-17 22:30:39,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0c575bc060ce15ecb6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:40,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a18575bc060ce15ed6e 2025-12-17 22:30:40,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a18575bc060ce15ed6e HTTP/1.1" 502 150 2025-12-17 22:30:40,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a18575bc060ce15ed6e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:40,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a1a575bc060ce15ed71 2025-12-17 22:30:40,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1a575bc060ce15ed71 HTTP/1.1" 502 150 2025-12-17 22:30:40,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a1a575bc060ce15ed71'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:40,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a20575bc060ce15ed7c 2025-12-17 22:30:40,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a20575bc060ce15ed7c HTTP/1.1" 502 150 2025-12-17 22:30:40,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a20575bc060ce15ed7c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:40,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a1c575bc060ce15ed74 2025-12-17 22:30:40,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1c575bc060ce15ed74 HTTP/1.1" 502 150 2025-12-17 22:30:40,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a1c575bc060ce15ed74'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:40,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408955575bc060ce15e9df 2025-12-17 22:30:40,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408955575bc060ce15e9df HTTP/1.1" 502 150 2025-12-17 22:30:40,935 - lava_callback - ERROR - Error processing callback for node 69408955575bc060ce15e9df: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408955575bc060ce15e9df (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408955575bc060ce15e9df (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408955575bc060ce15e9df (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:40,937 - lava_callback - INFO - Starting async job submit for node 69408a22575bc060ce15ed83 2025-12-17 22:30:40,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:40,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a22575bc060ce15ed83 HTTP/1.1" 502 150 2025-12-17 22:30:40,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a22575bc060ce15ed83'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:40,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a22575bc060ce15ed83 2025-12-17 22:30:40,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a22575bc060ce15ed83 HTTP/1.1" 502 150 2025-12-17 22:30:40,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a22575bc060ce15ed83'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:41,192 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a1e575bc060ce15ed77 2025-12-17 22:30:41,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1e575bc060ce15ed77 HTTP/1.1" 502 150 2025-12-17 22:30:41,193 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a1e575bc060ce15ed77'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:42,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a03575bc060ce15eca7 2025-12-17 22:30:42,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a03575bc060ce15eca7 HTTP/1.1" 502 150 2025-12-17 22:30:42,742 - lava_callback - ERROR - Error processing callback for node 69408a03575bc060ce15eca7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a03575bc060ce15eca7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a03575bc060ce15eca7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a03575bc060ce15eca7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:42,744 - lava_callback - INFO - Starting async job submit for node 69408a25575bc060ce15ed86 2025-12-17 22:30:42,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:42,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a25575bc060ce15ed86 HTTP/1.1" 502 150 2025-12-17 22:30:42,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a25575bc060ce15ed86'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:42,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a25575bc060ce15ed86 2025-12-17 22:30:42,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a25575bc060ce15ed86 HTTP/1.1" 502 150 2025-12-17 22:30:42,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a25575bc060ce15ed86'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:42,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a22575bc060ce15ed83 2025-12-17 22:30:42,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a22575bc060ce15ed83 HTTP/1.1" 502 150 2025-12-17 22:30:42,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a22575bc060ce15ed83'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:43,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0c575bc060ce15ecb6 2025-12-17 22:30:43,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0c575bc060ce15ecb6 HTTP/1.1" 502 150 2025-12-17 22:30:43,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0c575bc060ce15ecb6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:44,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a25575bc060ce15ed86 2025-12-17 22:30:44,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a20575bc060ce15ed7c 2025-12-17 22:30:44,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a20575bc060ce15ed7c HTTP/1.1" 502 150 2025-12-17 22:30:44,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a20575bc060ce15ed7c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:44,831 - lava_callback - INFO - Received callback for node 69408c02575bc060ce15f3e7 INFO: 172.21.0.1:52066 - "POST /node/69408c02575bc060ce15f3e7 HTTP/1.0" 202 Accepted 2025-12-17 22:30:44,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a10575bc060ce15ecc6 2025-12-17 22:30:44,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a10575bc060ce15ecc6 HTTP/1.1" 502 150 2025-12-17 22:30:44,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a10575bc060ce15ecc6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:45,486 - lava_callback - INFO - Received callback for node 69408c04575bc060ce15f3f0 INFO: 172.21.0.1:52070 - "POST /node/69408c04575bc060ce15f3f0 HTTP/1.0" 202 Accepted 2025-12-17 22:30:45,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a11575bc060ce15ecc9 2025-12-17 22:30:45,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a11575bc060ce15ecc9 HTTP/1.1" 502 150 2025-12-17 22:30:45,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a11575bc060ce15ecc9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:45,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a14575bc060ce15eccc 2025-12-17 22:30:45,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a14575bc060ce15eccc HTTP/1.1" 502 150 2025-12-17 22:30:45,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a14575bc060ce15eccc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:46,125 - lava_callback - INFO - Received callback for node 69408c0b575bc060ce15f437 INFO: 172.21.0.1:52086 - "POST /node/69408c0b575bc060ce15f437 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:52100 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:30:46,771 - lava_callback - INFO - Received callback for node 69408c0d575bc060ce15f43a INFO: 172.21.0.1:52106 - "POST /node/69408c0d575bc060ce15f43a HTTP/1.0" 202 Accepted 2025-12-17 22:30:46,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a22575bc060ce15ed83 2025-12-17 22:30:46,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a22575bc060ce15ed83 HTTP/1.1" 502 150 2025-12-17 22:30:46,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a22575bc060ce15ed83'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:46,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a04575bc060ce15ecaa 2025-12-17 22:30:46,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a04575bc060ce15ecaa HTTP/1.1" 502 150 2025-12-17 22:30:46,962 - lava_callback - ERROR - Error processing callback for node 69408a04575bc060ce15ecaa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a04575bc060ce15ecaa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a04575bc060ce15ecaa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a04575bc060ce15ecaa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:46,964 - lava_callback - INFO - Starting async job submit for node 69408a26575bc060ce15ed8e 2025-12-17 22:30:46,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:46,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a26575bc060ce15ed8e HTTP/1.1" 502 150 2025-12-17 22:30:46,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a26575bc060ce15ed8e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:46,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a26575bc060ce15ed8e 2025-12-17 22:30:46,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a26575bc060ce15ed8e HTTP/1.1" 502 150 2025-12-17 22:30:46,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a26575bc060ce15ed8e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:47,419 - lava_callback - INFO - Received callback for node 69408c0f575bc060ce15f43d INFO: 172.21.0.1:52114 - "POST /node/69408c0f575bc060ce15f43d HTTP/1.0" 202 Accepted 2025-12-17 22:30:47,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a09575bc060ce15ecad 2025-12-17 22:30:47,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a09575bc060ce15ecad HTTP/1.1" 502 150 2025-12-17 22:30:47,990 - lava_callback - ERROR - Error processing callback for node 69408a09575bc060ce15ecad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a09575bc060ce15ecad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a09575bc060ce15ecad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a09575bc060ce15ecad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:47,992 - lava_callback - INFO - Starting async job submit for node 69408a28575bc060ce15ed91 2025-12-17 22:30:47,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:48,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a28575bc060ce15ed91 HTTP/1.1" 502 150 2025-12-17 22:30:48,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a28575bc060ce15ed91'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:48,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a28575bc060ce15ed91 2025-12-17 22:30:48,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a28575bc060ce15ed91 HTTP/1.1" 502 150 2025-12-17 22:30:48,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a28575bc060ce15ed91'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:48,081 - lava_callback - INFO - Received callback for node 69408c11575bc060ce15f440 INFO: 172.21.0.1:52126 - "POST /node/69408c11575bc060ce15f440 HTTP/1.0" 202 Accepted 2025-12-17 22:30:48,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0b575bc060ce15ecb0 HTTP/1.1" 502 150 2025-12-17 22:30:48,240 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0b575bc060ce15ecb0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:48,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a18575bc060ce15ed6e 2025-12-17 22:30:48,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a18575bc060ce15ed6e HTTP/1.1" 502 150 2025-12-17 22:30:48,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a18575bc060ce15ed6e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:48,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a1a575bc060ce15ed71 2025-12-17 22:30:48,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1a575bc060ce15ed71 HTTP/1.1" 502 150 2025-12-17 22:30:48,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a1a575bc060ce15ed71'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:48,720 - lava_callback - INFO - Received callback for node 69408c13575bc060ce15f445 INFO: 172.21.0.1:52136 - "POST /node/69408c13575bc060ce15f445 HTTP/1.0" 202 Accepted 2025-12-17 22:30:48,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a1c575bc060ce15ed74 2025-12-17 22:30:48,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1c575bc060ce15ed74 HTTP/1.1" 502 150 2025-12-17 22:30:48,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a1c575bc060ce15ed74'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:48,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a26575bc060ce15ed8e 2025-12-17 22:30:48,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a26575bc060ce15ed8e HTTP/1.1" 502 150 2025-12-17 22:30:48,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a26575bc060ce15ed8e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:49,194 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a1e575bc060ce15ed77 2025-12-17 22:30:49,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1e575bc060ce15ed77 HTTP/1.1" 502 150 2025-12-17 22:30:49,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a1e575bc060ce15ed77'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:49,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0f575bc060ce15ecc3 HTTP/1.1" 502 150 2025-12-17 22:30:49,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0f575bc060ce15ecc3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:49,382 - lava_callback - INFO - Received callback for node 69408c00575bc060ce15f3dc INFO: 172.21.0.1:52144 - "POST /node/69408c00575bc060ce15f3dc HTTP/1.0" 202 Accepted 2025-12-17 22:30:50,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a28575bc060ce15ed91 2025-12-17 22:30:50,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a28575bc060ce15ed91 HTTP/1.1" 502 150 2025-12-17 22:30:50,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a28575bc060ce15ed91'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:51,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0c575bc060ce15ecb6 2025-12-17 22:30:51,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0c575bc060ce15ecb6 HTTP/1.1" 502 150 2025-12-17 22:30:51,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0c575bc060ce15ecb6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:52,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0d575bc060ce15ecbe 2025-12-17 22:30:52,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0d575bc060ce15ecbe HTTP/1.1" 502 150 2025-12-17 22:30:52,770 - lava_callback - ERROR - Error processing callback for node 69408a0d575bc060ce15ecbe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a0d575bc060ce15ecbe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a0d575bc060ce15ecbe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a0d575bc060ce15ecbe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:30:52,771 - lava_callback - INFO - Starting async job submit for node 69408a2a575bc060ce15ed94 2025-12-17 22:30:52,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:30:52,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a20575bc060ce15ed7c 2025-12-17 22:30:52,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a20575bc060ce15ed7c HTTP/1.1" 502 150 2025-12-17 22:30:52,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a20575bc060ce15ed7c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:52,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a2a575bc060ce15ed94 HTTP/1.1" 502 150 2025-12-17 22:30:52,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a2a575bc060ce15ed94'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:52,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a2a575bc060ce15ed94 2025-12-17 22:30:52,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a2a575bc060ce15ed94 HTTP/1.1" 502 150 2025-12-17 22:30:52,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a2a575bc060ce15ed94'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:52,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a26575bc060ce15ed8e 2025-12-17 22:30:52,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a26575bc060ce15ed8e HTTP/1.1" 502 150 2025-12-17 22:30:52,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a26575bc060ce15ed8e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:54,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a28575bc060ce15ed91 2025-12-17 22:30:54,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a28575bc060ce15ed91 HTTP/1.1" 502 150 2025-12-17 22:30:54,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a28575bc060ce15ed91'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:54,617 - lava_callback - INFO - Received callback for node 69408c1c575bc060ce15f465 INFO: 172.21.0.1:33498 - "POST /node/69408c1c575bc060ce15f465 HTTP/1.0" 202 Accepted 2025-12-17 22:30:54,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a2a575bc060ce15ed94 2025-12-17 22:30:54,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a2a575bc060ce15ed94 HTTP/1.1" 502 150 2025-12-17 22:30:54,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a2a575bc060ce15ed94'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:54,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a22575bc060ce15ed83 2025-12-17 22:30:54,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a22575bc060ce15ed83 HTTP/1.1" 502 150 2025-12-17 22:30:54,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a22575bc060ce15ed83'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33510 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:30:57,264 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0f575bc060ce15ecc3 2025-12-17 22:30:57,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0f575bc060ce15ecc3 HTTP/1.1" 502 150 2025-12-17 22:30:57,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a0f575bc060ce15ecc3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:30:58,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a2a575bc060ce15ed94 2025-12-17 22:30:58,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a2a575bc060ce15ed94 HTTP/1.1" 502 150 2025-12-17 22:30:58,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a2a575bc060ce15ed94'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:00,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a10575bc060ce15ecc6 2025-12-17 22:31:00,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a10575bc060ce15ecc6 HTTP/1.1" 502 150 2025-12-17 22:31:00,966 - lava_callback - ERROR - Error processing callback for node 69408a10575bc060ce15ecc6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a10575bc060ce15ecc6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a10575bc060ce15ecc6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a10575bc060ce15ecc6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:00,969 - lava_callback - INFO - Starting async job submit for node 69408a2c575bc060ce15ed97 2025-12-17 22:31:00,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:00,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a26575bc060ce15ed8e 2025-12-17 22:31:00,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a26575bc060ce15ed8e HTTP/1.1" 502 150 2025-12-17 22:31:00,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a26575bc060ce15ed8e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:00,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a2c575bc060ce15ed97 HTTP/1.1" 502 150 2025-12-17 22:31:00,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a2c575bc060ce15ed97'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:00,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a2c575bc060ce15ed97 2025-12-17 22:31:00,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a2c575bc060ce15ed97 HTTP/1.1" 502 150 2025-12-17 22:31:00,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a2c575bc060ce15ed97'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:01,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a11575bc060ce15ecc9 2025-12-17 22:31:01,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a14575bc060ce15eccc 2025-12-17 22:31:01,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a11575bc060ce15ecc9 HTTP/1.1" 502 150 2025-12-17 22:31:01,672 - lava_callback - ERROR - Error processing callback for node 69408a11575bc060ce15ecc9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a11575bc060ce15ecc9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a11575bc060ce15ecc9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a11575bc060ce15ecc9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:01,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a14575bc060ce15eccc HTTP/1.1" 502 150 2025-12-17 22:31:01,674 - lava_callback - INFO - Starting async job submit for node 69408a32575bc060ce15ed9a 2025-12-17 22:31:01,674 - lava_callback - ERROR - Error processing callback for node 69408a14575bc060ce15eccc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a14575bc060ce15eccc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a14575bc060ce15eccc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a14575bc060ce15eccc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:01,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:01,677 - lava_callback - INFO - Starting async job submit for node 69408a34575bc060ce15eda3 2025-12-17 22:31:01,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:01,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a32575bc060ce15ed9a HTTP/1.1" 502 150 2025-12-17 22:31:01,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a32575bc060ce15ed9a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:01,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a32575bc060ce15ed9a 2025-12-17 22:31:01,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a32575bc060ce15ed9a HTTP/1.1" 502 150 2025-12-17 22:31:01,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a32575bc060ce15ed9a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:01,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a34575bc060ce15eda3 HTTP/1.1" 502 150 2025-12-17 22:31:01,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a34575bc060ce15eda3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:01,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a34575bc060ce15eda3 2025-12-17 22:31:01,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a34575bc060ce15eda3 HTTP/1.1" 502 150 2025-12-17 22:31:01,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a34575bc060ce15eda3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:02,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a28575bc060ce15ed91 2025-12-17 22:31:02,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a28575bc060ce15ed91 HTTP/1.1" 502 150 2025-12-17 22:31:02,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a28575bc060ce15ed91'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44720 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:31:02,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a2c575bc060ce15ed97 2025-12-17 22:31:02,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a2c575bc060ce15ed97 HTTP/1.1" 502 150 2025-12-17 22:31:02,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a2c575bc060ce15ed97'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:03,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a32575bc060ce15ed9a 2025-12-17 22:31:03,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a32575bc060ce15ed9a HTTP/1.1" 502 150 2025-12-17 22:31:03,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a32575bc060ce15ed9a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:03,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a34575bc060ce15eda3 2025-12-17 22:31:03,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a34575bc060ce15eda3 HTTP/1.1" 502 150 2025-12-17 22:31:03,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a34575bc060ce15eda3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:04,240 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0b575bc060ce15ecb0 2025-12-17 22:31:04,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0b575bc060ce15ecb0 HTTP/1.1" 502 150 2025-12-17 22:31:04,242 - lava_callback - ERROR - Error processing callback for node 69408a0b575bc060ce15ecb0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a0b575bc060ce15ecb0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a0b575bc060ce15ecb0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a0b575bc060ce15ecb0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:04,245 - lava_callback - INFO - Starting async job submit for node 69408a36575bc060ce15edaa 2025-12-17 22:31:04,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:04,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a36575bc060ce15edaa HTTP/1.1" 502 150 2025-12-17 22:31:04,254 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a36575bc060ce15edaa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:04,254 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a36575bc060ce15edaa 2025-12-17 22:31:04,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a36575bc060ce15edaa HTTP/1.1" 502 150 2025-12-17 22:31:04,255 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a36575bc060ce15edaa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:04,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a18575bc060ce15ed6e 2025-12-17 22:31:04,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a18575bc060ce15ed6e HTTP/1.1" 502 150 2025-12-17 22:31:04,576 - lava_callback - ERROR - Error processing callback for node 69408a18575bc060ce15ed6e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a18575bc060ce15ed6e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a18575bc060ce15ed6e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a18575bc060ce15ed6e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:04,578 - lava_callback - INFO - Starting async job submit for node 69408a38575bc060ce15edad 2025-12-17 22:31:04,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:04,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a38575bc060ce15edad HTTP/1.1" 502 150 2025-12-17 22:31:04,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a38575bc060ce15edad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:04,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a38575bc060ce15edad 2025-12-17 22:31:04,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a38575bc060ce15edad HTTP/1.1" 502 150 2025-12-17 22:31:04,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a38575bc060ce15edad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:04,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a1a575bc060ce15ed71 2025-12-17 22:31:04,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1a575bc060ce15ed71 HTTP/1.1" 502 150 2025-12-17 22:31:04,694 - lava_callback - ERROR - Error processing callback for node 69408a1a575bc060ce15ed71: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a1a575bc060ce15ed71 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a1a575bc060ce15ed71 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a1a575bc060ce15ed71 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:04,695 - lava_callback - INFO - Starting async job submit for node 69408a3b575bc060ce15edb0 2025-12-17 22:31:04,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:04,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3b575bc060ce15edb0 HTTP/1.1" 502 150 2025-12-17 22:31:04,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a3b575bc060ce15edb0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:04,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a3b575bc060ce15edb0 2025-12-17 22:31:04,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3b575bc060ce15edb0 HTTP/1.1" 502 150 2025-12-17 22:31:04,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a3b575bc060ce15edb0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:04,851 - lava_callback - INFO - Received callback for node 69408c18575bc060ce15f45f INFO: 172.21.0.1:44726 - "POST /node/69408c18575bc060ce15f45f HTTP/1.0" 202 Accepted 2025-12-17 22:31:04,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a1c575bc060ce15ed74 2025-12-17 22:31:04,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1c575bc060ce15ed74 HTTP/1.1" 502 150 2025-12-17 22:31:04,906 - lava_callback - ERROR - Error processing callback for node 69408a1c575bc060ce15ed74: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a1c575bc060ce15ed74 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a1c575bc060ce15ed74 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a1c575bc060ce15ed74 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:04,907 - lava_callback - INFO - Starting async job submit for node 69408a3d575bc060ce15edb3 2025-12-17 22:31:04,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:04,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3d575bc060ce15edb3 HTTP/1.1" 502 150 2025-12-17 22:31:04,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a3d575bc060ce15edb3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:04,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a3d575bc060ce15edb3 2025-12-17 22:31:04,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3d575bc060ce15edb3 HTTP/1.1" 502 150 2025-12-17 22:31:04,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a3d575bc060ce15edb3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:05,195 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a1e575bc060ce15ed77 2025-12-17 22:31:05,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a1e575bc060ce15ed77 HTTP/1.1" 502 150 2025-12-17 22:31:05,196 - lava_callback - ERROR - Error processing callback for node 69408a1e575bc060ce15ed77: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a1e575bc060ce15ed77 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a1e575bc060ce15ed77 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a1e575bc060ce15ed77 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:05,199 - lava_callback - INFO - Starting async job submit for node 69408a3f575bc060ce15edb6 2025-12-17 22:31:05,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:05,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3f575bc060ce15edb6 HTTP/1.1" 502 150 2025-12-17 22:31:05,210 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a3f575bc060ce15edb6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:05,211 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a3f575bc060ce15edb6 2025-12-17 22:31:05,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3f575bc060ce15edb6 HTTP/1.1" 502 150 2025-12-17 22:31:05,212 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a3f575bc060ce15edb6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:05,504 - lava_callback - INFO - Received callback for node 69408c1a575bc060ce15f462 INFO: 172.21.0.1:44736 - "POST /node/69408c1a575bc060ce15f462 HTTP/1.0" 202 Accepted 2025-12-17 22:31:06,255 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a36575bc060ce15edaa 2025-12-17 22:31:06,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a36575bc060ce15edaa HTTP/1.1" 502 150 2025-12-17 22:31:06,256 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a36575bc060ce15edaa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44742 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:31:06,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a38575bc060ce15edad 2025-12-17 22:31:06,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a38575bc060ce15edad HTTP/1.1" 502 150 2025-12-17 22:31:06,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a38575bc060ce15edad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:06,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a3b575bc060ce15edb0 2025-12-17 22:31:06,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3b575bc060ce15edb0 HTTP/1.1" 502 150 2025-12-17 22:31:06,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a3b575bc060ce15edb0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:06,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a2a575bc060ce15ed94 2025-12-17 22:31:06,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a2a575bc060ce15ed94 HTTP/1.1" 502 150 2025-12-17 22:31:06,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a2a575bc060ce15ed94'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:06,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a3d575bc060ce15edb3 2025-12-17 22:31:06,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3d575bc060ce15edb3 HTTP/1.1" 502 150 2025-12-17 22:31:06,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a3d575bc060ce15edb3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:06,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a2c575bc060ce15ed97 2025-12-17 22:31:06,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a2c575bc060ce15ed97 HTTP/1.1" 502 150 2025-12-17 22:31:06,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a2c575bc060ce15ed97'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:07,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0c575bc060ce15ecb6 2025-12-17 22:31:07,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0c575bc060ce15ecb6 HTTP/1.1" 502 150 2025-12-17 22:31:07,028 - lava_callback - ERROR - Error processing callback for node 69408a0c575bc060ce15ecb6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a0c575bc060ce15ecb6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a0c575bc060ce15ecb6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a0c575bc060ce15ecb6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:07,030 - lava_callback - INFO - Starting async job submit for node 69408a42575bc060ce15edbc 2025-12-17 22:31:07,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:07,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a42575bc060ce15edbc HTTP/1.1" 502 150 2025-12-17 22:31:07,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a42575bc060ce15edbc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:07,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a42575bc060ce15edbc 2025-12-17 22:31:07,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a42575bc060ce15edbc HTTP/1.1" 502 150 2025-12-17 22:31:07,039 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a42575bc060ce15edbc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:07,212 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a3f575bc060ce15edb6 2025-12-17 22:31:07,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3f575bc060ce15edb6 HTTP/1.1" 502 150 2025-12-17 22:31:07,213 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a3f575bc060ce15edb6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:07,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a32575bc060ce15ed9a 2025-12-17 22:31:07,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a32575bc060ce15ed9a HTTP/1.1" 502 150 2025-12-17 22:31:07,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a32575bc060ce15ed9a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:07,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a34575bc060ce15eda3 2025-12-17 22:31:07,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a34575bc060ce15eda3 HTTP/1.1" 502 150 2025-12-17 22:31:07,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a34575bc060ce15eda3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:08,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a20575bc060ce15ed7c 2025-12-17 22:31:08,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a20575bc060ce15ed7c HTTP/1.1" 502 150 2025-12-17 22:31:08,781 - lava_callback - ERROR - Error processing callback for node 69408a20575bc060ce15ed7c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a20575bc060ce15ed7c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a20575bc060ce15ed7c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a20575bc060ce15ed7c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:08,783 - lava_callback - INFO - Starting async job submit for node 69408a44575bc060ce15edbf 2025-12-17 22:31:08,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:08,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a44575bc060ce15edbf HTTP/1.1" 502 150 2025-12-17 22:31:08,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a44575bc060ce15edbf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:08,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a44575bc060ce15edbf 2025-12-17 22:31:08,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a44575bc060ce15edbf HTTP/1.1" 502 150 2025-12-17 22:31:08,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a44575bc060ce15edbf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:09,040 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a42575bc060ce15edbc 2025-12-17 22:31:09,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a42575bc060ce15edbc HTTP/1.1" 502 150 2025-12-17 22:31:09,041 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a42575bc060ce15edbc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:10,256 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a36575bc060ce15edaa 2025-12-17 22:31:10,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a36575bc060ce15edaa HTTP/1.1" 502 150 2025-12-17 22:31:10,257 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a36575bc060ce15edaa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:10,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a38575bc060ce15edad 2025-12-17 22:31:10,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a38575bc060ce15edad HTTP/1.1" 502 150 2025-12-17 22:31:10,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a38575bc060ce15edad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:10,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a3b575bc060ce15edb0 2025-12-17 22:31:10,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3b575bc060ce15edb0 HTTP/1.1" 502 150 2025-12-17 22:31:10,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a3b575bc060ce15edb0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:10,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a44575bc060ce15edbf 2025-12-17 22:31:10,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a44575bc060ce15edbf HTTP/1.1" 502 150 2025-12-17 22:31:10,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a44575bc060ce15edbf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:10,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a3d575bc060ce15edb3 2025-12-17 22:31:10,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3d575bc060ce15edb3 HTTP/1.1" 502 150 2025-12-17 22:31:10,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a3d575bc060ce15edb3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:10,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a22575bc060ce15ed83 2025-12-17 22:31:10,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a22575bc060ce15ed83 HTTP/1.1" 502 150 2025-12-17 22:31:10,950 - lava_callback - ERROR - Error processing callback for node 69408a22575bc060ce15ed83: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a22575bc060ce15ed83 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a22575bc060ce15ed83 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a22575bc060ce15ed83 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:10,953 - lava_callback - INFO - Starting async job submit for node 69408a46575bc060ce15edc2 2025-12-17 22:31:10,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:10,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a46575bc060ce15edc2 HTTP/1.1" 502 150 2025-12-17 22:31:10,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a46575bc060ce15edc2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:10,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a46575bc060ce15edc2 2025-12-17 22:31:10,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a46575bc060ce15edc2 HTTP/1.1" 502 150 2025-12-17 22:31:10,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a46575bc060ce15edc2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:11,213 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a3f575bc060ce15edb6 2025-12-17 22:31:12,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a46575bc060ce15edc2 2025-12-17 22:31:12,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a46575bc060ce15edc2 HTTP/1.1" 502 150 2025-12-17 22:31:12,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a46575bc060ce15edc2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:13,041 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a42575bc060ce15edbc 2025-12-17 22:31:13,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a42575bc060ce15edbc HTTP/1.1" 502 150 2025-12-17 22:31:13,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a42575bc060ce15edbc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:13,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a0f575bc060ce15ecc3 2025-12-17 22:31:13,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a0f575bc060ce15ecc3 HTTP/1.1" 502 150 2025-12-17 22:31:13,267 - lava_callback - ERROR - Error processing callback for node 69408a0f575bc060ce15ecc3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a0f575bc060ce15ecc3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a0f575bc060ce15ecc3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a0f575bc060ce15ecc3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:13,269 - lava_callback - INFO - Starting async job submit for node 69408a48575bc060ce15edcd 2025-12-17 22:31:13,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:13,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a48575bc060ce15edcd HTTP/1.1" 502 150 2025-12-17 22:31:13,278 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a48575bc060ce15edcd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:13,278 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a48575bc060ce15edcd 2025-12-17 22:31:13,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a48575bc060ce15edcd HTTP/1.1" 502 150 2025-12-17 22:31:13,279 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a48575bc060ce15edcd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:13,996 - lava_callback - INFO - Received callback for node 69408c29575bc060ce15f488 INFO: 172.21.0.1:33316 - "POST /node/69408c29575bc060ce15f488 HTTP/1.0" 202 Accepted 2025-12-17 22:31:14,692 - lava_callback - INFO - Received callback for node 69408c1e575bc060ce15f468 INFO: 172.21.0.1:33324 - "POST /node/69408c1e575bc060ce15f468 HTTP/1.0" 202 Accepted 2025-12-17 22:31:14,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a44575bc060ce15edbf 2025-12-17 22:31:14,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a44575bc060ce15edbf HTTP/1.1" 502 150 2025-12-17 22:31:14,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a44575bc060ce15edbf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:14,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a2c575bc060ce15ed97 2025-12-17 22:31:14,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a2c575bc060ce15ed97 HTTP/1.1" 502 150 2025-12-17 22:31:14,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a2c575bc060ce15ed97'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33336 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:31:15,279 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a48575bc060ce15edcd 2025-12-17 22:31:15,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a48575bc060ce15edcd HTTP/1.1" 502 150 2025-12-17 22:31:15,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a48575bc060ce15edcd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:15,373 - lava_callback - INFO - Received callback for node 69408c1f575bc060ce15f46b INFO: 172.21.0.1:33350 - "POST /node/69408c1f575bc060ce15f46b HTTP/1.0" 202 Accepted 2025-12-17 22:31:15,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a25575bc060ce15ed86 HTTP/1.1" 502 150 2025-12-17 22:31:15,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a25575bc060ce15ed86'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:15,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a32575bc060ce15ed9a 2025-12-17 22:31:15,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a34575bc060ce15eda3 2025-12-17 22:31:15,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a32575bc060ce15ed9a HTTP/1.1" 502 150 2025-12-17 22:31:15,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a32575bc060ce15ed9a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:15,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a34575bc060ce15eda3 HTTP/1.1" 502 150 2025-12-17 22:31:15,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a34575bc060ce15eda3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:16,046 - lava_callback - INFO - Received callback for node 69408c20575bc060ce15f46e INFO: 172.21.0.1:33360 - "POST /node/69408c20575bc060ce15f46e HTTP/1.0" 202 Accepted INFO: 172.21.0.1:33368 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:31:16,736 - lava_callback - INFO - Received callback for node 69408c21575bc060ce15f471 INFO: 172.21.0.1:33370 - "POST /node/69408c21575bc060ce15f471 HTTP/1.0" 202 Accepted 2025-12-17 22:31:16,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a46575bc060ce15edc2 2025-12-17 22:31:16,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a46575bc060ce15edc2 HTTP/1.1" 502 150 2025-12-17 22:31:16,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a46575bc060ce15edc2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:16,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a26575bc060ce15ed8e 2025-12-17 22:31:16,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a26575bc060ce15ed8e HTTP/1.1" 502 150 2025-12-17 22:31:16,977 - lava_callback - ERROR - Error processing callback for node 69408a26575bc060ce15ed8e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a26575bc060ce15ed8e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a26575bc060ce15ed8e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a26575bc060ce15ed8e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:16,979 - lava_callback - INFO - Starting async job submit for node 69408a4a575bc060ce15edd2 2025-12-17 22:31:16,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:16,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4a575bc060ce15edd2 HTTP/1.1" 502 150 2025-12-17 22:31:16,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a4a575bc060ce15edd2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:16,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a4a575bc060ce15edd2 2025-12-17 22:31:16,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4a575bc060ce15edd2 HTTP/1.1" 502 150 2025-12-17 22:31:16,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a4a575bc060ce15edd2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:17,413 - lava_callback - INFO - Received callback for node 69408c23575bc060ce15f474 INFO: 172.21.0.1:33380 - "POST /node/69408c23575bc060ce15f474 HTTP/1.0" 202 Accepted 2025-12-17 22:31:18,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a28575bc060ce15ed91 2025-12-17 22:31:18,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a28575bc060ce15ed91 HTTP/1.1" 502 150 2025-12-17 22:31:18,021 - lava_callback - ERROR - Error processing callback for node 69408a28575bc060ce15ed91: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a28575bc060ce15ed91 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a28575bc060ce15ed91 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a28575bc060ce15ed91 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:18,023 - lava_callback - INFO - Starting async job submit for node 69408a4c575bc060ce15edd5 2025-12-17 22:31:18,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:18,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4c575bc060ce15edd5 HTTP/1.1" 502 150 2025-12-17 22:31:18,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a4c575bc060ce15edd5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:18,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a4c575bc060ce15edd5 2025-12-17 22:31:18,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4c575bc060ce15edd5 HTTP/1.1" 502 150 2025-12-17 22:31:18,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a4c575bc060ce15edd5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:18,078 - lava_callback - INFO - Received callback for node 69408c2a575bc060ce15f48f INFO: 172.21.0.1:33386 - "POST /node/69408c2a575bc060ce15f48f HTTP/1.0" 202 Accepted 2025-12-17 22:31:18,258 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a36575bc060ce15edaa 2025-12-17 22:31:18,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a36575bc060ce15edaa HTTP/1.1" 502 150 2025-12-17 22:31:18,259 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a36575bc060ce15edaa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:18,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a38575bc060ce15edad 2025-12-17 22:31:18,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a38575bc060ce15edad HTTP/1.1" 502 150 2025-12-17 22:31:18,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a38575bc060ce15edad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:18,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a3b575bc060ce15edb0 2025-12-17 22:31:18,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3b575bc060ce15edb0 HTTP/1.1" 502 150 2025-12-17 22:31:18,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a3b575bc060ce15edb0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:18,720 - lava_callback - INFO - Received callback for node 69408c2b575bc060ce15f496 INFO: 172.21.0.1:33398 - "POST /node/69408c2b575bc060ce15f496 HTTP/1.0" 202 Accepted 2025-12-17 22:31:18,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a3d575bc060ce15edb3 2025-12-17 22:31:18,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3d575bc060ce15edb3 HTTP/1.1" 502 150 2025-12-17 22:31:18,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a3d575bc060ce15edb3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:18,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a4a575bc060ce15edd2 2025-12-17 22:31:18,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4a575bc060ce15edd2 HTTP/1.1" 502 150 2025-12-17 22:31:18,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a4a575bc060ce15edd2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:19,280 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a48575bc060ce15edcd 2025-12-17 22:31:19,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a48575bc060ce15edcd HTTP/1.1" 502 150 2025-12-17 22:31:19,283 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a48575bc060ce15edcd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:19,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a25575bc060ce15ed86 2025-12-17 22:31:19,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a25575bc060ce15ed86 HTTP/1.1" 502 150 2025-12-17 22:31:19,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a25575bc060ce15ed86'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:19,382 - lava_callback - INFO - Received callback for node 69408c2d575bc060ce15f49d INFO: 172.21.0.1:33404 - "POST /node/69408c2d575bc060ce15f49d HTTP/1.0" 202 Accepted 2025-12-17 22:31:20,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a4c575bc060ce15edd5 2025-12-17 22:31:20,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4c575bc060ce15edd5 HTTP/1.1" 502 150 2025-12-17 22:31:20,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a4c575bc060ce15edd5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:21,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a42575bc060ce15edbc 2025-12-17 22:31:21,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a42575bc060ce15edbc HTTP/1.1" 502 150 2025-12-17 22:31:21,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a42575bc060ce15edbc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:22,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a44575bc060ce15edbf 2025-12-17 22:31:22,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a44575bc060ce15edbf HTTP/1.1" 502 150 2025-12-17 22:31:22,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a44575bc060ce15edbf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:22,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a2a575bc060ce15ed94 2025-12-17 22:31:22,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a2a575bc060ce15ed94 HTTP/1.1" 502 150 2025-12-17 22:31:22,806 - lava_callback - ERROR - Error processing callback for node 69408a2a575bc060ce15ed94: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a2a575bc060ce15ed94 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a2a575bc060ce15ed94 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a2a575bc060ce15ed94 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:22,808 - lava_callback - INFO - Starting async job submit for node 69408a4e575bc060ce15edd8 2025-12-17 22:31:22,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:22,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4e575bc060ce15edd8 HTTP/1.1" 502 150 2025-12-17 22:31:22,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a4e575bc060ce15edd8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:22,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a4e575bc060ce15edd8 2025-12-17 22:31:22,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4e575bc060ce15edd8 HTTP/1.1" 502 150 2025-12-17 22:31:22,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a4e575bc060ce15edd8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:22,852 - lava_callback - INFO - Received callback for node 69408c37575bc060ce15f4b2 INFO: 172.21.0.1:55892 - "POST /node/69408c37575bc060ce15f4b2 HTTP/1.0" 202 Accepted 2025-12-17 22:31:22,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a4a575bc060ce15edd2 2025-12-17 22:31:22,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4a575bc060ce15edd2 HTTP/1.1" 502 150 2025-12-17 22:31:22,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a4a575bc060ce15edd2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:23,630 - lava_callback - INFO - Received callback for node 69408c38575bc060ce15f4b5 INFO: 172.21.0.1:55896 - "POST /node/69408c38575bc060ce15f4b5 HTTP/1.0" 202 Accepted 2025-12-17 22:31:24,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a4c575bc060ce15edd5 2025-12-17 22:31:24,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4c575bc060ce15edd5 HTTP/1.1" 502 150 2025-12-17 22:31:24,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a4c575bc060ce15edd5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:24,377 - lava_callback - INFO - Received callback for node 69408c3a575bc060ce15f4b8 INFO: 172.21.0.1:55912 - "POST /node/69408c3a575bc060ce15f4b8 HTTP/1.0" 202 Accepted 2025-12-17 22:31:24,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a4e575bc060ce15edd8 2025-12-17 22:31:24,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4e575bc060ce15edd8 HTTP/1.1" 502 150 2025-12-17 22:31:24,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a4e575bc060ce15edd8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:24,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a46575bc060ce15edc2 2025-12-17 22:31:24,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a46575bc060ce15edc2 HTTP/1.1" 502 150 2025-12-17 22:31:24,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a46575bc060ce15edc2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:25,105 - lava_callback - INFO - Received callback for node 69408c3b575bc060ce15f4bb INFO: 172.21.0.1:55914 - "POST /node/69408c3b575bc060ce15f4bb HTTP/1.0" 202 Accepted 2025-12-17 22:31:25,887 - lava_callback - INFO - Received callback for node 69408c3c575bc060ce15f4c4 INFO: 172.21.0.1:55928 - "POST /node/69408c3c575bc060ce15f4c4 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:55936 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:31:26,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3f575bc060ce15edb6 HTTP/1.1" 502 150 2025-12-17 22:31:26,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a3f575bc060ce15edb6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:27,283 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a48575bc060ce15edcd 2025-12-17 22:31:27,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a48575bc060ce15edcd HTTP/1.1" 502 150 2025-12-17 22:31:27,285 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a48575bc060ce15edcd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:27,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a25575bc060ce15ed86 2025-12-17 22:31:27,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a25575bc060ce15ed86 HTTP/1.1" 502 150 2025-12-17 22:31:27,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a25575bc060ce15ed86'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:28,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a4e575bc060ce15edd8 2025-12-17 22:31:28,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4e575bc060ce15edd8 HTTP/1.1" 502 150 2025-12-17 22:31:28,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a4e575bc060ce15edd8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:30,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a2c575bc060ce15ed97 2025-12-17 22:31:30,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a2c575bc060ce15ed97 HTTP/1.1" 502 150 2025-12-17 22:31:30,986 - lava_callback - ERROR - Error processing callback for node 69408a2c575bc060ce15ed97: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a2c575bc060ce15ed97 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a2c575bc060ce15ed97 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a2c575bc060ce15ed97 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:30,988 - lava_callback - INFO - Starting async job submit for node 69408a50575bc060ce15eddb 2025-12-17 22:31:30,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:30,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a4a575bc060ce15edd2 2025-12-17 22:31:30,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4a575bc060ce15edd2 HTTP/1.1" 502 150 2025-12-17 22:31:30,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a4a575bc060ce15edd2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:31,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a50575bc060ce15eddb HTTP/1.1" 502 150 2025-12-17 22:31:31,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a50575bc060ce15eddb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:31,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a50575bc060ce15eddb 2025-12-17 22:31:31,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a50575bc060ce15eddb HTTP/1.1" 502 150 2025-12-17 22:31:31,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a50575bc060ce15eddb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:31,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a32575bc060ce15ed9a 2025-12-17 22:31:31,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a34575bc060ce15eda3 2025-12-17 22:31:31,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a32575bc060ce15ed9a HTTP/1.1" 502 150 2025-12-17 22:31:31,697 - lava_callback - ERROR - Error processing callback for node 69408a32575bc060ce15ed9a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a32575bc060ce15ed9a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a32575bc060ce15ed9a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a32575bc060ce15ed9a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:31,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a34575bc060ce15eda3 HTTP/1.1" 502 150 2025-12-17 22:31:31,700 - lava_callback - ERROR - Error processing callback for node 69408a34575bc060ce15eda3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a34575bc060ce15eda3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a34575bc060ce15eda3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a34575bc060ce15eda3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:31,702 - lava_callback - INFO - Starting async job submit for node 69408a52575bc060ce15ede2 2025-12-17 22:31:31,703 - lava_callback - INFO - Starting async job submit for node 69408a55575bc060ce15ee1f 2025-12-17 22:31:31,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:31,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:31,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a52575bc060ce15ede2 HTTP/1.1" 502 150 2025-12-17 22:31:31,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a52575bc060ce15ede2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:31,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a52575bc060ce15ede2 2025-12-17 22:31:31,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a52575bc060ce15ede2 HTTP/1.1" 502 150 2025-12-17 22:31:31,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a52575bc060ce15ede2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:32,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a4c575bc060ce15edd5 2025-12-17 22:31:32,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4c575bc060ce15edd5 HTTP/1.1" 502 150 2025-12-17 22:31:32,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a4c575bc060ce15edd5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:33,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a50575bc060ce15eddb 2025-12-17 22:31:33,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a50575bc060ce15eddb HTTP/1.1" 502 150 2025-12-17 22:31:33,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a50575bc060ce15eddb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:33,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a52575bc060ce15ede2 2025-12-17 22:31:33,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a52575bc060ce15ede2 HTTP/1.1" 502 150 2025-12-17 22:31:33,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a52575bc060ce15ede2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:34,259 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a36575bc060ce15edaa 2025-12-17 22:31:34,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a36575bc060ce15edaa HTTP/1.1" 502 150 2025-12-17 22:31:34,261 - lava_callback - ERROR - Error processing callback for node 69408a36575bc060ce15edaa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a36575bc060ce15edaa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a36575bc060ce15edaa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a36575bc060ce15edaa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:34,263 - lava_callback - INFO - Starting async job submit for node 69408a57575bc060ce15ee22 2025-12-17 22:31:34,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:34,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a57575bc060ce15ee22 HTTP/1.1" 502 150 2025-12-17 22:31:34,269 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a57575bc060ce15ee22'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:34,269 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a57575bc060ce15ee22 2025-12-17 22:31:34,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a57575bc060ce15ee22 HTTP/1.1" 502 150 2025-12-17 22:31:34,270 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a57575bc060ce15ee22'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:34,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a38575bc060ce15edad 2025-12-17 22:31:34,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a38575bc060ce15edad HTTP/1.1" 502 150 2025-12-17 22:31:34,611 - lava_callback - ERROR - Error processing callback for node 69408a38575bc060ce15edad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a38575bc060ce15edad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a38575bc060ce15edad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a38575bc060ce15edad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:34,613 - lava_callback - INFO - Starting async job submit for node 69408a5a575bc060ce15ee25 2025-12-17 22:31:34,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:34,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a5a575bc060ce15ee25 HTTP/1.1" 502 150 2025-12-17 22:31:34,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a5a575bc060ce15ee25'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:34,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a5a575bc060ce15ee25 2025-12-17 22:31:34,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a5a575bc060ce15ee25 HTTP/1.1" 502 150 2025-12-17 22:31:34,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a5a575bc060ce15ee25'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:34,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a3f575bc060ce15edb6 2025-12-17 22:31:34,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3f575bc060ce15edb6 HTTP/1.1" 502 150 2025-12-17 22:31:34,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a3f575bc060ce15edb6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:34,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a3b575bc060ce15edb0 2025-12-17 22:31:34,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3b575bc060ce15edb0 HTTP/1.1" 502 150 2025-12-17 22:31:34,721 - lava_callback - ERROR - Error processing callback for node 69408a3b575bc060ce15edb0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a3b575bc060ce15edb0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a3b575bc060ce15edb0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a3b575bc060ce15edb0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:34,723 - lava_callback - INFO - Starting async job submit for node 6940945d575bc060ce161000 2025-12-17 22:31:34,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:34,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940945d575bc060ce161000 HTTP/1.1" 502 150 2025-12-17 22:31:34,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940945d575bc060ce161000'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:34,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940945d575bc060ce161000 2025-12-17 22:31:34,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940945d575bc060ce161000 HTTP/1.1" 502 150 2025-12-17 22:31:34,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940945d575bc060ce161000'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:34,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a3d575bc060ce15edb3 2025-12-17 22:31:34,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3d575bc060ce15edb3 HTTP/1.1" 502 150 2025-12-17 22:31:34,922 - lava_callback - ERROR - Error processing callback for node 69408a3d575bc060ce15edb3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a3d575bc060ce15edb3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a3d575bc060ce15edb3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a3d575bc060ce15edb3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:34,924 - lava_callback - INFO - Starting async job submit for node 6940937b575bc060ce160d99 2025-12-17 22:31:34,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:34,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937b575bc060ce160d99 HTTP/1.1" 502 150 2025-12-17 22:31:34,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937b575bc060ce160d99'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:34,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937b575bc060ce160d99 2025-12-17 22:31:34,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937b575bc060ce160d99 HTTP/1.1" 502 150 2025-12-17 22:31:34,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937b575bc060ce160d99'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:36,270 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a57575bc060ce15ee22 2025-12-17 22:31:36,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a57575bc060ce15ee22 HTTP/1.1" 502 150 2025-12-17 22:31:36,271 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a57575bc060ce15ee22'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42168 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:31:36,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a5a575bc060ce15ee25 2025-12-17 22:31:36,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a5a575bc060ce15ee25 HTTP/1.1" 502 150 2025-12-17 22:31:36,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a5a575bc060ce15ee25'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:36,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940945d575bc060ce161000 2025-12-17 22:31:36,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940945d575bc060ce161000 HTTP/1.1" 502 150 2025-12-17 22:31:36,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940945d575bc060ce161000'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:36,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a4e575bc060ce15edd8 2025-12-17 22:31:36,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4e575bc060ce15edd8 HTTP/1.1" 502 150 2025-12-17 22:31:36,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a4e575bc060ce15edd8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:36,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937b575bc060ce160d99 2025-12-17 22:31:36,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937b575bc060ce160d99 HTTP/1.1" 502 150 2025-12-17 22:31:36,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937b575bc060ce160d99'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:37,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a50575bc060ce15eddb 2025-12-17 22:31:37,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a42575bc060ce15edbc 2025-12-17 22:31:37,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a52575bc060ce15ede2 2025-12-17 22:31:37,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a52575bc060ce15ede2 HTTP/1.1" 502 150 2025-12-17 22:31:37,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a52575bc060ce15ede2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:38,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a44575bc060ce15edbf 2025-12-17 22:31:38,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a44575bc060ce15edbf HTTP/1.1" 502 150 2025-12-17 22:31:38,797 - lava_callback - ERROR - Error processing callback for node 69408a44575bc060ce15edbf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a44575bc060ce15edbf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a44575bc060ce15edbf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a44575bc060ce15edbf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:38,799 - lava_callback - INFO - Starting async job submit for node 69408a5f575bc060ce15ee39 2025-12-17 22:31:38,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:38,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a5f575bc060ce15ee39 HTTP/1.1" 502 150 2025-12-17 22:31:38,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a5f575bc060ce15ee39'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:38,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a5f575bc060ce15ee39 2025-12-17 22:31:38,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a5f575bc060ce15ee39 HTTP/1.1" 502 150 2025-12-17 22:31:38,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a5f575bc060ce15ee39'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:40,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a57575bc060ce15ee22 2025-12-17 22:31:40,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a57575bc060ce15ee22 HTTP/1.1" 502 150 2025-12-17 22:31:40,273 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a57575bc060ce15ee22'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:40,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a5a575bc060ce15ee25 2025-12-17 22:31:40,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a5a575bc060ce15ee25 HTTP/1.1" 502 150 2025-12-17 22:31:40,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a5a575bc060ce15ee25'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:40,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940945d575bc060ce161000 2025-12-17 22:31:40,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940945d575bc060ce161000 HTTP/1.1" 502 150 2025-12-17 22:31:40,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940945d575bc060ce161000'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:40,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a5f575bc060ce15ee39 2025-12-17 22:31:40,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a5f575bc060ce15ee39 HTTP/1.1" 502 150 2025-12-17 22:31:40,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a5f575bc060ce15ee39'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:40,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a46575bc060ce15edc2 2025-12-17 22:31:40,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a46575bc060ce15edc2 HTTP/1.1" 502 150 2025-12-17 22:31:40,966 - lava_callback - ERROR - Error processing callback for node 69408a46575bc060ce15edc2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a46575bc060ce15edc2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a46575bc060ce15edc2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a46575bc060ce15edc2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:40,968 - lava_callback - INFO - Starting async job submit for node 69408a61575bc060ce15ee41 2025-12-17 22:31:40,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:40,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a61575bc060ce15ee41 HTTP/1.1" 502 150 2025-12-17 22:31:40,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a61575bc060ce15ee41'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:40,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a61575bc060ce15ee41 2025-12-17 22:31:40,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a61575bc060ce15ee41 HTTP/1.1" 502 150 2025-12-17 22:31:40,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a61575bc060ce15ee41'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:40,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937b575bc060ce160d99 2025-12-17 22:31:40,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937b575bc060ce160d99 HTTP/1.1" 502 150 2025-12-17 22:31:40,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937b575bc060ce160d99'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:42,869 - lava_callback - INFO - Received callback for node 69408c3e575bc060ce15f4d0 INFO: 172.21.0.1:41146 - "POST /node/69408c3e575bc060ce15f4d0 HTTP/1.0" 202 Accepted 2025-12-17 22:31:42,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a61575bc060ce15ee41 2025-12-17 22:31:42,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a61575bc060ce15ee41 HTTP/1.1" 502 150 2025-12-17 22:31:42,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a61575bc060ce15ee41'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:43,285 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a48575bc060ce15edcd 2025-12-17 22:31:43,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a48575bc060ce15edcd HTTP/1.1" 502 150 2025-12-17 22:31:43,286 - lava_callback - ERROR - Error processing callback for node 69408a48575bc060ce15edcd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a48575bc060ce15edcd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a48575bc060ce15edcd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a48575bc060ce15edcd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:43,288 - lava_callback - INFO - Starting async job submit for node 69408a63575bc060ce15ee44 2025-12-17 22:31:43,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:43,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a63575bc060ce15ee44 HTTP/1.1" 502 150 2025-12-17 22:31:43,296 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a63575bc060ce15ee44'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:43,296 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a63575bc060ce15ee44 2025-12-17 22:31:43,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a63575bc060ce15ee44 HTTP/1.1" 502 150 2025-12-17 22:31:43,297 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a63575bc060ce15ee44'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:43,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a25575bc060ce15ed86 2025-12-17 22:31:43,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a25575bc060ce15ed86 HTTP/1.1" 502 150 2025-12-17 22:31:43,384 - lava_callback - ERROR - Error processing callback for node 69408a25575bc060ce15ed86: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a25575bc060ce15ed86 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a25575bc060ce15ed86 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a25575bc060ce15ed86 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:43,386 - lava_callback - INFO - Starting async job submit for node 6940e125460bc8e26511373c 2025-12-17 22:31:43,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:43,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940e125460bc8e26511373c HTTP/1.1" 502 150 2025-12-17 22:31:43,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940e125460bc8e26511373c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:43,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940e125460bc8e26511373c 2025-12-17 22:31:43,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940e125460bc8e26511373c HTTP/1.1" 502 150 2025-12-17 22:31:43,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940e125460bc8e26511373c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:43,673 - lava_callback - INFO - Received callback for node 69408c3f575bc060ce15f4da INFO: 172.21.0.1:41158 - "POST /node/69408c3f575bc060ce15f4da HTTP/1.0" 202 Accepted 2025-12-17 22:31:44,138 - lava_callback - INFO - Received callback for node 694080d6575bc060ce15c906 INFO: 172.21.0.1:41162 - "POST /node/694080d6575bc060ce15c906 HTTP/1.0" 202 Accepted 2025-12-17 22:31:44,554 - lava_callback - INFO - Received callback for node 69408c45575bc060ce15f522 INFO: 172.21.0.1:41176 - "POST /node/69408c45575bc060ce15f522 HTTP/1.0" 202 Accepted 2025-12-17 22:31:44,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a5f575bc060ce15ee39 2025-12-17 22:31:44,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a5f575bc060ce15ee39 HTTP/1.1" 502 150 2025-12-17 22:31:44,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a5f575bc060ce15ee39'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:45,290 - lava_callback - INFO - Received callback for node 69408c46575bc060ce15f527 INFO: 172.21.0.1:41190 - "POST /node/69408c46575bc060ce15f527 HTTP/1.0" 202 Accepted 2025-12-17 22:31:45,298 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a63575bc060ce15ee44 2025-12-17 22:31:45,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a63575bc060ce15ee44 HTTP/1.1" 502 150 2025-12-17 22:31:45,299 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a63575bc060ce15ee44'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:45,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940e125460bc8e26511373c 2025-12-17 22:31:45,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940e125460bc8e26511373c HTTP/1.1" 502 150 2025-12-17 22:31:45,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940e125460bc8e26511373c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:45,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a52575bc060ce15ede2 2025-12-17 22:31:45,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a52575bc060ce15ede2 HTTP/1.1" 502 150 2025-12-17 22:31:45,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a52575bc060ce15ede2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:46,035 - lava_callback - INFO - Received callback for node 69408c2e575bc060ce15f4a0 INFO: 172.21.0.1:41200 - "POST /node/69408c2e575bc060ce15f4a0 HTTP/1.0" 202 Accepted 2025-12-17 22:31:46,103 - lava_callback - INFO - Received callback for node 69408c47575bc060ce15f52a INFO: 172.21.0.1:41206 - "POST /node/69408c47575bc060ce15f52a HTTP/1.0" 202 Accepted INFO: 172.21.0.1:41220 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:31:46,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a61575bc060ce15ee41 2025-12-17 22:31:46,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a61575bc060ce15ee41 HTTP/1.1" 502 150 2025-12-17 22:31:46,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a61575bc060ce15ee41'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:46,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a4a575bc060ce15edd2 2025-12-17 22:31:46,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4a575bc060ce15edd2 HTTP/1.1" 502 150 2025-12-17 22:31:46,994 - lava_callback - ERROR - Error processing callback for node 69408a4a575bc060ce15edd2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a4a575bc060ce15edd2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a4a575bc060ce15edd2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a4a575bc060ce15edd2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:46,996 - lava_callback - INFO - Starting async job submit for node 69408a67575bc060ce15ee4e 2025-12-17 22:31:46,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:47,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a67575bc060ce15ee4e HTTP/1.1" 502 150 2025-12-17 22:31:47,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a67575bc060ce15ee4e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:47,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a67575bc060ce15ee4e 2025-12-17 22:31:47,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a67575bc060ce15ee4e HTTP/1.1" 502 150 2025-12-17 22:31:47,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a67575bc060ce15ee4e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:47,222 - lava_callback - INFO - Received callback for node 69408c48575bc060ce15f52d INFO: 172.21.0.1:41232 - "POST /node/69408c48575bc060ce15f52d HTTP/1.0" 202 Accepted 2025-12-17 22:31:47,781 - lava_callback - INFO - Received callback for node 69408c30575bc060ce15f4a3 INFO: 172.21.0.1:41246 - "POST /node/69408c30575bc060ce15f4a3 HTTP/1.0" 202 Accepted 2025-12-17 22:31:48,013 - lava_callback - INFO - Received callback for node 69408c4a575bc060ce15f530 INFO: 172.21.0.1:41258 - "POST /node/69408c4a575bc060ce15f530 HTTP/1.0" 202 Accepted 2025-12-17 22:31:48,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a4c575bc060ce15edd5 2025-12-17 22:31:48,273 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a57575bc060ce15ee22 2025-12-17 22:31:48,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a57575bc060ce15ee22 HTTP/1.1" 502 150 2025-12-17 22:31:48,275 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a57575bc060ce15ee22'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:48,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a5a575bc060ce15ee25 2025-12-17 22:31:48,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a5a575bc060ce15ee25 HTTP/1.1" 502 150 2025-12-17 22:31:48,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a5a575bc060ce15ee25'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:48,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940945d575bc060ce161000 2025-12-17 22:31:48,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940945d575bc060ce161000 HTTP/1.1" 502 150 2025-12-17 22:31:48,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940945d575bc060ce161000'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:48,824 - lava_callback - INFO - Received callback for node 69408c4b575bc060ce15f533 INFO: 172.21.0.1:41268 - "POST /node/69408c4b575bc060ce15f533 HTTP/1.0" 202 Accepted 2025-12-17 22:31:48,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937b575bc060ce160d99 2025-12-17 22:31:48,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937b575bc060ce160d99 HTTP/1.1" 502 150 2025-12-17 22:31:48,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937b575bc060ce160d99'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:49,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a67575bc060ce15ee4e 2025-12-17 22:31:49,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a67575bc060ce15ee4e HTTP/1.1" 502 150 2025-12-17 22:31:49,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a67575bc060ce15ee4e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:49,299 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a63575bc060ce15ee44 2025-12-17 22:31:49,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a63575bc060ce15ee44 HTTP/1.1" 502 150 2025-12-17 22:31:49,300 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a63575bc060ce15ee44'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:49,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940e125460bc8e26511373c 2025-12-17 22:31:49,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940e125460bc8e26511373c HTTP/1.1" 502 150 2025-12-17 22:31:49,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940e125460bc8e26511373c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:49,555 - lava_callback - INFO - Received callback for node 69408c4c575bc060ce15f536 INFO: 172.21.0.1:41274 - "POST /node/69408c4c575bc060ce15f536 HTTP/1.0" 202 Accepted 2025-12-17 22:31:50,291 - lava_callback - INFO - Received callback for node 69408c4d575bc060ce15f539 INFO: 172.21.0.1:52866 - "POST /node/69408c4d575bc060ce15f539 HTTP/1.0" 202 Accepted 2025-12-17 22:31:50,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a3f575bc060ce15edb6 2025-12-17 22:31:50,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a3f575bc060ce15edb6 HTTP/1.1" 502 150 2025-12-17 22:31:50,643 - lava_callback - ERROR - Error processing callback for node 69408a3f575bc060ce15edb6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a3f575bc060ce15edb6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a3f575bc060ce15edb6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a3f575bc060ce15edb6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:50,645 - lava_callback - INFO - Starting async job submit for node 69408a6a575bc060ce15ee51 2025-12-17 22:31:50,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:50,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6a575bc060ce15ee51 HTTP/1.1" 502 150 2025-12-17 22:31:50,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a6a575bc060ce15ee51'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:50,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a6a575bc060ce15ee51 2025-12-17 22:31:50,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6a575bc060ce15ee51 HTTP/1.1" 502 150 2025-12-17 22:31:50,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a6a575bc060ce15ee51'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:51,053 - lava_callback - INFO - Received callback for node 69408c4f575bc060ce15f53c INFO: 172.21.0.1:52876 - "POST /node/69408c4f575bc060ce15f53c HTTP/1.0" 202 Accepted 2025-12-17 22:31:51,806 - lava_callback - INFO - Received callback for node 69408c50575bc060ce15f547 INFO: 172.21.0.1:52880 - "POST /node/69408c50575bc060ce15f547 HTTP/1.0" 202 Accepted 2025-12-17 22:31:52,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a50575bc060ce15eddb HTTP/1.1" 502 150 2025-12-17 22:31:52,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a50575bc060ce15eddb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:52,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a42575bc060ce15edbc HTTP/1.1" 502 150 2025-12-17 22:31:52,497 - lava_callback - ERROR - Error processing callback for node 69408a42575bc060ce15edbc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a42575bc060ce15edbc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a42575bc060ce15edbc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a42575bc060ce15edbc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:52,498 - lava_callback - INFO - Starting async job submit for node 69408a6c575bc060ce15ee54 2025-12-17 22:31:52,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:52,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6c575bc060ce15ee54 HTTP/1.1" 502 150 2025-12-17 22:31:52,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a6c575bc060ce15ee54'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:52,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a6c575bc060ce15ee54 2025-12-17 22:31:52,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6c575bc060ce15ee54 HTTP/1.1" 502 150 2025-12-17 22:31:52,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a6c575bc060ce15ee54'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:52,538 - lava_callback - INFO - Received callback for node 69408c52575bc060ce15f550 INFO: 172.21.0.1:52890 - "POST /node/69408c52575bc060ce15f550 HTTP/1.0" 202 Accepted 2025-12-17 22:31:52,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a6a575bc060ce15ee51 2025-12-17 22:31:52,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6a575bc060ce15ee51 HTTP/1.1" 502 150 2025-12-17 22:31:52,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a6a575bc060ce15ee51'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:52,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a5f575bc060ce15ee39 2025-12-17 22:31:52,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a5f575bc060ce15ee39 HTTP/1.1" 502 150 2025-12-17 22:31:52,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a5f575bc060ce15ee39'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:52,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a4e575bc060ce15edd8 2025-12-17 22:31:52,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4e575bc060ce15edd8 HTTP/1.1" 502 150 2025-12-17 22:31:52,824 - lava_callback - ERROR - Error processing callback for node 69408a4e575bc060ce15edd8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a4e575bc060ce15edd8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a4e575bc060ce15edd8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a4e575bc060ce15edd8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:31:52,826 - lava_callback - INFO - Starting async job submit for node 69408a6e575bc060ce15ee57 2025-12-17 22:31:52,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:31:52,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6e575bc060ce15ee57 HTTP/1.1" 502 150 2025-12-17 22:31:52,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a6e575bc060ce15ee57'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:52,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a6e575bc060ce15ee57 2025-12-17 22:31:52,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6e575bc060ce15ee57 HTTP/1.1" 502 150 2025-12-17 22:31:52,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a6e575bc060ce15ee57'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:53,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a67575bc060ce15ee4e 2025-12-17 22:31:53,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a67575bc060ce15ee4e HTTP/1.1" 502 150 2025-12-17 22:31:53,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a67575bc060ce15ee4e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:53,296 - lava_callback - INFO - Received callback for node 69408c53575bc060ce15f557 INFO: 172.21.0.1:52904 - "POST /node/69408c53575bc060ce15f557 HTTP/1.0" 202 Accepted 2025-12-17 22:31:54,046 - lava_callback - INFO - Received callback for node 69408c54575bc060ce15f55c INFO: 172.21.0.1:52910 - "POST /node/69408c54575bc060ce15f55c HTTP/1.0" 202 Accepted 2025-12-17 22:31:54,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a6c575bc060ce15ee54 2025-12-17 22:31:54,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6c575bc060ce15ee54 HTTP/1.1" 502 150 2025-12-17 22:31:54,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a6c575bc060ce15ee54'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:54,806 - lava_callback - INFO - Received callback for node 69408c56575bc060ce15f55f INFO: 172.21.0.1:52912 - "POST /node/69408c56575bc060ce15f55f HTTP/1.0" 202 Accepted 2025-12-17 22:31:54,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a6e575bc060ce15ee57 2025-12-17 22:31:54,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6e575bc060ce15ee57 HTTP/1.1" 502 150 2025-12-17 22:31:54,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a6e575bc060ce15ee57'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:54,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a61575bc060ce15ee41 2025-12-17 22:31:54,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a61575bc060ce15ee41 HTTP/1.1" 502 150 2025-12-17 22:31:54,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a61575bc060ce15ee41'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:55,539 - lava_callback - INFO - Received callback for node 69408c57575bc060ce15f562 INFO: 172.21.0.1:52924 - "POST /node/69408c57575bc060ce15f562 HTTP/1.0" 202 Accepted 2025-12-17 22:31:56,290 - lava_callback - INFO - Received callback for node 69408c58575bc060ce15f565 INFO: 172.21.0.1:52926 - "POST /node/69408c58575bc060ce15f565 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:52940 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:31:56,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a6a575bc060ce15ee51 2025-12-17 22:31:56,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6a575bc060ce15ee51 HTTP/1.1" 502 150 2025-12-17 22:31:56,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a6a575bc060ce15ee51'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:57,030 - lava_callback - INFO - Received callback for node 69408c5a575bc060ce15f568 INFO: 172.21.0.1:52942 - "POST /node/69408c5a575bc060ce15f568 HTTP/1.0" 202 Accepted 2025-12-17 22:31:57,301 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a63575bc060ce15ee44 2025-12-17 22:31:57,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a63575bc060ce15ee44 HTTP/1.1" 502 150 2025-12-17 22:31:57,302 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a63575bc060ce15ee44'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:57,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940e125460bc8e26511373c 2025-12-17 22:31:57,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940e125460bc8e26511373c HTTP/1.1" 502 150 2025-12-17 22:31:57,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940e125460bc8e26511373c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:57,773 - lava_callback - INFO - Received callback for node 69408c5b575bc060ce15f56b INFO: 172.21.0.1:52946 - "POST /node/69408c5b575bc060ce15f56b HTTP/1.0" 202 Accepted 2025-12-17 22:31:58,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a6c575bc060ce15ee54 2025-12-17 22:31:58,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6c575bc060ce15ee54 HTTP/1.1" 502 150 2025-12-17 22:31:58,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a6c575bc060ce15ee54'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:58,525 - lava_callback - INFO - Received callback for node 69408c5c575bc060ce15f56e INFO: 172.21.0.1:52952 - "POST /node/69408c5c575bc060ce15f56e HTTP/1.0" 202 Accepted 2025-12-17 22:31:58,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a6e575bc060ce15ee57 2025-12-17 22:31:58,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6e575bc060ce15ee57 HTTP/1.1" 502 150 2025-12-17 22:31:58,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a6e575bc060ce15ee57'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:31:59,296 - lava_callback - INFO - Received callback for node 69408c5d575bc060ce15f571 INFO: 172.21.0.1:52966 - "POST /node/69408c5d575bc060ce15f571 HTTP/1.0" 202 Accepted 2025-12-17 22:32:00,043 - lava_callback - INFO - Received callback for node 69408c5f575bc060ce15f574 INFO: 172.21.0.1:46798 - "POST /node/69408c5f575bc060ce15f574 HTTP/1.0" 202 Accepted 2025-12-17 22:32:00,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a50575bc060ce15eddb 2025-12-17 22:32:00,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a50575bc060ce15eddb HTTP/1.1" 502 150 2025-12-17 22:32:00,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a50575bc060ce15eddb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:01,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a67575bc060ce15ee4e 2025-12-17 22:32:01,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a67575bc060ce15ee4e HTTP/1.1" 502 150 2025-12-17 22:32:01,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a67575bc060ce15ee4e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:01,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a52575bc060ce15ede2 2025-12-17 22:32:01,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a52575bc060ce15ede2 HTTP/1.1" 502 150 2025-12-17 22:32:01,742 - lava_callback - ERROR - Error processing callback for node 69408a52575bc060ce15ede2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a52575bc060ce15ede2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a52575bc060ce15ede2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a52575bc060ce15ede2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:01,744 - lava_callback - INFO - Starting async job submit for node 69408a70575bc060ce15ee60 2025-12-17 22:32:01,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:01,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a70575bc060ce15ee60 HTTP/1.1" 502 150 2025-12-17 22:32:01,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a70575bc060ce15ee60'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:01,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a70575bc060ce15ee60 2025-12-17 22:32:01,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a70575bc060ce15ee60 HTTP/1.1" 502 150 2025-12-17 22:32:01,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a70575bc060ce15ee60'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46812 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:32:02,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a55575bc060ce15ee1f HTTP/1.1" 502 150 2025-12-17 22:32:02,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a55575bc060ce15ee1f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:02,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a55575bc060ce15ee1f 2025-12-17 22:32:03,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a4c575bc060ce15edd5 HTTP/1.1" 502 150 2025-12-17 22:32:03,504 - lava_callback - ERROR - Error processing callback for node 69408a4c575bc060ce15edd5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a4c575bc060ce15edd5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a4c575bc060ce15edd5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a4c575bc060ce15edd5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:03,506 - lava_callback - INFO - Starting async job submit for node 69408a72575bc060ce15ee69 2025-12-17 22:32:03,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:03,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a72575bc060ce15ee69 HTTP/1.1" 502 150 2025-12-17 22:32:03,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a72575bc060ce15ee69'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:03,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a72575bc060ce15ee69 2025-12-17 22:32:03,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a72575bc060ce15ee69 HTTP/1.1" 502 150 2025-12-17 22:32:03,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a72575bc060ce15ee69'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:03,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a70575bc060ce15ee60 2025-12-17 22:32:03,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a70575bc060ce15ee60 HTTP/1.1" 502 150 2025-12-17 22:32:03,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a70575bc060ce15ee60'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:04,275 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a57575bc060ce15ee22 2025-12-17 22:32:04,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a57575bc060ce15ee22 HTTP/1.1" 502 150 2025-12-17 22:32:04,276 - lava_callback - ERROR - Error processing callback for node 69408a57575bc060ce15ee22: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a57575bc060ce15ee22 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a57575bc060ce15ee22 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a57575bc060ce15ee22 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:04,278 - lava_callback - INFO - Starting async job submit for node 69408a74575bc060ce15ee6c 2025-12-17 22:32:04,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:04,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a74575bc060ce15ee6c HTTP/1.1" 502 150 2025-12-17 22:32:04,285 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a74575bc060ce15ee6c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:04,286 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a74575bc060ce15ee6c 2025-12-17 22:32:04,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a74575bc060ce15ee6c HTTP/1.1" 502 150 2025-12-17 22:32:04,286 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a74575bc060ce15ee6c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:04,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a5a575bc060ce15ee25 2025-12-17 22:32:04,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a5a575bc060ce15ee25 HTTP/1.1" 502 150 2025-12-17 22:32:04,627 - lava_callback - ERROR - Error processing callback for node 69408a5a575bc060ce15ee25: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a5a575bc060ce15ee25 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a5a575bc060ce15ee25 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a5a575bc060ce15ee25 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:04,629 - lava_callback - INFO - Starting async job submit for node 69408a77575bc060ce15ee6f 2025-12-17 22:32:04,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:04,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a77575bc060ce15ee6f HTTP/1.1" 502 150 2025-12-17 22:32:04,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a77575bc060ce15ee6f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:04,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a77575bc060ce15ee6f 2025-12-17 22:32:04,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a77575bc060ce15ee6f HTTP/1.1" 502 150 2025-12-17 22:32:04,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a77575bc060ce15ee6f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:04,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a6a575bc060ce15ee51 2025-12-17 22:32:04,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6a575bc060ce15ee51 HTTP/1.1" 502 150 2025-12-17 22:32:04,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a6a575bc060ce15ee51'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:04,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940945d575bc060ce161000 2025-12-17 22:32:04,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940945d575bc060ce161000 HTTP/1.1" 502 150 2025-12-17 22:32:04,787 - lava_callback - ERROR - Error processing callback for node 6940945d575bc060ce161000: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940945d575bc060ce161000 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940945d575bc060ce161000 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940945d575bc060ce161000 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:04,790 - lava_callback - INFO - Starting async job submit for node 69408a79575bc060ce15ee74 2025-12-17 22:32:04,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:04,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a79575bc060ce15ee74 HTTP/1.1" 502 150 2025-12-17 22:32:04,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a79575bc060ce15ee74'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:04,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a79575bc060ce15ee74 2025-12-17 22:32:04,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a79575bc060ce15ee74 HTTP/1.1" 502 150 2025-12-17 22:32:04,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a79575bc060ce15ee74'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:04,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937b575bc060ce160d99 2025-12-17 22:32:04,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937b575bc060ce160d99 HTTP/1.1" 502 150 2025-12-17 22:32:04,989 - lava_callback - ERROR - Error processing callback for node 6940937b575bc060ce160d99: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940937b575bc060ce160d99 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940937b575bc060ce160d99 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940937b575bc060ce160d99 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:04,991 - lava_callback - INFO - Starting async job submit for node 69408be6575bc060ce15f398 2025-12-17 22:32:05,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:05,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be6575bc060ce15f398 HTTP/1.1" 502 150 2025-12-17 22:32:05,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be6575bc060ce15f398'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:05,050 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be6575bc060ce15f398 2025-12-17 22:32:05,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be6575bc060ce15f398 HTTP/1.1" 502 150 2025-12-17 22:32:05,050 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be6575bc060ce15f398'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:05,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a72575bc060ce15ee69 2025-12-17 22:32:05,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a72575bc060ce15ee69 HTTP/1.1" 502 150 2025-12-17 22:32:05,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a72575bc060ce15ee69'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:06,286 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a74575bc060ce15ee6c 2025-12-17 22:32:06,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a74575bc060ce15ee6c HTTP/1.1" 502 150 2025-12-17 22:32:06,288 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a74575bc060ce15ee6c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:06,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a6c575bc060ce15ee54 2025-12-17 22:32:06,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6c575bc060ce15ee54 HTTP/1.1" 502 150 2025-12-17 22:32:06,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a6c575bc060ce15ee54'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46816 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:32:06,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a77575bc060ce15ee6f 2025-12-17 22:32:06,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a77575bc060ce15ee6f HTTP/1.1" 502 150 2025-12-17 22:32:06,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a77575bc060ce15ee6f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:06,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a79575bc060ce15ee74 2025-12-17 22:32:06,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a79575bc060ce15ee74 HTTP/1.1" 502 150 2025-12-17 22:32:06,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a79575bc060ce15ee74'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:06,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a6e575bc060ce15ee57 2025-12-17 22:32:06,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6e575bc060ce15ee57 HTTP/1.1" 502 150 2025-12-17 22:32:06,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a6e575bc060ce15ee57'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:07,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be6575bc060ce15f398 2025-12-17 22:32:07,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be6575bc060ce15f398 HTTP/1.1" 502 150 2025-12-17 22:32:07,052 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be6575bc060ce15f398'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:07,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a70575bc060ce15ee60 2025-12-17 22:32:07,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a70575bc060ce15ee60 HTTP/1.1" 502 150 2025-12-17 22:32:07,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a70575bc060ce15ee60'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:08,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a5f575bc060ce15ee39 2025-12-17 22:32:08,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a5f575bc060ce15ee39 HTTP/1.1" 502 150 2025-12-17 22:32:08,812 - lava_callback - ERROR - Error processing callback for node 69408a5f575bc060ce15ee39: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a5f575bc060ce15ee39 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a5f575bc060ce15ee39 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a5f575bc060ce15ee39 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:08,814 - lava_callback - INFO - Starting async job submit for node 69408a7d575bc060ce15ee7a 2025-12-17 22:32:08,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:08,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7d575bc060ce15ee7a HTTP/1.1" 502 150 2025-12-17 22:32:08,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a7d575bc060ce15ee7a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:08,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a7d575bc060ce15ee7a 2025-12-17 22:32:08,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7d575bc060ce15ee7a HTTP/1.1" 502 150 2025-12-17 22:32:08,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a7d575bc060ce15ee7a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:09,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a72575bc060ce15ee69 2025-12-17 22:32:09,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a72575bc060ce15ee69 HTTP/1.1" 502 150 2025-12-17 22:32:09,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a72575bc060ce15ee69'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:10,288 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a74575bc060ce15ee6c 2025-12-17 22:32:10,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a74575bc060ce15ee6c HTTP/1.1" 502 150 2025-12-17 22:32:10,289 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a74575bc060ce15ee6c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:10,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a77575bc060ce15ee6f 2025-12-17 22:32:10,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a77575bc060ce15ee6f HTTP/1.1" 502 150 2025-12-17 22:32:10,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a77575bc060ce15ee6f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:10,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a79575bc060ce15ee74 2025-12-17 22:32:10,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a79575bc060ce15ee74 HTTP/1.1" 502 150 2025-12-17 22:32:10,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a79575bc060ce15ee74'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:10,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a7d575bc060ce15ee7a 2025-12-17 22:32:10,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7d575bc060ce15ee7a HTTP/1.1" 502 150 2025-12-17 22:32:10,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a7d575bc060ce15ee7a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:10,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a61575bc060ce15ee41 2025-12-17 22:32:10,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a61575bc060ce15ee41 HTTP/1.1" 502 150 2025-12-17 22:32:10,980 - lava_callback - ERROR - Error processing callback for node 69408a61575bc060ce15ee41: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a61575bc060ce15ee41 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a61575bc060ce15ee41 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a61575bc060ce15ee41 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:10,983 - lava_callback - INFO - Starting async job submit for node 69408a7e575bc060ce15ee7d 2025-12-17 22:32:10,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:10,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7e575bc060ce15ee7d HTTP/1.1" 502 150 2025-12-17 22:32:10,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a7e575bc060ce15ee7d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:10,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a7e575bc060ce15ee7d 2025-12-17 22:32:10,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7e575bc060ce15ee7d HTTP/1.1" 502 150 2025-12-17 22:32:10,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a7e575bc060ce15ee7d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:11,052 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be6575bc060ce15f398 2025-12-17 22:32:11,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be6575bc060ce15f398 HTTP/1.1" 502 150 2025-12-17 22:32:11,053 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be6575bc060ce15f398'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:12,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a7e575bc060ce15ee7d 2025-12-17 22:32:12,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7e575bc060ce15ee7d HTTP/1.1" 502 150 2025-12-17 22:32:12,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a7e575bc060ce15ee7d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:13,302 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a63575bc060ce15ee44 2025-12-17 22:32:13,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a63575bc060ce15ee44 HTTP/1.1" 502 150 2025-12-17 22:32:13,304 - lava_callback - ERROR - Error processing callback for node 69408a63575bc060ce15ee44: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a63575bc060ce15ee44 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a63575bc060ce15ee44 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a63575bc060ce15ee44 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:13,306 - lava_callback - INFO - Starting async job submit for node 69408a80575bc060ce15ee80 2025-12-17 22:32:13,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:13,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a80575bc060ce15ee80 HTTP/1.1" 502 150 2025-12-17 22:32:13,312 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a80575bc060ce15ee80'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:13,312 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a80575bc060ce15ee80 2025-12-17 22:32:13,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a80575bc060ce15ee80 HTTP/1.1" 502 150 2025-12-17 22:32:13,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a80575bc060ce15ee80'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:13,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940e125460bc8e26511373c 2025-12-17 22:32:13,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940e125460bc8e26511373c HTTP/1.1" 502 150 2025-12-17 22:32:13,425 - lava_callback - ERROR - Error processing callback for node 6940e125460bc8e26511373c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940e125460bc8e26511373c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940e125460bc8e26511373c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940e125460bc8e26511373c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:13,427 - lava_callback - INFO - Starting async job submit for node 69408a80575bc060ce15ee83 2025-12-17 22:32:13,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:13,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a80575bc060ce15ee83 HTTP/1.1" 502 150 2025-12-17 22:32:13,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a80575bc060ce15ee83'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:13,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a80575bc060ce15ee83 2025-12-17 22:32:13,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a80575bc060ce15ee83 HTTP/1.1" 502 150 2025-12-17 22:32:13,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a80575bc060ce15ee83'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:14,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a7d575bc060ce15ee7a INFO: 172.21.0.1:50682 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:32:15,313 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a80575bc060ce15ee80 2025-12-17 22:32:15,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a80575bc060ce15ee80 HTTP/1.1" 502 150 2025-12-17 22:32:15,314 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a80575bc060ce15ee80'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:15,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a80575bc060ce15ee83 2025-12-17 22:32:15,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a80575bc060ce15ee83 HTTP/1.1" 502 150 2025-12-17 22:32:15,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a80575bc060ce15ee83'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:15,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a70575bc060ce15ee60 2025-12-17 22:32:15,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a70575bc060ce15ee60 HTTP/1.1" 502 150 2025-12-17 22:32:15,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a70575bc060ce15ee60'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:16,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a50575bc060ce15eddb 2025-12-17 22:32:16,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a50575bc060ce15eddb HTTP/1.1" 502 150 2025-12-17 22:32:16,499 - lava_callback - ERROR - Error processing callback for node 69408a50575bc060ce15eddb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a50575bc060ce15eddb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a50575bc060ce15eddb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a50575bc060ce15eddb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:16,501 - lava_callback - INFO - Starting async job submit for node 69408a83575bc060ce15ee86 2025-12-17 22:32:16,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:50684 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:32:16,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a83575bc060ce15ee86 HTTP/1.1" 502 150 2025-12-17 22:32:16,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a83575bc060ce15ee86'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:16,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a83575bc060ce15ee86 2025-12-17 22:32:16,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a83575bc060ce15ee86 HTTP/1.1" 502 150 2025-12-17 22:32:16,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a83575bc060ce15ee86'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:16,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a7e575bc060ce15ee7d 2025-12-17 22:32:16,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7e575bc060ce15ee7d HTTP/1.1" 502 150 2025-12-17 22:32:16,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a7e575bc060ce15ee7d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:17,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a67575bc060ce15ee4e 2025-12-17 22:32:17,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a67575bc060ce15ee4e HTTP/1.1" 502 150 2025-12-17 22:32:17,031 - lava_callback - ERROR - Error processing callback for node 69408a67575bc060ce15ee4e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a67575bc060ce15ee4e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a67575bc060ce15ee4e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a67575bc060ce15ee4e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:17,033 - lava_callback - INFO - Starting async job submit for node 69408be8575bc060ce15f39b 2025-12-17 22:32:17,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:17,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be8575bc060ce15f39b HTTP/1.1" 502 150 2025-12-17 22:32:17,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be8575bc060ce15f39b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:17,071 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be8575bc060ce15f39b 2025-12-17 22:32:17,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be8575bc060ce15f39b HTTP/1.1" 502 150 2025-12-17 22:32:17,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be8575bc060ce15f39b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:17,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a72575bc060ce15ee69 2025-12-17 22:32:17,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a72575bc060ce15ee69 HTTP/1.1" 502 150 2025-12-17 22:32:17,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a72575bc060ce15ee69'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:18,291 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a74575bc060ce15ee6c 2025-12-17 22:32:18,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a83575bc060ce15ee86 2025-12-17 22:32:18,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a83575bc060ce15ee86 HTTP/1.1" 502 150 2025-12-17 22:32:18,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a83575bc060ce15ee86'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:18,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a77575bc060ce15ee6f 2025-12-17 22:32:18,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a77575bc060ce15ee6f HTTP/1.1" 502 150 2025-12-17 22:32:18,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a77575bc060ce15ee6f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:18,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a79575bc060ce15ee74 2025-12-17 22:32:18,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a79575bc060ce15ee74 HTTP/1.1" 502 150 2025-12-17 22:32:18,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a79575bc060ce15ee74'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:19,054 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be6575bc060ce15f398 2025-12-17 22:32:19,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be6575bc060ce15f398 HTTP/1.1" 502 150 2025-12-17 22:32:19,055 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be6575bc060ce15f398'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:19,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be8575bc060ce15f39b 2025-12-17 22:32:19,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be8575bc060ce15f39b HTTP/1.1" 502 150 2025-12-17 22:32:19,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be8575bc060ce15f39b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:19,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a80575bc060ce15ee80 2025-12-17 22:32:19,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a80575bc060ce15ee80 HTTP/1.1" 502 150 2025-12-17 22:32:19,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a80575bc060ce15ee80'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:19,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a80575bc060ce15ee83 2025-12-17 22:32:19,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a80575bc060ce15ee83 HTTP/1.1" 502 150 2025-12-17 22:32:19,437 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a80575bc060ce15ee83'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:20,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a6a575bc060ce15ee51 2025-12-17 22:32:20,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6a575bc060ce15ee51 HTTP/1.1" 502 150 2025-12-17 22:32:20,661 - lava_callback - ERROR - Error processing callback for node 69408a6a575bc060ce15ee51: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a6a575bc060ce15ee51 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a6a575bc060ce15ee51 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a6a575bc060ce15ee51 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:20,662 - lava_callback - INFO - Starting async job submit for node 69408a7b575bc060ce15ee77 2025-12-17 22:32:20,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:20,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7b575bc060ce15ee77 HTTP/1.1" 502 150 2025-12-17 22:32:20,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a7b575bc060ce15ee77'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:20,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a7b575bc060ce15ee77 2025-12-17 22:32:20,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7b575bc060ce15ee77 HTTP/1.1" 502 150 2025-12-17 22:32:20,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a7b575bc060ce15ee77'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:22,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a6c575bc060ce15ee54 2025-12-17 22:32:22,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6c575bc060ce15ee54 HTTP/1.1" 502 150 2025-12-17 22:32:22,512 - lava_callback - ERROR - Error processing callback for node 69408a6c575bc060ce15ee54: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a6c575bc060ce15ee54 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a6c575bc060ce15ee54 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a6c575bc060ce15ee54 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:22,514 - lava_callback - INFO - Starting async job submit for node 69408a41575bc060ce15edb9 2025-12-17 22:32:22,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a83575bc060ce15ee86 2025-12-17 22:32:22,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a83575bc060ce15ee86 HTTP/1.1" 502 150 2025-12-17 22:32:22,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a83575bc060ce15ee86'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:22,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:22,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a41575bc060ce15edb9 HTTP/1.1" 502 150 2025-12-17 22:32:22,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a41575bc060ce15edb9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:22,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a41575bc060ce15edb9 2025-12-17 22:32:22,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a41575bc060ce15edb9 HTTP/1.1" 502 150 2025-12-17 22:32:22,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a41575bc060ce15edb9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:22,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a7b575bc060ce15ee77 2025-12-17 22:32:22,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7b575bc060ce15ee77 HTTP/1.1" 502 150 2025-12-17 22:32:22,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a7b575bc060ce15ee77'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:22,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a6e575bc060ce15ee57 2025-12-17 22:32:22,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a6e575bc060ce15ee57 HTTP/1.1" 502 150 2025-12-17 22:32:22,839 - lava_callback - ERROR - Error processing callback for node 69408a6e575bc060ce15ee57: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a6e575bc060ce15ee57 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a6e575bc060ce15ee57 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a6e575bc060ce15ee57 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:22,841 - lava_callback - INFO - Starting async job submit for node 69408a85575bc060ce15ee98 2025-12-17 22:32:22,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:22,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a85575bc060ce15ee98 HTTP/1.1" 502 150 2025-12-17 22:32:22,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a85575bc060ce15ee98'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:22,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a85575bc060ce15ee98 2025-12-17 22:32:22,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a85575bc060ce15ee98 HTTP/1.1" 502 150 2025-12-17 22:32:22,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a85575bc060ce15ee98'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:23,074 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be8575bc060ce15f39b 2025-12-17 22:32:23,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be8575bc060ce15f39b HTTP/1.1" 502 150 2025-12-17 22:32:23,075 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be8575bc060ce15f39b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:24,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a41575bc060ce15edb9 2025-12-17 22:32:24,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a41575bc060ce15edb9 HTTP/1.1" 502 150 2025-12-17 22:32:24,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a41575bc060ce15edb9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:24,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a85575bc060ce15ee98 2025-12-17 22:32:24,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a85575bc060ce15ee98 HTTP/1.1" 502 150 2025-12-17 22:32:24,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a85575bc060ce15ee98'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:24,910 - lava_callback - INFO - Received callback for node 69408c63575bc060ce15f57f INFO: 172.21.0.1:40176 - "POST /node/69408c63575bc060ce15f57f HTTP/1.0" 202 Accepted 2025-12-17 22:32:24,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a7e575bc060ce15ee7d 2025-12-17 22:32:24,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7e575bc060ce15ee7d HTTP/1.1" 502 150 2025-12-17 22:32:24,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a7e575bc060ce15ee7d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:25,560 - lava_callback - INFO - Received callback for node 69408c66575bc060ce15f58c INFO: 172.21.0.1:40184 - "POST /node/69408c66575bc060ce15f58c HTTP/1.0" 202 Accepted 2025-12-17 22:32:26,208 - lava_callback - INFO - Received callback for node 69408c68575bc060ce15f591 INFO: 172.21.0.1:40200 - "POST /node/69408c68575bc060ce15f591 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:40202 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:32:26,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a7b575bc060ce15ee77 2025-12-17 22:32:26,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7b575bc060ce15ee77 HTTP/1.1" 502 150 2025-12-17 22:32:26,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a7b575bc060ce15ee77'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:26,851 - lava_callback - INFO - Received callback for node 69408c6a575bc060ce15f594 INFO: 172.21.0.1:40216 - "POST /node/69408c6a575bc060ce15f594 HTTP/1.0" 202 Accepted 2025-12-17 22:32:27,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a80575bc060ce15ee80 2025-12-17 22:32:27,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a80575bc060ce15ee80 HTTP/1.1" 502 150 2025-12-17 22:32:27,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a80575bc060ce15ee80'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:27,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a80575bc060ce15ee83 2025-12-17 22:32:27,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a80575bc060ce15ee83 HTTP/1.1" 502 150 2025-12-17 22:32:27,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a80575bc060ce15ee83'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:27,502 - lava_callback - INFO - Received callback for node 69408c6c575bc060ce15f597 INFO: 172.21.0.1:40226 - "POST /node/69408c6c575bc060ce15f597 HTTP/1.0" 202 Accepted 2025-12-17 22:32:28,162 - lava_callback - INFO - Received callback for node 69408c6e575bc060ce15f59a INFO: 172.21.0.1:40234 - "POST /node/69408c6e575bc060ce15f59a HTTP/1.0" 202 Accepted 2025-12-17 22:32:28,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a41575bc060ce15edb9 2025-12-17 22:32:28,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a41575bc060ce15edb9 HTTP/1.1" 502 150 2025-12-17 22:32:28,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a41575bc060ce15edb9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:28,793 - lava_callback - INFO - Received callback for node 69408c70575bc060ce15f59d INFO: 172.21.0.1:40248 - "POST /node/69408c70575bc060ce15f59d HTTP/1.0" 202 Accepted 2025-12-17 22:32:28,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a85575bc060ce15ee98 2025-12-17 22:32:28,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a85575bc060ce15ee98 HTTP/1.1" 502 150 2025-12-17 22:32:28,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a85575bc060ce15ee98'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:29,427 - lava_callback - INFO - Received callback for node 69408c74575bc060ce15f5a3 INFO: 172.21.0.1:40256 - "POST /node/69408c74575bc060ce15f5a3 HTTP/1.0" 202 Accepted 2025-12-17 22:32:30,059 - lava_callback - INFO - Received callback for node 69408c76575bc060ce15f5a6 INFO: 172.21.0.1:42476 - "POST /node/69408c76575bc060ce15f5a6 HTTP/1.0" 202 Accepted 2025-12-17 22:32:30,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7d575bc060ce15ee7a HTTP/1.1" 502 150 2025-12-17 22:32:30,128 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a7d575bc060ce15ee7a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:30,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a83575bc060ce15ee86 2025-12-17 22:32:30,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a83575bc060ce15ee86 HTTP/1.1" 502 150 2025-12-17 22:32:30,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a83575bc060ce15ee86'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:31,075 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be8575bc060ce15f39b 2025-12-17 22:32:31,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be8575bc060ce15f39b HTTP/1.1" 502 150 2025-12-17 22:32:31,078 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be8575bc060ce15f39b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:31,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a70575bc060ce15ee60 2025-12-17 22:32:31,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a70575bc060ce15ee60 HTTP/1.1" 502 150 2025-12-17 22:32:31,758 - lava_callback - ERROR - Error processing callback for node 69408a70575bc060ce15ee60: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a70575bc060ce15ee60 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a70575bc060ce15ee60 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a70575bc060ce15ee60 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:31,761 - lava_callback - INFO - Starting async job submit for node 69408a8e575bc060ce15eead 2025-12-17 22:32:31,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:31,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8e575bc060ce15eead HTTP/1.1" 502 150 2025-12-17 22:32:31,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8e575bc060ce15eead'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:31,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8e575bc060ce15eead 2025-12-17 22:32:31,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8e575bc060ce15eead HTTP/1.1" 502 150 2025-12-17 22:32:31,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8e575bc060ce15eead'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:33,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a55575bc060ce15ee1f HTTP/1.1" 502 150 2025-12-17 22:32:33,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a55575bc060ce15ee1f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:33,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a72575bc060ce15ee69 2025-12-17 22:32:33,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a72575bc060ce15ee69 HTTP/1.1" 502 150 2025-12-17 22:32:33,526 - lava_callback - ERROR - Error processing callback for node 69408a72575bc060ce15ee69: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a72575bc060ce15ee69 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a72575bc060ce15ee69 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a72575bc060ce15ee69 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:33,528 - lava_callback - INFO - Starting async job submit for node 69408a84575bc060ce15ee91 2025-12-17 22:32:33,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:33,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a84575bc060ce15ee91 HTTP/1.1" 502 150 2025-12-17 22:32:33,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a84575bc060ce15ee91'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:33,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a84575bc060ce15ee91 2025-12-17 22:32:33,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a84575bc060ce15ee91 HTTP/1.1" 502 150 2025-12-17 22:32:33,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a84575bc060ce15ee91'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:33,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a74575bc060ce15ee6c HTTP/1.1" 502 150 2025-12-17 22:32:33,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a74575bc060ce15ee6c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:33,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8e575bc060ce15eead 2025-12-17 22:32:33,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8e575bc060ce15eead HTTP/1.1" 502 150 2025-12-17 22:32:33,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8e575bc060ce15eead'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:34,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a77575bc060ce15ee6f 2025-12-17 22:32:34,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a77575bc060ce15ee6f HTTP/1.1" 502 150 2025-12-17 22:32:34,644 - lava_callback - ERROR - Error processing callback for node 69408a77575bc060ce15ee6f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a77575bc060ce15ee6f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a77575bc060ce15ee6f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a77575bc060ce15ee6f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:34,646 - lava_callback - INFO - Starting async job submit for node 69408a8f575bc060ce15eeb0 2025-12-17 22:32:34,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:34,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a7b575bc060ce15ee77 2025-12-17 22:32:34,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7b575bc060ce15ee77 HTTP/1.1" 502 150 2025-12-17 22:32:34,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a7b575bc060ce15ee77'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:34,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8f575bc060ce15eeb0 HTTP/1.1" 502 150 2025-12-17 22:32:34,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8f575bc060ce15eeb0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:34,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8f575bc060ce15eeb0 2025-12-17 22:32:34,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8f575bc060ce15eeb0 HTTP/1.1" 502 150 2025-12-17 22:32:34,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8f575bc060ce15eeb0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:34,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a79575bc060ce15ee74 2025-12-17 22:32:34,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a79575bc060ce15ee74 HTTP/1.1" 502 150 2025-12-17 22:32:34,804 - lava_callback - ERROR - Error processing callback for node 69408a79575bc060ce15ee74: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a79575bc060ce15ee74 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a79575bc060ce15ee74 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a79575bc060ce15ee74 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:34,806 - lava_callback - INFO - Starting async job submit for node 69408a87575bc060ce15ee9f 2025-12-17 22:32:34,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:34,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a87575bc060ce15ee9f HTTP/1.1" 502 150 2025-12-17 22:32:34,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a87575bc060ce15ee9f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:34,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a87575bc060ce15ee9f 2025-12-17 22:32:34,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a87575bc060ce15ee9f HTTP/1.1" 502 150 2025-12-17 22:32:34,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a87575bc060ce15ee9f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:35,055 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be6575bc060ce15f398 2025-12-17 22:32:35,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be6575bc060ce15f398 HTTP/1.1" 502 150 2025-12-17 22:32:35,057 - lava_callback - ERROR - Error processing callback for node 69408be6575bc060ce15f398: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be6575bc060ce15f398 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be6575bc060ce15f398 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be6575bc060ce15f398 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:35,059 - lava_callback - INFO - Starting async job submit for node 69408a88575bc060ce15eea2 2025-12-17 22:32:35,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:35,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a88575bc060ce15eea2 HTTP/1.1" 502 150 2025-12-17 22:32:35,066 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a88575bc060ce15eea2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:35,066 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a88575bc060ce15eea2 2025-12-17 22:32:35,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a88575bc060ce15eea2 HTTP/1.1" 502 150 2025-12-17 22:32:35,067 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a88575bc060ce15eea2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:35,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a55575bc060ce15ee1f 2025-12-17 22:32:35,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a84575bc060ce15ee91 2025-12-17 22:32:35,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a84575bc060ce15ee91 HTTP/1.1" 502 150 2025-12-17 22:32:35,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a84575bc060ce15ee91'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42492 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:32:36,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a41575bc060ce15edb9 2025-12-17 22:32:36,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a41575bc060ce15edb9 HTTP/1.1" 502 150 2025-12-17 22:32:36,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a41575bc060ce15edb9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:36,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8f575bc060ce15eeb0 2025-12-17 22:32:36,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8f575bc060ce15eeb0 HTTP/1.1" 502 150 2025-12-17 22:32:36,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8f575bc060ce15eeb0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:36,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a87575bc060ce15ee9f 2025-12-17 22:32:36,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a87575bc060ce15ee9f HTTP/1.1" 502 150 2025-12-17 22:32:36,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a87575bc060ce15ee9f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:36,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a85575bc060ce15ee98 2025-12-17 22:32:36,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a85575bc060ce15ee98 HTTP/1.1" 502 150 2025-12-17 22:32:36,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a85575bc060ce15ee98'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:37,067 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a88575bc060ce15eea2 2025-12-17 22:32:37,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a88575bc060ce15eea2 HTTP/1.1" 502 150 2025-12-17 22:32:37,068 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a88575bc060ce15eea2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:37,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8e575bc060ce15eead 2025-12-17 22:32:37,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8e575bc060ce15eead HTTP/1.1" 502 150 2025-12-17 22:32:37,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8e575bc060ce15eead'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:38,128 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a7d575bc060ce15ee7a 2025-12-17 22:32:38,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7d575bc060ce15ee7a HTTP/1.1" 502 150 2025-12-17 22:32:38,129 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a7d575bc060ce15ee7a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:39,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a84575bc060ce15ee91 2025-12-17 22:32:39,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a84575bc060ce15ee91 HTTP/1.1" 502 150 2025-12-17 22:32:39,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a84575bc060ce15ee91'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:40,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8f575bc060ce15eeb0 2025-12-17 22:32:40,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8f575bc060ce15eeb0 HTTP/1.1" 502 150 2025-12-17 22:32:40,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8f575bc060ce15eeb0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:40,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a87575bc060ce15ee9f 2025-12-17 22:32:40,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a87575bc060ce15ee9f HTTP/1.1" 502 150 2025-12-17 22:32:40,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a87575bc060ce15ee9f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:40,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a7e575bc060ce15ee7d 2025-12-17 22:32:40,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7e575bc060ce15ee7d HTTP/1.1" 502 150 2025-12-17 22:32:40,997 - lava_callback - ERROR - Error processing callback for node 69408a7e575bc060ce15ee7d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a7e575bc060ce15ee7d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a7e575bc060ce15ee7d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a7e575bc060ce15ee7d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:40,999 - lava_callback - INFO - Starting async job submit for node 69408a96575bc060ce15eee7 2025-12-17 22:32:41,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:41,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a96575bc060ce15eee7 HTTP/1.1" 502 150 2025-12-17 22:32:41,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a96575bc060ce15eee7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:41,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a96575bc060ce15eee7 2025-12-17 22:32:41,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a96575bc060ce15eee7 HTTP/1.1" 502 150 2025-12-17 22:32:41,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a96575bc060ce15eee7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:41,069 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a88575bc060ce15eea2 2025-12-17 22:32:41,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a88575bc060ce15eea2 HTTP/1.1" 502 150 2025-12-17 22:32:41,069 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a88575bc060ce15eea2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:42,084 - lava_callback - INFO - Received callback for node 69408c60575bc060ce15f577 INFO: 172.21.0.1:46494 - "POST /node/69408c60575bc060ce15f577 HTTP/1.0" 202 Accepted 2025-12-17 22:32:43,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a96575bc060ce15eee7 2025-12-17 22:32:43,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a96575bc060ce15eee7 HTTP/1.1" 502 150 2025-12-17 22:32:43,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a96575bc060ce15eee7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:43,318 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a80575bc060ce15ee80 2025-12-17 22:32:43,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a80575bc060ce15ee80 HTTP/1.1" 502 150 2025-12-17 22:32:43,321 - lava_callback - ERROR - Error processing callback for node 69408a80575bc060ce15ee80: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a80575bc060ce15ee80 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a80575bc060ce15ee80 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a80575bc060ce15ee80 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:43,322 - lava_callback - INFO - Starting async job submit for node 69408a8d575bc060ce15eeaa 2025-12-17 22:32:43,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:43,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8d575bc060ce15eeaa HTTP/1.1" 502 150 2025-12-17 22:32:43,329 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8d575bc060ce15eeaa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:43,330 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8d575bc060ce15eeaa 2025-12-17 22:32:43,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8d575bc060ce15eeaa HTTP/1.1" 502 150 2025-12-17 22:32:43,330 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8d575bc060ce15eeaa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:43,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a80575bc060ce15ee83 2025-12-17 22:32:43,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a80575bc060ce15ee83 HTTP/1.1" 502 150 2025-12-17 22:32:43,440 - lava_callback - ERROR - Error processing callback for node 69408a80575bc060ce15ee83: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a80575bc060ce15ee83 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a80575bc060ce15ee83 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a80575bc060ce15ee83 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:43,443 - lava_callback - INFO - Starting async job submit for node 69408a99575bc060ce15eef3 2025-12-17 22:32:43,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:43,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a99575bc060ce15eef3 HTTP/1.1" 502 150 2025-12-17 22:32:43,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a99575bc060ce15eef3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:43,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a99575bc060ce15eef3 2025-12-17 22:32:43,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a99575bc060ce15eef3 HTTP/1.1" 502 150 2025-12-17 22:32:43,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a99575bc060ce15eef3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:44,939 - lava_callback - INFO - Received callback for node 69408c78575bc060ce15f5ad INFO: 172.21.0.1:46502 - "POST /node/69408c78575bc060ce15f5ad HTTP/1.0" 202 Accepted 2025-12-17 22:32:45,330 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8d575bc060ce15eeaa 2025-12-17 22:32:45,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8d575bc060ce15eeaa HTTP/1.1" 502 150 2025-12-17 22:32:45,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8d575bc060ce15eeaa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:45,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a99575bc060ce15eef3 2025-12-17 22:32:45,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a99575bc060ce15eef3 HTTP/1.1" 502 150 2025-12-17 22:32:45,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a99575bc060ce15eef3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:45,575 - lava_callback - INFO - Received callback for node 69408c7a575bc060ce15f5b6 INFO: 172.21.0.1:46504 - "POST /node/69408c7a575bc060ce15f5b6 HTTP/1.0" 202 Accepted 2025-12-17 22:32:45,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8e575bc060ce15eead 2025-12-17 22:32:45,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8e575bc060ce15eead HTTP/1.1" 502 150 2025-12-17 22:32:45,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8e575bc060ce15eead'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:46,220 - lava_callback - INFO - Received callback for node 69408c7d575bc060ce15f5bf INFO: 172.21.0.1:46516 - "POST /node/69408c7d575bc060ce15f5bf HTTP/1.0" 202 Accepted INFO: 172.21.0.1:46520 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:32:46,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a83575bc060ce15ee86 2025-12-17 22:32:46,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a83575bc060ce15ee86 HTTP/1.1" 502 150 2025-12-17 22:32:46,539 - lava_callback - ERROR - Error processing callback for node 69408a83575bc060ce15ee86: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a83575bc060ce15ee86 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a83575bc060ce15ee86 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a83575bc060ce15ee86 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:46,541 - lava_callback - INFO - Starting async job submit for node 69408a9b575bc060ce15eefd 2025-12-17 22:32:46,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:46,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9b575bc060ce15eefd HTTP/1.1" 502 150 2025-12-17 22:32:46,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a9b575bc060ce15eefd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:46,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a9b575bc060ce15eefd 2025-12-17 22:32:46,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9b575bc060ce15eefd HTTP/1.1" 502 150 2025-12-17 22:32:46,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a9b575bc060ce15eefd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:46,875 - lava_callback - INFO - Received callback for node 69408c7f575bc060ce15f5c6 INFO: 172.21.0.1:46536 - "POST /node/69408c7f575bc060ce15f5c6 HTTP/1.0" 202 Accepted 2025-12-17 22:32:47,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a96575bc060ce15eee7 2025-12-17 22:32:47,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a96575bc060ce15eee7 HTTP/1.1" 502 150 2025-12-17 22:32:47,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a96575bc060ce15eee7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:47,078 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be8575bc060ce15f39b 2025-12-17 22:32:47,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be8575bc060ce15f39b HTTP/1.1" 502 150 2025-12-17 22:32:47,079 - lava_callback - ERROR - Error processing callback for node 69408be8575bc060ce15f39b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be8575bc060ce15f39b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be8575bc060ce15f39b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be8575bc060ce15f39b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:47,081 - lava_callback - INFO - Starting async job submit for node 69408a9d575bc060ce15ef00 2025-12-17 22:32:47,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:47,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9d575bc060ce15ef00 HTTP/1.1" 502 150 2025-12-17 22:32:47,111 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a9d575bc060ce15ef00'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:47,111 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a9d575bc060ce15ef00 2025-12-17 22:32:47,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9d575bc060ce15ef00 HTTP/1.1" 502 150 2025-12-17 22:32:47,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a9d575bc060ce15ef00'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:47,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a84575bc060ce15ee91 2025-12-17 22:32:47,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a84575bc060ce15ee91 HTTP/1.1" 502 150 2025-12-17 22:32:47,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a84575bc060ce15ee91'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:48,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a9b575bc060ce15eefd 2025-12-17 22:32:48,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9b575bc060ce15eefd HTTP/1.1" 502 150 2025-12-17 22:32:48,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a9b575bc060ce15eefd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:48,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8f575bc060ce15eeb0 2025-12-17 22:32:48,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8f575bc060ce15eeb0 HTTP/1.1" 502 150 2025-12-17 22:32:48,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8f575bc060ce15eeb0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:48,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a87575bc060ce15ee9f 2025-12-17 22:32:48,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a87575bc060ce15ee9f HTTP/1.1" 502 150 2025-12-17 22:32:48,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a87575bc060ce15ee9f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:49,070 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a88575bc060ce15eea2 2025-12-17 22:32:49,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a88575bc060ce15eea2 HTTP/1.1" 502 150 2025-12-17 22:32:49,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a88575bc060ce15eea2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:49,112 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a9d575bc060ce15ef00 2025-12-17 22:32:49,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9d575bc060ce15ef00 HTTP/1.1" 502 150 2025-12-17 22:32:49,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a9d575bc060ce15ef00'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:49,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8d575bc060ce15eeaa 2025-12-17 22:32:49,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8d575bc060ce15eeaa HTTP/1.1" 502 150 2025-12-17 22:32:49,333 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8d575bc060ce15eeaa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:49,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a99575bc060ce15eef3 2025-12-17 22:32:49,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a99575bc060ce15eef3 HTTP/1.1" 502 150 2025-12-17 22:32:49,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a99575bc060ce15eef3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:49,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a74575bc060ce15ee6c 2025-12-17 22:32:49,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a74575bc060ce15ee6c HTTP/1.1" 502 150 2025-12-17 22:32:49,714 - lava_callback - ERROR - Error processing callback for node 69408a74575bc060ce15ee6c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a74575bc060ce15ee6c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a74575bc060ce15ee6c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a74575bc060ce15ee6c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:49,717 - lava_callback - INFO - Starting async job submit for node 69408a9f575bc060ce15ef04 2025-12-17 22:32:49,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:49,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9f575bc060ce15ef04 HTTP/1.1" 502 150 2025-12-17 22:32:49,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a9f575bc060ce15ef04'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:49,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a9f575bc060ce15ef04 2025-12-17 22:32:49,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9f575bc060ce15ef04 HTTP/1.1" 502 150 2025-12-17 22:32:49,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a9f575bc060ce15ef04'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:50,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a7b575bc060ce15ee77 2025-12-17 22:32:50,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7b575bc060ce15ee77 HTTP/1.1" 502 150 2025-12-17 22:32:50,675 - lava_callback - ERROR - Error processing callback for node 69408a7b575bc060ce15ee77: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a7b575bc060ce15ee77 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a7b575bc060ce15ee77 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a7b575bc060ce15ee77 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:50,677 - lava_callback - INFO - Starting async job submit for node 69408aa2575bc060ce15ef0d 2025-12-17 22:32:50,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:50,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa2575bc060ce15ef0d HTTP/1.1" 502 150 2025-12-17 22:32:50,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa2575bc060ce15ef0d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:50,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa2575bc060ce15ef0d 2025-12-17 22:32:50,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa2575bc060ce15ef0d HTTP/1.1" 502 150 2025-12-17 22:32:50,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa2575bc060ce15ef0d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:51,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a9f575bc060ce15ef04 2025-12-17 22:32:52,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a41575bc060ce15edb9 2025-12-17 22:32:52,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a9b575bc060ce15eefd 2025-12-17 22:32:52,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a41575bc060ce15edb9 HTTP/1.1" 502 150 2025-12-17 22:32:52,553 - lava_callback - ERROR - Error processing callback for node 69408a41575bc060ce15edb9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a41575bc060ce15edb9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a41575bc060ce15edb9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a41575bc060ce15edb9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:52,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9b575bc060ce15eefd HTTP/1.1" 502 150 2025-12-17 22:32:52,555 - lava_callback - INFO - Starting async job submit for node 69408aa4575bc060ce15ef10 2025-12-17 22:32:52,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a9b575bc060ce15eefd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:52,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:52,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa4575bc060ce15ef10 HTTP/1.1" 502 150 2025-12-17 22:32:52,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa4575bc060ce15ef10'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:52,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa4575bc060ce15ef10 2025-12-17 22:32:52,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa4575bc060ce15ef10 HTTP/1.1" 502 150 2025-12-17 22:32:52,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa4575bc060ce15ef10'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:52,669 - lava_callback - INFO - Received callback for node 6940f14b460bc8e2651152a9 INFO: 172.21.0.1:47168 - "POST /node/6940f14b460bc8e2651152a9 HTTP/1.0" 202 Accepted 2025-12-17 22:32:52,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa2575bc060ce15ef0d 2025-12-17 22:32:52,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa2575bc060ce15ef0d HTTP/1.1" 502 150 2025-12-17 22:32:52,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa2575bc060ce15ef0d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:52,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a85575bc060ce15ee98 2025-12-17 22:32:52,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a85575bc060ce15ee98 HTTP/1.1" 502 150 2025-12-17 22:32:52,854 - lava_callback - ERROR - Error processing callback for node 69408a85575bc060ce15ee98: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a85575bc060ce15ee98 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a85575bc060ce15ee98 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a85575bc060ce15ee98 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:52,856 - lava_callback - INFO - Starting async job submit for node 69408aa6575bc060ce15ef13 2025-12-17 22:32:52,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:52,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa6575bc060ce15ef13 HTTP/1.1" 502 150 2025-12-17 22:32:52,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa6575bc060ce15ef13'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:52,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa6575bc060ce15ef13 2025-12-17 22:32:52,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa6575bc060ce15ef13 HTTP/1.1" 502 150 2025-12-17 22:32:52,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa6575bc060ce15ef13'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:53,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a9d575bc060ce15ef00 2025-12-17 22:32:53,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9d575bc060ce15ef00 HTTP/1.1" 502 150 2025-12-17 22:32:53,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a9d575bc060ce15ef00'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:54,130 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a7d575bc060ce15ee7a 2025-12-17 22:32:54,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a7d575bc060ce15ee7a HTTP/1.1" 502 150 2025-12-17 22:32:54,131 - lava_callback - ERROR - Error processing callback for node 69408a7d575bc060ce15ee7a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a7d575bc060ce15ee7a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a7d575bc060ce15ee7a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a7d575bc060ce15ee7a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:32:54,133 - lava_callback - INFO - Starting async job submit for node 69408aa8575bc060ce15ef16 2025-12-17 22:32:54,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:32:54,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa8575bc060ce15ef16 HTTP/1.1" 502 150 2025-12-17 22:32:54,140 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa8575bc060ce15ef16'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:54,140 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa8575bc060ce15ef16 2025-12-17 22:32:54,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa8575bc060ce15ef16 HTTP/1.1" 502 150 2025-12-17 22:32:54,141 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa8575bc060ce15ef16'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:54,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa4575bc060ce15ef10 2025-12-17 22:32:54,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa4575bc060ce15ef10 HTTP/1.1" 502 150 2025-12-17 22:32:54,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa4575bc060ce15ef10'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:54,778 - lava_callback - INFO - Received callback for node 69408c82575bc060ce15f5ed INFO: 172.21.0.1:47180 - "POST /node/69408c82575bc060ce15f5ed HTTP/1.0" 202 Accepted 2025-12-17 22:32:54,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa6575bc060ce15ef13 2025-12-17 22:32:54,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa6575bc060ce15ef13 HTTP/1.1" 502 150 2025-12-17 22:32:54,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa6575bc060ce15ef13'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:55,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a96575bc060ce15eee7 2025-12-17 22:32:55,434 - lava_callback - INFO - Received callback for node 69408c84575bc060ce15f5f0 INFO: 172.21.0.1:47184 - "POST /node/69408c84575bc060ce15f5f0 HTTP/1.0" 202 Accepted 2025-12-17 22:32:56,105 - lava_callback - INFO - Received callback for node 69408c86575bc060ce15f5f3 INFO: 172.21.0.1:47190 - "POST /node/69408c86575bc060ce15f5f3 HTTP/1.0" 202 Accepted 2025-12-17 22:32:56,141 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa8575bc060ce15ef16 2025-12-17 22:32:56,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa8575bc060ce15ef16 HTTP/1.1" 502 150 2025-12-17 22:32:56,142 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa8575bc060ce15ef16'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47206 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:32:56,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa2575bc060ce15ef0d 2025-12-17 22:32:56,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa2575bc060ce15ef0d HTTP/1.1" 502 150 2025-12-17 22:32:56,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa2575bc060ce15ef0d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:57,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8d575bc060ce15eeaa 2025-12-17 22:32:57,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8d575bc060ce15eeaa HTTP/1.1" 502 150 2025-12-17 22:32:57,335 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a8d575bc060ce15eeaa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:57,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a99575bc060ce15eef3 2025-12-17 22:32:57,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a99575bc060ce15eef3 HTTP/1.1" 502 150 2025-12-17 22:32:57,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a99575bc060ce15eef3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:58,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa4575bc060ce15ef10 2025-12-17 22:32:58,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa4575bc060ce15ef10 HTTP/1.1" 502 150 2025-12-17 22:32:58,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa4575bc060ce15ef10'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:32:58,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa6575bc060ce15ef13 2025-12-17 22:32:58,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa6575bc060ce15ef13 HTTP/1.1" 502 150 2025-12-17 22:32:58,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa6575bc060ce15ef13'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:00,142 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa8575bc060ce15ef16 2025-12-17 22:33:00,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa8575bc060ce15ef16 HTTP/1.1" 502 150 2025-12-17 22:33:00,143 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa8575bc060ce15ef16'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:00,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a9b575bc060ce15eefd 2025-12-17 22:33:00,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9b575bc060ce15eefd HTTP/1.1" 502 150 2025-12-17 22:33:00,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a9b575bc060ce15eefd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:01,115 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a9d575bc060ce15ef00 2025-12-17 22:33:01,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9d575bc060ce15ef00 HTTP/1.1" 502 150 2025-12-17 22:33:01,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a9d575bc060ce15ef00'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:01,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8e575bc060ce15eead 2025-12-17 22:33:01,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8e575bc060ce15eead HTTP/1.1" 502 150 2025-12-17 22:33:01,793 - lava_callback - ERROR - Error processing callback for node 69408a8e575bc060ce15eead: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a8e575bc060ce15eead (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a8e575bc060ce15eead (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a8e575bc060ce15eead (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:01,795 - lava_callback - INFO - Starting async job submit for node 69408aaa575bc060ce15ef19 2025-12-17 22:33:01,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:01,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aaa575bc060ce15ef19 HTTP/1.1" 502 150 2025-12-17 22:33:01,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aaa575bc060ce15ef19'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:01,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aaa575bc060ce15ef19 2025-12-17 22:33:01,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aaa575bc060ce15ef19 HTTP/1.1" 502 150 2025-12-17 22:33:01,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aaa575bc060ce15ef19'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56988 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:33:02,823 - lava_callback - INFO - Received callback for node 69408c8b575bc060ce15f5fc INFO: 172.21.0.1:56992 - "POST /node/69408c8b575bc060ce15f5fc HTTP/1.0" 202 Accepted 2025-12-17 22:33:03,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a84575bc060ce15ee91 2025-12-17 22:33:03,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a84575bc060ce15ee91 HTTP/1.1" 502 150 2025-12-17 22:33:03,568 - lava_callback - ERROR - Error processing callback for node 69408a84575bc060ce15ee91: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a84575bc060ce15ee91 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a84575bc060ce15ee91 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a84575bc060ce15ee91 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:03,570 - lava_callback - INFO - Starting async job submit for node 69408ab0575bc060ce15ef30 2025-12-17 22:33:03,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:03,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab0575bc060ce15ef30 HTTP/1.1" 502 150 2025-12-17 22:33:03,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab0575bc060ce15ef30'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:03,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab0575bc060ce15ef30 2025-12-17 22:33:03,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab0575bc060ce15ef30 HTTP/1.1" 502 150 2025-12-17 22:33:03,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab0575bc060ce15ef30'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:03,605 - lava_callback - INFO - Received callback for node 69408c8c575bc060ce15f601 INFO: 172.21.0.1:57004 - "POST /node/69408c8c575bc060ce15f601 HTTP/1.0" 202 Accepted 2025-12-17 22:33:03,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aaa575bc060ce15ef19 2025-12-17 22:33:03,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aaa575bc060ce15ef19 HTTP/1.1" 502 150 2025-12-17 22:33:03,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aaa575bc060ce15ef19'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:04,320 - lava_callback - INFO - Received callback for node 69408c8e575bc060ce15f608 INFO: 172.21.0.1:57018 - "POST /node/69408c8e575bc060ce15f608 HTTP/1.0" 202 Accepted 2025-12-17 22:33:04,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8f575bc060ce15eeb0 2025-12-17 22:33:04,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa2575bc060ce15ef0d 2025-12-17 22:33:04,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8f575bc060ce15eeb0 HTTP/1.1" 502 150 2025-12-17 22:33:04,693 - lava_callback - ERROR - Error processing callback for node 69408a8f575bc060ce15eeb0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a8f575bc060ce15eeb0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a8f575bc060ce15eeb0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a8f575bc060ce15eeb0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:04,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa2575bc060ce15ef0d HTTP/1.1" 502 150 2025-12-17 22:33:04,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa2575bc060ce15ef0d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:04,695 - lava_callback - INFO - Starting async job submit for node 69408ab3575bc060ce15ef3a 2025-12-17 22:33:04,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:04,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab3575bc060ce15ef3a HTTP/1.1" 502 150 2025-12-17 22:33:04,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab3575bc060ce15ef3a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:04,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab3575bc060ce15ef3a 2025-12-17 22:33:04,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab3575bc060ce15ef3a HTTP/1.1" 502 150 2025-12-17 22:33:04,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab3575bc060ce15ef3a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:04,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a87575bc060ce15ee9f 2025-12-17 22:33:04,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a87575bc060ce15ee9f HTTP/1.1" 502 150 2025-12-17 22:33:04,826 - lava_callback - ERROR - Error processing callback for node 69408a87575bc060ce15ee9f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a87575bc060ce15ee9f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a87575bc060ce15ee9f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a87575bc060ce15ee9f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:04,827 - lava_callback - INFO - Starting async job submit for node 69408aac575bc060ce15ef1c 2025-12-17 22:33:04,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:04,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aac575bc060ce15ef1c HTTP/1.1" 502 150 2025-12-17 22:33:04,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aac575bc060ce15ef1c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:04,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aac575bc060ce15ef1c 2025-12-17 22:33:04,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aac575bc060ce15ef1c HTTP/1.1" 502 150 2025-12-17 22:33:04,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aac575bc060ce15ef1c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:04,936 - lava_callback - INFO - Received callback for node 69408c99575bc060ce15f62a INFO: 172.21.0.1:57022 - "POST /node/69408c99575bc060ce15f62a HTTP/1.0" 202 Accepted 2025-12-17 22:33:05,071 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a88575bc060ce15eea2 2025-12-17 22:33:05,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a88575bc060ce15eea2 HTTP/1.1" 502 150 2025-12-17 22:33:05,072 - lava_callback - ERROR - Error processing callback for node 69408a88575bc060ce15eea2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a88575bc060ce15eea2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a88575bc060ce15eea2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a88575bc060ce15eea2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:05,074 - lava_callback - INFO - Starting async job submit for node 69408abc575bc060ce15ef4d 2025-12-17 22:33:05,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:05,079 - lava_callback - INFO - Received callback for node 69408c8f575bc060ce15f610 INFO: 172.21.0.1:57032 - "POST /node/69408c8f575bc060ce15f610 HTTP/1.0" 202 Accepted 2025-12-17 22:33:05,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abc575bc060ce15ef4d HTTP/1.1" 502 150 2025-12-17 22:33:05,087 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abc575bc060ce15ef4d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:05,087 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abc575bc060ce15ef4d 2025-12-17 22:33:05,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abc575bc060ce15ef4d HTTP/1.1" 502 150 2025-12-17 22:33:05,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abc575bc060ce15ef4d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:05,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab0575bc060ce15ef30 2025-12-17 22:33:05,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab0575bc060ce15ef30 HTTP/1.1" 502 150 2025-12-17 22:33:05,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab0575bc060ce15ef30'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:05,604 - lava_callback - INFO - Received callback for node 69408c9b575bc060ce15f62d INFO: 172.21.0.1:57034 - "POST /node/69408c9b575bc060ce15f62d HTTP/1.0" 202 Accepted 2025-12-17 22:33:05,800 - lava_callback - INFO - Received callback for node 69408c91575bc060ce15f617 INFO: 172.21.0.1:57038 - "POST /node/69408c91575bc060ce15f617 HTTP/1.0" 202 Accepted 2025-12-17 22:33:05,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a55575bc060ce15ee1f HTTP/1.1" 502 150 2025-12-17 22:33:05,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a55575bc060ce15ee1f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:06,251 - lava_callback - INFO - Received callback for node 69408c9d575bc060ce15f630 INFO: 172.21.0.1:57042 - "POST /node/69408c9d575bc060ce15f630 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:57050 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:33:06,553 - lava_callback - INFO - Received callback for node 69408c95575bc060ce15f621 INFO: 172.21.0.1:57062 - "POST /node/69408c95575bc060ce15f621 HTTP/1.0" 202 Accepted 2025-12-17 22:33:06,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa4575bc060ce15ef10 2025-12-17 22:33:06,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa4575bc060ce15ef10 HTTP/1.1" 502 150 2025-12-17 22:33:06,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa4575bc060ce15ef10'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:06,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab3575bc060ce15ef3a 2025-12-17 22:33:06,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab3575bc060ce15ef3a HTTP/1.1" 502 150 2025-12-17 22:33:06,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab3575bc060ce15ef3a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:06,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aac575bc060ce15ef1c 2025-12-17 22:33:06,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aac575bc060ce15ef1c HTTP/1.1" 502 150 2025-12-17 22:33:06,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aac575bc060ce15ef1c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:06,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa6575bc060ce15ef13 2025-12-17 22:33:06,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa6575bc060ce15ef13 HTTP/1.1" 502 150 2025-12-17 22:33:06,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa6575bc060ce15ef13'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:06,901 - lava_callback - INFO - Received callback for node 69408c9f575bc060ce15f635 INFO: 172.21.0.1:57076 - "POST /node/69408c9f575bc060ce15f635 HTTP/1.0" 202 Accepted 2025-12-17 22:33:06,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9f575bc060ce15ef04 HTTP/1.1" 502 150 2025-12-17 22:33:06,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a9f575bc060ce15ef04'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:07,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abc575bc060ce15ef4d 2025-12-17 22:33:07,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abc575bc060ce15ef4d HTTP/1.1" 502 150 2025-12-17 22:33:07,089 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abc575bc060ce15ef4d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:07,326 - lava_callback - INFO - Received callback for node 69408c96575bc060ce15f624 INFO: 172.21.0.1:57080 - "POST /node/69408c96575bc060ce15f624 HTTP/1.0" 202 Accepted 2025-12-17 22:33:07,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aaa575bc060ce15ef19 2025-12-17 22:33:07,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aaa575bc060ce15ef19 HTTP/1.1" 502 150 2025-12-17 22:33:07,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aaa575bc060ce15ef19'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:08,144 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa8575bc060ce15ef16 2025-12-17 22:33:08,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa8575bc060ce15ef16 HTTP/1.1" 502 150 2025-12-17 22:33:08,145 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aa8575bc060ce15ef16'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:09,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab0575bc060ce15ef30 2025-12-17 22:33:09,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab0575bc060ce15ef30 HTTP/1.1" 502 150 2025-12-17 22:33:09,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab0575bc060ce15ef30'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:09,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a55575bc060ce15ee1f 2025-12-17 22:33:10,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a96575bc060ce15eee7 HTTP/1.1" 502 150 2025-12-17 22:33:10,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a96575bc060ce15eee7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:10,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab3575bc060ce15ef3a 2025-12-17 22:33:10,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab3575bc060ce15ef3a HTTP/1.1" 502 150 2025-12-17 22:33:10,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab3575bc060ce15ef3a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:10,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aac575bc060ce15ef1c 2025-12-17 22:33:10,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aac575bc060ce15ef1c HTTP/1.1" 502 150 2025-12-17 22:33:10,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aac575bc060ce15ef1c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:10,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a9f575bc060ce15ef04 2025-12-17 22:33:10,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9f575bc060ce15ef04 HTTP/1.1" 502 150 2025-12-17 22:33:10,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a9f575bc060ce15ef04'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:11,089 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abc575bc060ce15ef4d 2025-12-17 22:33:11,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abc575bc060ce15ef4d HTTP/1.1" 502 150 2025-12-17 22:33:11,091 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abc575bc060ce15ef4d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:13,335 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a8d575bc060ce15eeaa 2025-12-17 22:33:13,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a8d575bc060ce15eeaa HTTP/1.1" 502 150 2025-12-17 22:33:13,336 - lava_callback - ERROR - Error processing callback for node 69408a8d575bc060ce15eeaa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a8d575bc060ce15eeaa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a8d575bc060ce15eeaa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a8d575bc060ce15eeaa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:13,338 - lava_callback - INFO - Starting async job submit for node 69408abe575bc060ce15ef50 2025-12-17 22:33:13,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:13,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abe575bc060ce15ef50 HTTP/1.1" 502 150 2025-12-17 22:33:13,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abe575bc060ce15ef50'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:13,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abe575bc060ce15ef50 2025-12-17 22:33:13,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abe575bc060ce15ef50 HTTP/1.1" 502 150 2025-12-17 22:33:13,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abe575bc060ce15ef50'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:13,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a99575bc060ce15eef3 2025-12-17 22:33:13,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a99575bc060ce15eef3 HTTP/1.1" 502 150 2025-12-17 22:33:13,463 - lava_callback - ERROR - Error processing callback for node 69408a99575bc060ce15eef3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a99575bc060ce15eef3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a99575bc060ce15eef3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a99575bc060ce15eef3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:13,465 - lava_callback - INFO - Starting async job submit for node 69408abf575bc060ce15ef55 2025-12-17 22:33:13,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:13,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abf575bc060ce15ef55 HTTP/1.1" 502 150 2025-12-17 22:33:13,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abf575bc060ce15ef55'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:13,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abf575bc060ce15ef55 2025-12-17 22:33:13,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abf575bc060ce15ef55 HTTP/1.1" 502 150 2025-12-17 22:33:13,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abf575bc060ce15ef55'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45064 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:33:15,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abe575bc060ce15ef50 2025-12-17 22:33:15,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abe575bc060ce15ef50 HTTP/1.1" 502 150 2025-12-17 22:33:15,350 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abe575bc060ce15ef50'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:15,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abf575bc060ce15ef55 2025-12-17 22:33:15,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abf575bc060ce15ef55 HTTP/1.1" 502 150 2025-12-17 22:33:15,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abf575bc060ce15ef55'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:15,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aaa575bc060ce15ef19 2025-12-17 22:33:15,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aaa575bc060ce15ef19 HTTP/1.1" 502 150 2025-12-17 22:33:15,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aaa575bc060ce15ef19'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45078 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:33:16,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a9b575bc060ce15eefd 2025-12-17 22:33:16,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9b575bc060ce15eefd HTTP/1.1" 502 150 2025-12-17 22:33:16,558 - lava_callback - ERROR - Error processing callback for node 69408a9b575bc060ce15eefd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a9b575bc060ce15eefd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a9b575bc060ce15eefd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a9b575bc060ce15eefd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:16,560 - lava_callback - INFO - Starting async job submit for node 69408abb575bc060ce15ef4a 2025-12-17 22:33:16,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:16,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abb575bc060ce15ef4a HTTP/1.1" 502 150 2025-12-17 22:33:16,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abb575bc060ce15ef4a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:16,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abb575bc060ce15ef4a 2025-12-17 22:33:16,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abb575bc060ce15ef4a HTTP/1.1" 502 150 2025-12-17 22:33:16,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abb575bc060ce15ef4a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:17,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a9d575bc060ce15ef00 2025-12-17 22:33:17,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab0575bc060ce15ef30 2025-12-17 22:33:17,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab0575bc060ce15ef30 HTTP/1.1" 502 150 2025-12-17 22:33:17,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab0575bc060ce15ef30'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:18,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abb575bc060ce15ef4a 2025-12-17 22:33:18,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abb575bc060ce15ef4a HTTP/1.1" 502 150 2025-12-17 22:33:18,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abb575bc060ce15ef4a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:18,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab3575bc060ce15ef3a 2025-12-17 22:33:18,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab3575bc060ce15ef3a HTTP/1.1" 502 150 2025-12-17 22:33:18,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab3575bc060ce15ef3a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:18,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aac575bc060ce15ef1c 2025-12-17 22:33:18,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aac575bc060ce15ef1c HTTP/1.1" 502 150 2025-12-17 22:33:18,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aac575bc060ce15ef1c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:18,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a9f575bc060ce15ef04 2025-12-17 22:33:18,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9f575bc060ce15ef04 HTTP/1.1" 502 150 2025-12-17 22:33:18,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a9f575bc060ce15ef04'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:19,091 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abc575bc060ce15ef4d 2025-12-17 22:33:19,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abc575bc060ce15ef4d HTTP/1.1" 502 150 2025-12-17 22:33:19,093 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abc575bc060ce15ef4d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:19,350 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abe575bc060ce15ef50 2025-12-17 22:33:19,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abe575bc060ce15ef50 HTTP/1.1" 502 150 2025-12-17 22:33:19,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abe575bc060ce15ef50'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:19,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abf575bc060ce15ef55 2025-12-17 22:33:19,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abf575bc060ce15ef55 HTTP/1.1" 502 150 2025-12-17 22:33:19,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abf575bc060ce15ef55'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:20,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa2575bc060ce15ef0d 2025-12-17 22:33:20,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa2575bc060ce15ef0d HTTP/1.1" 502 150 2025-12-17 22:33:20,696 - lava_callback - ERROR - Error processing callback for node 69408aa2575bc060ce15ef0d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aa2575bc060ce15ef0d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aa2575bc060ce15ef0d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aa2575bc060ce15ef0d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:20,698 - lava_callback - INFO - Starting async job submit for node 69408ac2575bc060ce15ef63 2025-12-17 22:33:20,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:20,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac2575bc060ce15ef63 HTTP/1.1" 502 150 2025-12-17 22:33:20,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac2575bc060ce15ef63'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:20,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac2575bc060ce15ef63 2025-12-17 22:33:20,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac2575bc060ce15ef63 HTTP/1.1" 502 150 2025-12-17 22:33:20,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac2575bc060ce15ef63'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:22,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa4575bc060ce15ef10 2025-12-17 22:33:22,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa4575bc060ce15ef10 HTTP/1.1" 502 150 2025-12-17 22:33:22,570 - lava_callback - ERROR - Error processing callback for node 69408aa4575bc060ce15ef10: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aa4575bc060ce15ef10 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aa4575bc060ce15ef10 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aa4575bc060ce15ef10 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:22,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abb575bc060ce15ef4a 2025-12-17 22:33:22,572 - lava_callback - INFO - Starting async job submit for node 69408ac3575bc060ce15ef6a 2025-12-17 22:33:22,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:22,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abb575bc060ce15ef4a HTTP/1.1" 502 150 2025-12-17 22:33:22,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abb575bc060ce15ef4a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:22,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac3575bc060ce15ef6a HTTP/1.1" 502 150 2025-12-17 22:33:22,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac3575bc060ce15ef6a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:22,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac3575bc060ce15ef6a 2025-12-17 22:33:22,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac3575bc060ce15ef6a HTTP/1.1" 502 150 2025-12-17 22:33:22,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac3575bc060ce15ef6a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:22,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac2575bc060ce15ef63 2025-12-17 22:33:22,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac2575bc060ce15ef63 HTTP/1.1" 502 150 2025-12-17 22:33:22,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac2575bc060ce15ef63'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:22,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa6575bc060ce15ef13 2025-12-17 22:33:22,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa6575bc060ce15ef13 HTTP/1.1" 502 150 2025-12-17 22:33:22,873 - lava_callback - ERROR - Error processing callback for node 69408aa6575bc060ce15ef13: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aa6575bc060ce15ef13 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aa6575bc060ce15ef13 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aa6575bc060ce15ef13 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:22,875 - lava_callback - INFO - Starting async job submit for node 69408ab5575bc060ce15ef41 2025-12-17 22:33:22,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:22,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab5575bc060ce15ef41 HTTP/1.1" 502 150 2025-12-17 22:33:22,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab5575bc060ce15ef41'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:22,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab5575bc060ce15ef41 2025-12-17 22:33:22,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab5575bc060ce15ef41 HTTP/1.1" 502 150 2025-12-17 22:33:22,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab5575bc060ce15ef41'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:24,145 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aa8575bc060ce15ef16 2025-12-17 22:33:24,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aa8575bc060ce15ef16 HTTP/1.1" 502 150 2025-12-17 22:33:24,147 - lava_callback - ERROR - Error processing callback for node 69408aa8575bc060ce15ef16: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aa8575bc060ce15ef16 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aa8575bc060ce15ef16 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aa8575bc060ce15ef16 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:24,148 - lava_callback - INFO - Starting async job submit for node 69408ab7575bc060ce15ef44 2025-12-17 22:33:24,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:24,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab7575bc060ce15ef44 HTTP/1.1" 502 150 2025-12-17 22:33:24,162 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab7575bc060ce15ef44'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:24,162 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab7575bc060ce15ef44 2025-12-17 22:33:24,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab7575bc060ce15ef44 HTTP/1.1" 502 150 2025-12-17 22:33:24,166 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab7575bc060ce15ef44'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:24,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac3575bc060ce15ef6a 2025-12-17 22:33:24,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac3575bc060ce15ef6a HTTP/1.1" 502 150 2025-12-17 22:33:24,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac3575bc060ce15ef6a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:24,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab5575bc060ce15ef41 2025-12-17 22:33:24,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab5575bc060ce15ef41 HTTP/1.1" 502 150 2025-12-17 22:33:24,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab5575bc060ce15ef41'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:24,965 - lava_callback - INFO - Received callback for node 69408ca1575bc060ce15f63e INFO: 172.21.0.1:57296 - "POST /node/69408ca1575bc060ce15f63e HTTP/1.0" 202 Accepted 2025-12-17 22:33:25,597 - lava_callback - INFO - Received callback for node 69408ca3575bc060ce15f645 INFO: 172.21.0.1:57308 - "POST /node/69408ca3575bc060ce15f645 HTTP/1.0" 202 Accepted 2025-12-17 22:33:26,166 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab7575bc060ce15ef44 2025-12-17 22:33:26,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab7575bc060ce15ef44 HTTP/1.1" 502 150 2025-12-17 22:33:26,167 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab7575bc060ce15ef44'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:26,241 - lava_callback - INFO - Received callback for node 69408ca5575bc060ce15f648 INFO: 172.21.0.1:57320 - "POST /node/69408ca5575bc060ce15f648 HTTP/1.0" 202 Accepted 2025-12-17 22:33:26,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a96575bc060ce15eee7 2025-12-17 22:33:26,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a96575bc060ce15eee7 HTTP/1.1" 502 150 2025-12-17 22:33:26,322 - lava_callback - ERROR - Error processing callback for node 69408a96575bc060ce15eee7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a96575bc060ce15eee7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a96575bc060ce15eee7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a96575bc060ce15eee7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:26,324 - lava_callback - INFO - Starting async job submit for node 69408aba575bc060ce15ef47 2025-12-17 22:33:26,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:26,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aba575bc060ce15ef47 HTTP/1.1" 502 150 2025-12-17 22:33:26,330 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aba575bc060ce15ef47'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:26,330 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aba575bc060ce15ef47 2025-12-17 22:33:26,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aba575bc060ce15ef47 HTTP/1.1" 502 150 2025-12-17 22:33:26,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aba575bc060ce15ef47'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57324 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:33:26,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac2575bc060ce15ef63 2025-12-17 22:33:26,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac2575bc060ce15ef63 HTTP/1.1" 502 150 2025-12-17 22:33:26,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac2575bc060ce15ef63'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:26,879 - lava_callback - INFO - Received callback for node 69408ca7575bc060ce15f64b INFO: 172.21.0.1:57332 - "POST /node/69408ca7575bc060ce15f64b HTTP/1.0" 202 Accepted 2025-12-17 22:33:27,352 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abe575bc060ce15ef50 2025-12-17 22:33:27,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abe575bc060ce15ef50 HTTP/1.1" 502 150 2025-12-17 22:33:27,353 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abe575bc060ce15ef50'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:27,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abf575bc060ce15ef55 2025-12-17 22:33:27,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abf575bc060ce15ef55 HTTP/1.1" 502 150 2025-12-17 22:33:27,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abf575bc060ce15ef55'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:27,521 - lava_callback - INFO - Received callback for node 69408ca9575bc060ce15f64e INFO: 172.21.0.1:57346 - "POST /node/69408ca9575bc060ce15f64e HTTP/1.0" 202 Accepted 2025-12-17 22:33:28,163 - lava_callback - INFO - Received callback for node 69408cab575bc060ce15f651 INFO: 172.21.0.1:57362 - "POST /node/69408cab575bc060ce15f651 HTTP/1.0" 202 Accepted 2025-12-17 22:33:28,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aba575bc060ce15ef47 2025-12-17 22:33:28,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aba575bc060ce15ef47 HTTP/1.1" 502 150 2025-12-17 22:33:28,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aba575bc060ce15ef47'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:28,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac3575bc060ce15ef6a 2025-12-17 22:33:28,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac3575bc060ce15ef6a HTTP/1.1" 502 150 2025-12-17 22:33:28,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac3575bc060ce15ef6a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:28,813 - lava_callback - INFO - Received callback for node 69408cad575bc060ce15f654 INFO: 172.21.0.1:57378 - "POST /node/69408cad575bc060ce15f654 HTTP/1.0" 202 Accepted 2025-12-17 22:33:28,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab5575bc060ce15ef41 2025-12-17 22:33:28,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab5575bc060ce15ef41 HTTP/1.1" 502 150 2025-12-17 22:33:28,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab5575bc060ce15ef41'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:29,456 - lava_callback - INFO - Received callback for node 69408caf575bc060ce15f657 INFO: 172.21.0.1:57384 - "POST /node/69408caf575bc060ce15f657 HTTP/1.0" 202 Accepted 2025-12-17 22:33:30,090 - lava_callback - INFO - Received callback for node 69408cb1575bc060ce15f65a INFO: 172.21.0.1:46504 - "POST /node/69408cb1575bc060ce15f65a HTTP/1.0" 202 Accepted 2025-12-17 22:33:30,167 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab7575bc060ce15ef44 2025-12-17 22:33:30,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab7575bc060ce15ef44 HTTP/1.1" 502 150 2025-12-17 22:33:30,168 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab7575bc060ce15ef44'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:30,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abb575bc060ce15ef4a 2025-12-17 22:33:30,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abb575bc060ce15ef4a HTTP/1.1" 502 150 2025-12-17 22:33:30,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408abb575bc060ce15ef4a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:30,730 - lava_callback - INFO - Received callback for node 69408cb3575bc060ce15f65f INFO: 172.21.0.1:46508 - "POST /node/69408cb3575bc060ce15f65f HTTP/1.0" 202 Accepted 2025-12-17 22:33:31,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aaa575bc060ce15ef19 2025-12-17 22:33:31,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aaa575bc060ce15ef19 HTTP/1.1" 502 150 2025-12-17 22:33:31,810 - lava_callback - ERROR - Error processing callback for node 69408aaa575bc060ce15ef19: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aaa575bc060ce15ef19 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aaa575bc060ce15ef19 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aaa575bc060ce15ef19 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:31,811 - lava_callback - INFO - Starting async job submit for node 69408ac0575bc060ce15ef5c 2025-12-17 22:33:31,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:31,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac0575bc060ce15ef5c HTTP/1.1" 502 150 2025-12-17 22:33:31,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac0575bc060ce15ef5c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:31,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac0575bc060ce15ef5c 2025-12-17 22:33:31,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac0575bc060ce15ef5c HTTP/1.1" 502 150 2025-12-17 22:33:31,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac0575bc060ce15ef5c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:32,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aba575bc060ce15ef47 2025-12-17 22:33:32,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9d575bc060ce15ef00 HTTP/1.1" 502 150 2025-12-17 22:33:32,592 - lava_callback - ERROR - Error processing callback for node 69408a9d575bc060ce15ef00: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a9d575bc060ce15ef00 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a9d575bc060ce15ef00 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a9d575bc060ce15ef00 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:32,594 - lava_callback - INFO - Starting async job submit for node 69408ac4575bc060ce15ef6d 2025-12-17 22:33:32,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:32,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac4575bc060ce15ef6d HTTP/1.1" 502 150 2025-12-17 22:33:32,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac4575bc060ce15ef6d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:32,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac4575bc060ce15ef6d 2025-12-17 22:33:32,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac4575bc060ce15ef6d HTTP/1.1" 502 150 2025-12-17 22:33:32,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac4575bc060ce15ef6d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:33,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab0575bc060ce15ef30 2025-12-17 22:33:33,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab0575bc060ce15ef30 HTTP/1.1" 502 150 2025-12-17 22:33:33,596 - lava_callback - ERROR - Error processing callback for node 69408ab0575bc060ce15ef30: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ab0575bc060ce15ef30 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ab0575bc060ce15ef30 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ab0575bc060ce15ef30 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:33,598 - lava_callback - INFO - Starting async job submit for node 69408ac5575bc060ce15ef70 2025-12-17 22:33:33,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:33,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac5575bc060ce15ef70 HTTP/1.1" 502 150 2025-12-17 22:33:33,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac5575bc060ce15ef70'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:33,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac5575bc060ce15ef70 2025-12-17 22:33:33,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac5575bc060ce15ef70 HTTP/1.1" 502 150 2025-12-17 22:33:33,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac5575bc060ce15ef70'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:33,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac0575bc060ce15ef5c 2025-12-17 22:33:33,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac0575bc060ce15ef5c HTTP/1.1" 502 150 2025-12-17 22:33:33,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac0575bc060ce15ef5c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:34,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac4575bc060ce15ef6d 2025-12-17 22:33:34,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac4575bc060ce15ef6d HTTP/1.1" 502 150 2025-12-17 22:33:34,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac4575bc060ce15ef6d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:34,695 - lava_callback - INFO - Received callback for node 69408c31575bc060ce15f4a6 INFO: 172.21.0.1:46522 - "POST /node/69408c31575bc060ce15f4a6 HTTP/1.0" 202 Accepted 2025-12-17 22:33:34,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac2575bc060ce15ef63 2025-12-17 22:33:34,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac2575bc060ce15ef63 HTTP/1.1" 502 150 2025-12-17 22:33:34,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac2575bc060ce15ef63'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:34,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab3575bc060ce15ef3a 2025-12-17 22:33:34,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab3575bc060ce15ef3a HTTP/1.1" 502 150 2025-12-17 22:33:34,733 - lava_callback - ERROR - Error processing callback for node 69408ab3575bc060ce15ef3a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ab3575bc060ce15ef3a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ab3575bc060ce15ef3a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ab3575bc060ce15ef3a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:34,734 - lava_callback - INFO - Starting async job submit for node 69408ac7575bc060ce15ef73 2025-12-17 22:33:34,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:34,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac7575bc060ce15ef73 HTTP/1.1" 502 150 2025-12-17 22:33:34,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac7575bc060ce15ef73'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:34,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac7575bc060ce15ef73 2025-12-17 22:33:34,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac7575bc060ce15ef73 HTTP/1.1" 502 150 2025-12-17 22:33:34,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac7575bc060ce15ef73'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:34,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aac575bc060ce15ef1c 2025-12-17 22:33:34,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aac575bc060ce15ef1c HTTP/1.1" 502 150 2025-12-17 22:33:34,841 - lava_callback - ERROR - Error processing callback for node 69408aac575bc060ce15ef1c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aac575bc060ce15ef1c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aac575bc060ce15ef1c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aac575bc060ce15ef1c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:34,842 - lava_callback - INFO - Starting async job submit for node 69408ac8575bc060ce15ef76 2025-12-17 22:33:34,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:34,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac8575bc060ce15ef76 HTTP/1.1" 502 150 2025-12-17 22:33:34,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac8575bc060ce15ef76'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:34,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac8575bc060ce15ef76 2025-12-17 22:33:34,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac8575bc060ce15ef76 HTTP/1.1" 502 150 2025-12-17 22:33:34,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac8575bc060ce15ef76'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:34,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a9f575bc060ce15ef04 2025-12-17 22:33:34,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a9f575bc060ce15ef04 HTTP/1.1" 502 150 2025-12-17 22:33:34,997 - lava_callback - ERROR - Error processing callback for node 69408a9f575bc060ce15ef04: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a9f575bc060ce15ef04 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a9f575bc060ce15ef04 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a9f575bc060ce15ef04 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:35,000 - lava_callback - INFO - Starting async job submit for node 69408ac9575bc060ce15ef79 2025-12-17 22:33:35,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:35,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac9575bc060ce15ef79 HTTP/1.1" 502 150 2025-12-17 22:33:35,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac9575bc060ce15ef79'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:35,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac9575bc060ce15ef79 2025-12-17 22:33:35,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac9575bc060ce15ef79 HTTP/1.1" 502 150 2025-12-17 22:33:35,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac9575bc060ce15ef79'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:35,093 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abc575bc060ce15ef4d 2025-12-17 22:33:35,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abc575bc060ce15ef4d HTTP/1.1" 502 150 2025-12-17 22:33:35,094 - lava_callback - ERROR - Error processing callback for node 69408abc575bc060ce15ef4d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408abc575bc060ce15ef4d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408abc575bc060ce15ef4d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408abc575bc060ce15ef4d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:35,096 - lava_callback - INFO - Starting async job submit for node 69408aca575bc060ce15ef7c 2025-12-17 22:33:35,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:35,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aca575bc060ce15ef7c HTTP/1.1" 502 150 2025-12-17 22:33:35,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aca575bc060ce15ef7c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:35,113 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aca575bc060ce15ef7c 2025-12-17 22:33:35,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aca575bc060ce15ef7c HTTP/1.1" 502 150 2025-12-17 22:33:35,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aca575bc060ce15ef7c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:35,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac5575bc060ce15ef70 2025-12-17 22:33:35,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac5575bc060ce15ef70 HTTP/1.1" 502 150 2025-12-17 22:33:35,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac5575bc060ce15ef70'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46530 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:33:36,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac3575bc060ce15ef6a 2025-12-17 22:33:36,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac3575bc060ce15ef6a HTTP/1.1" 502 150 2025-12-17 22:33:36,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac3575bc060ce15ef6a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:36,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac7575bc060ce15ef73 2025-12-17 22:33:36,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac7575bc060ce15ef73 HTTP/1.1" 502 150 2025-12-17 22:33:36,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac7575bc060ce15ef73'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:36,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac8575bc060ce15ef76 2025-12-17 22:33:36,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac8575bc060ce15ef76 HTTP/1.1" 502 150 2025-12-17 22:33:36,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac8575bc060ce15ef76'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:36,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab5575bc060ce15ef41 2025-12-17 22:33:36,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab5575bc060ce15ef41 HTTP/1.1" 502 150 2025-12-17 22:33:36,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab5575bc060ce15ef41'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:37,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac9575bc060ce15ef79 2025-12-17 22:33:37,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac9575bc060ce15ef79 HTTP/1.1" 502 150 2025-12-17 22:33:37,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac9575bc060ce15ef79'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:37,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aca575bc060ce15ef7c 2025-12-17 22:33:37,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aca575bc060ce15ef7c HTTP/1.1" 502 150 2025-12-17 22:33:37,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aca575bc060ce15ef7c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:37,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac0575bc060ce15ef5c 2025-12-17 22:33:37,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac0575bc060ce15ef5c HTTP/1.1" 502 150 2025-12-17 22:33:37,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac0575bc060ce15ef5c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:38,169 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab7575bc060ce15ef44 2025-12-17 22:33:38,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab7575bc060ce15ef44 HTTP/1.1" 502 150 2025-12-17 22:33:38,170 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ab7575bc060ce15ef44'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:38,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac4575bc060ce15ef6d 2025-12-17 22:33:38,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac4575bc060ce15ef6d HTTP/1.1" 502 150 2025-12-17 22:33:38,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac4575bc060ce15ef6d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:39,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac5575bc060ce15ef70 2025-12-17 22:33:39,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac5575bc060ce15ef70 HTTP/1.1" 502 150 2025-12-17 22:33:39,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac5575bc060ce15ef70'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:40,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac7575bc060ce15ef73 2025-12-17 22:33:40,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac7575bc060ce15ef73 HTTP/1.1" 502 150 2025-12-17 22:33:40,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac7575bc060ce15ef73'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:40,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a55575bc060ce15ee1f HTTP/1.1" 502 150 2025-12-17 22:33:40,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a55575bc060ce15ee1f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:40,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac8575bc060ce15ef76 2025-12-17 22:33:40,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac8575bc060ce15ef76 HTTP/1.1" 502 150 2025-12-17 22:33:40,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac8575bc060ce15ef76'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:41,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac9575bc060ce15ef79 2025-12-17 22:33:41,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac9575bc060ce15ef79 HTTP/1.1" 502 150 2025-12-17 22:33:41,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac9575bc060ce15ef79'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:41,116 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aca575bc060ce15ef7c 2025-12-17 22:33:41,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aca575bc060ce15ef7c HTTP/1.1" 502 150 2025-12-17 22:33:41,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aca575bc060ce15ef7c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:42,046 - lava_callback - INFO - Received callback for node 69408c94575bc060ce15f61e INFO: 172.21.0.1:37358 - "POST /node/69408c94575bc060ce15f61e HTTP/1.0" 202 Accepted 2025-12-17 22:33:43,353 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abe575bc060ce15ef50 2025-12-17 22:33:43,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abf575bc060ce15ef55 2025-12-17 22:33:43,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abf575bc060ce15ef55 HTTP/1.1" 502 150 2025-12-17 22:33:43,478 - lava_callback - ERROR - Error processing callback for node 69408abf575bc060ce15ef55: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408abf575bc060ce15ef55 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408abf575bc060ce15ef55 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408abf575bc060ce15ef55 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:43,480 - lava_callback - INFO - Starting async job submit for node 69408acc575bc060ce15ef7f 2025-12-17 22:33:43,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:43,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acc575bc060ce15ef7f HTTP/1.1" 502 150 2025-12-17 22:33:43,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408acc575bc060ce15ef7f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:43,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408acc575bc060ce15ef7f 2025-12-17 22:33:43,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acc575bc060ce15ef7f HTTP/1.1" 502 150 2025-12-17 22:33:43,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408acc575bc060ce15ef7f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:44,968 - lava_callback - INFO - Received callback for node 69408cb6575bc060ce15f666 INFO: 172.21.0.1:37372 - "POST /node/69408cb6575bc060ce15f666 HTTP/1.0" 202 Accepted 2025-12-17 22:33:45,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408acc575bc060ce15ef7f 2025-12-17 22:33:45,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acc575bc060ce15ef7f HTTP/1.1" 502 150 2025-12-17 22:33:45,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408acc575bc060ce15ef7f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:45,605 - lava_callback - INFO - Received callback for node 69408cb8575bc060ce15f669 INFO: 172.21.0.1:37378 - "POST /node/69408cb8575bc060ce15f669 HTTP/1.0" 202 Accepted 2025-12-17 22:33:45,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac0575bc060ce15ef5c 2025-12-17 22:33:45,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac0575bc060ce15ef5c HTTP/1.1" 502 150 2025-12-17 22:33:45,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac0575bc060ce15ef5c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:46,268 - lava_callback - INFO - Received callback for node 69408cba575bc060ce15f66c INFO: 172.21.0.1:37386 - "POST /node/69408cba575bc060ce15f66c HTTP/1.0" 202 Accepted INFO: 172.21.0.1:37398 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:33:46,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408abb575bc060ce15ef4a 2025-12-17 22:33:46,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abb575bc060ce15ef4a HTTP/1.1" 502 150 2025-12-17 22:33:46,576 - lava_callback - ERROR - Error processing callback for node 69408abb575bc060ce15ef4a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408abb575bc060ce15ef4a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408abb575bc060ce15ef4a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408abb575bc060ce15ef4a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:46,578 - lava_callback - INFO - Starting async job submit for node 69408acd575bc060ce15ef82 2025-12-17 22:33:46,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:46,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac4575bc060ce15ef6d 2025-12-17 22:33:46,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac4575bc060ce15ef6d HTTP/1.1" 502 150 2025-12-17 22:33:46,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac4575bc060ce15ef6d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:46,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acd575bc060ce15ef82 HTTP/1.1" 502 150 2025-12-17 22:33:46,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408acd575bc060ce15ef82'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:46,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408acd575bc060ce15ef82 2025-12-17 22:33:46,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acd575bc060ce15ef82 HTTP/1.1" 502 150 2025-12-17 22:33:46,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408acd575bc060ce15ef82'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:46,901 - lava_callback - INFO - Received callback for node 69408cbc575bc060ce15f66f INFO: 172.21.0.1:37410 - "POST /node/69408cbc575bc060ce15f66f HTTP/1.0" 202 Accepted 2025-12-17 22:33:47,540 - lava_callback - INFO - Received callback for node 69408cbe575bc060ce15f68e INFO: 172.21.0.1:37414 - "POST /node/69408cbe575bc060ce15f68e HTTP/1.0" 202 Accepted 2025-12-17 22:33:47,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac5575bc060ce15ef70 2025-12-17 22:33:47,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac5575bc060ce15ef70 HTTP/1.1" 502 150 2025-12-17 22:33:47,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac5575bc060ce15ef70'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:47,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aba575bc060ce15ef47 HTTP/1.1" 502 150 2025-12-17 22:33:47,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aba575bc060ce15ef47'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:48,202 - lava_callback - INFO - Received callback for node 69408cc1575bc060ce15f691 INFO: 172.21.0.1:37428 - "POST /node/69408cc1575bc060ce15f691 HTTP/1.0" 202 Accepted 2025-12-17 22:33:48,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408acd575bc060ce15ef82 2025-12-17 22:33:48,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acd575bc060ce15ef82 HTTP/1.1" 502 150 2025-12-17 22:33:48,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408acd575bc060ce15ef82'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:48,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac7575bc060ce15ef73 2025-12-17 22:33:48,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac7575bc060ce15ef73 HTTP/1.1" 502 150 2025-12-17 22:33:48,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac7575bc060ce15ef73'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:48,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a55575bc060ce15ee1f 2025-12-17 22:33:48,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac8575bc060ce15ef76 2025-12-17 22:33:48,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac8575bc060ce15ef76 HTTP/1.1" 502 150 2025-12-17 22:33:48,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac8575bc060ce15ef76'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:48,870 - lava_callback - INFO - Received callback for node 69408cc3575bc060ce15f699 INFO: 172.21.0.1:37430 - "POST /node/69408cc3575bc060ce15f699 HTTP/1.0" 202 Accepted 2025-12-17 22:33:49,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac9575bc060ce15ef79 2025-12-17 22:33:49,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac9575bc060ce15ef79 HTTP/1.1" 502 150 2025-12-17 22:33:49,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ac9575bc060ce15ef79'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:49,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aca575bc060ce15ef7c 2025-12-17 22:33:49,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aca575bc060ce15ef7c HTTP/1.1" 502 150 2025-12-17 22:33:49,118 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aca575bc060ce15ef7c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:49,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408acc575bc060ce15ef7f 2025-12-17 22:33:49,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acc575bc060ce15ef7f HTTP/1.1" 502 150 2025-12-17 22:33:49,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408acc575bc060ce15ef7f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:49,536 - lava_callback - INFO - Received callback for node 69408cc5575bc060ce15f69c INFO: 172.21.0.1:37438 - "POST /node/69408cc5575bc060ce15f69c HTTP/1.0" 202 Accepted 2025-12-17 22:33:50,205 - lava_callback - INFO - Received callback for node 69408cc6575bc060ce15f69f INFO: 172.21.0.1:54908 - "POST /node/69408cc6575bc060ce15f69f HTTP/1.0" 202 Accepted 2025-12-17 22:33:50,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac2575bc060ce15ef63 2025-12-17 22:33:50,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac2575bc060ce15ef63 HTTP/1.1" 502 150 2025-12-17 22:33:50,717 - lava_callback - ERROR - Error processing callback for node 69408ac2575bc060ce15ef63: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac2575bc060ce15ef63 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac2575bc060ce15ef63 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac2575bc060ce15ef63 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:50,720 - lava_callback - INFO - Starting async job submit for node 69408acf575bc060ce15efba 2025-12-17 22:33:50,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:50,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acf575bc060ce15efba HTTP/1.1" 502 150 2025-12-17 22:33:50,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408acf575bc060ce15efba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:50,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408acf575bc060ce15efba 2025-12-17 22:33:50,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acf575bc060ce15efba HTTP/1.1" 502 150 2025-12-17 22:33:50,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408acf575bc060ce15efba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:52,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac3575bc060ce15ef6a 2025-12-17 22:33:52,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac3575bc060ce15ef6a HTTP/1.1" 502 150 2025-12-17 22:33:52,608 - lava_callback - ERROR - Error processing callback for node 69408ac3575bc060ce15ef6a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac3575bc060ce15ef6a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac3575bc060ce15ef6a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac3575bc060ce15ef6a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:52,611 - lava_callback - INFO - Starting async job submit for node 69408ad1575bc060ce15efd5 2025-12-17 22:33:52,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:52,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408acd575bc060ce15ef82 2025-12-17 22:33:52,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acd575bc060ce15ef82 HTTP/1.1" 502 150 2025-12-17 22:33:52,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408acd575bc060ce15ef82'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:52,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad1575bc060ce15efd5 HTTP/1.1" 502 150 2025-12-17 22:33:52,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad1575bc060ce15efd5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:52,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad1575bc060ce15efd5 2025-12-17 22:33:52,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad1575bc060ce15efd5 HTTP/1.1" 502 150 2025-12-17 22:33:52,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad1575bc060ce15efd5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:52,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408acf575bc060ce15efba 2025-12-17 22:33:52,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acf575bc060ce15efba HTTP/1.1" 502 150 2025-12-17 22:33:52,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408acf575bc060ce15efba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:52,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab5575bc060ce15ef41 2025-12-17 22:33:52,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab5575bc060ce15ef41 HTTP/1.1" 502 150 2025-12-17 22:33:52,895 - lava_callback - ERROR - Error processing callback for node 69408ab5575bc060ce15ef41: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ab5575bc060ce15ef41 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ab5575bc060ce15ef41 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ab5575bc060ce15ef41 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:52,897 - lava_callback - INFO - Starting async job submit for node 69408ad2575bc060ce15efd8 2025-12-17 22:33:52,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:52,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad2575bc060ce15efd8 HTTP/1.1" 502 150 2025-12-17 22:33:52,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad2575bc060ce15efd8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:52,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad2575bc060ce15efd8 2025-12-17 22:33:52,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad2575bc060ce15efd8 HTTP/1.1" 502 150 2025-12-17 22:33:52,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad2575bc060ce15efd8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:54,170 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ab7575bc060ce15ef44 2025-12-17 22:33:54,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ab7575bc060ce15ef44 HTTP/1.1" 502 150 2025-12-17 22:33:54,172 - lava_callback - ERROR - Error processing callback for node 69408ab7575bc060ce15ef44: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ab7575bc060ce15ef44 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ab7575bc060ce15ef44 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ab7575bc060ce15ef44 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:54,174 - lava_callback - INFO - Starting async job submit for node 69408fd6575bc060ce160272 2025-12-17 22:33:54,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:54,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd6575bc060ce160272 HTTP/1.1" 502 150 2025-12-17 22:33:54,239 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fd6575bc060ce160272'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:54,239 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fd6575bc060ce160272 2025-12-17 22:33:54,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd6575bc060ce160272 HTTP/1.1" 502 150 2025-12-17 22:33:54,239 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fd6575bc060ce160272'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:54,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad1575bc060ce15efd5 2025-12-17 22:33:54,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad1575bc060ce15efd5 HTTP/1.1" 502 150 2025-12-17 22:33:54,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad1575bc060ce15efd5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:54,817 - lava_callback - INFO - Received callback for node 69408cd0575bc060ce15f6b3 INFO: 172.21.0.1:54922 - "POST /node/69408cd0575bc060ce15f6b3 HTTP/1.0" 202 Accepted 2025-12-17 22:33:54,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad2575bc060ce15efd8 2025-12-17 22:33:54,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad2575bc060ce15efd8 HTTP/1.1" 502 150 2025-12-17 22:33:54,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad2575bc060ce15efd8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:55,462 - lava_callback - INFO - Received callback for node 69408ccd575bc060ce15f6b0 INFO: 172.21.0.1:54930 - "POST /node/69408ccd575bc060ce15f6b0 HTTP/1.0" 202 Accepted 2025-12-17 22:33:55,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aba575bc060ce15ef47 2025-12-17 22:33:56,240 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fd6575bc060ce160272 2025-12-17 22:33:56,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd6575bc060ce160272 HTTP/1.1" 502 150 2025-12-17 22:33:56,241 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fd6575bc060ce160272'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54934 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:33:56,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408acf575bc060ce15efba 2025-12-17 22:33:56,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acf575bc060ce15efba HTTP/1.1" 502 150 2025-12-17 22:33:56,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408acf575bc060ce15efba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:57,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408acc575bc060ce15ef7f 2025-12-17 22:33:57,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acc575bc060ce15ef7f HTTP/1.1" 502 150 2025-12-17 22:33:57,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408acc575bc060ce15ef7f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:58,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad1575bc060ce15efd5 2025-12-17 22:33:58,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad1575bc060ce15efd5 HTTP/1.1" 502 150 2025-12-17 22:33:58,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad1575bc060ce15efd5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:58,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408abe575bc060ce15ef50 HTTP/1.1" 502 150 2025-12-17 22:33:58,704 - lava_callback - ERROR - Error processing callback for node 69408abe575bc060ce15ef50: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408abe575bc060ce15ef50 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408abe575bc060ce15ef50 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408abe575bc060ce15ef50 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:33:58,706 - lava_callback - INFO - Starting async job submit for node 69408926575bc060ce15e98f 2025-12-17 22:33:58,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:33:58,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408926575bc060ce15e98f HTTP/1.1" 502 150 2025-12-17 22:33:58,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408926575bc060ce15e98f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:58,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408926575bc060ce15e98f 2025-12-17 22:33:58,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408926575bc060ce15e98f HTTP/1.1" 502 150 2025-12-17 22:33:58,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408926575bc060ce15e98f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:33:58,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad2575bc060ce15efd8 2025-12-17 22:33:58,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad2575bc060ce15efd8 HTTP/1.1" 502 150 2025-12-17 22:33:58,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad2575bc060ce15efd8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:00,241 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fd6575bc060ce160272 2025-12-17 22:34:00,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd6575bc060ce160272 HTTP/1.1" 502 150 2025-12-17 22:34:00,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fd6575bc060ce160272'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:00,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408acd575bc060ce15ef82 2025-12-17 22:34:00,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acd575bc060ce15ef82 HTTP/1.1" 502 150 2025-12-17 22:34:00,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408acd575bc060ce15ef82'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:00,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408926575bc060ce15e98f 2025-12-17 22:34:00,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408926575bc060ce15e98f HTTP/1.1" 502 150 2025-12-17 22:34:00,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408926575bc060ce15e98f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:01,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac0575bc060ce15ef5c 2025-12-17 22:34:01,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac0575bc060ce15ef5c HTTP/1.1" 502 150 2025-12-17 22:34:01,825 - lava_callback - ERROR - Error processing callback for node 69408ac0575bc060ce15ef5c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac0575bc060ce15ef5c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac0575bc060ce15ef5c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac0575bc060ce15ef5c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:01,827 - lava_callback - INFO - Starting async job submit for node 694098c0575bc060ce161b4f 2025-12-17 22:34:01,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:01,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c0575bc060ce161b4f HTTP/1.1" 502 150 2025-12-17 22:34:01,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c0575bc060ce161b4f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:01,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c0575bc060ce161b4f 2025-12-17 22:34:01,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c0575bc060ce161b4f HTTP/1.1" 502 150 2025-12-17 22:34:01,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c0575bc060ce161b4f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52324 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:34:02,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac4575bc060ce15ef6d 2025-12-17 22:34:02,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac4575bc060ce15ef6d HTTP/1.1" 502 150 2025-12-17 22:34:02,608 - lava_callback - ERROR - Error processing callback for node 69408ac4575bc060ce15ef6d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac4575bc060ce15ef6d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac4575bc060ce15ef6d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac4575bc060ce15ef6d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:02,610 - lava_callback - INFO - Starting async job submit for node 69408ad4575bc060ce15efdf 2025-12-17 22:34:02,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:02,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad4575bc060ce15efdf HTTP/1.1" 502 150 2025-12-17 22:34:02,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad4575bc060ce15efdf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:02,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad4575bc060ce15efdf 2025-12-17 22:34:02,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad4575bc060ce15efdf HTTP/1.1" 502 150 2025-12-17 22:34:02,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad4575bc060ce15efdf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:03,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac5575bc060ce15ef70 2025-12-17 22:34:03,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac5575bc060ce15ef70 HTTP/1.1" 502 150 2025-12-17 22:34:03,613 - lava_callback - ERROR - Error processing callback for node 69408ac5575bc060ce15ef70: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac5575bc060ce15ef70 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac5575bc060ce15ef70 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac5575bc060ce15ef70 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:03,615 - lava_callback - INFO - Starting async job submit for node 69408ad5575bc060ce15efe6 2025-12-17 22:34:03,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:03,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad5575bc060ce15efe6 HTTP/1.1" 502 150 2025-12-17 22:34:03,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad5575bc060ce15efe6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:03,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad5575bc060ce15efe6 2025-12-17 22:34:03,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad5575bc060ce15efe6 HTTP/1.1" 502 150 2025-12-17 22:34:03,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad5575bc060ce15efe6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:03,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c0575bc060ce161b4f 2025-12-17 22:34:03,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c0575bc060ce161b4f HTTP/1.1" 502 150 2025-12-17 22:34:03,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c0575bc060ce161b4f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:04,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad4575bc060ce15efdf 2025-12-17 22:34:04,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad4575bc060ce15efdf HTTP/1.1" 502 150 2025-12-17 22:34:04,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad4575bc060ce15efdf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:04,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408926575bc060ce15e98f 2025-12-17 22:34:04,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408926575bc060ce15e98f HTTP/1.1" 502 150 2025-12-17 22:34:04,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408926575bc060ce15e98f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:04,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408acf575bc060ce15efba 2025-12-17 22:34:04,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acf575bc060ce15efba HTTP/1.1" 502 150 2025-12-17 22:34:04,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408acf575bc060ce15efba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:04,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac7575bc060ce15ef73 2025-12-17 22:34:04,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac7575bc060ce15ef73 HTTP/1.1" 502 150 2025-12-17 22:34:04,760 - lava_callback - ERROR - Error processing callback for node 69408ac7575bc060ce15ef73: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac7575bc060ce15ef73 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac7575bc060ce15ef73 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac7575bc060ce15ef73 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:04,762 - lava_callback - INFO - Starting async job submit for node 69408ad6575bc060ce15efed 2025-12-17 22:34:04,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:04,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad6575bc060ce15efed HTTP/1.1" 502 150 2025-12-17 22:34:04,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad6575bc060ce15efed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:04,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad6575bc060ce15efed 2025-12-17 22:34:04,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad6575bc060ce15efed HTTP/1.1" 502 150 2025-12-17 22:34:04,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad6575bc060ce15efed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:04,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac8575bc060ce15ef76 2025-12-17 22:34:04,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac8575bc060ce15ef76 HTTP/1.1" 502 150 2025-12-17 22:34:04,855 - lava_callback - ERROR - Error processing callback for node 69408ac8575bc060ce15ef76: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac8575bc060ce15ef76 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac8575bc060ce15ef76 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac8575bc060ce15ef76 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:04,857 - lava_callback - INFO - Starting async job submit for node 69408ada575bc060ce15f001 2025-12-17 22:34:04,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:04,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ada575bc060ce15f001 HTTP/1.1" 502 150 2025-12-17 22:34:04,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ada575bc060ce15f001'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:04,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ada575bc060ce15f001 2025-12-17 22:34:04,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ada575bc060ce15f001 HTTP/1.1" 502 150 2025-12-17 22:34:04,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ada575bc060ce15f001'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:04,952 - lava_callback - INFO - Received callback for node 69408cc8575bc060ce15f6a4 INFO: 172.21.0.1:52332 - "POST /node/69408cc8575bc060ce15f6a4 HTTP/1.0" 202 Accepted 2025-12-17 22:34:05,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ac9575bc060ce15ef79 2025-12-17 22:34:05,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ac9575bc060ce15ef79 HTTP/1.1" 502 150 2025-12-17 22:34:05,030 - lava_callback - ERROR - Error processing callback for node 69408ac9575bc060ce15ef79: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac9575bc060ce15ef79 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac9575bc060ce15ef79 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ac9575bc060ce15ef79 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:05,032 - lava_callback - INFO - Starting async job submit for node 69408adc575bc060ce15f004 2025-12-17 22:34:05,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:05,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408adc575bc060ce15f004 HTTP/1.1" 502 150 2025-12-17 22:34:05,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408adc575bc060ce15f004'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:05,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408adc575bc060ce15f004 2025-12-17 22:34:05,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408adc575bc060ce15f004 HTTP/1.1" 502 150 2025-12-17 22:34:05,039 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408adc575bc060ce15f004'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:05,118 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aca575bc060ce15ef7c 2025-12-17 22:34:05,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aca575bc060ce15ef7c HTTP/1.1" 502 150 2025-12-17 22:34:05,120 - lava_callback - ERROR - Error processing callback for node 69408aca575bc060ce15ef7c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aca575bc060ce15ef7c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aca575bc060ce15ef7c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aca575bc060ce15ef7c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:05,123 - lava_callback - INFO - Starting async job submit for node 69408ade575bc060ce15f007 2025-12-17 22:34:05,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:05,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ade575bc060ce15f007 HTTP/1.1" 502 150 2025-12-17 22:34:05,132 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ade575bc060ce15f007'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:05,132 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ade575bc060ce15f007 2025-12-17 22:34:05,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ade575bc060ce15f007 HTTP/1.1" 502 150 2025-12-17 22:34:05,133 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ade575bc060ce15f007'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:05,600 - lava_callback - INFO - Received callback for node 69408cca575bc060ce15f6ad INFO: 172.21.0.1:52338 - "POST /node/69408cca575bc060ce15f6ad HTTP/1.0" 202 Accepted 2025-12-17 22:34:05,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad5575bc060ce15efe6 2025-12-17 22:34:05,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad5575bc060ce15efe6 HTTP/1.1" 502 150 2025-12-17 22:34:05,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad5575bc060ce15efe6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52354 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:34:06,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad1575bc060ce15efd5 2025-12-17 22:34:06,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad1575bc060ce15efd5 HTTP/1.1" 502 150 2025-12-17 22:34:06,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad1575bc060ce15efd5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:06,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad6575bc060ce15efed 2025-12-17 22:34:06,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad6575bc060ce15efed HTTP/1.1" 502 150 2025-12-17 22:34:06,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad6575bc060ce15efed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:06,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ada575bc060ce15f001 2025-12-17 22:34:06,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ada575bc060ce15f001 HTTP/1.1" 502 150 2025-12-17 22:34:06,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ada575bc060ce15f001'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:06,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad2575bc060ce15efd8 2025-12-17 22:34:06,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad2575bc060ce15efd8 HTTP/1.1" 502 150 2025-12-17 22:34:06,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad2575bc060ce15efd8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:07,039 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408adc575bc060ce15f004 2025-12-17 22:34:07,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408adc575bc060ce15f004 HTTP/1.1" 502 150 2025-12-17 22:34:07,041 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408adc575bc060ce15f004'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:07,133 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ade575bc060ce15f007 2025-12-17 22:34:07,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ade575bc060ce15f007 HTTP/1.1" 502 150 2025-12-17 22:34:07,135 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ade575bc060ce15f007'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:07,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c0575bc060ce161b4f 2025-12-17 22:34:07,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c0575bc060ce161b4f HTTP/1.1" 502 150 2025-12-17 22:34:07,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c0575bc060ce161b4f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:08,243 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fd6575bc060ce160272 2025-12-17 22:34:08,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd6575bc060ce160272 HTTP/1.1" 502 150 2025-12-17 22:34:08,244 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fd6575bc060ce160272'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:08,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad4575bc060ce15efdf 2025-12-17 22:34:08,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad4575bc060ce15efdf HTTP/1.1" 502 150 2025-12-17 22:34:08,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad4575bc060ce15efdf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:09,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad5575bc060ce15efe6 2025-12-17 22:34:09,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad5575bc060ce15efe6 HTTP/1.1" 502 150 2025-12-17 22:34:09,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad5575bc060ce15efe6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:10,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad6575bc060ce15efed 2025-12-17 22:34:10,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad6575bc060ce15efed HTTP/1.1" 502 150 2025-12-17 22:34:10,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad6575bc060ce15efed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:10,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ada575bc060ce15f001 2025-12-17 22:34:10,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ada575bc060ce15f001 HTTP/1.1" 502 150 2025-12-17 22:34:10,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ada575bc060ce15f001'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:10,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aba575bc060ce15ef47 HTTP/1.1" 502 150 2025-12-17 22:34:10,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aba575bc060ce15ef47'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:11,041 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408adc575bc060ce15f004 2025-12-17 22:34:11,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408adc575bc060ce15f004 HTTP/1.1" 502 150 2025-12-17 22:34:11,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408adc575bc060ce15f004'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:11,135 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ade575bc060ce15f007 2025-12-17 22:34:11,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ade575bc060ce15f007 HTTP/1.1" 502 150 2025-12-17 22:34:11,136 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ade575bc060ce15f007'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:12,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408926575bc060ce15e98f 2025-12-17 22:34:12,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408926575bc060ce15e98f HTTP/1.1" 502 150 2025-12-17 22:34:12,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408926575bc060ce15e98f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:13,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408acc575bc060ce15ef7f 2025-12-17 22:34:13,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acc575bc060ce15ef7f HTTP/1.1" 502 150 2025-12-17 22:34:13,495 - lava_callback - ERROR - Error processing callback for node 69408acc575bc060ce15ef7f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408acc575bc060ce15ef7f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408acc575bc060ce15ef7f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408acc575bc060ce15ef7f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:13,496 - lava_callback - INFO - Starting async job submit for node 69408ae0575bc060ce15f00a 2025-12-17 22:34:13,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:13,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae0575bc060ce15f00a HTTP/1.1" 502 150 2025-12-17 22:34:13,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae0575bc060ce15f00a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:13,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae0575bc060ce15f00a 2025-12-17 22:34:13,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae0575bc060ce15f00a HTTP/1.1" 502 150 2025-12-17 22:34:13,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae0575bc060ce15f00a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:14,816 - lava_callback - INFO - Received callback for node 69408cd2575bc060ce15f6b6 INFO: 172.21.0.1:49820 - "POST /node/69408cd2575bc060ce15f6b6 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:49834 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:34:15,466 - lava_callback - INFO - Received callback for node 69408cd4575bc060ce15f6b9 INFO: 172.21.0.1:49840 - "POST /node/69408cd4575bc060ce15f6b9 HTTP/1.0" 202 Accepted 2025-12-17 22:34:15,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae0575bc060ce15f00a 2025-12-17 22:34:15,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae0575bc060ce15f00a HTTP/1.1" 502 150 2025-12-17 22:34:15,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae0575bc060ce15f00a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:15,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c0575bc060ce161b4f 2025-12-17 22:34:15,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c0575bc060ce161b4f HTTP/1.1" 502 150 2025-12-17 22:34:15,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c0575bc060ce161b4f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:16,145 - lava_callback - INFO - Received callback for node 69408cd6575bc060ce15f6bc INFO: 172.21.0.1:49846 - "POST /node/69408cd6575bc060ce15f6bc HTTP/1.0" 202 Accepted INFO: 172.21.0.1:49854 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:34:16,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408acd575bc060ce15ef82 2025-12-17 22:34:16,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acd575bc060ce15ef82 HTTP/1.1" 502 150 2025-12-17 22:34:16,617 - lava_callback - ERROR - Error processing callback for node 69408acd575bc060ce15ef82: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408acd575bc060ce15ef82 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408acd575bc060ce15ef82 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408acd575bc060ce15ef82 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:16,619 - lava_callback - INFO - Starting async job submit for node 69408ae2575bc060ce15f00d 2025-12-17 22:34:16,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:16,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad4575bc060ce15efdf 2025-12-17 22:34:16,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad4575bc060ce15efdf HTTP/1.1" 502 150 2025-12-17 22:34:16,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad4575bc060ce15efdf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:16,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae2575bc060ce15f00d HTTP/1.1" 502 150 2025-12-17 22:34:16,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae2575bc060ce15f00d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:16,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae2575bc060ce15f00d 2025-12-17 22:34:16,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae2575bc060ce15f00d HTTP/1.1" 502 150 2025-12-17 22:34:16,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae2575bc060ce15f00d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:16,828 - lava_callback - INFO - Received callback for node 69408cd8575bc060ce15f6bf INFO: 172.21.0.1:49866 - "POST /node/69408cd8575bc060ce15f6bf HTTP/1.0" 202 Accepted 2025-12-17 22:34:17,484 - lava_callback - INFO - Received callback for node 69408cda575bc060ce15f6c2 INFO: 172.21.0.1:49876 - "POST /node/69408cda575bc060ce15f6c2 HTTP/1.0" 202 Accepted 2025-12-17 22:34:17,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad5575bc060ce15efe6 2025-12-17 22:34:17,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad5575bc060ce15efe6 HTTP/1.1" 502 150 2025-12-17 22:34:17,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad5575bc060ce15efe6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:18,125 - lava_callback - INFO - Received callback for node 69408cdc575bc060ce15f6c5 INFO: 172.21.0.1:49882 - "POST /node/69408cdc575bc060ce15f6c5 HTTP/1.0" 202 Accepted 2025-12-17 22:34:18,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae2575bc060ce15f00d 2025-12-17 22:34:18,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae2575bc060ce15f00d HTTP/1.1" 502 150 2025-12-17 22:34:18,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae2575bc060ce15f00d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:18,769 - lava_callback - INFO - Received callback for node 69408cde575bc060ce15f6ce INFO: 172.21.0.1:49892 - "POST /node/69408cde575bc060ce15f6ce HTTP/1.0" 202 Accepted 2025-12-17 22:34:18,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad6575bc060ce15efed 2025-12-17 22:34:18,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad6575bc060ce15efed HTTP/1.1" 502 150 2025-12-17 22:34:18,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ad6575bc060ce15efed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:18,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ada575bc060ce15f001 2025-12-17 22:34:18,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ada575bc060ce15f001 HTTP/1.1" 502 150 2025-12-17 22:34:18,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ada575bc060ce15f001'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:19,042 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408adc575bc060ce15f004 2025-12-17 22:34:19,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408adc575bc060ce15f004 HTTP/1.1" 502 150 2025-12-17 22:34:19,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408adc575bc060ce15f004'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:19,136 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ade575bc060ce15f007 2025-12-17 22:34:19,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ade575bc060ce15f007 HTTP/1.1" 502 150 2025-12-17 22:34:19,137 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ade575bc060ce15f007'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:19,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a55575bc060ce15ee1f HTTP/1.1" 502 150 2025-12-17 22:34:19,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408a55575bc060ce15ee1f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:19,443 - lava_callback - INFO - Received callback for node 69408ce0575bc060ce15f6d5 INFO: 172.21.0.1:49908 - "POST /node/69408ce0575bc060ce15f6d5 HTTP/1.0" 202 Accepted 2025-12-17 22:34:19,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae0575bc060ce15f00a 2025-12-17 22:34:19,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae0575bc060ce15f00a HTTP/1.1" 502 150 2025-12-17 22:34:19,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae0575bc060ce15f00a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:20,097 - lava_callback - INFO - Received callback for node 69408ce2575bc060ce15f6d8 INFO: 172.21.0.1:51348 - "POST /node/69408ce2575bc060ce15f6d8 HTTP/1.0" 202 Accepted 2025-12-17 22:34:20,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408acf575bc060ce15efba 2025-12-17 22:34:20,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408acf575bc060ce15efba HTTP/1.1" 502 150 2025-12-17 22:34:20,747 - lava_callback - ERROR - Error processing callback for node 69408acf575bc060ce15efba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408acf575bc060ce15efba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408acf575bc060ce15efba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408acf575bc060ce15efba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:20,749 - lava_callback - INFO - Starting async job submit for node 69408ae4575bc060ce15f010 2025-12-17 22:34:20,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:20,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae4575bc060ce15f010 HTTP/1.1" 502 150 2025-12-17 22:34:20,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae4575bc060ce15f010'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:20,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae4575bc060ce15f010 2025-12-17 22:34:20,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae4575bc060ce15f010 HTTP/1.1" 502 150 2025-12-17 22:34:20,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae4575bc060ce15f010'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:22,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad1575bc060ce15efd5 2025-12-17 22:34:22,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad1575bc060ce15efd5 HTTP/1.1" 502 150 2025-12-17 22:34:22,628 - lava_callback - ERROR - Error processing callback for node 69408ad1575bc060ce15efd5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ad1575bc060ce15efd5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ad1575bc060ce15efd5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ad1575bc060ce15efd5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:22,631 - lava_callback - INFO - Starting async job submit for node 69408ae6575bc060ce15f013 2025-12-17 22:34:22,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:22,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae6575bc060ce15f013 HTTP/1.1" 502 150 2025-12-17 22:34:22,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae6575bc060ce15f013'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:22,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae6575bc060ce15f013 2025-12-17 22:34:22,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae2575bc060ce15f00d 2025-12-17 22:34:22,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae6575bc060ce15f013 HTTP/1.1" 502 150 2025-12-17 22:34:22,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae6575bc060ce15f013'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:22,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae2575bc060ce15f00d HTTP/1.1" 502 150 2025-12-17 22:34:22,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae2575bc060ce15f00d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:22,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae4575bc060ce15f010 2025-12-17 22:34:22,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae4575bc060ce15f010 HTTP/1.1" 502 150 2025-12-17 22:34:22,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae4575bc060ce15f010'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:22,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad2575bc060ce15efd8 2025-12-17 22:34:22,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad2575bc060ce15efd8 HTTP/1.1" 502 150 2025-12-17 22:34:22,911 - lava_callback - ERROR - Error processing callback for node 69408ad2575bc060ce15efd8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ad2575bc060ce15efd8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ad2575bc060ce15efd8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ad2575bc060ce15efd8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:22,914 - lava_callback - INFO - Starting async job submit for node 6940924b575bc060ce160a69 2025-12-17 22:34:22,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:22,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924b575bc060ce160a69 HTTP/1.1" 502 150 2025-12-17 22:34:22,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940924b575bc060ce160a69'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:22,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940924b575bc060ce160a69 2025-12-17 22:34:22,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924b575bc060ce160a69 HTTP/1.1" 502 150 2025-12-17 22:34:22,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940924b575bc060ce160a69'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:24,244 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fd6575bc060ce160272 2025-12-17 22:34:24,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd6575bc060ce160272 HTTP/1.1" 502 150 2025-12-17 22:34:24,246 - lava_callback - ERROR - Error processing callback for node 69408fd6575bc060ce160272: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fd6575bc060ce160272 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fd6575bc060ce160272 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fd6575bc060ce160272 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:24,248 - lava_callback - INFO - Starting async job submit for node 69408ae8575bc060ce15f018 2025-12-17 22:34:24,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:24,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae8575bc060ce15f018 HTTP/1.1" 502 150 2025-12-17 22:34:24,258 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae8575bc060ce15f018'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:24,258 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae8575bc060ce15f018 2025-12-17 22:34:24,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae8575bc060ce15f018 HTTP/1.1" 502 150 2025-12-17 22:34:24,258 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae8575bc060ce15f018'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:24,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae6575bc060ce15f013 2025-12-17 22:34:24,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae6575bc060ce15f013 HTTP/1.1" 502 150 2025-12-17 22:34:24,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae6575bc060ce15f013'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:24,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940924b575bc060ce160a69 2025-12-17 22:34:24,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924b575bc060ce160a69 HTTP/1.1" 502 150 2025-12-17 22:34:24,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940924b575bc060ce160a69'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:25,013 - lava_callback - INFO - Received callback for node 69408ce4575bc060ce15f6db INFO: 172.21.0.1:51356 - "POST /node/69408ce4575bc060ce15f6db HTTP/1.0" 202 Accepted 2025-12-17 22:34:25,662 - lava_callback - INFO - Received callback for node 69408ce6575bc060ce15f6de INFO: 172.21.0.1:51358 - "POST /node/69408ce6575bc060ce15f6de HTTP/1.0" 202 Accepted 2025-12-17 22:34:26,259 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae8575bc060ce15f018 2025-12-17 22:34:26,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae8575bc060ce15f018 HTTP/1.1" 502 150 2025-12-17 22:34:26,260 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae8575bc060ce15f018'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:26,328 - lava_callback - INFO - Received callback for node 69408ce8575bc060ce15f6e1 INFO: 172.21.0.1:51372 - "POST /node/69408ce8575bc060ce15f6e1 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:51384 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:34:26,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae4575bc060ce15f010 2025-12-17 22:34:26,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae4575bc060ce15f010 HTTP/1.1" 502 150 2025-12-17 22:34:26,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae4575bc060ce15f010'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:26,992 - lava_callback - INFO - Received callback for node 69408cea575bc060ce15f6e4 2025-12-17 22:34:26,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aba575bc060ce15ef47 INFO: 172.21.0.1:51386 - "POST /node/69408cea575bc060ce15f6e4 HTTP/1.0" 202 Accepted 2025-12-17 22:34:26,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aba575bc060ce15ef47 HTTP/1.1" 502 150 2025-12-17 22:34:26,999 - lava_callback - ERROR - Error processing callback for node 69408aba575bc060ce15ef47: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aba575bc060ce15ef47 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aba575bc060ce15ef47 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aba575bc060ce15ef47 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:27,002 - lava_callback - INFO - Starting async job submit for node 69408aea575bc060ce15f01f 2025-12-17 22:34:27,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:27,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aea575bc060ce15f01f HTTP/1.1" 502 150 2025-12-17 22:34:27,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aea575bc060ce15f01f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:27,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aea575bc060ce15f01f 2025-12-17 22:34:27,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aea575bc060ce15f01f HTTP/1.1" 502 150 2025-12-17 22:34:27,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aea575bc060ce15f01f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:27,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae0575bc060ce15f00a 2025-12-17 22:34:27,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae0575bc060ce15f00a HTTP/1.1" 502 150 2025-12-17 22:34:27,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae0575bc060ce15f00a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:28,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae6575bc060ce15f013 2025-12-17 22:34:28,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae6575bc060ce15f013 HTTP/1.1" 502 150 2025-12-17 22:34:28,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae6575bc060ce15f013'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:28,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408926575bc060ce15e98f 2025-12-17 22:34:28,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408926575bc060ce15e98f HTTP/1.1" 502 150 2025-12-17 22:34:28,743 - lava_callback - ERROR - Error processing callback for node 69408926575bc060ce15e98f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408926575bc060ce15e98f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408926575bc060ce15e98f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408926575bc060ce15e98f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:28,745 - lava_callback - INFO - Starting async job submit for node 69408aec575bc060ce15f024 2025-12-17 22:34:28,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:28,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aec575bc060ce15f024 HTTP/1.1" 502 150 2025-12-17 22:34:28,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aec575bc060ce15f024'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:28,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aec575bc060ce15f024 2025-12-17 22:34:28,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aec575bc060ce15f024 HTTP/1.1" 502 150 2025-12-17 22:34:28,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aec575bc060ce15f024'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:28,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940924b575bc060ce160a69 2025-12-17 22:34:28,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924b575bc060ce160a69 HTTP/1.1" 502 150 2025-12-17 22:34:28,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940924b575bc060ce160a69'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:29,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aea575bc060ce15f01f 2025-12-17 22:34:29,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aea575bc060ce15f01f HTTP/1.1" 502 150 2025-12-17 22:34:29,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aea575bc060ce15f01f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:30,260 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae8575bc060ce15f018 2025-12-17 22:34:30,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae8575bc060ce15f018 HTTP/1.1" 502 150 2025-12-17 22:34:30,261 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae8575bc060ce15f018'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:30,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae2575bc060ce15f00d 2025-12-17 22:34:30,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae2575bc060ce15f00d HTTP/1.1" 502 150 2025-12-17 22:34:30,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae2575bc060ce15f00d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:30,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aec575bc060ce15f024 2025-12-17 22:34:30,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aec575bc060ce15f024 HTTP/1.1" 502 150 2025-12-17 22:34:30,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aec575bc060ce15f024'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:31,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c0575bc060ce161b4f 2025-12-17 22:34:31,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c0575bc060ce161b4f HTTP/1.1" 502 150 2025-12-17 22:34:31,859 - lava_callback - ERROR - Error processing callback for node 694098c0575bc060ce161b4f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c0575bc060ce161b4f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c0575bc060ce161b4f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c0575bc060ce161b4f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:31,861 - lava_callback - INFO - Starting async job submit for node 69408aee575bc060ce15f027 2025-12-17 22:34:31,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:31,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aee575bc060ce15f027 HTTP/1.1" 502 150 2025-12-17 22:34:31,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aee575bc060ce15f027'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:31,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aee575bc060ce15f027 2025-12-17 22:34:31,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aee575bc060ce15f027 HTTP/1.1" 502 150 2025-12-17 22:34:31,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aee575bc060ce15f027'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:32,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad4575bc060ce15efdf 2025-12-17 22:34:32,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad4575bc060ce15efdf HTTP/1.1" 502 150 2025-12-17 22:34:32,631 - lava_callback - ERROR - Error processing callback for node 69408ad4575bc060ce15efdf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ad4575bc060ce15efdf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ad4575bc060ce15efdf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ad4575bc060ce15efdf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:32,633 - lava_callback - INFO - Starting async job submit for node 69408af0575bc060ce15f02a 2025-12-17 22:34:32,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:32,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af0575bc060ce15f02a HTTP/1.1" 502 150 2025-12-17 22:34:32,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af0575bc060ce15f02a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:32,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af0575bc060ce15f02a 2025-12-17 22:34:32,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af0575bc060ce15f02a HTTP/1.1" 502 150 2025-12-17 22:34:32,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af0575bc060ce15f02a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:33,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aea575bc060ce15f01f 2025-12-17 22:34:33,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aea575bc060ce15f01f HTTP/1.1" 502 150 2025-12-17 22:34:33,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aea575bc060ce15f01f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:33,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad5575bc060ce15efe6 2025-12-17 22:34:33,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad5575bc060ce15efe6 HTTP/1.1" 502 150 2025-12-17 22:34:33,651 - lava_callback - ERROR - Error processing callback for node 69408ad5575bc060ce15efe6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ad5575bc060ce15efe6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ad5575bc060ce15efe6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ad5575bc060ce15efe6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:33,658 - lava_callback - INFO - Starting async job submit for node 69408af2575bc060ce15f02d 2025-12-17 22:34:33,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:33,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af2575bc060ce15f02d HTTP/1.1" 502 150 2025-12-17 22:34:33,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af2575bc060ce15f02d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:33,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af2575bc060ce15f02d 2025-12-17 22:34:33,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af2575bc060ce15f02d HTTP/1.1" 502 150 2025-12-17 22:34:33,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af2575bc060ce15f02d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:33,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aee575bc060ce15f027 2025-12-17 22:34:33,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aee575bc060ce15f027 HTTP/1.1" 502 150 2025-12-17 22:34:33,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aee575bc060ce15f027'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:34,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af0575bc060ce15f02a 2025-12-17 22:34:34,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af0575bc060ce15f02a HTTP/1.1" 502 150 2025-12-17 22:34:34,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af0575bc060ce15f02a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:34,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aec575bc060ce15f024 2025-12-17 22:34:34,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aec575bc060ce15f024 HTTP/1.1" 502 150 2025-12-17 22:34:34,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aec575bc060ce15f024'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:34,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae4575bc060ce15f010 2025-12-17 22:34:34,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae4575bc060ce15f010 HTTP/1.1" 502 150 2025-12-17 22:34:34,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae4575bc060ce15f010'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:34,798 - lava_callback - INFO - Received callback for node 69408cec575bc060ce15f6e7 2025-12-17 22:34:34,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ad6575bc060ce15efed INFO: 172.21.0.1:44158 - "POST /node/69408cec575bc060ce15f6e7 HTTP/1.0" 202 Accepted 2025-12-17 22:34:34,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ad6575bc060ce15efed HTTP/1.1" 502 150 2025-12-17 22:34:34,805 - lava_callback - ERROR - Error processing callback for node 69408ad6575bc060ce15efed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ad6575bc060ce15efed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ad6575bc060ce15efed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ad6575bc060ce15efed (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:34,807 - lava_callback - INFO - Starting async job submit for node 69408af4575bc060ce15f030 2025-12-17 22:34:34,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:34,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af4575bc060ce15f030 HTTP/1.1" 502 150 2025-12-17 22:34:34,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af4575bc060ce15f030'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:34,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af4575bc060ce15f030 2025-12-17 22:34:34,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af4575bc060ce15f030 HTTP/1.1" 502 150 2025-12-17 22:34:34,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af4575bc060ce15f030'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:34,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ada575bc060ce15f001 2025-12-17 22:34:34,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ada575bc060ce15f001 HTTP/1.1" 502 150 2025-12-17 22:34:34,869 - lava_callback - ERROR - Error processing callback for node 69408ada575bc060ce15f001: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ada575bc060ce15f001 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ada575bc060ce15f001 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ada575bc060ce15f001 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:34,872 - lava_callback - INFO - Starting async job submit for node 69408af5575bc060ce15f033 2025-12-17 22:34:34,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:34,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af5575bc060ce15f033 HTTP/1.1" 502 150 2025-12-17 22:34:34,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af5575bc060ce15f033'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:34,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af5575bc060ce15f033 2025-12-17 22:34:34,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af5575bc060ce15f033 HTTP/1.1" 502 150 2025-12-17 22:34:34,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af5575bc060ce15f033'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:35,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408adc575bc060ce15f004 2025-12-17 22:34:35,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408adc575bc060ce15f004 HTTP/1.1" 502 150 2025-12-17 22:34:35,045 - lava_callback - ERROR - Error processing callback for node 69408adc575bc060ce15f004: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408adc575bc060ce15f004 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408adc575bc060ce15f004 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408adc575bc060ce15f004 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:35,047 - lava_callback - INFO - Starting async job submit for node 69408af8575bc060ce15f036 2025-12-17 22:34:35,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:35,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af8575bc060ce15f036 HTTP/1.1" 502 150 2025-12-17 22:34:35,054 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af8575bc060ce15f036'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:35,054 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af8575bc060ce15f036 2025-12-17 22:34:35,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af8575bc060ce15f036 HTTP/1.1" 502 150 2025-12-17 22:34:35,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af8575bc060ce15f036'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:35,137 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ade575bc060ce15f007 2025-12-17 22:34:35,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ade575bc060ce15f007 HTTP/1.1" 502 150 2025-12-17 22:34:35,138 - lava_callback - ERROR - Error processing callback for node 69408ade575bc060ce15f007: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ade575bc060ce15f007 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ade575bc060ce15f007 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ade575bc060ce15f007 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:35,140 - lava_callback - INFO - Starting async job submit for node 69408afa575bc060ce15f039 2025-12-17 22:34:35,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:35,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afa575bc060ce15f039 HTTP/1.1" 502 150 2025-12-17 22:34:35,147 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408afa575bc060ce15f039'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:35,147 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408afa575bc060ce15f039 2025-12-17 22:34:35,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afa575bc060ce15f039 HTTP/1.1" 502 150 2025-12-17 22:34:35,148 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408afa575bc060ce15f039'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:35,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408a55575bc060ce15ee1f 2025-12-17 22:34:35,450 - lava_callback - INFO - Received callback for node 69408cee575bc060ce15f6ea INFO: 172.21.0.1:44168 - "POST /node/69408cee575bc060ce15f6ea HTTP/1.0" 202 Accepted 2025-12-17 22:34:35,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af2575bc060ce15f02d 2025-12-17 22:34:35,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af2575bc060ce15f02d HTTP/1.1" 502 150 2025-12-17 22:34:35,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af2575bc060ce15f02d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:36,121 - lava_callback - INFO - Received callback for node 69408cf2575bc060ce15f6f2 INFO: 172.21.0.1:44178 - "POST /node/69408cf2575bc060ce15f6f2 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:44192 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:34:36,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae6575bc060ce15f013 2025-12-17 22:34:36,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae6575bc060ce15f013 HTTP/1.1" 502 150 2025-12-17 22:34:36,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae6575bc060ce15f013'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:36,787 - lava_callback - INFO - Received callback for node 69408cf4575bc060ce15f6f5 INFO: 172.21.0.1:44204 - "POST /node/69408cf4575bc060ce15f6f5 HTTP/1.0" 202 Accepted 2025-12-17 22:34:36,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af4575bc060ce15f030 2025-12-17 22:34:36,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af4575bc060ce15f030 HTTP/1.1" 502 150 2025-12-17 22:34:36,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af4575bc060ce15f030'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:36,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af5575bc060ce15f033 2025-12-17 22:34:36,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af5575bc060ce15f033 HTTP/1.1" 502 150 2025-12-17 22:34:36,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af5575bc060ce15f033'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:36,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940924b575bc060ce160a69 2025-12-17 22:34:36,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924b575bc060ce160a69 HTTP/1.1" 502 150 2025-12-17 22:34:36,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940924b575bc060ce160a69'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:37,056 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af8575bc060ce15f036 2025-12-17 22:34:37,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af8575bc060ce15f036 HTTP/1.1" 502 150 2025-12-17 22:34:37,057 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af8575bc060ce15f036'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:37,148 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408afa575bc060ce15f039 2025-12-17 22:34:37,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afa575bc060ce15f039 HTTP/1.1" 502 150 2025-12-17 22:34:37,149 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408afa575bc060ce15f039'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:37,456 - lava_callback - INFO - Received callback for node 69408cf6575bc060ce15f6fd INFO: 172.21.0.1:44212 - "POST /node/69408cf6575bc060ce15f6fd HTTP/1.0" 202 Accepted 2025-12-17 22:34:37,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aee575bc060ce15f027 2025-12-17 22:34:37,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aee575bc060ce15f027 HTTP/1.1" 502 150 2025-12-17 22:34:37,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aee575bc060ce15f027'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:38,115 - lava_callback - INFO - Received callback for node 69408cf8575bc060ce15f700 INFO: 172.21.0.1:44222 - "POST /node/69408cf8575bc060ce15f700 HTTP/1.0" 202 Accepted 2025-12-17 22:34:38,261 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae8575bc060ce15f018 2025-12-17 22:34:38,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae8575bc060ce15f018 HTTP/1.1" 502 150 2025-12-17 22:34:38,262 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ae8575bc060ce15f018'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:38,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af0575bc060ce15f02a 2025-12-17 22:34:38,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af0575bc060ce15f02a HTTP/1.1" 502 150 2025-12-17 22:34:38,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af0575bc060ce15f02a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:38,773 - lava_callback - INFO - Received callback for node 69408cf9575bc060ce15f703 INFO: 172.21.0.1:44234 - "POST /node/69408cf9575bc060ce15f703 HTTP/1.0" 202 Accepted 2025-12-17 22:34:39,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af2575bc060ce15f02d 2025-12-17 22:34:39,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af2575bc060ce15f02d HTTP/1.1" 502 150 2025-12-17 22:34:39,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af2575bc060ce15f02d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:40,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af4575bc060ce15f030 2025-12-17 22:34:40,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af4575bc060ce15f030 HTTP/1.1" 502 150 2025-12-17 22:34:40,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af4575bc060ce15f030'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:40,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af5575bc060ce15f033 2025-12-17 22:34:40,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af5575bc060ce15f033 HTTP/1.1" 502 150 2025-12-17 22:34:40,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af5575bc060ce15f033'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:41,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aea575bc060ce15f01f 2025-12-17 22:34:41,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aea575bc060ce15f01f HTTP/1.1" 502 150 2025-12-17 22:34:41,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aea575bc060ce15f01f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:41,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af8575bc060ce15f036 2025-12-17 22:34:41,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af8575bc060ce15f036 HTTP/1.1" 502 150 2025-12-17 22:34:41,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af8575bc060ce15f036'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:41,149 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408afa575bc060ce15f039 2025-12-17 22:34:41,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afa575bc060ce15f039 HTTP/1.1" 502 150 2025-12-17 22:34:41,151 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408afa575bc060ce15f039'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:42,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aec575bc060ce15f024 2025-12-17 22:34:42,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aec575bc060ce15f024 HTTP/1.1" 502 150 2025-12-17 22:34:42,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aec575bc060ce15f024'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:43,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae0575bc060ce15f00a 2025-12-17 22:34:43,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae0575bc060ce15f00a HTTP/1.1" 502 150 2025-12-17 22:34:43,532 - lava_callback - ERROR - Error processing callback for node 69408ae0575bc060ce15f00a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ae0575bc060ce15f00a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ae0575bc060ce15f00a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ae0575bc060ce15f00a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:43,534 - lava_callback - INFO - Starting async job submit for node 69409253575bc060ce160a75 2025-12-17 22:34:43,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:43,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409253575bc060ce160a75 HTTP/1.1" 502 150 2025-12-17 22:34:43,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409253575bc060ce160a75'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:43,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409253575bc060ce160a75 2025-12-17 22:34:43,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409253575bc060ce160a75 HTTP/1.1" 502 150 2025-12-17 22:34:43,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409253575bc060ce160a75'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:45,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409253575bc060ce160a75 2025-12-17 22:34:45,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409253575bc060ce160a75 HTTP/1.1" 502 150 2025-12-17 22:34:45,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409253575bc060ce160a75'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:45,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aee575bc060ce15f027 2025-12-17 22:34:45,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aee575bc060ce15f027 HTTP/1.1" 502 150 2025-12-17 22:34:45,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408aee575bc060ce15f027'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54866 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:34:46,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af0575bc060ce15f02a 2025-12-17 22:34:46,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af0575bc060ce15f02a HTTP/1.1" 502 150 2025-12-17 22:34:46,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af0575bc060ce15f02a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:46,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae2575bc060ce15f00d 2025-12-17 22:34:46,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae2575bc060ce15f00d HTTP/1.1" 502 150 2025-12-17 22:34:46,652 - lava_callback - ERROR - Error processing callback for node 69408ae2575bc060ce15f00d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ae2575bc060ce15f00d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ae2575bc060ce15f00d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ae2575bc060ce15f00d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:46,655 - lava_callback - INFO - Starting async job submit for node 694086cb575bc060ce15e190 2025-12-17 22:34:46,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:46,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cb575bc060ce15e190 HTTP/1.1" 502 150 2025-12-17 22:34:46,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cb575bc060ce15e190'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:46,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cb575bc060ce15e190 2025-12-17 22:34:46,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cb575bc060ce15e190 HTTP/1.1" 502 150 2025-12-17 22:34:46,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cb575bc060ce15e190'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:47,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af2575bc060ce15f02d 2025-12-17 22:34:47,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af2575bc060ce15f02d HTTP/1.1" 502 150 2025-12-17 22:34:47,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af2575bc060ce15f02d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:48,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cb575bc060ce15e190 2025-12-17 22:34:48,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cb575bc060ce15e190 HTTP/1.1" 502 150 2025-12-17 22:34:48,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cb575bc060ce15e190'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:48,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af4575bc060ce15f030 2025-12-17 22:34:48,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af4575bc060ce15f030 HTTP/1.1" 502 150 2025-12-17 22:34:48,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af4575bc060ce15f030'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:48,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af5575bc060ce15f033 2025-12-17 22:34:48,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af5575bc060ce15f033 HTTP/1.1" 502 150 2025-12-17 22:34:48,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af5575bc060ce15f033'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:49,059 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af8575bc060ce15f036 2025-12-17 22:34:49,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af8575bc060ce15f036 HTTP/1.1" 502 150 2025-12-17 22:34:49,061 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408af8575bc060ce15f036'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:49,151 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408afa575bc060ce15f039 2025-12-17 22:34:49,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afa575bc060ce15f039 HTTP/1.1" 502 150 2025-12-17 22:34:49,152 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408afa575bc060ce15f039'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:49,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409253575bc060ce160a75 2025-12-17 22:34:49,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409253575bc060ce160a75 HTTP/1.1" 502 150 2025-12-17 22:34:49,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409253575bc060ce160a75'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:50,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae4575bc060ce15f010 2025-12-17 22:34:52,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae6575bc060ce15f013 2025-12-17 22:34:52,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae6575bc060ce15f013 HTTP/1.1" 502 150 2025-12-17 22:34:52,658 - lava_callback - ERROR - Error processing callback for node 69408ae6575bc060ce15f013: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ae6575bc060ce15f013 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ae6575bc060ce15f013 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ae6575bc060ce15f013 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:52,660 - lava_callback - INFO - Starting async job submit for node 69408be9575bc060ce15f39e 2025-12-17 22:34:52,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:52,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be9575bc060ce15f39e HTTP/1.1" 502 150 2025-12-17 22:34:52,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be9575bc060ce15f39e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:52,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be9575bc060ce15f39e 2025-12-17 22:34:52,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be9575bc060ce15f39e HTTP/1.1" 502 150 2025-12-17 22:34:52,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be9575bc060ce15f39e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:52,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cb575bc060ce15e190 2025-12-17 22:34:52,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cb575bc060ce15e190 HTTP/1.1" 502 150 2025-12-17 22:34:52,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cb575bc060ce15e190'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:52,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940924b575bc060ce160a69 2025-12-17 22:34:52,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924b575bc060ce160a69 HTTP/1.1" 502 150 2025-12-17 22:34:52,982 - lava_callback - ERROR - Error processing callback for node 6940924b575bc060ce160a69: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940924b575bc060ce160a69 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940924b575bc060ce160a69 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940924b575bc060ce160a69 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:52,984 - lava_callback - INFO - Starting async job submit for node 69408afc575bc060ce15f03c 2025-12-17 22:34:52,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:53,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afc575bc060ce15f03c HTTP/1.1" 502 150 2025-12-17 22:34:53,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408afc575bc060ce15f03c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:53,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408afc575bc060ce15f03c 2025-12-17 22:34:53,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afc575bc060ce15f03c HTTP/1.1" 502 150 2025-12-17 22:34:53,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408afc575bc060ce15f03c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:54,263 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ae8575bc060ce15f018 2025-12-17 22:34:54,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae8575bc060ce15f018 HTTP/1.1" 502 150 2025-12-17 22:34:54,264 - lava_callback - ERROR - Error processing callback for node 69408ae8575bc060ce15f018: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ae8575bc060ce15f018 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ae8575bc060ce15f018 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ae8575bc060ce15f018 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:54,266 - lava_callback - INFO - Starting async job submit for node 6940879d575bc060ce15e427 2025-12-17 22:34:54,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:54,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879d575bc060ce15e427 HTTP/1.1" 502 150 2025-12-17 22:34:54,340 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879d575bc060ce15e427'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:54,340 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879d575bc060ce15e427 2025-12-17 22:34:54,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879d575bc060ce15e427 HTTP/1.1" 502 150 2025-12-17 22:34:54,341 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879d575bc060ce15e427'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:54,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be9575bc060ce15f39e 2025-12-17 22:34:54,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be9575bc060ce15f39e HTTP/1.1" 502 150 2025-12-17 22:34:54,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be9575bc060ce15f39e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:54,833 - lava_callback - INFO - Received callback for node 69408cfc575bc060ce15f733 INFO: 172.21.0.1:58074 - "POST /node/69408cfc575bc060ce15f733 HTTP/1.0" 202 Accepted 2025-12-17 22:34:55,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408afc575bc060ce15f03c 2025-12-17 22:34:55,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afc575bc060ce15f03c HTTP/1.1" 502 150 2025-12-17 22:34:55,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408afc575bc060ce15f03c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:55,470 - lava_callback - INFO - Received callback for node 69408cfd575bc060ce15f736 INFO: 172.21.0.1:58076 - "POST /node/69408cfd575bc060ce15f736 HTTP/1.0" 202 Accepted 2025-12-17 22:34:56,135 - lava_callback - INFO - Received callback for node 69408cfe575bc060ce15f739 INFO: 172.21.0.1:58084 - "POST /node/69408cfe575bc060ce15f739 HTTP/1.0" 202 Accepted 2025-12-17 22:34:56,341 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879d575bc060ce15e427 2025-12-17 22:34:56,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879d575bc060ce15e427 HTTP/1.1" 502 150 2025-12-17 22:34:56,343 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879d575bc060ce15e427'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58100 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:34:56,810 - lava_callback - INFO - Received callback for node 69408d00575bc060ce15f73c INFO: 172.21.0.1:58110 - "POST /node/69408d00575bc060ce15f73c HTTP/1.0" 202 Accepted 2025-12-17 22:34:57,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aea575bc060ce15f01f 2025-12-17 22:34:57,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aea575bc060ce15f01f HTTP/1.1" 502 150 2025-12-17 22:34:57,021 - lava_callback - ERROR - Error processing callback for node 69408aea575bc060ce15f01f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aea575bc060ce15f01f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aea575bc060ce15f01f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aea575bc060ce15f01f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:57,023 - lava_callback - INFO - Starting async job submit for node 69408afe575bc060ce15f045 2025-12-17 22:34:57,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:57,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afe575bc060ce15f045 HTTP/1.1" 502 150 2025-12-17 22:34:57,040 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408afe575bc060ce15f045'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:57,040 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408afe575bc060ce15f045 2025-12-17 22:34:57,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afe575bc060ce15f045 HTTP/1.1" 502 150 2025-12-17 22:34:57,041 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408afe575bc060ce15f045'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:57,492 - lava_callback - INFO - Received callback for node 69408d05575bc060ce15f747 INFO: 172.21.0.1:58120 - "POST /node/69408d05575bc060ce15f747 HTTP/1.0" 202 Accepted 2025-12-17 22:34:57,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409253575bc060ce160a75 2025-12-17 22:34:57,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409253575bc060ce160a75 HTTP/1.1" 502 150 2025-12-17 22:34:57,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409253575bc060ce160a75'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:58,147 - lava_callback - INFO - Received callback for node 69408d09575bc060ce15f757 INFO: 172.21.0.1:58122 - "POST /node/69408d09575bc060ce15f757 HTTP/1.0" 202 Accepted 2025-12-17 22:34:58,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be9575bc060ce15f39e 2025-12-17 22:34:58,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be9575bc060ce15f39e HTTP/1.1" 502 150 2025-12-17 22:34:58,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be9575bc060ce15f39e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:58,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aec575bc060ce15f024 2025-12-17 22:34:58,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aec575bc060ce15f024 HTTP/1.1" 502 150 2025-12-17 22:34:58,758 - lava_callback - ERROR - Error processing callback for node 69408aec575bc060ce15f024: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aec575bc060ce15f024 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aec575bc060ce15f024 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aec575bc060ce15f024 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:34:58,760 - lava_callback - INFO - Starting async job submit for node 69408b02575bc060ce15f04e 2025-12-17 22:34:58,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:34:58,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b02575bc060ce15f04e HTTP/1.1" 502 150 2025-12-17 22:34:58,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b02575bc060ce15f04e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:58,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b02575bc060ce15f04e 2025-12-17 22:34:58,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b02575bc060ce15f04e HTTP/1.1" 502 150 2025-12-17 22:34:58,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b02575bc060ce15f04e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:58,812 - lava_callback - INFO - Received callback for node 69408d0b575bc060ce15f75a INFO: 172.21.0.1:58132 - "POST /node/69408d0b575bc060ce15f75a HTTP/1.0" 202 Accepted 2025-12-17 22:34:59,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408afc575bc060ce15f03c 2025-12-17 22:34:59,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afc575bc060ce15f03c HTTP/1.1" 502 150 2025-12-17 22:34:59,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408afc575bc060ce15f03c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:59,041 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408afe575bc060ce15f045 2025-12-17 22:34:59,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afe575bc060ce15f045 HTTP/1.1" 502 150 2025-12-17 22:34:59,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408afe575bc060ce15f045'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:34:59,485 - lava_callback - INFO - Received callback for node 69408d0c575bc060ce15f762 INFO: 172.21.0.1:58134 - "POST /node/69408d0c575bc060ce15f762 HTTP/1.0" 202 Accepted 2025-12-17 22:35:00,343 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879d575bc060ce15e427 2025-12-17 22:35:00,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879d575bc060ce15e427 HTTP/1.1" 502 150 2025-12-17 22:35:00,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879d575bc060ce15e427'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:00,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cb575bc060ce15e190 2025-12-17 22:35:00,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cb575bc060ce15e190 HTTP/1.1" 502 150 2025-12-17 22:35:00,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694086cb575bc060ce15e190'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:00,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b02575bc060ce15f04e 2025-12-17 22:35:00,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b02575bc060ce15f04e HTTP/1.1" 502 150 2025-12-17 22:35:00,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b02575bc060ce15f04e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:01,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408aee575bc060ce15f027 INFO: 172.21.0.1:34300 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:35:02,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af0575bc060ce15f02a 2025-12-17 22:35:02,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af0575bc060ce15f02a HTTP/1.1" 502 150 2025-12-17 22:35:02,650 - lava_callback - ERROR - Error processing callback for node 69408af0575bc060ce15f02a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408af0575bc060ce15f02a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408af0575bc060ce15f02a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408af0575bc060ce15f02a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:02,651 - lava_callback - INFO - Starting async job submit for node 69408b06575bc060ce15f054 2025-12-17 22:35:02,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:02,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b06575bc060ce15f054 HTTP/1.1" 502 150 2025-12-17 22:35:02,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b06575bc060ce15f054'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:02,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b06575bc060ce15f054 2025-12-17 22:35:02,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b06575bc060ce15f054 HTTP/1.1" 502 150 2025-12-17 22:35:02,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b06575bc060ce15f054'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:02,910 - lava_callback - INFO - Received callback for node 69408d09575bc060ce15f754 INFO: 172.21.0.1:34310 - "POST /node/69408d09575bc060ce15f754 HTTP/1.0" 202 Accepted 2025-12-17 22:35:03,042 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408afe575bc060ce15f045 2025-12-17 22:35:03,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afe575bc060ce15f045 HTTP/1.1" 502 150 2025-12-17 22:35:03,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408afe575bc060ce15f045'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:03,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af2575bc060ce15f02d 2025-12-17 22:35:03,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af2575bc060ce15f02d HTTP/1.1" 502 150 2025-12-17 22:35:03,685 - lava_callback - ERROR - Error processing callback for node 69408af2575bc060ce15f02d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408af2575bc060ce15f02d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408af2575bc060ce15f02d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408af2575bc060ce15f02d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:03,687 - lava_callback - INFO - Starting async job submit for node 69408b08575bc060ce15f057 2025-12-17 22:35:03,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:03,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b08575bc060ce15f057 HTTP/1.1" 502 150 2025-12-17 22:35:03,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b08575bc060ce15f057'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:03,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b08575bc060ce15f057 2025-12-17 22:35:03,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b08575bc060ce15f057 HTTP/1.1" 502 150 2025-12-17 22:35:03,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b08575bc060ce15f057'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:04,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b06575bc060ce15f054 2025-12-17 22:35:04,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b06575bc060ce15f054 HTTP/1.1" 502 150 2025-12-17 22:35:04,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b06575bc060ce15f054'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:04,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b02575bc060ce15f04e 2025-12-17 22:35:04,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b02575bc060ce15f04e HTTP/1.1" 502 150 2025-12-17 22:35:04,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b02575bc060ce15f04e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:04,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af4575bc060ce15f030 2025-12-17 22:35:04,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af4575bc060ce15f030 HTTP/1.1" 502 150 2025-12-17 22:35:04,822 - lava_callback - ERROR - Error processing callback for node 69408af4575bc060ce15f030: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408af4575bc060ce15f030 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408af4575bc060ce15f030 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408af4575bc060ce15f030 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:04,824 - lava_callback - INFO - Starting async job submit for node 69408b0a575bc060ce15f05a 2025-12-17 22:35:04,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:04,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b0a575bc060ce15f05a HTTP/1.1" 502 150 2025-12-17 22:35:04,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b0a575bc060ce15f05a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:04,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b0a575bc060ce15f05a 2025-12-17 22:35:04,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b0a575bc060ce15f05a HTTP/1.1" 502 150 2025-12-17 22:35:04,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b0a575bc060ce15f05a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:04,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af5575bc060ce15f033 2025-12-17 22:35:04,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af5575bc060ce15f033 HTTP/1.1" 502 150 2025-12-17 22:35:04,892 - lava_callback - ERROR - Error processing callback for node 69408af5575bc060ce15f033: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408af5575bc060ce15f033 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408af5575bc060ce15f033 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408af5575bc060ce15f033 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:04,894 - lava_callback - INFO - Starting async job submit for node 69408b0d575bc060ce15f07c 2025-12-17 22:35:04,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:04,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b0d575bc060ce15f07c HTTP/1.1" 502 150 2025-12-17 22:35:04,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b0d575bc060ce15f07c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:04,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b0d575bc060ce15f07c 2025-12-17 22:35:04,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b0d575bc060ce15f07c HTTP/1.1" 502 150 2025-12-17 22:35:04,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b0d575bc060ce15f07c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:05,048 - lava_callback - INFO - Received callback for node 69408d0f575bc060ce15f768 INFO: 172.21.0.1:34318 - "POST /node/69408d0f575bc060ce15f768 HTTP/1.0" 202 Accepted 2025-12-17 22:35:05,061 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408af8575bc060ce15f036 2025-12-17 22:35:05,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408af8575bc060ce15f036 HTTP/1.1" 502 150 2025-12-17 22:35:05,062 - lava_callback - ERROR - Error processing callback for node 69408af8575bc060ce15f036: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408af8575bc060ce15f036 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408af8575bc060ce15f036 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408af8575bc060ce15f036 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:05,064 - lava_callback - INFO - Starting async job submit for node 6940e7cf460bc8e2651142e7 2025-12-17 22:35:05,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:05,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940e7cf460bc8e2651142e7 HTTP/1.1" 502 150 2025-12-17 22:35:05,134 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940e7cf460bc8e2651142e7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:05,134 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940e7cf460bc8e2651142e7 2025-12-17 22:35:05,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940e7cf460bc8e2651142e7 HTTP/1.1" 502 150 2025-12-17 22:35:05,135 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940e7cf460bc8e2651142e7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:05,152 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408afa575bc060ce15f039 2025-12-17 22:35:05,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afa575bc060ce15f039 HTTP/1.1" 502 150 2025-12-17 22:35:05,153 - lava_callback - ERROR - Error processing callback for node 69408afa575bc060ce15f039: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408afa575bc060ce15f039 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408afa575bc060ce15f039 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408afa575bc060ce15f039 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:05,155 - lava_callback - INFO - Starting async job submit for node 69408b10575bc060ce15f0a5 2025-12-17 22:35:05,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:05,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b10575bc060ce15f0a5 HTTP/1.1" 502 150 2025-12-17 22:35:05,162 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b10575bc060ce15f0a5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:05,162 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b10575bc060ce15f0a5 2025-12-17 22:35:05,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b10575bc060ce15f0a5 HTTP/1.1" 502 150 2025-12-17 22:35:05,163 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b10575bc060ce15f0a5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:05,719 - lava_callback - INFO - Received callback for node 69408d11575bc060ce15f76b INFO: 172.21.0.1:34328 - "POST /node/69408d11575bc060ce15f76b HTTP/1.0" 202 Accepted 2025-12-17 22:35:05,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b08575bc060ce15f057 2025-12-17 22:35:05,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b08575bc060ce15f057 HTTP/1.1" 502 150 2025-12-17 22:35:05,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b08575bc060ce15f057'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:06,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ae4575bc060ce15f010 HTTP/1.1" 502 150 2025-12-17 22:35:06,033 - lava_callback - ERROR - Error processing callback for node 69408ae4575bc060ce15f010: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ae4575bc060ce15f010 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ae4575bc060ce15f010 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ae4575bc060ce15f010 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:06,035 - lava_callback - INFO - Starting async job submit for node 69408b14575bc060ce15f0b6 2025-12-17 22:35:06,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:06,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b14575bc060ce15f0b6 HTTP/1.1" 502 150 2025-12-17 22:35:06,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b14575bc060ce15f0b6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:06,042 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b14575bc060ce15f0b6 2025-12-17 22:35:06,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b14575bc060ce15f0b6 HTTP/1.1" 502 150 2025-12-17 22:35:06,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b14575bc060ce15f0b6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:06,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408a55575bc060ce15ee1f HTTP/1.1" 502 150 2025-12-17 22:35:06,288 - lava_callback - ERROR - Error processing callback for node 69408a55575bc060ce15ee1f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a55575bc060ce15ee1f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a55575bc060ce15ee1f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408a55575bc060ce15ee1f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:06,291 - lava_callback - INFO - Starting async job submit for node 69408b17575bc060ce15f0c4 2025-12-17 22:35:06,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:06,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b17575bc060ce15f0c4 HTTP/1.1" 502 150 2025-12-17 22:35:06,305 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b17575bc060ce15f0c4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:06,305 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b17575bc060ce15f0c4 2025-12-17 22:35:06,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b17575bc060ce15f0c4 HTTP/1.1" 502 150 2025-12-17 22:35:06,306 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b17575bc060ce15f0c4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34344 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:35:06,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be9575bc060ce15f39e 2025-12-17 22:35:06,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be9575bc060ce15f39e HTTP/1.1" 502 150 2025-12-17 22:35:06,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be9575bc060ce15f39e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:06,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b0a575bc060ce15f05a 2025-12-17 22:35:06,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b0a575bc060ce15f05a HTTP/1.1" 502 150 2025-12-17 22:35:06,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b0a575bc060ce15f05a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:06,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b0d575bc060ce15f07c 2025-12-17 22:35:06,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b0d575bc060ce15f07c HTTP/1.1" 502 150 2025-12-17 22:35:06,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b0d575bc060ce15f07c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:07,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408afc575bc060ce15f03c 2025-12-17 22:35:07,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afc575bc060ce15f03c HTTP/1.1" 502 150 2025-12-17 22:35:07,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408afc575bc060ce15f03c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:07,135 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940e7cf460bc8e2651142e7 2025-12-17 22:35:07,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940e7cf460bc8e2651142e7 HTTP/1.1" 502 150 2025-12-17 22:35:07,136 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940e7cf460bc8e2651142e7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:07,163 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b10575bc060ce15f0a5 2025-12-17 22:35:07,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b10575bc060ce15f0a5 HTTP/1.1" 502 150 2025-12-17 22:35:07,164 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b10575bc060ce15f0a5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:08,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b14575bc060ce15f0b6 2025-12-17 22:35:08,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b14575bc060ce15f0b6 HTTP/1.1" 502 150 2025-12-17 22:35:08,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b14575bc060ce15f0b6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:08,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b17575bc060ce15f0c4 2025-12-17 22:35:08,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b17575bc060ce15f0c4 HTTP/1.1" 502 150 2025-12-17 22:35:08,308 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b17575bc060ce15f0c4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:08,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879d575bc060ce15e427 2025-12-17 22:35:08,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879d575bc060ce15e427 HTTP/1.1" 502 150 2025-12-17 22:35:08,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940879d575bc060ce15e427'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:08,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b06575bc060ce15f054 2025-12-17 22:35:08,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b06575bc060ce15f054 HTTP/1.1" 502 150 2025-12-17 22:35:08,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b06575bc060ce15f054'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:09,306 - lava_callback - INFO - Received callback for node 69408d0d575bc060ce15f765 INFO: 172.21.0.1:34358 - "POST /node/69408d0d575bc060ce15f765 HTTP/1.0" 202 Accepted 2025-12-17 22:35:09,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b08575bc060ce15f057 2025-12-17 22:35:09,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b08575bc060ce15f057 HTTP/1.1" 502 150 2025-12-17 22:35:09,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b08575bc060ce15f057'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:10,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b0a575bc060ce15f05a 2025-12-17 22:35:10,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b0a575bc060ce15f05a HTTP/1.1" 502 150 2025-12-17 22:35:10,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b0a575bc060ce15f05a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:10,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b0d575bc060ce15f07c 2025-12-17 22:35:10,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b0d575bc060ce15f07c HTTP/1.1" 502 150 2025-12-17 22:35:10,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b0d575bc060ce15f07c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:11,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408afe575bc060ce15f045 2025-12-17 22:35:11,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afe575bc060ce15f045 HTTP/1.1" 502 150 2025-12-17 22:35:11,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408afe575bc060ce15f045'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:11,136 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940e7cf460bc8e2651142e7 2025-12-17 22:35:11,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940e7cf460bc8e2651142e7 HTTP/1.1" 502 150 2025-12-17 22:35:11,138 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940e7cf460bc8e2651142e7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:11,164 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b10575bc060ce15f0a5 2025-12-17 22:35:11,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b10575bc060ce15f0a5 HTTP/1.1" 502 150 2025-12-17 22:35:11,168 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b10575bc060ce15f0a5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:12,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b14575bc060ce15f0b6 2025-12-17 22:35:12,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b14575bc060ce15f0b6 HTTP/1.1" 502 150 2025-12-17 22:35:12,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b14575bc060ce15f0b6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:12,308 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b17575bc060ce15f0c4 2025-12-17 22:35:12,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b17575bc060ce15f0c4 HTTP/1.1" 502 150 2025-12-17 22:35:12,309 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b17575bc060ce15f0c4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:12,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b02575bc060ce15f04e 2025-12-17 22:35:12,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b02575bc060ce15f04e HTTP/1.1" 502 150 2025-12-17 22:35:12,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b02575bc060ce15f04e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:13,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409253575bc060ce160a75 INFO: 172.21.0.1:42788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42790 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:35:16,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b06575bc060ce15f054 2025-12-17 22:35:16,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b06575bc060ce15f054 HTTP/1.1" 502 150 2025-12-17 22:35:16,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b06575bc060ce15f054'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:16,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694086cb575bc060ce15e190 2025-12-17 22:35:16,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694086cb575bc060ce15e190 HTTP/1.1" 502 150 2025-12-17 22:35:16,737 - lava_callback - ERROR - Error processing callback for node 694086cb575bc060ce15e190: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086cb575bc060ce15e190 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086cb575bc060ce15e190 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694086cb575bc060ce15e190 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:16,739 - lava_callback - INFO - Starting async job submit for node 69408b19575bc060ce15f0c7 2025-12-17 22:35:16,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:16,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b19575bc060ce15f0c7 HTTP/1.1" 502 150 2025-12-17 22:35:16,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b19575bc060ce15f0c7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:16,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b19575bc060ce15f0c7 2025-12-17 22:35:16,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b19575bc060ce15f0c7 HTTP/1.1" 502 150 2025-12-17 22:35:16,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b19575bc060ce15f0c7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:17,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408aee575bc060ce15f027 HTTP/1.1" 502 150 2025-12-17 22:35:17,297 - lava_callback - ERROR - Error processing callback for node 69408aee575bc060ce15f027: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aee575bc060ce15f027 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aee575bc060ce15f027 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408aee575bc060ce15f027 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:17,299 - lava_callback - INFO - Starting async job submit for node 69408b1b575bc060ce15f0ca 2025-12-17 22:35:17,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:17,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1b575bc060ce15f0ca HTTP/1.1" 502 150 2025-12-17 22:35:17,305 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b1b575bc060ce15f0ca'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:17,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b1b575bc060ce15f0ca 2025-12-17 22:35:17,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1b575bc060ce15f0ca HTTP/1.1" 502 150 2025-12-17 22:35:17,306 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b1b575bc060ce15f0ca'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:17,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b08575bc060ce15f057 2025-12-17 22:35:17,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b08575bc060ce15f057 HTTP/1.1" 502 150 2025-12-17 22:35:17,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b08575bc060ce15f057'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:18,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b19575bc060ce15f0c7 2025-12-17 22:35:18,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b19575bc060ce15f0c7 HTTP/1.1" 502 150 2025-12-17 22:35:18,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b19575bc060ce15f0c7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:18,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b0a575bc060ce15f05a 2025-12-17 22:35:18,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b0a575bc060ce15f05a HTTP/1.1" 502 150 2025-12-17 22:35:18,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b0a575bc060ce15f05a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:18,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b0d575bc060ce15f07c 2025-12-17 22:35:18,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b0d575bc060ce15f07c HTTP/1.1" 502 150 2025-12-17 22:35:18,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b0d575bc060ce15f07c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:19,138 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940e7cf460bc8e2651142e7 2025-12-17 22:35:19,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940e7cf460bc8e2651142e7 HTTP/1.1" 502 150 2025-12-17 22:35:19,139 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940e7cf460bc8e2651142e7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:19,168 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b10575bc060ce15f0a5 2025-12-17 22:35:19,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b10575bc060ce15f0a5 HTTP/1.1" 502 150 2025-12-17 22:35:19,169 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b10575bc060ce15f0a5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:19,307 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b1b575bc060ce15f0ca 2025-12-17 22:35:19,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1b575bc060ce15f0ca HTTP/1.1" 502 150 2025-12-17 22:35:19,311 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b1b575bc060ce15f0ca'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:20,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b14575bc060ce15f0b6 2025-12-17 22:35:20,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b14575bc060ce15f0b6 HTTP/1.1" 502 150 2025-12-17 22:35:20,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b14575bc060ce15f0b6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:20,309 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b17575bc060ce15f0c4 2025-12-17 22:35:20,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b17575bc060ce15f0c4 HTTP/1.1" 502 150 2025-12-17 22:35:20,311 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b17575bc060ce15f0c4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:22,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be9575bc060ce15f39e 2025-12-17 22:35:22,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be9575bc060ce15f39e HTTP/1.1" 502 150 2025-12-17 22:35:22,702 - lava_callback - ERROR - Error processing callback for node 69408be9575bc060ce15f39e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be9575bc060ce15f39e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be9575bc060ce15f39e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be9575bc060ce15f39e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:22,704 - lava_callback - INFO - Starting async job submit for node 69408b1d575bc060ce15f0cd 2025-12-17 22:35:22,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:22,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1d575bc060ce15f0cd HTTP/1.1" 502 150 2025-12-17 22:35:22,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b1d575bc060ce15f0cd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:22,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b1d575bc060ce15f0cd 2025-12-17 22:35:22,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1d575bc060ce15f0cd HTTP/1.1" 502 150 2025-12-17 22:35:22,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b1d575bc060ce15f0cd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:22,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b19575bc060ce15f0c7 2025-12-17 22:35:22,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b19575bc060ce15f0c7 HTTP/1.1" 502 150 2025-12-17 22:35:22,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b19575bc060ce15f0c7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:22,975 - lava_callback - INFO - Received callback for node 69408d25575bc060ce15f794 INFO: 172.21.0.1:59758 - "POST /node/69408d25575bc060ce15f794 HTTP/1.0" 202 Accepted 2025-12-17 22:35:23,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408afc575bc060ce15f03c 2025-12-17 22:35:23,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afc575bc060ce15f03c HTTP/1.1" 502 150 2025-12-17 22:35:23,016 - lava_callback - ERROR - Error processing callback for node 69408afc575bc060ce15f03c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408afc575bc060ce15f03c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408afc575bc060ce15f03c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408afc575bc060ce15f03c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:23,018 - lava_callback - INFO - Starting async job submit for node 69408b1f575bc060ce15f0d0 2025-12-17 22:35:23,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:23,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1f575bc060ce15f0d0 HTTP/1.1" 502 150 2025-12-17 22:35:23,039 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b1f575bc060ce15f0d0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:23,039 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b1f575bc060ce15f0d0 2025-12-17 22:35:23,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1f575bc060ce15f0d0 HTTP/1.1" 502 150 2025-12-17 22:35:23,040 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b1f575bc060ce15f0d0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:23,311 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b1b575bc060ce15f0ca 2025-12-17 22:35:23,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1b575bc060ce15f0ca HTTP/1.1" 502 150 2025-12-17 22:35:23,312 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b1b575bc060ce15f0ca'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:23,817 - lava_callback - INFO - Received callback for node 69408d26575bc060ce15f797 INFO: 172.21.0.1:59766 - "POST /node/69408d26575bc060ce15f797 HTTP/1.0" 202 Accepted 2025-12-17 22:35:24,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940879d575bc060ce15e427 2025-12-17 22:35:24,550 - lava_callback - INFO - Received callback for node 69408d27575bc060ce15f79a INFO: 172.21.0.1:59774 - "POST /node/69408d27575bc060ce15f79a HTTP/1.0" 202 Accepted 2025-12-17 22:35:24,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b1d575bc060ce15f0cd 2025-12-17 22:35:24,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1d575bc060ce15f0cd HTTP/1.1" 502 150 2025-12-17 22:35:24,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b1d575bc060ce15f0cd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:25,040 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b1f575bc060ce15f0d0 2025-12-17 22:35:25,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1f575bc060ce15f0d0 HTTP/1.1" 502 150 2025-12-17 22:35:25,041 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b1f575bc060ce15f0d0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:25,042 - lava_callback - INFO - Received callback for node 69408d17575bc060ce15f770 INFO: 172.21.0.1:59780 - "POST /node/69408d17575bc060ce15f770 HTTP/1.0" 202 Accepted 2025-12-17 22:35:25,698 - lava_callback - INFO - Received callback for node 69408d1a575bc060ce15f77b INFO: 172.21.0.1:59784 - "POST /node/69408d1a575bc060ce15f77b HTTP/1.0" 202 Accepted 2025-12-17 22:35:26,368 - lava_callback - INFO - Received callback for node 69408d1c575bc060ce15f780 INFO: 172.21.0.1:59790 - "POST /node/69408d1c575bc060ce15f780 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:59806 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:35:27,007 - lava_callback - INFO - Received callback for node 69408d1e575bc060ce15f783 INFO: 172.21.0.1:59822 - "POST /node/69408d1e575bc060ce15f783 HTTP/1.0" 202 Accepted 2025-12-17 22:35:27,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408afe575bc060ce15f045 2025-12-17 22:35:27,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408afe575bc060ce15f045 HTTP/1.1" 502 150 2025-12-17 22:35:27,046 - lava_callback - ERROR - Error processing callback for node 69408afe575bc060ce15f045: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408afe575bc060ce15f045 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408afe575bc060ce15f045 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408afe575bc060ce15f045 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:27,049 - lava_callback - INFO - Starting async job submit for node 69408b21575bc060ce15f0d3 2025-12-17 22:35:27,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:27,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b21575bc060ce15f0d3 HTTP/1.1" 502 150 2025-12-17 22:35:27,057 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b21575bc060ce15f0d3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:27,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b21575bc060ce15f0d3 2025-12-17 22:35:27,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b21575bc060ce15f0d3 HTTP/1.1" 502 150 2025-12-17 22:35:27,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b21575bc060ce15f0d3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:27,271 - lava_callback - INFO - Received callback for node 6940c5381e3c24e95681d6aa INFO: 172.21.0.1:59838 - "POST /node/6940c5381e3c24e95681d6aa HTTP/1.0" 202 Accepted 2025-12-17 22:35:27,642 - lava_callback - INFO - Received callback for node 69408d20575bc060ce15f786 INFO: 172.21.0.1:59846 - "POST /node/69408d20575bc060ce15f786 HTTP/1.0" 202 Accepted 2025-12-17 22:35:28,273 - lava_callback - INFO - Received callback for node 69408d22575bc060ce15f791 INFO: 172.21.0.1:59848 - "POST /node/69408d22575bc060ce15f791 HTTP/1.0" 202 Accepted 2025-12-17 22:35:28,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b1d575bc060ce15f0cd 2025-12-17 22:35:28,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1d575bc060ce15f0cd HTTP/1.1" 502 150 2025-12-17 22:35:28,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b1d575bc060ce15f0cd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:28,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b02575bc060ce15f04e 2025-12-17 22:35:28,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b02575bc060ce15f04e HTTP/1.1" 502 150 2025-12-17 22:35:28,773 - lava_callback - ERROR - Error processing callback for node 69408b02575bc060ce15f04e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b02575bc060ce15f04e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b02575bc060ce15f04e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b02575bc060ce15f04e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:28,775 - lava_callback - INFO - Starting async job submit for node 69408b2c575bc060ce15f0f0 2025-12-17 22:35:28,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:28,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2c575bc060ce15f0f0 HTTP/1.1" 502 150 2025-12-17 22:35:28,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b2c575bc060ce15f0f0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:28,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b2c575bc060ce15f0f0 2025-12-17 22:35:28,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2c575bc060ce15f0f0 HTTP/1.1" 502 150 2025-12-17 22:35:28,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b2c575bc060ce15f0f0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:28,934 - lava_callback - INFO - Received callback for node 69408d2a575bc060ce15f7a0 INFO: 172.21.0.1:59852 - "POST /node/69408d2a575bc060ce15f7a0 HTTP/1.0" 202 Accepted 2025-12-17 22:35:29,041 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b1f575bc060ce15f0d0 2025-12-17 22:35:29,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1f575bc060ce15f0d0 HTTP/1.1" 502 150 2025-12-17 22:35:29,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b1f575bc060ce15f0d0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:29,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b21575bc060ce15f0d3 2025-12-17 22:35:29,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b21575bc060ce15f0d3 HTTP/1.1" 502 150 2025-12-17 22:35:29,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b21575bc060ce15f0d3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:29,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409253575bc060ce160a75 HTTP/1.1" 502 150 2025-12-17 22:35:29,072 - lava_callback - ERROR - Error processing callback for node 69409253575bc060ce160a75: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409253575bc060ce160a75 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409253575bc060ce160a75 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409253575bc060ce160a75 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:29,074 - lava_callback - INFO - Starting async job submit for node 69408b27575bc060ce15f0e5 2025-12-17 22:35:29,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:29,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b27575bc060ce15f0e5 HTTP/1.1" 502 150 2025-12-17 22:35:29,082 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b27575bc060ce15f0e5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:29,082 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b27575bc060ce15f0e5 2025-12-17 22:35:29,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b27575bc060ce15f0e5 HTTP/1.1" 502 150 2025-12-17 22:35:29,083 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b27575bc060ce15f0e5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:30,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b19575bc060ce15f0c7 2025-12-17 22:35:30,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b19575bc060ce15f0c7 HTTP/1.1" 502 150 2025-12-17 22:35:30,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b19575bc060ce15f0c7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:30,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b2c575bc060ce15f0f0 2025-12-17 22:35:30,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2c575bc060ce15f0f0 HTTP/1.1" 502 150 2025-12-17 22:35:30,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b2c575bc060ce15f0f0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:31,083 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b27575bc060ce15f0e5 2025-12-17 22:35:31,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b27575bc060ce15f0e5 HTTP/1.1" 502 150 2025-12-17 22:35:31,084 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b27575bc060ce15f0e5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:31,312 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b1b575bc060ce15f0ca 2025-12-17 22:35:31,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1b575bc060ce15f0ca HTTP/1.1" 502 150 2025-12-17 22:35:31,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b1b575bc060ce15f0ca'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:32,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b06575bc060ce15f054 2025-12-17 22:35:32,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b06575bc060ce15f054 HTTP/1.1" 502 150 2025-12-17 22:35:32,665 - lava_callback - ERROR - Error processing callback for node 69408b06575bc060ce15f054: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b06575bc060ce15f054 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b06575bc060ce15f054 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b06575bc060ce15f054 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:32,667 - lava_callback - INFO - Starting async job submit for node 69408b2a575bc060ce15f0ed 2025-12-17 22:35:32,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:32,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2a575bc060ce15f0ed HTTP/1.1" 502 150 2025-12-17 22:35:32,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b2a575bc060ce15f0ed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:32,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b2a575bc060ce15f0ed 2025-12-17 22:35:32,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2a575bc060ce15f0ed HTTP/1.1" 502 150 2025-12-17 22:35:32,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b2a575bc060ce15f0ed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:33,059 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b21575bc060ce15f0d3 2025-12-17 22:35:33,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b21575bc060ce15f0d3 HTTP/1.1" 502 150 2025-12-17 22:35:33,061 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b21575bc060ce15f0d3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:33,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b08575bc060ce15f057 2025-12-17 22:35:34,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b2a575bc060ce15f0ed 2025-12-17 22:35:34,732 - lava_callback - INFO - Received callback for node 69408e4e575bc060ce15fda9 INFO: 172.21.0.1:35444 - "POST /node/69408e4e575bc060ce15fda9 HTTP/1.0" 202 Accepted 2025-12-17 22:35:34,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b2c575bc060ce15f0f0 2025-12-17 22:35:34,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b0a575bc060ce15f05a 2025-12-17 22:35:34,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b0d575bc060ce15f07c 2025-12-17 22:35:35,084 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b27575bc060ce15f0e5 2025-12-17 22:35:35,140 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940e7cf460bc8e2651142e7 2025-12-17 22:35:35,169 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b10575bc060ce15f0a5 2025-12-17 22:35:36,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b14575bc060ce15f0b6 2025-12-17 22:35:36,311 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b17575bc060ce15f0c4 INFO: 172.21.0.1:35458 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:35:36,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b1d575bc060ce15f0cd 2025-12-17 22:35:37,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b1f575bc060ce15f0d0 2025-12-17 22:35:41,061 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b21575bc060ce15f0d3 2025-12-17 22:35:45,082 - lava_callback - INFO - Received callback for node 69408d2d575bc060ce15f7ad INFO: 172.21.0.1:50256 - "POST /node/69408d2d575bc060ce15f7ad HTTP/1.0" 202 Accepted 2025-12-17 22:35:45,734 - lava_callback - INFO - Received callback for node 69408d2f575bc060ce15f7b4 INFO: 172.21.0.1:50260 - "POST /node/69408d2f575bc060ce15f7b4 HTTP/1.0" 202 Accepted 2025-12-17 22:35:46,373 - lava_callback - INFO - Received callback for node 69408d31575bc060ce15f7b7 INFO: 172.21.0.1:50268 - "POST /node/69408d31575bc060ce15f7b7 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:50282 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:35:46,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b19575bc060ce15f0c7 2025-12-17 22:35:47,021 - lava_callback - INFO - Received callback for node 69408d33575bc060ce15f7ba INFO: 172.21.0.1:50298 - "POST /node/69408d33575bc060ce15f7ba HTTP/1.0" 202 Accepted 2025-12-17 22:35:47,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b1b575bc060ce15f0ca 2025-12-17 22:35:47,668 - lava_callback - INFO - Received callback for node 69408d35575bc060ce15f7bd INFO: 172.21.0.1:50302 - "POST /node/69408d35575bc060ce15f7bd HTTP/1.0" 202 Accepted 2025-12-17 22:35:48,322 - lava_callback - INFO - Received callback for node 69408d37575bc060ce15f7c0 INFO: 172.21.0.1:50314 - "POST /node/69408d37575bc060ce15f7c0 HTTP/1.0" 202 Accepted 2025-12-17 22:35:48,973 - lava_callback - INFO - Received callback for node 69408d3c575bc060ce15f7e8 INFO: 172.21.0.1:50322 - "POST /node/69408d3c575bc060ce15f7e8 HTTP/1.0" 202 Accepted 2025-12-17 22:35:49,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b08575bc060ce15f057 HTTP/1.1" 502 150 2025-12-17 22:35:49,040 - lava_callback - ERROR - Error processing callback for node 69408b08575bc060ce15f057: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b08575bc060ce15f057 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b08575bc060ce15f057 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b08575bc060ce15f057 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:49,042 - lava_callback - INFO - Starting async job submit for node 69408b2f575bc060ce15f0fb 2025-12-17 22:35:49,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:49,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2f575bc060ce15f0fb HTTP/1.1" 502 150 2025-12-17 22:35:49,063 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b2f575bc060ce15f0fb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:49,063 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b2f575bc060ce15f0fb 2025-12-17 22:35:49,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2f575bc060ce15f0fb HTTP/1.1" 502 150 2025-12-17 22:35:49,063 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b2f575bc060ce15f0fb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:49,619 - lava_callback - INFO - Received callback for node 69408d3f575bc060ce15f7eb INFO: 172.21.0.1:50330 - "POST /node/69408d3f575bc060ce15f7eb HTTP/1.0" 202 Accepted 2025-12-17 22:35:50,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2c575bc060ce15f0f0 HTTP/1.1" 502 150 2025-12-17 22:35:50,064 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b2c575bc060ce15f0f0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:50,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b0d575bc060ce15f07c HTTP/1.1" 502 150 2025-12-17 22:35:50,320 - lava_callback - ERROR - Error processing callback for node 69408b0d575bc060ce15f07c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b0d575bc060ce15f07c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b0d575bc060ce15f07c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b0d575bc060ce15f07c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:50,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b0a575bc060ce15f05a HTTP/1.1" 502 150 2025-12-17 22:35:50,322 - lava_callback - ERROR - Error processing callback for node 69408b0a575bc060ce15f05a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b0a575bc060ce15f05a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b0a575bc060ce15f05a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b0a575bc060ce15f05a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:50,323 - lava_callback - INFO - Starting async job submit for node 69408b33575bc060ce15f101 2025-12-17 22:35:50,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:50,325 - lava_callback - INFO - Starting async job submit for node 69408b37575bc060ce15f107 2025-12-17 22:35:50,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:50,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b33575bc060ce15f101 HTTP/1.1" 502 150 2025-12-17 22:35:50,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b33575bc060ce15f101'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:50,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b33575bc060ce15f101 2025-12-17 22:35:50,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b33575bc060ce15f101 HTTP/1.1" 502 150 2025-12-17 22:35:50,333 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b33575bc060ce15f101'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:50,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b37575bc060ce15f107 HTTP/1.1" 502 150 2025-12-17 22:35:50,333 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b37575bc060ce15f107'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:50,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b37575bc060ce15f107 2025-12-17 22:35:50,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b37575bc060ce15f107 HTTP/1.1" 502 150 2025-12-17 22:35:50,334 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b37575bc060ce15f107'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:50,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940e7cf460bc8e2651142e7 HTTP/1.1" 502 150 2025-12-17 22:35:50,577 - lava_callback - ERROR - Error processing callback for node 6940e7cf460bc8e2651142e7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940e7cf460bc8e2651142e7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940e7cf460bc8e2651142e7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940e7cf460bc8e2651142e7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:50,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b10575bc060ce15f0a5 HTTP/1.1" 502 150 2025-12-17 22:35:50,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b27575bc060ce15f0e5 HTTP/1.1" 502 150 2025-12-17 22:35:50,579 - lava_callback - INFO - Starting async job submit for node 69409228575bc060ce160a0c 2025-12-17 22:35:50,580 - lava_callback - ERROR - Error processing callback for node 69408b10575bc060ce15f0a5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b10575bc060ce15f0a5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b10575bc060ce15f0a5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b10575bc060ce15f0a5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:50,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b27575bc060ce15f0e5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:50,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:50,607 - lava_callback - INFO - Starting async job submit for node 69408b31575bc060ce15f0fe 2025-12-17 22:35:50,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:50,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409228575bc060ce160a0c HTTP/1.1" 502 150 2025-12-17 22:35:50,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409228575bc060ce160a0c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:50,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409228575bc060ce160a0c 2025-12-17 22:35:50,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409228575bc060ce160a0c HTTP/1.1" 502 150 2025-12-17 22:35:50,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409228575bc060ce160a0c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:50,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b31575bc060ce15f0fe HTTP/1.1" 502 150 2025-12-17 22:35:50,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b31575bc060ce15f0fe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:50,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b31575bc060ce15f0fe 2025-12-17 22:35:50,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b31575bc060ce15f0fe HTTP/1.1" 502 150 2025-12-17 22:35:50,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b31575bc060ce15f0fe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:51,064 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b2f575bc060ce15f0fb 2025-12-17 22:35:51,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2f575bc060ce15f0fb HTTP/1.1" 502 150 2025-12-17 22:35:51,065 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b2f575bc060ce15f0fb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:51,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b14575bc060ce15f0b6 HTTP/1.1" 502 150 2025-12-17 22:35:51,344 - lava_callback - ERROR - Error processing callback for node 69408b14575bc060ce15f0b6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b14575bc060ce15f0b6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b14575bc060ce15f0b6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b14575bc060ce15f0b6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:51,346 - lava_callback - INFO - Starting async job submit for node 69408b35575bc060ce15f104 2025-12-17 22:35:51,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:51,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b35575bc060ce15f104 HTTP/1.1" 502 150 2025-12-17 22:35:51,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b35575bc060ce15f104'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:51,354 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b35575bc060ce15f104 2025-12-17 22:35:51,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b35575bc060ce15f104 HTTP/1.1" 502 150 2025-12-17 22:35:51,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b35575bc060ce15f104'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:51,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b17575bc060ce15f0c4 HTTP/1.1" 502 150 2025-12-17 22:35:51,601 - lava_callback - ERROR - Error processing callback for node 69408b17575bc060ce15f0c4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b17575bc060ce15f0c4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b17575bc060ce15f0c4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b17575bc060ce15f0c4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:51,602 - lava_callback - INFO - Starting async job submit for node 69408b39575bc060ce15f10e 2025-12-17 22:35:51,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:51,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b39575bc060ce15f10e HTTP/1.1" 502 150 2025-12-17 22:35:51,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b39575bc060ce15f10e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:51,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b39575bc060ce15f10e 2025-12-17 22:35:51,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b39575bc060ce15f10e HTTP/1.1" 502 150 2025-12-17 22:35:51,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b39575bc060ce15f10e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:52,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1d575bc060ce15f0cd HTTP/1.1" 502 150 2025-12-17 22:35:52,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b1d575bc060ce15f0cd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:52,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b33575bc060ce15f101 2025-12-17 22:35:52,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b33575bc060ce15f101 HTTP/1.1" 502 150 2025-12-17 22:35:52,334 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b33575bc060ce15f101'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:52,334 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b37575bc060ce15f107 2025-12-17 22:35:52,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b37575bc060ce15f107 HTTP/1.1" 502 150 2025-12-17 22:35:52,335 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b37575bc060ce15f107'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:52,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1f575bc060ce15f0d0 HTTP/1.1" 502 150 2025-12-17 22:35:52,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b1f575bc060ce15f0d0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:52,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409228575bc060ce160a0c 2025-12-17 22:35:52,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409228575bc060ce160a0c HTTP/1.1" 502 150 2025-12-17 22:35:52,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409228575bc060ce160a0c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:52,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b31575bc060ce15f0fe 2025-12-17 22:35:52,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b31575bc060ce15f0fe HTTP/1.1" 502 150 2025-12-17 22:35:52,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b31575bc060ce15f0fe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:53,355 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b35575bc060ce15f104 2025-12-17 22:35:53,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b35575bc060ce15f104 HTTP/1.1" 502 150 2025-12-17 22:35:53,356 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b35575bc060ce15f104'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:53,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b39575bc060ce15f10e 2025-12-17 22:35:53,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b39575bc060ce15f10e HTTP/1.1" 502 150 2025-12-17 22:35:53,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b39575bc060ce15f10e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:54,923 - lava_callback - INFO - Received callback for node 69408d3a575bc060ce15f7e5 INFO: 172.21.0.1:36728 - "POST /node/69408d3a575bc060ce15f7e5 HTTP/1.0" 202 Accepted 2025-12-17 22:35:55,065 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b2f575bc060ce15f0fb 2025-12-17 22:35:55,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2f575bc060ce15f0fb HTTP/1.1" 502 150 2025-12-17 22:35:55,066 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b2f575bc060ce15f0fb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:55,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940879d575bc060ce15e427 HTTP/1.1" 502 150 2025-12-17 22:35:55,185 - lava_callback - ERROR - Error processing callback for node 6940879d575bc060ce15e427: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940879d575bc060ce15e427 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940879d575bc060ce15e427 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940879d575bc060ce15e427 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:35:55,188 - lava_callback - INFO - Starting async job submit for node 69408b3b575bc060ce15f115 2025-12-17 22:35:55,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:35:55,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3b575bc060ce15f115 HTTP/1.1" 502 150 2025-12-17 22:35:55,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b3b575bc060ce15f115'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:55,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b3b575bc060ce15f115 2025-12-17 22:35:55,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3b575bc060ce15f115 HTTP/1.1" 502 150 2025-12-17 22:35:55,201 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b3b575bc060ce15f115'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:56,334 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b33575bc060ce15f101 2025-12-17 22:35:56,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b33575bc060ce15f101 HTTP/1.1" 502 150 2025-12-17 22:35:56,335 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b33575bc060ce15f101'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:56,336 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b37575bc060ce15f107 2025-12-17 22:35:56,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b37575bc060ce15f107 HTTP/1.1" 502 150 2025-12-17 22:35:56,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b37575bc060ce15f107'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:56,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b21575bc060ce15f0d3 HTTP/1.1" 502 150 2025-12-17 22:35:56,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b21575bc060ce15f0d3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36738 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:35:56,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409228575bc060ce160a0c 2025-12-17 22:35:56,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b31575bc060ce15f0fe 2025-12-17 22:35:56,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409228575bc060ce160a0c HTTP/1.1" 502 150 2025-12-17 22:35:56,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409228575bc060ce160a0c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:56,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b31575bc060ce15f0fe HTTP/1.1" 502 150 2025-12-17 22:35:56,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b31575bc060ce15f0fe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:57,201 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b3b575bc060ce15f115 2025-12-17 22:35:57,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3b575bc060ce15f115 HTTP/1.1" 502 150 2025-12-17 22:35:57,202 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b3b575bc060ce15f115'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:57,357 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b35575bc060ce15f104 2025-12-17 22:35:57,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b35575bc060ce15f104 HTTP/1.1" 502 150 2025-12-17 22:35:57,358 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b35575bc060ce15f104'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:57,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b39575bc060ce15f10e 2025-12-17 22:35:57,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b39575bc060ce15f10e HTTP/1.1" 502 150 2025-12-17 22:35:57,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b39575bc060ce15f10e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:58,065 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b2c575bc060ce15f0f0 2025-12-17 22:35:58,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2c575bc060ce15f0f0 HTTP/1.1" 502 150 2025-12-17 22:35:58,066 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b2c575bc060ce15f0f0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:35:58,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b27575bc060ce15f0e5 2025-12-17 22:35:58,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b27575bc060ce15f0e5 HTTP/1.1" 502 150 2025-12-17 22:35:58,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b27575bc060ce15f0e5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:01,202 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b3b575bc060ce15f115 2025-12-17 22:36:02,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b19575bc060ce15f0c7 HTTP/1.1" 502 150 2025-12-17 22:36:02,096 - lava_callback - ERROR - Error processing callback for node 69408b19575bc060ce15f0c7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b19575bc060ce15f0c7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b19575bc060ce15f0c7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b19575bc060ce15f0c7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:02,098 - lava_callback - INFO - Starting async job submit for node 69408b3d575bc060ce15f11e 2025-12-17 22:36:02,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:02,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3d575bc060ce15f11e HTTP/1.1" 502 150 2025-12-17 22:36:02,109 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b3d575bc060ce15f11e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:02,109 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b3d575bc060ce15f11e 2025-12-17 22:36:02,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3d575bc060ce15f11e HTTP/1.1" 502 150 2025-12-17 22:36:02,110 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b3d575bc060ce15f11e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:02,196 - lava_callback - INFO - Received callback for node 6940f6b4460bc8e265115822 INFO: 172.21.0.1:56518 - "POST /node/6940f6b4460bc8e265115822 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:56524 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:36:02,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1b575bc060ce15f0ca HTTP/1.1" 502 150 2025-12-17 22:36:02,612 - lava_callback - ERROR - Error processing callback for node 69408b1b575bc060ce15f0ca: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b1b575bc060ce15f0ca (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b1b575bc060ce15f0ca (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b1b575bc060ce15f0ca (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:02,614 - lava_callback - INFO - Starting async job submit for node 69408b3f575bc060ce15f121 2025-12-17 22:36:02,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:02,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3f575bc060ce15f121 HTTP/1.1" 502 150 2025-12-17 22:36:02,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b3f575bc060ce15f121'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:02,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b3f575bc060ce15f121 2025-12-17 22:36:02,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3f575bc060ce15f121 HTTP/1.1" 502 150 2025-12-17 22:36:02,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b3f575bc060ce15f121'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:03,067 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b2f575bc060ce15f0fb 2025-12-17 22:36:03,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2f575bc060ce15f0fb HTTP/1.1" 502 150 2025-12-17 22:36:03,068 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b2f575bc060ce15f0fb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:04,110 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b3d575bc060ce15f11e 2025-12-17 22:36:04,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3d575bc060ce15f11e HTTP/1.1" 502 150 2025-12-17 22:36:04,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b3d575bc060ce15f11e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:04,336 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b33575bc060ce15f101 2025-12-17 22:36:04,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b33575bc060ce15f101 HTTP/1.1" 502 150 2025-12-17 22:36:04,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b33575bc060ce15f101'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:04,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b37575bc060ce15f107 2025-12-17 22:36:04,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b37575bc060ce15f107 HTTP/1.1" 502 150 2025-12-17 22:36:04,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b37575bc060ce15f107'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:04,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409228575bc060ce160a0c 2025-12-17 22:36:04,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b31575bc060ce15f0fe 2025-12-17 22:36:04,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b31575bc060ce15f0fe HTTP/1.1" 502 150 2025-12-17 22:36:04,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b31575bc060ce15f0fe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:04,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409228575bc060ce160a0c HTTP/1.1" 502 150 2025-12-17 22:36:04,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409228575bc060ce160a0c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:04,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b3f575bc060ce15f121 2025-12-17 22:36:04,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3f575bc060ce15f121 HTTP/1.1" 502 150 2025-12-17 22:36:04,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b3f575bc060ce15f121'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:05,072 - lava_callback - INFO - Received callback for node 69408d53575bc060ce15f814 INFO: 172.21.0.1:56536 - "POST /node/69408d53575bc060ce15f814 HTTP/1.0" 202 Accepted 2025-12-17 22:36:05,358 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b35575bc060ce15f104 2025-12-17 22:36:05,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b35575bc060ce15f104 HTTP/1.1" 502 150 2025-12-17 22:36:05,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b35575bc060ce15f104'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:05,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2a575bc060ce15f0ed HTTP/1.1" 502 150 2025-12-17 22:36:05,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b2a575bc060ce15f0ed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:05,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b39575bc060ce15f10e 2025-12-17 22:36:05,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b39575bc060ce15f10e HTTP/1.1" 502 150 2025-12-17 22:36:05,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b39575bc060ce15f10e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56538 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:36:08,112 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b3d575bc060ce15f11e 2025-12-17 22:36:08,113 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b1d575bc060ce15f0cd 2025-12-17 22:36:08,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3d575bc060ce15f11e HTTP/1.1" 502 150 2025-12-17 22:36:08,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1d575bc060ce15f0cd HTTP/1.1" 502 150 2025-12-17 22:36:08,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b3d575bc060ce15f11e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:08,115 - lava_callback - ERROR - Error processing callback for node 69408b1d575bc060ce15f0cd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b1d575bc060ce15f0cd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b1d575bc060ce15f0cd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b1d575bc060ce15f0cd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:08,117 - lava_callback - INFO - Starting async job submit for node 69408b41575bc060ce15f124 2025-12-17 22:36:08,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:08,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b41575bc060ce15f124 HTTP/1.1" 502 150 2025-12-17 22:36:08,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b41575bc060ce15f124'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:08,123 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b41575bc060ce15f124 2025-12-17 22:36:08,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b41575bc060ce15f124 HTTP/1.1" 502 150 2025-12-17 22:36:08,124 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b41575bc060ce15f124'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:08,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b1f575bc060ce15f0d0 2025-12-17 22:36:08,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b1f575bc060ce15f0d0 HTTP/1.1" 502 150 2025-12-17 22:36:08,370 - lava_callback - ERROR - Error processing callback for node 69408b1f575bc060ce15f0d0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b1f575bc060ce15f0d0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b1f575bc060ce15f0d0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b1f575bc060ce15f0d0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:08,371 - lava_callback - INFO - Starting async job submit for node 69408b43575bc060ce15f127 2025-12-17 22:36:08,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:08,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b43575bc060ce15f127 HTTP/1.1" 502 150 2025-12-17 22:36:08,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b43575bc060ce15f127'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:08,381 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b43575bc060ce15f127 2025-12-17 22:36:08,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b43575bc060ce15f127 HTTP/1.1" 502 150 2025-12-17 22:36:08,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b43575bc060ce15f127'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:08,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b3f575bc060ce15f121 2025-12-17 22:36:08,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3f575bc060ce15f121 HTTP/1.1" 502 150 2025-12-17 22:36:08,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b3f575bc060ce15f121'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:09,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b2a575bc060ce15f0ed 2025-12-17 22:36:09,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2a575bc060ce15f0ed HTTP/1.1" 502 150 2025-12-17 22:36:09,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b2a575bc060ce15f0ed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:10,124 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b41575bc060ce15f124 2025-12-17 22:36:10,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b41575bc060ce15f124 HTTP/1.1" 502 150 2025-12-17 22:36:10,125 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b41575bc060ce15f124'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:10,381 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b43575bc060ce15f127 2025-12-17 22:36:10,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b43575bc060ce15f127 HTTP/1.1" 502 150 2025-12-17 22:36:10,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b43575bc060ce15f127'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:12,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b21575bc060ce15f0d3 2025-12-17 22:36:12,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b21575bc060ce15f0d3 HTTP/1.1" 502 150 2025-12-17 22:36:12,466 - lava_callback - ERROR - Error processing callback for node 69408b21575bc060ce15f0d3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b21575bc060ce15f0d3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b21575bc060ce15f0d3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b21575bc060ce15f0d3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:12,468 - lava_callback - INFO - Starting async job submit for node 69408b45575bc060ce15f12a 2025-12-17 22:36:12,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:12,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b45575bc060ce15f12a HTTP/1.1" 502 150 2025-12-17 22:36:12,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b45575bc060ce15f12a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:12,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b45575bc060ce15f12a 2025-12-17 22:36:12,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b45575bc060ce15f12a HTTP/1.1" 502 150 2025-12-17 22:36:12,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b45575bc060ce15f12a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:14,066 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b2c575bc060ce15f0f0 2025-12-17 22:36:14,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2c575bc060ce15f0f0 HTTP/1.1" 502 150 2025-12-17 22:36:14,068 - lava_callback - ERROR - Error processing callback for node 69408b2c575bc060ce15f0f0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b2c575bc060ce15f0f0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b2c575bc060ce15f0f0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b2c575bc060ce15f0f0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:14,070 - lava_callback - INFO - Starting async job submit for node 69408b47575bc060ce15f12d 2025-12-17 22:36:14,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:14,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b47575bc060ce15f12d HTTP/1.1" 502 150 2025-12-17 22:36:14,104 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b47575bc060ce15f12d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:14,104 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b47575bc060ce15f12d 2025-12-17 22:36:14,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b47575bc060ce15f12d HTTP/1.1" 502 150 2025-12-17 22:36:14,105 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b47575bc060ce15f12d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:14,126 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b41575bc060ce15f124 2025-12-17 22:36:14,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b41575bc060ce15f124 HTTP/1.1" 502 150 2025-12-17 22:36:14,127 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b41575bc060ce15f124'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:14,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b43575bc060ce15f127 2025-12-17 22:36:14,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b43575bc060ce15f127 HTTP/1.1" 502 150 2025-12-17 22:36:14,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b43575bc060ce15f127'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:14,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b45575bc060ce15f12a 2025-12-17 22:36:14,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b45575bc060ce15f12a HTTP/1.1" 502 150 2025-12-17 22:36:14,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b45575bc060ce15f12a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:14,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b27575bc060ce15f0e5 2025-12-17 22:36:14,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b27575bc060ce15f0e5 HTTP/1.1" 502 150 2025-12-17 22:36:14,610 - lava_callback - ERROR - Error processing callback for node 69408b27575bc060ce15f0e5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b27575bc060ce15f0e5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b27575bc060ce15f0e5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b27575bc060ce15f0e5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:14,612 - lava_callback - INFO - Starting async job submit for node 69408b49575bc060ce15f130 2025-12-17 22:36:14,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:14,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b49575bc060ce15f130 HTTP/1.1" 502 150 2025-12-17 22:36:14,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b49575bc060ce15f130'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:14,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b49575bc060ce15f130 2025-12-17 22:36:14,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b49575bc060ce15f130 HTTP/1.1" 502 150 2025-12-17 22:36:14,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b49575bc060ce15f130'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36892 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:36:16,105 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b47575bc060ce15f12d 2025-12-17 22:36:16,115 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b3d575bc060ce15f11e 2025-12-17 22:36:16,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3d575bc060ce15f11e HTTP/1.1" 502 150 2025-12-17 22:36:16,116 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b3d575bc060ce15f11e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36908 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:36:16,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b3f575bc060ce15f121 2025-12-17 22:36:16,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3f575bc060ce15f121 HTTP/1.1" 502 150 2025-12-17 22:36:16,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b3f575bc060ce15f121'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:16,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b49575bc060ce15f130 2025-12-17 22:36:16,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b49575bc060ce15f130 HTTP/1.1" 502 150 2025-12-17 22:36:16,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b49575bc060ce15f130'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:16,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3b575bc060ce15f115 HTTP/1.1" 502 150 2025-12-17 22:36:16,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b3b575bc060ce15f115'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:17,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b2a575bc060ce15f0ed 2025-12-17 22:36:17,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2a575bc060ce15f0ed HTTP/1.1" 502 150 2025-12-17 22:36:17,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b2a575bc060ce15f0ed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:18,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b45575bc060ce15f12a 2025-12-17 22:36:18,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b45575bc060ce15f12a HTTP/1.1" 502 150 2025-12-17 22:36:18,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b45575bc060ce15f12a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:19,068 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b2f575bc060ce15f0fb 2025-12-17 22:36:19,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2f575bc060ce15f0fb HTTP/1.1" 502 150 2025-12-17 22:36:19,070 - lava_callback - ERROR - Error processing callback for node 69408b2f575bc060ce15f0fb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b2f575bc060ce15f0fb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b2f575bc060ce15f0fb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b2f575bc060ce15f0fb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:19,072 - lava_callback - INFO - Starting async job submit for node 69408b51575bc060ce15f193 2025-12-17 22:36:19,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:19,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b51575bc060ce15f193 HTTP/1.1" 502 150 2025-12-17 22:36:19,078 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b51575bc060ce15f193'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:19,079 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b51575bc060ce15f193 2025-12-17 22:36:19,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b51575bc060ce15f193 HTTP/1.1" 502 150 2025-12-17 22:36:19,079 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b51575bc060ce15f193'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:20,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b33575bc060ce15f101 2025-12-17 22:36:20,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b33575bc060ce15f101 HTTP/1.1" 502 150 2025-12-17 22:36:20,339 - lava_callback - ERROR - Error processing callback for node 69408b33575bc060ce15f101: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b33575bc060ce15f101 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b33575bc060ce15f101 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b33575bc060ce15f101 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:20,342 - lava_callback - INFO - Starting async job submit for node 69408b4d575bc060ce15f17a 2025-12-17 22:36:20,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:20,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b37575bc060ce15f107 2025-12-17 22:36:20,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b37575bc060ce15f107 HTTP/1.1" 502 150 2025-12-17 22:36:20,348 - lava_callback - ERROR - Error processing callback for node 69408b37575bc060ce15f107: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b37575bc060ce15f107 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b37575bc060ce15f107 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b37575bc060ce15f107 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:20,352 - lava_callback - INFO - Starting async job submit for node 69408b4f575bc060ce15f18a 2025-12-17 22:36:20,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:20,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b4d575bc060ce15f17a HTTP/1.1" 502 150 2025-12-17 22:36:20,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b4d575bc060ce15f17a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:20,357 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b4d575bc060ce15f17a 2025-12-17 22:36:20,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b4d575bc060ce15f17a HTTP/1.1" 502 150 2025-12-17 22:36:20,358 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b4d575bc060ce15f17a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:20,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b4f575bc060ce15f18a HTTP/1.1" 502 150 2025-12-17 22:36:20,364 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b4f575bc060ce15f18a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:20,364 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b4f575bc060ce15f18a 2025-12-17 22:36:20,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b4f575bc060ce15f18a HTTP/1.1" 502 150 2025-12-17 22:36:20,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b4f575bc060ce15f18a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:20,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b31575bc060ce15f0fe 2025-12-17 22:36:20,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409228575bc060ce160a0c 2025-12-17 22:36:20,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b31575bc060ce15f0fe HTTP/1.1" 502 150 2025-12-17 22:36:20,623 - lava_callback - ERROR - Error processing callback for node 69408b31575bc060ce15f0fe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b31575bc060ce15f0fe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b31575bc060ce15f0fe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b31575bc060ce15f0fe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:20,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409228575bc060ce160a0c HTTP/1.1" 502 150 2025-12-17 22:36:20,625 - lava_callback - ERROR - Error processing callback for node 69409228575bc060ce160a0c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409228575bc060ce160a0c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409228575bc060ce160a0c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409228575bc060ce160a0c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:20,626 - lava_callback - INFO - Starting async job submit for node 69408b53575bc060ce15f19c 2025-12-17 22:36:20,627 - lava_callback - INFO - Starting async job submit for node 69408b57575bc060ce15f1a4 2025-12-17 22:36:20,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:20,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:20,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b57575bc060ce15f1a4 HTTP/1.1" 502 150 2025-12-17 22:36:20,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b57575bc060ce15f1a4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:20,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b57575bc060ce15f1a4 2025-12-17 22:36:20,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b57575bc060ce15f1a4 HTTP/1.1" 502 150 2025-12-17 22:36:20,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b57575bc060ce15f1a4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:20,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b49575bc060ce15f130 2025-12-17 22:36:20,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b49575bc060ce15f130 HTTP/1.1" 502 150 2025-12-17 22:36:20,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b49575bc060ce15f130'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:20,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b53575bc060ce15f19c HTTP/1.1" 502 150 2025-12-17 22:36:20,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b53575bc060ce15f19c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:20,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b53575bc060ce15f19c 2025-12-17 22:36:20,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b53575bc060ce15f19c HTTP/1.1" 502 150 2025-12-17 22:36:20,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b53575bc060ce15f19c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:21,080 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b51575bc060ce15f193 2025-12-17 22:36:21,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b51575bc060ce15f193 HTTP/1.1" 502 150 2025-12-17 22:36:21,081 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b51575bc060ce15f193'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:21,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b35575bc060ce15f104 2025-12-17 22:36:21,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b35575bc060ce15f104 HTTP/1.1" 502 150 2025-12-17 22:36:21,362 - lava_callback - ERROR - Error processing callback for node 69408b35575bc060ce15f104: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b35575bc060ce15f104 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b35575bc060ce15f104 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b35575bc060ce15f104 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:21,364 - lava_callback - INFO - Starting async job submit for node 69408b59575bc060ce15f1a7 2025-12-17 22:36:21,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:21,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b59575bc060ce15f1a7 HTTP/1.1" 502 150 2025-12-17 22:36:21,377 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b59575bc060ce15f1a7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:21,377 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b59575bc060ce15f1a7 2025-12-17 22:36:21,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b59575bc060ce15f1a7 HTTP/1.1" 502 150 2025-12-17 22:36:21,378 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b59575bc060ce15f1a7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:21,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b39575bc060ce15f10e 2025-12-17 22:36:21,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b39575bc060ce15f10e HTTP/1.1" 502 150 2025-12-17 22:36:21,640 - lava_callback - ERROR - Error processing callback for node 69408b39575bc060ce15f10e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b39575bc060ce15f10e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b39575bc060ce15f10e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b39575bc060ce15f10e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:21,642 - lava_callback - INFO - Starting async job submit for node 69408b56575bc060ce15f1a1 2025-12-17 22:36:21,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:21,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b56575bc060ce15f1a1 HTTP/1.1" 502 150 2025-12-17 22:36:21,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b56575bc060ce15f1a1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:21,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b56575bc060ce15f1a1 2025-12-17 22:36:21,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b56575bc060ce15f1a1 HTTP/1.1" 502 150 2025-12-17 22:36:21,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b56575bc060ce15f1a1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:22,127 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b41575bc060ce15f124 2025-12-17 22:36:22,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b41575bc060ce15f124 HTTP/1.1" 502 150 2025-12-17 22:36:22,128 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b41575bc060ce15f124'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:22,358 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b4d575bc060ce15f17a 2025-12-17 22:36:22,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b4d575bc060ce15f17a HTTP/1.1" 502 150 2025-12-17 22:36:22,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b4d575bc060ce15f17a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:22,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b4f575bc060ce15f18a 2025-12-17 22:36:22,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b4f575bc060ce15f18a HTTP/1.1" 502 150 2025-12-17 22:36:22,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b4f575bc060ce15f18a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:22,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b43575bc060ce15f127 2025-12-17 22:36:22,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b43575bc060ce15f127 HTTP/1.1" 502 150 2025-12-17 22:36:22,385 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b43575bc060ce15f127'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:22,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b57575bc060ce15f1a4 2025-12-17 22:36:22,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b57575bc060ce15f1a4 HTTP/1.1" 502 150 2025-12-17 22:36:22,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b57575bc060ce15f1a4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:22,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b53575bc060ce15f19c 2025-12-17 22:36:22,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b53575bc060ce15f19c HTTP/1.1" 502 150 2025-12-17 22:36:22,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b53575bc060ce15f19c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:22,855 - lava_callback - INFO - Received callback for node 69408c89575bc060ce15f5f6 INFO: 172.21.0.1:44232 - "POST /node/69408c89575bc060ce15f5f6 HTTP/1.0" 202 Accepted 2025-12-17 22:36:23,378 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b59575bc060ce15f1a7 2025-12-17 22:36:23,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b59575bc060ce15f1a7 HTTP/1.1" 502 150 2025-12-17 22:36:23,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b59575bc060ce15f1a7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:23,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b56575bc060ce15f1a1 2025-12-17 22:36:23,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b56575bc060ce15f1a1 HTTP/1.1" 502 150 2025-12-17 22:36:23,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b56575bc060ce15f1a1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:24,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b3b575bc060ce15f115 2025-12-17 22:36:25,081 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b51575bc060ce15f193 2025-12-17 22:36:25,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b51575bc060ce15f193 HTTP/1.1" 502 150 2025-12-17 22:36:25,082 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b51575bc060ce15f193'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:25,088 - lava_callback - INFO - Received callback for node 69408d55575bc060ce15f81d INFO: 172.21.0.1:44242 - "POST /node/69408d55575bc060ce15f81d HTTP/1.0" 202 Accepted 2025-12-17 22:36:25,736 - lava_callback - INFO - Received callback for node 69408d57575bc060ce15f826 INFO: 172.21.0.1:44250 - "POST /node/69408d57575bc060ce15f826 HTTP/1.0" 202 Accepted 2025-12-17 22:36:26,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b4d575bc060ce15f17a 2025-12-17 22:36:26,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b4d575bc060ce15f17a HTTP/1.1" 502 150 2025-12-17 22:36:26,361 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b4d575bc060ce15f17a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:26,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b4f575bc060ce15f18a 2025-12-17 22:36:26,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b4f575bc060ce15f18a HTTP/1.1" 502 150 2025-12-17 22:36:26,367 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b4f575bc060ce15f18a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:26,392 - lava_callback - INFO - Received callback for node 69408d59575bc060ce15f829 INFO: 172.21.0.1:44258 - "POST /node/69408d59575bc060ce15f829 HTTP/1.0" 202 Accepted 2025-12-17 22:36:26,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b45575bc060ce15f12a 2025-12-17 22:36:26,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b45575bc060ce15f12a HTTP/1.1" 502 150 2025-12-17 22:36:26,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b45575bc060ce15f12a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44272 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:36:26,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b57575bc060ce15f1a4 2025-12-17 22:36:26,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b57575bc060ce15f1a4 HTTP/1.1" 502 150 2025-12-17 22:36:26,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b57575bc060ce15f1a4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:26,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b53575bc060ce15f19c 2025-12-17 22:36:26,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b53575bc060ce15f19c HTTP/1.1" 502 150 2025-12-17 22:36:26,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b53575bc060ce15f19c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:27,028 - lava_callback - INFO - Received callback for node 69408d5b575bc060ce15f82c INFO: 172.21.0.1:44282 - "POST /node/69408d5b575bc060ce15f82c HTTP/1.0" 202 Accepted 2025-12-17 22:36:27,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b59575bc060ce15f1a7 2025-12-17 22:36:27,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b56575bc060ce15f1a1 2025-12-17 22:36:27,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b56575bc060ce15f1a1 HTTP/1.1" 502 150 2025-12-17 22:36:27,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b56575bc060ce15f1a1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:27,692 - lava_callback - INFO - Received callback for node 69408d5d575bc060ce15f82f INFO: 172.21.0.1:44296 - "POST /node/69408d5d575bc060ce15f82f HTTP/1.0" 202 Accepted 2025-12-17 22:36:28,340 - lava_callback - INFO - Received callback for node 69408d5f575bc060ce15f832 INFO: 172.21.0.1:44312 - "POST /node/69408d5f575bc060ce15f832 HTTP/1.0" 202 Accepted 2025-12-17 22:36:28,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b49575bc060ce15f130 2025-12-17 22:36:28,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b49575bc060ce15f130 HTTP/1.1" 502 150 2025-12-17 22:36:28,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b49575bc060ce15f130'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:28,977 - lava_callback - INFO - Received callback for node 69408d61575bc060ce15f835 INFO: 172.21.0.1:44322 - "POST /node/69408d61575bc060ce15f835 HTTP/1.0" 202 Accepted 2025-12-17 22:36:29,614 - lava_callback - INFO - Received callback for node 69408d63575bc060ce15f838 INFO: 172.21.0.1:44324 - "POST /node/69408d63575bc060ce15f838 HTTP/1.0" 202 Accepted 2025-12-17 22:36:30,259 - lava_callback - INFO - Received callback for node 69408d65575bc060ce15f83b INFO: 172.21.0.1:51748 - "POST /node/69408d65575bc060ce15f83b HTTP/1.0" 202 Accepted 2025-12-17 22:36:31,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b47575bc060ce15f12d HTTP/1.1" 502 150 2025-12-17 22:36:31,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b47575bc060ce15f12d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:32,116 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b3d575bc060ce15f11e 2025-12-17 22:36:32,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3d575bc060ce15f11e HTTP/1.1" 502 150 2025-12-17 22:36:32,118 - lava_callback - ERROR - Error processing callback for node 69408b3d575bc060ce15f11e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b3d575bc060ce15f11e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b3d575bc060ce15f11e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b3d575bc060ce15f11e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:32,119 - lava_callback - INFO - Starting async job submit for node 69409287575bc060ce160adf 2025-12-17 22:36:32,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:32,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409287575bc060ce160adf HTTP/1.1" 502 150 2025-12-17 22:36:32,187 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409287575bc060ce160adf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:32,188 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409287575bc060ce160adf 2025-12-17 22:36:32,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409287575bc060ce160adf HTTP/1.1" 502 150 2025-12-17 22:36:32,188 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409287575bc060ce160adf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:32,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b3f575bc060ce15f121 2025-12-17 22:36:32,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3f575bc060ce15f121 HTTP/1.1" 502 150 2025-12-17 22:36:32,637 - lava_callback - ERROR - Error processing callback for node 69408b3f575bc060ce15f121: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b3f575bc060ce15f121 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b3f575bc060ce15f121 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b3f575bc060ce15f121 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:32,638 - lava_callback - INFO - Starting async job submit for node 69408b5b575bc060ce15f1aa 2025-12-17 22:36:32,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:32,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5b575bc060ce15f1aa HTTP/1.1" 502 150 2025-12-17 22:36:32,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5b575bc060ce15f1aa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:32,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5b575bc060ce15f1aa 2025-12-17 22:36:32,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5b575bc060ce15f1aa HTTP/1.1" 502 150 2025-12-17 22:36:32,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5b575bc060ce15f1aa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:33,083 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b51575bc060ce15f193 2025-12-17 22:36:33,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b51575bc060ce15f193 HTTP/1.1" 502 150 2025-12-17 22:36:33,084 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b51575bc060ce15f193'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:33,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b2a575bc060ce15f0ed 2025-12-17 22:36:33,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b2a575bc060ce15f0ed HTTP/1.1" 502 150 2025-12-17 22:36:33,429 - lava_callback - ERROR - Error processing callback for node 69408b2a575bc060ce15f0ed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b2a575bc060ce15f0ed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b2a575bc060ce15f0ed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b2a575bc060ce15f0ed (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:33,431 - lava_callback - INFO - Starting async job submit for node 69408b5c575bc060ce15f1ad 2025-12-17 22:36:33,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:33,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5c575bc060ce15f1ad HTTP/1.1" 502 150 2025-12-17 22:36:33,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5c575bc060ce15f1ad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:33,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5c575bc060ce15f1ad 2025-12-17 22:36:33,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5c575bc060ce15f1ad HTTP/1.1" 502 150 2025-12-17 22:36:33,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5c575bc060ce15f1ad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:34,189 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409287575bc060ce160adf 2025-12-17 22:36:34,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409287575bc060ce160adf HTTP/1.1" 502 150 2025-12-17 22:36:34,190 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409287575bc060ce160adf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:34,361 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b4d575bc060ce15f17a 2025-12-17 22:36:34,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b4d575bc060ce15f17a HTTP/1.1" 502 150 2025-12-17 22:36:34,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b4d575bc060ce15f17a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:34,367 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b4f575bc060ce15f18a 2025-12-17 22:36:34,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b4f575bc060ce15f18a HTTP/1.1" 502 150 2025-12-17 22:36:34,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b4f575bc060ce15f18a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:34,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b57575bc060ce15f1a4 2025-12-17 22:36:34,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b57575bc060ce15f1a4 HTTP/1.1" 502 150 2025-12-17 22:36:34,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b57575bc060ce15f1a4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:34,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b53575bc060ce15f19c 2025-12-17 22:36:34,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b53575bc060ce15f19c HTTP/1.1" 502 150 2025-12-17 22:36:34,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b53575bc060ce15f19c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:34,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5b575bc060ce15f1aa 2025-12-17 22:36:34,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5b575bc060ce15f1aa HTTP/1.1" 502 150 2025-12-17 22:36:34,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5b575bc060ce15f1aa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:35,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5c575bc060ce15f1ad 2025-12-17 22:36:35,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5c575bc060ce15f1ad HTTP/1.1" 502 150 2025-12-17 22:36:35,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5c575bc060ce15f1ad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:35,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b47575bc060ce15f12d 2025-12-17 22:36:35,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b47575bc060ce15f12d HTTP/1.1" 502 150 2025-12-17 22:36:35,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b47575bc060ce15f12d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:35,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b56575bc060ce15f1a1 2025-12-17 22:36:35,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b56575bc060ce15f1a1 HTTP/1.1" 502 150 2025-12-17 22:36:35,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b56575bc060ce15f1a1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51752 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:36:38,128 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b41575bc060ce15f124 2025-12-17 22:36:38,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b41575bc060ce15f124 HTTP/1.1" 502 150 2025-12-17 22:36:38,130 - lava_callback - ERROR - Error processing callback for node 69408b41575bc060ce15f124: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b41575bc060ce15f124 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b41575bc060ce15f124 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b41575bc060ce15f124 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:38,131 - lava_callback - INFO - Starting async job submit for node 69408b5d575bc060ce15f1b0 2025-12-17 22:36:38,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:38,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5d575bc060ce15f1b0 HTTP/1.1" 502 150 2025-12-17 22:36:38,143 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5d575bc060ce15f1b0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:38,143 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5d575bc060ce15f1b0 2025-12-17 22:36:38,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5d575bc060ce15f1b0 HTTP/1.1" 502 150 2025-12-17 22:36:38,143 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5d575bc060ce15f1b0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:38,190 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409287575bc060ce160adf 2025-12-17 22:36:38,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409287575bc060ce160adf HTTP/1.1" 502 150 2025-12-17 22:36:38,191 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409287575bc060ce160adf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:38,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b43575bc060ce15f127 2025-12-17 22:36:38,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b43575bc060ce15f127 HTTP/1.1" 502 150 2025-12-17 22:36:38,386 - lava_callback - ERROR - Error processing callback for node 69408b43575bc060ce15f127: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b43575bc060ce15f127 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b43575bc060ce15f127 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b43575bc060ce15f127 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:38,388 - lava_callback - INFO - Starting async job submit for node 69408b5f575bc060ce15f1b3 2025-12-17 22:36:38,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:38,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5f575bc060ce15f1b3 HTTP/1.1" 502 150 2025-12-17 22:36:38,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5f575bc060ce15f1b3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:38,395 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5f575bc060ce15f1b3 2025-12-17 22:36:38,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5f575bc060ce15f1b3 HTTP/1.1" 502 150 2025-12-17 22:36:38,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5f575bc060ce15f1b3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:38,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5b575bc060ce15f1aa 2025-12-17 22:36:38,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5b575bc060ce15f1aa HTTP/1.1" 502 150 2025-12-17 22:36:38,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5b575bc060ce15f1aa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:39,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5c575bc060ce15f1ad 2025-12-17 22:36:39,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5c575bc060ce15f1ad HTTP/1.1" 502 150 2025-12-17 22:36:39,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5c575bc060ce15f1ad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:39,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3b575bc060ce15f115 HTTP/1.1" 502 150 2025-12-17 22:36:39,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b3b575bc060ce15f115'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:40,144 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5d575bc060ce15f1b0 2025-12-17 22:36:40,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5d575bc060ce15f1b0 HTTP/1.1" 502 150 2025-12-17 22:36:40,145 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5d575bc060ce15f1b0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:40,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5f575bc060ce15f1b3 2025-12-17 22:36:40,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5f575bc060ce15f1b3 HTTP/1.1" 502 150 2025-12-17 22:36:40,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5f575bc060ce15f1b3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:42,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b45575bc060ce15f12a 2025-12-17 22:36:42,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b59575bc060ce15f1a7 HTTP/1.1" 502 150 2025-12-17 22:36:42,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b59575bc060ce15f1a7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:43,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b47575bc060ce15f12d 2025-12-17 22:36:43,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b47575bc060ce15f12d HTTP/1.1" 502 150 2025-12-17 22:36:43,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b47575bc060ce15f12d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:44,145 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5d575bc060ce15f1b0 2025-12-17 22:36:44,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5d575bc060ce15f1b0 HTTP/1.1" 502 150 2025-12-17 22:36:44,146 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5d575bc060ce15f1b0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:44,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5f575bc060ce15f1b3 2025-12-17 22:36:44,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5f575bc060ce15f1b3 HTTP/1.1" 502 150 2025-12-17 22:36:44,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5f575bc060ce15f1b3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:44,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b49575bc060ce15f130 2025-12-17 22:36:44,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b49575bc060ce15f130 HTTP/1.1" 502 150 2025-12-17 22:36:44,647 - lava_callback - ERROR - Error processing callback for node 69408b49575bc060ce15f130: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b49575bc060ce15f130 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b49575bc060ce15f130 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b49575bc060ce15f130 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:44,649 - lava_callback - INFO - Starting async job submit for node 69408b60575bc060ce15f1b8 2025-12-17 22:36:44,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:44,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b60575bc060ce15f1b8 HTTP/1.1" 502 150 2025-12-17 22:36:44,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b60575bc060ce15f1b8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:44,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b60575bc060ce15f1b8 2025-12-17 22:36:44,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b60575bc060ce15f1b8 HTTP/1.1" 502 150 2025-12-17 22:36:44,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b60575bc060ce15f1b8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:46,191 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409287575bc060ce160adf 2025-12-17 22:36:46,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409287575bc060ce160adf HTTP/1.1" 502 150 2025-12-17 22:36:46,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409287575bc060ce160adf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38840 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:36:46,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5b575bc060ce15f1aa 2025-12-17 22:36:46,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5b575bc060ce15f1aa HTTP/1.1" 502 150 2025-12-17 22:36:46,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5b575bc060ce15f1aa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:46,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b60575bc060ce15f1b8 2025-12-17 22:36:46,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b60575bc060ce15f1b8 HTTP/1.1" 502 150 2025-12-17 22:36:46,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b60575bc060ce15f1b8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:47,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5c575bc060ce15f1ad 2025-12-17 22:36:47,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5c575bc060ce15f1ad HTTP/1.1" 502 150 2025-12-17 22:36:47,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5c575bc060ce15f1ad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:49,084 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b51575bc060ce15f193 2025-12-17 22:36:49,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b51575bc060ce15f193 HTTP/1.1" 502 150 2025-12-17 22:36:49,086 - lava_callback - ERROR - Error processing callback for node 69408b51575bc060ce15f193: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b51575bc060ce15f193 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b51575bc060ce15f193 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b51575bc060ce15f193 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:49,088 - lava_callback - INFO - Starting async job submit for node 69408b61575bc060ce15f1bf 2025-12-17 22:36:49,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:49,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b61575bc060ce15f1bf HTTP/1.1" 502 150 2025-12-17 22:36:49,094 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b61575bc060ce15f1bf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:49,094 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b61575bc060ce15f1bf 2025-12-17 22:36:49,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b61575bc060ce15f1bf HTTP/1.1" 502 150 2025-12-17 22:36:49,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b61575bc060ce15f1bf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:50,363 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b4d575bc060ce15f17a 2025-12-17 22:36:50,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b4f575bc060ce15f18a 2025-12-17 22:36:50,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b4f575bc060ce15f18a HTTP/1.1" 502 150 2025-12-17 22:36:50,373 - lava_callback - ERROR - Error processing callback for node 69408b4f575bc060ce15f18a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b4f575bc060ce15f18a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b4f575bc060ce15f18a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b4f575bc060ce15f18a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:50,374 - lava_callback - INFO - Starting async job submit for node 69408b63575bc060ce15f1c6 2025-12-17 22:36:50,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:50,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b63575bc060ce15f1c6 HTTP/1.1" 502 150 2025-12-17 22:36:50,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b63575bc060ce15f1c6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:50,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b63575bc060ce15f1c6 2025-12-17 22:36:50,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b63575bc060ce15f1c6 HTTP/1.1" 502 150 2025-12-17 22:36:50,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b63575bc060ce15f1c6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:50,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b57575bc060ce15f1a4 2025-12-17 22:36:50,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b57575bc060ce15f1a4 HTTP/1.1" 502 150 2025-12-17 22:36:50,647 - lava_callback - ERROR - Error processing callback for node 69408b57575bc060ce15f1a4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b57575bc060ce15f1a4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b57575bc060ce15f1a4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b57575bc060ce15f1a4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:50,649 - lava_callback - INFO - Starting async job submit for node 69408b64575bc060ce15f1cb 2025-12-17 22:36:50,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:50,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b53575bc060ce15f19c 2025-12-17 22:36:50,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b53575bc060ce15f19c HTTP/1.1" 502 150 2025-12-17 22:36:50,655 - lava_callback - ERROR - Error processing callback for node 69408b53575bc060ce15f19c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b53575bc060ce15f19c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b53575bc060ce15f19c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b53575bc060ce15f19c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:50,657 - lava_callback - INFO - Starting async job submit for node 69408b65575bc060ce15f1d2 2025-12-17 22:36:50,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b64575bc060ce15f1cb HTTP/1.1" 502 150 2025-12-17 22:36:50,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b64575bc060ce15f1cb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:50,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b64575bc060ce15f1cb 2025-12-17 22:36:50,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:50,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b64575bc060ce15f1cb HTTP/1.1" 502 150 2025-12-17 22:36:50,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b64575bc060ce15f1cb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:50,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b65575bc060ce15f1d2 HTTP/1.1" 502 150 2025-12-17 22:36:50,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b65575bc060ce15f1d2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:50,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b65575bc060ce15f1d2 2025-12-17 22:36:50,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b60575bc060ce15f1b8 2025-12-17 22:36:50,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b65575bc060ce15f1d2 HTTP/1.1" 502 150 2025-12-17 22:36:50,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b65575bc060ce15f1d2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:50,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b60575bc060ce15f1b8 HTTP/1.1" 502 150 2025-12-17 22:36:50,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b60575bc060ce15f1b8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:50,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b59575bc060ce15f1a7 2025-12-17 22:36:50,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b59575bc060ce15f1a7 HTTP/1.1" 502 150 2025-12-17 22:36:50,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b59575bc060ce15f1a7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:51,095 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b61575bc060ce15f1bf 2025-12-17 22:36:51,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b61575bc060ce15f1bf HTTP/1.1" 502 150 2025-12-17 22:36:51,099 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b61575bc060ce15f1bf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:51,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b56575bc060ce15f1a1 2025-12-17 22:36:51,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b56575bc060ce15f1a1 HTTP/1.1" 502 150 2025-12-17 22:36:51,681 - lava_callback - ERROR - Error processing callback for node 69408b56575bc060ce15f1a1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b56575bc060ce15f1a1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b56575bc060ce15f1a1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b56575bc060ce15f1a1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:51,683 - lava_callback - INFO - Starting async job submit for node 69408b69575bc060ce15f1d8 2025-12-17 22:36:51,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:51,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b69575bc060ce15f1d8 HTTP/1.1" 502 150 2025-12-17 22:36:51,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b69575bc060ce15f1d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:51,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b69575bc060ce15f1d8 2025-12-17 22:36:51,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b69575bc060ce15f1d8 HTTP/1.1" 502 150 2025-12-17 22:36:51,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b69575bc060ce15f1d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:52,146 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5d575bc060ce15f1b0 2025-12-17 22:36:52,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5d575bc060ce15f1b0 HTTP/1.1" 502 150 2025-12-17 22:36:52,148 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5d575bc060ce15f1b0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:52,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b63575bc060ce15f1c6 2025-12-17 22:36:52,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b63575bc060ce15f1c6 HTTP/1.1" 502 150 2025-12-17 22:36:52,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b63575bc060ce15f1c6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:52,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5f575bc060ce15f1b3 2025-12-17 22:36:52,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5f575bc060ce15f1b3 HTTP/1.1" 502 150 2025-12-17 22:36:52,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b5f575bc060ce15f1b3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:52,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b64575bc060ce15f1cb 2025-12-17 22:36:52,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b64575bc060ce15f1cb HTTP/1.1" 502 150 2025-12-17 22:36:52,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b64575bc060ce15f1cb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:52,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b65575bc060ce15f1d2 2025-12-17 22:36:52,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b65575bc060ce15f1d2 HTTP/1.1" 502 150 2025-12-17 22:36:52,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b65575bc060ce15f1d2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:53,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b69575bc060ce15f1d8 2025-12-17 22:36:53,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b69575bc060ce15f1d8 HTTP/1.1" 502 150 2025-12-17 22:36:53,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b69575bc060ce15f1d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:54,848 - lava_callback - INFO - Received callback for node 69408d74575bc060ce15f857 INFO: 172.21.0.1:51848 - "POST /node/69408d74575bc060ce15f857 HTTP/1.0" 202 Accepted 2025-12-17 22:36:55,099 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b61575bc060ce15f1bf 2025-12-17 22:36:55,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b61575bc060ce15f1bf HTTP/1.1" 502 150 2025-12-17 22:36:55,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b61575bc060ce15f1bf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:55,484 - lava_callback - INFO - Received callback for node 69408d78575bc060ce15f9d5 INFO: 172.21.0.1:51860 - "POST /node/69408d78575bc060ce15f9d5 HTTP/1.0" 202 Accepted 2025-12-17 22:36:55,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b3b575bc060ce15f115 2025-12-17 22:36:55,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b3b575bc060ce15f115 HTTP/1.1" 502 150 2025-12-17 22:36:55,986 - lava_callback - ERROR - Error processing callback for node 69408b3b575bc060ce15f115: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b3b575bc060ce15f115 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b3b575bc060ce15f115 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b3b575bc060ce15f115 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:55,988 - lava_callback - INFO - Starting async job submit for node 69408b6b575bc060ce15f1db 2025-12-17 22:36:55,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:56,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6b575bc060ce15f1db HTTP/1.1" 502 150 2025-12-17 22:36:56,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6b575bc060ce15f1db'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:56,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6b575bc060ce15f1db 2025-12-17 22:36:56,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6b575bc060ce15f1db HTTP/1.1" 502 150 2025-12-17 22:36:56,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6b575bc060ce15f1db'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:56,215 - lava_callback - INFO - Received callback for node 69408d7a575bc060ce15f9d8 INFO: 172.21.0.1:51864 - "POST /node/69408d7a575bc060ce15f9d8 HTTP/1.0" 202 Accepted 2025-12-17 22:36:56,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b63575bc060ce15f1c6 2025-12-17 22:36:56,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b63575bc060ce15f1c6 HTTP/1.1" 502 150 2025-12-17 22:36:56,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b63575bc060ce15f1c6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51872 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:36:56,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b64575bc060ce15f1cb 2025-12-17 22:36:56,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b64575bc060ce15f1cb HTTP/1.1" 502 150 2025-12-17 22:36:56,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b64575bc060ce15f1cb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:56,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b65575bc060ce15f1d2 2025-12-17 22:36:56,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b65575bc060ce15f1d2 HTTP/1.1" 502 150 2025-12-17 22:36:56,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b65575bc060ce15f1d2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:56,863 - lava_callback - INFO - Received callback for node 69408d7c575bc060ce15f9df INFO: 172.21.0.1:51874 - "POST /node/69408d7c575bc060ce15f9df HTTP/1.0" 202 Accepted 2025-12-17 22:36:57,507 - lava_callback - INFO - Received callback for node 69408d7e575bc060ce15f9e8 INFO: 172.21.0.1:51878 - "POST /node/69408d7e575bc060ce15f9e8 HTTP/1.0" 202 Accepted 2025-12-17 22:36:57,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b69575bc060ce15f1d8 2025-12-17 22:36:57,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b69575bc060ce15f1d8 HTTP/1.1" 502 150 2025-12-17 22:36:57,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b69575bc060ce15f1d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:57,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b45575bc060ce15f12a HTTP/1.1" 502 150 2025-12-17 22:36:57,905 - lava_callback - ERROR - Error processing callback for node 69408b45575bc060ce15f12a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b45575bc060ce15f12a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b45575bc060ce15f12a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b45575bc060ce15f12a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:57,907 - lava_callback - INFO - Starting async job submit for node 69408b67575bc060ce15f1d5 2025-12-17 22:36:57,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:57,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b67575bc060ce15f1d5 HTTP/1.1" 502 150 2025-12-17 22:36:57,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b67575bc060ce15f1d5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:57,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b67575bc060ce15f1d5 2025-12-17 22:36:57,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b67575bc060ce15f1d5 HTTP/1.1" 502 150 2025-12-17 22:36:57,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b67575bc060ce15f1d5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:58,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6b575bc060ce15f1db 2025-12-17 22:36:58,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6b575bc060ce15f1db HTTP/1.1" 502 150 2025-12-17 22:36:58,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6b575bc060ce15f1db'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:58,170 - lava_callback - INFO - Received callback for node 69408d80575bc060ce15f9ed INFO: 172.21.0.1:51894 - "POST /node/69408d80575bc060ce15f9ed HTTP/1.0" 202 Accepted 2025-12-17 22:36:58,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b60575bc060ce15f1b8 2025-12-17 22:36:58,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b60575bc060ce15f1b8 HTTP/1.1" 502 150 2025-12-17 22:36:58,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b60575bc060ce15f1b8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:58,799 - lava_callback - INFO - Received callback for node 69408d83575bc060ce15f9f0 INFO: 172.21.0.1:51900 - "POST /node/69408d83575bc060ce15f9f0 HTTP/1.0" 202 Accepted 2025-12-17 22:36:59,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b47575bc060ce15f12d 2025-12-17 22:36:59,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b47575bc060ce15f12d HTTP/1.1" 502 150 2025-12-17 22:36:59,541 - lava_callback - ERROR - Error processing callback for node 69408b47575bc060ce15f12d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b47575bc060ce15f12d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b47575bc060ce15f12d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b47575bc060ce15f12d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:36:59,543 - lava_callback - INFO - Starting async job submit for node 69408b6c575bc060ce15f1de 2025-12-17 22:36:59,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:36:59,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6c575bc060ce15f1de HTTP/1.1" 502 150 2025-12-17 22:36:59,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6c575bc060ce15f1de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:59,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6c575bc060ce15f1de 2025-12-17 22:36:59,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6c575bc060ce15f1de HTTP/1.1" 502 150 2025-12-17 22:36:59,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6c575bc060ce15f1de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:36:59,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b67575bc060ce15f1d5 2025-12-17 22:36:59,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b67575bc060ce15f1d5 HTTP/1.1" 502 150 2025-12-17 22:36:59,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b67575bc060ce15f1d5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:01,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6c575bc060ce15f1de 2025-12-17 22:37:01,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6c575bc060ce15f1de HTTP/1.1" 502 150 2025-12-17 22:37:01,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6c575bc060ce15f1de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:02,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6b575bc060ce15f1db 2025-12-17 22:37:02,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6b575bc060ce15f1db HTTP/1.1" 502 150 2025-12-17 22:37:02,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6b575bc060ce15f1db'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:02,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409287575bc060ce160adf 2025-12-17 22:37:02,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409287575bc060ce160adf HTTP/1.1" 502 150 2025-12-17 22:37:02,194 - lava_callback - ERROR - Error processing callback for node 69409287575bc060ce160adf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409287575bc060ce160adf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409287575bc060ce160adf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409287575bc060ce160adf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:02,196 - lava_callback - INFO - Starting async job submit for node 69408b71575bc060ce15f1e7 2025-12-17 22:37:02,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:02,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b71575bc060ce15f1e7 HTTP/1.1" 502 150 2025-12-17 22:37:02,203 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b71575bc060ce15f1e7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:02,203 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b71575bc060ce15f1e7 2025-12-17 22:37:02,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b71575bc060ce15f1e7 HTTP/1.1" 502 150 2025-12-17 22:37:02,204 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b71575bc060ce15f1e7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51108 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:37:02,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5b575bc060ce15f1aa 2025-12-17 22:37:02,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5b575bc060ce15f1aa HTTP/1.1" 502 150 2025-12-17 22:37:02,662 - lava_callback - ERROR - Error processing callback for node 69408b5b575bc060ce15f1aa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b5b575bc060ce15f1aa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b5b575bc060ce15f1aa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b5b575bc060ce15f1aa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:02,664 - lava_callback - INFO - Starting async job submit for node 69408b72575bc060ce15f1ea 2025-12-17 22:37:02,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:02,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b72575bc060ce15f1ea HTTP/1.1" 502 150 2025-12-17 22:37:02,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b72575bc060ce15f1ea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:02,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b72575bc060ce15f1ea 2025-12-17 22:37:02,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b72575bc060ce15f1ea HTTP/1.1" 502 150 2025-12-17 22:37:02,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b72575bc060ce15f1ea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:03,101 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b61575bc060ce15f1bf 2025-12-17 22:37:03,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b61575bc060ce15f1bf HTTP/1.1" 502 150 2025-12-17 22:37:03,102 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b61575bc060ce15f1bf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:03,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5c575bc060ce15f1ad 2025-12-17 22:37:03,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5c575bc060ce15f1ad HTTP/1.1" 502 150 2025-12-17 22:37:03,463 - lava_callback - ERROR - Error processing callback for node 69408b5c575bc060ce15f1ad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b5c575bc060ce15f1ad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b5c575bc060ce15f1ad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b5c575bc060ce15f1ad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:03,465 - lava_callback - INFO - Starting async job submit for node 69408b73575bc060ce15f1ed 2025-12-17 22:37:03,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:03,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b73575bc060ce15f1ed HTTP/1.1" 502 150 2025-12-17 22:37:03,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b73575bc060ce15f1ed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:03,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b73575bc060ce15f1ed 2025-12-17 22:37:03,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b73575bc060ce15f1ed HTTP/1.1" 502 150 2025-12-17 22:37:03,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b73575bc060ce15f1ed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:03,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b67575bc060ce15f1d5 2025-12-17 22:37:03,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b67575bc060ce15f1d5 HTTP/1.1" 502 150 2025-12-17 22:37:03,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b67575bc060ce15f1d5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:04,204 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b71575bc060ce15f1e7 2025-12-17 22:37:04,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b71575bc060ce15f1e7 HTTP/1.1" 502 150 2025-12-17 22:37:04,206 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b71575bc060ce15f1e7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:04,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b63575bc060ce15f1c6 2025-12-17 22:37:04,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b63575bc060ce15f1c6 HTTP/1.1" 502 150 2025-12-17 22:37:04,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b63575bc060ce15f1c6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:04,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b64575bc060ce15f1cb 2025-12-17 22:37:04,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b64575bc060ce15f1cb HTTP/1.1" 502 150 2025-12-17 22:37:04,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b64575bc060ce15f1cb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:04,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b72575bc060ce15f1ea 2025-12-17 22:37:04,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b72575bc060ce15f1ea HTTP/1.1" 502 150 2025-12-17 22:37:04,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b72575bc060ce15f1ea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:04,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b65575bc060ce15f1d2 2025-12-17 22:37:04,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b65575bc060ce15f1d2 HTTP/1.1" 502 150 2025-12-17 22:37:04,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b65575bc060ce15f1d2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:05,095 - lava_callback - INFO - Received callback for node 69408d84575bc060ce15f9f3 INFO: 172.21.0.1:51124 - "POST /node/69408d84575bc060ce15f9f3 HTTP/1.0" 202 Accepted 2025-12-17 22:37:05,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b73575bc060ce15f1ed 2025-12-17 22:37:05,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b73575bc060ce15f1ed HTTP/1.1" 502 150 2025-12-17 22:37:05,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b73575bc060ce15f1ed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:05,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6c575bc060ce15f1de 2025-12-17 22:37:05,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6c575bc060ce15f1de HTTP/1.1" 502 150 2025-12-17 22:37:05,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6c575bc060ce15f1de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:05,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b69575bc060ce15f1d8 2025-12-17 22:37:05,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b69575bc060ce15f1d8 HTTP/1.1" 502 150 2025-12-17 22:37:05,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b69575bc060ce15f1d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:05,755 - lava_callback - INFO - Received callback for node 69408d86575bc060ce15f9f6 INFO: 172.21.0.1:51134 - "POST /node/69408d86575bc060ce15f9f6 HTTP/1.0" 202 Accepted 2025-12-17 22:37:05,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b4d575bc060ce15f17a HTTP/1.1" 502 150 2025-12-17 22:37:05,841 - lava_callback - ERROR - Error processing callback for node 69408b4d575bc060ce15f17a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b4d575bc060ce15f17a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b4d575bc060ce15f17a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b4d575bc060ce15f17a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:05,843 - lava_callback - INFO - Starting async job submit for node 69408b75575bc060ce15f1f4 2025-12-17 22:37:05,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:05,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b75575bc060ce15f1f4 HTTP/1.1" 502 150 2025-12-17 22:37:05,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b75575bc060ce15f1f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:05,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b75575bc060ce15f1f4 2025-12-17 22:37:05,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b75575bc060ce15f1f4 HTTP/1.1" 502 150 2025-12-17 22:37:05,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b75575bc060ce15f1f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:06,401 - lava_callback - INFO - Received callback for node 69408d89575bc060ce15f9fc INFO: 172.21.0.1:51138 - "POST /node/69408d89575bc060ce15f9fc HTTP/1.0" 202 Accepted INFO: 172.21.0.1:51144 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:37:06,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b59575bc060ce15f1a7 2025-12-17 22:37:06,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b59575bc060ce15f1a7 HTTP/1.1" 502 150 2025-12-17 22:37:06,803 - lava_callback - ERROR - Error processing callback for node 69408b59575bc060ce15f1a7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b59575bc060ce15f1a7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b59575bc060ce15f1a7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b59575bc060ce15f1a7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:06,805 - lava_callback - INFO - Starting async job submit for node 69408b76575bc060ce15f1fb 2025-12-17 22:37:06,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:06,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b76575bc060ce15f1fb HTTP/1.1" 502 150 2025-12-17 22:37:06,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b76575bc060ce15f1fb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:06,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b76575bc060ce15f1fb 2025-12-17 22:37:06,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b76575bc060ce15f1fb HTTP/1.1" 502 150 2025-12-17 22:37:06,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b76575bc060ce15f1fb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:07,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b75575bc060ce15f1f4 2025-12-17 22:37:07,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b75575bc060ce15f1f4 HTTP/1.1" 502 150 2025-12-17 22:37:07,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b75575bc060ce15f1f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:08,148 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5d575bc060ce15f1b0 2025-12-17 22:37:08,206 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b71575bc060ce15f1e7 2025-12-17 22:37:08,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b71575bc060ce15f1e7 HTTP/1.1" 502 150 2025-12-17 22:37:08,214 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b71575bc060ce15f1e7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:08,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b5f575bc060ce15f1b3 2025-12-17 22:37:08,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5f575bc060ce15f1b3 HTTP/1.1" 502 150 2025-12-17 22:37:08,401 - lava_callback - ERROR - Error processing callback for node 69408b5f575bc060ce15f1b3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b5f575bc060ce15f1b3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b5f575bc060ce15f1b3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b5f575bc060ce15f1b3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:08,403 - lava_callback - INFO - Starting async job submit for node 69408b77575bc060ce15f202 2025-12-17 22:37:08,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:08,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b77575bc060ce15f202 HTTP/1.1" 502 150 2025-12-17 22:37:08,410 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b77575bc060ce15f202'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:08,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b77575bc060ce15f202 2025-12-17 22:37:08,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b77575bc060ce15f202 HTTP/1.1" 502 150 2025-12-17 22:37:08,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b77575bc060ce15f202'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:08,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b72575bc060ce15f1ea 2025-12-17 22:37:08,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b72575bc060ce15f1ea HTTP/1.1" 502 150 2025-12-17 22:37:08,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b72575bc060ce15f1ea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:08,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b76575bc060ce15f1fb 2025-12-17 22:37:08,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b76575bc060ce15f1fb HTTP/1.1" 502 150 2025-12-17 22:37:08,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b76575bc060ce15f1fb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:09,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b73575bc060ce15f1ed 2025-12-17 22:37:09,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b73575bc060ce15f1ed HTTP/1.1" 502 150 2025-12-17 22:37:09,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b73575bc060ce15f1ed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:09,859 - lava_callback - INFO - Received callback for node 69408d93575bc060ce15fa19 INFO: 172.21.0.1:36690 - "POST /node/69408d93575bc060ce15fa19 HTTP/1.0" 202 Accepted 2025-12-17 22:37:10,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6b575bc060ce15f1db 2025-12-17 22:37:10,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6b575bc060ce15f1db HTTP/1.1" 502 150 2025-12-17 22:37:10,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6b575bc060ce15f1db'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:10,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b77575bc060ce15f202 2025-12-17 22:37:10,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b77575bc060ce15f202 HTTP/1.1" 502 150 2025-12-17 22:37:10,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b77575bc060ce15f202'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:10,527 - lava_callback - INFO - Received callback for node 69408d94575bc060ce15fa1c INFO: 172.21.0.1:36700 - "POST /node/69408d94575bc060ce15fa1c HTTP/1.0" 202 Accepted 2025-12-17 22:37:11,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b75575bc060ce15f1f4 2025-12-17 22:37:11,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b75575bc060ce15f1f4 HTTP/1.1" 502 150 2025-12-17 22:37:11,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b75575bc060ce15f1f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:11,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b67575bc060ce15f1d5 2025-12-17 22:37:11,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b67575bc060ce15f1d5 HTTP/1.1" 502 150 2025-12-17 22:37:11,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b67575bc060ce15f1d5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:12,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b76575bc060ce15f1fb 2025-12-17 22:37:12,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b76575bc060ce15f1fb HTTP/1.1" 502 150 2025-12-17 22:37:12,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b76575bc060ce15f1fb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:13,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6c575bc060ce15f1de 2025-12-17 22:37:13,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6c575bc060ce15f1de HTTP/1.1" 502 150 2025-12-17 22:37:13,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6c575bc060ce15f1de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:14,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b77575bc060ce15f202 2025-12-17 22:37:14,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b77575bc060ce15f202 HTTP/1.1" 502 150 2025-12-17 22:37:14,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b77575bc060ce15f202'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:14,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b60575bc060ce15f1b8 2025-12-17 22:37:14,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b60575bc060ce15f1b8 HTTP/1.1" 502 150 2025-12-17 22:37:14,669 - lava_callback - ERROR - Error processing callback for node 69408b60575bc060ce15f1b8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b60575bc060ce15f1b8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b60575bc060ce15f1b8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b60575bc060ce15f1b8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:14,671 - lava_callback - INFO - Starting async job submit for node 69408b79575bc060ce15f207 2025-12-17 22:37:14,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:14,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b79575bc060ce15f207 HTTP/1.1" 502 150 2025-12-17 22:37:14,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b79575bc060ce15f207'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:14,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b79575bc060ce15f207 2025-12-17 22:37:14,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b79575bc060ce15f207 HTTP/1.1" 502 150 2025-12-17 22:37:14,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b79575bc060ce15f207'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:14,850 - lava_callback - INFO - Received callback for node 69408d87575bc060ce15f9f9 INFO: 172.21.0.1:36710 - "POST /node/69408d87575bc060ce15f9f9 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:36718 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:37:15,527 - lava_callback - INFO - Received callback for node 69408d8b575bc060ce15f9ff INFO: 172.21.0.1:36728 - "POST /node/69408d8b575bc060ce15f9ff HTTP/1.0" 202 Accepted 2025-12-17 22:37:16,183 - lava_callback - INFO - Received callback for node 69408d8d575bc060ce15fa02 INFO: 172.21.0.1:36736 - "POST /node/69408d8d575bc060ce15fa02 HTTP/1.0" 202 Accepted 2025-12-17 22:37:16,214 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b71575bc060ce15f1e7 2025-12-17 22:37:16,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b71575bc060ce15f1e7 HTTP/1.1" 502 150 2025-12-17 22:37:16,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b71575bc060ce15f1e7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36738 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:37:16,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b72575bc060ce15f1ea 2025-12-17 22:37:16,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b72575bc060ce15f1ea HTTP/1.1" 502 150 2025-12-17 22:37:16,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b72575bc060ce15f1ea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:16,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b79575bc060ce15f207 2025-12-17 22:37:16,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b79575bc060ce15f207 HTTP/1.1" 502 150 2025-12-17 22:37:16,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b79575bc060ce15f207'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:16,815 - lava_callback - INFO - Received callback for node 69408d8f575bc060ce15fa05 INFO: 172.21.0.1:36740 - "POST /node/69408d8f575bc060ce15fa05 HTTP/1.0" 202 Accepted 2025-12-17 22:37:17,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b73575bc060ce15f1ed 2025-12-17 22:37:17,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b73575bc060ce15f1ed HTTP/1.1" 502 150 2025-12-17 22:37:17,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b73575bc060ce15f1ed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:19,103 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b61575bc060ce15f1bf 2025-12-17 22:37:19,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b75575bc060ce15f1f4 2025-12-17 22:37:19,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b75575bc060ce15f1f4 HTTP/1.1" 502 150 2025-12-17 22:37:19,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b75575bc060ce15f1f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:20,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b63575bc060ce15f1c6 2025-12-17 22:37:20,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b63575bc060ce15f1c6 HTTP/1.1" 502 150 2025-12-17 22:37:20,389 - lava_callback - ERROR - Error processing callback for node 69408b63575bc060ce15f1c6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b63575bc060ce15f1c6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b63575bc060ce15f1c6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b63575bc060ce15f1c6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:20,391 - lava_callback - INFO - Starting async job submit for node 69408b7a575bc060ce15f20e 2025-12-17 22:37:20,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:20,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7a575bc060ce15f20e HTTP/1.1" 502 150 2025-12-17 22:37:20,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7a575bc060ce15f20e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:20,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7a575bc060ce15f20e 2025-12-17 22:37:20,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7a575bc060ce15f20e HTTP/1.1" 502 150 2025-12-17 22:37:20,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7a575bc060ce15f20e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:20,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b64575bc060ce15f1cb 2025-12-17 22:37:20,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b64575bc060ce15f1cb HTTP/1.1" 502 150 2025-12-17 22:37:20,665 - lava_callback - ERROR - Error processing callback for node 69408b64575bc060ce15f1cb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b64575bc060ce15f1cb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b64575bc060ce15f1cb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b64575bc060ce15f1cb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:20,667 - lava_callback - INFO - Starting async job submit for node 69408b7b575bc060ce15f213 2025-12-17 22:37:20,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:20,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7b575bc060ce15f213 HTTP/1.1" 502 150 2025-12-17 22:37:20,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7b575bc060ce15f213'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:20,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7b575bc060ce15f213 2025-12-17 22:37:20,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b65575bc060ce15f1d2 2025-12-17 22:37:20,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b65575bc060ce15f1d2 HTTP/1.1" 502 150 2025-12-17 22:37:20,675 - lava_callback - ERROR - Error processing callback for node 69408b65575bc060ce15f1d2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b65575bc060ce15f1d2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b65575bc060ce15f1d2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b65575bc060ce15f1d2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:20,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7b575bc060ce15f213 HTTP/1.1" 502 150 2025-12-17 22:37:20,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7b575bc060ce15f213'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:20,678 - lava_callback - INFO - Starting async job submit for node 69408b7d575bc060ce15f21d 2025-12-17 22:37:20,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:20,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7d575bc060ce15f21d HTTP/1.1" 502 150 2025-12-17 22:37:20,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7d575bc060ce15f21d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:20,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7d575bc060ce15f21d 2025-12-17 22:37:20,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7d575bc060ce15f21d HTTP/1.1" 502 150 2025-12-17 22:37:20,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7d575bc060ce15f21d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:20,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b79575bc060ce15f207 2025-12-17 22:37:20,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b79575bc060ce15f207 HTTP/1.1" 502 150 2025-12-17 22:37:20,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b79575bc060ce15f207'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:20,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b76575bc060ce15f1fb 2025-12-17 22:37:20,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b76575bc060ce15f1fb HTTP/1.1" 502 150 2025-12-17 22:37:20,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b76575bc060ce15f1fb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:21,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b69575bc060ce15f1d8 2025-12-17 22:37:21,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b69575bc060ce15f1d8 HTTP/1.1" 502 150 2025-12-17 22:37:21,697 - lava_callback - ERROR - Error processing callback for node 69408b69575bc060ce15f1d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b69575bc060ce15f1d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b69575bc060ce15f1d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b69575bc060ce15f1d8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:21,699 - lava_callback - INFO - Starting async job submit for node 69408b82575bc060ce15f22e 2025-12-17 22:37:21,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:21,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b82575bc060ce15f22e HTTP/1.1" 502 150 2025-12-17 22:37:21,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b82575bc060ce15f22e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:21,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b82575bc060ce15f22e 2025-12-17 22:37:21,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b82575bc060ce15f22e HTTP/1.1" 502 150 2025-12-17 22:37:21,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b82575bc060ce15f22e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:22,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7a575bc060ce15f20e 2025-12-17 22:37:22,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7a575bc060ce15f20e HTTP/1.1" 502 150 2025-12-17 22:37:22,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7a575bc060ce15f20e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:22,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b77575bc060ce15f202 2025-12-17 22:37:22,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b77575bc060ce15f202 HTTP/1.1" 502 150 2025-12-17 22:37:22,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b77575bc060ce15f202'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:22,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7b575bc060ce15f213 2025-12-17 22:37:22,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7b575bc060ce15f213 HTTP/1.1" 502 150 2025-12-17 22:37:22,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7b575bc060ce15f213'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:22,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7d575bc060ce15f21d 2025-12-17 22:37:22,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7d575bc060ce15f21d HTTP/1.1" 502 150 2025-12-17 22:37:22,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7d575bc060ce15f21d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:23,049 - lava_callback - INFO - Received callback for node 69408da0575bc060ce15fa3a INFO: 172.21.0.1:52800 - "POST /node/69408da0575bc060ce15fa3a HTTP/1.0" 202 Accepted 2025-12-17 22:37:23,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b5d575bc060ce15f1b0 HTTP/1.1" 502 150 2025-12-17 22:37:23,504 - lava_callback - ERROR - Error processing callback for node 69408b5d575bc060ce15f1b0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b5d575bc060ce15f1b0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b5d575bc060ce15f1b0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b5d575bc060ce15f1b0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:23,506 - lava_callback - INFO - Starting async job submit for node 69408b86575bc060ce15f234 2025-12-17 22:37:23,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:23,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b86575bc060ce15f234 HTTP/1.1" 502 150 2025-12-17 22:37:23,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b86575bc060ce15f234'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:23,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b86575bc060ce15f234 2025-12-17 22:37:23,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b86575bc060ce15f234 HTTP/1.1" 502 150 2025-12-17 22:37:23,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b86575bc060ce15f234'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:23,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b82575bc060ce15f22e 2025-12-17 22:37:23,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b82575bc060ce15f22e HTTP/1.1" 502 150 2025-12-17 22:37:23,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b82575bc060ce15f22e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:25,114 - lava_callback - INFO - Received callback for node 69408d95575bc060ce15fa1f INFO: 172.21.0.1:52808 - "POST /node/69408d95575bc060ce15fa1f HTTP/1.0" 202 Accepted 2025-12-17 22:37:25,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b86575bc060ce15f234 2025-12-17 22:37:25,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b86575bc060ce15f234 HTTP/1.1" 502 150 2025-12-17 22:37:25,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b86575bc060ce15f234'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:25,775 - lava_callback - INFO - Received callback for node 69408d97575bc060ce15fa22 INFO: 172.21.0.1:52812 - "POST /node/69408d97575bc060ce15fa22 HTTP/1.0" 202 Accepted 2025-12-17 22:37:26,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6b575bc060ce15f1db 2025-12-17 22:37:26,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6b575bc060ce15f1db HTTP/1.1" 502 150 2025-12-17 22:37:26,010 - lava_callback - ERROR - Error processing callback for node 69408b6b575bc060ce15f1db: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b6b575bc060ce15f1db (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b6b575bc060ce15f1db (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b6b575bc060ce15f1db (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:26,013 - lava_callback - INFO - Starting async job submit for node 69408b7f575bc060ce15f225 2025-12-17 22:37:26,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:26,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7f575bc060ce15f225 HTTP/1.1" 502 150 2025-12-17 22:37:26,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7f575bc060ce15f225'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:26,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7f575bc060ce15f225 2025-12-17 22:37:26,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7f575bc060ce15f225 HTTP/1.1" 502 150 2025-12-17 22:37:26,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7f575bc060ce15f225'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:26,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7a575bc060ce15f20e 2025-12-17 22:37:26,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7a575bc060ce15f20e HTTP/1.1" 502 150 2025-12-17 22:37:26,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7a575bc060ce15f20e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:26,406 - lava_callback - INFO - Received callback for node 69408d98575bc060ce15fa25 INFO: 172.21.0.1:52822 - "POST /node/69408d98575bc060ce15fa25 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:52826 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:37:26,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7b575bc060ce15f213 2025-12-17 22:37:26,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7b575bc060ce15f213 HTTP/1.1" 502 150 2025-12-17 22:37:26,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7b575bc060ce15f213'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:26,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7d575bc060ce15f21d 2025-12-17 22:37:26,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7d575bc060ce15f21d HTTP/1.1" 502 150 2025-12-17 22:37:26,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7d575bc060ce15f21d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:27,045 - lava_callback - INFO - Received callback for node 69408d99575bc060ce15fa28 INFO: 172.21.0.1:52834 - "POST /node/69408d99575bc060ce15fa28 HTTP/1.0" 202 Accepted 2025-12-17 22:37:27,708 - lava_callback - INFO - Received callback for node 69408d9a575bc060ce15fa2b INFO: 172.21.0.1:52836 - "POST /node/69408d9a575bc060ce15fa2b HTTP/1.0" 202 Accepted 2025-12-17 22:37:27,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b82575bc060ce15f22e 2025-12-17 22:37:27,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b82575bc060ce15f22e HTTP/1.1" 502 150 2025-12-17 22:37:27,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b82575bc060ce15f22e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:27,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b67575bc060ce15f1d5 2025-12-17 22:37:27,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b67575bc060ce15f1d5 HTTP/1.1" 502 150 2025-12-17 22:37:27,945 - lava_callback - ERROR - Error processing callback for node 69408b67575bc060ce15f1d5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b67575bc060ce15f1d5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b67575bc060ce15f1d5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b67575bc060ce15f1d5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:27,946 - lava_callback - INFO - Starting async job submit for node 69408b80575bc060ce15f228 2025-12-17 22:37:27,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:27,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b80575bc060ce15f228 HTTP/1.1" 502 150 2025-12-17 22:37:27,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b80575bc060ce15f228'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:27,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b80575bc060ce15f228 2025-12-17 22:37:27,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b80575bc060ce15f228 HTTP/1.1" 502 150 2025-12-17 22:37:27,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b80575bc060ce15f228'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:28,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7f575bc060ce15f225 2025-12-17 22:37:28,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7f575bc060ce15f225 HTTP/1.1" 502 150 2025-12-17 22:37:28,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7f575bc060ce15f225'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:28,375 - lava_callback - INFO - Received callback for node 69408d9b575bc060ce15fa2e INFO: 172.21.0.1:52842 - "POST /node/69408d9b575bc060ce15fa2e HTTP/1.0" 202 Accepted 2025-12-17 22:37:28,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b79575bc060ce15f207 2025-12-17 22:37:28,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b79575bc060ce15f207 HTTP/1.1" 502 150 2025-12-17 22:37:28,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b79575bc060ce15f207'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:29,044 - lava_callback - INFO - Received callback for node 69408d9e575bc060ce15fa34 INFO: 172.21.0.1:52846 - "POST /node/69408d9e575bc060ce15fa34 HTTP/1.0" 202 Accepted 2025-12-17 22:37:29,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b86575bc060ce15f234 2025-12-17 22:37:29,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b86575bc060ce15f234 HTTP/1.1" 502 150 2025-12-17 22:37:29,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b86575bc060ce15f234'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:29,542 - lava_callback - INFO - Received callback for node 69408d9d575bc060ce15fa31 INFO: 172.21.0.1:52858 - "POST /node/69408d9d575bc060ce15fa31 HTTP/1.0" 202 Accepted 2025-12-17 22:37:29,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6c575bc060ce15f1de 2025-12-17 22:37:29,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6c575bc060ce15f1de HTTP/1.1" 502 150 2025-12-17 22:37:29,563 - lava_callback - ERROR - Error processing callback for node 69408b6c575bc060ce15f1de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b6c575bc060ce15f1de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b6c575bc060ce15f1de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b6c575bc060ce15f1de (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:29,566 - lava_callback - INFO - Starting async job submit for node 69408b81575bc060ce15f22b 2025-12-17 22:37:29,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:29,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b81575bc060ce15f22b HTTP/1.1" 502 150 2025-12-17 22:37:29,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b81575bc060ce15f22b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:29,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b81575bc060ce15f22b 2025-12-17 22:37:29,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b81575bc060ce15f22b HTTP/1.1" 502 150 2025-12-17 22:37:29,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b81575bc060ce15f22b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:29,703 - lava_callback - INFO - Received callback for node 69408d9f575bc060ce15fa37 INFO: 172.21.0.1:48166 - "POST /node/69408d9f575bc060ce15fa37 HTTP/1.0" 202 Accepted 2025-12-17 22:37:29,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b80575bc060ce15f228 2025-12-17 22:37:29,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b80575bc060ce15f228 HTTP/1.1" 502 150 2025-12-17 22:37:29,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b80575bc060ce15f228'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:30,224 - lava_callback - INFO - Received callback for node 69408da8575bc060ce15fa5c INFO: 172.21.0.1:48182 - "POST /node/69408da8575bc060ce15fa5c HTTP/1.0" 202 Accepted 2025-12-17 22:37:30,367 - lava_callback - INFO - Received callback for node 69408da2575bc060ce15fa3d INFO: 172.21.0.1:48198 - "POST /node/69408da2575bc060ce15fa3d HTTP/1.0" 202 Accepted 2025-12-17 22:37:31,021 - lava_callback - INFO - Received callback for node 69408da3575bc060ce15fa40 INFO: 172.21.0.1:48214 - "POST /node/69408da3575bc060ce15fa40 HTTP/1.0" 202 Accepted 2025-12-17 22:37:31,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b81575bc060ce15f22b 2025-12-17 22:37:31,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b81575bc060ce15f22b HTTP/1.1" 502 150 2025-12-17 22:37:31,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b81575bc060ce15f22b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:32,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7f575bc060ce15f225 2025-12-17 22:37:32,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7f575bc060ce15f225 HTTP/1.1" 502 150 2025-12-17 22:37:32,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7f575bc060ce15f225'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:32,216 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b71575bc060ce15f1e7 2025-12-17 22:37:32,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b71575bc060ce15f1e7 HTTP/1.1" 502 150 2025-12-17 22:37:32,217 - lava_callback - ERROR - Error processing callback for node 69408b71575bc060ce15f1e7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b71575bc060ce15f1e7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b71575bc060ce15f1e7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b71575bc060ce15f1e7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:32,220 - lava_callback - INFO - Starting async job submit for node 69408c24575bc060ce15f477 2025-12-17 22:37:32,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:32,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c24575bc060ce15f477 HTTP/1.1" 502 150 2025-12-17 22:37:32,279 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c24575bc060ce15f477'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:32,279 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c24575bc060ce15f477 2025-12-17 22:37:32,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c24575bc060ce15f477 HTTP/1.1" 502 150 2025-12-17 22:37:32,287 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c24575bc060ce15f477'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:32,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b72575bc060ce15f1ea 2025-12-17 22:37:32,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b72575bc060ce15f1ea HTTP/1.1" 502 150 2025-12-17 22:37:32,679 - lava_callback - ERROR - Error processing callback for node 69408b72575bc060ce15f1ea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b72575bc060ce15f1ea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b72575bc060ce15f1ea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b72575bc060ce15f1ea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:32,687 - lava_callback - INFO - Starting async job submit for node 69408b7e575bc060ce15f222 2025-12-17 22:37:32,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:32,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7e575bc060ce15f222 HTTP/1.1" 502 150 2025-12-17 22:37:32,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7e575bc060ce15f222'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:32,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7e575bc060ce15f222 2025-12-17 22:37:32,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7e575bc060ce15f222 HTTP/1.1" 502 150 2025-12-17 22:37:32,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7e575bc060ce15f222'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:33,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b73575bc060ce15f1ed 2025-12-17 22:37:33,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b73575bc060ce15f1ed HTTP/1.1" 502 150 2025-12-17 22:37:33,491 - lava_callback - ERROR - Error processing callback for node 69408b73575bc060ce15f1ed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b73575bc060ce15f1ed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b73575bc060ce15f1ed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b73575bc060ce15f1ed (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:33,493 - lava_callback - INFO - Starting async job submit for node 69408b85575bc060ce15f231 2025-12-17 22:37:33,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:33,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b85575bc060ce15f231 HTTP/1.1" 502 150 2025-12-17 22:37:33,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b85575bc060ce15f231'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:33,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b85575bc060ce15f231 2025-12-17 22:37:33,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b85575bc060ce15f231 HTTP/1.1" 502 150 2025-12-17 22:37:33,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b85575bc060ce15f231'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:33,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b80575bc060ce15f228 2025-12-17 22:37:33,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b80575bc060ce15f228 HTTP/1.1" 502 150 2025-12-17 22:37:33,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b80575bc060ce15f228'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:34,287 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c24575bc060ce15f477 2025-12-17 22:37:34,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7a575bc060ce15f20e 2025-12-17 22:37:34,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7a575bc060ce15f20e HTTP/1.1" 502 150 2025-12-17 22:37:34,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7a575bc060ce15f20e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:34,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b61575bc060ce15f1bf HTTP/1.1" 502 150 2025-12-17 22:37:34,517 - lava_callback - ERROR - Error processing callback for node 69408b61575bc060ce15f1bf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b61575bc060ce15f1bf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b61575bc060ce15f1bf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b61575bc060ce15f1bf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:34,520 - lava_callback - INFO - Starting async job submit for node 69408b8a575bc060ce15f277 2025-12-17 22:37:34,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:34,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8a575bc060ce15f277 HTTP/1.1" 502 150 2025-12-17 22:37:34,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b8a575bc060ce15f277'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:34,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b8a575bc060ce15f277 2025-12-17 22:37:34,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8a575bc060ce15f277 HTTP/1.1" 502 150 2025-12-17 22:37:34,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b8a575bc060ce15f277'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:34,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7b575bc060ce15f213 2025-12-17 22:37:34,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7b575bc060ce15f213 HTTP/1.1" 502 150 2025-12-17 22:37:34,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7b575bc060ce15f213'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:34,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7d575bc060ce15f21d 2025-12-17 22:37:34,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7d575bc060ce15f21d HTTP/1.1" 502 150 2025-12-17 22:37:34,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7d575bc060ce15f21d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:34,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7e575bc060ce15f222 2025-12-17 22:37:34,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7e575bc060ce15f222 HTTP/1.1" 502 150 2025-12-17 22:37:34,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7e575bc060ce15f222'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:34,933 - lava_callback - INFO - Received callback for node 69408daa575bc060ce15fa63 INFO: 172.21.0.1:48220 - "POST /node/69408daa575bc060ce15fa63 HTTP/1.0" 202 Accepted 2025-12-17 22:37:35,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b85575bc060ce15f231 2025-12-17 22:37:35,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b85575bc060ce15f231 HTTP/1.1" 502 150 2025-12-17 22:37:35,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b85575bc060ce15f231'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:35,574 - lava_callback - INFO - Received callback for node 69408dad575bc060ce15fa66 2025-12-17 22:37:35,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b81575bc060ce15f22b INFO: 172.21.0.1:48232 - "POST /node/69408dad575bc060ce15fa66 HTTP/1.0" 202 Accepted 2025-12-17 22:37:35,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b81575bc060ce15f22b HTTP/1.1" 502 150 2025-12-17 22:37:35,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b81575bc060ce15f22b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:35,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b82575bc060ce15f22e 2025-12-17 22:37:35,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b82575bc060ce15f22e HTTP/1.1" 502 150 2025-12-17 22:37:35,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b82575bc060ce15f22e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:35,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b75575bc060ce15f1f4 2025-12-17 22:37:35,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b75575bc060ce15f1f4 HTTP/1.1" 502 150 2025-12-17 22:37:35,869 - lava_callback - ERROR - Error processing callback for node 69408b75575bc060ce15f1f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b75575bc060ce15f1f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b75575bc060ce15f1f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b75575bc060ce15f1f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:35,870 - lava_callback - INFO - Starting async job submit for node 69408b8c575bc060ce15f298 2025-12-17 22:37:35,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:35,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8c575bc060ce15f298 HTTP/1.1" 502 150 2025-12-17 22:37:35,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b8c575bc060ce15f298'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:35,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b8c575bc060ce15f298 2025-12-17 22:37:35,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8c575bc060ce15f298 HTTP/1.1" 502 150 2025-12-17 22:37:35,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b8c575bc060ce15f298'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48244 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:37:36,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b8a575bc060ce15f277 2025-12-17 22:37:36,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8a575bc060ce15f277 HTTP/1.1" 502 150 2025-12-17 22:37:36,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b8a575bc060ce15f277'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:36,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b76575bc060ce15f1fb 2025-12-17 22:37:36,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b76575bc060ce15f1fb HTTP/1.1" 502 150 2025-12-17 22:37:36,819 - lava_callback - ERROR - Error processing callback for node 69408b76575bc060ce15f1fb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b76575bc060ce15f1fb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b76575bc060ce15f1fb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b76575bc060ce15f1fb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:36,822 - lava_callback - INFO - Starting async job submit for node 69408b91575bc060ce15f2a3 2025-12-17 22:37:36,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:36,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b91575bc060ce15f2a3 HTTP/1.1" 502 150 2025-12-17 22:37:36,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b91575bc060ce15f2a3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:36,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b91575bc060ce15f2a3 2025-12-17 22:37:36,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b91575bc060ce15f2a3 HTTP/1.1" 502 150 2025-12-17 22:37:36,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b91575bc060ce15f2a3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:37,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b86575bc060ce15f234 2025-12-17 22:37:37,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b86575bc060ce15f234 HTTP/1.1" 502 150 2025-12-17 22:37:37,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b86575bc060ce15f234'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:37,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b8c575bc060ce15f298 2025-12-17 22:37:37,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8c575bc060ce15f298 HTTP/1.1" 502 150 2025-12-17 22:37:37,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b8c575bc060ce15f298'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:38,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b77575bc060ce15f202 2025-12-17 22:37:38,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b77575bc060ce15f202 HTTP/1.1" 502 150 2025-12-17 22:37:38,416 - lava_callback - ERROR - Error processing callback for node 69408b77575bc060ce15f202: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b77575bc060ce15f202 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b77575bc060ce15f202 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b77575bc060ce15f202 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:38,418 - lava_callback - INFO - Starting async job submit for node 69408956575bc060ce15e9e2 2025-12-17 22:37:38,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7e575bc060ce15f222 2025-12-17 22:37:38,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7e575bc060ce15f222 HTTP/1.1" 502 150 2025-12-17 22:37:38,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7e575bc060ce15f222'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:38,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:38,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408956575bc060ce15e9e2 HTTP/1.1" 502 150 2025-12-17 22:37:38,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408956575bc060ce15e9e2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:38,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408956575bc060ce15e9e2 2025-12-17 22:37:38,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408956575bc060ce15e9e2 HTTP/1.1" 502 150 2025-12-17 22:37:38,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408956575bc060ce15e9e2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:38,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b91575bc060ce15f2a3 2025-12-17 22:37:38,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b91575bc060ce15f2a3 HTTP/1.1" 502 150 2025-12-17 22:37:38,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b91575bc060ce15f2a3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:39,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b85575bc060ce15f231 2025-12-17 22:37:39,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b85575bc060ce15f231 HTTP/1.1" 502 150 2025-12-17 22:37:39,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b85575bc060ce15f231'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:40,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7f575bc060ce15f225 2025-12-17 22:37:40,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7f575bc060ce15f225 HTTP/1.1" 502 150 2025-12-17 22:37:40,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7f575bc060ce15f225'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:40,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b8a575bc060ce15f277 2025-12-17 22:37:40,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8a575bc060ce15f277 HTTP/1.1" 502 150 2025-12-17 22:37:40,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b8a575bc060ce15f277'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:40,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408956575bc060ce15e9e2 2025-12-17 22:37:40,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408956575bc060ce15e9e2 HTTP/1.1" 502 150 2025-12-17 22:37:40,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408956575bc060ce15e9e2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:41,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b8c575bc060ce15f298 2025-12-17 22:37:41,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8c575bc060ce15f298 HTTP/1.1" 502 150 2025-12-17 22:37:41,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b8c575bc060ce15f298'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:41,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b80575bc060ce15f228 2025-12-17 22:37:41,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b80575bc060ce15f228 HTTP/1.1" 502 150 2025-12-17 22:37:41,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b80575bc060ce15f228'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:42,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b91575bc060ce15f2a3 2025-12-17 22:37:42,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b91575bc060ce15f2a3 HTTP/1.1" 502 150 2025-12-17 22:37:42,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b91575bc060ce15f2a3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:43,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b81575bc060ce15f22b 2025-12-17 22:37:43,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b81575bc060ce15f22b HTTP/1.1" 502 150 2025-12-17 22:37:43,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b81575bc060ce15f22b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:44,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b79575bc060ce15f207 2025-12-17 22:37:44,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b79575bc060ce15f207 HTTP/1.1" 502 150 2025-12-17 22:37:44,713 - lava_callback - ERROR - Error processing callback for node 69408b79575bc060ce15f207: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b79575bc060ce15f207 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b79575bc060ce15f207 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b79575bc060ce15f207 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:44,715 - lava_callback - INFO - Starting async job submit for node 69408b8e575bc060ce15f2a0 2025-12-17 22:37:44,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:44,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8e575bc060ce15f2a0 HTTP/1.1" 502 150 2025-12-17 22:37:44,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b8e575bc060ce15f2a0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:44,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b8e575bc060ce15f2a0 2025-12-17 22:37:44,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8e575bc060ce15f2a0 HTTP/1.1" 502 150 2025-12-17 22:37:44,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b8e575bc060ce15f2a0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:44,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408956575bc060ce15e9e2 2025-12-17 22:37:44,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408956575bc060ce15e9e2 HTTP/1.1" 502 150 2025-12-17 22:37:44,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408956575bc060ce15e9e2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:45,146 - lava_callback - INFO - Received callback for node 69408da4575bc060ce15fa4b INFO: 172.21.0.1:42004 - "POST /node/69408da4575bc060ce15fa4b HTTP/1.0" 202 Accepted 2025-12-17 22:37:45,801 - lava_callback - INFO - Received callback for node 69408da6575bc060ce15fa52 INFO: 172.21.0.1:42008 - "POST /node/69408da6575bc060ce15fa52 HTTP/1.0" 202 Accepted 2025-12-17 22:37:46,457 - lava_callback - INFO - Received callback for node 69408da7575bc060ce15fa59 INFO: 172.21.0.1:42012 - "POST /node/69408da7575bc060ce15fa59 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:42020 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:37:46,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b8e575bc060ce15f2a0 2025-12-17 22:37:46,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8e575bc060ce15f2a0 HTTP/1.1" 502 150 2025-12-17 22:37:46,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b8e575bc060ce15f2a0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:46,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7e575bc060ce15f222 2025-12-17 22:37:46,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7e575bc060ce15f222 HTTP/1.1" 502 150 2025-12-17 22:37:46,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b7e575bc060ce15f222'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:47,105 - lava_callback - INFO - Received callback for node 69408db1575bc060ce15fa6c INFO: 172.21.0.1:42026 - "POST /node/69408db1575bc060ce15fa6c HTTP/1.0" 202 Accepted 2025-12-17 22:37:47,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b85575bc060ce15f231 2025-12-17 22:37:47,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b85575bc060ce15f231 HTTP/1.1" 502 150 2025-12-17 22:37:47,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b85575bc060ce15f231'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:47,760 - lava_callback - INFO - Received callback for node 69408db4575bc060ce15fa8c INFO: 172.21.0.1:42030 - "POST /node/69408db4575bc060ce15fa8c HTTP/1.0" 202 Accepted 2025-12-17 22:37:48,427 - lava_callback - INFO - Received callback for node 69408db6575bc060ce15fa8f INFO: 172.21.0.1:42032 - "POST /node/69408db6575bc060ce15fa8f HTTP/1.0" 202 Accepted 2025-12-17 22:37:48,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b8a575bc060ce15f277 2025-12-17 22:37:48,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8a575bc060ce15f277 HTTP/1.1" 502 150 2025-12-17 22:37:48,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b8a575bc060ce15f277'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:49,076 - lava_callback - INFO - Received callback for node 69408db8575bc060ce15fa92 INFO: 172.21.0.1:42034 - "POST /node/69408db8575bc060ce15fa92 HTTP/1.0" 202 Accepted 2025-12-17 22:37:49,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c24575bc060ce15f477 HTTP/1.1" 502 150 2025-12-17 22:37:49,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c24575bc060ce15f477'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:49,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b8c575bc060ce15f298 2025-12-17 22:37:49,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8c575bc060ce15f298 HTTP/1.1" 502 150 2025-12-17 22:37:49,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b8c575bc060ce15f298'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:50,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7a575bc060ce15f20e 2025-12-17 22:37:50,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7a575bc060ce15f20e HTTP/1.1" 502 150 2025-12-17 22:37:50,406 - lava_callback - ERROR - Error processing callback for node 69408b7a575bc060ce15f20e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b7a575bc060ce15f20e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b7a575bc060ce15f20e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b7a575bc060ce15f20e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:50,408 - lava_callback - INFO - Starting async job submit for node 69408b92575bc060ce15f2a6 2025-12-17 22:37:50,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:50,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b92575bc060ce15f2a6 HTTP/1.1" 502 150 2025-12-17 22:37:50,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b92575bc060ce15f2a6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:50,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b92575bc060ce15f2a6 2025-12-17 22:37:50,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b92575bc060ce15f2a6 HTTP/1.1" 502 150 2025-12-17 22:37:50,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b92575bc060ce15f2a6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:50,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7b575bc060ce15f213 2025-12-17 22:37:50,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7b575bc060ce15f213 HTTP/1.1" 502 150 2025-12-17 22:37:50,684 - lava_callback - ERROR - Error processing callback for node 69408b7b575bc060ce15f213: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b7b575bc060ce15f213 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b7b575bc060ce15f213 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b7b575bc060ce15f213 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:50,686 - lava_callback - INFO - Starting async job submit for node 69408b95575bc060ce15f2a9 2025-12-17 22:37:50,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:50,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7d575bc060ce15f21d 2025-12-17 22:37:50,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7d575bc060ce15f21d HTTP/1.1" 502 150 2025-12-17 22:37:50,691 - lava_callback - ERROR - Error processing callback for node 69408b7d575bc060ce15f21d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b7d575bc060ce15f21d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b7d575bc060ce15f21d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b7d575bc060ce15f21d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:50,693 - lava_callback - INFO - Starting async job submit for node 69408b9e575bc060ce15f2b9 2025-12-17 22:37:50,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:50,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b95575bc060ce15f2a9 HTTP/1.1" 502 150 2025-12-17 22:37:50,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b95575bc060ce15f2a9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:50,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b95575bc060ce15f2a9 2025-12-17 22:37:50,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b95575bc060ce15f2a9 HTTP/1.1" 502 150 2025-12-17 22:37:50,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b95575bc060ce15f2a9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:50,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b9e575bc060ce15f2b9 HTTP/1.1" 502 150 2025-12-17 22:37:50,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b9e575bc060ce15f2b9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:50,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b9e575bc060ce15f2b9 2025-12-17 22:37:50,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b9e575bc060ce15f2b9 HTTP/1.1" 502 150 2025-12-17 22:37:50,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b9e575bc060ce15f2b9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:50,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b8e575bc060ce15f2a0 2025-12-17 22:37:50,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8e575bc060ce15f2a0 HTTP/1.1" 502 150 2025-12-17 22:37:50,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b8e575bc060ce15f2a0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:50,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b91575bc060ce15f2a3 2025-12-17 22:37:50,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b91575bc060ce15f2a3 HTTP/1.1" 502 150 2025-12-17 22:37:50,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b91575bc060ce15f2a3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:51,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b82575bc060ce15f22e 2025-12-17 22:37:51,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b82575bc060ce15f22e HTTP/1.1" 502 150 2025-12-17 22:37:51,733 - lava_callback - ERROR - Error processing callback for node 69408b82575bc060ce15f22e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b82575bc060ce15f22e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b82575bc060ce15f22e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b82575bc060ce15f22e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:51,735 - lava_callback - INFO - Starting async job submit for node 69408ba2575bc060ce15f2c1 2025-12-17 22:37:51,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:51,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba2575bc060ce15f2c1 HTTP/1.1" 502 150 2025-12-17 22:37:51,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba2575bc060ce15f2c1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:51,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba2575bc060ce15f2c1 2025-12-17 22:37:51,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba2575bc060ce15f2c1 HTTP/1.1" 502 150 2025-12-17 22:37:51,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba2575bc060ce15f2c1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:52,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b92575bc060ce15f2a6 2025-12-17 22:37:52,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b92575bc060ce15f2a6 HTTP/1.1" 502 150 2025-12-17 22:37:52,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b92575bc060ce15f2a6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:52,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b95575bc060ce15f2a9 2025-12-17 22:37:52,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b95575bc060ce15f2a9 HTTP/1.1" 502 150 2025-12-17 22:37:52,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b95575bc060ce15f2a9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:52,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b9e575bc060ce15f2b9 2025-12-17 22:37:52,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b9e575bc060ce15f2b9 HTTP/1.1" 502 150 2025-12-17 22:37:52,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b9e575bc060ce15f2b9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:52,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408956575bc060ce15e9e2 2025-12-17 22:37:52,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408956575bc060ce15e9e2 HTTP/1.1" 502 150 2025-12-17 22:37:52,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408956575bc060ce15e9e2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:53,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b86575bc060ce15f234 2025-12-17 22:37:53,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c24575bc060ce15f477 2025-12-17 22:37:53,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c24575bc060ce15f477 HTTP/1.1" 502 150 2025-12-17 22:37:53,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c24575bc060ce15f477'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:53,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba2575bc060ce15f2c1 2025-12-17 22:37:53,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba2575bc060ce15f2c1 HTTP/1.1" 502 150 2025-12-17 22:37:53,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba2575bc060ce15f2c1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:54,831 - lava_callback - INFO - Received callback for node 6940924d575bc060ce160a6c INFO: 172.21.0.1:57912 - "POST /node/6940924d575bc060ce160a6c HTTP/1.0" 202 Accepted 2025-12-17 22:37:54,984 - lava_callback - INFO - Received callback for node 69408daf575bc060ce15fa69 INFO: 172.21.0.1:57916 - "POST /node/69408daf575bc060ce15fa69 HTTP/1.0" 202 Accepted 2025-12-17 22:37:56,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7f575bc060ce15f225 2025-12-17 22:37:56,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7f575bc060ce15f225 HTTP/1.1" 502 150 2025-12-17 22:37:56,032 - lava_callback - ERROR - Error processing callback for node 69408b7f575bc060ce15f225: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b7f575bc060ce15f225 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b7f575bc060ce15f225 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b7f575bc060ce15f225 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:56,033 - lava_callback - INFO - Starting async job submit for node 69408b9b575bc060ce15f2ac 2025-12-17 22:37:56,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:56,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b9b575bc060ce15f2ac HTTP/1.1" 502 150 2025-12-17 22:37:56,041 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b9b575bc060ce15f2ac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:56,042 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b9b575bc060ce15f2ac 2025-12-17 22:37:56,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b9b575bc060ce15f2ac HTTP/1.1" 502 150 2025-12-17 22:37:56,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b9b575bc060ce15f2ac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:56,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b92575bc060ce15f2a6 2025-12-17 22:37:56,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b92575bc060ce15f2a6 HTTP/1.1" 502 150 2025-12-17 22:37:56,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b92575bc060ce15f2a6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57930 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:37:56,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b95575bc060ce15f2a9 2025-12-17 22:37:56,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b95575bc060ce15f2a9 HTTP/1.1" 502 150 2025-12-17 22:37:56,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b95575bc060ce15f2a9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:56,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b9e575bc060ce15f2b9 2025-12-17 22:37:56,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b9e575bc060ce15f2b9 HTTP/1.1" 502 150 2025-12-17 22:37:56,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b9e575bc060ce15f2b9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:57,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba2575bc060ce15f2c1 2025-12-17 22:37:57,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba2575bc060ce15f2c1 HTTP/1.1" 502 150 2025-12-17 22:37:57,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba2575bc060ce15f2c1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:57,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b80575bc060ce15f228 2025-12-17 22:37:57,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b80575bc060ce15f228 HTTP/1.1" 502 150 2025-12-17 22:37:57,960 - lava_callback - ERROR - Error processing callback for node 69408b80575bc060ce15f228: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b80575bc060ce15f228 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b80575bc060ce15f228 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b80575bc060ce15f228 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:57,962 - lava_callback - INFO - Starting async job submit for node 69408ba0575bc060ce15f2be 2025-12-17 22:37:57,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:57,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba0575bc060ce15f2be HTTP/1.1" 502 150 2025-12-17 22:37:57,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba0575bc060ce15f2be'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:57,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba0575bc060ce15f2be 2025-12-17 22:37:57,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba0575bc060ce15f2be HTTP/1.1" 502 150 2025-12-17 22:37:57,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba0575bc060ce15f2be'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:58,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b9b575bc060ce15f2ac 2025-12-17 22:37:58,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b9b575bc060ce15f2ac HTTP/1.1" 502 150 2025-12-17 22:37:58,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b9b575bc060ce15f2ac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:58,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b8e575bc060ce15f2a0 2025-12-17 22:37:58,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8e575bc060ce15f2a0 HTTP/1.1" 502 150 2025-12-17 22:37:58,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b8e575bc060ce15f2a0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:59,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b81575bc060ce15f22b 2025-12-17 22:37:59,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b81575bc060ce15f22b HTTP/1.1" 502 150 2025-12-17 22:37:59,594 - lava_callback - ERROR - Error processing callback for node 69408b81575bc060ce15f22b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b81575bc060ce15f22b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b81575bc060ce15f22b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b81575bc060ce15f22b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:37:59,596 - lava_callback - INFO - Starting async job submit for node 69408ba3575bc060ce15f2c4 2025-12-17 22:37:59,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:37:59,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba3575bc060ce15f2c4 HTTP/1.1" 502 150 2025-12-17 22:37:59,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba3575bc060ce15f2c4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:59,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba3575bc060ce15f2c4 2025-12-17 22:37:59,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba3575bc060ce15f2c4 HTTP/1.1" 502 150 2025-12-17 22:37:59,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba3575bc060ce15f2c4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:37:59,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba0575bc060ce15f2be 2025-12-17 22:37:59,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba0575bc060ce15f2be HTTP/1.1" 502 150 2025-12-17 22:37:59,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba0575bc060ce15f2be'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:01,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba3575bc060ce15f2c4 2025-12-17 22:38:01,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba3575bc060ce15f2c4 HTTP/1.1" 502 150 2025-12-17 22:38:01,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba3575bc060ce15f2c4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:01,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c24575bc060ce15f477 2025-12-17 22:38:01,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c24575bc060ce15f477 HTTP/1.1" 502 150 2025-12-17 22:38:01,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c24575bc060ce15f477'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:02,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b9b575bc060ce15f2ac 2025-12-17 22:38:02,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b9b575bc060ce15f2ac HTTP/1.1" 502 150 2025-12-17 22:38:02,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b9b575bc060ce15f2ac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54122 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:38:02,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b7e575bc060ce15f222 2025-12-17 22:38:02,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b7e575bc060ce15f222 HTTP/1.1" 502 150 2025-12-17 22:38:02,739 - lava_callback - ERROR - Error processing callback for node 69408b7e575bc060ce15f222: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b7e575bc060ce15f222 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b7e575bc060ce15f222 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b7e575bc060ce15f222 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:02,741 - lava_callback - INFO - Starting async job submit for node 69408ba5575bc060ce15f2c7 2025-12-17 22:38:02,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:02,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba5575bc060ce15f2c7 HTTP/1.1" 502 150 2025-12-17 22:38:02,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba5575bc060ce15f2c7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:02,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba5575bc060ce15f2c7 2025-12-17 22:38:02,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba5575bc060ce15f2c7 HTTP/1.1" 502 150 2025-12-17 22:38:02,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba5575bc060ce15f2c7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:03,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b85575bc060ce15f231 2025-12-17 22:38:03,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b85575bc060ce15f231 HTTP/1.1" 502 150 2025-12-17 22:38:03,522 - lava_callback - ERROR - Error processing callback for node 69408b85575bc060ce15f231: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b85575bc060ce15f231 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b85575bc060ce15f231 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b85575bc060ce15f231 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:03,525 - lava_callback - INFO - Starting async job submit for node 69408baa575bc060ce15f2d0 2025-12-17 22:38:03,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:03,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba0575bc060ce15f2be 2025-12-17 22:38:03,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba0575bc060ce15f2be HTTP/1.1" 502 150 2025-12-17 22:38:03,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba0575bc060ce15f2be'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:04,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b92575bc060ce15f2a6 2025-12-17 22:38:04,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b92575bc060ce15f2a6 HTTP/1.1" 502 150 2025-12-17 22:38:04,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b92575bc060ce15f2a6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:04,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b8a575bc060ce15f277 2025-12-17 22:38:04,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8a575bc060ce15f277 HTTP/1.1" 502 150 2025-12-17 22:38:04,541 - lava_callback - ERROR - Error processing callback for node 69408b8a575bc060ce15f277: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b8a575bc060ce15f277 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b8a575bc060ce15f277 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b8a575bc060ce15f277 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:04,543 - lava_callback - INFO - Starting async job submit for node 69408bac575bc060ce15f2d3 2025-12-17 22:38:04,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:04,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bac575bc060ce15f2d3 HTTP/1.1" 502 150 2025-12-17 22:38:04,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bac575bc060ce15f2d3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:04,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bac575bc060ce15f2d3 2025-12-17 22:38:04,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bac575bc060ce15f2d3 HTTP/1.1" 502 150 2025-12-17 22:38:04,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bac575bc060ce15f2d3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:04,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b95575bc060ce15f2a9 2025-12-17 22:38:04,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b95575bc060ce15f2a9 HTTP/1.1" 502 150 2025-12-17 22:38:04,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b95575bc060ce15f2a9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:04,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b9e575bc060ce15f2b9 2025-12-17 22:38:04,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b9e575bc060ce15f2b9 HTTP/1.1" 502 150 2025-12-17 22:38:04,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b9e575bc060ce15f2b9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:04,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba5575bc060ce15f2c7 2025-12-17 22:38:04,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba5575bc060ce15f2c7 HTTP/1.1" 502 150 2025-12-17 22:38:04,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba5575bc060ce15f2c7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:05,156 - lava_callback - INFO - Received callback for node 69408dba575bc060ce15fa9b INFO: 172.21.0.1:54124 - "POST /node/69408dba575bc060ce15fa9b HTTP/1.0" 202 Accepted 2025-12-17 22:38:05,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba3575bc060ce15f2c4 2025-12-17 22:38:05,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba3575bc060ce15f2c4 HTTP/1.1" 502 150 2025-12-17 22:38:05,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba3575bc060ce15f2c4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:05,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba2575bc060ce15f2c1 2025-12-17 22:38:05,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba2575bc060ce15f2c1 HTTP/1.1" 502 150 2025-12-17 22:38:05,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba2575bc060ce15f2c1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:05,825 - lava_callback - INFO - Received callback for node 69408dbc575bc060ce15faa2 INFO: 172.21.0.1:54138 - "POST /node/69408dbc575bc060ce15faa2 HTTP/1.0" 202 Accepted 2025-12-17 22:38:05,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b8c575bc060ce15f298 2025-12-17 22:38:05,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8c575bc060ce15f298 HTTP/1.1" 502 150 2025-12-17 22:38:05,884 - lava_callback - ERROR - Error processing callback for node 69408b8c575bc060ce15f298: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b8c575bc060ce15f298 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b8c575bc060ce15f298 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b8c575bc060ce15f298 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:05,885 - lava_callback - INFO - Starting async job submit for node 69408bae575bc060ce15f2d6 2025-12-17 22:38:05,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:05,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bae575bc060ce15f2d6 HTTP/1.1" 502 150 2025-12-17 22:38:05,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bae575bc060ce15f2d6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:05,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bae575bc060ce15f2d6 2025-12-17 22:38:05,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bae575bc060ce15f2d6 HTTP/1.1" 502 150 2025-12-17 22:38:05,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bae575bc060ce15f2d6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:06,493 - lava_callback - INFO - Received callback for node 69408dbe575bc060ce15faa5 INFO: 172.21.0.1:54154 - "POST /node/69408dbe575bc060ce15faa5 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:54166 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:38:06,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bac575bc060ce15f2d3 2025-12-17 22:38:06,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bac575bc060ce15f2d3 HTTP/1.1" 502 150 2025-12-17 22:38:06,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bac575bc060ce15f2d3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:06,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b91575bc060ce15f2a3 2025-12-17 22:38:06,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b91575bc060ce15f2a3 HTTP/1.1" 502 150 2025-12-17 22:38:06,857 - lava_callback - ERROR - Error processing callback for node 69408b91575bc060ce15f2a3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b91575bc060ce15f2a3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b91575bc060ce15f2a3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b91575bc060ce15f2a3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:06,860 - lava_callback - INFO - Starting async job submit for node 69408ba7575bc060ce15f2ca 2025-12-17 22:38:06,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:06,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba7575bc060ce15f2ca HTTP/1.1" 502 150 2025-12-17 22:38:06,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba7575bc060ce15f2ca'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:06,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba7575bc060ce15f2ca 2025-12-17 22:38:06,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba7575bc060ce15f2ca HTTP/1.1" 502 150 2025-12-17 22:38:06,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba7575bc060ce15f2ca'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:07,169 - lava_callback - INFO - Received callback for node 69408dc0575bc060ce15faa8 INFO: 172.21.0.1:54178 - "POST /node/69408dc0575bc060ce15faa8 HTTP/1.0" 202 Accepted 2025-12-17 22:38:07,794 - lava_callback - INFO - Received callback for node 69408dc2575bc060ce15faab INFO: 172.21.0.1:54182 - "POST /node/69408dc2575bc060ce15faab HTTP/1.0" 202 Accepted 2025-12-17 22:38:07,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bae575bc060ce15f2d6 2025-12-17 22:38:07,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bae575bc060ce15f2d6 HTTP/1.1" 502 150 2025-12-17 22:38:07,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bae575bc060ce15f2d6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:08,458 - lava_callback - INFO - Received callback for node 69408dc4575bc060ce15faae INFO: 172.21.0.1:54196 - "POST /node/69408dc4575bc060ce15faae HTTP/1.0" 202 Accepted 2025-12-17 22:38:08,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba5575bc060ce15f2c7 2025-12-17 22:38:08,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba5575bc060ce15f2c7 HTTP/1.1" 502 150 2025-12-17 22:38:08,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba5575bc060ce15f2c7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:08,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b86575bc060ce15f234 HTTP/1.1" 502 150 2025-12-17 22:38:08,816 - lava_callback - ERROR - Error processing callback for node 69408b86575bc060ce15f234: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b86575bc060ce15f234 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b86575bc060ce15f234 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b86575bc060ce15f234 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:08,818 - lava_callback - INFO - Starting async job submit for node 69408bb0575bc060ce15f2dd 2025-12-17 22:38:08,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:08,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb0575bc060ce15f2dd HTTP/1.1" 502 150 2025-12-17 22:38:08,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bb0575bc060ce15f2dd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:08,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bb0575bc060ce15f2dd 2025-12-17 22:38:08,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408956575bc060ce15e9e2 2025-12-17 22:38:08,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb0575bc060ce15f2dd HTTP/1.1" 502 150 2025-12-17 22:38:08,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bb0575bc060ce15f2dd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:08,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408956575bc060ce15e9e2 HTTP/1.1" 502 150 2025-12-17 22:38:08,826 - lava_callback - ERROR - Error processing callback for node 69408956575bc060ce15e9e2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408956575bc060ce15e9e2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408956575bc060ce15e9e2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408956575bc060ce15e9e2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:08,828 - lava_callback - INFO - Starting async job submit for node 69408bb6575bc060ce15f2ea 2025-12-17 22:38:08,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:08,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb6575bc060ce15f2ea HTTP/1.1" 502 150 2025-12-17 22:38:08,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bb6575bc060ce15f2ea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:08,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bb6575bc060ce15f2ea 2025-12-17 22:38:08,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb6575bc060ce15f2ea HTTP/1.1" 502 150 2025-12-17 22:38:08,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bb6575bc060ce15f2ea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:08,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba7575bc060ce15f2ca 2025-12-17 22:38:08,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba7575bc060ce15f2ca HTTP/1.1" 502 150 2025-12-17 22:38:08,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba7575bc060ce15f2ca'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:09,130 - lava_callback - INFO - Received callback for node 69408dc6575bc060ce15fab1 INFO: 172.21.0.1:54212 - "POST /node/69408dc6575bc060ce15fab1 HTTP/1.0" 202 Accepted 2025-12-17 22:38:09,794 - lava_callback - INFO - Received callback for node 69408dc8575bc060ce15fab4 INFO: 172.21.0.1:49484 - "POST /node/69408dc8575bc060ce15fab4 HTTP/1.0" 202 Accepted 2025-12-17 22:38:10,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b9b575bc060ce15f2ac 2025-12-17 22:38:10,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b9b575bc060ce15f2ac HTTP/1.1" 502 150 2025-12-17 22:38:10,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b9b575bc060ce15f2ac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:10,441 - lava_callback - INFO - Received callback for node 69408dca575bc060ce15fab7 INFO: 172.21.0.1:49486 - "POST /node/69408dca575bc060ce15fab7 HTTP/1.0" 202 Accepted 2025-12-17 22:38:10,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bac575bc060ce15f2d3 2025-12-17 22:38:10,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bac575bc060ce15f2d3 HTTP/1.1" 502 150 2025-12-17 22:38:10,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bac575bc060ce15f2d3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:10,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bb0575bc060ce15f2dd 2025-12-17 22:38:10,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb0575bc060ce15f2dd HTTP/1.1" 502 150 2025-12-17 22:38:10,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bb0575bc060ce15f2dd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:10,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bb6575bc060ce15f2ea 2025-12-17 22:38:10,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb6575bc060ce15f2ea HTTP/1.1" 502 150 2025-12-17 22:38:10,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bb6575bc060ce15f2ea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:11,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bae575bc060ce15f2d6 2025-12-17 22:38:11,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bae575bc060ce15f2d6 HTTP/1.1" 502 150 2025-12-17 22:38:11,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bae575bc060ce15f2d6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:11,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba0575bc060ce15f2be 2025-12-17 22:38:12,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba0575bc060ce15f2be HTTP/1.1" 502 150 2025-12-17 22:38:12,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba0575bc060ce15f2be'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:12,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba7575bc060ce15f2ca 2025-12-17 22:38:12,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba7575bc060ce15f2ca HTTP/1.1" 502 150 2025-12-17 22:38:12,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba7575bc060ce15f2ca'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:13,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba3575bc060ce15f2c4 2025-12-17 22:38:13,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba3575bc060ce15f2c4 HTTP/1.1" 502 150 2025-12-17 22:38:13,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba3575bc060ce15f2c4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:14,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b8e575bc060ce15f2a0 2025-12-17 22:38:14,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b8e575bc060ce15f2a0 HTTP/1.1" 502 150 2025-12-17 22:38:14,730 - lava_callback - ERROR - Error processing callback for node 69408b8e575bc060ce15f2a0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b8e575bc060ce15f2a0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b8e575bc060ce15f2a0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b8e575bc060ce15f2a0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:14,732 - lava_callback - INFO - Starting async job submit for node 69408bb8575bc060ce15f2ed 2025-12-17 22:38:14,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:14,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb8575bc060ce15f2ed HTTP/1.1" 502 150 2025-12-17 22:38:14,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bb8575bc060ce15f2ed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:14,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bb8575bc060ce15f2ed 2025-12-17 22:38:14,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb8575bc060ce15f2ed HTTP/1.1" 502 150 2025-12-17 22:38:14,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bb8575bc060ce15f2ed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:14,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bb0575bc060ce15f2dd 2025-12-17 22:38:14,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb0575bc060ce15f2dd HTTP/1.1" 502 150 2025-12-17 22:38:14,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bb0575bc060ce15f2dd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:14,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bb6575bc060ce15f2ea 2025-12-17 22:38:14,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb6575bc060ce15f2ea HTTP/1.1" 502 150 2025-12-17 22:38:14,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bb6575bc060ce15f2ea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49516 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:38:16,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bb8575bc060ce15f2ed 2025-12-17 22:38:16,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb8575bc060ce15f2ed HTTP/1.1" 502 150 2025-12-17 22:38:16,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bb8575bc060ce15f2ed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:16,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba5575bc060ce15f2c7 2025-12-17 22:38:16,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba5575bc060ce15f2c7 HTTP/1.1" 502 150 2025-12-17 22:38:16,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba5575bc060ce15f2c7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:17,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c24575bc060ce15f477 2025-12-17 22:38:17,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c24575bc060ce15f477 HTTP/1.1" 502 150 2025-12-17 22:38:17,620 - lava_callback - ERROR - Error processing callback for node 69408c24575bc060ce15f477: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c24575bc060ce15f477 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c24575bc060ce15f477 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c24575bc060ce15f477 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:17,622 - lava_callback - INFO - Starting async job submit for node 69408bba575bc060ce15f2f0 2025-12-17 22:38:17,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:17,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bba575bc060ce15f2f0 HTTP/1.1" 502 150 2025-12-17 22:38:17,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bba575bc060ce15f2f0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:17,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bba575bc060ce15f2f0 2025-12-17 22:38:17,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bba575bc060ce15f2f0 HTTP/1.1" 502 150 2025-12-17 22:38:17,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bba575bc060ce15f2f0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:18,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bac575bc060ce15f2d3 2025-12-17 22:38:18,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bac575bc060ce15f2d3 HTTP/1.1" 502 150 2025-12-17 22:38:18,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bac575bc060ce15f2d3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:19,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bba575bc060ce15f2f0 2025-12-17 22:38:19,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bae575bc060ce15f2d6 2025-12-17 22:38:19,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bae575bc060ce15f2d6 HTTP/1.1" 502 150 2025-12-17 22:38:19,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bae575bc060ce15f2d6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:20,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b92575bc060ce15f2a6 2025-12-17 22:38:20,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b92575bc060ce15f2a6 HTTP/1.1" 502 150 2025-12-17 22:38:20,436 - lava_callback - ERROR - Error processing callback for node 69408b92575bc060ce15f2a6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b92575bc060ce15f2a6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b92575bc060ce15f2a6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b92575bc060ce15f2a6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:20,438 - lava_callback - INFO - Starting async job submit for node 69408bbd575bc060ce15f2f3 2025-12-17 22:38:20,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:20,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bbd575bc060ce15f2f3 HTTP/1.1" 502 150 2025-12-17 22:38:20,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bbd575bc060ce15f2f3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:20,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bbd575bc060ce15f2f3 2025-12-17 22:38:20,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bbd575bc060ce15f2f3 HTTP/1.1" 502 150 2025-12-17 22:38:20,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bbd575bc060ce15f2f3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:20,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b95575bc060ce15f2a9 2025-12-17 22:38:20,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b95575bc060ce15f2a9 HTTP/1.1" 502 150 2025-12-17 22:38:20,702 - lava_callback - ERROR - Error processing callback for node 69408b95575bc060ce15f2a9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b95575bc060ce15f2a9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b95575bc060ce15f2a9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b95575bc060ce15f2a9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:20,704 - lava_callback - INFO - Starting async job submit for node 69408bbf575bc060ce15f2f6 2025-12-17 22:38:20,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:20,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b9e575bc060ce15f2b9 2025-12-17 22:38:20,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b9e575bc060ce15f2b9 HTTP/1.1" 502 150 2025-12-17 22:38:20,706 - lava_callback - ERROR - Error processing callback for node 69408b9e575bc060ce15f2b9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b9e575bc060ce15f2b9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b9e575bc060ce15f2b9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b9e575bc060ce15f2b9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:20,708 - lava_callback - INFO - Starting async job submit for node 69408bc1575bc060ce15f2f9 2025-12-17 22:38:20,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:20,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bbf575bc060ce15f2f6 HTTP/1.1" 502 150 2025-12-17 22:38:20,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bbf575bc060ce15f2f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:20,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bbf575bc060ce15f2f6 2025-12-17 22:38:20,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bbf575bc060ce15f2f6 HTTP/1.1" 502 150 2025-12-17 22:38:20,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bbf575bc060ce15f2f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:20,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc1575bc060ce15f2f9 HTTP/1.1" 502 150 2025-12-17 22:38:20,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc1575bc060ce15f2f9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:20,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc1575bc060ce15f2f9 2025-12-17 22:38:20,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc1575bc060ce15f2f9 HTTP/1.1" 502 150 2025-12-17 22:38:20,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc1575bc060ce15f2f9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:20,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bb8575bc060ce15f2ed 2025-12-17 22:38:20,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb8575bc060ce15f2ed HTTP/1.1" 502 150 2025-12-17 22:38:20,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bb8575bc060ce15f2ed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:20,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba7575bc060ce15f2ca 2025-12-17 22:38:20,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba7575bc060ce15f2ca HTTP/1.1" 502 150 2025-12-17 22:38:20,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ba7575bc060ce15f2ca'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:21,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba2575bc060ce15f2c1 2025-12-17 22:38:21,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba2575bc060ce15f2c1 HTTP/1.1" 502 150 2025-12-17 22:38:21,765 - lava_callback - ERROR - Error processing callback for node 69408ba2575bc060ce15f2c1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba2575bc060ce15f2c1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba2575bc060ce15f2c1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba2575bc060ce15f2c1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:21,768 - lava_callback - INFO - Starting async job submit for node 69408bc5575bc060ce15f307 2025-12-17 22:38:21,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:21,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc5575bc060ce15f307 HTTP/1.1" 502 150 2025-12-17 22:38:21,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc5575bc060ce15f307'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:21,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc5575bc060ce15f307 2025-12-17 22:38:21,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc5575bc060ce15f307 HTTP/1.1" 502 150 2025-12-17 22:38:21,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc5575bc060ce15f307'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:22,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bbd575bc060ce15f2f3 2025-12-17 22:38:22,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bbd575bc060ce15f2f3 HTTP/1.1" 502 150 2025-12-17 22:38:22,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bbd575bc060ce15f2f3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:22,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bbf575bc060ce15f2f6 2025-12-17 22:38:22,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bbf575bc060ce15f2f6 HTTP/1.1" 502 150 2025-12-17 22:38:22,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bbf575bc060ce15f2f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:22,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc1575bc060ce15f2f9 2025-12-17 22:38:22,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc1575bc060ce15f2f9 HTTP/1.1" 502 150 2025-12-17 22:38:22,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc1575bc060ce15f2f9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:22,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bb0575bc060ce15f2dd 2025-12-17 22:38:22,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb0575bc060ce15f2dd HTTP/1.1" 502 150 2025-12-17 22:38:22,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bb0575bc060ce15f2dd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:22,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bb6575bc060ce15f2ea 2025-12-17 22:38:22,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb6575bc060ce15f2ea HTTP/1.1" 502 150 2025-12-17 22:38:22,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bb6575bc060ce15f2ea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:23,086 - lava_callback - INFO - Received callback for node 69408dd7575bc060ce15fad7 INFO: 172.21.0.1:59170 - "POST /node/69408dd7575bc060ce15fad7 HTTP/1.0" 202 Accepted 2025-12-17 22:38:23,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc5575bc060ce15f307 2025-12-17 22:38:23,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc5575bc060ce15f307 HTTP/1.1" 502 150 2025-12-17 22:38:23,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc5575bc060ce15f307'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:23,888 - lava_callback - INFO - Received callback for node 69408ddb575bc060ce15fadd INFO: 172.21.0.1:59182 - "POST /node/69408ddb575bc060ce15fadd HTTP/1.0" 202 Accepted 2025-12-17 22:38:25,174 - lava_callback - INFO - Received callback for node 69408dcc575bc060ce15fabd INFO: 172.21.0.1:59184 - "POST /node/69408dcc575bc060ce15fabd HTTP/1.0" 202 Accepted 2025-12-17 22:38:25,809 - lava_callback - INFO - Received callback for node 69408dce575bc060ce15fac8 INFO: 172.21.0.1:59194 - "POST /node/69408dce575bc060ce15fac8 HTTP/1.0" 202 Accepted 2025-12-17 22:38:26,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b9b575bc060ce15f2ac 2025-12-17 22:38:26,455 - lava_callback - INFO - Received callback for node 69408dd0575bc060ce15facb INFO: 172.21.0.1:59200 - "POST /node/69408dd0575bc060ce15facb HTTP/1.0" 202 Accepted 2025-12-17 22:38:26,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bbd575bc060ce15f2f3 2025-12-17 22:38:26,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bbd575bc060ce15f2f3 HTTP/1.1" 502 150 2025-12-17 22:38:26,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bbd575bc060ce15f2f3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59208 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:38:26,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bbf575bc060ce15f2f6 2025-12-17 22:38:26,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bbf575bc060ce15f2f6 HTTP/1.1" 502 150 2025-12-17 22:38:26,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bbf575bc060ce15f2f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:26,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc1575bc060ce15f2f9 2025-12-17 22:38:26,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc1575bc060ce15f2f9 HTTP/1.1" 502 150 2025-12-17 22:38:26,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc1575bc060ce15f2f9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:27,099 - lava_callback - INFO - Received callback for node 69408dd2575bc060ce15face INFO: 172.21.0.1:59222 - "POST /node/69408dd2575bc060ce15face HTTP/1.0" 202 Accepted 2025-12-17 22:38:27,761 - lava_callback - INFO - Received callback for node 69408dd4575bc060ce15fad1 INFO: 172.21.0.1:59232 - "POST /node/69408dd4575bc060ce15fad1 HTTP/1.0" 202 Accepted 2025-12-17 22:38:27,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc5575bc060ce15f307 2025-12-17 22:38:27,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc5575bc060ce15f307 HTTP/1.1" 502 150 2025-12-17 22:38:27,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc5575bc060ce15f307'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:28,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba0575bc060ce15f2be 2025-12-17 22:38:28,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba0575bc060ce15f2be HTTP/1.1" 502 150 2025-12-17 22:38:28,002 - lava_callback - ERROR - Error processing callback for node 69408ba0575bc060ce15f2be: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba0575bc060ce15f2be (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba0575bc060ce15f2be (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba0575bc060ce15f2be (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:28,004 - lava_callback - INFO - Starting async job submit for node 69408bc9575bc060ce15f33f 2025-12-17 22:38:28,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:28,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc9575bc060ce15f33f HTTP/1.1" 502 150 2025-12-17 22:38:28,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc9575bc060ce15f33f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:28,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc9575bc060ce15f33f 2025-12-17 22:38:28,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc9575bc060ce15f33f HTTP/1.1" 502 150 2025-12-17 22:38:28,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc9575bc060ce15f33f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:28,402 - lava_callback - INFO - Received callback for node 69408dd6575bc060ce15fad4 INFO: 172.21.0.1:59240 - "POST /node/69408dd6575bc060ce15fad4 HTTP/1.0" 202 Accepted 2025-12-17 22:38:28,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bb8575bc060ce15f2ed 2025-12-17 22:38:28,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb8575bc060ce15f2ed HTTP/1.1" 502 150 2025-12-17 22:38:28,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bb8575bc060ce15f2ed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:29,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba3575bc060ce15f2c4 2025-12-17 22:38:29,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba3575bc060ce15f2c4 HTTP/1.1" 502 150 2025-12-17 22:38:29,618 - lava_callback - ERROR - Error processing callback for node 69408ba3575bc060ce15f2c4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba3575bc060ce15f2c4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba3575bc060ce15f2c4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba3575bc060ce15f2c4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:29,620 - lava_callback - INFO - Starting async job submit for node 69408bcb575bc060ce15f34d 2025-12-17 22:38:29,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:29,640 - lava_callback - INFO - Received callback for node 69408de1575bc060ce15faf2 INFO: 172.21.0.1:59254 - "POST /node/69408de1575bc060ce15faf2 HTTP/1.0" 202 Accepted 2025-12-17 22:38:29,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcb575bc060ce15f34d HTTP/1.1" 502 150 2025-12-17 22:38:29,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bcb575bc060ce15f34d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:29,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bcb575bc060ce15f34d 2025-12-17 22:38:29,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcb575bc060ce15f34d HTTP/1.1" 502 150 2025-12-17 22:38:29,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bcb575bc060ce15f34d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:30,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc9575bc060ce15f33f 2025-12-17 22:38:30,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc9575bc060ce15f33f HTTP/1.1" 502 150 2025-12-17 22:38:30,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc9575bc060ce15f33f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:31,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bcb575bc060ce15f34d 2025-12-17 22:38:31,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcb575bc060ce15f34d HTTP/1.1" 502 150 2025-12-17 22:38:31,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bcb575bc060ce15f34d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:32,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba5575bc060ce15f2c7 2025-12-17 22:38:32,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba5575bc060ce15f2c7 HTTP/1.1" 502 150 2025-12-17 22:38:32,759 - lava_callback - ERROR - Error processing callback for node 69408ba5575bc060ce15f2c7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba5575bc060ce15f2c7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba5575bc060ce15f2c7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba5575bc060ce15f2c7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:32,760 - lava_callback - INFO - Starting async job submit for node 6940ee14460bc8e265114da8 2025-12-17 22:38:32,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:32,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ee14460bc8e265114da8 HTTP/1.1" 502 150 2025-12-17 22:38:32,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ee14460bc8e265114da8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:32,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ee14460bc8e265114da8 2025-12-17 22:38:32,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ee14460bc8e265114da8 HTTP/1.1" 502 150 2025-12-17 22:38:32,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ee14460bc8e265114da8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:34,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc9575bc060ce15f33f 2025-12-17 22:38:34,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408baa575bc060ce15f2d0 HTTP/1.1" 502 150 2025-12-17 22:38:34,160 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408baa575bc060ce15f2d0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:34,160 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408baa575bc060ce15f2d0 2025-12-17 22:38:34,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bbd575bc060ce15f2f3 2025-12-17 22:38:34,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bbd575bc060ce15f2f3 HTTP/1.1" 502 150 2025-12-17 22:38:34,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bbd575bc060ce15f2f3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:34,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bac575bc060ce15f2d3 2025-12-17 22:38:34,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bac575bc060ce15f2d3 HTTP/1.1" 502 150 2025-12-17 22:38:34,557 - lava_callback - ERROR - Error processing callback for node 69408bac575bc060ce15f2d3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bac575bc060ce15f2d3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bac575bc060ce15f2d3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bac575bc060ce15f2d3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:34,560 - lava_callback - INFO - Starting async job submit for node 69408c26575bc060ce15f47a 2025-12-17 22:38:34,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:34,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c26575bc060ce15f47a HTTP/1.1" 502 150 2025-12-17 22:38:34,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c26575bc060ce15f47a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:34,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c26575bc060ce15f47a 2025-12-17 22:38:34,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c26575bc060ce15f47a HTTP/1.1" 502 150 2025-12-17 22:38:34,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c26575bc060ce15f47a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:34,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bbf575bc060ce15f2f6 2025-12-17 22:38:34,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bbf575bc060ce15f2f6 HTTP/1.1" 502 150 2025-12-17 22:38:34,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bbf575bc060ce15f2f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:34,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc1575bc060ce15f2f9 2025-12-17 22:38:34,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc1575bc060ce15f2f9 HTTP/1.1" 502 150 2025-12-17 22:38:34,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc1575bc060ce15f2f9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:34,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ee14460bc8e265114da8 2025-12-17 22:38:34,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ee14460bc8e265114da8 HTTP/1.1" 502 150 2025-12-17 22:38:34,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ee14460bc8e265114da8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:34,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bba575bc060ce15f2f0 HTTP/1.1" 502 150 2025-12-17 22:38:34,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bba575bc060ce15f2f0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:35,002 - lava_callback - INFO - Received callback for node 69408dd9575bc060ce15fada INFO: 172.21.0.1:60338 - "POST /node/69408dd9575bc060ce15fada HTTP/1.0" 202 Accepted 2025-12-17 22:38:35,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bcb575bc060ce15f34d 2025-12-17 22:38:35,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcb575bc060ce15f34d HTTP/1.1" 502 150 2025-12-17 22:38:35,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bcb575bc060ce15f34d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:35,680 - lava_callback - INFO - Received callback for node 69408ddc575bc060ce15fae0 INFO: 172.21.0.1:60342 - "POST /node/69408ddc575bc060ce15fae0 HTTP/1.0" 202 Accepted 2025-12-17 22:38:35,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc5575bc060ce15f307 2025-12-17 22:38:35,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc5575bc060ce15f307 HTTP/1.1" 502 150 2025-12-17 22:38:35,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc5575bc060ce15f307'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:35,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bae575bc060ce15f2d6 2025-12-17 22:38:35,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bae575bc060ce15f2d6 HTTP/1.1" 502 150 2025-12-17 22:38:35,901 - lava_callback - ERROR - Error processing callback for node 69408bae575bc060ce15f2d6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bae575bc060ce15f2d6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bae575bc060ce15f2d6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bae575bc060ce15f2d6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:35,903 - lava_callback - INFO - Starting async job submit for node 69408bc3575bc060ce15f2fe 2025-12-17 22:38:35,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:35,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc3575bc060ce15f2fe HTTP/1.1" 502 150 2025-12-17 22:38:35,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc3575bc060ce15f2fe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:35,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc3575bc060ce15f2fe 2025-12-17 22:38:35,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc3575bc060ce15f2fe HTTP/1.1" 502 150 2025-12-17 22:38:35,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc3575bc060ce15f2fe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:36,346 - lava_callback - INFO - Received callback for node 69408dde575bc060ce15fae3 INFO: 172.21.0.1:60354 - "POST /node/69408dde575bc060ce15fae3 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:60360 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:38:36,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c26575bc060ce15f47a 2025-12-17 22:38:36,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c26575bc060ce15f47a HTTP/1.1" 502 150 2025-12-17 22:38:36,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c26575bc060ce15f47a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:36,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ba7575bc060ce15f2ca 2025-12-17 22:38:36,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ba7575bc060ce15f2ca HTTP/1.1" 502 150 2025-12-17 22:38:36,892 - lava_callback - ERROR - Error processing callback for node 69408ba7575bc060ce15f2ca: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba7575bc060ce15f2ca (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba7575bc060ce15f2ca (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ba7575bc060ce15f2ca (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:36,893 - lava_callback - INFO - Starting async job submit for node 69408bcd575bc060ce15f354 2025-12-17 22:38:36,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:36,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcd575bc060ce15f354 HTTP/1.1" 502 150 2025-12-17 22:38:36,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bcd575bc060ce15f354'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:36,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bcd575bc060ce15f354 2025-12-17 22:38:36,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcd575bc060ce15f354 HTTP/1.1" 502 150 2025-12-17 22:38:36,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bcd575bc060ce15f354'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:37,007 - lava_callback - INFO - Received callback for node 69408ddf575bc060ce15fae6 INFO: 172.21.0.1:60370 - "POST /node/69408ddf575bc060ce15fae6 HTTP/1.0" 202 Accepted 2025-12-17 22:38:37,697 - lava_callback - INFO - Received callback for node 69408de0575bc060ce15faed INFO: 172.21.0.1:60386 - "POST /node/69408de0575bc060ce15faed HTTP/1.0" 202 Accepted 2025-12-17 22:38:37,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc3575bc060ce15f2fe 2025-12-17 22:38:37,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc3575bc060ce15f2fe HTTP/1.1" 502 150 2025-12-17 22:38:37,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc3575bc060ce15f2fe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:38,377 - lava_callback - INFO - Received callback for node 69408de3575bc060ce15faf5 INFO: 172.21.0.1:60402 - "POST /node/69408de3575bc060ce15faf5 HTTP/1.0" 202 Accepted 2025-12-17 22:38:38,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ee14460bc8e265114da8 2025-12-17 22:38:38,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ee14460bc8e265114da8 HTTP/1.1" 502 150 2025-12-17 22:38:38,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ee14460bc8e265114da8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:38,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bb0575bc060ce15f2dd 2025-12-17 22:38:38,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb0575bc060ce15f2dd HTTP/1.1" 502 150 2025-12-17 22:38:38,831 - lava_callback - ERROR - Error processing callback for node 69408bb0575bc060ce15f2dd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bb0575bc060ce15f2dd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bb0575bc060ce15f2dd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bb0575bc060ce15f2dd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:38,834 - lava_callback - INFO - Starting async job submit for node 69408bcf575bc060ce15f357 2025-12-17 22:38:38,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:38,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bb6575bc060ce15f2ea 2025-12-17 22:38:38,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb6575bc060ce15f2ea HTTP/1.1" 502 150 2025-12-17 22:38:38,839 - lava_callback - ERROR - Error processing callback for node 69408bb6575bc060ce15f2ea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bb6575bc060ce15f2ea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bb6575bc060ce15f2ea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bb6575bc060ce15f2ea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:38,841 - lava_callback - INFO - Starting async job submit for node 69408bd1575bc060ce15f35a 2025-12-17 22:38:38,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:38,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcf575bc060ce15f357 HTTP/1.1" 502 150 2025-12-17 22:38:38,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bcf575bc060ce15f357'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:38,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bcf575bc060ce15f357 2025-12-17 22:38:38,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcf575bc060ce15f357 HTTP/1.1" 502 150 2025-12-17 22:38:38,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bcf575bc060ce15f357'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:38,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd1575bc060ce15f35a HTTP/1.1" 502 150 2025-12-17 22:38:38,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd1575bc060ce15f35a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:38,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd1575bc060ce15f35a 2025-12-17 22:38:38,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd1575bc060ce15f35a HTTP/1.1" 502 150 2025-12-17 22:38:38,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd1575bc060ce15f35a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:38,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bcd575bc060ce15f354 2025-12-17 22:38:38,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcd575bc060ce15f354 HTTP/1.1" 502 150 2025-12-17 22:38:38,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bcd575bc060ce15f354'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:38,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bba575bc060ce15f2f0 2025-12-17 22:38:38,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bba575bc060ce15f2f0 HTTP/1.1" 502 150 2025-12-17 22:38:38,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bba575bc060ce15f2f0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:39,026 - lava_callback - INFO - Received callback for node 69408de4575bc060ce15faf8 INFO: 172.21.0.1:60408 - "POST /node/69408de4575bc060ce15faf8 HTTP/1.0" 202 Accepted 2025-12-17 22:38:39,678 - lava_callback - INFO - Received callback for node 69408de5575bc060ce15fafb INFO: 172.21.0.1:60424 - "POST /node/69408de5575bc060ce15fafb HTTP/1.0" 202 Accepted 2025-12-17 22:38:40,346 - lava_callback - INFO - Received callback for node 69408de6575bc060ce15fafe INFO: 172.21.0.1:44042 - "POST /node/69408de6575bc060ce15fafe HTTP/1.0" 202 Accepted 2025-12-17 22:38:40,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c26575bc060ce15f47a 2025-12-17 22:38:40,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c26575bc060ce15f47a HTTP/1.1" 502 150 2025-12-17 22:38:40,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c26575bc060ce15f47a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:40,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bcf575bc060ce15f357 2025-12-17 22:38:40,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcf575bc060ce15f357 HTTP/1.1" 502 150 2025-12-17 22:38:40,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bcf575bc060ce15f357'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:40,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd1575bc060ce15f35a 2025-12-17 22:38:40,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd1575bc060ce15f35a HTTP/1.1" 502 150 2025-12-17 22:38:40,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd1575bc060ce15f35a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:41,024 - lava_callback - INFO - Received callback for node 69408de8575bc060ce15fb01 INFO: 172.21.0.1:44054 - "POST /node/69408de8575bc060ce15fb01 HTTP/1.0" 202 Accepted 2025-12-17 22:38:41,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b9b575bc060ce15f2ac HTTP/1.1" 502 150 2025-12-17 22:38:41,328 - lava_callback - ERROR - Error processing callback for node 69408b9b575bc060ce15f2ac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b9b575bc060ce15f2ac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b9b575bc060ce15f2ac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b9b575bc060ce15f2ac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:41,330 - lava_callback - INFO - Starting async job submit for node 69408bd3575bc060ce15f35d 2025-12-17 22:38:41,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:41,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd3575bc060ce15f35d HTTP/1.1" 502 150 2025-12-17 22:38:41,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd3575bc060ce15f35d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:41,355 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd3575bc060ce15f35d 2025-12-17 22:38:41,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd3575bc060ce15f35d HTTP/1.1" 502 150 2025-12-17 22:38:41,356 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd3575bc060ce15f35d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:41,676 - lava_callback - INFO - Received callback for node 69408de9575bc060ce15fb04 INFO: 172.21.0.1:44066 - "POST /node/69408de9575bc060ce15fb04 HTTP/1.0" 202 Accepted 2025-12-17 22:38:41,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc3575bc060ce15f2fe 2025-12-17 22:38:41,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc3575bc060ce15f2fe HTTP/1.1" 502 150 2025-12-17 22:38:41,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc3575bc060ce15f2fe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:42,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bcd575bc060ce15f354 2025-12-17 22:38:42,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcd575bc060ce15f354 HTTP/1.1" 502 150 2025-12-17 22:38:42,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bcd575bc060ce15f354'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:43,356 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd3575bc060ce15f35d 2025-12-17 22:38:43,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd3575bc060ce15f35d HTTP/1.1" 502 150 2025-12-17 22:38:43,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd3575bc060ce15f35d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:43,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bcb575bc060ce15f34d 2025-12-17 22:38:43,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcb575bc060ce15f34d HTTP/1.1" 502 150 2025-12-17 22:38:43,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bcb575bc060ce15f34d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:44,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bb8575bc060ce15f2ed 2025-12-17 22:38:44,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bb8575bc060ce15f2ed HTTP/1.1" 502 150 2025-12-17 22:38:44,745 - lava_callback - ERROR - Error processing callback for node 69408bb8575bc060ce15f2ed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bb8575bc060ce15f2ed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bb8575bc060ce15f2ed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bb8575bc060ce15f2ed (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:44,747 - lava_callback - INFO - Starting async job submit for node 69408bd4575bc060ce15f360 2025-12-17 22:38:44,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:44,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd4575bc060ce15f360 HTTP/1.1" 502 150 2025-12-17 22:38:44,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd4575bc060ce15f360'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:44,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd4575bc060ce15f360 2025-12-17 22:38:44,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd4575bc060ce15f360 HTTP/1.1" 502 150 2025-12-17 22:38:44,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd4575bc060ce15f360'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:44,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bcf575bc060ce15f357 2025-12-17 22:38:44,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd1575bc060ce15f35a 2025-12-17 22:38:44,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcf575bc060ce15f357 HTTP/1.1" 502 150 2025-12-17 22:38:44,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bcf575bc060ce15f357'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:44,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd1575bc060ce15f35a HTTP/1.1" 502 150 2025-12-17 22:38:44,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd1575bc060ce15f35a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44068 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:38:46,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd4575bc060ce15f360 2025-12-17 22:38:46,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd4575bc060ce15f360 HTTP/1.1" 502 150 2025-12-17 22:38:46,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd4575bc060ce15f360'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:46,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ee14460bc8e265114da8 2025-12-17 22:38:46,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ee14460bc8e265114da8 HTTP/1.1" 502 150 2025-12-17 22:38:46,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ee14460bc8e265114da8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:46,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bba575bc060ce15f2f0 2025-12-17 22:38:46,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bba575bc060ce15f2f0 HTTP/1.1" 502 150 2025-12-17 22:38:46,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bba575bc060ce15f2f0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:47,357 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd3575bc060ce15f35d 2025-12-17 22:38:47,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd3575bc060ce15f35d HTTP/1.1" 502 150 2025-12-17 22:38:47,358 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd3575bc060ce15f35d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:47,359 - lava_callback - INFO - Received callback for node 6940fa30460bc8e265115bb6 INFO: 172.21.0.1:44080 - "POST /node/6940fa30460bc8e265115bb6 HTTP/1.0" 202 Accepted 2025-12-17 22:38:48,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c26575bc060ce15f47a 2025-12-17 22:38:48,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c26575bc060ce15f47a HTTP/1.1" 502 150 2025-12-17 22:38:48,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c26575bc060ce15f47a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:49,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc9575bc060ce15f33f HTTP/1.1" 502 150 2025-12-17 22:38:49,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc9575bc060ce15f33f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:49,765 - lava_callback - INFO - Received callback for node 69408dee575bc060ce15fb0d INFO: 172.21.0.1:36448 - "POST /node/69408dee575bc060ce15fb0d HTTP/1.0" 202 Accepted 2025-12-17 22:38:49,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc3575bc060ce15f2fe 2025-12-17 22:38:49,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc3575bc060ce15f2fe HTTP/1.1" 502 150 2025-12-17 22:38:49,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc3575bc060ce15f2fe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:50,461 - lava_callback - INFO - Received callback for node 69408df3575bc060ce15fb30 INFO: 172.21.0.1:36456 - "POST /node/69408df3575bc060ce15fb30 HTTP/1.0" 202 Accepted 2025-12-17 22:38:50,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bbd575bc060ce15f2f3 2025-12-17 22:38:50,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bbd575bc060ce15f2f3 HTTP/1.1" 502 150 2025-12-17 22:38:50,470 - lava_callback - ERROR - Error processing callback for node 69408bbd575bc060ce15f2f3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bbd575bc060ce15f2f3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bbd575bc060ce15f2f3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bbd575bc060ce15f2f3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:50,472 - lava_callback - INFO - Starting async job submit for node 69408bd5575bc060ce15f363 2025-12-17 22:38:50,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:50,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd5575bc060ce15f363 HTTP/1.1" 502 150 2025-12-17 22:38:50,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd5575bc060ce15f363'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:50,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd5575bc060ce15f363 2025-12-17 22:38:50,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd5575bc060ce15f363 HTTP/1.1" 502 150 2025-12-17 22:38:50,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd5575bc060ce15f363'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:50,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bbf575bc060ce15f2f6 2025-12-17 22:38:50,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bbf575bc060ce15f2f6 HTTP/1.1" 502 150 2025-12-17 22:38:50,719 - lava_callback - ERROR - Error processing callback for node 69408bbf575bc060ce15f2f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bbf575bc060ce15f2f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bbf575bc060ce15f2f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bbf575bc060ce15f2f6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:50,720 - lava_callback - INFO - Starting async job submit for node 69409289575bc060ce160ae7 2025-12-17 22:38:50,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc1575bc060ce15f2f9 2025-12-17 22:38:50,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc1575bc060ce15f2f9 HTTP/1.1" 502 150 2025-12-17 22:38:50,737 - lava_callback - ERROR - Error processing callback for node 69408bc1575bc060ce15f2f9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bc1575bc060ce15f2f9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bc1575bc060ce15f2f9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bc1575bc060ce15f2f9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:50,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd4575bc060ce15f360 2025-12-17 22:38:50,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:50,786 - lava_callback - INFO - Starting async job submit for node 69408bd6575bc060ce15f366 2025-12-17 22:38:50,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:50,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd4575bc060ce15f360 HTTP/1.1" 502 150 2025-12-17 22:38:50,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd4575bc060ce15f360'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:50,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409289575bc060ce160ae7 HTTP/1.1" 502 150 2025-12-17 22:38:50,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409289575bc060ce160ae7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:50,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409289575bc060ce160ae7 2025-12-17 22:38:50,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd6575bc060ce15f366 HTTP/1.1" 502 150 2025-12-17 22:38:50,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd6575bc060ce15f366'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:50,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd6575bc060ce15f366 2025-12-17 22:38:50,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409289575bc060ce160ae7 HTTP/1.1" 502 150 2025-12-17 22:38:50,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409289575bc060ce160ae7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:50,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd6575bc060ce15f366 HTTP/1.1" 502 150 2025-12-17 22:38:50,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd6575bc060ce15f366'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:50,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bcd575bc060ce15f354 2025-12-17 22:38:50,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcd575bc060ce15f354 HTTP/1.1" 502 150 2025-12-17 22:38:50,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bcd575bc060ce15f354'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:51,112 - lava_callback - INFO - Received callback for node 69408df4575bc060ce15fb33 INFO: 172.21.0.1:36472 - "POST /node/69408df4575bc060ce15fb33 HTTP/1.0" 202 Accepted 2025-12-17 22:38:51,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc5575bc060ce15f307 2025-12-17 22:38:51,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc5575bc060ce15f307 HTTP/1.1" 502 150 2025-12-17 22:38:51,782 - lava_callback - ERROR - Error processing callback for node 69408bc5575bc060ce15f307: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bc5575bc060ce15f307 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bc5575bc060ce15f307 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bc5575bc060ce15f307 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:51,784 - lava_callback - INFO - Starting async job submit for node 69408bd8575bc060ce15f36f 2025-12-17 22:38:51,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:51,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd8575bc060ce15f36f HTTP/1.1" 502 150 2025-12-17 22:38:51,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd8575bc060ce15f36f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:51,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd8575bc060ce15f36f 2025-12-17 22:38:51,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd8575bc060ce15f36f HTTP/1.1" 502 150 2025-12-17 22:38:51,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd8575bc060ce15f36f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:52,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd5575bc060ce15f363 2025-12-17 22:38:52,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd5575bc060ce15f363 HTTP/1.1" 502 150 2025-12-17 22:38:52,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd5575bc060ce15f363'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:52,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409289575bc060ce160ae7 2025-12-17 22:38:52,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd6575bc060ce15f366 2025-12-17 22:38:52,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409289575bc060ce160ae7 HTTP/1.1" 502 150 2025-12-17 22:38:52,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409289575bc060ce160ae7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:52,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd6575bc060ce15f366 HTTP/1.1" 502 150 2025-12-17 22:38:52,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd6575bc060ce15f366'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:52,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bcf575bc060ce15f357 2025-12-17 22:38:52,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd1575bc060ce15f35a 2025-12-17 22:38:52,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcf575bc060ce15f357 HTTP/1.1" 502 150 2025-12-17 22:38:52,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bcf575bc060ce15f357'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:52,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd1575bc060ce15f35a HTTP/1.1" 502 150 2025-12-17 22:38:52,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd1575bc060ce15f35a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:53,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd8575bc060ce15f36f 2025-12-17 22:38:53,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd8575bc060ce15f36f HTTP/1.1" 502 150 2025-12-17 22:38:53,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd8575bc060ce15f36f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:55,031 - lava_callback - INFO - Received callback for node 69408deb575bc060ce15fb07 INFO: 172.21.0.1:36478 - "POST /node/69408deb575bc060ce15fb07 HTTP/1.0" 202 Accepted 2025-12-17 22:38:55,359 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd3575bc060ce15f35d 2025-12-17 22:38:55,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd3575bc060ce15f35d HTTP/1.1" 502 150 2025-12-17 22:38:55,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd3575bc060ce15f35d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36492 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:38:56,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd5575bc060ce15f363 2025-12-17 22:38:56,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd5575bc060ce15f363 HTTP/1.1" 502 150 2025-12-17 22:38:56,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd5575bc060ce15f363'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:56,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409289575bc060ce160ae7 2025-12-17 22:38:56,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd6575bc060ce15f366 2025-12-17 22:38:56,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409289575bc060ce160ae7 HTTP/1.1" 502 150 2025-12-17 22:38:56,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409289575bc060ce160ae7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:56,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd6575bc060ce15f366 HTTP/1.1" 502 150 2025-12-17 22:38:56,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd6575bc060ce15f366'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:57,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc9575bc060ce15f33f 2025-12-17 22:38:57,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc9575bc060ce15f33f HTTP/1.1" 502 150 2025-12-17 22:38:57,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bc9575bc060ce15f33f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:57,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd8575bc060ce15f36f 2025-12-17 22:38:57,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd8575bc060ce15f36f HTTP/1.1" 502 150 2025-12-17 22:38:57,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd8575bc060ce15f36f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:58,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd4575bc060ce15f360 2025-12-17 22:38:58,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd4575bc060ce15f360 HTTP/1.1" 502 150 2025-12-17 22:38:58,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd4575bc060ce15f360'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:59,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bcb575bc060ce15f34d 2025-12-17 22:38:59,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcb575bc060ce15f34d HTTP/1.1" 502 150 2025-12-17 22:38:59,653 - lava_callback - ERROR - Error processing callback for node 69408bcb575bc060ce15f34d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bcb575bc060ce15f34d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bcb575bc060ce15f34d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bcb575bc060ce15f34d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:38:59,656 - lava_callback - INFO - Starting async job submit for node 69408bd9575bc060ce15f376 2025-12-17 22:38:59,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:38:59,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd9575bc060ce15f376 HTTP/1.1" 502 150 2025-12-17 22:38:59,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd9575bc060ce15f376'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:38:59,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd9575bc060ce15f376 2025-12-17 22:38:59,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd9575bc060ce15f376 HTTP/1.1" 502 150 2025-12-17 22:38:59,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd9575bc060ce15f376'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:01,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd9575bc060ce15f376 2025-12-17 22:39:01,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd9575bc060ce15f376 HTTP/1.1" 502 150 2025-12-17 22:39:01,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd9575bc060ce15f376'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43848 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:39:02,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ee14460bc8e265114da8 2025-12-17 22:39:02,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ee14460bc8e265114da8 HTTP/1.1" 502 150 2025-12-17 22:39:02,803 - lava_callback - ERROR - Error processing callback for node 6940ee14460bc8e265114da8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ee14460bc8e265114da8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ee14460bc8e265114da8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ee14460bc8e265114da8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:02,804 - lava_callback - INFO - Starting async job submit for node 69408bda575bc060ce15f379 2025-12-17 22:39:02,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:02,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bda575bc060ce15f379 HTTP/1.1" 502 150 2025-12-17 22:39:02,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bda575bc060ce15f379'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:02,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bda575bc060ce15f379 2025-12-17 22:39:02,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bda575bc060ce15f379 HTTP/1.1" 502 150 2025-12-17 22:39:02,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bda575bc060ce15f379'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:02,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bba575bc060ce15f2f0 2025-12-17 22:39:02,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bba575bc060ce15f2f0 HTTP/1.1" 502 150 2025-12-17 22:39:02,933 - lava_callback - ERROR - Error processing callback for node 69408bba575bc060ce15f2f0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bba575bc060ce15f2f0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bba575bc060ce15f2f0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bba575bc060ce15f2f0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:02,936 - lava_callback - INFO - Starting async job submit for node 69408bdb575bc060ce15f37c 2025-12-17 22:39:02,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:02,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bdb575bc060ce15f37c HTTP/1.1" 502 150 2025-12-17 22:39:02,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bdb575bc060ce15f37c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:02,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bdb575bc060ce15f37c 2025-12-17 22:39:02,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bdb575bc060ce15f37c HTTP/1.1" 502 150 2025-12-17 22:39:02,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bdb575bc060ce15f37c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:04,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd5575bc060ce15f363 2025-12-17 22:39:04,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd5575bc060ce15f363 HTTP/1.1" 502 150 2025-12-17 22:39:04,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd5575bc060ce15f363'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:04,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c26575bc060ce15f47a 2025-12-17 22:39:04,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c26575bc060ce15f47a HTTP/1.1" 502 150 2025-12-17 22:39:04,624 - lava_callback - ERROR - Error processing callback for node 69408c26575bc060ce15f47a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c26575bc060ce15f47a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c26575bc060ce15f47a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c26575bc060ce15f47a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:04,626 - lava_callback - INFO - Starting async job submit for node 69408bdd575bc060ce15f37f 2025-12-17 22:39:04,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:04,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bdd575bc060ce15f37f HTTP/1.1" 502 150 2025-12-17 22:39:04,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bdd575bc060ce15f37f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:04,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bdd575bc060ce15f37f 2025-12-17 22:39:04,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bdd575bc060ce15f37f HTTP/1.1" 502 150 2025-12-17 22:39:04,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bdd575bc060ce15f37f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:04,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409289575bc060ce160ae7 2025-12-17 22:39:04,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd6575bc060ce15f366 2025-12-17 22:39:04,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409289575bc060ce160ae7 HTTP/1.1" 502 150 2025-12-17 22:39:04,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409289575bc060ce160ae7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:04,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bda575bc060ce15f379 2025-12-17 22:39:04,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bda575bc060ce15f379 HTTP/1.1" 502 150 2025-12-17 22:39:04,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bda575bc060ce15f379'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:04,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408baa575bc060ce15f2d0 HTTP/1.1" 502 150 2025-12-17 22:39:04,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408baa575bc060ce15f2d0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:04,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bdb575bc060ce15f37c 2025-12-17 22:39:04,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bdb575bc060ce15f37c HTTP/1.1" 502 150 2025-12-17 22:39:04,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bdb575bc060ce15f37c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:05,211 - lava_callback - INFO - Received callback for node 69408df5575bc060ce15fb3a INFO: 172.21.0.1:43854 - "POST /node/69408df5575bc060ce15fb3a HTTP/1.0" 202 Accepted 2025-12-17 22:39:05,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd9575bc060ce15f376 2025-12-17 22:39:05,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd9575bc060ce15f376 HTTP/1.1" 502 150 2025-12-17 22:39:05,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd9575bc060ce15f376'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:05,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd8575bc060ce15f36f 2025-12-17 22:39:05,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd8575bc060ce15f36f HTTP/1.1" 502 150 2025-12-17 22:39:05,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd8575bc060ce15f36f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:05,846 - lava_callback - INFO - Received callback for node 69408df7575bc060ce15fb41 INFO: 172.21.0.1:43858 - "POST /node/69408df7575bc060ce15fb41 HTTP/1.0" 202 Accepted 2025-12-17 22:39:05,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc3575bc060ce15f2fe 2025-12-17 22:39:05,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc3575bc060ce15f2fe HTTP/1.1" 502 150 2025-12-17 22:39:05,922 - lava_callback - ERROR - Error processing callback for node 69408bc3575bc060ce15f2fe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bc3575bc060ce15f2fe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bc3575bc060ce15f2fe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bc3575bc060ce15f2fe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:05,924 - lava_callback - INFO - Starting async job submit for node 69408bde575bc060ce15f382 2025-12-17 22:39:05,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:05,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bde575bc060ce15f382 HTTP/1.1" 502 150 2025-12-17 22:39:05,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bde575bc060ce15f382'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:05,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bde575bc060ce15f382 2025-12-17 22:39:05,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bde575bc060ce15f382 HTTP/1.1" 502 150 2025-12-17 22:39:05,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bde575bc060ce15f382'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:06,498 - lava_callback - INFO - Received callback for node 69408df8575bc060ce15fb46 INFO: 172.21.0.1:43870 - "POST /node/69408df8575bc060ce15fb46 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:43884 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:39:06,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bdd575bc060ce15f37f 2025-12-17 22:39:06,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bdd575bc060ce15f37f HTTP/1.1" 502 150 2025-12-17 22:39:06,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bdd575bc060ce15f37f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:06,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408baa575bc060ce15f2d0 2025-12-17 22:39:06,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408baa575bc060ce15f2d0 HTTP/1.1" 502 150 2025-12-17 22:39:06,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408baa575bc060ce15f2d0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:06,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bcd575bc060ce15f354 2025-12-17 22:39:06,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcd575bc060ce15f354 HTTP/1.1" 502 150 2025-12-17 22:39:06,909 - lava_callback - ERROR - Error processing callback for node 69408bcd575bc060ce15f354: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bcd575bc060ce15f354 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bcd575bc060ce15f354 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bcd575bc060ce15f354 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:06,912 - lava_callback - INFO - Starting async job submit for node 69408be0575bc060ce15f389 2025-12-17 22:39:06,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:06,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be0575bc060ce15f389 HTTP/1.1" 502 150 2025-12-17 22:39:06,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be0575bc060ce15f389'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:06,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be0575bc060ce15f389 2025-12-17 22:39:06,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be0575bc060ce15f389 HTTP/1.1" 502 150 2025-12-17 22:39:06,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be0575bc060ce15f389'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:07,165 - lava_callback - INFO - Received callback for node 69408df9575bc060ce15fb49 INFO: 172.21.0.1:43888 - "POST /node/69408df9575bc060ce15fb49 HTTP/1.0" 202 Accepted 2025-12-17 22:39:07,819 - lava_callback - INFO - Received callback for node 69408dfb575bc060ce15fb4c INFO: 172.21.0.1:43894 - "POST /node/69408dfb575bc060ce15fb4c HTTP/1.0" 202 Accepted 2025-12-17 22:39:07,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bde575bc060ce15f382 2025-12-17 22:39:07,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bde575bc060ce15f382 HTTP/1.1" 502 150 2025-12-17 22:39:07,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bde575bc060ce15f382'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:08,469 - lava_callback - INFO - Received callback for node 69408dfc575bc060ce15fb4f INFO: 172.21.0.1:43896 - "POST /node/69408dfc575bc060ce15fb4f HTTP/1.0" 202 Accepted 2025-12-17 22:39:08,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bda575bc060ce15f379 2025-12-17 22:39:08,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bda575bc060ce15f379 HTTP/1.1" 502 150 2025-12-17 22:39:08,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bda575bc060ce15f379'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:08,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bcf575bc060ce15f357 2025-12-17 22:39:08,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd1575bc060ce15f35a 2025-12-17 22:39:08,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bcf575bc060ce15f357 HTTP/1.1" 502 150 2025-12-17 22:39:08,859 - lava_callback - ERROR - Error processing callback for node 69408bcf575bc060ce15f357: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bcf575bc060ce15f357 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bcf575bc060ce15f357 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bcf575bc060ce15f357 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:08,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd1575bc060ce15f35a HTTP/1.1" 502 150 2025-12-17 22:39:08,861 - lava_callback - ERROR - Error processing callback for node 69408bd1575bc060ce15f35a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd1575bc060ce15f35a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd1575bc060ce15f35a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd1575bc060ce15f35a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:08,861 - lava_callback - INFO - Starting async job submit for node 69408be1575bc060ce15f38c 2025-12-17 22:39:08,863 - lava_callback - INFO - Starting async job submit for node 69408be3575bc060ce15f38f 2025-12-17 22:39:08,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:08,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:08,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be1575bc060ce15f38c HTTP/1.1" 502 150 2025-12-17 22:39:08,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be1575bc060ce15f38c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:08,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be1575bc060ce15f38c 2025-12-17 22:39:08,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be1575bc060ce15f38c HTTP/1.1" 502 150 2025-12-17 22:39:08,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be1575bc060ce15f38c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:08,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be3575bc060ce15f38f HTTP/1.1" 502 150 2025-12-17 22:39:08,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be3575bc060ce15f38f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:08,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be3575bc060ce15f38f 2025-12-17 22:39:08,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be3575bc060ce15f38f HTTP/1.1" 502 150 2025-12-17 22:39:08,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be3575bc060ce15f38f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:08,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be0575bc060ce15f389 2025-12-17 22:39:08,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be0575bc060ce15f389 HTTP/1.1" 502 150 2025-12-17 22:39:08,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be0575bc060ce15f389'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:08,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bdb575bc060ce15f37c 2025-12-17 22:39:08,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bdb575bc060ce15f37c HTTP/1.1" 502 150 2025-12-17 22:39:08,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bdb575bc060ce15f37c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:09,147 - lava_callback - INFO - Received callback for node 69408dfd575bc060ce15fb52 INFO: 172.21.0.1:43906 - "POST /node/69408dfd575bc060ce15fb52 HTTP/1.0" 202 Accepted 2025-12-17 22:39:09,798 - lava_callback - INFO - Received callback for node 69408e04575bc060ce15fb5b INFO: 172.21.0.1:38970 - "POST /node/69408e04575bc060ce15fb5b HTTP/1.0" 202 Accepted 2025-12-17 22:39:10,446 - lava_callback - INFO - Received callback for node 69408e05575bc060ce15fb5e INFO: 172.21.0.1:38982 - "POST /node/69408e05575bc060ce15fb5e HTTP/1.0" 202 Accepted 2025-12-17 22:39:10,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bdd575bc060ce15f37f 2025-12-17 22:39:10,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bdd575bc060ce15f37f HTTP/1.1" 502 150 2025-12-17 22:39:10,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bdd575bc060ce15f37f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:10,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be1575bc060ce15f38c 2025-12-17 22:39:10,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be1575bc060ce15f38c HTTP/1.1" 502 150 2025-12-17 22:39:10,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be1575bc060ce15f38c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:10,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be3575bc060ce15f38f 2025-12-17 22:39:10,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be3575bc060ce15f38f HTTP/1.1" 502 150 2025-12-17 22:39:10,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be3575bc060ce15f38f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:10,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408baa575bc060ce15f2d0 2025-12-17 22:39:10,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408baa575bc060ce15f2d0 HTTP/1.1" 502 150 2025-12-17 22:39:10,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408baa575bc060ce15f2d0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:11,115 - lava_callback - INFO - Received callback for node 69408e07575bc060ce15fb61 INFO: 172.21.0.1:38988 - "POST /node/69408e07575bc060ce15fb61 HTTP/1.0" 202 Accepted 2025-12-17 22:39:11,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd3575bc060ce15f35d 2025-12-17 22:39:11,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd3575bc060ce15f35d HTTP/1.1" 502 150 2025-12-17 22:39:11,362 - lava_callback - ERROR - Error processing callback for node 69408bd3575bc060ce15f35d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd3575bc060ce15f35d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd3575bc060ce15f35d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd3575bc060ce15f35d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:11,363 - lava_callback - INFO - Starting async job submit for node 69408be4575bc060ce15f392 2025-12-17 22:39:11,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:11,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be4575bc060ce15f392 HTTP/1.1" 502 150 2025-12-17 22:39:11,371 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be4575bc060ce15f392'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:11,371 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be4575bc060ce15f392 2025-12-17 22:39:11,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be4575bc060ce15f392 HTTP/1.1" 502 150 2025-12-17 22:39:11,372 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be4575bc060ce15f392'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:11,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bde575bc060ce15f382 2025-12-17 22:39:11,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bde575bc060ce15f382 HTTP/1.1" 502 150 2025-12-17 22:39:11,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bde575bc060ce15f382'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:12,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be0575bc060ce15f389 2025-12-17 22:39:12,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be0575bc060ce15f389 HTTP/1.1" 502 150 2025-12-17 22:39:12,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be0575bc060ce15f389'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:13,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bc9575bc060ce15f33f 2025-12-17 22:39:13,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bc9575bc060ce15f33f HTTP/1.1" 502 150 2025-12-17 22:39:13,268 - lava_callback - ERROR - Error processing callback for node 69408bc9575bc060ce15f33f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bc9575bc060ce15f33f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bc9575bc060ce15f33f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bc9575bc060ce15f33f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:13,269 - lava_callback - INFO - Starting async job submit for node 69408be5575bc060ce15f395 2025-12-17 22:39:13,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:13,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be5575bc060ce15f395 HTTP/1.1" 502 150 2025-12-17 22:39:13,285 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be5575bc060ce15f395'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:13,285 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be5575bc060ce15f395 2025-12-17 22:39:13,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be5575bc060ce15f395 HTTP/1.1" 502 150 2025-12-17 22:39:13,286 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be5575bc060ce15f395'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:13,372 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be4575bc060ce15f392 2025-12-17 22:39:13,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be4575bc060ce15f392 HTTP/1.1" 502 150 2025-12-17 22:39:13,379 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be4575bc060ce15f392'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:13,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd9575bc060ce15f376 2025-12-17 22:39:13,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd9575bc060ce15f376 HTTP/1.1" 502 150 2025-12-17 22:39:13,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd9575bc060ce15f376'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:14,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd4575bc060ce15f360 2025-12-17 22:39:14,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd4575bc060ce15f360 HTTP/1.1" 502 150 2025-12-17 22:39:14,791 - lava_callback - ERROR - Error processing callback for node 69408bd4575bc060ce15f360: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd4575bc060ce15f360 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd4575bc060ce15f360 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd4575bc060ce15f360 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:14,792 - lava_callback - INFO - Starting async job submit for node 69408bee575bc060ce15f3b0 2025-12-17 22:39:14,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:14,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bee575bc060ce15f3b0 HTTP/1.1" 502 150 2025-12-17 22:39:14,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bee575bc060ce15f3b0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:14,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bee575bc060ce15f3b0 2025-12-17 22:39:14,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bee575bc060ce15f3b0 HTTP/1.1" 502 150 2025-12-17 22:39:14,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bee575bc060ce15f3b0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:14,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be1575bc060ce15f38c 2025-12-17 22:39:14,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be3575bc060ce15f38f 2025-12-17 22:39:14,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be1575bc060ce15f38c HTTP/1.1" 502 150 2025-12-17 22:39:14,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be1575bc060ce15f38c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:14,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be3575bc060ce15f38f HTTP/1.1" 502 150 2025-12-17 22:39:14,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be3575bc060ce15f38f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:15,021 - lava_callback - INFO - Received callback for node 69408dff575bc060ce15fb55 INFO: 172.21.0.1:39004 - "POST /node/69408dff575bc060ce15fb55 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:39014 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:39:15,286 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be5575bc060ce15f395 2025-12-17 22:39:15,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be5575bc060ce15f395 HTTP/1.1" 502 150 2025-12-17 22:39:15,287 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be5575bc060ce15f395'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:15,717 - lava_callback - INFO - Received callback for node 69408e02575bc060ce15fb58 INFO: 172.21.0.1:39018 - "POST /node/69408e02575bc060ce15fb58 HTTP/1.0" 202 Accepted 2025-12-17 22:39:16,407 - lava_callback - INFO - Received callback for node 69408e0c575bc060ce15fb6f INFO: 172.21.0.1:39032 - "POST /node/69408e0c575bc060ce15fb6f HTTP/1.0" 202 Accepted INFO: 172.21.0.1:39040 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:39:16,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bee575bc060ce15f3b0 2025-12-17 22:39:16,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bee575bc060ce15f3b0 HTTP/1.1" 502 150 2025-12-17 22:39:16,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bee575bc060ce15f3b0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:16,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bda575bc060ce15f379 2025-12-17 22:39:16,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bda575bc060ce15f379 HTTP/1.1" 502 150 2025-12-17 22:39:16,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bda575bc060ce15f379'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:16,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bdb575bc060ce15f37c 2025-12-17 22:39:16,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bdb575bc060ce15f37c HTTP/1.1" 502 150 2025-12-17 22:39:16,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bdb575bc060ce15f37c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:17,064 - lava_callback - INFO - Received callback for node 69408e10575bc060ce15fb75 INFO: 172.21.0.1:39054 - "POST /node/69408e10575bc060ce15fb75 HTTP/1.0" 202 Accepted 2025-12-17 22:39:17,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be4575bc060ce15f392 2025-12-17 22:39:17,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be4575bc060ce15f392 HTTP/1.1" 502 150 2025-12-17 22:39:17,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be4575bc060ce15f392'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:18,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bdd575bc060ce15f37f 2025-12-17 22:39:18,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bdd575bc060ce15f37f HTTP/1.1" 502 150 2025-12-17 22:39:18,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bdd575bc060ce15f37f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:18,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408baa575bc060ce15f2d0 2025-12-17 22:39:19,287 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be5575bc060ce15f395 2025-12-17 22:39:19,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be5575bc060ce15f395 HTTP/1.1" 502 150 2025-12-17 22:39:19,288 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be5575bc060ce15f395'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:19,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bde575bc060ce15f382 2025-12-17 22:39:19,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bde575bc060ce15f382 HTTP/1.1" 502 150 2025-12-17 22:39:19,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bde575bc060ce15f382'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:20,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd5575bc060ce15f363 2025-12-17 22:39:20,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd5575bc060ce15f363 HTTP/1.1" 502 150 2025-12-17 22:39:20,517 - lava_callback - ERROR - Error processing callback for node 69408bd5575bc060ce15f363: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd5575bc060ce15f363 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd5575bc060ce15f363 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd5575bc060ce15f363 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:20,520 - lava_callback - INFO - Starting async job submit for node 69408bf1575bc060ce15f3bf 2025-12-17 22:39:20,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:20,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf1575bc060ce15f3bf HTTP/1.1" 502 150 2025-12-17 22:39:20,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf1575bc060ce15f3bf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:20,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf1575bc060ce15f3bf 2025-12-17 22:39:20,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf1575bc060ce15f3bf HTTP/1.1" 502 150 2025-12-17 22:39:20,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf1575bc060ce15f3bf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:20,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409289575bc060ce160ae7 2025-12-17 22:39:20,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409289575bc060ce160ae7 HTTP/1.1" 502 150 2025-12-17 22:39:20,799 - lava_callback - ERROR - Error processing callback for node 69409289575bc060ce160ae7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409289575bc060ce160ae7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409289575bc060ce160ae7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409289575bc060ce160ae7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:20,801 - lava_callback - INFO - Starting async job submit for node 69408bf2575bc060ce15f3c2 2025-12-17 22:39:20,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bee575bc060ce15f3b0 2025-12-17 22:39:20,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:20,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bee575bc060ce15f3b0 HTTP/1.1" 502 150 2025-12-17 22:39:20,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bee575bc060ce15f3b0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:20,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf2575bc060ce15f3c2 HTTP/1.1" 502 150 2025-12-17 22:39:20,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf2575bc060ce15f3c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:20,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf2575bc060ce15f3c2 2025-12-17 22:39:20,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf2575bc060ce15f3c2 HTTP/1.1" 502 150 2025-12-17 22:39:20,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf2575bc060ce15f3c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:20,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be0575bc060ce15f389 2025-12-17 22:39:20,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be0575bc060ce15f389 HTTP/1.1" 502 150 2025-12-17 22:39:20,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be0575bc060ce15f389'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:21,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd8575bc060ce15f36f 2025-12-17 22:39:21,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd8575bc060ce15f36f HTTP/1.1" 502 150 2025-12-17 22:39:21,798 - lava_callback - ERROR - Error processing callback for node 69408bd8575bc060ce15f36f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd8575bc060ce15f36f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd8575bc060ce15f36f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd8575bc060ce15f36f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:21,800 - lava_callback - INFO - Starting async job submit for node 69408bf3575bc060ce15f3c5 2025-12-17 22:39:21,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:21,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf3575bc060ce15f3c5 HTTP/1.1" 502 150 2025-12-17 22:39:21,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf3575bc060ce15f3c5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:21,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf3575bc060ce15f3c5 2025-12-17 22:39:21,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf3575bc060ce15f3c5 HTTP/1.1" 502 150 2025-12-17 22:39:21,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf3575bc060ce15f3c5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:22,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf1575bc060ce15f3bf 2025-12-17 22:39:22,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf1575bc060ce15f3bf HTTP/1.1" 502 150 2025-12-17 22:39:22,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf1575bc060ce15f3bf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:22,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf2575bc060ce15f3c2 2025-12-17 22:39:22,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf2575bc060ce15f3c2 HTTP/1.1" 502 150 2025-12-17 22:39:22,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf2575bc060ce15f3c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:22,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be1575bc060ce15f38c 2025-12-17 22:39:22,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be3575bc060ce15f38f 2025-12-17 22:39:22,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be1575bc060ce15f38c HTTP/1.1" 502 150 2025-12-17 22:39:22,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be1575bc060ce15f38c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:22,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be3575bc060ce15f38f HTTP/1.1" 502 150 2025-12-17 22:39:22,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be3575bc060ce15f38f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:23,142 - lava_callback - INFO - Received callback for node 69408e0e575bc060ce15fb72 INFO: 172.21.0.1:43870 - "POST /node/69408e0e575bc060ce15fb72 HTTP/1.0" 202 Accepted 2025-12-17 22:39:23,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf3575bc060ce15f3c5 2025-12-17 22:39:25,218 - lava_callback - INFO - Received callback for node 69408e08575bc060ce15fb64 INFO: 172.21.0.1:43886 - "POST /node/69408e08575bc060ce15fb64 HTTP/1.0" 202 Accepted 2025-12-17 22:39:25,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be4575bc060ce15f392 2025-12-17 22:39:25,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be4575bc060ce15f392 HTTP/1.1" 502 150 2025-12-17 22:39:25,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be4575bc060ce15f392'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:25,889 - lava_callback - INFO - Received callback for node 69408e09575bc060ce15fb69 INFO: 172.21.0.1:43890 - "POST /node/69408e09575bc060ce15fb69 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:43902 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:39:26,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf1575bc060ce15f3bf 2025-12-17 22:39:26,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf1575bc060ce15f3bf HTTP/1.1" 502 150 2025-12-17 22:39:26,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf1575bc060ce15f3bf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:26,537 - lava_callback - INFO - Received callback for node 69408e0b575bc060ce15fb6c INFO: 172.21.0.1:43916 - "POST /node/69408e0b575bc060ce15fb6c HTTP/1.0" 202 Accepted 2025-12-17 22:39:26,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf2575bc060ce15f3c2 2025-12-17 22:39:26,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf2575bc060ce15f3c2 HTTP/1.1" 502 150 2025-12-17 22:39:26,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf2575bc060ce15f3c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:27,289 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be5575bc060ce15f395 2025-12-17 22:39:27,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be5575bc060ce15f395 HTTP/1.1" 502 150 2025-12-17 22:39:27,290 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408be5575bc060ce15f395'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:28,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bee575bc060ce15f3b0 2025-12-17 22:39:28,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bee575bc060ce15f3b0 HTTP/1.1" 502 150 2025-12-17 22:39:28,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bee575bc060ce15f3b0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:29,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd9575bc060ce15f376 2025-12-17 22:39:29,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd9575bc060ce15f376 HTTP/1.1" 502 150 2025-12-17 22:39:29,672 - lava_callback - ERROR - Error processing callback for node 69408bd9575bc060ce15f376: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd9575bc060ce15f376 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd9575bc060ce15f376 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd9575bc060ce15f376 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:29,674 - lava_callback - INFO - Starting async job submit for node 69408bf5575bc060ce15f3c8 2025-12-17 22:39:29,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:29,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf5575bc060ce15f3c8 HTTP/1.1" 502 150 2025-12-17 22:39:29,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf5575bc060ce15f3c8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:29,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf5575bc060ce15f3c8 2025-12-17 22:39:29,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf5575bc060ce15f3c8 HTTP/1.1" 502 150 2025-12-17 22:39:29,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf5575bc060ce15f3c8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:31,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf5575bc060ce15f3c8 2025-12-17 22:39:31,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf5575bc060ce15f3c8 HTTP/1.1" 502 150 2025-12-17 22:39:31,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf5575bc060ce15f3c8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:32,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bda575bc060ce15f379 2025-12-17 22:39:32,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bda575bc060ce15f379 HTTP/1.1" 502 150 2025-12-17 22:39:32,817 - lava_callback - ERROR - Error processing callback for node 69408bda575bc060ce15f379: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bda575bc060ce15f379 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bda575bc060ce15f379 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bda575bc060ce15f379 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:32,819 - lava_callback - INFO - Starting async job submit for node 694092fd575bc060ce160bf7 2025-12-17 22:39:32,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:32,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fd575bc060ce160bf7 HTTP/1.1" 502 150 2025-12-17 22:39:32,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092fd575bc060ce160bf7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:32,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092fd575bc060ce160bf7 2025-12-17 22:39:32,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fd575bc060ce160bf7 HTTP/1.1" 502 150 2025-12-17 22:39:32,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092fd575bc060ce160bf7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:32,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bdb575bc060ce15f37c 2025-12-17 22:39:32,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bdb575bc060ce15f37c HTTP/1.1" 502 150 2025-12-17 22:39:32,970 - lava_callback - ERROR - Error processing callback for node 69408bdb575bc060ce15f37c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bdb575bc060ce15f37c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bdb575bc060ce15f37c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bdb575bc060ce15f37c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:32,971 - lava_callback - INFO - Starting async job submit for node 69408bea575bc060ce15f3a1 2025-12-17 22:39:32,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:32,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bea575bc060ce15f3a1 HTTP/1.1" 502 150 2025-12-17 22:39:32,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bea575bc060ce15f3a1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:32,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bea575bc060ce15f3a1 2025-12-17 22:39:32,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bea575bc060ce15f3a1 HTTP/1.1" 502 150 2025-12-17 22:39:32,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bea575bc060ce15f3a1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:34,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf1575bc060ce15f3bf 2025-12-17 22:39:34,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf1575bc060ce15f3bf HTTP/1.1" 502 150 2025-12-17 22:39:34,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf1575bc060ce15f3bf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:34,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bdd575bc060ce15f37f 2025-12-17 22:39:34,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bdd575bc060ce15f37f HTTP/1.1" 502 150 2025-12-17 22:39:34,641 - lava_callback - ERROR - Error processing callback for node 69408bdd575bc060ce15f37f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bdd575bc060ce15f37f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bdd575bc060ce15f37f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bdd575bc060ce15f37f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:34,643 - lava_callback - INFO - Starting async job submit for node 69408beb575bc060ce15f3a4 2025-12-17 22:39:34,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:34,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408beb575bc060ce15f3a4 HTTP/1.1" 502 150 2025-12-17 22:39:34,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408beb575bc060ce15f3a4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:34,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408beb575bc060ce15f3a4 2025-12-17 22:39:34,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408beb575bc060ce15f3a4 HTTP/1.1" 502 150 2025-12-17 22:39:34,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408beb575bc060ce15f3a4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:34,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf2575bc060ce15f3c2 2025-12-17 22:39:34,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf2575bc060ce15f3c2 HTTP/1.1" 502 150 2025-12-17 22:39:34,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf2575bc060ce15f3c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:34,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092fd575bc060ce160bf7 2025-12-17 22:39:34,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fd575bc060ce160bf7 HTTP/1.1" 502 150 2025-12-17 22:39:34,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092fd575bc060ce160bf7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:34,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bea575bc060ce15f3a1 2025-12-17 22:39:34,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bea575bc060ce15f3a1 HTTP/1.1" 502 150 2025-12-17 22:39:34,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bea575bc060ce15f3a1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:35,157 - lava_callback - INFO - Received callback for node 69408e12575bc060ce15fb78 INFO: 172.21.0.1:55780 - "POST /node/69408e12575bc060ce15fb78 HTTP/1.0" 202 Accepted 2025-12-17 22:39:35,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd6575bc060ce15f366 HTTP/1.1" 502 150 2025-12-17 22:39:35,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bd6575bc060ce15f366'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:35,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf5575bc060ce15f3c8 2025-12-17 22:39:35,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf5575bc060ce15f3c8 HTTP/1.1" 502 150 2025-12-17 22:39:35,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf5575bc060ce15f3c8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:35,812 - lava_callback - INFO - Received callback for node 69408e14575bc060ce15fb7b INFO: 172.21.0.1:55784 - "POST /node/69408e14575bc060ce15fb7b HTTP/1.0" 202 Accepted 2025-12-17 22:39:35,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bde575bc060ce15f382 2025-12-17 22:39:35,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bde575bc060ce15f382 HTTP/1.1" 502 150 2025-12-17 22:39:35,964 - lava_callback - ERROR - Error processing callback for node 69408bde575bc060ce15f382: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bde575bc060ce15f382 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bde575bc060ce15f382 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bde575bc060ce15f382 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:35,966 - lava_callback - INFO - Starting async job submit for node 69408bed575bc060ce15f3a9 2025-12-17 22:39:35,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:35,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bed575bc060ce15f3a9 HTTP/1.1" 502 150 2025-12-17 22:39:35,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bed575bc060ce15f3a9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:35,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bed575bc060ce15f3a9 2025-12-17 22:39:35,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bed575bc060ce15f3a9 HTTP/1.1" 502 150 2025-12-17 22:39:35,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bed575bc060ce15f3a9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:36,484 - lava_callback - INFO - Received callback for node 69408e16575bc060ce15fb7e INFO: 172.21.0.1:55800 - "POST /node/69408e16575bc060ce15fb7e HTTP/1.0" 202 Accepted INFO: 172.21.0.1:55806 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:39:36,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408beb575bc060ce15f3a4 2025-12-17 22:39:36,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408beb575bc060ce15f3a4 HTTP/1.1" 502 150 2025-12-17 22:39:36,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408beb575bc060ce15f3a4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:36,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be0575bc060ce15f389 2025-12-17 22:39:36,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be0575bc060ce15f389 HTTP/1.1" 502 150 2025-12-17 22:39:36,925 - lava_callback - ERROR - Error processing callback for node 69408be0575bc060ce15f389: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be0575bc060ce15f389 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be0575bc060ce15f389 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be0575bc060ce15f389 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:36,927 - lava_callback - INFO - Starting async job submit for node 69408bf7575bc060ce15f3ce 2025-12-17 22:39:36,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:36,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf7575bc060ce15f3ce HTTP/1.1" 502 150 2025-12-17 22:39:36,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf7575bc060ce15f3ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:36,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf7575bc060ce15f3ce 2025-12-17 22:39:36,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf7575bc060ce15f3ce HTTP/1.1" 502 150 2025-12-17 22:39:36,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf7575bc060ce15f3ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:37,166 - lava_callback - INFO - Received callback for node 69408e18575bc060ce15fb81 INFO: 172.21.0.1:55808 - "POST /node/69408e18575bc060ce15fb81 HTTP/1.0" 202 Accepted 2025-12-17 22:39:37,828 - lava_callback - INFO - Received callback for node 69408e1a575bc060ce15fb84 INFO: 172.21.0.1:55810 - "POST /node/69408e1a575bc060ce15fb84 HTTP/1.0" 202 Accepted 2025-12-17 22:39:37,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bed575bc060ce15f3a9 2025-12-17 22:39:37,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bed575bc060ce15f3a9 HTTP/1.1" 502 150 2025-12-17 22:39:37,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bed575bc060ce15f3a9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:38,479 - lava_callback - INFO - Received callback for node 69408e1b575bc060ce15fb87 INFO: 172.21.0.1:55822 - "POST /node/69408e1b575bc060ce15fb87 HTTP/1.0" 202 Accepted 2025-12-17 22:39:38,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be1575bc060ce15f38c 2025-12-17 22:39:38,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be3575bc060ce15f38f 2025-12-17 22:39:38,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be1575bc060ce15f38c HTTP/1.1" 502 150 2025-12-17 22:39:38,880 - lava_callback - ERROR - Error processing callback for node 69408be1575bc060ce15f38c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be1575bc060ce15f38c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be1575bc060ce15f38c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be1575bc060ce15f38c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:38,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be3575bc060ce15f38f HTTP/1.1" 502 150 2025-12-17 22:39:38,883 - lava_callback - ERROR - Error processing callback for node 69408be3575bc060ce15f38f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be3575bc060ce15f38f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be3575bc060ce15f38f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be3575bc060ce15f38f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:38,883 - lava_callback - INFO - Starting async job submit for node 69408bfa575bc060ce15f3d1 2025-12-17 22:39:38,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:38,885 - lava_callback - INFO - Starting async job submit for node 69408bfc575bc060ce15f3d4 2025-12-17 22:39:38,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:38,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfc575bc060ce15f3d4 HTTP/1.1" 502 150 2025-12-17 22:39:38,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bfc575bc060ce15f3d4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:38,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bfc575bc060ce15f3d4 2025-12-17 22:39:38,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092fd575bc060ce160bf7 2025-12-17 22:39:38,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfc575bc060ce15f3d4 HTTP/1.1" 502 150 2025-12-17 22:39:38,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bfc575bc060ce15f3d4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:38,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fd575bc060ce160bf7 HTTP/1.1" 502 150 2025-12-17 22:39:38,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092fd575bc060ce160bf7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:38,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfa575bc060ce15f3d1 HTTP/1.1" 502 150 2025-12-17 22:39:38,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bfa575bc060ce15f3d1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:38,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bfa575bc060ce15f3d1 2025-12-17 22:39:38,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfa575bc060ce15f3d1 HTTP/1.1" 502 150 2025-12-17 22:39:38,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bfa575bc060ce15f3d1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:38,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf7575bc060ce15f3ce 2025-12-17 22:39:38,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf7575bc060ce15f3ce HTTP/1.1" 502 150 2025-12-17 22:39:38,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf7575bc060ce15f3ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:38,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bea575bc060ce15f3a1 2025-12-17 22:39:38,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bea575bc060ce15f3a1 HTTP/1.1" 502 150 2025-12-17 22:39:38,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bea575bc060ce15f3a1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:39,140 - lava_callback - INFO - Received callback for node 69408e1d575bc060ce15fb8d INFO: 172.21.0.1:55830 - "POST /node/69408e1d575bc060ce15fb8d HTTP/1.0" 202 Accepted 2025-12-17 22:39:39,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf3575bc060ce15f3c5 HTTP/1.1" 502 150 2025-12-17 22:39:39,184 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf3575bc060ce15f3c5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:39,798 - lava_callback - INFO - Received callback for node 69408e20575bc060ce15fb93 INFO: 172.21.0.1:58046 - "POST /node/69408e20575bc060ce15fb93 HTTP/1.0" 202 Accepted 2025-12-17 22:39:40,453 - lava_callback - INFO - Received callback for node 69408e22575bc060ce15fb9a INFO: 172.21.0.1:58054 - "POST /node/69408e22575bc060ce15fb9a HTTP/1.0" 202 Accepted 2025-12-17 22:39:40,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408beb575bc060ce15f3a4 2025-12-17 22:39:40,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408beb575bc060ce15f3a4 HTTP/1.1" 502 150 2025-12-17 22:39:40,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408beb575bc060ce15f3a4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:40,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bfc575bc060ce15f3d4 2025-12-17 22:39:40,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfc575bc060ce15f3d4 HTTP/1.1" 502 150 2025-12-17 22:39:40,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bfc575bc060ce15f3d4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:40,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bfa575bc060ce15f3d1 2025-12-17 22:39:40,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfa575bc060ce15f3d1 HTTP/1.1" 502 150 2025-12-17 22:39:40,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bfa575bc060ce15f3d1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:41,119 - lava_callback - INFO - Received callback for node 69408e23575bc060ce15fb9d INFO: 172.21.0.1:58068 - "POST /node/69408e23575bc060ce15fb9d HTTP/1.0" 202 Accepted 2025-12-17 22:39:41,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be4575bc060ce15f392 2025-12-17 22:39:41,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be4575bc060ce15f392 HTTP/1.1" 502 150 2025-12-17 22:39:41,384 - lava_callback - ERROR - Error processing callback for node 69408be4575bc060ce15f392: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be4575bc060ce15f392 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be4575bc060ce15f392 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be4575bc060ce15f392 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:41,386 - lava_callback - INFO - Starting async job submit for node 69408bfe575bc060ce15f3d7 2025-12-17 22:39:41,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:41,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfe575bc060ce15f3d7 HTTP/1.1" 502 150 2025-12-17 22:39:41,407 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bfe575bc060ce15f3d7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:41,407 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bfe575bc060ce15f3d7 2025-12-17 22:39:41,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfe575bc060ce15f3d7 HTTP/1.1" 502 150 2025-12-17 22:39:41,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bfe575bc060ce15f3d7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:41,780 - lava_callback - INFO - Received callback for node 69408e24575bc060ce15fba0 INFO: 172.21.0.1:58074 - "POST /node/69408e24575bc060ce15fba0 HTTP/1.0" 202 Accepted 2025-12-17 22:39:41,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bed575bc060ce15f3a9 2025-12-17 22:39:41,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bed575bc060ce15f3a9 HTTP/1.1" 502 150 2025-12-17 22:39:41,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bed575bc060ce15f3a9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:42,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf7575bc060ce15f3ce 2025-12-17 22:39:42,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf7575bc060ce15f3ce HTTP/1.1" 502 150 2025-12-17 22:39:42,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf7575bc060ce15f3ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:43,053 - lava_callback - INFO - Received callback for node 69408e1f575bc060ce15fb90 INFO: 172.21.0.1:58080 - "POST /node/69408e1f575bc060ce15fb90 HTTP/1.0" 202 Accepted 2025-12-17 22:39:43,185 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf3575bc060ce15f3c5 2025-12-17 22:39:43,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf3575bc060ce15f3c5 HTTP/1.1" 502 150 2025-12-17 22:39:43,186 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf3575bc060ce15f3c5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:43,290 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408be5575bc060ce15f395 2025-12-17 22:39:43,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408be5575bc060ce15f395 HTTP/1.1" 502 150 2025-12-17 22:39:43,291 - lava_callback - ERROR - Error processing callback for node 69408be5575bc060ce15f395: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be5575bc060ce15f395 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be5575bc060ce15f395 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408be5575bc060ce15f395 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:43,293 - lava_callback - INFO - Starting async job submit for node 69408bf6575bc060ce15f3cb 2025-12-17 22:39:43,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:43,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf6575bc060ce15f3cb HTTP/1.1" 502 150 2025-12-17 22:39:43,300 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf6575bc060ce15f3cb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:43,300 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf6575bc060ce15f3cb 2025-12-17 22:39:43,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf6575bc060ce15f3cb HTTP/1.1" 502 150 2025-12-17 22:39:43,301 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf6575bc060ce15f3cb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:43,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bfe575bc060ce15f3d7 2025-12-17 22:39:43,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfe575bc060ce15f3d7 HTTP/1.1" 502 150 2025-12-17 22:39:43,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bfe575bc060ce15f3d7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:43,412 - lava_callback - INFO - Received callback for node 69408c36575bc060ce15f4af INFO: 172.21.0.1:58084 - "POST /node/69408c36575bc060ce15f4af HTTP/1.0" 202 Accepted 2025-12-17 22:39:43,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf5575bc060ce15f3c8 2025-12-17 22:39:43,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf5575bc060ce15f3c8 HTTP/1.1" 502 150 2025-12-17 22:39:43,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf5575bc060ce15f3c8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:44,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bee575bc060ce15f3b0 2025-12-17 22:39:44,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bee575bc060ce15f3b0 HTTP/1.1" 502 150 2025-12-17 22:39:44,806 - lava_callback - ERROR - Error processing callback for node 69408bee575bc060ce15f3b0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bee575bc060ce15f3b0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bee575bc060ce15f3b0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bee575bc060ce15f3b0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:44,808 - lava_callback - INFO - Starting async job submit for node 69408c02575bc060ce15f3e7 2025-12-17 22:39:44,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:44,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c02575bc060ce15f3e7 HTTP/1.1" 502 150 2025-12-17 22:39:44,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c02575bc060ce15f3e7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:44,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c02575bc060ce15f3e7 2025-12-17 22:39:44,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c02575bc060ce15f3e7 HTTP/1.1" 502 150 2025-12-17 22:39:44,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c02575bc060ce15f3e7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:44,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bfc575bc060ce15f3d4 2025-12-17 22:39:44,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfc575bc060ce15f3d4 HTTP/1.1" 502 150 2025-12-17 22:39:44,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bfc575bc060ce15f3d4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:44,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bfa575bc060ce15f3d1 2025-12-17 22:39:44,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfa575bc060ce15f3d1 HTTP/1.1" 502 150 2025-12-17 22:39:44,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bfa575bc060ce15f3d1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:45,236 - lava_callback - INFO - Received callback for node 69408e25575bc060ce15fba3 INFO: 172.21.0.1:58096 - "POST /node/69408e25575bc060ce15fba3 HTTP/1.0" 202 Accepted 2025-12-17 22:39:45,301 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf6575bc060ce15f3cb 2025-12-17 22:39:45,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf6575bc060ce15f3cb HTTP/1.1" 502 150 2025-12-17 22:39:45,303 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf6575bc060ce15f3cb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:45,904 - lava_callback - INFO - Received callback for node 69408e27575bc060ce15fba6 INFO: 172.21.0.1:58100 - "POST /node/69408e27575bc060ce15fba6 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:58110 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:39:46,572 - lava_callback - INFO - Received callback for node 69408e2d575bc060ce15fd32 INFO: 172.21.0.1:58124 - "POST /node/69408e2d575bc060ce15fd32 HTTP/1.0" 202 Accepted 2025-12-17 22:39:46,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c02575bc060ce15f3e7 2025-12-17 22:39:46,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c02575bc060ce15f3e7 HTTP/1.1" 502 150 2025-12-17 22:39:46,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c02575bc060ce15f3e7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:46,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092fd575bc060ce160bf7 2025-12-17 22:39:46,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fd575bc060ce160bf7 HTTP/1.1" 502 150 2025-12-17 22:39:46,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092fd575bc060ce160bf7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:46,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bea575bc060ce15f3a1 2025-12-17 22:39:46,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bea575bc060ce15f3a1 HTTP/1.1" 502 150 2025-12-17 22:39:46,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bea575bc060ce15f3a1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:47,221 - lava_callback - INFO - Received callback for node 69408e2f575bc060ce15fd3a INFO: 172.21.0.1:58140 - "POST /node/69408e2f575bc060ce15fd3a HTTP/1.0" 202 Accepted 2025-12-17 22:39:47,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bfe575bc060ce15f3d7 2025-12-17 22:39:47,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfe575bc060ce15f3d7 HTTP/1.1" 502 150 2025-12-17 22:39:47,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bfe575bc060ce15f3d7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:48,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408beb575bc060ce15f3a4 2025-12-17 22:39:48,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408beb575bc060ce15f3a4 HTTP/1.1" 502 150 2025-12-17 22:39:48,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408beb575bc060ce15f3a4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:49,303 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf6575bc060ce15f3cb 2025-12-17 22:39:49,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf6575bc060ce15f3cb HTTP/1.1" 502 150 2025-12-17 22:39:49,306 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf6575bc060ce15f3cb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:49,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408baa575bc060ce15f2d0 HTTP/1.1" 502 150 2025-12-17 22:39:49,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408baa575bc060ce15f2d0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:49,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bed575bc060ce15f3a9 2025-12-17 22:39:49,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bed575bc060ce15f3a9 HTTP/1.1" 502 150 2025-12-17 22:39:49,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bed575bc060ce15f3a9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:50,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf1575bc060ce15f3bf 2025-12-17 22:39:50,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf2575bc060ce15f3c2 2025-12-17 22:39:50,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf2575bc060ce15f3c2 HTTP/1.1" 502 150 2025-12-17 22:39:50,819 - lava_callback - ERROR - Error processing callback for node 69408bf2575bc060ce15f3c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf2575bc060ce15f3c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf2575bc060ce15f3c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf2575bc060ce15f3c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:50,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c02575bc060ce15f3e7 2025-12-17 22:39:50,821 - lava_callback - INFO - Starting async job submit for node 69408c04575bc060ce15f3f0 2025-12-17 22:39:50,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:50,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c02575bc060ce15f3e7 HTTP/1.1" 502 150 2025-12-17 22:39:50,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c02575bc060ce15f3e7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:50,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c04575bc060ce15f3f0 HTTP/1.1" 502 150 2025-12-17 22:39:50,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c04575bc060ce15f3f0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:50,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c04575bc060ce15f3f0 2025-12-17 22:39:50,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c04575bc060ce15f3f0 HTTP/1.1" 502 150 2025-12-17 22:39:50,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c04575bc060ce15f3f0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:50,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf7575bc060ce15f3ce 2025-12-17 22:39:50,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf7575bc060ce15f3ce HTTP/1.1" 502 150 2025-12-17 22:39:50,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf7575bc060ce15f3ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:51,186 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf3575bc060ce15f3c5 2025-12-17 22:39:51,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf3575bc060ce15f3c5 HTTP/1.1" 502 150 2025-12-17 22:39:51,187 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf3575bc060ce15f3c5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:51,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bd6575bc060ce15f366 2025-12-17 22:39:52,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c04575bc060ce15f3f0 2025-12-17 22:39:52,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c04575bc060ce15f3f0 HTTP/1.1" 502 150 2025-12-17 22:39:52,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c04575bc060ce15f3f0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:52,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bfc575bc060ce15f3d4 2025-12-17 22:39:52,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfc575bc060ce15f3d4 HTTP/1.1" 502 150 2025-12-17 22:39:52,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bfc575bc060ce15f3d4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:52,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bfa575bc060ce15f3d1 2025-12-17 22:39:52,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfa575bc060ce15f3d1 HTTP/1.1" 502 150 2025-12-17 22:39:52,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bfa575bc060ce15f3d1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:54,870 - lava_callback - INFO - Received callback for node 69408e95575bc060ce15fe85 INFO: 172.21.0.1:44136 - "POST /node/69408e95575bc060ce15fe85 HTTP/1.0" 202 Accepted 2025-12-17 22:39:55,037 - lava_callback - INFO - Received callback for node 69408e34575bc060ce15fd54 INFO: 172.21.0.1:44152 - "POST /node/69408e34575bc060ce15fd54 HTTP/1.0" 202 Accepted 2025-12-17 22:39:55,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bfe575bc060ce15f3d7 2025-12-17 22:39:55,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfe575bc060ce15f3d7 HTTP/1.1" 502 150 2025-12-17 22:39:55,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bfe575bc060ce15f3d7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:55,685 - lava_callback - INFO - Received callback for node 69408e37575bc060ce15fd5f INFO: 172.21.0.1:44156 - "POST /node/69408e37575bc060ce15fd5f HTTP/1.0" 202 Accepted 2025-12-17 22:39:56,329 - lava_callback - INFO - Received callback for node 69408e38575bc060ce15fd62 INFO: 172.21.0.1:44160 - "POST /node/69408e38575bc060ce15fd62 HTTP/1.0" 202 Accepted 2025-12-17 22:39:56,482 - lava_callback - INFO - Received callback for node 6940924f575bc060ce160a6f INFO: 172.21.0.1:44176 - "POST /node/6940924f575bc060ce160a6f HTTP/1.0" 202 Accepted INFO: 172.21.0.1:44184 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:39:56,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c04575bc060ce15f3f0 2025-12-17 22:39:56,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c04575bc060ce15f3f0 HTTP/1.1" 502 150 2025-12-17 22:39:56,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c04575bc060ce15f3f0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:56,990 - lava_callback - INFO - Received callback for node 69408e39575bc060ce15fd65 INFO: 172.21.0.1:44196 - "POST /node/69408e39575bc060ce15fd65 HTTP/1.0" 202 Accepted 2025-12-17 22:39:57,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf6575bc060ce15f3cb 2025-12-17 22:39:58,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c02575bc060ce15f3e7 2025-12-17 22:39:58,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c02575bc060ce15f3e7 HTTP/1.1" 502 150 2025-12-17 22:39:58,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c02575bc060ce15f3e7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:59,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf5575bc060ce15f3c8 2025-12-17 22:39:59,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf5575bc060ce15f3c8 HTTP/1.1" 502 150 2025-12-17 22:39:59,694 - lava_callback - ERROR - Error processing callback for node 69408bf5575bc060ce15f3c8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf5575bc060ce15f3c8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf5575bc060ce15f3c8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf5575bc060ce15f3c8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:39:59,696 - lava_callback - INFO - Starting async job submit for node 69408c0b575bc060ce15f437 2025-12-17 22:39:59,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:39:59,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0b575bc060ce15f437 HTTP/1.1" 502 150 2025-12-17 22:39:59,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c0b575bc060ce15f437'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:39:59,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c0b575bc060ce15f437 2025-12-17 22:39:59,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0b575bc060ce15f437 HTTP/1.1" 502 150 2025-12-17 22:39:59,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c0b575bc060ce15f437'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:01,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c0b575bc060ce15f437 2025-12-17 22:40:01,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0b575bc060ce15f437 HTTP/1.1" 502 150 2025-12-17 22:40:01,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c0b575bc060ce15f437'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60844 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:40:02,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092fd575bc060ce160bf7 2025-12-17 22:40:02,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fd575bc060ce160bf7 HTTP/1.1" 502 150 2025-12-17 22:40:02,900 - lava_callback - ERROR - Error processing callback for node 694092fd575bc060ce160bf7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092fd575bc060ce160bf7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092fd575bc060ce160bf7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092fd575bc060ce160bf7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:02,902 - lava_callback - INFO - Starting async job submit for node 69408c0d575bc060ce15f43a 2025-12-17 22:40:02,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:02,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0d575bc060ce15f43a HTTP/1.1" 502 150 2025-12-17 22:40:02,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c0d575bc060ce15f43a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:02,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c0d575bc060ce15f43a 2025-12-17 22:40:02,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0d575bc060ce15f43a HTTP/1.1" 502 150 2025-12-17 22:40:02,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c0d575bc060ce15f43a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:02,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bea575bc060ce15f3a1 2025-12-17 22:40:02,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bea575bc060ce15f3a1 HTTP/1.1" 502 150 2025-12-17 22:40:02,986 - lava_callback - ERROR - Error processing callback for node 69408bea575bc060ce15f3a1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bea575bc060ce15f3a1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bea575bc060ce15f3a1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bea575bc060ce15f3a1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:02,987 - lava_callback - INFO - Starting async job submit for node 69408c0f575bc060ce15f43d 2025-12-17 22:40:02,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:02,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0f575bc060ce15f43d HTTP/1.1" 502 150 2025-12-17 22:40:02,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c0f575bc060ce15f43d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:02,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c0f575bc060ce15f43d 2025-12-17 22:40:02,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0f575bc060ce15f43d HTTP/1.1" 502 150 2025-12-17 22:40:02,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c0f575bc060ce15f43d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:03,883 - lava_callback - INFO - Received callback for node 69408e30575bc060ce15fd3d INFO: 172.21.0.1:60850 - "POST /node/69408e30575bc060ce15fd3d HTTP/1.0" 202 Accepted 2025-12-17 22:40:04,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408beb575bc060ce15f3a4 2025-12-17 22:40:04,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408beb575bc060ce15f3a4 HTTP/1.1" 502 150 2025-12-17 22:40:04,658 - lava_callback - ERROR - Error processing callback for node 69408beb575bc060ce15f3a4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408beb575bc060ce15f3a4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408beb575bc060ce15f3a4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408beb575bc060ce15f3a4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:04,660 - lava_callback - INFO - Starting async job submit for node 69408c11575bc060ce15f440 2025-12-17 22:40:04,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:04,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c11575bc060ce15f440 HTTP/1.1" 502 150 2025-12-17 22:40:04,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c11575bc060ce15f440'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:04,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c11575bc060ce15f440 2025-12-17 22:40:04,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c11575bc060ce15f440 HTTP/1.1" 502 150 2025-12-17 22:40:04,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c11575bc060ce15f440'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:04,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c04575bc060ce15f3f0 2025-12-17 22:40:04,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c04575bc060ce15f3f0 HTTP/1.1" 502 150 2025-12-17 22:40:04,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c04575bc060ce15f3f0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:04,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c0d575bc060ce15f43a 2025-12-17 22:40:04,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0d575bc060ce15f43a HTTP/1.1" 502 150 2025-12-17 22:40:04,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c0d575bc060ce15f43a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:04,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c0f575bc060ce15f43d 2025-12-17 22:40:04,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0f575bc060ce15f43d HTTP/1.1" 502 150 2025-12-17 22:40:04,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c0f575bc060ce15f43d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:05,269 - lava_callback - INFO - Received callback for node 69408e32575bc060ce15fd46 INFO: 172.21.0.1:60866 - "POST /node/69408e32575bc060ce15fd46 HTTP/1.0" 202 Accepted 2025-12-17 22:40:05,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408baa575bc060ce15f2d0 2025-12-17 22:40:05,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c0b575bc060ce15f437 2025-12-17 22:40:05,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0b575bc060ce15f437 HTTP/1.1" 502 150 2025-12-17 22:40:05,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c0b575bc060ce15f437'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:05,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf1575bc060ce15f3bf HTTP/1.1" 502 150 2025-12-17 22:40:05,808 - lava_callback - ERROR - Error processing callback for node 69408bf1575bc060ce15f3bf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf1575bc060ce15f3bf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf1575bc060ce15f3bf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf1575bc060ce15f3bf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:05,810 - lava_callback - INFO - Starting async job submit for node 69408c13575bc060ce15f445 2025-12-17 22:40:05,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:05,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c13575bc060ce15f445 HTTP/1.1" 502 150 2025-12-17 22:40:05,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c13575bc060ce15f445'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:05,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c13575bc060ce15f445 2025-12-17 22:40:05,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c13575bc060ce15f445 HTTP/1.1" 502 150 2025-12-17 22:40:05,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c13575bc060ce15f445'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:05,944 - lava_callback - INFO - Received callback for node 69408e33575bc060ce15fd4d INFO: 172.21.0.1:60874 - "POST /node/69408e33575bc060ce15fd4d HTTP/1.0" 202 Accepted 2025-12-17 22:40:05,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bed575bc060ce15f3a9 2025-12-17 22:40:05,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bed575bc060ce15f3a9 HTTP/1.1" 502 150 2025-12-17 22:40:05,985 - lava_callback - ERROR - Error processing callback for node 69408bed575bc060ce15f3a9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bed575bc060ce15f3a9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bed575bc060ce15f3a9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bed575bc060ce15f3a9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:05,987 - lava_callback - INFO - Starting async job submit for node 69408c00575bc060ce15f3dc 2025-12-17 22:40:05,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:06,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c00575bc060ce15f3dc HTTP/1.1" 502 150 2025-12-17 22:40:06,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c00575bc060ce15f3dc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:06,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c00575bc060ce15f3dc 2025-12-17 22:40:06,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c00575bc060ce15f3dc HTTP/1.1" 502 150 2025-12-17 22:40:06,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c00575bc060ce15f3dc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60876 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:40:06,612 - lava_callback - INFO - Received callback for node 69408e3e575bc060ce15fd71 INFO: 172.21.0.1:60892 - "POST /node/69408e3e575bc060ce15fd71 HTTP/1.0" 202 Accepted 2025-12-17 22:40:06,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c11575bc060ce15f440 2025-12-17 22:40:06,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c11575bc060ce15f440 HTTP/1.1" 502 150 2025-12-17 22:40:06,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c11575bc060ce15f440'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:06,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf7575bc060ce15f3ce 2025-12-17 22:40:06,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf7575bc060ce15f3ce HTTP/1.1" 502 150 2025-12-17 22:40:06,947 - lava_callback - ERROR - Error processing callback for node 69408bf7575bc060ce15f3ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf7575bc060ce15f3ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf7575bc060ce15f3ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf7575bc060ce15f3ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:06,949 - lava_callback - INFO - Starting async job submit for node 69408c1c575bc060ce15f465 2025-12-17 22:40:06,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:06,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1c575bc060ce15f465 HTTP/1.1" 502 150 2025-12-17 22:40:06,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1c575bc060ce15f465'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:06,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1c575bc060ce15f465 2025-12-17 22:40:06,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1c575bc060ce15f465 HTTP/1.1" 502 150 2025-12-17 22:40:06,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1c575bc060ce15f465'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:07,188 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf3575bc060ce15f3c5 2025-12-17 22:40:07,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf3575bc060ce15f3c5 HTTP/1.1" 502 150 2025-12-17 22:40:07,189 - lava_callback - ERROR - Error processing callback for node 69408bf3575bc060ce15f3c5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf3575bc060ce15f3c5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf3575bc060ce15f3c5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf3575bc060ce15f3c5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:07,191 - lava_callback - INFO - Starting async job submit for node 69408c18575bc060ce15f45f 2025-12-17 22:40:07,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:07,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c18575bc060ce15f45f HTTP/1.1" 502 150 2025-12-17 22:40:07,224 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c18575bc060ce15f45f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:07,224 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c18575bc060ce15f45f 2025-12-17 22:40:07,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c18575bc060ce15f45f HTTP/1.1" 502 150 2025-12-17 22:40:07,225 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c18575bc060ce15f45f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:07,254 - lava_callback - INFO - Received callback for node 69408e40575bc060ce15fd74 INFO: 172.21.0.1:60896 - "POST /node/69408e40575bc060ce15fd74 HTTP/1.0" 202 Accepted 2025-12-17 22:40:07,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c13575bc060ce15f445 2025-12-17 22:40:07,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c13575bc060ce15f445 HTTP/1.1" 502 150 2025-12-17 22:40:07,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c13575bc060ce15f445'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:07,916 - lava_callback - INFO - Received callback for node 69408e41575bc060ce15fd77 INFO: 172.21.0.1:60904 - "POST /node/69408e41575bc060ce15fd77 HTTP/1.0" 202 Accepted 2025-12-17 22:40:08,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c00575bc060ce15f3dc 2025-12-17 22:40:08,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c00575bc060ce15f3dc HTTP/1.1" 502 150 2025-12-17 22:40:08,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c00575bc060ce15f3dc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:08,569 - lava_callback - INFO - Received callback for node 69408e42575bc060ce15fd7a INFO: 172.21.0.1:60910 - "POST /node/69408e42575bc060ce15fd7a HTTP/1.0" 202 Accepted 2025-12-17 22:40:08,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bfc575bc060ce15f3d4 2025-12-17 22:40:08,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfc575bc060ce15f3d4 HTTP/1.1" 502 150 2025-12-17 22:40:08,902 - lava_callback - ERROR - Error processing callback for node 69408bfc575bc060ce15f3d4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bfc575bc060ce15f3d4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bfc575bc060ce15f3d4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bfc575bc060ce15f3d4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:08,904 - lava_callback - INFO - Starting async job submit for node 69408c1a575bc060ce15f462 2025-12-17 22:40:08,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:08,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bfa575bc060ce15f3d1 2025-12-17 22:40:08,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfa575bc060ce15f3d1 HTTP/1.1" 502 150 2025-12-17 22:40:08,906 - lava_callback - ERROR - Error processing callback for node 69408bfa575bc060ce15f3d1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bfa575bc060ce15f3d1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bfa575bc060ce15f3d1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bfa575bc060ce15f3d1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:08,908 - lava_callback - INFO - Starting async job submit for node 69408c29575bc060ce15f488 2025-12-17 22:40:08,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:08,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c0d575bc060ce15f43a 2025-12-17 22:40:08,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0d575bc060ce15f43a HTTP/1.1" 502 150 2025-12-17 22:40:08,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c0d575bc060ce15f43a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:08,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c29575bc060ce15f488 HTTP/1.1" 502 150 2025-12-17 22:40:08,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c29575bc060ce15f488'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:08,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c29575bc060ce15f488 2025-12-17 22:40:08,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c29575bc060ce15f488 HTTP/1.1" 502 150 2025-12-17 22:40:08,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c29575bc060ce15f488'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:08,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1a575bc060ce15f462 HTTP/1.1" 502 150 2025-12-17 22:40:08,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1a575bc060ce15f462'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:08,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1a575bc060ce15f462 2025-12-17 22:40:08,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1a575bc060ce15f462 HTTP/1.1" 502 150 2025-12-17 22:40:08,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1a575bc060ce15f462'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:08,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1c575bc060ce15f465 2025-12-17 22:40:08,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1c575bc060ce15f465 HTTP/1.1" 502 150 2025-12-17 22:40:08,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1c575bc060ce15f465'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:08,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c0f575bc060ce15f43d 2025-12-17 22:40:08,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0f575bc060ce15f43d HTTP/1.1" 502 150 2025-12-17 22:40:08,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c0f575bc060ce15f43d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:09,217 - lava_callback - INFO - Received callback for node 69408e43575bc060ce15fd7d INFO: 172.21.0.1:60912 - "POST /node/69408e43575bc060ce15fd7d HTTP/1.0" 202 Accepted 2025-12-17 22:40:09,225 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c18575bc060ce15f45f 2025-12-17 22:40:09,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c18575bc060ce15f45f HTTP/1.1" 502 150 2025-12-17 22:40:09,226 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c18575bc060ce15f45f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:10,349 - lava_callback - INFO - Received callback for node 69408e3c575bc060ce15fd6b INFO: 172.21.0.1:41026 - "POST /node/69408e3c575bc060ce15fd6b HTTP/1.0" 202 Accepted 2025-12-17 22:40:10,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c11575bc060ce15f440 2025-12-17 22:40:10,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c29575bc060ce15f488 2025-12-17 22:40:10,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c29575bc060ce15f488 HTTP/1.1" 502 150 2025-12-17 22:40:10,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c29575bc060ce15f488'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:10,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1a575bc060ce15f462 2025-12-17 22:40:10,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1a575bc060ce15f462 HTTP/1.1" 502 150 2025-12-17 22:40:10,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1a575bc060ce15f462'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:11,076 - lava_callback - INFO - Received callback for node 69408e3d575bc060ce15fd6e INFO: 172.21.0.1:41040 - "POST /node/69408e3d575bc060ce15fd6e HTTP/1.0" 202 Accepted 2025-12-17 22:40:11,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bfe575bc060ce15f3d7 2025-12-17 22:40:11,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bfe575bc060ce15f3d7 HTTP/1.1" 502 150 2025-12-17 22:40:11,414 - lava_callback - ERROR - Error processing callback for node 69408bfe575bc060ce15f3d7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bfe575bc060ce15f3d7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bfe575bc060ce15f3d7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bfe575bc060ce15f3d7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:11,416 - lava_callback - INFO - Starting async job submit for node 69408c1e575bc060ce15f468 2025-12-17 22:40:11,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:11,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1e575bc060ce15f468 HTTP/1.1" 502 150 2025-12-17 22:40:11,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1e575bc060ce15f468'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:11,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1e575bc060ce15f468 2025-12-17 22:40:11,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1e575bc060ce15f468 HTTP/1.1" 502 150 2025-12-17 22:40:11,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1e575bc060ce15f468'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:11,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c13575bc060ce15f445 2025-12-17 22:40:11,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c13575bc060ce15f445 HTTP/1.1" 502 150 2025-12-17 22:40:11,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c13575bc060ce15f445'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:12,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c00575bc060ce15f3dc 2025-12-17 22:40:12,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c00575bc060ce15f3dc HTTP/1.1" 502 150 2025-12-17 22:40:12,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c00575bc060ce15f3dc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:12,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf6575bc060ce15f3cb HTTP/1.1" 502 150 2025-12-17 22:40:12,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408bf6575bc060ce15f3cb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:12,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1c575bc060ce15f465 2025-12-17 22:40:12,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1c575bc060ce15f465 HTTP/1.1" 502 150 2025-12-17 22:40:12,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1c575bc060ce15f465'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:13,226 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c18575bc060ce15f45f 2025-12-17 22:40:13,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c18575bc060ce15f45f HTTP/1.1" 502 150 2025-12-17 22:40:13,227 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c18575bc060ce15f45f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:13,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1e575bc060ce15f468 2025-12-17 22:40:13,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1e575bc060ce15f468 HTTP/1.1" 502 150 2025-12-17 22:40:13,425 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1e575bc060ce15f468'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:13,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c0b575bc060ce15f437 2025-12-17 22:40:13,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0b575bc060ce15f437 HTTP/1.1" 502 150 2025-12-17 22:40:13,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c0b575bc060ce15f437'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:14,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c02575bc060ce15f3e7 2025-12-17 22:40:14,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c02575bc060ce15f3e7 HTTP/1.1" 502 150 2025-12-17 22:40:14,826 - lava_callback - ERROR - Error processing callback for node 69408c02575bc060ce15f3e7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c02575bc060ce15f3e7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c02575bc060ce15f3e7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c02575bc060ce15f3e7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:14,828 - lava_callback - INFO - Starting async job submit for node 69408c1f575bc060ce15f46b 2025-12-17 22:40:14,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:14,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1f575bc060ce15f46b HTTP/1.1" 502 150 2025-12-17 22:40:14,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1f575bc060ce15f46b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:14,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1f575bc060ce15f46b 2025-12-17 22:40:14,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1f575bc060ce15f46b HTTP/1.1" 502 150 2025-12-17 22:40:14,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1f575bc060ce15f46b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:14,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c29575bc060ce15f488 2025-12-17 22:40:14,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c29575bc060ce15f488 HTTP/1.1" 502 150 2025-12-17 22:40:14,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c29575bc060ce15f488'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:14,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1a575bc060ce15f462 2025-12-17 22:40:14,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1a575bc060ce15f462 HTTP/1.1" 502 150 2025-12-17 22:40:14,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1a575bc060ce15f462'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:15,028 - lava_callback - INFO - Received callback for node 69408e3b575bc060ce15fd68 INFO: 172.21.0.1:41054 - "POST /node/69408e3b575bc060ce15fd68 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:41070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41078 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:40:16,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1f575bc060ce15f46b 2025-12-17 22:40:16,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1f575bc060ce15f46b HTTP/1.1" 502 150 2025-12-17 22:40:16,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1f575bc060ce15f46b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:16,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c0d575bc060ce15f43a 2025-12-17 22:40:16,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0d575bc060ce15f43a HTTP/1.1" 502 150 2025-12-17 22:40:16,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c0d575bc060ce15f43a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:16,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c0f575bc060ce15f43d 2025-12-17 22:40:16,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0f575bc060ce15f43d HTTP/1.1" 502 150 2025-12-17 22:40:16,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c0f575bc060ce15f43d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:17,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1e575bc060ce15f468 2025-12-17 22:40:17,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1e575bc060ce15f468 HTTP/1.1" 502 150 2025-12-17 22:40:17,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1e575bc060ce15f468'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:19,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c13575bc060ce15f445 2025-12-17 22:40:19,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c13575bc060ce15f445 HTTP/1.1" 502 150 2025-12-17 22:40:19,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c13575bc060ce15f445'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:20,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c00575bc060ce15f3dc 2025-12-17 22:40:20,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c00575bc060ce15f3dc HTTP/1.1" 502 150 2025-12-17 22:40:20,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c00575bc060ce15f3dc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:20,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c04575bc060ce15f3f0 2025-12-17 22:40:20,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c04575bc060ce15f3f0 HTTP/1.1" 502 150 2025-12-17 22:40:20,835 - lava_callback - ERROR - Error processing callback for node 69408c04575bc060ce15f3f0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c04575bc060ce15f3f0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c04575bc060ce15f3f0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c04575bc060ce15f3f0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:20,837 - lava_callback - INFO - Starting async job submit for node 69408c20575bc060ce15f46e 2025-12-17 22:40:20,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1f575bc060ce15f46b 2025-12-17 22:40:20,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:20,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1f575bc060ce15f46b HTTP/1.1" 502 150 2025-12-17 22:40:20,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1f575bc060ce15f46b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:20,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c20575bc060ce15f46e HTTP/1.1" 502 150 2025-12-17 22:40:20,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c20575bc060ce15f46e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:20,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c20575bc060ce15f46e 2025-12-17 22:40:20,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c20575bc060ce15f46e HTTP/1.1" 502 150 2025-12-17 22:40:20,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c20575bc060ce15f46e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:20,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1c575bc060ce15f465 2025-12-17 22:40:20,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1c575bc060ce15f465 HTTP/1.1" 502 150 2025-12-17 22:40:20,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1c575bc060ce15f465'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:21,228 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c18575bc060ce15f45f 2025-12-17 22:40:21,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c18575bc060ce15f45f HTTP/1.1" 502 150 2025-12-17 22:40:21,229 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c18575bc060ce15f45f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:22,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bd6575bc060ce15f366 HTTP/1.1" 502 150 2025-12-17 22:40:22,448 - lava_callback - ERROR - Error processing callback for node 69408bd6575bc060ce15f366: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd6575bc060ce15f366 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd6575bc060ce15f366 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bd6575bc060ce15f366 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:22,450 - lava_callback - INFO - Starting async job submit for node 69408c21575bc060ce15f471 2025-12-17 22:40:22,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:22,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c21575bc060ce15f471 HTTP/1.1" 502 150 2025-12-17 22:40:22,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c21575bc060ce15f471'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:22,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c21575bc060ce15f471 2025-12-17 22:40:22,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c21575bc060ce15f471 HTTP/1.1" 502 150 2025-12-17 22:40:22,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c21575bc060ce15f471'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:22,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c20575bc060ce15f46e 2025-12-17 22:40:22,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c20575bc060ce15f46e HTTP/1.1" 502 150 2025-12-17 22:40:22,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c20575bc060ce15f46e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:22,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c29575bc060ce15f488 2025-12-17 22:40:22,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c29575bc060ce15f488 HTTP/1.1" 502 150 2025-12-17 22:40:22,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c29575bc060ce15f488'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:22,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1a575bc060ce15f462 2025-12-17 22:40:22,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1a575bc060ce15f462 HTTP/1.1" 502 150 2025-12-17 22:40:22,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1a575bc060ce15f462'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:23,260 - lava_callback - INFO - Received callback for node 69408e4f575bc060ce15fdac INFO: 172.21.0.1:56058 - "POST /node/69408e4f575bc060ce15fdac HTTP/1.0" 202 Accepted 2025-12-17 22:40:23,995 - lava_callback - INFO - Received callback for node 69408e50575bc060ce15fdaf INFO: 172.21.0.1:56068 - "POST /node/69408e50575bc060ce15fdaf HTTP/1.0" 202 Accepted 2025-12-17 22:40:24,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c21575bc060ce15f471 2025-12-17 22:40:24,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c21575bc060ce15f471 HTTP/1.1" 502 150 2025-12-17 22:40:24,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c21575bc060ce15f471'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:24,764 - lava_callback - INFO - Received callback for node 69408e51575bc060ce15fdb2 INFO: 172.21.0.1:56070 - "POST /node/69408e51575bc060ce15fdb2 HTTP/1.0" 202 Accepted 2025-12-17 22:40:25,279 - lava_callback - INFO - Received callback for node 69408e45575bc060ce15fd88 INFO: 172.21.0.1:56074 - "POST /node/69408e45575bc060ce15fd88 HTTP/1.0" 202 Accepted 2025-12-17 22:40:25,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1e575bc060ce15f468 2025-12-17 22:40:25,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1e575bc060ce15f468 HTTP/1.1" 502 150 2025-12-17 22:40:25,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1e575bc060ce15f468'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:25,931 - lava_callback - INFO - Received callback for node 69408e46575bc060ce15fd95 INFO: 172.21.0.1:56090 - "POST /node/69408e46575bc060ce15fd95 HTTP/1.0" 202 Accepted 2025-12-17 22:40:26,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c11575bc060ce15f440 HTTP/1.1" 502 150 2025-12-17 22:40:26,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c11575bc060ce15f440'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56092 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:40:26,583 - lava_callback - INFO - Received callback for node 69408e48575bc060ce15fd9a INFO: 172.21.0.1:56108 - "POST /node/69408e48575bc060ce15fd9a HTTP/1.0" 202 Accepted 2025-12-17 22:40:26,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c20575bc060ce15f46e 2025-12-17 22:40:26,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c20575bc060ce15f46e HTTP/1.1" 502 150 2025-12-17 22:40:26,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c20575bc060ce15f46e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:27,211 - lava_callback - INFO - Received callback for node 69408e49575bc060ce15fd9d INFO: 172.21.0.1:56114 - "POST /node/69408e49575bc060ce15fd9d HTTP/1.0" 202 Accepted 2025-12-17 22:40:27,860 - lava_callback - INFO - Received callback for node 69408e4a575bc060ce15fda0 INFO: 172.21.0.1:56124 - "POST /node/69408e4a575bc060ce15fda0 HTTP/1.0" 202 Accepted 2025-12-17 22:40:28,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c21575bc060ce15f471 2025-12-17 22:40:28,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c21575bc060ce15f471 HTTP/1.1" 502 150 2025-12-17 22:40:28,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c21575bc060ce15f471'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:28,510 - lava_callback - INFO - Received callback for node 69408e4b575bc060ce15fda3 INFO: 172.21.0.1:56130 - "POST /node/69408e4b575bc060ce15fda3 HTTP/1.0" 202 Accepted 2025-12-17 22:40:28,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408bf6575bc060ce15f3cb 2025-12-17 22:40:28,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408bf6575bc060ce15f3cb HTTP/1.1" 502 150 2025-12-17 22:40:28,722 - lava_callback - ERROR - Error processing callback for node 69408bf6575bc060ce15f3cb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf6575bc060ce15f3cb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf6575bc060ce15f3cb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408bf6575bc060ce15f3cb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:28,725 - lava_callback - INFO - Starting async job submit for node 69408c23575bc060ce15f474 2025-12-17 22:40:28,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:28,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c23575bc060ce15f474 HTTP/1.1" 502 150 2025-12-17 22:40:28,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c23575bc060ce15f474'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:28,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c23575bc060ce15f474 2025-12-17 22:40:28,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c23575bc060ce15f474 HTTP/1.1" 502 150 2025-12-17 22:40:28,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c23575bc060ce15f474'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:28,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1f575bc060ce15f46b 2025-12-17 22:40:28,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1f575bc060ce15f46b HTTP/1.1" 502 150 2025-12-17 22:40:28,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c1f575bc060ce15f46b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:29,158 - lava_callback - INFO - Received callback for node 69408e4d575bc060ce15fda6 INFO: 172.21.0.1:56140 - "POST /node/69408e4d575bc060ce15fda6 HTTP/1.0" 202 Accepted 2025-12-17 22:40:29,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c0b575bc060ce15f437 2025-12-17 22:40:29,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0b575bc060ce15f437 HTTP/1.1" 502 150 2025-12-17 22:40:29,726 - lava_callback - ERROR - Error processing callback for node 69408c0b575bc060ce15f437: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c0b575bc060ce15f437 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c0b575bc060ce15f437 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c0b575bc060ce15f437 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:29,727 - lava_callback - INFO - Starting async job submit for node 69408c2a575bc060ce15f48f 2025-12-17 22:40:29,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:29,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2a575bc060ce15f48f HTTP/1.1" 502 150 2025-12-17 22:40:29,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2a575bc060ce15f48f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:29,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2a575bc060ce15f48f 2025-12-17 22:40:29,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2a575bc060ce15f48f HTTP/1.1" 502 150 2025-12-17 22:40:29,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2a575bc060ce15f48f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:30,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c23575bc060ce15f474 2025-12-17 22:40:30,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c23575bc060ce15f474 HTTP/1.1" 502 150 2025-12-17 22:40:30,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c23575bc060ce15f474'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:31,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2a575bc060ce15f48f 2025-12-17 22:40:31,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2a575bc060ce15f48f HTTP/1.1" 502 150 2025-12-17 22:40:31,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2a575bc060ce15f48f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:32,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c0d575bc060ce15f43a 2025-12-17 22:40:32,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0d575bc060ce15f43a HTTP/1.1" 502 150 2025-12-17 22:40:32,915 - lava_callback - ERROR - Error processing callback for node 69408c0d575bc060ce15f43a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c0d575bc060ce15f43a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c0d575bc060ce15f43a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c0d575bc060ce15f43a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:32,916 - lava_callback - INFO - Starting async job submit for node 69408c2b575bc060ce15f496 2025-12-17 22:40:32,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:32,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2b575bc060ce15f496 HTTP/1.1" 502 150 2025-12-17 22:40:32,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2b575bc060ce15f496'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:32,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2b575bc060ce15f496 2025-12-17 22:40:32,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2b575bc060ce15f496 HTTP/1.1" 502 150 2025-12-17 22:40:32,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2b575bc060ce15f496'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:32,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c0f575bc060ce15f43d 2025-12-17 22:40:33,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c0f575bc060ce15f43d HTTP/1.1" 502 150 2025-12-17 22:40:33,000 - lava_callback - ERROR - Error processing callback for node 69408c0f575bc060ce15f43d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c0f575bc060ce15f43d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c0f575bc060ce15f43d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c0f575bc060ce15f43d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:33,002 - lava_callback - INFO - Starting async job submit for node 69408c2d575bc060ce15f49d 2025-12-17 22:40:33,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:33,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2d575bc060ce15f49d HTTP/1.1" 502 150 2025-12-17 22:40:33,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2d575bc060ce15f49d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:33,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2d575bc060ce15f49d 2025-12-17 22:40:33,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2d575bc060ce15f49d HTTP/1.1" 502 150 2025-12-17 22:40:33,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2d575bc060ce15f49d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:34,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c11575bc060ce15f440 2025-12-17 22:40:34,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c11575bc060ce15f440 HTTP/1.1" 502 150 2025-12-17 22:40:34,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c11575bc060ce15f440'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:34,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c23575bc060ce15f474 2025-12-17 22:40:34,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c23575bc060ce15f474 HTTP/1.1" 502 150 2025-12-17 22:40:34,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c23575bc060ce15f474'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:34,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c20575bc060ce15f46e 2025-12-17 22:40:34,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c20575bc060ce15f46e HTTP/1.1" 502 150 2025-12-17 22:40:34,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c20575bc060ce15f46e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:34,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2b575bc060ce15f496 2025-12-17 22:40:34,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2b575bc060ce15f496 HTTP/1.1" 502 150 2025-12-17 22:40:34,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2b575bc060ce15f496'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:35,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2d575bc060ce15f49d 2025-12-17 22:40:35,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2d575bc060ce15f49d HTTP/1.1" 502 150 2025-12-17 22:40:35,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2d575bc060ce15f49d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:35,057 - lava_callback - INFO - Received callback for node 69408e57575bc060ce15fdbe INFO: 172.21.0.1:50306 - "POST /node/69408e57575bc060ce15fdbe HTTP/1.0" 202 Accepted 2025-12-17 22:40:35,703 - lava_callback - INFO - Received callback for node 69408e53575bc060ce15fdb5 INFO: 172.21.0.1:50318 - "POST /node/69408e53575bc060ce15fdb5 HTTP/1.0" 202 Accepted 2025-12-17 22:40:35,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2a575bc060ce15f48f 2025-12-17 22:40:35,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2a575bc060ce15f48f HTTP/1.1" 502 150 2025-12-17 22:40:35,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2a575bc060ce15f48f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:35,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c13575bc060ce15f445 2025-12-17 22:40:35,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c13575bc060ce15f445 HTTP/1.1" 502 150 2025-12-17 22:40:35,828 - lava_callback - ERROR - Error processing callback for node 69408c13575bc060ce15f445: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c13575bc060ce15f445 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c13575bc060ce15f445 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c13575bc060ce15f445 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:35,830 - lava_callback - INFO - Starting async job submit for node 69408c37575bc060ce15f4b2 2025-12-17 22:40:35,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:35,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c37575bc060ce15f4b2 HTTP/1.1" 502 150 2025-12-17 22:40:35,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c37575bc060ce15f4b2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:35,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c37575bc060ce15f4b2 2025-12-17 22:40:35,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c37575bc060ce15f4b2 HTTP/1.1" 502 150 2025-12-17 22:40:35,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c37575bc060ce15f4b2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:36,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c00575bc060ce15f3dc 2025-12-17 22:40:36,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c00575bc060ce15f3dc HTTP/1.1" 502 150 2025-12-17 22:40:36,025 - lava_callback - ERROR - Error processing callback for node 69408c00575bc060ce15f3dc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c00575bc060ce15f3dc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c00575bc060ce15f3dc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c00575bc060ce15f3dc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:36,026 - lava_callback - INFO - Starting async job submit for node 69408c38575bc060ce15f4b5 2025-12-17 22:40:36,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:36,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c38575bc060ce15f4b5 HTTP/1.1" 502 150 2025-12-17 22:40:36,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c38575bc060ce15f4b5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:36,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c38575bc060ce15f4b5 2025-12-17 22:40:36,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c38575bc060ce15f4b5 HTTP/1.1" 502 150 2025-12-17 22:40:36,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c38575bc060ce15f4b5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:36,353 - lava_callback - INFO - Received callback for node 69408e55575bc060ce15fdb8 INFO: 172.21.0.1:50324 - "POST /node/69408e55575bc060ce15fdb8 HTTP/1.0" 202 Accepted 2025-12-17 22:40:36,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c21575bc060ce15f471 2025-12-17 22:40:36,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c21575bc060ce15f471 HTTP/1.1" 502 150 2025-12-17 22:40:36,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c21575bc060ce15f471'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50334 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:40:36,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408baa575bc060ce15f2d0 HTTP/1.1" 502 150 2025-12-17 22:40:36,528 - lava_callback - ERROR - Error processing callback for node 69408baa575bc060ce15f2d0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408baa575bc060ce15f2d0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408baa575bc060ce15f2d0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408baa575bc060ce15f2d0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:36,530 - lava_callback - INFO - Starting async job submit for node 69408c3a575bc060ce15f4b8 2025-12-17 22:40:36,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:36,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3a575bc060ce15f4b8 HTTP/1.1" 502 150 2025-12-17 22:40:36,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3a575bc060ce15f4b8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:36,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3a575bc060ce15f4b8 2025-12-17 22:40:36,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3a575bc060ce15f4b8 HTTP/1.1" 502 150 2025-12-17 22:40:36,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3a575bc060ce15f4b8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:36,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1c575bc060ce15f465 2025-12-17 22:40:36,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1c575bc060ce15f465 HTTP/1.1" 502 150 2025-12-17 22:40:36,966 - lava_callback - ERROR - Error processing callback for node 69408c1c575bc060ce15f465: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c1c575bc060ce15f465 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c1c575bc060ce15f465 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c1c575bc060ce15f465 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:36,968 - lava_callback - INFO - Starting async job submit for node 69408c3b575bc060ce15f4bb 2025-12-17 22:40:36,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:36,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3b575bc060ce15f4bb HTTP/1.1" 502 150 2025-12-17 22:40:36,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3b575bc060ce15f4bb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:36,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3b575bc060ce15f4bb 2025-12-17 22:40:36,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3b575bc060ce15f4bb HTTP/1.1" 502 150 2025-12-17 22:40:36,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3b575bc060ce15f4bb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:36,993 - lava_callback - INFO - Received callback for node 69408e56575bc060ce15fdbb INFO: 172.21.0.1:50350 - "POST /node/69408e56575bc060ce15fdbb HTTP/1.0" 202 Accepted 2025-12-17 22:40:37,229 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c18575bc060ce15f45f 2025-12-17 22:40:37,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c37575bc060ce15f4b2 2025-12-17 22:40:37,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c37575bc060ce15f4b2 HTTP/1.1" 502 150 2025-12-17 22:40:37,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c37575bc060ce15f4b2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:38,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c38575bc060ce15f4b5 2025-12-17 22:40:38,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c38575bc060ce15f4b5 HTTP/1.1" 502 150 2025-12-17 22:40:38,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c38575bc060ce15f4b5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:38,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3a575bc060ce15f4b8 2025-12-17 22:40:38,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3a575bc060ce15f4b8 HTTP/1.1" 502 150 2025-12-17 22:40:38,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3a575bc060ce15f4b8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:38,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c29575bc060ce15f488 2025-12-17 22:40:38,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c29575bc060ce15f488 HTTP/1.1" 502 150 2025-12-17 22:40:38,921 - lava_callback - ERROR - Error processing callback for node 69408c29575bc060ce15f488: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c29575bc060ce15f488 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c29575bc060ce15f488 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c29575bc060ce15f488 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:38,922 - lava_callback - INFO - Starting async job submit for node 69408c3c575bc060ce15f4c4 2025-12-17 22:40:38,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:38,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2b575bc060ce15f496 2025-12-17 22:40:38,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2b575bc060ce15f496 HTTP/1.1" 502 150 2025-12-17 22:40:38,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2b575bc060ce15f496'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:38,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1a575bc060ce15f462 2025-12-17 22:40:38,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1a575bc060ce15f462 HTTP/1.1" 502 150 2025-12-17 22:40:38,945 - lava_callback - ERROR - Error processing callback for node 69408c1a575bc060ce15f462: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c1a575bc060ce15f462 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c1a575bc060ce15f462 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c1a575bc060ce15f462 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:38,947 - lava_callback - INFO - Starting async job submit for node 69408c3e575bc060ce15f4d0 2025-12-17 22:40:38,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:38,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3c575bc060ce15f4c4 HTTP/1.1" 502 150 2025-12-17 22:40:38,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3c575bc060ce15f4c4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:38,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3c575bc060ce15f4c4 2025-12-17 22:40:38,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3c575bc060ce15f4c4 HTTP/1.1" 502 150 2025-12-17 22:40:38,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3c575bc060ce15f4c4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:38,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3e575bc060ce15f4d0 HTTP/1.1" 502 150 2025-12-17 22:40:38,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3e575bc060ce15f4d0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:38,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3e575bc060ce15f4d0 2025-12-17 22:40:38,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3e575bc060ce15f4d0 HTTP/1.1" 502 150 2025-12-17 22:40:38,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3e575bc060ce15f4d0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:38,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3b575bc060ce15f4bb 2025-12-17 22:40:38,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3b575bc060ce15f4bb HTTP/1.1" 502 150 2025-12-17 22:40:38,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3b575bc060ce15f4bb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:39,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2d575bc060ce15f49d 2025-12-17 22:40:39,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2d575bc060ce15f49d HTTP/1.1" 502 150 2025-12-17 22:40:39,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2d575bc060ce15f49d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:40,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3c575bc060ce15f4c4 2025-12-17 22:40:40,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3c575bc060ce15f4c4 HTTP/1.1" 502 150 2025-12-17 22:40:40,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3c575bc060ce15f4c4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:40,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3e575bc060ce15f4d0 2025-12-17 22:40:40,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3e575bc060ce15f4d0 HTTP/1.1" 502 150 2025-12-17 22:40:40,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3e575bc060ce15f4d0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:41,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1e575bc060ce15f468 2025-12-17 22:40:41,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1e575bc060ce15f468 HTTP/1.1" 502 150 2025-12-17 22:40:41,429 - lava_callback - ERROR - Error processing callback for node 69408c1e575bc060ce15f468: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c1e575bc060ce15f468 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c1e575bc060ce15f468 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c1e575bc060ce15f468 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:41,431 - lava_callback - INFO - Starting async job submit for node 69408c3f575bc060ce15f4da 2025-12-17 22:40:41,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:41,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3f575bc060ce15f4da HTTP/1.1" 502 150 2025-12-17 22:40:41,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3f575bc060ce15f4da'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:41,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3f575bc060ce15f4da 2025-12-17 22:40:41,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3f575bc060ce15f4da HTTP/1.1" 502 150 2025-12-17 22:40:41,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3f575bc060ce15f4da'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:41,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c37575bc060ce15f4b2 2025-12-17 22:40:41,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c37575bc060ce15f4b2 HTTP/1.1" 502 150 2025-12-17 22:40:41,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c37575bc060ce15f4b2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:42,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c38575bc060ce15f4b5 2025-12-17 22:40:42,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3a575bc060ce15f4b8 2025-12-17 22:40:42,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3a575bc060ce15f4b8 HTTP/1.1" 502 150 2025-12-17 22:40:42,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3a575bc060ce15f4b8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:42,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c23575bc060ce15f474 2025-12-17 22:40:42,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c23575bc060ce15f474 HTTP/1.1" 502 150 2025-12-17 22:40:42,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c23575bc060ce15f474'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:42,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3b575bc060ce15f4bb 2025-12-17 22:40:42,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3b575bc060ce15f4bb HTTP/1.1" 502 150 2025-12-17 22:40:42,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3b575bc060ce15f4bb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:43,109 - lava_callback - INFO - Received callback for node 69408e59575bc060ce15fdc6 INFO: 172.21.0.1:37594 - "POST /node/69408e59575bc060ce15fdc6 HTTP/1.0" 202 Accepted 2025-12-17 22:40:43,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3f575bc060ce15f4da 2025-12-17 22:40:43,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3f575bc060ce15f4da HTTP/1.1" 502 150 2025-12-17 22:40:43,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3f575bc060ce15f4da'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:43,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2a575bc060ce15f48f 2025-12-17 22:40:43,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2a575bc060ce15f48f HTTP/1.1" 502 150 2025-12-17 22:40:43,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2a575bc060ce15f48f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:43,934 - lava_callback - INFO - Received callback for node 69408e5e575bc060ce15fdd2 INFO: 172.21.0.1:37596 - "POST /node/69408e5e575bc060ce15fdd2 HTTP/1.0" 202 Accepted 2025-12-17 22:40:44,732 - lava_callback - INFO - Received callback for node 69408e69575bc060ce15fe03 INFO: 172.21.0.1:37610 - "POST /node/69408e69575bc060ce15fe03 HTTP/1.0" 202 Accepted 2025-12-17 22:40:44,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c1f575bc060ce15f46b 2025-12-17 22:40:44,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c1f575bc060ce15f46b HTTP/1.1" 502 150 2025-12-17 22:40:44,842 - lava_callback - ERROR - Error processing callback for node 69408c1f575bc060ce15f46b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c1f575bc060ce15f46b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c1f575bc060ce15f46b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c1f575bc060ce15f46b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:44,843 - lava_callback - INFO - Starting async job submit for node 694080d6575bc060ce15c906 2025-12-17 22:40:44,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3c575bc060ce15f4c4 2025-12-17 22:40:44,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3e575bc060ce15f4d0 2025-12-17 22:40:44,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3c575bc060ce15f4c4 HTTP/1.1" 502 150 2025-12-17 22:40:44,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3c575bc060ce15f4c4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:44,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3e575bc060ce15f4d0 HTTP/1.1" 502 150 2025-12-17 22:40:44,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3e575bc060ce15f4d0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:45,317 - lava_callback - INFO - Received callback for node 69408e5a575bc060ce15fdc9 INFO: 172.21.0.1:37620 - "POST /node/69408e5a575bc060ce15fdc9 HTTP/1.0" 202 Accepted 2025-12-17 22:40:46,057 - lava_callback - INFO - Received callback for node 69408e5b575bc060ce15fdcc INFO: 172.21.0.1:37632 - "POST /node/69408e5b575bc060ce15fdcc HTTP/1.0" 202 Accepted 2025-12-17 22:40:46,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:46,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080d6575bc060ce15c906 HTTP/1.1" 502 150 2025-12-17 22:40:46,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080d6575bc060ce15c906'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:46,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080d6575bc060ce15c906 2025-12-17 22:40:46,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080d6575bc060ce15c906 HTTP/1.1" 502 150 2025-12-17 22:40:46,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080d6575bc060ce15c906'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37634 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:40:46,717 - lava_callback - INFO - Received callback for node 69408e5d575bc060ce15fdcf INFO: 172.21.0.1:37638 - "POST /node/69408e5d575bc060ce15fdcf HTTP/1.0" 202 Accepted 2025-12-17 22:40:46,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2b575bc060ce15f496 2025-12-17 22:40:46,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2b575bc060ce15f496 HTTP/1.1" 502 150 2025-12-17 22:40:46,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2b575bc060ce15f496'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:47,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2d575bc060ce15f49d 2025-12-17 22:40:47,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2d575bc060ce15f49d HTTP/1.1" 502 150 2025-12-17 22:40:47,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2d575bc060ce15f49d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:47,351 - lava_callback - INFO - Received callback for node 69408e5f575bc060ce15fdd5 INFO: 172.21.0.1:37650 - "POST /node/69408e5f575bc060ce15fdd5 HTTP/1.0" 202 Accepted 2025-12-17 22:40:47,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3f575bc060ce15f4da 2025-12-17 22:40:47,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3f575bc060ce15f4da HTTP/1.1" 502 150 2025-12-17 22:40:47,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3f575bc060ce15f4da'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:48,010 - lava_callback - INFO - Received callback for node 69408e60575bc060ce15fdd8 INFO: 172.21.0.1:37662 - "POST /node/69408e60575bc060ce15fdd8 HTTP/1.0" 202 Accepted 2025-12-17 22:40:48,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080d6575bc060ce15c906 2025-12-17 22:40:48,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080d6575bc060ce15c906 HTTP/1.1" 502 150 2025-12-17 22:40:48,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080d6575bc060ce15c906'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:48,659 - lava_callback - INFO - Received callback for node 69408e61575bc060ce15fddb INFO: 172.21.0.1:37666 - "POST /node/69408e61575bc060ce15fddb HTTP/1.0" 202 Accepted 2025-12-17 22:40:49,306 - lava_callback - INFO - Received callback for node 69408e63575bc060ce15fdde INFO: 172.21.0.1:37682 - "POST /node/69408e63575bc060ce15fdde HTTP/1.0" 202 Accepted 2025-12-17 22:40:49,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c37575bc060ce15f4b2 2025-12-17 22:40:49,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c37575bc060ce15f4b2 HTTP/1.1" 502 150 2025-12-17 22:40:49,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c37575bc060ce15f4b2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:49,965 - lava_callback - INFO - Received callback for node 69408e64575bc060ce15fde1 INFO: 172.21.0.1:41872 - "POST /node/69408e64575bc060ce15fde1 HTTP/1.0" 202 Accepted 2025-12-17 22:40:50,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c11575bc060ce15f440 2025-12-17 22:40:50,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c11575bc060ce15f440 HTTP/1.1" 502 150 2025-12-17 22:40:50,035 - lava_callback - ERROR - Error processing callback for node 69408c11575bc060ce15f440: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c11575bc060ce15f440 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c11575bc060ce15f440 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c11575bc060ce15f440 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:50,037 - lava_callback - INFO - Starting async job submit for node 69408c45575bc060ce15f522 2025-12-17 22:40:50,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:50,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c45575bc060ce15f522 HTTP/1.1" 502 150 2025-12-17 22:40:50,069 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c45575bc060ce15f522'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:50,069 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c45575bc060ce15f522 2025-12-17 22:40:50,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c45575bc060ce15f522 HTTP/1.1" 502 150 2025-12-17 22:40:50,070 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c45575bc060ce15f522'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:50,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3a575bc060ce15f4b8 2025-12-17 22:40:50,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3a575bc060ce15f4b8 HTTP/1.1" 502 150 2025-12-17 22:40:50,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3a575bc060ce15f4b8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:50,636 - lava_callback - INFO - Received callback for node 69408e65575bc060ce15fde4 INFO: 172.21.0.1:41886 - "POST /node/69408e65575bc060ce15fde4 HTTP/1.0" 202 Accepted 2025-12-17 22:40:50,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c20575bc060ce15f46e 2025-12-17 22:40:50,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c20575bc060ce15f46e HTTP/1.1" 502 150 2025-12-17 22:40:50,854 - lava_callback - ERROR - Error processing callback for node 69408c20575bc060ce15f46e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c20575bc060ce15f46e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c20575bc060ce15f46e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c20575bc060ce15f46e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:50,856 - lava_callback - INFO - Starting async job submit for node 69408c46575bc060ce15f527 2025-12-17 22:40:50,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:50,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c46575bc060ce15f527 HTTP/1.1" 502 150 2025-12-17 22:40:50,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c46575bc060ce15f527'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:50,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c46575bc060ce15f527 2025-12-17 22:40:50,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c46575bc060ce15f527 HTTP/1.1" 502 150 2025-12-17 22:40:50,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c46575bc060ce15f527'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:50,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3b575bc060ce15f4bb 2025-12-17 22:40:50,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3b575bc060ce15f4bb HTTP/1.1" 502 150 2025-12-17 22:40:50,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3b575bc060ce15f4bb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:51,284 - lava_callback - INFO - Received callback for node 69408e66575bc060ce15fde7 INFO: 172.21.0.1:41888 - "POST /node/69408e66575bc060ce15fde7 HTTP/1.0" 202 Accepted 2025-12-17 22:40:51,916 - lava_callback - INFO - Received callback for node 69408e68575bc060ce15fdea INFO: 172.21.0.1:41900 - "POST /node/69408e68575bc060ce15fdea HTTP/1.0" 202 Accepted 2025-12-17 22:40:52,070 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c45575bc060ce15f522 2025-12-17 22:40:52,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c45575bc060ce15f522 HTTP/1.1" 502 150 2025-12-17 22:40:52,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c45575bc060ce15f522'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:52,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080d6575bc060ce15c906 2025-12-17 22:40:52,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c21575bc060ce15f471 2025-12-17 22:40:52,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080d6575bc060ce15c906 HTTP/1.1" 502 150 2025-12-17 22:40:52,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080d6575bc060ce15c906'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:52,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c21575bc060ce15f471 HTTP/1.1" 502 150 2025-12-17 22:40:52,484 - lava_callback - ERROR - Error processing callback for node 69408c21575bc060ce15f471: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c21575bc060ce15f471 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c21575bc060ce15f471 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c21575bc060ce15f471 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:52,487 - lava_callback - INFO - Starting async job submit for node 69408c2e575bc060ce15f4a0 2025-12-17 22:40:52,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:52,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2e575bc060ce15f4a0 HTTP/1.1" 502 150 2025-12-17 22:40:52,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2e575bc060ce15f4a0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:52,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2e575bc060ce15f4a0 2025-12-17 22:40:52,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2e575bc060ce15f4a0 HTTP/1.1" 502 150 2025-12-17 22:40:52,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2e575bc060ce15f4a0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:52,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c18575bc060ce15f45f HTTP/1.1" 502 150 2025-12-17 22:40:52,656 - lava_callback - ERROR - Error processing callback for node 69408c18575bc060ce15f45f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c18575bc060ce15f45f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c18575bc060ce15f45f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c18575bc060ce15f45f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:52,659 - lava_callback - INFO - Starting async job submit for node 69408c47575bc060ce15f52a 2025-12-17 22:40:52,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:52,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c47575bc060ce15f52a HTTP/1.1" 502 150 2025-12-17 22:40:52,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c47575bc060ce15f52a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:52,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c47575bc060ce15f52a 2025-12-17 22:40:52,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c47575bc060ce15f52a HTTP/1.1" 502 150 2025-12-17 22:40:52,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c47575bc060ce15f52a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:52,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c46575bc060ce15f527 2025-12-17 22:40:52,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c46575bc060ce15f527 HTTP/1.1" 502 150 2025-12-17 22:40:52,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c46575bc060ce15f527'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:52,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3c575bc060ce15f4c4 2025-12-17 22:40:52,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3c575bc060ce15f4c4 HTTP/1.1" 502 150 2025-12-17 22:40:52,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3c575bc060ce15f4c4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:52,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3e575bc060ce15f4d0 2025-12-17 22:40:52,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3e575bc060ce15f4d0 HTTP/1.1" 502 150 2025-12-17 22:40:52,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3e575bc060ce15f4d0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:54,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2e575bc060ce15f4a0 2025-12-17 22:40:54,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2e575bc060ce15f4a0 HTTP/1.1" 502 150 2025-12-17 22:40:54,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2e575bc060ce15f4a0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:54,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c47575bc060ce15f52a 2025-12-17 22:40:54,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c47575bc060ce15f52a HTTP/1.1" 502 150 2025-12-17 22:40:54,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c47575bc060ce15f52a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:55,083 - lava_callback - INFO - Received callback for node 69408e6b575bc060ce15fe0c INFO: 172.21.0.1:41914 - "POST /node/69408e6b575bc060ce15fe0c HTTP/1.0" 202 Accepted 2025-12-17 22:40:55,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3f575bc060ce15f4da 2025-12-17 22:40:55,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3f575bc060ce15f4da HTTP/1.1" 502 150 2025-12-17 22:40:55,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c3f575bc060ce15f4da'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:55,753 - lava_callback - INFO - Received callback for node 69408e6d575bc060ce15fe13 INFO: 172.21.0.1:41918 - "POST /node/69408e6d575bc060ce15fe13 HTTP/1.0" 202 Accepted 2025-12-17 22:40:56,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c45575bc060ce15f522 2025-12-17 22:40:56,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c45575bc060ce15f522 HTTP/1.1" 502 150 2025-12-17 22:40:56,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c45575bc060ce15f522'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:56,407 - lava_callback - INFO - Received callback for node 69408e6e575bc060ce15fe19 INFO: 172.21.0.1:41928 - "POST /node/69408e6e575bc060ce15fe19 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:41932 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:40:56,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c46575bc060ce15f527 2025-12-17 22:40:56,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c46575bc060ce15f527 HTTP/1.1" 502 150 2025-12-17 22:40:56,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c46575bc060ce15f527'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:57,074 - lava_callback - INFO - Received callback for node 69408e70575bc060ce15fe1c INFO: 172.21.0.1:41944 - "POST /node/69408e70575bc060ce15fe1c HTTP/1.0" 202 Accepted 2025-12-17 22:40:57,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c38575bc060ce15f4b5 HTTP/1.1" 502 150 2025-12-17 22:40:57,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c38575bc060ce15f4b5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:57,726 - lava_callback - INFO - Received callback for node 69408e71575bc060ce15fe1f INFO: 172.21.0.1:41960 - "POST /node/69408e71575bc060ce15fe1f HTTP/1.0" 202 Accepted 2025-12-17 22:40:58,403 - lava_callback - INFO - Received callback for node 69408e72575bc060ce15fe22 INFO: 172.21.0.1:41974 - "POST /node/69408e72575bc060ce15fe22 HTTP/1.0" 202 Accepted 2025-12-17 22:40:58,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2e575bc060ce15f4a0 2025-12-17 22:40:58,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2e575bc060ce15f4a0 HTTP/1.1" 502 150 2025-12-17 22:40:58,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2e575bc060ce15f4a0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:58,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c47575bc060ce15f52a 2025-12-17 22:40:58,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c47575bc060ce15f52a HTTP/1.1" 502 150 2025-12-17 22:40:58,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c47575bc060ce15f52a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:58,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c23575bc060ce15f474 2025-12-17 22:40:58,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c23575bc060ce15f474 HTTP/1.1" 502 150 2025-12-17 22:40:58,741 - lava_callback - ERROR - Error processing callback for node 69408c23575bc060ce15f474: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c23575bc060ce15f474 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c23575bc060ce15f474 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c23575bc060ce15f474 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:58,743 - lava_callback - INFO - Starting async job submit for node 69408c48575bc060ce15f52d 2025-12-17 22:40:58,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:58,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c48575bc060ce15f52d HTTP/1.1" 502 150 2025-12-17 22:40:58,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c48575bc060ce15f52d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:58,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c48575bc060ce15f52d 2025-12-17 22:40:58,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c48575bc060ce15f52d HTTP/1.1" 502 150 2025-12-17 22:40:58,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c48575bc060ce15f52d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:59,073 - lava_callback - INFO - Received callback for node 69408e74575bc060ce15fe25 INFO: 172.21.0.1:41976 - "POST /node/69408e74575bc060ce15fe25 HTTP/1.0" 202 Accepted 2025-12-17 22:40:59,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2a575bc060ce15f48f 2025-12-17 22:40:59,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2a575bc060ce15f48f HTTP/1.1" 502 150 2025-12-17 22:40:59,745 - lava_callback - ERROR - Error processing callback for node 69408c2a575bc060ce15f48f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c2a575bc060ce15f48f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c2a575bc060ce15f48f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c2a575bc060ce15f48f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:40:59,747 - lava_callback - INFO - Starting async job submit for node 69408c30575bc060ce15f4a3 2025-12-17 22:40:59,767 - lava_callback - INFO - Received callback for node 69408e75575bc060ce15fe28 INFO: 172.21.0.1:45624 - "POST /node/69408e75575bc060ce15fe28 HTTP/1.0" 202 Accepted 2025-12-17 22:40:59,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:40:59,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c30575bc060ce15f4a3 HTTP/1.1" 502 150 2025-12-17 22:40:59,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c30575bc060ce15f4a3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:40:59,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c30575bc060ce15f4a3 2025-12-17 22:40:59,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c30575bc060ce15f4a3 HTTP/1.1" 502 150 2025-12-17 22:40:59,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c30575bc060ce15f4a3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:00,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080d6575bc060ce15c906 2025-12-17 22:41:00,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080d6575bc060ce15c906 HTTP/1.1" 502 150 2025-12-17 22:41:00,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694080d6575bc060ce15c906'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:00,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c48575bc060ce15f52d 2025-12-17 22:41:00,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c48575bc060ce15f52d HTTP/1.1" 502 150 2025-12-17 22:41:00,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c48575bc060ce15f52d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:01,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c30575bc060ce15f4a3 2025-12-17 22:41:01,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c30575bc060ce15f4a3 HTTP/1.1" 502 150 2025-12-17 22:41:01,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c30575bc060ce15f4a3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45638 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:41:02,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2b575bc060ce15f496 2025-12-17 22:41:02,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2b575bc060ce15f496 HTTP/1.1" 502 150 2025-12-17 22:41:02,929 - lava_callback - ERROR - Error processing callback for node 69408c2b575bc060ce15f496: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c2b575bc060ce15f496 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c2b575bc060ce15f496 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c2b575bc060ce15f496 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:02,932 - lava_callback - INFO - Starting async job submit for node 69408c4a575bc060ce15f530 2025-12-17 22:41:02,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:02,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4a575bc060ce15f530 HTTP/1.1" 502 150 2025-12-17 22:41:02,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4a575bc060ce15f530'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:02,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4a575bc060ce15f530 2025-12-17 22:41:02,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4a575bc060ce15f530 HTTP/1.1" 502 150 2025-12-17 22:41:02,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4a575bc060ce15f530'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:03,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2d575bc060ce15f49d 2025-12-17 22:41:03,112 - lava_callback - INFO - Received callback for node 69408e98575bc060ce15fe91 INFO: 172.21.0.1:45652 - "POST /node/69408e98575bc060ce15fe91 HTTP/1.0" 202 Accepted 2025-12-17 22:41:03,165 - lava_callback - INFO - Received callback for node 69408e7b575bc060ce15fe34 INFO: 172.21.0.1:45656 - "POST /node/69408e7b575bc060ce15fe34 HTTP/1.0" 202 Accepted 2025-12-17 22:41:03,968 - lava_callback - INFO - Received callback for node 69408e7c575bc060ce15fe37 INFO: 172.21.0.1:45660 - "POST /node/69408e7c575bc060ce15fe37 HTTP/1.0" 202 Accepted 2025-12-17 22:41:04,074 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c45575bc060ce15f522 2025-12-17 22:41:04,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c45575bc060ce15f522 HTTP/1.1" 502 150 2025-12-17 22:41:04,075 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c45575bc060ce15f522'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:04,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c48575bc060ce15f52d 2025-12-17 22:41:04,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c48575bc060ce15f52d HTTP/1.1" 502 150 2025-12-17 22:41:04,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c48575bc060ce15f52d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:04,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c46575bc060ce15f527 2025-12-17 22:41:04,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c46575bc060ce15f527 HTTP/1.1" 502 150 2025-12-17 22:41:04,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c46575bc060ce15f527'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:04,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4a575bc060ce15f530 2025-12-17 22:41:04,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4a575bc060ce15f530 HTTP/1.1" 502 150 2025-12-17 22:41:04,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4a575bc060ce15f530'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:05,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c38575bc060ce15f4b5 2025-12-17 22:41:05,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c38575bc060ce15f4b5 HTTP/1.1" 502 150 2025-12-17 22:41:05,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c38575bc060ce15f4b5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:05,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c30575bc060ce15f4a3 2025-12-17 22:41:05,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c30575bc060ce15f4a3 HTTP/1.1" 502 150 2025-12-17 22:41:05,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c30575bc060ce15f4a3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:05,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c37575bc060ce15f4b2 2025-12-17 22:41:05,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c37575bc060ce15f4b2 HTTP/1.1" 502 150 2025-12-17 22:41:05,872 - lava_callback - ERROR - Error processing callback for node 69408c37575bc060ce15f4b2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c37575bc060ce15f4b2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c37575bc060ce15f4b2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c37575bc060ce15f4b2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:05,874 - lava_callback - INFO - Starting async job submit for node 69408c4b575bc060ce15f533 2025-12-17 22:41:05,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:05,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4b575bc060ce15f533 HTTP/1.1" 502 150 2025-12-17 22:41:05,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4b575bc060ce15f533'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:05,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4b575bc060ce15f533 2025-12-17 22:41:05,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4b575bc060ce15f533 HTTP/1.1" 502 150 2025-12-17 22:41:05,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4b575bc060ce15f533'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45670 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:41:06,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3a575bc060ce15f4b8 2025-12-17 22:41:06,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3a575bc060ce15f4b8 HTTP/1.1" 502 150 2025-12-17 22:41:06,543 - lava_callback - ERROR - Error processing callback for node 69408c3a575bc060ce15f4b8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c3a575bc060ce15f4b8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c3a575bc060ce15f4b8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c3a575bc060ce15f4b8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:06,545 - lava_callback - INFO - Starting async job submit for node 69408c4c575bc060ce15f536 2025-12-17 22:41:06,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:06,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4c575bc060ce15f536 HTTP/1.1" 502 150 2025-12-17 22:41:06,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4c575bc060ce15f536'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:06,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4c575bc060ce15f536 2025-12-17 22:41:06,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4c575bc060ce15f536 HTTP/1.1" 502 150 2025-12-17 22:41:06,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4c575bc060ce15f536'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:06,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2e575bc060ce15f4a0 2025-12-17 22:41:06,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2e575bc060ce15f4a0 HTTP/1.1" 502 150 2025-12-17 22:41:06,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c2e575bc060ce15f4a0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:06,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c47575bc060ce15f52a 2025-12-17 22:41:06,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c47575bc060ce15f52a HTTP/1.1" 502 150 2025-12-17 22:41:06,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c47575bc060ce15f52a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:06,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3b575bc060ce15f4bb 2025-12-17 22:41:06,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3b575bc060ce15f4bb HTTP/1.1" 502 150 2025-12-17 22:41:06,981 - lava_callback - ERROR - Error processing callback for node 69408c3b575bc060ce15f4bb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c3b575bc060ce15f4bb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c3b575bc060ce15f4bb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c3b575bc060ce15f4bb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:06,983 - lava_callback - INFO - Starting async job submit for node 69408c4d575bc060ce15f539 2025-12-17 22:41:06,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:06,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4d575bc060ce15f539 HTTP/1.1" 502 150 2025-12-17 22:41:06,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4d575bc060ce15f539'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:06,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4d575bc060ce15f539 2025-12-17 22:41:06,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4d575bc060ce15f539 HTTP/1.1" 502 150 2025-12-17 22:41:06,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4d575bc060ce15f539'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:07,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4b575bc060ce15f533 2025-12-17 22:41:07,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4b575bc060ce15f533 HTTP/1.1" 502 150 2025-12-17 22:41:07,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4b575bc060ce15f533'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:08,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4c575bc060ce15f536 2025-12-17 22:41:08,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4c575bc060ce15f536 HTTP/1.1" 502 150 2025-12-17 22:41:08,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4c575bc060ce15f536'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:08,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4a575bc060ce15f530 2025-12-17 22:41:08,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4a575bc060ce15f530 HTTP/1.1" 502 150 2025-12-17 22:41:08,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4a575bc060ce15f530'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:08,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3c575bc060ce15f4c4 2025-12-17 22:41:08,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3c575bc060ce15f4c4 HTTP/1.1" 502 150 2025-12-17 22:41:08,972 - lava_callback - ERROR - Error processing callback for node 69408c3c575bc060ce15f4c4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c3c575bc060ce15f4c4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c3c575bc060ce15f4c4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c3c575bc060ce15f4c4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:08,974 - lava_callback - INFO - Starting async job submit for node 69408c4f575bc060ce15f53c 2025-12-17 22:41:08,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:08,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4f575bc060ce15f53c HTTP/1.1" 502 150 2025-12-17 22:41:08,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4f575bc060ce15f53c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:08,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4f575bc060ce15f53c 2025-12-17 22:41:08,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4f575bc060ce15f53c HTTP/1.1" 502 150 2025-12-17 22:41:08,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4f575bc060ce15f53c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:08,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3e575bc060ce15f4d0 2025-12-17 22:41:08,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3e575bc060ce15f4d0 HTTP/1.1" 502 150 2025-12-17 22:41:08,987 - lava_callback - ERROR - Error processing callback for node 69408c3e575bc060ce15f4d0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c3e575bc060ce15f4d0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c3e575bc060ce15f4d0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c3e575bc060ce15f4d0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:08,989 - lava_callback - INFO - Starting async job submit for node 69408c50575bc060ce15f547 2025-12-17 22:41:08,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:08,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c50575bc060ce15f547 HTTP/1.1" 502 150 2025-12-17 22:41:08,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c50575bc060ce15f547'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:08,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c50575bc060ce15f547 2025-12-17 22:41:08,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c50575bc060ce15f547 HTTP/1.1" 502 150 2025-12-17 22:41:08,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c50575bc060ce15f547'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:08,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4d575bc060ce15f539 2025-12-17 22:41:08,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4d575bc060ce15f539 HTTP/1.1" 502 150 2025-12-17 22:41:08,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4d575bc060ce15f539'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:10,192 - lava_callback - INFO - Received callback for node 69408e7d575bc060ce15fe3d INFO: 172.21.0.1:52980 - "POST /node/69408e7d575bc060ce15fe3d HTTP/1.0" 202 Accepted 2025-12-17 22:41:10,893 - lava_callback - INFO - Received callback for node 69408e7e575bc060ce15fe40 INFO: 172.21.0.1:52992 - "POST /node/69408e7e575bc060ce15fe40 HTTP/1.0" 202 Accepted 2025-12-17 22:41:10,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4f575bc060ce15f53c 2025-12-17 22:41:10,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4f575bc060ce15f53c HTTP/1.1" 502 150 2025-12-17 22:41:10,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4f575bc060ce15f53c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:10,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c50575bc060ce15f547 2025-12-17 22:41:10,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c50575bc060ce15f547 HTTP/1.1" 502 150 2025-12-17 22:41:10,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c50575bc060ce15f547'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:11,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c3f575bc060ce15f4da 2025-12-17 22:41:11,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c3f575bc060ce15f4da HTTP/1.1" 502 150 2025-12-17 22:41:11,458 - lava_callback - ERROR - Error processing callback for node 69408c3f575bc060ce15f4da: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c3f575bc060ce15f4da (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c3f575bc060ce15f4da (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c3f575bc060ce15f4da (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:11,460 - lava_callback - INFO - Starting async job submit for node 69408c52575bc060ce15f550 2025-12-17 22:41:11,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:11,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c52575bc060ce15f550 HTTP/1.1" 502 150 2025-12-17 22:41:11,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c52575bc060ce15f550'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:11,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c52575bc060ce15f550 2025-12-17 22:41:11,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c52575bc060ce15f550 HTTP/1.1" 502 150 2025-12-17 22:41:11,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c52575bc060ce15f550'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:11,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4b575bc060ce15f533 2025-12-17 22:41:11,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4b575bc060ce15f533 HTTP/1.1" 502 150 2025-12-17 22:41:11,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4b575bc060ce15f533'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:12,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4c575bc060ce15f536 2025-12-17 22:41:12,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4c575bc060ce15f536 HTTP/1.1" 502 150 2025-12-17 22:41:12,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4c575bc060ce15f536'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:12,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c48575bc060ce15f52d 2025-12-17 22:41:12,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c48575bc060ce15f52d HTTP/1.1" 502 150 2025-12-17 22:41:12,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c48575bc060ce15f52d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:12,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4d575bc060ce15f539 2025-12-17 22:41:12,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4d575bc060ce15f539 HTTP/1.1" 502 150 2025-12-17 22:41:12,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4d575bc060ce15f539'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:13,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c52575bc060ce15f550 2025-12-17 22:41:13,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c52575bc060ce15f550 HTTP/1.1" 502 150 2025-12-17 22:41:13,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c52575bc060ce15f550'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:13,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c30575bc060ce15f4a3 2025-12-17 22:41:13,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c30575bc060ce15f4a3 HTTP/1.1" 502 150 2025-12-17 22:41:13,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c30575bc060ce15f4a3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:14,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4f575bc060ce15f53c 2025-12-17 22:41:14,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4f575bc060ce15f53c HTTP/1.1" 502 150 2025-12-17 22:41:14,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4f575bc060ce15f53c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:14,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c50575bc060ce15f547 2025-12-17 22:41:14,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c50575bc060ce15f547 HTTP/1.1" 502 150 2025-12-17 22:41:14,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c50575bc060ce15f547'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:15,069 - lava_callback - INFO - Received callback for node 69408e76575bc060ce15fe2b INFO: 172.21.0.1:53004 - "POST /node/69408e76575bc060ce15fe2b HTTP/1.0" 202 Accepted INFO: 172.21.0.1:53010 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:41:15,739 - lava_callback - INFO - Received callback for node 69408e77575bc060ce15fe2e INFO: 172.21.0.1:53018 - "POST /node/69408e77575bc060ce15fe2e HTTP/1.0" 202 Accepted 2025-12-17 22:41:16,394 - lava_callback - INFO - Received callback for node 69408e86575bc060ce15fe5a INFO: 172.21.0.1:53034 - "POST /node/69408e86575bc060ce15fe5a HTTP/1.0" 202 Accepted 2025-12-17 22:41:16,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694080d6575bc060ce15c906 2025-12-17 22:41:16,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694080d6575bc060ce15c906 HTTP/1.1" 502 150 2025-12-17 22:41:16,487 - lava_callback - ERROR - Error processing callback for node 694080d6575bc060ce15c906: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080d6575bc060ce15c906 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080d6575bc060ce15c906 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694080d6575bc060ce15c906 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:16,489 - lava_callback - INFO - Starting async job submit for node 69408c53575bc060ce15f557 2025-12-17 22:41:16,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:16,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c53575bc060ce15f557 HTTP/1.1" 502 150 2025-12-17 22:41:16,495 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c53575bc060ce15f557'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:16,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c53575bc060ce15f557 2025-12-17 22:41:16,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c53575bc060ce15f557 HTTP/1.1" 502 150 2025-12-17 22:41:16,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c53575bc060ce15f557'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53038 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:41:16,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4a575bc060ce15f530 2025-12-17 22:41:16,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4a575bc060ce15f530 HTTP/1.1" 502 150 2025-12-17 22:41:16,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4a575bc060ce15f530'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:17,075 - lava_callback - INFO - Received callback for node 69408e87575bc060ce15fe5d INFO: 172.21.0.1:53048 - "POST /node/69408e87575bc060ce15fe5d HTTP/1.0" 202 Accepted 2025-12-17 22:41:17,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c52575bc060ce15f550 2025-12-17 22:41:17,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c52575bc060ce15f550 HTTP/1.1" 502 150 2025-12-17 22:41:17,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c52575bc060ce15f550'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:17,751 - lava_callback - INFO - Received callback for node 69408e88575bc060ce15fe63 INFO: 172.21.0.1:53052 - "POST /node/69408e88575bc060ce15fe63 HTTP/1.0" 202 Accepted 2025-12-17 22:41:18,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2d575bc060ce15f49d HTTP/1.1" 502 150 2025-12-17 22:41:18,257 - lava_callback - ERROR - Error processing callback for node 69408c2d575bc060ce15f49d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c2d575bc060ce15f49d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c2d575bc060ce15f49d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c2d575bc060ce15f49d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:18,258 - lava_callback - INFO - Starting async job submit for node 69408c54575bc060ce15f55c 2025-12-17 22:41:18,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:18,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c54575bc060ce15f55c HTTP/1.1" 502 150 2025-12-17 22:41:18,290 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c54575bc060ce15f55c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:18,290 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c54575bc060ce15f55c 2025-12-17 22:41:18,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c54575bc060ce15f55c HTTP/1.1" 502 150 2025-12-17 22:41:18,291 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c54575bc060ce15f55c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:18,428 - lava_callback - INFO - Received callback for node 69408e8a575bc060ce15fe67 INFO: 172.21.0.1:53054 - "POST /node/69408e8a575bc060ce15fe67 HTTP/1.0" 202 Accepted 2025-12-17 22:41:18,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c53575bc060ce15f557 2025-12-17 22:41:18,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c53575bc060ce15f557 HTTP/1.1" 502 150 2025-12-17 22:41:18,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c53575bc060ce15f557'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:19,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4b575bc060ce15f533 2025-12-17 22:41:19,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4b575bc060ce15f533 HTTP/1.1" 502 150 2025-12-17 22:41:19,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4b575bc060ce15f533'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:20,075 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c45575bc060ce15f522 2025-12-17 22:41:20,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c45575bc060ce15f522 HTTP/1.1" 502 150 2025-12-17 22:41:20,077 - lava_callback - ERROR - Error processing callback for node 69408c45575bc060ce15f522: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c45575bc060ce15f522 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c45575bc060ce15f522 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c45575bc060ce15f522 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:20,079 - lava_callback - INFO - Starting async job submit for node 69408c56575bc060ce15f55f 2025-12-17 22:41:20,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:20,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c56575bc060ce15f55f HTTP/1.1" 502 150 2025-12-17 22:41:20,085 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c56575bc060ce15f55f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:20,085 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c56575bc060ce15f55f 2025-12-17 22:41:20,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c56575bc060ce15f55f HTTP/1.1" 502 150 2025-12-17 22:41:20,086 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c56575bc060ce15f55f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:20,291 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c54575bc060ce15f55c 2025-12-17 22:41:20,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c54575bc060ce15f55c HTTP/1.1" 502 150 2025-12-17 22:41:20,292 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c54575bc060ce15f55c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:20,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4c575bc060ce15f536 2025-12-17 22:41:20,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4c575bc060ce15f536 HTTP/1.1" 502 150 2025-12-17 22:41:20,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4c575bc060ce15f536'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:20,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c46575bc060ce15f527 2025-12-17 22:41:20,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c46575bc060ce15f527 HTTP/1.1" 502 150 2025-12-17 22:41:20,870 - lava_callback - ERROR - Error processing callback for node 69408c46575bc060ce15f527: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c46575bc060ce15f527 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c46575bc060ce15f527 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c46575bc060ce15f527 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:20,872 - lava_callback - INFO - Starting async job submit for node 69408c57575bc060ce15f562 2025-12-17 22:41:20,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:20,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c57575bc060ce15f562 HTTP/1.1" 502 150 2025-12-17 22:41:20,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c57575bc060ce15f562'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:20,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c57575bc060ce15f562 2025-12-17 22:41:20,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c57575bc060ce15f562 HTTP/1.1" 502 150 2025-12-17 22:41:20,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c57575bc060ce15f562'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:20,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4d575bc060ce15f539 2025-12-17 22:41:20,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4d575bc060ce15f539 HTTP/1.1" 502 150 2025-12-17 22:41:20,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4d575bc060ce15f539'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:21,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c38575bc060ce15f4b5 2025-12-17 22:41:21,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c38575bc060ce15f4b5 HTTP/1.1" 502 150 2025-12-17 22:41:21,525 - lava_callback - ERROR - Error processing callback for node 69408c38575bc060ce15f4b5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c38575bc060ce15f4b5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c38575bc060ce15f4b5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c38575bc060ce15f4b5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:21,528 - lava_callback - INFO - Starting async job submit for node 69408c58575bc060ce15f565 2025-12-17 22:41:21,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:21,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c58575bc060ce15f565 HTTP/1.1" 502 150 2025-12-17 22:41:21,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c58575bc060ce15f565'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:21,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c58575bc060ce15f565 2025-12-17 22:41:21,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c58575bc060ce15f565 HTTP/1.1" 502 150 2025-12-17 22:41:21,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c58575bc060ce15f565'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:22,086 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c56575bc060ce15f55f 2025-12-17 22:41:22,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c56575bc060ce15f55f HTTP/1.1" 502 150 2025-12-17 22:41:22,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c56575bc060ce15f55f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:22,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c53575bc060ce15f557 2025-12-17 22:41:22,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c53575bc060ce15f557 HTTP/1.1" 502 150 2025-12-17 22:41:22,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c53575bc060ce15f557'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:22,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c2e575bc060ce15f4a0 2025-12-17 22:41:22,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c2e575bc060ce15f4a0 HTTP/1.1" 502 150 2025-12-17 22:41:22,644 - lava_callback - ERROR - Error processing callback for node 69408c2e575bc060ce15f4a0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c2e575bc060ce15f4a0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c2e575bc060ce15f4a0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c2e575bc060ce15f4a0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:22,646 - lava_callback - INFO - Starting async job submit for node 69408c5a575bc060ce15f568 2025-12-17 22:41:22,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:22,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5a575bc060ce15f568 HTTP/1.1" 502 150 2025-12-17 22:41:22,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5a575bc060ce15f568'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:22,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5a575bc060ce15f568 2025-12-17 22:41:22,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5a575bc060ce15f568 HTTP/1.1" 502 150 2025-12-17 22:41:22,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5a575bc060ce15f568'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:22,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c47575bc060ce15f52a 2025-12-17 22:41:22,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c47575bc060ce15f52a HTTP/1.1" 502 150 2025-12-17 22:41:22,672 - lava_callback - ERROR - Error processing callback for node 69408c47575bc060ce15f52a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c47575bc060ce15f52a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c47575bc060ce15f52a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c47575bc060ce15f52a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:22,674 - lava_callback - INFO - Starting async job submit for node 69408c5b575bc060ce15f56b 2025-12-17 22:41:22,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:22,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5b575bc060ce15f56b HTTP/1.1" 502 150 2025-12-17 22:41:22,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5b575bc060ce15f56b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:22,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5b575bc060ce15f56b 2025-12-17 22:41:22,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5b575bc060ce15f56b HTTP/1.1" 502 150 2025-12-17 22:41:22,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5b575bc060ce15f56b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:22,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c57575bc060ce15f562 2025-12-17 22:41:22,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c57575bc060ce15f562 HTTP/1.1" 502 150 2025-12-17 22:41:22,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c57575bc060ce15f562'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:22,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4f575bc060ce15f53c 2025-12-17 22:41:22,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4f575bc060ce15f53c HTTP/1.1" 502 150 2025-12-17 22:41:22,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c4f575bc060ce15f53c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:22,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c50575bc060ce15f547 2025-12-17 22:41:22,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c50575bc060ce15f547 HTTP/1.1" 502 150 2025-12-17 22:41:22,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c50575bc060ce15f547'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:23,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c58575bc060ce15f565 2025-12-17 22:41:23,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c58575bc060ce15f565 HTTP/1.1" 502 150 2025-12-17 22:41:23,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c58575bc060ce15f565'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:24,292 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c54575bc060ce15f55c 2025-12-17 22:41:24,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c54575bc060ce15f55c HTTP/1.1" 502 150 2025-12-17 22:41:24,294 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c54575bc060ce15f55c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:24,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5a575bc060ce15f568 2025-12-17 22:41:24,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5a575bc060ce15f568 HTTP/1.1" 502 150 2025-12-17 22:41:24,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5a575bc060ce15f568'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:24,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5b575bc060ce15f56b 2025-12-17 22:41:24,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5b575bc060ce15f56b HTTP/1.1" 502 150 2025-12-17 22:41:24,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5b575bc060ce15f56b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:25,337 - lava_callback - INFO - Received callback for node 69408e80575bc060ce15fe45 INFO: 172.21.0.1:43390 - "POST /node/69408e80575bc060ce15fe45 HTTP/1.0" 202 Accepted 2025-12-17 22:41:25,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c52575bc060ce15f550 2025-12-17 22:41:25,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c52575bc060ce15f550 HTTP/1.1" 502 150 2025-12-17 22:41:25,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c52575bc060ce15f550'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:26,002 - lava_callback - INFO - Received callback for node 69408e81575bc060ce15fe4d INFO: 172.21.0.1:43394 - "POST /node/69408e81575bc060ce15fe4d HTTP/1.0" 202 Accepted 2025-12-17 22:41:26,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c56575bc060ce15f55f 2025-12-17 22:41:26,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c56575bc060ce15f55f HTTP/1.1" 502 150 2025-12-17 22:41:26,089 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c56575bc060ce15f55f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43408 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:41:26,643 - lava_callback - INFO - Received callback for node 69408e83575bc060ce15fe54 INFO: 172.21.0.1:43418 - "POST /node/69408e83575bc060ce15fe54 HTTP/1.0" 202 Accepted 2025-12-17 22:41:26,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c57575bc060ce15f562 2025-12-17 22:41:26,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c57575bc060ce15f562 HTTP/1.1" 502 150 2025-12-17 22:41:26,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c57575bc060ce15f562'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:26,955 - lava_callback - INFO - Received callback for node 6940fc16460bc8e265115db2 INFO: 172.21.0.1:43422 - "POST /node/6940fc16460bc8e265115db2 HTTP/1.0" 202 Accepted 2025-12-17 22:41:27,291 - lava_callback - INFO - Received callback for node 69408e84575bc060ce15fe57 INFO: 172.21.0.1:43432 - "POST /node/69408e84575bc060ce15fe57 HTTP/1.0" 202 Accepted 2025-12-17 22:41:27,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c58575bc060ce15f565 2025-12-17 22:41:27,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c58575bc060ce15f565 HTTP/1.1" 502 150 2025-12-17 22:41:27,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c58575bc060ce15f565'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:28,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5a575bc060ce15f568 2025-12-17 22:41:28,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5a575bc060ce15f568 HTTP/1.1" 502 150 2025-12-17 22:41:28,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5a575bc060ce15f568'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:28,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5b575bc060ce15f56b 2025-12-17 22:41:28,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5b575bc060ce15f56b HTTP/1.1" 502 150 2025-12-17 22:41:28,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5b575bc060ce15f56b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:28,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c48575bc060ce15f52d 2025-12-17 22:41:28,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c48575bc060ce15f52d HTTP/1.1" 502 150 2025-12-17 22:41:28,756 - lava_callback - ERROR - Error processing callback for node 69408c48575bc060ce15f52d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c48575bc060ce15f52d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c48575bc060ce15f52d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c48575bc060ce15f52d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:28,759 - lava_callback - INFO - Starting async job submit for node 69408c5c575bc060ce15f56e 2025-12-17 22:41:28,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:28,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5c575bc060ce15f56e HTTP/1.1" 502 150 2025-12-17 22:41:28,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5c575bc060ce15f56e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:28,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5c575bc060ce15f56e 2025-12-17 22:41:28,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5c575bc060ce15f56e HTTP/1.1" 502 150 2025-12-17 22:41:28,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5c575bc060ce15f56e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:29,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c30575bc060ce15f4a3 2025-12-17 22:41:30,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c53575bc060ce15f557 2025-12-17 22:41:30,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5c575bc060ce15f56e 2025-12-17 22:41:30,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5c575bc060ce15f56e HTTP/1.1" 502 150 2025-12-17 22:41:30,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5c575bc060ce15f56e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:32,294 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c54575bc060ce15f55c 2025-12-17 22:41:32,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c54575bc060ce15f55c HTTP/1.1" 502 150 2025-12-17 22:41:32,295 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c54575bc060ce15f55c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:32,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4a575bc060ce15f530 2025-12-17 22:41:32,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4a575bc060ce15f530 HTTP/1.1" 502 150 2025-12-17 22:41:32,949 - lava_callback - ERROR - Error processing callback for node 69408c4a575bc060ce15f530: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c4a575bc060ce15f530 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c4a575bc060ce15f530 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c4a575bc060ce15f530 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:32,950 - lava_callback - INFO - Starting async job submit for node 69408c5d575bc060ce15f571 2025-12-17 22:41:32,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:32,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5d575bc060ce15f571 HTTP/1.1" 502 150 2025-12-17 22:41:32,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5d575bc060ce15f571'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:32,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5d575bc060ce15f571 2025-12-17 22:41:32,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5d575bc060ce15f571 HTTP/1.1" 502 150 2025-12-17 22:41:32,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5d575bc060ce15f571'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:34,089 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c56575bc060ce15f55f 2025-12-17 22:41:34,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c56575bc060ce15f55f HTTP/1.1" 502 150 2025-12-17 22:41:34,090 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c56575bc060ce15f55f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:34,578 - lava_callback - INFO - Received callback for node 69408e9b575bc060ce15fea0 INFO: 172.21.0.1:51600 - "POST /node/69408e9b575bc060ce15fea0 HTTP/1.0" 202 Accepted 2025-12-17 22:41:34,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5c575bc060ce15f56e 2025-12-17 22:41:34,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5c575bc060ce15f56e HTTP/1.1" 502 150 2025-12-17 22:41:34,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5c575bc060ce15f56e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:34,843 - lava_callback - INFO - Received callback for node 6940949b575bc060ce161098 INFO: 172.21.0.1:51604 - "POST /node/6940949b575bc060ce161098 HTTP/1.0" 202 Accepted 2025-12-17 22:41:34,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c57575bc060ce15f562 2025-12-17 22:41:34,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c57575bc060ce15f562 HTTP/1.1" 502 150 2025-12-17 22:41:34,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c57575bc060ce15f562'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:34,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5d575bc060ce15f571 2025-12-17 22:41:34,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5d575bc060ce15f571 HTTP/1.1" 502 150 2025-12-17 22:41:34,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5d575bc060ce15f571'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:35,078 - lava_callback - INFO - Received callback for node 69408e96575bc060ce15fe8a INFO: 172.21.0.1:51606 - "POST /node/69408e96575bc060ce15fe8a HTTP/1.0" 202 Accepted 2025-12-17 22:41:35,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c58575bc060ce15f565 2025-12-17 22:41:35,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c58575bc060ce15f565 HTTP/1.1" 502 150 2025-12-17 22:41:35,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c58575bc060ce15f565'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:35,740 - lava_callback - INFO - Received callback for node 69408e8b575bc060ce15fe6a INFO: 172.21.0.1:51622 - "POST /node/69408e8b575bc060ce15fe6a HTTP/1.0" 202 Accepted 2025-12-17 22:41:35,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4b575bc060ce15f533 2025-12-17 22:41:35,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4b575bc060ce15f533 HTTP/1.1" 502 150 2025-12-17 22:41:35,890 - lava_callback - ERROR - Error processing callback for node 69408c4b575bc060ce15f533: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c4b575bc060ce15f533 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c4b575bc060ce15f533 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c4b575bc060ce15f533 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:35,893 - lava_callback - INFO - Starting async job submit for node 69408c5f575bc060ce15f574 2025-12-17 22:41:35,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:35,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5f575bc060ce15f574 HTTP/1.1" 502 150 2025-12-17 22:41:35,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5f575bc060ce15f574'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:35,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5f575bc060ce15f574 2025-12-17 22:41:35,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5f575bc060ce15f574 HTTP/1.1" 502 150 2025-12-17 22:41:35,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5f575bc060ce15f574'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:36,401 - lava_callback - INFO - Received callback for node 69408e8c575bc060ce15fe6d INFO: 172.21.0.1:51636 - "POST /node/69408e8c575bc060ce15fe6d HTTP/1.0" 202 Accepted INFO: 172.21.0.1:51638 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:41:36,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4c575bc060ce15f536 2025-12-17 22:41:36,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4c575bc060ce15f536 HTTP/1.1" 502 150 2025-12-17 22:41:36,558 - lava_callback - ERROR - Error processing callback for node 69408c4c575bc060ce15f536: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c4c575bc060ce15f536 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c4c575bc060ce15f536 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c4c575bc060ce15f536 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:36,560 - lava_callback - INFO - Starting async job submit for node 69408c63575bc060ce15f57f 2025-12-17 22:41:36,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:36,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c63575bc060ce15f57f HTTP/1.1" 502 150 2025-12-17 22:41:36,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c63575bc060ce15f57f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:36,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c63575bc060ce15f57f 2025-12-17 22:41:36,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c63575bc060ce15f57f HTTP/1.1" 502 150 2025-12-17 22:41:36,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c63575bc060ce15f57f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:36,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5a575bc060ce15f568 2025-12-17 22:41:36,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5a575bc060ce15f568 HTTP/1.1" 502 150 2025-12-17 22:41:36,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5a575bc060ce15f568'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:36,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5b575bc060ce15f56b 2025-12-17 22:41:36,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5b575bc060ce15f56b HTTP/1.1" 502 150 2025-12-17 22:41:36,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5b575bc060ce15f56b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:37,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4d575bc060ce15f539 2025-12-17 22:41:37,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4d575bc060ce15f539 HTTP/1.1" 502 150 2025-12-17 22:41:37,001 - lava_callback - ERROR - Error processing callback for node 69408c4d575bc060ce15f539: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c4d575bc060ce15f539 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c4d575bc060ce15f539 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c4d575bc060ce15f539 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:37,004 - lava_callback - INFO - Starting async job submit for node 69408c66575bc060ce15f58c 2025-12-17 22:41:37,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:37,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c66575bc060ce15f58c HTTP/1.1" 502 150 2025-12-17 22:41:37,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c66575bc060ce15f58c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:37,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c66575bc060ce15f58c 2025-12-17 22:41:37,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c66575bc060ce15f58c HTTP/1.1" 502 150 2025-12-17 22:41:37,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c66575bc060ce15f58c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:37,083 - lava_callback - INFO - Received callback for node 69408e8d575bc060ce15fe70 INFO: 172.21.0.1:51646 - "POST /node/69408e8d575bc060ce15fe70 HTTP/1.0" 202 Accepted 2025-12-17 22:41:37,731 - lava_callback - INFO - Received callback for node 69408e8f575bc060ce15fe73 INFO: 172.21.0.1:51662 - "POST /node/69408e8f575bc060ce15fe73 HTTP/1.0" 202 Accepted 2025-12-17 22:41:37,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5f575bc060ce15f574 2025-12-17 22:41:37,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5f575bc060ce15f574 HTTP/1.1" 502 150 2025-12-17 22:41:37,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5f575bc060ce15f574'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:38,371 - lava_callback - INFO - Received callback for node 69408e92575bc060ce15fe79 INFO: 172.21.0.1:51676 - "POST /node/69408e92575bc060ce15fe79 HTTP/1.0" 202 Accepted 2025-12-17 22:41:38,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c63575bc060ce15f57f 2025-12-17 22:41:38,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c63575bc060ce15f57f HTTP/1.1" 502 150 2025-12-17 22:41:38,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c63575bc060ce15f57f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:38,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5d575bc060ce15f571 2025-12-17 22:41:38,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5d575bc060ce15f571 HTTP/1.1" 502 150 2025-12-17 22:41:38,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5d575bc060ce15f571'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:38,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c4f575bc060ce15f53c 2025-12-17 22:41:38,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c4f575bc060ce15f53c HTTP/1.1" 502 150 2025-12-17 22:41:38,989 - lava_callback - ERROR - Error processing callback for node 69408c4f575bc060ce15f53c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c4f575bc060ce15f53c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c4f575bc060ce15f53c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c4f575bc060ce15f53c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:38,991 - lava_callback - INFO - Starting async job submit for node 69408c68575bc060ce15f591 2025-12-17 22:41:38,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:38,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c50575bc060ce15f547 2025-12-17 22:41:39,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c50575bc060ce15f547 HTTP/1.1" 502 150 2025-12-17 22:41:39,001 - lava_callback - ERROR - Error processing callback for node 69408c50575bc060ce15f547: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c50575bc060ce15f547 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c50575bc060ce15f547 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c50575bc060ce15f547 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:39,003 - lava_callback - INFO - Starting async job submit for node 69408c6a575bc060ce15f594 2025-12-17 22:41:39,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:39,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c68575bc060ce15f591 HTTP/1.1" 502 150 2025-12-17 22:41:39,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c68575bc060ce15f591'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:39,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c68575bc060ce15f591 2025-12-17 22:41:39,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c68575bc060ce15f591 HTTP/1.1" 502 150 2025-12-17 22:41:39,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c68575bc060ce15f591'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:39,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6a575bc060ce15f594 HTTP/1.1" 502 150 2025-12-17 22:41:39,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c6a575bc060ce15f594'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:39,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c6a575bc060ce15f594 2025-12-17 22:41:39,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6a575bc060ce15f594 HTTP/1.1" 502 150 2025-12-17 22:41:39,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c6a575bc060ce15f594'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:39,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c66575bc060ce15f58c 2025-12-17 22:41:39,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c66575bc060ce15f58c HTTP/1.1" 502 150 2025-12-17 22:41:39,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c66575bc060ce15f58c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:39,025 - lava_callback - INFO - Received callback for node 69408e93575bc060ce15fe7c INFO: 172.21.0.1:51680 - "POST /node/69408e93575bc060ce15fe7c HTTP/1.0" 202 Accepted 2025-12-17 22:41:41,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c68575bc060ce15f591 2025-12-17 22:41:41,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c6a575bc060ce15f594 2025-12-17 22:41:41,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6a575bc060ce15f594 HTTP/1.1" 502 150 2025-12-17 22:41:41,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c6a575bc060ce15f594'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:41,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c52575bc060ce15f550 2025-12-17 22:41:41,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c52575bc060ce15f550 HTTP/1.1" 502 150 2025-12-17 22:41:41,474 - lava_callback - ERROR - Error processing callback for node 69408c52575bc060ce15f550: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c52575bc060ce15f550 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c52575bc060ce15f550 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c52575bc060ce15f550 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:41,476 - lava_callback - INFO - Starting async job submit for node 69408c6c575bc060ce15f597 2025-12-17 22:41:41,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:41,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6c575bc060ce15f597 HTTP/1.1" 502 150 2025-12-17 22:41:41,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c6c575bc060ce15f597'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:41,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c6c575bc060ce15f597 2025-12-17 22:41:41,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6c575bc060ce15f597 HTTP/1.1" 502 150 2025-12-17 22:41:41,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c6c575bc060ce15f597'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:41,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5f575bc060ce15f574 2025-12-17 22:41:41,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5f575bc060ce15f574 HTTP/1.1" 502 150 2025-12-17 22:41:41,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5f575bc060ce15f574'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:42,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c63575bc060ce15f57f 2025-12-17 22:41:42,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c63575bc060ce15f57f HTTP/1.1" 502 150 2025-12-17 22:41:42,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c63575bc060ce15f57f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:42,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5c575bc060ce15f56e 2025-12-17 22:41:42,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5c575bc060ce15f56e HTTP/1.1" 502 150 2025-12-17 22:41:42,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5c575bc060ce15f56e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:43,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c66575bc060ce15f58c 2025-12-17 22:41:43,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c66575bc060ce15f58c HTTP/1.1" 502 150 2025-12-17 22:41:43,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c66575bc060ce15f58c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:43,275 - lava_callback - INFO - Received callback for node 69408ea3575bc060ce15fea9 INFO: 172.21.0.1:56788 - "POST /node/69408ea3575bc060ce15fea9 HTTP/1.0" 202 Accepted 2025-12-17 22:41:43,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c6c575bc060ce15f597 2025-12-17 22:41:43,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6c575bc060ce15f597 HTTP/1.1" 502 150 2025-12-17 22:41:43,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c6c575bc060ce15f597'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:44,029 - lava_callback - INFO - Received callback for node 69408ea6575bc060ce15febc INFO: 172.21.0.1:56792 - "POST /node/69408ea6575bc060ce15febc HTTP/1.0" 202 Accepted 2025-12-17 22:41:45,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c6a575bc060ce15f594 2025-12-17 22:41:45,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c30575bc060ce15f4a3 HTTP/1.1" 502 150 2025-12-17 22:41:45,136 - lava_callback - ERROR - Error processing callback for node 69408c30575bc060ce15f4a3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c30575bc060ce15f4a3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c30575bc060ce15f4a3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c30575bc060ce15f4a3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:45,138 - lava_callback - INFO - Starting async job submit for node 69408c6e575bc060ce15f59a 2025-12-17 22:41:45,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:45,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6e575bc060ce15f59a HTTP/1.1" 502 150 2025-12-17 22:41:45,145 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c6e575bc060ce15f59a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:45,145 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c6e575bc060ce15f59a 2025-12-17 22:41:45,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6e575bc060ce15f59a HTTP/1.1" 502 150 2025-12-17 22:41:45,145 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c6e575bc060ce15f59a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:45,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c53575bc060ce15f557 HTTP/1.1" 502 150 2025-12-17 22:41:45,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c53575bc060ce15f557'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56808 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:41:46,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5d575bc060ce15f571 2025-12-17 22:41:46,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5d575bc060ce15f571 HTTP/1.1" 502 150 2025-12-17 22:41:46,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5d575bc060ce15f571'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:47,146 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c6e575bc060ce15f59a 2025-12-17 22:41:47,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6e575bc060ce15f59a HTTP/1.1" 502 150 2025-12-17 22:41:47,147 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c6e575bc060ce15f59a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:47,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c6c575bc060ce15f597 2025-12-17 22:41:47,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6c575bc060ce15f597 HTTP/1.1" 502 150 2025-12-17 22:41:47,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c6c575bc060ce15f597'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:48,295 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c54575bc060ce15f55c 2025-12-17 22:41:48,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c54575bc060ce15f55c HTTP/1.1" 502 150 2025-12-17 22:41:48,297 - lava_callback - ERROR - Error processing callback for node 69408c54575bc060ce15f55c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c54575bc060ce15f55c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c54575bc060ce15f55c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c54575bc060ce15f55c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:48,299 - lava_callback - INFO - Starting async job submit for node 69408c70575bc060ce15f59d 2025-12-17 22:41:48,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:48,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c70575bc060ce15f59d HTTP/1.1" 502 150 2025-12-17 22:41:48,305 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c70575bc060ce15f59d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:48,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c70575bc060ce15f59d 2025-12-17 22:41:48,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c70575bc060ce15f59d HTTP/1.1" 502 150 2025-12-17 22:41:48,306 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c70575bc060ce15f59d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:49,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5f575bc060ce15f574 2025-12-17 22:41:50,091 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c56575bc060ce15f55f 2025-12-17 22:41:50,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c56575bc060ce15f55f HTTP/1.1" 502 150 2025-12-17 22:41:50,092 - lava_callback - ERROR - Error processing callback for node 69408c56575bc060ce15f55f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c56575bc060ce15f55f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c56575bc060ce15f55f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c56575bc060ce15f55f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:50,094 - lava_callback - INFO - Starting async job submit for node 69408c74575bc060ce15f5a3 2025-12-17 22:41:50,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:50,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c74575bc060ce15f5a3 HTTP/1.1" 502 150 2025-12-17 22:41:50,120 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c74575bc060ce15f5a3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:50,120 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c74575bc060ce15f5a3 2025-12-17 22:41:50,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c74575bc060ce15f5a3 HTTP/1.1" 502 150 2025-12-17 22:41:50,121 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c74575bc060ce15f5a3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:50,307 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c70575bc060ce15f59d 2025-12-17 22:41:50,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c70575bc060ce15f59d HTTP/1.1" 502 150 2025-12-17 22:41:50,308 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c70575bc060ce15f59d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:50,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c63575bc060ce15f57f 2025-12-17 22:41:50,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c63575bc060ce15f57f HTTP/1.1" 502 150 2025-12-17 22:41:50,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c63575bc060ce15f57f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:50,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c57575bc060ce15f562 2025-12-17 22:41:50,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c57575bc060ce15f562 HTTP/1.1" 502 150 2025-12-17 22:41:50,886 - lava_callback - ERROR - Error processing callback for node 69408c57575bc060ce15f562: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c57575bc060ce15f562 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c57575bc060ce15f562 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c57575bc060ce15f562 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:50,888 - lava_callback - INFO - Starting async job submit for node 69408c76575bc060ce15f5a6 2025-12-17 22:41:50,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:50,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c76575bc060ce15f5a6 HTTP/1.1" 502 150 2025-12-17 22:41:50,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c76575bc060ce15f5a6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:50,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c76575bc060ce15f5a6 2025-12-17 22:41:50,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c76575bc060ce15f5a6 HTTP/1.1" 502 150 2025-12-17 22:41:50,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c76575bc060ce15f5a6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:51,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c66575bc060ce15f58c 2025-12-17 22:41:51,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c66575bc060ce15f58c HTTP/1.1" 502 150 2025-12-17 22:41:51,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c66575bc060ce15f58c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:51,147 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c6e575bc060ce15f59a 2025-12-17 22:41:51,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6e575bc060ce15f59a HTTP/1.1" 502 150 2025-12-17 22:41:51,148 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c6e575bc060ce15f59a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:51,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c58575bc060ce15f565 2025-12-17 22:41:51,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c58575bc060ce15f565 HTTP/1.1" 502 150 2025-12-17 22:41:51,548 - lava_callback - ERROR - Error processing callback for node 69408c58575bc060ce15f565: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c58575bc060ce15f565 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c58575bc060ce15f565 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c58575bc060ce15f565 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:51,551 - lava_callback - INFO - Starting async job submit for node 69408c60575bc060ce15f577 2025-12-17 22:41:51,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:51,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c60575bc060ce15f577 HTTP/1.1" 502 150 2025-12-17 22:41:51,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c60575bc060ce15f577'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:51,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c60575bc060ce15f577 2025-12-17 22:41:51,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c60575bc060ce15f577 HTTP/1.1" 502 150 2025-12-17 22:41:51,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c60575bc060ce15f577'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:52,121 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c74575bc060ce15f5a3 2025-12-17 22:41:52,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c74575bc060ce15f5a3 HTTP/1.1" 502 150 2025-12-17 22:41:52,122 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c74575bc060ce15f5a3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:52,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5a575bc060ce15f568 2025-12-17 22:41:52,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5a575bc060ce15f568 HTTP/1.1" 502 150 2025-12-17 22:41:52,659 - lava_callback - ERROR - Error processing callback for node 69408c5a575bc060ce15f568: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c5a575bc060ce15f568 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c5a575bc060ce15f568 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c5a575bc060ce15f568 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:52,660 - lava_callback - INFO - Starting async job submit for node 69408c78575bc060ce15f5ad 2025-12-17 22:41:52,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:52,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c78575bc060ce15f5ad HTTP/1.1" 502 150 2025-12-17 22:41:52,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c78575bc060ce15f5ad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:52,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c78575bc060ce15f5ad 2025-12-17 22:41:52,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c78575bc060ce15f5ad HTTP/1.1" 502 150 2025-12-17 22:41:52,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c78575bc060ce15f5ad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:52,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5b575bc060ce15f56b 2025-12-17 22:41:52,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5b575bc060ce15f56b HTTP/1.1" 502 150 2025-12-17 22:41:52,689 - lava_callback - ERROR - Error processing callback for node 69408c5b575bc060ce15f56b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c5b575bc060ce15f56b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c5b575bc060ce15f56b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c5b575bc060ce15f56b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:52,691 - lava_callback - INFO - Starting async job submit for node 69408c7a575bc060ce15f5b6 2025-12-17 22:41:52,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:52,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7a575bc060ce15f5b6 HTTP/1.1" 502 150 2025-12-17 22:41:52,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c7a575bc060ce15f5b6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:52,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c7a575bc060ce15f5b6 2025-12-17 22:41:52,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7a575bc060ce15f5b6 HTTP/1.1" 502 150 2025-12-17 22:41:52,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c7a575bc060ce15f5b6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:52,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c76575bc060ce15f5a6 2025-12-17 22:41:52,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c76575bc060ce15f5a6 HTTP/1.1" 502 150 2025-12-17 22:41:52,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c76575bc060ce15f5a6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:53,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c60575bc060ce15f577 2025-12-17 22:41:53,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c60575bc060ce15f577 HTTP/1.1" 502 150 2025-12-17 22:41:53,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c60575bc060ce15f577'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:54,308 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c70575bc060ce15f59d 2025-12-17 22:41:54,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c70575bc060ce15f59d HTTP/1.1" 502 150 2025-12-17 22:41:54,309 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c70575bc060ce15f59d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:54,528 - lava_callback - INFO - Received callback for node 69408e9e575bc060ce15fea6 INFO: 172.21.0.1:48704 - "POST /node/69408e9e575bc060ce15fea6 HTTP/1.0" 202 Accepted 2025-12-17 22:41:54,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c78575bc060ce15f5ad 2025-12-17 22:41:54,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c78575bc060ce15f5ad HTTP/1.1" 502 150 2025-12-17 22:41:54,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c78575bc060ce15f5ad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:54,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c7a575bc060ce15f5b6 2025-12-17 22:41:54,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7a575bc060ce15f5b6 HTTP/1.1" 502 150 2025-12-17 22:41:54,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c7a575bc060ce15f5b6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:55,165 - lava_callback - INFO - Received callback for node 69408ea5575bc060ce15feac INFO: 172.21.0.1:48716 - "POST /node/69408ea5575bc060ce15feac HTTP/1.0" 202 Accepted 2025-12-17 22:41:55,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c6c575bc060ce15f597 2025-12-17 22:41:55,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6c575bc060ce15f597 HTTP/1.1" 502 150 2025-12-17 22:41:55,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c6c575bc060ce15f597'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:56,122 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c74575bc060ce15f5a3 2025-12-17 22:41:56,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c68575bc060ce15f591 HTTP/1.1" 502 150 2025-12-17 22:41:56,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c68575bc060ce15f591'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48730 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:41:56,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c76575bc060ce15f5a6 2025-12-17 22:41:56,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c76575bc060ce15f5a6 HTTP/1.1" 502 150 2025-12-17 22:41:56,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c76575bc060ce15f5a6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:57,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c60575bc060ce15f577 2025-12-17 22:41:57,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c60575bc060ce15f577 HTTP/1.1" 502 150 2025-12-17 22:41:57,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c60575bc060ce15f577'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:58,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c78575bc060ce15f5ad 2025-12-17 22:41:58,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c78575bc060ce15f5ad HTTP/1.1" 502 150 2025-12-17 22:41:58,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c78575bc060ce15f5ad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:58,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c7a575bc060ce15f5b6 2025-12-17 22:41:58,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7a575bc060ce15f5b6 HTTP/1.1" 502 150 2025-12-17 22:41:58,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c7a575bc060ce15f5b6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:58,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5c575bc060ce15f56e 2025-12-17 22:41:58,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5c575bc060ce15f56e HTTP/1.1" 502 150 2025-12-17 22:41:58,775 - lava_callback - ERROR - Error processing callback for node 69408c5c575bc060ce15f56e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c5c575bc060ce15f56e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c5c575bc060ce15f56e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c5c575bc060ce15f56e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:41:58,777 - lava_callback - INFO - Starting async job submit for node 69408c7d575bc060ce15f5bf 2025-12-17 22:41:58,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:41:58,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7d575bc060ce15f5bf HTTP/1.1" 502 150 2025-12-17 22:41:58,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c7d575bc060ce15f5bf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:58,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c7d575bc060ce15f5bf 2025-12-17 22:41:58,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7d575bc060ce15f5bf HTTP/1.1" 502 150 2025-12-17 22:41:58,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c7d575bc060ce15f5bf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:41:59,149 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c6e575bc060ce15f59a 2025-12-17 22:41:59,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6e575bc060ce15f59a HTTP/1.1" 502 150 2025-12-17 22:41:59,150 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c6e575bc060ce15f59a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:00,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c68575bc060ce15f591 2025-12-17 22:42:00,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c68575bc060ce15f591 HTTP/1.1" 502 150 2025-12-17 22:42:00,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c68575bc060ce15f591'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:00,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6a575bc060ce15f594 HTTP/1.1" 502 150 2025-12-17 22:42:00,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c6a575bc060ce15f594'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:00,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c7d575bc060ce15f5bf 2025-12-17 22:42:00,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7d575bc060ce15f5bf HTTP/1.1" 502 150 2025-12-17 22:42:00,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c7d575bc060ce15f5bf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:01,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c53575bc060ce15f557 2025-12-17 22:42:01,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c53575bc060ce15f557 HTTP/1.1" 502 150 2025-12-17 22:42:01,906 - lava_callback - ERROR - Error processing callback for node 69408c53575bc060ce15f557: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c53575bc060ce15f557 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c53575bc060ce15f557 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c53575bc060ce15f557 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:01,908 - lava_callback - INFO - Starting async job submit for node 69408c7f575bc060ce15f5c6 2025-12-17 22:42:01,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:01,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7f575bc060ce15f5c6 HTTP/1.1" 502 150 2025-12-17 22:42:01,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c7f575bc060ce15f5c6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:01,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c7f575bc060ce15f5c6 2025-12-17 22:42:01,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7f575bc060ce15f5c6 HTTP/1.1" 502 150 2025-12-17 22:42:01,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c7f575bc060ce15f5c6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:02,309 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c70575bc060ce15f59d 2025-12-17 22:42:02,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c70575bc060ce15f59d HTTP/1.1" 502 150 2025-12-17 22:42:02,311 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c70575bc060ce15f59d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59924 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:42:02,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5d575bc060ce15f571 2025-12-17 22:42:02,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5d575bc060ce15f571 HTTP/1.1" 502 150 2025-12-17 22:42:02,990 - lava_callback - ERROR - Error processing callback for node 69408c5d575bc060ce15f571: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c5d575bc060ce15f571 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c5d575bc060ce15f571 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c5d575bc060ce15f571 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:02,992 - lava_callback - INFO - Starting async job submit for node 6940f14b460bc8e2651152a9 2025-12-17 22:42:03,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:03,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940f14b460bc8e2651152a9 HTTP/1.1" 502 150 2025-12-17 22:42:03,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940f14b460bc8e2651152a9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:03,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940f14b460bc8e2651152a9 2025-12-17 22:42:03,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940f14b460bc8e2651152a9 HTTP/1.1" 502 150 2025-12-17 22:42:03,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940f14b460bc8e2651152a9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:03,235 - lava_callback - INFO - Received callback for node 69408ea8575bc060ce15fec1 INFO: 172.21.0.1:59932 - "POST /node/69408ea8575bc060ce15fec1 HTTP/1.0" 202 Accepted 2025-12-17 22:42:03,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c7f575bc060ce15f5c6 2025-12-17 22:42:03,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7f575bc060ce15f5c6 HTTP/1.1" 502 150 2025-12-17 22:42:03,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c7f575bc060ce15f5c6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:04,003 - lava_callback - INFO - Received callback for node 69408ea9575bc060ce15fec8 INFO: 172.21.0.1:59938 - "POST /node/69408ea9575bc060ce15fec8 HTTP/1.0" 202 Accepted 2025-12-17 22:42:04,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c7d575bc060ce15f5bf 2025-12-17 22:42:04,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7d575bc060ce15f5bf HTTP/1.1" 502 150 2025-12-17 22:42:04,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c7d575bc060ce15f5bf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:04,817 - lava_callback - INFO - Received callback for node 69408eaa575bc060ce15fecf INFO: 172.21.0.1:59950 - "POST /node/69408eaa575bc060ce15fecf HTTP/1.0" 202 Accepted 2025-12-17 22:42:04,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c76575bc060ce15f5a6 2025-12-17 22:42:04,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c76575bc060ce15f5a6 HTTP/1.1" 502 150 2025-12-17 22:42:04,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c76575bc060ce15f5a6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:05,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940f14b460bc8e2651152a9 2025-12-17 22:42:05,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940f14b460bc8e2651152a9 HTTP/1.1" 502 150 2025-12-17 22:42:05,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940f14b460bc8e2651152a9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:05,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5f575bc060ce15f574 HTTP/1.1" 502 150 2025-12-17 22:42:05,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c5f575bc060ce15f574'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:05,367 - lava_callback - INFO - Received callback for node 69408eaf575bc060ce15fedb INFO: 172.21.0.1:59952 - "POST /node/69408eaf575bc060ce15fedb HTTP/1.0" 202 Accepted 2025-12-17 22:42:05,543 - lava_callback - INFO - Received callback for node 69408eac575bc060ce15fed2 INFO: 172.21.0.1:59964 - "POST /node/69408eac575bc060ce15fed2 HTTP/1.0" 202 Accepted 2025-12-17 22:42:05,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c60575bc060ce15f577 2025-12-17 22:42:05,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c60575bc060ce15f577 HTTP/1.1" 502 150 2025-12-17 22:42:05,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c60575bc060ce15f577'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:06,025 - lava_callback - INFO - Received callback for node 69408eb0575bc060ce15fede INFO: 172.21.0.1:59970 - "POST /node/69408eb0575bc060ce15fede HTTP/1.0" 202 Accepted 2025-12-17 22:42:06,268 - lava_callback - INFO - Received callback for node 69408ead575bc060ce15fed5 INFO: 172.21.0.1:59984 - "POST /node/69408ead575bc060ce15fed5 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:59990 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:42:06,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c63575bc060ce15f57f 2025-12-17 22:42:06,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c63575bc060ce15f57f HTTP/1.1" 502 150 2025-12-17 22:42:06,574 - lava_callback - ERROR - Error processing callback for node 69408c63575bc060ce15f57f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c63575bc060ce15f57f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c63575bc060ce15f57f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c63575bc060ce15f57f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:06,576 - lava_callback - INFO - Starting async job submit for node 69408c82575bc060ce15f5ed 2025-12-17 22:42:06,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:06,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c82575bc060ce15f5ed HTTP/1.1" 502 150 2025-12-17 22:42:06,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c82575bc060ce15f5ed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:06,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c82575bc060ce15f5ed 2025-12-17 22:42:06,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c82575bc060ce15f5ed HTTP/1.1" 502 150 2025-12-17 22:42:06,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c82575bc060ce15f5ed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:06,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c78575bc060ce15f5ad 2025-12-17 22:42:06,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c78575bc060ce15f5ad HTTP/1.1" 502 150 2025-12-17 22:42:06,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c78575bc060ce15f5ad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:06,675 - lava_callback - INFO - Received callback for node 69408eb2575bc060ce15fee1 INFO: 172.21.0.1:60000 - "POST /node/69408eb2575bc060ce15fee1 HTTP/1.0" 202 Accepted 2025-12-17 22:42:06,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c7a575bc060ce15f5b6 2025-12-17 22:42:06,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7a575bc060ce15f5b6 HTTP/1.1" 502 150 2025-12-17 22:42:06,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c7a575bc060ce15f5b6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:07,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c66575bc060ce15f58c 2025-12-17 22:42:07,036 - lava_callback - INFO - Received callback for node 69408eae575bc060ce15fed8 INFO: 172.21.0.1:60004 - "POST /node/69408eae575bc060ce15fed8 HTTP/1.0" 202 Accepted 2025-12-17 22:42:07,331 - lava_callback - INFO - Received callback for node 69408eb3575bc060ce15ff3d INFO: 172.21.0.1:60018 - "POST /node/69408eb3575bc060ce15ff3d HTTP/1.0" 202 Accepted 2025-12-17 22:42:07,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c7f575bc060ce15f5c6 2025-12-17 22:42:07,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7f575bc060ce15f5c6 HTTP/1.1" 502 150 2025-12-17 22:42:07,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c7f575bc060ce15f5c6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:08,016 - lava_callback - INFO - Received callback for node 69408eb4575bc060ce15ff40 INFO: 172.21.0.1:60024 - "POST /node/69408eb4575bc060ce15ff40 HTTP/1.0" 202 Accepted 2025-12-17 22:42:08,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c68575bc060ce15f591 2025-12-17 22:42:08,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c68575bc060ce15f591 HTTP/1.1" 502 150 2025-12-17 22:42:08,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c68575bc060ce15f591'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:08,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c6a575bc060ce15f594 2025-12-17 22:42:08,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6a575bc060ce15f594 HTTP/1.1" 502 150 2025-12-17 22:42:08,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c6a575bc060ce15f594'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:08,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c82575bc060ce15f5ed 2025-12-17 22:42:08,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c82575bc060ce15f5ed HTTP/1.1" 502 150 2025-12-17 22:42:08,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c82575bc060ce15f5ed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:08,675 - lava_callback - INFO - Received callback for node 69408eb7575bc060ce15ff46 INFO: 172.21.0.1:60026 - "POST /node/69408eb7575bc060ce15ff46 HTTP/1.0" 202 Accepted 2025-12-17 22:42:09,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940f14b460bc8e2651152a9 2025-12-17 22:42:09,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940f14b460bc8e2651152a9 HTTP/1.1" 502 150 2025-12-17 22:42:09,054 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940f14b460bc8e2651152a9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:09,316 - lava_callback - INFO - Received callback for node 69408eb9575bc060ce15ff49 INFO: 172.21.0.1:60032 - "POST /node/69408eb9575bc060ce15ff49 HTTP/1.0" 202 Accepted 2025-12-17 22:42:09,958 - lava_callback - INFO - Received callback for node 69408ebb575bc060ce15ff4c INFO: 172.21.0.1:41100 - "POST /node/69408ebb575bc060ce15ff4c HTTP/1.0" 202 Accepted 2025-12-17 22:42:11,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c74575bc060ce15f5a3 HTTP/1.1" 502 150 2025-12-17 22:42:11,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c74575bc060ce15f5a3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:11,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c6c575bc060ce15f597 2025-12-17 22:42:11,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6c575bc060ce15f597 HTTP/1.1" 502 150 2025-12-17 22:42:11,511 - lava_callback - ERROR - Error processing callback for node 69408c6c575bc060ce15f597: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c6c575bc060ce15f597 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c6c575bc060ce15f597 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c6c575bc060ce15f597 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:11,513 - lava_callback - INFO - Starting async job submit for node 69408c84575bc060ce15f5f0 2025-12-17 22:42:11,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:11,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c84575bc060ce15f5f0 HTTP/1.1" 502 150 2025-12-17 22:42:11,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c84575bc060ce15f5f0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:11,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c84575bc060ce15f5f0 2025-12-17 22:42:11,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c84575bc060ce15f5f0 HTTP/1.1" 502 150 2025-12-17 22:42:11,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c84575bc060ce15f5f0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:12,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c82575bc060ce15f5ed 2025-12-17 22:42:12,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c82575bc060ce15f5ed HTTP/1.1" 502 150 2025-12-17 22:42:12,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c82575bc060ce15f5ed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:12,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c7d575bc060ce15f5bf 2025-12-17 22:42:12,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7d575bc060ce15f5bf HTTP/1.1" 502 150 2025-12-17 22:42:12,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c7d575bc060ce15f5bf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:13,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c84575bc060ce15f5f0 2025-12-17 22:42:13,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c84575bc060ce15f5f0 HTTP/1.1" 502 150 2025-12-17 22:42:13,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c84575bc060ce15f5f0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:15,150 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c6e575bc060ce15f59a 2025-12-17 22:42:15,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6e575bc060ce15f59a HTTP/1.1" 502 150 2025-12-17 22:42:15,151 - lava_callback - ERROR - Error processing callback for node 69408c6e575bc060ce15f59a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c6e575bc060ce15f59a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c6e575bc060ce15f59a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c6e575bc060ce15f59a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:15,153 - lava_callback - INFO - Starting async job submit for node 69408c86575bc060ce15f5f3 2025-12-17 22:42:15,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:15,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c86575bc060ce15f5f3 HTTP/1.1" 502 150 2025-12-17 22:42:15,160 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c86575bc060ce15f5f3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:15,160 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c86575bc060ce15f5f3 2025-12-17 22:42:15,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c86575bc060ce15f5f3 HTTP/1.1" 502 150 2025-12-17 22:42:15,161 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c86575bc060ce15f5f3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41108 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:42:15,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c7f575bc060ce15f5c6 2025-12-17 22:42:15,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7f575bc060ce15f5c6 HTTP/1.1" 502 150 2025-12-17 22:42:15,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c7f575bc060ce15f5c6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41112 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:42:17,054 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940f14b460bc8e2651152a9 2025-12-17 22:42:17,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940f14b460bc8e2651152a9 HTTP/1.1" 502 150 2025-12-17 22:42:17,055 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940f14b460bc8e2651152a9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:17,161 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c86575bc060ce15f5f3 2025-12-17 22:42:17,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c86575bc060ce15f5f3 HTTP/1.1" 502 150 2025-12-17 22:42:17,162 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c86575bc060ce15f5f3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:17,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c84575bc060ce15f5f0 2025-12-17 22:42:17,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c84575bc060ce15f5f0 HTTP/1.1" 502 150 2025-12-17 22:42:17,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c84575bc060ce15f5f0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:18,311 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c70575bc060ce15f59d 2025-12-17 22:42:19,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c74575bc060ce15f5a3 2025-12-17 22:42:19,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c74575bc060ce15f5a3 HTTP/1.1" 502 150 2025-12-17 22:42:19,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c74575bc060ce15f5a3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:20,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c82575bc060ce15f5ed 2025-12-17 22:42:20,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c82575bc060ce15f5ed HTTP/1.1" 502 150 2025-12-17 22:42:20,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c82575bc060ce15f5ed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:20,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c76575bc060ce15f5a6 2025-12-17 22:42:20,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c76575bc060ce15f5a6 HTTP/1.1" 502 150 2025-12-17 22:42:20,902 - lava_callback - ERROR - Error processing callback for node 69408c76575bc060ce15f5a6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c76575bc060ce15f5a6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c76575bc060ce15f5a6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c76575bc060ce15f5a6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:20,904 - lava_callback - INFO - Starting async job submit for node 69408c8b575bc060ce15f5fc 2025-12-17 22:42:20,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:20,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8b575bc060ce15f5fc HTTP/1.1" 502 150 2025-12-17 22:42:20,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8b575bc060ce15f5fc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:20,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8b575bc060ce15f5fc 2025-12-17 22:42:20,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8b575bc060ce15f5fc HTTP/1.1" 502 150 2025-12-17 22:42:20,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8b575bc060ce15f5fc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:21,163 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c86575bc060ce15f5f3 2025-12-17 22:42:21,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c86575bc060ce15f5f3 HTTP/1.1" 502 150 2025-12-17 22:42:21,164 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c86575bc060ce15f5f3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:21,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c5f575bc060ce15f574 2025-12-17 22:42:21,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c60575bc060ce15f577 2025-12-17 22:42:21,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c60575bc060ce15f577 HTTP/1.1" 502 150 2025-12-17 22:42:21,568 - lava_callback - ERROR - Error processing callback for node 69408c60575bc060ce15f577: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c60575bc060ce15f577 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c60575bc060ce15f577 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c60575bc060ce15f577 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:21,570 - lava_callback - INFO - Starting async job submit for node 69408c8c575bc060ce15f601 2025-12-17 22:42:21,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:21,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8c575bc060ce15f601 HTTP/1.1" 502 150 2025-12-17 22:42:21,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8c575bc060ce15f601'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:21,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8c575bc060ce15f601 2025-12-17 22:42:21,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8c575bc060ce15f601 HTTP/1.1" 502 150 2025-12-17 22:42:21,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8c575bc060ce15f601'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:22,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c78575bc060ce15f5ad 2025-12-17 22:42:22,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c78575bc060ce15f5ad HTTP/1.1" 502 150 2025-12-17 22:42:22,674 - lava_callback - ERROR - Error processing callback for node 69408c78575bc060ce15f5ad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c78575bc060ce15f5ad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c78575bc060ce15f5ad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c78575bc060ce15f5ad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:22,676 - lava_callback - INFO - Starting async job submit for node 69408c8e575bc060ce15f608 2025-12-17 22:42:22,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:22,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8e575bc060ce15f608 HTTP/1.1" 502 150 2025-12-17 22:42:22,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8e575bc060ce15f608'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:22,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8e575bc060ce15f608 2025-12-17 22:42:22,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8e575bc060ce15f608 HTTP/1.1" 502 150 2025-12-17 22:42:22,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8e575bc060ce15f608'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:22,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c7a575bc060ce15f5b6 2025-12-17 22:42:22,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7a575bc060ce15f5b6 HTTP/1.1" 502 150 2025-12-17 22:42:22,702 - lava_callback - ERROR - Error processing callback for node 69408c7a575bc060ce15f5b6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c7a575bc060ce15f5b6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c7a575bc060ce15f5b6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c7a575bc060ce15f5b6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:22,704 - lava_callback - INFO - Starting async job submit for node 69408c99575bc060ce15f62a 2025-12-17 22:42:22,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:22,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c99575bc060ce15f62a HTTP/1.1" 502 150 2025-12-17 22:42:22,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c99575bc060ce15f62a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:22,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c99575bc060ce15f62a 2025-12-17 22:42:22,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c99575bc060ce15f62a HTTP/1.1" 502 150 2025-12-17 22:42:22,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c99575bc060ce15f62a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:22,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8b575bc060ce15f5fc 2025-12-17 22:42:22,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8b575bc060ce15f5fc HTTP/1.1" 502 150 2025-12-17 22:42:22,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8b575bc060ce15f5fc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:23,394 - lava_callback - INFO - Received callback for node 69408ec9575bc060ce15ff6f INFO: 172.21.0.1:48566 - "POST /node/69408ec9575bc060ce15ff6f HTTP/1.0" 202 Accepted 2025-12-17 22:42:23,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8c575bc060ce15f601 2025-12-17 22:42:23,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8c575bc060ce15f601 HTTP/1.1" 502 150 2025-12-17 22:42:23,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8c575bc060ce15f601'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:24,259 - lava_callback - INFO - Received callback for node 69408ecd575bc060ce15ff75 INFO: 172.21.0.1:48580 - "POST /node/69408ecd575bc060ce15ff75 HTTP/1.0" 202 Accepted 2025-12-17 22:42:24,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c68575bc060ce15f591 2025-12-17 22:42:24,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c68575bc060ce15f591 HTTP/1.1" 502 150 2025-12-17 22:42:24,404 - lava_callback - ERROR - Error processing callback for node 69408c68575bc060ce15f591: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c68575bc060ce15f591 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c68575bc060ce15f591 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c68575bc060ce15f591 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:24,406 - lava_callback - INFO - Starting async job submit for node 69408c8f575bc060ce15f610 2025-12-17 22:42:24,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:24,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8f575bc060ce15f610 HTTP/1.1" 502 150 2025-12-17 22:42:24,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8f575bc060ce15f610'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:24,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8f575bc060ce15f610 2025-12-17 22:42:24,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8f575bc060ce15f610 HTTP/1.1" 502 150 2025-12-17 22:42:24,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8f575bc060ce15f610'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:24,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c6a575bc060ce15f594 2025-12-17 22:42:24,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c6a575bc060ce15f594 HTTP/1.1" 502 150 2025-12-17 22:42:24,499 - lava_callback - ERROR - Error processing callback for node 69408c6a575bc060ce15f594: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c6a575bc060ce15f594 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c6a575bc060ce15f594 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c6a575bc060ce15f594 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:24,502 - lava_callback - INFO - Starting async job submit for node 69408c9b575bc060ce15f62d 2025-12-17 22:42:24,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:24,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9b575bc060ce15f62d HTTP/1.1" 502 150 2025-12-17 22:42:24,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c9b575bc060ce15f62d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:24,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c9b575bc060ce15f62d 2025-12-17 22:42:24,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9b575bc060ce15f62d HTTP/1.1" 502 150 2025-12-17 22:42:24,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c9b575bc060ce15f62d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:24,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8e575bc060ce15f608 2025-12-17 22:42:24,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8e575bc060ce15f608 HTTP/1.1" 502 150 2025-12-17 22:42:24,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8e575bc060ce15f608'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:24,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c99575bc060ce15f62a 2025-12-17 22:42:24,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c99575bc060ce15f62a HTTP/1.1" 502 150 2025-12-17 22:42:24,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c99575bc060ce15f62a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:25,366 - lava_callback - INFO - Received callback for node 69408ebc575bc060ce15ff51 INFO: 172.21.0.1:48584 - "POST /node/69408ebc575bc060ce15ff51 HTTP/1.0" 202 Accepted 2025-12-17 22:42:25,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c84575bc060ce15f5f0 2025-12-17 22:42:25,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c84575bc060ce15f5f0 HTTP/1.1" 502 150 2025-12-17 22:42:25,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c84575bc060ce15f5f0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:26,008 - lava_callback - INFO - Received callback for node 69408ebe575bc060ce15ff5a INFO: 172.21.0.1:48588 - "POST /node/69408ebe575bc060ce15ff5a HTTP/1.0" 202 Accepted 2025-12-17 22:42:26,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8f575bc060ce15f610 2025-12-17 22:42:26,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8f575bc060ce15f610 HTTP/1.1" 502 150 2025-12-17 22:42:26,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8f575bc060ce15f610'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48602 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:42:26,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c9b575bc060ce15f62d 2025-12-17 22:42:26,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9b575bc060ce15f62d HTTP/1.1" 502 150 2025-12-17 22:42:26,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c9b575bc060ce15f62d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:26,646 - lava_callback - INFO - Received callback for node 69408ebf575bc060ce15ff60 INFO: 172.21.0.1:48618 - "POST /node/69408ebf575bc060ce15ff60 HTTP/1.0" 202 Accepted 2025-12-17 22:42:26,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8b575bc060ce15f5fc 2025-12-17 22:42:26,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8b575bc060ce15f5fc HTTP/1.1" 502 150 2025-12-17 22:42:26,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8b575bc060ce15f5fc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:27,287 - lava_callback - INFO - Received callback for node 69408ec1575bc060ce15ff63 INFO: 172.21.0.1:48622 - "POST /node/69408ec1575bc060ce15ff63 HTTP/1.0" 202 Accepted 2025-12-17 22:42:27,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8c575bc060ce15f601 2025-12-17 22:42:27,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8c575bc060ce15f601 HTTP/1.1" 502 150 2025-12-17 22:42:27,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8c575bc060ce15f601'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:27,952 - lava_callback - INFO - Received callback for node 69408ec4575bc060ce15ff66 INFO: 172.21.0.1:48632 - "POST /node/69408ec4575bc060ce15ff66 HTTP/1.0" 202 Accepted 2025-12-17 22:42:28,594 - lava_callback - INFO - Received callback for node 69408ec6575bc060ce15ff69 INFO: 172.21.0.1:48638 - "POST /node/69408ec6575bc060ce15ff69 HTTP/1.0" 202 Accepted 2025-12-17 22:42:28,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8e575bc060ce15f608 2025-12-17 22:42:28,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8e575bc060ce15f608 HTTP/1.1" 502 150 2025-12-17 22:42:28,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8e575bc060ce15f608'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:28,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c99575bc060ce15f62a 2025-12-17 22:42:28,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c99575bc060ce15f62a HTTP/1.1" 502 150 2025-12-17 22:42:28,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c99575bc060ce15f62a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:28,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c7d575bc060ce15f5bf 2025-12-17 22:42:28,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7d575bc060ce15f5bf HTTP/1.1" 502 150 2025-12-17 22:42:28,790 - lava_callback - ERROR - Error processing callback for node 69408c7d575bc060ce15f5bf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c7d575bc060ce15f5bf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c7d575bc060ce15f5bf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c7d575bc060ce15f5bf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:28,792 - lava_callback - INFO - Starting async job submit for node 69408c91575bc060ce15f617 2025-12-17 22:42:28,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:28,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c91575bc060ce15f617 HTTP/1.1" 502 150 2025-12-17 22:42:28,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c91575bc060ce15f617'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:28,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c91575bc060ce15f617 2025-12-17 22:42:28,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c91575bc060ce15f617 HTTP/1.1" 502 150 2025-12-17 22:42:28,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c91575bc060ce15f617'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:29,164 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c86575bc060ce15f5f3 2025-12-17 22:42:29,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c86575bc060ce15f5f3 HTTP/1.1" 502 150 2025-12-17 22:42:29,165 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c86575bc060ce15f5f3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:29,236 - lava_callback - INFO - Received callback for node 69408ec8575bc060ce15ff6c INFO: 172.21.0.1:48652 - "POST /node/69408ec8575bc060ce15ff6c HTTP/1.0" 202 Accepted 2025-12-17 22:42:29,890 - lava_callback - INFO - Received callback for node 69408ecb575bc060ce15ff72 INFO: 172.21.0.1:56086 - "POST /node/69408ecb575bc060ce15ff72 HTTP/1.0" 202 Accepted 2025-12-17 22:42:30,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8f575bc060ce15f610 2025-12-17 22:42:30,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8f575bc060ce15f610 HTTP/1.1" 502 150 2025-12-17 22:42:30,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8f575bc060ce15f610'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:30,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c9b575bc060ce15f62d 2025-12-17 22:42:30,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9b575bc060ce15f62d HTTP/1.1" 502 150 2025-12-17 22:42:30,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c9b575bc060ce15f62d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:30,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c91575bc060ce15f617 2025-12-17 22:42:30,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c91575bc060ce15f617 HTTP/1.1" 502 150 2025-12-17 22:42:30,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c91575bc060ce15f617'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:31,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c7f575bc060ce15f5c6 2025-12-17 22:42:31,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c7f575bc060ce15f5c6 HTTP/1.1" 502 150 2025-12-17 22:42:31,921 - lava_callback - ERROR - Error processing callback for node 69408c7f575bc060ce15f5c6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c7f575bc060ce15f5c6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c7f575bc060ce15f5c6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c7f575bc060ce15f5c6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:31,922 - lava_callback - INFO - Starting async job submit for node 69408c9d575bc060ce15f630 2025-12-17 22:42:31,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:31,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9d575bc060ce15f630 HTTP/1.1" 502 150 2025-12-17 22:42:31,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c9d575bc060ce15f630'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:31,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c9d575bc060ce15f630 2025-12-17 22:42:31,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9d575bc060ce15f630 HTTP/1.1" 502 150 2025-12-17 22:42:31,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c9d575bc060ce15f630'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:33,055 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940f14b460bc8e2651152a9 2025-12-17 22:42:33,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c70575bc060ce15f59d HTTP/1.1" 502 150 2025-12-17 22:42:33,777 - lava_callback - ERROR - Error processing callback for node 69408c70575bc060ce15f59d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c70575bc060ce15f59d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c70575bc060ce15f59d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c70575bc060ce15f59d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:33,779 - lava_callback - INFO - Starting async job submit for node 69408c95575bc060ce15f621 2025-12-17 22:42:33,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:33,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c95575bc060ce15f621 HTTP/1.1" 502 150 2025-12-17 22:42:33,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c95575bc060ce15f621'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:33,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c95575bc060ce15f621 2025-12-17 22:42:33,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c95575bc060ce15f621 HTTP/1.1" 502 150 2025-12-17 22:42:33,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c95575bc060ce15f621'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:33,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c9d575bc060ce15f630 2025-12-17 22:42:33,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9d575bc060ce15f630 HTTP/1.1" 502 150 2025-12-17 22:42:33,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c9d575bc060ce15f630'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:34,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c91575bc060ce15f617 2025-12-17 22:42:34,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c91575bc060ce15f617 HTTP/1.1" 502 150 2025-12-17 22:42:34,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c91575bc060ce15f617'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:34,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8b575bc060ce15f5fc 2025-12-17 22:42:34,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8b575bc060ce15f5fc HTTP/1.1" 502 150 2025-12-17 22:42:34,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8b575bc060ce15f5fc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:35,165 - lava_callback - INFO - Received callback for node 69408ece575bc060ce15ff78 INFO: 172.21.0.1:56100 - "POST /node/69408ece575bc060ce15ff78 HTTP/1.0" 202 Accepted 2025-12-17 22:42:35,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c74575bc060ce15f5a3 2025-12-17 22:42:35,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c74575bc060ce15f5a3 HTTP/1.1" 502 150 2025-12-17 22:42:35,508 - lava_callback - ERROR - Error processing callback for node 69408c74575bc060ce15f5a3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c74575bc060ce15f5a3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c74575bc060ce15f5a3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c74575bc060ce15f5a3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:35,510 - lava_callback - INFO - Starting async job submit for node 69408c9f575bc060ce15f635 2025-12-17 22:42:35,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:35,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9f575bc060ce15f635 HTTP/1.1" 502 150 2025-12-17 22:42:35,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c9f575bc060ce15f635'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:35,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c9f575bc060ce15f635 2025-12-17 22:42:35,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9f575bc060ce15f635 HTTP/1.1" 502 150 2025-12-17 22:42:35,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c9f575bc060ce15f635'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:35,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8c575bc060ce15f601 2025-12-17 22:42:35,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8c575bc060ce15f601 HTTP/1.1" 502 150 2025-12-17 22:42:35,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8c575bc060ce15f601'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:35,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c95575bc060ce15f621 2025-12-17 22:42:35,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c95575bc060ce15f621 HTTP/1.1" 502 150 2025-12-17 22:42:35,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c95575bc060ce15f621'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:35,844 - lava_callback - INFO - Received callback for node 69408ed0575bc060ce15ff7d INFO: 172.21.0.1:56104 - "POST /node/69408ed0575bc060ce15ff7d HTTP/1.0" 202 Accepted 2025-12-17 22:42:36,495 - lava_callback - INFO - Received callback for node 69408ed1575bc060ce15ff84 INFO: 172.21.0.1:56106 - "POST /node/69408ed1575bc060ce15ff84 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:56122 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:42:36,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c82575bc060ce15f5ed 2025-12-17 22:42:36,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c82575bc060ce15f5ed HTTP/1.1" 502 150 2025-12-17 22:42:36,601 - lava_callback - ERROR - Error processing callback for node 69408c82575bc060ce15f5ed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c82575bc060ce15f5ed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c82575bc060ce15f5ed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c82575bc060ce15f5ed (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:36,603 - lava_callback - INFO - Starting async job submit for node 69408c96575bc060ce15f624 2025-12-17 22:42:36,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:36,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c96575bc060ce15f624 HTTP/1.1" 502 150 2025-12-17 22:42:36,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c96575bc060ce15f624'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:36,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c96575bc060ce15f624 2025-12-17 22:42:36,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c96575bc060ce15f624 HTTP/1.1" 502 150 2025-12-17 22:42:36,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c96575bc060ce15f624'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:36,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8e575bc060ce15f608 2025-12-17 22:42:36,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8e575bc060ce15f608 HTTP/1.1" 502 150 2025-12-17 22:42:36,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8e575bc060ce15f608'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:36,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c99575bc060ce15f62a 2025-12-17 22:42:36,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c99575bc060ce15f62a HTTP/1.1" 502 150 2025-12-17 22:42:36,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c99575bc060ce15f62a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:36,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c5f575bc060ce15f574 HTTP/1.1" 502 150 2025-12-17 22:42:36,848 - lava_callback - ERROR - Error processing callback for node 69408c5f575bc060ce15f574: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c5f575bc060ce15f574 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c5f575bc060ce15f574 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c5f575bc060ce15f574 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:36,850 - lava_callback - INFO - Starting async job submit for node 69408ca1575bc060ce15f63e 2025-12-17 22:42:36,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:36,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca1575bc060ce15f63e HTTP/1.1" 502 150 2025-12-17 22:42:36,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca1575bc060ce15f63e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:36,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca1575bc060ce15f63e 2025-12-17 22:42:36,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca1575bc060ce15f63e HTTP/1.1" 502 150 2025-12-17 22:42:36,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca1575bc060ce15f63e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:37,171 - lava_callback - INFO - Received callback for node 69408ed3575bc060ce15ff8b INFO: 172.21.0.1:56126 - "POST /node/69408ed3575bc060ce15ff8b HTTP/1.0" 202 Accepted 2025-12-17 22:42:37,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c9f575bc060ce15f635 2025-12-17 22:42:37,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9f575bc060ce15f635 HTTP/1.1" 502 150 2025-12-17 22:42:37,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c9f575bc060ce15f635'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:37,833 - lava_callback - INFO - Received callback for node 69408ed6575bc060ce15ff95 INFO: 172.21.0.1:56130 - "POST /node/69408ed6575bc060ce15ff95 HTTP/1.0" 202 Accepted 2025-12-17 22:42:37,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c66575bc060ce15f58c HTTP/1.1" 502 150 2025-12-17 22:42:37,872 - lava_callback - ERROR - Error processing callback for node 69408c66575bc060ce15f58c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c66575bc060ce15f58c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c66575bc060ce15f58c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c66575bc060ce15f58c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:37,874 - lava_callback - INFO - Starting async job submit for node 69408ca3575bc060ce15f645 2025-12-17 22:42:37,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:37,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca3575bc060ce15f645 HTTP/1.1" 502 150 2025-12-17 22:42:37,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca3575bc060ce15f645'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:37,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca3575bc060ce15f645 2025-12-17 22:42:37,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca3575bc060ce15f645 HTTP/1.1" 502 150 2025-12-17 22:42:37,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca3575bc060ce15f645'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:37,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c9d575bc060ce15f630 2025-12-17 22:42:37,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9d575bc060ce15f630 HTTP/1.1" 502 150 2025-12-17 22:42:37,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c9d575bc060ce15f630'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:38,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8f575bc060ce15f610 2025-12-17 22:42:38,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8f575bc060ce15f610 HTTP/1.1" 502 150 2025-12-17 22:42:38,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8f575bc060ce15f610'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:38,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c9b575bc060ce15f62d 2025-12-17 22:42:38,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9b575bc060ce15f62d HTTP/1.1" 502 150 2025-12-17 22:42:38,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c9b575bc060ce15f62d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:38,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c96575bc060ce15f624 2025-12-17 22:42:38,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c96575bc060ce15f624 HTTP/1.1" 502 150 2025-12-17 22:42:38,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c96575bc060ce15f624'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:38,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca1575bc060ce15f63e 2025-12-17 22:42:38,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca1575bc060ce15f63e HTTP/1.1" 502 150 2025-12-17 22:42:38,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca1575bc060ce15f63e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:39,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c95575bc060ce15f621 2025-12-17 22:42:39,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c95575bc060ce15f621 HTTP/1.1" 502 150 2025-12-17 22:42:39,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c95575bc060ce15f621'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:39,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca3575bc060ce15f645 2025-12-17 22:42:39,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca3575bc060ce15f645 HTTP/1.1" 502 150 2025-12-17 22:42:39,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca3575bc060ce15f645'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:41,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c9f575bc060ce15f635 2025-12-17 22:42:41,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9f575bc060ce15f635 HTTP/1.1" 502 150 2025-12-17 22:42:41,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c9f575bc060ce15f635'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:41,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c84575bc060ce15f5f0 2025-12-17 22:42:41,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c84575bc060ce15f5f0 HTTP/1.1" 502 150 2025-12-17 22:42:41,535 - lava_callback - ERROR - Error processing callback for node 69408c84575bc060ce15f5f0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c84575bc060ce15f5f0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c84575bc060ce15f5f0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c84575bc060ce15f5f0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:41,536 - lava_callback - INFO - Starting async job submit for node 69408ca5575bc060ce15f648 2025-12-17 22:42:41,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:41,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca5575bc060ce15f648 HTTP/1.1" 502 150 2025-12-17 22:42:41,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca5575bc060ce15f648'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:41,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca5575bc060ce15f648 2025-12-17 22:42:41,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca5575bc060ce15f648 HTTP/1.1" 502 150 2025-12-17 22:42:41,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca5575bc060ce15f648'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:42,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c96575bc060ce15f624 2025-12-17 22:42:42,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c96575bc060ce15f624 HTTP/1.1" 502 150 2025-12-17 22:42:42,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c96575bc060ce15f624'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:42,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c91575bc060ce15f617 2025-12-17 22:42:42,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c91575bc060ce15f617 HTTP/1.1" 502 150 2025-12-17 22:42:42,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c91575bc060ce15f617'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:42,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca1575bc060ce15f63e 2025-12-17 22:42:42,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca1575bc060ce15f63e HTTP/1.1" 502 150 2025-12-17 22:42:42,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca1575bc060ce15f63e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:43,185 - lava_callback - INFO - Received callback for node 69408eda575bc060ce15ff9e INFO: 172.21.0.1:45686 - "POST /node/69408eda575bc060ce15ff9e HTTP/1.0" 202 Accepted 2025-12-17 22:42:43,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca5575bc060ce15f648 2025-12-17 22:42:43,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca5575bc060ce15f648 HTTP/1.1" 502 150 2025-12-17 22:42:43,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca5575bc060ce15f648'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:43,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca3575bc060ce15f645 2025-12-17 22:42:43,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca3575bc060ce15f645 HTTP/1.1" 502 150 2025-12-17 22:42:43,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca3575bc060ce15f645'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:43,913 - lava_callback - INFO - Received callback for node 69408ede575bc060ce15ffa7 INFO: 172.21.0.1:45688 - "POST /node/69408ede575bc060ce15ffa7 HTTP/1.0" 202 Accepted 2025-12-17 22:42:45,166 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c86575bc060ce15f5f3 2025-12-17 22:42:45,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c86575bc060ce15f5f3 HTTP/1.1" 502 150 2025-12-17 22:42:45,167 - lava_callback - ERROR - Error processing callback for node 69408c86575bc060ce15f5f3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c86575bc060ce15f5f3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c86575bc060ce15f5f3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c86575bc060ce15f5f3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:45,169 - lava_callback - INFO - Starting async job submit for node 69408ca7575bc060ce15f64b 2025-12-17 22:42:45,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:45,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca7575bc060ce15f64b HTTP/1.1" 502 150 2025-12-17 22:42:45,175 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca7575bc060ce15f64b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:45,175 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca7575bc060ce15f64b 2025-12-17 22:42:45,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca7575bc060ce15f64b HTTP/1.1" 502 150 2025-12-17 22:42:45,176 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca7575bc060ce15f64b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:45,367 - lava_callback - INFO - Received callback for node 69408ed8575bc060ce15ff9b INFO: 172.21.0.1:45690 - "POST /node/69408ed8575bc060ce15ff9b HTTP/1.0" 202 Accepted 2025-12-17 22:42:45,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c9d575bc060ce15f630 2025-12-17 22:42:45,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9d575bc060ce15f630 HTTP/1.1" 502 150 2025-12-17 22:42:45,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c9d575bc060ce15f630'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45692 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:42:47,176 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca7575bc060ce15f64b 2025-12-17 22:42:47,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca7575bc060ce15f64b HTTP/1.1" 502 150 2025-12-17 22:42:47,177 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca7575bc060ce15f64b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:47,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca5575bc060ce15f648 2025-12-17 22:42:47,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca5575bc060ce15f648 HTTP/1.1" 502 150 2025-12-17 22:42:47,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca5575bc060ce15f648'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:47,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c95575bc060ce15f621 2025-12-17 22:42:47,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c95575bc060ce15f621 HTTP/1.1" 502 150 2025-12-17 22:42:47,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c95575bc060ce15f621'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:49,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c9f575bc060ce15f635 2025-12-17 22:42:49,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9f575bc060ce15f635 HTTP/1.1" 502 150 2025-12-17 22:42:49,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c9f575bc060ce15f635'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:50,167 - lava_callback - INFO - Received callback for node 6940946f575bc060ce16103c INFO: 172.21.0.1:37878 - "POST /node/6940946f575bc060ce16103c HTTP/1.0" 202 Accepted 2025-12-17 22:42:50,246 - lava_callback - INFO - Received callback for node 69408ed7575bc060ce15ff98 INFO: 172.21.0.1:37880 - "POST /node/69408ed7575bc060ce15ff98 HTTP/1.0" 202 Accepted 2025-12-17 22:42:50,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c96575bc060ce15f624 2025-12-17 22:42:50,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c96575bc060ce15f624 HTTP/1.1" 502 150 2025-12-17 22:42:50,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c96575bc060ce15f624'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:50,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca1575bc060ce15f63e 2025-12-17 22:42:50,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca1575bc060ce15f63e HTTP/1.1" 502 150 2025-12-17 22:42:50,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca1575bc060ce15f63e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:50,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8b575bc060ce15f5fc 2025-12-17 22:42:50,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8b575bc060ce15f5fc HTTP/1.1" 502 150 2025-12-17 22:42:50,919 - lava_callback - ERROR - Error processing callback for node 69408c8b575bc060ce15f5fc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c8b575bc060ce15f5fc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c8b575bc060ce15f5fc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c8b575bc060ce15f5fc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:50,921 - lava_callback - INFO - Starting async job submit for node 69408ca9575bc060ce15f64e 2025-12-17 22:42:50,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:50,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca9575bc060ce15f64e HTTP/1.1" 502 150 2025-12-17 22:42:50,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca9575bc060ce15f64e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:50,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca9575bc060ce15f64e 2025-12-17 22:42:50,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca9575bc060ce15f64e HTTP/1.1" 502 150 2025-12-17 22:42:50,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca9575bc060ce15f64e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:51,177 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca7575bc060ce15f64b 2025-12-17 22:42:51,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca7575bc060ce15f64b HTTP/1.1" 502 150 2025-12-17 22:42:51,179 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca7575bc060ce15f64b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:51,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8c575bc060ce15f601 2025-12-17 22:42:51,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8c575bc060ce15f601 HTTP/1.1" 502 150 2025-12-17 22:42:51,595 - lava_callback - ERROR - Error processing callback for node 69408c8c575bc060ce15f601: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c8c575bc060ce15f601 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c8c575bc060ce15f601 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c8c575bc060ce15f601 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:51,597 - lava_callback - INFO - Starting async job submit for node 69408cab575bc060ce15f651 2025-12-17 22:42:51,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:51,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cab575bc060ce15f651 HTTP/1.1" 502 150 2025-12-17 22:42:51,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cab575bc060ce15f651'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:51,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cab575bc060ce15f651 2025-12-17 22:42:51,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cab575bc060ce15f651 HTTP/1.1" 502 150 2025-12-17 22:42:51,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cab575bc060ce15f651'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:51,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca3575bc060ce15f645 2025-12-17 22:42:51,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca3575bc060ce15f645 HTTP/1.1" 502 150 2025-12-17 22:42:51,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca3575bc060ce15f645'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:52,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8e575bc060ce15f608 2025-12-17 22:42:52,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8e575bc060ce15f608 HTTP/1.1" 502 150 2025-12-17 22:42:52,692 - lava_callback - ERROR - Error processing callback for node 69408c8e575bc060ce15f608: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c8e575bc060ce15f608 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c8e575bc060ce15f608 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c8e575bc060ce15f608 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:52,693 - lava_callback - INFO - Starting async job submit for node 69408cad575bc060ce15f654 2025-12-17 22:42:52,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:52,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cad575bc060ce15f654 HTTP/1.1" 502 150 2025-12-17 22:42:52,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cad575bc060ce15f654'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:52,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cad575bc060ce15f654 2025-12-17 22:42:52,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cad575bc060ce15f654 HTTP/1.1" 502 150 2025-12-17 22:42:52,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cad575bc060ce15f654'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:52,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c99575bc060ce15f62a 2025-12-17 22:42:52,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c99575bc060ce15f62a HTTP/1.1" 502 150 2025-12-17 22:42:52,715 - lava_callback - ERROR - Error processing callback for node 69408c99575bc060ce15f62a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c99575bc060ce15f62a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c99575bc060ce15f62a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c99575bc060ce15f62a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:52,717 - lava_callback - INFO - Starting async job submit for node 69408caf575bc060ce15f657 2025-12-17 22:42:52,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:52,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408caf575bc060ce15f657 HTTP/1.1" 502 150 2025-12-17 22:42:52,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408caf575bc060ce15f657'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:52,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408caf575bc060ce15f657 2025-12-17 22:42:52,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408caf575bc060ce15f657 HTTP/1.1" 502 150 2025-12-17 22:42:52,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408caf575bc060ce15f657'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:52,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca9575bc060ce15f64e 2025-12-17 22:42:52,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca9575bc060ce15f64e HTTP/1.1" 502 150 2025-12-17 22:42:52,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca9575bc060ce15f64e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:53,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cab575bc060ce15f651 2025-12-17 22:42:53,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cab575bc060ce15f651 HTTP/1.1" 502 150 2025-12-17 22:42:53,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cab575bc060ce15f651'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:54,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8f575bc060ce15f610 2025-12-17 22:42:54,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8f575bc060ce15f610 HTTP/1.1" 502 150 2025-12-17 22:42:54,420 - lava_callback - ERROR - Error processing callback for node 69408c8f575bc060ce15f610: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c8f575bc060ce15f610 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c8f575bc060ce15f610 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c8f575bc060ce15f610 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:54,422 - lava_callback - INFO - Starting async job submit for node 69408cb1575bc060ce15f65a 2025-12-17 22:42:54,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:54,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb1575bc060ce15f65a HTTP/1.1" 502 150 2025-12-17 22:42:54,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb1575bc060ce15f65a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:54,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb1575bc060ce15f65a 2025-12-17 22:42:54,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb1575bc060ce15f65a HTTP/1.1" 502 150 2025-12-17 22:42:54,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb1575bc060ce15f65a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:54,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c9b575bc060ce15f62d 2025-12-17 22:42:54,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9b575bc060ce15f62d HTTP/1.1" 502 150 2025-12-17 22:42:54,520 - lava_callback - ERROR - Error processing callback for node 69408c9b575bc060ce15f62d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c9b575bc060ce15f62d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c9b575bc060ce15f62d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c9b575bc060ce15f62d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:54,522 - lava_callback - INFO - Starting async job submit for node 69408cb3575bc060ce15f65f 2025-12-17 22:42:54,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:54,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb3575bc060ce15f65f HTTP/1.1" 502 150 2025-12-17 22:42:54,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb3575bc060ce15f65f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:54,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb3575bc060ce15f65f 2025-12-17 22:42:54,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb3575bc060ce15f65f HTTP/1.1" 502 150 2025-12-17 22:42:54,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb3575bc060ce15f65f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:54,558 - lava_callback - INFO - Received callback for node 69409251575bc060ce160a72 INFO: 172.21.0.1:37896 - "POST /node/69409251575bc060ce160a72 HTTP/1.0" 202 Accepted 2025-12-17 22:42:54,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cad575bc060ce15f654 2025-12-17 22:42:54,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cad575bc060ce15f654 HTTP/1.1" 502 150 2025-12-17 22:42:54,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cad575bc060ce15f654'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:54,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408caf575bc060ce15f657 2025-12-17 22:42:54,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408caf575bc060ce15f657 HTTP/1.1" 502 150 2025-12-17 22:42:54,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408caf575bc060ce15f657'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:55,113 - lava_callback - INFO - Received callback for node 69408ee4575bc060ce15ffba INFO: 172.21.0.1:37898 - "POST /node/69408ee4575bc060ce15ffba HTTP/1.0" 202 Accepted 2025-12-17 22:42:55,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca5575bc060ce15f648 2025-12-17 22:42:55,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca5575bc060ce15f648 HTTP/1.1" 502 150 2025-12-17 22:42:55,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca5575bc060ce15f648'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:55,758 - lava_callback - INFO - Received callback for node 69408ee7575bc060ce15ffc8 INFO: 172.21.0.1:37902 - "POST /node/69408ee7575bc060ce15ffc8 HTTP/1.0" 202 Accepted 2025-12-17 22:42:56,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb1575bc060ce15f65a 2025-12-17 22:42:56,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb1575bc060ce15f65a HTTP/1.1" 502 150 2025-12-17 22:42:56,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb1575bc060ce15f65a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37910 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:42:56,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb3575bc060ce15f65f 2025-12-17 22:42:56,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb3575bc060ce15f65f HTTP/1.1" 502 150 2025-12-17 22:42:56,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb3575bc060ce15f65f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:56,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca9575bc060ce15f64e 2025-12-17 22:42:56,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca9575bc060ce15f64e HTTP/1.1" 502 150 2025-12-17 22:42:56,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca9575bc060ce15f64e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:57,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cab575bc060ce15f651 2025-12-17 22:42:57,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cab575bc060ce15f651 HTTP/1.1" 502 150 2025-12-17 22:42:57,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cab575bc060ce15f651'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:58,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cad575bc060ce15f654 2025-12-17 22:42:58,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cad575bc060ce15f654 HTTP/1.1" 502 150 2025-12-17 22:42:58,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cad575bc060ce15f654'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:58,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408caf575bc060ce15f657 2025-12-17 22:42:58,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408caf575bc060ce15f657 HTTP/1.1" 502 150 2025-12-17 22:42:58,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408caf575bc060ce15f657'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:58,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c91575bc060ce15f617 2025-12-17 22:42:58,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c91575bc060ce15f617 HTTP/1.1" 502 150 2025-12-17 22:42:58,832 - lava_callback - ERROR - Error processing callback for node 69408c91575bc060ce15f617: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c91575bc060ce15f617 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c91575bc060ce15f617 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c91575bc060ce15f617 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:42:58,834 - lava_callback - INFO - Starting async job submit for node 69408c31575bc060ce15f4a6 2025-12-17 22:42:58,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:42:58,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c31575bc060ce15f4a6 HTTP/1.1" 502 150 2025-12-17 22:42:58,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c31575bc060ce15f4a6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:58,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c31575bc060ce15f4a6 2025-12-17 22:42:58,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c31575bc060ce15f4a6 HTTP/1.1" 502 150 2025-12-17 22:42:58,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c31575bc060ce15f4a6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:42:59,179 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca7575bc060ce15f64b 2025-12-17 22:43:00,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb1575bc060ce15f65a 2025-12-17 22:43:00,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb1575bc060ce15f65a HTTP/1.1" 502 150 2025-12-17 22:43:00,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb1575bc060ce15f65a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:00,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb3575bc060ce15f65f 2025-12-17 22:43:00,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb3575bc060ce15f65f HTTP/1.1" 502 150 2025-12-17 22:43:00,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb3575bc060ce15f65f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:00,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c31575bc060ce15f4a6 2025-12-17 22:43:00,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c31575bc060ce15f4a6 HTTP/1.1" 502 150 2025-12-17 22:43:00,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c31575bc060ce15f4a6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:01,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c9d575bc060ce15f630 2025-12-17 22:43:01,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9d575bc060ce15f630 HTTP/1.1" 502 150 2025-12-17 22:43:01,936 - lava_callback - ERROR - Error processing callback for node 69408c9d575bc060ce15f630: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c9d575bc060ce15f630 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c9d575bc060ce15f630 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c9d575bc060ce15f630 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:01,938 - lava_callback - INFO - Starting async job submit for node 69408c94575bc060ce15f61e 2025-12-17 22:43:01,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:01,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c94575bc060ce15f61e HTTP/1.1" 502 150 2025-12-17 22:43:01,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c94575bc060ce15f61e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:01,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c94575bc060ce15f61e 2025-12-17 22:43:01,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c94575bc060ce15f61e HTTP/1.1" 502 150 2025-12-17 22:43:01,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c94575bc060ce15f61e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57138 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:43:03,321 - lava_callback - INFO - Received callback for node 69408ee3575bc060ce15ffb5 INFO: 172.21.0.1:57144 - "POST /node/69408ee3575bc060ce15ffb5 HTTP/1.0" 202 Accepted 2025-12-17 22:43:03,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940f14b460bc8e2651152a9 HTTP/1.1" 502 150 2025-12-17 22:43:03,728 - lava_callback - ERROR - Error processing callback for node 6940f14b460bc8e2651152a9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940f14b460bc8e2651152a9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940f14b460bc8e2651152a9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940f14b460bc8e2651152a9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:03,730 - lava_callback - INFO - Starting async job submit for node 69408cb6575bc060ce15f666 2025-12-17 22:43:03,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:03,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb6575bc060ce15f666 HTTP/1.1" 502 150 2025-12-17 22:43:03,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb6575bc060ce15f666'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:03,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb6575bc060ce15f666 2025-12-17 22:43:03,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb6575bc060ce15f666 HTTP/1.1" 502 150 2025-12-17 22:43:03,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb6575bc060ce15f666'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:03,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c95575bc060ce15f621 2025-12-17 22:43:03,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c95575bc060ce15f621 HTTP/1.1" 502 150 2025-12-17 22:43:03,818 - lava_callback - ERROR - Error processing callback for node 69408c95575bc060ce15f621: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c95575bc060ce15f621 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c95575bc060ce15f621 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c95575bc060ce15f621 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:03,820 - lava_callback - INFO - Starting async job submit for node 69408cb8575bc060ce15f669 2025-12-17 22:43:03,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:03,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb8575bc060ce15f669 HTTP/1.1" 502 150 2025-12-17 22:43:03,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb8575bc060ce15f669'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:03,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb8575bc060ce15f669 2025-12-17 22:43:03,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb8575bc060ce15f669 HTTP/1.1" 502 150 2025-12-17 22:43:03,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb8575bc060ce15f669'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:03,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c94575bc060ce15f61e 2025-12-17 22:43:03,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c94575bc060ce15f61e HTTP/1.1" 502 150 2025-12-17 22:43:03,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c94575bc060ce15f61e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:04,056 - lava_callback - INFO - Received callback for node 69408ee8575bc060ce15ffcf INFO: 172.21.0.1:57156 - "POST /node/69408ee8575bc060ce15ffcf HTTP/1.0" 202 Accepted 2025-12-17 22:43:04,786 - lava_callback - INFO - Received callback for node 69408eea575bc060ce15ffd4 INFO: 172.21.0.1:57162 - "POST /node/69408eea575bc060ce15ffd4 HTTP/1.0" 202 Accepted 2025-12-17 22:43:04,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca9575bc060ce15f64e 2025-12-17 22:43:04,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca9575bc060ce15f64e HTTP/1.1" 502 150 2025-12-17 22:43:04,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca9575bc060ce15f64e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:04,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c31575bc060ce15f4a6 2025-12-17 22:43:04,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c31575bc060ce15f4a6 HTTP/1.1" 502 150 2025-12-17 22:43:04,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c31575bc060ce15f4a6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:05,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c9f575bc060ce15f635 2025-12-17 22:43:05,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c9f575bc060ce15f635 HTTP/1.1" 502 150 2025-12-17 22:43:05,525 - lava_callback - ERROR - Error processing callback for node 69408c9f575bc060ce15f635: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c9f575bc060ce15f635 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c9f575bc060ce15f635 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c9f575bc060ce15f635 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:05,528 - lava_callback - INFO - Starting async job submit for node 69408cba575bc060ce15f66c 2025-12-17 22:43:05,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:05,537 - lava_callback - INFO - Received callback for node 69408eeb575bc060ce15ffd7 INFO: 172.21.0.1:57174 - "POST /node/69408eeb575bc060ce15ffd7 HTTP/1.0" 202 Accepted 2025-12-17 22:43:05,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cba575bc060ce15f66c HTTP/1.1" 502 150 2025-12-17 22:43:05,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cba575bc060ce15f66c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:05,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cba575bc060ce15f66c 2025-12-17 22:43:05,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cba575bc060ce15f66c HTTP/1.1" 502 150 2025-12-17 22:43:05,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cba575bc060ce15f66c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:05,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cab575bc060ce15f651 2025-12-17 22:43:05,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cab575bc060ce15f651 HTTP/1.1" 502 150 2025-12-17 22:43:05,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cab575bc060ce15f651'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:05,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb6575bc060ce15f666 2025-12-17 22:43:05,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb6575bc060ce15f666 HTTP/1.1" 502 150 2025-12-17 22:43:05,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb6575bc060ce15f666'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:05,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb8575bc060ce15f669 2025-12-17 22:43:05,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb8575bc060ce15f669 HTTP/1.1" 502 150 2025-12-17 22:43:05,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb8575bc060ce15f669'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:06,290 - lava_callback - INFO - Received callback for node 69408eec575bc060ce15ffda INFO: 172.21.0.1:57188 - "POST /node/69408eec575bc060ce15ffda HTTP/1.0" 202 Accepted INFO: 172.21.0.1:57200 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:43:06,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c96575bc060ce15f624 2025-12-17 22:43:06,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c96575bc060ce15f624 HTTP/1.1" 502 150 2025-12-17 22:43:06,616 - lava_callback - ERROR - Error processing callback for node 69408c96575bc060ce15f624: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c96575bc060ce15f624 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c96575bc060ce15f624 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c96575bc060ce15f624 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:06,618 - lava_callback - INFO - Starting async job submit for node 69408cbc575bc060ce15f66f 2025-12-17 22:43:06,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:06,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cbc575bc060ce15f66f HTTP/1.1" 502 150 2025-12-17 22:43:06,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cbc575bc060ce15f66f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:06,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cbc575bc060ce15f66f 2025-12-17 22:43:06,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cbc575bc060ce15f66f HTTP/1.1" 502 150 2025-12-17 22:43:06,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cbc575bc060ce15f66f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:06,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cad575bc060ce15f654 2025-12-17 22:43:06,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cad575bc060ce15f654 HTTP/1.1" 502 150 2025-12-17 22:43:06,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cad575bc060ce15f654'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:06,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408caf575bc060ce15f657 2025-12-17 22:43:06,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408caf575bc060ce15f657 HTTP/1.1" 502 150 2025-12-17 22:43:06,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408caf575bc060ce15f657'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:06,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca1575bc060ce15f63e 2025-12-17 22:43:06,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca1575bc060ce15f63e HTTP/1.1" 502 150 2025-12-17 22:43:06,864 - lava_callback - ERROR - Error processing callback for node 69408ca1575bc060ce15f63e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ca1575bc060ce15f63e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ca1575bc060ce15f63e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ca1575bc060ce15f63e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:06,866 - lava_callback - INFO - Starting async job submit for node 69408cbe575bc060ce15f68e 2025-12-17 22:43:06,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:06,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cbe575bc060ce15f68e HTTP/1.1" 502 150 2025-12-17 22:43:06,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cbe575bc060ce15f68e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:06,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cbe575bc060ce15f68e 2025-12-17 22:43:06,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cbe575bc060ce15f68e HTTP/1.1" 502 150 2025-12-17 22:43:06,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cbe575bc060ce15f68e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:07,029 - lava_callback - INFO - Received callback for node 69408eed575bc060ce15ffdd INFO: 172.21.0.1:57216 - "POST /node/69408eed575bc060ce15ffdd HTTP/1.0" 202 Accepted 2025-12-17 22:43:07,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cba575bc060ce15f66c 2025-12-17 22:43:07,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cba575bc060ce15f66c HTTP/1.1" 502 150 2025-12-17 22:43:07,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cba575bc060ce15f66c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:07,829 - lava_callback - INFO - Received callback for node 69408eee575bc060ce15ffe0 INFO: 172.21.0.1:57224 - "POST /node/69408eee575bc060ce15ffe0 HTTP/1.0" 202 Accepted 2025-12-17 22:43:07,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca3575bc060ce15f645 2025-12-17 22:43:07,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca3575bc060ce15f645 HTTP/1.1" 502 150 2025-12-17 22:43:07,890 - lava_callback - ERROR - Error processing callback for node 69408ca3575bc060ce15f645: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ca3575bc060ce15f645 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ca3575bc060ce15f645 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ca3575bc060ce15f645 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:07,893 - lava_callback - INFO - Starting async job submit for node 69408cc1575bc060ce15f691 2025-12-17 22:43:07,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:07,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc1575bc060ce15f691 HTTP/1.1" 502 150 2025-12-17 22:43:07,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc1575bc060ce15f691'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:07,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc1575bc060ce15f691 2025-12-17 22:43:07,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc1575bc060ce15f691 HTTP/1.1" 502 150 2025-12-17 22:43:07,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc1575bc060ce15f691'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:07,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c94575bc060ce15f61e 2025-12-17 22:43:07,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c94575bc060ce15f61e HTTP/1.1" 502 150 2025-12-17 22:43:07,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c94575bc060ce15f61e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:08,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb1575bc060ce15f65a 2025-12-17 22:43:08,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb1575bc060ce15f65a HTTP/1.1" 502 150 2025-12-17 22:43:08,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb1575bc060ce15f65a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:08,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb3575bc060ce15f65f 2025-12-17 22:43:08,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb3575bc060ce15f65f HTTP/1.1" 502 150 2025-12-17 22:43:08,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb3575bc060ce15f65f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:08,562 - lava_callback - INFO - Received callback for node 69408ef0575bc060ce15ffe3 INFO: 172.21.0.1:57238 - "POST /node/69408ef0575bc060ce15ffe3 HTTP/1.0" 202 Accepted 2025-12-17 22:43:08,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cbc575bc060ce15f66f 2025-12-17 22:43:08,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cbc575bc060ce15f66f HTTP/1.1" 502 150 2025-12-17 22:43:08,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cbc575bc060ce15f66f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:08,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cbe575bc060ce15f68e 2025-12-17 22:43:08,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cbe575bc060ce15f68e HTTP/1.1" 502 150 2025-12-17 22:43:08,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cbe575bc060ce15f68e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:09,287 - lava_callback - INFO - Received callback for node 69408ef1575bc060ce15ffe6 INFO: 172.21.0.1:57240 - "POST /node/69408ef1575bc060ce15ffe6 HTTP/1.0" 202 Accepted 2025-12-17 22:43:09,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb6575bc060ce15f666 2025-12-17 22:43:09,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb6575bc060ce15f666 HTTP/1.1" 502 150 2025-12-17 22:43:09,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb6575bc060ce15f666'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:09,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb8575bc060ce15f669 2025-12-17 22:43:09,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb8575bc060ce15f669 HTTP/1.1" 502 150 2025-12-17 22:43:09,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb8575bc060ce15f669'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:09,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc1575bc060ce15f691 2025-12-17 22:43:09,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc1575bc060ce15f691 HTTP/1.1" 502 150 2025-12-17 22:43:09,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc1575bc060ce15f691'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:10,077 - lava_callback - INFO - Received callback for node 69408ef2575bc060ce15ffe9 INFO: 172.21.0.1:53942 - "POST /node/69408ef2575bc060ce15ffe9 HTTP/1.0" 202 Accepted 2025-12-17 22:43:10,813 - lava_callback - INFO - Received callback for node 69408ef4575bc060ce15ffec INFO: 172.21.0.1:53944 - "POST /node/69408ef4575bc060ce15ffec HTTP/1.0" 202 Accepted 2025-12-17 22:43:11,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca5575bc060ce15f648 2025-12-17 22:43:11,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca5575bc060ce15f648 HTTP/1.1" 502 150 2025-12-17 22:43:11,551 - lava_callback - ERROR - Error processing callback for node 69408ca5575bc060ce15f648: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ca5575bc060ce15f648 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ca5575bc060ce15f648 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ca5575bc060ce15f648 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:11,552 - lava_callback - INFO - Starting async job submit for node 69408cc3575bc060ce15f699 2025-12-17 22:43:11,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:11,554 - lava_callback - INFO - Received callback for node 69408ef5575bc060ce15ffef INFO: 172.21.0.1:53958 - "POST /node/69408ef5575bc060ce15ffef HTTP/1.0" 202 Accepted 2025-12-17 22:43:11,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cba575bc060ce15f66c 2025-12-17 22:43:11,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cba575bc060ce15f66c HTTP/1.1" 502 150 2025-12-17 22:43:11,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cba575bc060ce15f66c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:11,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc3575bc060ce15f699 HTTP/1.1" 502 150 2025-12-17 22:43:11,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc3575bc060ce15f699'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:11,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc3575bc060ce15f699 2025-12-17 22:43:11,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc3575bc060ce15f699 HTTP/1.1" 502 150 2025-12-17 22:43:11,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc3575bc060ce15f699'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:12,293 - lava_callback - INFO - Received callback for node 69408ef6575bc060ce15fff2 INFO: 172.21.0.1:53962 - "POST /node/69408ef6575bc060ce15fff2 HTTP/1.0" 202 Accepted 2025-12-17 22:43:12,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cbc575bc060ce15f66f 2025-12-17 22:43:12,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cbc575bc060ce15f66f HTTP/1.1" 502 150 2025-12-17 22:43:12,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cbc575bc060ce15f66f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:12,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cbe575bc060ce15f68e 2025-12-17 22:43:12,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cbe575bc060ce15f68e HTTP/1.1" 502 150 2025-12-17 22:43:12,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cbe575bc060ce15f68e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:12,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c31575bc060ce15f4a6 2025-12-17 22:43:12,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c31575bc060ce15f4a6 HTTP/1.1" 502 150 2025-12-17 22:43:12,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c31575bc060ce15f4a6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:13,046 - lava_callback - INFO - Received callback for node 69408ef7575bc060ce15fff5 INFO: 172.21.0.1:53970 - "POST /node/69408ef7575bc060ce15fff5 HTTP/1.0" 202 Accepted 2025-12-17 22:43:13,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc3575bc060ce15f699 2025-12-17 22:43:13,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc3575bc060ce15f699 HTTP/1.1" 502 150 2025-12-17 22:43:13,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc3575bc060ce15f699'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:13,863 - lava_callback - INFO - Received callback for node 69408ef9575bc060ce15fffc INFO: 172.21.0.1:53974 - "POST /node/69408ef9575bc060ce15fffc HTTP/1.0" 202 Accepted 2025-12-17 22:43:13,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc1575bc060ce15f691 2025-12-17 22:43:13,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc1575bc060ce15f691 HTTP/1.1" 502 150 2025-12-17 22:43:13,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc1575bc060ce15f691'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:14,660 - lava_callback - INFO - Received callback for node 69408efa575bc060ce160003 INFO: 172.21.0.1:53990 - "POST /node/69408efa575bc060ce160003 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:54002 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:43:15,214 - lava_callback - INFO - Received callback for node 694094bb575bc060ce1610e8 INFO: 172.21.0.1:54004 - "POST /node/694094bb575bc060ce1610e8 HTTP/1.0" 202 Accepted 2025-12-17 22:43:15,397 - lava_callback - INFO - Received callback for node 69408efb575bc060ce16000a INFO: 172.21.0.1:54012 - "POST /node/69408efb575bc060ce16000a HTTP/1.0" 202 Accepted 2025-12-17 22:43:15,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c94575bc060ce15f61e 2025-12-17 22:43:15,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c94575bc060ce15f61e HTTP/1.1" 502 150 2025-12-17 22:43:15,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c94575bc060ce15f61e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:16,130 - lava_callback - INFO - Received callback for node 69408efd575bc060ce16000d INFO: 172.21.0.1:54022 - "POST /node/69408efd575bc060ce16000d HTTP/1.0" 202 Accepted INFO: 172.21.0.1:54036 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:43:16,891 - lava_callback - INFO - Received callback for node 69408efe575bc060ce160010 INFO: 172.21.0.1:54050 - "POST /node/69408efe575bc060ce160010 HTTP/1.0" 202 Accepted 2025-12-17 22:43:17,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc3575bc060ce15f699 2025-12-17 22:43:17,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc3575bc060ce15f699 HTTP/1.1" 502 150 2025-12-17 22:43:17,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc3575bc060ce15f699'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:17,633 - lava_callback - INFO - Received callback for node 69408eff575bc060ce160013 INFO: 172.21.0.1:54062 - "POST /node/69408eff575bc060ce160013 HTTP/1.0" 202 Accepted 2025-12-17 22:43:17,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb6575bc060ce15f666 2025-12-17 22:43:17,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb6575bc060ce15f666 HTTP/1.1" 502 150 2025-12-17 22:43:17,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb6575bc060ce15f666'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:17,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb8575bc060ce15f669 2025-12-17 22:43:17,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb8575bc060ce15f669 HTTP/1.1" 502 150 2025-12-17 22:43:17,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cb8575bc060ce15f669'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:18,407 - lava_callback - INFO - Received callback for node 69408f00575bc060ce160016 INFO: 172.21.0.1:54070 - "POST /node/69408f00575bc060ce160016 HTTP/1.0" 202 Accepted 2025-12-17 22:43:19,130 - lava_callback - INFO - Received callback for node 69408f01575bc060ce160019 INFO: 172.21.0.1:54076 - "POST /node/69408f01575bc060ce160019 HTTP/1.0" 202 Accepted 2025-12-17 22:43:19,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cba575bc060ce15f66c 2025-12-17 22:43:19,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cba575bc060ce15f66c HTTP/1.1" 502 150 2025-12-17 22:43:19,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cba575bc060ce15f66c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:19,860 - lava_callback - INFO - Received callback for node 69408f03575bc060ce16001c INFO: 172.21.0.1:41926 - "POST /node/69408f03575bc060ce16001c HTTP/1.0" 202 Accepted 2025-12-17 22:43:20,632 - lava_callback - INFO - Received callback for node 69408f04575bc060ce16001f INFO: 172.21.0.1:41942 - "POST /node/69408f04575bc060ce16001f HTTP/1.0" 202 Accepted 2025-12-17 22:43:20,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cbc575bc060ce15f66f 2025-12-17 22:43:20,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cbc575bc060ce15f66f HTTP/1.1" 502 150 2025-12-17 22:43:20,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cbc575bc060ce15f66f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:20,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cbe575bc060ce15f68e 2025-12-17 22:43:20,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cbe575bc060ce15f68e HTTP/1.1" 502 150 2025-12-17 22:43:20,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cbe575bc060ce15f68e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:20,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca9575bc060ce15f64e 2025-12-17 22:43:20,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca9575bc060ce15f64e HTTP/1.1" 502 150 2025-12-17 22:43:20,959 - lava_callback - ERROR - Error processing callback for node 69408ca9575bc060ce15f64e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ca9575bc060ce15f64e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ca9575bc060ce15f64e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ca9575bc060ce15f64e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:20,960 - lava_callback - INFO - Starting async job submit for node 69408cc5575bc060ce15f69c 2025-12-17 22:43:20,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:20,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc5575bc060ce15f69c HTTP/1.1" 502 150 2025-12-17 22:43:20,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc5575bc060ce15f69c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:20,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc5575bc060ce15f69c 2025-12-17 22:43:20,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc5575bc060ce15f69c HTTP/1.1" 502 150 2025-12-17 22:43:20,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc5575bc060ce15f69c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:21,381 - lava_callback - INFO - Received callback for node 69408f05575bc060ce160022 INFO: 172.21.0.1:41954 - "POST /node/69408f05575bc060ce160022 HTTP/1.0" 202 Accepted 2025-12-17 22:43:21,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cab575bc060ce15f651 2025-12-17 22:43:21,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cab575bc060ce15f651 HTTP/1.1" 502 150 2025-12-17 22:43:21,617 - lava_callback - ERROR - Error processing callback for node 69408cab575bc060ce15f651: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cab575bc060ce15f651 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cab575bc060ce15f651 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cab575bc060ce15f651 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:21,619 - lava_callback - INFO - Starting async job submit for node 69408cc6575bc060ce15f69f 2025-12-17 22:43:21,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:21,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc6575bc060ce15f69f HTTP/1.1" 502 150 2025-12-17 22:43:21,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc6575bc060ce15f69f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:21,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc6575bc060ce15f69f 2025-12-17 22:43:21,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc6575bc060ce15f69f HTTP/1.1" 502 150 2025-12-17 22:43:21,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc6575bc060ce15f69f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:21,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc1575bc060ce15f691 2025-12-17 22:43:21,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc1575bc060ce15f691 HTTP/1.1" 502 150 2025-12-17 22:43:21,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc1575bc060ce15f691'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:22,102 - lava_callback - INFO - Received callback for node 69408f07575bc060ce160025 INFO: 172.21.0.1:41958 - "POST /node/69408f07575bc060ce160025 HTTP/1.0" 202 Accepted 2025-12-17 22:43:22,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cad575bc060ce15f654 2025-12-17 22:43:22,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cad575bc060ce15f654 HTTP/1.1" 502 150 2025-12-17 22:43:22,710 - lava_callback - ERROR - Error processing callback for node 69408cad575bc060ce15f654: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cad575bc060ce15f654 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cad575bc060ce15f654 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cad575bc060ce15f654 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:22,712 - lava_callback - INFO - Starting async job submit for node 69408cd0575bc060ce15f6b3 2025-12-17 22:43:22,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:22,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd0575bc060ce15f6b3 HTTP/1.1" 502 150 2025-12-17 22:43:22,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd0575bc060ce15f6b3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:22,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd0575bc060ce15f6b3 2025-12-17 22:43:22,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd0575bc060ce15f6b3 HTTP/1.1" 502 150 2025-12-17 22:43:22,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd0575bc060ce15f6b3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:22,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408caf575bc060ce15f657 2025-12-17 22:43:22,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408caf575bc060ce15f657 HTTP/1.1" 502 150 2025-12-17 22:43:22,738 - lava_callback - ERROR - Error processing callback for node 69408caf575bc060ce15f657: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408caf575bc060ce15f657 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408caf575bc060ce15f657 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408caf575bc060ce15f657 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:22,740 - lava_callback - INFO - Starting async job submit for node 69408ccd575bc060ce15f6b0 2025-12-17 22:43:22,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:22,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ccd575bc060ce15f6b0 HTTP/1.1" 502 150 2025-12-17 22:43:22,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ccd575bc060ce15f6b0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:22,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ccd575bc060ce15f6b0 2025-12-17 22:43:22,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ccd575bc060ce15f6b0 HTTP/1.1" 502 150 2025-12-17 22:43:22,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ccd575bc060ce15f6b0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:22,830 - lava_callback - INFO - Received callback for node 69408f08575bc060ce160028 INFO: 172.21.0.1:41974 - "POST /node/69408f08575bc060ce160028 HTTP/1.0" 202 Accepted 2025-12-17 22:43:22,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc5575bc060ce15f69c 2025-12-17 22:43:22,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc5575bc060ce15f69c HTTP/1.1" 502 150 2025-12-17 22:43:22,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc5575bc060ce15f69c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:23,564 - lava_callback - INFO - Received callback for node 69408f09575bc060ce16002b INFO: 172.21.0.1:41990 - "POST /node/69408f09575bc060ce16002b HTTP/1.0" 202 Accepted 2025-12-17 22:43:23,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc6575bc060ce15f69f 2025-12-17 22:43:23,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc6575bc060ce15f69f HTTP/1.1" 502 150 2025-12-17 22:43:23,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc6575bc060ce15f69f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:24,381 - lava_callback - INFO - Received callback for node 69408f0b575bc060ce160033 INFO: 172.21.0.1:41994 - "POST /node/69408f0b575bc060ce160033 HTTP/1.0" 202 Accepted 2025-12-17 22:43:24,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb1575bc060ce15f65a 2025-12-17 22:43:24,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb1575bc060ce15f65a HTTP/1.1" 502 150 2025-12-17 22:43:24,438 - lava_callback - ERROR - Error processing callback for node 69408cb1575bc060ce15f65a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cb1575bc060ce15f65a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cb1575bc060ce15f65a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cb1575bc060ce15f65a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:24,440 - lava_callback - INFO - Starting async job submit for node 69408cc8575bc060ce15f6a4 2025-12-17 22:43:24,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:24,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc8575bc060ce15f6a4 HTTP/1.1" 502 150 2025-12-17 22:43:24,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc8575bc060ce15f6a4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:24,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc8575bc060ce15f6a4 2025-12-17 22:43:24,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc8575bc060ce15f6a4 HTTP/1.1" 502 150 2025-12-17 22:43:24,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc8575bc060ce15f6a4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:24,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb3575bc060ce15f65f 2025-12-17 22:43:24,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb3575bc060ce15f65f HTTP/1.1" 502 150 2025-12-17 22:43:24,534 - lava_callback - ERROR - Error processing callback for node 69408cb3575bc060ce15f65f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cb3575bc060ce15f65f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cb3575bc060ce15f65f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cb3575bc060ce15f65f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:24,537 - lava_callback - INFO - Starting async job submit for node 69408cca575bc060ce15f6ad 2025-12-17 22:43:24,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:24,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cca575bc060ce15f6ad HTTP/1.1" 502 150 2025-12-17 22:43:24,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cca575bc060ce15f6ad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:24,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cca575bc060ce15f6ad 2025-12-17 22:43:24,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cca575bc060ce15f6ad HTTP/1.1" 502 150 2025-12-17 22:43:24,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cca575bc060ce15f6ad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:24,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd0575bc060ce15f6b3 2025-12-17 22:43:24,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd0575bc060ce15f6b3 HTTP/1.1" 502 150 2025-12-17 22:43:24,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd0575bc060ce15f6b3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:24,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ccd575bc060ce15f6b0 2025-12-17 22:43:24,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ccd575bc060ce15f6b0 HTTP/1.1" 502 150 2025-12-17 22:43:24,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ccd575bc060ce15f6b0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:25,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc3575bc060ce15f699 2025-12-17 22:43:25,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc3575bc060ce15f699 HTTP/1.1" 502 150 2025-12-17 22:43:25,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc3575bc060ce15f699'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:26,048 - lava_callback - INFO - Received callback for node 69408f0c575bc060ce16003d INFO: 172.21.0.1:41998 - "POST /node/69408f0c575bc060ce16003d HTTP/1.0" 202 Accepted 2025-12-17 22:43:26,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc8575bc060ce15f6a4 2025-12-17 22:43:26,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc8575bc060ce15f6a4 HTTP/1.1" 502 150 2025-12-17 22:43:26,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc8575bc060ce15f6a4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42004 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:43:26,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cca575bc060ce15f6ad 2025-12-17 22:43:26,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cca575bc060ce15f6ad HTTP/1.1" 502 150 2025-12-17 22:43:26,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cca575bc060ce15f6ad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:26,833 - lava_callback - INFO - Received callback for node 69408f0d575bc060ce160044 INFO: 172.21.0.1:42014 - "POST /node/69408f0d575bc060ce160044 HTTP/1.0" 202 Accepted 2025-12-17 22:43:26,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc5575bc060ce15f69c 2025-12-17 22:43:26,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc5575bc060ce15f69c HTTP/1.1" 502 150 2025-12-17 22:43:26,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc5575bc060ce15f69c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:27,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc6575bc060ce15f69f 2025-12-17 22:43:27,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc6575bc060ce15f69f HTTP/1.1" 502 150 2025-12-17 22:43:27,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc6575bc060ce15f69f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:28,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd0575bc060ce15f6b3 2025-12-17 22:43:28,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd0575bc060ce15f6b3 HTTP/1.1" 502 150 2025-12-17 22:43:28,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd0575bc060ce15f6b3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:28,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ccd575bc060ce15f6b0 2025-12-17 22:43:28,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ccd575bc060ce15f6b0 HTTP/1.1" 502 150 2025-12-17 22:43:28,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ccd575bc060ce15f6b0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:28,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c31575bc060ce15f4a6 2025-12-17 22:43:28,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c31575bc060ce15f4a6 HTTP/1.1" 502 150 2025-12-17 22:43:28,978 - lava_callback - ERROR - Error processing callback for node 69408c31575bc060ce15f4a6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c31575bc060ce15f4a6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c31575bc060ce15f4a6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c31575bc060ce15f4a6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:28,981 - lava_callback - INFO - Starting async job submit for node 69408cd2575bc060ce15f6b6 2025-12-17 22:43:28,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:28,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd2575bc060ce15f6b6 HTTP/1.1" 502 150 2025-12-17 22:43:28,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd2575bc060ce15f6b6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:28,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd2575bc060ce15f6b6 2025-12-17 22:43:28,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd2575bc060ce15f6b6 HTTP/1.1" 502 150 2025-12-17 22:43:28,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd2575bc060ce15f6b6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:29,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca7575bc060ce15f64b HTTP/1.1" 502 150 2025-12-17 22:43:29,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ca7575bc060ce15f64b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:30,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc8575bc060ce15f6a4 2025-12-17 22:43:30,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc8575bc060ce15f6a4 HTTP/1.1" 502 150 2025-12-17 22:43:30,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc8575bc060ce15f6a4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:30,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cca575bc060ce15f6ad 2025-12-17 22:43:30,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cca575bc060ce15f6ad HTTP/1.1" 502 150 2025-12-17 22:43:30,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cca575bc060ce15f6ad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:30,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd2575bc060ce15f6b6 2025-12-17 22:43:30,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd2575bc060ce15f6b6 HTTP/1.1" 502 150 2025-12-17 22:43:30,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd2575bc060ce15f6b6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:31,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c94575bc060ce15f61e 2025-12-17 22:43:31,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c94575bc060ce15f61e HTTP/1.1" 502 150 2025-12-17 22:43:31,954 - lava_callback - ERROR - Error processing callback for node 69408c94575bc060ce15f61e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c94575bc060ce15f61e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c94575bc060ce15f61e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c94575bc060ce15f61e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:31,956 - lava_callback - INFO - Starting async job submit for node 69408cd4575bc060ce15f6b9 2025-12-17 22:43:31,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:31,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd4575bc060ce15f6b9 HTTP/1.1" 502 150 2025-12-17 22:43:31,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd4575bc060ce15f6b9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:31,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd4575bc060ce15f6b9 2025-12-17 22:43:31,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd4575bc060ce15f6b9 HTTP/1.1" 502 150 2025-12-17 22:43:31,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd4575bc060ce15f6b9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:33,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb6575bc060ce15f666 2025-12-17 22:43:33,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb6575bc060ce15f666 HTTP/1.1" 502 150 2025-12-17 22:43:33,766 - lava_callback - ERROR - Error processing callback for node 69408cb6575bc060ce15f666: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cb6575bc060ce15f666 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cb6575bc060ce15f666 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cb6575bc060ce15f666 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:33,768 - lava_callback - INFO - Starting async job submit for node 69408cd6575bc060ce15f6bc 2025-12-17 22:43:33,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:33,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd6575bc060ce15f6bc HTTP/1.1" 502 150 2025-12-17 22:43:33,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd6575bc060ce15f6bc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:33,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd6575bc060ce15f6bc 2025-12-17 22:43:33,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd6575bc060ce15f6bc HTTP/1.1" 502 150 2025-12-17 22:43:33,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd6575bc060ce15f6bc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:33,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cb8575bc060ce15f669 2025-12-17 22:43:33,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cb8575bc060ce15f669 HTTP/1.1" 502 150 2025-12-17 22:43:33,844 - lava_callback - ERROR - Error processing callback for node 69408cb8575bc060ce15f669: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cb8575bc060ce15f669 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cb8575bc060ce15f669 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cb8575bc060ce15f669 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:33,846 - lava_callback - INFO - Starting async job submit for node 69408cd8575bc060ce15f6bf 2025-12-17 22:43:33,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:33,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd8575bc060ce15f6bf HTTP/1.1" 502 150 2025-12-17 22:43:33,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd8575bc060ce15f6bf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:33,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd8575bc060ce15f6bf 2025-12-17 22:43:33,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd8575bc060ce15f6bf HTTP/1.1" 502 150 2025-12-17 22:43:33,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd8575bc060ce15f6bf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:33,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd4575bc060ce15f6b9 2025-12-17 22:43:33,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd4575bc060ce15f6b9 HTTP/1.1" 502 150 2025-12-17 22:43:33,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd4575bc060ce15f6b9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:34,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc5575bc060ce15f69c 2025-12-17 22:43:34,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc5575bc060ce15f69c HTTP/1.1" 502 150 2025-12-17 22:43:34,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc5575bc060ce15f69c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:34,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd2575bc060ce15f6b6 2025-12-17 22:43:34,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd2575bc060ce15f6b6 HTTP/1.1" 502 150 2025-12-17 22:43:34,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd2575bc060ce15f6b6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:35,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cba575bc060ce15f66c 2025-12-17 22:43:35,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cba575bc060ce15f66c HTTP/1.1" 502 150 2025-12-17 22:43:35,563 - lava_callback - ERROR - Error processing callback for node 69408cba575bc060ce15f66c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cba575bc060ce15f66c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cba575bc060ce15f66c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cba575bc060ce15f66c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:35,565 - lava_callback - INFO - Starting async job submit for node 69408cda575bc060ce15f6c2 2025-12-17 22:43:35,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:35,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cda575bc060ce15f6c2 HTTP/1.1" 502 150 2025-12-17 22:43:35,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cda575bc060ce15f6c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:35,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cda575bc060ce15f6c2 2025-12-17 22:43:35,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cda575bc060ce15f6c2 HTTP/1.1" 502 150 2025-12-17 22:43:35,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cda575bc060ce15f6c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:35,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc6575bc060ce15f69f 2025-12-17 22:43:35,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc6575bc060ce15f69f HTTP/1.1" 502 150 2025-12-17 22:43:35,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc6575bc060ce15f69f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:35,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd6575bc060ce15f6bc 2025-12-17 22:43:35,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd6575bc060ce15f6bc HTTP/1.1" 502 150 2025-12-17 22:43:35,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd6575bc060ce15f6bc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:35,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd8575bc060ce15f6bf 2025-12-17 22:43:35,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd8575bc060ce15f6bf HTTP/1.1" 502 150 2025-12-17 22:43:35,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd8575bc060ce15f6bf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53902 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:43:36,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cbc575bc060ce15f66f 2025-12-17 22:43:36,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cbc575bc060ce15f66f HTTP/1.1" 502 150 2025-12-17 22:43:36,658 - lava_callback - ERROR - Error processing callback for node 69408cbc575bc060ce15f66f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cbc575bc060ce15f66f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cbc575bc060ce15f66f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cbc575bc060ce15f66f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:36,660 - lava_callback - INFO - Starting async job submit for node 69408cdc575bc060ce15f6c5 2025-12-17 22:43:36,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:36,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cdc575bc060ce15f6c5 HTTP/1.1" 502 150 2025-12-17 22:43:36,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cdc575bc060ce15f6c5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:36,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cdc575bc060ce15f6c5 2025-12-17 22:43:36,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cdc575bc060ce15f6c5 HTTP/1.1" 502 150 2025-12-17 22:43:36,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cdc575bc060ce15f6c5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:36,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd0575bc060ce15f6b3 2025-12-17 22:43:36,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd0575bc060ce15f6b3 HTTP/1.1" 502 150 2025-12-17 22:43:36,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd0575bc060ce15f6b3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:36,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ccd575bc060ce15f6b0 2025-12-17 22:43:36,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ccd575bc060ce15f6b0 HTTP/1.1" 502 150 2025-12-17 22:43:36,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ccd575bc060ce15f6b0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:36,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cbe575bc060ce15f68e 2025-12-17 22:43:36,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cbe575bc060ce15f68e HTTP/1.1" 502 150 2025-12-17 22:43:36,883 - lava_callback - ERROR - Error processing callback for node 69408cbe575bc060ce15f68e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cbe575bc060ce15f68e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cbe575bc060ce15f68e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cbe575bc060ce15f68e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:36,886 - lava_callback - INFO - Starting async job submit for node 69408cde575bc060ce15f6ce 2025-12-17 22:43:36,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:36,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cde575bc060ce15f6ce HTTP/1.1" 502 150 2025-12-17 22:43:36,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cde575bc060ce15f6ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:36,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cde575bc060ce15f6ce 2025-12-17 22:43:36,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cde575bc060ce15f6ce HTTP/1.1" 502 150 2025-12-17 22:43:36,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cde575bc060ce15f6ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:37,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cda575bc060ce15f6c2 2025-12-17 22:43:37,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cda575bc060ce15f6c2 HTTP/1.1" 502 150 2025-12-17 22:43:37,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cda575bc060ce15f6c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:37,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc1575bc060ce15f691 2025-12-17 22:43:37,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc1575bc060ce15f691 HTTP/1.1" 502 150 2025-12-17 22:43:37,919 - lava_callback - ERROR - Error processing callback for node 69408cc1575bc060ce15f691: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cc1575bc060ce15f691 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cc1575bc060ce15f691 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cc1575bc060ce15f691 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:37,921 - lava_callback - INFO - Starting async job submit for node 69408ce0575bc060ce15f6d5 2025-12-17 22:43:37,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:37,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce0575bc060ce15f6d5 HTTP/1.1" 502 150 2025-12-17 22:43:37,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce0575bc060ce15f6d5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:37,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce0575bc060ce15f6d5 2025-12-17 22:43:37,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce0575bc060ce15f6d5 HTTP/1.1" 502 150 2025-12-17 22:43:37,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce0575bc060ce15f6d5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:37,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd4575bc060ce15f6b9 2025-12-17 22:43:37,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd4575bc060ce15f6b9 HTTP/1.1" 502 150 2025-12-17 22:43:37,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd4575bc060ce15f6b9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:38,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc8575bc060ce15f6a4 2025-12-17 22:43:38,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc8575bc060ce15f6a4 HTTP/1.1" 502 150 2025-12-17 22:43:38,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cc8575bc060ce15f6a4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:38,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cca575bc060ce15f6ad 2025-12-17 22:43:38,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cca575bc060ce15f6ad HTTP/1.1" 502 150 2025-12-17 22:43:38,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cca575bc060ce15f6ad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:38,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cdc575bc060ce15f6c5 2025-12-17 22:43:38,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cdc575bc060ce15f6c5 HTTP/1.1" 502 150 2025-12-17 22:43:38,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cdc575bc060ce15f6c5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:38,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cde575bc060ce15f6ce 2025-12-17 22:43:38,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cde575bc060ce15f6ce HTTP/1.1" 502 150 2025-12-17 22:43:38,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cde575bc060ce15f6ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:39,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd6575bc060ce15f6bc 2025-12-17 22:43:39,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd6575bc060ce15f6bc HTTP/1.1" 502 150 2025-12-17 22:43:39,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd6575bc060ce15f6bc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:39,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd8575bc060ce15f6bf 2025-12-17 22:43:39,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd8575bc060ce15f6bf HTTP/1.1" 502 150 2025-12-17 22:43:39,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd8575bc060ce15f6bf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:39,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce0575bc060ce15f6d5 2025-12-17 22:43:39,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce0575bc060ce15f6d5 HTTP/1.1" 502 150 2025-12-17 22:43:39,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce0575bc060ce15f6d5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:41,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cda575bc060ce15f6c2 2025-12-17 22:43:41,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cda575bc060ce15f6c2 HTTP/1.1" 502 150 2025-12-17 22:43:41,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cda575bc060ce15f6c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:41,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc3575bc060ce15f699 2025-12-17 22:43:41,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc3575bc060ce15f699 HTTP/1.1" 502 150 2025-12-17 22:43:41,594 - lava_callback - ERROR - Error processing callback for node 69408cc3575bc060ce15f699: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cc3575bc060ce15f699 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cc3575bc060ce15f699 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cc3575bc060ce15f699 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:41,596 - lava_callback - INFO - Starting async job submit for node 69408ce2575bc060ce15f6d8 2025-12-17 22:43:41,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:41,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce2575bc060ce15f6d8 HTTP/1.1" 502 150 2025-12-17 22:43:41,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce2575bc060ce15f6d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:41,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce2575bc060ce15f6d8 2025-12-17 22:43:41,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce2575bc060ce15f6d8 HTTP/1.1" 502 150 2025-12-17 22:43:41,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce2575bc060ce15f6d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:42,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cdc575bc060ce15f6c5 2025-12-17 22:43:42,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cdc575bc060ce15f6c5 HTTP/1.1" 502 150 2025-12-17 22:43:42,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cdc575bc060ce15f6c5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:42,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cde575bc060ce15f6ce 2025-12-17 22:43:42,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cde575bc060ce15f6ce HTTP/1.1" 502 150 2025-12-17 22:43:42,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cde575bc060ce15f6ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:42,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd2575bc060ce15f6b6 2025-12-17 22:43:42,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd2575bc060ce15f6b6 HTTP/1.1" 502 150 2025-12-17 22:43:42,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd2575bc060ce15f6b6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:43,192 - lava_callback - INFO - Received callback for node 69408c8a575bc060ce15f5f9 INFO: 172.21.0.1:58558 - "POST /node/69408c8a575bc060ce15f5f9 HTTP/1.0" 202 Accepted 2025-12-17 22:43:43,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce2575bc060ce15f6d8 2025-12-17 22:43:43,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce2575bc060ce15f6d8 HTTP/1.1" 502 150 2025-12-17 22:43:43,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce2575bc060ce15f6d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:43,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce0575bc060ce15f6d5 2025-12-17 22:43:43,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce0575bc060ce15f6d5 HTTP/1.1" 502 150 2025-12-17 22:43:43,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce0575bc060ce15f6d5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:45,443 - lava_callback - INFO - Received callback for node 69408f11575bc060ce160057 INFO: 172.21.0.1:58572 - "POST /node/69408f11575bc060ce160057 HTTP/1.0" 202 Accepted 2025-12-17 22:43:45,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ca7575bc060ce15f64b 2025-12-17 22:43:45,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ca7575bc060ce15f64b HTTP/1.1" 502 150 2025-12-17 22:43:45,842 - lava_callback - ERROR - Error processing callback for node 69408ca7575bc060ce15f64b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ca7575bc060ce15f64b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ca7575bc060ce15f64b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ca7575bc060ce15f64b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:45,843 - lava_callback - INFO - Starting async job submit for node 69408ce4575bc060ce15f6db 2025-12-17 22:43:45,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:45,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce4575bc060ce15f6db HTTP/1.1" 502 150 2025-12-17 22:43:45,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce4575bc060ce15f6db'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:45,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce4575bc060ce15f6db 2025-12-17 22:43:45,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce4575bc060ce15f6db HTTP/1.1" 502 150 2025-12-17 22:43:45,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce4575bc060ce15f6db'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:45,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd4575bc060ce15f6b9 2025-12-17 22:43:45,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd4575bc060ce15f6b9 HTTP/1.1" 502 150 2025-12-17 22:43:45,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd4575bc060ce15f6b9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:46,066 - lava_callback - INFO - Received callback for node 69408f14575bc060ce160067 INFO: 172.21.0.1:58578 - "POST /node/69408f14575bc060ce160067 HTTP/1.0" 202 Accepted 2025-12-17 22:43:46,495 - lava_callback - INFO - Received callback for node 69408f0e575bc060ce16004b INFO: 172.21.0.1:58584 - "POST /node/69408f0e575bc060ce16004b HTTP/1.0" 202 Accepted INFO: 172.21.0.1:58594 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:43:47,240 - lava_callback - INFO - Received callback for node 69408f10575bc060ce160052 INFO: 172.21.0.1:58606 - "POST /node/69408f10575bc060ce160052 HTTP/1.0" 202 Accepted 2025-12-17 22:43:47,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce2575bc060ce15f6d8 2025-12-17 22:43:47,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce2575bc060ce15f6d8 HTTP/1.1" 502 150 2025-12-17 22:43:47,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce2575bc060ce15f6d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:47,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd6575bc060ce15f6bc 2025-12-17 22:43:47,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd6575bc060ce15f6bc HTTP/1.1" 502 150 2025-12-17 22:43:47,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd6575bc060ce15f6bc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:47,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce4575bc060ce15f6db 2025-12-17 22:43:47,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce4575bc060ce15f6db HTTP/1.1" 502 150 2025-12-17 22:43:47,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce4575bc060ce15f6db'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:47,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd8575bc060ce15f6bf 2025-12-17 22:43:47,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd8575bc060ce15f6bf HTTP/1.1" 502 150 2025-12-17 22:43:47,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cd8575bc060ce15f6bf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:48,029 - lava_callback - INFO - Received callback for node 69408f1c575bc060ce160076 INFO: 172.21.0.1:58612 - "POST /node/69408f1c575bc060ce160076 HTTP/1.0" 202 Accepted 2025-12-17 22:43:48,765 - lava_callback - INFO - Received callback for node 69408f1d575bc060ce160079 INFO: 172.21.0.1:58628 - "POST /node/69408f1d575bc060ce160079 HTTP/1.0" 202 Accepted 2025-12-17 22:43:49,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cda575bc060ce15f6c2 2025-12-17 22:43:49,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cda575bc060ce15f6c2 HTTP/1.1" 502 150 2025-12-17 22:43:49,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cda575bc060ce15f6c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:50,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cdc575bc060ce15f6c5 2025-12-17 22:43:50,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cdc575bc060ce15f6c5 HTTP/1.1" 502 150 2025-12-17 22:43:50,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cdc575bc060ce15f6c5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:50,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cde575bc060ce15f6ce 2025-12-17 22:43:50,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cde575bc060ce15f6ce HTTP/1.1" 502 150 2025-12-17 22:43:50,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cde575bc060ce15f6ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:50,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc5575bc060ce15f69c 2025-12-17 22:43:50,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc5575bc060ce15f69c HTTP/1.1" 502 150 2025-12-17 22:43:50,976 - lava_callback - ERROR - Error processing callback for node 69408cc5575bc060ce15f69c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cc5575bc060ce15f69c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cc5575bc060ce15f69c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cc5575bc060ce15f69c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:50,978 - lava_callback - INFO - Starting async job submit for node 69408ce6575bc060ce15f6de 2025-12-17 22:43:50,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:50,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce6575bc060ce15f6de HTTP/1.1" 502 150 2025-12-17 22:43:50,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce6575bc060ce15f6de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:50,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce6575bc060ce15f6de 2025-12-17 22:43:50,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce6575bc060ce15f6de HTTP/1.1" 502 150 2025-12-17 22:43:50,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce6575bc060ce15f6de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:51,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc6575bc060ce15f69f 2025-12-17 22:43:51,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc6575bc060ce15f69f HTTP/1.1" 502 150 2025-12-17 22:43:51,632 - lava_callback - ERROR - Error processing callback for node 69408cc6575bc060ce15f69f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cc6575bc060ce15f69f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cc6575bc060ce15f69f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cc6575bc060ce15f69f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:51,634 - lava_callback - INFO - Starting async job submit for node 69408ce8575bc060ce15f6e1 2025-12-17 22:43:51,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:51,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce8575bc060ce15f6e1 HTTP/1.1" 502 150 2025-12-17 22:43:51,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce8575bc060ce15f6e1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:51,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce8575bc060ce15f6e1 2025-12-17 22:43:51,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce8575bc060ce15f6e1 HTTP/1.1" 502 150 2025-12-17 22:43:51,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce8575bc060ce15f6e1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:51,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce4575bc060ce15f6db 2025-12-17 22:43:51,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce4575bc060ce15f6db HTTP/1.1" 502 150 2025-12-17 22:43:51,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce4575bc060ce15f6db'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:51,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce0575bc060ce15f6d5 2025-12-17 22:43:51,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce0575bc060ce15f6d5 HTTP/1.1" 502 150 2025-12-17 22:43:51,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce0575bc060ce15f6d5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:52,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd0575bc060ce15f6b3 2025-12-17 22:43:52,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd0575bc060ce15f6b3 HTTP/1.1" 502 150 2025-12-17 22:43:52,729 - lava_callback - ERROR - Error processing callback for node 69408cd0575bc060ce15f6b3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cd0575bc060ce15f6b3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cd0575bc060ce15f6b3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cd0575bc060ce15f6b3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:52,731 - lava_callback - INFO - Starting async job submit for node 69408cea575bc060ce15f6e4 2025-12-17 22:43:52,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:52,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ccd575bc060ce15f6b0 2025-12-17 22:43:52,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ccd575bc060ce15f6b0 HTTP/1.1" 502 150 2025-12-17 22:43:52,751 - lava_callback - ERROR - Error processing callback for node 69408ccd575bc060ce15f6b0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ccd575bc060ce15f6b0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ccd575bc060ce15f6b0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ccd575bc060ce15f6b0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:52,753 - lava_callback - INFO - Starting async job submit for node 69408cec575bc060ce15f6e7 2025-12-17 22:43:52,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:52,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cec575bc060ce15f6e7 HTTP/1.1" 502 150 2025-12-17 22:43:52,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cec575bc060ce15f6e7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:52,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cec575bc060ce15f6e7 2025-12-17 22:43:52,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cec575bc060ce15f6e7 HTTP/1.1" 502 150 2025-12-17 22:43:52,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cec575bc060ce15f6e7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:52,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cea575bc060ce15f6e4 HTTP/1.1" 502 150 2025-12-17 22:43:52,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cea575bc060ce15f6e4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:52,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cea575bc060ce15f6e4 2025-12-17 22:43:52,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cea575bc060ce15f6e4 HTTP/1.1" 502 150 2025-12-17 22:43:52,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cea575bc060ce15f6e4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:52,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce6575bc060ce15f6de 2025-12-17 22:43:52,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce6575bc060ce15f6de HTTP/1.1" 502 150 2025-12-17 22:43:52,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce6575bc060ce15f6de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:53,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce8575bc060ce15f6e1 2025-12-17 22:43:53,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce8575bc060ce15f6e1 HTTP/1.1" 502 150 2025-12-17 22:43:53,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce8575bc060ce15f6e1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:54,363 - lava_callback - INFO - Received callback for node 6942bf7e1b61f1c6005fe7f9 INFO: 172.21.0.1:45502 - "POST /node/6942bf7e1b61f1c6005fe7f9 HTTP/1.0" 202 Accepted 2025-12-17 22:43:54,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cc8575bc060ce15f6a4 2025-12-17 22:43:54,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cc8575bc060ce15f6a4 HTTP/1.1" 502 150 2025-12-17 22:43:54,457 - lava_callback - ERROR - Error processing callback for node 69408cc8575bc060ce15f6a4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cc8575bc060ce15f6a4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cc8575bc060ce15f6a4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cc8575bc060ce15f6a4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:54,460 - lava_callback - INFO - Starting async job submit for node 69408cee575bc060ce15f6ea 2025-12-17 22:43:54,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:54,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cee575bc060ce15f6ea HTTP/1.1" 502 150 2025-12-17 22:43:54,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cee575bc060ce15f6ea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:54,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cee575bc060ce15f6ea 2025-12-17 22:43:54,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cee575bc060ce15f6ea HTTP/1.1" 502 150 2025-12-17 22:43:54,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cee575bc060ce15f6ea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:54,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cca575bc060ce15f6ad 2025-12-17 22:43:54,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cca575bc060ce15f6ad HTTP/1.1" 502 150 2025-12-17 22:43:54,551 - lava_callback - ERROR - Error processing callback for node 69408cca575bc060ce15f6ad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cca575bc060ce15f6ad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cca575bc060ce15f6ad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cca575bc060ce15f6ad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:54,553 - lava_callback - INFO - Starting async job submit for node 69408cf2575bc060ce15f6f2 2025-12-17 22:43:54,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:54,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf2575bc060ce15f6f2 HTTP/1.1" 502 150 2025-12-17 22:43:54,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf2575bc060ce15f6f2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:54,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf2575bc060ce15f6f2 2025-12-17 22:43:54,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf2575bc060ce15f6f2 HTTP/1.1" 502 150 2025-12-17 22:43:54,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf2575bc060ce15f6f2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:54,622 - lava_callback - INFO - Received callback for node 69408f18575bc060ce16006d INFO: 172.21.0.1:45518 - "POST /node/69408f18575bc060ce16006d HTTP/1.0" 202 Accepted 2025-12-17 22:43:54,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cec575bc060ce15f6e7 2025-12-17 22:43:54,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cec575bc060ce15f6e7 HTTP/1.1" 502 150 2025-12-17 22:43:54,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cec575bc060ce15f6e7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:54,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cea575bc060ce15f6e4 2025-12-17 22:43:54,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cea575bc060ce15f6e4 HTTP/1.1" 502 150 2025-12-17 22:43:54,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cea575bc060ce15f6e4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:55,177 - lava_callback - INFO - Received callback for node 69408f16575bc060ce16006a INFO: 172.21.0.1:45532 - "POST /node/69408f16575bc060ce16006a HTTP/1.0" 202 Accepted 2025-12-17 22:43:55,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce2575bc060ce15f6d8 2025-12-17 22:43:55,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce2575bc060ce15f6d8 HTTP/1.1" 502 150 2025-12-17 22:43:55,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce2575bc060ce15f6d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:55,851 - lava_callback - INFO - Received callback for node 69408f19575bc060ce160070 INFO: 172.21.0.1:45540 - "POST /node/69408f19575bc060ce160070 HTTP/1.0" 202 Accepted 2025-12-17 22:43:56,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cee575bc060ce15f6ea 2025-12-17 22:43:56,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cee575bc060ce15f6ea HTTP/1.1" 502 150 2025-12-17 22:43:56,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cee575bc060ce15f6ea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45556 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:43:56,533 - lava_callback - INFO - Received callback for node 69408f1f575bc060ce160086 INFO: 172.21.0.1:45568 - "POST /node/69408f1f575bc060ce160086 HTTP/1.0" 202 Accepted 2025-12-17 22:43:56,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf2575bc060ce15f6f2 2025-12-17 22:43:56,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf2575bc060ce15f6f2 HTTP/1.1" 502 150 2025-12-17 22:43:56,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf2575bc060ce15f6f2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:56,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce6575bc060ce15f6de 2025-12-17 22:43:56,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce6575bc060ce15f6de HTTP/1.1" 502 150 2025-12-17 22:43:56,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce6575bc060ce15f6de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:57,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce8575bc060ce15f6e1 2025-12-17 22:43:57,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce8575bc060ce15f6e1 HTTP/1.1" 502 150 2025-12-17 22:43:57,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce8575bc060ce15f6e1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:58,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cec575bc060ce15f6e7 2025-12-17 22:43:58,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cec575bc060ce15f6e7 HTTP/1.1" 502 150 2025-12-17 22:43:58,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cec575bc060ce15f6e7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:58,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cea575bc060ce15f6e4 2025-12-17 22:43:58,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cea575bc060ce15f6e4 HTTP/1.1" 502 150 2025-12-17 22:43:58,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cea575bc060ce15f6e4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:58,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd2575bc060ce15f6b6 2025-12-17 22:43:59,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd2575bc060ce15f6b6 HTTP/1.1" 502 150 2025-12-17 22:43:59,001 - lava_callback - ERROR - Error processing callback for node 69408cd2575bc060ce15f6b6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cd2575bc060ce15f6b6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cd2575bc060ce15f6b6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cd2575bc060ce15f6b6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:43:59,002 - lava_callback - INFO - Starting async job submit for node 69408cf4575bc060ce15f6f5 2025-12-17 22:43:59,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:43:59,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf4575bc060ce15f6f5 HTTP/1.1" 502 150 2025-12-17 22:43:59,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf4575bc060ce15f6f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:59,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf4575bc060ce15f6f5 2025-12-17 22:43:59,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf4575bc060ce15f6f5 HTTP/1.1" 502 150 2025-12-17 22:43:59,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf4575bc060ce15f6f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:43:59,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce4575bc060ce15f6db 2025-12-17 22:43:59,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce4575bc060ce15f6db HTTP/1.1" 502 150 2025-12-17 22:43:59,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce4575bc060ce15f6db'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:00,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cee575bc060ce15f6ea 2025-12-17 22:44:00,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cee575bc060ce15f6ea HTTP/1.1" 502 150 2025-12-17 22:44:00,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cee575bc060ce15f6ea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:00,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf2575bc060ce15f6f2 2025-12-17 22:44:00,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf2575bc060ce15f6f2 HTTP/1.1" 502 150 2025-12-17 22:44:00,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf2575bc060ce15f6f2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:01,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf4575bc060ce15f6f5 2025-12-17 22:44:01,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf4575bc060ce15f6f5 HTTP/1.1" 502 150 2025-12-17 22:44:01,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf4575bc060ce15f6f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:01,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd4575bc060ce15f6b9 2025-12-17 22:44:01,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd4575bc060ce15f6b9 HTTP/1.1" 502 150 2025-12-17 22:44:01,976 - lava_callback - ERROR - Error processing callback for node 69408cd4575bc060ce15f6b9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cd4575bc060ce15f6b9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cd4575bc060ce15f6b9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cd4575bc060ce15f6b9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:01,977 - lava_callback - INFO - Starting async job submit for node 69408cf6575bc060ce15f6fd 2025-12-17 22:44:01,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:51928 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:44:03,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd6575bc060ce15f6bc 2025-12-17 22:44:03,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd6575bc060ce15f6bc HTTP/1.1" 502 150 2025-12-17 22:44:03,806 - lava_callback - ERROR - Error processing callback for node 69408cd6575bc060ce15f6bc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cd6575bc060ce15f6bc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cd6575bc060ce15f6bc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cd6575bc060ce15f6bc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:03,807 - lava_callback - INFO - Starting async job submit for node 69408cf8575bc060ce15f700 2025-12-17 22:44:03,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:03,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf8575bc060ce15f700 HTTP/1.1" 502 150 2025-12-17 22:44:03,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf8575bc060ce15f700'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:03,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf8575bc060ce15f700 2025-12-17 22:44:03,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf8575bc060ce15f700 HTTP/1.1" 502 150 2025-12-17 22:44:03,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf8575bc060ce15f700'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:03,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cd8575bc060ce15f6bf 2025-12-17 22:44:03,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cd8575bc060ce15f6bf HTTP/1.1" 502 150 2025-12-17 22:44:03,861 - lava_callback - ERROR - Error processing callback for node 69408cd8575bc060ce15f6bf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cd8575bc060ce15f6bf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cd8575bc060ce15f6bf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cd8575bc060ce15f6bf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:03,862 - lava_callback - INFO - Starting async job submit for node 69408cf9575bc060ce15f703 2025-12-17 22:44:03,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:03,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf9575bc060ce15f703 HTTP/1.1" 502 150 2025-12-17 22:44:03,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf9575bc060ce15f703'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:03,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf9575bc060ce15f703 2025-12-17 22:44:03,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf9575bc060ce15f703 HTTP/1.1" 502 150 2025-12-17 22:44:03,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf9575bc060ce15f703'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:04,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce6575bc060ce15f6de 2025-12-17 22:44:04,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce6575bc060ce15f6de HTTP/1.1" 502 150 2025-12-17 22:44:04,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce6575bc060ce15f6de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:05,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf4575bc060ce15f6f5 2025-12-17 22:44:05,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf4575bc060ce15f6f5 HTTP/1.1" 502 150 2025-12-17 22:44:05,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf4575bc060ce15f6f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:05,459 - lava_callback - INFO - Received callback for node 69408f27575bc060ce160097 INFO: 172.21.0.1:51942 - "POST /node/69408f27575bc060ce160097 HTTP/1.0" 202 Accepted 2025-12-17 22:44:05,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cda575bc060ce15f6c2 2025-12-17 22:44:05,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cda575bc060ce15f6c2 HTTP/1.1" 502 150 2025-12-17 22:44:05,579 - lava_callback - ERROR - Error processing callback for node 69408cda575bc060ce15f6c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cda575bc060ce15f6c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cda575bc060ce15f6c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cda575bc060ce15f6c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:05,580 - lava_callback - INFO - Starting async job submit for node 69408cfc575bc060ce15f733 2025-12-17 22:44:05,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:05,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfc575bc060ce15f733 HTTP/1.1" 502 150 2025-12-17 22:44:05,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cfc575bc060ce15f733'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:05,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cfc575bc060ce15f733 2025-12-17 22:44:05,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfc575bc060ce15f733 HTTP/1.1" 502 150 2025-12-17 22:44:05,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cfc575bc060ce15f733'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:05,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce8575bc060ce15f6e1 2025-12-17 22:44:05,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce8575bc060ce15f6e1 HTTP/1.1" 502 150 2025-12-17 22:44:05,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ce8575bc060ce15f6e1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:05,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf8575bc060ce15f700 2025-12-17 22:44:05,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf8575bc060ce15f700 HTTP/1.1" 502 150 2025-12-17 22:44:05,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf8575bc060ce15f700'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:05,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf9575bc060ce15f703 2025-12-17 22:44:05,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf9575bc060ce15f703 HTTP/1.1" 502 150 2025-12-17 22:44:05,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf9575bc060ce15f703'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:06,049 - lava_callback - INFO - Received callback for node 69408f25575bc060ce160091 INFO: 172.21.0.1:51948 - "POST /node/69408f25575bc060ce160091 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:51960 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:44:06,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cdc575bc060ce15f6c5 2025-12-17 22:44:06,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cdc575bc060ce15f6c5 HTTP/1.1" 502 150 2025-12-17 22:44:06,677 - lava_callback - ERROR - Error processing callback for node 69408cdc575bc060ce15f6c5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cdc575bc060ce15f6c5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cdc575bc060ce15f6c5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cdc575bc060ce15f6c5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:06,679 - lava_callback - INFO - Starting async job submit for node 69408cfd575bc060ce15f736 2025-12-17 22:44:06,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:06,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfd575bc060ce15f736 HTTP/1.1" 502 150 2025-12-17 22:44:06,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cfd575bc060ce15f736'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:06,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cfd575bc060ce15f736 2025-12-17 22:44:06,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfd575bc060ce15f736 HTTP/1.1" 502 150 2025-12-17 22:44:06,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cfd575bc060ce15f736'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:06,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cec575bc060ce15f6e7 2025-12-17 22:44:06,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cec575bc060ce15f6e7 HTTP/1.1" 502 150 2025-12-17 22:44:06,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cec575bc060ce15f6e7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:06,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cea575bc060ce15f6e4 2025-12-17 22:44:06,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cea575bc060ce15f6e4 HTTP/1.1" 502 150 2025-12-17 22:44:06,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cea575bc060ce15f6e4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:06,783 - lava_callback - INFO - Received callback for node 69408f26575bc060ce160094 INFO: 172.21.0.1:51974 - "POST /node/69408f26575bc060ce160094 HTTP/1.0" 202 Accepted 2025-12-17 22:44:06,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cde575bc060ce15f6ce 2025-12-17 22:44:06,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cde575bc060ce15f6ce HTTP/1.1" 502 150 2025-12-17 22:44:06,929 - lava_callback - ERROR - Error processing callback for node 69408cde575bc060ce15f6ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cde575bc060ce15f6ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cde575bc060ce15f6ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cde575bc060ce15f6ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:06,931 - lava_callback - INFO - Starting async job submit for node 69408cfe575bc060ce15f739 2025-12-17 22:44:06,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:06,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfe575bc060ce15f739 HTTP/1.1" 502 150 2025-12-17 22:44:06,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cfe575bc060ce15f739'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:06,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cfe575bc060ce15f739 2025-12-17 22:44:06,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfe575bc060ce15f739 HTTP/1.1" 502 150 2025-12-17 22:44:06,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cfe575bc060ce15f739'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:07,007 - lava_callback - INFO - Received callback for node 6940fea7460bc8e265116410 INFO: 172.21.0.1:51976 - "POST /node/6940fea7460bc8e265116410 HTTP/1.0" 202 Accepted 2025-12-17 22:44:07,533 - lava_callback - INFO - Received callback for node 69408f2a575bc060ce1600a0 INFO: 172.21.0.1:51980 - "POST /node/69408f2a575bc060ce1600a0 HTTP/1.0" 202 Accepted 2025-12-17 22:44:07,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cfc575bc060ce15f733 2025-12-17 22:44:07,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfc575bc060ce15f733 HTTP/1.1" 502 150 2025-12-17 22:44:07,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cfc575bc060ce15f733'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:07,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce0575bc060ce15f6d5 2025-12-17 22:44:07,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce0575bc060ce15f6d5 HTTP/1.1" 502 150 2025-12-17 22:44:07,958 - lava_callback - ERROR - Error processing callback for node 69408ce0575bc060ce15f6d5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ce0575bc060ce15f6d5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ce0575bc060ce15f6d5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ce0575bc060ce15f6d5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:07,960 - lava_callback - INFO - Starting async job submit for node 69408d00575bc060ce15f73c 2025-12-17 22:44:07,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:07,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d00575bc060ce15f73c HTTP/1.1" 502 150 2025-12-17 22:44:07,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d00575bc060ce15f73c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:07,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d00575bc060ce15f73c 2025-12-17 22:44:07,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d00575bc060ce15f73c HTTP/1.1" 502 150 2025-12-17 22:44:07,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d00575bc060ce15f73c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:08,325 - lava_callback - INFO - Received callback for node 69408f2b575bc060ce1600a3 INFO: 172.21.0.1:51984 - "POST /node/69408f2b575bc060ce1600a3 HTTP/1.0" 202 Accepted 2025-12-17 22:44:08,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cee575bc060ce15f6ea 2025-12-17 22:44:08,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cee575bc060ce15f6ea HTTP/1.1" 502 150 2025-12-17 22:44:08,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cee575bc060ce15f6ea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:08,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf2575bc060ce15f6f2 2025-12-17 22:44:08,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf2575bc060ce15f6f2 HTTP/1.1" 502 150 2025-12-17 22:44:08,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf2575bc060ce15f6f2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:08,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cfd575bc060ce15f736 2025-12-17 22:44:08,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfd575bc060ce15f736 HTTP/1.1" 502 150 2025-12-17 22:44:08,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cfd575bc060ce15f736'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:08,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cfe575bc060ce15f739 2025-12-17 22:44:08,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfe575bc060ce15f739 HTTP/1.1" 502 150 2025-12-17 22:44:08,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cfe575bc060ce15f739'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:09,057 - lava_callback - INFO - Received callback for node 69408f2d575bc060ce1600a6 INFO: 172.21.0.1:51990 - "POST /node/69408f2d575bc060ce1600a6 HTTP/1.0" 202 Accepted 2025-12-17 22:44:09,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf8575bc060ce15f700 2025-12-17 22:44:09,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf8575bc060ce15f700 HTTP/1.1" 502 150 2025-12-17 22:44:09,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf8575bc060ce15f700'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:09,827 - lava_callback - INFO - Received callback for node 69408f2f575bc060ce1600ac INFO: 172.21.0.1:58098 - "POST /node/69408f2f575bc060ce1600ac HTTP/1.0" 202 Accepted 2025-12-17 22:44:09,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf9575bc060ce15f703 2025-12-17 22:44:09,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf9575bc060ce15f703 HTTP/1.1" 502 150 2025-12-17 22:44:09,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf9575bc060ce15f703'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:09,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d00575bc060ce15f73c 2025-12-17 22:44:09,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d00575bc060ce15f73c HTTP/1.1" 502 150 2025-12-17 22:44:09,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d00575bc060ce15f73c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:10,568 - lava_callback - INFO - Received callback for node 69408f30575bc060ce1600af INFO: 172.21.0.1:58110 - "POST /node/69408f30575bc060ce1600af HTTP/1.0" 202 Accepted 2025-12-17 22:44:11,303 - lava_callback - INFO - Received callback for node 69408f32575bc060ce1600b2 INFO: 172.21.0.1:58116 - "POST /node/69408f32575bc060ce1600b2 HTTP/1.0" 202 Accepted 2025-12-17 22:44:11,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cfc575bc060ce15f733 2025-12-17 22:44:11,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfc575bc060ce15f733 HTTP/1.1" 502 150 2025-12-17 22:44:11,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cfc575bc060ce15f733'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:11,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce2575bc060ce15f6d8 2025-12-17 22:44:11,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce2575bc060ce15f6d8 HTTP/1.1" 502 150 2025-12-17 22:44:11,613 - lava_callback - ERROR - Error processing callback for node 69408ce2575bc060ce15f6d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ce2575bc060ce15f6d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ce2575bc060ce15f6d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ce2575bc060ce15f6d8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:11,615 - lava_callback - INFO - Starting async job submit for node 69408d05575bc060ce15f747 2025-12-17 22:44:11,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:11,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d05575bc060ce15f747 HTTP/1.1" 502 150 2025-12-17 22:44:11,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d05575bc060ce15f747'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:11,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d05575bc060ce15f747 2025-12-17 22:44:11,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d05575bc060ce15f747 HTTP/1.1" 502 150 2025-12-17 22:44:11,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d05575bc060ce15f747'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:12,039 - lava_callback - INFO - Received callback for node 69408f33575bc060ce1600b5 INFO: 172.21.0.1:58132 - "POST /node/69408f33575bc060ce1600b5 HTTP/1.0" 202 Accepted 2025-12-17 22:44:12,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cfd575bc060ce15f736 2025-12-17 22:44:12,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfd575bc060ce15f736 HTTP/1.1" 502 150 2025-12-17 22:44:12,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cfd575bc060ce15f736'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:12,816 - lava_callback - INFO - Received callback for node 69408f36575bc060ce1600c3 INFO: 172.21.0.1:58146 - "POST /node/69408f36575bc060ce1600c3 HTTP/1.0" 202 Accepted 2025-12-17 22:44:12,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cfe575bc060ce15f739 2025-12-17 22:44:12,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfe575bc060ce15f739 HTTP/1.1" 502 150 2025-12-17 22:44:12,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cfe575bc060ce15f739'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:13,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf4575bc060ce15f6f5 2025-12-17 22:44:13,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf4575bc060ce15f6f5 HTTP/1.1" 502 150 2025-12-17 22:44:13,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf4575bc060ce15f6f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:13,575 - lava_callback - INFO - Received callback for node 69408f3a575bc060ce1600cd INFO: 172.21.0.1:58162 - "POST /node/69408f3a575bc060ce1600cd HTTP/1.0" 202 Accepted 2025-12-17 22:44:13,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d05575bc060ce15f747 2025-12-17 22:44:13,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d05575bc060ce15f747 HTTP/1.1" 502 150 2025-12-17 22:44:13,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d05575bc060ce15f747'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:13,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d00575bc060ce15f73c 2025-12-17 22:44:13,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d00575bc060ce15f73c HTTP/1.1" 502 150 2025-12-17 22:44:13,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d00575bc060ce15f73c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:14,323 - lava_callback - INFO - Received callback for node 69408f3b575bc060ce1600d0 INFO: 172.21.0.1:58170 - "POST /node/69408f3b575bc060ce1600d0 HTTP/1.0" 202 Accepted 2025-12-17 22:44:15,140 - lava_callback - INFO - Received callback for node 69408f2d575bc060ce1600a9 INFO: 172.21.0.1:58184 - "POST /node/69408f2d575bc060ce1600a9 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:58186 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:44:15,830 - lava_callback - INFO - Received callback for node 69408f34575bc060ce1600ba INFO: 172.21.0.1:58196 - "POST /node/69408f34575bc060ce1600ba HTTP/1.0" 202 Accepted 2025-12-17 22:44:15,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce4575bc060ce15f6db 2025-12-17 22:44:15,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce4575bc060ce15f6db HTTP/1.1" 502 150 2025-12-17 22:44:15,863 - lava_callback - ERROR - Error processing callback for node 69408ce4575bc060ce15f6db: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ce4575bc060ce15f6db (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ce4575bc060ce15f6db (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ce4575bc060ce15f6db (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:15,864 - lava_callback - INFO - Starting async job submit for node 69408d09575bc060ce15f757 2025-12-17 22:44:15,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:15,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d09575bc060ce15f757 HTTP/1.1" 502 150 2025-12-17 22:44:15,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d09575bc060ce15f757'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:15,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d09575bc060ce15f757 2025-12-17 22:44:15,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d09575bc060ce15f757 HTTP/1.1" 502 150 2025-12-17 22:44:15,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d09575bc060ce15f757'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:16,492 - lava_callback - INFO - Received callback for node 69408f3c575bc060ce1600d3 INFO: 172.21.0.1:58204 - "POST /node/69408f3c575bc060ce1600d3 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:58220 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:44:17,506 - lava_callback - INFO - Received callback for node 69409533575bc060ce16126e INFO: 172.21.0.1:58232 - "POST /node/69409533575bc060ce16126e HTTP/1.0" 202 Accepted 2025-12-17 22:44:17,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d05575bc060ce15f747 2025-12-17 22:44:17,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d05575bc060ce15f747 HTTP/1.1" 502 150 2025-12-17 22:44:17,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d05575bc060ce15f747'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:17,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf8575bc060ce15f700 2025-12-17 22:44:17,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf8575bc060ce15f700 HTTP/1.1" 502 150 2025-12-17 22:44:17,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf8575bc060ce15f700'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:17,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d09575bc060ce15f757 2025-12-17 22:44:17,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d09575bc060ce15f757 HTTP/1.1" 502 150 2025-12-17 22:44:17,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d09575bc060ce15f757'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:17,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf9575bc060ce15f703 2025-12-17 22:44:17,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf9575bc060ce15f703 HTTP/1.1" 502 150 2025-12-17 22:44:17,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf9575bc060ce15f703'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:18,129 - lava_callback - INFO - Received callback for node 69409613575bc060ce1614d1 INFO: 172.21.0.1:58240 - "POST /node/69409613575bc060ce1614d1 HTTP/1.0" 202 Accepted 2025-12-17 22:44:19,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cfc575bc060ce15f733 2025-12-17 22:44:19,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfc575bc060ce15f733 HTTP/1.1" 502 150 2025-12-17 22:44:19,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cfc575bc060ce15f733'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:20,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cfd575bc060ce15f736 2025-12-17 22:44:20,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfd575bc060ce15f736 HTTP/1.1" 502 150 2025-12-17 22:44:20,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cfd575bc060ce15f736'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:20,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cfe575bc060ce15f739 2025-12-17 22:44:20,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfe575bc060ce15f739 HTTP/1.1" 502 150 2025-12-17 22:44:20,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cfe575bc060ce15f739'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:20,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce6575bc060ce15f6de 2025-12-17 22:44:20,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce6575bc060ce15f6de HTTP/1.1" 502 150 2025-12-17 22:44:20,990 - lava_callback - ERROR - Error processing callback for node 69408ce6575bc060ce15f6de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ce6575bc060ce15f6de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ce6575bc060ce15f6de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ce6575bc060ce15f6de (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:20,992 - lava_callback - INFO - Starting async job submit for node 69408d0b575bc060ce15f75a 2025-12-17 22:44:20,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:20,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0b575bc060ce15f75a HTTP/1.1" 502 150 2025-12-17 22:44:20,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0b575bc060ce15f75a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:20,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0b575bc060ce15f75a 2025-12-17 22:44:20,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0b575bc060ce15f75a HTTP/1.1" 502 150 2025-12-17 22:44:20,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0b575bc060ce15f75a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:21,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ce8575bc060ce15f6e1 2025-12-17 22:44:21,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ce8575bc060ce15f6e1 HTTP/1.1" 502 150 2025-12-17 22:44:21,650 - lava_callback - ERROR - Error processing callback for node 69408ce8575bc060ce15f6e1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ce8575bc060ce15f6e1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ce8575bc060ce15f6e1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ce8575bc060ce15f6e1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:21,652 - lava_callback - INFO - Starting async job submit for node 69408d0c575bc060ce15f762 2025-12-17 22:44:21,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:21,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0c575bc060ce15f762 HTTP/1.1" 502 150 2025-12-17 22:44:21,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0c575bc060ce15f762'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:21,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0c575bc060ce15f762 2025-12-17 22:44:21,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0c575bc060ce15f762 HTTP/1.1" 502 150 2025-12-17 22:44:21,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0c575bc060ce15f762'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:21,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d09575bc060ce15f757 2025-12-17 22:44:21,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d09575bc060ce15f757 HTTP/1.1" 502 150 2025-12-17 22:44:21,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d09575bc060ce15f757'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:21,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d00575bc060ce15f73c 2025-12-17 22:44:21,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d00575bc060ce15f73c HTTP/1.1" 502 150 2025-12-17 22:44:21,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d00575bc060ce15f73c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:22,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cec575bc060ce15f6e7 2025-12-17 22:44:22,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cec575bc060ce15f6e7 HTTP/1.1" 502 150 2025-12-17 22:44:22,766 - lava_callback - ERROR - Error processing callback for node 69408cec575bc060ce15f6e7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cec575bc060ce15f6e7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cec575bc060ce15f6e7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cec575bc060ce15f6e7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:22,768 - lava_callback - INFO - Starting async job submit for node 69408d09575bc060ce15f754 2025-12-17 22:44:22,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:22,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d09575bc060ce15f754 HTTP/1.1" 502 150 2025-12-17 22:44:22,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d09575bc060ce15f754'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:22,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d09575bc060ce15f754 2025-12-17 22:44:22,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d09575bc060ce15f754 HTTP/1.1" 502 150 2025-12-17 22:44:22,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d09575bc060ce15f754'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:22,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cea575bc060ce15f6e4 2025-12-17 22:44:22,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cea575bc060ce15f6e4 HTTP/1.1" 502 150 2025-12-17 22:44:22,778 - lava_callback - ERROR - Error processing callback for node 69408cea575bc060ce15f6e4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cea575bc060ce15f6e4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cea575bc060ce15f6e4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cea575bc060ce15f6e4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:22,780 - lava_callback - INFO - Starting async job submit for node 69408d0f575bc060ce15f768 2025-12-17 22:44:22,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:22,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0f575bc060ce15f768 HTTP/1.1" 502 150 2025-12-17 22:44:22,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0f575bc060ce15f768'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:22,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0f575bc060ce15f768 2025-12-17 22:44:22,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0f575bc060ce15f768 HTTP/1.1" 502 150 2025-12-17 22:44:22,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0f575bc060ce15f768'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:22,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0b575bc060ce15f75a 2025-12-17 22:44:23,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0b575bc060ce15f75a HTTP/1.1" 502 150 2025-12-17 22:44:23,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0b575bc060ce15f75a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:23,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0c575bc060ce15f762 2025-12-17 22:44:23,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0c575bc060ce15f762 HTTP/1.1" 502 150 2025-12-17 22:44:23,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0c575bc060ce15f762'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:24,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cee575bc060ce15f6ea 2025-12-17 22:44:24,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cee575bc060ce15f6ea HTTP/1.1" 502 150 2025-12-17 22:44:24,478 - lava_callback - ERROR - Error processing callback for node 69408cee575bc060ce15f6ea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cee575bc060ce15f6ea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cee575bc060ce15f6ea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cee575bc060ce15f6ea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:24,479 - lava_callback - INFO - Starting async job submit for node 69408d11575bc060ce15f76b 2025-12-17 22:44:24,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:24,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d11575bc060ce15f76b HTTP/1.1" 502 150 2025-12-17 22:44:24,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d11575bc060ce15f76b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:24,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d11575bc060ce15f76b 2025-12-17 22:44:24,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d11575bc060ce15f76b HTTP/1.1" 502 150 2025-12-17 22:44:24,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d11575bc060ce15f76b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:24,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf2575bc060ce15f6f2 2025-12-17 22:44:24,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf2575bc060ce15f6f2 HTTP/1.1" 502 150 2025-12-17 22:44:24,568 - lava_callback - ERROR - Error processing callback for node 69408cf2575bc060ce15f6f2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf2575bc060ce15f6f2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf2575bc060ce15f6f2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf2575bc060ce15f6f2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:24,570 - lava_callback - INFO - Starting async job submit for node 69408d0d575bc060ce15f765 2025-12-17 22:44:24,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:24,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0d575bc060ce15f765 HTTP/1.1" 502 150 2025-12-17 22:44:24,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0d575bc060ce15f765'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:24,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0d575bc060ce15f765 2025-12-17 22:44:24,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0d575bc060ce15f765 HTTP/1.1" 502 150 2025-12-17 22:44:24,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0d575bc060ce15f765'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:24,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d09575bc060ce15f754 2025-12-17 22:44:24,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d09575bc060ce15f754 HTTP/1.1" 502 150 2025-12-17 22:44:24,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d09575bc060ce15f754'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:24,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0f575bc060ce15f768 2025-12-17 22:44:24,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0f575bc060ce15f768 HTTP/1.1" 502 150 2025-12-17 22:44:24,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0f575bc060ce15f768'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:25,474 - lava_callback - INFO - Received callback for node 69408f37575bc060ce1600c8 INFO: 172.21.0.1:34940 - "POST /node/69408f37575bc060ce1600c8 HTTP/1.0" 202 Accepted 2025-12-17 22:44:25,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d05575bc060ce15f747 2025-12-17 22:44:25,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d05575bc060ce15f747 HTTP/1.1" 502 150 2025-12-17 22:44:25,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d05575bc060ce15f747'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:26,083 - lava_callback - INFO - Received callback for node 69408f44575bc060ce1600db INFO: 172.21.0.1:34956 - "POST /node/69408f44575bc060ce1600db HTTP/1.0" 202 Accepted 2025-12-17 22:44:26,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d11575bc060ce15f76b 2025-12-17 22:44:26,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d11575bc060ce15f76b HTTP/1.1" 502 150 2025-12-17 22:44:26,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d11575bc060ce15f76b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34964 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:44:26,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0d575bc060ce15f765 2025-12-17 22:44:26,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0d575bc060ce15f765 HTTP/1.1" 502 150 2025-12-17 22:44:26,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0d575bc060ce15f765'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:26,967 - lava_callback - INFO - Received callback for node 69408f46575bc060ce1600de INFO: 172.21.0.1:34976 - "POST /node/69408f46575bc060ce1600de HTTP/1.0" 202 Accepted 2025-12-17 22:44:27,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0b575bc060ce15f75a 2025-12-17 22:44:27,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0b575bc060ce15f75a HTTP/1.1" 502 150 2025-12-17 22:44:27,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0b575bc060ce15f75a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:27,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0c575bc060ce15f762 2025-12-17 22:44:27,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0c575bc060ce15f762 HTTP/1.1" 502 150 2025-12-17 22:44:27,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0c575bc060ce15f762'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:28,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d09575bc060ce15f754 2025-12-17 22:44:28,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d09575bc060ce15f754 HTTP/1.1" 502 150 2025-12-17 22:44:28,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d09575bc060ce15f754'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:28,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0f575bc060ce15f768 2025-12-17 22:44:28,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0f575bc060ce15f768 HTTP/1.1" 502 150 2025-12-17 22:44:28,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0f575bc060ce15f768'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:29,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf4575bc060ce15f6f5 2025-12-17 22:44:29,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf4575bc060ce15f6f5 HTTP/1.1" 502 150 2025-12-17 22:44:29,018 - lava_callback - ERROR - Error processing callback for node 69408cf4575bc060ce15f6f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf4575bc060ce15f6f5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf4575bc060ce15f6f5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf4575bc060ce15f6f5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:29,021 - lava_callback - INFO - Starting async job submit for node 69408d25575bc060ce15f794 2025-12-17 22:44:29,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:29,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d25575bc060ce15f794 HTTP/1.1" 502 150 2025-12-17 22:44:29,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d25575bc060ce15f794'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:29,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d25575bc060ce15f794 2025-12-17 22:44:29,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d25575bc060ce15f794 HTTP/1.1" 502 150 2025-12-17 22:44:29,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d25575bc060ce15f794'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:29,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d09575bc060ce15f757 2025-12-17 22:44:29,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d09575bc060ce15f757 HTTP/1.1" 502 150 2025-12-17 22:44:29,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d09575bc060ce15f757'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:30,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d11575bc060ce15f76b 2025-12-17 22:44:30,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d11575bc060ce15f76b HTTP/1.1" 502 150 2025-12-17 22:44:30,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d11575bc060ce15f76b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:30,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0d575bc060ce15f765 2025-12-17 22:44:30,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0d575bc060ce15f765 HTTP/1.1" 502 150 2025-12-17 22:44:30,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0d575bc060ce15f765'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:31,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d25575bc060ce15f794 2025-12-17 22:44:31,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d25575bc060ce15f794 HTTP/1.1" 502 150 2025-12-17 22:44:31,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d25575bc060ce15f794'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:32,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf6575bc060ce15f6fd HTTP/1.1" 502 150 2025-12-17 22:44:32,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf6575bc060ce15f6fd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:32,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf6575bc060ce15f6fd 2025-12-17 22:44:32,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf6575bc060ce15f6fd HTTP/1.1" 502 150 2025-12-17 22:44:32,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf6575bc060ce15f6fd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:33,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf8575bc060ce15f700 2025-12-17 22:44:33,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf8575bc060ce15f700 HTTP/1.1" 502 150 2025-12-17 22:44:33,826 - lava_callback - ERROR - Error processing callback for node 69408cf8575bc060ce15f700: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf8575bc060ce15f700 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf8575bc060ce15f700 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf8575bc060ce15f700 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:33,827 - lava_callback - INFO - Starting async job submit for node 69408d26575bc060ce15f797 2025-12-17 22:44:33,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:33,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d26575bc060ce15f797 HTTP/1.1" 502 150 2025-12-17 22:44:33,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d26575bc060ce15f797'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:33,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d26575bc060ce15f797 2025-12-17 22:44:33,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d26575bc060ce15f797 HTTP/1.1" 502 150 2025-12-17 22:44:33,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d26575bc060ce15f797'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:33,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf9575bc060ce15f703 2025-12-17 22:44:33,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf9575bc060ce15f703 HTTP/1.1" 502 150 2025-12-17 22:44:33,878 - lava_callback - ERROR - Error processing callback for node 69408cf9575bc060ce15f703: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf9575bc060ce15f703 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf9575bc060ce15f703 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf9575bc060ce15f703 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:33,881 - lava_callback - INFO - Starting async job submit for node 69408d27575bc060ce15f79a 2025-12-17 22:44:33,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:33,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d27575bc060ce15f79a HTTP/1.1" 502 150 2025-12-17 22:44:33,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d27575bc060ce15f79a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:33,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d27575bc060ce15f79a 2025-12-17 22:44:33,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d27575bc060ce15f79a HTTP/1.1" 502 150 2025-12-17 22:44:33,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d27575bc060ce15f79a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:34,671 - lava_callback - INFO - Received callback for node 69408f49575bc060ce1600e8 INFO: 172.21.0.1:36802 - "POST /node/69408f49575bc060ce1600e8 HTTP/1.0" 202 Accepted 2025-12-17 22:44:34,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf6575bc060ce15f6fd 2025-12-17 22:44:34,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf6575bc060ce15f6fd HTTP/1.1" 502 150 2025-12-17 22:44:34,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf6575bc060ce15f6fd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:34,951 - lava_callback - INFO - Received callback for node 69409228575bc060ce160a0f INFO: 172.21.0.1:36812 - "POST /node/69409228575bc060ce160a0f HTTP/1.0" 202 Accepted 2025-12-17 22:44:35,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0b575bc060ce15f75a 2025-12-17 22:44:35,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0b575bc060ce15f75a HTTP/1.1" 502 150 2025-12-17 22:44:35,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0b575bc060ce15f75a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:35,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d25575bc060ce15f794 2025-12-17 22:44:35,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d25575bc060ce15f794 HTTP/1.1" 502 150 2025-12-17 22:44:35,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d25575bc060ce15f794'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:35,164 - lava_callback - INFO - Received callback for node 69408f42575bc060ce1600d8 INFO: 172.21.0.1:36822 - "POST /node/69408f42575bc060ce1600d8 HTTP/1.0" 202 Accepted 2025-12-17 22:44:35,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cfc575bc060ce15f733 2025-12-17 22:44:35,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfc575bc060ce15f733 HTTP/1.1" 502 150 2025-12-17 22:44:35,594 - lava_callback - ERROR - Error processing callback for node 69408cfc575bc060ce15f733: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cfc575bc060ce15f733 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cfc575bc060ce15f733 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cfc575bc060ce15f733 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:35,596 - lava_callback - INFO - Starting async job submit for node 69408d17575bc060ce15f770 2025-12-17 22:44:35,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:35,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d17575bc060ce15f770 HTTP/1.1" 502 150 2025-12-17 22:44:35,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d17575bc060ce15f770'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:35,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d17575bc060ce15f770 2025-12-17 22:44:35,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d17575bc060ce15f770 HTTP/1.1" 502 150 2025-12-17 22:44:35,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d17575bc060ce15f770'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:35,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0c575bc060ce15f762 2025-12-17 22:44:35,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0c575bc060ce15f762 HTTP/1.1" 502 150 2025-12-17 22:44:35,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0c575bc060ce15f762'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:35,806 - lava_callback - INFO - Received callback for node 69408f47575bc060ce1600e1 INFO: 172.21.0.1:36830 - "POST /node/69408f47575bc060ce1600e1 HTTP/1.0" 202 Accepted 2025-12-17 22:44:35,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d26575bc060ce15f797 2025-12-17 22:44:35,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d26575bc060ce15f797 HTTP/1.1" 502 150 2025-12-17 22:44:35,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d26575bc060ce15f797'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:35,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d27575bc060ce15f79a 2025-12-17 22:44:35,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d27575bc060ce15f79a HTTP/1.1" 502 150 2025-12-17 22:44:35,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d27575bc060ce15f79a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:36,090 - lava_callback - INFO - Received callback for node 6942c65d1b61f1c6005ff1fb INFO: 172.21.0.1:36836 - "POST /node/6942c65d1b61f1c6005ff1fb HTTP/1.0" 202 Accepted INFO: 172.21.0.1:36846 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:44:36,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cfd575bc060ce15f736 2025-12-17 22:44:36,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfd575bc060ce15f736 HTTP/1.1" 502 150 2025-12-17 22:44:36,704 - lava_callback - ERROR - Error processing callback for node 69408cfd575bc060ce15f736: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cfd575bc060ce15f736 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cfd575bc060ce15f736 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cfd575bc060ce15f736 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:36,707 - lava_callback - INFO - Starting async job submit for node 69408d1a575bc060ce15f77b 2025-12-17 22:44:36,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:36,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1a575bc060ce15f77b HTTP/1.1" 502 150 2025-12-17 22:44:36,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d1a575bc060ce15f77b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:36,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d1a575bc060ce15f77b 2025-12-17 22:44:36,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1a575bc060ce15f77b HTTP/1.1" 502 150 2025-12-17 22:44:36,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d1a575bc060ce15f77b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:36,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d09575bc060ce15f754 2025-12-17 22:44:36,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d09575bc060ce15f754 HTTP/1.1" 502 150 2025-12-17 22:44:36,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d09575bc060ce15f754'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:36,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0f575bc060ce15f768 2025-12-17 22:44:36,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0f575bc060ce15f768 HTTP/1.1" 502 150 2025-12-17 22:44:36,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0f575bc060ce15f768'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:36,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cfe575bc060ce15f739 2025-12-17 22:44:36,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cfe575bc060ce15f739 HTTP/1.1" 502 150 2025-12-17 22:44:36,945 - lava_callback - ERROR - Error processing callback for node 69408cfe575bc060ce15f739: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cfe575bc060ce15f739 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cfe575bc060ce15f739 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cfe575bc060ce15f739 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:36,947 - lava_callback - INFO - Starting async job submit for node 69408d1c575bc060ce15f780 2025-12-17 22:44:36,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:36,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1c575bc060ce15f780 HTTP/1.1" 502 150 2025-12-17 22:44:36,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d1c575bc060ce15f780'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:36,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d1c575bc060ce15f780 2025-12-17 22:44:36,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1c575bc060ce15f780 HTTP/1.1" 502 150 2025-12-17 22:44:36,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d1c575bc060ce15f780'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:37,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d17575bc060ce15f770 2025-12-17 22:44:37,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d17575bc060ce15f770 HTTP/1.1" 502 150 2025-12-17 22:44:37,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d17575bc060ce15f770'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:37,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d00575bc060ce15f73c 2025-12-17 22:44:37,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d00575bc060ce15f73c HTTP/1.1" 502 150 2025-12-17 22:44:37,988 - lava_callback - ERROR - Error processing callback for node 69408d00575bc060ce15f73c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d00575bc060ce15f73c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d00575bc060ce15f73c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d00575bc060ce15f73c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:37,990 - lava_callback - INFO - Starting async job submit for node 69408d1e575bc060ce15f783 2025-12-17 22:44:37,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:37,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1e575bc060ce15f783 HTTP/1.1" 502 150 2025-12-17 22:44:37,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d1e575bc060ce15f783'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:37,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d1e575bc060ce15f783 2025-12-17 22:44:37,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1e575bc060ce15f783 HTTP/1.1" 502 150 2025-12-17 22:44:37,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d1e575bc060ce15f783'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:38,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d11575bc060ce15f76b 2025-12-17 22:44:38,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d11575bc060ce15f76b HTTP/1.1" 502 150 2025-12-17 22:44:38,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d11575bc060ce15f76b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:38,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0d575bc060ce15f765 2025-12-17 22:44:38,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0d575bc060ce15f765 HTTP/1.1" 502 150 2025-12-17 22:44:38,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d0d575bc060ce15f765'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:38,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d1a575bc060ce15f77b 2025-12-17 22:44:38,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1a575bc060ce15f77b HTTP/1.1" 502 150 2025-12-17 22:44:38,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d1a575bc060ce15f77b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:38,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf6575bc060ce15f6fd 2025-12-17 22:44:38,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf6575bc060ce15f6fd HTTP/1.1" 502 150 2025-12-17 22:44:38,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf6575bc060ce15f6fd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:38,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d1c575bc060ce15f780 2025-12-17 22:44:38,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1c575bc060ce15f780 HTTP/1.1" 502 150 2025-12-17 22:44:38,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d1c575bc060ce15f780'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:39,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d26575bc060ce15f797 2025-12-17 22:44:39,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d27575bc060ce15f79a 2025-12-17 22:44:39,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d27575bc060ce15f79a HTTP/1.1" 502 150 2025-12-17 22:44:39,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d27575bc060ce15f79a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:40,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d1e575bc060ce15f783 2025-12-17 22:44:40,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1e575bc060ce15f783 HTTP/1.1" 502 150 2025-12-17 22:44:40,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d1e575bc060ce15f783'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:41,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d17575bc060ce15f770 2025-12-17 22:44:41,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d17575bc060ce15f770 HTTP/1.1" 502 150 2025-12-17 22:44:41,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d17575bc060ce15f770'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:41,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d05575bc060ce15f747 2025-12-17 22:44:41,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d05575bc060ce15f747 HTTP/1.1" 502 150 2025-12-17 22:44:41,631 - lava_callback - ERROR - Error processing callback for node 69408d05575bc060ce15f747: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d05575bc060ce15f747 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d05575bc060ce15f747 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d05575bc060ce15f747 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:41,633 - lava_callback - INFO - Starting async job submit for node 6940c5381e3c24e95681d6aa 2025-12-17 22:44:41,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:41,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5381e3c24e95681d6aa HTTP/1.1" 502 150 2025-12-17 22:44:41,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5381e3c24e95681d6aa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:41,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5381e3c24e95681d6aa 2025-12-17 22:44:41,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5381e3c24e95681d6aa HTTP/1.1" 502 150 2025-12-17 22:44:41,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5381e3c24e95681d6aa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:42,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d1a575bc060ce15f77b 2025-12-17 22:44:42,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1a575bc060ce15f77b HTTP/1.1" 502 150 2025-12-17 22:44:42,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d1a575bc060ce15f77b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:42,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d1c575bc060ce15f780 2025-12-17 22:44:42,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1c575bc060ce15f780 HTTP/1.1" 502 150 2025-12-17 22:44:42,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d1c575bc060ce15f780'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:43,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d25575bc060ce15f794 2025-12-17 22:44:43,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5381e3c24e95681d6aa 2025-12-17 22:44:43,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5381e3c24e95681d6aa HTTP/1.1" 502 150 2025-12-17 22:44:43,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5381e3c24e95681d6aa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:44,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d1e575bc060ce15f783 2025-12-17 22:44:44,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1e575bc060ce15f783 HTTP/1.1" 502 150 2025-12-17 22:44:44,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d1e575bc060ce15f783'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:45,501 - lava_callback - INFO - Received callback for node 69408f4a575bc060ce1600f6 INFO: 172.21.0.1:54286 - "POST /node/69408f4a575bc060ce1600f6 HTTP/1.0" 202 Accepted 2025-12-17 22:44:45,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d09575bc060ce15f757 2025-12-17 22:44:45,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d09575bc060ce15f757 HTTP/1.1" 502 150 2025-12-17 22:44:45,879 - lava_callback - ERROR - Error processing callback for node 69408d09575bc060ce15f757: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d09575bc060ce15f757 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d09575bc060ce15f757 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d09575bc060ce15f757 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:45,881 - lava_callback - INFO - Starting async job submit for node 69408d20575bc060ce15f786 2025-12-17 22:44:45,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:45,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d20575bc060ce15f786 HTTP/1.1" 502 150 2025-12-17 22:44:45,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d20575bc060ce15f786'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:45,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d20575bc060ce15f786 2025-12-17 22:44:45,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d20575bc060ce15f786 HTTP/1.1" 502 150 2025-12-17 22:44:45,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d20575bc060ce15f786'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:46,142 - lava_callback - INFO - Received callback for node 69408f4d575bc060ce1600fd INFO: 172.21.0.1:54302 - "POST /node/69408f4d575bc060ce1600fd HTTP/1.0" 202 Accepted INFO: 172.21.0.1:54314 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:44:46,799 - lava_callback - INFO - Received callback for node 69408f4f575bc060ce160100 INFO: 172.21.0.1:54326 - "POST /node/69408f4f575bc060ce160100 HTTP/1.0" 202 Accepted 2025-12-17 22:44:46,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf6575bc060ce15f6fd 2025-12-17 22:44:46,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf6575bc060ce15f6fd HTTP/1.1" 502 150 2025-12-17 22:44:46,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408cf6575bc060ce15f6fd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:47,447 - lava_callback - INFO - Received callback for node 69408f51575bc060ce160105 INFO: 172.21.0.1:54332 - "POST /node/69408f51575bc060ce160105 HTTP/1.0" 202 Accepted 2025-12-17 22:44:47,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5381e3c24e95681d6aa 2025-12-17 22:44:47,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5381e3c24e95681d6aa HTTP/1.1" 502 150 2025-12-17 22:44:47,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5381e3c24e95681d6aa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:47,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d20575bc060ce15f786 2025-12-17 22:44:47,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d20575bc060ce15f786 HTTP/1.1" 502 150 2025-12-17 22:44:47,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d20575bc060ce15f786'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:47,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d27575bc060ce15f79a 2025-12-17 22:44:47,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d27575bc060ce15f79a HTTP/1.1" 502 150 2025-12-17 22:44:47,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d27575bc060ce15f79a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:48,078 - lava_callback - INFO - Received callback for node 69408f54575bc060ce16010b INFO: 172.21.0.1:54342 - "POST /node/69408f54575bc060ce16010b HTTP/1.0" 202 Accepted 2025-12-17 22:44:48,715 - lava_callback - INFO - Received callback for node 69408f56575bc060ce16010e INFO: 172.21.0.1:54356 - "POST /node/69408f56575bc060ce16010e HTTP/1.0" 202 Accepted 2025-12-17 22:44:49,364 - lava_callback - INFO - Received callback for node 69408f58575bc060ce160111 INFO: 172.21.0.1:54362 - "POST /node/69408f58575bc060ce160111 HTTP/1.0" 202 Accepted 2025-12-17 22:44:49,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d17575bc060ce15f770 2025-12-17 22:44:49,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d17575bc060ce15f770 HTTP/1.1" 502 150 2025-12-17 22:44:49,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d17575bc060ce15f770'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:50,035 - lava_callback - INFO - Received callback for node 69408f5a575bc060ce16011b INFO: 172.21.0.1:57992 - "POST /node/69408f5a575bc060ce16011b HTTP/1.0" 202 Accepted 2025-12-17 22:44:50,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d1a575bc060ce15f77b 2025-12-17 22:44:50,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1a575bc060ce15f77b HTTP/1.1" 502 150 2025-12-17 22:44:50,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d1a575bc060ce15f77b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:50,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d1c575bc060ce15f780 2025-12-17 22:44:50,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1c575bc060ce15f780 HTTP/1.1" 502 150 2025-12-17 22:44:50,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d1c575bc060ce15f780'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:51,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0b575bc060ce15f75a 2025-12-17 22:44:51,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0b575bc060ce15f75a HTTP/1.1" 502 150 2025-12-17 22:44:51,005 - lava_callback - ERROR - Error processing callback for node 69408d0b575bc060ce15f75a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d0b575bc060ce15f75a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d0b575bc060ce15f75a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d0b575bc060ce15f75a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:51,007 - lava_callback - INFO - Starting async job submit for node 69408d22575bc060ce15f791 2025-12-17 22:44:51,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:51,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d22575bc060ce15f791 HTTP/1.1" 502 150 2025-12-17 22:44:51,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d22575bc060ce15f791'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:51,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d22575bc060ce15f791 2025-12-17 22:44:51,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d22575bc060ce15f791 HTTP/1.1" 502 150 2025-12-17 22:44:51,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d22575bc060ce15f791'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:51,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0c575bc060ce15f762 2025-12-17 22:44:51,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0c575bc060ce15f762 HTTP/1.1" 502 150 2025-12-17 22:44:51,679 - lava_callback - ERROR - Error processing callback for node 69408d0c575bc060ce15f762: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d0c575bc060ce15f762 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d0c575bc060ce15f762 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d0c575bc060ce15f762 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:51,681 - lava_callback - INFO - Starting async job submit for node 69408d2a575bc060ce15f7a0 2025-12-17 22:44:51,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:51,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2a575bc060ce15f7a0 HTTP/1.1" 502 150 2025-12-17 22:44:51,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d2a575bc060ce15f7a0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:51,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d2a575bc060ce15f7a0 2025-12-17 22:44:51,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2a575bc060ce15f7a0 HTTP/1.1" 502 150 2025-12-17 22:44:51,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d2a575bc060ce15f7a0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:51,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d20575bc060ce15f786 2025-12-17 22:44:51,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d20575bc060ce15f786 HTTP/1.1" 502 150 2025-12-17 22:44:51,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d20575bc060ce15f786'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:52,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d1e575bc060ce15f783 2025-12-17 22:44:52,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1e575bc060ce15f783 HTTP/1.1" 502 150 2025-12-17 22:44:52,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d1e575bc060ce15f783'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:52,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d09575bc060ce15f754 2025-12-17 22:44:52,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d09575bc060ce15f754 HTTP/1.1" 502 150 2025-12-17 22:44:52,780 - lava_callback - ERROR - Error processing callback for node 69408d09575bc060ce15f754: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d09575bc060ce15f754 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d09575bc060ce15f754 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d09575bc060ce15f754 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:52,782 - lava_callback - INFO - Starting async job submit for node 69408e4e575bc060ce15fda9 2025-12-17 22:44:52,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0f575bc060ce15f768 2025-12-17 22:44:52,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0f575bc060ce15f768 HTTP/1.1" 502 150 2025-12-17 22:44:52,808 - lava_callback - ERROR - Error processing callback for node 69408d0f575bc060ce15f768: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d0f575bc060ce15f768 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d0f575bc060ce15f768 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d0f575bc060ce15f768 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:52,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:52,811 - lava_callback - INFO - Starting async job submit for node 69408d2d575bc060ce15f7ad 2025-12-17 22:44:52,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:52,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4e575bc060ce15fda9 HTTP/1.1" 502 150 2025-12-17 22:44:52,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4e575bc060ce15fda9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:52,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4e575bc060ce15fda9 2025-12-17 22:44:52,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4e575bc060ce15fda9 HTTP/1.1" 502 150 2025-12-17 22:44:52,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4e575bc060ce15fda9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:52,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2d575bc060ce15f7ad HTTP/1.1" 502 150 2025-12-17 22:44:52,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d2d575bc060ce15f7ad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:52,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d2d575bc060ce15f7ad 2025-12-17 22:44:52,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2d575bc060ce15f7ad HTTP/1.1" 502 150 2025-12-17 22:44:52,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d2d575bc060ce15f7ad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:53,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d22575bc060ce15f791 2025-12-17 22:44:53,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d22575bc060ce15f791 HTTP/1.1" 502 150 2025-12-17 22:44:53,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d22575bc060ce15f791'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:53,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d2a575bc060ce15f7a0 2025-12-17 22:44:53,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2a575bc060ce15f7a0 HTTP/1.1" 502 150 2025-12-17 22:44:53,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d2a575bc060ce15f7a0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:54,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d11575bc060ce15f76b 2025-12-17 22:44:54,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d0d575bc060ce15f765 2025-12-17 22:44:54,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d0d575bc060ce15f765 HTTP/1.1" 502 150 2025-12-17 22:44:54,604 - lava_callback - ERROR - Error processing callback for node 69408d0d575bc060ce15f765: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d0d575bc060ce15f765 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d0d575bc060ce15f765 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d0d575bc060ce15f765 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:44:54,605 - lava_callback - INFO - Starting async job submit for node 69408d2f575bc060ce15f7b4 2025-12-17 22:44:54,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:44:54,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2f575bc060ce15f7b4 HTTP/1.1" 502 150 2025-12-17 22:44:54,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d2f575bc060ce15f7b4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:54,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d2f575bc060ce15f7b4 2025-12-17 22:44:54,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2f575bc060ce15f7b4 HTTP/1.1" 502 150 2025-12-17 22:44:54,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d2f575bc060ce15f7b4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:54,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4e575bc060ce15fda9 2025-12-17 22:44:54,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4e575bc060ce15fda9 HTTP/1.1" 502 150 2025-12-17 22:44:54,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4e575bc060ce15fda9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:54,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d2d575bc060ce15f7ad 2025-12-17 22:44:54,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2d575bc060ce15f7ad HTTP/1.1" 502 150 2025-12-17 22:44:54,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d2d575bc060ce15f7ad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:55,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d26575bc060ce15f797 HTTP/1.1" 502 150 2025-12-17 22:44:55,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d26575bc060ce15f797'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:55,493 - lava_callback - INFO - Received callback for node 69408e9a575bc060ce15fe9d INFO: 172.21.0.1:58008 - "POST /node/69408e9a575bc060ce15fe9d HTTP/1.0" 202 Accepted 2025-12-17 22:44:55,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5381e3c24e95681d6aa 2025-12-17 22:44:55,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5381e3c24e95681d6aa HTTP/1.1" 502 150 2025-12-17 22:44:55,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5381e3c24e95681d6aa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58024 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:44:56,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d2f575bc060ce15f7b4 2025-12-17 22:44:56,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2f575bc060ce15f7b4 HTTP/1.1" 502 150 2025-12-17 22:44:56,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d2f575bc060ce15f7b4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:57,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d22575bc060ce15f791 2025-12-17 22:44:57,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d22575bc060ce15f791 HTTP/1.1" 502 150 2025-12-17 22:44:57,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d22575bc060ce15f791'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:57,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d2a575bc060ce15f7a0 2025-12-17 22:44:57,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2a575bc060ce15f7a0 HTTP/1.1" 502 150 2025-12-17 22:44:57,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d2a575bc060ce15f7a0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:58,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d25575bc060ce15f794 HTTP/1.1" 502 150 2025-12-17 22:44:58,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d25575bc060ce15f794'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:58,670 - lava_callback - INFO - Received callback for node 69408c27575bc060ce15f47d INFO: 172.21.0.1:58026 - "POST /node/69408c27575bc060ce15f47d HTTP/1.0" 202 Accepted 2025-12-17 22:44:58,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4e575bc060ce15fda9 2025-12-17 22:44:58,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4e575bc060ce15fda9 HTTP/1.1" 502 150 2025-12-17 22:44:58,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4e575bc060ce15fda9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:58,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d2d575bc060ce15f7ad 2025-12-17 22:44:58,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2d575bc060ce15f7ad HTTP/1.1" 502 150 2025-12-17 22:44:58,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d2d575bc060ce15f7ad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:44:59,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d20575bc060ce15f786 2025-12-17 22:44:59,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d20575bc060ce15f786 HTTP/1.1" 502 150 2025-12-17 22:44:59,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d20575bc060ce15f786'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:00,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d2f575bc060ce15f7b4 2025-12-17 22:45:00,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2f575bc060ce15f7b4 HTTP/1.1" 502 150 2025-12-17 22:45:00,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d2f575bc060ce15f7b4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33898 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:45:02,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408cf6575bc060ce15f6fd 2025-12-17 22:45:02,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408cf6575bc060ce15f6fd HTTP/1.1" 502 150 2025-12-17 22:45:02,823 - lava_callback - ERROR - Error processing callback for node 69408cf6575bc060ce15f6fd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf6575bc060ce15f6fd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf6575bc060ce15f6fd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408cf6575bc060ce15f6fd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:02,825 - lava_callback - INFO - Starting async job submit for node 69408d31575bc060ce15f7b7 2025-12-17 22:45:02,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:02,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d31575bc060ce15f7b7 HTTP/1.1" 502 150 2025-12-17 22:45:02,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d31575bc060ce15f7b7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:02,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d31575bc060ce15f7b7 2025-12-17 22:45:02,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d31575bc060ce15f7b7 HTTP/1.1" 502 150 2025-12-17 22:45:02,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d31575bc060ce15f7b7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:03,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d26575bc060ce15f797 2025-12-17 22:45:03,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d26575bc060ce15f797 HTTP/1.1" 502 150 2025-12-17 22:45:03,090 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d26575bc060ce15f797'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:03,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d27575bc060ce15f79a 2025-12-17 22:45:03,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d27575bc060ce15f79a HTTP/1.1" 502 150 2025-12-17 22:45:03,918 - lava_callback - ERROR - Error processing callback for node 69408d27575bc060ce15f79a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d27575bc060ce15f79a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d27575bc060ce15f79a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d27575bc060ce15f79a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:03,920 - lava_callback - INFO - Starting async job submit for node 69408d33575bc060ce15f7ba 2025-12-17 22:45:03,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:03,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d33575bc060ce15f7ba HTTP/1.1" 502 150 2025-12-17 22:45:03,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d33575bc060ce15f7ba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:03,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d33575bc060ce15f7ba 2025-12-17 22:45:03,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d33575bc060ce15f7ba HTTP/1.1" 502 150 2025-12-17 22:45:03,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d33575bc060ce15f7ba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:04,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d31575bc060ce15f7b7 2025-12-17 22:45:04,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d31575bc060ce15f7b7 HTTP/1.1" 502 150 2025-12-17 22:45:04,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d31575bc060ce15f7b7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:05,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d22575bc060ce15f791 2025-12-17 22:45:05,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d22575bc060ce15f791 HTTP/1.1" 502 150 2025-12-17 22:45:05,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d22575bc060ce15f791'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:05,509 - lava_callback - INFO - Received callback for node 69408f5d575bc060ce16012e INFO: 172.21.0.1:33914 - "POST /node/69408f5d575bc060ce16012e HTTP/1.0" 202 Accepted 2025-12-17 22:45:05,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d17575bc060ce15f770 2025-12-17 22:45:05,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d17575bc060ce15f770 HTTP/1.1" 502 150 2025-12-17 22:45:05,613 - lava_callback - ERROR - Error processing callback for node 69408d17575bc060ce15f770: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d17575bc060ce15f770 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d17575bc060ce15f770 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d17575bc060ce15f770 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:05,616 - lava_callback - INFO - Starting async job submit for node 69408d35575bc060ce15f7bd 2025-12-17 22:45:05,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:05,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d35575bc060ce15f7bd HTTP/1.1" 502 150 2025-12-17 22:45:05,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d35575bc060ce15f7bd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:05,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d35575bc060ce15f7bd 2025-12-17 22:45:05,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d35575bc060ce15f7bd HTTP/1.1" 502 150 2025-12-17 22:45:05,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d35575bc060ce15f7bd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:05,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d2a575bc060ce15f7a0 2025-12-17 22:45:05,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2a575bc060ce15f7a0 HTTP/1.1" 502 150 2025-12-17 22:45:05,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d2a575bc060ce15f7a0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:05,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d33575bc060ce15f7ba 2025-12-17 22:45:05,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d33575bc060ce15f7ba HTTP/1.1" 502 150 2025-12-17 22:45:05,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d33575bc060ce15f7ba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:06,152 - lava_callback - INFO - Received callback for node 69408f5f575bc060ce160139 INFO: 172.21.0.1:33916 - "POST /node/69408f5f575bc060ce160139 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:33922 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:45:06,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d1a575bc060ce15f77b 2025-12-17 22:45:06,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1a575bc060ce15f77b HTTP/1.1" 502 150 2025-12-17 22:45:06,728 - lava_callback - ERROR - Error processing callback for node 69408d1a575bc060ce15f77b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d1a575bc060ce15f77b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d1a575bc060ce15f77b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d1a575bc060ce15f77b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:06,729 - lava_callback - INFO - Starting async job submit for node 69408d37575bc060ce15f7c0 2025-12-17 22:45:06,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:06,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d37575bc060ce15f7c0 HTTP/1.1" 502 150 2025-12-17 22:45:06,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d37575bc060ce15f7c0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:06,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d37575bc060ce15f7c0 2025-12-17 22:45:06,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d37575bc060ce15f7c0 HTTP/1.1" 502 150 2025-12-17 22:45:06,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d37575bc060ce15f7c0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:06,801 - lava_callback - INFO - Received callback for node 69408f62575bc060ce160140 INFO: 172.21.0.1:33930 - "POST /node/69408f62575bc060ce160140 HTTP/1.0" 202 Accepted 2025-12-17 22:45:06,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4e575bc060ce15fda9 2025-12-17 22:45:06,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4e575bc060ce15fda9 HTTP/1.1" 502 150 2025-12-17 22:45:06,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4e575bc060ce15fda9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:06,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d2d575bc060ce15f7ad 2025-12-17 22:45:06,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2d575bc060ce15f7ad HTTP/1.1" 502 150 2025-12-17 22:45:06,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d2d575bc060ce15f7ad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:06,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d1c575bc060ce15f780 2025-12-17 22:45:06,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1c575bc060ce15f780 HTTP/1.1" 502 150 2025-12-17 22:45:06,962 - lava_callback - ERROR - Error processing callback for node 69408d1c575bc060ce15f780: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d1c575bc060ce15f780 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d1c575bc060ce15f780 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d1c575bc060ce15f780 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:06,963 - lava_callback - INFO - Starting async job submit for node 69408d3c575bc060ce15f7e8 2025-12-17 22:45:06,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:06,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3c575bc060ce15f7e8 HTTP/1.1" 502 150 2025-12-17 22:45:06,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d3c575bc060ce15f7e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:06,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d3c575bc060ce15f7e8 2025-12-17 22:45:06,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3c575bc060ce15f7e8 HTTP/1.1" 502 150 2025-12-17 22:45:06,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d3c575bc060ce15f7e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:07,470 - lava_callback - INFO - Received callback for node 69408f64575bc060ce160143 INFO: 172.21.0.1:33946 - "POST /node/69408f64575bc060ce160143 HTTP/1.0" 202 Accepted 2025-12-17 22:45:07,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d35575bc060ce15f7bd 2025-12-17 22:45:07,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d35575bc060ce15f7bd HTTP/1.1" 502 150 2025-12-17 22:45:07,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d35575bc060ce15f7bd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:08,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d1e575bc060ce15f783 2025-12-17 22:45:08,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d1e575bc060ce15f783 HTTP/1.1" 502 150 2025-12-17 22:45:08,005 - lava_callback - ERROR - Error processing callback for node 69408d1e575bc060ce15f783: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d1e575bc060ce15f783 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d1e575bc060ce15f783 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d1e575bc060ce15f783 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:08,007 - lava_callback - INFO - Starting async job submit for node 69408d3f575bc060ce15f7eb 2025-12-17 22:45:08,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:08,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3f575bc060ce15f7eb HTTP/1.1" 502 150 2025-12-17 22:45:08,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d3f575bc060ce15f7eb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:08,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d3f575bc060ce15f7eb 2025-12-17 22:45:08,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3f575bc060ce15f7eb HTTP/1.1" 502 150 2025-12-17 22:45:08,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d3f575bc060ce15f7eb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:08,136 - lava_callback - INFO - Received callback for node 69408f68575bc060ce160149 INFO: 172.21.0.1:33950 - "POST /node/69408f68575bc060ce160149 HTTP/1.0" 202 Accepted 2025-12-17 22:45:08,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d2f575bc060ce15f7b4 2025-12-17 22:45:08,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2f575bc060ce15f7b4 HTTP/1.1" 502 150 2025-12-17 22:45:08,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d2f575bc060ce15f7b4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:08,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d37575bc060ce15f7c0 2025-12-17 22:45:08,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d37575bc060ce15f7c0 HTTP/1.1" 502 150 2025-12-17 22:45:08,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d37575bc060ce15f7c0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:08,780 - lava_callback - INFO - Received callback for node 69408f6a575bc060ce16014c INFO: 172.21.0.1:33954 - "POST /node/69408f6a575bc060ce16014c HTTP/1.0" 202 Accepted 2025-12-17 22:45:08,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d31575bc060ce15f7b7 2025-12-17 22:45:08,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d31575bc060ce15f7b7 HTTP/1.1" 502 150 2025-12-17 22:45:08,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d31575bc060ce15f7b7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:08,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d3c575bc060ce15f7e8 2025-12-17 22:45:08,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3c575bc060ce15f7e8 HTTP/1.1" 502 150 2025-12-17 22:45:08,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d3c575bc060ce15f7e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:09,443 - lava_callback - INFO - Received callback for node 69408f6c575bc060ce16014f INFO: 172.21.0.1:33970 - "POST /node/69408f6c575bc060ce16014f HTTP/1.0" 202 Accepted 2025-12-17 22:45:09,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d11575bc060ce15f76b HTTP/1.1" 502 150 2025-12-17 22:45:09,936 - lava_callback - ERROR - Error processing callback for node 69408d11575bc060ce15f76b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d11575bc060ce15f76b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d11575bc060ce15f76b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d11575bc060ce15f76b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:09,938 - lava_callback - INFO - Starting async job submit for node 69408d3a575bc060ce15f7e5 2025-12-17 22:45:09,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:09,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d33575bc060ce15f7ba 2025-12-17 22:45:09,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d33575bc060ce15f7ba HTTP/1.1" 502 150 2025-12-17 22:45:09,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d33575bc060ce15f7ba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:09,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3a575bc060ce15f7e5 HTTP/1.1" 502 150 2025-12-17 22:45:09,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d3a575bc060ce15f7e5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:09,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d3a575bc060ce15f7e5 2025-12-17 22:45:09,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3a575bc060ce15f7e5 HTTP/1.1" 502 150 2025-12-17 22:45:09,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d3a575bc060ce15f7e5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:10,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d3f575bc060ce15f7eb 2025-12-17 22:45:10,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3f575bc060ce15f7eb HTTP/1.1" 502 150 2025-12-17 22:45:10,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d3f575bc060ce15f7eb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:10,088 - lava_callback - INFO - Received callback for node 69408f6e575bc060ce160152 INFO: 172.21.0.1:35934 - "POST /node/69408f6e575bc060ce160152 HTTP/1.0" 202 Accepted 2025-12-17 22:45:10,740 - lava_callback - INFO - Received callback for node 69408f6f575bc060ce160155 INFO: 172.21.0.1:35942 - "POST /node/69408f6f575bc060ce160155 HTTP/1.0" 202 Accepted 2025-12-17 22:45:11,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d35575bc060ce15f7bd 2025-12-17 22:45:11,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d35575bc060ce15f7bd HTTP/1.1" 502 150 2025-12-17 22:45:11,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d35575bc060ce15f7bd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:11,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5381e3c24e95681d6aa 2025-12-17 22:45:11,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5381e3c24e95681d6aa HTTP/1.1" 502 150 2025-12-17 22:45:11,717 - lava_callback - ERROR - Error processing callback for node 6940c5381e3c24e95681d6aa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5381e3c24e95681d6aa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5381e3c24e95681d6aa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5381e3c24e95681d6aa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:11,719 - lava_callback - INFO - Starting async job submit for node 6940f6b4460bc8e265115822 2025-12-17 22:45:11,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:11,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940f6b4460bc8e265115822 HTTP/1.1" 502 150 2025-12-17 22:45:11,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940f6b4460bc8e265115822'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:11,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940f6b4460bc8e265115822 2025-12-17 22:45:11,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940f6b4460bc8e265115822 HTTP/1.1" 502 150 2025-12-17 22:45:11,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940f6b4460bc8e265115822'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:11,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d3a575bc060ce15f7e5 2025-12-17 22:45:11,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3a575bc060ce15f7e5 HTTP/1.1" 502 150 2025-12-17 22:45:11,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d3a575bc060ce15f7e5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:12,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d37575bc060ce15f7c0 2025-12-17 22:45:12,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d37575bc060ce15f7c0 HTTP/1.1" 502 150 2025-12-17 22:45:12,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d37575bc060ce15f7c0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:12,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d3c575bc060ce15f7e8 2025-12-17 22:45:12,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3c575bc060ce15f7e8 HTTP/1.1" 502 150 2025-12-17 22:45:12,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d3c575bc060ce15f7e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:13,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940f6b4460bc8e265115822 2025-12-17 22:45:13,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940f6b4460bc8e265115822 HTTP/1.1" 502 150 2025-12-17 22:45:13,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940f6b4460bc8e265115822'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:14,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d3f575bc060ce15f7eb 2025-12-17 22:45:14,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3f575bc060ce15f7eb HTTP/1.1" 502 150 2025-12-17 22:45:14,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d3f575bc060ce15f7eb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:14,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d25575bc060ce15f794 2025-12-17 22:45:14,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d25575bc060ce15f794 HTTP/1.1" 502 150 2025-12-17 22:45:14,418 - lava_callback - ERROR - Error processing callback for node 69408d25575bc060ce15f794: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d25575bc060ce15f794 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d25575bc060ce15f794 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d25575bc060ce15f794 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:14,419 - lava_callback - INFO - Starting async job submit for node 69408d53575bc060ce15f814 2025-12-17 22:45:14,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:14,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d53575bc060ce15f814 HTTP/1.1" 502 150 2025-12-17 22:45:14,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d53575bc060ce15f814'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:14,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d53575bc060ce15f814 2025-12-17 22:45:14,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d53575bc060ce15f814 HTTP/1.1" 502 150 2025-12-17 22:45:14,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d53575bc060ce15f814'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35956 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:45:15,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d20575bc060ce15f786 2025-12-17 22:45:15,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d20575bc060ce15f786 HTTP/1.1" 502 150 2025-12-17 22:45:15,897 - lava_callback - ERROR - Error processing callback for node 69408d20575bc060ce15f786: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d20575bc060ce15f786 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d20575bc060ce15f786 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d20575bc060ce15f786 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:15,899 - lava_callback - INFO - Starting async job submit for node 69408c89575bc060ce15f5f6 2025-12-17 22:45:15,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:15,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d3a575bc060ce15f7e5 2025-12-17 22:45:15,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3a575bc060ce15f7e5 HTTP/1.1" 502 150 2025-12-17 22:45:15,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d3a575bc060ce15f7e5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:15,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c89575bc060ce15f5f6 HTTP/1.1" 502 150 2025-12-17 22:45:15,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c89575bc060ce15f5f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:15,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c89575bc060ce15f5f6 2025-12-17 22:45:15,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c89575bc060ce15f5f6 HTTP/1.1" 502 150 2025-12-17 22:45:15,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c89575bc060ce15f5f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:16,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d53575bc060ce15f814 2025-12-17 22:45:16,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d53575bc060ce15f814 HTTP/1.1" 502 150 2025-12-17 22:45:16,437 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d53575bc060ce15f814'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35972 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:45:16,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d31575bc060ce15f7b7 2025-12-17 22:45:16,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d31575bc060ce15f7b7 HTTP/1.1" 502 150 2025-12-17 22:45:16,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d31575bc060ce15f7b7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:17,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940f6b4460bc8e265115822 2025-12-17 22:45:17,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940f6b4460bc8e265115822 HTTP/1.1" 502 150 2025-12-17 22:45:17,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940f6b4460bc8e265115822'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:17,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d33575bc060ce15f7ba 2025-12-17 22:45:17,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d33575bc060ce15f7ba HTTP/1.1" 502 150 2025-12-17 22:45:17,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d33575bc060ce15f7ba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:17,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c89575bc060ce15f5f6 2025-12-17 22:45:17,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c89575bc060ce15f5f6 HTTP/1.1" 502 150 2025-12-17 22:45:17,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c89575bc060ce15f5f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:18,548 - lava_callback - INFO - Received callback for node 694094cd575bc060ce161115 INFO: 172.21.0.1:35984 - "POST /node/694094cd575bc060ce161115 HTTP/1.0" 202 Accepted 2025-12-17 22:45:19,090 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d26575bc060ce15f797 2025-12-17 22:45:19,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d26575bc060ce15f797 HTTP/1.1" 502 150 2025-12-17 22:45:19,091 - lava_callback - ERROR - Error processing callback for node 69408d26575bc060ce15f797: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d26575bc060ce15f797 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d26575bc060ce15f797 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d26575bc060ce15f797 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:19,093 - lava_callback - INFO - Starting async job submit for node 69408d55575bc060ce15f81d 2025-12-17 22:45:19,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:19,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d55575bc060ce15f81d HTTP/1.1" 502 150 2025-12-17 22:45:19,111 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d55575bc060ce15f81d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:19,111 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d55575bc060ce15f81d 2025-12-17 22:45:19,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d55575bc060ce15f81d HTTP/1.1" 502 150 2025-12-17 22:45:19,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d55575bc060ce15f81d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:19,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d35575bc060ce15f7bd 2025-12-17 22:45:19,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d35575bc060ce15f7bd HTTP/1.1" 502 150 2025-12-17 22:45:19,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d35575bc060ce15f7bd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:20,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d53575bc060ce15f814 2025-12-17 22:45:20,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d53575bc060ce15f814 HTTP/1.1" 502 150 2025-12-17 22:45:20,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d53575bc060ce15f814'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:20,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d37575bc060ce15f7c0 2025-12-17 22:45:20,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d37575bc060ce15f7c0 HTTP/1.1" 502 150 2025-12-17 22:45:20,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d37575bc060ce15f7c0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:20,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d3c575bc060ce15f7e8 2025-12-17 22:45:20,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3c575bc060ce15f7e8 HTTP/1.1" 502 150 2025-12-17 22:45:20,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d3c575bc060ce15f7e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:21,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d22575bc060ce15f791 2025-12-17 22:45:21,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d22575bc060ce15f791 HTTP/1.1" 502 150 2025-12-17 22:45:21,022 - lava_callback - ERROR - Error processing callback for node 69408d22575bc060ce15f791: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d22575bc060ce15f791 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d22575bc060ce15f791 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d22575bc060ce15f791 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:21,025 - lava_callback - INFO - Starting async job submit for node 69408d57575bc060ce15f826 2025-12-17 22:45:21,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:21,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d57575bc060ce15f826 HTTP/1.1" 502 150 2025-12-17 22:45:21,053 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d57575bc060ce15f826'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:21,053 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d57575bc060ce15f826 2025-12-17 22:45:21,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d57575bc060ce15f826 HTTP/1.1" 502 150 2025-12-17 22:45:21,054 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d57575bc060ce15f826'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:21,112 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d55575bc060ce15f81d 2025-12-17 22:45:21,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d55575bc060ce15f81d HTTP/1.1" 502 150 2025-12-17 22:45:21,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d55575bc060ce15f81d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:21,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d2a575bc060ce15f7a0 2025-12-17 22:45:21,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2a575bc060ce15f7a0 HTTP/1.1" 502 150 2025-12-17 22:45:21,694 - lava_callback - ERROR - Error processing callback for node 69408d2a575bc060ce15f7a0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d2a575bc060ce15f7a0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d2a575bc060ce15f7a0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d2a575bc060ce15f7a0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:21,696 - lava_callback - INFO - Starting async job submit for node 69408d59575bc060ce15f829 2025-12-17 22:45:21,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:21,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d59575bc060ce15f829 HTTP/1.1" 502 150 2025-12-17 22:45:21,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d59575bc060ce15f829'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:21,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d59575bc060ce15f829 2025-12-17 22:45:21,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d59575bc060ce15f829 HTTP/1.1" 502 150 2025-12-17 22:45:21,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d59575bc060ce15f829'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:21,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c89575bc060ce15f5f6 2025-12-17 22:45:21,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c89575bc060ce15f5f6 HTTP/1.1" 502 150 2025-12-17 22:45:21,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c89575bc060ce15f5f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:22,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d3f575bc060ce15f7eb 2025-12-17 22:45:22,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3f575bc060ce15f7eb HTTP/1.1" 502 150 2025-12-17 22:45:22,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d3f575bc060ce15f7eb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:22,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4e575bc060ce15fda9 2025-12-17 22:45:22,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4e575bc060ce15fda9 HTTP/1.1" 502 150 2025-12-17 22:45:22,831 - lava_callback - ERROR - Error processing callback for node 69408e4e575bc060ce15fda9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e4e575bc060ce15fda9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e4e575bc060ce15fda9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e4e575bc060ce15fda9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:22,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d2d575bc060ce15f7ad 2025-12-17 22:45:22,833 - lava_callback - INFO - Starting async job submit for node 69408d5b575bc060ce15f82c 2025-12-17 22:45:22,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:22,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2d575bc060ce15f7ad HTTP/1.1" 502 150 2025-12-17 22:45:22,834 - lava_callback - ERROR - Error processing callback for node 69408d2d575bc060ce15f7ad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d2d575bc060ce15f7ad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d2d575bc060ce15f7ad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d2d575bc060ce15f7ad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:22,836 - lava_callback - INFO - Starting async job submit for node 69408d5d575bc060ce15f82f 2025-12-17 22:45:22,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:22,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5b575bc060ce15f82c HTTP/1.1" 502 150 2025-12-17 22:45:22,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d5b575bc060ce15f82c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:22,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d5b575bc060ce15f82c 2025-12-17 22:45:22,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5b575bc060ce15f82c HTTP/1.1" 502 150 2025-12-17 22:45:22,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d5b575bc060ce15f82c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:22,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5d575bc060ce15f82f HTTP/1.1" 502 150 2025-12-17 22:45:22,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d5d575bc060ce15f82f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:22,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d5d575bc060ce15f82f 2025-12-17 22:45:22,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5d575bc060ce15f82f HTTP/1.1" 502 150 2025-12-17 22:45:22,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d5d575bc060ce15f82f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:23,054 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d57575bc060ce15f826 2025-12-17 22:45:23,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d57575bc060ce15f826 HTTP/1.1" 502 150 2025-12-17 22:45:23,055 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d57575bc060ce15f826'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:23,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d59575bc060ce15f829 2025-12-17 22:45:23,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d59575bc060ce15f829 HTTP/1.1" 502 150 2025-12-17 22:45:23,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d59575bc060ce15f829'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:23,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d3a575bc060ce15f7e5 2025-12-17 22:45:23,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3a575bc060ce15f7e5 HTTP/1.1" 502 150 2025-12-17 22:45:23,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d3a575bc060ce15f7e5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:24,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d2f575bc060ce15f7b4 2025-12-17 22:45:24,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d2f575bc060ce15f7b4 HTTP/1.1" 502 150 2025-12-17 22:45:24,619 - lava_callback - ERROR - Error processing callback for node 69408d2f575bc060ce15f7b4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d2f575bc060ce15f7b4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d2f575bc060ce15f7b4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d2f575bc060ce15f7b4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:24,620 - lava_callback - INFO - Starting async job submit for node 69408d5f575bc060ce15f832 2025-12-17 22:45:24,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:24,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5f575bc060ce15f832 HTTP/1.1" 502 150 2025-12-17 22:45:24,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d5f575bc060ce15f832'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:24,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d5f575bc060ce15f832 2025-12-17 22:45:24,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5f575bc060ce15f832 HTTP/1.1" 502 150 2025-12-17 22:45:24,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d5f575bc060ce15f832'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:24,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d5b575bc060ce15f82c 2025-12-17 22:45:24,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5b575bc060ce15f82c HTTP/1.1" 502 150 2025-12-17 22:45:24,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d5b575bc060ce15f82c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:24,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d5d575bc060ce15f82f 2025-12-17 22:45:24,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5d575bc060ce15f82f HTTP/1.1" 502 150 2025-12-17 22:45:24,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d5d575bc060ce15f82f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:25,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d55575bc060ce15f81d 2025-12-17 22:45:25,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d55575bc060ce15f81d HTTP/1.1" 502 150 2025-12-17 22:45:25,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d55575bc060ce15f81d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:25,526 - lava_callback - INFO - Received callback for node 69408f72575bc060ce160162 INFO: 172.21.0.1:58094 - "POST /node/69408f72575bc060ce160162 HTTP/1.0" 202 Accepted 2025-12-17 22:45:25,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940f6b4460bc8e265115822 2025-12-17 22:45:25,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940f6b4460bc8e265115822 HTTP/1.1" 502 150 2025-12-17 22:45:25,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940f6b4460bc8e265115822'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:26,170 - lava_callback - INFO - Received callback for node 69408f74575bc060ce160169 INFO: 172.21.0.1:58110 - "POST /node/69408f74575bc060ce160169 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:58120 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:45:26,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d5f575bc060ce15f832 2025-12-17 22:45:26,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5f575bc060ce15f832 HTTP/1.1" 502 150 2025-12-17 22:45:26,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d5f575bc060ce15f832'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:26,807 - lava_callback - INFO - Received callback for node 69408f76575bc060ce16016c INFO: 172.21.0.1:58134 - "POST /node/69408f76575bc060ce16016c HTTP/1.0" 202 Accepted 2025-12-17 22:45:27,055 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d57575bc060ce15f826 2025-12-17 22:45:27,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d57575bc060ce15f826 HTTP/1.1" 502 150 2025-12-17 22:45:27,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d57575bc060ce15f826'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:27,462 - lava_callback - INFO - Received callback for node 69408f78575bc060ce16016f INFO: 172.21.0.1:58142 - "POST /node/69408f78575bc060ce16016f HTTP/1.0" 202 Accepted 2025-12-17 22:45:27,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d59575bc060ce15f829 2025-12-17 22:45:27,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d59575bc060ce15f829 HTTP/1.1" 502 150 2025-12-17 22:45:27,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d59575bc060ce15f829'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:28,110 - lava_callback - INFO - Received callback for node 69408f79575bc060ce160172 INFO: 172.21.0.1:58156 - "POST /node/69408f79575bc060ce160172 HTTP/1.0" 202 Accepted 2025-12-17 22:45:28,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d53575bc060ce15f814 2025-12-17 22:45:28,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d53575bc060ce15f814 HTTP/1.1" 502 150 2025-12-17 22:45:28,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d53575bc060ce15f814'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:28,763 - lava_callback - INFO - Received callback for node 69408f7b575bc060ce160175 INFO: 172.21.0.1:58168 - "POST /node/69408f7b575bc060ce160175 HTTP/1.0" 202 Accepted 2025-12-17 22:45:28,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d5b575bc060ce15f82c 2025-12-17 22:45:28,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5b575bc060ce15f82c HTTP/1.1" 502 150 2025-12-17 22:45:28,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d5b575bc060ce15f82c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:28,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d5d575bc060ce15f82f 2025-12-17 22:45:28,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5d575bc060ce15f82f HTTP/1.1" 502 150 2025-12-17 22:45:28,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d5d575bc060ce15f82f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:29,424 - lava_callback - INFO - Received callback for node 69408f7d575bc060ce160178 INFO: 172.21.0.1:58176 - "POST /node/69408f7d575bc060ce160178 HTTP/1.0" 202 Accepted 2025-12-17 22:45:29,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c89575bc060ce15f5f6 2025-12-17 22:45:29,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c89575bc060ce15f5f6 HTTP/1.1" 502 150 2025-12-17 22:45:29,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c89575bc060ce15f5f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:30,092 - lava_callback - INFO - Received callback for node 69408f7f575bc060ce16017b INFO: 172.21.0.1:38390 - "POST /node/69408f7f575bc060ce16017b HTTP/1.0" 202 Accepted 2025-12-17 22:45:30,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d5f575bc060ce15f832 2025-12-17 22:45:30,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5f575bc060ce15f832 HTTP/1.1" 502 150 2025-12-17 22:45:30,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d5f575bc060ce15f832'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:30,762 - lava_callback - INFO - Received callback for node 69408f81575bc060ce16017e INFO: 172.21.0.1:38404 - "POST /node/69408f81575bc060ce16017e HTTP/1.0" 202 Accepted 2025-12-17 22:45:31,409 - lava_callback - INFO - Received callback for node 69408f83575bc060ce160181 INFO: 172.21.0.1:38408 - "POST /node/69408f83575bc060ce160181 HTTP/1.0" 202 Accepted 2025-12-17 22:45:32,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d31575bc060ce15f7b7 2025-12-17 22:45:32,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d31575bc060ce15f7b7 HTTP/1.1" 502 150 2025-12-17 22:45:32,838 - lava_callback - ERROR - Error processing callback for node 69408d31575bc060ce15f7b7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d31575bc060ce15f7b7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d31575bc060ce15f7b7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d31575bc060ce15f7b7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:32,840 - lava_callback - INFO - Starting async job submit for node 69408d61575bc060ce15f835 2025-12-17 22:45:32,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:32,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d61575bc060ce15f835 HTTP/1.1" 502 150 2025-12-17 22:45:32,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d61575bc060ce15f835'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:32,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d61575bc060ce15f835 2025-12-17 22:45:32,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d61575bc060ce15f835 HTTP/1.1" 502 150 2025-12-17 22:45:32,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d61575bc060ce15f835'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:33,115 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d55575bc060ce15f81d 2025-12-17 22:45:33,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d55575bc060ce15f81d HTTP/1.1" 502 150 2025-12-17 22:45:33,116 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d55575bc060ce15f81d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:33,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d33575bc060ce15f7ba 2025-12-17 22:45:33,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d33575bc060ce15f7ba HTTP/1.1" 502 150 2025-12-17 22:45:33,943 - lava_callback - ERROR - Error processing callback for node 69408d33575bc060ce15f7ba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d33575bc060ce15f7ba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d33575bc060ce15f7ba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d33575bc060ce15f7ba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:33,945 - lava_callback - INFO - Starting async job submit for node 69408d63575bc060ce15f838 2025-12-17 22:45:33,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:33,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d63575bc060ce15f838 HTTP/1.1" 502 150 2025-12-17 22:45:33,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d63575bc060ce15f838'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:33,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d63575bc060ce15f838 2025-12-17 22:45:33,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d63575bc060ce15f838 HTTP/1.1" 502 150 2025-12-17 22:45:33,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d63575bc060ce15f838'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:34,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d61575bc060ce15f835 2025-12-17 22:45:34,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d61575bc060ce15f835 HTTP/1.1" 502 150 2025-12-17 22:45:34,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d61575bc060ce15f835'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:35,056 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d57575bc060ce15f826 2025-12-17 22:45:35,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d57575bc060ce15f826 HTTP/1.1" 502 150 2025-12-17 22:45:35,057 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d57575bc060ce15f826'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:35,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d35575bc060ce15f7bd 2025-12-17 22:45:35,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d35575bc060ce15f7bd HTTP/1.1" 502 150 2025-12-17 22:45:35,644 - lava_callback - ERROR - Error processing callback for node 69408d35575bc060ce15f7bd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d35575bc060ce15f7bd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d35575bc060ce15f7bd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d35575bc060ce15f7bd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:35,646 - lava_callback - INFO - Starting async job submit for node 69408d65575bc060ce15f83b 2025-12-17 22:45:35,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:35,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d65575bc060ce15f83b HTTP/1.1" 502 150 2025-12-17 22:45:35,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d65575bc060ce15f83b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:35,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d65575bc060ce15f83b 2025-12-17 22:45:35,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d65575bc060ce15f83b HTTP/1.1" 502 150 2025-12-17 22:45:35,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d65575bc060ce15f83b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:35,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d59575bc060ce15f829 2025-12-17 22:45:35,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d59575bc060ce15f829 HTTP/1.1" 502 150 2025-12-17 22:45:35,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d59575bc060ce15f829'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:35,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d63575bc060ce15f838 2025-12-17 22:45:35,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d63575bc060ce15f838 HTTP/1.1" 502 150 2025-12-17 22:45:35,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d63575bc060ce15f838'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:36,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d37575bc060ce15f7c0 2025-12-17 22:45:36,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d37575bc060ce15f7c0 HTTP/1.1" 502 150 2025-12-17 22:45:36,742 - lava_callback - ERROR - Error processing callback for node 69408d37575bc060ce15f7c0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d37575bc060ce15f7c0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d37575bc060ce15f7c0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d37575bc060ce15f7c0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:36,744 - lava_callback - INFO - Starting async job submit for node 69408d74575bc060ce15f857 2025-12-17 22:45:36,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:36,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d74575bc060ce15f857 HTTP/1.1" 502 150 2025-12-17 22:45:36,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d74575bc060ce15f857'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:36,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d74575bc060ce15f857 2025-12-17 22:45:36,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d74575bc060ce15f857 HTTP/1.1" 502 150 2025-12-17 22:45:36,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d74575bc060ce15f857'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38420 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:45:36,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d5b575bc060ce15f82c 2025-12-17 22:45:36,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5b575bc060ce15f82c HTTP/1.1" 502 150 2025-12-17 22:45:36,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d5b575bc060ce15f82c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:36,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d5d575bc060ce15f82f 2025-12-17 22:45:36,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5d575bc060ce15f82f HTTP/1.1" 502 150 2025-12-17 22:45:36,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d5d575bc060ce15f82f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:36,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d3c575bc060ce15f7e8 2025-12-17 22:45:36,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3c575bc060ce15f7e8 HTTP/1.1" 502 150 2025-12-17 22:45:36,977 - lava_callback - ERROR - Error processing callback for node 69408d3c575bc060ce15f7e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d3c575bc060ce15f7e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d3c575bc060ce15f7e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d3c575bc060ce15f7e8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:36,978 - lava_callback - INFO - Starting async job submit for node 69408d78575bc060ce15f9d5 2025-12-17 22:45:36,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:36,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d78575bc060ce15f9d5 HTTP/1.1" 502 150 2025-12-17 22:45:36,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d78575bc060ce15f9d5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:36,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d78575bc060ce15f9d5 2025-12-17 22:45:36,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d78575bc060ce15f9d5 HTTP/1.1" 502 150 2025-12-17 22:45:36,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d78575bc060ce15f9d5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:37,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d65575bc060ce15f83b 2025-12-17 22:45:37,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d65575bc060ce15f83b HTTP/1.1" 502 150 2025-12-17 22:45:37,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d65575bc060ce15f83b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:38,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d3f575bc060ce15f7eb 2025-12-17 22:45:38,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3f575bc060ce15f7eb HTTP/1.1" 502 150 2025-12-17 22:45:38,023 - lava_callback - ERROR - Error processing callback for node 69408d3f575bc060ce15f7eb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d3f575bc060ce15f7eb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d3f575bc060ce15f7eb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d3f575bc060ce15f7eb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:38,024 - lava_callback - INFO - Starting async job submit for node 69408d7a575bc060ce15f9d8 2025-12-17 22:45:38,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:38,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7a575bc060ce15f9d8 HTTP/1.1" 502 150 2025-12-17 22:45:38,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d7a575bc060ce15f9d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:38,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d7a575bc060ce15f9d8 2025-12-17 22:45:38,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7a575bc060ce15f9d8 HTTP/1.1" 502 150 2025-12-17 22:45:38,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d7a575bc060ce15f9d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:38,551 - lava_callback - INFO - Received callback for node 69409618575bc060ce1614eb INFO: 172.21.0.1:38430 - "POST /node/69409618575bc060ce1614eb HTTP/1.0" 202 Accepted 2025-12-17 22:45:38,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d5f575bc060ce15f832 2025-12-17 22:45:38,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5f575bc060ce15f832 HTTP/1.1" 502 150 2025-12-17 22:45:38,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d5f575bc060ce15f832'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:38,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d74575bc060ce15f857 2025-12-17 22:45:38,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d74575bc060ce15f857 HTTP/1.1" 502 150 2025-12-17 22:45:38,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d74575bc060ce15f857'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:38,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d61575bc060ce15f835 2025-12-17 22:45:38,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d61575bc060ce15f835 HTTP/1.1" 502 150 2025-12-17 22:45:38,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d61575bc060ce15f835'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:38,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d78575bc060ce15f9d5 2025-12-17 22:45:38,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d78575bc060ce15f9d5 HTTP/1.1" 502 150 2025-12-17 22:45:38,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d78575bc060ce15f9d5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:39,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d3a575bc060ce15f7e5 2025-12-17 22:45:39,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d3a575bc060ce15f7e5 HTTP/1.1" 502 150 2025-12-17 22:45:39,953 - lava_callback - ERROR - Error processing callback for node 69408d3a575bc060ce15f7e5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d3a575bc060ce15f7e5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d3a575bc060ce15f7e5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d3a575bc060ce15f7e5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:39,955 - lava_callback - INFO - Starting async job submit for node 69408d7c575bc060ce15f9df 2025-12-17 22:45:39,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:39,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d63575bc060ce15f838 2025-12-17 22:45:39,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d63575bc060ce15f838 HTTP/1.1" 502 150 2025-12-17 22:45:39,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d63575bc060ce15f838'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:39,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7c575bc060ce15f9df HTTP/1.1" 502 150 2025-12-17 22:45:39,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d7c575bc060ce15f9df'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:39,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d7c575bc060ce15f9df 2025-12-17 22:45:39,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7c575bc060ce15f9df HTTP/1.1" 502 150 2025-12-17 22:45:39,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d7c575bc060ce15f9df'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:40,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d7a575bc060ce15f9d8 2025-12-17 22:45:40,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7a575bc060ce15f9d8 HTTP/1.1" 502 150 2025-12-17 22:45:40,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d7a575bc060ce15f9d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:41,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d65575bc060ce15f83b 2025-12-17 22:45:41,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d65575bc060ce15f83b HTTP/1.1" 502 150 2025-12-17 22:45:41,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d65575bc060ce15f83b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:41,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940f6b4460bc8e265115822 2025-12-17 22:45:41,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940f6b4460bc8e265115822 HTTP/1.1" 502 150 2025-12-17 22:45:41,769 - lava_callback - ERROR - Error processing callback for node 6940f6b4460bc8e265115822: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940f6b4460bc8e265115822 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940f6b4460bc8e265115822 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940f6b4460bc8e265115822 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:41,771 - lava_callback - INFO - Starting async job submit for node 69408d7e575bc060ce15f9e8 2025-12-17 22:45:41,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:41,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7e575bc060ce15f9e8 HTTP/1.1" 502 150 2025-12-17 22:45:41,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d7e575bc060ce15f9e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:41,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d7e575bc060ce15f9e8 2025-12-17 22:45:41,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7e575bc060ce15f9e8 HTTP/1.1" 502 150 2025-12-17 22:45:41,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d7e575bc060ce15f9e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:41,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d7c575bc060ce15f9df 2025-12-17 22:45:41,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7c575bc060ce15f9df HTTP/1.1" 502 150 2025-12-17 22:45:41,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d7c575bc060ce15f9df'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:42,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d74575bc060ce15f857 2025-12-17 22:45:42,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d74575bc060ce15f857 HTTP/1.1" 502 150 2025-12-17 22:45:42,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d74575bc060ce15f857'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:42,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d78575bc060ce15f9d5 2025-12-17 22:45:42,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d78575bc060ce15f9d5 HTTP/1.1" 502 150 2025-12-17 22:45:42,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d78575bc060ce15f9d5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:43,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d7e575bc060ce15f9e8 2025-12-17 22:45:43,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7e575bc060ce15f9e8 HTTP/1.1" 502 150 2025-12-17 22:45:43,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d7e575bc060ce15f9e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:44,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d7a575bc060ce15f9d8 2025-12-17 22:45:44,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7a575bc060ce15f9d8 HTTP/1.1" 502 150 2025-12-17 22:45:44,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d7a575bc060ce15f9d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:44,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d53575bc060ce15f814 2025-12-17 22:45:44,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d53575bc060ce15f814 HTTP/1.1" 502 150 2025-12-17 22:45:44,441 - lava_callback - ERROR - Error processing callback for node 69408d53575bc060ce15f814: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d53575bc060ce15f814 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d53575bc060ce15f814 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d53575bc060ce15f814 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:44,443 - lava_callback - INFO - Starting async job submit for node 69408d80575bc060ce15f9ed 2025-12-17 22:45:44,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:44,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d80575bc060ce15f9ed HTTP/1.1" 502 150 2025-12-17 22:45:44,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d80575bc060ce15f9ed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:44,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d80575bc060ce15f9ed 2025-12-17 22:45:44,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d80575bc060ce15f9ed HTTP/1.1" 502 150 2025-12-17 22:45:44,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d80575bc060ce15f9ed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:45,545 - lava_callback - INFO - Received callback for node 69408f86575bc060ce160192 INFO: 172.21.0.1:57252 - "POST /node/69408f86575bc060ce160192 HTTP/1.0" 202 Accepted 2025-12-17 22:45:45,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c89575bc060ce15f5f6 2025-12-17 22:45:45,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c89575bc060ce15f5f6 HTTP/1.1" 502 150 2025-12-17 22:45:45,961 - lava_callback - ERROR - Error processing callback for node 69408c89575bc060ce15f5f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c89575bc060ce15f5f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c89575bc060ce15f5f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c89575bc060ce15f5f6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:45,963 - lava_callback - INFO - Starting async job submit for node 69408d83575bc060ce15f9f0 2025-12-17 22:45:45,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:45,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d7c575bc060ce15f9df 2025-12-17 22:45:45,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7c575bc060ce15f9df HTTP/1.1" 502 150 2025-12-17 22:45:45,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d7c575bc060ce15f9df'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:45,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d83575bc060ce15f9f0 HTTP/1.1" 502 150 2025-12-17 22:45:45,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d83575bc060ce15f9f0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:45,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d83575bc060ce15f9f0 2025-12-17 22:45:45,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d83575bc060ce15f9f0 HTTP/1.1" 502 150 2025-12-17 22:45:45,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d83575bc060ce15f9f0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:46,212 - lava_callback - INFO - Received callback for node 69408f88575bc060ce16019b INFO: 172.21.0.1:57260 - "POST /node/69408f88575bc060ce16019b HTTP/1.0" 202 Accepted 2025-12-17 22:45:46,219 - lava_callback - INFO - Received callback for node 69408b6d575bc060ce15f1e1 INFO: 172.21.0.1:57266 - "POST /node/69408b6d575bc060ce15f1e1 HTTP/1.0" 202 Accepted 2025-12-17 22:45:46,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d80575bc060ce15f9ed INFO: 172.21.0.1:57272 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:45:46,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d61575bc060ce15f835 2025-12-17 22:45:46,876 - lava_callback - INFO - Received callback for node 69408f8a575bc060ce16019e INFO: 172.21.0.1:57288 - "POST /node/69408f8a575bc060ce16019e HTTP/1.0" 202 Accepted 2025-12-17 22:45:47,539 - lava_callback - INFO - Received callback for node 69408f8c575bc060ce1601a1 INFO: 172.21.0.1:57304 - "POST /node/69408f8c575bc060ce1601a1 HTTP/1.0" 202 Accepted 2025-12-17 22:45:47,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d7e575bc060ce15f9e8 2025-12-17 22:45:47,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7e575bc060ce15f9e8 HTTP/1.1" 502 150 2025-12-17 22:45:47,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d7e575bc060ce15f9e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:47,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d63575bc060ce15f838 2025-12-17 22:45:47,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d63575bc060ce15f838 HTTP/1.1" 502 150 2025-12-17 22:45:47,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d63575bc060ce15f838'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:47,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d83575bc060ce15f9f0 2025-12-17 22:45:47,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d83575bc060ce15f9f0 HTTP/1.1" 502 150 2025-12-17 22:45:47,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d83575bc060ce15f9f0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:48,181 - lava_callback - INFO - Received callback for node 69408f8d575bc060ce1601a4 INFO: 172.21.0.1:57316 - "POST /node/69408f8d575bc060ce1601a4 HTTP/1.0" 202 Accepted 2025-12-17 22:45:48,838 - lava_callback - INFO - Received callback for node 69408f8f575bc060ce1601a7 INFO: 172.21.0.1:57320 - "POST /node/69408f8f575bc060ce1601a7 HTTP/1.0" 202 Accepted 2025-12-17 22:45:49,116 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d55575bc060ce15f81d 2025-12-17 22:45:49,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d55575bc060ce15f81d HTTP/1.1" 502 150 2025-12-17 22:45:49,118 - lava_callback - ERROR - Error processing callback for node 69408d55575bc060ce15f81d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d55575bc060ce15f81d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d55575bc060ce15f81d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d55575bc060ce15f81d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:49,120 - lava_callback - INFO - Starting async job submit for node 69408d84575bc060ce15f9f3 2025-12-17 22:45:49,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:49,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d84575bc060ce15f9f3 HTTP/1.1" 502 150 2025-12-17 22:45:49,127 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d84575bc060ce15f9f3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:49,127 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d84575bc060ce15f9f3 2025-12-17 22:45:49,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d84575bc060ce15f9f3 HTTP/1.1" 502 150 2025-12-17 22:45:49,127 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d84575bc060ce15f9f3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:49,511 - lava_callback - INFO - Received callback for node 69408f91575bc060ce1601aa INFO: 172.21.0.1:57328 - "POST /node/69408f91575bc060ce1601aa HTTP/1.0" 202 Accepted 2025-12-17 22:45:49,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d65575bc060ce15f83b 2025-12-17 22:45:49,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d65575bc060ce15f83b HTTP/1.1" 502 150 2025-12-17 22:45:49,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d65575bc060ce15f83b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:50,172 - lava_callback - INFO - Received callback for node 69408f93575bc060ce1601ad INFO: 172.21.0.1:49340 - "POST /node/69408f93575bc060ce1601ad HTTP/1.0" 202 Accepted 2025-12-17 22:45:50,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d74575bc060ce15f857 2025-12-17 22:45:50,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d74575bc060ce15f857 HTTP/1.1" 502 150 2025-12-17 22:45:50,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d74575bc060ce15f857'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:50,831 - lava_callback - INFO - Received callback for node 69408f98575bc060ce1601cc INFO: 172.21.0.1:49348 - "POST /node/69408f98575bc060ce1601cc HTTP/1.0" 202 Accepted 2025-12-17 22:45:50,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d78575bc060ce15f9d5 2025-12-17 22:45:50,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d78575bc060ce15f9d5 HTTP/1.1" 502 150 2025-12-17 22:45:50,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d78575bc060ce15f9d5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:51,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d57575bc060ce15f826 2025-12-17 22:45:51,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d57575bc060ce15f826 HTTP/1.1" 502 150 2025-12-17 22:45:51,059 - lava_callback - ERROR - Error processing callback for node 69408d57575bc060ce15f826: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d57575bc060ce15f826 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d57575bc060ce15f826 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d57575bc060ce15f826 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:51,061 - lava_callback - INFO - Starting async job submit for node 69408d86575bc060ce15f9f6 2025-12-17 22:45:51,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:51,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d86575bc060ce15f9f6 HTTP/1.1" 502 150 2025-12-17 22:45:51,068 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d86575bc060ce15f9f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:51,068 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d86575bc060ce15f9f6 2025-12-17 22:45:51,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d86575bc060ce15f9f6 HTTP/1.1" 502 150 2025-12-17 22:45:51,069 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d86575bc060ce15f9f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:51,128 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d84575bc060ce15f9f3 2025-12-17 22:45:51,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d84575bc060ce15f9f3 HTTP/1.1" 502 150 2025-12-17 22:45:51,129 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d84575bc060ce15f9f3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:51,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d59575bc060ce15f829 2025-12-17 22:45:51,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d59575bc060ce15f829 HTTP/1.1" 502 150 2025-12-17 22:45:51,710 - lava_callback - ERROR - Error processing callback for node 69408d59575bc060ce15f829: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d59575bc060ce15f829 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d59575bc060ce15f829 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d59575bc060ce15f829 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:51,712 - lava_callback - INFO - Starting async job submit for node 69408d89575bc060ce15f9fc 2025-12-17 22:45:51,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:51,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d89575bc060ce15f9fc HTTP/1.1" 502 150 2025-12-17 22:45:51,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d89575bc060ce15f9fc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:51,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d89575bc060ce15f9fc 2025-12-17 22:45:51,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d89575bc060ce15f9fc HTTP/1.1" 502 150 2025-12-17 22:45:51,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d89575bc060ce15f9fc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:51,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d83575bc060ce15f9f0 2025-12-17 22:45:51,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d83575bc060ce15f9f0 HTTP/1.1" 502 150 2025-12-17 22:45:51,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d83575bc060ce15f9f0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:52,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d7a575bc060ce15f9d8 2025-12-17 22:45:52,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7a575bc060ce15f9d8 HTTP/1.1" 502 150 2025-12-17 22:45:52,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d7a575bc060ce15f9d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:52,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d5b575bc060ce15f82c 2025-12-17 22:45:52,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d5d575bc060ce15f82f 2025-12-17 22:45:52,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5d575bc060ce15f82f HTTP/1.1" 502 150 2025-12-17 22:45:52,848 - lava_callback - ERROR - Error processing callback for node 69408d5d575bc060ce15f82f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d5d575bc060ce15f82f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d5d575bc060ce15f82f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d5d575bc060ce15f82f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:52,851 - lava_callback - INFO - Starting async job submit for node 69408d93575bc060ce15fa19 2025-12-17 22:45:52,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5b575bc060ce15f82c HTTP/1.1" 502 150 2025-12-17 22:45:52,852 - lava_callback - ERROR - Error processing callback for node 69408d5b575bc060ce15f82c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d5b575bc060ce15f82c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d5b575bc060ce15f82c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d5b575bc060ce15f82c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:52,855 - lava_callback - INFO - Starting async job submit for node 69408d94575bc060ce15fa1c 2025-12-17 22:45:52,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:52,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:52,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d94575bc060ce15fa1c HTTP/1.1" 502 150 2025-12-17 22:45:52,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d94575bc060ce15fa1c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:52,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d94575bc060ce15fa1c 2025-12-17 22:45:52,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d93575bc060ce15fa19 HTTP/1.1" 502 150 2025-12-17 22:45:52,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d93575bc060ce15fa19'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:52,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d93575bc060ce15fa19 2025-12-17 22:45:52,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d94575bc060ce15fa1c HTTP/1.1" 502 150 2025-12-17 22:45:52,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d94575bc060ce15fa1c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:52,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d93575bc060ce15fa19 HTTP/1.1" 502 150 2025-12-17 22:45:52,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d93575bc060ce15fa19'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:53,071 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d86575bc060ce15f9f6 2025-12-17 22:45:53,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d86575bc060ce15f9f6 HTTP/1.1" 502 150 2025-12-17 22:45:53,078 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d86575bc060ce15f9f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:53,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d89575bc060ce15f9fc 2025-12-17 22:45:53,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d89575bc060ce15f9fc HTTP/1.1" 502 150 2025-12-17 22:45:53,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d89575bc060ce15f9fc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:53,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d7c575bc060ce15f9df 2025-12-17 22:45:53,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7c575bc060ce15f9df HTTP/1.1" 502 150 2025-12-17 22:45:53,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d7c575bc060ce15f9df'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:54,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d5f575bc060ce15f832 2025-12-17 22:45:54,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d5f575bc060ce15f832 HTTP/1.1" 502 150 2025-12-17 22:45:54,634 - lava_callback - ERROR - Error processing callback for node 69408d5f575bc060ce15f832: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d5f575bc060ce15f832 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d5f575bc060ce15f832 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d5f575bc060ce15f832 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:45:54,636 - lava_callback - INFO - Starting async job submit for node 69408d87575bc060ce15f9f9 2025-12-17 22:45:54,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:45:54,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d87575bc060ce15f9f9 HTTP/1.1" 502 150 2025-12-17 22:45:54,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d87575bc060ce15f9f9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:54,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d87575bc060ce15f9f9 2025-12-17 22:45:54,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d87575bc060ce15f9f9 HTTP/1.1" 502 150 2025-12-17 22:45:54,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d87575bc060ce15f9f9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:54,795 - lava_callback - INFO - Received callback for node 6940928f575bc060ce160aea INFO: 172.21.0.1:49354 - "POST /node/6940928f575bc060ce160aea HTTP/1.0" 202 Accepted 2025-12-17 22:45:54,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d94575bc060ce15fa1c 2025-12-17 22:45:54,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d93575bc060ce15fa19 2025-12-17 22:45:54,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d94575bc060ce15fa1c HTTP/1.1" 502 150 2025-12-17 22:45:54,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d94575bc060ce15fa1c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:54,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d93575bc060ce15fa19 HTTP/1.1" 502 150 2025-12-17 22:45:54,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d93575bc060ce15fa19'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:55,129 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d84575bc060ce15f9f3 2025-12-17 22:45:55,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d84575bc060ce15f9f3 HTTP/1.1" 502 150 2025-12-17 22:45:55,130 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d84575bc060ce15f9f3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:55,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d7e575bc060ce15f9e8 2025-12-17 22:45:55,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7e575bc060ce15f9e8 HTTP/1.1" 502 150 2025-12-17 22:45:55,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d7e575bc060ce15f9e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49356 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:45:56,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d87575bc060ce15f9f9 2025-12-17 22:45:56,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d87575bc060ce15f9f9 HTTP/1.1" 502 150 2025-12-17 22:45:56,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d87575bc060ce15f9f9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:57,079 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d86575bc060ce15f9f6 2025-12-17 22:45:57,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d86575bc060ce15f9f6 HTTP/1.1" 502 150 2025-12-17 22:45:57,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d86575bc060ce15f9f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:57,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d89575bc060ce15f9fc 2025-12-17 22:45:57,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d89575bc060ce15f9fc HTTP/1.1" 502 150 2025-12-17 22:45:57,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d89575bc060ce15f9fc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:58,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d94575bc060ce15fa1c 2025-12-17 22:45:58,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d93575bc060ce15fa19 2025-12-17 22:45:58,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d94575bc060ce15fa1c HTTP/1.1" 502 150 2025-12-17 22:45:58,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d94575bc060ce15fa1c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:58,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d93575bc060ce15fa19 HTTP/1.1" 502 150 2025-12-17 22:45:58,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d93575bc060ce15fa19'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:45:59,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d83575bc060ce15f9f0 2025-12-17 22:45:59,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d83575bc060ce15f9f0 HTTP/1.1" 502 150 2025-12-17 22:45:59,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d83575bc060ce15f9f0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:00,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d87575bc060ce15f9f9 2025-12-17 22:46:00,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d87575bc060ce15f9f9 HTTP/1.1" 502 150 2025-12-17 22:46:00,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d87575bc060ce15f9f9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:01,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d80575bc060ce15f9ed HTTP/1.1" 502 150 2025-12-17 22:46:01,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d80575bc060ce15f9ed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:02,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d61575bc060ce15f835 HTTP/1.1" 502 150 2025-12-17 22:46:02,160 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d61575bc060ce15f835'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43490 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:46:03,130 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d84575bc060ce15f9f3 2025-12-17 22:46:03,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d84575bc060ce15f9f3 HTTP/1.1" 502 150 2025-12-17 22:46:03,131 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d84575bc060ce15f9f3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:03,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d63575bc060ce15f838 2025-12-17 22:46:03,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d63575bc060ce15f838 HTTP/1.1" 502 150 2025-12-17 22:46:03,965 - lava_callback - ERROR - Error processing callback for node 69408d63575bc060ce15f838: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d63575bc060ce15f838 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d63575bc060ce15f838 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d63575bc060ce15f838 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:03,967 - lava_callback - INFO - Starting async job submit for node 69408d8b575bc060ce15f9ff 2025-12-17 22:46:03,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:03,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8b575bc060ce15f9ff HTTP/1.1" 502 150 2025-12-17 22:46:03,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d8b575bc060ce15f9ff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:03,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d8b575bc060ce15f9ff 2025-12-17 22:46:03,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8b575bc060ce15f9ff HTTP/1.1" 502 150 2025-12-17 22:46:03,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d8b575bc060ce15f9ff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:05,080 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d86575bc060ce15f9f6 2025-12-17 22:46:05,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d86575bc060ce15f9f6 HTTP/1.1" 502 150 2025-12-17 22:46:05,084 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d86575bc060ce15f9f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:05,569 - lava_callback - INFO - Received callback for node 69408f9a575bc060ce1601db INFO: 172.21.0.1:43500 - "POST /node/69408f9a575bc060ce1601db HTTP/1.0" 202 Accepted 2025-12-17 22:46:05,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d65575bc060ce15f83b 2025-12-17 22:46:05,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d65575bc060ce15f83b HTTP/1.1" 502 150 2025-12-17 22:46:05,668 - lava_callback - ERROR - Error processing callback for node 69408d65575bc060ce15f83b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d65575bc060ce15f83b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d65575bc060ce15f83b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d65575bc060ce15f83b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:05,669 - lava_callback - INFO - Starting async job submit for node 69408d8d575bc060ce15fa02 2025-12-17 22:46:05,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:05,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8d575bc060ce15fa02 HTTP/1.1" 502 150 2025-12-17 22:46:05,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d8d575bc060ce15fa02'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:05,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d8d575bc060ce15fa02 2025-12-17 22:46:05,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8d575bc060ce15fa02 HTTP/1.1" 502 150 2025-12-17 22:46:05,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d8d575bc060ce15fa02'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:05,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d89575bc060ce15f9fc 2025-12-17 22:46:05,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d89575bc060ce15f9fc HTTP/1.1" 502 150 2025-12-17 22:46:05,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d89575bc060ce15f9fc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:05,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d80575bc060ce15f9ed 2025-12-17 22:46:05,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d80575bc060ce15f9ed HTTP/1.1" 502 150 2025-12-17 22:46:05,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d80575bc060ce15f9ed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:05,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d8b575bc060ce15f9ff 2025-12-17 22:46:05,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8b575bc060ce15f9ff HTTP/1.1" 502 150 2025-12-17 22:46:05,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d8b575bc060ce15f9ff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:06,219 - lava_callback - INFO - Received callback for node 69408f9c575bc060ce1601e5 INFO: 172.21.0.1:43516 - "POST /node/69408f9c575bc060ce1601e5 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:43518 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:46:06,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d74575bc060ce15f857 2025-12-17 22:46:06,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d74575bc060ce15f857 HTTP/1.1" 502 150 2025-12-17 22:46:06,787 - lava_callback - ERROR - Error processing callback for node 69408d74575bc060ce15f857: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d74575bc060ce15f857 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d74575bc060ce15f857 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d74575bc060ce15f857 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:06,789 - lava_callback - INFO - Starting async job submit for node 69408d8f575bc060ce15fa05 2025-12-17 22:46:06,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:06,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8f575bc060ce15fa05 HTTP/1.1" 502 150 2025-12-17 22:46:06,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d8f575bc060ce15fa05'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:06,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d8f575bc060ce15fa05 2025-12-17 22:46:06,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8f575bc060ce15fa05 HTTP/1.1" 502 150 2025-12-17 22:46:06,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d8f575bc060ce15fa05'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:06,875 - lava_callback - INFO - Received callback for node 69408f9e575bc060ce1601ee INFO: 172.21.0.1:43526 - "POST /node/69408f9e575bc060ce1601ee HTTP/1.0" 202 Accepted 2025-12-17 22:46:06,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d94575bc060ce15fa1c 2025-12-17 22:46:06,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d93575bc060ce15fa19 2025-12-17 22:46:06,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d94575bc060ce15fa1c HTTP/1.1" 502 150 2025-12-17 22:46:06,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d94575bc060ce15fa1c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:06,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d93575bc060ce15fa19 HTTP/1.1" 502 150 2025-12-17 22:46:06,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d93575bc060ce15fa19'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:06,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d78575bc060ce15f9d5 2025-12-17 22:46:06,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d78575bc060ce15f9d5 HTTP/1.1" 502 150 2025-12-17 22:46:06,992 - lava_callback - ERROR - Error processing callback for node 69408d78575bc060ce15f9d5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d78575bc060ce15f9d5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d78575bc060ce15f9d5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d78575bc060ce15f9d5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:06,994 - lava_callback - INFO - Starting async job submit for node 69408da0575bc060ce15fa3a 2025-12-17 22:46:06,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:07,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da0575bc060ce15fa3a HTTP/1.1" 502 150 2025-12-17 22:46:07,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da0575bc060ce15fa3a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:07,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da0575bc060ce15fa3a 2025-12-17 22:46:07,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da0575bc060ce15fa3a HTTP/1.1" 502 150 2025-12-17 22:46:07,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da0575bc060ce15fa3a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:07,516 - lava_callback - INFO - Received callback for node 69408fa1575bc060ce1601f3 INFO: 172.21.0.1:43534 - "POST /node/69408fa1575bc060ce1601f3 HTTP/1.0" 202 Accepted 2025-12-17 22:46:07,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d8d575bc060ce15fa02 2025-12-17 22:46:07,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8d575bc060ce15fa02 HTTP/1.1" 502 150 2025-12-17 22:46:07,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d8d575bc060ce15fa02'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:08,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d7a575bc060ce15f9d8 2025-12-17 22:46:08,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7a575bc060ce15f9d8 HTTP/1.1" 502 150 2025-12-17 22:46:08,037 - lava_callback - ERROR - Error processing callback for node 69408d7a575bc060ce15f9d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d7a575bc060ce15f9d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d7a575bc060ce15f9d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d7a575bc060ce15f9d8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:08,039 - lava_callback - INFO - Starting async job submit for node 69408d95575bc060ce15fa1f 2025-12-17 22:46:08,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:08,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d95575bc060ce15fa1f HTTP/1.1" 502 150 2025-12-17 22:46:08,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d95575bc060ce15fa1f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:08,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d95575bc060ce15fa1f 2025-12-17 22:46:08,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d95575bc060ce15fa1f HTTP/1.1" 502 150 2025-12-17 22:46:08,047 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d95575bc060ce15fa1f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:08,157 - lava_callback - INFO - Received callback for node 69408fa7575bc060ce1601f6 INFO: 172.21.0.1:43538 - "POST /node/69408fa7575bc060ce1601f6 HTTP/1.0" 202 Accepted 2025-12-17 22:46:08,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d87575bc060ce15f9f9 2025-12-17 22:46:08,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d87575bc060ce15f9f9 HTTP/1.1" 502 150 2025-12-17 22:46:08,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d87575bc060ce15f9f9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:08,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d8f575bc060ce15fa05 2025-12-17 22:46:08,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8f575bc060ce15fa05 HTTP/1.1" 502 150 2025-12-17 22:46:08,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d8f575bc060ce15fa05'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:08,801 - lava_callback - INFO - Received callback for node 69408fa9575bc060ce1601f9 INFO: 172.21.0.1:43548 - "POST /node/69408fa9575bc060ce1601f9 HTTP/1.0" 202 Accepted 2025-12-17 22:46:09,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da0575bc060ce15fa3a 2025-12-17 22:46:09,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da0575bc060ce15fa3a HTTP/1.1" 502 150 2025-12-17 22:46:09,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da0575bc060ce15fa3a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:09,447 - lava_callback - INFO - Received callback for node 69408fab575bc060ce1601fc INFO: 172.21.0.1:43550 - "POST /node/69408fab575bc060ce1601fc HTTP/1.0" 202 Accepted 2025-12-17 22:46:09,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d7c575bc060ce15f9df 2025-12-17 22:46:09,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7c575bc060ce15f9df HTTP/1.1" 502 150 2025-12-17 22:46:09,968 - lava_callback - ERROR - Error processing callback for node 69408d7c575bc060ce15f9df: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d7c575bc060ce15f9df (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d7c575bc060ce15f9df (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d7c575bc060ce15f9df (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:09,970 - lava_callback - INFO - Starting async job submit for node 69408d97575bc060ce15fa22 2025-12-17 22:46:09,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:09,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d97575bc060ce15fa22 HTTP/1.1" 502 150 2025-12-17 22:46:09,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d97575bc060ce15fa22'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:09,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d97575bc060ce15fa22 2025-12-17 22:46:09,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d97575bc060ce15fa22 HTTP/1.1" 502 150 2025-12-17 22:46:09,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d97575bc060ce15fa22'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:09,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d8b575bc060ce15f9ff 2025-12-17 22:46:09,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8b575bc060ce15f9ff HTTP/1.1" 502 150 2025-12-17 22:46:09,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d8b575bc060ce15f9ff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:10,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d95575bc060ce15fa1f 2025-12-17 22:46:10,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d95575bc060ce15fa1f HTTP/1.1" 502 150 2025-12-17 22:46:10,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d95575bc060ce15fa1f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:11,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d8d575bc060ce15fa02 2025-12-17 22:46:11,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8d575bc060ce15fa02 HTTP/1.1" 502 150 2025-12-17 22:46:11,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d8d575bc060ce15fa02'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:11,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d7e575bc060ce15f9e8 2025-12-17 22:46:11,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d7e575bc060ce15f9e8 HTTP/1.1" 502 150 2025-12-17 22:46:11,806 - lava_callback - ERROR - Error processing callback for node 69408d7e575bc060ce15f9e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d7e575bc060ce15f9e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d7e575bc060ce15f9e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d7e575bc060ce15f9e8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:11,808 - lava_callback - INFO - Starting async job submit for node 69408d98575bc060ce15fa25 2025-12-17 22:46:11,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:11,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d98575bc060ce15fa25 HTTP/1.1" 502 150 2025-12-17 22:46:11,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d98575bc060ce15fa25'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:11,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d98575bc060ce15fa25 2025-12-17 22:46:11,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d98575bc060ce15fa25 HTTP/1.1" 502 150 2025-12-17 22:46:11,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d98575bc060ce15fa25'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:11,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d97575bc060ce15fa22 2025-12-17 22:46:11,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d97575bc060ce15fa22 HTTP/1.1" 502 150 2025-12-17 22:46:11,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d97575bc060ce15fa22'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:12,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d8f575bc060ce15fa05 2025-12-17 22:46:12,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8f575bc060ce15fa05 HTTP/1.1" 502 150 2025-12-17 22:46:12,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d8f575bc060ce15fa05'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:13,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da0575bc060ce15fa3a 2025-12-17 22:46:13,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d98575bc060ce15fa25 2025-12-17 22:46:13,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d98575bc060ce15fa25 HTTP/1.1" 502 150 2025-12-17 22:46:13,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d98575bc060ce15fa25'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:13,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d80575bc060ce15f9ed 2025-12-17 22:46:13,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d80575bc060ce15f9ed HTTP/1.1" 502 150 2025-12-17 22:46:13,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d80575bc060ce15f9ed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:14,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d95575bc060ce15fa1f 2025-12-17 22:46:14,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d95575bc060ce15fa1f HTTP/1.1" 502 150 2025-12-17 22:46:14,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d95575bc060ce15fa1f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:15,028 - lava_callback - INFO - Received callback for node 694098e5575bc060ce161b9b INFO: 172.21.0.1:53296 - "POST /node/694098e5575bc060ce161b9b HTTP/1.0" 202 Accepted INFO: 172.21.0.1:53308 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:46:15,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d83575bc060ce15f9f0 2025-12-17 22:46:15,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d83575bc060ce15f9f0 HTTP/1.1" 502 150 2025-12-17 22:46:15,978 - lava_callback - ERROR - Error processing callback for node 69408d83575bc060ce15f9f0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d83575bc060ce15f9f0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d83575bc060ce15f9f0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d83575bc060ce15f9f0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:15,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d97575bc060ce15fa22 2025-12-17 22:46:15,980 - lava_callback - INFO - Starting async job submit for node 69408d99575bc060ce15fa28 2025-12-17 22:46:15,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:15,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d97575bc060ce15fa22 HTTP/1.1" 502 150 2025-12-17 22:46:15,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d97575bc060ce15fa22'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:16,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d99575bc060ce15fa28 HTTP/1.1" 502 150 2025-12-17 22:46:16,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d99575bc060ce15fa28'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:16,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d99575bc060ce15fa28 2025-12-17 22:46:16,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d99575bc060ce15fa28 HTTP/1.1" 502 150 2025-12-17 22:46:16,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d99575bc060ce15fa28'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53324 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:46:17,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d98575bc060ce15fa25 2025-12-17 22:46:17,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d98575bc060ce15fa25 HTTP/1.1" 502 150 2025-12-17 22:46:17,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d98575bc060ce15fa25'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:17,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d8b575bc060ce15f9ff 2025-12-17 22:46:17,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8b575bc060ce15f9ff HTTP/1.1" 502 150 2025-12-17 22:46:17,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d8b575bc060ce15f9ff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:18,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d99575bc060ce15fa28 2025-12-17 22:46:18,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d99575bc060ce15fa28 HTTP/1.1" 502 150 2025-12-17 22:46:18,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d99575bc060ce15fa28'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:18,160 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d61575bc060ce15f835 2025-12-17 22:46:18,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d61575bc060ce15f835 HTTP/1.1" 502 150 2025-12-17 22:46:18,162 - lava_callback - ERROR - Error processing callback for node 69408d61575bc060ce15f835: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d61575bc060ce15f835 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d61575bc060ce15f835 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d61575bc060ce15f835 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:18,163 - lava_callback - INFO - Starting async job submit for node 69408d9a575bc060ce15fa2b 2025-12-17 22:46:18,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:18,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9a575bc060ce15fa2b HTTP/1.1" 502 150 2025-12-17 22:46:18,170 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9a575bc060ce15fa2b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:18,170 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9a575bc060ce15fa2b 2025-12-17 22:46:18,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9a575bc060ce15fa2b HTTP/1.1" 502 150 2025-12-17 22:46:18,171 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9a575bc060ce15fa2b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:19,132 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d84575bc060ce15f9f3 2025-12-17 22:46:19,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d84575bc060ce15f9f3 HTTP/1.1" 502 150 2025-12-17 22:46:19,133 - lava_callback - ERROR - Error processing callback for node 69408d84575bc060ce15f9f3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d84575bc060ce15f9f3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d84575bc060ce15f9f3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d84575bc060ce15f9f3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:19,135 - lava_callback - INFO - Starting async job submit for node 69408d9b575bc060ce15fa2e 2025-12-17 22:46:19,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:19,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9b575bc060ce15fa2e HTTP/1.1" 502 150 2025-12-17 22:46:19,142 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9b575bc060ce15fa2e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:19,142 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9b575bc060ce15fa2e 2025-12-17 22:46:19,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9b575bc060ce15fa2e HTTP/1.1" 502 150 2025-12-17 22:46:19,143 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9b575bc060ce15fa2e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:19,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d8d575bc060ce15fa02 2025-12-17 22:46:19,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8d575bc060ce15fa02 HTTP/1.1" 502 150 2025-12-17 22:46:19,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d8d575bc060ce15fa02'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:20,171 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9a575bc060ce15fa2b 2025-12-17 22:46:20,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9a575bc060ce15fa2b HTTP/1.1" 502 150 2025-12-17 22:46:20,173 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9a575bc060ce15fa2b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:20,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d8f575bc060ce15fa05 2025-12-17 22:46:20,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8f575bc060ce15fa05 HTTP/1.1" 502 150 2025-12-17 22:46:20,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d8f575bc060ce15fa05'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:21,084 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d86575bc060ce15f9f6 2025-12-17 22:46:21,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d86575bc060ce15f9f6 HTTP/1.1" 502 150 2025-12-17 22:46:21,086 - lava_callback - ERROR - Error processing callback for node 69408d86575bc060ce15f9f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d86575bc060ce15f9f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d86575bc060ce15f9f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d86575bc060ce15f9f6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:21,088 - lava_callback - INFO - Starting async job submit for node 69408d9e575bc060ce15fa34 2025-12-17 22:46:21,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:21,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9e575bc060ce15fa34 HTTP/1.1" 502 150 2025-12-17 22:46:21,094 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9e575bc060ce15fa34'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:21,094 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9e575bc060ce15fa34 2025-12-17 22:46:21,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9e575bc060ce15fa34 HTTP/1.1" 502 150 2025-12-17 22:46:21,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9e575bc060ce15fa34'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:21,143 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9b575bc060ce15fa2e 2025-12-17 22:46:21,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9b575bc060ce15fa2e HTTP/1.1" 502 150 2025-12-17 22:46:21,144 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9b575bc060ce15fa2e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:21,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d89575bc060ce15f9fc 2025-12-17 22:46:21,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d89575bc060ce15f9fc HTTP/1.1" 502 150 2025-12-17 22:46:21,725 - lava_callback - ERROR - Error processing callback for node 69408d89575bc060ce15f9fc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d89575bc060ce15f9fc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d89575bc060ce15f9fc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d89575bc060ce15f9fc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:21,727 - lava_callback - INFO - Starting async job submit for node 69408d9d575bc060ce15fa31 2025-12-17 22:46:21,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:21,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9d575bc060ce15fa31 HTTP/1.1" 502 150 2025-12-17 22:46:21,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9d575bc060ce15fa31'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:21,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9d575bc060ce15fa31 2025-12-17 22:46:21,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9d575bc060ce15fa31 HTTP/1.1" 502 150 2025-12-17 22:46:21,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9d575bc060ce15fa31'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:22,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d99575bc060ce15fa28 2025-12-17 22:46:22,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d99575bc060ce15fa28 HTTP/1.1" 502 150 2025-12-17 22:46:22,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d99575bc060ce15fa28'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:22,049 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d95575bc060ce15fa1f 2025-12-17 22:46:22,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d95575bc060ce15fa1f HTTP/1.1" 502 150 2025-12-17 22:46:22,050 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d95575bc060ce15fa1f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:22,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d94575bc060ce15fa1c 2025-12-17 22:46:22,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d93575bc060ce15fa19 2025-12-17 22:46:22,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d94575bc060ce15fa1c HTTP/1.1" 502 150 2025-12-17 22:46:22,885 - lava_callback - ERROR - Error processing callback for node 69408d94575bc060ce15fa1c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d94575bc060ce15fa1c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d94575bc060ce15fa1c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d94575bc060ce15fa1c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:22,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d93575bc060ce15fa19 HTTP/1.1" 502 150 2025-12-17 22:46:22,886 - lava_callback - INFO - Starting async job submit for node 69408d9f575bc060ce15fa37 2025-12-17 22:46:22,887 - lava_callback - ERROR - Error processing callback for node 69408d93575bc060ce15fa19: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d93575bc060ce15fa19 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d93575bc060ce15fa19 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d93575bc060ce15fa19 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:22,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:22,889 - lava_callback - INFO - Starting async job submit for node 69408da8575bc060ce15fa5c 2025-12-17 22:46:22,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:22,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da8575bc060ce15fa5c HTTP/1.1" 502 150 2025-12-17 22:46:22,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da8575bc060ce15fa5c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:22,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da8575bc060ce15fa5c 2025-12-17 22:46:22,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da8575bc060ce15fa5c HTTP/1.1" 502 150 2025-12-17 22:46:22,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da8575bc060ce15fa5c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:22,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9f575bc060ce15fa37 HTTP/1.1" 502 150 2025-12-17 22:46:22,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9f575bc060ce15fa37'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:22,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9f575bc060ce15fa37 2025-12-17 22:46:22,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9f575bc060ce15fa37 HTTP/1.1" 502 150 2025-12-17 22:46:22,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9f575bc060ce15fa37'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:23,095 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9e575bc060ce15fa34 2025-12-17 22:46:23,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9e575bc060ce15fa34 HTTP/1.1" 502 150 2025-12-17 22:46:23,097 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9e575bc060ce15fa34'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:23,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9d575bc060ce15fa31 2025-12-17 22:46:23,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9d575bc060ce15fa31 HTTP/1.1" 502 150 2025-12-17 22:46:23,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9d575bc060ce15fa31'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:23,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d97575bc060ce15fa22 2025-12-17 22:46:23,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d97575bc060ce15fa22 HTTP/1.1" 502 150 2025-12-17 22:46:23,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d97575bc060ce15fa22'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:24,173 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9a575bc060ce15fa2b 2025-12-17 22:46:24,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d87575bc060ce15f9f9 2025-12-17 22:46:24,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d87575bc060ce15f9f9 HTTP/1.1" 502 150 2025-12-17 22:46:24,650 - lava_callback - ERROR - Error processing callback for node 69408d87575bc060ce15f9f9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d87575bc060ce15f9f9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d87575bc060ce15f9f9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d87575bc060ce15f9f9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:24,652 - lava_callback - INFO - Starting async job submit for node 69408da2575bc060ce15fa3d 2025-12-17 22:46:24,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:24,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da2575bc060ce15fa3d HTTP/1.1" 502 150 2025-12-17 22:46:24,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da2575bc060ce15fa3d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:24,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da2575bc060ce15fa3d 2025-12-17 22:46:24,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da2575bc060ce15fa3d HTTP/1.1" 502 150 2025-12-17 22:46:24,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da2575bc060ce15fa3d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:24,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da8575bc060ce15fa5c 2025-12-17 22:46:24,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da8575bc060ce15fa5c HTTP/1.1" 502 150 2025-12-17 22:46:24,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da8575bc060ce15fa5c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:24,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9f575bc060ce15fa37 2025-12-17 22:46:24,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9f575bc060ce15fa37 HTTP/1.1" 502 150 2025-12-17 22:46:24,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9f575bc060ce15fa37'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:25,144 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9b575bc060ce15fa2e 2025-12-17 22:46:25,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9b575bc060ce15fa2e HTTP/1.1" 502 150 2025-12-17 22:46:25,145 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9b575bc060ce15fa2e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:25,577 - lava_callback - INFO - Received callback for node 69408faf575bc060ce16020c INFO: 172.21.0.1:35870 - "POST /node/69408faf575bc060ce16020c HTTP/1.0" 202 Accepted 2025-12-17 22:46:25,732 - lava_callback - INFO - Received callback for node 69408e9d575bc060ce15fea3 INFO: 172.21.0.1:35880 - "POST /node/69408e9d575bc060ce15fea3 HTTP/1.0" 202 Accepted 2025-12-17 22:46:25,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d98575bc060ce15fa25 2025-12-17 22:46:25,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d98575bc060ce15fa25 HTTP/1.1" 502 150 2025-12-17 22:46:25,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d98575bc060ce15fa25'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:26,244 - lava_callback - INFO - Received callback for node 69408fb1575bc060ce160211 INFO: 172.21.0.1:35884 - "POST /node/69408fb1575bc060ce160211 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:35898 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:46:26,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da2575bc060ce15fa3d 2025-12-17 22:46:26,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da2575bc060ce15fa3d HTTP/1.1" 502 150 2025-12-17 22:46:26,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da2575bc060ce15fa3d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:26,895 - lava_callback - INFO - Received callback for node 69408fb3575bc060ce160214 INFO: 172.21.0.1:35904 - "POST /node/69408fb3575bc060ce160214 HTTP/1.0" 202 Accepted 2025-12-17 22:46:27,097 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9e575bc060ce15fa34 2025-12-17 22:46:27,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9e575bc060ce15fa34 HTTP/1.1" 502 150 2025-12-17 22:46:27,098 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9e575bc060ce15fa34'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:27,555 - lava_callback - INFO - Received callback for node 69408fb5575bc060ce160217 INFO: 172.21.0.1:35918 - "POST /node/69408fb5575bc060ce160217 HTTP/1.0" 202 Accepted 2025-12-17 22:46:27,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9d575bc060ce15fa31 2025-12-17 22:46:27,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9d575bc060ce15fa31 HTTP/1.1" 502 150 2025-12-17 22:46:27,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9d575bc060ce15fa31'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:28,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da0575bc060ce15fa3a HTTP/1.1" 502 150 2025-12-17 22:46:28,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da0575bc060ce15fa3a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:28,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da8575bc060ce15fa5c 2025-12-17 22:46:28,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da8575bc060ce15fa5c HTTP/1.1" 502 150 2025-12-17 22:46:28,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da8575bc060ce15fa5c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:28,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9f575bc060ce15fa37 2025-12-17 22:46:28,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9f575bc060ce15fa37 HTTP/1.1" 502 150 2025-12-17 22:46:28,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9f575bc060ce15fa37'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:29,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d80575bc060ce15f9ed 2025-12-17 22:46:29,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d80575bc060ce15f9ed HTTP/1.1" 502 150 2025-12-17 22:46:29,909 - lava_callback - ERROR - Error processing callback for node 69408d80575bc060ce15f9ed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d80575bc060ce15f9ed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d80575bc060ce15f9ed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d80575bc060ce15f9ed (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:29,910 - lava_callback - INFO - Starting async job submit for node 69408da3575bc060ce15fa40 2025-12-17 22:46:29,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:29,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da3575bc060ce15fa40 HTTP/1.1" 502 150 2025-12-17 22:46:29,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da3575bc060ce15fa40'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:29,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da3575bc060ce15fa40 2025-12-17 22:46:29,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da3575bc060ce15fa40 HTTP/1.1" 502 150 2025-12-17 22:46:29,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da3575bc060ce15fa40'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:30,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d99575bc060ce15fa28 2025-12-17 22:46:30,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d99575bc060ce15fa28 HTTP/1.1" 502 150 2025-12-17 22:46:30,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d99575bc060ce15fa28'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:30,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da2575bc060ce15fa3d 2025-12-17 22:46:30,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da2575bc060ce15fa3d HTTP/1.1" 502 150 2025-12-17 22:46:30,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da2575bc060ce15fa3d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:31,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da3575bc060ce15fa40 2025-12-17 22:46:31,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da3575bc060ce15fa40 HTTP/1.1" 502 150 2025-12-17 22:46:31,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da3575bc060ce15fa40'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:33,146 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9b575bc060ce15fa2e 2025-12-17 22:46:33,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9b575bc060ce15fa2e HTTP/1.1" 502 150 2025-12-17 22:46:33,147 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9b575bc060ce15fa2e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:33,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d8b575bc060ce15f9ff 2025-12-17 22:46:33,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8b575bc060ce15f9ff HTTP/1.1" 502 150 2025-12-17 22:46:33,985 - lava_callback - ERROR - Error processing callback for node 69408d8b575bc060ce15f9ff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d8b575bc060ce15f9ff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d8b575bc060ce15f9ff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d8b575bc060ce15f9ff (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:33,986 - lava_callback - INFO - Starting async job submit for node 69408daa575bc060ce15fa63 2025-12-17 22:46:33,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:34,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408daa575bc060ce15fa63 HTTP/1.1" 502 150 2025-12-17 22:46:34,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408daa575bc060ce15fa63'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:34,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408daa575bc060ce15fa63 2025-12-17 22:46:34,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408daa575bc060ce15fa63 HTTP/1.1" 502 150 2025-12-17 22:46:34,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408daa575bc060ce15fa63'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:35,098 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9e575bc060ce15fa34 2025-12-17 22:46:35,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9e575bc060ce15fa34 HTTP/1.1" 502 150 2025-12-17 22:46:35,100 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9e575bc060ce15fa34'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:35,270 - lava_callback - INFO - Received callback for node 69408fb7575bc060ce16021a INFO: 172.21.0.1:59022 - "POST /node/69408fb7575bc060ce16021a HTTP/1.0" 202 Accepted 2025-12-17 22:46:35,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d8d575bc060ce15fa02 2025-12-17 22:46:35,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8d575bc060ce15fa02 HTTP/1.1" 502 150 2025-12-17 22:46:35,685 - lava_callback - ERROR - Error processing callback for node 69408d8d575bc060ce15fa02: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d8d575bc060ce15fa02 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d8d575bc060ce15fa02 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d8d575bc060ce15fa02 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:35,687 - lava_callback - INFO - Starting async job submit for node 69408dad575bc060ce15fa66 2025-12-17 22:46:35,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:35,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dad575bc060ce15fa66 HTTP/1.1" 502 150 2025-12-17 22:46:35,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dad575bc060ce15fa66'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:35,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dad575bc060ce15fa66 2025-12-17 22:46:35,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dad575bc060ce15fa66 HTTP/1.1" 502 150 2025-12-17 22:46:35,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dad575bc060ce15fa66'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:35,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9d575bc060ce15fa31 2025-12-17 22:46:35,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9d575bc060ce15fa31 HTTP/1.1" 502 150 2025-12-17 22:46:35,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9d575bc060ce15fa31'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:35,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da3575bc060ce15fa40 2025-12-17 22:46:35,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da3575bc060ce15fa40 HTTP/1.1" 502 150 2025-12-17 22:46:35,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da3575bc060ce15fa40'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:35,957 - lava_callback - INFO - Received callback for node 69408fba575bc060ce16021d INFO: 172.21.0.1:59036 - "POST /node/69408fba575bc060ce16021d HTTP/1.0" 202 Accepted 2025-12-17 22:46:36,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408daa575bc060ce15fa63 2025-12-17 22:46:36,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408daa575bc060ce15fa63 HTTP/1.1" 502 150 2025-12-17 22:46:36,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408daa575bc060ce15fa63'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:36,273 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da0575bc060ce15fa3a 2025-12-17 22:46:36,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da0575bc060ce15fa3a HTTP/1.1" 502 150 2025-12-17 22:46:36,274 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da0575bc060ce15fa3a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59046 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:46:36,620 - lava_callback - INFO - Received callback for node 69408fbc575bc060ce160220 INFO: 172.21.0.1:59054 - "POST /node/69408fbc575bc060ce160220 HTTP/1.0" 202 Accepted 2025-12-17 22:46:36,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d8f575bc060ce15fa05 2025-12-17 22:46:36,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d8f575bc060ce15fa05 HTTP/1.1" 502 150 2025-12-17 22:46:36,804 - lava_callback - ERROR - Error processing callback for node 69408d8f575bc060ce15fa05: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d8f575bc060ce15fa05 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d8f575bc060ce15fa05 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d8f575bc060ce15fa05 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:36,806 - lava_callback - INFO - Starting async job submit for node 69408da4575bc060ce15fa4b 2025-12-17 22:46:36,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:36,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da4575bc060ce15fa4b HTTP/1.1" 502 150 2025-12-17 22:46:36,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da4575bc060ce15fa4b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:36,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da4575bc060ce15fa4b 2025-12-17 22:46:36,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da4575bc060ce15fa4b HTTP/1.1" 502 150 2025-12-17 22:46:36,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da4575bc060ce15fa4b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:36,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da8575bc060ce15fa5c 2025-12-17 22:46:36,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da8575bc060ce15fa5c HTTP/1.1" 502 150 2025-12-17 22:46:36,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da8575bc060ce15fa5c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:36,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9f575bc060ce15fa37 2025-12-17 22:46:36,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9f575bc060ce15fa37 HTTP/1.1" 502 150 2025-12-17 22:46:36,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9f575bc060ce15fa37'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:37,308 - lava_callback - INFO - Received callback for node 69408fbf575bc060ce160223 INFO: 172.21.0.1:59068 - "POST /node/69408fbf575bc060ce160223 HTTP/1.0" 202 Accepted 2025-12-17 22:46:37,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dad575bc060ce15fa66 2025-12-17 22:46:37,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dad575bc060ce15fa66 HTTP/1.1" 502 150 2025-12-17 22:46:37,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dad575bc060ce15fa66'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:37,957 - lava_callback - INFO - Received callback for node 69408fc1575bc060ce160228 INFO: 172.21.0.1:59076 - "POST /node/69408fc1575bc060ce160228 HTTP/1.0" 202 Accepted 2025-12-17 22:46:38,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d95575bc060ce15fa1f 2025-12-17 22:46:38,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da2575bc060ce15fa3d 2025-12-17 22:46:38,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da2575bc060ce15fa3d HTTP/1.1" 502 150 2025-12-17 22:46:38,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da2575bc060ce15fa3d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:38,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da4575bc060ce15fa4b 2025-12-17 22:46:38,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da4575bc060ce15fa4b HTTP/1.1" 502 150 2025-12-17 22:46:38,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da4575bc060ce15fa4b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:39,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9a575bc060ce15fa2b HTTP/1.1" 502 150 2025-12-17 22:46:39,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9a575bc060ce15fa2b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:39,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d97575bc060ce15fa22 2025-12-17 22:46:39,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d97575bc060ce15fa22 HTTP/1.1" 502 150 2025-12-17 22:46:39,989 - lava_callback - ERROR - Error processing callback for node 69408d97575bc060ce15fa22: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d97575bc060ce15fa22 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d97575bc060ce15fa22 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d97575bc060ce15fa22 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:39,991 - lava_callback - INFO - Starting async job submit for node 69408da6575bc060ce15fa52 2025-12-17 22:46:39,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:39,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da6575bc060ce15fa52 HTTP/1.1" 502 150 2025-12-17 22:46:39,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da6575bc060ce15fa52'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:39,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da6575bc060ce15fa52 2025-12-17 22:46:39,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da6575bc060ce15fa52 HTTP/1.1" 502 150 2025-12-17 22:46:39,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da6575bc060ce15fa52'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:40,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408daa575bc060ce15fa63 2025-12-17 22:46:40,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408daa575bc060ce15fa63 HTTP/1.1" 502 150 2025-12-17 22:46:40,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408daa575bc060ce15fa63'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:41,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dad575bc060ce15fa66 2025-12-17 22:46:41,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dad575bc060ce15fa66 HTTP/1.1" 502 150 2025-12-17 22:46:41,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dad575bc060ce15fa66'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:41,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d98575bc060ce15fa25 2025-12-17 22:46:41,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d98575bc060ce15fa25 HTTP/1.1" 502 150 2025-12-17 22:46:41,820 - lava_callback - ERROR - Error processing callback for node 69408d98575bc060ce15fa25: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d98575bc060ce15fa25 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d98575bc060ce15fa25 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d98575bc060ce15fa25 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:41,822 - lava_callback - INFO - Starting async job submit for node 69408da7575bc060ce15fa59 2025-12-17 22:46:41,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:41,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da7575bc060ce15fa59 HTTP/1.1" 502 150 2025-12-17 22:46:41,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da7575bc060ce15fa59'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:41,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da7575bc060ce15fa59 2025-12-17 22:46:41,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da7575bc060ce15fa59 HTTP/1.1" 502 150 2025-12-17 22:46:41,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da7575bc060ce15fa59'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:41,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da6575bc060ce15fa52 2025-12-17 22:46:42,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da6575bc060ce15fa52 HTTP/1.1" 502 150 2025-12-17 22:46:42,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da6575bc060ce15fa52'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:42,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da4575bc060ce15fa4b 2025-12-17 22:46:42,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da4575bc060ce15fa4b HTTP/1.1" 502 150 2025-12-17 22:46:42,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da4575bc060ce15fa4b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:43,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da7575bc060ce15fa59 2025-12-17 22:46:43,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da7575bc060ce15fa59 HTTP/1.1" 502 150 2025-12-17 22:46:43,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da7575bc060ce15fa59'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:43,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da3575bc060ce15fa40 2025-12-17 22:46:43,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da3575bc060ce15fa40 HTTP/1.1" 502 150 2025-12-17 22:46:43,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da3575bc060ce15fa40'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:45,579 - lava_callback - INFO - Received callback for node 69408fc7575bc060ce160240 INFO: 172.21.0.1:42488 - "POST /node/69408fc7575bc060ce160240 HTTP/1.0" 202 Accepted 2025-12-17 22:46:46,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da6575bc060ce15fa52 2025-12-17 22:46:46,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da6575bc060ce15fa52 HTTP/1.1" 502 150 2025-12-17 22:46:46,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da6575bc060ce15fa52'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:46,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d99575bc060ce15fa28 2025-12-17 22:46:46,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d99575bc060ce15fa28 HTTP/1.1" 502 150 2025-12-17 22:46:46,013 - lava_callback - ERROR - Error processing callback for node 69408d99575bc060ce15fa28: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d99575bc060ce15fa28 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d99575bc060ce15fa28 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d99575bc060ce15fa28 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:46,015 - lava_callback - INFO - Starting async job submit for node 69408db1575bc060ce15fa6c 2025-12-17 22:46:46,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:46,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db1575bc060ce15fa6c HTTP/1.1" 502 150 2025-12-17 22:46:46,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db1575bc060ce15fa6c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:46,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db1575bc060ce15fa6c 2025-12-17 22:46:46,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db1575bc060ce15fa6c HTTP/1.1" 502 150 2025-12-17 22:46:46,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db1575bc060ce15fa6c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:46,227 - lava_callback - INFO - Received callback for node 69408fca575bc060ce160243 INFO: 172.21.0.1:42498 - "POST /node/69408fca575bc060ce160243 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:42506 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:46:46,897 - lava_callback - INFO - Received callback for node 69408fcc575bc060ce160246 INFO: 172.21.0.1:42512 - "POST /node/69408fcc575bc060ce160246 HTTP/1.0" 202 Accepted 2025-12-17 22:46:47,444 - lava_callback - INFO - Received callback for node 6941017a460bc8e265116a25 INFO: 172.21.0.1:42522 - "POST /node/6941017a460bc8e265116a25 HTTP/1.0" 202 Accepted 2025-12-17 22:46:47,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9a575bc060ce15fa2b 2025-12-17 22:46:47,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9a575bc060ce15fa2b HTTP/1.1" 502 150 2025-12-17 22:46:47,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d9a575bc060ce15fa2b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:47,560 - lava_callback - INFO - Received callback for node 69408fce575bc060ce160249 INFO: 172.21.0.1:42534 - "POST /node/69408fce575bc060ce160249 HTTP/1.0" 202 Accepted 2025-12-17 22:46:47,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da7575bc060ce15fa59 2025-12-17 22:46:47,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da7575bc060ce15fa59 HTTP/1.1" 502 150 2025-12-17 22:46:47,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da7575bc060ce15fa59'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:48,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408daa575bc060ce15fa63 2025-12-17 22:46:48,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408daa575bc060ce15fa63 HTTP/1.1" 502 150 2025-12-17 22:46:48,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408daa575bc060ce15fa63'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:48,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db1575bc060ce15fa6c 2025-12-17 22:46:48,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db1575bc060ce15fa6c HTTP/1.1" 502 150 2025-12-17 22:46:48,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db1575bc060ce15fa6c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:49,147 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9b575bc060ce15fa2e 2025-12-17 22:46:49,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9b575bc060ce15fa2e HTTP/1.1" 502 150 2025-12-17 22:46:49,149 - lava_callback - ERROR - Error processing callback for node 69408d9b575bc060ce15fa2e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d9b575bc060ce15fa2e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d9b575bc060ce15fa2e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d9b575bc060ce15fa2e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:49,151 - lava_callback - INFO - Starting async job submit for node 69408db4575bc060ce15fa8c 2025-12-17 22:46:49,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:49,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db4575bc060ce15fa8c HTTP/1.1" 502 150 2025-12-17 22:46:49,158 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db4575bc060ce15fa8c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:49,158 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db4575bc060ce15fa8c 2025-12-17 22:46:49,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db4575bc060ce15fa8c HTTP/1.1" 502 150 2025-12-17 22:46:49,159 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db4575bc060ce15fa8c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:49,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dad575bc060ce15fa66 2025-12-17 22:46:49,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dad575bc060ce15fa66 HTTP/1.1" 502 150 2025-12-17 22:46:49,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dad575bc060ce15fa66'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:50,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da4575bc060ce15fa4b 2025-12-17 22:46:51,100 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9e575bc060ce15fa34 2025-12-17 22:46:51,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9e575bc060ce15fa34 HTTP/1.1" 502 150 2025-12-17 22:46:51,101 - lava_callback - ERROR - Error processing callback for node 69408d9e575bc060ce15fa34: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d9e575bc060ce15fa34 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d9e575bc060ce15fa34 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d9e575bc060ce15fa34 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:51,103 - lava_callback - INFO - Starting async job submit for node 69408db6575bc060ce15fa8f 2025-12-17 22:46:51,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:51,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db6575bc060ce15fa8f HTTP/1.1" 502 150 2025-12-17 22:46:51,135 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db6575bc060ce15fa8f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:51,135 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db6575bc060ce15fa8f 2025-12-17 22:46:51,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db6575bc060ce15fa8f HTTP/1.1" 502 150 2025-12-17 22:46:51,136 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db6575bc060ce15fa8f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:51,159 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db4575bc060ce15fa8c 2025-12-17 22:46:51,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db4575bc060ce15fa8c HTTP/1.1" 502 150 2025-12-17 22:46:51,160 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db4575bc060ce15fa8c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:51,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9d575bc060ce15fa31 2025-12-17 22:46:51,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9d575bc060ce15fa31 HTTP/1.1" 502 150 2025-12-17 22:46:51,746 - lava_callback - ERROR - Error processing callback for node 69408d9d575bc060ce15fa31: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d9d575bc060ce15fa31 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d9d575bc060ce15fa31 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d9d575bc060ce15fa31 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:51,748 - lava_callback - INFO - Starting async job submit for node 69408db8575bc060ce15fa92 2025-12-17 22:46:51,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:51,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db8575bc060ce15fa92 HTTP/1.1" 502 150 2025-12-17 22:46:51,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db8575bc060ce15fa92'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:51,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db8575bc060ce15fa92 2025-12-17 22:46:51,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db8575bc060ce15fa92 HTTP/1.1" 502 150 2025-12-17 22:46:51,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db8575bc060ce15fa92'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:52,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db1575bc060ce15fa6c 2025-12-17 22:46:52,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db1575bc060ce15fa6c HTTP/1.1" 502 150 2025-12-17 22:46:52,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db1575bc060ce15fa6c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:52,274 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da0575bc060ce15fa3a 2025-12-17 22:46:52,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da0575bc060ce15fa3a HTTP/1.1" 502 150 2025-12-17 22:46:52,275 - lava_callback - ERROR - Error processing callback for node 69408da0575bc060ce15fa3a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da0575bc060ce15fa3a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da0575bc060ce15fa3a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da0575bc060ce15fa3a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:52,277 - lava_callback - INFO - Starting async job submit for node 6940924d575bc060ce160a6c 2025-12-17 22:46:52,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:52,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924d575bc060ce160a6c HTTP/1.1" 502 150 2025-12-17 22:46:52,302 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940924d575bc060ce160a6c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:52,302 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940924d575bc060ce160a6c 2025-12-17 22:46:52,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924d575bc060ce160a6c HTTP/1.1" 502 150 2025-12-17 22:46:52,303 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940924d575bc060ce160a6c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:52,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da8575bc060ce15fa5c 2025-12-17 22:46:52,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da8575bc060ce15fa5c HTTP/1.1" 502 150 2025-12-17 22:46:52,907 - lava_callback - ERROR - Error processing callback for node 69408da8575bc060ce15fa5c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da8575bc060ce15fa5c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da8575bc060ce15fa5c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da8575bc060ce15fa5c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:52,908 - lava_callback - INFO - Starting async job submit for node 69408daf575bc060ce15fa69 2025-12-17 22:46:52,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:52,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9f575bc060ce15fa37 2025-12-17 22:46:52,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9f575bc060ce15fa37 HTTP/1.1" 502 150 2025-12-17 22:46:52,924 - lava_callback - ERROR - Error processing callback for node 69408d9f575bc060ce15fa37: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d9f575bc060ce15fa37 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d9f575bc060ce15fa37 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d9f575bc060ce15fa37 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:52,926 - lava_callback - INFO - Starting async job submit for node 69408dba575bc060ce15fa9b 2025-12-17 22:46:52,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408daf575bc060ce15fa69 HTTP/1.1" 502 150 2025-12-17 22:46:52,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:52,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408daf575bc060ce15fa69'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:52,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408daf575bc060ce15fa69 2025-12-17 22:46:52,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408daf575bc060ce15fa69 HTTP/1.1" 502 150 2025-12-17 22:46:52,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408daf575bc060ce15fa69'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:52,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dba575bc060ce15fa9b HTTP/1.1" 502 150 2025-12-17 22:46:52,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dba575bc060ce15fa9b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:52,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dba575bc060ce15fa9b 2025-12-17 22:46:52,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dba575bc060ce15fa9b HTTP/1.1" 502 150 2025-12-17 22:46:52,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dba575bc060ce15fa9b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:53,136 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db6575bc060ce15fa8f 2025-12-17 22:46:53,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db6575bc060ce15fa8f HTTP/1.1" 502 150 2025-12-17 22:46:53,137 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db6575bc060ce15fa8f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:53,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d95575bc060ce15fa1f HTTP/1.1" 502 150 2025-12-17 22:46:53,360 - lava_callback - ERROR - Error processing callback for node 69408d95575bc060ce15fa1f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d95575bc060ce15fa1f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d95575bc060ce15fa1f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d95575bc060ce15fa1f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:53,363 - lava_callback - INFO - Starting async job submit for node 69408dbc575bc060ce15faa2 2025-12-17 22:46:53,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:53,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dbc575bc060ce15faa2 HTTP/1.1" 502 150 2025-12-17 22:46:53,370 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dbc575bc060ce15faa2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:53,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dbc575bc060ce15faa2 2025-12-17 22:46:53,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dbc575bc060ce15faa2 HTTP/1.1" 502 150 2025-12-17 22:46:53,371 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dbc575bc060ce15faa2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:53,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db8575bc060ce15fa92 2025-12-17 22:46:53,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db8575bc060ce15fa92 HTTP/1.1" 502 150 2025-12-17 22:46:53,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db8575bc060ce15fa92'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:54,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da6575bc060ce15fa52 2025-12-17 22:46:54,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da6575bc060ce15fa52 HTTP/1.1" 502 150 2025-12-17 22:46:54,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da6575bc060ce15fa52'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:54,303 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940924d575bc060ce160a6c 2025-12-17 22:46:54,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924d575bc060ce160a6c HTTP/1.1" 502 150 2025-12-17 22:46:54,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940924d575bc060ce160a6c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:54,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da2575bc060ce15fa3d 2025-12-17 22:46:54,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da2575bc060ce15fa3d HTTP/1.1" 502 150 2025-12-17 22:46:54,669 - lava_callback - ERROR - Error processing callback for node 69408da2575bc060ce15fa3d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da2575bc060ce15fa3d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da2575bc060ce15fa3d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da2575bc060ce15fa3d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:54,671 - lava_callback - INFO - Starting async job submit for node 69408dbe575bc060ce15faa5 2025-12-17 22:46:54,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:54,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dbe575bc060ce15faa5 HTTP/1.1" 502 150 2025-12-17 22:46:54,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dbe575bc060ce15faa5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:54,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dbe575bc060ce15faa5 2025-12-17 22:46:54,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dbe575bc060ce15faa5 HTTP/1.1" 502 150 2025-12-17 22:46:54,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dbe575bc060ce15faa5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:54,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408daf575bc060ce15fa69 2025-12-17 22:46:54,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408daf575bc060ce15fa69 HTTP/1.1" 502 150 2025-12-17 22:46:54,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408daf575bc060ce15fa69'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:54,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dba575bc060ce15fa9b 2025-12-17 22:46:54,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dba575bc060ce15fa9b HTTP/1.1" 502 150 2025-12-17 22:46:54,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dba575bc060ce15fa9b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:55,160 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db4575bc060ce15fa8c 2025-12-17 22:46:55,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db4575bc060ce15fa8c HTTP/1.1" 502 150 2025-12-17 22:46:55,162 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db4575bc060ce15fa8c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:55,250 - lava_callback - INFO - Received callback for node 69408fd7575bc060ce160277 INFO: 172.21.0.1:46402 - "POST /node/69408fd7575bc060ce160277 HTTP/1.0" 202 Accepted 2025-12-17 22:46:55,371 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dbc575bc060ce15faa2 2025-12-17 22:46:55,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dbc575bc060ce15faa2 HTTP/1.1" 502 150 2025-12-17 22:46:55,373 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dbc575bc060ce15faa2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:55,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da7575bc060ce15fa59 2025-12-17 22:46:55,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da7575bc060ce15fa59 HTTP/1.1" 502 150 2025-12-17 22:46:55,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da7575bc060ce15fa59'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46408 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:46:56,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dbe575bc060ce15faa5 2025-12-17 22:46:56,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dbe575bc060ce15faa5 HTTP/1.1" 502 150 2025-12-17 22:46:56,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dbe575bc060ce15faa5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:57,138 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db6575bc060ce15fa8f 2025-12-17 22:46:57,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db6575bc060ce15fa8f HTTP/1.1" 502 150 2025-12-17 22:46:57,139 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db6575bc060ce15fa8f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:57,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db8575bc060ce15fa92 2025-12-17 22:46:57,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db8575bc060ce15fa92 HTTP/1.1" 502 150 2025-12-17 22:46:57,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db8575bc060ce15fa92'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:58,305 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940924d575bc060ce160a6c 2025-12-17 22:46:58,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924d575bc060ce160a6c HTTP/1.1" 502 150 2025-12-17 22:46:58,306 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940924d575bc060ce160a6c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:58,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408daf575bc060ce15fa69 2025-12-17 22:46:58,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408daf575bc060ce15fa69 HTTP/1.1" 502 150 2025-12-17 22:46:58,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408daf575bc060ce15fa69'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:58,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dba575bc060ce15fa9b 2025-12-17 22:46:58,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dba575bc060ce15fa9b HTTP/1.1" 502 150 2025-12-17 22:46:58,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dba575bc060ce15fa9b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:59,373 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dbc575bc060ce15faa2 2025-12-17 22:46:59,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dbc575bc060ce15faa2 HTTP/1.1" 502 150 2025-12-17 22:46:59,374 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dbc575bc060ce15faa2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:59,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da3575bc060ce15fa40 2025-12-17 22:46:59,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da3575bc060ce15fa40 HTTP/1.1" 502 150 2025-12-17 22:46:59,929 - lava_callback - ERROR - Error processing callback for node 69408da3575bc060ce15fa40: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da3575bc060ce15fa40 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da3575bc060ce15fa40 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da3575bc060ce15fa40 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:46:59,931 - lava_callback - INFO - Starting async job submit for node 69408dc0575bc060ce15faa8 2025-12-17 22:46:59,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:46:59,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc0575bc060ce15faa8 HTTP/1.1" 502 150 2025-12-17 22:46:59,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc0575bc060ce15faa8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:46:59,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc0575bc060ce15faa8 2025-12-17 22:46:59,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc0575bc060ce15faa8 HTTP/1.1" 502 150 2025-12-17 22:46:59,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc0575bc060ce15faa8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:00,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db1575bc060ce15fa6c 2025-12-17 22:47:00,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db1575bc060ce15fa6c HTTP/1.1" 502 150 2025-12-17 22:47:00,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db1575bc060ce15fa6c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:00,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dbe575bc060ce15faa5 2025-12-17 22:47:00,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dbe575bc060ce15faa5 HTTP/1.1" 502 150 2025-12-17 22:47:00,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dbe575bc060ce15faa5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:01,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc0575bc060ce15faa8 2025-12-17 22:47:01,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc0575bc060ce15faa8 HTTP/1.1" 502 150 2025-12-17 22:47:01,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc0575bc060ce15faa8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46354 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:47:03,162 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db4575bc060ce15fa8c 2025-12-17 22:47:03,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db4575bc060ce15fa8c HTTP/1.1" 502 150 2025-12-17 22:47:03,163 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db4575bc060ce15fa8c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:03,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d9a575bc060ce15fa2b 2025-12-17 22:47:03,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d9a575bc060ce15fa2b HTTP/1.1" 502 150 2025-12-17 22:47:03,539 - lava_callback - ERROR - Error processing callback for node 69408d9a575bc060ce15fa2b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d9a575bc060ce15fa2b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d9a575bc060ce15fa2b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d9a575bc060ce15fa2b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:03,542 - lava_callback - INFO - Starting async job submit for node 69408dc2575bc060ce15faab 2025-12-17 22:47:03,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:03,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc2575bc060ce15faab HTTP/1.1" 502 150 2025-12-17 22:47:03,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc2575bc060ce15faab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:03,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc2575bc060ce15faab 2025-12-17 22:47:03,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc2575bc060ce15faab HTTP/1.1" 502 150 2025-12-17 22:47:03,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc2575bc060ce15faab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:04,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408daa575bc060ce15fa63 2025-12-17 22:47:05,139 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db6575bc060ce15fa8f 2025-12-17 22:47:05,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc2575bc060ce15faab 2025-12-17 22:47:05,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc2575bc060ce15faab HTTP/1.1" 502 150 2025-12-17 22:47:05,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc2575bc060ce15faab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:05,621 - lava_callback - INFO - Received callback for node 69408fd4575bc060ce160255 INFO: 172.21.0.1:46358 - "POST /node/69408fd4575bc060ce160255 HTTP/1.0" 202 Accepted 2025-12-17 22:47:05,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dad575bc060ce15fa66 2025-12-17 22:47:05,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dad575bc060ce15fa66 HTTP/1.1" 502 150 2025-12-17 22:47:05,702 - lava_callback - ERROR - Error processing callback for node 69408dad575bc060ce15fa66: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dad575bc060ce15fa66 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dad575bc060ce15fa66 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dad575bc060ce15fa66 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:05,704 - lava_callback - INFO - Starting async job submit for node 69408dc4575bc060ce15faae 2025-12-17 22:47:05,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:05,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc4575bc060ce15faae HTTP/1.1" 502 150 2025-12-17 22:47:05,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc4575bc060ce15faae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:05,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc4575bc060ce15faae 2025-12-17 22:47:05,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc4575bc060ce15faae HTTP/1.1" 502 150 2025-12-17 22:47:05,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc4575bc060ce15faae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:05,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db8575bc060ce15fa92 2025-12-17 22:47:05,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db8575bc060ce15fa92 HTTP/1.1" 502 150 2025-12-17 22:47:05,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db8575bc060ce15fa92'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:05,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc0575bc060ce15faa8 2025-12-17 22:47:05,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc0575bc060ce15faa8 HTTP/1.1" 502 150 2025-12-17 22:47:05,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc0575bc060ce15faa8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:06,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da4575bc060ce15fa4b HTTP/1.1" 502 150 2025-12-17 22:47:06,160 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408da4575bc060ce15fa4b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:06,285 - lava_callback - INFO - Received callback for node 69408fda575bc060ce160282 INFO: 172.21.0.1:46370 - "POST /node/69408fda575bc060ce160282 HTTP/1.0" 202 Accepted 2025-12-17 22:47:06,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940924d575bc060ce160a6c 2025-12-17 22:47:06,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924d575bc060ce160a6c HTTP/1.1" 502 150 2025-12-17 22:47:06,307 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940924d575bc060ce160a6c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46386 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:47:06,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408daf575bc060ce15fa69 2025-12-17 22:47:06,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408daf575bc060ce15fa69 HTTP/1.1" 502 150 2025-12-17 22:47:06,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408daf575bc060ce15fa69'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:06,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dba575bc060ce15fa9b 2025-12-17 22:47:06,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dba575bc060ce15fa9b HTTP/1.1" 502 150 2025-12-17 22:47:06,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dba575bc060ce15fa9b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:07,375 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dbc575bc060ce15faa2 2025-12-17 22:47:07,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dbc575bc060ce15faa2 HTTP/1.1" 502 150 2025-12-17 22:47:07,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dbc575bc060ce15faa2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:07,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc4575bc060ce15faae 2025-12-17 22:47:07,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc4575bc060ce15faae HTTP/1.1" 502 150 2025-12-17 22:47:07,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc4575bc060ce15faae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:08,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dbe575bc060ce15faa5 2025-12-17 22:47:08,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dbe575bc060ce15faa5 HTTP/1.1" 502 150 2025-12-17 22:47:08,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dbe575bc060ce15faa5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:09,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc2575bc060ce15faab 2025-12-17 22:47:09,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc2575bc060ce15faab HTTP/1.1" 502 150 2025-12-17 22:47:09,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc2575bc060ce15faab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:10,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da6575bc060ce15fa52 2025-12-17 22:47:10,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da6575bc060ce15fa52 HTTP/1.1" 502 150 2025-12-17 22:47:10,005 - lava_callback - ERROR - Error processing callback for node 69408da6575bc060ce15fa52: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da6575bc060ce15fa52 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da6575bc060ce15fa52 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da6575bc060ce15fa52 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:10,007 - lava_callback - INFO - Starting async job submit for node 69408dc6575bc060ce15fab1 2025-12-17 22:47:10,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:10,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc6575bc060ce15fab1 HTTP/1.1" 502 150 2025-12-17 22:47:10,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc6575bc060ce15fab1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:10,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc6575bc060ce15fab1 2025-12-17 22:47:10,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc6575bc060ce15fab1 HTTP/1.1" 502 150 2025-12-17 22:47:10,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc6575bc060ce15fab1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:11,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc4575bc060ce15faae 2025-12-17 22:47:11,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc4575bc060ce15faae HTTP/1.1" 502 150 2025-12-17 22:47:11,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc4575bc060ce15faae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:11,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da7575bc060ce15fa59 2025-12-17 22:47:11,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da7575bc060ce15fa59 HTTP/1.1" 502 150 2025-12-17 22:47:11,862 - lava_callback - ERROR - Error processing callback for node 69408da7575bc060ce15fa59: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da7575bc060ce15fa59 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da7575bc060ce15fa59 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da7575bc060ce15fa59 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:11,864 - lava_callback - INFO - Starting async job submit for node 69408dc8575bc060ce15fab4 2025-12-17 22:47:11,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:11,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc8575bc060ce15fab4 HTTP/1.1" 502 150 2025-12-17 22:47:11,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc8575bc060ce15fab4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:11,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc8575bc060ce15fab4 2025-12-17 22:47:11,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc8575bc060ce15fab4 HTTP/1.1" 502 150 2025-12-17 22:47:11,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc8575bc060ce15fab4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:12,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc6575bc060ce15fab1 2025-12-17 22:47:12,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc6575bc060ce15fab1 HTTP/1.1" 502 150 2025-12-17 22:47:12,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc6575bc060ce15fab1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:13,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc8575bc060ce15fab4 2025-12-17 22:47:13,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc8575bc060ce15fab4 HTTP/1.1" 502 150 2025-12-17 22:47:13,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc8575bc060ce15fab4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:13,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc0575bc060ce15faa8 2025-12-17 22:47:13,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc0575bc060ce15faa8 HTTP/1.1" 502 150 2025-12-17 22:47:13,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc0575bc060ce15faa8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40868 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:47:15,287 - lava_callback - INFO - Received callback for node 69408fe1575bc060ce16028b INFO: 172.21.0.1:40874 - "POST /node/69408fe1575bc060ce16028b HTTP/1.0" 202 Accepted 2025-12-17 22:47:15,963 - lava_callback - INFO - Received callback for node 69408fe3575bc060ce16028e INFO: 172.21.0.1:40882 - "POST /node/69408fe3575bc060ce16028e HTTP/1.0" 202 Accepted 2025-12-17 22:47:16,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc6575bc060ce15fab1 2025-12-17 22:47:16,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db1575bc060ce15fa6c 2025-12-17 22:47:16,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db1575bc060ce15fa6c HTTP/1.1" 502 150 2025-12-17 22:47:16,028 - lava_callback - ERROR - Error processing callback for node 69408db1575bc060ce15fa6c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408db1575bc060ce15fa6c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408db1575bc060ce15fa6c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408db1575bc060ce15fa6c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:16,030 - lava_callback - INFO - Starting async job submit for node 69408dca575bc060ce15fab7 2025-12-17 22:47:16,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:16,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dca575bc060ce15fab7 HTTP/1.1" 502 150 2025-12-17 22:47:16,037 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dca575bc060ce15fab7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:16,037 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dca575bc060ce15fab7 2025-12-17 22:47:16,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dca575bc060ce15fab7 HTTP/1.1" 502 150 2025-12-17 22:47:16,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dca575bc060ce15fab7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40898 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:47:16,647 - lava_callback - INFO - Received callback for node 69408fe5575bc060ce160291 INFO: 172.21.0.1:40906 - "POST /node/69408fe5575bc060ce160291 HTTP/1.0" 202 Accepted 2025-12-17 22:47:17,317 - lava_callback - INFO - Received callback for node 69408fe7575bc060ce160294 INFO: 172.21.0.1:40922 - "POST /node/69408fe7575bc060ce160294 HTTP/1.0" 202 Accepted 2025-12-17 22:47:17,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc2575bc060ce15faab 2025-12-17 22:47:17,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc8575bc060ce15fab4 2025-12-17 22:47:17,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc8575bc060ce15fab4 HTTP/1.1" 502 150 2025-12-17 22:47:17,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc8575bc060ce15fab4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:17,957 - lava_callback - INFO - Received callback for node 69408fe9575bc060ce160299 INFO: 172.21.0.1:40938 - "POST /node/69408fe9575bc060ce160299 HTTP/1.0" 202 Accepted 2025-12-17 22:47:18,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dca575bc060ce15fab7 2025-12-17 22:47:18,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dca575bc060ce15fab7 HTTP/1.1" 502 150 2025-12-17 22:47:18,039 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dca575bc060ce15fab7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:18,606 - lava_callback - INFO - Received callback for node 69408fea575bc060ce16029f INFO: 172.21.0.1:40940 - "POST /node/69408fea575bc060ce16029f HTTP/1.0" 202 Accepted 2025-12-17 22:47:19,164 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db4575bc060ce15fa8c 2025-12-17 22:47:19,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db4575bc060ce15fa8c HTTP/1.1" 502 150 2025-12-17 22:47:19,165 - lava_callback - ERROR - Error processing callback for node 69408db4575bc060ce15fa8c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408db4575bc060ce15fa8c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408db4575bc060ce15fa8c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408db4575bc060ce15fa8c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:19,167 - lava_callback - INFO - Starting async job submit for node 69408dd7575bc060ce15fad7 2025-12-17 22:47:19,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:19,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd7575bc060ce15fad7 HTTP/1.1" 502 150 2025-12-17 22:47:19,197 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd7575bc060ce15fad7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:19,197 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd7575bc060ce15fad7 2025-12-17 22:47:19,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd7575bc060ce15fad7 HTTP/1.1" 502 150 2025-12-17 22:47:19,198 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd7575bc060ce15fad7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:19,263 - lava_callback - INFO - Received callback for node 69408fec575bc060ce1602a5 INFO: 172.21.0.1:40944 - "POST /node/69408fec575bc060ce1602a5 HTTP/1.0" 202 Accepted 2025-12-17 22:47:19,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408daa575bc060ce15fa63 HTTP/1.1" 502 150 2025-12-17 22:47:19,472 - lava_callback - ERROR - Error processing callback for node 69408daa575bc060ce15fa63: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408daa575bc060ce15fa63 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408daa575bc060ce15fa63 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408daa575bc060ce15fa63 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:19,474 - lava_callback - INFO - Starting async job submit for node 69408ddb575bc060ce15fadd 2025-12-17 22:47:19,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:19,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddb575bc060ce15fadd HTTP/1.1" 502 150 2025-12-17 22:47:19,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ddb575bc060ce15fadd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:19,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ddb575bc060ce15fadd 2025-12-17 22:47:19,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddb575bc060ce15fadd HTTP/1.1" 502 150 2025-12-17 22:47:19,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ddb575bc060ce15fadd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:19,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc4575bc060ce15faae 2025-12-17 22:47:19,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc4575bc060ce15faae HTTP/1.1" 502 150 2025-12-17 22:47:19,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc4575bc060ce15faae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:19,908 - lava_callback - INFO - Received callback for node 69408ff0575bc060ce1602ba INFO: 172.21.0.1:41004 - "POST /node/69408ff0575bc060ce1602ba HTTP/1.0" 202 Accepted 2025-12-17 22:47:20,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db6575bc060ce15fa8f HTTP/1.1" 502 150 2025-12-17 22:47:20,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408db6575bc060ce15fa8f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:20,574 - lava_callback - INFO - Received callback for node 69408ff1575bc060ce1602bd INFO: 172.21.0.1:41016 - "POST /node/69408ff1575bc060ce1602bd HTTP/1.0" 202 Accepted 2025-12-17 22:47:21,198 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd7575bc060ce15fad7 2025-12-17 22:47:21,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd7575bc060ce15fad7 HTTP/1.1" 502 150 2025-12-17 22:47:21,199 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd7575bc060ce15fad7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:21,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ddb575bc060ce15fadd 2025-12-17 22:47:21,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddb575bc060ce15fadd HTTP/1.1" 502 150 2025-12-17 22:47:21,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ddb575bc060ce15fadd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:21,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db8575bc060ce15fa92 2025-12-17 22:47:21,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db8575bc060ce15fa92 HTTP/1.1" 502 150 2025-12-17 22:47:21,761 - lava_callback - ERROR - Error processing callback for node 69408db8575bc060ce15fa92: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408db8575bc060ce15fa92 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408db8575bc060ce15fa92 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408db8575bc060ce15fa92 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:21,763 - lava_callback - INFO - Starting async job submit for node 69408dcc575bc060ce15fabd 2025-12-17 22:47:21,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:21,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dcc575bc060ce15fabd HTTP/1.1" 502 150 2025-12-17 22:47:21,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dcc575bc060ce15fabd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:21,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dcc575bc060ce15fabd 2025-12-17 22:47:21,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dcc575bc060ce15fabd HTTP/1.1" 502 150 2025-12-17 22:47:21,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dcc575bc060ce15fabd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:22,039 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dca575bc060ce15fab7 2025-12-17 22:47:22,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dca575bc060ce15fab7 HTTP/1.1" 502 150 2025-12-17 22:47:22,041 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dca575bc060ce15fab7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:22,160 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408da4575bc060ce15fa4b 2025-12-17 22:47:22,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408da4575bc060ce15fa4b HTTP/1.1" 502 150 2025-12-17 22:47:22,162 - lava_callback - ERROR - Error processing callback for node 69408da4575bc060ce15fa4b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da4575bc060ce15fa4b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da4575bc060ce15fa4b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408da4575bc060ce15fa4b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:22,164 - lava_callback - INFO - Starting async job submit for node 69408dce575bc060ce15fac8 2025-12-17 22:47:22,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:22,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dce575bc060ce15fac8 HTTP/1.1" 502 150 2025-12-17 22:47:22,170 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dce575bc060ce15fac8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:22,170 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dce575bc060ce15fac8 2025-12-17 22:47:22,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dce575bc060ce15fac8 HTTP/1.1" 502 150 2025-12-17 22:47:22,171 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dce575bc060ce15fac8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:22,307 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940924d575bc060ce160a6c 2025-12-17 22:47:22,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924d575bc060ce160a6c HTTP/1.1" 502 150 2025-12-17 22:47:22,308 - lava_callback - ERROR - Error processing callback for node 6940924d575bc060ce160a6c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940924d575bc060ce160a6c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940924d575bc060ce160a6c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940924d575bc060ce160a6c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:22,311 - lava_callback - INFO - Starting async job submit for node 69408dd0575bc060ce15facb 2025-12-17 22:47:22,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:22,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd0575bc060ce15facb HTTP/1.1" 502 150 2025-12-17 22:47:22,321 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd0575bc060ce15facb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:22,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd0575bc060ce15facb 2025-12-17 22:47:22,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd0575bc060ce15facb HTTP/1.1" 502 150 2025-12-17 22:47:22,322 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd0575bc060ce15facb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:22,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408daf575bc060ce15fa69 2025-12-17 22:47:22,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408daf575bc060ce15fa69 HTTP/1.1" 502 150 2025-12-17 22:47:22,934 - lava_callback - ERROR - Error processing callback for node 69408daf575bc060ce15fa69: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408daf575bc060ce15fa69 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408daf575bc060ce15fa69 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408daf575bc060ce15fa69 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:22,937 - lava_callback - INFO - Starting async job submit for node 69408dd2575bc060ce15face 2025-12-17 22:47:22,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:22,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd2575bc060ce15face HTTP/1.1" 502 150 2025-12-17 22:47:22,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd2575bc060ce15face'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:22,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd2575bc060ce15face 2025-12-17 22:47:22,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd2575bc060ce15face HTTP/1.1" 502 150 2025-12-17 22:47:22,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd2575bc060ce15face'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:22,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dba575bc060ce15fa9b 2025-12-17 22:47:22,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dba575bc060ce15fa9b HTTP/1.1" 502 150 2025-12-17 22:47:22,949 - lava_callback - ERROR - Error processing callback for node 69408dba575bc060ce15fa9b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dba575bc060ce15fa9b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dba575bc060ce15fa9b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dba575bc060ce15fa9b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:22,952 - lava_callback - INFO - Starting async job submit for node 69408dd4575bc060ce15fad1 2025-12-17 22:47:22,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:22,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd4575bc060ce15fad1 HTTP/1.1" 502 150 2025-12-17 22:47:22,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd4575bc060ce15fad1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:22,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd4575bc060ce15fad1 2025-12-17 22:47:22,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd4575bc060ce15fad1 HTTP/1.1" 502 150 2025-12-17 22:47:22,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd4575bc060ce15fad1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:23,376 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dbc575bc060ce15faa2 2025-12-17 22:47:23,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dbc575bc060ce15faa2 HTTP/1.1" 502 150 2025-12-17 22:47:23,383 - lava_callback - ERROR - Error processing callback for node 69408dbc575bc060ce15faa2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dbc575bc060ce15faa2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dbc575bc060ce15faa2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dbc575bc060ce15faa2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:23,385 - lava_callback - INFO - Starting async job submit for node 69408dd6575bc060ce15fad4 2025-12-17 22:47:23,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:23,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd6575bc060ce15fad4 HTTP/1.1" 502 150 2025-12-17 22:47:23,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd6575bc060ce15fad4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:23,395 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd6575bc060ce15fad4 2025-12-17 22:47:23,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd6575bc060ce15fad4 HTTP/1.1" 502 150 2025-12-17 22:47:23,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd6575bc060ce15fad4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:23,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dcc575bc060ce15fabd 2025-12-17 22:47:23,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dcc575bc060ce15fabd HTTP/1.1" 502 150 2025-12-17 22:47:23,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dcc575bc060ce15fabd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:24,171 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dce575bc060ce15fac8 2025-12-17 22:47:24,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dce575bc060ce15fac8 HTTP/1.1" 502 150 2025-12-17 22:47:24,172 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dce575bc060ce15fac8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:24,322 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd0575bc060ce15facb 2025-12-17 22:47:24,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd0575bc060ce15facb HTTP/1.1" 502 150 2025-12-17 22:47:24,323 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd0575bc060ce15facb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:24,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dbe575bc060ce15faa5 2025-12-17 22:47:24,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dbe575bc060ce15faa5 HTTP/1.1" 502 150 2025-12-17 22:47:24,684 - lava_callback - ERROR - Error processing callback for node 69408dbe575bc060ce15faa5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dbe575bc060ce15faa5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dbe575bc060ce15faa5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dbe575bc060ce15faa5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:24,686 - lava_callback - INFO - Starting async job submit for node 69408de1575bc060ce15faf2 2025-12-17 22:47:24,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:24,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de1575bc060ce15faf2 HTTP/1.1" 502 150 2025-12-17 22:47:24,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de1575bc060ce15faf2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:24,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de1575bc060ce15faf2 2025-12-17 22:47:24,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de1575bc060ce15faf2 HTTP/1.1" 502 150 2025-12-17 22:47:24,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de1575bc060ce15faf2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:24,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd2575bc060ce15face 2025-12-17 22:47:24,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd2575bc060ce15face HTTP/1.1" 502 150 2025-12-17 22:47:24,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd2575bc060ce15face'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:24,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd4575bc060ce15fad1 2025-12-17 22:47:24,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd4575bc060ce15fad1 HTTP/1.1" 502 150 2025-12-17 22:47:24,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd4575bc060ce15fad1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:25,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd7575bc060ce15fad7 2025-12-17 22:47:25,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd7575bc060ce15fad7 HTTP/1.1" 502 150 2025-12-17 22:47:25,201 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd7575bc060ce15fad7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:25,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd6575bc060ce15fad4 2025-12-17 22:47:25,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd6575bc060ce15fad4 HTTP/1.1" 502 150 2025-12-17 22:47:25,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd6575bc060ce15fad4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:25,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ddb575bc060ce15fadd 2025-12-17 22:47:25,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddb575bc060ce15fadd HTTP/1.1" 502 150 2025-12-17 22:47:25,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ddb575bc060ce15fadd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:25,629 - lava_callback - INFO - Received callback for node 69408fed575bc060ce1602ac INFO: 172.21.0.1:41030 - "POST /node/69408fed575bc060ce1602ac HTTP/1.0" 202 Accepted 2025-12-17 22:47:25,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc8575bc060ce15fab4 2025-12-17 22:47:25,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc8575bc060ce15fab4 HTTP/1.1" 502 150 2025-12-17 22:47:25,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc8575bc060ce15fab4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:26,274 - lava_callback - INFO - Received callback for node 69408fee575bc060ce1602b7 INFO: 172.21.0.1:41036 - "POST /node/69408fee575bc060ce1602b7 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:41052 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:47:26,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de1575bc060ce15faf2 2025-12-17 22:47:26,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de1575bc060ce15faf2 HTTP/1.1" 502 150 2025-12-17 22:47:26,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de1575bc060ce15faf2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:26,926 - lava_callback - INFO - Received callback for node 69408ff7575bc060ce1602cc INFO: 172.21.0.1:41066 - "POST /node/69408ff7575bc060ce1602cc HTTP/1.0" 202 Accepted 2025-12-17 22:47:27,600 - lava_callback - INFO - Received callback for node 69408ffb575bc060ce1602d2 INFO: 172.21.0.1:41068 - "POST /node/69408ffb575bc060ce1602d2 HTTP/1.0" 202 Accepted 2025-12-17 22:47:27,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dcc575bc060ce15fabd 2025-12-17 22:47:27,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dcc575bc060ce15fabd HTTP/1.1" 502 150 2025-12-17 22:47:27,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dcc575bc060ce15fabd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:28,173 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dce575bc060ce15fac8 2025-12-17 22:47:28,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dce575bc060ce15fac8 HTTP/1.1" 502 150 2025-12-17 22:47:28,174 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dce575bc060ce15fac8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:28,324 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd0575bc060ce15facb 2025-12-17 22:47:28,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd0575bc060ce15facb HTTP/1.1" 502 150 2025-12-17 22:47:28,325 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd0575bc060ce15facb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:28,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd2575bc060ce15face 2025-12-17 22:47:28,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd2575bc060ce15face HTTP/1.1" 502 150 2025-12-17 22:47:28,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd2575bc060ce15face'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:28,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd4575bc060ce15fad1 2025-12-17 22:47:28,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd4575bc060ce15fad1 HTTP/1.1" 502 150 2025-12-17 22:47:28,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd4575bc060ce15fad1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:29,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd6575bc060ce15fad4 2025-12-17 22:47:29,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd6575bc060ce15fad4 HTTP/1.1" 502 150 2025-12-17 22:47:29,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd6575bc060ce15fad4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:29,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc0575bc060ce15faa8 2025-12-17 22:47:29,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc0575bc060ce15faa8 HTTP/1.1" 502 150 2025-12-17 22:47:29,945 - lava_callback - ERROR - Error processing callback for node 69408dc0575bc060ce15faa8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dc0575bc060ce15faa8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dc0575bc060ce15faa8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dc0575bc060ce15faa8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:29,946 - lava_callback - INFO - Starting async job submit for node 69408dd9575bc060ce15fada 2025-12-17 22:47:29,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:29,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd9575bc060ce15fada HTTP/1.1" 502 150 2025-12-17 22:47:29,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd9575bc060ce15fada'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:29,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd9575bc060ce15fada 2025-12-17 22:47:29,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd9575bc060ce15fada HTTP/1.1" 502 150 2025-12-17 22:47:29,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd9575bc060ce15fada'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:30,041 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dca575bc060ce15fab7 2025-12-17 22:47:30,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de1575bc060ce15faf2 2025-12-17 22:47:30,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de1575bc060ce15faf2 HTTP/1.1" 502 150 2025-12-17 22:47:30,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de1575bc060ce15faf2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:31,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc6575bc060ce15fab1 HTTP/1.1" 502 150 2025-12-17 22:47:31,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc6575bc060ce15fab1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:31,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd9575bc060ce15fada 2025-12-17 22:47:31,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd9575bc060ce15fada HTTP/1.1" 502 150 2025-12-17 22:47:31,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd9575bc060ce15fada'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:33,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc2575bc060ce15faab HTTP/1.1" 502 150 2025-12-17 22:47:33,040 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc2575bc060ce15faab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:33,201 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd7575bc060ce15fad7 2025-12-17 22:47:33,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd7575bc060ce15fad7 HTTP/1.1" 502 150 2025-12-17 22:47:33,202 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd7575bc060ce15fad7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:33,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ddb575bc060ce15fadd 2025-12-17 22:47:33,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddb575bc060ce15fadd HTTP/1.1" 502 150 2025-12-17 22:47:33,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ddb575bc060ce15fadd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:35,312 - lava_callback - INFO - Received callback for node 69408ff2575bc060ce1602c0 INFO: 172.21.0.1:60446 - "POST /node/69408ff2575bc060ce1602c0 HTTP/1.0" 202 Accepted 2025-12-17 22:47:35,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc4575bc060ce15faae 2025-12-17 22:47:35,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc4575bc060ce15faae HTTP/1.1" 502 150 2025-12-17 22:47:35,727 - lava_callback - ERROR - Error processing callback for node 69408dc4575bc060ce15faae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dc4575bc060ce15faae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dc4575bc060ce15faae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dc4575bc060ce15faae (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:35,729 - lava_callback - INFO - Starting async job submit for node 69408ddc575bc060ce15fae0 2025-12-17 22:47:35,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:35,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddc575bc060ce15fae0 HTTP/1.1" 502 150 2025-12-17 22:47:35,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ddc575bc060ce15fae0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:35,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ddc575bc060ce15fae0 2025-12-17 22:47:35,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddc575bc060ce15fae0 HTTP/1.1" 502 150 2025-12-17 22:47:35,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ddc575bc060ce15fae0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:35,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dcc575bc060ce15fabd 2025-12-17 22:47:35,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dcc575bc060ce15fabd HTTP/1.1" 502 150 2025-12-17 22:47:35,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dcc575bc060ce15fabd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:35,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd9575bc060ce15fada 2025-12-17 22:47:35,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd9575bc060ce15fada HTTP/1.1" 502 150 2025-12-17 22:47:35,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd9575bc060ce15fada'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:35,997 - lava_callback - INFO - Received callback for node 69408ff4575bc060ce1602c3 INFO: 172.21.0.1:60454 - "POST /node/69408ff4575bc060ce1602c3 HTTP/1.0" 202 Accepted 2025-12-17 22:47:36,174 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dce575bc060ce15fac8 2025-12-17 22:47:36,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dce575bc060ce15fac8 HTTP/1.1" 502 150 2025-12-17 22:47:36,175 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dce575bc060ce15fac8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:36,325 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd0575bc060ce15facb 2025-12-17 22:47:36,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd0575bc060ce15facb HTTP/1.1" 502 150 2025-12-17 22:47:36,326 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd0575bc060ce15facb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:36,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408db6575bc060ce15fa8f 2025-12-17 22:47:36,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408db6575bc060ce15fa8f HTTP/1.1" 502 150 2025-12-17 22:47:36,498 - lava_callback - ERROR - Error processing callback for node 69408db6575bc060ce15fa8f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408db6575bc060ce15fa8f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408db6575bc060ce15fa8f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408db6575bc060ce15fa8f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:36,501 - lava_callback - INFO - Starting async job submit for node 69408dde575bc060ce15fae3 2025-12-17 22:47:36,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:60460 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:47:36,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dde575bc060ce15fae3 HTTP/1.1" 502 150 2025-12-17 22:47:36,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dde575bc060ce15fae3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:36,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dde575bc060ce15fae3 2025-12-17 22:47:36,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dde575bc060ce15fae3 HTTP/1.1" 502 150 2025-12-17 22:47:36,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dde575bc060ce15fae3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:36,667 - lava_callback - INFO - Received callback for node 69408ff5575bc060ce1602c6 INFO: 172.21.0.1:60462 - "POST /node/69408ff5575bc060ce1602c6 HTTP/1.0" 202 Accepted 2025-12-17 22:47:36,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd2575bc060ce15face 2025-12-17 22:47:36,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd2575bc060ce15face HTTP/1.1" 502 150 2025-12-17 22:47:36,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd2575bc060ce15face'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:36,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd4575bc060ce15fad1 2025-12-17 22:47:36,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd4575bc060ce15fad1 HTTP/1.1" 502 150 2025-12-17 22:47:36,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd4575bc060ce15fad1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:37,388 - lava_callback - INFO - Received callback for node 69408ff6575bc060ce1602c9 INFO: 172.21.0.1:60472 - "POST /node/69408ff6575bc060ce1602c9 HTTP/1.0" 202 Accepted 2025-12-17 22:47:37,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd6575bc060ce15fad4 2025-12-17 22:47:37,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd6575bc060ce15fad4 HTTP/1.1" 502 150 2025-12-17 22:47:37,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd6575bc060ce15fad4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:37,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ddc575bc060ce15fae0 2025-12-17 22:47:37,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddc575bc060ce15fae0 HTTP/1.1" 502 150 2025-12-17 22:47:37,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ddc575bc060ce15fae0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:38,081 - lava_callback - INFO - Received callback for node 69408ffa575bc060ce1602cf INFO: 172.21.0.1:60484 - "POST /node/69408ffa575bc060ce1602cf HTTP/1.0" 202 Accepted 2025-12-17 22:47:38,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dde575bc060ce15fae3 2025-12-17 22:47:38,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dde575bc060ce15fae3 HTTP/1.1" 502 150 2025-12-17 22:47:38,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dde575bc060ce15fae3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:38,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de1575bc060ce15faf2 2025-12-17 22:47:38,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de1575bc060ce15faf2 HTTP/1.1" 502 150 2025-12-17 22:47:38,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de1575bc060ce15faf2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:39,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc6575bc060ce15fab1 2025-12-17 22:47:39,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc6575bc060ce15fab1 HTTP/1.1" 502 150 2025-12-17 22:47:39,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dc6575bc060ce15fab1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:41,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ddc575bc060ce15fae0 2025-12-17 22:47:41,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddc575bc060ce15fae0 HTTP/1.1" 502 150 2025-12-17 22:47:41,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ddc575bc060ce15fae0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:41,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc8575bc060ce15fab4 2025-12-17 22:47:41,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc8575bc060ce15fab4 HTTP/1.1" 502 150 2025-12-17 22:47:41,890 - lava_callback - ERROR - Error processing callback for node 69408dc8575bc060ce15fab4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dc8575bc060ce15fab4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dc8575bc060ce15fab4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dc8575bc060ce15fab4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:41,891 - lava_callback - INFO - Starting async job submit for node 69408ddf575bc060ce15fae6 2025-12-17 22:47:41,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:41,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddf575bc060ce15fae6 HTTP/1.1" 502 150 2025-12-17 22:47:41,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ddf575bc060ce15fae6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:41,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ddf575bc060ce15fae6 2025-12-17 22:47:41,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddf575bc060ce15fae6 HTTP/1.1" 502 150 2025-12-17 22:47:41,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ddf575bc060ce15fae6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:42,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dde575bc060ce15fae3 2025-12-17 22:47:42,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dde575bc060ce15fae3 HTTP/1.1" 502 150 2025-12-17 22:47:42,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dde575bc060ce15fae3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:43,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ddf575bc060ce15fae6 2025-12-17 22:47:43,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddf575bc060ce15fae6 HTTP/1.1" 502 150 2025-12-17 22:47:43,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ddf575bc060ce15fae6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:43,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd9575bc060ce15fada 2025-12-17 22:47:43,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd9575bc060ce15fada HTTP/1.1" 502 150 2025-12-17 22:47:43,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dd9575bc060ce15fada'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:45,643 - lava_callback - INFO - Received callback for node 69408ffd575bc060ce1602db INFO: 172.21.0.1:60710 - "POST /node/69408ffd575bc060ce1602db HTTP/1.0" 202 Accepted 2025-12-17 22:47:46,297 - lava_callback - INFO - Received callback for node 69409004575bc060ce1602e0 INFO: 172.21.0.1:60716 - "POST /node/69409004575bc060ce1602e0 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:60726 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:47:46,942 - lava_callback - INFO - Received callback for node 69409006575bc060ce1602ee INFO: 172.21.0.1:60728 - "POST /node/69409006575bc060ce1602ee HTTP/1.0" 202 Accepted 2025-12-17 22:47:47,595 - lava_callback - INFO - Received callback for node 69409008575bc060ce1602f1 INFO: 172.21.0.1:60730 - "POST /node/69409008575bc060ce1602f1 HTTP/1.0" 202 Accepted 2025-12-17 22:47:47,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ddf575bc060ce15fae6 2025-12-17 22:47:47,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddf575bc060ce15fae6 HTTP/1.1" 502 150 2025-12-17 22:47:47,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ddf575bc060ce15fae6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:48,243 - lava_callback - INFO - Received callback for node 6940900a575bc060ce1602f4 INFO: 172.21.0.1:60744 - "POST /node/6940900a575bc060ce1602f4 HTTP/1.0" 202 Accepted 2025-12-17 22:47:48,907 - lava_callback - INFO - Received callback for node 6940900c575bc060ce1602f7 INFO: 172.21.0.1:60752 - "POST /node/6940900c575bc060ce1602f7 HTTP/1.0" 202 Accepted 2025-12-17 22:47:49,041 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc2575bc060ce15faab 2025-12-17 22:47:49,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc2575bc060ce15faab HTTP/1.1" 502 150 2025-12-17 22:47:49,042 - lava_callback - ERROR - Error processing callback for node 69408dc2575bc060ce15faab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dc2575bc060ce15faab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dc2575bc060ce15faab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dc2575bc060ce15faab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:49,043 - lava_callback - INFO - Starting async job submit for node 69408de0575bc060ce15faed 2025-12-17 22:47:49,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:49,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de0575bc060ce15faed HTTP/1.1" 502 150 2025-12-17 22:47:49,050 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de0575bc060ce15faed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:49,050 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de0575bc060ce15faed 2025-12-17 22:47:49,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de0575bc060ce15faed HTTP/1.1" 502 150 2025-12-17 22:47:49,051 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de0575bc060ce15faed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:49,203 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd7575bc060ce15fad7 2025-12-17 22:47:49,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd7575bc060ce15fad7 HTTP/1.1" 502 150 2025-12-17 22:47:49,204 - lava_callback - ERROR - Error processing callback for node 69408dd7575bc060ce15fad7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd7575bc060ce15fad7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd7575bc060ce15fad7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd7575bc060ce15fad7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:49,206 - lava_callback - INFO - Starting async job submit for node 69408de3575bc060ce15faf5 2025-12-17 22:47:49,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:49,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de3575bc060ce15faf5 HTTP/1.1" 502 150 2025-12-17 22:47:49,213 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de3575bc060ce15faf5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:49,213 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de3575bc060ce15faf5 2025-12-17 22:47:49,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de3575bc060ce15faf5 HTTP/1.1" 502 150 2025-12-17 22:47:49,213 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de3575bc060ce15faf5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:49,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ddb575bc060ce15fadd 2025-12-17 22:47:49,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddb575bc060ce15fadd HTTP/1.1" 502 150 2025-12-17 22:47:49,489 - lava_callback - ERROR - Error processing callback for node 69408ddb575bc060ce15fadd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ddb575bc060ce15fadd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ddb575bc060ce15fadd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ddb575bc060ce15fadd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:49,491 - lava_callback - INFO - Starting async job submit for node 69408de4575bc060ce15faf8 2025-12-17 22:47:49,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:49,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de4575bc060ce15faf8 HTTP/1.1" 502 150 2025-12-17 22:47:49,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de4575bc060ce15faf8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:49,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de4575bc060ce15faf8 2025-12-17 22:47:49,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de4575bc060ce15faf8 HTTP/1.1" 502 150 2025-12-17 22:47:49,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de4575bc060ce15faf8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:49,572 - lava_callback - INFO - Received callback for node 6940900e575bc060ce1602fa INFO: 172.21.0.1:60762 - "POST /node/6940900e575bc060ce1602fa HTTP/1.0" 202 Accepted 2025-12-17 22:47:49,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ddc575bc060ce15fae0 2025-12-17 22:47:49,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddc575bc060ce15fae0 HTTP/1.1" 502 150 2025-12-17 22:47:49,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ddc575bc060ce15fae0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:50,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dde575bc060ce15fae3 2025-12-17 22:47:50,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dde575bc060ce15fae3 HTTP/1.1" 502 150 2025-12-17 22:47:50,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dde575bc060ce15fae3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:51,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de0575bc060ce15faed 2025-12-17 22:47:51,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de0575bc060ce15faed HTTP/1.1" 502 150 2025-12-17 22:47:51,052 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de0575bc060ce15faed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:51,214 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de3575bc060ce15faf5 2025-12-17 22:47:51,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de3575bc060ce15faf5 HTTP/1.1" 502 150 2025-12-17 22:47:51,214 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de3575bc060ce15faf5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:51,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de4575bc060ce15faf8 2025-12-17 22:47:51,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de4575bc060ce15faf8 HTTP/1.1" 502 150 2025-12-17 22:47:51,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de4575bc060ce15faf8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:51,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dcc575bc060ce15fabd 2025-12-17 22:47:51,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dcc575bc060ce15fabd HTTP/1.1" 502 150 2025-12-17 22:47:51,786 - lava_callback - ERROR - Error processing callback for node 69408dcc575bc060ce15fabd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dcc575bc060ce15fabd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dcc575bc060ce15fabd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dcc575bc060ce15fabd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:51,788 - lava_callback - INFO - Starting async job submit for node 69408de5575bc060ce15fafb 2025-12-17 22:47:51,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:51,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de5575bc060ce15fafb HTTP/1.1" 502 150 2025-12-17 22:47:51,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de5575bc060ce15fafb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:51,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de5575bc060ce15fafb 2025-12-17 22:47:51,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de5575bc060ce15fafb HTTP/1.1" 502 150 2025-12-17 22:47:51,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de5575bc060ce15fafb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:52,176 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dce575bc060ce15fac8 2025-12-17 22:47:52,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dce575bc060ce15fac8 HTTP/1.1" 502 150 2025-12-17 22:47:52,177 - lava_callback - ERROR - Error processing callback for node 69408dce575bc060ce15fac8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dce575bc060ce15fac8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dce575bc060ce15fac8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dce575bc060ce15fac8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:52,179 - lava_callback - INFO - Starting async job submit for node 69408de6575bc060ce15fafe 2025-12-17 22:47:52,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:52,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de6575bc060ce15fafe HTTP/1.1" 502 150 2025-12-17 22:47:52,186 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de6575bc060ce15fafe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:52,186 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de6575bc060ce15fafe 2025-12-17 22:47:52,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de6575bc060ce15fafe HTTP/1.1" 502 150 2025-12-17 22:47:52,187 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de6575bc060ce15fafe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:52,326 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd0575bc060ce15facb 2025-12-17 22:47:52,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd0575bc060ce15facb HTTP/1.1" 502 150 2025-12-17 22:47:52,328 - lava_callback - ERROR - Error processing callback for node 69408dd0575bc060ce15facb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd0575bc060ce15facb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd0575bc060ce15facb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd0575bc060ce15facb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:52,329 - lava_callback - INFO - Starting async job submit for node 69408de8575bc060ce15fb01 2025-12-17 22:47:52,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:52,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de8575bc060ce15fb01 HTTP/1.1" 502 150 2025-12-17 22:47:52,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de8575bc060ce15fb01'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:52,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de8575bc060ce15fb01 2025-12-17 22:47:52,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de8575bc060ce15fb01 HTTP/1.1" 502 150 2025-12-17 22:47:52,338 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de8575bc060ce15fb01'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:52,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd2575bc060ce15face 2025-12-17 22:47:52,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd2575bc060ce15face HTTP/1.1" 502 150 2025-12-17 22:47:52,954 - lava_callback - ERROR - Error processing callback for node 69408dd2575bc060ce15face: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd2575bc060ce15face (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd2575bc060ce15face (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd2575bc060ce15face (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:52,955 - lava_callback - INFO - Starting async job submit for node 69408de9575bc060ce15fb04 2025-12-17 22:47:52,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:52,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de9575bc060ce15fb04 HTTP/1.1" 502 150 2025-12-17 22:47:52,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de9575bc060ce15fb04'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:52,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de9575bc060ce15fb04 2025-12-17 22:47:52,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de9575bc060ce15fb04 HTTP/1.1" 502 150 2025-12-17 22:47:52,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de9575bc060ce15fb04'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:52,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd4575bc060ce15fad1 2025-12-17 22:47:52,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd4575bc060ce15fad1 HTTP/1.1" 502 150 2025-12-17 22:47:52,964 - lava_callback - ERROR - Error processing callback for node 69408dd4575bc060ce15fad1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd4575bc060ce15fad1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd4575bc060ce15fad1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd4575bc060ce15fad1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:52,966 - lava_callback - INFO - Starting async job submit for node 6940fa30460bc8e265115bb6 2025-12-17 22:47:52,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:52,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fa30460bc8e265115bb6 HTTP/1.1" 502 150 2025-12-17 22:47:52,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940fa30460bc8e265115bb6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:52,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940fa30460bc8e265115bb6 2025-12-17 22:47:52,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fa30460bc8e265115bb6 HTTP/1.1" 502 150 2025-12-17 22:47:52,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940fa30460bc8e265115bb6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:53,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd6575bc060ce15fad4 2025-12-17 22:47:53,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd6575bc060ce15fad4 HTTP/1.1" 502 150 2025-12-17 22:47:53,401 - lava_callback - ERROR - Error processing callback for node 69408dd6575bc060ce15fad4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd6575bc060ce15fad4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd6575bc060ce15fad4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd6575bc060ce15fad4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:53,403 - lava_callback - INFO - Starting async job submit for node 69408dee575bc060ce15fb0d 2025-12-17 22:47:53,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:53,404 - lava_callback - INFO - Received callback for node 69409988575bc060ce161d2d INFO: 172.21.0.1:44048 - "POST /node/69409988575bc060ce161d2d HTTP/1.0" 202 Accepted 2025-12-17 22:47:53,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dee575bc060ce15fb0d HTTP/1.1" 502 150 2025-12-17 22:47:53,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dee575bc060ce15fb0d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:53,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dee575bc060ce15fb0d 2025-12-17 22:47:53,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dee575bc060ce15fb0d HTTP/1.1" 502 150 2025-12-17 22:47:53,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dee575bc060ce15fb0d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:53,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de5575bc060ce15fafb 2025-12-17 22:47:53,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de5575bc060ce15fafb HTTP/1.1" 502 150 2025-12-17 22:47:53,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de5575bc060ce15fafb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:54,187 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de6575bc060ce15fafe 2025-12-17 22:47:54,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de6575bc060ce15fafe HTTP/1.1" 502 150 2025-12-17 22:47:54,189 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de6575bc060ce15fafe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:54,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de8575bc060ce15fb01 2025-12-17 22:47:54,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de8575bc060ce15fb01 HTTP/1.1" 502 150 2025-12-17 22:47:54,339 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de8575bc060ce15fb01'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:54,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de1575bc060ce15faf2 2025-12-17 22:47:54,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de1575bc060ce15faf2 HTTP/1.1" 502 150 2025-12-17 22:47:54,700 - lava_callback - ERROR - Error processing callback for node 69408de1575bc060ce15faf2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de1575bc060ce15faf2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de1575bc060ce15faf2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de1575bc060ce15faf2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:54,701 - lava_callback - INFO - Starting async job submit for node 69408df3575bc060ce15fb30 2025-12-17 22:47:54,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:54,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df3575bc060ce15fb30 HTTP/1.1" 502 150 2025-12-17 22:47:54,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df3575bc060ce15fb30'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:54,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df3575bc060ce15fb30 2025-12-17 22:47:54,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df3575bc060ce15fb30 HTTP/1.1" 502 150 2025-12-17 22:47:54,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df3575bc060ce15fb30'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:54,872 - lava_callback - INFO - Received callback for node 694092d3575bc060ce160b88 INFO: 172.21.0.1:44060 - "POST /node/694092d3575bc060ce160b88 HTTP/1.0" 202 Accepted 2025-12-17 22:47:54,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de9575bc060ce15fb04 2025-12-17 22:47:54,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de9575bc060ce15fb04 HTTP/1.1" 502 150 2025-12-17 22:47:54,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de9575bc060ce15fb04'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:54,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940fa30460bc8e265115bb6 2025-12-17 22:47:54,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fa30460bc8e265115bb6 HTTP/1.1" 502 150 2025-12-17 22:47:54,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940fa30460bc8e265115bb6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:55,052 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de0575bc060ce15faed 2025-12-17 22:47:55,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de0575bc060ce15faed HTTP/1.1" 502 150 2025-12-17 22:47:55,055 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de0575bc060ce15faed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:55,215 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de3575bc060ce15faf5 2025-12-17 22:47:55,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de3575bc060ce15faf5 HTTP/1.1" 502 150 2025-12-17 22:47:55,216 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de3575bc060ce15faf5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:55,290 - lava_callback - INFO - Received callback for node 69409017575bc060ce16032a INFO: 172.21.0.1:44074 - "POST /node/69409017575bc060ce16032a HTTP/1.0" 202 Accepted 2025-12-17 22:47:55,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dee575bc060ce15fb0d 2025-12-17 22:47:55,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dee575bc060ce15fb0d HTTP/1.1" 502 150 2025-12-17 22:47:55,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dee575bc060ce15fb0d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:55,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de4575bc060ce15faf8 2025-12-17 22:47:55,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de4575bc060ce15faf8 HTTP/1.1" 502 150 2025-12-17 22:47:55,501 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de4575bc060ce15faf8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:55,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dc6575bc060ce15fab1 2025-12-17 22:47:55,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dc6575bc060ce15fab1 HTTP/1.1" 502 150 2025-12-17 22:47:55,507 - lava_callback - ERROR - Error processing callback for node 69408dc6575bc060ce15fab1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dc6575bc060ce15fab1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dc6575bc060ce15fab1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dc6575bc060ce15fab1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:55,509 - lava_callback - INFO - Starting async job submit for node 69408df4575bc060ce15fb33 2025-12-17 22:47:55,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:47:55,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df4575bc060ce15fb33 HTTP/1.1" 502 150 2025-12-17 22:47:55,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df4575bc060ce15fb33'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:55,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df4575bc060ce15fb33 2025-12-17 22:47:55,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df4575bc060ce15fb33 HTTP/1.1" 502 150 2025-12-17 22:47:55,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df4575bc060ce15fb33'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:55,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ddf575bc060ce15fae6 2025-12-17 22:47:55,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddf575bc060ce15fae6 HTTP/1.1" 502 150 2025-12-17 22:47:55,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ddf575bc060ce15fae6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:55,956 - lava_callback - INFO - Received callback for node 69409010575bc060ce160301 INFO: 172.21.0.1:44086 - "POST /node/69409010575bc060ce160301 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:44092 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:47:56,650 - lava_callback - INFO - Received callback for node 69409013575bc060ce160320 INFO: 172.21.0.1:44094 - "POST /node/69409013575bc060ce160320 HTTP/1.0" 202 Accepted 2025-12-17 22:47:56,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df3575bc060ce15fb30 2025-12-17 22:47:56,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df3575bc060ce15fb30 HTTP/1.1" 502 150 2025-12-17 22:47:56,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df3575bc060ce15fb30'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:57,315 - lava_callback - INFO - Received callback for node 69409015575bc060ce160327 INFO: 172.21.0.1:44106 - "POST /node/69409015575bc060ce160327 HTTP/1.0" 202 Accepted 2025-12-17 22:47:57,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df4575bc060ce15fb33 2025-12-17 22:47:57,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de5575bc060ce15fafb 2025-12-17 22:47:57,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de5575bc060ce15fafb HTTP/1.1" 502 150 2025-12-17 22:47:57,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de5575bc060ce15fafb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:58,189 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de6575bc060ce15fafe 2025-12-17 22:47:58,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de6575bc060ce15fafe HTTP/1.1" 502 150 2025-12-17 22:47:58,190 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de6575bc060ce15fafe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:58,339 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de8575bc060ce15fb01 2025-12-17 22:47:58,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de8575bc060ce15fb01 HTTP/1.1" 502 150 2025-12-17 22:47:58,341 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de8575bc060ce15fb01'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:58,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de9575bc060ce15fb04 2025-12-17 22:47:58,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de9575bc060ce15fb04 HTTP/1.1" 502 150 2025-12-17 22:47:58,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de9575bc060ce15fb04'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:58,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940fa30460bc8e265115bb6 2025-12-17 22:47:58,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fa30460bc8e265115bb6 HTTP/1.1" 502 150 2025-12-17 22:47:58,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940fa30460bc8e265115bb6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:59,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dee575bc060ce15fb0d 2025-12-17 22:47:59,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dee575bc060ce15fb0d HTTP/1.1" 502 150 2025-12-17 22:47:59,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dee575bc060ce15fb0d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:47:59,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dd9575bc060ce15fada 2025-12-17 22:47:59,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dd9575bc060ce15fada HTTP/1.1" 502 150 2025-12-17 22:47:59,969 - lava_callback - ERROR - Error processing callback for node 69408dd9575bc060ce15fada: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd9575bc060ce15fada (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd9575bc060ce15fada (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dd9575bc060ce15fada (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:47:59,972 - lava_callback - INFO - Starting async job submit for node 69408deb575bc060ce15fb07 2025-12-17 22:47:59,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:00,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408deb575bc060ce15fb07 HTTP/1.1" 502 150 2025-12-17 22:48:00,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408deb575bc060ce15fb07'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:00,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408deb575bc060ce15fb07 2025-12-17 22:48:00,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408deb575bc060ce15fb07 HTTP/1.1" 502 150 2025-12-17 22:48:00,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408deb575bc060ce15fb07'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:00,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dca575bc060ce15fab7 HTTP/1.1" 502 150 2025-12-17 22:48:00,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dca575bc060ce15fab7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:00,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df3575bc060ce15fb30 2025-12-17 22:48:00,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df3575bc060ce15fb30 HTTP/1.1" 502 150 2025-12-17 22:48:00,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df3575bc060ce15fb30'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:02,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408deb575bc060ce15fb07 2025-12-17 22:48:02,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408deb575bc060ce15fb07 HTTP/1.1" 502 150 2025-12-17 22:48:02,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408deb575bc060ce15fb07'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39390 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:48:03,056 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de0575bc060ce15faed 2025-12-17 22:48:03,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de0575bc060ce15faed HTTP/1.1" 502 150 2025-12-17 22:48:03,057 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de0575bc060ce15faed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:03,216 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de3575bc060ce15faf5 2025-12-17 22:48:03,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de3575bc060ce15faf5 HTTP/1.1" 502 150 2025-12-17 22:48:03,218 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de3575bc060ce15faf5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:03,346 - lava_callback - INFO - Received callback for node 69408f1b575bc060ce160073 INFO: 172.21.0.1:39392 - "POST /node/69408f1b575bc060ce160073 HTTP/1.0" 202 Accepted 2025-12-17 22:48:03,501 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de4575bc060ce15faf8 2025-12-17 22:48:03,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de4575bc060ce15faf8 HTTP/1.1" 502 150 2025-12-17 22:48:03,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de4575bc060ce15faf8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:05,643 - lava_callback - INFO - Received callback for node 69409019575bc060ce16032d INFO: 172.21.0.1:39394 - "POST /node/69409019575bc060ce16032d HTTP/1.0" 202 Accepted 2025-12-17 22:48:05,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ddc575bc060ce15fae0 2025-12-17 22:48:05,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddc575bc060ce15fae0 HTTP/1.1" 502 150 2025-12-17 22:48:05,742 - lava_callback - ERROR - Error processing callback for node 69408ddc575bc060ce15fae0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ddc575bc060ce15fae0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ddc575bc060ce15fae0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ddc575bc060ce15fae0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:05,744 - lava_callback - INFO - Starting async job submit for node 69408df5575bc060ce15fb3a 2025-12-17 22:48:05,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:05,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df5575bc060ce15fb3a HTTP/1.1" 502 150 2025-12-17 22:48:05,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df5575bc060ce15fb3a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:05,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df5575bc060ce15fb3a 2025-12-17 22:48:05,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df5575bc060ce15fb3a HTTP/1.1" 502 150 2025-12-17 22:48:05,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df5575bc060ce15fb3a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:05,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de5575bc060ce15fafb 2025-12-17 22:48:05,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de5575bc060ce15fafb HTTP/1.1" 502 150 2025-12-17 22:48:05,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de5575bc060ce15fafb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:06,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408deb575bc060ce15fb07 2025-12-17 22:48:06,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408deb575bc060ce15fb07 HTTP/1.1" 502 150 2025-12-17 22:48:06,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408deb575bc060ce15fb07'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:06,190 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de6575bc060ce15fafe 2025-12-17 22:48:06,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de6575bc060ce15fafe HTTP/1.1" 502 150 2025-12-17 22:48:06,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de6575bc060ce15fafe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:06,290 - lava_callback - INFO - Received callback for node 6940901c575bc060ce160330 INFO: 172.21.0.1:39406 - "POST /node/6940901c575bc060ce160330 HTTP/1.0" 202 Accepted 2025-12-17 22:48:06,341 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de8575bc060ce15fb01 2025-12-17 22:48:06,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de8575bc060ce15fb01 HTTP/1.1" 502 150 2025-12-17 22:48:06,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de8575bc060ce15fb01'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39412 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:48:06,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dde575bc060ce15fae3 2025-12-17 22:48:06,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dde575bc060ce15fae3 HTTP/1.1" 502 150 2025-12-17 22:48:06,542 - lava_callback - ERROR - Error processing callback for node 69408dde575bc060ce15fae3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dde575bc060ce15fae3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dde575bc060ce15fae3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dde575bc060ce15fae3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:06,544 - lava_callback - INFO - Starting async job submit for node 69408df7575bc060ce15fb41 2025-12-17 22:48:06,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:06,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df7575bc060ce15fb41 HTTP/1.1" 502 150 2025-12-17 22:48:06,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df7575bc060ce15fb41'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:06,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df7575bc060ce15fb41 2025-12-17 22:48:06,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df7575bc060ce15fb41 HTTP/1.1" 502 150 2025-12-17 22:48:06,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df7575bc060ce15fb41'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:06,937 - lava_callback - INFO - Received callback for node 6940901e575bc060ce160333 INFO: 172.21.0.1:39416 - "POST /node/6940901e575bc060ce160333 HTTP/1.0" 202 Accepted 2025-12-17 22:48:06,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de9575bc060ce15fb04 2025-12-17 22:48:06,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de9575bc060ce15fb04 HTTP/1.1" 502 150 2025-12-17 22:48:06,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408de9575bc060ce15fb04'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:06,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940fa30460bc8e265115bb6 2025-12-17 22:48:06,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fa30460bc8e265115bb6 HTTP/1.1" 502 150 2025-12-17 22:48:06,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940fa30460bc8e265115bb6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:07,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dee575bc060ce15fb0d 2025-12-17 22:48:07,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dee575bc060ce15fb0d HTTP/1.1" 502 150 2025-12-17 22:48:07,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dee575bc060ce15fb0d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:07,603 - lava_callback - INFO - Received callback for node 69409022575bc060ce160339 INFO: 172.21.0.1:39430 - "POST /node/69409022575bc060ce160339 HTTP/1.0" 202 Accepted 2025-12-17 22:48:07,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df5575bc060ce15fb3a 2025-12-17 22:48:07,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df5575bc060ce15fb3a HTTP/1.1" 502 150 2025-12-17 22:48:07,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df5575bc060ce15fb3a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:08,265 - lava_callback - INFO - Received callback for node 69409024575bc060ce16033e INFO: 172.21.0.1:39434 - "POST /node/69409024575bc060ce16033e HTTP/1.0" 202 Accepted 2025-12-17 22:48:08,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df7575bc060ce15fb41 2025-12-17 22:48:08,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df7575bc060ce15fb41 HTTP/1.1" 502 150 2025-12-17 22:48:08,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df7575bc060ce15fb41'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:08,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df3575bc060ce15fb30 2025-12-17 22:48:08,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df3575bc060ce15fb30 HTTP/1.1" 502 150 2025-12-17 22:48:08,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df3575bc060ce15fb30'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:11,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df5575bc060ce15fb3a 2025-12-17 22:48:11,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ddf575bc060ce15fae6 2025-12-17 22:48:11,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ddf575bc060ce15fae6 HTTP/1.1" 502 150 2025-12-17 22:48:11,904 - lava_callback - ERROR - Error processing callback for node 69408ddf575bc060ce15fae6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ddf575bc060ce15fae6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ddf575bc060ce15fae6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ddf575bc060ce15fae6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:11,906 - lava_callback - INFO - Starting async job submit for node 69408df8575bc060ce15fb46 2025-12-17 22:48:11,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:11,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df8575bc060ce15fb46 HTTP/1.1" 502 150 2025-12-17 22:48:11,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df8575bc060ce15fb46'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:11,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df8575bc060ce15fb46 2025-12-17 22:48:11,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df8575bc060ce15fb46 HTTP/1.1" 502 150 2025-12-17 22:48:11,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df8575bc060ce15fb46'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:12,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df7575bc060ce15fb41 2025-12-17 22:48:12,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df7575bc060ce15fb41 HTTP/1.1" 502 150 2025-12-17 22:48:12,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df7575bc060ce15fb41'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:12,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df4575bc060ce15fb33 HTTP/1.1" 502 150 2025-12-17 22:48:12,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df4575bc060ce15fb33'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:13,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df8575bc060ce15fb46 2025-12-17 22:48:13,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df8575bc060ce15fb46 HTTP/1.1" 502 150 2025-12-17 22:48:13,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df8575bc060ce15fb46'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:14,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408deb575bc060ce15fb07 2025-12-17 22:48:14,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408deb575bc060ce15fb07 HTTP/1.1" 502 150 2025-12-17 22:48:14,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408deb575bc060ce15fb07'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38708 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:48:16,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dca575bc060ce15fab7 2025-12-17 22:48:16,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dca575bc060ce15fab7 HTTP/1.1" 502 150 2025-12-17 22:48:16,692 - lava_callback - ERROR - Error processing callback for node 69408dca575bc060ce15fab7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dca575bc060ce15fab7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dca575bc060ce15fab7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dca575bc060ce15fab7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:16,695 - lava_callback - INFO - Starting async job submit for node 69408df9575bc060ce15fb49 2025-12-17 22:48:16,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:16,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df9575bc060ce15fb49 HTTP/1.1" 502 150 2025-12-17 22:48:16,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df9575bc060ce15fb49'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:16,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df9575bc060ce15fb49 2025-12-17 22:48:16,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df9575bc060ce15fb49 HTTP/1.1" 502 150 2025-12-17 22:48:16,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df9575bc060ce15fb49'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:16,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df4575bc060ce15fb33 2025-12-17 22:48:16,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df4575bc060ce15fb33 HTTP/1.1" 502 150 2025-12-17 22:48:16,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df4575bc060ce15fb33'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:17,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df8575bc060ce15fb46 2025-12-17 22:48:17,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df8575bc060ce15fb46 HTTP/1.1" 502 150 2025-12-17 22:48:17,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df8575bc060ce15fb46'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:18,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df9575bc060ce15fb49 2025-12-17 22:48:18,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df9575bc060ce15fb49 HTTP/1.1" 502 150 2025-12-17 22:48:18,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df9575bc060ce15fb49'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:19,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de0575bc060ce15faed 2025-12-17 22:48:19,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de0575bc060ce15faed HTTP/1.1" 502 150 2025-12-17 22:48:19,059 - lava_callback - ERROR - Error processing callback for node 69408de0575bc060ce15faed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de0575bc060ce15faed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de0575bc060ce15faed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de0575bc060ce15faed (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:19,061 - lava_callback - INFO - Starting async job submit for node 69408dfb575bc060ce15fb4c 2025-12-17 22:48:19,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:19,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfb575bc060ce15fb4c HTTP/1.1" 502 150 2025-12-17 22:48:19,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dfb575bc060ce15fb4c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:19,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dfb575bc060ce15fb4c 2025-12-17 22:48:19,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfb575bc060ce15fb4c HTTP/1.1" 502 150 2025-12-17 22:48:19,075 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dfb575bc060ce15fb4c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:19,218 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de3575bc060ce15faf5 2025-12-17 22:48:19,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de3575bc060ce15faf5 HTTP/1.1" 502 150 2025-12-17 22:48:19,219 - lava_callback - ERROR - Error processing callback for node 69408de3575bc060ce15faf5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de3575bc060ce15faf5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de3575bc060ce15faf5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de3575bc060ce15faf5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:19,222 - lava_callback - INFO - Starting async job submit for node 69408dfc575bc060ce15fb4f 2025-12-17 22:48:19,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:19,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfc575bc060ce15fb4f HTTP/1.1" 502 150 2025-12-17 22:48:19,257 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dfc575bc060ce15fb4f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:19,257 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dfc575bc060ce15fb4f 2025-12-17 22:48:19,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfc575bc060ce15fb4f HTTP/1.1" 502 150 2025-12-17 22:48:19,259 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dfc575bc060ce15fb4f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:19,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de4575bc060ce15faf8 2025-12-17 22:48:19,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de4575bc060ce15faf8 HTTP/1.1" 502 150 2025-12-17 22:48:19,506 - lava_callback - ERROR - Error processing callback for node 69408de4575bc060ce15faf8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de4575bc060ce15faf8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de4575bc060ce15faf8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de4575bc060ce15faf8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:19,508 - lava_callback - INFO - Starting async job submit for node 69408dfd575bc060ce15fb52 2025-12-17 22:48:19,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:19,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfd575bc060ce15fb52 HTTP/1.1" 502 150 2025-12-17 22:48:19,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dfd575bc060ce15fb52'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:19,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dfd575bc060ce15fb52 2025-12-17 22:48:19,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfd575bc060ce15fb52 HTTP/1.1" 502 150 2025-12-17 22:48:19,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dfd575bc060ce15fb52'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:20,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df7575bc060ce15fb41 2025-12-17 22:48:20,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df7575bc060ce15fb41 HTTP/1.1" 502 150 2025-12-17 22:48:20,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df7575bc060ce15fb41'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:21,075 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dfb575bc060ce15fb4c 2025-12-17 22:48:21,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfb575bc060ce15fb4c HTTP/1.1" 502 150 2025-12-17 22:48:21,077 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dfb575bc060ce15fb4c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:21,259 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dfc575bc060ce15fb4f 2025-12-17 22:48:21,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfc575bc060ce15fb4f HTTP/1.1" 502 150 2025-12-17 22:48:21,260 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dfc575bc060ce15fb4f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:21,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dfd575bc060ce15fb52 2025-12-17 22:48:21,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfd575bc060ce15fb52 HTTP/1.1" 502 150 2025-12-17 22:48:21,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dfd575bc060ce15fb52'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:21,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de5575bc060ce15fafb 2025-12-17 22:48:21,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de5575bc060ce15fafb HTTP/1.1" 502 150 2025-12-17 22:48:21,812 - lava_callback - ERROR - Error processing callback for node 69408de5575bc060ce15fafb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de5575bc060ce15fafb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de5575bc060ce15fafb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de5575bc060ce15fafb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:21,814 - lava_callback - INFO - Starting async job submit for node 69408e04575bc060ce15fb5b 2025-12-17 22:48:21,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:21,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e04575bc060ce15fb5b HTTP/1.1" 502 150 2025-12-17 22:48:21,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e04575bc060ce15fb5b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:21,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e04575bc060ce15fb5b 2025-12-17 22:48:21,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e04575bc060ce15fb5b HTTP/1.1" 502 150 2025-12-17 22:48:21,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e04575bc060ce15fb5b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:22,192 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de6575bc060ce15fafe 2025-12-17 22:48:22,342 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de8575bc060ce15fb01 2025-12-17 22:48:22,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de8575bc060ce15fb01 HTTP/1.1" 502 150 2025-12-17 22:48:22,343 - lava_callback - ERROR - Error processing callback for node 69408de8575bc060ce15fb01: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de8575bc060ce15fb01 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de8575bc060ce15fb01 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de8575bc060ce15fb01 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:22,345 - lava_callback - INFO - Starting async job submit for node 69408e05575bc060ce15fb5e 2025-12-17 22:48:22,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:22,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e05575bc060ce15fb5e HTTP/1.1" 502 150 2025-12-17 22:48:22,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e05575bc060ce15fb5e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:22,354 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e05575bc060ce15fb5e 2025-12-17 22:48:22,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e05575bc060ce15fb5e HTTP/1.1" 502 150 2025-12-17 22:48:22,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e05575bc060ce15fb5e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:22,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df9575bc060ce15fb49 2025-12-17 22:48:22,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df9575bc060ce15fb49 HTTP/1.1" 502 150 2025-12-17 22:48:22,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df9575bc060ce15fb49'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:22,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408de9575bc060ce15fb04 2025-12-17 22:48:22,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de9575bc060ce15fb04 HTTP/1.1" 502 150 2025-12-17 22:48:22,968 - lava_callback - ERROR - Error processing callback for node 69408de9575bc060ce15fb04: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de9575bc060ce15fb04 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de9575bc060ce15fb04 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de9575bc060ce15fb04 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:22,970 - lava_callback - INFO - Starting async job submit for node 69408e07575bc060ce15fb61 2025-12-17 22:48:22,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:22,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e07575bc060ce15fb61 HTTP/1.1" 502 150 2025-12-17 22:48:22,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e07575bc060ce15fb61'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:22,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e07575bc060ce15fb61 2025-12-17 22:48:22,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e07575bc060ce15fb61 HTTP/1.1" 502 150 2025-12-17 22:48:22,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e07575bc060ce15fb61'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:22,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940fa30460bc8e265115bb6 2025-12-17 22:48:22,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fa30460bc8e265115bb6 HTTP/1.1" 502 150 2025-12-17 22:48:22,997 - lava_callback - ERROR - Error processing callback for node 6940fa30460bc8e265115bb6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940fa30460bc8e265115bb6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940fa30460bc8e265115bb6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940fa30460bc8e265115bb6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:22,999 - lava_callback - INFO - Starting async job submit for node 69408dff575bc060ce15fb55 2025-12-17 22:48:23,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:23,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dee575bc060ce15fb0d 2025-12-17 22:48:23,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dee575bc060ce15fb0d HTTP/1.1" 502 150 2025-12-17 22:48:23,425 - lava_callback - ERROR - Error processing callback for node 69408dee575bc060ce15fb0d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dee575bc060ce15fb0d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dee575bc060ce15fb0d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dee575bc060ce15fb0d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:23,427 - lava_callback - INFO - Starting async job submit for node 69408e02575bc060ce15fb58 2025-12-17 22:48:23,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:23,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e02575bc060ce15fb58 HTTP/1.1" 502 150 2025-12-17 22:48:23,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e02575bc060ce15fb58'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:23,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e02575bc060ce15fb58 2025-12-17 22:48:23,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e02575bc060ce15fb58 HTTP/1.1" 502 150 2025-12-17 22:48:23,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e02575bc060ce15fb58'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:23,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e04575bc060ce15fb5b 2025-12-17 22:48:23,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e04575bc060ce15fb5b HTTP/1.1" 502 150 2025-12-17 22:48:23,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e04575bc060ce15fb5b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:23,956 - lava_callback - INFO - Received callback for node 69408c34575bc060ce15f4ac INFO: 172.21.0.1:45278 - "POST /node/69408c34575bc060ce15f4ac HTTP/1.0" 202 Accepted 2025-12-17 22:48:24,355 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e05575bc060ce15fb5e 2025-12-17 22:48:24,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e05575bc060ce15fb5e HTTP/1.1" 502 150 2025-12-17 22:48:24,356 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e05575bc060ce15fb5e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:24,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df3575bc060ce15fb30 2025-12-17 22:48:24,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df3575bc060ce15fb30 HTTP/1.1" 502 150 2025-12-17 22:48:24,714 - lava_callback - ERROR - Error processing callback for node 69408df3575bc060ce15fb30: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df3575bc060ce15fb30 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df3575bc060ce15fb30 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df3575bc060ce15fb30 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:24,716 - lava_callback - INFO - Starting async job submit for node 69408e0c575bc060ce15fb6f 2025-12-17 22:48:24,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:24,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0c575bc060ce15fb6f HTTP/1.1" 502 150 2025-12-17 22:48:24,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e0c575bc060ce15fb6f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:24,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e0c575bc060ce15fb6f 2025-12-17 22:48:24,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0c575bc060ce15fb6f HTTP/1.1" 502 150 2025-12-17 22:48:24,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e0c575bc060ce15fb6f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:24,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df4575bc060ce15fb33 2025-12-17 22:48:24,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e07575bc060ce15fb61 2025-12-17 22:48:24,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df4575bc060ce15fb33 HTTP/1.1" 502 150 2025-12-17 22:48:24,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df4575bc060ce15fb33'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:24,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e07575bc060ce15fb61 HTTP/1.1" 502 150 2025-12-17 22:48:24,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e07575bc060ce15fb61'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:25,077 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dfb575bc060ce15fb4c 2025-12-17 22:48:25,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfb575bc060ce15fb4c HTTP/1.1" 502 150 2025-12-17 22:48:25,078 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dfb575bc060ce15fb4c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:25,260 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dfc575bc060ce15fb4f 2025-12-17 22:48:25,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfc575bc060ce15fb4f HTTP/1.1" 502 150 2025-12-17 22:48:25,261 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dfc575bc060ce15fb4f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:25,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e02575bc060ce15fb58 2025-12-17 22:48:25,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e02575bc060ce15fb58 HTTP/1.1" 502 150 2025-12-17 22:48:25,437 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e02575bc060ce15fb58'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:25,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dfd575bc060ce15fb52 2025-12-17 22:48:25,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfd575bc060ce15fb52 HTTP/1.1" 502 150 2025-12-17 22:48:25,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dfd575bc060ce15fb52'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:25,662 - lava_callback - INFO - Received callback for node 69409026575bc060ce160349 INFO: 172.21.0.1:45286 - "POST /node/69409026575bc060ce160349 HTTP/1.0" 202 Accepted 2025-12-17 22:48:25,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df8575bc060ce15fb46 2025-12-17 22:48:25,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df8575bc060ce15fb46 HTTP/1.1" 502 150 2025-12-17 22:48:25,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df8575bc060ce15fb46'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:26,304 - lava_callback - INFO - Received callback for node 69409028575bc060ce160357 INFO: 172.21.0.1:45296 - "POST /node/69409028575bc060ce160357 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:45310 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:48:26,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e0c575bc060ce15fb6f 2025-12-17 22:48:26,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0c575bc060ce15fb6f HTTP/1.1" 502 150 2025-12-17 22:48:26,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e0c575bc060ce15fb6f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:26,951 - lava_callback - INFO - Received callback for node 6940902a575bc060ce16035a INFO: 172.21.0.1:45314 - "POST /node/6940902a575bc060ce16035a HTTP/1.0" 202 Accepted 2025-12-17 22:48:27,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df5575bc060ce15fb3a HTTP/1.1" 502 150 2025-12-17 22:48:27,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df5575bc060ce15fb3a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:27,612 - lava_callback - INFO - Received callback for node 6940902c575bc060ce16035d INFO: 172.21.0.1:45330 - "POST /node/6940902c575bc060ce16035d HTTP/1.0" 202 Accepted 2025-12-17 22:48:27,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e04575bc060ce15fb5b 2025-12-17 22:48:27,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e04575bc060ce15fb5b HTTP/1.1" 502 150 2025-12-17 22:48:27,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e04575bc060ce15fb5b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:28,259 - lava_callback - INFO - Received callback for node 6940902e575bc060ce160360 INFO: 172.21.0.1:45338 - "POST /node/6940902e575bc060ce160360 HTTP/1.0" 202 Accepted 2025-12-17 22:48:28,356 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e05575bc060ce15fb5e 2025-12-17 22:48:28,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e05575bc060ce15fb5e HTTP/1.1" 502 150 2025-12-17 22:48:28,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e05575bc060ce15fb5e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:28,907 - lava_callback - INFO - Received callback for node 69409030575bc060ce160363 INFO: 172.21.0.1:45342 - "POST /node/69409030575bc060ce160363 HTTP/1.0" 202 Accepted 2025-12-17 22:48:28,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e07575bc060ce15fb61 2025-12-17 22:48:28,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e07575bc060ce15fb61 HTTP/1.1" 502 150 2025-12-17 22:48:28,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e07575bc060ce15fb61'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:29,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e02575bc060ce15fb58 2025-12-17 22:48:29,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e02575bc060ce15fb58 HTTP/1.1" 502 150 2025-12-17 22:48:29,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e02575bc060ce15fb58'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:29,571 - lava_callback - INFO - Received callback for node 69409032575bc060ce160366 INFO: 172.21.0.1:45350 - "POST /node/69409032575bc060ce160366 HTTP/1.0" 202 Accepted 2025-12-17 22:48:30,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408deb575bc060ce15fb07 2025-12-17 22:48:30,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408deb575bc060ce15fb07 HTTP/1.1" 502 150 2025-12-17 22:48:30,010 - lava_callback - ERROR - Error processing callback for node 69408deb575bc060ce15fb07: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408deb575bc060ce15fb07 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408deb575bc060ce15fb07 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408deb575bc060ce15fb07 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:30,011 - lava_callback - INFO - Starting async job submit for node 69408e10575bc060ce15fb75 2025-12-17 22:48:30,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:30,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e10575bc060ce15fb75 HTTP/1.1" 502 150 2025-12-17 22:48:30,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e10575bc060ce15fb75'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:30,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e10575bc060ce15fb75 2025-12-17 22:48:30,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e10575bc060ce15fb75 HTTP/1.1" 502 150 2025-12-17 22:48:30,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e10575bc060ce15fb75'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:30,235 - lava_callback - INFO - Received callback for node 69409034575bc060ce160369 INFO: 172.21.0.1:43866 - "POST /node/69409034575bc060ce160369 HTTP/1.0" 202 Accepted 2025-12-17 22:48:30,442 - lava_callback - INFO - Received callback for node 69409037575bc060ce16036f INFO: 172.21.0.1:43880 - "POST /node/69409037575bc060ce16036f HTTP/1.0" 202 Accepted 2025-12-17 22:48:30,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e0c575bc060ce15fb6f 2025-12-17 22:48:30,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0c575bc060ce15fb6f HTTP/1.1" 502 150 2025-12-17 22:48:30,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e0c575bc060ce15fb6f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:30,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df9575bc060ce15fb49 2025-12-17 22:48:30,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df9575bc060ce15fb49 HTTP/1.1" 502 150 2025-12-17 22:48:30,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df9575bc060ce15fb49'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:31,127 - lava_callback - INFO - Received callback for node 69409038575bc060ce160374 INFO: 172.21.0.1:43894 - "POST /node/69409038575bc060ce160374 HTTP/1.0" 202 Accepted 2025-12-17 22:48:31,817 - lava_callback - INFO - Received callback for node 6940903d575bc060ce16038d INFO: 172.21.0.1:43898 - "POST /node/6940903d575bc060ce16038d HTTP/1.0" 202 Accepted 2025-12-17 22:48:32,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e10575bc060ce15fb75 2025-12-17 22:48:32,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e10575bc060ce15fb75 HTTP/1.1" 502 150 2025-12-17 22:48:32,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e10575bc060ce15fb75'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:33,078 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dfb575bc060ce15fb4c 2025-12-17 22:48:33,262 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dfc575bc060ce15fb4f 2025-12-17 22:48:33,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfc575bc060ce15fb4f HTTP/1.1" 502 150 2025-12-17 22:48:33,269 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dfc575bc060ce15fb4f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:33,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dfd575bc060ce15fb52 2025-12-17 22:48:33,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfd575bc060ce15fb52 HTTP/1.1" 502 150 2025-12-17 22:48:33,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dfd575bc060ce15fb52'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:35,056 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df5575bc060ce15fb3a 2025-12-17 22:48:35,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df5575bc060ce15fb3a HTTP/1.1" 502 150 2025-12-17 22:48:35,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408df5575bc060ce15fb3a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:35,367 - lava_callback - INFO - Received callback for node 6940903e575bc060ce160390 INFO: 172.21.0.1:43912 - "POST /node/6940903e575bc060ce160390 HTTP/1.0" 202 Accepted 2025-12-17 22:48:35,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e04575bc060ce15fb5b 2025-12-17 22:48:35,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e04575bc060ce15fb5b HTTP/1.1" 502 150 2025-12-17 22:48:35,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e04575bc060ce15fb5b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:36,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e10575bc060ce15fb75 2025-12-17 22:48:36,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e10575bc060ce15fb75 HTTP/1.1" 502 150 2025-12-17 22:48:36,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e10575bc060ce15fb75'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:36,358 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e05575bc060ce15fb5e 2025-12-17 22:48:36,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e05575bc060ce15fb5e HTTP/1.1" 502 150 2025-12-17 22:48:36,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e05575bc060ce15fb5e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43928 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:48:36,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df7575bc060ce15fb41 2025-12-17 22:48:36,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df7575bc060ce15fb41 HTTP/1.1" 502 150 2025-12-17 22:48:36,557 - lava_callback - ERROR - Error processing callback for node 69408df7575bc060ce15fb41: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df7575bc060ce15fb41 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df7575bc060ce15fb41 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df7575bc060ce15fb41 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:36,558 - lava_callback - INFO - Starting async job submit for node 69408e0e575bc060ce15fb72 2025-12-17 22:48:36,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:36,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0e575bc060ce15fb72 HTTP/1.1" 502 150 2025-12-17 22:48:36,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e0e575bc060ce15fb72'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:36,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e0e575bc060ce15fb72 2025-12-17 22:48:36,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0e575bc060ce15fb72 HTTP/1.1" 502 150 2025-12-17 22:48:36,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e0e575bc060ce15fb72'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:36,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e07575bc060ce15fb61 2025-12-17 22:48:36,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e07575bc060ce15fb61 HTTP/1.1" 502 150 2025-12-17 22:48:36,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e07575bc060ce15fb61'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:37,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e02575bc060ce15fb58 2025-12-17 22:48:37,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e02575bc060ce15fb58 HTTP/1.1" 502 150 2025-12-17 22:48:37,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e02575bc060ce15fb58'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:37,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408de6575bc060ce15fafe HTTP/1.1" 502 150 2025-12-17 22:48:37,552 - lava_callback - ERROR - Error processing callback for node 69408de6575bc060ce15fafe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de6575bc060ce15fafe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de6575bc060ce15fafe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408de6575bc060ce15fafe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:37,554 - lava_callback - INFO - Starting async job submit for node 69408e08575bc060ce15fb64 2025-12-17 22:48:37,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:37,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e08575bc060ce15fb64 HTTP/1.1" 502 150 2025-12-17 22:48:37,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e08575bc060ce15fb64'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:37,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e08575bc060ce15fb64 2025-12-17 22:48:37,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e08575bc060ce15fb64 HTTP/1.1" 502 150 2025-12-17 22:48:37,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e08575bc060ce15fb64'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:38,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dff575bc060ce15fb55 HTTP/1.1" 502 150 2025-12-17 22:48:38,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dff575bc060ce15fb55'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:38,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dff575bc060ce15fb55 2025-12-17 22:48:38,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dff575bc060ce15fb55 HTTP/1.1" 502 150 2025-12-17 22:48:38,321 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dff575bc060ce15fb55'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:38,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e0e575bc060ce15fb72 2025-12-17 22:48:38,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0e575bc060ce15fb72 HTTP/1.1" 502 150 2025-12-17 22:48:38,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e0e575bc060ce15fb72'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:38,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e0c575bc060ce15fb6f 2025-12-17 22:48:38,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0c575bc060ce15fb6f HTTP/1.1" 502 150 2025-12-17 22:48:38,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e0c575bc060ce15fb6f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:39,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e08575bc060ce15fb64 2025-12-17 22:48:39,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e08575bc060ce15fb64 HTTP/1.1" 502 150 2025-12-17 22:48:39,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e08575bc060ce15fb64'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:40,322 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dff575bc060ce15fb55 2025-12-17 22:48:40,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dff575bc060ce15fb55 HTTP/1.1" 502 150 2025-12-17 22:48:40,323 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dff575bc060ce15fb55'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:40,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df4575bc060ce15fb33 2025-12-17 22:48:40,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df4575bc060ce15fb33 HTTP/1.1" 502 150 2025-12-17 22:48:40,983 - lava_callback - ERROR - Error processing callback for node 69408df4575bc060ce15fb33: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df4575bc060ce15fb33 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df4575bc060ce15fb33 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df4575bc060ce15fb33 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:40,985 - lava_callback - INFO - Starting async job submit for node 69408e09575bc060ce15fb69 2025-12-17 22:48:40,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:40,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e09575bc060ce15fb69 HTTP/1.1" 502 150 2025-12-17 22:48:40,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e09575bc060ce15fb69'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:40,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e09575bc060ce15fb69 2025-12-17 22:48:40,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e09575bc060ce15fb69 HTTP/1.1" 502 150 2025-12-17 22:48:40,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e09575bc060ce15fb69'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:41,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df8575bc060ce15fb46 2025-12-17 22:48:41,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df8575bc060ce15fb46 HTTP/1.1" 502 150 2025-12-17 22:48:41,918 - lava_callback - ERROR - Error processing callback for node 69408df8575bc060ce15fb46: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df8575bc060ce15fb46 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df8575bc060ce15fb46 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df8575bc060ce15fb46 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:41,920 - lava_callback - INFO - Starting async job submit for node 69408e0b575bc060ce15fb6c 2025-12-17 22:48:41,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:41,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0b575bc060ce15fb6c HTTP/1.1" 502 150 2025-12-17 22:48:41,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e0b575bc060ce15fb6c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:41,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e0b575bc060ce15fb6c 2025-12-17 22:48:41,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0b575bc060ce15fb6c HTTP/1.1" 502 150 2025-12-17 22:48:41,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e0b575bc060ce15fb6c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:42,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e0e575bc060ce15fb72 2025-12-17 22:48:42,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0e575bc060ce15fb72 HTTP/1.1" 502 150 2025-12-17 22:48:42,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e0e575bc060ce15fb72'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:42,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e09575bc060ce15fb69 2025-12-17 22:48:42,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e09575bc060ce15fb69 HTTP/1.1" 502 150 2025-12-17 22:48:42,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e09575bc060ce15fb69'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:43,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e08575bc060ce15fb64 2025-12-17 22:48:43,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e08575bc060ce15fb64 HTTP/1.1" 502 150 2025-12-17 22:48:43,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e08575bc060ce15fb64'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:43,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e0b575bc060ce15fb6c 2025-12-17 22:48:43,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0b575bc060ce15fb6c HTTP/1.1" 502 150 2025-12-17 22:48:43,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e0b575bc060ce15fb6c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:44,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e10575bc060ce15fb75 2025-12-17 22:48:44,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e10575bc060ce15fb75 HTTP/1.1" 502 150 2025-12-17 22:48:44,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e10575bc060ce15fb75'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:44,323 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dff575bc060ce15fb55 2025-12-17 22:48:44,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dff575bc060ce15fb55 HTTP/1.1" 502 150 2025-12-17 22:48:44,324 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dff575bc060ce15fb55'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:45,681 - lava_callback - INFO - Received callback for node 6940903a575bc060ce16037f INFO: 172.21.0.1:32978 - "POST /node/6940903a575bc060ce16037f HTTP/1.0" 202 Accepted 2025-12-17 22:48:46,346 - lava_callback - INFO - Received callback for node 6940903b575bc060ce160386 INFO: 172.21.0.1:32986 - "POST /node/6940903b575bc060ce160386 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:32988 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:48:46,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df9575bc060ce15fb49 2025-12-17 22:48:46,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df9575bc060ce15fb49 HTTP/1.1" 502 150 2025-12-17 22:48:46,753 - lava_callback - ERROR - Error processing callback for node 69408df9575bc060ce15fb49: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df9575bc060ce15fb49 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df9575bc060ce15fb49 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df9575bc060ce15fb49 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:46,755 - lava_callback - INFO - Starting async job submit for node 69408e12575bc060ce15fb78 2025-12-17 22:48:46,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:46,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e12575bc060ce15fb78 HTTP/1.1" 502 150 2025-12-17 22:48:46,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e12575bc060ce15fb78'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:46,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e12575bc060ce15fb78 2025-12-17 22:48:46,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e12575bc060ce15fb78 HTTP/1.1" 502 150 2025-12-17 22:48:46,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e12575bc060ce15fb78'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:46,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e09575bc060ce15fb69 2025-12-17 22:48:46,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e09575bc060ce15fb69 HTTP/1.1" 502 150 2025-12-17 22:48:46,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e09575bc060ce15fb69'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:47,010 - lava_callback - INFO - Received callback for node 69409041575bc060ce160393 INFO: 172.21.0.1:33004 - "POST /node/69409041575bc060ce160393 HTTP/1.0" 202 Accepted 2025-12-17 22:48:47,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e0b575bc060ce15fb6c 2025-12-17 22:48:47,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0b575bc060ce15fb6c HTTP/1.1" 502 150 2025-12-17 22:48:47,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e0b575bc060ce15fb6c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:48,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfb575bc060ce15fb4c HTTP/1.1" 502 150 2025-12-17 22:48:48,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dfb575bc060ce15fb4c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:48,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e12575bc060ce15fb78 2025-12-17 22:48:48,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e12575bc060ce15fb78 HTTP/1.1" 502 150 2025-12-17 22:48:48,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e12575bc060ce15fb78'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:49,270 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dfc575bc060ce15fb4f 2025-12-17 22:48:49,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfc575bc060ce15fb4f HTTP/1.1" 502 150 2025-12-17 22:48:49,271 - lava_callback - ERROR - Error processing callback for node 69408dfc575bc060ce15fb4f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dfc575bc060ce15fb4f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dfc575bc060ce15fb4f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dfc575bc060ce15fb4f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:49,273 - lava_callback - INFO - Starting async job submit for node 69408e14575bc060ce15fb7b 2025-12-17 22:48:49,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:49,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e14575bc060ce15fb7b HTTP/1.1" 502 150 2025-12-17 22:48:49,292 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e14575bc060ce15fb7b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:49,293 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e14575bc060ce15fb7b 2025-12-17 22:48:49,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e14575bc060ce15fb7b HTTP/1.1" 502 150 2025-12-17 22:48:49,294 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e14575bc060ce15fb7b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:49,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dfd575bc060ce15fb52 2025-12-17 22:48:49,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfd575bc060ce15fb52 HTTP/1.1" 502 150 2025-12-17 22:48:49,540 - lava_callback - ERROR - Error processing callback for node 69408dfd575bc060ce15fb52: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dfd575bc060ce15fb52 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dfd575bc060ce15fb52 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dfd575bc060ce15fb52 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:49,543 - lava_callback - INFO - Starting async job submit for node 69408e16575bc060ce15fb7e 2025-12-17 22:48:49,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:49,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e16575bc060ce15fb7e HTTP/1.1" 502 150 2025-12-17 22:48:49,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e16575bc060ce15fb7e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:49,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e16575bc060ce15fb7e 2025-12-17 22:48:49,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e16575bc060ce15fb7e HTTP/1.1" 502 150 2025-12-17 22:48:49,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e16575bc060ce15fb7e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:50,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e0e575bc060ce15fb72 2025-12-17 22:48:50,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0e575bc060ce15fb72 HTTP/1.1" 502 150 2025-12-17 22:48:50,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e0e575bc060ce15fb72'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:51,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408df5575bc060ce15fb3a 2025-12-17 22:48:51,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408df5575bc060ce15fb3a HTTP/1.1" 502 150 2025-12-17 22:48:51,063 - lava_callback - ERROR - Error processing callback for node 69408df5575bc060ce15fb3a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df5575bc060ce15fb3a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df5575bc060ce15fb3a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408df5575bc060ce15fb3a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:51,066 - lava_callback - INFO - Starting async job submit for node 69408e18575bc060ce15fb81 2025-12-17 22:48:51,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:51,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e18575bc060ce15fb81 HTTP/1.1" 502 150 2025-12-17 22:48:51,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e18575bc060ce15fb81'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:51,096 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e18575bc060ce15fb81 2025-12-17 22:48:51,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e18575bc060ce15fb81 HTTP/1.1" 502 150 2025-12-17 22:48:51,097 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e18575bc060ce15fb81'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:51,294 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e14575bc060ce15fb7b 2025-12-17 22:48:51,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e14575bc060ce15fb7b HTTP/1.1" 502 150 2025-12-17 22:48:51,298 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e14575bc060ce15fb7b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:51,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e16575bc060ce15fb7e 2025-12-17 22:48:51,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e16575bc060ce15fb7e HTTP/1.1" 502 150 2025-12-17 22:48:51,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e16575bc060ce15fb7e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:51,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e08575bc060ce15fb64 2025-12-17 22:48:51,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e08575bc060ce15fb64 HTTP/1.1" 502 150 2025-12-17 22:48:51,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e08575bc060ce15fb64'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:51,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e04575bc060ce15fb5b 2025-12-17 22:48:51,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e04575bc060ce15fb5b HTTP/1.1" 502 150 2025-12-17 22:48:51,837 - lava_callback - ERROR - Error processing callback for node 69408e04575bc060ce15fb5b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e04575bc060ce15fb5b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e04575bc060ce15fb5b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e04575bc060ce15fb5b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:51,840 - lava_callback - INFO - Starting async job submit for node 69408e1a575bc060ce15fb84 2025-12-17 22:48:51,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:51,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1a575bc060ce15fb84 HTTP/1.1" 502 150 2025-12-17 22:48:51,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1a575bc060ce15fb84'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:51,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1a575bc060ce15fb84 2025-12-17 22:48:51,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1a575bc060ce15fb84 HTTP/1.1" 502 150 2025-12-17 22:48:51,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1a575bc060ce15fb84'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:52,325 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dff575bc060ce15fb55 2025-12-17 22:48:52,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dff575bc060ce15fb55 HTTP/1.1" 502 150 2025-12-17 22:48:52,326 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408dff575bc060ce15fb55'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:52,359 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e05575bc060ce15fb5e 2025-12-17 22:48:52,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e05575bc060ce15fb5e HTTP/1.1" 502 150 2025-12-17 22:48:52,361 - lava_callback - ERROR - Error processing callback for node 69408e05575bc060ce15fb5e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e05575bc060ce15fb5e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e05575bc060ce15fb5e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e05575bc060ce15fb5e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:52,364 - lava_callback - INFO - Starting async job submit for node 69408e1b575bc060ce15fb87 2025-12-17 22:48:52,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:52,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1b575bc060ce15fb87 HTTP/1.1" 502 150 2025-12-17 22:48:52,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1b575bc060ce15fb87'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:52,381 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1b575bc060ce15fb87 2025-12-17 22:48:52,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1b575bc060ce15fb87 HTTP/1.1" 502 150 2025-12-17 22:48:52,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1b575bc060ce15fb87'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:52,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e12575bc060ce15fb78 2025-12-17 22:48:52,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e12575bc060ce15fb78 HTTP/1.1" 502 150 2025-12-17 22:48:52,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e12575bc060ce15fb78'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:52,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e07575bc060ce15fb61 2025-12-17 22:48:52,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e07575bc060ce15fb61 HTTP/1.1" 502 150 2025-12-17 22:48:52,985 - lava_callback - ERROR - Error processing callback for node 69408e07575bc060ce15fb61: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e07575bc060ce15fb61 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e07575bc060ce15fb61 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e07575bc060ce15fb61 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:52,987 - lava_callback - INFO - Starting async job submit for node 69408e1d575bc060ce15fb8d 2025-12-17 22:48:52,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:52,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1d575bc060ce15fb8d HTTP/1.1" 502 150 2025-12-17 22:48:52,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1d575bc060ce15fb8d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:52,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1d575bc060ce15fb8d 2025-12-17 22:48:52,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1d575bc060ce15fb8d HTTP/1.1" 502 150 2025-12-17 22:48:52,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1d575bc060ce15fb8d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:53,097 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e18575bc060ce15fb81 2025-12-17 22:48:53,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e18575bc060ce15fb81 HTTP/1.1" 502 150 2025-12-17 22:48:53,099 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e18575bc060ce15fb81'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:53,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e02575bc060ce15fb58 2025-12-17 22:48:53,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e02575bc060ce15fb58 HTTP/1.1" 502 150 2025-12-17 22:48:53,442 - lava_callback - ERROR - Error processing callback for node 69408e02575bc060ce15fb58: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e02575bc060ce15fb58 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e02575bc060ce15fb58 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e02575bc060ce15fb58 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:53,445 - lava_callback - INFO - Starting async job submit for node 69408e20575bc060ce15fb93 2025-12-17 22:48:53,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:53,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e20575bc060ce15fb93 HTTP/1.1" 502 150 2025-12-17 22:48:53,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e20575bc060ce15fb93'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:53,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e20575bc060ce15fb93 2025-12-17 22:48:53,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e20575bc060ce15fb93 HTTP/1.1" 502 150 2025-12-17 22:48:53,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e20575bc060ce15fb93'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:53,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1a575bc060ce15fb84 2025-12-17 22:48:53,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1a575bc060ce15fb84 HTTP/1.1" 502 150 2025-12-17 22:48:53,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1a575bc060ce15fb84'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:54,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1b575bc060ce15fb87 2025-12-17 22:48:54,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1b575bc060ce15fb87 HTTP/1.1" 502 150 2025-12-17 22:48:54,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1b575bc060ce15fb87'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:54,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e0c575bc060ce15fb6f 2025-12-17 22:48:54,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0c575bc060ce15fb6f HTTP/1.1" 502 150 2025-12-17 22:48:54,730 - lava_callback - ERROR - Error processing callback for node 69408e0c575bc060ce15fb6f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e0c575bc060ce15fb6f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e0c575bc060ce15fb6f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e0c575bc060ce15fb6f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:48:54,731 - lava_callback - INFO - Starting async job submit for node 69408e22575bc060ce15fb9a 2025-12-17 22:48:54,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:48:54,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e22575bc060ce15fb9a HTTP/1.1" 502 150 2025-12-17 22:48:54,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e22575bc060ce15fb9a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:54,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e22575bc060ce15fb9a 2025-12-17 22:48:54,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e22575bc060ce15fb9a HTTP/1.1" 502 150 2025-12-17 22:48:54,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e22575bc060ce15fb9a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:54,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e09575bc060ce15fb69 2025-12-17 22:48:54,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1d575bc060ce15fb8d 2025-12-17 22:48:54,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e09575bc060ce15fb69 HTTP/1.1" 502 150 2025-12-17 22:48:54,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e09575bc060ce15fb69'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:54,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1d575bc060ce15fb8d HTTP/1.1" 502 150 2025-12-17 22:48:54,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1d575bc060ce15fb8d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:55,298 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e14575bc060ce15fb7b 2025-12-17 22:48:55,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e14575bc060ce15fb7b HTTP/1.1" 502 150 2025-12-17 22:48:55,299 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e14575bc060ce15fb7b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:55,353 - lava_callback - INFO - Received callback for node 69409047575bc060ce160396 INFO: 172.21.0.1:46770 - "POST /node/69409047575bc060ce160396 HTTP/1.0" 202 Accepted 2025-12-17 22:48:55,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e20575bc060ce15fb93 2025-12-17 22:48:55,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e20575bc060ce15fb93 HTTP/1.1" 502 150 2025-12-17 22:48:55,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e20575bc060ce15fb93'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:55,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e16575bc060ce15fb7e 2025-12-17 22:48:55,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e16575bc060ce15fb7e HTTP/1.1" 502 150 2025-12-17 22:48:55,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e16575bc060ce15fb7e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:55,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e0b575bc060ce15fb6c 2025-12-17 22:48:55,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0b575bc060ce15fb6c HTTP/1.1" 502 150 2025-12-17 22:48:55,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e0b575bc060ce15fb6c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:56,029 - lava_callback - INFO - Received callback for node 69409048575bc060ce160399 INFO: 172.21.0.1:46784 - "POST /node/69409048575bc060ce160399 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:46788 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:48:56,691 - lava_callback - INFO - Received callback for node 69409049575bc060ce16039c INFO: 172.21.0.1:46792 - "POST /node/69409049575bc060ce16039c HTTP/1.0" 202 Accepted 2025-12-17 22:48:56,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e22575bc060ce15fb9a 2025-12-17 22:48:56,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e22575bc060ce15fb9a HTTP/1.1" 502 150 2025-12-17 22:48:56,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e22575bc060ce15fb9a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:57,099 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e18575bc060ce15fb81 2025-12-17 22:48:57,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e18575bc060ce15fb81 HTTP/1.1" 502 150 2025-12-17 22:48:57,100 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e18575bc060ce15fb81'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:57,348 - lava_callback - INFO - Received callback for node 6940904b575bc060ce16039f INFO: 172.21.0.1:46798 - "POST /node/6940904b575bc060ce16039f HTTP/1.0" 202 Accepted 2025-12-17 22:48:57,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1a575bc060ce15fb84 2025-12-17 22:48:57,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1a575bc060ce15fb84 HTTP/1.1" 502 150 2025-12-17 22:48:57,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1a575bc060ce15fb84'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:58,014 - lava_callback - INFO - Received callback for node 6940904c575bc060ce1603a4 INFO: 172.21.0.1:46812 - "POST /node/6940904c575bc060ce1603a4 HTTP/1.0" 202 Accepted 2025-12-17 22:48:58,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1b575bc060ce15fb87 2025-12-17 22:48:58,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1b575bc060ce15fb87 HTTP/1.1" 502 150 2025-12-17 22:48:58,385 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1b575bc060ce15fb87'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:58,663 - lava_callback - INFO - Received callback for node 69409050575bc060ce1603ce INFO: 172.21.0.1:46826 - "POST /node/69409050575bc060ce1603ce HTTP/1.0" 202 Accepted 2025-12-17 22:48:58,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1d575bc060ce15fb8d 2025-12-17 22:48:59,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1d575bc060ce15fb8d HTTP/1.1" 502 150 2025-12-17 22:48:59,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1d575bc060ce15fb8d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:48:59,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e20575bc060ce15fb93 2025-12-17 22:49:00,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e10575bc060ce15fb75 2025-12-17 22:49:00,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e22575bc060ce15fb9a 2025-12-17 22:49:00,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e22575bc060ce15fb9a HTTP/1.1" 502 150 2025-12-17 22:49:00,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e22575bc060ce15fb9a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:00,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e12575bc060ce15fb78 2025-12-17 22:49:00,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e12575bc060ce15fb78 HTTP/1.1" 502 150 2025-12-17 22:49:00,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e12575bc060ce15fb78'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43264 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:49:03,299 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e14575bc060ce15fb7b 2025-12-17 22:49:03,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e14575bc060ce15fb7b HTTP/1.1" 502 150 2025-12-17 22:49:03,301 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e14575bc060ce15fb7b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:03,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e16575bc060ce15fb7e 2025-12-17 22:49:03,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e16575bc060ce15fb7e HTTP/1.1" 502 150 2025-12-17 22:49:03,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e16575bc060ce15fb7e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:04,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dfb575bc060ce15fb4c 2025-12-17 22:49:04,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dfb575bc060ce15fb4c HTTP/1.1" 502 150 2025-12-17 22:49:04,562 - lava_callback - ERROR - Error processing callback for node 69408dfb575bc060ce15fb4c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dfb575bc060ce15fb4c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dfb575bc060ce15fb4c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dfb575bc060ce15fb4c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:04,564 - lava_callback - INFO - Starting async job submit for node 69408e23575bc060ce15fb9d 2025-12-17 22:49:04,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:04,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e23575bc060ce15fb9d HTTP/1.1" 502 150 2025-12-17 22:49:04,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e23575bc060ce15fb9d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:04,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e23575bc060ce15fb9d 2025-12-17 22:49:04,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e23575bc060ce15fb9d HTTP/1.1" 502 150 2025-12-17 22:49:04,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e23575bc060ce15fb9d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:05,100 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e18575bc060ce15fb81 2025-12-17 22:49:05,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e18575bc060ce15fb81 HTTP/1.1" 502 150 2025-12-17 22:49:05,102 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e18575bc060ce15fb81'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:05,688 - lava_callback - INFO - Received callback for node 6940904d575bc060ce1603ac INFO: 172.21.0.1:43270 - "POST /node/6940904d575bc060ce1603ac HTTP/1.0" 202 Accepted 2025-12-17 22:49:05,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1a575bc060ce15fb84 2025-12-17 22:49:05,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1a575bc060ce15fb84 HTTP/1.1" 502 150 2025-12-17 22:49:05,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1a575bc060ce15fb84'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:06,311 - lava_callback - INFO - Received callback for node 6940905e575bc060ce160556 INFO: 172.21.0.1:43286 - "POST /node/6940905e575bc060ce160556 HTTP/1.0" 202 Accepted 2025-12-17 22:49:06,353 - lava_callback - INFO - Received callback for node 6940904f575bc060ce1603cb INFO: 172.21.0.1:43300 - "POST /node/6940904f575bc060ce1603cb HTTP/1.0" 202 Accepted 2025-12-17 22:49:06,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1b575bc060ce15fb87 2025-12-17 22:49:06,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1b575bc060ce15fb87 HTTP/1.1" 502 150 2025-12-17 22:49:06,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1b575bc060ce15fb87'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43304 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:49:06,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e0e575bc060ce15fb72 2025-12-17 22:49:06,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0e575bc060ce15fb72 HTTP/1.1" 502 150 2025-12-17 22:49:06,573 - lava_callback - ERROR - Error processing callback for node 69408e0e575bc060ce15fb72: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e0e575bc060ce15fb72 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e0e575bc060ce15fb72 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e0e575bc060ce15fb72 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:06,575 - lava_callback - INFO - Starting async job submit for node 69408e24575bc060ce15fba0 2025-12-17 22:49:06,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e23575bc060ce15fb9d 2025-12-17 22:49:06,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:06,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e23575bc060ce15fb9d HTTP/1.1" 502 150 2025-12-17 22:49:06,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e23575bc060ce15fb9d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:06,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e24575bc060ce15fba0 HTTP/1.1" 502 150 2025-12-17 22:49:06,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e24575bc060ce15fba0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:06,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e24575bc060ce15fba0 2025-12-17 22:49:06,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e24575bc060ce15fba0 HTTP/1.1" 502 150 2025-12-17 22:49:06,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e24575bc060ce15fba0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:07,000 - lava_callback - INFO - Received callback for node 69409051575bc060ce1603d1 2025-12-17 22:49:07,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1d575bc060ce15fb8d INFO: 172.21.0.1:43318 - "POST /node/69409051575bc060ce1603d1 HTTP/1.0" 202 Accepted 2025-12-17 22:49:07,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1d575bc060ce15fb8d HTTP/1.1" 502 150 2025-12-17 22:49:07,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1d575bc060ce15fb8d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:07,104 - lava_callback - INFO - Received callback for node 69409061575bc060ce160560 INFO: 172.21.0.1:43326 - "POST /node/69409061575bc060ce160560 HTTP/1.0" 202 Accepted 2025-12-17 22:49:07,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e08575bc060ce15fb64 2025-12-17 22:49:07,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e08575bc060ce15fb64 HTTP/1.1" 502 150 2025-12-17 22:49:07,591 - lava_callback - ERROR - Error processing callback for node 69408e08575bc060ce15fb64: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e08575bc060ce15fb64 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e08575bc060ce15fb64 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e08575bc060ce15fb64 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:07,592 - lava_callback - INFO - Starting async job submit for node 69408e1f575bc060ce15fb90 2025-12-17 22:49:07,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:07,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1f575bc060ce15fb90 HTTP/1.1" 502 150 2025-12-17 22:49:07,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1f575bc060ce15fb90'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:07,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1f575bc060ce15fb90 2025-12-17 22:49:07,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1f575bc060ce15fb90 HTTP/1.1" 502 150 2025-12-17 22:49:07,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1f575bc060ce15fb90'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:07,644 - lava_callback - INFO - Received callback for node 69409053575bc060ce1603d4 INFO: 172.21.0.1:43336 - "POST /node/69409053575bc060ce1603d4 HTTP/1.0" 202 Accepted 2025-12-17 22:49:08,287 - lava_callback - INFO - Received callback for node 69409054575bc060ce1603d7 INFO: 172.21.0.1:43352 - "POST /node/69409054575bc060ce1603d7 HTTP/1.0" 202 Accepted 2025-12-17 22:49:08,326 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408dff575bc060ce15fb55 2025-12-17 22:49:08,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408dff575bc060ce15fb55 HTTP/1.1" 502 150 2025-12-17 22:49:08,328 - lava_callback - ERROR - Error processing callback for node 69408dff575bc060ce15fb55: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dff575bc060ce15fb55 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dff575bc060ce15fb55 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408dff575bc060ce15fb55 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:08,329 - lava_callback - INFO - Starting async job submit for node 69408c36575bc060ce15f4af 2025-12-17 22:49:08,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e24575bc060ce15fba0 2025-12-17 22:49:08,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e24575bc060ce15fba0 HTTP/1.1" 502 150 2025-12-17 22:49:08,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e24575bc060ce15fba0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:08,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:08,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c36575bc060ce15f4af HTTP/1.1" 502 150 2025-12-17 22:49:08,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c36575bc060ce15f4af'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:08,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c36575bc060ce15f4af 2025-12-17 22:49:08,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c36575bc060ce15f4af HTTP/1.1" 502 150 2025-12-17 22:49:08,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c36575bc060ce15f4af'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:08,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e22575bc060ce15fb9a 2025-12-17 22:49:08,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e22575bc060ce15fb9a HTTP/1.1" 502 150 2025-12-17 22:49:08,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e22575bc060ce15fb9a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:08,956 - lava_callback - INFO - Received callback for node 69409055575bc060ce1603e0 INFO: 172.21.0.1:43358 - "POST /node/69409055575bc060ce1603e0 HTTP/1.0" 202 Accepted 2025-12-17 22:49:09,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1f575bc060ce15fb90 2025-12-17 22:49:09,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1f575bc060ce15fb90 HTTP/1.1" 502 150 2025-12-17 22:49:09,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1f575bc060ce15fb90'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:09,620 - lava_callback - INFO - Received callback for node 69409057575bc060ce1603e8 INFO: 172.21.0.1:43368 - "POST /node/69409057575bc060ce1603e8 HTTP/1.0" 202 Accepted 2025-12-17 22:49:10,272 - lava_callback - INFO - Received callback for node 69409058575bc060ce1603eb INFO: 172.21.0.1:42640 - "POST /node/69409058575bc060ce1603eb HTTP/1.0" 202 Accepted 2025-12-17 22:49:10,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e23575bc060ce15fb9d 2025-12-17 22:49:10,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c36575bc060ce15f4af 2025-12-17 22:49:10,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c36575bc060ce15f4af HTTP/1.1" 502 150 2025-12-17 22:49:10,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c36575bc060ce15f4af'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:10,921 - lava_callback - INFO - Received callback for node 69409059575bc060ce1603ee INFO: 172.21.0.1:42656 - "POST /node/69409059575bc060ce1603ee HTTP/1.0" 202 Accepted 2025-12-17 22:49:10,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e09575bc060ce15fb69 2025-12-17 22:49:10,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e09575bc060ce15fb69 HTTP/1.1" 502 150 2025-12-17 22:49:10,999 - lava_callback - ERROR - Error processing callback for node 69408e09575bc060ce15fb69: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e09575bc060ce15fb69 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e09575bc060ce15fb69 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e09575bc060ce15fb69 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:11,002 - lava_callback - INFO - Starting async job submit for node 69408e25575bc060ce15fba3 2025-12-17 22:49:11,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:11,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e25575bc060ce15fba3 HTTP/1.1" 502 150 2025-12-17 22:49:11,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e25575bc060ce15fba3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:11,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e25575bc060ce15fba3 2025-12-17 22:49:11,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e25575bc060ce15fba3 HTTP/1.1" 502 150 2025-12-17 22:49:11,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e25575bc060ce15fba3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:11,570 - lava_callback - INFO - Received callback for node 6940905b575bc060ce160550 INFO: 172.21.0.1:42660 - "POST /node/6940905b575bc060ce160550 HTTP/1.0" 202 Accepted 2025-12-17 22:49:11,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e0b575bc060ce15fb6c 2025-12-17 22:49:11,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e0b575bc060ce15fb6c HTTP/1.1" 502 150 2025-12-17 22:49:11,957 - lava_callback - ERROR - Error processing callback for node 69408e0b575bc060ce15fb6c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e0b575bc060ce15fb6c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e0b575bc060ce15fb6c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e0b575bc060ce15fb6c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:11,959 - lava_callback - INFO - Starting async job submit for node 69408e27575bc060ce15fba6 2025-12-17 22:49:11,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:11,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e27575bc060ce15fba6 HTTP/1.1" 502 150 2025-12-17 22:49:11,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e27575bc060ce15fba6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:11,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e27575bc060ce15fba6 2025-12-17 22:49:11,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e27575bc060ce15fba6 HTTP/1.1" 502 150 2025-12-17 22:49:11,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e27575bc060ce15fba6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:12,227 - lava_callback - INFO - Received callback for node 6940905d575bc060ce160553 INFO: 172.21.0.1:42676 - "POST /node/6940905d575bc060ce160553 HTTP/1.0" 202 Accepted 2025-12-17 22:49:12,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e24575bc060ce15fba0 2025-12-17 22:49:12,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e24575bc060ce15fba0 HTTP/1.1" 502 150 2025-12-17 22:49:12,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e24575bc060ce15fba0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:12,908 - lava_callback - INFO - Received callback for node 6940905f575bc060ce160559 INFO: 172.21.0.1:42680 - "POST /node/6940905f575bc060ce160559 HTTP/1.0" 202 Accepted 2025-12-17 22:49:13,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e25575bc060ce15fba3 2025-12-17 22:49:13,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e25575bc060ce15fba3 HTTP/1.1" 502 150 2025-12-17 22:49:13,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e25575bc060ce15fba3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:13,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1f575bc060ce15fb90 2025-12-17 22:49:13,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1f575bc060ce15fb90 HTTP/1.1" 502 150 2025-12-17 22:49:13,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1f575bc060ce15fb90'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:13,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e27575bc060ce15fba6 2025-12-17 22:49:13,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e27575bc060ce15fba6 HTTP/1.1" 502 150 2025-12-17 22:49:13,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e27575bc060ce15fba6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:14,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c36575bc060ce15f4af 2025-12-17 22:49:14,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c36575bc060ce15f4af HTTP/1.1" 502 150 2025-12-17 22:49:14,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c36575bc060ce15f4af'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:14,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e20575bc060ce15fb93 HTTP/1.1" 502 150 2025-12-17 22:49:14,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e20575bc060ce15fb93'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42696 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:49:15,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e10575bc060ce15fb75 HTTP/1.1" 502 150 2025-12-17 22:49:15,440 - lava_callback - ERROR - Error processing callback for node 69408e10575bc060ce15fb75: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e10575bc060ce15fb75 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e10575bc060ce15fb75 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e10575bc060ce15fb75 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:15,442 - lava_callback - INFO - Starting async job submit for node 69408e2d575bc060ce15fd32 2025-12-17 22:49:15,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:15,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e2d575bc060ce15fd32 HTTP/1.1" 502 150 2025-12-17 22:49:15,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e2d575bc060ce15fd32'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:15,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e2d575bc060ce15fd32 2025-12-17 22:49:15,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e2d575bc060ce15fd32 HTTP/1.1" 502 150 2025-12-17 22:49:15,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e2d575bc060ce15fd32'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42712 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:49:16,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e12575bc060ce15fb78 2025-12-17 22:49:16,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e12575bc060ce15fb78 HTTP/1.1" 502 150 2025-12-17 22:49:16,791 - lava_callback - ERROR - Error processing callback for node 69408e12575bc060ce15fb78: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e12575bc060ce15fb78 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e12575bc060ce15fb78 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e12575bc060ce15fb78 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:16,793 - lava_callback - INFO - Starting async job submit for node 69408e2f575bc060ce15fd3a 2025-12-17 22:49:16,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:16,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e2f575bc060ce15fd3a HTTP/1.1" 502 150 2025-12-17 22:49:16,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e2f575bc060ce15fd3a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:16,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e2f575bc060ce15fd3a 2025-12-17 22:49:16,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e2f575bc060ce15fd3a HTTP/1.1" 502 150 2025-12-17 22:49:16,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e2f575bc060ce15fd3a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:17,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e25575bc060ce15fba3 2025-12-17 22:49:17,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e25575bc060ce15fba3 HTTP/1.1" 502 150 2025-12-17 22:49:17,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e25575bc060ce15fba3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:17,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e2d575bc060ce15fd32 2025-12-17 22:49:17,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e2d575bc060ce15fd32 HTTP/1.1" 502 150 2025-12-17 22:49:17,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e2d575bc060ce15fd32'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:17,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e27575bc060ce15fba6 2025-12-17 22:49:17,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e27575bc060ce15fba6 HTTP/1.1" 502 150 2025-12-17 22:49:17,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e27575bc060ce15fba6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:18,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e2f575bc060ce15fd3a 2025-12-17 22:49:18,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e2f575bc060ce15fd3a HTTP/1.1" 502 150 2025-12-17 22:49:18,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e2f575bc060ce15fd3a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:19,301 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e14575bc060ce15fb7b 2025-12-17 22:49:19,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e14575bc060ce15fb7b HTTP/1.1" 502 150 2025-12-17 22:49:19,303 - lava_callback - ERROR - Error processing callback for node 69408e14575bc060ce15fb7b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e14575bc060ce15fb7b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e14575bc060ce15fb7b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e14575bc060ce15fb7b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:19,305 - lava_callback - INFO - Starting async job submit for node 69408e95575bc060ce15fe85 2025-12-17 22:49:19,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:19,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e95575bc060ce15fe85 HTTP/1.1" 502 150 2025-12-17 22:49:19,338 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e95575bc060ce15fe85'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:19,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e95575bc060ce15fe85 2025-12-17 22:49:19,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e95575bc060ce15fe85 HTTP/1.1" 502 150 2025-12-17 22:49:19,339 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e95575bc060ce15fe85'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:19,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e16575bc060ce15fb7e 2025-12-17 22:49:19,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e16575bc060ce15fb7e HTTP/1.1" 502 150 2025-12-17 22:49:19,571 - lava_callback - ERROR - Error processing callback for node 69408e16575bc060ce15fb7e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e16575bc060ce15fb7e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e16575bc060ce15fb7e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e16575bc060ce15fb7e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:19,572 - lava_callback - INFO - Starting async job submit for node 69408e34575bc060ce15fd54 2025-12-17 22:49:19,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:19,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e34575bc060ce15fd54 HTTP/1.1" 502 150 2025-12-17 22:49:19,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e34575bc060ce15fd54'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:19,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e34575bc060ce15fd54 2025-12-17 22:49:19,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e34575bc060ce15fd54 HTTP/1.1" 502 150 2025-12-17 22:49:19,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e34575bc060ce15fd54'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:20,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e24575bc060ce15fba0 2025-12-17 22:49:20,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e24575bc060ce15fba0 HTTP/1.1" 502 150 2025-12-17 22:49:20,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e24575bc060ce15fba0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:21,102 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e18575bc060ce15fb81 2025-12-17 22:49:21,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e18575bc060ce15fb81 HTTP/1.1" 502 150 2025-12-17 22:49:21,103 - lava_callback - ERROR - Error processing callback for node 69408e18575bc060ce15fb81: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e18575bc060ce15fb81 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e18575bc060ce15fb81 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e18575bc060ce15fb81 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:21,106 - lava_callback - INFO - Starting async job submit for node 69408e37575bc060ce15fd5f 2025-12-17 22:49:21,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:21,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e37575bc060ce15fd5f HTTP/1.1" 502 150 2025-12-17 22:49:21,118 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e37575bc060ce15fd5f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:21,118 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e37575bc060ce15fd5f 2025-12-17 22:49:21,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e37575bc060ce15fd5f HTTP/1.1" 502 150 2025-12-17 22:49:21,119 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e37575bc060ce15fd5f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:21,339 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e95575bc060ce15fe85 2025-12-17 22:49:21,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e95575bc060ce15fe85 HTTP/1.1" 502 150 2025-12-17 22:49:21,340 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e95575bc060ce15fe85'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:21,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e2d575bc060ce15fd32 2025-12-17 22:49:21,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e2d575bc060ce15fd32 HTTP/1.1" 502 150 2025-12-17 22:49:21,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e2d575bc060ce15fd32'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:21,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e34575bc060ce15fd54 2025-12-17 22:49:21,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e34575bc060ce15fd54 HTTP/1.1" 502 150 2025-12-17 22:49:21,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e34575bc060ce15fd54'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:21,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1f575bc060ce15fb90 2025-12-17 22:49:21,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1f575bc060ce15fb90 HTTP/1.1" 502 150 2025-12-17 22:49:21,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e1f575bc060ce15fb90'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:21,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1a575bc060ce15fb84 2025-12-17 22:49:21,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1a575bc060ce15fb84 HTTP/1.1" 502 150 2025-12-17 22:49:21,877 - lava_callback - ERROR - Error processing callback for node 69408e1a575bc060ce15fb84: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e1a575bc060ce15fb84 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e1a575bc060ce15fb84 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e1a575bc060ce15fb84 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:21,879 - lava_callback - INFO - Starting async job submit for node 69408e38575bc060ce15fd62 2025-12-17 22:49:21,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:21,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e38575bc060ce15fd62 HTTP/1.1" 502 150 2025-12-17 22:49:21,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e38575bc060ce15fd62'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:21,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e38575bc060ce15fd62 2025-12-17 22:49:21,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e38575bc060ce15fd62 HTTP/1.1" 502 150 2025-12-17 22:49:21,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e38575bc060ce15fd62'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:22,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1b575bc060ce15fb87 2025-12-17 22:49:22,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1b575bc060ce15fb87 HTTP/1.1" 502 150 2025-12-17 22:49:22,388 - lava_callback - ERROR - Error processing callback for node 69408e1b575bc060ce15fb87: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e1b575bc060ce15fb87 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e1b575bc060ce15fb87 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e1b575bc060ce15fb87 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:22,391 - lava_callback - INFO - Starting async job submit for node 6940924f575bc060ce160a6f 2025-12-17 22:49:22,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:22,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924f575bc060ce160a6f HTTP/1.1" 502 150 2025-12-17 22:49:22,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940924f575bc060ce160a6f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:22,427 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940924f575bc060ce160a6f 2025-12-17 22:49:22,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924f575bc060ce160a6f HTTP/1.1" 502 150 2025-12-17 22:49:22,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940924f575bc060ce160a6f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:22,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c36575bc060ce15f4af 2025-12-17 22:49:22,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c36575bc060ce15f4af HTTP/1.1" 502 150 2025-12-17 22:49:22,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c36575bc060ce15f4af'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:22,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e2f575bc060ce15fd3a 2025-12-17 22:49:22,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e2f575bc060ce15fd3a HTTP/1.1" 502 150 2025-12-17 22:49:22,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e2f575bc060ce15fd3a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:22,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e20575bc060ce15fb93 2025-12-17 22:49:22,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e20575bc060ce15fb93 HTTP/1.1" 502 150 2025-12-17 22:49:22,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e20575bc060ce15fb93'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:23,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1d575bc060ce15fb8d 2025-12-17 22:49:23,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1d575bc060ce15fb8d HTTP/1.1" 502 150 2025-12-17 22:49:23,008 - lava_callback - ERROR - Error processing callback for node 69408e1d575bc060ce15fb8d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e1d575bc060ce15fb8d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e1d575bc060ce15fb8d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e1d575bc060ce15fb8d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:23,011 - lava_callback - INFO - Starting async job submit for node 69408e39575bc060ce15fd65 2025-12-17 22:49:23,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:23,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e39575bc060ce15fd65 HTTP/1.1" 502 150 2025-12-17 22:49:23,050 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e39575bc060ce15fd65'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:23,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e39575bc060ce15fd65 2025-12-17 22:49:23,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e39575bc060ce15fd65 HTTP/1.1" 502 150 2025-12-17 22:49:23,053 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e39575bc060ce15fd65'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:23,119 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e37575bc060ce15fd5f 2025-12-17 22:49:23,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e37575bc060ce15fd5f HTTP/1.1" 502 150 2025-12-17 22:49:23,120 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e37575bc060ce15fd5f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:23,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e38575bc060ce15fd62 2025-12-17 22:49:23,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e38575bc060ce15fd62 HTTP/1.1" 502 150 2025-12-17 22:49:23,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e38575bc060ce15fd62'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:24,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940924f575bc060ce160a6f 2025-12-17 22:49:24,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924f575bc060ce160a6f HTTP/1.1" 502 150 2025-12-17 22:49:24,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940924f575bc060ce160a6f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:24,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e22575bc060ce15fb9a 2025-12-17 22:49:24,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e22575bc060ce15fb9a HTTP/1.1" 502 150 2025-12-17 22:49:24,745 - lava_callback - ERROR - Error processing callback for node 69408e22575bc060ce15fb9a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e22575bc060ce15fb9a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e22575bc060ce15fb9a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e22575bc060ce15fb9a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:24,747 - lava_callback - INFO - Starting async job submit for node 69408e30575bc060ce15fd3d 2025-12-17 22:49:24,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:24,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e30575bc060ce15fd3d HTTP/1.1" 502 150 2025-12-17 22:49:24,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e30575bc060ce15fd3d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:24,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e30575bc060ce15fd3d 2025-12-17 22:49:24,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e30575bc060ce15fd3d HTTP/1.1" 502 150 2025-12-17 22:49:24,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e30575bc060ce15fd3d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:25,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e25575bc060ce15fba3 2025-12-17 22:49:25,053 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e39575bc060ce15fd65 2025-12-17 22:49:25,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e39575bc060ce15fd65 HTTP/1.1" 502 150 2025-12-17 22:49:25,064 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e39575bc060ce15fd65'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:25,341 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e95575bc060ce15fe85 2025-12-17 22:49:25,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e95575bc060ce15fe85 HTTP/1.1" 502 150 2025-12-17 22:49:25,343 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e95575bc060ce15fe85'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:25,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e34575bc060ce15fd54 2025-12-17 22:49:25,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e34575bc060ce15fd54 HTTP/1.1" 502 150 2025-12-17 22:49:25,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e34575bc060ce15fd54'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:25,689 - lava_callback - INFO - Received callback for node 69409062575bc060ce16056b INFO: 172.21.0.1:60016 - "POST /node/69409062575bc060ce16056b HTTP/1.0" 202 Accepted 2025-12-17 22:49:25,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e23575bc060ce15fb9d HTTP/1.1" 502 150 2025-12-17 22:49:25,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e23575bc060ce15fb9d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:25,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e27575bc060ce15fba6 2025-12-17 22:49:25,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e27575bc060ce15fba6 HTTP/1.1" 502 150 2025-12-17 22:49:25,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e27575bc060ce15fba6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:26,351 - lava_callback - INFO - Received callback for node 69409064575bc060ce160577 INFO: 172.21.0.1:60028 - "POST /node/69409064575bc060ce160577 HTTP/1.0" 202 Accepted 2025-12-17 22:49:26,440 - lava_callback - INFO - Received callback for node 69409063575bc060ce160572 INFO: 172.21.0.1:60042 - "POST /node/69409063575bc060ce160572 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:60046 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:49:26,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e30575bc060ce15fd3d 2025-12-17 22:49:26,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e30575bc060ce15fd3d HTTP/1.1" 502 150 2025-12-17 22:49:26,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e30575bc060ce15fd3d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:26,949 - lava_callback - INFO - Received callback for node 694101ba460bc8e265116a92 INFO: 172.21.0.1:60062 - "POST /node/694101ba460bc8e265116a92 HTTP/1.0" 202 Accepted 2025-12-17 22:49:26,987 - lava_callback - INFO - Received callback for node 69409067575bc060ce16057d INFO: 172.21.0.1:60074 - "POST /node/69409067575bc060ce16057d HTTP/1.0" 202 Accepted 2025-12-17 22:49:27,120 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e37575bc060ce15fd5f 2025-12-17 22:49:27,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e37575bc060ce15fd5f HTTP/1.1" 502 150 2025-12-17 22:49:27,122 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e37575bc060ce15fd5f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:27,241 - lava_callback - INFO - Received callback for node 69409066575bc060ce16057a INFO: 172.21.0.1:60088 - "POST /node/69409066575bc060ce16057a HTTP/1.0" 202 Accepted 2025-12-17 22:49:27,638 - lava_callback - INFO - Received callback for node 69409068575bc060ce160580 INFO: 172.21.0.1:60094 - "POST /node/69409068575bc060ce160580 HTTP/1.0" 202 Accepted 2025-12-17 22:49:27,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e38575bc060ce15fd62 2025-12-17 22:49:27,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e38575bc060ce15fd62 HTTP/1.1" 502 150 2025-12-17 22:49:27,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e38575bc060ce15fd62'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:28,288 - lava_callback - INFO - Received callback for node 69409069575bc060ce160583 INFO: 172.21.0.1:60104 - "POST /node/69409069575bc060ce160583 HTTP/1.0" 202 Accepted 2025-12-17 22:49:28,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940924f575bc060ce160a6f 2025-12-17 22:49:28,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924f575bc060ce160a6f HTTP/1.1" 502 150 2025-12-17 22:49:28,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940924f575bc060ce160a6f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:28,933 - lava_callback - INFO - Received callback for node 6940906b575bc060ce160586 INFO: 172.21.0.1:60118 - "POST /node/6940906b575bc060ce160586 HTTP/1.0" 202 Accepted 2025-12-17 22:49:29,064 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e39575bc060ce15fd65 2025-12-17 22:49:29,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e39575bc060ce15fd65 HTTP/1.1" 502 150 2025-12-17 22:49:29,066 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e39575bc060ce15fd65'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:29,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e2d575bc060ce15fd32 2025-12-17 22:49:29,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e2d575bc060ce15fd32 HTTP/1.1" 502 150 2025-12-17 22:49:29,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e2d575bc060ce15fd32'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:30,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e30575bc060ce15fd3d 2025-12-17 22:49:30,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e30575bc060ce15fd3d HTTP/1.1" 502 150 2025-12-17 22:49:30,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e30575bc060ce15fd3d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:30,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e2f575bc060ce15fd3a 2025-12-17 22:49:30,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e2f575bc060ce15fd3a HTTP/1.1" 502 150 2025-12-17 22:49:30,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e2f575bc060ce15fd3a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:33,343 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e95575bc060ce15fe85 2025-12-17 22:49:33,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e95575bc060ce15fe85 HTTP/1.1" 502 150 2025-12-17 22:49:33,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e95575bc060ce15fe85'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:33,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e34575bc060ce15fd54 2025-12-17 22:49:33,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e34575bc060ce15fd54 HTTP/1.1" 502 150 2025-12-17 22:49:33,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e34575bc060ce15fd54'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:33,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e23575bc060ce15fb9d 2025-12-17 22:49:33,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e23575bc060ce15fb9d HTTP/1.1" 502 150 2025-12-17 22:49:33,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e23575bc060ce15fb9d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:35,122 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e37575bc060ce15fd5f 2025-12-17 22:49:35,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e37575bc060ce15fd5f HTTP/1.1" 502 150 2025-12-17 22:49:35,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e37575bc060ce15fd5f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:35,430 - lava_callback - INFO - Received callback for node 69409072575bc060ce16058f INFO: 172.21.0.1:50178 - "POST /node/69409072575bc060ce16058f HTTP/1.0" 202 Accepted 2025-12-17 22:49:35,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e38575bc060ce15fd62 2025-12-17 22:49:35,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e38575bc060ce15fd62 HTTP/1.1" 502 150 2025-12-17 22:49:35,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e38575bc060ce15fd62'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:36,083 - lava_callback - INFO - Received callback for node 69409074575bc060ce160592 INFO: 172.21.0.1:50180 - "POST /node/69409074575bc060ce160592 HTTP/1.0" 202 Accepted 2025-12-17 22:49:36,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940924f575bc060ce160a6f INFO: 172.21.0.1:50190 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:49:36,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e24575bc060ce15fba0 2025-12-17 22:49:36,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e24575bc060ce15fba0 HTTP/1.1" 502 150 2025-12-17 22:49:36,637 - lava_callback - ERROR - Error processing callback for node 69408e24575bc060ce15fba0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e24575bc060ce15fba0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e24575bc060ce15fba0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e24575bc060ce15fba0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:36,638 - lava_callback - INFO - Starting async job submit for node 69408e32575bc060ce15fd46 2025-12-17 22:49:36,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:36,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e32575bc060ce15fd46 HTTP/1.1" 502 150 2025-12-17 22:49:36,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e32575bc060ce15fd46'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:36,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e32575bc060ce15fd46 2025-12-17 22:49:36,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e32575bc060ce15fd46 HTTP/1.1" 502 150 2025-12-17 22:49:36,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e32575bc060ce15fd46'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:36,745 - lava_callback - INFO - Received callback for node 6940906d575bc060ce160589 INFO: 172.21.0.1:50192 - "POST /node/6940906d575bc060ce160589 HTTP/1.0" 202 Accepted 2025-12-17 22:49:37,066 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e39575bc060ce15fd65 2025-12-17 22:49:37,419 - lava_callback - INFO - Received callback for node 69409070575bc060ce16058c INFO: 172.21.0.1:50194 - "POST /node/69409070575bc060ce16058c HTTP/1.0" 202 Accepted 2025-12-17 22:49:37,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e1f575bc060ce15fb90 2025-12-17 22:49:37,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e1f575bc060ce15fb90 HTTP/1.1" 502 150 2025-12-17 22:49:37,606 - lava_callback - ERROR - Error processing callback for node 69408e1f575bc060ce15fb90: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e1f575bc060ce15fb90 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e1f575bc060ce15fb90 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e1f575bc060ce15fb90 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:37,608 - lava_callback - INFO - Starting async job submit for node 69408e33575bc060ce15fd4d 2025-12-17 22:49:37,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:37,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e33575bc060ce15fd4d HTTP/1.1" 502 150 2025-12-17 22:49:37,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e33575bc060ce15fd4d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:37,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e33575bc060ce15fd4d 2025-12-17 22:49:37,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e33575bc060ce15fd4d HTTP/1.1" 502 150 2025-12-17 22:49:37,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e33575bc060ce15fd4d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:38,076 - lava_callback - INFO - Received callback for node 69409076575bc060ce16059b INFO: 172.21.0.1:50200 - "POST /node/69409076575bc060ce16059b HTTP/1.0" 202 Accepted 2025-12-17 22:49:38,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e32575bc060ce15fd46 2025-12-17 22:49:38,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e32575bc060ce15fd46 HTTP/1.1" 502 150 2025-12-17 22:49:38,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e32575bc060ce15fd46'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:38,725 - lava_callback - INFO - Received callback for node 69409078575bc060ce1605a6 2025-12-17 22:49:38,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c36575bc060ce15f4af INFO: 172.21.0.1:50208 - "POST /node/69409078575bc060ce1605a6 HTTP/1.0" 202 Accepted 2025-12-17 22:49:38,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c36575bc060ce15f4af HTTP/1.1" 502 150 2025-12-17 22:49:38,732 - lava_callback - ERROR - Error processing callback for node 69408c36575bc060ce15f4af: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c36575bc060ce15f4af (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c36575bc060ce15f4af (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c36575bc060ce15f4af (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:38,734 - lava_callback - INFO - Starting async job submit for node 69408e3e575bc060ce15fd71 2025-12-17 22:49:38,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:38,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3e575bc060ce15fd71 HTTP/1.1" 502 150 2025-12-17 22:49:38,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3e575bc060ce15fd71'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:38,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3e575bc060ce15fd71 2025-12-17 22:49:38,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3e575bc060ce15fd71 HTTP/1.1" 502 150 2025-12-17 22:49:38,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3e575bc060ce15fd71'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:38,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e30575bc060ce15fd3d 2025-12-17 22:49:38,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e30575bc060ce15fd3d HTTP/1.1" 502 150 2025-12-17 22:49:38,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e30575bc060ce15fd3d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:38,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e20575bc060ce15fb93 2025-12-17 22:49:38,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e20575bc060ce15fb93 HTTP/1.1" 502 150 2025-12-17 22:49:38,932 - lava_callback - ERROR - Error processing callback for node 69408e20575bc060ce15fb93: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e20575bc060ce15fb93 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e20575bc060ce15fb93 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e20575bc060ce15fb93 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:38,933 - lava_callback - INFO - Starting async job submit for node 69408e40575bc060ce15fd74 2025-12-17 22:49:38,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:38,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e40575bc060ce15fd74 HTTP/1.1" 502 150 2025-12-17 22:49:38,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e40575bc060ce15fd74'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:38,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e40575bc060ce15fd74 2025-12-17 22:49:38,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e40575bc060ce15fd74 HTTP/1.1" 502 150 2025-12-17 22:49:38,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e40575bc060ce15fd74'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:39,379 - lava_callback - INFO - Received callback for node 6940907a575bc060ce1605ad INFO: 172.21.0.1:50216 - "POST /node/6940907a575bc060ce1605ad HTTP/1.0" 202 Accepted 2025-12-17 22:49:39,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e33575bc060ce15fd4d 2025-12-17 22:49:39,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e33575bc060ce15fd4d HTTP/1.1" 502 150 2025-12-17 22:49:39,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e33575bc060ce15fd4d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:40,023 - lava_callback - INFO - Received callback for node 6940907c575bc060ce1605b0 INFO: 172.21.0.1:42534 - "POST /node/6940907c575bc060ce1605b0 HTTP/1.0" 202 Accepted 2025-12-17 22:49:40,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e25575bc060ce15fba3 HTTP/1.1" 502 150 2025-12-17 22:49:40,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e25575bc060ce15fba3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:40,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3e575bc060ce15fd71 2025-12-17 22:49:40,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3e575bc060ce15fd71 HTTP/1.1" 502 150 2025-12-17 22:49:40,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3e575bc060ce15fd71'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:40,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e40575bc060ce15fd74 2025-12-17 22:49:40,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e40575bc060ce15fd74 HTTP/1.1" 502 150 2025-12-17 22:49:40,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e40575bc060ce15fd74'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:41,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e27575bc060ce15fba6 2025-12-17 22:49:41,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e27575bc060ce15fba6 HTTP/1.1" 502 150 2025-12-17 22:49:41,974 - lava_callback - ERROR - Error processing callback for node 69408e27575bc060ce15fba6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e27575bc060ce15fba6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e27575bc060ce15fba6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e27575bc060ce15fba6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:41,976 - lava_callback - INFO - Starting async job submit for node 69408e41575bc060ce15fd77 2025-12-17 22:49:41,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:41,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e41575bc060ce15fd77 HTTP/1.1" 502 150 2025-12-17 22:49:41,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e41575bc060ce15fd77'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:41,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e41575bc060ce15fd77 2025-12-17 22:49:41,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e41575bc060ce15fd77 HTTP/1.1" 502 150 2025-12-17 22:49:41,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e41575bc060ce15fd77'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:42,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e32575bc060ce15fd46 2025-12-17 22:49:42,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e32575bc060ce15fd46 HTTP/1.1" 502 150 2025-12-17 22:49:42,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e32575bc060ce15fd46'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:43,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e33575bc060ce15fd4d 2025-12-17 22:49:43,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e33575bc060ce15fd4d HTTP/1.1" 502 150 2025-12-17 22:49:43,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e33575bc060ce15fd4d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:43,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e41575bc060ce15fd77 2025-12-17 22:49:43,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e41575bc060ce15fd77 HTTP/1.1" 502 150 2025-12-17 22:49:43,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e41575bc060ce15fd77'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:44,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3e575bc060ce15fd71 2025-12-17 22:49:44,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3e575bc060ce15fd71 HTTP/1.1" 502 150 2025-12-17 22:49:44,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3e575bc060ce15fd71'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:44,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e40575bc060ce15fd74 2025-12-17 22:49:44,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e40575bc060ce15fd74 HTTP/1.1" 502 150 2025-12-17 22:49:44,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e40575bc060ce15fd74'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:45,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e2d575bc060ce15fd32 2025-12-17 22:49:45,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e2d575bc060ce15fd32 HTTP/1.1" 502 150 2025-12-17 22:49:45,488 - lava_callback - ERROR - Error processing callback for node 69408e2d575bc060ce15fd32: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e2d575bc060ce15fd32 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e2d575bc060ce15fd32 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e2d575bc060ce15fd32 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:45,490 - lava_callback - INFO - Starting async job submit for node 69408e42575bc060ce15fd7a 2025-12-17 22:49:45,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:45,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e42575bc060ce15fd7a HTTP/1.1" 502 150 2025-12-17 22:49:45,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e42575bc060ce15fd7a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:45,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e42575bc060ce15fd7a 2025-12-17 22:49:45,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e42575bc060ce15fd7a HTTP/1.1" 502 150 2025-12-17 22:49:45,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e42575bc060ce15fd7a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:45,695 - lava_callback - INFO - Received callback for node 69409082575bc060ce1605b9 INFO: 172.21.0.1:42540 - "POST /node/69409082575bc060ce1605b9 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:42550 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:49:46,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e2f575bc060ce15fd3a 2025-12-17 22:49:46,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e2f575bc060ce15fd3a HTTP/1.1" 502 150 2025-12-17 22:49:46,806 - lava_callback - ERROR - Error processing callback for node 69408e2f575bc060ce15fd3a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e2f575bc060ce15fd3a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e2f575bc060ce15fd3a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e2f575bc060ce15fd3a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:46,808 - lava_callback - INFO - Starting async job submit for node 69408e43575bc060ce15fd7d 2025-12-17 22:49:46,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:46,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e43575bc060ce15fd7d HTTP/1.1" 502 150 2025-12-17 22:49:46,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e43575bc060ce15fd7d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:46,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e43575bc060ce15fd7d 2025-12-17 22:49:46,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e43575bc060ce15fd7d HTTP/1.1" 502 150 2025-12-17 22:49:46,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e43575bc060ce15fd7d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:47,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e42575bc060ce15fd7a 2025-12-17 22:49:47,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e42575bc060ce15fd7a HTTP/1.1" 502 150 2025-12-17 22:49:47,501 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e42575bc060ce15fd7a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:47,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e41575bc060ce15fd77 2025-12-17 22:49:47,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e41575bc060ce15fd77 HTTP/1.1" 502 150 2025-12-17 22:49:47,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e41575bc060ce15fd77'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:48,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e43575bc060ce15fd7d 2025-12-17 22:49:48,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e43575bc060ce15fd7d HTTP/1.1" 502 150 2025-12-17 22:49:48,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e43575bc060ce15fd7d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:49,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e95575bc060ce15fe85 2025-12-17 22:49:49,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e95575bc060ce15fe85 HTTP/1.1" 502 150 2025-12-17 22:49:49,346 - lava_callback - ERROR - Error processing callback for node 69408e95575bc060ce15fe85: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e95575bc060ce15fe85 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e95575bc060ce15fe85 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e95575bc060ce15fe85 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:49,348 - lava_callback - INFO - Starting async job submit for node 69408e3c575bc060ce15fd6b 2025-12-17 22:49:49,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:49,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3c575bc060ce15fd6b HTTP/1.1" 502 150 2025-12-17 22:49:49,372 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3c575bc060ce15fd6b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:49,372 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3c575bc060ce15fd6b 2025-12-17 22:49:49,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3c575bc060ce15fd6b HTTP/1.1" 502 150 2025-12-17 22:49:49,373 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3c575bc060ce15fd6b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:49,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e34575bc060ce15fd54 2025-12-17 22:49:49,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e34575bc060ce15fd54 HTTP/1.1" 502 150 2025-12-17 22:49:49,586 - lava_callback - ERROR - Error processing callback for node 69408e34575bc060ce15fd54: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e34575bc060ce15fd54 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e34575bc060ce15fd54 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e34575bc060ce15fd54 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:49,587 - lava_callback - INFO - Starting async job submit for node 69408e3d575bc060ce15fd6e 2025-12-17 22:49:49,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:49,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3d575bc060ce15fd6e HTTP/1.1" 502 150 2025-12-17 22:49:49,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3d575bc060ce15fd6e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:49,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3d575bc060ce15fd6e 2025-12-17 22:49:49,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3d575bc060ce15fd6e HTTP/1.1" 502 150 2025-12-17 22:49:49,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3d575bc060ce15fd6e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:49,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e23575bc060ce15fb9d 2025-12-17 22:49:49,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e23575bc060ce15fb9d HTTP/1.1" 502 150 2025-12-17 22:49:49,939 - lava_callback - ERROR - Error processing callback for node 69408e23575bc060ce15fb9d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e23575bc060ce15fb9d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e23575bc060ce15fb9d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e23575bc060ce15fb9d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:49,941 - lava_callback - INFO - Starting async job submit for node 69408e3b575bc060ce15fd68 2025-12-17 22:49:49,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:49,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3b575bc060ce15fd68 HTTP/1.1" 502 150 2025-12-17 22:49:49,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3b575bc060ce15fd68'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:49,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3b575bc060ce15fd68 2025-12-17 22:49:49,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3b575bc060ce15fd68 HTTP/1.1" 502 150 2025-12-17 22:49:49,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3b575bc060ce15fd68'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:50,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e32575bc060ce15fd46 2025-12-17 22:49:50,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e32575bc060ce15fd46 HTTP/1.1" 502 150 2025-12-17 22:49:50,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e32575bc060ce15fd46'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:51,123 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e37575bc060ce15fd5f 2025-12-17 22:49:51,373 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3c575bc060ce15fd6b 2025-12-17 22:49:51,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3c575bc060ce15fd6b HTTP/1.1" 502 150 2025-12-17 22:49:51,374 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3c575bc060ce15fd6b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:51,501 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e42575bc060ce15fd7a 2025-12-17 22:49:51,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e42575bc060ce15fd7a HTTP/1.1" 502 150 2025-12-17 22:49:51,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e42575bc060ce15fd7a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:51,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3d575bc060ce15fd6e 2025-12-17 22:49:51,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3d575bc060ce15fd6e HTTP/1.1" 502 150 2025-12-17 22:49:51,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3d575bc060ce15fd6e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:51,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e33575bc060ce15fd4d 2025-12-17 22:49:51,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e33575bc060ce15fd4d HTTP/1.1" 502 150 2025-12-17 22:49:51,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e33575bc060ce15fd4d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:51,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924f575bc060ce160a6f HTTP/1.1" 502 150 2025-12-17 22:49:51,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940924f575bc060ce160a6f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:51,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e38575bc060ce15fd62 2025-12-17 22:49:51,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e38575bc060ce15fd62 HTTP/1.1" 502 150 2025-12-17 22:49:51,913 - lava_callback - ERROR - Error processing callback for node 69408e38575bc060ce15fd62: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e38575bc060ce15fd62 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e38575bc060ce15fd62 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e38575bc060ce15fd62 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:51,915 - lava_callback - INFO - Starting async job submit for node 69408e4f575bc060ce15fdac 2025-12-17 22:49:51,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:51,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4f575bc060ce15fdac HTTP/1.1" 502 150 2025-12-17 22:49:51,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4f575bc060ce15fdac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:51,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4f575bc060ce15fdac 2025-12-17 22:49:51,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4f575bc060ce15fdac HTTP/1.1" 502 150 2025-12-17 22:49:51,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4f575bc060ce15fdac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:51,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3b575bc060ce15fd68 2025-12-17 22:49:51,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3b575bc060ce15fd68 HTTP/1.1" 502 150 2025-12-17 22:49:51,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3b575bc060ce15fd68'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:52,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e39575bc060ce15fd65 HTTP/1.1" 502 150 2025-12-17 22:49:52,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e39575bc060ce15fd65'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:52,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3e575bc060ce15fd71 2025-12-17 22:49:52,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3e575bc060ce15fd71 HTTP/1.1" 502 150 2025-12-17 22:49:52,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3e575bc060ce15fd71'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:52,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e43575bc060ce15fd7d 2025-12-17 22:49:52,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e43575bc060ce15fd7d HTTP/1.1" 502 150 2025-12-17 22:49:52,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e43575bc060ce15fd7d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:52,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e40575bc060ce15fd74 2025-12-17 22:49:52,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e40575bc060ce15fd74 HTTP/1.1" 502 150 2025-12-17 22:49:52,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e40575bc060ce15fd74'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:53,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4f575bc060ce15fdac 2025-12-17 22:49:53,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4f575bc060ce15fdac HTTP/1.1" 502 150 2025-12-17 22:49:53,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4f575bc060ce15fdac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:54,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e30575bc060ce15fd3d 2025-12-17 22:49:54,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e30575bc060ce15fd3d HTTP/1.1" 502 150 2025-12-17 22:49:54,785 - lava_callback - ERROR - Error processing callback for node 69408e30575bc060ce15fd3d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e30575bc060ce15fd3d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e30575bc060ce15fd3d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e30575bc060ce15fd3d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:54,786 - lava_callback - INFO - Starting async job submit for node 69408e50575bc060ce15fdaf 2025-12-17 22:49:54,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:54,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e50575bc060ce15fdaf HTTP/1.1" 502 150 2025-12-17 22:49:54,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e50575bc060ce15fdaf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:54,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e50575bc060ce15fdaf 2025-12-17 22:49:54,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e50575bc060ce15fdaf HTTP/1.1" 502 150 2025-12-17 22:49:54,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e50575bc060ce15fdaf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:55,375 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3c575bc060ce15fd6b 2025-12-17 22:49:55,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3c575bc060ce15fd6b HTTP/1.1" 502 150 2025-12-17 22:49:55,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3c575bc060ce15fd6b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:55,387 - lava_callback - INFO - Received callback for node 6940908f575bc060ce1605e5 INFO: 172.21.0.1:43782 - "POST /node/6940908f575bc060ce1605e5 HTTP/1.0" 202 Accepted 2025-12-17 22:49:55,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3d575bc060ce15fd6e 2025-12-17 22:49:55,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3d575bc060ce15fd6e HTTP/1.1" 502 150 2025-12-17 22:49:55,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3d575bc060ce15fd6e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:55,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3b575bc060ce15fd68 2025-12-17 22:49:55,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3b575bc060ce15fd68 HTTP/1.1" 502 150 2025-12-17 22:49:55,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3b575bc060ce15fd68'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:55,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e41575bc060ce15fd77 2025-12-17 22:49:55,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e41575bc060ce15fd77 HTTP/1.1" 502 150 2025-12-17 22:49:55,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e41575bc060ce15fd77'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:56,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e25575bc060ce15fba3 2025-12-17 22:49:56,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e25575bc060ce15fba3 HTTP/1.1" 502 150 2025-12-17 22:49:56,274 - lava_callback - ERROR - Error processing callback for node 69408e25575bc060ce15fba3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e25575bc060ce15fba3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e25575bc060ce15fba3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e25575bc060ce15fba3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:49:56,275 - lava_callback - INFO - Starting async job submit for node 69408e51575bc060ce15fdb2 2025-12-17 22:49:56,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:49:56,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e51575bc060ce15fdb2 HTTP/1.1" 502 150 2025-12-17 22:49:56,305 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e51575bc060ce15fdb2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:56,305 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e51575bc060ce15fdb2 2025-12-17 22:49:56,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e51575bc060ce15fdb2 HTTP/1.1" 502 150 2025-12-17 22:49:56,306 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e51575bc060ce15fdb2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43784 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:49:56,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e50575bc060ce15fdaf 2025-12-17 22:49:56,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e50575bc060ce15fdaf HTTP/1.1" 502 150 2025-12-17 22:49:56,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e50575bc060ce15fdaf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:57,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4f575bc060ce15fdac 2025-12-17 22:49:57,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4f575bc060ce15fdac HTTP/1.1" 502 150 2025-12-17 22:49:57,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4f575bc060ce15fdac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:58,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e51575bc060ce15fdb2 2025-12-17 22:49:58,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e51575bc060ce15fdb2 HTTP/1.1" 502 150 2025-12-17 22:49:58,308 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e51575bc060ce15fdb2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:49:59,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e42575bc060ce15fd7a 2025-12-17 22:49:59,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e42575bc060ce15fd7a HTTP/1.1" 502 150 2025-12-17 22:49:59,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e42575bc060ce15fd7a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:00,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e50575bc060ce15fdaf 2025-12-17 22:50:00,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e50575bc060ce15fdaf HTTP/1.1" 502 150 2025-12-17 22:50:00,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e50575bc060ce15fdaf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:00,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e43575bc060ce15fd7d 2025-12-17 22:50:00,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e43575bc060ce15fd7d HTTP/1.1" 502 150 2025-12-17 22:50:00,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e43575bc060ce15fd7d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:02,308 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e51575bc060ce15fdb2 INFO: 172.21.0.1:53920 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:50:03,376 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3c575bc060ce15fd6b 2025-12-17 22:50:03,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3d575bc060ce15fd6e 2025-12-17 22:50:03,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3d575bc060ce15fd6e HTTP/1.1" 502 150 2025-12-17 22:50:03,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3d575bc060ce15fd6e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:03,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3b575bc060ce15fd68 2025-12-17 22:50:03,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3b575bc060ce15fd68 HTTP/1.1" 502 150 2025-12-17 22:50:03,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3b575bc060ce15fd68'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:05,751 - lava_callback - INFO - Received callback for node 69409088575bc060ce1605bc INFO: 172.21.0.1:53932 - "POST /node/69409088575bc060ce1605bc HTTP/1.0" 202 Accepted 2025-12-17 22:50:05,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4f575bc060ce15fdac 2025-12-17 22:50:05,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4f575bc060ce15fdac HTTP/1.1" 502 150 2025-12-17 22:50:05,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4f575bc060ce15fdac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:06,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e37575bc060ce15fd5f HTTP/1.1" 502 150 2025-12-17 22:50:06,384 - lava_callback - ERROR - Error processing callback for node 69408e37575bc060ce15fd5f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e37575bc060ce15fd5f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e37575bc060ce15fd5f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e37575bc060ce15fd5f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:06,386 - lava_callback - INFO - Starting async job submit for node 69408e45575bc060ce15fd88 2025-12-17 22:50:06,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:06,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e45575bc060ce15fd88 HTTP/1.1" 502 150 2025-12-17 22:50:06,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e45575bc060ce15fd88'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:06,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e45575bc060ce15fd88 2025-12-17 22:50:06,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e45575bc060ce15fd88 HTTP/1.1" 502 150 2025-12-17 22:50:06,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e45575bc060ce15fd88'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:06,395 - lava_callback - INFO - Received callback for node 6940908b575bc060ce1605d9 INFO: 172.21.0.1:53938 - "POST /node/6940908b575bc060ce1605d9 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:53948 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:50:06,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e32575bc060ce15fd46 2025-12-17 22:50:06,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e32575bc060ce15fd46 HTTP/1.1" 502 150 2025-12-17 22:50:06,676 - lava_callback - ERROR - Error processing callback for node 69408e32575bc060ce15fd46: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e32575bc060ce15fd46 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e32575bc060ce15fd46 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e32575bc060ce15fd46 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:06,677 - lava_callback - INFO - Starting async job submit for node 69408e46575bc060ce15fd95 2025-12-17 22:50:06,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:06,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e46575bc060ce15fd95 HTTP/1.1" 502 150 2025-12-17 22:50:06,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e46575bc060ce15fd95'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:06,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e46575bc060ce15fd95 2025-12-17 22:50:06,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e46575bc060ce15fd95 HTTP/1.1" 502 150 2025-12-17 22:50:06,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e46575bc060ce15fd95'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:07,037 - lava_callback - INFO - Received callback for node 6940908d575bc060ce1605e2 INFO: 172.21.0.1:53952 - "POST /node/6940908d575bc060ce1605e2 HTTP/1.0" 202 Accepted 2025-12-17 22:50:07,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e33575bc060ce15fd4d 2025-12-17 22:50:07,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e33575bc060ce15fd4d HTTP/1.1" 502 150 2025-12-17 22:50:07,623 - lava_callback - ERROR - Error processing callback for node 69408e33575bc060ce15fd4d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e33575bc060ce15fd4d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e33575bc060ce15fd4d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e33575bc060ce15fd4d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:07,625 - lava_callback - INFO - Starting async job submit for node 69408e48575bc060ce15fd9a 2025-12-17 22:50:07,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:07,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e48575bc060ce15fd9a HTTP/1.1" 502 150 2025-12-17 22:50:07,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e48575bc060ce15fd9a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:07,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e48575bc060ce15fd9a 2025-12-17 22:50:07,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e48575bc060ce15fd9a HTTP/1.1" 502 150 2025-12-17 22:50:07,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e48575bc060ce15fd9a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:07,698 - lava_callback - INFO - Received callback for node 69409096575bc060ce1605f0 INFO: 172.21.0.1:53958 - "POST /node/69409096575bc060ce1605f0 HTTP/1.0" 202 Accepted 2025-12-17 22:50:07,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940924f575bc060ce160a6f 2025-12-17 22:50:07,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940924f575bc060ce160a6f HTTP/1.1" 502 150 2025-12-17 22:50:07,797 - lava_callback - ERROR - Error processing callback for node 6940924f575bc060ce160a6f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940924f575bc060ce160a6f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940924f575bc060ce160a6f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940924f575bc060ce160a6f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:07,799 - lava_callback - INFO - Starting async job submit for node 69408e49575bc060ce15fd9d 2025-12-17 22:50:07,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:07,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e49575bc060ce15fd9d HTTP/1.1" 502 150 2025-12-17 22:50:07,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e49575bc060ce15fd9d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:07,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e49575bc060ce15fd9d 2025-12-17 22:50:07,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e49575bc060ce15fd9d HTTP/1.1" 502 150 2025-12-17 22:50:07,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e49575bc060ce15fd9d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:08,357 - lava_callback - INFO - Received callback for node 69409098575bc060ce1605f3 INFO: 172.21.0.1:53974 - "POST /node/69409098575bc060ce1605f3 HTTP/1.0" 202 Accepted 2025-12-17 22:50:08,393 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e45575bc060ce15fd88 2025-12-17 22:50:08,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e45575bc060ce15fd88 HTTP/1.1" 502 150 2025-12-17 22:50:08,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e45575bc060ce15fd88'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:08,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e39575bc060ce15fd65 2025-12-17 22:50:08,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e39575bc060ce15fd65 HTTP/1.1" 502 150 2025-12-17 22:50:08,562 - lava_callback - ERROR - Error processing callback for node 69408e39575bc060ce15fd65: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e39575bc060ce15fd65 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e39575bc060ce15fd65 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e39575bc060ce15fd65 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:08,563 - lava_callback - INFO - Starting async job submit for node 69408e4a575bc060ce15fda0 2025-12-17 22:50:08,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:08,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4a575bc060ce15fda0 HTTP/1.1" 502 150 2025-12-17 22:50:08,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4a575bc060ce15fda0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:08,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4a575bc060ce15fda0 2025-12-17 22:50:08,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4a575bc060ce15fda0 HTTP/1.1" 502 150 2025-12-17 22:50:08,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4a575bc060ce15fda0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:08,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e46575bc060ce15fd95 2025-12-17 22:50:08,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e46575bc060ce15fd95 HTTP/1.1" 502 150 2025-12-17 22:50:08,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e46575bc060ce15fd95'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:08,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3e575bc060ce15fd71 2025-12-17 22:50:08,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3e575bc060ce15fd71 HTTP/1.1" 502 150 2025-12-17 22:50:08,748 - lava_callback - ERROR - Error processing callback for node 69408e3e575bc060ce15fd71: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e3e575bc060ce15fd71 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e3e575bc060ce15fd71 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e3e575bc060ce15fd71 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:08,750 - lava_callback - INFO - Starting async job submit for node 69408e4b575bc060ce15fda3 2025-12-17 22:50:08,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:08,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4b575bc060ce15fda3 HTTP/1.1" 502 150 2025-12-17 22:50:08,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4b575bc060ce15fda3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:08,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4b575bc060ce15fda3 2025-12-17 22:50:08,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4b575bc060ce15fda3 HTTP/1.1" 502 150 2025-12-17 22:50:08,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4b575bc060ce15fda3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:08,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e50575bc060ce15fdaf 2025-12-17 22:50:08,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e50575bc060ce15fdaf HTTP/1.1" 502 150 2025-12-17 22:50:08,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e50575bc060ce15fdaf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:08,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e40575bc060ce15fd74 2025-12-17 22:50:08,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e40575bc060ce15fd74 HTTP/1.1" 502 150 2025-12-17 22:50:08,946 - lava_callback - ERROR - Error processing callback for node 69408e40575bc060ce15fd74: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e40575bc060ce15fd74 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e40575bc060ce15fd74 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e40575bc060ce15fd74 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:08,948 - lava_callback - INFO - Starting async job submit for node 69408e4d575bc060ce15fda6 2025-12-17 22:50:08,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:08,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4d575bc060ce15fda6 HTTP/1.1" 502 150 2025-12-17 22:50:08,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4d575bc060ce15fda6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:08,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4d575bc060ce15fda6 2025-12-17 22:50:08,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4d575bc060ce15fda6 HTTP/1.1" 502 150 2025-12-17 22:50:08,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4d575bc060ce15fda6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:09,006 - lava_callback - INFO - Received callback for node 6940909a575bc060ce1605f6 INFO: 172.21.0.1:53978 - "POST /node/6940909a575bc060ce1605f6 HTTP/1.0" 202 Accepted 2025-12-17 22:50:09,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e48575bc060ce15fd9a 2025-12-17 22:50:09,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e48575bc060ce15fd9a HTTP/1.1" 502 150 2025-12-17 22:50:09,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e48575bc060ce15fd9a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:09,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e49575bc060ce15fd9d 2025-12-17 22:50:09,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e49575bc060ce15fd9d HTTP/1.1" 502 150 2025-12-17 22:50:09,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e49575bc060ce15fd9d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:10,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4a575bc060ce15fda0 2025-12-17 22:50:10,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4a575bc060ce15fda0 HTTP/1.1" 502 150 2025-12-17 22:50:10,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4a575bc060ce15fda0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:10,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4b575bc060ce15fda3 2025-12-17 22:50:10,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4b575bc060ce15fda3 HTTP/1.1" 502 150 2025-12-17 22:50:10,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4b575bc060ce15fda3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:10,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4d575bc060ce15fda6 2025-12-17 22:50:10,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4d575bc060ce15fda6 HTTP/1.1" 502 150 2025-12-17 22:50:10,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4d575bc060ce15fda6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:11,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e41575bc060ce15fd77 2025-12-17 22:50:11,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e41575bc060ce15fd77 HTTP/1.1" 502 150 2025-12-17 22:50:11,990 - lava_callback - ERROR - Error processing callback for node 69408e41575bc060ce15fd77: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e41575bc060ce15fd77 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e41575bc060ce15fd77 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e41575bc060ce15fd77 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:11,992 - lava_callback - INFO - Starting async job submit for node 69408e57575bc060ce15fdbe 2025-12-17 22:50:11,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:11,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e57575bc060ce15fdbe HTTP/1.1" 502 150 2025-12-17 22:50:11,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e57575bc060ce15fdbe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:11,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e57575bc060ce15fdbe 2025-12-17 22:50:12,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e57575bc060ce15fdbe HTTP/1.1" 502 150 2025-12-17 22:50:12,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e57575bc060ce15fdbe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:12,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e45575bc060ce15fd88 2025-12-17 22:50:12,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e45575bc060ce15fd88 HTTP/1.1" 502 150 2025-12-17 22:50:12,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e45575bc060ce15fd88'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:12,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e46575bc060ce15fd95 2025-12-17 22:50:12,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e46575bc060ce15fd95 HTTP/1.1" 502 150 2025-12-17 22:50:12,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e46575bc060ce15fd95'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:13,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e48575bc060ce15fd9a 2025-12-17 22:50:13,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e48575bc060ce15fd9a HTTP/1.1" 502 150 2025-12-17 22:50:13,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e48575bc060ce15fd9a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:13,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e49575bc060ce15fd9d 2025-12-17 22:50:13,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e49575bc060ce15fd9d HTTP/1.1" 502 150 2025-12-17 22:50:13,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e49575bc060ce15fd9d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:14,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e57575bc060ce15fdbe 2025-12-17 22:50:14,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4a575bc060ce15fda0 2025-12-17 22:50:14,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4a575bc060ce15fda0 HTTP/1.1" 502 150 2025-12-17 22:50:14,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4a575bc060ce15fda0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:14,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4b575bc060ce15fda3 2025-12-17 22:50:14,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4b575bc060ce15fda3 HTTP/1.1" 502 150 2025-12-17 22:50:14,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4b575bc060ce15fda3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:14,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4d575bc060ce15fda6 2025-12-17 22:50:14,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4d575bc060ce15fda6 HTTP/1.1" 502 150 2025-12-17 22:50:14,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4d575bc060ce15fda6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57300 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:50:15,403 - lava_callback - INFO - Received callback for node 69409092575bc060ce1605ea INFO: 172.21.0.1:57304 - "POST /node/69409092575bc060ce1605ea HTTP/1.0" 202 Accepted 2025-12-17 22:50:15,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e42575bc060ce15fd7a 2025-12-17 22:50:15,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e42575bc060ce15fd7a HTTP/1.1" 502 150 2025-12-17 22:50:15,506 - lava_callback - ERROR - Error processing callback for node 69408e42575bc060ce15fd7a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e42575bc060ce15fd7a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e42575bc060ce15fd7a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e42575bc060ce15fd7a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:15,508 - lava_callback - INFO - Starting async job submit for node 69408e53575bc060ce15fdb5 2025-12-17 22:50:15,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:15,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e53575bc060ce15fdb5 HTTP/1.1" 502 150 2025-12-17 22:50:15,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e53575bc060ce15fdb5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:15,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e53575bc060ce15fdb5 2025-12-17 22:50:15,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e53575bc060ce15fdb5 HTTP/1.1" 502 150 2025-12-17 22:50:15,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e53575bc060ce15fdb5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:16,074 - lava_callback - INFO - Received callback for node 694090a1575bc060ce16060b INFO: 172.21.0.1:57320 - "POST /node/694090a1575bc060ce16060b HTTP/1.0" 202 Accepted INFO: 172.21.0.1:57328 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:50:16,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e43575bc060ce15fd7d 2025-12-17 22:50:16,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e43575bc060ce15fd7d HTTP/1.1" 502 150 2025-12-17 22:50:16,824 - lava_callback - ERROR - Error processing callback for node 69408e43575bc060ce15fd7d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e43575bc060ce15fd7d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e43575bc060ce15fd7d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e43575bc060ce15fd7d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:16,826 - lava_callback - INFO - Starting async job submit for node 69408e55575bc060ce15fdb8 2025-12-17 22:50:16,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:16,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e55575bc060ce15fdb8 HTTP/1.1" 502 150 2025-12-17 22:50:16,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e55575bc060ce15fdb8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:16,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e55575bc060ce15fdb8 2025-12-17 22:50:16,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e55575bc060ce15fdb8 HTTP/1.1" 502 150 2025-12-17 22:50:16,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e55575bc060ce15fdb8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:17,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e53575bc060ce15fdb5 2025-12-17 22:50:17,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e53575bc060ce15fdb5 HTTP/1.1" 502 150 2025-12-17 22:50:17,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e53575bc060ce15fdb5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:17,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e51575bc060ce15fdb2 HTTP/1.1" 502 150 2025-12-17 22:50:17,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e51575bc060ce15fdb2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:18,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3c575bc060ce15fd6b HTTP/1.1" 502 150 2025-12-17 22:50:18,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e3c575bc060ce15fd6b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:18,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e55575bc060ce15fdb8 2025-12-17 22:50:18,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e55575bc060ce15fdb8 HTTP/1.1" 502 150 2025-12-17 22:50:18,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e55575bc060ce15fdb8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:19,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3d575bc060ce15fd6e 2025-12-17 22:50:19,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3d575bc060ce15fd6e HTTP/1.1" 502 150 2025-12-17 22:50:19,601 - lava_callback - ERROR - Error processing callback for node 69408e3d575bc060ce15fd6e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e3d575bc060ce15fd6e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e3d575bc060ce15fd6e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e3d575bc060ce15fd6e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:19,603 - lava_callback - INFO - Starting async job submit for node 69408e56575bc060ce15fdbb 2025-12-17 22:50:19,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:19,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e56575bc060ce15fdbb HTTP/1.1" 502 150 2025-12-17 22:50:19,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e56575bc060ce15fdbb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:19,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e56575bc060ce15fdbb 2025-12-17 22:50:19,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e56575bc060ce15fdbb HTTP/1.1" 502 150 2025-12-17 22:50:19,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e56575bc060ce15fdbb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:19,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3b575bc060ce15fd68 2025-12-17 22:50:19,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3b575bc060ce15fd68 HTTP/1.1" 502 150 2025-12-17 22:50:19,954 - lava_callback - ERROR - Error processing callback for node 69408e3b575bc060ce15fd68: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e3b575bc060ce15fd68 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e3b575bc060ce15fd68 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e3b575bc060ce15fd68 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:19,956 - lava_callback - INFO - Starting async job submit for node 69408e59575bc060ce15fdc6 2025-12-17 22:50:19,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:19,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e59575bc060ce15fdc6 HTTP/1.1" 502 150 2025-12-17 22:50:19,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e59575bc060ce15fdc6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:19,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e59575bc060ce15fdc6 2025-12-17 22:50:19,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e59575bc060ce15fdc6 HTTP/1.1" 502 150 2025-12-17 22:50:19,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e59575bc060ce15fdc6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:20,395 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e45575bc060ce15fd88 2025-12-17 22:50:20,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e45575bc060ce15fd88 HTTP/1.1" 502 150 2025-12-17 22:50:20,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e45575bc060ce15fd88'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:20,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e46575bc060ce15fd95 2025-12-17 22:50:20,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e46575bc060ce15fd95 HTTP/1.1" 502 150 2025-12-17 22:50:20,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e46575bc060ce15fd95'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:21,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e53575bc060ce15fdb5 2025-12-17 22:50:21,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e53575bc060ce15fdb5 HTTP/1.1" 502 150 2025-12-17 22:50:21,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e53575bc060ce15fdb5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:21,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e56575bc060ce15fdbb 2025-12-17 22:50:21,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e56575bc060ce15fdbb HTTP/1.1" 502 150 2025-12-17 22:50:21,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e56575bc060ce15fdbb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:21,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e48575bc060ce15fd9a 2025-12-17 22:50:21,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e48575bc060ce15fd9a HTTP/1.1" 502 150 2025-12-17 22:50:21,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e48575bc060ce15fd9a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:21,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e49575bc060ce15fd9d 2025-12-17 22:50:21,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e49575bc060ce15fd9d HTTP/1.1" 502 150 2025-12-17 22:50:21,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e49575bc060ce15fd9d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:21,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4f575bc060ce15fdac 2025-12-17 22:50:21,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4f575bc060ce15fdac HTTP/1.1" 502 150 2025-12-17 22:50:21,957 - lava_callback - ERROR - Error processing callback for node 69408e4f575bc060ce15fdac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e4f575bc060ce15fdac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e4f575bc060ce15fdac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e4f575bc060ce15fdac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:21,959 - lava_callback - INFO - Starting async job submit for node 69408e5e575bc060ce15fdd2 2025-12-17 22:50:21,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:21,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e59575bc060ce15fdc6 2025-12-17 22:50:21,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e59575bc060ce15fdc6 HTTP/1.1" 502 150 2025-12-17 22:50:21,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e59575bc060ce15fdc6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:21,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5e575bc060ce15fdd2 HTTP/1.1" 502 150 2025-12-17 22:50:21,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5e575bc060ce15fdd2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:21,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5e575bc060ce15fdd2 2025-12-17 22:50:21,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5e575bc060ce15fdd2 HTTP/1.1" 502 150 2025-12-17 22:50:21,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5e575bc060ce15fdd2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:22,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4a575bc060ce15fda0 2025-12-17 22:50:22,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4a575bc060ce15fda0 HTTP/1.1" 502 150 2025-12-17 22:50:22,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4a575bc060ce15fda0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:22,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4b575bc060ce15fda3 2025-12-17 22:50:22,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4b575bc060ce15fda3 HTTP/1.1" 502 150 2025-12-17 22:50:22,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4b575bc060ce15fda3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:22,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e55575bc060ce15fdb8 2025-12-17 22:50:22,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e55575bc060ce15fdb8 HTTP/1.1" 502 150 2025-12-17 22:50:22,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e55575bc060ce15fdb8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:22,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4d575bc060ce15fda6 2025-12-17 22:50:22,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4d575bc060ce15fda6 HTTP/1.1" 502 150 2025-12-17 22:50:22,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e4d575bc060ce15fda6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:23,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5e575bc060ce15fdd2 2025-12-17 22:50:23,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5e575bc060ce15fdd2 HTTP/1.1" 502 150 2025-12-17 22:50:23,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5e575bc060ce15fdd2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:24,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e50575bc060ce15fdaf 2025-12-17 22:50:24,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e50575bc060ce15fdaf HTTP/1.1" 502 150 2025-12-17 22:50:24,808 - lava_callback - ERROR - Error processing callback for node 69408e50575bc060ce15fdaf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e50575bc060ce15fdaf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e50575bc060ce15fdaf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e50575bc060ce15fdaf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:24,810 - lava_callback - INFO - Starting async job submit for node 69408e69575bc060ce15fe03 2025-12-17 22:50:24,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:24,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e69575bc060ce15fe03 HTTP/1.1" 502 150 2025-12-17 22:50:24,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e69575bc060ce15fe03'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:24,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e69575bc060ce15fe03 2025-12-17 22:50:24,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e69575bc060ce15fe03 HTTP/1.1" 502 150 2025-12-17 22:50:24,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e69575bc060ce15fe03'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:25,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e56575bc060ce15fdbb 2025-12-17 22:50:25,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e56575bc060ce15fdbb HTTP/1.1" 502 150 2025-12-17 22:50:25,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e56575bc060ce15fdbb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:25,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e51575bc060ce15fdb2 2025-12-17 22:50:25,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e51575bc060ce15fdb2 HTTP/1.1" 502 150 2025-12-17 22:50:25,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e51575bc060ce15fdb2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:25,730 - lava_callback - INFO - Received callback for node 6940909d575bc060ce1605fd INFO: 172.21.0.1:57568 - "POST /node/6940909d575bc060ce1605fd HTTP/1.0" 202 Accepted 2025-12-17 22:50:25,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e59575bc060ce15fdc6 2025-12-17 22:50:25,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e59575bc060ce15fdc6 HTTP/1.1" 502 150 2025-12-17 22:50:25,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e59575bc060ce15fdc6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:26,374 - lava_callback - INFO - Received callback for node 6940909f575bc060ce160608 INFO: 172.21.0.1:57580 - "POST /node/6940909f575bc060ce160608 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:57590 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:50:26,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e69575bc060ce15fe03 2025-12-17 22:50:26,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e69575bc060ce15fe03 HTTP/1.1" 502 150 2025-12-17 22:50:26,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e69575bc060ce15fe03'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:27,018 - lava_callback - INFO - Received callback for node 694090a5575bc060ce160617 INFO: 172.21.0.1:57606 - "POST /node/694090a5575bc060ce160617 HTTP/1.0" 202 Accepted 2025-12-17 22:50:27,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5e575bc060ce15fdd2 2025-12-17 22:50:27,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5e575bc060ce15fdd2 HTTP/1.1" 502 150 2025-12-17 22:50:27,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5e575bc060ce15fdd2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:29,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e57575bc060ce15fdbe HTTP/1.1" 502 150 2025-12-17 22:50:29,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e57575bc060ce15fdbe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:29,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e53575bc060ce15fdb5 2025-12-17 22:50:29,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e53575bc060ce15fdb5 HTTP/1.1" 502 150 2025-12-17 22:50:29,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e53575bc060ce15fdb5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:30,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e55575bc060ce15fdb8 2025-12-17 22:50:30,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e55575bc060ce15fdb8 HTTP/1.1" 502 150 2025-12-17 22:50:30,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e55575bc060ce15fdb8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:30,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e69575bc060ce15fe03 2025-12-17 22:50:30,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e69575bc060ce15fe03 HTTP/1.1" 502 150 2025-12-17 22:50:30,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e69575bc060ce15fe03'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:33,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e57575bc060ce15fdbe 2025-12-17 22:50:33,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e57575bc060ce15fdbe HTTP/1.1" 502 150 2025-12-17 22:50:33,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e57575bc060ce15fdbe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:33,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e56575bc060ce15fdbb 2025-12-17 22:50:33,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e56575bc060ce15fdbb HTTP/1.1" 502 150 2025-12-17 22:50:33,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e56575bc060ce15fdbb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:33,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e59575bc060ce15fdc6 2025-12-17 22:50:33,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e59575bc060ce15fdc6 HTTP/1.1" 502 150 2025-12-17 22:50:33,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e59575bc060ce15fdc6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:34,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e3c575bc060ce15fd6b 2025-12-17 22:50:34,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e3c575bc060ce15fd6b HTTP/1.1" 502 150 2025-12-17 22:50:34,674 - lava_callback - ERROR - Error processing callback for node 69408e3c575bc060ce15fd6b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e3c575bc060ce15fd6b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e3c575bc060ce15fd6b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e3c575bc060ce15fd6b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:34,676 - lava_callback - INFO - Starting async job submit for node 69408e5a575bc060ce15fdc9 2025-12-17 22:50:34,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:34,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5a575bc060ce15fdc9 HTTP/1.1" 502 150 2025-12-17 22:50:34,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5a575bc060ce15fdc9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:34,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5a575bc060ce15fdc9 2025-12-17 22:50:34,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5a575bc060ce15fdc9 HTTP/1.1" 502 150 2025-12-17 22:50:34,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5a575bc060ce15fdc9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:35,442 - lava_callback - INFO - Received callback for node 694090ab575bc060ce160620 INFO: 172.21.0.1:42048 - "POST /node/694090ab575bc060ce160620 HTTP/1.0" 202 Accepted 2025-12-17 22:50:35,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5e575bc060ce15fdd2 2025-12-17 22:50:35,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5e575bc060ce15fdd2 HTTP/1.1" 502 150 2025-12-17 22:50:35,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5e575bc060ce15fdd2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:36,101 - lava_callback - INFO - Received callback for node 694090ad575bc060ce160623 INFO: 172.21.0.1:42052 - "POST /node/694090ad575bc060ce160623 HTTP/1.0" 202 Accepted 2025-12-17 22:50:36,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e45575bc060ce15fd88 2025-12-17 22:50:36,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e45575bc060ce15fd88 HTTP/1.1" 502 150 2025-12-17 22:50:36,399 - lava_callback - ERROR - Error processing callback for node 69408e45575bc060ce15fd88: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e45575bc060ce15fd88 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e45575bc060ce15fd88 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e45575bc060ce15fd88 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:36,401 - lava_callback - INFO - Starting async job submit for node 69408e5b575bc060ce15fdcc 2025-12-17 22:50:36,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:36,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5b575bc060ce15fdcc HTTP/1.1" 502 150 2025-12-17 22:50:36,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5b575bc060ce15fdcc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:36,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5b575bc060ce15fdcc 2025-12-17 22:50:36,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5b575bc060ce15fdcc HTTP/1.1" 502 150 2025-12-17 22:50:36,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5b575bc060ce15fdcc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42068 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:50:36,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e46575bc060ce15fd95 2025-12-17 22:50:36,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e46575bc060ce15fd95 HTTP/1.1" 502 150 2025-12-17 22:50:36,690 - lava_callback - ERROR - Error processing callback for node 69408e46575bc060ce15fd95: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e46575bc060ce15fd95 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e46575bc060ce15fd95 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e46575bc060ce15fd95 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:36,693 - lava_callback - INFO - Starting async job submit for node 69408e5d575bc060ce15fdcf 2025-12-17 22:50:36,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:36,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5d575bc060ce15fdcf HTTP/1.1" 502 150 2025-12-17 22:50:36,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5d575bc060ce15fdcf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:36,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5d575bc060ce15fdcf 2025-12-17 22:50:36,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5d575bc060ce15fdcf HTTP/1.1" 502 150 2025-12-17 22:50:36,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5d575bc060ce15fdcf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:36,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5a575bc060ce15fdc9 2025-12-17 22:50:36,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5a575bc060ce15fdc9 HTTP/1.1" 502 150 2025-12-17 22:50:36,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5a575bc060ce15fdc9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:36,772 - lava_callback - INFO - Received callback for node 694090af575bc060ce160626 INFO: 172.21.0.1:42078 - "POST /node/694090af575bc060ce160626 HTTP/1.0" 202 Accepted 2025-12-17 22:50:37,455 - lava_callback - INFO - Received callback for node 694090b6575bc060ce16064e INFO: 172.21.0.1:42082 - "POST /node/694090b6575bc060ce16064e HTTP/1.0" 202 Accepted 2025-12-17 22:50:37,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e48575bc060ce15fd9a 2025-12-17 22:50:37,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e48575bc060ce15fd9a HTTP/1.1" 502 150 2025-12-17 22:50:37,638 - lava_callback - ERROR - Error processing callback for node 69408e48575bc060ce15fd9a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e48575bc060ce15fd9a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e48575bc060ce15fd9a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e48575bc060ce15fd9a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:37,641 - lava_callback - INFO - Starting async job submit for node 69408e5f575bc060ce15fdd5 2025-12-17 22:50:37,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:37,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5f575bc060ce15fdd5 HTTP/1.1" 502 150 2025-12-17 22:50:37,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5f575bc060ce15fdd5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:37,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5f575bc060ce15fdd5 2025-12-17 22:50:37,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5f575bc060ce15fdd5 HTTP/1.1" 502 150 2025-12-17 22:50:37,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5f575bc060ce15fdd5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:37,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e49575bc060ce15fd9d 2025-12-17 22:50:37,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e49575bc060ce15fd9d HTTP/1.1" 502 150 2025-12-17 22:50:37,814 - lava_callback - ERROR - Error processing callback for node 69408e49575bc060ce15fd9d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e49575bc060ce15fd9d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e49575bc060ce15fd9d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e49575bc060ce15fd9d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:37,816 - lava_callback - INFO - Starting async job submit for node 69408e60575bc060ce15fdd8 2025-12-17 22:50:37,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:37,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e60575bc060ce15fdd8 HTTP/1.1" 502 150 2025-12-17 22:50:37,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e60575bc060ce15fdd8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:37,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e60575bc060ce15fdd8 2025-12-17 22:50:37,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e60575bc060ce15fdd8 HTTP/1.1" 502 150 2025-12-17 22:50:37,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e60575bc060ce15fdd8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:38,108 - lava_callback - INFO - Received callback for node 694090b8575bc060ce160654 INFO: 172.21.0.1:42084 - "POST /node/694090b8575bc060ce160654 HTTP/1.0" 202 Accepted 2025-12-17 22:50:38,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5b575bc060ce15fdcc 2025-12-17 22:50:38,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5b575bc060ce15fdcc HTTP/1.1" 502 150 2025-12-17 22:50:38,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5b575bc060ce15fdcc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:38,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4a575bc060ce15fda0 2025-12-17 22:50:38,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4a575bc060ce15fda0 HTTP/1.1" 502 150 2025-12-17 22:50:38,612 - lava_callback - ERROR - Error processing callback for node 69408e4a575bc060ce15fda0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e4a575bc060ce15fda0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e4a575bc060ce15fda0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e4a575bc060ce15fda0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:38,614 - lava_callback - INFO - Starting async job submit for node 69408e61575bc060ce15fddb 2025-12-17 22:50:38,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:38,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e61575bc060ce15fddb HTTP/1.1" 502 150 2025-12-17 22:50:38,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e61575bc060ce15fddb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:38,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e61575bc060ce15fddb 2025-12-17 22:50:38,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e61575bc060ce15fddb HTTP/1.1" 502 150 2025-12-17 22:50:38,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e61575bc060ce15fddb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:38,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5d575bc060ce15fdcf 2025-12-17 22:50:38,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5d575bc060ce15fdcf HTTP/1.1" 502 150 2025-12-17 22:50:38,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5d575bc060ce15fdcf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:38,758 - lava_callback - INFO - Received callback for node 694090b9575bc060ce160657 INFO: 172.21.0.1:42098 - "POST /node/694090b9575bc060ce160657 HTTP/1.0" 202 Accepted 2025-12-17 22:50:38,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4b575bc060ce15fda3 2025-12-17 22:50:38,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4b575bc060ce15fda3 HTTP/1.1" 502 150 2025-12-17 22:50:38,772 - lava_callback - ERROR - Error processing callback for node 69408e4b575bc060ce15fda3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e4b575bc060ce15fda3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e4b575bc060ce15fda3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e4b575bc060ce15fda3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:38,774 - lava_callback - INFO - Starting async job submit for node 69408e63575bc060ce15fdde 2025-12-17 22:50:38,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:38,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e63575bc060ce15fdde HTTP/1.1" 502 150 2025-12-17 22:50:38,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e63575bc060ce15fdde'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:38,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e63575bc060ce15fdde 2025-12-17 22:50:38,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e63575bc060ce15fdde HTTP/1.1" 502 150 2025-12-17 22:50:38,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e63575bc060ce15fdde'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:38,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e69575bc060ce15fe03 2025-12-17 22:50:38,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e69575bc060ce15fe03 HTTP/1.1" 502 150 2025-12-17 22:50:38,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e69575bc060ce15fe03'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:38,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e4d575bc060ce15fda6 2025-12-17 22:50:38,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e4d575bc060ce15fda6 HTTP/1.1" 502 150 2025-12-17 22:50:38,961 - lava_callback - ERROR - Error processing callback for node 69408e4d575bc060ce15fda6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e4d575bc060ce15fda6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e4d575bc060ce15fda6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e4d575bc060ce15fda6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:38,963 - lava_callback - INFO - Starting async job submit for node 69408e64575bc060ce15fde1 2025-12-17 22:50:38,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:38,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e64575bc060ce15fde1 HTTP/1.1" 502 150 2025-12-17 22:50:38,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e64575bc060ce15fde1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:38,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e64575bc060ce15fde1 2025-12-17 22:50:38,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e64575bc060ce15fde1 HTTP/1.1" 502 150 2025-12-17 22:50:38,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e64575bc060ce15fde1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:39,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5f575bc060ce15fdd5 2025-12-17 22:50:39,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5f575bc060ce15fdd5 HTTP/1.1" 502 150 2025-12-17 22:50:39,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5f575bc060ce15fdd5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:39,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e60575bc060ce15fdd8 2025-12-17 22:50:39,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e60575bc060ce15fdd8 HTTP/1.1" 502 150 2025-12-17 22:50:39,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e60575bc060ce15fdd8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:40,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e61575bc060ce15fddb 2025-12-17 22:50:40,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e61575bc060ce15fddb HTTP/1.1" 502 150 2025-12-17 22:50:40,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e61575bc060ce15fddb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:40,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5a575bc060ce15fdc9 2025-12-17 22:50:40,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5a575bc060ce15fdc9 HTTP/1.1" 502 150 2025-12-17 22:50:40,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5a575bc060ce15fdc9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:40,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e63575bc060ce15fdde 2025-12-17 22:50:40,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e63575bc060ce15fdde HTTP/1.1" 502 150 2025-12-17 22:50:40,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e63575bc060ce15fdde'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:40,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e64575bc060ce15fde1 2025-12-17 22:50:40,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e64575bc060ce15fde1 HTTP/1.1" 502 150 2025-12-17 22:50:40,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e64575bc060ce15fde1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:41,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e57575bc060ce15fdbe 2025-12-17 22:50:41,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e57575bc060ce15fdbe HTTP/1.1" 502 150 2025-12-17 22:50:41,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e57575bc060ce15fdbe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:41,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e51575bc060ce15fdb2 2025-12-17 22:50:41,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e51575bc060ce15fdb2 HTTP/1.1" 502 150 2025-12-17 22:50:41,651 - lava_callback - ERROR - Error processing callback for node 69408e51575bc060ce15fdb2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e51575bc060ce15fdb2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e51575bc060ce15fdb2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e51575bc060ce15fdb2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:41,653 - lava_callback - INFO - Starting async job submit for node 69408e65575bc060ce15fde4 2025-12-17 22:50:41,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:41,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e65575bc060ce15fde4 HTTP/1.1" 502 150 2025-12-17 22:50:41,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e65575bc060ce15fde4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:41,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e65575bc060ce15fde4 2025-12-17 22:50:41,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e65575bc060ce15fde4 HTTP/1.1" 502 150 2025-12-17 22:50:41,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e65575bc060ce15fde4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:42,427 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5b575bc060ce15fdcc 2025-12-17 22:50:42,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5b575bc060ce15fdcc HTTP/1.1" 502 150 2025-12-17 22:50:42,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5b575bc060ce15fdcc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:42,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5d575bc060ce15fdcf 2025-12-17 22:50:42,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5d575bc060ce15fdcf HTTP/1.1" 502 150 2025-12-17 22:50:42,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5d575bc060ce15fdcf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:43,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5f575bc060ce15fdd5 2025-12-17 22:50:43,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e65575bc060ce15fde4 2025-12-17 22:50:43,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e65575bc060ce15fde4 HTTP/1.1" 502 150 2025-12-17 22:50:43,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e65575bc060ce15fde4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:43,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e60575bc060ce15fdd8 2025-12-17 22:50:43,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e60575bc060ce15fdd8 HTTP/1.1" 502 150 2025-12-17 22:50:43,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e60575bc060ce15fdd8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:44,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e61575bc060ce15fddb 2025-12-17 22:50:44,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e61575bc060ce15fddb HTTP/1.1" 502 150 2025-12-17 22:50:44,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e61575bc060ce15fddb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:44,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e63575bc060ce15fdde 2025-12-17 22:50:44,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e63575bc060ce15fdde HTTP/1.1" 502 150 2025-12-17 22:50:44,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e63575bc060ce15fdde'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:44,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e64575bc060ce15fde1 2025-12-17 22:50:44,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e64575bc060ce15fde1 HTTP/1.1" 502 150 2025-12-17 22:50:44,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e64575bc060ce15fde1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:45,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e53575bc060ce15fdb5 2025-12-17 22:50:45,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e53575bc060ce15fdb5 HTTP/1.1" 502 150 2025-12-17 22:50:45,521 - lava_callback - ERROR - Error processing callback for node 69408e53575bc060ce15fdb5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e53575bc060ce15fdb5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e53575bc060ce15fdb5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e53575bc060ce15fdb5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:45,524 - lava_callback - INFO - Starting async job submit for node 69408e66575bc060ce15fde7 2025-12-17 22:50:45,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:45,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e66575bc060ce15fde7 HTTP/1.1" 502 150 2025-12-17 22:50:45,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e66575bc060ce15fde7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:45,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e66575bc060ce15fde7 2025-12-17 22:50:45,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e66575bc060ce15fde7 HTTP/1.1" 502 150 2025-12-17 22:50:45,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e66575bc060ce15fde7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:45,782 - lava_callback - INFO - Received callback for node 694090b1575bc060ce16062d INFO: 172.21.0.1:41712 - "POST /node/694090b1575bc060ce16062d HTTP/1.0" 202 Accepted 2025-12-17 22:50:46,446 - lava_callback - INFO - Received callback for node 694090b4575bc060ce160649 INFO: 172.21.0.1:41716 - "POST /node/694090b4575bc060ce160649 HTTP/1.0" 202 Accepted 2025-12-17 22:50:46,501 - lava_callback - INFO - Received callback for node 694090b4575bc060ce160644 INFO: 172.21.0.1:41728 - "POST /node/694090b4575bc060ce160644 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:41730 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:50:46,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e55575bc060ce15fdb8 2025-12-17 22:50:46,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e55575bc060ce15fdb8 HTTP/1.1" 502 150 2025-12-17 22:50:46,840 - lava_callback - ERROR - Error processing callback for node 69408e55575bc060ce15fdb8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e55575bc060ce15fdb8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e55575bc060ce15fdb8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e55575bc060ce15fdb8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:46,842 - lava_callback - INFO - Starting async job submit for node 69408e68575bc060ce15fdea 2025-12-17 22:50:46,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:46,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e68575bc060ce15fdea HTTP/1.1" 502 150 2025-12-17 22:50:46,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e68575bc060ce15fdea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:46,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e68575bc060ce15fdea 2025-12-17 22:50:46,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e68575bc060ce15fdea HTTP/1.1" 502 150 2025-12-17 22:50:46,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e68575bc060ce15fdea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:47,111 - lava_callback - INFO - Received callback for node 694090bf575bc060ce160667 INFO: 172.21.0.1:41734 - "POST /node/694090bf575bc060ce160667 HTTP/1.0" 202 Accepted 2025-12-17 22:50:47,325 - lava_callback - INFO - Received callback for node 694090b7575bc060ce160651 INFO: 172.21.0.1:41740 - "POST /node/694090b7575bc060ce160651 HTTP/1.0" 202 Accepted 2025-12-17 22:50:47,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e66575bc060ce15fde7 2025-12-17 22:50:47,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e66575bc060ce15fde7 HTTP/1.1" 502 150 2025-12-17 22:50:47,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e66575bc060ce15fde7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:47,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e65575bc060ce15fde4 2025-12-17 22:50:47,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e65575bc060ce15fde4 HTTP/1.1" 502 150 2025-12-17 22:50:47,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e65575bc060ce15fde4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:48,133 - lava_callback - INFO - Received callback for node 694090c2575bc060ce16066a INFO: 172.21.0.1:41742 - "POST /node/694090c2575bc060ce16066a HTTP/1.0" 202 Accepted 2025-12-17 22:50:48,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5a575bc060ce15fdc9 2025-12-17 22:50:48,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5a575bc060ce15fdc9 HTTP/1.1" 502 150 2025-12-17 22:50:48,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5a575bc060ce15fdc9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:48,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e68575bc060ce15fdea 2025-12-17 22:50:48,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e68575bc060ce15fdea HTTP/1.1" 502 150 2025-12-17 22:50:48,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e68575bc060ce15fdea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:48,915 - lava_callback - INFO - Received callback for node 694090c4575bc060ce160674 INFO: 172.21.0.1:41748 - "POST /node/694090c4575bc060ce160674 HTTP/1.0" 202 Accepted 2025-12-17 22:50:49,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e56575bc060ce15fdbb 2025-12-17 22:50:49,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e56575bc060ce15fdbb HTTP/1.1" 502 150 2025-12-17 22:50:49,617 - lava_callback - ERROR - Error processing callback for node 69408e56575bc060ce15fdbb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e56575bc060ce15fdbb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e56575bc060ce15fdbb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e56575bc060ce15fdbb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:49,618 - lava_callback - INFO - Starting async job submit for node 69408e6b575bc060ce15fe0c 2025-12-17 22:50:49,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:49,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6b575bc060ce15fe0c HTTP/1.1" 502 150 2025-12-17 22:50:49,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e6b575bc060ce15fe0c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:49,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e6b575bc060ce15fe0c 2025-12-17 22:50:49,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6b575bc060ce15fe0c HTTP/1.1" 502 150 2025-12-17 22:50:49,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e6b575bc060ce15fe0c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:49,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e59575bc060ce15fdc6 2025-12-17 22:50:49,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e59575bc060ce15fdc6 HTTP/1.1" 502 150 2025-12-17 22:50:49,981 - lava_callback - ERROR - Error processing callback for node 69408e59575bc060ce15fdc6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e59575bc060ce15fdc6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e59575bc060ce15fdc6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e59575bc060ce15fdc6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:49,984 - lava_callback - INFO - Starting async job submit for node 69408e6d575bc060ce15fe13 2025-12-17 22:50:49,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:49,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6d575bc060ce15fe13 HTTP/1.1" 502 150 2025-12-17 22:50:49,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e6d575bc060ce15fe13'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:49,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e6d575bc060ce15fe13 2025-12-17 22:50:49,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6d575bc060ce15fe13 HTTP/1.1" 502 150 2025-12-17 22:50:49,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e6d575bc060ce15fe13'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:50,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5b575bc060ce15fdcc 2025-12-17 22:50:50,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5b575bc060ce15fdcc HTTP/1.1" 502 150 2025-12-17 22:50:50,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5b575bc060ce15fdcc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:50,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5d575bc060ce15fdcf 2025-12-17 22:50:50,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5d575bc060ce15fdcf HTTP/1.1" 502 150 2025-12-17 22:50:50,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5d575bc060ce15fdcf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:51,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e66575bc060ce15fde7 2025-12-17 22:50:51,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e66575bc060ce15fde7 HTTP/1.1" 502 150 2025-12-17 22:50:51,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e66575bc060ce15fde7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:51,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e6b575bc060ce15fe0c 2025-12-17 22:50:51,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6b575bc060ce15fe0c HTTP/1.1" 502 150 2025-12-17 22:50:51,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e6b575bc060ce15fe0c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:51,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e60575bc060ce15fdd8 2025-12-17 22:50:51,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e60575bc060ce15fdd8 HTTP/1.1" 502 150 2025-12-17 22:50:51,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e60575bc060ce15fdd8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:51,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5e575bc060ce15fdd2 2025-12-17 22:50:51,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5e575bc060ce15fdd2 HTTP/1.1" 502 150 2025-12-17 22:50:51,994 - lava_callback - ERROR - Error processing callback for node 69408e5e575bc060ce15fdd2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e5e575bc060ce15fdd2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e5e575bc060ce15fdd2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e5e575bc060ce15fdd2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:51,995 - lava_callback - INFO - Starting async job submit for node 69408e6e575bc060ce15fe19 2025-12-17 22:50:51,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:52,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e6d575bc060ce15fe13 2025-12-17 22:50:52,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6e575bc060ce15fe19 HTTP/1.1" 502 150 2025-12-17 22:50:52,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e6e575bc060ce15fe19'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:52,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e6e575bc060ce15fe19 2025-12-17 22:50:52,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6d575bc060ce15fe13 HTTP/1.1" 502 150 2025-12-17 22:50:52,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e6d575bc060ce15fe13'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:52,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6e575bc060ce15fe19 HTTP/1.1" 502 150 2025-12-17 22:50:52,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e6e575bc060ce15fe19'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:52,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e61575bc060ce15fddb 2025-12-17 22:50:52,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e61575bc060ce15fddb HTTP/1.1" 502 150 2025-12-17 22:50:52,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e61575bc060ce15fddb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:52,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e63575bc060ce15fdde 2025-12-17 22:50:52,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e63575bc060ce15fdde HTTP/1.1" 502 150 2025-12-17 22:50:52,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e63575bc060ce15fdde'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:52,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e68575bc060ce15fdea 2025-12-17 22:50:52,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e68575bc060ce15fdea HTTP/1.1" 502 150 2025-12-17 22:50:52,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e68575bc060ce15fdea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:52,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e64575bc060ce15fde1 2025-12-17 22:50:52,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e64575bc060ce15fde1 HTTP/1.1" 502 150 2025-12-17 22:50:52,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e64575bc060ce15fde1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:54,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e6e575bc060ce15fe19 2025-12-17 22:50:54,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e69575bc060ce15fe03 2025-12-17 22:50:54,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e69575bc060ce15fe03 HTTP/1.1" 502 150 2025-12-17 22:50:54,842 - lava_callback - ERROR - Error processing callback for node 69408e69575bc060ce15fe03: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e69575bc060ce15fe03 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e69575bc060ce15fe03 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e69575bc060ce15fe03 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:54,844 - lava_callback - INFO - Starting async job submit for node 69408e70575bc060ce15fe1c 2025-12-17 22:50:54,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:54,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e70575bc060ce15fe1c HTTP/1.1" 502 150 2025-12-17 22:50:54,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e70575bc060ce15fe1c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:54,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e70575bc060ce15fe1c 2025-12-17 22:50:54,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e70575bc060ce15fe1c HTTP/1.1" 502 150 2025-12-17 22:50:54,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e70575bc060ce15fe1c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:55,070 - lava_callback - INFO - Received callback for node 694092ff575bc060ce160bfc INFO: 172.21.0.1:59180 - "POST /node/694092ff575bc060ce160bfc HTTP/1.0" 202 Accepted 2025-12-17 22:50:55,410 - lava_callback - INFO - Received callback for node 694090bb575bc060ce16065a INFO: 172.21.0.1:59182 - "POST /node/694090bb575bc060ce16065a HTTP/1.0" 202 Accepted 2025-12-17 22:50:55,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e6b575bc060ce15fe0c 2025-12-17 22:50:55,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e65575bc060ce15fde4 2025-12-17 22:50:55,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e65575bc060ce15fde4 HTTP/1.1" 502 150 2025-12-17 22:50:55,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e65575bc060ce15fde4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:56,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e6d575bc060ce15fe13 2025-12-17 22:50:56,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6d575bc060ce15fe13 HTTP/1.1" 502 150 2025-12-17 22:50:56,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e6d575bc060ce15fe13'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:56,079 - lava_callback - INFO - Received callback for node 694090bc575bc060ce16065d INFO: 172.21.0.1:59190 - "POST /node/694090bc575bc060ce16065d HTTP/1.0" 202 Accepted INFO: 172.21.0.1:59206 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:50:56,764 - lava_callback - INFO - Received callback for node 694090be575bc060ce160664 INFO: 172.21.0.1:59216 - "POST /node/694090be575bc060ce160664 HTTP/1.0" 202 Accepted 2025-12-17 22:50:56,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e70575bc060ce15fe1c 2025-12-17 22:50:56,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e70575bc060ce15fe1c HTTP/1.1" 502 150 2025-12-17 22:50:56,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e70575bc060ce15fe1c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:57,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e57575bc060ce15fdbe 2025-12-17 22:50:57,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e57575bc060ce15fdbe HTTP/1.1" 502 150 2025-12-17 22:50:57,429 - lava_callback - ERROR - Error processing callback for node 69408e57575bc060ce15fdbe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e57575bc060ce15fdbe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e57575bc060ce15fdbe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e57575bc060ce15fdbe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:50:57,431 - lava_callback - INFO - Starting async job submit for node 69408e71575bc060ce15fe1f 2025-12-17 22:50:57,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:50:57,439 - lava_callback - INFO - Received callback for node 694090c3575bc060ce16066d INFO: 172.21.0.1:59226 - "POST /node/694090c3575bc060ce16066d HTTP/1.0" 202 Accepted 2025-12-17 22:50:57,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e71575bc060ce15fe1f HTTP/1.1" 502 150 2025-12-17 22:50:57,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e71575bc060ce15fe1f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:57,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e71575bc060ce15fe1f 2025-12-17 22:50:57,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e71575bc060ce15fe1f HTTP/1.1" 502 150 2025-12-17 22:50:57,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e71575bc060ce15fe1f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:58,121 - lava_callback - INFO - Received callback for node 694090c5575bc060ce16067b INFO: 172.21.0.1:59242 - "POST /node/694090c5575bc060ce16067b HTTP/1.0" 202 Accepted 2025-12-17 22:50:58,795 - lava_callback - INFO - Received callback for node 694090c7575bc060ce160692 INFO: 172.21.0.1:59258 - "POST /node/694090c7575bc060ce160692 HTTP/1.0" 202 Accepted 2025-12-17 22:50:59,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5f575bc060ce15fdd5 HTTP/1.1" 502 150 2025-12-17 22:50:59,120 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5f575bc060ce15fdd5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:59,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e71575bc060ce15fe1f 2025-12-17 22:50:59,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e71575bc060ce15fe1f HTTP/1.1" 502 150 2025-12-17 22:50:59,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e71575bc060ce15fe1f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:50:59,466 - lava_callback - INFO - Received callback for node 694090c9575bc060ce1606b0 INFO: 172.21.0.1:59266 - "POST /node/694090c9575bc060ce1606b0 HTTP/1.0" 202 Accepted 2025-12-17 22:50:59,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e66575bc060ce15fde7 2025-12-17 22:50:59,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e66575bc060ce15fde7 HTTP/1.1" 502 150 2025-12-17 22:50:59,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e66575bc060ce15fde7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:00,141 - lava_callback - INFO - Received callback for node 694090cb575bc060ce1606ba INFO: 172.21.0.1:52562 - "POST /node/694090cb575bc060ce1606ba HTTP/1.0" 202 Accepted 2025-12-17 22:51:00,801 - lava_callback - INFO - Received callback for node 694090cd575bc060ce1606bd INFO: 172.21.0.1:52572 - "POST /node/694090cd575bc060ce1606bd HTTP/1.0" 202 Accepted 2025-12-17 22:51:00,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e68575bc060ce15fdea 2025-12-17 22:51:00,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e68575bc060ce15fdea HTTP/1.1" 502 150 2025-12-17 22:51:00,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e68575bc060ce15fdea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:00,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e70575bc060ce15fe1c 2025-12-17 22:51:00,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e70575bc060ce15fe1c HTTP/1.1" 502 150 2025-12-17 22:51:00,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e70575bc060ce15fe1c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:01,458 - lava_callback - INFO - Received callback for node 694090ce575bc060ce1606c0 INFO: 172.21.0.1:52574 - "POST /node/694090ce575bc060ce1606c0 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:52582 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:51:03,447 - lava_callback - INFO - Received callback for node 69408e99575bc060ce15fe96 INFO: 172.21.0.1:52586 - "POST /node/69408e99575bc060ce15fe96 HTTP/1.0" 202 Accepted 2025-12-17 22:51:03,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e71575bc060ce15fe1f 2025-12-17 22:51:03,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e71575bc060ce15fe1f HTTP/1.1" 502 150 2025-12-17 22:51:03,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e71575bc060ce15fe1f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:04,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e6d575bc060ce15fe13 2025-12-17 22:51:04,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6d575bc060ce15fe13 HTTP/1.1" 502 150 2025-12-17 22:51:04,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e6d575bc060ce15fe13'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:04,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5a575bc060ce15fdc9 2025-12-17 22:51:04,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5a575bc060ce15fdc9 HTTP/1.1" 502 150 2025-12-17 22:51:04,718 - lava_callback - ERROR - Error processing callback for node 69408e5a575bc060ce15fdc9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e5a575bc060ce15fdc9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e5a575bc060ce15fdc9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e5a575bc060ce15fdc9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:04,721 - lava_callback - INFO - Starting async job submit for node 69408e72575bc060ce15fe22 2025-12-17 22:51:04,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:04,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e72575bc060ce15fe22 HTTP/1.1" 502 150 2025-12-17 22:51:04,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e72575bc060ce15fe22'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:04,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e72575bc060ce15fe22 2025-12-17 22:51:04,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e72575bc060ce15fe22 HTTP/1.1" 502 150 2025-12-17 22:51:04,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e72575bc060ce15fe22'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:05,083 - lava_callback - INFO - Received callback for node 694091a6575bc060ce1608ec INFO: 172.21.0.1:52588 - "POST /node/694091a6575bc060ce1608ec HTTP/1.0" 202 Accepted 2025-12-17 22:51:06,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5b575bc060ce15fdcc 2025-12-17 22:51:06,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5b575bc060ce15fdcc HTTP/1.1" 502 150 2025-12-17 22:51:06,431 - lava_callback - ERROR - Error processing callback for node 69408e5b575bc060ce15fdcc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e5b575bc060ce15fdcc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e5b575bc060ce15fdcc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e5b575bc060ce15fdcc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:06,433 - lava_callback - INFO - Starting async job submit for node 69408e74575bc060ce15fe25 2025-12-17 22:51:06,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:06,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e74575bc060ce15fe25 HTTP/1.1" 502 150 2025-12-17 22:51:06,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e74575bc060ce15fe25'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:06,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e74575bc060ce15fe25 2025-12-17 22:51:06,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e74575bc060ce15fe25 HTTP/1.1" 502 150 2025-12-17 22:51:06,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e74575bc060ce15fe25'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52590 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:51:06,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5d575bc060ce15fdcf 2025-12-17 22:51:06,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5d575bc060ce15fdcf HTTP/1.1" 502 150 2025-12-17 22:51:06,711 - lava_callback - ERROR - Error processing callback for node 69408e5d575bc060ce15fdcf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e5d575bc060ce15fdcf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e5d575bc060ce15fdcf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e5d575bc060ce15fdcf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:06,713 - lava_callback - INFO - Starting async job submit for node 69408e75575bc060ce15fe28 2025-12-17 22:51:06,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:06,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e75575bc060ce15fe28 HTTP/1.1" 502 150 2025-12-17 22:51:06,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e75575bc060ce15fe28'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:06,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e75575bc060ce15fe28 2025-12-17 22:51:06,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e75575bc060ce15fe28 HTTP/1.1" 502 150 2025-12-17 22:51:06,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e75575bc060ce15fe28'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:06,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e72575bc060ce15fe22 2025-12-17 22:51:06,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e72575bc060ce15fe22 HTTP/1.1" 502 150 2025-12-17 22:51:06,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e72575bc060ce15fe22'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:06,832 - lava_callback - INFO - Received callback for node 694090cb575bc060ce1606b7 INFO: 172.21.0.1:52594 - "POST /node/694090cb575bc060ce1606b7 HTTP/1.0" 202 Accepted 2025-12-17 22:51:07,121 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5f575bc060ce15fdd5 2025-12-17 22:51:07,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5f575bc060ce15fdd5 HTTP/1.1" 502 150 2025-12-17 22:51:07,122 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e5f575bc060ce15fdd5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:07,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e60575bc060ce15fdd8 2025-12-17 22:51:07,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e60575bc060ce15fdd8 HTTP/1.1" 502 150 2025-12-17 22:51:07,830 - lava_callback - ERROR - Error processing callback for node 69408e60575bc060ce15fdd8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e60575bc060ce15fdd8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e60575bc060ce15fdd8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e60575bc060ce15fdd8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:07,833 - lava_callback - INFO - Starting async job submit for node 69408e98575bc060ce15fe91 2025-12-17 22:51:07,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:07,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e98575bc060ce15fe91 HTTP/1.1" 502 150 2025-12-17 22:51:07,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e98575bc060ce15fe91'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:07,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e98575bc060ce15fe91 2025-12-17 22:51:07,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e98575bc060ce15fe91 HTTP/1.1" 502 150 2025-12-17 22:51:07,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e98575bc060ce15fe91'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:08,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e74575bc060ce15fe25 2025-12-17 22:51:08,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e74575bc060ce15fe25 HTTP/1.1" 502 150 2025-12-17 22:51:08,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e74575bc060ce15fe25'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:08,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e61575bc060ce15fddb 2025-12-17 22:51:08,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e61575bc060ce15fddb HTTP/1.1" 502 150 2025-12-17 22:51:08,627 - lava_callback - ERROR - Error processing callback for node 69408e61575bc060ce15fddb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e61575bc060ce15fddb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e61575bc060ce15fddb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e61575bc060ce15fddb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:08,630 - lava_callback - INFO - Starting async job submit for node 69408e7b575bc060ce15fe34 2025-12-17 22:51:08,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:08,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7b575bc060ce15fe34 HTTP/1.1" 502 150 2025-12-17 22:51:08,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7b575bc060ce15fe34'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:08,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7b575bc060ce15fe34 2025-12-17 22:51:08,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7b575bc060ce15fe34 HTTP/1.1" 502 150 2025-12-17 22:51:08,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7b575bc060ce15fe34'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:08,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e75575bc060ce15fe28 2025-12-17 22:51:08,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e75575bc060ce15fe28 HTTP/1.1" 502 150 2025-12-17 22:51:08,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e75575bc060ce15fe28'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:08,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e63575bc060ce15fdde 2025-12-17 22:51:08,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e63575bc060ce15fdde HTTP/1.1" 502 150 2025-12-17 22:51:08,792 - lava_callback - ERROR - Error processing callback for node 69408e63575bc060ce15fdde: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e63575bc060ce15fdde (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e63575bc060ce15fdde (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e63575bc060ce15fdde (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:08,798 - lava_callback - INFO - Starting async job submit for node 69408e7c575bc060ce15fe37 2025-12-17 22:51:08,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:08,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7c575bc060ce15fe37 HTTP/1.1" 502 150 2025-12-17 22:51:08,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7c575bc060ce15fe37'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:08,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7c575bc060ce15fe37 2025-12-17 22:51:08,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7c575bc060ce15fe37 HTTP/1.1" 502 150 2025-12-17 22:51:08,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7c575bc060ce15fe37'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:08,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e70575bc060ce15fe1c 2025-12-17 22:51:08,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e70575bc060ce15fe1c HTTP/1.1" 502 150 2025-12-17 22:51:08,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e70575bc060ce15fe1c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:08,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e64575bc060ce15fde1 2025-12-17 22:51:08,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e64575bc060ce15fde1 HTTP/1.1" 502 150 2025-12-17 22:51:08,976 - lava_callback - ERROR - Error processing callback for node 69408e64575bc060ce15fde1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e64575bc060ce15fde1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e64575bc060ce15fde1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e64575bc060ce15fde1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:08,979 - lava_callback - INFO - Starting async job submit for node 69408e7d575bc060ce15fe3d 2025-12-17 22:51:08,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:09,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7d575bc060ce15fe3d HTTP/1.1" 502 150 2025-12-17 22:51:09,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7d575bc060ce15fe3d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:09,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7d575bc060ce15fe3d 2025-12-17 22:51:09,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7d575bc060ce15fe3d HTTP/1.1" 502 150 2025-12-17 22:51:09,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7d575bc060ce15fe3d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:09,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6e575bc060ce15fe19 HTTP/1.1" 502 150 2025-12-17 22:51:09,364 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e6e575bc060ce15fe19'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:09,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e98575bc060ce15fe91 2025-12-17 22:51:09,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e98575bc060ce15fe91 HTTP/1.1" 502 150 2025-12-17 22:51:09,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e98575bc060ce15fe91'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:10,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7b575bc060ce15fe34 2025-12-17 22:51:10,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7b575bc060ce15fe34 HTTP/1.1" 502 150 2025-12-17 22:51:10,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7b575bc060ce15fe34'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:10,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e72575bc060ce15fe22 2025-12-17 22:51:10,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e72575bc060ce15fe22 HTTP/1.1" 502 150 2025-12-17 22:51:10,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e72575bc060ce15fe22'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:10,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7c575bc060ce15fe37 2025-12-17 22:51:10,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7c575bc060ce15fe37 HTTP/1.1" 502 150 2025-12-17 22:51:10,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7c575bc060ce15fe37'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:10,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6b575bc060ce15fe0c HTTP/1.1" 502 150 2025-12-17 22:51:10,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e6b575bc060ce15fe0c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:11,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7d575bc060ce15fe3d 2025-12-17 22:51:11,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7d575bc060ce15fe3d HTTP/1.1" 502 150 2025-12-17 22:51:11,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7d575bc060ce15fe3d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:11,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e71575bc060ce15fe1f 2025-12-17 22:51:11,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e71575bc060ce15fe1f HTTP/1.1" 502 150 2025-12-17 22:51:11,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e71575bc060ce15fe1f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:11,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e65575bc060ce15fde4 2025-12-17 22:51:11,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e65575bc060ce15fde4 HTTP/1.1" 502 150 2025-12-17 22:51:11,666 - lava_callback - ERROR - Error processing callback for node 69408e65575bc060ce15fde4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e65575bc060ce15fde4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e65575bc060ce15fde4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e65575bc060ce15fde4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:11,668 - lava_callback - INFO - Starting async job submit for node 69408e7e575bc060ce15fe40 2025-12-17 22:51:11,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:11,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7e575bc060ce15fe40 HTTP/1.1" 502 150 2025-12-17 22:51:11,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7e575bc060ce15fe40'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:11,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7e575bc060ce15fe40 2025-12-17 22:51:11,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7e575bc060ce15fe40 HTTP/1.1" 502 150 2025-12-17 22:51:11,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7e575bc060ce15fe40'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:12,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e74575bc060ce15fe25 2025-12-17 22:51:12,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e74575bc060ce15fe25 HTTP/1.1" 502 150 2025-12-17 22:51:12,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e74575bc060ce15fe25'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:12,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e75575bc060ce15fe28 2025-12-17 22:51:12,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e75575bc060ce15fe28 HTTP/1.1" 502 150 2025-12-17 22:51:12,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e75575bc060ce15fe28'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:13,364 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e6e575bc060ce15fe19 2025-12-17 22:51:13,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6e575bc060ce15fe19 HTTP/1.1" 502 150 2025-12-17 22:51:13,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e6e575bc060ce15fe19'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:13,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7e575bc060ce15fe40 2025-12-17 22:51:13,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7e575bc060ce15fe40 HTTP/1.1" 502 150 2025-12-17 22:51:13,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7e575bc060ce15fe40'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:13,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e98575bc060ce15fe91 2025-12-17 22:51:13,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e98575bc060ce15fe91 HTTP/1.1" 502 150 2025-12-17 22:51:13,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e98575bc060ce15fe91'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:14,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7b575bc060ce15fe34 2025-12-17 22:51:14,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7b575bc060ce15fe34 HTTP/1.1" 502 150 2025-12-17 22:51:14,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7b575bc060ce15fe34'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:14,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7c575bc060ce15fe37 2025-12-17 22:51:14,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7c575bc060ce15fe37 HTTP/1.1" 502 150 2025-12-17 22:51:14,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7c575bc060ce15fe37'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:15,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7d575bc060ce15fe3d 2025-12-17 22:51:15,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7d575bc060ce15fe3d HTTP/1.1" 502 150 2025-12-17 22:51:15,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7d575bc060ce15fe3d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41384 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:51:15,416 - lava_callback - INFO - Received callback for node 694090cf575bc060ce1606c3 INFO: 172.21.0.1:41398 - "POST /node/694090cf575bc060ce1606c3 HTTP/1.0" 202 Accepted 2025-12-17 22:51:15,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e66575bc060ce15fde7 2025-12-17 22:51:15,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e66575bc060ce15fde7 HTTP/1.1" 502 150 2025-12-17 22:51:15,546 - lava_callback - ERROR - Error processing callback for node 69408e66575bc060ce15fde7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e66575bc060ce15fde7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e66575bc060ce15fde7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e66575bc060ce15fde7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:15,548 - lava_callback - INFO - Starting async job submit for node 69408e76575bc060ce15fe2b 2025-12-17 22:51:15,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:15,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e76575bc060ce15fe2b HTTP/1.1" 502 150 2025-12-17 22:51:15,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e76575bc060ce15fe2b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:15,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e76575bc060ce15fe2b 2025-12-17 22:51:15,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e76575bc060ce15fe2b HTTP/1.1" 502 150 2025-12-17 22:51:15,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e76575bc060ce15fe2b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:16,088 - lava_callback - INFO - Received callback for node 694090d0575bc060ce1606c6 INFO: 172.21.0.1:41406 - "POST /node/694090d0575bc060ce1606c6 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:41416 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:51:16,754 - lava_callback - INFO - Received callback for node 694090d2575bc060ce1606c9 INFO: 172.21.0.1:41418 - "POST /node/694090d2575bc060ce1606c9 HTTP/1.0" 202 Accepted 2025-12-17 22:51:16,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e68575bc060ce15fdea 2025-12-17 22:51:16,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e68575bc060ce15fdea HTTP/1.1" 502 150 2025-12-17 22:51:16,854 - lava_callback - ERROR - Error processing callback for node 69408e68575bc060ce15fdea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e68575bc060ce15fdea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e68575bc060ce15fdea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e68575bc060ce15fdea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:16,856 - lava_callback - INFO - Starting async job submit for node 69408e77575bc060ce15fe2e 2025-12-17 22:51:16,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:16,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e77575bc060ce15fe2e HTTP/1.1" 502 150 2025-12-17 22:51:16,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e77575bc060ce15fe2e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:16,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e77575bc060ce15fe2e 2025-12-17 22:51:16,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e77575bc060ce15fe2e HTTP/1.1" 502 150 2025-12-17 22:51:16,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e77575bc060ce15fe2e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:17,434 - lava_callback - INFO - Received callback for node 694090d3575bc060ce1606cc INFO: 172.21.0.1:41434 - "POST /node/694090d3575bc060ce1606cc HTTP/1.0" 202 Accepted 2025-12-17 22:51:17,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e76575bc060ce15fe2b 2025-12-17 22:51:17,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e76575bc060ce15fe2b HTTP/1.1" 502 150 2025-12-17 22:51:17,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e76575bc060ce15fe2b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:17,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7e575bc060ce15fe40 2025-12-17 22:51:17,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7e575bc060ce15fe40 HTTP/1.1" 502 150 2025-12-17 22:51:17,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7e575bc060ce15fe40'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:18,103 - lava_callback - INFO - Received callback for node 694090d4575bc060ce1606cf INFO: 172.21.0.1:41450 - "POST /node/694090d4575bc060ce1606cf HTTP/1.0" 202 Accepted 2025-12-17 22:51:18,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e72575bc060ce15fe22 2025-12-17 22:51:18,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e72575bc060ce15fe22 HTTP/1.1" 502 150 2025-12-17 22:51:18,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e72575bc060ce15fe22'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:18,768 - lava_callback - INFO - Received callback for node 694090d5575bc060ce1606d2 INFO: 172.21.0.1:41460 - "POST /node/694090d5575bc060ce1606d2 HTTP/1.0" 202 Accepted 2025-12-17 22:51:18,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e77575bc060ce15fe2e 2025-12-17 22:51:18,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e77575bc060ce15fe2e HTTP/1.1" 502 150 2025-12-17 22:51:18,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e77575bc060ce15fe2e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:18,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e6b575bc060ce15fe0c 2025-12-17 22:51:18,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6b575bc060ce15fe0c HTTP/1.1" 502 150 2025-12-17 22:51:18,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e6b575bc060ce15fe0c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:19,413 - lava_callback - INFO - Received callback for node 6940c53e1e3c24e95681d6b0 INFO: 172.21.0.1:41462 - "POST /node/6940c53e1e3c24e95681d6b0 HTTP/1.0" 202 Accepted 2025-12-17 22:51:19,458 - lava_callback - INFO - Received callback for node 694090d7575bc060ce1606d5 INFO: 172.21.0.1:41464 - "POST /node/694090d7575bc060ce1606d5 HTTP/1.0" 202 Accepted 2025-12-17 22:51:20,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e6d575bc060ce15fe13 2025-12-17 22:51:20,134 - lava_callback - INFO - Received callback for node 694090d9575bc060ce1606d8 INFO: 172.21.0.1:43792 - "POST /node/694090d9575bc060ce1606d8 HTTP/1.0" 202 Accepted 2025-12-17 22:51:20,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e74575bc060ce15fe25 2025-12-17 22:51:20,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e74575bc060ce15fe25 HTTP/1.1" 502 150 2025-12-17 22:51:20,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e74575bc060ce15fe25'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:20,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e75575bc060ce15fe28 2025-12-17 22:51:20,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e75575bc060ce15fe28 HTTP/1.1" 502 150 2025-12-17 22:51:20,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e75575bc060ce15fe28'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:20,797 - lava_callback - INFO - Received callback for node 694090db575bc060ce1606e3 INFO: 172.21.0.1:43796 - "POST /node/694090db575bc060ce1606e3 HTTP/1.0" 202 Accepted 2025-12-17 22:51:21,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e6e575bc060ce15fe19 2025-12-17 22:51:21,490 - lava_callback - INFO - Received callback for node 694090dd575bc060ce1606e6 INFO: 172.21.0.1:43808 - "POST /node/694090dd575bc060ce1606e6 HTTP/1.0" 202 Accepted 2025-12-17 22:51:21,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e76575bc060ce15fe2b 2025-12-17 22:51:21,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e76575bc060ce15fe2b HTTP/1.1" 502 150 2025-12-17 22:51:21,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e76575bc060ce15fe2b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:21,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e98575bc060ce15fe91 2025-12-17 22:51:21,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e98575bc060ce15fe91 HTTP/1.1" 502 150 2025-12-17 22:51:21,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e98575bc060ce15fe91'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:22,174 - lava_callback - INFO - Received callback for node 694090df575bc060ce1606e9 INFO: 172.21.0.1:43812 - "POST /node/694090df575bc060ce1606e9 HTTP/1.0" 202 Accepted 2025-12-17 22:51:22,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7b575bc060ce15fe34 2025-12-17 22:51:22,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7b575bc060ce15fe34 HTTP/1.1" 502 150 2025-12-17 22:51:22,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7b575bc060ce15fe34'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:22,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7c575bc060ce15fe37 2025-12-17 22:51:22,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7c575bc060ce15fe37 HTTP/1.1" 502 150 2025-12-17 22:51:22,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7c575bc060ce15fe37'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:22,836 - lava_callback - INFO - Received callback for node 694090e1575bc060ce1606ec INFO: 172.21.0.1:43822 - "POST /node/694090e1575bc060ce1606ec HTTP/1.0" 202 Accepted 2025-12-17 22:51:22,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e77575bc060ce15fe2e 2025-12-17 22:51:22,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e77575bc060ce15fe2e HTTP/1.1" 502 150 2025-12-17 22:51:22,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e77575bc060ce15fe2e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:23,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7d575bc060ce15fe3d 2025-12-17 22:51:23,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7d575bc060ce15fe3d HTTP/1.1" 502 150 2025-12-17 22:51:23,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7d575bc060ce15fe3d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:23,122 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e5f575bc060ce15fdd5 2025-12-17 22:51:23,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e5f575bc060ce15fdd5 HTTP/1.1" 502 150 2025-12-17 22:51:23,123 - lava_callback - ERROR - Error processing callback for node 69408e5f575bc060ce15fdd5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e5f575bc060ce15fdd5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e5f575bc060ce15fdd5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e5f575bc060ce15fdd5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:23,125 - lava_callback - INFO - Starting async job submit for node 69408e86575bc060ce15fe5a 2025-12-17 22:51:23,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:23,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e86575bc060ce15fe5a HTTP/1.1" 502 150 2025-12-17 22:51:23,131 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e86575bc060ce15fe5a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:23,132 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e86575bc060ce15fe5a 2025-12-17 22:51:23,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e86575bc060ce15fe5a HTTP/1.1" 502 150 2025-12-17 22:51:23,132 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e86575bc060ce15fe5a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:24,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e70575bc060ce15fe1c 2025-12-17 22:51:24,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e70575bc060ce15fe1c HTTP/1.1" 502 150 2025-12-17 22:51:24,876 - lava_callback - ERROR - Error processing callback for node 69408e70575bc060ce15fe1c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e70575bc060ce15fe1c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e70575bc060ce15fe1c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e70575bc060ce15fe1c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:24,878 - lava_callback - INFO - Starting async job submit for node 69408e87575bc060ce15fe5d 2025-12-17 22:51:24,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:24,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e87575bc060ce15fe5d HTTP/1.1" 502 150 2025-12-17 22:51:24,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e87575bc060ce15fe5d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:24,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e87575bc060ce15fe5d 2025-12-17 22:51:24,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e87575bc060ce15fe5d HTTP/1.1" 502 150 2025-12-17 22:51:24,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e87575bc060ce15fe5d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:25,133 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e86575bc060ce15fe5a 2025-12-17 22:51:25,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e86575bc060ce15fe5a HTTP/1.1" 502 150 2025-12-17 22:51:25,134 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e86575bc060ce15fe5a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:25,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7e575bc060ce15fe40 2025-12-17 22:51:25,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7e575bc060ce15fe40 HTTP/1.1" 502 150 2025-12-17 22:51:25,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e7e575bc060ce15fe40'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43824 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:51:26,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e87575bc060ce15fe5d 2025-12-17 22:51:26,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e87575bc060ce15fe5d HTTP/1.1" 502 150 2025-12-17 22:51:26,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e87575bc060ce15fe5d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:27,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e71575bc060ce15fe1f 2025-12-17 22:51:27,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e71575bc060ce15fe1f HTTP/1.1" 502 150 2025-12-17 22:51:27,460 - lava_callback - ERROR - Error processing callback for node 69408e71575bc060ce15fe1f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e71575bc060ce15fe1f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e71575bc060ce15fe1f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e71575bc060ce15fe1f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:27,461 - lava_callback - INFO - Starting async job submit for node 69408e88575bc060ce15fe63 2025-12-17 22:51:27,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:27,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e88575bc060ce15fe63 HTTP/1.1" 502 150 2025-12-17 22:51:27,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e88575bc060ce15fe63'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:27,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e88575bc060ce15fe63 2025-12-17 22:51:27,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e88575bc060ce15fe63 HTTP/1.1" 502 150 2025-12-17 22:51:27,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e88575bc060ce15fe63'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:29,134 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e86575bc060ce15fe5a 2025-12-17 22:51:29,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e86575bc060ce15fe5a HTTP/1.1" 502 150 2025-12-17 22:51:29,135 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e86575bc060ce15fe5a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:29,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e88575bc060ce15fe63 2025-12-17 22:51:29,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e88575bc060ce15fe63 HTTP/1.1" 502 150 2025-12-17 22:51:29,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e88575bc060ce15fe63'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:29,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e76575bc060ce15fe2b 2025-12-17 22:51:29,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e76575bc060ce15fe2b HTTP/1.1" 502 150 2025-12-17 22:51:29,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e76575bc060ce15fe2b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:30,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e77575bc060ce15fe2e 2025-12-17 22:51:30,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e77575bc060ce15fe2e HTTP/1.1" 502 150 2025-12-17 22:51:30,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e77575bc060ce15fe2e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:30,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e87575bc060ce15fe5d 2025-12-17 22:51:30,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e87575bc060ce15fe5d HTTP/1.1" 502 150 2025-12-17 22:51:30,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e87575bc060ce15fe5d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:33,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e88575bc060ce15fe63 2025-12-17 22:51:33,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e88575bc060ce15fe63 HTTP/1.1" 502 150 2025-12-17 22:51:33,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e88575bc060ce15fe63'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:34,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e72575bc060ce15fe22 2025-12-17 22:51:34,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e72575bc060ce15fe22 HTTP/1.1" 502 150 2025-12-17 22:51:34,752 - lava_callback - ERROR - Error processing callback for node 69408e72575bc060ce15fe22: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e72575bc060ce15fe22 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e72575bc060ce15fe22 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e72575bc060ce15fe22 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:34,754 - lava_callback - INFO - Starting async job submit for node 69408e8a575bc060ce15fe67 2025-12-17 22:51:34,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:34,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8a575bc060ce15fe67 HTTP/1.1" 502 150 2025-12-17 22:51:34,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8a575bc060ce15fe67'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:34,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8a575bc060ce15fe67 2025-12-17 22:51:34,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8a575bc060ce15fe67 HTTP/1.1" 502 150 2025-12-17 22:51:34,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8a575bc060ce15fe67'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:34,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e6b575bc060ce15fe0c 2025-12-17 22:51:34,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6b575bc060ce15fe0c HTTP/1.1" 502 150 2025-12-17 22:51:34,898 - lava_callback - ERROR - Error processing callback for node 69408e6b575bc060ce15fe0c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e6b575bc060ce15fe0c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e6b575bc060ce15fe0c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e6b575bc060ce15fe0c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:34,900 - lava_callback - INFO - Starting async job submit for node 69408e80575bc060ce15fe45 2025-12-17 22:51:34,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:34,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e80575bc060ce15fe45 HTTP/1.1" 502 150 2025-12-17 22:51:34,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e80575bc060ce15fe45'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:34,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e80575bc060ce15fe45 2025-12-17 22:51:34,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e80575bc060ce15fe45 HTTP/1.1" 502 150 2025-12-17 22:51:34,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e80575bc060ce15fe45'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:35,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6d575bc060ce15fe13 HTTP/1.1" 502 150 2025-12-17 22:51:35,474 - lava_callback - ERROR - Error processing callback for node 69408e6d575bc060ce15fe13: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e6d575bc060ce15fe13 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e6d575bc060ce15fe13 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e6d575bc060ce15fe13 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:35,476 - lava_callback - INFO - Starting async job submit for node 69408e81575bc060ce15fe4d 2025-12-17 22:51:35,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:35,478 - lava_callback - INFO - Received callback for node 694090e3575bc060ce1606ef INFO: 172.21.0.1:58460 - "POST /node/694090e3575bc060ce1606ef HTTP/1.0" 202 Accepted 2025-12-17 22:51:35,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e81575bc060ce15fe4d HTTP/1.1" 502 150 2025-12-17 22:51:35,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e81575bc060ce15fe4d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:35,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e81575bc060ce15fe4d 2025-12-17 22:51:35,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e81575bc060ce15fe4d HTTP/1.1" 502 150 2025-12-17 22:51:35,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e81575bc060ce15fe4d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:36,136 - lava_callback - INFO - Received callback for node 694090e5575bc060ce1606f2 INFO: 172.21.0.1:58464 - "POST /node/694090e5575bc060ce1606f2 HTTP/1.0" 202 Accepted 2025-12-17 22:51:36,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e74575bc060ce15fe25 2025-12-17 22:51:36,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e74575bc060ce15fe25 HTTP/1.1" 502 150 2025-12-17 22:51:36,455 - lava_callback - ERROR - Error processing callback for node 69408e74575bc060ce15fe25: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e74575bc060ce15fe25 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e74575bc060ce15fe25 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e74575bc060ce15fe25 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:36,458 - lava_callback - INFO - Starting async job submit for node 69408e83575bc060ce15fe54 2025-12-17 22:51:36,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:36,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e83575bc060ce15fe54 HTTP/1.1" 502 150 2025-12-17 22:51:36,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e83575bc060ce15fe54'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:36,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e83575bc060ce15fe54 2025-12-17 22:51:36,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e83575bc060ce15fe54 HTTP/1.1" 502 150 2025-12-17 22:51:36,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e83575bc060ce15fe54'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58468 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:51:36,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e75575bc060ce15fe28 2025-12-17 22:51:36,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e75575bc060ce15fe28 HTTP/1.1" 502 150 2025-12-17 22:51:36,745 - lava_callback - ERROR - Error processing callback for node 69408e75575bc060ce15fe28: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e75575bc060ce15fe28 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e75575bc060ce15fe28 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e75575bc060ce15fe28 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:36,748 - lava_callback - INFO - Starting async job submit for node 6940fc16460bc8e265115db2 2025-12-17 22:51:36,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6e575bc060ce15fe19 HTTP/1.1" 502 150 2025-12-17 22:51:36,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e6e575bc060ce15fe19'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:36,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8a575bc060ce15fe67 2025-12-17 22:51:36,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8a575bc060ce15fe67 HTTP/1.1" 502 150 2025-12-17 22:51:36,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8a575bc060ce15fe67'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:36,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:36,794 - lava_callback - INFO - Received callback for node 694090e7575bc060ce1606f5 INFO: 172.21.0.1:58482 - "POST /node/694090e7575bc060ce1606f5 HTTP/1.0" 202 Accepted 2025-12-17 22:51:36,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fc16460bc8e265115db2 HTTP/1.1" 502 150 2025-12-17 22:51:36,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940fc16460bc8e265115db2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:36,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940fc16460bc8e265115db2 2025-12-17 22:51:36,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fc16460bc8e265115db2 HTTP/1.1" 502 150 2025-12-17 22:51:36,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940fc16460bc8e265115db2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:36,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e80575bc060ce15fe45 2025-12-17 22:51:36,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e80575bc060ce15fe45 HTTP/1.1" 502 150 2025-12-17 22:51:36,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e80575bc060ce15fe45'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:37,136 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e86575bc060ce15fe5a 2025-12-17 22:51:37,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e86575bc060ce15fe5a HTTP/1.1" 502 150 2025-12-17 22:51:37,137 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e86575bc060ce15fe5a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:37,460 - lava_callback - INFO - Received callback for node 694090ea575bc060ce1606f8 INFO: 172.21.0.1:58484 - "POST /node/694090ea575bc060ce1606f8 HTTP/1.0" 202 Accepted 2025-12-17 22:51:37,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e81575bc060ce15fe4d 2025-12-17 22:51:37,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e81575bc060ce15fe4d HTTP/1.1" 502 150 2025-12-17 22:51:37,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e81575bc060ce15fe4d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:37,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e98575bc060ce15fe91 2025-12-17 22:51:37,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e98575bc060ce15fe91 HTTP/1.1" 502 150 2025-12-17 22:51:37,914 - lava_callback - ERROR - Error processing callback for node 69408e98575bc060ce15fe91: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e98575bc060ce15fe91 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e98575bc060ce15fe91 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e98575bc060ce15fe91 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:37,916 - lava_callback - INFO - Starting async job submit for node 69408e84575bc060ce15fe57 2025-12-17 22:51:37,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:37,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e84575bc060ce15fe57 HTTP/1.1" 502 150 2025-12-17 22:51:37,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e84575bc060ce15fe57'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:37,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e84575bc060ce15fe57 2025-12-17 22:51:37,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e84575bc060ce15fe57 HTTP/1.1" 502 150 2025-12-17 22:51:37,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e84575bc060ce15fe57'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:38,164 - lava_callback - INFO - Received callback for node 694090ec575bc060ce1606fb INFO: 172.21.0.1:58488 - "POST /node/694090ec575bc060ce1606fb HTTP/1.0" 202 Accepted 2025-12-17 22:51:38,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e83575bc060ce15fe54 2025-12-17 22:51:38,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e83575bc060ce15fe54 HTTP/1.1" 502 150 2025-12-17 22:51:38,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e83575bc060ce15fe54'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:38,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7b575bc060ce15fe34 2025-12-17 22:51:38,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7b575bc060ce15fe34 HTTP/1.1" 502 150 2025-12-17 22:51:38,655 - lava_callback - ERROR - Error processing callback for node 69408e7b575bc060ce15fe34: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e7b575bc060ce15fe34 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e7b575bc060ce15fe34 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e7b575bc060ce15fe34 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:38,657 - lava_callback - INFO - Starting async job submit for node 69408e9b575bc060ce15fea0 2025-12-17 22:51:38,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:38,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9b575bc060ce15fea0 HTTP/1.1" 502 150 2025-12-17 22:51:38,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9b575bc060ce15fea0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:38,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9b575bc060ce15fea0 2025-12-17 22:51:38,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9b575bc060ce15fea0 HTTP/1.1" 502 150 2025-12-17 22:51:38,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9b575bc060ce15fea0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:38,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7c575bc060ce15fe37 2025-12-17 22:51:38,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7c575bc060ce15fe37 HTTP/1.1" 502 150 2025-12-17 22:51:38,822 - lava_callback - ERROR - Error processing callback for node 69408e7c575bc060ce15fe37: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e7c575bc060ce15fe37 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e7c575bc060ce15fe37 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e7c575bc060ce15fe37 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:38,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940fc16460bc8e265115db2 2025-12-17 22:51:38,824 - lava_callback - INFO - Starting async job submit for node 6940949b575bc060ce161098 2025-12-17 22:51:38,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fc16460bc8e265115db2 HTTP/1.1" 502 150 2025-12-17 22:51:38,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940fc16460bc8e265115db2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:38,846 - lava_callback - INFO - Received callback for node 694090ee575bc060ce160702 INFO: 172.21.0.1:58504 - "POST /node/694090ee575bc060ce160702 HTTP/1.0" 202 Accepted 2025-12-17 22:51:38,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:38,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949b575bc060ce161098 HTTP/1.1" 502 150 2025-12-17 22:51:38,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940949b575bc060ce161098'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:38,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940949b575bc060ce161098 2025-12-17 22:51:38,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949b575bc060ce161098 HTTP/1.1" 502 150 2025-12-17 22:51:38,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940949b575bc060ce161098'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:38,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e87575bc060ce15fe5d 2025-12-17 22:51:38,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e87575bc060ce15fe5d HTTP/1.1" 502 150 2025-12-17 22:51:38,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e87575bc060ce15fe5d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:39,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7d575bc060ce15fe3d 2025-12-17 22:51:39,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7d575bc060ce15fe3d HTTP/1.1" 502 150 2025-12-17 22:51:39,017 - lava_callback - ERROR - Error processing callback for node 69408e7d575bc060ce15fe3d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e7d575bc060ce15fe3d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e7d575bc060ce15fe3d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e7d575bc060ce15fe3d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:39,019 - lava_callback - INFO - Starting async job submit for node 69408e96575bc060ce15fe8a 2025-12-17 22:51:39,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:39,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e96575bc060ce15fe8a HTTP/1.1" 502 150 2025-12-17 22:51:39,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e96575bc060ce15fe8a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:39,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e96575bc060ce15fe8a 2025-12-17 22:51:39,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e96575bc060ce15fe8a HTTP/1.1" 502 150 2025-12-17 22:51:39,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e96575bc060ce15fe8a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:39,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e84575bc060ce15fe57 2025-12-17 22:51:39,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e84575bc060ce15fe57 HTTP/1.1" 502 150 2025-12-17 22:51:39,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e84575bc060ce15fe57'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:40,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9b575bc060ce15fea0 2025-12-17 22:51:40,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9b575bc060ce15fea0 HTTP/1.1" 502 150 2025-12-17 22:51:40,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9b575bc060ce15fea0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:40,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8a575bc060ce15fe67 2025-12-17 22:51:40,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8a575bc060ce15fe67 HTTP/1.1" 502 150 2025-12-17 22:51:40,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8a575bc060ce15fe67'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:40,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940949b575bc060ce161098 2025-12-17 22:51:40,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949b575bc060ce161098 HTTP/1.1" 502 150 2025-12-17 22:51:40,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940949b575bc060ce161098'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:40,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e80575bc060ce15fe45 2025-12-17 22:51:40,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e80575bc060ce15fe45 HTTP/1.1" 502 150 2025-12-17 22:51:40,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e80575bc060ce15fe45'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:41,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e96575bc060ce15fe8a 2025-12-17 22:51:41,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e96575bc060ce15fe8a HTTP/1.1" 502 150 2025-12-17 22:51:41,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e96575bc060ce15fe8a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:41,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e88575bc060ce15fe63 2025-12-17 22:51:41,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e88575bc060ce15fe63 HTTP/1.1" 502 150 2025-12-17 22:51:41,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e88575bc060ce15fe63'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:41,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e81575bc060ce15fe4d 2025-12-17 22:51:41,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e81575bc060ce15fe4d HTTP/1.1" 502 150 2025-12-17 22:51:41,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e81575bc060ce15fe4d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:41,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e7e575bc060ce15fe40 2025-12-17 22:51:41,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e7e575bc060ce15fe40 HTTP/1.1" 502 150 2025-12-17 22:51:41,701 - lava_callback - ERROR - Error processing callback for node 69408e7e575bc060ce15fe40: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e7e575bc060ce15fe40 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e7e575bc060ce15fe40 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e7e575bc060ce15fe40 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:41,702 - lava_callback - INFO - Starting async job submit for node 69408e8b575bc060ce15fe6a 2025-12-17 22:51:41,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:41,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8b575bc060ce15fe6a HTTP/1.1" 502 150 2025-12-17 22:51:41,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8b575bc060ce15fe6a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:41,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8b575bc060ce15fe6a 2025-12-17 22:51:41,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8b575bc060ce15fe6a HTTP/1.1" 502 150 2025-12-17 22:51:41,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8b575bc060ce15fe6a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:42,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e83575bc060ce15fe54 2025-12-17 22:51:42,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e83575bc060ce15fe54 HTTP/1.1" 502 150 2025-12-17 22:51:42,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e83575bc060ce15fe54'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:42,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940fc16460bc8e265115db2 2025-12-17 22:51:42,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fc16460bc8e265115db2 HTTP/1.1" 502 150 2025-12-17 22:51:42,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940fc16460bc8e265115db2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:43,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8b575bc060ce15fe6a 2025-12-17 22:51:43,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8b575bc060ce15fe6a HTTP/1.1" 502 150 2025-12-17 22:51:43,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8b575bc060ce15fe6a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:43,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e84575bc060ce15fe57 2025-12-17 22:51:43,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e84575bc060ce15fe57 HTTP/1.1" 502 150 2025-12-17 22:51:43,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e84575bc060ce15fe57'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:44,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9b575bc060ce15fea0 2025-12-17 22:51:44,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9b575bc060ce15fea0 HTTP/1.1" 502 150 2025-12-17 22:51:44,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9b575bc060ce15fea0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:44,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940949b575bc060ce161098 2025-12-17 22:51:44,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949b575bc060ce161098 HTTP/1.1" 502 150 2025-12-17 22:51:44,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940949b575bc060ce161098'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:45,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e96575bc060ce15fe8a 2025-12-17 22:51:45,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e96575bc060ce15fe8a HTTP/1.1" 502 150 2025-12-17 22:51:45,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e96575bc060ce15fe8a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:45,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e76575bc060ce15fe2b 2025-12-17 22:51:45,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e76575bc060ce15fe2b HTTP/1.1" 502 150 2025-12-17 22:51:45,561 - lava_callback - ERROR - Error processing callback for node 69408e76575bc060ce15fe2b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e76575bc060ce15fe2b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e76575bc060ce15fe2b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e76575bc060ce15fe2b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:45,563 - lava_callback - INFO - Starting async job submit for node 69408e8c575bc060ce15fe6d 2025-12-17 22:51:45,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:45,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8c575bc060ce15fe6d HTTP/1.1" 502 150 2025-12-17 22:51:45,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8c575bc060ce15fe6d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:45,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8c575bc060ce15fe6d 2025-12-17 22:51:45,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8c575bc060ce15fe6d HTTP/1.1" 502 150 2025-12-17 22:51:45,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8c575bc060ce15fe6d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:45,827 - lava_callback - INFO - Received callback for node 694090f0575bc060ce160705 INFO: 172.21.0.1:58338 - "POST /node/694090f0575bc060ce160705 HTTP/1.0" 202 Accepted 2025-12-17 22:51:46,492 - lava_callback - INFO - Received callback for node 694090f2575bc060ce160708 INFO: 172.21.0.1:58340 - "POST /node/694090f2575bc060ce160708 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:58342 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:51:46,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e77575bc060ce15fe2e 2025-12-17 22:51:46,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e77575bc060ce15fe2e HTTP/1.1" 502 150 2025-12-17 22:51:46,872 - lava_callback - ERROR - Error processing callback for node 69408e77575bc060ce15fe2e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e77575bc060ce15fe2e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e77575bc060ce15fe2e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e77575bc060ce15fe2e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:46,874 - lava_callback - INFO - Starting async job submit for node 69408e8d575bc060ce15fe70 2025-12-17 22:51:46,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:46,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8d575bc060ce15fe70 HTTP/1.1" 502 150 2025-12-17 22:51:46,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8d575bc060ce15fe70'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:46,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8d575bc060ce15fe70 2025-12-17 22:51:46,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8d575bc060ce15fe70 HTTP/1.1" 502 150 2025-12-17 22:51:46,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8d575bc060ce15fe70'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:47,162 - lava_callback - INFO - Received callback for node 694090f4575bc060ce16070e INFO: 172.21.0.1:58352 - "POST /node/694090f4575bc060ce16070e HTTP/1.0" 202 Accepted 2025-12-17 22:51:47,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8c575bc060ce15fe6d 2025-12-17 22:51:47,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8b575bc060ce15fe6a 2025-12-17 22:51:47,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8b575bc060ce15fe6a HTTP/1.1" 502 150 2025-12-17 22:51:47,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8b575bc060ce15fe6a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:47,827 - lava_callback - INFO - Received callback for node 694090f6575bc060ce160711 INFO: 172.21.0.1:58364 - "POST /node/694090f6575bc060ce160711 HTTP/1.0" 202 Accepted 2025-12-17 22:51:48,476 - lava_callback - INFO - Received callback for node 694090f8575bc060ce160714 INFO: 172.21.0.1:58366 - "POST /node/694090f8575bc060ce160714 HTTP/1.0" 202 Accepted 2025-12-17 22:51:48,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8a575bc060ce15fe67 2025-12-17 22:51:48,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8a575bc060ce15fe67 HTTP/1.1" 502 150 2025-12-17 22:51:48,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8a575bc060ce15fe67'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:48,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8d575bc060ce15fe70 2025-12-17 22:51:48,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8d575bc060ce15fe70 HTTP/1.1" 502 150 2025-12-17 22:51:48,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8d575bc060ce15fe70'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:48,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e80575bc060ce15fe45 2025-12-17 22:51:48,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e80575bc060ce15fe45 HTTP/1.1" 502 150 2025-12-17 22:51:48,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e80575bc060ce15fe45'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:49,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e81575bc060ce15fe4d 2025-12-17 22:51:49,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e81575bc060ce15fe4d HTTP/1.1" 502 150 2025-12-17 22:51:49,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e81575bc060ce15fe4d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:50,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e83575bc060ce15fe54 2025-12-17 22:51:50,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e83575bc060ce15fe54 HTTP/1.1" 502 150 2025-12-17 22:51:50,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e83575bc060ce15fe54'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:50,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940fc16460bc8e265115db2 2025-12-17 22:51:50,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fc16460bc8e265115db2 HTTP/1.1" 502 150 2025-12-17 22:51:50,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940fc16460bc8e265115db2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:51,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e84575bc060ce15fe57 2025-12-17 22:51:51,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e84575bc060ce15fe57 HTTP/1.1" 502 150 2025-12-17 22:51:51,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e84575bc060ce15fe57'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:52,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9b575bc060ce15fea0 2025-12-17 22:51:52,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9b575bc060ce15fea0 HTTP/1.1" 502 150 2025-12-17 22:51:52,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9b575bc060ce15fea0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:52,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e6e575bc060ce15fe19 2025-12-17 22:51:52,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e6e575bc060ce15fe19 HTTP/1.1" 502 150 2025-12-17 22:51:52,763 - lava_callback - ERROR - Error processing callback for node 69408e6e575bc060ce15fe19: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e6e575bc060ce15fe19 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e6e575bc060ce15fe19 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e6e575bc060ce15fe19 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:52,766 - lava_callback - INFO - Starting async job submit for node 69408e8f575bc060ce15fe73 2025-12-17 22:51:52,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:52,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8f575bc060ce15fe73 HTTP/1.1" 502 150 2025-12-17 22:51:52,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8f575bc060ce15fe73'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:52,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8f575bc060ce15fe73 2025-12-17 22:51:52,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8f575bc060ce15fe73 HTTP/1.1" 502 150 2025-12-17 22:51:52,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8f575bc060ce15fe73'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:52,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8d575bc060ce15fe70 2025-12-17 22:51:52,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8d575bc060ce15fe70 HTTP/1.1" 502 150 2025-12-17 22:51:52,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8d575bc060ce15fe70'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:52,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940949b575bc060ce161098 2025-12-17 22:51:52,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949b575bc060ce161098 HTTP/1.1" 502 150 2025-12-17 22:51:52,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940949b575bc060ce161098'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:53,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e96575bc060ce15fe8a 2025-12-17 22:51:53,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e96575bc060ce15fe8a HTTP/1.1" 502 150 2025-12-17 22:51:53,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e96575bc060ce15fe8a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:53,150 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e86575bc060ce15fe5a 2025-12-17 22:51:53,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e86575bc060ce15fe5a HTTP/1.1" 502 150 2025-12-17 22:51:53,152 - lava_callback - ERROR - Error processing callback for node 69408e86575bc060ce15fe5a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e86575bc060ce15fe5a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e86575bc060ce15fe5a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e86575bc060ce15fe5a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:53,155 - lava_callback - INFO - Starting async job submit for node 69408e92575bc060ce15fe79 2025-12-17 22:51:53,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:53,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e92575bc060ce15fe79 HTTP/1.1" 502 150 2025-12-17 22:51:53,170 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e92575bc060ce15fe79'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:53,171 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e92575bc060ce15fe79 2025-12-17 22:51:53,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e92575bc060ce15fe79 HTTP/1.1" 502 150 2025-12-17 22:51:53,171 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e92575bc060ce15fe79'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:54,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8f575bc060ce15fe73 2025-12-17 22:51:54,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8f575bc060ce15fe73 HTTP/1.1" 502 150 2025-12-17 22:51:54,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8f575bc060ce15fe73'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:54,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e87575bc060ce15fe5d 2025-12-17 22:51:54,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e87575bc060ce15fe5d HTTP/1.1" 502 150 2025-12-17 22:51:54,914 - lava_callback - ERROR - Error processing callback for node 69408e87575bc060ce15fe5d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e87575bc060ce15fe5d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e87575bc060ce15fe5d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e87575bc060ce15fe5d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:54,917 - lava_callback - INFO - Starting async job submit for node 69408e93575bc060ce15fe7c 2025-12-17 22:51:54,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:54,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e93575bc060ce15fe7c HTTP/1.1" 502 150 2025-12-17 22:51:54,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e93575bc060ce15fe7c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:54,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e93575bc060ce15fe7c 2025-12-17 22:51:54,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e93575bc060ce15fe7c HTTP/1.1" 502 150 2025-12-17 22:51:54,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e93575bc060ce15fe7c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:55,172 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e92575bc060ce15fe79 2025-12-17 22:51:55,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e92575bc060ce15fe79 HTTP/1.1" 502 150 2025-12-17 22:51:55,174 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e92575bc060ce15fe79'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:55,448 - lava_callback - INFO - Received callback for node 694090fb575bc060ce160717 INFO: 172.21.0.1:41744 - "POST /node/694090fb575bc060ce160717 HTTP/1.0" 202 Accepted 2025-12-17 22:51:55,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8b575bc060ce15fe6a 2025-12-17 22:51:55,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8b575bc060ce15fe6a HTTP/1.1" 502 150 2025-12-17 22:51:55,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8b575bc060ce15fe6a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:56,092 - lava_callback - INFO - Received callback for node 694090fc575bc060ce16071a INFO: 172.21.0.1:41750 - "POST /node/694090fc575bc060ce16071a HTTP/1.0" 202 Accepted INFO: 172.21.0.1:41760 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:51:56,750 - lava_callback - INFO - Received callback for node 694090fd575bc060ce16071d INFO: 172.21.0.1:41768 - "POST /node/694090fd575bc060ce16071d HTTP/1.0" 202 Accepted 2025-12-17 22:51:56,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e93575bc060ce15fe7c 2025-12-17 22:51:56,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e93575bc060ce15fe7c HTTP/1.1" 502 150 2025-12-17 22:51:56,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e93575bc060ce15fe7c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:57,436 - lava_callback - INFO - Received callback for node 694090fe575bc060ce160720 INFO: 172.21.0.1:41772 - "POST /node/694090fe575bc060ce160720 HTTP/1.0" 202 Accepted 2025-12-17 22:51:57,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e88575bc060ce15fe63 2025-12-17 22:51:57,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e88575bc060ce15fe63 HTTP/1.1" 502 150 2025-12-17 22:51:57,477 - lava_callback - ERROR - Error processing callback for node 69408e88575bc060ce15fe63: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e88575bc060ce15fe63 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e88575bc060ce15fe63 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e88575bc060ce15fe63 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:51:57,479 - lava_callback - INFO - Starting async job submit for node 69408ea3575bc060ce15fea9 2025-12-17 22:51:57,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:51:57,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea3575bc060ce15fea9 HTTP/1.1" 502 150 2025-12-17 22:51:57,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea3575bc060ce15fea9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:57,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea3575bc060ce15fea9 2025-12-17 22:51:57,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea3575bc060ce15fea9 HTTP/1.1" 502 150 2025-12-17 22:51:57,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea3575bc060ce15fea9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:58,105 - lava_callback - INFO - Received callback for node 69409103575bc060ce16072b INFO: 172.21.0.1:41782 - "POST /node/69409103575bc060ce16072b HTTP/1.0" 202 Accepted 2025-12-17 22:51:58,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8f575bc060ce15fe73 2025-12-17 22:51:58,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8f575bc060ce15fe73 HTTP/1.1" 502 150 2025-12-17 22:51:58,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8f575bc060ce15fe73'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:51:58,786 - lava_callback - INFO - Received callback for node 69409105575bc060ce16074a INFO: 172.21.0.1:41792 - "POST /node/69409105575bc060ce16074a HTTP/1.0" 202 Accepted 2025-12-17 22:51:59,174 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e92575bc060ce15fe79 2025-12-17 22:51:59,460 - lava_callback - INFO - Received callback for node 69409106575bc060ce160751 INFO: 172.21.0.1:41802 - "POST /node/69409106575bc060ce160751 HTTP/1.0" 202 Accepted 2025-12-17 22:51:59,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea3575bc060ce15fea9 2025-12-17 22:51:59,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea3575bc060ce15fea9 HTTP/1.1" 502 150 2025-12-17 22:51:59,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea3575bc060ce15fea9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:00,105 - lava_callback - INFO - Received callback for node 69409107575bc060ce160754 INFO: 172.21.0.1:33280 - "POST /node/69409107575bc060ce160754 HTTP/1.0" 202 Accepted 2025-12-17 22:52:00,777 - lava_callback - INFO - Received callback for node 69409109575bc060ce16075b INFO: 172.21.0.1:33286 - "POST /node/69409109575bc060ce16075b HTTP/1.0" 202 Accepted 2025-12-17 22:52:00,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8d575bc060ce15fe70 2025-12-17 22:52:00,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8d575bc060ce15fe70 HTTP/1.1" 502 150 2025-12-17 22:52:00,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8d575bc060ce15fe70'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:00,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e93575bc060ce15fe7c 2025-12-17 22:52:00,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e93575bc060ce15fe7c HTTP/1.1" 502 150 2025-12-17 22:52:00,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e93575bc060ce15fe7c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33294 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:52:02,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8c575bc060ce15fe6d HTTP/1.1" 502 150 2025-12-17 22:52:02,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8c575bc060ce15fe6d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:03,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea3575bc060ce15fea9 2025-12-17 22:52:03,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea3575bc060ce15fea9 HTTP/1.1" 502 150 2025-12-17 22:52:03,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea3575bc060ce15fea9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:04,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8a575bc060ce15fe67 2025-12-17 22:52:04,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8a575bc060ce15fe67 HTTP/1.1" 502 150 2025-12-17 22:52:04,789 - lava_callback - ERROR - Error processing callback for node 69408e8a575bc060ce15fe67: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e8a575bc060ce15fe67 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e8a575bc060ce15fe67 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e8a575bc060ce15fe67 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:04,791 - lava_callback - INFO - Starting async job submit for node 69408ea6575bc060ce15febc 2025-12-17 22:52:04,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:04,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea6575bc060ce15febc HTTP/1.1" 502 150 2025-12-17 22:52:04,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea6575bc060ce15febc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:04,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea6575bc060ce15febc 2025-12-17 22:52:04,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea6575bc060ce15febc HTTP/1.1" 502 150 2025-12-17 22:52:04,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea6575bc060ce15febc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:04,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e80575bc060ce15fe45 2025-12-17 22:52:04,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e80575bc060ce15fe45 HTTP/1.1" 502 150 2025-12-17 22:52:04,912 - lava_callback - ERROR - Error processing callback for node 69408e80575bc060ce15fe45: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e80575bc060ce15fe45 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e80575bc060ce15fe45 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e80575bc060ce15fe45 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:04,914 - lava_callback - INFO - Starting async job submit for node 69408e9e575bc060ce15fea6 2025-12-17 22:52:04,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:04,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9e575bc060ce15fea6 HTTP/1.1" 502 150 2025-12-17 22:52:04,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9e575bc060ce15fea6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:04,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9e575bc060ce15fea6 2025-12-17 22:52:04,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9e575bc060ce15fea6 HTTP/1.1" 502 150 2025-12-17 22:52:04,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9e575bc060ce15fea6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:05,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e81575bc060ce15fe4d 2025-12-17 22:52:05,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e81575bc060ce15fe4d HTTP/1.1" 502 150 2025-12-17 22:52:05,515 - lava_callback - ERROR - Error processing callback for node 69408e81575bc060ce15fe4d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e81575bc060ce15fe4d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e81575bc060ce15fe4d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e81575bc060ce15fe4d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:05,517 - lava_callback - INFO - Starting async job submit for node 69408ea5575bc060ce15feac 2025-12-17 22:52:05,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:05,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea5575bc060ce15feac HTTP/1.1" 502 150 2025-12-17 22:52:05,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea5575bc060ce15feac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:05,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea5575bc060ce15feac 2025-12-17 22:52:05,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea5575bc060ce15feac HTTP/1.1" 502 150 2025-12-17 22:52:05,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea5575bc060ce15feac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:05,828 - lava_callback - INFO - Received callback for node 6940910a575bc060ce16075e INFO: 172.21.0.1:33302 - "POST /node/6940910a575bc060ce16075e HTTP/1.0" 202 Accepted 2025-12-17 22:52:06,470 - lava_callback - INFO - Received callback for node 6940910c575bc060ce160761 INFO: 172.21.0.1:33304 - "POST /node/6940910c575bc060ce160761 HTTP/1.0" 202 Accepted 2025-12-17 22:52:06,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e83575bc060ce15fe54 2025-12-17 22:52:06,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e83575bc060ce15fe54 HTTP/1.1" 502 150 2025-12-17 22:52:06,477 - lava_callback - ERROR - Error processing callback for node 69408e83575bc060ce15fe54: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e83575bc060ce15fe54 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e83575bc060ce15fe54 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e83575bc060ce15fe54 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:06,478 - lava_callback - INFO - Starting async job submit for node 69408ea8575bc060ce15fec1 2025-12-17 22:52:06,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:06,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea8575bc060ce15fec1 HTTP/1.1" 502 150 2025-12-17 22:52:06,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea8575bc060ce15fec1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:06,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea8575bc060ce15fec1 2025-12-17 22:52:06,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea8575bc060ce15fec1 HTTP/1.1" 502 150 2025-12-17 22:52:06,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea8575bc060ce15fec1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33314 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:52:06,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8f575bc060ce15fe73 2025-12-17 22:52:06,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8f575bc060ce15fe73 HTTP/1.1" 502 150 2025-12-17 22:52:06,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8f575bc060ce15fe73'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:06,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea6575bc060ce15febc 2025-12-17 22:52:06,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea6575bc060ce15febc HTTP/1.1" 502 150 2025-12-17 22:52:06,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea6575bc060ce15febc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:06,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940fc16460bc8e265115db2 2025-12-17 22:52:06,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fc16460bc8e265115db2 HTTP/1.1" 502 150 2025-12-17 22:52:06,849 - lava_callback - ERROR - Error processing callback for node 6940fc16460bc8e265115db2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940fc16460bc8e265115db2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940fc16460bc8e265115db2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940fc16460bc8e265115db2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:06,851 - lava_callback - INFO - Starting async job submit for node 69408ea9575bc060ce15fec8 2025-12-17 22:52:06,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:06,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea9575bc060ce15fec8 HTTP/1.1" 502 150 2025-12-17 22:52:06,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea9575bc060ce15fec8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:06,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea9575bc060ce15fec8 2025-12-17 22:52:06,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea9575bc060ce15fec8 HTTP/1.1" 502 150 2025-12-17 22:52:06,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea9575bc060ce15fec8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:06,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8c575bc060ce15fe6d 2025-12-17 22:52:06,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8c575bc060ce15fe6d HTTP/1.1" 502 150 2025-12-17 22:52:06,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8c575bc060ce15fe6d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:06,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9e575bc060ce15fea6 2025-12-17 22:52:06,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9e575bc060ce15fea6 HTTP/1.1" 502 150 2025-12-17 22:52:06,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9e575bc060ce15fea6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:07,118 - lava_callback - INFO - Received callback for node 6940910e575bc060ce160768 INFO: 172.21.0.1:33320 - "POST /node/6940910e575bc060ce160768 HTTP/1.0" 202 Accepted 2025-12-17 22:52:07,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea5575bc060ce15feac 2025-12-17 22:52:07,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea5575bc060ce15feac HTTP/1.1" 502 150 2025-12-17 22:52:07,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea5575bc060ce15feac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:07,768 - lava_callback - INFO - Received callback for node 69409110575bc060ce16076b INFO: 172.21.0.1:33326 - "POST /node/69409110575bc060ce16076b HTTP/1.0" 202 Accepted 2025-12-17 22:52:07,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e84575bc060ce15fe57 2025-12-17 22:52:07,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e84575bc060ce15fe57 HTTP/1.1" 502 150 2025-12-17 22:52:07,937 - lava_callback - ERROR - Error processing callback for node 69408e84575bc060ce15fe57: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e84575bc060ce15fe57 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e84575bc060ce15fe57 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e84575bc060ce15fe57 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:07,939 - lava_callback - INFO - Starting async job submit for node 69408eaa575bc060ce15fecf 2025-12-17 22:52:07,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:07,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eaa575bc060ce15fecf HTTP/1.1" 502 150 2025-12-17 22:52:07,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eaa575bc060ce15fecf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:07,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eaa575bc060ce15fecf 2025-12-17 22:52:07,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eaa575bc060ce15fecf HTTP/1.1" 502 150 2025-12-17 22:52:07,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eaa575bc060ce15fecf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:08,424 - lava_callback - INFO - Received callback for node 69409112575bc060ce16076e INFO: 172.21.0.1:33332 - "POST /node/69409112575bc060ce16076e HTTP/1.0" 202 Accepted 2025-12-17 22:52:08,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea8575bc060ce15fec1 2025-12-17 22:52:08,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea8575bc060ce15fec1 HTTP/1.1" 502 150 2025-12-17 22:52:08,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea8575bc060ce15fec1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:08,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9b575bc060ce15fea0 2025-12-17 22:52:08,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9b575bc060ce15fea0 HTTP/1.1" 502 150 2025-12-17 22:52:08,671 - lava_callback - ERROR - Error processing callback for node 69408e9b575bc060ce15fea0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e9b575bc060ce15fea0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e9b575bc060ce15fea0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e9b575bc060ce15fea0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:08,674 - lava_callback - INFO - Starting async job submit for node 69408eaf575bc060ce15fedb 2025-12-17 22:52:08,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:08,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eaf575bc060ce15fedb HTTP/1.1" 502 150 2025-12-17 22:52:08,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eaf575bc060ce15fedb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:08,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eaf575bc060ce15fedb 2025-12-17 22:52:08,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eaf575bc060ce15fedb HTTP/1.1" 502 150 2025-12-17 22:52:08,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eaf575bc060ce15fedb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:08,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea9575bc060ce15fec8 2025-12-17 22:52:08,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea9575bc060ce15fec8 HTTP/1.1" 502 150 2025-12-17 22:52:08,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea9575bc060ce15fec8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:08,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940949b575bc060ce161098 2025-12-17 22:52:08,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949b575bc060ce161098 HTTP/1.1" 502 150 2025-12-17 22:52:08,907 - lava_callback - ERROR - Error processing callback for node 6940949b575bc060ce161098: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940949b575bc060ce161098 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940949b575bc060ce161098 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940949b575bc060ce161098 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:08,910 - lava_callback - INFO - Starting async job submit for node 69408eac575bc060ce15fed2 2025-12-17 22:52:08,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:08,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eac575bc060ce15fed2 HTTP/1.1" 502 150 2025-12-17 22:52:08,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eac575bc060ce15fed2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:08,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eac575bc060ce15fed2 2025-12-17 22:52:08,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eac575bc060ce15fed2 HTTP/1.1" 502 150 2025-12-17 22:52:08,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eac575bc060ce15fed2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:08,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e93575bc060ce15fe7c 2025-12-17 22:52:08,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e93575bc060ce15fe7c HTTP/1.1" 502 150 2025-12-17 22:52:08,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e93575bc060ce15fe7c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:09,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e96575bc060ce15fe8a 2025-12-17 22:52:09,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e96575bc060ce15fe8a HTTP/1.1" 502 150 2025-12-17 22:52:09,033 - lava_callback - ERROR - Error processing callback for node 69408e96575bc060ce15fe8a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e96575bc060ce15fe8a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e96575bc060ce15fe8a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e96575bc060ce15fe8a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:09,036 - lava_callback - INFO - Starting async job submit for node 69408eb0575bc060ce15fede 2025-12-17 22:52:09,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:09,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb0575bc060ce15fede HTTP/1.1" 502 150 2025-12-17 22:52:09,047 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb0575bc060ce15fede'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:09,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb0575bc060ce15fede 2025-12-17 22:52:09,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb0575bc060ce15fede HTTP/1.1" 502 150 2025-12-17 22:52:09,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb0575bc060ce15fede'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:09,071 - lava_callback - INFO - Received callback for node 69409114575bc060ce160773 INFO: 172.21.0.1:33340 - "POST /node/69409114575bc060ce160773 HTTP/1.0" 202 Accepted 2025-12-17 22:52:09,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eaa575bc060ce15fecf 2025-12-17 22:52:09,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eaa575bc060ce15fecf HTTP/1.1" 502 150 2025-12-17 22:52:09,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eaa575bc060ce15fecf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:10,036 - lava_callback - INFO - Received callback for node 694106df460bc8e265116fe2 INFO: 172.21.0.1:44298 - "POST /node/694106df460bc8e265116fe2 HTTP/1.0" 202 Accepted 2025-12-17 22:52:10,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eaf575bc060ce15fedb 2025-12-17 22:52:10,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eaf575bc060ce15fedb HTTP/1.1" 502 150 2025-12-17 22:52:10,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eaf575bc060ce15fedb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:10,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea6575bc060ce15febc 2025-12-17 22:52:10,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea6575bc060ce15febc HTTP/1.1" 502 150 2025-12-17 22:52:10,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea6575bc060ce15febc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:10,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eac575bc060ce15fed2 2025-12-17 22:52:10,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eac575bc060ce15fed2 HTTP/1.1" 502 150 2025-12-17 22:52:10,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eac575bc060ce15fed2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:10,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9e575bc060ce15fea6 2025-12-17 22:52:10,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9e575bc060ce15fea6 HTTP/1.1" 502 150 2025-12-17 22:52:10,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9e575bc060ce15fea6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:11,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb0575bc060ce15fede 2025-12-17 22:52:11,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb0575bc060ce15fede HTTP/1.1" 502 150 2025-12-17 22:52:11,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb0575bc060ce15fede'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:11,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea3575bc060ce15fea9 2025-12-17 22:52:11,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea3575bc060ce15fea9 HTTP/1.1" 502 150 2025-12-17 22:52:11,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea3575bc060ce15fea9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:11,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea5575bc060ce15feac 2025-12-17 22:52:11,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea5575bc060ce15feac HTTP/1.1" 502 150 2025-12-17 22:52:11,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea5575bc060ce15feac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:11,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8b575bc060ce15fe6a 2025-12-17 22:52:11,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8b575bc060ce15fe6a HTTP/1.1" 502 150 2025-12-17 22:52:11,718 - lava_callback - ERROR - Error processing callback for node 69408e8b575bc060ce15fe6a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e8b575bc060ce15fe6a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e8b575bc060ce15fe6a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e8b575bc060ce15fe6a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:11,721 - lava_callback - INFO - Starting async job submit for node 69408ead575bc060ce15fed5 2025-12-17 22:52:11,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:11,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ead575bc060ce15fed5 HTTP/1.1" 502 150 2025-12-17 22:52:11,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ead575bc060ce15fed5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:11,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ead575bc060ce15fed5 2025-12-17 22:52:11,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ead575bc060ce15fed5 HTTP/1.1" 502 150 2025-12-17 22:52:11,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ead575bc060ce15fed5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:12,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea8575bc060ce15fec1 2025-12-17 22:52:12,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea8575bc060ce15fec1 HTTP/1.1" 502 150 2025-12-17 22:52:12,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea8575bc060ce15fec1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:12,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea9575bc060ce15fec8 2025-12-17 22:52:12,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea9575bc060ce15fec8 HTTP/1.1" 502 150 2025-12-17 22:52:12,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea9575bc060ce15fec8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:13,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ead575bc060ce15fed5 2025-12-17 22:52:13,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eaa575bc060ce15fecf 2025-12-17 22:52:13,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eaa575bc060ce15fecf HTTP/1.1" 502 150 2025-12-17 22:52:13,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eaa575bc060ce15fecf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:14,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e92575bc060ce15fe79 HTTP/1.1" 502 150 2025-12-17 22:52:14,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e92575bc060ce15fe79'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:14,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eaf575bc060ce15fedb 2025-12-17 22:52:14,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eaf575bc060ce15fedb HTTP/1.1" 502 150 2025-12-17 22:52:14,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eaf575bc060ce15fedb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:14,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8c575bc060ce15fe6d 2025-12-17 22:52:14,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8c575bc060ce15fe6d HTTP/1.1" 502 150 2025-12-17 22:52:14,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e8c575bc060ce15fe6d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:14,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eac575bc060ce15fed2 2025-12-17 22:52:14,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eac575bc060ce15fed2 HTTP/1.1" 502 150 2025-12-17 22:52:14,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eac575bc060ce15fed2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:15,049 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb0575bc060ce15fede 2025-12-17 22:52:15,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb0575bc060ce15fede HTTP/1.1" 502 150 2025-12-17 22:52:15,051 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb0575bc060ce15fede'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44304 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:52:15,467 - lava_callback - INFO - Received callback for node 6940911b575bc060ce160784 INFO: 172.21.0.1:44314 - "POST /node/6940911b575bc060ce160784 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:44330 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:52:16,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8d575bc060ce15fe70 2025-12-17 22:52:16,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8d575bc060ce15fe70 HTTP/1.1" 502 150 2025-12-17 22:52:16,890 - lava_callback - ERROR - Error processing callback for node 69408e8d575bc060ce15fe70: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e8d575bc060ce15fe70 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e8d575bc060ce15fe70 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e8d575bc060ce15fe70 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:16,892 - lava_callback - INFO - Starting async job submit for node 69408eb2575bc060ce15fee1 2025-12-17 22:52:16,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:16,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb2575bc060ce15fee1 HTTP/1.1" 502 150 2025-12-17 22:52:16,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb2575bc060ce15fee1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:16,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb2575bc060ce15fee1 2025-12-17 22:52:16,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb2575bc060ce15fee1 HTTP/1.1" 502 150 2025-12-17 22:52:16,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb2575bc060ce15fee1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:18,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea6575bc060ce15febc 2025-12-17 22:52:18,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea6575bc060ce15febc HTTP/1.1" 502 150 2025-12-17 22:52:18,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea6575bc060ce15febc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:18,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb2575bc060ce15fee1 2025-12-17 22:52:18,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb2575bc060ce15fee1 HTTP/1.1" 502 150 2025-12-17 22:52:18,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb2575bc060ce15fee1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:18,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9e575bc060ce15fea6 2025-12-17 22:52:18,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9e575bc060ce15fea6 HTTP/1.1" 502 150 2025-12-17 22:52:18,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9e575bc060ce15fea6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:19,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea5575bc060ce15feac 2025-12-17 22:52:19,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea5575bc060ce15feac HTTP/1.1" 502 150 2025-12-17 22:52:19,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea5575bc060ce15feac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:20,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea8575bc060ce15fec1 2025-12-17 22:52:20,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea8575bc060ce15fec1 HTTP/1.1" 502 150 2025-12-17 22:52:20,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea8575bc060ce15fec1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:20,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea9575bc060ce15fec8 2025-12-17 22:52:20,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea9575bc060ce15fec8 HTTP/1.1" 502 150 2025-12-17 22:52:20,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ea9575bc060ce15fec8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:21,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eaa575bc060ce15fecf 2025-12-17 22:52:21,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eaa575bc060ce15fecf HTTP/1.1" 502 150 2025-12-17 22:52:21,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eaa575bc060ce15fecf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:22,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e92575bc060ce15fe79 2025-12-17 22:52:22,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e92575bc060ce15fe79 HTTP/1.1" 502 150 2025-12-17 22:52:22,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e92575bc060ce15fe79'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:22,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eaf575bc060ce15fedb 2025-12-17 22:52:22,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eaf575bc060ce15fedb HTTP/1.1" 502 150 2025-12-17 22:52:22,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eaf575bc060ce15fedb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:22,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8f575bc060ce15fe73 2025-12-17 22:52:22,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8f575bc060ce15fe73 HTTP/1.1" 502 150 2025-12-17 22:52:22,788 - lava_callback - ERROR - Error processing callback for node 69408e8f575bc060ce15fe73: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e8f575bc060ce15fe73 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e8f575bc060ce15fe73 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e8f575bc060ce15fe73 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:22,790 - lava_callback - INFO - Starting async job submit for node 69408eae575bc060ce15fed8 2025-12-17 22:52:22,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:22,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eae575bc060ce15fed8 HTTP/1.1" 502 150 2025-12-17 22:52:22,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eae575bc060ce15fed8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:22,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eae575bc060ce15fed8 2025-12-17 22:52:22,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eae575bc060ce15fed8 HTTP/1.1" 502 150 2025-12-17 22:52:22,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eae575bc060ce15fed8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:22,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb2575bc060ce15fee1 2025-12-17 22:52:22,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb2575bc060ce15fee1 HTTP/1.1" 502 150 2025-12-17 22:52:22,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb2575bc060ce15fee1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:22,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eac575bc060ce15fed2 2025-12-17 22:52:22,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eac575bc060ce15fed2 HTTP/1.1" 502 150 2025-12-17 22:52:22,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eac575bc060ce15fed2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:23,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb0575bc060ce15fede 2025-12-17 22:52:23,432 - lava_callback - INFO - Received callback for node 694091a7575bc060ce1608f4 INFO: 172.21.0.1:44834 - "POST /node/694091a7575bc060ce1608f4 HTTP/1.0" 202 Accepted 2025-12-17 22:52:24,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eae575bc060ce15fed8 2025-12-17 22:52:24,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eae575bc060ce15fed8 HTTP/1.1" 502 150 2025-12-17 22:52:24,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eae575bc060ce15fed8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:24,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e93575bc060ce15fe7c 2025-12-17 22:52:24,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e93575bc060ce15fe7c HTTP/1.1" 502 150 2025-12-17 22:52:24,932 - lava_callback - ERROR - Error processing callback for node 69408e93575bc060ce15fe7c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e93575bc060ce15fe7c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e93575bc060ce15fe7c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e93575bc060ce15fe7c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:24,934 - lava_callback - INFO - Starting async job submit for node 69408eb3575bc060ce15ff3d 2025-12-17 22:52:24,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:24,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb3575bc060ce15ff3d HTTP/1.1" 502 150 2025-12-17 22:52:24,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb3575bc060ce15ff3d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:24,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb3575bc060ce15ff3d 2025-12-17 22:52:24,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb3575bc060ce15ff3d HTTP/1.1" 502 150 2025-12-17 22:52:24,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb3575bc060ce15ff3d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:25,856 - lava_callback - INFO - Received callback for node 69409116575bc060ce16077c INFO: 172.21.0.1:44842 - "POST /node/69409116575bc060ce16077c HTTP/1.0" 202 Accepted INFO: 172.21.0.1:44856 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:52:26,519 - lava_callback - INFO - Received callback for node 69409118575bc060ce160781 INFO: 172.21.0.1:44866 - "POST /node/69409118575bc060ce160781 HTTP/1.0" 202 Accepted 2025-12-17 22:52:26,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb3575bc060ce15ff3d 2025-12-17 22:52:26,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb3575bc060ce15ff3d HTTP/1.1" 502 150 2025-12-17 22:52:26,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb3575bc060ce15ff3d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:27,187 - lava_callback - INFO - Received callback for node 6940911d575bc060ce160789 INFO: 172.21.0.1:44878 - "POST /node/6940911d575bc060ce160789 HTTP/1.0" 202 Accepted 2025-12-17 22:52:27,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea3575bc060ce15fea9 2025-12-17 22:52:27,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea3575bc060ce15fea9 HTTP/1.1" 502 150 2025-12-17 22:52:27,493 - lava_callback - ERROR - Error processing callback for node 69408ea3575bc060ce15fea9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ea3575bc060ce15fea9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ea3575bc060ce15fea9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ea3575bc060ce15fea9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:27,495 - lava_callback - INFO - Starting async job submit for node 69408eb4575bc060ce15ff40 2025-12-17 22:52:27,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:27,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb4575bc060ce15ff40 HTTP/1.1" 502 150 2025-12-17 22:52:27,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb4575bc060ce15ff40'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:27,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb4575bc060ce15ff40 2025-12-17 22:52:27,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb4575bc060ce15ff40 HTTP/1.1" 502 150 2025-12-17 22:52:27,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb4575bc060ce15ff40'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:27,825 - lava_callback - INFO - Received callback for node 69409120575bc060ce16078c INFO: 172.21.0.1:44884 - "POST /node/69409120575bc060ce16078c HTTP/1.0" 202 Accepted 2025-12-17 22:52:28,466 - lava_callback - INFO - Received callback for node 69409122575bc060ce16078f INFO: 172.21.0.1:44888 - "POST /node/69409122575bc060ce16078f HTTP/1.0" 202 Accepted 2025-12-17 22:52:28,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eae575bc060ce15fed8 2025-12-17 22:52:28,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eae575bc060ce15fed8 HTTP/1.1" 502 150 2025-12-17 22:52:28,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eae575bc060ce15fed8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:29,122 - lava_callback - INFO - Received callback for node 69409123575bc060ce160792 INFO: 172.21.0.1:44896 - "POST /node/69409123575bc060ce160792 HTTP/1.0" 202 Accepted 2025-12-17 22:52:29,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ead575bc060ce15fed5 HTTP/1.1" 502 150 2025-12-17 22:52:29,232 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ead575bc060ce15fed5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:29,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb4575bc060ce15ff40 2025-12-17 22:52:29,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb4575bc060ce15ff40 HTTP/1.1" 502 150 2025-12-17 22:52:29,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb4575bc060ce15ff40'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:29,770 - lava_callback - INFO - Received callback for node 69409126575bc060ce160799 INFO: 172.21.0.1:55746 - "POST /node/69409126575bc060ce160799 HTTP/1.0" 202 Accepted 2025-12-17 22:52:30,404 - lava_callback - INFO - Received callback for node 69409127575bc060ce16079c INFO: 172.21.0.1:55748 - "POST /node/69409127575bc060ce16079c HTTP/1.0" 202 Accepted 2025-12-17 22:52:30,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e8c575bc060ce15fe6d 2025-12-17 22:52:30,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e8c575bc060ce15fe6d HTTP/1.1" 502 150 2025-12-17 22:52:30,868 - lava_callback - ERROR - Error processing callback for node 69408e8c575bc060ce15fe6d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e8c575bc060ce15fe6d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e8c575bc060ce15fe6d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e8c575bc060ce15fe6d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:30,871 - lava_callback - INFO - Starting async job submit for node 69408eb7575bc060ce15ff46 2025-12-17 22:52:30,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:30,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb7575bc060ce15ff46 HTTP/1.1" 502 150 2025-12-17 22:52:30,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb7575bc060ce15ff46'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:30,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb7575bc060ce15ff46 2025-12-17 22:52:30,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb7575bc060ce15ff46 HTTP/1.1" 502 150 2025-12-17 22:52:30,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb7575bc060ce15ff46'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:30,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb2575bc060ce15fee1 2025-12-17 22:52:30,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb2575bc060ce15fee1 HTTP/1.1" 502 150 2025-12-17 22:52:30,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb2575bc060ce15fee1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:30,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb3575bc060ce15ff3d 2025-12-17 22:52:30,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb3575bc060ce15ff3d HTTP/1.1" 502 150 2025-12-17 22:52:30,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb3575bc060ce15ff3d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:32,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb7575bc060ce15ff46 2025-12-17 22:52:32,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb7575bc060ce15ff46 HTTP/1.1" 502 150 2025-12-17 22:52:32,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb7575bc060ce15ff46'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:33,232 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ead575bc060ce15fed5 2025-12-17 22:52:33,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ead575bc060ce15fed5 HTTP/1.1" 502 150 2025-12-17 22:52:33,234 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ead575bc060ce15fed5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:33,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb4575bc060ce15ff40 2025-12-17 22:52:33,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb4575bc060ce15ff40 HTTP/1.1" 502 150 2025-12-17 22:52:33,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb4575bc060ce15ff40'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:34,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea6575bc060ce15febc 2025-12-17 22:52:34,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9e575bc060ce15fea6 2025-12-17 22:52:34,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9e575bc060ce15fea6 HTTP/1.1" 502 150 2025-12-17 22:52:34,925 - lava_callback - ERROR - Error processing callback for node 69408e9e575bc060ce15fea6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e9e575bc060ce15fea6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e9e575bc060ce15fea6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e9e575bc060ce15fea6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:34,927 - lava_callback - INFO - Starting async job submit for node 69408eb9575bc060ce15ff49 2025-12-17 22:52:34,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:34,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb9575bc060ce15ff49 HTTP/1.1" 502 150 2025-12-17 22:52:34,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb9575bc060ce15ff49'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:34,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb9575bc060ce15ff49 2025-12-17 22:52:34,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb9575bc060ce15ff49 HTTP/1.1" 502 150 2025-12-17 22:52:34,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb9575bc060ce15ff49'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:35,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea5575bc060ce15feac 2025-12-17 22:52:35,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea5575bc060ce15feac HTTP/1.1" 502 150 2025-12-17 22:52:35,553 - lava_callback - ERROR - Error processing callback for node 69408ea5575bc060ce15feac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ea5575bc060ce15feac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ea5575bc060ce15feac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ea5575bc060ce15feac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:35,555 - lava_callback - INFO - Starting async job submit for node 69408ebb575bc060ce15ff4c 2025-12-17 22:52:35,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:35,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebb575bc060ce15ff4c HTTP/1.1" 502 150 2025-12-17 22:52:35,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebb575bc060ce15ff4c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:35,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebb575bc060ce15ff4c 2025-12-17 22:52:35,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebb575bc060ce15ff4c HTTP/1.1" 502 150 2025-12-17 22:52:35,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebb575bc060ce15ff4c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55756 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:52:36,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea8575bc060ce15fec1 2025-12-17 22:52:36,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea8575bc060ce15fec1 HTTP/1.1" 502 150 2025-12-17 22:52:36,520 - lava_callback - ERROR - Error processing callback for node 69408ea8575bc060ce15fec1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ea8575bc060ce15fec1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ea8575bc060ce15fec1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ea8575bc060ce15fec1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:36,522 - lava_callback - INFO - Starting async job submit for node 69408ec9575bc060ce15ff6f 2025-12-17 22:52:36,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:36,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec9575bc060ce15ff6f HTTP/1.1" 502 150 2025-12-17 22:52:36,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec9575bc060ce15ff6f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:36,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec9575bc060ce15ff6f 2025-12-17 22:52:36,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec9575bc060ce15ff6f HTTP/1.1" 502 150 2025-12-17 22:52:36,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec9575bc060ce15ff6f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:36,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eae575bc060ce15fed8 2025-12-17 22:52:36,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eae575bc060ce15fed8 HTTP/1.1" 502 150 2025-12-17 22:52:36,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eae575bc060ce15fed8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:36,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ea9575bc060ce15fec8 2025-12-17 22:52:36,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea9575bc060ce15fec8 HTTP/1.1" 502 150 2025-12-17 22:52:36,870 - lava_callback - ERROR - Error processing callback for node 69408ea9575bc060ce15fec8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ea9575bc060ce15fec8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ea9575bc060ce15fec8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ea9575bc060ce15fec8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:36,872 - lava_callback - INFO - Starting async job submit for node 69408ecd575bc060ce15ff75 2025-12-17 22:52:36,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:36,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb7575bc060ce15ff46 2025-12-17 22:52:36,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb7575bc060ce15ff46 HTTP/1.1" 502 150 2025-12-17 22:52:36,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb7575bc060ce15ff46'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:36,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ecd575bc060ce15ff75 HTTP/1.1" 502 150 2025-12-17 22:52:36,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ecd575bc060ce15ff75'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:36,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ecd575bc060ce15ff75 2025-12-17 22:52:36,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ecd575bc060ce15ff75 HTTP/1.1" 502 150 2025-12-17 22:52:36,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ecd575bc060ce15ff75'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:36,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb9575bc060ce15ff49 2025-12-17 22:52:36,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb9575bc060ce15ff49 HTTP/1.1" 502 150 2025-12-17 22:52:36,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb9575bc060ce15ff49'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:37,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebb575bc060ce15ff4c 2025-12-17 22:52:37,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebb575bc060ce15ff4c HTTP/1.1" 502 150 2025-12-17 22:52:37,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebb575bc060ce15ff4c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:37,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eaa575bc060ce15fecf 2025-12-17 22:52:37,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eaa575bc060ce15fecf HTTP/1.1" 502 150 2025-12-17 22:52:37,954 - lava_callback - ERROR - Error processing callback for node 69408eaa575bc060ce15fecf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eaa575bc060ce15fecf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eaa575bc060ce15fecf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eaa575bc060ce15fecf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:37,955 - lava_callback - INFO - Starting async job submit for node 69408ebc575bc060ce15ff51 2025-12-17 22:52:37,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:37,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebc575bc060ce15ff51 HTTP/1.1" 502 150 2025-12-17 22:52:37,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebc575bc060ce15ff51'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:37,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebc575bc060ce15ff51 2025-12-17 22:52:37,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebc575bc060ce15ff51 HTTP/1.1" 502 150 2025-12-17 22:52:37,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebc575bc060ce15ff51'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:38,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb0575bc060ce15fede HTTP/1.1" 502 150 2025-12-17 22:52:38,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb0575bc060ce15fede'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:38,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec9575bc060ce15ff6f 2025-12-17 22:52:38,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec9575bc060ce15ff6f HTTP/1.1" 502 150 2025-12-17 22:52:38,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec9575bc060ce15ff6f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:38,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e92575bc060ce15fe79 2025-12-17 22:52:38,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e92575bc060ce15fe79 HTTP/1.1" 502 150 2025-12-17 22:52:38,643 - lava_callback - ERROR - Error processing callback for node 69408e92575bc060ce15fe79: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e92575bc060ce15fe79 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e92575bc060ce15fe79 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e92575bc060ce15fe79 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:38,645 - lava_callback - INFO - Starting async job submit for node 69408ebe575bc060ce15ff5a 2025-12-17 22:52:38,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:38,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebe575bc060ce15ff5a HTTP/1.1" 502 150 2025-12-17 22:52:38,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebe575bc060ce15ff5a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:38,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebe575bc060ce15ff5a 2025-12-17 22:52:38,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebe575bc060ce15ff5a HTTP/1.1" 502 150 2025-12-17 22:52:38,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebe575bc060ce15ff5a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:38,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eaf575bc060ce15fedb 2025-12-17 22:52:38,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eaf575bc060ce15fedb HTTP/1.1" 502 150 2025-12-17 22:52:38,724 - lava_callback - ERROR - Error processing callback for node 69408eaf575bc060ce15fedb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eaf575bc060ce15fedb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eaf575bc060ce15fedb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eaf575bc060ce15fedb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:38,726 - lava_callback - INFO - Starting async job submit for node 69408ebf575bc060ce15ff60 2025-12-17 22:52:38,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:38,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebf575bc060ce15ff60 HTTP/1.1" 502 150 2025-12-17 22:52:38,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebf575bc060ce15ff60'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:38,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebf575bc060ce15ff60 2025-12-17 22:52:38,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebf575bc060ce15ff60 HTTP/1.1" 502 150 2025-12-17 22:52:38,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebf575bc060ce15ff60'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:38,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ecd575bc060ce15ff75 2025-12-17 22:52:38,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ecd575bc060ce15ff75 HTTP/1.1" 502 150 2025-12-17 22:52:38,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ecd575bc060ce15ff75'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:38,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eac575bc060ce15fed2 2025-12-17 22:52:38,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eac575bc060ce15fed2 HTTP/1.1" 502 150 2025-12-17 22:52:38,925 - lava_callback - ERROR - Error processing callback for node 69408eac575bc060ce15fed2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eac575bc060ce15fed2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eac575bc060ce15fed2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eac575bc060ce15fed2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:38,927 - lava_callback - INFO - Starting async job submit for node 69408ec1575bc060ce15ff63 2025-12-17 22:52:38,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:38,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb3575bc060ce15ff3d 2025-12-17 22:52:38,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb3575bc060ce15ff3d HTTP/1.1" 502 150 2025-12-17 22:52:38,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb3575bc060ce15ff3d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:38,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec1575bc060ce15ff63 HTTP/1.1" 502 150 2025-12-17 22:52:38,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec1575bc060ce15ff63'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:38,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec1575bc060ce15ff63 2025-12-17 22:52:38,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec1575bc060ce15ff63 HTTP/1.1" 502 150 2025-12-17 22:52:38,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec1575bc060ce15ff63'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:39,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebc575bc060ce15ff51 2025-12-17 22:52:39,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebc575bc060ce15ff51 HTTP/1.1" 502 150 2025-12-17 22:52:39,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebc575bc060ce15ff51'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:40,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebe575bc060ce15ff5a 2025-12-17 22:52:40,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebe575bc060ce15ff5a HTTP/1.1" 502 150 2025-12-17 22:52:40,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebe575bc060ce15ff5a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:40,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebf575bc060ce15ff60 2025-12-17 22:52:40,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebf575bc060ce15ff60 HTTP/1.1" 502 150 2025-12-17 22:52:40,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebf575bc060ce15ff60'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:40,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb9575bc060ce15ff49 2025-12-17 22:52:40,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb9575bc060ce15ff49 HTTP/1.1" 502 150 2025-12-17 22:52:40,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb9575bc060ce15ff49'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:40,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec1575bc060ce15ff63 2025-12-17 22:52:40,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec1575bc060ce15ff63 HTTP/1.1" 502 150 2025-12-17 22:52:40,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec1575bc060ce15ff63'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:41,234 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ead575bc060ce15fed5 2025-12-17 22:52:41,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ead575bc060ce15fed5 HTTP/1.1" 502 150 2025-12-17 22:52:41,235 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ead575bc060ce15fed5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:41,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb4575bc060ce15ff40 2025-12-17 22:52:41,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb4575bc060ce15ff40 HTTP/1.1" 502 150 2025-12-17 22:52:41,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb4575bc060ce15ff40'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:41,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebb575bc060ce15ff4c 2025-12-17 22:52:41,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebb575bc060ce15ff4c HTTP/1.1" 502 150 2025-12-17 22:52:41,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebb575bc060ce15ff4c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:42,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec9575bc060ce15ff6f 2025-12-17 22:52:42,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec9575bc060ce15ff6f HTTP/1.1" 502 150 2025-12-17 22:52:42,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec9575bc060ce15ff6f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:42,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ecd575bc060ce15ff75 2025-12-17 22:52:42,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ecd575bc060ce15ff75 HTTP/1.1" 502 150 2025-12-17 22:52:42,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ecd575bc060ce15ff75'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:43,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebc575bc060ce15ff51 2025-12-17 22:52:43,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebc575bc060ce15ff51 HTTP/1.1" 502 150 2025-12-17 22:52:43,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebc575bc060ce15ff51'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:44,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebe575bc060ce15ff5a 2025-12-17 22:52:44,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebe575bc060ce15ff5a HTTP/1.1" 502 150 2025-12-17 22:52:44,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebe575bc060ce15ff5a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:44,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebf575bc060ce15ff60 2025-12-17 22:52:44,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebf575bc060ce15ff60 HTTP/1.1" 502 150 2025-12-17 22:52:44,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebf575bc060ce15ff60'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:44,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb7575bc060ce15ff46 2025-12-17 22:52:44,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb7575bc060ce15ff46 HTTP/1.1" 502 150 2025-12-17 22:52:44,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb7575bc060ce15ff46'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:44,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec1575bc060ce15ff63 2025-12-17 22:52:44,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec1575bc060ce15ff63 HTTP/1.1" 502 150 2025-12-17 22:52:44,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec1575bc060ce15ff63'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:45,884 - lava_callback - INFO - Received callback for node 69409129575bc060ce1607a5 INFO: 172.21.0.1:40384 - "POST /node/69409129575bc060ce1607a5 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:40392 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:52:46,551 - lava_callback - INFO - Received callback for node 6940912c575bc060ce1607ac INFO: 172.21.0.1:40402 - "POST /node/6940912c575bc060ce1607ac HTTP/1.0" 202 Accepted 2025-12-17 22:52:46,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb2575bc060ce15fee1 2025-12-17 22:52:46,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb2575bc060ce15fee1 HTTP/1.1" 502 150 2025-12-17 22:52:46,905 - lava_callback - ERROR - Error processing callback for node 69408eb2575bc060ce15fee1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb2575bc060ce15fee1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb2575bc060ce15fee1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb2575bc060ce15fee1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:46,907 - lava_callback - INFO - Starting async job submit for node 69408ec4575bc060ce15ff66 2025-12-17 22:52:46,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:46,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec4575bc060ce15ff66 HTTP/1.1" 502 150 2025-12-17 22:52:46,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec4575bc060ce15ff66'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:46,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec4575bc060ce15ff66 2025-12-17 22:52:46,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec4575bc060ce15ff66 HTTP/1.1" 502 150 2025-12-17 22:52:46,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec4575bc060ce15ff66'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:47,217 - lava_callback - INFO - Received callback for node 6940912d575bc060ce1607af INFO: 172.21.0.1:40408 - "POST /node/6940912d575bc060ce1607af HTTP/1.0" 202 Accepted 2025-12-17 22:52:47,882 - lava_callback - INFO - Received callback for node 69409130575bc060ce1607b5 INFO: 172.21.0.1:40414 - "POST /node/69409130575bc060ce1607b5 HTTP/1.0" 202 Accepted 2025-12-17 22:52:48,530 - lava_callback - INFO - Received callback for node 69409132575bc060ce1607b8 INFO: 172.21.0.1:40418 - "POST /node/69409132575bc060ce1607b8 HTTP/1.0" 202 Accepted 2025-12-17 22:52:48,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec4575bc060ce15ff66 2025-12-17 22:52:48,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec4575bc060ce15ff66 HTTP/1.1" 502 150 2025-12-17 22:52:48,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec4575bc060ce15ff66'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:48,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb9575bc060ce15ff49 2025-12-17 22:52:48,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb9575bc060ce15ff49 HTTP/1.1" 502 150 2025-12-17 22:52:48,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eb9575bc060ce15ff49'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:49,192 - lava_callback - INFO - Received callback for node 69409134575bc060ce1607bb INFO: 172.21.0.1:40428 - "POST /node/69409134575bc060ce1607bb HTTP/1.0" 202 Accepted 2025-12-17 22:52:49,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebb575bc060ce15ff4c 2025-12-17 22:52:49,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebb575bc060ce15ff4c HTTP/1.1" 502 150 2025-12-17 22:52:49,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebb575bc060ce15ff4c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:50,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ea6575bc060ce15febc HTTP/1.1" 502 150 2025-12-17 22:52:50,224 - lava_callback - ERROR - Error processing callback for node 69408ea6575bc060ce15febc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ea6575bc060ce15febc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ea6575bc060ce15febc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ea6575bc060ce15febc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:50,226 - lava_callback - INFO - Starting async job submit for node 69408ec6575bc060ce15ff69 2025-12-17 22:52:50,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:50,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec6575bc060ce15ff69 HTTP/1.1" 502 150 2025-12-17 22:52:50,233 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec6575bc060ce15ff69'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:50,233 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec6575bc060ce15ff69 2025-12-17 22:52:50,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec6575bc060ce15ff69 HTTP/1.1" 502 150 2025-12-17 22:52:50,234 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec6575bc060ce15ff69'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:50,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec9575bc060ce15ff6f 2025-12-17 22:52:50,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec9575bc060ce15ff6f HTTP/1.1" 502 150 2025-12-17 22:52:50,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec9575bc060ce15ff6f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:50,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ecd575bc060ce15ff75 2025-12-17 22:52:50,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ecd575bc060ce15ff75 HTTP/1.1" 502 150 2025-12-17 22:52:50,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ecd575bc060ce15ff75'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:51,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebc575bc060ce15ff51 2025-12-17 22:52:51,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebc575bc060ce15ff51 HTTP/1.1" 502 150 2025-12-17 22:52:51,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebc575bc060ce15ff51'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:52,234 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec6575bc060ce15ff69 2025-12-17 22:52:52,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebe575bc060ce15ff5a 2025-12-17 22:52:52,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebe575bc060ce15ff5a HTTP/1.1" 502 150 2025-12-17 22:52:52,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebe575bc060ce15ff5a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:52,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebf575bc060ce15ff60 2025-12-17 22:52:52,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebf575bc060ce15ff60 HTTP/1.1" 502 150 2025-12-17 22:52:52,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ebf575bc060ce15ff60'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:52,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eae575bc060ce15fed8 2025-12-17 22:52:52,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eae575bc060ce15fed8 HTTP/1.1" 502 150 2025-12-17 22:52:52,826 - lava_callback - ERROR - Error processing callback for node 69408eae575bc060ce15fed8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eae575bc060ce15fed8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eae575bc060ce15fed8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eae575bc060ce15fed8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:52,828 - lava_callback - INFO - Starting async job submit for node 69408ec8575bc060ce15ff6c 2025-12-17 22:52:52,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:52,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec8575bc060ce15ff6c HTTP/1.1" 502 150 2025-12-17 22:52:52,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec8575bc060ce15ff6c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:52,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec8575bc060ce15ff6c 2025-12-17 22:52:52,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec8575bc060ce15ff6c HTTP/1.1" 502 150 2025-12-17 22:52:52,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec8575bc060ce15ff6c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:52,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec4575bc060ce15ff66 2025-12-17 22:52:52,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec4575bc060ce15ff66 HTTP/1.1" 502 150 2025-12-17 22:52:52,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec4575bc060ce15ff66'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:52,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec1575bc060ce15ff63 2025-12-17 22:52:52,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec1575bc060ce15ff63 HTTP/1.1" 502 150 2025-12-17 22:52:52,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec1575bc060ce15ff63'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:54,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb0575bc060ce15fede 2025-12-17 22:52:54,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb0575bc060ce15fede HTTP/1.1" 502 150 2025-12-17 22:52:54,450 - lava_callback - ERROR - Error processing callback for node 69408eb0575bc060ce15fede: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb0575bc060ce15fede (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb0575bc060ce15fede (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb0575bc060ce15fede (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:54,452 - lava_callback - INFO - Starting async job submit for node 69408ecb575bc060ce15ff72 2025-12-17 22:52:54,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:54,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ecb575bc060ce15ff72 HTTP/1.1" 502 150 2025-12-17 22:52:54,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ecb575bc060ce15ff72'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:54,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ecb575bc060ce15ff72 2025-12-17 22:52:54,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ecb575bc060ce15ff72 HTTP/1.1" 502 150 2025-12-17 22:52:54,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ecb575bc060ce15ff72'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:54,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec8575bc060ce15ff6c 2025-12-17 22:52:54,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec8575bc060ce15ff6c HTTP/1.1" 502 150 2025-12-17 22:52:54,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec8575bc060ce15ff6c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:54,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb3575bc060ce15ff3d 2025-12-17 22:52:54,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb3575bc060ce15ff3d HTTP/1.1" 502 150 2025-12-17 22:52:54,952 - lava_callback - ERROR - Error processing callback for node 69408eb3575bc060ce15ff3d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb3575bc060ce15ff3d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb3575bc060ce15ff3d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb3575bc060ce15ff3d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:54,954 - lava_callback - INFO - Starting async job submit for node 69408ece575bc060ce15ff78 2025-12-17 22:52:54,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:54,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ece575bc060ce15ff78 HTTP/1.1" 502 150 2025-12-17 22:52:54,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ece575bc060ce15ff78'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:54,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ece575bc060ce15ff78 2025-12-17 22:52:54,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ece575bc060ce15ff78 HTTP/1.1" 502 150 2025-12-17 22:52:54,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ece575bc060ce15ff78'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:55,522 - lava_callback - INFO - Received callback for node 69409136575bc060ce1607be INFO: 172.21.0.1:60988 - "POST /node/69409136575bc060ce1607be HTTP/1.0" 202 Accepted 2025-12-17 22:52:56,195 - lava_callback - INFO - Received callback for node 6940913b575bc060ce1607c4 INFO: 172.21.0.1:60990 - "POST /node/6940913b575bc060ce1607c4 HTTP/1.0" 202 Accepted 2025-12-17 22:52:56,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ecb575bc060ce15ff72 2025-12-17 22:52:56,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ecb575bc060ce15ff72 HTTP/1.1" 502 150 2025-12-17 22:52:56,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ecb575bc060ce15ff72'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60996 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:52:56,875 - lava_callback - INFO - Received callback for node 6940913d575bc060ce1607cb INFO: 172.21.0.1:32776 - "POST /node/6940913d575bc060ce1607cb HTTP/1.0" 202 Accepted 2025-12-17 22:52:56,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ece575bc060ce15ff78 2025-12-17 22:52:56,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ece575bc060ce15ff78 HTTP/1.1" 502 150 2025-12-17 22:52:56,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ece575bc060ce15ff78'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:57,235 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ead575bc060ce15fed5 2025-12-17 22:52:57,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ead575bc060ce15fed5 HTTP/1.1" 502 150 2025-12-17 22:52:57,237 - lava_callback - ERROR - Error processing callback for node 69408ead575bc060ce15fed5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ead575bc060ce15fed5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ead575bc060ce15fed5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ead575bc060ce15fed5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:57,240 - lava_callback - INFO - Starting async job submit for node 69408ed0575bc060ce15ff7d 2025-12-17 22:52:57,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:57,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed0575bc060ce15ff7d HTTP/1.1" 502 150 2025-12-17 22:52:57,247 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed0575bc060ce15ff7d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:57,247 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed0575bc060ce15ff7d 2025-12-17 22:52:57,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed0575bc060ce15ff7d HTTP/1.1" 502 150 2025-12-17 22:52:57,248 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed0575bc060ce15ff7d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:57,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb4575bc060ce15ff40 2025-12-17 22:52:57,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb4575bc060ce15ff40 HTTP/1.1" 502 150 2025-12-17 22:52:57,533 - lava_callback - ERROR - Error processing callback for node 69408eb4575bc060ce15ff40: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb4575bc060ce15ff40 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb4575bc060ce15ff40 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb4575bc060ce15ff40 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:52:57,534 - lava_callback - INFO - Starting async job submit for node 69408ed1575bc060ce15ff84 2025-12-17 22:52:57,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:52:57,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed1575bc060ce15ff84 HTTP/1.1" 502 150 2025-12-17 22:52:57,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed1575bc060ce15ff84'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:57,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed1575bc060ce15ff84 2025-12-17 22:52:57,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed1575bc060ce15ff84 HTTP/1.1" 502 150 2025-12-17 22:52:57,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed1575bc060ce15ff84'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:57,552 - lava_callback - INFO - Received callback for node 6940913f575bc060ce1607d2 INFO: 172.21.0.1:32778 - "POST /node/6940913f575bc060ce1607d2 HTTP/1.0" 202 Accepted 2025-12-17 22:52:58,210 - lava_callback - INFO - Received callback for node 69409141575bc060ce1607e3 INFO: 172.21.0.1:32792 - "POST /node/69409141575bc060ce1607e3 HTTP/1.0" 202 Accepted 2025-12-17 22:52:58,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec8575bc060ce15ff6c 2025-12-17 22:52:58,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec8575bc060ce15ff6c HTTP/1.1" 502 150 2025-12-17 22:52:58,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec8575bc060ce15ff6c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:59,248 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed0575bc060ce15ff7d 2025-12-17 22:52:59,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed0575bc060ce15ff7d HTTP/1.1" 502 150 2025-12-17 22:52:59,249 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed0575bc060ce15ff7d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:52:59,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed1575bc060ce15ff84 2025-12-17 22:52:59,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed1575bc060ce15ff84 HTTP/1.1" 502 150 2025-12-17 22:52:59,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed1575bc060ce15ff84'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:00,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ecb575bc060ce15ff72 2025-12-17 22:53:00,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ecb575bc060ce15ff72 HTTP/1.1" 502 150 2025-12-17 22:53:00,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ecb575bc060ce15ff72'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:00,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb7575bc060ce15ff46 2025-12-17 22:53:00,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb7575bc060ce15ff46 HTTP/1.1" 502 150 2025-12-17 22:53:00,889 - lava_callback - ERROR - Error processing callback for node 69408eb7575bc060ce15ff46: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb7575bc060ce15ff46 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb7575bc060ce15ff46 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb7575bc060ce15ff46 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:00,891 - lava_callback - INFO - Starting async job submit for node 69408ed3575bc060ce15ff8b 2025-12-17 22:53:00,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:00,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed3575bc060ce15ff8b HTTP/1.1" 502 150 2025-12-17 22:53:00,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed3575bc060ce15ff8b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:00,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed3575bc060ce15ff8b 2025-12-17 22:53:00,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed3575bc060ce15ff8b HTTP/1.1" 502 150 2025-12-17 22:53:00,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed3575bc060ce15ff8b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:00,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec4575bc060ce15ff66 2025-12-17 22:53:00,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec4575bc060ce15ff66 HTTP/1.1" 502 150 2025-12-17 22:53:00,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec4575bc060ce15ff66'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:00,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ece575bc060ce15ff78 2025-12-17 22:53:00,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ece575bc060ce15ff78 HTTP/1.1" 502 150 2025-12-17 22:53:00,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ece575bc060ce15ff78'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42930 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:53:02,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed3575bc060ce15ff8b 2025-12-17 22:53:02,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed3575bc060ce15ff8b HTTP/1.1" 502 150 2025-12-17 22:53:02,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed3575bc060ce15ff8b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:03,249 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed0575bc060ce15ff7d 2025-12-17 22:53:03,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed0575bc060ce15ff7d HTTP/1.1" 502 150 2025-12-17 22:53:03,251 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed0575bc060ce15ff7d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:03,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed1575bc060ce15ff84 2025-12-17 22:53:03,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed1575bc060ce15ff84 HTTP/1.1" 502 150 2025-12-17 22:53:03,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed1575bc060ce15ff84'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:04,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eb9575bc060ce15ff49 2025-12-17 22:53:04,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eb9575bc060ce15ff49 HTTP/1.1" 502 150 2025-12-17 22:53:04,942 - lava_callback - ERROR - Error processing callback for node 69408eb9575bc060ce15ff49: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb9575bc060ce15ff49 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb9575bc060ce15ff49 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eb9575bc060ce15ff49 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:04,944 - lava_callback - INFO - Starting async job submit for node 69408ed6575bc060ce15ff95 2025-12-17 22:53:04,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:04,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed6575bc060ce15ff95 HTTP/1.1" 502 150 2025-12-17 22:53:04,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed6575bc060ce15ff95'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:04,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed6575bc060ce15ff95 2025-12-17 22:53:04,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed6575bc060ce15ff95 HTTP/1.1" 502 150 2025-12-17 22:53:04,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed6575bc060ce15ff95'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:05,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebb575bc060ce15ff4c 2025-12-17 22:53:05,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebb575bc060ce15ff4c HTTP/1.1" 502 150 2025-12-17 22:53:05,572 - lava_callback - ERROR - Error processing callback for node 69408ebb575bc060ce15ff4c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ebb575bc060ce15ff4c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ebb575bc060ce15ff4c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ebb575bc060ce15ff4c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:05,575 - lava_callback - INFO - Starting async job submit for node 69408eda575bc060ce15ff9e 2025-12-17 22:53:05,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:05,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eda575bc060ce15ff9e HTTP/1.1" 502 150 2025-12-17 22:53:05,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eda575bc060ce15ff9e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:05,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eda575bc060ce15ff9e 2025-12-17 22:53:05,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eda575bc060ce15ff9e HTTP/1.1" 502 150 2025-12-17 22:53:05,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eda575bc060ce15ff9e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:05,873 - lava_callback - INFO - Received callback for node 6940914a575bc060ce1607f5 INFO: 172.21.0.1:42932 - "POST /node/6940914a575bc060ce1607f5 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:42940 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:53:06,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec9575bc060ce15ff6f 2025-12-17 22:53:06,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec8575bc060ce15ff6c 2025-12-17 22:53:06,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec8575bc060ce15ff6c HTTP/1.1" 502 150 2025-12-17 22:53:06,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec8575bc060ce15ff6c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:06,854 - lava_callback - INFO - Received callback for node 69409144575bc060ce1607e6 INFO: 172.21.0.1:42942 - "POST /node/69409144575bc060ce1607e6 HTTP/1.0" 202 Accepted 2025-12-17 22:53:06,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed3575bc060ce15ff8b 2025-12-17 22:53:06,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed3575bc060ce15ff8b HTTP/1.1" 502 150 2025-12-17 22:53:06,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed3575bc060ce15ff8b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:06,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ecd575bc060ce15ff75 2025-12-17 22:53:06,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ecd575bc060ce15ff75 HTTP/1.1" 502 150 2025-12-17 22:53:06,914 - lava_callback - ERROR - Error processing callback for node 69408ecd575bc060ce15ff75: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ecd575bc060ce15ff75 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ecd575bc060ce15ff75 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ecd575bc060ce15ff75 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:06,916 - lava_callback - INFO - Starting async job submit for node 69408ede575bc060ce15ffa7 2025-12-17 22:53:06,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:06,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ede575bc060ce15ffa7 HTTP/1.1" 502 150 2025-12-17 22:53:06,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ede575bc060ce15ffa7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:06,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ede575bc060ce15ffa7 2025-12-17 22:53:06,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ede575bc060ce15ffa7 HTTP/1.1" 502 150 2025-12-17 22:53:06,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ede575bc060ce15ffa7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:06,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed6575bc060ce15ff95 2025-12-17 22:53:06,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed6575bc060ce15ff95 HTTP/1.1" 502 150 2025-12-17 22:53:06,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed6575bc060ce15ff95'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:07,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eda575bc060ce15ff9e 2025-12-17 22:53:07,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eda575bc060ce15ff9e HTTP/1.1" 502 150 2025-12-17 22:53:07,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eda575bc060ce15ff9e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:07,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec6575bc060ce15ff69 HTTP/1.1" 502 150 2025-12-17 22:53:07,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec6575bc060ce15ff69'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:07,721 - lava_callback - INFO - Received callback for node 69409145575bc060ce1607e9 INFO: 172.21.0.1:42956 - "POST /node/69409145575bc060ce1607e9 HTTP/1.0" 202 Accepted 2025-12-17 22:53:07,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebc575bc060ce15ff51 2025-12-17 22:53:07,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebc575bc060ce15ff51 HTTP/1.1" 502 150 2025-12-17 22:53:07,971 - lava_callback - ERROR - Error processing callback for node 69408ebc575bc060ce15ff51: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ebc575bc060ce15ff51 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ebc575bc060ce15ff51 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ebc575bc060ce15ff51 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:07,973 - lava_callback - INFO - Starting async job submit for node 69408ed8575bc060ce15ff9b 2025-12-17 22:53:07,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:07,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed8575bc060ce15ff9b HTTP/1.1" 502 150 2025-12-17 22:53:07,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed8575bc060ce15ff9b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:07,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed8575bc060ce15ff9b 2025-12-17 22:53:07,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed8575bc060ce15ff9b HTTP/1.1" 502 150 2025-12-17 22:53:07,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed8575bc060ce15ff9b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:08,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ecb575bc060ce15ff72 2025-12-17 22:53:08,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ecb575bc060ce15ff72 HTTP/1.1" 502 150 2025-12-17 22:53:08,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ecb575bc060ce15ff72'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:08,497 - lava_callback - INFO - Received callback for node 69409147575bc060ce1607ec INFO: 172.21.0.1:42970 - "POST /node/69409147575bc060ce1607ec HTTP/1.0" 202 Accepted 2025-12-17 22:53:08,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebe575bc060ce15ff5a 2025-12-17 22:53:08,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebe575bc060ce15ff5a HTTP/1.1" 502 150 2025-12-17 22:53:08,687 - lava_callback - ERROR - Error processing callback for node 69408ebe575bc060ce15ff5a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ebe575bc060ce15ff5a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ebe575bc060ce15ff5a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ebe575bc060ce15ff5a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:08,689 - lava_callback - INFO - Starting async job submit for node 6940946f575bc060ce16103c 2025-12-17 22:53:08,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:08,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946f575bc060ce16103c HTTP/1.1" 502 150 2025-12-17 22:53:08,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940946f575bc060ce16103c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:08,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940946f575bc060ce16103c 2025-12-17 22:53:08,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946f575bc060ce16103c HTTP/1.1" 502 150 2025-12-17 22:53:08,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940946f575bc060ce16103c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:08,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ebf575bc060ce15ff60 2025-12-17 22:53:08,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ebf575bc060ce15ff60 HTTP/1.1" 502 150 2025-12-17 22:53:08,770 - lava_callback - ERROR - Error processing callback for node 69408ebf575bc060ce15ff60: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ebf575bc060ce15ff60 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ebf575bc060ce15ff60 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ebf575bc060ce15ff60 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:08,772 - lava_callback - INFO - Starting async job submit for node 69408ed7575bc060ce15ff98 2025-12-17 22:53:08,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:08,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed7575bc060ce15ff98 HTTP/1.1" 502 150 2025-12-17 22:53:08,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed7575bc060ce15ff98'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:08,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed7575bc060ce15ff98 2025-12-17 22:53:08,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed7575bc060ce15ff98 HTTP/1.1" 502 150 2025-12-17 22:53:08,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed7575bc060ce15ff98'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:08,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ede575bc060ce15ffa7 2025-12-17 22:53:08,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ede575bc060ce15ffa7 HTTP/1.1" 502 150 2025-12-17 22:53:08,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ede575bc060ce15ffa7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:08,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec1575bc060ce15ff63 2025-12-17 22:53:08,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec1575bc060ce15ff63 HTTP/1.1" 502 150 2025-12-17 22:53:08,958 - lava_callback - ERROR - Error processing callback for node 69408ec1575bc060ce15ff63: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ec1575bc060ce15ff63 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ec1575bc060ce15ff63 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ec1575bc060ce15ff63 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:08,960 - lava_callback - INFO - Starting async job submit for node 69409251575bc060ce160a72 2025-12-17 22:53:08,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ece575bc060ce15ff78 2025-12-17 22:53:08,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:08,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ece575bc060ce15ff78 HTTP/1.1" 502 150 2025-12-17 22:53:08,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ece575bc060ce15ff78'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:08,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409251575bc060ce160a72 HTTP/1.1" 502 150 2025-12-17 22:53:08,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409251575bc060ce160a72'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:08,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409251575bc060ce160a72 2025-12-17 22:53:08,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409251575bc060ce160a72 HTTP/1.1" 502 150 2025-12-17 22:53:08,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409251575bc060ce160a72'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:09,313 - lava_callback - INFO - Received callback for node 69409148575bc060ce1607ef INFO: 172.21.0.1:42980 - "POST /node/69409148575bc060ce1607ef HTTP/1.0" 202 Accepted 2025-12-17 22:53:09,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed8575bc060ce15ff9b 2025-12-17 22:53:09,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed8575bc060ce15ff9b HTTP/1.1" 502 150 2025-12-17 22:53:09,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed8575bc060ce15ff9b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:10,041 - lava_callback - INFO - Received callback for node 69409149575bc060ce1607f2 INFO: 172.21.0.1:60208 - "POST /node/69409149575bc060ce1607f2 HTTP/1.0" 202 Accepted 2025-12-17 22:53:10,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940946f575bc060ce16103c 2025-12-17 22:53:10,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946f575bc060ce16103c HTTP/1.1" 502 150 2025-12-17 22:53:10,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940946f575bc060ce16103c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:10,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed7575bc060ce15ff98 2025-12-17 22:53:10,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed7575bc060ce15ff98 HTTP/1.1" 502 150 2025-12-17 22:53:10,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed7575bc060ce15ff98'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:10,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed6575bc060ce15ff95 2025-12-17 22:53:10,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed6575bc060ce15ff95 HTTP/1.1" 502 150 2025-12-17 22:53:10,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed6575bc060ce15ff95'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:10,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409251575bc060ce160a72 2025-12-17 22:53:10,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409251575bc060ce160a72 HTTP/1.1" 502 150 2025-12-17 22:53:10,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409251575bc060ce160a72'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:11,251 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed0575bc060ce15ff7d 2025-12-17 22:53:11,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed0575bc060ce15ff7d HTTP/1.1" 502 150 2025-12-17 22:53:11,252 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed0575bc060ce15ff7d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:11,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed1575bc060ce15ff84 2025-12-17 22:53:11,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed1575bc060ce15ff84 HTTP/1.1" 502 150 2025-12-17 22:53:11,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed1575bc060ce15ff84'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:11,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eda575bc060ce15ff9e 2025-12-17 22:53:11,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eda575bc060ce15ff9e HTTP/1.1" 502 150 2025-12-17 22:53:11,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eda575bc060ce15ff9e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:11,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec6575bc060ce15ff69 2025-12-17 22:53:11,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec6575bc060ce15ff69 HTTP/1.1" 502 150 2025-12-17 22:53:11,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec6575bc060ce15ff69'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:12,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ede575bc060ce15ffa7 2025-12-17 22:53:12,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ede575bc060ce15ffa7 HTTP/1.1" 502 150 2025-12-17 22:53:12,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ede575bc060ce15ffa7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:13,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed8575bc060ce15ff9b 2025-12-17 22:53:13,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed8575bc060ce15ff9b HTTP/1.1" 502 150 2025-12-17 22:53:13,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed8575bc060ce15ff9b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:14,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940946f575bc060ce16103c 2025-12-17 22:53:14,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946f575bc060ce16103c HTTP/1.1" 502 150 2025-12-17 22:53:14,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940946f575bc060ce16103c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:14,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed7575bc060ce15ff98 2025-12-17 22:53:14,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed7575bc060ce15ff98 HTTP/1.1" 502 150 2025-12-17 22:53:14,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed7575bc060ce15ff98'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:14,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed3575bc060ce15ff8b 2025-12-17 22:53:14,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed3575bc060ce15ff8b HTTP/1.1" 502 150 2025-12-17 22:53:14,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed3575bc060ce15ff8b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:14,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409251575bc060ce160a72 2025-12-17 22:53:14,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409251575bc060ce160a72 HTTP/1.1" 502 150 2025-12-17 22:53:14,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409251575bc060ce160a72'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60212 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:53:15,508 - lava_callback - INFO - Received callback for node 69409153575bc060ce160807 INFO: 172.21.0.1:60220 - "POST /node/69409153575bc060ce160807 HTTP/1.0" 202 Accepted 2025-12-17 22:53:15,722 - lava_callback - INFO - Received callback for node 694091a5575bc060ce1608e9 INFO: 172.21.0.1:60230 - "POST /node/694091a5575bc060ce1608e9 HTTP/1.0" 202 Accepted 2025-12-17 22:53:16,156 - lava_callback - INFO - Received callback for node 69409159575bc060ce16080e INFO: 172.21.0.1:60240 - "POST /node/69409159575bc060ce16080e HTTP/1.0" 202 Accepted INFO: 172.21.0.1:60252 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:53:16,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec4575bc060ce15ff66 2025-12-17 22:53:16,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec4575bc060ce15ff66 HTTP/1.1" 502 150 2025-12-17 22:53:16,921 - lava_callback - ERROR - Error processing callback for node 69408ec4575bc060ce15ff66: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ec4575bc060ce15ff66 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ec4575bc060ce15ff66 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ec4575bc060ce15ff66 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:16,922 - lava_callback - INFO - Starting async job submit for node 69408ee4575bc060ce15ffba 2025-12-17 22:53:16,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:16,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee4575bc060ce15ffba HTTP/1.1" 502 150 2025-12-17 22:53:16,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee4575bc060ce15ffba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:16,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee4575bc060ce15ffba 2025-12-17 22:53:16,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee4575bc060ce15ffba HTTP/1.1" 502 150 2025-12-17 22:53:16,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee4575bc060ce15ffba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:18,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee4575bc060ce15ffba 2025-12-17 22:53:18,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee4575bc060ce15ffba HTTP/1.1" 502 150 2025-12-17 22:53:18,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee4575bc060ce15ffba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:18,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed6575bc060ce15ff95 2025-12-17 22:53:18,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed6575bc060ce15ff95 HTTP/1.1" 502 150 2025-12-17 22:53:18,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed6575bc060ce15ff95'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:19,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eda575bc060ce15ff9e 2025-12-17 22:53:19,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eda575bc060ce15ff9e HTTP/1.1" 502 150 2025-12-17 22:53:19,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eda575bc060ce15ff9e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:19,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec6575bc060ce15ff69 2025-12-17 22:53:19,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec6575bc060ce15ff69 HTTP/1.1" 502 150 2025-12-17 22:53:19,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ec6575bc060ce15ff69'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:20,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ede575bc060ce15ffa7 2025-12-17 22:53:20,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ede575bc060ce15ffa7 HTTP/1.1" 502 150 2025-12-17 22:53:20,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ede575bc060ce15ffa7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:21,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec9575bc060ce15ff6f HTTP/1.1" 502 150 2025-12-17 22:53:21,968 - lava_callback - ERROR - Error processing callback for node 69408ec9575bc060ce15ff6f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ec9575bc060ce15ff6f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ec9575bc060ce15ff6f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ec9575bc060ce15ff6f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:21,970 - lava_callback - INFO - Starting async job submit for node 69408ee7575bc060ce15ffc8 2025-12-17 22:53:21,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:21,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee7575bc060ce15ffc8 HTTP/1.1" 502 150 2025-12-17 22:53:21,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee7575bc060ce15ffc8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:21,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee7575bc060ce15ffc8 2025-12-17 22:53:21,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee7575bc060ce15ffc8 HTTP/1.1" 502 150 2025-12-17 22:53:21,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee7575bc060ce15ffc8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:21,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed8575bc060ce15ff9b 2025-12-17 22:53:21,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed8575bc060ce15ff9b HTTP/1.1" 502 150 2025-12-17 22:53:21,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed8575bc060ce15ff9b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:22,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940946f575bc060ce16103c 2025-12-17 22:53:22,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946f575bc060ce16103c HTTP/1.1" 502 150 2025-12-17 22:53:22,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940946f575bc060ce16103c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:22,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed7575bc060ce15ff98 2025-12-17 22:53:22,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed7575bc060ce15ff98 HTTP/1.1" 502 150 2025-12-17 22:53:22,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ed7575bc060ce15ff98'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:22,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec8575bc060ce15ff6c 2025-12-17 22:53:22,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec8575bc060ce15ff6c HTTP/1.1" 502 150 2025-12-17 22:53:22,842 - lava_callback - ERROR - Error processing callback for node 69408ec8575bc060ce15ff6c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ec8575bc060ce15ff6c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ec8575bc060ce15ff6c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ec8575bc060ce15ff6c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:22,844 - lava_callback - INFO - Starting async job submit for node 69408ee3575bc060ce15ffb5 2025-12-17 22:53:22,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:22,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee3575bc060ce15ffb5 HTTP/1.1" 502 150 2025-12-17 22:53:22,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee3575bc060ce15ffb5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:22,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee3575bc060ce15ffb5 2025-12-17 22:53:22,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee3575bc060ce15ffb5 HTTP/1.1" 502 150 2025-12-17 22:53:22,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee3575bc060ce15ffb5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:22,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee4575bc060ce15ffba 2025-12-17 22:53:22,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee4575bc060ce15ffba HTTP/1.1" 502 150 2025-12-17 22:53:22,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee4575bc060ce15ffba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:22,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409251575bc060ce160a72 2025-12-17 22:53:22,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409251575bc060ce160a72 HTTP/1.1" 502 150 2025-12-17 22:53:22,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409251575bc060ce160a72'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:23,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee7575bc060ce15ffc8 2025-12-17 22:53:23,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee7575bc060ce15ffc8 HTTP/1.1" 502 150 2025-12-17 22:53:23,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee7575bc060ce15ffc8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:24,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ecb575bc060ce15ff72 2025-12-17 22:53:24,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ecb575bc060ce15ff72 HTTP/1.1" 502 150 2025-12-17 22:53:24,465 - lava_callback - ERROR - Error processing callback for node 69408ecb575bc060ce15ff72: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ecb575bc060ce15ff72 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ecb575bc060ce15ff72 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ecb575bc060ce15ff72 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:24,467 - lava_callback - INFO - Starting async job submit for node 69408ee8575bc060ce15ffcf 2025-12-17 22:53:24,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:24,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee8575bc060ce15ffcf HTTP/1.1" 502 150 2025-12-17 22:53:24,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee8575bc060ce15ffcf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:24,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee8575bc060ce15ffcf 2025-12-17 22:53:24,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee8575bc060ce15ffcf HTTP/1.1" 502 150 2025-12-17 22:53:24,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee8575bc060ce15ffcf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:24,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee3575bc060ce15ffb5 2025-12-17 22:53:24,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee3575bc060ce15ffb5 HTTP/1.1" 502 150 2025-12-17 22:53:24,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee3575bc060ce15ffb5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:24,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ece575bc060ce15ff78 2025-12-17 22:53:24,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ece575bc060ce15ff78 HTTP/1.1" 502 150 2025-12-17 22:53:24,982 - lava_callback - ERROR - Error processing callback for node 69408ece575bc060ce15ff78: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ece575bc060ce15ff78 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ece575bc060ce15ff78 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ece575bc060ce15ff78 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:24,984 - lava_callback - INFO - Starting async job submit for node 69408eea575bc060ce15ffd4 2025-12-17 22:53:24,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:25,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eea575bc060ce15ffd4 HTTP/1.1" 502 150 2025-12-17 22:53:25,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eea575bc060ce15ffd4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:25,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eea575bc060ce15ffd4 2025-12-17 22:53:25,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eea575bc060ce15ffd4 HTTP/1.1" 502 150 2025-12-17 22:53:25,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eea575bc060ce15ffd4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:26,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee8575bc060ce15ffcf 2025-12-17 22:53:26,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee8575bc060ce15ffcf HTTP/1.1" 502 150 2025-12-17 22:53:26,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee8575bc060ce15ffcf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59550 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:53:27,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eea575bc060ce15ffd4 2025-12-17 22:53:27,252 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed0575bc060ce15ff7d 2025-12-17 22:53:27,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed0575bc060ce15ff7d HTTP/1.1" 502 150 2025-12-17 22:53:27,254 - lava_callback - ERROR - Error processing callback for node 69408ed0575bc060ce15ff7d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed0575bc060ce15ff7d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed0575bc060ce15ff7d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed0575bc060ce15ff7d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:27,256 - lava_callback - INFO - Starting async job submit for node 69408eeb575bc060ce15ffd7 2025-12-17 22:53:27,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:27,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eeb575bc060ce15ffd7 HTTP/1.1" 502 150 2025-12-17 22:53:27,262 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eeb575bc060ce15ffd7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:27,262 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eeb575bc060ce15ffd7 2025-12-17 22:53:27,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eeb575bc060ce15ffd7 HTTP/1.1" 502 150 2025-12-17 22:53:27,263 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eeb575bc060ce15ffd7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:27,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed1575bc060ce15ff84 2025-12-17 22:53:27,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed1575bc060ce15ff84 HTTP/1.1" 502 150 2025-12-17 22:53:27,548 - lava_callback - ERROR - Error processing callback for node 69408ed1575bc060ce15ff84: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed1575bc060ce15ff84 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed1575bc060ce15ff84 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed1575bc060ce15ff84 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:27,550 - lava_callback - INFO - Starting async job submit for node 69408eec575bc060ce15ffda 2025-12-17 22:53:27,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:27,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eec575bc060ce15ffda HTTP/1.1" 502 150 2025-12-17 22:53:27,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eec575bc060ce15ffda'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:27,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eec575bc060ce15ffda 2025-12-17 22:53:27,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eec575bc060ce15ffda HTTP/1.1" 502 150 2025-12-17 22:53:27,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eec575bc060ce15ffda'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:27,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee7575bc060ce15ffc8 2025-12-17 22:53:27,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee7575bc060ce15ffc8 HTTP/1.1" 502 150 2025-12-17 22:53:27,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee7575bc060ce15ffc8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:28,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee3575bc060ce15ffb5 2025-12-17 22:53:28,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee3575bc060ce15ffb5 HTTP/1.1" 502 150 2025-12-17 22:53:28,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee3575bc060ce15ffb5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:29,263 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eeb575bc060ce15ffd7 2025-12-17 22:53:29,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eeb575bc060ce15ffd7 HTTP/1.1" 502 150 2025-12-17 22:53:29,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eeb575bc060ce15ffd7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:29,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eec575bc060ce15ffda 2025-12-17 22:53:29,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eec575bc060ce15ffda HTTP/1.1" 502 150 2025-12-17 22:53:29,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eec575bc060ce15ffda'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:30,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee8575bc060ce15ffcf 2025-12-17 22:53:30,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee8575bc060ce15ffcf HTTP/1.1" 502 150 2025-12-17 22:53:30,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee8575bc060ce15ffcf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:30,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed3575bc060ce15ff8b 2025-12-17 22:53:30,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed3575bc060ce15ff8b HTTP/1.1" 502 150 2025-12-17 22:53:30,917 - lava_callback - ERROR - Error processing callback for node 69408ed3575bc060ce15ff8b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed3575bc060ce15ff8b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed3575bc060ce15ff8b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed3575bc060ce15ff8b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:30,919 - lava_callback - INFO - Starting async job submit for node 69408eed575bc060ce15ffdd 2025-12-17 22:53:30,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:30,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eed575bc060ce15ffdd HTTP/1.1" 502 150 2025-12-17 22:53:30,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eed575bc060ce15ffdd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:30,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eed575bc060ce15ffdd 2025-12-17 22:53:30,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eed575bc060ce15ffdd HTTP/1.1" 502 150 2025-12-17 22:53:30,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eed575bc060ce15ffdd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:30,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee4575bc060ce15ffba 2025-12-17 22:53:30,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee4575bc060ce15ffba HTTP/1.1" 502 150 2025-12-17 22:53:30,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee4575bc060ce15ffba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:32,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eed575bc060ce15ffdd 2025-12-17 22:53:32,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eed575bc060ce15ffdd HTTP/1.1" 502 150 2025-12-17 22:53:32,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eed575bc060ce15ffdd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:33,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eeb575bc060ce15ffd7 2025-12-17 22:53:33,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eeb575bc060ce15ffd7 HTTP/1.1" 502 150 2025-12-17 22:53:33,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eeb575bc060ce15ffd7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:33,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eec575bc060ce15ffda 2025-12-17 22:53:33,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eec575bc060ce15ffda HTTP/1.1" 502 150 2025-12-17 22:53:33,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eec575bc060ce15ffda'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:34,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed6575bc060ce15ff95 2025-12-17 22:53:34,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed6575bc060ce15ff95 HTTP/1.1" 502 150 2025-12-17 22:53:34,958 - lava_callback - ERROR - Error processing callback for node 69408ed6575bc060ce15ff95: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed6575bc060ce15ff95 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed6575bc060ce15ff95 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed6575bc060ce15ff95 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:34,960 - lava_callback - INFO - Starting async job submit for node 69408eee575bc060ce15ffe0 2025-12-17 22:53:34,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:34,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eee575bc060ce15ffe0 HTTP/1.1" 502 150 2025-12-17 22:53:34,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eee575bc060ce15ffe0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:34,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eee575bc060ce15ffe0 2025-12-17 22:53:34,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eee575bc060ce15ffe0 HTTP/1.1" 502 150 2025-12-17 22:53:34,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eee575bc060ce15ffe0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:35,522 - lava_callback - INFO - Received callback for node 6940915b575bc060ce160811 INFO: 172.21.0.1:42512 - "POST /node/6940915b575bc060ce160811 HTTP/1.0" 202 Accepted 2025-12-17 22:53:35,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eda575bc060ce15ff9e 2025-12-17 22:53:35,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eda575bc060ce15ff9e HTTP/1.1" 502 150 2025-12-17 22:53:35,594 - lava_callback - ERROR - Error processing callback for node 69408eda575bc060ce15ff9e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eda575bc060ce15ff9e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eda575bc060ce15ff9e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eda575bc060ce15ff9e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:35,596 - lava_callback - INFO - Starting async job submit for node 69408ef0575bc060ce15ffe3 2025-12-17 22:53:35,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:35,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef0575bc060ce15ffe3 HTTP/1.1" 502 150 2025-12-17 22:53:35,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef0575bc060ce15ffe3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:35,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef0575bc060ce15ffe3 2025-12-17 22:53:35,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef0575bc060ce15ffe3 HTTP/1.1" 502 150 2025-12-17 22:53:35,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef0575bc060ce15ffe3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:35,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ec6575bc060ce15ff69 2025-12-17 22:53:35,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ec6575bc060ce15ff69 HTTP/1.1" 502 150 2025-12-17 22:53:35,637 - lava_callback - ERROR - Error processing callback for node 69408ec6575bc060ce15ff69: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ec6575bc060ce15ff69 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ec6575bc060ce15ff69 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ec6575bc060ce15ff69 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:35,639 - lava_callback - INFO - Starting async job submit for node 69408ef1575bc060ce15ffe6 2025-12-17 22:53:35,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:35,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef1575bc060ce15ffe6 HTTP/1.1" 502 150 2025-12-17 22:53:35,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef1575bc060ce15ffe6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:35,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef1575bc060ce15ffe6 2025-12-17 22:53:35,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef1575bc060ce15ffe6 HTTP/1.1" 502 150 2025-12-17 22:53:35,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef1575bc060ce15ffe6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:35,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee7575bc060ce15ffc8 2025-12-17 22:53:35,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee7575bc060ce15ffc8 HTTP/1.1" 502 150 2025-12-17 22:53:35,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee7575bc060ce15ffc8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:36,191 - lava_callback - INFO - Received callback for node 6940915d575bc060ce160814 INFO: 172.21.0.1:42514 - "POST /node/6940915d575bc060ce160814 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:42522 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:53:36,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee3575bc060ce15ffb5 2025-12-17 22:53:36,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee3575bc060ce15ffb5 HTTP/1.1" 502 150 2025-12-17 22:53:36,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee3575bc060ce15ffb5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:36,867 - lava_callback - INFO - Received callback for node 69409160575bc060ce160817 INFO: 172.21.0.1:42526 - "POST /node/69409160575bc060ce160817 HTTP/1.0" 202 Accepted 2025-12-17 22:53:36,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eed575bc060ce15ffdd 2025-12-17 22:53:36,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eed575bc060ce15ffdd HTTP/1.1" 502 150 2025-12-17 22:53:36,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eed575bc060ce15ffdd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:36,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ede575bc060ce15ffa7 2025-12-17 22:53:36,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ede575bc060ce15ffa7 HTTP/1.1" 502 150 2025-12-17 22:53:36,932 - lava_callback - ERROR - Error processing callback for node 69408ede575bc060ce15ffa7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ede575bc060ce15ffa7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ede575bc060ce15ffa7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ede575bc060ce15ffa7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:36,934 - lava_callback - INFO - Starting async job submit for node 69408ef2575bc060ce15ffe9 2025-12-17 22:53:36,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:36,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef2575bc060ce15ffe9 HTTP/1.1" 502 150 2025-12-17 22:53:36,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef2575bc060ce15ffe9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:36,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef2575bc060ce15ffe9 2025-12-17 22:53:36,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef2575bc060ce15ffe9 HTTP/1.1" 502 150 2025-12-17 22:53:36,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef2575bc060ce15ffe9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:36,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eee575bc060ce15ffe0 2025-12-17 22:53:36,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eee575bc060ce15ffe0 HTTP/1.1" 502 150 2025-12-17 22:53:36,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eee575bc060ce15ffe0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:37,516 - lava_callback - INFO - Received callback for node 69409162575bc060ce16081a INFO: 172.21.0.1:42542 - "POST /node/69409162575bc060ce16081a HTTP/1.0" 202 Accepted 2025-12-17 22:53:37,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef0575bc060ce15ffe3 2025-12-17 22:53:37,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef0575bc060ce15ffe3 HTTP/1.1" 502 150 2025-12-17 22:53:37,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef0575bc060ce15ffe3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:37,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef1575bc060ce15ffe6 2025-12-17 22:53:37,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef1575bc060ce15ffe6 HTTP/1.1" 502 150 2025-12-17 22:53:37,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef1575bc060ce15ffe6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:37,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed8575bc060ce15ff9b 2025-12-17 22:53:37,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed8575bc060ce15ff9b HTTP/1.1" 502 150 2025-12-17 22:53:37,999 - lava_callback - ERROR - Error processing callback for node 69408ed8575bc060ce15ff9b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed8575bc060ce15ff9b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed8575bc060ce15ff9b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed8575bc060ce15ff9b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:38,001 - lava_callback - INFO - Starting async job submit for node 69408ef4575bc060ce15ffec 2025-12-17 22:53:38,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:38,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef4575bc060ce15ffec HTTP/1.1" 502 150 2025-12-17 22:53:38,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef4575bc060ce15ffec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:38,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef4575bc060ce15ffec 2025-12-17 22:53:38,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef4575bc060ce15ffec HTTP/1.1" 502 150 2025-12-17 22:53:38,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef4575bc060ce15ffec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:38,165 - lava_callback - INFO - Received callback for node 69409164575bc060ce16081d INFO: 172.21.0.1:42548 - "POST /node/69409164575bc060ce16081d HTTP/1.0" 202 Accepted 2025-12-17 22:53:38,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee8575bc060ce15ffcf 2025-12-17 22:53:38,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee8575bc060ce15ffcf HTTP/1.1" 502 150 2025-12-17 22:53:38,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ee8575bc060ce15ffcf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:38,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940946f575bc060ce16103c 2025-12-17 22:53:38,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946f575bc060ce16103c HTTP/1.1" 502 150 2025-12-17 22:53:38,773 - lava_callback - ERROR - Error processing callback for node 6940946f575bc060ce16103c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940946f575bc060ce16103c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940946f575bc060ce16103c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940946f575bc060ce16103c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:38,775 - lava_callback - INFO - Starting async job submit for node 69408ef5575bc060ce15ffef 2025-12-17 22:53:38,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:38,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef5575bc060ce15ffef HTTP/1.1" 502 150 2025-12-17 22:53:38,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef5575bc060ce15ffef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:38,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef5575bc060ce15ffef 2025-12-17 22:53:38,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef5575bc060ce15ffef HTTP/1.1" 502 150 2025-12-17 22:53:38,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef5575bc060ce15ffef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:38,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ed7575bc060ce15ff98 2025-12-17 22:53:38,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ed7575bc060ce15ff98 HTTP/1.1" 502 150 2025-12-17 22:53:38,785 - lava_callback - ERROR - Error processing callback for node 69408ed7575bc060ce15ff98: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed7575bc060ce15ff98 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed7575bc060ce15ff98 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ed7575bc060ce15ff98 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:38,786 - lava_callback - INFO - Starting async job submit for node 69408ef6575bc060ce15fff2 2025-12-17 22:53:38,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:38,804 - lava_callback - INFO - Received callback for node 69409166575bc060ce160828 INFO: 172.21.0.1:42560 - "POST /node/69409166575bc060ce160828 HTTP/1.0" 202 Accepted 2025-12-17 22:53:38,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef6575bc060ce15fff2 HTTP/1.1" 502 150 2025-12-17 22:53:38,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef6575bc060ce15fff2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:38,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef6575bc060ce15fff2 2025-12-17 22:53:38,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef6575bc060ce15fff2 HTTP/1.1" 502 150 2025-12-17 22:53:38,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef6575bc060ce15fff2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:38,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef2575bc060ce15ffe9 2025-12-17 22:53:38,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef2575bc060ce15ffe9 HTTP/1.1" 502 150 2025-12-17 22:53:38,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef2575bc060ce15ffe9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:38,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409251575bc060ce160a72 2025-12-17 22:53:38,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409251575bc060ce160a72 HTTP/1.1" 502 150 2025-12-17 22:53:38,997 - lava_callback - ERROR - Error processing callback for node 69409251575bc060ce160a72: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409251575bc060ce160a72 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409251575bc060ce160a72 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409251575bc060ce160a72 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:38,999 - lava_callback - INFO - Starting async job submit for node 69408ef7575bc060ce15fff5 2025-12-17 22:53:39,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:39,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef7575bc060ce15fff5 HTTP/1.1" 502 150 2025-12-17 22:53:39,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef7575bc060ce15fff5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:39,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef7575bc060ce15fff5 2025-12-17 22:53:39,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef7575bc060ce15fff5 HTTP/1.1" 502 150 2025-12-17 22:53:39,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef7575bc060ce15fff5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:40,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef4575bc060ce15ffec 2025-12-17 22:53:40,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef4575bc060ce15ffec HTTP/1.1" 502 150 2025-12-17 22:53:40,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef4575bc060ce15ffec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:40,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef5575bc060ce15ffef 2025-12-17 22:53:40,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef5575bc060ce15ffef HTTP/1.1" 502 150 2025-12-17 22:53:40,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef5575bc060ce15ffef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:40,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef6575bc060ce15fff2 2025-12-17 22:53:40,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef6575bc060ce15fff2 HTTP/1.1" 502 150 2025-12-17 22:53:40,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef6575bc060ce15fff2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:40,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eee575bc060ce15ffe0 2025-12-17 22:53:40,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eee575bc060ce15ffe0 HTTP/1.1" 502 150 2025-12-17 22:53:40,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eee575bc060ce15ffe0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:41,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef7575bc060ce15fff5 2025-12-17 22:53:41,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef7575bc060ce15fff5 HTTP/1.1" 502 150 2025-12-17 22:53:41,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef7575bc060ce15fff5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:41,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eeb575bc060ce15ffd7 2025-12-17 22:53:41,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eeb575bc060ce15ffd7 HTTP/1.1" 502 150 2025-12-17 22:53:41,267 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eeb575bc060ce15ffd7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:41,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eec575bc060ce15ffda 2025-12-17 22:53:41,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eec575bc060ce15ffda HTTP/1.1" 502 150 2025-12-17 22:53:41,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eec575bc060ce15ffda'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:41,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef0575bc060ce15ffe3 2025-12-17 22:53:41,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef0575bc060ce15ffe3 HTTP/1.1" 502 150 2025-12-17 22:53:41,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef0575bc060ce15ffe3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:41,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef1575bc060ce15ffe6 2025-12-17 22:53:41,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef1575bc060ce15ffe6 HTTP/1.1" 502 150 2025-12-17 22:53:41,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef1575bc060ce15ffe6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:42,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eea575bc060ce15ffd4 HTTP/1.1" 502 150 2025-12-17 22:53:42,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eea575bc060ce15ffd4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:42,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef2575bc060ce15ffe9 2025-12-17 22:53:42,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef2575bc060ce15ffe9 HTTP/1.1" 502 150 2025-12-17 22:53:42,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef2575bc060ce15ffe9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:44,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef4575bc060ce15ffec 2025-12-17 22:53:44,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef5575bc060ce15ffef 2025-12-17 22:53:44,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef5575bc060ce15ffef HTTP/1.1" 502 150 2025-12-17 22:53:44,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef5575bc060ce15ffef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:44,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef6575bc060ce15fff2 2025-12-17 22:53:44,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef6575bc060ce15fff2 HTTP/1.1" 502 150 2025-12-17 22:53:44,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef6575bc060ce15fff2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:44,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eed575bc060ce15ffdd 2025-12-17 22:53:44,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eed575bc060ce15ffdd HTTP/1.1" 502 150 2025-12-17 22:53:44,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eed575bc060ce15ffdd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:45,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef7575bc060ce15fff5 2025-12-17 22:53:45,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef7575bc060ce15fff5 HTTP/1.1" 502 150 2025-12-17 22:53:45,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef7575bc060ce15fff5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:45,934 - lava_callback - INFO - Received callback for node 69409168575bc060ce160835 INFO: 172.21.0.1:37860 - "POST /node/69409168575bc060ce160835 HTTP/1.0" 202 Accepted 2025-12-17 22:53:46,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eea575bc060ce15ffd4 2025-12-17 22:53:46,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eea575bc060ce15ffd4 HTTP/1.1" 502 150 2025-12-17 22:53:46,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eea575bc060ce15ffd4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37872 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:53:46,573 - lava_callback - INFO - Received callback for node 6940916b575bc060ce16083c INFO: 172.21.0.1:37878 - "POST /node/6940916b575bc060ce16083c HTTP/1.0" 202 Accepted 2025-12-17 22:53:46,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee4575bc060ce15ffba 2025-12-17 22:53:46,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee4575bc060ce15ffba HTTP/1.1" 502 150 2025-12-17 22:53:46,943 - lava_callback - ERROR - Error processing callback for node 69408ee4575bc060ce15ffba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ee4575bc060ce15ffba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ee4575bc060ce15ffba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ee4575bc060ce15ffba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:46,946 - lava_callback - INFO - Starting async job submit for node 69408ef9575bc060ce15fffc 2025-12-17 22:53:46,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:46,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef9575bc060ce15fffc HTTP/1.1" 502 150 2025-12-17 22:53:46,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef9575bc060ce15fffc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:46,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef9575bc060ce15fffc 2025-12-17 22:53:46,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef9575bc060ce15fffc HTTP/1.1" 502 150 2025-12-17 22:53:46,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef9575bc060ce15fffc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:47,239 - lava_callback - INFO - Received callback for node 6940916d575bc060ce16083f INFO: 172.21.0.1:37894 - "POST /node/6940916d575bc060ce16083f HTTP/1.0" 202 Accepted 2025-12-17 22:53:47,904 - lava_callback - INFO - Received callback for node 6940916e575bc060ce160846 INFO: 172.21.0.1:37896 - "POST /node/6940916e575bc060ce160846 HTTP/1.0" 202 Accepted 2025-12-17 22:53:48,559 - lava_callback - INFO - Received callback for node 69409170575bc060ce160849 INFO: 172.21.0.1:37906 - "POST /node/69409170575bc060ce160849 HTTP/1.0" 202 Accepted 2025-12-17 22:53:48,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef9575bc060ce15fffc 2025-12-17 22:53:48,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef9575bc060ce15fffc HTTP/1.1" 502 150 2025-12-17 22:53:48,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef9575bc060ce15fffc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:48,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eee575bc060ce15ffe0 2025-12-17 22:53:48,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eee575bc060ce15ffe0 HTTP/1.1" 502 150 2025-12-17 22:53:48,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eee575bc060ce15ffe0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:49,226 - lava_callback - INFO - Received callback for node 69409172575bc060ce16084c INFO: 172.21.0.1:37918 - "POST /node/69409172575bc060ce16084c HTTP/1.0" 202 Accepted 2025-12-17 22:53:49,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef0575bc060ce15ffe3 2025-12-17 22:53:49,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef0575bc060ce15ffe3 HTTP/1.1" 502 150 2025-12-17 22:53:49,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef0575bc060ce15ffe3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:49,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef1575bc060ce15ffe6 2025-12-17 22:53:49,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef1575bc060ce15ffe6 HTTP/1.1" 502 150 2025-12-17 22:53:49,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef1575bc060ce15ffe6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:49,880 - lava_callback - INFO - Received callback for node 69409176575bc060ce160852 INFO: 172.21.0.1:51550 - "POST /node/69409176575bc060ce160852 HTTP/1.0" 202 Accepted 2025-12-17 22:53:50,511 - lava_callback - INFO - Received callback for node 69409178575bc060ce160857 INFO: 172.21.0.1:51552 - "POST /node/69409178575bc060ce160857 HTTP/1.0" 202 Accepted 2025-12-17 22:53:50,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef2575bc060ce15ffe9 2025-12-17 22:53:50,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef2575bc060ce15ffe9 HTTP/1.1" 502 150 2025-12-17 22:53:50,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef2575bc060ce15ffe9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:51,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee7575bc060ce15ffc8 2025-12-17 22:53:51,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee7575bc060ce15ffc8 HTTP/1.1" 502 150 2025-12-17 22:53:51,998 - lava_callback - ERROR - Error processing callback for node 69408ee7575bc060ce15ffc8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ee7575bc060ce15ffc8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ee7575bc060ce15ffc8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ee7575bc060ce15ffc8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:52,001 - lava_callback - INFO - Starting async job submit for node 69408efa575bc060ce160003 2025-12-17 22:53:52,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:52,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efa575bc060ce160003 HTTP/1.1" 502 150 2025-12-17 22:53:52,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efa575bc060ce160003'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:52,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efa575bc060ce160003 2025-12-17 22:53:52,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efa575bc060ce160003 HTTP/1.1" 502 150 2025-12-17 22:53:52,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efa575bc060ce160003'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:52,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef5575bc060ce15ffef 2025-12-17 22:53:52,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef5575bc060ce15ffef HTTP/1.1" 502 150 2025-12-17 22:53:52,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef5575bc060ce15ffef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:52,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef6575bc060ce15fff2 2025-12-17 22:53:52,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef6575bc060ce15fff2 HTTP/1.1" 502 150 2025-12-17 22:53:52,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef6575bc060ce15fff2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:52,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee3575bc060ce15ffb5 2025-12-17 22:53:52,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee3575bc060ce15ffb5 HTTP/1.1" 502 150 2025-12-17 22:53:52,857 - lava_callback - ERROR - Error processing callback for node 69408ee3575bc060ce15ffb5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ee3575bc060ce15ffb5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ee3575bc060ce15ffb5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ee3575bc060ce15ffb5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:52,859 - lava_callback - INFO - Starting async job submit for node 694094bb575bc060ce1610e8 2025-12-17 22:53:52,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:52,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bb575bc060ce1610e8 HTTP/1.1" 502 150 2025-12-17 22:53:52,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094bb575bc060ce1610e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:52,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094bb575bc060ce1610e8 2025-12-17 22:53:52,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bb575bc060ce1610e8 HTTP/1.1" 502 150 2025-12-17 22:53:52,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094bb575bc060ce1610e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:52,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef9575bc060ce15fffc 2025-12-17 22:53:52,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef9575bc060ce15fffc HTTP/1.1" 502 150 2025-12-17 22:53:52,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef9575bc060ce15fffc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:53,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef7575bc060ce15fff5 2025-12-17 22:53:54,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efa575bc060ce160003 2025-12-17 22:53:54,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efa575bc060ce160003 HTTP/1.1" 502 150 2025-12-17 22:53:54,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efa575bc060ce160003'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:54,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eea575bc060ce15ffd4 2025-12-17 22:53:54,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eea575bc060ce15ffd4 HTTP/1.1" 502 150 2025-12-17 22:53:54,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eea575bc060ce15ffd4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:54,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ee8575bc060ce15ffcf 2025-12-17 22:53:54,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ee8575bc060ce15ffcf HTTP/1.1" 502 150 2025-12-17 22:53:54,483 - lava_callback - ERROR - Error processing callback for node 69408ee8575bc060ce15ffcf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ee8575bc060ce15ffcf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ee8575bc060ce15ffcf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ee8575bc060ce15ffcf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:54,485 - lava_callback - INFO - Starting async job submit for node 69408efb575bc060ce16000a 2025-12-17 22:53:54,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:54,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efb575bc060ce16000a HTTP/1.1" 502 150 2025-12-17 22:53:54,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efb575bc060ce16000a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:54,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efb575bc060ce16000a 2025-12-17 22:53:54,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efb575bc060ce16000a HTTP/1.1" 502 150 2025-12-17 22:53:54,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efb575bc060ce16000a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:54,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094bb575bc060ce1610e8 2025-12-17 22:53:54,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bb575bc060ce1610e8 HTTP/1.1" 502 150 2025-12-17 22:53:54,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094bb575bc060ce1610e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:55,110 - lava_callback - INFO - Received callback for node 69409334575bc060ce160c6a INFO: 172.21.0.1:51558 - "POST /node/69409334575bc060ce160c6a HTTP/1.0" 202 Accepted 2025-12-17 22:53:56,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efb575bc060ce16000a 2025-12-17 22:53:56,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efb575bc060ce16000a HTTP/1.1" 502 150 2025-12-17 22:53:56,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efb575bc060ce16000a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51570 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:53:57,267 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eeb575bc060ce15ffd7 2025-12-17 22:53:57,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eeb575bc060ce15ffd7 HTTP/1.1" 502 150 2025-12-17 22:53:57,269 - lava_callback - ERROR - Error processing callback for node 69408eeb575bc060ce15ffd7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eeb575bc060ce15ffd7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eeb575bc060ce15ffd7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eeb575bc060ce15ffd7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:57,271 - lava_callback - INFO - Starting async job submit for node 69408efd575bc060ce16000d 2025-12-17 22:53:57,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:57,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efd575bc060ce16000d HTTP/1.1" 502 150 2025-12-17 22:53:57,283 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efd575bc060ce16000d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:57,283 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efd575bc060ce16000d 2025-12-17 22:53:57,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efd575bc060ce16000d HTTP/1.1" 502 150 2025-12-17 22:53:57,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efd575bc060ce16000d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:57,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eec575bc060ce15ffda 2025-12-17 22:53:57,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eec575bc060ce15ffda HTTP/1.1" 502 150 2025-12-17 22:53:57,601 - lava_callback - ERROR - Error processing callback for node 69408eec575bc060ce15ffda: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eec575bc060ce15ffda (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eec575bc060ce15ffda (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eec575bc060ce15ffda (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:53:57,603 - lava_callback - INFO - Starting async job submit for node 69408efe575bc060ce160010 2025-12-17 22:53:57,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:53:57,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efe575bc060ce160010 HTTP/1.1" 502 150 2025-12-17 22:53:57,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efe575bc060ce160010'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:57,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efe575bc060ce160010 2025-12-17 22:53:57,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efe575bc060ce160010 HTTP/1.1" 502 150 2025-12-17 22:53:57,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efe575bc060ce160010'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:58,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efa575bc060ce160003 2025-12-17 22:53:58,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094bb575bc060ce1610e8 2025-12-17 22:53:58,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bb575bc060ce1610e8 HTTP/1.1" 502 150 2025-12-17 22:53:58,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094bb575bc060ce1610e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:59,284 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efd575bc060ce16000d 2025-12-17 22:53:59,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efd575bc060ce16000d HTTP/1.1" 502 150 2025-12-17 22:53:59,286 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efd575bc060ce16000d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:59,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef4575bc060ce15ffec HTTP/1.1" 502 150 2025-12-17 22:53:59,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef4575bc060ce15ffec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:53:59,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efe575bc060ce160010 2025-12-17 22:53:59,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efe575bc060ce160010 HTTP/1.1" 502 150 2025-12-17 22:53:59,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efe575bc060ce160010'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:00,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efb575bc060ce16000a 2025-12-17 22:54:00,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efb575bc060ce16000a HTTP/1.1" 502 150 2025-12-17 22:54:00,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efb575bc060ce16000a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:00,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eed575bc060ce15ffdd 2025-12-17 22:54:00,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eed575bc060ce15ffdd HTTP/1.1" 502 150 2025-12-17 22:54:00,933 - lava_callback - ERROR - Error processing callback for node 69408eed575bc060ce15ffdd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eed575bc060ce15ffdd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eed575bc060ce15ffdd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eed575bc060ce15ffdd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:00,935 - lava_callback - INFO - Starting async job submit for node 69408eff575bc060ce160013 2025-12-17 22:54:00,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:00,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eff575bc060ce160013 HTTP/1.1" 502 150 2025-12-17 22:54:00,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eff575bc060ce160013'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:00,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eff575bc060ce160013 2025-12-17 22:54:00,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eff575bc060ce160013 HTTP/1.1" 502 150 2025-12-17 22:54:00,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eff575bc060ce160013'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:00,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef9575bc060ce15fffc 2025-12-17 22:54:00,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef9575bc060ce15fffc HTTP/1.1" 502 150 2025-12-17 22:54:00,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef9575bc060ce15fffc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37842 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:54:02,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eff575bc060ce160013 2025-12-17 22:54:02,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eff575bc060ce160013 HTTP/1.1" 502 150 2025-12-17 22:54:02,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eff575bc060ce160013'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:03,286 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efd575bc060ce16000d 2025-12-17 22:54:03,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efd575bc060ce16000d HTTP/1.1" 502 150 2025-12-17 22:54:03,287 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efd575bc060ce16000d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:03,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efe575bc060ce160010 2025-12-17 22:54:03,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efe575bc060ce160010 HTTP/1.1" 502 150 2025-12-17 22:54:03,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efe575bc060ce160010'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:04,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eee575bc060ce15ffe0 2025-12-17 22:54:04,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eee575bc060ce15ffe0 HTTP/1.1" 502 150 2025-12-17 22:54:04,982 - lava_callback - ERROR - Error processing callback for node 69408eee575bc060ce15ffe0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eee575bc060ce15ffe0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eee575bc060ce15ffe0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eee575bc060ce15ffe0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:04,984 - lava_callback - INFO - Starting async job submit for node 69408f00575bc060ce160016 2025-12-17 22:54:04,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:04,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f00575bc060ce160016 HTTP/1.1" 502 150 2025-12-17 22:54:04,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f00575bc060ce160016'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:04,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f00575bc060ce160016 2025-12-17 22:54:04,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f00575bc060ce160016 HTTP/1.1" 502 150 2025-12-17 22:54:04,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f00575bc060ce160016'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:05,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef0575bc060ce15ffe3 2025-12-17 22:54:05,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef0575bc060ce15ffe3 HTTP/1.1" 502 150 2025-12-17 22:54:05,611 - lava_callback - ERROR - Error processing callback for node 69408ef0575bc060ce15ffe3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef0575bc060ce15ffe3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef0575bc060ce15ffe3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef0575bc060ce15ffe3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:05,613 - lava_callback - INFO - Starting async job submit for node 69408f01575bc060ce160019 2025-12-17 22:54:05,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:05,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f01575bc060ce160019 HTTP/1.1" 502 150 2025-12-17 22:54:05,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f01575bc060ce160019'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:05,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f01575bc060ce160019 2025-12-17 22:54:05,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f01575bc060ce160019 HTTP/1.1" 502 150 2025-12-17 22:54:05,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f01575bc060ce160019'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:05,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef1575bc060ce15ffe6 2025-12-17 22:54:05,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef1575bc060ce15ffe6 HTTP/1.1" 502 150 2025-12-17 22:54:05,656 - lava_callback - ERROR - Error processing callback for node 69408ef1575bc060ce15ffe6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef1575bc060ce15ffe6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef1575bc060ce15ffe6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef1575bc060ce15ffe6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:05,658 - lava_callback - INFO - Starting async job submit for node 69408f03575bc060ce16001c 2025-12-17 22:54:05,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:05,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f03575bc060ce16001c HTTP/1.1" 502 150 2025-12-17 22:54:05,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f03575bc060ce16001c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:05,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f03575bc060ce16001c 2025-12-17 22:54:05,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f03575bc060ce16001c HTTP/1.1" 502 150 2025-12-17 22:54:05,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f03575bc060ce16001c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:05,931 - lava_callback - INFO - Received callback for node 6940917a575bc060ce160860 INFO: 172.21.0.1:37844 - "POST /node/6940917a575bc060ce160860 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:37860 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:54:06,566 - lava_callback - INFO - Received callback for node 6940917c575bc060ce160869 INFO: 172.21.0.1:37862 - "POST /node/6940917c575bc060ce160869 HTTP/1.0" 202 Accepted 2025-12-17 22:54:06,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094bb575bc060ce1610e8 2025-12-17 22:54:06,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bb575bc060ce1610e8 HTTP/1.1" 502 150 2025-12-17 22:54:06,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094bb575bc060ce1610e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:06,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef2575bc060ce15ffe9 2025-12-17 22:54:06,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef2575bc060ce15ffe9 HTTP/1.1" 502 150 2025-12-17 22:54:06,946 - lava_callback - ERROR - Error processing callback for node 69408ef2575bc060ce15ffe9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef2575bc060ce15ffe9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef2575bc060ce15ffe9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef2575bc060ce15ffe9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:06,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eff575bc060ce160013 2025-12-17 22:54:06,948 - lava_callback - INFO - Starting async job submit for node 69408f04575bc060ce16001f 2025-12-17 22:54:06,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:06,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eff575bc060ce160013 HTTP/1.1" 502 150 2025-12-17 22:54:06,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eff575bc060ce160013'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:06,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f04575bc060ce16001f HTTP/1.1" 502 150 2025-12-17 22:54:06,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f04575bc060ce16001f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:06,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f04575bc060ce16001f 2025-12-17 22:54:06,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f04575bc060ce16001f HTTP/1.1" 502 150 2025-12-17 22:54:06,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f04575bc060ce16001f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:06,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f00575bc060ce160016 2025-12-17 22:54:06,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f00575bc060ce160016 HTTP/1.1" 502 150 2025-12-17 22:54:06,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f00575bc060ce160016'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:07,213 - lava_callback - INFO - Received callback for node 6940917f575bc060ce160887 INFO: 172.21.0.1:37870 - "POST /node/6940917f575bc060ce160887 HTTP/1.0" 202 Accepted 2025-12-17 22:54:07,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef4575bc060ce15ffec 2025-12-17 22:54:07,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef4575bc060ce15ffec HTTP/1.1" 502 150 2025-12-17 22:54:07,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef4575bc060ce15ffec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:07,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f01575bc060ce160019 2025-12-17 22:54:07,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f01575bc060ce160019 HTTP/1.1" 502 150 2025-12-17 22:54:07,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f01575bc060ce160019'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:07,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f03575bc060ce16001c 2025-12-17 22:54:07,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f03575bc060ce16001c HTTP/1.1" 502 150 2025-12-17 22:54:07,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f03575bc060ce16001c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:07,867 - lava_callback - INFO - Received callback for node 69409181575bc060ce16088a INFO: 172.21.0.1:37880 - "POST /node/69409181575bc060ce16088a HTTP/1.0" 202 Accepted 2025-12-17 22:54:08,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef7575bc060ce15fff5 HTTP/1.1" 502 150 2025-12-17 22:54:08,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ef7575bc060ce15fff5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:08,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efb575bc060ce16000a 2025-12-17 22:54:08,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efb575bc060ce16000a HTTP/1.1" 502 150 2025-12-17 22:54:08,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efb575bc060ce16000a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:08,505 - lava_callback - INFO - Received callback for node 69409183575bc060ce16088f INFO: 172.21.0.1:37886 - "POST /node/69409183575bc060ce16088f HTTP/1.0" 202 Accepted 2025-12-17 22:54:08,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef5575bc060ce15ffef 2025-12-17 22:54:08,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef5575bc060ce15ffef HTTP/1.1" 502 150 2025-12-17 22:54:08,788 - lava_callback - ERROR - Error processing callback for node 69408ef5575bc060ce15ffef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef5575bc060ce15ffef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef5575bc060ce15ffef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef5575bc060ce15ffef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:08,790 - lava_callback - INFO - Starting async job submit for node 69408f05575bc060ce160022 2025-12-17 22:54:08,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:08,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f05575bc060ce160022 HTTP/1.1" 502 150 2025-12-17 22:54:08,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f05575bc060ce160022'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:08,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f05575bc060ce160022 2025-12-17 22:54:08,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f05575bc060ce160022 HTTP/1.1" 502 150 2025-12-17 22:54:08,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f05575bc060ce160022'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:08,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef6575bc060ce15fff2 2025-12-17 22:54:08,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef6575bc060ce15fff2 HTTP/1.1" 502 150 2025-12-17 22:54:08,824 - lava_callback - ERROR - Error processing callback for node 69408ef6575bc060ce15fff2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef6575bc060ce15fff2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef6575bc060ce15fff2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef6575bc060ce15fff2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:08,826 - lava_callback - INFO - Starting async job submit for node 69408f07575bc060ce160025 2025-12-17 22:54:08,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:08,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f07575bc060ce160025 HTTP/1.1" 502 150 2025-12-17 22:54:08,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f07575bc060ce160025'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:08,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f07575bc060ce160025 2025-12-17 22:54:08,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f07575bc060ce160025 HTTP/1.1" 502 150 2025-12-17 22:54:08,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f07575bc060ce160025'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:08,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f04575bc060ce16001f 2025-12-17 22:54:08,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f04575bc060ce16001f HTTP/1.1" 502 150 2025-12-17 22:54:08,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f04575bc060ce16001f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:09,151 - lava_callback - INFO - Received callback for node 69409185575bc060ce160897 INFO: 172.21.0.1:37898 - "POST /node/69409185575bc060ce160897 HTTP/1.0" 202 Accepted 2025-12-17 22:54:09,798 - lava_callback - INFO - Received callback for node 69409187575bc060ce16089a INFO: 172.21.0.1:59654 - "POST /node/69409187575bc060ce16089a HTTP/1.0" 202 Accepted 2025-12-17 22:54:10,436 - lava_callback - INFO - Received callback for node 69409189575bc060ce16089d INFO: 172.21.0.1:59670 - "POST /node/69409189575bc060ce16089d HTTP/1.0" 202 Accepted 2025-12-17 22:54:10,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eea575bc060ce15ffd4 2025-12-17 22:54:10,655 - lava_callback - INFO - Received callback for node 6940918f575bc060ce1608b8 INFO: 172.21.0.1:59682 - "POST /node/6940918f575bc060ce1608b8 HTTP/1.0" 202 Accepted 2025-12-17 22:54:10,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f05575bc060ce160022 2025-12-17 22:54:10,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f05575bc060ce160022 HTTP/1.1" 502 150 2025-12-17 22:54:10,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f05575bc060ce160022'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:10,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f07575bc060ce160025 2025-12-17 22:54:10,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f07575bc060ce160025 HTTP/1.1" 502 150 2025-12-17 22:54:10,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f07575bc060ce160025'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:10,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f00575bc060ce160016 2025-12-17 22:54:10,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f00575bc060ce160016 HTTP/1.1" 502 150 2025-12-17 22:54:10,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f00575bc060ce160016'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:11,088 - lava_callback - INFO - Received callback for node 6940918b575bc060ce1608a0 INFO: 172.21.0.1:59696 - "POST /node/6940918b575bc060ce1608a0 HTTP/1.0" 202 Accepted 2025-12-17 22:54:11,288 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efd575bc060ce16000d 2025-12-17 22:54:11,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efd575bc060ce16000d HTTP/1.1" 502 150 2025-12-17 22:54:11,289 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efd575bc060ce16000d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:11,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efe575bc060ce160010 2025-12-17 22:54:11,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efe575bc060ce160010 HTTP/1.1" 502 150 2025-12-17 22:54:11,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efe575bc060ce160010'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:11,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f01575bc060ce160019 2025-12-17 22:54:11,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f01575bc060ce160019 HTTP/1.1" 502 150 2025-12-17 22:54:11,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f01575bc060ce160019'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:11,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f03575bc060ce16001c 2025-12-17 22:54:11,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f03575bc060ce16001c HTTP/1.1" 502 150 2025-12-17 22:54:11,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f03575bc060ce16001c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:12,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f04575bc060ce16001f 2025-12-17 22:54:12,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f04575bc060ce16001f HTTP/1.1" 502 150 2025-12-17 22:54:12,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f04575bc060ce16001f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:13,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efa575bc060ce160003 HTTP/1.1" 502 150 2025-12-17 22:54:13,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efa575bc060ce160003'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:14,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f05575bc060ce160022 2025-12-17 22:54:14,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f05575bc060ce160022 HTTP/1.1" 502 150 2025-12-17 22:54:14,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f05575bc060ce160022'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:14,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f07575bc060ce160025 2025-12-17 22:54:14,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f07575bc060ce160025 HTTP/1.1" 502 150 2025-12-17 22:54:14,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f07575bc060ce160025'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:14,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eff575bc060ce160013 2025-12-17 22:54:14,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eff575bc060ce160013 HTTP/1.1" 502 150 2025-12-17 22:54:14,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408eff575bc060ce160013'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59710 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:54:16,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef9575bc060ce15fffc 2025-12-17 22:54:16,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef9575bc060ce15fffc HTTP/1.1" 502 150 2025-12-17 22:54:16,978 - lava_callback - ERROR - Error processing callback for node 69408ef9575bc060ce15fffc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef9575bc060ce15fffc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef9575bc060ce15fffc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef9575bc060ce15fffc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:16,980 - lava_callback - INFO - Starting async job submit for node 69408f08575bc060ce160028 2025-12-17 22:54:16,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:16,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f08575bc060ce160028 HTTP/1.1" 502 150 2025-12-17 22:54:16,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f08575bc060ce160028'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:16,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f08575bc060ce160028 2025-12-17 22:54:16,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f08575bc060ce160028 HTTP/1.1" 502 150 2025-12-17 22:54:16,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f08575bc060ce160028'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:18,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f08575bc060ce160028 2025-12-17 22:54:18,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f08575bc060ce160028 HTTP/1.1" 502 150 2025-12-17 22:54:18,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f08575bc060ce160028'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:18,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f00575bc060ce160016 2025-12-17 22:54:18,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f00575bc060ce160016 HTTP/1.1" 502 150 2025-12-17 22:54:18,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f00575bc060ce160016'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:19,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f01575bc060ce160019 2025-12-17 22:54:19,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f01575bc060ce160019 HTTP/1.1" 502 150 2025-12-17 22:54:19,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f01575bc060ce160019'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:19,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f03575bc060ce16001c 2025-12-17 22:54:19,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f03575bc060ce16001c HTTP/1.1" 502 150 2025-12-17 22:54:19,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f03575bc060ce16001c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:20,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f04575bc060ce16001f 2025-12-17 22:54:20,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f04575bc060ce16001f HTTP/1.1" 502 150 2025-12-17 22:54:20,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f04575bc060ce16001f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:21,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efa575bc060ce160003 2025-12-17 22:54:21,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efa575bc060ce160003 HTTP/1.1" 502 150 2025-12-17 22:54:21,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408efa575bc060ce160003'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:22,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f05575bc060ce160022 2025-12-17 22:54:22,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f05575bc060ce160022 HTTP/1.1" 502 150 2025-12-17 22:54:22,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f05575bc060ce160022'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:22,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f07575bc060ce160025 2025-12-17 22:54:22,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f07575bc060ce160025 HTTP/1.1" 502 150 2025-12-17 22:54:22,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f07575bc060ce160025'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:22,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094bb575bc060ce1610e8 2025-12-17 22:54:22,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bb575bc060ce1610e8 HTTP/1.1" 502 150 2025-12-17 22:54:22,942 - lava_callback - ERROR - Error processing callback for node 694094bb575bc060ce1610e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094bb575bc060ce1610e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094bb575bc060ce1610e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094bb575bc060ce1610e8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:22,944 - lava_callback - INFO - Starting async job submit for node 69408f09575bc060ce16002b 2025-12-17 22:54:22,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:22,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f09575bc060ce16002b HTTP/1.1" 502 150 2025-12-17 22:54:22,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f09575bc060ce16002b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:22,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f09575bc060ce16002b 2025-12-17 22:54:22,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f09575bc060ce16002b HTTP/1.1" 502 150 2025-12-17 22:54:22,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f09575bc060ce16002b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:22,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f08575bc060ce160028 2025-12-17 22:54:22,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f08575bc060ce160028 HTTP/1.1" 502 150 2025-12-17 22:54:22,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f08575bc060ce160028'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:23,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef4575bc060ce15ffec 2025-12-17 22:54:23,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef4575bc060ce15ffec HTTP/1.1" 502 150 2025-12-17 22:54:23,348 - lava_callback - ERROR - Error processing callback for node 69408ef4575bc060ce15ffec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef4575bc060ce15ffec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef4575bc060ce15ffec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef4575bc060ce15ffec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:23,350 - lava_callback - INFO - Starting async job submit for node 69408f0b575bc060ce160033 2025-12-17 22:54:23,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:23,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0b575bc060ce160033 HTTP/1.1" 502 150 2025-12-17 22:54:23,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0b575bc060ce160033'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:23,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0b575bc060ce160033 2025-12-17 22:54:23,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0b575bc060ce160033 HTTP/1.1" 502 150 2025-12-17 22:54:23,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0b575bc060ce160033'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:24,304 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ef7575bc060ce15fff5 2025-12-17 22:54:24,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ef7575bc060ce15fff5 HTTP/1.1" 502 150 2025-12-17 22:54:24,306 - lava_callback - ERROR - Error processing callback for node 69408ef7575bc060ce15fff5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef7575bc060ce15fff5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef7575bc060ce15fff5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ef7575bc060ce15fff5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:24,307 - lava_callback - INFO - Starting async job submit for node 69408f0c575bc060ce16003d 2025-12-17 22:54:24,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:24,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0c575bc060ce16003d HTTP/1.1" 502 150 2025-12-17 22:54:24,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0c575bc060ce16003d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:24,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0c575bc060ce16003d 2025-12-17 22:54:24,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0c575bc060ce16003d HTTP/1.1" 502 150 2025-12-17 22:54:24,315 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0c575bc060ce16003d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:24,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efb575bc060ce16000a 2025-12-17 22:54:24,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efb575bc060ce16000a HTTP/1.1" 502 150 2025-12-17 22:54:24,500 - lava_callback - ERROR - Error processing callback for node 69408efb575bc060ce16000a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408efb575bc060ce16000a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408efb575bc060ce16000a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408efb575bc060ce16000a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:24,501 - lava_callback - INFO - Starting async job submit for node 69408f0d575bc060ce160044 2025-12-17 22:54:24,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:24,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0d575bc060ce160044 HTTP/1.1" 502 150 2025-12-17 22:54:24,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0d575bc060ce160044'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:24,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0d575bc060ce160044 2025-12-17 22:54:24,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0d575bc060ce160044 HTTP/1.1" 502 150 2025-12-17 22:54:24,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0d575bc060ce160044'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:24,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f09575bc060ce16002b 2025-12-17 22:54:24,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f09575bc060ce16002b HTTP/1.1" 502 150 2025-12-17 22:54:24,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f09575bc060ce16002b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:25,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0b575bc060ce160033 2025-12-17 22:54:25,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0b575bc060ce160033 HTTP/1.1" 502 150 2025-12-17 22:54:25,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0b575bc060ce160033'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:25,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eea575bc060ce15ffd4 HTTP/1.1" 502 150 2025-12-17 22:54:25,712 - lava_callback - ERROR - Error processing callback for node 69408eea575bc060ce15ffd4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eea575bc060ce15ffd4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eea575bc060ce15ffd4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eea575bc060ce15ffd4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:25,714 - lava_callback - INFO - Starting async job submit for node 69408c8a575bc060ce15f5f9 2025-12-17 22:54:25,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:25,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8a575bc060ce15f5f9 HTTP/1.1" 502 150 2025-12-17 22:54:25,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8a575bc060ce15f5f9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:25,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8a575bc060ce15f5f9 2025-12-17 22:54:25,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8a575bc060ce15f5f9 HTTP/1.1" 502 150 2025-12-17 22:54:25,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8a575bc060ce15f5f9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:25,951 - lava_callback - INFO - Received callback for node 6940918d575bc060ce1608a9 INFO: 172.21.0.1:39010 - "POST /node/6940918d575bc060ce1608a9 HTTP/1.0" 202 Accepted 2025-12-17 22:54:26,315 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0c575bc060ce16003d 2025-12-17 22:54:26,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0c575bc060ce16003d HTTP/1.1" 502 150 2025-12-17 22:54:26,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0c575bc060ce16003d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:26,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0d575bc060ce160044 2025-12-17 22:54:26,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0d575bc060ce160044 HTTP/1.1" 502 150 2025-12-17 22:54:26,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0d575bc060ce160044'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:26,513 - lava_callback - INFO - Received callback for node 69409197575bc060ce1608c8 INFO: 172.21.0.1:39026 - "POST /node/69409197575bc060ce1608c8 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:39038 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:54:26,586 - lava_callback - INFO - Received callback for node 69409190575bc060ce1608bd INFO: 172.21.0.1:39052 - "POST /node/69409190575bc060ce1608bd HTTP/1.0" 202 Accepted 2025-12-17 22:54:27,231 - lava_callback - INFO - Received callback for node 69409192575bc060ce1608c0 INFO: 172.21.0.1:39066 - "POST /node/69409192575bc060ce1608c0 HTTP/1.0" 202 Accepted 2025-12-17 22:54:27,289 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efd575bc060ce16000d 2025-12-17 22:54:27,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efd575bc060ce16000d HTTP/1.1" 502 150 2025-12-17 22:54:27,291 - lava_callback - ERROR - Error processing callback for node 69408efd575bc060ce16000d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408efd575bc060ce16000d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408efd575bc060ce16000d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408efd575bc060ce16000d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:27,292 - lava_callback - INFO - Starting async job submit for node 69408f11575bc060ce160057 2025-12-17 22:54:27,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:27,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f11575bc060ce160057 HTTP/1.1" 502 150 2025-12-17 22:54:27,299 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f11575bc060ce160057'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:27,299 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f11575bc060ce160057 2025-12-17 22:54:27,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f11575bc060ce160057 HTTP/1.1" 502 150 2025-12-17 22:54:27,300 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f11575bc060ce160057'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:27,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efe575bc060ce160010 2025-12-17 22:54:27,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efe575bc060ce160010 HTTP/1.1" 502 150 2025-12-17 22:54:27,620 - lava_callback - ERROR - Error processing callback for node 69408efe575bc060ce160010: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408efe575bc060ce160010 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408efe575bc060ce160010 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408efe575bc060ce160010 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:27,622 - lava_callback - INFO - Starting async job submit for node 69408f14575bc060ce160067 2025-12-17 22:54:27,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:27,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f14575bc060ce160067 HTTP/1.1" 502 150 2025-12-17 22:54:27,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f14575bc060ce160067'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:27,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f14575bc060ce160067 2025-12-17 22:54:27,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f14575bc060ce160067 HTTP/1.1" 502 150 2025-12-17 22:54:27,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f14575bc060ce160067'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:27,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8a575bc060ce15f5f9 2025-12-17 22:54:27,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8a575bc060ce15f5f9 HTTP/1.1" 502 150 2025-12-17 22:54:27,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8a575bc060ce15f5f9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:27,883 - lava_callback - INFO - Received callback for node 69409194575bc060ce1608c3 INFO: 172.21.0.1:39080 - "POST /node/69409194575bc060ce1608c3 HTTP/1.0" 202 Accepted 2025-12-17 22:54:28,536 - lava_callback - INFO - Received callback for node 69409198575bc060ce1608cb INFO: 172.21.0.1:39094 - "POST /node/69409198575bc060ce1608cb HTTP/1.0" 202 Accepted 2025-12-17 22:54:28,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f09575bc060ce16002b 2025-12-17 22:54:28,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f09575bc060ce16002b HTTP/1.1" 502 150 2025-12-17 22:54:28,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f09575bc060ce16002b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:29,180 - lava_callback - INFO - Received callback for node 6940919a575bc060ce1608ce INFO: 172.21.0.1:39108 - "POST /node/6940919a575bc060ce1608ce HTTP/1.0" 202 Accepted 2025-12-17 22:54:29,300 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f11575bc060ce160057 2025-12-17 22:54:29,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f11575bc060ce160057 HTTP/1.1" 502 150 2025-12-17 22:54:29,301 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f11575bc060ce160057'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:29,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0b575bc060ce160033 2025-12-17 22:54:29,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0b575bc060ce160033 HTTP/1.1" 502 150 2025-12-17 22:54:29,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0b575bc060ce160033'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:29,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f14575bc060ce160067 2025-12-17 22:54:29,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f14575bc060ce160067 HTTP/1.1" 502 150 2025-12-17 22:54:29,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f14575bc060ce160067'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:29,820 - lava_callback - INFO - Received callback for node 6940919c575bc060ce1608d1 INFO: 172.21.0.1:40718 - "POST /node/6940919c575bc060ce1608d1 HTTP/1.0" 202 Accepted 2025-12-17 22:54:30,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0c575bc060ce16003d 2025-12-17 22:54:30,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0c575bc060ce16003d HTTP/1.1" 502 150 2025-12-17 22:54:30,318 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0c575bc060ce16003d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:30,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0d575bc060ce160044 2025-12-17 22:54:30,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0d575bc060ce160044 HTTP/1.1" 502 150 2025-12-17 22:54:30,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0d575bc060ce160044'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:30,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408eff575bc060ce160013 2025-12-17 22:54:30,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408eff575bc060ce160013 HTTP/1.1" 502 150 2025-12-17 22:54:30,953 - lava_callback - ERROR - Error processing callback for node 69408eff575bc060ce160013: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eff575bc060ce160013 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eff575bc060ce160013 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408eff575bc060ce160013 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:30,955 - lava_callback - INFO - Starting async job submit for node 69408f0e575bc060ce16004b 2025-12-17 22:54:30,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:30,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0e575bc060ce16004b HTTP/1.1" 502 150 2025-12-17 22:54:30,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0e575bc060ce16004b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:30,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0e575bc060ce16004b 2025-12-17 22:54:30,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0e575bc060ce16004b HTTP/1.1" 502 150 2025-12-17 22:54:30,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0e575bc060ce16004b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:30,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f08575bc060ce160028 2025-12-17 22:54:30,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f08575bc060ce160028 HTTP/1.1" 502 150 2025-12-17 22:54:30,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f08575bc060ce160028'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:31,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8a575bc060ce15f5f9 2025-12-17 22:54:32,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0e575bc060ce16004b 2025-12-17 22:54:32,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0e575bc060ce16004b HTTP/1.1" 502 150 2025-12-17 22:54:32,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0e575bc060ce16004b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:33,301 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f11575bc060ce160057 2025-12-17 22:54:33,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f11575bc060ce160057 HTTP/1.1" 502 150 2025-12-17 22:54:33,303 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f11575bc060ce160057'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:33,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f14575bc060ce160067 2025-12-17 22:54:33,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f14575bc060ce160067 HTTP/1.1" 502 150 2025-12-17 22:54:33,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f14575bc060ce160067'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:34,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f00575bc060ce160016 2025-12-17 22:54:34,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f00575bc060ce160016 HTTP/1.1" 502 150 2025-12-17 22:54:34,997 - lava_callback - ERROR - Error processing callback for node 69408f00575bc060ce160016: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f00575bc060ce160016 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f00575bc060ce160016 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f00575bc060ce160016 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:34,999 - lava_callback - INFO - Starting async job submit for node 69408f10575bc060ce160052 2025-12-17 22:54:35,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:35,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f10575bc060ce160052 HTTP/1.1" 502 150 2025-12-17 22:54:35,198 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f10575bc060ce160052'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:35,199 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f10575bc060ce160052 2025-12-17 22:54:35,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f10575bc060ce160052 HTTP/1.1" 502 150 2025-12-17 22:54:35,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f10575bc060ce160052'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:35,603 - lava_callback - INFO - Received callback for node 694091a0575bc060ce1608d9 INFO: 172.21.0.1:40730 - "POST /node/694091a0575bc060ce1608d9 HTTP/1.0" 202 Accepted 2025-12-17 22:54:35,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f01575bc060ce160019 2025-12-17 22:54:35,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f01575bc060ce160019 HTTP/1.1" 502 150 2025-12-17 22:54:35,627 - lava_callback - ERROR - Error processing callback for node 69408f01575bc060ce160019: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f01575bc060ce160019 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f01575bc060ce160019 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f01575bc060ce160019 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:35,632 - lava_callback - INFO - Starting async job submit for node 69408f1c575bc060ce160076 2025-12-17 22:54:35,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:35,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1c575bc060ce160076 HTTP/1.1" 502 150 2025-12-17 22:54:35,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1c575bc060ce160076'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:35,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1c575bc060ce160076 2025-12-17 22:54:35,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1c575bc060ce160076 HTTP/1.1" 502 150 2025-12-17 22:54:35,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1c575bc060ce160076'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:35,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f03575bc060ce16001c 2025-12-17 22:54:35,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f03575bc060ce16001c HTTP/1.1" 502 150 2025-12-17 22:54:35,671 - lava_callback - ERROR - Error processing callback for node 69408f03575bc060ce16001c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f03575bc060ce16001c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f03575bc060ce16001c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f03575bc060ce16001c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:35,674 - lava_callback - INFO - Starting async job submit for node 69408f1d575bc060ce160079 2025-12-17 22:54:35,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:35,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1d575bc060ce160079 HTTP/1.1" 502 150 2025-12-17 22:54:35,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1d575bc060ce160079'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:35,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1d575bc060ce160079 2025-12-17 22:54:35,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1d575bc060ce160079 HTTP/1.1" 502 150 2025-12-17 22:54:35,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1d575bc060ce160079'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:36,284 - lava_callback - INFO - Received callback for node 694091a8575bc060ce1608f7 INFO: 172.21.0.1:40746 - "POST /node/694091a8575bc060ce1608f7 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:40756 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:54:36,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f09575bc060ce16002b 2025-12-17 22:54:36,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f09575bc060ce16002b HTTP/1.1" 502 150 2025-12-17 22:54:36,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f09575bc060ce16002b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:36,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f04575bc060ce16001f 2025-12-17 22:54:36,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f04575bc060ce16001f HTTP/1.1" 502 150 2025-12-17 22:54:36,967 - lava_callback - ERROR - Error processing callback for node 69408f04575bc060ce16001f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f04575bc060ce16001f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f04575bc060ce16001f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f04575bc060ce16001f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:36,969 - lava_callback - INFO - Starting async job submit for node 6942bf7e1b61f1c6005fe7f9 2025-12-17 22:54:36,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:36,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942bf7e1b61f1c6005fe7f9 HTTP/1.1" 502 150 2025-12-17 22:54:36,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942bf7e1b61f1c6005fe7f9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:36,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942bf7e1b61f1c6005fe7f9 2025-12-17 22:54:36,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942bf7e1b61f1c6005fe7f9 HTTP/1.1" 502 150 2025-12-17 22:54:36,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942bf7e1b61f1c6005fe7f9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:36,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0e575bc060ce16004b 2025-12-17 22:54:36,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0e575bc060ce16004b HTTP/1.1" 502 150 2025-12-17 22:54:36,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0e575bc060ce16004b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:37,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f10575bc060ce160052 2025-12-17 22:54:37,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f10575bc060ce160052 HTTP/1.1" 502 150 2025-12-17 22:54:37,201 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f10575bc060ce160052'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:37,369 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0b575bc060ce160033 2025-12-17 22:54:37,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0b575bc060ce160033 HTTP/1.1" 502 150 2025-12-17 22:54:37,371 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0b575bc060ce160033'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:37,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408efa575bc060ce160003 2025-12-17 22:54:37,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408efa575bc060ce160003 HTTP/1.1" 502 150 2025-12-17 22:54:37,427 - lava_callback - ERROR - Error processing callback for node 69408efa575bc060ce160003: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408efa575bc060ce160003 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408efa575bc060ce160003 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408efa575bc060ce160003 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:37,429 - lava_callback - INFO - Starting async job submit for node 69408f18575bc060ce16006d 2025-12-17 22:54:37,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:37,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f18575bc060ce16006d HTTP/1.1" 502 150 2025-12-17 22:54:37,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f18575bc060ce16006d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:37,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f18575bc060ce16006d 2025-12-17 22:54:37,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f18575bc060ce16006d HTTP/1.1" 502 150 2025-12-17 22:54:37,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f18575bc060ce16006d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:37,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1c575bc060ce160076 2025-12-17 22:54:37,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1c575bc060ce160076 HTTP/1.1" 502 150 2025-12-17 22:54:37,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1c575bc060ce160076'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:37,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1d575bc060ce160079 2025-12-17 22:54:37,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1d575bc060ce160079 HTTP/1.1" 502 150 2025-12-17 22:54:37,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1d575bc060ce160079'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:38,318 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0c575bc060ce16003d 2025-12-17 22:54:38,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0c575bc060ce16003d HTTP/1.1" 502 150 2025-12-17 22:54:38,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0c575bc060ce16003d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:38,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0d575bc060ce160044 2025-12-17 22:54:38,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0d575bc060ce160044 HTTP/1.1" 502 150 2025-12-17 22:54:38,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0d575bc060ce160044'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:38,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f05575bc060ce160022 2025-12-17 22:54:38,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f05575bc060ce160022 HTTP/1.1" 502 150 2025-12-17 22:54:38,803 - lava_callback - ERROR - Error processing callback for node 69408f05575bc060ce160022: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f05575bc060ce160022 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f05575bc060ce160022 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f05575bc060ce160022 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:38,804 - lava_callback - INFO - Starting async job submit for node 69408f16575bc060ce16006a 2025-12-17 22:54:38,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:38,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f16575bc060ce16006a HTTP/1.1" 502 150 2025-12-17 22:54:38,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f16575bc060ce16006a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:38,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f16575bc060ce16006a 2025-12-17 22:54:38,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f16575bc060ce16006a HTTP/1.1" 502 150 2025-12-17 22:54:38,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f16575bc060ce16006a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:38,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f07575bc060ce160025 2025-12-17 22:54:38,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f07575bc060ce160025 HTTP/1.1" 502 150 2025-12-17 22:54:38,837 - lava_callback - ERROR - Error processing callback for node 69408f07575bc060ce160025: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f07575bc060ce160025 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f07575bc060ce160025 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f07575bc060ce160025 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:38,839 - lava_callback - INFO - Starting async job submit for node 69408f19575bc060ce160070 2025-12-17 22:54:38,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:38,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f19575bc060ce160070 HTTP/1.1" 502 150 2025-12-17 22:54:38,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f19575bc060ce160070'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:38,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f19575bc060ce160070 2025-12-17 22:54:38,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f19575bc060ce160070 HTTP/1.1" 502 150 2025-12-17 22:54:38,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f19575bc060ce160070'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:38,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942bf7e1b61f1c6005fe7f9 2025-12-17 22:54:38,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942bf7e1b61f1c6005fe7f9 HTTP/1.1" 502 150 2025-12-17 22:54:38,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942bf7e1b61f1c6005fe7f9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:39,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f18575bc060ce16006d 2025-12-17 22:54:39,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f18575bc060ce16006d HTTP/1.1" 502 150 2025-12-17 22:54:39,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f18575bc060ce16006d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:40,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f16575bc060ce16006a 2025-12-17 22:54:40,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f16575bc060ce16006a HTTP/1.1" 502 150 2025-12-17 22:54:40,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f16575bc060ce16006a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:40,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f19575bc060ce160070 2025-12-17 22:54:40,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f19575bc060ce160070 HTTP/1.1" 502 150 2025-12-17 22:54:40,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f19575bc060ce160070'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:41,201 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f10575bc060ce160052 2025-12-17 22:54:41,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f10575bc060ce160052 HTTP/1.1" 502 150 2025-12-17 22:54:41,203 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f10575bc060ce160052'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:41,303 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f11575bc060ce160057 2025-12-17 22:54:41,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f11575bc060ce160057 HTTP/1.1" 502 150 2025-12-17 22:54:41,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f11575bc060ce160057'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:41,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f14575bc060ce160067 2025-12-17 22:54:41,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f14575bc060ce160067 HTTP/1.1" 502 150 2025-12-17 22:54:41,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f14575bc060ce160067'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:41,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1c575bc060ce160076 2025-12-17 22:54:41,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1c575bc060ce160076 HTTP/1.1" 502 150 2025-12-17 22:54:41,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1c575bc060ce160076'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:41,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1d575bc060ce160079 2025-12-17 22:54:41,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1d575bc060ce160079 HTTP/1.1" 502 150 2025-12-17 22:54:41,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1d575bc060ce160079'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:42,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942bf7e1b61f1c6005fe7f9 2025-12-17 22:54:42,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942bf7e1b61f1c6005fe7f9 HTTP/1.1" 502 150 2025-12-17 22:54:42,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942bf7e1b61f1c6005fe7f9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:43,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f18575bc060ce16006d 2025-12-17 22:54:43,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f18575bc060ce16006d HTTP/1.1" 502 150 2025-12-17 22:54:43,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f18575bc060ce16006d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:44,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f16575bc060ce16006a 2025-12-17 22:54:44,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f16575bc060ce16006a HTTP/1.1" 502 150 2025-12-17 22:54:44,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f16575bc060ce16006a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:44,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f19575bc060ce160070 2025-12-17 22:54:44,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f19575bc060ce160070 HTTP/1.1" 502 150 2025-12-17 22:54:44,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f19575bc060ce160070'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:44,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0e575bc060ce16004b 2025-12-17 22:54:44,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0e575bc060ce16004b HTTP/1.1" 502 150 2025-12-17 22:54:44,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f0e575bc060ce16004b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:45,937 - lava_callback - INFO - Received callback for node 694091ac575bc060ce1608fd INFO: 172.21.0.1:47404 - "POST /node/694091ac575bc060ce1608fd HTTP/1.0" 202 Accepted 2025-12-17 22:54:46,497 - lava_callback - INFO - Received callback for node 694091a4575bc060ce1608e6 INFO: 172.21.0.1:47410 - "POST /node/694091a4575bc060ce1608e6 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:47426 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:54:46,575 - lava_callback - INFO - Received callback for node 694091af575bc060ce160900 INFO: 172.21.0.1:47430 - "POST /node/694091af575bc060ce160900 HTTP/1.0" 202 Accepted 2025-12-17 22:54:46,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f08575bc060ce160028 2025-12-17 22:54:47,212 - lava_callback - INFO - Received callback for node 694091b0575bc060ce160903 INFO: 172.21.0.1:47442 - "POST /node/694091b0575bc060ce160903 HTTP/1.0" 202 Accepted 2025-12-17 22:54:47,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8a575bc060ce15f5f9 HTTP/1.1" 502 150 2025-12-17 22:54:47,220 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8a575bc060ce15f5f9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:47,856 - lava_callback - INFO - Received callback for node 694091b2575bc060ce160906 INFO: 172.21.0.1:47454 - "POST /node/694091b2575bc060ce160906 HTTP/1.0" 202 Accepted 2025-12-17 22:54:48,159 - lava_callback - INFO - Received callback for node 69410c49460bc8e265117571 INFO: 172.21.0.1:47470 - "POST /node/69410c49460bc8e265117571 HTTP/1.0" 202 Accepted 2025-12-17 22:54:48,513 - lava_callback - INFO - Received callback for node 694091b4575bc060ce16090b INFO: 172.21.0.1:47478 - "POST /node/694091b4575bc060ce16090b HTTP/1.0" 202 Accepted 2025-12-17 22:54:49,203 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f10575bc060ce160052 2025-12-17 22:54:49,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f10575bc060ce160052 HTTP/1.1" 502 150 2025-12-17 22:54:49,204 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f10575bc060ce160052'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:49,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1c575bc060ce160076 2025-12-17 22:54:49,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1c575bc060ce160076 HTTP/1.1" 502 150 2025-12-17 22:54:49,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1c575bc060ce160076'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:49,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1d575bc060ce160079 2025-12-17 22:54:49,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1d575bc060ce160079 HTTP/1.1" 502 150 2025-12-17 22:54:49,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1d575bc060ce160079'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:50,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942bf7e1b61f1c6005fe7f9 2025-12-17 22:54:50,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942bf7e1b61f1c6005fe7f9 HTTP/1.1" 502 150 2025-12-17 22:54:50,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942bf7e1b61f1c6005fe7f9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:51,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f18575bc060ce16006d 2025-12-17 22:54:51,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f18575bc060ce16006d HTTP/1.1" 502 150 2025-12-17 22:54:51,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f18575bc060ce16006d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:52,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f16575bc060ce16006a 2025-12-17 22:54:52,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f16575bc060ce16006a HTTP/1.1" 502 150 2025-12-17 22:54:52,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f16575bc060ce16006a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:52,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f19575bc060ce160070 2025-12-17 22:54:52,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f19575bc060ce160070 HTTP/1.1" 502 150 2025-12-17 22:54:52,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f19575bc060ce160070'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:52,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f09575bc060ce16002b 2025-12-17 22:54:52,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f09575bc060ce16002b HTTP/1.1" 502 150 2025-12-17 22:54:52,958 - lava_callback - ERROR - Error processing callback for node 69408f09575bc060ce16002b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f09575bc060ce16002b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f09575bc060ce16002b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f09575bc060ce16002b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:52,960 - lava_callback - INFO - Starting async job submit for node 69408f1f575bc060ce160086 2025-12-17 22:54:52,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:52,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1f575bc060ce160086 HTTP/1.1" 502 150 2025-12-17 22:54:52,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1f575bc060ce160086'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:52,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1f575bc060ce160086 2025-12-17 22:54:52,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1f575bc060ce160086 HTTP/1.1" 502 150 2025-12-17 22:54:52,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1f575bc060ce160086'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:53,371 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0b575bc060ce160033 2025-12-17 22:54:53,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0b575bc060ce160033 HTTP/1.1" 502 150 2025-12-17 22:54:53,372 - lava_callback - ERROR - Error processing callback for node 69408f0b575bc060ce160033: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f0b575bc060ce160033 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f0b575bc060ce160033 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f0b575bc060ce160033 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:53,374 - lava_callback - INFO - Starting async job submit for node 69408f27575bc060ce160097 2025-12-17 22:54:53,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:53,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f27575bc060ce160097 HTTP/1.1" 502 150 2025-12-17 22:54:53,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f27575bc060ce160097'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:53,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f27575bc060ce160097 2025-12-17 22:54:53,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f27575bc060ce160097 HTTP/1.1" 502 150 2025-12-17 22:54:53,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f27575bc060ce160097'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:54,319 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0c575bc060ce16003d 2025-12-17 22:54:54,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0c575bc060ce16003d HTTP/1.1" 502 150 2025-12-17 22:54:54,320 - lava_callback - ERROR - Error processing callback for node 69408f0c575bc060ce16003d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f0c575bc060ce16003d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f0c575bc060ce16003d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f0c575bc060ce16003d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:54,322 - lava_callback - INFO - Starting async job submit for node 69408f25575bc060ce160091 2025-12-17 22:54:54,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:54,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f25575bc060ce160091 HTTP/1.1" 502 150 2025-12-17 22:54:54,334 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f25575bc060ce160091'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:54,334 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f25575bc060ce160091 2025-12-17 22:54:54,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f25575bc060ce160091 HTTP/1.1" 502 150 2025-12-17 22:54:54,335 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f25575bc060ce160091'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:54,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0d575bc060ce160044 2025-12-17 22:54:54,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0d575bc060ce160044 HTTP/1.1" 502 150 2025-12-17 22:54:54,518 - lava_callback - ERROR - Error processing callback for node 69408f0d575bc060ce160044: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f0d575bc060ce160044 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f0d575bc060ce160044 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f0d575bc060ce160044 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:54,520 - lava_callback - INFO - Starting async job submit for node 69408f26575bc060ce160094 2025-12-17 22:54:54,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:54,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f26575bc060ce160094 HTTP/1.1" 502 150 2025-12-17 22:54:54,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f26575bc060ce160094'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:54,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f26575bc060ce160094 2025-12-17 22:54:54,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f26575bc060ce160094 HTTP/1.1" 502 150 2025-12-17 22:54:54,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f26575bc060ce160094'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:54,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1f575bc060ce160086 2025-12-17 22:54:54,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1f575bc060ce160086 HTTP/1.1" 502 150 2025-12-17 22:54:54,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1f575bc060ce160086'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:55,220 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8a575bc060ce15f5f9 2025-12-17 22:54:55,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8a575bc060ce15f5f9 HTTP/1.1" 502 150 2025-12-17 22:54:55,222 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c8a575bc060ce15f5f9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:55,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f27575bc060ce160097 2025-12-17 22:54:55,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f27575bc060ce160097 HTTP/1.1" 502 150 2025-12-17 22:54:55,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f27575bc060ce160097'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:55,578 - lava_callback - INFO - Received callback for node 694091aa575bc060ce1608fa INFO: 172.21.0.1:41448 - "POST /node/694091aa575bc060ce1608fa HTTP/1.0" 202 Accepted 2025-12-17 22:54:56,335 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f25575bc060ce160091 2025-12-17 22:54:56,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f25575bc060ce160091 HTTP/1.1" 502 150 2025-12-17 22:54:56,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f25575bc060ce160091'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:56,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f26575bc060ce160094 2025-12-17 22:54:56,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f26575bc060ce160094 HTTP/1.1" 502 150 2025-12-17 22:54:56,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f26575bc060ce160094'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41456 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:54:57,304 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f11575bc060ce160057 2025-12-17 22:54:57,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f11575bc060ce160057 HTTP/1.1" 502 150 2025-12-17 22:54:57,306 - lava_callback - ERROR - Error processing callback for node 69408f11575bc060ce160057: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f11575bc060ce160057 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f11575bc060ce160057 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f11575bc060ce160057 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:57,307 - lava_callback - INFO - Starting async job submit for node 6940fea7460bc8e265116410 2025-12-17 22:54:57,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:57,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fea7460bc8e265116410 HTTP/1.1" 502 150 2025-12-17 22:54:57,333 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940fea7460bc8e265116410'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:57,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940fea7460bc8e265116410 2025-12-17 22:54:57,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fea7460bc8e265116410 HTTP/1.1" 502 150 2025-12-17 22:54:57,334 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940fea7460bc8e265116410'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:57,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f14575bc060ce160067 2025-12-17 22:54:57,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f14575bc060ce160067 HTTP/1.1" 502 150 2025-12-17 22:54:57,635 - lava_callback - ERROR - Error processing callback for node 69408f14575bc060ce160067: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f14575bc060ce160067 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f14575bc060ce160067 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f14575bc060ce160067 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:54:57,637 - lava_callback - INFO - Starting async job submit for node 69408f2a575bc060ce1600a0 2025-12-17 22:54:57,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:54:57,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2a575bc060ce1600a0 HTTP/1.1" 502 150 2025-12-17 22:54:57,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2a575bc060ce1600a0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:57,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2a575bc060ce1600a0 2025-12-17 22:54:57,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2a575bc060ce1600a0 HTTP/1.1" 502 150 2025-12-17 22:54:57,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2a575bc060ce1600a0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:58,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1f575bc060ce160086 2025-12-17 22:54:58,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1f575bc060ce160086 HTTP/1.1" 502 150 2025-12-17 22:54:58,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1f575bc060ce160086'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:59,334 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940fea7460bc8e265116410 2025-12-17 22:54:59,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fea7460bc8e265116410 HTTP/1.1" 502 150 2025-12-17 22:54:59,335 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940fea7460bc8e265116410'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:59,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f27575bc060ce160097 2025-12-17 22:54:59,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f27575bc060ce160097 HTTP/1.1" 502 150 2025-12-17 22:54:59,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f27575bc060ce160097'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:54:59,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2a575bc060ce1600a0 2025-12-17 22:54:59,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2a575bc060ce1600a0 HTTP/1.1" 502 150 2025-12-17 22:54:59,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2a575bc060ce1600a0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:00,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f25575bc060ce160091 2025-12-17 22:55:00,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f25575bc060ce160091 HTTP/1.1" 502 150 2025-12-17 22:55:00,338 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f25575bc060ce160091'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:00,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f26575bc060ce160094 2025-12-17 22:55:00,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f26575bc060ce160094 HTTP/1.1" 502 150 2025-12-17 22:55:00,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f26575bc060ce160094'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:00,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f0e575bc060ce16004b 2025-12-17 22:55:00,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f0e575bc060ce16004b HTTP/1.1" 502 150 2025-12-17 22:55:00,993 - lava_callback - ERROR - Error processing callback for node 69408f0e575bc060ce16004b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f0e575bc060ce16004b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f0e575bc060ce16004b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f0e575bc060ce16004b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:00,995 - lava_callback - INFO - Starting async job submit for node 69408f2b575bc060ce1600a3 2025-12-17 22:55:00,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:01,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2b575bc060ce1600a3 HTTP/1.1" 502 150 2025-12-17 22:55:01,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2b575bc060ce1600a3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:01,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2b575bc060ce1600a3 2025-12-17 22:55:01,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2b575bc060ce1600a3 HTTP/1.1" 502 150 2025-12-17 22:55:01,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2b575bc060ce1600a3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40654 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:55:03,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2b575bc060ce1600a3 2025-12-17 22:55:03,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2b575bc060ce1600a3 HTTP/1.1" 502 150 2025-12-17 22:55:03,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2b575bc060ce1600a3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:03,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940fea7460bc8e265116410 2025-12-17 22:55:03,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fea7460bc8e265116410 HTTP/1.1" 502 150 2025-12-17 22:55:03,338 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940fea7460bc8e265116410'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:03,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2a575bc060ce1600a0 2025-12-17 22:55:03,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2a575bc060ce1600a0 HTTP/1.1" 502 150 2025-12-17 22:55:03,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2a575bc060ce1600a0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:05,205 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f10575bc060ce160052 2025-12-17 22:55:05,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f10575bc060ce160052 HTTP/1.1" 502 150 2025-12-17 22:55:05,206 - lava_callback - ERROR - Error processing callback for node 69408f10575bc060ce160052: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f10575bc060ce160052 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f10575bc060ce160052 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f10575bc060ce160052 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:05,208 - lava_callback - INFO - Starting async job submit for node 69408f2d575bc060ce1600a6 2025-12-17 22:55:05,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:05,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2d575bc060ce1600a6 HTTP/1.1" 502 150 2025-12-17 22:55:05,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2d575bc060ce1600a6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:05,215 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2d575bc060ce1600a6 2025-12-17 22:55:05,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2d575bc060ce1600a6 HTTP/1.1" 502 150 2025-12-17 22:55:05,216 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2d575bc060ce1600a6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:05,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1c575bc060ce160076 2025-12-17 22:55:05,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1c575bc060ce160076 HTTP/1.1" 502 150 2025-12-17 22:55:05,648 - lava_callback - ERROR - Error processing callback for node 69408f1c575bc060ce160076: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f1c575bc060ce160076 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f1c575bc060ce160076 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f1c575bc060ce160076 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:05,649 - lava_callback - INFO - Starting async job submit for node 69408f2f575bc060ce1600ac 2025-12-17 22:55:05,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:05,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2f575bc060ce1600ac HTTP/1.1" 502 150 2025-12-17 22:55:05,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2f575bc060ce1600ac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:05,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2f575bc060ce1600ac 2025-12-17 22:55:05,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2f575bc060ce1600ac HTTP/1.1" 502 150 2025-12-17 22:55:05,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2f575bc060ce1600ac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:05,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1d575bc060ce160079 2025-12-17 22:55:05,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1d575bc060ce160079 HTTP/1.1" 502 150 2025-12-17 22:55:05,689 - lava_callback - ERROR - Error processing callback for node 69408f1d575bc060ce160079: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f1d575bc060ce160079 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f1d575bc060ce160079 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f1d575bc060ce160079 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:05,691 - lava_callback - INFO - Starting async job submit for node 69408f30575bc060ce1600af 2025-12-17 22:55:05,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:05,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f30575bc060ce1600af HTTP/1.1" 502 150 2025-12-17 22:55:05,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f30575bc060ce1600af'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:05,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f30575bc060ce1600af 2025-12-17 22:55:05,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f30575bc060ce1600af HTTP/1.1" 502 150 2025-12-17 22:55:05,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f30575bc060ce1600af'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:05,971 - lava_callback - INFO - Received callback for node 694091b7575bc060ce160916 INFO: 172.21.0.1:40670 - "POST /node/694091b7575bc060ce160916 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:40672 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:55:06,633 - lava_callback - INFO - Received callback for node 694091bb575bc060ce16092e INFO: 172.21.0.1:40674 - "POST /node/694091bb575bc060ce16092e HTTP/1.0" 202 Accepted 2025-12-17 22:55:06,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1f575bc060ce160086 2025-12-17 22:55:06,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1f575bc060ce160086 HTTP/1.1" 502 150 2025-12-17 22:55:06,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1f575bc060ce160086'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:06,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942bf7e1b61f1c6005fe7f9 2025-12-17 22:55:06,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942bf7e1b61f1c6005fe7f9 HTTP/1.1" 502 150 2025-12-17 22:55:06,986 - lava_callback - ERROR - Error processing callback for node 6942bf7e1b61f1c6005fe7f9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942bf7e1b61f1c6005fe7f9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942bf7e1b61f1c6005fe7f9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942bf7e1b61f1c6005fe7f9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:06,988 - lava_callback - INFO - Starting async job submit for node 69408f32575bc060ce1600b2 2025-12-17 22:55:06,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:06,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f32575bc060ce1600b2 HTTP/1.1" 502 150 2025-12-17 22:55:06,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f32575bc060ce1600b2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:06,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f32575bc060ce1600b2 2025-12-17 22:55:06,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f32575bc060ce1600b2 HTTP/1.1" 502 150 2025-12-17 22:55:06,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f32575bc060ce1600b2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:07,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2b575bc060ce1600a3 2025-12-17 22:55:07,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2b575bc060ce1600a3 HTTP/1.1" 502 150 2025-12-17 22:55:07,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2b575bc060ce1600a3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:07,216 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2d575bc060ce1600a6 2025-12-17 22:55:07,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2d575bc060ce1600a6 HTTP/1.1" 502 150 2025-12-17 22:55:07,217 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2d575bc060ce1600a6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:07,291 - lava_callback - INFO - Received callback for node 694091bd575bc060ce160931 INFO: 172.21.0.1:40676 - "POST /node/694091bd575bc060ce160931 HTTP/1.0" 202 Accepted 2025-12-17 22:55:07,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f18575bc060ce16006d 2025-12-17 22:55:07,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f18575bc060ce16006d HTTP/1.1" 502 150 2025-12-17 22:55:07,443 - lava_callback - ERROR - Error processing callback for node 69408f18575bc060ce16006d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f18575bc060ce16006d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f18575bc060ce16006d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f18575bc060ce16006d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:07,445 - lava_callback - INFO - Starting async job submit for node 69408f33575bc060ce1600b5 2025-12-17 22:55:07,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:07,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f33575bc060ce1600b5 HTTP/1.1" 502 150 2025-12-17 22:55:07,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f33575bc060ce1600b5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:07,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f33575bc060ce1600b5 2025-12-17 22:55:07,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f33575bc060ce1600b5 HTTP/1.1" 502 150 2025-12-17 22:55:07,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f33575bc060ce1600b5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:07,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f27575bc060ce160097 2025-12-17 22:55:07,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f27575bc060ce160097 HTTP/1.1" 502 150 2025-12-17 22:55:07,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f27575bc060ce160097'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:07,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2f575bc060ce1600ac 2025-12-17 22:55:07,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2f575bc060ce1600ac HTTP/1.1" 502 150 2025-12-17 22:55:07,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2f575bc060ce1600ac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:07,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f30575bc060ce1600af 2025-12-17 22:55:07,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f30575bc060ce1600af HTTP/1.1" 502 150 2025-12-17 22:55:07,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f30575bc060ce1600af'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:07,928 - lava_callback - INFO - Received callback for node 694091bf575bc060ce160934 INFO: 172.21.0.1:40678 - "POST /node/694091bf575bc060ce160934 HTTP/1.0" 202 Accepted 2025-12-17 22:55:08,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f25575bc060ce160091 2025-12-17 22:55:08,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f25575bc060ce160091 HTTP/1.1" 502 150 2025-12-17 22:55:08,340 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f25575bc060ce160091'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:08,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f26575bc060ce160094 2025-12-17 22:55:08,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f26575bc060ce160094 HTTP/1.1" 502 150 2025-12-17 22:55:08,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f26575bc060ce160094'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:08,597 - lava_callback - INFO - Received callback for node 694091c1575bc060ce160937 INFO: 172.21.0.1:40686 - "POST /node/694091c1575bc060ce160937 HTTP/1.0" 202 Accepted 2025-12-17 22:55:08,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f16575bc060ce16006a 2025-12-17 22:55:08,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f16575bc060ce16006a HTTP/1.1" 502 150 2025-12-17 22:55:08,817 - lava_callback - ERROR - Error processing callback for node 69408f16575bc060ce16006a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f16575bc060ce16006a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f16575bc060ce16006a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f16575bc060ce16006a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:08,820 - lava_callback - INFO - Starting async job submit for node 69408f36575bc060ce1600c3 2025-12-17 22:55:08,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:08,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f36575bc060ce1600c3 HTTP/1.1" 502 150 2025-12-17 22:55:08,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f36575bc060ce1600c3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:08,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f36575bc060ce1600c3 2025-12-17 22:55:08,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f36575bc060ce1600c3 HTTP/1.1" 502 150 2025-12-17 22:55:08,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f36575bc060ce1600c3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:08,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f19575bc060ce160070 2025-12-17 22:55:08,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f19575bc060ce160070 HTTP/1.1" 502 150 2025-12-17 22:55:08,856 - lava_callback - ERROR - Error processing callback for node 69408f19575bc060ce160070: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f19575bc060ce160070 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f19575bc060ce160070 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f19575bc060ce160070 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:08,859 - lava_callback - INFO - Starting async job submit for node 69408f3a575bc060ce1600cd 2025-12-17 22:55:08,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:08,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3a575bc060ce1600cd HTTP/1.1" 502 150 2025-12-17 22:55:08,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f3a575bc060ce1600cd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:08,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f3a575bc060ce1600cd 2025-12-17 22:55:08,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3a575bc060ce1600cd HTTP/1.1" 502 150 2025-12-17 22:55:08,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f3a575bc060ce1600cd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:08,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f32575bc060ce1600b2 2025-12-17 22:55:08,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f32575bc060ce1600b2 HTTP/1.1" 502 150 2025-12-17 22:55:08,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f32575bc060ce1600b2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:09,269 - lava_callback - INFO - Received callback for node 694091c3575bc060ce16093a INFO: 172.21.0.1:40700 - "POST /node/694091c3575bc060ce16093a HTTP/1.0" 202 Accepted 2025-12-17 22:55:09,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f33575bc060ce1600b5 2025-12-17 22:55:09,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f33575bc060ce1600b5 HTTP/1.1" 502 150 2025-12-17 22:55:09,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f33575bc060ce1600b5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:09,921 - lava_callback - INFO - Received callback for node 694091c5575bc060ce16093d INFO: 172.21.0.1:36342 - "POST /node/694091c5575bc060ce16093d HTTP/1.0" 202 Accepted 2025-12-17 22:55:10,567 - lava_callback - INFO - Received callback for node 694091c7575bc060ce160940 INFO: 172.21.0.1:36348 - "POST /node/694091c7575bc060ce160940 HTTP/1.0" 202 Accepted 2025-12-17 22:55:10,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f36575bc060ce1600c3 2025-12-17 22:55:10,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f36575bc060ce1600c3 HTTP/1.1" 502 150 2025-12-17 22:55:10,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f36575bc060ce1600c3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:10,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f3a575bc060ce1600cd 2025-12-17 22:55:10,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3a575bc060ce1600cd HTTP/1.1" 502 150 2025-12-17 22:55:10,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f3a575bc060ce1600cd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:11,218 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2d575bc060ce1600a6 2025-12-17 22:55:11,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2d575bc060ce1600a6 HTTP/1.1" 502 150 2025-12-17 22:55:11,219 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2d575bc060ce1600a6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:11,222 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c8a575bc060ce15f5f9 2025-12-17 22:55:11,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c8a575bc060ce15f5f9 HTTP/1.1" 502 150 2025-12-17 22:55:11,223 - lava_callback - ERROR - Error processing callback for node 69408c8a575bc060ce15f5f9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c8a575bc060ce15f5f9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c8a575bc060ce15f5f9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c8a575bc060ce15f5f9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:11,225 - lava_callback - INFO - Starting async job submit for node 69408f3b575bc060ce1600d0 2025-12-17 22:55:11,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:11,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3b575bc060ce1600d0 HTTP/1.1" 502 150 2025-12-17 22:55:11,231 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f3b575bc060ce1600d0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:11,231 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f3b575bc060ce1600d0 2025-12-17 22:55:11,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3b575bc060ce1600d0 HTTP/1.1" 502 150 2025-12-17 22:55:11,232 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f3b575bc060ce1600d0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:11,339 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940fea7460bc8e265116410 2025-12-17 22:55:11,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fea7460bc8e265116410 HTTP/1.1" 502 150 2025-12-17 22:55:11,339 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940fea7460bc8e265116410'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:11,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2a575bc060ce1600a0 2025-12-17 22:55:11,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2a575bc060ce1600a0 HTTP/1.1" 502 150 2025-12-17 22:55:11,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2a575bc060ce1600a0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:11,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2f575bc060ce1600ac 2025-12-17 22:55:11,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2f575bc060ce1600ac HTTP/1.1" 502 150 2025-12-17 22:55:11,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2f575bc060ce1600ac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:11,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f30575bc060ce1600af 2025-12-17 22:55:11,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f30575bc060ce1600af HTTP/1.1" 502 150 2025-12-17 22:55:11,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f30575bc060ce1600af'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:12,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f32575bc060ce1600b2 2025-12-17 22:55:12,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f32575bc060ce1600b2 HTTP/1.1" 502 150 2025-12-17 22:55:13,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f32575bc060ce1600b2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:13,232 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f3b575bc060ce1600d0 2025-12-17 22:55:13,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f33575bc060ce1600b5 2025-12-17 22:55:13,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f33575bc060ce1600b5 HTTP/1.1" 502 150 2025-12-17 22:55:13,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f33575bc060ce1600b5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:14,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f36575bc060ce1600c3 2025-12-17 22:55:14,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f36575bc060ce1600c3 HTTP/1.1" 502 150 2025-12-17 22:55:14,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f36575bc060ce1600c3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:14,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f3a575bc060ce1600cd 2025-12-17 22:55:14,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3a575bc060ce1600cd HTTP/1.1" 502 150 2025-12-17 22:55:14,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f3a575bc060ce1600cd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:15,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2b575bc060ce1600a3 2025-12-17 22:55:15,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2b575bc060ce1600a3 HTTP/1.1" 502 150 2025-12-17 22:55:15,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2b575bc060ce1600a3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36354 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:55:15,590 - lava_callback - INFO - Received callback for node 694091cc575bc060ce16094c INFO: 172.21.0.1:36368 - "POST /node/694091cc575bc060ce16094c HTTP/1.0" 202 Accepted INFO: 172.21.0.1:36374 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:55:17,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f08575bc060ce160028 HTTP/1.1" 502 150 2025-12-17 22:55:17,680 - lava_callback - ERROR - Error processing callback for node 69408f08575bc060ce160028: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f08575bc060ce160028 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f08575bc060ce160028 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f08575bc060ce160028 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:17,682 - lava_callback - INFO - Starting async job submit for node 69408f2d575bc060ce1600a9 2025-12-17 22:55:17,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:17,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2d575bc060ce1600a9 HTTP/1.1" 502 150 2025-12-17 22:55:17,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2d575bc060ce1600a9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:17,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2d575bc060ce1600a9 2025-12-17 22:55:17,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2d575bc060ce1600a9 HTTP/1.1" 502 150 2025-12-17 22:55:17,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2d575bc060ce1600a9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:19,219 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2d575bc060ce1600a6 2025-12-17 22:55:19,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2d575bc060ce1600a6 HTTP/1.1" 502 150 2025-12-17 22:55:19,221 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2d575bc060ce1600a6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:19,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2f575bc060ce1600ac 2025-12-17 22:55:19,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2f575bc060ce1600ac HTTP/1.1" 502 150 2025-12-17 22:55:19,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2f575bc060ce1600ac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:19,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2d575bc060ce1600a9 2025-12-17 22:55:19,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2d575bc060ce1600a9 HTTP/1.1" 502 150 2025-12-17 22:55:19,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2d575bc060ce1600a9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:19,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f30575bc060ce1600af 2025-12-17 22:55:19,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f30575bc060ce1600af HTTP/1.1" 502 150 2025-12-17 22:55:19,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f30575bc060ce1600af'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:21,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f32575bc060ce1600b2 2025-12-17 22:55:21,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f32575bc060ce1600b2 HTTP/1.1" 502 150 2025-12-17 22:55:21,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f32575bc060ce1600b2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:21,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f33575bc060ce1600b5 2025-12-17 22:55:21,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f33575bc060ce1600b5 HTTP/1.1" 502 150 2025-12-17 22:55:21,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f33575bc060ce1600b5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:22,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f36575bc060ce1600c3 2025-12-17 22:55:22,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f36575bc060ce1600c3 HTTP/1.1" 502 150 2025-12-17 22:55:22,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f36575bc060ce1600c3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:22,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f3a575bc060ce1600cd 2025-12-17 22:55:22,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3a575bc060ce1600cd HTTP/1.1" 502 150 2025-12-17 22:55:22,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f3a575bc060ce1600cd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:22,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1f575bc060ce160086 2025-12-17 22:55:22,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1f575bc060ce160086 HTTP/1.1" 502 150 2025-12-17 22:55:22,978 - lava_callback - ERROR - Error processing callback for node 69408f1f575bc060ce160086: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f1f575bc060ce160086 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f1f575bc060ce160086 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f1f575bc060ce160086 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:22,980 - lava_callback - INFO - Starting async job submit for node 69408f34575bc060ce1600ba 2025-12-17 22:55:22,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:22,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f34575bc060ce1600ba HTTP/1.1" 502 150 2025-12-17 22:55:22,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f34575bc060ce1600ba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:22,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f34575bc060ce1600ba 2025-12-17 22:55:22,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f34575bc060ce1600ba HTTP/1.1" 502 150 2025-12-17 22:55:22,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f34575bc060ce1600ba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:23,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f27575bc060ce160097 2025-12-17 22:55:23,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f27575bc060ce160097 HTTP/1.1" 502 150 2025-12-17 22:55:23,578 - lava_callback - ERROR - Error processing callback for node 69408f27575bc060ce160097: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f27575bc060ce160097 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f27575bc060ce160097 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f27575bc060ce160097 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:23,580 - lava_callback - INFO - Starting async job submit for node 69408f3c575bc060ce1600d3 2025-12-17 22:55:23,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:23,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3c575bc060ce1600d3 HTTP/1.1" 502 150 2025-12-17 22:55:23,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f3c575bc060ce1600d3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:23,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f3c575bc060ce1600d3 2025-12-17 22:55:23,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3c575bc060ce1600d3 HTTP/1.1" 502 150 2025-12-17 22:55:23,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f3c575bc060ce1600d3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:23,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2d575bc060ce1600a9 2025-12-17 22:55:23,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2d575bc060ce1600a9 HTTP/1.1" 502 150 2025-12-17 22:55:23,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2d575bc060ce1600a9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:24,340 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f25575bc060ce160091 2025-12-17 22:55:24,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f26575bc060ce160094 2025-12-17 22:55:24,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f26575bc060ce160094 HTTP/1.1" 502 150 2025-12-17 22:55:24,542 - lava_callback - ERROR - Error processing callback for node 69408f26575bc060ce160094: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f26575bc060ce160094 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f26575bc060ce160094 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f26575bc060ce160094 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:24,544 - lava_callback - INFO - Starting async job submit for node 69409533575bc060ce16126e 2025-12-17 22:55:24,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:24,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409533575bc060ce16126e HTTP/1.1" 502 150 2025-12-17 22:55:24,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409533575bc060ce16126e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:24,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409533575bc060ce16126e 2025-12-17 22:55:24,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409533575bc060ce16126e HTTP/1.1" 502 150 2025-12-17 22:55:24,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409533575bc060ce16126e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:24,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f34575bc060ce1600ba 2025-12-17 22:55:24,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f34575bc060ce1600ba HTTP/1.1" 502 150 2025-12-17 22:55:24,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f34575bc060ce1600ba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:25,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f3c575bc060ce1600d3 2025-12-17 22:55:25,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3c575bc060ce1600d3 HTTP/1.1" 502 150 2025-12-17 22:55:25,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f3c575bc060ce1600d3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:25,991 - lava_callback - INFO - Received callback for node 694091ca575bc060ce160949 INFO: 172.21.0.1:59764 - "POST /node/694091ca575bc060ce160949 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:59774 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:55:26,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409533575bc060ce16126e 2025-12-17 22:55:26,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409533575bc060ce16126e HTTP/1.1" 502 150 2025-12-17 22:55:26,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409533575bc060ce16126e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:27,340 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940fea7460bc8e265116410 2025-12-17 22:55:27,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940fea7460bc8e265116410 HTTP/1.1" 502 150 2025-12-17 22:55:27,341 - lava_callback - ERROR - Error processing callback for node 6940fea7460bc8e265116410: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940fea7460bc8e265116410 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940fea7460bc8e265116410 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940fea7460bc8e265116410 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:27,343 - lava_callback - INFO - Starting async job submit for node 69409613575bc060ce1614d1 2025-12-17 22:55:27,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:27,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409613575bc060ce1614d1 HTTP/1.1" 502 150 2025-12-17 22:55:27,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409613575bc060ce1614d1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:27,381 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409613575bc060ce1614d1 2025-12-17 22:55:27,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409613575bc060ce1614d1 HTTP/1.1" 502 150 2025-12-17 22:55:27,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409613575bc060ce1614d1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:27,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2a575bc060ce1600a0 2025-12-17 22:55:27,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2a575bc060ce1600a0 HTTP/1.1" 502 150 2025-12-17 22:55:27,654 - lava_callback - ERROR - Error processing callback for node 69408f2a575bc060ce1600a0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f2a575bc060ce1600a0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f2a575bc060ce1600a0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f2a575bc060ce1600a0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:27,656 - lava_callback - INFO - Starting async job submit for node 69408f37575bc060ce1600c8 2025-12-17 22:55:27,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:27,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f37575bc060ce1600c8 HTTP/1.1" 502 150 2025-12-17 22:55:27,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f37575bc060ce1600c8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:27,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f37575bc060ce1600c8 2025-12-17 22:55:27,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f37575bc060ce1600c8 HTTP/1.1" 502 150 2025-12-17 22:55:27,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f37575bc060ce1600c8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:28,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3b575bc060ce1600d0 HTTP/1.1" 502 150 2025-12-17 22:55:28,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f3b575bc060ce1600d0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:28,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f34575bc060ce1600ba 2025-12-17 22:55:28,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f34575bc060ce1600ba HTTP/1.1" 502 150 2025-12-17 22:55:28,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f34575bc060ce1600ba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:29,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409613575bc060ce1614d1 2025-12-17 22:55:29,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409613575bc060ce1614d1 HTTP/1.1" 502 150 2025-12-17 22:55:29,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409613575bc060ce1614d1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:29,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f3c575bc060ce1600d3 2025-12-17 22:55:29,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3c575bc060ce1600d3 HTTP/1.1" 502 150 2025-12-17 22:55:29,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f3c575bc060ce1600d3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:29,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f37575bc060ce1600c8 2025-12-17 22:55:29,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f37575bc060ce1600c8 HTTP/1.1" 502 150 2025-12-17 22:55:29,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f37575bc060ce1600c8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:30,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409533575bc060ce16126e 2025-12-17 22:55:30,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409533575bc060ce16126e HTTP/1.1" 502 150 2025-12-17 22:55:30,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409533575bc060ce16126e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:31,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2b575bc060ce1600a3 2025-12-17 22:55:31,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2b575bc060ce1600a3 HTTP/1.1" 502 150 2025-12-17 22:55:31,008 - lava_callback - ERROR - Error processing callback for node 69408f2b575bc060ce1600a3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f2b575bc060ce1600a3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f2b575bc060ce1600a3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f2b575bc060ce1600a3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:31,010 - lava_callback - INFO - Starting async job submit for node 69408f44575bc060ce1600db 2025-12-17 22:55:31,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:31,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f44575bc060ce1600db HTTP/1.1" 502 150 2025-12-17 22:55:31,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f44575bc060ce1600db'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:31,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f44575bc060ce1600db 2025-12-17 22:55:31,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f44575bc060ce1600db HTTP/1.1" 502 150 2025-12-17 22:55:31,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f44575bc060ce1600db'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:31,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2d575bc060ce1600a9 2025-12-17 22:55:31,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2d575bc060ce1600a9 HTTP/1.1" 502 150 2025-12-17 22:55:31,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f2d575bc060ce1600a9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:32,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f3b575bc060ce1600d0 2025-12-17 22:55:32,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3b575bc060ce1600d0 HTTP/1.1" 502 150 2025-12-17 22:55:32,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f3b575bc060ce1600d0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:33,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f44575bc060ce1600db 2025-12-17 22:55:33,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f44575bc060ce1600db HTTP/1.1" 502 150 2025-12-17 22:55:33,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f44575bc060ce1600db'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:33,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409613575bc060ce1614d1 2025-12-17 22:55:33,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409613575bc060ce1614d1 HTTP/1.1" 502 150 2025-12-17 22:55:33,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409613575bc060ce1614d1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:33,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f37575bc060ce1600c8 2025-12-17 22:55:33,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f37575bc060ce1600c8 HTTP/1.1" 502 150 2025-12-17 22:55:33,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f37575bc060ce1600c8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:35,221 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2d575bc060ce1600a6 2025-12-17 22:55:35,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2d575bc060ce1600a6 HTTP/1.1" 502 150 2025-12-17 22:55:35,222 - lava_callback - ERROR - Error processing callback for node 69408f2d575bc060ce1600a6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f2d575bc060ce1600a6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f2d575bc060ce1600a6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f2d575bc060ce1600a6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:35,224 - lava_callback - INFO - Starting async job submit for node 69408f46575bc060ce1600de 2025-12-17 22:55:35,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:35,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f46575bc060ce1600de HTTP/1.1" 502 150 2025-12-17 22:55:35,406 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f46575bc060ce1600de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:35,406 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f46575bc060ce1600de 2025-12-17 22:55:35,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f46575bc060ce1600de HTTP/1.1" 502 150 2025-12-17 22:55:35,407 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f46575bc060ce1600de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:35,626 - lava_callback - INFO - Received callback for node 694091d7575bc060ce16094f INFO: 172.21.0.1:34376 - "POST /node/694091d7575bc060ce16094f HTTP/1.0" 202 Accepted 2025-12-17 22:55:35,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2f575bc060ce1600ac 2025-12-17 22:55:35,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2f575bc060ce1600ac HTTP/1.1" 502 150 2025-12-17 22:55:35,686 - lava_callback - ERROR - Error processing callback for node 69408f2f575bc060ce1600ac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f2f575bc060ce1600ac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f2f575bc060ce1600ac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f2f575bc060ce1600ac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:35,689 - lava_callback - INFO - Starting async job submit for node 69408f49575bc060ce1600e8 2025-12-17 22:55:35,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:35,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f30575bc060ce1600af 2025-12-17 22:55:35,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f30575bc060ce1600af HTTP/1.1" 502 150 2025-12-17 22:55:35,706 - lava_callback - ERROR - Error processing callback for node 69408f30575bc060ce1600af: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f30575bc060ce1600af (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f30575bc060ce1600af (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f30575bc060ce1600af (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:35,709 - lava_callback - INFO - Starting async job submit for node 69409228575bc060ce160a0f 2025-12-17 22:55:35,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:35,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409228575bc060ce160a0f HTTP/1.1" 502 150 2025-12-17 22:55:35,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409228575bc060ce160a0f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:35,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409228575bc060ce160a0f 2025-12-17 22:55:35,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409228575bc060ce160a0f HTTP/1.1" 502 150 2025-12-17 22:55:35,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409228575bc060ce160a0f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:35,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f49575bc060ce1600e8 HTTP/1.1" 502 150 2025-12-17 22:55:35,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f49575bc060ce1600e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:35,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f49575bc060ce1600e8 2025-12-17 22:55:35,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f49575bc060ce1600e8 HTTP/1.1" 502 150 2025-12-17 22:55:35,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f49575bc060ce1600e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:35,866 - lava_callback - INFO - Received callback for node 69409150575bc060ce1607fe INFO: 172.21.0.1:34380 - "POST /node/69409150575bc060ce1607fe HTTP/1.0" 202 Accepted 2025-12-17 22:55:36,292 - lava_callback - INFO - Received callback for node 694091d9575bc060ce160952 INFO: 172.21.0.1:34396 - "POST /node/694091d9575bc060ce160952 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:34400 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:55:36,939 - lava_callback - INFO - Received callback for node 694091db575bc060ce160955 INFO: 172.21.0.1:34412 - "POST /node/694091db575bc060ce160955 HTTP/1.0" 202 Accepted 2025-12-17 22:55:36,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f34575bc060ce1600ba 2025-12-17 22:55:36,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f34575bc060ce1600ba HTTP/1.1" 502 150 2025-12-17 22:55:36,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f34575bc060ce1600ba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:37,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f32575bc060ce1600b2 2025-12-17 22:55:37,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f32575bc060ce1600b2 HTTP/1.1" 502 150 2025-12-17 22:55:37,002 - lava_callback - ERROR - Error processing callback for node 69408f32575bc060ce1600b2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f32575bc060ce1600b2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f32575bc060ce1600b2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f32575bc060ce1600b2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:37,004 - lava_callback - INFO - Starting async job submit for node 69408f42575bc060ce1600d8 2025-12-17 22:55:37,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:37,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f42575bc060ce1600d8 HTTP/1.1" 502 150 2025-12-17 22:55:37,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f42575bc060ce1600d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:37,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f42575bc060ce1600d8 2025-12-17 22:55:37,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f42575bc060ce1600d8 HTTP/1.1" 502 150 2025-12-17 22:55:37,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f42575bc060ce1600d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:37,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f44575bc060ce1600db 2025-12-17 22:55:37,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f44575bc060ce1600db HTTP/1.1" 502 150 2025-12-17 22:55:37,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f44575bc060ce1600db'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:37,407 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f46575bc060ce1600de 2025-12-17 22:55:37,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f46575bc060ce1600de HTTP/1.1" 502 150 2025-12-17 22:55:37,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f46575bc060ce1600de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:37,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f33575bc060ce1600b5 2025-12-17 22:55:37,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f33575bc060ce1600b5 HTTP/1.1" 502 150 2025-12-17 22:55:37,464 - lava_callback - ERROR - Error processing callback for node 69408f33575bc060ce1600b5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f33575bc060ce1600b5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f33575bc060ce1600b5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f33575bc060ce1600b5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:37,466 - lava_callback - INFO - Starting async job submit for node 69408f47575bc060ce1600e1 2025-12-17 22:55:37,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:37,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f47575bc060ce1600e1 HTTP/1.1" 502 150 2025-12-17 22:55:37,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f47575bc060ce1600e1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:37,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f47575bc060ce1600e1 2025-12-17 22:55:37,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f47575bc060ce1600e1 HTTP/1.1" 502 150 2025-12-17 22:55:37,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f47575bc060ce1600e1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:37,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f3c575bc060ce1600d3 2025-12-17 22:55:37,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3c575bc060ce1600d3 HTTP/1.1" 502 150 2025-12-17 22:55:37,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f3c575bc060ce1600d3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:37,635 - lava_callback - INFO - Received callback for node 694091dd575bc060ce160960 INFO: 172.21.0.1:34424 - "POST /node/694091dd575bc060ce160960 HTTP/1.0" 202 Accepted 2025-12-17 22:55:37,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409228575bc060ce160a0f 2025-12-17 22:55:37,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409228575bc060ce160a0f HTTP/1.1" 502 150 2025-12-17 22:55:37,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409228575bc060ce160a0f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:37,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f49575bc060ce1600e8 2025-12-17 22:55:37,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f49575bc060ce1600e8 HTTP/1.1" 502 150 2025-12-17 22:55:37,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f49575bc060ce1600e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:38,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409533575bc060ce16126e 2025-12-17 22:55:38,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409533575bc060ce16126e HTTP/1.1" 502 150 2025-12-17 22:55:38,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409533575bc060ce16126e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:38,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f36575bc060ce1600c3 2025-12-17 22:55:38,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f36575bc060ce1600c3 HTTP/1.1" 502 150 2025-12-17 22:55:38,835 - lava_callback - ERROR - Error processing callback for node 69408f36575bc060ce1600c3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f36575bc060ce1600c3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f36575bc060ce1600c3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f36575bc060ce1600c3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:38,837 - lava_callback - INFO - Starting async job submit for node 6942c65d1b61f1c6005ff1fb 2025-12-17 22:55:38,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:38,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942c65d1b61f1c6005ff1fb HTTP/1.1" 502 150 2025-12-17 22:55:38,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942c65d1b61f1c6005ff1fb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:38,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942c65d1b61f1c6005ff1fb 2025-12-17 22:55:38,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942c65d1b61f1c6005ff1fb HTTP/1.1" 502 150 2025-12-17 22:55:38,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942c65d1b61f1c6005ff1fb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:38,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f3a575bc060ce1600cd 2025-12-17 22:55:38,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3a575bc060ce1600cd HTTP/1.1" 502 150 2025-12-17 22:55:38,873 - lava_callback - ERROR - Error processing callback for node 69408f3a575bc060ce1600cd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f3a575bc060ce1600cd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f3a575bc060ce1600cd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f3a575bc060ce1600cd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:38,874 - lava_callback - INFO - Starting async job submit for node 69408f4a575bc060ce1600f6 2025-12-17 22:55:38,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:38,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4a575bc060ce1600f6 HTTP/1.1" 502 150 2025-12-17 22:55:38,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4a575bc060ce1600f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:38,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4a575bc060ce1600f6 2025-12-17 22:55:38,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4a575bc060ce1600f6 HTTP/1.1" 502 150 2025-12-17 22:55:38,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4a575bc060ce1600f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:39,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f42575bc060ce1600d8 2025-12-17 22:55:39,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f47575bc060ce1600e1 2025-12-17 22:55:39,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f47575bc060ce1600e1 HTTP/1.1" 502 150 2025-12-17 22:55:39,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f47575bc060ce1600e1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:39,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f25575bc060ce160091 HTTP/1.1" 502 150 2025-12-17 22:55:39,702 - lava_callback - ERROR - Error processing callback for node 69408f25575bc060ce160091: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f25575bc060ce160091 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f25575bc060ce160091 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f25575bc060ce160091 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:39,705 - lava_callback - INFO - Starting async job submit for node 69408f4d575bc060ce1600fd 2025-12-17 22:55:39,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:39,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4d575bc060ce1600fd HTTP/1.1" 502 150 2025-12-17 22:55:39,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4d575bc060ce1600fd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:39,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4d575bc060ce1600fd 2025-12-17 22:55:39,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4d575bc060ce1600fd HTTP/1.1" 502 150 2025-12-17 22:55:39,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4d575bc060ce1600fd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:40,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f3b575bc060ce1600d0 2025-12-17 22:55:40,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3b575bc060ce1600d0 HTTP/1.1" 502 150 2025-12-17 22:55:40,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f3b575bc060ce1600d0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:40,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942c65d1b61f1c6005ff1fb 2025-12-17 22:55:40,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942c65d1b61f1c6005ff1fb HTTP/1.1" 502 150 2025-12-17 22:55:40,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942c65d1b61f1c6005ff1fb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:40,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4a575bc060ce1600f6 2025-12-17 22:55:40,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4a575bc060ce1600f6 HTTP/1.1" 502 150 2025-12-17 22:55:40,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4a575bc060ce1600f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:41,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409613575bc060ce1614d1 2025-12-17 22:55:41,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409613575bc060ce1614d1 HTTP/1.1" 502 150 2025-12-17 22:55:41,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409613575bc060ce1614d1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:41,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f46575bc060ce1600de 2025-12-17 22:55:41,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f46575bc060ce1600de HTTP/1.1" 502 150 2025-12-17 22:55:41,410 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f46575bc060ce1600de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:41,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f37575bc060ce1600c8 2025-12-17 22:55:41,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f37575bc060ce1600c8 HTTP/1.1" 502 150 2025-12-17 22:55:41,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f37575bc060ce1600c8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:41,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4d575bc060ce1600fd 2025-12-17 22:55:41,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4d575bc060ce1600fd HTTP/1.1" 502 150 2025-12-17 22:55:41,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4d575bc060ce1600fd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:41,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409228575bc060ce160a0f 2025-12-17 22:55:41,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409228575bc060ce160a0f HTTP/1.1" 502 150 2025-12-17 22:55:41,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409228575bc060ce160a0f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:41,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f49575bc060ce1600e8 2025-12-17 22:55:41,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f49575bc060ce1600e8 HTTP/1.1" 502 150 2025-12-17 22:55:41,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f49575bc060ce1600e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:43,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f47575bc060ce1600e1 2025-12-17 22:55:43,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f47575bc060ce1600e1 HTTP/1.1" 502 150 2025-12-17 22:55:43,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f47575bc060ce1600e1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:44,384 - lava_callback - INFO - Received callback for node 6940955a575bc060ce1612df INFO: 172.21.0.1:46344 - "POST /node/6940955a575bc060ce1612df HTTP/1.0" 202 Accepted 2025-12-17 22:55:44,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942c65d1b61f1c6005ff1fb 2025-12-17 22:55:44,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942c65d1b61f1c6005ff1fb HTTP/1.1" 502 150 2025-12-17 22:55:44,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942c65d1b61f1c6005ff1fb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:44,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4a575bc060ce1600f6 2025-12-17 22:55:44,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4a575bc060ce1600f6 HTTP/1.1" 502 150 2025-12-17 22:55:44,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4a575bc060ce1600f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:45,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f44575bc060ce1600db 2025-12-17 22:55:45,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f44575bc060ce1600db HTTP/1.1" 502 150 2025-12-17 22:55:45,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f44575bc060ce1600db'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:45,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4d575bc060ce1600fd 2025-12-17 22:55:45,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4d575bc060ce1600fd HTTP/1.1" 502 150 2025-12-17 22:55:45,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4d575bc060ce1600fd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:46,034 - lava_callback - INFO - Received callback for node 694091df575bc060ce160969 INFO: 172.21.0.1:46352 - "POST /node/694091df575bc060ce160969 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:46362 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:55:46,694 - lava_callback - INFO - Received callback for node 694091e1575bc060ce16096c INFO: 172.21.0.1:46378 - "POST /node/694091e1575bc060ce16096c HTTP/1.0" 202 Accepted 2025-12-17 22:55:47,336 - lava_callback - INFO - Received callback for node 694091e3575bc060ce160971 INFO: 172.21.0.1:46384 - "POST /node/694091e3575bc060ce160971 HTTP/1.0" 202 Accepted 2025-12-17 22:55:47,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f2d575bc060ce1600a9 2025-12-17 22:55:47,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f2d575bc060ce1600a9 HTTP/1.1" 502 150 2025-12-17 22:55:47,704 - lava_callback - ERROR - Error processing callback for node 69408f2d575bc060ce1600a9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f2d575bc060ce1600a9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f2d575bc060ce1600a9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f2d575bc060ce1600a9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:47,706 - lava_callback - INFO - Starting async job submit for node 69408f4f575bc060ce160100 2025-12-17 22:55:47,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:47,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4f575bc060ce160100 HTTP/1.1" 502 150 2025-12-17 22:55:47,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4f575bc060ce160100'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:47,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4f575bc060ce160100 2025-12-17 22:55:47,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4f575bc060ce160100 HTTP/1.1" 502 150 2025-12-17 22:55:47,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4f575bc060ce160100'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:47,984 - lava_callback - INFO - Received callback for node 694091e5575bc060ce160974 INFO: 172.21.0.1:46396 - "POST /node/694091e5575bc060ce160974 HTTP/1.0" 202 Accepted 2025-12-17 22:55:48,631 - lava_callback - INFO - Received callback for node 694091e7575bc060ce160977 INFO: 172.21.0.1:46398 - "POST /node/694091e7575bc060ce160977 HTTP/1.0" 202 Accepted 2025-12-17 22:55:49,281 - lava_callback - INFO - Received callback for node 694091e9575bc060ce16097a INFO: 172.21.0.1:46412 - "POST /node/694091e9575bc060ce16097a HTTP/1.0" 202 Accepted 2025-12-17 22:55:49,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f46575bc060ce1600de 2025-12-17 22:55:49,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f46575bc060ce1600de HTTP/1.1" 502 150 2025-12-17 22:55:49,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f46575bc060ce1600de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:49,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4f575bc060ce160100 2025-12-17 22:55:49,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4f575bc060ce160100 HTTP/1.1" 502 150 2025-12-17 22:55:49,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4f575bc060ce160100'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:49,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409228575bc060ce160a0f 2025-12-17 22:55:49,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409228575bc060ce160a0f HTTP/1.1" 502 150 2025-12-17 22:55:49,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409228575bc060ce160a0f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:49,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f49575bc060ce1600e8 2025-12-17 22:55:49,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f49575bc060ce1600e8 HTTP/1.1" 502 150 2025-12-17 22:55:49,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f49575bc060ce1600e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:49,930 - lava_callback - INFO - Received callback for node 694091eb575bc060ce16097d INFO: 172.21.0.1:54256 - "POST /node/694091eb575bc060ce16097d HTTP/1.0" 202 Accepted 2025-12-17 22:55:50,574 - lava_callback - INFO - Received callback for node 694091ed575bc060ce160980 INFO: 172.21.0.1:54268 - "POST /node/694091ed575bc060ce160980 HTTP/1.0" 202 Accepted 2025-12-17 22:55:51,222 - lava_callback - INFO - Received callback for node 694091ef575bc060ce160983 INFO: 172.21.0.1:54274 - "POST /node/694091ef575bc060ce160983 HTTP/1.0" 202 Accepted 2025-12-17 22:55:51,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f47575bc060ce1600e1 2025-12-17 22:55:51,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f47575bc060ce1600e1 HTTP/1.1" 502 150 2025-12-17 22:55:51,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f47575bc060ce1600e1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:52,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942c65d1b61f1c6005ff1fb 2025-12-17 22:55:52,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942c65d1b61f1c6005ff1fb HTTP/1.1" 502 150 2025-12-17 22:55:52,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942c65d1b61f1c6005ff1fb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:52,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4a575bc060ce1600f6 2025-12-17 22:55:52,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4a575bc060ce1600f6 HTTP/1.1" 502 150 2025-12-17 22:55:52,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4a575bc060ce1600f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:52,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f34575bc060ce1600ba 2025-12-17 22:55:53,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f34575bc060ce1600ba HTTP/1.1" 502 150 2025-12-17 22:55:53,001 - lava_callback - ERROR - Error processing callback for node 69408f34575bc060ce1600ba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f34575bc060ce1600ba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f34575bc060ce1600ba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f34575bc060ce1600ba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:53,003 - lava_callback - INFO - Starting async job submit for node 69408f51575bc060ce160105 2025-12-17 22:55:53,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:53,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f51575bc060ce160105 HTTP/1.1" 502 150 2025-12-17 22:55:53,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f51575bc060ce160105'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:53,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f51575bc060ce160105 2025-12-17 22:55:53,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f51575bc060ce160105 HTTP/1.1" 502 150 2025-12-17 22:55:53,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f51575bc060ce160105'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:53,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f3c575bc060ce1600d3 2025-12-17 22:55:53,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3c575bc060ce1600d3 HTTP/1.1" 502 150 2025-12-17 22:55:53,617 - lava_callback - ERROR - Error processing callback for node 69408f3c575bc060ce1600d3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f3c575bc060ce1600d3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f3c575bc060ce1600d3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f3c575bc060ce1600d3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:53,620 - lava_callback - INFO - Starting async job submit for node 69408f54575bc060ce16010b 2025-12-17 22:55:53,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:53,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f54575bc060ce16010b HTTP/1.1" 502 150 2025-12-17 22:55:53,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f54575bc060ce16010b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:53,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f54575bc060ce16010b 2025-12-17 22:55:53,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f54575bc060ce16010b HTTP/1.1" 502 150 2025-12-17 22:55:53,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f54575bc060ce16010b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:53,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4d575bc060ce1600fd 2025-12-17 22:55:53,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4d575bc060ce1600fd HTTP/1.1" 502 150 2025-12-17 22:55:53,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4d575bc060ce1600fd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:53,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4f575bc060ce160100 2025-12-17 22:55:53,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4f575bc060ce160100 HTTP/1.1" 502 150 2025-12-17 22:55:53,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4f575bc060ce160100'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:54,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409533575bc060ce16126e 2025-12-17 22:55:54,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409533575bc060ce16126e HTTP/1.1" 502 150 2025-12-17 22:55:54,595 - lava_callback - ERROR - Error processing callback for node 69409533575bc060ce16126e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409533575bc060ce16126e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409533575bc060ce16126e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409533575bc060ce16126e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:54,597 - lava_callback - INFO - Starting async job submit for node 69408f56575bc060ce16010e 2025-12-17 22:55:54,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:54,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f56575bc060ce16010e HTTP/1.1" 502 150 2025-12-17 22:55:54,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f56575bc060ce16010e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:54,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f56575bc060ce16010e 2025-12-17 22:55:54,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f56575bc060ce16010e HTTP/1.1" 502 150 2025-12-17 22:55:54,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f56575bc060ce16010e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:54,854 - lava_callback - INFO - Received callback for node 6942e2f3e43d2b76e902392e INFO: 172.21.0.1:54288 - "POST /node/6942e2f3e43d2b76e902392e HTTP/1.0" 202 Accepted 2025-12-17 22:55:55,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f51575bc060ce160105 2025-12-17 22:55:55,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f51575bc060ce160105 HTTP/1.1" 502 150 2025-12-17 22:55:55,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f51575bc060ce160105'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:55,599 - lava_callback - INFO - Received callback for node 694091f1575bc060ce16098a INFO: 172.21.0.1:54302 - "POST /node/694091f1575bc060ce16098a HTTP/1.0" 202 Accepted 2025-12-17 22:55:55,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f54575bc060ce16010b 2025-12-17 22:55:55,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f54575bc060ce16010b HTTP/1.1" 502 150 2025-12-17 22:55:55,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f54575bc060ce16010b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:56,274 - lava_callback - INFO - Received callback for node 694091f4575bc060ce160995 INFO: 172.21.0.1:54304 - "POST /node/694091f4575bc060ce160995 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:54316 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:55:56,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f56575bc060ce16010e 2025-12-17 22:55:56,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f56575bc060ce16010e HTTP/1.1" 502 150 2025-12-17 22:55:56,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f56575bc060ce16010e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:56,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f3b575bc060ce1600d0 2025-12-17 22:55:56,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f3b575bc060ce1600d0 HTTP/1.1" 502 150 2025-12-17 22:55:56,697 - lava_callback - ERROR - Error processing callback for node 69408f3b575bc060ce1600d0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f3b575bc060ce1600d0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f3b575bc060ce1600d0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f3b575bc060ce1600d0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:56,700 - lava_callback - INFO - Starting async job submit for node 69408f58575bc060ce160111 2025-12-17 22:55:56,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:56,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f58575bc060ce160111 HTTP/1.1" 502 150 2025-12-17 22:55:56,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f58575bc060ce160111'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:56,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f58575bc060ce160111 2025-12-17 22:55:56,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f58575bc060ce160111 HTTP/1.1" 502 150 2025-12-17 22:55:56,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f58575bc060ce160111'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:56,932 - lava_callback - INFO - Received callback for node 694091f6575bc060ce16099c INFO: 172.21.0.1:54324 - "POST /node/694091f6575bc060ce16099c HTTP/1.0" 202 Accepted 2025-12-17 22:55:57,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409613575bc060ce1614d1 2025-12-17 22:55:57,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409613575bc060ce1614d1 HTTP/1.1" 502 150 2025-12-17 22:55:57,388 - lava_callback - ERROR - Error processing callback for node 69409613575bc060ce1614d1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409613575bc060ce1614d1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409613575bc060ce1614d1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409613575bc060ce1614d1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:57,391 - lava_callback - INFO - Starting async job submit for node 69408f5a575bc060ce16011b 2025-12-17 22:55:57,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:57,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5a575bc060ce16011b HTTP/1.1" 502 150 2025-12-17 22:55:57,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f5a575bc060ce16011b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:57,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f5a575bc060ce16011b 2025-12-17 22:55:57,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5a575bc060ce16011b HTTP/1.1" 502 150 2025-12-17 22:55:57,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f5a575bc060ce16011b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:57,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f37575bc060ce1600c8 2025-12-17 22:55:57,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f37575bc060ce1600c8 HTTP/1.1" 502 150 2025-12-17 22:55:57,676 - lava_callback - ERROR - Error processing callback for node 69408f37575bc060ce1600c8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f37575bc060ce1600c8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f37575bc060ce1600c8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f37575bc060ce1600c8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:55:57,678 - lava_callback - INFO - Starting async job submit for node 69408e9a575bc060ce15fe9d 2025-12-17 22:55:57,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:55:57,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9a575bc060ce15fe9d HTTP/1.1" 502 150 2025-12-17 22:55:57,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9a575bc060ce15fe9d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:57,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9a575bc060ce15fe9d 2025-12-17 22:55:57,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9a575bc060ce15fe9d HTTP/1.1" 502 150 2025-12-17 22:55:57,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9a575bc060ce15fe9d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:58,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f58575bc060ce160111 2025-12-17 22:55:58,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f58575bc060ce160111 HTTP/1.1" 502 150 2025-12-17 22:55:58,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f58575bc060ce160111'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:59,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f51575bc060ce160105 2025-12-17 22:55:59,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f51575bc060ce160105 HTTP/1.1" 502 150 2025-12-17 22:55:59,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f51575bc060ce160105'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:59,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f5a575bc060ce16011b 2025-12-17 22:55:59,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5a575bc060ce16011b HTTP/1.1" 502 150 2025-12-17 22:55:59,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f5a575bc060ce16011b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:59,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f54575bc060ce16010b 2025-12-17 22:55:59,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f54575bc060ce16010b HTTP/1.1" 502 150 2025-12-17 22:55:59,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f54575bc060ce16010b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:55:59,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9a575bc060ce15fe9d 2025-12-17 22:55:59,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9a575bc060ce15fe9d HTTP/1.1" 502 150 2025-12-17 22:55:59,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9a575bc060ce15fe9d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:00,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f56575bc060ce16010e 2025-12-17 22:56:00,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f56575bc060ce16010e HTTP/1.1" 502 150 2025-12-17 22:56:00,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f56575bc060ce16010e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:01,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f44575bc060ce1600db 2025-12-17 22:56:01,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f44575bc060ce1600db HTTP/1.1" 502 150 2025-12-17 22:56:01,024 - lava_callback - ERROR - Error processing callback for node 69408f44575bc060ce1600db: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f44575bc060ce1600db (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f44575bc060ce1600db (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f44575bc060ce1600db (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:01,026 - lava_callback - INFO - Starting async job submit for node 69408c27575bc060ce15f47d 2025-12-17 22:56:01,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:01,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c27575bc060ce15f47d HTTP/1.1" 502 150 2025-12-17 22:56:01,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c27575bc060ce15f47d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:01,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c27575bc060ce15f47d 2025-12-17 22:56:01,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c27575bc060ce15f47d HTTP/1.1" 502 150 2025-12-17 22:56:01,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c27575bc060ce15f47d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:01,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4f575bc060ce160100 2025-12-17 22:56:01,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4f575bc060ce160100 HTTP/1.1" 502 150 2025-12-17 22:56:01,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f4f575bc060ce160100'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:02,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f58575bc060ce160111 2025-12-17 22:56:02,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f58575bc060ce160111 HTTP/1.1" 502 150 2025-12-17 22:56:02,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f58575bc060ce160111'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34572 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:56:03,059 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c27575bc060ce15f47d 2025-12-17 22:56:03,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c27575bc060ce15f47d HTTP/1.1" 502 150 2025-12-17 22:56:03,060 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c27575bc060ce15f47d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:03,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f5a575bc060ce16011b 2025-12-17 22:56:03,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5a575bc060ce16011b HTTP/1.1" 502 150 2025-12-17 22:56:03,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f5a575bc060ce16011b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:03,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9a575bc060ce15fe9d 2025-12-17 22:56:03,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9a575bc060ce15fe9d HTTP/1.1" 502 150 2025-12-17 22:56:03,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9a575bc060ce15fe9d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:05,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f46575bc060ce1600de 2025-12-17 22:56:05,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409228575bc060ce160a0f 2025-12-17 22:56:05,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409228575bc060ce160a0f HTTP/1.1" 502 150 2025-12-17 22:56:05,814 - lava_callback - ERROR - Error processing callback for node 69409228575bc060ce160a0f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409228575bc060ce160a0f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409228575bc060ce160a0f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409228575bc060ce160a0f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:05,816 - lava_callback - INFO - Starting async job submit for node 69408f5d575bc060ce16012e 2025-12-17 22:56:05,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:05,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5d575bc060ce16012e HTTP/1.1" 502 150 2025-12-17 22:56:05,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f5d575bc060ce16012e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:05,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f5d575bc060ce16012e 2025-12-17 22:56:05,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5d575bc060ce16012e HTTP/1.1" 502 150 2025-12-17 22:56:05,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f5d575bc060ce16012e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:05,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f49575bc060ce1600e8 2025-12-17 22:56:05,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f49575bc060ce1600e8 HTTP/1.1" 502 150 2025-12-17 22:56:05,871 - lava_callback - ERROR - Error processing callback for node 69408f49575bc060ce1600e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f49575bc060ce1600e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f49575bc060ce1600e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f49575bc060ce1600e8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:05,873 - lava_callback - INFO - Starting async job submit for node 69408f5f575bc060ce160139 2025-12-17 22:56:05,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:05,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5f575bc060ce160139 HTTP/1.1" 502 150 2025-12-17 22:56:05,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f5f575bc060ce160139'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:05,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f5f575bc060ce160139 2025-12-17 22:56:05,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5f575bc060ce160139 HTTP/1.1" 502 150 2025-12-17 22:56:05,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f5f575bc060ce160139'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:06,025 - lava_callback - INFO - Received callback for node 694091f8575bc060ce16099f INFO: 172.21.0.1:34576 - "POST /node/694091f8575bc060ce16099f HTTP/1.0" 202 Accepted INFO: 172.21.0.1:34586 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:56:06,679 - lava_callback - INFO - Received callback for node 694091fa575bc060ce1609a2 INFO: 172.21.0.1:34602 - "POST /node/694091fa575bc060ce1609a2 HTTP/1.0" 202 Accepted 2025-12-17 22:56:07,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f51575bc060ce160105 2025-12-17 22:56:07,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f51575bc060ce160105 HTTP/1.1" 502 150 2025-12-17 22:56:07,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f51575bc060ce160105'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:07,060 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c27575bc060ce15f47d 2025-12-17 22:56:07,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c27575bc060ce15f47d HTTP/1.1" 502 150 2025-12-17 22:56:07,061 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c27575bc060ce15f47d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:07,317 - lava_callback - INFO - Received callback for node 694091fc575bc060ce1609a5 INFO: 172.21.0.1:34608 - "POST /node/694091fc575bc060ce1609a5 HTTP/1.0" 202 Accepted 2025-12-17 22:56:07,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f47575bc060ce1600e1 2025-12-17 22:56:07,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f47575bc060ce1600e1 HTTP/1.1" 502 150 2025-12-17 22:56:07,481 - lava_callback - ERROR - Error processing callback for node 69408f47575bc060ce1600e1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f47575bc060ce1600e1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f47575bc060ce1600e1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f47575bc060ce1600e1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:07,483 - lava_callback - INFO - Starting async job submit for node 69408f62575bc060ce160140 2025-12-17 22:56:07,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:07,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f62575bc060ce160140 HTTP/1.1" 502 150 2025-12-17 22:56:07,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f62575bc060ce160140'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:07,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f62575bc060ce160140 2025-12-17 22:56:07,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f62575bc060ce160140 HTTP/1.1" 502 150 2025-12-17 22:56:07,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f62575bc060ce160140'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:07,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f54575bc060ce16010b 2025-12-17 22:56:07,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f54575bc060ce16010b HTTP/1.1" 502 150 2025-12-17 22:56:07,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f54575bc060ce16010b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:07,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f5d575bc060ce16012e 2025-12-17 22:56:07,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5d575bc060ce16012e HTTP/1.1" 502 150 2025-12-17 22:56:07,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f5d575bc060ce16012e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:07,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f5f575bc060ce160139 2025-12-17 22:56:07,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5f575bc060ce160139 HTTP/1.1" 502 150 2025-12-17 22:56:07,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f5f575bc060ce160139'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:07,984 - lava_callback - INFO - Received callback for node 694091fe575bc060ce1609a8 INFO: 172.21.0.1:34618 - "POST /node/694091fe575bc060ce1609a8 HTTP/1.0" 202 Accepted 2025-12-17 22:56:08,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f56575bc060ce16010e 2025-12-17 22:56:08,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f56575bc060ce16010e HTTP/1.1" 502 150 2025-12-17 22:56:08,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f56575bc060ce16010e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:08,645 - lava_callback - INFO - Received callback for node 69409200575bc060ce1609ab INFO: 172.21.0.1:34634 - "POST /node/69409200575bc060ce1609ab HTTP/1.0" 202 Accepted 2025-12-17 22:56:08,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942c65d1b61f1c6005ff1fb 2025-12-17 22:56:08,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942c65d1b61f1c6005ff1fb HTTP/1.1" 502 150 2025-12-17 22:56:08,866 - lava_callback - ERROR - Error processing callback for node 6942c65d1b61f1c6005ff1fb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942c65d1b61f1c6005ff1fb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942c65d1b61f1c6005ff1fb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942c65d1b61f1c6005ff1fb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:08,869 - lava_callback - INFO - Starting async job submit for node 69408f64575bc060ce160143 2025-12-17 22:56:08,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:08,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f64575bc060ce160143 HTTP/1.1" 502 150 2025-12-17 22:56:08,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f64575bc060ce160143'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:08,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f64575bc060ce160143 2025-12-17 22:56:08,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f64575bc060ce160143 HTTP/1.1" 502 150 2025-12-17 22:56:08,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f64575bc060ce160143'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:08,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4a575bc060ce1600f6 2025-12-17 22:56:08,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4a575bc060ce1600f6 HTTP/1.1" 502 150 2025-12-17 22:56:08,887 - lava_callback - ERROR - Error processing callback for node 69408f4a575bc060ce1600f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f4a575bc060ce1600f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f4a575bc060ce1600f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f4a575bc060ce1600f6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:08,889 - lava_callback - INFO - Starting async job submit for node 69408f68575bc060ce160149 2025-12-17 22:56:08,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:08,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f68575bc060ce160149 HTTP/1.1" 502 150 2025-12-17 22:56:08,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f68575bc060ce160149'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:08,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f68575bc060ce160149 2025-12-17 22:56:08,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f68575bc060ce160149 HTTP/1.1" 502 150 2025-12-17 22:56:08,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f68575bc060ce160149'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:09,312 - lava_callback - INFO - Received callback for node 69409202575bc060ce1609ae INFO: 172.21.0.1:34650 - "POST /node/69409202575bc060ce1609ae HTTP/1.0" 202 Accepted 2025-12-17 22:56:09,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f62575bc060ce160140 2025-12-17 22:56:09,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f62575bc060ce160140 HTTP/1.1" 502 150 2025-12-17 22:56:09,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f62575bc060ce160140'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:09,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f42575bc060ce1600d8 HTTP/1.1" 502 150 2025-12-17 22:56:09,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f42575bc060ce1600d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:09,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4d575bc060ce1600fd 2025-12-17 22:56:09,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4d575bc060ce1600fd HTTP/1.1" 502 150 2025-12-17 22:56:09,725 - lava_callback - ERROR - Error processing callback for node 69408f4d575bc060ce1600fd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f4d575bc060ce1600fd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f4d575bc060ce1600fd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f4d575bc060ce1600fd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:09,727 - lava_callback - INFO - Starting async job submit for node 69408f6a575bc060ce16014c 2025-12-17 22:56:09,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:09,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6a575bc060ce16014c HTTP/1.1" 502 150 2025-12-17 22:56:09,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6a575bc060ce16014c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:09,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6a575bc060ce16014c 2025-12-17 22:56:09,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6a575bc060ce16014c HTTP/1.1" 502 150 2025-12-17 22:56:09,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6a575bc060ce16014c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:09,968 - lava_callback - INFO - Received callback for node 69409204575bc060ce1609b1 INFO: 172.21.0.1:43108 - "POST /node/69409204575bc060ce1609b1 HTTP/1.0" 202 Accepted 2025-12-17 22:56:10,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f58575bc060ce160111 2025-12-17 22:56:10,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f58575bc060ce160111 HTTP/1.1" 502 150 2025-12-17 22:56:10,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f58575bc060ce160111'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:10,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f64575bc060ce160143 2025-12-17 22:56:10,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f64575bc060ce160143 HTTP/1.1" 502 150 2025-12-17 22:56:10,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f64575bc060ce160143'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:10,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f68575bc060ce160149 2025-12-17 22:56:10,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f68575bc060ce160149 HTTP/1.1" 502 150 2025-12-17 22:56:10,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f68575bc060ce160149'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:11,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f5a575bc060ce16011b 2025-12-17 22:56:11,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5a575bc060ce16011b HTTP/1.1" 502 150 2025-12-17 22:56:11,405 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f5a575bc060ce16011b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:11,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6a575bc060ce16014c 2025-12-17 22:56:11,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6a575bc060ce16014c HTTP/1.1" 502 150 2025-12-17 22:56:11,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6a575bc060ce16014c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:11,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f5d575bc060ce16012e 2025-12-17 22:56:11,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5d575bc060ce16012e HTTP/1.1" 502 150 2025-12-17 22:56:11,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f5d575bc060ce16012e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:11,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f5f575bc060ce160139 2025-12-17 22:56:11,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5f575bc060ce160139 HTTP/1.1" 502 150 2025-12-17 22:56:11,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f5f575bc060ce160139'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:11,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9a575bc060ce15fe9d 2025-12-17 22:56:11,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9a575bc060ce15fe9d HTTP/1.1" 502 150 2025-12-17 22:56:11,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9a575bc060ce15fe9d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:13,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f62575bc060ce160140 2025-12-17 22:56:13,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f62575bc060ce160140 HTTP/1.1" 502 150 2025-12-17 22:56:13,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f62575bc060ce160140'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:13,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f42575bc060ce1600d8 2025-12-17 22:56:13,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f42575bc060ce1600d8 HTTP/1.1" 502 150 2025-12-17 22:56:13,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f42575bc060ce1600d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:14,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f64575bc060ce160143 2025-12-17 22:56:14,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f64575bc060ce160143 HTTP/1.1" 502 150 2025-12-17 22:56:14,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f64575bc060ce160143'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:14,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f68575bc060ce160149 2025-12-17 22:56:14,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f68575bc060ce160149 HTTP/1.1" 502 150 2025-12-17 22:56:14,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f68575bc060ce160149'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:15,061 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c27575bc060ce15f47d 2025-12-17 22:56:15,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c27575bc060ce15f47d HTTP/1.1" 502 150 2025-12-17 22:56:15,062 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c27575bc060ce15f47d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43118 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:56:15,623 - lava_callback - INFO - Received callback for node 6940920c575bc060ce1609bf INFO: 172.21.0.1:43130 - "POST /node/6940920c575bc060ce1609bf HTTP/1.0" 202 Accepted 2025-12-17 22:56:15,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6a575bc060ce16014c 2025-12-17 22:56:15,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6a575bc060ce16014c HTTP/1.1" 502 150 2025-12-17 22:56:15,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6a575bc060ce16014c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43138 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:56:17,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f4f575bc060ce160100 2025-12-17 22:56:19,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f5d575bc060ce16012e 2025-12-17 22:56:19,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5d575bc060ce16012e HTTP/1.1" 502 150 2025-12-17 22:56:19,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f5d575bc060ce16012e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:19,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f5f575bc060ce160139 2025-12-17 22:56:19,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5f575bc060ce160139 HTTP/1.1" 502 150 2025-12-17 22:56:19,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f5f575bc060ce160139'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:21,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f62575bc060ce160140 2025-12-17 22:56:21,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f62575bc060ce160140 HTTP/1.1" 502 150 2025-12-17 22:56:21,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f62575bc060ce160140'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:21,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f42575bc060ce1600d8 2025-12-17 22:56:21,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f42575bc060ce1600d8 HTTP/1.1" 502 150 2025-12-17 22:56:21,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f42575bc060ce1600d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46530 - "GET /node/69402588575bc060ce1540ed%5Cn HTTP/1.0" 405 Method Not Allowed 2025-12-17 22:56:22,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f64575bc060ce160143 2025-12-17 22:56:22,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f64575bc060ce160143 HTTP/1.1" 502 150 2025-12-17 22:56:22,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f64575bc060ce160143'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:22,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f68575bc060ce160149 2025-12-17 22:56:22,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f68575bc060ce160149 HTTP/1.1" 502 150 2025-12-17 22:56:22,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f68575bc060ce160149'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:23,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f51575bc060ce160105 2025-12-17 22:56:23,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f51575bc060ce160105 HTTP/1.1" 502 150 2025-12-17 22:56:23,016 - lava_callback - ERROR - Error processing callback for node 69408f51575bc060ce160105: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f51575bc060ce160105 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f51575bc060ce160105 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f51575bc060ce160105 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:23,018 - lava_callback - INFO - Starting async job submit for node 69408f6c575bc060ce16014f 2025-12-17 22:56:23,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:23,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6c575bc060ce16014f HTTP/1.1" 502 150 2025-12-17 22:56:23,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6c575bc060ce16014f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:23,028 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6c575bc060ce16014f 2025-12-17 22:56:23,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6c575bc060ce16014f HTTP/1.1" 502 150 2025-12-17 22:56:23,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6c575bc060ce16014f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:23,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f54575bc060ce16010b 2025-12-17 22:56:23,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f54575bc060ce16010b HTTP/1.1" 502 150 2025-12-17 22:56:23,657 - lava_callback - ERROR - Error processing callback for node 69408f54575bc060ce16010b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f54575bc060ce16010b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f54575bc060ce16010b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f54575bc060ce16010b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:23,659 - lava_callback - INFO - Starting async job submit for node 69408f6e575bc060ce160152 2025-12-17 22:56:23,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:23,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6e575bc060ce160152 HTTP/1.1" 502 150 2025-12-17 22:56:23,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6e575bc060ce160152'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:23,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6e575bc060ce160152 2025-12-17 22:56:23,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6e575bc060ce160152 HTTP/1.1" 502 150 2025-12-17 22:56:23,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6e575bc060ce160152'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:23,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6a575bc060ce16014c 2025-12-17 22:56:23,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6a575bc060ce16014c HTTP/1.1" 502 150 2025-12-17 22:56:23,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6a575bc060ce16014c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:24,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f56575bc060ce16010e 2025-12-17 22:56:24,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f56575bc060ce16010e HTTP/1.1" 502 150 2025-12-17 22:56:24,610 - lava_callback - ERROR - Error processing callback for node 69408f56575bc060ce16010e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f56575bc060ce16010e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f56575bc060ce16010e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f56575bc060ce16010e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:24,612 - lava_callback - INFO - Starting async job submit for node 69408f6f575bc060ce160155 2025-12-17 22:56:24,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:24,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6f575bc060ce160155 HTTP/1.1" 502 150 2025-12-17 22:56:24,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6f575bc060ce160155'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:24,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6f575bc060ce160155 2025-12-17 22:56:24,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6f575bc060ce160155 HTTP/1.1" 502 150 2025-12-17 22:56:24,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6f575bc060ce160155'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:25,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6c575bc060ce16014f 2025-12-17 22:56:25,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6c575bc060ce16014f HTTP/1.1" 502 150 2025-12-17 22:56:25,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6c575bc060ce16014f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:25,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6e575bc060ce160152 2025-12-17 22:56:25,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6e575bc060ce160152 HTTP/1.1" 502 150 2025-12-17 22:56:25,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6e575bc060ce160152'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:26,031 - lava_callback - INFO - Received callback for node 69409206575bc060ce1609b8 INFO: 172.21.0.1:46542 - "POST /node/69409206575bc060ce1609b8 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:46558 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:56:26,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6f575bc060ce160155 2025-12-17 22:56:26,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6f575bc060ce160155 HTTP/1.1" 502 150 2025-12-17 22:56:26,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6f575bc060ce160155'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:26,689 - lava_callback - INFO - Received callback for node 6940920f575bc060ce1609c2 INFO: 172.21.0.1:46568 - "POST /node/6940920f575bc060ce1609c2 HTTP/1.0" 202 Accepted 2025-12-17 22:56:26,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f58575bc060ce160111 2025-12-17 22:56:26,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f58575bc060ce160111 HTTP/1.1" 502 150 2025-12-17 22:56:26,718 - lava_callback - ERROR - Error processing callback for node 69408f58575bc060ce160111: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f58575bc060ce160111 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f58575bc060ce160111 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f58575bc060ce160111 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:26,720 - lava_callback - INFO - Starting async job submit for node 694094cd575bc060ce161115 2025-12-17 22:56:26,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:26,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cd575bc060ce161115 HTTP/1.1" 502 150 2025-12-17 22:56:26,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094cd575bc060ce161115'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:26,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094cd575bc060ce161115 2025-12-17 22:56:26,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cd575bc060ce161115 HTTP/1.1" 502 150 2025-12-17 22:56:26,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094cd575bc060ce161115'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:27,351 - lava_callback - INFO - Received callback for node 69409213575bc060ce1609c8 INFO: 172.21.0.1:46578 - "POST /node/69409213575bc060ce1609c8 HTTP/1.0" 202 Accepted 2025-12-17 22:56:27,405 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f5a575bc060ce16011b 2025-12-17 22:56:27,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5a575bc060ce16011b HTTP/1.1" 502 150 2025-12-17 22:56:27,406 - lava_callback - ERROR - Error processing callback for node 69408f5a575bc060ce16011b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f5a575bc060ce16011b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f5a575bc060ce16011b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f5a575bc060ce16011b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:27,409 - lava_callback - INFO - Starting async job submit for node 69408f72575bc060ce160162 2025-12-17 22:56:27,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:27,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f72575bc060ce160162 HTTP/1.1" 502 150 2025-12-17 22:56:27,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f72575bc060ce160162'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:27,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f72575bc060ce160162 2025-12-17 22:56:27,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f72575bc060ce160162 HTTP/1.1" 502 150 2025-12-17 22:56:27,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f72575bc060ce160162'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:27,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9a575bc060ce15fe9d 2025-12-17 22:56:27,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9a575bc060ce15fe9d HTTP/1.1" 502 150 2025-12-17 22:56:27,908 - lava_callback - ERROR - Error processing callback for node 69408e9a575bc060ce15fe9d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e9a575bc060ce15fe9d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e9a575bc060ce15fe9d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e9a575bc060ce15fe9d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:27,910 - lava_callback - INFO - Starting async job submit for node 69408f74575bc060ce160169 2025-12-17 22:56:27,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:27,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f74575bc060ce160169 HTTP/1.1" 502 150 2025-12-17 22:56:27,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f74575bc060ce160169'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:27,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f74575bc060ce160169 2025-12-17 22:56:27,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f74575bc060ce160169 HTTP/1.1" 502 150 2025-12-17 22:56:27,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f74575bc060ce160169'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:27,999 - lava_callback - INFO - Received callback for node 69409216575bc060ce1609cb INFO: 172.21.0.1:46582 - "POST /node/69409216575bc060ce1609cb HTTP/1.0" 202 Accepted 2025-12-17 22:56:28,661 - lava_callback - INFO - Received callback for node 69409217575bc060ce1609ce INFO: 172.21.0.1:46586 - "POST /node/69409217575bc060ce1609ce HTTP/1.0" 202 Accepted 2025-12-17 22:56:28,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094cd575bc060ce161115 2025-12-17 22:56:28,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cd575bc060ce161115 HTTP/1.1" 502 150 2025-12-17 22:56:28,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094cd575bc060ce161115'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:29,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6c575bc060ce16014f 2025-12-17 22:56:29,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6c575bc060ce16014f HTTP/1.1" 502 150 2025-12-17 22:56:29,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6c575bc060ce16014f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:29,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f72575bc060ce160162 2025-12-17 22:56:29,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f72575bc060ce160162 HTTP/1.1" 502 150 2025-12-17 22:56:29,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f72575bc060ce160162'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:29,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6e575bc060ce160152 2025-12-17 22:56:29,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6e575bc060ce160152 HTTP/1.1" 502 150 2025-12-17 22:56:29,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6e575bc060ce160152'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:29,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f74575bc060ce160169 2025-12-17 22:56:29,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f74575bc060ce160169 HTTP/1.1" 502 150 2025-12-17 22:56:29,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f74575bc060ce160169'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:30,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6f575bc060ce160155 2025-12-17 22:56:30,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6f575bc060ce160155 HTTP/1.1" 502 150 2025-12-17 22:56:30,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6f575bc060ce160155'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:31,063 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c27575bc060ce15f47d 2025-12-17 22:56:32,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094cd575bc060ce161115 2025-12-17 22:56:32,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cd575bc060ce161115 HTTP/1.1" 502 150 2025-12-17 22:56:32,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094cd575bc060ce161115'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:33,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f4f575bc060ce160100 HTTP/1.1" 502 150 2025-12-17 22:56:33,200 - lava_callback - ERROR - Error processing callback for node 69408f4f575bc060ce160100: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f4f575bc060ce160100 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f4f575bc060ce160100 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f4f575bc060ce160100 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:33,202 - lava_callback - INFO - Starting async job submit for node 69408f76575bc060ce16016c 2025-12-17 22:56:33,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:33,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f76575bc060ce16016c HTTP/1.1" 502 150 2025-12-17 22:56:33,212 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f76575bc060ce16016c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:33,212 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f76575bc060ce16016c 2025-12-17 22:56:33,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f76575bc060ce16016c HTTP/1.1" 502 150 2025-12-17 22:56:33,213 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f76575bc060ce16016c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:33,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f72575bc060ce160162 2025-12-17 22:56:33,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f72575bc060ce160162 HTTP/1.1" 502 150 2025-12-17 22:56:33,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f72575bc060ce160162'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:33,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f74575bc060ce160169 2025-12-17 22:56:33,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f74575bc060ce160169 HTTP/1.1" 502 150 2025-12-17 22:56:33,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f74575bc060ce160169'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:35,213 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f76575bc060ce16016c 2025-12-17 22:56:35,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f76575bc060ce16016c HTTP/1.1" 502 150 2025-12-17 22:56:35,214 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f76575bc060ce16016c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:35,665 - lava_callback - INFO - Received callback for node 6940921c575bc060ce1609e6 INFO: 172.21.0.1:60618 - "POST /node/6940921c575bc060ce1609e6 HTTP/1.0" 202 Accepted 2025-12-17 22:56:35,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f5d575bc060ce16012e 2025-12-17 22:56:35,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5d575bc060ce16012e HTTP/1.1" 502 150 2025-12-17 22:56:35,851 - lava_callback - ERROR - Error processing callback for node 69408f5d575bc060ce16012e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f5d575bc060ce16012e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f5d575bc060ce16012e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f5d575bc060ce16012e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:35,852 - lava_callback - INFO - Starting async job submit for node 69408f78575bc060ce16016f 2025-12-17 22:56:35,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:35,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f78575bc060ce16016f HTTP/1.1" 502 150 2025-12-17 22:56:35,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f78575bc060ce16016f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:35,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f78575bc060ce16016f 2025-12-17 22:56:35,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f78575bc060ce16016f HTTP/1.1" 502 150 2025-12-17 22:56:35,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f78575bc060ce16016f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:35,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f5f575bc060ce160139 2025-12-17 22:56:35,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f5f575bc060ce160139 HTTP/1.1" 502 150 2025-12-17 22:56:35,885 - lava_callback - ERROR - Error processing callback for node 69408f5f575bc060ce160139: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f5f575bc060ce160139 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f5f575bc060ce160139 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f5f575bc060ce160139 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:35,887 - lava_callback - INFO - Starting async job submit for node 69408f79575bc060ce160172 2025-12-17 22:56:35,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:35,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f79575bc060ce160172 HTTP/1.1" 502 150 2025-12-17 22:56:35,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f79575bc060ce160172'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:35,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f79575bc060ce160172 2025-12-17 22:56:35,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f79575bc060ce160172 HTTP/1.1" 502 150 2025-12-17 22:56:35,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f79575bc060ce160172'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:36,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f46575bc060ce1600de HTTP/1.1" 502 150 2025-12-17 22:56:36,016 - lava_callback - ERROR - Error processing callback for node 69408f46575bc060ce1600de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f46575bc060ce1600de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f46575bc060ce1600de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f46575bc060ce1600de (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:36,018 - lava_callback - INFO - Starting async job submit for node 69408f7b575bc060ce160175 2025-12-17 22:56:36,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:36,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7b575bc060ce160175 HTTP/1.1" 502 150 2025-12-17 22:56:36,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f7b575bc060ce160175'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:36,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f7b575bc060ce160175 2025-12-17 22:56:36,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7b575bc060ce160175 HTTP/1.1" 502 150 2025-12-17 22:56:36,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f7b575bc060ce160175'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:36,344 - lava_callback - INFO - Received callback for node 69409212575bc060ce1609c5 INFO: 172.21.0.1:60624 - "POST /node/69409212575bc060ce1609c5 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:60632 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:56:37,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6c575bc060ce16014f 2025-12-17 22:56:37,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6c575bc060ce16014f HTTP/1.1" 502 150 2025-12-17 22:56:37,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6c575bc060ce16014f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:37,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f62575bc060ce160140 2025-12-17 22:56:37,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f62575bc060ce160140 HTTP/1.1" 502 150 2025-12-17 22:56:37,497 - lava_callback - ERROR - Error processing callback for node 69408f62575bc060ce160140: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f62575bc060ce160140 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f62575bc060ce160140 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f62575bc060ce160140 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:37,499 - lava_callback - INFO - Starting async job submit for node 69408f7d575bc060ce160178 2025-12-17 22:56:37,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:37,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7d575bc060ce160178 HTTP/1.1" 502 150 2025-12-17 22:56:37,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f7d575bc060ce160178'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:37,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f7d575bc060ce160178 2025-12-17 22:56:37,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7d575bc060ce160178 HTTP/1.1" 502 150 2025-12-17 22:56:37,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f7d575bc060ce160178'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:37,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f42575bc060ce1600d8 2025-12-17 22:56:37,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f42575bc060ce1600d8 HTTP/1.1" 502 150 2025-12-17 22:56:37,652 - lava_callback - ERROR - Error processing callback for node 69408f42575bc060ce1600d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f42575bc060ce1600d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f42575bc060ce1600d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f42575bc060ce1600d8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:37,654 - lava_callback - INFO - Starting async job submit for node 69408f7f575bc060ce16017b 2025-12-17 22:56:37,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:37,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7f575bc060ce16017b HTTP/1.1" 502 150 2025-12-17 22:56:37,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f7f575bc060ce16017b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:37,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f7f575bc060ce16017b 2025-12-17 22:56:37,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7f575bc060ce16017b HTTP/1.1" 502 150 2025-12-17 22:56:37,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f7f575bc060ce16017b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:37,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6e575bc060ce160152 2025-12-17 22:56:37,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6e575bc060ce160152 HTTP/1.1" 502 150 2025-12-17 22:56:37,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6e575bc060ce160152'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:37,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f78575bc060ce16016f 2025-12-17 22:56:37,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f78575bc060ce16016f HTTP/1.1" 502 150 2025-12-17 22:56:37,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f78575bc060ce16016f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:37,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f79575bc060ce160172 2025-12-17 22:56:37,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f79575bc060ce160172 HTTP/1.1" 502 150 2025-12-17 22:56:37,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f79575bc060ce160172'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:38,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f7b575bc060ce160175 2025-12-17 22:56:38,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7b575bc060ce160175 HTTP/1.1" 502 150 2025-12-17 22:56:38,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f7b575bc060ce160175'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:38,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6f575bc060ce160155 2025-12-17 22:56:38,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6f575bc060ce160155 HTTP/1.1" 502 150 2025-12-17 22:56:38,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f6f575bc060ce160155'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:38,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f64575bc060ce160143 2025-12-17 22:56:38,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f64575bc060ce160143 HTTP/1.1" 502 150 2025-12-17 22:56:38,882 - lava_callback - ERROR - Error processing callback for node 69408f64575bc060ce160143: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f64575bc060ce160143 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f64575bc060ce160143 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f64575bc060ce160143 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:38,884 - lava_callback - INFO - Starting async job submit for node 69408f81575bc060ce16017e 2025-12-17 22:56:38,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:38,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f81575bc060ce16017e HTTP/1.1" 502 150 2025-12-17 22:56:38,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f81575bc060ce16017e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:38,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f81575bc060ce16017e 2025-12-17 22:56:38,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f81575bc060ce16017e HTTP/1.1" 502 150 2025-12-17 22:56:38,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f81575bc060ce16017e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:38,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f68575bc060ce160149 2025-12-17 22:56:38,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f68575bc060ce160149 HTTP/1.1" 502 150 2025-12-17 22:56:38,900 - lava_callback - ERROR - Error processing callback for node 69408f68575bc060ce160149: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f68575bc060ce160149 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f68575bc060ce160149 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f68575bc060ce160149 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:38,902 - lava_callback - INFO - Starting async job submit for node 69408f83575bc060ce160181 2025-12-17 22:56:38,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:38,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f83575bc060ce160181 HTTP/1.1" 502 150 2025-12-17 22:56:38,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f83575bc060ce160181'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:38,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f83575bc060ce160181 2025-12-17 22:56:38,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f83575bc060ce160181 HTTP/1.1" 502 150 2025-12-17 22:56:38,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f83575bc060ce160181'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:39,214 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f76575bc060ce16016c 2025-12-17 22:56:39,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f76575bc060ce16016c HTTP/1.1" 502 150 2025-12-17 22:56:39,216 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f76575bc060ce16016c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:39,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f7d575bc060ce160178 2025-12-17 22:56:39,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7d575bc060ce160178 HTTP/1.1" 502 150 2025-12-17 22:56:39,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f7d575bc060ce160178'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:39,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f7f575bc060ce16017b 2025-12-17 22:56:39,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7f575bc060ce16017b HTTP/1.1" 502 150 2025-12-17 22:56:39,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f7f575bc060ce16017b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:39,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6a575bc060ce16014c 2025-12-17 22:56:39,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6a575bc060ce16014c HTTP/1.1" 502 150 2025-12-17 22:56:39,740 - lava_callback - ERROR - Error processing callback for node 69408f6a575bc060ce16014c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f6a575bc060ce16014c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f6a575bc060ce16014c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f6a575bc060ce16014c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:39,742 - lava_callback - INFO - Starting async job submit for node 69409618575bc060ce1614eb 2025-12-17 22:56:39,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:39,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409618575bc060ce1614eb HTTP/1.1" 502 150 2025-12-17 22:56:39,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409618575bc060ce1614eb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:39,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409618575bc060ce1614eb 2025-12-17 22:56:39,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409618575bc060ce1614eb HTTP/1.1" 502 150 2025-12-17 22:56:39,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409618575bc060ce1614eb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:39,990 - lava_callback - INFO - Received callback for node 6942e382e43d2b76e9023a74 INFO: 172.21.0.1:45584 - "POST /node/6942e382e43d2b76e9023a74 HTTP/1.0" 202 Accepted 2025-12-17 22:56:40,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094cd575bc060ce161115 2025-12-17 22:56:40,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cd575bc060ce161115 HTTP/1.1" 502 150 2025-12-17 22:56:40,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094cd575bc060ce161115'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:40,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f81575bc060ce16017e 2025-12-17 22:56:40,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f81575bc060ce16017e HTTP/1.1" 502 150 2025-12-17 22:56:40,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f81575bc060ce16017e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:40,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f83575bc060ce160181 2025-12-17 22:56:40,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f83575bc060ce160181 HTTP/1.1" 502 150 2025-12-17 22:56:40,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f83575bc060ce160181'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:41,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f72575bc060ce160162 2025-12-17 22:56:41,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f72575bc060ce160162 HTTP/1.1" 502 150 2025-12-17 22:56:41,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f72575bc060ce160162'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:41,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409618575bc060ce1614eb 2025-12-17 22:56:41,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409618575bc060ce1614eb HTTP/1.1" 502 150 2025-12-17 22:56:41,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409618575bc060ce1614eb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:41,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f78575bc060ce16016f 2025-12-17 22:56:41,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f78575bc060ce16016f HTTP/1.1" 502 150 2025-12-17 22:56:41,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f78575bc060ce16016f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:41,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f74575bc060ce160169 2025-12-17 22:56:41,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f74575bc060ce160169 HTTP/1.1" 502 150 2025-12-17 22:56:41,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f74575bc060ce160169'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:41,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f79575bc060ce160172 2025-12-17 22:56:41,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f79575bc060ce160172 HTTP/1.1" 502 150 2025-12-17 22:56:41,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f79575bc060ce160172'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:42,028 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f7b575bc060ce160175 2025-12-17 22:56:42,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7b575bc060ce160175 HTTP/1.1" 502 150 2025-12-17 22:56:42,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f7b575bc060ce160175'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:43,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f7d575bc060ce160178 2025-12-17 22:56:43,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7d575bc060ce160178 HTTP/1.1" 502 150 2025-12-17 22:56:43,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f7d575bc060ce160178'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:43,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f7f575bc060ce16017b 2025-12-17 22:56:43,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7f575bc060ce16017b HTTP/1.1" 502 150 2025-12-17 22:56:43,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f7f575bc060ce16017b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:44,187 - lava_callback - INFO - Received callback for node 6940955b575bc060ce1612e2 INFO: 172.21.0.1:45590 - "POST /node/6940955b575bc060ce1612e2 HTTP/1.0" 202 Accepted 2025-12-17 22:56:44,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f81575bc060ce16017e 2025-12-17 22:56:44,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f81575bc060ce16017e HTTP/1.1" 502 150 2025-12-17 22:56:44,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f81575bc060ce16017e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:44,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f83575bc060ce160181 2025-12-17 22:56:44,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f83575bc060ce160181 HTTP/1.1" 502 150 2025-12-17 22:56:44,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f83575bc060ce160181'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:45,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409618575bc060ce1614eb 2025-12-17 22:56:45,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409618575bc060ce1614eb HTTP/1.1" 502 150 2025-12-17 22:56:45,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409618575bc060ce1614eb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:46,051 - lava_callback - INFO - Received callback for node 69409219575bc060ce1609d6 INFO: 172.21.0.1:45606 - "POST /node/69409219575bc060ce1609d6 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:45612 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:56:46,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c27575bc060ce15f47d HTTP/1.1" 502 150 2025-12-17 22:56:46,512 - lava_callback - ERROR - Error processing callback for node 69408c27575bc060ce15f47d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c27575bc060ce15f47d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c27575bc060ce15f47d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c27575bc060ce15f47d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:46,515 - lava_callback - INFO - Starting async job submit for node 69408f86575bc060ce160192 2025-12-17 22:56:46,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:46,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f86575bc060ce160192 HTTP/1.1" 502 150 2025-12-17 22:56:46,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f86575bc060ce160192'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:46,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f86575bc060ce160192 2025-12-17 22:56:46,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f86575bc060ce160192 HTTP/1.1" 502 150 2025-12-17 22:56:46,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f86575bc060ce160192'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:46,685 - lava_callback - INFO - Received callback for node 6940921e575bc060ce1609ef INFO: 172.21.0.1:45624 - "POST /node/6940921e575bc060ce1609ef HTTP/1.0" 202 Accepted 2025-12-17 22:56:47,213 - lava_callback - INFO - Received callback for node 6940921a575bc060ce1609dd INFO: 172.21.0.1:45638 - "POST /node/6940921a575bc060ce1609dd HTTP/1.0" 202 Accepted 2025-12-17 22:56:47,218 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f76575bc060ce16016c 2025-12-17 22:56:47,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f76575bc060ce16016c HTTP/1.1" 502 150 2025-12-17 22:56:47,219 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f76575bc060ce16016c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:47,345 - lava_callback - INFO - Received callback for node 6940921f575bc060ce1609f6 INFO: 172.21.0.1:45648 - "POST /node/6940921f575bc060ce1609f6 HTTP/1.0" 202 Accepted 2025-12-17 22:56:47,974 - lava_callback - INFO - Received callback for node 69409220575bc060ce1609fb INFO: 172.21.0.1:45662 - "POST /node/69409220575bc060ce1609fb HTTP/1.0" 202 Accepted 2025-12-17 22:56:48,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f86575bc060ce160192 2025-12-17 22:56:48,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f86575bc060ce160192 HTTP/1.1" 502 150 2025-12-17 22:56:48,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f86575bc060ce160192'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:48,615 - lava_callback - INFO - Received callback for node 69409222575bc060ce160a00 INFO: 172.21.0.1:45672 - "POST /node/69409222575bc060ce160a00 HTTP/1.0" 202 Accepted 2025-12-17 22:56:49,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f78575bc060ce16016f 2025-12-17 22:56:49,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f78575bc060ce16016f HTTP/1.1" 502 150 2025-12-17 22:56:49,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f78575bc060ce16016f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:49,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f79575bc060ce160172 2025-12-17 22:56:49,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f79575bc060ce160172 HTTP/1.1" 502 150 2025-12-17 22:56:49,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f79575bc060ce160172'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:50,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f7b575bc060ce160175 2025-12-17 22:56:50,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7b575bc060ce160175 HTTP/1.1" 502 150 2025-12-17 22:56:50,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f7b575bc060ce160175'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:50,746 - lava_callback - INFO - Received callback for node 69409223575bc060ce160a03 INFO: 172.21.0.1:41890 - "POST /node/69409223575bc060ce160a03 HTTP/1.0" 202 Accepted 2025-12-17 22:56:51,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f7d575bc060ce160178 2025-12-17 22:56:51,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7d575bc060ce160178 HTTP/1.1" 502 150 2025-12-17 22:56:51,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f7d575bc060ce160178'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:51,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f7f575bc060ce16017b 2025-12-17 22:56:51,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7f575bc060ce16017b HTTP/1.1" 502 150 2025-12-17 22:56:51,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f7f575bc060ce16017b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:52,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f86575bc060ce160192 2025-12-17 22:56:52,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f86575bc060ce160192 HTTP/1.1" 502 150 2025-12-17 22:56:52,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f86575bc060ce160192'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:52,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f81575bc060ce16017e 2025-12-17 22:56:52,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f81575bc060ce16017e HTTP/1.1" 502 150 2025-12-17 22:56:52,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f81575bc060ce16017e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:52,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f83575bc060ce160181 2025-12-17 22:56:52,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f83575bc060ce160181 HTTP/1.1" 502 150 2025-12-17 22:56:52,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f83575bc060ce160181'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:53,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6c575bc060ce16014f 2025-12-17 22:56:53,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6c575bc060ce16014f HTTP/1.1" 502 150 2025-12-17 22:56:53,035 - lava_callback - ERROR - Error processing callback for node 69408f6c575bc060ce16014f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f6c575bc060ce16014f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f6c575bc060ce16014f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f6c575bc060ce16014f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:53,037 - lava_callback - INFO - Starting async job submit for node 69408f88575bc060ce16019b 2025-12-17 22:56:53,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:53,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f88575bc060ce16019b HTTP/1.1" 502 150 2025-12-17 22:56:53,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f88575bc060ce16019b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:53,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f88575bc060ce16019b 2025-12-17 22:56:53,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f88575bc060ce16019b HTTP/1.1" 502 150 2025-12-17 22:56:53,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f88575bc060ce16019b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:53,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6e575bc060ce160152 2025-12-17 22:56:53,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6e575bc060ce160152 HTTP/1.1" 502 150 2025-12-17 22:56:53,683 - lava_callback - ERROR - Error processing callback for node 69408f6e575bc060ce160152: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f6e575bc060ce160152 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f6e575bc060ce160152 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f6e575bc060ce160152 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:53,685 - lava_callback - INFO - Starting async job submit for node 69408b6d575bc060ce15f1e1 2025-12-17 22:56:53,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:53,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6d575bc060ce15f1e1 HTTP/1.1" 502 150 2025-12-17 22:56:53,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6d575bc060ce15f1e1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:53,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6d575bc060ce15f1e1 2025-12-17 22:56:53,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6d575bc060ce15f1e1 HTTP/1.1" 502 150 2025-12-17 22:56:53,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6d575bc060ce15f1e1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:53,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409618575bc060ce1614eb 2025-12-17 22:56:53,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409618575bc060ce1614eb HTTP/1.1" 502 150 2025-12-17 22:56:53,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409618575bc060ce1614eb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:54,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f6f575bc060ce160155 2025-12-17 22:56:54,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f6f575bc060ce160155 HTTP/1.1" 502 150 2025-12-17 22:56:54,627 - lava_callback - ERROR - Error processing callback for node 69408f6f575bc060ce160155: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f6f575bc060ce160155 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f6f575bc060ce160155 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f6f575bc060ce160155 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:54,629 - lava_callback - INFO - Starting async job submit for node 69408f8a575bc060ce16019e 2025-12-17 22:56:54,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:54,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8a575bc060ce16019e HTTP/1.1" 502 150 2025-12-17 22:56:54,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8a575bc060ce16019e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:54,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8a575bc060ce16019e 2025-12-17 22:56:54,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8a575bc060ce16019e HTTP/1.1" 502 150 2025-12-17 22:56:54,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8a575bc060ce16019e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:55,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f88575bc060ce16019b 2025-12-17 22:56:55,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f88575bc060ce16019b HTTP/1.1" 502 150 2025-12-17 22:56:55,047 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f88575bc060ce16019b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:55,292 - lava_callback - INFO - Received callback for node 6940936b575bc060ce160d6c INFO: 172.21.0.1:41898 - "POST /node/6940936b575bc060ce160d6c HTTP/1.0" 202 Accepted 2025-12-17 22:56:55,623 - lava_callback - INFO - Received callback for node 69409225575bc060ce160a06 INFO: 172.21.0.1:41912 - "POST /node/69409225575bc060ce160a06 HTTP/1.0" 202 Accepted 2025-12-17 22:56:55,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6d575bc060ce15f1e1 2025-12-17 22:56:55,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6d575bc060ce15f1e1 HTTP/1.1" 502 150 2025-12-17 22:56:55,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6d575bc060ce15f1e1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:56,262 - lava_callback - INFO - Received callback for node 69409227575bc060ce160a09 INFO: 172.21.0.1:41914 - "POST /node/69409227575bc060ce160a09 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:41928 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:56:56,527 - lava_callback - INFO - Received callback for node 69409790575bc060ce16186d INFO: 172.21.0.1:41932 - "POST /node/69409790575bc060ce16186d HTTP/1.0" 202 Accepted 2025-12-17 22:56:56,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8a575bc060ce16019e 2025-12-17 22:56:56,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8a575bc060ce16019e HTTP/1.1" 502 150 2025-12-17 22:56:56,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8a575bc060ce16019e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:56,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094cd575bc060ce161115 2025-12-17 22:56:56,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cd575bc060ce161115 HTTP/1.1" 502 150 2025-12-17 22:56:56,818 - lava_callback - ERROR - Error processing callback for node 694094cd575bc060ce161115: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094cd575bc060ce161115 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094cd575bc060ce161115 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094cd575bc060ce161115 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:56,820 - lava_callback - INFO - Starting async job submit for node 69408f8c575bc060ce1601a1 2025-12-17 22:56:56,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:56,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8c575bc060ce1601a1 HTTP/1.1" 502 150 2025-12-17 22:56:56,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8c575bc060ce1601a1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:56,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8c575bc060ce1601a1 2025-12-17 22:56:56,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8c575bc060ce1601a1 HTTP/1.1" 502 150 2025-12-17 22:56:56,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8c575bc060ce1601a1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:56,931 - lava_callback - INFO - Received callback for node 69409229575bc060ce160a12 INFO: 172.21.0.1:41940 - "POST /node/69409229575bc060ce160a12 HTTP/1.0" 202 Accepted 2025-12-17 22:56:57,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f72575bc060ce160162 2025-12-17 22:56:57,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f72575bc060ce160162 HTTP/1.1" 502 150 2025-12-17 22:56:57,448 - lava_callback - ERROR - Error processing callback for node 69408f72575bc060ce160162: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f72575bc060ce160162 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f72575bc060ce160162 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f72575bc060ce160162 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:57,449 - lava_callback - INFO - Starting async job submit for node 69408f8d575bc060ce1601a4 2025-12-17 22:56:57,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:57,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8d575bc060ce1601a4 HTTP/1.1" 502 150 2025-12-17 22:56:57,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8d575bc060ce1601a4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:57,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8d575bc060ce1601a4 2025-12-17 22:56:57,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8d575bc060ce1601a4 HTTP/1.1" 502 150 2025-12-17 22:56:57,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8d575bc060ce1601a4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:57,584 - lava_callback - INFO - Received callback for node 6940922b575bc060ce160a15 INFO: 172.21.0.1:41944 - "POST /node/6940922b575bc060ce160a15 HTTP/1.0" 202 Accepted 2025-12-17 22:56:57,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f74575bc060ce160169 2025-12-17 22:56:57,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f74575bc060ce160169 HTTP/1.1" 502 150 2025-12-17 22:56:57,925 - lava_callback - ERROR - Error processing callback for node 69408f74575bc060ce160169: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f74575bc060ce160169 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f74575bc060ce160169 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f74575bc060ce160169 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:56:57,927 - lava_callback - INFO - Starting async job submit for node 69408f8f575bc060ce1601a7 2025-12-17 22:56:57,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:56:57,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8f575bc060ce1601a7 HTTP/1.1" 502 150 2025-12-17 22:56:57,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8f575bc060ce1601a7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:57,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8f575bc060ce1601a7 2025-12-17 22:56:57,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8f575bc060ce1601a7 HTTP/1.1" 502 150 2025-12-17 22:56:57,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8f575bc060ce1601a7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:58,245 - lava_callback - INFO - Received callback for node 6940922d575bc060ce160a1e INFO: 172.21.0.1:41956 - "POST /node/6940922d575bc060ce160a1e HTTP/1.0" 202 Accepted 2025-12-17 22:56:58,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8c575bc060ce1601a1 2025-12-17 22:56:58,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8c575bc060ce1601a1 HTTP/1.1" 502 150 2025-12-17 22:56:58,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8c575bc060ce1601a1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:58,923 - lava_callback - INFO - Received callback for node 69409230575bc060ce160a28 INFO: 172.21.0.1:41972 - "POST /node/69409230575bc060ce160a28 HTTP/1.0" 202 Accepted 2025-12-17 22:56:59,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f88575bc060ce16019b 2025-12-17 22:56:59,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f88575bc060ce16019b HTTP/1.1" 502 150 2025-12-17 22:56:59,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f88575bc060ce16019b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:59,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8d575bc060ce1601a4 2025-12-17 22:56:59,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8d575bc060ce1601a4 HTTP/1.1" 502 150 2025-12-17 22:56:59,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8d575bc060ce1601a4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:59,582 - lava_callback - INFO - Received callback for node 69409232575bc060ce160a34 INFO: 172.21.0.1:41988 - "POST /node/69409232575bc060ce160a34 HTTP/1.0" 202 Accepted 2025-12-17 22:56:59,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6d575bc060ce15f1e1 2025-12-17 22:56:59,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6d575bc060ce15f1e1 HTTP/1.1" 502 150 2025-12-17 22:56:59,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6d575bc060ce15f1e1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:56:59,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8f575bc060ce1601a7 2025-12-17 22:56:59,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8f575bc060ce1601a7 HTTP/1.1" 502 150 2025-12-17 22:56:59,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8f575bc060ce1601a7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:00,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f86575bc060ce160192 2025-12-17 22:57:00,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f86575bc060ce160192 HTTP/1.1" 502 150 2025-12-17 22:57:00,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f86575bc060ce160192'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:00,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8a575bc060ce16019e 2025-12-17 22:57:00,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8a575bc060ce16019e HTTP/1.1" 502 150 2025-12-17 22:57:00,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8a575bc060ce16019e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46068 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:57:02,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8c575bc060ce1601a1 2025-12-17 22:57:02,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8c575bc060ce1601a1 HTTP/1.1" 502 150 2025-12-17 22:57:02,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8c575bc060ce1601a1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:03,219 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f76575bc060ce16016c 2025-12-17 22:57:03,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f76575bc060ce16016c HTTP/1.1" 502 150 2025-12-17 22:57:03,221 - lava_callback - ERROR - Error processing callback for node 69408f76575bc060ce16016c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f76575bc060ce16016c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f76575bc060ce16016c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f76575bc060ce16016c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:03,222 - lava_callback - INFO - Starting async job submit for node 69408f91575bc060ce1601aa 2025-12-17 22:57:03,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:03,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f91575bc060ce1601aa HTTP/1.1" 502 150 2025-12-17 22:57:03,249 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f91575bc060ce1601aa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:03,249 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f91575bc060ce1601aa 2025-12-17 22:57:03,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f91575bc060ce1601aa HTTP/1.1" 502 150 2025-12-17 22:57:03,250 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f91575bc060ce1601aa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:03,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8d575bc060ce1601a4 2025-12-17 22:57:03,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8d575bc060ce1601a4 HTTP/1.1" 502 150 2025-12-17 22:57:03,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8d575bc060ce1601a4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:03,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8f575bc060ce1601a7 2025-12-17 22:57:03,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8f575bc060ce1601a7 HTTP/1.1" 502 150 2025-12-17 22:57:03,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8f575bc060ce1601a7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:05,250 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f91575bc060ce1601aa 2025-12-17 22:57:05,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f91575bc060ce1601aa HTTP/1.1" 502 150 2025-12-17 22:57:05,252 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f91575bc060ce1601aa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:05,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f78575bc060ce16016f 2025-12-17 22:57:05,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f78575bc060ce16016f HTTP/1.1" 502 150 2025-12-17 22:57:05,886 - lava_callback - ERROR - Error processing callback for node 69408f78575bc060ce16016f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f78575bc060ce16016f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f78575bc060ce16016f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f78575bc060ce16016f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:05,888 - lava_callback - INFO - Starting async job submit for node 69408f93575bc060ce1601ad 2025-12-17 22:57:05,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:05,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f93575bc060ce1601ad HTTP/1.1" 502 150 2025-12-17 22:57:05,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f93575bc060ce1601ad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:05,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f93575bc060ce1601ad 2025-12-17 22:57:05,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f93575bc060ce1601ad HTTP/1.1" 502 150 2025-12-17 22:57:05,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f93575bc060ce1601ad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:05,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f79575bc060ce160172 2025-12-17 22:57:05,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f79575bc060ce160172 HTTP/1.1" 502 150 2025-12-17 22:57:05,930 - lava_callback - ERROR - Error processing callback for node 69408f79575bc060ce160172: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f79575bc060ce160172 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f79575bc060ce160172 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f79575bc060ce160172 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:05,931 - lava_callback - INFO - Starting async job submit for node 69408f98575bc060ce1601cc 2025-12-17 22:57:05,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:05,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f98575bc060ce1601cc HTTP/1.1" 502 150 2025-12-17 22:57:05,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f98575bc060ce1601cc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:05,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f98575bc060ce1601cc 2025-12-17 22:57:05,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f98575bc060ce1601cc HTTP/1.1" 502 150 2025-12-17 22:57:05,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f98575bc060ce1601cc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:06,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f7b575bc060ce160175 2025-12-17 22:57:06,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7b575bc060ce160175 HTTP/1.1" 502 150 2025-12-17 22:57:06,032 - lava_callback - ERROR - Error processing callback for node 69408f7b575bc060ce160175: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f7b575bc060ce160175 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f7b575bc060ce160175 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f7b575bc060ce160175 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:06,033 - lava_callback - INFO - Starting async job submit for node 6940928f575bc060ce160aea 2025-12-17 22:57:06,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:06,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940928f575bc060ce160aea HTTP/1.1" 502 150 2025-12-17 22:57:06,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940928f575bc060ce160aea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:06,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940928f575bc060ce160aea 2025-12-17 22:57:06,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940928f575bc060ce160aea HTTP/1.1" 502 150 2025-12-17 22:57:06,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940928f575bc060ce160aea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:06,068 - lava_callback - INFO - Received callback for node 69409235575bc060ce160a38 INFO: 172.21.0.1:46076 - "POST /node/69409235575bc060ce160a38 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:46088 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:57:06,707 - lava_callback - INFO - Received callback for node 6940923f575bc060ce160a4b INFO: 172.21.0.1:46104 - "POST /node/6940923f575bc060ce160a4b HTTP/1.0" 202 Accepted 2025-12-17 22:57:07,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f88575bc060ce16019b 2025-12-17 22:57:07,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f88575bc060ce16019b HTTP/1.1" 502 150 2025-12-17 22:57:07,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f88575bc060ce16019b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:07,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f7d575bc060ce160178 2025-12-17 22:57:07,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7d575bc060ce160178 HTTP/1.1" 502 150 2025-12-17 22:57:07,539 - lava_callback - ERROR - Error processing callback for node 69408f7d575bc060ce160178: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f7d575bc060ce160178 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f7d575bc060ce160178 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f7d575bc060ce160178 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:07,541 - lava_callback - INFO - Starting async job submit for node 69408f9a575bc060ce1601db 2025-12-17 22:57:07,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:07,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9a575bc060ce1601db HTTP/1.1" 502 150 2025-12-17 22:57:07,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f9a575bc060ce1601db'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:07,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f9a575bc060ce1601db 2025-12-17 22:57:07,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9a575bc060ce1601db HTTP/1.1" 502 150 2025-12-17 22:57:07,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f9a575bc060ce1601db'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:07,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f7f575bc060ce16017b 2025-12-17 22:57:07,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f7f575bc060ce16017b HTTP/1.1" 502 150 2025-12-17 22:57:07,668 - lava_callback - ERROR - Error processing callback for node 69408f7f575bc060ce16017b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f7f575bc060ce16017b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f7f575bc060ce16017b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f7f575bc060ce16017b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:07,669 - lava_callback - INFO - Starting async job submit for node 69408f9c575bc060ce1601e5 2025-12-17 22:57:07,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:07,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9c575bc060ce1601e5 HTTP/1.1" 502 150 2025-12-17 22:57:07,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f9c575bc060ce1601e5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:07,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f9c575bc060ce1601e5 2025-12-17 22:57:07,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9c575bc060ce1601e5 HTTP/1.1" 502 150 2025-12-17 22:57:07,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f9c575bc060ce1601e5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:07,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6d575bc060ce15f1e1 2025-12-17 22:57:07,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6d575bc060ce15f1e1 HTTP/1.1" 502 150 2025-12-17 22:57:07,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408b6d575bc060ce15f1e1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:07,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f93575bc060ce1601ad 2025-12-17 22:57:07,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f93575bc060ce1601ad HTTP/1.1" 502 150 2025-12-17 22:57:07,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f93575bc060ce1601ad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:07,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f98575bc060ce1601cc 2025-12-17 22:57:07,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f98575bc060ce1601cc HTTP/1.1" 502 150 2025-12-17 22:57:07,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f98575bc060ce1601cc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:08,059 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940928f575bc060ce160aea 2025-12-17 22:57:08,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8a575bc060ce16019e 2025-12-17 22:57:08,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8a575bc060ce16019e HTTP/1.1" 502 150 2025-12-17 22:57:08,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8a575bc060ce16019e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:08,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f81575bc060ce16017e 2025-12-17 22:57:08,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f81575bc060ce16017e HTTP/1.1" 502 150 2025-12-17 22:57:08,898 - lava_callback - ERROR - Error processing callback for node 69408f81575bc060ce16017e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f81575bc060ce16017e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f81575bc060ce16017e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f81575bc060ce16017e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:08,900 - lava_callback - INFO - Starting async job submit for node 69408f9e575bc060ce1601ee 2025-12-17 22:57:08,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:08,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9e575bc060ce1601ee HTTP/1.1" 502 150 2025-12-17 22:57:08,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f9e575bc060ce1601ee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:08,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f9e575bc060ce1601ee 2025-12-17 22:57:08,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9e575bc060ce1601ee HTTP/1.1" 502 150 2025-12-17 22:57:08,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f9e575bc060ce1601ee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:08,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f83575bc060ce160181 2025-12-17 22:57:08,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f83575bc060ce160181 HTTP/1.1" 502 150 2025-12-17 22:57:08,935 - lava_callback - ERROR - Error processing callback for node 69408f83575bc060ce160181: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f83575bc060ce160181 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f83575bc060ce160181 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f83575bc060ce160181 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:08,936 - lava_callback - INFO - Starting async job submit for node 69408fa1575bc060ce1601f3 2025-12-17 22:57:08,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:08,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa1575bc060ce1601f3 HTTP/1.1" 502 150 2025-12-17 22:57:08,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fa1575bc060ce1601f3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:08,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fa1575bc060ce1601f3 2025-12-17 22:57:08,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa1575bc060ce1601f3 HTTP/1.1" 502 150 2025-12-17 22:57:08,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fa1575bc060ce1601f3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:09,252 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f91575bc060ce1601aa 2025-12-17 22:57:09,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f91575bc060ce1601aa HTTP/1.1" 502 150 2025-12-17 22:57:09,253 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f91575bc060ce1601aa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:09,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f9a575bc060ce1601db 2025-12-17 22:57:09,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9a575bc060ce1601db HTTP/1.1" 502 150 2025-12-17 22:57:09,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f9a575bc060ce1601db'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:09,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f9c575bc060ce1601e5 2025-12-17 22:57:09,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9c575bc060ce1601e5 HTTP/1.1" 502 150 2025-12-17 22:57:09,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f9c575bc060ce1601e5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:09,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409618575bc060ce1614eb 2025-12-17 22:57:09,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409618575bc060ce1614eb HTTP/1.1" 502 150 2025-12-17 22:57:09,832 - lava_callback - ERROR - Error processing callback for node 69409618575bc060ce1614eb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409618575bc060ce1614eb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409618575bc060ce1614eb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409618575bc060ce1614eb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:09,834 - lava_callback - INFO - Starting async job submit for node 69408fa7575bc060ce1601f6 2025-12-17 22:57:09,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:09,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa7575bc060ce1601f6 HTTP/1.1" 502 150 2025-12-17 22:57:09,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fa7575bc060ce1601f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:09,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fa7575bc060ce1601f6 2025-12-17 22:57:09,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa7575bc060ce1601f6 HTTP/1.1" 502 150 2025-12-17 22:57:09,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fa7575bc060ce1601f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:10,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8c575bc060ce1601a1 2025-12-17 22:57:10,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8c575bc060ce1601a1 HTTP/1.1" 502 150 2025-12-17 22:57:10,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8c575bc060ce1601a1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:10,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f9e575bc060ce1601ee 2025-12-17 22:57:10,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9e575bc060ce1601ee HTTP/1.1" 502 150 2025-12-17 22:57:10,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f9e575bc060ce1601ee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:10,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fa1575bc060ce1601f3 2025-12-17 22:57:10,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa1575bc060ce1601f3 HTTP/1.1" 502 150 2025-12-17 22:57:10,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fa1575bc060ce1601f3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:11,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8d575bc060ce1601a4 2025-12-17 22:57:11,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8d575bc060ce1601a4 HTTP/1.1" 502 150 2025-12-17 22:57:11,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8d575bc060ce1601a4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:11,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fa7575bc060ce1601f6 2025-12-17 22:57:11,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa7575bc060ce1601f6 HTTP/1.1" 502 150 2025-12-17 22:57:11,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fa7575bc060ce1601f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:11,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f93575bc060ce1601ad 2025-12-17 22:57:11,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f93575bc060ce1601ad HTTP/1.1" 502 150 2025-12-17 22:57:11,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f93575bc060ce1601ad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:11,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f98575bc060ce1601cc 2025-12-17 22:57:11,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f98575bc060ce1601cc HTTP/1.1" 502 150 2025-12-17 22:57:11,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f98575bc060ce1601cc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:11,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8f575bc060ce1601a7 2025-12-17 22:57:11,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8f575bc060ce1601a7 HTTP/1.1" 502 150 2025-12-17 22:57:11,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f8f575bc060ce1601a7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:13,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f9a575bc060ce1601db 2025-12-17 22:57:13,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9a575bc060ce1601db HTTP/1.1" 502 150 2025-12-17 22:57:13,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f9a575bc060ce1601db'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:13,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f9c575bc060ce1601e5 2025-12-17 22:57:13,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9c575bc060ce1601e5 HTTP/1.1" 502 150 2025-12-17 22:57:13,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f9c575bc060ce1601e5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:14,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f9e575bc060ce1601ee 2025-12-17 22:57:14,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9e575bc060ce1601ee HTTP/1.1" 502 150 2025-12-17 22:57:14,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f9e575bc060ce1601ee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:14,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fa1575bc060ce1601f3 2025-12-17 22:57:14,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa1575bc060ce1601f3 HTTP/1.1" 502 150 2025-12-17 22:57:14,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fa1575bc060ce1601f3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60342 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:57:15,674 - lava_callback - INFO - Received callback for node 69409237575bc060ce160a3f INFO: 172.21.0.1:60358 - "POST /node/69409237575bc060ce160a3f HTTP/1.0" 202 Accepted 2025-12-17 22:57:15,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fa7575bc060ce1601f6 2025-12-17 22:57:15,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa7575bc060ce1601f6 HTTP/1.1" 502 150 2025-12-17 22:57:15,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fa7575bc060ce1601f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:16,341 - lava_callback - INFO - Received callback for node 69409239575bc060ce160a42 INFO: 172.21.0.1:60366 - "POST /node/69409239575bc060ce160a42 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:60374 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:57:16,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f86575bc060ce160192 2025-12-17 22:57:16,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f86575bc060ce160192 HTTP/1.1" 502 150 2025-12-17 22:57:16,536 - lava_callback - ERROR - Error processing callback for node 69408f86575bc060ce160192: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f86575bc060ce160192 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f86575bc060ce160192 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f86575bc060ce160192 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:16,539 - lava_callback - INFO - Starting async job submit for node 69408fa9575bc060ce1601f9 2025-12-17 22:57:16,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:16,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa9575bc060ce1601f9 HTTP/1.1" 502 150 2025-12-17 22:57:16,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fa9575bc060ce1601f9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:16,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fa9575bc060ce1601f9 2025-12-17 22:57:16,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa9575bc060ce1601f9 HTTP/1.1" 502 150 2025-12-17 22:57:16,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fa9575bc060ce1601f9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:17,022 - lava_callback - INFO - Received callback for node 6940923b575bc060ce160a45 INFO: 172.21.0.1:60380 - "POST /node/6940923b575bc060ce160a45 HTTP/1.0" 202 Accepted 2025-12-17 22:57:17,253 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f91575bc060ce1601aa 2025-12-17 22:57:17,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f91575bc060ce1601aa HTTP/1.1" 502 150 2025-12-17 22:57:17,255 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f91575bc060ce1601aa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:17,689 - lava_callback - INFO - Received callback for node 6940923d575bc060ce160a48 INFO: 172.21.0.1:60382 - "POST /node/6940923d575bc060ce160a48 HTTP/1.0" 202 Accepted 2025-12-17 22:57:18,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fa9575bc060ce1601f9 2025-12-17 22:57:18,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa9575bc060ce1601f9 HTTP/1.1" 502 150 2025-12-17 22:57:18,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fa9575bc060ce1601f9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:19,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f93575bc060ce1601ad 2025-12-17 22:57:19,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f93575bc060ce1601ad HTTP/1.1" 502 150 2025-12-17 22:57:19,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f93575bc060ce1601ad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:19,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f98575bc060ce1601cc 2025-12-17 22:57:19,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f98575bc060ce1601cc HTTP/1.1" 502 150 2025-12-17 22:57:19,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f98575bc060ce1601cc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:21,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f9a575bc060ce1601db 2025-12-17 22:57:21,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9a575bc060ce1601db HTTP/1.1" 502 150 2025-12-17 22:57:21,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f9a575bc060ce1601db'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:21,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f9c575bc060ce1601e5 2025-12-17 22:57:21,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9c575bc060ce1601e5 HTTP/1.1" 502 150 2025-12-17 22:57:21,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f9c575bc060ce1601e5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:22,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fa9575bc060ce1601f9 2025-12-17 22:57:22,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa9575bc060ce1601f9 HTTP/1.1" 502 150 2025-12-17 22:57:22,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fa9575bc060ce1601f9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:22,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f9e575bc060ce1601ee 2025-12-17 22:57:22,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9e575bc060ce1601ee HTTP/1.1" 502 150 2025-12-17 22:57:22,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f9e575bc060ce1601ee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:22,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fa1575bc060ce1601f3 2025-12-17 22:57:22,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa1575bc060ce1601f3 HTTP/1.1" 502 150 2025-12-17 22:57:22,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fa1575bc060ce1601f3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:23,049 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f88575bc060ce16019b 2025-12-17 22:57:23,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940928f575bc060ce160aea HTTP/1.1" 502 150 2025-12-17 22:57:23,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940928f575bc060ce160aea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:23,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408b6d575bc060ce15f1e1 2025-12-17 22:57:23,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408b6d575bc060ce15f1e1 HTTP/1.1" 502 150 2025-12-17 22:57:23,730 - lava_callback - ERROR - Error processing callback for node 69408b6d575bc060ce15f1e1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b6d575bc060ce15f1e1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b6d575bc060ce15f1e1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408b6d575bc060ce15f1e1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:23,738 - lava_callback - INFO - Starting async job submit for node 69408fab575bc060ce1601fc 2025-12-17 22:57:23,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:23,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fab575bc060ce1601fc HTTP/1.1" 502 150 2025-12-17 22:57:23,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fab575bc060ce1601fc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:23,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fab575bc060ce1601fc 2025-12-17 22:57:23,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fab575bc060ce1601fc HTTP/1.1" 502 150 2025-12-17 22:57:23,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fab575bc060ce1601fc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:23,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fa7575bc060ce1601f6 2025-12-17 22:57:23,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa7575bc060ce1601f6 HTTP/1.1" 502 150 2025-12-17 22:57:23,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fa7575bc060ce1601f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:24,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8a575bc060ce16019e 2025-12-17 22:57:24,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8a575bc060ce16019e HTTP/1.1" 502 150 2025-12-17 22:57:24,647 - lava_callback - ERROR - Error processing callback for node 69408f8a575bc060ce16019e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f8a575bc060ce16019e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f8a575bc060ce16019e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f8a575bc060ce16019e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:24,648 - lava_callback - INFO - Starting async job submit for node 694098e5575bc060ce161b9b 2025-12-17 22:57:24,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:24,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e5575bc060ce161b9b HTTP/1.1" 502 150 2025-12-17 22:57:24,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e5575bc060ce161b9b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:24,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e5575bc060ce161b9b 2025-12-17 22:57:24,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e5575bc060ce161b9b HTTP/1.1" 502 150 2025-12-17 22:57:24,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e5575bc060ce161b9b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:25,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fab575bc060ce1601fc 2025-12-17 22:57:25,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fab575bc060ce1601fc HTTP/1.1" 502 150 2025-12-17 22:57:25,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fab575bc060ce1601fc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:26,099 - lava_callback - INFO - Received callback for node 69409241575bc060ce160a54 INFO: 172.21.0.1:46358 - "POST /node/69409241575bc060ce160a54 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:46364 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:57:26,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e5575bc060ce161b9b 2025-12-17 22:57:26,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e5575bc060ce161b9b HTTP/1.1" 502 150 2025-12-17 22:57:26,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e5575bc060ce161b9b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:26,752 - lava_callback - INFO - Received callback for node 69409243575bc060ce160a5d INFO: 172.21.0.1:46370 - "POST /node/69409243575bc060ce160a5d HTTP/1.0" 202 Accepted 2025-12-17 22:57:26,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8c575bc060ce1601a1 2025-12-17 22:57:26,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8c575bc060ce1601a1 HTTP/1.1" 502 150 2025-12-17 22:57:26,849 - lava_callback - ERROR - Error processing callback for node 69408f8c575bc060ce1601a1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f8c575bc060ce1601a1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f8c575bc060ce1601a1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f8c575bc060ce1601a1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:26,851 - lava_callback - INFO - Starting async job submit for node 69408faf575bc060ce16020c 2025-12-17 22:57:26,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:26,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408faf575bc060ce16020c HTTP/1.1" 502 150 2025-12-17 22:57:26,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408faf575bc060ce16020c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:26,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408faf575bc060ce16020c 2025-12-17 22:57:26,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408faf575bc060ce16020c HTTP/1.1" 502 150 2025-12-17 22:57:26,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408faf575bc060ce16020c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:27,377 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940928f575bc060ce160aea 2025-12-17 22:57:27,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940928f575bc060ce160aea HTTP/1.1" 502 150 2025-12-17 22:57:27,378 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940928f575bc060ce160aea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:27,422 - lava_callback - INFO - Received callback for node 69409245575bc060ce160a60 INFO: 172.21.0.1:46386 - "POST /node/69409245575bc060ce160a60 HTTP/1.0" 202 Accepted 2025-12-17 22:57:27,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8d575bc060ce1601a4 2025-12-17 22:57:27,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8d575bc060ce1601a4 HTTP/1.1" 502 150 2025-12-17 22:57:27,472 - lava_callback - ERROR - Error processing callback for node 69408f8d575bc060ce1601a4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f8d575bc060ce1601a4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f8d575bc060ce1601a4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f8d575bc060ce1601a4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:27,473 - lava_callback - INFO - Starting async job submit for node 69408e9d575bc060ce15fea3 2025-12-17 22:57:27,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:27,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9d575bc060ce15fea3 HTTP/1.1" 502 150 2025-12-17 22:57:27,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9d575bc060ce15fea3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:27,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9d575bc060ce15fea3 2025-12-17 22:57:27,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9d575bc060ce15fea3 HTTP/1.1" 502 150 2025-12-17 22:57:27,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9d575bc060ce15fea3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:27,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f8f575bc060ce1601a7 2025-12-17 22:57:27,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f8f575bc060ce1601a7 HTTP/1.1" 502 150 2025-12-17 22:57:27,944 - lava_callback - ERROR - Error processing callback for node 69408f8f575bc060ce1601a7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f8f575bc060ce1601a7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f8f575bc060ce1601a7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f8f575bc060ce1601a7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:27,947 - lava_callback - INFO - Starting async job submit for node 69408fb1575bc060ce160211 2025-12-17 22:57:27,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:27,988 - lava_callback - INFO - Received callback for node 69410fd9460bc8e265117ad5 INFO: 172.21.0.1:46398 - "POST /node/69410fd9460bc8e265117ad5 HTTP/1.0" 202 Accepted 2025-12-17 22:57:28,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb1575bc060ce160211 HTTP/1.1" 502 150 2025-12-17 22:57:28,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb1575bc060ce160211'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:28,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb1575bc060ce160211 2025-12-17 22:57:28,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb1575bc060ce160211 HTTP/1.1" 502 150 2025-12-17 22:57:28,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb1575bc060ce160211'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:28,073 - lava_callback - INFO - Received callback for node 69409247575bc060ce160a63 INFO: 172.21.0.1:46404 - "POST /node/69409247575bc060ce160a63 HTTP/1.0" 202 Accepted 2025-12-17 22:57:28,722 - lava_callback - INFO - Received callback for node 69409249575bc060ce160a66 INFO: 172.21.0.1:46406 - "POST /node/69409249575bc060ce160a66 HTTP/1.0" 202 Accepted 2025-12-17 22:57:28,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408faf575bc060ce16020c 2025-12-17 22:57:28,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408faf575bc060ce16020c HTTP/1.1" 502 150 2025-12-17 22:57:28,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408faf575bc060ce16020c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:29,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9d575bc060ce15fea3 2025-12-17 22:57:29,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9d575bc060ce15fea3 HTTP/1.1" 502 150 2025-12-17 22:57:29,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9d575bc060ce15fea3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:29,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fab575bc060ce1601fc 2025-12-17 22:57:29,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fab575bc060ce1601fc HTTP/1.1" 502 150 2025-12-17 22:57:29,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fab575bc060ce1601fc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:30,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb1575bc060ce160211 2025-12-17 22:57:30,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb1575bc060ce160211 HTTP/1.1" 502 150 2025-12-17 22:57:30,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb1575bc060ce160211'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:30,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fa9575bc060ce1601f9 2025-12-17 22:57:30,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa9575bc060ce1601f9 HTTP/1.1" 502 150 2025-12-17 22:57:30,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fa9575bc060ce1601f9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:30,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e5575bc060ce161b9b 2025-12-17 22:57:30,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e5575bc060ce161b9b HTTP/1.1" 502 150 2025-12-17 22:57:30,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e5575bc060ce161b9b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:32,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408faf575bc060ce16020c 2025-12-17 22:57:32,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408faf575bc060ce16020c HTTP/1.1" 502 150 2025-12-17 22:57:32,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408faf575bc060ce16020c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:33,255 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f91575bc060ce1601aa 2025-12-17 22:57:33,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f91575bc060ce1601aa HTTP/1.1" 502 150 2025-12-17 22:57:33,256 - lava_callback - ERROR - Error processing callback for node 69408f91575bc060ce1601aa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f91575bc060ce1601aa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f91575bc060ce1601aa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f91575bc060ce1601aa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:33,258 - lava_callback - INFO - Starting async job submit for node 69408fb3575bc060ce160214 2025-12-17 22:57:33,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:33,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb3575bc060ce160214 HTTP/1.1" 502 150 2025-12-17 22:57:33,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb3575bc060ce160214'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:33,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb3575bc060ce160214 2025-12-17 22:57:33,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb3575bc060ce160214 HTTP/1.1" 502 150 2025-12-17 22:57:33,267 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb3575bc060ce160214'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:33,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9d575bc060ce15fea3 2025-12-17 22:57:33,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9d575bc060ce15fea3 HTTP/1.1" 502 150 2025-12-17 22:57:33,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9d575bc060ce15fea3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:34,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb1575bc060ce160211 2025-12-17 22:57:35,267 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb3575bc060ce160214 2025-12-17 22:57:35,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb3575bc060ce160214 HTTP/1.1" 502 150 2025-12-17 22:57:35,268 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb3575bc060ce160214'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:35,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940928f575bc060ce160aea 2025-12-17 22:57:35,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940928f575bc060ce160aea HTTP/1.1" 502 150 2025-12-17 22:57:35,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940928f575bc060ce160aea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:35,716 - lava_callback - INFO - Received callback for node 6940955d575bc060ce1612e5 INFO: 172.21.0.1:48646 - "POST /node/6940955d575bc060ce1612e5 HTTP/1.0" 202 Accepted 2025-12-17 22:57:35,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f93575bc060ce1601ad 2025-12-17 22:57:35,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f93575bc060ce1601ad HTTP/1.1" 502 150 2025-12-17 22:57:35,909 - lava_callback - ERROR - Error processing callback for node 69408f93575bc060ce1601ad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f93575bc060ce1601ad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f93575bc060ce1601ad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f93575bc060ce1601ad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:35,911 - lava_callback - INFO - Starting async job submit for node 69408fb5575bc060ce160217 2025-12-17 22:57:35,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:35,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb5575bc060ce160217 HTTP/1.1" 502 150 2025-12-17 22:57:35,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb5575bc060ce160217'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:35,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb5575bc060ce160217 2025-12-17 22:57:35,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb5575bc060ce160217 HTTP/1.1" 502 150 2025-12-17 22:57:35,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb5575bc060ce160217'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:35,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f98575bc060ce1601cc 2025-12-17 22:57:35,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f98575bc060ce1601cc HTTP/1.1" 502 150 2025-12-17 22:57:35,943 - lava_callback - ERROR - Error processing callback for node 69408f98575bc060ce1601cc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f98575bc060ce1601cc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f98575bc060ce1601cc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f98575bc060ce1601cc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:35,945 - lava_callback - INFO - Starting async job submit for node 69408fb7575bc060ce16021a 2025-12-17 22:57:35,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:35,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb7575bc060ce16021a HTTP/1.1" 502 150 2025-12-17 22:57:35,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb7575bc060ce16021a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:35,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb7575bc060ce16021a 2025-12-17 22:57:35,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb7575bc060ce16021a HTTP/1.1" 502 150 2025-12-17 22:57:35,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb7575bc060ce16021a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48650 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:57:37,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f9a575bc060ce1601db 2025-12-17 22:57:37,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9a575bc060ce1601db HTTP/1.1" 502 150 2025-12-17 22:57:37,554 - lava_callback - ERROR - Error processing callback for node 69408f9a575bc060ce1601db: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f9a575bc060ce1601db (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f9a575bc060ce1601db (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f9a575bc060ce1601db (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:37,556 - lava_callback - INFO - Starting async job submit for node 69408fba575bc060ce16021d 2025-12-17 22:57:37,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:37,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fba575bc060ce16021d HTTP/1.1" 502 150 2025-12-17 22:57:37,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fba575bc060ce16021d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:37,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fba575bc060ce16021d 2025-12-17 22:57:37,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fba575bc060ce16021d HTTP/1.1" 502 150 2025-12-17 22:57:37,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fba575bc060ce16021d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:37,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f9c575bc060ce1601e5 2025-12-17 22:57:37,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9c575bc060ce1601e5 HTTP/1.1" 502 150 2025-12-17 22:57:37,683 - lava_callback - ERROR - Error processing callback for node 69408f9c575bc060ce1601e5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f9c575bc060ce1601e5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f9c575bc060ce1601e5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f9c575bc060ce1601e5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:37,685 - lava_callback - INFO - Starting async job submit for node 69408fbc575bc060ce160220 2025-12-17 22:57:37,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:37,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fbc575bc060ce160220 HTTP/1.1" 502 150 2025-12-17 22:57:37,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fbc575bc060ce160220'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:37,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fbc575bc060ce160220 2025-12-17 22:57:37,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fbc575bc060ce160220 HTTP/1.1" 502 150 2025-12-17 22:57:37,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fbc575bc060ce160220'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:37,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fab575bc060ce1601fc 2025-12-17 22:57:37,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fab575bc060ce1601fc HTTP/1.1" 502 150 2025-12-17 22:57:37,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fab575bc060ce1601fc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:37,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb5575bc060ce160217 2025-12-17 22:57:37,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb5575bc060ce160217 HTTP/1.1" 502 150 2025-12-17 22:57:37,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb5575bc060ce160217'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:37,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb7575bc060ce16021a 2025-12-17 22:57:37,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb7575bc060ce16021a HTTP/1.1" 502 150 2025-12-17 22:57:37,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb7575bc060ce16021a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:38,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f88575bc060ce16019b HTTP/1.1" 502 150 2025-12-17 22:57:38,484 - lava_callback - ERROR - Error processing callback for node 69408f88575bc060ce16019b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f88575bc060ce16019b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f88575bc060ce16019b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f88575bc060ce16019b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:38,486 - lava_callback - INFO - Starting async job submit for node 69408fbf575bc060ce160223 2025-12-17 22:57:38,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:38,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fbf575bc060ce160223 HTTP/1.1" 502 150 2025-12-17 22:57:38,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fbf575bc060ce160223'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:38,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fbf575bc060ce160223 2025-12-17 22:57:38,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fbf575bc060ce160223 HTTP/1.1" 502 150 2025-12-17 22:57:38,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fbf575bc060ce160223'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:38,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e5575bc060ce161b9b 2025-12-17 22:57:38,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e5575bc060ce161b9b HTTP/1.1" 502 150 2025-12-17 22:57:38,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e5575bc060ce161b9b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:38,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f9e575bc060ce1601ee 2025-12-17 22:57:38,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f9e575bc060ce1601ee HTTP/1.1" 502 150 2025-12-17 22:57:38,921 - lava_callback - ERROR - Error processing callback for node 69408f9e575bc060ce1601ee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f9e575bc060ce1601ee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f9e575bc060ce1601ee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f9e575bc060ce1601ee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:38,923 - lava_callback - INFO - Starting async job submit for node 69408fc1575bc060ce160228 2025-12-17 22:57:38,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:38,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fc1575bc060ce160228 HTTP/1.1" 502 150 2025-12-17 22:57:38,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fc1575bc060ce160228'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:38,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fc1575bc060ce160228 2025-12-17 22:57:38,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fc1575bc060ce160228 HTTP/1.1" 502 150 2025-12-17 22:57:38,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fc1575bc060ce160228'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:38,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fa1575bc060ce1601f3 2025-12-17 22:57:38,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa1575bc060ce1601f3 HTTP/1.1" 502 150 2025-12-17 22:57:38,947 - lava_callback - ERROR - Error processing callback for node 69408fa1575bc060ce1601f3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fa1575bc060ce1601f3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fa1575bc060ce1601f3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fa1575bc060ce1601f3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:38,949 - lava_callback - INFO - Starting async job submit for node 69408fc7575bc060ce160240 2025-12-17 22:57:38,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:38,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fc7575bc060ce160240 HTTP/1.1" 502 150 2025-12-17 22:57:38,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fc7575bc060ce160240'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:38,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fc7575bc060ce160240 2025-12-17 22:57:38,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fc7575bc060ce160240 HTTP/1.1" 502 150 2025-12-17 22:57:38,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fc7575bc060ce160240'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:39,268 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb3575bc060ce160214 2025-12-17 22:57:39,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb3575bc060ce160214 HTTP/1.1" 502 150 2025-12-17 22:57:39,270 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb3575bc060ce160214'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:39,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fba575bc060ce16021d 2025-12-17 22:57:39,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fba575bc060ce16021d HTTP/1.1" 502 150 2025-12-17 22:57:39,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fba575bc060ce16021d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:39,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fbc575bc060ce160220 2025-12-17 22:57:39,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fbc575bc060ce160220 HTTP/1.1" 502 150 2025-12-17 22:57:39,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fbc575bc060ce160220'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:39,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fa7575bc060ce1601f6 2025-12-17 22:57:39,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa7575bc060ce1601f6 HTTP/1.1" 502 150 2025-12-17 22:57:39,848 - lava_callback - ERROR - Error processing callback for node 69408fa7575bc060ce1601f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fa7575bc060ce1601f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fa7575bc060ce1601f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fa7575bc060ce1601f6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:39,850 - lava_callback - INFO - Starting async job submit for node 69408fca575bc060ce160243 2025-12-17 22:57:39,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:39,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fca575bc060ce160243 HTTP/1.1" 502 150 2025-12-17 22:57:39,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fca575bc060ce160243'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:39,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fca575bc060ce160243 2025-12-17 22:57:39,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fca575bc060ce160243 HTTP/1.1" 502 150 2025-12-17 22:57:39,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fca575bc060ce160243'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:40,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fbf575bc060ce160223 2025-12-17 22:57:40,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fbf575bc060ce160223 HTTP/1.1" 502 150 2025-12-17 22:57:40,495 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fbf575bc060ce160223'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:40,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408faf575bc060ce16020c 2025-12-17 22:57:40,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408faf575bc060ce16020c HTTP/1.1" 502 150 2025-12-17 22:57:40,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408faf575bc060ce16020c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:40,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fc1575bc060ce160228 2025-12-17 22:57:40,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fc1575bc060ce160228 HTTP/1.1" 502 150 2025-12-17 22:57:40,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fc1575bc060ce160228'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:40,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fc7575bc060ce160240 2025-12-17 22:57:40,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fc7575bc060ce160240 HTTP/1.1" 502 150 2025-12-17 22:57:40,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fc7575bc060ce160240'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:41,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9d575bc060ce15fea3 2025-12-17 22:57:41,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9d575bc060ce15fea3 HTTP/1.1" 502 150 2025-12-17 22:57:41,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e9d575bc060ce15fea3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:41,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fca575bc060ce160243 2025-12-17 22:57:41,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fca575bc060ce160243 HTTP/1.1" 502 150 2025-12-17 22:57:41,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fca575bc060ce160243'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:41,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb5575bc060ce160217 2025-12-17 22:57:41,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb5575bc060ce160217 HTTP/1.1" 502 150 2025-12-17 22:57:41,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb5575bc060ce160217'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:41,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb7575bc060ce16021a 2025-12-17 22:57:41,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb7575bc060ce16021a HTTP/1.1" 502 150 2025-12-17 22:57:41,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb7575bc060ce16021a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:43,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fba575bc060ce16021d 2025-12-17 22:57:43,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fba575bc060ce16021d HTTP/1.1" 502 150 2025-12-17 22:57:43,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fba575bc060ce16021d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:43,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fbc575bc060ce160220 2025-12-17 22:57:43,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fbc575bc060ce160220 HTTP/1.1" 502 150 2025-12-17 22:57:43,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fbc575bc060ce160220'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:44,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fbf575bc060ce160223 2025-12-17 22:57:44,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fbf575bc060ce160223 HTTP/1.1" 502 150 2025-12-17 22:57:44,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fbf575bc060ce160223'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:44,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fc1575bc060ce160228 2025-12-17 22:57:44,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fc1575bc060ce160228 HTTP/1.1" 502 150 2025-12-17 22:57:44,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fc1575bc060ce160228'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:44,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fc7575bc060ce160240 2025-12-17 22:57:44,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fc7575bc060ce160240 HTTP/1.1" 502 150 2025-12-17 22:57:44,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fc7575bc060ce160240'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:45,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fca575bc060ce160243 2025-12-17 22:57:45,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fca575bc060ce160243 HTTP/1.1" 502 150 2025-12-17 22:57:45,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fca575bc060ce160243'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:46,116 - lava_callback - INFO - Received callback for node 69409255575bc060ce160a7c INFO: 172.21.0.1:50628 - "POST /node/69409255575bc060ce160a7c HTTP/1.0" 202 Accepted INFO: 172.21.0.1:50642 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:57:46,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fa9575bc060ce1601f9 2025-12-17 22:57:46,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fa9575bc060ce1601f9 HTTP/1.1" 502 150 2025-12-17 22:57:46,558 - lava_callback - ERROR - Error processing callback for node 69408fa9575bc060ce1601f9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fa9575bc060ce1601f9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fa9575bc060ce1601f9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fa9575bc060ce1601f9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:46,559 - lava_callback - INFO - Starting async job submit for node 69408fcc575bc060ce160246 2025-12-17 22:57:46,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:46,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fcc575bc060ce160246 HTTP/1.1" 502 150 2025-12-17 22:57:46,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fcc575bc060ce160246'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:46,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fcc575bc060ce160246 2025-12-17 22:57:46,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fcc575bc060ce160246 HTTP/1.1" 502 150 2025-12-17 22:57:46,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fcc575bc060ce160246'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:46,783 - lava_callback - INFO - Received callback for node 69409257575bc060ce160a86 INFO: 172.21.0.1:50658 - "POST /node/69409257575bc060ce160a86 HTTP/1.0" 202 Accepted 2025-12-17 22:57:47,270 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb3575bc060ce160214 2025-12-17 22:57:47,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb3575bc060ce160214 HTTP/1.1" 502 150 2025-12-17 22:57:47,271 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb3575bc060ce160214'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:47,452 - lava_callback - INFO - Received callback for node 69409259575bc060ce160a8a INFO: 172.21.0.1:50662 - "POST /node/69409259575bc060ce160a8a HTTP/1.0" 202 Accepted 2025-12-17 22:57:48,102 - lava_callback - INFO - Received callback for node 6940925a575bc060ce160a8d INFO: 172.21.0.1:50668 - "POST /node/6940925a575bc060ce160a8d HTTP/1.0" 202 Accepted 2025-12-17 22:57:48,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fcc575bc060ce160246 2025-12-17 22:57:48,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fcc575bc060ce160246 HTTP/1.1" 502 150 2025-12-17 22:57:48,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fcc575bc060ce160246'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:48,768 - lava_callback - INFO - Received callback for node 6940925b575bc060ce160a90 INFO: 172.21.0.1:50680 - "POST /node/6940925b575bc060ce160a90 HTTP/1.0" 202 Accepted 2025-12-17 22:57:49,422 - lava_callback - INFO - Received callback for node 6940925c575bc060ce160a93 INFO: 172.21.0.1:50694 - "POST /node/6940925c575bc060ce160a93 HTTP/1.0" 202 Accepted 2025-12-17 22:57:49,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb1575bc060ce160211 HTTP/1.1" 502 150 2025-12-17 22:57:49,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb1575bc060ce160211'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:49,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb5575bc060ce160217 2025-12-17 22:57:49,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb5575bc060ce160217 HTTP/1.1" 502 150 2025-12-17 22:57:49,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb5575bc060ce160217'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:49,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb7575bc060ce16021a 2025-12-17 22:57:49,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb7575bc060ce16021a HTTP/1.1" 502 150 2025-12-17 22:57:49,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb7575bc060ce16021a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:50,074 - lava_callback - INFO - Received callback for node 6940925e575bc060ce160a96 INFO: 172.21.0.1:52164 - "POST /node/6940925e575bc060ce160a96 HTTP/1.0" 202 Accepted 2025-12-17 22:57:50,707 - lava_callback - INFO - Received callback for node 69409260575bc060ce160a99 INFO: 172.21.0.1:52168 - "POST /node/69409260575bc060ce160a99 HTTP/1.0" 202 Accepted 2025-12-17 22:57:51,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940928f575bc060ce160aea 2025-12-17 22:57:51,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940928f575bc060ce160aea HTTP/1.1" 502 150 2025-12-17 22:57:51,383 - lava_callback - ERROR - Error processing callback for node 6940928f575bc060ce160aea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940928f575bc060ce160aea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940928f575bc060ce160aea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940928f575bc060ce160aea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:51,385 - lava_callback - INFO - Starting async job submit for node 6941017a460bc8e265116a25 2025-12-17 22:57:51,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:51,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941017a460bc8e265116a25 HTTP/1.1" 502 150 2025-12-17 22:57:51,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941017a460bc8e265116a25'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:51,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941017a460bc8e265116a25 2025-12-17 22:57:51,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941017a460bc8e265116a25 HTTP/1.1" 502 150 2025-12-17 22:57:51,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941017a460bc8e265116a25'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:51,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fba575bc060ce16021d 2025-12-17 22:57:51,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fba575bc060ce16021d HTTP/1.1" 502 150 2025-12-17 22:57:51,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fba575bc060ce16021d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:51,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fbc575bc060ce160220 2025-12-17 22:57:51,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fbc575bc060ce160220 HTTP/1.1" 502 150 2025-12-17 22:57:51,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fbc575bc060ce160220'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:52,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fbf575bc060ce160223 2025-12-17 22:57:52,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fbf575bc060ce160223 HTTP/1.1" 502 150 2025-12-17 22:57:52,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fbf575bc060ce160223'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:52,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fcc575bc060ce160246 2025-12-17 22:57:52,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fcc575bc060ce160246 HTTP/1.1" 502 150 2025-12-17 22:57:52,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fcc575bc060ce160246'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:52,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fc1575bc060ce160228 2025-12-17 22:57:52,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fc1575bc060ce160228 HTTP/1.1" 502 150 2025-12-17 22:57:52,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fc1575bc060ce160228'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:52,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fc7575bc060ce160240 2025-12-17 22:57:52,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fc7575bc060ce160240 HTTP/1.1" 502 150 2025-12-17 22:57:52,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fc7575bc060ce160240'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:53,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941017a460bc8e265116a25 2025-12-17 22:57:53,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941017a460bc8e265116a25 HTTP/1.1" 502 150 2025-12-17 22:57:53,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941017a460bc8e265116a25'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:53,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fab575bc060ce1601fc 2025-12-17 22:57:53,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fab575bc060ce1601fc HTTP/1.1" 502 150 2025-12-17 22:57:53,761 - lava_callback - ERROR - Error processing callback for node 69408fab575bc060ce1601fc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fab575bc060ce1601fc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fab575bc060ce1601fc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fab575bc060ce1601fc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:53,762 - lava_callback - INFO - Starting async job submit for node 69408fce575bc060ce160249 2025-12-17 22:57:53,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:53,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fce575bc060ce160249 HTTP/1.1" 502 150 2025-12-17 22:57:53,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fce575bc060ce160249'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:53,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fce575bc060ce160249 2025-12-17 22:57:53,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fce575bc060ce160249 HTTP/1.1" 502 150 2025-12-17 22:57:53,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fce575bc060ce160249'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:53,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fca575bc060ce160243 2025-12-17 22:57:53,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fca575bc060ce160243 HTTP/1.1" 502 150 2025-12-17 22:57:53,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fca575bc060ce160243'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:54,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e5575bc060ce161b9b 2025-12-17 22:57:54,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e5575bc060ce161b9b HTTP/1.1" 502 150 2025-12-17 22:57:54,695 - lava_callback - ERROR - Error processing callback for node 694098e5575bc060ce161b9b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098e5575bc060ce161b9b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098e5575bc060ce161b9b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098e5575bc060ce161b9b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:54,696 - lava_callback - INFO - Starting async job submit for node 69408fd7575bc060ce160277 2025-12-17 22:57:54,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:54,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd7575bc060ce160277 HTTP/1.1" 502 150 2025-12-17 22:57:54,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fd7575bc060ce160277'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:54,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fd7575bc060ce160277 2025-12-17 22:57:54,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd7575bc060ce160277 HTTP/1.1" 502 150 2025-12-17 22:57:54,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fd7575bc060ce160277'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:55,728 - lava_callback - INFO - Received callback for node 69409262575bc060ce160a9c INFO: 172.21.0.1:52182 - "POST /node/69409262575bc060ce160a9c HTTP/1.0" 202 Accepted 2025-12-17 22:57:55,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fce575bc060ce160249 2025-12-17 22:57:55,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fce575bc060ce160249 HTTP/1.1" 502 150 2025-12-17 22:57:55,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fce575bc060ce160249'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:56,398 - lava_callback - INFO - Received callback for node 69409265575bc060ce160a9f INFO: 172.21.0.1:52192 - "POST /node/69409265575bc060ce160a9f HTTP/1.0" 202 Accepted INFO: 172.21.0.1:52194 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:57:56,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fd7575bc060ce160277 2025-12-17 22:57:56,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd7575bc060ce160277 HTTP/1.1" 502 150 2025-12-17 22:57:56,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fd7575bc060ce160277'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:56,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408faf575bc060ce16020c 2025-12-17 22:57:56,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408faf575bc060ce16020c HTTP/1.1" 502 150 2025-12-17 22:57:56,865 - lava_callback - ERROR - Error processing callback for node 69408faf575bc060ce16020c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408faf575bc060ce16020c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408faf575bc060ce16020c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408faf575bc060ce16020c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:56,867 - lava_callback - INFO - Starting async job submit for node 69408fd4575bc060ce160255 2025-12-17 22:57:56,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:56,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd4575bc060ce160255 HTTP/1.1" 502 150 2025-12-17 22:57:56,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fd4575bc060ce160255'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:56,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fd4575bc060ce160255 2025-12-17 22:57:56,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd4575bc060ce160255 HTTP/1.1" 502 150 2025-12-17 22:57:56,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fd4575bc060ce160255'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:57,073 - lava_callback - INFO - Received callback for node 69409267575bc060ce160aa2 INFO: 172.21.0.1:52196 - "POST /node/69409267575bc060ce160aa2 HTTP/1.0" 202 Accepted 2025-12-17 22:57:57,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941017a460bc8e265116a25 2025-12-17 22:57:57,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941017a460bc8e265116a25 HTTP/1.1" 502 150 2025-12-17 22:57:57,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941017a460bc8e265116a25'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:57,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb1575bc060ce160211 2025-12-17 22:57:57,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb1575bc060ce160211 HTTP/1.1" 502 150 2025-12-17 22:57:57,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fb1575bc060ce160211'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:57,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e9d575bc060ce15fea3 2025-12-17 22:57:57,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e9d575bc060ce15fea3 HTTP/1.1" 502 150 2025-12-17 22:57:57,593 - lava_callback - ERROR - Error processing callback for node 69408e9d575bc060ce15fea3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e9d575bc060ce15fea3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e9d575bc060ce15fea3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e9d575bc060ce15fea3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:57:57,595 - lava_callback - INFO - Starting async job submit for node 69408fda575bc060ce160282 2025-12-17 22:57:57,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:57:57,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fda575bc060ce160282 HTTP/1.1" 502 150 2025-12-17 22:57:57,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fda575bc060ce160282'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:57,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fda575bc060ce160282 2025-12-17 22:57:57,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fda575bc060ce160282 HTTP/1.1" 502 150 2025-12-17 22:57:57,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fda575bc060ce160282'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:57,734 - lava_callback - INFO - Received callback for node 69409269575bc060ce160aa7 INFO: 172.21.0.1:52202 - "POST /node/69409269575bc060ce160aa7 HTTP/1.0" 202 Accepted 2025-12-17 22:57:58,407 - lava_callback - INFO - Received callback for node 6940926b575bc060ce160aaa INFO: 172.21.0.1:52214 - "POST /node/6940926b575bc060ce160aaa HTTP/1.0" 202 Accepted 2025-12-17 22:57:58,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fd4575bc060ce160255 2025-12-17 22:57:58,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd4575bc060ce160255 HTTP/1.1" 502 150 2025-12-17 22:57:58,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fd4575bc060ce160255'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:59,053 - lava_callback - INFO - Received callback for node 6940926c575bc060ce160aad INFO: 172.21.0.1:52216 - "POST /node/6940926c575bc060ce160aad HTTP/1.0" 202 Accepted 2025-12-17 22:57:59,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fda575bc060ce160282 2025-12-17 22:57:59,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fda575bc060ce160282 HTTP/1.1" 502 150 2025-12-17 22:57:59,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fda575bc060ce160282'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:57:59,735 - lava_callback - INFO - Received callback for node 6940926f575bc060ce160ab3 INFO: 172.21.0.1:52570 - "POST /node/6940926f575bc060ce160ab3 HTTP/1.0" 202 Accepted 2025-12-17 22:57:59,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fce575bc060ce160249 2025-12-17 22:57:59,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fce575bc060ce160249 HTTP/1.1" 502 150 2025-12-17 22:57:59,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fce575bc060ce160249'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:00,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fcc575bc060ce160246 2025-12-17 22:58:00,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fd7575bc060ce160277 2025-12-17 22:58:00,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd7575bc060ce160277 HTTP/1.1" 502 150 2025-12-17 22:58:00,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fd7575bc060ce160277'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52576 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:58:02,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fd4575bc060ce160255 2025-12-17 22:58:02,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd4575bc060ce160255 HTTP/1.1" 502 150 2025-12-17 22:58:02,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fd4575bc060ce160255'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:03,271 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb3575bc060ce160214 2025-12-17 22:58:03,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb3575bc060ce160214 HTTP/1.1" 502 150 2025-12-17 22:58:03,273 - lava_callback - ERROR - Error processing callback for node 69408fb3575bc060ce160214: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fb3575bc060ce160214 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fb3575bc060ce160214 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fb3575bc060ce160214 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:03,275 - lava_callback - INFO - Starting async job submit for node 69408fe1575bc060ce16028b 2025-12-17 22:58:03,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:03,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe1575bc060ce16028b HTTP/1.1" 502 150 2025-12-17 22:58:03,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe1575bc060ce16028b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:03,284 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe1575bc060ce16028b 2025-12-17 22:58:03,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe1575bc060ce16028b HTTP/1.1" 502 150 2025-12-17 22:58:03,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe1575bc060ce16028b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:03,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fda575bc060ce160282 2025-12-17 22:58:03,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fda575bc060ce160282 HTTP/1.1" 502 150 2025-12-17 22:58:03,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fda575bc060ce160282'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:05,285 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe1575bc060ce16028b 2025-12-17 22:58:05,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe1575bc060ce16028b HTTP/1.1" 502 150 2025-12-17 22:58:05,286 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe1575bc060ce16028b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:05,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941017a460bc8e265116a25 2025-12-17 22:58:05,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941017a460bc8e265116a25 HTTP/1.1" 502 150 2025-12-17 22:58:05,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941017a460bc8e265116a25'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:05,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb5575bc060ce160217 2025-12-17 22:58:05,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb5575bc060ce160217 HTTP/1.1" 502 150 2025-12-17 22:58:05,924 - lava_callback - ERROR - Error processing callback for node 69408fb5575bc060ce160217: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fb5575bc060ce160217 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fb5575bc060ce160217 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fb5575bc060ce160217 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:05,927 - lava_callback - INFO - Starting async job submit for node 69408fe3575bc060ce16028e 2025-12-17 22:58:05,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:05,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe3575bc060ce16028e HTTP/1.1" 502 150 2025-12-17 22:58:05,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe3575bc060ce16028e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:05,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe3575bc060ce16028e 2025-12-17 22:58:05,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe3575bc060ce16028e HTTP/1.1" 502 150 2025-12-17 22:58:05,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe3575bc060ce16028e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:05,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb7575bc060ce16021a 2025-12-17 22:58:05,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb7575bc060ce16021a HTTP/1.1" 502 150 2025-12-17 22:58:05,983 - lava_callback - ERROR - Error processing callback for node 69408fb7575bc060ce16021a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fb7575bc060ce16021a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fb7575bc060ce16021a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fb7575bc060ce16021a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:05,986 - lava_callback - INFO - Starting async job submit for node 69408fe5575bc060ce160291 2025-12-17 22:58:05,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:05,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe5575bc060ce160291 HTTP/1.1" 502 150 2025-12-17 22:58:05,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe5575bc060ce160291'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:05,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe5575bc060ce160291 2025-12-17 22:58:05,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe5575bc060ce160291 HTTP/1.1" 502 150 2025-12-17 22:58:05,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe5575bc060ce160291'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:06,119 - lava_callback - INFO - Received callback for node 69409270575bc060ce160ab6 INFO: 172.21.0.1:52580 - "POST /node/69409270575bc060ce160ab6 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:52596 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:58:06,789 - lava_callback - INFO - Received callback for node 69409272575bc060ce160ab9 INFO: 172.21.0.1:52606 - "POST /node/69409272575bc060ce160ab9 HTTP/1.0" 202 Accepted 2025-12-17 22:58:06,973 - lava_callback - INFO - Received callback for node 69409277575bc060ce160ac2 INFO: 172.21.0.1:52612 - "POST /node/69409277575bc060ce160ac2 HTTP/1.0" 202 Accepted 2025-12-17 22:58:07,447 - lava_callback - INFO - Received callback for node 69409274575bc060ce160abc INFO: 172.21.0.1:52614 - "POST /node/69409274575bc060ce160abc HTTP/1.0" 202 Accepted 2025-12-17 22:58:07,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fba575bc060ce16021d 2025-12-17 22:58:07,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fba575bc060ce16021d HTTP/1.1" 502 150 2025-12-17 22:58:07,572 - lava_callback - ERROR - Error processing callback for node 69408fba575bc060ce16021d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fba575bc060ce16021d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fba575bc060ce16021d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fba575bc060ce16021d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:07,574 - lava_callback - INFO - Starting async job submit for node 69408fe7575bc060ce160294 2025-12-17 22:58:07,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:07,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe7575bc060ce160294 HTTP/1.1" 502 150 2025-12-17 22:58:07,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe7575bc060ce160294'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:07,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe7575bc060ce160294 2025-12-17 22:58:07,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe7575bc060ce160294 HTTP/1.1" 502 150 2025-12-17 22:58:07,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe7575bc060ce160294'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:07,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fbc575bc060ce160220 2025-12-17 22:58:07,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fbc575bc060ce160220 HTTP/1.1" 502 150 2025-12-17 22:58:07,700 - lava_callback - ERROR - Error processing callback for node 69408fbc575bc060ce160220: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fbc575bc060ce160220 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fbc575bc060ce160220 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fbc575bc060ce160220 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:07,703 - lava_callback - INFO - Starting async job submit for node 69408fe9575bc060ce160299 2025-12-17 22:58:07,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:07,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe9575bc060ce160299 HTTP/1.1" 502 150 2025-12-17 22:58:07,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe9575bc060ce160299'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:07,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe9575bc060ce160299 2025-12-17 22:58:07,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe9575bc060ce160299 HTTP/1.1" 502 150 2025-12-17 22:58:07,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe9575bc060ce160299'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:07,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fce575bc060ce160249 2025-12-17 22:58:07,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fce575bc060ce160249 HTTP/1.1" 502 150 2025-12-17 22:58:07,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fce575bc060ce160249'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:07,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe3575bc060ce16028e 2025-12-17 22:58:07,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe3575bc060ce16028e HTTP/1.1" 502 150 2025-12-17 22:58:07,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe3575bc060ce16028e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:07,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe5575bc060ce160291 2025-12-17 22:58:07,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe5575bc060ce160291 HTTP/1.1" 502 150 2025-12-17 22:58:07,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe5575bc060ce160291'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:08,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fbf575bc060ce160223 2025-12-17 22:58:08,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fbf575bc060ce160223 HTTP/1.1" 502 150 2025-12-17 22:58:08,500 - lava_callback - ERROR - Error processing callback for node 69408fbf575bc060ce160223: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fbf575bc060ce160223 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fbf575bc060ce160223 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fbf575bc060ce160223 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:08,503 - lava_callback - INFO - Starting async job submit for node 69408fea575bc060ce16029f 2025-12-17 22:58:08,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:08,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fea575bc060ce16029f HTTP/1.1" 502 150 2025-12-17 22:58:08,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fea575bc060ce16029f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:08,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fea575bc060ce16029f 2025-12-17 22:58:08,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fea575bc060ce16029f HTTP/1.1" 502 150 2025-12-17 22:58:08,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fea575bc060ce16029f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:08,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fd7575bc060ce160277 2025-12-17 22:58:08,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd7575bc060ce160277 HTTP/1.1" 502 150 2025-12-17 22:58:08,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fd7575bc060ce160277'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:08,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fc1575bc060ce160228 2025-12-17 22:58:08,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fc1575bc060ce160228 HTTP/1.1" 502 150 2025-12-17 22:58:08,937 - lava_callback - ERROR - Error processing callback for node 69408fc1575bc060ce160228: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fc1575bc060ce160228 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fc1575bc060ce160228 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fc1575bc060ce160228 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:08,939 - lava_callback - INFO - Starting async job submit for node 69408fec575bc060ce1602a5 2025-12-17 22:58:08,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:08,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fec575bc060ce1602a5 HTTP/1.1" 502 150 2025-12-17 22:58:08,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fec575bc060ce1602a5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:08,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fec575bc060ce1602a5 2025-12-17 22:58:08,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fec575bc060ce1602a5 HTTP/1.1" 502 150 2025-12-17 22:58:08,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fec575bc060ce1602a5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:08,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fc7575bc060ce160240 2025-12-17 22:58:08,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fc7575bc060ce160240 HTTP/1.1" 502 150 2025-12-17 22:58:08,961 - lava_callback - ERROR - Error processing callback for node 69408fc7575bc060ce160240: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fc7575bc060ce160240 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fc7575bc060ce160240 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fc7575bc060ce160240 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:08,962 - lava_callback - INFO - Starting async job submit for node 69408ff0575bc060ce1602ba 2025-12-17 22:58:08,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:08,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff0575bc060ce1602ba HTTP/1.1" 502 150 2025-12-17 22:58:08,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff0575bc060ce1602ba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:08,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff0575bc060ce1602ba 2025-12-17 22:58:08,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff0575bc060ce1602ba HTTP/1.1" 502 150 2025-12-17 22:58:08,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff0575bc060ce1602ba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:09,286 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe1575bc060ce16028b 2025-12-17 22:58:09,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe1575bc060ce16028b HTTP/1.1" 502 150 2025-12-17 22:58:09,288 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe1575bc060ce16028b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:09,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe7575bc060ce160294 2025-12-17 22:58:09,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe7575bc060ce160294 HTTP/1.1" 502 150 2025-12-17 22:58:09,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe7575bc060ce160294'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:09,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe9575bc060ce160299 2025-12-17 22:58:09,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe9575bc060ce160299 HTTP/1.1" 502 150 2025-12-17 22:58:09,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe9575bc060ce160299'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:09,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fca575bc060ce160243 2025-12-17 22:58:09,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fca575bc060ce160243 HTTP/1.1" 502 150 2025-12-17 22:58:09,884 - lava_callback - ERROR - Error processing callback for node 69408fca575bc060ce160243: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fca575bc060ce160243 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fca575bc060ce160243 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fca575bc060ce160243 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:09,886 - lava_callback - INFO - Starting async job submit for node 69408ff1575bc060ce1602bd 2025-12-17 22:58:09,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:09,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff1575bc060ce1602bd HTTP/1.1" 502 150 2025-12-17 22:58:09,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff1575bc060ce1602bd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:09,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff1575bc060ce1602bd 2025-12-17 22:58:09,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff1575bc060ce1602bd HTTP/1.1" 502 150 2025-12-17 22:58:09,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff1575bc060ce1602bd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:10,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fea575bc060ce16029f 2025-12-17 22:58:10,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fea575bc060ce16029f HTTP/1.1" 502 150 2025-12-17 22:58:10,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fea575bc060ce16029f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:10,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fd4575bc060ce160255 2025-12-17 22:58:10,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd4575bc060ce160255 HTTP/1.1" 502 150 2025-12-17 22:58:10,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fd4575bc060ce160255'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:10,905 - lava_callback - INFO - Received callback for node 69409276575bc060ce160abf INFO: 172.21.0.1:51592 - "POST /node/69409276575bc060ce160abf HTTP/1.0" 202 Accepted 2025-12-17 22:58:10,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fec575bc060ce1602a5 2025-12-17 22:58:10,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fec575bc060ce1602a5 HTTP/1.1" 502 150 2025-12-17 22:58:10,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fec575bc060ce1602a5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:10,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff0575bc060ce1602ba 2025-12-17 22:58:10,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff0575bc060ce1602ba HTTP/1.1" 502 150 2025-12-17 22:58:10,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff0575bc060ce1602ba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:11,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fda575bc060ce160282 2025-12-17 22:58:11,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fda575bc060ce160282 HTTP/1.1" 502 150 2025-12-17 22:58:11,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fda575bc060ce160282'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:11,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff1575bc060ce1602bd 2025-12-17 22:58:11,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff1575bc060ce1602bd HTTP/1.1" 502 150 2025-12-17 22:58:11,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff1575bc060ce1602bd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:11,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe3575bc060ce16028e 2025-12-17 22:58:11,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe3575bc060ce16028e HTTP/1.1" 502 150 2025-12-17 22:58:11,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe3575bc060ce16028e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:11,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe5575bc060ce160291 2025-12-17 22:58:11,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe5575bc060ce160291 HTTP/1.1" 502 150 2025-12-17 22:58:11,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe5575bc060ce160291'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:13,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fb1575bc060ce160211 2025-12-17 22:58:13,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe7575bc060ce160294 2025-12-17 22:58:13,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe7575bc060ce160294 HTTP/1.1" 502 150 2025-12-17 22:58:13,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe7575bc060ce160294'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:13,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe9575bc060ce160299 2025-12-17 22:58:13,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe9575bc060ce160299 HTTP/1.1" 502 150 2025-12-17 22:58:13,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe9575bc060ce160299'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:14,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fea575bc060ce16029f 2025-12-17 22:58:14,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fea575bc060ce16029f HTTP/1.1" 502 150 2025-12-17 22:58:14,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fea575bc060ce16029f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:14,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fec575bc060ce1602a5 2025-12-17 22:58:14,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fec575bc060ce1602a5 HTTP/1.1" 502 150 2025-12-17 22:58:14,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fec575bc060ce1602a5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:14,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff0575bc060ce1602ba 2025-12-17 22:58:14,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff0575bc060ce1602ba HTTP/1.1" 502 150 2025-12-17 22:58:14,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff0575bc060ce1602ba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51606 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:58:15,661 - lava_callback - INFO - Received callback for node 69409279575bc060ce160ac5 INFO: 172.21.0.1:51614 - "POST /node/69409279575bc060ce160ac5 HTTP/1.0" 202 Accepted 2025-12-17 22:58:15,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fcc575bc060ce160246 HTTP/1.1" 502 150 2025-12-17 22:58:15,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fcc575bc060ce160246'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:15,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff1575bc060ce1602bd 2025-12-17 22:58:15,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff1575bc060ce1602bd HTTP/1.1" 502 150 2025-12-17 22:58:15,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff1575bc060ce1602bd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:16,332 - lava_callback - INFO - Received callback for node 6940927a575bc060ce160ac8 INFO: 172.21.0.1:51618 - "POST /node/6940927a575bc060ce160ac8 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:51632 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:58:17,288 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe1575bc060ce16028b 2025-12-17 22:58:17,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe1575bc060ce16028b HTTP/1.1" 502 150 2025-12-17 22:58:17,289 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe1575bc060ce16028b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:19,500 - lava_callback - INFO - Received callback for node 6940988e575bc060ce161ae3 INFO: 172.21.0.1:51642 - "POST /node/6940988e575bc060ce161ae3 HTTP/1.0" 202 Accepted 2025-12-17 22:58:19,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe3575bc060ce16028e 2025-12-17 22:58:19,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe3575bc060ce16028e HTTP/1.1" 502 150 2025-12-17 22:58:19,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe3575bc060ce16028e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:19,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe5575bc060ce160291 2025-12-17 22:58:20,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe5575bc060ce160291 HTTP/1.1" 502 150 2025-12-17 22:58:20,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe5575bc060ce160291'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:21,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941017a460bc8e265116a25 2025-12-17 22:58:21,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941017a460bc8e265116a25 HTTP/1.1" 502 150 2025-12-17 22:58:21,437 - lava_callback - ERROR - Error processing callback for node 6941017a460bc8e265116a25: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941017a460bc8e265116a25 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941017a460bc8e265116a25 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941017a460bc8e265116a25 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:21,439 - lava_callback - INFO - Starting async job submit for node 69408fed575bc060ce1602ac 2025-12-17 22:58:21,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:21,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fed575bc060ce1602ac HTTP/1.1" 502 150 2025-12-17 22:58:21,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fed575bc060ce1602ac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:21,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fed575bc060ce1602ac 2025-12-17 22:58:21,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fed575bc060ce1602ac HTTP/1.1" 502 150 2025-12-17 22:58:21,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fed575bc060ce1602ac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:21,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe7575bc060ce160294 2025-12-17 22:58:21,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe7575bc060ce160294 HTTP/1.1" 502 150 2025-12-17 22:58:21,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe7575bc060ce160294'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:21,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe9575bc060ce160299 2025-12-17 22:58:21,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe9575bc060ce160299 HTTP/1.1" 502 150 2025-12-17 22:58:21,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fe9575bc060ce160299'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:22,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fea575bc060ce16029f 2025-12-17 22:58:22,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fea575bc060ce16029f HTTP/1.1" 502 150 2025-12-17 22:58:22,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fea575bc060ce16029f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:22,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fec575bc060ce1602a5 2025-12-17 22:58:22,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fec575bc060ce1602a5 HTTP/1.1" 502 150 2025-12-17 22:58:22,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fec575bc060ce1602a5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:22,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff0575bc060ce1602ba 2025-12-17 22:58:22,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff0575bc060ce1602ba HTTP/1.1" 502 150 2025-12-17 22:58:22,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff0575bc060ce1602ba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:23,447 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fed575bc060ce1602ac 2025-12-17 22:58:23,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fed575bc060ce1602ac HTTP/1.1" 502 150 2025-12-17 22:58:23,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fed575bc060ce1602ac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:23,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fce575bc060ce160249 2025-12-17 22:58:23,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fce575bc060ce160249 HTTP/1.1" 502 150 2025-12-17 22:58:23,781 - lava_callback - ERROR - Error processing callback for node 69408fce575bc060ce160249: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fce575bc060ce160249 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fce575bc060ce160249 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fce575bc060ce160249 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:23,783 - lava_callback - INFO - Starting async job submit for node 69408fee575bc060ce1602b7 2025-12-17 22:58:23,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:23,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fee575bc060ce1602b7 HTTP/1.1" 502 150 2025-12-17 22:58:23,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fee575bc060ce1602b7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:23,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fee575bc060ce1602b7 2025-12-17 22:58:23,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fee575bc060ce1602b7 HTTP/1.1" 502 150 2025-12-17 22:58:23,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fee575bc060ce1602b7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:23,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff1575bc060ce1602bd 2025-12-17 22:58:23,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff1575bc060ce1602bd HTTP/1.1" 502 150 2025-12-17 22:58:23,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff1575bc060ce1602bd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:24,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fd7575bc060ce160277 2025-12-17 22:58:24,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd7575bc060ce160277 HTTP/1.1" 502 150 2025-12-17 22:58:24,712 - lava_callback - ERROR - Error processing callback for node 69408fd7575bc060ce160277: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fd7575bc060ce160277 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fd7575bc060ce160277 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fd7575bc060ce160277 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:24,714 - lava_callback - INFO - Starting async job submit for node 69408ff7575bc060ce1602cc 2025-12-17 22:58:24,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:24,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff7575bc060ce1602cc HTTP/1.1" 502 150 2025-12-17 22:58:24,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff7575bc060ce1602cc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:24,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff7575bc060ce1602cc 2025-12-17 22:58:24,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff7575bc060ce1602cc HTTP/1.1" 502 150 2025-12-17 22:58:24,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff7575bc060ce1602cc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:25,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fee575bc060ce1602b7 2025-12-17 22:58:25,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fee575bc060ce1602b7 HTTP/1.1" 502 150 2025-12-17 22:58:25,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fee575bc060ce1602b7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50828 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:58:26,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff7575bc060ce1602cc 2025-12-17 22:58:26,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff7575bc060ce1602cc HTTP/1.1" 502 150 2025-12-17 22:58:26,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff7575bc060ce1602cc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:26,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fd4575bc060ce160255 2025-12-17 22:58:26,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fd4575bc060ce160255 HTTP/1.1" 502 150 2025-12-17 22:58:26,905 - lava_callback - ERROR - Error processing callback for node 69408fd4575bc060ce160255: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fd4575bc060ce160255 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fd4575bc060ce160255 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fd4575bc060ce160255 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:26,907 - lava_callback - INFO - Starting async job submit for node 69408ffb575bc060ce1602d2 2025-12-17 22:58:26,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:26,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffb575bc060ce1602d2 HTTP/1.1" 502 150 2025-12-17 22:58:26,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ffb575bc060ce1602d2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:26,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ffb575bc060ce1602d2 2025-12-17 22:58:26,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffb575bc060ce1602d2 HTTP/1.1" 502 150 2025-12-17 22:58:26,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ffb575bc060ce1602d2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:27,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fed575bc060ce1602ac 2025-12-17 22:58:27,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fed575bc060ce1602ac HTTP/1.1" 502 150 2025-12-17 22:58:27,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fed575bc060ce1602ac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:27,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fda575bc060ce160282 2025-12-17 22:58:27,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fda575bc060ce160282 HTTP/1.1" 502 150 2025-12-17 22:58:27,610 - lava_callback - ERROR - Error processing callback for node 69408fda575bc060ce160282: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fda575bc060ce160282 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fda575bc060ce160282 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fda575bc060ce160282 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:27,613 - lava_callback - INFO - Starting async job submit for node 69408ff2575bc060ce1602c0 2025-12-17 22:58:27,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:27,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff2575bc060ce1602c0 HTTP/1.1" 502 150 2025-12-17 22:58:27,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff2575bc060ce1602c0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:27,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff2575bc060ce1602c0 2025-12-17 22:58:27,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff2575bc060ce1602c0 HTTP/1.1" 502 150 2025-12-17 22:58:27,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff2575bc060ce1602c0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:28,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fb1575bc060ce160211 HTTP/1.1" 502 150 2025-12-17 22:58:28,916 - lava_callback - ERROR - Error processing callback for node 69408fb1575bc060ce160211: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fb1575bc060ce160211 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fb1575bc060ce160211 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fb1575bc060ce160211 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:28,919 - lava_callback - INFO - Starting async job submit for node 69408ff4575bc060ce1602c3 2025-12-17 22:58:28,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:28,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ffb575bc060ce1602d2 2025-12-17 22:58:28,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffb575bc060ce1602d2 HTTP/1.1" 502 150 2025-12-17 22:58:28,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ffb575bc060ce1602d2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:28,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff4575bc060ce1602c3 HTTP/1.1" 502 150 2025-12-17 22:58:28,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff4575bc060ce1602c3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:28,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff4575bc060ce1602c3 2025-12-17 22:58:28,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff4575bc060ce1602c3 HTTP/1.1" 502 150 2025-12-17 22:58:28,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff4575bc060ce1602c3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:29,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff2575bc060ce1602c0 2025-12-17 22:58:29,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff2575bc060ce1602c0 HTTP/1.1" 502 150 2025-12-17 22:58:29,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff2575bc060ce1602c0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:29,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fee575bc060ce1602b7 2025-12-17 22:58:29,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fee575bc060ce1602b7 HTTP/1.1" 502 150 2025-12-17 22:58:29,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fee575bc060ce1602b7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:30,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff7575bc060ce1602cc 2025-12-17 22:58:30,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff7575bc060ce1602cc HTTP/1.1" 502 150 2025-12-17 22:58:30,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff7575bc060ce1602cc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:30,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff4575bc060ce1602c3 2025-12-17 22:58:30,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff4575bc060ce1602c3 HTTP/1.1" 502 150 2025-12-17 22:58:30,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff4575bc060ce1602c3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:31,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fcc575bc060ce160246 2025-12-17 22:58:31,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fcc575bc060ce160246 HTTP/1.1" 502 150 2025-12-17 22:58:31,858 - lava_callback - ERROR - Error processing callback for node 69408fcc575bc060ce160246: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fcc575bc060ce160246 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fcc575bc060ce160246 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fcc575bc060ce160246 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:31,860 - lava_callback - INFO - Starting async job submit for node 69408ff5575bc060ce1602c6 2025-12-17 22:58:31,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:31,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff5575bc060ce1602c6 HTTP/1.1" 502 150 2025-12-17 22:58:31,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff5575bc060ce1602c6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:31,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff5575bc060ce1602c6 2025-12-17 22:58:31,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff5575bc060ce1602c6 HTTP/1.1" 502 150 2025-12-17 22:58:31,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff5575bc060ce1602c6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:32,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ffb575bc060ce1602d2 2025-12-17 22:58:32,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffb575bc060ce1602d2 HTTP/1.1" 502 150 2025-12-17 22:58:32,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ffb575bc060ce1602d2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:33,290 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe1575bc060ce16028b 2025-12-17 22:58:33,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe1575bc060ce16028b HTTP/1.1" 502 150 2025-12-17 22:58:33,291 - lava_callback - ERROR - Error processing callback for node 69408fe1575bc060ce16028b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fe1575bc060ce16028b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fe1575bc060ce16028b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fe1575bc060ce16028b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:33,293 - lava_callback - INFO - Starting async job submit for node 69408ff6575bc060ce1602c9 2025-12-17 22:58:33,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:33,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff6575bc060ce1602c9 HTTP/1.1" 502 150 2025-12-17 22:58:33,302 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff6575bc060ce1602c9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:33,302 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff6575bc060ce1602c9 2025-12-17 22:58:33,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff6575bc060ce1602c9 HTTP/1.1" 502 150 2025-12-17 22:58:33,302 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff6575bc060ce1602c9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:33,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff2575bc060ce1602c0 2025-12-17 22:58:33,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff2575bc060ce1602c0 HTTP/1.1" 502 150 2025-12-17 22:58:33,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff2575bc060ce1602c0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:33,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff5575bc060ce1602c6 2025-12-17 22:58:33,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff5575bc060ce1602c6 HTTP/1.1" 502 150 2025-12-17 22:58:33,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff5575bc060ce1602c6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:34,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff4575bc060ce1602c3 2025-12-17 22:58:34,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff4575bc060ce1602c3 HTTP/1.1" 502 150 2025-12-17 22:58:34,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff4575bc060ce1602c3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:35,303 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff6575bc060ce1602c9 2025-12-17 22:58:35,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff6575bc060ce1602c9 HTTP/1.1" 502 150 2025-12-17 22:58:35,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff6575bc060ce1602c9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:35,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fed575bc060ce1602ac 2025-12-17 22:58:35,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fed575bc060ce1602ac HTTP/1.1" 502 150 2025-12-17 22:58:35,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fed575bc060ce1602ac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:35,713 - lava_callback - INFO - Received callback for node 69409299575bc060ce160af9 INFO: 172.21.0.1:34140 - "POST /node/69409299575bc060ce160af9 HTTP/1.0" 202 Accepted 2025-12-17 22:58:35,845 - lava_callback - INFO - Received callback for node 69409373575bc060ce160d87 INFO: 172.21.0.1:34146 - "POST /node/69409373575bc060ce160d87 HTTP/1.0" 202 Accepted 2025-12-17 22:58:35,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe3575bc060ce16028e 2025-12-17 22:58:35,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe3575bc060ce16028e HTTP/1.1" 502 150 2025-12-17 22:58:35,963 - lava_callback - ERROR - Error processing callback for node 69408fe3575bc060ce16028e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fe3575bc060ce16028e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fe3575bc060ce16028e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fe3575bc060ce16028e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:35,964 - lava_callback - INFO - Starting async job submit for node 69408ffa575bc060ce1602cf 2025-12-17 22:58:35,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:35,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffa575bc060ce1602cf HTTP/1.1" 502 150 2025-12-17 22:58:35,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ffa575bc060ce1602cf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:35,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ffa575bc060ce1602cf 2025-12-17 22:58:35,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffa575bc060ce1602cf HTTP/1.1" 502 150 2025-12-17 22:58:35,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ffa575bc060ce1602cf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:36,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe5575bc060ce160291 2025-12-17 22:58:36,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe5575bc060ce160291 HTTP/1.1" 502 150 2025-12-17 22:58:36,005 - lava_callback - ERROR - Error processing callback for node 69408fe5575bc060ce160291: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fe5575bc060ce160291 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fe5575bc060ce160291 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fe5575bc060ce160291 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:36,007 - lava_callback - INFO - Starting async job submit for node 69408ffd575bc060ce1602db 2025-12-17 22:58:36,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:36,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffd575bc060ce1602db HTTP/1.1" 502 150 2025-12-17 22:58:36,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ffd575bc060ce1602db'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:36,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ffd575bc060ce1602db 2025-12-17 22:58:36,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffd575bc060ce1602db HTTP/1.1" 502 150 2025-12-17 22:58:36,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ffd575bc060ce1602db'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34162 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:58:37,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe7575bc060ce160294 2025-12-17 22:58:37,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe7575bc060ce160294 HTTP/1.1" 502 150 2025-12-17 22:58:37,593 - lava_callback - ERROR - Error processing callback for node 69408fe7575bc060ce160294: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fe7575bc060ce160294 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fe7575bc060ce160294 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fe7575bc060ce160294 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:37,595 - lava_callback - INFO - Starting async job submit for node 69409004575bc060ce1602e0 2025-12-17 22:58:37,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:37,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409004575bc060ce1602e0 HTTP/1.1" 502 150 2025-12-17 22:58:37,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409004575bc060ce1602e0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:37,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409004575bc060ce1602e0 2025-12-17 22:58:37,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409004575bc060ce1602e0 HTTP/1.1" 502 150 2025-12-17 22:58:37,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409004575bc060ce1602e0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:37,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fe9575bc060ce160299 2025-12-17 22:58:37,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fe9575bc060ce160299 HTTP/1.1" 502 150 2025-12-17 22:58:37,716 - lava_callback - ERROR - Error processing callback for node 69408fe9575bc060ce160299: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fe9575bc060ce160299 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fe9575bc060ce160299 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fe9575bc060ce160299 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:37,718 - lava_callback - INFO - Starting async job submit for node 69409006575bc060ce1602ee 2025-12-17 22:58:37,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:37,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409006575bc060ce1602ee HTTP/1.1" 502 150 2025-12-17 22:58:37,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409006575bc060ce1602ee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:37,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409006575bc060ce1602ee 2025-12-17 22:58:37,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409006575bc060ce1602ee HTTP/1.1" 502 150 2025-12-17 22:58:37,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409006575bc060ce1602ee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:37,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fee575bc060ce1602b7 2025-12-17 22:58:37,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fee575bc060ce1602b7 HTTP/1.1" 502 150 2025-12-17 22:58:37,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408fee575bc060ce1602b7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:37,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff5575bc060ce1602c6 2025-12-17 22:58:37,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff5575bc060ce1602c6 HTTP/1.1" 502 150 2025-12-17 22:58:37,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff5575bc060ce1602c6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:37,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ffa575bc060ce1602cf 2025-12-17 22:58:37,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffa575bc060ce1602cf HTTP/1.1" 502 150 2025-12-17 22:58:37,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ffa575bc060ce1602cf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:38,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ffd575bc060ce1602db 2025-12-17 22:58:38,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffd575bc060ce1602db HTTP/1.1" 502 150 2025-12-17 22:58:38,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ffd575bc060ce1602db'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:38,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fea575bc060ce16029f 2025-12-17 22:58:38,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fea575bc060ce16029f HTTP/1.1" 502 150 2025-12-17 22:58:38,524 - lava_callback - ERROR - Error processing callback for node 69408fea575bc060ce16029f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fea575bc060ce16029f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fea575bc060ce16029f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fea575bc060ce16029f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:38,526 - lava_callback - INFO - Starting async job submit for node 69409008575bc060ce1602f1 2025-12-17 22:58:38,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:38,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409008575bc060ce1602f1 HTTP/1.1" 502 150 2025-12-17 22:58:38,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409008575bc060ce1602f1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:38,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409008575bc060ce1602f1 2025-12-17 22:58:38,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409008575bc060ce1602f1 HTTP/1.1" 502 150 2025-12-17 22:58:38,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409008575bc060ce1602f1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:38,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff7575bc060ce1602cc 2025-12-17 22:58:38,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff7575bc060ce1602cc HTTP/1.1" 502 150 2025-12-17 22:58:38,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff7575bc060ce1602cc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:38,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fec575bc060ce1602a5 2025-12-17 22:58:38,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fec575bc060ce1602a5 HTTP/1.1" 502 150 2025-12-17 22:58:38,952 - lava_callback - ERROR - Error processing callback for node 69408fec575bc060ce1602a5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fec575bc060ce1602a5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fec575bc060ce1602a5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fec575bc060ce1602a5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:38,955 - lava_callback - INFO - Starting async job submit for node 6940900a575bc060ce1602f4 2025-12-17 22:58:38,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:38,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900a575bc060ce1602f4 HTTP/1.1" 502 150 2025-12-17 22:58:38,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940900a575bc060ce1602f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:38,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940900a575bc060ce1602f4 2025-12-17 22:58:38,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900a575bc060ce1602f4 HTTP/1.1" 502 150 2025-12-17 22:58:38,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940900a575bc060ce1602f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:38,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff0575bc060ce1602ba 2025-12-17 22:58:38,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff0575bc060ce1602ba HTTP/1.1" 502 150 2025-12-17 22:58:38,979 - lava_callback - ERROR - Error processing callback for node 69408ff0575bc060ce1602ba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff0575bc060ce1602ba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff0575bc060ce1602ba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff0575bc060ce1602ba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:38,982 - lava_callback - INFO - Starting async job submit for node 6940900c575bc060ce1602f7 2025-12-17 22:58:38,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:38,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900c575bc060ce1602f7 HTTP/1.1" 502 150 2025-12-17 22:58:38,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940900c575bc060ce1602f7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:38,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940900c575bc060ce1602f7 2025-12-17 22:58:38,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900c575bc060ce1602f7 HTTP/1.1" 502 150 2025-12-17 22:58:38,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940900c575bc060ce1602f7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:39,304 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff6575bc060ce1602c9 2025-12-17 22:58:39,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409004575bc060ce1602e0 2025-12-17 22:58:39,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409004575bc060ce1602e0 HTTP/1.1" 502 150 2025-12-17 22:58:39,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409004575bc060ce1602e0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:39,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409006575bc060ce1602ee 2025-12-17 22:58:39,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409006575bc060ce1602ee HTTP/1.1" 502 150 2025-12-17 22:58:39,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409006575bc060ce1602ee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:39,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff1575bc060ce1602bd 2025-12-17 22:58:39,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff1575bc060ce1602bd HTTP/1.1" 502 150 2025-12-17 22:58:39,906 - lava_callback - ERROR - Error processing callback for node 69408ff1575bc060ce1602bd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff1575bc060ce1602bd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff1575bc060ce1602bd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff1575bc060ce1602bd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:39,908 - lava_callback - INFO - Starting async job submit for node 6940900e575bc060ce1602fa 2025-12-17 22:58:39,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:39,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900e575bc060ce1602fa HTTP/1.1" 502 150 2025-12-17 22:58:39,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940900e575bc060ce1602fa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:39,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940900e575bc060ce1602fa 2025-12-17 22:58:39,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900e575bc060ce1602fa HTTP/1.1" 502 150 2025-12-17 22:58:39,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940900e575bc060ce1602fa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:40,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409008575bc060ce1602f1 2025-12-17 22:58:40,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409008575bc060ce1602f1 HTTP/1.1" 502 150 2025-12-17 22:58:40,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409008575bc060ce1602f1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:40,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ffb575bc060ce1602d2 2025-12-17 22:58:40,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffb575bc060ce1602d2 HTTP/1.1" 502 150 2025-12-17 22:58:40,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ffb575bc060ce1602d2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:40,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940900a575bc060ce1602f4 2025-12-17 22:58:40,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900a575bc060ce1602f4 HTTP/1.1" 502 150 2025-12-17 22:58:40,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940900a575bc060ce1602f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:40,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940900c575bc060ce1602f7 2025-12-17 22:58:40,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900c575bc060ce1602f7 HTTP/1.1" 502 150 2025-12-17 22:58:40,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940900c575bc060ce1602f7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:41,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff2575bc060ce1602c0 2025-12-17 22:58:41,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff2575bc060ce1602c0 HTTP/1.1" 502 150 2025-12-17 22:58:41,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff2575bc060ce1602c0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:41,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940900e575bc060ce1602fa 2025-12-17 22:58:41,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900e575bc060ce1602fa HTTP/1.1" 502 150 2025-12-17 22:58:41,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940900e575bc060ce1602fa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:41,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ffa575bc060ce1602cf 2025-12-17 22:58:41,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffa575bc060ce1602cf HTTP/1.1" 502 150 2025-12-17 22:58:41,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ffa575bc060ce1602cf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:42,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ffd575bc060ce1602db 2025-12-17 22:58:42,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffd575bc060ce1602db HTTP/1.1" 502 150 2025-12-17 22:58:42,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ffd575bc060ce1602db'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:42,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff4575bc060ce1602c3 2025-12-17 22:58:42,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff4575bc060ce1602c3 HTTP/1.1" 502 150 2025-12-17 22:58:42,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff4575bc060ce1602c3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:43,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409004575bc060ce1602e0 2025-12-17 22:58:43,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409004575bc060ce1602e0 HTTP/1.1" 502 150 2025-12-17 22:58:43,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409004575bc060ce1602e0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:43,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409006575bc060ce1602ee 2025-12-17 22:58:43,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409006575bc060ce1602ee HTTP/1.1" 502 150 2025-12-17 22:58:43,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409006575bc060ce1602ee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:44,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409008575bc060ce1602f1 2025-12-17 22:58:44,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409008575bc060ce1602f1 HTTP/1.1" 502 150 2025-12-17 22:58:44,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409008575bc060ce1602f1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:44,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940900a575bc060ce1602f4 2025-12-17 22:58:44,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900a575bc060ce1602f4 HTTP/1.1" 502 150 2025-12-17 22:58:44,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940900a575bc060ce1602f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:44,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940900c575bc060ce1602f7 2025-12-17 22:58:44,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900c575bc060ce1602f7 HTTP/1.1" 502 150 2025-12-17 22:58:44,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940900c575bc060ce1602f7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:45,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff5575bc060ce1602c6 2025-12-17 22:58:45,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff5575bc060ce1602c6 HTTP/1.1" 502 150 2025-12-17 22:58:45,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff5575bc060ce1602c6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:45,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940900e575bc060ce1602fa 2025-12-17 22:58:45,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900e575bc060ce1602fa HTTP/1.1" 502 150 2025-12-17 22:58:45,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940900e575bc060ce1602fa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36564 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:58:47,237 - lava_callback - INFO - Received callback for node 694092a2575bc060ce160b09 INFO: 172.21.0.1:36574 - "POST /node/694092a2575bc060ce160b09 HTTP/1.0" 202 Accepted 2025-12-17 22:58:49,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ffa575bc060ce1602cf 2025-12-17 22:58:49,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffa575bc060ce1602cf HTTP/1.1" 502 150 2025-12-17 22:58:49,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ffa575bc060ce1602cf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:50,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ffd575bc060ce1602db 2025-12-17 22:58:51,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fed575bc060ce1602ac 2025-12-17 22:58:51,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fed575bc060ce1602ac HTTP/1.1" 502 150 2025-12-17 22:58:51,453 - lava_callback - ERROR - Error processing callback for node 69408fed575bc060ce1602ac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fed575bc060ce1602ac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fed575bc060ce1602ac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fed575bc060ce1602ac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:51,455 - lava_callback - INFO - Starting async job submit for node 69409988575bc060ce161d2d 2025-12-17 22:58:51,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:51,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409988575bc060ce161d2d HTTP/1.1" 502 150 2025-12-17 22:58:51,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409988575bc060ce161d2d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:51,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409988575bc060ce161d2d 2025-12-17 22:58:51,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409988575bc060ce161d2d HTTP/1.1" 502 150 2025-12-17 22:58:51,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409988575bc060ce161d2d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:51,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409004575bc060ce1602e0 2025-12-17 22:58:51,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409004575bc060ce1602e0 HTTP/1.1" 502 150 2025-12-17 22:58:51,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409004575bc060ce1602e0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:51,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409006575bc060ce1602ee 2025-12-17 22:58:51,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409006575bc060ce1602ee HTTP/1.1" 502 150 2025-12-17 22:58:51,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409006575bc060ce1602ee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:52,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409008575bc060ce1602f1 2025-12-17 22:58:52,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940900a575bc060ce1602f4 2025-12-17 22:58:52,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900a575bc060ce1602f4 HTTP/1.1" 502 150 2025-12-17 22:58:52,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940900a575bc060ce1602f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:52,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940900c575bc060ce1602f7 2025-12-17 22:58:53,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900c575bc060ce1602f7 HTTP/1.1" 502 150 2025-12-17 22:58:53,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940900c575bc060ce1602f7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:53,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409988575bc060ce161d2d 2025-12-17 22:58:53,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409988575bc060ce161d2d HTTP/1.1" 502 150 2025-12-17 22:58:53,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409988575bc060ce161d2d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:53,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408fee575bc060ce1602b7 2025-12-17 22:58:53,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408fee575bc060ce1602b7 HTTP/1.1" 502 150 2025-12-17 22:58:53,804 - lava_callback - ERROR - Error processing callback for node 69408fee575bc060ce1602b7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fee575bc060ce1602b7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fee575bc060ce1602b7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408fee575bc060ce1602b7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:53,807 - lava_callback - INFO - Starting async job submit for node 694092d3575bc060ce160b88 2025-12-17 22:58:53,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:53,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d3575bc060ce160b88 HTTP/1.1" 502 150 2025-12-17 22:58:53,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d3575bc060ce160b88'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:53,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d3575bc060ce160b88 2025-12-17 22:58:53,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d3575bc060ce160b88 HTTP/1.1" 502 150 2025-12-17 22:58:53,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d3575bc060ce160b88'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:53,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940900e575bc060ce1602fa 2025-12-17 22:58:53,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900e575bc060ce1602fa HTTP/1.1" 502 150 2025-12-17 22:58:53,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940900e575bc060ce1602fa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:54,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff7575bc060ce1602cc 2025-12-17 22:58:54,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff7575bc060ce1602cc HTTP/1.1" 502 150 2025-12-17 22:58:54,727 - lava_callback - ERROR - Error processing callback for node 69408ff7575bc060ce1602cc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff7575bc060ce1602cc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff7575bc060ce1602cc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff7575bc060ce1602cc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:54,729 - lava_callback - INFO - Starting async job submit for node 69409017575bc060ce16032a 2025-12-17 22:58:54,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:54,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409017575bc060ce16032a HTTP/1.1" 502 150 2025-12-17 22:58:54,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409017575bc060ce16032a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:54,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409017575bc060ce16032a 2025-12-17 22:58:54,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409017575bc060ce16032a HTTP/1.1" 502 150 2025-12-17 22:58:54,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409017575bc060ce16032a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:54,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff6575bc060ce1602c9 HTTP/1.1" 502 150 2025-12-17 22:58:54,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff6575bc060ce1602c9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:55,753 - lava_callback - INFO - Received callback for node 6940929b575bc060ce160afc INFO: 172.21.0.1:33012 - "POST /node/6940929b575bc060ce160afc HTTP/1.0" 202 Accepted 2025-12-17 22:58:55,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d3575bc060ce160b88 2025-12-17 22:58:55,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d3575bc060ce160b88 HTTP/1.1" 502 150 2025-12-17 22:58:55,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d3575bc060ce160b88'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:56,418 - lava_callback - INFO - Received callback for node 6940929d575bc060ce160aff INFO: 172.21.0.1:33014 - "POST /node/6940929d575bc060ce160aff HTTP/1.0" 202 Accepted INFO: 172.21.0.1:33030 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:58:56,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409017575bc060ce16032a 2025-12-17 22:58:56,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409017575bc060ce16032a HTTP/1.1" 502 150 2025-12-17 22:58:56,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409017575bc060ce16032a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:56,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ffb575bc060ce1602d2 2025-12-17 22:58:56,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffb575bc060ce1602d2 HTTP/1.1" 502 150 2025-12-17 22:58:56,931 - lava_callback - ERROR - Error processing callback for node 69408ffb575bc060ce1602d2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ffb575bc060ce1602d2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ffb575bc060ce1602d2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ffb575bc060ce1602d2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:56,933 - lava_callback - INFO - Starting async job submit for node 69409010575bc060ce160301 2025-12-17 22:58:56,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:56,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409010575bc060ce160301 HTTP/1.1" 502 150 2025-12-17 22:58:56,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409010575bc060ce160301'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:56,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409010575bc060ce160301 2025-12-17 22:58:56,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409010575bc060ce160301 HTTP/1.1" 502 150 2025-12-17 22:58:56,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409010575bc060ce160301'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:57,086 - lava_callback - INFO - Received callback for node 6940929f575bc060ce160b06 INFO: 172.21.0.1:33036 - "POST /node/6940929f575bc060ce160b06 HTTP/1.0" 202 Accepted 2025-12-17 22:58:57,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409988575bc060ce161d2d 2025-12-17 22:58:57,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409988575bc060ce161d2d HTTP/1.1" 502 150 2025-12-17 22:58:57,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409988575bc060ce161d2d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:57,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff2575bc060ce1602c0 2025-12-17 22:58:57,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff2575bc060ce1602c0 HTTP/1.1" 502 150 2025-12-17 22:58:57,628 - lava_callback - ERROR - Error processing callback for node 69408ff2575bc060ce1602c0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff2575bc060ce1602c0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff2575bc060ce1602c0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff2575bc060ce1602c0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:57,629 - lava_callback - INFO - Starting async job submit for node 69409013575bc060ce160320 2025-12-17 22:58:57,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:57,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409013575bc060ce160320 HTTP/1.1" 502 150 2025-12-17 22:58:57,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409013575bc060ce160320'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:57,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409013575bc060ce160320 2025-12-17 22:58:57,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409013575bc060ce160320 HTTP/1.1" 502 150 2025-12-17 22:58:57,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409013575bc060ce160320'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:57,764 - lava_callback - INFO - Received callback for node 694092a2575bc060ce160b0c INFO: 172.21.0.1:33038 - "POST /node/694092a2575bc060ce160b0c HTTP/1.0" 202 Accepted 2025-12-17 22:58:58,456 - lava_callback - INFO - Received callback for node 694092a4575bc060ce160b0f INFO: 172.21.0.1:33050 - "POST /node/694092a4575bc060ce160b0f HTTP/1.0" 202 Accepted 2025-12-17 22:58:58,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff4575bc060ce1602c3 2025-12-17 22:58:58,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff4575bc060ce1602c3 HTTP/1.1" 502 150 2025-12-17 22:58:58,936 - lava_callback - ERROR - Error processing callback for node 69408ff4575bc060ce1602c3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff4575bc060ce1602c3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff4575bc060ce1602c3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff4575bc060ce1602c3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:58:58,938 - lava_callback - INFO - Starting async job submit for node 69409015575bc060ce160327 2025-12-17 22:58:58,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:58:58,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409015575bc060ce160327 HTTP/1.1" 502 150 2025-12-17 22:58:58,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409015575bc060ce160327'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:58,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409015575bc060ce160327 2025-12-17 22:58:58,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409010575bc060ce160301 2025-12-17 22:58:58,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409015575bc060ce160327 HTTP/1.1" 502 150 2025-12-17 22:58:58,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409015575bc060ce160327'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:58,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409010575bc060ce160301 HTTP/1.1" 502 150 2025-12-17 22:58:58,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409010575bc060ce160301'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:59,113 - lava_callback - INFO - Received callback for node 694092a6575bc060ce160b14 INFO: 172.21.0.1:33056 - "POST /node/694092a6575bc060ce160b14 HTTP/1.0" 202 Accepted 2025-12-17 22:58:59,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409013575bc060ce160320 2025-12-17 22:58:59,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409013575bc060ce160320 HTTP/1.1" 502 150 2025-12-17 22:58:59,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409013575bc060ce160320'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:58:59,763 - lava_callback - INFO - Received callback for node 694092a7575bc060ce160b17 INFO: 172.21.0.1:45290 - "POST /node/694092a7575bc060ce160b17 HTTP/1.0" 202 Accepted 2025-12-17 22:58:59,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d3575bc060ce160b88 2025-12-17 22:58:59,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d3575bc060ce160b88 HTTP/1.1" 502 150 2025-12-17 22:58:59,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d3575bc060ce160b88'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:00,433 - lava_callback - INFO - Received callback for node 694092a9575bc060ce160b1a INFO: 172.21.0.1:45292 - "POST /node/694092a9575bc060ce160b1a HTTP/1.0" 202 Accepted 2025-12-17 22:59:00,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409017575bc060ce16032a 2025-12-17 22:59:00,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409017575bc060ce16032a HTTP/1.1" 502 150 2025-12-17 22:59:00,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409017575bc060ce16032a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:00,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409015575bc060ce160327 2025-12-17 22:59:00,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409015575bc060ce160327 HTTP/1.1" 502 150 2025-12-17 22:59:00,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409015575bc060ce160327'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:01,114 - lava_callback - INFO - Received callback for node 694092aa575bc060ce160b1d INFO: 172.21.0.1:45294 - "POST /node/694092aa575bc060ce160b1d HTTP/1.0" 202 Accepted 2025-12-17 22:59:01,784 - lava_callback - INFO - Received callback for node 694092ac575bc060ce160b34 INFO: 172.21.0.1:45310 - "POST /node/694092ac575bc060ce160b34 HTTP/1.0" 202 Accepted 2025-12-17 22:59:01,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff5575bc060ce1602c6 2025-12-17 22:59:01,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff5575bc060ce1602c6 HTTP/1.1" 502 150 2025-12-17 22:59:01,872 - lava_callback - ERROR - Error processing callback for node 69408ff5575bc060ce1602c6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff5575bc060ce1602c6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff5575bc060ce1602c6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff5575bc060ce1602c6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:01,875 - lava_callback - INFO - Starting async job submit for node 69408f1b575bc060ce160073 2025-12-17 22:59:01,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:01,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1b575bc060ce160073 HTTP/1.1" 502 150 2025-12-17 22:59:01,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1b575bc060ce160073'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:01,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1b575bc060ce160073 2025-12-17 22:59:01,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1b575bc060ce160073 HTTP/1.1" 502 150 2025-12-17 22:59:01,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1b575bc060ce160073'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:02,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff6575bc060ce1602c9 2025-12-17 22:59:02,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff6575bc060ce1602c9 HTTP/1.1" 502 150 2025-12-17 22:59:02,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ff6575bc060ce1602c9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45326 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:59:02,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409010575bc060ce160301 2025-12-17 22:59:02,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409010575bc060ce160301 HTTP/1.1" 502 150 2025-12-17 22:59:02,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409010575bc060ce160301'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:03,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409013575bc060ce160320 2025-12-17 22:59:03,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409013575bc060ce160320 HTTP/1.1" 502 150 2025-12-17 22:59:03,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409013575bc060ce160320'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:03,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1b575bc060ce160073 2025-12-17 22:59:03,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1b575bc060ce160073 HTTP/1.1" 502 150 2025-12-17 22:59:03,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1b575bc060ce160073'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:04,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409015575bc060ce160327 2025-12-17 22:59:04,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409015575bc060ce160327 HTTP/1.1" 502 150 2025-12-17 22:59:04,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409015575bc060ce160327'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:05,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffd575bc060ce1602db HTTP/1.1" 502 150 2025-12-17 22:59:05,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408ffd575bc060ce1602db'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:05,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409988575bc060ce161d2d 2025-12-17 22:59:05,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409988575bc060ce161d2d HTTP/1.1" 502 150 2025-12-17 22:59:05,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409988575bc060ce161d2d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:05,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ffa575bc060ce1602cf 2025-12-17 22:59:05,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffa575bc060ce1602cf HTTP/1.1" 502 150 2025-12-17 22:59:05,981 - lava_callback - ERROR - Error processing callback for node 69408ffa575bc060ce1602cf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ffa575bc060ce1602cf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ffa575bc060ce1602cf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ffa575bc060ce1602cf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:05,983 - lava_callback - INFO - Starting async job submit for node 69409019575bc060ce16032d 2025-12-17 22:59:05,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:05,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409019575bc060ce16032d HTTP/1.1" 502 150 2025-12-17 22:59:05,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409019575bc060ce16032d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:05,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409019575bc060ce16032d 2025-12-17 22:59:05,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409019575bc060ce16032d HTTP/1.1" 502 150 2025-12-17 22:59:05,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409019575bc060ce16032d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:06,184 - lava_callback - INFO - Received callback for node 694092ad575bc060ce160b37 INFO: 172.21.0.1:45342 - "POST /node/694092ad575bc060ce160b37 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:45358 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:59:06,834 - lava_callback - INFO - Received callback for node 694092ae575bc060ce160b3a INFO: 172.21.0.1:45364 - "POST /node/694092ae575bc060ce160b3a HTTP/1.0" 202 Accepted 2025-12-17 22:59:06,889 - lava_callback - INFO - Received callback for node 694092b0575bc060ce160b3d INFO: 172.21.0.1:45374 - "POST /node/694092b0575bc060ce160b3d HTTP/1.0" 202 Accepted 2025-12-17 22:59:07,502 - lava_callback - INFO - Received callback for node 694092b2575bc060ce160b43 INFO: 172.21.0.1:45386 - "POST /node/694092b2575bc060ce160b43 HTTP/1.0" 202 Accepted 2025-12-17 22:59:07,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409004575bc060ce1602e0 2025-12-17 22:59:07,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409004575bc060ce1602e0 HTTP/1.1" 502 150 2025-12-17 22:59:07,630 - lava_callback - ERROR - Error processing callback for node 69409004575bc060ce1602e0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409004575bc060ce1602e0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409004575bc060ce1602e0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409004575bc060ce1602e0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:07,632 - lava_callback - INFO - Starting async job submit for node 6940901c575bc060ce160330 2025-12-17 22:59:07,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:07,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940901c575bc060ce160330 HTTP/1.1" 502 150 2025-12-17 22:59:07,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940901c575bc060ce160330'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:07,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940901c575bc060ce160330 2025-12-17 22:59:07,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940901c575bc060ce160330 HTTP/1.1" 502 150 2025-12-17 22:59:07,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940901c575bc060ce160330'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:07,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409006575bc060ce1602ee 2025-12-17 22:59:07,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409006575bc060ce1602ee HTTP/1.1" 502 150 2025-12-17 22:59:07,731 - lava_callback - ERROR - Error processing callback for node 69409006575bc060ce1602ee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409006575bc060ce1602ee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409006575bc060ce1602ee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409006575bc060ce1602ee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:07,733 - lava_callback - INFO - Starting async job submit for node 6940901e575bc060ce160333 2025-12-17 22:59:07,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:07,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940901e575bc060ce160333 HTTP/1.1" 502 150 2025-12-17 22:59:07,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940901e575bc060ce160333'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:07,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940901e575bc060ce160333 2025-12-17 22:59:07,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940901e575bc060ce160333 HTTP/1.1" 502 150 2025-12-17 22:59:07,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940901e575bc060ce160333'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:07,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409008575bc060ce1602f1 HTTP/1.1" 502 150 2025-12-17 22:59:07,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409008575bc060ce1602f1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:07,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d3575bc060ce160b88 2025-12-17 22:59:07,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d3575bc060ce160b88 HTTP/1.1" 502 150 2025-12-17 22:59:07,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d3575bc060ce160b88'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:07,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1b575bc060ce160073 2025-12-17 22:59:07,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1b575bc060ce160073 HTTP/1.1" 502 150 2025-12-17 22:59:07,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1b575bc060ce160073'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:07,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409019575bc060ce16032d 2025-12-17 22:59:07,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409019575bc060ce16032d HTTP/1.1" 502 150 2025-12-17 22:59:07,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409019575bc060ce16032d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:08,152 - lava_callback - INFO - Received callback for node 694092b5575bc060ce160b49 INFO: 172.21.0.1:45400 - "POST /node/694092b5575bc060ce160b49 HTTP/1.0" 202 Accepted 2025-12-17 22:59:08,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409017575bc060ce16032a 2025-12-17 22:59:08,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409017575bc060ce16032a HTTP/1.1" 502 150 2025-12-17 22:59:08,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409017575bc060ce16032a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:08,813 - lava_callback - INFO - Received callback for node 694092b7575bc060ce160b4f INFO: 172.21.0.1:45402 - "POST /node/694092b7575bc060ce160b4f HTTP/1.0" 202 Accepted 2025-12-17 22:59:08,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940900a575bc060ce1602f4 2025-12-17 22:59:08,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900a575bc060ce1602f4 HTTP/1.1" 502 150 2025-12-17 22:59:08,971 - lava_callback - ERROR - Error processing callback for node 6940900a575bc060ce1602f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940900a575bc060ce1602f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940900a575bc060ce1602f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940900a575bc060ce1602f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:08,973 - lava_callback - INFO - Starting async job submit for node 69409022575bc060ce160339 2025-12-17 22:59:08,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:08,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409022575bc060ce160339 HTTP/1.1" 502 150 2025-12-17 22:59:08,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409022575bc060ce160339'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:08,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409022575bc060ce160339 2025-12-17 22:59:08,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409022575bc060ce160339 HTTP/1.1" 502 150 2025-12-17 22:59:08,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409022575bc060ce160339'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:09,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940900c575bc060ce1602f7 2025-12-17 22:59:09,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900c575bc060ce1602f7 HTTP/1.1" 502 150 2025-12-17 22:59:09,002 - lava_callback - ERROR - Error processing callback for node 6940900c575bc060ce1602f7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940900c575bc060ce1602f7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940900c575bc060ce1602f7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940900c575bc060ce1602f7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:09,005 - lava_callback - INFO - Starting async job submit for node 69409024575bc060ce16033e 2025-12-17 22:59:09,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:09,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409024575bc060ce16033e HTTP/1.1" 502 150 2025-12-17 22:59:09,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409024575bc060ce16033e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:09,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409024575bc060ce16033e 2025-12-17 22:59:09,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409024575bc060ce16033e HTTP/1.1" 502 150 2025-12-17 22:59:09,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409024575bc060ce16033e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:09,473 - lava_callback - INFO - Received callback for node 694092b8575bc060ce160b52 INFO: 172.21.0.1:45412 - "POST /node/694092b8575bc060ce160b52 HTTP/1.0" 202 Accepted 2025-12-17 22:59:09,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940901c575bc060ce160330 2025-12-17 22:59:09,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940901c575bc060ce160330 HTTP/1.1" 502 150 2025-12-17 22:59:09,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940901c575bc060ce160330'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:09,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940901e575bc060ce160333 2025-12-17 22:59:09,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940901e575bc060ce160333 HTTP/1.1" 502 150 2025-12-17 22:59:09,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940901e575bc060ce160333'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:09,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940900e575bc060ce1602fa 2025-12-17 22:59:09,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940900e575bc060ce1602fa HTTP/1.1" 502 150 2025-12-17 22:59:09,934 - lava_callback - ERROR - Error processing callback for node 6940900e575bc060ce1602fa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940900e575bc060ce1602fa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940900e575bc060ce1602fa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940900e575bc060ce1602fa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:09,936 - lava_callback - INFO - Starting async job submit for node 69408c34575bc060ce15f4ac 2025-12-17 22:59:10,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:10,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c34575bc060ce15f4ac HTTP/1.1" 502 150 2025-12-17 22:59:10,269 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c34575bc060ce15f4ac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:10,269 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c34575bc060ce15f4ac 2025-12-17 22:59:10,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c34575bc060ce15f4ac HTTP/1.1" 502 150 2025-12-17 22:59:10,270 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c34575bc060ce15f4ac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:10,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409010575bc060ce160301 2025-12-17 22:59:10,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409010575bc060ce160301 HTTP/1.1" 502 150 2025-12-17 22:59:10,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409010575bc060ce160301'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:10,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409022575bc060ce160339 2025-12-17 22:59:10,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409022575bc060ce160339 HTTP/1.1" 502 150 2025-12-17 22:59:10,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409022575bc060ce160339'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:11,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409024575bc060ce16033e 2025-12-17 22:59:11,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409024575bc060ce16033e HTTP/1.1" 502 150 2025-12-17 22:59:11,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409024575bc060ce16033e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:11,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409013575bc060ce160320 2025-12-17 22:59:11,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409013575bc060ce160320 HTTP/1.1" 502 150 2025-12-17 22:59:11,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409013575bc060ce160320'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:11,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409019575bc060ce16032d 2025-12-17 22:59:11,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409019575bc060ce16032d HTTP/1.1" 502 150 2025-12-17 22:59:11,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409019575bc060ce16032d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:12,271 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c34575bc060ce15f4ac 2025-12-17 22:59:12,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c34575bc060ce15f4ac HTTP/1.1" 502 150 2025-12-17 22:59:12,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c34575bc060ce15f4ac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:12,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409015575bc060ce160327 2025-12-17 22:59:12,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409015575bc060ce160327 HTTP/1.1" 502 150 2025-12-17 22:59:12,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409015575bc060ce160327'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:13,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940901c575bc060ce160330 2025-12-17 22:59:13,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940901c575bc060ce160330 HTTP/1.1" 502 150 2025-12-17 22:59:13,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940901c575bc060ce160330'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:13,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940901e575bc060ce160333 2025-12-17 22:59:13,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940901e575bc060ce160333 HTTP/1.1" 502 150 2025-12-17 22:59:13,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940901e575bc060ce160333'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:14,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409022575bc060ce160339 2025-12-17 22:59:14,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409022575bc060ce160339 HTTP/1.1" 502 150 2025-12-17 22:59:14,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409022575bc060ce160339'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:15,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409024575bc060ce16033e 2025-12-17 22:59:15,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409024575bc060ce16033e HTTP/1.1" 502 150 2025-12-17 22:59:15,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409024575bc060ce16033e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54308 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 22:59:15,732 - lava_callback - INFO - Received callback for node 694092b1575bc060ce160b40 INFO: 172.21.0.1:54314 - "POST /node/694092b1575bc060ce160b40 HTTP/1.0" 202 Accepted 2025-12-17 22:59:15,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1b575bc060ce160073 2025-12-17 22:59:15,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1b575bc060ce160073 HTTP/1.1" 502 150 2025-12-17 22:59:15,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408f1b575bc060ce160073'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:16,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c34575bc060ce15f4ac 2025-12-17 22:59:16,378 - lava_callback - INFO - Received callback for node 694092b4575bc060ce160b46 INFO: 172.21.0.1:54320 - "POST /node/694092b4575bc060ce160b46 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:54324 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:59:17,025 - lava_callback - INFO - Received callback for node 694092b6575bc060ce160b4c INFO: 172.21.0.1:54338 - "POST /node/694092b6575bc060ce160b4c HTTP/1.0" 202 Accepted 2025-12-17 22:59:17,712 - lava_callback - INFO - Received callback for node 694092ba575bc060ce160b58 INFO: 172.21.0.1:54350 - "POST /node/694092ba575bc060ce160b58 HTTP/1.0" 202 Accepted 2025-12-17 22:59:18,381 - lava_callback - INFO - Received callback for node 694092bb575bc060ce160b60 INFO: 172.21.0.1:54364 - "POST /node/694092bb575bc060ce160b60 HTTP/1.0" 202 Accepted 2025-12-17 22:59:18,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ff6575bc060ce1602c9 2025-12-17 22:59:18,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ff6575bc060ce1602c9 HTTP/1.1" 502 150 2025-12-17 22:59:18,779 - lava_callback - ERROR - Error processing callback for node 69408ff6575bc060ce1602c9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff6575bc060ce1602c9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff6575bc060ce1602c9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ff6575bc060ce1602c9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:18,781 - lava_callback - INFO - Starting async job submit for node 69409026575bc060ce160349 2025-12-17 22:59:18,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:18,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409026575bc060ce160349 HTTP/1.1" 502 150 2025-12-17 22:59:18,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409026575bc060ce160349'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:18,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409026575bc060ce160349 2025-12-17 22:59:18,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409026575bc060ce160349 HTTP/1.1" 502 150 2025-12-17 22:59:18,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409026575bc060ce160349'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:19,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409019575bc060ce16032d 2025-12-17 22:59:19,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409019575bc060ce16032d HTTP/1.1" 502 150 2025-12-17 22:59:19,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409019575bc060ce16032d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:20,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409026575bc060ce160349 2025-12-17 22:59:20,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409026575bc060ce160349 HTTP/1.1" 502 150 2025-12-17 22:59:20,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409026575bc060ce160349'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:21,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408ffd575bc060ce1602db 2025-12-17 22:59:21,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408ffd575bc060ce1602db HTTP/1.1" 502 150 2025-12-17 22:59:21,266 - lava_callback - ERROR - Error processing callback for node 69408ffd575bc060ce1602db: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ffd575bc060ce1602db (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ffd575bc060ce1602db (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408ffd575bc060ce1602db (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:21,268 - lava_callback - INFO - Starting async job submit for node 69409028575bc060ce160357 2025-12-17 22:59:21,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:21,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409028575bc060ce160357 HTTP/1.1" 502 150 2025-12-17 22:59:21,278 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409028575bc060ce160357'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:21,278 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409028575bc060ce160357 2025-12-17 22:59:21,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409028575bc060ce160357 HTTP/1.1" 502 150 2025-12-17 22:59:21,279 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409028575bc060ce160357'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:21,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409988575bc060ce161d2d 2025-12-17 22:59:21,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409988575bc060ce161d2d HTTP/1.1" 502 150 2025-12-17 22:59:21,515 - lava_callback - ERROR - Error processing callback for node 69409988575bc060ce161d2d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409988575bc060ce161d2d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409988575bc060ce161d2d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409988575bc060ce161d2d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:21,517 - lava_callback - INFO - Starting async job submit for node 6940902a575bc060ce16035a 2025-12-17 22:59:21,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:21,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902a575bc060ce16035a HTTP/1.1" 502 150 2025-12-17 22:59:21,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940902a575bc060ce16035a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:21,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940902a575bc060ce16035a 2025-12-17 22:59:21,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902a575bc060ce16035a HTTP/1.1" 502 150 2025-12-17 22:59:21,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940902a575bc060ce16035a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:21,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940901c575bc060ce160330 2025-12-17 22:59:21,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940901c575bc060ce160330 HTTP/1.1" 502 150 2025-12-17 22:59:21,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940901c575bc060ce160330'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:21,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940901e575bc060ce160333 2025-12-17 22:59:21,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940901e575bc060ce160333 HTTP/1.1" 502 150 2025-12-17 22:59:21,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940901e575bc060ce160333'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:22,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409022575bc060ce160339 2025-12-17 22:59:22,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409022575bc060ce160339 HTTP/1.1" 502 150 2025-12-17 22:59:22,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409022575bc060ce160339'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:23,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409024575bc060ce16033e 2025-12-17 22:59:23,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409024575bc060ce16033e HTTP/1.1" 502 150 2025-12-17 22:59:23,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409024575bc060ce16033e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:23,279 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409028575bc060ce160357 2025-12-17 22:59:23,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409028575bc060ce160357 HTTP/1.1" 502 150 2025-12-17 22:59:23,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409028575bc060ce160357'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:23,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940902a575bc060ce16035a 2025-12-17 22:59:23,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902a575bc060ce16035a HTTP/1.1" 502 150 2025-12-17 22:59:23,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940902a575bc060ce16035a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:23,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409008575bc060ce1602f1 2025-12-17 22:59:23,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409008575bc060ce1602f1 HTTP/1.1" 502 150 2025-12-17 22:59:23,826 - lava_callback - ERROR - Error processing callback for node 69409008575bc060ce1602f1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409008575bc060ce1602f1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409008575bc060ce1602f1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409008575bc060ce1602f1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:23,828 - lava_callback - INFO - Starting async job submit for node 6940902c575bc060ce16035d 2025-12-17 22:59:23,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:23,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902c575bc060ce16035d HTTP/1.1" 502 150 2025-12-17 22:59:23,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940902c575bc060ce16035d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:23,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940902c575bc060ce16035d 2025-12-17 22:59:23,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902c575bc060ce16035d HTTP/1.1" 502 150 2025-12-17 22:59:23,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940902c575bc060ce16035d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:23,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d3575bc060ce160b88 2025-12-17 22:59:23,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d3575bc060ce160b88 HTTP/1.1" 502 150 2025-12-17 22:59:23,849 - lava_callback - ERROR - Error processing callback for node 694092d3575bc060ce160b88: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092d3575bc060ce160b88 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092d3575bc060ce160b88 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092d3575bc060ce160b88 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:23,850 - lava_callback - INFO - Starting async job submit for node 6940902e575bc060ce160360 2025-12-17 22:59:23,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:23,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902e575bc060ce160360 HTTP/1.1" 502 150 2025-12-17 22:59:23,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940902e575bc060ce160360'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:23,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940902e575bc060ce160360 2025-12-17 22:59:23,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902e575bc060ce160360 HTTP/1.1" 502 150 2025-12-17 22:59:23,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940902e575bc060ce160360'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:24,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409017575bc060ce16032a 2025-12-17 22:59:24,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409017575bc060ce16032a HTTP/1.1" 502 150 2025-12-17 22:59:24,752 - lava_callback - ERROR - Error processing callback for node 69409017575bc060ce16032a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409017575bc060ce16032a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409017575bc060ce16032a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409017575bc060ce16032a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:24,754 - lava_callback - INFO - Starting async job submit for node 69409030575bc060ce160363 2025-12-17 22:59:24,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:24,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409030575bc060ce160363 HTTP/1.1" 502 150 2025-12-17 22:59:24,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409030575bc060ce160363'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:24,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409030575bc060ce160363 2025-12-17 22:59:24,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409030575bc060ce160363 HTTP/1.1" 502 150 2025-12-17 22:59:24,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409030575bc060ce160363'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:24,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409026575bc060ce160349 2025-12-17 22:59:24,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409026575bc060ce160349 HTTP/1.1" 502 150 2025-12-17 22:59:24,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409026575bc060ce160349'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:25,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940902c575bc060ce16035d 2025-12-17 22:59:25,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902c575bc060ce16035d HTTP/1.1" 502 150 2025-12-17 22:59:25,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940902c575bc060ce16035d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:25,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940902e575bc060ce160360 2025-12-17 22:59:25,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902e575bc060ce160360 HTTP/1.1" 502 150 2025-12-17 22:59:25,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940902e575bc060ce160360'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:26,192 - lava_callback - INFO - Received callback for node 694092bd575bc060ce160b65 INFO: 172.21.0.1:38796 - "POST /node/694092bd575bc060ce160b65 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:38808 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:59:26,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409030575bc060ce160363 2025-12-17 22:59:26,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409030575bc060ce160363 HTTP/1.1" 502 150 2025-12-17 22:59:26,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409030575bc060ce160363'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:26,859 - lava_callback - INFO - Received callback for node 694092bf575bc060ce160b68 INFO: 172.21.0.1:38822 - "POST /node/694092bf575bc060ce160b68 HTTP/1.0" 202 Accepted 2025-12-17 22:59:26,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409010575bc060ce160301 2025-12-17 22:59:26,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409010575bc060ce160301 HTTP/1.1" 502 150 2025-12-17 22:59:26,950 - lava_callback - ERROR - Error processing callback for node 69409010575bc060ce160301: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409010575bc060ce160301 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409010575bc060ce160301 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409010575bc060ce160301 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:26,952 - lava_callback - INFO - Starting async job submit for node 69409032575bc060ce160366 2025-12-17 22:59:26,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:26,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409032575bc060ce160366 HTTP/1.1" 502 150 2025-12-17 22:59:26,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409032575bc060ce160366'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:26,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409032575bc060ce160366 2025-12-17 22:59:26,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409032575bc060ce160366 HTTP/1.1" 502 150 2025-12-17 22:59:26,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409032575bc060ce160366'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:27,280 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409028575bc060ce160357 2025-12-17 22:59:27,507 - lava_callback - INFO - Received callback for node 694092c9575bc060ce160b6b INFO: 172.21.0.1:38826 - "POST /node/694092c9575bc060ce160b6b HTTP/1.0" 202 Accepted 2025-12-17 22:59:27,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940902a575bc060ce16035a 2025-12-17 22:59:27,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902a575bc060ce16035a HTTP/1.1" 502 150 2025-12-17 22:59:27,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940902a575bc060ce16035a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:27,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409013575bc060ce160320 2025-12-17 22:59:27,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409013575bc060ce160320 HTTP/1.1" 502 150 2025-12-17 22:59:27,664 - lava_callback - ERROR - Error processing callback for node 69409013575bc060ce160320: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409013575bc060ce160320 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409013575bc060ce160320 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409013575bc060ce160320 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:27,666 - lava_callback - INFO - Starting async job submit for node 69409034575bc060ce160369 2025-12-17 22:59:27,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:27,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409034575bc060ce160369 HTTP/1.1" 502 150 2025-12-17 22:59:27,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409034575bc060ce160369'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:27,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409034575bc060ce160369 2025-12-17 22:59:27,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409034575bc060ce160369 HTTP/1.1" 502 150 2025-12-17 22:59:27,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409034575bc060ce160369'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:28,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409015575bc060ce160327 2025-12-17 22:59:28,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409015575bc060ce160327 HTTP/1.1" 502 150 2025-12-17 22:59:28,952 - lava_callback - ERROR - Error processing callback for node 69409015575bc060ce160327: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409015575bc060ce160327 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409015575bc060ce160327 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409015575bc060ce160327 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:28,954 - lava_callback - INFO - Starting async job submit for node 69409037575bc060ce16036f 2025-12-17 22:59:28,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:28,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409037575bc060ce16036f HTTP/1.1" 502 150 2025-12-17 22:59:28,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409037575bc060ce16036f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:28,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409037575bc060ce16036f 2025-12-17 22:59:28,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409037575bc060ce16036f HTTP/1.1" 502 150 2025-12-17 22:59:28,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409037575bc060ce16036f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:28,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409032575bc060ce160366 2025-12-17 22:59:28,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409032575bc060ce160366 HTTP/1.1" 502 150 2025-12-17 22:59:28,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409032575bc060ce160366'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:29,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409034575bc060ce160369 2025-12-17 22:59:29,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409034575bc060ce160369 HTTP/1.1" 502 150 2025-12-17 22:59:29,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409034575bc060ce160369'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:29,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940902c575bc060ce16035d 2025-12-17 22:59:29,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902c575bc060ce16035d HTTP/1.1" 502 150 2025-12-17 22:59:29,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940902c575bc060ce16035d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:29,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940902e575bc060ce160360 2025-12-17 22:59:29,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902e575bc060ce160360 HTTP/1.1" 502 150 2025-12-17 22:59:29,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940902e575bc060ce160360'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:30,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409030575bc060ce160363 2025-12-17 22:59:30,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409030575bc060ce160363 HTTP/1.1" 502 150 2025-12-17 22:59:30,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409030575bc060ce160363'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:30,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409037575bc060ce16036f 2025-12-17 22:59:30,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409037575bc060ce16036f HTTP/1.1" 502 150 2025-12-17 22:59:30,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409037575bc060ce16036f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:31,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c34575bc060ce15f4ac HTTP/1.1" 502 150 2025-12-17 22:59:31,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c34575bc060ce15f4ac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:31,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408f1b575bc060ce160073 2025-12-17 22:59:31,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408f1b575bc060ce160073 HTTP/1.1" 502 150 2025-12-17 22:59:31,934 - lava_callback - ERROR - Error processing callback for node 69408f1b575bc060ce160073: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f1b575bc060ce160073 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f1b575bc060ce160073 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408f1b575bc060ce160073 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:31,936 - lava_callback - INFO - Starting async job submit for node 69409038575bc060ce160374 2025-12-17 22:59:31,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:31,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409038575bc060ce160374 HTTP/1.1" 502 150 2025-12-17 22:59:31,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409038575bc060ce160374'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:31,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409038575bc060ce160374 2025-12-17 22:59:31,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409038575bc060ce160374 HTTP/1.1" 502 150 2025-12-17 22:59:31,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409038575bc060ce160374'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:32,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409026575bc060ce160349 2025-12-17 22:59:32,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409026575bc060ce160349 HTTP/1.1" 502 150 2025-12-17 22:59:32,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409026575bc060ce160349'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:32,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409032575bc060ce160366 2025-12-17 22:59:32,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409032575bc060ce160366 HTTP/1.1" 502 150 2025-12-17 22:59:32,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409032575bc060ce160366'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:33,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409034575bc060ce160369 2025-12-17 22:59:33,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409034575bc060ce160369 HTTP/1.1" 502 150 2025-12-17 22:59:33,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409034575bc060ce160369'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:33,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409038575bc060ce160374 2025-12-17 22:59:33,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409038575bc060ce160374 HTTP/1.1" 502 150 2025-12-17 22:59:33,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409038575bc060ce160374'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:34,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409037575bc060ce16036f 2025-12-17 22:59:34,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409037575bc060ce16036f HTTP/1.1" 502 150 2025-12-17 22:59:34,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409037575bc060ce16036f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:35,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940902a575bc060ce16035a 2025-12-17 22:59:35,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902a575bc060ce16035a HTTP/1.1" 502 150 2025-12-17 22:59:35,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940902a575bc060ce16035a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:35,745 - lava_callback - INFO - Received callback for node 694092cc575bc060ce160b71 INFO: 172.21.0.1:52428 - "POST /node/694092cc575bc060ce160b71 HTTP/1.0" 202 Accepted 2025-12-17 22:59:35,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409019575bc060ce16032d 2025-12-17 22:59:35,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409019575bc060ce16032d HTTP/1.1" 502 150 2025-12-17 22:59:35,999 - lava_callback - ERROR - Error processing callback for node 69409019575bc060ce16032d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409019575bc060ce16032d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409019575bc060ce16032d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409019575bc060ce16032d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:36,001 - lava_callback - INFO - Starting async job submit for node 6940903d575bc060ce16038d 2025-12-17 22:59:36,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:36,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903d575bc060ce16038d HTTP/1.1" 502 150 2025-12-17 22:59:36,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903d575bc060ce16038d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:36,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903d575bc060ce16038d 2025-12-17 22:59:36,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903d575bc060ce16038d HTTP/1.1" 502 150 2025-12-17 22:59:36,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903d575bc060ce16038d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:36,428 - lava_callback - INFO - Received callback for node 694092d5575bc060ce160b8b INFO: 172.21.0.1:52432 - "POST /node/694092d5575bc060ce160b8b HTTP/1.0" 202 Accepted INFO: 172.21.0.1:52436 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:59:37,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940901c575bc060ce160330 2025-12-17 22:59:37,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940901c575bc060ce160330 HTTP/1.1" 502 150 2025-12-17 22:59:37,653 - lava_callback - ERROR - Error processing callback for node 6940901c575bc060ce160330: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940901c575bc060ce160330 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940901c575bc060ce160330 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940901c575bc060ce160330 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:37,655 - lava_callback - INFO - Starting async job submit for node 6940903e575bc060ce160390 2025-12-17 22:59:37,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:37,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903e575bc060ce160390 HTTP/1.1" 502 150 2025-12-17 22:59:37,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903e575bc060ce160390'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:37,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903e575bc060ce160390 2025-12-17 22:59:37,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903e575bc060ce160390 HTTP/1.1" 502 150 2025-12-17 22:59:37,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903e575bc060ce160390'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:37,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940901e575bc060ce160333 2025-12-17 22:59:37,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940901e575bc060ce160333 HTTP/1.1" 502 150 2025-12-17 22:59:37,761 - lava_callback - ERROR - Error processing callback for node 6940901e575bc060ce160333: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940901e575bc060ce160333 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940901e575bc060ce160333 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940901e575bc060ce160333 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:37,763 - lava_callback - INFO - Starting async job submit for node 6940903a575bc060ce16037f 2025-12-17 22:59:37,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:37,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903a575bc060ce16037f HTTP/1.1" 502 150 2025-12-17 22:59:37,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903a575bc060ce16037f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:37,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903a575bc060ce16037f 2025-12-17 22:59:37,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903a575bc060ce16037f HTTP/1.1" 502 150 2025-12-17 22:59:37,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903a575bc060ce16037f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:37,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940902c575bc060ce16035d 2025-12-17 22:59:37,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902c575bc060ce16035d HTTP/1.1" 502 150 2025-12-17 22:59:37,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940902c575bc060ce16035d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:37,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940902e575bc060ce160360 2025-12-17 22:59:37,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902e575bc060ce160360 HTTP/1.1" 502 150 2025-12-17 22:59:37,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940902e575bc060ce160360'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:37,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409038575bc060ce160374 2025-12-17 22:59:37,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409038575bc060ce160374 HTTP/1.1" 502 150 2025-12-17 22:59:37,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409038575bc060ce160374'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:38,028 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903d575bc060ce16038d 2025-12-17 22:59:38,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903d575bc060ce16038d HTTP/1.1" 502 150 2025-12-17 22:59:38,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903d575bc060ce16038d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:38,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409030575bc060ce160363 2025-12-17 22:59:38,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409030575bc060ce160363 HTTP/1.1" 502 150 2025-12-17 22:59:38,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409030575bc060ce160363'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:38,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409022575bc060ce160339 2025-12-17 22:59:38,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409022575bc060ce160339 HTTP/1.1" 502 150 2025-12-17 22:59:38,989 - lava_callback - ERROR - Error processing callback for node 69409022575bc060ce160339: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409022575bc060ce160339 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409022575bc060ce160339 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409022575bc060ce160339 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:38,990 - lava_callback - INFO - Starting async job submit for node 6940903b575bc060ce160386 2025-12-17 22:59:38,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:39,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409024575bc060ce16033e 2025-12-17 22:59:39,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409024575bc060ce16033e HTTP/1.1" 502 150 2025-12-17 22:59:39,020 - lava_callback - ERROR - Error processing callback for node 69409024575bc060ce16033e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409024575bc060ce16033e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409024575bc060ce16033e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409024575bc060ce16033e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:39,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903b575bc060ce160386 HTTP/1.1" 502 150 2025-12-17 22:59:39,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903b575bc060ce160386'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:39,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903b575bc060ce160386 2025-12-17 22:59:39,022 - lava_callback - INFO - Starting async job submit for node 69409041575bc060ce160393 2025-12-17 22:59:39,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:39,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903b575bc060ce160386 HTTP/1.1" 502 150 2025-12-17 22:59:39,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903b575bc060ce160386'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:39,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409041575bc060ce160393 HTTP/1.1" 502 150 2025-12-17 22:59:39,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409041575bc060ce160393'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:39,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409041575bc060ce160393 2025-12-17 22:59:39,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409041575bc060ce160393 HTTP/1.1" 502 150 2025-12-17 22:59:39,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409041575bc060ce160393'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:39,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c34575bc060ce15f4ac 2025-12-17 22:59:39,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c34575bc060ce15f4ac HTTP/1.1" 502 150 2025-12-17 22:59:39,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408c34575bc060ce15f4ac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:39,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903e575bc060ce160390 2025-12-17 22:59:39,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903e575bc060ce160390 HTTP/1.1" 502 150 2025-12-17 22:59:39,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903e575bc060ce160390'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:39,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903a575bc060ce16037f 2025-12-17 22:59:39,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903a575bc060ce16037f HTTP/1.1" 502 150 2025-12-17 22:59:39,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903a575bc060ce16037f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:40,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409032575bc060ce160366 2025-12-17 22:59:40,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409032575bc060ce160366 HTTP/1.1" 502 150 2025-12-17 22:59:40,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409032575bc060ce160366'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:41,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903b575bc060ce160386 2025-12-17 22:59:41,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903b575bc060ce160386 HTTP/1.1" 502 150 2025-12-17 22:59:41,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903b575bc060ce160386'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:41,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409041575bc060ce160393 2025-12-17 22:59:41,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409041575bc060ce160393 HTTP/1.1" 502 150 2025-12-17 22:59:41,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409041575bc060ce160393'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:41,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409034575bc060ce160369 2025-12-17 22:59:41,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409034575bc060ce160369 HTTP/1.1" 502 150 2025-12-17 22:59:41,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409034575bc060ce160369'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:42,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903d575bc060ce16038d 2025-12-17 22:59:42,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409028575bc060ce160357 HTTP/1.1" 502 150 2025-12-17 22:59:42,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409028575bc060ce160357'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:42,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409037575bc060ce16036f 2025-12-17 22:59:42,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409037575bc060ce16036f HTTP/1.1" 502 150 2025-12-17 22:59:42,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409037575bc060ce16036f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:43,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903e575bc060ce160390 2025-12-17 22:59:43,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903e575bc060ce160390 HTTP/1.1" 502 150 2025-12-17 22:59:43,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903e575bc060ce160390'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:43,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903a575bc060ce16037f 2025-12-17 22:59:43,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903a575bc060ce16037f HTTP/1.1" 502 150 2025-12-17 22:59:43,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903a575bc060ce16037f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:45,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903b575bc060ce160386 2025-12-17 22:59:45,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903b575bc060ce160386 HTTP/1.1" 502 150 2025-12-17 22:59:45,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903b575bc060ce160386'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:45,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409041575bc060ce160393 2025-12-17 22:59:45,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409041575bc060ce160393 HTTP/1.1" 502 150 2025-12-17 22:59:45,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409041575bc060ce160393'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:45,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409038575bc060ce160374 2025-12-17 22:59:45,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409038575bc060ce160374 HTTP/1.1" 502 150 2025-12-17 22:59:45,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409038575bc060ce160374'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:46,199 - lava_callback - INFO - Received callback for node 694092ce575bc060ce160b78 INFO: 172.21.0.1:56286 - "POST /node/694092ce575bc060ce160b78 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:56296 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:59:46,841 - lava_callback - INFO - Received callback for node 694092d1575bc060ce160b83 INFO: 172.21.0.1:56308 - "POST /node/694092d1575bc060ce160b83 HTTP/1.0" 202 Accepted 2025-12-17 22:59:47,478 - lava_callback - INFO - Received callback for node 694092da575bc060ce160b99 INFO: 172.21.0.1:56318 - "POST /node/694092da575bc060ce160b99 HTTP/1.0" 202 Accepted 2025-12-17 22:59:48,121 - lava_callback - INFO - Received callback for node 694092dd575bc060ce160b9c INFO: 172.21.0.1:56324 - "POST /node/694092dd575bc060ce160b9c HTTP/1.0" 202 Accepted 2025-12-17 22:59:48,751 - lava_callback - INFO - Received callback for node 694092e0575bc060ce160ba2 INFO: 172.21.0.1:56326 - "POST /node/694092e0575bc060ce160ba2 HTTP/1.0" 202 Accepted 2025-12-17 22:59:48,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409026575bc060ce160349 2025-12-17 22:59:48,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409026575bc060ce160349 HTTP/1.1" 502 150 2025-12-17 22:59:48,798 - lava_callback - ERROR - Error processing callback for node 69409026575bc060ce160349: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409026575bc060ce160349 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409026575bc060ce160349 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409026575bc060ce160349 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:48,800 - lava_callback - INFO - Starting async job submit for node 69409047575bc060ce160396 2025-12-17 22:59:48,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:48,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409047575bc060ce160396 HTTP/1.1" 502 150 2025-12-17 22:59:48,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409047575bc060ce160396'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:48,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409047575bc060ce160396 2025-12-17 22:59:48,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409047575bc060ce160396 HTTP/1.1" 502 150 2025-12-17 22:59:48,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409047575bc060ce160396'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:50,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409028575bc060ce160357 2025-12-17 22:59:50,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409028575bc060ce160357 HTTP/1.1" 502 150 2025-12-17 22:59:50,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409028575bc060ce160357'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:50,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409047575bc060ce160396 2025-12-17 22:59:50,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409047575bc060ce160396 HTTP/1.1" 502 150 2025-12-17 22:59:50,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409047575bc060ce160396'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:51,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940902a575bc060ce16035a 2025-12-17 22:59:51,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902a575bc060ce16035a HTTP/1.1" 502 150 2025-12-17 22:59:51,530 - lava_callback - ERROR - Error processing callback for node 6940902a575bc060ce16035a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940902a575bc060ce16035a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940902a575bc060ce16035a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940902a575bc060ce16035a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:51,533 - lava_callback - INFO - Starting async job submit for node 69409048575bc060ce160399 2025-12-17 22:59:51,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:51,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409048575bc060ce160399 HTTP/1.1" 502 150 2025-12-17 22:59:51,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409048575bc060ce160399'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:51,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409048575bc060ce160399 2025-12-17 22:59:51,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409048575bc060ce160399 HTTP/1.1" 502 150 2025-12-17 22:59:51,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409048575bc060ce160399'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:51,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903e575bc060ce160390 2025-12-17 22:59:51,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903e575bc060ce160390 HTTP/1.1" 502 150 2025-12-17 22:59:51,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903e575bc060ce160390'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:51,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903a575bc060ce16037f 2025-12-17 22:59:51,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903a575bc060ce16037f HTTP/1.1" 502 150 2025-12-17 22:59:51,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903a575bc060ce16037f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:53,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903b575bc060ce160386 2025-12-17 22:59:53,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409041575bc060ce160393 2025-12-17 22:59:53,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409041575bc060ce160393 HTTP/1.1" 502 150 2025-12-17 22:59:53,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409041575bc060ce160393'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:53,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409048575bc060ce160399 2025-12-17 22:59:53,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409048575bc060ce160399 HTTP/1.1" 502 150 2025-12-17 22:59:53,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409048575bc060ce160399'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:53,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940902c575bc060ce16035d 2025-12-17 22:59:53,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902c575bc060ce16035d HTTP/1.1" 502 150 2025-12-17 22:59:53,840 - lava_callback - ERROR - Error processing callback for node 6940902c575bc060ce16035d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940902c575bc060ce16035d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940902c575bc060ce16035d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940902c575bc060ce16035d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:53,842 - lava_callback - INFO - Starting async job submit for node 69409049575bc060ce16039c 2025-12-17 22:59:53,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:53,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409049575bc060ce16039c HTTP/1.1" 502 150 2025-12-17 22:59:53,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409049575bc060ce16039c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:53,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409049575bc060ce16039c 2025-12-17 22:59:53,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409049575bc060ce16039c HTTP/1.1" 502 150 2025-12-17 22:59:53,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409049575bc060ce16039c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:53,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940902e575bc060ce160360 2025-12-17 22:59:53,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940902e575bc060ce160360 HTTP/1.1" 502 150 2025-12-17 22:59:53,864 - lava_callback - ERROR - Error processing callback for node 6940902e575bc060ce160360: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940902e575bc060ce160360 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940902e575bc060ce160360 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940902e575bc060ce160360 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:53,866 - lava_callback - INFO - Starting async job submit for node 6940904b575bc060ce16039f 2025-12-17 22:59:53,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:53,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904b575bc060ce16039f HTTP/1.1" 502 150 2025-12-17 22:59:53,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904b575bc060ce16039f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:53,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904b575bc060ce16039f 2025-12-17 22:59:53,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904b575bc060ce16039f HTTP/1.1" 502 150 2025-12-17 22:59:53,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904b575bc060ce16039f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:54,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409030575bc060ce160363 2025-12-17 22:59:54,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409030575bc060ce160363 HTTP/1.1" 502 150 2025-12-17 22:59:54,767 - lava_callback - ERROR - Error processing callback for node 69409030575bc060ce160363: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409030575bc060ce160363 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409030575bc060ce160363 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409030575bc060ce160363 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:54,769 - lava_callback - INFO - Starting async job submit for node 6940904c575bc060ce1603a4 2025-12-17 22:59:54,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:54,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904c575bc060ce1603a4 HTTP/1.1" 502 150 2025-12-17 22:59:54,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904c575bc060ce1603a4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:54,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904c575bc060ce1603a4 2025-12-17 22:59:54,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904c575bc060ce1603a4 HTTP/1.1" 502 150 2025-12-17 22:59:54,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904c575bc060ce1603a4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:54,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409047575bc060ce160396 2025-12-17 22:59:54,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409047575bc060ce160396 HTTP/1.1" 502 150 2025-12-17 22:59:54,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409047575bc060ce160396'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:55,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408c34575bc060ce15f4ac 2025-12-17 22:59:55,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408c34575bc060ce15f4ac HTTP/1.1" 502 150 2025-12-17 22:59:55,639 - lava_callback - ERROR - Error processing callback for node 69408c34575bc060ce15f4ac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c34575bc060ce15f4ac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c34575bc060ce15f4ac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408c34575bc060ce15f4ac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:55,641 - lava_callback - INFO - Starting async job submit for node 69409050575bc060ce1603ce 2025-12-17 22:59:55,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:55,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409050575bc060ce1603ce HTTP/1.1" 502 150 2025-12-17 22:59:55,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409050575bc060ce1603ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:55,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409050575bc060ce1603ce 2025-12-17 22:59:55,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409050575bc060ce1603ce HTTP/1.1" 502 150 2025-12-17 22:59:55,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409050575bc060ce1603ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:55,781 - lava_callback - INFO - Received callback for node 6940939a575bc060ce160de3 INFO: 172.21.0.1:55746 - "POST /node/6940939a575bc060ce160de3 HTTP/1.0" 202 Accepted 2025-12-17 22:59:55,814 - lava_callback - INFO - Received callback for node 694092d7575bc060ce160b8e INFO: 172.21.0.1:55754 - "POST /node/694092d7575bc060ce160b8e HTTP/1.0" 202 Accepted 2025-12-17 22:59:55,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409049575bc060ce16039c 2025-12-17 22:59:55,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409049575bc060ce16039c HTTP/1.1" 502 150 2025-12-17 22:59:55,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409049575bc060ce16039c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:55,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904b575bc060ce16039f 2025-12-17 22:59:55,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904b575bc060ce16039f HTTP/1.1" 502 150 2025-12-17 22:59:55,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904b575bc060ce16039f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:56,488 - lava_callback - INFO - Received callback for node 694092d9575bc060ce160b96 INFO: 172.21.0.1:55760 - "POST /node/694092d9575bc060ce160b96 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:55776 - "GET / HTTP/1.0" 200 OK 2025-12-17 22:59:56,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904c575bc060ce1603a4 2025-12-17 22:59:56,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904c575bc060ce1603a4 HTTP/1.1" 502 150 2025-12-17 22:59:56,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904c575bc060ce1603a4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:56,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409032575bc060ce160366 2025-12-17 22:59:56,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409032575bc060ce160366 HTTP/1.1" 502 150 2025-12-17 22:59:56,968 - lava_callback - ERROR - Error processing callback for node 69409032575bc060ce160366: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409032575bc060ce160366 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409032575bc060ce160366 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409032575bc060ce160366 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:56,970 - lava_callback - INFO - Starting async job submit for node 6940904d575bc060ce1603ac 2025-12-17 22:59:56,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:56,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904d575bc060ce1603ac HTTP/1.1" 502 150 2025-12-17 22:59:56,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904d575bc060ce1603ac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:56,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904d575bc060ce1603ac 2025-12-17 22:59:56,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904d575bc060ce1603ac HTTP/1.1" 502 150 2025-12-17 22:59:56,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904d575bc060ce1603ac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:57,163 - lava_callback - INFO - Received callback for node 694092df575bc060ce160b9f INFO: 172.21.0.1:55790 - "POST /node/694092df575bc060ce160b9f HTTP/1.0" 202 Accepted 2025-12-17 22:59:57,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903d575bc060ce16038d HTTP/1.1" 502 150 2025-12-17 22:59:57,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903d575bc060ce16038d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:57,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409048575bc060ce160399 2025-12-17 22:59:57,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409048575bc060ce160399 HTTP/1.1" 502 150 2025-12-17 22:59:57,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409048575bc060ce160399'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:57,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409050575bc060ce1603ce 2025-12-17 22:59:57,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409050575bc060ce1603ce HTTP/1.1" 502 150 2025-12-17 22:59:57,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409050575bc060ce1603ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:57,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409034575bc060ce160369 2025-12-17 22:59:57,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409034575bc060ce160369 HTTP/1.1" 502 150 2025-12-17 22:59:57,706 - lava_callback - ERROR - Error processing callback for node 69409034575bc060ce160369: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409034575bc060ce160369 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409034575bc060ce160369 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409034575bc060ce160369 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:57,708 - lava_callback - INFO - Starting async job submit for node 6940905e575bc060ce160556 2025-12-17 22:59:57,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:57,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905e575bc060ce160556 HTTP/1.1" 502 150 2025-12-17 22:59:57,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905e575bc060ce160556'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:57,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905e575bc060ce160556 2025-12-17 22:59:57,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905e575bc060ce160556 HTTP/1.1" 502 150 2025-12-17 22:59:57,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905e575bc060ce160556'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:58,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409037575bc060ce16036f 2025-12-17 22:59:58,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409037575bc060ce16036f HTTP/1.1" 502 150 2025-12-17 22:59:58,967 - lava_callback - ERROR - Error processing callback for node 69409037575bc060ce16036f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409037575bc060ce16036f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409037575bc060ce16036f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409037575bc060ce16036f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 22:59:58,969 - lava_callback - INFO - Starting async job submit for node 6940904f575bc060ce1603cb 2025-12-17 22:59:58,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 22:59:58,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904d575bc060ce1603ac 2025-12-17 22:59:58,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904d575bc060ce1603ac HTTP/1.1" 502 150 2025-12-17 22:59:58,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904d575bc060ce1603ac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:58,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904f575bc060ce1603cb HTTP/1.1" 502 150 2025-12-17 22:59:58,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904f575bc060ce1603cb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:58,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904f575bc060ce1603cb 2025-12-17 22:59:58,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904f575bc060ce1603cb HTTP/1.1" 502 150 2025-12-17 22:59:58,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904f575bc060ce1603cb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:59,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905e575bc060ce160556 2025-12-17 22:59:59,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905e575bc060ce160556 HTTP/1.1" 502 150 2025-12-17 22:59:59,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905e575bc060ce160556'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:59,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409049575bc060ce16039c 2025-12-17 22:59:59,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409049575bc060ce16039c HTTP/1.1" 502 150 2025-12-17 22:59:59,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409049575bc060ce16039c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 22:59:59,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904b575bc060ce16039f 2025-12-17 22:59:59,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904b575bc060ce16039f HTTP/1.1" 502 150 2025-12-17 22:59:59,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904b575bc060ce16039f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:00,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904c575bc060ce1603a4 2025-12-17 23:00:00,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904c575bc060ce1603a4 HTTP/1.1" 502 150 2025-12-17 23:00:00,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904c575bc060ce1603a4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:00,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904f575bc060ce1603cb 2025-12-17 23:00:00,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904f575bc060ce1603cb HTTP/1.1" 502 150 2025-12-17 23:00:00,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904f575bc060ce1603cb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:01,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409050575bc060ce1603ce 2025-12-17 23:00:01,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409050575bc060ce1603ce HTTP/1.1" 502 150 2025-12-17 23:00:01,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409050575bc060ce1603ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:01,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409038575bc060ce160374 2025-12-17 23:00:01,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409038575bc060ce160374 HTTP/1.1" 502 150 2025-12-17 23:00:01,951 - lava_callback - ERROR - Error processing callback for node 69409038575bc060ce160374: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409038575bc060ce160374 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409038575bc060ce160374 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409038575bc060ce160374 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:01,954 - lava_callback - INFO - Starting async job submit for node 69409051575bc060ce1603d1 2025-12-17 23:00:01,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:01,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409051575bc060ce1603d1 HTTP/1.1" 502 150 2025-12-17 23:00:01,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409051575bc060ce1603d1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:01,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409051575bc060ce1603d1 2025-12-17 23:00:01,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409051575bc060ce1603d1 HTTP/1.1" 502 150 2025-12-17 23:00:01,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409051575bc060ce1603d1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:02,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409047575bc060ce160396 2025-12-17 23:00:02,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409047575bc060ce160396 HTTP/1.1" 502 150 2025-12-17 23:00:02,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409047575bc060ce160396'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41046 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:00:02,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904d575bc060ce1603ac 2025-12-17 23:00:02,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904d575bc060ce1603ac HTTP/1.1" 502 150 2025-12-17 23:00:02,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904d575bc060ce1603ac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:03,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905e575bc060ce160556 2025-12-17 23:00:03,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905e575bc060ce160556 HTTP/1.1" 502 150 2025-12-17 23:00:03,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905e575bc060ce160556'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:03,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409051575bc060ce1603d1 2025-12-17 23:00:03,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409051575bc060ce1603d1 HTTP/1.1" 502 150 2025-12-17 23:00:03,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409051575bc060ce1603d1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:04,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904f575bc060ce1603cb 2025-12-17 23:00:04,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904f575bc060ce1603cb HTTP/1.1" 502 150 2025-12-17 23:00:04,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904f575bc060ce1603cb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:05,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903d575bc060ce16038d 2025-12-17 23:00:05,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903d575bc060ce16038d HTTP/1.1" 502 150 2025-12-17 23:00:05,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903d575bc060ce16038d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:05,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409048575bc060ce160399 2025-12-17 23:00:05,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409048575bc060ce160399 HTTP/1.1" 502 150 2025-12-17 23:00:05,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409048575bc060ce160399'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:06,238 - lava_callback - INFO - Received callback for node 694092e2575bc060ce160baf INFO: 172.21.0.1:41054 - "POST /node/694092e2575bc060ce160baf HTTP/1.0" 202 Accepted INFO: 172.21.0.1:41070 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:00:06,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409028575bc060ce160357 2025-12-17 23:00:06,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409028575bc060ce160357 HTTP/1.1" 502 150 2025-12-17 23:00:06,652 - lava_callback - ERROR - Error processing callback for node 69409028575bc060ce160357: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409028575bc060ce160357 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409028575bc060ce160357 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409028575bc060ce160357 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:06,654 - lava_callback - INFO - Starting async job submit for node 69409061575bc060ce160560 2025-12-17 23:00:06,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:06,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409061575bc060ce160560 HTTP/1.1" 502 150 2025-12-17 23:00:06,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409061575bc060ce160560'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:06,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409061575bc060ce160560 2025-12-17 23:00:06,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409061575bc060ce160560 HTTP/1.1" 502 150 2025-12-17 23:00:06,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409061575bc060ce160560'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:06,875 - lava_callback - INFO - Received callback for node 694092e4575bc060ce160bb8 INFO: 172.21.0.1:41086 - "POST /node/694092e4575bc060ce160bb8 HTTP/1.0" 202 Accepted 2025-12-17 23:00:06,984 - lava_callback - INFO - Received callback for node 694092e7575bc060ce160bbd INFO: 172.21.0.1:41094 - "POST /node/694092e7575bc060ce160bbd HTTP/1.0" 202 Accepted 2025-12-17 23:00:07,524 - lava_callback - INFO - Received callback for node 694092e8575bc060ce160bc5 INFO: 172.21.0.1:41100 - "POST /node/694092e8575bc060ce160bc5 HTTP/1.0" 202 Accepted 2025-12-17 23:00:07,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903e575bc060ce160390 2025-12-17 23:00:07,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903e575bc060ce160390 HTTP/1.1" 502 150 2025-12-17 23:00:07,667 - lava_callback - ERROR - Error processing callback for node 6940903e575bc060ce160390: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940903e575bc060ce160390 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940903e575bc060ce160390 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940903e575bc060ce160390 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:07,669 - lava_callback - INFO - Starting async job submit for node 69409053575bc060ce1603d4 2025-12-17 23:00:07,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:07,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409053575bc060ce1603d4 HTTP/1.1" 502 150 2025-12-17 23:00:07,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409053575bc060ce1603d4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:07,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409053575bc060ce1603d4 2025-12-17 23:00:07,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409053575bc060ce1603d4 HTTP/1.1" 502 150 2025-12-17 23:00:07,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409053575bc060ce1603d4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:07,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903a575bc060ce16037f 2025-12-17 23:00:07,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903a575bc060ce16037f HTTP/1.1" 502 150 2025-12-17 23:00:07,778 - lava_callback - ERROR - Error processing callback for node 6940903a575bc060ce16037f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940903a575bc060ce16037f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940903a575bc060ce16037f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940903a575bc060ce16037f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:07,780 - lava_callback - INFO - Starting async job submit for node 69409054575bc060ce1603d7 2025-12-17 23:00:07,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:07,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409054575bc060ce1603d7 HTTP/1.1" 502 150 2025-12-17 23:00:07,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409054575bc060ce1603d7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:07,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409054575bc060ce1603d7 2025-12-17 23:00:07,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409054575bc060ce1603d7 HTTP/1.1" 502 150 2025-12-17 23:00:07,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409054575bc060ce1603d7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:07,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409049575bc060ce16039c 2025-12-17 23:00:07,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409049575bc060ce16039c HTTP/1.1" 502 150 2025-12-17 23:00:07,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409049575bc060ce16039c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:07,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904b575bc060ce16039f 2025-12-17 23:00:07,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904b575bc060ce16039f HTTP/1.1" 502 150 2025-12-17 23:00:07,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904b575bc060ce16039f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:07,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409051575bc060ce1603d1 2025-12-17 23:00:07,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409051575bc060ce1603d1 HTTP/1.1" 502 150 2025-12-17 23:00:07,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409051575bc060ce1603d1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:08,190 - lava_callback - INFO - Received callback for node 694092ea575bc060ce160bce INFO: 172.21.0.1:41108 - "POST /node/694092ea575bc060ce160bce HTTP/1.0" 202 Accepted 2025-12-17 23:00:08,379 - lava_callback - INFO - Received callback for node 694099b1575bc060ce161d96 INFO: 172.21.0.1:41112 - "POST /node/694099b1575bc060ce161d96 HTTP/1.0" 202 Accepted 2025-12-17 23:00:08,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903b575bc060ce160386 HTTP/1.1" 502 150 2025-12-17 23:00:08,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940903b575bc060ce160386'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:08,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409061575bc060ce160560 2025-12-17 23:00:08,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409061575bc060ce160560 HTTP/1.1" 502 150 2025-12-17 23:00:08,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409061575bc060ce160560'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:08,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904c575bc060ce1603a4 2025-12-17 23:00:08,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904c575bc060ce1603a4 HTTP/1.1" 502 150 2025-12-17 23:00:08,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904c575bc060ce1603a4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:08,831 - lava_callback - INFO - Received callback for node 694092eb575bc060ce160bd1 INFO: 172.21.0.1:41120 - "POST /node/694092eb575bc060ce160bd1 HTTP/1.0" 202 Accepted 2025-12-17 23:00:09,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409041575bc060ce160393 2025-12-17 23:00:09,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409041575bc060ce160393 HTTP/1.1" 502 150 2025-12-17 23:00:09,035 - lava_callback - ERROR - Error processing callback for node 69409041575bc060ce160393: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409041575bc060ce160393 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409041575bc060ce160393 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409041575bc060ce160393 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:09,037 - lava_callback - INFO - Starting async job submit for node 69409055575bc060ce1603e0 2025-12-17 23:00:09,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:09,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409055575bc060ce1603e0 HTTP/1.1" 502 150 2025-12-17 23:00:09,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409055575bc060ce1603e0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:09,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409055575bc060ce1603e0 2025-12-17 23:00:09,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409055575bc060ce1603e0 HTTP/1.1" 502 150 2025-12-17 23:00:09,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409055575bc060ce1603e0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:09,215 - lava_callback - INFO - Received callback for node 694111a8460bc8e265117c2f INFO: 172.21.0.1:41126 - "POST /node/694111a8460bc8e265117c2f HTTP/1.0" 202 Accepted 2025-12-17 23:00:09,479 - lava_callback - INFO - Received callback for node 694092ef575bc060ce160bda INFO: 172.21.0.1:41136 - "POST /node/694092ef575bc060ce160bda HTTP/1.0" 202 Accepted 2025-12-17 23:00:09,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409050575bc060ce1603ce 2025-12-17 23:00:09,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409050575bc060ce1603ce HTTP/1.1" 502 150 2025-12-17 23:00:09,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409050575bc060ce1603ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:09,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409053575bc060ce1603d4 2025-12-17 23:00:09,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409053575bc060ce1603d4 HTTP/1.1" 502 150 2025-12-17 23:00:09,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409053575bc060ce1603d4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:09,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409054575bc060ce1603d7 2025-12-17 23:00:09,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409054575bc060ce1603d7 HTTP/1.1" 502 150 2025-12-17 23:00:09,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409054575bc060ce1603d7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:10,127 - lava_callback - INFO - Received callback for node 694092f1575bc060ce160bdd INFO: 172.21.0.1:37862 - "POST /node/694092f1575bc060ce160bdd HTTP/1.0" 202 Accepted 2025-12-17 23:00:10,781 - lava_callback - INFO - Received callback for node 694092f3575bc060ce160be0 INFO: 172.21.0.1:37872 - "POST /node/694092f3575bc060ce160be0 HTTP/1.0" 202 Accepted 2025-12-17 23:00:10,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904d575bc060ce1603ac 2025-12-17 23:00:10,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904d575bc060ce1603ac HTTP/1.1" 502 150 2025-12-17 23:00:10,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904d575bc060ce1603ac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:11,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409055575bc060ce1603e0 2025-12-17 23:00:11,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409055575bc060ce1603e0 HTTP/1.1" 502 150 2025-12-17 23:00:11,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409055575bc060ce1603e0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:11,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905e575bc060ce160556 2025-12-17 23:00:11,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905e575bc060ce160556 HTTP/1.1" 502 150 2025-12-17 23:00:11,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905e575bc060ce160556'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:12,270 - lava_callback - INFO - Received callback for node 694092ed575bc060ce160bd4 INFO: 172.21.0.1:37884 - "POST /node/694092ed575bc060ce160bd4 HTTP/1.0" 202 Accepted 2025-12-17 23:00:12,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409061575bc060ce160560 2025-12-17 23:00:12,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409061575bc060ce160560 HTTP/1.1" 502 150 2025-12-17 23:00:12,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409061575bc060ce160560'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:12,995 - lava_callback - INFO - Received callback for node 694092ee575bc060ce160bd7 2025-12-17 23:00:13,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904f575bc060ce1603cb INFO: 172.21.0.1:37898 - "POST /node/694092ee575bc060ce160bd7 HTTP/1.0" 202 Accepted 2025-12-17 23:00:13,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904f575bc060ce1603cb HTTP/1.1" 502 150 2025-12-17 23:00:13,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940904f575bc060ce1603cb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:13,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409053575bc060ce1603d4 2025-12-17 23:00:13,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409053575bc060ce1603d4 HTTP/1.1" 502 150 2025-12-17 23:00:13,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409053575bc060ce1603d4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:13,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409054575bc060ce1603d7 2025-12-17 23:00:13,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409054575bc060ce1603d7 HTTP/1.1" 502 150 2025-12-17 23:00:13,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409054575bc060ce1603d7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:15,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409055575bc060ce1603e0 2025-12-17 23:00:15,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409055575bc060ce1603e0 HTTP/1.1" 502 150 2025-12-17 23:00:15,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409055575bc060ce1603e0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37914 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:00:15,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409051575bc060ce1603d1 2025-12-17 23:00:15,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409051575bc060ce1603d1 HTTP/1.1" 502 150 2025-12-17 23:00:15,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409051575bc060ce1603d1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37926 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:00:18,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409047575bc060ce160396 2025-12-17 23:00:18,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409047575bc060ce160396 HTTP/1.1" 502 150 2025-12-17 23:00:18,813 - lava_callback - ERROR - Error processing callback for node 69409047575bc060ce160396: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409047575bc060ce160396 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409047575bc060ce160396 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409047575bc060ce160396 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:18,815 - lava_callback - INFO - Starting async job submit for node 69409057575bc060ce1603e8 2025-12-17 23:00:18,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:18,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409057575bc060ce1603e8 HTTP/1.1" 502 150 2025-12-17 23:00:18,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409057575bc060ce1603e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:18,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409057575bc060ce1603e8 2025-12-17 23:00:18,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409057575bc060ce1603e8 HTTP/1.1" 502 150 2025-12-17 23:00:18,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409057575bc060ce1603e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:20,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409061575bc060ce160560 2025-12-17 23:00:20,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409061575bc060ce160560 HTTP/1.1" 502 150 2025-12-17 23:00:20,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409061575bc060ce160560'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:20,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409057575bc060ce1603e8 2025-12-17 23:00:20,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409057575bc060ce1603e8 HTTP/1.1" 502 150 2025-12-17 23:00:20,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409057575bc060ce1603e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:21,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903d575bc060ce16038d 2025-12-17 23:00:21,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903d575bc060ce16038d HTTP/1.1" 502 150 2025-12-17 23:00:21,492 - lava_callback - ERROR - Error processing callback for node 6940903d575bc060ce16038d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940903d575bc060ce16038d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940903d575bc060ce16038d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940903d575bc060ce16038d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:21,493 - lava_callback - INFO - Starting async job submit for node 69409058575bc060ce1603eb 2025-12-17 23:00:21,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:21,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409058575bc060ce1603eb HTTP/1.1" 502 150 2025-12-17 23:00:21,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409058575bc060ce1603eb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:21,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409058575bc060ce1603eb 2025-12-17 23:00:21,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409058575bc060ce1603eb HTTP/1.1" 502 150 2025-12-17 23:00:21,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409058575bc060ce1603eb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:21,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409048575bc060ce160399 2025-12-17 23:00:21,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409048575bc060ce160399 HTTP/1.1" 502 150 2025-12-17 23:00:21,569 - lava_callback - ERROR - Error processing callback for node 69409048575bc060ce160399: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409048575bc060ce160399 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409048575bc060ce160399 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409048575bc060ce160399 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:21,571 - lava_callback - INFO - Starting async job submit for node 69409059575bc060ce1603ee 2025-12-17 23:00:21,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:21,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409059575bc060ce1603ee HTTP/1.1" 502 150 2025-12-17 23:00:21,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409059575bc060ce1603ee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:21,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409059575bc060ce1603ee 2025-12-17 23:00:21,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409059575bc060ce1603ee HTTP/1.1" 502 150 2025-12-17 23:00:21,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409059575bc060ce1603ee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:21,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409053575bc060ce1603d4 2025-12-17 23:00:21,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409053575bc060ce1603d4 HTTP/1.1" 502 150 2025-12-17 23:00:21,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409053575bc060ce1603d4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:21,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409054575bc060ce1603d7 2025-12-17 23:00:21,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409054575bc060ce1603d7 HTTP/1.1" 502 150 2025-12-17 23:00:21,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409054575bc060ce1603d7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:23,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409055575bc060ce1603e0 2025-12-17 23:00:23,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409055575bc060ce1603e0 HTTP/1.1" 502 150 2025-12-17 23:00:23,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409055575bc060ce1603e0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:23,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409058575bc060ce1603eb 2025-12-17 23:00:23,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409058575bc060ce1603eb HTTP/1.1" 502 150 2025-12-17 23:00:23,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409058575bc060ce1603eb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:23,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409059575bc060ce1603ee 2025-12-17 23:00:23,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409059575bc060ce1603ee HTTP/1.1" 502 150 2025-12-17 23:00:23,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409059575bc060ce1603ee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:23,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409049575bc060ce16039c 2025-12-17 23:00:23,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409049575bc060ce16039c HTTP/1.1" 502 150 2025-12-17 23:00:23,856 - lava_callback - ERROR - Error processing callback for node 69409049575bc060ce16039c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409049575bc060ce16039c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409049575bc060ce16039c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409049575bc060ce16039c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:23,858 - lava_callback - INFO - Starting async job submit for node 6940905b575bc060ce160550 2025-12-17 23:00:23,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:23,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905b575bc060ce160550 HTTP/1.1" 502 150 2025-12-17 23:00:23,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905b575bc060ce160550'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:23,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905b575bc060ce160550 2025-12-17 23:00:23,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905b575bc060ce160550 HTTP/1.1" 502 150 2025-12-17 23:00:23,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905b575bc060ce160550'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:23,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904b575bc060ce16039f 2025-12-17 23:00:23,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904b575bc060ce16039f HTTP/1.1" 502 150 2025-12-17 23:00:23,885 - lava_callback - ERROR - Error processing callback for node 6940904b575bc060ce16039f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940904b575bc060ce16039f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940904b575bc060ce16039f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940904b575bc060ce16039f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:23,887 - lava_callback - INFO - Starting async job submit for node 6940905d575bc060ce160553 2025-12-17 23:00:23,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:23,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905d575bc060ce160553 HTTP/1.1" 502 150 2025-12-17 23:00:23,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905d575bc060ce160553'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:23,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905d575bc060ce160553 2025-12-17 23:00:23,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905d575bc060ce160553 HTTP/1.1" 502 150 2025-12-17 23:00:23,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905d575bc060ce160553'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:24,501 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940903b575bc060ce160386 2025-12-17 23:00:24,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940903b575bc060ce160386 HTTP/1.1" 502 150 2025-12-17 23:00:24,502 - lava_callback - ERROR - Error processing callback for node 6940903b575bc060ce160386: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940903b575bc060ce160386 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940903b575bc060ce160386 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940903b575bc060ce160386 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:24,504 - lava_callback - INFO - Starting async job submit for node 6940905f575bc060ce160559 2025-12-17 23:00:24,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:24,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905f575bc060ce160559 HTTP/1.1" 502 150 2025-12-17 23:00:24,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905f575bc060ce160559'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:24,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905f575bc060ce160559 2025-12-17 23:00:24,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905f575bc060ce160559 HTTP/1.1" 502 150 2025-12-17 23:00:24,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905f575bc060ce160559'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:24,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904c575bc060ce1603a4 2025-12-17 23:00:24,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904c575bc060ce1603a4 HTTP/1.1" 502 150 2025-12-17 23:00:24,788 - lava_callback - ERROR - Error processing callback for node 6940904c575bc060ce1603a4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940904c575bc060ce1603a4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940904c575bc060ce1603a4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940904c575bc060ce1603a4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:24,791 - lava_callback - INFO - Starting async job submit for node 69409062575bc060ce16056b 2025-12-17 23:00:24,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:24,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409062575bc060ce16056b HTTP/1.1" 502 150 2025-12-17 23:00:24,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409062575bc060ce16056b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:24,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409062575bc060ce16056b 2025-12-17 23:00:24,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409062575bc060ce16056b HTTP/1.1" 502 150 2025-12-17 23:00:24,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409062575bc060ce16056b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:24,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409057575bc060ce1603e8 2025-12-17 23:00:24,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409057575bc060ce1603e8 HTTP/1.1" 502 150 2025-12-17 23:00:24,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409057575bc060ce1603e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:25,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409050575bc060ce1603ce 2025-12-17 23:00:25,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409050575bc060ce1603ce HTTP/1.1" 502 150 2025-12-17 23:00:25,654 - lava_callback - ERROR - Error processing callback for node 69409050575bc060ce1603ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409050575bc060ce1603ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409050575bc060ce1603ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409050575bc060ce1603ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:25,656 - lava_callback - INFO - Starting async job submit for node 69409064575bc060ce160577 2025-12-17 23:00:25,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:25,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409064575bc060ce160577 HTTP/1.1" 502 150 2025-12-17 23:00:25,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409064575bc060ce160577'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:25,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409064575bc060ce160577 2025-12-17 23:00:25,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409064575bc060ce160577 HTTP/1.1" 502 150 2025-12-17 23:00:25,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409064575bc060ce160577'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:25,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905b575bc060ce160550 2025-12-17 23:00:25,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905b575bc060ce160550 HTTP/1.1" 502 150 2025-12-17 23:00:25,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905b575bc060ce160550'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:25,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905d575bc060ce160553 2025-12-17 23:00:25,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905d575bc060ce160553 HTTP/1.1" 502 150 2025-12-17 23:00:25,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905d575bc060ce160553'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:26,256 - lava_callback - INFO - Received callback for node 694092f6575bc060ce160be9 INFO: 172.21.0.1:45980 - "POST /node/694092f6575bc060ce160be9 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:45986 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:00:26,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905f575bc060ce160559 2025-12-17 23:00:26,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905f575bc060ce160559 HTTP/1.1" 502 150 2025-12-17 23:00:26,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905f575bc060ce160559'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:26,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409062575bc060ce16056b 2025-12-17 23:00:26,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409062575bc060ce16056b HTTP/1.1" 502 150 2025-12-17 23:00:26,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409062575bc060ce16056b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:26,892 - lava_callback - INFO - Received callback for node 694092f8575bc060ce160bee INFO: 172.21.0.1:45998 - "POST /node/694092f8575bc060ce160bee HTTP/1.0" 202 Accepted 2025-12-17 23:00:26,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904d575bc060ce1603ac 2025-12-17 23:00:26,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904d575bc060ce1603ac HTTP/1.1" 502 150 2025-12-17 23:00:26,985 - lava_callback - ERROR - Error processing callback for node 6940904d575bc060ce1603ac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940904d575bc060ce1603ac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940904d575bc060ce1603ac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940904d575bc060ce1603ac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:26,987 - lava_callback - INFO - Starting async job submit for node 69409063575bc060ce160572 2025-12-17 23:00:26,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:26,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409063575bc060ce160572 HTTP/1.1" 502 150 2025-12-17 23:00:26,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409063575bc060ce160572'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:26,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409063575bc060ce160572 2025-12-17 23:00:26,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409063575bc060ce160572 HTTP/1.1" 502 150 2025-12-17 23:00:26,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409063575bc060ce160572'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:27,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409058575bc060ce1603eb 2025-12-17 23:00:27,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409058575bc060ce1603eb HTTP/1.1" 502 150 2025-12-17 23:00:27,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409058575bc060ce1603eb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:27,538 - lava_callback - INFO - Received callback for node 694092fa575bc060ce160bf1 INFO: 172.21.0.1:46012 - "POST /node/694092fa575bc060ce160bf1 HTTP/1.0" 202 Accepted 2025-12-17 23:00:27,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409059575bc060ce1603ee 2025-12-17 23:00:27,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409059575bc060ce1603ee HTTP/1.1" 502 150 2025-12-17 23:00:27,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409059575bc060ce1603ee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:27,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409064575bc060ce160577 2025-12-17 23:00:27,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409064575bc060ce160577 HTTP/1.1" 502 150 2025-12-17 23:00:27,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409064575bc060ce160577'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:27,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905e575bc060ce160556 2025-12-17 23:00:27,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905e575bc060ce160556 HTTP/1.1" 502 150 2025-12-17 23:00:27,739 - lava_callback - ERROR - Error processing callback for node 6940905e575bc060ce160556: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940905e575bc060ce160556 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940905e575bc060ce160556 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940905e575bc060ce160556 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:27,740 - lava_callback - INFO - Starting async job submit for node 694101ba460bc8e265116a92 2025-12-17 23:00:27,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:27,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694101ba460bc8e265116a92 HTTP/1.1" 502 150 2025-12-17 23:00:27,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694101ba460bc8e265116a92'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:27,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694101ba460bc8e265116a92 2025-12-17 23:00:27,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694101ba460bc8e265116a92 HTTP/1.1" 502 150 2025-12-17 23:00:27,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694101ba460bc8e265116a92'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:28,177 - lava_callback - INFO - Received callback for node 694092fc575bc060ce160bf4 INFO: 172.21.0.1:46026 - "POST /node/694092fc575bc060ce160bf4 HTTP/1.0" 202 Accepted 2025-12-17 23:00:28,840 - lava_callback - INFO - Received callback for node 69409308575bc060ce160c08 INFO: 172.21.0.1:46038 - "POST /node/69409308575bc060ce160c08 HTTP/1.0" 202 Accepted 2025-12-17 23:00:28,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409063575bc060ce160572 2025-12-17 23:00:28,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409063575bc060ce160572 HTTP/1.1" 502 150 2025-12-17 23:00:28,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409063575bc060ce160572'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:29,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940904f575bc060ce1603cb 2025-12-17 23:00:29,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940904f575bc060ce1603cb HTTP/1.1" 502 150 2025-12-17 23:00:29,007 - lava_callback - ERROR - Error processing callback for node 6940904f575bc060ce1603cb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940904f575bc060ce1603cb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940904f575bc060ce1603cb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940904f575bc060ce1603cb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:29,008 - lava_callback - INFO - Starting async job submit for node 69409067575bc060ce16057d 2025-12-17 23:00:29,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:29,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409067575bc060ce16057d HTTP/1.1" 502 150 2025-12-17 23:00:29,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409067575bc060ce16057d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:29,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409067575bc060ce16057d 2025-12-17 23:00:29,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409067575bc060ce16057d HTTP/1.1" 502 150 2025-12-17 23:00:29,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409067575bc060ce16057d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:29,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694101ba460bc8e265116a92 2025-12-17 23:00:29,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694101ba460bc8e265116a92 HTTP/1.1" 502 150 2025-12-17 23:00:29,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694101ba460bc8e265116a92'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:29,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905b575bc060ce160550 2025-12-17 23:00:29,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905b575bc060ce160550 HTTP/1.1" 502 150 2025-12-17 23:00:29,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905b575bc060ce160550'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:29,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905d575bc060ce160553 2025-12-17 23:00:29,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905d575bc060ce160553 HTTP/1.1" 502 150 2025-12-17 23:00:29,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905d575bc060ce160553'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:30,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905f575bc060ce160559 2025-12-17 23:00:30,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905f575bc060ce160559 HTTP/1.1" 502 150 2025-12-17 23:00:30,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905f575bc060ce160559'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:30,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409062575bc060ce16056b 2025-12-17 23:00:30,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409062575bc060ce16056b HTTP/1.1" 502 150 2025-12-17 23:00:30,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409062575bc060ce16056b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:31,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409067575bc060ce16057d 2025-12-17 23:00:31,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409067575bc060ce16057d HTTP/1.1" 502 150 2025-12-17 23:00:31,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409067575bc060ce16057d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:31,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409064575bc060ce160577 2025-12-17 23:00:31,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409064575bc060ce160577 HTTP/1.1" 502 150 2025-12-17 23:00:31,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409064575bc060ce160577'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:31,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409051575bc060ce1603d1 2025-12-17 23:00:31,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409051575bc060ce1603d1 HTTP/1.1" 502 150 2025-12-17 23:00:31,972 - lava_callback - ERROR - Error processing callback for node 69409051575bc060ce1603d1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409051575bc060ce1603d1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409051575bc060ce1603d1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409051575bc060ce1603d1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:31,974 - lava_callback - INFO - Starting async job submit for node 69409066575bc060ce16057a 2025-12-17 23:00:31,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:31,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409066575bc060ce16057a HTTP/1.1" 502 150 2025-12-17 23:00:31,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409066575bc060ce16057a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:31,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409066575bc060ce16057a 2025-12-17 23:00:31,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409066575bc060ce16057a HTTP/1.1" 502 150 2025-12-17 23:00:31,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409066575bc060ce16057a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:32,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409057575bc060ce1603e8 2025-12-17 23:00:32,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409063575bc060ce160572 2025-12-17 23:00:32,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409063575bc060ce160572 HTTP/1.1" 502 150 2025-12-17 23:00:32,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409063575bc060ce160572'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:33,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694101ba460bc8e265116a92 2025-12-17 23:00:33,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694101ba460bc8e265116a92 HTTP/1.1" 502 150 2025-12-17 23:00:33,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694101ba460bc8e265116a92'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:33,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409066575bc060ce16057a 2025-12-17 23:00:33,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409066575bc060ce16057a HTTP/1.1" 502 150 2025-12-17 23:00:33,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409066575bc060ce16057a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:35,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409067575bc060ce16057d 2025-12-17 23:00:35,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409067575bc060ce16057d HTTP/1.1" 502 150 2025-12-17 23:00:35,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409067575bc060ce16057d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:35,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409058575bc060ce1603eb 2025-12-17 23:00:35,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409058575bc060ce1603eb HTTP/1.1" 502 150 2025-12-17 23:00:35,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409058575bc060ce1603eb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:35,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409059575bc060ce1603ee 2025-12-17 23:00:35,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409059575bc060ce1603ee HTTP/1.1" 502 150 2025-12-17 23:00:35,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409059575bc060ce1603ee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:35,808 - lava_callback - INFO - Received callback for node 69409301575bc060ce160bff INFO: 172.21.0.1:58896 - "POST /node/69409301575bc060ce160bff HTTP/1.0" 202 Accepted 2025-12-17 23:00:36,472 - lava_callback - INFO - Received callback for node 69409304575bc060ce160c02 INFO: 172.21.0.1:58906 - "POST /node/69409304575bc060ce160c02 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:58922 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:00:36,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409061575bc060ce160560 2025-12-17 23:00:36,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409061575bc060ce160560 HTTP/1.1" 502 150 2025-12-17 23:00:36,667 - lava_callback - ERROR - Error processing callback for node 69409061575bc060ce160560: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409061575bc060ce160560 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409061575bc060ce160560 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409061575bc060ce160560 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:36,671 - lava_callback - INFO - Starting async job submit for node 69409068575bc060ce160580 2025-12-17 23:00:36,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:36,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409068575bc060ce160580 HTTP/1.1" 502 150 2025-12-17 23:00:36,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409068575bc060ce160580'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:36,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409068575bc060ce160580 2025-12-17 23:00:36,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409068575bc060ce160580 HTTP/1.1" 502 150 2025-12-17 23:00:36,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409068575bc060ce160580'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:37,146 - lava_callback - INFO - Received callback for node 69409306575bc060ce160c05 INFO: 172.21.0.1:58924 - "POST /node/69409306575bc060ce160c05 HTTP/1.0" 202 Accepted 2025-12-17 23:00:37,487 - lava_callback - INFO - Received callback for node 69409a1a575bc060ce161ee2 INFO: 172.21.0.1:58930 - "POST /node/69409a1a575bc060ce161ee2 HTTP/1.0" 202 Accepted 2025-12-17 23:00:37,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409053575bc060ce1603d4 2025-12-17 23:00:37,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409053575bc060ce1603d4 HTTP/1.1" 502 150 2025-12-17 23:00:37,708 - lava_callback - ERROR - Error processing callback for node 69409053575bc060ce1603d4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409053575bc060ce1603d4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409053575bc060ce1603d4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409053575bc060ce1603d4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:37,710 - lava_callback - INFO - Starting async job submit for node 69409069575bc060ce160583 2025-12-17 23:00:37,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:37,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409069575bc060ce160583 HTTP/1.1" 502 150 2025-12-17 23:00:37,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409069575bc060ce160583'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:37,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409069575bc060ce160583 2025-12-17 23:00:37,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409069575bc060ce160583 HTTP/1.1" 502 150 2025-12-17 23:00:37,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409069575bc060ce160583'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:37,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409054575bc060ce1603d7 2025-12-17 23:00:37,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409054575bc060ce1603d7 HTTP/1.1" 502 150 2025-12-17 23:00:37,812 - lava_callback - ERROR - Error processing callback for node 69409054575bc060ce1603d7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409054575bc060ce1603d7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409054575bc060ce1603d7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409054575bc060ce1603d7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:37,814 - lava_callback - INFO - Starting async job submit for node 6940906b575bc060ce160586 2025-12-17 23:00:37,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:37,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940906b575bc060ce160586 HTTP/1.1" 502 150 2025-12-17 23:00:37,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940906b575bc060ce160586'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:37,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940906b575bc060ce160586 2025-12-17 23:00:37,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940906b575bc060ce160586 HTTP/1.1" 502 150 2025-12-17 23:00:37,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940906b575bc060ce160586'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:37,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905b575bc060ce160550 2025-12-17 23:00:37,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905b575bc060ce160550 HTTP/1.1" 502 150 2025-12-17 23:00:37,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905b575bc060ce160550'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:37,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905d575bc060ce160553 2025-12-17 23:00:37,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905d575bc060ce160553 HTTP/1.1" 502 150 2025-12-17 23:00:37,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905d575bc060ce160553'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:37,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409066575bc060ce16057a 2025-12-17 23:00:37,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409066575bc060ce16057a HTTP/1.1" 502 150 2025-12-17 23:00:37,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409066575bc060ce16057a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:38,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905f575bc060ce160559 2025-12-17 23:00:38,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905f575bc060ce160559 HTTP/1.1" 502 150 2025-12-17 23:00:38,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940905f575bc060ce160559'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:38,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409068575bc060ce160580 2025-12-17 23:00:38,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409068575bc060ce160580 HTTP/1.1" 502 150 2025-12-17 23:00:38,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409068575bc060ce160580'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:38,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409062575bc060ce16056b 2025-12-17 23:00:38,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409062575bc060ce16056b HTTP/1.1" 502 150 2025-12-17 23:00:38,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409062575bc060ce16056b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:39,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409055575bc060ce1603e0 2025-12-17 23:00:39,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409055575bc060ce1603e0 HTTP/1.1" 502 150 2025-12-17 23:00:39,049 - lava_callback - ERROR - Error processing callback for node 69409055575bc060ce1603e0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409055575bc060ce1603e0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409055575bc060ce1603e0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409055575bc060ce1603e0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:39,051 - lava_callback - INFO - Starting async job submit for node 69409072575bc060ce16058f 2025-12-17 23:00:39,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:39,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409072575bc060ce16058f HTTP/1.1" 502 150 2025-12-17 23:00:39,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409072575bc060ce16058f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:39,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409072575bc060ce16058f 2025-12-17 23:00:39,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409072575bc060ce16058f HTTP/1.1" 502 150 2025-12-17 23:00:39,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409072575bc060ce16058f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:39,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409064575bc060ce160577 2025-12-17 23:00:39,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409064575bc060ce160577 HTTP/1.1" 502 150 2025-12-17 23:00:39,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409064575bc060ce160577'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:39,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409069575bc060ce160583 2025-12-17 23:00:39,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409069575bc060ce160583 HTTP/1.1" 502 150 2025-12-17 23:00:39,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409069575bc060ce160583'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:39,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940906b575bc060ce160586 2025-12-17 23:00:39,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940906b575bc060ce160586 HTTP/1.1" 502 150 2025-12-17 23:00:39,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940906b575bc060ce160586'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:40,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409063575bc060ce160572 2025-12-17 23:00:40,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409063575bc060ce160572 HTTP/1.1" 502 150 2025-12-17 23:00:40,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409063575bc060ce160572'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:41,059 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409072575bc060ce16058f 2025-12-17 23:00:41,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409072575bc060ce16058f HTTP/1.1" 502 150 2025-12-17 23:00:41,060 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409072575bc060ce16058f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:41,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694101ba460bc8e265116a92 2025-12-17 23:00:41,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694101ba460bc8e265116a92 HTTP/1.1" 502 150 2025-12-17 23:00:41,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694101ba460bc8e265116a92'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:42,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409068575bc060ce160580 2025-12-17 23:00:42,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409068575bc060ce160580 HTTP/1.1" 502 150 2025-12-17 23:00:42,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409068575bc060ce160580'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:43,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409067575bc060ce16057d 2025-12-17 23:00:43,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409067575bc060ce16057d HTTP/1.1" 502 150 2025-12-17 23:00:43,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409067575bc060ce16057d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:43,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409069575bc060ce160583 2025-12-17 23:00:43,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409069575bc060ce160583 HTTP/1.1" 502 150 2025-12-17 23:00:43,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409069575bc060ce160583'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:43,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940906b575bc060ce160586 2025-12-17 23:00:43,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940906b575bc060ce160586 HTTP/1.1" 502 150 2025-12-17 23:00:43,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940906b575bc060ce160586'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:45,060 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409072575bc060ce16058f 2025-12-17 23:00:45,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409066575bc060ce16057a 2025-12-17 23:00:45,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409066575bc060ce16057a HTTP/1.1" 502 150 2025-12-17 23:00:45,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409066575bc060ce16057a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:46,249 - lava_callback - INFO - Received callback for node 6940930b575bc060ce160c13 INFO: 172.21.0.1:45112 - "POST /node/6940930b575bc060ce160c13 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:45124 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:00:46,881 - lava_callback - INFO - Received callback for node 6940930d575bc060ce160c1a INFO: 172.21.0.1:45126 - "POST /node/6940930d575bc060ce160c1a HTTP/1.0" 202 Accepted 2025-12-17 23:00:47,518 - lava_callback - INFO - Received callback for node 6940930f575bc060ce160c1d INFO: 172.21.0.1:45140 - "POST /node/6940930f575bc060ce160c1d HTTP/1.0" 202 Accepted 2025-12-17 23:00:48,170 - lava_callback - INFO - Received callback for node 69409311575bc060ce160c20 INFO: 172.21.0.1:45152 - "POST /node/69409311575bc060ce160c20 HTTP/1.0" 202 Accepted 2025-12-17 23:00:48,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409057575bc060ce1603e8 HTTP/1.1" 502 150 2025-12-17 23:00:48,185 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409057575bc060ce1603e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:48,821 - lava_callback - INFO - Received callback for node 69409313575bc060ce160c23 INFO: 172.21.0.1:45156 - "POST /node/69409313575bc060ce160c23 HTTP/1.0" 202 Accepted 2025-12-17 23:00:49,466 - lava_callback - INFO - Received callback for node 69409315575bc060ce160c26 INFO: 172.21.0.1:45160 - "POST /node/69409315575bc060ce160c26 HTTP/1.0" 202 Accepted 2025-12-17 23:00:50,116 - lava_callback - INFO - Received callback for node 6940931b575bc060ce160c29 INFO: 172.21.0.1:44340 - "POST /node/6940931b575bc060ce160c29 HTTP/1.0" 202 Accepted 2025-12-17 23:00:50,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409068575bc060ce160580 2025-12-17 23:00:50,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409068575bc060ce160580 HTTP/1.1" 502 150 2025-12-17 23:00:50,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409068575bc060ce160580'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:51,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409058575bc060ce1603eb 2025-12-17 23:00:51,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409058575bc060ce1603eb HTTP/1.1" 502 150 2025-12-17 23:00:51,528 - lava_callback - ERROR - Error processing callback for node 69409058575bc060ce1603eb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409058575bc060ce1603eb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409058575bc060ce1603eb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409058575bc060ce1603eb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:51,529 - lava_callback - INFO - Starting async job submit for node 69409074575bc060ce160592 2025-12-17 23:00:51,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:51,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409074575bc060ce160592 HTTP/1.1" 502 150 2025-12-17 23:00:51,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409074575bc060ce160592'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:51,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409074575bc060ce160592 2025-12-17 23:00:51,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409074575bc060ce160592 HTTP/1.1" 502 150 2025-12-17 23:00:51,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409074575bc060ce160592'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:51,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409059575bc060ce1603ee 2025-12-17 23:00:51,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409059575bc060ce1603ee HTTP/1.1" 502 150 2025-12-17 23:00:51,584 - lava_callback - ERROR - Error processing callback for node 69409059575bc060ce1603ee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409059575bc060ce1603ee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409059575bc060ce1603ee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409059575bc060ce1603ee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:51,585 - lava_callback - INFO - Starting async job submit for node 6940906d575bc060ce160589 2025-12-17 23:00:51,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:51,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940906d575bc060ce160589 HTTP/1.1" 502 150 2025-12-17 23:00:51,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940906d575bc060ce160589'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:51,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940906d575bc060ce160589 2025-12-17 23:00:51,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940906d575bc060ce160589 HTTP/1.1" 502 150 2025-12-17 23:00:51,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940906d575bc060ce160589'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:51,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409069575bc060ce160583 2025-12-17 23:00:51,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409069575bc060ce160583 HTTP/1.1" 502 150 2025-12-17 23:00:51,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409069575bc060ce160583'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:51,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940906b575bc060ce160586 2025-12-17 23:00:51,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940906b575bc060ce160586 HTTP/1.1" 502 150 2025-12-17 23:00:51,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940906b575bc060ce160586'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:53,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409074575bc060ce160592 2025-12-17 23:00:53,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409074575bc060ce160592 HTTP/1.1" 502 150 2025-12-17 23:00:53,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409074575bc060ce160592'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:53,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940906d575bc060ce160589 2025-12-17 23:00:53,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940906d575bc060ce160589 HTTP/1.1" 502 150 2025-12-17 23:00:53,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940906d575bc060ce160589'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:53,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905b575bc060ce160550 2025-12-17 23:00:53,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905b575bc060ce160550 HTTP/1.1" 502 150 2025-12-17 23:00:53,877 - lava_callback - ERROR - Error processing callback for node 6940905b575bc060ce160550: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940905b575bc060ce160550 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940905b575bc060ce160550 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940905b575bc060ce160550 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:53,880 - lava_callback - INFO - Starting async job submit for node 69409070575bc060ce16058c 2025-12-17 23:00:53,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:53,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409070575bc060ce16058c HTTP/1.1" 502 150 2025-12-17 23:00:53,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409070575bc060ce16058c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:53,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409070575bc060ce16058c 2025-12-17 23:00:53,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409070575bc060ce16058c HTTP/1.1" 502 150 2025-12-17 23:00:53,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409070575bc060ce16058c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:53,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905d575bc060ce160553 2025-12-17 23:00:53,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905d575bc060ce160553 HTTP/1.1" 502 150 2025-12-17 23:00:53,902 - lava_callback - ERROR - Error processing callback for node 6940905d575bc060ce160553: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940905d575bc060ce160553 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940905d575bc060ce160553 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940905d575bc060ce160553 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:53,904 - lava_callback - INFO - Starting async job submit for node 69409076575bc060ce16059b 2025-12-17 23:00:53,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:53,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409076575bc060ce16059b HTTP/1.1" 502 150 2025-12-17 23:00:53,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409076575bc060ce16059b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:53,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409076575bc060ce16059b 2025-12-17 23:00:53,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409076575bc060ce16059b HTTP/1.1" 502 150 2025-12-17 23:00:53,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409076575bc060ce16059b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:54,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940905f575bc060ce160559 2025-12-17 23:00:54,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940905f575bc060ce160559 HTTP/1.1" 502 150 2025-12-17 23:00:54,517 - lava_callback - ERROR - Error processing callback for node 6940905f575bc060ce160559: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940905f575bc060ce160559 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940905f575bc060ce160559 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940905f575bc060ce160559 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:54,519 - lava_callback - INFO - Starting async job submit for node 69409078575bc060ce1605a6 2025-12-17 23:00:54,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:54,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409078575bc060ce1605a6 HTTP/1.1" 502 150 2025-12-17 23:00:54,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409078575bc060ce1605a6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:54,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409078575bc060ce1605a6 2025-12-17 23:00:54,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409078575bc060ce1605a6 HTTP/1.1" 502 150 2025-12-17 23:00:54,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409078575bc060ce1605a6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:54,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409062575bc060ce16056b 2025-12-17 23:00:54,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409062575bc060ce16056b HTTP/1.1" 502 150 2025-12-17 23:00:54,803 - lava_callback - ERROR - Error processing callback for node 69409062575bc060ce16056b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409062575bc060ce16056b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409062575bc060ce16056b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409062575bc060ce16056b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:54,805 - lava_callback - INFO - Starting async job submit for node 6940907a575bc060ce1605ad 2025-12-17 23:00:54,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:54,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907a575bc060ce1605ad HTTP/1.1" 502 150 2025-12-17 23:00:54,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940907a575bc060ce1605ad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:54,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940907a575bc060ce1605ad 2025-12-17 23:00:54,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907a575bc060ce1605ad HTTP/1.1" 502 150 2025-12-17 23:00:54,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940907a575bc060ce1605ad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:55,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409064575bc060ce160577 2025-12-17 23:00:55,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409064575bc060ce160577 HTTP/1.1" 502 150 2025-12-17 23:00:55,670 - lava_callback - ERROR - Error processing callback for node 69409064575bc060ce160577: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409064575bc060ce160577 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409064575bc060ce160577 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409064575bc060ce160577 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:55,673 - lava_callback - INFO - Starting async job submit for node 6940907c575bc060ce1605b0 2025-12-17 23:00:55,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:55,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907c575bc060ce1605b0 HTTP/1.1" 502 150 2025-12-17 23:00:55,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940907c575bc060ce1605b0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:55,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940907c575bc060ce1605b0 2025-12-17 23:00:55,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907c575bc060ce1605b0 HTTP/1.1" 502 150 2025-12-17 23:00:55,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940907c575bc060ce1605b0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:55,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409070575bc060ce16058c 2025-12-17 23:00:55,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409070575bc060ce16058c HTTP/1.1" 502 150 2025-12-17 23:00:55,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409070575bc060ce16058c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:55,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409076575bc060ce16059b 2025-12-17 23:00:55,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409076575bc060ce16059b HTTP/1.1" 502 150 2025-12-17 23:00:55,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409076575bc060ce16059b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44346 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:00:56,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409078575bc060ce1605a6 2025-12-17 23:00:56,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409078575bc060ce1605a6 HTTP/1.1" 502 150 2025-12-17 23:00:56,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409078575bc060ce1605a6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:56,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940907a575bc060ce1605ad 2025-12-17 23:00:56,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907a575bc060ce1605ad HTTP/1.1" 502 150 2025-12-17 23:00:56,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940907a575bc060ce1605ad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:56,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409063575bc060ce160572 2025-12-17 23:00:57,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409063575bc060ce160572 HTTP/1.1" 502 150 2025-12-17 23:00:57,000 - lava_callback - ERROR - Error processing callback for node 69409063575bc060ce160572: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409063575bc060ce160572 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409063575bc060ce160572 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409063575bc060ce160572 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:57,002 - lava_callback - INFO - Starting async job submit for node 69409082575bc060ce1605b9 2025-12-17 23:00:57,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:57,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409082575bc060ce1605b9 HTTP/1.1" 502 150 2025-12-17 23:00:57,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409082575bc060ce1605b9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:57,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409082575bc060ce1605b9 2025-12-17 23:00:57,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409082575bc060ce1605b9 HTTP/1.1" 502 150 2025-12-17 23:00:57,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409082575bc060ce1605b9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:57,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409074575bc060ce160592 2025-12-17 23:00:57,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409074575bc060ce160592 HTTP/1.1" 502 150 2025-12-17 23:00:57,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409074575bc060ce160592'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:57,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940906d575bc060ce160589 2025-12-17 23:00:57,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940906d575bc060ce160589 HTTP/1.1" 502 150 2025-12-17 23:00:57,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940906d575bc060ce160589'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:57,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940907c575bc060ce1605b0 2025-12-17 23:00:57,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907c575bc060ce1605b0 HTTP/1.1" 502 150 2025-12-17 23:00:57,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940907c575bc060ce1605b0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:57,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694101ba460bc8e265116a92 2025-12-17 23:00:57,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694101ba460bc8e265116a92 HTTP/1.1" 502 150 2025-12-17 23:00:57,774 - lava_callback - ERROR - Error processing callback for node 694101ba460bc8e265116a92: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694101ba460bc8e265116a92 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694101ba460bc8e265116a92 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694101ba460bc8e265116a92 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:57,776 - lava_callback - INFO - Starting async job submit for node 6940908f575bc060ce1605e5 2025-12-17 23:00:57,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:57,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908f575bc060ce1605e5 HTTP/1.1" 502 150 2025-12-17 23:00:57,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940908f575bc060ce1605e5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:57,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940908f575bc060ce1605e5 2025-12-17 23:00:57,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908f575bc060ce1605e5 HTTP/1.1" 502 150 2025-12-17 23:00:57,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940908f575bc060ce1605e5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:59,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409082575bc060ce1605b9 2025-12-17 23:00:59,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409067575bc060ce16057d 2025-12-17 23:00:59,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409067575bc060ce16057d HTTP/1.1" 502 150 2025-12-17 23:00:59,022 - lava_callback - ERROR - Error processing callback for node 69409067575bc060ce16057d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409067575bc060ce16057d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409067575bc060ce16057d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409067575bc060ce16057d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:00:59,023 - lava_callback - INFO - Starting async job submit for node 69409088575bc060ce1605bc 2025-12-17 23:00:59,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:00:59,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409088575bc060ce1605bc HTTP/1.1" 502 150 2025-12-17 23:00:59,062 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409088575bc060ce1605bc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:59,062 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409088575bc060ce1605bc 2025-12-17 23:00:59,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409088575bc060ce1605bc HTTP/1.1" 502 150 2025-12-17 23:00:59,062 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409088575bc060ce1605bc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:59,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940908f575bc060ce1605e5 2025-12-17 23:00:59,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908f575bc060ce1605e5 HTTP/1.1" 502 150 2025-12-17 23:00:59,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940908f575bc060ce1605e5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:59,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409070575bc060ce16058c 2025-12-17 23:00:59,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409070575bc060ce16058c HTTP/1.1" 502 150 2025-12-17 23:00:59,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409070575bc060ce16058c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:00:59,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409076575bc060ce16059b 2025-12-17 23:00:59,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409076575bc060ce16059b HTTP/1.1" 502 150 2025-12-17 23:00:59,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409076575bc060ce16059b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:00,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409072575bc060ce16058f HTTP/1.1" 502 150 2025-12-17 23:01:00,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409072575bc060ce16058f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:00,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409078575bc060ce1605a6 2025-12-17 23:01:00,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409078575bc060ce1605a6 HTTP/1.1" 502 150 2025-12-17 23:01:00,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409078575bc060ce1605a6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:00,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940907a575bc060ce1605ad 2025-12-17 23:01:00,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907a575bc060ce1605ad HTTP/1.1" 502 150 2025-12-17 23:01:00,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940907a575bc060ce1605ad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:01,063 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409088575bc060ce1605bc 2025-12-17 23:01:01,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409088575bc060ce1605bc HTTP/1.1" 502 150 2025-12-17 23:01:01,064 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409088575bc060ce1605bc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:01,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940907c575bc060ce1605b0 2025-12-17 23:01:01,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907c575bc060ce1605b0 HTTP/1.1" 502 150 2025-12-17 23:01:01,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940907c575bc060ce1605b0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:01,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409066575bc060ce16057a 2025-12-17 23:01:02,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409066575bc060ce16057a HTTP/1.1" 502 150 2025-12-17 23:01:02,000 - lava_callback - ERROR - Error processing callback for node 69409066575bc060ce16057a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409066575bc060ce16057a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409066575bc060ce16057a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409066575bc060ce16057a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:02,002 - lava_callback - INFO - Starting async job submit for node 6940908b575bc060ce1605d9 2025-12-17 23:01:02,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:02,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908b575bc060ce1605d9 HTTP/1.1" 502 150 2025-12-17 23:01:02,040 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940908b575bc060ce1605d9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:02,041 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940908b575bc060ce1605d9 2025-12-17 23:01:02,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908b575bc060ce1605d9 HTTP/1.1" 502 150 2025-12-17 23:01:02,041 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940908b575bc060ce1605d9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49572 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:01:03,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940908f575bc060ce1605e5 2025-12-17 23:01:03,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908f575bc060ce1605e5 HTTP/1.1" 502 150 2025-12-17 23:01:03,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940908f575bc060ce1605e5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:04,042 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940908b575bc060ce1605d9 2025-12-17 23:01:04,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908b575bc060ce1605d9 HTTP/1.1" 502 150 2025-12-17 23:01:04,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940908b575bc060ce1605d9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:04,185 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409057575bc060ce1603e8 2025-12-17 23:01:04,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409057575bc060ce1603e8 HTTP/1.1" 502 150 2025-12-17 23:01:04,187 - lava_callback - ERROR - Error processing callback for node 69409057575bc060ce1603e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409057575bc060ce1603e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409057575bc060ce1603e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409057575bc060ce1603e8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:04,189 - lava_callback - INFO - Starting async job submit for node 6940908d575bc060ce1605e2 2025-12-17 23:01:04,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:04,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908d575bc060ce1605e2 HTTP/1.1" 502 150 2025-12-17 23:01:04,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940908d575bc060ce1605e2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:04,195 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940908d575bc060ce1605e2 2025-12-17 23:01:04,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908d575bc060ce1605e2 HTTP/1.1" 502 150 2025-12-17 23:01:04,196 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940908d575bc060ce1605e2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:05,064 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409088575bc060ce1605bc 2025-12-17 23:01:05,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409088575bc060ce1605bc HTTP/1.1" 502 150 2025-12-17 23:01:05,065 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409088575bc060ce1605bc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:05,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409074575bc060ce160592 2025-12-17 23:01:05,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409074575bc060ce160592 HTTP/1.1" 502 150 2025-12-17 23:01:05,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409074575bc060ce160592'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:05,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940906d575bc060ce160589 2025-12-17 23:01:05,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940906d575bc060ce160589 HTTP/1.1" 502 150 2025-12-17 23:01:05,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940906d575bc060ce160589'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:06,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940908d575bc060ce1605e2 2025-12-17 23:01:06,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908d575bc060ce1605e2 HTTP/1.1" 502 150 2025-12-17 23:01:06,198 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940908d575bc060ce1605e2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:06,258 - lava_callback - INFO - Received callback for node 6940931d575bc060ce160c32 INFO: 172.21.0.1:49580 - "POST /node/6940931d575bc060ce160c32 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:49596 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:01:06,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409068575bc060ce160580 2025-12-17 23:01:06,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409068575bc060ce160580 HTTP/1.1" 502 150 2025-12-17 23:01:06,708 - lava_callback - ERROR - Error processing callback for node 69409068575bc060ce160580: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409068575bc060ce160580 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409068575bc060ce160580 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409068575bc060ce160580 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:06,710 - lava_callback - INFO - Starting async job submit for node 69409096575bc060ce1605f0 2025-12-17 23:01:06,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:06,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409096575bc060ce1605f0 HTTP/1.1" 502 150 2025-12-17 23:01:06,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409096575bc060ce1605f0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:06,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409096575bc060ce1605f0 2025-12-17 23:01:06,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409096575bc060ce1605f0 HTTP/1.1" 502 150 2025-12-17 23:01:06,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409096575bc060ce1605f0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:06,908 - lava_callback - INFO - Received callback for node 69409323575bc060ce160c3d INFO: 172.21.0.1:49602 - "POST /node/69409323575bc060ce160c3d HTTP/1.0" 202 Accepted 2025-12-17 23:01:07,570 - lava_callback - INFO - Received callback for node 69409326575bc060ce160c49 INFO: 172.21.0.1:49606 - "POST /node/69409326575bc060ce160c49 HTTP/1.0" 202 Accepted 2025-12-17 23:01:07,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409069575bc060ce160583 2025-12-17 23:01:07,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409069575bc060ce160583 HTTP/1.1" 502 150 2025-12-17 23:01:07,725 - lava_callback - ERROR - Error processing callback for node 69409069575bc060ce160583: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409069575bc060ce160583 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409069575bc060ce160583 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409069575bc060ce160583 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:07,727 - lava_callback - INFO - Starting async job submit for node 69409098575bc060ce1605f3 2025-12-17 23:01:07,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:07,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409098575bc060ce1605f3 HTTP/1.1" 502 150 2025-12-17 23:01:07,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409098575bc060ce1605f3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:07,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409098575bc060ce1605f3 2025-12-17 23:01:07,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409098575bc060ce1605f3 HTTP/1.1" 502 150 2025-12-17 23:01:07,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409098575bc060ce1605f3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:07,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940906b575bc060ce160586 2025-12-17 23:01:07,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940906b575bc060ce160586 HTTP/1.1" 502 150 2025-12-17 23:01:07,834 - lava_callback - ERROR - Error processing callback for node 6940906b575bc060ce160586: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940906b575bc060ce160586 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940906b575bc060ce160586 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940906b575bc060ce160586 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:07,837 - lava_callback - INFO - Starting async job submit for node 6940909a575bc060ce1605f6 2025-12-17 23:01:07,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:07,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909a575bc060ce1605f6 HTTP/1.1" 502 150 2025-12-17 23:01:07,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940909a575bc060ce1605f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:07,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940909a575bc060ce1605f6 2025-12-17 23:01:07,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909a575bc060ce1605f6 HTTP/1.1" 502 150 2025-12-17 23:01:07,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940909a575bc060ce1605f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:07,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409070575bc060ce16058c 2025-12-17 23:01:07,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409070575bc060ce16058c HTTP/1.1" 502 150 2025-12-17 23:01:07,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409070575bc060ce16058c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:07,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409076575bc060ce16059b 2025-12-17 23:01:07,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409076575bc060ce16059b HTTP/1.1" 502 150 2025-12-17 23:01:07,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409076575bc060ce16059b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:08,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940908b575bc060ce1605d9 2025-12-17 23:01:08,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908b575bc060ce1605d9 HTTP/1.1" 502 150 2025-12-17 23:01:08,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940908b575bc060ce1605d9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:08,237 - lava_callback - INFO - Received callback for node 69409328575bc060ce160c4c INFO: 172.21.0.1:49608 - "POST /node/69409328575bc060ce160c4c HTTP/1.0" 202 Accepted 2025-12-17 23:01:08,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409072575bc060ce16058f 2025-12-17 23:01:08,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409072575bc060ce16058f HTTP/1.1" 502 150 2025-12-17 23:01:08,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409072575bc060ce16058f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:08,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409078575bc060ce1605a6 2025-12-17 23:01:08,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409078575bc060ce1605a6 HTTP/1.1" 502 150 2025-12-17 23:01:08,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409078575bc060ce1605a6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:08,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409096575bc060ce1605f0 2025-12-17 23:01:08,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409096575bc060ce1605f0 HTTP/1.1" 502 150 2025-12-17 23:01:08,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409096575bc060ce1605f0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:08,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940907a575bc060ce1605ad 2025-12-17 23:01:08,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907a575bc060ce1605ad HTTP/1.1" 502 150 2025-12-17 23:01:08,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940907a575bc060ce1605ad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:08,891 - lava_callback - INFO - Received callback for node 6940932a575bc060ce160c4f INFO: 172.21.0.1:49616 - "POST /node/6940932a575bc060ce160c4f HTTP/1.0" 202 Accepted 2025-12-17 23:01:09,533 - lava_callback - INFO - Received callback for node 6940932c575bc060ce160c52 INFO: 172.21.0.1:49620 - "POST /node/6940932c575bc060ce160c52 HTTP/1.0" 202 Accepted 2025-12-17 23:01:09,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940907c575bc060ce1605b0 2025-12-17 23:01:09,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907c575bc060ce1605b0 HTTP/1.1" 502 150 2025-12-17 23:01:09,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940907c575bc060ce1605b0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:09,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409098575bc060ce1605f3 2025-12-17 23:01:09,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409098575bc060ce1605f3 HTTP/1.1" 502 150 2025-12-17 23:01:09,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409098575bc060ce1605f3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:09,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940909a575bc060ce1605f6 2025-12-17 23:01:09,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909a575bc060ce1605f6 HTTP/1.1" 502 150 2025-12-17 23:01:09,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940909a575bc060ce1605f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:10,179 - lava_callback - INFO - Received callback for node 6940932e575bc060ce160c55 INFO: 172.21.0.1:48538 - "POST /node/6940932e575bc060ce160c55 HTTP/1.0" 202 Accepted 2025-12-17 23:01:10,198 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940908d575bc060ce1605e2 2025-12-17 23:01:10,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908d575bc060ce1605e2 HTTP/1.1" 502 150 2025-12-17 23:01:10,199 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940908d575bc060ce1605e2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:10,810 - lava_callback - INFO - Received callback for node 69409330575bc060ce160c58 INFO: 172.21.0.1:48544 - "POST /node/69409330575bc060ce160c58 HTTP/1.0" 202 Accepted 2025-12-17 23:01:11,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940908f575bc060ce1605e5 2025-12-17 23:01:11,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908f575bc060ce1605e5 HTTP/1.1" 502 150 2025-12-17 23:01:11,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940908f575bc060ce1605e5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:12,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409096575bc060ce1605f0 2025-12-17 23:01:12,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409096575bc060ce1605f0 HTTP/1.1" 502 150 2025-12-17 23:01:12,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409096575bc060ce1605f0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:13,066 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409088575bc060ce1605bc 2025-12-17 23:01:13,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409088575bc060ce1605bc HTTP/1.1" 502 150 2025-12-17 23:01:13,067 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409088575bc060ce1605bc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:13,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409098575bc060ce1605f3 2025-12-17 23:01:13,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409098575bc060ce1605f3 HTTP/1.1" 502 150 2025-12-17 23:01:13,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409098575bc060ce1605f3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:13,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940909a575bc060ce1605f6 2025-12-17 23:01:13,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909a575bc060ce1605f6 HTTP/1.1" 502 150 2025-12-17 23:01:13,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940909a575bc060ce1605f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:14,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409082575bc060ce1605b9 HTTP/1.1" 502 150 2025-12-17 23:01:14,288 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409082575bc060ce1605b9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48546 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:01:16,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940908b575bc060ce1605d9 2025-12-17 23:01:16,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908b575bc060ce1605d9 HTTP/1.1" 502 150 2025-12-17 23:01:16,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940908b575bc060ce1605d9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48554 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:01:18,199 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940908d575bc060ce1605e2 2025-12-17 23:01:18,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908d575bc060ce1605e2 HTTP/1.1" 502 150 2025-12-17 23:01:18,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940908d575bc060ce1605e2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:18,288 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409082575bc060ce1605b9 2025-12-17 23:01:18,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409082575bc060ce1605b9 HTTP/1.1" 502 150 2025-12-17 23:01:18,289 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409082575bc060ce1605b9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:20,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409096575bc060ce1605f0 2025-12-17 23:01:20,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409096575bc060ce1605f0 HTTP/1.1" 502 150 2025-12-17 23:01:20,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409096575bc060ce1605f0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:21,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409074575bc060ce160592 2025-12-17 23:01:21,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409074575bc060ce160592 HTTP/1.1" 502 150 2025-12-17 23:01:21,543 - lava_callback - ERROR - Error processing callback for node 69409074575bc060ce160592: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409074575bc060ce160592 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409074575bc060ce160592 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409074575bc060ce160592 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:21,545 - lava_callback - INFO - Starting async job submit for node 69409092575bc060ce1605ea 2025-12-17 23:01:21,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:21,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409092575bc060ce1605ea HTTP/1.1" 502 150 2025-12-17 23:01:21,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409092575bc060ce1605ea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:21,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409092575bc060ce1605ea 2025-12-17 23:01:21,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409092575bc060ce1605ea HTTP/1.1" 502 150 2025-12-17 23:01:21,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409092575bc060ce1605ea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:21,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940906d575bc060ce160589 2025-12-17 23:01:21,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940906d575bc060ce160589 HTTP/1.1" 502 150 2025-12-17 23:01:21,597 - lava_callback - ERROR - Error processing callback for node 6940906d575bc060ce160589: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940906d575bc060ce160589 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940906d575bc060ce160589 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940906d575bc060ce160589 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:21,600 - lava_callback - INFO - Starting async job submit for node 694090a1575bc060ce16060b 2025-12-17 23:01:21,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:21,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090a1575bc060ce16060b HTTP/1.1" 502 150 2025-12-17 23:01:21,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090a1575bc060ce16060b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:21,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090a1575bc060ce16060b 2025-12-17 23:01:21,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090a1575bc060ce16060b HTTP/1.1" 502 150 2025-12-17 23:01:21,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090a1575bc060ce16060b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:21,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409098575bc060ce1605f3 2025-12-17 23:01:21,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409098575bc060ce1605f3 HTTP/1.1" 502 150 2025-12-17 23:01:21,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409098575bc060ce1605f3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:21,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940909a575bc060ce1605f6 2025-12-17 23:01:21,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909a575bc060ce1605f6 HTTP/1.1" 502 150 2025-12-17 23:01:21,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940909a575bc060ce1605f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:23,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409092575bc060ce1605ea 2025-12-17 23:01:23,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409092575bc060ce1605ea HTTP/1.1" 502 150 2025-12-17 23:01:23,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409092575bc060ce1605ea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:23,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090a1575bc060ce16060b 2025-12-17 23:01:23,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090a1575bc060ce16060b HTTP/1.1" 502 150 2025-12-17 23:01:23,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090a1575bc060ce16060b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:23,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409070575bc060ce16058c 2025-12-17 23:01:23,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409070575bc060ce16058c HTTP/1.1" 502 150 2025-12-17 23:01:23,895 - lava_callback - ERROR - Error processing callback for node 69409070575bc060ce16058c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409070575bc060ce16058c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409070575bc060ce16058c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409070575bc060ce16058c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:23,897 - lava_callback - INFO - Starting async job submit for node 6940909d575bc060ce1605fd 2025-12-17 23:01:23,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:23,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409076575bc060ce16059b 2025-12-17 23:01:23,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409076575bc060ce16059b HTTP/1.1" 502 150 2025-12-17 23:01:23,917 - lava_callback - ERROR - Error processing callback for node 69409076575bc060ce16059b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409076575bc060ce16059b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409076575bc060ce16059b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409076575bc060ce16059b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:23,918 - lava_callback - INFO - Starting async job submit for node 6940909f575bc060ce160608 2025-12-17 23:01:23,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:23,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909d575bc060ce1605fd HTTP/1.1" 502 150 2025-12-17 23:01:23,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940909d575bc060ce1605fd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:23,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940909d575bc060ce1605fd 2025-12-17 23:01:23,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909d575bc060ce1605fd HTTP/1.1" 502 150 2025-12-17 23:01:23,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940909d575bc060ce1605fd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:24,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409072575bc060ce16058f 2025-12-17 23:01:24,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409072575bc060ce16058f HTTP/1.1" 502 150 2025-12-17 23:01:24,468 - lava_callback - ERROR - Error processing callback for node 69409072575bc060ce16058f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409072575bc060ce16058f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409072575bc060ce16058f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409072575bc060ce16058f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:24,469 - lava_callback - INFO - Starting async job submit for node 694090a5575bc060ce160617 2025-12-17 23:01:24,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:24,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090a5575bc060ce160617 HTTP/1.1" 502 150 2025-12-17 23:01:24,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090a5575bc060ce160617'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:24,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090a5575bc060ce160617 2025-12-17 23:01:24,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090a5575bc060ce160617 HTTP/1.1" 502 150 2025-12-17 23:01:24,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090a5575bc060ce160617'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:24,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409078575bc060ce1605a6 2025-12-17 23:01:24,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409078575bc060ce1605a6 HTTP/1.1" 502 150 2025-12-17 23:01:24,540 - lava_callback - ERROR - Error processing callback for node 69409078575bc060ce1605a6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409078575bc060ce1605a6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409078575bc060ce1605a6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409078575bc060ce1605a6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:24,541 - lava_callback - INFO - Starting async job submit for node 694090ab575bc060ce160620 2025-12-17 23:01:24,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:24,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ab575bc060ce160620 HTTP/1.1" 502 150 2025-12-17 23:01:24,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ab575bc060ce160620'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:24,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ab575bc060ce160620 2025-12-17 23:01:24,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ab575bc060ce160620 HTTP/1.1" 502 150 2025-12-17 23:01:24,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ab575bc060ce160620'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:24,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940907a575bc060ce1605ad 2025-12-17 23:01:24,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907a575bc060ce1605ad HTTP/1.1" 502 150 2025-12-17 23:01:24,847 - lava_callback - ERROR - Error processing callback for node 6940907a575bc060ce1605ad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940907a575bc060ce1605ad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940907a575bc060ce1605ad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940907a575bc060ce1605ad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:24,849 - lava_callback - INFO - Starting async job submit for node 694090ad575bc060ce160623 2025-12-17 23:01:24,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:24,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ad575bc060ce160623 HTTP/1.1" 502 150 2025-12-17 23:01:24,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ad575bc060ce160623'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:24,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ad575bc060ce160623 2025-12-17 23:01:24,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ad575bc060ce160623 HTTP/1.1" 502 150 2025-12-17 23:01:24,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ad575bc060ce160623'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:25,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940907c575bc060ce1605b0 2025-12-17 23:01:25,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940907c575bc060ce1605b0 HTTP/1.1" 502 150 2025-12-17 23:01:25,686 - lava_callback - ERROR - Error processing callback for node 6940907c575bc060ce1605b0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940907c575bc060ce1605b0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940907c575bc060ce1605b0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940907c575bc060ce1605b0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:25,688 - lava_callback - INFO - Starting async job submit for node 694090af575bc060ce160626 2025-12-17 23:01:25,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:25,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090af575bc060ce160626 HTTP/1.1" 502 150 2025-12-17 23:01:25,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090af575bc060ce160626'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:25,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090af575bc060ce160626 2025-12-17 23:01:25,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090af575bc060ce160626 HTTP/1.1" 502 150 2025-12-17 23:01:25,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090af575bc060ce160626'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:25,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940909d575bc060ce1605fd 2025-12-17 23:01:25,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909d575bc060ce1605fd HTTP/1.1" 502 150 2025-12-17 23:01:25,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940909d575bc060ce1605fd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:26,285 - lava_callback - INFO - Received callback for node 69409332575bc060ce160c61 INFO: 172.21.0.1:53170 - "POST /node/69409332575bc060ce160c61 HTTP/1.0" 202 Accepted 2025-12-17 23:01:26,292 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409082575bc060ce1605b9 2025-12-17 23:01:26,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409082575bc060ce1605b9 HTTP/1.1" 502 150 2025-12-17 23:01:26,293 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409082575bc060ce1605b9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:26,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090a5575bc060ce160617 2025-12-17 23:01:26,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090a5575bc060ce160617 HTTP/1.1" 502 150 2025-12-17 23:01:26,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090a5575bc060ce160617'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53176 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:01:26,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ab575bc060ce160620 2025-12-17 23:01:26,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ab575bc060ce160620 HTTP/1.1" 502 150 2025-12-17 23:01:26,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ab575bc060ce160620'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:26,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ad575bc060ce160623 2025-12-17 23:01:26,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ad575bc060ce160623 HTTP/1.1" 502 150 2025-12-17 23:01:26,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ad575bc060ce160623'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:26,937 - lava_callback - INFO - Received callback for node 69409336575bc060ce160c73 INFO: 172.21.0.1:53180 - "POST /node/69409336575bc060ce160c73 HTTP/1.0" 202 Accepted 2025-12-17 23:01:27,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409092575bc060ce1605ea 2025-12-17 23:01:27,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409092575bc060ce1605ea HTTP/1.1" 502 150 2025-12-17 23:01:27,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409092575bc060ce1605ea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:27,586 - lava_callback - INFO - Received callback for node 69409338575bc060ce160c7a INFO: 172.21.0.1:53182 - "POST /node/69409338575bc060ce160c7a HTTP/1.0" 202 Accepted 2025-12-17 23:01:27,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090a1575bc060ce16060b 2025-12-17 23:01:27,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090a1575bc060ce16060b HTTP/1.1" 502 150 2025-12-17 23:01:27,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090a1575bc060ce16060b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:27,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090af575bc060ce160626 2025-12-17 23:01:27,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090af575bc060ce160626 HTTP/1.1" 502 150 2025-12-17 23:01:27,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090af575bc060ce160626'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:27,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940908f575bc060ce1605e5 2025-12-17 23:01:27,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908f575bc060ce1605e5 HTTP/1.1" 502 150 2025-12-17 23:01:27,790 - lava_callback - ERROR - Error processing callback for node 6940908f575bc060ce1605e5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940908f575bc060ce1605e5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940908f575bc060ce1605e5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940908f575bc060ce1605e5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:27,791 - lava_callback - INFO - Starting async job submit for node 694090b6575bc060ce16064e 2025-12-17 23:01:27,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:27,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b6575bc060ce16064e HTTP/1.1" 502 150 2025-12-17 23:01:27,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b6575bc060ce16064e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:27,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b6575bc060ce16064e 2025-12-17 23:01:27,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b6575bc060ce16064e HTTP/1.1" 502 150 2025-12-17 23:01:27,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b6575bc060ce16064e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:28,247 - lava_callback - INFO - Received callback for node 6940933a575bc060ce160c7f INFO: 172.21.0.1:53190 - "POST /node/6940933a575bc060ce160c7f HTTP/1.0" 202 Accepted 2025-12-17 23:01:28,897 - lava_callback - INFO - Received callback for node 6940933c575bc060ce160ce3 INFO: 172.21.0.1:53196 - "POST /node/6940933c575bc060ce160ce3 HTTP/1.0" 202 Accepted 2025-12-17 23:01:29,067 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409088575bc060ce1605bc 2025-12-17 23:01:29,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409088575bc060ce1605bc HTTP/1.1" 502 150 2025-12-17 23:01:29,069 - lava_callback - ERROR - Error processing callback for node 69409088575bc060ce1605bc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409088575bc060ce1605bc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409088575bc060ce1605bc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409088575bc060ce1605bc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:29,071 - lava_callback - INFO - Starting async job submit for node 694090b8575bc060ce160654 2025-12-17 23:01:29,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:29,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b8575bc060ce160654 HTTP/1.1" 502 150 2025-12-17 23:01:29,079 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b8575bc060ce160654'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:29,079 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b8575bc060ce160654 2025-12-17 23:01:29,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b8575bc060ce160654 HTTP/1.1" 502 150 2025-12-17 23:01:29,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b8575bc060ce160654'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:29,530 - lava_callback - INFO - Received callback for node 6940933e575bc060ce160ce6 INFO: 172.21.0.1:53208 - "POST /node/6940933e575bc060ce160ce6 HTTP/1.0" 202 Accepted 2025-12-17 23:01:29,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b6575bc060ce16064e 2025-12-17 23:01:29,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b6575bc060ce16064e HTTP/1.1" 502 150 2025-12-17 23:01:29,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b6575bc060ce16064e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:29,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940909d575bc060ce1605fd 2025-12-17 23:01:29,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909d575bc060ce1605fd HTTP/1.1" 502 150 2025-12-17 23:01:29,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940909d575bc060ce1605fd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:30,168 - lava_callback - INFO - Received callback for node 69409341575bc060ce160ce9 INFO: 172.21.0.1:52660 - "POST /node/69409341575bc060ce160ce9 HTTP/1.0" 202 Accepted 2025-12-17 23:01:30,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090a5575bc060ce160617 2025-12-17 23:01:30,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090a5575bc060ce160617 HTTP/1.1" 502 150 2025-12-17 23:01:30,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090a5575bc060ce160617'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:30,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ab575bc060ce160620 2025-12-17 23:01:30,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ab575bc060ce160620 HTTP/1.1" 502 150 2025-12-17 23:01:30,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ab575bc060ce160620'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:30,816 - lava_callback - INFO - Received callback for node 69409343575bc060ce160cec INFO: 172.21.0.1:52676 - "POST /node/69409343575bc060ce160cec HTTP/1.0" 202 Accepted 2025-12-17 23:01:30,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ad575bc060ce160623 2025-12-17 23:01:30,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ad575bc060ce160623 HTTP/1.1" 502 150 2025-12-17 23:01:30,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ad575bc060ce160623'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:31,080 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b8575bc060ce160654 2025-12-17 23:01:31,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b8575bc060ce160654 HTTP/1.1" 502 150 2025-12-17 23:01:31,081 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b8575bc060ce160654'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:31,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090af575bc060ce160626 2025-12-17 23:01:31,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090af575bc060ce160626 HTTP/1.1" 502 150 2025-12-17 23:01:31,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090af575bc060ce160626'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:32,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940908b575bc060ce1605d9 2025-12-17 23:01:32,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908b575bc060ce1605d9 HTTP/1.1" 502 150 2025-12-17 23:01:32,048 - lava_callback - ERROR - Error processing callback for node 6940908b575bc060ce1605d9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940908b575bc060ce1605d9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940908b575bc060ce1605d9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940908b575bc060ce1605d9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:32,049 - lava_callback - INFO - Starting async job submit for node 694090b9575bc060ce160657 2025-12-17 23:01:32,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:32,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b9575bc060ce160657 HTTP/1.1" 502 150 2025-12-17 23:01:32,057 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b9575bc060ce160657'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:32,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b9575bc060ce160657 2025-12-17 23:01:32,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b9575bc060ce160657 HTTP/1.1" 502 150 2025-12-17 23:01:32,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b9575bc060ce160657'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:33,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b6575bc060ce16064e 2025-12-17 23:01:33,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b6575bc060ce16064e HTTP/1.1" 502 150 2025-12-17 23:01:33,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b6575bc060ce16064e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:34,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b9575bc060ce160657 2025-12-17 23:01:34,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b9575bc060ce160657 HTTP/1.1" 502 150 2025-12-17 23:01:34,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b9575bc060ce160657'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:34,201 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940908d575bc060ce1605e2 2025-12-17 23:01:34,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940908d575bc060ce1605e2 HTTP/1.1" 502 150 2025-12-17 23:01:34,202 - lava_callback - ERROR - Error processing callback for node 6940908d575bc060ce1605e2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940908d575bc060ce1605e2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940908d575bc060ce1605e2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940908d575bc060ce1605e2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:34,204 - lava_callback - INFO - Starting async job submit for node 694090b1575bc060ce16062d 2025-12-17 23:01:34,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:34,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b1575bc060ce16062d HTTP/1.1" 502 150 2025-12-17 23:01:34,229 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b1575bc060ce16062d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:34,229 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b1575bc060ce16062d 2025-12-17 23:01:34,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b1575bc060ce16062d HTTP/1.1" 502 150 2025-12-17 23:01:34,230 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b1575bc060ce16062d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:35,081 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b8575bc060ce160654 2025-12-17 23:01:35,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b8575bc060ce160654 HTTP/1.1" 502 150 2025-12-17 23:01:35,083 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b8575bc060ce160654'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:35,281 - lava_callback - INFO - Received callback for node 69409418575bc060ce160f60 INFO: 172.21.0.1:52688 - "POST /node/69409418575bc060ce160f60 HTTP/1.0" 202 Accepted 2025-12-17 23:01:35,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409092575bc060ce1605ea 2025-12-17 23:01:35,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409092575bc060ce1605ea HTTP/1.1" 502 150 2025-12-17 23:01:35,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409092575bc060ce1605ea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:35,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090a1575bc060ce16060b 2025-12-17 23:01:35,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090a1575bc060ce16060b HTTP/1.1" 502 150 2025-12-17 23:01:35,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090a1575bc060ce16060b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:36,230 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b1575bc060ce16062d 2025-12-17 23:01:36,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b1575bc060ce16062d HTTP/1.1" 502 150 2025-12-17 23:01:36,231 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b1575bc060ce16062d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52696 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:01:36,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409096575bc060ce1605f0 2025-12-17 23:01:36,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409096575bc060ce1605f0 HTTP/1.1" 502 150 2025-12-17 23:01:36,743 - lava_callback - ERROR - Error processing callback for node 69409096575bc060ce1605f0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409096575bc060ce1605f0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409096575bc060ce1605f0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409096575bc060ce1605f0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:36,745 - lava_callback - INFO - Starting async job submit for node 694090b4575bc060ce160649 2025-12-17 23:01:36,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:36,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b4575bc060ce160649 HTTP/1.1" 502 150 2025-12-17 23:01:36,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b4575bc060ce160649'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:36,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b4575bc060ce160649 2025-12-17 23:01:36,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b4575bc060ce160649 HTTP/1.1" 502 150 2025-12-17 23:01:36,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b4575bc060ce160649'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:37,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409098575bc060ce1605f3 2025-12-17 23:01:37,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409098575bc060ce1605f3 HTTP/1.1" 502 150 2025-12-17 23:01:37,742 - lava_callback - ERROR - Error processing callback for node 69409098575bc060ce1605f3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409098575bc060ce1605f3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409098575bc060ce1605f3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409098575bc060ce1605f3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:37,743 - lava_callback - INFO - Starting async job submit for node 694090b4575bc060ce160644 2025-12-17 23:01:37,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:37,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b4575bc060ce160644 HTTP/1.1" 502 150 2025-12-17 23:01:37,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b4575bc060ce160644'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:37,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b4575bc060ce160644 2025-12-17 23:01:37,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b4575bc060ce160644 HTTP/1.1" 502 150 2025-12-17 23:01:37,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b4575bc060ce160644'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:37,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940909a575bc060ce1605f6 2025-12-17 23:01:37,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909a575bc060ce1605f6 HTTP/1.1" 502 150 2025-12-17 23:01:37,876 - lava_callback - ERROR - Error processing callback for node 6940909a575bc060ce1605f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940909a575bc060ce1605f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940909a575bc060ce1605f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940909a575bc060ce1605f6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:37,878 - lava_callback - INFO - Starting async job submit for node 694090bf575bc060ce160667 2025-12-17 23:01:37,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:37,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bf575bc060ce160667 HTTP/1.1" 502 150 2025-12-17 23:01:37,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090bf575bc060ce160667'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:37,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090bf575bc060ce160667 2025-12-17 23:01:37,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bf575bc060ce160667 HTTP/1.1" 502 150 2025-12-17 23:01:37,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090bf575bc060ce160667'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:37,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940909d575bc060ce1605fd 2025-12-17 23:01:37,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909d575bc060ce1605fd HTTP/1.1" 502 150 2025-12-17 23:01:37,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940909d575bc060ce1605fd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:38,059 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b9575bc060ce160657 2025-12-17 23:01:38,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b9575bc060ce160657 HTTP/1.1" 502 150 2025-12-17 23:01:38,060 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b9575bc060ce160657'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:38,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090a5575bc060ce160617 2025-12-17 23:01:38,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090a5575bc060ce160617 HTTP/1.1" 502 150 2025-12-17 23:01:38,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090a5575bc060ce160617'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:38,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ab575bc060ce160620 2025-12-17 23:01:38,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ab575bc060ce160620 HTTP/1.1" 502 150 2025-12-17 23:01:38,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ab575bc060ce160620'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:38,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b4575bc060ce160649 2025-12-17 23:01:38,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b4575bc060ce160649 HTTP/1.1" 502 150 2025-12-17 23:01:38,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b4575bc060ce160649'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:38,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ad575bc060ce160623 2025-12-17 23:01:38,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ad575bc060ce160623 HTTP/1.1" 502 150 2025-12-17 23:01:38,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ad575bc060ce160623'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:39,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090af575bc060ce160626 2025-12-17 23:01:39,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090af575bc060ce160626 HTTP/1.1" 502 150 2025-12-17 23:01:39,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090af575bc060ce160626'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:39,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b4575bc060ce160644 2025-12-17 23:01:39,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b4575bc060ce160644 HTTP/1.1" 502 150 2025-12-17 23:01:39,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b4575bc060ce160644'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:39,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090bf575bc060ce160667 2025-12-17 23:01:39,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bf575bc060ce160667 HTTP/1.1" 502 150 2025-12-17 23:01:39,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090bf575bc060ce160667'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:40,231 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b1575bc060ce16062d 2025-12-17 23:01:40,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b1575bc060ce16062d HTTP/1.1" 502 150 2025-12-17 23:01:40,232 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b1575bc060ce16062d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:41,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b6575bc060ce16064e 2025-12-17 23:01:41,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b6575bc060ce16064e HTTP/1.1" 502 150 2025-12-17 23:01:41,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b6575bc060ce16064e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:42,293 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409082575bc060ce1605b9 2025-12-17 23:01:42,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409082575bc060ce1605b9 HTTP/1.1" 502 150 2025-12-17 23:01:42,294 - lava_callback - ERROR - Error processing callback for node 69409082575bc060ce1605b9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409082575bc060ce1605b9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409082575bc060ce1605b9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409082575bc060ce1605b9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:42,296 - lava_callback - INFO - Starting async job submit for node 694090b7575bc060ce160651 2025-12-17 23:01:42,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:42,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b7575bc060ce160651 HTTP/1.1" 502 150 2025-12-17 23:01:42,303 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b7575bc060ce160651'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:42,303 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b7575bc060ce160651 2025-12-17 23:01:42,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b7575bc060ce160651 HTTP/1.1" 502 150 2025-12-17 23:01:42,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b7575bc060ce160651'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:42,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b4575bc060ce160649 2025-12-17 23:01:42,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b4575bc060ce160649 HTTP/1.1" 502 150 2025-12-17 23:01:42,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b4575bc060ce160649'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:43,083 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b8575bc060ce160654 2025-12-17 23:01:43,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b8575bc060ce160654 HTTP/1.1" 502 150 2025-12-17 23:01:43,084 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b8575bc060ce160654'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:43,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b4575bc060ce160644 2025-12-17 23:01:43,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b4575bc060ce160644 HTTP/1.1" 502 150 2025-12-17 23:01:43,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b4575bc060ce160644'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:43,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090bf575bc060ce160667 2025-12-17 23:01:43,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bf575bc060ce160667 HTTP/1.1" 502 150 2025-12-17 23:01:43,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090bf575bc060ce160667'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:44,304 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b7575bc060ce160651 2025-12-17 23:01:44,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b7575bc060ce160651 HTTP/1.1" 502 150 2025-12-17 23:01:44,305 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b7575bc060ce160651'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:46,061 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b9575bc060ce160657 2025-12-17 23:01:46,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b9575bc060ce160657 HTTP/1.1" 502 150 2025-12-17 23:01:46,062 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b9575bc060ce160657'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:46,299 - lava_callback - INFO - Received callback for node 69409347575bc060ce160cfc INFO: 172.21.0.1:47006 - "POST /node/69409347575bc060ce160cfc HTTP/1.0" 202 Accepted INFO: 172.21.0.1:47014 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:01:46,946 - lava_callback - INFO - Received callback for node 69409349575bc060ce160d05 INFO: 172.21.0.1:47030 - "POST /node/69409349575bc060ce160d05 HTTP/1.0" 202 Accepted 2025-12-17 23:01:47,605 - lava_callback - INFO - Received callback for node 6940934b575bc060ce160d0a INFO: 172.21.0.1:47044 - "POST /node/6940934b575bc060ce160d0a HTTP/1.0" 202 Accepted 2025-12-17 23:01:48,233 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b1575bc060ce16062d 2025-12-17 23:01:48,256 - lava_callback - INFO - Received callback for node 6940934d575bc060ce160d0d INFO: 172.21.0.1:47052 - "POST /node/6940934d575bc060ce160d0d HTTP/1.0" 202 Accepted 2025-12-17 23:01:48,305 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b7575bc060ce160651 2025-12-17 23:01:48,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b7575bc060ce160651 HTTP/1.1" 502 150 2025-12-17 23:01:48,307 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b7575bc060ce160651'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:48,898 - lava_callback - INFO - Received callback for node 69409350575bc060ce160d10 INFO: 172.21.0.1:47068 - "POST /node/69409350575bc060ce160d10 HTTP/1.0" 202 Accepted 2025-12-17 23:01:49,544 - lava_callback - INFO - Received callback for node 69409351575bc060ce160d13 INFO: 172.21.0.1:47082 - "POST /node/69409351575bc060ce160d13 HTTP/1.0" 202 Accepted 2025-12-17 23:01:50,192 - lava_callback - INFO - Received callback for node 69409357575bc060ce160d1c INFO: 172.21.0.1:57790 - "POST /node/69409357575bc060ce160d1c HTTP/1.0" 202 Accepted 2025-12-17 23:01:50,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b4575bc060ce160649 2025-12-17 23:01:50,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b4575bc060ce160649 HTTP/1.1" 502 150 2025-12-17 23:01:50,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b4575bc060ce160649'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:51,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409092575bc060ce1605ea 2025-12-17 23:01:51,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409092575bc060ce1605ea HTTP/1.1" 502 150 2025-12-17 23:01:51,561 - lava_callback - ERROR - Error processing callback for node 69409092575bc060ce1605ea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409092575bc060ce1605ea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409092575bc060ce1605ea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409092575bc060ce1605ea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:51,563 - lava_callback - INFO - Starting async job submit for node 694090c2575bc060ce16066a 2025-12-17 23:01:51,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:51,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c2575bc060ce16066a HTTP/1.1" 502 150 2025-12-17 23:01:51,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c2575bc060ce16066a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:51,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c2575bc060ce16066a 2025-12-17 23:01:51,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c2575bc060ce16066a HTTP/1.1" 502 150 2025-12-17 23:01:51,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c2575bc060ce16066a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:51,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090a1575bc060ce16060b 2025-12-17 23:01:51,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090a1575bc060ce16060b HTTP/1.1" 502 150 2025-12-17 23:01:51,617 - lava_callback - ERROR - Error processing callback for node 694090a1575bc060ce16060b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090a1575bc060ce16060b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090a1575bc060ce16060b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090a1575bc060ce16060b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:51,619 - lava_callback - INFO - Starting async job submit for node 694090c4575bc060ce160674 2025-12-17 23:01:51,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:51,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c4575bc060ce160674 HTTP/1.1" 502 150 2025-12-17 23:01:51,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c4575bc060ce160674'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:51,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c4575bc060ce160674 2025-12-17 23:01:51,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c4575bc060ce160674 HTTP/1.1" 502 150 2025-12-17 23:01:51,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c4575bc060ce160674'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:51,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b4575bc060ce160644 2025-12-17 23:01:51,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b4575bc060ce160644 HTTP/1.1" 502 150 2025-12-17 23:01:51,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b4575bc060ce160644'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:51,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090bf575bc060ce160667 2025-12-17 23:01:51,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bf575bc060ce160667 HTTP/1.1" 502 150 2025-12-17 23:01:51,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090bf575bc060ce160667'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:53,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c2575bc060ce16066a 2025-12-17 23:01:53,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c2575bc060ce16066a HTTP/1.1" 502 150 2025-12-17 23:01:53,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c2575bc060ce16066a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:53,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c4575bc060ce160674 2025-12-17 23:01:53,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c4575bc060ce160674 HTTP/1.1" 502 150 2025-12-17 23:01:53,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c4575bc060ce160674'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:53,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940909d575bc060ce1605fd 2025-12-17 23:01:53,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909d575bc060ce1605fd HTTP/1.1" 502 150 2025-12-17 23:01:53,933 - lava_callback - ERROR - Error processing callback for node 6940909d575bc060ce1605fd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940909d575bc060ce1605fd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940909d575bc060ce1605fd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940909d575bc060ce1605fd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:53,936 - lava_callback - INFO - Starting async job submit for node 694092ff575bc060ce160bfc 2025-12-17 23:01:53,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:53,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ff575bc060ce160bfc HTTP/1.1" 502 150 2025-12-17 23:01:53,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ff575bc060ce160bfc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:53,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ff575bc060ce160bfc 2025-12-17 23:01:53,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ff575bc060ce160bfc HTTP/1.1" 502 150 2025-12-17 23:01:53,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ff575bc060ce160bfc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:54,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090a5575bc060ce160617 2025-12-17 23:01:54,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090a5575bc060ce160617 HTTP/1.1" 502 150 2025-12-17 23:01:54,482 - lava_callback - ERROR - Error processing callback for node 694090a5575bc060ce160617: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090a5575bc060ce160617 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090a5575bc060ce160617 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090a5575bc060ce160617 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:54,485 - lava_callback - INFO - Starting async job submit for node 694090bb575bc060ce16065a 2025-12-17 23:01:54,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:54,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bb575bc060ce16065a HTTP/1.1" 502 150 2025-12-17 23:01:54,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090bb575bc060ce16065a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:54,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090bb575bc060ce16065a 2025-12-17 23:01:54,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bb575bc060ce16065a HTTP/1.1" 502 150 2025-12-17 23:01:54,495 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090bb575bc060ce16065a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:54,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ab575bc060ce160620 2025-12-17 23:01:54,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ab575bc060ce160620 HTTP/1.1" 502 150 2025-12-17 23:01:54,574 - lava_callback - ERROR - Error processing callback for node 694090ab575bc060ce160620: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ab575bc060ce160620 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ab575bc060ce160620 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ab575bc060ce160620 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:54,577 - lava_callback - INFO - Starting async job submit for node 694090bc575bc060ce16065d 2025-12-17 23:01:54,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:54,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bc575bc060ce16065d HTTP/1.1" 502 150 2025-12-17 23:01:54,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090bc575bc060ce16065d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:54,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090bc575bc060ce16065d 2025-12-17 23:01:54,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bc575bc060ce16065d HTTP/1.1" 502 150 2025-12-17 23:01:54,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090bc575bc060ce16065d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:54,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909f575bc060ce160608 HTTP/1.1" 502 150 2025-12-17 23:01:54,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940909f575bc060ce160608'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:54,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940909f575bc060ce160608 2025-12-17 23:01:54,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ad575bc060ce160623 2025-12-17 23:01:54,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ad575bc060ce160623 HTTP/1.1" 502 150 2025-12-17 23:01:54,861 - lava_callback - ERROR - Error processing callback for node 694090ad575bc060ce160623: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ad575bc060ce160623 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ad575bc060ce160623 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ad575bc060ce160623 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:54,864 - lava_callback - INFO - Starting async job submit for node 694090be575bc060ce160664 2025-12-17 23:01:54,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:54,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090be575bc060ce160664 HTTP/1.1" 502 150 2025-12-17 23:01:54,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090be575bc060ce160664'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:54,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090be575bc060ce160664 2025-12-17 23:01:54,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090be575bc060ce160664 HTTP/1.1" 502 150 2025-12-17 23:01:54,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090be575bc060ce160664'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:55,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090af575bc060ce160626 2025-12-17 23:01:55,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090af575bc060ce160626 HTTP/1.1" 502 150 2025-12-17 23:01:55,704 - lava_callback - ERROR - Error processing callback for node 694090af575bc060ce160626: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090af575bc060ce160626 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090af575bc060ce160626 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090af575bc060ce160626 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:55,706 - lava_callback - INFO - Starting async job submit for node 694090c3575bc060ce16066d 2025-12-17 23:01:55,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:55,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c3575bc060ce16066d HTTP/1.1" 502 150 2025-12-17 23:01:55,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c3575bc060ce16066d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:55,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c3575bc060ce16066d 2025-12-17 23:01:55,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c3575bc060ce16066d HTTP/1.1" 502 150 2025-12-17 23:01:55,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c3575bc060ce16066d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:55,843 - lava_callback - INFO - Received callback for node 69409561575bc060ce1612ee INFO: 172.21.0.1:57796 - "POST /node/69409561575bc060ce1612ee HTTP/1.0" 202 Accepted 2025-12-17 23:01:55,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ff575bc060ce160bfc 2025-12-17 23:01:55,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ff575bc060ce160bfc HTTP/1.1" 502 150 2025-12-17 23:01:55,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ff575bc060ce160bfc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:56,307 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b7575bc060ce160651 2025-12-17 23:01:56,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b7575bc060ce160651 HTTP/1.1" 502 150 2025-12-17 23:01:56,308 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b7575bc060ce160651'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:56,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090bb575bc060ce16065a 2025-12-17 23:01:56,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bb575bc060ce16065a HTTP/1.1" 502 150 2025-12-17 23:01:56,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090bb575bc060ce16065a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57800 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:01:56,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090bc575bc060ce16065d 2025-12-17 23:01:56,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bc575bc060ce16065d HTTP/1.1" 502 150 2025-12-17 23:01:56,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090bc575bc060ce16065d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:56,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090be575bc060ce160664 2025-12-17 23:01:56,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090be575bc060ce160664 HTTP/1.1" 502 150 2025-12-17 23:01:56,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090be575bc060ce160664'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:57,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c2575bc060ce16066a 2025-12-17 23:01:57,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c2575bc060ce16066a HTTP/1.1" 502 150 2025-12-17 23:01:57,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c2575bc060ce16066a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:57,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c4575bc060ce160674 2025-12-17 23:01:57,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c4575bc060ce160674 HTTP/1.1" 502 150 2025-12-17 23:01:57,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c4575bc060ce160674'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:57,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c3575bc060ce16066d 2025-12-17 23:01:57,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c3575bc060ce16066d HTTP/1.1" 502 150 2025-12-17 23:01:57,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c3575bc060ce16066d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:57,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b6575bc060ce16064e 2025-12-17 23:01:57,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b6575bc060ce16064e HTTP/1.1" 502 150 2025-12-17 23:01:57,804 - lava_callback - ERROR - Error processing callback for node 694090b6575bc060ce16064e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b6575bc060ce16064e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b6575bc060ce16064e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b6575bc060ce16064e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:57,807 - lava_callback - INFO - Starting async job submit for node 694090c5575bc060ce16067b 2025-12-17 23:01:57,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:57,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c5575bc060ce16067b HTTP/1.1" 502 150 2025-12-17 23:01:57,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c5575bc060ce16067b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:57,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c5575bc060ce16067b 2025-12-17 23:01:57,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c5575bc060ce16067b HTTP/1.1" 502 150 2025-12-17 23:01:57,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c5575bc060ce16067b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:59,084 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b8575bc060ce160654 2025-12-17 23:01:59,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b8575bc060ce160654 HTTP/1.1" 502 150 2025-12-17 23:01:59,086 - lava_callback - ERROR - Error processing callback for node 694090b8575bc060ce160654: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b8575bc060ce160654 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b8575bc060ce160654 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b8575bc060ce160654 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:01:59,088 - lava_callback - INFO - Starting async job submit for node 694090c7575bc060ce160692 2025-12-17 23:01:59,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:01:59,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c7575bc060ce160692 HTTP/1.1" 502 150 2025-12-17 23:01:59,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c7575bc060ce160692'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:59,095 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c7575bc060ce160692 2025-12-17 23:01:59,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c7575bc060ce160692 HTTP/1.1" 502 150 2025-12-17 23:01:59,096 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c7575bc060ce160692'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:59,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c5575bc060ce16067b 2025-12-17 23:01:59,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c5575bc060ce16067b HTTP/1.1" 502 150 2025-12-17 23:01:59,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c5575bc060ce16067b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:01:59,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ff575bc060ce160bfc 2025-12-17 23:01:59,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ff575bc060ce160bfc HTTP/1.1" 502 150 2025-12-17 23:01:59,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ff575bc060ce160bfc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:00,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090bb575bc060ce16065a 2025-12-17 23:02:00,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bb575bc060ce16065a HTTP/1.1" 502 150 2025-12-17 23:02:00,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090bb575bc060ce16065a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:00,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090bc575bc060ce16065d 2025-12-17 23:02:00,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bc575bc060ce16065d HTTP/1.1" 502 150 2025-12-17 23:02:00,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090bc575bc060ce16065d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:00,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090be575bc060ce160664 2025-12-17 23:02:00,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090be575bc060ce160664 HTTP/1.1" 502 150 2025-12-17 23:02:00,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090be575bc060ce160664'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:01,096 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c7575bc060ce160692 2025-12-17 23:02:01,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c7575bc060ce160692 HTTP/1.1" 502 150 2025-12-17 23:02:01,097 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c7575bc060ce160692'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:01,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c3575bc060ce16066d 2025-12-17 23:02:01,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c3575bc060ce16066d HTTP/1.1" 502 150 2025-12-17 23:02:01,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c3575bc060ce16066d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:02,062 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b9575bc060ce160657 INFO: 172.21.0.1:37330 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:02:03,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b1575bc060ce16062d HTTP/1.1" 502 150 2025-12-17 23:02:03,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090b1575bc060ce16062d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:03,804 - lava_callback - INFO - Received callback for node 69409559575bc060ce1612dc INFO: 172.21.0.1:37342 - "POST /node/69409559575bc060ce1612dc HTTP/1.0" 202 Accepted 2025-12-17 23:02:03,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c5575bc060ce16067b 2025-12-17 23:02:03,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c5575bc060ce16067b HTTP/1.1" 502 150 2025-12-17 23:02:03,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c5575bc060ce16067b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:05,098 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c7575bc060ce160692 2025-12-17 23:02:05,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c7575bc060ce160692 HTTP/1.1" 502 150 2025-12-17 23:02:05,099 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c7575bc060ce160692'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:05,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c2575bc060ce16066a 2025-12-17 23:02:05,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c2575bc060ce16066a HTTP/1.1" 502 150 2025-12-17 23:02:05,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c2575bc060ce16066a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:05,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c4575bc060ce160674 2025-12-17 23:02:05,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c4575bc060ce160674 HTTP/1.1" 502 150 2025-12-17 23:02:05,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c4575bc060ce160674'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:06,302 - lava_callback - INFO - Received callback for node 6940935a575bc060ce160d25 INFO: 172.21.0.1:37344 - "POST /node/6940935a575bc060ce160d25 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:37360 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:02:06,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b4575bc060ce160649 2025-12-17 23:02:06,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b4575bc060ce160649 HTTP/1.1" 502 150 2025-12-17 23:02:06,761 - lava_callback - ERROR - Error processing callback for node 694090b4575bc060ce160649: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b4575bc060ce160649 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b4575bc060ce160649 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b4575bc060ce160649 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:06,763 - lava_callback - INFO - Starting async job submit for node 694090c9575bc060ce1606b0 2025-12-17 23:02:06,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:06,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c9575bc060ce1606b0 HTTP/1.1" 502 150 2025-12-17 23:02:06,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c9575bc060ce1606b0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:06,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c9575bc060ce1606b0 2025-12-17 23:02:06,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c9575bc060ce1606b0 HTTP/1.1" 502 150 2025-12-17 23:02:06,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c9575bc060ce1606b0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:06,947 - lava_callback - INFO - Received callback for node 6940935b575bc060ce160d2e INFO: 172.21.0.1:37374 - "POST /node/6940935b575bc060ce160d2e HTTP/1.0" 202 Accepted 2025-12-17 23:02:07,616 - lava_callback - INFO - Received callback for node 6940935c575bc060ce160d35 INFO: 172.21.0.1:37376 - "POST /node/6940935c575bc060ce160d35 HTTP/1.0" 202 Accepted 2025-12-17 23:02:07,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b4575bc060ce160644 2025-12-17 23:02:07,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b4575bc060ce160644 HTTP/1.1" 502 150 2025-12-17 23:02:07,757 - lava_callback - ERROR - Error processing callback for node 694090b4575bc060ce160644: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b4575bc060ce160644 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b4575bc060ce160644 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b4575bc060ce160644 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:07,759 - lava_callback - INFO - Starting async job submit for node 694090cb575bc060ce1606ba 2025-12-17 23:02:07,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:07,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cb575bc060ce1606ba HTTP/1.1" 502 150 2025-12-17 23:02:07,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cb575bc060ce1606ba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:07,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cb575bc060ce1606ba 2025-12-17 23:02:07,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cb575bc060ce1606ba HTTP/1.1" 502 150 2025-12-17 23:02:07,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cb575bc060ce1606ba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:07,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090bf575bc060ce160667 2025-12-17 23:02:07,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bf575bc060ce160667 HTTP/1.1" 502 150 2025-12-17 23:02:07,891 - lava_callback - ERROR - Error processing callback for node 694090bf575bc060ce160667: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090bf575bc060ce160667 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090bf575bc060ce160667 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090bf575bc060ce160667 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:07,894 - lava_callback - INFO - Starting async job submit for node 694090cd575bc060ce1606bd 2025-12-17 23:02:07,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:07,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cd575bc060ce1606bd HTTP/1.1" 502 150 2025-12-17 23:02:07,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cd575bc060ce1606bd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:07,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cd575bc060ce1606bd 2025-12-17 23:02:07,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cd575bc060ce1606bd HTTP/1.1" 502 150 2025-12-17 23:02:07,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cd575bc060ce1606bd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:07,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ff575bc060ce160bfc 2025-12-17 23:02:07,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ff575bc060ce160bfc HTTP/1.1" 502 150 2025-12-17 23:02:07,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ff575bc060ce160bfc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:08,259 - lava_callback - INFO - Received callback for node 6940935e575bc060ce160d3a INFO: 172.21.0.1:37382 - "POST /node/6940935e575bc060ce160d3a HTTP/1.0" 202 Accepted 2025-12-17 23:02:08,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090bb575bc060ce16065a 2025-12-17 23:02:08,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bb575bc060ce16065a HTTP/1.1" 502 150 2025-12-17 23:02:08,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090bb575bc060ce16065a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:08,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090bc575bc060ce16065d 2025-12-17 23:02:08,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bc575bc060ce16065d HTTP/1.1" 502 150 2025-12-17 23:02:08,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090bc575bc060ce16065d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:08,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c9575bc060ce1606b0 2025-12-17 23:02:08,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c9575bc060ce1606b0 HTTP/1.1" 502 150 2025-12-17 23:02:08,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c9575bc060ce1606b0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:08,821 - lava_callback - INFO - Received callback for node 6940936a575bc060ce160d69 INFO: 172.21.0.1:37394 - "POST /node/6940936a575bc060ce160d69 HTTP/1.0" 202 Accepted 2025-12-17 23:02:08,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090be575bc060ce160664 2025-12-17 23:02:08,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090be575bc060ce160664 HTTP/1.1" 502 150 2025-12-17 23:02:08,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090be575bc060ce160664'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:08,913 - lava_callback - INFO - Received callback for node 6940935f575bc060ce160d3d INFO: 172.21.0.1:37406 - "POST /node/6940935f575bc060ce160d3d HTTP/1.0" 202 Accepted 2025-12-17 23:02:09,580 - lava_callback - INFO - Received callback for node 69409363575bc060ce160d5a INFO: 172.21.0.1:37420 - "POST /node/69409363575bc060ce160d5a HTTP/1.0" 202 Accepted 2025-12-17 23:02:09,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c3575bc060ce16066d 2025-12-17 23:02:09,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c3575bc060ce16066d HTTP/1.1" 502 150 2025-12-17 23:02:09,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c3575bc060ce16066d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:09,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cb575bc060ce1606ba 2025-12-17 23:02:09,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cb575bc060ce1606ba HTTP/1.1" 502 150 2025-12-17 23:02:09,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cb575bc060ce1606ba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:09,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cd575bc060ce1606bd 2025-12-17 23:02:09,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cd575bc060ce1606bd HTTP/1.1" 502 150 2025-12-17 23:02:09,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cd575bc060ce1606bd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:10,249 - lava_callback - INFO - Received callback for node 69409365575bc060ce160d5d INFO: 172.21.0.1:39634 - "POST /node/69409365575bc060ce160d5d HTTP/1.0" 202 Accepted 2025-12-17 23:02:10,895 - lava_callback - INFO - Received callback for node 69409366575bc060ce160d60 INFO: 172.21.0.1:39638 - "POST /node/69409366575bc060ce160d60 HTTP/1.0" 202 Accepted 2025-12-17 23:02:11,561 - lava_callback - INFO - Received callback for node 69409367575bc060ce160d63 INFO: 172.21.0.1:39646 - "POST /node/69409367575bc060ce160d63 HTTP/1.0" 202 Accepted 2025-12-17 23:02:11,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c5575bc060ce16067b 2025-12-17 23:02:11,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c5575bc060ce16067b HTTP/1.1" 502 150 2025-12-17 23:02:11,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c5575bc060ce16067b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:12,287 - lava_callback - INFO - Received callback for node 69409369575bc060ce160d66 INFO: 172.21.0.1:39652 - "POST /node/69409369575bc060ce160d66 HTTP/1.0" 202 Accepted 2025-12-17 23:02:12,308 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b7575bc060ce160651 2025-12-17 23:02:12,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b7575bc060ce160651 HTTP/1.1" 502 150 2025-12-17 23:02:12,309 - lava_callback - ERROR - Error processing callback for node 694090b7575bc060ce160651: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b7575bc060ce160651 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b7575bc060ce160651 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b7575bc060ce160651 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:12,311 - lava_callback - INFO - Starting async job submit for node 694090ce575bc060ce1606c0 2025-12-17 23:02:12,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:12,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ce575bc060ce1606c0 HTTP/1.1" 502 150 2025-12-17 23:02:12,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ce575bc060ce1606c0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:12,319 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ce575bc060ce1606c0 2025-12-17 23:02:12,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ce575bc060ce1606c0 HTTP/1.1" 502 150 2025-12-17 23:02:12,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ce575bc060ce1606c0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:12,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c9575bc060ce1606b0 2025-12-17 23:02:13,099 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c7575bc060ce160692 2025-12-17 23:02:13,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cb575bc060ce1606ba 2025-12-17 23:02:13,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cd575bc060ce1606bd 2025-12-17 23:02:14,320 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ce575bc060ce1606c0 2025-12-17 23:02:14,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cb575bc060ce1606ba HTTP/1.1" 502 150 2025-12-17 23:02:14,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cb575bc060ce1606ba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:14,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cd575bc060ce1606bd HTTP/1.1" 502 150 2025-12-17 23:02:14,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ce575bc060ce1606c0 HTTP/1.1" 502 150 2025-12-17 23:02:14,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cd575bc060ce1606bd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:14,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ce575bc060ce1606c0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39664 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:02:15,945 - lava_callback - INFO - Received callback for node 6940936e575bc060ce160d75 INFO: 172.21.0.1:39676 - "POST /node/6940936e575bc060ce160d75 HTTP/1.0" 202 Accepted 2025-12-17 23:02:16,190 - lava_callback - INFO - Received callback for node 69409691575bc060ce161615 INFO: 172.21.0.1:39682 - "POST /node/69409691575bc060ce161615 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:39692 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:02:16,609 - lava_callback - INFO - Received callback for node 69409371575bc060ce160d80 INFO: 172.21.0.1:39694 - "POST /node/69409371575bc060ce160d80 HTTP/1.0" 202 Accepted 2025-12-17 23:02:17,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b9575bc060ce160657 HTTP/1.1" 502 150 2025-12-17 23:02:17,520 - lava_callback - ERROR - Error processing callback for node 694090b9575bc060ce160657: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b9575bc060ce160657 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b9575bc060ce160657 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b9575bc060ce160657 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:17,522 - lava_callback - INFO - Starting async job submit for node 69408e99575bc060ce15fe96 2025-12-17 23:02:17,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:17,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e99575bc060ce15fe96 HTTP/1.1" 502 150 2025-12-17 23:02:17,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e99575bc060ce15fe96'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:17,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e99575bc060ce15fe96 2025-12-17 23:02:17,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e99575bc060ce15fe96 HTTP/1.1" 502 150 2025-12-17 23:02:17,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e99575bc060ce15fe96'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:18,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ce575bc060ce1606c0 2025-12-17 23:02:18,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ce575bc060ce1606c0 HTTP/1.1" 502 150 2025-12-17 23:02:18,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ce575bc060ce1606c0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:19,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e99575bc060ce15fe96 2025-12-17 23:02:19,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e99575bc060ce15fe96 HTTP/1.1" 502 150 2025-12-17 23:02:19,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e99575bc060ce15fe96'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:19,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090b1575bc060ce16062d 2025-12-17 23:02:19,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090b1575bc060ce16062d HTTP/1.1" 502 150 2025-12-17 23:02:19,698 - lava_callback - ERROR - Error processing callback for node 694090b1575bc060ce16062d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b1575bc060ce16062d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b1575bc060ce16062d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090b1575bc060ce16062d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:19,699 - lava_callback - INFO - Starting async job submit for node 694091a6575bc060ce1608ec 2025-12-17 23:02:19,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:19,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a6575bc060ce1608ec HTTP/1.1" 502 150 2025-12-17 23:02:19,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a6575bc060ce1608ec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:19,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a6575bc060ce1608ec 2025-12-17 23:02:19,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a6575bc060ce1608ec HTTP/1.1" 502 150 2025-12-17 23:02:19,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a6575bc060ce1608ec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:21,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c2575bc060ce16066a 2025-12-17 23:02:21,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c2575bc060ce16066a HTTP/1.1" 502 150 2025-12-17 23:02:21,580 - lava_callback - ERROR - Error processing callback for node 694090c2575bc060ce16066a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c2575bc060ce16066a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c2575bc060ce16066a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c2575bc060ce16066a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:21,583 - lava_callback - INFO - Starting async job submit for node 694090cb575bc060ce1606b7 2025-12-17 23:02:21,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:21,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cb575bc060ce1606b7 HTTP/1.1" 502 150 2025-12-17 23:02:21,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cb575bc060ce1606b7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:21,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cb575bc060ce1606b7 2025-12-17 23:02:21,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cb575bc060ce1606b7 HTTP/1.1" 502 150 2025-12-17 23:02:21,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cb575bc060ce1606b7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:21,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c4575bc060ce160674 2025-12-17 23:02:21,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c4575bc060ce160674 HTTP/1.1" 502 150 2025-12-17 23:02:21,632 - lava_callback - ERROR - Error processing callback for node 694090c4575bc060ce160674: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c4575bc060ce160674 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c4575bc060ce160674 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c4575bc060ce160674 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:21,634 - lava_callback - INFO - Starting async job submit for node 694090cf575bc060ce1606c3 2025-12-17 23:02:21,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:21,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cf575bc060ce1606c3 HTTP/1.1" 502 150 2025-12-17 23:02:21,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cf575bc060ce1606c3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:21,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cf575bc060ce1606c3 2025-12-17 23:02:21,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cf575bc060ce1606c3 HTTP/1.1" 502 150 2025-12-17 23:02:21,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cf575bc060ce1606c3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:21,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a6575bc060ce1608ec 2025-12-17 23:02:21,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a6575bc060ce1608ec HTTP/1.1" 502 150 2025-12-17 23:02:21,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a6575bc060ce1608ec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:22,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cb575bc060ce1606ba 2025-12-17 23:02:22,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cd575bc060ce1606bd 2025-12-17 23:02:22,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cb575bc060ce1606ba HTTP/1.1" 502 150 2025-12-17 23:02:22,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cb575bc060ce1606ba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:22,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cd575bc060ce1606bd HTTP/1.1" 502 150 2025-12-17 23:02:22,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cd575bc060ce1606bd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:23,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e99575bc060ce15fe96 2025-12-17 23:02:23,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e99575bc060ce15fe96 HTTP/1.1" 502 150 2025-12-17 23:02:23,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e99575bc060ce15fe96'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:23,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cb575bc060ce1606b7 2025-12-17 23:02:23,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cb575bc060ce1606b7 HTTP/1.1" 502 150 2025-12-17 23:02:23,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cb575bc060ce1606b7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:23,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cf575bc060ce1606c3 2025-12-17 23:02:23,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cf575bc060ce1606c3 HTTP/1.1" 502 150 2025-12-17 23:02:23,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cf575bc060ce1606c3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:23,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ff575bc060ce160bfc 2025-12-17 23:02:23,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ff575bc060ce160bfc HTTP/1.1" 502 150 2025-12-17 23:02:23,985 - lava_callback - ERROR - Error processing callback for node 694092ff575bc060ce160bfc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ff575bc060ce160bfc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ff575bc060ce160bfc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ff575bc060ce160bfc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:23,986 - lava_callback - INFO - Starting async job submit for node 694090d0575bc060ce1606c6 2025-12-17 23:02:23,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:23,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d0575bc060ce1606c6 HTTP/1.1" 502 150 2025-12-17 23:02:23,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d0575bc060ce1606c6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:23,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d0575bc060ce1606c6 2025-12-17 23:02:23,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d0575bc060ce1606c6 HTTP/1.1" 502 150 2025-12-17 23:02:23,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d0575bc060ce1606c6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:24,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090bb575bc060ce16065a 2025-12-17 23:02:24,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bb575bc060ce16065a HTTP/1.1" 502 150 2025-12-17 23:02:24,501 - lava_callback - ERROR - Error processing callback for node 694090bb575bc060ce16065a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090bb575bc060ce16065a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090bb575bc060ce16065a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090bb575bc060ce16065a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:24,503 - lava_callback - INFO - Starting async job submit for node 694090d2575bc060ce1606c9 2025-12-17 23:02:24,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:24,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d2575bc060ce1606c9 HTTP/1.1" 502 150 2025-12-17 23:02:24,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d2575bc060ce1606c9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:24,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d2575bc060ce1606c9 2025-12-17 23:02:24,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d2575bc060ce1606c9 HTTP/1.1" 502 150 2025-12-17 23:02:24,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d2575bc060ce1606c9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:24,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090bc575bc060ce16065d 2025-12-17 23:02:24,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090bc575bc060ce16065d HTTP/1.1" 502 150 2025-12-17 23:02:24,602 - lava_callback - ERROR - Error processing callback for node 694090bc575bc060ce16065d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090bc575bc060ce16065d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090bc575bc060ce16065d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090bc575bc060ce16065d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:24,604 - lava_callback - INFO - Starting async job submit for node 694090d3575bc060ce1606cc 2025-12-17 23:02:24,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:24,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d3575bc060ce1606cc HTTP/1.1" 502 150 2025-12-17 23:02:24,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d3575bc060ce1606cc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:24,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d3575bc060ce1606cc 2025-12-17 23:02:24,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d3575bc060ce1606cc HTTP/1.1" 502 150 2025-12-17 23:02:24,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d3575bc060ce1606cc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:24,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090be575bc060ce160664 2025-12-17 23:02:24,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090be575bc060ce160664 HTTP/1.1" 502 150 2025-12-17 23:02:24,905 - lava_callback - ERROR - Error processing callback for node 694090be575bc060ce160664: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090be575bc060ce160664 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090be575bc060ce160664 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090be575bc060ce160664 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:24,907 - lava_callback - INFO - Starting async job submit for node 694090d4575bc060ce1606cf 2025-12-17 23:02:24,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:24,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d4575bc060ce1606cf HTTP/1.1" 502 150 2025-12-17 23:02:24,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d4575bc060ce1606cf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:24,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d4575bc060ce1606cf 2025-12-17 23:02:24,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d4575bc060ce1606cf HTTP/1.1" 502 150 2025-12-17 23:02:24,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d4575bc060ce1606cf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:25,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909f575bc060ce160608 HTTP/1.1" 502 150 2025-12-17 23:02:25,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940909f575bc060ce160608'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:25,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c3575bc060ce16066d 2025-12-17 23:02:25,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c3575bc060ce16066d HTTP/1.1" 502 150 2025-12-17 23:02:25,719 - lava_callback - ERROR - Error processing callback for node 694090c3575bc060ce16066d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c3575bc060ce16066d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c3575bc060ce16066d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c3575bc060ce16066d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:25,721 - lava_callback - INFO - Starting async job submit for node 694090d5575bc060ce1606d2 2025-12-17 23:02:25,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:25,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d5575bc060ce1606d2 HTTP/1.1" 502 150 2025-12-17 23:02:25,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d5575bc060ce1606d2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:25,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d5575bc060ce1606d2 2025-12-17 23:02:25,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d5575bc060ce1606d2 HTTP/1.1" 502 150 2025-12-17 23:02:25,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d5575bc060ce1606d2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:25,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a6575bc060ce1608ec 2025-12-17 23:02:25,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a6575bc060ce1608ec HTTP/1.1" 502 150 2025-12-17 23:02:25,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a6575bc060ce1608ec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:25,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d0575bc060ce1606c6 2025-12-17 23:02:26,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d0575bc060ce1606c6 HTTP/1.1" 502 150 2025-12-17 23:02:26,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d0575bc060ce1606c6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:26,307 - lava_callback - INFO - Received callback for node 69409374575bc060ce160d8a INFO: 172.21.0.1:48922 - "POST /node/69409374575bc060ce160d8a HTTP/1.0" 202 Accepted INFO: 172.21.0.1:48938 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:02:26,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d2575bc060ce1606c9 2025-12-17 23:02:26,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d2575bc060ce1606c9 HTTP/1.1" 502 150 2025-12-17 23:02:26,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d2575bc060ce1606c9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:26,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ce575bc060ce1606c0 2025-12-17 23:02:26,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ce575bc060ce1606c0 HTTP/1.1" 502 150 2025-12-17 23:02:26,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ce575bc060ce1606c0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:26,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d3575bc060ce1606cc 2025-12-17 23:02:26,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d3575bc060ce1606cc HTTP/1.1" 502 150 2025-12-17 23:02:26,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d3575bc060ce1606cc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:26,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d4575bc060ce1606cf 2025-12-17 23:02:26,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d4575bc060ce1606cf HTTP/1.1" 502 150 2025-12-17 23:02:26,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d4575bc060ce1606cf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:26,960 - lava_callback - INFO - Received callback for node 6940937b575bc060ce160d9c INFO: 172.21.0.1:48942 - "POST /node/6940937b575bc060ce160d9c HTTP/1.0" 202 Accepted 2025-12-17 23:02:27,047 - lava_callback - INFO - Received callback for node 69409376575bc060ce160d8d INFO: 172.21.0.1:48946 - "POST /node/69409376575bc060ce160d8d HTTP/1.0" 202 Accepted 2025-12-17 23:02:27,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940909f575bc060ce160608 2025-12-17 23:02:27,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cb575bc060ce1606b7 2025-12-17 23:02:27,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cb575bc060ce1606b7 HTTP/1.1" 502 150 2025-12-17 23:02:27,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cb575bc060ce1606b7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:27,596 - lava_callback - INFO - Received callback for node 6940937e575bc060ce160d9f INFO: 172.21.0.1:48962 - "POST /node/6940937e575bc060ce160d9f HTTP/1.0" 202 Accepted 2025-12-17 23:02:27,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cf575bc060ce1606c3 2025-12-17 23:02:27,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cf575bc060ce1606c3 HTTP/1.1" 502 150 2025-12-17 23:02:27,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cf575bc060ce1606c3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:27,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d5575bc060ce1606d2 2025-12-17 23:02:27,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d5575bc060ce1606d2 HTTP/1.1" 502 150 2025-12-17 23:02:27,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d5575bc060ce1606d2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:27,846 - lava_callback - INFO - Received callback for node 69409378575bc060ce160d93 2025-12-17 23:02:27,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c5575bc060ce16067b INFO: 172.21.0.1:48966 - "POST /node/69409378575bc060ce160d93 HTTP/1.0" 202 Accepted 2025-12-17 23:02:27,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c5575bc060ce16067b HTTP/1.1" 502 150 2025-12-17 23:02:27,857 - lava_callback - ERROR - Error processing callback for node 694090c5575bc060ce16067b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c5575bc060ce16067b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c5575bc060ce16067b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c5575bc060ce16067b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:27,858 - lava_callback - INFO - Starting async job submit for node 6940c53e1e3c24e95681d6b0 2025-12-17 23:02:27,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:27,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c53e1e3c24e95681d6b0 HTTP/1.1" 502 150 2025-12-17 23:02:27,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c53e1e3c24e95681d6b0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:27,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c53e1e3c24e95681d6b0 2025-12-17 23:02:27,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c53e1e3c24e95681d6b0 HTTP/1.1" 502 150 2025-12-17 23:02:27,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c53e1e3c24e95681d6b0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:28,290 - lava_callback - INFO - Received callback for node 69409380575bc060ce160da2 INFO: 172.21.0.1:48978 - "POST /node/69409380575bc060ce160da2 HTTP/1.0" 202 Accepted 2025-12-17 23:02:29,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c53e1e3c24e95681d6b0 2025-12-17 23:02:29,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c53e1e3c24e95681d6b0 HTTP/1.1" 502 150 2025-12-17 23:02:29,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c53e1e3c24e95681d6b0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:30,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d0575bc060ce1606c6 2025-12-17 23:02:30,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d0575bc060ce1606c6 HTTP/1.1" 502 150 2025-12-17 23:02:30,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d0575bc060ce1606c6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:30,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c7575bc060ce160692 HTTP/1.1" 502 150 2025-12-17 23:02:30,064 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c7575bc060ce160692'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:30,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c9575bc060ce1606b0 HTTP/1.1" 502 150 2025-12-17 23:02:30,068 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c9575bc060ce1606b0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:30,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d2575bc060ce1606c9 2025-12-17 23:02:30,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d2575bc060ce1606c9 HTTP/1.1" 502 150 2025-12-17 23:02:30,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d2575bc060ce1606c9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:30,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d3575bc060ce1606cc 2025-12-17 23:02:30,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d3575bc060ce1606cc HTTP/1.1" 502 150 2025-12-17 23:02:30,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d3575bc060ce1606cc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:30,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d4575bc060ce1606cf 2025-12-17 23:02:30,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d4575bc060ce1606cf HTTP/1.1" 502 150 2025-12-17 23:02:30,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d4575bc060ce1606cf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:31,027 - lava_callback - INFO - Received callback for node 6940937a575bc060ce160d96 INFO: 172.21.0.1:35176 - "POST /node/6940937a575bc060ce160d96 HTTP/1.0" 202 Accepted 2025-12-17 23:02:31,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e99575bc060ce15fe96 2025-12-17 23:02:31,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e99575bc060ce15fe96 HTTP/1.1" 502 150 2025-12-17 23:02:31,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408e99575bc060ce15fe96'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:31,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d5575bc060ce1606d2 2025-12-17 23:02:31,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d5575bc060ce1606d2 HTTP/1.1" 502 150 2025-12-17 23:02:31,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d5575bc060ce1606d2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:32,040 - lava_callback - INFO - Received callback for node 69409a22575bc060ce161eee INFO: 172.21.0.1:35178 - "POST /node/69409a22575bc060ce161eee HTTP/1.0" 202 Accepted 2025-12-17 23:02:33,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a6575bc060ce1608ec 2025-12-17 23:02:33,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a6575bc060ce1608ec HTTP/1.1" 502 150 2025-12-17 23:02:33,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a6575bc060ce1608ec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:33,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c53e1e3c24e95681d6b0 2025-12-17 23:02:33,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c53e1e3c24e95681d6b0 HTTP/1.1" 502 150 2025-12-17 23:02:33,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c53e1e3c24e95681d6b0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:35,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cb575bc060ce1606b7 2025-12-17 23:02:35,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cb575bc060ce1606b7 HTTP/1.1" 502 150 2025-12-17 23:02:35,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cb575bc060ce1606b7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:35,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cf575bc060ce1606c3 2025-12-17 23:02:35,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cf575bc060ce1606c3 HTTP/1.1" 502 150 2025-12-17 23:02:35,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090cf575bc060ce1606c3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:35,889 - lava_callback - INFO - Received callback for node 69409377575bc060ce160d90 INFO: 172.21.0.1:35184 - "POST /node/69409377575bc060ce160d90 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:35186 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:02:38,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d0575bc060ce1606c6 2025-12-17 23:02:38,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d0575bc060ce1606c6 HTTP/1.1" 502 150 2025-12-17 23:02:38,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d0575bc060ce1606c6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:38,068 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c9575bc060ce1606b0 2025-12-17 23:02:38,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c9575bc060ce1606b0 HTTP/1.1" 502 150 2025-12-17 23:02:38,069 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090c9575bc060ce1606b0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:38,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d2575bc060ce1606c9 2025-12-17 23:02:38,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d2575bc060ce1606c9 HTTP/1.1" 502 150 2025-12-17 23:02:38,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d2575bc060ce1606c9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:38,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cb575bc060ce1606ba 2025-12-17 23:02:38,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cd575bc060ce1606bd 2025-12-17 23:02:38,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cb575bc060ce1606ba HTTP/1.1" 502 150 2025-12-17 23:02:38,582 - lava_callback - ERROR - Error processing callback for node 694090cb575bc060ce1606ba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090cb575bc060ce1606ba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090cb575bc060ce1606ba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090cb575bc060ce1606ba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:38,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cd575bc060ce1606bd HTTP/1.1" 502 150 2025-12-17 23:02:38,584 - lava_callback - ERROR - Error processing callback for node 694090cd575bc060ce1606bd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090cd575bc060ce1606bd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090cd575bc060ce1606bd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090cd575bc060ce1606bd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:38,584 - lava_callback - INFO - Starting async job submit for node 694090d7575bc060ce1606d5 2025-12-17 23:02:38,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:38,587 - lava_callback - INFO - Starting async job submit for node 694090d9575bc060ce1606d8 2025-12-17 23:02:38,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:38,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d9575bc060ce1606d8 HTTP/1.1" 502 150 2025-12-17 23:02:38,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d9575bc060ce1606d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:38,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d9575bc060ce1606d8 2025-12-17 23:02:38,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d9575bc060ce1606d8 HTTP/1.1" 502 150 2025-12-17 23:02:38,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d9575bc060ce1606d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:38,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d3575bc060ce1606cc 2025-12-17 23:02:38,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d3575bc060ce1606cc HTTP/1.1" 502 150 2025-12-17 23:02:38,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d3575bc060ce1606cc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:38,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d4575bc060ce1606cf 2025-12-17 23:02:38,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d4575bc060ce1606cf HTTP/1.1" 502 150 2025-12-17 23:02:38,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d4575bc060ce1606cf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:39,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d5575bc060ce1606d2 2025-12-17 23:02:39,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d5575bc060ce1606d2 HTTP/1.1" 502 150 2025-12-17 23:02:39,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d5575bc060ce1606d2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:40,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d9575bc060ce1606d8 2025-12-17 23:02:40,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d9575bc060ce1606d8 HTTP/1.1" 502 150 2025-12-17 23:02:40,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d9575bc060ce1606d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:41,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c53e1e3c24e95681d6b0 2025-12-17 23:02:42,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ce575bc060ce1606c0 2025-12-17 23:02:42,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ce575bc060ce1606c0 HTTP/1.1" 502 150 2025-12-17 23:02:42,585 - lava_callback - ERROR - Error processing callback for node 694090ce575bc060ce1606c0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ce575bc060ce1606c0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ce575bc060ce1606c0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ce575bc060ce1606c0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:42,587 - lava_callback - INFO - Starting async job submit for node 694090db575bc060ce1606e3 2025-12-17 23:02:42,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:42,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090db575bc060ce1606e3 HTTP/1.1" 502 150 2025-12-17 23:02:42,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090db575bc060ce1606e3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:42,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090db575bc060ce1606e3 2025-12-17 23:02:42,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090db575bc060ce1606e3 HTTP/1.1" 502 150 2025-12-17 23:02:42,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090db575bc060ce1606e3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:44,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090db575bc060ce1606e3 2025-12-17 23:02:44,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090db575bc060ce1606e3 HTTP/1.1" 502 150 2025-12-17 23:02:44,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090db575bc060ce1606e3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:44,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d9575bc060ce1606d8 2025-12-17 23:02:44,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d9575bc060ce1606d8 HTTP/1.1" 502 150 2025-12-17 23:02:44,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d9575bc060ce1606d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:46,064 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c7575bc060ce160692 2025-12-17 23:02:46,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c7575bc060ce160692 HTTP/1.1" 502 150 2025-12-17 23:02:46,066 - lava_callback - ERROR - Error processing callback for node 694090c7575bc060ce160692: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c7575bc060ce160692 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c7575bc060ce160692 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c7575bc060ce160692 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:46,067 - lava_callback - INFO - Starting async job submit for node 694090dd575bc060ce1606e6 2025-12-17 23:02:46,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:46,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090dd575bc060ce1606e6 HTTP/1.1" 502 150 2025-12-17 23:02:46,074 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090dd575bc060ce1606e6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:46,074 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090dd575bc060ce1606e6 2025-12-17 23:02:46,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090dd575bc060ce1606e6 HTTP/1.1" 502 150 2025-12-17 23:02:46,075 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090dd575bc060ce1606e6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:46,316 - lava_callback - INFO - Received callback for node 69409382575bc060ce160dab INFO: 172.21.0.1:52154 - "POST /node/69409382575bc060ce160dab HTTP/1.0" 202 Accepted INFO: 172.21.0.1:52158 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:02:46,960 - lava_callback - INFO - Received callback for node 69409384575bc060ce160db4 INFO: 172.21.0.1:52160 - "POST /node/69409384575bc060ce160db4 HTTP/1.0" 202 Accepted 2025-12-17 23:02:47,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408e99575bc060ce15fe96 2025-12-17 23:02:47,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408e99575bc060ce15fe96 HTTP/1.1" 502 150 2025-12-17 23:02:47,569 - lava_callback - ERROR - Error processing callback for node 69408e99575bc060ce15fe96: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e99575bc060ce15fe96 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e99575bc060ce15fe96 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408e99575bc060ce15fe96 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:47,570 - lava_callback - INFO - Starting async job submit for node 694090df575bc060ce1606e9 2025-12-17 23:02:47,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:47,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090df575bc060ce1606e9 HTTP/1.1" 502 150 2025-12-17 23:02:47,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090df575bc060ce1606e9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:47,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090df575bc060ce1606e9 2025-12-17 23:02:47,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090df575bc060ce1606e9 HTTP/1.1" 502 150 2025-12-17 23:02:47,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090df575bc060ce1606e9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:47,607 - lava_callback - INFO - Received callback for node 69409386575bc060ce160db7 INFO: 172.21.0.1:52162 - "POST /node/69409386575bc060ce160db7 HTTP/1.0" 202 Accepted 2025-12-17 23:02:48,075 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090dd575bc060ce1606e6 2025-12-17 23:02:48,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090dd575bc060ce1606e6 HTTP/1.1" 502 150 2025-12-17 23:02:48,076 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090dd575bc060ce1606e6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:48,244 - lava_callback - INFO - Received callback for node 69409389575bc060ce160dbd INFO: 172.21.0.1:52168 - "POST /node/69409389575bc060ce160dbd HTTP/1.0" 202 Accepted 2025-12-17 23:02:48,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090db575bc060ce1606e3 2025-12-17 23:02:48,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090db575bc060ce1606e3 HTTP/1.1" 502 150 2025-12-17 23:02:48,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090db575bc060ce1606e3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:48,887 - lava_callback - INFO - Received callback for node 6940938a575bc060ce160dc0 INFO: 172.21.0.1:52170 - "POST /node/6940938a575bc060ce160dc0 HTTP/1.0" 202 Accepted 2025-12-17 23:02:49,551 - lava_callback - INFO - Received callback for node 6940938e575bc060ce160dc6 INFO: 172.21.0.1:52178 - "POST /node/6940938e575bc060ce160dc6 HTTP/1.0" 202 Accepted 2025-12-17 23:02:49,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090df575bc060ce1606e9 2025-12-17 23:02:49,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090df575bc060ce1606e9 HTTP/1.1" 502 150 2025-12-17 23:02:49,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090df575bc060ce1606e9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:49,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a6575bc060ce1608ec 2025-12-17 23:02:49,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a6575bc060ce1608ec HTTP/1.1" 502 150 2025-12-17 23:02:49,742 - lava_callback - ERROR - Error processing callback for node 694091a6575bc060ce1608ec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a6575bc060ce1608ec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a6575bc060ce1608ec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a6575bc060ce1608ec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:49,743 - lava_callback - INFO - Starting async job submit for node 694090e1575bc060ce1606ec 2025-12-17 23:02:49,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:49,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e1575bc060ce1606ec HTTP/1.1" 502 150 2025-12-17 23:02:49,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e1575bc060ce1606ec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:49,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e1575bc060ce1606ec 2025-12-17 23:02:49,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e1575bc060ce1606ec HTTP/1.1" 502 150 2025-12-17 23:02:49,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e1575bc060ce1606ec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:50,193 - lava_callback - INFO - Received callback for node 69409390575bc060ce160dc9 INFO: 172.21.0.1:59524 - "POST /node/69409390575bc060ce160dc9 HTTP/1.0" 202 Accepted 2025-12-17 23:02:50,847 - lava_callback - INFO - Received callback for node 69409392575bc060ce160dcc INFO: 172.21.0.1:59536 - "POST /node/69409392575bc060ce160dcc HTTP/1.0" 202 Accepted 2025-12-17 23:02:51,160 - lava_callback - INFO - Received callback for node 69409388575bc060ce160dba INFO: 172.21.0.1:59542 - "POST /node/69409388575bc060ce160dba HTTP/1.0" 202 Accepted 2025-12-17 23:02:51,484 - lava_callback - INFO - Received callback for node 69409393575bc060ce160dcf INFO: 172.21.0.1:59546 - "POST /node/69409393575bc060ce160dcf HTTP/1.0" 202 Accepted 2025-12-17 23:02:51,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cb575bc060ce1606b7 2025-12-17 23:02:51,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cb575bc060ce1606b7 HTTP/1.1" 502 150 2025-12-17 23:02:51,599 - lava_callback - ERROR - Error processing callback for node 694090cb575bc060ce1606b7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090cb575bc060ce1606b7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090cb575bc060ce1606b7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090cb575bc060ce1606b7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:51,600 - lava_callback - INFO - Starting async job submit for node 694090e3575bc060ce1606ef 2025-12-17 23:02:51,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:51,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e3575bc060ce1606ef HTTP/1.1" 502 150 2025-12-17 23:02:51,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e3575bc060ce1606ef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:51,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e3575bc060ce1606ef 2025-12-17 23:02:51,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e3575bc060ce1606ef HTTP/1.1" 502 150 2025-12-17 23:02:51,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e3575bc060ce1606ef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:51,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090cf575bc060ce1606c3 2025-12-17 23:02:51,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090cf575bc060ce1606c3 HTTP/1.1" 502 150 2025-12-17 23:02:51,651 - lava_callback - ERROR - Error processing callback for node 694090cf575bc060ce1606c3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090cf575bc060ce1606c3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090cf575bc060ce1606c3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090cf575bc060ce1606c3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:51,653 - lava_callback - INFO - Starting async job submit for node 694090e5575bc060ce1606f2 2025-12-17 23:02:51,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:51,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e5575bc060ce1606f2 HTTP/1.1" 502 150 2025-12-17 23:02:51,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e5575bc060ce1606f2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:51,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e5575bc060ce1606f2 2025-12-17 23:02:51,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e5575bc060ce1606f2 HTTP/1.1" 502 150 2025-12-17 23:02:51,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e5575bc060ce1606f2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:51,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e1575bc060ce1606ec 2025-12-17 23:02:51,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e1575bc060ce1606ec HTTP/1.1" 502 150 2025-12-17 23:02:51,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e1575bc060ce1606ec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:51,832 - lava_callback - INFO - Received callback for node 69409396575bc060ce160dd9 INFO: 172.21.0.1:59548 - "POST /node/69409396575bc060ce160dd9 HTTP/1.0" 202 Accepted 2025-12-17 23:02:52,077 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090dd575bc060ce1606e6 2025-12-17 23:02:52,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090dd575bc060ce1606e6 HTTP/1.1" 502 150 2025-12-17 23:02:52,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090dd575bc060ce1606e6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:52,133 - lava_callback - INFO - Received callback for node 69409395575bc060ce160dd2 INFO: 172.21.0.1:59560 - "POST /node/69409395575bc060ce160dd2 HTTP/1.0" 202 Accepted 2025-12-17 23:02:52,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d9575bc060ce1606d8 2025-12-17 23:02:52,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d9575bc060ce1606d8 HTTP/1.1" 502 150 2025-12-17 23:02:52,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d9575bc060ce1606d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:53,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090df575bc060ce1606e9 2025-12-17 23:02:53,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090df575bc060ce1606e9 HTTP/1.1" 502 150 2025-12-17 23:02:53,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090df575bc060ce1606e9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:53,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e3575bc060ce1606ef 2025-12-17 23:02:53,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e3575bc060ce1606ef HTTP/1.1" 502 150 2025-12-17 23:02:53,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e3575bc060ce1606ef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:53,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e5575bc060ce1606f2 2025-12-17 23:02:53,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e5575bc060ce1606f2 HTTP/1.1" 502 150 2025-12-17 23:02:53,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e5575bc060ce1606f2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:54,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d0575bc060ce1606c6 2025-12-17 23:02:54,069 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090c9575bc060ce1606b0 2025-12-17 23:02:54,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090c9575bc060ce1606b0 HTTP/1.1" 502 150 2025-12-17 23:02:54,071 - lava_callback - ERROR - Error processing callback for node 694090c9575bc060ce1606b0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c9575bc060ce1606b0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c9575bc060ce1606b0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090c9575bc060ce1606b0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:54,072 - lava_callback - INFO - Starting async job submit for node 694090e7575bc060ce1606f5 2025-12-17 23:02:54,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:54,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e7575bc060ce1606f5 HTTP/1.1" 502 150 2025-12-17 23:02:54,083 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e7575bc060ce1606f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:54,083 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e7575bc060ce1606f5 2025-12-17 23:02:54,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e7575bc060ce1606f5 HTTP/1.1" 502 150 2025-12-17 23:02:54,084 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e7575bc060ce1606f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:54,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d2575bc060ce1606c9 2025-12-17 23:02:54,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d2575bc060ce1606c9 HTTP/1.1" 502 150 2025-12-17 23:02:54,536 - lava_callback - ERROR - Error processing callback for node 694090d2575bc060ce1606c9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d2575bc060ce1606c9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d2575bc060ce1606c9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d2575bc060ce1606c9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:54,538 - lava_callback - INFO - Starting async job submit for node 694090ea575bc060ce1606f8 2025-12-17 23:02:54,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:54,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ea575bc060ce1606f8 HTTP/1.1" 502 150 2025-12-17 23:02:54,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ea575bc060ce1606f8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:54,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ea575bc060ce1606f8 2025-12-17 23:02:54,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ea575bc060ce1606f8 HTTP/1.1" 502 150 2025-12-17 23:02:54,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ea575bc060ce1606f8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:54,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d3575bc060ce1606cc 2025-12-17 23:02:54,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d3575bc060ce1606cc HTTP/1.1" 502 150 2025-12-17 23:02:54,641 - lava_callback - ERROR - Error processing callback for node 694090d3575bc060ce1606cc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d3575bc060ce1606cc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d3575bc060ce1606cc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d3575bc060ce1606cc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:54,643 - lava_callback - INFO - Starting async job submit for node 694090ec575bc060ce1606fb 2025-12-17 23:02:54,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:54,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ec575bc060ce1606fb HTTP/1.1" 502 150 2025-12-17 23:02:54,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ec575bc060ce1606fb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:54,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ec575bc060ce1606fb 2025-12-17 23:02:54,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ec575bc060ce1606fb HTTP/1.1" 502 150 2025-12-17 23:02:54,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ec575bc060ce1606fb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:54,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d4575bc060ce1606cf 2025-12-17 23:02:54,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d4575bc060ce1606cf HTTP/1.1" 502 150 2025-12-17 23:02:54,920 - lava_callback - ERROR - Error processing callback for node 694090d4575bc060ce1606cf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d4575bc060ce1606cf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d4575bc060ce1606cf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d4575bc060ce1606cf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:54,922 - lava_callback - INFO - Starting async job submit for node 694090ee575bc060ce160702 2025-12-17 23:02:54,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:54,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ee575bc060ce160702 HTTP/1.1" 502 150 2025-12-17 23:02:54,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ee575bc060ce160702'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:54,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ee575bc060ce160702 2025-12-17 23:02:54,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ee575bc060ce160702 HTTP/1.1" 502 150 2025-12-17 23:02:54,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ee575bc060ce160702'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:55,435 - lava_callback - INFO - Received callback for node 69409420575bc060ce160f6c INFO: 172.21.0.1:59562 - "POST /node/69409420575bc060ce160f6c HTTP/1.0" 202 Accepted 2025-12-17 23:02:55,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d5575bc060ce1606d2 2025-12-17 23:02:55,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d5575bc060ce1606d2 HTTP/1.1" 502 150 2025-12-17 23:02:55,735 - lava_callback - ERROR - Error processing callback for node 694090d5575bc060ce1606d2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d5575bc060ce1606d2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d5575bc060ce1606d2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d5575bc060ce1606d2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:02:55,737 - lava_callback - INFO - Starting async job submit for node 694090f0575bc060ce160705 2025-12-17 23:02:55,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:02:55,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f0575bc060ce160705 HTTP/1.1" 502 150 2025-12-17 23:02:55,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f0575bc060ce160705'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:55,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f0575bc060ce160705 2025-12-17 23:02:55,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f0575bc060ce160705 HTTP/1.1" 502 150 2025-12-17 23:02:55,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f0575bc060ce160705'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:55,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e1575bc060ce1606ec 2025-12-17 23:02:55,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e1575bc060ce1606ec HTTP/1.1" 502 150 2025-12-17 23:02:55,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e1575bc060ce1606ec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:55,893 - lava_callback - INFO - Received callback for node 6940938c575bc060ce160dc3 INFO: 172.21.0.1:59572 - "POST /node/6940938c575bc060ce160dc3 HTTP/1.0" 202 Accepted 2025-12-17 23:02:56,084 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e7575bc060ce1606f5 2025-12-17 23:02:56,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e7575bc060ce1606f5 HTTP/1.1" 502 150 2025-12-17 23:02:56,085 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e7575bc060ce1606f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59586 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:02:56,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ea575bc060ce1606f8 2025-12-17 23:02:56,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ea575bc060ce1606f8 HTTP/1.1" 502 150 2025-12-17 23:02:56,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ea575bc060ce1606f8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:56,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090db575bc060ce1606e3 2025-12-17 23:02:56,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090db575bc060ce1606e3 HTTP/1.1" 502 150 2025-12-17 23:02:56,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090db575bc060ce1606e3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:56,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ec575bc060ce1606fb 2025-12-17 23:02:56,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ec575bc060ce1606fb HTTP/1.1" 502 150 2025-12-17 23:02:56,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ec575bc060ce1606fb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:56,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ee575bc060ce160702 2025-12-17 23:02:56,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ee575bc060ce160702 HTTP/1.1" 502 150 2025-12-17 23:02:56,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ee575bc060ce160702'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:57,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c53e1e3c24e95681d6b0 HTTP/1.1" 502 150 2025-12-17 23:02:57,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c53e1e3c24e95681d6b0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:57,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e3575bc060ce1606ef 2025-12-17 23:02:57,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e3575bc060ce1606ef HTTP/1.1" 502 150 2025-12-17 23:02:57,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e3575bc060ce1606ef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:57,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e5575bc060ce1606f2 2025-12-17 23:02:57,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e5575bc060ce1606f2 HTTP/1.1" 502 150 2025-12-17 23:02:57,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e5575bc060ce1606f2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:57,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f0575bc060ce160705 2025-12-17 23:02:57,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f0575bc060ce160705 HTTP/1.1" 502 150 2025-12-17 23:02:57,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f0575bc060ce160705'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:02:58,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909f575bc060ce160608 HTTP/1.1" 502 150 2025-12-17 23:02:58,224 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940909f575bc060ce160608'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:00,081 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090dd575bc060ce1606e6 2025-12-17 23:03:00,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090dd575bc060ce1606e6 HTTP/1.1" 502 150 2025-12-17 23:03:00,082 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090dd575bc060ce1606e6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:00,086 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e7575bc060ce1606f5 2025-12-17 23:03:00,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e7575bc060ce1606f5 HTTP/1.1" 502 150 2025-12-17 23:03:00,087 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e7575bc060ce1606f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:00,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ea575bc060ce1606f8 2025-12-17 23:03:00,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ea575bc060ce1606f8 HTTP/1.1" 502 150 2025-12-17 23:03:00,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ea575bc060ce1606f8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:00,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ec575bc060ce1606fb 2025-12-17 23:03:00,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ec575bc060ce1606fb HTTP/1.1" 502 150 2025-12-17 23:03:00,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ec575bc060ce1606fb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:00,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ee575bc060ce160702 2025-12-17 23:03:00,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ee575bc060ce160702 HTTP/1.1" 502 150 2025-12-17 23:03:00,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ee575bc060ce160702'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:01,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090df575bc060ce1606e9 2025-12-17 23:03:01,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090df575bc060ce1606e9 HTTP/1.1" 502 150 2025-12-17 23:03:01,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090df575bc060ce1606e9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:01,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f0575bc060ce160705 2025-12-17 23:03:01,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f0575bc060ce160705 HTTP/1.1" 502 150 2025-12-17 23:03:01,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f0575bc060ce160705'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:02,224 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940909f575bc060ce160608 2025-12-17 23:03:02,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909f575bc060ce160608 HTTP/1.1" 502 150 2025-12-17 23:03:02,227 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940909f575bc060ce160608'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41682 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:03:03,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e1575bc060ce1606ec 2025-12-17 23:03:03,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e1575bc060ce1606ec HTTP/1.1" 502 150 2025-12-17 23:03:03,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e1575bc060ce1606ec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:05,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e3575bc060ce1606ef 2025-12-17 23:03:05,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e3575bc060ce1606ef HTTP/1.1" 502 150 2025-12-17 23:03:05,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e3575bc060ce1606ef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:05,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e5575bc060ce1606f2 2025-12-17 23:03:05,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e5575bc060ce1606f2 HTTP/1.1" 502 150 2025-12-17 23:03:05,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e5575bc060ce1606f2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:06,319 - lava_callback - INFO - Received callback for node 69409398575bc060ce160de0 INFO: 172.21.0.1:41698 - "POST /node/69409398575bc060ce160de0 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:41710 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:03:06,965 - lava_callback - INFO - Received callback for node 6940939c575bc060ce160de6 INFO: 172.21.0.1:41714 - "POST /node/6940939c575bc060ce160de6 HTTP/1.0" 202 Accepted 2025-12-17 23:03:07,630 - lava_callback - INFO - Received callback for node 6940939e575bc060ce160dee INFO: 172.21.0.1:41718 - "POST /node/6940939e575bc060ce160dee HTTP/1.0" 202 Accepted 2025-12-17 23:03:08,087 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e7575bc060ce1606f5 2025-12-17 23:03:08,283 - lava_callback - INFO - Received callback for node 694093a0575bc060ce160df1 INFO: 172.21.0.1:41724 - "POST /node/694093a0575bc060ce160df1 HTTP/1.0" 202 Accepted 2025-12-17 23:03:08,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ea575bc060ce1606f8 2025-12-17 23:03:08,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ea575bc060ce1606f8 HTTP/1.1" 502 150 2025-12-17 23:03:08,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ea575bc060ce1606f8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:08,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d9575bc060ce1606d8 2025-12-17 23:03:08,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d9575bc060ce1606d8 HTTP/1.1" 502 150 2025-12-17 23:03:08,602 - lava_callback - ERROR - Error processing callback for node 694090d9575bc060ce1606d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d9575bc060ce1606d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d9575bc060ce1606d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d9575bc060ce1606d8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:08,604 - lava_callback - INFO - Starting async job submit for node 694090f2575bc060ce160708 2025-12-17 23:03:08,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:08,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f2575bc060ce160708 HTTP/1.1" 502 150 2025-12-17 23:03:08,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f2575bc060ce160708'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:08,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f2575bc060ce160708 2025-12-17 23:03:08,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f2575bc060ce160708 HTTP/1.1" 502 150 2025-12-17 23:03:08,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f2575bc060ce160708'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:08,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ec575bc060ce1606fb 2025-12-17 23:03:08,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ec575bc060ce1606fb HTTP/1.1" 502 150 2025-12-17 23:03:08,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ec575bc060ce1606fb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:08,894 - lava_callback - INFO - Received callback for node 694113d8460bc8e265117e35 INFO: 172.21.0.1:41734 - "POST /node/694113d8460bc8e265117e35 HTTP/1.0" 202 Accepted 2025-12-17 23:03:08,929 - lava_callback - INFO - Received callback for node 694093a2575bc060ce160df4 INFO: 172.21.0.1:41748 - "POST /node/694093a2575bc060ce160df4 HTTP/1.0" 202 Accepted 2025-12-17 23:03:08,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ee575bc060ce160702 2025-12-17 23:03:08,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ee575bc060ce160702 HTTP/1.1" 502 150 2025-12-17 23:03:08,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090ee575bc060ce160702'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:09,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d7575bc060ce1606d5 HTTP/1.1" 502 150 2025-12-17 23:03:09,232 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d7575bc060ce1606d5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:09,232 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d7575bc060ce1606d5 2025-12-17 23:03:09,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d0575bc060ce1606c6 HTTP/1.1" 502 150 2025-12-17 23:03:09,488 - lava_callback - ERROR - Error processing callback for node 694090d0575bc060ce1606c6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d0575bc060ce1606c6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d0575bc060ce1606c6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d0575bc060ce1606c6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:09,491 - lava_callback - INFO - Starting async job submit for node 694090f4575bc060ce16070e 2025-12-17 23:03:09,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:09,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f4575bc060ce16070e HTTP/1.1" 502 150 2025-12-17 23:03:09,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f4575bc060ce16070e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:09,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f4575bc060ce16070e 2025-12-17 23:03:09,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f4575bc060ce16070e HTTP/1.1" 502 150 2025-12-17 23:03:09,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f4575bc060ce16070e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:09,596 - lava_callback - INFO - Received callback for node 694093a4575bc060ce160df7 INFO: 172.21.0.1:41764 - "POST /node/694093a4575bc060ce160df7 HTTP/1.0" 202 Accepted 2025-12-17 23:03:09,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f0575bc060ce160705 2025-12-17 23:03:09,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f0575bc060ce160705 HTTP/1.1" 502 150 2025-12-17 23:03:09,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f0575bc060ce160705'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:10,227 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940909f575bc060ce160608 2025-12-17 23:03:10,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909f575bc060ce160608 HTTP/1.1" 502 150 2025-12-17 23:03:10,228 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940909f575bc060ce160608'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:10,262 - lava_callback - INFO - Received callback for node 694093a6575bc060ce160dfa INFO: 172.21.0.1:38108 - "POST /node/694093a6575bc060ce160dfa HTTP/1.0" 202 Accepted 2025-12-17 23:03:10,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f2575bc060ce160708 2025-12-17 23:03:10,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f2575bc060ce160708 HTTP/1.1" 502 150 2025-12-17 23:03:10,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f2575bc060ce160708'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:10,928 - lava_callback - INFO - Received callback for node 694093a8575bc060ce160dfd INFO: 172.21.0.1:38110 - "POST /node/694093a8575bc060ce160dfd HTTP/1.0" 202 Accepted 2025-12-17 23:03:11,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f4575bc060ce16070e 2025-12-17 23:03:11,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f4575bc060ce16070e HTTP/1.1" 502 150 2025-12-17 23:03:11,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f4575bc060ce16070e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:12,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090db575bc060ce1606e3 2025-12-17 23:03:12,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090db575bc060ce1606e3 HTTP/1.1" 502 150 2025-12-17 23:03:12,602 - lava_callback - ERROR - Error processing callback for node 694090db575bc060ce1606e3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090db575bc060ce1606e3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090db575bc060ce1606e3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090db575bc060ce1606e3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:12,604 - lava_callback - INFO - Starting async job submit for node 694090f6575bc060ce160711 2025-12-17 23:03:12,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:12,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f6575bc060ce160711 HTTP/1.1" 502 150 2025-12-17 23:03:12,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f6575bc060ce160711'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:12,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f6575bc060ce160711 2025-12-17 23:03:12,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f6575bc060ce160711 HTTP/1.1" 502 150 2025-12-17 23:03:12,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f6575bc060ce160711'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:13,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c53e1e3c24e95681d6b0 2025-12-17 23:03:13,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c53e1e3c24e95681d6b0 HTTP/1.1" 502 150 2025-12-17 23:03:13,202 - lava_callback - ERROR - Error processing callback for node 6940c53e1e3c24e95681d6b0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c53e1e3c24e95681d6b0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c53e1e3c24e95681d6b0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c53e1e3c24e95681d6b0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:13,204 - lava_callback - INFO - Starting async job submit for node 694090f8575bc060ce160714 2025-12-17 23:03:13,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:13,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f8575bc060ce160714 HTTP/1.1" 502 150 2025-12-17 23:03:13,211 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f8575bc060ce160714'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:13,211 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f8575bc060ce160714 2025-12-17 23:03:13,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f8575bc060ce160714 HTTP/1.1" 502 150 2025-12-17 23:03:13,212 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f8575bc060ce160714'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:14,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f6575bc060ce160711 2025-12-17 23:03:14,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f6575bc060ce160711 HTTP/1.1" 502 150 2025-12-17 23:03:14,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f6575bc060ce160711'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:14,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f2575bc060ce160708 2025-12-17 23:03:14,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f2575bc060ce160708 HTTP/1.1" 502 150 2025-12-17 23:03:14,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f2575bc060ce160708'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38112 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:03:15,212 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f8575bc060ce160714 2025-12-17 23:03:15,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f8575bc060ce160714 HTTP/1.1" 502 150 2025-12-17 23:03:15,213 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f8575bc060ce160714'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:15,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f4575bc060ce16070e 2025-12-17 23:03:15,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f4575bc060ce16070e HTTP/1.1" 502 150 2025-12-17 23:03:15,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f4575bc060ce16070e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:16,082 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090dd575bc060ce1606e6 2025-12-17 23:03:16,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090dd575bc060ce1606e6 HTTP/1.1" 502 150 2025-12-17 23:03:16,083 - lava_callback - ERROR - Error processing callback for node 694090dd575bc060ce1606e6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090dd575bc060ce1606e6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090dd575bc060ce1606e6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090dd575bc060ce1606e6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:16,085 - lava_callback - INFO - Starting async job submit for node 694090fb575bc060ce160717 2025-12-17 23:03:16,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:16,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fb575bc060ce160717 HTTP/1.1" 502 150 2025-12-17 23:03:16,118 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fb575bc060ce160717'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:16,118 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fb575bc060ce160717 2025-12-17 23:03:16,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fb575bc060ce160717 HTTP/1.1" 502 150 2025-12-17 23:03:16,119 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fb575bc060ce160717'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38120 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:03:17,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090df575bc060ce1606e9 2025-12-17 23:03:17,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090df575bc060ce1606e9 HTTP/1.1" 502 150 2025-12-17 23:03:17,583 - lava_callback - ERROR - Error processing callback for node 694090df575bc060ce1606e9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090df575bc060ce1606e9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090df575bc060ce1606e9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090df575bc060ce1606e9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:17,585 - lava_callback - INFO - Starting async job submit for node 694090fc575bc060ce16071a 2025-12-17 23:03:17,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:17,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fc575bc060ce16071a HTTP/1.1" 502 150 2025-12-17 23:03:17,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fc575bc060ce16071a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:17,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fc575bc060ce16071a 2025-12-17 23:03:17,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fc575bc060ce16071a HTTP/1.1" 502 150 2025-12-17 23:03:17,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fc575bc060ce16071a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:18,119 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fb575bc060ce160717 2025-12-17 23:03:18,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fb575bc060ce160717 HTTP/1.1" 502 150 2025-12-17 23:03:18,120 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fb575bc060ce160717'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:18,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f6575bc060ce160711 2025-12-17 23:03:18,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f6575bc060ce160711 HTTP/1.1" 502 150 2025-12-17 23:03:18,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f6575bc060ce160711'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:19,213 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f8575bc060ce160714 2025-12-17 23:03:19,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f8575bc060ce160714 HTTP/1.1" 502 150 2025-12-17 23:03:19,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f8575bc060ce160714'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:19,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fc575bc060ce16071a 2025-12-17 23:03:19,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fc575bc060ce16071a HTTP/1.1" 502 150 2025-12-17 23:03:19,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fc575bc060ce16071a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:19,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e1575bc060ce1606ec 2025-12-17 23:03:19,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e1575bc060ce1606ec HTTP/1.1" 502 150 2025-12-17 23:03:19,757 - lava_callback - ERROR - Error processing callback for node 694090e1575bc060ce1606ec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090e1575bc060ce1606ec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090e1575bc060ce1606ec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090e1575bc060ce1606ec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:19,758 - lava_callback - INFO - Starting async job submit for node 694090fd575bc060ce16071d 2025-12-17 23:03:19,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:19,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fd575bc060ce16071d HTTP/1.1" 502 150 2025-12-17 23:03:19,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fd575bc060ce16071d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:19,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fd575bc060ce16071d 2025-12-17 23:03:19,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fd575bc060ce16071d HTTP/1.1" 502 150 2025-12-17 23:03:19,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fd575bc060ce16071d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:21,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e3575bc060ce1606ef 2025-12-17 23:03:21,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e5575bc060ce1606f2 2025-12-17 23:03:21,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fd575bc060ce16071d 2025-12-17 23:03:21,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fd575bc060ce16071d HTTP/1.1" 502 150 2025-12-17 23:03:21,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fd575bc060ce16071d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:22,121 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fb575bc060ce160717 2025-12-17 23:03:22,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fb575bc060ce160717 HTTP/1.1" 502 150 2025-12-17 23:03:22,122 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fb575bc060ce160717'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:22,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f2575bc060ce160708 2025-12-17 23:03:22,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f2575bc060ce160708 HTTP/1.1" 502 150 2025-12-17 23:03:22,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f2575bc060ce160708'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:23,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f4575bc060ce16070e 2025-12-17 23:03:23,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f4575bc060ce16070e HTTP/1.1" 502 150 2025-12-17 23:03:23,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f4575bc060ce16070e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:23,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e7575bc060ce1606f5 HTTP/1.1" 502 150 2025-12-17 23:03:23,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090e7575bc060ce1606f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:23,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fc575bc060ce16071a 2025-12-17 23:03:23,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fc575bc060ce16071a HTTP/1.1" 502 150 2025-12-17 23:03:23,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fc575bc060ce16071a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:24,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ea575bc060ce1606f8 2025-12-17 23:03:24,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ea575bc060ce1606f8 HTTP/1.1" 502 150 2025-12-17 23:03:24,559 - lava_callback - ERROR - Error processing callback for node 694090ea575bc060ce1606f8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ea575bc060ce1606f8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ea575bc060ce1606f8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ea575bc060ce1606f8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:24,561 - lava_callback - INFO - Starting async job submit for node 694090fe575bc060ce160720 2025-12-17 23:03:24,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:24,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fe575bc060ce160720 HTTP/1.1" 502 150 2025-12-17 23:03:24,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fe575bc060ce160720'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:24,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fe575bc060ce160720 2025-12-17 23:03:24,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fe575bc060ce160720 HTTP/1.1" 502 150 2025-12-17 23:03:24,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fe575bc060ce160720'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:24,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ec575bc060ce1606fb 2025-12-17 23:03:24,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ec575bc060ce1606fb HTTP/1.1" 502 150 2025-12-17 23:03:24,666 - lava_callback - ERROR - Error processing callback for node 694090ec575bc060ce1606fb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ec575bc060ce1606fb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ec575bc060ce1606fb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ec575bc060ce1606fb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:24,667 - lava_callback - INFO - Starting async job submit for node 69409103575bc060ce16072b 2025-12-17 23:03:24,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:24,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409103575bc060ce16072b HTTP/1.1" 502 150 2025-12-17 23:03:24,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409103575bc060ce16072b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:24,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409103575bc060ce16072b 2025-12-17 23:03:24,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409103575bc060ce16072b HTTP/1.1" 502 150 2025-12-17 23:03:24,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409103575bc060ce16072b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:24,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090ee575bc060ce160702 2025-12-17 23:03:24,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090ee575bc060ce160702 HTTP/1.1" 502 150 2025-12-17 23:03:24,941 - lava_callback - ERROR - Error processing callback for node 694090ee575bc060ce160702: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ee575bc060ce160702 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ee575bc060ce160702 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090ee575bc060ce160702 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:24,943 - lava_callback - INFO - Starting async job submit for node 69409105575bc060ce16074a 2025-12-17 23:03:24,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:24,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409105575bc060ce16074a HTTP/1.1" 502 150 2025-12-17 23:03:24,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409105575bc060ce16074a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:24,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409105575bc060ce16074a 2025-12-17 23:03:24,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409105575bc060ce16074a HTTP/1.1" 502 150 2025-12-17 23:03:24,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409105575bc060ce16074a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:25,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f0575bc060ce160705 2025-12-17 23:03:25,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f0575bc060ce160705 HTTP/1.1" 502 150 2025-12-17 23:03:25,751 - lava_callback - ERROR - Error processing callback for node 694090f0575bc060ce160705: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f0575bc060ce160705 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f0575bc060ce160705 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f0575bc060ce160705 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:25,752 - lava_callback - INFO - Starting async job submit for node 69409106575bc060ce160751 2025-12-17 23:03:25,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:25,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409106575bc060ce160751 HTTP/1.1" 502 150 2025-12-17 23:03:25,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409106575bc060ce160751'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:25,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409106575bc060ce160751 2025-12-17 23:03:25,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409106575bc060ce160751 HTTP/1.1" 502 150 2025-12-17 23:03:25,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409106575bc060ce160751'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:25,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fd575bc060ce16071d 2025-12-17 23:03:25,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fd575bc060ce16071d HTTP/1.1" 502 150 2025-12-17 23:03:25,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fd575bc060ce16071d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:26,228 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940909f575bc060ce160608 2025-12-17 23:03:26,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940909f575bc060ce160608 HTTP/1.1" 502 150 2025-12-17 23:03:26,230 - lava_callback - ERROR - Error processing callback for node 6940909f575bc060ce160608: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940909f575bc060ce160608 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940909f575bc060ce160608 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940909f575bc060ce160608 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:26,232 - lava_callback - INFO - Starting async job submit for node 69409107575bc060ce160754 2025-12-17 23:03:26,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:26,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409107575bc060ce160754 HTTP/1.1" 502 150 2025-12-17 23:03:26,239 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409107575bc060ce160754'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:26,239 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409107575bc060ce160754 2025-12-17 23:03:26,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409107575bc060ce160754 HTTP/1.1" 502 150 2025-12-17 23:03:26,240 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409107575bc060ce160754'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:26,356 - lava_callback - INFO - Received callback for node 694093a9575bc060ce160e04 INFO: 172.21.0.1:50392 - "POST /node/694093a9575bc060ce160e04 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:50394 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:03:26,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fe575bc060ce160720 2025-12-17 23:03:26,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fe575bc060ce160720 HTTP/1.1" 502 150 2025-12-17 23:03:26,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fe575bc060ce160720'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:26,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f6575bc060ce160711 2025-12-17 23:03:26,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f6575bc060ce160711 HTTP/1.1" 502 150 2025-12-17 23:03:26,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f6575bc060ce160711'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:26,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409103575bc060ce16072b 2025-12-17 23:03:26,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409103575bc060ce16072b HTTP/1.1" 502 150 2025-12-17 23:03:26,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409103575bc060ce16072b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:26,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409105575bc060ce16074a 2025-12-17 23:03:26,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409105575bc060ce16074a HTTP/1.1" 502 150 2025-12-17 23:03:26,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409105575bc060ce16074a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:27,018 - lava_callback - INFO - Received callback for node 694093ab575bc060ce160e0f INFO: 172.21.0.1:50400 - "POST /node/694093ab575bc060ce160e0f HTTP/1.0" 202 Accepted 2025-12-17 23:03:27,100 - lava_callback - INFO - Received callback for node 694093b7575bc060ce160e29 INFO: 172.21.0.1:50412 - "POST /node/694093b7575bc060ce160e29 HTTP/1.0" 202 Accepted 2025-12-17 23:03:27,215 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f8575bc060ce160714 2025-12-17 23:03:27,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f8575bc060ce160714 HTTP/1.1" 502 150 2025-12-17 23:03:27,216 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090f8575bc060ce160714'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:27,684 - lava_callback - INFO - Received callback for node 694093ad575bc060ce160e14 INFO: 172.21.0.1:50422 - "POST /node/694093ad575bc060ce160e14 HTTP/1.0" 202 Accepted 2025-12-17 23:03:27,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409106575bc060ce160751 2025-12-17 23:03:27,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409106575bc060ce160751 HTTP/1.1" 502 150 2025-12-17 23:03:27,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409106575bc060ce160751'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:28,240 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409107575bc060ce160754 2025-12-17 23:03:28,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409107575bc060ce160754 HTTP/1.1" 502 150 2025-12-17 23:03:28,241 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409107575bc060ce160754'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:28,334 - lava_callback - INFO - Received callback for node 694093ae575bc060ce160e17 INFO: 172.21.0.1:50430 - "POST /node/694093ae575bc060ce160e17 HTTP/1.0" 202 Accepted 2025-12-17 23:03:28,999 - lava_callback - INFO - Received callback for node 694093b0575bc060ce160e1a INFO: 172.21.0.1:50442 - "POST /node/694093b0575bc060ce160e1a HTTP/1.0" 202 Accepted 2025-12-17 23:03:29,668 - lava_callback - INFO - Received callback for node 694093b2575bc060ce160e1d INFO: 172.21.0.1:50454 - "POST /node/694093b2575bc060ce160e1d HTTP/1.0" 202 Accepted 2025-12-17 23:03:30,122 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fb575bc060ce160717 2025-12-17 23:03:30,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fb575bc060ce160717 HTTP/1.1" 502 150 2025-12-17 23:03:30,124 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fb575bc060ce160717'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:30,304 - lava_callback - INFO - Received callback for node 694093b3575bc060ce160e20 INFO: 172.21.0.1:43518 - "POST /node/694093b3575bc060ce160e20 HTTP/1.0" 202 Accepted 2025-12-17 23:03:30,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fe575bc060ce160720 2025-12-17 23:03:30,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fe575bc060ce160720 HTTP/1.1" 502 150 2025-12-17 23:03:30,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fe575bc060ce160720'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:30,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409103575bc060ce16072b 2025-12-17 23:03:30,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409103575bc060ce16072b HTTP/1.1" 502 150 2025-12-17 23:03:30,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409103575bc060ce16072b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:30,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409105575bc060ce16074a 2025-12-17 23:03:30,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409105575bc060ce16074a HTTP/1.1" 502 150 2025-12-17 23:03:30,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409105575bc060ce16074a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:30,972 - lava_callback - INFO - Received callback for node 694093b4575bc060ce160e23 INFO: 172.21.0.1:43520 - "POST /node/694093b4575bc060ce160e23 HTTP/1.0" 202 Accepted 2025-12-17 23:03:31,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fc575bc060ce16071a 2025-12-17 23:03:31,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fc575bc060ce16071a HTTP/1.1" 502 150 2025-12-17 23:03:31,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fc575bc060ce16071a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:31,700 - lava_callback - INFO - Received callback for node 694093b6575bc060ce160e26 INFO: 172.21.0.1:43528 - "POST /node/694093b6575bc060ce160e26 HTTP/1.0" 202 Accepted 2025-12-17 23:03:31,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409106575bc060ce160751 2025-12-17 23:03:31,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409106575bc060ce160751 HTTP/1.1" 502 150 2025-12-17 23:03:31,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409106575bc060ce160751'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:32,242 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409107575bc060ce160754 2025-12-17 23:03:32,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409107575bc060ce160754 HTTP/1.1" 502 150 2025-12-17 23:03:32,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409107575bc060ce160754'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:32,347 - lava_callback - INFO - Received callback for node 694093b9575bc060ce160e2c INFO: 172.21.0.1:43544 - "POST /node/694093b9575bc060ce160e2c HTTP/1.0" 202 Accepted 2025-12-17 23:03:33,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fd575bc060ce16071d 2025-12-17 23:03:33,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fd575bc060ce16071d HTTP/1.1" 502 150 2025-12-17 23:03:33,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fd575bc060ce16071d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:35,365 - lava_callback - INFO - Received callback for node 694095e0575bc060ce161466 INFO: 172.21.0.1:43550 - "POST /node/694095e0575bc060ce161466 HTTP/1.0" 202 Accepted 2025-12-17 23:03:35,971 - lava_callback - INFO - Received callback for node 694093ba575bc060ce160e2f INFO: 172.21.0.1:43558 - "POST /node/694093ba575bc060ce160e2f HTTP/1.0" 202 Accepted INFO: 172.21.0.1:43566 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:03:36,639 - lava_callback - INFO - Received callback for node 694093bd575bc060ce160e34 INFO: 172.21.0.1:43568 - "POST /node/694093bd575bc060ce160e34 HTTP/1.0" 202 Accepted 2025-12-17 23:03:36,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e3575bc060ce1606ef HTTP/1.1" 502 150 2025-12-17 23:03:36,880 - lava_callback - ERROR - Error processing callback for node 694090e3575bc060ce1606ef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090e3575bc060ce1606ef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090e3575bc060ce1606ef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090e3575bc060ce1606ef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:36,882 - lava_callback - INFO - Starting async job submit for node 69409109575bc060ce16075b 2025-12-17 23:03:36,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:36,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409109575bc060ce16075b HTTP/1.1" 502 150 2025-12-17 23:03:36,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409109575bc060ce16075b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:36,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409109575bc060ce16075b 2025-12-17 23:03:36,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409109575bc060ce16075b HTTP/1.1" 502 150 2025-12-17 23:03:36,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409109575bc060ce16075b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:37,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e5575bc060ce1606f2 HTTP/1.1" 502 150 2025-12-17 23:03:37,136 - lava_callback - ERROR - Error processing callback for node 694090e5575bc060ce1606f2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090e5575bc060ce1606f2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090e5575bc060ce1606f2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090e5575bc060ce1606f2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:37,138 - lava_callback - INFO - Starting async job submit for node 6940910a575bc060ce16075e 2025-12-17 23:03:37,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:37,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910a575bc060ce16075e HTTP/1.1" 502 150 2025-12-17 23:03:37,145 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940910a575bc060ce16075e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:37,145 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940910a575bc060ce16075e 2025-12-17 23:03:37,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910a575bc060ce16075e HTTP/1.1" 502 150 2025-12-17 23:03:37,146 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940910a575bc060ce16075e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:37,296 - lava_callback - INFO - Received callback for node 694093bf575bc060ce160e45 INFO: 172.21.0.1:43572 - "POST /node/694093bf575bc060ce160e45 HTTP/1.0" 202 Accepted 2025-12-17 23:03:37,948 - lava_callback - INFO - Received callback for node 694093c2575bc060ce160e4c INFO: 172.21.0.1:43582 - "POST /node/694093c2575bc060ce160e4c HTTP/1.0" 202 Accepted 2025-12-17 23:03:38,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fe575bc060ce160720 2025-12-17 23:03:38,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fe575bc060ce160720 HTTP/1.1" 502 150 2025-12-17 23:03:38,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090fe575bc060ce160720'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:38,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f2575bc060ce160708 2025-12-17 23:03:38,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f2575bc060ce160708 HTTP/1.1" 502 150 2025-12-17 23:03:38,616 - lava_callback - ERROR - Error processing callback for node 694090f2575bc060ce160708: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f2575bc060ce160708 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f2575bc060ce160708 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f2575bc060ce160708 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:38,618 - lava_callback - INFO - Starting async job submit for node 6940910c575bc060ce160761 2025-12-17 23:03:38,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:38,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910c575bc060ce160761 HTTP/1.1" 502 150 2025-12-17 23:03:38,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940910c575bc060ce160761'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:38,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940910c575bc060ce160761 2025-12-17 23:03:38,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910c575bc060ce160761 HTTP/1.1" 502 150 2025-12-17 23:03:38,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940910c575bc060ce160761'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:38,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409103575bc060ce16072b 2025-12-17 23:03:38,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409103575bc060ce16072b HTTP/1.1" 502 150 2025-12-17 23:03:38,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409103575bc060ce16072b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:38,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409109575bc060ce16075b 2025-12-17 23:03:38,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409109575bc060ce16075b HTTP/1.1" 502 150 2025-12-17 23:03:38,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409109575bc060ce16075b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:38,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409105575bc060ce16074a 2025-12-17 23:03:38,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409105575bc060ce16074a HTTP/1.1" 502 150 2025-12-17 23:03:38,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409105575bc060ce16074a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:39,146 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940910a575bc060ce16075e 2025-12-17 23:03:39,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910a575bc060ce16075e HTTP/1.1" 502 150 2025-12-17 23:03:39,148 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940910a575bc060ce16075e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:39,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f4575bc060ce16070e 2025-12-17 23:03:39,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f4575bc060ce16070e HTTP/1.1" 502 150 2025-12-17 23:03:39,505 - lava_callback - ERROR - Error processing callback for node 694090f4575bc060ce16070e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f4575bc060ce16070e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f4575bc060ce16070e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f4575bc060ce16070e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:39,508 - lava_callback - INFO - Starting async job submit for node 6940910e575bc060ce160768 2025-12-17 23:03:39,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:39,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910e575bc060ce160768 HTTP/1.1" 502 150 2025-12-17 23:03:39,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940910e575bc060ce160768'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:39,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940910e575bc060ce160768 2025-12-17 23:03:39,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910e575bc060ce160768 HTTP/1.1" 502 150 2025-12-17 23:03:39,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940910e575bc060ce160768'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:39,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090e7575bc060ce1606f5 2025-12-17 23:03:39,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090e7575bc060ce1606f5 HTTP/1.1" 502 150 2025-12-17 23:03:39,570 - lava_callback - ERROR - Error processing callback for node 694090e7575bc060ce1606f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090e7575bc060ce1606f5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090e7575bc060ce1606f5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090e7575bc060ce1606f5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:39,571 - lava_callback - INFO - Starting async job submit for node 69409110575bc060ce16076b 2025-12-17 23:03:39,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:39,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409110575bc060ce16076b HTTP/1.1" 502 150 2025-12-17 23:03:39,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409110575bc060ce16076b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:39,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409110575bc060ce16076b 2025-12-17 23:03:39,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409110575bc060ce16076b HTTP/1.1" 502 150 2025-12-17 23:03:39,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409110575bc060ce16076b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:39,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409106575bc060ce160751 2025-12-17 23:03:39,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409106575bc060ce160751 HTTP/1.1" 502 150 2025-12-17 23:03:39,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409106575bc060ce160751'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:39,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d7575bc060ce1606d5 HTTP/1.1" 502 150 2025-12-17 23:03:39,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d7575bc060ce1606d5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:40,243 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409107575bc060ce160754 2025-12-17 23:03:40,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409107575bc060ce160754 HTTP/1.1" 502 150 2025-12-17 23:03:40,244 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409107575bc060ce160754'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:40,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940910c575bc060ce160761 2025-12-17 23:03:40,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910c575bc060ce160761 HTTP/1.1" 502 150 2025-12-17 23:03:40,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940910c575bc060ce160761'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:41,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940910e575bc060ce160768 2025-12-17 23:03:41,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910e575bc060ce160768 HTTP/1.1" 502 150 2025-12-17 23:03:41,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940910e575bc060ce160768'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:41,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409110575bc060ce16076b 2025-12-17 23:03:41,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409110575bc060ce16076b HTTP/1.1" 502 150 2025-12-17 23:03:41,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409110575bc060ce16076b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:41,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d7575bc060ce1606d5 2025-12-17 23:03:42,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f6575bc060ce160711 2025-12-17 23:03:42,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f6575bc060ce160711 HTTP/1.1" 502 150 2025-12-17 23:03:42,617 - lava_callback - ERROR - Error processing callback for node 694090f6575bc060ce160711: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f6575bc060ce160711 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f6575bc060ce160711 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f6575bc060ce160711 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:42,619 - lava_callback - INFO - Starting async job submit for node 69409112575bc060ce16076e 2025-12-17 23:03:42,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:42,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409112575bc060ce16076e HTTP/1.1" 502 150 2025-12-17 23:03:42,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409112575bc060ce16076e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:42,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409112575bc060ce16076e 2025-12-17 23:03:42,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409112575bc060ce16076e HTTP/1.1" 502 150 2025-12-17 23:03:42,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409112575bc060ce16076e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:42,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409109575bc060ce16075b 2025-12-17 23:03:42,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409109575bc060ce16075b HTTP/1.1" 502 150 2025-12-17 23:03:42,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409109575bc060ce16075b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:43,150 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940910a575bc060ce16075e 2025-12-17 23:03:43,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910a575bc060ce16075e HTTP/1.1" 502 150 2025-12-17 23:03:43,153 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940910a575bc060ce16075e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:43,216 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090f8575bc060ce160714 2025-12-17 23:03:43,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090f8575bc060ce160714 HTTP/1.1" 502 150 2025-12-17 23:03:43,218 - lava_callback - ERROR - Error processing callback for node 694090f8575bc060ce160714: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f8575bc060ce160714 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f8575bc060ce160714 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090f8575bc060ce160714 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:43,221 - lava_callback - INFO - Starting async job submit for node 69409114575bc060ce160773 2025-12-17 23:03:43,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:43,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409114575bc060ce160773 HTTP/1.1" 502 150 2025-12-17 23:03:43,246 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409114575bc060ce160773'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:43,246 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409114575bc060ce160773 2025-12-17 23:03:43,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409114575bc060ce160773 HTTP/1.1" 502 150 2025-12-17 23:03:43,248 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409114575bc060ce160773'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:44,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940910c575bc060ce160761 2025-12-17 23:03:44,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910c575bc060ce160761 HTTP/1.1" 502 150 2025-12-17 23:03:44,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940910c575bc060ce160761'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:44,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409112575bc060ce16076e 2025-12-17 23:03:44,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409112575bc060ce16076e HTTP/1.1" 502 150 2025-12-17 23:03:44,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409112575bc060ce16076e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:45,250 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409114575bc060ce160773 2025-12-17 23:03:45,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409114575bc060ce160773 HTTP/1.1" 502 150 2025-12-17 23:03:45,251 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409114575bc060ce160773'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:45,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940910e575bc060ce160768 2025-12-17 23:03:45,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910e575bc060ce160768 HTTP/1.1" 502 150 2025-12-17 23:03:45,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940910e575bc060ce160768'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:45,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409110575bc060ce16076b 2025-12-17 23:03:45,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409110575bc060ce16076b HTTP/1.1" 502 150 2025-12-17 23:03:45,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409110575bc060ce16076b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:46,124 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fb575bc060ce160717 2025-12-17 23:03:46,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fb575bc060ce160717 HTTP/1.1" 502 150 2025-12-17 23:03:46,125 - lava_callback - ERROR - Error processing callback for node 694090fb575bc060ce160717: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090fb575bc060ce160717 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090fb575bc060ce160717 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090fb575bc060ce160717 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:46,127 - lava_callback - INFO - Starting async job submit for node 694106df460bc8e265116fe2 2025-12-17 23:03:46,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:46,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694106df460bc8e265116fe2 HTTP/1.1" 502 150 2025-12-17 23:03:46,153 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694106df460bc8e265116fe2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:46,154 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694106df460bc8e265116fe2 2025-12-17 23:03:46,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694106df460bc8e265116fe2 HTTP/1.1" 502 150 2025-12-17 23:03:46,154 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694106df460bc8e265116fe2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:46,372 - lava_callback - INFO - Received callback for node 694093c3575bc060ce160e4f INFO: 172.21.0.1:48398 - "POST /node/694093c3575bc060ce160e4f HTTP/1.0" 202 Accepted INFO: 172.21.0.1:48412 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:03:47,042 - lava_callback - INFO - Received callback for node 694093c6575bc060ce160e55 INFO: 172.21.0.1:48418 - "POST /node/694093c6575bc060ce160e55 HTTP/1.0" 202 Accepted 2025-12-17 23:03:47,218 - lava_callback - INFO - Received callback for node 694093c5575bc060ce160e52 INFO: 172.21.0.1:48424 - "POST /node/694093c5575bc060ce160e52 HTTP/1.0" 202 Accepted 2025-12-17 23:03:47,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fc575bc060ce16071a 2025-12-17 23:03:47,695 - lava_callback - INFO - Received callback for node 694093c8575bc060ce160e5b INFO: 172.21.0.1:48426 - "POST /node/694093c8575bc060ce160e5b HTTP/1.0" 202 Accepted 2025-12-17 23:03:48,154 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694106df460bc8e265116fe2 2025-12-17 23:03:48,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694106df460bc8e265116fe2 HTTP/1.1" 502 150 2025-12-17 23:03:48,156 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694106df460bc8e265116fe2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:48,338 - lava_callback - INFO - Received callback for node 694093cb575bc060ce160e61 INFO: 172.21.0.1:48434 - "POST /node/694093cb575bc060ce160e61 HTTP/1.0" 202 Accepted 2025-12-17 23:03:48,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409112575bc060ce16076e 2025-12-17 23:03:48,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409112575bc060ce16076e HTTP/1.1" 502 150 2025-12-17 23:03:48,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409112575bc060ce16076e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:48,988 - lava_callback - INFO - Received callback for node 694093cc575bc060ce160e64 INFO: 172.21.0.1:48444 - "POST /node/694093cc575bc060ce160e64 HTTP/1.0" 202 Accepted 2025-12-17 23:03:49,251 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409114575bc060ce160773 2025-12-17 23:03:49,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409114575bc060ce160773 HTTP/1.1" 502 150 2025-12-17 23:03:49,252 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409114575bc060ce160773'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:49,627 - lava_callback - INFO - Received callback for node 694093ce575bc060ce160e67 INFO: 172.21.0.1:48450 - "POST /node/694093ce575bc060ce160e67 HTTP/1.0" 202 Accepted 2025-12-17 23:03:49,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fd575bc060ce16071d 2025-12-17 23:03:49,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fd575bc060ce16071d HTTP/1.1" 502 150 2025-12-17 23:03:49,772 - lava_callback - ERROR - Error processing callback for node 694090fd575bc060ce16071d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090fd575bc060ce16071d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090fd575bc060ce16071d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090fd575bc060ce16071d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:49,774 - lava_callback - INFO - Starting async job submit for node 6940911b575bc060ce160784 2025-12-17 23:03:49,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:49,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940911b575bc060ce160784 HTTP/1.1" 502 150 2025-12-17 23:03:49,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940911b575bc060ce160784'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:49,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940911b575bc060ce160784 2025-12-17 23:03:49,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940911b575bc060ce160784 HTTP/1.1" 502 150 2025-12-17 23:03:49,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940911b575bc060ce160784'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:50,266 - lava_callback - INFO - Received callback for node 694093cf575bc060ce160e6a INFO: 172.21.0.1:55126 - "POST /node/694093cf575bc060ce160e6a HTTP/1.0" 202 Accepted 2025-12-17 23:03:50,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409109575bc060ce16075b 2025-12-17 23:03:50,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409109575bc060ce16075b HTTP/1.1" 502 150 2025-12-17 23:03:50,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409109575bc060ce16075b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:50,914 - lava_callback - INFO - Received callback for node 694093d0575bc060ce160e6d INFO: 172.21.0.1:55132 - "POST /node/694093d0575bc060ce160e6d HTTP/1.0" 202 Accepted 2025-12-17 23:03:51,153 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940910a575bc060ce16075e 2025-12-17 23:03:51,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910a575bc060ce16075e HTTP/1.1" 502 150 2025-12-17 23:03:51,155 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940910a575bc060ce16075e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:51,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940911b575bc060ce160784 2025-12-17 23:03:51,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940911b575bc060ce160784 HTTP/1.1" 502 150 2025-12-17 23:03:51,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940911b575bc060ce160784'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:52,156 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694106df460bc8e265116fe2 2025-12-17 23:03:52,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694106df460bc8e265116fe2 HTTP/1.1" 502 150 2025-12-17 23:03:52,157 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694106df460bc8e265116fe2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:52,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940910c575bc060ce160761 2025-12-17 23:03:52,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910c575bc060ce160761 HTTP/1.1" 502 150 2025-12-17 23:03:52,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940910c575bc060ce160761'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:53,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940910e575bc060ce160768 2025-12-17 23:03:53,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910e575bc060ce160768 HTTP/1.1" 502 150 2025-12-17 23:03:53,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940910e575bc060ce160768'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:53,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409110575bc060ce16076b 2025-12-17 23:03:53,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409110575bc060ce16076b HTTP/1.1" 502 150 2025-12-17 23:03:53,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409110575bc060ce16076b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:54,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090fe575bc060ce160720 2025-12-17 23:03:54,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fe575bc060ce160720 HTTP/1.1" 502 150 2025-12-17 23:03:54,584 - lava_callback - ERROR - Error processing callback for node 694090fe575bc060ce160720: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090fe575bc060ce160720 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090fe575bc060ce160720 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090fe575bc060ce160720 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:54,586 - lava_callback - INFO - Starting async job submit for node 694091a7575bc060ce1608f4 2025-12-17 23:03:54,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:54,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a7575bc060ce1608f4 HTTP/1.1" 502 150 2025-12-17 23:03:54,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a7575bc060ce1608f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:54,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a7575bc060ce1608f4 2025-12-17 23:03:54,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a7575bc060ce1608f4 HTTP/1.1" 502 150 2025-12-17 23:03:54,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a7575bc060ce1608f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:54,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409103575bc060ce16072b 2025-12-17 23:03:54,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409103575bc060ce16072b HTTP/1.1" 502 150 2025-12-17 23:03:54,679 - lava_callback - ERROR - Error processing callback for node 69409103575bc060ce16072b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409103575bc060ce16072b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409103575bc060ce16072b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409103575bc060ce16072b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:54,681 - lava_callback - INFO - Starting async job submit for node 69409116575bc060ce16077c 2025-12-17 23:03:54,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:54,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409116575bc060ce16077c HTTP/1.1" 502 150 2025-12-17 23:03:54,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409116575bc060ce16077c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:54,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409116575bc060ce16077c 2025-12-17 23:03:54,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409116575bc060ce16077c HTTP/1.1" 502 150 2025-12-17 23:03:54,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409116575bc060ce16077c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:54,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409105575bc060ce16074a 2025-12-17 23:03:54,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409105575bc060ce16074a HTTP/1.1" 502 150 2025-12-17 23:03:54,955 - lava_callback - ERROR - Error processing callback for node 69409105575bc060ce16074a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409105575bc060ce16074a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409105575bc060ce16074a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409105575bc060ce16074a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:54,957 - lava_callback - INFO - Starting async job submit for node 69409118575bc060ce160781 2025-12-17 23:03:54,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:54,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409118575bc060ce160781 HTTP/1.1" 502 150 2025-12-17 23:03:54,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409118575bc060ce160781'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:54,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409118575bc060ce160781 2025-12-17 23:03:54,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409118575bc060ce160781 HTTP/1.1" 502 150 2025-12-17 23:03:54,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409118575bc060ce160781'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:55,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409106575bc060ce160751 2025-12-17 23:03:55,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409106575bc060ce160751 HTTP/1.1" 502 150 2025-12-17 23:03:55,766 - lava_callback - ERROR - Error processing callback for node 69409106575bc060ce160751: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409106575bc060ce160751 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409106575bc060ce160751 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409106575bc060ce160751 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:55,768 - lava_callback - INFO - Starting async job submit for node 6940911d575bc060ce160789 2025-12-17 23:03:55,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:55,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940911d575bc060ce160789 HTTP/1.1" 502 150 2025-12-17 23:03:55,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940911d575bc060ce160789'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:55,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940911d575bc060ce160789 2025-12-17 23:03:55,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940911d575bc060ce160789 HTTP/1.1" 502 150 2025-12-17 23:03:55,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940911d575bc060ce160789'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:55,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940911b575bc060ce160784 2025-12-17 23:03:55,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940911b575bc060ce160784 HTTP/1.1" 502 150 2025-12-17 23:03:55,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940911b575bc060ce160784'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:55,953 - lava_callback - INFO - Received callback for node 694093c7575bc060ce160e58 INFO: 172.21.0.1:55144 - "POST /node/694093c7575bc060ce160e58 HTTP/1.0" 202 Accepted 2025-12-17 23:03:56,245 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409107575bc060ce160754 2025-12-17 23:03:56,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409107575bc060ce160754 HTTP/1.1" 502 150 2025-12-17 23:03:56,246 - lava_callback - ERROR - Error processing callback for node 69409107575bc060ce160754: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409107575bc060ce160754 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409107575bc060ce160754 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409107575bc060ce160754 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:03:56,248 - lava_callback - INFO - Starting async job submit for node 69409120575bc060ce16078c 2025-12-17 23:03:56,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:03:56,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409120575bc060ce16078c HTTP/1.1" 502 150 2025-12-17 23:03:56,277 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409120575bc060ce16078c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:56,277 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409120575bc060ce16078c 2025-12-17 23:03:56,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409120575bc060ce16078c HTTP/1.1" 502 150 2025-12-17 23:03:56,279 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409120575bc060ce16078c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55158 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:03:56,625 - lava_callback - INFO - Received callback for node 694093ca575bc060ce160e5e INFO: 172.21.0.1:55174 - "POST /node/694093ca575bc060ce160e5e HTTP/1.0" 202 Accepted 2025-12-17 23:03:56,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a7575bc060ce1608f4 2025-12-17 23:03:56,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a7575bc060ce1608f4 HTTP/1.1" 502 150 2025-12-17 23:03:56,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a7575bc060ce1608f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:56,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409112575bc060ce16076e 2025-12-17 23:03:56,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409112575bc060ce16076e HTTP/1.1" 502 150 2025-12-17 23:03:56,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409112575bc060ce16076e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:56,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409116575bc060ce16077c 2025-12-17 23:03:56,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409116575bc060ce16077c HTTP/1.1" 502 150 2025-12-17 23:03:56,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409116575bc060ce16077c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:56,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409118575bc060ce160781 2025-12-17 23:03:56,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409118575bc060ce160781 HTTP/1.1" 502 150 2025-12-17 23:03:56,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409118575bc060ce160781'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:57,252 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409114575bc060ce160773 2025-12-17 23:03:57,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409114575bc060ce160773 HTTP/1.1" 502 150 2025-12-17 23:03:57,253 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409114575bc060ce160773'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:57,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940911d575bc060ce160789 2025-12-17 23:03:57,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940911d575bc060ce160789 HTTP/1.1" 502 150 2025-12-17 23:03:57,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940911d575bc060ce160789'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:03:58,279 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409120575bc060ce16078c 2025-12-17 23:04:00,158 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694106df460bc8e265116fe2 2025-12-17 23:04:00,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694106df460bc8e265116fe2 HTTP/1.1" 502 150 2025-12-17 23:04:00,159 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694106df460bc8e265116fe2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:00,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a7575bc060ce1608f4 2025-12-17 23:04:00,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a7575bc060ce1608f4 HTTP/1.1" 502 150 2025-12-17 23:04:00,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a7575bc060ce1608f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:00,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409116575bc060ce16077c 2025-12-17 23:04:00,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409116575bc060ce16077c HTTP/1.1" 502 150 2025-12-17 23:04:00,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409116575bc060ce16077c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:00,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409118575bc060ce160781 2025-12-17 23:04:00,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409118575bc060ce160781 HTTP/1.1" 502 150 2025-12-17 23:04:00,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409118575bc060ce160781'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:01,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940911d575bc060ce160789 2025-12-17 23:04:01,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940911d575bc060ce160789 HTTP/1.1" 502 150 2025-12-17 23:04:01,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940911d575bc060ce160789'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58380 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:04:02,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090fc575bc060ce16071a HTTP/1.1" 502 150 2025-12-17 23:04:02,992 - lava_callback - ERROR - Error processing callback for node 694090fc575bc060ce16071a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090fc575bc060ce16071a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090fc575bc060ce16071a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090fc575bc060ce16071a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:02,994 - lava_callback - INFO - Starting async job submit for node 69409122575bc060ce16078f 2025-12-17 23:04:02,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:03,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409122575bc060ce16078f HTTP/1.1" 502 150 2025-12-17 23:04:03,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409122575bc060ce16078f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:03,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409122575bc060ce16078f 2025-12-17 23:04:03,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409122575bc060ce16078f HTTP/1.1" 502 150 2025-12-17 23:04:03,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409122575bc060ce16078f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:03,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940911b575bc060ce160784 2025-12-17 23:04:03,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940911b575bc060ce160784 HTTP/1.1" 502 150 2025-12-17 23:04:03,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940911b575bc060ce160784'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:05,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409122575bc060ce16078f 2025-12-17 23:04:05,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409122575bc060ce16078f HTTP/1.1" 502 150 2025-12-17 23:04:05,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409122575bc060ce16078f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:06,368 - lava_callback - INFO - Received callback for node 694093d3575bc060ce160e7a INFO: 172.21.0.1:58386 - "POST /node/694093d3575bc060ce160e7a HTTP/1.0" 202 Accepted INFO: 172.21.0.1:58402 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:04:06,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409109575bc060ce16075b 2025-12-17 23:04:06,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409109575bc060ce16075b HTTP/1.1" 502 150 2025-12-17 23:04:06,895 - lava_callback - ERROR - Error processing callback for node 69409109575bc060ce16075b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409109575bc060ce16075b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409109575bc060ce16075b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409109575bc060ce16075b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:06,897 - lava_callback - INFO - Starting async job submit for node 69409123575bc060ce160792 2025-12-17 23:04:06,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:06,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409123575bc060ce160792 HTTP/1.1" 502 150 2025-12-17 23:04:06,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409123575bc060ce160792'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:06,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409123575bc060ce160792 2025-12-17 23:04:06,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409123575bc060ce160792 HTTP/1.1" 502 150 2025-12-17 23:04:06,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409123575bc060ce160792'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:07,030 - lava_callback - INFO - Received callback for node 694093d5575bc060ce160e85 INFO: 172.21.0.1:58418 - "POST /node/694093d5575bc060ce160e85 HTTP/1.0" 202 Accepted 2025-12-17 23:04:07,155 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940910a575bc060ce16075e 2025-12-17 23:04:07,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910a575bc060ce16075e HTTP/1.1" 502 150 2025-12-17 23:04:07,156 - lava_callback - ERROR - Error processing callback for node 6940910a575bc060ce16075e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940910a575bc060ce16075e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940910a575bc060ce16075e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940910a575bc060ce16075e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:07,158 - lava_callback - INFO - Starting async job submit for node 69409126575bc060ce160799 2025-12-17 23:04:07,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:07,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409126575bc060ce160799 HTTP/1.1" 502 150 2025-12-17 23:04:07,166 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409126575bc060ce160799'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:07,166 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409126575bc060ce160799 2025-12-17 23:04:07,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409126575bc060ce160799 HTTP/1.1" 502 150 2025-12-17 23:04:07,167 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409126575bc060ce160799'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:07,696 - lava_callback - INFO - Received callback for node 694093d7575bc060ce160e8c INFO: 172.21.0.1:58432 - "POST /node/694093d7575bc060ce160e8c HTTP/1.0" 202 Accepted 2025-12-17 23:04:08,343 - lava_callback - INFO - Received callback for node 694093d9575bc060ce160e99 INFO: 172.21.0.1:58446 - "POST /node/694093d9575bc060ce160e99 HTTP/1.0" 202 Accepted 2025-12-17 23:04:08,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940910c575bc060ce160761 2025-12-17 23:04:08,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910c575bc060ce160761 HTTP/1.1" 502 150 2025-12-17 23:04:08,633 - lava_callback - ERROR - Error processing callback for node 6940910c575bc060ce160761: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940910c575bc060ce160761 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940910c575bc060ce160761 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940910c575bc060ce160761 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:08,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a7575bc060ce1608f4 2025-12-17 23:04:08,635 - lava_callback - INFO - Starting async job submit for node 69409127575bc060ce16079c 2025-12-17 23:04:08,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:08,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a7575bc060ce1608f4 HTTP/1.1" 502 150 2025-12-17 23:04:08,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a7575bc060ce1608f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:08,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409127575bc060ce16079c HTTP/1.1" 502 150 2025-12-17 23:04:08,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409127575bc060ce16079c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:08,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409127575bc060ce16079c 2025-12-17 23:04:08,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409127575bc060ce16079c HTTP/1.1" 502 150 2025-12-17 23:04:08,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409127575bc060ce16079c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:08,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409116575bc060ce16077c 2025-12-17 23:04:08,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409116575bc060ce16077c HTTP/1.1" 502 150 2025-12-17 23:04:08,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409116575bc060ce16077c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:08,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409123575bc060ce160792 2025-12-17 23:04:08,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409123575bc060ce160792 HTTP/1.1" 502 150 2025-12-17 23:04:08,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409123575bc060ce160792'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:08,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409118575bc060ce160781 2025-12-17 23:04:08,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409118575bc060ce160781 HTTP/1.1" 502 150 2025-12-17 23:04:08,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409118575bc060ce160781'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:08,990 - lava_callback - INFO - Received callback for node 694093db575bc060ce160e9c INFO: 172.21.0.1:58458 - "POST /node/694093db575bc060ce160e9c HTTP/1.0" 202 Accepted 2025-12-17 23:04:09,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409122575bc060ce16078f 2025-12-17 23:04:09,167 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409126575bc060ce160799 2025-12-17 23:04:09,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409126575bc060ce160799 HTTP/1.1" 502 150 2025-12-17 23:04:09,168 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409126575bc060ce160799'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:09,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940910e575bc060ce160768 2025-12-17 23:04:09,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940910e575bc060ce160768 HTTP/1.1" 502 150 2025-12-17 23:04:09,531 - lava_callback - ERROR - Error processing callback for node 6940910e575bc060ce160768: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940910e575bc060ce160768 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940910e575bc060ce160768 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940910e575bc060ce160768 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:09,532 - lava_callback - INFO - Starting async job submit for node 69409129575bc060ce1607a5 2025-12-17 23:04:09,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:09,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409129575bc060ce1607a5 HTTP/1.1" 502 150 2025-12-17 23:04:09,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409129575bc060ce1607a5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:09,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409129575bc060ce1607a5 2025-12-17 23:04:09,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409129575bc060ce1607a5 HTTP/1.1" 502 150 2025-12-17 23:04:09,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409129575bc060ce1607a5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:09,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409110575bc060ce16076b 2025-12-17 23:04:09,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409110575bc060ce16076b HTTP/1.1" 502 150 2025-12-17 23:04:09,584 - lava_callback - ERROR - Error processing callback for node 69409110575bc060ce16076b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409110575bc060ce16076b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409110575bc060ce16076b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409110575bc060ce16076b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:09,586 - lava_callback - INFO - Starting async job submit for node 6940912c575bc060ce1607ac 2025-12-17 23:04:09,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:09,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912c575bc060ce1607ac HTTP/1.1" 502 150 2025-12-17 23:04:09,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940912c575bc060ce1607ac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:09,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940912c575bc060ce1607ac 2025-12-17 23:04:09,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912c575bc060ce1607ac HTTP/1.1" 502 150 2025-12-17 23:04:09,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940912c575bc060ce1607ac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:09,642 - lava_callback - INFO - Received callback for node 694093dd575bc060ce160e9f INFO: 172.21.0.1:58466 - "POST /node/694093dd575bc060ce160e9f HTTP/1.0" 202 Accepted 2025-12-17 23:04:09,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940911d575bc060ce160789 2025-12-17 23:04:09,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940911d575bc060ce160789 HTTP/1.1" 502 150 2025-12-17 23:04:09,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940911d575bc060ce160789'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:10,281 - lava_callback - INFO - Received callback for node 694093df575bc060ce160ea2 INFO: 172.21.0.1:41708 - "POST /node/694093df575bc060ce160ea2 HTTP/1.0" 202 Accepted 2025-12-17 23:04:10,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409127575bc060ce16079c 2025-12-17 23:04:10,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409127575bc060ce16079c HTTP/1.1" 502 150 2025-12-17 23:04:10,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409127575bc060ce16079c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:10,940 - lava_callback - INFO - Received callback for node 694093e1575bc060ce160ea5 INFO: 172.21.0.1:41724 - "POST /node/694093e1575bc060ce160ea5 HTTP/1.0" 202 Accepted 2025-12-17 23:04:11,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409129575bc060ce1607a5 2025-12-17 23:04:11,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409129575bc060ce1607a5 HTTP/1.1" 502 150 2025-12-17 23:04:11,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409129575bc060ce1607a5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:11,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940912c575bc060ce1607ac 2025-12-17 23:04:11,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912c575bc060ce1607ac HTTP/1.1" 502 150 2025-12-17 23:04:11,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940912c575bc060ce1607ac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:12,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409112575bc060ce16076e 2025-12-17 23:04:12,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409112575bc060ce16076e HTTP/1.1" 502 150 2025-12-17 23:04:12,678 - lava_callback - ERROR - Error processing callback for node 69409112575bc060ce16076e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409112575bc060ce16076e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409112575bc060ce16076e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409112575bc060ce16076e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:12,681 - lava_callback - INFO - Starting async job submit for node 6940912d575bc060ce1607af 2025-12-17 23:04:12,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:12,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912d575bc060ce1607af HTTP/1.1" 502 150 2025-12-17 23:04:12,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940912d575bc060ce1607af'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:12,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940912d575bc060ce1607af 2025-12-17 23:04:12,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912d575bc060ce1607af HTTP/1.1" 502 150 2025-12-17 23:04:12,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940912d575bc060ce1607af'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:12,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d7575bc060ce1606d5 HTTP/1.1" 502 150 2025-12-17 23:04:12,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d7575bc060ce1606d5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:12,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409123575bc060ce160792 2025-12-17 23:04:12,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409123575bc060ce160792 HTTP/1.1" 502 150 2025-12-17 23:04:12,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409123575bc060ce160792'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:13,168 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409126575bc060ce160799 2025-12-17 23:04:13,254 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409114575bc060ce160773 2025-12-17 23:04:13,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409114575bc060ce160773 HTTP/1.1" 502 150 2025-12-17 23:04:13,255 - lava_callback - ERROR - Error processing callback for node 69409114575bc060ce160773: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409114575bc060ce160773 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409114575bc060ce160773 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409114575bc060ce160773 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:13,258 - lava_callback - INFO - Starting async job submit for node 69409130575bc060ce1607b5 2025-12-17 23:04:13,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:13,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409130575bc060ce1607b5 HTTP/1.1" 502 150 2025-12-17 23:04:13,291 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409130575bc060ce1607b5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:13,291 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409130575bc060ce1607b5 2025-12-17 23:04:13,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409130575bc060ce1607b5 HTTP/1.1" 502 150 2025-12-17 23:04:13,293 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409130575bc060ce1607b5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:13,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409120575bc060ce16078c HTTP/1.1" 502 150 2025-12-17 23:04:13,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409120575bc060ce16078c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:14,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409127575bc060ce16079c 2025-12-17 23:04:14,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409127575bc060ce16079c HTTP/1.1" 502 150 2025-12-17 23:04:14,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409127575bc060ce16079c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:14,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940912d575bc060ce1607af 2025-12-17 23:04:14,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912d575bc060ce1607af HTTP/1.1" 502 150 2025-12-17 23:04:14,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940912d575bc060ce1607af'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41732 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:04:15,293 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409130575bc060ce1607b5 2025-12-17 23:04:15,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409130575bc060ce1607b5 HTTP/1.1" 502 150 2025-12-17 23:04:15,295 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409130575bc060ce1607b5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:15,450 - lava_callback - INFO - Received callback for node 69409835575bc060ce161a10 INFO: 172.21.0.1:41748 - "POST /node/69409835575bc060ce161a10 HTTP/1.0" 202 Accepted 2025-12-17 23:04:15,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409129575bc060ce1607a5 2025-12-17 23:04:15,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409129575bc060ce1607a5 HTTP/1.1" 502 150 2025-12-17 23:04:15,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409129575bc060ce1607a5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:15,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940912c575bc060ce1607ac 2025-12-17 23:04:15,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912c575bc060ce1607ac HTTP/1.1" 502 150 2025-12-17 23:04:15,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940912c575bc060ce1607ac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:15,963 - lava_callback - INFO - Received callback for node 694093e3575bc060ce160ea8 INFO: 172.21.0.1:41750 - "POST /node/694093e3575bc060ce160ea8 HTTP/1.0" 202 Accepted 2025-12-17 23:04:16,159 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694106df460bc8e265116fe2 2025-12-17 23:04:16,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694106df460bc8e265116fe2 HTTP/1.1" 502 150 2025-12-17 23:04:16,161 - lava_callback - ERROR - Error processing callback for node 694106df460bc8e265116fe2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694106df460bc8e265116fe2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694106df460bc8e265116fe2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694106df460bc8e265116fe2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:16,166 - lava_callback - INFO - Starting async job submit for node 69409132575bc060ce1607b8 2025-12-17 23:04:16,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:16,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409132575bc060ce1607b8 HTTP/1.1" 502 150 2025-12-17 23:04:16,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409132575bc060ce1607b8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:16,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409132575bc060ce1607b8 2025-12-17 23:04:16,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409132575bc060ce1607b8 HTTP/1.1" 502 150 2025-12-17 23:04:16,201 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409132575bc060ce1607b8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41752 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:04:16,651 - lava_callback - INFO - Received callback for node 694093ed575bc060ce160eca INFO: 172.21.0.1:41764 - "POST /node/694093ed575bc060ce160eca HTTP/1.0" 202 Accepted 2025-12-17 23:04:16,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d7575bc060ce1606d5 2025-12-17 23:04:17,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409120575bc060ce16078c 2025-12-17 23:04:17,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409120575bc060ce16078c HTTP/1.1" 502 150 2025-12-17 23:04:17,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409120575bc060ce16078c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:18,201 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409132575bc060ce1607b8 2025-12-17 23:04:18,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409132575bc060ce1607b8 HTTP/1.1" 502 150 2025-12-17 23:04:18,202 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409132575bc060ce1607b8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:18,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940912d575bc060ce1607af 2025-12-17 23:04:18,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912d575bc060ce1607af HTTP/1.1" 502 150 2025-12-17 23:04:18,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940912d575bc060ce1607af'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:19,295 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409130575bc060ce1607b5 2025-12-17 23:04:19,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409130575bc060ce1607b5 HTTP/1.1" 502 150 2025-12-17 23:04:19,296 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409130575bc060ce1607b5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:19,482 - lava_callback - INFO - Received callback for node 6940919e575bc060ce1608d4 INFO: 172.21.0.1:41772 - "POST /node/6940919e575bc060ce1608d4 HTTP/1.0" 202 Accepted 2025-12-17 23:04:19,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940911b575bc060ce160784 2025-12-17 23:04:19,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940911b575bc060ce160784 HTTP/1.1" 502 150 2025-12-17 23:04:19,819 - lava_callback - ERROR - Error processing callback for node 6940911b575bc060ce160784: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940911b575bc060ce160784 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940911b575bc060ce160784 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940911b575bc060ce160784 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:19,822 - lava_callback - INFO - Starting async job submit for node 69409134575bc060ce1607bb 2025-12-17 23:04:19,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:19,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409134575bc060ce1607bb HTTP/1.1" 502 150 2025-12-17 23:04:19,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409134575bc060ce1607bb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:19,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409134575bc060ce1607bb 2025-12-17 23:04:19,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409134575bc060ce1607bb HTTP/1.1" 502 150 2025-12-17 23:04:19,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409134575bc060ce1607bb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:20,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409123575bc060ce160792 2025-12-17 23:04:20,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409123575bc060ce160792 HTTP/1.1" 502 150 2025-12-17 23:04:20,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409123575bc060ce160792'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:21,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409134575bc060ce1607bb 2025-12-17 23:04:21,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409134575bc060ce1607bb HTTP/1.1" 502 150 2025-12-17 23:04:21,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409134575bc060ce1607bb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:22,203 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409132575bc060ce1607b8 2025-12-17 23:04:22,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409132575bc060ce1607b8 HTTP/1.1" 502 150 2025-12-17 23:04:22,204 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409132575bc060ce1607b8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:22,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409127575bc060ce16079c 2025-12-17 23:04:22,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409127575bc060ce16079c HTTP/1.1" 502 150 2025-12-17 23:04:22,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409127575bc060ce16079c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:23,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409129575bc060ce1607a5 2025-12-17 23:04:23,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409129575bc060ce1607a5 HTTP/1.1" 502 150 2025-12-17 23:04:23,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409129575bc060ce1607a5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:23,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940912c575bc060ce1607ac 2025-12-17 23:04:23,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912c575bc060ce1607ac HTTP/1.1" 502 150 2025-12-17 23:04:23,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940912c575bc060ce1607ac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:24,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409122575bc060ce16078f HTTP/1.1" 502 150 2025-12-17 23:04:24,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409122575bc060ce16078f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:24,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a7575bc060ce1608f4 2025-12-17 23:04:24,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a7575bc060ce1608f4 HTTP/1.1" 502 150 2025-12-17 23:04:24,638 - lava_callback - ERROR - Error processing callback for node 694091a7575bc060ce1608f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a7575bc060ce1608f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a7575bc060ce1608f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a7575bc060ce1608f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:24,640 - lava_callback - INFO - Starting async job submit for node 69409136575bc060ce1607be 2025-12-17 23:04:24,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:24,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409136575bc060ce1607be HTTP/1.1" 502 150 2025-12-17 23:04:24,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409136575bc060ce1607be'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:24,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409136575bc060ce1607be 2025-12-17 23:04:24,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409136575bc060ce1607be HTTP/1.1" 502 150 2025-12-17 23:04:24,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409136575bc060ce1607be'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:24,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409116575bc060ce16077c 2025-12-17 23:04:24,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409116575bc060ce16077c HTTP/1.1" 502 150 2025-12-17 23:04:24,692 - lava_callback - ERROR - Error processing callback for node 69409116575bc060ce16077c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409116575bc060ce16077c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409116575bc060ce16077c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409116575bc060ce16077c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:24,694 - lava_callback - INFO - Starting async job submit for node 6940913b575bc060ce1607c4 2025-12-17 23:04:24,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:24,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913b575bc060ce1607c4 HTTP/1.1" 502 150 2025-12-17 23:04:24,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940913b575bc060ce1607c4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:24,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940913b575bc060ce1607c4 2025-12-17 23:04:24,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913b575bc060ce1607c4 HTTP/1.1" 502 150 2025-12-17 23:04:24,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940913b575bc060ce1607c4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:24,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409118575bc060ce160781 2025-12-17 23:04:24,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409118575bc060ce160781 HTTP/1.1" 502 150 2025-12-17 23:04:24,971 - lava_callback - ERROR - Error processing callback for node 69409118575bc060ce160781: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409118575bc060ce160781 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409118575bc060ce160781 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409118575bc060ce160781 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:24,972 - lava_callback - INFO - Starting async job submit for node 6940913d575bc060ce1607cb 2025-12-17 23:04:24,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:24,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913d575bc060ce1607cb HTTP/1.1" 502 150 2025-12-17 23:04:24,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940913d575bc060ce1607cb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:24,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940913d575bc060ce1607cb 2025-12-17 23:04:25,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913d575bc060ce1607cb HTTP/1.1" 502 150 2025-12-17 23:04:25,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940913d575bc060ce1607cb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:25,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409120575bc060ce16078c 2025-12-17 23:04:25,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409120575bc060ce16078c HTTP/1.1" 502 150 2025-12-17 23:04:25,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409120575bc060ce16078c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:25,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940911d575bc060ce160789 2025-12-17 23:04:25,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940911d575bc060ce160789 HTTP/1.1" 502 150 2025-12-17 23:04:25,781 - lava_callback - ERROR - Error processing callback for node 6940911d575bc060ce160789: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940911d575bc060ce160789 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940911d575bc060ce160789 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940911d575bc060ce160789 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:25,782 - lava_callback - INFO - Starting async job submit for node 6940913f575bc060ce1607d2 2025-12-17 23:04:25,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:25,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913f575bc060ce1607d2 HTTP/1.1" 502 150 2025-12-17 23:04:25,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940913f575bc060ce1607d2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:25,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940913f575bc060ce1607d2 2025-12-17 23:04:25,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913f575bc060ce1607d2 HTTP/1.1" 502 150 2025-12-17 23:04:25,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940913f575bc060ce1607d2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:25,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409134575bc060ce1607bb 2025-12-17 23:04:25,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409134575bc060ce1607bb HTTP/1.1" 502 150 2025-12-17 23:04:25,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409134575bc060ce1607bb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:26,365 - lava_callback - INFO - Received callback for node 694093e8575bc060ce160eb8 INFO: 172.21.0.1:40476 - "POST /node/694093e8575bc060ce160eb8 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:40490 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:04:26,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409136575bc060ce1607be 2025-12-17 23:04:26,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409136575bc060ce1607be HTTP/1.1" 502 150 2025-12-17 23:04:26,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409136575bc060ce1607be'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:26,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940912d575bc060ce1607af 2025-12-17 23:04:26,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912d575bc060ce1607af HTTP/1.1" 502 150 2025-12-17 23:04:26,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940912d575bc060ce1607af'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:26,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940913b575bc060ce1607c4 2025-12-17 23:04:26,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913b575bc060ce1607c4 HTTP/1.1" 502 150 2025-12-17 23:04:26,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940913b575bc060ce1607c4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:27,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940913d575bc060ce1607cb 2025-12-17 23:04:27,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913d575bc060ce1607cb HTTP/1.1" 502 150 2025-12-17 23:04:27,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940913d575bc060ce1607cb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:27,026 - lava_callback - INFO - Received callback for node 694093eb575bc060ce160ec3 INFO: 172.21.0.1:40500 - "POST /node/694093eb575bc060ce160ec3 HTTP/1.0" 202 Accepted 2025-12-17 23:04:27,296 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409130575bc060ce1607b5 2025-12-17 23:04:27,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409130575bc060ce1607b5 HTTP/1.1" 502 150 2025-12-17 23:04:27,298 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409130575bc060ce1607b5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:27,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940913f575bc060ce1607d2 2025-12-17 23:04:27,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913f575bc060ce1607d2 HTTP/1.1" 502 150 2025-12-17 23:04:27,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940913f575bc060ce1607d2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:28,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409126575bc060ce160799 HTTP/1.1" 502 150 2025-12-17 23:04:28,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409126575bc060ce160799'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:30,204 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409132575bc060ce1607b8 2025-12-17 23:04:30,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409132575bc060ce1607b8 HTTP/1.1" 502 150 2025-12-17 23:04:30,206 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409132575bc060ce1607b8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:30,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409136575bc060ce1607be 2025-12-17 23:04:30,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409136575bc060ce1607be HTTP/1.1" 502 150 2025-12-17 23:04:30,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409136575bc060ce1607be'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:30,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940913b575bc060ce1607c4 2025-12-17 23:04:30,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913b575bc060ce1607c4 HTTP/1.1" 502 150 2025-12-17 23:04:30,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940913b575bc060ce1607c4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:31,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940913d575bc060ce1607cb 2025-12-17 23:04:31,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913d575bc060ce1607cb HTTP/1.1" 502 150 2025-12-17 23:04:31,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940913d575bc060ce1607cb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:31,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940913f575bc060ce1607d2 2025-12-17 23:04:31,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913f575bc060ce1607d2 HTTP/1.1" 502 150 2025-12-17 23:04:31,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940913f575bc060ce1607d2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:32,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d7575bc060ce1606d5 HTTP/1.1" 502 150 2025-12-17 23:04:32,180 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d7575bc060ce1606d5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:32,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409122575bc060ce16078f 2025-12-17 23:04:32,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409122575bc060ce16078f HTTP/1.1" 502 150 2025-12-17 23:04:32,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409122575bc060ce16078f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:33,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409134575bc060ce1607bb 2025-12-17 23:04:33,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409134575bc060ce1607bb HTTP/1.1" 502 150 2025-12-17 23:04:33,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409134575bc060ce1607bb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:35,990 - lava_callback - INFO - Received callback for node 694093ef575bc060ce160ecd INFO: 172.21.0.1:56452 - "POST /node/694093ef575bc060ce160ecd HTTP/1.0" 202 Accepted 2025-12-17 23:04:36,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409126575bc060ce160799 2025-12-17 23:04:36,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409126575bc060ce160799 HTTP/1.1" 502 150 2025-12-17 23:04:36,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409126575bc060ce160799'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:36,666 - lava_callback - INFO - Received callback for node 694093f1575bc060ce160ed4 INFO: 172.21.0.1:56468 - "POST /node/694093f1575bc060ce160ed4 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:56478 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:04:36,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409123575bc060ce160792 2025-12-17 23:04:36,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409123575bc060ce160792 HTTP/1.1" 502 150 2025-12-17 23:04:36,913 - lava_callback - ERROR - Error processing callback for node 69409123575bc060ce160792: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409123575bc060ce160792 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409123575bc060ce160792 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409123575bc060ce160792 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:36,915 - lava_callback - INFO - Starting async job submit for node 69409141575bc060ce1607e3 2025-12-17 23:04:36,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:36,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409141575bc060ce1607e3 HTTP/1.1" 502 150 2025-12-17 23:04:36,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409141575bc060ce1607e3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:36,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409141575bc060ce1607e3 2025-12-17 23:04:36,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409141575bc060ce1607e3 HTTP/1.1" 502 150 2025-12-17 23:04:36,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409141575bc060ce1607e3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:37,346 - lava_callback - INFO - Received callback for node 694093f2575bc060ce160ed7 INFO: 172.21.0.1:56486 - "POST /node/694093f2575bc060ce160ed7 HTTP/1.0" 202 Accepted 2025-12-17 23:04:38,011 - lava_callback - INFO - Received callback for node 694093f3575bc060ce160eda INFO: 172.21.0.1:56502 - "POST /node/694093f3575bc060ce160eda HTTP/1.0" 202 Accepted 2025-12-17 23:04:38,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409127575bc060ce16079c 2025-12-17 23:04:38,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409127575bc060ce16079c HTTP/1.1" 502 150 2025-12-17 23:04:38,648 - lava_callback - ERROR - Error processing callback for node 69409127575bc060ce16079c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409127575bc060ce16079c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409127575bc060ce16079c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409127575bc060ce16079c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:38,650 - lava_callback - INFO - Starting async job submit for node 6940914a575bc060ce1607f5 2025-12-17 23:04:38,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:38,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409136575bc060ce1607be 2025-12-17 23:04:38,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409136575bc060ce1607be HTTP/1.1" 502 150 2025-12-17 23:04:38,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409136575bc060ce1607be'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:38,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940914a575bc060ce1607f5 HTTP/1.1" 502 150 2025-12-17 23:04:38,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940914a575bc060ce1607f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:38,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940914a575bc060ce1607f5 2025-12-17 23:04:38,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940914a575bc060ce1607f5 HTTP/1.1" 502 150 2025-12-17 23:04:38,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940914a575bc060ce1607f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:38,682 - lava_callback - INFO - Received callback for node 694093f4575bc060ce160edd INFO: 172.21.0.1:56518 - "POST /node/694093f4575bc060ce160edd HTTP/1.0" 202 Accepted 2025-12-17 23:04:38,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940913b575bc060ce1607c4 2025-12-17 23:04:38,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913b575bc060ce1607c4 HTTP/1.1" 502 150 2025-12-17 23:04:38,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940913b575bc060ce1607c4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:38,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409141575bc060ce1607e3 2025-12-17 23:04:38,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409141575bc060ce1607e3 HTTP/1.1" 502 150 2025-12-17 23:04:38,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409141575bc060ce1607e3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:39,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940913d575bc060ce1607cb 2025-12-17 23:04:39,341 - lava_callback - INFO - Received callback for node 694093f6575bc060ce160ee0 INFO: 172.21.0.1:56526 - "POST /node/694093f6575bc060ce160ee0 HTTP/1.0" 202 Accepted 2025-12-17 23:04:39,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409129575bc060ce1607a5 2025-12-17 23:04:39,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409129575bc060ce1607a5 HTTP/1.1" 502 150 2025-12-17 23:04:39,551 - lava_callback - ERROR - Error processing callback for node 69409129575bc060ce1607a5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409129575bc060ce1607a5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409129575bc060ce1607a5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409129575bc060ce1607a5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:39,552 - lava_callback - INFO - Starting async job submit for node 69409144575bc060ce1607e6 2025-12-17 23:04:39,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:39,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409144575bc060ce1607e6 HTTP/1.1" 502 150 2025-12-17 23:04:39,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409144575bc060ce1607e6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:39,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409144575bc060ce1607e6 2025-12-17 23:04:39,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409144575bc060ce1607e6 HTTP/1.1" 502 150 2025-12-17 23:04:39,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409144575bc060ce1607e6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:39,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940912c575bc060ce1607ac 2025-12-17 23:04:39,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912c575bc060ce1607ac HTTP/1.1" 502 150 2025-12-17 23:04:39,600 - lava_callback - ERROR - Error processing callback for node 6940912c575bc060ce1607ac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940912c575bc060ce1607ac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940912c575bc060ce1607ac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940912c575bc060ce1607ac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:39,603 - lava_callback - INFO - Starting async job submit for node 69409145575bc060ce1607e9 2025-12-17 23:04:39,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:39,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409145575bc060ce1607e9 HTTP/1.1" 502 150 2025-12-17 23:04:39,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409145575bc060ce1607e9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:39,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409145575bc060ce1607e9 2025-12-17 23:04:39,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409145575bc060ce1607e9 HTTP/1.1" 502 150 2025-12-17 23:04:39,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409145575bc060ce1607e9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:39,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940913f575bc060ce1607d2 2025-12-17 23:04:39,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913f575bc060ce1607d2 HTTP/1.1" 502 150 2025-12-17 23:04:39,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940913f575bc060ce1607d2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:40,027 - lava_callback - INFO - Received callback for node 694093f7575bc060ce160ee3 INFO: 172.21.0.1:54066 - "POST /node/694093f7575bc060ce160ee3 HTTP/1.0" 202 Accepted 2025-12-17 23:04:40,180 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d7575bc060ce1606d5 2025-12-17 23:04:40,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d7575bc060ce1606d5 HTTP/1.1" 502 150 2025-12-17 23:04:40,182 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694090d7575bc060ce1606d5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:40,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940914a575bc060ce1607f5 2025-12-17 23:04:40,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940914a575bc060ce1607f5 HTTP/1.1" 502 150 2025-12-17 23:04:40,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940914a575bc060ce1607f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:40,678 - lava_callback - INFO - Received callback for node 694093f8575bc060ce160ee6 INFO: 172.21.0.1:54068 - "POST /node/694093f8575bc060ce160ee6 HTTP/1.0" 202 Accepted 2025-12-17 23:04:41,355 - lava_callback - INFO - Received callback for node 694093fa575bc060ce160eef INFO: 172.21.0.1:54072 - "POST /node/694093fa575bc060ce160eef HTTP/1.0" 202 Accepted 2025-12-17 23:04:41,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409144575bc060ce1607e6 2025-12-17 23:04:41,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409144575bc060ce1607e6 HTTP/1.1" 502 150 2025-12-17 23:04:41,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409144575bc060ce1607e6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:41,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409145575bc060ce1607e9 2025-12-17 23:04:41,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409145575bc060ce1607e9 HTTP/1.1" 502 150 2025-12-17 23:04:41,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409145575bc060ce1607e9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:41,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409120575bc060ce16078c 2025-12-17 23:04:41,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409120575bc060ce16078c HTTP/1.1" 502 150 2025-12-17 23:04:41,753 - lava_callback - ERROR - Error processing callback for node 69409120575bc060ce16078c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409120575bc060ce16078c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409120575bc060ce16078c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409120575bc060ce16078c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:41,755 - lava_callback - INFO - Starting async job submit for node 69409147575bc060ce1607ec 2025-12-17 23:04:41,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:41,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409147575bc060ce1607ec HTTP/1.1" 502 150 2025-12-17 23:04:41,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409147575bc060ce1607ec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:41,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409147575bc060ce1607ec 2025-12-17 23:04:41,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409147575bc060ce1607ec HTTP/1.1" 502 150 2025-12-17 23:04:41,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409147575bc060ce1607ec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:42,023 - lava_callback - INFO - Received callback for node 694093fc575bc060ce160efa INFO: 172.21.0.1:54080 - "POST /node/694093fc575bc060ce160efa HTTP/1.0" 202 Accepted 2025-12-17 23:04:42,664 - lava_callback - INFO - Received callback for node 694093fe575bc060ce160eff INFO: 172.21.0.1:54090 - "POST /node/694093fe575bc060ce160eff HTTP/1.0" 202 Accepted 2025-12-17 23:04:42,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940912d575bc060ce1607af 2025-12-17 23:04:42,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940912d575bc060ce1607af HTTP/1.1" 502 150 2025-12-17 23:04:42,701 - lava_callback - ERROR - Error processing callback for node 6940912d575bc060ce1607af: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940912d575bc060ce1607af (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940912d575bc060ce1607af (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940912d575bc060ce1607af (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:42,703 - lava_callback - INFO - Starting async job submit for node 69409148575bc060ce1607ef 2025-12-17 23:04:42,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:42,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409148575bc060ce1607ef HTTP/1.1" 502 150 2025-12-17 23:04:42,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409148575bc060ce1607ef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:42,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409148575bc060ce1607ef 2025-12-17 23:04:42,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409148575bc060ce1607ef HTTP/1.1" 502 150 2025-12-17 23:04:42,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409148575bc060ce1607ef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:42,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409141575bc060ce1607e3 2025-12-17 23:04:42,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409141575bc060ce1607e3 HTTP/1.1" 502 150 2025-12-17 23:04:42,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409141575bc060ce1607e3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:43,298 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409130575bc060ce1607b5 2025-12-17 23:04:43,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409130575bc060ce1607b5 HTTP/1.1" 502 150 2025-12-17 23:04:43,299 - lava_callback - ERROR - Error processing callback for node 69409130575bc060ce1607b5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409130575bc060ce1607b5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409130575bc060ce1607b5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409130575bc060ce1607b5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:43,302 - lava_callback - INFO - Starting async job submit for node 69409149575bc060ce1607f2 2025-12-17 23:04:43,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:43,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409149575bc060ce1607f2 HTTP/1.1" 502 150 2025-12-17 23:04:43,310 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409149575bc060ce1607f2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:43,310 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409149575bc060ce1607f2 2025-12-17 23:04:43,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409149575bc060ce1607f2 HTTP/1.1" 502 150 2025-12-17 23:04:43,311 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409149575bc060ce1607f2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:43,332 - lava_callback - INFO - Received callback for node 69409400575bc060ce160f06 INFO: 172.21.0.1:54096 - "POST /node/69409400575bc060ce160f06 HTTP/1.0" 202 Accepted 2025-12-17 23:04:43,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409147575bc060ce1607ec 2025-12-17 23:04:43,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409147575bc060ce1607ec HTTP/1.1" 502 150 2025-12-17 23:04:43,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409147575bc060ce1607ec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:44,012 - lava_callback - INFO - Received callback for node 69409402575bc060ce160f09 INFO: 172.21.0.1:54100 - "POST /node/69409402575bc060ce160f09 HTTP/1.0" 202 Accepted 2025-12-17 23:04:44,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940914a575bc060ce1607f5 2025-12-17 23:04:44,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940914a575bc060ce1607f5 HTTP/1.1" 502 150 2025-12-17 23:04:44,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940914a575bc060ce1607f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:44,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409148575bc060ce1607ef 2025-12-17 23:04:44,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409148575bc060ce1607ef HTTP/1.1" 502 150 2025-12-17 23:04:44,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409148575bc060ce1607ef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:45,311 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409149575bc060ce1607f2 2025-12-17 23:04:45,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409149575bc060ce1607f2 HTTP/1.1" 502 150 2025-12-17 23:04:45,312 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409149575bc060ce1607f2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:45,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409144575bc060ce1607e6 2025-12-17 23:04:45,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409144575bc060ce1607e6 HTTP/1.1" 502 150 2025-12-17 23:04:45,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409144575bc060ce1607e6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:45,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409145575bc060ce1607e9 2025-12-17 23:04:45,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409145575bc060ce1607e9 HTTP/1.1" 502 150 2025-12-17 23:04:45,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409145575bc060ce1607e9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:46,206 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409132575bc060ce1607b8 2025-12-17 23:04:46,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409132575bc060ce1607b8 HTTP/1.1" 502 150 2025-12-17 23:04:46,207 - lava_callback - ERROR - Error processing callback for node 69409132575bc060ce1607b8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409132575bc060ce1607b8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409132575bc060ce1607b8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409132575bc060ce1607b8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:46,210 - lava_callback - INFO - Starting async job submit for node 69409153575bc060ce160807 2025-12-17 23:04:46,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:46,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409153575bc060ce160807 HTTP/1.1" 502 150 2025-12-17 23:04:46,223 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409153575bc060ce160807'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:46,223 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409153575bc060ce160807 2025-12-17 23:04:46,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409153575bc060ce160807 HTTP/1.1" 502 150 2025-12-17 23:04:46,224 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409153575bc060ce160807'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54108 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:04:47,137 - lava_callback - INFO - Received callback for node 69409404575bc060ce160f13 INFO: 172.21.0.1:54120 - "POST /node/69409404575bc060ce160f13 HTTP/1.0" 202 Accepted 2025-12-17 23:04:47,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409147575bc060ce1607ec 2025-12-17 23:04:47,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409147575bc060ce1607ec HTTP/1.1" 502 150 2025-12-17 23:04:47,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409147575bc060ce1607ec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:48,224 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409153575bc060ce160807 2025-12-17 23:04:48,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409153575bc060ce160807 HTTP/1.1" 502 150 2025-12-17 23:04:48,226 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409153575bc060ce160807'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:48,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409122575bc060ce16078f 2025-12-17 23:04:48,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409122575bc060ce16078f HTTP/1.1" 502 150 2025-12-17 23:04:48,500 - lava_callback - ERROR - Error processing callback for node 69409122575bc060ce16078f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409122575bc060ce16078f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409122575bc060ce16078f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409122575bc060ce16078f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:48,503 - lava_callback - INFO - Starting async job submit for node 694091a5575bc060ce1608e9 2025-12-17 23:04:48,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:48,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a5575bc060ce1608e9 HTTP/1.1" 502 150 2025-12-17 23:04:48,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a5575bc060ce1608e9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:48,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a5575bc060ce1608e9 2025-12-17 23:04:48,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a5575bc060ce1608e9 HTTP/1.1" 502 150 2025-12-17 23:04:48,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a5575bc060ce1608e9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:48,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409148575bc060ce1607ef 2025-12-17 23:04:48,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409148575bc060ce1607ef HTTP/1.1" 502 150 2025-12-17 23:04:48,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409148575bc060ce1607ef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:49,312 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409149575bc060ce1607f2 2025-12-17 23:04:49,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409149575bc060ce1607f2 HTTP/1.1" 502 150 2025-12-17 23:04:49,314 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409149575bc060ce1607f2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:49,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409134575bc060ce1607bb 2025-12-17 23:04:49,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409134575bc060ce1607bb HTTP/1.1" 502 150 2025-12-17 23:04:49,836 - lava_callback - ERROR - Error processing callback for node 69409134575bc060ce1607bb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409134575bc060ce1607bb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409134575bc060ce1607bb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409134575bc060ce1607bb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:49,838 - lava_callback - INFO - Starting async job submit for node 69409159575bc060ce16080e 2025-12-17 23:04:49,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:49,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409159575bc060ce16080e HTTP/1.1" 502 150 2025-12-17 23:04:49,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409159575bc060ce16080e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:49,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409159575bc060ce16080e 2025-12-17 23:04:49,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409159575bc060ce16080e HTTP/1.1" 502 150 2025-12-17 23:04:49,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409159575bc060ce16080e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:50,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a5575bc060ce1608e9 2025-12-17 23:04:50,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a5575bc060ce1608e9 HTTP/1.1" 502 150 2025-12-17 23:04:50,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a5575bc060ce1608e9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:50,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409141575bc060ce1607e3 2025-12-17 23:04:50,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409141575bc060ce1607e3 HTTP/1.1" 502 150 2025-12-17 23:04:50,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409141575bc060ce1607e3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:51,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409159575bc060ce16080e 2025-12-17 23:04:51,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409159575bc060ce16080e HTTP/1.1" 502 150 2025-12-17 23:04:51,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409159575bc060ce16080e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:52,226 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409153575bc060ce160807 2025-12-17 23:04:52,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409153575bc060ce160807 HTTP/1.1" 502 150 2025-12-17 23:04:52,227 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409153575bc060ce160807'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:52,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409126575bc060ce160799 2025-12-17 23:04:52,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409126575bc060ce160799 HTTP/1.1" 502 150 2025-12-17 23:04:52,595 - lava_callback - ERROR - Error processing callback for node 69409126575bc060ce160799: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409126575bc060ce160799 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409126575bc060ce160799 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409126575bc060ce160799 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:52,597 - lava_callback - INFO - Starting async job submit for node 6940915b575bc060ce160811 2025-12-17 23:04:52,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:52,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940915b575bc060ce160811 HTTP/1.1" 502 150 2025-12-17 23:04:52,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940915b575bc060ce160811'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:52,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940915b575bc060ce160811 2025-12-17 23:04:52,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940915b575bc060ce160811 HTTP/1.1" 502 150 2025-12-17 23:04:52,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940915b575bc060ce160811'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:52,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940914a575bc060ce1607f5 2025-12-17 23:04:52,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940914a575bc060ce1607f5 HTTP/1.1" 502 150 2025-12-17 23:04:52,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940914a575bc060ce1607f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:53,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409144575bc060ce1607e6 2025-12-17 23:04:53,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409144575bc060ce1607e6 HTTP/1.1" 502 150 2025-12-17 23:04:53,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409144575bc060ce1607e6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:53,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409145575bc060ce1607e9 2025-12-17 23:04:53,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409145575bc060ce1607e9 HTTP/1.1" 502 150 2025-12-17 23:04:53,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409145575bc060ce1607e9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:54,428 - lava_callback - INFO - Received callback for node 6940993f575bc060ce161c85 INFO: 172.21.0.1:53372 - "POST /node/6940993f575bc060ce161c85 HTTP/1.0" 202 Accepted 2025-12-17 23:04:54,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913d575bc060ce1607cb HTTP/1.1" 502 150 2025-12-17 23:04:54,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940913d575bc060ce1607cb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:54,476 - lava_callback - INFO - Received callback for node 6940992c575bc060ce161c57 INFO: 172.21.0.1:53386 - "POST /node/6940992c575bc060ce161c57 HTTP/1.0" 202 Accepted 2025-12-17 23:04:54,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940915b575bc060ce160811 2025-12-17 23:04:54,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940915b575bc060ce160811 HTTP/1.1" 502 150 2025-12-17 23:04:54,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940915b575bc060ce160811'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:54,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409136575bc060ce1607be 2025-12-17 23:04:54,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409136575bc060ce1607be HTTP/1.1" 502 150 2025-12-17 23:04:54,653 - lava_callback - ERROR - Error processing callback for node 69409136575bc060ce1607be: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409136575bc060ce1607be (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409136575bc060ce1607be (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409136575bc060ce1607be (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:54,654 - lava_callback - INFO - Starting async job submit for node 6940915d575bc060ce160814 2025-12-17 23:04:54,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:54,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940915d575bc060ce160814 HTTP/1.1" 502 150 2025-12-17 23:04:54,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940915d575bc060ce160814'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:54,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940915d575bc060ce160814 2025-12-17 23:04:54,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940915d575bc060ce160814 HTTP/1.1" 502 150 2025-12-17 23:04:54,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940915d575bc060ce160814'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:54,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a5575bc060ce1608e9 2025-12-17 23:04:54,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a5575bc060ce1608e9 HTTP/1.1" 502 150 2025-12-17 23:04:54,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a5575bc060ce1608e9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:54,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940913b575bc060ce1607c4 2025-12-17 23:04:54,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913b575bc060ce1607c4 HTTP/1.1" 502 150 2025-12-17 23:04:54,713 - lava_callback - ERROR - Error processing callback for node 6940913b575bc060ce1607c4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940913b575bc060ce1607c4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940913b575bc060ce1607c4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940913b575bc060ce1607c4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:54,715 - lava_callback - INFO - Starting async job submit for node 69409160575bc060ce160817 2025-12-17 23:04:54,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:54,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409160575bc060ce160817 HTTP/1.1" 502 150 2025-12-17 23:04:54,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409160575bc060ce160817'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:54,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409160575bc060ce160817 2025-12-17 23:04:54,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409160575bc060ce160817 HTTP/1.1" 502 150 2025-12-17 23:04:54,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409160575bc060ce160817'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:55,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409147575bc060ce1607ec 2025-12-17 23:04:55,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409147575bc060ce1607ec HTTP/1.1" 502 150 2025-12-17 23:04:55,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409147575bc060ce1607ec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:55,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940913f575bc060ce1607d2 2025-12-17 23:04:55,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913f575bc060ce1607d2 HTTP/1.1" 502 150 2025-12-17 23:04:55,805 - lava_callback - ERROR - Error processing callback for node 6940913f575bc060ce1607d2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940913f575bc060ce1607d2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940913f575bc060ce1607d2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940913f575bc060ce1607d2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:04:55,808 - lava_callback - INFO - Starting async job submit for node 69409162575bc060ce16081a 2025-12-17 23:04:55,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:04:55,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409162575bc060ce16081a HTTP/1.1" 502 150 2025-12-17 23:04:55,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409162575bc060ce16081a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:55,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409162575bc060ce16081a 2025-12-17 23:04:55,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409162575bc060ce16081a HTTP/1.1" 502 150 2025-12-17 23:04:55,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409162575bc060ce16081a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:55,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409159575bc060ce16080e 2025-12-17 23:04:55,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409159575bc060ce16080e HTTP/1.1" 502 150 2025-12-17 23:04:55,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409159575bc060ce16080e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:55,949 - lava_callback - INFO - Received callback for node 69409405575bc060ce160f16 INFO: 172.21.0.1:53390 - "POST /node/69409405575bc060ce160f16 HTTP/1.0" 202 Accepted 2025-12-17 23:04:56,182 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694090d7575bc060ce1606d5 INFO: 172.21.0.1:53406 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:04:56,597 - lava_callback - INFO - Received callback for node 69409407575bc060ce160f19 INFO: 172.21.0.1:53412 - "POST /node/69409407575bc060ce160f19 HTTP/1.0" 202 Accepted 2025-12-17 23:04:56,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940915d575bc060ce160814 2025-12-17 23:04:56,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940915d575bc060ce160814 HTTP/1.1" 502 150 2025-12-17 23:04:56,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940915d575bc060ce160814'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:56,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409148575bc060ce1607ef 2025-12-17 23:04:56,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409148575bc060ce1607ef HTTP/1.1" 502 150 2025-12-17 23:04:56,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409148575bc060ce1607ef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:56,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409160575bc060ce160817 2025-12-17 23:04:56,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409160575bc060ce160817 HTTP/1.1" 502 150 2025-12-17 23:04:56,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409160575bc060ce160817'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:57,276 - lava_callback - INFO - Received callback for node 69409409575bc060ce160f1c INFO: 172.21.0.1:53416 - "POST /node/69409409575bc060ce160f1c HTTP/1.0" 202 Accepted 2025-12-17 23:04:57,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409149575bc060ce1607f2 2025-12-17 23:04:57,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409149575bc060ce1607f2 HTTP/1.1" 502 150 2025-12-17 23:04:57,315 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409149575bc060ce1607f2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:57,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409162575bc060ce16081a 2025-12-17 23:04:57,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409162575bc060ce16081a HTTP/1.1" 502 150 2025-12-17 23:04:57,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409162575bc060ce16081a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:57,953 - lava_callback - INFO - Received callback for node 6940940c575bc060ce160f1f INFO: 172.21.0.1:53420 - "POST /node/6940940c575bc060ce160f1f HTTP/1.0" 202 Accepted 2025-12-17 23:04:58,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940915b575bc060ce160811 2025-12-17 23:04:58,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940915b575bc060ce160811 HTTP/1.1" 502 150 2025-12-17 23:04:58,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940915b575bc060ce160811'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:04:58,635 - lava_callback - INFO - Received callback for node 6940940d575bc060ce160f27 INFO: 172.21.0.1:53434 - "POST /node/6940940d575bc060ce160f27 HTTP/1.0" 202 Accepted 2025-12-17 23:04:59,280 - lava_callback - INFO - Received callback for node 6940940f575bc060ce160f31 INFO: 172.21.0.1:53438 - "POST /node/6940940f575bc060ce160f31 HTTP/1.0" 202 Accepted 2025-12-17 23:04:59,937 - lava_callback - INFO - Received callback for node 69409411575bc060ce160f39 INFO: 172.21.0.1:50494 - "POST /node/69409411575bc060ce160f39 HTTP/1.0" 202 Accepted 2025-12-17 23:05:00,227 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409153575bc060ce160807 2025-12-17 23:05:00,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409153575bc060ce160807 HTTP/1.1" 502 150 2025-12-17 23:05:00,229 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409153575bc060ce160807'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:00,604 - lava_callback - INFO - Received callback for node 69409413575bc060ce160f3c INFO: 172.21.0.1:50508 - "POST /node/69409413575bc060ce160f3c HTTP/1.0" 202 Accepted 2025-12-17 23:05:00,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940915d575bc060ce160814 2025-12-17 23:05:00,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940915d575bc060ce160814 HTTP/1.1" 502 150 2025-12-17 23:05:00,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940915d575bc060ce160814'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:00,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409160575bc060ce160817 2025-12-17 23:05:00,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409160575bc060ce160817 HTTP/1.1" 502 150 2025-12-17 23:05:00,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409160575bc060ce160817'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:01,267 - lava_callback - INFO - Received callback for node 69409416575bc060ce160f58 INFO: 172.21.0.1:50520 - "POST /node/69409416575bc060ce160f58 HTTP/1.0" 202 Accepted 2025-12-17 23:05:01,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409162575bc060ce16081a 2025-12-17 23:05:01,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409162575bc060ce16081a HTTP/1.1" 502 150 2025-12-17 23:05:01,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409162575bc060ce16081a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:02,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a5575bc060ce1608e9 2025-12-17 23:05:02,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a5575bc060ce1608e9 HTTP/1.1" 502 150 2025-12-17 23:05:02,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a5575bc060ce1608e9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50528 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:05:03,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409159575bc060ce16080e 2025-12-17 23:05:03,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409159575bc060ce16080e HTTP/1.1" 502 150 2025-12-17 23:05:03,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409159575bc060ce16080e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:03,955 - lava_callback - INFO - Received callback for node 6940955e575bc060ce1612e8 INFO: 172.21.0.1:50534 - "POST /node/6940955e575bc060ce1612e8 HTTP/1.0" 202 Accepted 2025-12-17 23:05:06,347 - lava_callback - INFO - Received callback for node 69409036575bc060ce16036c INFO: 172.21.0.1:50546 - "POST /node/69409036575bc060ce16036c HTTP/1.0" 202 Accepted 2025-12-17 23:05:06,440 - lava_callback - INFO - Received callback for node 6940941a575bc060ce160f63 INFO: 172.21.0.1:50548 - "POST /node/6940941a575bc060ce160f63 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:50550 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:05:06,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940915b575bc060ce160811 2025-12-17 23:05:06,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940915b575bc060ce160811 HTTP/1.1" 502 150 2025-12-17 23:05:06,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940915b575bc060ce160811'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:06,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409141575bc060ce1607e3 2025-12-17 23:05:06,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409141575bc060ce1607e3 HTTP/1.1" 502 150 2025-12-17 23:05:06,950 - lava_callback - ERROR - Error processing callback for node 69409141575bc060ce1607e3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409141575bc060ce1607e3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409141575bc060ce1607e3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409141575bc060ce1607e3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:06,952 - lava_callback - INFO - Starting async job submit for node 69409164575bc060ce16081d 2025-12-17 23:05:06,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:06,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409164575bc060ce16081d HTTP/1.1" 502 150 2025-12-17 23:05:06,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409164575bc060ce16081d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:06,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409164575bc060ce16081d 2025-12-17 23:05:06,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409164575bc060ce16081d HTTP/1.1" 502 150 2025-12-17 23:05:06,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409164575bc060ce16081d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:07,080 - lava_callback - INFO - Received callback for node 6940941c575bc060ce160f66 INFO: 172.21.0.1:50566 - "POST /node/6940941c575bc060ce160f66 HTTP/1.0" 202 Accepted 2025-12-17 23:05:07,716 - lava_callback - INFO - Received callback for node 6940941e575bc060ce160f69 INFO: 172.21.0.1:50568 - "POST /node/6940941e575bc060ce160f69 HTTP/1.0" 202 Accepted 2025-12-17 23:05:08,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940914a575bc060ce1607f5 2025-12-17 23:05:08,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940914a575bc060ce1607f5 HTTP/1.1" 502 150 2025-12-17 23:05:08,663 - lava_callback - ERROR - Error processing callback for node 6940914a575bc060ce1607f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940914a575bc060ce1607f5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940914a575bc060ce1607f5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940914a575bc060ce1607f5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:08,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940915d575bc060ce160814 2025-12-17 23:05:08,666 - lava_callback - INFO - Starting async job submit for node 69409166575bc060ce160828 2025-12-17 23:05:08,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940915d575bc060ce160814 HTTP/1.1" 502 150 2025-12-17 23:05:08,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:08,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940915d575bc060ce160814'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:08,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409166575bc060ce160828 HTTP/1.1" 502 150 2025-12-17 23:05:08,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409166575bc060ce160828'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:08,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409166575bc060ce160828 2025-12-17 23:05:08,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409166575bc060ce160828 HTTP/1.1" 502 150 2025-12-17 23:05:08,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409166575bc060ce160828'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:08,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409160575bc060ce160817 2025-12-17 23:05:08,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409160575bc060ce160817 HTTP/1.1" 502 150 2025-12-17 23:05:08,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409160575bc060ce160817'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:08,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409164575bc060ce16081d 2025-12-17 23:05:08,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409164575bc060ce16081d HTTP/1.1" 502 150 2025-12-17 23:05:08,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409164575bc060ce16081d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:09,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409144575bc060ce1607e6 2025-12-17 23:05:09,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409144575bc060ce1607e6 HTTP/1.1" 502 150 2025-12-17 23:05:09,575 - lava_callback - ERROR - Error processing callback for node 69409144575bc060ce1607e6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409144575bc060ce1607e6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409144575bc060ce1607e6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409144575bc060ce1607e6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:09,577 - lava_callback - INFO - Starting async job submit for node 69409168575bc060ce160835 2025-12-17 23:05:09,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:09,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409168575bc060ce160835 HTTP/1.1" 502 150 2025-12-17 23:05:09,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409168575bc060ce160835'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:09,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409168575bc060ce160835 2025-12-17 23:05:09,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409168575bc060ce160835 HTTP/1.1" 502 150 2025-12-17 23:05:09,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409168575bc060ce160835'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:09,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409145575bc060ce1607e9 2025-12-17 23:05:09,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409145575bc060ce1607e9 HTTP/1.1" 502 150 2025-12-17 23:05:09,619 - lava_callback - ERROR - Error processing callback for node 69409145575bc060ce1607e9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409145575bc060ce1607e9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409145575bc060ce1607e9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409145575bc060ce1607e9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:09,621 - lava_callback - INFO - Starting async job submit for node 6940916b575bc060ce16083c 2025-12-17 23:05:09,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:09,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916b575bc060ce16083c HTTP/1.1" 502 150 2025-12-17 23:05:09,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940916b575bc060ce16083c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:09,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940916b575bc060ce16083c 2025-12-17 23:05:09,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916b575bc060ce16083c HTTP/1.1" 502 150 2025-12-17 23:05:09,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940916b575bc060ce16083c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:09,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409162575bc060ce16081a 2025-12-17 23:05:09,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409162575bc060ce16081a HTTP/1.1" 502 150 2025-12-17 23:05:09,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409162575bc060ce16081a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:10,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940913d575bc060ce1607cb 2025-12-17 23:05:10,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940913d575bc060ce1607cb HTTP/1.1" 502 150 2025-12-17 23:05:10,454 - lava_callback - ERROR - Error processing callback for node 6940913d575bc060ce1607cb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940913d575bc060ce1607cb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940913d575bc060ce1607cb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940913d575bc060ce1607cb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:10,455 - lava_callback - INFO - Starting async job submit for node 6940916d575bc060ce16083f 2025-12-17 23:05:10,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:10,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916d575bc060ce16083f HTTP/1.1" 502 150 2025-12-17 23:05:10,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940916d575bc060ce16083f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:10,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940916d575bc060ce16083f 2025-12-17 23:05:10,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916d575bc060ce16083f HTTP/1.1" 502 150 2025-12-17 23:05:10,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940916d575bc060ce16083f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:10,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409166575bc060ce160828 2025-12-17 23:05:10,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409166575bc060ce160828 HTTP/1.1" 502 150 2025-12-17 23:05:10,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409166575bc060ce160828'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:11,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409168575bc060ce160835 2025-12-17 23:05:11,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409168575bc060ce160835 HTTP/1.1" 502 150 2025-12-17 23:05:11,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409168575bc060ce160835'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:11,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694090d7575bc060ce1606d5 HTTP/1.1" 502 150 2025-12-17 23:05:11,601 - lava_callback - ERROR - Error processing callback for node 694090d7575bc060ce1606d5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d7575bc060ce1606d5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d7575bc060ce1606d5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694090d7575bc060ce1606d5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:11,603 - lava_callback - INFO - Starting async job submit for node 6940916e575bc060ce160846 2025-12-17 23:05:11,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:11,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916e575bc060ce160846 HTTP/1.1" 502 150 2025-12-17 23:05:11,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940916e575bc060ce160846'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:11,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940916e575bc060ce160846 2025-12-17 23:05:11,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916e575bc060ce160846 HTTP/1.1" 502 150 2025-12-17 23:05:11,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940916e575bc060ce160846'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:11,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940916b575bc060ce16083c 2025-12-17 23:05:11,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916b575bc060ce16083c HTTP/1.1" 502 150 2025-12-17 23:05:11,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940916b575bc060ce16083c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:11,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409147575bc060ce1607ec 2025-12-17 23:05:11,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409147575bc060ce1607ec HTTP/1.1" 502 150 2025-12-17 23:05:11,771 - lava_callback - ERROR - Error processing callback for node 69409147575bc060ce1607ec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409147575bc060ce1607ec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409147575bc060ce1607ec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409147575bc060ce1607ec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:11,773 - lava_callback - INFO - Starting async job submit for node 69409170575bc060ce160849 2025-12-17 23:05:11,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:11,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409170575bc060ce160849 HTTP/1.1" 502 150 2025-12-17 23:05:11,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409170575bc060ce160849'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:11,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409170575bc060ce160849 2025-12-17 23:05:11,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409170575bc060ce160849 HTTP/1.1" 502 150 2025-12-17 23:05:11,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409170575bc060ce160849'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:12,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940916d575bc060ce16083f 2025-12-17 23:05:12,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409148575bc060ce1607ef 2025-12-17 23:05:12,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409148575bc060ce1607ef HTTP/1.1" 502 150 2025-12-17 23:05:12,727 - lava_callback - ERROR - Error processing callback for node 69409148575bc060ce1607ef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409148575bc060ce1607ef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409148575bc060ce1607ef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409148575bc060ce1607ef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:12,729 - lava_callback - INFO - Starting async job submit for node 69409172575bc060ce16084c 2025-12-17 23:05:12,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:12,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409172575bc060ce16084c HTTP/1.1" 502 150 2025-12-17 23:05:12,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409172575bc060ce16084c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:12,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409172575bc060ce16084c 2025-12-17 23:05:12,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409172575bc060ce16084c HTTP/1.1" 502 150 2025-12-17 23:05:12,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409172575bc060ce16084c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:12,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409164575bc060ce16081d 2025-12-17 23:05:12,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409164575bc060ce16081d HTTP/1.1" 502 150 2025-12-17 23:05:12,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409164575bc060ce16081d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:13,315 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409149575bc060ce1607f2 2025-12-17 23:05:13,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409149575bc060ce1607f2 HTTP/1.1" 502 150 2025-12-17 23:05:13,316 - lava_callback - ERROR - Error processing callback for node 69409149575bc060ce1607f2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409149575bc060ce1607f2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409149575bc060ce1607f2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409149575bc060ce1607f2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:13,318 - lava_callback - INFO - Starting async job submit for node 69409176575bc060ce160852 2025-12-17 23:05:13,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:13,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409176575bc060ce160852 HTTP/1.1" 502 150 2025-12-17 23:05:13,325 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409176575bc060ce160852'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:13,325 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409176575bc060ce160852 2025-12-17 23:05:13,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409176575bc060ce160852 HTTP/1.1" 502 150 2025-12-17 23:05:13,326 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409176575bc060ce160852'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:13,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940916e575bc060ce160846 2025-12-17 23:05:13,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916e575bc060ce160846 HTTP/1.1" 502 150 2025-12-17 23:05:13,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940916e575bc060ce160846'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:13,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409170575bc060ce160849 2025-12-17 23:05:13,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409170575bc060ce160849 HTTP/1.1" 502 150 2025-12-17 23:05:13,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409170575bc060ce160849'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:14,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409166575bc060ce160828 2025-12-17 23:05:14,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409166575bc060ce160828 HTTP/1.1" 502 150 2025-12-17 23:05:14,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409166575bc060ce160828'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:14,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409172575bc060ce16084c 2025-12-17 23:05:14,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409172575bc060ce16084c HTTP/1.1" 502 150 2025-12-17 23:05:14,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409172575bc060ce16084c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40464 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:05:15,326 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409176575bc060ce160852 2025-12-17 23:05:15,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409176575bc060ce160852 HTTP/1.1" 502 150 2025-12-17 23:05:15,327 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409176575bc060ce160852'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:15,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409168575bc060ce160835 2025-12-17 23:05:15,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409168575bc060ce160835 HTTP/1.1" 502 150 2025-12-17 23:05:15,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409168575bc060ce160835'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:15,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940916b575bc060ce16083c 2025-12-17 23:05:15,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916b575bc060ce16083c HTTP/1.1" 502 150 2025-12-17 23:05:15,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940916b575bc060ce16083c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:16,020 - lava_callback - INFO - Received callback for node 69409427575bc060ce160f85 INFO: 172.21.0.1:40472 - "POST /node/69409427575bc060ce160f85 HTTP/1.0" 202 Accepted 2025-12-17 23:05:16,229 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409153575bc060ce160807 INFO: 172.21.0.1:40486 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:05:16,682 - lava_callback - INFO - Received callback for node 6940942a575bc060ce160f8a INFO: 172.21.0.1:40490 - "POST /node/6940942a575bc060ce160f8a HTTP/1.0" 202 Accepted 2025-12-17 23:05:17,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940916e575bc060ce160846 2025-12-17 23:05:17,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916e575bc060ce160846 HTTP/1.1" 502 150 2025-12-17 23:05:17,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940916e575bc060ce160846'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:17,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409170575bc060ce160849 2025-12-17 23:05:17,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409170575bc060ce160849 HTTP/1.1" 502 150 2025-12-17 23:05:17,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409170575bc060ce160849'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:18,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a5575bc060ce1608e9 2025-12-17 23:05:18,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a5575bc060ce1608e9 HTTP/1.1" 502 150 2025-12-17 23:05:18,676 - lava_callback - ERROR - Error processing callback for node 694091a5575bc060ce1608e9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a5575bc060ce1608e9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a5575bc060ce1608e9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a5575bc060ce1608e9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:18,679 - lava_callback - INFO - Starting async job submit for node 69409178575bc060ce160857 2025-12-17 23:05:18,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:18,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409178575bc060ce160857 HTTP/1.1" 502 150 2025-12-17 23:05:18,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409178575bc060ce160857'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:18,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409178575bc060ce160857 2025-12-17 23:05:18,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409178575bc060ce160857 HTTP/1.1" 502 150 2025-12-17 23:05:18,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409178575bc060ce160857'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:18,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409172575bc060ce16084c 2025-12-17 23:05:18,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409172575bc060ce16084c HTTP/1.1" 502 150 2025-12-17 23:05:18,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409172575bc060ce16084c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:19,328 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409176575bc060ce160852 2025-12-17 23:05:19,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409176575bc060ce160852 HTTP/1.1" 502 150 2025-12-17 23:05:19,329 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409176575bc060ce160852'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:19,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409159575bc060ce16080e 2025-12-17 23:05:19,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409159575bc060ce16080e HTTP/1.1" 502 150 2025-12-17 23:05:19,861 - lava_callback - ERROR - Error processing callback for node 69409159575bc060ce16080e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409159575bc060ce16080e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409159575bc060ce16080e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409159575bc060ce16080e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:19,863 - lava_callback - INFO - Starting async job submit for node 69409334575bc060ce160c6a 2025-12-17 23:05:19,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:19,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409334575bc060ce160c6a HTTP/1.1" 502 150 2025-12-17 23:05:19,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409334575bc060ce160c6a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:19,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409334575bc060ce160c6a 2025-12-17 23:05:19,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409334575bc060ce160c6a HTTP/1.1" 502 150 2025-12-17 23:05:19,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409334575bc060ce160c6a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:20,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409178575bc060ce160857 2025-12-17 23:05:20,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409178575bc060ce160857 HTTP/1.1" 502 150 2025-12-17 23:05:20,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409178575bc060ce160857'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:20,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409164575bc060ce16081d 2025-12-17 23:05:20,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409164575bc060ce16081d HTTP/1.1" 502 150 2025-12-17 23:05:20,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409164575bc060ce16081d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:21,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409334575bc060ce160c6a 2025-12-17 23:05:21,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409334575bc060ce160c6a HTTP/1.1" 502 150 2025-12-17 23:05:21,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409334575bc060ce160c6a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:22,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940915b575bc060ce160811 2025-12-17 23:05:22,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940915b575bc060ce160811 HTTP/1.1" 502 150 2025-12-17 23:05:22,627 - lava_callback - ERROR - Error processing callback for node 6940915b575bc060ce160811: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940915b575bc060ce160811 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940915b575bc060ce160811 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940915b575bc060ce160811 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:22,628 - lava_callback - INFO - Starting async job submit for node 6940917a575bc060ce160860 2025-12-17 23:05:22,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:22,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917a575bc060ce160860 HTTP/1.1" 502 150 2025-12-17 23:05:22,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940917a575bc060ce160860'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:22,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940917a575bc060ce160860 2025-12-17 23:05:22,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917a575bc060ce160860 HTTP/1.1" 502 150 2025-12-17 23:05:22,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940917a575bc060ce160860'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:22,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409166575bc060ce160828 2025-12-17 23:05:22,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409166575bc060ce160828 HTTP/1.1" 502 150 2025-12-17 23:05:22,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409166575bc060ce160828'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:23,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409168575bc060ce160835 2025-12-17 23:05:23,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409168575bc060ce160835 HTTP/1.1" 502 150 2025-12-17 23:05:23,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409168575bc060ce160835'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:23,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940916b575bc060ce16083c 2025-12-17 23:05:23,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916b575bc060ce16083c HTTP/1.1" 502 150 2025-12-17 23:05:23,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940916b575bc060ce16083c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:24,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940917a575bc060ce160860 2025-12-17 23:05:24,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917a575bc060ce160860 HTTP/1.1" 502 150 2025-12-17 23:05:24,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940917a575bc060ce160860'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:24,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940915d575bc060ce160814 2025-12-17 23:05:24,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940915d575bc060ce160814 HTTP/1.1" 502 150 2025-12-17 23:05:24,669 - lava_callback - ERROR - Error processing callback for node 6940915d575bc060ce160814: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940915d575bc060ce160814 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940915d575bc060ce160814 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940915d575bc060ce160814 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:24,671 - lava_callback - INFO - Starting async job submit for node 6940917c575bc060ce160869 2025-12-17 23:05:24,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:24,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917c575bc060ce160869 HTTP/1.1" 502 150 2025-12-17 23:05:24,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940917c575bc060ce160869'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:24,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940917c575bc060ce160869 2025-12-17 23:05:24,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917c575bc060ce160869 HTTP/1.1" 502 150 2025-12-17 23:05:24,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940917c575bc060ce160869'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:24,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409178575bc060ce160857 2025-12-17 23:05:24,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409178575bc060ce160857 HTTP/1.1" 502 150 2025-12-17 23:05:24,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409178575bc060ce160857'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:24,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409160575bc060ce160817 2025-12-17 23:05:24,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409160575bc060ce160817 HTTP/1.1" 502 150 2025-12-17 23:05:24,730 - lava_callback - ERROR - Error processing callback for node 69409160575bc060ce160817: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409160575bc060ce160817 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409160575bc060ce160817 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409160575bc060ce160817 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:24,732 - lava_callback - INFO - Starting async job submit for node 6940917f575bc060ce160887 2025-12-17 23:05:24,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:24,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917f575bc060ce160887 HTTP/1.1" 502 150 2025-12-17 23:05:24,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940917f575bc060ce160887'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:24,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940917f575bc060ce160887 2025-12-17 23:05:24,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917f575bc060ce160887 HTTP/1.1" 502 150 2025-12-17 23:05:24,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940917f575bc060ce160887'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:25,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940916e575bc060ce160846 2025-12-17 23:05:25,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916e575bc060ce160846 HTTP/1.1" 502 150 2025-12-17 23:05:25,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940916e575bc060ce160846'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:25,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409170575bc060ce160849 2025-12-17 23:05:25,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409170575bc060ce160849 HTTP/1.1" 502 150 2025-12-17 23:05:25,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409170575bc060ce160849'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:25,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409162575bc060ce16081a 2025-12-17 23:05:25,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409162575bc060ce16081a HTTP/1.1" 502 150 2025-12-17 23:05:25,851 - lava_callback - ERROR - Error processing callback for node 69409162575bc060ce16081a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409162575bc060ce16081a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409162575bc060ce16081a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409162575bc060ce16081a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:25,853 - lava_callback - INFO - Starting async job submit for node 69409181575bc060ce16088a 2025-12-17 23:05:25,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:25,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409181575bc060ce16088a HTTP/1.1" 502 150 2025-12-17 23:05:25,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409181575bc060ce16088a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:25,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409181575bc060ce16088a 2025-12-17 23:05:25,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409181575bc060ce16088a HTTP/1.1" 502 150 2025-12-17 23:05:25,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409181575bc060ce16088a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:25,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409334575bc060ce160c6a 2025-12-17 23:05:25,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409334575bc060ce160c6a HTTP/1.1" 502 150 2025-12-17 23:05:25,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409334575bc060ce160c6a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:26,451 - lava_callback - INFO - Received callback for node 69409422575bc060ce160f77 INFO: 172.21.0.1:56854 - "POST /node/69409422575bc060ce160f77 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:56862 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:05:26,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940917c575bc060ce160869 2025-12-17 23:05:26,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917c575bc060ce160869 HTTP/1.1" 502 150 2025-12-17 23:05:26,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940917c575bc060ce160869'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:26,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940917f575bc060ce160887 2025-12-17 23:05:26,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917f575bc060ce160887 HTTP/1.1" 502 150 2025-12-17 23:05:26,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940917f575bc060ce160887'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:26,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409172575bc060ce16084c 2025-12-17 23:05:26,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409172575bc060ce16084c HTTP/1.1" 502 150 2025-12-17 23:05:26,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409172575bc060ce16084c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:27,121 - lava_callback - INFO - Received callback for node 69409424575bc060ce160f82 INFO: 172.21.0.1:56874 - "POST /node/69409424575bc060ce160f82 HTTP/1.0" 202 Accepted 2025-12-17 23:05:27,329 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409176575bc060ce160852 2025-12-17 23:05:27,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916d575bc060ce16083f HTTP/1.1" 502 150 2025-12-17 23:05:27,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940916d575bc060ce16083f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:27,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409181575bc060ce16088a 2025-12-17 23:05:27,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409181575bc060ce16088a HTTP/1.1" 502 150 2025-12-17 23:05:27,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409181575bc060ce16088a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:28,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940917a575bc060ce160860 2025-12-17 23:05:28,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917a575bc060ce160860 HTTP/1.1" 502 150 2025-12-17 23:05:28,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940917a575bc060ce160860'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:30,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940917c575bc060ce160869 2025-12-17 23:05:30,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917c575bc060ce160869 HTTP/1.1" 502 150 2025-12-17 23:05:30,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940917c575bc060ce160869'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:30,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940917f575bc060ce160887 2025-12-17 23:05:30,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917f575bc060ce160887 HTTP/1.1" 502 150 2025-12-17 23:05:30,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940917f575bc060ce160887'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:31,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409153575bc060ce160807 HTTP/1.1" 502 150 2025-12-17 23:05:31,568 - lava_callback - ERROR - Error processing callback for node 69409153575bc060ce160807: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409153575bc060ce160807 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409153575bc060ce160807 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409153575bc060ce160807 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:31,570 - lava_callback - INFO - Starting async job submit for node 69409183575bc060ce16088f 2025-12-17 23:05:31,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:31,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409183575bc060ce16088f HTTP/1.1" 502 150 2025-12-17 23:05:31,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409183575bc060ce16088f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:31,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409183575bc060ce16088f 2025-12-17 23:05:31,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409183575bc060ce16088f HTTP/1.1" 502 150 2025-12-17 23:05:31,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409183575bc060ce16088f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:31,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940916d575bc060ce16083f 2025-12-17 23:05:31,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916d575bc060ce16083f HTTP/1.1" 502 150 2025-12-17 23:05:31,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940916d575bc060ce16083f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:31,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409181575bc060ce16088a 2025-12-17 23:05:31,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409181575bc060ce16088a HTTP/1.1" 502 150 2025-12-17 23:05:31,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409181575bc060ce16088a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:32,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409178575bc060ce160857 2025-12-17 23:05:32,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409178575bc060ce160857 HTTP/1.1" 502 150 2025-12-17 23:05:32,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409178575bc060ce160857'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:33,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409183575bc060ce16088f 2025-12-17 23:05:33,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409183575bc060ce16088f HTTP/1.1" 502 150 2025-12-17 23:05:33,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409183575bc060ce16088f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:33,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409334575bc060ce160c6a 2025-12-17 23:05:33,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409334575bc060ce160c6a HTTP/1.1" 502 150 2025-12-17 23:05:33,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409334575bc060ce160c6a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:36,046 - lava_callback - INFO - Received callback for node 6940942c575bc060ce160f8d INFO: 172.21.0.1:52054 - "POST /node/6940942c575bc060ce160f8d HTTP/1.0" 202 Accepted INFO: 172.21.0.1:52062 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:05:36,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940917a575bc060ce160860 2025-12-17 23:05:36,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917a575bc060ce160860 HTTP/1.1" 502 150 2025-12-17 23:05:36,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940917a575bc060ce160860'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:36,731 - lava_callback - INFO - Received callback for node 6940942e575bc060ce160f90 INFO: 172.21.0.1:52074 - "POST /node/6940942e575bc060ce160f90 HTTP/1.0" 202 Accepted 2025-12-17 23:05:36,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409164575bc060ce16081d 2025-12-17 23:05:36,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409164575bc060ce16081d HTTP/1.1" 502 150 2025-12-17 23:05:36,971 - lava_callback - ERROR - Error processing callback for node 69409164575bc060ce16081d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409164575bc060ce16081d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409164575bc060ce16081d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409164575bc060ce16081d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:36,973 - lava_callback - INFO - Starting async job submit for node 69409185575bc060ce160897 2025-12-17 23:05:36,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:36,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409185575bc060ce160897 HTTP/1.1" 502 150 2025-12-17 23:05:36,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409185575bc060ce160897'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:36,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409185575bc060ce160897 2025-12-17 23:05:36,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409185575bc060ce160897 HTTP/1.1" 502 150 2025-12-17 23:05:36,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409185575bc060ce160897'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:37,390 - lava_callback - INFO - Received callback for node 69409430575bc060ce160f93 INFO: 172.21.0.1:52088 - "POST /node/69409430575bc060ce160f93 HTTP/1.0" 202 Accepted 2025-12-17 23:05:37,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409183575bc060ce16088f 2025-12-17 23:05:37,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409183575bc060ce16088f HTTP/1.1" 502 150 2025-12-17 23:05:37,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409183575bc060ce16088f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:38,082 - lava_callback - INFO - Received callback for node 69409432575bc060ce160f96 INFO: 172.21.0.1:52092 - "POST /node/69409432575bc060ce160f96 HTTP/1.0" 202 Accepted 2025-12-17 23:05:38,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409166575bc060ce160828 2025-12-17 23:05:38,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409166575bc060ce160828 HTTP/1.1" 502 150 2025-12-17 23:05:38,680 - lava_callback - ERROR - Error processing callback for node 69409166575bc060ce160828: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409166575bc060ce160828 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409166575bc060ce160828 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409166575bc060ce160828 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:38,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940917c575bc060ce160869 2025-12-17 23:05:38,682 - lava_callback - INFO - Starting async job submit for node 69409187575bc060ce16089a 2025-12-17 23:05:38,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:38,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917c575bc060ce160869 HTTP/1.1" 502 150 2025-12-17 23:05:38,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940917c575bc060ce160869'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:38,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409187575bc060ce16089a HTTP/1.1" 502 150 2025-12-17 23:05:38,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409187575bc060ce16089a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:38,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409187575bc060ce16089a 2025-12-17 23:05:38,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409187575bc060ce16089a HTTP/1.1" 502 150 2025-12-17 23:05:38,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409187575bc060ce16089a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:38,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940917f575bc060ce160887 2025-12-17 23:05:38,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917f575bc060ce160887 HTTP/1.1" 502 150 2025-12-17 23:05:38,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940917f575bc060ce160887'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:38,786 - lava_callback - INFO - Received callback for node 69409434575bc060ce160f99 INFO: 172.21.0.1:52096 - "POST /node/69409434575bc060ce160f99 HTTP/1.0" 202 Accepted 2025-12-17 23:05:38,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409185575bc060ce160897 2025-12-17 23:05:39,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409185575bc060ce160897 HTTP/1.1" 502 150 2025-12-17 23:05:39,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409185575bc060ce160897'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:39,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409168575bc060ce160835 2025-12-17 23:05:39,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409168575bc060ce160835 HTTP/1.1" 502 150 2025-12-17 23:05:39,599 - lava_callback - ERROR - Error processing callback for node 69409168575bc060ce160835: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409168575bc060ce160835 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409168575bc060ce160835 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409168575bc060ce160835 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:39,601 - lava_callback - INFO - Starting async job submit for node 69409189575bc060ce16089d 2025-12-17 23:05:39,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:39,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409189575bc060ce16089d HTTP/1.1" 502 150 2025-12-17 23:05:39,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409189575bc060ce16089d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:39,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409189575bc060ce16089d 2025-12-17 23:05:39,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409189575bc060ce16089d HTTP/1.1" 502 150 2025-12-17 23:05:39,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409189575bc060ce16089d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:39,634 - lava_callback - INFO - Received callback for node 69409436575bc060ce160f9e INFO: 172.21.0.1:52108 - "POST /node/69409436575bc060ce160f9e HTTP/1.0" 202 Accepted 2025-12-17 23:05:39,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940916b575bc060ce16083c 2025-12-17 23:05:39,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916b575bc060ce16083c HTTP/1.1" 502 150 2025-12-17 23:05:39,645 - lava_callback - ERROR - Error processing callback for node 6940916b575bc060ce16083c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940916b575bc060ce16083c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940916b575bc060ce16083c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940916b575bc060ce16083c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:39,646 - lava_callback - INFO - Starting async job submit for node 6940918f575bc060ce1608b8 2025-12-17 23:05:39,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:39,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918f575bc060ce1608b8 HTTP/1.1" 502 150 2025-12-17 23:05:39,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940918f575bc060ce1608b8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:39,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940918f575bc060ce1608b8 2025-12-17 23:05:39,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918f575bc060ce1608b8 HTTP/1.1" 502 150 2025-12-17 23:05:39,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940918f575bc060ce1608b8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:39,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940916d575bc060ce16083f 2025-12-17 23:05:39,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916d575bc060ce16083f HTTP/1.1" 502 150 2025-12-17 23:05:39,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940916d575bc060ce16083f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:39,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409181575bc060ce16088a 2025-12-17 23:05:39,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409181575bc060ce16088a HTTP/1.1" 502 150 2025-12-17 23:05:39,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409181575bc060ce16088a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:40,292 - lava_callback - INFO - Received callback for node 69409438575bc060ce160fa7 INFO: 172.21.0.1:41132 - "POST /node/69409438575bc060ce160fa7 HTTP/1.0" 202 Accepted 2025-12-17 23:05:40,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409187575bc060ce16089a 2025-12-17 23:05:40,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409187575bc060ce16089a HTTP/1.1" 502 150 2025-12-17 23:05:40,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409187575bc060ce16089a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:40,953 - lava_callback - INFO - Received callback for node 69409439575bc060ce160fae INFO: 172.21.0.1:41146 - "POST /node/69409439575bc060ce160fae HTTP/1.0" 202 Accepted 2025-12-17 23:05:41,604 - lava_callback - INFO - Received callback for node 6940943b575bc060ce160fb3 INFO: 172.21.0.1:41150 - "POST /node/6940943b575bc060ce160fb3 HTTP/1.0" 202 Accepted 2025-12-17 23:05:41,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409189575bc060ce16089d 2025-12-17 23:05:41,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409189575bc060ce16089d HTTP/1.1" 502 150 2025-12-17 23:05:41,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409189575bc060ce16089d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:41,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940916e575bc060ce160846 2025-12-17 23:05:41,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916e575bc060ce160846 HTTP/1.1" 502 150 2025-12-17 23:05:41,616 - lava_callback - ERROR - Error processing callback for node 6940916e575bc060ce160846: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940916e575bc060ce160846 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940916e575bc060ce160846 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940916e575bc060ce160846 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:41,618 - lava_callback - INFO - Starting async job submit for node 6940918b575bc060ce1608a0 2025-12-17 23:05:41,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:41,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918b575bc060ce1608a0 HTTP/1.1" 502 150 2025-12-17 23:05:41,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940918b575bc060ce1608a0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:41,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940918b575bc060ce1608a0 2025-12-17 23:05:41,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918b575bc060ce1608a0 HTTP/1.1" 502 150 2025-12-17 23:05:41,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940918b575bc060ce1608a0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:41,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940918f575bc060ce1608b8 2025-12-17 23:05:41,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918f575bc060ce1608b8 HTTP/1.1" 502 150 2025-12-17 23:05:41,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940918f575bc060ce1608b8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:41,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409170575bc060ce160849 2025-12-17 23:05:41,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409170575bc060ce160849 HTTP/1.1" 502 150 2025-12-17 23:05:41,787 - lava_callback - ERROR - Error processing callback for node 69409170575bc060ce160849: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409170575bc060ce160849 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409170575bc060ce160849 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409170575bc060ce160849 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:41,789 - lava_callback - INFO - Starting async job submit for node 6940918d575bc060ce1608a9 2025-12-17 23:05:41,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:41,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918d575bc060ce1608a9 HTTP/1.1" 502 150 2025-12-17 23:05:41,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940918d575bc060ce1608a9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:41,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940918d575bc060ce1608a9 2025-12-17 23:05:41,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918d575bc060ce1608a9 HTTP/1.1" 502 150 2025-12-17 23:05:41,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940918d575bc060ce1608a9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:42,290 - lava_callback - INFO - Received callback for node 6940943d575bc060ce160fb6 INFO: 172.21.0.1:41160 - "POST /node/6940943d575bc060ce160fb6 HTTP/1.0" 202 Accepted 2025-12-17 23:05:42,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409176575bc060ce160852 HTTP/1.1" 502 150 2025-12-17 23:05:42,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409176575bc060ce160852'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:42,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409172575bc060ce16084c 2025-12-17 23:05:42,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409172575bc060ce16084c HTTP/1.1" 502 150 2025-12-17 23:05:42,751 - lava_callback - ERROR - Error processing callback for node 69409172575bc060ce16084c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409172575bc060ce16084c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409172575bc060ce16084c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409172575bc060ce16084c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:42,753 - lava_callback - INFO - Starting async job submit for node 69409197575bc060ce1608c8 2025-12-17 23:05:42,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:42,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409197575bc060ce1608c8 HTTP/1.1" 502 150 2025-12-17 23:05:42,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409197575bc060ce1608c8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:42,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409197575bc060ce1608c8 2025-12-17 23:05:42,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409197575bc060ce1608c8 HTTP/1.1" 502 150 2025-12-17 23:05:42,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409197575bc060ce1608c8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:43,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409185575bc060ce160897 2025-12-17 23:05:43,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409185575bc060ce160897 HTTP/1.1" 502 150 2025-12-17 23:05:43,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409185575bc060ce160897'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:43,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940918b575bc060ce1608a0 2025-12-17 23:05:43,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918b575bc060ce1608a0 HTTP/1.1" 502 150 2025-12-17 23:05:43,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940918b575bc060ce1608a0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:43,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940918d575bc060ce1608a9 2025-12-17 23:05:43,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918d575bc060ce1608a9 HTTP/1.1" 502 150 2025-12-17 23:05:43,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940918d575bc060ce1608a9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:44,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409187575bc060ce16089a 2025-12-17 23:05:44,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409187575bc060ce16089a HTTP/1.1" 502 150 2025-12-17 23:05:44,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409187575bc060ce16089a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:44,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409197575bc060ce1608c8 2025-12-17 23:05:44,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409197575bc060ce1608c8 HTTP/1.1" 502 150 2025-12-17 23:05:44,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409197575bc060ce1608c8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:45,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409183575bc060ce16088f 2025-12-17 23:05:45,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409183575bc060ce16088f HTTP/1.1" 502 150 2025-12-17 23:05:45,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409183575bc060ce16088f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:45,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409189575bc060ce16089d 2025-12-17 23:05:45,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409189575bc060ce16089d HTTP/1.1" 502 150 2025-12-17 23:05:45,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409189575bc060ce16089d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:45,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940918f575bc060ce1608b8 2025-12-17 23:05:45,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918f575bc060ce1608b8 HTTP/1.1" 502 150 2025-12-17 23:05:45,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940918f575bc060ce1608b8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41176 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:05:47,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940918b575bc060ce1608a0 2025-12-17 23:05:47,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918b575bc060ce1608a0 HTTP/1.1" 502 150 2025-12-17 23:05:47,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940918b575bc060ce1608a0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:47,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940918d575bc060ce1608a9 2025-12-17 23:05:47,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918d575bc060ce1608a9 HTTP/1.1" 502 150 2025-12-17 23:05:47,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940918d575bc060ce1608a9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:47,829 - lava_callback - INFO - Received callback for node 694187d5ecc59770562e6d7d INFO: 172.21.0.1:41188 - "POST /node/694187d5ecc59770562e6d7d HTTP/1.0" 202 Accepted 2025-12-17 23:05:48,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409178575bc060ce160857 2025-12-17 23:05:48,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409178575bc060ce160857 HTTP/1.1" 502 150 2025-12-17 23:05:48,699 - lava_callback - ERROR - Error processing callback for node 69409178575bc060ce160857: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409178575bc060ce160857 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409178575bc060ce160857 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409178575bc060ce160857 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:48,702 - lava_callback - INFO - Starting async job submit for node 69409190575bc060ce1608bd 2025-12-17 23:05:48,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:48,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409190575bc060ce1608bd HTTP/1.1" 502 150 2025-12-17 23:05:48,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409190575bc060ce1608bd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:48,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409190575bc060ce1608bd 2025-12-17 23:05:48,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409190575bc060ce1608bd HTTP/1.1" 502 150 2025-12-17 23:05:48,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409190575bc060ce1608bd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:48,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409197575bc060ce1608c8 2025-12-17 23:05:48,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409197575bc060ce1608c8 HTTP/1.1" 502 150 2025-12-17 23:05:48,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409197575bc060ce1608c8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:49,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409334575bc060ce160c6a 2025-12-17 23:05:49,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409334575bc060ce160c6a HTTP/1.1" 502 150 2025-12-17 23:05:49,910 - lava_callback - ERROR - Error processing callback for node 69409334575bc060ce160c6a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409334575bc060ce160c6a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409334575bc060ce160c6a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409334575bc060ce160c6a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:49,912 - lava_callback - INFO - Starting async job submit for node 69409192575bc060ce1608c0 2025-12-17 23:05:49,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:49,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409192575bc060ce1608c0 HTTP/1.1" 502 150 2025-12-17 23:05:49,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409192575bc060ce1608c0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:49,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409192575bc060ce1608c0 2025-12-17 23:05:49,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409192575bc060ce1608c0 HTTP/1.1" 502 150 2025-12-17 23:05:49,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409192575bc060ce1608c0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:50,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409190575bc060ce1608bd 2025-12-17 23:05:50,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409190575bc060ce1608bd HTTP/1.1" 502 150 2025-12-17 23:05:50,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409190575bc060ce1608bd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:51,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409185575bc060ce160897 2025-12-17 23:05:51,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409185575bc060ce160897 HTTP/1.1" 502 150 2025-12-17 23:05:51,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409185575bc060ce160897'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:51,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409192575bc060ce1608c0 2025-12-17 23:05:51,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409192575bc060ce1608c0 HTTP/1.1" 502 150 2025-12-17 23:05:51,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409192575bc060ce1608c0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:52,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940917a575bc060ce160860 2025-12-17 23:05:52,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917a575bc060ce160860 HTTP/1.1" 502 150 2025-12-17 23:05:52,641 - lava_callback - ERROR - Error processing callback for node 6940917a575bc060ce160860: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940917a575bc060ce160860 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940917a575bc060ce160860 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940917a575bc060ce160860 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:52,643 - lava_callback - INFO - Starting async job submit for node 69409194575bc060ce1608c3 2025-12-17 23:05:52,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:52,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409194575bc060ce1608c3 HTTP/1.1" 502 150 2025-12-17 23:05:52,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409194575bc060ce1608c3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:52,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409194575bc060ce1608c3 2025-12-17 23:05:52,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409194575bc060ce1608c3 HTTP/1.1" 502 150 2025-12-17 23:05:52,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409194575bc060ce1608c3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:52,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409187575bc060ce16089a 2025-12-17 23:05:52,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409187575bc060ce16089a HTTP/1.1" 502 150 2025-12-17 23:05:52,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409187575bc060ce16089a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:53,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409189575bc060ce16089d 2025-12-17 23:05:53,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940918f575bc060ce1608b8 2025-12-17 23:05:53,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918f575bc060ce1608b8 HTTP/1.1" 502 150 2025-12-17 23:05:53,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940918f575bc060ce1608b8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:53,867 - lava_callback - INFO - Received callback for node 6941170f459ff5fc64d293c2 INFO: 172.21.0.1:60110 - "POST /node/6941170f459ff5fc64d293c2 HTTP/1.0" 202 Accepted 2025-12-17 23:05:54,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409194575bc060ce1608c3 2025-12-17 23:05:54,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409194575bc060ce1608c3 HTTP/1.1" 502 150 2025-12-17 23:05:54,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409194575bc060ce1608c3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:54,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940917c575bc060ce160869 2025-12-17 23:05:54,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917c575bc060ce160869 HTTP/1.1" 502 150 2025-12-17 23:05:54,685 - lava_callback - ERROR - Error processing callback for node 6940917c575bc060ce160869: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940917c575bc060ce160869 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940917c575bc060ce160869 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940917c575bc060ce160869 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:54,688 - lava_callback - INFO - Starting async job submit for node 69409198575bc060ce1608cb 2025-12-17 23:05:54,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:54,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409198575bc060ce1608cb HTTP/1.1" 502 150 2025-12-17 23:05:54,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409198575bc060ce1608cb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:54,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409198575bc060ce1608cb 2025-12-17 23:05:54,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409198575bc060ce1608cb HTTP/1.1" 502 150 2025-12-17 23:05:54,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409198575bc060ce1608cb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:54,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409190575bc060ce1608bd 2025-12-17 23:05:54,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409190575bc060ce1608bd HTTP/1.1" 502 150 2025-12-17 23:05:54,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409190575bc060ce1608bd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:54,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940917f575bc060ce160887 2025-12-17 23:05:54,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940917f575bc060ce160887 HTTP/1.1" 502 150 2025-12-17 23:05:54,744 - lava_callback - ERROR - Error processing callback for node 6940917f575bc060ce160887: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940917f575bc060ce160887 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940917f575bc060ce160887 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940917f575bc060ce160887 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:54,745 - lava_callback - INFO - Starting async job submit for node 6940919a575bc060ce1608ce 2025-12-17 23:05:54,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:54,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919a575bc060ce1608ce HTTP/1.1" 502 150 2025-12-17 23:05:54,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940919a575bc060ce1608ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:54,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940919a575bc060ce1608ce 2025-12-17 23:05:54,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919a575bc060ce1608ce HTTP/1.1" 502 150 2025-12-17 23:05:54,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940919a575bc060ce1608ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:55,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940918b575bc060ce1608a0 2025-12-17 23:05:55,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918b575bc060ce1608a0 HTTP/1.1" 502 150 2025-12-17 23:05:55,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940918b575bc060ce1608a0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:55,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940916d575bc060ce16083f 2025-12-17 23:05:55,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940916d575bc060ce16083f HTTP/1.1" 502 150 2025-12-17 23:05:55,733 - lava_callback - ERROR - Error processing callback for node 6940916d575bc060ce16083f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940916d575bc060ce16083f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940916d575bc060ce16083f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940916d575bc060ce16083f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:55,736 - lava_callback - INFO - Starting async job submit for node 6940919c575bc060ce1608d1 2025-12-17 23:05:55,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:55,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919c575bc060ce1608d1 HTTP/1.1" 502 150 2025-12-17 23:05:55,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940919c575bc060ce1608d1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:55,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940919c575bc060ce1608d1 2025-12-17 23:05:55,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919c575bc060ce1608d1 HTTP/1.1" 502 150 2025-12-17 23:05:55,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940919c575bc060ce1608d1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:55,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940918d575bc060ce1608a9 2025-12-17 23:05:55,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918d575bc060ce1608a9 HTTP/1.1" 502 150 2025-12-17 23:05:55,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940918d575bc060ce1608a9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:55,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409181575bc060ce16088a 2025-12-17 23:05:55,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409181575bc060ce16088a HTTP/1.1" 502 150 2025-12-17 23:05:55,865 - lava_callback - ERROR - Error processing callback for node 69409181575bc060ce16088a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409181575bc060ce16088a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409181575bc060ce16088a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409181575bc060ce16088a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:55,867 - lava_callback - INFO - Starting async job submit for node 694091a0575bc060ce1608d9 2025-12-17 23:05:55,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:55,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a0575bc060ce1608d9 HTTP/1.1" 502 150 2025-12-17 23:05:55,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a0575bc060ce1608d9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:55,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a0575bc060ce1608d9 2025-12-17 23:05:55,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a0575bc060ce1608d9 HTTP/1.1" 502 150 2025-12-17 23:05:55,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a0575bc060ce1608d9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:55,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409192575bc060ce1608c0 2025-12-17 23:05:55,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409192575bc060ce1608c0 HTTP/1.1" 502 150 2025-12-17 23:05:55,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409192575bc060ce1608c0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:56,017 - lava_callback - INFO - Received callback for node 6940943f575bc060ce160fb9 INFO: 172.21.0.1:60124 - "POST /node/6940943f575bc060ce160fb9 HTTP/1.0" 202 Accepted 2025-12-17 23:05:56,100 - lava_callback - INFO - Received callback for node 69409565575bc060ce161303 INFO: 172.21.0.1:60140 - "POST /node/69409565575bc060ce161303 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:60150 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:05:56,674 - lava_callback - INFO - Received callback for node 69409441575bc060ce160fbc INFO: 172.21.0.1:60166 - "POST /node/69409441575bc060ce160fbc HTTP/1.0" 202 Accepted 2025-12-17 23:05:56,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409198575bc060ce1608cb 2025-12-17 23:05:56,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409198575bc060ce1608cb HTTP/1.1" 502 150 2025-12-17 23:05:56,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409198575bc060ce1608cb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:56,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940919a575bc060ce1608ce 2025-12-17 23:05:56,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919a575bc060ce1608ce HTTP/1.1" 502 150 2025-12-17 23:05:56,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940919a575bc060ce1608ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:56,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409197575bc060ce1608c8 2025-12-17 23:05:56,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409197575bc060ce1608c8 HTTP/1.1" 502 150 2025-12-17 23:05:56,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409197575bc060ce1608c8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:57,314 - lava_callback - INFO - Received callback for node 69409442575bc060ce160fbf INFO: 172.21.0.1:60182 - "POST /node/69409442575bc060ce160fbf HTTP/1.0" 202 Accepted 2025-12-17 23:05:57,656 - lava_callback - INFO - Received callback for node 69409638575bc060ce161543 INFO: 172.21.0.1:60186 - "POST /node/69409638575bc060ce161543 HTTP/1.0" 202 Accepted 2025-12-17 23:05:57,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940919c575bc060ce1608d1 2025-12-17 23:05:57,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919c575bc060ce1608d1 HTTP/1.1" 502 150 2025-12-17 23:05:57,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940919c575bc060ce1608d1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:57,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a0575bc060ce1608d9 2025-12-17 23:05:57,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a0575bc060ce1608d9 HTTP/1.1" 502 150 2025-12-17 23:05:57,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a0575bc060ce1608d9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:57,976 - lava_callback - INFO - Received callback for node 69409444575bc060ce160fc2 INFO: 172.21.0.1:60200 - "POST /node/69409444575bc060ce160fc2 HTTP/1.0" 202 Accepted 2025-12-17 23:05:58,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409176575bc060ce160852 2025-12-17 23:05:58,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409176575bc060ce160852 HTTP/1.1" 502 150 2025-12-17 23:05:58,578 - lava_callback - ERROR - Error processing callback for node 69409176575bc060ce160852: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409176575bc060ce160852 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409176575bc060ce160852 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409176575bc060ce160852 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:05:58,579 - lava_callback - INFO - Starting async job submit for node 694091a8575bc060ce1608f7 2025-12-17 23:05:58,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:05:58,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a8575bc060ce1608f7 HTTP/1.1" 502 150 2025-12-17 23:05:58,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a8575bc060ce1608f7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:58,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a8575bc060ce1608f7 2025-12-17 23:05:58,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a8575bc060ce1608f7 HTTP/1.1" 502 150 2025-12-17 23:05:58,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a8575bc060ce1608f7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:58,639 - lava_callback - INFO - Received callback for node 69409445575bc060ce160fc5 INFO: 172.21.0.1:60216 - "POST /node/69409445575bc060ce160fc5 HTTP/1.0" 202 Accepted 2025-12-17 23:05:58,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409194575bc060ce1608c3 2025-12-17 23:05:58,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409194575bc060ce1608c3 HTTP/1.1" 502 150 2025-12-17 23:05:58,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409194575bc060ce1608c3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:05:59,310 - lava_callback - INFO - Received callback for node 69409447575bc060ce160fc8 INFO: 172.21.0.1:60218 - "POST /node/69409447575bc060ce160fc8 HTTP/1.0" 202 Accepted 2025-12-17 23:05:59,991 - lava_callback - INFO - Received callback for node 69409449575bc060ce160fcb INFO: 172.21.0.1:60708 - "POST /node/69409449575bc060ce160fcb HTTP/1.0" 202 Accepted 2025-12-17 23:06:00,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a8575bc060ce1608f7 2025-12-17 23:06:00,649 - lava_callback - INFO - Received callback for node 6940944a575bc060ce160fd2 INFO: 172.21.0.1:60724 - "POST /node/6940944a575bc060ce160fd2 HTTP/1.0" 202 Accepted 2025-12-17 23:06:00,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409198575bc060ce1608cb 2025-12-17 23:06:00,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409198575bc060ce1608cb HTTP/1.1" 502 150 2025-12-17 23:06:00,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409198575bc060ce1608cb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:00,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940919a575bc060ce1608ce 2025-12-17 23:06:00,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919a575bc060ce1608ce HTTP/1.1" 502 150 2025-12-17 23:06:00,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940919a575bc060ce1608ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:01,328 - lava_callback - INFO - Received callback for node 6940944c575bc060ce160fd9 INFO: 172.21.0.1:60736 - "POST /node/6940944c575bc060ce160fd9 HTTP/1.0" 202 Accepted 2025-12-17 23:06:01,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409183575bc060ce16088f 2025-12-17 23:06:01,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409183575bc060ce16088f HTTP/1.1" 502 150 2025-12-17 23:06:01,611 - lava_callback - ERROR - Error processing callback for node 69409183575bc060ce16088f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409183575bc060ce16088f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409183575bc060ce16088f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409183575bc060ce16088f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:01,612 - lava_callback - INFO - Starting async job submit for node 694091ac575bc060ce1608fd 2025-12-17 23:06:01,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:01,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ac575bc060ce1608fd HTTP/1.1" 502 150 2025-12-17 23:06:01,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ac575bc060ce1608fd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:01,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ac575bc060ce1608fd 2025-12-17 23:06:01,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ac575bc060ce1608fd HTTP/1.1" 502 150 2025-12-17 23:06:01,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ac575bc060ce1608fd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:01,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940919c575bc060ce1608d1 2025-12-17 23:06:01,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919c575bc060ce1608d1 HTTP/1.1" 502 150 2025-12-17 23:06:01,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940919c575bc060ce1608d1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:01,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a0575bc060ce1608d9 2025-12-17 23:06:01,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a0575bc060ce1608d9 HTTP/1.1" 502 150 2025-12-17 23:06:01,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a0575bc060ce1608d9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:01,985 - lava_callback - INFO - Received callback for node 6940944e575bc060ce160fe4 INFO: 172.21.0.1:60740 - "POST /node/6940944e575bc060ce160fe4 HTTP/1.0" 202 Accepted 2025-12-17 23:06:02,627 - lava_callback - INFO - Received callback for node 69409450575bc060ce160fe9 INFO: 172.21.0.1:60742 - "POST /node/69409450575bc060ce160fe9 HTTP/1.0" 202 Accepted 2025-12-17 23:06:02,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409190575bc060ce1608bd 2025-12-17 23:06:02,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409190575bc060ce1608bd HTTP/1.1" 502 150 2025-12-17 23:06:02,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409190575bc060ce1608bd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60748 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:06:03,282 - lava_callback - INFO - Received callback for node 69409452575bc060ce160fed INFO: 172.21.0.1:60754 - "POST /node/69409452575bc060ce160fed HTTP/1.0" 202 Accepted 2025-12-17 23:06:03,402 - lava_callback - INFO - Received callback for node 694191c6ecc59770562e821a INFO: 172.21.0.1:60756 - "POST /node/694191c6ecc59770562e821a HTTP/1.0" 202 Accepted 2025-12-17 23:06:03,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ac575bc060ce1608fd 2025-12-17 23:06:03,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ac575bc060ce1608fd HTTP/1.1" 502 150 2025-12-17 23:06:03,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ac575bc060ce1608fd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:03,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409192575bc060ce1608c0 2025-12-17 23:06:03,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409192575bc060ce1608c0 HTTP/1.1" 502 150 2025-12-17 23:06:03,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409192575bc060ce1608c0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60758 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:06:06,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409194575bc060ce1608c3 2025-12-17 23:06:06,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409194575bc060ce1608c3 HTTP/1.1" 502 150 2025-12-17 23:06:06,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409194575bc060ce1608c3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:07,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409185575bc060ce160897 2025-12-17 23:06:07,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409185575bc060ce160897 HTTP/1.1" 502 150 2025-12-17 23:06:07,008 - lava_callback - ERROR - Error processing callback for node 69409185575bc060ce160897: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409185575bc060ce160897 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409185575bc060ce160897 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409185575bc060ce160897 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:07,010 - lava_callback - INFO - Starting async job submit for node 694091a4575bc060ce1608e6 2025-12-17 23:06:07,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:07,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a4575bc060ce1608e6 HTTP/1.1" 502 150 2025-12-17 23:06:07,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a4575bc060ce1608e6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:07,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a4575bc060ce1608e6 2025-12-17 23:06:07,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a4575bc060ce1608e6 HTTP/1.1" 502 150 2025-12-17 23:06:07,047 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a4575bc060ce1608e6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:07,523 - lava_callback - INFO - Received callback for node 69409454575bc060ce160ff0 INFO: 172.21.0.1:60774 - "POST /node/69409454575bc060ce160ff0 HTTP/1.0" 202 Accepted 2025-12-17 23:06:07,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ac575bc060ce1608fd 2025-12-17 23:06:07,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ac575bc060ce1608fd HTTP/1.1" 502 150 2025-12-17 23:06:07,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ac575bc060ce1608fd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:08,437 - lava_callback - INFO - Received callback for node 69409457575bc060ce160ff6 INFO: 172.21.0.1:60786 - "POST /node/69409457575bc060ce160ff6 HTTP/1.0" 202 Accepted 2025-12-17 23:06:08,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409187575bc060ce16089a 2025-12-17 23:06:08,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409187575bc060ce16089a HTTP/1.1" 502 150 2025-12-17 23:06:08,697 - lava_callback - ERROR - Error processing callback for node 69409187575bc060ce16089a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409187575bc060ce16089a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409187575bc060ce16089a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409187575bc060ce16089a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:08,699 - lava_callback - INFO - Starting async job submit for node 694091af575bc060ce160900 2025-12-17 23:06:08,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:08,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409198575bc060ce1608cb 2025-12-17 23:06:08,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409198575bc060ce1608cb HTTP/1.1" 502 150 2025-12-17 23:06:08,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409198575bc060ce1608cb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:08,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091af575bc060ce160900 HTTP/1.1" 502 150 2025-12-17 23:06:08,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091af575bc060ce160900'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:08,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091af575bc060ce160900 2025-12-17 23:06:08,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091af575bc060ce160900 HTTP/1.1" 502 150 2025-12-17 23:06:08,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091af575bc060ce160900'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:08,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940919a575bc060ce1608ce 2025-12-17 23:06:08,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919a575bc060ce1608ce HTTP/1.1" 502 150 2025-12-17 23:06:08,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940919a575bc060ce1608ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:08,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409189575bc060ce16089d HTTP/1.1" 502 150 2025-12-17 23:06:08,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409189575bc060ce16089d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:09,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a4575bc060ce1608e6 2025-12-17 23:06:09,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a4575bc060ce1608e6 HTTP/1.1" 502 150 2025-12-17 23:06:09,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a4575bc060ce1608e6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:09,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940918f575bc060ce1608b8 2025-12-17 23:06:09,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918f575bc060ce1608b8 HTTP/1.1" 502 150 2025-12-17 23:06:09,681 - lava_callback - ERROR - Error processing callback for node 6940918f575bc060ce1608b8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940918f575bc060ce1608b8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940918f575bc060ce1608b8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940918f575bc060ce1608b8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:09,682 - lava_callback - INFO - Starting async job submit for node 694091b0575bc060ce160903 2025-12-17 23:06:09,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:09,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b0575bc060ce160903 HTTP/1.1" 502 150 2025-12-17 23:06:09,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b0575bc060ce160903'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:09,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b0575bc060ce160903 2025-12-17 23:06:09,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b0575bc060ce160903 HTTP/1.1" 502 150 2025-12-17 23:06:09,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b0575bc060ce160903'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:09,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940919c575bc060ce1608d1 2025-12-17 23:06:09,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919c575bc060ce1608d1 HTTP/1.1" 502 150 2025-12-17 23:06:09,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940919c575bc060ce1608d1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:09,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a0575bc060ce1608d9 2025-12-17 23:06:09,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a0575bc060ce1608d9 HTTP/1.1" 502 150 2025-12-17 23:06:09,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a0575bc060ce1608d9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:10,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091af575bc060ce160900 2025-12-17 23:06:10,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091af575bc060ce160900 HTTP/1.1" 502 150 2025-12-17 23:06:10,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091af575bc060ce160900'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:11,270 - lava_callback - INFO - Received callback for node 69409460575bc060ce161015 INFO: 172.21.0.1:46296 - "POST /node/69409460575bc060ce161015 HTTP/1.0" 202 Accepted 2025-12-17 23:06:11,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940918b575bc060ce1608a0 2025-12-17 23:06:11,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918b575bc060ce1608a0 HTTP/1.1" 502 150 2025-12-17 23:06:11,631 - lava_callback - ERROR - Error processing callback for node 6940918b575bc060ce1608a0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940918b575bc060ce1608a0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940918b575bc060ce1608a0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940918b575bc060ce1608a0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:11,632 - lava_callback - INFO - Starting async job submit for node 694091b2575bc060ce160906 2025-12-17 23:06:11,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:11,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b2575bc060ce160906 HTTP/1.1" 502 150 2025-12-17 23:06:11,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b2575bc060ce160906'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:11,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b2575bc060ce160906 2025-12-17 23:06:11,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b2575bc060ce160906 HTTP/1.1" 502 150 2025-12-17 23:06:11,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b2575bc060ce160906'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:11,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b0575bc060ce160903 2025-12-17 23:06:11,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b0575bc060ce160903 HTTP/1.1" 502 150 2025-12-17 23:06:11,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b0575bc060ce160903'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:11,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940918d575bc060ce1608a9 2025-12-17 23:06:11,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940918d575bc060ce1608a9 HTTP/1.1" 502 150 2025-12-17 23:06:11,802 - lava_callback - ERROR - Error processing callback for node 6940918d575bc060ce1608a9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940918d575bc060ce1608a9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940918d575bc060ce1608a9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940918d575bc060ce1608a9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:11,804 - lava_callback - INFO - Starting async job submit for node 69410c49460bc8e265117571 2025-12-17 23:06:11,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:11,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69410c49460bc8e265117571 HTTP/1.1" 502 150 2025-12-17 23:06:11,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69410c49460bc8e265117571'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:11,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69410c49460bc8e265117571 2025-12-17 23:06:11,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69410c49460bc8e265117571 HTTP/1.1" 502 150 2025-12-17 23:06:11,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69410c49460bc8e265117571'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:12,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409197575bc060ce1608c8 2025-12-17 23:06:12,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409197575bc060ce1608c8 HTTP/1.1" 502 150 2025-12-17 23:06:12,772 - lava_callback - ERROR - Error processing callback for node 69409197575bc060ce1608c8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409197575bc060ce1608c8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409197575bc060ce1608c8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409197575bc060ce1608c8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:12,774 - lava_callback - INFO - Starting async job submit for node 694091b4575bc060ce16090b 2025-12-17 23:06:12,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:12,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b4575bc060ce16090b HTTP/1.1" 502 150 2025-12-17 23:06:12,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b4575bc060ce16090b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:12,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b4575bc060ce16090b 2025-12-17 23:06:12,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b4575bc060ce16090b HTTP/1.1" 502 150 2025-12-17 23:06:12,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b4575bc060ce16090b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:13,049 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a4575bc060ce1608e6 2025-12-17 23:06:13,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a4575bc060ce1608e6 HTTP/1.1" 502 150 2025-12-17 23:06:13,050 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a4575bc060ce1608e6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:13,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b2575bc060ce160906 2025-12-17 23:06:13,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b2575bc060ce160906 HTTP/1.1" 502 150 2025-12-17 23:06:13,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b2575bc060ce160906'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:13,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69410c49460bc8e265117571 2025-12-17 23:06:13,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69410c49460bc8e265117571 HTTP/1.1" 502 150 2025-12-17 23:06:13,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69410c49460bc8e265117571'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:14,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091af575bc060ce160900 2025-12-17 23:06:14,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091af575bc060ce160900 HTTP/1.1" 502 150 2025-12-17 23:06:14,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091af575bc060ce160900'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:14,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b4575bc060ce16090b 2025-12-17 23:06:14,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b4575bc060ce16090b HTTP/1.1" 502 150 2025-12-17 23:06:14,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b4575bc060ce16090b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46310 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:06:15,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ac575bc060ce1608fd 2025-12-17 23:06:15,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b0575bc060ce160903 2025-12-17 23:06:15,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b0575bc060ce160903 HTTP/1.1" 502 150 2025-12-17 23:06:15,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b0575bc060ce160903'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:15,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a8575bc060ce1608f7 HTTP/1.1" 502 150 2025-12-17 23:06:15,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a8575bc060ce1608f7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:16,120 - lava_callback - INFO - Received callback for node 69409455575bc060ce160ff3 INFO: 172.21.0.1:46324 - "POST /node/69409455575bc060ce160ff3 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:46336 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:06:16,772 - lava_callback - INFO - Received callback for node 69409457575bc060ce160ff9 INFO: 172.21.0.1:46344 - "POST /node/69409457575bc060ce160ff9 HTTP/1.0" 202 Accepted 2025-12-17 23:06:17,455 - lava_callback - INFO - Received callback for node 69409464575bc060ce161024 INFO: 172.21.0.1:46354 - "POST /node/69409464575bc060ce161024 HTTP/1.0" 202 Accepted 2025-12-17 23:06:17,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b2575bc060ce160906 2025-12-17 23:06:17,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b2575bc060ce160906 HTTP/1.1" 502 150 2025-12-17 23:06:17,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b2575bc060ce160906'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:17,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69410c49460bc8e265117571 2025-12-17 23:06:17,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69410c49460bc8e265117571 HTTP/1.1" 502 150 2025-12-17 23:06:17,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69410c49460bc8e265117571'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:18,150 - lava_callback - INFO - Received callback for node 69409466575bc060ce161027 INFO: 172.21.0.1:46366 - "POST /node/69409466575bc060ce161027 HTTP/1.0" 202 Accepted 2025-12-17 23:06:18,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409190575bc060ce1608bd 2025-12-17 23:06:18,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409190575bc060ce1608bd HTTP/1.1" 502 150 2025-12-17 23:06:18,716 - lava_callback - ERROR - Error processing callback for node 69409190575bc060ce1608bd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409190575bc060ce1608bd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409190575bc060ce1608bd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409190575bc060ce1608bd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:18,719 - lava_callback - INFO - Starting async job submit for node 694091aa575bc060ce1608fa 2025-12-17 23:06:18,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:18,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091aa575bc060ce1608fa HTTP/1.1" 502 150 2025-12-17 23:06:18,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091aa575bc060ce1608fa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:18,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091aa575bc060ce1608fa 2025-12-17 23:06:18,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091aa575bc060ce1608fa HTTP/1.1" 502 150 2025-12-17 23:06:18,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091aa575bc060ce1608fa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:18,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b4575bc060ce16090b 2025-12-17 23:06:18,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b4575bc060ce16090b HTTP/1.1" 502 150 2025-12-17 23:06:18,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b4575bc060ce16090b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:19,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a8575bc060ce1608f7 2025-12-17 23:06:19,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a8575bc060ce1608f7 HTTP/1.1" 502 150 2025-12-17 23:06:19,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a8575bc060ce1608f7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:19,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409192575bc060ce1608c0 2025-12-17 23:06:19,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409192575bc060ce1608c0 HTTP/1.1" 502 150 2025-12-17 23:06:19,928 - lava_callback - ERROR - Error processing callback for node 69409192575bc060ce1608c0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409192575bc060ce1608c0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409192575bc060ce1608c0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409192575bc060ce1608c0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:19,929 - lava_callback - INFO - Starting async job submit for node 694091b7575bc060ce160916 2025-12-17 23:06:19,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:19,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b7575bc060ce160916 HTTP/1.1" 502 150 2025-12-17 23:06:19,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b7575bc060ce160916'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:19,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b7575bc060ce160916 2025-12-17 23:06:19,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b7575bc060ce160916 HTTP/1.1" 502 150 2025-12-17 23:06:19,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b7575bc060ce160916'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:20,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091aa575bc060ce1608fa 2025-12-17 23:06:20,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091aa575bc060ce1608fa HTTP/1.1" 502 150 2025-12-17 23:06:20,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091aa575bc060ce1608fa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:21,050 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a4575bc060ce1608e6 2025-12-17 23:06:21,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a4575bc060ce1608e6 HTTP/1.1" 502 150 2025-12-17 23:06:21,052 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a4575bc060ce1608e6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:21,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b7575bc060ce160916 2025-12-17 23:06:21,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b7575bc060ce160916 HTTP/1.1" 502 150 2025-12-17 23:06:21,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b7575bc060ce160916'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:22,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409194575bc060ce1608c3 2025-12-17 23:06:22,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409194575bc060ce1608c3 HTTP/1.1" 502 150 2025-12-17 23:06:22,656 - lava_callback - ERROR - Error processing callback for node 69409194575bc060ce1608c3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409194575bc060ce1608c3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409194575bc060ce1608c3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409194575bc060ce1608c3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:22,658 - lava_callback - INFO - Starting async job submit for node 694091bb575bc060ce16092e 2025-12-17 23:06:22,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:22,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bb575bc060ce16092e HTTP/1.1" 502 150 2025-12-17 23:06:22,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091bb575bc060ce16092e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:22,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091bb575bc060ce16092e 2025-12-17 23:06:22,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bb575bc060ce16092e HTTP/1.1" 502 150 2025-12-17 23:06:22,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091bb575bc060ce16092e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:22,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091af575bc060ce160900 2025-12-17 23:06:22,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091af575bc060ce160900 HTTP/1.1" 502 150 2025-12-17 23:06:22,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091af575bc060ce160900'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:23,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b0575bc060ce160903 2025-12-17 23:06:23,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b0575bc060ce160903 HTTP/1.1" 502 150 2025-12-17 23:06:23,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b0575bc060ce160903'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:24,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091bb575bc060ce16092e 2025-12-17 23:06:24,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bb575bc060ce16092e HTTP/1.1" 502 150 2025-12-17 23:06:24,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091bb575bc060ce16092e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:24,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409198575bc060ce1608cb 2025-12-17 23:06:24,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409198575bc060ce1608cb HTTP/1.1" 502 150 2025-12-17 23:06:24,705 - lava_callback - ERROR - Error processing callback for node 69409198575bc060ce1608cb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409198575bc060ce1608cb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409198575bc060ce1608cb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409198575bc060ce1608cb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:24,706 - lava_callback - INFO - Starting async job submit for node 694091bd575bc060ce160931 2025-12-17 23:06:24,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:24,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bd575bc060ce160931 HTTP/1.1" 502 150 2025-12-17 23:06:24,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091bd575bc060ce160931'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:24,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091bd575bc060ce160931 2025-12-17 23:06:24,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bd575bc060ce160931 HTTP/1.1" 502 150 2025-12-17 23:06:24,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091bd575bc060ce160931'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:24,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091aa575bc060ce1608fa 2025-12-17 23:06:24,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091aa575bc060ce1608fa HTTP/1.1" 502 150 2025-12-17 23:06:24,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091aa575bc060ce1608fa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:24,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940919a575bc060ce1608ce 2025-12-17 23:06:24,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919a575bc060ce1608ce HTTP/1.1" 502 150 2025-12-17 23:06:24,765 - lava_callback - ERROR - Error processing callback for node 6940919a575bc060ce1608ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940919a575bc060ce1608ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940919a575bc060ce1608ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940919a575bc060ce1608ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:24,766 - lava_callback - INFO - Starting async job submit for node 694091bf575bc060ce160934 2025-12-17 23:06:24,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:24,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bf575bc060ce160934 HTTP/1.1" 502 150 2025-12-17 23:06:24,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091bf575bc060ce160934'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:24,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091bf575bc060ce160934 2025-12-17 23:06:24,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bf575bc060ce160934 HTTP/1.1" 502 150 2025-12-17 23:06:24,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091bf575bc060ce160934'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:24,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409189575bc060ce16089d 2025-12-17 23:06:24,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409189575bc060ce16089d HTTP/1.1" 502 150 2025-12-17 23:06:24,946 - lava_callback - ERROR - Error processing callback for node 69409189575bc060ce16089d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409189575bc060ce16089d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409189575bc060ce16089d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409189575bc060ce16089d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:24,947 - lava_callback - INFO - Starting async job submit for node 694091c1575bc060ce160937 2025-12-17 23:06:24,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:24,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c1575bc060ce160937 HTTP/1.1" 502 150 2025-12-17 23:06:24,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c1575bc060ce160937'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:24,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c1575bc060ce160937 2025-12-17 23:06:24,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c1575bc060ce160937 HTTP/1.1" 502 150 2025-12-17 23:06:24,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c1575bc060ce160937'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:25,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b2575bc060ce160906 2025-12-17 23:06:25,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b2575bc060ce160906 HTTP/1.1" 502 150 2025-12-17 23:06:25,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b2575bc060ce160906'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:25,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940919c575bc060ce1608d1 2025-12-17 23:06:25,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919c575bc060ce1608d1 HTTP/1.1" 502 150 2025-12-17 23:06:25,754 - lava_callback - ERROR - Error processing callback for node 6940919c575bc060ce1608d1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940919c575bc060ce1608d1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940919c575bc060ce1608d1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940919c575bc060ce1608d1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:25,756 - lava_callback - INFO - Starting async job submit for node 694091c3575bc060ce16093a 2025-12-17 23:06:25,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:25,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c3575bc060ce16093a HTTP/1.1" 502 150 2025-12-17 23:06:25,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c3575bc060ce16093a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:25,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c3575bc060ce16093a 2025-12-17 23:06:25,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c3575bc060ce16093a HTTP/1.1" 502 150 2025-12-17 23:06:25,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c3575bc060ce16093a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:25,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69410c49460bc8e265117571 2025-12-17 23:06:25,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69410c49460bc8e265117571 HTTP/1.1" 502 150 2025-12-17 23:06:25,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69410c49460bc8e265117571'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:25,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a0575bc060ce1608d9 2025-12-17 23:06:25,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a0575bc060ce1608d9 HTTP/1.1" 502 150 2025-12-17 23:06:25,902 - lava_callback - ERROR - Error processing callback for node 694091a0575bc060ce1608d9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a0575bc060ce1608d9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a0575bc060ce1608d9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a0575bc060ce1608d9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:25,904 - lava_callback - INFO - Starting async job submit for node 694091c5575bc060ce16093d 2025-12-17 23:06:25,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:25,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c5575bc060ce16093d HTTP/1.1" 502 150 2025-12-17 23:06:25,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c5575bc060ce16093d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:25,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c5575bc060ce16093d 2025-12-17 23:06:25,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c5575bc060ce16093d HTTP/1.1" 502 150 2025-12-17 23:06:25,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c5575bc060ce16093d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:25,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b7575bc060ce160916 2025-12-17 23:06:25,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b7575bc060ce160916 HTTP/1.1" 502 150 2025-12-17 23:06:25,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b7575bc060ce160916'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:26,481 - lava_callback - INFO - Received callback for node 69409461575bc060ce16101c INFO: 172.21.0.1:43756 - "POST /node/69409461575bc060ce16101c HTTP/1.0" 202 Accepted INFO: 172.21.0.1:43772 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:06:26,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091bd575bc060ce160931 2025-12-17 23:06:26,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bd575bc060ce160931 HTTP/1.1" 502 150 2025-12-17 23:06:26,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091bd575bc060ce160931'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:26,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091bf575bc060ce160934 2025-12-17 23:06:26,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bf575bc060ce160934 HTTP/1.1" 502 150 2025-12-17 23:06:26,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091bf575bc060ce160934'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:26,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b4575bc060ce16090b 2025-12-17 23:06:26,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b4575bc060ce16090b HTTP/1.1" 502 150 2025-12-17 23:06:26,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b4575bc060ce16090b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:26,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c1575bc060ce160937 2025-12-17 23:06:26,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c1575bc060ce160937 HTTP/1.1" 502 150 2025-12-17 23:06:26,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c1575bc060ce160937'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:27,145 - lava_callback - INFO - Received callback for node 69409463575bc060ce161021 INFO: 172.21.0.1:43784 - "POST /node/69409463575bc060ce161021 HTTP/1.0" 202 Accepted 2025-12-17 23:06:27,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c3575bc060ce16093a 2025-12-17 23:06:27,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c3575bc060ce16093a HTTP/1.1" 502 150 2025-12-17 23:06:27,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c3575bc060ce16093a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:27,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a8575bc060ce1608f7 2025-12-17 23:06:27,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a8575bc060ce1608f7 HTTP/1.1" 502 150 2025-12-17 23:06:27,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091a8575bc060ce1608f7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:27,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c5575bc060ce16093d 2025-12-17 23:06:27,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c5575bc060ce16093d HTTP/1.1" 502 150 2025-12-17 23:06:27,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c5575bc060ce16093d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:28,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091bb575bc060ce16092e 2025-12-17 23:06:28,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bb575bc060ce16092e HTTP/1.1" 502 150 2025-12-17 23:06:28,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091bb575bc060ce16092e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:30,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091bd575bc060ce160931 2025-12-17 23:06:30,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091bf575bc060ce160934 2025-12-17 23:06:30,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bf575bc060ce160934 HTTP/1.1" 502 150 2025-12-17 23:06:30,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091bf575bc060ce160934'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:30,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c1575bc060ce160937 2025-12-17 23:06:30,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c1575bc060ce160937 HTTP/1.1" 502 150 2025-12-17 23:06:30,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c1575bc060ce160937'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:30,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ac575bc060ce1608fd HTTP/1.1" 502 150 2025-12-17 23:06:30,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ac575bc060ce1608fd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:31,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c3575bc060ce16093a 2025-12-17 23:06:31,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c3575bc060ce16093a HTTP/1.1" 502 150 2025-12-17 23:06:31,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c3575bc060ce16093a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:31,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c5575bc060ce16093d 2025-12-17 23:06:31,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c5575bc060ce16093d HTTP/1.1" 502 150 2025-12-17 23:06:31,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c5575bc060ce16093d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:32,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091aa575bc060ce1608fa 2025-12-17 23:06:32,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091aa575bc060ce1608fa HTTP/1.1" 502 150 2025-12-17 23:06:32,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091aa575bc060ce1608fa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:33,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b7575bc060ce160916 2025-12-17 23:06:33,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b7575bc060ce160916 HTTP/1.1" 502 150 2025-12-17 23:06:33,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091b7575bc060ce160916'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:35,580 - lava_callback - INFO - Received callback for node 694096b9575bc060ce16167a INFO: 172.21.0.1:58392 - "POST /node/694096b9575bc060ce16167a HTTP/1.0" 202 Accepted 2025-12-17 23:06:36,051 - lava_callback - INFO - Received callback for node 69409469575bc060ce16102e INFO: 172.21.0.1:58400 - "POST /node/69409469575bc060ce16102e HTTP/1.0" 202 Accepted INFO: 172.21.0.1:58410 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:06:36,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091bb575bc060ce16092e 2025-12-17 23:06:36,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bb575bc060ce16092e HTTP/1.1" 502 150 2025-12-17 23:06:36,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091bb575bc060ce16092e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:36,720 - lava_callback - INFO - Received callback for node 6940946b575bc060ce161031 INFO: 172.21.0.1:58414 - "POST /node/6940946b575bc060ce161031 HTTP/1.0" 202 Accepted 2025-12-17 23:06:37,052 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a4575bc060ce1608e6 2025-12-17 23:06:37,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a4575bc060ce1608e6 HTTP/1.1" 502 150 2025-12-17 23:06:37,053 - lava_callback - ERROR - Error processing callback for node 694091a4575bc060ce1608e6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a4575bc060ce1608e6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a4575bc060ce1608e6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a4575bc060ce1608e6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:37,055 - lava_callback - INFO - Starting async job submit for node 694091c7575bc060ce160940 2025-12-17 23:06:37,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:37,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c7575bc060ce160940 HTTP/1.1" 502 150 2025-12-17 23:06:37,086 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c7575bc060ce160940'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:37,087 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c7575bc060ce160940 2025-12-17 23:06:37,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c7575bc060ce160940 HTTP/1.1" 502 150 2025-12-17 23:06:37,087 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c7575bc060ce160940'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:37,379 - lava_callback - INFO - Received callback for node 6940946d575bc060ce161039 INFO: 172.21.0.1:58430 - "POST /node/6940946d575bc060ce161039 HTTP/1.0" 202 Accepted 2025-12-17 23:06:38,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091af575bc060ce160900 2025-12-17 23:06:38,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091af575bc060ce160900 HTTP/1.1" 502 150 2025-12-17 23:06:38,714 - lava_callback - ERROR - Error processing callback for node 694091af575bc060ce160900: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091af575bc060ce160900 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091af575bc060ce160900 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091af575bc060ce160900 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:38,716 - lava_callback - INFO - Starting async job submit for node 694091cc575bc060ce16094c 2025-12-17 23:06:38,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:38,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091cc575bc060ce16094c HTTP/1.1" 502 150 2025-12-17 23:06:38,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091cc575bc060ce16094c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:38,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091cc575bc060ce16094c 2025-12-17 23:06:38,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091cc575bc060ce16094c HTTP/1.1" 502 150 2025-12-17 23:06:38,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091cc575bc060ce16094c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:38,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091bf575bc060ce160934 2025-12-17 23:06:38,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bf575bc060ce160934 HTTP/1.1" 502 150 2025-12-17 23:06:38,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091bf575bc060ce160934'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:38,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c1575bc060ce160937 2025-12-17 23:06:38,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c1575bc060ce160937 HTTP/1.1" 502 150 2025-12-17 23:06:38,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c1575bc060ce160937'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:39,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c7575bc060ce160940 2025-12-17 23:06:39,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c7575bc060ce160940 HTTP/1.1" 502 150 2025-12-17 23:06:39,089 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c7575bc060ce160940'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:39,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b0575bc060ce160903 2025-12-17 23:06:39,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b0575bc060ce160903 HTTP/1.1" 502 150 2025-12-17 23:06:39,696 - lava_callback - ERROR - Error processing callback for node 694091b0575bc060ce160903: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091b0575bc060ce160903 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091b0575bc060ce160903 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091b0575bc060ce160903 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:39,697 - lava_callback - INFO - Starting async job submit for node 694091ca575bc060ce160949 2025-12-17 23:06:39,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:39,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ca575bc060ce160949 HTTP/1.1" 502 150 2025-12-17 23:06:39,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ca575bc060ce160949'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:39,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ca575bc060ce160949 2025-12-17 23:06:39,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ca575bc060ce160949 HTTP/1.1" 502 150 2025-12-17 23:06:39,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ca575bc060ce160949'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:39,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c3575bc060ce16093a 2025-12-17 23:06:39,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c3575bc060ce16093a HTTP/1.1" 502 150 2025-12-17 23:06:39,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c3575bc060ce16093a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:39,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c5575bc060ce16093d 2025-12-17 23:06:39,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c5575bc060ce16093d HTTP/1.1" 502 150 2025-12-17 23:06:39,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c5575bc060ce16093d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:40,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091cc575bc060ce16094c 2025-12-17 23:06:40,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091cc575bc060ce16094c HTTP/1.1" 502 150 2025-12-17 23:06:40,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091cc575bc060ce16094c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:41,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b2575bc060ce160906 2025-12-17 23:06:41,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ca575bc060ce160949 2025-12-17 23:06:41,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ca575bc060ce160949 HTTP/1.1" 502 150 2025-12-17 23:06:41,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ca575bc060ce160949'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:41,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69410c49460bc8e265117571 2025-12-17 23:06:41,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69410c49460bc8e265117571 HTTP/1.1" 502 150 2025-12-17 23:06:41,842 - lava_callback - ERROR - Error processing callback for node 69410c49460bc8e265117571: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69410c49460bc8e265117571 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69410c49460bc8e265117571 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69410c49460bc8e265117571 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:41,845 - lava_callback - INFO - Starting async job submit for node 694091d7575bc060ce16094f 2025-12-17 23:06:41,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:41,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091d7575bc060ce16094f HTTP/1.1" 502 150 2025-12-17 23:06:41,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091d7575bc060ce16094f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:41,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091d7575bc060ce16094f 2025-12-17 23:06:41,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091d7575bc060ce16094f HTTP/1.1" 502 150 2025-12-17 23:06:41,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091d7575bc060ce16094f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:42,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b4575bc060ce16090b 2025-12-17 23:06:42,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b4575bc060ce16090b HTTP/1.1" 502 150 2025-12-17 23:06:42,810 - lava_callback - ERROR - Error processing callback for node 694091b4575bc060ce16090b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091b4575bc060ce16090b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091b4575bc060ce16090b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091b4575bc060ce16090b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:42,811 - lava_callback - INFO - Starting async job submit for node 69409150575bc060ce1607fe 2025-12-17 23:06:42,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:42,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409150575bc060ce1607fe HTTP/1.1" 502 150 2025-12-17 23:06:42,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409150575bc060ce1607fe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:42,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409150575bc060ce1607fe 2025-12-17 23:06:42,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409150575bc060ce1607fe HTTP/1.1" 502 150 2025-12-17 23:06:42,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409150575bc060ce1607fe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:43,089 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c7575bc060ce160940 2025-12-17 23:06:43,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c7575bc060ce160940 HTTP/1.1" 502 150 2025-12-17 23:06:43,091 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c7575bc060ce160940'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:43,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091d7575bc060ce16094f 2025-12-17 23:06:43,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091d7575bc060ce16094f HTTP/1.1" 502 150 2025-12-17 23:06:43,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091d7575bc060ce16094f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:43,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091a8575bc060ce1608f7 2025-12-17 23:06:43,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091a8575bc060ce1608f7 HTTP/1.1" 502 150 2025-12-17 23:06:43,864 - lava_callback - ERROR - Error processing callback for node 694091a8575bc060ce1608f7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a8575bc060ce1608f7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a8575bc060ce1608f7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091a8575bc060ce1608f7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:43,866 - lava_callback - INFO - Starting async job submit for node 694091d9575bc060ce160952 2025-12-17 23:06:43,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:43,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091d9575bc060ce160952 HTTP/1.1" 502 150 2025-12-17 23:06:43,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091d9575bc060ce160952'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:43,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091d9575bc060ce160952 2025-12-17 23:06:43,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091d9575bc060ce160952 HTTP/1.1" 502 150 2025-12-17 23:06:43,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091d9575bc060ce160952'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:44,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091cc575bc060ce16094c 2025-12-17 23:06:44,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091cc575bc060ce16094c HTTP/1.1" 502 150 2025-12-17 23:06:44,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091cc575bc060ce16094c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:44,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409150575bc060ce1607fe 2025-12-17 23:06:44,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409150575bc060ce1607fe HTTP/1.1" 502 150 2025-12-17 23:06:44,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409150575bc060ce1607fe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:45,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ca575bc060ce160949 2025-12-17 23:06:45,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ca575bc060ce160949 HTTP/1.1" 502 150 2025-12-17 23:06:45,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ca575bc060ce160949'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:45,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091d9575bc060ce160952 2025-12-17 23:06:45,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091d9575bc060ce160952 HTTP/1.1" 502 150 2025-12-17 23:06:45,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091d9575bc060ce160952'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:46,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bd575bc060ce160931 HTTP/1.1" 502 150 2025-12-17 23:06:46,064 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091bd575bc060ce160931'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:46,494 - lava_callback - INFO - Received callback for node 69409471575bc060ce16103f INFO: 172.21.0.1:36946 - "POST /node/69409471575bc060ce16103f HTTP/1.0" 202 Accepted INFO: 172.21.0.1:36956 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:06:46,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ac575bc060ce1608fd 2025-12-17 23:06:46,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ac575bc060ce1608fd HTTP/1.1" 502 150 2025-12-17 23:06:46,968 - lava_callback - ERROR - Error processing callback for node 694091ac575bc060ce1608fd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091ac575bc060ce1608fd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091ac575bc060ce1608fd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091ac575bc060ce1608fd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:46,969 - lava_callback - INFO - Starting async job submit for node 694091db575bc060ce160955 2025-12-17 23:06:46,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:46,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091db575bc060ce160955 HTTP/1.1" 502 150 2025-12-17 23:06:46,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091db575bc060ce160955'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:46,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091db575bc060ce160955 2025-12-17 23:06:46,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091db575bc060ce160955 HTTP/1.1" 502 150 2025-12-17 23:06:46,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091db575bc060ce160955'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:47,138 - lava_callback - INFO - Received callback for node 69409473575bc060ce16104a INFO: 172.21.0.1:36966 - "POST /node/69409473575bc060ce16104a HTTP/1.0" 202 Accepted 2025-12-17 23:06:47,775 - lava_callback - INFO - Received callback for node 69409475575bc060ce16104d INFO: 172.21.0.1:36980 - "POST /node/69409475575bc060ce16104d HTTP/1.0" 202 Accepted 2025-12-17 23:06:47,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091d7575bc060ce16094f 2025-12-17 23:06:47,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091d7575bc060ce16094f HTTP/1.1" 502 150 2025-12-17 23:06:47,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091d7575bc060ce16094f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:48,445 - lava_callback - INFO - Received callback for node 69409477575bc060ce161054 INFO: 172.21.0.1:36992 - "POST /node/69409477575bc060ce161054 HTTP/1.0" 202 Accepted 2025-12-17 23:06:48,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091aa575bc060ce1608fa 2025-12-17 23:06:48,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091aa575bc060ce1608fa HTTP/1.1" 502 150 2025-12-17 23:06:48,754 - lava_callback - ERROR - Error processing callback for node 694091aa575bc060ce1608fa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091aa575bc060ce1608fa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091aa575bc060ce1608fa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091aa575bc060ce1608fa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:48,756 - lava_callback - INFO - Starting async job submit for node 694091dd575bc060ce160960 2025-12-17 23:06:48,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:48,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091dd575bc060ce160960 HTTP/1.1" 502 150 2025-12-17 23:06:48,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091dd575bc060ce160960'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:48,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091dd575bc060ce160960 2025-12-17 23:06:48,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091dd575bc060ce160960 HTTP/1.1" 502 150 2025-12-17 23:06:48,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091dd575bc060ce160960'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:48,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409150575bc060ce1607fe 2025-12-17 23:06:48,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409150575bc060ce1607fe HTTP/1.1" 502 150 2025-12-17 23:06:48,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409150575bc060ce1607fe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:48,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091db575bc060ce160955 2025-12-17 23:06:48,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091db575bc060ce160955 HTTP/1.1" 502 150 2025-12-17 23:06:48,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091db575bc060ce160955'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:49,090 - lava_callback - INFO - Received callback for node 69409479575bc060ce161057 INFO: 172.21.0.1:36994 - "POST /node/69409479575bc060ce161057 HTTP/1.0" 202 Accepted 2025-12-17 23:06:49,756 - lava_callback - INFO - Received callback for node 6940947b575bc060ce16105a INFO: 172.21.0.1:45384 - "POST /node/6940947b575bc060ce16105a HTTP/1.0" 202 Accepted 2025-12-17 23:06:49,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091d9575bc060ce160952 2025-12-17 23:06:49,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091d9575bc060ce160952 HTTP/1.1" 502 150 2025-12-17 23:06:49,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091d9575bc060ce160952'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:49,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091b7575bc060ce160916 2025-12-17 23:06:49,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b7575bc060ce160916 HTTP/1.1" 502 150 2025-12-17 23:06:49,950 - lava_callback - ERROR - Error processing callback for node 694091b7575bc060ce160916: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091b7575bc060ce160916 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091b7575bc060ce160916 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091b7575bc060ce160916 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:49,952 - lava_callback - INFO - Starting async job submit for node 6940955a575bc060ce1612df 2025-12-17 23:06:49,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:49,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955a575bc060ce1612df HTTP/1.1" 502 150 2025-12-17 23:06:49,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955a575bc060ce1612df'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:49,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955a575bc060ce1612df 2025-12-17 23:06:49,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955a575bc060ce1612df HTTP/1.1" 502 150 2025-12-17 23:06:49,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955a575bc060ce1612df'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:50,426 - lava_callback - INFO - Received callback for node 6940947c575bc060ce16105d INFO: 172.21.0.1:45400 - "POST /node/6940947c575bc060ce16105d HTTP/1.0" 202 Accepted 2025-12-17 23:06:50,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091dd575bc060ce160960 2025-12-17 23:06:50,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091dd575bc060ce160960 HTTP/1.1" 502 150 2025-12-17 23:06:50,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091dd575bc060ce160960'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:51,091 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c7575bc060ce160940 2025-12-17 23:06:51,093 - lava_callback - INFO - Received callback for node 6940947d575bc060ce161060 2025-12-17 23:06:51,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c7575bc060ce160940 HTTP/1.1" 502 150 2025-12-17 23:06:51,098 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091c7575bc060ce160940'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45404 - "POST /node/6940947d575bc060ce161060 HTTP/1.0" 202 Accepted 2025-12-17 23:06:51,551 - lava_callback - INFO - Received callback for node 6940c5401e3c24e95681d6b3 INFO: 172.21.0.1:45410 - "POST /node/6940c5401e3c24e95681d6b3 HTTP/1.0" 202 Accepted 2025-12-17 23:06:51,762 - lava_callback - INFO - Received callback for node 6940947e575bc060ce161063 INFO: 172.21.0.1:45418 - "POST /node/6940947e575bc060ce161063 HTTP/1.0" 202 Accepted 2025-12-17 23:06:51,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955a575bc060ce1612df 2025-12-17 23:06:51,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955a575bc060ce1612df HTTP/1.1" 502 150 2025-12-17 23:06:51,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955a575bc060ce1612df'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:52,441 - lava_callback - INFO - Received callback for node 69409480575bc060ce161066 INFO: 172.21.0.1:45434 - "POST /node/69409480575bc060ce161066 HTTP/1.0" 202 Accepted 2025-12-17 23:06:52,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091bb575bc060ce16092e 2025-12-17 23:06:52,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bb575bc060ce16092e HTTP/1.1" 502 150 2025-12-17 23:06:52,682 - lava_callback - ERROR - Error processing callback for node 694091bb575bc060ce16092e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091bb575bc060ce16092e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091bb575bc060ce16092e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091bb575bc060ce16092e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:52,685 - lava_callback - INFO - Starting async job submit for node 694091df575bc060ce160969 2025-12-17 23:06:52,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:52,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091df575bc060ce160969 HTTP/1.1" 502 150 2025-12-17 23:06:52,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091df575bc060ce160969'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:52,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091df575bc060ce160969 2025-12-17 23:06:52,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091df575bc060ce160969 HTTP/1.1" 502 150 2025-12-17 23:06:52,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091df575bc060ce160969'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:52,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091cc575bc060ce16094c 2025-12-17 23:06:52,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091cc575bc060ce16094c HTTP/1.1" 502 150 2025-12-17 23:06:52,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091cc575bc060ce16094c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:52,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091db575bc060ce160955 2025-12-17 23:06:52,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091db575bc060ce160955 HTTP/1.1" 502 150 2025-12-17 23:06:52,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091db575bc060ce160955'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:53,120 - lava_callback - INFO - Received callback for node 69409481575bc060ce161069 INFO: 172.21.0.1:45442 - "POST /node/69409481575bc060ce161069 HTTP/1.0" 202 Accepted 2025-12-17 23:06:53,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ca575bc060ce160949 2025-12-17 23:06:53,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ca575bc060ce160949 HTTP/1.1" 502 150 2025-12-17 23:06:53,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ca575bc060ce160949'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:54,065 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091bd575bc060ce160931 2025-12-17 23:06:54,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bd575bc060ce160931 HTTP/1.1" 502 150 2025-12-17 23:06:54,066 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091bd575bc060ce160931'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:54,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091df575bc060ce160969 2025-12-17 23:06:54,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091df575bc060ce160969 HTTP/1.1" 502 150 2025-12-17 23:06:54,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091df575bc060ce160969'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:54,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091dd575bc060ce160960 2025-12-17 23:06:54,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091dd575bc060ce160960 HTTP/1.1" 502 150 2025-12-17 23:06:54,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091dd575bc060ce160960'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:54,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091bf575bc060ce160934 2025-12-17 23:06:54,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bf575bc060ce160934 HTTP/1.1" 502 150 2025-12-17 23:06:54,793 - lava_callback - ERROR - Error processing callback for node 694091bf575bc060ce160934: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091bf575bc060ce160934 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091bf575bc060ce160934 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091bf575bc060ce160934 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:54,794 - lava_callback - INFO - Starting async job submit for node 694091e1575bc060ce16096c 2025-12-17 23:06:54,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:54,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e1575bc060ce16096c HTTP/1.1" 502 150 2025-12-17 23:06:54,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e1575bc060ce16096c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:54,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e1575bc060ce16096c 2025-12-17 23:06:54,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e1575bc060ce16096c HTTP/1.1" 502 150 2025-12-17 23:06:54,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e1575bc060ce16096c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:54,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c1575bc060ce160937 2025-12-17 23:06:54,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c1575bc060ce160937 HTTP/1.1" 502 150 2025-12-17 23:06:54,963 - lava_callback - ERROR - Error processing callback for node 694091c1575bc060ce160937: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091c1575bc060ce160937 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091c1575bc060ce160937 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091c1575bc060ce160937 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:54,965 - lava_callback - INFO - Starting async job submit for node 694091e3575bc060ce160971 2025-12-17 23:06:54,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:54,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e3575bc060ce160971 HTTP/1.1" 502 150 2025-12-17 23:06:54,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e3575bc060ce160971'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:54,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e3575bc060ce160971 2025-12-17 23:06:54,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e3575bc060ce160971 HTTP/1.1" 502 150 2025-12-17 23:06:54,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e3575bc060ce160971'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:55,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c3575bc060ce16093a 2025-12-17 23:06:55,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c3575bc060ce16093a HTTP/1.1" 502 150 2025-12-17 23:06:55,770 - lava_callback - ERROR - Error processing callback for node 694091c3575bc060ce16093a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091c3575bc060ce16093a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091c3575bc060ce16093a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091c3575bc060ce16093a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:55,772 - lava_callback - INFO - Starting async job submit for node 694091e5575bc060ce160974 2025-12-17 23:06:55,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:55,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e5575bc060ce160974 HTTP/1.1" 502 150 2025-12-17 23:06:55,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e5575bc060ce160974'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:55,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e5575bc060ce160974 2025-12-17 23:06:55,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e5575bc060ce160974 HTTP/1.1" 502 150 2025-12-17 23:06:55,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e5575bc060ce160974'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:55,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091d7575bc060ce16094f 2025-12-17 23:06:55,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091d7575bc060ce16094f HTTP/1.1" 502 150 2025-12-17 23:06:55,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091d7575bc060ce16094f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:55,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c5575bc060ce16093d 2025-12-17 23:06:55,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c5575bc060ce16093d HTTP/1.1" 502 150 2025-12-17 23:06:55,922 - lava_callback - ERROR - Error processing callback for node 694091c5575bc060ce16093d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091c5575bc060ce16093d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091c5575bc060ce16093d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091c5575bc060ce16093d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:06:55,924 - lava_callback - INFO - Starting async job submit for node 694091e7575bc060ce160977 2025-12-17 23:06:55,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:06:55,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e7575bc060ce160977 HTTP/1.1" 502 150 2025-12-17 23:06:55,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e7575bc060ce160977'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:55,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e7575bc060ce160977 2025-12-17 23:06:55,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e7575bc060ce160977 HTTP/1.1" 502 150 2025-12-17 23:06:55,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e7575bc060ce160977'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:55,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955a575bc060ce1612df 2025-12-17 23:06:55,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955a575bc060ce1612df HTTP/1.1" 502 150 2025-12-17 23:06:55,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955a575bc060ce1612df'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:56,057 - lava_callback - INFO - Received callback for node 6940948c575bc060ce161075 INFO: 172.21.0.1:45452 - "POST /node/6940948c575bc060ce161075 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:45464 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:06:56,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e1575bc060ce16096c 2025-12-17 23:06:56,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e1575bc060ce16096c HTTP/1.1" 502 150 2025-12-17 23:06:56,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e1575bc060ce16096c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:56,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409150575bc060ce1607fe 2025-12-17 23:06:56,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409150575bc060ce1607fe HTTP/1.1" 502 150 2025-12-17 23:06:56,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409150575bc060ce1607fe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:56,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e3575bc060ce160971 2025-12-17 23:06:56,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e3575bc060ce160971 HTTP/1.1" 502 150 2025-12-17 23:06:56,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e3575bc060ce160971'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:57,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e5575bc060ce160974 2025-12-17 23:06:57,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e5575bc060ce160974 HTTP/1.1" 502 150 2025-12-17 23:06:57,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e5575bc060ce160974'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:57,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091d9575bc060ce160952 2025-12-17 23:06:57,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091d9575bc060ce160952 HTTP/1.1" 502 150 2025-12-17 23:06:57,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091d9575bc060ce160952'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:57,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e7575bc060ce160977 2025-12-17 23:06:57,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e7575bc060ce160977 HTTP/1.1" 502 150 2025-12-17 23:06:57,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e7575bc060ce160977'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:06:58,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091df575bc060ce160969 2025-12-17 23:06:58,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091df575bc060ce160969 HTTP/1.1" 502 150 2025-12-17 23:06:58,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091df575bc060ce160969'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:00,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e1575bc060ce16096c 2025-12-17 23:07:00,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e1575bc060ce16096c HTTP/1.1" 502 150 2025-12-17 23:07:00,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e1575bc060ce16096c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:00,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091db575bc060ce160955 2025-12-17 23:07:00,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091db575bc060ce160955 HTTP/1.1" 502 150 2025-12-17 23:07:00,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091db575bc060ce160955'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:00,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e3575bc060ce160971 2025-12-17 23:07:00,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e3575bc060ce160971 HTTP/1.1" 502 150 2025-12-17 23:07:00,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e3575bc060ce160971'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:01,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e5575bc060ce160974 2025-12-17 23:07:01,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e5575bc060ce160974 HTTP/1.1" 502 150 2025-12-17 23:07:01,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e5575bc060ce160974'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:01,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e7575bc060ce160977 2025-12-17 23:07:01,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e7575bc060ce160977 HTTP/1.1" 502 150 2025-12-17 23:07:01,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e7575bc060ce160977'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:02,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091dd575bc060ce160960 2025-12-17 23:07:02,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091dd575bc060ce160960 HTTP/1.1" 502 150 2025-12-17 23:07:02,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091dd575bc060ce160960'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33716 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:07:03,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955a575bc060ce1612df 2025-12-17 23:07:03,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955a575bc060ce1612df HTTP/1.1" 502 150 2025-12-17 23:07:03,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955a575bc060ce1612df'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:06,503 - lava_callback - INFO - Received callback for node 69409486575bc060ce161072 INFO: 172.21.0.1:33730 - "POST /node/69409486575bc060ce161072 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:33732 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:07:06,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091df575bc060ce160969 2025-12-17 23:07:06,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091df575bc060ce160969 HTTP/1.1" 502 150 2025-12-17 23:07:06,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091df575bc060ce160969'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:07,098 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091c7575bc060ce160940 2025-12-17 23:07:07,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091c7575bc060ce160940 HTTP/1.1" 502 150 2025-12-17 23:07:07,100 - lava_callback - ERROR - Error processing callback for node 694091c7575bc060ce160940: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091c7575bc060ce160940 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091c7575bc060ce160940 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091c7575bc060ce160940 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:07,102 - lava_callback - INFO - Starting async job submit for node 694091e9575bc060ce16097a 2025-12-17 23:07:07,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:07,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e9575bc060ce16097a HTTP/1.1" 502 150 2025-12-17 23:07:07,108 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e9575bc060ce16097a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:07,108 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e9575bc060ce16097a 2025-12-17 23:07:07,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e9575bc060ce16097a HTTP/1.1" 502 150 2025-12-17 23:07:07,109 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e9575bc060ce16097a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:07,149 - lava_callback - INFO - Received callback for node 6940948f575bc060ce161081 INFO: 172.21.0.1:33746 - "POST /node/6940948f575bc060ce161081 HTTP/1.0" 202 Accepted 2025-12-17 23:07:07,797 - lava_callback - INFO - Received callback for node 69409494575bc060ce161084 INFO: 172.21.0.1:33754 - "POST /node/69409494575bc060ce161084 HTTP/1.0" 202 Accepted 2025-12-17 23:07:08,447 - lava_callback - INFO - Received callback for node 69409496575bc060ce161087 INFO: 172.21.0.1:33768 - "POST /node/69409496575bc060ce161087 HTTP/1.0" 202 Accepted 2025-12-17 23:07:08,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091cc575bc060ce16094c 2025-12-17 23:07:08,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091cc575bc060ce16094c HTTP/1.1" 502 150 2025-12-17 23:07:08,755 - lava_callback - ERROR - Error processing callback for node 694091cc575bc060ce16094c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091cc575bc060ce16094c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091cc575bc060ce16094c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091cc575bc060ce16094c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:08,757 - lava_callback - INFO - Starting async job submit for node 694091eb575bc060ce16097d 2025-12-17 23:07:08,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:08,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091eb575bc060ce16097d HTTP/1.1" 502 150 2025-12-17 23:07:08,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091eb575bc060ce16097d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:08,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091eb575bc060ce16097d 2025-12-17 23:07:08,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091eb575bc060ce16097d HTTP/1.1" 502 150 2025-12-17 23:07:08,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091eb575bc060ce16097d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:08,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e1575bc060ce16096c 2025-12-17 23:07:08,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e1575bc060ce16096c HTTP/1.1" 502 150 2025-12-17 23:07:08,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e1575bc060ce16096c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:08,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e3575bc060ce160971 2025-12-17 23:07:08,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e3575bc060ce160971 HTTP/1.1" 502 150 2025-12-17 23:07:08,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e3575bc060ce160971'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:09,089 - lava_callback - INFO - Received callback for node 69409498575bc060ce16108a INFO: 172.21.0.1:33778 - "POST /node/69409498575bc060ce16108a HTTP/1.0" 202 Accepted 2025-12-17 23:07:09,109 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e9575bc060ce16097a 2025-12-17 23:07:09,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e9575bc060ce16097a HTTP/1.1" 502 150 2025-12-17 23:07:09,110 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e9575bc060ce16097a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:09,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ca575bc060ce160949 2025-12-17 23:07:09,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ca575bc060ce160949 HTTP/1.1" 502 150 2025-12-17 23:07:09,711 - lava_callback - ERROR - Error processing callback for node 694091ca575bc060ce160949: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091ca575bc060ce160949 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091ca575bc060ce160949 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091ca575bc060ce160949 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:09,713 - lava_callback - INFO - Starting async job submit for node 694091ed575bc060ce160980 2025-12-17 23:07:09,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:09,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ed575bc060ce160980 HTTP/1.1" 502 150 2025-12-17 23:07:09,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ed575bc060ce160980'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:09,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ed575bc060ce160980 2025-12-17 23:07:09,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ed575bc060ce160980 HTTP/1.1" 502 150 2025-12-17 23:07:09,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ed575bc060ce160980'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:09,733 - lava_callback - INFO - Received callback for node 69409499575bc060ce16108d INFO: 172.21.0.1:43862 - "POST /node/69409499575bc060ce16108d HTTP/1.0" 202 Accepted 2025-12-17 23:07:09,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e5575bc060ce160974 2025-12-17 23:07:09,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e5575bc060ce160974 HTTP/1.1" 502 150 2025-12-17 23:07:09,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e5575bc060ce160974'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:09,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e7575bc060ce160977 2025-12-17 23:07:09,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e7575bc060ce160977 HTTP/1.1" 502 150 2025-12-17 23:07:09,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e7575bc060ce160977'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:10,066 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091bd575bc060ce160931 2025-12-17 23:07:10,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091bd575bc060ce160931 HTTP/1.1" 502 150 2025-12-17 23:07:10,068 - lava_callback - ERROR - Error processing callback for node 694091bd575bc060ce160931: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091bd575bc060ce160931 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091bd575bc060ce160931 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091bd575bc060ce160931 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:10,070 - lava_callback - INFO - Starting async job submit for node 694091ef575bc060ce160983 2025-12-17 23:07:10,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:10,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ef575bc060ce160983 HTTP/1.1" 502 150 2025-12-17 23:07:10,089 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ef575bc060ce160983'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:10,089 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ef575bc060ce160983 2025-12-17 23:07:10,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ef575bc060ce160983 HTTP/1.1" 502 150 2025-12-17 23:07:10,090 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ef575bc060ce160983'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:10,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091eb575bc060ce16097d 2025-12-17 23:07:10,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091eb575bc060ce16097d HTTP/1.1" 502 150 2025-12-17 23:07:10,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091eb575bc060ce16097d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:11,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ed575bc060ce160980 2025-12-17 23:07:11,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ed575bc060ce160980 HTTP/1.1" 502 150 2025-12-17 23:07:11,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ed575bc060ce160980'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:11,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091d7575bc060ce16094f 2025-12-17 23:07:11,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091d7575bc060ce16094f HTTP/1.1" 502 150 2025-12-17 23:07:11,861 - lava_callback - ERROR - Error processing callback for node 694091d7575bc060ce16094f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091d7575bc060ce16094f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091d7575bc060ce16094f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091d7575bc060ce16094f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:11,863 - lava_callback - INFO - Starting async job submit for node 6942e2f3e43d2b76e902392e 2025-12-17 23:07:11,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:11,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f3e43d2b76e902392e HTTP/1.1" 502 150 2025-12-17 23:07:11,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e2f3e43d2b76e902392e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:11,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e2f3e43d2b76e902392e 2025-12-17 23:07:11,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f3e43d2b76e902392e HTTP/1.1" 502 150 2025-12-17 23:07:11,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e2f3e43d2b76e902392e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:12,090 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ef575bc060ce160983 2025-12-17 23:07:12,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ef575bc060ce160983 HTTP/1.1" 502 150 2025-12-17 23:07:12,093 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ef575bc060ce160983'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:12,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091b2575bc060ce160906 HTTP/1.1" 502 150 2025-12-17 23:07:12,433 - lava_callback - ERROR - Error processing callback for node 694091b2575bc060ce160906: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091b2575bc060ce160906 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091b2575bc060ce160906 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091b2575bc060ce160906 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:12,435 - lava_callback - INFO - Starting async job submit for node 694091f1575bc060ce16098a 2025-12-17 23:07:12,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:12,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f1575bc060ce16098a HTTP/1.1" 502 150 2025-12-17 23:07:12,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f1575bc060ce16098a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:12,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f1575bc060ce16098a 2025-12-17 23:07:12,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f1575bc060ce16098a HTTP/1.1" 502 150 2025-12-17 23:07:12,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f1575bc060ce16098a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:12,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409150575bc060ce1607fe 2025-12-17 23:07:12,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409150575bc060ce1607fe HTTP/1.1" 502 150 2025-12-17 23:07:12,850 - lava_callback - ERROR - Error processing callback for node 69409150575bc060ce1607fe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409150575bc060ce1607fe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409150575bc060ce1607fe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409150575bc060ce1607fe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:12,852 - lava_callback - INFO - Starting async job submit for node 694091f4575bc060ce160995 2025-12-17 23:07:12,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:12,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f4575bc060ce160995 HTTP/1.1" 502 150 2025-12-17 23:07:12,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f4575bc060ce160995'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:12,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f4575bc060ce160995 2025-12-17 23:07:12,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f4575bc060ce160995 HTTP/1.1" 502 150 2025-12-17 23:07:12,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f4575bc060ce160995'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:13,111 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e9575bc060ce16097a 2025-12-17 23:07:13,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e9575bc060ce16097a HTTP/1.1" 502 150 2025-12-17 23:07:13,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e9575bc060ce16097a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:13,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091d9575bc060ce160952 2025-12-17 23:07:13,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091d9575bc060ce160952 HTTP/1.1" 502 150 2025-12-17 23:07:13,879 - lava_callback - ERROR - Error processing callback for node 694091d9575bc060ce160952: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091d9575bc060ce160952 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091d9575bc060ce160952 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091d9575bc060ce160952 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:13,881 - lava_callback - INFO - Starting async job submit for node 694091f6575bc060ce16099c 2025-12-17 23:07:13,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:13,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e2f3e43d2b76e902392e 2025-12-17 23:07:13,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f3e43d2b76e902392e HTTP/1.1" 502 150 2025-12-17 23:07:13,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e2f3e43d2b76e902392e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:13,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f6575bc060ce16099c HTTP/1.1" 502 150 2025-12-17 23:07:13,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f6575bc060ce16099c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:13,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f6575bc060ce16099c 2025-12-17 23:07:13,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f6575bc060ce16099c HTTP/1.1" 502 150 2025-12-17 23:07:13,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f6575bc060ce16099c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:14,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f1575bc060ce16098a 2025-12-17 23:07:14,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f1575bc060ce16098a HTTP/1.1" 502 150 2025-12-17 23:07:14,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f1575bc060ce16098a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:14,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091eb575bc060ce16097d 2025-12-17 23:07:14,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091eb575bc060ce16097d HTTP/1.1" 502 150 2025-12-17 23:07:14,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091eb575bc060ce16097d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:14,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f4575bc060ce160995 2025-12-17 23:07:14,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f4575bc060ce160995 HTTP/1.1" 502 150 2025-12-17 23:07:14,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f4575bc060ce160995'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43870 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:07:15,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ed575bc060ce160980 2025-12-17 23:07:15,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ed575bc060ce160980 HTTP/1.1" 502 150 2025-12-17 23:07:15,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ed575bc060ce160980'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:15,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f6575bc060ce16099c 2025-12-17 23:07:15,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f6575bc060ce16099c HTTP/1.1" 502 150 2025-12-17 23:07:15,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f6575bc060ce16099c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:16,093 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ef575bc060ce160983 2025-12-17 23:07:16,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ef575bc060ce160983 HTTP/1.1" 502 150 2025-12-17 23:07:16,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ef575bc060ce160983'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43874 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:07:16,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091db575bc060ce160955 2025-12-17 23:07:16,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091db575bc060ce160955 HTTP/1.1" 502 150 2025-12-17 23:07:16,986 - lava_callback - ERROR - Error processing callback for node 694091db575bc060ce160955: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091db575bc060ce160955 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091db575bc060ce160955 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091db575bc060ce160955 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:16,989 - lava_callback - INFO - Starting async job submit for node 694091f8575bc060ce16099f 2025-12-17 23:07:16,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:17,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f8575bc060ce16099f HTTP/1.1" 502 150 2025-12-17 23:07:17,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f8575bc060ce16099f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:17,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f8575bc060ce16099f 2025-12-17 23:07:17,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f8575bc060ce16099f HTTP/1.1" 502 150 2025-12-17 23:07:17,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f8575bc060ce16099f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:17,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e2f3e43d2b76e902392e 2025-12-17 23:07:17,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f3e43d2b76e902392e HTTP/1.1" 502 150 2025-12-17 23:07:17,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e2f3e43d2b76e902392e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:18,447 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f1575bc060ce16098a 2025-12-17 23:07:18,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f1575bc060ce16098a HTTP/1.1" 502 150 2025-12-17 23:07:18,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f1575bc060ce16098a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:18,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091dd575bc060ce160960 2025-12-17 23:07:18,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091dd575bc060ce160960 HTTP/1.1" 502 150 2025-12-17 23:07:18,769 - lava_callback - ERROR - Error processing callback for node 694091dd575bc060ce160960: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091dd575bc060ce160960 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091dd575bc060ce160960 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091dd575bc060ce160960 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:18,771 - lava_callback - INFO - Starting async job submit for node 694091fa575bc060ce1609a2 2025-12-17 23:07:18,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:18,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fa575bc060ce1609a2 HTTP/1.1" 502 150 2025-12-17 23:07:18,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091fa575bc060ce1609a2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:18,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091fa575bc060ce1609a2 2025-12-17 23:07:18,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fa575bc060ce1609a2 HTTP/1.1" 502 150 2025-12-17 23:07:18,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091fa575bc060ce1609a2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:18,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f4575bc060ce160995 2025-12-17 23:07:18,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f4575bc060ce160995 HTTP/1.1" 502 150 2025-12-17 23:07:18,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f4575bc060ce160995'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:19,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f8575bc060ce16099f 2025-12-17 23:07:19,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f6575bc060ce16099c 2025-12-17 23:07:19,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f6575bc060ce16099c HTTP/1.1" 502 150 2025-12-17 23:07:19,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f6575bc060ce16099c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:19,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955a575bc060ce1612df 2025-12-17 23:07:19,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955a575bc060ce1612df HTTP/1.1" 502 150 2025-12-17 23:07:19,992 - lava_callback - ERROR - Error processing callback for node 6940955a575bc060ce1612df: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940955a575bc060ce1612df (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940955a575bc060ce1612df (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940955a575bc060ce1612df (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:19,994 - lava_callback - INFO - Starting async job submit for node 694091fc575bc060ce1609a5 2025-12-17 23:07:19,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:20,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fc575bc060ce1609a5 HTTP/1.1" 502 150 2025-12-17 23:07:20,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091fc575bc060ce1609a5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:20,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091fc575bc060ce1609a5 2025-12-17 23:07:20,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fc575bc060ce1609a5 HTTP/1.1" 502 150 2025-12-17 23:07:20,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091fc575bc060ce1609a5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:20,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091fa575bc060ce1609a2 2025-12-17 23:07:20,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fa575bc060ce1609a2 HTTP/1.1" 502 150 2025-12-17 23:07:20,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091fa575bc060ce1609a2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:21,112 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e9575bc060ce16097a 2025-12-17 23:07:21,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e9575bc060ce16097a HTTP/1.1" 502 150 2025-12-17 23:07:21,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091e9575bc060ce16097a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:22,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091fc575bc060ce1609a5 2025-12-17 23:07:22,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fc575bc060ce1609a5 HTTP/1.1" 502 150 2025-12-17 23:07:22,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091fc575bc060ce1609a5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:22,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091df575bc060ce160969 2025-12-17 23:07:22,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091df575bc060ce160969 HTTP/1.1" 502 150 2025-12-17 23:07:22,706 - lava_callback - ERROR - Error processing callback for node 694091df575bc060ce160969: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091df575bc060ce160969 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091df575bc060ce160969 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091df575bc060ce160969 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:22,708 - lava_callback - INFO - Starting async job submit for node 694091fe575bc060ce1609a8 2025-12-17 23:07:22,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:22,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fe575bc060ce1609a8 HTTP/1.1" 502 150 2025-12-17 23:07:22,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091fe575bc060ce1609a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:22,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091fe575bc060ce1609a8 2025-12-17 23:07:22,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fe575bc060ce1609a8 HTTP/1.1" 502 150 2025-12-17 23:07:22,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091fe575bc060ce1609a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:22,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091eb575bc060ce16097d 2025-12-17 23:07:22,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091eb575bc060ce16097d HTTP/1.1" 502 150 2025-12-17 23:07:22,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091eb575bc060ce16097d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:23,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ed575bc060ce160980 2025-12-17 23:07:24,095 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ef575bc060ce160983 2025-12-17 23:07:24,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ef575bc060ce160983 HTTP/1.1" 502 150 2025-12-17 23:07:24,096 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ef575bc060ce160983'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:24,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091fe575bc060ce1609a8 2025-12-17 23:07:24,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fe575bc060ce1609a8 HTTP/1.1" 502 150 2025-12-17 23:07:24,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091fe575bc060ce1609a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:24,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091fa575bc060ce1609a2 2025-12-17 23:07:24,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fa575bc060ce1609a2 HTTP/1.1" 502 150 2025-12-17 23:07:24,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091fa575bc060ce1609a2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:24,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e1575bc060ce16096c 2025-12-17 23:07:24,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e1575bc060ce16096c HTTP/1.1" 502 150 2025-12-17 23:07:24,807 - lava_callback - ERROR - Error processing callback for node 694091e1575bc060ce16096c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091e1575bc060ce16096c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091e1575bc060ce16096c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091e1575bc060ce16096c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:24,809 - lava_callback - INFO - Starting async job submit for node 69409200575bc060ce1609ab 2025-12-17 23:07:24,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:24,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409200575bc060ce1609ab HTTP/1.1" 502 150 2025-12-17 23:07:24,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409200575bc060ce1609ab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:24,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409200575bc060ce1609ab 2025-12-17 23:07:24,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409200575bc060ce1609ab HTTP/1.1" 502 150 2025-12-17 23:07:24,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409200575bc060ce1609ab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:24,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e3575bc060ce160971 2025-12-17 23:07:25,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e3575bc060ce160971 HTTP/1.1" 502 150 2025-12-17 23:07:25,001 - lava_callback - ERROR - Error processing callback for node 694091e3575bc060ce160971: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091e3575bc060ce160971 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091e3575bc060ce160971 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091e3575bc060ce160971 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:25,003 - lava_callback - INFO - Starting async job submit for node 69409202575bc060ce1609ae 2025-12-17 23:07:25,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:25,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409202575bc060ce1609ae HTTP/1.1" 502 150 2025-12-17 23:07:25,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409202575bc060ce1609ae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:25,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409202575bc060ce1609ae 2025-12-17 23:07:25,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409202575bc060ce1609ae HTTP/1.1" 502 150 2025-12-17 23:07:25,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409202575bc060ce1609ae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:25,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e5575bc060ce160974 2025-12-17 23:07:25,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e5575bc060ce160974 HTTP/1.1" 502 150 2025-12-17 23:07:25,786 - lava_callback - ERROR - Error processing callback for node 694091e5575bc060ce160974: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091e5575bc060ce160974 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091e5575bc060ce160974 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091e5575bc060ce160974 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:25,787 - lava_callback - INFO - Starting async job submit for node 69409204575bc060ce1609b1 2025-12-17 23:07:25,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:25,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409204575bc060ce1609b1 HTTP/1.1" 502 150 2025-12-17 23:07:25,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409204575bc060ce1609b1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:25,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409204575bc060ce1609b1 2025-12-17 23:07:25,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409204575bc060ce1609b1 HTTP/1.1" 502 150 2025-12-17 23:07:25,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409204575bc060ce1609b1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:25,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e2f3e43d2b76e902392e 2025-12-17 23:07:25,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f3e43d2b76e902392e HTTP/1.1" 502 150 2025-12-17 23:07:25,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e2f3e43d2b76e902392e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:25,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e7575bc060ce160977 2025-12-17 23:07:25,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e7575bc060ce160977 HTTP/1.1" 502 150 2025-12-17 23:07:25,935 - lava_callback - ERROR - Error processing callback for node 694091e7575bc060ce160977: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091e7575bc060ce160977 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091e7575bc060ce160977 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091e7575bc060ce160977 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:25,937 - lava_callback - INFO - Starting async job submit for node 6940920c575bc060ce1609bf 2025-12-17 23:07:25,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:25,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940920c575bc060ce1609bf HTTP/1.1" 502 150 2025-12-17 23:07:25,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940920c575bc060ce1609bf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:25,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940920c575bc060ce1609bf 2025-12-17 23:07:25,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940920c575bc060ce1609bf HTTP/1.1" 502 150 2025-12-17 23:07:25,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940920c575bc060ce1609bf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:26,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091fc575bc060ce1609a5 2025-12-17 23:07:26,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fc575bc060ce1609a5 HTTP/1.1" 502 150 2025-12-17 23:07:26,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091fc575bc060ce1609a5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:26,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f1575bc060ce16098a 2025-12-17 23:07:26,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f1575bc060ce16098a HTTP/1.1" 502 150 2025-12-17 23:07:26,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f1575bc060ce16098a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:26,509 - lava_callback - INFO - Received callback for node 6940949d575bc060ce16109f INFO: 172.21.0.1:56106 - "POST /node/6940949d575bc060ce16109f HTTP/1.0" 202 Accepted INFO: 172.21.0.1:56108 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:07:26,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409200575bc060ce1609ab 2025-12-17 23:07:26,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409200575bc060ce1609ab HTTP/1.1" 502 150 2025-12-17 23:07:26,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409200575bc060ce1609ab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:26,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f4575bc060ce160995 2025-12-17 23:07:26,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f4575bc060ce160995 HTTP/1.1" 502 150 2025-12-17 23:07:26,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f4575bc060ce160995'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:27,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409202575bc060ce1609ae 2025-12-17 23:07:27,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409202575bc060ce1609ae HTTP/1.1" 502 150 2025-12-17 23:07:27,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409202575bc060ce1609ae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:27,194 - lava_callback - INFO - Received callback for node 694094a0575bc060ce1610ad INFO: 172.21.0.1:56124 - "POST /node/694094a0575bc060ce1610ad HTTP/1.0" 202 Accepted 2025-12-17 23:07:27,434 - lava_callback - INFO - Received callback for node 6940949f575bc060ce1610a8 INFO: 172.21.0.1:56136 - "POST /node/6940949f575bc060ce1610a8 HTTP/1.0" 202 Accepted 2025-12-17 23:07:27,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409204575bc060ce1609b1 2025-12-17 23:07:27,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409204575bc060ce1609b1 HTTP/1.1" 502 150 2025-12-17 23:07:27,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409204575bc060ce1609b1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:27,860 - lava_callback - INFO - Received callback for node 694094ac575bc060ce1610c6 INFO: 172.21.0.1:56144 - "POST /node/694094ac575bc060ce1610c6 HTTP/1.0" 202 Accepted 2025-12-17 23:07:27,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f6575bc060ce16099c 2025-12-17 23:07:27,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f6575bc060ce16099c HTTP/1.1" 502 150 2025-12-17 23:07:27,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f6575bc060ce16099c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:27,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940920c575bc060ce1609bf 2025-12-17 23:07:27,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940920c575bc060ce1609bf HTTP/1.1" 502 150 2025-12-17 23:07:27,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940920c575bc060ce1609bf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:28,266 - lava_callback - INFO - Received callback for node 694094af575bc060ce1610cf INFO: 172.21.0.1:56160 - "POST /node/694094af575bc060ce1610cf HTTP/1.0" 202 Accepted 2025-12-17 23:07:28,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091fe575bc060ce1609a8 2025-12-17 23:07:28,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fe575bc060ce1609a8 HTTP/1.1" 502 150 2025-12-17 23:07:28,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091fe575bc060ce1609a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:30,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409200575bc060ce1609ab 2025-12-17 23:07:30,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409200575bc060ce1609ab HTTP/1.1" 502 150 2025-12-17 23:07:30,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409200575bc060ce1609ab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:31,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409202575bc060ce1609ae 2025-12-17 23:07:31,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409202575bc060ce1609ae HTTP/1.1" 502 150 2025-12-17 23:07:31,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409202575bc060ce1609ae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:31,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409204575bc060ce1609b1 2025-12-17 23:07:31,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409204575bc060ce1609b1 HTTP/1.1" 502 150 2025-12-17 23:07:31,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409204575bc060ce1609b1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:31,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940920c575bc060ce1609bf 2025-12-17 23:07:31,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940920c575bc060ce1609bf HTTP/1.1" 502 150 2025-12-17 23:07:31,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940920c575bc060ce1609bf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:32,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091fa575bc060ce1609a2 2025-12-17 23:07:32,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fa575bc060ce1609a2 HTTP/1.1" 502 150 2025-12-17 23:07:32,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091fa575bc060ce1609a2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:34,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091fc575bc060ce1609a5 2025-12-17 23:07:34,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f8575bc060ce16099f HTTP/1.1" 502 150 2025-12-17 23:07:34,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f8575bc060ce16099f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:36,119 - lava_callback - INFO - Received callback for node 694094a2575bc060ce1610b4 INFO: 172.21.0.1:50178 - "POST /node/694094a2575bc060ce1610b4 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:50184 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:07:36,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091fe575bc060ce1609a8 2025-12-17 23:07:36,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fe575bc060ce1609a8 HTTP/1.1" 502 150 2025-12-17 23:07:36,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091fe575bc060ce1609a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:36,779 - lava_callback - INFO - Received callback for node 694094a5575bc060ce1610b7 INFO: 172.21.0.1:50198 - "POST /node/694094a5575bc060ce1610b7 HTTP/1.0" 202 Accepted 2025-12-17 23:07:37,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091e9575bc060ce16097a 2025-12-17 23:07:37,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091e9575bc060ce16097a HTTP/1.1" 502 150 2025-12-17 23:07:37,115 - lava_callback - ERROR - Error processing callback for node 694091e9575bc060ce16097a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091e9575bc060ce16097a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091e9575bc060ce16097a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091e9575bc060ce16097a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:37,117 - lava_callback - INFO - Starting async job submit for node 69409206575bc060ce1609b8 2025-12-17 23:07:37,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:37,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409206575bc060ce1609b8 HTTP/1.1" 502 150 2025-12-17 23:07:37,149 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409206575bc060ce1609b8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:37,149 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409206575bc060ce1609b8 2025-12-17 23:07:37,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409206575bc060ce1609b8 HTTP/1.1" 502 150 2025-12-17 23:07:37,150 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409206575bc060ce1609b8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:37,451 - lava_callback - INFO - Received callback for node 694094a6575bc060ce1610ba INFO: 172.21.0.1:50204 - "POST /node/694094a6575bc060ce1610ba HTTP/1.0" 202 Accepted 2025-12-17 23:07:38,116 - lava_callback - INFO - Received callback for node 694094a9575bc060ce1610bd INFO: 172.21.0.1:50212 - "POST /node/694094a9575bc060ce1610bd HTTP/1.0" 202 Accepted 2025-12-17 23:07:38,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f8575bc060ce16099f 2025-12-17 23:07:38,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f8575bc060ce16099f HTTP/1.1" 502 150 2025-12-17 23:07:38,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f8575bc060ce16099f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:38,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091eb575bc060ce16097d 2025-12-17 23:07:38,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091eb575bc060ce16097d HTTP/1.1" 502 150 2025-12-17 23:07:38,770 - lava_callback - ERROR - Error processing callback for node 694091eb575bc060ce16097d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091eb575bc060ce16097d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091eb575bc060ce16097d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091eb575bc060ce16097d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:38,772 - lava_callback - INFO - Received callback for node 694094aa575bc060ce1610c0 2025-12-17 23:07:38,773 - lava_callback - INFO - Starting async job submit for node 6940920f575bc060ce1609c2 2025-12-17 23:07:38,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:50220 - "POST /node/694094aa575bc060ce1610c0 HTTP/1.0" 202 Accepted 2025-12-17 23:07:38,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940920f575bc060ce1609c2 HTTP/1.1" 502 150 2025-12-17 23:07:38,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940920f575bc060ce1609c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:38,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940920f575bc060ce1609c2 2025-12-17 23:07:38,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940920f575bc060ce1609c2 HTTP/1.1" 502 150 2025-12-17 23:07:38,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940920f575bc060ce1609c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:38,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409200575bc060ce1609ab 2025-12-17 23:07:38,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409200575bc060ce1609ab HTTP/1.1" 502 150 2025-12-17 23:07:38,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409200575bc060ce1609ab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:39,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409202575bc060ce1609ae 2025-12-17 23:07:39,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409202575bc060ce1609ae HTTP/1.1" 502 150 2025-12-17 23:07:39,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409202575bc060ce1609ae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:39,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ed575bc060ce160980 HTTP/1.1" 502 150 2025-12-17 23:07:39,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091ed575bc060ce160980'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:39,150 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409206575bc060ce1609b8 2025-12-17 23:07:39,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409206575bc060ce1609b8 HTTP/1.1" 502 150 2025-12-17 23:07:39,151 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409206575bc060ce1609b8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:39,456 - lava_callback - INFO - Received callback for node 694094ab575bc060ce1610c3 INFO: 172.21.0.1:50226 - "POST /node/694094ab575bc060ce1610c3 HTTP/1.0" 202 Accepted 2025-12-17 23:07:39,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409204575bc060ce1609b1 2025-12-17 23:07:39,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409204575bc060ce1609b1 HTTP/1.1" 502 150 2025-12-17 23:07:39,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409204575bc060ce1609b1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:39,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940920c575bc060ce1609bf 2025-12-17 23:07:39,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940920c575bc060ce1609bf HTTP/1.1" 502 150 2025-12-17 23:07:39,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940920c575bc060ce1609bf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:40,096 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ef575bc060ce160983 2025-12-17 23:07:40,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ef575bc060ce160983 HTTP/1.1" 502 150 2025-12-17 23:07:40,097 - lava_callback - ERROR - Error processing callback for node 694091ef575bc060ce160983: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091ef575bc060ce160983 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091ef575bc060ce160983 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091ef575bc060ce160983 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:40,100 - lava_callback - INFO - Starting async job submit for node 69409213575bc060ce1609c8 2025-12-17 23:07:40,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:40,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409213575bc060ce1609c8 HTTP/1.1" 502 150 2025-12-17 23:07:40,106 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409213575bc060ce1609c8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:40,106 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409213575bc060ce1609c8 2025-12-17 23:07:40,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409213575bc060ce1609c8 HTTP/1.1" 502 150 2025-12-17 23:07:40,107 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409213575bc060ce1609c8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:40,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940920f575bc060ce1609c2 2025-12-17 23:07:40,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940920f575bc060ce1609c2 HTTP/1.1" 502 150 2025-12-17 23:07:40,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940920f575bc060ce1609c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:41,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e2f3e43d2b76e902392e 2025-12-17 23:07:41,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f3e43d2b76e902392e HTTP/1.1" 502 150 2025-12-17 23:07:41,891 - lava_callback - ERROR - Error processing callback for node 6942e2f3e43d2b76e902392e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e2f3e43d2b76e902392e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e2f3e43d2b76e902392e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e2f3e43d2b76e902392e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:41,893 - lava_callback - INFO - Starting async job submit for node 69409216575bc060ce1609cb 2025-12-17 23:07:41,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:41,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409216575bc060ce1609cb HTTP/1.1" 502 150 2025-12-17 23:07:41,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409216575bc060ce1609cb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:41,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409216575bc060ce1609cb 2025-12-17 23:07:41,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409216575bc060ce1609cb HTTP/1.1" 502 150 2025-12-17 23:07:41,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409216575bc060ce1609cb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:42,107 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409213575bc060ce1609c8 2025-12-17 23:07:42,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409213575bc060ce1609c8 HTTP/1.1" 502 150 2025-12-17 23:07:42,109 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409213575bc060ce1609c8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:42,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f1575bc060ce16098a 2025-12-17 23:07:42,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f1575bc060ce16098a HTTP/1.1" 502 150 2025-12-17 23:07:42,452 - lava_callback - ERROR - Error processing callback for node 694091f1575bc060ce16098a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091f1575bc060ce16098a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091f1575bc060ce16098a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091f1575bc060ce16098a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:42,454 - lava_callback - INFO - Starting async job submit for node 69409217575bc060ce1609ce 2025-12-17 23:07:42,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:42,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409217575bc060ce1609ce HTTP/1.1" 502 150 2025-12-17 23:07:42,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409217575bc060ce1609ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:42,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409217575bc060ce1609ce 2025-12-17 23:07:42,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409217575bc060ce1609ce HTTP/1.1" 502 150 2025-12-17 23:07:42,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409217575bc060ce1609ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:42,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f4575bc060ce160995 2025-12-17 23:07:42,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f4575bc060ce160995 HTTP/1.1" 502 150 2025-12-17 23:07:42,868 - lava_callback - ERROR - Error processing callback for node 694091f4575bc060ce160995: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091f4575bc060ce160995 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091f4575bc060ce160995 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091f4575bc060ce160995 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:42,870 - lava_callback - INFO - Starting async job submit for node 6940921c575bc060ce1609e6 2025-12-17 23:07:42,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:42,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921c575bc060ce1609e6 HTTP/1.1" 502 150 2025-12-17 23:07:42,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921c575bc060ce1609e6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:42,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921c575bc060ce1609e6 2025-12-17 23:07:42,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921c575bc060ce1609e6 HTTP/1.1" 502 150 2025-12-17 23:07:42,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921c575bc060ce1609e6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:43,152 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409206575bc060ce1609b8 2025-12-17 23:07:43,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409206575bc060ce1609b8 HTTP/1.1" 502 150 2025-12-17 23:07:43,153 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409206575bc060ce1609b8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:43,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409216575bc060ce1609cb 2025-12-17 23:07:43,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409216575bc060ce1609cb HTTP/1.1" 502 150 2025-12-17 23:07:43,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409216575bc060ce1609cb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:43,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f6575bc060ce16099c 2025-12-17 23:07:43,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f6575bc060ce16099c HTTP/1.1" 502 150 2025-12-17 23:07:43,906 - lava_callback - ERROR - Error processing callback for node 694091f6575bc060ce16099c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091f6575bc060ce16099c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091f6575bc060ce16099c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091f6575bc060ce16099c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:43,908 - lava_callback - INFO - Starting async job submit for node 69409212575bc060ce1609c5 2025-12-17 23:07:43,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:43,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409212575bc060ce1609c5 HTTP/1.1" 502 150 2025-12-17 23:07:43,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409212575bc060ce1609c5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:43,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409212575bc060ce1609c5 2025-12-17 23:07:43,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409212575bc060ce1609c5 HTTP/1.1" 502 150 2025-12-17 23:07:43,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409212575bc060ce1609c5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:44,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409217575bc060ce1609ce 2025-12-17 23:07:44,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409217575bc060ce1609ce HTTP/1.1" 502 150 2025-12-17 23:07:44,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409217575bc060ce1609ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:44,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940920f575bc060ce1609c2 2025-12-17 23:07:44,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940920f575bc060ce1609c2 HTTP/1.1" 502 150 2025-12-17 23:07:44,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940920f575bc060ce1609c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:44,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921c575bc060ce1609e6 2025-12-17 23:07:44,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921c575bc060ce1609e6 HTTP/1.1" 502 150 2025-12-17 23:07:44,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921c575bc060ce1609e6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:45,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409212575bc060ce1609c5 2025-12-17 23:07:45,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409212575bc060ce1609c5 HTTP/1.1" 502 150 2025-12-17 23:07:45,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409212575bc060ce1609c5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:46,109 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409213575bc060ce1609c8 2025-12-17 23:07:46,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409213575bc060ce1609c8 HTTP/1.1" 502 150 2025-12-17 23:07:46,110 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409213575bc060ce1609c8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:46,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f8575bc060ce16099f 2025-12-17 23:07:46,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f8575bc060ce16099f HTTP/1.1" 502 150 2025-12-17 23:07:46,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091f8575bc060ce16099f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42094 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:07:46,542 - lava_callback - INFO - Received callback for node 694094af575bc060ce1610d4 INFO: 172.21.0.1:42100 - "POST /node/694094af575bc060ce1610d4 HTTP/1.0" 202 Accepted 2025-12-17 23:07:47,209 - lava_callback - INFO - Received callback for node 694094b1575bc060ce1610d9 INFO: 172.21.0.1:42104 - "POST /node/694094b1575bc060ce1610d9 HTTP/1.0" 202 Accepted 2025-12-17 23:07:47,846 - lava_callback - INFO - Received callback for node 694094b3575bc060ce1610dc INFO: 172.21.0.1:42120 - "POST /node/694094b3575bc060ce1610dc HTTP/1.0" 202 Accepted 2025-12-17 23:07:47,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409216575bc060ce1609cb 2025-12-17 23:07:47,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409216575bc060ce1609cb HTTP/1.1" 502 150 2025-12-17 23:07:47,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409216575bc060ce1609cb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:48,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409217575bc060ce1609ce 2025-12-17 23:07:48,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409217575bc060ce1609ce HTTP/1.1" 502 150 2025-12-17 23:07:48,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409217575bc060ce1609ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:48,481 - lava_callback - INFO - Received callback for node 694094b5575bc060ce1610df INFO: 172.21.0.1:42126 - "POST /node/694094b5575bc060ce1610df HTTP/1.0" 202 Accepted 2025-12-17 23:07:48,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091fa575bc060ce1609a2 2025-12-17 23:07:48,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fa575bc060ce1609a2 HTTP/1.1" 502 150 2025-12-17 23:07:48,812 - lava_callback - ERROR - Error processing callback for node 694091fa575bc060ce1609a2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091fa575bc060ce1609a2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091fa575bc060ce1609a2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091fa575bc060ce1609a2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:48,814 - lava_callback - INFO - Starting async job submit for node 6942e382e43d2b76e9023a74 2025-12-17 23:07:48,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:48,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e382e43d2b76e9023a74 HTTP/1.1" 502 150 2025-12-17 23:07:48,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e382e43d2b76e9023a74'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:48,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e382e43d2b76e9023a74 2025-12-17 23:07:48,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e382e43d2b76e9023a74 HTTP/1.1" 502 150 2025-12-17 23:07:48,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e382e43d2b76e9023a74'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:48,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921c575bc060ce1609e6 2025-12-17 23:07:48,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921c575bc060ce1609e6 HTTP/1.1" 502 150 2025-12-17 23:07:48,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921c575bc060ce1609e6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:49,140 - lava_callback - INFO - Received callback for node 694094b7575bc060ce1610e2 INFO: 172.21.0.1:42138 - "POST /node/694094b7575bc060ce1610e2 HTTP/1.0" 202 Accepted 2025-12-17 23:07:49,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fc575bc060ce1609a5 HTTP/1.1" 502 150 2025-12-17 23:07:49,296 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694091fc575bc060ce1609a5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:49,788 - lava_callback - INFO - Received callback for node 694094bd575bc060ce1610eb INFO: 172.21.0.1:36262 - "POST /node/694094bd575bc060ce1610eb HTTP/1.0" 202 Accepted 2025-12-17 23:07:49,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409212575bc060ce1609c5 2025-12-17 23:07:49,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409212575bc060ce1609c5 HTTP/1.1" 502 150 2025-12-17 23:07:49,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409212575bc060ce1609c5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:50,444 - lava_callback - INFO - Received callback for node 694094bf575bc060ce1610ee INFO: 172.21.0.1:36266 - "POST /node/694094bf575bc060ce1610ee HTTP/1.0" 202 Accepted 2025-12-17 23:07:50,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e382e43d2b76e9023a74 2025-12-17 23:07:50,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e382e43d2b76e9023a74 HTTP/1.1" 502 150 2025-12-17 23:07:50,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e382e43d2b76e9023a74'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:51,153 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409206575bc060ce1609b8 2025-12-17 23:07:51,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409206575bc060ce1609b8 HTTP/1.1" 502 150 2025-12-17 23:07:51,154 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409206575bc060ce1609b8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:52,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091fe575bc060ce1609a8 2025-12-17 23:07:52,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fe575bc060ce1609a8 HTTP/1.1" 502 150 2025-12-17 23:07:52,732 - lava_callback - ERROR - Error processing callback for node 694091fe575bc060ce1609a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091fe575bc060ce1609a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091fe575bc060ce1609a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091fe575bc060ce1609a8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:52,733 - lava_callback - INFO - Starting async job submit for node 6940955b575bc060ce1612e2 2025-12-17 23:07:52,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:52,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955b575bc060ce1612e2 HTTP/1.1" 502 150 2025-12-17 23:07:52,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955b575bc060ce1612e2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:52,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955b575bc060ce1612e2 2025-12-17 23:07:52,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955b575bc060ce1612e2 HTTP/1.1" 502 150 2025-12-17 23:07:52,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955b575bc060ce1612e2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:52,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940920f575bc060ce1609c2 2025-12-17 23:07:52,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940920f575bc060ce1609c2 HTTP/1.1" 502 150 2025-12-17 23:07:52,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940920f575bc060ce1609c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:54,110 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409213575bc060ce1609c8 2025-12-17 23:07:54,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409213575bc060ce1609c8 HTTP/1.1" 502 150 2025-12-17 23:07:54,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409213575bc060ce1609c8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:54,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955b575bc060ce1612e2 2025-12-17 23:07:54,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955b575bc060ce1612e2 HTTP/1.1" 502 150 2025-12-17 23:07:54,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955b575bc060ce1612e2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:54,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409200575bc060ce1609ab 2025-12-17 23:07:54,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409200575bc060ce1609ab HTTP/1.1" 502 150 2025-12-17 23:07:54,821 - lava_callback - ERROR - Error processing callback for node 69409200575bc060ce1609ab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409200575bc060ce1609ab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409200575bc060ce1609ab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409200575bc060ce1609ab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:54,823 - lava_callback - INFO - Starting async job submit for node 69409219575bc060ce1609d6 2025-12-17 23:07:54,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:54,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409219575bc060ce1609d6 HTTP/1.1" 502 150 2025-12-17 23:07:54,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409219575bc060ce1609d6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:54,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409219575bc060ce1609d6 2025-12-17 23:07:54,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409219575bc060ce1609d6 HTTP/1.1" 502 150 2025-12-17 23:07:54,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409219575bc060ce1609d6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:54,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e382e43d2b76e9023a74 2025-12-17 23:07:54,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e382e43d2b76e9023a74 HTTP/1.1" 502 150 2025-12-17 23:07:54,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e382e43d2b76e9023a74'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:55,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409202575bc060ce1609ae 2025-12-17 23:07:55,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409202575bc060ce1609ae HTTP/1.1" 502 150 2025-12-17 23:07:55,018 - lava_callback - ERROR - Error processing callback for node 69409202575bc060ce1609ae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409202575bc060ce1609ae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409202575bc060ce1609ae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409202575bc060ce1609ae (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:55,019 - lava_callback - INFO - Starting async job submit for node 6940921e575bc060ce1609ef 2025-12-17 23:07:55,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:55,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921e575bc060ce1609ef HTTP/1.1" 502 150 2025-12-17 23:07:55,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921e575bc060ce1609ef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:55,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921e575bc060ce1609ef 2025-12-17 23:07:55,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921e575bc060ce1609ef HTTP/1.1" 502 150 2025-12-17 23:07:55,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921e575bc060ce1609ef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:55,056 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091ed575bc060ce160980 2025-12-17 23:07:55,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091ed575bc060ce160980 HTTP/1.1" 502 150 2025-12-17 23:07:55,057 - lava_callback - ERROR - Error processing callback for node 694091ed575bc060ce160980: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091ed575bc060ce160980 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091ed575bc060ce160980 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091ed575bc060ce160980 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:55,059 - lava_callback - INFO - Starting async job submit for node 6940921a575bc060ce1609dd 2025-12-17 23:07:55,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:55,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921a575bc060ce1609dd HTTP/1.1" 502 150 2025-12-17 23:07:55,066 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921a575bc060ce1609dd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:55,066 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921a575bc060ce1609dd 2025-12-17 23:07:55,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921a575bc060ce1609dd HTTP/1.1" 502 150 2025-12-17 23:07:55,067 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921a575bc060ce1609dd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:55,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409204575bc060ce1609b1 2025-12-17 23:07:55,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409204575bc060ce1609b1 HTTP/1.1" 502 150 2025-12-17 23:07:55,833 - lava_callback - ERROR - Error processing callback for node 69409204575bc060ce1609b1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409204575bc060ce1609b1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409204575bc060ce1609b1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409204575bc060ce1609b1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:55,836 - lava_callback - INFO - Starting async job submit for node 6940921f575bc060ce1609f6 2025-12-17 23:07:55,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:55,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921f575bc060ce1609f6 HTTP/1.1" 502 150 2025-12-17 23:07:55,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921f575bc060ce1609f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:55,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921f575bc060ce1609f6 2025-12-17 23:07:55,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921f575bc060ce1609f6 HTTP/1.1" 502 150 2025-12-17 23:07:55,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921f575bc060ce1609f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:55,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409216575bc060ce1609cb 2025-12-17 23:07:55,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409216575bc060ce1609cb HTTP/1.1" 502 150 2025-12-17 23:07:55,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409216575bc060ce1609cb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:55,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940920c575bc060ce1609bf 2025-12-17 23:07:55,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940920c575bc060ce1609bf HTTP/1.1" 502 150 2025-12-17 23:07:55,962 - lava_callback - ERROR - Error processing callback for node 6940920c575bc060ce1609bf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940920c575bc060ce1609bf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940920c575bc060ce1609bf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940920c575bc060ce1609bf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:07:55,963 - lava_callback - INFO - Starting async job submit for node 69409220575bc060ce1609fb 2025-12-17 23:07:55,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:07:55,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409220575bc060ce1609fb HTTP/1.1" 502 150 2025-12-17 23:07:55,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409220575bc060ce1609fb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:55,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409220575bc060ce1609fb 2025-12-17 23:07:55,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409220575bc060ce1609fb HTTP/1.1" 502 150 2025-12-17 23:07:55,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409220575bc060ce1609fb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:56,125 - lava_callback - INFO - Received callback for node 694094b9575bc060ce1610e5 INFO: 172.21.0.1:36272 - "POST /node/694094b9575bc060ce1610e5 HTTP/1.0" 202 Accepted 2025-12-17 23:07:56,442 - lava_callback - INFO - Received callback for node 69409764575bc060ce1617f6 INFO: 172.21.0.1:36288 - "POST /node/69409764575bc060ce1617f6 HTTP/1.0" 202 Accepted 2025-12-17 23:07:56,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409217575bc060ce1609ce 2025-12-17 23:07:56,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409217575bc060ce1609ce HTTP/1.1" 502 150 2025-12-17 23:07:56,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409217575bc060ce1609ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36298 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:07:56,773 - lava_callback - INFO - Received callback for node 694094c1575bc060ce1610f5 INFO: 172.21.0.1:36304 - "POST /node/694094c1575bc060ce1610f5 HTTP/1.0" 202 Accepted 2025-12-17 23:07:56,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409219575bc060ce1609d6 2025-12-17 23:07:56,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409219575bc060ce1609d6 HTTP/1.1" 502 150 2025-12-17 23:07:56,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409219575bc060ce1609d6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:56,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921c575bc060ce1609e6 2025-12-17 23:07:56,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921c575bc060ce1609e6 HTTP/1.1" 502 150 2025-12-17 23:07:56,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921c575bc060ce1609e6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:57,049 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921e575bc060ce1609ef 2025-12-17 23:07:57,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921e575bc060ce1609ef HTTP/1.1" 502 150 2025-12-17 23:07:57,050 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921e575bc060ce1609ef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:57,067 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921a575bc060ce1609dd 2025-12-17 23:07:57,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921a575bc060ce1609dd HTTP/1.1" 502 150 2025-12-17 23:07:57,068 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921a575bc060ce1609dd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:57,454 - lava_callback - INFO - Received callback for node 694094c3575bc060ce1610fe INFO: 172.21.0.1:36320 - "POST /node/694094c3575bc060ce1610fe HTTP/1.0" 202 Accepted 2025-12-17 23:07:57,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921f575bc060ce1609f6 2025-12-17 23:07:57,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921f575bc060ce1609f6 HTTP/1.1" 502 150 2025-12-17 23:07:57,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921f575bc060ce1609f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:57,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409212575bc060ce1609c5 2025-12-17 23:07:57,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409212575bc060ce1609c5 HTTP/1.1" 502 150 2025-12-17 23:07:57,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409212575bc060ce1609c5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:57,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409220575bc060ce1609fb 2025-12-17 23:07:57,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409220575bc060ce1609fb HTTP/1.1" 502 150 2025-12-17 23:07:57,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409220575bc060ce1609fb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:58,143 - lava_callback - INFO - Received callback for node 694094c5575bc060ce161101 INFO: 172.21.0.1:36330 - "POST /node/694094c5575bc060ce161101 HTTP/1.0" 202 Accepted 2025-12-17 23:07:58,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955b575bc060ce1612e2 2025-12-17 23:07:58,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955b575bc060ce1612e2 HTTP/1.1" 502 150 2025-12-17 23:07:58,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955b575bc060ce1612e2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:07:58,844 - lava_callback - INFO - Received callback for node 694094c8575bc060ce161108 INFO: 172.21.0.1:36346 - "POST /node/694094c8575bc060ce161108 HTTP/1.0" 202 Accepted 2025-12-17 23:08:00,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409219575bc060ce1609d6 2025-12-17 23:08:00,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409219575bc060ce1609d6 HTTP/1.1" 502 150 2025-12-17 23:08:00,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409219575bc060ce1609d6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:01,051 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921e575bc060ce1609ef 2025-12-17 23:08:01,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921e575bc060ce1609ef HTTP/1.1" 502 150 2025-12-17 23:08:01,052 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921e575bc060ce1609ef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:01,068 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921a575bc060ce1609dd 2025-12-17 23:08:01,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921a575bc060ce1609dd HTTP/1.1" 502 150 2025-12-17 23:08:01,069 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921a575bc060ce1609dd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:01,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921f575bc060ce1609f6 2025-12-17 23:08:01,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921f575bc060ce1609f6 HTTP/1.1" 502 150 2025-12-17 23:08:01,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921f575bc060ce1609f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:01,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409220575bc060ce1609fb 2025-12-17 23:08:01,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409220575bc060ce1609fb HTTP/1.1" 502 150 2025-12-17 23:08:01,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409220575bc060ce1609fb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:02,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091f8575bc060ce16099f 2025-12-17 23:08:02,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091f8575bc060ce16099f HTTP/1.1" 502 150 2025-12-17 23:08:02,454 - lava_callback - ERROR - Error processing callback for node 694091f8575bc060ce16099f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091f8575bc060ce16099f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091f8575bc060ce16099f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091f8575bc060ce16099f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:02,457 - lava_callback - INFO - Starting async job submit for node 69409222575bc060ce160a00 2025-12-17 23:08:02,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:02,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409222575bc060ce160a00 HTTP/1.1" 502 150 2025-12-17 23:08:02,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409222575bc060ce160a00'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:02,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409222575bc060ce160a00 2025-12-17 23:08:02,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409222575bc060ce160a00 HTTP/1.1" 502 150 2025-12-17 23:08:02,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409222575bc060ce160a00'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:02,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e382e43d2b76e9023a74 2025-12-17 23:08:02,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e382e43d2b76e9023a74 HTTP/1.1" 502 150 2025-12-17 23:08:02,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e382e43d2b76e9023a74'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41576 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:08:04,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409222575bc060ce160a00 2025-12-17 23:08:05,296 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694091fc575bc060ce1609a5 2025-12-17 23:08:05,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694091fc575bc060ce1609a5 HTTP/1.1" 502 150 2025-12-17 23:08:05,298 - lava_callback - ERROR - Error processing callback for node 694091fc575bc060ce1609a5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091fc575bc060ce1609a5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091fc575bc060ce1609a5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694091fc575bc060ce1609a5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:05,300 - lava_callback - INFO - Starting async job submit for node 69409223575bc060ce160a03 2025-12-17 23:08:05,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:05,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409223575bc060ce160a03 HTTP/1.1" 502 150 2025-12-17 23:08:05,307 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409223575bc060ce160a03'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:05,307 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409223575bc060ce160a03 2025-12-17 23:08:05,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409223575bc060ce160a03 HTTP/1.1" 502 150 2025-12-17 23:08:05,308 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409223575bc060ce160a03'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41578 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:08:06,554 - lava_callback - INFO - Received callback for node 694094ca575bc060ce16110b INFO: 172.21.0.1:41590 - "POST /node/694094ca575bc060ce16110b HTTP/1.0" 202 Accepted 2025-12-17 23:08:06,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955b575bc060ce1612e2 2025-12-17 23:08:06,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955b575bc060ce1612e2 HTTP/1.1" 502 150 2025-12-17 23:08:06,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955b575bc060ce1612e2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:07,155 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409206575bc060ce1609b8 2025-12-17 23:08:07,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409206575bc060ce1609b8 HTTP/1.1" 502 150 2025-12-17 23:08:07,156 - lava_callback - ERROR - Error processing callback for node 69409206575bc060ce1609b8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409206575bc060ce1609b8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409206575bc060ce1609b8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409206575bc060ce1609b8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:07,158 - lava_callback - INFO - Starting async job submit for node 6940936b575bc060ce160d6c 2025-12-17 23:08:07,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:07,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936b575bc060ce160d6c HTTP/1.1" 502 150 2025-12-17 23:08:07,198 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940936b575bc060ce160d6c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:07,198 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940936b575bc060ce160d6c 2025-12-17 23:08:07,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936b575bc060ce160d6c HTTP/1.1" 502 150 2025-12-17 23:08:07,199 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940936b575bc060ce160d6c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:07,204 - lava_callback - INFO - Received callback for node 694094cf575bc060ce161118 INFO: 172.21.0.1:41600 - "POST /node/694094cf575bc060ce161118 HTTP/1.0" 202 Accepted 2025-12-17 23:08:07,308 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409223575bc060ce160a03 2025-12-17 23:08:07,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409223575bc060ce160a03 HTTP/1.1" 502 150 2025-12-17 23:08:07,310 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409223575bc060ce160a03'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:07,416 - lava_callback - INFO - Received callback for node 694094d2575bc060ce16111e INFO: 172.21.0.1:41608 - "POST /node/694094d2575bc060ce16111e HTTP/1.0" 202 Accepted 2025-12-17 23:08:07,871 - lava_callback - INFO - Received callback for node 694094d0575bc060ce16111b INFO: 172.21.0.1:41618 - "POST /node/694094d0575bc060ce16111b HTTP/1.0" 202 Accepted 2025-12-17 23:08:08,208 - lava_callback - INFO - Received callback for node 694094d4575bc060ce161124 INFO: 172.21.0.1:41626 - "POST /node/694094d4575bc060ce161124 HTTP/1.0" 202 Accepted 2025-12-17 23:08:08,534 - lava_callback - INFO - Received callback for node 694094d3575bc060ce161121 INFO: 172.21.0.1:41640 - "POST /node/694094d3575bc060ce161121 HTTP/1.0" 202 Accepted 2025-12-17 23:08:08,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940920f575bc060ce1609c2 2025-12-17 23:08:08,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940920f575bc060ce1609c2 HTTP/1.1" 502 150 2025-12-17 23:08:08,796 - lava_callback - ERROR - Error processing callback for node 6940920f575bc060ce1609c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940920f575bc060ce1609c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940920f575bc060ce1609c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940920f575bc060ce1609c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:08,798 - lava_callback - INFO - Starting async job submit for node 69409225575bc060ce160a06 2025-12-17 23:08:08,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:08,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409225575bc060ce160a06 HTTP/1.1" 502 150 2025-12-17 23:08:08,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409225575bc060ce160a06'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:08,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409225575bc060ce160a06 2025-12-17 23:08:08,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409225575bc060ce160a06 HTTP/1.1" 502 150 2025-12-17 23:08:08,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409225575bc060ce160a06'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:08,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409219575bc060ce1609d6 2025-12-17 23:08:08,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409219575bc060ce1609d6 HTTP/1.1" 502 150 2025-12-17 23:08:08,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409219575bc060ce1609d6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:09,052 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921e575bc060ce1609ef 2025-12-17 23:08:09,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921e575bc060ce1609ef HTTP/1.1" 502 150 2025-12-17 23:08:09,053 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921e575bc060ce1609ef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:09,070 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921a575bc060ce1609dd 2025-12-17 23:08:09,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921a575bc060ce1609dd HTTP/1.1" 502 150 2025-12-17 23:08:09,070 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921a575bc060ce1609dd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:09,199 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940936b575bc060ce160d6c 2025-12-17 23:08:09,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936b575bc060ce160d6c HTTP/1.1" 502 150 2025-12-17 23:08:09,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940936b575bc060ce160d6c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:09,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921f575bc060ce1609f6 2025-12-17 23:08:09,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921f575bc060ce1609f6 HTTP/1.1" 502 150 2025-12-17 23:08:09,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940921f575bc060ce1609f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:09,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409220575bc060ce1609fb 2025-12-17 23:08:09,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409220575bc060ce1609fb HTTP/1.1" 502 150 2025-12-17 23:08:09,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409220575bc060ce1609fb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:10,112 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409213575bc060ce1609c8 2025-12-17 23:08:10,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409213575bc060ce1609c8 HTTP/1.1" 502 150 2025-12-17 23:08:10,113 - lava_callback - ERROR - Error processing callback for node 69409213575bc060ce1609c8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409213575bc060ce1609c8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409213575bc060ce1609c8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409213575bc060ce1609c8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:10,115 - lava_callback - INFO - Starting async job submit for node 69409227575bc060ce160a09 2025-12-17 23:08:10,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:10,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409227575bc060ce160a09 HTTP/1.1" 502 150 2025-12-17 23:08:10,122 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409227575bc060ce160a09'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:10,122 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409227575bc060ce160a09 2025-12-17 23:08:10,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409227575bc060ce160a09 HTTP/1.1" 502 150 2025-12-17 23:08:10,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409227575bc060ce160a09'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:10,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409225575bc060ce160a06 2025-12-17 23:08:10,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409225575bc060ce160a06 HTTP/1.1" 502 150 2025-12-17 23:08:10,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409225575bc060ce160a06'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:11,266 - lava_callback - INFO - Received callback for node 694094cc575bc060ce161112 INFO: 172.21.0.1:41986 - "POST /node/694094cc575bc060ce161112 HTTP/1.0" 202 Accepted 2025-12-17 23:08:11,310 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409223575bc060ce160a03 2025-12-17 23:08:11,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409216575bc060ce1609cb 2025-12-17 23:08:11,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409216575bc060ce1609cb HTTP/1.1" 502 150 2025-12-17 23:08:11,909 - lava_callback - ERROR - Error processing callback for node 69409216575bc060ce1609cb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409216575bc060ce1609cb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409216575bc060ce1609cb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409216575bc060ce1609cb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:11,911 - lava_callback - INFO - Starting async job submit for node 69409790575bc060ce16186d 2025-12-17 23:08:11,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:11,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409790575bc060ce16186d HTTP/1.1" 502 150 2025-12-17 23:08:11,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409790575bc060ce16186d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:11,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409790575bc060ce16186d 2025-12-17 23:08:11,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409790575bc060ce16186d HTTP/1.1" 502 150 2025-12-17 23:08:11,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409790575bc060ce16186d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:12,123 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409227575bc060ce160a09 2025-12-17 23:08:12,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409227575bc060ce160a09 HTTP/1.1" 502 150 2025-12-17 23:08:12,125 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409227575bc060ce160a09'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:12,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409217575bc060ce1609ce 2025-12-17 23:08:12,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409217575bc060ce1609ce HTTP/1.1" 502 150 2025-12-17 23:08:12,467 - lava_callback - ERROR - Error processing callback for node 69409217575bc060ce1609ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409217575bc060ce1609ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409217575bc060ce1609ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409217575bc060ce1609ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:12,469 - lava_callback - INFO - Starting async job submit for node 69409229575bc060ce160a12 2025-12-17 23:08:12,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:12,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409229575bc060ce160a12 HTTP/1.1" 502 150 2025-12-17 23:08:12,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409229575bc060ce160a12'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:12,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409229575bc060ce160a12 2025-12-17 23:08:12,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409229575bc060ce160a12 HTTP/1.1" 502 150 2025-12-17 23:08:12,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409229575bc060ce160a12'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:12,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921c575bc060ce1609e6 2025-12-17 23:08:12,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921c575bc060ce1609e6 HTTP/1.1" 502 150 2025-12-17 23:08:12,888 - lava_callback - ERROR - Error processing callback for node 6940921c575bc060ce1609e6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940921c575bc060ce1609e6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940921c575bc060ce1609e6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940921c575bc060ce1609e6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:12,890 - lava_callback - INFO - Starting async job submit for node 6940922b575bc060ce160a15 2025-12-17 23:08:12,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:12,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940922b575bc060ce160a15 HTTP/1.1" 502 150 2025-12-17 23:08:12,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940922b575bc060ce160a15'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:12,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940922b575bc060ce160a15 2025-12-17 23:08:12,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940922b575bc060ce160a15 HTTP/1.1" 502 150 2025-12-17 23:08:12,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940922b575bc060ce160a15'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:13,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940936b575bc060ce160d6c 2025-12-17 23:08:13,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936b575bc060ce160d6c HTTP/1.1" 502 150 2025-12-17 23:08:13,201 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940936b575bc060ce160d6c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:13,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409212575bc060ce1609c5 2025-12-17 23:08:13,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409212575bc060ce1609c5 HTTP/1.1" 502 150 2025-12-17 23:08:13,925 - lava_callback - ERROR - Error processing callback for node 69409212575bc060ce1609c5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409212575bc060ce1609c5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409212575bc060ce1609c5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409212575bc060ce1609c5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:13,927 - lava_callback - INFO - Starting async job submit for node 6940922d575bc060ce160a1e 2025-12-17 23:08:13,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:13,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940922d575bc060ce160a1e HTTP/1.1" 502 150 2025-12-17 23:08:13,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940922d575bc060ce160a1e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:13,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940922d575bc060ce160a1e 2025-12-17 23:08:13,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940922d575bc060ce160a1e HTTP/1.1" 502 150 2025-12-17 23:08:13,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940922d575bc060ce160a1e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:13,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409790575bc060ce16186d 2025-12-17 23:08:13,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409790575bc060ce16186d HTTP/1.1" 502 150 2025-12-17 23:08:13,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409790575bc060ce16186d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:14,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409229575bc060ce160a12 2025-12-17 23:08:14,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409229575bc060ce160a12 HTTP/1.1" 502 150 2025-12-17 23:08:14,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409229575bc060ce160a12'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:14,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409225575bc060ce160a06 2025-12-17 23:08:14,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409225575bc060ce160a06 HTTP/1.1" 502 150 2025-12-17 23:08:14,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409225575bc060ce160a06'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:14,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940922b575bc060ce160a15 2025-12-17 23:08:14,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940922b575bc060ce160a15 HTTP/1.1" 502 150 2025-12-17 23:08:14,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940922b575bc060ce160a15'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41990 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:08:15,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940922d575bc060ce160a1e 2025-12-17 23:08:15,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940922d575bc060ce160a1e HTTP/1.1" 502 150 2025-12-17 23:08:15,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940922d575bc060ce160a1e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:16,125 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409227575bc060ce160a09 INFO: 172.21.0.1:41996 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:08:17,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409790575bc060ce16186d 2025-12-17 23:08:17,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409790575bc060ce16186d HTTP/1.1" 502 150 2025-12-17 23:08:17,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409790575bc060ce16186d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:18,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409229575bc060ce160a12 2025-12-17 23:08:18,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409229575bc060ce160a12 HTTP/1.1" 502 150 2025-12-17 23:08:18,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409229575bc060ce160a12'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:18,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e382e43d2b76e9023a74 2025-12-17 23:08:18,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e382e43d2b76e9023a74 HTTP/1.1" 502 150 2025-12-17 23:08:18,844 - lava_callback - ERROR - Error processing callback for node 6942e382e43d2b76e9023a74: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e382e43d2b76e9023a74 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e382e43d2b76e9023a74 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e382e43d2b76e9023a74 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:18,845 - lava_callback - INFO - Starting async job submit for node 69409230575bc060ce160a28 2025-12-17 23:08:18,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:18,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409230575bc060ce160a28 HTTP/1.1" 502 150 2025-12-17 23:08:18,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409230575bc060ce160a28'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:18,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409230575bc060ce160a28 2025-12-17 23:08:18,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409230575bc060ce160a28 HTTP/1.1" 502 150 2025-12-17 23:08:18,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409230575bc060ce160a28'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:18,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940922b575bc060ce160a15 2025-12-17 23:08:18,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940922b575bc060ce160a15 HTTP/1.1" 502 150 2025-12-17 23:08:18,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940922b575bc060ce160a15'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:19,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409222575bc060ce160a00 HTTP/1.1" 502 150 2025-12-17 23:08:19,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409222575bc060ce160a00'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:19,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940922d575bc060ce160a1e 2025-12-17 23:08:19,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940922d575bc060ce160a1e HTTP/1.1" 502 150 2025-12-17 23:08:19,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940922d575bc060ce160a1e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:20,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409230575bc060ce160a28 2025-12-17 23:08:20,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409230575bc060ce160a28 HTTP/1.1" 502 150 2025-12-17 23:08:20,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409230575bc060ce160a28'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:21,202 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940936b575bc060ce160d6c 2025-12-17 23:08:21,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936b575bc060ce160d6c HTTP/1.1" 502 150 2025-12-17 23:08:21,203 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940936b575bc060ce160d6c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:22,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955b575bc060ce1612e2 2025-12-17 23:08:22,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955b575bc060ce1612e2 HTTP/1.1" 502 150 2025-12-17 23:08:22,779 - lava_callback - ERROR - Error processing callback for node 6940955b575bc060ce1612e2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940955b575bc060ce1612e2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940955b575bc060ce1612e2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940955b575bc060ce1612e2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:22,781 - lava_callback - INFO - Starting async job submit for node 69409232575bc060ce160a34 2025-12-17 23:08:22,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:22,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409232575bc060ce160a34 HTTP/1.1" 502 150 2025-12-17 23:08:22,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409232575bc060ce160a34'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:22,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409232575bc060ce160a34 2025-12-17 23:08:22,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409232575bc060ce160a34 HTTP/1.1" 502 150 2025-12-17 23:08:22,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409232575bc060ce160a34'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:22,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409225575bc060ce160a06 2025-12-17 23:08:22,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409225575bc060ce160a06 HTTP/1.1" 502 150 2025-12-17 23:08:22,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409225575bc060ce160a06'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:23,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409222575bc060ce160a00 2025-12-17 23:08:23,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409222575bc060ce160a00 HTTP/1.1" 502 150 2025-12-17 23:08:23,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409222575bc060ce160a00'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:24,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409232575bc060ce160a34 2025-12-17 23:08:24,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409232575bc060ce160a34 HTTP/1.1" 502 150 2025-12-17 23:08:24,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409232575bc060ce160a34'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:24,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409219575bc060ce1609d6 2025-12-17 23:08:24,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409219575bc060ce1609d6 HTTP/1.1" 502 150 2025-12-17 23:08:24,838 - lava_callback - ERROR - Error processing callback for node 69409219575bc060ce1609d6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409219575bc060ce1609d6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409219575bc060ce1609d6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409219575bc060ce1609d6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:24,840 - lava_callback - INFO - Starting async job submit for node 69409235575bc060ce160a38 2025-12-17 23:08:24,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:24,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409235575bc060ce160a38 HTTP/1.1" 502 150 2025-12-17 23:08:24,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409235575bc060ce160a38'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:24,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409235575bc060ce160a38 2025-12-17 23:08:24,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409235575bc060ce160a38 HTTP/1.1" 502 150 2025-12-17 23:08:24,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409235575bc060ce160a38'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:24,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409230575bc060ce160a28 2025-12-17 23:08:24,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409230575bc060ce160a28 HTTP/1.1" 502 150 2025-12-17 23:08:24,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409230575bc060ce160a28'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:25,053 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921e575bc060ce1609ef 2025-12-17 23:08:25,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921e575bc060ce1609ef HTTP/1.1" 502 150 2025-12-17 23:08:25,055 - lava_callback - ERROR - Error processing callback for node 6940921e575bc060ce1609ef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940921e575bc060ce1609ef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940921e575bc060ce1609ef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940921e575bc060ce1609ef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:25,057 - lava_callback - INFO - Starting async job submit for node 6940923f575bc060ce160a4b 2025-12-17 23:08:25,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:25,071 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921a575bc060ce1609dd 2025-12-17 23:08:25,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921a575bc060ce1609dd HTTP/1.1" 502 150 2025-12-17 23:08:25,072 - lava_callback - ERROR - Error processing callback for node 6940921a575bc060ce1609dd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940921a575bc060ce1609dd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940921a575bc060ce1609dd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940921a575bc060ce1609dd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:25,073 - lava_callback - INFO - Starting async job submit for node 69409237575bc060ce160a3f 2025-12-17 23:08:25,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:25,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409237575bc060ce160a3f HTTP/1.1" 502 150 2025-12-17 23:08:25,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409237575bc060ce160a3f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:25,080 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409237575bc060ce160a3f 2025-12-17 23:08:25,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409237575bc060ce160a3f HTTP/1.1" 502 150 2025-12-17 23:08:25,081 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409237575bc060ce160a3f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:25,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923f575bc060ce160a4b HTTP/1.1" 502 150 2025-12-17 23:08:25,084 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940923f575bc060ce160a4b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:25,084 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940923f575bc060ce160a4b 2025-12-17 23:08:25,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923f575bc060ce160a4b HTTP/1.1" 502 150 2025-12-17 23:08:25,085 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940923f575bc060ce160a4b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:25,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940921f575bc060ce1609f6 2025-12-17 23:08:25,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940921f575bc060ce1609f6 HTTP/1.1" 502 150 2025-12-17 23:08:25,853 - lava_callback - ERROR - Error processing callback for node 6940921f575bc060ce1609f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940921f575bc060ce1609f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940921f575bc060ce1609f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940921f575bc060ce1609f6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:25,855 - lava_callback - INFO - Starting async job submit for node 69409239575bc060ce160a42 2025-12-17 23:08:25,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:25,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409239575bc060ce160a42 HTTP/1.1" 502 150 2025-12-17 23:08:25,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409239575bc060ce160a42'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:25,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409239575bc060ce160a42 2025-12-17 23:08:25,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409239575bc060ce160a42 HTTP/1.1" 502 150 2025-12-17 23:08:25,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409239575bc060ce160a42'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:25,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409220575bc060ce1609fb 2025-12-17 23:08:25,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409220575bc060ce1609fb HTTP/1.1" 502 150 2025-12-17 23:08:25,978 - lava_callback - ERROR - Error processing callback for node 69409220575bc060ce1609fb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409220575bc060ce1609fb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409220575bc060ce1609fb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409220575bc060ce1609fb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:25,980 - lava_callback - INFO - Starting async job submit for node 6940923b575bc060ce160a45 2025-12-17 23:08:25,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:25,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409790575bc060ce16186d 2025-12-17 23:08:25,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409790575bc060ce16186d HTTP/1.1" 502 150 2025-12-17 23:08:25,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409790575bc060ce16186d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:26,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409229575bc060ce160a12 2025-12-17 23:08:26,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409229575bc060ce160a12 HTTP/1.1" 502 150 2025-12-17 23:08:26,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409229575bc060ce160a12'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:32844 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:08:26,589 - lava_callback - INFO - Received callback for node 694094d5575bc060ce16112d INFO: 172.21.0.1:32860 - "POST /node/694094d5575bc060ce16112d HTTP/1.0" 202 Accepted 2025-12-17 23:08:26,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409223575bc060ce160a03 HTTP/1.1" 502 150 2025-12-17 23:08:26,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409223575bc060ce160a03'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:26,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409235575bc060ce160a38 2025-12-17 23:08:26,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409235575bc060ce160a38 HTTP/1.1" 502 150 2025-12-17 23:08:26,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409235575bc060ce160a38'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:26,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940922b575bc060ce160a15 2025-12-17 23:08:26,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940922b575bc060ce160a15 HTTP/1.1" 502 150 2025-12-17 23:08:26,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940922b575bc060ce160a15'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:27,081 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409237575bc060ce160a3f 2025-12-17 23:08:27,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409237575bc060ce160a3f HTTP/1.1" 502 150 2025-12-17 23:08:27,083 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409237575bc060ce160a3f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:27,085 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940923f575bc060ce160a4b 2025-12-17 23:08:27,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923f575bc060ce160a4b HTTP/1.1" 502 150 2025-12-17 23:08:27,086 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940923f575bc060ce160a4b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:27,257 - lava_callback - INFO - Received callback for node 694094d7575bc060ce161136 INFO: 172.21.0.1:32870 - "POST /node/694094d7575bc060ce161136 HTTP/1.0" 202 Accepted 2025-12-17 23:08:27,326 - lava_callback - INFO - Received callback for node 694094df575bc060ce161142 INFO: 172.21.0.1:32882 - "POST /node/694094df575bc060ce161142 HTTP/1.0" 202 Accepted 2025-12-17 23:08:27,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409239575bc060ce160a42 2025-12-17 23:08:27,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409239575bc060ce160a42 HTTP/1.1" 502 150 2025-12-17 23:08:27,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409239575bc060ce160a42'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:27,926 - lava_callback - INFO - Received callback for node 694094d9575bc060ce161139 INFO: 172.21.0.1:32886 - "POST /node/694094d9575bc060ce161139 HTTP/1.0" 202 Accepted 2025-12-17 23:08:27,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940922d575bc060ce160a1e 2025-12-17 23:08:27,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940922d575bc060ce160a1e HTTP/1.1" 502 150 2025-12-17 23:08:27,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940922d575bc060ce160a1e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:28,570 - lava_callback - INFO - Received callback for node 694094db575bc060ce16113c INFO: 172.21.0.1:32896 - "POST /node/694094db575bc060ce16113c HTTP/1.0" 202 Accepted 2025-12-17 23:08:28,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409232575bc060ce160a34 2025-12-17 23:08:28,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409232575bc060ce160a34 HTTP/1.1" 502 150 2025-12-17 23:08:28,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409232575bc060ce160a34'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:29,232 - lava_callback - INFO - Received callback for node 694094dd575bc060ce16113f INFO: 172.21.0.1:32906 - "POST /node/694094dd575bc060ce16113f HTTP/1.0" 202 Accepted 2025-12-17 23:08:30,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409235575bc060ce160a38 2025-12-17 23:08:30,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409235575bc060ce160a38 HTTP/1.1" 502 150 2025-12-17 23:08:30,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409235575bc060ce160a38'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:31,083 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409237575bc060ce160a3f 2025-12-17 23:08:31,086 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940923f575bc060ce160a4b 2025-12-17 23:08:31,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409237575bc060ce160a3f HTTP/1.1" 502 150 2025-12-17 23:08:31,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409237575bc060ce160a3f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:31,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923f575bc060ce160a4b HTTP/1.1" 502 150 2025-12-17 23:08:31,089 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940923f575bc060ce160a4b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:31,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409227575bc060ce160a09 HTTP/1.1" 502 150 2025-12-17 23:08:31,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409227575bc060ce160a09'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:31,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409222575bc060ce160a00 2025-12-17 23:08:31,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409222575bc060ce160a00 HTTP/1.1" 502 150 2025-12-17 23:08:31,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409222575bc060ce160a00'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:31,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409239575bc060ce160a42 2025-12-17 23:08:31,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409239575bc060ce160a42 HTTP/1.1" 502 150 2025-12-17 23:08:31,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409239575bc060ce160a42'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:32,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409230575bc060ce160a28 2025-12-17 23:08:32,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409230575bc060ce160a28 HTTP/1.1" 502 150 2025-12-17 23:08:32,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409230575bc060ce160a28'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:34,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409223575bc060ce160a03 2025-12-17 23:08:34,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409223575bc060ce160a03 HTTP/1.1" 502 150 2025-12-17 23:08:34,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409223575bc060ce160a03'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:35,310 - lava_callback - INFO - Received callback for node 6940998e575bc060ce161d3c INFO: 172.21.0.1:57084 - "POST /node/6940998e575bc060ce161d3c HTTP/1.0" 202 Accepted 2025-12-17 23:08:35,549 - lava_callback - INFO - Received callback for node 694097a2575bc060ce1618ad INFO: 172.21.0.1:57086 - "POST /node/694097a2575bc060ce1618ad HTTP/1.0" 202 Accepted 2025-12-17 23:08:36,206 - lava_callback - INFO - Received callback for node 694094e0575bc060ce161145 INFO: 172.21.0.1:57088 - "POST /node/694094e0575bc060ce161145 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:57094 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:08:36,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409232575bc060ce160a34 2025-12-17 23:08:36,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409232575bc060ce160a34 HTTP/1.1" 502 150 2025-12-17 23:08:36,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409232575bc060ce160a34'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:36,877 - lava_callback - INFO - Received callback for node 694094e2575bc060ce161148 INFO: 172.21.0.1:57098 - "POST /node/694094e2575bc060ce161148 HTTP/1.0" 202 Accepted 2025-12-17 23:08:37,203 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940936b575bc060ce160d6c 2025-12-17 23:08:37,577 - lava_callback - INFO - Received callback for node 694094e3575bc060ce16114b INFO: 172.21.0.1:57106 - "POST /node/694094e3575bc060ce16114b HTTP/1.0" 202 Accepted 2025-12-17 23:08:38,243 - lava_callback - INFO - Received callback for node 694094e4575bc060ce16114e INFO: 172.21.0.1:57118 - "POST /node/694094e4575bc060ce16114e HTTP/1.0" 202 Accepted 2025-12-17 23:08:38,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409225575bc060ce160a06 2025-12-17 23:08:38,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409225575bc060ce160a06 HTTP/1.1" 502 150 2025-12-17 23:08:38,815 - lava_callback - ERROR - Error processing callback for node 69409225575bc060ce160a06: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409225575bc060ce160a06 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409225575bc060ce160a06 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409225575bc060ce160a06 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:38,816 - lava_callback - INFO - Starting async job submit for node 6940923d575bc060ce160a48 2025-12-17 23:08:38,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:38,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923d575bc060ce160a48 HTTP/1.1" 502 150 2025-12-17 23:08:38,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940923d575bc060ce160a48'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:38,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940923d575bc060ce160a48 2025-12-17 23:08:38,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923d575bc060ce160a48 HTTP/1.1" 502 150 2025-12-17 23:08:38,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940923d575bc060ce160a48'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:38,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409235575bc060ce160a38 2025-12-17 23:08:38,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409235575bc060ce160a38 HTTP/1.1" 502 150 2025-12-17 23:08:38,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409235575bc060ce160a38'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:38,904 - lava_callback - INFO - Received callback for node 694094e6575bc060ce161151 INFO: 172.21.0.1:57132 - "POST /node/694094e6575bc060ce161151 HTTP/1.0" 202 Accepted 2025-12-17 23:08:39,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409237575bc060ce160a3f 2025-12-17 23:08:39,089 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940923f575bc060ce160a4b 2025-12-17 23:08:39,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409237575bc060ce160a3f HTTP/1.1" 502 150 2025-12-17 23:08:39,090 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409237575bc060ce160a3f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:39,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923f575bc060ce160a4b HTTP/1.1" 502 150 2025-12-17 23:08:39,090 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940923f575bc060ce160a4b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:39,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409227575bc060ce160a09 2025-12-17 23:08:39,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409227575bc060ce160a09 HTTP/1.1" 502 150 2025-12-17 23:08:39,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409227575bc060ce160a09'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:39,557 - lava_callback - INFO - Received callback for node 694094e7575bc060ce161154 INFO: 172.21.0.1:57144 - "POST /node/694094e7575bc060ce161154 HTTP/1.0" 202 Accepted 2025-12-17 23:08:39,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409239575bc060ce160a42 2025-12-17 23:08:39,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409239575bc060ce160a42 HTTP/1.1" 502 150 2025-12-17 23:08:39,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409239575bc060ce160a42'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:40,229 - lava_callback - INFO - Received callback for node 694094e8575bc060ce161157 INFO: 172.21.0.1:49262 - "POST /node/694094e8575bc060ce161157 HTTP/1.0" 202 Accepted 2025-12-17 23:08:40,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940923d575bc060ce160a48 2025-12-17 23:08:40,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923d575bc060ce160a48 HTTP/1.1" 502 150 2025-12-17 23:08:40,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940923d575bc060ce160a48'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:40,873 - lava_callback - INFO - Received callback for node 694094ea575bc060ce161160 INFO: 172.21.0.1:49278 - "POST /node/694094ea575bc060ce161160 HTTP/1.0" 202 Accepted 2025-12-17 23:08:41,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923b575bc060ce160a45 HTTP/1.1" 502 150 2025-12-17 23:08:41,264 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940923b575bc060ce160a45'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:41,264 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940923b575bc060ce160a45 2025-12-17 23:08:41,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923b575bc060ce160a45 HTTP/1.1" 502 150 2025-12-17 23:08:41,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940923b575bc060ce160a45'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:41,513 - lava_callback - INFO - Received callback for node 694094eb575bc060ce161165 INFO: 172.21.0.1:49286 - "POST /node/694094eb575bc060ce161165 HTTP/1.0" 202 Accepted 2025-12-17 23:08:41,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409790575bc060ce16186d 2025-12-17 23:08:41,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409790575bc060ce16186d HTTP/1.1" 502 150 2025-12-17 23:08:41,989 - lava_callback - ERROR - Error processing callback for node 69409790575bc060ce16186d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409790575bc060ce16186d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409790575bc060ce16186d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409790575bc060ce16186d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:41,991 - lava_callback - INFO - Starting async job submit for node 69409241575bc060ce160a54 2025-12-17 23:08:41,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:41,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409241575bc060ce160a54 HTTP/1.1" 502 150 2025-12-17 23:08:41,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409241575bc060ce160a54'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:42,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409241575bc060ce160a54 2025-12-17 23:08:42,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409229575bc060ce160a12 2025-12-17 23:08:42,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409229575bc060ce160a12 HTTP/1.1" 502 150 2025-12-17 23:08:42,484 - lava_callback - ERROR - Error processing callback for node 69409229575bc060ce160a12: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409229575bc060ce160a12 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409229575bc060ce160a12 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409229575bc060ce160a12 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:42,485 - lava_callback - INFO - Starting async job submit for node 69409243575bc060ce160a5d 2025-12-17 23:08:42,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:42,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409243575bc060ce160a5d HTTP/1.1" 502 150 2025-12-17 23:08:42,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409243575bc060ce160a5d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:42,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409243575bc060ce160a5d 2025-12-17 23:08:42,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409243575bc060ce160a5d HTTP/1.1" 502 150 2025-12-17 23:08:42,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409243575bc060ce160a5d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:42,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940922b575bc060ce160a15 2025-12-17 23:08:42,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940922b575bc060ce160a15 HTTP/1.1" 502 150 2025-12-17 23:08:42,905 - lava_callback - ERROR - Error processing callback for node 6940922b575bc060ce160a15: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940922b575bc060ce160a15 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940922b575bc060ce160a15 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940922b575bc060ce160a15 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:42,907 - lava_callback - INFO - Starting async job submit for node 69409245575bc060ce160a60 2025-12-17 23:08:42,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:42,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409245575bc060ce160a60 HTTP/1.1" 502 150 2025-12-17 23:08:42,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409245575bc060ce160a60'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:42,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409245575bc060ce160a60 2025-12-17 23:08:42,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409245575bc060ce160a60 HTTP/1.1" 502 150 2025-12-17 23:08:42,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409245575bc060ce160a60'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:43,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940923b575bc060ce160a45 2025-12-17 23:08:43,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923b575bc060ce160a45 HTTP/1.1" 502 150 2025-12-17 23:08:43,267 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940923b575bc060ce160a45'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:43,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940922d575bc060ce160a1e 2025-12-17 23:08:43,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940922d575bc060ce160a1e HTTP/1.1" 502 150 2025-12-17 23:08:43,942 - lava_callback - ERROR - Error processing callback for node 6940922d575bc060ce160a1e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940922d575bc060ce160a1e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940922d575bc060ce160a1e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940922d575bc060ce160a1e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:43,944 - lava_callback - INFO - Starting async job submit for node 69410fd9460bc8e265117ad5 2025-12-17 23:08:43,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:43,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69410fd9460bc8e265117ad5 HTTP/1.1" 502 150 2025-12-17 23:08:43,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69410fd9460bc8e265117ad5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:43,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69410fd9460bc8e265117ad5 2025-12-17 23:08:43,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69410fd9460bc8e265117ad5 HTTP/1.1" 502 150 2025-12-17 23:08:43,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69410fd9460bc8e265117ad5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:44,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409243575bc060ce160a5d 2025-12-17 23:08:44,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409243575bc060ce160a5d HTTP/1.1" 502 150 2025-12-17 23:08:44,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409243575bc060ce160a5d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:44,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940923d575bc060ce160a48 2025-12-17 23:08:44,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923d575bc060ce160a48 HTTP/1.1" 502 150 2025-12-17 23:08:44,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940923d575bc060ce160a48'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:44,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409245575bc060ce160a60 2025-12-17 23:08:44,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409245575bc060ce160a60 HTTP/1.1" 502 150 2025-12-17 23:08:44,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409245575bc060ce160a60'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:45,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69410fd9460bc8e265117ad5 2025-12-17 23:08:45,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69410fd9460bc8e265117ad5 HTTP/1.1" 502 150 2025-12-17 23:08:45,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69410fd9460bc8e265117ad5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49302 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:08:46,609 - lava_callback - INFO - Received callback for node 694094ec575bc060ce161168 INFO: 172.21.0.1:49316 - "POST /node/694094ec575bc060ce161168 HTTP/1.0" 202 Accepted 2025-12-17 23:08:47,243 - lava_callback - INFO - Received callback for node 694094ef575bc060ce16116b INFO: 172.21.0.1:49318 - "POST /node/694094ef575bc060ce16116b HTTP/1.0" 202 Accepted 2025-12-17 23:08:47,267 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940923b575bc060ce160a45 2025-12-17 23:08:47,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923b575bc060ce160a45 HTTP/1.1" 502 150 2025-12-17 23:08:47,268 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940923b575bc060ce160a45'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:47,298 - lava_callback - INFO - Received callback for node 694094f7575bc060ce1611d9 INFO: 172.21.0.1:49330 - "POST /node/694094f7575bc060ce1611d9 HTTP/1.0" 202 Accepted 2025-12-17 23:08:47,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409222575bc060ce160a00 2025-12-17 23:08:47,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409222575bc060ce160a00 HTTP/1.1" 502 150 2025-12-17 23:08:47,765 - lava_callback - ERROR - Error processing callback for node 69409222575bc060ce160a00: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409222575bc060ce160a00 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409222575bc060ce160a00 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409222575bc060ce160a00 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:47,767 - lava_callback - INFO - Starting async job submit for node 69409247575bc060ce160a63 2025-12-17 23:08:47,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:47,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409247575bc060ce160a63 HTTP/1.1" 502 150 2025-12-17 23:08:47,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409247575bc060ce160a63'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:47,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409247575bc060ce160a63 2025-12-17 23:08:47,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409247575bc060ce160a63 HTTP/1.1" 502 150 2025-12-17 23:08:47,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409247575bc060ce160a63'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:47,911 - lava_callback - INFO - Received callback for node 694094f1575bc060ce16116e INFO: 172.21.0.1:49346 - "POST /node/694094f1575bc060ce16116e HTTP/1.0" 202 Accepted 2025-12-17 23:08:48,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409243575bc060ce160a5d 2025-12-17 23:08:48,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409243575bc060ce160a5d HTTP/1.1" 502 150 2025-12-17 23:08:48,495 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409243575bc060ce160a5d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:48,550 - lava_callback - INFO - Received callback for node 694094f3575bc060ce161171 INFO: 172.21.0.1:49348 - "POST /node/694094f3575bc060ce161171 HTTP/1.0" 202 Accepted 2025-12-17 23:08:48,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409230575bc060ce160a28 2025-12-17 23:08:48,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409230575bc060ce160a28 HTTP/1.1" 502 150 2025-12-17 23:08:48,859 - lava_callback - ERROR - Error processing callback for node 69409230575bc060ce160a28: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409230575bc060ce160a28 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409230575bc060ce160a28 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409230575bc060ce160a28 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:48,861 - lava_callback - INFO - Starting async job submit for node 69409249575bc060ce160a66 2025-12-17 23:08:48,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:48,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409249575bc060ce160a66 HTTP/1.1" 502 150 2025-12-17 23:08:48,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409249575bc060ce160a66'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:48,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409249575bc060ce160a66 2025-12-17 23:08:48,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409249575bc060ce160a66 HTTP/1.1" 502 150 2025-12-17 23:08:48,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409249575bc060ce160a66'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:48,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409245575bc060ce160a60 2025-12-17 23:08:48,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409245575bc060ce160a60 HTTP/1.1" 502 150 2025-12-17 23:08:48,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409245575bc060ce160a60'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:49,202 - lava_callback - INFO - Received callback for node 694094f5575bc060ce1611d6 INFO: 172.21.0.1:49350 - "POST /node/694094f5575bc060ce1611d6 HTTP/1.0" 202 Accepted 2025-12-17 23:08:49,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409247575bc060ce160a63 2025-12-17 23:08:49,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409247575bc060ce160a63 HTTP/1.1" 502 150 2025-12-17 23:08:49,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409247575bc060ce160a63'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:49,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69410fd9460bc8e265117ad5 2025-12-17 23:08:49,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69410fd9460bc8e265117ad5 HTTP/1.1" 502 150 2025-12-17 23:08:49,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69410fd9460bc8e265117ad5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:50,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409223575bc060ce160a03 2025-12-17 23:08:50,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409223575bc060ce160a03 HTTP/1.1" 502 150 2025-12-17 23:08:50,675 - lava_callback - ERROR - Error processing callback for node 69409223575bc060ce160a03: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409223575bc060ce160a03 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409223575bc060ce160a03 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409223575bc060ce160a03 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:50,677 - lava_callback - INFO - Starting async job submit for node 6940955d575bc060ce1612e5 2025-12-17 23:08:50,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:50,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955d575bc060ce1612e5 HTTP/1.1" 502 150 2025-12-17 23:08:50,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955d575bc060ce1612e5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:50,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955d575bc060ce1612e5 2025-12-17 23:08:50,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955d575bc060ce1612e5 HTTP/1.1" 502 150 2025-12-17 23:08:50,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955d575bc060ce1612e5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:50,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409249575bc060ce160a66 2025-12-17 23:08:50,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409249575bc060ce160a66 HTTP/1.1" 502 150 2025-12-17 23:08:50,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409249575bc060ce160a66'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:51,653 - lava_callback - INFO - Received callback for node 69411958459ff5fc64d2a0ce INFO: 172.21.0.1:45340 - "POST /node/69411958459ff5fc64d2a0ce HTTP/1.0" 202 Accepted 2025-12-17 23:08:52,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936b575bc060ce160d6c HTTP/1.1" 502 150 2025-12-17 23:08:52,528 - lava_callback - ERROR - Error processing callback for node 6940936b575bc060ce160d6c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940936b575bc060ce160d6c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940936b575bc060ce160d6c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940936b575bc060ce160d6c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:52,530 - lava_callback - INFO - Starting async job submit for node 69409255575bc060ce160a7c 2025-12-17 23:08:52,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:52,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409255575bc060ce160a7c HTTP/1.1" 502 150 2025-12-17 23:08:52,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409255575bc060ce160a7c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:52,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409255575bc060ce160a7c 2025-12-17 23:08:52,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409255575bc060ce160a7c HTTP/1.1" 502 150 2025-12-17 23:08:52,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409255575bc060ce160a7c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:52,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955d575bc060ce1612e5 2025-12-17 23:08:52,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955d575bc060ce1612e5 HTTP/1.1" 502 150 2025-12-17 23:08:52,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955d575bc060ce1612e5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:52,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409232575bc060ce160a34 2025-12-17 23:08:52,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409232575bc060ce160a34 HTTP/1.1" 502 150 2025-12-17 23:08:52,805 - lava_callback - ERROR - Error processing callback for node 69409232575bc060ce160a34: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409232575bc060ce160a34 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409232575bc060ce160a34 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409232575bc060ce160a34 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:52,807 - lava_callback - INFO - Starting async job submit for node 69409257575bc060ce160a86 2025-12-17 23:08:52,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:52,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409257575bc060ce160a86 HTTP/1.1" 502 150 2025-12-17 23:08:52,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409257575bc060ce160a86'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:52,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409257575bc060ce160a86 2025-12-17 23:08:52,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409257575bc060ce160a86 HTTP/1.1" 502 150 2025-12-17 23:08:52,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409257575bc060ce160a86'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:52,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940923d575bc060ce160a48 2025-12-17 23:08:52,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923d575bc060ce160a48 HTTP/1.1" 502 150 2025-12-17 23:08:52,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940923d575bc060ce160a48'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:53,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409247575bc060ce160a63 2025-12-17 23:08:53,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409247575bc060ce160a63 HTTP/1.1" 502 150 2025-12-17 23:08:53,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409247575bc060ce160a63'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:54,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409255575bc060ce160a7c 2025-12-17 23:08:54,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409255575bc060ce160a7c HTTP/1.1" 502 150 2025-12-17 23:08:54,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409255575bc060ce160a7c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:54,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409257575bc060ce160a86 2025-12-17 23:08:54,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409257575bc060ce160a86 HTTP/1.1" 502 150 2025-12-17 23:08:54,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409257575bc060ce160a86'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:54,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409235575bc060ce160a38 2025-12-17 23:08:54,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409235575bc060ce160a38 HTTP/1.1" 502 150 2025-12-17 23:08:54,854 - lava_callback - ERROR - Error processing callback for node 69409235575bc060ce160a38: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409235575bc060ce160a38 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409235575bc060ce160a38 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409235575bc060ce160a38 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:54,856 - lava_callback - INFO - Starting async job submit for node 69409259575bc060ce160a8a 2025-12-17 23:08:54,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:54,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409259575bc060ce160a8a HTTP/1.1" 502 150 2025-12-17 23:08:54,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409259575bc060ce160a8a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:54,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409259575bc060ce160a8a 2025-12-17 23:08:54,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409259575bc060ce160a8a HTTP/1.1" 502 150 2025-12-17 23:08:54,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409259575bc060ce160a8a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:54,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409249575bc060ce160a66 2025-12-17 23:08:54,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409249575bc060ce160a66 HTTP/1.1" 502 150 2025-12-17 23:08:54,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409249575bc060ce160a66'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:55,090 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409237575bc060ce160a3f 2025-12-17 23:08:55,090 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940923f575bc060ce160a4b 2025-12-17 23:08:55,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409237575bc060ce160a3f HTTP/1.1" 502 150 2025-12-17 23:08:55,091 - lava_callback - ERROR - Error processing callback for node 69409237575bc060ce160a3f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409237575bc060ce160a3f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409237575bc060ce160a3f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409237575bc060ce160a3f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:55,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923f575bc060ce160a4b HTTP/1.1" 502 150 2025-12-17 23:08:55,093 - lava_callback - INFO - Starting async job submit for node 6940925a575bc060ce160a8d 2025-12-17 23:08:55,093 - lava_callback - ERROR - Error processing callback for node 6940923f575bc060ce160a4b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940923f575bc060ce160a4b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940923f575bc060ce160a4b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940923f575bc060ce160a4b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:55,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:55,096 - lava_callback - INFO - Starting async job submit for node 6940925b575bc060ce160a90 2025-12-17 23:08:55,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:55,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925a575bc060ce160a8d HTTP/1.1" 502 150 2025-12-17 23:08:55,104 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925a575bc060ce160a8d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:55,104 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925a575bc060ce160a8d 2025-12-17 23:08:55,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925a575bc060ce160a8d HTTP/1.1" 502 150 2025-12-17 23:08:55,105 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925a575bc060ce160a8d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:55,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925b575bc060ce160a90 HTTP/1.1" 502 150 2025-12-17 23:08:55,130 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925b575bc060ce160a90'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:55,130 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925b575bc060ce160a90 2025-12-17 23:08:55,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925b575bc060ce160a90 HTTP/1.1" 502 150 2025-12-17 23:08:55,131 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925b575bc060ce160a90'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:55,268 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940923b575bc060ce160a45 2025-12-17 23:08:55,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923b575bc060ce160a45 HTTP/1.1" 502 150 2025-12-17 23:08:55,269 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940923b575bc060ce160a45'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:55,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409227575bc060ce160a09 2025-12-17 23:08:55,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409227575bc060ce160a09 HTTP/1.1" 502 150 2025-12-17 23:08:55,539 - lava_callback - ERROR - Error processing callback for node 69409227575bc060ce160a09: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409227575bc060ce160a09 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409227575bc060ce160a09 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409227575bc060ce160a09 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:55,541 - lava_callback - INFO - Starting async job submit for node 6940925c575bc060ce160a93 2025-12-17 23:08:55,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:55,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925c575bc060ce160a93 HTTP/1.1" 502 150 2025-12-17 23:08:55,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925c575bc060ce160a93'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:55,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925c575bc060ce160a93 2025-12-17 23:08:55,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925c575bc060ce160a93 HTTP/1.1" 502 150 2025-12-17 23:08:55,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925c575bc060ce160a93'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:55,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409239575bc060ce160a42 2025-12-17 23:08:55,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409239575bc060ce160a42 HTTP/1.1" 502 150 2025-12-17 23:08:55,869 - lava_callback - ERROR - Error processing callback for node 69409239575bc060ce160a42: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409239575bc060ce160a42 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409239575bc060ce160a42 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409239575bc060ce160a42 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:08:55,871 - lava_callback - INFO - Starting async job submit for node 6940925e575bc060ce160a96 2025-12-17 23:08:55,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:08:55,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925e575bc060ce160a96 HTTP/1.1" 502 150 2025-12-17 23:08:55,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925e575bc060ce160a96'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:55,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925e575bc060ce160a96 2025-12-17 23:08:55,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925e575bc060ce160a96 HTTP/1.1" 502 150 2025-12-17 23:08:55,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925e575bc060ce160a96'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:56,155 - lava_callback - INFO - Received callback for node 694094f8575bc060ce1611dc INFO: 172.21.0.1:45354 - "POST /node/694094f8575bc060ce1611dc HTTP/1.0" 202 Accepted 2025-12-17 23:08:56,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409243575bc060ce160a5d 2025-12-17 23:08:56,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409243575bc060ce160a5d HTTP/1.1" 502 150 2025-12-17 23:08:56,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409243575bc060ce160a5d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45366 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:08:56,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955d575bc060ce1612e5 2025-12-17 23:08:56,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955d575bc060ce1612e5 HTTP/1.1" 502 150 2025-12-17 23:08:56,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955d575bc060ce1612e5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:56,816 - lava_callback - INFO - Received callback for node 694094fa575bc060ce1611df INFO: 172.21.0.1:45378 - "POST /node/694094fa575bc060ce1611df HTTP/1.0" 202 Accepted 2025-12-17 23:08:56,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409259575bc060ce160a8a 2025-12-17 23:08:56,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409259575bc060ce160a8a HTTP/1.1" 502 150 2025-12-17 23:08:56,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409259575bc060ce160a8a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:56,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409245575bc060ce160a60 2025-12-17 23:08:56,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409245575bc060ce160a60 HTTP/1.1" 502 150 2025-12-17 23:08:56,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409245575bc060ce160a60'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:57,105 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925a575bc060ce160a8d 2025-12-17 23:08:57,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925a575bc060ce160a8d HTTP/1.1" 502 150 2025-12-17 23:08:57,106 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925a575bc060ce160a8d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:57,131 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925b575bc060ce160a90 2025-12-17 23:08:57,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925b575bc060ce160a90 HTTP/1.1" 502 150 2025-12-17 23:08:57,132 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925b575bc060ce160a90'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:57,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409241575bc060ce160a54 HTTP/1.1" 502 150 2025-12-17 23:08:57,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409241575bc060ce160a54'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:57,471 - lava_callback - INFO - Received callback for node 694094fc575bc060ce1611e2 INFO: 172.21.0.1:45384 - "POST /node/694094fc575bc060ce1611e2 HTTP/1.0" 202 Accepted 2025-12-17 23:08:57,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925c575bc060ce160a93 2025-12-17 23:08:57,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925c575bc060ce160a93 HTTP/1.1" 502 150 2025-12-17 23:08:57,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925c575bc060ce160a93'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:57,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925e575bc060ce160a96 2025-12-17 23:08:57,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925e575bc060ce160a96 HTTP/1.1" 502 150 2025-12-17 23:08:57,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925e575bc060ce160a96'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:57,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69410fd9460bc8e265117ad5 2025-12-17 23:08:57,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69410fd9460bc8e265117ad5 HTTP/1.1" 502 150 2025-12-17 23:08:57,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69410fd9460bc8e265117ad5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:58,161 - lava_callback - INFO - Received callback for node 69409500575bc060ce1611f5 INFO: 172.21.0.1:45394 - "POST /node/69409500575bc060ce1611f5 HTTP/1.0" 202 Accepted 2025-12-17 23:08:58,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409255575bc060ce160a7c 2025-12-17 23:08:58,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409255575bc060ce160a7c HTTP/1.1" 502 150 2025-12-17 23:08:58,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409255575bc060ce160a7c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:58,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409257575bc060ce160a86 2025-12-17 23:08:58,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409257575bc060ce160a86 HTTP/1.1" 502 150 2025-12-17 23:08:58,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409257575bc060ce160a86'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:58,842 - lava_callback - INFO - Received callback for node 69409501575bc060ce1611fa INFO: 172.21.0.1:45402 - "POST /node/69409501575bc060ce1611fa HTTP/1.0" 202 Accepted 2025-12-17 23:08:59,393 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409241575bc060ce160a54 2025-12-17 23:08:59,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409241575bc060ce160a54 HTTP/1.1" 502 150 2025-12-17 23:08:59,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409241575bc060ce160a54'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:08:59,520 - lava_callback - INFO - Received callback for node 69409503575bc060ce161204 INFO: 172.21.0.1:45416 - "POST /node/69409503575bc060ce161204 HTTP/1.0" 202 Accepted 2025-12-17 23:09:00,160 - lava_callback - INFO - Received callback for node 69409506575bc060ce16120a INFO: 172.21.0.1:41620 - "POST /node/69409506575bc060ce16120a HTTP/1.0" 202 Accepted 2025-12-17 23:09:00,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409259575bc060ce160a8a 2025-12-17 23:09:00,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409259575bc060ce160a8a HTTP/1.1" 502 150 2025-12-17 23:09:00,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409259575bc060ce160a8a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:01,106 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925a575bc060ce160a8d 2025-12-17 23:09:01,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925a575bc060ce160a8d HTTP/1.1" 502 150 2025-12-17 23:09:01,108 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925a575bc060ce160a8d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:01,132 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925b575bc060ce160a90 2025-12-17 23:09:01,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925b575bc060ce160a90 HTTP/1.1" 502 150 2025-12-17 23:09:01,133 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925b575bc060ce160a90'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:01,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925c575bc060ce160a93 2025-12-17 23:09:01,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925c575bc060ce160a93 HTTP/1.1" 502 150 2025-12-17 23:09:01,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925c575bc060ce160a93'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:01,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409247575bc060ce160a63 2025-12-17 23:09:01,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409247575bc060ce160a63 HTTP/1.1" 502 150 2025-12-17 23:09:01,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409247575bc060ce160a63'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:01,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925e575bc060ce160a96 2025-12-17 23:09:01,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925e575bc060ce160a96 HTTP/1.1" 502 150 2025-12-17 23:09:01,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925e575bc060ce160a96'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:02,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409249575bc060ce160a66 2025-12-17 23:09:02,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409249575bc060ce160a66 HTTP/1.1" 502 150 2025-12-17 23:09:02,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409249575bc060ce160a66'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41636 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:09:03,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409241575bc060ce160a54 2025-12-17 23:09:04,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955d575bc060ce1612e5 2025-12-17 23:09:04,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955d575bc060ce1612e5 HTTP/1.1" 502 150 2025-12-17 23:09:04,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955d575bc060ce1612e5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41646 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:09:06,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409255575bc060ce160a7c 2025-12-17 23:09:06,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409255575bc060ce160a7c HTTP/1.1" 502 150 2025-12-17 23:09:06,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409255575bc060ce160a7c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:06,605 - lava_callback - INFO - Received callback for node 694094fe575bc060ce1611e9 INFO: 172.21.0.1:41648 - "POST /node/694094fe575bc060ce1611e9 HTTP/1.0" 202 Accepted 2025-12-17 23:09:06,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409257575bc060ce160a86 2025-12-17 23:09:06,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409257575bc060ce160a86 HTTP/1.1" 502 150 2025-12-17 23:09:06,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409257575bc060ce160a86'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:07,260 - lava_callback - INFO - Received callback for node 69409509575bc060ce161216 INFO: 172.21.0.1:41662 - "POST /node/69409509575bc060ce161216 HTTP/1.0" 202 Accepted 2025-12-17 23:09:07,883 - lava_callback - INFO - Received callback for node 6940950b575bc060ce16121b INFO: 172.21.0.1:41668 - "POST /node/6940950b575bc060ce16121b HTTP/1.0" 202 Accepted 2025-12-17 23:09:08,509 - lava_callback - INFO - Received callback for node 6940950d575bc060ce161221 INFO: 172.21.0.1:41684 - "POST /node/6940950d575bc060ce161221 HTTP/1.0" 202 Accepted 2025-12-17 23:09:08,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940923d575bc060ce160a48 2025-12-17 23:09:08,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923d575bc060ce160a48 HTTP/1.1" 502 150 2025-12-17 23:09:08,855 - lava_callback - ERROR - Error processing callback for node 6940923d575bc060ce160a48: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940923d575bc060ce160a48 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940923d575bc060ce160a48 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940923d575bc060ce160a48 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:08,858 - lava_callback - INFO - Starting async job submit for node 69409260575bc060ce160a99 2025-12-17 23:09:08,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:08,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409259575bc060ce160a8a 2025-12-17 23:09:08,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409260575bc060ce160a99 HTTP/1.1" 502 150 2025-12-17 23:09:08,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409260575bc060ce160a99'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:08,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409260575bc060ce160a99 2025-12-17 23:09:08,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409259575bc060ce160a8a HTTP/1.1" 502 150 2025-12-17 23:09:08,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409259575bc060ce160a8a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:08,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409260575bc060ce160a99 HTTP/1.1" 502 150 2025-12-17 23:09:08,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409260575bc060ce160a99'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:09,108 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925a575bc060ce160a8d 2025-12-17 23:09:09,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925a575bc060ce160a8d HTTP/1.1" 502 150 2025-12-17 23:09:09,109 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925a575bc060ce160a8d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:09,134 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925b575bc060ce160a90 2025-12-17 23:09:09,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925b575bc060ce160a90 HTTP/1.1" 502 150 2025-12-17 23:09:09,134 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925b575bc060ce160a90'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:09,152 - lava_callback - INFO - Received callback for node 69409510575bc060ce161227 INFO: 172.21.0.1:41688 - "POST /node/69409510575bc060ce161227 HTTP/1.0" 202 Accepted 2025-12-17 23:09:09,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925c575bc060ce160a93 2025-12-17 23:09:09,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925c575bc060ce160a93 HTTP/1.1" 502 150 2025-12-17 23:09:09,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925c575bc060ce160a93'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:09,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925e575bc060ce160a96 2025-12-17 23:09:09,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925e575bc060ce160a96 HTTP/1.1" 502 150 2025-12-17 23:09:09,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940925e575bc060ce160a96'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:10,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409260575bc060ce160a99 2025-12-17 23:09:10,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409260575bc060ce160a99 HTTP/1.1" 502 150 2025-12-17 23:09:10,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409260575bc060ce160a99'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:11,269 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940923b575bc060ce160a45 2025-12-17 23:09:11,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940923b575bc060ce160a45 HTTP/1.1" 502 150 2025-12-17 23:09:11,271 - lava_callback - ERROR - Error processing callback for node 6940923b575bc060ce160a45: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940923b575bc060ce160a45 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940923b575bc060ce160a45 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940923b575bc060ce160a45 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:11,273 - lava_callback - INFO - Starting async job submit for node 69409262575bc060ce160a9c 2025-12-17 23:09:11,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:11,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409262575bc060ce160a9c HTTP/1.1" 502 150 2025-12-17 23:09:11,282 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409262575bc060ce160a9c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:11,283 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409262575bc060ce160a9c 2025-12-17 23:09:11,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409262575bc060ce160a9c HTTP/1.1" 502 150 2025-12-17 23:09:11,283 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409262575bc060ce160a9c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:11,320 - lava_callback - INFO - Received callback for node 69409508575bc060ce161210 INFO: 172.21.0.1:43842 - "POST /node/69409508575bc060ce161210 HTTP/1.0" 202 Accepted 2025-12-17 23:09:12,001 - lava_callback - INFO - Received callback for node 69409511575bc060ce16122a INFO: 172.21.0.1:43856 - "POST /node/69409511575bc060ce16122a HTTP/1.0" 202 Accepted 2025-12-17 23:09:12,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409243575bc060ce160a5d 2025-12-17 23:09:12,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409243575bc060ce160a5d HTTP/1.1" 502 150 2025-12-17 23:09:12,498 - lava_callback - ERROR - Error processing callback for node 69409243575bc060ce160a5d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409243575bc060ce160a5d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409243575bc060ce160a5d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409243575bc060ce160a5d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:12,501 - lava_callback - INFO - Starting async job submit for node 69409265575bc060ce160a9f 2025-12-17 23:09:12,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:12,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409265575bc060ce160a9f HTTP/1.1" 502 150 2025-12-17 23:09:12,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409265575bc060ce160a9f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:12,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409265575bc060ce160a9f 2025-12-17 23:09:12,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409265575bc060ce160a9f HTTP/1.1" 502 150 2025-12-17 23:09:12,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409265575bc060ce160a9f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:12,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409245575bc060ce160a60 2025-12-17 23:09:12,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409245575bc060ce160a60 HTTP/1.1" 502 150 2025-12-17 23:09:12,920 - lava_callback - ERROR - Error processing callback for node 69409245575bc060ce160a60: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409245575bc060ce160a60 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409245575bc060ce160a60 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409245575bc060ce160a60 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:12,923 - lava_callback - INFO - Starting async job submit for node 69409267575bc060ce160aa2 2025-12-17 23:09:12,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:12,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409267575bc060ce160aa2 HTTP/1.1" 502 150 2025-12-17 23:09:12,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409267575bc060ce160aa2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:12,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409267575bc060ce160aa2 2025-12-17 23:09:12,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409267575bc060ce160aa2 HTTP/1.1" 502 150 2025-12-17 23:09:12,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409267575bc060ce160aa2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:13,284 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409262575bc060ce160a9c 2025-12-17 23:09:13,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409262575bc060ce160a9c HTTP/1.1" 502 150 2025-12-17 23:09:13,285 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409262575bc060ce160a9c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:13,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69410fd9460bc8e265117ad5 2025-12-17 23:09:13,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69410fd9460bc8e265117ad5 HTTP/1.1" 502 150 2025-12-17 23:09:13,985 - lava_callback - ERROR - Error processing callback for node 69410fd9460bc8e265117ad5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69410fd9460bc8e265117ad5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69410fd9460bc8e265117ad5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69410fd9460bc8e265117ad5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:13,987 - lava_callback - INFO - Starting async job submit for node 69409269575bc060ce160aa7 2025-12-17 23:09:13,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:14,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409269575bc060ce160aa7 HTTP/1.1" 502 150 2025-12-17 23:09:14,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409269575bc060ce160aa7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:14,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409269575bc060ce160aa7 2025-12-17 23:09:14,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409269575bc060ce160aa7 HTTP/1.1" 502 150 2025-12-17 23:09:14,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409269575bc060ce160aa7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:14,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409265575bc060ce160a9f 2025-12-17 23:09:14,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409265575bc060ce160a9f HTTP/1.1" 502 150 2025-12-17 23:09:14,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409265575bc060ce160a9f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:14,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409260575bc060ce160a99 2025-12-17 23:09:14,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409260575bc060ce160a99 HTTP/1.1" 502 150 2025-12-17 23:09:14,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409260575bc060ce160a99'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:14,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409267575bc060ce160aa2 2025-12-17 23:09:14,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409267575bc060ce160aa2 HTTP/1.1" 502 150 2025-12-17 23:09:14,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409267575bc060ce160aa2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43864 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:09:16,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409269575bc060ce160aa7 2025-12-17 23:09:16,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409269575bc060ce160aa7 HTTP/1.1" 502 150 2025-12-17 23:09:16,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409269575bc060ce160aa7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:16,123 - lava_callback - INFO - Received callback for node 6940950c575bc060ce16121e INFO: 172.21.0.1:43872 - "POST /node/6940950c575bc060ce16121e HTTP/1.0" 202 Accepted INFO: 172.21.0.1:43880 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:09:16,781 - lava_callback - INFO - Received callback for node 6940950e575bc060ce161224 INFO: 172.21.0.1:43882 - "POST /node/6940950e575bc060ce161224 HTTP/1.0" 202 Accepted 2025-12-17 23:09:17,285 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409262575bc060ce160a9c 2025-12-17 23:09:17,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409262575bc060ce160a9c HTTP/1.1" 502 150 2025-12-17 23:09:17,287 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409262575bc060ce160a9c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:17,447 - lava_callback - INFO - Received callback for node 6940951a575bc060ce161236 INFO: 172.21.0.1:43890 - "POST /node/6940951a575bc060ce161236 HTTP/1.0" 202 Accepted 2025-12-17 23:09:17,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409247575bc060ce160a63 2025-12-17 23:09:17,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409247575bc060ce160a63 HTTP/1.1" 502 150 2025-12-17 23:09:17,780 - lava_callback - ERROR - Error processing callback for node 69409247575bc060ce160a63: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409247575bc060ce160a63 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409247575bc060ce160a63 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409247575bc060ce160a63 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:17,782 - lava_callback - INFO - Starting async job submit for node 6940926b575bc060ce160aaa 2025-12-17 23:09:17,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:17,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926b575bc060ce160aaa HTTP/1.1" 502 150 2025-12-17 23:09:17,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940926b575bc060ce160aaa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:17,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940926b575bc060ce160aaa 2025-12-17 23:09:17,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926b575bc060ce160aaa HTTP/1.1" 502 150 2025-12-17 23:09:17,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940926b575bc060ce160aaa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:18,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409265575bc060ce160a9f 2025-12-17 23:09:18,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409265575bc060ce160a9f HTTP/1.1" 502 150 2025-12-17 23:09:18,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409265575bc060ce160a9f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:18,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409241575bc060ce160a54 HTTP/1.1" 502 150 2025-12-17 23:09:18,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409241575bc060ce160a54'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:18,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409249575bc060ce160a66 2025-12-17 23:09:18,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409249575bc060ce160a66 HTTP/1.1" 502 150 2025-12-17 23:09:18,880 - lava_callback - ERROR - Error processing callback for node 69409249575bc060ce160a66: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409249575bc060ce160a66 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409249575bc060ce160a66 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409249575bc060ce160a66 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:18,882 - lava_callback - INFO - Starting async job submit for node 6940926c575bc060ce160aad 2025-12-17 23:09:18,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:18,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926c575bc060ce160aad HTTP/1.1" 502 150 2025-12-17 23:09:18,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940926c575bc060ce160aad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:18,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940926c575bc060ce160aad 2025-12-17 23:09:18,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926c575bc060ce160aad HTTP/1.1" 502 150 2025-12-17 23:09:18,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940926c575bc060ce160aad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:18,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409267575bc060ce160aa2 2025-12-17 23:09:18,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409267575bc060ce160aa2 HTTP/1.1" 502 150 2025-12-17 23:09:18,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409267575bc060ce160aa2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:19,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940926b575bc060ce160aaa 2025-12-17 23:09:19,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926b575bc060ce160aaa HTTP/1.1" 502 150 2025-12-17 23:09:19,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940926b575bc060ce160aaa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:20,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409269575bc060ce160aa7 2025-12-17 23:09:20,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409269575bc060ce160aa7 HTTP/1.1" 502 150 2025-12-17 23:09:20,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409269575bc060ce160aa7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:20,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955d575bc060ce1612e5 2025-12-17 23:09:20,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955d575bc060ce1612e5 HTTP/1.1" 502 150 2025-12-17 23:09:20,717 - lava_callback - ERROR - Error processing callback for node 6940955d575bc060ce1612e5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940955d575bc060ce1612e5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940955d575bc060ce1612e5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940955d575bc060ce1612e5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:20,719 - lava_callback - INFO - Starting async job submit for node 6940926f575bc060ce160ab3 2025-12-17 23:09:20,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:20,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926f575bc060ce160ab3 HTTP/1.1" 502 150 2025-12-17 23:09:20,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940926f575bc060ce160ab3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:20,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940926f575bc060ce160ab3 2025-12-17 23:09:20,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926f575bc060ce160ab3 HTTP/1.1" 502 150 2025-12-17 23:09:20,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940926f575bc060ce160ab3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:20,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940926c575bc060ce160aad 2025-12-17 23:09:20,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926c575bc060ce160aad HTTP/1.1" 502 150 2025-12-17 23:09:20,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940926c575bc060ce160aad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:22,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409255575bc060ce160a7c 2025-12-17 23:09:22,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940926f575bc060ce160ab3 2025-12-17 23:09:22,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926f575bc060ce160ab3 HTTP/1.1" 502 150 2025-12-17 23:09:22,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940926f575bc060ce160ab3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:22,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409257575bc060ce160a86 2025-12-17 23:09:22,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409257575bc060ce160a86 HTTP/1.1" 502 150 2025-12-17 23:09:22,821 - lava_callback - ERROR - Error processing callback for node 69409257575bc060ce160a86: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409257575bc060ce160a86 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409257575bc060ce160a86 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409257575bc060ce160a86 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:22,823 - lava_callback - INFO - Starting async job submit for node 69409270575bc060ce160ab6 2025-12-17 23:09:22,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:22,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409270575bc060ce160ab6 HTTP/1.1" 502 150 2025-12-17 23:09:22,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409270575bc060ce160ab6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:22,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409270575bc060ce160ab6 2025-12-17 23:09:22,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409270575bc060ce160ab6 HTTP/1.1" 502 150 2025-12-17 23:09:22,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409270575bc060ce160ab6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:22,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409260575bc060ce160a99 2025-12-17 23:09:22,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409260575bc060ce160a99 HTTP/1.1" 502 150 2025-12-17 23:09:22,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409260575bc060ce160a99'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:23,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940926b575bc060ce160aaa 2025-12-17 23:09:23,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926b575bc060ce160aaa HTTP/1.1" 502 150 2025-12-17 23:09:23,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940926b575bc060ce160aaa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:24,260 - lava_callback - INFO - Received callback for node 69409543575bc060ce16129f INFO: 172.21.0.1:33598 - "POST /node/69409543575bc060ce16129f HTTP/1.0" 202 Accepted 2025-12-17 23:09:24,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409270575bc060ce160ab6 2025-12-17 23:09:24,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409270575bc060ce160ab6 HTTP/1.1" 502 150 2025-12-17 23:09:24,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409270575bc060ce160ab6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:24,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409259575bc060ce160a8a 2025-12-17 23:09:24,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409259575bc060ce160a8a HTTP/1.1" 502 150 2025-12-17 23:09:24,869 - lava_callback - ERROR - Error processing callback for node 69409259575bc060ce160a8a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409259575bc060ce160a8a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409259575bc060ce160a8a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409259575bc060ce160a8a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:24,871 - lava_callback - INFO - Starting async job submit for node 69409272575bc060ce160ab9 2025-12-17 23:09:24,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:24,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409272575bc060ce160ab9 HTTP/1.1" 502 150 2025-12-17 23:09:24,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409272575bc060ce160ab9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:24,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409272575bc060ce160ab9 2025-12-17 23:09:24,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409272575bc060ce160ab9 HTTP/1.1" 502 150 2025-12-17 23:09:24,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409272575bc060ce160ab9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:24,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940926c575bc060ce160aad 2025-12-17 23:09:24,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926c575bc060ce160aad HTTP/1.1" 502 150 2025-12-17 23:09:24,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940926c575bc060ce160aad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:25,109 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925a575bc060ce160a8d 2025-12-17 23:09:25,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925a575bc060ce160a8d HTTP/1.1" 502 150 2025-12-17 23:09:25,112 - lava_callback - ERROR - Error processing callback for node 6940925a575bc060ce160a8d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940925a575bc060ce160a8d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940925a575bc060ce160a8d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940925a575bc060ce160a8d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:25,114 - lava_callback - INFO - Starting async job submit for node 69409277575bc060ce160ac2 2025-12-17 23:09:25,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:25,135 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925b575bc060ce160a90 2025-12-17 23:09:25,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925b575bc060ce160a90 HTTP/1.1" 502 150 2025-12-17 23:09:25,136 - lava_callback - ERROR - Error processing callback for node 6940925b575bc060ce160a90: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940925b575bc060ce160a90 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940925b575bc060ce160a90 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940925b575bc060ce160a90 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:25,137 - lava_callback - INFO - Starting async job submit for node 69409274575bc060ce160abc 2025-12-17 23:09:25,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:25,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409277575bc060ce160ac2 HTTP/1.1" 502 150 2025-12-17 23:09:25,142 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409277575bc060ce160ac2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:25,142 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409277575bc060ce160ac2 2025-12-17 23:09:25,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409277575bc060ce160ac2 HTTP/1.1" 502 150 2025-12-17 23:09:25,143 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409277575bc060ce160ac2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:25,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409274575bc060ce160abc HTTP/1.1" 502 150 2025-12-17 23:09:25,144 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409274575bc060ce160abc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:25,144 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409274575bc060ce160abc 2025-12-17 23:09:25,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409274575bc060ce160abc HTTP/1.1" 502 150 2025-12-17 23:09:25,144 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409274575bc060ce160abc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:25,287 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409262575bc060ce160a9c 2025-12-17 23:09:25,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409262575bc060ce160a9c HTTP/1.1" 502 150 2025-12-17 23:09:25,288 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409262575bc060ce160a9c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:25,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925c575bc060ce160a93 2025-12-17 23:09:25,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925c575bc060ce160a93 HTTP/1.1" 502 150 2025-12-17 23:09:25,556 - lava_callback - ERROR - Error processing callback for node 6940925c575bc060ce160a93: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940925c575bc060ce160a93 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940925c575bc060ce160a93 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940925c575bc060ce160a93 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:25,557 - lava_callback - INFO - Starting async job submit for node 69409276575bc060ce160abf 2025-12-17 23:09:25,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:25,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409276575bc060ce160abf HTTP/1.1" 502 150 2025-12-17 23:09:25,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409276575bc060ce160abf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:25,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409276575bc060ce160abf 2025-12-17 23:09:25,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409276575bc060ce160abf HTTP/1.1" 502 150 2025-12-17 23:09:25,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409276575bc060ce160abf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:25,823 - lava_callback - INFO - Received callback for node 69409566575bc060ce161306 INFO: 172.21.0.1:33602 - "POST /node/69409566575bc060ce161306 HTTP/1.0" 202 Accepted 2025-12-17 23:09:25,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940925e575bc060ce160a96 2025-12-17 23:09:25,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940925e575bc060ce160a96 HTTP/1.1" 502 150 2025-12-17 23:09:25,892 - lava_callback - ERROR - Error processing callback for node 6940925e575bc060ce160a96: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940925e575bc060ce160a96 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940925e575bc060ce160a96 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940925e575bc060ce160a96 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:25,894 - lava_callback - INFO - Starting async job submit for node 69409279575bc060ce160ac5 2025-12-17 23:09:25,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:25,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409279575bc060ce160ac5 HTTP/1.1" 502 150 2025-12-17 23:09:25,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409279575bc060ce160ac5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:25,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409279575bc060ce160ac5 2025-12-17 23:09:25,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409279575bc060ce160ac5 HTTP/1.1" 502 150 2025-12-17 23:09:25,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409279575bc060ce160ac5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:26,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409265575bc060ce160a9f 2025-12-17 23:09:26,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409265575bc060ce160a9f HTTP/1.1" 502 150 2025-12-17 23:09:26,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409265575bc060ce160a9f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33614 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:09:26,619 - lava_callback - INFO - Received callback for node 69409516575bc060ce16122d INFO: 172.21.0.1:33618 - "POST /node/69409516575bc060ce16122d HTTP/1.0" 202 Accepted 2025-12-17 23:09:26,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409241575bc060ce160a54 2025-12-17 23:09:26,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409241575bc060ce160a54 HTTP/1.1" 502 150 2025-12-17 23:09:26,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409241575bc060ce160a54'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:26,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940926f575bc060ce160ab3 2025-12-17 23:09:26,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926f575bc060ce160ab3 HTTP/1.1" 502 150 2025-12-17 23:09:26,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940926f575bc060ce160ab3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:26,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409272575bc060ce160ab9 2025-12-17 23:09:26,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409272575bc060ce160ab9 HTTP/1.1" 502 150 2025-12-17 23:09:26,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409272575bc060ce160ab9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:26,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409267575bc060ce160aa2 2025-12-17 23:09:26,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409267575bc060ce160aa2 HTTP/1.1" 502 150 2025-12-17 23:09:26,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409267575bc060ce160aa2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:27,143 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409277575bc060ce160ac2 2025-12-17 23:09:27,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409277575bc060ce160ac2 HTTP/1.1" 502 150 2025-12-17 23:09:27,144 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409277575bc060ce160ac2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:27,145 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409274575bc060ce160abc 2025-12-17 23:09:27,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409274575bc060ce160abc HTTP/1.1" 502 150 2025-12-17 23:09:27,145 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409274575bc060ce160abc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:27,260 - lava_callback - INFO - Received callback for node 69409518575bc060ce161230 INFO: 172.21.0.1:33630 - "POST /node/69409518575bc060ce161230 HTTP/1.0" 202 Accepted 2025-12-17 23:09:27,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409276575bc060ce160abf 2025-12-17 23:09:27,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409276575bc060ce160abf HTTP/1.1" 502 150 2025-12-17 23:09:27,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409276575bc060ce160abf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:27,672 - lava_callback - INFO - Received callback for node 69409526575bc060ce161251 INFO: 172.21.0.1:33642 - "POST /node/69409526575bc060ce161251 HTTP/1.0" 202 Accepted 2025-12-17 23:09:27,903 - lava_callback - INFO - Received callback for node 6940951b575bc060ce161239 INFO: 172.21.0.1:33644 - "POST /node/6940951b575bc060ce161239 HTTP/1.0" 202 Accepted 2025-12-17 23:09:27,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409279575bc060ce160ac5 2025-12-17 23:09:27,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409279575bc060ce160ac5 HTTP/1.1" 502 150 2025-12-17 23:09:27,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409279575bc060ce160ac5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:28,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409269575bc060ce160aa7 2025-12-17 23:09:28,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409269575bc060ce160aa7 HTTP/1.1" 502 150 2025-12-17 23:09:28,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409269575bc060ce160aa7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:28,543 - lava_callback - INFO - Received callback for node 6940951e575bc060ce16123f INFO: 172.21.0.1:33648 - "POST /node/6940951e575bc060ce16123f HTTP/1.0" 202 Accepted 2025-12-17 23:09:28,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409270575bc060ce160ab6 2025-12-17 23:09:28,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409270575bc060ce160ab6 HTTP/1.1" 502 150 2025-12-17 23:09:28,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409270575bc060ce160ab6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:29,205 - lava_callback - INFO - Received callback for node 6940951f575bc060ce161242 INFO: 172.21.0.1:33654 - "POST /node/6940951f575bc060ce161242 HTTP/1.0" 202 Accepted 2025-12-17 23:09:29,860 - lava_callback - INFO - Received callback for node 69409520575bc060ce161245 INFO: 172.21.0.1:37980 - "POST /node/69409520575bc060ce161245 HTTP/1.0" 202 Accepted 2025-12-17 23:09:30,520 - lava_callback - INFO - Received callback for node 69409521575bc060ce161248 INFO: 172.21.0.1:37994 - "POST /node/69409521575bc060ce161248 HTTP/1.0" 202 Accepted 2025-12-17 23:09:30,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409272575bc060ce160ab9 2025-12-17 23:09:30,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409272575bc060ce160ab9 HTTP/1.1" 502 150 2025-12-17 23:09:30,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409272575bc060ce160ab9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:31,141 - lava_callback - INFO - Received callback for node 69409519575bc060ce161233 INFO: 172.21.0.1:38006 - "POST /node/69409519575bc060ce161233 HTTP/1.0" 202 Accepted 2025-12-17 23:09:31,146 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409277575bc060ce160ac2 2025-12-17 23:09:31,146 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409274575bc060ce160abc 2025-12-17 23:09:31,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409277575bc060ce160ac2 HTTP/1.1" 502 150 2025-12-17 23:09:31,149 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409277575bc060ce160ac2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:31,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409274575bc060ce160abc HTTP/1.1" 502 150 2025-12-17 23:09:31,152 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409274575bc060ce160abc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:31,166 - lava_callback - INFO - Received callback for node 69409523575bc060ce16124b INFO: 172.21.0.1:38018 - "POST /node/69409523575bc060ce16124b HTTP/1.0" 202 Accepted 2025-12-17 23:09:31,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409276575bc060ce160abf 2025-12-17 23:09:31,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409276575bc060ce160abf HTTP/1.1" 502 150 2025-12-17 23:09:31,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409276575bc060ce160abf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:31,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940926b575bc060ce160aaa 2025-12-17 23:09:31,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926b575bc060ce160aaa HTTP/1.1" 502 150 2025-12-17 23:09:31,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940926b575bc060ce160aaa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:31,800 - lava_callback - INFO - Received callback for node 69409524575bc060ce16124e INFO: 172.21.0.1:38026 - "POST /node/69409524575bc060ce16124e HTTP/1.0" 202 Accepted 2025-12-17 23:09:31,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409279575bc060ce160ac5 2025-12-17 23:09:31,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409279575bc060ce160ac5 HTTP/1.1" 502 150 2025-12-17 23:09:31,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409279575bc060ce160ac5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:32,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940926c575bc060ce160aad 2025-12-17 23:09:32,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926c575bc060ce160aad HTTP/1.1" 502 150 2025-12-17 23:09:32,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940926c575bc060ce160aad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:34,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940926f575bc060ce160ab3 2025-12-17 23:09:34,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926f575bc060ce160ab3 HTTP/1.1" 502 150 2025-12-17 23:09:34,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940926f575bc060ce160ab3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:36,202 - lava_callback - INFO - Received callback for node 6940951d575bc060ce16123c INFO: 172.21.0.1:38032 - "POST /node/6940951d575bc060ce16123c HTTP/1.0" 202 Accepted INFO: 172.21.0.1:38038 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:09:36,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409270575bc060ce160ab6 2025-12-17 23:09:36,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409270575bc060ce160ab6 HTTP/1.1" 502 150 2025-12-17 23:09:36,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409270575bc060ce160ab6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:36,847 - lava_callback - INFO - Received callback for node 69409527575bc060ce161254 INFO: 172.21.0.1:38048 - "POST /node/69409527575bc060ce161254 HTTP/1.0" 202 Accepted 2025-12-17 23:09:37,499 - lava_callback - INFO - Received callback for node 6940952a575bc060ce16125c INFO: 172.21.0.1:38050 - "POST /node/6940952a575bc060ce16125c HTTP/1.0" 202 Accepted 2025-12-17 23:09:37,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409255575bc060ce160a7c HTTP/1.1" 502 150 2025-12-17 23:09:37,841 - lava_callback - ERROR - Error processing callback for node 69409255575bc060ce160a7c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409255575bc060ce160a7c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409255575bc060ce160a7c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409255575bc060ce160a7c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:37,843 - lava_callback - INFO - Starting async job submit for node 6940927a575bc060ce160ac8 2025-12-17 23:09:37,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:37,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940927a575bc060ce160ac8 HTTP/1.1" 502 150 2025-12-17 23:09:37,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940927a575bc060ce160ac8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:37,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940927a575bc060ce160ac8 2025-12-17 23:09:37,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940927a575bc060ce160ac8 HTTP/1.1" 502 150 2025-12-17 23:09:37,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940927a575bc060ce160ac8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:38,185 - lava_callback - INFO - Received callback for node 6940952b575bc060ce16125f INFO: 172.21.0.1:38056 - "POST /node/6940952b575bc060ce16125f HTTP/1.0" 202 Accepted 2025-12-17 23:09:38,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409260575bc060ce160a99 2025-12-17 23:09:38,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409260575bc060ce160a99 HTTP/1.1" 502 150 2025-12-17 23:09:38,876 - lava_callback - ERROR - Error processing callback for node 69409260575bc060ce160a99: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409260575bc060ce160a99 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409260575bc060ce160a99 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409260575bc060ce160a99 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:38,878 - lava_callback - INFO - Starting async job submit for node 6940988e575bc060ce161ae3 2025-12-17 23:09:38,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409272575bc060ce160ab9 2025-12-17 23:09:38,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409272575bc060ce160ab9 HTTP/1.1" 502 150 2025-12-17 23:09:38,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409272575bc060ce160ab9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:38,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:38,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988e575bc060ce161ae3 HTTP/1.1" 502 150 2025-12-17 23:09:38,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988e575bc060ce161ae3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:38,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988e575bc060ce161ae3 2025-12-17 23:09:38,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988e575bc060ce161ae3 HTTP/1.1" 502 150 2025-12-17 23:09:38,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988e575bc060ce161ae3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:39,149 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409277575bc060ce160ac2 2025-12-17 23:09:39,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409277575bc060ce160ac2 HTTP/1.1" 502 150 2025-12-17 23:09:39,150 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409277575bc060ce160ac2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:39,152 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409274575bc060ce160abc 2025-12-17 23:09:39,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409274575bc060ce160abc HTTP/1.1" 502 150 2025-12-17 23:09:39,153 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409274575bc060ce160abc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:39,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409276575bc060ce160abf 2025-12-17 23:09:39,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409276575bc060ce160abf HTTP/1.1" 502 150 2025-12-17 23:09:39,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409276575bc060ce160abf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:39,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940927a575bc060ce160ac8 2025-12-17 23:09:39,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940927a575bc060ce160ac8 HTTP/1.1" 502 150 2025-12-17 23:09:39,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940927a575bc060ce160ac8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:39,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409279575bc060ce160ac5 2025-12-17 23:09:39,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409279575bc060ce160ac5 HTTP/1.1" 502 150 2025-12-17 23:09:39,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409279575bc060ce160ac5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:40,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988e575bc060ce161ae3 2025-12-17 23:09:40,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988e575bc060ce161ae3 HTTP/1.1" 502 150 2025-12-17 23:09:40,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988e575bc060ce161ae3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:41,288 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409262575bc060ce160a9c 2025-12-17 23:09:41,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409262575bc060ce160a9c HTTP/1.1" 502 150 2025-12-17 23:09:41,290 - lava_callback - ERROR - Error processing callback for node 69409262575bc060ce160a9c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409262575bc060ce160a9c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409262575bc060ce160a9c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409262575bc060ce160a9c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:41,293 - lava_callback - INFO - Starting async job submit for node 69409299575bc060ce160af9 2025-12-17 23:09:41,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:41,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409299575bc060ce160af9 HTTP/1.1" 502 150 2025-12-17 23:09:41,310 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409299575bc060ce160af9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:41,310 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409299575bc060ce160af9 2025-12-17 23:09:41,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409299575bc060ce160af9 HTTP/1.1" 502 150 2025-12-17 23:09:41,311 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409299575bc060ce160af9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:42,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409265575bc060ce160a9f 2025-12-17 23:09:42,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409265575bc060ce160a9f HTTP/1.1" 502 150 2025-12-17 23:09:42,514 - lava_callback - ERROR - Error processing callback for node 69409265575bc060ce160a9f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409265575bc060ce160a9f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409265575bc060ce160a9f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409265575bc060ce160a9f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:42,516 - lava_callback - INFO - Starting async job submit for node 69409373575bc060ce160d87 2025-12-17 23:09:42,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:42,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409373575bc060ce160d87 HTTP/1.1" 502 150 2025-12-17 23:09:42,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409373575bc060ce160d87'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:42,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409373575bc060ce160d87 2025-12-17 23:09:42,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409373575bc060ce160d87 HTTP/1.1" 502 150 2025-12-17 23:09:42,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409373575bc060ce160d87'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:42,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409241575bc060ce160a54 2025-12-17 23:09:42,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409241575bc060ce160a54 HTTP/1.1" 502 150 2025-12-17 23:09:42,644 - lava_callback - ERROR - Error processing callback for node 69409241575bc060ce160a54: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409241575bc060ce160a54 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409241575bc060ce160a54 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409241575bc060ce160a54 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:42,646 - lava_callback - INFO - Starting async job submit for node 694092a2575bc060ce160b09 2025-12-17 23:09:42,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:42,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a2575bc060ce160b09 HTTP/1.1" 502 150 2025-12-17 23:09:42,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a2575bc060ce160b09'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:42,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a2575bc060ce160b09 2025-12-17 23:09:42,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a2575bc060ce160b09 HTTP/1.1" 502 150 2025-12-17 23:09:42,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a2575bc060ce160b09'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:42,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409267575bc060ce160aa2 2025-12-17 23:09:42,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409267575bc060ce160aa2 HTTP/1.1" 502 150 2025-12-17 23:09:42,937 - lava_callback - ERROR - Error processing callback for node 69409267575bc060ce160aa2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409267575bc060ce160aa2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409267575bc060ce160aa2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409267575bc060ce160aa2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:42,939 - lava_callback - INFO - Starting async job submit for node 6940929b575bc060ce160afc 2025-12-17 23:09:42,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:42,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929b575bc060ce160afc HTTP/1.1" 502 150 2025-12-17 23:09:42,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940929b575bc060ce160afc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:42,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940929b575bc060ce160afc 2025-12-17 23:09:42,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929b575bc060ce160afc HTTP/1.1" 502 150 2025-12-17 23:09:42,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940929b575bc060ce160afc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:43,312 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409299575bc060ce160af9 2025-12-17 23:09:43,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409299575bc060ce160af9 HTTP/1.1" 502 150 2025-12-17 23:09:43,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409299575bc060ce160af9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:43,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940927a575bc060ce160ac8 2025-12-17 23:09:43,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940927a575bc060ce160ac8 HTTP/1.1" 502 150 2025-12-17 23:09:43,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940927a575bc060ce160ac8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:44,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409269575bc060ce160aa7 2025-12-17 23:09:44,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409373575bc060ce160d87 2025-12-17 23:09:44,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409373575bc060ce160d87 HTTP/1.1" 502 150 2025-12-17 23:09:44,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409373575bc060ce160d87'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:44,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a2575bc060ce160b09 2025-12-17 23:09:44,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a2575bc060ce160b09 HTTP/1.1" 502 150 2025-12-17 23:09:44,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a2575bc060ce160b09'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:44,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940929b575bc060ce160afc 2025-12-17 23:09:44,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929b575bc060ce160afc HTTP/1.1" 502 150 2025-12-17 23:09:44,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940929b575bc060ce160afc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:44,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988e575bc060ce161ae3 2025-12-17 23:09:44,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988e575bc060ce161ae3 HTTP/1.1" 502 150 2025-12-17 23:09:44,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988e575bc060ce161ae3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49962 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:09:46,634 - lava_callback - INFO - Received callback for node 6940952c575bc060ce161262 INFO: 172.21.0.1:49970 - "POST /node/6940952c575bc060ce161262 HTTP/1.0" 202 Accepted 2025-12-17 23:09:47,282 - lava_callback - INFO - Received callback for node 6940952e575bc060ce161265 INFO: 172.21.0.1:49972 - "POST /node/6940952e575bc060ce161265 HTTP/1.0" 202 Accepted 2025-12-17 23:09:47,313 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409299575bc060ce160af9 2025-12-17 23:09:47,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409299575bc060ce160af9 HTTP/1.1" 502 150 2025-12-17 23:09:47,314 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409299575bc060ce160af9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:47,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940926b575bc060ce160aaa 2025-12-17 23:09:47,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926b575bc060ce160aaa HTTP/1.1" 502 150 2025-12-17 23:09:47,797 - lava_callback - ERROR - Error processing callback for node 6940926b575bc060ce160aaa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940926b575bc060ce160aaa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940926b575bc060ce160aaa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940926b575bc060ce160aaa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:47,799 - lava_callback - INFO - Starting async job submit for node 6940929d575bc060ce160aff 2025-12-17 23:09:47,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:47,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929d575bc060ce160aff HTTP/1.1" 502 150 2025-12-17 23:09:47,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940929d575bc060ce160aff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:47,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940929d575bc060ce160aff 2025-12-17 23:09:47,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929d575bc060ce160aff HTTP/1.1" 502 150 2025-12-17 23:09:47,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940929d575bc060ce160aff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:47,912 - lava_callback - INFO - Received callback for node 69409529575bc060ce161257 INFO: 172.21.0.1:49988 - "POST /node/69409529575bc060ce161257 HTTP/1.0" 202 Accepted 2025-12-17 23:09:47,940 - lava_callback - INFO - Received callback for node 69409537575bc060ce161277 INFO: 172.21.0.1:49996 - "POST /node/69409537575bc060ce161277 HTTP/1.0" 202 Accepted 2025-12-17 23:09:48,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409373575bc060ce160d87 2025-12-17 23:09:48,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409373575bc060ce160d87 HTTP/1.1" 502 150 2025-12-17 23:09:48,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409373575bc060ce160d87'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:48,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a2575bc060ce160b09 2025-12-17 23:09:48,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a2575bc060ce160b09 HTTP/1.1" 502 150 2025-12-17 23:09:48,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a2575bc060ce160b09'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:48,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940926c575bc060ce160aad 2025-12-17 23:09:48,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926c575bc060ce160aad HTTP/1.1" 502 150 2025-12-17 23:09:48,917 - lava_callback - ERROR - Error processing callback for node 6940926c575bc060ce160aad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940926c575bc060ce160aad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940926c575bc060ce160aad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940926c575bc060ce160aad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:48,920 - lava_callback - INFO - Starting async job submit for node 6940929f575bc060ce160b06 2025-12-17 23:09:48,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:48,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929f575bc060ce160b06 HTTP/1.1" 502 150 2025-12-17 23:09:48,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940929f575bc060ce160b06'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:48,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940929f575bc060ce160b06 2025-12-17 23:09:48,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929f575bc060ce160b06 HTTP/1.1" 502 150 2025-12-17 23:09:48,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940929f575bc060ce160b06'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:48,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940929b575bc060ce160afc 2025-12-17 23:09:48,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929b575bc060ce160afc HTTP/1.1" 502 150 2025-12-17 23:09:48,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940929b575bc060ce160afc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:49,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940929d575bc060ce160aff 2025-12-17 23:09:49,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929d575bc060ce160aff HTTP/1.1" 502 150 2025-12-17 23:09:49,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940929d575bc060ce160aff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:50,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940926f575bc060ce160ab3 2025-12-17 23:09:50,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940926f575bc060ce160ab3 HTTP/1.1" 502 150 2025-12-17 23:09:50,732 - lava_callback - ERROR - Error processing callback for node 6940926f575bc060ce160ab3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940926f575bc060ce160ab3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940926f575bc060ce160ab3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940926f575bc060ce160ab3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:50,733 - lava_callback - INFO - Starting async job submit for node 694092a2575bc060ce160b0c 2025-12-17 23:09:50,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:50,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a2575bc060ce160b0c HTTP/1.1" 502 150 2025-12-17 23:09:50,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a2575bc060ce160b0c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:50,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a2575bc060ce160b0c 2025-12-17 23:09:50,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a2575bc060ce160b0c HTTP/1.1" 502 150 2025-12-17 23:09:50,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a2575bc060ce160b0c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:50,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940929f575bc060ce160b06 2025-12-17 23:09:50,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929f575bc060ce160b06 HTTP/1.1" 502 150 2025-12-17 23:09:50,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940929f575bc060ce160b06'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:51,002 - lava_callback - INFO - Received callback for node 69409536575bc060ce161274 INFO: 172.21.0.1:34786 - "POST /node/69409536575bc060ce161274 HTTP/1.0" 202 Accepted 2025-12-17 23:09:51,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940927a575bc060ce160ac8 2025-12-17 23:09:51,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940927a575bc060ce160ac8 HTTP/1.1" 502 150 2025-12-17 23:09:51,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940927a575bc060ce160ac8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:52,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a2575bc060ce160b0c 2025-12-17 23:09:52,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a2575bc060ce160b0c HTTP/1.1" 502 150 2025-12-17 23:09:52,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a2575bc060ce160b0c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:52,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409270575bc060ce160ab6 2025-12-17 23:09:52,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409270575bc060ce160ab6 HTTP/1.1" 502 150 2025-12-17 23:09:52,844 - lava_callback - ERROR - Error processing callback for node 69409270575bc060ce160ab6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409270575bc060ce160ab6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409270575bc060ce160ab6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409270575bc060ce160ab6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:52,846 - lava_callback - INFO - Starting async job submit for node 694092a4575bc060ce160b0f 2025-12-17 23:09:52,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:52,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a4575bc060ce160b0f HTTP/1.1" 502 150 2025-12-17 23:09:52,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a4575bc060ce160b0f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:52,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a4575bc060ce160b0f 2025-12-17 23:09:52,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a4575bc060ce160b0f HTTP/1.1" 502 150 2025-12-17 23:09:52,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a4575bc060ce160b0f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:52,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988e575bc060ce161ae3 2025-12-17 23:09:53,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988e575bc060ce161ae3 HTTP/1.1" 502 150 2025-12-17 23:09:53,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988e575bc060ce161ae3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:53,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940929d575bc060ce160aff 2025-12-17 23:09:53,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929d575bc060ce160aff HTTP/1.1" 502 150 2025-12-17 23:09:53,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940929d575bc060ce160aff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:54,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a4575bc060ce160b0f 2025-12-17 23:09:54,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a4575bc060ce160b0f HTTP/1.1" 502 150 2025-12-17 23:09:54,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a4575bc060ce160b0f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:54,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409272575bc060ce160ab9 2025-12-17 23:09:54,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409272575bc060ce160ab9 HTTP/1.1" 502 150 2025-12-17 23:09:54,903 - lava_callback - ERROR - Error processing callback for node 69409272575bc060ce160ab9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409272575bc060ce160ab9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409272575bc060ce160ab9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409272575bc060ce160ab9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:54,905 - lava_callback - INFO - Starting async job submit for node 694092a6575bc060ce160b14 2025-12-17 23:09:54,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:54,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a6575bc060ce160b14 HTTP/1.1" 502 150 2025-12-17 23:09:54,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a6575bc060ce160b14'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:54,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a6575bc060ce160b14 2025-12-17 23:09:54,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a6575bc060ce160b14 HTTP/1.1" 502 150 2025-12-17 23:09:54,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a6575bc060ce160b14'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:54,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940929f575bc060ce160b06 2025-12-17 23:09:54,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929f575bc060ce160b06 HTTP/1.1" 502 150 2025-12-17 23:09:54,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940929f575bc060ce160b06'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:55,151 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409277575bc060ce160ac2 2025-12-17 23:09:55,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409277575bc060ce160ac2 HTTP/1.1" 502 150 2025-12-17 23:09:55,152 - lava_callback - ERROR - Error processing callback for node 69409277575bc060ce160ac2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409277575bc060ce160ac2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409277575bc060ce160ac2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409277575bc060ce160ac2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:55,153 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409274575bc060ce160abc 2025-12-17 23:09:55,154 - lava_callback - INFO - Starting async job submit for node 694092a7575bc060ce160b17 2025-12-17 23:09:55,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:55,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409274575bc060ce160abc HTTP/1.1" 502 150 2025-12-17 23:09:55,156 - lava_callback - ERROR - Error processing callback for node 69409274575bc060ce160abc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409274575bc060ce160abc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409274575bc060ce160abc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409274575bc060ce160abc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:55,157 - lava_callback - INFO - Starting async job submit for node 694092a9575bc060ce160b1a 2025-12-17 23:09:55,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:55,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a7575bc060ce160b17 HTTP/1.1" 502 150 2025-12-17 23:09:55,168 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a7575bc060ce160b17'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:55,168 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a7575bc060ce160b17 2025-12-17 23:09:55,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a7575bc060ce160b17 HTTP/1.1" 502 150 2025-12-17 23:09:55,169 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a7575bc060ce160b17'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:55,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a9575bc060ce160b1a HTTP/1.1" 502 150 2025-12-17 23:09:55,169 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a9575bc060ce160b1a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:55,170 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a9575bc060ce160b1a 2025-12-17 23:09:55,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a9575bc060ce160b1a HTTP/1.1" 502 150 2025-12-17 23:09:55,170 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a9575bc060ce160b1a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:55,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409299575bc060ce160af9 2025-12-17 23:09:55,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409299575bc060ce160af9 HTTP/1.1" 502 150 2025-12-17 23:09:55,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409299575bc060ce160af9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:55,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409276575bc060ce160abf 2025-12-17 23:09:55,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409276575bc060ce160abf HTTP/1.1" 502 150 2025-12-17 23:09:55,577 - lava_callback - ERROR - Error processing callback for node 69409276575bc060ce160abf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409276575bc060ce160abf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409276575bc060ce160abf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409276575bc060ce160abf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:55,579 - lava_callback - INFO - Starting async job submit for node 694092aa575bc060ce160b1d 2025-12-17 23:09:55,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:55,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092aa575bc060ce160b1d HTTP/1.1" 502 150 2025-12-17 23:09:55,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092aa575bc060ce160b1d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:55,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092aa575bc060ce160b1d 2025-12-17 23:09:55,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092aa575bc060ce160b1d HTTP/1.1" 502 150 2025-12-17 23:09:55,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092aa575bc060ce160b1d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:55,754 - lava_callback - INFO - Received callback for node 694097f8575bc060ce16197b INFO: 172.21.0.1:34788 - "POST /node/694097f8575bc060ce16197b HTTP/1.0" 202 Accepted 2025-12-17 23:09:55,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409279575bc060ce160ac5 2025-12-17 23:09:55,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409279575bc060ce160ac5 HTTP/1.1" 502 150 2025-12-17 23:09:55,914 - lava_callback - ERROR - Error processing callback for node 69409279575bc060ce160ac5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409279575bc060ce160ac5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409279575bc060ce160ac5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409279575bc060ce160ac5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:09:55,916 - lava_callback - INFO - Starting async job submit for node 694092ac575bc060ce160b34 2025-12-17 23:09:55,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:09:55,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ac575bc060ce160b34 HTTP/1.1" 502 150 2025-12-17 23:09:55,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ac575bc060ce160b34'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:55,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ac575bc060ce160b34 2025-12-17 23:09:55,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ac575bc060ce160b34 HTTP/1.1" 502 150 2025-12-17 23:09:55,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ac575bc060ce160b34'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:56,180 - lava_callback - INFO - Received callback for node 6940952f575bc060ce161268 INFO: 172.21.0.1:34800 - "POST /node/6940952f575bc060ce161268 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:34802 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:09:56,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409373575bc060ce160d87 2025-12-17 23:09:56,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409373575bc060ce160d87 HTTP/1.1" 502 150 2025-12-17 23:09:56,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409373575bc060ce160d87'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:56,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a2575bc060ce160b09 2025-12-17 23:09:56,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a2575bc060ce160b09 HTTP/1.1" 502 150 2025-12-17 23:09:56,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a2575bc060ce160b09'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:56,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a2575bc060ce160b0c 2025-12-17 23:09:56,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a2575bc060ce160b0c HTTP/1.1" 502 150 2025-12-17 23:09:56,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a2575bc060ce160b0c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:56,822 - lava_callback - INFO - Received callback for node 69409531575bc060ce16126b INFO: 172.21.0.1:34812 - "POST /node/69409531575bc060ce16126b HTTP/1.0" 202 Accepted 2025-12-17 23:09:56,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a6575bc060ce160b14 2025-12-17 23:09:56,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a6575bc060ce160b14 HTTP/1.1" 502 150 2025-12-17 23:09:56,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a6575bc060ce160b14'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:56,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940929b575bc060ce160afc 2025-12-17 23:09:56,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929b575bc060ce160afc HTTP/1.1" 502 150 2025-12-17 23:09:56,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940929b575bc060ce160afc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:57,169 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a7575bc060ce160b17 2025-12-17 23:09:57,170 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a9575bc060ce160b1a 2025-12-17 23:09:57,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a9575bc060ce160b1a HTTP/1.1" 502 150 2025-12-17 23:09:57,171 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a9575bc060ce160b1a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:57,462 - lava_callback - INFO - Received callback for node 69409534575bc060ce161271 INFO: 172.21.0.1:34820 - "POST /node/69409534575bc060ce161271 HTTP/1.0" 202 Accepted 2025-12-17 23:09:57,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092aa575bc060ce160b1d 2025-12-17 23:09:57,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092aa575bc060ce160b1d HTTP/1.1" 502 150 2025-12-17 23:09:57,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092aa575bc060ce160b1d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:57,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ac575bc060ce160b34 2025-12-17 23:09:57,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ac575bc060ce160b34 HTTP/1.1" 502 150 2025-12-17 23:09:57,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ac575bc060ce160b34'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:58,130 - lava_callback - INFO - Received callback for node 69409539575bc060ce16127a INFO: 172.21.0.1:34832 - "POST /node/69409539575bc060ce16127a HTTP/1.0" 202 Accepted 2025-12-17 23:09:58,795 - lava_callback - INFO - Received callback for node 6940953e575bc060ce16128d INFO: 172.21.0.1:34848 - "POST /node/6940953e575bc060ce16128d HTTP/1.0" 202 Accepted 2025-12-17 23:09:58,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a4575bc060ce160b0f 2025-12-17 23:09:58,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a4575bc060ce160b0f HTTP/1.1" 502 150 2025-12-17 23:09:58,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a4575bc060ce160b0f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:09:59,452 - lava_callback - INFO - Received callback for node 6940953f575bc060ce161290 INFO: 172.21.0.1:34864 - "POST /node/6940953f575bc060ce161290 HTTP/1.0" 202 Accepted 2025-12-17 23:10:00,122 - lava_callback - INFO - Received callback for node 69409540575bc060ce161293 INFO: 172.21.0.1:35734 - "POST /node/69409540575bc060ce161293 HTTP/1.0" 202 Accepted 2025-12-17 23:10:00,769 - lava_callback - INFO - Received callback for node 69409542575bc060ce161298 INFO: 172.21.0.1:35740 - "POST /node/69409542575bc060ce161298 HTTP/1.0" 202 Accepted 2025-12-17 23:10:00,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a6575bc060ce160b14 2025-12-17 23:10:00,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a6575bc060ce160b14 HTTP/1.1" 502 150 2025-12-17 23:10:00,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a6575bc060ce160b14'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:01,172 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a9575bc060ce160b1a 2025-12-17 23:10:01,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a9575bc060ce160b1a HTTP/1.1" 502 150 2025-12-17 23:10:01,173 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a9575bc060ce160b1a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:01,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092aa575bc060ce160b1d 2025-12-17 23:10:01,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092aa575bc060ce160b1d HTTP/1.1" 502 150 2025-12-17 23:10:01,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092aa575bc060ce160b1d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:01,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940929d575bc060ce160aff 2025-12-17 23:10:01,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929d575bc060ce160aff HTTP/1.1" 502 150 2025-12-17 23:10:01,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940929d575bc060ce160aff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:01,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ac575bc060ce160b34 2025-12-17 23:10:01,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ac575bc060ce160b34 HTTP/1.1" 502 150 2025-12-17 23:10:01,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ac575bc060ce160b34'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:02,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940929f575bc060ce160b06 2025-12-17 23:10:02,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929f575bc060ce160b06 HTTP/1.1" 502 150 2025-12-17 23:10:02,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940929f575bc060ce160b06'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35750 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:10:04,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a2575bc060ce160b0c 2025-12-17 23:10:04,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a2575bc060ce160b0c HTTP/1.1" 502 150 2025-12-17 23:10:04,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a2575bc060ce160b0c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:06,657 - lava_callback - INFO - Received callback for node 6940953a575bc060ce16127f INFO: 172.21.0.1:35752 - "POST /node/6940953a575bc060ce16127f HTTP/1.0" 202 Accepted INFO: 172.21.0.1:35758 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:10:06,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a4575bc060ce160b0f 2025-12-17 23:10:06,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a4575bc060ce160b0f HTTP/1.1" 502 150 2025-12-17 23:10:06,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a4575bc060ce160b0f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:07,323 - lava_callback - INFO - Received callback for node 6940953b575bc060ce161284 INFO: 172.21.0.1:35766 - "POST /node/6940953b575bc060ce161284 HTTP/1.0" 202 Accepted 2025-12-17 23:10:07,523 - lava_callback - INFO - Received callback for node 6940953c575bc060ce161289 INFO: 172.21.0.1:35768 - "POST /node/6940953c575bc060ce161289 HTTP/1.0" 202 Accepted 2025-12-17 23:10:07,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940927a575bc060ce160ac8 2025-12-17 23:10:07,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940927a575bc060ce160ac8 HTTP/1.1" 502 150 2025-12-17 23:10:07,881 - lava_callback - ERROR - Error processing callback for node 6940927a575bc060ce160ac8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940927a575bc060ce160ac8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940927a575bc060ce160ac8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940927a575bc060ce160ac8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:07,884 - lava_callback - INFO - Starting async job submit for node 694092ad575bc060ce160b37 2025-12-17 23:10:07,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:07,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ad575bc060ce160b37 HTTP/1.1" 502 150 2025-12-17 23:10:07,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ad575bc060ce160b37'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:07,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ad575bc060ce160b37 2025-12-17 23:10:07,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ad575bc060ce160b37 HTTP/1.1" 502 150 2025-12-17 23:10:07,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ad575bc060ce160b37'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:08,339 - lava_callback - INFO - Received callback for node 6940954e575bc060ce1612ba INFO: 172.21.0.1:35782 - "POST /node/6940954e575bc060ce1612ba HTTP/1.0" 202 Accepted 2025-12-17 23:10:08,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a6575bc060ce160b14 2025-12-17 23:10:08,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a6575bc060ce160b14 HTTP/1.1" 502 150 2025-12-17 23:10:08,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a6575bc060ce160b14'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:09,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988e575bc060ce161ae3 2025-12-17 23:10:09,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988e575bc060ce161ae3 HTTP/1.1" 502 150 2025-12-17 23:10:09,002 - lava_callback - ERROR - Error processing callback for node 6940988e575bc060ce161ae3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940988e575bc060ce161ae3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940988e575bc060ce161ae3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940988e575bc060ce161ae3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:09,004 - lava_callback - INFO - Starting async job submit for node 694092ae575bc060ce160b3a 2025-12-17 23:10:09,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:09,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ae575bc060ce160b3a HTTP/1.1" 502 150 2025-12-17 23:10:09,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ae575bc060ce160b3a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:09,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ae575bc060ce160b3a 2025-12-17 23:10:09,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ae575bc060ce160b3a HTTP/1.1" 502 150 2025-12-17 23:10:09,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ae575bc060ce160b3a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:09,173 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a9575bc060ce160b1a 2025-12-17 23:10:09,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a9575bc060ce160b1a HTTP/1.1" 502 150 2025-12-17 23:10:09,174 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a9575bc060ce160b1a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:09,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092aa575bc060ce160b1d 2025-12-17 23:10:09,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092aa575bc060ce160b1d HTTP/1.1" 502 150 2025-12-17 23:10:09,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092aa575bc060ce160b1d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:09,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ad575bc060ce160b37 2025-12-17 23:10:09,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ad575bc060ce160b37 HTTP/1.1" 502 150 2025-12-17 23:10:09,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ad575bc060ce160b37'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:09,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ac575bc060ce160b34 2025-12-17 23:10:09,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ac575bc060ce160b34 HTTP/1.1" 502 150 2025-12-17 23:10:09,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ac575bc060ce160b34'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:11,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ae575bc060ce160b3a 2025-12-17 23:10:11,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ae575bc060ce160b3a HTTP/1.1" 502 150 2025-12-17 23:10:11,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ae575bc060ce160b3a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:11,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409299575bc060ce160af9 2025-12-17 23:10:11,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409299575bc060ce160af9 HTTP/1.1" 502 150 2025-12-17 23:10:11,317 - lava_callback - ERROR - Error processing callback for node 69409299575bc060ce160af9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409299575bc060ce160af9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409299575bc060ce160af9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409299575bc060ce160af9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:11,319 - lava_callback - INFO - Starting async job submit for node 694092b0575bc060ce160b3d 2025-12-17 23:10:11,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:11,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b0575bc060ce160b3d HTTP/1.1" 502 150 2025-12-17 23:10:11,326 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b0575bc060ce160b3d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:11,326 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b0575bc060ce160b3d 2025-12-17 23:10:11,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b0575bc060ce160b3d HTTP/1.1" 502 150 2025-12-17 23:10:11,327 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b0575bc060ce160b3d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:12,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409373575bc060ce160d87 2025-12-17 23:10:12,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409373575bc060ce160d87 HTTP/1.1" 502 150 2025-12-17 23:10:12,549 - lava_callback - ERROR - Error processing callback for node 69409373575bc060ce160d87: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409373575bc060ce160d87 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409373575bc060ce160d87 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409373575bc060ce160d87 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:12,551 - lava_callback - INFO - Starting async job submit for node 694092b2575bc060ce160b43 2025-12-17 23:10:12,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:12,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b2575bc060ce160b43 HTTP/1.1" 502 150 2025-12-17 23:10:12,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b2575bc060ce160b43'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:12,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b2575bc060ce160b43 2025-12-17 23:10:12,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b2575bc060ce160b43 HTTP/1.1" 502 150 2025-12-17 23:10:12,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b2575bc060ce160b43'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:12,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a7575bc060ce160b17 HTTP/1.1" 502 150 2025-12-17 23:10:12,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a7575bc060ce160b17'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:12,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a2575bc060ce160b09 2025-12-17 23:10:12,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a2575bc060ce160b09 HTTP/1.1" 502 150 2025-12-17 23:10:12,664 - lava_callback - ERROR - Error processing callback for node 694092a2575bc060ce160b09: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a2575bc060ce160b09 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a2575bc060ce160b09 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a2575bc060ce160b09 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:12,667 - lava_callback - INFO - Starting async job submit for node 694092b5575bc060ce160b49 2025-12-17 23:10:12,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:12,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b5575bc060ce160b49 HTTP/1.1" 502 150 2025-12-17 23:10:12,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b5575bc060ce160b49'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:12,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b5575bc060ce160b49 2025-12-17 23:10:12,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b5575bc060ce160b49 HTTP/1.1" 502 150 2025-12-17 23:10:12,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b5575bc060ce160b49'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:12,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940929b575bc060ce160afc 2025-12-17 23:10:12,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929b575bc060ce160afc HTTP/1.1" 502 150 2025-12-17 23:10:12,952 - lava_callback - ERROR - Error processing callback for node 6940929b575bc060ce160afc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940929b575bc060ce160afc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940929b575bc060ce160afc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940929b575bc060ce160afc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:12,955 - lava_callback - INFO - Starting async job submit for node 694092b7575bc060ce160b4f 2025-12-17 23:10:12,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:12,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b7575bc060ce160b4f HTTP/1.1" 502 150 2025-12-17 23:10:12,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b7575bc060ce160b4f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:12,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b7575bc060ce160b4f 2025-12-17 23:10:12,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b7575bc060ce160b4f HTTP/1.1" 502 150 2025-12-17 23:10:12,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b7575bc060ce160b4f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:13,327 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b0575bc060ce160b3d 2025-12-17 23:10:13,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b0575bc060ce160b3d HTTP/1.1" 502 150 2025-12-17 23:10:13,328 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b0575bc060ce160b3d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:13,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ad575bc060ce160b37 2025-12-17 23:10:13,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ad575bc060ce160b37 HTTP/1.1" 502 150 2025-12-17 23:10:13,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ad575bc060ce160b37'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:14,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b2575bc060ce160b43 2025-12-17 23:10:14,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b2575bc060ce160b43 HTTP/1.1" 502 150 2025-12-17 23:10:14,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b2575bc060ce160b43'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:14,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b5575bc060ce160b49 2025-12-17 23:10:14,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b5575bc060ce160b49 HTTP/1.1" 502 150 2025-12-17 23:10:14,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b5575bc060ce160b49'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:14,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409269575bc060ce160aa7 HTTP/1.1" 502 150 2025-12-17 23:10:14,708 - lava_callback - ERROR - Error processing callback for node 69409269575bc060ce160aa7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409269575bc060ce160aa7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409269575bc060ce160aa7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409269575bc060ce160aa7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:14,710 - lava_callback - INFO - Starting async job submit for node 694092b8575bc060ce160b52 2025-12-17 23:10:14,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:14,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b8575bc060ce160b52 HTTP/1.1" 502 150 2025-12-17 23:10:14,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b8575bc060ce160b52'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:14,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b8575bc060ce160b52 2025-12-17 23:10:14,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b8575bc060ce160b52 HTTP/1.1" 502 150 2025-12-17 23:10:14,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b8575bc060ce160b52'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:14,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b7575bc060ce160b4f 2025-12-17 23:10:14,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b7575bc060ce160b4f HTTP/1.1" 502 150 2025-12-17 23:10:14,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b7575bc060ce160b4f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:15,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ae575bc060ce160b3a 2025-12-17 23:10:15,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ae575bc060ce160b3a HTTP/1.1" 502 150 2025-12-17 23:10:15,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ae575bc060ce160b3a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38104 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:10:16,183 - lava_callback - INFO - Received callback for node 69409547575bc060ce1612a5 INFO: 172.21.0.1:38120 - "POST /node/69409547575bc060ce1612a5 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:38134 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:10:16,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a7575bc060ce160b17 2025-12-17 23:10:16,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a7575bc060ce160b17 HTTP/1.1" 502 150 2025-12-17 23:10:16,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a7575bc060ce160b17'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:16,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b8575bc060ce160b52 2025-12-17 23:10:16,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b8575bc060ce160b52 HTTP/1.1" 502 150 2025-12-17 23:10:16,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b8575bc060ce160b52'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:16,835 - lava_callback - INFO - Received callback for node 69409548575bc060ce1612a8 INFO: 172.21.0.1:38140 - "POST /node/69409548575bc060ce1612a8 HTTP/1.0" 202 Accepted 2025-12-17 23:10:17,328 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b0575bc060ce160b3d 2025-12-17 23:10:17,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b0575bc060ce160b3d HTTP/1.1" 502 150 2025-12-17 23:10:17,330 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b0575bc060ce160b3d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:17,480 - lava_callback - INFO - Received callback for node 69409549575bc060ce1612ab INFO: 172.21.0.1:38150 - "POST /node/69409549575bc060ce1612ab HTTP/1.0" 202 Accepted 2025-12-17 23:10:17,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940929d575bc060ce160aff 2025-12-17 23:10:17,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929d575bc060ce160aff HTTP/1.1" 502 150 2025-12-17 23:10:17,816 - lava_callback - ERROR - Error processing callback for node 6940929d575bc060ce160aff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940929d575bc060ce160aff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940929d575bc060ce160aff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940929d575bc060ce160aff (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:17,817 - lava_callback - INFO - Starting async job submit for node 694092b1575bc060ce160b40 2025-12-17 23:10:17,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:17,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b1575bc060ce160b40 HTTP/1.1" 502 150 2025-12-17 23:10:17,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b1575bc060ce160b40'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:17,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b1575bc060ce160b40 2025-12-17 23:10:17,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b1575bc060ce160b40 HTTP/1.1" 502 150 2025-12-17 23:10:17,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b1575bc060ce160b40'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:18,153 - lava_callback - INFO - Received callback for node 6940954a575bc060ce1612ae INFO: 172.21.0.1:38166 - "POST /node/6940954a575bc060ce1612ae HTTP/1.0" 202 Accepted 2025-12-17 23:10:18,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b2575bc060ce160b43 2025-12-17 23:10:18,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b2575bc060ce160b43 HTTP/1.1" 502 150 2025-12-17 23:10:18,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b2575bc060ce160b43'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:18,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b5575bc060ce160b49 2025-12-17 23:10:18,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b5575bc060ce160b49 HTTP/1.1" 502 150 2025-12-17 23:10:18,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b5575bc060ce160b49'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:18,808 - lava_callback - INFO - Received callback for node 6940954c575bc060ce1612b1 INFO: 172.21.0.1:38168 - "POST /node/6940954c575bc060ce1612b1 HTTP/1.0" 202 Accepted 2025-12-17 23:10:18,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940929f575bc060ce160b06 2025-12-17 23:10:18,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940929f575bc060ce160b06 HTTP/1.1" 502 150 2025-12-17 23:10:18,954 - lava_callback - ERROR - Error processing callback for node 6940929f575bc060ce160b06: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940929f575bc060ce160b06 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940929f575bc060ce160b06 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940929f575bc060ce160b06 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:18,956 - lava_callback - INFO - Starting async job submit for node 694092b4575bc060ce160b46 2025-12-17 23:10:18,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:18,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b7575bc060ce160b4f 2025-12-17 23:10:18,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b7575bc060ce160b4f HTTP/1.1" 502 150 2025-12-17 23:10:18,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b7575bc060ce160b4f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:18,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b4575bc060ce160b46 HTTP/1.1" 502 150 2025-12-17 23:10:18,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b4575bc060ce160b46'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:18,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b4575bc060ce160b46 2025-12-17 23:10:18,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b4575bc060ce160b46 HTTP/1.1" 502 150 2025-12-17 23:10:18,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b4575bc060ce160b46'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:19,471 - lava_callback - INFO - Received callback for node 6940954e575bc060ce1612be INFO: 172.21.0.1:38172 - "POST /node/6940954e575bc060ce1612be HTTP/1.0" 202 Accepted 2025-12-17 23:10:19,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b1575bc060ce160b40 2025-12-17 23:10:19,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b1575bc060ce160b40 HTTP/1.1" 502 150 2025-12-17 23:10:19,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b1575bc060ce160b40'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:20,125 - lava_callback - INFO - Received callback for node 69409551575bc060ce1612cd INFO: 172.21.0.1:41780 - "POST /node/69409551575bc060ce1612cd HTTP/1.0" 202 Accepted 2025-12-17 23:10:20,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b8575bc060ce160b52 2025-12-17 23:10:20,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b8575bc060ce160b52 HTTP/1.1" 502 150 2025-12-17 23:10:20,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b8575bc060ce160b52'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:20,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a2575bc060ce160b0c 2025-12-17 23:10:20,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a2575bc060ce160b0c HTTP/1.1" 502 150 2025-12-17 23:10:20,755 - lava_callback - ERROR - Error processing callback for node 694092a2575bc060ce160b0c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a2575bc060ce160b0c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a2575bc060ce160b0c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a2575bc060ce160b0c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:20,757 - lava_callback - INFO - Starting async job submit for node 694092b6575bc060ce160b4c 2025-12-17 23:10:20,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:20,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b6575bc060ce160b4c HTTP/1.1" 502 150 2025-12-17 23:10:20,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b6575bc060ce160b4c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:20,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b6575bc060ce160b4c 2025-12-17 23:10:20,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b6575bc060ce160b4c HTTP/1.1" 502 150 2025-12-17 23:10:20,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b6575bc060ce160b4c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:20,797 - lava_callback - INFO - Received callback for node 69409556575bc060ce1612d6 INFO: 172.21.0.1:41794 - "POST /node/69409556575bc060ce1612d6 HTTP/1.0" 202 Accepted 2025-12-17 23:10:20,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b4575bc060ce160b46 2025-12-17 23:10:20,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b4575bc060ce160b46 HTTP/1.1" 502 150 2025-12-17 23:10:20,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b4575bc060ce160b46'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:21,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ad575bc060ce160b37 2025-12-17 23:10:21,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ad575bc060ce160b37 HTTP/1.1" 502 150 2025-12-17 23:10:21,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ad575bc060ce160b37'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:22,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b6575bc060ce160b4c 2025-12-17 23:10:22,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b6575bc060ce160b4c HTTP/1.1" 502 150 2025-12-17 23:10:22,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b6575bc060ce160b4c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:22,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a4575bc060ce160b0f 2025-12-17 23:10:22,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a4575bc060ce160b0f HTTP/1.1" 502 150 2025-12-17 23:10:22,866 - lava_callback - ERROR - Error processing callback for node 694092a4575bc060ce160b0f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a4575bc060ce160b0f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a4575bc060ce160b0f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a4575bc060ce160b0f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:22,868 - lava_callback - INFO - Starting async job submit for node 694092ba575bc060ce160b58 2025-12-17 23:10:22,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:22,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ba575bc060ce160b58 HTTP/1.1" 502 150 2025-12-17 23:10:22,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ba575bc060ce160b58'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:22,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ba575bc060ce160b58 2025-12-17 23:10:22,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ba575bc060ce160b58 HTTP/1.1" 502 150 2025-12-17 23:10:22,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ba575bc060ce160b58'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:23,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ae575bc060ce160b3a 2025-12-17 23:10:23,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b1575bc060ce160b40 2025-12-17 23:10:23,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b1575bc060ce160b40 HTTP/1.1" 502 150 2025-12-17 23:10:23,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b1575bc060ce160b40'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:24,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a7575bc060ce160b17 2025-12-17 23:10:24,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a7575bc060ce160b17 HTTP/1.1" 502 150 2025-12-17 23:10:24,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092a7575bc060ce160b17'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:24,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ba575bc060ce160b58 2025-12-17 23:10:24,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ba575bc060ce160b58 HTTP/1.1" 502 150 2025-12-17 23:10:24,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ba575bc060ce160b58'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:24,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a6575bc060ce160b14 2025-12-17 23:10:24,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a6575bc060ce160b14 HTTP/1.1" 502 150 2025-12-17 23:10:24,920 - lava_callback - ERROR - Error processing callback for node 694092a6575bc060ce160b14: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a6575bc060ce160b14 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a6575bc060ce160b14 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a6575bc060ce160b14 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:24,921 - lava_callback - INFO - Starting async job submit for node 694092bb575bc060ce160b60 2025-12-17 23:10:24,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:24,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bb575bc060ce160b60 HTTP/1.1" 502 150 2025-12-17 23:10:24,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092bb575bc060ce160b60'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:24,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092bb575bc060ce160b60 2025-12-17 23:10:24,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bb575bc060ce160b60 HTTP/1.1" 502 150 2025-12-17 23:10:24,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092bb575bc060ce160b60'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:24,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b4575bc060ce160b46 2025-12-17 23:10:24,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b4575bc060ce160b46 HTTP/1.1" 502 150 2025-12-17 23:10:24,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b4575bc060ce160b46'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:25,174 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a9575bc060ce160b1a 2025-12-17 23:10:25,330 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b0575bc060ce160b3d 2025-12-17 23:10:25,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b0575bc060ce160b3d HTTP/1.1" 502 150 2025-12-17 23:10:25,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b0575bc060ce160b3d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:25,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092aa575bc060ce160b1d 2025-12-17 23:10:25,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092aa575bc060ce160b1d HTTP/1.1" 502 150 2025-12-17 23:10:25,610 - lava_callback - ERROR - Error processing callback for node 694092aa575bc060ce160b1d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092aa575bc060ce160b1d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092aa575bc060ce160b1d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092aa575bc060ce160b1d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:25,612 - lava_callback - INFO - Starting async job submit for node 694092bd575bc060ce160b65 2025-12-17 23:10:25,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:25,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bd575bc060ce160b65 HTTP/1.1" 502 150 2025-12-17 23:10:25,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092bd575bc060ce160b65'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:25,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092bd575bc060ce160b65 2025-12-17 23:10:25,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bd575bc060ce160b65 HTTP/1.1" 502 150 2025-12-17 23:10:25,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092bd575bc060ce160b65'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:25,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ac575bc060ce160b34 2025-12-17 23:10:25,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ac575bc060ce160b34 HTTP/1.1" 502 150 2025-12-17 23:10:25,943 - lava_callback - ERROR - Error processing callback for node 694092ac575bc060ce160b34: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ac575bc060ce160b34 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ac575bc060ce160b34 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ac575bc060ce160b34 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:25,945 - lava_callback - INFO - Starting async job submit for node 694092bf575bc060ce160b68 2025-12-17 23:10:25,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:25,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bf575bc060ce160b68 HTTP/1.1" 502 150 2025-12-17 23:10:25,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092bf575bc060ce160b68'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:25,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092bf575bc060ce160b68 2025-12-17 23:10:25,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bf575bc060ce160b68 HTTP/1.1" 502 150 2025-12-17 23:10:25,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092bf575bc060ce160b68'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41810 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:10:26,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b2575bc060ce160b43 2025-12-17 23:10:26,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b2575bc060ce160b43 HTTP/1.1" 502 150 2025-12-17 23:10:26,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b2575bc060ce160b43'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:26,682 - lava_callback - INFO - Received callback for node 69409553575bc060ce1612d0 INFO: 172.21.0.1:41822 - "POST /node/69409553575bc060ce1612d0 HTTP/1.0" 202 Accepted 2025-12-17 23:10:26,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b5575bc060ce160b49 2025-12-17 23:10:26,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b5575bc060ce160b49 HTTP/1.1" 502 150 2025-12-17 23:10:26,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b5575bc060ce160b49'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:26,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b6575bc060ce160b4c 2025-12-17 23:10:26,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b6575bc060ce160b4c HTTP/1.1" 502 150 2025-12-17 23:10:26,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b6575bc060ce160b4c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:26,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092bb575bc060ce160b60 2025-12-17 23:10:26,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bb575bc060ce160b60 HTTP/1.1" 502 150 2025-12-17 23:10:26,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092bb575bc060ce160b60'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:26,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b7575bc060ce160b4f 2025-12-17 23:10:26,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b7575bc060ce160b4f HTTP/1.1" 502 150 2025-12-17 23:10:26,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b7575bc060ce160b4f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:27,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092bd575bc060ce160b65 2025-12-17 23:10:27,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bd575bc060ce160b65 HTTP/1.1" 502 150 2025-12-17 23:10:27,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092bd575bc060ce160b65'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:27,680 - lava_callback - INFO - Received callback for node 69409550575bc060ce1612c4 INFO: 172.21.0.1:41830 - "POST /node/69409550575bc060ce1612c4 HTTP/1.0" 202 Accepted 2025-12-17 23:10:27,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092bf575bc060ce160b68 2025-12-17 23:10:27,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bf575bc060ce160b68 HTTP/1.1" 502 150 2025-12-17 23:10:27,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092bf575bc060ce160b68'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:28,465 - lava_callback - INFO - Received callback for node 69409555575bc060ce1612d3 INFO: 172.21.0.1:41832 - "POST /node/69409555575bc060ce1612d3 HTTP/1.0" 202 Accepted 2025-12-17 23:10:28,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b8575bc060ce160b52 2025-12-17 23:10:28,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b8575bc060ce160b52 HTTP/1.1" 502 150 2025-12-17 23:10:28,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b8575bc060ce160b52'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:28,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ba575bc060ce160b58 2025-12-17 23:10:28,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ba575bc060ce160b58 HTTP/1.1" 502 150 2025-12-17 23:10:28,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ba575bc060ce160b58'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:29,214 - lava_callback - INFO - Received callback for node 69409562575bc060ce1612f5 INFO: 172.21.0.1:41840 - "POST /node/69409562575bc060ce1612f5 HTTP/1.0" 202 Accepted 2025-12-17 23:10:30,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092bb575bc060ce160b60 2025-12-17 23:10:30,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bb575bc060ce160b60 HTTP/1.1" 502 150 2025-12-17 23:10:30,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092bb575bc060ce160b60'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:31,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092bd575bc060ce160b65 2025-12-17 23:10:31,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bd575bc060ce160b65 HTTP/1.1" 502 150 2025-12-17 23:10:31,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092bd575bc060ce160b65'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:31,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b1575bc060ce160b40 2025-12-17 23:10:31,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b1575bc060ce160b40 HTTP/1.1" 502 150 2025-12-17 23:10:31,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b1575bc060ce160b40'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:31,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092bf575bc060ce160b68 2025-12-17 23:10:31,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bf575bc060ce160b68 HTTP/1.1" 502 150 2025-12-17 23:10:31,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092bf575bc060ce160b68'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:32,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b4575bc060ce160b46 2025-12-17 23:10:32,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b4575bc060ce160b46 HTTP/1.1" 502 150 2025-12-17 23:10:32,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b4575bc060ce160b46'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:34,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b6575bc060ce160b4c 2025-12-17 23:10:34,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b6575bc060ce160b4c HTTP/1.1" 502 150 2025-12-17 23:10:34,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092b6575bc060ce160b4c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:36,198 - lava_callback - INFO - Received callback for node 69409557575bc060ce1612d9 INFO: 172.21.0.1:52646 - "POST /node/69409557575bc060ce1612d9 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:52654 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:10:36,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ba575bc060ce160b58 2025-12-17 23:10:36,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ba575bc060ce160b58 HTTP/1.1" 502 150 2025-12-17 23:10:36,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ba575bc060ce160b58'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:37,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ad575bc060ce160b37 2025-12-17 23:10:37,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ad575bc060ce160b37 HTTP/1.1" 502 150 2025-12-17 23:10:37,903 - lava_callback - ERROR - Error processing callback for node 694092ad575bc060ce160b37: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ad575bc060ce160b37 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ad575bc060ce160b37 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ad575bc060ce160b37 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:37,905 - lava_callback - INFO - Starting async job submit for node 694092c9575bc060ce160b6b 2025-12-17 23:10:37,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:37,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092c9575bc060ce160b6b HTTP/1.1" 502 150 2025-12-17 23:10:37,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092c9575bc060ce160b6b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:37,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092c9575bc060ce160b6b 2025-12-17 23:10:37,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092c9575bc060ce160b6b HTTP/1.1" 502 150 2025-12-17 23:10:37,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092c9575bc060ce160b6b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:38,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ae575bc060ce160b3a HTTP/1.1" 502 150 2025-12-17 23:10:38,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ae575bc060ce160b3a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:38,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092bb575bc060ce160b60 2025-12-17 23:10:38,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bb575bc060ce160b60 HTTP/1.1" 502 150 2025-12-17 23:10:38,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092bb575bc060ce160b60'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:39,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092bd575bc060ce160b65 2025-12-17 23:10:39,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bd575bc060ce160b65 HTTP/1.1" 502 150 2025-12-17 23:10:39,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092bd575bc060ce160b65'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:39,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092c9575bc060ce160b6b 2025-12-17 23:10:39,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092c9575bc060ce160b6b HTTP/1.1" 502 150 2025-12-17 23:10:39,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092c9575bc060ce160b6b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:39,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092bf575bc060ce160b68 2025-12-17 23:10:39,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bf575bc060ce160b68 HTTP/1.1" 502 150 2025-12-17 23:10:39,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092bf575bc060ce160b68'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:40,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a9575bc060ce160b1a HTTP/1.1" 502 150 2025-12-17 23:10:40,560 - lava_callback - ERROR - Error processing callback for node 694092a9575bc060ce160b1a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a9575bc060ce160b1a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a9575bc060ce160b1a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a9575bc060ce160b1a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:40,562 - lava_callback - INFO - Starting async job submit for node 694092cc575bc060ce160b71 2025-12-17 23:10:40,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:40,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092cc575bc060ce160b71 HTTP/1.1" 502 150 2025-12-17 23:10:40,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092cc575bc060ce160b71'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:40,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092cc575bc060ce160b71 2025-12-17 23:10:40,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092cc575bc060ce160b71 HTTP/1.1" 502 150 2025-12-17 23:10:40,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092cc575bc060ce160b71'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:40,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092a7575bc060ce160b17 2025-12-17 23:10:40,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092a7575bc060ce160b17 HTTP/1.1" 502 150 2025-12-17 23:10:40,664 - lava_callback - ERROR - Error processing callback for node 694092a7575bc060ce160b17: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a7575bc060ce160b17 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a7575bc060ce160b17 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092a7575bc060ce160b17 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:40,665 - lava_callback - INFO - Starting async job submit for node 694092d5575bc060ce160b8b 2025-12-17 23:10:40,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:40,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d5575bc060ce160b8b HTTP/1.1" 502 150 2025-12-17 23:10:40,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d5575bc060ce160b8b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:40,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d5575bc060ce160b8b 2025-12-17 23:10:40,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d5575bc060ce160b8b HTTP/1.1" 502 150 2025-12-17 23:10:40,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d5575bc060ce160b8b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:41,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b0575bc060ce160b3d 2025-12-17 23:10:41,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b0575bc060ce160b3d HTTP/1.1" 502 150 2025-12-17 23:10:41,333 - lava_callback - ERROR - Error processing callback for node 694092b0575bc060ce160b3d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b0575bc060ce160b3d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b0575bc060ce160b3d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b0575bc060ce160b3d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:41,335 - lava_callback - INFO - Starting async job submit for node 694092ce575bc060ce160b78 2025-12-17 23:10:41,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:41,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ce575bc060ce160b78 HTTP/1.1" 502 150 2025-12-17 23:10:41,341 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ce575bc060ce160b78'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:41,341 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ce575bc060ce160b78 2025-12-17 23:10:41,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ce575bc060ce160b78 HTTP/1.1" 502 150 2025-12-17 23:10:41,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ce575bc060ce160b78'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:42,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092cc575bc060ce160b71 2025-12-17 23:10:42,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092cc575bc060ce160b71 HTTP/1.1" 502 150 2025-12-17 23:10:42,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092cc575bc060ce160b71'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:42,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b2575bc060ce160b43 2025-12-17 23:10:42,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b2575bc060ce160b43 HTTP/1.1" 502 150 2025-12-17 23:10:42,589 - lava_callback - ERROR - Error processing callback for node 694092b2575bc060ce160b43: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b2575bc060ce160b43 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b2575bc060ce160b43 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b2575bc060ce160b43 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:42,591 - lava_callback - INFO - Starting async job submit for node 694092d1575bc060ce160b83 2025-12-17 23:10:42,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:42,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d1575bc060ce160b83 HTTP/1.1" 502 150 2025-12-17 23:10:42,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d1575bc060ce160b83'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:42,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d1575bc060ce160b83 2025-12-17 23:10:42,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d1575bc060ce160b83 HTTP/1.1" 502 150 2025-12-17 23:10:42,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d1575bc060ce160b83'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:42,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b5575bc060ce160b49 2025-12-17 23:10:42,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b5575bc060ce160b49 HTTP/1.1" 502 150 2025-12-17 23:10:42,691 - lava_callback - ERROR - Error processing callback for node 694092b5575bc060ce160b49: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b5575bc060ce160b49 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b5575bc060ce160b49 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b5575bc060ce160b49 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:42,693 - lava_callback - INFO - Starting async job submit for node 694092da575bc060ce160b99 2025-12-17 23:10:42,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:42,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d5575bc060ce160b8b 2025-12-17 23:10:42,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d5575bc060ce160b8b HTTP/1.1" 502 150 2025-12-17 23:10:42,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d5575bc060ce160b8b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:42,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092da575bc060ce160b99 HTTP/1.1" 502 150 2025-12-17 23:10:42,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092da575bc060ce160b99'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:42,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092da575bc060ce160b99 2025-12-17 23:10:42,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092da575bc060ce160b99 HTTP/1.1" 502 150 2025-12-17 23:10:42,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092da575bc060ce160b99'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:42,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b7575bc060ce160b4f 2025-12-17 23:10:42,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b7575bc060ce160b4f HTTP/1.1" 502 150 2025-12-17 23:10:42,979 - lava_callback - ERROR - Error processing callback for node 694092b7575bc060ce160b4f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b7575bc060ce160b4f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b7575bc060ce160b4f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b7575bc060ce160b4f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:42,981 - lava_callback - INFO - Starting async job submit for node 694092dd575bc060ce160b9c 2025-12-17 23:10:42,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:42,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092dd575bc060ce160b9c HTTP/1.1" 502 150 2025-12-17 23:10:42,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092dd575bc060ce160b9c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:42,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092dd575bc060ce160b9c 2025-12-17 23:10:42,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092dd575bc060ce160b9c HTTP/1.1" 502 150 2025-12-17 23:10:42,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092dd575bc060ce160b9c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:43,342 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ce575bc060ce160b78 2025-12-17 23:10:43,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ce575bc060ce160b78 HTTP/1.1" 502 150 2025-12-17 23:10:43,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ce575bc060ce160b78'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:43,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092c9575bc060ce160b6b 2025-12-17 23:10:43,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092c9575bc060ce160b6b HTTP/1.1" 502 150 2025-12-17 23:10:43,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092c9575bc060ce160b6b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:44,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d1575bc060ce160b83 2025-12-17 23:10:44,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d1575bc060ce160b83 HTTP/1.1" 502 150 2025-12-17 23:10:44,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d1575bc060ce160b83'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:44,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092da575bc060ce160b99 2025-12-17 23:10:44,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092da575bc060ce160b99 HTTP/1.1" 502 150 2025-12-17 23:10:44,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092da575bc060ce160b99'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:44,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b8575bc060ce160b52 2025-12-17 23:10:44,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b8575bc060ce160b52 HTTP/1.1" 502 150 2025-12-17 23:10:44,724 - lava_callback - ERROR - Error processing callback for node 694092b8575bc060ce160b52: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b8575bc060ce160b52 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b8575bc060ce160b52 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b8575bc060ce160b52 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:44,727 - lava_callback - INFO - Starting async job submit for node 694092e0575bc060ce160ba2 2025-12-17 23:10:44,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:44,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e0575bc060ce160ba2 HTTP/1.1" 502 150 2025-12-17 23:10:44,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e0575bc060ce160ba2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:44,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e0575bc060ce160ba2 2025-12-17 23:10:44,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e0575bc060ce160ba2 HTTP/1.1" 502 150 2025-12-17 23:10:44,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e0575bc060ce160ba2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:44,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092dd575bc060ce160b9c 2025-12-17 23:10:44,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092dd575bc060ce160b9c HTTP/1.1" 502 150 2025-12-17 23:10:44,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092dd575bc060ce160b9c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:32860 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:10:46,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092cc575bc060ce160b71 2025-12-17 23:10:46,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092cc575bc060ce160b71 HTTP/1.1" 502 150 2025-12-17 23:10:46,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092cc575bc060ce160b71'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:46,693 - lava_callback - INFO - Received callback for node 6940956a575bc060ce161312 2025-12-17 23:10:46,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d5575bc060ce160b8b INFO: 172.21.0.1:32874 - "POST /node/6940956a575bc060ce161312 HTTP/1.0" 202 Accepted 2025-12-17 23:10:46,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d5575bc060ce160b8b HTTP/1.1" 502 150 2025-12-17 23:10:46,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d5575bc060ce160b8b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:46,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e0575bc060ce160ba2 2025-12-17 23:10:46,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e0575bc060ce160ba2 HTTP/1.1" 502 150 2025-12-17 23:10:46,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e0575bc060ce160ba2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:47,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ce575bc060ce160b78 2025-12-17 23:10:47,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ce575bc060ce160b78 HTTP/1.1" 502 150 2025-12-17 23:10:47,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ce575bc060ce160b78'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:47,350 - lava_callback - INFO - Received callback for node 6940956c575bc060ce161315 INFO: 172.21.0.1:32890 - "POST /node/6940956c575bc060ce161315 HTTP/1.0" 202 Accepted 2025-12-17 23:10:47,411 - lava_callback - INFO - Received callback for node 69409567575bc060ce161309 INFO: 172.21.0.1:32900 - "POST /node/69409567575bc060ce161309 HTTP/1.0" 202 Accepted 2025-12-17 23:10:47,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b1575bc060ce160b40 2025-12-17 23:10:47,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b1575bc060ce160b40 HTTP/1.1" 502 150 2025-12-17 23:10:47,833 - lava_callback - ERROR - Error processing callback for node 694092b1575bc060ce160b40: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b1575bc060ce160b40 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b1575bc060ce160b40 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b1575bc060ce160b40 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:47,836 - lava_callback - INFO - Starting async job submit for node 6940939a575bc060ce160de3 2025-12-17 23:10:47,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:47,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939a575bc060ce160de3 HTTP/1.1" 502 150 2025-12-17 23:10:47,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940939a575bc060ce160de3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:47,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940939a575bc060ce160de3 2025-12-17 23:10:47,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939a575bc060ce160de3 HTTP/1.1" 502 150 2025-12-17 23:10:47,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940939a575bc060ce160de3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:48,018 - lava_callback - INFO - Received callback for node 6940956f575bc060ce16131e INFO: 172.21.0.1:32912 - "POST /node/6940956f575bc060ce16131e HTTP/1.0" 202 Accepted 2025-12-17 23:10:48,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d1575bc060ce160b83 2025-12-17 23:10:48,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d1575bc060ce160b83 HTTP/1.1" 502 150 2025-12-17 23:10:48,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d1575bc060ce160b83'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:48,668 - lava_callback - INFO - Received callback for node 69409571575bc060ce161321 INFO: 172.21.0.1:32922 - "POST /node/69409571575bc060ce161321 HTTP/1.0" 202 Accepted 2025-12-17 23:10:48,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092da575bc060ce160b99 2025-12-17 23:10:48,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092da575bc060ce160b99 HTTP/1.1" 502 150 2025-12-17 23:10:48,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092da575bc060ce160b99'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:48,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b4575bc060ce160b46 2025-12-17 23:10:48,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b4575bc060ce160b46 HTTP/1.1" 502 150 2025-12-17 23:10:48,992 - lava_callback - ERROR - Error processing callback for node 694092b4575bc060ce160b46: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b4575bc060ce160b46 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b4575bc060ce160b46 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b4575bc060ce160b46 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:48,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092dd575bc060ce160b9c 2025-12-17 23:10:48,995 - lava_callback - INFO - Starting async job submit for node 694092d7575bc060ce160b8e 2025-12-17 23:10:48,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092dd575bc060ce160b9c HTTP/1.1" 502 150 2025-12-17 23:10:48,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092dd575bc060ce160b9c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:48,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:49,334 - lava_callback - INFO - Received callback for node 69409572575bc060ce161324 INFO: 172.21.0.1:32924 - "POST /node/69409572575bc060ce161324 HTTP/1.0" 202 Accepted 2025-12-17 23:10:49,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940939a575bc060ce160de3 2025-12-17 23:10:49,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939a575bc060ce160de3 HTTP/1.1" 502 150 2025-12-17 23:10:49,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940939a575bc060ce160de3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:49,998 - lava_callback - INFO - Received callback for node 69409573575bc060ce161327 INFO: 172.21.0.1:58030 - "POST /node/69409573575bc060ce161327 HTTP/1.0" 202 Accepted 2025-12-17 23:10:50,649 - lava_callback - INFO - Received callback for node 69409574575bc060ce16132a INFO: 172.21.0.1:58034 - "POST /node/69409574575bc060ce16132a HTTP/1.0" 202 Accepted 2025-12-17 23:10:50,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e0575bc060ce160ba2 2025-12-17 23:10:50,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e0575bc060ce160ba2 HTTP/1.1" 502 150 2025-12-17 23:10:50,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e0575bc060ce160ba2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:50,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092b6575bc060ce160b4c 2025-12-17 23:10:50,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092b6575bc060ce160b4c HTTP/1.1" 502 150 2025-12-17 23:10:50,768 - lava_callback - ERROR - Error processing callback for node 694092b6575bc060ce160b4c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b6575bc060ce160b4c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b6575bc060ce160b4c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092b6575bc060ce160b4c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:50,770 - lava_callback - INFO - Starting async job submit for node 694092d9575bc060ce160b96 2025-12-17 23:10:50,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:50,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d9575bc060ce160b96 HTTP/1.1" 502 150 2025-12-17 23:10:50,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d9575bc060ce160b96'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:50,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d9575bc060ce160b96 2025-12-17 23:10:50,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d9575bc060ce160b96 HTTP/1.1" 502 150 2025-12-17 23:10:50,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d9575bc060ce160b96'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:51,256 - lava_callback - INFO - Received callback for node 69409569575bc060ce16130f INFO: 172.21.0.1:58040 - "POST /node/69409569575bc060ce16130f HTTP/1.0" 202 Accepted 2025-12-17 23:10:51,910 - lava_callback - INFO - Received callback for node 6940956d575bc060ce161318 INFO: 172.21.0.1:58056 - "POST /node/6940956d575bc060ce161318 HTTP/1.0" 202 Accepted 2025-12-17 23:10:51,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092c9575bc060ce160b6b 2025-12-17 23:10:51,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092c9575bc060ce160b6b HTTP/1.1" 502 150 2025-12-17 23:10:51,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092c9575bc060ce160b6b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:52,567 - lava_callback - INFO - Received callback for node 6940956e575bc060ce16131b INFO: 172.21.0.1:58058 - "POST /node/6940956e575bc060ce16131b HTTP/1.0" 202 Accepted 2025-12-17 23:10:52,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d9575bc060ce160b96 2025-12-17 23:10:52,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d9575bc060ce160b96 HTTP/1.1" 502 150 2025-12-17 23:10:52,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d9575bc060ce160b96'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:52,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ba575bc060ce160b58 2025-12-17 23:10:52,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ba575bc060ce160b58 HTTP/1.1" 502 150 2025-12-17 23:10:52,880 - lava_callback - ERROR - Error processing callback for node 694092ba575bc060ce160b58: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ba575bc060ce160b58 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ba575bc060ce160b58 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ba575bc060ce160b58 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:52,882 - lava_callback - INFO - Starting async job submit for node 694092df575bc060ce160b9f 2025-12-17 23:10:52,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:52,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092df575bc060ce160b9f HTTP/1.1" 502 150 2025-12-17 23:10:52,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092df575bc060ce160b9f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:52,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092df575bc060ce160b9f 2025-12-17 23:10:52,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092df575bc060ce160b9f HTTP/1.1" 502 150 2025-12-17 23:10:52,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092df575bc060ce160b9f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:53,232 - lava_callback - INFO - Received callback for node 69409576575bc060ce16132f INFO: 172.21.0.1:58074 - "POST /node/69409576575bc060ce16132f HTTP/1.0" 202 Accepted 2025-12-17 23:10:53,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940939a575bc060ce160de3 2025-12-17 23:10:53,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939a575bc060ce160de3 HTTP/1.1" 502 150 2025-12-17 23:10:53,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940939a575bc060ce160de3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:54,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ae575bc060ce160b3a 2025-12-17 23:10:54,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ae575bc060ce160b3a HTTP/1.1" 502 150 2025-12-17 23:10:54,514 - lava_callback - ERROR - Error processing callback for node 694092ae575bc060ce160b3a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ae575bc060ce160b3a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ae575bc060ce160b3a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ae575bc060ce160b3a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:54,516 - lava_callback - INFO - Starting async job submit for node 694092e2575bc060ce160baf 2025-12-17 23:10:54,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:54,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e2575bc060ce160baf HTTP/1.1" 502 150 2025-12-17 23:10:54,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e2575bc060ce160baf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:54,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e2575bc060ce160baf 2025-12-17 23:10:54,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e2575bc060ce160baf HTTP/1.1" 502 150 2025-12-17 23:10:54,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e2575bc060ce160baf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:54,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092cc575bc060ce160b71 2025-12-17 23:10:54,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092cc575bc060ce160b71 HTTP/1.1" 502 150 2025-12-17 23:10:54,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092cc575bc060ce160b71'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:54,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d5575bc060ce160b8b 2025-12-17 23:10:54,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d5575bc060ce160b8b HTTP/1.1" 502 150 2025-12-17 23:10:54,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d5575bc060ce160b8b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:54,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092df575bc060ce160b9f 2025-12-17 23:10:54,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092df575bc060ce160b9f HTTP/1.1" 502 150 2025-12-17 23:10:54,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092df575bc060ce160b9f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:54,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092bb575bc060ce160b60 2025-12-17 23:10:54,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bb575bc060ce160b60 HTTP/1.1" 502 150 2025-12-17 23:10:54,934 - lava_callback - ERROR - Error processing callback for node 694092bb575bc060ce160b60: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092bb575bc060ce160b60 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092bb575bc060ce160b60 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092bb575bc060ce160b60 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:54,937 - lava_callback - INFO - Starting async job submit for node 694092e4575bc060ce160bb8 2025-12-17 23:10:54,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:54,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e4575bc060ce160bb8 HTTP/1.1" 502 150 2025-12-17 23:10:54,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e4575bc060ce160bb8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:54,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e4575bc060ce160bb8 2025-12-17 23:10:54,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e4575bc060ce160bb8 HTTP/1.1" 502 150 2025-12-17 23:10:54,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e4575bc060ce160bb8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:55,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ce575bc060ce160b78 2025-12-17 23:10:55,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ce575bc060ce160b78 HTTP/1.1" 502 150 2025-12-17 23:10:55,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ce575bc060ce160b78'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:55,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092bd575bc060ce160b65 2025-12-17 23:10:55,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bd575bc060ce160b65 HTTP/1.1" 502 150 2025-12-17 23:10:55,625 - lava_callback - ERROR - Error processing callback for node 694092bd575bc060ce160b65: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092bd575bc060ce160b65 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092bd575bc060ce160b65 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092bd575bc060ce160b65 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:55,627 - lava_callback - INFO - Starting async job submit for node 694092e7575bc060ce160bbd 2025-12-17 23:10:55,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:55,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e7575bc060ce160bbd HTTP/1.1" 502 150 2025-12-17 23:10:55,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e7575bc060ce160bbd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:55,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e7575bc060ce160bbd 2025-12-17 23:10:55,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e7575bc060ce160bbd HTTP/1.1" 502 150 2025-12-17 23:10:55,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e7575bc060ce160bbd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:55,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092bf575bc060ce160b68 2025-12-17 23:10:55,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092bf575bc060ce160b68 HTTP/1.1" 502 150 2025-12-17 23:10:55,958 - lava_callback - ERROR - Error processing callback for node 694092bf575bc060ce160b68: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092bf575bc060ce160b68 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092bf575bc060ce160b68 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092bf575bc060ce160b68 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:10:55,960 - lava_callback - INFO - Starting async job submit for node 694092e8575bc060ce160bc5 2025-12-17 23:10:55,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:10:55,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e8575bc060ce160bc5 HTTP/1.1" 502 150 2025-12-17 23:10:55,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e8575bc060ce160bc5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:55,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e8575bc060ce160bc5 2025-12-17 23:10:55,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e8575bc060ce160bc5 HTTP/1.1" 502 150 2025-12-17 23:10:55,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e8575bc060ce160bc5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58080 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:10:56,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e2575bc060ce160baf 2025-12-17 23:10:56,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e2575bc060ce160baf HTTP/1.1" 502 150 2025-12-17 23:10:56,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e2575bc060ce160baf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:56,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d1575bc060ce160b83 2025-12-17 23:10:56,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d1575bc060ce160b83 HTTP/1.1" 502 150 2025-12-17 23:10:56,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d1575bc060ce160b83'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:56,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092da575bc060ce160b99 2025-12-17 23:10:56,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092da575bc060ce160b99 HTTP/1.1" 502 150 2025-12-17 23:10:56,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092da575bc060ce160b99'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:56,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d9575bc060ce160b96 2025-12-17 23:10:56,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d9575bc060ce160b96 HTTP/1.1" 502 150 2025-12-17 23:10:56,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d9575bc060ce160b96'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:56,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e4575bc060ce160bb8 2025-12-17 23:10:56,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e4575bc060ce160bb8 HTTP/1.1" 502 150 2025-12-17 23:10:56,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e4575bc060ce160bb8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:56,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092dd575bc060ce160b9c 2025-12-17 23:10:56,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092dd575bc060ce160b9c HTTP/1.1" 502 150 2025-12-17 23:10:56,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092dd575bc060ce160b9c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:57,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e7575bc060ce160bbd 2025-12-17 23:10:57,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e7575bc060ce160bbd HTTP/1.1" 502 150 2025-12-17 23:10:57,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e7575bc060ce160bbd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:57,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e8575bc060ce160bc5 2025-12-17 23:10:57,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e8575bc060ce160bc5 HTTP/1.1" 502 150 2025-12-17 23:10:57,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e8575bc060ce160bc5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:58,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e0575bc060ce160ba2 2025-12-17 23:10:58,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e0575bc060ce160ba2 HTTP/1.1" 502 150 2025-12-17 23:10:58,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e0575bc060ce160ba2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:10:58,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092df575bc060ce160b9f 2025-12-17 23:10:58,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092df575bc060ce160b9f HTTP/1.1" 502 150 2025-12-17 23:10:58,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092df575bc060ce160b9f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:00,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e2575bc060ce160baf 2025-12-17 23:11:00,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e2575bc060ce160baf HTTP/1.1" 502 150 2025-12-17 23:11:00,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e2575bc060ce160baf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:00,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e4575bc060ce160bb8 2025-12-17 23:11:00,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e4575bc060ce160bb8 HTTP/1.1" 502 150 2025-12-17 23:11:00,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e4575bc060ce160bb8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:01,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e7575bc060ce160bbd 2025-12-17 23:11:01,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e7575bc060ce160bbd HTTP/1.1" 502 150 2025-12-17 23:11:01,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e7575bc060ce160bbd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:01,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940939a575bc060ce160de3 2025-12-17 23:11:01,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939a575bc060ce160de3 HTTP/1.1" 502 150 2025-12-17 23:11:01,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940939a575bc060ce160de3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:01,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e8575bc060ce160bc5 2025-12-17 23:11:01,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e8575bc060ce160bc5 HTTP/1.1" 502 150 2025-12-17 23:11:01,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e8575bc060ce160bc5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52038 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:11:04,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d7575bc060ce160b8e HTTP/1.1" 502 150 2025-12-17 23:11:04,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d7575bc060ce160b8e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:04,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d7575bc060ce160b8e 2025-12-17 23:11:04,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d7575bc060ce160b8e HTTP/1.1" 502 150 2025-12-17 23:11:04,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d7575bc060ce160b8e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:04,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d9575bc060ce160b96 2025-12-17 23:11:04,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d9575bc060ce160b96 HTTP/1.1" 502 150 2025-12-17 23:11:04,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d9575bc060ce160b96'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:06,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d7575bc060ce160b8e 2025-12-17 23:11:06,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d7575bc060ce160b8e HTTP/1.1" 502 150 2025-12-17 23:11:06,371 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d7575bc060ce160b8e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52048 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:11:06,692 - lava_callback - INFO - Received callback for node 69409577575bc060ce16133f INFO: 172.21.0.1:52054 - "POST /node/69409577575bc060ce16133f HTTP/1.0" 202 Accepted 2025-12-17 23:11:06,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092df575bc060ce160b9f 2025-12-17 23:11:06,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092df575bc060ce160b9f HTTP/1.1" 502 150 2025-12-17 23:11:06,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092df575bc060ce160b9f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:07,342 - lava_callback - INFO - Received callback for node 69409579575bc060ce161346 INFO: 172.21.0.1:52058 - "POST /node/69409579575bc060ce161346 HTTP/1.0" 202 Accepted 2025-12-17 23:11:07,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092c9575bc060ce160b6b 2025-12-17 23:11:07,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092c9575bc060ce160b6b HTTP/1.1" 502 150 2025-12-17 23:11:07,919 - lava_callback - ERROR - Error processing callback for node 694092c9575bc060ce160b6b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092c9575bc060ce160b6b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092c9575bc060ce160b6b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092c9575bc060ce160b6b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:07,921 - lava_callback - INFO - Starting async job submit for node 694092ea575bc060ce160bce 2025-12-17 23:11:07,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:07,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ea575bc060ce160bce HTTP/1.1" 502 150 2025-12-17 23:11:07,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ea575bc060ce160bce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:07,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ea575bc060ce160bce 2025-12-17 23:11:07,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ea575bc060ce160bce HTTP/1.1" 502 150 2025-12-17 23:11:07,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ea575bc060ce160bce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:07,980 - lava_callback - INFO - Received callback for node 6940957b575bc060ce16134d INFO: 172.21.0.1:52072 - "POST /node/6940957b575bc060ce16134d HTTP/1.0" 202 Accepted 2025-12-17 23:11:08,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e2575bc060ce160baf 2025-12-17 23:11:08,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e2575bc060ce160baf HTTP/1.1" 502 150 2025-12-17 23:11:08,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e2575bc060ce160baf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:08,634 - lava_callback - INFO - Received callback for node 6940957d575bc060ce161350 INFO: 172.21.0.1:52076 - "POST /node/6940957d575bc060ce161350 HTTP/1.0" 202 Accepted 2025-12-17 23:11:08,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e4575bc060ce160bb8 2025-12-17 23:11:08,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e4575bc060ce160bb8 HTTP/1.1" 502 150 2025-12-17 23:11:08,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e4575bc060ce160bb8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:09,296 - lava_callback - INFO - Received callback for node 6940957f575bc060ce161356 INFO: 172.21.0.1:52090 - "POST /node/6940957f575bc060ce161356 HTTP/1.0" 202 Accepted 2025-12-17 23:11:09,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e7575bc060ce160bbd 2025-12-17 23:11:09,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e7575bc060ce160bbd HTTP/1.1" 502 150 2025-12-17 23:11:09,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e7575bc060ce160bbd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:09,939 - lava_callback - INFO - Received callback for node 69409581575bc060ce161359 INFO: 172.21.0.1:38468 - "POST /node/69409581575bc060ce161359 HTTP/1.0" 202 Accepted 2025-12-17 23:11:09,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ea575bc060ce160bce 2025-12-17 23:11:09,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ea575bc060ce160bce HTTP/1.1" 502 150 2025-12-17 23:11:09,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ea575bc060ce160bce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:09,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e8575bc060ce160bc5 2025-12-17 23:11:09,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e8575bc060ce160bc5 HTTP/1.1" 502 150 2025-12-17 23:11:09,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092e8575bc060ce160bc5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:10,371 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d7575bc060ce160b8e 2025-12-17 23:11:10,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d7575bc060ce160b8e HTTP/1.1" 502 150 2025-12-17 23:11:10,372 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d7575bc060ce160b8e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:10,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092cc575bc060ce160b71 2025-12-17 23:11:10,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092cc575bc060ce160b71 HTTP/1.1" 502 150 2025-12-17 23:11:10,575 - lava_callback - ERROR - Error processing callback for node 694092cc575bc060ce160b71: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092cc575bc060ce160b71 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092cc575bc060ce160b71 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092cc575bc060ce160b71 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:10,576 - lava_callback - INFO - Starting async job submit for node 694099b1575bc060ce161d96 2025-12-17 23:11:10,610 - lava_callback - INFO - Received callback for node 69409584575bc060ce16135c INFO: 172.21.0.1:38472 - "POST /node/69409584575bc060ce16135c HTTP/1.0" 202 Accepted 2025-12-17 23:11:10,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:10,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b1575bc060ce161d96 HTTP/1.1" 502 150 2025-12-17 23:11:10,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b1575bc060ce161d96'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:10,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b1575bc060ce161d96 2025-12-17 23:11:10,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b1575bc060ce161d96 HTTP/1.1" 502 150 2025-12-17 23:11:10,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b1575bc060ce161d96'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:10,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d5575bc060ce160b8b 2025-12-17 23:11:10,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d5575bc060ce160b8b HTTP/1.1" 502 150 2025-12-17 23:11:10,708 - lava_callback - ERROR - Error processing callback for node 694092d5575bc060ce160b8b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092d5575bc060ce160b8b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092d5575bc060ce160b8b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092d5575bc060ce160b8b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:10,710 - lava_callback - INFO - Starting async job submit for node 694092eb575bc060ce160bd1 2025-12-17 23:11:10,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:10,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092eb575bc060ce160bd1 HTTP/1.1" 502 150 2025-12-17 23:11:10,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092eb575bc060ce160bd1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:10,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092eb575bc060ce160bd1 2025-12-17 23:11:10,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092eb575bc060ce160bd1 HTTP/1.1" 502 150 2025-12-17 23:11:10,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092eb575bc060ce160bd1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:11,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ce575bc060ce160b78 2025-12-17 23:11:11,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ce575bc060ce160b78 HTTP/1.1" 502 150 2025-12-17 23:11:11,350 - lava_callback - ERROR - Error processing callback for node 694092ce575bc060ce160b78: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ce575bc060ce160b78 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ce575bc060ce160b78 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ce575bc060ce160b78 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:11,351 - lava_callback - INFO - Starting async job submit for node 694111a8460bc8e265117c2f 2025-12-17 23:11:11,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:11,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694111a8460bc8e265117c2f HTTP/1.1" 502 150 2025-12-17 23:11:11,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694111a8460bc8e265117c2f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:11,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694111a8460bc8e265117c2f 2025-12-17 23:11:11,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694111a8460bc8e265117c2f HTTP/1.1" 502 150 2025-12-17 23:11:11,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694111a8460bc8e265117c2f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:12,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d1575bc060ce160b83 2025-12-17 23:11:12,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d1575bc060ce160b83 HTTP/1.1" 502 150 2025-12-17 23:11:12,631 - lava_callback - ERROR - Error processing callback for node 694092d1575bc060ce160b83: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092d1575bc060ce160b83 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092d1575bc060ce160b83 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092d1575bc060ce160b83 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:12,633 - lava_callback - INFO - Starting async job submit for node 694092ef575bc060ce160bda 2025-12-17 23:11:12,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:12,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b1575bc060ce161d96 2025-12-17 23:11:12,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b1575bc060ce161d96 HTTP/1.1" 502 150 2025-12-17 23:11:12,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b1575bc060ce161d96'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:12,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ef575bc060ce160bda HTTP/1.1" 502 150 2025-12-17 23:11:12,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ef575bc060ce160bda'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:12,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ef575bc060ce160bda 2025-12-17 23:11:12,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ef575bc060ce160bda HTTP/1.1" 502 150 2025-12-17 23:11:12,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ef575bc060ce160bda'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:12,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092da575bc060ce160b99 2025-12-17 23:11:12,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092da575bc060ce160b99 HTTP/1.1" 502 150 2025-12-17 23:11:12,706 - lava_callback - ERROR - Error processing callback for node 694092da575bc060ce160b99: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092da575bc060ce160b99 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092da575bc060ce160b99 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092da575bc060ce160b99 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:12,708 - lava_callback - INFO - Starting async job submit for node 694092f1575bc060ce160bdd 2025-12-17 23:11:12,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:12,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f1575bc060ce160bdd HTTP/1.1" 502 150 2025-12-17 23:11:12,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f1575bc060ce160bdd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:12,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f1575bc060ce160bdd 2025-12-17 23:11:12,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f1575bc060ce160bdd HTTP/1.1" 502 150 2025-12-17 23:11:12,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f1575bc060ce160bdd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:12,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092eb575bc060ce160bd1 2025-12-17 23:11:12,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092eb575bc060ce160bd1 HTTP/1.1" 502 150 2025-12-17 23:11:12,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092eb575bc060ce160bd1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:12,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092dd575bc060ce160b9c 2025-12-17 23:11:12,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092dd575bc060ce160b9c HTTP/1.1" 502 150 2025-12-17 23:11:12,999 - lava_callback - ERROR - Error processing callback for node 694092dd575bc060ce160b9c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092dd575bc060ce160b9c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092dd575bc060ce160b9c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092dd575bc060ce160b9c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:13,001 - lava_callback - INFO - Starting async job submit for node 694092f3575bc060ce160be0 2025-12-17 23:11:13,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:13,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f3575bc060ce160be0 HTTP/1.1" 502 150 2025-12-17 23:11:13,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f3575bc060ce160be0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:13,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f3575bc060ce160be0 2025-12-17 23:11:13,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f3575bc060ce160be0 HTTP/1.1" 502 150 2025-12-17 23:11:13,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f3575bc060ce160be0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:13,381 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694111a8460bc8e265117c2f 2025-12-17 23:11:13,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694111a8460bc8e265117c2f HTTP/1.1" 502 150 2025-12-17 23:11:13,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694111a8460bc8e265117c2f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:13,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ea575bc060ce160bce 2025-12-17 23:11:13,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ea575bc060ce160bce HTTP/1.1" 502 150 2025-12-17 23:11:13,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ea575bc060ce160bce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:14,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ef575bc060ce160bda 2025-12-17 23:11:14,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ef575bc060ce160bda HTTP/1.1" 502 150 2025-12-17 23:11:14,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ef575bc060ce160bda'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:14,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f1575bc060ce160bdd 2025-12-17 23:11:14,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f1575bc060ce160bdd HTTP/1.1" 502 150 2025-12-17 23:11:14,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f1575bc060ce160bdd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:14,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e0575bc060ce160ba2 2025-12-17 23:11:14,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e0575bc060ce160ba2 HTTP/1.1" 502 150 2025-12-17 23:11:14,763 - lava_callback - ERROR - Error processing callback for node 694092e0575bc060ce160ba2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092e0575bc060ce160ba2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092e0575bc060ce160ba2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092e0575bc060ce160ba2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:14,764 - lava_callback - INFO - Starting async job submit for node 694092ed575bc060ce160bd4 2025-12-17 23:11:14,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:14,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ed575bc060ce160bd4 HTTP/1.1" 502 150 2025-12-17 23:11:14,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ed575bc060ce160bd4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:14,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ed575bc060ce160bd4 2025-12-17 23:11:14,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ed575bc060ce160bd4 HTTP/1.1" 502 150 2025-12-17 23:11:14,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ed575bc060ce160bd4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:15,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f3575bc060ce160be0 INFO: 172.21.0.1:38484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38486 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:11:16,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b1575bc060ce161d96 2025-12-17 23:11:16,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b1575bc060ce161d96 HTTP/1.1" 502 150 2025-12-17 23:11:16,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b1575bc060ce161d96'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:16,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092eb575bc060ce160bd1 2025-12-17 23:11:16,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092eb575bc060ce160bd1 HTTP/1.1" 502 150 2025-12-17 23:11:16,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092eb575bc060ce160bd1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:16,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ed575bc060ce160bd4 2025-12-17 23:11:16,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ed575bc060ce160bd4 HTTP/1.1" 502 150 2025-12-17 23:11:16,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ed575bc060ce160bd4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:17,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694111a8460bc8e265117c2f 2025-12-17 23:11:17,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694111a8460bc8e265117c2f HTTP/1.1" 502 150 2025-12-17 23:11:17,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694111a8460bc8e265117c2f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:17,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940939a575bc060ce160de3 2025-12-17 23:11:17,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939a575bc060ce160de3 HTTP/1.1" 502 150 2025-12-17 23:11:17,883 - lava_callback - ERROR - Error processing callback for node 6940939a575bc060ce160de3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940939a575bc060ce160de3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940939a575bc060ce160de3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940939a575bc060ce160de3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:17,885 - lava_callback - INFO - Starting async job submit for node 694092ee575bc060ce160bd7 2025-12-17 23:11:17,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:17,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ee575bc060ce160bd7 HTTP/1.1" 502 150 2025-12-17 23:11:17,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ee575bc060ce160bd7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:17,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ee575bc060ce160bd7 2025-12-17 23:11:17,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ee575bc060ce160bd7 HTTP/1.1" 502 150 2025-12-17 23:11:17,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ee575bc060ce160bd7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:18,372 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d7575bc060ce160b8e 2025-12-17 23:11:18,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d7575bc060ce160b8e HTTP/1.1" 502 150 2025-12-17 23:11:18,374 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092d7575bc060ce160b8e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:18,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ef575bc060ce160bda 2025-12-17 23:11:18,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ef575bc060ce160bda HTTP/1.1" 502 150 2025-12-17 23:11:18,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ef575bc060ce160bda'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:18,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f1575bc060ce160bdd 2025-12-17 23:11:18,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f1575bc060ce160bdd HTTP/1.1" 502 150 2025-12-17 23:11:18,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f1575bc060ce160bdd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:19,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ee575bc060ce160bd7 2025-12-17 23:11:19,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ee575bc060ce160bd7 HTTP/1.1" 502 150 2025-12-17 23:11:19,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ee575bc060ce160bd7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:20,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ed575bc060ce160bd4 2025-12-17 23:11:20,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ed575bc060ce160bd4 HTTP/1.1" 502 150 2025-12-17 23:11:20,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ed575bc060ce160bd4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:20,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d9575bc060ce160b96 2025-12-17 23:11:20,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d9575bc060ce160b96 HTTP/1.1" 502 150 2025-12-17 23:11:20,787 - lava_callback - ERROR - Error processing callback for node 694092d9575bc060ce160b96: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092d9575bc060ce160b96 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092d9575bc060ce160b96 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092d9575bc060ce160b96 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:20,789 - lava_callback - INFO - Starting async job submit for node 694092f6575bc060ce160be9 2025-12-17 23:11:20,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:20,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f6575bc060ce160be9 HTTP/1.1" 502 150 2025-12-17 23:11:20,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f6575bc060ce160be9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:20,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f6575bc060ce160be9 2025-12-17 23:11:20,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f6575bc060ce160be9 HTTP/1.1" 502 150 2025-12-17 23:11:20,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f6575bc060ce160be9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:21,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ea575bc060ce160bce 2025-12-17 23:11:21,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ea575bc060ce160bce HTTP/1.1" 502 150 2025-12-17 23:11:21,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ea575bc060ce160bce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:22,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f6575bc060ce160be9 2025-12-17 23:11:22,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f6575bc060ce160be9 HTTP/1.1" 502 150 2025-12-17 23:11:22,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f6575bc060ce160be9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:22,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092df575bc060ce160b9f 2025-12-17 23:11:22,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092df575bc060ce160b9f HTTP/1.1" 502 150 2025-12-17 23:11:22,897 - lava_callback - ERROR - Error processing callback for node 694092df575bc060ce160b9f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092df575bc060ce160b9f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092df575bc060ce160b9f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092df575bc060ce160b9f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:22,898 - lava_callback - INFO - Starting async job submit for node 694092f8575bc060ce160bee 2025-12-17 23:11:22,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:22,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f8575bc060ce160bee HTTP/1.1" 502 150 2025-12-17 23:11:22,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f8575bc060ce160bee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:22,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f8575bc060ce160bee 2025-12-17 23:11:22,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f8575bc060ce160bee HTTP/1.1" 502 150 2025-12-17 23:11:22,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f8575bc060ce160bee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:23,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ee575bc060ce160bd7 2025-12-17 23:11:23,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ee575bc060ce160bd7 HTTP/1.1" 502 150 2025-12-17 23:11:23,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ee575bc060ce160bd7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:24,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e2575bc060ce160baf 2025-12-17 23:11:24,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e2575bc060ce160baf HTTP/1.1" 502 150 2025-12-17 23:11:24,534 - lava_callback - ERROR - Error processing callback for node 694092e2575bc060ce160baf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092e2575bc060ce160baf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092e2575bc060ce160baf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092e2575bc060ce160baf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:24,536 - lava_callback - INFO - Starting async job submit for node 694092fa575bc060ce160bf1 2025-12-17 23:11:24,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:24,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fa575bc060ce160bf1 HTTP/1.1" 502 150 2025-12-17 23:11:24,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092fa575bc060ce160bf1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:24,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092fa575bc060ce160bf1 2025-12-17 23:11:24,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fa575bc060ce160bf1 HTTP/1.1" 502 150 2025-12-17 23:11:24,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092fa575bc060ce160bf1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:24,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b1575bc060ce161d96 2025-12-17 23:11:24,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b1575bc060ce161d96 HTTP/1.1" 502 150 2025-12-17 23:11:24,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b1575bc060ce161d96'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:24,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092eb575bc060ce160bd1 2025-12-17 23:11:24,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092eb575bc060ce160bd1 HTTP/1.1" 502 150 2025-12-17 23:11:24,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092eb575bc060ce160bd1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:24,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f8575bc060ce160bee 2025-12-17 23:11:24,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f8575bc060ce160bee HTTP/1.1" 502 150 2025-12-17 23:11:24,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f8575bc060ce160bee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:24,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e4575bc060ce160bb8 2025-12-17 23:11:24,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e4575bc060ce160bb8 HTTP/1.1" 502 150 2025-12-17 23:11:24,952 - lava_callback - ERROR - Error processing callback for node 694092e4575bc060ce160bb8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092e4575bc060ce160bb8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092e4575bc060ce160bb8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092e4575bc060ce160bb8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:24,954 - lava_callback - INFO - Starting async job submit for node 694092fc575bc060ce160bf4 2025-12-17 23:11:24,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:24,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fc575bc060ce160bf4 HTTP/1.1" 502 150 2025-12-17 23:11:24,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092fc575bc060ce160bf4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:24,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092fc575bc060ce160bf4 2025-12-17 23:11:24,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fc575bc060ce160bf4 HTTP/1.1" 502 150 2025-12-17 23:11:24,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092fc575bc060ce160bf4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:25,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694111a8460bc8e265117c2f 2025-12-17 23:11:25,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694111a8460bc8e265117c2f HTTP/1.1" 502 150 2025-12-17 23:11:25,385 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694111a8460bc8e265117c2f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:25,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e7575bc060ce160bbd 2025-12-17 23:11:25,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e7575bc060ce160bbd HTTP/1.1" 502 150 2025-12-17 23:11:25,642 - lava_callback - ERROR - Error processing callback for node 694092e7575bc060ce160bbd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092e7575bc060ce160bbd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092e7575bc060ce160bbd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092e7575bc060ce160bbd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:25,644 - lava_callback - INFO - Starting async job submit for node 69409308575bc060ce160c08 2025-12-17 23:11:25,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:25,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409308575bc060ce160c08 HTTP/1.1" 502 150 2025-12-17 23:11:25,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409308575bc060ce160c08'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:25,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409308575bc060ce160c08 2025-12-17 23:11:25,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409308575bc060ce160c08 HTTP/1.1" 502 150 2025-12-17 23:11:25,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409308575bc060ce160c08'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:25,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092e8575bc060ce160bc5 2025-12-17 23:11:25,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092e8575bc060ce160bc5 HTTP/1.1" 502 150 2025-12-17 23:11:25,978 - lava_callback - ERROR - Error processing callback for node 694092e8575bc060ce160bc5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092e8575bc060ce160bc5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092e8575bc060ce160bc5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092e8575bc060ce160bc5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:25,980 - lava_callback - INFO - Starting async job submit for node 69409301575bc060ce160bff 2025-12-17 23:11:25,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:25,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409301575bc060ce160bff HTTP/1.1" 502 150 2025-12-17 23:11:25,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409301575bc060ce160bff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:25,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409301575bc060ce160bff 2025-12-17 23:11:25,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409301575bc060ce160bff HTTP/1.1" 502 150 2025-12-17 23:11:25,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409301575bc060ce160bff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57734 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:11:26,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092fa575bc060ce160bf1 2025-12-17 23:11:26,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fa575bc060ce160bf1 HTTP/1.1" 502 150 2025-12-17 23:11:26,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092fa575bc060ce160bf1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:26,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ef575bc060ce160bda 2025-12-17 23:11:26,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ef575bc060ce160bda HTTP/1.1" 502 150 2025-12-17 23:11:26,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ef575bc060ce160bda'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:26,697 - lava_callback - INFO - Received callback for node 6940958a575bc060ce161365 INFO: 172.21.0.1:57742 - "POST /node/6940958a575bc060ce161365 HTTP/1.0" 202 Accepted 2025-12-17 23:11:26,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f1575bc060ce160bdd 2025-12-17 23:11:26,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f1575bc060ce160bdd HTTP/1.1" 502 150 2025-12-17 23:11:26,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f1575bc060ce160bdd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:26,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f6575bc060ce160be9 2025-12-17 23:11:26,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f6575bc060ce160be9 HTTP/1.1" 502 150 2025-12-17 23:11:26,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f6575bc060ce160be9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:26,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092fc575bc060ce160bf4 2025-12-17 23:11:26,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fc575bc060ce160bf4 HTTP/1.1" 502 150 2025-12-17 23:11:26,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092fc575bc060ce160bf4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:27,360 - lava_callback - INFO - Received callback for node 6940958c575bc060ce16136e INFO: 172.21.0.1:57750 - "POST /node/6940958c575bc060ce16136e HTTP/1.0" 202 Accepted 2025-12-17 23:11:27,483 - lava_callback - INFO - Received callback for node 69409593575bc060ce16137d INFO: 172.21.0.1:57762 - "POST /node/69409593575bc060ce16137d HTTP/1.0" 202 Accepted 2025-12-17 23:11:27,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409308575bc060ce160c08 2025-12-17 23:11:27,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409308575bc060ce160c08 HTTP/1.1" 502 150 2025-12-17 23:11:27,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409308575bc060ce160c08'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:27,997 - lava_callback - INFO - Received callback for node 6940958e575bc060ce161371 INFO: 172.21.0.1:57776 - "POST /node/6940958e575bc060ce161371 HTTP/1.0" 202 Accepted 2025-12-17 23:11:28,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409301575bc060ce160bff 2025-12-17 23:11:28,644 - lava_callback - INFO - Received callback for node 6940958f575bc060ce161374 INFO: 172.21.0.1:57784 - "POST /node/6940958f575bc060ce161374 HTTP/1.0" 202 Accepted 2025-12-17 23:11:28,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ed575bc060ce160bd4 2025-12-17 23:11:28,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ed575bc060ce160bd4 HTTP/1.1" 502 150 2025-12-17 23:11:28,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ed575bc060ce160bd4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:28,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f8575bc060ce160bee 2025-12-17 23:11:28,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f8575bc060ce160bee HTTP/1.1" 502 150 2025-12-17 23:11:28,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f8575bc060ce160bee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:29,297 - lava_callback - INFO - Received callback for node 69409590575bc060ce161377 INFO: 172.21.0.1:57786 - "POST /node/69409590575bc060ce161377 HTTP/1.0" 202 Accepted 2025-12-17 23:11:29,944 - lava_callback - INFO - Received callback for node 69409592575bc060ce16137a INFO: 172.21.0.1:50504 - "POST /node/69409592575bc060ce16137a HTTP/1.0" 202 Accepted 2025-12-17 23:11:30,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f3575bc060ce160be0 HTTP/1.1" 502 150 2025-12-17 23:11:30,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f3575bc060ce160be0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:30,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092fa575bc060ce160bf1 2025-12-17 23:11:30,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fa575bc060ce160bf1 HTTP/1.1" 502 150 2025-12-17 23:11:30,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092fa575bc060ce160bf1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:30,607 - lava_callback - INFO - Received callback for node 69409595575bc060ce161383 INFO: 172.21.0.1:50508 - "POST /node/69409595575bc060ce161383 HTTP/1.0" 202 Accepted 2025-12-17 23:11:30,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092fc575bc060ce160bf4 2025-12-17 23:11:30,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fc575bc060ce160bf4 HTTP/1.1" 502 150 2025-12-17 23:11:30,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092fc575bc060ce160bf4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:31,242 - lava_callback - INFO - Received callback for node 69409596575bc060ce161386 INFO: 172.21.0.1:50516 - "POST /node/69409596575bc060ce161386 HTTP/1.0" 202 Accepted 2025-12-17 23:11:31,249 - lava_callback - INFO - Received callback for node 69409594575bc060ce161380 INFO: 172.21.0.1:50526 - "POST /node/69409594575bc060ce161380 HTTP/1.0" 202 Accepted 2025-12-17 23:11:31,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409308575bc060ce160c08 2025-12-17 23:11:31,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409308575bc060ce160c08 HTTP/1.1" 502 150 2025-12-17 23:11:31,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409308575bc060ce160c08'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:31,888 - lava_callback - INFO - Received callback for node 69409598575bc060ce161389 INFO: 172.21.0.1:50528 - "POST /node/69409598575bc060ce161389 HTTP/1.0" 202 Accepted 2025-12-17 23:11:31,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ee575bc060ce160bd7 2025-12-17 23:11:31,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ee575bc060ce160bd7 HTTP/1.1" 502 150 2025-12-17 23:11:31,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092ee575bc060ce160bd7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:31,909 - lava_callback - INFO - Received callback for node 6940959c575bc060ce161392 INFO: 172.21.0.1:50530 - "POST /node/6940959c575bc060ce161392 HTTP/1.0" 202 Accepted 2025-12-17 23:11:32,524 - lava_callback - INFO - Received callback for node 69409599575bc060ce16138c INFO: 172.21.0.1:50546 - "POST /node/69409599575bc060ce16138c HTTP/1.0" 202 Accepted 2025-12-17 23:11:32,539 - lava_callback - INFO - Received callback for node 6940959d575bc060ce161395 INFO: 172.21.0.1:50554 - "POST /node/6940959d575bc060ce161395 HTTP/1.0" 202 Accepted 2025-12-17 23:11:33,167 - lava_callback - INFO - Received callback for node 6940959a575bc060ce16138f INFO: 172.21.0.1:50556 - "POST /node/6940959a575bc060ce16138f HTTP/1.0" 202 Accepted 2025-12-17 23:11:34,374 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092d7575bc060ce160b8e 2025-12-17 23:11:34,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092d7575bc060ce160b8e HTTP/1.1" 502 150 2025-12-17 23:11:34,375 - lava_callback - ERROR - Error processing callback for node 694092d7575bc060ce160b8e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092d7575bc060ce160b8e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092d7575bc060ce160b8e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092d7575bc060ce160b8e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:34,377 - lava_callback - INFO - Starting async job submit for node 69409304575bc060ce160c02 2025-12-17 23:11:34,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:34,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409304575bc060ce160c02 HTTP/1.1" 502 150 2025-12-17 23:11:34,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409304575bc060ce160c02'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:34,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409304575bc060ce160c02 2025-12-17 23:11:34,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409304575bc060ce160c02 HTTP/1.1" 502 150 2025-12-17 23:11:34,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409304575bc060ce160c02'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:34,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f3575bc060ce160be0 2025-12-17 23:11:34,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f3575bc060ce160be0 HTTP/1.1" 502 150 2025-12-17 23:11:34,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f3575bc060ce160be0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:34,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f6575bc060ce160be9 2025-12-17 23:11:34,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f6575bc060ce160be9 HTTP/1.1" 502 150 2025-12-17 23:11:34,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f6575bc060ce160be9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:35,665 - lava_callback - INFO - Received callback for node 69409843575bc060ce161a33 INFO: 172.21.0.1:50568 - "POST /node/69409843575bc060ce161a33 HTTP/1.0" 202 Accepted 2025-12-17 23:11:36,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409304575bc060ce160c02 2025-12-17 23:11:36,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409304575bc060ce160c02 HTTP/1.1" 502 150 2025-12-17 23:11:36,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409304575bc060ce160c02'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50574 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:11:36,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f8575bc060ce160bee 2025-12-17 23:11:36,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f8575bc060ce160bee HTTP/1.1" 502 150 2025-12-17 23:11:36,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f8575bc060ce160bee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:37,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ea575bc060ce160bce 2025-12-17 23:11:37,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ea575bc060ce160bce HTTP/1.1" 502 150 2025-12-17 23:11:37,953 - lava_callback - ERROR - Error processing callback for node 694092ea575bc060ce160bce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ea575bc060ce160bce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ea575bc060ce160bce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ea575bc060ce160bce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:37,955 - lava_callback - INFO - Starting async job submit for node 69409306575bc060ce160c05 2025-12-17 23:11:37,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:37,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409306575bc060ce160c05 HTTP/1.1" 502 150 2025-12-17 23:11:37,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409306575bc060ce160c05'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:37,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409306575bc060ce160c05 2025-12-17 23:11:37,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409306575bc060ce160c05 HTTP/1.1" 502 150 2025-12-17 23:11:37,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409306575bc060ce160c05'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:38,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092fa575bc060ce160bf1 2025-12-17 23:11:38,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fa575bc060ce160bf1 HTTP/1.1" 502 150 2025-12-17 23:11:38,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092fa575bc060ce160bf1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:38,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092fc575bc060ce160bf4 2025-12-17 23:11:38,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fc575bc060ce160bf4 HTTP/1.1" 502 150 2025-12-17 23:11:38,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092fc575bc060ce160bf4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:39,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409308575bc060ce160c08 2025-12-17 23:11:39,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409308575bc060ce160c08 HTTP/1.1" 502 150 2025-12-17 23:11:39,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409308575bc060ce160c08'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:39,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409306575bc060ce160c05 2025-12-17 23:11:39,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409306575bc060ce160c05 HTTP/1.1" 502 150 2025-12-17 23:11:39,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409306575bc060ce160c05'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:40,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409304575bc060ce160c02 2025-12-17 23:11:40,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409304575bc060ce160c02 HTTP/1.1" 502 150 2025-12-17 23:11:40,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409304575bc060ce160c02'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:40,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b1575bc060ce161d96 2025-12-17 23:11:40,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b1575bc060ce161d96 HTTP/1.1" 502 150 2025-12-17 23:11:40,651 - lava_callback - ERROR - Error processing callback for node 694099b1575bc060ce161d96: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099b1575bc060ce161d96 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099b1575bc060ce161d96 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099b1575bc060ce161d96 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:40,653 - lava_callback - INFO - Starting async job submit for node 69409a1a575bc060ce161ee2 2025-12-17 23:11:40,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:40,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092eb575bc060ce160bd1 2025-12-17 23:11:40,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1a575bc060ce161ee2 HTTP/1.1" 502 150 2025-12-17 23:11:40,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a1a575bc060ce161ee2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:40,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a1a575bc060ce161ee2 2025-12-17 23:11:40,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092eb575bc060ce160bd1 HTTP/1.1" 502 150 2025-12-17 23:11:40,723 - lava_callback - ERROR - Error processing callback for node 694092eb575bc060ce160bd1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092eb575bc060ce160bd1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092eb575bc060ce160bd1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092eb575bc060ce160bd1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:40,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1a575bc060ce161ee2 HTTP/1.1" 502 150 2025-12-17 23:11:40,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a1a575bc060ce161ee2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:40,725 - lava_callback - INFO - Starting async job submit for node 6940930b575bc060ce160c13 2025-12-17 23:11:40,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:40,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930b575bc060ce160c13 HTTP/1.1" 502 150 2025-12-17 23:11:40,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940930b575bc060ce160c13'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:40,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940930b575bc060ce160c13 2025-12-17 23:11:40,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930b575bc060ce160c13 HTTP/1.1" 502 150 2025-12-17 23:11:40,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940930b575bc060ce160c13'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:41,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694111a8460bc8e265117c2f 2025-12-17 23:11:42,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f3575bc060ce160be0 2025-12-17 23:11:42,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f3575bc060ce160be0 HTTP/1.1" 502 150 2025-12-17 23:11:42,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694092f3575bc060ce160be0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:42,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ef575bc060ce160bda 2025-12-17 23:11:42,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ef575bc060ce160bda HTTP/1.1" 502 150 2025-12-17 23:11:42,658 - lava_callback - ERROR - Error processing callback for node 694092ef575bc060ce160bda: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ef575bc060ce160bda (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ef575bc060ce160bda (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ef575bc060ce160bda (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:42,661 - lava_callback - INFO - Starting async job submit for node 6940930d575bc060ce160c1a 2025-12-17 23:11:42,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:42,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930d575bc060ce160c1a HTTP/1.1" 502 150 2025-12-17 23:11:42,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940930d575bc060ce160c1a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:42,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940930d575bc060ce160c1a 2025-12-17 23:11:42,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930d575bc060ce160c1a HTTP/1.1" 502 150 2025-12-17 23:11:42,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940930d575bc060ce160c1a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:42,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f1575bc060ce160bdd 2025-12-17 23:11:42,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f1575bc060ce160bdd HTTP/1.1" 502 150 2025-12-17 23:11:42,724 - lava_callback - ERROR - Error processing callback for node 694092f1575bc060ce160bdd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092f1575bc060ce160bdd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092f1575bc060ce160bdd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092f1575bc060ce160bdd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:42,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a1a575bc060ce161ee2 2025-12-17 23:11:42,726 - lava_callback - INFO - Starting async job submit for node 6940930f575bc060ce160c1d 2025-12-17 23:11:42,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:42,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1a575bc060ce161ee2 HTTP/1.1" 502 150 2025-12-17 23:11:42,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a1a575bc060ce161ee2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:42,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940930b575bc060ce160c13 2025-12-17 23:11:42,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930b575bc060ce160c13 HTTP/1.1" 502 150 2025-12-17 23:11:42,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940930b575bc060ce160c13'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:42,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930f575bc060ce160c1d HTTP/1.1" 502 150 2025-12-17 23:11:42,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940930f575bc060ce160c1d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:42,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940930f575bc060ce160c1d 2025-12-17 23:11:42,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930f575bc060ce160c1d HTTP/1.1" 502 150 2025-12-17 23:11:42,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940930f575bc060ce160c1d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:43,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409301575bc060ce160bff HTTP/1.1" 502 150 2025-12-17 23:11:43,281 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409301575bc060ce160bff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:43,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409306575bc060ce160c05 2025-12-17 23:11:43,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409306575bc060ce160c05 HTTP/1.1" 502 150 2025-12-17 23:11:43,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409306575bc060ce160c05'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:44,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940930d575bc060ce160c1a 2025-12-17 23:11:44,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930d575bc060ce160c1a HTTP/1.1" 502 150 2025-12-17 23:11:44,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940930d575bc060ce160c1a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:44,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ed575bc060ce160bd4 2025-12-17 23:11:44,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940930f575bc060ce160c1d 2025-12-17 23:11:44,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930f575bc060ce160c1d HTTP/1.1" 502 150 2025-12-17 23:11:44,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940930f575bc060ce160c1d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:44,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ed575bc060ce160bd4 HTTP/1.1" 502 150 2025-12-17 23:11:44,782 - lava_callback - ERROR - Error processing callback for node 694092ed575bc060ce160bd4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ed575bc060ce160bd4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ed575bc060ce160bd4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ed575bc060ce160bd4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:44,784 - lava_callback - INFO - Starting async job submit for node 69409311575bc060ce160c20 2025-12-17 23:11:44,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:44,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409311575bc060ce160c20 HTTP/1.1" 502 150 2025-12-17 23:11:44,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409311575bc060ce160c20'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:44,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409311575bc060ce160c20 2025-12-17 23:11:44,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409311575bc060ce160c20 HTTP/1.1" 502 150 2025-12-17 23:11:44,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409311575bc060ce160c20'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59024 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:11:46,713 - lava_callback - INFO - Received callback for node 6940959e575bc060ce16139c INFO: 172.21.0.1:59036 - "POST /node/6940959e575bc060ce16139c HTTP/1.0" 202 Accepted 2025-12-17 23:11:46,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a1a575bc060ce161ee2 2025-12-17 23:11:46,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1a575bc060ce161ee2 HTTP/1.1" 502 150 2025-12-17 23:11:46,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a1a575bc060ce161ee2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:46,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940930b575bc060ce160c13 2025-12-17 23:11:46,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930b575bc060ce160c13 HTTP/1.1" 502 150 2025-12-17 23:11:46,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940930b575bc060ce160c13'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:46,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409311575bc060ce160c20 2025-12-17 23:11:46,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409311575bc060ce160c20 HTTP/1.1" 502 150 2025-12-17 23:11:46,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409311575bc060ce160c20'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:47,281 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409301575bc060ce160bff 2025-12-17 23:11:47,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409301575bc060ce160bff HTTP/1.1" 502 150 2025-12-17 23:11:47,282 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409301575bc060ce160bff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:47,355 - lava_callback - INFO - Received callback for node 6940959f575bc060ce1613a3 INFO: 172.21.0.1:59050 - "POST /node/6940959f575bc060ce1613a3 HTTP/1.0" 202 Accepted 2025-12-17 23:11:47,506 - lava_callback - INFO - Received callback for node 694095a2575bc060ce1613ac INFO: 172.21.0.1:59062 - "POST /node/694095a2575bc060ce1613ac HTTP/1.0" 202 Accepted 2025-12-17 23:11:47,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092ee575bc060ce160bd7 2025-12-17 23:11:47,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092ee575bc060ce160bd7 HTTP/1.1" 502 150 2025-12-17 23:11:47,902 - lava_callback - ERROR - Error processing callback for node 694092ee575bc060ce160bd7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ee575bc060ce160bd7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ee575bc060ce160bd7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092ee575bc060ce160bd7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:47,904 - lava_callback - INFO - Starting async job submit for node 69409313575bc060ce160c23 2025-12-17 23:11:47,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:47,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409313575bc060ce160c23 HTTP/1.1" 502 150 2025-12-17 23:11:47,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409313575bc060ce160c23'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:47,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409313575bc060ce160c23 2025-12-17 23:11:47,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409313575bc060ce160c23 HTTP/1.1" 502 150 2025-12-17 23:11:47,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409313575bc060ce160c23'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:48,035 - lava_callback - INFO - Received callback for node 694095a2575bc060ce1613b1 INFO: 172.21.0.1:59072 - "POST /node/694095a2575bc060ce1613b1 HTTP/1.0" 202 Accepted 2025-12-17 23:11:48,279 - lava_callback - INFO - Received callback for node 694095a4575bc060ce1613bb INFO: 172.21.0.1:59088 - "POST /node/694095a4575bc060ce1613bb HTTP/1.0" 202 Accepted 2025-12-17 23:11:48,365 - lava_callback - INFO - Received callback for node 69411cc4459ff5fc64d2ab11 INFO: 172.21.0.1:59096 - "POST /node/69411cc4459ff5fc64d2ab11 HTTP/1.0" 202 Accepted 2025-12-17 23:11:48,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409304575bc060ce160c02 2025-12-17 23:11:48,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409304575bc060ce160c02 HTTP/1.1" 502 150 2025-12-17 23:11:48,389 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409304575bc060ce160c02'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:48,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940930d575bc060ce160c1a 2025-12-17 23:11:48,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930d575bc060ce160c1a HTTP/1.1" 502 150 2025-12-17 23:11:48,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940930d575bc060ce160c1a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:48,694 - lava_callback - INFO - Received callback for node 694095a5575bc060ce1613be INFO: 172.21.0.1:59112 - "POST /node/694095a5575bc060ce1613be HTTP/1.0" 202 Accepted 2025-12-17 23:11:48,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940930f575bc060ce160c1d 2025-12-17 23:11:48,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930f575bc060ce160c1d HTTP/1.1" 502 150 2025-12-17 23:11:48,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940930f575bc060ce160c1d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:49,359 - lava_callback - INFO - Received callback for node 694095a6575bc060ce1613c1 INFO: 172.21.0.1:59126 - "POST /node/694095a6575bc060ce1613c1 HTTP/1.0" 202 Accepted 2025-12-17 23:11:49,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409313575bc060ce160c23 2025-12-17 23:11:49,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409313575bc060ce160c23 HTTP/1.1" 502 150 2025-12-17 23:11:49,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409313575bc060ce160c23'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:50,003 - lava_callback - INFO - Received callback for node 694095a7575bc060ce1613c4 INFO: 172.21.0.1:40780 - "POST /node/694095a7575bc060ce1613c4 HTTP/1.0" 202 Accepted 2025-12-17 23:11:50,658 - lava_callback - INFO - Received callback for node 694095a9575bc060ce1613c7 INFO: 172.21.0.1:40786 - "POST /node/694095a9575bc060ce1613c7 HTTP/1.0" 202 Accepted 2025-12-17 23:11:50,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409311575bc060ce160c20 2025-12-17 23:11:50,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409311575bc060ce160c20 HTTP/1.1" 502 150 2025-12-17 23:11:50,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409311575bc060ce160c20'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:50,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f6575bc060ce160be9 2025-12-17 23:11:50,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f6575bc060ce160be9 HTTP/1.1" 502 150 2025-12-17 23:11:50,802 - lava_callback - ERROR - Error processing callback for node 694092f6575bc060ce160be9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092f6575bc060ce160be9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092f6575bc060ce160be9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092f6575bc060ce160be9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:50,804 - lava_callback - INFO - Starting async job submit for node 69409315575bc060ce160c26 2025-12-17 23:11:50,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:50,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409315575bc060ce160c26 HTTP/1.1" 502 150 2025-12-17 23:11:50,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409315575bc060ce160c26'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:50,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409315575bc060ce160c26 2025-12-17 23:11:50,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409315575bc060ce160c26 HTTP/1.1" 502 150 2025-12-17 23:11:50,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409315575bc060ce160c26'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:51,310 - lava_callback - INFO - Received callback for node 694095aa575bc060ce1613ca INFO: 172.21.0.1:40800 - "POST /node/694095aa575bc060ce1613ca HTTP/1.0" 202 Accepted 2025-12-17 23:11:51,944 - lava_callback - INFO - Received callback for node 694095ab575bc060ce1613cd INFO: 172.21.0.1:40806 - "POST /node/694095ab575bc060ce1613cd HTTP/1.0" 202 Accepted 2025-12-17 23:11:51,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409306575bc060ce160c05 2025-12-17 23:11:51,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409306575bc060ce160c05 HTTP/1.1" 502 150 2025-12-17 23:11:51,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409306575bc060ce160c05'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:52,595 - lava_callback - INFO - Received callback for node 694095ac575bc060ce1613d0 INFO: 172.21.0.1:40808 - "POST /node/694095ac575bc060ce1613d0 HTTP/1.0" 202 Accepted 2025-12-17 23:11:52,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409315575bc060ce160c26 2025-12-17 23:11:52,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409315575bc060ce160c26 HTTP/1.1" 502 150 2025-12-17 23:11:52,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409315575bc060ce160c26'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:52,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f8575bc060ce160bee 2025-12-17 23:11:52,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f8575bc060ce160bee HTTP/1.1" 502 150 2025-12-17 23:11:52,917 - lava_callback - ERROR - Error processing callback for node 694092f8575bc060ce160bee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092f8575bc060ce160bee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092f8575bc060ce160bee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092f8575bc060ce160bee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:52,918 - lava_callback - INFO - Starting async job submit for node 6940931b575bc060ce160c29 2025-12-17 23:11:52,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:52,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940931b575bc060ce160c29 HTTP/1.1" 502 150 2025-12-17 23:11:52,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940931b575bc060ce160c29'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:52,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940931b575bc060ce160c29 2025-12-17 23:11:52,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940931b575bc060ce160c29 HTTP/1.1" 502 150 2025-12-17 23:11:52,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940931b575bc060ce160c29'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:53,231 - lava_callback - INFO - Received callback for node 694095ae575bc060ce1613d3 INFO: 172.21.0.1:40816 - "POST /node/694095ae575bc060ce1613d3 HTTP/1.0" 202 Accepted 2025-12-17 23:11:53,862 - lava_callback - INFO - Received callback for node 694095af575bc060ce1613d6 INFO: 172.21.0.1:40820 - "POST /node/694095af575bc060ce1613d6 HTTP/1.0" 202 Accepted 2025-12-17 23:11:53,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409313575bc060ce160c23 2025-12-17 23:11:53,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409313575bc060ce160c23 HTTP/1.1" 502 150 2025-12-17 23:11:53,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409313575bc060ce160c23'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:54,503 - lava_callback - INFO - Received callback for node 694095b0575bc060ce1613d9 INFO: 172.21.0.1:40830 - "POST /node/694095b0575bc060ce1613d9 HTTP/1.0" 202 Accepted 2025-12-17 23:11:54,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092fa575bc060ce160bf1 2025-12-17 23:11:54,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a1a575bc060ce161ee2 2025-12-17 23:11:54,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1a575bc060ce161ee2 HTTP/1.1" 502 150 2025-12-17 23:11:54,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a1a575bc060ce161ee2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:54,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940930b575bc060ce160c13 2025-12-17 23:11:54,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930b575bc060ce160c13 HTTP/1.1" 502 150 2025-12-17 23:11:54,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940930b575bc060ce160c13'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:54,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940931b575bc060ce160c29 2025-12-17 23:11:54,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940931b575bc060ce160c29 HTTP/1.1" 502 150 2025-12-17 23:11:54,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940931b575bc060ce160c29'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:54,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092fc575bc060ce160bf4 2025-12-17 23:11:54,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fc575bc060ce160bf4 HTTP/1.1" 502 150 2025-12-17 23:11:54,968 - lava_callback - ERROR - Error processing callback for node 694092fc575bc060ce160bf4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092fc575bc060ce160bf4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092fc575bc060ce160bf4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092fc575bc060ce160bf4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:54,969 - lava_callback - INFO - Starting async job submit for node 6940931d575bc060ce160c32 2025-12-17 23:11:54,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:54,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940931d575bc060ce160c32 HTTP/1.1" 502 150 2025-12-17 23:11:54,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940931d575bc060ce160c32'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:54,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940931d575bc060ce160c32 2025-12-17 23:11:54,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940931d575bc060ce160c32 HTTP/1.1" 502 150 2025-12-17 23:11:54,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940931d575bc060ce160c32'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:55,282 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409301575bc060ce160bff 2025-12-17 23:11:55,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409308575bc060ce160c08 2025-12-17 23:11:55,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409308575bc060ce160c08 HTTP/1.1" 502 150 2025-12-17 23:11:55,665 - lava_callback - ERROR - Error processing callback for node 69409308575bc060ce160c08: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409308575bc060ce160c08 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409308575bc060ce160c08 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409308575bc060ce160c08 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:55,667 - lava_callback - INFO - Starting async job submit for node 69409323575bc060ce160c3d 2025-12-17 23:11:55,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:55,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409323575bc060ce160c3d HTTP/1.1" 502 150 2025-12-17 23:11:55,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409323575bc060ce160c3d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:55,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409323575bc060ce160c3d 2025-12-17 23:11:55,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409323575bc060ce160c3d HTTP/1.1" 502 150 2025-12-17 23:11:55,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409323575bc060ce160c3d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:56,250 - lava_callback - INFO - Received callback for node 694095b7575bc060ce1613f9 INFO: 172.21.0.1:40834 - "POST /node/694095b7575bc060ce1613f9 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:40850 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:11:56,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940930d575bc060ce160c1a 2025-12-17 23:11:56,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930d575bc060ce160c1a HTTP/1.1" 502 150 2025-12-17 23:11:56,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940930d575bc060ce160c1a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:56,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940930f575bc060ce160c1d 2025-12-17 23:11:56,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930f575bc060ce160c1d HTTP/1.1" 502 150 2025-12-17 23:11:56,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940930f575bc060ce160c1d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:56,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409315575bc060ce160c26 2025-12-17 23:11:56,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409315575bc060ce160c26 HTTP/1.1" 502 150 2025-12-17 23:11:56,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409315575bc060ce160c26'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:56,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694111a8460bc8e265117c2f HTTP/1.1" 502 150 2025-12-17 23:11:56,848 - lava_callback - ERROR - Error processing callback for node 694111a8460bc8e265117c2f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694111a8460bc8e265117c2f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694111a8460bc8e265117c2f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694111a8460bc8e265117c2f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:56,850 - lava_callback - INFO - Starting async job submit for node 69409326575bc060ce160c49 2025-12-17 23:11:56,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:56,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409326575bc060ce160c49 HTTP/1.1" 502 150 2025-12-17 23:11:56,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409326575bc060ce160c49'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:56,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409326575bc060ce160c49 2025-12-17 23:11:56,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409326575bc060ce160c49 HTTP/1.1" 502 150 2025-12-17 23:11:56,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409326575bc060ce160c49'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:56,898 - lava_callback - INFO - Received callback for node 694095b9575bc060ce1613fc INFO: 172.21.0.1:40854 - "POST /node/694095b9575bc060ce1613fc HTTP/1.0" 202 Accepted 2025-12-17 23:11:56,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940931d575bc060ce160c32 2025-12-17 23:11:56,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940931d575bc060ce160c32 HTTP/1.1" 502 150 2025-12-17 23:11:56,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940931d575bc060ce160c32'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:57,539 - lava_callback - INFO - Received callback for node 694095bb575bc060ce1613ff INFO: 172.21.0.1:40870 - "POST /node/694095bb575bc060ce1613ff HTTP/1.0" 202 Accepted 2025-12-17 23:11:57,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409323575bc060ce160c3d 2025-12-17 23:11:57,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409323575bc060ce160c3d HTTP/1.1" 502 150 2025-12-17 23:11:57,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409323575bc060ce160c3d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:58,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694092f3575bc060ce160be0 2025-12-17 23:11:58,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092f3575bc060ce160be0 HTTP/1.1" 502 150 2025-12-17 23:11:58,485 - lava_callback - ERROR - Error processing callback for node 694092f3575bc060ce160be0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092f3575bc060ce160be0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092f3575bc060ce160be0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092f3575bc060ce160be0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:11:58,487 - lava_callback - INFO - Starting async job submit for node 69409328575bc060ce160c4c 2025-12-17 23:11:58,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:11:58,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409328575bc060ce160c4c HTTP/1.1" 502 150 2025-12-17 23:11:58,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409328575bc060ce160c4c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:58,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409328575bc060ce160c4c 2025-12-17 23:11:58,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409328575bc060ce160c4c HTTP/1.1" 502 150 2025-12-17 23:11:58,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409328575bc060ce160c4c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:58,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409311575bc060ce160c20 2025-12-17 23:11:58,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409311575bc060ce160c20 HTTP/1.1" 502 150 2025-12-17 23:11:58,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409311575bc060ce160c20'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:58,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409326575bc060ce160c49 2025-12-17 23:11:58,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409326575bc060ce160c49 HTTP/1.1" 502 150 2025-12-17 23:11:58,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409326575bc060ce160c49'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:11:58,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940931b575bc060ce160c29 2025-12-17 23:11:58,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940931b575bc060ce160c29 HTTP/1.1" 502 150 2025-12-17 23:11:58,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940931b575bc060ce160c29'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:00,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409328575bc060ce160c4c 2025-12-17 23:12:00,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409328575bc060ce160c4c HTTP/1.1" 502 150 2025-12-17 23:12:00,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409328575bc060ce160c4c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:00,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940931d575bc060ce160c32 2025-12-17 23:12:00,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940931d575bc060ce160c32 HTTP/1.1" 502 150 2025-12-17 23:12:00,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940931d575bc060ce160c32'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:01,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409323575bc060ce160c3d 2025-12-17 23:12:01,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409323575bc060ce160c3d HTTP/1.1" 502 150 2025-12-17 23:12:01,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409323575bc060ce160c3d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:01,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409313575bc060ce160c23 2025-12-17 23:12:01,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409313575bc060ce160c23 HTTP/1.1" 502 150 2025-12-17 23:12:01,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409313575bc060ce160c23'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:02,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409326575bc060ce160c49 2025-12-17 23:12:02,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409326575bc060ce160c49 HTTP/1.1" 502 150 2025-12-17 23:12:02,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409326575bc060ce160c49'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45370 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:12:04,389 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409304575bc060ce160c02 2025-12-17 23:12:04,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409304575bc060ce160c02 HTTP/1.1" 502 150 2025-12-17 23:12:04,390 - lava_callback - ERROR - Error processing callback for node 69409304575bc060ce160c02: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409304575bc060ce160c02 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409304575bc060ce160c02 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409304575bc060ce160c02 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:04,392 - lava_callback - INFO - Starting async job submit for node 6940932a575bc060ce160c4f 2025-12-17 23:12:04,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:04,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932a575bc060ce160c4f HTTP/1.1" 502 150 2025-12-17 23:12:04,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940932a575bc060ce160c4f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:04,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940932a575bc060ce160c4f 2025-12-17 23:12:04,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932a575bc060ce160c4f HTTP/1.1" 502 150 2025-12-17 23:12:04,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940932a575bc060ce160c4f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:04,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409328575bc060ce160c4c 2025-12-17 23:12:04,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409328575bc060ce160c4c HTTP/1.1" 502 150 2025-12-17 23:12:04,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409328575bc060ce160c4c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:04,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409315575bc060ce160c26 2025-12-17 23:12:04,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409315575bc060ce160c26 HTTP/1.1" 502 150 2025-12-17 23:12:04,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409315575bc060ce160c26'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:06,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940932a575bc060ce160c4f 2025-12-17 23:12:06,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932a575bc060ce160c4f HTTP/1.1" 502 150 2025-12-17 23:12:06,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940932a575bc060ce160c4f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45372 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:12:06,706 - lava_callback - INFO - Received callback for node 694095b1575bc060ce1613de INFO: 172.21.0.1:45376 - "POST /node/694095b1575bc060ce1613de HTTP/1.0" 202 Accepted 2025-12-17 23:12:06,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940931b575bc060ce160c29 2025-12-17 23:12:06,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940931b575bc060ce160c29 HTTP/1.1" 502 150 2025-12-17 23:12:06,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940931b575bc060ce160c29'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:07,370 - lava_callback - INFO - Received callback for node 694095b3575bc060ce1613e5 INFO: 172.21.0.1:45388 - "POST /node/694095b3575bc060ce1613e5 HTTP/1.0" 202 Accepted 2025-12-17 23:12:07,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409306575bc060ce160c05 2025-12-17 23:12:07,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409306575bc060ce160c05 HTTP/1.1" 502 150 2025-12-17 23:12:07,998 - lava_callback - ERROR - Error processing callback for node 69409306575bc060ce160c05: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409306575bc060ce160c05 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409306575bc060ce160c05 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409306575bc060ce160c05 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:08,001 - lava_callback - INFO - Starting async job submit for node 6940932c575bc060ce160c52 2025-12-17 23:12:08,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:08,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932c575bc060ce160c52 HTTP/1.1" 502 150 2025-12-17 23:12:08,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940932c575bc060ce160c52'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:08,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940932c575bc060ce160c52 2025-12-17 23:12:08,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932c575bc060ce160c52 HTTP/1.1" 502 150 2025-12-17 23:12:08,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940932c575bc060ce160c52'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:08,038 - lava_callback - INFO - Received callback for node 694095b4575bc060ce1613ec INFO: 172.21.0.1:45390 - "POST /node/694095b4575bc060ce1613ec HTTP/1.0" 202 Accepted 2025-12-17 23:12:08,672 - lava_callback - INFO - Received callback for node 694095b5575bc060ce1613f1 INFO: 172.21.0.1:45404 - "POST /node/694095b5575bc060ce1613f1 HTTP/1.0" 202 Accepted 2025-12-17 23:12:08,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940931d575bc060ce160c32 2025-12-17 23:12:08,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940931d575bc060ce160c32 HTTP/1.1" 502 150 2025-12-17 23:12:08,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940931d575bc060ce160c32'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:09,329 - lava_callback - INFO - Received callback for node 694095bc575bc060ce161402 INFO: 172.21.0.1:45414 - "POST /node/694095bc575bc060ce161402 HTTP/1.0" 202 Accepted 2025-12-17 23:12:09,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409323575bc060ce160c3d 2025-12-17 23:12:09,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409323575bc060ce160c3d HTTP/1.1" 502 150 2025-12-17 23:12:09,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409323575bc060ce160c3d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:09,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694092fa575bc060ce160bf1 HTTP/1.1" 502 150 2025-12-17 23:12:09,904 - lava_callback - ERROR - Error processing callback for node 694092fa575bc060ce160bf1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092fa575bc060ce160bf1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092fa575bc060ce160bf1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694092fa575bc060ce160bf1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:09,906 - lava_callback - INFO - Starting async job submit for node 6940932e575bc060ce160c55 2025-12-17 23:12:09,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:09,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932e575bc060ce160c55 HTTP/1.1" 502 150 2025-12-17 23:12:09,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940932e575bc060ce160c55'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:09,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940932e575bc060ce160c55 2025-12-17 23:12:09,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932e575bc060ce160c55 HTTP/1.1" 502 150 2025-12-17 23:12:09,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940932e575bc060ce160c55'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:09,970 - lava_callback - INFO - Received callback for node 694095bd575bc060ce161405 INFO: 172.21.0.1:38718 - "POST /node/694095bd575bc060ce161405 HTTP/1.0" 202 Accepted 2025-12-17 23:12:10,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940932c575bc060ce160c52 2025-12-17 23:12:10,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932c575bc060ce160c52 HTTP/1.1" 502 150 2025-12-17 23:12:10,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940932c575bc060ce160c52'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:10,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940932a575bc060ce160c4f 2025-12-17 23:12:10,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932a575bc060ce160c4f HTTP/1.1" 502 150 2025-12-17 23:12:10,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940932a575bc060ce160c4f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:10,632 - lava_callback - INFO - Received callback for node 694095be575bc060ce161408 INFO: 172.21.0.1:38730 - "POST /node/694095be575bc060ce161408 HTTP/1.0" 202 Accepted 2025-12-17 23:12:10,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409301575bc060ce160bff HTTP/1.1" 502 150 2025-12-17 23:12:10,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409301575bc060ce160bff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:10,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a1a575bc060ce161ee2 2025-12-17 23:12:10,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1a575bc060ce161ee2 HTTP/1.1" 502 150 2025-12-17 23:12:10,733 - lava_callback - ERROR - Error processing callback for node 69409a1a575bc060ce161ee2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a1a575bc060ce161ee2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a1a575bc060ce161ee2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a1a575bc060ce161ee2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:10,735 - lava_callback - INFO - Starting async job submit for node 69409330575bc060ce160c58 2025-12-17 23:12:10,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:10,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940930b575bc060ce160c13 2025-12-17 23:12:10,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930b575bc060ce160c13 HTTP/1.1" 502 150 2025-12-17 23:12:10,736 - lava_callback - ERROR - Error processing callback for node 6940930b575bc060ce160c13: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940930b575bc060ce160c13 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940930b575bc060ce160c13 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940930b575bc060ce160c13 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:10,738 - lava_callback - INFO - Starting async job submit for node 69409332575bc060ce160c61 2025-12-17 23:12:10,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:10,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409330575bc060ce160c58 HTTP/1.1" 502 150 2025-12-17 23:12:10,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409330575bc060ce160c58'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:10,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409330575bc060ce160c58 2025-12-17 23:12:10,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409330575bc060ce160c58 HTTP/1.1" 502 150 2025-12-17 23:12:10,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409330575bc060ce160c58'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:10,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409332575bc060ce160c61 HTTP/1.1" 502 150 2025-12-17 23:12:10,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409332575bc060ce160c61'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:10,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409332575bc060ce160c61 2025-12-17 23:12:10,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409332575bc060ce160c61 HTTP/1.1" 502 150 2025-12-17 23:12:10,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409332575bc060ce160c61'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:10,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409326575bc060ce160c49 2025-12-17 23:12:10,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409326575bc060ce160c49 HTTP/1.1" 502 150 2025-12-17 23:12:10,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409326575bc060ce160c49'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:11,288 - lava_callback - INFO - Received callback for node 694095c3575bc060ce16140b INFO: 172.21.0.1:38742 - "POST /node/694095c3575bc060ce16140b HTTP/1.0" 202 Accepted 2025-12-17 23:12:11,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940932e575bc060ce160c55 2025-12-17 23:12:11,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932e575bc060ce160c55 HTTP/1.1" 502 150 2025-12-17 23:12:11,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940932e575bc060ce160c55'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:11,948 - lava_callback - INFO - Received callback for node 694095c4575bc060ce16140e INFO: 172.21.0.1:38748 - "POST /node/694095c4575bc060ce16140e HTTP/1.0" 202 Accepted 2025-12-17 23:12:12,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409328575bc060ce160c4c 2025-12-17 23:12:12,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409328575bc060ce160c4c HTTP/1.1" 502 150 2025-12-17 23:12:12,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409328575bc060ce160c4c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:12,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940930d575bc060ce160c1a 2025-12-17 23:12:12,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930d575bc060ce160c1a HTTP/1.1" 502 150 2025-12-17 23:12:12,681 - lava_callback - ERROR - Error processing callback for node 6940930d575bc060ce160c1a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940930d575bc060ce160c1a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940930d575bc060ce160c1a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940930d575bc060ce160c1a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:12,683 - lava_callback - INFO - Starting async job submit for node 69409336575bc060ce160c73 2025-12-17 23:12:12,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:12,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409336575bc060ce160c73 HTTP/1.1" 502 150 2025-12-17 23:12:12,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409336575bc060ce160c73'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:12,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409336575bc060ce160c73 2025-12-17 23:12:12,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409336575bc060ce160c73 HTTP/1.1" 502 150 2025-12-17 23:12:12,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409336575bc060ce160c73'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:12,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409330575bc060ce160c58 2025-12-17 23:12:12,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409332575bc060ce160c61 2025-12-17 23:12:12,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409332575bc060ce160c61 HTTP/1.1" 502 150 2025-12-17 23:12:12,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409332575bc060ce160c61'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:12,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409330575bc060ce160c58 HTTP/1.1" 502 150 2025-12-17 23:12:12,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409330575bc060ce160c58'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:12,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940930f575bc060ce160c1d 2025-12-17 23:12:12,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940930f575bc060ce160c1d HTTP/1.1" 502 150 2025-12-17 23:12:12,786 - lava_callback - ERROR - Error processing callback for node 6940930f575bc060ce160c1d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940930f575bc060ce160c1d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940930f575bc060ce160c1d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940930f575bc060ce160c1d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:12,787 - lava_callback - INFO - Starting async job submit for node 69409338575bc060ce160c7a 2025-12-17 23:12:12,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:12,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409338575bc060ce160c7a HTTP/1.1" 502 150 2025-12-17 23:12:12,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409338575bc060ce160c7a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:12,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409338575bc060ce160c7a 2025-12-17 23:12:12,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409338575bc060ce160c7a HTTP/1.1" 502 150 2025-12-17 23:12:12,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409338575bc060ce160c7a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:14,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940932c575bc060ce160c52 2025-12-17 23:12:14,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932c575bc060ce160c52 HTTP/1.1" 502 150 2025-12-17 23:12:14,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940932c575bc060ce160c52'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:14,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409336575bc060ce160c73 2025-12-17 23:12:14,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409336575bc060ce160c73 HTTP/1.1" 502 150 2025-12-17 23:12:14,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409336575bc060ce160c73'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:14,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409311575bc060ce160c20 2025-12-17 23:12:14,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409338575bc060ce160c7a 2025-12-17 23:12:14,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409311575bc060ce160c20 HTTP/1.1" 502 150 2025-12-17 23:12:14,796 - lava_callback - ERROR - Error processing callback for node 69409311575bc060ce160c20: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409311575bc060ce160c20 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409311575bc060ce160c20 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409311575bc060ce160c20 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:14,799 - lava_callback - INFO - Starting async job submit for node 6940933a575bc060ce160c7f 2025-12-17 23:12:14,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:14,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409338575bc060ce160c7a HTTP/1.1" 502 150 2025-12-17 23:12:14,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409338575bc060ce160c7a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:14,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933a575bc060ce160c7f HTTP/1.1" 502 150 2025-12-17 23:12:14,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940933a575bc060ce160c7f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:14,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940933a575bc060ce160c7f 2025-12-17 23:12:14,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933a575bc060ce160c7f HTTP/1.1" 502 150 2025-12-17 23:12:14,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940933a575bc060ce160c7f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38750 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:12:15,741 - lava_callback - INFO - Received callback for node 69409ac4575bc060ce16204a INFO: 172.21.0.1:38762 - "POST /node/69409ac4575bc060ce16204a HTTP/1.0" 202 Accepted 2025-12-17 23:12:15,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940932e575bc060ce160c55 2025-12-17 23:12:15,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932e575bc060ce160c55 HTTP/1.1" 502 150 2025-12-17 23:12:15,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940932e575bc060ce160c55'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:16,257 - lava_callback - INFO - Received callback for node 694095c7575bc060ce16141a INFO: 172.21.0.1:38770 - "POST /node/694095c7575bc060ce16141a HTTP/1.0" 202 Accepted INFO: 172.21.0.1:38774 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:12:16,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409332575bc060ce160c61 2025-12-17 23:12:16,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409330575bc060ce160c58 2025-12-17 23:12:16,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409332575bc060ce160c61 HTTP/1.1" 502 150 2025-12-17 23:12:16,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409332575bc060ce160c61'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:16,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409330575bc060ce160c58 HTTP/1.1" 502 150 2025-12-17 23:12:16,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409330575bc060ce160c58'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:16,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940933a575bc060ce160c7f 2025-12-17 23:12:16,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933a575bc060ce160c7f HTTP/1.1" 502 150 2025-12-17 23:12:16,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940933a575bc060ce160c7f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:16,906 - lava_callback - INFO - Received callback for node 694095c8575bc060ce161421 INFO: 172.21.0.1:38788 - "POST /node/694095c8575bc060ce161421 HTTP/1.0" 202 Accepted 2025-12-17 23:12:17,567 - lava_callback - INFO - Received callback for node 694095cf575bc060ce161434 INFO: 172.21.0.1:38796 - "POST /node/694095cf575bc060ce161434 HTTP/1.0" 202 Accepted 2025-12-17 23:12:17,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409313575bc060ce160c23 2025-12-17 23:12:17,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409313575bc060ce160c23 HTTP/1.1" 502 150 2025-12-17 23:12:17,934 - lava_callback - ERROR - Error processing callback for node 69409313575bc060ce160c23: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409313575bc060ce160c23 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409313575bc060ce160c23 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409313575bc060ce160c23 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:17,936 - lava_callback - INFO - Starting async job submit for node 6940933c575bc060ce160ce3 2025-12-17 23:12:17,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:17,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933c575bc060ce160ce3 HTTP/1.1" 502 150 2025-12-17 23:12:17,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940933c575bc060ce160ce3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:17,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940933c575bc060ce160ce3 2025-12-17 23:12:17,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933c575bc060ce160ce3 HTTP/1.1" 502 150 2025-12-17 23:12:17,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940933c575bc060ce160ce3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:18,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940932a575bc060ce160c4f 2025-12-17 23:12:18,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932a575bc060ce160c4f HTTP/1.1" 502 150 2025-12-17 23:12:18,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940932a575bc060ce160c4f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:18,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409336575bc060ce160c73 2025-12-17 23:12:18,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409336575bc060ce160c73 HTTP/1.1" 502 150 2025-12-17 23:12:18,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409336575bc060ce160c73'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:18,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409338575bc060ce160c7a 2025-12-17 23:12:18,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409338575bc060ce160c7a HTTP/1.1" 502 150 2025-12-17 23:12:18,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409338575bc060ce160c7a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:19,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940933c575bc060ce160ce3 2025-12-17 23:12:19,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933c575bc060ce160ce3 HTTP/1.1" 502 150 2025-12-17 23:12:19,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940933c575bc060ce160ce3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:20,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940933a575bc060ce160c7f 2025-12-17 23:12:20,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933a575bc060ce160c7f HTTP/1.1" 502 150 2025-12-17 23:12:20,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940933a575bc060ce160c7f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:20,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409315575bc060ce160c26 2025-12-17 23:12:20,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409315575bc060ce160c26 HTTP/1.1" 502 150 2025-12-17 23:12:20,840 - lava_callback - ERROR - Error processing callback for node 69409315575bc060ce160c26: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409315575bc060ce160c26 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409315575bc060ce160c26 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409315575bc060ce160c26 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:20,843 - lava_callback - INFO - Starting async job submit for node 6940933e575bc060ce160ce6 2025-12-17 23:12:20,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:20,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933e575bc060ce160ce6 HTTP/1.1" 502 150 2025-12-17 23:12:20,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940933e575bc060ce160ce6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:20,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940933e575bc060ce160ce6 2025-12-17 23:12:20,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933e575bc060ce160ce6 HTTP/1.1" 502 150 2025-12-17 23:12:20,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940933e575bc060ce160ce6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:22,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940932c575bc060ce160c52 2025-12-17 23:12:22,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932c575bc060ce160c52 HTTP/1.1" 502 150 2025-12-17 23:12:22,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940932c575bc060ce160c52'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:22,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940933e575bc060ce160ce6 2025-12-17 23:12:22,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933e575bc060ce160ce6 HTTP/1.1" 502 150 2025-12-17 23:12:22,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940933e575bc060ce160ce6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:22,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940931b575bc060ce160c29 2025-12-17 23:12:22,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940931b575bc060ce160c29 HTTP/1.1" 502 150 2025-12-17 23:12:22,939 - lava_callback - ERROR - Error processing callback for node 6940931b575bc060ce160c29: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940931b575bc060ce160c29 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940931b575bc060ce160c29 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940931b575bc060ce160c29 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:22,941 - lava_callback - INFO - Starting async job submit for node 69409341575bc060ce160ce9 2025-12-17 23:12:22,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:22,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409341575bc060ce160ce9 HTTP/1.1" 502 150 2025-12-17 23:12:22,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409341575bc060ce160ce9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:22,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409341575bc060ce160ce9 2025-12-17 23:12:22,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409341575bc060ce160ce9 HTTP/1.1" 502 150 2025-12-17 23:12:22,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409341575bc060ce160ce9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:23,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940932e575bc060ce160c55 2025-12-17 23:12:23,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932e575bc060ce160c55 HTTP/1.1" 502 150 2025-12-17 23:12:23,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940932e575bc060ce160c55'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:23,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940933c575bc060ce160ce3 2025-12-17 23:12:23,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933c575bc060ce160ce3 HTTP/1.1" 502 150 2025-12-17 23:12:23,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940933c575bc060ce160ce3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:24,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409332575bc060ce160c61 2025-12-17 23:12:24,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409330575bc060ce160c58 2025-12-17 23:12:24,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409332575bc060ce160c61 HTTP/1.1" 502 150 2025-12-17 23:12:24,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409332575bc060ce160c61'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:24,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409330575bc060ce160c58 HTTP/1.1" 502 150 2025-12-17 23:12:24,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409330575bc060ce160c58'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:24,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409341575bc060ce160ce9 2025-12-17 23:12:24,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409341575bc060ce160ce9 HTTP/1.1" 502 150 2025-12-17 23:12:24,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409341575bc060ce160ce9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:24,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940931d575bc060ce160c32 2025-12-17 23:12:24,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940931d575bc060ce160c32 HTTP/1.1" 502 150 2025-12-17 23:12:24,985 - lava_callback - ERROR - Error processing callback for node 6940931d575bc060ce160c32: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940931d575bc060ce160c32 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940931d575bc060ce160c32 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940931d575bc060ce160c32 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:24,987 - lava_callback - INFO - Starting async job submit for node 69409343575bc060ce160cec 2025-12-17 23:12:24,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:25,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409343575bc060ce160cec HTTP/1.1" 502 150 2025-12-17 23:12:25,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409343575bc060ce160cec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:25,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409343575bc060ce160cec 2025-12-17 23:12:25,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409343575bc060ce160cec HTTP/1.1" 502 150 2025-12-17 23:12:25,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409343575bc060ce160cec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:25,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409323575bc060ce160c3d 2025-12-17 23:12:25,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409323575bc060ce160c3d HTTP/1.1" 502 150 2025-12-17 23:12:25,680 - lava_callback - ERROR - Error processing callback for node 69409323575bc060ce160c3d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409323575bc060ce160c3d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409323575bc060ce160c3d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409323575bc060ce160c3d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:25,682 - lava_callback - INFO - Starting async job submit for node 69409418575bc060ce160f60 2025-12-17 23:12:25,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:25,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409418575bc060ce160f60 HTTP/1.1" 502 150 2025-12-17 23:12:25,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409418575bc060ce160f60'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:25,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409418575bc060ce160f60 2025-12-17 23:12:25,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409418575bc060ce160f60 HTTP/1.1" 502 150 2025-12-17 23:12:25,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409418575bc060ce160f60'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55158 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:12:26,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409301575bc060ce160bff 2025-12-17 23:12:26,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409301575bc060ce160bff HTTP/1.1" 502 150 2025-12-17 23:12:26,674 - lava_callback - ERROR - Error processing callback for node 69409301575bc060ce160bff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409301575bc060ce160bff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409301575bc060ce160bff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409301575bc060ce160bff (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:26,676 - lava_callback - INFO - Starting async job submit for node 69409347575bc060ce160cfc 2025-12-17 23:12:26,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:26,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409336575bc060ce160c73 2025-12-17 23:12:26,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409336575bc060ce160c73 HTTP/1.1" 502 150 2025-12-17 23:12:26,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409336575bc060ce160c73'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:26,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409347575bc060ce160cfc HTTP/1.1" 502 150 2025-12-17 23:12:26,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409347575bc060ce160cfc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:26,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409347575bc060ce160cfc 2025-12-17 23:12:26,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409347575bc060ce160cfc HTTP/1.1" 502 150 2025-12-17 23:12:26,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409347575bc060ce160cfc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:26,727 - lava_callback - INFO - Received callback for node 694095c6575bc060ce161413 INFO: 172.21.0.1:55174 - "POST /node/694095c6575bc060ce161413 HTTP/1.0" 202 Accepted 2025-12-17 23:12:26,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409338575bc060ce160c7a 2025-12-17 23:12:26,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409338575bc060ce160c7a HTTP/1.1" 502 150 2025-12-17 23:12:26,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409338575bc060ce160c7a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:26,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940933e575bc060ce160ce6 2025-12-17 23:12:26,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933e575bc060ce160ce6 HTTP/1.1" 502 150 2025-12-17 23:12:26,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940933e575bc060ce160ce6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:26,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409326575bc060ce160c49 2025-12-17 23:12:26,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409326575bc060ce160c49 HTTP/1.1" 502 150 2025-12-17 23:12:26,888 - lava_callback - ERROR - Error processing callback for node 69409326575bc060ce160c49: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409326575bc060ce160c49 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409326575bc060ce160c49 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409326575bc060ce160c49 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:26,890 - lava_callback - INFO - Starting async job submit for node 69409349575bc060ce160d05 2025-12-17 23:12:26,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:26,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409349575bc060ce160d05 HTTP/1.1" 502 150 2025-12-17 23:12:26,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409349575bc060ce160d05'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:26,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409349575bc060ce160d05 2025-12-17 23:12:26,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409349575bc060ce160d05 HTTP/1.1" 502 150 2025-12-17 23:12:26,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409349575bc060ce160d05'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:27,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409343575bc060ce160cec 2025-12-17 23:12:27,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409343575bc060ce160cec HTTP/1.1" 502 150 2025-12-17 23:12:27,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409343575bc060ce160cec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:27,367 - lava_callback - INFO - Received callback for node 694095cd575bc060ce161431 INFO: 172.21.0.1:55184 - "POST /node/694095cd575bc060ce161431 HTTP/1.0" 202 Accepted 2025-12-17 23:12:27,478 - lava_callback - INFO - Received callback for node 694095ca575bc060ce161428 INFO: 172.21.0.1:55198 - "POST /node/694095ca575bc060ce161428 HTTP/1.0" 202 Accepted 2025-12-17 23:12:27,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409418575bc060ce160f60 2025-12-17 23:12:27,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409418575bc060ce160f60 HTTP/1.1" 502 150 2025-12-17 23:12:27,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409418575bc060ce160f60'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:27,999 - lava_callback - INFO - Received callback for node 694095d0575bc060ce161437 INFO: 172.21.0.1:55214 - "POST /node/694095d0575bc060ce161437 HTTP/1.0" 202 Accepted 2025-12-17 23:12:28,273 - lava_callback - INFO - Received callback for node 694095cc575bc060ce16142e INFO: 172.21.0.1:55228 - "POST /node/694095cc575bc060ce16142e HTTP/1.0" 202 Accepted 2025-12-17 23:12:28,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409328575bc060ce160c4c 2025-12-17 23:12:28,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409328575bc060ce160c4c HTTP/1.1" 502 150 2025-12-17 23:12:28,507 - lava_callback - ERROR - Error processing callback for node 69409328575bc060ce160c4c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409328575bc060ce160c4c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409328575bc060ce160c4c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409328575bc060ce160c4c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:28,509 - lava_callback - INFO - Starting async job submit for node 6940934b575bc060ce160d0a 2025-12-17 23:12:28,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:28,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940934b575bc060ce160d0a HTTP/1.1" 502 150 2025-12-17 23:12:28,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940934b575bc060ce160d0a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:28,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940934b575bc060ce160d0a 2025-12-17 23:12:28,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940934b575bc060ce160d0a HTTP/1.1" 502 150 2025-12-17 23:12:28,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940934b575bc060ce160d0a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:28,649 - lava_callback - INFO - Received callback for node 694095d2575bc060ce16143d INFO: 172.21.0.1:55230 - "POST /node/694095d2575bc060ce16143d HTTP/1.0" 202 Accepted 2025-12-17 23:12:28,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409347575bc060ce160cfc 2025-12-17 23:12:28,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409347575bc060ce160cfc HTTP/1.1" 502 150 2025-12-17 23:12:28,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409347575bc060ce160cfc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:28,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940933a575bc060ce160c7f 2025-12-17 23:12:28,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933a575bc060ce160c7f HTTP/1.1" 502 150 2025-12-17 23:12:28,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940933a575bc060ce160c7f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:28,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409349575bc060ce160d05 2025-12-17 23:12:28,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409349575bc060ce160d05 HTTP/1.1" 502 150 2025-12-17 23:12:28,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409349575bc060ce160d05'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:28,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409341575bc060ce160ce9 2025-12-17 23:12:28,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409341575bc060ce160ce9 HTTP/1.1" 502 150 2025-12-17 23:12:28,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409341575bc060ce160ce9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:29,035 - lava_callback - INFO - Received callback for node 694095d1575bc060ce16143a INFO: 172.21.0.1:55240 - "POST /node/694095d1575bc060ce16143a HTTP/1.0" 202 Accepted 2025-12-17 23:12:29,307 - lava_callback - INFO - Received callback for node 694095d5575bc060ce161443 INFO: 172.21.0.1:55250 - "POST /node/694095d5575bc060ce161443 HTTP/1.0" 202 Accepted 2025-12-17 23:12:29,853 - lava_callback - INFO - Received callback for node 694095d3575bc060ce161440 INFO: 172.21.0.1:37624 - "POST /node/694095d3575bc060ce161440 HTTP/1.0" 202 Accepted 2025-12-17 23:12:29,972 - lava_callback - INFO - Received callback for node 694095d6575bc060ce161446 INFO: 172.21.0.1:37640 - "POST /node/694095d6575bc060ce161446 HTTP/1.0" 202 Accepted 2025-12-17 23:12:30,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940934b575bc060ce160d0a 2025-12-17 23:12:30,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940934b575bc060ce160d0a HTTP/1.1" 502 150 2025-12-17 23:12:30,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940934b575bc060ce160d0a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:30,619 - lava_callback - INFO - Received callback for node 694095d8575bc060ce16144c INFO: 172.21.0.1:37642 - "POST /node/694095d8575bc060ce16144c HTTP/1.0" 202 Accepted 2025-12-17 23:12:30,632 - lava_callback - INFO - Received callback for node 694095da575bc060ce161451 INFO: 172.21.0.1:37644 - "POST /node/694095da575bc060ce161451 HTTP/1.0" 202 Accepted 2025-12-17 23:12:31,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409343575bc060ce160cec 2025-12-17 23:12:31,267 - lava_callback - INFO - Received callback for node 694095d7575bc060ce161449 INFO: 172.21.0.1:37658 - "POST /node/694095d7575bc060ce161449 HTTP/1.0" 202 Accepted 2025-12-17 23:12:31,461 - lava_callback - INFO - Received callback for node 694095dd575bc060ce161460 INFO: 172.21.0.1:37660 - "POST /node/694095dd575bc060ce161460 HTTP/1.0" 202 Accepted 2025-12-17 23:12:31,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409418575bc060ce160f60 2025-12-17 23:12:31,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409418575bc060ce160f60 HTTP/1.1" 502 150 2025-12-17 23:12:31,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409418575bc060ce160f60'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:31,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940933c575bc060ce160ce3 2025-12-17 23:12:31,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933c575bc060ce160ce3 HTTP/1.1" 502 150 2025-12-17 23:12:31,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940933c575bc060ce160ce3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:32,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409347575bc060ce160cfc 2025-12-17 23:12:32,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409347575bc060ce160cfc HTTP/1.1" 502 150 2025-12-17 23:12:32,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409347575bc060ce160cfc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:32,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409349575bc060ce160d05 2025-12-17 23:12:32,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409349575bc060ce160d05 HTTP/1.1" 502 150 2025-12-17 23:12:32,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409349575bc060ce160d05'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:34,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940932a575bc060ce160c4f 2025-12-17 23:12:34,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940934b575bc060ce160d0a 2025-12-17 23:12:34,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940934b575bc060ce160d0a HTTP/1.1" 502 150 2025-12-17 23:12:34,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940934b575bc060ce160d0a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:34,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940933e575bc060ce160ce6 2025-12-17 23:12:34,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933e575bc060ce160ce6 HTTP/1.1" 502 150 2025-12-17 23:12:34,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940933e575bc060ce160ce6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:36,278 - lava_callback - INFO - Received callback for node 694095dc575bc060ce16145d INFO: 172.21.0.1:37664 - "POST /node/694095dc575bc060ce16145d HTTP/1.0" 202 Accepted INFO: 172.21.0.1:37678 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:12:36,933 - lava_callback - INFO - Received callback for node 694095df575bc060ce161463 INFO: 172.21.0.1:37694 - "POST /node/694095df575bc060ce161463 HTTP/1.0" 202 Accepted 2025-12-17 23:12:36,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409341575bc060ce160ce9 2025-12-17 23:12:36,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409341575bc060ce160ce9 HTTP/1.1" 502 150 2025-12-17 23:12:36,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409341575bc060ce160ce9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:38,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940932c575bc060ce160c52 2025-12-17 23:12:38,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932c575bc060ce160c52 HTTP/1.1" 502 150 2025-12-17 23:12:38,021 - lava_callback - ERROR - Error processing callback for node 6940932c575bc060ce160c52: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940932c575bc060ce160c52 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940932c575bc060ce160c52 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940932c575bc060ce160c52 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:38,022 - lava_callback - INFO - Starting async job submit for node 6940934d575bc060ce160d0d 2025-12-17 23:12:38,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:38,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940934d575bc060ce160d0d HTTP/1.1" 502 150 2025-12-17 23:12:38,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940934d575bc060ce160d0d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:38,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940934d575bc060ce160d0d 2025-12-17 23:12:38,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940934d575bc060ce160d0d HTTP/1.1" 502 150 2025-12-17 23:12:38,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940934d575bc060ce160d0d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:39,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409418575bc060ce160f60 2025-12-17 23:12:39,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409418575bc060ce160f60 HTTP/1.1" 502 150 2025-12-17 23:12:39,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409418575bc060ce160f60'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:39,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940932e575bc060ce160c55 2025-12-17 23:12:39,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932e575bc060ce160c55 HTTP/1.1" 502 150 2025-12-17 23:12:39,920 - lava_callback - ERROR - Error processing callback for node 6940932e575bc060ce160c55: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940932e575bc060ce160c55 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940932e575bc060ce160c55 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940932e575bc060ce160c55 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:39,923 - lava_callback - INFO - Starting async job submit for node 69409350575bc060ce160d10 2025-12-17 23:12:39,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:39,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409350575bc060ce160d10 HTTP/1.1" 502 150 2025-12-17 23:12:39,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409350575bc060ce160d10'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:39,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409350575bc060ce160d10 2025-12-17 23:12:39,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409350575bc060ce160d10 HTTP/1.1" 502 150 2025-12-17 23:12:39,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409350575bc060ce160d10'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:40,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940934d575bc060ce160d0d 2025-12-17 23:12:40,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940934d575bc060ce160d0d HTTP/1.1" 502 150 2025-12-17 23:12:40,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940934d575bc060ce160d0d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:40,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409347575bc060ce160cfc 2025-12-17 23:12:40,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409347575bc060ce160cfc HTTP/1.1" 502 150 2025-12-17 23:12:40,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409347575bc060ce160cfc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:40,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409332575bc060ce160c61 2025-12-17 23:12:40,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409330575bc060ce160c58 2025-12-17 23:12:40,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409332575bc060ce160c61 HTTP/1.1" 502 150 2025-12-17 23:12:40,757 - lava_callback - ERROR - Error processing callback for node 69409332575bc060ce160c61: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409332575bc060ce160c61 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409332575bc060ce160c61 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409332575bc060ce160c61 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:40,760 - lava_callback - INFO - Starting async job submit for node 69409351575bc060ce160d13 2025-12-17 23:12:40,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:40,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409330575bc060ce160c58 HTTP/1.1" 502 150 2025-12-17 23:12:40,762 - lava_callback - ERROR - Error processing callback for node 69409330575bc060ce160c58: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409330575bc060ce160c58 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409330575bc060ce160c58 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409330575bc060ce160c58 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:40,765 - lava_callback - INFO - Starting async job submit for node 69409357575bc060ce160d1c 2025-12-17 23:12:40,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:40,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409351575bc060ce160d13 HTTP/1.1" 502 150 2025-12-17 23:12:40,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409351575bc060ce160d13'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:40,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409351575bc060ce160d13 2025-12-17 23:12:40,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409357575bc060ce160d1c HTTP/1.1" 502 150 2025-12-17 23:12:40,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409357575bc060ce160d1c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:40,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409357575bc060ce160d1c 2025-12-17 23:12:40,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409351575bc060ce160d13 HTTP/1.1" 502 150 2025-12-17 23:12:40,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409351575bc060ce160d13'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:40,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409357575bc060ce160d1c HTTP/1.1" 502 150 2025-12-17 23:12:40,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409357575bc060ce160d1c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:40,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409349575bc060ce160d05 2025-12-17 23:12:40,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409349575bc060ce160d05 HTTP/1.1" 502 150 2025-12-17 23:12:40,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409349575bc060ce160d05'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:41,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409350575bc060ce160d10 2025-12-17 23:12:41,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409350575bc060ce160d10 HTTP/1.1" 502 150 2025-12-17 23:12:41,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409350575bc060ce160d10'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:42,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940934b575bc060ce160d0a 2025-12-17 23:12:42,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940934b575bc060ce160d0a HTTP/1.1" 502 150 2025-12-17 23:12:42,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940934b575bc060ce160d0a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:42,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409336575bc060ce160c73 2025-12-17 23:12:42,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409336575bc060ce160c73 HTTP/1.1" 502 150 2025-12-17 23:12:42,700 - lava_callback - ERROR - Error processing callback for node 69409336575bc060ce160c73: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409336575bc060ce160c73 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409336575bc060ce160c73 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409336575bc060ce160c73 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:42,702 - lava_callback - INFO - Starting async job submit for node 69409561575bc060ce1612ee 2025-12-17 23:12:42,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:42,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409561575bc060ce1612ee HTTP/1.1" 502 150 2025-12-17 23:12:42,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409561575bc060ce1612ee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:42,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409561575bc060ce1612ee 2025-12-17 23:12:42,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409561575bc060ce1612ee HTTP/1.1" 502 150 2025-12-17 23:12:42,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409561575bc060ce1612ee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:42,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409351575bc060ce160d13 2025-12-17 23:12:42,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409351575bc060ce160d13 HTTP/1.1" 502 150 2025-12-17 23:12:42,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409351575bc060ce160d13'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:42,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409357575bc060ce160d1c 2025-12-17 23:12:42,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409357575bc060ce160d1c HTTP/1.1" 502 150 2025-12-17 23:12:42,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409357575bc060ce160d1c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:42,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409338575bc060ce160c7a 2025-12-17 23:12:42,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409338575bc060ce160c7a HTTP/1.1" 502 150 2025-12-17 23:12:42,804 - lava_callback - ERROR - Error processing callback for node 69409338575bc060ce160c7a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409338575bc060ce160c7a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409338575bc060ce160c7a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409338575bc060ce160c7a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:42,806 - lava_callback - INFO - Starting async job submit for node 69409559575bc060ce1612dc 2025-12-17 23:12:42,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:42,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409559575bc060ce1612dc HTTP/1.1" 502 150 2025-12-17 23:12:42,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409559575bc060ce1612dc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:42,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409559575bc060ce1612dc 2025-12-17 23:12:42,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409559575bc060ce1612dc HTTP/1.1" 502 150 2025-12-17 23:12:42,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409559575bc060ce1612dc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:44,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940934d575bc060ce160d0d 2025-12-17 23:12:44,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940934d575bc060ce160d0d HTTP/1.1" 502 150 2025-12-17 23:12:44,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940934d575bc060ce160d0d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:44,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409561575bc060ce1612ee 2025-12-17 23:12:44,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409561575bc060ce1612ee HTTP/1.1" 502 150 2025-12-17 23:12:44,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409561575bc060ce1612ee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:44,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940933a575bc060ce160c7f 2025-12-17 23:12:44,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933a575bc060ce160c7f HTTP/1.1" 502 150 2025-12-17 23:12:44,816 - lava_callback - ERROR - Error processing callback for node 6940933a575bc060ce160c7f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940933a575bc060ce160c7f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940933a575bc060ce160c7f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940933a575bc060ce160c7f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:44,817 - lava_callback - INFO - Starting async job submit for node 6940935a575bc060ce160d25 2025-12-17 23:12:44,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:44,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935a575bc060ce160d25 HTTP/1.1" 502 150 2025-12-17 23:12:44,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935a575bc060ce160d25'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:44,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935a575bc060ce160d25 2025-12-17 23:12:44,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935a575bc060ce160d25 HTTP/1.1" 502 150 2025-12-17 23:12:44,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935a575bc060ce160d25'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:44,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409559575bc060ce1612dc 2025-12-17 23:12:44,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409559575bc060ce1612dc HTTP/1.1" 502 150 2025-12-17 23:12:44,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409559575bc060ce1612dc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:45,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409350575bc060ce160d10 2025-12-17 23:12:45,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409350575bc060ce160d10 HTTP/1.1" 502 150 2025-12-17 23:12:45,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409350575bc060ce160d10'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:46,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409343575bc060ce160cec HTTP/1.1" 502 150 2025-12-17 23:12:46,256 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409343575bc060ce160cec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38856 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:12:46,770 - lava_callback - INFO - Received callback for node 694095db575bc060ce16145a INFO: 172.21.0.1:38860 - "POST /node/694095db575bc060ce16145a HTTP/1.0" 202 Accepted 2025-12-17 23:12:46,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409351575bc060ce160d13 2025-12-17 23:12:46,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409351575bc060ce160d13 HTTP/1.1" 502 150 2025-12-17 23:12:46,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409351575bc060ce160d13'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:46,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409357575bc060ce160d1c 2025-12-17 23:12:46,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409357575bc060ce160d1c HTTP/1.1" 502 150 2025-12-17 23:12:46,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409357575bc060ce160d1c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:46,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935a575bc060ce160d25 2025-12-17 23:12:46,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935a575bc060ce160d25 HTTP/1.1" 502 150 2025-12-17 23:12:46,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935a575bc060ce160d25'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:47,436 - lava_callback - INFO - Received callback for node 694095e2575bc060ce161469 INFO: 172.21.0.1:38862 - "POST /node/694095e2575bc060ce161469 HTTP/1.0" 202 Accepted 2025-12-17 23:12:47,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940933c575bc060ce160ce3 2025-12-17 23:12:47,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933c575bc060ce160ce3 HTTP/1.1" 502 150 2025-12-17 23:12:47,949 - lava_callback - ERROR - Error processing callback for node 6940933c575bc060ce160ce3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940933c575bc060ce160ce3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940933c575bc060ce160ce3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940933c575bc060ce160ce3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:47,951 - lava_callback - INFO - Starting async job submit for node 6940935b575bc060ce160d2e 2025-12-17 23:12:47,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:47,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935b575bc060ce160d2e HTTP/1.1" 502 150 2025-12-17 23:12:47,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935b575bc060ce160d2e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:47,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935b575bc060ce160d2e 2025-12-17 23:12:47,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935b575bc060ce160d2e HTTP/1.1" 502 150 2025-12-17 23:12:47,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935b575bc060ce160d2e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:48,083 - lava_callback - INFO - Received callback for node 694095e5575bc060ce16146c INFO: 172.21.0.1:38868 - "POST /node/694095e5575bc060ce16146c HTTP/1.0" 202 Accepted 2025-12-17 23:12:48,728 - lava_callback - INFO - Received callback for node 694095e7575bc060ce16146f INFO: 172.21.0.1:38884 - "POST /node/694095e7575bc060ce16146f HTTP/1.0" 202 Accepted 2025-12-17 23:12:48,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409561575bc060ce1612ee 2025-12-17 23:12:48,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409561575bc060ce1612ee HTTP/1.1" 502 150 2025-12-17 23:12:48,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409561575bc060ce1612ee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:48,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409559575bc060ce1612dc 2025-12-17 23:12:48,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409559575bc060ce1612dc HTTP/1.1" 502 150 2025-12-17 23:12:48,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409559575bc060ce1612dc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:49,379 - lava_callback - INFO - Received callback for node 694095e8575bc060ce161472 INFO: 172.21.0.1:38894 - "POST /node/694095e8575bc060ce161472 HTTP/1.0" 202 Accepted 2025-12-17 23:12:49,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940932a575bc060ce160c4f HTTP/1.1" 502 150 2025-12-17 23:12:49,840 - lava_callback - ERROR - Error processing callback for node 6940932a575bc060ce160c4f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940932a575bc060ce160c4f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940932a575bc060ce160c4f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940932a575bc060ce160c4f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:49,842 - lava_callback - INFO - Starting async job submit for node 6940935c575bc060ce160d35 2025-12-17 23:12:49,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:49,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935c575bc060ce160d35 HTTP/1.1" 502 150 2025-12-17 23:12:49,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935c575bc060ce160d35'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:49,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935c575bc060ce160d35 2025-12-17 23:12:49,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935c575bc060ce160d35 HTTP/1.1" 502 150 2025-12-17 23:12:49,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935c575bc060ce160d35'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:49,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935b575bc060ce160d2e 2025-12-17 23:12:49,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935b575bc060ce160d2e HTTP/1.1" 502 150 2025-12-17 23:12:49,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935b575bc060ce160d2e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:50,040 - lava_callback - INFO - Received callback for node 694095ea575bc060ce161475 INFO: 172.21.0.1:44694 - "POST /node/694095ea575bc060ce161475 HTTP/1.0" 202 Accepted 2025-12-17 23:12:50,687 - lava_callback - INFO - Received callback for node 694095ec575bc060ce161478 INFO: 172.21.0.1:44704 - "POST /node/694095ec575bc060ce161478 HTTP/1.0" 202 Accepted 2025-12-17 23:12:50,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935a575bc060ce160d25 2025-12-17 23:12:50,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935a575bc060ce160d25 HTTP/1.1" 502 150 2025-12-17 23:12:50,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935a575bc060ce160d25'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:50,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940933e575bc060ce160ce6 2025-12-17 23:12:50,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940933e575bc060ce160ce6 HTTP/1.1" 502 150 2025-12-17 23:12:50,856 - lava_callback - ERROR - Error processing callback for node 6940933e575bc060ce160ce6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940933e575bc060ce160ce6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940933e575bc060ce160ce6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940933e575bc060ce160ce6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:50,859 - lava_callback - INFO - Starting async job submit for node 6940935e575bc060ce160d3a 2025-12-17 23:12:50,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:50,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935e575bc060ce160d3a HTTP/1.1" 502 150 2025-12-17 23:12:50,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935e575bc060ce160d3a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:50,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935e575bc060ce160d3a 2025-12-17 23:12:50,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935e575bc060ce160d3a HTTP/1.1" 502 150 2025-12-17 23:12:50,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935e575bc060ce160d3a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:51,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935c575bc060ce160d35 2025-12-17 23:12:51,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935c575bc060ce160d35 HTTP/1.1" 502 150 2025-12-17 23:12:51,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935c575bc060ce160d35'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:52,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940934d575bc060ce160d0d 2025-12-17 23:12:52,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940934d575bc060ce160d0d HTTP/1.1" 502 150 2025-12-17 23:12:52,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940934d575bc060ce160d0d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:52,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935e575bc060ce160d3a 2025-12-17 23:12:52,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935e575bc060ce160d3a HTTP/1.1" 502 150 2025-12-17 23:12:52,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935e575bc060ce160d3a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:52,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409341575bc060ce160ce9 2025-12-17 23:12:52,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409341575bc060ce160ce9 HTTP/1.1" 502 150 2025-12-17 23:12:52,956 - lava_callback - ERROR - Error processing callback for node 69409341575bc060ce160ce9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409341575bc060ce160ce9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409341575bc060ce160ce9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409341575bc060ce160ce9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:52,958 - lava_callback - INFO - Starting async job submit for node 6940936a575bc060ce160d69 2025-12-17 23:12:52,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:52,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936a575bc060ce160d69 HTTP/1.1" 502 150 2025-12-17 23:12:52,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940936a575bc060ce160d69'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:52,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940936a575bc060ce160d69 2025-12-17 23:12:52,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936a575bc060ce160d69 HTTP/1.1" 502 150 2025-12-17 23:12:52,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940936a575bc060ce160d69'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:53,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409350575bc060ce160d10 2025-12-17 23:12:53,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409350575bc060ce160d10 HTTP/1.1" 502 150 2025-12-17 23:12:53,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409350575bc060ce160d10'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:53,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935b575bc060ce160d2e 2025-12-17 23:12:53,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935b575bc060ce160d2e HTTP/1.1" 502 150 2025-12-17 23:12:53,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935b575bc060ce160d2e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:54,256 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409343575bc060ce160cec 2025-12-17 23:12:54,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409343575bc060ce160cec HTTP/1.1" 502 150 2025-12-17 23:12:54,258 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409343575bc060ce160cec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:54,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409357575bc060ce160d1c 2025-12-17 23:12:54,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409357575bc060ce160d1c HTTP/1.1" 502 150 2025-12-17 23:12:54,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409357575bc060ce160d1c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:54,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409351575bc060ce160d13 2025-12-17 23:12:54,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409351575bc060ce160d13 HTTP/1.1" 502 150 2025-12-17 23:12:54,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409351575bc060ce160d13'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:54,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940936a575bc060ce160d69 2025-12-17 23:12:54,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936a575bc060ce160d69 HTTP/1.1" 502 150 2025-12-17 23:12:54,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940936a575bc060ce160d69'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:55,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409418575bc060ce160f60 2025-12-17 23:12:55,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409418575bc060ce160f60 HTTP/1.1" 502 150 2025-12-17 23:12:55,719 - lava_callback - ERROR - Error processing callback for node 69409418575bc060ce160f60: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409418575bc060ce160f60 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409418575bc060ce160f60 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409418575bc060ce160f60 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:55,721 - lava_callback - INFO - Starting async job submit for node 6940935f575bc060ce160d3d 2025-12-17 23:12:55,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:55,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935f575bc060ce160d3d HTTP/1.1" 502 150 2025-12-17 23:12:55,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935f575bc060ce160d3d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:55,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935f575bc060ce160d3d 2025-12-17 23:12:55,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935f575bc060ce160d3d HTTP/1.1" 502 150 2025-12-17 23:12:55,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935f575bc060ce160d3d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:55,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935c575bc060ce160d35 2025-12-17 23:12:55,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935c575bc060ce160d35 HTTP/1.1" 502 150 2025-12-17 23:12:55,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935c575bc060ce160d35'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:56,016 - lava_callback - INFO - Received callback for node 6940989a575bc060ce161b02 INFO: 172.21.0.1:44712 - "POST /node/6940989a575bc060ce161b02 HTTP/1.0" 202 Accepted 2025-12-17 23:12:56,301 - lava_callback - INFO - Received callback for node 694095f0575bc060ce161486 INFO: 172.21.0.1:44716 - "POST /node/694095f0575bc060ce161486 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:44720 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:12:56,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409347575bc060ce160cfc 2025-12-17 23:12:56,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409347575bc060ce160cfc HTTP/1.1" 502 150 2025-12-17 23:12:56,708 - lava_callback - ERROR - Error processing callback for node 69409347575bc060ce160cfc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409347575bc060ce160cfc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409347575bc060ce160cfc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409347575bc060ce160cfc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:56,710 - lava_callback - INFO - Starting async job submit for node 69409363575bc060ce160d5a 2025-12-17 23:12:56,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:56,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409363575bc060ce160d5a HTTP/1.1" 502 150 2025-12-17 23:12:56,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409363575bc060ce160d5a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:56,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409363575bc060ce160d5a 2025-12-17 23:12:56,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409363575bc060ce160d5a HTTP/1.1" 502 150 2025-12-17 23:12:56,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409363575bc060ce160d5a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:56,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409561575bc060ce1612ee 2025-12-17 23:12:56,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409561575bc060ce1612ee HTTP/1.1" 502 150 2025-12-17 23:12:56,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409561575bc060ce1612ee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:56,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935e575bc060ce160d3a 2025-12-17 23:12:56,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935e575bc060ce160d3a HTTP/1.1" 502 150 2025-12-17 23:12:56,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935e575bc060ce160d3a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:56,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409349575bc060ce160d05 2025-12-17 23:12:56,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409349575bc060ce160d05 HTTP/1.1" 502 150 2025-12-17 23:12:56,903 - lava_callback - ERROR - Error processing callback for node 69409349575bc060ce160d05: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409349575bc060ce160d05 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409349575bc060ce160d05 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409349575bc060ce160d05 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:56,905 - lava_callback - INFO - Starting async job submit for node 69409365575bc060ce160d5d 2025-12-17 23:12:56,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:56,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409559575bc060ce1612dc 2025-12-17 23:12:56,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409559575bc060ce1612dc HTTP/1.1" 502 150 2025-12-17 23:12:56,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409559575bc060ce1612dc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:56,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409365575bc060ce160d5d HTTP/1.1" 502 150 2025-12-17 23:12:56,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409365575bc060ce160d5d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:56,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409365575bc060ce160d5d 2025-12-17 23:12:56,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409365575bc060ce160d5d HTTP/1.1" 502 150 2025-12-17 23:12:56,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409365575bc060ce160d5d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:56,953 - lava_callback - INFO - Received callback for node 694095f3575bc060ce16148b INFO: 172.21.0.1:44730 - "POST /node/694095f3575bc060ce16148b HTTP/1.0" 202 Accepted 2025-12-17 23:12:57,609 - lava_callback - INFO - Received callback for node 694095f4575bc060ce16148e INFO: 172.21.0.1:44740 - "POST /node/694095f4575bc060ce16148e HTTP/1.0" 202 Accepted 2025-12-17 23:12:57,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935f575bc060ce160d3d 2025-12-17 23:12:57,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935f575bc060ce160d3d HTTP/1.1" 502 150 2025-12-17 23:12:57,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935f575bc060ce160d3d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:58,274 - lava_callback - INFO - Received callback for node 694095f5575bc060ce161491 INFO: 172.21.0.1:44742 - "POST /node/694095f5575bc060ce161491 HTTP/1.0" 202 Accepted 2025-12-17 23:12:58,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940934b575bc060ce160d0a 2025-12-17 23:12:58,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940934b575bc060ce160d0a HTTP/1.1" 502 150 2025-12-17 23:12:58,528 - lava_callback - ERROR - Error processing callback for node 6940934b575bc060ce160d0a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940934b575bc060ce160d0a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940934b575bc060ce160d0a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940934b575bc060ce160d0a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:12:58,530 - lava_callback - INFO - Starting async job submit for node 69409366575bc060ce160d60 2025-12-17 23:12:58,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:12:58,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409366575bc060ce160d60 HTTP/1.1" 502 150 2025-12-17 23:12:58,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409366575bc060ce160d60'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:58,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409366575bc060ce160d60 2025-12-17 23:12:58,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409366575bc060ce160d60 HTTP/1.1" 502 150 2025-12-17 23:12:58,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409366575bc060ce160d60'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:58,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409363575bc060ce160d5a 2025-12-17 23:12:58,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409363575bc060ce160d5a HTTP/1.1" 502 150 2025-12-17 23:12:58,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409363575bc060ce160d5a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:58,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935a575bc060ce160d25 2025-12-17 23:12:58,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935a575bc060ce160d25 HTTP/1.1" 502 150 2025-12-17 23:12:58,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935a575bc060ce160d25'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:58,933 - lava_callback - INFO - Received callback for node 694095f7575bc060ce161497 INFO: 172.21.0.1:44752 - "POST /node/694095f7575bc060ce161497 HTTP/1.0" 202 Accepted 2025-12-17 23:12:58,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409365575bc060ce160d5d 2025-12-17 23:12:58,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409365575bc060ce160d5d HTTP/1.1" 502 150 2025-12-17 23:12:58,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409365575bc060ce160d5d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:12:58,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940936a575bc060ce160d69 2025-12-17 23:12:58,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936a575bc060ce160d69 HTTP/1.1" 502 150 2025-12-17 23:12:58,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940936a575bc060ce160d69'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:00,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409366575bc060ce160d60 2025-12-17 23:13:01,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935f575bc060ce160d3d 2025-12-17 23:13:01,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935f575bc060ce160d3d HTTP/1.1" 502 150 2025-12-17 23:13:01,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935f575bc060ce160d3d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:01,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935b575bc060ce160d2e 2025-12-17 23:13:01,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935b575bc060ce160d2e HTTP/1.1" 502 150 2025-12-17 23:13:01,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935b575bc060ce160d2e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:02,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409363575bc060ce160d5a 2025-12-17 23:13:02,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409363575bc060ce160d5a HTTP/1.1" 502 150 2025-12-17 23:13:02,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409363575bc060ce160d5a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:02,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409365575bc060ce160d5d 2025-12-17 23:13:02,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409365575bc060ce160d5d HTTP/1.1" 502 150 2025-12-17 23:13:02,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409365575bc060ce160d5d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55846 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:13:03,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935c575bc060ce160d35 2025-12-17 23:13:03,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935c575bc060ce160d35 HTTP/1.1" 502 150 2025-12-17 23:13:03,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935c575bc060ce160d35'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:04,243 - lava_callback - INFO - Received callback for node 69409563575bc060ce1612fc INFO: 172.21.0.1:55852 - "POST /node/69409563575bc060ce1612fc HTTP/1.0" 202 Accepted 2025-12-17 23:13:04,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935e575bc060ce160d3a 2025-12-17 23:13:04,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935e575bc060ce160d3a HTTP/1.1" 502 150 2025-12-17 23:13:04,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935e575bc060ce160d3a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55864 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:13:06,769 - lava_callback - INFO - Received callback for node 694095ee575bc060ce16147d INFO: 172.21.0.1:55870 - "POST /node/694095ee575bc060ce16147d HTTP/1.0" 202 Accepted 2025-12-17 23:13:06,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940936a575bc060ce160d69 2025-12-17 23:13:06,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936a575bc060ce160d69 HTTP/1.1" 502 150 2025-12-17 23:13:06,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940936a575bc060ce160d69'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:07,421 - lava_callback - INFO - Received callback for node 694095fa575bc060ce1614a1 INFO: 172.21.0.1:55876 - "POST /node/694095fa575bc060ce1614a1 HTTP/1.0" 202 Accepted 2025-12-17 23:13:08,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940934d575bc060ce160d0d 2025-12-17 23:13:08,070 - lava_callback - INFO - Received callback for node 694095fd575bc060ce1614a4 INFO: 172.21.0.1:55888 - "POST /node/694095fd575bc060ce1614a4 HTTP/1.0" 202 Accepted 2025-12-17 23:13:08,538 - lava_callback - INFO - Received callback for node 694095f9575bc060ce16149b INFO: 172.21.0.1:55898 - "POST /node/694095f9575bc060ce16149b HTTP/1.0" 202 Accepted 2025-12-17 23:13:08,710 - lava_callback - INFO - Received callback for node 694095ff575bc060ce1614a7 INFO: 172.21.0.1:55900 - "POST /node/694095ff575bc060ce1614a7 HTTP/1.0" 202 Accepted 2025-12-17 23:13:09,348 - lava_callback - INFO - Received callback for node 69409601575bc060ce1614aa INFO: 172.21.0.1:55902 - "POST /node/69409601575bc060ce1614aa HTTP/1.0" 202 Accepted 2025-12-17 23:13:09,359 - lava_callback - INFO - Received callback for node 694095fa575bc060ce16149e INFO: 172.21.0.1:55906 - "POST /node/694095fa575bc060ce16149e HTTP/1.0" 202 Accepted 2025-12-17 23:13:09,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935f575bc060ce160d3d 2025-12-17 23:13:09,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935f575bc060ce160d3d HTTP/1.1" 502 150 2025-12-17 23:13:09,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940935f575bc060ce160d3d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:09,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409350575bc060ce160d10 2025-12-17 23:13:09,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409350575bc060ce160d10 HTTP/1.1" 502 150 2025-12-17 23:13:09,946 - lava_callback - ERROR - Error processing callback for node 69409350575bc060ce160d10: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409350575bc060ce160d10 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409350575bc060ce160d10 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409350575bc060ce160d10 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:09,948 - lava_callback - INFO - Starting async job submit for node 69409367575bc060ce160d63 2025-12-17 23:13:09,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:09,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409367575bc060ce160d63 HTTP/1.1" 502 150 2025-12-17 23:13:09,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409367575bc060ce160d63'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:09,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409367575bc060ce160d63 2025-12-17 23:13:09,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409367575bc060ce160d63 HTTP/1.1" 502 150 2025-12-17 23:13:09,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409367575bc060ce160d63'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:10,258 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409343575bc060ce160cec 2025-12-17 23:13:10,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409343575bc060ce160cec HTTP/1.1" 502 150 2025-12-17 23:13:10,259 - lava_callback - ERROR - Error processing callback for node 69409343575bc060ce160cec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409343575bc060ce160cec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409343575bc060ce160cec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409343575bc060ce160cec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:10,261 - lava_callback - INFO - Starting async job submit for node 69409369575bc060ce160d66 2025-12-17 23:13:10,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:10,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409369575bc060ce160d66 HTTP/1.1" 502 150 2025-12-17 23:13:10,268 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409369575bc060ce160d66'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:10,268 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409369575bc060ce160d66 2025-12-17 23:13:10,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409369575bc060ce160d66 HTTP/1.1" 502 150 2025-12-17 23:13:10,269 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409369575bc060ce160d66'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:10,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409363575bc060ce160d5a 2025-12-17 23:13:10,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409363575bc060ce160d5a HTTP/1.1" 502 150 2025-12-17 23:13:10,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409363575bc060ce160d5a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:10,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409357575bc060ce160d1c 2025-12-17 23:13:10,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409357575bc060ce160d1c HTTP/1.1" 502 150 2025-12-17 23:13:10,792 - lava_callback - ERROR - Error processing callback for node 69409357575bc060ce160d1c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409357575bc060ce160d1c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409357575bc060ce160d1c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409357575bc060ce160d1c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:10,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409351575bc060ce160d13 2025-12-17 23:13:10,794 - lava_callback - INFO - Starting async job submit for node 6940936e575bc060ce160d75 2025-12-17 23:13:10,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:10,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409351575bc060ce160d13 HTTP/1.1" 502 150 2025-12-17 23:13:10,796 - lava_callback - ERROR - Error processing callback for node 69409351575bc060ce160d13: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409351575bc060ce160d13 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409351575bc060ce160d13 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409351575bc060ce160d13 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:10,797 - lava_callback - INFO - Starting async job submit for node 69409691575bc060ce161615 2025-12-17 23:13:10,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:10,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936e575bc060ce160d75 HTTP/1.1" 502 150 2025-12-17 23:13:10,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940936e575bc060ce160d75'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:10,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940936e575bc060ce160d75 2025-12-17 23:13:10,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936e575bc060ce160d75 HTTP/1.1" 502 150 2025-12-17 23:13:10,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940936e575bc060ce160d75'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:10,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409691575bc060ce161615 HTTP/1.1" 502 150 2025-12-17 23:13:10,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409691575bc060ce161615'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:10,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409691575bc060ce161615 2025-12-17 23:13:10,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409691575bc060ce161615 HTTP/1.1" 502 150 2025-12-17 23:13:10,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409691575bc060ce161615'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:10,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409365575bc060ce160d5d 2025-12-17 23:13:10,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409365575bc060ce160d5d HTTP/1.1" 502 150 2025-12-17 23:13:10,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409365575bc060ce160d5d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:11,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409367575bc060ce160d63 2025-12-17 23:13:11,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409367575bc060ce160d63 HTTP/1.1" 502 150 2025-12-17 23:13:11,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409367575bc060ce160d63'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:12,269 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409369575bc060ce160d66 2025-12-17 23:13:12,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409369575bc060ce160d66 HTTP/1.1" 502 150 2025-12-17 23:13:12,270 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409369575bc060ce160d66'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:12,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409561575bc060ce1612ee 2025-12-17 23:13:12,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409561575bc060ce1612ee HTTP/1.1" 502 150 2025-12-17 23:13:12,751 - lava_callback - ERROR - Error processing callback for node 69409561575bc060ce1612ee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409561575bc060ce1612ee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409561575bc060ce1612ee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409561575bc060ce1612ee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:12,753 - lava_callback - INFO - Starting async job submit for node 69409371575bc060ce160d80 2025-12-17 23:13:12,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:12,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409371575bc060ce160d80 HTTP/1.1" 502 150 2025-12-17 23:13:12,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409371575bc060ce160d80'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:12,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409371575bc060ce160d80 2025-12-17 23:13:12,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409371575bc060ce160d80 HTTP/1.1" 502 150 2025-12-17 23:13:12,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409371575bc060ce160d80'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:12,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940936e575bc060ce160d75 2025-12-17 23:13:12,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936e575bc060ce160d75 HTTP/1.1" 502 150 2025-12-17 23:13:12,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940936e575bc060ce160d75'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:12,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409691575bc060ce161615 2025-12-17 23:13:12,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409691575bc060ce161615 HTTP/1.1" 502 150 2025-12-17 23:13:12,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409691575bc060ce161615'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:12,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409559575bc060ce1612dc 2025-12-17 23:13:12,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409559575bc060ce1612dc HTTP/1.1" 502 150 2025-12-17 23:13:12,933 - lava_callback - ERROR - Error processing callback for node 69409559575bc060ce1612dc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409559575bc060ce1612dc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409559575bc060ce1612dc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409559575bc060ce1612dc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:12,935 - lava_callback - INFO - Starting async job submit for node 69409374575bc060ce160d8a 2025-12-17 23:13:12,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:12,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409374575bc060ce160d8a HTTP/1.1" 502 150 2025-12-17 23:13:12,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409374575bc060ce160d8a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:12,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409374575bc060ce160d8a 2025-12-17 23:13:12,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409374575bc060ce160d8a HTTP/1.1" 502 150 2025-12-17 23:13:12,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409374575bc060ce160d8a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:14,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409371575bc060ce160d80 2025-12-17 23:13:14,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409371575bc060ce160d80 HTTP/1.1" 502 150 2025-12-17 23:13:14,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409371575bc060ce160d80'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:14,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935a575bc060ce160d25 2025-12-17 23:13:14,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935a575bc060ce160d25 HTTP/1.1" 502 150 2025-12-17 23:13:14,833 - lava_callback - ERROR - Error processing callback for node 6940935a575bc060ce160d25: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940935a575bc060ce160d25 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940935a575bc060ce160d25 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940935a575bc060ce160d25 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:14,834 - lava_callback - INFO - Starting async job submit for node 6940937b575bc060ce160d9c 2025-12-17 23:13:14,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:14,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937b575bc060ce160d9c HTTP/1.1" 502 150 2025-12-17 23:13:14,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937b575bc060ce160d9c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:14,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937b575bc060ce160d9c 2025-12-17 23:13:14,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937b575bc060ce160d9c HTTP/1.1" 502 150 2025-12-17 23:13:14,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937b575bc060ce160d9c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:14,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409374575bc060ce160d8a 2025-12-17 23:13:14,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409374575bc060ce160d8a HTTP/1.1" 502 150 2025-12-17 23:13:14,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409374575bc060ce160d8a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33044 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:13:15,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409366575bc060ce160d60 HTTP/1.1" 502 150 2025-12-17 23:13:15,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409366575bc060ce160d60'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:15,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409367575bc060ce160d63 2025-12-17 23:13:15,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409367575bc060ce160d63 HTTP/1.1" 502 150 2025-12-17 23:13:15,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409367575bc060ce160d63'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:16,270 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409369575bc060ce160d66 2025-12-17 23:13:16,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409369575bc060ce160d66 HTTP/1.1" 502 150 2025-12-17 23:13:16,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409369575bc060ce160d66'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:16,302 - lava_callback - INFO - Received callback for node 69409605575bc060ce1614b2 INFO: 172.21.0.1:33054 - "POST /node/69409605575bc060ce1614b2 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:33066 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:13:16,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937b575bc060ce160d9c 2025-12-17 23:13:16,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937b575bc060ce160d9c HTTP/1.1" 502 150 2025-12-17 23:13:16,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937b575bc060ce160d9c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:16,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940936e575bc060ce160d75 2025-12-17 23:13:16,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936e575bc060ce160d75 HTTP/1.1" 502 150 2025-12-17 23:13:16,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940936e575bc060ce160d75'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:16,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409691575bc060ce161615 2025-12-17 23:13:16,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409691575bc060ce161615 HTTP/1.1" 502 150 2025-12-17 23:13:16,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409691575bc060ce161615'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:17,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935b575bc060ce160d2e 2025-12-17 23:13:17,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935b575bc060ce160d2e HTTP/1.1" 502 150 2025-12-17 23:13:17,967 - lava_callback - ERROR - Error processing callback for node 6940935b575bc060ce160d2e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940935b575bc060ce160d2e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940935b575bc060ce160d2e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940935b575bc060ce160d2e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:17,968 - lava_callback - INFO - Starting async job submit for node 69409376575bc060ce160d8d 2025-12-17 23:13:17,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:17,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409376575bc060ce160d8d HTTP/1.1" 502 150 2025-12-17 23:13:17,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409376575bc060ce160d8d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:17,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409376575bc060ce160d8d 2025-12-17 23:13:17,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409376575bc060ce160d8d HTTP/1.1" 502 150 2025-12-17 23:13:17,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409376575bc060ce160d8d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:18,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409371575bc060ce160d80 2025-12-17 23:13:18,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409371575bc060ce160d80 HTTP/1.1" 502 150 2025-12-17 23:13:18,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409371575bc060ce160d80'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:18,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409374575bc060ce160d8a 2025-12-17 23:13:18,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409374575bc060ce160d8a HTTP/1.1" 502 150 2025-12-17 23:13:18,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409374575bc060ce160d8a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:19,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935c575bc060ce160d35 2025-12-17 23:13:19,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935c575bc060ce160d35 HTTP/1.1" 502 150 2025-12-17 23:13:19,882 - lava_callback - ERROR - Error processing callback for node 6940935c575bc060ce160d35: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940935c575bc060ce160d35 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940935c575bc060ce160d35 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940935c575bc060ce160d35 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:19,884 - lava_callback - INFO - Starting async job submit for node 6940937e575bc060ce160d9f 2025-12-17 23:13:19,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:19,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937e575bc060ce160d9f HTTP/1.1" 502 150 2025-12-17 23:13:19,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937e575bc060ce160d9f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:19,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937e575bc060ce160d9f 2025-12-17 23:13:19,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937e575bc060ce160d9f HTTP/1.1" 502 150 2025-12-17 23:13:19,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937e575bc060ce160d9f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:19,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409366575bc060ce160d60 2025-12-17 23:13:19,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409366575bc060ce160d60 HTTP/1.1" 502 150 2025-12-17 23:13:19,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409366575bc060ce160d60'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:19,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409376575bc060ce160d8d 2025-12-17 23:13:19,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409376575bc060ce160d8d HTTP/1.1" 502 150 2025-12-17 23:13:19,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409376575bc060ce160d8d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:20,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937b575bc060ce160d9c 2025-12-17 23:13:20,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937b575bc060ce160d9c HTTP/1.1" 502 150 2025-12-17 23:13:20,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937b575bc060ce160d9c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:20,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935e575bc060ce160d3a 2025-12-17 23:13:20,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935e575bc060ce160d3a HTTP/1.1" 502 150 2025-12-17 23:13:20,873 - lava_callback - ERROR - Error processing callback for node 6940935e575bc060ce160d3a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940935e575bc060ce160d3a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940935e575bc060ce160d3a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940935e575bc060ce160d3a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:20,875 - lava_callback - INFO - Starting async job submit for node 69409378575bc060ce160d93 2025-12-17 23:13:20,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:20,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409378575bc060ce160d93 HTTP/1.1" 502 150 2025-12-17 23:13:20,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409378575bc060ce160d93'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:20,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409378575bc060ce160d93 2025-12-17 23:13:20,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409378575bc060ce160d93 HTTP/1.1" 502 150 2025-12-17 23:13:20,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409378575bc060ce160d93'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:21,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937e575bc060ce160d9f 2025-12-17 23:13:21,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937e575bc060ce160d9f HTTP/1.1" 502 150 2025-12-17 23:13:21,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937e575bc060ce160d9f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:22,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409378575bc060ce160d93 2025-12-17 23:13:22,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409378575bc060ce160d93 HTTP/1.1" 502 150 2025-12-17 23:13:22,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409378575bc060ce160d93'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:22,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940936a575bc060ce160d69 2025-12-17 23:13:22,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936a575bc060ce160d69 HTTP/1.1" 502 150 2025-12-17 23:13:22,972 - lava_callback - ERROR - Error processing callback for node 6940936a575bc060ce160d69: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940936a575bc060ce160d69 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940936a575bc060ce160d69 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940936a575bc060ce160d69 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:22,973 - lava_callback - INFO - Starting async job submit for node 69409380575bc060ce160da2 2025-12-17 23:13:22,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:22,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409380575bc060ce160da2 HTTP/1.1" 502 150 2025-12-17 23:13:22,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409380575bc060ce160da2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:22,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409380575bc060ce160da2 2025-12-17 23:13:22,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409380575bc060ce160da2 HTTP/1.1" 502 150 2025-12-17 23:13:22,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409380575bc060ce160da2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:23,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940934d575bc060ce160d0d HTTP/1.1" 502 150 2025-12-17 23:13:23,377 - lava_callback - ERROR - Error processing callback for node 6940934d575bc060ce160d0d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940934d575bc060ce160d0d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940934d575bc060ce160d0d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940934d575bc060ce160d0d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:23,380 - lava_callback - INFO - Starting async job submit for node 6940937a575bc060ce160d96 2025-12-17 23:13:23,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:23,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937a575bc060ce160d96 HTTP/1.1" 502 150 2025-12-17 23:13:23,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937a575bc060ce160d96'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:23,388 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937a575bc060ce160d96 2025-12-17 23:13:23,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937a575bc060ce160d96 HTTP/1.1" 502 150 2025-12-17 23:13:23,388 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937a575bc060ce160d96'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:23,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409367575bc060ce160d63 2025-12-17 23:13:23,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409367575bc060ce160d63 HTTP/1.1" 502 150 2025-12-17 23:13:23,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409367575bc060ce160d63'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:23,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409376575bc060ce160d8d 2025-12-17 23:13:23,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409376575bc060ce160d8d HTTP/1.1" 502 150 2025-12-17 23:13:23,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409376575bc060ce160d8d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:24,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409369575bc060ce160d66 2025-12-17 23:13:24,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409369575bc060ce160d66 HTTP/1.1" 502 150 2025-12-17 23:13:24,273 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409369575bc060ce160d66'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:24,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940936e575bc060ce160d75 2025-12-17 23:13:24,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936e575bc060ce160d75 HTTP/1.1" 502 150 2025-12-17 23:13:24,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940936e575bc060ce160d75'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:24,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409691575bc060ce161615 2025-12-17 23:13:24,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409691575bc060ce161615 HTTP/1.1" 502 150 2025-12-17 23:13:24,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409691575bc060ce161615'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:24,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409380575bc060ce160da2 2025-12-17 23:13:24,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409380575bc060ce160da2 HTTP/1.1" 502 150 2025-12-17 23:13:24,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409380575bc060ce160da2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:25,388 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937a575bc060ce160d96 2025-12-17 23:13:25,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937a575bc060ce160d96 HTTP/1.1" 502 150 2025-12-17 23:13:25,390 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937a575bc060ce160d96'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:25,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940935f575bc060ce160d3d 2025-12-17 23:13:25,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940935f575bc060ce160d3d HTTP/1.1" 502 150 2025-12-17 23:13:25,735 - lava_callback - ERROR - Error processing callback for node 6940935f575bc060ce160d3d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940935f575bc060ce160d3d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940935f575bc060ce160d3d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940935f575bc060ce160d3d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:25,737 - lava_callback - INFO - Starting async job submit for node 69409a22575bc060ce161eee 2025-12-17 23:13:25,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:25,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a22575bc060ce161eee HTTP/1.1" 502 150 2025-12-17 23:13:25,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a22575bc060ce161eee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:25,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a22575bc060ce161eee 2025-12-17 23:13:25,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a22575bc060ce161eee HTTP/1.1" 502 150 2025-12-17 23:13:25,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a22575bc060ce161eee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:25,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937e575bc060ce160d9f 2025-12-17 23:13:25,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937e575bc060ce160d9f HTTP/1.1" 502 150 2025-12-17 23:13:25,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937e575bc060ce160d9f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43010 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:13:26,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409363575bc060ce160d5a 2025-12-17 23:13:26,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409371575bc060ce160d80 2025-12-17 23:13:26,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409371575bc060ce160d80 HTTP/1.1" 502 150 2025-12-17 23:13:26,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409371575bc060ce160d80'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:26,791 - lava_callback - INFO - Received callback for node 69409603575bc060ce1614af INFO: 172.21.0.1:43014 - "POST /node/69409603575bc060ce1614af HTTP/1.0" 202 Accepted 2025-12-17 23:13:26,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409378575bc060ce160d93 2025-12-17 23:13:26,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409378575bc060ce160d93 HTTP/1.1" 502 150 2025-12-17 23:13:26,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409378575bc060ce160d93'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:26,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409365575bc060ce160d5d 2025-12-17 23:13:26,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409365575bc060ce160d5d HTTP/1.1" 502 150 2025-12-17 23:13:26,943 - lava_callback - ERROR - Error processing callback for node 69409365575bc060ce160d5d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409365575bc060ce160d5d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409365575bc060ce160d5d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409365575bc060ce160d5d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:26,945 - lava_callback - INFO - Starting async job submit for node 69409377575bc060ce160d90 2025-12-17 23:13:26,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:26,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409377575bc060ce160d90 HTTP/1.1" 502 150 2025-12-17 23:13:26,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409377575bc060ce160d90'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:26,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409377575bc060ce160d90 2025-12-17 23:13:26,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409377575bc060ce160d90 HTTP/1.1" 502 150 2025-12-17 23:13:26,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409377575bc060ce160d90'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:26,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409374575bc060ce160d8a 2025-12-17 23:13:26,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409374575bc060ce160d8a HTTP/1.1" 502 150 2025-12-17 23:13:26,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409374575bc060ce160d8a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:27,458 - lava_callback - INFO - Received callback for node 69409607575bc060ce1614b5 INFO: 172.21.0.1:43016 - "POST /node/69409607575bc060ce1614b5 HTTP/1.0" 202 Accepted 2025-12-17 23:13:27,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a22575bc060ce161eee 2025-12-17 23:13:27,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a22575bc060ce161eee HTTP/1.1" 502 150 2025-12-17 23:13:27,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a22575bc060ce161eee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:27,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409366575bc060ce160d60 2025-12-17 23:13:27,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409366575bc060ce160d60 HTTP/1.1" 502 150 2025-12-17 23:13:27,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409366575bc060ce160d60'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:28,110 - lava_callback - INFO - Received callback for node 69409609575bc060ce1614b8 INFO: 172.21.0.1:43032 - "POST /node/69409609575bc060ce1614b8 HTTP/1.0" 202 Accepted 2025-12-17 23:13:28,766 - lava_callback - INFO - Received callback for node 6940960b575bc060ce1614bb INFO: 172.21.0.1:43046 - "POST /node/6940960b575bc060ce1614bb HTTP/1.0" 202 Accepted 2025-12-17 23:13:28,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937b575bc060ce160d9c 2025-12-17 23:13:28,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937b575bc060ce160d9c HTTP/1.1" 502 150 2025-12-17 23:13:28,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937b575bc060ce160d9c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:28,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409377575bc060ce160d90 2025-12-17 23:13:28,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409377575bc060ce160d90 HTTP/1.1" 502 150 2025-12-17 23:13:28,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409377575bc060ce160d90'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:28,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409380575bc060ce160da2 2025-12-17 23:13:28,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409380575bc060ce160da2 HTTP/1.1" 502 150 2025-12-17 23:13:28,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409380575bc060ce160da2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:29,390 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937a575bc060ce160d96 2025-12-17 23:13:29,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937a575bc060ce160d96 HTTP/1.1" 502 150 2025-12-17 23:13:29,391 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937a575bc060ce160d96'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:31,351 - lava_callback - INFO - Received callback for node 69409611575bc060ce1614ce INFO: 172.21.0.1:39050 - "POST /node/69409611575bc060ce1614ce HTTP/1.0" 202 Accepted 2025-12-17 23:13:31,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a22575bc060ce161eee 2025-12-17 23:13:31,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a22575bc060ce161eee HTTP/1.1" 502 150 2025-12-17 23:13:31,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a22575bc060ce161eee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:31,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409376575bc060ce160d8d 2025-12-17 23:13:31,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409376575bc060ce160d8d HTTP/1.1" 502 150 2025-12-17 23:13:31,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409376575bc060ce160d8d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:32,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409377575bc060ce160d90 2025-12-17 23:13:32,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409377575bc060ce160d90 HTTP/1.1" 502 150 2025-12-17 23:13:32,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409377575bc060ce160d90'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:33,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937e575bc060ce160d9f 2025-12-17 23:13:33,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937e575bc060ce160d9f HTTP/1.1" 502 150 2025-12-17 23:13:33,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937e575bc060ce160d9f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:34,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409378575bc060ce160d93 2025-12-17 23:13:35,815 - lava_callback - INFO - Received callback for node 694098c0575bc060ce161b52 INFO: 172.21.0.1:39054 - "POST /node/694098c0575bc060ce161b52 HTTP/1.0" 202 Accepted 2025-12-17 23:13:36,312 - lava_callback - INFO - Received callback for node 6940960c575bc060ce1614be INFO: 172.21.0.1:39062 - "POST /node/6940960c575bc060ce1614be HTTP/1.0" 202 Accepted INFO: 172.21.0.1:39078 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:13:36,984 - lava_callback - INFO - Received callback for node 6940960e575bc060ce1614c1 2025-12-17 23:13:36,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409380575bc060ce160da2 INFO: 172.21.0.1:39080 - "POST /node/6940960e575bc060ce1614c1 HTTP/1.0" 202 Accepted 2025-12-17 23:13:36,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409380575bc060ce160da2 HTTP/1.1" 502 150 2025-12-17 23:13:36,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409380575bc060ce160da2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:37,391 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937a575bc060ce160d96 2025-12-17 23:13:37,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937a575bc060ce160d96 HTTP/1.1" 502 150 2025-12-17 23:13:37,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940937a575bc060ce160d96'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:37,686 - lava_callback - INFO - Received callback for node 6940960f575bc060ce1614c4 INFO: 172.21.0.1:39090 - "POST /node/6940960f575bc060ce1614c4 HTTP/1.0" 202 Accepted 2025-12-17 23:13:38,352 - lava_callback - INFO - Received callback for node 69409610575bc060ce1614cb INFO: 172.21.0.1:39096 - "POST /node/69409610575bc060ce1614cb HTTP/1.0" 202 Accepted 2025-12-17 23:13:39,030 - lava_callback - INFO - Received callback for node 69409613575bc060ce1614d4 INFO: 172.21.0.1:39110 - "POST /node/69409613575bc060ce1614d4 HTTP/1.0" 202 Accepted 2025-12-17 23:13:39,699 - lava_callback - INFO - Received callback for node 69409614575bc060ce1614d7 INFO: 172.21.0.1:52942 - "POST /node/69409614575bc060ce1614d7 HTTP/1.0" 202 Accepted 2025-12-17 23:13:39,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a22575bc060ce161eee 2025-12-17 23:13:39,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a22575bc060ce161eee HTTP/1.1" 502 150 2025-12-17 23:13:39,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a22575bc060ce161eee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:39,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409367575bc060ce160d63 2025-12-17 23:13:39,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409367575bc060ce160d63 HTTP/1.1" 502 150 2025-12-17 23:13:39,975 - lava_callback - ERROR - Error processing callback for node 69409367575bc060ce160d63: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409367575bc060ce160d63 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409367575bc060ce160d63 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409367575bc060ce160d63 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:39,978 - lava_callback - INFO - Starting async job submit for node 69409382575bc060ce160dab 2025-12-17 23:13:39,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:39,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409382575bc060ce160dab HTTP/1.1" 502 150 2025-12-17 23:13:39,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409382575bc060ce160dab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:39,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409382575bc060ce160dab 2025-12-17 23:13:39,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409382575bc060ce160dab HTTP/1.1" 502 150 2025-12-17 23:13:39,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409382575bc060ce160dab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:40,273 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409369575bc060ce160d66 2025-12-17 23:13:40,340 - lava_callback - INFO - Received callback for node 69409616575bc060ce1614de INFO: 172.21.0.1:52952 - "POST /node/69409616575bc060ce1614de HTTP/1.0" 202 Accepted 2025-12-17 23:13:40,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940936e575bc060ce160d75 2025-12-17 23:13:40,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940936e575bc060ce160d75 HTTP/1.1" 502 150 2025-12-17 23:13:40,869 - lava_callback - ERROR - Error processing callback for node 6940936e575bc060ce160d75: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940936e575bc060ce160d75 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940936e575bc060ce160d75 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940936e575bc060ce160d75 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:40,872 - lava_callback - INFO - Starting async job submit for node 69409384575bc060ce160db4 2025-12-17 23:13:40,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:40,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409691575bc060ce161615 2025-12-17 23:13:40,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409691575bc060ce161615 HTTP/1.1" 502 150 2025-12-17 23:13:40,875 - lava_callback - ERROR - Error processing callback for node 69409691575bc060ce161615: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409691575bc060ce161615 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409691575bc060ce161615 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409691575bc060ce161615 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:40,877 - lava_callback - INFO - Starting async job submit for node 69409386575bc060ce160db7 2025-12-17 23:13:40,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:40,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409384575bc060ce160db4 HTTP/1.1" 502 150 2025-12-17 23:13:40,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409384575bc060ce160db4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:40,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409384575bc060ce160db4 2025-12-17 23:13:40,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409384575bc060ce160db4 HTTP/1.1" 502 150 2025-12-17 23:13:40,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409384575bc060ce160db4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:40,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409386575bc060ce160db7 HTTP/1.1" 502 150 2025-12-17 23:13:40,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409386575bc060ce160db7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:40,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409386575bc060ce160db7 2025-12-17 23:13:40,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409386575bc060ce160db7 HTTP/1.1" 502 150 2025-12-17 23:13:40,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409386575bc060ce160db7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:40,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409377575bc060ce160d90 2025-12-17 23:13:40,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409377575bc060ce160d90 HTTP/1.1" 502 150 2025-12-17 23:13:40,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409377575bc060ce160d90'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:41,019 - lava_callback - INFO - Received callback for node 69409617575bc060ce1614e4 INFO: 172.21.0.1:52962 - "POST /node/69409617575bc060ce1614e4 HTTP/1.0" 202 Accepted 2025-12-17 23:13:41,680 - lava_callback - INFO - Received callback for node 6940961b575bc060ce1614f1 INFO: 172.21.0.1:52966 - "POST /node/6940961b575bc060ce1614f1 HTTP/1.0" 202 Accepted 2025-12-17 23:13:41,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409382575bc060ce160dab 2025-12-17 23:13:41,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409382575bc060ce160dab HTTP/1.1" 502 150 2025-12-17 23:13:41,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409382575bc060ce160dab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:42,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409363575bc060ce160d5a HTTP/1.1" 502 150 2025-12-17 23:13:42,068 - lava_callback - ERROR - Error processing callback for node 69409363575bc060ce160d5a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409363575bc060ce160d5a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409363575bc060ce160d5a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409363575bc060ce160d5a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:42,071 - lava_callback - INFO - Starting async job submit for node 69409389575bc060ce160dbd 2025-12-17 23:13:42,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:42,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409389575bc060ce160dbd HTTP/1.1" 502 150 2025-12-17 23:13:42,078 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409389575bc060ce160dbd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:42,078 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409389575bc060ce160dbd 2025-12-17 23:13:42,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409389575bc060ce160dbd HTTP/1.1" 502 150 2025-12-17 23:13:42,079 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409389575bc060ce160dbd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:42,334 - lava_callback - INFO - Received callback for node 6940961c575bc060ce1614f4 INFO: 172.21.0.1:52980 - "POST /node/6940961c575bc060ce1614f4 HTTP/1.0" 202 Accepted 2025-12-17 23:13:42,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409371575bc060ce160d80 2025-12-17 23:13:42,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409371575bc060ce160d80 HTTP/1.1" 502 150 2025-12-17 23:13:42,772 - lava_callback - ERROR - Error processing callback for node 69409371575bc060ce160d80: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409371575bc060ce160d80 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409371575bc060ce160d80 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409371575bc060ce160d80 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:42,774 - lava_callback - INFO - Starting async job submit for node 6940938a575bc060ce160dc0 2025-12-17 23:13:42,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:42,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938a575bc060ce160dc0 HTTP/1.1" 502 150 2025-12-17 23:13:42,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940938a575bc060ce160dc0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:42,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940938a575bc060ce160dc0 2025-12-17 23:13:42,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938a575bc060ce160dc0 HTTP/1.1" 502 150 2025-12-17 23:13:42,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940938a575bc060ce160dc0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:42,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409384575bc060ce160db4 2025-12-17 23:13:42,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409384575bc060ce160db4 HTTP/1.1" 502 150 2025-12-17 23:13:42,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409384575bc060ce160db4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:42,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409386575bc060ce160db7 2025-12-17 23:13:42,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409386575bc060ce160db7 HTTP/1.1" 502 150 2025-12-17 23:13:42,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409386575bc060ce160db7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:42,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409374575bc060ce160d8a 2025-12-17 23:13:42,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409374575bc060ce160d8a HTTP/1.1" 502 150 2025-12-17 23:13:42,979 - lava_callback - ERROR - Error processing callback for node 69409374575bc060ce160d8a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409374575bc060ce160d8a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409374575bc060ce160d8a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409374575bc060ce160d8a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:42,982 - lava_callback - INFO - Starting async job submit for node 6940938e575bc060ce160dc6 2025-12-17 23:13:42,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:42,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938e575bc060ce160dc6 HTTP/1.1" 502 150 2025-12-17 23:13:42,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940938e575bc060ce160dc6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:42,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940938e575bc060ce160dc6 2025-12-17 23:13:42,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938e575bc060ce160dc6 HTTP/1.1" 502 150 2025-12-17 23:13:42,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940938e575bc060ce160dc6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:43,020 - lava_callback - INFO - Received callback for node 6940961d575bc060ce1614f7 INFO: 172.21.0.1:52994 - "POST /node/6940961d575bc060ce1614f7 HTTP/1.0" 202 Accepted 2025-12-17 23:13:43,698 - lava_callback - INFO - Received callback for node 6940961e575bc060ce1614fa INFO: 172.21.0.1:52996 - "POST /node/6940961e575bc060ce1614fa HTTP/1.0" 202 Accepted 2025-12-17 23:13:43,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409366575bc060ce160d60 2025-12-17 23:13:43,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409366575bc060ce160d60 HTTP/1.1" 502 150 2025-12-17 23:13:43,956 - lava_callback - ERROR - Error processing callback for node 69409366575bc060ce160d60: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409366575bc060ce160d60 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409366575bc060ce160d60 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409366575bc060ce160d60 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:43,958 - lava_callback - INFO - Starting async job submit for node 69409390575bc060ce160dc9 2025-12-17 23:13:43,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:43,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409390575bc060ce160dc9 HTTP/1.1" 502 150 2025-12-17 23:13:43,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409390575bc060ce160dc9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:43,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409390575bc060ce160dc9 2025-12-17 23:13:43,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409390575bc060ce160dc9 HTTP/1.1" 502 150 2025-12-17 23:13:43,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409390575bc060ce160dc9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:44,079 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409389575bc060ce160dbd 2025-12-17 23:13:44,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409389575bc060ce160dbd HTTP/1.1" 502 150 2025-12-17 23:13:44,081 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409389575bc060ce160dbd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:44,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940938a575bc060ce160dc0 2025-12-17 23:13:44,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938a575bc060ce160dc0 HTTP/1.1" 502 150 2025-12-17 23:13:44,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940938a575bc060ce160dc0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:44,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937b575bc060ce160d9c 2025-12-17 23:13:44,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937b575bc060ce160d9c HTTP/1.1" 502 150 2025-12-17 23:13:44,846 - lava_callback - ERROR - Error processing callback for node 6940937b575bc060ce160d9c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940937b575bc060ce160d9c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940937b575bc060ce160d9c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940937b575bc060ce160d9c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:44,848 - lava_callback - INFO - Starting async job submit for node 69409392575bc060ce160dcc 2025-12-17 23:13:44,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:44,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409392575bc060ce160dcc HTTP/1.1" 502 150 2025-12-17 23:13:44,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409392575bc060ce160dcc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:44,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409392575bc060ce160dcc 2025-12-17 23:13:44,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409392575bc060ce160dcc HTTP/1.1" 502 150 2025-12-17 23:13:44,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409392575bc060ce160dcc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:44,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940938e575bc060ce160dc6 2025-12-17 23:13:44,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938e575bc060ce160dc6 HTTP/1.1" 502 150 2025-12-17 23:13:44,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940938e575bc060ce160dc6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:45,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409390575bc060ce160dc9 2025-12-17 23:13:45,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409390575bc060ce160dc9 HTTP/1.1" 502 150 2025-12-17 23:13:45,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409390575bc060ce160dc9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:45,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409382575bc060ce160dab 2025-12-17 23:13:45,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409382575bc060ce160dab HTTP/1.1" 502 150 2025-12-17 23:13:45,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409382575bc060ce160dab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52998 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:13:46,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409392575bc060ce160dcc 2025-12-17 23:13:46,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409392575bc060ce160dcc HTTP/1.1" 502 150 2025-12-17 23:13:46,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409392575bc060ce160dcc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:46,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409384575bc060ce160db4 2025-12-17 23:13:46,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409384575bc060ce160db4 HTTP/1.1" 502 150 2025-12-17 23:13:46,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409384575bc060ce160db4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:46,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409386575bc060ce160db7 2025-12-17 23:13:46,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409386575bc060ce160db7 HTTP/1.1" 502 150 2025-12-17 23:13:46,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409386575bc060ce160db7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:47,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409376575bc060ce160d8d 2025-12-17 23:13:47,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409376575bc060ce160d8d HTTP/1.1" 502 150 2025-12-17 23:13:47,981 - lava_callback - ERROR - Error processing callback for node 69409376575bc060ce160d8d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409376575bc060ce160d8d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409376575bc060ce160d8d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409376575bc060ce160d8d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:47,983 - lava_callback - INFO - Starting async job submit for node 69409388575bc060ce160dba 2025-12-17 23:13:47,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:47,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409388575bc060ce160dba HTTP/1.1" 502 150 2025-12-17 23:13:47,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409388575bc060ce160dba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:47,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409388575bc060ce160dba 2025-12-17 23:13:47,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409388575bc060ce160dba HTTP/1.1" 502 150 2025-12-17 23:13:47,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409388575bc060ce160dba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:48,081 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409389575bc060ce160dbd 2025-12-17 23:13:48,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409389575bc060ce160dbd HTTP/1.1" 502 150 2025-12-17 23:13:48,082 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409389575bc060ce160dbd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:48,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940938a575bc060ce160dc0 2025-12-17 23:13:48,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938a575bc060ce160dc0 HTTP/1.1" 502 150 2025-12-17 23:13:48,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940938a575bc060ce160dc0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:48,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940938e575bc060ce160dc6 2025-12-17 23:13:49,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937e575bc060ce160d9f 2025-12-17 23:13:49,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937e575bc060ce160d9f HTTP/1.1" 502 150 2025-12-17 23:13:49,900 - lava_callback - ERROR - Error processing callback for node 6940937e575bc060ce160d9f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940937e575bc060ce160d9f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940937e575bc060ce160d9f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940937e575bc060ce160d9f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:49,902 - lava_callback - INFO - Starting async job submit for node 69409393575bc060ce160dcf 2025-12-17 23:13:49,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:49,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409393575bc060ce160dcf HTTP/1.1" 502 150 2025-12-17 23:13:49,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409393575bc060ce160dcf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:49,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409393575bc060ce160dcf 2025-12-17 23:13:49,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409393575bc060ce160dcf HTTP/1.1" 502 150 2025-12-17 23:13:49,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409393575bc060ce160dcf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:49,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409390575bc060ce160dc9 2025-12-17 23:13:49,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409390575bc060ce160dc9 HTTP/1.1" 502 150 2025-12-17 23:13:49,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409390575bc060ce160dc9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:49,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409388575bc060ce160dba 2025-12-17 23:13:49,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409388575bc060ce160dba HTTP/1.1" 502 150 2025-12-17 23:13:49,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409388575bc060ce160dba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:50,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409378575bc060ce160d93 HTTP/1.1" 502 150 2025-12-17 23:13:50,256 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409378575bc060ce160d93'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:50,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409392575bc060ce160dcc 2025-12-17 23:13:50,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409392575bc060ce160dcc HTTP/1.1" 502 150 2025-12-17 23:13:50,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409392575bc060ce160dcc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:51,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409393575bc060ce160dcf 2025-12-17 23:13:51,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409393575bc060ce160dcf HTTP/1.1" 502 150 2025-12-17 23:13:51,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409393575bc060ce160dcf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:52,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409380575bc060ce160da2 2025-12-17 23:13:52,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409380575bc060ce160da2 HTTP/1.1" 502 150 2025-12-17 23:13:52,995 - lava_callback - ERROR - Error processing callback for node 69409380575bc060ce160da2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409380575bc060ce160da2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409380575bc060ce160da2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409380575bc060ce160da2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:52,997 - lava_callback - INFO - Starting async job submit for node 69409396575bc060ce160dd9 2025-12-17 23:13:52,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:53,393 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940937a575bc060ce160d96 2025-12-17 23:13:53,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940937a575bc060ce160d96 HTTP/1.1" 502 150 2025-12-17 23:13:53,394 - lava_callback - ERROR - Error processing callback for node 6940937a575bc060ce160d96: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940937a575bc060ce160d96 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940937a575bc060ce160d96 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940937a575bc060ce160d96 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:53,397 - lava_callback - INFO - Starting async job submit for node 69409395575bc060ce160dd2 2025-12-17 23:13:53,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:53,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409395575bc060ce160dd2 HTTP/1.1" 502 150 2025-12-17 23:13:53,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409395575bc060ce160dd2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:53,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409395575bc060ce160dd2 2025-12-17 23:13:53,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409395575bc060ce160dd2 HTTP/1.1" 502 150 2025-12-17 23:13:53,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409395575bc060ce160dd2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:53,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409382575bc060ce160dab 2025-12-17 23:13:53,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409382575bc060ce160dab HTTP/1.1" 502 150 2025-12-17 23:13:53,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409382575bc060ce160dab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:53,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409388575bc060ce160dba 2025-12-17 23:13:53,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409388575bc060ce160dba HTTP/1.1" 502 150 2025-12-17 23:13:53,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409388575bc060ce160dba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:54,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409384575bc060ce160db4 2025-12-17 23:13:54,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409384575bc060ce160db4 HTTP/1.1" 502 150 2025-12-17 23:13:54,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409384575bc060ce160db4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:54,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409386575bc060ce160db7 2025-12-17 23:13:54,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409386575bc060ce160db7 HTTP/1.1" 502 150 2025-12-17 23:13:54,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409386575bc060ce160db7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:55,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409395575bc060ce160dd2 2025-12-17 23:13:55,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409395575bc060ce160dd2 HTTP/1.1" 502 150 2025-12-17 23:13:55,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409395575bc060ce160dd2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:55,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409369575bc060ce160d66 HTTP/1.1" 502 150 2025-12-17 23:13:55,632 - lava_callback - ERROR - Error processing callback for node 69409369575bc060ce160d66: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409369575bc060ce160d66 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409369575bc060ce160d66 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409369575bc060ce160d66 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:55,634 - lava_callback - INFO - Starting async job submit for node 69409420575bc060ce160f6c 2025-12-17 23:13:55,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:55,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409420575bc060ce160f6c HTTP/1.1" 502 150 2025-12-17 23:13:55,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409420575bc060ce160f6c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:55,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409420575bc060ce160f6c 2025-12-17 23:13:55,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409420575bc060ce160f6c HTTP/1.1" 502 150 2025-12-17 23:13:55,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409420575bc060ce160f6c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:55,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a22575bc060ce161eee 2025-12-17 23:13:55,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a22575bc060ce161eee HTTP/1.1" 502 150 2025-12-17 23:13:55,814 - lava_callback - ERROR - Error processing callback for node 69409a22575bc060ce161eee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a22575bc060ce161eee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a22575bc060ce161eee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a22575bc060ce161eee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:55,816 - lava_callback - INFO - Starting async job submit for node 6940938c575bc060ce160dc3 2025-12-17 23:13:55,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:55,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938c575bc060ce160dc3 HTTP/1.1" 502 150 2025-12-17 23:13:55,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940938c575bc060ce160dc3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:55,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940938c575bc060ce160dc3 2025-12-17 23:13:55,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938c575bc060ce160dc3 HTTP/1.1" 502 150 2025-12-17 23:13:55,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940938c575bc060ce160dc3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:55,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409393575bc060ce160dcf 2025-12-17 23:13:55,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409393575bc060ce160dcf HTTP/1.1" 502 150 2025-12-17 23:13:55,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409393575bc060ce160dcf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:56,082 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409389575bc060ce160dbd 2025-12-17 23:13:56,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409389575bc060ce160dbd HTTP/1.1" 502 150 2025-12-17 23:13:56,084 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409389575bc060ce160dbd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:56,345 - lava_callback - INFO - Received callback for node 6940961f575bc060ce1614fd INFO: 172.21.0.1:32890 - "POST /node/6940961f575bc060ce1614fd HTTP/1.0" 202 Accepted INFO: 172.21.0.1:32896 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:13:56,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940938a575bc060ce160dc0 2025-12-17 23:13:56,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938a575bc060ce160dc0 HTTP/1.1" 502 150 2025-12-17 23:13:56,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940938a575bc060ce160dc0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:56,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409377575bc060ce160d90 2025-12-17 23:13:56,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409377575bc060ce160d90 HTTP/1.1" 502 150 2025-12-17 23:13:56,978 - lava_callback - ERROR - Error processing callback for node 69409377575bc060ce160d90: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409377575bc060ce160d90 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409377575bc060ce160d90 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409377575bc060ce160d90 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:13:56,979 - lava_callback - INFO - Starting async job submit for node 69409398575bc060ce160de0 2025-12-17 23:13:56,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:13:56,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409398575bc060ce160de0 HTTP/1.1" 502 150 2025-12-17 23:13:56,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409398575bc060ce160de0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:56,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409398575bc060ce160de0 2025-12-17 23:13:56,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409398575bc060ce160de0 HTTP/1.1" 502 150 2025-12-17 23:13:56,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409398575bc060ce160de0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:56,994 - lava_callback - INFO - Received callback for node 69409621575bc060ce161500 INFO: 172.21.0.1:32900 - "POST /node/69409621575bc060ce161500 HTTP/1.0" 202 Accepted 2025-12-17 23:13:57,640 - lava_callback - INFO - Received callback for node 69409622575bc060ce161503 INFO: 172.21.0.1:32910 - "POST /node/69409622575bc060ce161503 HTTP/1.0" 202 Accepted 2025-12-17 23:13:57,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409420575bc060ce160f6c 2025-12-17 23:13:57,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409420575bc060ce160f6c HTTP/1.1" 502 150 2025-12-17 23:13:57,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409420575bc060ce160f6c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:57,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940938c575bc060ce160dc3 2025-12-17 23:13:57,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938c575bc060ce160dc3 HTTP/1.1" 502 150 2025-12-17 23:13:57,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940938c575bc060ce160dc3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:57,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409390575bc060ce160dc9 2025-12-17 23:13:57,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409390575bc060ce160dc9 HTTP/1.1" 502 150 2025-12-17 23:13:57,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409390575bc060ce160dc9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:58,285 - lava_callback - INFO - Received callback for node 69409624575bc060ce161506 INFO: 172.21.0.1:32912 - "POST /node/69409624575bc060ce161506 HTTP/1.0" 202 Accepted 2025-12-17 23:13:58,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409392575bc060ce160dcc 2025-12-17 23:13:58,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409392575bc060ce160dcc HTTP/1.1" 502 150 2025-12-17 23:13:58,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409392575bc060ce160dcc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:58,966 - lava_callback - INFO - Received callback for node 69409625575bc060ce161509 INFO: 172.21.0.1:32924 - "POST /node/69409625575bc060ce161509 HTTP/1.0" 202 Accepted 2025-12-17 23:13:58,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409398575bc060ce160de0 2025-12-17 23:13:58,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409398575bc060ce160de0 HTTP/1.1" 502 150 2025-12-17 23:13:58,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409398575bc060ce160de0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:59,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409395575bc060ce160dd2 2025-12-17 23:13:59,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409395575bc060ce160dd2 HTTP/1.1" 502 150 2025-12-17 23:13:59,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409395575bc060ce160dd2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:13:59,651 - lava_callback - INFO - Received callback for node 69409626575bc060ce16150c INFO: 172.21.0.1:32930 - "POST /node/69409626575bc060ce16150c HTTP/1.0" 202 Accepted 2025-12-17 23:14:00,307 - lava_callback - INFO - Received callback for node 69409627575bc060ce16150f INFO: 172.21.0.1:50688 - "POST /node/69409627575bc060ce16150f HTTP/1.0" 202 Accepted 2025-12-17 23:14:00,990 - lava_callback - INFO - Received callback for node 69409628575bc060ce161512 INFO: 172.21.0.1:50694 - "POST /node/69409628575bc060ce161512 HTTP/1.0" 202 Accepted 2025-12-17 23:14:01,646 - lava_callback - INFO - Received callback for node 6940962e575bc060ce161527 INFO: 172.21.0.1:50696 - "POST /node/6940962e575bc060ce161527 HTTP/1.0" 202 Accepted 2025-12-17 23:14:01,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409420575bc060ce160f6c 2025-12-17 23:14:01,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940938c575bc060ce160dc3 2025-12-17 23:14:01,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938c575bc060ce160dc3 HTTP/1.1" 502 150 2025-12-17 23:14:01,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940938c575bc060ce160dc3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:01,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409388575bc060ce160dba 2025-12-17 23:14:01,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409388575bc060ce160dba HTTP/1.1" 502 150 2025-12-17 23:14:01,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409388575bc060ce160dba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:02,319 - lava_callback - INFO - Received callback for node 69409630575bc060ce16152f INFO: 172.21.0.1:50702 - "POST /node/69409630575bc060ce16152f HTTP/1.0" 202 Accepted 2025-12-17 23:14:02,976 - lava_callback - INFO - Received callback for node 69409631575bc060ce161532 INFO: 172.21.0.1:50706 - "POST /node/69409631575bc060ce161532 HTTP/1.0" 202 Accepted 2025-12-17 23:14:02,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409398575bc060ce160de0 2025-12-17 23:14:02,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409398575bc060ce160de0 HTTP/1.1" 502 150 2025-12-17 23:14:02,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409398575bc060ce160de0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50714 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:14:03,637 - lava_callback - INFO - Received callback for node 69409632575bc060ce161535 INFO: 172.21.0.1:50718 - "POST /node/69409632575bc060ce161535 HTTP/1.0" 202 Accepted 2025-12-17 23:14:03,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409393575bc060ce160dcf 2025-12-17 23:14:03,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409393575bc060ce160dcf HTTP/1.1" 502 150 2025-12-17 23:14:03,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409393575bc060ce160dcf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:04,316 - lava_callback - INFO - Received callback for node 694096a0575bc060ce161630 INFO: 172.21.0.1:50732 - "POST /node/694096a0575bc060ce161630 HTTP/1.0" 202 Accepted 2025-12-17 23:14:04,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938e575bc060ce160dc6 HTTP/1.1" 502 150 2025-12-17 23:14:04,336 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940938e575bc060ce160dc6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:06,021 - lava_callback - INFO - Received callback for node 694097fc575bc060ce161985 INFO: 172.21.0.1:50742 - "POST /node/694097fc575bc060ce161985 HTTP/1.0" 202 Accepted 2025-12-17 23:14:06,257 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409378575bc060ce160d93 INFO: 172.21.0.1:50758 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:14:06,838 - lava_callback - INFO - Received callback for node 6940962a575bc060ce161519 INFO: 172.21.0.1:50762 - "POST /node/6940962a575bc060ce161519 HTTP/1.0" 202 Accepted 2025-12-17 23:14:07,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409395575bc060ce160dd2 2025-12-17 23:14:07,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409395575bc060ce160dd2 HTTP/1.1" 502 150 2025-12-17 23:14:07,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409395575bc060ce160dd2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:07,488 - lava_callback - INFO - Received callback for node 6940962c575bc060ce161524 INFO: 172.21.0.1:50764 - "POST /node/6940962c575bc060ce161524 HTTP/1.0" 202 Accepted 2025-12-17 23:14:08,128 - lava_callback - INFO - Received callback for node 6940963b575bc060ce161549 INFO: 172.21.0.1:50780 - "POST /node/6940963b575bc060ce161549 HTTP/1.0" 202 Accepted 2025-12-17 23:14:08,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409396575bc060ce160dd9 HTTP/1.1" 502 150 2025-12-17 23:14:08,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409396575bc060ce160dd9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:08,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409396575bc060ce160dd9 2025-12-17 23:14:08,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409396575bc060ce160dd9 HTTP/1.1" 502 150 2025-12-17 23:14:08,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409396575bc060ce160dd9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:08,768 - lava_callback - INFO - Received callback for node 6940963d575bc060ce16154e INFO: 172.21.0.1:50796 - "POST /node/6940963d575bc060ce16154e HTTP/1.0" 202 Accepted 2025-12-17 23:14:09,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940938c575bc060ce160dc3 2025-12-17 23:14:09,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938c575bc060ce160dc3 HTTP/1.1" 502 150 2025-12-17 23:14:09,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940938c575bc060ce160dc3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:09,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409382575bc060ce160dab 2025-12-17 23:14:09,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409382575bc060ce160dab HTTP/1.1" 502 150 2025-12-17 23:14:09,991 - lava_callback - ERROR - Error processing callback for node 69409382575bc060ce160dab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409382575bc060ce160dab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409382575bc060ce160dab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409382575bc060ce160dab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:09,993 - lava_callback - INFO - Starting async job submit for node 6940939c575bc060ce160de6 2025-12-17 23:14:09,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:10,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939c575bc060ce160de6 HTTP/1.1" 502 150 2025-12-17 23:14:10,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940939c575bc060ce160de6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:10,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940939c575bc060ce160de6 2025-12-17 23:14:10,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939c575bc060ce160de6 HTTP/1.1" 502 150 2025-12-17 23:14:10,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940939c575bc060ce160de6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:10,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409396575bc060ce160dd9 2025-12-17 23:14:10,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409396575bc060ce160dd9 HTTP/1.1" 502 150 2025-12-17 23:14:10,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409396575bc060ce160dd9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:10,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409384575bc060ce160db4 2025-12-17 23:14:10,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409384575bc060ce160db4 HTTP/1.1" 502 150 2025-12-17 23:14:10,891 - lava_callback - ERROR - Error processing callback for node 69409384575bc060ce160db4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409384575bc060ce160db4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409384575bc060ce160db4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409384575bc060ce160db4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:10,893 - lava_callback - INFO - Starting async job submit for node 6940939e575bc060ce160dee 2025-12-17 23:14:10,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409386575bc060ce160db7 2025-12-17 23:14:10,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:10,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409386575bc060ce160db7 HTTP/1.1" 502 150 2025-12-17 23:14:10,895 - lava_callback - ERROR - Error processing callback for node 69409386575bc060ce160db7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409386575bc060ce160db7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409386575bc060ce160db7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409386575bc060ce160db7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:10,898 - lava_callback - INFO - Starting async job submit for node 694093a0575bc060ce160df1 2025-12-17 23:14:10,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:10,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a0575bc060ce160df1 HTTP/1.1" 502 150 2025-12-17 23:14:10,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a0575bc060ce160df1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:10,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a0575bc060ce160df1 2025-12-17 23:14:10,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939e575bc060ce160dee HTTP/1.1" 502 150 2025-12-17 23:14:10,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940939e575bc060ce160dee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:10,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940939e575bc060ce160dee 2025-12-17 23:14:10,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a0575bc060ce160df1 HTTP/1.1" 502 150 2025-12-17 23:14:10,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a0575bc060ce160df1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:10,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939e575bc060ce160dee HTTP/1.1" 502 150 2025-12-17 23:14:10,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940939e575bc060ce160dee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:10,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409398575bc060ce160de0 2025-12-17 23:14:10,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409398575bc060ce160de0 HTTP/1.1" 502 150 2025-12-17 23:14:10,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409398575bc060ce160de0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:11,431 - lava_callback - INFO - Received callback for node 6940963a575bc060ce161546 INFO: 172.21.0.1:33952 - "POST /node/6940963a575bc060ce161546 HTTP/1.0" 202 Accepted 2025-12-17 23:14:12,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940939c575bc060ce160de6 2025-12-17 23:14:12,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939c575bc060ce160de6 HTTP/1.1" 502 150 2025-12-17 23:14:12,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940939c575bc060ce160de6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:12,084 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409389575bc060ce160dbd 2025-12-17 23:14:12,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409389575bc060ce160dbd HTTP/1.1" 502 150 2025-12-17 23:14:12,085 - lava_callback - ERROR - Error processing callback for node 69409389575bc060ce160dbd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409389575bc060ce160dbd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409389575bc060ce160dbd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409389575bc060ce160dbd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:12,088 - lava_callback - INFO - Starting async job submit for node 694113d8460bc8e265117e35 2025-12-17 23:14:12,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:12,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694113d8460bc8e265117e35 HTTP/1.1" 502 150 2025-12-17 23:14:12,124 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694113d8460bc8e265117e35'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:12,124 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694113d8460bc8e265117e35 2025-12-17 23:14:12,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694113d8460bc8e265117e35 HTTP/1.1" 502 150 2025-12-17 23:14:12,125 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694113d8460bc8e265117e35'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:12,336 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940938e575bc060ce160dc6 2025-12-17 23:14:12,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938e575bc060ce160dc6 HTTP/1.1" 502 150 2025-12-17 23:14:12,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940938e575bc060ce160dc6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:12,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940938a575bc060ce160dc0 2025-12-17 23:14:12,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938a575bc060ce160dc0 HTTP/1.1" 502 150 2025-12-17 23:14:12,789 - lava_callback - ERROR - Error processing callback for node 6940938a575bc060ce160dc0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940938a575bc060ce160dc0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940938a575bc060ce160dc0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940938a575bc060ce160dc0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:12,792 - lava_callback - INFO - Starting async job submit for node 694093a2575bc060ce160df4 2025-12-17 23:14:12,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:12,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a2575bc060ce160df4 HTTP/1.1" 502 150 2025-12-17 23:14:12,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a2575bc060ce160df4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:12,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a2575bc060ce160df4 2025-12-17 23:14:12,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a2575bc060ce160df4 HTTP/1.1" 502 150 2025-12-17 23:14:12,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a2575bc060ce160df4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:12,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a0575bc060ce160df1 2025-12-17 23:14:12,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940939e575bc060ce160dee 2025-12-17 23:14:12,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a0575bc060ce160df1 HTTP/1.1" 502 150 2025-12-17 23:14:12,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a0575bc060ce160df1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:12,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939e575bc060ce160dee HTTP/1.1" 502 150 2025-12-17 23:14:12,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940939e575bc060ce160dee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:13,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409390575bc060ce160dc9 2025-12-17 23:14:13,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409390575bc060ce160dc9 HTTP/1.1" 502 150 2025-12-17 23:14:13,970 - lava_callback - ERROR - Error processing callback for node 69409390575bc060ce160dc9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409390575bc060ce160dc9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409390575bc060ce160dc9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409390575bc060ce160dc9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:13,972 - lava_callback - INFO - Starting async job submit for node 694093a4575bc060ce160df7 2025-12-17 23:14:13,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:13,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a4575bc060ce160df7 HTTP/1.1" 502 150 2025-12-17 23:14:13,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a4575bc060ce160df7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:13,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a4575bc060ce160df7 2025-12-17 23:14:13,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a4575bc060ce160df7 HTTP/1.1" 502 150 2025-12-17 23:14:13,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a4575bc060ce160df7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:14,125 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694113d8460bc8e265117e35 2025-12-17 23:14:14,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694113d8460bc8e265117e35 HTTP/1.1" 502 150 2025-12-17 23:14:14,126 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694113d8460bc8e265117e35'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:14,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409396575bc060ce160dd9 2025-12-17 23:14:14,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409396575bc060ce160dd9 HTTP/1.1" 502 150 2025-12-17 23:14:14,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409396575bc060ce160dd9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:14,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a2575bc060ce160df4 2025-12-17 23:14:14,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a2575bc060ce160df4 HTTP/1.1" 502 150 2025-12-17 23:14:14,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a2575bc060ce160df4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:14,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409392575bc060ce160dcc 2025-12-17 23:14:14,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409392575bc060ce160dcc HTTP/1.1" 502 150 2025-12-17 23:14:14,861 - lava_callback - ERROR - Error processing callback for node 69409392575bc060ce160dcc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409392575bc060ce160dcc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409392575bc060ce160dcc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409392575bc060ce160dcc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:14,863 - lava_callback - INFO - Starting async job submit for node 694093a6575bc060ce160dfa 2025-12-17 23:14:14,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:14,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a6575bc060ce160dfa HTTP/1.1" 502 150 2025-12-17 23:14:14,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a6575bc060ce160dfa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:14,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a6575bc060ce160dfa 2025-12-17 23:14:14,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a6575bc060ce160dfa HTTP/1.1" 502 150 2025-12-17 23:14:14,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a6575bc060ce160dfa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33962 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:14:15,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a4575bc060ce160df7 2025-12-17 23:14:15,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a4575bc060ce160df7 HTTP/1.1" 502 150 2025-12-17 23:14:15,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a4575bc060ce160df7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:16,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940939c575bc060ce160de6 2025-12-17 23:14:16,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939c575bc060ce160de6 HTTP/1.1" 502 150 2025-12-17 23:14:16,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940939c575bc060ce160de6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:16,331 - lava_callback - INFO - Received callback for node 69409633575bc060ce16153a INFO: 172.21.0.1:33978 - "POST /node/69409633575bc060ce16153a HTTP/1.0" 202 Accepted INFO: 172.21.0.1:33986 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:14:16,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a6575bc060ce160dfa 2025-12-17 23:14:16,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a6575bc060ce160dfa HTTP/1.1" 502 150 2025-12-17 23:14:16,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a6575bc060ce160dfa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:16,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a0575bc060ce160df1 2025-12-17 23:14:16,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940939e575bc060ce160dee 2025-12-17 23:14:16,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a0575bc060ce160df1 HTTP/1.1" 502 150 2025-12-17 23:14:16,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a0575bc060ce160df1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:16,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939e575bc060ce160dee HTTP/1.1" 502 150 2025-12-17 23:14:16,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940939e575bc060ce160dee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:16,970 - lava_callback - INFO - Received callback for node 69409635575bc060ce16153d INFO: 172.21.0.1:33994 - "POST /node/69409635575bc060ce16153d HTTP/1.0" 202 Accepted 2025-12-17 23:14:17,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409420575bc060ce160f6c HTTP/1.1" 502 150 2025-12-17 23:14:17,136 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409420575bc060ce160f6c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:17,627 - lava_callback - INFO - Received callback for node 69409636575bc060ce161540 INFO: 172.21.0.1:34002 - "POST /node/69409636575bc060ce161540 HTTP/1.0" 202 Accepted 2025-12-17 23:14:17,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409388575bc060ce160dba 2025-12-17 23:14:18,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409388575bc060ce160dba HTTP/1.1" 502 150 2025-12-17 23:14:18,000 - lava_callback - ERROR - Error processing callback for node 69409388575bc060ce160dba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409388575bc060ce160dba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409388575bc060ce160dba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409388575bc060ce160dba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:18,002 - lava_callback - INFO - Starting async job submit for node 694093a8575bc060ce160dfd 2025-12-17 23:14:18,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:18,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a8575bc060ce160dfd HTTP/1.1" 502 150 2025-12-17 23:14:18,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a8575bc060ce160dfd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:18,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a8575bc060ce160dfd 2025-12-17 23:14:18,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a8575bc060ce160dfd HTTP/1.1" 502 150 2025-12-17 23:14:18,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a8575bc060ce160dfd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:18,127 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694113d8460bc8e265117e35 2025-12-17 23:14:18,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694113d8460bc8e265117e35 HTTP/1.1" 502 150 2025-12-17 23:14:18,128 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694113d8460bc8e265117e35'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:18,289 - lava_callback - INFO - Received callback for node 69409640575bc060ce161559 INFO: 172.21.0.1:34014 - "POST /node/69409640575bc060ce161559 HTTP/1.0" 202 Accepted 2025-12-17 23:14:18,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a2575bc060ce160df4 2025-12-17 23:14:18,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a2575bc060ce160df4 HTTP/1.1" 502 150 2025-12-17 23:14:18,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a2575bc060ce160df4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:18,981 - lava_callback - INFO - Received callback for node 69409644575bc060ce161563 INFO: 172.21.0.1:34030 - "POST /node/69409644575bc060ce161563 HTTP/1.0" 202 Accepted 2025-12-17 23:14:19,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409393575bc060ce160dcf 2025-12-17 23:14:19,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409393575bc060ce160dcf HTTP/1.1" 502 150 2025-12-17 23:14:19,914 - lava_callback - ERROR - Error processing callback for node 69409393575bc060ce160dcf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409393575bc060ce160dcf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409393575bc060ce160dcf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409393575bc060ce160dcf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:19,916 - lava_callback - INFO - Starting async job submit for node 694093a9575bc060ce160e04 2025-12-17 23:14:19,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:19,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a9575bc060ce160e04 HTTP/1.1" 502 150 2025-12-17 23:14:19,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a9575bc060ce160e04'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:19,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a9575bc060ce160e04 2025-12-17 23:14:19,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a9575bc060ce160e04 HTTP/1.1" 502 150 2025-12-17 23:14:19,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a9575bc060ce160e04'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:19,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a4575bc060ce160df7 2025-12-17 23:14:19,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a4575bc060ce160df7 HTTP/1.1" 502 150 2025-12-17 23:14:19,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a4575bc060ce160df7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:20,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a8575bc060ce160dfd 2025-12-17 23:14:20,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a8575bc060ce160dfd HTTP/1.1" 502 150 2025-12-17 23:14:20,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a8575bc060ce160dfd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:20,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a6575bc060ce160dfa 2025-12-17 23:14:20,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a6575bc060ce160dfa HTTP/1.1" 502 150 2025-12-17 23:14:20,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a6575bc060ce160dfa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:21,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409378575bc060ce160d93 HTTP/1.1" 502 150 2025-12-17 23:14:21,744 - lava_callback - ERROR - Error processing callback for node 69409378575bc060ce160d93: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409378575bc060ce160d93 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409378575bc060ce160d93 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409378575bc060ce160d93 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:21,746 - lava_callback - INFO - Starting async job submit for node 694093ab575bc060ce160e0f 2025-12-17 23:14:21,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:21,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ab575bc060ce160e0f HTTP/1.1" 502 150 2025-12-17 23:14:21,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ab575bc060ce160e0f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:21,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ab575bc060ce160e0f 2025-12-17 23:14:21,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ab575bc060ce160e0f HTTP/1.1" 502 150 2025-12-17 23:14:21,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ab575bc060ce160e0f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:21,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a9575bc060ce160e04 2025-12-17 23:14:21,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a9575bc060ce160e04 HTTP/1.1" 502 150 2025-12-17 23:14:21,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a9575bc060ce160e04'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:22,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409396575bc060ce160dd9 2025-12-17 23:14:22,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409396575bc060ce160dd9 HTTP/1.1" 502 150 2025-12-17 23:14:22,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409396575bc060ce160dd9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:23,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409395575bc060ce160dd2 2025-12-17 23:14:23,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409395575bc060ce160dd2 HTTP/1.1" 502 150 2025-12-17 23:14:23,435 - lava_callback - ERROR - Error processing callback for node 69409395575bc060ce160dd2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409395575bc060ce160dd2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409395575bc060ce160dd2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409395575bc060ce160dd2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:23,437 - lava_callback - INFO - Starting async job submit for node 694093b7575bc060ce160e29 2025-12-17 23:14:23,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:23,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b7575bc060ce160e29 HTTP/1.1" 502 150 2025-12-17 23:14:23,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b7575bc060ce160e29'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:23,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b7575bc060ce160e29 2025-12-17 23:14:23,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b7575bc060ce160e29 HTTP/1.1" 502 150 2025-12-17 23:14:23,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b7575bc060ce160e29'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:23,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ab575bc060ce160e0f 2025-12-17 23:14:23,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ab575bc060ce160e0f HTTP/1.1" 502 150 2025-12-17 23:14:23,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ab575bc060ce160e0f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:24,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940939c575bc060ce160de6 2025-12-17 23:14:24,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939c575bc060ce160de6 HTTP/1.1" 502 150 2025-12-17 23:14:24,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940939c575bc060ce160de6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:24,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a8575bc060ce160dfd 2025-12-17 23:14:24,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a8575bc060ce160dfd HTTP/1.1" 502 150 2025-12-17 23:14:24,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a8575bc060ce160dfd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:24,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a0575bc060ce160df1 2025-12-17 23:14:24,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940939e575bc060ce160dee 2025-12-17 23:14:24,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a0575bc060ce160df1 HTTP/1.1" 502 150 2025-12-17 23:14:24,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a0575bc060ce160df1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:24,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939e575bc060ce160dee HTTP/1.1" 502 150 2025-12-17 23:14:24,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940939e575bc060ce160dee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:25,136 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409420575bc060ce160f6c 2025-12-17 23:14:25,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409420575bc060ce160f6c HTTP/1.1" 502 150 2025-12-17 23:14:25,137 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409420575bc060ce160f6c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:25,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b7575bc060ce160e29 2025-12-17 23:14:25,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b7575bc060ce160e29 HTTP/1.1" 502 150 2025-12-17 23:14:25,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b7575bc060ce160e29'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:25,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940938c575bc060ce160dc3 2025-12-17 23:14:25,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938c575bc060ce160dc3 HTTP/1.1" 502 150 2025-12-17 23:14:25,854 - lava_callback - ERROR - Error processing callback for node 6940938c575bc060ce160dc3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940938c575bc060ce160dc3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940938c575bc060ce160dc3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940938c575bc060ce160dc3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:25,855 - lava_callback - INFO - Starting async job submit for node 694093ad575bc060ce160e14 2025-12-17 23:14:25,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:25,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ad575bc060ce160e14 HTTP/1.1" 502 150 2025-12-17 23:14:25,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ad575bc060ce160e14'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:25,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ad575bc060ce160e14 2025-12-17 23:14:25,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ad575bc060ce160e14 HTTP/1.1" 502 150 2025-12-17 23:14:25,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ad575bc060ce160e14'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:25,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a9575bc060ce160e04 2025-12-17 23:14:25,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a9575bc060ce160e04 HTTP/1.1" 502 150 2025-12-17 23:14:25,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a9575bc060ce160e04'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:26,128 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694113d8460bc8e265117e35 2025-12-17 23:14:26,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694113d8460bc8e265117e35 HTTP/1.1" 502 150 2025-12-17 23:14:26,129 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694113d8460bc8e265117e35'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35720 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:14:26,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a2575bc060ce160df4 2025-12-17 23:14:26,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a2575bc060ce160df4 HTTP/1.1" 502 150 2025-12-17 23:14:26,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a2575bc060ce160df4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:26,844 - lava_callback - INFO - Received callback for node 69409642575bc060ce161560 INFO: 172.21.0.1:35724 - "POST /node/69409642575bc060ce161560 HTTP/1.0" 202 Accepted 2025-12-17 23:14:26,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409398575bc060ce160de0 2025-12-17 23:14:26,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409398575bc060ce160de0 HTTP/1.1" 502 150 2025-12-17 23:14:26,995 - lava_callback - ERROR - Error processing callback for node 69409398575bc060ce160de0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409398575bc060ce160de0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409398575bc060ce160de0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409398575bc060ce160de0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:26,997 - lava_callback - INFO - Starting async job submit for node 694093ae575bc060ce160e17 2025-12-17 23:14:26,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:27,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ae575bc060ce160e17 HTTP/1.1" 502 150 2025-12-17 23:14:27,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ae575bc060ce160e17'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:27,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ae575bc060ce160e17 2025-12-17 23:14:27,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ae575bc060ce160e17 HTTP/1.1" 502 150 2025-12-17 23:14:27,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ae575bc060ce160e17'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:27,489 - lava_callback - INFO - Received callback for node 69409646575bc060ce161566 INFO: 172.21.0.1:35736 - "POST /node/69409646575bc060ce161566 HTTP/1.0" 202 Accepted 2025-12-17 23:14:27,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ab575bc060ce160e0f 2025-12-17 23:14:27,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ab575bc060ce160e0f HTTP/1.1" 502 150 2025-12-17 23:14:27,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ab575bc060ce160e0f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:27,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ad575bc060ce160e14 2025-12-17 23:14:27,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ad575bc060ce160e14 HTTP/1.1" 502 150 2025-12-17 23:14:27,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ad575bc060ce160e14'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:27,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a4575bc060ce160df7 2025-12-17 23:14:27,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a4575bc060ce160df7 HTTP/1.1" 502 150 2025-12-17 23:14:27,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a4575bc060ce160df7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:28,073 - lava_callback - INFO - Received callback for node 69411ffa459ff5fc64d2b337 INFO: 172.21.0.1:35742 - "POST /node/69411ffa459ff5fc64d2b337 HTTP/1.0" 202 Accepted 2025-12-17 23:14:28,138 - lava_callback - INFO - Received callback for node 69409648575bc060ce161569 INFO: 172.21.0.1:35752 - "POST /node/69409648575bc060ce161569 HTTP/1.0" 202 Accepted 2025-12-17 23:14:28,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940938e575bc060ce160dc6 2025-12-17 23:14:28,796 - lava_callback - INFO - Received callback for node 6940964a575bc060ce16156c INFO: 172.21.0.1:35760 - "POST /node/6940964a575bc060ce16156c HTTP/1.0" 202 Accepted 2025-12-17 23:14:28,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a6575bc060ce160dfa 2025-12-17 23:14:28,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a6575bc060ce160dfa HTTP/1.1" 502 150 2025-12-17 23:14:28,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a6575bc060ce160dfa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:29,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ae575bc060ce160e17 2025-12-17 23:14:29,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ae575bc060ce160e17 HTTP/1.1" 502 150 2025-12-17 23:14:29,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ae575bc060ce160e17'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:29,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b7575bc060ce160e29 2025-12-17 23:14:29,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b7575bc060ce160e29 HTTP/1.1" 502 150 2025-12-17 23:14:29,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b7575bc060ce160e29'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:29,448 - lava_callback - INFO - Received callback for node 6940964e575bc060ce161572 INFO: 172.21.0.1:35772 - "POST /node/6940964e575bc060ce161572 HTTP/1.0" 202 Accepted 2025-12-17 23:14:30,092 - lava_callback - INFO - Received callback for node 69409650575bc060ce161575 INFO: 172.21.0.1:38720 - "POST /node/69409650575bc060ce161575 HTTP/1.0" 202 Accepted 2025-12-17 23:14:31,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ad575bc060ce160e14 2025-12-17 23:14:31,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ad575bc060ce160e14 HTTP/1.1" 502 150 2025-12-17 23:14:31,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ad575bc060ce160e14'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:32,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a8575bc060ce160dfd 2025-12-17 23:14:33,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ae575bc060ce160e17 2025-12-17 23:14:33,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ae575bc060ce160e17 HTTP/1.1" 502 150 2025-12-17 23:14:33,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ae575bc060ce160e17'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:33,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a9575bc060ce160e04 2025-12-17 23:14:33,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a9575bc060ce160e04 HTTP/1.1" 502 150 2025-12-17 23:14:33,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a9575bc060ce160e04'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:35,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ab575bc060ce160e0f 2025-12-17 23:14:35,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ab575bc060ce160e0f HTTP/1.1" 502 150 2025-12-17 23:14:35,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ab575bc060ce160e0f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:36,333 - lava_callback - INFO - Received callback for node 6940964c575bc060ce16156f INFO: 172.21.0.1:38730 - "POST /node/6940964c575bc060ce16156f HTTP/1.0" 202 Accepted INFO: 172.21.0.1:38738 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:14:37,047 - lava_callback - INFO - Received callback for node 69409658575bc060ce16158d INFO: 172.21.0.1:38750 - "POST /node/69409658575bc060ce16158d HTTP/1.0" 202 Accepted 2025-12-17 23:14:37,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b7575bc060ce160e29 2025-12-17 23:14:37,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b7575bc060ce160e29 HTTP/1.1" 502 150 2025-12-17 23:14:37,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b7575bc060ce160e29'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:37,723 - lava_callback - INFO - Received callback for node 6940965a575bc060ce161593 INFO: 172.21.0.1:38756 - "POST /node/6940965a575bc060ce161593 HTTP/1.0" 202 Accepted 2025-12-17 23:14:38,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409396575bc060ce160dd9 2025-12-17 23:14:38,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409396575bc060ce160dd9 HTTP/1.1" 502 150 2025-12-17 23:14:38,439 - lava_callback - ERROR - Error processing callback for node 69409396575bc060ce160dd9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409396575bc060ce160dd9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409396575bc060ce160dd9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409396575bc060ce160dd9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:38,442 - lava_callback - INFO - Starting async job submit for node 694093b0575bc060ce160e1a 2025-12-17 23:14:38,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:38,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b0575bc060ce160e1a HTTP/1.1" 502 150 2025-12-17 23:14:38,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b0575bc060ce160e1a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:38,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b0575bc060ce160e1a 2025-12-17 23:14:38,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b0575bc060ce160e1a HTTP/1.1" 502 150 2025-12-17 23:14:38,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b0575bc060ce160e1a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:39,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ad575bc060ce160e14 2025-12-17 23:14:39,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ad575bc060ce160e14 HTTP/1.1" 502 150 2025-12-17 23:14:39,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ad575bc060ce160e14'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:40,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940939c575bc060ce160de6 2025-12-17 23:14:40,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939c575bc060ce160de6 HTTP/1.1" 502 150 2025-12-17 23:14:40,007 - lava_callback - ERROR - Error processing callback for node 6940939c575bc060ce160de6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940939c575bc060ce160de6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940939c575bc060ce160de6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940939c575bc060ce160de6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:40,009 - lava_callback - INFO - Starting async job submit for node 694093b2575bc060ce160e1d 2025-12-17 23:14:40,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:40,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b2575bc060ce160e1d HTTP/1.1" 502 150 2025-12-17 23:14:40,041 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b2575bc060ce160e1d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:40,041 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b2575bc060ce160e1d 2025-12-17 23:14:40,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b2575bc060ce160e1d HTTP/1.1" 502 150 2025-12-17 23:14:40,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b2575bc060ce160e1d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:40,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b0575bc060ce160e1a 2025-12-17 23:14:40,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b0575bc060ce160e1a HTTP/1.1" 502 150 2025-12-17 23:14:40,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b0575bc060ce160e1a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:40,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a0575bc060ce160df1 2025-12-17 23:14:40,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940939e575bc060ce160dee 2025-12-17 23:14:40,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a0575bc060ce160df1 HTTP/1.1" 502 150 2025-12-17 23:14:40,936 - lava_callback - ERROR - Error processing callback for node 694093a0575bc060ce160df1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a0575bc060ce160df1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a0575bc060ce160df1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a0575bc060ce160df1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:40,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940939e575bc060ce160dee HTTP/1.1" 502 150 2025-12-17 23:14:40,938 - lava_callback - ERROR - Error processing callback for node 6940939e575bc060ce160dee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940939e575bc060ce160dee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940939e575bc060ce160dee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940939e575bc060ce160dee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:40,939 - lava_callback - INFO - Starting async job submit for node 694093b3575bc060ce160e20 2025-12-17 23:14:40,940 - lava_callback - INFO - Starting async job submit for node 694093b4575bc060ce160e23 2025-12-17 23:14:40,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:40,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:40,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b3575bc060ce160e20 HTTP/1.1" 502 150 2025-12-17 23:14:40,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b3575bc060ce160e20'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:40,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b3575bc060ce160e20 2025-12-17 23:14:40,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b3575bc060ce160e20 HTTP/1.1" 502 150 2025-12-17 23:14:40,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b3575bc060ce160e20'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:40,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b4575bc060ce160e23 HTTP/1.1" 502 150 2025-12-17 23:14:40,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b4575bc060ce160e23'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:40,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b4575bc060ce160e23 2025-12-17 23:14:40,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b4575bc060ce160e23 HTTP/1.1" 502 150 2025-12-17 23:14:40,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b4575bc060ce160e23'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:41,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ae575bc060ce160e17 2025-12-17 23:14:41,138 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409420575bc060ce160f6c 2025-12-17 23:14:41,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409420575bc060ce160f6c HTTP/1.1" 502 150 2025-12-17 23:14:41,141 - lava_callback - ERROR - Error processing callback for node 69409420575bc060ce160f6c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409420575bc060ce160f6c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409420575bc060ce160f6c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409420575bc060ce160f6c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:41,143 - lava_callback - INFO - Starting async job submit for node 694093b6575bc060ce160e26 2025-12-17 23:14:41,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:41,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b6575bc060ce160e26 HTTP/1.1" 502 150 2025-12-17 23:14:41,160 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b6575bc060ce160e26'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:41,160 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b6575bc060ce160e26 2025-12-17 23:14:41,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b6575bc060ce160e26 HTTP/1.1" 502 150 2025-12-17 23:14:41,161 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b6575bc060ce160e26'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:42,042 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b2575bc060ce160e1d 2025-12-17 23:14:42,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b2575bc060ce160e1d HTTP/1.1" 502 150 2025-12-17 23:14:42,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b2575bc060ce160e1d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:42,129 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694113d8460bc8e265117e35 2025-12-17 23:14:42,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694113d8460bc8e265117e35 HTTP/1.1" 502 150 2025-12-17 23:14:42,131 - lava_callback - ERROR - Error processing callback for node 694113d8460bc8e265117e35: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694113d8460bc8e265117e35 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694113d8460bc8e265117e35 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694113d8460bc8e265117e35 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:42,133 - lava_callback - INFO - Starting async job submit for node 694093b9575bc060ce160e2c 2025-12-17 23:14:42,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:42,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b9575bc060ce160e2c HTTP/1.1" 502 150 2025-12-17 23:14:42,145 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b9575bc060ce160e2c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:42,146 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b9575bc060ce160e2c 2025-12-17 23:14:42,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b9575bc060ce160e2c HTTP/1.1" 502 150 2025-12-17 23:14:42,147 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b9575bc060ce160e2c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:42,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a2575bc060ce160df4 2025-12-17 23:14:42,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a2575bc060ce160df4 HTTP/1.1" 502 150 2025-12-17 23:14:42,808 - lava_callback - ERROR - Error processing callback for node 694093a2575bc060ce160df4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a2575bc060ce160df4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a2575bc060ce160df4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a2575bc060ce160df4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:42,810 - lava_callback - INFO - Starting async job submit for node 694095e0575bc060ce161466 2025-12-17 23:14:42,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:42,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e0575bc060ce161466 HTTP/1.1" 502 150 2025-12-17 23:14:42,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e0575bc060ce161466'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:42,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e0575bc060ce161466 2025-12-17 23:14:42,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e0575bc060ce161466 HTTP/1.1" 502 150 2025-12-17 23:14:42,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e0575bc060ce161466'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:42,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b3575bc060ce160e20 2025-12-17 23:14:42,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b3575bc060ce160e20 HTTP/1.1" 502 150 2025-12-17 23:14:42,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b3575bc060ce160e20'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:42,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b4575bc060ce160e23 2025-12-17 23:14:42,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b4575bc060ce160e23 HTTP/1.1" 502 150 2025-12-17 23:14:42,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b4575bc060ce160e23'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:43,161 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b6575bc060ce160e26 2025-12-17 23:14:43,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b6575bc060ce160e26 HTTP/1.1" 502 150 2025-12-17 23:14:43,163 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b6575bc060ce160e26'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:43,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940938e575bc060ce160dc6 HTTP/1.1" 502 150 2025-12-17 23:14:43,760 - lava_callback - ERROR - Error processing callback for node 6940938e575bc060ce160dc6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940938e575bc060ce160dc6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940938e575bc060ce160dc6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940938e575bc060ce160dc6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:43,762 - lava_callback - INFO - Starting async job submit for node 694093ba575bc060ce160e2f 2025-12-17 23:14:43,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:43,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ba575bc060ce160e2f HTTP/1.1" 502 150 2025-12-17 23:14:43,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ba575bc060ce160e2f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:43,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ba575bc060ce160e2f 2025-12-17 23:14:43,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ba575bc060ce160e2f HTTP/1.1" 502 150 2025-12-17 23:14:43,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ba575bc060ce160e2f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:43,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a4575bc060ce160df7 2025-12-17 23:14:43,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a4575bc060ce160df7 HTTP/1.1" 502 150 2025-12-17 23:14:43,985 - lava_callback - ERROR - Error processing callback for node 694093a4575bc060ce160df7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a4575bc060ce160df7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a4575bc060ce160df7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a4575bc060ce160df7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:43,987 - lava_callback - INFO - Starting async job submit for node 694093bd575bc060ce160e34 2025-12-17 23:14:43,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:43,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093bd575bc060ce160e34 HTTP/1.1" 502 150 2025-12-17 23:14:43,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093bd575bc060ce160e34'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:43,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093bd575bc060ce160e34 2025-12-17 23:14:43,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093bd575bc060ce160e34 HTTP/1.1" 502 150 2025-12-17 23:14:43,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093bd575bc060ce160e34'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:44,147 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b9575bc060ce160e2c 2025-12-17 23:14:44,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b9575bc060ce160e2c HTTP/1.1" 502 150 2025-12-17 23:14:44,148 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b9575bc060ce160e2c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:44,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b0575bc060ce160e1a 2025-12-17 23:14:44,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b0575bc060ce160e1a HTTP/1.1" 502 150 2025-12-17 23:14:44,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b0575bc060ce160e1a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:44,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e0575bc060ce161466 2025-12-17 23:14:44,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e0575bc060ce161466 HTTP/1.1" 502 150 2025-12-17 23:14:44,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e0575bc060ce161466'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:44,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a6575bc060ce160dfa 2025-12-17 23:14:44,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a6575bc060ce160dfa HTTP/1.1" 502 150 2025-12-17 23:14:44,875 - lava_callback - ERROR - Error processing callback for node 694093a6575bc060ce160dfa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a6575bc060ce160dfa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a6575bc060ce160dfa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a6575bc060ce160dfa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:44,877 - lava_callback - INFO - Starting async job submit for node 694093bf575bc060ce160e45 2025-12-17 23:14:44,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:44,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093bf575bc060ce160e45 HTTP/1.1" 502 150 2025-12-17 23:14:44,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093bf575bc060ce160e45'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:44,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093bf575bc060ce160e45 2025-12-17 23:14:44,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093bf575bc060ce160e45 HTTP/1.1" 502 150 2025-12-17 23:14:44,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093bf575bc060ce160e45'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:45,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ba575bc060ce160e2f 2025-12-17 23:14:45,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ba575bc060ce160e2f HTTP/1.1" 502 150 2025-12-17 23:14:45,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ba575bc060ce160e2f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:45,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093bd575bc060ce160e34 2025-12-17 23:14:45,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093bd575bc060ce160e34 HTTP/1.1" 502 150 2025-12-17 23:14:45,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093bd575bc060ce160e34'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:46,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b2575bc060ce160e1d 2025-12-17 23:14:46,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b2575bc060ce160e1d HTTP/1.1" 502 150 2025-12-17 23:14:46,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b2575bc060ce160e1d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34066 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:14:46,855 - lava_callback - INFO - Received callback for node 69409652575bc060ce16157c INFO: 172.21.0.1:34078 - "POST /node/69409652575bc060ce16157c HTTP/1.0" 202 Accepted 2025-12-17 23:14:46,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093bf575bc060ce160e45 2025-12-17 23:14:46,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093bf575bc060ce160e45 HTTP/1.1" 502 150 2025-12-17 23:14:46,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093bf575bc060ce160e45'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:46,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b3575bc060ce160e20 2025-12-17 23:14:46,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b3575bc060ce160e20 HTTP/1.1" 502 150 2025-12-17 23:14:46,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b3575bc060ce160e20'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:46,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b4575bc060ce160e23 2025-12-17 23:14:46,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b4575bc060ce160e23 HTTP/1.1" 502 150 2025-12-17 23:14:46,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b4575bc060ce160e23'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:47,163 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b6575bc060ce160e26 2025-12-17 23:14:47,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b6575bc060ce160e26 HTTP/1.1" 502 150 2025-12-17 23:14:47,164 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b6575bc060ce160e26'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:47,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a8575bc060ce160dfd HTTP/1.1" 502 150 2025-12-17 23:14:47,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093a8575bc060ce160dfd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:47,519 - lava_callback - INFO - Received callback for node 69409654575bc060ce161585 INFO: 172.21.0.1:34086 - "POST /node/69409654575bc060ce161585 HTTP/1.0" 202 Accepted 2025-12-17 23:14:48,148 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b9575bc060ce160e2c 2025-12-17 23:14:48,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b9575bc060ce160e2c HTTP/1.1" 502 150 2025-12-17 23:14:48,154 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b9575bc060ce160e2c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:48,168 - lava_callback - INFO - Received callback for node 69409656575bc060ce16158a INFO: 172.21.0.1:34088 - "POST /node/69409656575bc060ce16158a HTTP/1.0" 202 Accepted 2025-12-17 23:14:48,832 - lava_callback - INFO - Received callback for node 6940965c575bc060ce161596 INFO: 172.21.0.1:34090 - "POST /node/6940965c575bc060ce161596 HTTP/1.0" 202 Accepted 2025-12-17 23:14:48,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e0575bc060ce161466 2025-12-17 23:14:48,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e0575bc060ce161466 HTTP/1.1" 502 150 2025-12-17 23:14:48,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e0575bc060ce161466'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:49,463 - lava_callback - INFO - Received callback for node 6940965f575bc060ce161599 INFO: 172.21.0.1:34096 - "POST /node/6940965f575bc060ce161599 HTTP/1.0" 202 Accepted 2025-12-17 23:14:49,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ba575bc060ce160e2f 2025-12-17 23:14:49,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ba575bc060ce160e2f HTTP/1.1" 502 150 2025-12-17 23:14:49,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ba575bc060ce160e2f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:49,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a9575bc060ce160e04 2025-12-17 23:14:49,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a9575bc060ce160e04 HTTP/1.1" 502 150 2025-12-17 23:14:49,928 - lava_callback - ERROR - Error processing callback for node 694093a9575bc060ce160e04: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a9575bc060ce160e04 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a9575bc060ce160e04 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a9575bc060ce160e04 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:49,930 - lava_callback - INFO - Starting async job submit for node 694093c2575bc060ce160e4c 2025-12-17 23:14:49,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:49,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c2575bc060ce160e4c HTTP/1.1" 502 150 2025-12-17 23:14:49,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c2575bc060ce160e4c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:49,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c2575bc060ce160e4c 2025-12-17 23:14:49,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c2575bc060ce160e4c HTTP/1.1" 502 150 2025-12-17 23:14:49,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c2575bc060ce160e4c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:49,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093bd575bc060ce160e34 2025-12-17 23:14:49,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093bd575bc060ce160e34 HTTP/1.1" 502 150 2025-12-17 23:14:49,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093bd575bc060ce160e34'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:50,112 - lava_callback - INFO - Received callback for node 69409661575bc060ce16159c INFO: 172.21.0.1:41208 - "POST /node/69409661575bc060ce16159c HTTP/1.0" 202 Accepted 2025-12-17 23:14:50,769 - lava_callback - INFO - Received callback for node 69409663575bc060ce16159f INFO: 172.21.0.1:41218 - "POST /node/69409663575bc060ce16159f HTTP/1.0" 202 Accepted 2025-12-17 23:14:50,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093bf575bc060ce160e45 2025-12-17 23:14:50,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093bf575bc060ce160e45 HTTP/1.1" 502 150 2025-12-17 23:14:50,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093bf575bc060ce160e45'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:51,434 - lava_callback - INFO - Received callback for node 69409665575bc060ce1615a4 INFO: 172.21.0.1:41228 - "POST /node/69409665575bc060ce1615a4 HTTP/1.0" 202 Accepted 2025-12-17 23:14:51,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ab575bc060ce160e0f 2025-12-17 23:14:51,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ab575bc060ce160e0f HTTP/1.1" 502 150 2025-12-17 23:14:51,760 - lava_callback - ERROR - Error processing callback for node 694093ab575bc060ce160e0f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ab575bc060ce160e0f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ab575bc060ce160e0f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ab575bc060ce160e0f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:51,762 - lava_callback - INFO - Starting async job submit for node 694093c3575bc060ce160e4f 2025-12-17 23:14:51,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:51,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c3575bc060ce160e4f HTTP/1.1" 502 150 2025-12-17 23:14:51,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c3575bc060ce160e4f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:51,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c3575bc060ce160e4f 2025-12-17 23:14:51,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c3575bc060ce160e4f HTTP/1.1" 502 150 2025-12-17 23:14:51,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c3575bc060ce160e4f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:51,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c2575bc060ce160e4c 2025-12-17 23:14:51,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c2575bc060ce160e4c HTTP/1.1" 502 150 2025-12-17 23:14:51,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c2575bc060ce160e4c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:52,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b0575bc060ce160e1a 2025-12-17 23:14:52,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b0575bc060ce160e1a HTTP/1.1" 502 150 2025-12-17 23:14:52,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b0575bc060ce160e1a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:53,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b7575bc060ce160e29 2025-12-17 23:14:53,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b7575bc060ce160e29 HTTP/1.1" 502 150 2025-12-17 23:14:53,451 - lava_callback - ERROR - Error processing callback for node 694093b7575bc060ce160e29: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b7575bc060ce160e29 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b7575bc060ce160e29 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b7575bc060ce160e29 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:53,453 - lava_callback - INFO - Starting async job submit for node 694093c6575bc060ce160e55 2025-12-17 23:14:53,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:53,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c6575bc060ce160e55 HTTP/1.1" 502 150 2025-12-17 23:14:53,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c6575bc060ce160e55'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:53,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c6575bc060ce160e55 2025-12-17 23:14:53,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c6575bc060ce160e55 HTTP/1.1" 502 150 2025-12-17 23:14:53,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c6575bc060ce160e55'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:53,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c3575bc060ce160e4f 2025-12-17 23:14:53,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c3575bc060ce160e4f HTTP/1.1" 502 150 2025-12-17 23:14:53,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c3575bc060ce160e4f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:54,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b2575bc060ce160e1d 2025-12-17 23:14:54,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b3575bc060ce160e20 2025-12-17 23:14:54,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b3575bc060ce160e20 HTTP/1.1" 502 150 2025-12-17 23:14:54,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b3575bc060ce160e20'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:54,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b4575bc060ce160e23 2025-12-17 23:14:54,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b4575bc060ce160e23 HTTP/1.1" 502 150 2025-12-17 23:14:54,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b4575bc060ce160e23'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:55,164 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b6575bc060ce160e26 2025-12-17 23:14:55,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b6575bc060ce160e26 HTTP/1.1" 502 150 2025-12-17 23:14:55,166 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b6575bc060ce160e26'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:55,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c6575bc060ce160e55 2025-12-17 23:14:55,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c6575bc060ce160e55 HTTP/1.1" 502 150 2025-12-17 23:14:55,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c6575bc060ce160e55'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:55,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ad575bc060ce160e14 2025-12-17 23:14:55,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ad575bc060ce160e14 HTTP/1.1" 502 150 2025-12-17 23:14:55,869 - lava_callback - ERROR - Error processing callback for node 694093ad575bc060ce160e14: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ad575bc060ce160e14 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ad575bc060ce160e14 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ad575bc060ce160e14 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:14:55,872 - lava_callback - INFO - Starting async job submit for node 694093c5575bc060ce160e52 2025-12-17 23:14:55,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:14:55,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c5575bc060ce160e52 HTTP/1.1" 502 150 2025-12-17 23:14:55,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c5575bc060ce160e52'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:55,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c5575bc060ce160e52 2025-12-17 23:14:55,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c5575bc060ce160e52 HTTP/1.1" 502 150 2025-12-17 23:14:55,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c5575bc060ce160e52'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:55,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c2575bc060ce160e4c 2025-12-17 23:14:55,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c2575bc060ce160e4c HTTP/1.1" 502 150 2025-12-17 23:14:55,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c2575bc060ce160e4c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:56,154 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b9575bc060ce160e2c 2025-12-17 23:14:56,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b9575bc060ce160e2c HTTP/1.1" 502 150 2025-12-17 23:14:56,156 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b9575bc060ce160e2c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:56,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ae575bc060ce160e17 HTTP/1.1" 502 150 2025-12-17 23:14:56,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ae575bc060ce160e17'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:56,421 - lava_callback - INFO - Received callback for node 6940966e575bc060ce1615c1 INFO: 172.21.0.1:41236 - "POST /node/6940966e575bc060ce1615c1 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:41252 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:14:56,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e0575bc060ce161466 2025-12-17 23:14:56,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e0575bc060ce161466 HTTP/1.1" 502 150 2025-12-17 23:14:56,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e0575bc060ce161466'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:57,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c3575bc060ce160e4f 2025-12-17 23:14:57,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c3575bc060ce160e4f HTTP/1.1" 502 150 2025-12-17 23:14:57,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c3575bc060ce160e4f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:57,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ba575bc060ce160e2f 2025-12-17 23:14:57,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ba575bc060ce160e2f HTTP/1.1" 502 150 2025-12-17 23:14:57,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ba575bc060ce160e2f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:57,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c5575bc060ce160e52 2025-12-17 23:14:57,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c5575bc060ce160e52 HTTP/1.1" 502 150 2025-12-17 23:14:57,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c5575bc060ce160e52'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:57,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093bd575bc060ce160e34 2025-12-17 23:14:57,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093bd575bc060ce160e34 HTTP/1.1" 502 150 2025-12-17 23:14:57,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093bd575bc060ce160e34'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:58,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093bf575bc060ce160e45 2025-12-17 23:14:58,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093bf575bc060ce160e45 HTTP/1.1" 502 150 2025-12-17 23:14:58,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093bf575bc060ce160e45'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:14:59,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c6575bc060ce160e55 2025-12-17 23:14:59,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c6575bc060ce160e55 HTTP/1.1" 502 150 2025-12-17 23:14:59,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c6575bc060ce160e55'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:01,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c5575bc060ce160e52 2025-12-17 23:15:01,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c5575bc060ce160e52 HTTP/1.1" 502 150 2025-12-17 23:15:01,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c5575bc060ce160e52'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37662 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:15:03,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093a8575bc060ce160dfd 2025-12-17 23:15:03,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093a8575bc060ce160dfd HTTP/1.1" 502 150 2025-12-17 23:15:03,346 - lava_callback - ERROR - Error processing callback for node 694093a8575bc060ce160dfd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a8575bc060ce160dfd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a8575bc060ce160dfd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093a8575bc060ce160dfd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:03,348 - lava_callback - INFO - Starting async job submit for node 694093c8575bc060ce160e5b 2025-12-17 23:15:03,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:03,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c8575bc060ce160e5b HTTP/1.1" 502 150 2025-12-17 23:15:03,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c8575bc060ce160e5b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:03,355 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c8575bc060ce160e5b 2025-12-17 23:15:03,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c8575bc060ce160e5b HTTP/1.1" 502 150 2025-12-17 23:15:03,356 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c8575bc060ce160e5b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:03,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c2575bc060ce160e4c 2025-12-17 23:15:03,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c2575bc060ce160e4c HTTP/1.1" 502 150 2025-12-17 23:15:03,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c2575bc060ce160e4c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:05,356 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c8575bc060ce160e5b 2025-12-17 23:15:05,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c8575bc060ce160e5b HTTP/1.1" 502 150 2025-12-17 23:15:05,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c8575bc060ce160e5b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:05,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c3575bc060ce160e4f 2025-12-17 23:15:05,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c3575bc060ce160e4f HTTP/1.1" 502 150 2025-12-17 23:15:05,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c3575bc060ce160e4f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37676 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:15:06,876 - lava_callback - INFO - Received callback for node 69409667575bc060ce1615ad INFO: 172.21.0.1:37682 - "POST /node/69409667575bc060ce1615ad HTTP/1.0" 202 Accepted 2025-12-17 23:15:07,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c6575bc060ce160e55 2025-12-17 23:15:07,530 - lava_callback - INFO - Received callback for node 69409669575bc060ce1615b6 INFO: 172.21.0.1:37688 - "POST /node/69409669575bc060ce1615b6 HTTP/1.0" 202 Accepted 2025-12-17 23:15:08,187 - lava_callback - INFO - Received callback for node 6940966c575bc060ce1615be INFO: 172.21.0.1:37694 - "POST /node/6940966c575bc060ce1615be HTTP/1.0" 202 Accepted 2025-12-17 23:15:08,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b0575bc060ce160e1a 2025-12-17 23:15:08,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b0575bc060ce160e1a HTTP/1.1" 502 150 2025-12-17 23:15:08,461 - lava_callback - ERROR - Error processing callback for node 694093b0575bc060ce160e1a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b0575bc060ce160e1a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b0575bc060ce160e1a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b0575bc060ce160e1a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:08,463 - lava_callback - INFO - Starting async job submit for node 694093cb575bc060ce160e61 2025-12-17 23:15:08,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:08,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cb575bc060ce160e61 HTTP/1.1" 502 150 2025-12-17 23:15:08,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093cb575bc060ce160e61'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:08,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093cb575bc060ce160e61 2025-12-17 23:15:08,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cb575bc060ce160e61 HTTP/1.1" 502 150 2025-12-17 23:15:08,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093cb575bc060ce160e61'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:09,357 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c8575bc060ce160e5b 2025-12-17 23:15:09,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c8575bc060ce160e5b HTTP/1.1" 502 150 2025-12-17 23:15:09,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c8575bc060ce160e5b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:09,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b2575bc060ce160e1d HTTP/1.1" 502 150 2025-12-17 23:15:09,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093b2575bc060ce160e1d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:09,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c5575bc060ce160e52 2025-12-17 23:15:09,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c5575bc060ce160e52 HTTP/1.1" 502 150 2025-12-17 23:15:09,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c5575bc060ce160e52'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:10,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093cb575bc060ce160e61 2025-12-17 23:15:10,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cb575bc060ce160e61 HTTP/1.1" 502 150 2025-12-17 23:15:10,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093cb575bc060ce160e61'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:10,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b3575bc060ce160e20 2025-12-17 23:15:10,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b3575bc060ce160e20 HTTP/1.1" 502 150 2025-12-17 23:15:10,955 - lava_callback - ERROR - Error processing callback for node 694093b3575bc060ce160e20: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b3575bc060ce160e20 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b3575bc060ce160e20 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b3575bc060ce160e20 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:10,957 - lava_callback - INFO - Starting async job submit for node 694093cc575bc060ce160e64 2025-12-17 23:15:10,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:10,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cc575bc060ce160e64 HTTP/1.1" 502 150 2025-12-17 23:15:10,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093cc575bc060ce160e64'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:10,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093cc575bc060ce160e64 2025-12-17 23:15:10,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cc575bc060ce160e64 HTTP/1.1" 502 150 2025-12-17 23:15:10,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093cc575bc060ce160e64'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:10,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b4575bc060ce160e23 2025-12-17 23:15:10,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b4575bc060ce160e23 HTTP/1.1" 502 150 2025-12-17 23:15:10,966 - lava_callback - ERROR - Error processing callback for node 694093b4575bc060ce160e23: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b4575bc060ce160e23 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b4575bc060ce160e23 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b4575bc060ce160e23 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:10,968 - lava_callback - INFO - Starting async job submit for node 694093ce575bc060ce160e67 2025-12-17 23:15:10,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:10,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ce575bc060ce160e67 HTTP/1.1" 502 150 2025-12-17 23:15:10,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ce575bc060ce160e67'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:10,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ce575bc060ce160e67 2025-12-17 23:15:10,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ce575bc060ce160e67 HTTP/1.1" 502 150 2025-12-17 23:15:10,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ce575bc060ce160e67'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:11,166 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b6575bc060ce160e26 2025-12-17 23:15:11,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b6575bc060ce160e26 HTTP/1.1" 502 150 2025-12-17 23:15:11,168 - lava_callback - ERROR - Error processing callback for node 694093b6575bc060ce160e26: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b6575bc060ce160e26 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b6575bc060ce160e26 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b6575bc060ce160e26 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:11,169 - lava_callback - INFO - Starting async job submit for node 694093cf575bc060ce160e6a 2025-12-17 23:15:11,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:11,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cf575bc060ce160e6a HTTP/1.1" 502 150 2025-12-17 23:15:11,180 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093cf575bc060ce160e6a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:11,180 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093cf575bc060ce160e6a 2025-12-17 23:15:11,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cf575bc060ce160e6a HTTP/1.1" 502 150 2025-12-17 23:15:11,180 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093cf575bc060ce160e6a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:12,156 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b9575bc060ce160e2c 2025-12-17 23:15:12,304 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ae575bc060ce160e17 2025-12-17 23:15:12,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ae575bc060ce160e17 HTTP/1.1" 502 150 2025-12-17 23:15:12,305 - lava_callback - ERROR - Error processing callback for node 694093ae575bc060ce160e17: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ae575bc060ce160e17 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ae575bc060ce160e17 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ae575bc060ce160e17 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:12,307 - lava_callback - INFO - Starting async job submit for node 694093d0575bc060ce160e6d 2025-12-17 23:15:12,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:12,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d0575bc060ce160e6d HTTP/1.1" 502 150 2025-12-17 23:15:12,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d0575bc060ce160e6d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:12,313 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d0575bc060ce160e6d 2025-12-17 23:15:12,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d0575bc060ce160e6d HTTP/1.1" 502 150 2025-12-17 23:15:12,314 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d0575bc060ce160e6d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:12,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e0575bc060ce161466 2025-12-17 23:15:12,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e0575bc060ce161466 HTTP/1.1" 502 150 2025-12-17 23:15:12,856 - lava_callback - ERROR - Error processing callback for node 694095e0575bc060ce161466: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095e0575bc060ce161466 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095e0575bc060ce161466 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095e0575bc060ce161466 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:12,858 - lava_callback - INFO - Starting async job submit for node 694093c7575bc060ce160e58 2025-12-17 23:15:12,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:12,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c7575bc060ce160e58 HTTP/1.1" 502 150 2025-12-17 23:15:12,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c7575bc060ce160e58'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:12,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c7575bc060ce160e58 2025-12-17 23:15:12,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c7575bc060ce160e58 HTTP/1.1" 502 150 2025-12-17 23:15:12,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c7575bc060ce160e58'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:12,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093cc575bc060ce160e64 2025-12-17 23:15:12,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cc575bc060ce160e64 HTTP/1.1" 502 150 2025-12-17 23:15:12,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093cc575bc060ce160e64'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:12,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ce575bc060ce160e67 2025-12-17 23:15:12,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ce575bc060ce160e67 HTTP/1.1" 502 150 2025-12-17 23:15:12,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ce575bc060ce160e67'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:13,141 - lava_callback - INFO - Received callback for node 69409b50575bc060ce16219b INFO: 172.21.0.1:52684 - "POST /node/69409b50575bc060ce16219b HTTP/1.0" 202 Accepted 2025-12-17 23:15:13,181 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093cf575bc060ce160e6a 2025-12-17 23:15:13,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cf575bc060ce160e6a HTTP/1.1" 502 150 2025-12-17 23:15:13,182 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093cf575bc060ce160e6a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:13,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ba575bc060ce160e2f 2025-12-17 23:15:13,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ba575bc060ce160e2f HTTP/1.1" 502 150 2025-12-17 23:15:13,780 - lava_callback - ERROR - Error processing callback for node 694093ba575bc060ce160e2f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ba575bc060ce160e2f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ba575bc060ce160e2f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ba575bc060ce160e2f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:13,782 - lava_callback - INFO - Starting async job submit for node 694093ca575bc060ce160e5e 2025-12-17 23:15:13,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:13,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ca575bc060ce160e5e HTTP/1.1" 502 150 2025-12-17 23:15:13,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ca575bc060ce160e5e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:13,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ca575bc060ce160e5e 2025-12-17 23:15:13,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ca575bc060ce160e5e HTTP/1.1" 502 150 2025-12-17 23:15:13,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ca575bc060ce160e5e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:13,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093bd575bc060ce160e34 2025-12-17 23:15:14,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093bd575bc060ce160e34 HTTP/1.1" 502 150 2025-12-17 23:15:14,000 - lava_callback - ERROR - Error processing callback for node 694093bd575bc060ce160e34: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093bd575bc060ce160e34 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093bd575bc060ce160e34 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093bd575bc060ce160e34 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:14,003 - lava_callback - INFO - Starting async job submit for node 694093d3575bc060ce160e7a 2025-12-17 23:15:14,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:14,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d3575bc060ce160e7a HTTP/1.1" 502 150 2025-12-17 23:15:14,037 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d3575bc060ce160e7a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:14,037 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d3575bc060ce160e7a 2025-12-17 23:15:14,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d3575bc060ce160e7a HTTP/1.1" 502 150 2025-12-17 23:15:14,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d3575bc060ce160e7a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:14,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d0575bc060ce160e6d 2025-12-17 23:15:14,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d0575bc060ce160e6d HTTP/1.1" 502 150 2025-12-17 23:15:14,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d0575bc060ce160e6d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:14,346 - lava_callback - INFO - Received callback for node 69409c69575bc060ce1625af INFO: 172.21.0.1:52692 - "POST /node/69409c69575bc060ce1625af HTTP/1.0" 202 Accepted 2025-12-17 23:15:14,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093cb575bc060ce160e61 2025-12-17 23:15:14,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cb575bc060ce160e61 HTTP/1.1" 502 150 2025-12-17 23:15:14,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093cb575bc060ce160e61'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:14,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c7575bc060ce160e58 2025-12-17 23:15:14,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c7575bc060ce160e58 HTTP/1.1" 502 150 2025-12-17 23:15:14,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c7575bc060ce160e58'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:14,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093bf575bc060ce160e45 2025-12-17 23:15:14,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093bf575bc060ce160e45 HTTP/1.1" 502 150 2025-12-17 23:15:14,890 - lava_callback - ERROR - Error processing callback for node 694093bf575bc060ce160e45: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093bf575bc060ce160e45 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093bf575bc060ce160e45 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093bf575bc060ce160e45 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:14,892 - lava_callback - INFO - Starting async job submit for node 694093d5575bc060ce160e85 2025-12-17 23:15:14,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:14,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d5575bc060ce160e85 HTTP/1.1" 502 150 2025-12-17 23:15:14,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d5575bc060ce160e85'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:14,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d5575bc060ce160e85 2025-12-17 23:15:14,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d5575bc060ce160e85 HTTP/1.1" 502 150 2025-12-17 23:15:14,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d5575bc060ce160e85'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52702 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:15:15,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ca575bc060ce160e5e 2025-12-17 23:15:15,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ca575bc060ce160e5e HTTP/1.1" 502 150 2025-12-17 23:15:15,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ca575bc060ce160e5e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:16,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d3575bc060ce160e7a 2025-12-17 23:15:16,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d3575bc060ce160e7a HTTP/1.1" 502 150 2025-12-17 23:15:16,040 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d3575bc060ce160e7a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:16,390 - lava_callback - INFO - Received callback for node 6940967f575bc060ce1615e9 INFO: 172.21.0.1:52716 - "POST /node/6940967f575bc060ce1615e9 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:52728 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:15:16,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d5575bc060ce160e85 2025-12-17 23:15:16,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d5575bc060ce160e85 HTTP/1.1" 502 150 2025-12-17 23:15:16,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d5575bc060ce160e85'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:16,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093cc575bc060ce160e64 2025-12-17 23:15:16,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cc575bc060ce160e64 HTTP/1.1" 502 150 2025-12-17 23:15:16,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093cc575bc060ce160e64'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:16,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ce575bc060ce160e67 2025-12-17 23:15:16,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ce575bc060ce160e67 HTTP/1.1" 502 150 2025-12-17 23:15:16,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ce575bc060ce160e67'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:17,044 - lava_callback - INFO - Received callback for node 69409681575bc060ce1615ec INFO: 172.21.0.1:52744 - "POST /node/69409681575bc060ce1615ec HTTP/1.0" 202 Accepted 2025-12-17 23:15:17,182 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093cf575bc060ce160e6a 2025-12-17 23:15:17,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cf575bc060ce160e6a HTTP/1.1" 502 150 2025-12-17 23:15:17,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093cf575bc060ce160e6a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:17,359 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c8575bc060ce160e5b 2025-12-17 23:15:17,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c8575bc060ce160e5b HTTP/1.1" 502 150 2025-12-17 23:15:17,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c8575bc060ce160e5b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:17,697 - lava_callback - INFO - Received callback for node 69409671575bc060ce1615cb INFO: 172.21.0.1:52748 - "POST /node/69409671575bc060ce1615cb HTTP/1.0" 202 Accepted 2025-12-17 23:15:18,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d0575bc060ce160e6d 2025-12-17 23:15:18,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d0575bc060ce160e6d HTTP/1.1" 502 150 2025-12-17 23:15:18,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d0575bc060ce160e6d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:18,378 - lava_callback - INFO - Received callback for node 69409673575bc060ce1615ce INFO: 172.21.0.1:52762 - "POST /node/69409673575bc060ce1615ce HTTP/1.0" 202 Accepted 2025-12-17 23:15:18,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c7575bc060ce160e58 2025-12-17 23:15:18,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c7575bc060ce160e58 HTTP/1.1" 502 150 2025-12-17 23:15:18,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c7575bc060ce160e58'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:19,032 - lava_callback - INFO - Received callback for node 69409675575bc060ce1615d1 INFO: 172.21.0.1:52772 - "POST /node/69409675575bc060ce1615d1 HTTP/1.0" 202 Accepted 2025-12-17 23:15:19,699 - lava_callback - INFO - Received callback for node 69409677575bc060ce1615d4 INFO: 172.21.0.1:36726 - "POST /node/69409677575bc060ce1615d4 HTTP/1.0" 202 Accepted 2025-12-17 23:15:19,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ca575bc060ce160e5e 2025-12-17 23:15:19,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ca575bc060ce160e5e HTTP/1.1" 502 150 2025-12-17 23:15:19,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ca575bc060ce160e5e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:19,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c2575bc060ce160e4c 2025-12-17 23:15:19,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c2575bc060ce160e4c HTTP/1.1" 502 150 2025-12-17 23:15:19,947 - lava_callback - ERROR - Error processing callback for node 694093c2575bc060ce160e4c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c2575bc060ce160e4c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c2575bc060ce160e4c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c2575bc060ce160e4c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:19,949 - lava_callback - INFO - Starting async job submit for node 694093d7575bc060ce160e8c 2025-12-17 23:15:19,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:19,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d7575bc060ce160e8c HTTP/1.1" 502 150 2025-12-17 23:15:19,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d7575bc060ce160e8c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:19,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d7575bc060ce160e8c 2025-12-17 23:15:19,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d7575bc060ce160e8c HTTP/1.1" 502 150 2025-12-17 23:15:19,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d7575bc060ce160e8c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:20,040 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d3575bc060ce160e7a 2025-12-17 23:15:20,370 - lava_callback - INFO - Received callback for node 69409679575bc060ce1615d7 INFO: 172.21.0.1:36740 - "POST /node/69409679575bc060ce1615d7 HTTP/1.0" 202 Accepted 2025-12-17 23:15:20,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d5575bc060ce160e85 2025-12-17 23:15:20,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d5575bc060ce160e85 HTTP/1.1" 502 150 2025-12-17 23:15:20,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d5575bc060ce160e85'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:21,030 - lava_callback - INFO - Received callback for node 6940967b575bc060ce1615de INFO: 172.21.0.1:36752 - "POST /node/6940967b575bc060ce1615de HTTP/1.0" 202 Accepted 2025-12-17 23:15:21,714 - lava_callback - INFO - Received callback for node 6940967d575bc060ce1615e1 INFO: 172.21.0.1:36760 - "POST /node/6940967d575bc060ce1615e1 HTTP/1.0" 202 Accepted 2025-12-17 23:15:21,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c3575bc060ce160e4f 2025-12-17 23:15:21,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c3575bc060ce160e4f HTTP/1.1" 502 150 2025-12-17 23:15:21,775 - lava_callback - ERROR - Error processing callback for node 694093c3575bc060ce160e4f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c3575bc060ce160e4f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c3575bc060ce160e4f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c3575bc060ce160e4f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:21,777 - lava_callback - INFO - Starting async job submit for node 694093d9575bc060ce160e99 2025-12-17 23:15:21,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:21,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d9575bc060ce160e99 HTTP/1.1" 502 150 2025-12-17 23:15:21,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d9575bc060ce160e99'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:21,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d9575bc060ce160e99 2025-12-17 23:15:21,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d9575bc060ce160e99 HTTP/1.1" 502 150 2025-12-17 23:15:21,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d9575bc060ce160e99'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:21,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d7575bc060ce160e8c 2025-12-17 23:15:21,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d7575bc060ce160e8c HTTP/1.1" 502 150 2025-12-17 23:15:21,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d7575bc060ce160e8c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:22,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093cb575bc060ce160e61 2025-12-17 23:15:22,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cb575bc060ce160e61 HTTP/1.1" 502 150 2025-12-17 23:15:22,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093cb575bc060ce160e61'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:22,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c6575bc060ce160e55 HTTP/1.1" 502 150 2025-12-17 23:15:22,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c6575bc060ce160e55'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:23,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d9575bc060ce160e99 2025-12-17 23:15:24,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093cc575bc060ce160e64 2025-12-17 23:15:24,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cc575bc060ce160e64 HTTP/1.1" 502 150 2025-12-17 23:15:24,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093cc575bc060ce160e64'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:24,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ce575bc060ce160e67 2025-12-17 23:15:24,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ce575bc060ce160e67 HTTP/1.1" 502 150 2025-12-17 23:15:24,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ce575bc060ce160e67'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:25,183 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093cf575bc060ce160e6a 2025-12-17 23:15:25,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cf575bc060ce160e6a HTTP/1.1" 502 150 2025-12-17 23:15:25,184 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093cf575bc060ce160e6a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:25,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093b2575bc060ce160e1d 2025-12-17 23:15:25,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b2575bc060ce160e1d HTTP/1.1" 502 150 2025-12-17 23:15:25,361 - lava_callback - ERROR - Error processing callback for node 694093b2575bc060ce160e1d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b2575bc060ce160e1d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b2575bc060ce160e1d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b2575bc060ce160e1d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:25,363 - lava_callback - INFO - Starting async job submit for node 694093db575bc060ce160e9c 2025-12-17 23:15:25,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:25,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093db575bc060ce160e9c HTTP/1.1" 502 150 2025-12-17 23:15:25,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093db575bc060ce160e9c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:25,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093db575bc060ce160e9c 2025-12-17 23:15:25,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093db575bc060ce160e9c HTTP/1.1" 502 150 2025-12-17 23:15:25,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093db575bc060ce160e9c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:25,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c5575bc060ce160e52 2025-12-17 23:15:25,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c5575bc060ce160e52 HTTP/1.1" 502 150 2025-12-17 23:15:25,887 - lava_callback - ERROR - Error processing callback for node 694093c5575bc060ce160e52: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c5575bc060ce160e52 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c5575bc060ce160e52 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c5575bc060ce160e52 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:25,890 - lava_callback - INFO - Starting async job submit for node 694093dd575bc060ce160e9f 2025-12-17 23:15:25,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:25,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093dd575bc060ce160e9f HTTP/1.1" 502 150 2025-12-17 23:15:25,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093dd575bc060ce160e9f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:25,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093dd575bc060ce160e9f 2025-12-17 23:15:25,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093dd575bc060ce160e9f HTTP/1.1" 502 150 2025-12-17 23:15:25,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093dd575bc060ce160e9f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:25,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d7575bc060ce160e8c 2025-12-17 23:15:25,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d7575bc060ce160e8c HTTP/1.1" 502 150 2025-12-17 23:15:25,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d7575bc060ce160e8c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:26,317 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d0575bc060ce160e6d 2025-12-17 23:15:26,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d0575bc060ce160e6d HTTP/1.1" 502 150 2025-12-17 23:15:26,318 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d0575bc060ce160e6d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36762 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:15:26,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c7575bc060ce160e58 2025-12-17 23:15:26,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c7575bc060ce160e58 HTTP/1.1" 502 150 2025-12-17 23:15:26,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093c7575bc060ce160e58'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:26,880 - lava_callback - INFO - Received callback for node 69409688575bc060ce1615f8 INFO: 172.21.0.1:36770 - "POST /node/69409688575bc060ce1615f8 HTTP/1.0" 202 Accepted 2025-12-17 23:15:27,395 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093db575bc060ce160e9c 2025-12-17 23:15:27,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093db575bc060ce160e9c HTTP/1.1" 502 150 2025-12-17 23:15:27,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093db575bc060ce160e9c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:27,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093b9575bc060ce160e2c HTTP/1.1" 502 150 2025-12-17 23:15:27,536 - lava_callback - ERROR - Error processing callback for node 694093b9575bc060ce160e2c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b9575bc060ce160e2c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b9575bc060ce160e2c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093b9575bc060ce160e2c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:27,538 - lava_callback - INFO - Received callback for node 6940968b575bc060ce1615fe 2025-12-17 23:15:27,539 - lava_callback - INFO - Starting async job submit for node 694093df575bc060ce160ea2 2025-12-17 23:15:27,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:36782 - "POST /node/6940968b575bc060ce1615fe HTTP/1.0" 202 Accepted 2025-12-17 23:15:27,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093df575bc060ce160ea2 HTTP/1.1" 502 150 2025-12-17 23:15:27,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093df575bc060ce160ea2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:27,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093df575bc060ce160ea2 2025-12-17 23:15:27,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093df575bc060ce160ea2 HTTP/1.1" 502 150 2025-12-17 23:15:27,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093df575bc060ce160ea2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:27,632 - lava_callback - INFO - Received callback for node 6940968a575bc060ce1615fb INFO: 172.21.0.1:36788 - "POST /node/6940968a575bc060ce1615fb HTTP/1.0" 202 Accepted 2025-12-17 23:15:27,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ca575bc060ce160e5e 2025-12-17 23:15:27,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ca575bc060ce160e5e HTTP/1.1" 502 150 2025-12-17 23:15:27,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ca575bc060ce160e5e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:27,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093dd575bc060ce160e9f 2025-12-17 23:15:27,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093dd575bc060ce160e9f HTTP/1.1" 502 150 2025-12-17 23:15:27,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093dd575bc060ce160e9f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:28,208 - lava_callback - INFO - Received callback for node 6940968c575bc060ce161601 INFO: 172.21.0.1:36804 - "POST /node/6940968c575bc060ce161601 HTTP/1.0" 202 Accepted 2025-12-17 23:15:28,373 - lava_callback - INFO - Received callback for node 6940968e575bc060ce161606 INFO: 172.21.0.1:36820 - "POST /node/6940968e575bc060ce161606 HTTP/1.0" 202 Accepted 2025-12-17 23:15:28,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d5575bc060ce160e85 2025-12-17 23:15:28,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d5575bc060ce160e85 HTTP/1.1" 502 150 2025-12-17 23:15:28,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d5575bc060ce160e85'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:29,153 - lava_callback - INFO - Received callback for node 69409690575bc060ce16160d INFO: 172.21.0.1:36824 - "POST /node/69409690575bc060ce16160d HTTP/1.0" 202 Accepted 2025-12-17 23:15:29,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093df575bc060ce160ea2 2025-12-17 23:15:29,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093df575bc060ce160ea2 HTTP/1.1" 502 150 2025-12-17 23:15:29,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093df575bc060ce160ea2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:31,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093db575bc060ce160e9c 2025-12-17 23:15:31,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093db575bc060ce160e9c HTTP/1.1" 502 150 2025-12-17 23:15:31,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093db575bc060ce160e9c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:31,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093dd575bc060ce160e9f 2025-12-17 23:15:31,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093dd575bc060ce160e9f HTTP/1.1" 502 150 2025-12-17 23:15:31,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093dd575bc060ce160e9f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:33,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c8575bc060ce160e5b 2025-12-17 23:15:33,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c8575bc060ce160e5b HTTP/1.1" 502 150 2025-12-17 23:15:33,362 - lava_callback - ERROR - Error processing callback for node 694093c8575bc060ce160e5b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c8575bc060ce160e5b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c8575bc060ce160e5b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c8575bc060ce160e5b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:33,364 - lava_callback - INFO - Starting async job submit for node 694093e1575bc060ce160ea5 2025-12-17 23:15:33,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:33,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e1575bc060ce160ea5 HTTP/1.1" 502 150 2025-12-17 23:15:33,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093e1575bc060ce160ea5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:33,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093e1575bc060ce160ea5 2025-12-17 23:15:33,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e1575bc060ce160ea5 HTTP/1.1" 502 150 2025-12-17 23:15:33,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093e1575bc060ce160ea5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:33,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093df575bc060ce160ea2 2025-12-17 23:15:33,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093df575bc060ce160ea2 HTTP/1.1" 502 150 2025-12-17 23:15:33,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093df575bc060ce160ea2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:33,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d7575bc060ce160e8c 2025-12-17 23:15:33,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d7575bc060ce160e8c HTTP/1.1" 502 150 2025-12-17 23:15:33,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d7575bc060ce160e8c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:35,393 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093e1575bc060ce160ea5 2025-12-17 23:15:35,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e1575bc060ce160ea5 HTTP/1.1" 502 150 2025-12-17 23:15:35,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093e1575bc060ce160ea5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:35,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d3575bc060ce160e7a HTTP/1.1" 502 150 2025-12-17 23:15:35,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d3575bc060ce160e7a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:36,408 - lava_callback - INFO - Received callback for node 69409683575bc060ce1615ef INFO: 172.21.0.1:46556 - "POST /node/69409683575bc060ce1615ef HTTP/1.0" 202 Accepted INFO: 172.21.0.1:46566 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:15:37,076 - lava_callback - INFO - Received callback for node 69409685575bc060ce1615f2 INFO: 172.21.0.1:46568 - "POST /node/69409685575bc060ce1615f2 HTTP/1.0" 202 Accepted 2025-12-17 23:15:38,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093cb575bc060ce160e61 2025-12-17 23:15:38,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cb575bc060ce160e61 HTTP/1.1" 502 150 2025-12-17 23:15:38,486 - lava_callback - ERROR - Error processing callback for node 694093cb575bc060ce160e61: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093cb575bc060ce160e61 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093cb575bc060ce160e61 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093cb575bc060ce160e61 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:38,489 - lava_callback - INFO - Starting async job submit for node 69409835575bc060ce161a10 2025-12-17 23:15:38,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:38,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409835575bc060ce161a10 HTTP/1.1" 502 150 2025-12-17 23:15:38,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409835575bc060ce161a10'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:38,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409835575bc060ce161a10 2025-12-17 23:15:38,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409835575bc060ce161a10 HTTP/1.1" 502 150 2025-12-17 23:15:38,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409835575bc060ce161a10'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:38,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c6575bc060ce160e55 2025-12-17 23:15:38,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c6575bc060ce160e55 HTTP/1.1" 502 150 2025-12-17 23:15:38,932 - lava_callback - ERROR - Error processing callback for node 694093c6575bc060ce160e55: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c6575bc060ce160e55 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c6575bc060ce160e55 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c6575bc060ce160e55 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:38,935 - lava_callback - INFO - Starting async job submit for node 694093e3575bc060ce160ea8 2025-12-17 23:15:38,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:38,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e3575bc060ce160ea8 HTTP/1.1" 502 150 2025-12-17 23:15:38,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093e3575bc060ce160ea8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:38,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093e3575bc060ce160ea8 2025-12-17 23:15:38,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e3575bc060ce160ea8 HTTP/1.1" 502 150 2025-12-17 23:15:38,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093e3575bc060ce160ea8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:39,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d9575bc060ce160e99 HTTP/1.1" 502 150 2025-12-17 23:15:39,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d9575bc060ce160e99'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:39,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093e1575bc060ce160ea5 2025-12-17 23:15:39,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e1575bc060ce160ea5 HTTP/1.1" 502 150 2025-12-17 23:15:39,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093e1575bc060ce160ea5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:39,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093db575bc060ce160e9c 2025-12-17 23:15:39,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093db575bc060ce160e9c HTTP/1.1" 502 150 2025-12-17 23:15:39,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093db575bc060ce160e9c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:39,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093dd575bc060ce160e9f 2025-12-17 23:15:39,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093dd575bc060ce160e9f HTTP/1.1" 502 150 2025-12-17 23:15:39,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093dd575bc060ce160e9f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:40,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409835575bc060ce161a10 2025-12-17 23:15:40,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409835575bc060ce161a10 HTTP/1.1" 502 150 2025-12-17 23:15:40,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409835575bc060ce161a10'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:40,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093e3575bc060ce160ea8 2025-12-17 23:15:40,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e3575bc060ce160ea8 HTTP/1.1" 502 150 2025-12-17 23:15:40,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093e3575bc060ce160ea8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:40,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093cc575bc060ce160e64 2025-12-17 23:15:40,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cc575bc060ce160e64 HTTP/1.1" 502 150 2025-12-17 23:15:40,970 - lava_callback - ERROR - Error processing callback for node 694093cc575bc060ce160e64: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093cc575bc060ce160e64 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093cc575bc060ce160e64 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093cc575bc060ce160e64 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:40,972 - lava_callback - INFO - Starting async job submit for node 694093ed575bc060ce160eca 2025-12-17 23:15:40,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:40,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ce575bc060ce160e67 2025-12-17 23:15:40,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ed575bc060ce160eca HTTP/1.1" 502 150 2025-12-17 23:15:40,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ed575bc060ce160eca'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:40,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ed575bc060ce160eca 2025-12-17 23:15:40,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ce575bc060ce160e67 HTTP/1.1" 502 150 2025-12-17 23:15:40,980 - lava_callback - ERROR - Error processing callback for node 694093ce575bc060ce160e67: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ce575bc060ce160e67 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ce575bc060ce160e67 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ce575bc060ce160e67 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:40,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ed575bc060ce160eca HTTP/1.1" 502 150 2025-12-17 23:15:40,982 - lava_callback - INFO - Starting async job submit for node 6940919e575bc060ce1608d4 2025-12-17 23:15:40,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ed575bc060ce160eca'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:41,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:41,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919e575bc060ce1608d4 HTTP/1.1" 502 150 2025-12-17 23:15:41,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940919e575bc060ce1608d4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:41,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940919e575bc060ce1608d4 2025-12-17 23:15:41,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919e575bc060ce1608d4 HTTP/1.1" 502 150 2025-12-17 23:15:41,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940919e575bc060ce1608d4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:41,185 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093cf575bc060ce160e6a 2025-12-17 23:15:41,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093cf575bc060ce160e6a HTTP/1.1" 502 150 2025-12-17 23:15:41,187 - lava_callback - ERROR - Error processing callback for node 694093cf575bc060ce160e6a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093cf575bc060ce160e6a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093cf575bc060ce160e6a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093cf575bc060ce160e6a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:41,189 - lava_callback - INFO - Starting async job submit for node 694093e8575bc060ce160eb8 2025-12-17 23:15:41,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:41,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e8575bc060ce160eb8 HTTP/1.1" 502 150 2025-12-17 23:15:41,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093e8575bc060ce160eb8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:41,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093e8575bc060ce160eb8 2025-12-17 23:15:41,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e8575bc060ce160eb8 HTTP/1.1" 502 150 2025-12-17 23:15:41,196 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093e8575bc060ce160eb8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:41,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093df575bc060ce160ea2 2025-12-17 23:15:41,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093df575bc060ce160ea2 HTTP/1.1" 502 150 2025-12-17 23:15:41,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093df575bc060ce160ea2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:42,319 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d0575bc060ce160e6d 2025-12-17 23:15:42,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d0575bc060ce160e6d HTTP/1.1" 502 150 2025-12-17 23:15:42,320 - lava_callback - ERROR - Error processing callback for node 694093d0575bc060ce160e6d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093d0575bc060ce160e6d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093d0575bc060ce160e6d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093d0575bc060ce160e6d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:42,323 - lava_callback - INFO - Starting async job submit for node 694093eb575bc060ce160ec3 2025-12-17 23:15:42,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:42,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093eb575bc060ce160ec3 HTTP/1.1" 502 150 2025-12-17 23:15:42,334 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093eb575bc060ce160ec3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:42,334 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093eb575bc060ce160ec3 2025-12-17 23:15:42,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093eb575bc060ce160ec3 HTTP/1.1" 502 150 2025-12-17 23:15:42,335 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093eb575bc060ce160ec3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:42,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093c7575bc060ce160e58 2025-12-17 23:15:42,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093c7575bc060ce160e58 HTTP/1.1" 502 150 2025-12-17 23:15:42,873 - lava_callback - ERROR - Error processing callback for node 694093c7575bc060ce160e58: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c7575bc060ce160e58 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c7575bc060ce160e58 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093c7575bc060ce160e58 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:42,876 - lava_callback - INFO - Starting async job submit for node 694093ef575bc060ce160ecd 2025-12-17 23:15:42,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:42,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ef575bc060ce160ecd HTTP/1.1" 502 150 2025-12-17 23:15:42,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ef575bc060ce160ecd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:42,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ef575bc060ce160ecd 2025-12-17 23:15:42,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ef575bc060ce160ecd HTTP/1.1" 502 150 2025-12-17 23:15:42,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ef575bc060ce160ecd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:42,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ed575bc060ce160eca 2025-12-17 23:15:42,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ed575bc060ce160eca HTTP/1.1" 502 150 2025-12-17 23:15:42,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ed575bc060ce160eca'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:43,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940919e575bc060ce1608d4 2025-12-17 23:15:43,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919e575bc060ce1608d4 HTTP/1.1" 502 150 2025-12-17 23:15:43,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940919e575bc060ce1608d4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:43,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d9575bc060ce160e99 2025-12-17 23:15:43,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d9575bc060ce160e99 HTTP/1.1" 502 150 2025-12-17 23:15:43,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d9575bc060ce160e99'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:43,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093e8575bc060ce160eb8 2025-12-17 23:15:43,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e8575bc060ce160eb8 HTTP/1.1" 502 150 2025-12-17 23:15:43,198 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093e8575bc060ce160eb8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:43,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d3575bc060ce160e7a 2025-12-17 23:15:43,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d3575bc060ce160e7a HTTP/1.1" 502 150 2025-12-17 23:15:43,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d3575bc060ce160e7a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:43,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ca575bc060ce160e5e 2025-12-17 23:15:43,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ca575bc060ce160e5e HTTP/1.1" 502 150 2025-12-17 23:15:43,797 - lava_callback - ERROR - Error processing callback for node 694093ca575bc060ce160e5e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ca575bc060ce160e5e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ca575bc060ce160e5e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ca575bc060ce160e5e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:43,799 - lava_callback - INFO - Starting async job submit for node 694093f1575bc060ce160ed4 2025-12-17 23:15:43,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:43,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f1575bc060ce160ed4 HTTP/1.1" 502 150 2025-12-17 23:15:43,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f1575bc060ce160ed4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:43,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f1575bc060ce160ed4 2025-12-17 23:15:43,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f1575bc060ce160ed4 HTTP/1.1" 502 150 2025-12-17 23:15:43,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f1575bc060ce160ed4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:44,335 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093eb575bc060ce160ec3 2025-12-17 23:15:44,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093eb575bc060ce160ec3 HTTP/1.1" 502 150 2025-12-17 23:15:44,336 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093eb575bc060ce160ec3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:44,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409835575bc060ce161a10 2025-12-17 23:15:44,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409835575bc060ce161a10 HTTP/1.1" 502 150 2025-12-17 23:15:44,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409835575bc060ce161a10'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:44,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d5575bc060ce160e85 2025-12-17 23:15:44,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d5575bc060ce160e85 HTTP/1.1" 502 150 2025-12-17 23:15:44,904 - lava_callback - ERROR - Error processing callback for node 694093d5575bc060ce160e85: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093d5575bc060ce160e85 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093d5575bc060ce160e85 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093d5575bc060ce160e85 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:44,906 - lava_callback - INFO - Starting async job submit for node 694093f2575bc060ce160ed7 2025-12-17 23:15:44,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:44,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ef575bc060ce160ecd 2025-12-17 23:15:44,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ef575bc060ce160ecd HTTP/1.1" 502 150 2025-12-17 23:15:44,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ef575bc060ce160ecd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:44,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f2575bc060ce160ed7 HTTP/1.1" 502 150 2025-12-17 23:15:44,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f2575bc060ce160ed7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:44,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f2575bc060ce160ed7 2025-12-17 23:15:44,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f2575bc060ce160ed7 HTTP/1.1" 502 150 2025-12-17 23:15:44,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f2575bc060ce160ed7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:44,935 - lava_callback - INFO - Received callback for node 6940955f575bc060ce1612eb INFO: 172.21.0.1:46278 - "POST /node/6940955f575bc060ce1612eb HTTP/1.0" 202 Accepted 2025-12-17 23:15:44,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093e3575bc060ce160ea8 2025-12-17 23:15:44,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e3575bc060ce160ea8 HTTP/1.1" 502 150 2025-12-17 23:15:44,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093e3575bc060ce160ea8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:45,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f1575bc060ce160ed4 2025-12-17 23:15:45,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f1575bc060ce160ed4 HTTP/1.1" 502 150 2025-12-17 23:15:45,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f1575bc060ce160ed4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46292 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:15:46,894 - lava_callback - INFO - Received callback for node 69409697575bc060ce161624 INFO: 172.21.0.1:46300 - "POST /node/69409697575bc060ce161624 HTTP/1.0" 202 Accepted 2025-12-17 23:15:46,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f2575bc060ce160ed7 2025-12-17 23:15:46,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f2575bc060ce160ed7 HTTP/1.1" 502 150 2025-12-17 23:15:46,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f2575bc060ce160ed7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:46,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ed575bc060ce160eca 2025-12-17 23:15:46,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ed575bc060ce160eca HTTP/1.1" 502 150 2025-12-17 23:15:46,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ed575bc060ce160eca'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:47,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940919e575bc060ce1608d4 2025-12-17 23:15:47,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919e575bc060ce1608d4 HTTP/1.1" 502 150 2025-12-17 23:15:47,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940919e575bc060ce1608d4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:47,198 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093e8575bc060ce160eb8 2025-12-17 23:15:47,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e8575bc060ce160eb8 HTTP/1.1" 502 150 2025-12-17 23:15:47,199 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093e8575bc060ce160eb8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:47,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093e1575bc060ce160ea5 2025-12-17 23:15:47,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e1575bc060ce160ea5 HTTP/1.1" 502 150 2025-12-17 23:15:47,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093e1575bc060ce160ea5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:47,531 - lava_callback - INFO - Received callback for node 69409699575bc060ce161627 INFO: 172.21.0.1:46302 - "POST /node/69409699575bc060ce161627 HTTP/1.0" 202 Accepted 2025-12-17 23:15:48,091 - lava_callback - INFO - Received callback for node 69409691575bc060ce161612 INFO: 172.21.0.1:46304 - "POST /node/69409691575bc060ce161612 HTTP/1.0" 202 Accepted 2025-12-17 23:15:48,194 - lava_callback - INFO - Received callback for node 694096a0575bc060ce161633 INFO: 172.21.0.1:46310 - "POST /node/694096a0575bc060ce161633 HTTP/1.0" 202 Accepted 2025-12-17 23:15:48,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093eb575bc060ce160ec3 2025-12-17 23:15:48,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093eb575bc060ce160ec3 HTTP/1.1" 502 150 2025-12-17 23:15:48,338 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093eb575bc060ce160ec3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:48,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ef575bc060ce160ecd 2025-12-17 23:15:48,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ef575bc060ce160ecd HTTP/1.1" 502 150 2025-12-17 23:15:48,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ef575bc060ce160ecd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:49,001 - lava_callback - INFO - Received callback for node 69409693575bc060ce161618 INFO: 172.21.0.1:46320 - "POST /node/69409693575bc060ce161618 HTTP/1.0" 202 Accepted 2025-12-17 23:15:49,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f1575bc060ce160ed4 2025-12-17 23:15:49,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f1575bc060ce160ed4 HTTP/1.1" 502 150 2025-12-17 23:15:49,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f1575bc060ce160ed4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:49,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d7575bc060ce160e8c 2025-12-17 23:15:49,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d7575bc060ce160e8c HTTP/1.1" 502 150 2025-12-17 23:15:49,990 - lava_callback - ERROR - Error processing callback for node 694093d7575bc060ce160e8c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093d7575bc060ce160e8c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093d7575bc060ce160e8c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093d7575bc060ce160e8c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:49,992 - lava_callback - INFO - Starting async job submit for node 694093f3575bc060ce160eda 2025-12-17 23:15:49,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:49,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f3575bc060ce160eda HTTP/1.1" 502 150 2025-12-17 23:15:49,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f3575bc060ce160eda'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:49,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f3575bc060ce160eda 2025-12-17 23:15:50,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f3575bc060ce160eda HTTP/1.1" 502 150 2025-12-17 23:15:50,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f3575bc060ce160eda'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:50,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f2575bc060ce160ed7 2025-12-17 23:15:51,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d9575bc060ce160e99 2025-12-17 23:15:51,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d9575bc060ce160e99 HTTP/1.1" 502 150 2025-12-17 23:15:51,060 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093d9575bc060ce160e99'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:51,314 - lava_callback - INFO - Received callback for node 69409693575bc060ce16161b INFO: 172.21.0.1:35564 - "POST /node/69409693575bc060ce16161b HTTP/1.0" 202 Accepted 2025-12-17 23:15:51,994 - lava_callback - INFO - Received callback for node 69409694575bc060ce16161e INFO: 172.21.0.1:35580 - "POST /node/69409694575bc060ce16161e HTTP/1.0" 202 Accepted 2025-12-17 23:15:52,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f3575bc060ce160eda 2025-12-17 23:15:52,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f3575bc060ce160eda HTTP/1.1" 502 150 2025-12-17 23:15:52,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f3575bc060ce160eda'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:52,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409835575bc060ce161a10 2025-12-17 23:15:52,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409835575bc060ce161a10 HTTP/1.1" 502 150 2025-12-17 23:15:52,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409835575bc060ce161a10'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:52,669 - lava_callback - INFO - Received callback for node 69409695575bc060ce161621 INFO: 172.21.0.1:35582 - "POST /node/69409695575bc060ce161621 HTTP/1.0" 202 Accepted 2025-12-17 23:15:52,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093e3575bc060ce160ea8 2025-12-17 23:15:52,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e3575bc060ce160ea8 HTTP/1.1" 502 150 2025-12-17 23:15:52,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093e3575bc060ce160ea8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:54,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ed575bc060ce160eca 2025-12-17 23:15:54,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ed575bc060ce160eca HTTP/1.1" 502 150 2025-12-17 23:15:54,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ed575bc060ce160eca'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:55,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940919e575bc060ce1608d4 2025-12-17 23:15:55,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919e575bc060ce1608d4 HTTP/1.1" 502 150 2025-12-17 23:15:55,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940919e575bc060ce1608d4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:55,199 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093e8575bc060ce160eb8 2025-12-17 23:15:55,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e8575bc060ce160eb8 HTTP/1.1" 502 150 2025-12-17 23:15:55,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093e8575bc060ce160eb8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:55,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093db575bc060ce160e9c 2025-12-17 23:15:55,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093db575bc060ce160e9c HTTP/1.1" 502 150 2025-12-17 23:15:55,400 - lava_callback - ERROR - Error processing callback for node 694093db575bc060ce160e9c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093db575bc060ce160e9c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093db575bc060ce160e9c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093db575bc060ce160e9c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:55,403 - lava_callback - INFO - Starting async job submit for node 694093f4575bc060ce160edd 2025-12-17 23:15:55,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:55,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f4575bc060ce160edd HTTP/1.1" 502 150 2025-12-17 23:15:55,410 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f4575bc060ce160edd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:55,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f4575bc060ce160edd 2025-12-17 23:15:55,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f4575bc060ce160edd HTTP/1.1" 502 150 2025-12-17 23:15:55,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f4575bc060ce160edd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:55,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093dd575bc060ce160e9f 2025-12-17 23:15:55,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093dd575bc060ce160e9f HTTP/1.1" 502 150 2025-12-17 23:15:55,903 - lava_callback - ERROR - Error processing callback for node 694093dd575bc060ce160e9f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093dd575bc060ce160e9f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093dd575bc060ce160e9f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093dd575bc060ce160e9f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:55,905 - lava_callback - INFO - Starting async job submit for node 694093f6575bc060ce160ee0 2025-12-17 23:15:55,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:55,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f6575bc060ce160ee0 HTTP/1.1" 502 150 2025-12-17 23:15:55,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f6575bc060ce160ee0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:55,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f6575bc060ce160ee0 2025-12-17 23:15:55,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f6575bc060ce160ee0 HTTP/1.1" 502 150 2025-12-17 23:15:55,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f6575bc060ce160ee0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:56,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f3575bc060ce160eda 2025-12-17 23:15:56,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f3575bc060ce160eda HTTP/1.1" 502 150 2025-12-17 23:15:56,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f3575bc060ce160eda'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:56,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093eb575bc060ce160ec3 2025-12-17 23:15:56,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093eb575bc060ce160ec3 HTTP/1.1" 502 150 2025-12-17 23:15:56,339 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093eb575bc060ce160ec3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:56,421 - lava_callback - INFO - Received callback for node 6940969a575bc060ce16162a INFO: 172.21.0.1:35586 - "POST /node/6940969a575bc060ce16162a HTTP/1.0" 202 Accepted INFO: 172.21.0.1:35600 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:15:56,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ef575bc060ce160ecd 2025-12-17 23:15:56,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ef575bc060ce160ecd HTTP/1.1" 502 150 2025-12-17 23:15:56,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093ef575bc060ce160ecd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:57,093 - lava_callback - INFO - Received callback for node 6940969d575bc060ce16162d INFO: 172.21.0.1:35614 - "POST /node/6940969d575bc060ce16162d HTTP/1.0" 202 Accepted 2025-12-17 23:15:57,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f4575bc060ce160edd 2025-12-17 23:15:57,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f4575bc060ce160edd HTTP/1.1" 502 150 2025-12-17 23:15:57,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f4575bc060ce160edd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:57,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093df575bc060ce160ea2 2025-12-17 23:15:57,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093df575bc060ce160ea2 HTTP/1.1" 502 150 2025-12-17 23:15:57,591 - lava_callback - ERROR - Error processing callback for node 694093df575bc060ce160ea2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093df575bc060ce160ea2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093df575bc060ce160ea2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093df575bc060ce160ea2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:57,594 - lava_callback - INFO - Starting async job submit for node 694093f7575bc060ce160ee3 2025-12-17 23:15:57,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:57,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f7575bc060ce160ee3 HTTP/1.1" 502 150 2025-12-17 23:15:57,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f7575bc060ce160ee3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:57,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f7575bc060ce160ee3 2025-12-17 23:15:57,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f7575bc060ce160ee3 HTTP/1.1" 502 150 2025-12-17 23:15:57,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f7575bc060ce160ee3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:57,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f1575bc060ce160ed4 2025-12-17 23:15:57,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f1575bc060ce160ed4 HTTP/1.1" 502 150 2025-12-17 23:15:57,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f1575bc060ce160ed4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:57,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f6575bc060ce160ee0 2025-12-17 23:15:57,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f6575bc060ce160ee0 HTTP/1.1" 502 150 2025-12-17 23:15:57,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f6575bc060ce160ee0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:59,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d3575bc060ce160e7a 2025-12-17 23:15:59,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d3575bc060ce160e7a HTTP/1.1" 502 150 2025-12-17 23:15:59,475 - lava_callback - ERROR - Error processing callback for node 694093d3575bc060ce160e7a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093d3575bc060ce160e7a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093d3575bc060ce160e7a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093d3575bc060ce160e7a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:15:59,477 - lava_callback - INFO - Starting async job submit for node 694093f8575bc060ce160ee6 2025-12-17 23:15:59,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:15:59,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f8575bc060ce160ee6 HTTP/1.1" 502 150 2025-12-17 23:15:59,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f8575bc060ce160ee6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:59,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f8575bc060ce160ee6 2025-12-17 23:15:59,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f8575bc060ce160ee6 HTTP/1.1" 502 150 2025-12-17 23:15:59,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f8575bc060ce160ee6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:15:59,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f7575bc060ce160ee3 2025-12-17 23:15:59,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f7575bc060ce160ee3 HTTP/1.1" 502 150 2025-12-17 23:15:59,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f7575bc060ce160ee3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:00,441 - lava_callback - INFO - Received callback for node 69409c6c575bc060ce1625bf INFO: 172.21.0.1:56568 - "POST /node/69409c6c575bc060ce1625bf HTTP/1.0" 202 Accepted 2025-12-17 23:16:01,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f4575bc060ce160edd 2025-12-17 23:16:01,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f4575bc060ce160edd HTTP/1.1" 502 150 2025-12-17 23:16:01,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f4575bc060ce160edd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:01,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f8575bc060ce160ee6 2025-12-17 23:16:01,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f8575bc060ce160ee6 HTTP/1.1" 502 150 2025-12-17 23:16:01,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f8575bc060ce160ee6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:01,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f6575bc060ce160ee0 2025-12-17 23:16:01,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f6575bc060ce160ee0 HTTP/1.1" 502 150 2025-12-17 23:16:01,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f6575bc060ce160ee0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56578 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:16:03,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093e1575bc060ce160ea5 2025-12-17 23:16:03,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e1575bc060ce160ea5 HTTP/1.1" 502 150 2025-12-17 23:16:03,399 - lava_callback - ERROR - Error processing callback for node 694093e1575bc060ce160ea5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093e1575bc060ce160ea5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093e1575bc060ce160ea5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093e1575bc060ce160ea5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:03,401 - lava_callback - INFO - Starting async job submit for node 694093fa575bc060ce160eef 2025-12-17 23:16:03,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:03,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fa575bc060ce160eef HTTP/1.1" 502 150 2025-12-17 23:16:03,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093fa575bc060ce160eef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:03,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093fa575bc060ce160eef 2025-12-17 23:16:03,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fa575bc060ce160eef HTTP/1.1" 502 150 2025-12-17 23:16:03,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093fa575bc060ce160eef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:03,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f7575bc060ce160ee3 2025-12-17 23:16:03,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f7575bc060ce160ee3 HTTP/1.1" 502 150 2025-12-17 23:16:03,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f7575bc060ce160ee3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:04,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f3575bc060ce160eda 2025-12-17 23:16:05,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093fa575bc060ce160eef 2025-12-17 23:16:05,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fa575bc060ce160eef HTTP/1.1" 502 150 2025-12-17 23:16:05,410 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093fa575bc060ce160eef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:05,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f8575bc060ce160ee6 2025-12-17 23:16:05,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f8575bc060ce160ee6 HTTP/1.1" 502 150 2025-12-17 23:16:05,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f8575bc060ce160ee6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:06,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f2575bc060ce160ed7 HTTP/1.1" 502 150 2025-12-17 23:16:06,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f2575bc060ce160ed7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56582 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:16:06,921 - lava_callback - INFO - Received callback for node 694096a3575bc060ce161640 INFO: 172.21.0.1:56594 - "POST /node/694096a3575bc060ce161640 HTTP/1.0" 202 Accepted 2025-12-17 23:16:07,060 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093d9575bc060ce160e99 2025-12-17 23:16:07,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093d9575bc060ce160e99 HTTP/1.1" 502 150 2025-12-17 23:16:07,061 - lava_callback - ERROR - Error processing callback for node 694093d9575bc060ce160e99: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093d9575bc060ce160e99 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093d9575bc060ce160e99 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093d9575bc060ce160e99 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:07,063 - lava_callback - INFO - Starting async job submit for node 694093fc575bc060ce160efa 2025-12-17 23:16:07,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:07,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fc575bc060ce160efa HTTP/1.1" 502 150 2025-12-17 23:16:07,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093fc575bc060ce160efa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:07,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093fc575bc060ce160efa 2025-12-17 23:16:07,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fc575bc060ce160efa HTTP/1.1" 502 150 2025-12-17 23:16:07,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093fc575bc060ce160efa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:07,587 - lava_callback - INFO - Received callback for node 694096a5575bc060ce161643 INFO: 172.21.0.1:56606 - "POST /node/694096a5575bc060ce161643 HTTP/1.0" 202 Accepted 2025-12-17 23:16:07,817 - lava_callback - INFO - Received callback for node 694096af575bc060ce16165a INFO: 172.21.0.1:56610 - "POST /node/694096af575bc060ce16165a HTTP/1.0" 202 Accepted 2025-12-17 23:16:08,253 - lava_callback - INFO - Received callback for node 694096a7575bc060ce161646 INFO: 172.21.0.1:56618 - "POST /node/694096a7575bc060ce161646 HTTP/1.0" 202 Accepted 2025-12-17 23:16:08,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409835575bc060ce161a10 2025-12-17 23:16:08,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409835575bc060ce161a10 HTTP/1.1" 502 150 2025-12-17 23:16:08,547 - lava_callback - ERROR - Error processing callback for node 69409835575bc060ce161a10: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409835575bc060ce161a10 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409835575bc060ce161a10 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409835575bc060ce161a10 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:08,549 - lava_callback - INFO - Starting async job submit for node 694093fe575bc060ce160eff 2025-12-17 23:16:08,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:08,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fe575bc060ce160eff HTTP/1.1" 502 150 2025-12-17 23:16:08,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093fe575bc060ce160eff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:08,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093fe575bc060ce160eff 2025-12-17 23:16:08,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fe575bc060ce160eff HTTP/1.1" 502 150 2025-12-17 23:16:08,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093fe575bc060ce160eff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:08,902 - lava_callback - INFO - Received callback for node 694096a9575bc060ce161649 INFO: 172.21.0.1:56626 - "POST /node/694096a9575bc060ce161649 HTTP/1.0" 202 Accepted 2025-12-17 23:16:08,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093e3575bc060ce160ea8 2025-12-17 23:16:08,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e3575bc060ce160ea8 HTTP/1.1" 502 150 2025-12-17 23:16:08,949 - lava_callback - ERROR - Error processing callback for node 694093e3575bc060ce160ea8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093e3575bc060ce160ea8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093e3575bc060ce160ea8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093e3575bc060ce160ea8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:08,951 - lava_callback - INFO - Starting async job submit for node 69409400575bc060ce160f06 2025-12-17 23:16:08,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:08,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409400575bc060ce160f06 HTTP/1.1" 502 150 2025-12-17 23:16:08,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409400575bc060ce160f06'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:08,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409400575bc060ce160f06 2025-12-17 23:16:08,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409400575bc060ce160f06 HTTP/1.1" 502 150 2025-12-17 23:16:08,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409400575bc060ce160f06'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:09,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093fc575bc060ce160efa 2025-12-17 23:16:09,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fc575bc060ce160efa HTTP/1.1" 502 150 2025-12-17 23:16:09,075 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093fc575bc060ce160efa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:09,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093fa575bc060ce160eef 2025-12-17 23:16:09,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fa575bc060ce160eef HTTP/1.1" 502 150 2025-12-17 23:16:09,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093fa575bc060ce160eef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:09,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f4575bc060ce160edd 2025-12-17 23:16:09,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f4575bc060ce160edd HTTP/1.1" 502 150 2025-12-17 23:16:09,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f4575bc060ce160edd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:09,553 - lava_callback - INFO - Received callback for node 694096ab575bc060ce16164c INFO: 172.21.0.1:56638 - "POST /node/694096ab575bc060ce16164c HTTP/1.0" 202 Accepted 2025-12-17 23:16:09,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f6575bc060ce160ee0 2025-12-17 23:16:09,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f6575bc060ce160ee0 HTTP/1.1" 502 150 2025-12-17 23:16:09,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f6575bc060ce160ee0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:10,216 - lava_callback - INFO - Received callback for node 694096ad575bc060ce161657 INFO: 172.21.0.1:39552 - "POST /node/694096ad575bc060ce161657 HTTP/1.0" 202 Accepted 2025-12-17 23:16:10,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093fe575bc060ce160eff 2025-12-17 23:16:10,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fe575bc060ce160eff HTTP/1.1" 502 150 2025-12-17 23:16:10,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093fe575bc060ce160eff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:10,879 - lava_callback - INFO - Received callback for node 694096b0575bc060ce16165d INFO: 172.21.0.1:39560 - "POST /node/694096b0575bc060ce16165d HTTP/1.0" 202 Accepted 2025-12-17 23:16:10,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409400575bc060ce160f06 2025-12-17 23:16:10,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409400575bc060ce160f06 HTTP/1.1" 502 150 2025-12-17 23:16:10,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409400575bc060ce160f06'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:10,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ed575bc060ce160eca 2025-12-17 23:16:10,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ed575bc060ce160eca HTTP/1.1" 502 150 2025-12-17 23:16:10,993 - lava_callback - ERROR - Error processing callback for node 694093ed575bc060ce160eca: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ed575bc060ce160eca (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ed575bc060ce160eca (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ed575bc060ce160eca (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:10,994 - lava_callback - INFO - Starting async job submit for node 69409402575bc060ce160f09 2025-12-17 23:16:10,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:11,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409402575bc060ce160f09 HTTP/1.1" 502 150 2025-12-17 23:16:11,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409402575bc060ce160f09'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:11,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409402575bc060ce160f09 2025-12-17 23:16:11,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409402575bc060ce160f09 HTTP/1.1" 502 150 2025-12-17 23:16:11,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409402575bc060ce160f09'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:11,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940919e575bc060ce1608d4 2025-12-17 23:16:11,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940919e575bc060ce1608d4 HTTP/1.1" 502 150 2025-12-17 23:16:11,019 - lava_callback - ERROR - Error processing callback for node 6940919e575bc060ce1608d4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940919e575bc060ce1608d4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940919e575bc060ce1608d4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940919e575bc060ce1608d4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:11,021 - lava_callback - INFO - Starting async job submit for node 69409404575bc060ce160f13 2025-12-17 23:16:11,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:11,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409404575bc060ce160f13 HTTP/1.1" 502 150 2025-12-17 23:16:11,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409404575bc060ce160f13'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:11,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409404575bc060ce160f13 2025-12-17 23:16:11,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409404575bc060ce160f13 HTTP/1.1" 502 150 2025-12-17 23:16:11,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409404575bc060ce160f13'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:11,201 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093e8575bc060ce160eb8 2025-12-17 23:16:11,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093e8575bc060ce160eb8 HTTP/1.1" 502 150 2025-12-17 23:16:11,202 - lava_callback - ERROR - Error processing callback for node 694093e8575bc060ce160eb8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093e8575bc060ce160eb8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093e8575bc060ce160eb8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093e8575bc060ce160eb8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:11,205 - lava_callback - INFO - Starting async job submit for node 6940993f575bc060ce161c85 2025-12-17 23:16:11,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:11,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993f575bc060ce161c85 HTTP/1.1" 502 150 2025-12-17 23:16:11,260 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940993f575bc060ce161c85'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:11,260 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940993f575bc060ce161c85 2025-12-17 23:16:11,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993f575bc060ce161c85 HTTP/1.1" 502 150 2025-12-17 23:16:11,261 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940993f575bc060ce161c85'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:11,536 - lava_callback - INFO - Received callback for node 694096b2575bc060ce161660 INFO: 172.21.0.1:39572 - "POST /node/694096b2575bc060ce161660 HTTP/1.0" 202 Accepted 2025-12-17 23:16:11,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f7575bc060ce160ee3 2025-12-17 23:16:11,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f7575bc060ce160ee3 HTTP/1.1" 502 150 2025-12-17 23:16:11,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f7575bc060ce160ee3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:12,184 - lava_callback - INFO - Received callback for node 694096b3575bc060ce161663 INFO: 172.21.0.1:39580 - "POST /node/694096b3575bc060ce161663 HTTP/1.0" 202 Accepted 2025-12-17 23:16:12,339 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093eb575bc060ce160ec3 2025-12-17 23:16:12,827 - lava_callback - INFO - Received callback for node 694096b5575bc060ce161667 INFO: 172.21.0.1:39596 - "POST /node/694096b5575bc060ce161667 HTTP/1.0" 202 Accepted 2025-12-17 23:16:12,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093ef575bc060ce160ecd 2025-12-17 23:16:12,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093ef575bc060ce160ecd HTTP/1.1" 502 150 2025-12-17 23:16:12,912 - lava_callback - ERROR - Error processing callback for node 694093ef575bc060ce160ecd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ef575bc060ce160ecd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ef575bc060ce160ecd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093ef575bc060ce160ecd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:12,914 - lava_callback - INFO - Starting async job submit for node 6940992c575bc060ce161c57 2025-12-17 23:16:12,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:12,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992c575bc060ce161c57 HTTP/1.1" 502 150 2025-12-17 23:16:12,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992c575bc060ce161c57'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:12,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992c575bc060ce161c57 2025-12-17 23:16:12,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992c575bc060ce161c57 HTTP/1.1" 502 150 2025-12-17 23:16:12,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992c575bc060ce161c57'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:13,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409402575bc060ce160f09 2025-12-17 23:16:13,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409402575bc060ce160f09 HTTP/1.1" 502 150 2025-12-17 23:16:13,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409402575bc060ce160f09'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:13,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409404575bc060ce160f13 2025-12-17 23:16:13,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409404575bc060ce160f13 HTTP/1.1" 502 150 2025-12-17 23:16:13,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409404575bc060ce160f13'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:13,075 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093fc575bc060ce160efa 2025-12-17 23:16:13,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fc575bc060ce160efa HTTP/1.1" 502 150 2025-12-17 23:16:13,076 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093fc575bc060ce160efa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:13,261 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940993f575bc060ce161c85 2025-12-17 23:16:13,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993f575bc060ce161c85 HTTP/1.1" 502 150 2025-12-17 23:16:13,262 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940993f575bc060ce161c85'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:13,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f8575bc060ce160ee6 2025-12-17 23:16:13,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f8575bc060ce160ee6 HTTP/1.1" 502 150 2025-12-17 23:16:13,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f8575bc060ce160ee6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:13,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f1575bc060ce160ed4 2025-12-17 23:16:13,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f1575bc060ce160ed4 HTTP/1.1" 502 150 2025-12-17 23:16:13,814 - lava_callback - ERROR - Error processing callback for node 694093f1575bc060ce160ed4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f1575bc060ce160ed4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f1575bc060ce160ed4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f1575bc060ce160ed4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:13,816 - lava_callback - INFO - Starting async job submit for node 69409405575bc060ce160f16 2025-12-17 23:16:13,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:13,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409405575bc060ce160f16 HTTP/1.1" 502 150 2025-12-17 23:16:13,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409405575bc060ce160f16'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:13,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409405575bc060ce160f16 2025-12-17 23:16:13,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409405575bc060ce160f16 HTTP/1.1" 502 150 2025-12-17 23:16:13,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409405575bc060ce160f16'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:14,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f2575bc060ce160ed7 2025-12-17 23:16:14,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f2575bc060ce160ed7 HTTP/1.1" 502 150 2025-12-17 23:16:14,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f2575bc060ce160ed7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:14,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093fe575bc060ce160eff 2025-12-17 23:16:14,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fe575bc060ce160eff HTTP/1.1" 502 150 2025-12-17 23:16:14,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093fe575bc060ce160eff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:14,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992c575bc060ce161c57 2025-12-17 23:16:14,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992c575bc060ce161c57 HTTP/1.1" 502 150 2025-12-17 23:16:14,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992c575bc060ce161c57'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:14,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409400575bc060ce160f06 2025-12-17 23:16:14,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409400575bc060ce160f06 HTTP/1.1" 502 150 2025-12-17 23:16:14,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409400575bc060ce160f06'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39598 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:16:15,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409405575bc060ce160f16 2025-12-17 23:16:15,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409405575bc060ce160f16 HTTP/1.1" 502 150 2025-12-17 23:16:15,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409405575bc060ce160f16'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:16,419 - lava_callback - INFO - Received callback for node 694096bb575bc060ce16167d INFO: 172.21.0.1:39606 - "POST /node/694096bb575bc060ce16167d HTTP/1.0" 202 Accepted INFO: 172.21.0.1:39616 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:16:17,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409402575bc060ce160f09 2025-12-17 23:16:17,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409404575bc060ce160f13 2025-12-17 23:16:17,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409404575bc060ce160f13 HTTP/1.1" 502 150 2025-12-17 23:16:17,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409404575bc060ce160f13'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:17,262 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940993f575bc060ce161c85 2025-12-17 23:16:17,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993f575bc060ce161c85 HTTP/1.1" 502 150 2025-12-17 23:16:17,263 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940993f575bc060ce161c85'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:17,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093fa575bc060ce160eef 2025-12-17 23:16:17,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fa575bc060ce160eef HTTP/1.1" 502 150 2025-12-17 23:16:17,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093fa575bc060ce160eef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:18,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992c575bc060ce161c57 2025-12-17 23:16:18,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992c575bc060ce161c57 HTTP/1.1" 502 150 2025-12-17 23:16:18,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992c575bc060ce161c57'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:19,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f3575bc060ce160eda HTTP/1.1" 502 150 2025-12-17 23:16:19,260 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093f3575bc060ce160eda'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:19,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409405575bc060ce160f16 2025-12-17 23:16:19,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409405575bc060ce160f16 HTTP/1.1" 502 150 2025-12-17 23:16:19,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409405575bc060ce160f16'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:21,076 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093fc575bc060ce160efa 2025-12-17 23:16:21,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fc575bc060ce160efa HTTP/1.1" 502 150 2025-12-17 23:16:21,077 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093fc575bc060ce160efa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:22,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093fe575bc060ce160eff 2025-12-17 23:16:22,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fe575bc060ce160eff HTTP/1.1" 502 150 2025-12-17 23:16:22,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694093fe575bc060ce160eff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:22,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409400575bc060ce160f06 2025-12-17 23:16:22,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409400575bc060ce160f06 HTTP/1.1" 502 150 2025-12-17 23:16:22,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409400575bc060ce160f06'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:25,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409404575bc060ce160f13 2025-12-17 23:16:25,263 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940993f575bc060ce161c85 2025-12-17 23:16:25,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993f575bc060ce161c85 HTTP/1.1" 502 150 2025-12-17 23:16:25,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940993f575bc060ce161c85'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:25,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f4575bc060ce160edd 2025-12-17 23:16:25,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f4575bc060ce160edd HTTP/1.1" 502 150 2025-12-17 23:16:25,416 - lava_callback - ERROR - Error processing callback for node 694093f4575bc060ce160edd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f4575bc060ce160edd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f4575bc060ce160edd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f4575bc060ce160edd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:25,418 - lava_callback - INFO - Starting async job submit for node 69409407575bc060ce160f19 2025-12-17 23:16:25,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:25,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409407575bc060ce160f19 HTTP/1.1" 502 150 2025-12-17 23:16:25,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409407575bc060ce160f19'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:25,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409407575bc060ce160f19 2025-12-17 23:16:25,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409407575bc060ce160f19 HTTP/1.1" 502 150 2025-12-17 23:16:25,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409407575bc060ce160f19'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:25,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f6575bc060ce160ee0 2025-12-17 23:16:25,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f6575bc060ce160ee0 HTTP/1.1" 502 150 2025-12-17 23:16:25,919 - lava_callback - ERROR - Error processing callback for node 694093f6575bc060ce160ee0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f6575bc060ce160ee0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f6575bc060ce160ee0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f6575bc060ce160ee0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:25,921 - lava_callback - INFO - Starting async job submit for node 69409409575bc060ce160f1c 2025-12-17 23:16:25,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:25,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409409575bc060ce160f1c HTTP/1.1" 502 150 2025-12-17 23:16:25,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409409575bc060ce160f1c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:25,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409409575bc060ce160f1c 2025-12-17 23:16:25,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409409575bc060ce160f1c HTTP/1.1" 502 150 2025-12-17 23:16:25,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409409575bc060ce160f1c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38106 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:16:26,948 - lava_callback - INFO - Received callback for node 694096b7575bc060ce161671 INFO: 172.21.0.1:38114 - "POST /node/694096b7575bc060ce161671 HTTP/1.0" 202 Accepted 2025-12-17 23:16:26,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992c575bc060ce161c57 2025-12-17 23:16:26,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992c575bc060ce161c57 HTTP/1.1" 502 150 2025-12-17 23:16:26,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992c575bc060ce161c57'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:27,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409407575bc060ce160f19 2025-12-17 23:16:27,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409407575bc060ce160f19 HTTP/1.1" 502 150 2025-12-17 23:16:27,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409407575bc060ce160f19'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:27,604 - lava_callback - INFO - Received callback for node 694096c0575bc060ce161687 INFO: 172.21.0.1:38130 - "POST /node/694096c0575bc060ce161687 HTTP/1.0" 202 Accepted 2025-12-17 23:16:27,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f7575bc060ce160ee3 2025-12-17 23:16:27,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f7575bc060ce160ee3 HTTP/1.1" 502 150 2025-12-17 23:16:27,611 - lava_callback - ERROR - Error processing callback for node 694093f7575bc060ce160ee3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f7575bc060ce160ee3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f7575bc060ce160ee3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f7575bc060ce160ee3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:27,614 - lava_callback - INFO - Starting async job submit for node 6940940c575bc060ce160f1f 2025-12-17 23:16:27,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:27,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940c575bc060ce160f1f HTTP/1.1" 502 150 2025-12-17 23:16:27,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940940c575bc060ce160f1f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:27,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940940c575bc060ce160f1f 2025-12-17 23:16:27,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940c575bc060ce160f1f HTTP/1.1" 502 150 2025-12-17 23:16:27,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940940c575bc060ce160f1f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:27,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093eb575bc060ce160ec3 HTTP/1.1" 502 150 2025-12-17 23:16:27,696 - lava_callback - ERROR - Error processing callback for node 694093eb575bc060ce160ec3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093eb575bc060ce160ec3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093eb575bc060ce160ec3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093eb575bc060ce160ec3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:27,698 - lava_callback - INFO - Starting async job submit for node 6940940d575bc060ce160f27 2025-12-17 23:16:27,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:27,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940d575bc060ce160f27 HTTP/1.1" 502 150 2025-12-17 23:16:27,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940940d575bc060ce160f27'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:27,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940940d575bc060ce160f27 2025-12-17 23:16:27,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940d575bc060ce160f27 HTTP/1.1" 502 150 2025-12-17 23:16:27,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940940d575bc060ce160f27'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:27,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409405575bc060ce160f16 2025-12-17 23:16:27,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409405575bc060ce160f16 HTTP/1.1" 502 150 2025-12-17 23:16:27,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409405575bc060ce160f16'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:27,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409409575bc060ce160f1c 2025-12-17 23:16:27,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409409575bc060ce160f1c HTTP/1.1" 502 150 2025-12-17 23:16:27,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409409575bc060ce160f1c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:28,260 - lava_callback - INFO - Received callback for node 694096c1575bc060ce16168a INFO: 172.21.0.1:38142 - "POST /node/694096c1575bc060ce16168a HTTP/1.0" 202 Accepted 2025-12-17 23:16:28,908 - lava_callback - INFO - Received callback for node 694096c2575bc060ce16168d INFO: 172.21.0.1:38156 - "POST /node/694096c2575bc060ce16168d HTTP/1.0" 202 Accepted 2025-12-17 23:16:29,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f8575bc060ce160ee6 2025-12-17 23:16:29,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f8575bc060ce160ee6 HTTP/1.1" 502 150 2025-12-17 23:16:29,506 - lava_callback - ERROR - Error processing callback for node 694093f8575bc060ce160ee6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f8575bc060ce160ee6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f8575bc060ce160ee6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f8575bc060ce160ee6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:29,508 - lava_callback - INFO - Starting async job submit for node 6940940f575bc060ce160f31 2025-12-17 23:16:29,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:29,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940f575bc060ce160f31 HTTP/1.1" 502 150 2025-12-17 23:16:29,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940940f575bc060ce160f31'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:29,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940940f575bc060ce160f31 2025-12-17 23:16:29,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940f575bc060ce160f31 HTTP/1.1" 502 150 2025-12-17 23:16:29,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940940f575bc060ce160f31'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:29,566 - lava_callback - INFO - Received callback for node 694096c4575bc060ce161690 INFO: 172.21.0.1:38162 - "POST /node/694096c4575bc060ce161690 HTTP/1.0" 202 Accepted 2025-12-17 23:16:29,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940940c575bc060ce160f1f 2025-12-17 23:16:29,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940c575bc060ce160f1f HTTP/1.1" 502 150 2025-12-17 23:16:29,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940940c575bc060ce160f1f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:29,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940940d575bc060ce160f27 2025-12-17 23:16:29,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940d575bc060ce160f27 HTTP/1.1" 502 150 2025-12-17 23:16:29,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940940d575bc060ce160f27'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:30,194 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f2575bc060ce160ed7 2025-12-17 23:16:30,221 - lava_callback - INFO - Received callback for node 694096c5575bc060ce161693 INFO: 172.21.0.1:53850 - "POST /node/694096c5575bc060ce161693 HTTP/1.0" 202 Accepted 2025-12-17 23:16:30,865 - lava_callback - INFO - Received callback for node 694096c6575bc060ce161696 INFO: 172.21.0.1:53856 - "POST /node/694096c6575bc060ce161696 HTTP/1.0" 202 Accepted 2025-12-17 23:16:31,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409407575bc060ce160f19 2025-12-17 23:16:31,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409407575bc060ce160f19 HTTP/1.1" 502 150 2025-12-17 23:16:31,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409407575bc060ce160f19'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:31,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940940f575bc060ce160f31 2025-12-17 23:16:31,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940f575bc060ce160f31 HTTP/1.1" 502 150 2025-12-17 23:16:31,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940940f575bc060ce160f31'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:31,519 - lava_callback - INFO - Received callback for node 694096c7575bc060ce161699 INFO: 172.21.0.1:53870 - "POST /node/694096c7575bc060ce161699 HTTP/1.0" 202 Accepted 2025-12-17 23:16:31,528 - lava_callback - INFO - Received callback for node 694096be575bc060ce161684 INFO: 172.21.0.1:53874 - "POST /node/694096be575bc060ce161684 HTTP/1.0" 202 Accepted 2025-12-17 23:16:31,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409409575bc060ce160f1c 2025-12-17 23:16:31,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409409575bc060ce160f1c HTTP/1.1" 502 150 2025-12-17 23:16:31,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409409575bc060ce160f1c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:32,162 - lava_callback - INFO - Received callback for node 694096c8575bc060ce16169c INFO: 172.21.0.1:53884 - "POST /node/694096c8575bc060ce16169c HTTP/1.0" 202 Accepted 2025-12-17 23:16:32,196 - lava_callback - INFO - Received callback for node 694096cd575bc060ce1616b4 INFO: 172.21.0.1:53886 - "POST /node/694096cd575bc060ce1616b4 HTTP/1.0" 202 Accepted 2025-12-17 23:16:32,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409402575bc060ce160f09 HTTP/1.1" 502 150 2025-12-17 23:16:32,308 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409402575bc060ce160f09'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:33,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093fa575bc060ce160eef 2025-12-17 23:16:33,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fa575bc060ce160eef HTTP/1.1" 502 150 2025-12-17 23:16:33,414 - lava_callback - ERROR - Error processing callback for node 694093fa575bc060ce160eef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093fa575bc060ce160eef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093fa575bc060ce160eef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093fa575bc060ce160eef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:33,416 - lava_callback - INFO - Starting async job submit for node 69409411575bc060ce160f39 2025-12-17 23:16:33,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:33,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409411575bc060ce160f39 HTTP/1.1" 502 150 2025-12-17 23:16:33,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409411575bc060ce160f39'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:33,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409411575bc060ce160f39 2025-12-17 23:16:33,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409411575bc060ce160f39 HTTP/1.1" 502 150 2025-12-17 23:16:33,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409411575bc060ce160f39'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:33,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940940c575bc060ce160f1f 2025-12-17 23:16:33,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940c575bc060ce160f1f HTTP/1.1" 502 150 2025-12-17 23:16:33,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940940c575bc060ce160f1f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:33,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940940d575bc060ce160f27 2025-12-17 23:16:33,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940d575bc060ce160f27 HTTP/1.1" 502 150 2025-12-17 23:16:33,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940940d575bc060ce160f27'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:35,260 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093f3575bc060ce160eda 2025-12-17 23:16:35,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f3575bc060ce160eda HTTP/1.1" 502 150 2025-12-17 23:16:35,261 - lava_callback - ERROR - Error processing callback for node 694093f3575bc060ce160eda: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f3575bc060ce160eda (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f3575bc060ce160eda (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f3575bc060ce160eda (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:35,263 - lava_callback - INFO - Starting async job submit for node 69409413575bc060ce160f3c 2025-12-17 23:16:35,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:35,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409413575bc060ce160f3c HTTP/1.1" 502 150 2025-12-17 23:16:35,270 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409413575bc060ce160f3c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:35,270 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409413575bc060ce160f3c 2025-12-17 23:16:35,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409413575bc060ce160f3c HTTP/1.1" 502 150 2025-12-17 23:16:35,271 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409413575bc060ce160f3c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:35,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409411575bc060ce160f39 2025-12-17 23:16:35,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409411575bc060ce160f39 HTTP/1.1" 502 150 2025-12-17 23:16:35,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409411575bc060ce160f39'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:35,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940940f575bc060ce160f31 2025-12-17 23:16:35,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940f575bc060ce160f31 HTTP/1.1" 502 150 2025-12-17 23:16:35,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940940f575bc060ce160f31'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:36,023 - lava_callback - INFO - Received callback for node 6940990f575bc060ce161c08 INFO: 172.21.0.1:53894 - "POST /node/6940990f575bc060ce161c08 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:53898 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:16:37,078 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093fc575bc060ce160efa 2025-12-17 23:16:37,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fc575bc060ce160efa HTTP/1.1" 502 150 2025-12-17 23:16:37,079 - lava_callback - ERROR - Error processing callback for node 694093fc575bc060ce160efa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093fc575bc060ce160efa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093fc575bc060ce160efa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093fc575bc060ce160efa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:37,081 - lava_callback - INFO - Starting async job submit for node 69409416575bc060ce160f58 2025-12-17 23:16:37,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:37,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409416575bc060ce160f58 HTTP/1.1" 502 150 2025-12-17 23:16:37,087 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409416575bc060ce160f58'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:37,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409416575bc060ce160f58 2025-12-17 23:16:37,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409416575bc060ce160f58 HTTP/1.1" 502 150 2025-12-17 23:16:37,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409416575bc060ce160f58'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:37,271 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409413575bc060ce160f3c 2025-12-17 23:16:37,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409413575bc060ce160f3c HTTP/1.1" 502 150 2025-12-17 23:16:37,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409413575bc060ce160f3c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:38,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694093fe575bc060ce160eff 2025-12-17 23:16:38,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093fe575bc060ce160eff HTTP/1.1" 502 150 2025-12-17 23:16:38,562 - lava_callback - ERROR - Error processing callback for node 694093fe575bc060ce160eff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093fe575bc060ce160eff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093fe575bc060ce160eff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093fe575bc060ce160eff (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:38,564 - lava_callback - INFO - Starting async job submit for node 6940955e575bc060ce1612e8 2025-12-17 23:16:38,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:38,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955e575bc060ce1612e8 HTTP/1.1" 502 150 2025-12-17 23:16:38,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955e575bc060ce1612e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:38,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955e575bc060ce1612e8 2025-12-17 23:16:38,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955e575bc060ce1612e8 HTTP/1.1" 502 150 2025-12-17 23:16:38,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955e575bc060ce1612e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:38,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409400575bc060ce160f06 2025-12-17 23:16:38,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409400575bc060ce160f06 HTTP/1.1" 502 150 2025-12-17 23:16:38,963 - lava_callback - ERROR - Error processing callback for node 69409400575bc060ce160f06: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409400575bc060ce160f06 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409400575bc060ce160f06 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409400575bc060ce160f06 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:38,965 - lava_callback - INFO - Starting async job submit for node 69409036575bc060ce16036c 2025-12-17 23:16:38,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:39,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409036575bc060ce16036c HTTP/1.1" 502 150 2025-12-17 23:16:39,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409036575bc060ce16036c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:39,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409036575bc060ce16036c 2025-12-17 23:16:39,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409036575bc060ce16036c HTTP/1.1" 502 150 2025-12-17 23:16:39,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409036575bc060ce16036c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:39,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409416575bc060ce160f58 2025-12-17 23:16:39,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409416575bc060ce160f58 HTTP/1.1" 502 150 2025-12-17 23:16:39,089 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409416575bc060ce160f58'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:39,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409411575bc060ce160f39 2025-12-17 23:16:39,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409411575bc060ce160f39 HTTP/1.1" 502 150 2025-12-17 23:16:39,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409411575bc060ce160f39'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:39,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409407575bc060ce160f19 2025-12-17 23:16:39,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409407575bc060ce160f19 HTTP/1.1" 502 150 2025-12-17 23:16:39,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409407575bc060ce160f19'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:39,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409409575bc060ce160f1c 2025-12-17 23:16:39,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409409575bc060ce160f1c HTTP/1.1" 502 150 2025-12-17 23:16:39,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409409575bc060ce160f1c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:40,308 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409402575bc060ce160f09 2025-12-17 23:16:40,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409402575bc060ce160f09 HTTP/1.1" 502 150 2025-12-17 23:16:40,310 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409402575bc060ce160f09'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:40,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409404575bc060ce160f13 HTTP/1.1" 502 150 2025-12-17 23:16:40,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409404575bc060ce160f13'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:40,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955e575bc060ce1612e8 2025-12-17 23:16:40,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955e575bc060ce1612e8 HTTP/1.1" 502 150 2025-12-17 23:16:40,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955e575bc060ce1612e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:41,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409036575bc060ce16036c 2025-12-17 23:16:41,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409036575bc060ce16036c HTTP/1.1" 502 150 2025-12-17 23:16:41,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409036575bc060ce16036c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:41,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940993f575bc060ce161c85 2025-12-17 23:16:41,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993f575bc060ce161c85 HTTP/1.1" 502 150 2025-12-17 23:16:41,266 - lava_callback - ERROR - Error processing callback for node 6940993f575bc060ce161c85: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940993f575bc060ce161c85 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940993f575bc060ce161c85 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940993f575bc060ce161c85 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:41,268 - lava_callback - INFO - Starting async job submit for node 6940941a575bc060ce160f63 2025-12-17 23:16:41,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:41,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409413575bc060ce160f3c 2025-12-17 23:16:41,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409413575bc060ce160f3c HTTP/1.1" 502 150 2025-12-17 23:16:41,277 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409413575bc060ce160f3c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:41,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941a575bc060ce160f63 HTTP/1.1" 502 150 2025-12-17 23:16:41,278 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940941a575bc060ce160f63'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:41,278 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940941a575bc060ce160f63 2025-12-17 23:16:41,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941a575bc060ce160f63 HTTP/1.1" 502 150 2025-12-17 23:16:41,279 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940941a575bc060ce160f63'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:41,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940940c575bc060ce160f1f 2025-12-17 23:16:41,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940c575bc060ce160f1f HTTP/1.1" 502 150 2025-12-17 23:16:41,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940940c575bc060ce160f1f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:41,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940940d575bc060ce160f27 2025-12-17 23:16:41,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940d575bc060ce160f27 HTTP/1.1" 502 150 2025-12-17 23:16:41,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940940d575bc060ce160f27'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:42,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992c575bc060ce161c57 2025-12-17 23:16:42,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992c575bc060ce161c57 HTTP/1.1" 502 150 2025-12-17 23:16:42,959 - lava_callback - ERROR - Error processing callback for node 6940992c575bc060ce161c57: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940992c575bc060ce161c57 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940992c575bc060ce161c57 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940992c575bc060ce161c57 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:42,961 - lava_callback - INFO - Starting async job submit for node 6940941c575bc060ce160f66 2025-12-17 23:16:42,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:42,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941c575bc060ce160f66 HTTP/1.1" 502 150 2025-12-17 23:16:42,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940941c575bc060ce160f66'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:42,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940941c575bc060ce160f66 2025-12-17 23:16:42,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941c575bc060ce160f66 HTTP/1.1" 502 150 2025-12-17 23:16:42,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940941c575bc060ce160f66'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:43,090 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409416575bc060ce160f58 2025-12-17 23:16:43,279 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940941a575bc060ce160f63 2025-12-17 23:16:43,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941a575bc060ce160f63 HTTP/1.1" 502 150 2025-12-17 23:16:43,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940941a575bc060ce160f63'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:43,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940940f575bc060ce160f31 2025-12-17 23:16:43,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940f575bc060ce160f31 HTTP/1.1" 502 150 2025-12-17 23:16:43,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940940f575bc060ce160f31'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:43,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409405575bc060ce160f16 2025-12-17 23:16:43,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409405575bc060ce160f16 HTTP/1.1" 502 150 2025-12-17 23:16:43,835 - lava_callback - ERROR - Error processing callback for node 69409405575bc060ce160f16: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409405575bc060ce160f16 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409405575bc060ce160f16 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409405575bc060ce160f16 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:43,838 - lava_callback - INFO - Starting async job submit for node 6940941e575bc060ce160f69 2025-12-17 23:16:43,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:43,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941e575bc060ce160f69 HTTP/1.1" 502 150 2025-12-17 23:16:43,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940941e575bc060ce160f69'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:43,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940941e575bc060ce160f69 2025-12-17 23:16:43,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941e575bc060ce160f69 HTTP/1.1" 502 150 2025-12-17 23:16:43,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940941e575bc060ce160f69'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:44,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955e575bc060ce1612e8 2025-12-17 23:16:44,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955e575bc060ce1612e8 HTTP/1.1" 502 150 2025-12-17 23:16:44,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955e575bc060ce1612e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:44,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940941c575bc060ce160f66 2025-12-17 23:16:45,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941c575bc060ce160f66 HTTP/1.1" 502 150 2025-12-17 23:16:45,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940941c575bc060ce160f66'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:45,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409036575bc060ce16036c 2025-12-17 23:16:45,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409036575bc060ce16036c HTTP/1.1" 502 150 2025-12-17 23:16:45,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409036575bc060ce16036c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:45,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694093f2575bc060ce160ed7 HTTP/1.1" 502 150 2025-12-17 23:16:45,616 - lava_callback - ERROR - Error processing callback for node 694093f2575bc060ce160ed7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f2575bc060ce160ed7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f2575bc060ce160ed7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694093f2575bc060ce160ed7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:45,618 - lava_callback - INFO - Starting async job submit for node 69409427575bc060ce160f85 2025-12-17 23:16:45,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:45,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409427575bc060ce160f85 HTTP/1.1" 502 150 2025-12-17 23:16:45,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409427575bc060ce160f85'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:45,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409427575bc060ce160f85 2025-12-17 23:16:45,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409427575bc060ce160f85 HTTP/1.1" 502 150 2025-12-17 23:16:45,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409427575bc060ce160f85'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:45,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940941e575bc060ce160f69 2025-12-17 23:16:45,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941e575bc060ce160f69 HTTP/1.1" 502 150 2025-12-17 23:16:45,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940941e575bc060ce160f69'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40914 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:16:46,945 - lava_callback - INFO - Received callback for node 694096ca575bc060ce1616a1 INFO: 172.21.0.1:40920 - "POST /node/694096ca575bc060ce1616a1 HTTP/1.0" 202 Accepted 2025-12-17 23:16:47,280 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940941a575bc060ce160f63 2025-12-17 23:16:47,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941a575bc060ce160f63 HTTP/1.1" 502 150 2025-12-17 23:16:47,282 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940941a575bc060ce160f63'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:47,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409411575bc060ce160f39 2025-12-17 23:16:47,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409411575bc060ce160f39 HTTP/1.1" 502 150 2025-12-17 23:16:47,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409411575bc060ce160f39'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:47,595 - lava_callback - INFO - Received callback for node 694096cb575bc060ce1616a8 INFO: 172.21.0.1:40926 - "POST /node/694096cb575bc060ce1616a8 HTTP/1.0" 202 Accepted 2025-12-17 23:16:47,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409427575bc060ce160f85 2025-12-17 23:16:47,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409427575bc060ce160f85 HTTP/1.1" 502 150 2025-12-17 23:16:47,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409427575bc060ce160f85'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:47,862 - lava_callback - INFO - Received callback for node 694096dc575bc060ce1616d5 INFO: 172.21.0.1:40928 - "POST /node/694096dc575bc060ce1616d5 HTTP/1.0" 202 Accepted 2025-12-17 23:16:48,237 - lava_callback - INFO - Received callback for node 694096cc575bc060ce1616af INFO: 172.21.0.1:40942 - "POST /node/694096cc575bc060ce1616af HTTP/1.0" 202 Accepted 2025-12-17 23:16:48,886 - lava_callback - INFO - Received callback for node 694096ce575bc060ce1616b7 INFO: 172.21.0.1:40958 - "POST /node/694096ce575bc060ce1616b7 HTTP/1.0" 202 Accepted 2025-12-17 23:16:49,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940941c575bc060ce160f66 2025-12-17 23:16:49,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941c575bc060ce160f66 HTTP/1.1" 502 150 2025-12-17 23:16:49,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940941c575bc060ce160f66'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:49,277 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409413575bc060ce160f3c 2025-12-17 23:16:49,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409413575bc060ce160f3c HTTP/1.1" 502 150 2025-12-17 23:16:49,278 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409413575bc060ce160f3c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:49,524 - lava_callback - INFO - Received callback for node 694096d0575bc060ce1616ba INFO: 172.21.0.1:40966 - "POST /node/694096d0575bc060ce1616ba HTTP/1.0" 202 Accepted 2025-12-17 23:16:49,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940941e575bc060ce160f69 2025-12-17 23:16:49,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941e575bc060ce160f69 HTTP/1.1" 502 150 2025-12-17 23:16:49,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940941e575bc060ce160f69'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:50,172 - lava_callback - INFO - Received callback for node 694096d1575bc060ce1616bd INFO: 172.21.0.1:53660 - "POST /node/694096d1575bc060ce1616bd HTTP/1.0" 202 Accepted 2025-12-17 23:16:50,828 - lava_callback - INFO - Received callback for node 694096d2575bc060ce1616c0 INFO: 172.21.0.1:53674 - "POST /node/694096d2575bc060ce1616c0 HTTP/1.0" 202 Accepted 2025-12-17 23:16:51,473 - lava_callback - INFO - Received callback for node 694096d3575bc060ce1616c3 INFO: 172.21.0.1:53680 - "POST /node/694096d3575bc060ce1616c3 HTTP/1.0" 202 Accepted 2025-12-17 23:16:51,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409427575bc060ce160f85 2025-12-17 23:16:51,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409427575bc060ce160f85 HTTP/1.1" 502 150 2025-12-17 23:16:51,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409427575bc060ce160f85'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:52,121 - lava_callback - INFO - Received callback for node 694096d5575bc060ce1616c6 INFO: 172.21.0.1:53686 - "POST /node/694096d5575bc060ce1616c6 HTTP/1.0" 202 Accepted 2025-12-17 23:16:52,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955e575bc060ce1612e8 2025-12-17 23:16:52,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955e575bc060ce1612e8 HTTP/1.1" 502 150 2025-12-17 23:16:52,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955e575bc060ce1612e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:52,764 - lava_callback - INFO - Received callback for node 694096d7575bc060ce1616c9 INFO: 172.21.0.1:53688 - "POST /node/694096d7575bc060ce1616c9 HTTP/1.0" 202 Accepted 2025-12-17 23:16:53,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409036575bc060ce16036c 2025-12-17 23:16:53,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409036575bc060ce16036c HTTP/1.1" 502 150 2025-12-17 23:16:53,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409036575bc060ce16036c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:53,416 - lava_callback - INFO - Received callback for node 694096d8575bc060ce1616cc INFO: 172.21.0.1:53696 - "POST /node/694096d8575bc060ce1616cc HTTP/1.0" 202 Accepted 2025-12-17 23:16:54,062 - lava_callback - INFO - Received callback for node 694096da575bc060ce1616cf INFO: 172.21.0.1:53700 - "POST /node/694096da575bc060ce1616cf HTTP/1.0" 202 Accepted 2025-12-17 23:16:54,712 - lava_callback - INFO - Received callback for node 694096db575bc060ce1616d2 INFO: 172.21.0.1:53712 - "POST /node/694096db575bc060ce1616d2 HTTP/1.0" 202 Accepted 2025-12-17 23:16:55,282 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940941a575bc060ce160f63 2025-12-17 23:16:55,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941a575bc060ce160f63 HTTP/1.1" 502 150 2025-12-17 23:16:55,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940941a575bc060ce160f63'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:55,381 - lava_callback - INFO - Received callback for node 694096de575bc060ce1616da INFO: 172.21.0.1:53726 - "POST /node/694096de575bc060ce1616da HTTP/1.0" 202 Accepted 2025-12-17 23:16:55,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409407575bc060ce160f19 2025-12-17 23:16:55,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409407575bc060ce160f19 HTTP/1.1" 502 150 2025-12-17 23:16:55,434 - lava_callback - ERROR - Error processing callback for node 69409407575bc060ce160f19: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409407575bc060ce160f19 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409407575bc060ce160f19 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409407575bc060ce160f19 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:55,436 - lava_callback - INFO - Starting async job submit for node 6940942a575bc060ce160f8a 2025-12-17 23:16:55,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:55,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942a575bc060ce160f8a HTTP/1.1" 502 150 2025-12-17 23:16:55,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940942a575bc060ce160f8a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:55,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940942a575bc060ce160f8a 2025-12-17 23:16:55,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942a575bc060ce160f8a HTTP/1.1" 502 150 2025-12-17 23:16:55,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940942a575bc060ce160f8a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:55,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409409575bc060ce160f1c 2025-12-17 23:16:55,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409409575bc060ce160f1c HTTP/1.1" 502 150 2025-12-17 23:16:55,939 - lava_callback - ERROR - Error processing callback for node 69409409575bc060ce160f1c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409409575bc060ce160f1c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409409575bc060ce160f1c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409409575bc060ce160f1c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:55,941 - lava_callback - INFO - Starting async job submit for node 69409422575bc060ce160f77 2025-12-17 23:16:55,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:55,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409422575bc060ce160f77 HTTP/1.1" 502 150 2025-12-17 23:16:55,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409422575bc060ce160f77'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:55,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409422575bc060ce160f77 2025-12-17 23:16:55,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409422575bc060ce160f77 HTTP/1.1" 502 150 2025-12-17 23:16:55,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409422575bc060ce160f77'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:56,310 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409402575bc060ce160f09 2025-12-17 23:16:56,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409404575bc060ce160f13 2025-12-17 23:16:56,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409404575bc060ce160f13 HTTP/1.1" 502 150 2025-12-17 23:16:56,501 - lava_callback - ERROR - Error processing callback for node 69409404575bc060ce160f13: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409404575bc060ce160f13 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409404575bc060ce160f13 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409404575bc060ce160f13 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:56,503 - lava_callback - INFO - Starting async job submit for node 69409424575bc060ce160f82 2025-12-17 23:16:56,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:53738 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:16:56,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409424575bc060ce160f82 HTTP/1.1" 502 150 2025-12-17 23:16:56,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409424575bc060ce160f82'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:56,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409424575bc060ce160f82 2025-12-17 23:16:56,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409424575bc060ce160f82 HTTP/1.1" 502 150 2025-12-17 23:16:56,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409424575bc060ce160f82'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:57,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940941c575bc060ce160f66 2025-12-17 23:16:57,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941c575bc060ce160f66 HTTP/1.1" 502 150 2025-12-17 23:16:57,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940941c575bc060ce160f66'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:57,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940942a575bc060ce160f8a 2025-12-17 23:16:57,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942a575bc060ce160f8a HTTP/1.1" 502 150 2025-12-17 23:16:57,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940942a575bc060ce160f8a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:57,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940940c575bc060ce160f1f 2025-12-17 23:16:57,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940c575bc060ce160f1f HTTP/1.1" 502 150 2025-12-17 23:16:57,646 - lava_callback - ERROR - Error processing callback for node 6940940c575bc060ce160f1f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940940c575bc060ce160f1f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940940c575bc060ce160f1f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940940c575bc060ce160f1f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:57,648 - lava_callback - INFO - Starting async job submit for node 6940942c575bc060ce160f8d 2025-12-17 23:16:57,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:57,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942c575bc060ce160f8d HTTP/1.1" 502 150 2025-12-17 23:16:57,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940942c575bc060ce160f8d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:57,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940942c575bc060ce160f8d 2025-12-17 23:16:57,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942c575bc060ce160f8d HTTP/1.1" 502 150 2025-12-17 23:16:57,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940942c575bc060ce160f8d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:57,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940940d575bc060ce160f27 2025-12-17 23:16:57,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940d575bc060ce160f27 HTTP/1.1" 502 150 2025-12-17 23:16:57,736 - lava_callback - ERROR - Error processing callback for node 6940940d575bc060ce160f27: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940940d575bc060ce160f27 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940940d575bc060ce160f27 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940940d575bc060ce160f27 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:57,738 - lava_callback - INFO - Starting async job submit for node 6940942e575bc060ce160f90 2025-12-17 23:16:57,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:57,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942e575bc060ce160f90 HTTP/1.1" 502 150 2025-12-17 23:16:57,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940942e575bc060ce160f90'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:57,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940942e575bc060ce160f90 2025-12-17 23:16:57,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942e575bc060ce160f90 HTTP/1.1" 502 150 2025-12-17 23:16:57,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940942e575bc060ce160f90'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:57,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940941e575bc060ce160f69 2025-12-17 23:16:57,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941e575bc060ce160f69 HTTP/1.1" 502 150 2025-12-17 23:16:57,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940941e575bc060ce160f69'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:57,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409422575bc060ce160f77 2025-12-17 23:16:57,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409422575bc060ce160f77 HTTP/1.1" 502 150 2025-12-17 23:16:57,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409422575bc060ce160f77'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:58,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409416575bc060ce160f58 HTTP/1.1" 502 150 2025-12-17 23:16:58,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409416575bc060ce160f58'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:58,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409424575bc060ce160f82 2025-12-17 23:16:58,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409424575bc060ce160f82 HTTP/1.1" 502 150 2025-12-17 23:16:58,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409424575bc060ce160f82'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:59,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940940f575bc060ce160f31 2025-12-17 23:16:59,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940940f575bc060ce160f31 HTTP/1.1" 502 150 2025-12-17 23:16:59,521 - lava_callback - ERROR - Error processing callback for node 6940940f575bc060ce160f31: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940940f575bc060ce160f31 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940940f575bc060ce160f31 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940940f575bc060ce160f31 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:16:59,523 - lava_callback - INFO - Starting async job submit for node 69409430575bc060ce160f93 2025-12-17 23:16:59,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:16:59,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409430575bc060ce160f93 HTTP/1.1" 502 150 2025-12-17 23:16:59,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409430575bc060ce160f93'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:59,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409430575bc060ce160f93 2025-12-17 23:16:59,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409430575bc060ce160f93 HTTP/1.1" 502 150 2025-12-17 23:16:59,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409430575bc060ce160f93'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:59,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940942c575bc060ce160f8d 2025-12-17 23:16:59,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942c575bc060ce160f8d HTTP/1.1" 502 150 2025-12-17 23:16:59,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940942c575bc060ce160f8d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:59,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409427575bc060ce160f85 2025-12-17 23:16:59,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409427575bc060ce160f85 HTTP/1.1" 502 150 2025-12-17 23:16:59,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409427575bc060ce160f85'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:16:59,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940942e575bc060ce160f90 2025-12-17 23:16:59,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942e575bc060ce160f90 HTTP/1.1" 502 150 2025-12-17 23:16:59,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940942e575bc060ce160f90'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:01,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940942a575bc060ce160f8a 2025-12-17 23:17:01,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942a575bc060ce160f8a HTTP/1.1" 502 150 2025-12-17 23:17:01,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940942a575bc060ce160f8a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:01,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409430575bc060ce160f93 2025-12-17 23:17:01,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409430575bc060ce160f93 HTTP/1.1" 502 150 2025-12-17 23:17:01,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409430575bc060ce160f93'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:01,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409422575bc060ce160f77 2025-12-17 23:17:01,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409422575bc060ce160f77 HTTP/1.1" 502 150 2025-12-17 23:17:01,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409422575bc060ce160f77'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:02,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409424575bc060ce160f82 INFO: 172.21.0.1:38410 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:17:03,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409411575bc060ce160f39 2025-12-17 23:17:03,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409411575bc060ce160f39 HTTP/1.1" 502 150 2025-12-17 23:17:03,437 - lava_callback - ERROR - Error processing callback for node 69409411575bc060ce160f39: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409411575bc060ce160f39 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409411575bc060ce160f39 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409411575bc060ce160f39 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:03,439 - lava_callback - INFO - Starting async job submit for node 69409432575bc060ce160f96 2025-12-17 23:17:03,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:03,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409432575bc060ce160f96 HTTP/1.1" 502 150 2025-12-17 23:17:03,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409432575bc060ce160f96'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:03,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409432575bc060ce160f96 2025-12-17 23:17:03,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409432575bc060ce160f96 HTTP/1.1" 502 150 2025-12-17 23:17:03,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409432575bc060ce160f96'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:03,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940942c575bc060ce160f8d 2025-12-17 23:17:03,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942c575bc060ce160f8d HTTP/1.1" 502 150 2025-12-17 23:17:03,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940942c575bc060ce160f8d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:03,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940942e575bc060ce160f90 2025-12-17 23:17:03,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942e575bc060ce160f90 HTTP/1.1" 502 150 2025-12-17 23:17:03,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940942e575bc060ce160f90'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:05,278 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409413575bc060ce160f3c 2025-12-17 23:17:05,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409413575bc060ce160f3c HTTP/1.1" 502 150 2025-12-17 23:17:05,280 - lava_callback - ERROR - Error processing callback for node 69409413575bc060ce160f3c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409413575bc060ce160f3c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409413575bc060ce160f3c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409413575bc060ce160f3c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:05,282 - lava_callback - INFO - Starting async job submit for node 69409434575bc060ce160f99 2025-12-17 23:17:05,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:05,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409434575bc060ce160f99 HTTP/1.1" 502 150 2025-12-17 23:17:05,288 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409434575bc060ce160f99'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:05,288 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409434575bc060ce160f99 2025-12-17 23:17:05,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409434575bc060ce160f99 HTTP/1.1" 502 150 2025-12-17 23:17:05,289 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409434575bc060ce160f99'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:05,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409432575bc060ce160f96 2025-12-17 23:17:05,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409432575bc060ce160f96 HTTP/1.1" 502 150 2025-12-17 23:17:05,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409432575bc060ce160f96'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:05,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409430575bc060ce160f93 2025-12-17 23:17:05,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409430575bc060ce160f93 HTTP/1.1" 502 150 2025-12-17 23:17:05,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409430575bc060ce160f93'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:06,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409416575bc060ce160f58 2025-12-17 23:17:06,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409416575bc060ce160f58 HTTP/1.1" 502 150 2025-12-17 23:17:06,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409416575bc060ce160f58'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38418 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:17:06,939 - lava_callback - INFO - Received callback for node 694096df575bc060ce1616e1 INFO: 172.21.0.1:38420 - "POST /node/694096df575bc060ce1616e1 HTTP/1.0" 202 Accepted 2025-12-17 23:17:07,289 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409434575bc060ce160f99 2025-12-17 23:17:07,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409434575bc060ce160f99 HTTP/1.1" 502 150 2025-12-17 23:17:07,290 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409434575bc060ce160f99'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:07,584 - lava_callback - INFO - Received callback for node 694096e2575bc060ce1616eb INFO: 172.21.0.1:38428 - "POST /node/694096e2575bc060ce1616eb HTTP/1.0" 202 Accepted 2025-12-17 23:17:07,670 - lava_callback - INFO - Received callback for node 694096e0575bc060ce1616e8 INFO: 172.21.0.1:38430 - "POST /node/694096e0575bc060ce1616e8 HTTP/1.0" 202 Accepted 2025-12-17 23:17:08,227 - lava_callback - INFO - Received callback for node 694096e4575bc060ce1616f3 INFO: 172.21.0.1:38438 - "POST /node/694096e4575bc060ce1616f3 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:38448 - "GET /node/693aebf0fee8383e92b667a9%5Cn HTTP/1.0" 405 Method Not Allowed 2025-12-17 23:17:08,461 - lava_callback - INFO - Received callback for node 694121ed459ff5fc64d2b7f1 INFO: 172.21.0.1:38456 - "POST /node/694121ed459ff5fc64d2b7f1 HTTP/1.0" 202 Accepted 2025-12-17 23:17:08,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955e575bc060ce1612e8 2025-12-17 23:17:08,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955e575bc060ce1612e8 HTTP/1.1" 502 150 2025-12-17 23:17:08,638 - lava_callback - ERROR - Error processing callback for node 6940955e575bc060ce1612e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940955e575bc060ce1612e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940955e575bc060ce1612e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940955e575bc060ce1612e8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:08,640 - lava_callback - INFO - Starting async job submit for node 69409436575bc060ce160f9e 2025-12-17 23:17:08,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:08,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409436575bc060ce160f9e HTTP/1.1" 502 150 2025-12-17 23:17:08,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409436575bc060ce160f9e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:08,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409436575bc060ce160f9e 2025-12-17 23:17:08,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409436575bc060ce160f9e HTTP/1.1" 502 150 2025-12-17 23:17:08,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409436575bc060ce160f9e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:08,877 - lava_callback - INFO - Received callback for node 694096e5575bc060ce1616f6 INFO: 172.21.0.1:38462 - "POST /node/694096e5575bc060ce1616f6 HTTP/1.0" 202 Accepted 2025-12-17 23:17:09,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409036575bc060ce16036c 2025-12-17 23:17:09,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940942a575bc060ce160f8a 2025-12-17 23:17:09,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942a575bc060ce160f8a HTTP/1.1" 502 150 2025-12-17 23:17:09,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940942a575bc060ce160f8a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:09,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409432575bc060ce160f96 2025-12-17 23:17:09,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409432575bc060ce160f96 HTTP/1.1" 502 150 2025-12-17 23:17:09,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409432575bc060ce160f96'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:09,530 - lava_callback - INFO - Received callback for node 694096e7575bc060ce1616f9 INFO: 172.21.0.1:38468 - "POST /node/694096e7575bc060ce1616f9 HTTP/1.0" 202 Accepted 2025-12-17 23:17:09,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409422575bc060ce160f77 2025-12-17 23:17:09,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409422575bc060ce160f77 HTTP/1.1" 502 150 2025-12-17 23:17:09,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409422575bc060ce160f77'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:10,190 - lava_callback - INFO - Received callback for node 694096ed575bc060ce161704 INFO: 172.21.0.1:46118 - "POST /node/694096ed575bc060ce161704 HTTP/1.0" 202 Accepted 2025-12-17 23:17:10,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409436575bc060ce160f9e 2025-12-17 23:17:10,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409436575bc060ce160f9e HTTP/1.1" 502 150 2025-12-17 23:17:10,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409436575bc060ce160f9e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:11,284 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940941a575bc060ce160f63 2025-12-17 23:17:11,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941a575bc060ce160f63 HTTP/1.1" 502 150 2025-12-17 23:17:11,285 - lava_callback - ERROR - Error processing callback for node 6940941a575bc060ce160f63: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940941a575bc060ce160f63 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940941a575bc060ce160f63 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940941a575bc060ce160f63 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:11,287 - lava_callback - INFO - Starting async job submit for node 69409438575bc060ce160fa7 2025-12-17 23:17:11,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:11,290 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409434575bc060ce160f99 2025-12-17 23:17:11,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409434575bc060ce160f99 HTTP/1.1" 502 150 2025-12-17 23:17:11,296 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409434575bc060ce160f99'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:11,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409438575bc060ce160fa7 HTTP/1.1" 502 150 2025-12-17 23:17:11,297 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409438575bc060ce160fa7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:11,297 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409438575bc060ce160fa7 2025-12-17 23:17:11,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409438575bc060ce160fa7 HTTP/1.1" 502 150 2025-12-17 23:17:11,298 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409438575bc060ce160fa7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:11,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940942c575bc060ce160f8d 2025-12-17 23:17:11,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942c575bc060ce160f8d HTTP/1.1" 502 150 2025-12-17 23:17:11,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940942c575bc060ce160f8d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:11,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409402575bc060ce160f09 HTTP/1.1" 502 150 2025-12-17 23:17:11,728 - lava_callback - ERROR - Error processing callback for node 69409402575bc060ce160f09: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409402575bc060ce160f09 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409402575bc060ce160f09 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409402575bc060ce160f09 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:11,730 - lava_callback - INFO - Starting async job submit for node 69409439575bc060ce160fae 2025-12-17 23:17:11,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:11,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409439575bc060ce160fae HTTP/1.1" 502 150 2025-12-17 23:17:11,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409439575bc060ce160fae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:11,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409439575bc060ce160fae 2025-12-17 23:17:11,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409439575bc060ce160fae HTTP/1.1" 502 150 2025-12-17 23:17:11,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409439575bc060ce160fae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:11,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940942e575bc060ce160f90 2025-12-17 23:17:11,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942e575bc060ce160f90 HTTP/1.1" 502 150 2025-12-17 23:17:11,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940942e575bc060ce160f90'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:13,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940941c575bc060ce160f66 2025-12-17 23:17:13,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941c575bc060ce160f66 HTTP/1.1" 502 150 2025-12-17 23:17:13,005 - lava_callback - ERROR - Error processing callback for node 6940941c575bc060ce160f66: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940941c575bc060ce160f66 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940941c575bc060ce160f66 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940941c575bc060ce160f66 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:13,006 - lava_callback - INFO - Starting async job submit for node 6940943b575bc060ce160fb3 2025-12-17 23:17:13,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:13,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943b575bc060ce160fb3 HTTP/1.1" 502 150 2025-12-17 23:17:13,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940943b575bc060ce160fb3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:13,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940943b575bc060ce160fb3 2025-12-17 23:17:13,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943b575bc060ce160fb3 HTTP/1.1" 502 150 2025-12-17 23:17:13,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940943b575bc060ce160fb3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:13,298 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409438575bc060ce160fa7 2025-12-17 23:17:13,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409438575bc060ce160fa7 HTTP/1.1" 502 150 2025-12-17 23:17:13,299 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409438575bc060ce160fa7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:13,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409430575bc060ce160f93 2025-12-17 23:17:13,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409430575bc060ce160f93 HTTP/1.1" 502 150 2025-12-17 23:17:13,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409430575bc060ce160f93'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:13,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409439575bc060ce160fae 2025-12-17 23:17:13,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409439575bc060ce160fae HTTP/1.1" 502 150 2025-12-17 23:17:13,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409439575bc060ce160fae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:13,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940941e575bc060ce160f69 2025-12-17 23:17:13,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940941e575bc060ce160f69 HTTP/1.1" 502 150 2025-12-17 23:17:13,867 - lava_callback - ERROR - Error processing callback for node 6940941e575bc060ce160f69: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940941e575bc060ce160f69 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940941e575bc060ce160f69 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940941e575bc060ce160f69 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:13,869 - lava_callback - INFO - Starting async job submit for node 6940943d575bc060ce160fb6 2025-12-17 23:17:13,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:13,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943d575bc060ce160fb6 HTTP/1.1" 502 150 2025-12-17 23:17:13,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940943d575bc060ce160fb6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:13,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940943d575bc060ce160fb6 2025-12-17 23:17:13,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943d575bc060ce160fb6 HTTP/1.1" 502 150 2025-12-17 23:17:13,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940943d575bc060ce160fb6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:14,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409436575bc060ce160f9e 2025-12-17 23:17:14,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409436575bc060ce160f9e HTTP/1.1" 502 150 2025-12-17 23:17:14,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409436575bc060ce160f9e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:15,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940943b575bc060ce160fb3 2025-12-17 23:17:15,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943b575bc060ce160fb3 HTTP/1.1" 502 150 2025-12-17 23:17:15,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940943b575bc060ce160fb3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46134 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:17:15,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409427575bc060ce160f85 2025-12-17 23:17:15,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409427575bc060ce160f85 HTTP/1.1" 502 150 2025-12-17 23:17:15,672 - lava_callback - ERROR - Error processing callback for node 69409427575bc060ce160f85: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409427575bc060ce160f85 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409427575bc060ce160f85 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409427575bc060ce160f85 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:15,674 - lava_callback - INFO - Starting async job submit for node 694187d5ecc59770562e6d7d 2025-12-17 23:17:15,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:15,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694187d5ecc59770562e6d7d HTTP/1.1" 502 150 2025-12-17 23:17:15,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694187d5ecc59770562e6d7d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:15,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694187d5ecc59770562e6d7d 2025-12-17 23:17:15,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694187d5ecc59770562e6d7d HTTP/1.1" 502 150 2025-12-17 23:17:15,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694187d5ecc59770562e6d7d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:15,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940943d575bc060ce160fb6 2025-12-17 23:17:15,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943d575bc060ce160fb6 HTTP/1.1" 502 150 2025-12-17 23:17:15,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940943d575bc060ce160fb6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:16,489 - lava_callback - INFO - Received callback for node 694096e8575bc060ce1616fe INFO: 172.21.0.1:46138 - "POST /node/694096e8575bc060ce1616fe HTTP/1.0" 202 Accepted INFO: 172.21.0.1:46148 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:17:17,171 - lava_callback - INFO - Received callback for node 694096eb575bc060ce161701 INFO: 172.21.0.1:46150 - "POST /node/694096eb575bc060ce161701 HTTP/1.0" 202 Accepted 2025-12-17 23:17:17,299 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409438575bc060ce160fa7 2025-12-17 23:17:17,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409432575bc060ce160f96 2025-12-17 23:17:17,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409432575bc060ce160f96 HTTP/1.1" 502 150 2025-12-17 23:17:17,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409432575bc060ce160f96'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:17,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694187d5ecc59770562e6d7d 2025-12-17 23:17:17,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694187d5ecc59770562e6d7d HTTP/1.1" 502 150 2025-12-17 23:17:17,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694187d5ecc59770562e6d7d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:17,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409439575bc060ce160fae 2025-12-17 23:17:17,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409439575bc060ce160fae HTTP/1.1" 502 150 2025-12-17 23:17:17,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409439575bc060ce160fae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:17,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409424575bc060ce160f82 HTTP/1.1" 502 150 2025-12-17 23:17:17,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409424575bc060ce160f82'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:19,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940943b575bc060ce160fb3 2025-12-17 23:17:19,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943b575bc060ce160fb3 HTTP/1.1" 502 150 2025-12-17 23:17:19,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940943b575bc060ce160fb3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:19,297 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409434575bc060ce160f99 2025-12-17 23:17:19,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409434575bc060ce160f99 HTTP/1.1" 502 150 2025-12-17 23:17:19,298 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409434575bc060ce160f99'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:19,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940943d575bc060ce160fb6 2025-12-17 23:17:19,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943d575bc060ce160fb6 HTTP/1.1" 502 150 2025-12-17 23:17:19,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940943d575bc060ce160fb6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:21,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694187d5ecc59770562e6d7d 2025-12-17 23:17:21,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694187d5ecc59770562e6d7d HTTP/1.1" 502 150 2025-12-17 23:17:21,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694187d5ecc59770562e6d7d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:22,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409416575bc060ce160f58 2025-12-17 23:17:22,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409436575bc060ce160f9e 2025-12-17 23:17:22,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409436575bc060ce160f9e HTTP/1.1" 502 150 2025-12-17 23:17:22,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409436575bc060ce160f9e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:24,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409036575bc060ce16036c HTTP/1.1" 502 150 2025-12-17 23:17:24,272 - lava_callback - ERROR - Error processing callback for node 69409036575bc060ce16036c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409036575bc060ce16036c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409036575bc060ce16036c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409036575bc060ce16036c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:24,274 - lava_callback - INFO - Starting async job submit for node 6941170f459ff5fc64d293c2 2025-12-17 23:17:24,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:24,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941170f459ff5fc64d293c2 HTTP/1.1" 502 150 2025-12-17 23:17:24,301 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941170f459ff5fc64d293c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:24,301 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941170f459ff5fc64d293c2 2025-12-17 23:17:24,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941170f459ff5fc64d293c2 HTTP/1.1" 502 150 2025-12-17 23:17:24,302 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941170f459ff5fc64d293c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:25,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940942a575bc060ce160f8a 2025-12-17 23:17:25,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942a575bc060ce160f8a HTTP/1.1" 502 150 2025-12-17 23:17:25,449 - lava_callback - ERROR - Error processing callback for node 6940942a575bc060ce160f8a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940942a575bc060ce160f8a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940942a575bc060ce160f8a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940942a575bc060ce160f8a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:25,451 - lava_callback - INFO - Starting async job submit for node 6940943f575bc060ce160fb9 2025-12-17 23:17:25,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:25,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943f575bc060ce160fb9 HTTP/1.1" 502 150 2025-12-17 23:17:25,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940943f575bc060ce160fb9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:25,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940943f575bc060ce160fb9 2025-12-17 23:17:25,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943f575bc060ce160fb9 HTTP/1.1" 502 150 2025-12-17 23:17:25,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940943f575bc060ce160fb9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:25,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409439575bc060ce160fae 2025-12-17 23:17:25,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409439575bc060ce160fae HTTP/1.1" 502 150 2025-12-17 23:17:25,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409439575bc060ce160fae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:25,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409424575bc060ce160f82 2025-12-17 23:17:25,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409424575bc060ce160f82 HTTP/1.1" 502 150 2025-12-17 23:17:25,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409424575bc060ce160f82'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:25,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409422575bc060ce160f77 2025-12-17 23:17:25,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409422575bc060ce160f77 HTTP/1.1" 502 150 2025-12-17 23:17:25,953 - lava_callback - ERROR - Error processing callback for node 69409422575bc060ce160f77: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409422575bc060ce160f77 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409422575bc060ce160f77 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409422575bc060ce160f77 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:25,955 - lava_callback - INFO - Starting async job submit for node 69409565575bc060ce161303 2025-12-17 23:17:25,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:25,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409565575bc060ce161303 HTTP/1.1" 502 150 2025-12-17 23:17:25,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409565575bc060ce161303'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:25,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409565575bc060ce161303 2025-12-17 23:17:25,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409565575bc060ce161303 HTTP/1.1" 502 150 2025-12-17 23:17:25,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409565575bc060ce161303'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:26,302 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941170f459ff5fc64d293c2 2025-12-17 23:17:26,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941170f459ff5fc64d293c2 HTTP/1.1" 502 150 2025-12-17 23:17:26,303 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941170f459ff5fc64d293c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56206 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:17:26,965 - lava_callback - INFO - Received callback for node 694096f0575bc060ce161707 INFO: 172.21.0.1:56214 - "POST /node/694096f0575bc060ce161707 HTTP/1.0" 202 Accepted 2025-12-17 23:17:27,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940943b575bc060ce160fb3 2025-12-17 23:17:27,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943b575bc060ce160fb3 HTTP/1.1" 502 150 2025-12-17 23:17:27,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940943b575bc060ce160fb3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:27,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940943f575bc060ce160fb9 2025-12-17 23:17:27,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943f575bc060ce160fb9 HTTP/1.1" 502 150 2025-12-17 23:17:27,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940943f575bc060ce160fb9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:27,607 - lava_callback - INFO - Received callback for node 694096f4575bc060ce161714 INFO: 172.21.0.1:56226 - "POST /node/694096f4575bc060ce161714 HTTP/1.0" 202 Accepted 2025-12-17 23:17:27,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940942c575bc060ce160f8d 2025-12-17 23:17:27,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942c575bc060ce160f8d HTTP/1.1" 502 150 2025-12-17 23:17:27,662 - lava_callback - ERROR - Error processing callback for node 6940942c575bc060ce160f8d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940942c575bc060ce160f8d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940942c575bc060ce160f8d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940942c575bc060ce160f8d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:27,664 - lava_callback - INFO - Starting async job submit for node 69409441575bc060ce160fbc 2025-12-17 23:17:27,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:27,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409441575bc060ce160fbc HTTP/1.1" 502 150 2025-12-17 23:17:27,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409441575bc060ce160fbc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:27,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409441575bc060ce160fbc 2025-12-17 23:17:27,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409441575bc060ce160fbc HTTP/1.1" 502 150 2025-12-17 23:17:27,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409441575bc060ce160fbc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:27,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940942e575bc060ce160f90 2025-12-17 23:17:27,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940942e575bc060ce160f90 HTTP/1.1" 502 150 2025-12-17 23:17:27,756 - lava_callback - ERROR - Error processing callback for node 6940942e575bc060ce160f90: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940942e575bc060ce160f90 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940942e575bc060ce160f90 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940942e575bc060ce160f90 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:27,758 - lava_callback - INFO - Starting async job submit for node 69409442575bc060ce160fbf 2025-12-17 23:17:27,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:27,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409442575bc060ce160fbf HTTP/1.1" 502 150 2025-12-17 23:17:27,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409442575bc060ce160fbf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:27,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409442575bc060ce160fbf 2025-12-17 23:17:27,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409442575bc060ce160fbf HTTP/1.1" 502 150 2025-12-17 23:17:27,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409442575bc060ce160fbf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:27,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940943d575bc060ce160fb6 2025-12-17 23:17:27,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943d575bc060ce160fb6 HTTP/1.1" 502 150 2025-12-17 23:17:27,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940943d575bc060ce160fb6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:27,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409565575bc060ce161303 2025-12-17 23:17:27,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409565575bc060ce161303 HTTP/1.1" 502 150 2025-12-17 23:17:27,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409565575bc060ce161303'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:28,265 - lava_callback - INFO - Received callback for node 694096f6575bc060ce161719 INFO: 172.21.0.1:56232 - "POST /node/694096f6575bc060ce161719 HTTP/1.0" 202 Accepted 2025-12-17 23:17:28,929 - lava_callback - INFO - Received callback for node 694096f8575bc060ce16171c INFO: 172.21.0.1:56246 - "POST /node/694096f8575bc060ce16171c HTTP/1.0" 202 Accepted 2025-12-17 23:17:29,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409430575bc060ce160f93 2025-12-17 23:17:29,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409430575bc060ce160f93 HTTP/1.1" 502 150 2025-12-17 23:17:29,538 - lava_callback - ERROR - Error processing callback for node 69409430575bc060ce160f93: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409430575bc060ce160f93 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409430575bc060ce160f93 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409430575bc060ce160f93 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:29,540 - lava_callback - INFO - Starting async job submit for node 69409638575bc060ce161543 2025-12-17 23:17:29,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:29,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409638575bc060ce161543 HTTP/1.1" 502 150 2025-12-17 23:17:29,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409638575bc060ce161543'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:29,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409638575bc060ce161543 2025-12-17 23:17:29,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409638575bc060ce161543 HTTP/1.1" 502 150 2025-12-17 23:17:29,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409638575bc060ce161543'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:29,580 - lava_callback - INFO - Received callback for node 694096fa575bc060ce16171f INFO: 172.21.0.1:56254 - "POST /node/694096fa575bc060ce16171f HTTP/1.0" 202 Accepted 2025-12-17 23:17:29,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409441575bc060ce160fbc 2025-12-17 23:17:29,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409441575bc060ce160fbc HTTP/1.1" 502 150 2025-12-17 23:17:29,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409441575bc060ce160fbc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:29,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694187d5ecc59770562e6d7d 2025-12-17 23:17:29,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694187d5ecc59770562e6d7d HTTP/1.1" 502 150 2025-12-17 23:17:29,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694187d5ecc59770562e6d7d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:29,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409442575bc060ce160fbf 2025-12-17 23:17:29,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409442575bc060ce160fbf HTTP/1.1" 502 150 2025-12-17 23:17:29,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409442575bc060ce160fbf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:30,303 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941170f459ff5fc64d293c2 2025-12-17 23:17:30,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941170f459ff5fc64d293c2 HTTP/1.1" 502 150 2025-12-17 23:17:30,305 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941170f459ff5fc64d293c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:30,924 - lava_callback - INFO - Received callback for node 69409701575bc060ce16172d INFO: 172.21.0.1:33566 - "POST /node/69409701575bc060ce16172d HTTP/1.0" 202 Accepted 2025-12-17 23:17:31,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940943f575bc060ce160fb9 2025-12-17 23:17:31,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943f575bc060ce160fb9 HTTP/1.1" 502 150 2025-12-17 23:17:31,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940943f575bc060ce160fb9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:31,567 - lava_callback - INFO - Received callback for node 69409704575bc060ce161730 INFO: 172.21.0.1:33576 - "POST /node/69409704575bc060ce161730 HTTP/1.0" 202 Accepted 2025-12-17 23:17:31,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409638575bc060ce161543 2025-12-17 23:17:31,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409638575bc060ce161543 HTTP/1.1" 502 150 2025-12-17 23:17:31,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409638575bc060ce161543'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:31,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409565575bc060ce161303 2025-12-17 23:17:31,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409565575bc060ce161303 HTTP/1.1" 502 150 2025-12-17 23:17:31,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409565575bc060ce161303'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:32,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409438575bc060ce160fa7 HTTP/1.1" 502 150 2025-12-17 23:17:32,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409438575bc060ce160fa7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:33,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409432575bc060ce160f96 2025-12-17 23:17:33,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409432575bc060ce160f96 HTTP/1.1" 502 150 2025-12-17 23:17:33,461 - lava_callback - ERROR - Error processing callback for node 69409432575bc060ce160f96: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409432575bc060ce160f96 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409432575bc060ce160f96 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409432575bc060ce160f96 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:33,463 - lava_callback - INFO - Starting async job submit for node 69409444575bc060ce160fc2 2025-12-17 23:17:33,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:33,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409444575bc060ce160fc2 HTTP/1.1" 502 150 2025-12-17 23:17:33,495 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409444575bc060ce160fc2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:33,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409444575bc060ce160fc2 2025-12-17 23:17:33,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409444575bc060ce160fc2 HTTP/1.1" 502 150 2025-12-17 23:17:33,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409444575bc060ce160fc2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:33,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409441575bc060ce160fbc 2025-12-17 23:17:33,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409441575bc060ce160fbc HTTP/1.1" 502 150 2025-12-17 23:17:33,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409441575bc060ce160fbc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:33,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409442575bc060ce160fbf 2025-12-17 23:17:33,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409442575bc060ce160fbf HTTP/1.1" 502 150 2025-12-17 23:17:33,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409442575bc060ce160fbf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:35,298 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409434575bc060ce160f99 2025-12-17 23:17:35,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409444575bc060ce160fc2 2025-12-17 23:17:35,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409444575bc060ce160fc2 HTTP/1.1" 502 150 2025-12-17 23:17:35,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409444575bc060ce160fc2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:35,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409638575bc060ce161543 2025-12-17 23:17:35,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409638575bc060ce161543 HTTP/1.1" 502 150 2025-12-17 23:17:35,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409638575bc060ce161543'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:36,469 - lava_callback - INFO - Received callback for node 694096fd575bc060ce161722 INFO: 172.21.0.1:33588 - "POST /node/694096fd575bc060ce161722 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:33598 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:17:37,138 - lava_callback - INFO - Received callback for node 69409700575bc060ce16172a INFO: 172.21.0.1:33610 - "POST /node/69409700575bc060ce16172a HTTP/1.0" 202 Accepted 2025-12-17 23:17:37,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409416575bc060ce160f58 HTTP/1.1" 502 150 2025-12-17 23:17:37,841 - lava_callback - ERROR - Error processing callback for node 69409416575bc060ce160f58: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409416575bc060ce160f58 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409416575bc060ce160f58 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409416575bc060ce160f58 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:37,843 - lava_callback - INFO - Starting async job submit for node 69409445575bc060ce160fc5 2025-12-17 23:17:37,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:37,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409445575bc060ce160fc5 HTTP/1.1" 502 150 2025-12-17 23:17:37,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409445575bc060ce160fc5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:37,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409445575bc060ce160fc5 2025-12-17 23:17:37,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409445575bc060ce160fc5 HTTP/1.1" 502 150 2025-12-17 23:17:37,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409445575bc060ce160fc5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:38,305 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941170f459ff5fc64d293c2 2025-12-17 23:17:38,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941170f459ff5fc64d293c2 HTTP/1.1" 502 150 2025-12-17 23:17:38,306 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941170f459ff5fc64d293c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:38,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409436575bc060ce160f9e 2025-12-17 23:17:38,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409436575bc060ce160f9e HTTP/1.1" 502 150 2025-12-17 23:17:38,654 - lava_callback - ERROR - Error processing callback for node 69409436575bc060ce160f9e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409436575bc060ce160f9e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409436575bc060ce160f9e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409436575bc060ce160f9e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:38,656 - lava_callback - INFO - Starting async job submit for node 69409447575bc060ce160fc8 2025-12-17 23:17:38,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:38,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409447575bc060ce160fc8 HTTP/1.1" 502 150 2025-12-17 23:17:38,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409447575bc060ce160fc8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:38,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409447575bc060ce160fc8 2025-12-17 23:17:38,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409447575bc060ce160fc8 HTTP/1.1" 502 150 2025-12-17 23:17:38,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409447575bc060ce160fc8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:39,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940943f575bc060ce160fb9 2025-12-17 23:17:39,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943f575bc060ce160fb9 HTTP/1.1" 502 150 2025-12-17 23:17:39,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940943f575bc060ce160fb9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:39,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409444575bc060ce160fc2 2025-12-17 23:17:39,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409444575bc060ce160fc2 HTTP/1.1" 502 150 2025-12-17 23:17:39,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409444575bc060ce160fc2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:39,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409445575bc060ce160fc5 2025-12-17 23:17:39,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409445575bc060ce160fc5 HTTP/1.1" 502 150 2025-12-17 23:17:39,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409445575bc060ce160fc5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:39,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409565575bc060ce161303 2025-12-17 23:17:39,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409565575bc060ce161303 HTTP/1.1" 502 150 2025-12-17 23:17:39,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409565575bc060ce161303'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:40,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409447575bc060ce160fc8 2025-12-17 23:17:40,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409447575bc060ce160fc8 HTTP/1.1" 502 150 2025-12-17 23:17:40,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409447575bc060ce160fc8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:40,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409438575bc060ce160fa7 2025-12-17 23:17:40,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409438575bc060ce160fa7 HTTP/1.1" 502 150 2025-12-17 23:17:40,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409438575bc060ce160fa7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:41,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409441575bc060ce160fbc 2025-12-17 23:17:41,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409441575bc060ce160fbc HTTP/1.1" 502 150 2025-12-17 23:17:41,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409441575bc060ce160fbc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:41,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409439575bc060ce160fae 2025-12-17 23:17:41,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409439575bc060ce160fae HTTP/1.1" 502 150 2025-12-17 23:17:41,743 - lava_callback - ERROR - Error processing callback for node 69409439575bc060ce160fae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409439575bc060ce160fae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409439575bc060ce160fae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409439575bc060ce160fae (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:41,745 - lava_callback - INFO - Starting async job submit for node 69409449575bc060ce160fcb 2025-12-17 23:17:41,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:41,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409449575bc060ce160fcb HTTP/1.1" 502 150 2025-12-17 23:17:41,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409449575bc060ce160fcb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:41,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409449575bc060ce160fcb 2025-12-17 23:17:41,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409449575bc060ce160fcb HTTP/1.1" 502 150 2025-12-17 23:17:41,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409449575bc060ce160fcb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:41,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409442575bc060ce160fbf 2025-12-17 23:17:41,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409442575bc060ce160fbf HTTP/1.1" 502 150 2025-12-17 23:17:41,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409442575bc060ce160fbf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:41,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409424575bc060ce160f82 2025-12-17 23:17:41,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409424575bc060ce160f82 HTTP/1.1" 502 150 2025-12-17 23:17:41,875 - lava_callback - ERROR - Error processing callback for node 69409424575bc060ce160f82: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409424575bc060ce160f82 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409424575bc060ce160f82 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409424575bc060ce160f82 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:41,877 - lava_callback - INFO - Starting async job submit for node 6940944a575bc060ce160fd2 2025-12-17 23:17:41,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:41,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944a575bc060ce160fd2 HTTP/1.1" 502 150 2025-12-17 23:17:41,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940944a575bc060ce160fd2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:41,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940944a575bc060ce160fd2 2025-12-17 23:17:41,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944a575bc060ce160fd2 HTTP/1.1" 502 150 2025-12-17 23:17:41,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940944a575bc060ce160fd2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:43,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940943b575bc060ce160fb3 2025-12-17 23:17:43,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409638575bc060ce161543 2025-12-17 23:17:43,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409638575bc060ce161543 HTTP/1.1" 502 150 2025-12-17 23:17:43,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409638575bc060ce161543'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:43,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409449575bc060ce160fcb 2025-12-17 23:17:43,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409449575bc060ce160fcb HTTP/1.1" 502 150 2025-12-17 23:17:43,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409449575bc060ce160fcb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:43,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409445575bc060ce160fc5 2025-12-17 23:17:43,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409445575bc060ce160fc5 HTTP/1.1" 502 150 2025-12-17 23:17:43,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409445575bc060ce160fc5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:43,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940943d575bc060ce160fb6 2025-12-17 23:17:43,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943d575bc060ce160fb6 HTTP/1.1" 502 150 2025-12-17 23:17:43,891 - lava_callback - ERROR - Error processing callback for node 6940943d575bc060ce160fb6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940943d575bc060ce160fb6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940943d575bc060ce160fb6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940943d575bc060ce160fb6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:43,893 - lava_callback - INFO - Starting async job submit for node 6940944c575bc060ce160fd9 2025-12-17 23:17:43,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:43,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944c575bc060ce160fd9 HTTP/1.1" 502 150 2025-12-17 23:17:43,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940944c575bc060ce160fd9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:43,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940944c575bc060ce160fd9 2025-12-17 23:17:43,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944c575bc060ce160fd9 HTTP/1.1" 502 150 2025-12-17 23:17:43,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940944c575bc060ce160fd9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:43,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940944a575bc060ce160fd2 2025-12-17 23:17:43,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944a575bc060ce160fd2 HTTP/1.1" 502 150 2025-12-17 23:17:43,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940944a575bc060ce160fd2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:44,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409447575bc060ce160fc8 2025-12-17 23:17:44,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409447575bc060ce160fc8 HTTP/1.1" 502 150 2025-12-17 23:17:44,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409447575bc060ce160fc8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:45,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694187d5ecc59770562e6d7d 2025-12-17 23:17:45,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694187d5ecc59770562e6d7d HTTP/1.1" 502 150 2025-12-17 23:17:45,692 - lava_callback - ERROR - Error processing callback for node 694187d5ecc59770562e6d7d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694187d5ecc59770562e6d7d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694187d5ecc59770562e6d7d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694187d5ecc59770562e6d7d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:45,693 - lava_callback - INFO - Starting async job submit for node 6940944e575bc060ce160fe4 2025-12-17 23:17:45,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:45,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944e575bc060ce160fe4 HTTP/1.1" 502 150 2025-12-17 23:17:45,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940944e575bc060ce160fe4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:45,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940944e575bc060ce160fe4 2025-12-17 23:17:45,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944e575bc060ce160fe4 HTTP/1.1" 502 150 2025-12-17 23:17:45,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940944e575bc060ce160fe4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:45,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940944c575bc060ce160fd9 2025-12-17 23:17:45,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944c575bc060ce160fd9 HTTP/1.1" 502 150 2025-12-17 23:17:45,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940944c575bc060ce160fd9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41356 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:17:46,982 - lava_callback - INFO - Received callback for node 69409706575bc060ce161739 INFO: 172.21.0.1:41366 - "POST /node/69409706575bc060ce161739 HTTP/1.0" 202 Accepted 2025-12-17 23:17:47,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409444575bc060ce160fc2 2025-12-17 23:17:47,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409444575bc060ce160fc2 HTTP/1.1" 502 150 2025-12-17 23:17:47,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409444575bc060ce160fc2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:47,628 - lava_callback - INFO - Received callback for node 69409708575bc060ce16173c INFO: 172.21.0.1:41374 - "POST /node/69409708575bc060ce16173c HTTP/1.0" 202 Accepted 2025-12-17 23:17:47,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940944e575bc060ce160fe4 2025-12-17 23:17:47,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944e575bc060ce160fe4 HTTP/1.1" 502 150 2025-12-17 23:17:47,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940944e575bc060ce160fe4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:47,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409449575bc060ce160fcb 2025-12-17 23:17:47,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409449575bc060ce160fcb HTTP/1.1" 502 150 2025-12-17 23:17:47,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409449575bc060ce160fcb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:47,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940944a575bc060ce160fd2 2025-12-17 23:17:47,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944a575bc060ce160fd2 HTTP/1.1" 502 150 2025-12-17 23:17:47,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940944a575bc060ce160fd2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:48,262 - lava_callback - INFO - Received callback for node 6940970e575bc060ce16173f INFO: 172.21.0.1:41380 - "POST /node/6940970e575bc060ce16173f HTTP/1.0" 202 Accepted 2025-12-17 23:17:48,915 - lava_callback - INFO - Received callback for node 69409710575bc060ce161742 INFO: 172.21.0.1:41386 - "POST /node/69409710575bc060ce161742 HTTP/1.0" 202 Accepted 2025-12-17 23:17:49,565 - lava_callback - INFO - Received callback for node 69409711575bc060ce161745 INFO: 172.21.0.1:41392 - "POST /node/69409711575bc060ce161745 HTTP/1.0" 202 Accepted 2025-12-17 23:17:49,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940944c575bc060ce160fd9 2025-12-17 23:17:49,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944c575bc060ce160fd9 HTTP/1.1" 502 150 2025-12-17 23:17:49,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940944c575bc060ce160fd9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:50,205 - lava_callback - INFO - Received callback for node 69409713575bc060ce161748 INFO: 172.21.0.1:34200 - "POST /node/69409713575bc060ce161748 HTTP/1.0" 202 Accepted 2025-12-17 23:17:50,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409434575bc060ce160f99 HTTP/1.1" 502 150 2025-12-17 23:17:50,640 - lava_callback - ERROR - Error processing callback for node 69409434575bc060ce160f99: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409434575bc060ce160f99 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409434575bc060ce160f99 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409434575bc060ce160f99 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:50,642 - lava_callback - INFO - Starting async job submit for node 69409450575bc060ce160fe9 2025-12-17 23:17:50,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:50,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409450575bc060ce160fe9 HTTP/1.1" 502 150 2025-12-17 23:17:50,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409450575bc060ce160fe9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:50,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409450575bc060ce160fe9 2025-12-17 23:17:50,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409450575bc060ce160fe9 HTTP/1.1" 502 150 2025-12-17 23:17:50,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409450575bc060ce160fe9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:50,840 - lava_callback - INFO - Received callback for node 69409714575bc060ce16174b INFO: 172.21.0.1:34202 - "POST /node/69409714575bc060ce16174b HTTP/1.0" 202 Accepted 2025-12-17 23:17:51,474 - lava_callback - INFO - Received callback for node 69409715575bc060ce16174e INFO: 172.21.0.1:34206 - "POST /node/69409715575bc060ce16174e HTTP/1.0" 202 Accepted 2025-12-17 23:17:51,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940944e575bc060ce160fe4 2025-12-17 23:17:51,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944e575bc060ce160fe4 HTTP/1.1" 502 150 2025-12-17 23:17:51,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940944e575bc060ce160fe4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:51,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409445575bc060ce160fc5 2025-12-17 23:17:51,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409445575bc060ce160fc5 HTTP/1.1" 502 150 2025-12-17 23:17:51,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409445575bc060ce160fc5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:52,119 - lava_callback - INFO - Received callback for node 69409717575bc060ce161751 INFO: 172.21.0.1:34212 - "POST /node/69409717575bc060ce161751 HTTP/1.0" 202 Accepted 2025-12-17 23:17:52,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409450575bc060ce160fe9 2025-12-17 23:17:52,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409450575bc060ce160fe9 HTTP/1.1" 502 150 2025-12-17 23:17:52,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409450575bc060ce160fe9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:52,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409447575bc060ce160fc8 2025-12-17 23:17:52,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409447575bc060ce160fc8 HTTP/1.1" 502 150 2025-12-17 23:17:52,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409447575bc060ce160fc8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:52,767 - lava_callback - INFO - Received callback for node 69409718575bc060ce161754 INFO: 172.21.0.1:34220 - "POST /node/69409718575bc060ce161754 HTTP/1.0" 202 Accepted 2025-12-17 23:17:54,307 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941170f459ff5fc64d293c2 2025-12-17 23:17:55,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940943f575bc060ce160fb9 2025-12-17 23:17:55,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943f575bc060ce160fb9 HTTP/1.1" 502 150 2025-12-17 23:17:55,466 - lava_callback - ERROR - Error processing callback for node 6940943f575bc060ce160fb9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940943f575bc060ce160fb9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940943f575bc060ce160fb9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940943f575bc060ce160fb9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:55,468 - lava_callback - INFO - Starting async job submit for node 69409452575bc060ce160fed 2025-12-17 23:17:55,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:55,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409452575bc060ce160fed HTTP/1.1" 502 150 2025-12-17 23:17:55,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409452575bc060ce160fed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:55,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409452575bc060ce160fed 2025-12-17 23:17:55,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409452575bc060ce160fed HTTP/1.1" 502 150 2025-12-17 23:17:55,495 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409452575bc060ce160fed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:55,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409449575bc060ce160fcb 2025-12-17 23:17:55,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409449575bc060ce160fcb HTTP/1.1" 502 150 2025-12-17 23:17:55,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409449575bc060ce160fcb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:55,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940944a575bc060ce160fd2 2025-12-17 23:17:55,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944a575bc060ce160fd2 HTTP/1.1" 502 150 2025-12-17 23:17:55,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940944a575bc060ce160fd2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:55,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409565575bc060ce161303 2025-12-17 23:17:55,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409565575bc060ce161303 HTTP/1.1" 502 150 2025-12-17 23:17:55,993 - lava_callback - ERROR - Error processing callback for node 69409565575bc060ce161303: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409565575bc060ce161303 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409565575bc060ce161303 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409565575bc060ce161303 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:55,994 - lava_callback - INFO - Starting async job submit for node 694191c6ecc59770562e821a 2025-12-17 23:17:55,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:56,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694191c6ecc59770562e821a HTTP/1.1" 502 150 2025-12-17 23:17:56,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694191c6ecc59770562e821a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:56,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694191c6ecc59770562e821a 2025-12-17 23:17:56,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694191c6ecc59770562e821a HTTP/1.1" 502 150 2025-12-17 23:17:56,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694191c6ecc59770562e821a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34234 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:17:56,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409450575bc060ce160fe9 2025-12-17 23:17:56,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409450575bc060ce160fe9 HTTP/1.1" 502 150 2025-12-17 23:17:56,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409450575bc060ce160fe9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:56,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409438575bc060ce160fa7 2025-12-17 23:17:56,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409438575bc060ce160fa7 HTTP/1.1" 502 150 2025-12-17 23:17:56,722 - lava_callback - ERROR - Error processing callback for node 69409438575bc060ce160fa7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409438575bc060ce160fa7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409438575bc060ce160fa7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409438575bc060ce160fa7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:56,724 - lava_callback - INFO - Starting async job submit for node 69409454575bc060ce160ff0 2025-12-17 23:17:56,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:56,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409454575bc060ce160ff0 HTTP/1.1" 502 150 2025-12-17 23:17:56,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409454575bc060ce160ff0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:56,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409454575bc060ce160ff0 2025-12-17 23:17:56,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409454575bc060ce160ff0 HTTP/1.1" 502 150 2025-12-17 23:17:56,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409454575bc060ce160ff0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:57,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409452575bc060ce160fed 2025-12-17 23:17:57,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409452575bc060ce160fed HTTP/1.1" 502 150 2025-12-17 23:17:57,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409452575bc060ce160fed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:57,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409441575bc060ce160fbc 2025-12-17 23:17:57,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409441575bc060ce160fbc HTTP/1.1" 502 150 2025-12-17 23:17:57,687 - lava_callback - ERROR - Error processing callback for node 69409441575bc060ce160fbc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409441575bc060ce160fbc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409441575bc060ce160fbc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409441575bc060ce160fbc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:57,689 - lava_callback - INFO - Starting async job submit for node 69409457575bc060ce160ff6 2025-12-17 23:17:57,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:57,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409457575bc060ce160ff6 HTTP/1.1" 502 150 2025-12-17 23:17:57,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409457575bc060ce160ff6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:57,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409457575bc060ce160ff6 2025-12-17 23:17:57,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409457575bc060ce160ff6 HTTP/1.1" 502 150 2025-12-17 23:17:57,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409457575bc060ce160ff6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:57,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409442575bc060ce160fbf 2025-12-17 23:17:57,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409442575bc060ce160fbf HTTP/1.1" 502 150 2025-12-17 23:17:57,776 - lava_callback - ERROR - Error processing callback for node 69409442575bc060ce160fbf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409442575bc060ce160fbf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409442575bc060ce160fbf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409442575bc060ce160fbf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:57,778 - lava_callback - INFO - Starting async job submit for node 69409460575bc060ce161015 2025-12-17 23:17:57,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:57,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409460575bc060ce161015 HTTP/1.1" 502 150 2025-12-17 23:17:57,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409460575bc060ce161015'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:57,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409460575bc060ce161015 2025-12-17 23:17:57,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409460575bc060ce161015 HTTP/1.1" 502 150 2025-12-17 23:17:57,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409460575bc060ce161015'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:57,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940944c575bc060ce160fd9 2025-12-17 23:17:57,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944c575bc060ce160fd9 HTTP/1.1" 502 150 2025-12-17 23:17:57,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940944c575bc060ce160fd9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:58,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694191c6ecc59770562e821a 2025-12-17 23:17:58,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694191c6ecc59770562e821a HTTP/1.1" 502 150 2025-12-17 23:17:58,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694191c6ecc59770562e821a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:58,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940943b575bc060ce160fb3 HTTP/1.1" 502 150 2025-12-17 23:17:58,320 - lava_callback - ERROR - Error processing callback for node 6940943b575bc060ce160fb3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940943b575bc060ce160fb3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940943b575bc060ce160fb3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940943b575bc060ce160fb3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:58,322 - lava_callback - INFO - Starting async job submit for node 69409455575bc060ce160ff3 2025-12-17 23:17:58,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:58,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409455575bc060ce160ff3 HTTP/1.1" 502 150 2025-12-17 23:17:58,336 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409455575bc060ce160ff3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:58,336 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409455575bc060ce160ff3 2025-12-17 23:17:58,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409455575bc060ce160ff3 HTTP/1.1" 502 150 2025-12-17 23:17:58,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409455575bc060ce160ff3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:58,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409454575bc060ce160ff0 2025-12-17 23:17:58,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409454575bc060ce160ff0 HTTP/1.1" 502 150 2025-12-17 23:17:58,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409454575bc060ce160ff0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:59,414 - lava_callback - INFO - Received callback for node 69409c7a575bc060ce1625db INFO: 172.21.0.1:34246 - "POST /node/69409c7a575bc060ce1625db HTTP/1.0" 202 Accepted 2025-12-17 23:17:59,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409638575bc060ce161543 2025-12-17 23:17:59,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409638575bc060ce161543 HTTP/1.1" 502 150 2025-12-17 23:17:59,580 - lava_callback - ERROR - Error processing callback for node 69409638575bc060ce161543: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409638575bc060ce161543 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409638575bc060ce161543 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409638575bc060ce161543 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:17:59,582 - lava_callback - INFO - Starting async job submit for node 69409457575bc060ce160ff9 2025-12-17 23:17:59,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:17:59,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409457575bc060ce160ff9 HTTP/1.1" 502 150 2025-12-17 23:17:59,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409457575bc060ce160ff9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:59,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409457575bc060ce160ff9 2025-12-17 23:17:59,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409457575bc060ce160ff9 HTTP/1.1" 502 150 2025-12-17 23:17:59,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409457575bc060ce160ff9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:59,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409457575bc060ce160ff6 2025-12-17 23:17:59,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409457575bc060ce160ff6 HTTP/1.1" 502 150 2025-12-17 23:17:59,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409457575bc060ce160ff6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:59,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940944e575bc060ce160fe4 2025-12-17 23:17:59,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944e575bc060ce160fe4 HTTP/1.1" 502 150 2025-12-17 23:17:59,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940944e575bc060ce160fe4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:17:59,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409460575bc060ce161015 2025-12-17 23:17:59,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409460575bc060ce161015 HTTP/1.1" 502 150 2025-12-17 23:17:59,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409460575bc060ce161015'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:00,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409455575bc060ce160ff3 2025-12-17 23:18:00,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409455575bc060ce160ff3 HTTP/1.1" 502 150 2025-12-17 23:18:00,338 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409455575bc060ce160ff3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:01,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409452575bc060ce160fed 2025-12-17 23:18:01,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409457575bc060ce160ff9 2025-12-17 23:18:01,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409457575bc060ce160ff9 HTTP/1.1" 502 150 2025-12-17 23:18:01,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409457575bc060ce160ff9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:02,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694191c6ecc59770562e821a 2025-12-17 23:18:02,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694191c6ecc59770562e821a HTTP/1.1" 502 150 2025-12-17 23:18:02,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694191c6ecc59770562e821a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:02,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409454575bc060ce160ff0 2025-12-17 23:18:02,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409454575bc060ce160ff0 HTTP/1.1" 502 150 2025-12-17 23:18:02,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409454575bc060ce160ff0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38114 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:18:03,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409444575bc060ce160fc2 2025-12-17 23:18:03,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409444575bc060ce160fc2 HTTP/1.1" 502 150 2025-12-17 23:18:03,502 - lava_callback - ERROR - Error processing callback for node 69409444575bc060ce160fc2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409444575bc060ce160fc2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409444575bc060ce160fc2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409444575bc060ce160fc2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:03,505 - lava_callback - INFO - Starting async job submit for node 69409464575bc060ce161024 2025-12-17 23:18:03,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:03,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409464575bc060ce161024 HTTP/1.1" 502 150 2025-12-17 23:18:03,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409464575bc060ce161024'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:03,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409464575bc060ce161024 2025-12-17 23:18:03,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409464575bc060ce161024 HTTP/1.1" 502 150 2025-12-17 23:18:03,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409464575bc060ce161024'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:03,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409457575bc060ce160ff6 2025-12-17 23:18:03,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409457575bc060ce160ff6 HTTP/1.1" 502 150 2025-12-17 23:18:03,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409457575bc060ce160ff6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:03,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409460575bc060ce161015 2025-12-17 23:18:03,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409460575bc060ce161015 HTTP/1.1" 502 150 2025-12-17 23:18:03,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409460575bc060ce161015'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:04,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409455575bc060ce160ff3 2025-12-17 23:18:04,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409455575bc060ce160ff3 HTTP/1.1" 502 150 2025-12-17 23:18:04,340 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409455575bc060ce160ff3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:04,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409450575bc060ce160fe9 2025-12-17 23:18:04,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409450575bc060ce160fe9 HTTP/1.1" 502 150 2025-12-17 23:18:04,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409450575bc060ce160fe9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:05,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409464575bc060ce161024 2025-12-17 23:18:05,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409464575bc060ce161024 HTTP/1.1" 502 150 2025-12-17 23:18:05,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409464575bc060ce161024'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:05,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409457575bc060ce160ff9 2025-12-17 23:18:05,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409457575bc060ce160ff9 HTTP/1.1" 502 150 2025-12-17 23:18:05,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409457575bc060ce160ff9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38128 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:18:07,001 - lava_callback - INFO - Received callback for node 6940971b575bc060ce16175f INFO: 172.21.0.1:38132 - "POST /node/6940971b575bc060ce16175f HTTP/1.0" 202 Accepted 2025-12-17 23:18:07,657 - lava_callback - INFO - Received callback for node 6940971d575bc060ce161764 INFO: 172.21.0.1:38140 - "POST /node/6940971d575bc060ce161764 HTTP/1.0" 202 Accepted 2025-12-17 23:18:07,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409445575bc060ce160fc5 2025-12-17 23:18:07,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409445575bc060ce160fc5 HTTP/1.1" 502 150 2025-12-17 23:18:07,861 - lava_callback - ERROR - Error processing callback for node 69409445575bc060ce160fc5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409445575bc060ce160fc5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409445575bc060ce160fc5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409445575bc060ce160fc5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:07,864 - lava_callback - INFO - Starting async job submit for node 69409466575bc060ce161027 2025-12-17 23:18:07,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:07,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409466575bc060ce161027 HTTP/1.1" 502 150 2025-12-17 23:18:07,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409466575bc060ce161027'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:07,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409466575bc060ce161027 2025-12-17 23:18:07,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409466575bc060ce161027 HTTP/1.1" 502 150 2025-12-17 23:18:07,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409466575bc060ce161027'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:08,320 - lava_callback - INFO - Received callback for node 6940971f575bc060ce16176b INFO: 172.21.0.1:38148 - "POST /node/6940971f575bc060ce16176b HTTP/1.0" 202 Accepted 2025-12-17 23:18:08,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409447575bc060ce160fc8 2025-12-17 23:18:08,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409447575bc060ce160fc8 HTTP/1.1" 502 150 2025-12-17 23:18:08,678 - lava_callback - ERROR - Error processing callback for node 69409447575bc060ce160fc8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409447575bc060ce160fc8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409447575bc060ce160fc8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409447575bc060ce160fc8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:08,681 - lava_callback - INFO - Starting async job submit for node 69409461575bc060ce16101c 2025-12-17 23:18:08,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:08,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409461575bc060ce16101c HTTP/1.1" 502 150 2025-12-17 23:18:08,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409461575bc060ce16101c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:08,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409461575bc060ce16101c 2025-12-17 23:18:08,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409461575bc060ce16101c HTTP/1.1" 502 150 2025-12-17 23:18:08,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409461575bc060ce16101c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:08,979 - lava_callback - INFO - Received callback for node 69409721575bc060ce16176e INFO: 172.21.0.1:38160 - "POST /node/69409721575bc060ce16176e HTTP/1.0" 202 Accepted 2025-12-17 23:18:09,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409464575bc060ce161024 2025-12-17 23:18:09,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409464575bc060ce161024 HTTP/1.1" 502 150 2025-12-17 23:18:09,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409464575bc060ce161024'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:09,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941170f459ff5fc64d293c2 HTTP/1.1" 502 150 2025-12-17 23:18:09,584 - lava_callback - ERROR - Error processing callback for node 6941170f459ff5fc64d293c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941170f459ff5fc64d293c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941170f459ff5fc64d293c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941170f459ff5fc64d293c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:09,587 - lava_callback - INFO - Starting async job submit for node 69409463575bc060ce161021 2025-12-17 23:18:09,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:09,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409463575bc060ce161021 HTTP/1.1" 502 150 2025-12-17 23:18:09,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409463575bc060ce161021'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:09,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409463575bc060ce161021 2025-12-17 23:18:09,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409463575bc060ce161021 HTTP/1.1" 502 150 2025-12-17 23:18:09,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409463575bc060ce161021'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:09,637 - lava_callback - INFO - Received callback for node 69409723575bc060ce161771 INFO: 172.21.0.1:38174 - "POST /node/69409723575bc060ce161771 HTTP/1.0" 202 Accepted 2025-12-17 23:18:09,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409466575bc060ce161027 2025-12-17 23:18:09,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409466575bc060ce161027 HTTP/1.1" 502 150 2025-12-17 23:18:09,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409466575bc060ce161027'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:10,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694191c6ecc59770562e821a 2025-12-17 23:18:10,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694191c6ecc59770562e821a HTTP/1.1" 502 150 2025-12-17 23:18:10,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694191c6ecc59770562e821a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:10,288 - lava_callback - INFO - Received callback for node 69409724575bc060ce161774 INFO: 172.21.0.1:38166 - "POST /node/69409724575bc060ce161774 HTTP/1.0" 202 Accepted 2025-12-17 23:18:10,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409461575bc060ce16101c 2025-12-17 23:18:10,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409461575bc060ce16101c HTTP/1.1" 502 150 2025-12-17 23:18:10,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409461575bc060ce16101c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:10,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409454575bc060ce160ff0 2025-12-17 23:18:10,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409454575bc060ce160ff0 HTTP/1.1" 502 150 2025-12-17 23:18:10,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409454575bc060ce160ff0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:11,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409463575bc060ce161021 2025-12-17 23:18:11,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409463575bc060ce161021 HTTP/1.1" 502 150 2025-12-17 23:18:11,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409463575bc060ce161021'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:11,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409457575bc060ce160ff6 2025-12-17 23:18:11,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409457575bc060ce160ff6 HTTP/1.1" 502 150 2025-12-17 23:18:11,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409457575bc060ce160ff6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:11,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409449575bc060ce160fcb 2025-12-17 23:18:11,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409449575bc060ce160fcb HTTP/1.1" 502 150 2025-12-17 23:18:11,767 - lava_callback - ERROR - Error processing callback for node 69409449575bc060ce160fcb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409449575bc060ce160fcb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409449575bc060ce160fcb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409449575bc060ce160fcb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:11,769 - lava_callback - INFO - Starting async job submit for node 694096b9575bc060ce16167a 2025-12-17 23:18:11,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:11,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409460575bc060ce161015 2025-12-17 23:18:11,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409460575bc060ce161015 HTTP/1.1" 502 150 2025-12-17 23:18:11,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409460575bc060ce161015'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:11,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b9575bc060ce16167a HTTP/1.1" 502 150 2025-12-17 23:18:11,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b9575bc060ce16167a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:11,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b9575bc060ce16167a 2025-12-17 23:18:11,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b9575bc060ce16167a HTTP/1.1" 502 150 2025-12-17 23:18:11,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b9575bc060ce16167a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:11,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940944a575bc060ce160fd2 2025-12-17 23:18:11,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944a575bc060ce160fd2 HTTP/1.1" 502 150 2025-12-17 23:18:11,909 - lava_callback - ERROR - Error processing callback for node 6940944a575bc060ce160fd2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940944a575bc060ce160fd2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940944a575bc060ce160fd2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940944a575bc060ce160fd2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:11,910 - lava_callback - INFO - Starting async job submit for node 69409469575bc060ce16102e 2025-12-17 23:18:11,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:11,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409469575bc060ce16102e HTTP/1.1" 502 150 2025-12-17 23:18:11,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409469575bc060ce16102e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:11,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409469575bc060ce16102e 2025-12-17 23:18:11,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409469575bc060ce16102e HTTP/1.1" 502 150 2025-12-17 23:18:11,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409469575bc060ce16102e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:12,340 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409455575bc060ce160ff3 2025-12-17 23:18:12,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409455575bc060ce160ff3 HTTP/1.1" 502 150 2025-12-17 23:18:12,343 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409455575bc060ce160ff3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:13,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409457575bc060ce160ff9 2025-12-17 23:18:13,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409457575bc060ce160ff9 HTTP/1.1" 502 150 2025-12-17 23:18:13,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409457575bc060ce160ff9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:13,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b9575bc060ce16167a 2025-12-17 23:18:13,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b9575bc060ce16167a HTTP/1.1" 502 150 2025-12-17 23:18:13,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b9575bc060ce16167a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:13,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409466575bc060ce161027 2025-12-17 23:18:13,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409466575bc060ce161027 HTTP/1.1" 502 150 2025-12-17 23:18:13,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409466575bc060ce161027'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:13,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940944c575bc060ce160fd9 2025-12-17 23:18:13,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944c575bc060ce160fd9 HTTP/1.1" 502 150 2025-12-17 23:18:13,911 - lava_callback - ERROR - Error processing callback for node 6940944c575bc060ce160fd9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940944c575bc060ce160fd9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940944c575bc060ce160fd9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940944c575bc060ce160fd9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:13,912 - lava_callback - INFO - Starting async job submit for node 6940946b575bc060ce161031 2025-12-17 23:18:13,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:13,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409469575bc060ce16102e 2025-12-17 23:18:13,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409469575bc060ce16102e HTTP/1.1" 502 150 2025-12-17 23:18:13,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409469575bc060ce16102e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:13,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946b575bc060ce161031 HTTP/1.1" 502 150 2025-12-17 23:18:13,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940946b575bc060ce161031'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:13,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940946b575bc060ce161031 2025-12-17 23:18:13,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946b575bc060ce161031 HTTP/1.1" 502 150 2025-12-17 23:18:13,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940946b575bc060ce161031'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:14,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409461575bc060ce16101c 2025-12-17 23:18:14,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409461575bc060ce16101c HTTP/1.1" 502 150 2025-12-17 23:18:14,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409461575bc060ce16101c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38182 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:18:15,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409463575bc060ce161021 2025-12-17 23:18:15,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409463575bc060ce161021 HTTP/1.1" 502 150 2025-12-17 23:18:15,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409463575bc060ce161021'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:15,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940944e575bc060ce160fe4 2025-12-17 23:18:15,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940944e575bc060ce160fe4 HTTP/1.1" 502 150 2025-12-17 23:18:15,746 - lava_callback - ERROR - Error processing callback for node 6940944e575bc060ce160fe4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940944e575bc060ce160fe4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940944e575bc060ce160fe4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940944e575bc060ce160fe4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:15,748 - lava_callback - INFO - Starting async job submit for node 6940946d575bc060ce161039 2025-12-17 23:18:15,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:15,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946d575bc060ce161039 HTTP/1.1" 502 150 2025-12-17 23:18:15,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940946d575bc060ce161039'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:15,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940946d575bc060ce161039 2025-12-17 23:18:15,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946d575bc060ce161039 HTTP/1.1" 502 150 2025-12-17 23:18:15,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940946d575bc060ce161039'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:15,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940946b575bc060ce161031 2025-12-17 23:18:15,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946b575bc060ce161031 HTTP/1.1" 502 150 2025-12-17 23:18:15,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940946b575bc060ce161031'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:16,489 - lava_callback - INFO - Received callback for node 69409726575bc060ce161777 INFO: 172.21.0.1:38196 - "POST /node/69409726575bc060ce161777 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:38202 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:18:16,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409452575bc060ce160fed HTTP/1.1" 502 150 2025-12-17 23:18:16,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409452575bc060ce160fed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:17,163 - lava_callback - INFO - Received callback for node 69409729575bc060ce16177e INFO: 172.21.0.1:38210 - "POST /node/69409729575bc060ce16177e HTTP/1.0" 202 Accepted 2025-12-17 23:18:17,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409464575bc060ce161024 2025-12-17 23:18:17,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409464575bc060ce161024 HTTP/1.1" 502 150 2025-12-17 23:18:17,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409464575bc060ce161024'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:17,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940946d575bc060ce161039 2025-12-17 23:18:17,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946d575bc060ce161039 HTTP/1.1" 502 150 2025-12-17 23:18:17,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940946d575bc060ce161039'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:17,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b9575bc060ce16167a 2025-12-17 23:18:17,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b9575bc060ce16167a HTTP/1.1" 502 150 2025-12-17 23:18:17,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b9575bc060ce16167a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:17,846 - lava_callback - INFO - Received callback for node 6940972b575bc060ce161781 INFO: 172.21.0.1:38226 - "POST /node/6940972b575bc060ce161781 HTTP/1.0" 202 Accepted 2025-12-17 23:18:17,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409469575bc060ce16102e 2025-12-17 23:18:17,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409469575bc060ce16102e HTTP/1.1" 502 150 2025-12-17 23:18:17,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409469575bc060ce16102e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:18,499 - lava_callback - INFO - Received callback for node 6940972d575bc060ce161784 INFO: 172.21.0.1:38236 - "POST /node/6940972d575bc060ce161784 HTTP/1.0" 202 Accepted 2025-12-17 23:18:19,172 - lava_callback - INFO - Received callback for node 6940972f575bc060ce161787 INFO: 172.21.0.1:38242 - "POST /node/6940972f575bc060ce161787 HTTP/1.0" 202 Accepted 2025-12-17 23:18:19,852 - lava_callback - INFO - Received callback for node 69409731575bc060ce16178a INFO: 172.21.0.1:39284 - "POST /node/69409731575bc060ce16178a HTTP/1.0" 202 Accepted 2025-12-17 23:18:19,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940946b575bc060ce161031 2025-12-17 23:18:19,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946b575bc060ce161031 HTTP/1.1" 502 150 2025-12-17 23:18:19,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940946b575bc060ce161031'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:20,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409450575bc060ce160fe9 2025-12-17 23:18:20,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409450575bc060ce160fe9 HTTP/1.1" 502 150 2025-12-17 23:18:20,659 - lava_callback - ERROR - Error processing callback for node 69409450575bc060ce160fe9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409450575bc060ce160fe9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409450575bc060ce160fe9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409450575bc060ce160fe9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:20,661 - lava_callback - INFO - Starting async job submit for node 69409471575bc060ce16103f 2025-12-17 23:18:20,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:20,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409471575bc060ce16103f HTTP/1.1" 502 150 2025-12-17 23:18:20,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409471575bc060ce16103f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:20,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409471575bc060ce16103f 2025-12-17 23:18:20,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409471575bc060ce16103f HTTP/1.1" 502 150 2025-12-17 23:18:20,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409471575bc060ce16103f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:21,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940946d575bc060ce161039 2025-12-17 23:18:21,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946d575bc060ce161039 HTTP/1.1" 502 150 2025-12-17 23:18:21,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940946d575bc060ce161039'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:21,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409466575bc060ce161027 2025-12-17 23:18:21,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409466575bc060ce161027 HTTP/1.1" 502 150 2025-12-17 23:18:21,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409466575bc060ce161027'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:22,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409471575bc060ce16103f 2025-12-17 23:18:22,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409471575bc060ce16103f HTTP/1.1" 502 150 2025-12-17 23:18:22,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409471575bc060ce16103f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:22,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409461575bc060ce16101c 2025-12-17 23:18:22,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409461575bc060ce16101c HTTP/1.1" 502 150 2025-12-17 23:18:22,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409461575bc060ce16101c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:23,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409463575bc060ce161021 2025-12-17 23:18:23,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409463575bc060ce161021 HTTP/1.1" 502 150 2025-12-17 23:18:23,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409463575bc060ce161021'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:24,430 - lava_callback - INFO - Received callback for node 694097ff575bc060ce16198b INFO: 172.21.0.1:39286 - "POST /node/694097ff575bc060ce16198b HTTP/1.0" 202 Accepted 2025-12-17 23:18:24,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409452575bc060ce160fed 2025-12-17 23:18:24,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409452575bc060ce160fed HTTP/1.1" 502 150 2025-12-17 23:18:24,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409452575bc060ce160fed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:25,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b9575bc060ce16167a 2025-12-17 23:18:25,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b9575bc060ce16167a HTTP/1.1" 502 150 2025-12-17 23:18:25,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b9575bc060ce16167a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:25,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409469575bc060ce16102e 2025-12-17 23:18:25,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409469575bc060ce16102e HTTP/1.1" 502 150 2025-12-17 23:18:25,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409469575bc060ce16102e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:26,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694191c6ecc59770562e821a 2025-12-17 23:18:26,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694191c6ecc59770562e821a HTTP/1.1" 502 150 2025-12-17 23:18:26,011 - lava_callback - ERROR - Error processing callback for node 694191c6ecc59770562e821a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694191c6ecc59770562e821a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694191c6ecc59770562e821a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694191c6ecc59770562e821a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:26,012 - lava_callback - INFO - Starting async job submit for node 69409473575bc060ce16104a 2025-12-17 23:18:26,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:26,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409473575bc060ce16104a HTTP/1.1" 502 150 2025-12-17 23:18:26,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409473575bc060ce16104a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:26,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409473575bc060ce16104a 2025-12-17 23:18:26,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409473575bc060ce16104a HTTP/1.1" 502 150 2025-12-17 23:18:26,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409473575bc060ce16104a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39288 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:18:26,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409471575bc060ce16103f 2025-12-17 23:18:26,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409471575bc060ce16103f HTTP/1.1" 502 150 2025-12-17 23:18:26,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409471575bc060ce16103f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:26,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409454575bc060ce160ff0 2025-12-17 23:18:26,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409454575bc060ce160ff0 HTTP/1.1" 502 150 2025-12-17 23:18:26,736 - lava_callback - ERROR - Error processing callback for node 69409454575bc060ce160ff0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409454575bc060ce160ff0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409454575bc060ce160ff0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409454575bc060ce160ff0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:26,738 - lava_callback - INFO - Starting async job submit for node 69409475575bc060ce16104d 2025-12-17 23:18:26,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:26,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409475575bc060ce16104d HTTP/1.1" 502 150 2025-12-17 23:18:26,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409475575bc060ce16104d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:26,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409475575bc060ce16104d 2025-12-17 23:18:26,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409475575bc060ce16104d HTTP/1.1" 502 150 2025-12-17 23:18:26,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409475575bc060ce16104d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:27,016 - lava_callback - INFO - Received callback for node 69409733575bc060ce16178d INFO: 172.21.0.1:39296 - "POST /node/69409733575bc060ce16178d HTTP/1.0" 202 Accepted 2025-12-17 23:18:27,672 - lava_callback - INFO - Received callback for node 69409737575bc060ce161793 INFO: 172.21.0.1:39302 - "POST /node/69409737575bc060ce161793 HTTP/1.0" 202 Accepted 2025-12-17 23:18:27,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409457575bc060ce160ff6 2025-12-17 23:18:27,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409460575bc060ce161015 2025-12-17 23:18:27,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409460575bc060ce161015 HTTP/1.1" 502 150 2025-12-17 23:18:27,790 - lava_callback - ERROR - Error processing callback for node 69409460575bc060ce161015: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409460575bc060ce161015 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409460575bc060ce161015 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409460575bc060ce161015 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:27,792 - lava_callback - INFO - Starting async job submit for node 69409477575bc060ce161054 2025-12-17 23:18:27,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:27,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409477575bc060ce161054 HTTP/1.1" 502 150 2025-12-17 23:18:27,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409477575bc060ce161054'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:27,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409477575bc060ce161054 2025-12-17 23:18:27,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409477575bc060ce161054 HTTP/1.1" 502 150 2025-12-17 23:18:27,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409477575bc060ce161054'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:27,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940946b575bc060ce161031 2025-12-17 23:18:27,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946b575bc060ce161031 HTTP/1.1" 502 150 2025-12-17 23:18:27,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940946b575bc060ce161031'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:28,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409473575bc060ce16104a 2025-12-17 23:18:28,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409473575bc060ce16104a HTTP/1.1" 502 150 2025-12-17 23:18:28,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409473575bc060ce16104a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:28,301 - lava_callback - INFO - Received callback for node 69409735575bc060ce161790 INFO: 172.21.0.1:39318 - "POST /node/69409735575bc060ce161790 HTTP/1.0" 202 Accepted 2025-12-17 23:18:28,317 - lava_callback - INFO - Received callback for node 6940973b575bc060ce161799 INFO: 172.21.0.1:39330 - "POST /node/6940973b575bc060ce161799 HTTP/1.0" 202 Accepted 2025-12-17 23:18:28,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409455575bc060ce160ff3 2025-12-17 23:18:28,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409455575bc060ce160ff3 HTTP/1.1" 502 150 2025-12-17 23:18:28,345 - lava_callback - ERROR - Error processing callback for node 69409455575bc060ce160ff3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409455575bc060ce160ff3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409455575bc060ce160ff3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409455575bc060ce160ff3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:28,347 - lava_callback - INFO - Starting async job submit for node 69409479575bc060ce161057 2025-12-17 23:18:28,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:28,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409479575bc060ce161057 HTTP/1.1" 502 150 2025-12-17 23:18:28,353 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409479575bc060ce161057'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:28,353 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409479575bc060ce161057 2025-12-17 23:18:28,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409479575bc060ce161057 HTTP/1.1" 502 150 2025-12-17 23:18:28,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409479575bc060ce161057'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:28,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409475575bc060ce16104d 2025-12-17 23:18:28,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409475575bc060ce16104d HTTP/1.1" 502 150 2025-12-17 23:18:28,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409475575bc060ce16104d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:28,987 - lava_callback - INFO - Received callback for node 6940973d575bc060ce16179c INFO: 172.21.0.1:39340 - "POST /node/6940973d575bc060ce16179c HTTP/1.0" 202 Accepted 2025-12-17 23:18:29,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409457575bc060ce160ff9 2025-12-17 23:18:29,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409457575bc060ce160ff9 HTTP/1.1" 502 150 2025-12-17 23:18:29,597 - lava_callback - ERROR - Error processing callback for node 69409457575bc060ce160ff9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409457575bc060ce160ff9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409457575bc060ce160ff9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409457575bc060ce160ff9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:29,598 - lava_callback - INFO - Starting async job submit for node 6940947b575bc060ce16105a 2025-12-17 23:18:29,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:29,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947b575bc060ce16105a HTTP/1.1" 502 150 2025-12-17 23:18:29,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947b575bc060ce16105a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:29,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947b575bc060ce16105a 2025-12-17 23:18:29,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947b575bc060ce16105a HTTP/1.1" 502 150 2025-12-17 23:18:29,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947b575bc060ce16105a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:29,628 - lava_callback - INFO - Received callback for node 6940973e575bc060ce16179f INFO: 172.21.0.1:39344 - "POST /node/6940973e575bc060ce16179f HTTP/1.0" 202 Accepted 2025-12-17 23:18:29,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940946d575bc060ce161039 2025-12-17 23:18:29,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946d575bc060ce161039 HTTP/1.1" 502 150 2025-12-17 23:18:29,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940946d575bc060ce161039'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:29,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409477575bc060ce161054 2025-12-17 23:18:29,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409477575bc060ce161054 HTTP/1.1" 502 150 2025-12-17 23:18:29,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409477575bc060ce161054'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:30,258 - lava_callback - INFO - Received callback for node 69409740575bc060ce1617a2 INFO: 172.21.0.1:53832 - "POST /node/69409740575bc060ce1617a2 HTTP/1.0" 202 Accepted 2025-12-17 23:18:30,354 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409479575bc060ce161057 2025-12-17 23:18:30,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409479575bc060ce161057 HTTP/1.1" 502 150 2025-12-17 23:18:30,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409479575bc060ce161057'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:31,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947b575bc060ce16105a 2025-12-17 23:18:32,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409473575bc060ce16104a 2025-12-17 23:18:32,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409473575bc060ce16104a HTTP/1.1" 502 150 2025-12-17 23:18:32,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409473575bc060ce16104a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:32,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409475575bc060ce16104d 2025-12-17 23:18:32,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409475575bc060ce16104d HTTP/1.1" 502 150 2025-12-17 23:18:32,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409475575bc060ce16104d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:33,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409464575bc060ce161024 2025-12-17 23:18:33,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409464575bc060ce161024 HTTP/1.1" 502 150 2025-12-17 23:18:33,544 - lava_callback - ERROR - Error processing callback for node 69409464575bc060ce161024: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409464575bc060ce161024 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409464575bc060ce161024 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409464575bc060ce161024 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:33,546 - lava_callback - INFO - Starting async job submit for node 6940947c575bc060ce16105d 2025-12-17 23:18:33,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:33,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947c575bc060ce16105d HTTP/1.1" 502 150 2025-12-17 23:18:33,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947c575bc060ce16105d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:33,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947c575bc060ce16105d 2025-12-17 23:18:33,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947c575bc060ce16105d HTTP/1.1" 502 150 2025-12-17 23:18:33,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947c575bc060ce16105d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:33,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409477575bc060ce161054 2025-12-17 23:18:33,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409477575bc060ce161054 HTTP/1.1" 502 150 2025-12-17 23:18:33,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409477575bc060ce161054'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:34,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409479575bc060ce161057 2025-12-17 23:18:34,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409479575bc060ce161057 HTTP/1.1" 502 150 2025-12-17 23:18:34,361 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409479575bc060ce161057'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:34,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409471575bc060ce16103f 2025-12-17 23:18:34,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409471575bc060ce16103f HTTP/1.1" 502 150 2025-12-17 23:18:34,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409471575bc060ce16103f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:35,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947c575bc060ce16105d 2025-12-17 23:18:35,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947c575bc060ce16105d HTTP/1.1" 502 150 2025-12-17 23:18:35,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947c575bc060ce16105d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:36,483 - lava_callback - INFO - Received callback for node 69409739575bc060ce161796 INFO: 172.21.0.1:53838 - "POST /node/69409739575bc060ce161796 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:53844 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:18:37,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409466575bc060ce161027 2025-12-17 23:18:37,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409466575bc060ce161027 HTTP/1.1" 502 150 2025-12-17 23:18:37,883 - lava_callback - ERROR - Error processing callback for node 69409466575bc060ce161027: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409466575bc060ce161027 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409466575bc060ce161027 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409466575bc060ce161027 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:37,886 - lava_callback - INFO - Starting async job submit for node 6940947d575bc060ce161060 2025-12-17 23:18:37,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:37,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947d575bc060ce161060 HTTP/1.1" 502 150 2025-12-17 23:18:37,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947d575bc060ce161060'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:37,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947d575bc060ce161060 2025-12-17 23:18:37,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947d575bc060ce161060 HTTP/1.1" 502 150 2025-12-17 23:18:37,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947d575bc060ce161060'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:38,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409461575bc060ce16101c 2025-12-17 23:18:38,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409461575bc060ce16101c HTTP/1.1" 502 150 2025-12-17 23:18:38,720 - lava_callback - ERROR - Error processing callback for node 69409461575bc060ce16101c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409461575bc060ce16101c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409461575bc060ce16101c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409461575bc060ce16101c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:38,724 - lava_callback - INFO - Starting async job submit for node 6940c5401e3c24e95681d6b3 2025-12-17 23:18:38,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:38,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5401e3c24e95681d6b3 HTTP/1.1" 502 150 2025-12-17 23:18:38,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5401e3c24e95681d6b3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:38,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5401e3c24e95681d6b3 2025-12-17 23:18:38,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5401e3c24e95681d6b3 HTTP/1.1" 502 150 2025-12-17 23:18:38,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5401e3c24e95681d6b3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:39,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947c575bc060ce16105d 2025-12-17 23:18:39,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947c575bc060ce16105d HTTP/1.1" 502 150 2025-12-17 23:18:39,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947c575bc060ce16105d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:39,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409463575bc060ce161021 2025-12-17 23:18:39,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409463575bc060ce161021 HTTP/1.1" 502 150 2025-12-17 23:18:39,627 - lava_callback - ERROR - Error processing callback for node 69409463575bc060ce161021: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409463575bc060ce161021 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409463575bc060ce161021 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409463575bc060ce161021 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:39,630 - lava_callback - INFO - Starting async job submit for node 6940947e575bc060ce161063 2025-12-17 23:18:39,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:39,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947e575bc060ce161063 HTTP/1.1" 502 150 2025-12-17 23:18:39,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947e575bc060ce161063'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:39,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947e575bc060ce161063 2025-12-17 23:18:39,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947e575bc060ce161063 HTTP/1.1" 502 150 2025-12-17 23:18:39,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947e575bc060ce161063'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:39,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947d575bc060ce161060 2025-12-17 23:18:39,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947d575bc060ce161060 HTTP/1.1" 502 150 2025-12-17 23:18:39,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947d575bc060ce161060'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:40,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409473575bc060ce16104a 2025-12-17 23:18:40,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409475575bc060ce16104d 2025-12-17 23:18:40,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409475575bc060ce16104d HTTP/1.1" 502 150 2025-12-17 23:18:40,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409475575bc060ce16104d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:40,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409452575bc060ce160fed 2025-12-17 23:18:40,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409452575bc060ce160fed HTTP/1.1" 502 150 2025-12-17 23:18:40,760 - lava_callback - ERROR - Error processing callback for node 69409452575bc060ce160fed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409452575bc060ce160fed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409452575bc060ce160fed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409452575bc060ce160fed (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:40,762 - lava_callback - INFO - Starting async job submit for node 69409480575bc060ce161066 2025-12-17 23:18:40,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:40,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409480575bc060ce161066 HTTP/1.1" 502 150 2025-12-17 23:18:40,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409480575bc060ce161066'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:40,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409480575bc060ce161066 2025-12-17 23:18:40,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409480575bc060ce161066 HTTP/1.1" 502 150 2025-12-17 23:18:40,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409480575bc060ce161066'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:40,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5401e3c24e95681d6b3 2025-12-17 23:18:40,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5401e3c24e95681d6b3 HTTP/1.1" 502 150 2025-12-17 23:18:40,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5401e3c24e95681d6b3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:41,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947e575bc060ce161063 2025-12-17 23:18:41,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947e575bc060ce161063 HTTP/1.1" 502 150 2025-12-17 23:18:41,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947e575bc060ce161063'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:41,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b9575bc060ce16167a 2025-12-17 23:18:41,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409477575bc060ce161054 2025-12-17 23:18:41,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b9575bc060ce16167a HTTP/1.1" 502 150 2025-12-17 23:18:41,804 - lava_callback - ERROR - Error processing callback for node 694096b9575bc060ce16167a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b9575bc060ce16167a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b9575bc060ce16167a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b9575bc060ce16167a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:41,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409477575bc060ce161054 HTTP/1.1" 502 150 2025-12-17 23:18:41,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409477575bc060ce161054'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:41,806 - lava_callback - INFO - Starting async job submit for node 69409481575bc060ce161069 2025-12-17 23:18:41,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:41,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409481575bc060ce161069 HTTP/1.1" 502 150 2025-12-17 23:18:41,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409481575bc060ce161069'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:41,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409481575bc060ce161069 2025-12-17 23:18:41,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409481575bc060ce161069 HTTP/1.1" 502 150 2025-12-17 23:18:41,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409481575bc060ce161069'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:41,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409469575bc060ce16102e 2025-12-17 23:18:41,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409469575bc060ce16102e HTTP/1.1" 502 150 2025-12-17 23:18:41,924 - lava_callback - ERROR - Error processing callback for node 69409469575bc060ce16102e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409469575bc060ce16102e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409469575bc060ce16102e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409469575bc060ce16102e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:41,926 - lava_callback - INFO - Starting async job submit for node 6940948c575bc060ce161075 2025-12-17 23:18:41,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:41,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940948c575bc060ce161075 HTTP/1.1" 502 150 2025-12-17 23:18:41,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940948c575bc060ce161075'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:41,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940948c575bc060ce161075 2025-12-17 23:18:41,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940948c575bc060ce161075 HTTP/1.1" 502 150 2025-12-17 23:18:41,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940948c575bc060ce161075'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:42,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409479575bc060ce161057 2025-12-17 23:18:42,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409479575bc060ce161057 HTTP/1.1" 502 150 2025-12-17 23:18:42,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409479575bc060ce161057'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:42,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409480575bc060ce161066 2025-12-17 23:18:42,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409480575bc060ce161066 HTTP/1.1" 502 150 2025-12-17 23:18:42,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409480575bc060ce161066'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:43,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409457575bc060ce160ff6 HTTP/1.1" 502 150 2025-12-17 23:18:43,121 - lava_callback - ERROR - Error processing callback for node 69409457575bc060ce160ff6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409457575bc060ce160ff6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409457575bc060ce160ff6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409457575bc060ce160ff6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:43,123 - lava_callback - INFO - Starting async job submit for node 69409486575bc060ce161072 2025-12-17 23:18:43,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:43,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409486575bc060ce161072 HTTP/1.1" 502 150 2025-12-17 23:18:43,134 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409486575bc060ce161072'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:43,134 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409486575bc060ce161072 2025-12-17 23:18:43,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409486575bc060ce161072 HTTP/1.1" 502 150 2025-12-17 23:18:43,135 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409486575bc060ce161072'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:43,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409481575bc060ce161069 2025-12-17 23:18:43,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409481575bc060ce161069 HTTP/1.1" 502 150 2025-12-17 23:18:43,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409481575bc060ce161069'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:43,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947d575bc060ce161060 2025-12-17 23:18:43,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947d575bc060ce161060 HTTP/1.1" 502 150 2025-12-17 23:18:43,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947d575bc060ce161060'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:43,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940946b575bc060ce161031 2025-12-17 23:18:43,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946b575bc060ce161031 HTTP/1.1" 502 150 2025-12-17 23:18:43,932 - lava_callback - ERROR - Error processing callback for node 6940946b575bc060ce161031: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940946b575bc060ce161031 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940946b575bc060ce161031 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940946b575bc060ce161031 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:43,933 - lava_callback - INFO - Starting async job submit for node 6940948f575bc060ce161081 2025-12-17 23:18:43,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:43,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940948c575bc060ce161075 2025-12-17 23:18:43,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940948c575bc060ce161075 HTTP/1.1" 502 150 2025-12-17 23:18:43,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940948c575bc060ce161075'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:43,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940948f575bc060ce161081 HTTP/1.1" 502 150 2025-12-17 23:18:43,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940948f575bc060ce161081'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:43,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940948f575bc060ce161081 2025-12-17 23:18:43,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940948f575bc060ce161081 HTTP/1.1" 502 150 2025-12-17 23:18:43,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940948f575bc060ce161081'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:44,850 - lava_callback - INFO - Received callback for node 69409800575bc060ce16198e 2025-12-17 23:18:44,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5401e3c24e95681d6b3 INFO: 172.21.0.1:50738 - "POST /node/69409800575bc060ce16198e HTTP/1.0" 202 Accepted 2025-12-17 23:18:44,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5401e3c24e95681d6b3 HTTP/1.1" 502 150 2025-12-17 23:18:44,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5401e3c24e95681d6b3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:45,135 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409486575bc060ce161072 2025-12-17 23:18:45,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409486575bc060ce161072 HTTP/1.1" 502 150 2025-12-17 23:18:45,136 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409486575bc060ce161072'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:45,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947e575bc060ce161063 2025-12-17 23:18:45,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947e575bc060ce161063 HTTP/1.1" 502 150 2025-12-17 23:18:45,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947e575bc060ce161063'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:45,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940946d575bc060ce161039 2025-12-17 23:18:45,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940946d575bc060ce161039 HTTP/1.1" 502 150 2025-12-17 23:18:45,760 - lava_callback - ERROR - Error processing callback for node 6940946d575bc060ce161039: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940946d575bc060ce161039 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940946d575bc060ce161039 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940946d575bc060ce161039 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:45,762 - lava_callback - INFO - Starting async job submit for node 69409494575bc060ce161084 2025-12-17 23:18:45,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:45,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409494575bc060ce161084 HTTP/1.1" 502 150 2025-12-17 23:18:45,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409494575bc060ce161084'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:45,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409494575bc060ce161084 2025-12-17 23:18:45,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409494575bc060ce161084 HTTP/1.1" 502 150 2025-12-17 23:18:45,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409494575bc060ce161084'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:45,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940948f575bc060ce161081 2025-12-17 23:18:45,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940948f575bc060ce161081 HTTP/1.1" 502 150 2025-12-17 23:18:45,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940948f575bc060ce161081'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50742 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:18:46,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409480575bc060ce161066 2025-12-17 23:18:46,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409480575bc060ce161066 HTTP/1.1" 502 150 2025-12-17 23:18:46,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409480575bc060ce161066'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:46,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947b575bc060ce16105a HTTP/1.1" 502 150 2025-12-17 23:18:46,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947b575bc060ce16105a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:47,037 - lava_callback - INFO - Received callback for node 69409742575bc060ce1617a7 INFO: 172.21.0.1:50754 - "POST /node/69409742575bc060ce1617a7 HTTP/1.0" 202 Accepted 2025-12-17 23:18:47,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947c575bc060ce16105d 2025-12-17 23:18:47,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947c575bc060ce16105d HTTP/1.1" 502 150 2025-12-17 23:18:47,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947c575bc060ce16105d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:47,694 - lava_callback - INFO - Received callback for node 69409746575bc060ce1617ad INFO: 172.21.0.1:50770 - "POST /node/69409746575bc060ce1617ad HTTP/1.0" 202 Accepted 2025-12-17 23:18:47,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409494575bc060ce161084 2025-12-17 23:18:47,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409494575bc060ce161084 HTTP/1.1" 502 150 2025-12-17 23:18:47,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409494575bc060ce161084'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:47,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409481575bc060ce161069 2025-12-17 23:18:47,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409481575bc060ce161069 HTTP/1.1" 502 150 2025-12-17 23:18:47,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409481575bc060ce161069'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:47,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940948c575bc060ce161075 2025-12-17 23:18:47,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940948c575bc060ce161075 HTTP/1.1" 502 150 2025-12-17 23:18:47,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940948c575bc060ce161075'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:48,338 - lava_callback - INFO - Received callback for node 69409745575bc060ce1617aa INFO: 172.21.0.1:50780 - "POST /node/69409745575bc060ce1617aa HTTP/1.0" 202 Accepted 2025-12-17 23:18:48,358 - lava_callback - INFO - Received callback for node 69409747575bc060ce1617b0 INFO: 172.21.0.1:50796 - "POST /node/69409747575bc060ce1617b0 HTTP/1.0" 202 Accepted 2025-12-17 23:18:49,010 - lava_callback - INFO - Received callback for node 69409749575bc060ce1617b3 INFO: 172.21.0.1:50798 - "POST /node/69409749575bc060ce1617b3 HTTP/1.0" 202 Accepted 2025-12-17 23:18:49,091 - lava_callback - INFO - Received callback for node 69409751575bc060ce1617bf INFO: 172.21.0.1:50800 - "POST /node/69409751575bc060ce1617bf HTTP/1.0" 202 Accepted 2025-12-17 23:18:49,137 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409486575bc060ce161072 2025-12-17 23:18:49,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409486575bc060ce161072 HTTP/1.1" 502 150 2025-12-17 23:18:49,142 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409486575bc060ce161072'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:49,665 - lava_callback - INFO - Received callback for node 6940974b575bc060ce1617b6 INFO: 172.21.0.1:50814 - "POST /node/6940974b575bc060ce1617b6 HTTP/1.0" 202 Accepted 2025-12-17 23:18:49,908 - lava_callback - INFO - Received callback for node 69409756575bc060ce1617c8 INFO: 172.21.0.1:43848 - "POST /node/69409756575bc060ce1617c8 HTTP/1.0" 202 Accepted 2025-12-17 23:18:49,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940948f575bc060ce161081 2025-12-17 23:18:49,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940948f575bc060ce161081 HTTP/1.1" 502 150 2025-12-17 23:18:49,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940948f575bc060ce161081'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:50,301 - lava_callback - INFO - Received callback for node 6940974d575bc060ce1617b9 INFO: 172.21.0.1:43856 - "POST /node/6940974d575bc060ce1617b9 HTTP/1.0" 202 Accepted 2025-12-17 23:18:50,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409471575bc060ce16103f 2025-12-17 23:18:50,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409471575bc060ce16103f HTTP/1.1" 502 150 2025-12-17 23:18:50,674 - lava_callback - ERROR - Error processing callback for node 69409471575bc060ce16103f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409471575bc060ce16103f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409471575bc060ce16103f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409471575bc060ce16103f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:50,676 - lava_callback - INFO - Starting async job submit for node 69409496575bc060ce161087 2025-12-17 23:18:50,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:50,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409496575bc060ce161087 HTTP/1.1" 502 150 2025-12-17 23:18:50,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409496575bc060ce161087'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:50,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409496575bc060ce161087 2025-12-17 23:18:50,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409496575bc060ce161087 HTTP/1.1" 502 150 2025-12-17 23:18:50,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409496575bc060ce161087'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:50,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947b575bc060ce16105a 2025-12-17 23:18:50,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947b575bc060ce16105a HTTP/1.1" 502 150 2025-12-17 23:18:50,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947b575bc060ce16105a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:50,968 - lava_callback - INFO - Received callback for node 6940974f575bc060ce1617bc INFO: 172.21.0.1:43858 - "POST /node/6940974f575bc060ce1617bc HTTP/1.0" 202 Accepted 2025-12-17 23:18:51,616 - lava_callback - INFO - Received callback for node 69409752575bc060ce1617c2 INFO: 172.21.0.1:43868 - "POST /node/69409752575bc060ce1617c2 HTTP/1.0" 202 Accepted 2025-12-17 23:18:51,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409494575bc060ce161084 2025-12-17 23:18:51,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409494575bc060ce161084 HTTP/1.1" 502 150 2025-12-17 23:18:51,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409494575bc060ce161084'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:51,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947d575bc060ce161060 2025-12-17 23:18:51,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947d575bc060ce161060 HTTP/1.1" 502 150 2025-12-17 23:18:51,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947d575bc060ce161060'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:52,262 - lava_callback - INFO - Received callback for node 69409754575bc060ce1617c5 INFO: 172.21.0.1:43878 - "POST /node/69409754575bc060ce1617c5 HTTP/1.0" 202 Accepted 2025-12-17 23:18:52,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409496575bc060ce161087 2025-12-17 23:18:52,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409496575bc060ce161087 HTTP/1.1" 502 150 2025-12-17 23:18:52,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409496575bc060ce161087'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:52,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5401e3c24e95681d6b3 2025-12-17 23:18:52,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5401e3c24e95681d6b3 HTTP/1.1" 502 150 2025-12-17 23:18:52,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5401e3c24e95681d6b3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:53,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947e575bc060ce161063 2025-12-17 23:18:53,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947e575bc060ce161063 HTTP/1.1" 502 150 2025-12-17 23:18:53,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947e575bc060ce161063'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:54,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409480575bc060ce161066 2025-12-17 23:18:54,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409480575bc060ce161066 HTTP/1.1" 502 150 2025-12-17 23:18:54,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409480575bc060ce161066'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:55,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409473575bc060ce16104a HTTP/1.1" 502 150 2025-12-17 23:18:55,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409473575bc060ce16104a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:55,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409481575bc060ce161069 2025-12-17 23:18:55,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409481575bc060ce161069 HTTP/1.1" 502 150 2025-12-17 23:18:55,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409481575bc060ce161069'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:55,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940948c575bc060ce161075 2025-12-17 23:18:55,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940948c575bc060ce161075 HTTP/1.1" 502 150 2025-12-17 23:18:55,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940948c575bc060ce161075'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:56,492 - lava_callback - INFO - Received callback for node 6940975c575bc060ce1617d5 INFO: 172.21.0.1:43894 - "POST /node/6940975c575bc060ce1617d5 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:43904 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:18:56,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409496575bc060ce161087 2025-12-17 23:18:56,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409496575bc060ce161087 HTTP/1.1" 502 150 2025-12-17 23:18:56,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409496575bc060ce161087'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:56,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409475575bc060ce16104d 2025-12-17 23:18:56,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409475575bc060ce16104d HTTP/1.1" 502 150 2025-12-17 23:18:56,751 - lava_callback - ERROR - Error processing callback for node 69409475575bc060ce16104d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409475575bc060ce16104d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409475575bc060ce16104d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409475575bc060ce16104d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:56,753 - lava_callback - INFO - Starting async job submit for node 69409498575bc060ce16108a 2025-12-17 23:18:56,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:56,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409498575bc060ce16108a HTTP/1.1" 502 150 2025-12-17 23:18:56,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409498575bc060ce16108a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:56,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409498575bc060ce16108a 2025-12-17 23:18:56,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409498575bc060ce16108a HTTP/1.1" 502 150 2025-12-17 23:18:56,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409498575bc060ce16108a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:57,142 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409486575bc060ce161072 2025-12-17 23:18:57,171 - lava_callback - INFO - Received callback for node 6940975e575bc060ce1617d8 INFO: 172.21.0.1:43910 - "POST /node/6940975e575bc060ce1617d8 HTTP/1.0" 202 Accepted 2025-12-17 23:18:57,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409477575bc060ce161054 2025-12-17 23:18:57,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409477575bc060ce161054 HTTP/1.1" 502 150 2025-12-17 23:18:57,807 - lava_callback - ERROR - Error processing callback for node 69409477575bc060ce161054: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409477575bc060ce161054 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409477575bc060ce161054 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409477575bc060ce161054 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:57,809 - lava_callback - INFO - Starting async job submit for node 69409499575bc060ce16108d 2025-12-17 23:18:57,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:57,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409499575bc060ce16108d HTTP/1.1" 502 150 2025-12-17 23:18:57,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409499575bc060ce16108d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:57,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409499575bc060ce16108d 2025-12-17 23:18:57,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409499575bc060ce16108d HTTP/1.1" 502 150 2025-12-17 23:18:57,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409499575bc060ce16108d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:57,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940948f575bc060ce161081 2025-12-17 23:18:57,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940948f575bc060ce161081 HTTP/1.1" 502 150 2025-12-17 23:18:57,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940948f575bc060ce161081'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:58,363 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409479575bc060ce161057 2025-12-17 23:18:58,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409479575bc060ce161057 HTTP/1.1" 502 150 2025-12-17 23:18:58,365 - lava_callback - ERROR - Error processing callback for node 69409479575bc060ce161057: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409479575bc060ce161057 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409479575bc060ce161057 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409479575bc060ce161057 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:18:58,367 - lava_callback - INFO - Starting async job submit for node 6940949d575bc060ce16109f 2025-12-17 23:18:58,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:18:58,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949d575bc060ce16109f HTTP/1.1" 502 150 2025-12-17 23:18:58,373 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940949d575bc060ce16109f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:58,374 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940949d575bc060ce16109f 2025-12-17 23:18:58,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949d575bc060ce16109f HTTP/1.1" 502 150 2025-12-17 23:18:58,374 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940949d575bc060ce16109f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:58,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409498575bc060ce16108a 2025-12-17 23:18:58,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409498575bc060ce16108a HTTP/1.1" 502 150 2025-12-17 23:18:58,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409498575bc060ce16108a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:58,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947b575bc060ce16105a 2025-12-17 23:18:58,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947b575bc060ce16105a HTTP/1.1" 502 150 2025-12-17 23:18:58,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940947b575bc060ce16105a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:59,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409494575bc060ce161084 2025-12-17 23:18:59,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409494575bc060ce161084 HTTP/1.1" 502 150 2025-12-17 23:18:59,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409494575bc060ce161084'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:18:59,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409499575bc060ce16108d 2025-12-17 23:18:59,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409499575bc060ce16108d HTTP/1.1" 502 150 2025-12-17 23:18:59,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409499575bc060ce16108d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:00,375 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940949d575bc060ce16109f 2025-12-17 23:19:00,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949d575bc060ce16109f HTTP/1.1" 502 150 2025-12-17 23:19:00,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940949d575bc060ce16109f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:02,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409498575bc060ce16108a 2025-12-17 23:19:02,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409498575bc060ce16108a HTTP/1.1" 502 150 2025-12-17 23:19:02,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409498575bc060ce16108a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37560 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:19:03,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947c575bc060ce16105d 2025-12-17 23:19:03,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947c575bc060ce16105d HTTP/1.1" 502 150 2025-12-17 23:19:03,559 - lava_callback - ERROR - Error processing callback for node 6940947c575bc060ce16105d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940947c575bc060ce16105d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940947c575bc060ce16105d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940947c575bc060ce16105d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:03,561 - lava_callback - INFO - Starting async job submit for node 694094a0575bc060ce1610ad 2025-12-17 23:19:03,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:03,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a0575bc060ce1610ad HTTP/1.1" 502 150 2025-12-17 23:19:03,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a0575bc060ce1610ad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:03,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a0575bc060ce1610ad 2025-12-17 23:19:03,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a0575bc060ce1610ad HTTP/1.1" 502 150 2025-12-17 23:19:03,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a0575bc060ce1610ad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:03,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409499575bc060ce16108d 2025-12-17 23:19:03,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409499575bc060ce16108d HTTP/1.1" 502 150 2025-12-17 23:19:03,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409499575bc060ce16108d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:04,376 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940949d575bc060ce16109f 2025-12-17 23:19:04,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949d575bc060ce16109f HTTP/1.1" 502 150 2025-12-17 23:19:04,377 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940949d575bc060ce16109f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:04,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409496575bc060ce161087 2025-12-17 23:19:04,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409496575bc060ce161087 HTTP/1.1" 502 150 2025-12-17 23:19:04,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409496575bc060ce161087'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:05,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a0575bc060ce1610ad 2025-12-17 23:19:05,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a0575bc060ce1610ad HTTP/1.1" 502 150 2025-12-17 23:19:05,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a0575bc060ce1610ad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37574 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:19:07,046 - lava_callback - INFO - Received callback for node 69409757575bc060ce1617cf INFO: 172.21.0.1:37586 - "POST /node/69409757575bc060ce1617cf HTTP/1.0" 202 Accepted 2025-12-17 23:19:07,704 - lava_callback - INFO - Received callback for node 69409759575bc060ce1617d2 INFO: 172.21.0.1:37596 - "POST /node/69409759575bc060ce1617d2 HTTP/1.0" 202 Accepted 2025-12-17 23:19:07,717 - lava_callback - INFO - Received callback for node 69409763575bc060ce1617f3 INFO: 172.21.0.1:37608 - "POST /node/69409763575bc060ce1617f3 HTTP/1.0" 202 Accepted 2025-12-17 23:19:07,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947d575bc060ce161060 2025-12-17 23:19:07,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947d575bc060ce161060 HTTP/1.1" 502 150 2025-12-17 23:19:07,909 - lava_callback - ERROR - Error processing callback for node 6940947d575bc060ce161060: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940947d575bc060ce161060 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940947d575bc060ce161060 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940947d575bc060ce161060 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:07,911 - lava_callback - INFO - Starting async job submit for node 6940949f575bc060ce1610a8 2025-12-17 23:19:07,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:07,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949f575bc060ce1610a8 HTTP/1.1" 502 150 2025-12-17 23:19:07,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940949f575bc060ce1610a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:07,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940949f575bc060ce1610a8 2025-12-17 23:19:07,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949f575bc060ce1610a8 HTTP/1.1" 502 150 2025-12-17 23:19:07,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940949f575bc060ce1610a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:08,372 - lava_callback - INFO - Received callback for node 69409766575bc060ce1617f9 INFO: 172.21.0.1:37616 - "POST /node/69409766575bc060ce1617f9 HTTP/1.0" 202 Accepted 2025-12-17 23:19:08,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5401e3c24e95681d6b3 2025-12-17 23:19:08,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5401e3c24e95681d6b3 HTTP/1.1" 502 150 2025-12-17 23:19:08,861 - lava_callback - ERROR - Error processing callback for node 6940c5401e3c24e95681d6b3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5401e3c24e95681d6b3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5401e3c24e95681d6b3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5401e3c24e95681d6b3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:08,863 - lava_callback - INFO - Starting async job submit for node 694094ac575bc060ce1610c6 2025-12-17 23:19:08,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:08,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ac575bc060ce1610c6 HTTP/1.1" 502 150 2025-12-17 23:19:08,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ac575bc060ce1610c6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:08,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ac575bc060ce1610c6 2025-12-17 23:19:08,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ac575bc060ce1610c6 HTTP/1.1" 502 150 2025-12-17 23:19:08,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ac575bc060ce1610c6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:09,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a0575bc060ce1610ad 2025-12-17 23:19:09,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a0575bc060ce1610ad HTTP/1.1" 502 150 2025-12-17 23:19:09,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a0575bc060ce1610ad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:09,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947e575bc060ce161063 2025-12-17 23:19:09,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947e575bc060ce161063 HTTP/1.1" 502 150 2025-12-17 23:19:09,686 - lava_callback - ERROR - Error processing callback for node 6940947e575bc060ce161063: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940947e575bc060ce161063 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940947e575bc060ce161063 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940947e575bc060ce161063 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:09,689 - lava_callback - INFO - Starting async job submit for node 694094af575bc060ce1610cf 2025-12-17 23:19:09,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:09,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094af575bc060ce1610cf HTTP/1.1" 502 150 2025-12-17 23:19:09,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094af575bc060ce1610cf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:09,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094af575bc060ce1610cf 2025-12-17 23:19:09,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094af575bc060ce1610cf HTTP/1.1" 502 150 2025-12-17 23:19:09,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094af575bc060ce1610cf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:09,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940949f575bc060ce1610a8 2025-12-17 23:19:09,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949f575bc060ce1610a8 HTTP/1.1" 502 150 2025-12-17 23:19:09,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940949f575bc060ce1610a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:10,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409498575bc060ce16108a 2025-12-17 23:19:10,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409498575bc060ce16108a HTTP/1.1" 502 150 2025-12-17 23:19:10,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409498575bc060ce16108a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:10,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409480575bc060ce161066 2025-12-17 23:19:10,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409480575bc060ce161066 HTTP/1.1" 502 150 2025-12-17 23:19:10,782 - lava_callback - ERROR - Error processing callback for node 69409480575bc060ce161066: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409480575bc060ce161066 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409480575bc060ce161066 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409480575bc060ce161066 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:10,788 - lava_callback - INFO - Starting async job submit for node 694094a2575bc060ce1610b4 2025-12-17 23:19:10,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:10,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a2575bc060ce1610b4 HTTP/1.1" 502 150 2025-12-17 23:19:10,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a2575bc060ce1610b4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:10,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a2575bc060ce1610b4 2025-12-17 23:19:10,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a2575bc060ce1610b4 HTTP/1.1" 502 150 2025-12-17 23:19:10,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a2575bc060ce1610b4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:10,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ac575bc060ce1610c6 2025-12-17 23:19:10,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ac575bc060ce1610c6 HTTP/1.1" 502 150 2025-12-17 23:19:10,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ac575bc060ce1610c6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:11,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409473575bc060ce16104a 2025-12-17 23:19:11,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409473575bc060ce16104a HTTP/1.1" 502 150 2025-12-17 23:19:11,410 - lava_callback - ERROR - Error processing callback for node 69409473575bc060ce16104a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409473575bc060ce16104a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409473575bc060ce16104a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409473575bc060ce16104a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:11,413 - lava_callback - INFO - Starting async job submit for node 694094a5575bc060ce1610b7 2025-12-17 23:19:11,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:11,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a5575bc060ce1610b7 HTTP/1.1" 502 150 2025-12-17 23:19:11,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a5575bc060ce1610b7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:11,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a5575bc060ce1610b7 2025-12-17 23:19:11,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a5575bc060ce1610b7 HTTP/1.1" 502 150 2025-12-17 23:19:11,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a5575bc060ce1610b7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:11,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094af575bc060ce1610cf 2025-12-17 23:19:11,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094af575bc060ce1610cf HTTP/1.1" 502 150 2025-12-17 23:19:11,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094af575bc060ce1610cf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:11,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409481575bc060ce161069 2025-12-17 23:19:11,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409481575bc060ce161069 HTTP/1.1" 502 150 2025-12-17 23:19:11,819 - lava_callback - ERROR - Error processing callback for node 69409481575bc060ce161069: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409481575bc060ce161069 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409481575bc060ce161069 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409481575bc060ce161069 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:11,821 - lava_callback - INFO - Starting async job submit for node 694094a6575bc060ce1610ba 2025-12-17 23:19:11,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:11,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409499575bc060ce16108d 2025-12-17 23:19:11,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409499575bc060ce16108d HTTP/1.1" 502 150 2025-12-17 23:19:11,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409499575bc060ce16108d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:11,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a6575bc060ce1610ba HTTP/1.1" 502 150 2025-12-17 23:19:11,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a6575bc060ce1610ba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:11,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a6575bc060ce1610ba 2025-12-17 23:19:11,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a6575bc060ce1610ba HTTP/1.1" 502 150 2025-12-17 23:19:11,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a6575bc060ce1610ba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:11,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940948c575bc060ce161075 2025-12-17 23:19:11,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940948c575bc060ce161075 HTTP/1.1" 502 150 2025-12-17 23:19:11,939 - lava_callback - ERROR - Error processing callback for node 6940948c575bc060ce161075: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940948c575bc060ce161075 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940948c575bc060ce161075 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940948c575bc060ce161075 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:11,941 - lava_callback - INFO - Starting async job submit for node 694094a9575bc060ce1610bd 2025-12-17 23:19:11,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:11,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a9575bc060ce1610bd HTTP/1.1" 502 150 2025-12-17 23:19:11,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a9575bc060ce1610bd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:11,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a9575bc060ce1610bd 2025-12-17 23:19:11,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a9575bc060ce1610bd HTTP/1.1" 502 150 2025-12-17 23:19:11,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a9575bc060ce1610bd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:12,378 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940949d575bc060ce16109f 2025-12-17 23:19:12,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949d575bc060ce16109f HTTP/1.1" 502 150 2025-12-17 23:19:12,379 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940949d575bc060ce16109f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:12,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409486575bc060ce161072 HTTP/1.1" 502 150 2025-12-17 23:19:12,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409486575bc060ce161072'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:12,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a2575bc060ce1610b4 2025-12-17 23:19:12,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a2575bc060ce1610b4 HTTP/1.1" 502 150 2025-12-17 23:19:12,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a2575bc060ce1610b4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:13,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a5575bc060ce1610b7 2025-12-17 23:19:13,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a5575bc060ce1610b7 HTTP/1.1" 502 150 2025-12-17 23:19:13,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a5575bc060ce1610b7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:13,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a6575bc060ce1610ba 2025-12-17 23:19:13,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a6575bc060ce1610ba HTTP/1.1" 502 150 2025-12-17 23:19:13,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a6575bc060ce1610ba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:13,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940949f575bc060ce1610a8 2025-12-17 23:19:13,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949f575bc060ce1610a8 HTTP/1.1" 502 150 2025-12-17 23:19:13,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940949f575bc060ce1610a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:13,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940948f575bc060ce161081 2025-12-17 23:19:13,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940948f575bc060ce161081 HTTP/1.1" 502 150 2025-12-17 23:19:13,947 - lava_callback - ERROR - Error processing callback for node 6940948f575bc060ce161081: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940948f575bc060ce161081 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940948f575bc060ce161081 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940948f575bc060ce161081 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:13,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a9575bc060ce1610bd 2025-12-17 23:19:13,949 - lava_callback - INFO - Starting async job submit for node 694094aa575bc060ce1610c0 2025-12-17 23:19:13,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:13,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a9575bc060ce1610bd HTTP/1.1" 502 150 2025-12-17 23:19:13,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a9575bc060ce1610bd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:13,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094aa575bc060ce1610c0 HTTP/1.1" 502 150 2025-12-17 23:19:13,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094aa575bc060ce1610c0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:13,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094aa575bc060ce1610c0 2025-12-17 23:19:13,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094aa575bc060ce1610c0 HTTP/1.1" 502 150 2025-12-17 23:19:13,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094aa575bc060ce1610c0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:14,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ac575bc060ce1610c6 2025-12-17 23:19:14,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ac575bc060ce1610c6 HTTP/1.1" 502 150 2025-12-17 23:19:14,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ac575bc060ce1610c6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:14,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940947b575bc060ce16105a 2025-12-17 23:19:14,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940947b575bc060ce16105a HTTP/1.1" 502 150 2025-12-17 23:19:14,965 - lava_callback - ERROR - Error processing callback for node 6940947b575bc060ce16105a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940947b575bc060ce16105a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940947b575bc060ce16105a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940947b575bc060ce16105a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:14,966 - lava_callback - INFO - Starting async job submit for node 694094ab575bc060ce1610c3 2025-12-17 23:19:14,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:14,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ab575bc060ce1610c3 HTTP/1.1" 502 150 2025-12-17 23:19:14,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ab575bc060ce1610c3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:14,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ab575bc060ce1610c3 2025-12-17 23:19:14,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ab575bc060ce1610c3 HTTP/1.1" 502 150 2025-12-17 23:19:14,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ab575bc060ce1610c3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60996 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:19:15,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094af575bc060ce1610cf 2025-12-17 23:19:15,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094af575bc060ce1610cf HTTP/1.1" 502 150 2025-12-17 23:19:15,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094af575bc060ce1610cf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:15,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409494575bc060ce161084 2025-12-17 23:19:15,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409494575bc060ce161084 HTTP/1.1" 502 150 2025-12-17 23:19:15,774 - lava_callback - ERROR - Error processing callback for node 69409494575bc060ce161084: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409494575bc060ce161084 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409494575bc060ce161084 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409494575bc060ce161084 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:15,776 - lava_callback - INFO - Starting async job submit for node 694094af575bc060ce1610d4 2025-12-17 23:19:15,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:15,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094af575bc060ce1610d4 HTTP/1.1" 502 150 2025-12-17 23:19:15,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094af575bc060ce1610d4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:15,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094af575bc060ce1610d4 2025-12-17 23:19:15,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094af575bc060ce1610d4 HTTP/1.1" 502 150 2025-12-17 23:19:15,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094af575bc060ce1610d4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:15,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094aa575bc060ce1610c0 2025-12-17 23:19:15,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094aa575bc060ce1610c0 HTTP/1.1" 502 150 2025-12-17 23:19:15,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094aa575bc060ce1610c0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:32780 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:19:16,553 - lava_callback - INFO - Received callback for node 69409761575bc060ce1617f0 INFO: 172.21.0.1:32782 - "POST /node/69409761575bc060ce1617f0 HTTP/1.0" 202 Accepted 2025-12-17 23:19:16,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a2575bc060ce1610b4 2025-12-17 23:19:16,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a2575bc060ce1610b4 HTTP/1.1" 502 150 2025-12-17 23:19:16,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a2575bc060ce1610b4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:16,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ab575bc060ce1610c3 2025-12-17 23:19:16,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ab575bc060ce1610c3 HTTP/1.1" 502 150 2025-12-17 23:19:16,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ab575bc060ce1610c3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:17,224 - lava_callback - INFO - Received callback for node 69409768575bc060ce1617fc INFO: 172.21.0.1:32784 - "POST /node/69409768575bc060ce1617fc HTTP/1.0" 202 Accepted 2025-12-17 23:19:17,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a5575bc060ce1610b7 2025-12-17 23:19:17,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a5575bc060ce1610b7 HTTP/1.1" 502 150 2025-12-17 23:19:17,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a5575bc060ce1610b7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:17,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a0575bc060ce1610ad 2025-12-17 23:19:17,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a0575bc060ce1610ad HTTP/1.1" 502 150 2025-12-17 23:19:17,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a0575bc060ce1610ad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:17,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094af575bc060ce1610d4 2025-12-17 23:19:17,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094af575bc060ce1610d4 HTTP/1.1" 502 150 2025-12-17 23:19:17,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094af575bc060ce1610d4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:17,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a6575bc060ce1610ba 2025-12-17 23:19:17,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a6575bc060ce1610ba HTTP/1.1" 502 150 2025-12-17 23:19:17,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a6575bc060ce1610ba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:17,865 - lava_callback - INFO - Received callback for node 6940976a575bc060ce161802 INFO: 172.21.0.1:32794 - "POST /node/6940976a575bc060ce161802 HTTP/1.0" 202 Accepted 2025-12-17 23:19:17,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a9575bc060ce1610bd 2025-12-17 23:19:17,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a9575bc060ce1610bd HTTP/1.1" 502 150 2025-12-17 23:19:17,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a9575bc060ce1610bd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:18,527 - lava_callback - INFO - Received callback for node 6940976b575bc060ce16180a INFO: 172.21.0.1:32808 - "POST /node/6940976b575bc060ce16180a HTTP/1.0" 202 Accepted 2025-12-17 23:19:19,174 - lava_callback - INFO - Received callback for node 6940976c575bc060ce161811 INFO: 172.21.0.1:32816 - "POST /node/6940976c575bc060ce161811 HTTP/1.0" 202 Accepted 2025-12-17 23:19:19,845 - lava_callback - INFO - Received callback for node 6940976e575bc060ce161818 INFO: 172.21.0.1:35936 - "POST /node/6940976e575bc060ce161818 HTTP/1.0" 202 Accepted 2025-12-17 23:19:19,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094aa575bc060ce1610c0 2025-12-17 23:19:19,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094aa575bc060ce1610c0 HTTP/1.1" 502 150 2025-12-17 23:19:19,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094aa575bc060ce1610c0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:20,510 - lava_callback - INFO - Received callback for node 6940976f575bc060ce16181d INFO: 172.21.0.1:35944 - "POST /node/6940976f575bc060ce16181d HTTP/1.0" 202 Accepted 2025-12-17 23:19:20,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409496575bc060ce161087 2025-12-17 23:19:20,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ab575bc060ce1610c3 2025-12-17 23:19:20,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ab575bc060ce1610c3 HTTP/1.1" 502 150 2025-12-17 23:19:20,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ab575bc060ce1610c3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:21,183 - lava_callback - INFO - Received callback for node 69409770575bc060ce161820 INFO: 172.21.0.1:35952 - "POST /node/69409770575bc060ce161820 HTTP/1.0" 202 Accepted 2025-12-17 23:19:21,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094af575bc060ce1610d4 2025-12-17 23:19:21,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094af575bc060ce1610d4 HTTP/1.1" 502 150 2025-12-17 23:19:21,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094af575bc060ce1610d4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:21,845 - lava_callback - INFO - Received callback for node 69409771575bc060ce161823 INFO: 172.21.0.1:35960 - "POST /node/69409771575bc060ce161823 HTTP/1.0" 202 Accepted 2025-12-17 23:19:21,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940949f575bc060ce1610a8 2025-12-17 23:19:21,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949f575bc060ce1610a8 HTTP/1.1" 502 150 2025-12-17 23:19:21,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940949f575bc060ce1610a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:22,504 - lava_callback - INFO - Received callback for node 69409773575bc060ce161826 INFO: 172.21.0.1:35968 - "POST /node/69409773575bc060ce161826 HTTP/1.0" 202 Accepted 2025-12-17 23:19:22,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ac575bc060ce1610c6 2025-12-17 23:19:22,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ac575bc060ce1610c6 HTTP/1.1" 502 150 2025-12-17 23:19:22,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ac575bc060ce1610c6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:23,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094af575bc060ce1610cf 2025-12-17 23:19:23,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094af575bc060ce1610cf HTTP/1.1" 502 150 2025-12-17 23:19:23,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094af575bc060ce1610cf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:24,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a2575bc060ce1610b4 2025-12-17 23:19:24,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a2575bc060ce1610b4 HTTP/1.1" 502 150 2025-12-17 23:19:24,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a2575bc060ce1610b4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:25,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a5575bc060ce1610b7 2025-12-17 23:19:25,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a5575bc060ce1610b7 HTTP/1.1" 502 150 2025-12-17 23:19:25,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a5575bc060ce1610b7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:25,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a6575bc060ce1610ba 2025-12-17 23:19:25,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a6575bc060ce1610ba HTTP/1.1" 502 150 2025-12-17 23:19:25,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a6575bc060ce1610ba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:25,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a9575bc060ce1610bd 2025-12-17 23:19:25,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a9575bc060ce1610bd HTTP/1.1" 502 150 2025-12-17 23:19:25,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094a9575bc060ce1610bd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35976 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:19:26,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409498575bc060ce16108a 2025-12-17 23:19:26,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409498575bc060ce16108a HTTP/1.1" 502 150 2025-12-17 23:19:26,769 - lava_callback - ERROR - Error processing callback for node 69409498575bc060ce16108a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409498575bc060ce16108a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409498575bc060ce16108a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409498575bc060ce16108a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:26,772 - lava_callback - INFO - Starting async job submit for node 694094b1575bc060ce1610d9 2025-12-17 23:19:26,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:26,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b1575bc060ce1610d9 HTTP/1.1" 502 150 2025-12-17 23:19:26,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b1575bc060ce1610d9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:26,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b1575bc060ce1610d9 2025-12-17 23:19:26,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b1575bc060ce1610d9 HTTP/1.1" 502 150 2025-12-17 23:19:26,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b1575bc060ce1610d9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:27,067 - lava_callback - INFO - Received callback for node 69409775575bc060ce16182c INFO: 172.21.0.1:35982 - "POST /node/69409775575bc060ce16182c HTTP/1.0" 202 Accepted 2025-12-17 23:19:27,741 - lava_callback - INFO - Received callback for node 69409777575bc060ce16182f INFO: 172.21.0.1:35996 - "POST /node/69409777575bc060ce16182f HTTP/1.0" 202 Accepted 2025-12-17 23:19:27,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409499575bc060ce16108d 2025-12-17 23:19:27,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409499575bc060ce16108d HTTP/1.1" 502 150 2025-12-17 23:19:27,840 - lava_callback - ERROR - Error processing callback for node 69409499575bc060ce16108d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409499575bc060ce16108d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409499575bc060ce16108d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409499575bc060ce16108d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:27,842 - lava_callback - INFO - Starting async job submit for node 694094b3575bc060ce1610dc 2025-12-17 23:19:27,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:27,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b3575bc060ce1610dc HTTP/1.1" 502 150 2025-12-17 23:19:27,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b3575bc060ce1610dc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:27,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b3575bc060ce1610dc 2025-12-17 23:19:27,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b3575bc060ce1610dc HTTP/1.1" 502 150 2025-12-17 23:19:27,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b3575bc060ce1610dc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:27,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094aa575bc060ce1610c0 2025-12-17 23:19:27,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094aa575bc060ce1610c0 HTTP/1.1" 502 150 2025-12-17 23:19:27,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094aa575bc060ce1610c0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:28,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940949d575bc060ce16109f 2025-12-17 23:19:28,380 - lava_callback - INFO - Received callback for node 6940977b575bc060ce161835 INFO: 172.21.0.1:36008 - "POST /node/6940977b575bc060ce161835 HTTP/1.0" 202 Accepted 2025-12-17 23:19:28,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949d575bc060ce16109f HTTP/1.1" 502 150 2025-12-17 23:19:28,386 - lava_callback - ERROR - Error processing callback for node 6940949d575bc060ce16109f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940949d575bc060ce16109f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940949d575bc060ce16109f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940949d575bc060ce16109f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:28,389 - lava_callback - INFO - Starting async job submit for node 694094b5575bc060ce1610df 2025-12-17 23:19:28,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:28,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b5575bc060ce1610df HTTP/1.1" 502 150 2025-12-17 23:19:28,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b5575bc060ce1610df'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:28,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b5575bc060ce1610df 2025-12-17 23:19:28,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b5575bc060ce1610df HTTP/1.1" 502 150 2025-12-17 23:19:28,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b5575bc060ce1610df'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:28,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409486575bc060ce161072 2025-12-17 23:19:28,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409486575bc060ce161072 HTTP/1.1" 502 150 2025-12-17 23:19:28,562 - lava_callback - ERROR - Error processing callback for node 69409486575bc060ce161072: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409486575bc060ce161072 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409486575bc060ce161072 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409486575bc060ce161072 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:28,563 - lava_callback - INFO - Starting async job submit for node 694094b7575bc060ce1610e2 2025-12-17 23:19:28,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:28,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b7575bc060ce1610e2 HTTP/1.1" 502 150 2025-12-17 23:19:28,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b7575bc060ce1610e2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:28,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b7575bc060ce1610e2 2025-12-17 23:19:28,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b7575bc060ce1610e2 HTTP/1.1" 502 150 2025-12-17 23:19:28,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b7575bc060ce1610e2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:28,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b1575bc060ce1610d9 2025-12-17 23:19:28,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b1575bc060ce1610d9 HTTP/1.1" 502 150 2025-12-17 23:19:28,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b1575bc060ce1610d9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:28,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ab575bc060ce1610c3 2025-12-17 23:19:28,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ab575bc060ce1610c3 HTTP/1.1" 502 150 2025-12-17 23:19:28,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ab575bc060ce1610c3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:29,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094af575bc060ce1610d4 2025-12-17 23:19:29,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094af575bc060ce1610d4 HTTP/1.1" 502 150 2025-12-17 23:19:29,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094af575bc060ce1610d4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:29,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b3575bc060ce1610dc 2025-12-17 23:19:29,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b3575bc060ce1610dc HTTP/1.1" 502 150 2025-12-17 23:19:29,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b3575bc060ce1610dc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:30,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b5575bc060ce1610df 2025-12-17 23:19:30,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b5575bc060ce1610df HTTP/1.1" 502 150 2025-12-17 23:19:30,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b5575bc060ce1610df'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:30,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b7575bc060ce1610e2 2025-12-17 23:19:30,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b7575bc060ce1610e2 HTTP/1.1" 502 150 2025-12-17 23:19:30,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b7575bc060ce1610e2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:32,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b1575bc060ce1610d9 2025-12-17 23:19:32,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b1575bc060ce1610d9 HTTP/1.1" 502 150 2025-12-17 23:19:32,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b1575bc060ce1610d9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:33,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a0575bc060ce1610ad 2025-12-17 23:19:33,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a0575bc060ce1610ad HTTP/1.1" 502 150 2025-12-17 23:19:33,581 - lava_callback - ERROR - Error processing callback for node 694094a0575bc060ce1610ad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094a0575bc060ce1610ad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094a0575bc060ce1610ad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094a0575bc060ce1610ad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:33,583 - lava_callback - INFO - Starting async job submit for node 694094bd575bc060ce1610eb 2025-12-17 23:19:33,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:33,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bd575bc060ce1610eb HTTP/1.1" 502 150 2025-12-17 23:19:33,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094bd575bc060ce1610eb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:33,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094bd575bc060ce1610eb 2025-12-17 23:19:33,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bd575bc060ce1610eb HTTP/1.1" 502 150 2025-12-17 23:19:33,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094bd575bc060ce1610eb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:33,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b3575bc060ce1610dc 2025-12-17 23:19:33,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b3575bc060ce1610dc HTTP/1.1" 502 150 2025-12-17 23:19:33,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b3575bc060ce1610dc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:34,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b5575bc060ce1610df 2025-12-17 23:19:34,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b5575bc060ce1610df HTTP/1.1" 502 150 2025-12-17 23:19:34,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b5575bc060ce1610df'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:34,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b7575bc060ce1610e2 2025-12-17 23:19:34,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b7575bc060ce1610e2 HTTP/1.1" 502 150 2025-12-17 23:19:34,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b7575bc060ce1610e2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:35,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094bd575bc060ce1610eb 2025-12-17 23:19:35,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bd575bc060ce1610eb HTTP/1.1" 502 150 2025-12-17 23:19:35,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094bd575bc060ce1610eb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:36,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409496575bc060ce161087 HTTP/1.1" 502 150 2025-12-17 23:19:36,113 - lava_callback - ERROR - Error processing callback for node 69409496575bc060ce161087: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409496575bc060ce161087 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409496575bc060ce161087 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409496575bc060ce161087 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:36,115 - lava_callback - INFO - Starting async job submit for node 694094bf575bc060ce1610ee 2025-12-17 23:19:36,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:36,137 - lava_callback - INFO - Received callback for node 6940992d575bc060ce161c5a INFO: 172.21.0.1:42850 - "POST /node/6940992d575bc060ce161c5a HTTP/1.0" 202 Accepted 2025-12-17 23:19:36,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bf575bc060ce1610ee HTTP/1.1" 502 150 2025-12-17 23:19:36,288 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094bf575bc060ce1610ee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:36,289 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094bf575bc060ce1610ee 2025-12-17 23:19:36,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bf575bc060ce1610ee HTTP/1.1" 502 150 2025-12-17 23:19:36,289 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094bf575bc060ce1610ee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42862 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:19:36,525 - lava_callback - INFO - Received callback for node 69409774575bc060ce161829 INFO: 172.21.0.1:42870 - "POST /node/69409774575bc060ce161829 HTTP/1.0" 202 Accepted 2025-12-17 23:19:37,181 - lava_callback - INFO - Received callback for node 6940977a575bc060ce161832 INFO: 172.21.0.1:42882 - "POST /node/6940977a575bc060ce161832 HTTP/1.0" 202 Accepted 2025-12-17 23:19:37,860 - lava_callback - INFO - Received callback for node 6940977d575bc060ce161838 INFO: 172.21.0.1:42884 - "POST /node/6940977d575bc060ce161838 HTTP/1.0" 202 Accepted 2025-12-17 23:19:37,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940949f575bc060ce1610a8 2025-12-17 23:19:37,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940949f575bc060ce1610a8 HTTP/1.1" 502 150 2025-12-17 23:19:37,924 - lava_callback - ERROR - Error processing callback for node 6940949f575bc060ce1610a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940949f575bc060ce1610a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940949f575bc060ce1610a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940949f575bc060ce1610a8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:37,926 - lava_callback - INFO - Starting async job submit for node 694094b9575bc060ce1610e5 2025-12-17 23:19:37,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:37,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b9575bc060ce1610e5 HTTP/1.1" 502 150 2025-12-17 23:19:37,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b9575bc060ce1610e5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:37,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b9575bc060ce1610e5 2025-12-17 23:19:37,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b9575bc060ce1610e5 HTTP/1.1" 502 150 2025-12-17 23:19:37,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b9575bc060ce1610e5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:38,290 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094bf575bc060ce1610ee 2025-12-17 23:19:38,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bf575bc060ce1610ee HTTP/1.1" 502 150 2025-12-17 23:19:38,291 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094bf575bc060ce1610ee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:38,551 - lava_callback - INFO - Received callback for node 6940977e575bc060ce16183d INFO: 172.21.0.1:42888 - "POST /node/6940977e575bc060ce16183d HTTP/1.0" 202 Accepted 2025-12-17 23:19:38,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ac575bc060ce1610c6 2025-12-17 23:19:38,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ac575bc060ce1610c6 HTTP/1.1" 502 150 2025-12-17 23:19:38,876 - lava_callback - ERROR - Error processing callback for node 694094ac575bc060ce1610c6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ac575bc060ce1610c6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ac575bc060ce1610c6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ac575bc060ce1610c6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:38,878 - lava_callback - INFO - Starting async job submit for node 69409764575bc060ce1617f6 2025-12-17 23:19:38,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:38,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409764575bc060ce1617f6 HTTP/1.1" 502 150 2025-12-17 23:19:38,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409764575bc060ce1617f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:38,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409764575bc060ce1617f6 2025-12-17 23:19:38,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409764575bc060ce1617f6 HTTP/1.1" 502 150 2025-12-17 23:19:38,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409764575bc060ce1617f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:39,208 - lava_callback - INFO - Received callback for node 69409780575bc060ce161844 INFO: 172.21.0.1:42890 - "POST /node/69409780575bc060ce161844 HTTP/1.0" 202 Accepted 2025-12-17 23:19:39,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094bd575bc060ce1610eb 2025-12-17 23:19:39,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bd575bc060ce1610eb HTTP/1.1" 502 150 2025-12-17 23:19:39,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094bd575bc060ce1610eb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:39,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094af575bc060ce1610cf 2025-12-17 23:19:39,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094af575bc060ce1610cf HTTP/1.1" 502 150 2025-12-17 23:19:39,704 - lava_callback - ERROR - Error processing callback for node 694094af575bc060ce1610cf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094af575bc060ce1610cf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094af575bc060ce1610cf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094af575bc060ce1610cf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:39,706 - lava_callback - INFO - Starting async job submit for node 694094c1575bc060ce1610f5 2025-12-17 23:19:39,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:39,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c1575bc060ce1610f5 HTTP/1.1" 502 150 2025-12-17 23:19:39,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c1575bc060ce1610f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:39,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c1575bc060ce1610f5 2025-12-17 23:19:39,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c1575bc060ce1610f5 HTTP/1.1" 502 150 2025-12-17 23:19:39,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c1575bc060ce1610f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:39,910 - lava_callback - INFO - Received callback for node 69409782575bc060ce16184d INFO: 172.21.0.1:59070 - "POST /node/69409782575bc060ce16184d HTTP/1.0" 202 Accepted 2025-12-17 23:19:39,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b9575bc060ce1610e5 2025-12-17 23:19:39,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b9575bc060ce1610e5 HTTP/1.1" 502 150 2025-12-17 23:19:39,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b9575bc060ce1610e5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:40,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b1575bc060ce1610d9 2025-12-17 23:19:40,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b1575bc060ce1610d9 HTTP/1.1" 502 150 2025-12-17 23:19:40,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b1575bc060ce1610d9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:40,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a2575bc060ce1610b4 2025-12-17 23:19:40,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a2575bc060ce1610b4 HTTP/1.1" 502 150 2025-12-17 23:19:40,811 - lava_callback - ERROR - Error processing callback for node 694094a2575bc060ce1610b4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094a2575bc060ce1610b4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094a2575bc060ce1610b4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094a2575bc060ce1610b4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:40,812 - lava_callback - INFO - Starting async job submit for node 694094c3575bc060ce1610fe 2025-12-17 23:19:40,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:40,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c3575bc060ce1610fe HTTP/1.1" 502 150 2025-12-17 23:19:40,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c3575bc060ce1610fe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:40,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c3575bc060ce1610fe 2025-12-17 23:19:40,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c3575bc060ce1610fe HTTP/1.1" 502 150 2025-12-17 23:19:40,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c3575bc060ce1610fe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:40,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409764575bc060ce1617f6 2025-12-17 23:19:40,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409764575bc060ce1617f6 HTTP/1.1" 502 150 2025-12-17 23:19:40,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409764575bc060ce1617f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:41,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a5575bc060ce1610b7 2025-12-17 23:19:41,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a5575bc060ce1610b7 HTTP/1.1" 502 150 2025-12-17 23:19:41,453 - lava_callback - ERROR - Error processing callback for node 694094a5575bc060ce1610b7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094a5575bc060ce1610b7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094a5575bc060ce1610b7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094a5575bc060ce1610b7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:41,455 - lava_callback - INFO - Starting async job submit for node 694094c5575bc060ce161101 2025-12-17 23:19:41,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:41,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c5575bc060ce161101 HTTP/1.1" 502 150 2025-12-17 23:19:41,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c5575bc060ce161101'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:41,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c5575bc060ce161101 2025-12-17 23:19:41,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c5575bc060ce161101 HTTP/1.1" 502 150 2025-12-17 23:19:41,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c5575bc060ce161101'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:41,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c1575bc060ce1610f5 2025-12-17 23:19:41,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c1575bc060ce1610f5 HTTP/1.1" 502 150 2025-12-17 23:19:41,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c1575bc060ce1610f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:41,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b3575bc060ce1610dc 2025-12-17 23:19:41,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b3575bc060ce1610dc HTTP/1.1" 502 150 2025-12-17 23:19:41,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b3575bc060ce1610dc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:41,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a6575bc060ce1610ba 2025-12-17 23:19:41,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a6575bc060ce1610ba HTTP/1.1" 502 150 2025-12-17 23:19:41,865 - lava_callback - ERROR - Error processing callback for node 694094a6575bc060ce1610ba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094a6575bc060ce1610ba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094a6575bc060ce1610ba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094a6575bc060ce1610ba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:41,867 - lava_callback - INFO - Starting async job submit for node 694094c8575bc060ce161108 2025-12-17 23:19:41,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:41,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c8575bc060ce161108 HTTP/1.1" 502 150 2025-12-17 23:19:41,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c8575bc060ce161108'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:41,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c8575bc060ce161108 2025-12-17 23:19:41,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c8575bc060ce161108 HTTP/1.1" 502 150 2025-12-17 23:19:41,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c8575bc060ce161108'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:41,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094a9575bc060ce1610bd 2025-12-17 23:19:41,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094a9575bc060ce1610bd HTTP/1.1" 502 150 2025-12-17 23:19:41,955 - lava_callback - ERROR - Error processing callback for node 694094a9575bc060ce1610bd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094a9575bc060ce1610bd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094a9575bc060ce1610bd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094a9575bc060ce1610bd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:41,957 - lava_callback - INFO - Starting async job submit for node 694094ca575bc060ce16110b 2025-12-17 23:19:41,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:41,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ca575bc060ce16110b HTTP/1.1" 502 150 2025-12-17 23:19:41,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ca575bc060ce16110b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:41,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ca575bc060ce16110b 2025-12-17 23:19:41,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ca575bc060ce16110b HTTP/1.1" 502 150 2025-12-17 23:19:41,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ca575bc060ce16110b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:42,291 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094bf575bc060ce1610ee 2025-12-17 23:19:42,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bf575bc060ce1610ee HTTP/1.1" 502 150 2025-12-17 23:19:42,292 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094bf575bc060ce1610ee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:42,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b5575bc060ce1610df 2025-12-17 23:19:42,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b5575bc060ce1610df HTTP/1.1" 502 150 2025-12-17 23:19:42,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b5575bc060ce1610df'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:42,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b7575bc060ce1610e2 2025-12-17 23:19:42,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b7575bc060ce1610e2 HTTP/1.1" 502 150 2025-12-17 23:19:42,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b7575bc060ce1610e2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:42,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c3575bc060ce1610fe 2025-12-17 23:19:42,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c3575bc060ce1610fe HTTP/1.1" 502 150 2025-12-17 23:19:42,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c3575bc060ce1610fe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:43,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c5575bc060ce161101 2025-12-17 23:19:43,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c5575bc060ce161101 HTTP/1.1" 502 150 2025-12-17 23:19:43,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c5575bc060ce161101'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:43,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c8575bc060ce161108 2025-12-17 23:19:43,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c8575bc060ce161108 HTTP/1.1" 502 150 2025-12-17 23:19:43,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c8575bc060ce161108'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:43,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b9575bc060ce1610e5 2025-12-17 23:19:43,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b9575bc060ce1610e5 HTTP/1.1" 502 150 2025-12-17 23:19:43,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b9575bc060ce1610e5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:43,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094aa575bc060ce1610c0 2025-12-17 23:19:43,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094aa575bc060ce1610c0 HTTP/1.1" 502 150 2025-12-17 23:19:43,963 - lava_callback - ERROR - Error processing callback for node 694094aa575bc060ce1610c0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094aa575bc060ce1610c0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094aa575bc060ce1610c0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094aa575bc060ce1610c0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:43,966 - lava_callback - INFO - Starting async job submit for node 694094cf575bc060ce161118 2025-12-17 23:19:43,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:43,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ca575bc060ce16110b 2025-12-17 23:19:43,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ca575bc060ce16110b HTTP/1.1" 502 150 2025-12-17 23:19:43,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ca575bc060ce16110b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:43,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cf575bc060ce161118 HTTP/1.1" 502 150 2025-12-17 23:19:43,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094cf575bc060ce161118'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:43,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094cf575bc060ce161118 2025-12-17 23:19:43,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cf575bc060ce161118 HTTP/1.1" 502 150 2025-12-17 23:19:43,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094cf575bc060ce161118'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:44,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409764575bc060ce1617f6 2025-12-17 23:19:44,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409764575bc060ce1617f6 HTTP/1.1" 502 150 2025-12-17 23:19:44,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409764575bc060ce1617f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:44,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ab575bc060ce1610c3 2025-12-17 23:19:44,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ab575bc060ce1610c3 HTTP/1.1" 502 150 2025-12-17 23:19:44,979 - lava_callback - ERROR - Error processing callback for node 694094ab575bc060ce1610c3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ab575bc060ce1610c3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ab575bc060ce1610c3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ab575bc060ce1610c3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:44,981 - lava_callback - INFO - Starting async job submit for node 694094d2575bc060ce16111e 2025-12-17 23:19:44,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:45,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d2575bc060ce16111e HTTP/1.1" 502 150 2025-12-17 23:19:45,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d2575bc060ce16111e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:45,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d2575bc060ce16111e 2025-12-17 23:19:45,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d2575bc060ce16111e HTTP/1.1" 502 150 2025-12-17 23:19:45,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d2575bc060ce16111e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:45,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c1575bc060ce1610f5 2025-12-17 23:19:45,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c1575bc060ce1610f5 HTTP/1.1" 502 150 2025-12-17 23:19:45,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c1575bc060ce1610f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:45,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094af575bc060ce1610d4 2025-12-17 23:19:45,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094af575bc060ce1610d4 HTTP/1.1" 502 150 2025-12-17 23:19:45,797 - lava_callback - ERROR - Error processing callback for node 694094af575bc060ce1610d4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094af575bc060ce1610d4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094af575bc060ce1610d4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094af575bc060ce1610d4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:45,799 - lava_callback - INFO - Starting async job submit for node 694094d0575bc060ce16111b 2025-12-17 23:19:45,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:45,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d0575bc060ce16111b HTTP/1.1" 502 150 2025-12-17 23:19:45,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d0575bc060ce16111b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:45,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d0575bc060ce16111b 2025-12-17 23:19:45,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d0575bc060ce16111b HTTP/1.1" 502 150 2025-12-17 23:19:45,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d0575bc060ce16111b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:45,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094cf575bc060ce161118 2025-12-17 23:19:45,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cf575bc060ce161118 HTTP/1.1" 502 150 2025-12-17 23:19:45,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094cf575bc060ce161118'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59078 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:19:46,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c3575bc060ce1610fe 2025-12-17 23:19:46,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c3575bc060ce1610fe HTTP/1.1" 502 150 2025-12-17 23:19:46,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c3575bc060ce1610fe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:47,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d2575bc060ce16111e 2025-12-17 23:19:47,055 - lava_callback - INFO - Received callback for node 69409784575bc060ce161852 INFO: 172.21.0.1:59086 - "POST /node/69409784575bc060ce161852 HTTP/1.0" 202 Accepted 2025-12-17 23:19:47,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c5575bc060ce161101 2025-12-17 23:19:47,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c5575bc060ce161101 HTTP/1.1" 502 150 2025-12-17 23:19:47,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c5575bc060ce161101'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:47,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094bd575bc060ce1610eb 2025-12-17 23:19:47,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bd575bc060ce1610eb HTTP/1.1" 502 150 2025-12-17 23:19:47,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094bd575bc060ce1610eb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:47,705 - lava_callback - INFO - Received callback for node 69409786575bc060ce161859 INFO: 172.21.0.1:59096 - "POST /node/69409786575bc060ce161859 HTTP/1.0" 202 Accepted 2025-12-17 23:19:47,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d0575bc060ce16111b 2025-12-17 23:19:47,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d0575bc060ce16111b HTTP/1.1" 502 150 2025-12-17 23:19:47,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d0575bc060ce16111b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:47,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c8575bc060ce161108 2025-12-17 23:19:47,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c8575bc060ce161108 HTTP/1.1" 502 150 2025-12-17 23:19:47,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c8575bc060ce161108'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:47,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ca575bc060ce16110b 2025-12-17 23:19:47,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ca575bc060ce16110b HTTP/1.1" 502 150 2025-12-17 23:19:47,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ca575bc060ce16110b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:48,018 - lava_callback - INFO - Received callback for node 694122fc459ff5fc64d2bae1 INFO: 172.21.0.1:59102 - "POST /node/694122fc459ff5fc64d2bae1 HTTP/1.0" 202 Accepted 2025-12-17 23:19:48,351 - lava_callback - INFO - Received callback for node 6940978a575bc060ce161864 INFO: 172.21.0.1:59108 - "POST /node/6940978a575bc060ce161864 HTTP/1.0" 202 Accepted 2025-12-17 23:19:49,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094cf575bc060ce161118 2025-12-17 23:19:50,293 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094bf575bc060ce1610ee 2025-12-17 23:19:50,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bf575bc060ce1610ee HTTP/1.1" 502 150 2025-12-17 23:19:50,294 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094bf575bc060ce1610ee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:51,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d0575bc060ce16111b 2025-12-17 23:19:51,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d0575bc060ce16111b HTTP/1.1" 502 150 2025-12-17 23:19:51,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d0575bc060ce16111b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:51,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b9575bc060ce1610e5 2025-12-17 23:19:51,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b9575bc060ce1610e5 HTTP/1.1" 502 150 2025-12-17 23:19:51,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094b9575bc060ce1610e5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:52,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409764575bc060ce1617f6 2025-12-17 23:19:52,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409764575bc060ce1617f6 HTTP/1.1" 502 150 2025-12-17 23:19:52,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409764575bc060ce1617f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:53,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c1575bc060ce1610f5 2025-12-17 23:19:53,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c1575bc060ce1610f5 HTTP/1.1" 502 150 2025-12-17 23:19:53,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c1575bc060ce1610f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:54,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c3575bc060ce1610fe 2025-12-17 23:19:54,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c3575bc060ce1610fe HTTP/1.1" 502 150 2025-12-17 23:19:54,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c3575bc060ce1610fe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:55,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c5575bc060ce161101 2025-12-17 23:19:55,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c5575bc060ce161101 HTTP/1.1" 502 150 2025-12-17 23:19:55,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c5575bc060ce161101'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:55,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c8575bc060ce161108 2025-12-17 23:19:55,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c8575bc060ce161108 HTTP/1.1" 502 150 2025-12-17 23:19:55,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094c8575bc060ce161108'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:55,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ca575bc060ce16110b 2025-12-17 23:19:55,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ca575bc060ce16110b HTTP/1.1" 502 150 2025-12-17 23:19:55,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ca575bc060ce16110b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:56,526 - lava_callback - INFO - Received callback for node 69409789575bc060ce16185c INFO: 172.21.0.1:60620 - "POST /node/69409789575bc060ce16185c HTTP/1.0" 202 Accepted INFO: 172.21.0.1:60628 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:19:56,754 - lava_callback - INFO - Received callback for node 69409838575bc060ce161a1b INFO: 172.21.0.1:60642 - "POST /node/69409838575bc060ce161a1b HTTP/1.0" 202 Accepted 2025-12-17 23:19:56,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b1575bc060ce1610d9 2025-12-17 23:19:56,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b1575bc060ce1610d9 HTTP/1.1" 502 150 2025-12-17 23:19:56,801 - lava_callback - ERROR - Error processing callback for node 694094b1575bc060ce1610d9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094b1575bc060ce1610d9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094b1575bc060ce1610d9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094b1575bc060ce1610d9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:56,803 - lava_callback - INFO - Starting async job submit for node 694094d4575bc060ce161124 2025-12-17 23:19:56,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:56,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d4575bc060ce161124 HTTP/1.1" 502 150 2025-12-17 23:19:56,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d4575bc060ce161124'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:56,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d4575bc060ce161124 2025-12-17 23:19:56,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d4575bc060ce161124 HTTP/1.1" 502 150 2025-12-17 23:19:56,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d4575bc060ce161124'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:57,203 - lava_callback - INFO - Received callback for node 6940978d575bc060ce161867 INFO: 172.21.0.1:60656 - "POST /node/6940978d575bc060ce161867 HTTP/1.0" 202 Accepted 2025-12-17 23:19:57,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b3575bc060ce1610dc 2025-12-17 23:19:57,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b3575bc060ce1610dc HTTP/1.1" 502 150 2025-12-17 23:19:57,857 - lava_callback - ERROR - Error processing callback for node 694094b3575bc060ce1610dc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094b3575bc060ce1610dc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094b3575bc060ce1610dc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094b3575bc060ce1610dc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:57,860 - lava_callback - INFO - Starting async job submit for node 694094d3575bc060ce161121 2025-12-17 23:19:57,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:57,901 - lava_callback - INFO - Received callback for node 6940978f575bc060ce16186a INFO: 172.21.0.1:60672 - "POST /node/6940978f575bc060ce16186a HTTP/1.0" 202 Accepted 2025-12-17 23:19:57,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d3575bc060ce161121 HTTP/1.1" 502 150 2025-12-17 23:19:57,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d3575bc060ce161121'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:57,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d3575bc060ce161121 2025-12-17 23:19:57,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d3575bc060ce161121 HTTP/1.1" 502 150 2025-12-17 23:19:57,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d3575bc060ce161121'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:58,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b5575bc060ce1610df 2025-12-17 23:19:58,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b5575bc060ce1610df HTTP/1.1" 502 150 2025-12-17 23:19:58,404 - lava_callback - ERROR - Error processing callback for node 694094b5575bc060ce1610df: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094b5575bc060ce1610df (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094b5575bc060ce1610df (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094b5575bc060ce1610df (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:58,406 - lava_callback - INFO - Starting async job submit for node 694094cc575bc060ce161112 2025-12-17 23:19:58,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:58,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cc575bc060ce161112 HTTP/1.1" 502 150 2025-12-17 23:19:58,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094cc575bc060ce161112'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:58,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094cc575bc060ce161112 2025-12-17 23:19:58,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cc575bc060ce161112 HTTP/1.1" 502 150 2025-12-17 23:19:58,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094cc575bc060ce161112'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:58,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b7575bc060ce1610e2 2025-12-17 23:19:58,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b7575bc060ce1610e2 HTTP/1.1" 502 150 2025-12-17 23:19:58,583 - lava_callback - ERROR - Error processing callback for node 694094b7575bc060ce1610e2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094b7575bc060ce1610e2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094b7575bc060ce1610e2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094b7575bc060ce1610e2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:19:58,585 - lava_callback - INFO - Starting async job submit for node 694094d5575bc060ce16112d 2025-12-17 23:19:58,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:19:58,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d5575bc060ce16112d HTTP/1.1" 502 150 2025-12-17 23:19:58,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d5575bc060ce16112d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:58,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d5575bc060ce16112d 2025-12-17 23:19:58,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d5575bc060ce16112d HTTP/1.1" 502 150 2025-12-17 23:19:58,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d5575bc060ce16112d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:58,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d4575bc060ce161124 2025-12-17 23:19:58,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d4575bc060ce161124 HTTP/1.1" 502 150 2025-12-17 23:19:58,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d4575bc060ce161124'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:59,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d0575bc060ce16111b 2025-12-17 23:19:59,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d0575bc060ce16111b HTTP/1.1" 502 150 2025-12-17 23:19:59,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d0575bc060ce16111b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:19:59,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d3575bc060ce161121 2025-12-17 23:19:59,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d3575bc060ce161121 HTTP/1.1" 502 150 2025-12-17 23:19:59,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d3575bc060ce161121'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:00,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094cc575bc060ce161112 2025-12-17 23:20:00,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cc575bc060ce161112 HTTP/1.1" 502 150 2025-12-17 23:20:00,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094cc575bc060ce161112'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:00,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d5575bc060ce16112d 2025-12-17 23:20:00,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d5575bc060ce16112d HTTP/1.1" 502 150 2025-12-17 23:20:00,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d5575bc060ce16112d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:02,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d2575bc060ce16111e HTTP/1.1" 502 150 2025-12-17 23:20:02,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d2575bc060ce16111e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:02,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d4575bc060ce161124 2025-12-17 23:20:02,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d4575bc060ce161124 HTTP/1.1" 502 150 2025-12-17 23:20:02,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d4575bc060ce161124'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60592 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:20:03,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094bd575bc060ce1610eb 2025-12-17 23:20:03,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bd575bc060ce1610eb HTTP/1.1" 502 150 2025-12-17 23:20:03,597 - lava_callback - ERROR - Error processing callback for node 694094bd575bc060ce1610eb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094bd575bc060ce1610eb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094bd575bc060ce1610eb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094bd575bc060ce1610eb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:03,599 - lava_callback - INFO - Starting async job submit for node 694094d7575bc060ce161136 2025-12-17 23:20:03,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:03,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d7575bc060ce161136 HTTP/1.1" 502 150 2025-12-17 23:20:03,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d7575bc060ce161136'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:03,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d7575bc060ce161136 2025-12-17 23:20:03,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d7575bc060ce161136 HTTP/1.1" 502 150 2025-12-17 23:20:03,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d7575bc060ce161136'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:03,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d3575bc060ce161121 2025-12-17 23:20:03,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d3575bc060ce161121 HTTP/1.1" 502 150 2025-12-17 23:20:03,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d3575bc060ce161121'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:04,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094cc575bc060ce161112 2025-12-17 23:20:04,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d5575bc060ce16112d 2025-12-17 23:20:04,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d5575bc060ce16112d HTTP/1.1" 502 150 2025-12-17 23:20:04,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d5575bc060ce16112d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:05,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cf575bc060ce161118 HTTP/1.1" 502 150 2025-12-17 23:20:05,296 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094cf575bc060ce161118'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:05,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d7575bc060ce161136 2025-12-17 23:20:05,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d7575bc060ce161136 HTTP/1.1" 502 150 2025-12-17 23:20:05,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d7575bc060ce161136'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:06,294 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094bf575bc060ce1610ee 2025-12-17 23:20:06,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094bf575bc060ce1610ee HTTP/1.1" 502 150 2025-12-17 23:20:06,299 - lava_callback - ERROR - Error processing callback for node 694094bf575bc060ce1610ee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094bf575bc060ce1610ee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094bf575bc060ce1610ee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094bf575bc060ce1610ee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:06,301 - lava_callback - INFO - Starting async job submit for node 694094df575bc060ce161142 2025-12-17 23:20:06,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:06,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094df575bc060ce161142 HTTP/1.1" 502 150 2025-12-17 23:20:06,310 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094df575bc060ce161142'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:06,310 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094df575bc060ce161142 2025-12-17 23:20:06,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094df575bc060ce161142 HTTP/1.1" 502 150 2025-12-17 23:20:06,310 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094df575bc060ce161142'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:06,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d2575bc060ce16111e 2025-12-17 23:20:06,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d2575bc060ce16111e HTTP/1.1" 502 150 2025-12-17 23:20:06,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d2575bc060ce16111e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60596 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:20:07,129 - lava_callback - INFO - Received callback for node 69409793575bc060ce161872 INFO: 172.21.0.1:60602 - "POST /node/69409793575bc060ce161872 HTTP/1.0" 202 Accepted 2025-12-17 23:20:07,796 - lava_callback - INFO - Received callback for node 69409795575bc060ce16187d INFO: 172.21.0.1:60608 - "POST /node/69409795575bc060ce16187d HTTP/1.0" 202 Accepted 2025-12-17 23:20:07,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094b9575bc060ce1610e5 2025-12-17 23:20:07,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094b9575bc060ce1610e5 HTTP/1.1" 502 150 2025-12-17 23:20:07,964 - lava_callback - ERROR - Error processing callback for node 694094b9575bc060ce1610e5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094b9575bc060ce1610e5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094b9575bc060ce1610e5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094b9575bc060ce1610e5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:07,966 - lava_callback - INFO - Starting async job submit for node 694094d9575bc060ce161139 2025-12-17 23:20:07,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:07,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d9575bc060ce161139 HTTP/1.1" 502 150 2025-12-17 23:20:07,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d9575bc060ce161139'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:07,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d9575bc060ce161139 2025-12-17 23:20:07,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d9575bc060ce161139 HTTP/1.1" 502 150 2025-12-17 23:20:07,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d9575bc060ce161139'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:08,311 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094df575bc060ce161142 2025-12-17 23:20:08,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094df575bc060ce161142 HTTP/1.1" 502 150 2025-12-17 23:20:08,312 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094df575bc060ce161142'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:08,446 - lava_callback - INFO - Received callback for node 69409797575bc060ce161888 INFO: 172.21.0.1:60612 - "POST /node/69409797575bc060ce161888 HTTP/1.0" 202 Accepted 2025-12-17 23:20:08,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409764575bc060ce1617f6 2025-12-17 23:20:08,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409764575bc060ce1617f6 HTTP/1.1" 502 150 2025-12-17 23:20:08,907 - lava_callback - ERROR - Error processing callback for node 69409764575bc060ce1617f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409764575bc060ce1617f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409764575bc060ce1617f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409764575bc060ce1617f6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:08,909 - lava_callback - INFO - Starting async job submit for node 694094db575bc060ce16113c 2025-12-17 23:20:08,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:08,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094db575bc060ce16113c HTTP/1.1" 502 150 2025-12-17 23:20:08,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094db575bc060ce16113c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:08,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094db575bc060ce16113c 2025-12-17 23:20:08,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094db575bc060ce16113c HTTP/1.1" 502 150 2025-12-17 23:20:08,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094db575bc060ce16113c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:09,110 - lava_callback - INFO - Received callback for node 6940979a575bc060ce161894 INFO: 172.21.0.1:60616 - "POST /node/6940979a575bc060ce161894 HTTP/1.0" 202 Accepted 2025-12-17 23:20:09,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d7575bc060ce161136 2025-12-17 23:20:09,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d7575bc060ce161136 HTTP/1.1" 502 150 2025-12-17 23:20:09,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d7575bc060ce161136'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:09,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c1575bc060ce1610f5 2025-12-17 23:20:09,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c1575bc060ce1610f5 HTTP/1.1" 502 150 2025-12-17 23:20:09,721 - lava_callback - ERROR - Error processing callback for node 694094c1575bc060ce1610f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094c1575bc060ce1610f5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094c1575bc060ce1610f5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094c1575bc060ce1610f5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:09,723 - lava_callback - INFO - Starting async job submit for node 694094dd575bc060ce16113f 2025-12-17 23:20:09,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:09,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094dd575bc060ce16113f HTTP/1.1" 502 150 2025-12-17 23:20:09,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094dd575bc060ce16113f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:09,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094dd575bc060ce16113f 2025-12-17 23:20:09,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094dd575bc060ce16113f HTTP/1.1" 502 150 2025-12-17 23:20:09,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094dd575bc060ce16113f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:09,767 - lava_callback - INFO - Received callback for node 6940979c575bc060ce16189b INFO: 172.21.0.1:36688 - "POST /node/6940979c575bc060ce16189b HTTP/1.0" 202 Accepted 2025-12-17 23:20:09,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d9575bc060ce161139 2025-12-17 23:20:09,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d9575bc060ce161139 HTTP/1.1" 502 150 2025-12-17 23:20:09,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d9575bc060ce161139'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:10,409 - lava_callback - INFO - Received callback for node 6940979e575bc060ce1618a7 INFO: 172.21.0.1:36702 - "POST /node/6940979e575bc060ce1618a7 HTTP/1.0" 202 Accepted 2025-12-17 23:20:10,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d4575bc060ce161124 2025-12-17 23:20:10,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d4575bc060ce161124 HTTP/1.1" 502 150 2025-12-17 23:20:10,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d4575bc060ce161124'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:10,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c3575bc060ce1610fe 2025-12-17 23:20:10,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c3575bc060ce1610fe HTTP/1.1" 502 150 2025-12-17 23:20:10,826 - lava_callback - ERROR - Error processing callback for node 694094c3575bc060ce1610fe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094c3575bc060ce1610fe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094c3575bc060ce1610fe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094c3575bc060ce1610fe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:10,827 - lava_callback - INFO - Starting async job submit for node 6940998e575bc060ce161d3c 2025-12-17 23:20:10,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:10,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998e575bc060ce161d3c HTTP/1.1" 502 150 2025-12-17 23:20:10,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940998e575bc060ce161d3c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:10,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940998e575bc060ce161d3c 2025-12-17 23:20:10,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998e575bc060ce161d3c HTTP/1.1" 502 150 2025-12-17 23:20:10,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940998e575bc060ce161d3c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:10,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094db575bc060ce16113c 2025-12-17 23:20:10,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094db575bc060ce16113c HTTP/1.1" 502 150 2025-12-17 23:20:10,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094db575bc060ce16113c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:11,072 - lava_callback - INFO - Received callback for node 694097a0575bc060ce1618aa INFO: 172.21.0.1:36708 - "POST /node/694097a0575bc060ce1618aa HTTP/1.0" 202 Accepted 2025-12-17 23:20:11,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c5575bc060ce161101 2025-12-17 23:20:11,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c5575bc060ce161101 HTTP/1.1" 502 150 2025-12-17 23:20:11,473 - lava_callback - ERROR - Error processing callback for node 694094c5575bc060ce161101: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094c5575bc060ce161101 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094c5575bc060ce161101 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094c5575bc060ce161101 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:11,475 - lava_callback - INFO - Starting async job submit for node 694097a2575bc060ce1618ad 2025-12-17 23:20:11,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:11,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a2575bc060ce1618ad HTTP/1.1" 502 150 2025-12-17 23:20:11,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a2575bc060ce1618ad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:11,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a2575bc060ce1618ad 2025-12-17 23:20:11,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a2575bc060ce1618ad HTTP/1.1" 502 150 2025-12-17 23:20:11,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a2575bc060ce1618ad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:11,720 - lava_callback - INFO - Received callback for node 694097a4575bc060ce1618b0 INFO: 172.21.0.1:36716 - "POST /node/694097a4575bc060ce1618b0 HTTP/1.0" 202 Accepted 2025-12-17 23:20:11,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094dd575bc060ce16113f 2025-12-17 23:20:11,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094dd575bc060ce16113f HTTP/1.1" 502 150 2025-12-17 23:20:11,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094dd575bc060ce16113f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:11,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094c8575bc060ce161108 2025-12-17 23:20:11,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094c8575bc060ce161108 HTTP/1.1" 502 150 2025-12-17 23:20:11,880 - lava_callback - ERROR - Error processing callback for node 694094c8575bc060ce161108: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094c8575bc060ce161108 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094c8575bc060ce161108 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094c8575bc060ce161108 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:11,882 - lava_callback - INFO - Starting async job submit for node 694094e0575bc060ce161145 2025-12-17 23:20:11,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:11,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e0575bc060ce161145 HTTP/1.1" 502 150 2025-12-17 23:20:11,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e0575bc060ce161145'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:11,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e0575bc060ce161145 2025-12-17 23:20:11,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e0575bc060ce161145 HTTP/1.1" 502 150 2025-12-17 23:20:11,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e0575bc060ce161145'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:11,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d3575bc060ce161121 2025-12-17 23:20:11,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d3575bc060ce161121 HTTP/1.1" 502 150 2025-12-17 23:20:11,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d3575bc060ce161121'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:11,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ca575bc060ce16110b 2025-12-17 23:20:11,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ca575bc060ce16110b HTTP/1.1" 502 150 2025-12-17 23:20:11,972 - lava_callback - ERROR - Error processing callback for node 694094ca575bc060ce16110b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ca575bc060ce16110b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ca575bc060ce16110b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ca575bc060ce16110b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:11,974 - lava_callback - INFO - Starting async job submit for node 694094e2575bc060ce161148 2025-12-17 23:20:11,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:12,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e2575bc060ce161148 HTTP/1.1" 502 150 2025-12-17 23:20:12,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e2575bc060ce161148'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:12,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e2575bc060ce161148 2025-12-17 23:20:12,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e2575bc060ce161148 HTTP/1.1" 502 150 2025-12-17 23:20:12,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e2575bc060ce161148'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:12,312 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094df575bc060ce161142 2025-12-17 23:20:12,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094df575bc060ce161142 HTTP/1.1" 502 150 2025-12-17 23:20:12,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094df575bc060ce161142'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:12,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d5575bc060ce16112d 2025-12-17 23:20:12,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d5575bc060ce16112d HTTP/1.1" 502 150 2025-12-17 23:20:12,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d5575bc060ce16112d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:12,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940998e575bc060ce161d3c 2025-12-17 23:20:12,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998e575bc060ce161d3c HTTP/1.1" 502 150 2025-12-17 23:20:12,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940998e575bc060ce161d3c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:13,297 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094cf575bc060ce161118 2025-12-17 23:20:13,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a2575bc060ce1618ad 2025-12-17 23:20:13,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a2575bc060ce1618ad HTTP/1.1" 502 150 2025-12-17 23:20:13,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a2575bc060ce1618ad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:13,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e0575bc060ce161145 2025-12-17 23:20:13,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e0575bc060ce161145 HTTP/1.1" 502 150 2025-12-17 23:20:13,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e0575bc060ce161145'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:13,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d9575bc060ce161139 2025-12-17 23:20:13,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d9575bc060ce161139 HTTP/1.1" 502 150 2025-12-17 23:20:13,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d9575bc060ce161139'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:14,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e2575bc060ce161148 2025-12-17 23:20:14,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e2575bc060ce161148 HTTP/1.1" 502 150 2025-12-17 23:20:14,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e2575bc060ce161148'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:14,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d2575bc060ce16111e 2025-12-17 23:20:14,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d2575bc060ce16111e HTTP/1.1" 502 150 2025-12-17 23:20:14,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d2575bc060ce16111e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:14,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094db575bc060ce16113c 2025-12-17 23:20:14,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094db575bc060ce16113c HTTP/1.1" 502 150 2025-12-17 23:20:14,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094db575bc060ce16113c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36732 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:20:15,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094dd575bc060ce16113f 2025-12-17 23:20:15,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094dd575bc060ce16113f HTTP/1.1" 502 150 2025-12-17 23:20:15,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094dd575bc060ce16113f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:15,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d0575bc060ce16111b 2025-12-17 23:20:15,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d0575bc060ce16111b HTTP/1.1" 502 150 2025-12-17 23:20:15,816 - lava_callback - ERROR - Error processing callback for node 694094d0575bc060ce16111b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d0575bc060ce16111b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d0575bc060ce16111b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d0575bc060ce16111b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:15,821 - lava_callback - INFO - Starting async job submit for node 694094e3575bc060ce16114b 2025-12-17 23:20:15,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:15,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e3575bc060ce16114b HTTP/1.1" 502 150 2025-12-17 23:20:15,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e3575bc060ce16114b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:15,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e3575bc060ce16114b 2025-12-17 23:20:15,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e3575bc060ce16114b HTTP/1.1" 502 150 2025-12-17 23:20:15,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e3575bc060ce16114b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36736 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:20:16,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940998e575bc060ce161d3c 2025-12-17 23:20:16,883 - lava_callback - INFO - Received callback for node 694097a8575bc060ce1618be INFO: 172.21.0.1:36740 - "POST /node/694097a8575bc060ce1618be HTTP/1.0" 202 Accepted 2025-12-17 23:20:17,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a2575bc060ce1618ad 2025-12-17 23:20:17,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a2575bc060ce1618ad HTTP/1.1" 502 150 2025-12-17 23:20:17,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a2575bc060ce1618ad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:17,578 - lava_callback - INFO - Received callback for node 694097ab575bc060ce1618c1 INFO: 172.21.0.1:36754 - "POST /node/694097ab575bc060ce1618c1 HTTP/1.0" 202 Accepted 2025-12-17 23:20:17,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d7575bc060ce161136 2025-12-17 23:20:17,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d7575bc060ce161136 HTTP/1.1" 502 150 2025-12-17 23:20:17,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d7575bc060ce161136'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:17,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e3575bc060ce16114b 2025-12-17 23:20:17,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e3575bc060ce16114b HTTP/1.1" 502 150 2025-12-17 23:20:17,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e3575bc060ce16114b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:17,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e0575bc060ce161145 2025-12-17 23:20:17,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e0575bc060ce161145 HTTP/1.1" 502 150 2025-12-17 23:20:17,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e0575bc060ce161145'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:18,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e2575bc060ce161148 2025-12-17 23:20:18,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e2575bc060ce161148 HTTP/1.1" 502 150 2025-12-17 23:20:18,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e2575bc060ce161148'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:18,264 - lava_callback - INFO - Received callback for node 694097ad575bc060ce1618c4 INFO: 172.21.0.1:36766 - "POST /node/694097ad575bc060ce1618c4 HTTP/1.0" 202 Accepted 2025-12-17 23:20:18,917 - lava_callback - INFO - Received callback for node 694097af575bc060ce1618c7 INFO: 172.21.0.1:36780 - "POST /node/694097af575bc060ce1618c7 HTTP/1.0" 202 Accepted 2025-12-17 23:20:19,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cc575bc060ce161112 HTTP/1.1" 502 150 2025-12-17 23:20:19,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094cc575bc060ce161112'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:20,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094df575bc060ce161142 2025-12-17 23:20:20,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094df575bc060ce161142 HTTP/1.1" 502 150 2025-12-17 23:20:20,315 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094df575bc060ce161142'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:21,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e3575bc060ce16114b 2025-12-17 23:20:21,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e3575bc060ce16114b HTTP/1.1" 502 150 2025-12-17 23:20:21,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e3575bc060ce16114b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:21,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d9575bc060ce161139 2025-12-17 23:20:21,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d9575bc060ce161139 HTTP/1.1" 502 150 2025-12-17 23:20:21,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094d9575bc060ce161139'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:22,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094db575bc060ce16113c 2025-12-17 23:20:22,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094db575bc060ce16113c HTTP/1.1" 502 150 2025-12-17 23:20:22,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094db575bc060ce16113c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:23,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094dd575bc060ce16113f 2025-12-17 23:20:23,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094dd575bc060ce16113f HTTP/1.1" 502 150 2025-12-17 23:20:23,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094dd575bc060ce16113f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:25,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a2575bc060ce1618ad 2025-12-17 23:20:25,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e0575bc060ce161145 2025-12-17 23:20:25,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e0575bc060ce161145 HTTP/1.1" 502 150 2025-12-17 23:20:25,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e0575bc060ce161145'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:26,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e2575bc060ce161148 2025-12-17 23:20:26,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e2575bc060ce161148 HTTP/1.1" 502 150 2025-12-17 23:20:26,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e2575bc060ce161148'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38336 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:20:26,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d4575bc060ce161124 2025-12-17 23:20:26,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d4575bc060ce161124 HTTP/1.1" 502 150 2025-12-17 23:20:26,817 - lava_callback - ERROR - Error processing callback for node 694094d4575bc060ce161124: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d4575bc060ce161124 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d4575bc060ce161124 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d4575bc060ce161124 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:26,819 - lava_callback - INFO - Starting async job submit for node 694094e4575bc060ce16114e 2025-12-17 23:20:26,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:26,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e4575bc060ce16114e HTTP/1.1" 502 150 2025-12-17 23:20:26,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e4575bc060ce16114e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:26,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e4575bc060ce16114e 2025-12-17 23:20:26,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e4575bc060ce16114e HTTP/1.1" 502 150 2025-12-17 23:20:26,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e4575bc060ce16114e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:27,109 - lava_callback - INFO - Received callback for node 694097a6575bc060ce1618b7 INFO: 172.21.0.1:38346 - "POST /node/694097a6575bc060ce1618b7 HTTP/1.0" 202 Accepted 2025-12-17 23:20:27,744 - lava_callback - INFO - Received callback for node 694097b3575bc060ce1618cd INFO: 172.21.0.1:38356 - "POST /node/694097b3575bc060ce1618cd HTTP/1.0" 202 Accepted 2025-12-17 23:20:27,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094cc575bc060ce161112 2025-12-17 23:20:27,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cc575bc060ce161112 HTTP/1.1" 502 150 2025-12-17 23:20:27,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094cc575bc060ce161112'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:27,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d3575bc060ce161121 2025-12-17 23:20:27,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d3575bc060ce161121 HTTP/1.1" 502 150 2025-12-17 23:20:27,924 - lava_callback - ERROR - Error processing callback for node 694094d3575bc060ce161121: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d3575bc060ce161121 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d3575bc060ce161121 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d3575bc060ce161121 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:27,926 - lava_callback - INFO - Starting async job submit for node 694094e6575bc060ce161151 2025-12-17 23:20:27,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:27,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e6575bc060ce161151 HTTP/1.1" 502 150 2025-12-17 23:20:27,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e6575bc060ce161151'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:27,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e6575bc060ce161151 2025-12-17 23:20:27,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e6575bc060ce161151 HTTP/1.1" 502 150 2025-12-17 23:20:27,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e6575bc060ce161151'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:28,421 - lava_callback - INFO - Received callback for node 694097b5575bc060ce1618d0 INFO: 172.21.0.1:38372 - "POST /node/694097b5575bc060ce1618d0 HTTP/1.0" 202 Accepted 2025-12-17 23:20:28,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cf575bc060ce161118 HTTP/1.1" 502 150 2025-12-17 23:20:28,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094cf575bc060ce161118'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:28,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d5575bc060ce16112d 2025-12-17 23:20:28,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d5575bc060ce16112d HTTP/1.1" 502 150 2025-12-17 23:20:28,601 - lava_callback - ERROR - Error processing callback for node 694094d5575bc060ce16112d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d5575bc060ce16112d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d5575bc060ce16112d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d5575bc060ce16112d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:28,603 - lava_callback - INFO - Starting async job submit for node 694094e7575bc060ce161154 2025-12-17 23:20:28,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:28,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e7575bc060ce161154 HTTP/1.1" 502 150 2025-12-17 23:20:28,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e7575bc060ce161154'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:28,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e7575bc060ce161154 2025-12-17 23:20:28,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e7575bc060ce161154 HTTP/1.1" 502 150 2025-12-17 23:20:28,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e7575bc060ce161154'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:28,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e4575bc060ce16114e 2025-12-17 23:20:28,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e4575bc060ce16114e HTTP/1.1" 502 150 2025-12-17 23:20:28,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e4575bc060ce16114e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:29,081 - lava_callback - INFO - Received callback for node 694097b9575bc060ce1618d6 INFO: 172.21.0.1:38384 - "POST /node/694097b9575bc060ce1618d6 HTTP/1.0" 202 Accepted 2025-12-17 23:20:29,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e3575bc060ce16114b 2025-12-17 23:20:29,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e3575bc060ce16114b HTTP/1.1" 502 150 2025-12-17 23:20:29,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e3575bc060ce16114b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:29,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e6575bc060ce161151 2025-12-17 23:20:29,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e6575bc060ce161151 HTTP/1.1" 502 150 2025-12-17 23:20:29,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e6575bc060ce161151'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:30,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d2575bc060ce16111e 2025-12-17 23:20:30,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d2575bc060ce16111e HTTP/1.1" 502 150 2025-12-17 23:20:30,486 - lava_callback - ERROR - Error processing callback for node 694094d2575bc060ce16111e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d2575bc060ce16111e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d2575bc060ce16111e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d2575bc060ce16111e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:30,488 - lava_callback - INFO - Starting async job submit for node 694094e8575bc060ce161157 2025-12-17 23:20:30,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:30,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e8575bc060ce161157 HTTP/1.1" 502 150 2025-12-17 23:20:30,495 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e8575bc060ce161157'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:30,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e8575bc060ce161157 2025-12-17 23:20:30,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e8575bc060ce161157 HTTP/1.1" 502 150 2025-12-17 23:20:30,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e8575bc060ce161157'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:30,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e7575bc060ce161154 2025-12-17 23:20:30,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e7575bc060ce161154 HTTP/1.1" 502 150 2025-12-17 23:20:30,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e7575bc060ce161154'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:32,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998e575bc060ce161d3c HTTP/1.1" 502 150 2025-12-17 23:20:32,176 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940998e575bc060ce161d3c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:32,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e8575bc060ce161157 2025-12-17 23:20:32,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e8575bc060ce161157 HTTP/1.1" 502 150 2025-12-17 23:20:32,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e8575bc060ce161157'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:32,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e4575bc060ce16114e 2025-12-17 23:20:32,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e4575bc060ce16114e HTTP/1.1" 502 150 2025-12-17 23:20:32,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e4575bc060ce16114e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:33,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d7575bc060ce161136 2025-12-17 23:20:33,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d7575bc060ce161136 HTTP/1.1" 502 150 2025-12-17 23:20:33,620 - lava_callback - ERROR - Error processing callback for node 694094d7575bc060ce161136: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d7575bc060ce161136 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d7575bc060ce161136 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d7575bc060ce161136 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:33,623 - lava_callback - INFO - Starting async job submit for node 694094ea575bc060ce161160 2025-12-17 23:20:33,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:33,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ea575bc060ce161160 HTTP/1.1" 502 150 2025-12-17 23:20:33,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ea575bc060ce161160'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:33,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ea575bc060ce161160 2025-12-17 23:20:33,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ea575bc060ce161160 HTTP/1.1" 502 150 2025-12-17 23:20:33,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ea575bc060ce161160'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:33,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e6575bc060ce161151 2025-12-17 23:20:33,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e6575bc060ce161151 HTTP/1.1" 502 150 2025-12-17 23:20:33,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e6575bc060ce161151'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:34,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e7575bc060ce161154 2025-12-17 23:20:34,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e7575bc060ce161154 HTTP/1.1" 502 150 2025-12-17 23:20:34,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e7575bc060ce161154'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:35,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ea575bc060ce161160 2025-12-17 23:20:35,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ea575bc060ce161160 HTTP/1.1" 502 150 2025-12-17 23:20:35,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ea575bc060ce161160'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:36,315 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094df575bc060ce161142 2025-12-17 23:20:36,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094df575bc060ce161142 HTTP/1.1" 502 150 2025-12-17 23:20:36,316 - lava_callback - ERROR - Error processing callback for node 694094df575bc060ce161142: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094df575bc060ce161142 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094df575bc060ce161142 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094df575bc060ce161142 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:36,318 - lava_callback - INFO - Starting async job submit for node 694094eb575bc060ce161165 2025-12-17 23:20:36,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:36,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094eb575bc060ce161165 HTTP/1.1" 502 150 2025-12-17 23:20:36,325 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094eb575bc060ce161165'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:36,325 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094eb575bc060ce161165 2025-12-17 23:20:36,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094eb575bc060ce161165 HTTP/1.1" 502 150 2025-12-17 23:20:36,326 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094eb575bc060ce161165'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:36,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e8575bc060ce161157 2025-12-17 23:20:36,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e8575bc060ce161157 HTTP/1.1" 502 150 2025-12-17 23:20:36,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e8575bc060ce161157'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34068 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:20:36,557 - lava_callback - INFO - Received callback for node 694097b1575bc060ce1618ca INFO: 172.21.0.1:34070 - "POST /node/694097b1575bc060ce1618ca HTTP/1.0" 202 Accepted 2025-12-17 23:20:37,213 - lava_callback - INFO - Received callback for node 694097b7575bc060ce1618d3 INFO: 172.21.0.1:34080 - "POST /node/694097b7575bc060ce1618d3 HTTP/1.0" 202 Accepted 2025-12-17 23:20:37,874 - lava_callback - INFO - Received callback for node 694097bb575bc060ce1618dd INFO: 172.21.0.1:34088 - "POST /node/694097bb575bc060ce1618dd HTTP/1.0" 202 Accepted 2025-12-17 23:20:37,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094d9575bc060ce161139 2025-12-17 23:20:37,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094d9575bc060ce161139 HTTP/1.1" 502 150 2025-12-17 23:20:37,985 - lava_callback - ERROR - Error processing callback for node 694094d9575bc060ce161139: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d9575bc060ce161139 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d9575bc060ce161139 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094d9575bc060ce161139 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:37,986 - lava_callback - INFO - Starting async job submit for node 694094ec575bc060ce161168 2025-12-17 23:20:37,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:38,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ec575bc060ce161168 HTTP/1.1" 502 150 2025-12-17 23:20:38,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ec575bc060ce161168'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:38,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ec575bc060ce161168 2025-12-17 23:20:38,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ec575bc060ce161168 HTTP/1.1" 502 150 2025-12-17 23:20:38,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ec575bc060ce161168'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:38,326 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094eb575bc060ce161165 2025-12-17 23:20:38,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094eb575bc060ce161165 HTTP/1.1" 502 150 2025-12-17 23:20:38,328 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094eb575bc060ce161165'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:38,544 - lava_callback - INFO - Received callback for node 694097bc575bc060ce1618e0 INFO: 172.21.0.1:34100 - "POST /node/694097bc575bc060ce1618e0 HTTP/1.0" 202 Accepted 2025-12-17 23:20:38,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094db575bc060ce16113c 2025-12-17 23:20:38,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094db575bc060ce16113c HTTP/1.1" 502 150 2025-12-17 23:20:38,946 - lava_callback - ERROR - Error processing callback for node 694094db575bc060ce16113c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094db575bc060ce16113c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094db575bc060ce16113c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094db575bc060ce16113c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:38,949 - lava_callback - INFO - Starting async job submit for node 694094ef575bc060ce16116b 2025-12-17 23:20:38,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:38,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ef575bc060ce16116b HTTP/1.1" 502 150 2025-12-17 23:20:38,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ef575bc060ce16116b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:38,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ef575bc060ce16116b 2025-12-17 23:20:38,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ef575bc060ce16116b HTTP/1.1" 502 150 2025-12-17 23:20:38,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ef575bc060ce16116b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:39,210 - lava_callback - INFO - Received callback for node 694097be575bc060ce1618e3 INFO: 172.21.0.1:34102 - "POST /node/694097be575bc060ce1618e3 HTTP/1.0" 202 Accepted 2025-12-17 23:20:39,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ea575bc060ce161160 2025-12-17 23:20:39,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ea575bc060ce161160 HTTP/1.1" 502 150 2025-12-17 23:20:39,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ea575bc060ce161160'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:39,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094dd575bc060ce16113f 2025-12-17 23:20:39,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094dd575bc060ce16113f HTTP/1.1" 502 150 2025-12-17 23:20:39,738 - lava_callback - ERROR - Error processing callback for node 694094dd575bc060ce16113f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094dd575bc060ce16113f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094dd575bc060ce16113f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094dd575bc060ce16113f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:39,740 - lava_callback - INFO - Starting async job submit for node 694094f7575bc060ce1611d9 2025-12-17 23:20:39,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:39,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f7575bc060ce1611d9 HTTP/1.1" 502 150 2025-12-17 23:20:39,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f7575bc060ce1611d9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:39,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f7575bc060ce1611d9 2025-12-17 23:20:39,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f7575bc060ce1611d9 HTTP/1.1" 502 150 2025-12-17 23:20:39,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f7575bc060ce1611d9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:39,871 - lava_callback - INFO - Received callback for node 694097bf575bc060ce1618e6 INFO: 172.21.0.1:52218 - "POST /node/694097bf575bc060ce1618e6 HTTP/1.0" 202 Accepted 2025-12-17 23:20:40,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ec575bc060ce161168 2025-12-17 23:20:40,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ec575bc060ce161168 HTTP/1.1" 502 150 2025-12-17 23:20:40,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ec575bc060ce161168'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:40,176 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940998e575bc060ce161d3c 2025-12-17 23:20:40,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998e575bc060ce161d3c HTTP/1.1" 502 150 2025-12-17 23:20:40,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940998e575bc060ce161d3c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:40,542 - lava_callback - INFO - Received callback for node 694097c1575bc060ce1618ec INFO: 172.21.0.1:52230 - "POST /node/694097c1575bc060ce1618ec HTTP/1.0" 202 Accepted 2025-12-17 23:20:40,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e4575bc060ce16114e 2025-12-17 23:20:40,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e4575bc060ce16114e HTTP/1.1" 502 150 2025-12-17 23:20:40,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e4575bc060ce16114e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:40,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a2575bc060ce1618ad HTTP/1.1" 502 150 2025-12-17 23:20:40,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a2575bc060ce1618ad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:40,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ef575bc060ce16116b 2025-12-17 23:20:40,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ef575bc060ce16116b HTTP/1.1" 502 150 2025-12-17 23:20:40,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ef575bc060ce16116b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:41,215 - lava_callback - INFO - Received callback for node 694097c3575bc060ce1618ef INFO: 172.21.0.1:52240 - "POST /node/694097c3575bc060ce1618ef HTTP/1.0" 202 Accepted 2025-12-17 23:20:41,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f7575bc060ce1611d9 2025-12-17 23:20:41,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f7575bc060ce1611d9 HTTP/1.1" 502 150 2025-12-17 23:20:41,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f7575bc060ce1611d9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:41,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e0575bc060ce161145 2025-12-17 23:20:41,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e0575bc060ce161145 HTTP/1.1" 502 150 2025-12-17 23:20:41,900 - lava_callback - ERROR - Error processing callback for node 694094e0575bc060ce161145: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e0575bc060ce161145 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e0575bc060ce161145 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e0575bc060ce161145 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:41,902 - lava_callback - INFO - Starting async job submit for node 694094f1575bc060ce16116e 2025-12-17 23:20:41,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:41,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f1575bc060ce16116e HTTP/1.1" 502 150 2025-12-17 23:20:41,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f1575bc060ce16116e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:41,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f1575bc060ce16116e 2025-12-17 23:20:41,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f1575bc060ce16116e HTTP/1.1" 502 150 2025-12-17 23:20:41,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f1575bc060ce16116e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:41,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e6575bc060ce161151 2025-12-17 23:20:41,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e6575bc060ce161151 HTTP/1.1" 502 150 2025-12-17 23:20:41,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e6575bc060ce161151'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:42,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e2575bc060ce161148 2025-12-17 23:20:42,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e2575bc060ce161148 HTTP/1.1" 502 150 2025-12-17 23:20:42,008 - lava_callback - ERROR - Error processing callback for node 694094e2575bc060ce161148: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e2575bc060ce161148 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e2575bc060ce161148 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e2575bc060ce161148 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:42,009 - lava_callback - INFO - Starting async job submit for node 694094f3575bc060ce161171 2025-12-17 23:20:42,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:42,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f3575bc060ce161171 HTTP/1.1" 502 150 2025-12-17 23:20:42,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f3575bc060ce161171'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:42,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f3575bc060ce161171 2025-12-17 23:20:42,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f3575bc060ce161171 HTTP/1.1" 502 150 2025-12-17 23:20:42,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f3575bc060ce161171'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:42,328 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094eb575bc060ce161165 2025-12-17 23:20:42,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094eb575bc060ce161165 HTTP/1.1" 502 150 2025-12-17 23:20:42,329 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094eb575bc060ce161165'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:42,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e7575bc060ce161154 2025-12-17 23:20:42,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e7575bc060ce161154 HTTP/1.1" 502 150 2025-12-17 23:20:42,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e7575bc060ce161154'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:43,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094cc575bc060ce161112 2025-12-17 23:20:43,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cc575bc060ce161112 HTTP/1.1" 502 150 2025-12-17 23:20:43,891 - lava_callback - ERROR - Error processing callback for node 694094cc575bc060ce161112: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094cc575bc060ce161112 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094cc575bc060ce161112 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094cc575bc060ce161112 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:43,893 - lava_callback - INFO - Starting async job submit for node 694094f5575bc060ce1611d6 2025-12-17 23:20:43,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:43,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f5575bc060ce1611d6 HTTP/1.1" 502 150 2025-12-17 23:20:43,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f5575bc060ce1611d6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:43,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f5575bc060ce1611d6 2025-12-17 23:20:43,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f5575bc060ce1611d6 HTTP/1.1" 502 150 2025-12-17 23:20:43,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f5575bc060ce1611d6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:43,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f1575bc060ce16116e 2025-12-17 23:20:43,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f1575bc060ce16116e HTTP/1.1" 502 150 2025-12-17 23:20:43,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f1575bc060ce16116e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:44,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ec575bc060ce161168 2025-12-17 23:20:44,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ec575bc060ce161168 HTTP/1.1" 502 150 2025-12-17 23:20:44,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ec575bc060ce161168'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:44,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f3575bc060ce161171 2025-12-17 23:20:44,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f3575bc060ce161171 HTTP/1.1" 502 150 2025-12-17 23:20:44,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f3575bc060ce161171'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:44,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e8575bc060ce161157 2025-12-17 23:20:44,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e8575bc060ce161157 HTTP/1.1" 502 150 2025-12-17 23:20:44,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094e8575bc060ce161157'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:44,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094cf575bc060ce161118 2025-12-17 23:20:44,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094cf575bc060ce161118 HTTP/1.1" 502 150 2025-12-17 23:20:44,604 - lava_callback - ERROR - Error processing callback for node 694094cf575bc060ce161118: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094cf575bc060ce161118 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094cf575bc060ce161118 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094cf575bc060ce161118 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:44,606 - lava_callback - INFO - Starting async job submit for node 69411958459ff5fc64d2a0ce 2025-12-17 23:20:44,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:44,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411958459ff5fc64d2a0ce HTTP/1.1" 502 150 2025-12-17 23:20:44,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69411958459ff5fc64d2a0ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:44,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69411958459ff5fc64d2a0ce 2025-12-17 23:20:44,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411958459ff5fc64d2a0ce HTTP/1.1" 502 150 2025-12-17 23:20:44,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69411958459ff5fc64d2a0ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:44,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ef575bc060ce16116b 2025-12-17 23:20:44,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ef575bc060ce16116b HTTP/1.1" 502 150 2025-12-17 23:20:44,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ef575bc060ce16116b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:45,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f7575bc060ce1611d9 2025-12-17 23:20:45,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f7575bc060ce1611d9 HTTP/1.1" 502 150 2025-12-17 23:20:45,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f7575bc060ce1611d9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:45,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e3575bc060ce16114b 2025-12-17 23:20:45,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e3575bc060ce16114b HTTP/1.1" 502 150 2025-12-17 23:20:45,837 - lava_callback - ERROR - Error processing callback for node 694094e3575bc060ce16114b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e3575bc060ce16114b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e3575bc060ce16114b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e3575bc060ce16114b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:45,839 - lava_callback - INFO - Starting async job submit for node 694094f8575bc060ce1611dc 2025-12-17 23:20:45,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:45,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f8575bc060ce1611dc HTTP/1.1" 502 150 2025-12-17 23:20:45,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f8575bc060ce1611dc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:45,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f8575bc060ce1611dc 2025-12-17 23:20:45,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f8575bc060ce1611dc HTTP/1.1" 502 150 2025-12-17 23:20:45,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f8575bc060ce1611dc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:45,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f5575bc060ce1611d6 2025-12-17 23:20:45,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f5575bc060ce1611d6 HTTP/1.1" 502 150 2025-12-17 23:20:45,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f5575bc060ce1611d6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52248 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:20:46,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69411958459ff5fc64d2a0ce 2025-12-17 23:20:46,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411958459ff5fc64d2a0ce HTTP/1.1" 502 150 2025-12-17 23:20:46,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69411958459ff5fc64d2a0ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:47,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ea575bc060ce161160 2025-12-17 23:20:47,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ea575bc060ce161160 HTTP/1.1" 502 150 2025-12-17 23:20:47,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ea575bc060ce161160'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:47,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f8575bc060ce1611dc 2025-12-17 23:20:47,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f8575bc060ce1611dc HTTP/1.1" 502 150 2025-12-17 23:20:47,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f8575bc060ce1611dc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:47,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f1575bc060ce16116e 2025-12-17 23:20:47,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f1575bc060ce16116e HTTP/1.1" 502 150 2025-12-17 23:20:47,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f1575bc060ce16116e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:48,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f3575bc060ce161171 2025-12-17 23:20:48,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f3575bc060ce161171 HTTP/1.1" 502 150 2025-12-17 23:20:48,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f3575bc060ce161171'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:49,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f5575bc060ce1611d6 2025-12-17 23:20:49,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f5575bc060ce1611d6 HTTP/1.1" 502 150 2025-12-17 23:20:49,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f5575bc060ce1611d6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:50,329 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094eb575bc060ce161165 2025-12-17 23:20:50,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094eb575bc060ce161165 HTTP/1.1" 502 150 2025-12-17 23:20:50,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094eb575bc060ce161165'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:50,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69411958459ff5fc64d2a0ce 2025-12-17 23:20:50,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411958459ff5fc64d2a0ce HTTP/1.1" 502 150 2025-12-17 23:20:50,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69411958459ff5fc64d2a0ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:51,423 - lava_callback - INFO - Received callback for node 694097c0575bc060ce1618e9 INFO: 172.21.0.1:53180 - "POST /node/694097c0575bc060ce1618e9 HTTP/1.0" 202 Accepted 2025-12-17 23:20:51,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f8575bc060ce1611dc 2025-12-17 23:20:51,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f8575bc060ce1611dc HTTP/1.1" 502 150 2025-12-17 23:20:51,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f8575bc060ce1611dc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:52,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ec575bc060ce161168 2025-12-17 23:20:52,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ec575bc060ce161168 HTTP/1.1" 502 150 2025-12-17 23:20:52,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ec575bc060ce161168'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:52,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ef575bc060ce16116b 2025-12-17 23:20:52,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ef575bc060ce16116b HTTP/1.1" 502 150 2025-12-17 23:20:52,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094ef575bc060ce16116b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:53,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f7575bc060ce1611d9 2025-12-17 23:20:53,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f7575bc060ce1611d9 HTTP/1.1" 502 150 2025-12-17 23:20:53,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f7575bc060ce1611d9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:55,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f1575bc060ce16116e 2025-12-17 23:20:55,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f1575bc060ce16116e HTTP/1.1" 502 150 2025-12-17 23:20:55,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f1575bc060ce16116e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:56,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f3575bc060ce161171 2025-12-17 23:20:56,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f3575bc060ce161171 HTTP/1.1" 502 150 2025-12-17 23:20:56,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f3575bc060ce161171'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:56,178 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940998e575bc060ce161d3c 2025-12-17 23:20:56,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998e575bc060ce161d3c HTTP/1.1" 502 150 2025-12-17 23:20:56,179 - lava_callback - ERROR - Error processing callback for node 6940998e575bc060ce161d3c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940998e575bc060ce161d3c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940998e575bc060ce161d3c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940998e575bc060ce161d3c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:56,181 - lava_callback - INFO - Starting async job submit for node 694094fa575bc060ce1611df 2025-12-17 23:20:56,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:56,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fa575bc060ce1611df HTTP/1.1" 502 150 2025-12-17 23:20:56,189 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094fa575bc060ce1611df'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:56,189 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094fa575bc060ce1611df 2025-12-17 23:20:56,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fa575bc060ce1611df HTTP/1.1" 502 150 2025-12-17 23:20:56,189 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094fa575bc060ce1611df'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53186 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:20:56,574 - lava_callback - INFO - Received callback for node 694097c5575bc060ce1618f6 INFO: 172.21.0.1:53194 - "POST /node/694097c5575bc060ce1618f6 HTTP/1.0" 202 Accepted 2025-12-17 23:20:56,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e4575bc060ce16114e 2025-12-17 23:20:56,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e4575bc060ce16114e HTTP/1.1" 502 150 2025-12-17 23:20:56,832 - lava_callback - ERROR - Error processing callback for node 694094e4575bc060ce16114e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e4575bc060ce16114e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e4575bc060ce16114e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e4575bc060ce16114e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:56,834 - lava_callback - INFO - Starting async job submit for node 694094fc575bc060ce1611e2 2025-12-17 23:20:56,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:56,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fc575bc060ce1611e2 HTTP/1.1" 502 150 2025-12-17 23:20:56,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094fc575bc060ce1611e2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:56,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094fc575bc060ce1611e2 2025-12-17 23:20:56,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fc575bc060ce1611e2 HTTP/1.1" 502 150 2025-12-17 23:20:56,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094fc575bc060ce1611e2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:56,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a2575bc060ce1618ad 2025-12-17 23:20:56,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a2575bc060ce1618ad HTTP/1.1" 502 150 2025-12-17 23:20:56,881 - lava_callback - ERROR - Error processing callback for node 694097a2575bc060ce1618ad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097a2575bc060ce1618ad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097a2575bc060ce1618ad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097a2575bc060ce1618ad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:56,883 - lava_callback - INFO - Starting async job submit for node 69409500575bc060ce1611f5 2025-12-17 23:20:56,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:56,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409500575bc060ce1611f5 HTTP/1.1" 502 150 2025-12-17 23:20:56,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409500575bc060ce1611f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:56,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409500575bc060ce1611f5 2025-12-17 23:20:56,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409500575bc060ce1611f5 HTTP/1.1" 502 150 2025-12-17 23:20:56,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409500575bc060ce1611f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:57,225 - lava_callback - INFO - Received callback for node 694097c7575bc060ce1618f9 INFO: 172.21.0.1:53200 - "POST /node/694097c7575bc060ce1618f9 HTTP/1.0" 202 Accepted 2025-12-17 23:20:57,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f5575bc060ce1611d6 2025-12-17 23:20:57,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f5575bc060ce1611d6 HTTP/1.1" 502 150 2025-12-17 23:20:57,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f5575bc060ce1611d6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:57,921 - lava_callback - INFO - Received callback for node 694097c8575bc060ce1618fc INFO: 172.21.0.1:53206 - "POST /node/694097c8575bc060ce1618fc HTTP/1.0" 202 Accepted 2025-12-17 23:20:57,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e6575bc060ce161151 2025-12-17 23:20:57,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e6575bc060ce161151 HTTP/1.1" 502 150 2025-12-17 23:20:57,949 - lava_callback - ERROR - Error processing callback for node 694094e6575bc060ce161151: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e6575bc060ce161151 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e6575bc060ce161151 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e6575bc060ce161151 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:57,951 - lava_callback - INFO - Starting async job submit for node 69409501575bc060ce1611fa 2025-12-17 23:20:57,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:57,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409501575bc060ce1611fa HTTP/1.1" 502 150 2025-12-17 23:20:57,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409501575bc060ce1611fa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:57,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409501575bc060ce1611fa 2025-12-17 23:20:57,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409501575bc060ce1611fa HTTP/1.1" 502 150 2025-12-17 23:20:57,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409501575bc060ce1611fa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:58,190 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094fa575bc060ce1611df 2025-12-17 23:20:58,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fa575bc060ce1611df HTTP/1.1" 502 150 2025-12-17 23:20:58,191 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094fa575bc060ce1611df'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:58,587 - lava_callback - INFO - Received callback for node 694097c9575bc060ce1618ff INFO: 172.21.0.1:53220 - "POST /node/694097c9575bc060ce1618ff HTTP/1.0" 202 Accepted 2025-12-17 23:20:58,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e7575bc060ce161154 2025-12-17 23:20:58,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e7575bc060ce161154 HTTP/1.1" 502 150 2025-12-17 23:20:58,617 - lava_callback - ERROR - Error processing callback for node 694094e7575bc060ce161154: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e7575bc060ce161154 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e7575bc060ce161154 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e7575bc060ce161154 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:20:58,619 - lava_callback - INFO - Starting async job submit for node 69409503575bc060ce161204 2025-12-17 23:20:58,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:20:58,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409503575bc060ce161204 HTTP/1.1" 502 150 2025-12-17 23:20:58,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409503575bc060ce161204'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:58,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409503575bc060ce161204 2025-12-17 23:20:58,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409503575bc060ce161204 HTTP/1.1" 502 150 2025-12-17 23:20:58,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409503575bc060ce161204'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:58,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69411958459ff5fc64d2a0ce 2025-12-17 23:20:58,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411958459ff5fc64d2a0ce HTTP/1.1" 502 150 2025-12-17 23:20:58,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69411958459ff5fc64d2a0ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:58,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094fc575bc060ce1611e2 2025-12-17 23:20:58,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fc575bc060ce1611e2 HTTP/1.1" 502 150 2025-12-17 23:20:58,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094fc575bc060ce1611e2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:58,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409500575bc060ce1611f5 2025-12-17 23:20:58,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409500575bc060ce1611f5 HTTP/1.1" 502 150 2025-12-17 23:20:58,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409500575bc060ce1611f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:59,269 - lava_callback - INFO - Received callback for node 694097ca575bc060ce161902 INFO: 172.21.0.1:53236 - "POST /node/694097ca575bc060ce161902 HTTP/1.0" 202 Accepted 2025-12-17 23:20:59,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f8575bc060ce1611dc 2025-12-17 23:20:59,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f8575bc060ce1611dc HTTP/1.1" 502 150 2025-12-17 23:20:59,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094f8575bc060ce1611dc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:20:59,946 - lava_callback - INFO - Received callback for node 694097cc575bc060ce161905 INFO: 172.21.0.1:54660 - "POST /node/694097cc575bc060ce161905 HTTP/1.0" 202 Accepted 2025-12-17 23:20:59,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409501575bc060ce1611fa 2025-12-17 23:20:59,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409501575bc060ce1611fa HTTP/1.1" 502 150 2025-12-17 23:20:59,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409501575bc060ce1611fa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:00,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094e8575bc060ce161157 2025-12-17 23:21:00,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094e8575bc060ce161157 HTTP/1.1" 502 150 2025-12-17 23:21:00,501 - lava_callback - ERROR - Error processing callback for node 694094e8575bc060ce161157: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e8575bc060ce161157 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e8575bc060ce161157 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094e8575bc060ce161157 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:00,503 - lava_callback - INFO - Starting async job submit for node 69409506575bc060ce16120a 2025-12-17 23:21:00,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:00,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409506575bc060ce16120a HTTP/1.1" 502 150 2025-12-17 23:21:00,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409506575bc060ce16120a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:00,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409506575bc060ce16120a 2025-12-17 23:21:00,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409506575bc060ce16120a HTTP/1.1" 502 150 2025-12-17 23:21:00,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409506575bc060ce16120a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:00,618 - lava_callback - INFO - Received callback for node 694097cd575bc060ce161908 INFO: 172.21.0.1:54676 - "POST /node/694097cd575bc060ce161908 HTTP/1.0" 202 Accepted 2025-12-17 23:21:00,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409503575bc060ce161204 2025-12-17 23:21:00,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409503575bc060ce161204 HTTP/1.1" 502 150 2025-12-17 23:21:00,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409503575bc060ce161204'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:01,281 - lava_callback - INFO - Received callback for node 694097ce575bc060ce161913 INFO: 172.21.0.1:54678 - "POST /node/694097ce575bc060ce161913 HTTP/1.0" 202 Accepted 2025-12-17 23:21:01,953 - lava_callback - INFO - Received callback for node 694097d0575bc060ce16191c INFO: 172.21.0.1:54690 - "POST /node/694097d0575bc060ce16191c HTTP/1.0" 202 Accepted 2025-12-17 23:21:02,191 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094fa575bc060ce1611df 2025-12-17 23:21:02,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fa575bc060ce1611df HTTP/1.1" 502 150 2025-12-17 23:21:02,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094fa575bc060ce1611df'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:02,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409506575bc060ce16120a 2025-12-17 23:21:02,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409506575bc060ce16120a HTTP/1.1" 502 150 2025-12-17 23:21:02,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409506575bc060ce16120a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:02,612 - lava_callback - INFO - Received callback for node 694097d2575bc060ce16191f INFO: 172.21.0.1:54706 - "POST /node/694097d2575bc060ce16191f HTTP/1.0" 202 Accepted 2025-12-17 23:21:02,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094fc575bc060ce1611e2 2025-12-17 23:21:02,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fc575bc060ce1611e2 HTTP/1.1" 502 150 2025-12-17 23:21:02,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094fc575bc060ce1611e2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:02,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409500575bc060ce1611f5 2025-12-17 23:21:02,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409500575bc060ce1611f5 HTTP/1.1" 502 150 2025-12-17 23:21:02,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409500575bc060ce1611f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54720 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:21:03,296 - lava_callback - INFO - Received callback for node 694097d3575bc060ce161922 INFO: 172.21.0.1:54722 - "POST /node/694097d3575bc060ce161922 HTTP/1.0" 202 Accepted 2025-12-17 23:21:03,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ea575bc060ce161160 2025-12-17 23:21:03,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ea575bc060ce161160 HTTP/1.1" 502 150 2025-12-17 23:21:03,643 - lava_callback - ERROR - Error processing callback for node 694094ea575bc060ce161160: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ea575bc060ce161160 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ea575bc060ce161160 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ea575bc060ce161160 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:03,644 - lava_callback - INFO - Starting async job submit for node 694094fe575bc060ce1611e9 2025-12-17 23:21:03,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:03,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fe575bc060ce1611e9 HTTP/1.1" 502 150 2025-12-17 23:21:03,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094fe575bc060ce1611e9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:03,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094fe575bc060ce1611e9 2025-12-17 23:21:03,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fe575bc060ce1611e9 HTTP/1.1" 502 150 2025-12-17 23:21:03,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094fe575bc060ce1611e9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:03,959 - lava_callback - INFO - Received callback for node 694097d5575bc060ce161925 INFO: 172.21.0.1:54738 - "POST /node/694097d5575bc060ce161925 HTTP/1.0" 202 Accepted 2025-12-17 23:21:03,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409501575bc060ce1611fa 2025-12-17 23:21:03,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409501575bc060ce1611fa HTTP/1.1" 502 150 2025-12-17 23:21:03,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409501575bc060ce1611fa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:04,612 - lava_callback - INFO - Received callback for node 694097d6575bc060ce161928 INFO: 172.21.0.1:54740 - "POST /node/694097d6575bc060ce161928 HTTP/1.0" 202 Accepted 2025-12-17 23:21:04,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409503575bc060ce161204 2025-12-17 23:21:04,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409503575bc060ce161204 HTTP/1.1" 502 150 2025-12-17 23:21:04,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409503575bc060ce161204'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:05,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094fe575bc060ce1611e9 2025-12-17 23:21:06,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094eb575bc060ce161165 2025-12-17 23:21:06,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094eb575bc060ce161165 HTTP/1.1" 502 150 2025-12-17 23:21:06,333 - lava_callback - ERROR - Error processing callback for node 694094eb575bc060ce161165: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094eb575bc060ce161165 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094eb575bc060ce161165 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094eb575bc060ce161165 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:06,335 - lava_callback - INFO - Starting async job submit for node 69409509575bc060ce161216 2025-12-17 23:21:06,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:06,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409509575bc060ce161216 HTTP/1.1" 502 150 2025-12-17 23:21:06,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409509575bc060ce161216'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:06,343 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409509575bc060ce161216 2025-12-17 23:21:06,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409509575bc060ce161216 HTTP/1.1" 502 150 2025-12-17 23:21:06,343 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409509575bc060ce161216'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:06,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409506575bc060ce16120a 2025-12-17 23:21:06,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409506575bc060ce16120a HTTP/1.1" 502 150 2025-12-17 23:21:06,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409506575bc060ce16120a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54746 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:21:08,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ec575bc060ce161168 2025-12-17 23:21:08,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ec575bc060ce161168 HTTP/1.1" 502 150 2025-12-17 23:21:08,024 - lava_callback - ERROR - Error processing callback for node 694094ec575bc060ce161168: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ec575bc060ce161168 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ec575bc060ce161168 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ec575bc060ce161168 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:08,025 - lava_callback - INFO - Starting async job submit for node 6940950b575bc060ce16121b 2025-12-17 23:21:08,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:08,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950b575bc060ce16121b HTTP/1.1" 502 150 2025-12-17 23:21:08,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950b575bc060ce16121b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:08,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950b575bc060ce16121b 2025-12-17 23:21:08,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950b575bc060ce16121b HTTP/1.1" 502 150 2025-12-17 23:21:08,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950b575bc060ce16121b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:08,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409509575bc060ce161216 2025-12-17 23:21:08,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409509575bc060ce161216 HTTP/1.1" 502 150 2025-12-17 23:21:08,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409509575bc060ce161216'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:08,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094ef575bc060ce16116b 2025-12-17 23:21:08,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094ef575bc060ce16116b HTTP/1.1" 502 150 2025-12-17 23:21:08,996 - lava_callback - ERROR - Error processing callback for node 694094ef575bc060ce16116b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ef575bc060ce16116b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ef575bc060ce16116b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094ef575bc060ce16116b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:08,998 - lava_callback - INFO - Starting async job submit for node 6940950d575bc060ce161221 2025-12-17 23:21:08,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:09,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950d575bc060ce161221 HTTP/1.1" 502 150 2025-12-17 23:21:09,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950d575bc060ce161221'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:09,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950d575bc060ce161221 2025-12-17 23:21:09,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950d575bc060ce161221 HTTP/1.1" 502 150 2025-12-17 23:21:09,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950d575bc060ce161221'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:09,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f7575bc060ce1611d9 2025-12-17 23:21:09,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f7575bc060ce1611d9 HTTP/1.1" 502 150 2025-12-17 23:21:09,760 - lava_callback - ERROR - Error processing callback for node 694094f7575bc060ce1611d9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094f7575bc060ce1611d9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094f7575bc060ce1611d9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094f7575bc060ce1611d9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:09,761 - lava_callback - INFO - Starting async job submit for node 69409510575bc060ce161227 2025-12-17 23:21:09,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:09,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409510575bc060ce161227 HTTP/1.1" 502 150 2025-12-17 23:21:09,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409510575bc060ce161227'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:09,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409510575bc060ce161227 2025-12-17 23:21:09,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409510575bc060ce161227 HTTP/1.1" 502 150 2025-12-17 23:21:09,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409510575bc060ce161227'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:10,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950b575bc060ce16121b 2025-12-17 23:21:10,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094fa575bc060ce1611df 2025-12-17 23:21:10,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fa575bc060ce1611df HTTP/1.1" 502 150 2025-12-17 23:21:10,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094fa575bc060ce1611df'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:10,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094fc575bc060ce1611e2 2025-12-17 23:21:10,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fc575bc060ce1611e2 HTTP/1.1" 502 150 2025-12-17 23:21:10,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094fc575bc060ce1611e2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:10,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409500575bc060ce1611f5 2025-12-17 23:21:10,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409500575bc060ce1611f5 HTTP/1.1" 502 150 2025-12-17 23:21:10,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409500575bc060ce1611f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:11,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950d575bc060ce161221 2025-12-17 23:21:11,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950d575bc060ce161221 HTTP/1.1" 502 150 2025-12-17 23:21:11,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950d575bc060ce161221'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:11,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409510575bc060ce161227 2025-12-17 23:21:11,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409510575bc060ce161227 HTTP/1.1" 502 150 2025-12-17 23:21:11,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409510575bc060ce161227'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:11,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f1575bc060ce16116e 2025-12-17 23:21:11,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f1575bc060ce16116e HTTP/1.1" 502 150 2025-12-17 23:21:11,920 - lava_callback - ERROR - Error processing callback for node 694094f1575bc060ce16116e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094f1575bc060ce16116e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094f1575bc060ce16116e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094f1575bc060ce16116e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:11,922 - lava_callback - INFO - Starting async job submit for node 69409508575bc060ce161210 2025-12-17 23:21:11,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:11,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409508575bc060ce161210 HTTP/1.1" 502 150 2025-12-17 23:21:11,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409508575bc060ce161210'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:11,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409508575bc060ce161210 2025-12-17 23:21:11,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409508575bc060ce161210 HTTP/1.1" 502 150 2025-12-17 23:21:11,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409508575bc060ce161210'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:11,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409501575bc060ce1611fa 2025-12-17 23:21:11,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409501575bc060ce1611fa HTTP/1.1" 502 150 2025-12-17 23:21:11,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409501575bc060ce1611fa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:12,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f3575bc060ce161171 2025-12-17 23:21:12,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f3575bc060ce161171 HTTP/1.1" 502 150 2025-12-17 23:21:12,028 - lava_callback - ERROR - Error processing callback for node 694094f3575bc060ce161171: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094f3575bc060ce161171 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094f3575bc060ce161171 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094f3575bc060ce161171 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:12,029 - lava_callback - INFO - Starting async job submit for node 69409511575bc060ce16122a 2025-12-17 23:21:12,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:12,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409511575bc060ce16122a HTTP/1.1" 502 150 2025-12-17 23:21:12,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409511575bc060ce16122a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:12,059 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409511575bc060ce16122a 2025-12-17 23:21:12,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409511575bc060ce16122a HTTP/1.1" 502 150 2025-12-17 23:21:12,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409511575bc060ce16122a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:12,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409509575bc060ce161216 2025-12-17 23:21:12,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409509575bc060ce161216 HTTP/1.1" 502 150 2025-12-17 23:21:12,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409509575bc060ce161216'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:12,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409503575bc060ce161204 2025-12-17 23:21:12,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409503575bc060ce161204 HTTP/1.1" 502 150 2025-12-17 23:21:12,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409503575bc060ce161204'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:13,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f5575bc060ce1611d6 2025-12-17 23:21:13,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f5575bc060ce1611d6 HTTP/1.1" 502 150 2025-12-17 23:21:13,907 - lava_callback - ERROR - Error processing callback for node 694094f5575bc060ce1611d6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094f5575bc060ce1611d6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094f5575bc060ce1611d6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094f5575bc060ce1611d6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:13,909 - lava_callback - INFO - Starting async job submit for node 6940950c575bc060ce16121e 2025-12-17 23:21:13,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:13,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950c575bc060ce16121e HTTP/1.1" 502 150 2025-12-17 23:21:13,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950c575bc060ce16121e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:13,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950c575bc060ce16121e 2025-12-17 23:21:13,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950c575bc060ce16121e HTTP/1.1" 502 150 2025-12-17 23:21:13,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950c575bc060ce16121e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:13,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409508575bc060ce161210 2025-12-17 23:21:13,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409508575bc060ce161210 HTTP/1.1" 502 150 2025-12-17 23:21:13,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409508575bc060ce161210'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:14,060 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409511575bc060ce16122a 2025-12-17 23:21:14,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409511575bc060ce16122a HTTP/1.1" 502 150 2025-12-17 23:21:14,061 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409511575bc060ce16122a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:14,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409506575bc060ce16120a 2025-12-17 23:21:14,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409506575bc060ce16120a HTTP/1.1" 502 150 2025-12-17 23:21:14,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409506575bc060ce16120a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:14,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69411958459ff5fc64d2a0ce 2025-12-17 23:21:14,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411958459ff5fc64d2a0ce HTTP/1.1" 502 150 2025-12-17 23:21:14,644 - lava_callback - ERROR - Error processing callback for node 69411958459ff5fc64d2a0ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69411958459ff5fc64d2a0ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69411958459ff5fc64d2a0ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69411958459ff5fc64d2a0ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:14,645 - lava_callback - INFO - Starting async job submit for node 6940950e575bc060ce161224 2025-12-17 23:21:14,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:14,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950e575bc060ce161224 HTTP/1.1" 502 150 2025-12-17 23:21:14,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950e575bc060ce161224'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:14,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950e575bc060ce161224 2025-12-17 23:21:14,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950e575bc060ce161224 HTTP/1.1" 502 150 2025-12-17 23:21:14,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950e575bc060ce161224'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:15,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950d575bc060ce161221 2025-12-17 23:21:15,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950d575bc060ce161221 HTTP/1.1" 502 150 2025-12-17 23:21:15,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950d575bc060ce161221'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37512 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:21:15,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409510575bc060ce161227 2025-12-17 23:21:15,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409510575bc060ce161227 HTTP/1.1" 502 150 2025-12-17 23:21:15,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409510575bc060ce161227'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:15,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094f8575bc060ce1611dc 2025-12-17 23:21:15,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094f8575bc060ce1611dc HTTP/1.1" 502 150 2025-12-17 23:21:15,852 - lava_callback - ERROR - Error processing callback for node 694094f8575bc060ce1611dc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094f8575bc060ce1611dc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094f8575bc060ce1611dc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094f8575bc060ce1611dc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:15,855 - lava_callback - INFO - Starting async job submit for node 6940951a575bc060ce161236 2025-12-17 23:21:15,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:15,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951a575bc060ce161236 HTTP/1.1" 502 150 2025-12-17 23:21:15,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951a575bc060ce161236'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:15,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951a575bc060ce161236 2025-12-17 23:21:15,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951a575bc060ce161236 HTTP/1.1" 502 150 2025-12-17 23:21:15,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951a575bc060ce161236'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:15,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950c575bc060ce16121e 2025-12-17 23:21:15,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950c575bc060ce16121e HTTP/1.1" 502 150 2025-12-17 23:21:15,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950c575bc060ce16121e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37528 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:21:16,608 - lava_callback - INFO - Received callback for node 694097d9575bc060ce161930 INFO: 172.21.0.1:37540 - "POST /node/694097d9575bc060ce161930 HTTP/1.0" 202 Accepted 2025-12-17 23:21:16,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950e575bc060ce161224 2025-12-17 23:21:16,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950e575bc060ce161224 HTTP/1.1" 502 150 2025-12-17 23:21:16,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950e575bc060ce161224'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:17,297 - lava_callback - INFO - Received callback for node 694097db575bc060ce161934 INFO: 172.21.0.1:37552 - "POST /node/694097db575bc060ce161934 HTTP/1.0" 202 Accepted 2025-12-17 23:21:17,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951a575bc060ce161236 2025-12-17 23:21:17,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951a575bc060ce161236 HTTP/1.1" 502 150 2025-12-17 23:21:17,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951a575bc060ce161236'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:17,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409508575bc060ce161210 2025-12-17 23:21:17,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409508575bc060ce161210 HTTP/1.1" 502 150 2025-12-17 23:21:17,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409508575bc060ce161210'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:17,977 - lava_callback - INFO - Received callback for node 694097dd575bc060ce161937 INFO: 172.21.0.1:37558 - "POST /node/694097dd575bc060ce161937 HTTP/1.0" 202 Accepted 2025-12-17 23:21:18,061 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409511575bc060ce16122a 2025-12-17 23:21:18,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409511575bc060ce16122a HTTP/1.1" 502 150 2025-12-17 23:21:18,063 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409511575bc060ce16122a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:18,635 - lava_callback - INFO - Received callback for node 694097df575bc060ce16193a INFO: 172.21.0.1:37570 - "POST /node/694097df575bc060ce16193a HTTP/1.0" 202 Accepted 2025-12-17 23:21:19,291 - lava_callback - INFO - Received callback for node 694097e1575bc060ce16193d INFO: 172.21.0.1:37574 - "POST /node/694097e1575bc060ce16193d HTTP/1.0" 202 Accepted 2025-12-17 23:21:19,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950c575bc060ce16121e 2025-12-17 23:21:19,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950c575bc060ce16121e HTTP/1.1" 502 150 2025-12-17 23:21:19,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950c575bc060ce16121e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:19,931 - lava_callback - INFO - Received callback for node 694097e2575bc060ce161944 INFO: 172.21.0.1:40592 - "POST /node/694097e2575bc060ce161944 HTTP/1.0" 202 Accepted 2025-12-17 23:21:20,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409509575bc060ce161216 2025-12-17 23:21:20,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409509575bc060ce161216 HTTP/1.1" 502 150 2025-12-17 23:21:20,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409509575bc060ce161216'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:20,586 - lava_callback - INFO - Received callback for node 694097e4575bc060ce16194c INFO: 172.21.0.1:40608 - "POST /node/694097e4575bc060ce16194c HTTP/1.0" 202 Accepted 2025-12-17 23:21:20,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950e575bc060ce161224 2025-12-17 23:21:20,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950e575bc060ce161224 HTTP/1.1" 502 150 2025-12-17 23:21:20,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950e575bc060ce161224'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:21,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fe575bc060ce1611e9 HTTP/1.1" 502 150 2025-12-17 23:21:21,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094fe575bc060ce1611e9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:21,228 - lava_callback - INFO - Received callback for node 694097e6575bc060ce161952 INFO: 172.21.0.1:40620 - "POST /node/694097e6575bc060ce161952 HTTP/1.0" 202 Accepted 2025-12-17 23:21:21,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951a575bc060ce161236 2025-12-17 23:21:21,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951a575bc060ce161236 HTTP/1.1" 502 150 2025-12-17 23:21:21,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951a575bc060ce161236'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:23,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950d575bc060ce161221 2025-12-17 23:21:23,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950d575bc060ce161221 HTTP/1.1" 502 150 2025-12-17 23:21:23,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950d575bc060ce161221'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:23,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409510575bc060ce161227 2025-12-17 23:21:23,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409510575bc060ce161227 HTTP/1.1" 502 150 2025-12-17 23:21:23,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409510575bc060ce161227'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:24,613 - lava_callback - INFO - Received callback for node 6941dc3becc59770562efc10 INFO: 172.21.0.1:40632 - "POST /node/6941dc3becc59770562efc10 HTTP/1.0" 202 Accepted 2025-12-17 23:21:25,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094fe575bc060ce1611e9 2025-12-17 23:21:25,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fe575bc060ce1611e9 HTTP/1.1" 502 150 2025-12-17 23:21:25,074 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094fe575bc060ce1611e9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:25,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950b575bc060ce16121b HTTP/1.1" 502 150 2025-12-17 23:21:25,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950b575bc060ce16121b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:25,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409508575bc060ce161210 2025-12-17 23:21:25,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409508575bc060ce161210 HTTP/1.1" 502 150 2025-12-17 23:21:25,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409508575bc060ce161210'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:26,063 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409511575bc060ce16122a 2025-12-17 23:21:26,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409511575bc060ce16122a HTTP/1.1" 502 150 2025-12-17 23:21:26,064 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409511575bc060ce16122a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:26,195 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094fa575bc060ce1611df 2025-12-17 23:21:26,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fa575bc060ce1611df HTTP/1.1" 502 150 2025-12-17 23:21:26,196 - lava_callback - ERROR - Error processing callback for node 694094fa575bc060ce1611df: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094fa575bc060ce1611df (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094fa575bc060ce1611df (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094fa575bc060ce1611df (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:26,198 - lava_callback - INFO - Starting async job submit for node 69409543575bc060ce16129f 2025-12-17 23:21:26,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:26,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409543575bc060ce16129f HTTP/1.1" 502 150 2025-12-17 23:21:26,341 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409543575bc060ce16129f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:26,341 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409543575bc060ce16129f 2025-12-17 23:21:26,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409543575bc060ce16129f HTTP/1.1" 502 150 2025-12-17 23:21:26,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409543575bc060ce16129f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40644 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:21:26,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094fc575bc060ce1611e2 2025-12-17 23:21:26,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fc575bc060ce1611e2 HTTP/1.1" 502 150 2025-12-17 23:21:26,848 - lava_callback - ERROR - Error processing callback for node 694094fc575bc060ce1611e2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094fc575bc060ce1611e2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094fc575bc060ce1611e2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094fc575bc060ce1611e2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:26,850 - lava_callback - INFO - Starting async job submit for node 69409566575bc060ce161306 2025-12-17 23:21:26,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:26,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409566575bc060ce161306 HTTP/1.1" 502 150 2025-12-17 23:21:26,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409566575bc060ce161306'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:26,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409566575bc060ce161306 2025-12-17 23:21:26,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409500575bc060ce1611f5 2025-12-17 23:21:26,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409500575bc060ce1611f5 HTTP/1.1" 502 150 2025-12-17 23:21:26,896 - lava_callback - ERROR - Error processing callback for node 69409500575bc060ce1611f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409500575bc060ce1611f5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409500575bc060ce1611f5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409500575bc060ce1611f5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:26,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409566575bc060ce161306 HTTP/1.1" 502 150 2025-12-17 23:21:26,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409566575bc060ce161306'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:26,897 - lava_callback - INFO - Starting async job submit for node 69409516575bc060ce16122d 2025-12-17 23:21:26,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:26,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409516575bc060ce16122d HTTP/1.1" 502 150 2025-12-17 23:21:26,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409516575bc060ce16122d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:26,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409516575bc060ce16122d 2025-12-17 23:21:26,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409516575bc060ce16122d HTTP/1.1" 502 150 2025-12-17 23:21:26,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409516575bc060ce16122d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:27,179 - lava_callback - INFO - Received callback for node 694097f1575bc060ce161964 INFO: 172.21.0.1:40652 - "POST /node/694097f1575bc060ce161964 HTTP/1.0" 202 Accepted 2025-12-17 23:21:27,823 - lava_callback - INFO - Received callback for node 694097f4575bc060ce16196b INFO: 172.21.0.1:40660 - "POST /node/694097f4575bc060ce16196b HTTP/1.0" 202 Accepted 2025-12-17 23:21:27,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950c575bc060ce16121e 2025-12-17 23:21:27,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950c575bc060ce16121e HTTP/1.1" 502 150 2025-12-17 23:21:27,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950c575bc060ce16121e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:27,973 - lava_callback - INFO - Received callback for node 694097ea575bc060ce161958 INFO: 172.21.0.1:40662 - "POST /node/694097ea575bc060ce161958 HTTP/1.0" 202 Accepted 2025-12-17 23:21:27,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409501575bc060ce1611fa 2025-12-17 23:21:27,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409501575bc060ce1611fa HTTP/1.1" 502 150 2025-12-17 23:21:27,986 - lava_callback - ERROR - Error processing callback for node 69409501575bc060ce1611fa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409501575bc060ce1611fa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409501575bc060ce1611fa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409501575bc060ce1611fa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:27,988 - lava_callback - INFO - Starting async job submit for node 69409518575bc060ce161230 2025-12-17 23:21:27,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:27,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409518575bc060ce161230 HTTP/1.1" 502 150 2025-12-17 23:21:27,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409518575bc060ce161230'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:27,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409518575bc060ce161230 2025-12-17 23:21:27,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409518575bc060ce161230 HTTP/1.1" 502 150 2025-12-17 23:21:27,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409518575bc060ce161230'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:28,342 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409543575bc060ce16129f 2025-12-17 23:21:28,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409543575bc060ce16129f HTTP/1.1" 502 150 2025-12-17 23:21:28,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409543575bc060ce16129f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:28,507 - lava_callback - INFO - Received callback for node 694097f6575bc060ce161970 INFO: 172.21.0.1:40668 - "POST /node/694097f6575bc060ce161970 HTTP/1.0" 202 Accepted 2025-12-17 23:21:28,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409503575bc060ce161204 2025-12-17 23:21:28,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409503575bc060ce161204 HTTP/1.1" 502 150 2025-12-17 23:21:28,631 - lava_callback - ERROR - Error processing callback for node 69409503575bc060ce161204: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409503575bc060ce161204 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409503575bc060ce161204 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409503575bc060ce161204 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:28,633 - lava_callback - INFO - Starting async job submit for node 69409526575bc060ce161251 2025-12-17 23:21:28,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:28,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409526575bc060ce161251 HTTP/1.1" 502 150 2025-12-17 23:21:28,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409526575bc060ce161251'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:28,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409526575bc060ce161251 2025-12-17 23:21:28,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409526575bc060ce161251 HTTP/1.1" 502 150 2025-12-17 23:21:28,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409526575bc060ce161251'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:28,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950e575bc060ce161224 2025-12-17 23:21:28,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950e575bc060ce161224 HTTP/1.1" 502 150 2025-12-17 23:21:28,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950e575bc060ce161224'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:28,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409566575bc060ce161306 2025-12-17 23:21:28,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409566575bc060ce161306 HTTP/1.1" 502 150 2025-12-17 23:21:28,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409566575bc060ce161306'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:28,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409516575bc060ce16122d 2025-12-17 23:21:28,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409516575bc060ce16122d HTTP/1.1" 502 150 2025-12-17 23:21:28,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409516575bc060ce16122d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:29,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950b575bc060ce16121b 2025-12-17 23:21:29,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950b575bc060ce16121b HTTP/1.1" 502 150 2025-12-17 23:21:29,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950b575bc060ce16121b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:29,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951a575bc060ce161236 2025-12-17 23:21:29,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951a575bc060ce161236 HTTP/1.1" 502 150 2025-12-17 23:21:29,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951a575bc060ce161236'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:29,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409518575bc060ce161230 2025-12-17 23:21:29,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409518575bc060ce161230 HTTP/1.1" 502 150 2025-12-17 23:21:29,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409518575bc060ce161230'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:30,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409506575bc060ce16120a 2025-12-17 23:21:30,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409506575bc060ce16120a HTTP/1.1" 502 150 2025-12-17 23:21:30,516 - lava_callback - ERROR - Error processing callback for node 69409506575bc060ce16120a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409506575bc060ce16120a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409506575bc060ce16120a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409506575bc060ce16120a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:30,519 - lava_callback - INFO - Starting async job submit for node 6940951b575bc060ce161239 2025-12-17 23:21:30,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:30,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951b575bc060ce161239 HTTP/1.1" 502 150 2025-12-17 23:21:30,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951b575bc060ce161239'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:30,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951b575bc060ce161239 2025-12-17 23:21:30,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951b575bc060ce161239 HTTP/1.1" 502 150 2025-12-17 23:21:30,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951b575bc060ce161239'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:30,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409526575bc060ce161251 2025-12-17 23:21:30,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409526575bc060ce161251 HTTP/1.1" 502 150 2025-12-17 23:21:30,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409526575bc060ce161251'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:32,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409543575bc060ce16129f 2025-12-17 23:21:32,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951b575bc060ce161239 2025-12-17 23:21:32,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951b575bc060ce161239 HTTP/1.1" 502 150 2025-12-17 23:21:32,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951b575bc060ce161239'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:32,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409566575bc060ce161306 2025-12-17 23:21:32,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409566575bc060ce161306 HTTP/1.1" 502 150 2025-12-17 23:21:32,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409566575bc060ce161306'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:32,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409516575bc060ce16122d 2025-12-17 23:21:32,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409516575bc060ce16122d HTTP/1.1" 502 150 2025-12-17 23:21:32,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409516575bc060ce16122d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:33,074 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094fe575bc060ce1611e9 2025-12-17 23:21:33,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fe575bc060ce1611e9 HTTP/1.1" 502 150 2025-12-17 23:21:33,076 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694094fe575bc060ce1611e9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:33,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409518575bc060ce161230 2025-12-17 23:21:33,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409518575bc060ce161230 HTTP/1.1" 502 150 2025-12-17 23:21:33,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409518575bc060ce161230'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:34,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409526575bc060ce161251 2025-12-17 23:21:34,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409526575bc060ce161251 HTTP/1.1" 502 150 2025-12-17 23:21:34,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409526575bc060ce161251'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:36,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409509575bc060ce161216 INFO: 172.21.0.1:35046 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:21:36,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951b575bc060ce161239 2025-12-17 23:21:36,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951b575bc060ce161239 HTTP/1.1" 502 150 2025-12-17 23:21:36,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951b575bc060ce161239'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:36,586 - lava_callback - INFO - Received callback for node 694097eb575bc060ce16195b INFO: 172.21.0.1:35052 - "POST /node/694097eb575bc060ce16195b HTTP/1.0" 202 Accepted 2025-12-17 23:21:37,254 - lava_callback - INFO - Received callback for node 694097ed575bc060ce16195e INFO: 172.21.0.1:35056 - "POST /node/694097ed575bc060ce16195e HTTP/1.0" 202 Accepted 2025-12-17 23:21:37,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950b575bc060ce16121b 2025-12-17 23:21:37,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950b575bc060ce16121b HTTP/1.1" 502 150 2025-12-17 23:21:37,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940950b575bc060ce16121b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:37,942 - lava_callback - INFO - Received callback for node 694097ef575bc060ce161961 INFO: 172.21.0.1:35066 - "POST /node/694097ef575bc060ce161961 HTTP/1.0" 202 Accepted 2025-12-17 23:21:38,609 - lava_callback - INFO - Received callback for node 694097fd575bc060ce161988 INFO: 172.21.0.1:35076 - "POST /node/694097fd575bc060ce161988 HTTP/1.0" 202 Accepted 2025-12-17 23:21:39,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950d575bc060ce161221 2025-12-17 23:21:39,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950d575bc060ce161221 HTTP/1.1" 502 150 2025-12-17 23:21:39,011 - lava_callback - ERROR - Error processing callback for node 6940950d575bc060ce161221: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940950d575bc060ce161221 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940950d575bc060ce161221 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940950d575bc060ce161221 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:39,013 - lava_callback - INFO - Starting async job submit for node 6940951e575bc060ce16123f 2025-12-17 23:21:39,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:39,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951e575bc060ce16123f HTTP/1.1" 502 150 2025-12-17 23:21:39,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951e575bc060ce16123f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:39,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951e575bc060ce16123f 2025-12-17 23:21:39,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951e575bc060ce16123f HTTP/1.1" 502 150 2025-12-17 23:21:39,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951e575bc060ce16123f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:39,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409510575bc060ce161227 2025-12-17 23:21:39,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409510575bc060ce161227 HTTP/1.1" 502 150 2025-12-17 23:21:39,775 - lava_callback - ERROR - Error processing callback for node 69409510575bc060ce161227: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409510575bc060ce161227 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409510575bc060ce161227 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409510575bc060ce161227 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:39,776 - lava_callback - INFO - Starting async job submit for node 6940951f575bc060ce161242 2025-12-17 23:21:39,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:39,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951f575bc060ce161242 HTTP/1.1" 502 150 2025-12-17 23:21:39,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951f575bc060ce161242'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:39,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951f575bc060ce161242 2025-12-17 23:21:39,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951f575bc060ce161242 HTTP/1.1" 502 150 2025-12-17 23:21:39,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951f575bc060ce161242'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:40,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409566575bc060ce161306 2025-12-17 23:21:40,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409566575bc060ce161306 HTTP/1.1" 502 150 2025-12-17 23:21:40,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409566575bc060ce161306'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:40,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409516575bc060ce16122d 2025-12-17 23:21:40,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409516575bc060ce16122d HTTP/1.1" 502 150 2025-12-17 23:21:40,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409516575bc060ce16122d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:41,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951e575bc060ce16123f 2025-12-17 23:21:41,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951e575bc060ce16123f HTTP/1.1" 502 150 2025-12-17 23:21:41,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951e575bc060ce16123f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:41,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951f575bc060ce161242 2025-12-17 23:21:41,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951f575bc060ce161242 HTTP/1.1" 502 150 2025-12-17 23:21:41,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951f575bc060ce161242'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:41,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409508575bc060ce161210 2025-12-17 23:21:41,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409508575bc060ce161210 HTTP/1.1" 502 150 2025-12-17 23:21:41,940 - lava_callback - ERROR - Error processing callback for node 69409508575bc060ce161210: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409508575bc060ce161210 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409508575bc060ce161210 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409508575bc060ce161210 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:41,942 - lava_callback - INFO - Starting async job submit for node 69409520575bc060ce161245 2025-12-17 23:21:41,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:41,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409520575bc060ce161245 HTTP/1.1" 502 150 2025-12-17 23:21:41,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409520575bc060ce161245'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:41,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409520575bc060ce161245 2025-12-17 23:21:41,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409520575bc060ce161245 HTTP/1.1" 502 150 2025-12-17 23:21:41,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409520575bc060ce161245'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:41,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409518575bc060ce161230 2025-12-17 23:21:41,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409518575bc060ce161230 HTTP/1.1" 502 150 2025-12-17 23:21:41,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409518575bc060ce161230'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:42,065 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409511575bc060ce16122a 2025-12-17 23:21:42,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409511575bc060ce16122a HTTP/1.1" 502 150 2025-12-17 23:21:42,066 - lava_callback - ERROR - Error processing callback for node 69409511575bc060ce16122a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409511575bc060ce16122a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409511575bc060ce16122a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409511575bc060ce16122a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:42,068 - lava_callback - INFO - Starting async job submit for node 69409521575bc060ce161248 2025-12-17 23:21:42,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:42,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409521575bc060ce161248 HTTP/1.1" 502 150 2025-12-17 23:21:42,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409521575bc060ce161248'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:42,080 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409521575bc060ce161248 2025-12-17 23:21:42,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409521575bc060ce161248 HTTP/1.1" 502 150 2025-12-17 23:21:42,081 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409521575bc060ce161248'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:42,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409526575bc060ce161251 2025-12-17 23:21:42,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409526575bc060ce161251 HTTP/1.1" 502 150 2025-12-17 23:21:42,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409526575bc060ce161251'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:43,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950c575bc060ce16121e 2025-12-17 23:21:43,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409520575bc060ce161245 2025-12-17 23:21:43,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409520575bc060ce161245 HTTP/1.1" 502 150 2025-12-17 23:21:43,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409520575bc060ce161245'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:44,081 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409521575bc060ce161248 2025-12-17 23:21:44,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409521575bc060ce161248 HTTP/1.1" 502 150 2025-12-17 23:21:44,082 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409521575bc060ce161248'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:44,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951b575bc060ce161239 2025-12-17 23:21:44,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951b575bc060ce161239 HTTP/1.1" 502 150 2025-12-17 23:21:44,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951b575bc060ce161239'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:44,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950e575bc060ce161224 2025-12-17 23:21:44,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950e575bc060ce161224 HTTP/1.1" 502 150 2025-12-17 23:21:44,683 - lava_callback - ERROR - Error processing callback for node 6940950e575bc060ce161224: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940950e575bc060ce161224 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940950e575bc060ce161224 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940950e575bc060ce161224 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:44,685 - lava_callback - INFO - Starting async job submit for node 69409519575bc060ce161233 2025-12-17 23:21:44,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:44,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409519575bc060ce161233 HTTP/1.1" 502 150 2025-12-17 23:21:44,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409519575bc060ce161233'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:44,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409519575bc060ce161233 2025-12-17 23:21:44,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409519575bc060ce161233 HTTP/1.1" 502 150 2025-12-17 23:21:44,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409519575bc060ce161233'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:45,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951e575bc060ce16123f 2025-12-17 23:21:45,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951e575bc060ce16123f HTTP/1.1" 502 150 2025-12-17 23:21:45,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951e575bc060ce16123f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:45,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951f575bc060ce161242 2025-12-17 23:21:45,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951f575bc060ce161242 HTTP/1.1" 502 150 2025-12-17 23:21:45,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951f575bc060ce161242'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:45,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951a575bc060ce161236 2025-12-17 23:21:45,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951a575bc060ce161236 HTTP/1.1" 502 150 2025-12-17 23:21:45,869 - lava_callback - ERROR - Error processing callback for node 6940951a575bc060ce161236: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940951a575bc060ce161236 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940951a575bc060ce161236 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940951a575bc060ce161236 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:45,871 - lava_callback - INFO - Starting async job submit for node 69409523575bc060ce16124b 2025-12-17 23:21:45,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:45,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409523575bc060ce16124b HTTP/1.1" 502 150 2025-12-17 23:21:45,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409523575bc060ce16124b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:45,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409523575bc060ce16124b 2025-12-17 23:21:45,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409523575bc060ce16124b HTTP/1.1" 502 150 2025-12-17 23:21:45,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409523575bc060ce16124b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53770 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:21:46,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409519575bc060ce161233 2025-12-17 23:21:46,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409519575bc060ce161233 HTTP/1.1" 502 150 2025-12-17 23:21:46,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409519575bc060ce161233'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:47,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409543575bc060ce16129f HTTP/1.1" 502 150 2025-12-17 23:21:47,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409543575bc060ce16129f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:47,867 - lava_callback - INFO - Received callback for node 694097fa575bc060ce161982 INFO: 172.21.0.1:53776 - "POST /node/694097fa575bc060ce161982 HTTP/1.0" 202 Accepted 2025-12-17 23:21:47,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409523575bc060ce16124b 2025-12-17 23:21:47,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409523575bc060ce16124b HTTP/1.1" 502 150 2025-12-17 23:21:47,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409523575bc060ce16124b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:47,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409520575bc060ce161245 2025-12-17 23:21:47,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409520575bc060ce161245 HTTP/1.1" 502 150 2025-12-17 23:21:47,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409520575bc060ce161245'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:48,082 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409521575bc060ce161248 2025-12-17 23:21:48,660 - lava_callback - INFO - Received callback for node 69409805575bc060ce16199a INFO: 172.21.0.1:53788 - "POST /node/69409805575bc060ce16199a HTTP/1.0" 202 Accepted 2025-12-17 23:21:49,076 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694094fe575bc060ce1611e9 2025-12-17 23:21:49,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694094fe575bc060ce1611e9 HTTP/1.1" 502 150 2025-12-17 23:21:49,077 - lava_callback - ERROR - Error processing callback for node 694094fe575bc060ce1611e9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094fe575bc060ce1611e9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094fe575bc060ce1611e9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694094fe575bc060ce1611e9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:49,079 - lava_callback - INFO - Starting async job submit for node 69409524575bc060ce16124e 2025-12-17 23:21:49,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:49,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409524575bc060ce16124e HTTP/1.1" 502 150 2025-12-17 23:21:49,086 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409524575bc060ce16124e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:49,086 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409524575bc060ce16124e 2025-12-17 23:21:49,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409524575bc060ce16124e HTTP/1.1" 502 150 2025-12-17 23:21:49,087 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409524575bc060ce16124e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:49,467 - lava_callback - INFO - Received callback for node 69409808575bc060ce1619a5 INFO: 172.21.0.1:53794 - "POST /node/69409808575bc060ce1619a5 HTTP/1.0" 202 Accepted 2025-12-17 23:21:50,202 - lava_callback - INFO - Received callback for node 69409809575bc060ce1619a8 INFO: 172.21.0.1:37636 - "POST /node/69409809575bc060ce1619a8 HTTP/1.0" 202 Accepted 2025-12-17 23:21:50,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409519575bc060ce161233 2025-12-17 23:21:50,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409519575bc060ce161233 HTTP/1.1" 502 150 2025-12-17 23:21:50,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409519575bc060ce161233'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:51,087 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409524575bc060ce16124e 2025-12-17 23:21:51,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409524575bc060ce16124e HTTP/1.1" 502 150 2025-12-17 23:21:51,089 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409524575bc060ce16124e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:51,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409509575bc060ce161216 HTTP/1.1" 502 150 2025-12-17 23:21:51,792 - lava_callback - ERROR - Error processing callback for node 69409509575bc060ce161216: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409509575bc060ce161216 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409509575bc060ce161216 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409509575bc060ce161216 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:51,794 - lava_callback - INFO - Starting async job submit for node 6940951d575bc060ce16123c 2025-12-17 23:21:51,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:51,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951d575bc060ce16123c HTTP/1.1" 502 150 2025-12-17 23:21:51,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951d575bc060ce16123c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:51,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951d575bc060ce16123c 2025-12-17 23:21:51,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951d575bc060ce16123c HTTP/1.1" 502 150 2025-12-17 23:21:51,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951d575bc060ce16123c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:51,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409523575bc060ce16124b 2025-12-17 23:21:51,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409523575bc060ce16124b HTTP/1.1" 502 150 2025-12-17 23:21:51,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409523575bc060ce16124b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:53,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951e575bc060ce16123f 2025-12-17 23:21:53,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951e575bc060ce16123f HTTP/1.1" 502 150 2025-12-17 23:21:53,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951e575bc060ce16123f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:53,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940950b575bc060ce16121b 2025-12-17 23:21:53,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950b575bc060ce16121b HTTP/1.1" 502 150 2025-12-17 23:21:53,433 - lava_callback - ERROR - Error processing callback for node 6940950b575bc060ce16121b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940950b575bc060ce16121b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940950b575bc060ce16121b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940950b575bc060ce16121b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:53,435 - lava_callback - INFO - Starting async job submit for node 69409527575bc060ce161254 2025-12-17 23:21:53,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:53,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409527575bc060ce161254 HTTP/1.1" 502 150 2025-12-17 23:21:53,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409527575bc060ce161254'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:53,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409527575bc060ce161254 2025-12-17 23:21:53,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409527575bc060ce161254 HTTP/1.1" 502 150 2025-12-17 23:21:53,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409527575bc060ce161254'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:53,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951f575bc060ce161242 2025-12-17 23:21:53,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951f575bc060ce161242 HTTP/1.1" 502 150 2025-12-17 23:21:53,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951f575bc060ce161242'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:53,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951d575bc060ce16123c 2025-12-17 23:21:53,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951d575bc060ce16123c HTTP/1.1" 502 150 2025-12-17 23:21:53,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951d575bc060ce16123c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:55,089 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409524575bc060ce16124e 2025-12-17 23:21:55,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409524575bc060ce16124e HTTP/1.1" 502 150 2025-12-17 23:21:55,090 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409524575bc060ce16124e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:55,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409527575bc060ce161254 2025-12-17 23:21:55,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409527575bc060ce161254 HTTP/1.1" 502 150 2025-12-17 23:21:55,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409527575bc060ce161254'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:55,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409543575bc060ce16129f 2025-12-17 23:21:55,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409543575bc060ce16129f HTTP/1.1" 502 150 2025-12-17 23:21:55,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409543575bc060ce16129f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:55,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409520575bc060ce161245 2025-12-17 23:21:55,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409520575bc060ce161245 HTTP/1.1" 502 150 2025-12-17 23:21:55,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409520575bc060ce161245'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37644 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:21:56,602 - lava_callback - INFO - Received callback for node 6940980a575bc060ce1619ad INFO: 172.21.0.1:37648 - "POST /node/6940980a575bc060ce1619ad HTTP/1.0" 202 Accepted 2025-12-17 23:21:56,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409566575bc060ce161306 2025-12-17 23:21:56,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409566575bc060ce161306 HTTP/1.1" 502 150 2025-12-17 23:21:56,904 - lava_callback - ERROR - Error processing callback for node 69409566575bc060ce161306: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409566575bc060ce161306 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409566575bc060ce161306 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409566575bc060ce161306 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:56,907 - lava_callback - INFO - Starting async job submit for node 6940952a575bc060ce16125c 2025-12-17 23:21:56,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:56,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409516575bc060ce16122d 2025-12-17 23:21:56,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409516575bc060ce16122d HTTP/1.1" 502 150 2025-12-17 23:21:56,914 - lava_callback - ERROR - Error processing callback for node 69409516575bc060ce16122d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409516575bc060ce16122d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409516575bc060ce16122d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409516575bc060ce16122d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:56,917 - lava_callback - INFO - Starting async job submit for node 6940952b575bc060ce16125f 2025-12-17 23:21:56,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:56,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952a575bc060ce16125c HTTP/1.1" 502 150 2025-12-17 23:21:56,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952a575bc060ce16125c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:56,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952a575bc060ce16125c 2025-12-17 23:21:56,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952a575bc060ce16125c HTTP/1.1" 502 150 2025-12-17 23:21:56,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952a575bc060ce16125c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:56,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952b575bc060ce16125f HTTP/1.1" 502 150 2025-12-17 23:21:56,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952b575bc060ce16125f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:56,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952b575bc060ce16125f 2025-12-17 23:21:56,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952b575bc060ce16125f HTTP/1.1" 502 150 2025-12-17 23:21:56,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952b575bc060ce16125f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:57,276 - lava_callback - INFO - Received callback for node 6940980c575bc060ce1619b8 INFO: 172.21.0.1:37662 - "POST /node/6940980c575bc060ce1619b8 HTTP/1.0" 202 Accepted 2025-12-17 23:21:57,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951d575bc060ce16123c 2025-12-17 23:21:57,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951d575bc060ce16123c HTTP/1.1" 502 150 2025-12-17 23:21:57,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951d575bc060ce16123c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:57,979 - lava_callback - INFO - Received callback for node 6940980f575bc060ce1619bf INFO: 172.21.0.1:37672 - "POST /node/6940980f575bc060ce1619bf HTTP/1.0" 202 Accepted 2025-12-17 23:21:57,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409518575bc060ce161230 2025-12-17 23:21:58,659 - lava_callback - INFO - Received callback for node 69409811575bc060ce1619c2 2025-12-17 23:21:58,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409526575bc060ce161251 INFO: 172.21.0.1:37682 - "POST /node/69409811575bc060ce1619c2 HTTP/1.0" 202 Accepted 2025-12-17 23:21:58,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409526575bc060ce161251 HTTP/1.1" 502 150 2025-12-17 23:21:58,670 - lava_callback - ERROR - Error processing callback for node 69409526575bc060ce161251: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409526575bc060ce161251 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409526575bc060ce161251 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409526575bc060ce161251 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:58,672 - lava_callback - INFO - Starting async job submit for node 6940952c575bc060ce161262 2025-12-17 23:21:58,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:58,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952c575bc060ce161262 HTTP/1.1" 502 150 2025-12-17 23:21:58,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952c575bc060ce161262'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:58,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952c575bc060ce161262 2025-12-17 23:21:58,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952c575bc060ce161262 HTTP/1.1" 502 150 2025-12-17 23:21:58,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952c575bc060ce161262'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:58,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409519575bc060ce161233 2025-12-17 23:21:58,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409519575bc060ce161233 HTTP/1.1" 502 150 2025-12-17 23:21:58,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409519575bc060ce161233'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:58,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952a575bc060ce16125c 2025-12-17 23:21:58,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952a575bc060ce16125c HTTP/1.1" 502 150 2025-12-17 23:21:58,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952a575bc060ce16125c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:58,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952b575bc060ce16125f 2025-12-17 23:21:58,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952b575bc060ce16125f HTTP/1.1" 502 150 2025-12-17 23:21:58,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952b575bc060ce16125f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:59,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940950c575bc060ce16121e HTTP/1.1" 502 150 2025-12-17 23:21:59,216 - lava_callback - ERROR - Error processing callback for node 6940950c575bc060ce16121e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940950c575bc060ce16121e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940950c575bc060ce16121e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940950c575bc060ce16121e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:21:59,218 - lava_callback - INFO - Starting async job submit for node 6940952e575bc060ce161265 2025-12-17 23:21:59,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:21:59,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952e575bc060ce161265 HTTP/1.1" 502 150 2025-12-17 23:21:59,226 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952e575bc060ce161265'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:59,226 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952e575bc060ce161265 2025-12-17 23:21:59,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952e575bc060ce161265 HTTP/1.1" 502 150 2025-12-17 23:21:59,227 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952e575bc060ce161265'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:59,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409527575bc060ce161254 2025-12-17 23:21:59,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409527575bc060ce161254 HTTP/1.1" 502 150 2025-12-17 23:21:59,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409527575bc060ce161254'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:21:59,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409523575bc060ce16124b 2025-12-17 23:21:59,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409523575bc060ce16124b HTTP/1.1" 502 150 2025-12-17 23:21:59,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409523575bc060ce16124b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:00,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951b575bc060ce161239 2025-12-17 23:22:00,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951b575bc060ce161239 HTTP/1.1" 502 150 2025-12-17 23:22:00,540 - lava_callback - ERROR - Error processing callback for node 6940951b575bc060ce161239: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940951b575bc060ce161239 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940951b575bc060ce161239 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940951b575bc060ce161239 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:00,542 - lava_callback - INFO - Starting async job submit for node 69409529575bc060ce161257 2025-12-17 23:22:00,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:00,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409529575bc060ce161257 HTTP/1.1" 502 150 2025-12-17 23:22:00,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409529575bc060ce161257'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:00,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409529575bc060ce161257 2025-12-17 23:22:00,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409529575bc060ce161257 HTTP/1.1" 502 150 2025-12-17 23:22:00,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409529575bc060ce161257'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:00,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952c575bc060ce161262 2025-12-17 23:22:00,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952c575bc060ce161262 HTTP/1.1" 502 150 2025-12-17 23:22:00,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952c575bc060ce161262'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:01,227 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952e575bc060ce161265 2025-12-17 23:22:01,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952e575bc060ce161265 HTTP/1.1" 502 150 2025-12-17 23:22:01,231 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952e575bc060ce161265'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:02,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409529575bc060ce161257 2025-12-17 23:22:02,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952a575bc060ce16125c 2025-12-17 23:22:02,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952a575bc060ce16125c HTTP/1.1" 502 150 2025-12-17 23:22:02,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952a575bc060ce16125c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:02,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952b575bc060ce16125f 2025-12-17 23:22:02,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952b575bc060ce16125f HTTP/1.1" 502 150 2025-12-17 23:22:02,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952b575bc060ce16125f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:03,090 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409524575bc060ce16124e 2025-12-17 23:22:03,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409524575bc060ce16124e HTTP/1.1" 502 150 2025-12-17 23:22:03,092 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409524575bc060ce16124e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52522 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:22:03,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409521575bc060ce161248 HTTP/1.1" 502 150 2025-12-17 23:22:03,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409521575bc060ce161248'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:04,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952c575bc060ce161262 2025-12-17 23:22:04,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952c575bc060ce161262 HTTP/1.1" 502 150 2025-12-17 23:22:04,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952c575bc060ce161262'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:05,231 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952e575bc060ce161265 2025-12-17 23:22:05,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952e575bc060ce161265 HTTP/1.1" 502 150 2025-12-17 23:22:05,232 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952e575bc060ce161265'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:05,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951d575bc060ce16123c 2025-12-17 23:22:05,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951d575bc060ce16123c HTTP/1.1" 502 150 2025-12-17 23:22:05,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940951d575bc060ce16123c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52530 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:22:07,227 - lava_callback - INFO - Received callback for node 69409812575bc060ce1619c5 INFO: 172.21.0.1:52532 - "POST /node/69409812575bc060ce1619c5 HTTP/1.0" 202 Accepted 2025-12-17 23:22:07,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409527575bc060ce161254 2025-12-17 23:22:07,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409527575bc060ce161254 HTTP/1.1" 502 150 2025-12-17 23:22:07,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409527575bc060ce161254'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:07,903 - lava_callback - INFO - Received callback for node 69409816575bc060ce1619cd INFO: 172.21.0.1:52542 - "POST /node/69409816575bc060ce1619cd HTTP/1.0" 202 Accepted 2025-12-17 23:22:08,010 - lava_callback - INFO - Received callback for node 69409815575bc060ce1619ca INFO: 172.21.0.1:52544 - "POST /node/69409815575bc060ce1619ca HTTP/1.0" 202 Accepted 2025-12-17 23:22:08,812 - lava_callback - INFO - Received callback for node 6940981a575bc060ce1619d3 INFO: 172.21.0.1:52546 - "POST /node/6940981a575bc060ce1619d3 HTTP/1.0" 202 Accepted 2025-12-17 23:22:09,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951e575bc060ce16123f 2025-12-17 23:22:09,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951e575bc060ce16123f HTTP/1.1" 502 150 2025-12-17 23:22:09,026 - lava_callback - ERROR - Error processing callback for node 6940951e575bc060ce16123f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940951e575bc060ce16123f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940951e575bc060ce16123f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940951e575bc060ce16123f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:09,028 - lava_callback - INFO - Starting async job submit for node 69409537575bc060ce161277 2025-12-17 23:22:09,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:09,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409537575bc060ce161277 HTTP/1.1" 502 150 2025-12-17 23:22:09,041 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409537575bc060ce161277'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:09,041 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409537575bc060ce161277 2025-12-17 23:22:09,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409537575bc060ce161277 HTTP/1.1" 502 150 2025-12-17 23:22:09,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409537575bc060ce161277'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:09,589 - lava_callback - INFO - Received callback for node 6940981c575bc060ce1619d9 INFO: 172.21.0.1:52548 - "POST /node/6940981c575bc060ce1619d9 HTTP/1.0" 202 Accepted 2025-12-17 23:22:09,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951f575bc060ce161242 2025-12-17 23:22:09,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951f575bc060ce161242 HTTP/1.1" 502 150 2025-12-17 23:22:09,803 - lava_callback - ERROR - Error processing callback for node 6940951f575bc060ce161242: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940951f575bc060ce161242 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940951f575bc060ce161242 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940951f575bc060ce161242 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:09,806 - lava_callback - INFO - Starting async job submit for node 69409536575bc060ce161274 2025-12-17 23:22:09,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:09,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409536575bc060ce161274 HTTP/1.1" 502 150 2025-12-17 23:22:09,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409536575bc060ce161274'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:09,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409536575bc060ce161274 2025-12-17 23:22:09,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409536575bc060ce161274 HTTP/1.1" 502 150 2025-12-17 23:22:09,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409536575bc060ce161274'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:10,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952a575bc060ce16125c 2025-12-17 23:22:10,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952a575bc060ce16125c HTTP/1.1" 502 150 2025-12-17 23:22:10,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952a575bc060ce16125c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:10,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952b575bc060ce16125f 2025-12-17 23:22:10,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952b575bc060ce16125f HTTP/1.1" 502 150 2025-12-17 23:22:10,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952b575bc060ce16125f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:11,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409537575bc060ce161277 2025-12-17 23:22:11,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409537575bc060ce161277 HTTP/1.1" 502 150 2025-12-17 23:22:11,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409537575bc060ce161277'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:11,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409521575bc060ce161248 2025-12-17 23:22:11,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409521575bc060ce161248 HTTP/1.1" 502 150 2025-12-17 23:22:11,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409521575bc060ce161248'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:11,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409543575bc060ce16129f 2025-12-17 23:22:11,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409543575bc060ce16129f HTTP/1.1" 502 150 2025-12-17 23:22:11,699 - lava_callback - ERROR - Error processing callback for node 69409543575bc060ce16129f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409543575bc060ce16129f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409543575bc060ce16129f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409543575bc060ce16129f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:11,701 - lava_callback - INFO - Starting async job submit for node 694097f8575bc060ce16197b 2025-12-17 23:22:11,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:11,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f8575bc060ce16197b HTTP/1.1" 502 150 2025-12-17 23:22:11,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f8575bc060ce16197b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:11,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f8575bc060ce16197b 2025-12-17 23:22:11,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f8575bc060ce16197b HTTP/1.1" 502 150 2025-12-17 23:22:11,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f8575bc060ce16197b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:11,829 - lava_callback - INFO - Received callback for node 6940981b575bc060ce1619d6 INFO: 172.21.0.1:33650 - "POST /node/6940981b575bc060ce1619d6 HTTP/1.0" 202 Accepted 2025-12-17 23:22:11,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409536575bc060ce161274 2025-12-17 23:22:11,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409536575bc060ce161274 HTTP/1.1" 502 150 2025-12-17 23:22:11,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409536575bc060ce161274'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:11,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409520575bc060ce161245 2025-12-17 23:22:11,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409520575bc060ce161245 HTTP/1.1" 502 150 2025-12-17 23:22:11,955 - lava_callback - ERROR - Error processing callback for node 69409520575bc060ce161245: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409520575bc060ce161245 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409520575bc060ce161245 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409520575bc060ce161245 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:11,956 - lava_callback - INFO - Starting async job submit for node 6940952f575bc060ce161268 2025-12-17 23:22:11,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:11,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952f575bc060ce161268 HTTP/1.1" 502 150 2025-12-17 23:22:11,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952f575bc060ce161268'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:11,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952f575bc060ce161268 2025-12-17 23:22:11,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952f575bc060ce161268 HTTP/1.1" 502 150 2025-12-17 23:22:11,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952f575bc060ce161268'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:12,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952c575bc060ce161262 2025-12-17 23:22:12,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952c575bc060ce161262 HTTP/1.1" 502 150 2025-12-17 23:22:12,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952c575bc060ce161262'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:13,233 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952e575bc060ce161265 2025-12-17 23:22:13,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952e575bc060ce161265 HTTP/1.1" 502 150 2025-12-17 23:22:13,234 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952e575bc060ce161265'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:13,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409518575bc060ce161230 HTTP/1.1" 502 150 2025-12-17 23:22:13,296 - lava_callback - ERROR - Error processing callback for node 69409518575bc060ce161230: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409518575bc060ce161230 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409518575bc060ce161230 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409518575bc060ce161230 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:13,298 - lava_callback - INFO - Starting async job submit for node 69409531575bc060ce16126b 2025-12-17 23:22:13,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:13,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409531575bc060ce16126b HTTP/1.1" 502 150 2025-12-17 23:22:13,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409531575bc060ce16126b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:13,304 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409531575bc060ce16126b 2025-12-17 23:22:13,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409531575bc060ce16126b HTTP/1.1" 502 150 2025-12-17 23:22:13,305 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409531575bc060ce16126b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:13,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f8575bc060ce16197b 2025-12-17 23:22:13,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f8575bc060ce16197b HTTP/1.1" 502 150 2025-12-17 23:22:13,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f8575bc060ce16197b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:13,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952f575bc060ce161268 2025-12-17 23:22:13,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952f575bc060ce161268 HTTP/1.1" 502 150 2025-12-17 23:22:13,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952f575bc060ce161268'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:14,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409519575bc060ce161233 2025-12-17 23:22:15,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409537575bc060ce161277 2025-12-17 23:22:15,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409537575bc060ce161277 HTTP/1.1" 502 150 2025-12-17 23:22:15,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409537575bc060ce161277'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33664 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:22:15,305 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409531575bc060ce16126b 2025-12-17 23:22:15,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409531575bc060ce16126b HTTP/1.1" 502 150 2025-12-17 23:22:15,306 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409531575bc060ce16126b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:15,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409536575bc060ce161274 2025-12-17 23:22:15,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409536575bc060ce161274 HTTP/1.1" 502 150 2025-12-17 23:22:15,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409536575bc060ce161274'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:15,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409523575bc060ce16124b 2025-12-17 23:22:15,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409523575bc060ce16124b HTTP/1.1" 502 150 2025-12-17 23:22:15,887 - lava_callback - ERROR - Error processing callback for node 69409523575bc060ce16124b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409523575bc060ce16124b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409523575bc060ce16124b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409523575bc060ce16124b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:15,889 - lava_callback - INFO - Starting async job submit for node 69409534575bc060ce161271 2025-12-17 23:22:15,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:15,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409534575bc060ce161271 HTTP/1.1" 502 150 2025-12-17 23:22:15,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409534575bc060ce161271'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:15,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409534575bc060ce161271 2025-12-17 23:22:15,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409534575bc060ce161271 HTTP/1.1" 502 150 2025-12-17 23:22:15,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409534575bc060ce161271'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33674 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:22:16,636 - lava_callback - INFO - Received callback for node 69409818575bc060ce1619d0 INFO: 172.21.0.1:33684 - "POST /node/69409818575bc060ce1619d0 HTTP/1.0" 202 Accepted 2025-12-17 23:22:17,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f8575bc060ce16197b 2025-12-17 23:22:17,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f8575bc060ce16197b HTTP/1.1" 502 150 2025-12-17 23:22:17,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f8575bc060ce16197b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:17,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409529575bc060ce161257 HTTP/1.1" 502 150 2025-12-17 23:22:17,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409529575bc060ce161257'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:17,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409534575bc060ce161271 2025-12-17 23:22:17,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409534575bc060ce161271 HTTP/1.1" 502 150 2025-12-17 23:22:17,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409534575bc060ce161271'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:17,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952f575bc060ce161268 2025-12-17 23:22:17,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952f575bc060ce161268 HTTP/1.1" 502 150 2025-12-17 23:22:17,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952f575bc060ce161268'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:19,092 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409524575bc060ce16124e 2025-12-17 23:22:19,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409524575bc060ce16124e HTTP/1.1" 502 150 2025-12-17 23:22:19,094 - lava_callback - ERROR - Error processing callback for node 69409524575bc060ce16124e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409524575bc060ce16124e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409524575bc060ce16124e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409524575bc060ce16124e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:19,096 - lava_callback - INFO - Starting async job submit for node 69409539575bc060ce16127a 2025-12-17 23:22:19,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:19,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409539575bc060ce16127a HTTP/1.1" 502 150 2025-12-17 23:22:19,124 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409539575bc060ce16127a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:19,124 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409539575bc060ce16127a 2025-12-17 23:22:19,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409539575bc060ce16127a HTTP/1.1" 502 150 2025-12-17 23:22:19,125 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409539575bc060ce16127a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:19,307 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409531575bc060ce16126b 2025-12-17 23:22:19,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409531575bc060ce16126b HTTP/1.1" 502 150 2025-12-17 23:22:19,308 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409531575bc060ce16126b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:21,125 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409539575bc060ce16127a 2025-12-17 23:22:21,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409539575bc060ce16127a HTTP/1.1" 502 150 2025-12-17 23:22:21,126 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409539575bc060ce16127a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:21,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940951d575bc060ce16123c 2025-12-17 23:22:21,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940951d575bc060ce16123c HTTP/1.1" 502 150 2025-12-17 23:22:21,811 - lava_callback - ERROR - Error processing callback for node 6940951d575bc060ce16123c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940951d575bc060ce16123c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940951d575bc060ce16123c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940951d575bc060ce16123c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:21,813 - lava_callback - INFO - Starting async job submit for node 6940953e575bc060ce16128d 2025-12-17 23:22:21,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:21,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953e575bc060ce16128d HTTP/1.1" 502 150 2025-12-17 23:22:21,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953e575bc060ce16128d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:21,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953e575bc060ce16128d 2025-12-17 23:22:21,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953e575bc060ce16128d HTTP/1.1" 502 150 2025-12-17 23:22:21,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953e575bc060ce16128d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:21,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409529575bc060ce161257 2025-12-17 23:22:21,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409529575bc060ce161257 HTTP/1.1" 502 150 2025-12-17 23:22:21,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409529575bc060ce161257'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:21,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409534575bc060ce161271 2025-12-17 23:22:21,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409534575bc060ce161271 HTTP/1.1" 502 150 2025-12-17 23:22:21,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409534575bc060ce161271'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:23,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409537575bc060ce161277 2025-12-17 23:22:23,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409537575bc060ce161277 HTTP/1.1" 502 150 2025-12-17 23:22:23,047 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409537575bc060ce161277'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:23,447 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409527575bc060ce161254 2025-12-17 23:22:23,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409527575bc060ce161254 HTTP/1.1" 502 150 2025-12-17 23:22:23,449 - lava_callback - ERROR - Error processing callback for node 69409527575bc060ce161254: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409527575bc060ce161254 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409527575bc060ce161254 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409527575bc060ce161254 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:23,451 - lava_callback - INFO - Starting async job submit for node 6940953f575bc060ce161290 2025-12-17 23:22:23,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:23,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953f575bc060ce161290 HTTP/1.1" 502 150 2025-12-17 23:22:23,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953f575bc060ce161290'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:23,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953f575bc060ce161290 2025-12-17 23:22:23,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953f575bc060ce161290 HTTP/1.1" 502 150 2025-12-17 23:22:23,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953f575bc060ce161290'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:23,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953e575bc060ce16128d 2025-12-17 23:22:23,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953e575bc060ce16128d HTTP/1.1" 502 150 2025-12-17 23:22:23,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953e575bc060ce16128d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:23,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409536575bc060ce161274 2025-12-17 23:22:23,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409536575bc060ce161274 HTTP/1.1" 502 150 2025-12-17 23:22:23,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409536575bc060ce161274'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:25,127 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409539575bc060ce16127a 2025-12-17 23:22:25,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953f575bc060ce161290 2025-12-17 23:22:25,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953f575bc060ce161290 HTTP/1.1" 502 150 2025-12-17 23:22:25,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953f575bc060ce161290'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:25,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f8575bc060ce16197b 2025-12-17 23:22:25,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f8575bc060ce16197b HTTP/1.1" 502 150 2025-12-17 23:22:25,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f8575bc060ce16197b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:25,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952f575bc060ce161268 2025-12-17 23:22:25,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952f575bc060ce161268 HTTP/1.1" 502 150 2025-12-17 23:22:25,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940952f575bc060ce161268'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52874 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:22:26,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952a575bc060ce16125c 2025-12-17 23:22:26,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952a575bc060ce16125c HTTP/1.1" 502 150 2025-12-17 23:22:26,928 - lava_callback - ERROR - Error processing callback for node 6940952a575bc060ce16125c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940952a575bc060ce16125c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940952a575bc060ce16125c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940952a575bc060ce16125c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:26,930 - lava_callback - INFO - Starting async job submit for node 69409540575bc060ce161293 2025-12-17 23:22:26,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:26,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952b575bc060ce16125f 2025-12-17 23:22:26,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952b575bc060ce16125f HTTP/1.1" 502 150 2025-12-17 23:22:26,951 - lava_callback - ERROR - Error processing callback for node 6940952b575bc060ce16125f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940952b575bc060ce16125f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940952b575bc060ce16125f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940952b575bc060ce16125f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:26,953 - lava_callback - INFO - Starting async job submit for node 69409542575bc060ce161298 2025-12-17 23:22:26,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:26,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409540575bc060ce161293 HTTP/1.1" 502 150 2025-12-17 23:22:26,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409540575bc060ce161293'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:26,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409540575bc060ce161293 2025-12-17 23:22:26,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409540575bc060ce161293 HTTP/1.1" 502 150 2025-12-17 23:22:26,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409540575bc060ce161293'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:26,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409542575bc060ce161298 HTTP/1.1" 502 150 2025-12-17 23:22:26,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409542575bc060ce161298'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:26,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409542575bc060ce161298 2025-12-17 23:22:26,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409542575bc060ce161298 HTTP/1.1" 502 150 2025-12-17 23:22:26,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409542575bc060ce161298'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:27,241 - lava_callback - INFO - Received callback for node 6940981e575bc060ce1619dc INFO: 172.21.0.1:52882 - "POST /node/6940981e575bc060ce1619dc HTTP/1.0" 202 Accepted 2025-12-17 23:22:27,308 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409531575bc060ce16126b 2025-12-17 23:22:27,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409531575bc060ce16126b HTTP/1.1" 502 150 2025-12-17 23:22:27,309 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409531575bc060ce16126b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:27,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409521575bc060ce161248 2025-12-17 23:22:27,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409521575bc060ce161248 HTTP/1.1" 502 150 2025-12-17 23:22:27,571 - lava_callback - ERROR - Error processing callback for node 69409521575bc060ce161248: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409521575bc060ce161248 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409521575bc060ce161248 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409521575bc060ce161248 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:27,574 - lava_callback - INFO - Starting async job submit for node 6940953a575bc060ce16127f 2025-12-17 23:22:27,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:27,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953a575bc060ce16127f HTTP/1.1" 502 150 2025-12-17 23:22:27,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953a575bc060ce16127f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:27,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953a575bc060ce16127f 2025-12-17 23:22:27,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953a575bc060ce16127f HTTP/1.1" 502 150 2025-12-17 23:22:27,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953a575bc060ce16127f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:27,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953e575bc060ce16128d 2025-12-17 23:22:27,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953e575bc060ce16128d HTTP/1.1" 502 150 2025-12-17 23:22:27,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953e575bc060ce16128d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:27,891 - lava_callback - INFO - Received callback for node 69409823575bc060ce1619e6 INFO: 172.21.0.1:52888 - "POST /node/69409823575bc060ce1619e6 HTTP/1.0" 202 Accepted 2025-12-17 23:22:27,949 - lava_callback - INFO - Received callback for node 694124cd459ff5fc64d2bd7b INFO: 172.21.0.1:52904 - "POST /node/694124cd459ff5fc64d2bd7b HTTP/1.0" 202 Accepted 2025-12-17 23:22:27,963 - lava_callback - INFO - Received callback for node 69409821575bc060ce1619e3 INFO: 172.21.0.1:52910 - "POST /node/69409821575bc060ce1619e3 HTTP/1.0" 202 Accepted 2025-12-17 23:22:28,561 - lava_callback - INFO - Received callback for node 69409826575bc060ce1619ec INFO: 172.21.0.1:52918 - "POST /node/69409826575bc060ce1619ec HTTP/1.0" 202 Accepted 2025-12-17 23:22:28,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952c575bc060ce161262 2025-12-17 23:22:28,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952c575bc060ce161262 HTTP/1.1" 502 150 2025-12-17 23:22:28,691 - lava_callback - ERROR - Error processing callback for node 6940952c575bc060ce161262: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940952c575bc060ce161262 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940952c575bc060ce161262 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940952c575bc060ce161262 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:28,693 - lava_callback - INFO - Starting async job submit for node 6940953b575bc060ce161284 2025-12-17 23:22:28,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:28,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953b575bc060ce161284 HTTP/1.1" 502 150 2025-12-17 23:22:28,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953b575bc060ce161284'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:28,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953b575bc060ce161284 2025-12-17 23:22:28,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953b575bc060ce161284 HTTP/1.1" 502 150 2025-12-17 23:22:28,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953b575bc060ce161284'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:28,759 - lava_callback - INFO - Received callback for node 69409825575bc060ce1619e9 INFO: 172.21.0.1:52930 - "POST /node/69409825575bc060ce1619e9 HTTP/1.0" 202 Accepted 2025-12-17 23:22:28,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409540575bc060ce161293 2025-12-17 23:22:28,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409540575bc060ce161293 HTTP/1.1" 502 150 2025-12-17 23:22:28,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409540575bc060ce161293'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:28,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409542575bc060ce161298 2025-12-17 23:22:28,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409542575bc060ce161298 HTTP/1.1" 502 150 2025-12-17 23:22:28,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409542575bc060ce161298'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:29,234 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952e575bc060ce161265 2025-12-17 23:22:29,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952e575bc060ce161265 HTTP/1.1" 502 150 2025-12-17 23:22:29,236 - lava_callback - ERROR - Error processing callback for node 6940952e575bc060ce161265: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940952e575bc060ce161265 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940952e575bc060ce161265 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940952e575bc060ce161265 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:29,239 - lava_callback - INFO - Starting async job submit for node 6940953c575bc060ce161289 2025-12-17 23:22:29,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:29,240 - lava_callback - INFO - Received callback for node 69409828575bc060ce1619f4 INFO: 172.21.0.1:52934 - "POST /node/69409828575bc060ce1619f4 HTTP/1.0" 202 Accepted 2025-12-17 23:22:29,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953c575bc060ce161289 HTTP/1.1" 502 150 2025-12-17 23:22:29,257 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953c575bc060ce161289'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:29,257 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953c575bc060ce161289 2025-12-17 23:22:29,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953c575bc060ce161289 HTTP/1.1" 502 150 2025-12-17 23:22:29,258 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953c575bc060ce161289'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:29,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953f575bc060ce161290 2025-12-17 23:22:29,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953f575bc060ce161290 HTTP/1.1" 502 150 2025-12-17 23:22:29,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953f575bc060ce161290'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:29,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953a575bc060ce16127f 2025-12-17 23:22:29,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953a575bc060ce16127f HTTP/1.1" 502 150 2025-12-17 23:22:29,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953a575bc060ce16127f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:29,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409529575bc060ce161257 2025-12-17 23:22:29,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409529575bc060ce161257 HTTP/1.1" 502 150 2025-12-17 23:22:29,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409529575bc060ce161257'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:29,915 - lava_callback - INFO - Received callback for node 6940982a575bc060ce1619f7 INFO: 172.21.0.1:58368 - "POST /node/6940982a575bc060ce1619f7 HTTP/1.0" 202 Accepted 2025-12-17 23:22:29,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409534575bc060ce161271 2025-12-17 23:22:29,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409534575bc060ce161271 HTTP/1.1" 502 150 2025-12-17 23:22:29,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409534575bc060ce161271'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:30,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409519575bc060ce161233 HTTP/1.1" 502 150 2025-12-17 23:22:30,196 - lava_callback - ERROR - Error processing callback for node 69409519575bc060ce161233: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409519575bc060ce161233 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409519575bc060ce161233 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409519575bc060ce161233 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:30,199 - lava_callback - INFO - Starting async job submit for node 6940954e575bc060ce1612ba 2025-12-17 23:22:30,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:30,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954e575bc060ce1612ba HTTP/1.1" 502 150 2025-12-17 23:22:30,209 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954e575bc060ce1612ba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:30,209 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954e575bc060ce1612ba 2025-12-17 23:22:30,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954e575bc060ce1612ba HTTP/1.1" 502 150 2025-12-17 23:22:30,210 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954e575bc060ce1612ba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:30,580 - lava_callback - INFO - Received callback for node 6940982c575bc060ce1619fe INFO: 172.21.0.1:58378 - "POST /node/6940982c575bc060ce1619fe HTTP/1.0" 202 Accepted 2025-12-17 23:22:30,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953b575bc060ce161284 2025-12-17 23:22:30,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953b575bc060ce161284 HTTP/1.1" 502 150 2025-12-17 23:22:30,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953b575bc060ce161284'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:31,238 - lava_callback - INFO - Received callback for node 69409831575bc060ce161a04 INFO: 172.21.0.1:58388 - "POST /node/69409831575bc060ce161a04 HTTP/1.0" 202 Accepted 2025-12-17 23:22:31,258 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953c575bc060ce161289 2025-12-17 23:22:31,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953c575bc060ce161289 HTTP/1.1" 502 150 2025-12-17 23:22:31,259 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953c575bc060ce161289'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:32,210 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954e575bc060ce1612ba 2025-12-17 23:22:32,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954e575bc060ce1612ba HTTP/1.1" 502 150 2025-12-17 23:22:32,212 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954e575bc060ce1612ba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:32,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409540575bc060ce161293 2025-12-17 23:22:32,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409540575bc060ce161293 HTTP/1.1" 502 150 2025-12-17 23:22:32,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409540575bc060ce161293'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:32,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409542575bc060ce161298 2025-12-17 23:22:32,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409542575bc060ce161298 HTTP/1.1" 502 150 2025-12-17 23:22:32,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409542575bc060ce161298'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:33,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953a575bc060ce16127f 2025-12-17 23:22:33,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953a575bc060ce16127f HTTP/1.1" 502 150 2025-12-17 23:22:33,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953a575bc060ce16127f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:34,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953b575bc060ce161284 2025-12-17 23:22:34,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953b575bc060ce161284 HTTP/1.1" 502 150 2025-12-17 23:22:34,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953b575bc060ce161284'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:35,260 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953c575bc060ce161289 2025-12-17 23:22:35,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953c575bc060ce161289 HTTP/1.1" 502 150 2025-12-17 23:22:35,261 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953c575bc060ce161289'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:35,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953e575bc060ce16128d 2025-12-17 23:22:35,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953e575bc060ce16128d HTTP/1.1" 502 150 2025-12-17 23:22:35,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953e575bc060ce16128d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:36,212 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954e575bc060ce1612ba 2025-12-17 23:22:36,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954e575bc060ce1612ba HTTP/1.1" 502 150 2025-12-17 23:22:36,213 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954e575bc060ce1612ba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:36,234 - lava_callback - INFO - Received callback for node 69409958575bc060ce161cb9 INFO: 172.21.0.1:58392 - "POST /node/69409958575bc060ce161cb9 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:58406 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:22:36,620 - lava_callback - INFO - Received callback for node 6940982e575bc060ce161a01 INFO: 172.21.0.1:58422 - "POST /node/6940982e575bc060ce161a01 HTTP/1.0" 202 Accepted 2025-12-17 23:22:37,271 - lava_callback - INFO - Received callback for node 69409833575bc060ce161a0d INFO: 172.21.0.1:58430 - "POST /node/69409833575bc060ce161a0d HTTP/1.0" 202 Accepted 2025-12-17 23:22:37,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953f575bc060ce161290 2025-12-17 23:22:37,942 - lava_callback - INFO - Received callback for node 69409836575bc060ce161a18 INFO: 172.21.0.1:58436 - "POST /node/69409836575bc060ce161a18 HTTP/1.0" 202 Accepted 2025-12-17 23:22:38,622 - lava_callback - INFO - Received callback for node 69409839575bc060ce161a1e INFO: 172.21.0.1:58448 - "POST /node/69409839575bc060ce161a1e HTTP/1.0" 202 Accepted 2025-12-17 23:22:39,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409537575bc060ce161277 2025-12-17 23:22:39,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409537575bc060ce161277 HTTP/1.1" 502 150 2025-12-17 23:22:39,049 - lava_callback - ERROR - Error processing callback for node 69409537575bc060ce161277: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409537575bc060ce161277 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409537575bc060ce161277 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409537575bc060ce161277 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:39,051 - lava_callback - INFO - Starting async job submit for node 69409547575bc060ce1612a5 2025-12-17 23:22:39,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:39,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409547575bc060ce1612a5 HTTP/1.1" 502 150 2025-12-17 23:22:39,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409547575bc060ce1612a5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:39,080 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409547575bc060ce1612a5 2025-12-17 23:22:39,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409547575bc060ce1612a5 HTTP/1.1" 502 150 2025-12-17 23:22:39,081 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409547575bc060ce1612a5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:39,281 - lava_callback - INFO - Received callback for node 6940983b575bc060ce161a21 INFO: 172.21.0.1:58458 - "POST /node/6940983b575bc060ce161a21 HTTP/1.0" 202 Accepted 2025-12-17 23:22:39,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409536575bc060ce161274 2025-12-17 23:22:39,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409536575bc060ce161274 HTTP/1.1" 502 150 2025-12-17 23:22:39,857 - lava_callback - ERROR - Error processing callback for node 69409536575bc060ce161274: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409536575bc060ce161274 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409536575bc060ce161274 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409536575bc060ce161274 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:39,859 - lava_callback - INFO - Starting async job submit for node 69409548575bc060ce1612a8 2025-12-17 23:22:39,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:39,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409548575bc060ce1612a8 HTTP/1.1" 502 150 2025-12-17 23:22:39,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409548575bc060ce1612a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:39,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409548575bc060ce1612a8 2025-12-17 23:22:39,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409548575bc060ce1612a8 HTTP/1.1" 502 150 2025-12-17 23:22:39,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409548575bc060ce1612a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:40,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409539575bc060ce16127a HTTP/1.1" 502 150 2025-12-17 23:22:40,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409539575bc060ce16127a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:40,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409540575bc060ce161293 2025-12-17 23:22:40,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409540575bc060ce161293 HTTP/1.1" 502 150 2025-12-17 23:22:40,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409540575bc060ce161293'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:40,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409542575bc060ce161298 2025-12-17 23:22:40,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409542575bc060ce161298 HTTP/1.1" 502 150 2025-12-17 23:22:40,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409542575bc060ce161298'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:41,081 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409547575bc060ce1612a5 2025-12-17 23:22:41,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953a575bc060ce16127f 2025-12-17 23:22:41,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953a575bc060ce16127f HTTP/1.1" 502 150 2025-12-17 23:22:41,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953a575bc060ce16127f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:41,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f8575bc060ce16197b 2025-12-17 23:22:41,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f8575bc060ce16197b HTTP/1.1" 502 150 2025-12-17 23:22:41,731 - lava_callback - ERROR - Error processing callback for node 694097f8575bc060ce16197b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097f8575bc060ce16197b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097f8575bc060ce16197b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097f8575bc060ce16197b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:41,734 - lava_callback - INFO - Starting async job submit for node 69409549575bc060ce1612ab 2025-12-17 23:22:41,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:41,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409549575bc060ce1612ab HTTP/1.1" 502 150 2025-12-17 23:22:41,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409549575bc060ce1612ab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:41,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409549575bc060ce1612ab 2025-12-17 23:22:41,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409549575bc060ce1612ab HTTP/1.1" 502 150 2025-12-17 23:22:41,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409549575bc060ce1612ab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:41,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409548575bc060ce1612a8 2025-12-17 23:22:41,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409548575bc060ce1612a8 HTTP/1.1" 502 150 2025-12-17 23:22:41,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409548575bc060ce1612a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:41,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940952f575bc060ce161268 2025-12-17 23:22:41,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940952f575bc060ce161268 HTTP/1.1" 502 150 2025-12-17 23:22:41,969 - lava_callback - ERROR - Error processing callback for node 6940952f575bc060ce161268: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940952f575bc060ce161268 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940952f575bc060ce161268 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940952f575bc060ce161268 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:41,971 - lava_callback - INFO - Starting async job submit for node 6940954a575bc060ce1612ae 2025-12-17 23:22:41,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:42,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954a575bc060ce1612ae HTTP/1.1" 502 150 2025-12-17 23:22:42,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954a575bc060ce1612ae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:42,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954a575bc060ce1612ae 2025-12-17 23:22:42,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954a575bc060ce1612ae HTTP/1.1" 502 150 2025-12-17 23:22:42,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954a575bc060ce1612ae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:42,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953b575bc060ce161284 2025-12-17 23:22:42,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953b575bc060ce161284 HTTP/1.1" 502 150 2025-12-17 23:22:42,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953b575bc060ce161284'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:43,261 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953c575bc060ce161289 2025-12-17 23:22:43,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953c575bc060ce161289 HTTP/1.1" 502 150 2025-12-17 23:22:43,262 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953c575bc060ce161289'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:43,310 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409531575bc060ce16126b 2025-12-17 23:22:43,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409531575bc060ce16126b HTTP/1.1" 502 150 2025-12-17 23:22:43,311 - lava_callback - ERROR - Error processing callback for node 69409531575bc060ce16126b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409531575bc060ce16126b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409531575bc060ce16126b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409531575bc060ce16126b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:43,313 - lava_callback - INFO - Starting async job submit for node 6940954c575bc060ce1612b1 2025-12-17 23:22:43,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:43,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954c575bc060ce1612b1 HTTP/1.1" 502 150 2025-12-17 23:22:43,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954c575bc060ce1612b1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:43,319 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954c575bc060ce1612b1 2025-12-17 23:22:43,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954c575bc060ce1612b1 HTTP/1.1" 502 150 2025-12-17 23:22:43,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954c575bc060ce1612b1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:43,740 - lava_callback - INFO - Received callback for node 6940c5431e3c24e95681d6b6 2025-12-17 23:22:43,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409549575bc060ce1612ab INFO: 172.21.0.1:43422 - "POST /node/6940c5431e3c24e95681d6b6 HTTP/1.0" 202 Accepted 2025-12-17 23:22:43,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409549575bc060ce1612ab HTTP/1.1" 502 150 2025-12-17 23:22:43,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409549575bc060ce1612ab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:44,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954a575bc060ce1612ae 2025-12-17 23:22:44,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954a575bc060ce1612ae HTTP/1.1" 502 150 2025-12-17 23:22:44,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954a575bc060ce1612ae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:44,213 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954e575bc060ce1612ba 2025-12-17 23:22:44,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954e575bc060ce1612ba HTTP/1.1" 502 150 2025-12-17 23:22:44,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954e575bc060ce1612ba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:44,868 - lava_callback - INFO - Received callback for node 69409804575bc060ce161997 INFO: 172.21.0.1:43426 - "POST /node/69409804575bc060ce161997 HTTP/1.0" 202 Accepted 2025-12-17 23:22:45,320 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954c575bc060ce1612b1 2025-12-17 23:22:45,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954c575bc060ce1612b1 HTTP/1.1" 502 150 2025-12-17 23:22:45,321 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954c575bc060ce1612b1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:45,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409548575bc060ce1612a8 2025-12-17 23:22:45,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409548575bc060ce1612a8 HTTP/1.1" 502 150 2025-12-17 23:22:45,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409548575bc060ce1612a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:45,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409529575bc060ce161257 2025-12-17 23:22:45,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409529575bc060ce161257 HTTP/1.1" 502 150 2025-12-17 23:22:45,908 - lava_callback - ERROR - Error processing callback for node 69409529575bc060ce161257: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409529575bc060ce161257 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409529575bc060ce161257 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409529575bc060ce161257 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:45,910 - lava_callback - INFO - Starting async job submit for node 6940954e575bc060ce1612be 2025-12-17 23:22:45,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:45,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954e575bc060ce1612be HTTP/1.1" 502 150 2025-12-17 23:22:45,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954e575bc060ce1612be'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:45,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954e575bc060ce1612be 2025-12-17 23:22:45,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954e575bc060ce1612be HTTP/1.1" 502 150 2025-12-17 23:22:45,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954e575bc060ce1612be'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:45,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409534575bc060ce161271 2025-12-17 23:22:45,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409534575bc060ce161271 HTTP/1.1" 502 150 2025-12-17 23:22:45,925 - lava_callback - ERROR - Error processing callback for node 69409534575bc060ce161271: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409534575bc060ce161271 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409534575bc060ce161271 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409534575bc060ce161271 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:45,926 - lava_callback - INFO - Starting async job submit for node 69409551575bc060ce1612cd 2025-12-17 23:22:45,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:45,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409551575bc060ce1612cd HTTP/1.1" 502 150 2025-12-17 23:22:45,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409551575bc060ce1612cd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:45,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409551575bc060ce1612cd 2025-12-17 23:22:45,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409551575bc060ce1612cd HTTP/1.1" 502 150 2025-12-17 23:22:45,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409551575bc060ce1612cd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43440 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:22:47,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409549575bc060ce1612ab 2025-12-17 23:22:47,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409549575bc060ce1612ab HTTP/1.1" 502 150 2025-12-17 23:22:47,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409549575bc060ce1612ab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:47,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954e575bc060ce1612be 2025-12-17 23:22:47,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954e575bc060ce1612be HTTP/1.1" 502 150 2025-12-17 23:22:47,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954e575bc060ce1612be'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:47,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409551575bc060ce1612cd 2025-12-17 23:22:47,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409551575bc060ce1612cd HTTP/1.1" 502 150 2025-12-17 23:22:47,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409551575bc060ce1612cd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:48,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954a575bc060ce1612ae 2025-12-17 23:22:48,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954a575bc060ce1612ae HTTP/1.1" 502 150 2025-12-17 23:22:48,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954a575bc060ce1612ae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:48,347 - lava_callback - INFO - Received callback for node 6940983d575bc060ce161a24 INFO: 172.21.0.1:43454 - "POST /node/6940983d575bc060ce161a24 HTTP/1.0" 202 Accepted 2025-12-17 23:22:48,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409539575bc060ce16127a 2025-12-17 23:22:48,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409539575bc060ce16127a HTTP/1.1" 502 150 2025-12-17 23:22:48,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409539575bc060ce16127a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:49,095 - lava_callback - INFO - Received callback for node 6940983e575bc060ce161a27 INFO: 172.21.0.1:43458 - "POST /node/6940983e575bc060ce161a27 HTTP/1.0" 202 Accepted 2025-12-17 23:22:49,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954c575bc060ce1612b1 2025-12-17 23:22:49,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954c575bc060ce1612b1 HTTP/1.1" 502 150 2025-12-17 23:22:49,322 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954c575bc060ce1612b1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:49,834 - lava_callback - INFO - Received callback for node 69409843575bc060ce161a30 INFO: 172.21.0.1:48672 - "POST /node/69409843575bc060ce161a30 HTTP/1.0" 202 Accepted 2025-12-17 23:22:51,674 - lava_callback - INFO - Received callback for node 69409841575bc060ce161a2d INFO: 172.21.0.1:48678 - "POST /node/69409841575bc060ce161a2d HTTP/1.0" 202 Accepted 2025-12-17 23:22:51,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953e575bc060ce16128d 2025-12-17 23:22:51,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953e575bc060ce16128d HTTP/1.1" 502 150 2025-12-17 23:22:51,826 - lava_callback - ERROR - Error processing callback for node 6940953e575bc060ce16128d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940953e575bc060ce16128d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940953e575bc060ce16128d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940953e575bc060ce16128d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:51,828 - lava_callback - INFO - Starting async job submit for node 69409556575bc060ce1612d6 2025-12-17 23:22:51,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:51,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409556575bc060ce1612d6 HTTP/1.1" 502 150 2025-12-17 23:22:51,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409556575bc060ce1612d6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:51,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409556575bc060ce1612d6 2025-12-17 23:22:51,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409556575bc060ce1612d6 HTTP/1.1" 502 150 2025-12-17 23:22:51,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409556575bc060ce1612d6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:51,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954e575bc060ce1612be 2025-12-17 23:22:51,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954e575bc060ce1612be HTTP/1.1" 502 150 2025-12-17 23:22:51,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954e575bc060ce1612be'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:51,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409551575bc060ce1612cd 2025-12-17 23:22:51,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409551575bc060ce1612cd HTTP/1.1" 502 150 2025-12-17 23:22:51,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409551575bc060ce1612cd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:52,310 - lava_callback - INFO - Received callback for node 69409845575bc060ce161a36 INFO: 172.21.0.1:48688 - "POST /node/69409845575bc060ce161a36 HTTP/1.0" 202 Accepted 2025-12-17 23:22:52,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953f575bc060ce161290 HTTP/1.1" 502 150 2025-12-17 23:22:52,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940953f575bc060ce161290'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:53,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409556575bc060ce1612d6 2025-12-17 23:22:53,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409556575bc060ce1612d6 HTTP/1.1" 502 150 2025-12-17 23:22:53,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409556575bc060ce1612d6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:53,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409548575bc060ce1612a8 2025-12-17 23:22:53,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409548575bc060ce1612a8 HTTP/1.1" 502 150 2025-12-17 23:22:53,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409548575bc060ce1612a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:55,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409549575bc060ce1612ab 2025-12-17 23:22:55,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409549575bc060ce1612ab HTTP/1.1" 502 150 2025-12-17 23:22:55,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409549575bc060ce1612ab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:56,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954a575bc060ce1612ae 2025-12-17 23:22:56,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954a575bc060ce1612ae HTTP/1.1" 502 150 2025-12-17 23:22:56,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954a575bc060ce1612ae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48698 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:22:56,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409547575bc060ce1612a5 HTTP/1.1" 502 150 2025-12-17 23:22:56,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409547575bc060ce1612a5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:56,912 - lava_callback - INFO - Received callback for node 6940983f575bc060ce161a2a INFO: 172.21.0.1:48712 - "POST /node/6940983f575bc060ce161a2a HTTP/1.0" 202 Accepted 2025-12-17 23:22:56,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409540575bc060ce161293 2025-12-17 23:22:56,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409542575bc060ce161298 2025-12-17 23:22:56,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409540575bc060ce161293 HTTP/1.1" 502 150 2025-12-17 23:22:56,967 - lava_callback - ERROR - Error processing callback for node 69409540575bc060ce161293: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409540575bc060ce161293 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409540575bc060ce161293 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409540575bc060ce161293 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:56,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409542575bc060ce161298 HTTP/1.1" 502 150 2025-12-17 23:22:56,969 - lava_callback - ERROR - Error processing callback for node 69409542575bc060ce161298: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409542575bc060ce161298 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409542575bc060ce161298 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409542575bc060ce161298 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:56,969 - lava_callback - INFO - Starting async job submit for node 69409553575bc060ce1612d0 2025-12-17 23:22:56,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:56,971 - lava_callback - INFO - Starting async job submit for node 69409550575bc060ce1612c4 2025-12-17 23:22:56,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:56,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409553575bc060ce1612d0 HTTP/1.1" 502 150 2025-12-17 23:22:56,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409553575bc060ce1612d0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:56,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409553575bc060ce1612d0 2025-12-17 23:22:56,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409553575bc060ce1612d0 HTTP/1.1" 502 150 2025-12-17 23:22:56,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409553575bc060ce1612d0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:57,322 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954c575bc060ce1612b1 2025-12-17 23:22:57,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954c575bc060ce1612b1 HTTP/1.1" 502 150 2025-12-17 23:22:57,323 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954c575bc060ce1612b1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:57,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953a575bc060ce16127f 2025-12-17 23:22:57,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953a575bc060ce16127f HTTP/1.1" 502 150 2025-12-17 23:22:57,590 - lava_callback - ERROR - Error processing callback for node 6940953a575bc060ce16127f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940953a575bc060ce16127f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940953a575bc060ce16127f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940953a575bc060ce16127f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:57,592 - lava_callback - INFO - Starting async job submit for node 69409555575bc060ce1612d3 2025-12-17 23:22:57,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:57,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409555575bc060ce1612d3 HTTP/1.1" 502 150 2025-12-17 23:22:57,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409555575bc060ce1612d3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:57,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409555575bc060ce1612d3 2025-12-17 23:22:57,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409555575bc060ce1612d3 HTTP/1.1" 502 150 2025-12-17 23:22:57,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409555575bc060ce1612d3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:57,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409556575bc060ce1612d6 2025-12-17 23:22:57,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409556575bc060ce1612d6 HTTP/1.1" 502 150 2025-12-17 23:22:57,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409556575bc060ce1612d6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:58,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953b575bc060ce161284 2025-12-17 23:22:58,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953b575bc060ce161284 HTTP/1.1" 502 150 2025-12-17 23:22:58,712 - lava_callback - ERROR - Error processing callback for node 6940953b575bc060ce161284: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940953b575bc060ce161284 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940953b575bc060ce161284 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940953b575bc060ce161284 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:58,714 - lava_callback - INFO - Starting async job submit for node 69409562575bc060ce1612f5 2025-12-17 23:22:58,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:58,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409562575bc060ce1612f5 HTTP/1.1" 502 150 2025-12-17 23:22:58,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409562575bc060ce1612f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:58,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409562575bc060ce1612f5 2025-12-17 23:22:58,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409562575bc060ce1612f5 HTTP/1.1" 502 150 2025-12-17 23:22:58,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409562575bc060ce1612f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:58,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409553575bc060ce1612d0 2025-12-17 23:22:58,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409553575bc060ce1612d0 HTTP/1.1" 502 150 2025-12-17 23:22:58,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409553575bc060ce1612d0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:59,262 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953c575bc060ce161289 2025-12-17 23:22:59,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953c575bc060ce161289 HTTP/1.1" 502 150 2025-12-17 23:22:59,267 - lava_callback - ERROR - Error processing callback for node 6940953c575bc060ce161289: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940953c575bc060ce161289 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940953c575bc060ce161289 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940953c575bc060ce161289 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:22:59,270 - lava_callback - INFO - Starting async job submit for node 69409557575bc060ce1612d9 2025-12-17 23:22:59,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:22:59,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409557575bc060ce1612d9 HTTP/1.1" 502 150 2025-12-17 23:22:59,309 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409557575bc060ce1612d9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:59,309 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409557575bc060ce1612d9 2025-12-17 23:22:59,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409557575bc060ce1612d9 HTTP/1.1" 502 150 2025-12-17 23:22:59,310 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409557575bc060ce1612d9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:59,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409555575bc060ce1612d3 2025-12-17 23:22:59,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409555575bc060ce1612d3 HTTP/1.1" 502 150 2025-12-17 23:22:59,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409555575bc060ce1612d3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:59,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954e575bc060ce1612be 2025-12-17 23:22:59,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954e575bc060ce1612be HTTP/1.1" 502 150 2025-12-17 23:22:59,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940954e575bc060ce1612be'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:22:59,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409551575bc060ce1612cd 2025-12-17 23:22:59,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409551575bc060ce1612cd HTTP/1.1" 502 150 2025-12-17 23:22:59,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409551575bc060ce1612cd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:00,215 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954e575bc060ce1612ba 2025-12-17 23:23:00,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954e575bc060ce1612ba HTTP/1.1" 502 150 2025-12-17 23:23:00,216 - lava_callback - ERROR - Error processing callback for node 6940954e575bc060ce1612ba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940954e575bc060ce1612ba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940954e575bc060ce1612ba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940954e575bc060ce1612ba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:00,219 - lava_callback - INFO - Starting async job submit for node 6940956a575bc060ce161312 2025-12-17 23:23:00,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:00,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956a575bc060ce161312 HTTP/1.1" 502 150 2025-12-17 23:23:00,237 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956a575bc060ce161312'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:00,237 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956a575bc060ce161312 2025-12-17 23:23:00,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956a575bc060ce161312 HTTP/1.1" 502 150 2025-12-17 23:23:00,238 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956a575bc060ce161312'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:00,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409547575bc060ce1612a5 2025-12-17 23:23:00,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409547575bc060ce1612a5 HTTP/1.1" 502 150 2025-12-17 23:23:00,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409547575bc060ce1612a5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:00,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409562575bc060ce1612f5 2025-12-17 23:23:00,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409562575bc060ce1612f5 HTTP/1.1" 502 150 2025-12-17 23:23:00,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409562575bc060ce1612f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:01,310 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409557575bc060ce1612d9 2025-12-17 23:23:01,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409557575bc060ce1612d9 HTTP/1.1" 502 150 2025-12-17 23:23:01,312 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409557575bc060ce1612d9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:02,239 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956a575bc060ce161312 2025-12-17 23:23:02,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956a575bc060ce161312 HTTP/1.1" 502 150 2025-12-17 23:23:02,240 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956a575bc060ce161312'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:02,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409553575bc060ce1612d0 2025-12-17 23:23:02,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409553575bc060ce1612d0 HTTP/1.1" 502 150 2025-12-17 23:23:02,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409553575bc060ce1612d0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55318 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:23:03,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409555575bc060ce1612d3 2025-12-17 23:23:03,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409555575bc060ce1612d3 HTTP/1.1" 502 150 2025-12-17 23:23:03,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409555575bc060ce1612d3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:04,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409539575bc060ce16127a 2025-12-17 23:23:04,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409539575bc060ce16127a HTTP/1.1" 502 150 2025-12-17 23:23:04,436 - lava_callback - ERROR - Error processing callback for node 69409539575bc060ce16127a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409539575bc060ce16127a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409539575bc060ce16127a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409539575bc060ce16127a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:04,438 - lava_callback - INFO - Starting async job submit for node 6940956c575bc060ce161315 2025-12-17 23:23:04,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:04,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956c575bc060ce161315 HTTP/1.1" 502 150 2025-12-17 23:23:04,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956c575bc060ce161315'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:04,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956c575bc060ce161315 2025-12-17 23:23:04,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956c575bc060ce161315 HTTP/1.1" 502 150 2025-12-17 23:23:04,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956c575bc060ce161315'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:04,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409562575bc060ce1612f5 2025-12-17 23:23:04,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409562575bc060ce1612f5 HTTP/1.1" 502 150 2025-12-17 23:23:04,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409562575bc060ce1612f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:05,312 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409557575bc060ce1612d9 2025-12-17 23:23:05,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409557575bc060ce1612d9 HTTP/1.1" 502 150 2025-12-17 23:23:05,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409557575bc060ce1612d9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:05,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409556575bc060ce1612d6 2025-12-17 23:23:05,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409556575bc060ce1612d6 HTTP/1.1" 502 150 2025-12-17 23:23:05,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409556575bc060ce1612d6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:06,190 - lava_callback - INFO - Received callback for node 69408d07575bc060ce15f74f INFO: 172.21.0.1:55330 - "POST /node/69408d07575bc060ce15f74f HTTP/1.0" 202 Accepted 2025-12-17 23:23:06,240 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956a575bc060ce161312 2025-12-17 23:23:06,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956a575bc060ce161312 HTTP/1.1" 502 150 2025-12-17 23:23:06,241 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956a575bc060ce161312'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:06,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956c575bc060ce161315 2025-12-17 23:23:06,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956c575bc060ce161315 HTTP/1.1" 502 150 2025-12-17 23:23:06,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956c575bc060ce161315'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55336 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:23:07,195 - lava_callback - INFO - Received callback for node 6942e2f1e43d2b76e9023918 INFO: 172.21.0.1:55340 - "POST /node/6942e2f1e43d2b76e9023918 HTTP/1.0" 202 Accepted 2025-12-17 23:23:07,260 - lava_callback - INFO - Received callback for node 69409846575bc060ce161a3f INFO: 172.21.0.1:55348 - "POST /node/69409846575bc060ce161a3f HTTP/1.0" 202 Accepted 2025-12-17 23:23:07,929 - lava_callback - INFO - Received callback for node 69409849575bc060ce161a46 INFO: 172.21.0.1:55356 - "POST /node/69409849575bc060ce161a46 HTTP/1.0" 202 Accepted 2025-12-17 23:23:08,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409547575bc060ce1612a5 2025-12-17 23:23:08,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409547575bc060ce1612a5 HTTP/1.1" 502 150 2025-12-17 23:23:08,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409547575bc060ce1612a5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:08,605 - lava_callback - INFO - Received callback for node 6940984b575bc060ce161a49 INFO: 172.21.0.1:55358 - "POST /node/6940984b575bc060ce161a49 HTTP/1.0" 202 Accepted 2025-12-17 23:23:08,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940953f575bc060ce161290 2025-12-17 23:23:08,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940953f575bc060ce161290 HTTP/1.1" 502 150 2025-12-17 23:23:08,978 - lava_callback - ERROR - Error processing callback for node 6940953f575bc060ce161290: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940953f575bc060ce161290 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940953f575bc060ce161290 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940953f575bc060ce161290 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:08,980 - lava_callback - INFO - Starting async job submit for node 69409567575bc060ce161309 2025-12-17 23:23:08,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:08,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409567575bc060ce161309 HTTP/1.1" 502 150 2025-12-17 23:23:08,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409567575bc060ce161309'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:08,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409567575bc060ce161309 2025-12-17 23:23:08,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409567575bc060ce161309 HTTP/1.1" 502 150 2025-12-17 23:23:08,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409567575bc060ce161309'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:09,260 - lava_callback - INFO - Received callback for node 6940984d575bc060ce161a54 INFO: 172.21.0.1:55364 - "POST /node/6940984d575bc060ce161a54 HTTP/1.0" 202 Accepted 2025-12-17 23:23:09,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409548575bc060ce1612a8 2025-12-17 23:23:09,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409548575bc060ce1612a8 HTTP/1.1" 502 150 2025-12-17 23:23:09,873 - lava_callback - ERROR - Error processing callback for node 69409548575bc060ce1612a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409548575bc060ce1612a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409548575bc060ce1612a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409548575bc060ce1612a8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:09,875 - lava_callback - INFO - Starting async job submit for node 6940956f575bc060ce16131e 2025-12-17 23:23:09,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:09,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956f575bc060ce16131e HTTP/1.1" 502 150 2025-12-17 23:23:09,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956f575bc060ce16131e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:09,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956f575bc060ce16131e 2025-12-17 23:23:09,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956f575bc060ce16131e HTTP/1.1" 502 150 2025-12-17 23:23:09,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956f575bc060ce16131e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:09,922 - lava_callback - INFO - Received callback for node 6940984f575bc060ce161a57 INFO: 172.21.0.1:44836 - "POST /node/6940984f575bc060ce161a57 HTTP/1.0" 202 Accepted 2025-12-17 23:23:10,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956c575bc060ce161315 2025-12-17 23:23:10,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956c575bc060ce161315 HTTP/1.1" 502 150 2025-12-17 23:23:10,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956c575bc060ce161315'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:10,585 - lava_callback - INFO - Received callback for node 69409851575bc060ce161a60 INFO: 172.21.0.1:44848 - "POST /node/69409851575bc060ce161a60 HTTP/1.0" 202 Accepted 2025-12-17 23:23:10,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409553575bc060ce1612d0 2025-12-17 23:23:10,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409553575bc060ce1612d0 HTTP/1.1" 502 150 2025-12-17 23:23:10,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409553575bc060ce1612d0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:10,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409567575bc060ce161309 2025-12-17 23:23:10,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409567575bc060ce161309 HTTP/1.1" 502 150 2025-12-17 23:23:10,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409567575bc060ce161309'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:11,270 - lava_callback - INFO - Received callback for node 69409853575bc060ce161a63 INFO: 172.21.0.1:44862 - "POST /node/69409853575bc060ce161a63 HTTP/1.0" 202 Accepted 2025-12-17 23:23:11,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409555575bc060ce1612d3 2025-12-17 23:23:11,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409555575bc060ce1612d3 HTTP/1.1" 502 150 2025-12-17 23:23:11,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409555575bc060ce1612d3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:11,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409549575bc060ce1612ab 2025-12-17 23:23:11,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409549575bc060ce1612ab HTTP/1.1" 502 150 2025-12-17 23:23:11,755 - lava_callback - ERROR - Error processing callback for node 69409549575bc060ce1612ab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409549575bc060ce1612ab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409549575bc060ce1612ab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409549575bc060ce1612ab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:11,757 - lava_callback - INFO - Starting async job submit for node 69409571575bc060ce161321 2025-12-17 23:23:11,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:11,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409571575bc060ce161321 HTTP/1.1" 502 150 2025-12-17 23:23:11,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409571575bc060ce161321'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:11,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409571575bc060ce161321 2025-12-17 23:23:11,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409571575bc060ce161321 HTTP/1.1" 502 150 2025-12-17 23:23:11,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409571575bc060ce161321'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:11,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956f575bc060ce16131e 2025-12-17 23:23:11,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956f575bc060ce16131e HTTP/1.1" 502 150 2025-12-17 23:23:11,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956f575bc060ce16131e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:11,915 - lava_callback - INFO - Received callback for node 69409855575bc060ce161a66 INFO: 172.21.0.1:44874 - "POST /node/69409855575bc060ce161a66 HTTP/1.0" 202 Accepted 2025-12-17 23:23:12,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954a575bc060ce1612ae 2025-12-17 23:23:12,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954a575bc060ce1612ae HTTP/1.1" 502 150 2025-12-17 23:23:12,006 - lava_callback - ERROR - Error processing callback for node 6940954a575bc060ce1612ae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940954a575bc060ce1612ae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940954a575bc060ce1612ae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940954a575bc060ce1612ae (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:12,008 - lava_callback - INFO - Starting async job submit for node 69409572575bc060ce161324 2025-12-17 23:23:12,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:12,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409572575bc060ce161324 HTTP/1.1" 502 150 2025-12-17 23:23:12,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409572575bc060ce161324'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:12,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409572575bc060ce161324 2025-12-17 23:23:12,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409572575bc060ce161324 HTTP/1.1" 502 150 2025-12-17 23:23:12,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409572575bc060ce161324'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:12,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409550575bc060ce1612c4 HTTP/1.1" 502 150 2025-12-17 23:23:12,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409550575bc060ce1612c4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:12,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409550575bc060ce1612c4 2025-12-17 23:23:12,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409550575bc060ce1612c4 HTTP/1.1" 502 150 2025-12-17 23:23:12,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409550575bc060ce1612c4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:12,583 - lava_callback - INFO - Received callback for node 69409857575bc060ce161a69 INFO: 172.21.0.1:44886 - "POST /node/69409857575bc060ce161a69 HTTP/1.0" 202 Accepted 2025-12-17 23:23:12,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409562575bc060ce1612f5 2025-12-17 23:23:12,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409562575bc060ce1612f5 HTTP/1.1" 502 150 2025-12-17 23:23:12,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409562575bc060ce1612f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:13,257 - lava_callback - INFO - Received callback for node 69409859575bc060ce161a6c INFO: 172.21.0.1:44892 - "POST /node/69409859575bc060ce161a6c HTTP/1.0" 202 Accepted 2025-12-17 23:23:13,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409557575bc060ce1612d9 2025-12-17 23:23:13,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409557575bc060ce1612d9 HTTP/1.1" 502 150 2025-12-17 23:23:13,315 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409557575bc060ce1612d9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:13,324 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954c575bc060ce1612b1 2025-12-17 23:23:13,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954c575bc060ce1612b1 HTTP/1.1" 502 150 2025-12-17 23:23:13,325 - lava_callback - ERROR - Error processing callback for node 6940954c575bc060ce1612b1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940954c575bc060ce1612b1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940954c575bc060ce1612b1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940954c575bc060ce1612b1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:13,327 - lava_callback - INFO - Starting async job submit for node 69409573575bc060ce161327 2025-12-17 23:23:13,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:13,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409573575bc060ce161327 HTTP/1.1" 502 150 2025-12-17 23:23:13,334 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409573575bc060ce161327'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:13,334 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409573575bc060ce161327 2025-12-17 23:23:13,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409573575bc060ce161327 HTTP/1.1" 502 150 2025-12-17 23:23:13,335 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409573575bc060ce161327'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:13,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409571575bc060ce161321 2025-12-17 23:23:13,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409571575bc060ce161321 HTTP/1.1" 502 150 2025-12-17 23:23:13,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409571575bc060ce161321'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:14,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409572575bc060ce161324 2025-12-17 23:23:14,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409572575bc060ce161324 HTTP/1.1" 502 150 2025-12-17 23:23:14,037 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409572575bc060ce161324'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:14,241 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956a575bc060ce161312 2025-12-17 23:23:14,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956a575bc060ce161312 HTTP/1.1" 502 150 2025-12-17 23:23:14,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956a575bc060ce161312'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:14,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409550575bc060ce1612c4 2025-12-17 23:23:14,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409550575bc060ce1612c4 HTTP/1.1" 502 150 2025-12-17 23:23:14,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409550575bc060ce1612c4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:14,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409567575bc060ce161309 2025-12-17 23:23:14,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409567575bc060ce161309 HTTP/1.1" 502 150 2025-12-17 23:23:14,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409567575bc060ce161309'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44894 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:23:15,335 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409573575bc060ce161327 2025-12-17 23:23:15,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409573575bc060ce161327 HTTP/1.1" 502 150 2025-12-17 23:23:15,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409573575bc060ce161327'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:15,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956f575bc060ce16131e 2025-12-17 23:23:15,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956f575bc060ce16131e HTTP/1.1" 502 150 2025-12-17 23:23:15,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956f575bc060ce16131e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:15,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940954e575bc060ce1612be 2025-12-17 23:23:15,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940954e575bc060ce1612be HTTP/1.1" 502 150 2025-12-17 23:23:15,926 - lava_callback - ERROR - Error processing callback for node 6940954e575bc060ce1612be: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940954e575bc060ce1612be (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940954e575bc060ce1612be (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940954e575bc060ce1612be (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:15,928 - lava_callback - INFO - Starting async job submit for node 69409574575bc060ce16132a 2025-12-17 23:23:15,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:15,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409551575bc060ce1612cd 2025-12-17 23:23:15,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409551575bc060ce1612cd HTTP/1.1" 502 150 2025-12-17 23:23:15,937 - lava_callback - ERROR - Error processing callback for node 69409551575bc060ce1612cd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409551575bc060ce1612cd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409551575bc060ce1612cd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409551575bc060ce1612cd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:15,939 - lava_callback - INFO - Starting async job submit for node 69409569575bc060ce16130f 2025-12-17 23:23:15,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:15,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409569575bc060ce16130f HTTP/1.1" 502 150 2025-12-17 23:23:15,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409569575bc060ce16130f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:15,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409569575bc060ce16130f 2025-12-17 23:23:15,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409569575bc060ce16130f HTTP/1.1" 502 150 2025-12-17 23:23:15,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409569575bc060ce16130f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:15,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409574575bc060ce16132a HTTP/1.1" 502 150 2025-12-17 23:23:15,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409574575bc060ce16132a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:15,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409574575bc060ce16132a 2025-12-17 23:23:15,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409574575bc060ce16132a HTTP/1.1" 502 150 2025-12-17 23:23:15,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409574575bc060ce16132a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44902 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:23:17,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409571575bc060ce161321 2025-12-17 23:23:17,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409571575bc060ce161321 HTTP/1.1" 502 150 2025-12-17 23:23:17,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409571575bc060ce161321'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:17,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409569575bc060ce16130f 2025-12-17 23:23:17,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409569575bc060ce16130f HTTP/1.1" 502 150 2025-12-17 23:23:17,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409569575bc060ce16130f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:17,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409574575bc060ce16132a 2025-12-17 23:23:17,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409574575bc060ce16132a HTTP/1.1" 502 150 2025-12-17 23:23:17,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409574575bc060ce16132a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:18,037 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409572575bc060ce161324 2025-12-17 23:23:18,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409550575bc060ce1612c4 2025-12-17 23:23:18,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956c575bc060ce161315 2025-12-17 23:23:18,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956c575bc060ce161315 HTTP/1.1" 502 150 2025-12-17 23:23:18,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956c575bc060ce161315'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:19,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409573575bc060ce161327 2025-12-17 23:23:19,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409573575bc060ce161327 HTTP/1.1" 502 150 2025-12-17 23:23:19,338 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409573575bc060ce161327'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:21,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409556575bc060ce1612d6 2025-12-17 23:23:21,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409556575bc060ce1612d6 HTTP/1.1" 502 150 2025-12-17 23:23:21,842 - lava_callback - ERROR - Error processing callback for node 69409556575bc060ce1612d6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409556575bc060ce1612d6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409556575bc060ce1612d6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409556575bc060ce1612d6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:21,844 - lava_callback - INFO - Starting async job submit for node 6940956d575bc060ce161318 2025-12-17 23:23:21,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:21,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956d575bc060ce161318 HTTP/1.1" 502 150 2025-12-17 23:23:21,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956d575bc060ce161318'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:21,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956d575bc060ce161318 2025-12-17 23:23:21,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956d575bc060ce161318 HTTP/1.1" 502 150 2025-12-17 23:23:21,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956d575bc060ce161318'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:21,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409569575bc060ce16130f 2025-12-17 23:23:21,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409569575bc060ce16130f HTTP/1.1" 502 150 2025-12-17 23:23:21,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409569575bc060ce16130f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:21,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409574575bc060ce16132a 2025-12-17 23:23:21,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409574575bc060ce16132a HTTP/1.1" 502 150 2025-12-17 23:23:21,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409574575bc060ce16132a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:22,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409567575bc060ce161309 2025-12-17 23:23:22,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409567575bc060ce161309 HTTP/1.1" 502 150 2025-12-17 23:23:22,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409567575bc060ce161309'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:23,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956d575bc060ce161318 2025-12-17 23:23:23,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956d575bc060ce161318 HTTP/1.1" 502 150 2025-12-17 23:23:23,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956d575bc060ce161318'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:23,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956f575bc060ce16131e 2025-12-17 23:23:23,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956f575bc060ce16131e HTTP/1.1" 502 150 2025-12-17 23:23:23,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956f575bc060ce16131e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:24,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409547575bc060ce1612a5 2025-12-17 23:23:24,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409547575bc060ce1612a5 HTTP/1.1" 502 150 2025-12-17 23:23:24,565 - lava_callback - ERROR - Error processing callback for node 69409547575bc060ce1612a5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409547575bc060ce1612a5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409547575bc060ce1612a5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409547575bc060ce1612a5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:24,567 - lava_callback - INFO - Starting async job submit for node 6940956e575bc060ce16131b 2025-12-17 23:23:24,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:24,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956e575bc060ce16131b HTTP/1.1" 502 150 2025-12-17 23:23:24,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956e575bc060ce16131b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:24,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956e575bc060ce16131b 2025-12-17 23:23:24,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956e575bc060ce16131b HTTP/1.1" 502 150 2025-12-17 23:23:24,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956e575bc060ce16131b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:25,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409571575bc060ce161321 2025-12-17 23:23:25,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409571575bc060ce161321 HTTP/1.1" 502 150 2025-12-17 23:23:25,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409571575bc060ce161321'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33612 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:23:26,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956e575bc060ce16131b 2025-12-17 23:23:26,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956e575bc060ce16131b HTTP/1.1" 502 150 2025-12-17 23:23:26,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956e575bc060ce16131b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:26,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409553575bc060ce1612d0 2025-12-17 23:23:26,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409553575bc060ce1612d0 HTTP/1.1" 502 150 2025-12-17 23:23:26,984 - lava_callback - ERROR - Error processing callback for node 69409553575bc060ce1612d0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409553575bc060ce1612d0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409553575bc060ce1612d0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409553575bc060ce1612d0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:26,987 - lava_callback - INFO - Starting async job submit for node 69409576575bc060ce16132f 2025-12-17 23:23:26,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:26,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409576575bc060ce16132f HTTP/1.1" 502 150 2025-12-17 23:23:26,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409576575bc060ce16132f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:26,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409576575bc060ce16132f 2025-12-17 23:23:26,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409576575bc060ce16132f HTTP/1.1" 502 150 2025-12-17 23:23:26,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409576575bc060ce16132f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:27,301 - lava_callback - INFO - Received callback for node 6940985b575bc060ce161a75 INFO: 172.21.0.1:33622 - "POST /node/6940985b575bc060ce161a75 HTTP/1.0" 202 Accepted 2025-12-17 23:23:27,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409573575bc060ce161327 2025-12-17 23:23:27,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409573575bc060ce161327 HTTP/1.1" 502 150 2025-12-17 23:23:27,340 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409573575bc060ce161327'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:27,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409555575bc060ce1612d3 2025-12-17 23:23:27,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409555575bc060ce1612d3 HTTP/1.1" 502 150 2025-12-17 23:23:27,620 - lava_callback - ERROR - Error processing callback for node 69409555575bc060ce1612d3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409555575bc060ce1612d3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409555575bc060ce1612d3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409555575bc060ce1612d3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:27,621 - lava_callback - INFO - Starting async job submit for node 69409577575bc060ce16133f 2025-12-17 23:23:27,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:27,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409577575bc060ce16133f HTTP/1.1" 502 150 2025-12-17 23:23:27,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409577575bc060ce16133f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:27,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409577575bc060ce16133f 2025-12-17 23:23:27,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409577575bc060ce16133f HTTP/1.1" 502 150 2025-12-17 23:23:27,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409577575bc060ce16133f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:27,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956d575bc060ce161318 2025-12-17 23:23:27,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956d575bc060ce161318 HTTP/1.1" 502 150 2025-12-17 23:23:27,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956d575bc060ce161318'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:27,962 - lava_callback - INFO - Received callback for node 6940985d575bc060ce161a7c INFO: 172.21.0.1:33638 - "POST /node/6940985d575bc060ce161a7c HTTP/1.0" 202 Accepted 2025-12-17 23:23:28,666 - lava_callback - INFO - Received callback for node 6940985f575bc060ce161a7f INFO: 172.21.0.1:33644 - "POST /node/6940985f575bc060ce161a7f HTTP/1.0" 202 Accepted 2025-12-17 23:23:28,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409562575bc060ce1612f5 2025-12-17 23:23:28,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409562575bc060ce1612f5 HTTP/1.1" 502 150 2025-12-17 23:23:28,736 - lava_callback - ERROR - Error processing callback for node 69409562575bc060ce1612f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409562575bc060ce1612f5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409562575bc060ce1612f5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409562575bc060ce1612f5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:28,738 - lava_callback - INFO - Starting async job submit for node 69409579575bc060ce161346 2025-12-17 23:23:28,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:28,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409579575bc060ce161346 HTTP/1.1" 502 150 2025-12-17 23:23:28,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409579575bc060ce161346'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:28,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409579575bc060ce161346 2025-12-17 23:23:28,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409579575bc060ce161346 HTTP/1.1" 502 150 2025-12-17 23:23:28,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409579575bc060ce161346'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:28,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409576575bc060ce16132f 2025-12-17 23:23:28,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409576575bc060ce16132f HTTP/1.1" 502 150 2025-12-17 23:23:28,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409576575bc060ce16132f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:29,315 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409557575bc060ce1612d9 2025-12-17 23:23:29,347 - lava_callback - INFO - Received callback for node 69409861575bc060ce161a82 INFO: 172.21.0.1:33650 - "POST /node/69409861575bc060ce161a82 HTTP/1.0" 202 Accepted 2025-12-17 23:23:29,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409577575bc060ce16133f 2025-12-17 23:23:29,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409577575bc060ce16133f HTTP/1.1" 502 150 2025-12-17 23:23:29,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409577575bc060ce16133f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:29,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409569575bc060ce16130f 2025-12-17 23:23:29,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409569575bc060ce16130f HTTP/1.1" 502 150 2025-12-17 23:23:29,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409569575bc060ce16130f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:29,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409574575bc060ce16132a 2025-12-17 23:23:29,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409574575bc060ce16132a HTTP/1.1" 502 150 2025-12-17 23:23:29,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409574575bc060ce16132a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:30,009 - lava_callback - INFO - Received callback for node 69409866575bc060ce161a88 INFO: 172.21.0.1:42174 - "POST /node/69409866575bc060ce161a88 HTTP/1.0" 202 Accepted 2025-12-17 23:23:30,243 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956a575bc060ce161312 2025-12-17 23:23:30,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956a575bc060ce161312 HTTP/1.1" 502 150 2025-12-17 23:23:30,244 - lava_callback - ERROR - Error processing callback for node 6940956a575bc060ce161312: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940956a575bc060ce161312 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940956a575bc060ce161312 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940956a575bc060ce161312 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:30,246 - lava_callback - INFO - Starting async job submit for node 6940957b575bc060ce16134d 2025-12-17 23:23:30,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:30,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957b575bc060ce16134d HTTP/1.1" 502 150 2025-12-17 23:23:30,254 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940957b575bc060ce16134d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:30,254 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940957b575bc060ce16134d 2025-12-17 23:23:30,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957b575bc060ce16134d HTTP/1.1" 502 150 2025-12-17 23:23:30,255 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940957b575bc060ce16134d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:30,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956e575bc060ce16131b 2025-12-17 23:23:30,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956e575bc060ce16131b HTTP/1.1" 502 150 2025-12-17 23:23:30,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956e575bc060ce16131b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:30,698 - lava_callback - INFO - Received callback for node 6940986a575bc060ce161a8e INFO: 172.21.0.1:42176 - "POST /node/6940986a575bc060ce161a8e HTTP/1.0" 202 Accepted 2025-12-17 23:23:30,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409579575bc060ce161346 2025-12-17 23:23:30,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409579575bc060ce161346 HTTP/1.1" 502 150 2025-12-17 23:23:30,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409579575bc060ce161346'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:31,364 - lava_callback - INFO - Received callback for node 6940986c575bc060ce161a91 INFO: 172.21.0.1:42178 - "POST /node/6940986c575bc060ce161a91 HTTP/1.0" 202 Accepted 2025-12-17 23:23:32,030 - lava_callback - INFO - Received callback for node 6940986e575bc060ce161a96 INFO: 172.21.0.1:42186 - "POST /node/6940986e575bc060ce161a96 HTTP/1.0" 202 Accepted 2025-12-17 23:23:32,255 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940957b575bc060ce16134d 2025-12-17 23:23:32,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957b575bc060ce16134d HTTP/1.1" 502 150 2025-12-17 23:23:32,257 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940957b575bc060ce16134d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:32,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409576575bc060ce16132f 2025-12-17 23:23:33,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409576575bc060ce16132f HTTP/1.1" 502 150 2025-12-17 23:23:33,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409576575bc060ce16132f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:33,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409572575bc060ce161324 HTTP/1.1" 502 150 2025-12-17 23:23:33,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409572575bc060ce161324'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:33,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409577575bc060ce16133f 2025-12-17 23:23:33,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409577575bc060ce16133f HTTP/1.1" 502 150 2025-12-17 23:23:33,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409577575bc060ce16133f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:33,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409550575bc060ce1612c4 HTTP/1.1" 502 150 2025-12-17 23:23:33,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409550575bc060ce1612c4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:34,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956c575bc060ce161315 2025-12-17 23:23:34,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956c575bc060ce161315 HTTP/1.1" 502 150 2025-12-17 23:23:34,450 - lava_callback - ERROR - Error processing callback for node 6940956c575bc060ce161315: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940956c575bc060ce161315 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940956c575bc060ce161315 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940956c575bc060ce161315 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:34,452 - lava_callback - INFO - Starting async job submit for node 6940957d575bc060ce161350 2025-12-17 23:23:34,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:34,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957d575bc060ce161350 HTTP/1.1" 502 150 2025-12-17 23:23:34,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940957d575bc060ce161350'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:34,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940957d575bc060ce161350 2025-12-17 23:23:34,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957d575bc060ce161350 HTTP/1.1" 502 150 2025-12-17 23:23:34,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940957d575bc060ce161350'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:34,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409579575bc060ce161346 2025-12-17 23:23:34,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409579575bc060ce161346 HTTP/1.1" 502 150 2025-12-17 23:23:34,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409579575bc060ce161346'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:35,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956d575bc060ce161318 2025-12-17 23:23:35,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956d575bc060ce161318 HTTP/1.1" 502 150 2025-12-17 23:23:35,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956d575bc060ce161318'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:36,257 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940957b575bc060ce16134d 2025-12-17 23:23:36,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957b575bc060ce16134d HTTP/1.1" 502 150 2025-12-17 23:23:36,258 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940957b575bc060ce16134d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:36,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940957d575bc060ce161350 2025-12-17 23:23:36,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957d575bc060ce161350 HTTP/1.1" 502 150 2025-12-17 23:23:36,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940957d575bc060ce161350'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42194 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:23:36,563 - lava_callback - INFO - Received callback for node 69409863575bc060ce161a85 INFO: 172.21.0.1:42206 - "POST /node/69409863575bc060ce161a85 HTTP/1.0" 202 Accepted 2025-12-17 23:23:37,472 - lava_callback - INFO - Received callback for node 69409876575bc060ce161aaa INFO: 172.21.0.1:42208 - "POST /node/69409876575bc060ce161aaa HTTP/1.0" 202 Accepted 2025-12-17 23:23:38,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956e575bc060ce16131b 2025-12-17 23:23:38,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956e575bc060ce16131b HTTP/1.1" 502 150 2025-12-17 23:23:38,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940956e575bc060ce16131b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:38,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409567575bc060ce161309 2025-12-17 23:23:38,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409567575bc060ce161309 HTTP/1.1" 502 150 2025-12-17 23:23:38,993 - lava_callback - ERROR - Error processing callback for node 69409567575bc060ce161309: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409567575bc060ce161309 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409567575bc060ce161309 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409567575bc060ce161309 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:38,995 - lava_callback - INFO - Starting async job submit for node 6940957f575bc060ce161356 2025-12-17 23:23:38,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:39,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957f575bc060ce161356 HTTP/1.1" 502 150 2025-12-17 23:23:39,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940957f575bc060ce161356'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:39,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940957f575bc060ce161356 2025-12-17 23:23:39,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957f575bc060ce161356 HTTP/1.1" 502 150 2025-12-17 23:23:39,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940957f575bc060ce161356'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:39,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956f575bc060ce16131e 2025-12-17 23:23:39,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956f575bc060ce16131e HTTP/1.1" 502 150 2025-12-17 23:23:39,888 - lava_callback - ERROR - Error processing callback for node 6940956f575bc060ce16131e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940956f575bc060ce16131e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940956f575bc060ce16131e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940956f575bc060ce16131e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:39,890 - lava_callback - INFO - Starting async job submit for node 69409581575bc060ce161359 2025-12-17 23:23:39,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:39,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409581575bc060ce161359 HTTP/1.1" 502 150 2025-12-17 23:23:39,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409581575bc060ce161359'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:39,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409581575bc060ce161359 2025-12-17 23:23:39,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409581575bc060ce161359 HTTP/1.1" 502 150 2025-12-17 23:23:39,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409581575bc060ce161359'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:40,277 - lava_callback - INFO - Received callback for node 69409545575bc060ce1612a2 INFO: 172.21.0.1:53638 - "POST /node/69409545575bc060ce1612a2 HTTP/1.0" 202 Accepted 2025-12-17 23:23:40,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940957d575bc060ce161350 2025-12-17 23:23:40,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957d575bc060ce161350 HTTP/1.1" 502 150 2025-12-17 23:23:40,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940957d575bc060ce161350'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:41,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409576575bc060ce16132f 2025-12-17 23:23:41,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409576575bc060ce16132f HTTP/1.1" 502 150 2025-12-17 23:23:41,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409576575bc060ce16132f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:41,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940957f575bc060ce161356 2025-12-17 23:23:41,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957f575bc060ce161356 HTTP/1.1" 502 150 2025-12-17 23:23:41,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940957f575bc060ce161356'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:41,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409572575bc060ce161324 2025-12-17 23:23:41,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409572575bc060ce161324 HTTP/1.1" 502 150 2025-12-17 23:23:41,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409572575bc060ce161324'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:41,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409577575bc060ce16133f 2025-12-17 23:23:41,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409577575bc060ce16133f HTTP/1.1" 502 150 2025-12-17 23:23:41,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409577575bc060ce16133f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:41,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409550575bc060ce1612c4 2025-12-17 23:23:41,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409550575bc060ce1612c4 HTTP/1.1" 502 150 2025-12-17 23:23:41,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409550575bc060ce1612c4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:41,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409571575bc060ce161321 2025-12-17 23:23:41,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409571575bc060ce161321 HTTP/1.1" 502 150 2025-12-17 23:23:41,770 - lava_callback - ERROR - Error processing callback for node 69409571575bc060ce161321: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409571575bc060ce161321 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409571575bc060ce161321 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409571575bc060ce161321 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:41,772 - lava_callback - INFO - Starting async job submit for node 69409584575bc060ce16135c 2025-12-17 23:23:41,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:41,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409584575bc060ce16135c HTTP/1.1" 502 150 2025-12-17 23:23:41,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409584575bc060ce16135c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:41,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409584575bc060ce16135c 2025-12-17 23:23:41,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409584575bc060ce16135c HTTP/1.1" 502 150 2025-12-17 23:23:41,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409584575bc060ce16135c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:41,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409581575bc060ce161359 2025-12-17 23:23:41,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409581575bc060ce161359 HTTP/1.1" 502 150 2025-12-17 23:23:41,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409581575bc060ce161359'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:42,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409579575bc060ce161346 2025-12-17 23:23:42,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409579575bc060ce161346 HTTP/1.1" 502 150 2025-12-17 23:23:42,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409579575bc060ce161346'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:43,340 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409573575bc060ce161327 2025-12-17 23:23:43,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409573575bc060ce161327 HTTP/1.1" 502 150 2025-12-17 23:23:43,341 - lava_callback - ERROR - Error processing callback for node 69409573575bc060ce161327: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409573575bc060ce161327 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409573575bc060ce161327 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409573575bc060ce161327 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:43,343 - lava_callback - INFO - Starting async job submit for node 6940958a575bc060ce161365 2025-12-17 23:23:43,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:43,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958a575bc060ce161365 HTTP/1.1" 502 150 2025-12-17 23:23:43,350 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958a575bc060ce161365'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:43,350 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958a575bc060ce161365 2025-12-17 23:23:43,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958a575bc060ce161365 HTTP/1.1" 502 150 2025-12-17 23:23:43,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958a575bc060ce161365'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:43,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409584575bc060ce16135c 2025-12-17 23:23:43,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409584575bc060ce16135c HTTP/1.1" 502 150 2025-12-17 23:23:43,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409584575bc060ce16135c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:44,259 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940957b575bc060ce16134d 2025-12-17 23:23:44,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409557575bc060ce1612d9 HTTP/1.1" 502 150 2025-12-17 23:23:44,692 - lava_callback - ERROR - Error processing callback for node 69409557575bc060ce1612d9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409557575bc060ce1612d9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409557575bc060ce1612d9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409557575bc060ce1612d9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:44,694 - lava_callback - INFO - Starting async job submit for node 6940958c575bc060ce16136e 2025-12-17 23:23:44,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:44,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958c575bc060ce16136e HTTP/1.1" 502 150 2025-12-17 23:23:44,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958c575bc060ce16136e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:44,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958c575bc060ce16136e 2025-12-17 23:23:44,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958c575bc060ce16136e HTTP/1.1" 502 150 2025-12-17 23:23:44,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958c575bc060ce16136e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:45,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940957f575bc060ce161356 2025-12-17 23:23:45,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957f575bc060ce161356 HTTP/1.1" 502 150 2025-12-17 23:23:45,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940957f575bc060ce161356'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:45,351 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958a575bc060ce161365 2025-12-17 23:23:45,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958a575bc060ce161365 HTTP/1.1" 502 150 2025-12-17 23:23:45,352 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958a575bc060ce161365'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:45,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409581575bc060ce161359 2025-12-17 23:23:45,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409581575bc060ce161359 HTTP/1.1" 502 150 2025-12-17 23:23:45,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409581575bc060ce161359'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:45,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409569575bc060ce16130f 2025-12-17 23:23:45,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409569575bc060ce16130f HTTP/1.1" 502 150 2025-12-17 23:23:45,962 - lava_callback - ERROR - Error processing callback for node 69409569575bc060ce16130f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409569575bc060ce16130f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409569575bc060ce16130f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409569575bc060ce16130f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:45,964 - lava_callback - INFO - Starting async job submit for node 69409593575bc060ce16137d 2025-12-17 23:23:45,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:45,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409574575bc060ce16132a 2025-12-17 23:23:45,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409574575bc060ce16132a HTTP/1.1" 502 150 2025-12-17 23:23:45,976 - lava_callback - ERROR - Error processing callback for node 69409574575bc060ce16132a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409574575bc060ce16132a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409574575bc060ce16132a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409574575bc060ce16132a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:45,978 - lava_callback - INFO - Starting async job submit for node 6940958e575bc060ce161371 2025-12-17 23:23:45,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:45,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409593575bc060ce16137d HTTP/1.1" 502 150 2025-12-17 23:23:45,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409593575bc060ce16137d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:45,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409593575bc060ce16137d 2025-12-17 23:23:45,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409593575bc060ce16137d HTTP/1.1" 502 150 2025-12-17 23:23:45,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409593575bc060ce16137d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:45,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958e575bc060ce161371 HTTP/1.1" 502 150 2025-12-17 23:23:45,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958e575bc060ce161371'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:45,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958e575bc060ce161371 2025-12-17 23:23:45,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958e575bc060ce161371 HTTP/1.1" 502 150 2025-12-17 23:23:45,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958e575bc060ce161371'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53654 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:23:46,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958c575bc060ce16136e 2025-12-17 23:23:46,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958c575bc060ce16136e HTTP/1.1" 502 150 2025-12-17 23:23:46,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958c575bc060ce16136e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:47,301 - lava_callback - INFO - Received callback for node 69409871575bc060ce161aa4 INFO: 172.21.0.1:53662 - "POST /node/69409871575bc060ce161aa4 HTTP/1.0" 202 Accepted 2025-12-17 23:23:47,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409584575bc060ce16135c 2025-12-17 23:23:47,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409584575bc060ce16135c HTTP/1.1" 502 150 2025-12-17 23:23:47,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409584575bc060ce16135c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:47,972 - lava_callback - INFO - Received callback for node 69409873575bc060ce161aa7 INFO: 172.21.0.1:53678 - "POST /node/69409873575bc060ce161aa7 HTTP/1.0" 202 Accepted 2025-12-17 23:23:47,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409593575bc060ce16137d 2025-12-17 23:23:47,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409593575bc060ce16137d HTTP/1.1" 502 150 2025-12-17 23:23:47,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409593575bc060ce16137d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:47,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958e575bc060ce161371 2025-12-17 23:23:47,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958e575bc060ce161371 HTTP/1.1" 502 150 2025-12-17 23:23:47,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958e575bc060ce161371'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:48,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940957d575bc060ce161350 2025-12-17 23:23:48,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957d575bc060ce161350 HTTP/1.1" 502 150 2025-12-17 23:23:48,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940957d575bc060ce161350'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:48,633 - lava_callback - INFO - Received callback for node 69409878575bc060ce161aad INFO: 172.21.0.1:53694 - "POST /node/69409878575bc060ce161aad HTTP/1.0" 202 Accepted 2025-12-17 23:23:49,298 - lava_callback - INFO - Received callback for node 6940987a575bc060ce161ab0 INFO: 172.21.0.1:53700 - "POST /node/6940987a575bc060ce161ab0 HTTP/1.0" 202 Accepted 2025-12-17 23:23:49,353 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958a575bc060ce161365 2025-12-17 23:23:49,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958a575bc060ce161365 HTTP/1.1" 502 150 2025-12-17 23:23:49,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958a575bc060ce161365'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:49,960 - lava_callback - INFO - Received callback for node 6940987c575bc060ce161ab3 INFO: 172.21.0.1:41576 - "POST /node/6940987c575bc060ce161ab3 HTTP/1.0" 202 Accepted 2025-12-17 23:23:50,651 - lava_callback - INFO - Received callback for node 6940987e575bc060ce161ab6 INFO: 172.21.0.1:41582 - "POST /node/6940987e575bc060ce161ab6 HTTP/1.0" 202 Accepted 2025-12-17 23:23:50,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958c575bc060ce16136e 2025-12-17 23:23:50,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958c575bc060ce16136e HTTP/1.1" 502 150 2025-12-17 23:23:50,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958c575bc060ce16136e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:51,309 - lava_callback - INFO - Received callback for node 69409880575bc060ce161ab9 INFO: 172.21.0.1:41592 - "POST /node/69409880575bc060ce161ab9 HTTP/1.0" 202 Accepted 2025-12-17 23:23:51,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956d575bc060ce161318 2025-12-17 23:23:51,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956d575bc060ce161318 HTTP/1.1" 502 150 2025-12-17 23:23:51,860 - lava_callback - ERROR - Error processing callback for node 6940956d575bc060ce161318: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940956d575bc060ce161318 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940956d575bc060ce161318 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940956d575bc060ce161318 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:51,862 - lava_callback - INFO - Starting async job submit for node 6940958f575bc060ce161374 2025-12-17 23:23:51,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:51,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958f575bc060ce161374 HTTP/1.1" 502 150 2025-12-17 23:23:51,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958f575bc060ce161374'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:51,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958f575bc060ce161374 2025-12-17 23:23:51,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958f575bc060ce161374 HTTP/1.1" 502 150 2025-12-17 23:23:51,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958f575bc060ce161374'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:51,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409593575bc060ce16137d 2025-12-17 23:23:51,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409593575bc060ce16137d HTTP/1.1" 502 150 2025-12-17 23:23:51,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409593575bc060ce16137d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:51,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958e575bc060ce161371 2025-12-17 23:23:51,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958e575bc060ce161371 HTTP/1.1" 502 150 2025-12-17 23:23:51,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958e575bc060ce161371'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:53,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940957f575bc060ce161356 2025-12-17 23:23:53,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957f575bc060ce161356 HTTP/1.1" 502 150 2025-12-17 23:23:53,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940957f575bc060ce161356'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:53,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958f575bc060ce161374 2025-12-17 23:23:53,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958f575bc060ce161374 HTTP/1.1" 502 150 2025-12-17 23:23:53,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958f575bc060ce161374'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:53,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409581575bc060ce161359 2025-12-17 23:23:53,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409581575bc060ce161359 HTTP/1.1" 502 150 2025-12-17 23:23:53,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409581575bc060ce161359'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:54,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940956e575bc060ce16131b 2025-12-17 23:23:54,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940956e575bc060ce16131b HTTP/1.1" 502 150 2025-12-17 23:23:54,602 - lava_callback - ERROR - Error processing callback for node 6940956e575bc060ce16131b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940956e575bc060ce16131b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940956e575bc060ce16131b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940956e575bc060ce16131b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:54,604 - lava_callback - INFO - Starting async job submit for node 69409590575bc060ce161377 2025-12-17 23:23:54,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:54,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409590575bc060ce161377 HTTP/1.1" 502 150 2025-12-17 23:23:54,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409590575bc060ce161377'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:54,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409590575bc060ce161377 2025-12-17 23:23:54,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409590575bc060ce161377 HTTP/1.1" 502 150 2025-12-17 23:23:54,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409590575bc060ce161377'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:55,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409584575bc060ce16135c INFO: 172.21.0.1:41604 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:23:56,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409590575bc060ce161377 2025-12-17 23:23:56,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409590575bc060ce161377 HTTP/1.1" 502 150 2025-12-17 23:23:56,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409590575bc060ce161377'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:57,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409576575bc060ce16132f 2025-12-17 23:23:57,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409576575bc060ce16132f HTTP/1.1" 502 150 2025-12-17 23:23:57,003 - lava_callback - ERROR - Error processing callback for node 69409576575bc060ce16132f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409576575bc060ce16132f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409576575bc060ce16132f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409576575bc060ce16132f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:57,005 - lava_callback - INFO - Starting async job submit for node 69409592575bc060ce16137a 2025-12-17 23:23:57,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:57,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409592575bc060ce16137a HTTP/1.1" 502 150 2025-12-17 23:23:57,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409592575bc060ce16137a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:57,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409592575bc060ce16137a 2025-12-17 23:23:57,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409592575bc060ce16137a HTTP/1.1" 502 150 2025-12-17 23:23:57,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409592575bc060ce16137a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:57,354 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958a575bc060ce161365 2025-12-17 23:23:57,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958a575bc060ce161365 HTTP/1.1" 502 150 2025-12-17 23:23:57,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958a575bc060ce161365'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:57,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409572575bc060ce161324 2025-12-17 23:23:57,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409572575bc060ce161324 HTTP/1.1" 502 150 2025-12-17 23:23:57,427 - lava_callback - ERROR - Error processing callback for node 69409572575bc060ce161324: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409572575bc060ce161324 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409572575bc060ce161324 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409572575bc060ce161324 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:57,429 - lava_callback - INFO - Starting async job submit for node 69409595575bc060ce161383 2025-12-17 23:23:57,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:57,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409595575bc060ce161383 HTTP/1.1" 502 150 2025-12-17 23:23:57,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409595575bc060ce161383'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:57,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409595575bc060ce161383 2025-12-17 23:23:57,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409595575bc060ce161383 HTTP/1.1" 502 150 2025-12-17 23:23:57,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409595575bc060ce161383'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:57,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409577575bc060ce16133f 2025-12-17 23:23:57,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409577575bc060ce16133f HTTP/1.1" 502 150 2025-12-17 23:23:57,638 - lava_callback - ERROR - Error processing callback for node 69409577575bc060ce16133f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409577575bc060ce16133f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409577575bc060ce16133f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409577575bc060ce16133f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:57,640 - lava_callback - INFO - Starting async job submit for node 69409596575bc060ce161386 2025-12-17 23:23:57,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:57,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409596575bc060ce161386 HTTP/1.1" 502 150 2025-12-17 23:23:57,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409596575bc060ce161386'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:57,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409596575bc060ce161386 2025-12-17 23:23:57,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409596575bc060ce161386 HTTP/1.1" 502 150 2025-12-17 23:23:57,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409596575bc060ce161386'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:57,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409550575bc060ce1612c4 2025-12-17 23:23:57,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409550575bc060ce1612c4 HTTP/1.1" 502 150 2025-12-17 23:23:57,683 - lava_callback - ERROR - Error processing callback for node 69409550575bc060ce1612c4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409550575bc060ce1612c4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409550575bc060ce1612c4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409550575bc060ce1612c4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:57,684 - lava_callback - INFO - Starting async job submit for node 69409594575bc060ce161380 2025-12-17 23:23:57,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:57,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409594575bc060ce161380 HTTP/1.1" 502 150 2025-12-17 23:23:57,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409594575bc060ce161380'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:57,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409594575bc060ce161380 2025-12-17 23:23:57,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409594575bc060ce161380 HTTP/1.1" 502 150 2025-12-17 23:23:57,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409594575bc060ce161380'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:57,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958f575bc060ce161374 2025-12-17 23:23:57,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958f575bc060ce161374 HTTP/1.1" 502 150 2025-12-17 23:23:57,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958f575bc060ce161374'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:58,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958c575bc060ce16136e 2025-12-17 23:23:58,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958c575bc060ce16136e HTTP/1.1" 502 150 2025-12-17 23:23:58,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958c575bc060ce16136e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:58,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409579575bc060ce161346 2025-12-17 23:23:58,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409579575bc060ce161346 HTTP/1.1" 502 150 2025-12-17 23:23:58,762 - lava_callback - ERROR - Error processing callback for node 69409579575bc060ce161346: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409579575bc060ce161346 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409579575bc060ce161346 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409579575bc060ce161346 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:23:58,764 - lava_callback - INFO - Starting async job submit for node 69409598575bc060ce161389 2025-12-17 23:23:58,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:23:58,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409598575bc060ce161389 HTTP/1.1" 502 150 2025-12-17 23:23:58,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409598575bc060ce161389'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:58,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409598575bc060ce161389 2025-12-17 23:23:58,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409598575bc060ce161389 HTTP/1.1" 502 150 2025-12-17 23:23:58,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409598575bc060ce161389'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:59,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409592575bc060ce16137a 2025-12-17 23:23:59,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409595575bc060ce161383 2025-12-17 23:23:59,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409595575bc060ce161383 HTTP/1.1" 502 150 2025-12-17 23:23:59,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409595575bc060ce161383'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:59,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957b575bc060ce16134d HTTP/1.1" 502 150 2025-12-17 23:23:59,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940957b575bc060ce16134d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:59,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409596575bc060ce161386 2025-12-17 23:23:59,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409596575bc060ce161386 HTTP/1.1" 502 150 2025-12-17 23:23:59,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409596575bc060ce161386'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:59,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409594575bc060ce161380 2025-12-17 23:23:59,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409594575bc060ce161380 HTTP/1.1" 502 150 2025-12-17 23:23:59,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409594575bc060ce161380'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:59,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409593575bc060ce16137d 2025-12-17 23:23:59,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409593575bc060ce16137d HTTP/1.1" 502 150 2025-12-17 23:23:59,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409593575bc060ce16137d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:23:59,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958e575bc060ce161371 2025-12-17 23:23:59,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958e575bc060ce161371 HTTP/1.1" 502 150 2025-12-17 23:23:59,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958e575bc060ce161371'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:00,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409590575bc060ce161377 2025-12-17 23:24:00,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409590575bc060ce161377 HTTP/1.1" 502 150 2025-12-17 23:24:00,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409590575bc060ce161377'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:00,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409598575bc060ce161389 2025-12-17 23:24:00,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409598575bc060ce161389 HTTP/1.1" 502 150 2025-12-17 23:24:00,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409598575bc060ce161389'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58336 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:24:03,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409595575bc060ce161383 2025-12-17 23:24:03,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409595575bc060ce161383 HTTP/1.1" 502 150 2025-12-17 23:24:03,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409595575bc060ce161383'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:03,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409596575bc060ce161386 2025-12-17 23:24:03,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409596575bc060ce161386 HTTP/1.1" 502 150 2025-12-17 23:24:03,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409596575bc060ce161386'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:03,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409594575bc060ce161380 2025-12-17 23:24:03,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409594575bc060ce161380 HTTP/1.1" 502 150 2025-12-17 23:24:03,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409594575bc060ce161380'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:04,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940957d575bc060ce161350 2025-12-17 23:24:04,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957d575bc060ce161350 HTTP/1.1" 502 150 2025-12-17 23:24:04,469 - lava_callback - ERROR - Error processing callback for node 6940957d575bc060ce161350: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940957d575bc060ce161350 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940957d575bc060ce161350 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940957d575bc060ce161350 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:04,471 - lava_callback - INFO - Starting async job submit for node 6940959c575bc060ce161392 2025-12-17 23:24:04,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:04,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959c575bc060ce161392 HTTP/1.1" 502 150 2025-12-17 23:24:04,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959c575bc060ce161392'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:04,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959c575bc060ce161392 2025-12-17 23:24:04,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959c575bc060ce161392 HTTP/1.1" 502 150 2025-12-17 23:24:04,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959c575bc060ce161392'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:04,608 - lava_callback - INFO - Received callback for node 69409802575bc060ce161991 INFO: 172.21.0.1:58352 - "POST /node/69409802575bc060ce161991 HTTP/1.0" 202 Accepted 2025-12-17 23:24:04,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409598575bc060ce161389 2025-12-17 23:24:04,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409598575bc060ce161389 HTTP/1.1" 502 150 2025-12-17 23:24:04,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409598575bc060ce161389'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:05,547 - lava_callback - INFO - Received callback for node 6942e37fe43d2b76e9023a5a INFO: 172.21.0.1:58356 - "POST /node/6942e37fe43d2b76e9023a5a HTTP/1.0" 202 Accepted 2025-12-17 23:24:05,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958f575bc060ce161374 2025-12-17 23:24:05,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958f575bc060ce161374 HTTP/1.1" 502 150 2025-12-17 23:24:05,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940958f575bc060ce161374'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:06,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959c575bc060ce161392 2025-12-17 23:24:06,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959c575bc060ce161392 HTTP/1.1" 502 150 2025-12-17 23:24:06,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959c575bc060ce161392'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58372 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:24:07,310 - lava_callback - INFO - Received callback for node 69409882575bc060ce161ac0 INFO: 172.21.0.1:58380 - "POST /node/69409882575bc060ce161ac0 HTTP/1.0" 202 Accepted 2025-12-17 23:24:08,000 - lava_callback - INFO - Received callback for node 69409884575bc060ce161ac9 INFO: 172.21.0.1:58396 - "POST /node/69409884575bc060ce161ac9 HTTP/1.0" 202 Accepted 2025-12-17 23:24:08,539 - lava_callback - INFO - Received callback for node 6941e655ecc59770562f0bb7 INFO: 172.21.0.1:58398 - "POST /node/6941e655ecc59770562f0bb7 HTTP/1.0" 202 Accepted 2025-12-17 23:24:08,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409590575bc060ce161377 2025-12-17 23:24:08,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409590575bc060ce161377 HTTP/1.1" 502 150 2025-12-17 23:24:08,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409590575bc060ce161377'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:08,679 - lava_callback - INFO - Received callback for node 69409887575bc060ce161ad0 INFO: 172.21.0.1:58406 - "POST /node/69409887575bc060ce161ad0 HTTP/1.0" 202 Accepted 2025-12-17 23:24:09,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940957f575bc060ce161356 2025-12-17 23:24:09,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957f575bc060ce161356 HTTP/1.1" 502 150 2025-12-17 23:24:09,030 - lava_callback - ERROR - Error processing callback for node 6940957f575bc060ce161356: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940957f575bc060ce161356 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940957f575bc060ce161356 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940957f575bc060ce161356 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:09,032 - lava_callback - INFO - Starting async job submit for node 69409599575bc060ce16138c 2025-12-17 23:24:09,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:09,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409599575bc060ce16138c HTTP/1.1" 502 150 2025-12-17 23:24:09,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409599575bc060ce16138c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:09,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409599575bc060ce16138c 2025-12-17 23:24:09,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409599575bc060ce16138c HTTP/1.1" 502 150 2025-12-17 23:24:09,039 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409599575bc060ce16138c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:09,375 - lava_callback - INFO - Received callback for node 69409888575bc060ce161ad3 INFO: 172.21.0.1:58420 - "POST /node/69409888575bc060ce161ad3 HTTP/1.0" 202 Accepted 2025-12-17 23:24:09,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409581575bc060ce161359 2025-12-17 23:24:09,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409581575bc060ce161359 HTTP/1.1" 502 150 2025-12-17 23:24:09,903 - lava_callback - ERROR - Error processing callback for node 69409581575bc060ce161359: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409581575bc060ce161359 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409581575bc060ce161359 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409581575bc060ce161359 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:09,905 - lava_callback - INFO - Starting async job submit for node 6940959d575bc060ce161395 2025-12-17 23:24:09,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:09,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959d575bc060ce161395 HTTP/1.1" 502 150 2025-12-17 23:24:09,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959d575bc060ce161395'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:09,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959d575bc060ce161395 2025-12-17 23:24:09,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959d575bc060ce161395 HTTP/1.1" 502 150 2025-12-17 23:24:09,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959d575bc060ce161395'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:10,052 - lava_callback - INFO - Received callback for node 6940988a575bc060ce161ad6 INFO: 172.21.0.1:34836 - "POST /node/6940988a575bc060ce161ad6 HTTP/1.0" 202 Accepted 2025-12-17 23:24:10,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959c575bc060ce161392 2025-12-17 23:24:10,724 - lava_callback - INFO - Received callback for node 6940988c575bc060ce161add INFO: 172.21.0.1:34842 - "POST /node/6940988c575bc060ce161add HTTP/1.0" 202 Accepted 2025-12-17 23:24:11,039 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409599575bc060ce16138c 2025-12-17 23:24:11,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409599575bc060ce16138c HTTP/1.1" 502 150 2025-12-17 23:24:11,041 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409599575bc060ce16138c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:11,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409584575bc060ce16135c HTTP/1.1" 502 150 2025-12-17 23:24:11,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409584575bc060ce16135c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:11,398 - lava_callback - INFO - Received callback for node 6940988d575bc060ce161ae0 INFO: 172.21.0.1:34858 - "POST /node/6940988d575bc060ce161ae0 HTTP/1.0" 202 Accepted 2025-12-17 23:24:11,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409595575bc060ce161383 2025-12-17 23:24:11,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409595575bc060ce161383 HTTP/1.1" 502 150 2025-12-17 23:24:11,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409595575bc060ce161383'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:11,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409596575bc060ce161386 2025-12-17 23:24:11,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409596575bc060ce161386 HTTP/1.1" 502 150 2025-12-17 23:24:11,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409596575bc060ce161386'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:11,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409594575bc060ce161380 2025-12-17 23:24:11,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409594575bc060ce161380 HTTP/1.1" 502 150 2025-12-17 23:24:11,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409594575bc060ce161380'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:11,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959d575bc060ce161395 2025-12-17 23:24:11,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959d575bc060ce161395 HTTP/1.1" 502 150 2025-12-17 23:24:11,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959d575bc060ce161395'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:12,055 - lava_callback - INFO - Received callback for node 69409890575bc060ce161ae6 INFO: 172.21.0.1:34872 - "POST /node/69409890575bc060ce161ae6 HTTP/1.0" 202 Accepted 2025-12-17 23:24:12,744 - lava_callback - INFO - Received callback for node 69409891575bc060ce161ae9 INFO: 172.21.0.1:34878 - "POST /node/69409891575bc060ce161ae9 HTTP/1.0" 202 Accepted 2025-12-17 23:24:12,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409598575bc060ce161389 2025-12-17 23:24:12,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409598575bc060ce161389 HTTP/1.1" 502 150 2025-12-17 23:24:12,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409598575bc060ce161389'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:13,355 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958a575bc060ce161365 2025-12-17 23:24:13,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958a575bc060ce161365 HTTP/1.1" 502 150 2025-12-17 23:24:13,357 - lava_callback - ERROR - Error processing callback for node 6940958a575bc060ce161365: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940958a575bc060ce161365 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940958a575bc060ce161365 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940958a575bc060ce161365 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:13,359 - lava_callback - INFO - Starting async job submit for node 6940959a575bc060ce16138f 2025-12-17 23:24:13,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:13,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959a575bc060ce16138f HTTP/1.1" 502 150 2025-12-17 23:24:13,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959a575bc060ce16138f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:13,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959a575bc060ce16138f 2025-12-17 23:24:13,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959a575bc060ce16138f HTTP/1.1" 502 150 2025-12-17 23:24:13,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959a575bc060ce16138f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:13,439 - lava_callback - INFO - Received callback for node 69409893575bc060ce161aec INFO: 172.21.0.1:34884 - "POST /node/69409893575bc060ce161aec HTTP/1.0" 202 Accepted 2025-12-17 23:24:14,131 - lava_callback - INFO - Received callback for node 69409895575bc060ce161aef INFO: 172.21.0.1:34890 - "POST /node/69409895575bc060ce161aef HTTP/1.0" 202 Accepted 2025-12-17 23:24:14,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409592575bc060ce16137a HTTP/1.1" 502 150 2025-12-17 23:24:14,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409592575bc060ce16137a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:14,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958c575bc060ce16136e 2025-12-17 23:24:14,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958c575bc060ce16136e HTTP/1.1" 502 150 2025-12-17 23:24:14,708 - lava_callback - ERROR - Error processing callback for node 6940958c575bc060ce16136e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940958c575bc060ce16136e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940958c575bc060ce16136e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940958c575bc060ce16136e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:14,710 - lava_callback - INFO - Starting async job submit for node 69409843575bc060ce161a33 2025-12-17 23:24:14,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:14,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409843575bc060ce161a33 HTTP/1.1" 502 150 2025-12-17 23:24:14,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409843575bc060ce161a33'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:14,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409843575bc060ce161a33 2025-12-17 23:24:14,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409843575bc060ce161a33 HTTP/1.1" 502 150 2025-12-17 23:24:14,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409843575bc060ce161a33'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:15,041 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409599575bc060ce16138c 2025-12-17 23:24:15,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409599575bc060ce16138c HTTP/1.1" 502 150 2025-12-17 23:24:15,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409599575bc060ce16138c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34904 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:24:15,367 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959a575bc060ce16138f 2025-12-17 23:24:15,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959a575bc060ce16138f HTTP/1.1" 502 150 2025-12-17 23:24:15,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959a575bc060ce16138f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:15,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940957b575bc060ce16134d 2025-12-17 23:24:15,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940957b575bc060ce16134d HTTP/1.1" 502 150 2025-12-17 23:24:15,538 - lava_callback - ERROR - Error processing callback for node 6940957b575bc060ce16134d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940957b575bc060ce16134d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940957b575bc060ce16134d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940957b575bc060ce16134d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:15,540 - lava_callback - INFO - Starting async job submit for node 6940959e575bc060ce16139c 2025-12-17 23:24:15,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:15,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959e575bc060ce16139c HTTP/1.1" 502 150 2025-12-17 23:24:15,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959e575bc060ce16139c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:15,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959e575bc060ce16139c 2025-12-17 23:24:15,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959e575bc060ce16139c HTTP/1.1" 502 150 2025-12-17 23:24:15,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959e575bc060ce16139c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:15,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959d575bc060ce161395 2025-12-17 23:24:15,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959d575bc060ce161395 HTTP/1.1" 502 150 2025-12-17 23:24:15,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959d575bc060ce161395'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:15,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409593575bc060ce16137d 2025-12-17 23:24:15,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409593575bc060ce16137d HTTP/1.1" 502 150 2025-12-17 23:24:15,987 - lava_callback - ERROR - Error processing callback for node 69409593575bc060ce16137d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409593575bc060ce16137d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409593575bc060ce16137d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409593575bc060ce16137d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:15,988 - lava_callback - INFO - Starting async job submit for node 6940959f575bc060ce1613a3 2025-12-17 23:24:15,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:15,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959f575bc060ce1613a3 HTTP/1.1" 502 150 2025-12-17 23:24:15,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959f575bc060ce1613a3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:15,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959f575bc060ce1613a3 2025-12-17 23:24:15,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958e575bc060ce161371 2025-12-17 23:24:15,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959f575bc060ce1613a3 HTTP/1.1" 502 150 2025-12-17 23:24:15,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959f575bc060ce1613a3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:15,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958e575bc060ce161371 HTTP/1.1" 502 150 2025-12-17 23:24:15,997 - lava_callback - ERROR - Error processing callback for node 6940958e575bc060ce161371: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940958e575bc060ce161371 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940958e575bc060ce161371 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940958e575bc060ce161371 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:15,998 - lava_callback - INFO - Starting async job submit for node 694095a2575bc060ce1613ac 2025-12-17 23:24:15,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:16,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a2575bc060ce1613ac HTTP/1.1" 502 150 2025-12-17 23:24:16,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a2575bc060ce1613ac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:16,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a2575bc060ce1613ac 2025-12-17 23:24:16,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a2575bc060ce1613ac HTTP/1.1" 502 150 2025-12-17 23:24:16,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a2575bc060ce1613ac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34920 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:24:16,584 - lava_callback - INFO - Received callback for node 6940989e575bc060ce161b08 INFO: 172.21.0.1:34926 - "POST /node/6940989e575bc060ce161b08 HTTP/1.0" 202 Accepted 2025-12-17 23:24:16,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409843575bc060ce161a33 2025-12-17 23:24:16,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409843575bc060ce161a33 HTTP/1.1" 502 150 2025-12-17 23:24:16,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409843575bc060ce161a33'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:17,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959e575bc060ce16139c 2025-12-17 23:24:17,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959e575bc060ce16139c HTTP/1.1" 502 150 2025-12-17 23:24:17,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959e575bc060ce16139c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:17,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959f575bc060ce1613a3 2025-12-17 23:24:17,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959f575bc060ce1613a3 HTTP/1.1" 502 150 2025-12-17 23:24:17,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959f575bc060ce1613a3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:18,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a2575bc060ce1613ac 2025-12-17 23:24:18,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a2575bc060ce1613ac HTTP/1.1" 502 150 2025-12-17 23:24:18,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a2575bc060ce1613ac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:18,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409592575bc060ce16137a 2025-12-17 23:24:18,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409592575bc060ce16137a HTTP/1.1" 502 150 2025-12-17 23:24:18,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409592575bc060ce16137a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:19,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959a575bc060ce16138f 2025-12-17 23:24:19,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959a575bc060ce16138f HTTP/1.1" 502 150 2025-12-17 23:24:19,371 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959a575bc060ce16138f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:20,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409843575bc060ce161a33 2025-12-17 23:24:20,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409843575bc060ce161a33 HTTP/1.1" 502 150 2025-12-17 23:24:20,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409843575bc060ce161a33'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:21,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959e575bc060ce16139c 2025-12-17 23:24:21,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959e575bc060ce16139c HTTP/1.1" 502 150 2025-12-17 23:24:21,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959e575bc060ce16139c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:21,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940958f575bc060ce161374 2025-12-17 23:24:21,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940958f575bc060ce161374 HTTP/1.1" 502 150 2025-12-17 23:24:21,875 - lava_callback - ERROR - Error processing callback for node 6940958f575bc060ce161374: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940958f575bc060ce161374 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940958f575bc060ce161374 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940958f575bc060ce161374 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:21,877 - lava_callback - INFO - Starting async job submit for node 694095a2575bc060ce1613b1 2025-12-17 23:24:21,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:21,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a2575bc060ce1613b1 HTTP/1.1" 502 150 2025-12-17 23:24:21,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a2575bc060ce1613b1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:21,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a2575bc060ce1613b1 2025-12-17 23:24:21,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a2575bc060ce1613b1 HTTP/1.1" 502 150 2025-12-17 23:24:21,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a2575bc060ce1613b1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:21,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959f575bc060ce1613a3 2025-12-17 23:24:22,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a2575bc060ce1613ac 2025-12-17 23:24:22,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a2575bc060ce1613ac HTTP/1.1" 502 150 2025-12-17 23:24:22,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a2575bc060ce1613ac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:23,042 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409599575bc060ce16138c 2025-12-17 23:24:23,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409599575bc060ce16138c HTTP/1.1" 502 150 2025-12-17 23:24:23,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409599575bc060ce16138c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:23,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a2575bc060ce1613b1 2025-12-17 23:24:23,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a2575bc060ce1613b1 HTTP/1.1" 502 150 2025-12-17 23:24:23,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a2575bc060ce1613b1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:23,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959d575bc060ce161395 2025-12-17 23:24:23,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959d575bc060ce161395 HTTP/1.1" 502 150 2025-12-17 23:24:23,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959d575bc060ce161395'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:24,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409590575bc060ce161377 2025-12-17 23:24:24,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409590575bc060ce161377 HTTP/1.1" 502 150 2025-12-17 23:24:24,624 - lava_callback - ERROR - Error processing callback for node 69409590575bc060ce161377: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409590575bc060ce161377 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409590575bc060ce161377 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409590575bc060ce161377 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:24,625 - lava_callback - INFO - Starting async job submit for node 694095a4575bc060ce1613bb 2025-12-17 23:24:24,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:24,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a4575bc060ce1613bb HTTP/1.1" 502 150 2025-12-17 23:24:24,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a4575bc060ce1613bb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:24,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a4575bc060ce1613bb 2025-12-17 23:24:24,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a4575bc060ce1613bb HTTP/1.1" 502 150 2025-12-17 23:24:24,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a4575bc060ce1613bb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:25,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959c575bc060ce161392 HTTP/1.1" 502 150 2025-12-17 23:24:25,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959c575bc060ce161392'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:26,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409592575bc060ce16137a 2025-12-17 23:24:26,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409592575bc060ce16137a HTTP/1.1" 502 150 2025-12-17 23:24:26,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409592575bc060ce16137a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37442 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:24:26,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a4575bc060ce1613bb 2025-12-17 23:24:26,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a4575bc060ce1613bb HTTP/1.1" 502 150 2025-12-17 23:24:26,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a4575bc060ce1613bb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:27,056 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409584575bc060ce16135c 2025-12-17 23:24:27,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409584575bc060ce16135c HTTP/1.1" 502 150 2025-12-17 23:24:27,058 - lava_callback - ERROR - Error processing callback for node 69409584575bc060ce16135c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409584575bc060ce16135c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409584575bc060ce16135c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409584575bc060ce16135c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:27,060 - lava_callback - INFO - Starting async job submit for node 69411cc4459ff5fc64d2ab11 2025-12-17 23:24:27,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:27,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411cc4459ff5fc64d2ab11 HTTP/1.1" 502 150 2025-12-17 23:24:27,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69411cc4459ff5fc64d2ab11'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:27,112 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69411cc4459ff5fc64d2ab11 2025-12-17 23:24:27,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411cc4459ff5fc64d2ab11 HTTP/1.1" 502 150 2025-12-17 23:24:27,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69411cc4459ff5fc64d2ab11'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:27,157 - lava_callback - INFO - Received callback for node 6940945f575bc060ce16100e INFO: 172.21.0.1:37458 - "POST /node/6940945f575bc060ce16100e HTTP/1.0" 202 Accepted 2025-12-17 23:24:27,340 - lava_callback - INFO - Received callback for node 69409897575bc060ce161af8 INFO: 172.21.0.1:37462 - "POST /node/69409897575bc060ce161af8 HTTP/1.0" 202 Accepted 2025-12-17 23:24:27,371 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959a575bc060ce16138f 2025-12-17 23:24:27,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959a575bc060ce16138f HTTP/1.1" 502 150 2025-12-17 23:24:27,372 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959a575bc060ce16138f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:27,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409595575bc060ce161383 2025-12-17 23:24:27,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409595575bc060ce161383 HTTP/1.1" 502 150 2025-12-17 23:24:27,447 - lava_callback - ERROR - Error processing callback for node 69409595575bc060ce161383: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409595575bc060ce161383 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409595575bc060ce161383 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409595575bc060ce161383 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:27,448 - lava_callback - INFO - Starting async job submit for node 694095a5575bc060ce1613be 2025-12-17 23:24:27,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:27,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a5575bc060ce1613be HTTP/1.1" 502 150 2025-12-17 23:24:27,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a5575bc060ce1613be'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:27,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a5575bc060ce1613be 2025-12-17 23:24:27,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a5575bc060ce1613be HTTP/1.1" 502 150 2025-12-17 23:24:27,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a5575bc060ce1613be'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:27,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409596575bc060ce161386 2025-12-17 23:24:27,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409596575bc060ce161386 HTTP/1.1" 502 150 2025-12-17 23:24:27,658 - lava_callback - ERROR - Error processing callback for node 69409596575bc060ce161386: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409596575bc060ce161386 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409596575bc060ce161386 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409596575bc060ce161386 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:27,660 - lava_callback - INFO - Starting async job submit for node 694095a6575bc060ce1613c1 2025-12-17 23:24:27,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:27,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a6575bc060ce1613c1 HTTP/1.1" 502 150 2025-12-17 23:24:27,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a6575bc060ce1613c1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:27,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a6575bc060ce1613c1 2025-12-17 23:24:27,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a6575bc060ce1613c1 HTTP/1.1" 502 150 2025-12-17 23:24:27,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a6575bc060ce1613c1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:27,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409594575bc060ce161380 2025-12-17 23:24:27,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409594575bc060ce161380 HTTP/1.1" 502 150 2025-12-17 23:24:27,697 - lava_callback - ERROR - Error processing callback for node 69409594575bc060ce161380: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409594575bc060ce161380 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409594575bc060ce161380 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409594575bc060ce161380 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:27,698 - lava_callback - INFO - Starting async job submit for node 694095a7575bc060ce1613c4 2025-12-17 23:24:27,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:27,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a7575bc060ce1613c4 HTTP/1.1" 502 150 2025-12-17 23:24:27,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a7575bc060ce1613c4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:27,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a7575bc060ce1613c4 2025-12-17 23:24:27,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a7575bc060ce1613c4 HTTP/1.1" 502 150 2025-12-17 23:24:27,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a7575bc060ce1613c4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:27,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a2575bc060ce1613b1 2025-12-17 23:24:27,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a2575bc060ce1613b1 HTTP/1.1" 502 150 2025-12-17 23:24:27,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a2575bc060ce1613b1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:28,005 - lava_callback - INFO - Received callback for node 69409898575bc060ce161aff INFO: 172.21.0.1:37468 - "POST /node/69409898575bc060ce161aff HTTP/1.0" 202 Accepted 2025-12-17 23:24:28,664 - lava_callback - INFO - Received callback for node 6940989c575bc060ce161b05 INFO: 172.21.0.1:37476 - "POST /node/6940989c575bc060ce161b05 HTTP/1.0" 202 Accepted 2025-12-17 23:24:28,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409843575bc060ce161a33 2025-12-17 23:24:28,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409843575bc060ce161a33 HTTP/1.1" 502 150 2025-12-17 23:24:28,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409843575bc060ce161a33'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:28,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409598575bc060ce161389 2025-12-17 23:24:28,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409598575bc060ce161389 HTTP/1.1" 502 150 2025-12-17 23:24:28,778 - lava_callback - ERROR - Error processing callback for node 69409598575bc060ce161389: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409598575bc060ce161389 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409598575bc060ce161389 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409598575bc060ce161389 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:28,780 - lava_callback - INFO - Starting async job submit for node 694095a9575bc060ce1613c7 2025-12-17 23:24:28,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:28,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a9575bc060ce1613c7 HTTP/1.1" 502 150 2025-12-17 23:24:28,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a9575bc060ce1613c7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:28,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a9575bc060ce1613c7 2025-12-17 23:24:28,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a9575bc060ce1613c7 HTTP/1.1" 502 150 2025-12-17 23:24:28,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a9575bc060ce1613c7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:29,113 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69411cc4459ff5fc64d2ab11 2025-12-17 23:24:29,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411cc4459ff5fc64d2ab11 HTTP/1.1" 502 150 2025-12-17 23:24:29,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69411cc4459ff5fc64d2ab11'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:29,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a5575bc060ce1613be 2025-12-17 23:24:29,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a5575bc060ce1613be HTTP/1.1" 502 150 2025-12-17 23:24:29,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a5575bc060ce1613be'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:29,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959e575bc060ce16139c 2025-12-17 23:24:29,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959e575bc060ce16139c HTTP/1.1" 502 150 2025-12-17 23:24:29,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959e575bc060ce16139c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:29,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a6575bc060ce1613c1 2025-12-17 23:24:29,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a6575bc060ce1613c1 HTTP/1.1" 502 150 2025-12-17 23:24:29,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a6575bc060ce1613c1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:29,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a7575bc060ce1613c4 2025-12-17 23:24:29,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a7575bc060ce1613c4 HTTP/1.1" 502 150 2025-12-17 23:24:29,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a7575bc060ce1613c4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:30,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a2575bc060ce1613ac 2025-12-17 23:24:30,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a2575bc060ce1613ac HTTP/1.1" 502 150 2025-12-17 23:24:30,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a2575bc060ce1613ac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:30,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a4575bc060ce1613bb 2025-12-17 23:24:30,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a4575bc060ce1613bb HTTP/1.1" 502 150 2025-12-17 23:24:30,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a4575bc060ce1613bb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:30,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a9575bc060ce1613c7 2025-12-17 23:24:30,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a9575bc060ce1613c7 HTTP/1.1" 502 150 2025-12-17 23:24:30,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a9575bc060ce1613c7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:33,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69411cc4459ff5fc64d2ab11 2025-12-17 23:24:33,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411cc4459ff5fc64d2ab11 HTTP/1.1" 502 150 2025-12-17 23:24:33,116 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69411cc4459ff5fc64d2ab11'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:33,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a5575bc060ce1613be 2025-12-17 23:24:33,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a5575bc060ce1613be HTTP/1.1" 502 150 2025-12-17 23:24:33,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a5575bc060ce1613be'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:33,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a6575bc060ce1613c1 2025-12-17 23:24:33,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a6575bc060ce1613c1 HTTP/1.1" 502 150 2025-12-17 23:24:33,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a6575bc060ce1613c1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:33,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a7575bc060ce1613c4 2025-12-17 23:24:33,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a7575bc060ce1613c4 HTTP/1.1" 502 150 2025-12-17 23:24:33,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a7575bc060ce1613c4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:33,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959c575bc060ce161392 2025-12-17 23:24:33,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959c575bc060ce161392 HTTP/1.1" 502 150 2025-12-17 23:24:33,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959c575bc060ce161392'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:34,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a9575bc060ce1613c7 2025-12-17 23:24:34,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a9575bc060ce1613c7 HTTP/1.1" 502 150 2025-12-17 23:24:34,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a9575bc060ce1613c7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:35,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a2575bc060ce1613b1 2025-12-17 23:24:35,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a2575bc060ce1613b1 HTTP/1.1" 502 150 2025-12-17 23:24:35,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a2575bc060ce1613b1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47080 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:24:36,650 - lava_callback - INFO - Received callback for node 694098a1575bc060ce161b0f INFO: 172.21.0.1:47090 - "POST /node/694098a1575bc060ce161b0f HTTP/1.0" 202 Accepted 2025-12-17 23:24:37,314 - lava_callback - INFO - Received callback for node 694098a3575bc060ce161b12 INFO: 172.21.0.1:47100 - "POST /node/694098a3575bc060ce161b12 HTTP/1.0" 202 Accepted 2025-12-17 23:24:37,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959f575bc060ce1613a3 HTTP/1.1" 502 150 2025-12-17 23:24:37,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959f575bc060ce1613a3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:37,946 - lava_callback - INFO - Received callback for node 694098a5575bc060ce161b1a INFO: 172.21.0.1:47112 - "POST /node/694098a5575bc060ce161b1a HTTP/1.0" 202 Accepted 2025-12-17 23:24:38,607 - lava_callback - INFO - Received callback for node 694098a7575bc060ce161b1d INFO: 172.21.0.1:47128 - "POST /node/694098a7575bc060ce161b1d HTTP/1.0" 202 Accepted 2025-12-17 23:24:38,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a4575bc060ce1613bb 2025-12-17 23:24:38,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a4575bc060ce1613bb HTTP/1.1" 502 150 2025-12-17 23:24:38,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a4575bc060ce1613bb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:39,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409599575bc060ce16138c 2025-12-17 23:24:39,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409599575bc060ce16138c HTTP/1.1" 502 150 2025-12-17 23:24:39,045 - lava_callback - ERROR - Error processing callback for node 69409599575bc060ce16138c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409599575bc060ce16138c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409599575bc060ce16138c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409599575bc060ce16138c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:39,048 - lava_callback - INFO - Starting async job submit for node 694095aa575bc060ce1613ca 2025-12-17 23:24:39,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:39,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095aa575bc060ce1613ca HTTP/1.1" 502 150 2025-12-17 23:24:39,224 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095aa575bc060ce1613ca'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:39,225 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095aa575bc060ce1613ca 2025-12-17 23:24:39,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095aa575bc060ce1613ca HTTP/1.1" 502 150 2025-12-17 23:24:39,225 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095aa575bc060ce1613ca'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:39,258 - lava_callback - INFO - Received callback for node 694098a9575bc060ce161b20 INFO: 172.21.0.1:47140 - "POST /node/694098a9575bc060ce161b20 HTTP/1.0" 202 Accepted 2025-12-17 23:24:39,894 - lava_callback - INFO - Received callback for node 694098af575bc060ce161b36 INFO: 172.21.0.1:40688 - "POST /node/694098af575bc060ce161b36 HTTP/1.0" 202 Accepted 2025-12-17 23:24:39,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959d575bc060ce161395 2025-12-17 23:24:39,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959d575bc060ce161395 HTTP/1.1" 502 150 2025-12-17 23:24:39,923 - lava_callback - ERROR - Error processing callback for node 6940959d575bc060ce161395: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940959d575bc060ce161395 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940959d575bc060ce161395 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940959d575bc060ce161395 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:39,926 - lava_callback - INFO - Starting async job submit for node 694095ab575bc060ce1613cd 2025-12-17 23:24:39,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:39,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ab575bc060ce1613cd HTTP/1.1" 502 150 2025-12-17 23:24:39,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ab575bc060ce1613cd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:39,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ab575bc060ce1613cd 2025-12-17 23:24:39,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ab575bc060ce1613cd HTTP/1.1" 502 150 2025-12-17 23:24:39,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ab575bc060ce1613cd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:40,543 - lava_callback - INFO - Received callback for node 694098b1575bc060ce161b39 INFO: 172.21.0.1:40690 - "POST /node/694098b1575bc060ce161b39 HTTP/1.0" 202 Accepted 2025-12-17 23:24:41,116 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69411cc4459ff5fc64d2ab11 2025-12-17 23:24:41,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411cc4459ff5fc64d2ab11 HTTP/1.1" 502 150 2025-12-17 23:24:41,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69411cc4459ff5fc64d2ab11'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:41,226 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095aa575bc060ce1613ca 2025-12-17 23:24:41,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095aa575bc060ce1613ca HTTP/1.1" 502 150 2025-12-17 23:24:41,227 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095aa575bc060ce1613ca'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:41,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a5575bc060ce1613be 2025-12-17 23:24:41,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a5575bc060ce1613be HTTP/1.1" 502 150 2025-12-17 23:24:41,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a5575bc060ce1613be'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:41,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a6575bc060ce1613c1 2025-12-17 23:24:41,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a6575bc060ce1613c1 HTTP/1.1" 502 150 2025-12-17 23:24:41,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a6575bc060ce1613c1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:41,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a7575bc060ce1613c4 2025-12-17 23:24:41,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a7575bc060ce1613c4 HTTP/1.1" 502 150 2025-12-17 23:24:41,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a7575bc060ce1613c4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:41,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ab575bc060ce1613cd 2025-12-17 23:24:41,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ab575bc060ce1613cd HTTP/1.1" 502 150 2025-12-17 23:24:41,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ab575bc060ce1613cd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:42,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409592575bc060ce16137a 2025-12-17 23:24:42,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409592575bc060ce16137a HTTP/1.1" 502 150 2025-12-17 23:24:42,389 - lava_callback - ERROR - Error processing callback for node 69409592575bc060ce16137a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409592575bc060ce16137a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409592575bc060ce16137a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409592575bc060ce16137a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:42,391 - lava_callback - INFO - Starting async job submit for node 694095ac575bc060ce1613d0 2025-12-17 23:24:42,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:42,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ac575bc060ce1613d0 HTTP/1.1" 502 150 2025-12-17 23:24:42,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ac575bc060ce1613d0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:42,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ac575bc060ce1613d0 2025-12-17 23:24:42,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ac575bc060ce1613d0 HTTP/1.1" 502 150 2025-12-17 23:24:42,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ac575bc060ce1613d0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:42,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a9575bc060ce1613c7 2025-12-17 23:24:42,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a9575bc060ce1613c7 HTTP/1.1" 502 150 2025-12-17 23:24:42,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095a9575bc060ce1613c7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:43,372 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959a575bc060ce16138f 2025-12-17 23:24:43,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959a575bc060ce16138f HTTP/1.1" 502 150 2025-12-17 23:24:43,374 - lava_callback - ERROR - Error processing callback for node 6940959a575bc060ce16138f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940959a575bc060ce16138f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940959a575bc060ce16138f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940959a575bc060ce16138f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:43,375 - lava_callback - INFO - Starting async job submit for node 694095ae575bc060ce1613d3 2025-12-17 23:24:43,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:43,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ae575bc060ce1613d3 HTTP/1.1" 502 150 2025-12-17 23:24:43,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ae575bc060ce1613d3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:43,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ae575bc060ce1613d3 2025-12-17 23:24:43,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ae575bc060ce1613d3 HTTP/1.1" 502 150 2025-12-17 23:24:43,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ae575bc060ce1613d3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:44,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ac575bc060ce1613d0 2025-12-17 23:24:44,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ac575bc060ce1613d0 HTTP/1.1" 502 150 2025-12-17 23:24:44,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ac575bc060ce1613d0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:44,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409843575bc060ce161a33 2025-12-17 23:24:44,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409843575bc060ce161a33 HTTP/1.1" 502 150 2025-12-17 23:24:44,741 - lava_callback - ERROR - Error processing callback for node 69409843575bc060ce161a33: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409843575bc060ce161a33 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409843575bc060ce161a33 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409843575bc060ce161a33 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:44,743 - lava_callback - INFO - Starting async job submit for node 694095af575bc060ce1613d6 2025-12-17 23:24:44,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:44,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095af575bc060ce1613d6 HTTP/1.1" 502 150 2025-12-17 23:24:44,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095af575bc060ce1613d6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:44,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095af575bc060ce1613d6 2025-12-17 23:24:44,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095af575bc060ce1613d6 HTTP/1.1" 502 150 2025-12-17 23:24:44,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095af575bc060ce1613d6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:45,227 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095aa575bc060ce1613ca 2025-12-17 23:24:45,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095aa575bc060ce1613ca HTTP/1.1" 502 150 2025-12-17 23:24:45,228 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095aa575bc060ce1613ca'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:45,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ae575bc060ce1613d3 2025-12-17 23:24:45,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ae575bc060ce1613d3 HTTP/1.1" 502 150 2025-12-17 23:24:45,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ae575bc060ce1613d3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:45,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959f575bc060ce1613a3 2025-12-17 23:24:45,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959f575bc060ce1613a3 HTTP/1.1" 502 150 2025-12-17 23:24:45,425 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940959f575bc060ce1613a3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:45,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959e575bc060ce16139c 2025-12-17 23:24:45,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959e575bc060ce16139c HTTP/1.1" 502 150 2025-12-17 23:24:45,573 - lava_callback - ERROR - Error processing callback for node 6940959e575bc060ce16139c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940959e575bc060ce16139c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940959e575bc060ce16139c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940959e575bc060ce16139c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:45,575 - lava_callback - INFO - Starting async job submit for node 694095b0575bc060ce1613d9 2025-12-17 23:24:45,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:45,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b0575bc060ce1613d9 HTTP/1.1" 502 150 2025-12-17 23:24:45,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b0575bc060ce1613d9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:45,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b0575bc060ce1613d9 2025-12-17 23:24:45,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b0575bc060ce1613d9 HTTP/1.1" 502 150 2025-12-17 23:24:45,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b0575bc060ce1613d9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:45,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ab575bc060ce1613cd 2025-12-17 23:24:45,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ab575bc060ce1613cd HTTP/1.1" 502 150 2025-12-17 23:24:45,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ab575bc060ce1613cd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:46,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a2575bc060ce1613ac 2025-12-17 23:24:46,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a2575bc060ce1613ac HTTP/1.1" 502 150 2025-12-17 23:24:46,010 - lava_callback - ERROR - Error processing callback for node 694095a2575bc060ce1613ac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a2575bc060ce1613ac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a2575bc060ce1613ac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a2575bc060ce1613ac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:46,012 - lava_callback - INFO - Starting async job submit for node 694095b7575bc060ce1613f9 2025-12-17 23:24:46,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:46,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b7575bc060ce1613f9 HTTP/1.1" 502 150 2025-12-17 23:24:46,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b7575bc060ce1613f9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:46,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b7575bc060ce1613f9 2025-12-17 23:24:46,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b7575bc060ce1613f9 HTTP/1.1" 502 150 2025-12-17 23:24:46,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b7575bc060ce1613f9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40692 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:24:46,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095af575bc060ce1613d6 2025-12-17 23:24:46,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095af575bc060ce1613d6 HTTP/1.1" 502 150 2025-12-17 23:24:46,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095af575bc060ce1613d6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:47,374 - lava_callback - INFO - Received callback for node 694098ab575bc060ce161b28 INFO: 172.21.0.1:40698 - "POST /node/694098ab575bc060ce161b28 HTTP/1.0" 202 Accepted 2025-12-17 23:24:47,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b0575bc060ce1613d9 2025-12-17 23:24:47,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b0575bc060ce1613d9 HTTP/1.1" 502 150 2025-12-17 23:24:47,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b0575bc060ce1613d9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:48,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b7575bc060ce1613f9 2025-12-17 23:24:48,034 - lava_callback - INFO - Received callback for node 694098ad575bc060ce161b31 INFO: 172.21.0.1:40704 - "POST /node/694098ad575bc060ce161b31 HTTP/1.0" 202 Accepted 2025-12-17 23:24:48,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ac575bc060ce1613d0 2025-12-17 23:24:48,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ac575bc060ce1613d0 HTTP/1.1" 502 150 2025-12-17 23:24:48,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ac575bc060ce1613d0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:49,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ae575bc060ce1613d3 2025-12-17 23:24:49,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ae575bc060ce1613d3 HTTP/1.1" 502 150 2025-12-17 23:24:49,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ae575bc060ce1613d3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:49,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959c575bc060ce161392 2025-12-17 23:24:49,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959c575bc060ce161392 HTTP/1.1" 502 150 2025-12-17 23:24:49,907 - lava_callback - ERROR - Error processing callback for node 6940959c575bc060ce161392: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940959c575bc060ce161392 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940959c575bc060ce161392 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940959c575bc060ce161392 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:49,909 - lava_callback - INFO - Starting async job submit for node 694095b9575bc060ce1613fc 2025-12-17 23:24:49,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:49,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b9575bc060ce1613fc HTTP/1.1" 502 150 2025-12-17 23:24:49,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b9575bc060ce1613fc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:49,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b9575bc060ce1613fc 2025-12-17 23:24:49,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b9575bc060ce1613fc HTTP/1.1" 502 150 2025-12-17 23:24:49,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b9575bc060ce1613fc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:50,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095af575bc060ce1613d6 2025-12-17 23:24:50,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095af575bc060ce1613d6 HTTP/1.1" 502 150 2025-12-17 23:24:50,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095af575bc060ce1613d6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:51,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b0575bc060ce1613d9 2025-12-17 23:24:51,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b0575bc060ce1613d9 HTTP/1.1" 502 150 2025-12-17 23:24:51,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b0575bc060ce1613d9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:51,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a2575bc060ce1613b1 2025-12-17 23:24:51,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a2575bc060ce1613b1 HTTP/1.1" 502 150 2025-12-17 23:24:51,897 - lava_callback - ERROR - Error processing callback for node 694095a2575bc060ce1613b1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a2575bc060ce1613b1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a2575bc060ce1613b1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a2575bc060ce1613b1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:51,899 - lava_callback - INFO - Starting async job submit for node 694095bb575bc060ce1613ff 2025-12-17 23:24:51,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:51,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b9575bc060ce1613fc 2025-12-17 23:24:51,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b9575bc060ce1613fc HTTP/1.1" 502 150 2025-12-17 23:24:51,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b9575bc060ce1613fc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:51,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bb575bc060ce1613ff HTTP/1.1" 502 150 2025-12-17 23:24:51,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095bb575bc060ce1613ff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:51,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095bb575bc060ce1613ff 2025-12-17 23:24:51,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bb575bc060ce1613ff HTTP/1.1" 502 150 2025-12-17 23:24:51,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095bb575bc060ce1613ff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:53,229 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095aa575bc060ce1613ca 2025-12-17 23:24:53,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095aa575bc060ce1613ca HTTP/1.1" 502 150 2025-12-17 23:24:53,230 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095aa575bc060ce1613ca'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:53,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095bb575bc060ce1613ff 2025-12-17 23:24:53,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bb575bc060ce1613ff HTTP/1.1" 502 150 2025-12-17 23:24:53,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095bb575bc060ce1613ff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:53,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ab575bc060ce1613cd 2025-12-17 23:24:53,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ab575bc060ce1613cd HTTP/1.1" 502 150 2025-12-17 23:24:53,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ab575bc060ce1613cd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:54,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a4575bc060ce1613bb 2025-12-17 23:24:54,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a4575bc060ce1613bb HTTP/1.1" 502 150 2025-12-17 23:24:54,638 - lava_callback - ERROR - Error processing callback for node 694095a4575bc060ce1613bb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a4575bc060ce1613bb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a4575bc060ce1613bb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a4575bc060ce1613bb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:54,640 - lava_callback - INFO - Starting async job submit for node 694095b1575bc060ce1613de 2025-12-17 23:24:54,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:54,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b1575bc060ce1613de HTTP/1.1" 502 150 2025-12-17 23:24:54,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b1575bc060ce1613de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:54,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b1575bc060ce1613de 2025-12-17 23:24:54,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b1575bc060ce1613de HTTP/1.1" 502 150 2025-12-17 23:24:54,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b1575bc060ce1613de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:55,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b9575bc060ce1613fc 2025-12-17 23:24:55,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b9575bc060ce1613fc HTTP/1.1" 502 150 2025-12-17 23:24:55,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b9575bc060ce1613fc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:56,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ac575bc060ce1613d0 2025-12-17 23:24:56,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ac575bc060ce1613d0 HTTP/1.1" 502 150 2025-12-17 23:24:56,405 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ac575bc060ce1613d0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:56,468 - lava_callback - INFO - Received callback for node 69409569575bc060ce16130c INFO: 172.21.0.1:34248 - "POST /node/69409569575bc060ce16130c HTTP/1.0" 202 Accepted INFO: 172.21.0.1:34256 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:24:56,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b1575bc060ce1613de 2025-12-17 23:24:56,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b1575bc060ce1613de HTTP/1.1" 502 150 2025-12-17 23:24:56,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b1575bc060ce1613de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:56,718 - lava_callback - INFO - Received callback for node 694098b4575bc060ce161b3c INFO: 172.21.0.1:34266 - "POST /node/694098b4575bc060ce161b3c HTTP/1.0" 202 Accepted 2025-12-17 23:24:57,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69411cc4459ff5fc64d2ab11 2025-12-17 23:24:57,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411cc4459ff5fc64d2ab11 HTTP/1.1" 502 150 2025-12-17 23:24:57,119 - lava_callback - ERROR - Error processing callback for node 69411cc4459ff5fc64d2ab11: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69411cc4459ff5fc64d2ab11 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69411cc4459ff5fc64d2ab11 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69411cc4459ff5fc64d2ab11 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:57,121 - lava_callback - INFO - Starting async job submit for node 694095b3575bc060ce1613e5 2025-12-17 23:24:57,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:57,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b3575bc060ce1613e5 HTTP/1.1" 502 150 2025-12-17 23:24:57,128 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b3575bc060ce1613e5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:57,128 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b3575bc060ce1613e5 2025-12-17 23:24:57,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b3575bc060ce1613e5 HTTP/1.1" 502 150 2025-12-17 23:24:57,129 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b3575bc060ce1613e5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:57,373 - lava_callback - INFO - Received callback for node 694098b6575bc060ce161b3f INFO: 172.21.0.1:34280 - "POST /node/694098b6575bc060ce161b3f HTTP/1.0" 202 Accepted 2025-12-17 23:24:57,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ae575bc060ce1613d3 2025-12-17 23:24:57,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ae575bc060ce1613d3 HTTP/1.1" 502 150 2025-12-17 23:24:57,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ae575bc060ce1613d3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:57,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a5575bc060ce1613be 2025-12-17 23:24:57,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a5575bc060ce1613be HTTP/1.1" 502 150 2025-12-17 23:24:57,461 - lava_callback - ERROR - Error processing callback for node 694095a5575bc060ce1613be: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a5575bc060ce1613be (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a5575bc060ce1613be (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a5575bc060ce1613be (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:57,463 - lava_callback - INFO - Starting async job submit for node 694095b4575bc060ce1613ec 2025-12-17 23:24:57,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:57,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b4575bc060ce1613ec HTTP/1.1" 502 150 2025-12-17 23:24:57,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b4575bc060ce1613ec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:57,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b4575bc060ce1613ec 2025-12-17 23:24:57,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b4575bc060ce1613ec HTTP/1.1" 502 150 2025-12-17 23:24:57,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b4575bc060ce1613ec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:57,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a6575bc060ce1613c1 2025-12-17 23:24:57,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a6575bc060ce1613c1 HTTP/1.1" 502 150 2025-12-17 23:24:57,674 - lava_callback - ERROR - Error processing callback for node 694095a6575bc060ce1613c1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a6575bc060ce1613c1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a6575bc060ce1613c1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a6575bc060ce1613c1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:57,675 - lava_callback - INFO - Starting async job submit for node 694095b5575bc060ce1613f1 2025-12-17 23:24:57,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:57,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b5575bc060ce1613f1 HTTP/1.1" 502 150 2025-12-17 23:24:57,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b5575bc060ce1613f1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:57,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b5575bc060ce1613f1 2025-12-17 23:24:57,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b5575bc060ce1613f1 HTTP/1.1" 502 150 2025-12-17 23:24:57,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b5575bc060ce1613f1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:57,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a7575bc060ce1613c4 2025-12-17 23:24:57,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a7575bc060ce1613c4 HTTP/1.1" 502 150 2025-12-17 23:24:57,710 - lava_callback - ERROR - Error processing callback for node 694095a7575bc060ce1613c4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a7575bc060ce1613c4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a7575bc060ce1613c4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a7575bc060ce1613c4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:57,712 - lava_callback - INFO - Starting async job submit for node 694095bc575bc060ce161402 2025-12-17 23:24:57,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:57,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bc575bc060ce161402 HTTP/1.1" 502 150 2025-12-17 23:24:57,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095bc575bc060ce161402'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:57,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095bc575bc060ce161402 2025-12-17 23:24:57,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bc575bc060ce161402 HTTP/1.1" 502 150 2025-12-17 23:24:57,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095bc575bc060ce161402'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:57,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095bb575bc060ce1613ff 2025-12-17 23:24:57,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bb575bc060ce1613ff HTTP/1.1" 502 150 2025-12-17 23:24:57,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095bb575bc060ce1613ff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:58,081 - lava_callback - INFO - Received callback for node 694098b8575bc060ce161b46 INFO: 172.21.0.1:34286 - "POST /node/694098b8575bc060ce161b46 HTTP/1.0" 202 Accepted 2025-12-17 23:24:58,740 - lava_callback - INFO - Received callback for node 694098ba575bc060ce161b49 INFO: 172.21.0.1:34302 - "POST /node/694098ba575bc060ce161b49 HTTP/1.0" 202 Accepted 2025-12-17 23:24:58,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095af575bc060ce1613d6 2025-12-17 23:24:58,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095af575bc060ce1613d6 HTTP/1.1" 502 150 2025-12-17 23:24:58,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095af575bc060ce1613d6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:58,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095a9575bc060ce1613c7 2025-12-17 23:24:58,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095a9575bc060ce1613c7 HTTP/1.1" 502 150 2025-12-17 23:24:58,800 - lava_callback - ERROR - Error processing callback for node 694095a9575bc060ce1613c7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a9575bc060ce1613c7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a9575bc060ce1613c7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095a9575bc060ce1613c7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:24:58,803 - lava_callback - INFO - Starting async job submit for node 694095bd575bc060ce161405 2025-12-17 23:24:58,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:24:58,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bd575bc060ce161405 HTTP/1.1" 502 150 2025-12-17 23:24:58,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095bd575bc060ce161405'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:58,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095bd575bc060ce161405 2025-12-17 23:24:58,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bd575bc060ce161405 HTTP/1.1" 502 150 2025-12-17 23:24:58,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095bd575bc060ce161405'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:59,129 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b3575bc060ce1613e5 2025-12-17 23:24:59,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b3575bc060ce1613e5 HTTP/1.1" 502 150 2025-12-17 23:24:59,131 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b3575bc060ce1613e5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:59,415 - lava_callback - INFO - Received callback for node 694098bc575bc060ce161b4c INFO: 172.21.0.1:34310 - "POST /node/694098bc575bc060ce161b4c HTTP/1.0" 202 Accepted 2025-12-17 23:24:59,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b4575bc060ce1613ec 2025-12-17 23:24:59,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b4575bc060ce1613ec HTTP/1.1" 502 150 2025-12-17 23:24:59,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b4575bc060ce1613ec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:59,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b0575bc060ce1613d9 2025-12-17 23:24:59,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b0575bc060ce1613d9 HTTP/1.1" 502 150 2025-12-17 23:24:59,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b0575bc060ce1613d9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:59,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b5575bc060ce1613f1 2025-12-17 23:24:59,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b5575bc060ce1613f1 HTTP/1.1" 502 150 2025-12-17 23:24:59,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b5575bc060ce1613f1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:24:59,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095bc575bc060ce161402 2025-12-17 23:24:59,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bc575bc060ce161402 HTTP/1.1" 502 150 2025-12-17 23:24:59,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095bc575bc060ce161402'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:00,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b1575bc060ce1613de 2025-12-17 23:25:00,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b1575bc060ce1613de HTTP/1.1" 502 150 2025-12-17 23:25:00,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b1575bc060ce1613de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:00,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095bd575bc060ce161405 2025-12-17 23:25:00,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bd575bc060ce161405 HTTP/1.1" 502 150 2025-12-17 23:25:00,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095bd575bc060ce161405'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:01,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940959f575bc060ce1613a3 2025-12-17 23:25:01,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940959f575bc060ce1613a3 HTTP/1.1" 502 150 2025-12-17 23:25:01,427 - lava_callback - ERROR - Error processing callback for node 6940959f575bc060ce1613a3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940959f575bc060ce1613a3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940959f575bc060ce1613a3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940959f575bc060ce1613a3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:01,430 - lava_callback - INFO - Starting async job submit for node 694095be575bc060ce161408 2025-12-17 23:25:01,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:01,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095be575bc060ce161408 HTTP/1.1" 502 150 2025-12-17 23:25:01,437 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095be575bc060ce161408'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:01,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095be575bc060ce161408 2025-12-17 23:25:01,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095be575bc060ce161408 HTTP/1.1" 502 150 2025-12-17 23:25:01,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095be575bc060ce161408'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:03,131 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b3575bc060ce1613e5 INFO: 172.21.0.1:41754 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:25:03,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b7575bc060ce1613f9 HTTP/1.1" 502 150 2025-12-17 23:25:03,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b7575bc060ce1613f9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:03,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095be575bc060ce161408 2025-12-17 23:25:03,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095be575bc060ce161408 HTTP/1.1" 502 150 2025-12-17 23:25:03,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095be575bc060ce161408'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:03,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b4575bc060ce1613ec 2025-12-17 23:25:03,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b4575bc060ce1613ec HTTP/1.1" 502 150 2025-12-17 23:25:03,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b4575bc060ce1613ec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:03,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b5575bc060ce1613f1 2025-12-17 23:25:03,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b5575bc060ce1613f1 HTTP/1.1" 502 150 2025-12-17 23:25:03,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b5575bc060ce1613f1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:03,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095bc575bc060ce161402 2025-12-17 23:25:03,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bc575bc060ce161402 HTTP/1.1" 502 150 2025-12-17 23:25:03,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095bc575bc060ce161402'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:03,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b9575bc060ce1613fc 2025-12-17 23:25:03,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b9575bc060ce1613fc HTTP/1.1" 502 150 2025-12-17 23:25:03,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b9575bc060ce1613fc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:04,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095bd575bc060ce161405 2025-12-17 23:25:04,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bd575bc060ce161405 HTTP/1.1" 502 150 2025-12-17 23:25:04,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095bd575bc060ce161405'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:05,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095bb575bc060ce1613ff 2025-12-17 23:25:05,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bb575bc060ce1613ff HTTP/1.1" 502 150 2025-12-17 23:25:05,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095bb575bc060ce1613ff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41756 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:25:07,284 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b7575bc060ce1613f9 2025-12-17 23:25:07,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b7575bc060ce1613f9 HTTP/1.1" 502 150 2025-12-17 23:25:07,286 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b7575bc060ce1613f9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:07,375 - lava_callback - INFO - Received callback for node 694098c6575bc060ce161b5f INFO: 172.21.0.1:41760 - "POST /node/694098c6575bc060ce161b5f HTTP/1.0" 202 Accepted 2025-12-17 23:25:07,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095be575bc060ce161408 2025-12-17 23:25:07,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095be575bc060ce161408 HTTP/1.1" 502 150 2025-12-17 23:25:07,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095be575bc060ce161408'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:08,525 - lava_callback - INFO - Received callback for node 694098c2575bc060ce161b55 INFO: 172.21.0.1:41770 - "POST /node/694098c2575bc060ce161b55 HTTP/1.0" 202 Accepted 2025-12-17 23:25:08,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b1575bc060ce1613de 2025-12-17 23:25:08,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b1575bc060ce1613de HTTP/1.1" 502 150 2025-12-17 23:25:08,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b1575bc060ce1613de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:09,230 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095aa575bc060ce1613ca 2025-12-17 23:25:09,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095aa575bc060ce1613ca HTTP/1.1" 502 150 2025-12-17 23:25:09,232 - lava_callback - ERROR - Error processing callback for node 694095aa575bc060ce1613ca: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095aa575bc060ce1613ca (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095aa575bc060ce1613ca (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095aa575bc060ce1613ca (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:09,235 - lava_callback - INFO - Starting async job submit for node 694095c3575bc060ce16140b 2025-12-17 23:25:09,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:09,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c3575bc060ce16140b HTTP/1.1" 502 150 2025-12-17 23:25:09,242 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c3575bc060ce16140b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:09,242 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c3575bc060ce16140b 2025-12-17 23:25:09,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c3575bc060ce16140b HTTP/1.1" 502 150 2025-12-17 23:25:09,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c3575bc060ce16140b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:09,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ab575bc060ce1613cd 2025-12-17 23:25:09,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ab575bc060ce1613cd HTTP/1.1" 502 150 2025-12-17 23:25:09,946 - lava_callback - ERROR - Error processing callback for node 694095ab575bc060ce1613cd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ab575bc060ce1613cd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ab575bc060ce1613cd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ab575bc060ce1613cd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:09,949 - lava_callback - INFO - Starting async job submit for node 694095c4575bc060ce16140e 2025-12-17 23:25:09,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:09,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c4575bc060ce16140e HTTP/1.1" 502 150 2025-12-17 23:25:09,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c4575bc060ce16140e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:09,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c4575bc060ce16140e 2025-12-17 23:25:09,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c4575bc060ce16140e HTTP/1.1" 502 150 2025-12-17 23:25:09,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c4575bc060ce16140e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:11,243 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c3575bc060ce16140b 2025-12-17 23:25:11,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c3575bc060ce16140b HTTP/1.1" 502 150 2025-12-17 23:25:11,244 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c3575bc060ce16140b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:11,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b4575bc060ce1613ec 2025-12-17 23:25:11,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b4575bc060ce1613ec HTTP/1.1" 502 150 2025-12-17 23:25:11,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b4575bc060ce1613ec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:11,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b5575bc060ce1613f1 2025-12-17 23:25:11,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b5575bc060ce1613f1 HTTP/1.1" 502 150 2025-12-17 23:25:11,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b5575bc060ce1613f1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:11,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095bc575bc060ce161402 2025-12-17 23:25:11,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bc575bc060ce161402 HTTP/1.1" 502 150 2025-12-17 23:25:11,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095bc575bc060ce161402'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:11,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c4575bc060ce16140e 2025-12-17 23:25:11,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c4575bc060ce16140e HTTP/1.1" 502 150 2025-12-17 23:25:11,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c4575bc060ce16140e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:12,405 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ac575bc060ce1613d0 2025-12-17 23:25:12,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ac575bc060ce1613d0 HTTP/1.1" 502 150 2025-12-17 23:25:12,407 - lava_callback - ERROR - Error processing callback for node 694095ac575bc060ce1613d0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ac575bc060ce1613d0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ac575bc060ce1613d0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ac575bc060ce1613d0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:12,408 - lava_callback - INFO - Starting async job submit for node 69409ac4575bc060ce16204a 2025-12-17 23:25:12,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:12,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac4575bc060ce16204a HTTP/1.1" 502 150 2025-12-17 23:25:12,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac4575bc060ce16204a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:12,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac4575bc060ce16204a 2025-12-17 23:25:12,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac4575bc060ce16204a HTTP/1.1" 502 150 2025-12-17 23:25:12,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac4575bc060ce16204a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:12,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095bd575bc060ce161405 2025-12-17 23:25:12,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bd575bc060ce161405 HTTP/1.1" 502 150 2025-12-17 23:25:12,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095bd575bc060ce161405'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:13,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ae575bc060ce1613d3 2025-12-17 23:25:13,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ae575bc060ce1613d3 HTTP/1.1" 502 150 2025-12-17 23:25:13,388 - lava_callback - ERROR - Error processing callback for node 694095ae575bc060ce1613d3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ae575bc060ce1613d3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ae575bc060ce1613d3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ae575bc060ce1613d3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:13,390 - lava_callback - INFO - Starting async job submit for node 694095c7575bc060ce16141a 2025-12-17 23:25:13,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:13,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c7575bc060ce16141a HTTP/1.1" 502 150 2025-12-17 23:25:13,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c7575bc060ce16141a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:13,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c7575bc060ce16141a 2025-12-17 23:25:13,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c7575bc060ce16141a HTTP/1.1" 502 150 2025-12-17 23:25:13,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c7575bc060ce16141a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:14,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac4575bc060ce16204a 2025-12-17 23:25:14,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095af575bc060ce1613d6 INFO: 172.21.0.1:55444 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:25:15,244 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c3575bc060ce16140b 2025-12-17 23:25:15,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c3575bc060ce16140b HTTP/1.1" 502 150 2025-12-17 23:25:15,245 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c3575bc060ce16140b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:15,286 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b7575bc060ce1613f9 2025-12-17 23:25:15,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b7575bc060ce1613f9 HTTP/1.1" 502 150 2025-12-17 23:25:15,287 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b7575bc060ce1613f9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:15,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c7575bc060ce16141a 2025-12-17 23:25:15,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c7575bc060ce16141a HTTP/1.1" 502 150 2025-12-17 23:25:15,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c7575bc060ce16141a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:15,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095be575bc060ce161408 2025-12-17 23:25:15,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095be575bc060ce161408 HTTP/1.1" 502 150 2025-12-17 23:25:15,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095be575bc060ce161408'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:15,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b0575bc060ce1613d9 2025-12-17 23:25:15,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b0575bc060ce1613d9 HTTP/1.1" 502 150 2025-12-17 23:25:15,594 - lava_callback - ERROR - Error processing callback for node 694095b0575bc060ce1613d9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b0575bc060ce1613d9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b0575bc060ce1613d9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b0575bc060ce1613d9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:15,596 - lava_callback - INFO - Starting async job submit for node 694095c8575bc060ce161421 2025-12-17 23:25:15,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:15,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c8575bc060ce161421 HTTP/1.1" 502 150 2025-12-17 23:25:15,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c8575bc060ce161421'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:15,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c8575bc060ce161421 2025-12-17 23:25:15,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c8575bc060ce161421 HTTP/1.1" 502 150 2025-12-17 23:25:15,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c8575bc060ce161421'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:15,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c4575bc060ce16140e 2025-12-17 23:25:15,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c4575bc060ce16140e HTTP/1.1" 502 150 2025-12-17 23:25:15,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c4575bc060ce16140e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55456 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:25:16,750 - lava_callback - INFO - Received callback for node 694098c8575bc060ce161b62 INFO: 172.21.0.1:55472 - "POST /node/694098c8575bc060ce161b62 HTTP/1.0" 202 Accepted 2025-12-17 23:25:17,424 - lava_callback - INFO - Received callback for node 694098cb575bc060ce161b69 INFO: 172.21.0.1:55486 - "POST /node/694098cb575bc060ce161b69 HTTP/1.0" 202 Accepted 2025-12-17 23:25:17,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c8575bc060ce161421 2025-12-17 23:25:17,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c8575bc060ce161421 HTTP/1.1" 502 150 2025-12-17 23:25:17,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c8575bc060ce161421'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:18,105 - lava_callback - INFO - Received callback for node 694098cc575bc060ce161b6c INFO: 172.21.0.1:55492 - "POST /node/694098cc575bc060ce161b6c HTTP/1.0" 202 Accepted 2025-12-17 23:25:18,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b3575bc060ce1613e5 HTTP/1.1" 502 150 2025-12-17 23:25:18,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b3575bc060ce1613e5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:18,769 - lava_callback - INFO - Received callback for node 694098ce575bc060ce161b6f INFO: 172.21.0.1:55508 - "POST /node/694098ce575bc060ce161b6f HTTP/1.0" 202 Accepted 2025-12-17 23:25:19,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c7575bc060ce16141a 2025-12-17 23:25:19,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c7575bc060ce16141a HTTP/1.1" 502 150 2025-12-17 23:25:19,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c7575bc060ce16141a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:19,440 - lava_callback - INFO - Received callback for node 694098d0575bc060ce161b72 INFO: 172.21.0.1:55524 - "POST /node/694098d0575bc060ce161b72 HTTP/1.0" 202 Accepted 2025-12-17 23:25:19,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b9575bc060ce1613fc 2025-12-17 23:25:19,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b9575bc060ce1613fc HTTP/1.1" 502 150 2025-12-17 23:25:19,928 - lava_callback - ERROR - Error processing callback for node 694095b9575bc060ce1613fc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b9575bc060ce1613fc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b9575bc060ce1613fc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b9575bc060ce1613fc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:19,930 - lava_callback - INFO - Starting async job submit for node 694095cf575bc060ce161434 2025-12-17 23:25:19,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:19,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cf575bc060ce161434 HTTP/1.1" 502 150 2025-12-17 23:25:19,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095cf575bc060ce161434'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:19,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095cf575bc060ce161434 2025-12-17 23:25:19,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cf575bc060ce161434 HTTP/1.1" 502 150 2025-12-17 23:25:19,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095cf575bc060ce161434'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:20,097 - lava_callback - INFO - Received callback for node 694098d2575bc060ce161b75 INFO: 172.21.0.1:49696 - "POST /node/694098d2575bc060ce161b75 HTTP/1.0" 202 Accepted 2025-12-17 23:25:21,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c8575bc060ce161421 2025-12-17 23:25:21,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c8575bc060ce161421 HTTP/1.1" 502 150 2025-12-17 23:25:21,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c8575bc060ce161421'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:21,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095bb575bc060ce1613ff 2025-12-17 23:25:21,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bb575bc060ce1613ff HTTP/1.1" 502 150 2025-12-17 23:25:21,935 - lava_callback - ERROR - Error processing callback for node 694095bb575bc060ce1613ff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095bb575bc060ce1613ff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095bb575bc060ce1613ff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095bb575bc060ce1613ff (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:21,938 - lava_callback - INFO - Starting async job submit for node 694095c6575bc060ce161413 2025-12-17 23:25:21,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:21,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095cf575bc060ce161434 2025-12-17 23:25:21,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cf575bc060ce161434 HTTP/1.1" 502 150 2025-12-17 23:25:21,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095cf575bc060ce161434'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:21,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c6575bc060ce161413 HTTP/1.1" 502 150 2025-12-17 23:25:21,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c6575bc060ce161413'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:21,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c6575bc060ce161413 2025-12-17 23:25:21,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c6575bc060ce161413 HTTP/1.1" 502 150 2025-12-17 23:25:21,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c6575bc060ce161413'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:23,246 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c3575bc060ce16140b 2025-12-17 23:25:23,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c3575bc060ce16140b HTTP/1.1" 502 150 2025-12-17 23:25:23,247 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c3575bc060ce16140b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:23,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c6575bc060ce161413 2025-12-17 23:25:23,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c6575bc060ce161413 HTTP/1.1" 502 150 2025-12-17 23:25:23,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c6575bc060ce161413'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:23,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c4575bc060ce16140e 2025-12-17 23:25:23,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c4575bc060ce16140e HTTP/1.1" 502 150 2025-12-17 23:25:23,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c4575bc060ce16140e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:24,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b1575bc060ce1613de 2025-12-17 23:25:24,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b1575bc060ce1613de HTTP/1.1" 502 150 2025-12-17 23:25:24,654 - lava_callback - ERROR - Error processing callback for node 694095b1575bc060ce1613de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b1575bc060ce1613de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b1575bc060ce1613de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b1575bc060ce1613de (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:24,656 - lava_callback - INFO - Starting async job submit for node 694095cd575bc060ce161431 2025-12-17 23:25:24,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:24,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cd575bc060ce161431 HTTP/1.1" 502 150 2025-12-17 23:25:24,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095cd575bc060ce161431'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:24,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095cd575bc060ce161431 2025-12-17 23:25:24,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cd575bc060ce161431 HTTP/1.1" 502 150 2025-12-17 23:25:24,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095cd575bc060ce161431'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:25,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095cf575bc060ce161434 2025-12-17 23:25:25,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cf575bc060ce161434 HTTP/1.1" 502 150 2025-12-17 23:25:25,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095cf575bc060ce161434'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:26,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b3575bc060ce1613e5 2025-12-17 23:25:26,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b3575bc060ce1613e5 HTTP/1.1" 502 150 2025-12-17 23:25:26,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095b3575bc060ce1613e5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49698 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:25:26,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095cd575bc060ce161431 2025-12-17 23:25:26,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cd575bc060ce161431 HTTP/1.1" 502 150 2025-12-17 23:25:26,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095cd575bc060ce161431'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:27,395 - lava_callback - INFO - Received callback for node 694098d3575bc060ce161b7e INFO: 172.21.0.1:49706 - "POST /node/694098d3575bc060ce161b7e HTTP/1.0" 202 Accepted 2025-12-17 23:25:27,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c7575bc060ce16141a 2025-12-17 23:25:27,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c7575bc060ce16141a HTTP/1.1" 502 150 2025-12-17 23:25:27,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c7575bc060ce16141a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:27,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b4575bc060ce1613ec 2025-12-17 23:25:27,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b4575bc060ce1613ec HTTP/1.1" 502 150 2025-12-17 23:25:27,475 - lava_callback - ERROR - Error processing callback for node 694095b4575bc060ce1613ec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b4575bc060ce1613ec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b4575bc060ce1613ec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b4575bc060ce1613ec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:27,477 - lava_callback - INFO - Starting async job submit for node 694095ca575bc060ce161428 2025-12-17 23:25:27,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:27,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ca575bc060ce161428 HTTP/1.1" 502 150 2025-12-17 23:25:27,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ca575bc060ce161428'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:27,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ca575bc060ce161428 2025-12-17 23:25:27,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ca575bc060ce161428 HTTP/1.1" 502 150 2025-12-17 23:25:27,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ca575bc060ce161428'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:27,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b5575bc060ce1613f1 2025-12-17 23:25:27,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b5575bc060ce1613f1 HTTP/1.1" 502 150 2025-12-17 23:25:27,694 - lava_callback - ERROR - Error processing callback for node 694095b5575bc060ce1613f1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b5575bc060ce1613f1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b5575bc060ce1613f1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b5575bc060ce1613f1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:27,696 - lava_callback - INFO - Starting async job submit for node 694095d0575bc060ce161437 2025-12-17 23:25:27,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:27,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d0575bc060ce161437 HTTP/1.1" 502 150 2025-12-17 23:25:27,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d0575bc060ce161437'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:27,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d0575bc060ce161437 2025-12-17 23:25:27,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d0575bc060ce161437 HTTP/1.1" 502 150 2025-12-17 23:25:27,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d0575bc060ce161437'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:27,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095bc575bc060ce161402 2025-12-17 23:25:27,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bc575bc060ce161402 HTTP/1.1" 502 150 2025-12-17 23:25:27,724 - lava_callback - ERROR - Error processing callback for node 694095bc575bc060ce161402: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095bc575bc060ce161402 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095bc575bc060ce161402 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095bc575bc060ce161402 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:27,726 - lava_callback - INFO - Starting async job submit for node 694095cc575bc060ce16142e 2025-12-17 23:25:27,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:27,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cc575bc060ce16142e HTTP/1.1" 502 150 2025-12-17 23:25:27,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095cc575bc060ce16142e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:27,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095cc575bc060ce16142e 2025-12-17 23:25:27,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cc575bc060ce16142e HTTP/1.1" 502 150 2025-12-17 23:25:27,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095cc575bc060ce16142e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:27,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c6575bc060ce161413 2025-12-17 23:25:27,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c6575bc060ce161413 HTTP/1.1" 502 150 2025-12-17 23:25:27,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c6575bc060ce161413'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:28,035 - lava_callback - INFO - Received callback for node 694098d6575bc060ce161b85 INFO: 172.21.0.1:49720 - "POST /node/694098d6575bc060ce161b85 HTTP/1.0" 202 Accepted 2025-12-17 23:25:28,724 - lava_callback - INFO - Received callback for node 694098d8575bc060ce161b88 INFO: 172.21.0.1:49732 - "POST /node/694098d8575bc060ce161b88 HTTP/1.0" 202 Accepted 2025-12-17 23:25:28,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095bd575bc060ce161405 2025-12-17 23:25:28,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095bd575bc060ce161405 HTTP/1.1" 502 150 2025-12-17 23:25:28,829 - lava_callback - ERROR - Error processing callback for node 694095bd575bc060ce161405: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095bd575bc060ce161405 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095bd575bc060ce161405 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095bd575bc060ce161405 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:28,831 - lava_callback - INFO - Starting async job submit for node 694095d2575bc060ce16143d 2025-12-17 23:25:28,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:28,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d2575bc060ce16143d HTTP/1.1" 502 150 2025-12-17 23:25:28,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d2575bc060ce16143d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:28,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d2575bc060ce16143d 2025-12-17 23:25:28,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d2575bc060ce16143d HTTP/1.1" 502 150 2025-12-17 23:25:28,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d2575bc060ce16143d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:29,412 - lava_callback - INFO - Received callback for node 694098da575bc060ce161b8f INFO: 172.21.0.1:49748 - "POST /node/694098da575bc060ce161b8f HTTP/1.0" 202 Accepted 2025-12-17 23:25:29,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ca575bc060ce161428 2025-12-17 23:25:29,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c8575bc060ce161421 2025-12-17 23:25:29,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c8575bc060ce161421 HTTP/1.1" 502 150 2025-12-17 23:25:29,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c8575bc060ce161421'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:29,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d0575bc060ce161437 2025-12-17 23:25:29,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d0575bc060ce161437 HTTP/1.1" 502 150 2025-12-17 23:25:29,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d0575bc060ce161437'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:29,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095cc575bc060ce16142e 2025-12-17 23:25:29,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cc575bc060ce16142e HTTP/1.1" 502 150 2025-12-17 23:25:29,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095cc575bc060ce16142e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:29,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac4575bc060ce16204a HTTP/1.1" 502 150 2025-12-17 23:25:29,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac4575bc060ce16204a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:30,090 - lava_callback - INFO - Received callback for node 694098dc575bc060ce161b92 INFO: 172.21.0.1:53652 - "POST /node/694098dc575bc060ce161b92 HTTP/1.0" 202 Accepted 2025-12-17 23:25:30,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095af575bc060ce1613d6 HTTP/1.1" 502 150 2025-12-17 23:25:30,160 - lava_callback - ERROR - Error processing callback for node 694095af575bc060ce1613d6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095af575bc060ce1613d6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095af575bc060ce1613d6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095af575bc060ce1613d6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:30,162 - lava_callback - INFO - Starting async job submit for node 694095d1575bc060ce16143a 2025-12-17 23:25:30,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:30,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d1575bc060ce16143a HTTP/1.1" 502 150 2025-12-17 23:25:30,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d1575bc060ce16143a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:30,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d1575bc060ce16143a 2025-12-17 23:25:30,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d1575bc060ce16143a HTTP/1.1" 502 150 2025-12-17 23:25:30,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d1575bc060ce16143a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:30,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095cd575bc060ce161431 2025-12-17 23:25:30,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cd575bc060ce161431 HTTP/1.1" 502 150 2025-12-17 23:25:30,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095cd575bc060ce161431'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:30,773 - lava_callback - INFO - Received callback for node 694098de575bc060ce161b95 INFO: 172.21.0.1:53668 - "POST /node/694098de575bc060ce161b95 HTTP/1.0" 202 Accepted 2025-12-17 23:25:30,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d2575bc060ce16143d 2025-12-17 23:25:30,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d2575bc060ce16143d HTTP/1.1" 502 150 2025-12-17 23:25:30,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d2575bc060ce16143d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:31,287 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b7575bc060ce1613f9 2025-12-17 23:25:31,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b7575bc060ce1613f9 HTTP/1.1" 502 150 2025-12-17 23:25:31,289 - lava_callback - ERROR - Error processing callback for node 694095b7575bc060ce1613f9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b7575bc060ce1613f9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b7575bc060ce1613f9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b7575bc060ce1613f9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:31,291 - lava_callback - INFO - Starting async job submit for node 694095d5575bc060ce161443 2025-12-17 23:25:31,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:31,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d5575bc060ce161443 HTTP/1.1" 502 150 2025-12-17 23:25:31,298 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d5575bc060ce161443'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:31,298 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d5575bc060ce161443 2025-12-17 23:25:31,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d5575bc060ce161443 HTTP/1.1" 502 150 2025-12-17 23:25:31,299 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d5575bc060ce161443'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:31,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095be575bc060ce161408 2025-12-17 23:25:31,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095be575bc060ce161408 HTTP/1.1" 502 150 2025-12-17 23:25:31,444 - lava_callback - ERROR - Error processing callback for node 694095be575bc060ce161408: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095be575bc060ce161408 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095be575bc060ce161408 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095be575bc060ce161408 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:31,446 - lava_callback - INFO - Starting async job submit for node 694095d3575bc060ce161440 2025-12-17 23:25:31,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:31,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d3575bc060ce161440 HTTP/1.1" 502 150 2025-12-17 23:25:31,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d3575bc060ce161440'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:31,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d3575bc060ce161440 2025-12-17 23:25:31,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d3575bc060ce161440 HTTP/1.1" 502 150 2025-12-17 23:25:31,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d3575bc060ce161440'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:32,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d1575bc060ce16143a 2025-12-17 23:25:32,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d1575bc060ce16143a HTTP/1.1" 502 150 2025-12-17 23:25:32,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d1575bc060ce16143a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:33,299 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d5575bc060ce161443 2025-12-17 23:25:33,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d5575bc060ce161443 HTTP/1.1" 502 150 2025-12-17 23:25:33,300 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d5575bc060ce161443'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:33,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d3575bc060ce161440 2025-12-17 23:25:33,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d3575bc060ce161440 HTTP/1.1" 502 150 2025-12-17 23:25:33,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d3575bc060ce161440'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:33,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d0575bc060ce161437 2025-12-17 23:25:33,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d0575bc060ce161437 HTTP/1.1" 502 150 2025-12-17 23:25:33,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d0575bc060ce161437'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:33,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095cc575bc060ce16142e 2025-12-17 23:25:33,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cc575bc060ce16142e HTTP/1.1" 502 150 2025-12-17 23:25:33,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095cc575bc060ce16142e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:33,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac4575bc060ce16204a 2025-12-17 23:25:33,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac4575bc060ce16204a HTTP/1.1" 502 150 2025-12-17 23:25:33,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac4575bc060ce16204a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:33,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095cf575bc060ce161434 2025-12-17 23:25:33,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cf575bc060ce161434 HTTP/1.1" 502 150 2025-12-17 23:25:33,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095cf575bc060ce161434'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:34,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d2575bc060ce16143d 2025-12-17 23:25:34,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d2575bc060ce16143d HTTP/1.1" 502 150 2025-12-17 23:25:34,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d2575bc060ce16143d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:35,468 - lava_callback - INFO - Received callback for node 69409a86575bc060ce161fc5 INFO: 172.21.0.1:53676 - "POST /node/69409a86575bc060ce161fc5 HTTP/1.0" 202 Accepted 2025-12-17 23:25:35,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c6575bc060ce161413 2025-12-17 23:25:35,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c6575bc060ce161413 HTTP/1.1" 502 150 2025-12-17 23:25:35,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095c6575bc060ce161413'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:36,351 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d1575bc060ce16143a 2025-12-17 23:25:36,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d1575bc060ce16143a HTTP/1.1" 502 150 2025-12-17 23:25:36,353 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d1575bc060ce16143a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53684 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:25:37,080 - lava_callback - INFO - Received callback for node 69409c78575bc060ce1625d8 INFO: 172.21.0.1:53698 - "POST /node/69409c78575bc060ce1625d8 HTTP/1.0" 202 Accepted 2025-12-17 23:25:37,301 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d5575bc060ce161443 2025-12-17 23:25:37,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d5575bc060ce161443 HTTP/1.1" 502 150 2025-12-17 23:25:37,302 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d5575bc060ce161443'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:37,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d3575bc060ce161440 2025-12-17 23:25:37,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d3575bc060ce161440 HTTP/1.1" 502 150 2025-12-17 23:25:37,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d3575bc060ce161440'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:38,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095cd575bc060ce161431 2025-12-17 23:25:38,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cd575bc060ce161431 HTTP/1.1" 502 150 2025-12-17 23:25:38,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095cd575bc060ce161431'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:39,248 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c3575bc060ce16140b 2025-12-17 23:25:39,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c3575bc060ce16140b HTTP/1.1" 502 150 2025-12-17 23:25:39,249 - lava_callback - ERROR - Error processing callback for node 694095c3575bc060ce16140b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095c3575bc060ce16140b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095c3575bc060ce16140b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095c3575bc060ce16140b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:39,251 - lava_callback - INFO - Starting async job submit for node 694095d6575bc060ce161446 2025-12-17 23:25:39,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:39,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d6575bc060ce161446 HTTP/1.1" 502 150 2025-12-17 23:25:39,283 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d6575bc060ce161446'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:39,283 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d6575bc060ce161446 2025-12-17 23:25:39,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d6575bc060ce161446 HTTP/1.1" 502 150 2025-12-17 23:25:39,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d6575bc060ce161446'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:39,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c4575bc060ce16140e 2025-12-17 23:25:39,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c4575bc060ce16140e HTTP/1.1" 502 150 2025-12-17 23:25:39,988 - lava_callback - ERROR - Error processing callback for node 694095c4575bc060ce16140e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095c4575bc060ce16140e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095c4575bc060ce16140e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095c4575bc060ce16140e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:39,991 - lava_callback - INFO - Starting async job submit for node 694095d8575bc060ce16144c 2025-12-17 23:25:39,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:41,284 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d6575bc060ce161446 2025-12-17 23:25:41,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d6575bc060ce161446 HTTP/1.1" 502 150 2025-12-17 23:25:41,285 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d6575bc060ce161446'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:41,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d0575bc060ce161437 2025-12-17 23:25:41,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d0575bc060ce161437 HTTP/1.1" 502 150 2025-12-17 23:25:41,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d0575bc060ce161437'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:41,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095cc575bc060ce16142e 2025-12-17 23:25:41,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cc575bc060ce16142e HTTP/1.1" 502 150 2025-12-17 23:25:41,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095cc575bc060ce16142e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:41,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac4575bc060ce16204a 2025-12-17 23:25:41,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac4575bc060ce16204a HTTP/1.1" 502 150 2025-12-17 23:25:41,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac4575bc060ce16204a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:42,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095b3575bc060ce1613e5 2025-12-17 23:25:42,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095b3575bc060ce1613e5 HTTP/1.1" 502 150 2025-12-17 23:25:42,387 - lava_callback - ERROR - Error processing callback for node 694095b3575bc060ce1613e5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b3575bc060ce1613e5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b3575bc060ce1613e5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095b3575bc060ce1613e5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:42,389 - lava_callback - INFO - Starting async job submit for node 694095da575bc060ce161451 2025-12-17 23:25:42,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:42,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095da575bc060ce161451 HTTP/1.1" 502 150 2025-12-17 23:25:42,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095da575bc060ce161451'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:42,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095da575bc060ce161451 2025-12-17 23:25:42,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095da575bc060ce161451 HTTP/1.1" 502 150 2025-12-17 23:25:42,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095da575bc060ce161451'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:42,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d2575bc060ce16143d 2025-12-17 23:25:42,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d2575bc060ce16143d HTTP/1.1" 502 150 2025-12-17 23:25:42,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d2575bc060ce16143d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:43,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c7575bc060ce16141a 2025-12-17 23:25:43,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c7575bc060ce16141a HTTP/1.1" 502 150 2025-12-17 23:25:43,404 - lava_callback - ERROR - Error processing callback for node 694095c7575bc060ce16141a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095c7575bc060ce16141a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095c7575bc060ce16141a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095c7575bc060ce16141a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:43,406 - lava_callback - INFO - Starting async job submit for node 694095d7575bc060ce161449 2025-12-17 23:25:43,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:43,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d7575bc060ce161449 HTTP/1.1" 502 150 2025-12-17 23:25:43,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d7575bc060ce161449'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:43,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d7575bc060ce161449 2025-12-17 23:25:43,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d7575bc060ce161449 HTTP/1.1" 502 150 2025-12-17 23:25:43,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d7575bc060ce161449'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:44,353 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d1575bc060ce16143a 2025-12-17 23:25:44,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d1575bc060ce16143a HTTP/1.1" 502 150 2025-12-17 23:25:44,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d1575bc060ce16143a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:44,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095da575bc060ce161451 2025-12-17 23:25:44,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095da575bc060ce161451 HTTP/1.1" 502 150 2025-12-17 23:25:44,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095da575bc060ce161451'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:44,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ca575bc060ce161428 HTTP/1.1" 502 150 2025-12-17 23:25:44,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ca575bc060ce161428'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:45,286 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d6575bc060ce161446 2025-12-17 23:25:45,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d6575bc060ce161446 HTTP/1.1" 502 150 2025-12-17 23:25:45,287 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d6575bc060ce161446'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:45,302 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d5575bc060ce161443 2025-12-17 23:25:45,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d5575bc060ce161443 HTTP/1.1" 502 150 2025-12-17 23:25:45,303 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d5575bc060ce161443'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:45,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d7575bc060ce161449 2025-12-17 23:25:45,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d7575bc060ce161449 HTTP/1.1" 502 150 2025-12-17 23:25:45,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d7575bc060ce161449'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:45,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d3575bc060ce161440 2025-12-17 23:25:45,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d3575bc060ce161440 HTTP/1.1" 502 150 2025-12-17 23:25:45,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d3575bc060ce161440'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:45,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c8575bc060ce161421 2025-12-17 23:25:45,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c8575bc060ce161421 HTTP/1.1" 502 150 2025-12-17 23:25:45,610 - lava_callback - ERROR - Error processing callback for node 694095c8575bc060ce161421: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095c8575bc060ce161421 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095c8575bc060ce161421 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095c8575bc060ce161421 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:45,612 - lava_callback - INFO - Starting async job submit for node 694095dd575bc060ce161460 2025-12-17 23:25:45,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:45,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095dd575bc060ce161460 HTTP/1.1" 502 150 2025-12-17 23:25:45,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095dd575bc060ce161460'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:45,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095dd575bc060ce161460 2025-12-17 23:25:45,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095dd575bc060ce161460 HTTP/1.1" 502 150 2025-12-17 23:25:45,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095dd575bc060ce161460'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60756 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:25:47,387 - lava_callback - INFO - Received callback for node 694098e6575bc060ce161ba0 INFO: 172.21.0.1:60772 - "POST /node/694098e6575bc060ce161ba0 HTTP/1.0" 202 Accepted 2025-12-17 23:25:47,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095dd575bc060ce161460 2025-12-17 23:25:47,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095dd575bc060ce161460 HTTP/1.1" 502 150 2025-12-17 23:25:47,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095dd575bc060ce161460'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:48,052 - lava_callback - INFO - Received callback for node 694098e9575bc060ce161bab INFO: 172.21.0.1:60774 - "POST /node/694098e9575bc060ce161bab HTTP/1.0" 202 Accepted 2025-12-17 23:25:48,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095da575bc060ce161451 2025-12-17 23:25:48,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095da575bc060ce161451 HTTP/1.1" 502 150 2025-12-17 23:25:48,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095da575bc060ce161451'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:48,713 - lava_callback - INFO - Received callback for node 694098eb575bc060ce161bb6 INFO: 172.21.0.1:60784 - "POST /node/694098eb575bc060ce161bb6 HTTP/1.0" 202 Accepted 2025-12-17 23:25:48,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ca575bc060ce161428 2025-12-17 23:25:48,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ca575bc060ce161428 HTTP/1.1" 502 150 2025-12-17 23:25:48,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ca575bc060ce161428'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:49,400 - lava_callback - INFO - Received callback for node 694098ed575bc060ce161bb9 INFO: 172.21.0.1:60794 - "POST /node/694098ed575bc060ce161bb9 HTTP/1.0" 202 Accepted 2025-12-17 23:25:49,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d7575bc060ce161449 2025-12-17 23:25:49,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d7575bc060ce161449 HTTP/1.1" 502 150 2025-12-17 23:25:49,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d7575bc060ce161449'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:49,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095cf575bc060ce161434 2025-12-17 23:25:49,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cf575bc060ce161434 HTTP/1.1" 502 150 2025-12-17 23:25:49,951 - lava_callback - ERROR - Error processing callback for node 694095cf575bc060ce161434: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095cf575bc060ce161434 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095cf575bc060ce161434 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095cf575bc060ce161434 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:49,953 - lava_callback - INFO - Starting async job submit for node 694095dc575bc060ce16145d 2025-12-17 23:25:49,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:49,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095dc575bc060ce16145d HTTP/1.1" 502 150 2025-12-17 23:25:49,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095dc575bc060ce16145d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:49,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095dc575bc060ce16145d 2025-12-17 23:25:49,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095dc575bc060ce16145d HTTP/1.1" 502 150 2025-12-17 23:25:49,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095dc575bc060ce16145d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:50,071 - lava_callback - INFO - Received callback for node 694098ef575bc060ce161bbc INFO: 172.21.0.1:35560 - "POST /node/694098ef575bc060ce161bbc HTTP/1.0" 202 Accepted 2025-12-17 23:25:50,732 - lava_callback - INFO - Received callback for node 694098f1575bc060ce161bbf INFO: 172.21.0.1:35570 - "POST /node/694098f1575bc060ce161bbf HTTP/1.0" 202 Accepted 2025-12-17 23:25:51,399 - lava_callback - INFO - Received callback for node 694098f3575bc060ce161bc2 INFO: 172.21.0.1:35572 - "POST /node/694098f3575bc060ce161bc2 HTTP/1.0" 202 Accepted 2025-12-17 23:25:51,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095dd575bc060ce161460 2025-12-17 23:25:51,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095dd575bc060ce161460 HTTP/1.1" 502 150 2025-12-17 23:25:51,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095dd575bc060ce161460'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:51,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095c6575bc060ce161413 2025-12-17 23:25:51,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095c6575bc060ce161413 HTTP/1.1" 502 150 2025-12-17 23:25:51,955 - lava_callback - ERROR - Error processing callback for node 694095c6575bc060ce161413: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095c6575bc060ce161413 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095c6575bc060ce161413 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095c6575bc060ce161413 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:51,957 - lava_callback - INFO - Starting async job submit for node 694095df575bc060ce161463 2025-12-17 23:25:51,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:51,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095dc575bc060ce16145d 2025-12-17 23:25:51,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095dc575bc060ce16145d HTTP/1.1" 502 150 2025-12-17 23:25:51,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095dc575bc060ce16145d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:51,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095df575bc060ce161463 HTTP/1.1" 502 150 2025-12-17 23:25:51,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095df575bc060ce161463'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:51,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095df575bc060ce161463 2025-12-17 23:25:51,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095df575bc060ce161463 HTTP/1.1" 502 150 2025-12-17 23:25:51,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095df575bc060ce161463'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:52,070 - lava_callback - INFO - Received callback for node 694098f5575bc060ce161bc5 INFO: 172.21.0.1:35578 - "POST /node/694098f5575bc060ce161bc5 HTTP/1.0" 202 Accepted 2025-12-17 23:25:52,745 - lava_callback - INFO - Received callback for node 694098f7575bc060ce161bc8 INFO: 172.21.0.1:35586 - "POST /node/694098f7575bc060ce161bc8 HTTP/1.0" 202 Accepted 2025-12-17 23:25:53,287 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d6575bc060ce161446 2025-12-17 23:25:53,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d6575bc060ce161446 HTTP/1.1" 502 150 2025-12-17 23:25:53,288 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d6575bc060ce161446'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:53,421 - lava_callback - INFO - Received callback for node 694098f9575bc060ce161bcb INFO: 172.21.0.1:35596 - "POST /node/694098f9575bc060ce161bcb HTTP/1.0" 202 Accepted 2025-12-17 23:25:53,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095df575bc060ce161463 2025-12-17 23:25:53,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095df575bc060ce161463 HTTP/1.1" 502 150 2025-12-17 23:25:53,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095df575bc060ce161463'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:54,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095cd575bc060ce161431 2025-12-17 23:25:54,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cd575bc060ce161431 HTTP/1.1" 502 150 2025-12-17 23:25:54,678 - lava_callback - ERROR - Error processing callback for node 694095cd575bc060ce161431: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095cd575bc060ce161431 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095cd575bc060ce161431 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095cd575bc060ce161431 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:54,680 - lava_callback - INFO - Starting async job submit for node 694095db575bc060ce16145a 2025-12-17 23:25:54,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:54,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095db575bc060ce16145a HTTP/1.1" 502 150 2025-12-17 23:25:54,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095db575bc060ce16145a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:54,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095db575bc060ce16145a 2025-12-17 23:25:54,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095db575bc060ce16145a HTTP/1.1" 502 150 2025-12-17 23:25:54,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095db575bc060ce16145a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:55,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d8575bc060ce16144c HTTP/1.1" 502 150 2025-12-17 23:25:55,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d8575bc060ce16144c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:55,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d8575bc060ce16144c 2025-12-17 23:25:55,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d8575bc060ce16144c HTTP/1.1" 502 150 2025-12-17 23:25:55,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d8575bc060ce16144c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:55,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095dc575bc060ce16145d 2025-12-17 23:25:55,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095dc575bc060ce16145d HTTP/1.1" 502 150 2025-12-17 23:25:55,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095dc575bc060ce16145d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:56,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095da575bc060ce161451 2025-12-17 23:25:56,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095da575bc060ce161451 HTTP/1.1" 502 150 2025-12-17 23:25:56,425 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095da575bc060ce161451'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35610 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:25:56,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095db575bc060ce16145a 2025-12-17 23:25:56,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095db575bc060ce16145a HTTP/1.1" 502 150 2025-12-17 23:25:56,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095db575bc060ce16145a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:56,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ca575bc060ce161428 2025-12-17 23:25:56,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ca575bc060ce161428 HTTP/1.1" 502 150 2025-12-17 23:25:56,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ca575bc060ce161428'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:57,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d7575bc060ce161449 2025-12-17 23:25:57,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d7575bc060ce161449 HTTP/1.1" 502 150 2025-12-17 23:25:57,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d7575bc060ce161449'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:57,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d8575bc060ce16144c 2025-12-17 23:25:57,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d8575bc060ce16144c HTTP/1.1" 502 150 2025-12-17 23:25:57,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d8575bc060ce16144c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:57,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d0575bc060ce161437 2025-12-17 23:25:57,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d0575bc060ce161437 HTTP/1.1" 502 150 2025-12-17 23:25:57,710 - lava_callback - ERROR - Error processing callback for node 694095d0575bc060ce161437: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d0575bc060ce161437 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d0575bc060ce161437 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d0575bc060ce161437 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:57,713 - lava_callback - INFO - Starting async job submit for node 694095e2575bc060ce161469 2025-12-17 23:25:57,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:57,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e2575bc060ce161469 HTTP/1.1" 502 150 2025-12-17 23:25:57,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e2575bc060ce161469'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:57,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e2575bc060ce161469 2025-12-17 23:25:57,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e2575bc060ce161469 HTTP/1.1" 502 150 2025-12-17 23:25:57,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e2575bc060ce161469'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:57,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095cc575bc060ce16142e 2025-12-17 23:25:57,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095cc575bc060ce16142e HTTP/1.1" 502 150 2025-12-17 23:25:57,737 - lava_callback - ERROR - Error processing callback for node 694095cc575bc060ce16142e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095cc575bc060ce16142e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095cc575bc060ce16142e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095cc575bc060ce16142e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:57,739 - lava_callback - INFO - Starting async job submit for node 694095e5575bc060ce16146c 2025-12-17 23:25:57,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:57,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e5575bc060ce16146c HTTP/1.1" 502 150 2025-12-17 23:25:57,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e5575bc060ce16146c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:57,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e5575bc060ce16146c 2025-12-17 23:25:57,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e5575bc060ce16146c HTTP/1.1" 502 150 2025-12-17 23:25:57,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e5575bc060ce16146c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:57,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac4575bc060ce16204a 2025-12-17 23:25:57,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac4575bc060ce16204a HTTP/1.1" 502 150 2025-12-17 23:25:57,909 - lava_callback - ERROR - Error processing callback for node 69409ac4575bc060ce16204a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ac4575bc060ce16204a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ac4575bc060ce16204a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ac4575bc060ce16204a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:57,912 - lava_callback - INFO - Starting async job submit for node 694095e7575bc060ce16146f 2025-12-17 23:25:57,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:57,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e7575bc060ce16146f HTTP/1.1" 502 150 2025-12-17 23:25:57,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e7575bc060ce16146f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:57,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e7575bc060ce16146f 2025-12-17 23:25:57,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e7575bc060ce16146f HTTP/1.1" 502 150 2025-12-17 23:25:57,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e7575bc060ce16146f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:57,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095df575bc060ce161463 2025-12-17 23:25:57,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095df575bc060ce161463 HTTP/1.1" 502 150 2025-12-17 23:25:57,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095df575bc060ce161463'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:58,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d2575bc060ce16143d 2025-12-17 23:25:58,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d2575bc060ce16143d HTTP/1.1" 502 150 2025-12-17 23:25:58,851 - lava_callback - ERROR - Error processing callback for node 694095d2575bc060ce16143d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d2575bc060ce16143d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d2575bc060ce16143d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d2575bc060ce16143d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:25:58,853 - lava_callback - INFO - Starting async job submit for node 694095e8575bc060ce161472 2025-12-17 23:25:58,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:25:58,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e8575bc060ce161472 HTTP/1.1" 502 150 2025-12-17 23:25:58,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e8575bc060ce161472'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:58,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e8575bc060ce161472 2025-12-17 23:25:58,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e8575bc060ce161472 HTTP/1.1" 502 150 2025-12-17 23:25:58,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e8575bc060ce161472'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:59,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e2575bc060ce161469 2025-12-17 23:25:59,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e2575bc060ce161469 HTTP/1.1" 502 150 2025-12-17 23:25:59,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e2575bc060ce161469'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:59,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e5575bc060ce16146c 2025-12-17 23:25:59,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e5575bc060ce16146c HTTP/1.1" 502 150 2025-12-17 23:25:59,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e5575bc060ce16146c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:59,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095dd575bc060ce161460 2025-12-17 23:25:59,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095dd575bc060ce161460 HTTP/1.1" 502 150 2025-12-17 23:25:59,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095dd575bc060ce161460'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:25:59,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e7575bc060ce16146f 2025-12-17 23:25:59,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e7575bc060ce16146f HTTP/1.1" 502 150 2025-12-17 23:25:59,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e7575bc060ce16146f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:00,355 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d1575bc060ce16143a 2025-12-17 23:26:00,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d1575bc060ce16143a HTTP/1.1" 502 150 2025-12-17 23:26:00,356 - lava_callback - ERROR - Error processing callback for node 694095d1575bc060ce16143a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d1575bc060ce16143a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d1575bc060ce16143a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d1575bc060ce16143a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:00,358 - lava_callback - INFO - Starting async job submit for node 694095ea575bc060ce161475 2025-12-17 23:26:00,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:00,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ea575bc060ce161475 HTTP/1.1" 502 150 2025-12-17 23:26:00,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ea575bc060ce161475'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:00,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ea575bc060ce161475 2025-12-17 23:26:00,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ea575bc060ce161475 HTTP/1.1" 502 150 2025-12-17 23:26:00,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ea575bc060ce161475'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:00,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095db575bc060ce16145a 2025-12-17 23:26:00,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095db575bc060ce16145a HTTP/1.1" 502 150 2025-12-17 23:26:00,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095db575bc060ce16145a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:00,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e8575bc060ce161472 2025-12-17 23:26:00,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e8575bc060ce161472 HTTP/1.1" 502 150 2025-12-17 23:26:00,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e8575bc060ce161472'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:01,303 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d5575bc060ce161443 2025-12-17 23:26:01,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d5575bc060ce161443 HTTP/1.1" 502 150 2025-12-17 23:26:01,305 - lava_callback - ERROR - Error processing callback for node 694095d5575bc060ce161443: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d5575bc060ce161443 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d5575bc060ce161443 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d5575bc060ce161443 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:01,307 - lava_callback - INFO - Starting async job submit for node 694095ec575bc060ce161478 2025-12-17 23:26:01,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:01,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ec575bc060ce161478 HTTP/1.1" 502 150 2025-12-17 23:26:01,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ec575bc060ce161478'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:01,313 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ec575bc060ce161478 2025-12-17 23:26:01,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ec575bc060ce161478 HTTP/1.1" 502 150 2025-12-17 23:26:01,314 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ec575bc060ce161478'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:01,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d3575bc060ce161440 2025-12-17 23:26:01,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d3575bc060ce161440 HTTP/1.1" 502 150 2025-12-17 23:26:01,459 - lava_callback - ERROR - Error processing callback for node 694095d3575bc060ce161440: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d3575bc060ce161440 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d3575bc060ce161440 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d3575bc060ce161440 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:01,460 - lava_callback - INFO - Starting async job submit for node 6940989a575bc060ce161b02 2025-12-17 23:26:01,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:01,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989a575bc060ce161b02 HTTP/1.1" 502 150 2025-12-17 23:26:01,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940989a575bc060ce161b02'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:01,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940989a575bc060ce161b02 2025-12-17 23:26:01,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989a575bc060ce161b02 HTTP/1.1" 502 150 2025-12-17 23:26:01,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940989a575bc060ce161b02'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:01,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d8575bc060ce16144c 2025-12-17 23:26:01,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d8575bc060ce16144c HTTP/1.1" 502 150 2025-12-17 23:26:01,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d8575bc060ce16144c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:02,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ea575bc060ce161475 2025-12-17 23:26:02,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ea575bc060ce161475 HTTP/1.1" 502 150 2025-12-17 23:26:02,367 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ea575bc060ce161475'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34212 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:26:03,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ec575bc060ce161478 2025-12-17 23:26:03,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940989a575bc060ce161b02 2025-12-17 23:26:03,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989a575bc060ce161b02 HTTP/1.1" 502 150 2025-12-17 23:26:03,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940989a575bc060ce161b02'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:03,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e2575bc060ce161469 2025-12-17 23:26:03,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e2575bc060ce161469 HTTP/1.1" 502 150 2025-12-17 23:26:03,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e2575bc060ce161469'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:03,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e5575bc060ce16146c 2025-12-17 23:26:03,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e5575bc060ce16146c HTTP/1.1" 502 150 2025-12-17 23:26:03,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e5575bc060ce16146c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:03,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e7575bc060ce16146f 2025-12-17 23:26:03,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e7575bc060ce16146f HTTP/1.1" 502 150 2025-12-17 23:26:03,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e7575bc060ce16146f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:03,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095dc575bc060ce16145d 2025-12-17 23:26:03,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095dc575bc060ce16145d HTTP/1.1" 502 150 2025-12-17 23:26:03,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095dc575bc060ce16145d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:04,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e8575bc060ce161472 2025-12-17 23:26:04,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e8575bc060ce161472 HTTP/1.1" 502 150 2025-12-17 23:26:04,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e8575bc060ce161472'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:05,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095df575bc060ce161463 2025-12-17 23:26:05,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095df575bc060ce161463 HTTP/1.1" 502 150 2025-12-17 23:26:05,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095df575bc060ce161463'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:06,367 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ea575bc060ce161475 INFO: 172.21.0.1:34218 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:26:07,410 - lava_callback - INFO - Received callback for node 694098fb575bc060ce161bd4 INFO: 172.21.0.1:34226 - "POST /node/694098fb575bc060ce161bd4 HTTP/1.0" 202 Accepted 2025-12-17 23:26:07,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940989a575bc060ce161b02 2025-12-17 23:26:07,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989a575bc060ce161b02 HTTP/1.1" 502 150 2025-12-17 23:26:07,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940989a575bc060ce161b02'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:08,072 - lava_callback - INFO - Received callback for node 694098fd575bc060ce161bdf INFO: 172.21.0.1:34234 - "POST /node/694098fd575bc060ce161bdf HTTP/1.0" 202 Accepted 2025-12-17 23:26:08,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095db575bc060ce16145a 2025-12-17 23:26:08,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095db575bc060ce16145a HTTP/1.1" 502 150 2025-12-17 23:26:08,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095db575bc060ce16145a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:08,735 - lava_callback - INFO - Received callback for node 694098ff575bc060ce161be2 INFO: 172.21.0.1:34246 - "POST /node/694098ff575bc060ce161be2 HTTP/1.0" 202 Accepted 2025-12-17 23:26:09,289 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d6575bc060ce161446 2025-12-17 23:26:09,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d6575bc060ce161446 HTTP/1.1" 502 150 2025-12-17 23:26:09,290 - lava_callback - ERROR - Error processing callback for node 694095d6575bc060ce161446: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d6575bc060ce161446 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d6575bc060ce161446 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d6575bc060ce161446 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:09,292 - lava_callback - INFO - Starting async job submit for node 694095f0575bc060ce161486 2025-12-17 23:26:09,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:09,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f0575bc060ce161486 HTTP/1.1" 502 150 2025-12-17 23:26:09,307 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f0575bc060ce161486'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:09,307 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f0575bc060ce161486 2025-12-17 23:26:09,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f0575bc060ce161486 HTTP/1.1" 502 150 2025-12-17 23:26:09,308 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f0575bc060ce161486'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:09,413 - lava_callback - INFO - Received callback for node 69409901575bc060ce161be9 INFO: 172.21.0.1:34254 - "POST /node/69409901575bc060ce161be9 HTTP/1.0" 202 Accepted 2025-12-17 23:26:09,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d8575bc060ce16144c 2025-12-17 23:26:09,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d8575bc060ce16144c HTTP/1.1" 502 150 2025-12-17 23:26:09,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095d8575bc060ce16144c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:10,091 - lava_callback - INFO - Received callback for node 69409903575bc060ce161bec INFO: 172.21.0.1:43156 - "POST /node/69409903575bc060ce161bec HTTP/1.0" 202 Accepted 2025-12-17 23:26:10,757 - lava_callback - INFO - Received callback for node 69409905575bc060ce161bef INFO: 172.21.0.1:43162 - "POST /node/69409905575bc060ce161bef HTTP/1.0" 202 Accepted 2025-12-17 23:26:11,308 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f0575bc060ce161486 2025-12-17 23:26:11,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f0575bc060ce161486 HTTP/1.1" 502 150 2025-12-17 23:26:11,309 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f0575bc060ce161486'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:11,435 - lava_callback - INFO - Received callback for node 69409907575bc060ce161bf6 INFO: 172.21.0.1:43174 - "POST /node/69409907575bc060ce161bf6 HTTP/1.0" 202 Accepted 2025-12-17 23:26:11,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e2575bc060ce161469 2025-12-17 23:26:11,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e2575bc060ce161469 HTTP/1.1" 502 150 2025-12-17 23:26:11,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e2575bc060ce161469'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:11,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e5575bc060ce16146c 2025-12-17 23:26:11,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e5575bc060ce16146c HTTP/1.1" 502 150 2025-12-17 23:26:11,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e5575bc060ce16146c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:11,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e7575bc060ce16146f 2025-12-17 23:26:11,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e7575bc060ce16146f HTTP/1.1" 502 150 2025-12-17 23:26:11,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e7575bc060ce16146f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:12,085 - lava_callback - INFO - Received callback for node 69409909575bc060ce161bf9 INFO: 172.21.0.1:43178 - "POST /node/69409909575bc060ce161bf9 HTTP/1.0" 202 Accepted 2025-12-17 23:26:12,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095da575bc060ce161451 2025-12-17 23:26:12,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095da575bc060ce161451 HTTP/1.1" 502 150 2025-12-17 23:26:12,427 - lava_callback - ERROR - Error processing callback for node 694095da575bc060ce161451: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095da575bc060ce161451 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095da575bc060ce161451 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095da575bc060ce161451 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:12,430 - lava_callback - INFO - Starting async job submit for node 694095f3575bc060ce16148b 2025-12-17 23:26:12,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:12,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f3575bc060ce16148b HTTP/1.1" 502 150 2025-12-17 23:26:12,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f3575bc060ce16148b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:12,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f3575bc060ce16148b 2025-12-17 23:26:12,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f3575bc060ce16148b HTTP/1.1" 502 150 2025-12-17 23:26:12,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f3575bc060ce16148b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:12,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ca575bc060ce161428 2025-12-17 23:26:12,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ca575bc060ce161428 HTTP/1.1" 502 150 2025-12-17 23:26:12,757 - lava_callback - ERROR - Error processing callback for node 694095ca575bc060ce161428: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ca575bc060ce161428 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ca575bc060ce161428 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ca575bc060ce161428 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:12,760 - lava_callback - INFO - Starting async job submit for node 694095f4575bc060ce16148e 2025-12-17 23:26:12,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:12,764 - lava_callback - INFO - Received callback for node 6940990b575bc060ce161bfc INFO: 172.21.0.1:43184 - "POST /node/6940990b575bc060ce161bfc HTTP/1.0" 202 Accepted 2025-12-17 23:26:12,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f4575bc060ce16148e HTTP/1.1" 502 150 2025-12-17 23:26:12,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f4575bc060ce16148e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:12,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f4575bc060ce16148e 2025-12-17 23:26:12,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f4575bc060ce16148e HTTP/1.1" 502 150 2025-12-17 23:26:12,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f4575bc060ce16148e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:12,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e8575bc060ce161472 2025-12-17 23:26:12,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e8575bc060ce161472 HTTP/1.1" 502 150 2025-12-17 23:26:12,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095e8575bc060ce161472'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:13,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d7575bc060ce161449 2025-12-17 23:26:13,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d7575bc060ce161449 HTTP/1.1" 502 150 2025-12-17 23:26:13,419 - lava_callback - ERROR - Error processing callback for node 694095d7575bc060ce161449: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d7575bc060ce161449 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d7575bc060ce161449 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d7575bc060ce161449 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:13,421 - lava_callback - INFO - Starting async job submit for node 694095f5575bc060ce161491 2025-12-17 23:26:13,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:13,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f5575bc060ce161491 HTTP/1.1" 502 150 2025-12-17 23:26:13,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f5575bc060ce161491'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:13,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f5575bc060ce161491 2025-12-17 23:26:13,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f5575bc060ce161491 HTTP/1.1" 502 150 2025-12-17 23:26:13,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f5575bc060ce161491'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:13,431 - lava_callback - INFO - Received callback for node 6940990d575bc060ce161bff INFO: 172.21.0.1:43186 - "POST /node/6940990d575bc060ce161bff HTTP/1.0" 202 Accepted 2025-12-17 23:26:14,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f3575bc060ce16148b 2025-12-17 23:26:14,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f3575bc060ce16148b HTTP/1.1" 502 150 2025-12-17 23:26:14,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f3575bc060ce16148b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:14,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f4575bc060ce16148e 2025-12-17 23:26:14,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f4575bc060ce16148e HTTP/1.1" 502 150 2025-12-17 23:26:14,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f4575bc060ce16148e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43188 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:26:15,310 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f0575bc060ce161486 2025-12-17 23:26:15,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f0575bc060ce161486 HTTP/1.1" 502 150 2025-12-17 23:26:15,311 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f0575bc060ce161486'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:15,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f5575bc060ce161491 2025-12-17 23:26:15,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f5575bc060ce161491 HTTP/1.1" 502 150 2025-12-17 23:26:15,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f5575bc060ce161491'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:15,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940989a575bc060ce161b02 2025-12-17 23:26:15,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989a575bc060ce161b02 HTTP/1.1" 502 150 2025-12-17 23:26:15,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940989a575bc060ce161b02'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:15,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095dd575bc060ce161460 2025-12-17 23:26:15,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095dd575bc060ce161460 HTTP/1.1" 502 150 2025-12-17 23:26:15,808 - lava_callback - ERROR - Error processing callback for node 694095dd575bc060ce161460: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095dd575bc060ce161460 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095dd575bc060ce161460 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095dd575bc060ce161460 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:15,810 - lava_callback - INFO - Starting async job submit for node 694095f7575bc060ce161497 2025-12-17 23:26:15,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:15,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f7575bc060ce161497 HTTP/1.1" 502 150 2025-12-17 23:26:15,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f7575bc060ce161497'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:15,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f7575bc060ce161497 2025-12-17 23:26:15,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f7575bc060ce161497 HTTP/1.1" 502 150 2025-12-17 23:26:15,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f7575bc060ce161497'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43200 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:26:17,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f7575bc060ce161497 2025-12-17 23:26:17,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f7575bc060ce161497 HTTP/1.1" 502 150 2025-12-17 23:26:17,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f7575bc060ce161497'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:18,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f3575bc060ce16148b 2025-12-17 23:26:18,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f3575bc060ce16148b HTTP/1.1" 502 150 2025-12-17 23:26:18,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f3575bc060ce16148b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:18,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f4575bc060ce16148e 2025-12-17 23:26:18,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f4575bc060ce16148e HTTP/1.1" 502 150 2025-12-17 23:26:18,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f4575bc060ce16148e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:18,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ec575bc060ce161478 HTTP/1.1" 502 150 2025-12-17 23:26:18,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ec575bc060ce161478'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:19,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f5575bc060ce161491 2025-12-17 23:26:19,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f5575bc060ce161491 HTTP/1.1" 502 150 2025-12-17 23:26:19,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f5575bc060ce161491'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:19,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095dc575bc060ce16145d 2025-12-17 23:26:19,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095dc575bc060ce16145d HTTP/1.1" 502 150 2025-12-17 23:26:19,969 - lava_callback - ERROR - Error processing callback for node 694095dc575bc060ce16145d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095dc575bc060ce16145d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095dc575bc060ce16145d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095dc575bc060ce16145d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:19,971 - lava_callback - INFO - Starting async job submit for node 69409563575bc060ce1612fc 2025-12-17 23:26:20,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:20,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409563575bc060ce1612fc HTTP/1.1" 502 150 2025-12-17 23:26:20,057 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409563575bc060ce1612fc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:20,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409563575bc060ce1612fc 2025-12-17 23:26:20,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409563575bc060ce1612fc HTTP/1.1" 502 150 2025-12-17 23:26:20,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409563575bc060ce1612fc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:21,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ea575bc060ce161475 HTTP/1.1" 502 150 2025-12-17 23:26:21,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ea575bc060ce161475'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:21,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f7575bc060ce161497 2025-12-17 23:26:21,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f7575bc060ce161497 HTTP/1.1" 502 150 2025-12-17 23:26:21,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f7575bc060ce161497'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:21,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095df575bc060ce161463 2025-12-17 23:26:21,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095df575bc060ce161463 HTTP/1.1" 502 150 2025-12-17 23:26:21,972 - lava_callback - ERROR - Error processing callback for node 694095df575bc060ce161463: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095df575bc060ce161463 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095df575bc060ce161463 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095df575bc060ce161463 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:21,974 - lava_callback - INFO - Starting async job submit for node 694095ee575bc060ce16147d 2025-12-17 23:26:21,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:22,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409563575bc060ce1612fc 2025-12-17 23:26:22,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409563575bc060ce1612fc HTTP/1.1" 502 150 2025-12-17 23:26:22,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409563575bc060ce1612fc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:22,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ee575bc060ce16147d HTTP/1.1" 502 150 2025-12-17 23:26:22,163 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ee575bc060ce16147d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:22,163 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ee575bc060ce16147d 2025-12-17 23:26:22,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ee575bc060ce16147d HTTP/1.1" 502 150 2025-12-17 23:26:22,164 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ee575bc060ce16147d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:22,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ec575bc060ce161478 2025-12-17 23:26:22,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ec575bc060ce161478 HTTP/1.1" 502 150 2025-12-17 23:26:22,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ec575bc060ce161478'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:23,311 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f0575bc060ce161486 2025-12-17 23:26:23,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f0575bc060ce161486 HTTP/1.1" 502 150 2025-12-17 23:26:23,312 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f0575bc060ce161486'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:24,164 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ee575bc060ce16147d 2025-12-17 23:26:24,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ee575bc060ce16147d HTTP/1.1" 502 150 2025-12-17 23:26:24,165 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ee575bc060ce16147d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:24,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095db575bc060ce16145a 2025-12-17 23:26:24,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095db575bc060ce16145a HTTP/1.1" 502 150 2025-12-17 23:26:24,718 - lava_callback - ERROR - Error processing callback for node 694095db575bc060ce16145a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095db575bc060ce16145a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095db575bc060ce16145a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095db575bc060ce16145a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:24,720 - lava_callback - INFO - Starting async job submit for node 694095fa575bc060ce1614a1 2025-12-17 23:26:24,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:24,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fa575bc060ce1614a1 HTTP/1.1" 502 150 2025-12-17 23:26:24,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095fa575bc060ce1614a1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:24,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095fa575bc060ce1614a1 2025-12-17 23:26:24,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fa575bc060ce1614a1 HTTP/1.1" 502 150 2025-12-17 23:26:24,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095fa575bc060ce1614a1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:25,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095d8575bc060ce16144c 2025-12-17 23:26:25,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095d8575bc060ce16144c HTTP/1.1" 502 150 2025-12-17 23:26:25,511 - lava_callback - ERROR - Error processing callback for node 694095d8575bc060ce16144c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d8575bc060ce16144c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d8575bc060ce16144c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095d8575bc060ce16144c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:25,513 - lava_callback - INFO - Starting async job submit for node 694095fd575bc060ce1614a4 2025-12-17 23:26:25,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:25,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fd575bc060ce1614a4 HTTP/1.1" 502 150 2025-12-17 23:26:25,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095fd575bc060ce1614a4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:25,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095fd575bc060ce1614a4 2025-12-17 23:26:25,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fd575bc060ce1614a4 HTTP/1.1" 502 150 2025-12-17 23:26:25,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095fd575bc060ce1614a4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:26,060 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409563575bc060ce1612fc 2025-12-17 23:26:26,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409563575bc060ce1612fc HTTP/1.1" 502 150 2025-12-17 23:26:26,061 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409563575bc060ce1612fc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:26,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f3575bc060ce16148b 2025-12-17 23:26:26,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f3575bc060ce16148b HTTP/1.1" 502 150 2025-12-17 23:26:26,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f3575bc060ce16148b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42630 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:26:26,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095fa575bc060ce1614a1 2025-12-17 23:26:26,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fa575bc060ce1614a1 HTTP/1.1" 502 150 2025-12-17 23:26:26,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095fa575bc060ce1614a1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:26,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f4575bc060ce16148e 2025-12-17 23:26:26,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f4575bc060ce16148e HTTP/1.1" 502 150 2025-12-17 23:26:26,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f4575bc060ce16148e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:27,416 - lava_callback - INFO - Received callback for node 69409915575bc060ce161c1c INFO: 172.21.0.1:42640 - "POST /node/69409915575bc060ce161c1c HTTP/1.0" 202 Accepted 2025-12-17 23:26:27,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f5575bc060ce161491 2025-12-17 23:26:27,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f5575bc060ce161491 HTTP/1.1" 502 150 2025-12-17 23:26:27,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f5575bc060ce161491'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:27,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095fd575bc060ce1614a4 2025-12-17 23:26:27,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fd575bc060ce1614a4 HTTP/1.1" 502 150 2025-12-17 23:26:27,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095fd575bc060ce1614a4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:27,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e2575bc060ce161469 2025-12-17 23:26:27,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e2575bc060ce161469 HTTP/1.1" 502 150 2025-12-17 23:26:27,728 - lava_callback - ERROR - Error processing callback for node 694095e2575bc060ce161469: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095e2575bc060ce161469 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095e2575bc060ce161469 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095e2575bc060ce161469 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:27,730 - lava_callback - INFO - Starting async job submit for node 694095f9575bc060ce16149b 2025-12-17 23:26:27,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:27,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f9575bc060ce16149b HTTP/1.1" 502 150 2025-12-17 23:26:27,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f9575bc060ce16149b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:27,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f9575bc060ce16149b 2025-12-17 23:26:27,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f9575bc060ce16149b HTTP/1.1" 502 150 2025-12-17 23:26:27,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f9575bc060ce16149b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:27,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e5575bc060ce16146c 2025-12-17 23:26:27,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e5575bc060ce16146c HTTP/1.1" 502 150 2025-12-17 23:26:27,752 - lava_callback - ERROR - Error processing callback for node 694095e5575bc060ce16146c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095e5575bc060ce16146c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095e5575bc060ce16146c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095e5575bc060ce16146c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:27,754 - lava_callback - INFO - Starting async job submit for node 694095ff575bc060ce1614a7 2025-12-17 23:26:27,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:27,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ff575bc060ce1614a7 HTTP/1.1" 502 150 2025-12-17 23:26:27,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ff575bc060ce1614a7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:27,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ff575bc060ce1614a7 2025-12-17 23:26:27,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ff575bc060ce1614a7 HTTP/1.1" 502 150 2025-12-17 23:26:27,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ff575bc060ce1614a7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:27,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e7575bc060ce16146f 2025-12-17 23:26:27,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e7575bc060ce16146f HTTP/1.1" 502 150 2025-12-17 23:26:27,930 - lava_callback - ERROR - Error processing callback for node 694095e7575bc060ce16146f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095e7575bc060ce16146f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095e7575bc060ce16146f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095e7575bc060ce16146f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:27,933 - lava_callback - INFO - Starting async job submit for node 69409601575bc060ce1614aa 2025-12-17 23:26:27,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:27,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409601575bc060ce1614aa HTTP/1.1" 502 150 2025-12-17 23:26:27,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409601575bc060ce1614aa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:27,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409601575bc060ce1614aa 2025-12-17 23:26:27,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409601575bc060ce1614aa HTTP/1.1" 502 150 2025-12-17 23:26:27,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409601575bc060ce1614aa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:28,095 - lava_callback - INFO - Received callback for node 69409917575bc060ce161c1f INFO: 172.21.0.1:42646 - "POST /node/69409917575bc060ce161c1f HTTP/1.0" 202 Accepted 2025-12-17 23:26:28,165 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ee575bc060ce16147d 2025-12-17 23:26:28,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ee575bc060ce16147d HTTP/1.1" 502 150 2025-12-17 23:26:28,167 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ee575bc060ce16147d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:28,227 - lava_callback - INFO - Received callback for node 69409912575bc060ce161c16 INFO: 172.21.0.1:42654 - "POST /node/69409912575bc060ce161c16 HTTP/1.0" 202 Accepted 2025-12-17 23:26:28,810 - lava_callback - INFO - Received callback for node 6941f90cbb27571da24af054 INFO: 172.21.0.1:42666 - "POST /node/6941f90cbb27571da24af054 HTTP/1.0" 202 Accepted 2025-12-17 23:26:28,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095e8575bc060ce161472 2025-12-17 23:26:28,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095e8575bc060ce161472 HTTP/1.1" 502 150 2025-12-17 23:26:28,866 - lava_callback - ERROR - Error processing callback for node 694095e8575bc060ce161472: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095e8575bc060ce161472 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095e8575bc060ce161472 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095e8575bc060ce161472 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:28,867 - lava_callback - INFO - Starting async job submit for node 694095fa575bc060ce16149e 2025-12-17 23:26:28,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:28,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fa575bc060ce16149e HTTP/1.1" 502 150 2025-12-17 23:26:28,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095fa575bc060ce16149e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:28,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095fa575bc060ce16149e 2025-12-17 23:26:28,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fa575bc060ce16149e HTTP/1.1" 502 150 2025-12-17 23:26:28,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095fa575bc060ce16149e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:29,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ea575bc060ce161475 2025-12-17 23:26:29,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f9575bc060ce16149b 2025-12-17 23:26:29,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f9575bc060ce16149b HTTP/1.1" 502 150 2025-12-17 23:26:29,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f9575bc060ce16149b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:29,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ff575bc060ce1614a7 2025-12-17 23:26:29,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ff575bc060ce1614a7 HTTP/1.1" 502 150 2025-12-17 23:26:29,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ff575bc060ce1614a7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:29,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f7575bc060ce161497 2025-12-17 23:26:29,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f7575bc060ce161497 HTTP/1.1" 502 150 2025-12-17 23:26:29,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f7575bc060ce161497'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:29,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409601575bc060ce1614aa 2025-12-17 23:26:29,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409601575bc060ce1614aa HTTP/1.1" 502 150 2025-12-17 23:26:29,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409601575bc060ce1614aa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:30,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095fa575bc060ce1614a1 2025-12-17 23:26:30,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fa575bc060ce1614a1 HTTP/1.1" 502 150 2025-12-17 23:26:30,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095fa575bc060ce1614a1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:30,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ec575bc060ce161478 2025-12-17 23:26:30,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ec575bc060ce161478 HTTP/1.1" 502 150 2025-12-17 23:26:30,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ec575bc060ce161478'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:30,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095fa575bc060ce16149e 2025-12-17 23:26:30,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fa575bc060ce16149e HTTP/1.1" 502 150 2025-12-17 23:26:30,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095fa575bc060ce16149e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:31,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940989a575bc060ce161b02 2025-12-17 23:26:31,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989a575bc060ce161b02 HTTP/1.1" 502 150 2025-12-17 23:26:31,493 - lava_callback - ERROR - Error processing callback for node 6940989a575bc060ce161b02: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940989a575bc060ce161b02 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940989a575bc060ce161b02 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940989a575bc060ce161b02 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:31,495 - lava_callback - INFO - Starting async job submit for node 69409605575bc060ce1614b2 2025-12-17 23:26:31,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:31,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409605575bc060ce1614b2 HTTP/1.1" 502 150 2025-12-17 23:26:31,501 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409605575bc060ce1614b2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:31,501 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409605575bc060ce1614b2 2025-12-17 23:26:31,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409605575bc060ce1614b2 HTTP/1.1" 502 150 2025-12-17 23:26:31,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409605575bc060ce1614b2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:31,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095fd575bc060ce1614a4 2025-12-17 23:26:31,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fd575bc060ce1614a4 HTTP/1.1" 502 150 2025-12-17 23:26:31,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095fd575bc060ce1614a4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:32,286 - lava_callback - INFO - Received callback for node 69409913575bc060ce161c19 INFO: 172.21.0.1:51026 - "POST /node/69409913575bc060ce161c19 HTTP/1.0" 202 Accepted 2025-12-17 23:26:33,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409605575bc060ce1614b2 2025-12-17 23:26:33,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409605575bc060ce1614b2 HTTP/1.1" 502 150 2025-12-17 23:26:33,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409605575bc060ce1614b2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:33,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f9575bc060ce16149b 2025-12-17 23:26:33,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f9575bc060ce16149b HTTP/1.1" 502 150 2025-12-17 23:26:33,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f9575bc060ce16149b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:33,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ff575bc060ce1614a7 2025-12-17 23:26:33,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ff575bc060ce1614a7 HTTP/1.1" 502 150 2025-12-17 23:26:33,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ff575bc060ce1614a7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:33,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409601575bc060ce1614aa 2025-12-17 23:26:33,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409601575bc060ce1614aa HTTP/1.1" 502 150 2025-12-17 23:26:33,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409601575bc060ce1614aa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:34,061 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409563575bc060ce1612fc 2025-12-17 23:26:34,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409563575bc060ce1612fc HTTP/1.1" 502 150 2025-12-17 23:26:34,062 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409563575bc060ce1612fc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:34,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095fa575bc060ce16149e 2025-12-17 23:26:34,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fa575bc060ce16149e HTTP/1.1" 502 150 2025-12-17 23:26:34,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095fa575bc060ce16149e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:36,167 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ee575bc060ce16147d 2025-12-17 23:26:36,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ee575bc060ce16147d HTTP/1.1" 502 150 2025-12-17 23:26:36,168 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ee575bc060ce16147d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51032 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:26:36,874 - lava_callback - INFO - Received callback for node 6940991a575bc060ce161c29 INFO: 172.21.0.1:51048 - "POST /node/6940991a575bc060ce161c29 HTTP/1.0" 202 Accepted 2025-12-17 23:26:37,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409605575bc060ce1614b2 2025-12-17 23:26:37,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409605575bc060ce1614b2 HTTP/1.1" 502 150 2025-12-17 23:26:37,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409605575bc060ce1614b2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:37,524 - lava_callback - INFO - Received callback for node 6940991d575bc060ce161c2c INFO: 172.21.0.1:51056 - "POST /node/6940991d575bc060ce161c2c HTTP/1.0" 202 Accepted 2025-12-17 23:26:38,202 - lava_callback - INFO - Received callback for node 6940991f575bc060ce161c2f INFO: 172.21.0.1:51062 - "POST /node/6940991f575bc060ce161c2f HTTP/1.0" 202 Accepted 2025-12-17 23:26:38,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095fa575bc060ce1614a1 2025-12-17 23:26:38,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fa575bc060ce1614a1 HTTP/1.1" 502 150 2025-12-17 23:26:38,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095fa575bc060ce1614a1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:38,862 - lava_callback - INFO - Received callback for node 69409920575bc060ce161c32 INFO: 172.21.0.1:51078 - "POST /node/69409920575bc060ce161c32 HTTP/1.0" 202 Accepted 2025-12-17 23:26:39,312 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f0575bc060ce161486 2025-12-17 23:26:39,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f0575bc060ce161486 HTTP/1.1" 502 150 2025-12-17 23:26:39,314 - lava_callback - ERROR - Error processing callback for node 694095f0575bc060ce161486: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f0575bc060ce161486 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f0575bc060ce161486 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f0575bc060ce161486 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:39,316 - lava_callback - INFO - Starting async job submit for node 69409603575bc060ce1614af 2025-12-17 23:26:39,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:39,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409603575bc060ce1614af HTTP/1.1" 502 150 2025-12-17 23:26:39,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409603575bc060ce1614af'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:39,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409603575bc060ce1614af 2025-12-17 23:26:39,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409603575bc060ce1614af HTTP/1.1" 502 150 2025-12-17 23:26:39,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409603575bc060ce1614af'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:39,523 - lava_callback - INFO - Received callback for node 69409927575bc060ce161c48 2025-12-17 23:26:39,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095fd575bc060ce1614a4 INFO: 172.21.0.1:51088 - "POST /node/69409927575bc060ce161c48 HTTP/1.0" 202 Accepted 2025-12-17 23:26:39,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fd575bc060ce1614a4 HTTP/1.1" 502 150 2025-12-17 23:26:39,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095fd575bc060ce1614a4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:40,186 - lava_callback - INFO - Received callback for node 6940992b575bc060ce161c54 INFO: 172.21.0.1:51620 - "POST /node/6940992b575bc060ce161c54 HTTP/1.0" 202 Accepted 2025-12-17 23:26:40,910 - lava_callback - INFO - Received callback for node 6941fcdcbb27571da24afbdb INFO: 172.21.0.1:51634 - "POST /node/6941fcdcbb27571da24afbdb HTTP/1.0" 202 Accepted 2025-12-17 23:26:41,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409603575bc060ce1614af 2025-12-17 23:26:41,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409603575bc060ce1614af HTTP/1.1" 502 150 2025-12-17 23:26:41,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409603575bc060ce1614af'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:41,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f9575bc060ce16149b 2025-12-17 23:26:41,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f9575bc060ce16149b HTTP/1.1" 502 150 2025-12-17 23:26:41,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095f9575bc060ce16149b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:41,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ff575bc060ce1614a7 2025-12-17 23:26:41,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ff575bc060ce1614a7 HTTP/1.1" 502 150 2025-12-17 23:26:41,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ff575bc060ce1614a7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:41,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409601575bc060ce1614aa 2025-12-17 23:26:41,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409601575bc060ce1614aa HTTP/1.1" 502 150 2025-12-17 23:26:41,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409601575bc060ce1614aa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:42,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f3575bc060ce16148b 2025-12-17 23:26:42,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f3575bc060ce16148b HTTP/1.1" 502 150 2025-12-17 23:26:42,447 - lava_callback - ERROR - Error processing callback for node 694095f3575bc060ce16148b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f3575bc060ce16148b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f3575bc060ce16148b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f3575bc060ce16148b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:42,449 - lava_callback - INFO - Starting async job submit for node 69409607575bc060ce1614b5 2025-12-17 23:26:42,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:42,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409607575bc060ce1614b5 HTTP/1.1" 502 150 2025-12-17 23:26:42,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409607575bc060ce1614b5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:42,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409607575bc060ce1614b5 2025-12-17 23:26:42,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409607575bc060ce1614b5 HTTP/1.1" 502 150 2025-12-17 23:26:42,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409607575bc060ce1614b5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:42,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f4575bc060ce16148e 2025-12-17 23:26:42,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f4575bc060ce16148e HTTP/1.1" 502 150 2025-12-17 23:26:42,790 - lava_callback - ERROR - Error processing callback for node 694095f4575bc060ce16148e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f4575bc060ce16148e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f4575bc060ce16148e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f4575bc060ce16148e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:42,792 - lava_callback - INFO - Starting async job submit for node 69409609575bc060ce1614b8 2025-12-17 23:26:42,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:42,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409609575bc060ce1614b8 HTTP/1.1" 502 150 2025-12-17 23:26:42,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409609575bc060ce1614b8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:42,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409609575bc060ce1614b8 2025-12-17 23:26:42,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409609575bc060ce1614b8 HTTP/1.1" 502 150 2025-12-17 23:26:42,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409609575bc060ce1614b8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:42,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095fa575bc060ce16149e 2025-12-17 23:26:42,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fa575bc060ce16149e HTTP/1.1" 502 150 2025-12-17 23:26:42,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095fa575bc060ce16149e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:43,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f5575bc060ce161491 2025-12-17 23:26:43,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f5575bc060ce161491 HTTP/1.1" 502 150 2025-12-17 23:26:43,436 - lava_callback - ERROR - Error processing callback for node 694095f5575bc060ce161491: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f5575bc060ce161491 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f5575bc060ce161491 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f5575bc060ce161491 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:43,439 - lava_callback - INFO - Starting async job submit for node 6940960b575bc060ce1614bb 2025-12-17 23:26:43,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:43,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960b575bc060ce1614bb HTTP/1.1" 502 150 2025-12-17 23:26:43,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960b575bc060ce1614bb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:43,447 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960b575bc060ce1614bb 2025-12-17 23:26:43,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960b575bc060ce1614bb HTTP/1.1" 502 150 2025-12-17 23:26:43,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960b575bc060ce1614bb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:44,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409607575bc060ce1614b5 2025-12-17 23:26:44,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409609575bc060ce1614b8 2025-12-17 23:26:44,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409609575bc060ce1614b8 HTTP/1.1" 502 150 2025-12-17 23:26:44,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409609575bc060ce1614b8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:44,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ea575bc060ce161475 HTTP/1.1" 502 150 2025-12-17 23:26:44,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694095ea575bc060ce161475'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:45,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409603575bc060ce1614af 2025-12-17 23:26:45,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409603575bc060ce1614af HTTP/1.1" 502 150 2025-12-17 23:26:45,338 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409603575bc060ce1614af'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:45,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960b575bc060ce1614bb 2025-12-17 23:26:45,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960b575bc060ce1614bb HTTP/1.1" 502 150 2025-12-17 23:26:45,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960b575bc060ce1614bb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:45,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409605575bc060ce1614b2 2025-12-17 23:26:45,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409605575bc060ce1614b2 HTTP/1.1" 502 150 2025-12-17 23:26:45,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409605575bc060ce1614b2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:45,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f7575bc060ce161497 2025-12-17 23:26:45,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f7575bc060ce161497 HTTP/1.1" 502 150 2025-12-17 23:26:45,832 - lava_callback - ERROR - Error processing callback for node 694095f7575bc060ce161497: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f7575bc060ce161497 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f7575bc060ce161497 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f7575bc060ce161497 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:45,834 - lava_callback - INFO - Starting async job submit for node 69409611575bc060ce1614ce 2025-12-17 23:26:45,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:45,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409611575bc060ce1614ce HTTP/1.1" 502 150 2025-12-17 23:26:45,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409611575bc060ce1614ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:45,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409611575bc060ce1614ce 2025-12-17 23:26:45,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409611575bc060ce1614ce HTTP/1.1" 502 150 2025-12-17 23:26:45,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409611575bc060ce1614ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51636 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:26:46,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ec575bc060ce161478 2025-12-17 23:26:46,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ec575bc060ce161478 HTTP/1.1" 502 150 2025-12-17 23:26:46,804 - lava_callback - ERROR - Error processing callback for node 694095ec575bc060ce161478: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ec575bc060ce161478 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ec575bc060ce161478 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ec575bc060ce161478 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:46,806 - lava_callback - INFO - Starting async job submit for node 694098c0575bc060ce161b52 2025-12-17 23:26:46,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:46,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c0575bc060ce161b52 HTTP/1.1" 502 150 2025-12-17 23:26:46,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c0575bc060ce161b52'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:46,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c0575bc060ce161b52 2025-12-17 23:26:46,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c0575bc060ce161b52 HTTP/1.1" 502 150 2025-12-17 23:26:46,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c0575bc060ce161b52'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:47,423 - lava_callback - INFO - Received callback for node 69409922575bc060ce161c35 INFO: 172.21.0.1:51640 - "POST /node/69409922575bc060ce161c35 HTTP/1.0" 202 Accepted 2025-12-17 23:26:47,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409611575bc060ce1614ce 2025-12-17 23:26:47,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409611575bc060ce1614ce HTTP/1.1" 502 150 2025-12-17 23:26:47,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409611575bc060ce1614ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:48,084 - lava_callback - INFO - Received callback for node 69409924575bc060ce161c3e INFO: 172.21.0.1:51642 - "POST /node/69409924575bc060ce161c3e HTTP/1.0" 202 Accepted 2025-12-17 23:26:48,767 - lava_callback - INFO - Received callback for node 69409929575bc060ce161c51 INFO: 172.21.0.1:51658 - "POST /node/69409929575bc060ce161c51 HTTP/1.0" 202 Accepted 2025-12-17 23:26:48,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409609575bc060ce1614b8 2025-12-17 23:26:48,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409609575bc060ce1614b8 HTTP/1.1" 502 150 2025-12-17 23:26:48,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409609575bc060ce1614b8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:48,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c0575bc060ce161b52 2025-12-17 23:26:48,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c0575bc060ce161b52 HTTP/1.1" 502 150 2025-12-17 23:26:48,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c0575bc060ce161b52'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:49,433 - lava_callback - INFO - Received callback for node 69409930575bc060ce161c60 INFO: 172.21.0.1:51660 - "POST /node/69409930575bc060ce161c60 HTTP/1.0" 202 Accepted 2025-12-17 23:26:49,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960b575bc060ce1614bb 2025-12-17 23:26:49,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960b575bc060ce1614bb HTTP/1.1" 502 150 2025-12-17 23:26:49,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960b575bc060ce1614bb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:50,062 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409563575bc060ce1612fc 2025-12-17 23:26:50,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409563575bc060ce1612fc HTTP/1.1" 502 150 2025-12-17 23:26:50,064 - lava_callback - ERROR - Error processing callback for node 69409563575bc060ce1612fc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409563575bc060ce1612fc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409563575bc060ce1612fc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409563575bc060ce1612fc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:50,066 - lava_callback - INFO - Starting async job submit for node 6940960c575bc060ce1614be 2025-12-17 23:26:50,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:50,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960c575bc060ce1614be HTTP/1.1" 502 150 2025-12-17 23:26:50,090 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960c575bc060ce1614be'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:50,090 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960c575bc060ce1614be 2025-12-17 23:26:50,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960c575bc060ce1614be HTTP/1.1" 502 150 2025-12-17 23:26:50,091 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960c575bc060ce1614be'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:50,107 - lava_callback - INFO - Received callback for node 69409933575bc060ce161c63 INFO: 172.21.0.1:40962 - "POST /node/69409933575bc060ce161c63 HTTP/1.0" 202 Accepted 2025-12-17 23:26:50,780 - lava_callback - INFO - Received callback for node 69409935575bc060ce161c66 INFO: 172.21.0.1:40964 - "POST /node/69409935575bc060ce161c66 HTTP/1.0" 202 Accepted 2025-12-17 23:26:51,842 - lava_callback - INFO - Received callback for node 6940992f575bc060ce161c5d INFO: 172.21.0.1:40970 - "POST /node/6940992f575bc060ce161c5d HTTP/1.0" 202 Accepted 2025-12-17 23:26:51,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409611575bc060ce1614ce 2025-12-17 23:26:51,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409611575bc060ce1614ce HTTP/1.1" 502 150 2025-12-17 23:26:51,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409611575bc060ce1614ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:52,091 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960c575bc060ce1614be 2025-12-17 23:26:52,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960c575bc060ce1614be HTTP/1.1" 502 150 2025-12-17 23:26:52,092 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960c575bc060ce1614be'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:52,168 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ee575bc060ce16147d 2025-12-17 23:26:52,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ee575bc060ce16147d HTTP/1.1" 502 150 2025-12-17 23:26:52,169 - lava_callback - ERROR - Error processing callback for node 694095ee575bc060ce16147d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ee575bc060ce16147d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ee575bc060ce16147d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ee575bc060ce16147d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:52,171 - lava_callback - INFO - Starting async job submit for node 6940960e575bc060ce1614c1 2025-12-17 23:26:52,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:52,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960e575bc060ce1614c1 HTTP/1.1" 502 150 2025-12-17 23:26:52,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960e575bc060ce1614c1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:52,178 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960e575bc060ce1614c1 2025-12-17 23:26:52,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960e575bc060ce1614c1 HTTP/1.1" 502 150 2025-12-17 23:26:52,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960e575bc060ce1614c1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:52,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c0575bc060ce161b52 2025-12-17 23:26:52,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c0575bc060ce161b52 HTTP/1.1" 502 150 2025-12-17 23:26:52,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c0575bc060ce161b52'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:53,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409603575bc060ce1614af 2025-12-17 23:26:53,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409603575bc060ce1614af HTTP/1.1" 502 150 2025-12-17 23:26:53,340 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409603575bc060ce1614af'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:54,179 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960e575bc060ce1614c1 2025-12-17 23:26:54,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960e575bc060ce1614c1 HTTP/1.1" 502 150 2025-12-17 23:26:54,180 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960e575bc060ce1614c1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:54,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095fa575bc060ce1614a1 2025-12-17 23:26:54,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fa575bc060ce1614a1 HTTP/1.1" 502 150 2025-12-17 23:26:54,758 - lava_callback - ERROR - Error processing callback for node 694095fa575bc060ce1614a1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095fa575bc060ce1614a1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095fa575bc060ce1614a1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095fa575bc060ce1614a1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:54,760 - lava_callback - INFO - Starting async job submit for node 6940960f575bc060ce1614c4 2025-12-17 23:26:54,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:54,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960f575bc060ce1614c4 HTTP/1.1" 502 150 2025-12-17 23:26:54,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960f575bc060ce1614c4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:54,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960f575bc060ce1614c4 2025-12-17 23:26:54,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960f575bc060ce1614c4 HTTP/1.1" 502 150 2025-12-17 23:26:54,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960f575bc060ce1614c4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:55,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095fd575bc060ce1614a4 2025-12-17 23:26:55,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fd575bc060ce1614a4 HTTP/1.1" 502 150 2025-12-17 23:26:55,535 - lava_callback - ERROR - Error processing callback for node 694095fd575bc060ce1614a4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095fd575bc060ce1614a4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095fd575bc060ce1614a4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095fd575bc060ce1614a4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:55,537 - lava_callback - INFO - Starting async job submit for node 69409610575bc060ce1614cb 2025-12-17 23:26:55,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:55,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409610575bc060ce1614cb HTTP/1.1" 502 150 2025-12-17 23:26:55,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409610575bc060ce1614cb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:55,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409610575bc060ce1614cb 2025-12-17 23:26:55,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409610575bc060ce1614cb HTTP/1.1" 502 150 2025-12-17 23:26:55,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409610575bc060ce1614cb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:56,092 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960c575bc060ce1614be 2025-12-17 23:26:56,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960c575bc060ce1614be HTTP/1.1" 502 150 2025-12-17 23:26:56,094 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960c575bc060ce1614be'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40986 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:26:56,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960f575bc060ce1614c4 2025-12-17 23:26:56,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960f575bc060ce1614c4 HTTP/1.1" 502 150 2025-12-17 23:26:56,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960f575bc060ce1614c4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:56,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409609575bc060ce1614b8 2025-12-17 23:26:56,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409609575bc060ce1614b8 HTTP/1.1" 502 150 2025-12-17 23:26:56,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409609575bc060ce1614b8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:57,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960b575bc060ce1614bb 2025-12-17 23:26:57,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960b575bc060ce1614bb HTTP/1.1" 502 150 2025-12-17 23:26:57,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960b575bc060ce1614bb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:57,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409610575bc060ce1614cb 2025-12-17 23:26:57,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409610575bc060ce1614cb HTTP/1.1" 502 150 2025-12-17 23:26:57,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409610575bc060ce1614cb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:57,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095f9575bc060ce16149b 2025-12-17 23:26:57,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095f9575bc060ce16149b HTTP/1.1" 502 150 2025-12-17 23:26:57,743 - lava_callback - ERROR - Error processing callback for node 694095f9575bc060ce16149b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f9575bc060ce16149b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f9575bc060ce16149b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095f9575bc060ce16149b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:57,745 - lava_callback - INFO - Starting async job submit for node 69409613575bc060ce1614d4 2025-12-17 23:26:57,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:57,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409613575bc060ce1614d4 HTTP/1.1" 502 150 2025-12-17 23:26:57,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409613575bc060ce1614d4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:57,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409613575bc060ce1614d4 2025-12-17 23:26:57,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409613575bc060ce1614d4 HTTP/1.1" 502 150 2025-12-17 23:26:57,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409613575bc060ce1614d4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:57,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ff575bc060ce1614a7 2025-12-17 23:26:57,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ff575bc060ce1614a7 HTTP/1.1" 502 150 2025-12-17 23:26:57,766 - lava_callback - ERROR - Error processing callback for node 694095ff575bc060ce1614a7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ff575bc060ce1614a7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ff575bc060ce1614a7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ff575bc060ce1614a7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:57,767 - lava_callback - INFO - Starting async job submit for node 69409614575bc060ce1614d7 2025-12-17 23:26:57,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:57,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409614575bc060ce1614d7 HTTP/1.1" 502 150 2025-12-17 23:26:57,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409614575bc060ce1614d7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:57,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409614575bc060ce1614d7 2025-12-17 23:26:57,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409614575bc060ce1614d7 HTTP/1.1" 502 150 2025-12-17 23:26:57,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409614575bc060ce1614d7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:57,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409601575bc060ce1614aa 2025-12-17 23:26:57,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409601575bc060ce1614aa HTTP/1.1" 502 150 2025-12-17 23:26:57,947 - lava_callback - ERROR - Error processing callback for node 69409601575bc060ce1614aa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409601575bc060ce1614aa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409601575bc060ce1614aa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409601575bc060ce1614aa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:57,949 - lava_callback - INFO - Starting async job submit for node 69409616575bc060ce1614de 2025-12-17 23:26:57,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:57,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409616575bc060ce1614de HTTP/1.1" 502 150 2025-12-17 23:26:57,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409616575bc060ce1614de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:57,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409616575bc060ce1614de 2025-12-17 23:26:57,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409616575bc060ce1614de HTTP/1.1" 502 150 2025-12-17 23:26:57,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409616575bc060ce1614de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:58,180 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960e575bc060ce1614c1 2025-12-17 23:26:58,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960e575bc060ce1614c1 HTTP/1.1" 502 150 2025-12-17 23:26:58,181 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960e575bc060ce1614c1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:58,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095fa575bc060ce16149e 2025-12-17 23:26:58,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095fa575bc060ce16149e HTTP/1.1" 502 150 2025-12-17 23:26:58,881 - lava_callback - ERROR - Error processing callback for node 694095fa575bc060ce16149e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095fa575bc060ce16149e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095fa575bc060ce16149e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095fa575bc060ce16149e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:26:58,883 - lava_callback - INFO - Starting async job submit for node 69409617575bc060ce1614e4 2025-12-17 23:26:58,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:26:58,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409617575bc060ce1614e4 HTTP/1.1" 502 150 2025-12-17 23:26:58,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409617575bc060ce1614e4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:58,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409617575bc060ce1614e4 2025-12-17 23:26:58,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409617575bc060ce1614e4 HTTP/1.1" 502 150 2025-12-17 23:26:58,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409617575bc060ce1614e4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:59,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409613575bc060ce1614d4 2025-12-17 23:26:59,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409613575bc060ce1614d4 HTTP/1.1" 502 150 2025-12-17 23:26:59,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409613575bc060ce1614d4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:59,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409607575bc060ce1614b5 HTTP/1.1" 502 150 2025-12-17 23:26:59,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409607575bc060ce1614b5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:59,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409614575bc060ce1614d7 2025-12-17 23:26:59,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409614575bc060ce1614d7 HTTP/1.1" 502 150 2025-12-17 23:26:59,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409614575bc060ce1614d7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:59,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409611575bc060ce1614ce 2025-12-17 23:26:59,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409611575bc060ce1614ce HTTP/1.1" 502 150 2025-12-17 23:26:59,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409611575bc060ce1614ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:26:59,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409616575bc060ce1614de 2025-12-17 23:26:59,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409616575bc060ce1614de HTTP/1.1" 502 150 2025-12-17 23:26:59,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409616575bc060ce1614de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:00,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960f575bc060ce1614c4 2025-12-17 23:27:00,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960f575bc060ce1614c4 HTTP/1.1" 502 150 2025-12-17 23:27:00,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960f575bc060ce1614c4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:00,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c0575bc060ce161b52 2025-12-17 23:27:00,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c0575bc060ce161b52 HTTP/1.1" 502 150 2025-12-17 23:27:00,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c0575bc060ce161b52'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:00,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694095ea575bc060ce161475 2025-12-17 23:27:00,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694095ea575bc060ce161475 HTTP/1.1" 502 150 2025-12-17 23:27:00,913 - lava_callback - ERROR - Error processing callback for node 694095ea575bc060ce161475: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ea575bc060ce161475 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ea575bc060ce161475 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694095ea575bc060ce161475 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:00,916 - lava_callback - INFO - Starting async job submit for node 6940961b575bc060ce1614f1 2025-12-17 23:27:00,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:00,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409617575bc060ce1614e4 2025-12-17 23:27:00,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409617575bc060ce1614e4 HTTP/1.1" 502 150 2025-12-17 23:27:00,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409617575bc060ce1614e4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:00,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961b575bc060ce1614f1 HTTP/1.1" 502 150 2025-12-17 23:27:00,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961b575bc060ce1614f1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:00,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961b575bc060ce1614f1 2025-12-17 23:27:00,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961b575bc060ce1614f1 HTTP/1.1" 502 150 2025-12-17 23:27:00,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961b575bc060ce1614f1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:01,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409605575bc060ce1614b2 2025-12-17 23:27:01,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409605575bc060ce1614b2 HTTP/1.1" 502 150 2025-12-17 23:27:01,508 - lava_callback - ERROR - Error processing callback for node 69409605575bc060ce1614b2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409605575bc060ce1614b2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409605575bc060ce1614b2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409605575bc060ce1614b2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:01,511 - lava_callback - INFO - Starting async job submit for node 6940961c575bc060ce1614f4 2025-12-17 23:27:01,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:01,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961c575bc060ce1614f4 HTTP/1.1" 502 150 2025-12-17 23:27:01,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961c575bc060ce1614f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:01,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961c575bc060ce1614f4 2025-12-17 23:27:01,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961c575bc060ce1614f4 HTTP/1.1" 502 150 2025-12-17 23:27:01,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961c575bc060ce1614f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:01,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409610575bc060ce1614cb 2025-12-17 23:27:01,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409610575bc060ce1614cb HTTP/1.1" 502 150 2025-12-17 23:27:01,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409610575bc060ce1614cb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:02,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961b575bc060ce1614f1 2025-12-17 23:27:02,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961b575bc060ce1614f1 HTTP/1.1" 502 150 2025-12-17 23:27:02,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961b575bc060ce1614f1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39794 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:27:03,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961c575bc060ce1614f4 2025-12-17 23:27:03,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961c575bc060ce1614f4 HTTP/1.1" 502 150 2025-12-17 23:27:03,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961c575bc060ce1614f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:03,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409613575bc060ce1614d4 2025-12-17 23:27:03,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409613575bc060ce1614d4 HTTP/1.1" 502 150 2025-12-17 23:27:03,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409613575bc060ce1614d4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:03,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409607575bc060ce1614b5 2025-12-17 23:27:03,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409607575bc060ce1614b5 HTTP/1.1" 502 150 2025-12-17 23:27:03,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409607575bc060ce1614b5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:03,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409614575bc060ce1614d7 2025-12-17 23:27:03,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409614575bc060ce1614d7 HTTP/1.1" 502 150 2025-12-17 23:27:03,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409614575bc060ce1614d7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:03,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409616575bc060ce1614de 2025-12-17 23:27:03,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409616575bc060ce1614de HTTP/1.1" 502 150 2025-12-17 23:27:03,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409616575bc060ce1614de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:04,094 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960c575bc060ce1614be 2025-12-17 23:27:04,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960c575bc060ce1614be HTTP/1.1" 502 150 2025-12-17 23:27:04,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960c575bc060ce1614be'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:04,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409617575bc060ce1614e4 2025-12-17 23:27:04,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409617575bc060ce1614e4 HTTP/1.1" 502 150 2025-12-17 23:27:04,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409617575bc060ce1614e4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:06,181 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960e575bc060ce1614c1 INFO: 172.21.0.1:39804 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:27:06,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961b575bc060ce1614f1 2025-12-17 23:27:06,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961b575bc060ce1614f1 HTTP/1.1" 502 150 2025-12-17 23:27:06,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961b575bc060ce1614f1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:07,471 - lava_callback - INFO - Received callback for node 69409937575bc060ce161c6f INFO: 172.21.0.1:39820 - "POST /node/69409937575bc060ce161c6f HTTP/1.0" 202 Accepted 2025-12-17 23:27:07,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961c575bc060ce1614f4 2025-12-17 23:27:07,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961c575bc060ce1614f4 HTTP/1.1" 502 150 2025-12-17 23:27:07,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961c575bc060ce1614f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:08,165 - lava_callback - INFO - Received callback for node 69409939575bc060ce161c78 INFO: 172.21.0.1:39828 - "POST /node/69409939575bc060ce161c78 HTTP/1.0" 202 Accepted 2025-12-17 23:27:08,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960f575bc060ce1614c4 2025-12-17 23:27:08,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960f575bc060ce1614c4 HTTP/1.1" 502 150 2025-12-17 23:27:08,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960f575bc060ce1614c4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:08,835 - lava_callback - INFO - Received callback for node 6940993b575bc060ce161c7b INFO: 172.21.0.1:39834 - "POST /node/6940993b575bc060ce161c7b HTTP/1.0" 202 Accepted 2025-12-17 23:27:09,340 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409603575bc060ce1614af 2025-12-17 23:27:09,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409603575bc060ce1614af HTTP/1.1" 502 150 2025-12-17 23:27:09,341 - lava_callback - ERROR - Error processing callback for node 69409603575bc060ce1614af: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409603575bc060ce1614af (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409603575bc060ce1614af (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409603575bc060ce1614af (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:09,343 - lava_callback - INFO - Starting async job submit for node 6940961d575bc060ce1614f7 2025-12-17 23:27:09,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:09,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961d575bc060ce1614f7 HTTP/1.1" 502 150 2025-12-17 23:27:09,350 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961d575bc060ce1614f7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:09,350 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961d575bc060ce1614f7 2025-12-17 23:27:09,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961d575bc060ce1614f7 HTTP/1.1" 502 150 2025-12-17 23:27:09,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961d575bc060ce1614f7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:09,523 - lava_callback - INFO - Received callback for node 6940993d575bc060ce161c82 INFO: 172.21.0.1:39850 - "POST /node/6940993d575bc060ce161c82 HTTP/1.0" 202 Accepted 2025-12-17 23:27:09,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409610575bc060ce1614cb 2025-12-17 23:27:09,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409610575bc060ce1614cb HTTP/1.1" 502 150 2025-12-17 23:27:09,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409610575bc060ce1614cb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:10,185 - lava_callback - INFO - Received callback for node 69409940575bc060ce161c88 INFO: 172.21.0.1:50428 - "POST /node/69409940575bc060ce161c88 HTTP/1.0" 202 Accepted 2025-12-17 23:27:10,847 - lava_callback - INFO - Received callback for node 69409942575bc060ce161c91 INFO: 172.21.0.1:50438 - "POST /node/69409942575bc060ce161c91 HTTP/1.0" 202 Accepted 2025-12-17 23:27:11,351 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961d575bc060ce1614f7 2025-12-17 23:27:11,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961d575bc060ce1614f7 HTTP/1.1" 502 150 2025-12-17 23:27:11,352 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961d575bc060ce1614f7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:11,523 - lava_callback - INFO - Received callback for node 69409944575bc060ce161c94 INFO: 172.21.0.1:50444 - "POST /node/69409944575bc060ce161c94 HTTP/1.0" 202 Accepted 2025-12-17 23:27:11,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409613575bc060ce1614d4 2025-12-17 23:27:11,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409613575bc060ce1614d4 HTTP/1.1" 502 150 2025-12-17 23:27:11,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409613575bc060ce1614d4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:11,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409607575bc060ce1614b5 2025-12-17 23:27:11,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409607575bc060ce1614b5 HTTP/1.1" 502 150 2025-12-17 23:27:11,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409607575bc060ce1614b5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:11,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409614575bc060ce1614d7 2025-12-17 23:27:11,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409614575bc060ce1614d7 HTTP/1.1" 502 150 2025-12-17 23:27:11,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409614575bc060ce1614d7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:11,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409616575bc060ce1614de 2025-12-17 23:27:11,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409616575bc060ce1614de HTTP/1.1" 502 150 2025-12-17 23:27:11,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409616575bc060ce1614de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:12,196 - lava_callback - INFO - Received callback for node 69409946575bc060ce161c97 INFO: 172.21.0.1:50452 - "POST /node/69409946575bc060ce161c97 HTTP/1.0" 202 Accepted 2025-12-17 23:27:12,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409609575bc060ce1614b8 2025-12-17 23:27:12,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409609575bc060ce1614b8 HTTP/1.1" 502 150 2025-12-17 23:27:12,829 - lava_callback - ERROR - Error processing callback for node 69409609575bc060ce1614b8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409609575bc060ce1614b8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409609575bc060ce1614b8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409609575bc060ce1614b8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:12,831 - lava_callback - INFO - Starting async job submit for node 6940961e575bc060ce1614fa 2025-12-17 23:27:12,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:12,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961e575bc060ce1614fa HTTP/1.1" 502 150 2025-12-17 23:27:12,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961e575bc060ce1614fa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:12,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961e575bc060ce1614fa 2025-12-17 23:27:12,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961e575bc060ce1614fa HTTP/1.1" 502 150 2025-12-17 23:27:12,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961e575bc060ce1614fa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:12,853 - lava_callback - INFO - Received callback for node 69409947575bc060ce161c9a INFO: 172.21.0.1:50454 - "POST /node/69409947575bc060ce161c9a HTTP/1.0" 202 Accepted 2025-12-17 23:27:12,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409617575bc060ce1614e4 2025-12-17 23:27:12,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409617575bc060ce1614e4 HTTP/1.1" 502 150 2025-12-17 23:27:12,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409617575bc060ce1614e4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:13,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960b575bc060ce1614bb 2025-12-17 23:27:13,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960b575bc060ce1614bb HTTP/1.1" 502 150 2025-12-17 23:27:13,454 - lava_callback - ERROR - Error processing callback for node 6940960b575bc060ce1614bb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940960b575bc060ce1614bb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940960b575bc060ce1614bb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940960b575bc060ce1614bb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:13,455 - lava_callback - INFO - Starting async job submit for node 6940961f575bc060ce1614fd 2025-12-17 23:27:13,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:13,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961f575bc060ce1614fd HTTP/1.1" 502 150 2025-12-17 23:27:13,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961f575bc060ce1614fd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:13,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961f575bc060ce1614fd 2025-12-17 23:27:13,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961f575bc060ce1614fd HTTP/1.1" 502 150 2025-12-17 23:27:13,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961f575bc060ce1614fd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:13,527 - lava_callback - INFO - Received callback for node 6940994a575bc060ce161c9d INFO: 172.21.0.1:50468 - "POST /node/6940994a575bc060ce161c9d HTTP/1.0" 202 Accepted 2025-12-17 23:27:14,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961e575bc060ce1614fa 2025-12-17 23:27:14,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961e575bc060ce1614fa HTTP/1.1" 502 150 2025-12-17 23:27:14,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961e575bc060ce1614fa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:14,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961b575bc060ce1614f1 2025-12-17 23:27:14,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961b575bc060ce1614f1 HTTP/1.1" 502 150 2025-12-17 23:27:14,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961b575bc060ce1614f1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50476 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:27:15,352 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961d575bc060ce1614f7 2025-12-17 23:27:15,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961d575bc060ce1614f7 HTTP/1.1" 502 150 2025-12-17 23:27:15,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961d575bc060ce1614f7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:15,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961f575bc060ce1614fd 2025-12-17 23:27:15,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961f575bc060ce1614fd HTTP/1.1" 502 150 2025-12-17 23:27:15,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961f575bc060ce1614fd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:15,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961c575bc060ce1614f4 2025-12-17 23:27:15,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961c575bc060ce1614f4 HTTP/1.1" 502 150 2025-12-17 23:27:15,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961c575bc060ce1614f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:15,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409611575bc060ce1614ce 2025-12-17 23:27:15,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409611575bc060ce1614ce HTTP/1.1" 502 150 2025-12-17 23:27:15,852 - lava_callback - ERROR - Error processing callback for node 69409611575bc060ce1614ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409611575bc060ce1614ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409611575bc060ce1614ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409611575bc060ce1614ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:15,854 - lava_callback - INFO - Starting async job submit for node 69409621575bc060ce161500 2025-12-17 23:27:15,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:15,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409621575bc060ce161500 HTTP/1.1" 502 150 2025-12-17 23:27:15,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409621575bc060ce161500'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:15,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409621575bc060ce161500 2025-12-17 23:27:15,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409621575bc060ce161500 HTTP/1.1" 502 150 2025-12-17 23:27:15,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409621575bc060ce161500'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50482 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:27:16,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c0575bc060ce161b52 2025-12-17 23:27:16,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c0575bc060ce161b52 HTTP/1.1" 502 150 2025-12-17 23:27:16,834 - lava_callback - ERROR - Error processing callback for node 694098c0575bc060ce161b52: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c0575bc060ce161b52 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c0575bc060ce161b52 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c0575bc060ce161b52 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:16,836 - lava_callback - INFO - Starting async job submit for node 69409622575bc060ce161503 2025-12-17 23:27:16,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:16,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409622575bc060ce161503 HTTP/1.1" 502 150 2025-12-17 23:27:16,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409622575bc060ce161503'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:16,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409622575bc060ce161503 2025-12-17 23:27:16,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409622575bc060ce161503 HTTP/1.1" 502 150 2025-12-17 23:27:16,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409622575bc060ce161503'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:17,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409621575bc060ce161500 2025-12-17 23:27:17,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409621575bc060ce161500 HTTP/1.1" 502 150 2025-12-17 23:27:17,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409621575bc060ce161500'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:18,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961e575bc060ce1614fa 2025-12-17 23:27:18,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961e575bc060ce1614fa HTTP/1.1" 502 150 2025-12-17 23:27:18,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961e575bc060ce1614fa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:18,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409622575bc060ce161503 2025-12-17 23:27:18,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409622575bc060ce161503 HTTP/1.1" 502 150 2025-12-17 23:27:18,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409622575bc060ce161503'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:19,501 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961f575bc060ce1614fd 2025-12-17 23:27:19,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961f575bc060ce1614fd HTTP/1.1" 502 150 2025-12-17 23:27:19,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961f575bc060ce1614fd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:20,095 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960c575bc060ce1614be 2025-12-17 23:27:20,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960c575bc060ce1614be HTTP/1.1" 502 150 2025-12-17 23:27:20,096 - lava_callback - ERROR - Error processing callback for node 6940960c575bc060ce1614be: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940960c575bc060ce1614be (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940960c575bc060ce1614be (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940960c575bc060ce1614be (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:20,098 - lava_callback - INFO - Starting async job submit for node 69409624575bc060ce161506 2025-12-17 23:27:20,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:20,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409624575bc060ce161506 HTTP/1.1" 502 150 2025-12-17 23:27:20,105 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409624575bc060ce161506'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:20,105 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409624575bc060ce161506 2025-12-17 23:27:20,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409624575bc060ce161506 HTTP/1.1" 502 150 2025-12-17 23:27:20,106 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409624575bc060ce161506'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:21,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409621575bc060ce161500 2025-12-17 23:27:21,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409621575bc060ce161500 HTTP/1.1" 502 150 2025-12-17 23:27:21,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409621575bc060ce161500'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:22,106 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409624575bc060ce161506 2025-12-17 23:27:22,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409624575bc060ce161506 HTTP/1.1" 502 150 2025-12-17 23:27:22,107 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409624575bc060ce161506'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:22,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409622575bc060ce161503 2025-12-17 23:27:22,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409622575bc060ce161503 HTTP/1.1" 502 150 2025-12-17 23:27:22,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409622575bc060ce161503'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:23,354 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961d575bc060ce1614f7 2025-12-17 23:27:23,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961d575bc060ce1614f7 HTTP/1.1" 502 150 2025-12-17 23:27:23,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961d575bc060ce1614f7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:24,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960f575bc060ce1614c4 2025-12-17 23:27:24,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960f575bc060ce1614c4 HTTP/1.1" 502 150 2025-12-17 23:27:24,788 - lava_callback - ERROR - Error processing callback for node 6940960f575bc060ce1614c4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940960f575bc060ce1614c4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940960f575bc060ce1614c4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940960f575bc060ce1614c4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:24,790 - lava_callback - INFO - Starting async job submit for node 69409625575bc060ce161509 2025-12-17 23:27:24,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:24,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409625575bc060ce161509 HTTP/1.1" 502 150 2025-12-17 23:27:24,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409625575bc060ce161509'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:24,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409625575bc060ce161509 2025-12-17 23:27:24,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409625575bc060ce161509 HTTP/1.1" 502 150 2025-12-17 23:27:24,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409625575bc060ce161509'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:25,157 - lava_callback - INFO - Received callback for node 69409807575bc060ce1619a2 INFO: 172.21.0.1:46346 - "POST /node/69409807575bc060ce1619a2 HTTP/1.0" 202 Accepted 2025-12-17 23:27:25,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409610575bc060ce1614cb 2025-12-17 23:27:25,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409610575bc060ce1614cb HTTP/1.1" 502 150 2025-12-17 23:27:25,574 - lava_callback - ERROR - Error processing callback for node 69409610575bc060ce1614cb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409610575bc060ce1614cb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409610575bc060ce1614cb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409610575bc060ce1614cb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:25,576 - lava_callback - INFO - Starting async job submit for node 69409626575bc060ce16150c 2025-12-17 23:27:25,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:25,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409626575bc060ce16150c HTTP/1.1" 502 150 2025-12-17 23:27:25,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409626575bc060ce16150c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:25,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409626575bc060ce16150c 2025-12-17 23:27:25,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409626575bc060ce16150c HTTP/1.1" 502 150 2025-12-17 23:27:25,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409626575bc060ce16150c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:26,107 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409624575bc060ce161506 2025-12-17 23:27:26,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409624575bc060ce161506 HTTP/1.1" 502 150 2025-12-17 23:27:26,109 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409624575bc060ce161506'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46362 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:27:26,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409625575bc060ce161509 2025-12-17 23:27:26,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409625575bc060ce161509 HTTP/1.1" 502 150 2025-12-17 23:27:26,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409625575bc060ce161509'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:26,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961e575bc060ce1614fa 2025-12-17 23:27:26,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961e575bc060ce1614fa HTTP/1.1" 502 150 2025-12-17 23:27:26,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961e575bc060ce1614fa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:27,467 - lava_callback - INFO - Received callback for node 6940994c575bc060ce161caa INFO: 172.21.0.1:46368 - "POST /node/6940994c575bc060ce161caa HTTP/1.0" 202 Accepted 2025-12-17 23:27:27,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961f575bc060ce1614fd 2025-12-17 23:27:27,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961f575bc060ce1614fd HTTP/1.1" 502 150 2025-12-17 23:27:27,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940961f575bc060ce1614fd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:27,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409626575bc060ce16150c 2025-12-17 23:27:27,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409626575bc060ce16150c HTTP/1.1" 502 150 2025-12-17 23:27:27,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409626575bc060ce16150c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:27,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409613575bc060ce1614d4 2025-12-17 23:27:27,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409613575bc060ce1614d4 HTTP/1.1" 502 150 2025-12-17 23:27:27,758 - lava_callback - ERROR - Error processing callback for node 69409613575bc060ce1614d4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409613575bc060ce1614d4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409613575bc060ce1614d4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409613575bc060ce1614d4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:27,760 - lava_callback - INFO - Starting async job submit for node 69409627575bc060ce16150f 2025-12-17 23:27:27,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:27,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409607575bc060ce1614b5 2025-12-17 23:27:27,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409607575bc060ce1614b5 HTTP/1.1" 502 150 2025-12-17 23:27:27,763 - lava_callback - ERROR - Error processing callback for node 69409607575bc060ce1614b5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409607575bc060ce1614b5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409607575bc060ce1614b5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409607575bc060ce1614b5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:27,765 - lava_callback - INFO - Starting async job submit for node 69409628575bc060ce161512 2025-12-17 23:27:27,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:27,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409627575bc060ce16150f HTTP/1.1" 502 150 2025-12-17 23:27:27,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409627575bc060ce16150f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:27,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409627575bc060ce16150f 2025-12-17 23:27:27,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409627575bc060ce16150f HTTP/1.1" 502 150 2025-12-17 23:27:27,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409627575bc060ce16150f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:27,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409614575bc060ce1614d7 2025-12-17 23:27:27,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409614575bc060ce1614d7 HTTP/1.1" 502 150 2025-12-17 23:27:27,779 - lava_callback - ERROR - Error processing callback for node 69409614575bc060ce1614d7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409614575bc060ce1614d7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409614575bc060ce1614d7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409614575bc060ce1614d7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:27,781 - lava_callback - INFO - Starting async job submit for node 6940962e575bc060ce161527 2025-12-17 23:27:27,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:27,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962e575bc060ce161527 HTTP/1.1" 502 150 2025-12-17 23:27:27,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940962e575bc060ce161527'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:27,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940962e575bc060ce161527 2025-12-17 23:27:27,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962e575bc060ce161527 HTTP/1.1" 502 150 2025-12-17 23:27:27,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940962e575bc060ce161527'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:27,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409616575bc060ce1614de 2025-12-17 23:27:27,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409616575bc060ce1614de HTTP/1.1" 502 150 2025-12-17 23:27:27,961 - lava_callback - ERROR - Error processing callback for node 69409616575bc060ce1614de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409616575bc060ce1614de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409616575bc060ce1614de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409616575bc060ce1614de (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:27,963 - lava_callback - INFO - Starting async job submit for node 69409630575bc060ce16152f 2025-12-17 23:27:27,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:27,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409630575bc060ce16152f HTTP/1.1" 502 150 2025-12-17 23:27:27,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409630575bc060ce16152f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:27,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409630575bc060ce16152f 2025-12-17 23:27:27,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409630575bc060ce16152f HTTP/1.1" 502 150 2025-12-17 23:27:27,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409630575bc060ce16152f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:28,143 - lava_callback - INFO - Received callback for node 6940994e575bc060ce161cad INFO: 172.21.0.1:46370 - "POST /node/6940994e575bc060ce161cad HTTP/1.0" 202 Accepted 2025-12-17 23:27:28,511 - lava_callback - INFO - Received callback for node 6940995a575bc060ce161cbc INFO: 172.21.0.1:46382 - "POST /node/6940995a575bc060ce161cbc HTTP/1.0" 202 Accepted 2025-12-17 23:27:28,783 - lava_callback - INFO - Received callback for node 69409950575bc060ce161cb0 INFO: 172.21.0.1:46384 - "POST /node/69409950575bc060ce161cb0 HTTP/1.0" 202 Accepted 2025-12-17 23:27:28,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409617575bc060ce1614e4 2025-12-17 23:27:28,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409617575bc060ce1614e4 HTTP/1.1" 502 150 2025-12-17 23:27:28,923 - lava_callback - ERROR - Error processing callback for node 69409617575bc060ce1614e4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409617575bc060ce1614e4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409617575bc060ce1614e4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409617575bc060ce1614e4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:28,925 - lava_callback - INFO - Starting async job submit for node 69409631575bc060ce161532 2025-12-17 23:27:28,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:28,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409631575bc060ce161532 HTTP/1.1" 502 150 2025-12-17 23:27:28,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409631575bc060ce161532'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:28,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409631575bc060ce161532 2025-12-17 23:27:28,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409631575bc060ce161532 HTTP/1.1" 502 150 2025-12-17 23:27:28,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409631575bc060ce161532'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:29,470 - lava_callback - INFO - Received callback for node 69409954575bc060ce161cb3 INFO: 172.21.0.1:46392 - "POST /node/69409954575bc060ce161cb3 HTTP/1.0" 202 Accepted 2025-12-17 23:27:29,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409627575bc060ce16150f 2025-12-17 23:27:29,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409627575bc060ce16150f HTTP/1.1" 502 150 2025-12-17 23:27:29,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409627575bc060ce16150f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:29,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940962e575bc060ce161527 2025-12-17 23:27:29,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962e575bc060ce161527 HTTP/1.1" 502 150 2025-12-17 23:27:29,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940962e575bc060ce161527'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:29,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409621575bc060ce161500 2025-12-17 23:27:29,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409621575bc060ce161500 HTTP/1.1" 502 150 2025-12-17 23:27:29,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409621575bc060ce161500'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:29,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409630575bc060ce16152f 2025-12-17 23:27:29,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409630575bc060ce16152f HTTP/1.1" 502 150 2025-12-17 23:27:29,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409630575bc060ce16152f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:30,144 - lava_callback - INFO - Received callback for node 69409956575bc060ce161cb6 INFO: 172.21.0.1:54936 - "POST /node/69409956575bc060ce161cb6 HTTP/1.0" 202 Accepted 2025-12-17 23:27:30,821 - lava_callback - INFO - Received callback for node 6940995d575bc060ce161cc2 INFO: 172.21.0.1:54952 - "POST /node/6940995d575bc060ce161cc2 HTTP/1.0" 202 Accepted 2025-12-17 23:27:30,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409625575bc060ce161509 2025-12-17 23:27:30,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409625575bc060ce161509 HTTP/1.1" 502 150 2025-12-17 23:27:30,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409625575bc060ce161509'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:30,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409622575bc060ce161503 2025-12-17 23:27:30,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409622575bc060ce161503 HTTP/1.1" 502 150 2025-12-17 23:27:30,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409622575bc060ce161503'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:30,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409631575bc060ce161532 2025-12-17 23:27:30,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409631575bc060ce161532 HTTP/1.1" 502 150 2025-12-17 23:27:30,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409631575bc060ce161532'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:30,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961b575bc060ce1614f1 2025-12-17 23:27:30,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961b575bc060ce1614f1 HTTP/1.1" 502 150 2025-12-17 23:27:30,976 - lava_callback - ERROR - Error processing callback for node 6940961b575bc060ce1614f1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940961b575bc060ce1614f1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940961b575bc060ce1614f1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940961b575bc060ce1614f1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:30,978 - lava_callback - INFO - Starting async job submit for node 69409632575bc060ce161535 2025-12-17 23:27:30,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:30,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409632575bc060ce161535 HTTP/1.1" 502 150 2025-12-17 23:27:30,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409632575bc060ce161535'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:30,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409632575bc060ce161535 2025-12-17 23:27:30,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409632575bc060ce161535 HTTP/1.1" 502 150 2025-12-17 23:27:30,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409632575bc060ce161535'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:31,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961c575bc060ce1614f4 2025-12-17 23:27:31,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961c575bc060ce1614f4 HTTP/1.1" 502 150 2025-12-17 23:27:31,528 - lava_callback - ERROR - Error processing callback for node 6940961c575bc060ce1614f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940961c575bc060ce1614f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940961c575bc060ce1614f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940961c575bc060ce1614f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:31,531 - lava_callback - INFO - Starting async job submit for node 694096a0575bc060ce161630 2025-12-17 23:27:31,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:31,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a0575bc060ce161630 HTTP/1.1" 502 150 2025-12-17 23:27:31,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a0575bc060ce161630'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:31,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a0575bc060ce161630 2025-12-17 23:27:31,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a0575bc060ce161630 HTTP/1.1" 502 150 2025-12-17 23:27:31,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a0575bc060ce161630'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:31,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409626575bc060ce16150c 2025-12-17 23:27:31,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409626575bc060ce16150c HTTP/1.1" 502 150 2025-12-17 23:27:31,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409626575bc060ce16150c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:32,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409632575bc060ce161535 2025-12-17 23:27:32,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409632575bc060ce161535 HTTP/1.1" 502 150 2025-12-17 23:27:32,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409632575bc060ce161535'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:33,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a0575bc060ce161630 2025-12-17 23:27:33,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a0575bc060ce161630 HTTP/1.1" 502 150 2025-12-17 23:27:33,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a0575bc060ce161630'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:33,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409627575bc060ce16150f 2025-12-17 23:27:33,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409627575bc060ce16150f HTTP/1.1" 502 150 2025-12-17 23:27:33,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409627575bc060ce16150f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:33,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940962e575bc060ce161527 2025-12-17 23:27:33,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962e575bc060ce161527 HTTP/1.1" 502 150 2025-12-17 23:27:33,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940962e575bc060ce161527'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:33,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409630575bc060ce16152f 2025-12-17 23:27:33,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409630575bc060ce16152f HTTP/1.1" 502 150 2025-12-17 23:27:33,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409630575bc060ce16152f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:34,109 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409624575bc060ce161506 2025-12-17 23:27:34,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409624575bc060ce161506 HTTP/1.1" 502 150 2025-12-17 23:27:34,110 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409624575bc060ce161506'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:34,605 - lava_callback - INFO - Received callback for node 69409d1b575bc060ce16274b INFO: 172.21.0.1:54958 - "POST /node/69409d1b575bc060ce16274b HTTP/1.0" 202 Accepted 2025-12-17 23:27:34,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409631575bc060ce161532 2025-12-17 23:27:34,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409631575bc060ce161532 HTTP/1.1" 502 150 2025-12-17 23:27:34,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409631575bc060ce161532'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:35,301 - lava_callback - INFO - Received callback for node 69409c27575bc060ce162508 INFO: 172.21.0.1:54966 - "POST /node/69409c27575bc060ce162508 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:54976 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:27:36,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960e575bc060ce1614c1 HTTP/1.1" 502 150 2025-12-17 23:27:36,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940960e575bc060ce1614c1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:36,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409632575bc060ce161535 2025-12-17 23:27:36,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409632575bc060ce161535 HTTP/1.1" 502 150 2025-12-17 23:27:36,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409632575bc060ce161535'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:37,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a0575bc060ce161630 2025-12-17 23:27:37,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a0575bc060ce161630 HTTP/1.1" 502 150 2025-12-17 23:27:37,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a0575bc060ce161630'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:38,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409625575bc060ce161509 2025-12-17 23:27:38,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409625575bc060ce161509 HTTP/1.1" 502 150 2025-12-17 23:27:38,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409625575bc060ce161509'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:39,355 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961d575bc060ce1614f7 2025-12-17 23:27:39,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961d575bc060ce1614f7 HTTP/1.1" 502 150 2025-12-17 23:27:39,357 - lava_callback - ERROR - Error processing callback for node 6940961d575bc060ce1614f7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940961d575bc060ce1614f7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940961d575bc060ce1614f7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940961d575bc060ce1614f7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:39,358 - lava_callback - INFO - Starting async job submit for node 694097fc575bc060ce161985 2025-12-17 23:27:39,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:39,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fc575bc060ce161985 HTTP/1.1" 502 150 2025-12-17 23:27:39,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097fc575bc060ce161985'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:39,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097fc575bc060ce161985 2025-12-17 23:27:39,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fc575bc060ce161985 HTTP/1.1" 502 150 2025-12-17 23:27:39,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097fc575bc060ce161985'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:39,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409626575bc060ce16150c 2025-12-17 23:27:39,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409626575bc060ce16150c HTTP/1.1" 502 150 2025-12-17 23:27:39,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409626575bc060ce16150c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:41,395 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097fc575bc060ce161985 2025-12-17 23:27:41,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fc575bc060ce161985 HTTP/1.1" 502 150 2025-12-17 23:27:41,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097fc575bc060ce161985'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:41,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409627575bc060ce16150f 2025-12-17 23:27:41,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409627575bc060ce16150f HTTP/1.1" 502 150 2025-12-17 23:27:41,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409627575bc060ce16150f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:41,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940962e575bc060ce161527 2025-12-17 23:27:41,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962e575bc060ce161527 HTTP/1.1" 502 150 2025-12-17 23:27:41,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940962e575bc060ce161527'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:41,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409630575bc060ce16152f 2025-12-17 23:27:41,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409630575bc060ce16152f HTTP/1.1" 502 150 2025-12-17 23:27:41,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409630575bc060ce16152f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:42,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961e575bc060ce1614fa 2025-12-17 23:27:42,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961e575bc060ce1614fa HTTP/1.1" 502 150 2025-12-17 23:27:42,844 - lava_callback - ERROR - Error processing callback for node 6940961e575bc060ce1614fa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940961e575bc060ce1614fa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940961e575bc060ce1614fa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940961e575bc060ce1614fa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:42,847 - lava_callback - INFO - Starting async job submit for node 6940962a575bc060ce161519 2025-12-17 23:27:42,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:42,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962a575bc060ce161519 HTTP/1.1" 502 150 2025-12-17 23:27:42,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940962a575bc060ce161519'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:42,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940962a575bc060ce161519 2025-12-17 23:27:42,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962a575bc060ce161519 HTTP/1.1" 502 150 2025-12-17 23:27:42,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940962a575bc060ce161519'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:42,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409631575bc060ce161532 2025-12-17 23:27:42,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409631575bc060ce161532 HTTP/1.1" 502 150 2025-12-17 23:27:42,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409631575bc060ce161532'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:43,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940961f575bc060ce1614fd 2025-12-17 23:27:43,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940961f575bc060ce1614fd HTTP/1.1" 502 150 2025-12-17 23:27:43,504 - lava_callback - ERROR - Error processing callback for node 6940961f575bc060ce1614fd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940961f575bc060ce1614fd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940961f575bc060ce1614fd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940961f575bc060ce1614fd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:43,506 - lava_callback - INFO - Starting async job submit for node 6940962c575bc060ce161524 2025-12-17 23:27:43,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:43,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962c575bc060ce161524 HTTP/1.1" 502 150 2025-12-17 23:27:43,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940962c575bc060ce161524'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:43,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940962c575bc060ce161524 2025-12-17 23:27:43,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962c575bc060ce161524 HTTP/1.1" 502 150 2025-12-17 23:27:43,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940962c575bc060ce161524'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:44,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940962a575bc060ce161519 2025-12-17 23:27:44,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409632575bc060ce161535 2025-12-17 23:27:44,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409632575bc060ce161535 HTTP/1.1" 502 150 2025-12-17 23:27:44,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409632575bc060ce161535'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:45,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097fc575bc060ce161985 2025-12-17 23:27:45,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fc575bc060ce161985 HTTP/1.1" 502 150 2025-12-17 23:27:45,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097fc575bc060ce161985'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:45,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940962c575bc060ce161524 2025-12-17 23:27:45,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962c575bc060ce161524 HTTP/1.1" 502 150 2025-12-17 23:27:45,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940962c575bc060ce161524'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:45,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a0575bc060ce161630 2025-12-17 23:27:45,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a0575bc060ce161630 HTTP/1.1" 502 150 2025-12-17 23:27:45,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a0575bc060ce161630'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:45,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409621575bc060ce161500 2025-12-17 23:27:45,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409621575bc060ce161500 HTTP/1.1" 502 150 2025-12-17 23:27:45,873 - lava_callback - ERROR - Error processing callback for node 69409621575bc060ce161500: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409621575bc060ce161500 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409621575bc060ce161500 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409621575bc060ce161500 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:45,875 - lava_callback - INFO - Starting async job submit for node 6940963b575bc060ce161549 2025-12-17 23:27:45,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:45,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963b575bc060ce161549 HTTP/1.1" 502 150 2025-12-17 23:27:45,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940963b575bc060ce161549'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:45,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940963b575bc060ce161549 2025-12-17 23:27:45,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963b575bc060ce161549 HTTP/1.1" 502 150 2025-12-17 23:27:45,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940963b575bc060ce161549'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37838 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:27:46,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409622575bc060ce161503 2025-12-17 23:27:46,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409622575bc060ce161503 HTTP/1.1" 502 150 2025-12-17 23:27:46,855 - lava_callback - ERROR - Error processing callback for node 69409622575bc060ce161503: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409622575bc060ce161503 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409622575bc060ce161503 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409622575bc060ce161503 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:46,858 - lava_callback - INFO - Starting async job submit for node 6940963d575bc060ce16154e 2025-12-17 23:27:46,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:46,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963d575bc060ce16154e HTTP/1.1" 502 150 2025-12-17 23:27:46,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940963d575bc060ce16154e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:46,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940963d575bc060ce16154e 2025-12-17 23:27:46,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963d575bc060ce16154e HTTP/1.1" 502 150 2025-12-17 23:27:46,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940963d575bc060ce16154e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:47,461 - lava_callback - INFO - Received callback for node 69409960575bc060ce161ccd INFO: 172.21.0.1:37850 - "POST /node/69409960575bc060ce161ccd HTTP/1.0" 202 Accepted 2025-12-17 23:27:47,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940963b575bc060ce161549 2025-12-17 23:27:47,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963b575bc060ce161549 HTTP/1.1" 502 150 2025-12-17 23:27:47,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940963b575bc060ce161549'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:48,126 - lava_callback - INFO - Received callback for node 69409962575bc060ce161cd4 INFO: 172.21.0.1:37854 - "POST /node/69409962575bc060ce161cd4 HTTP/1.0" 202 Accepted 2025-12-17 23:27:48,807 - lava_callback - INFO - Received callback for node 69409964575bc060ce161cd7 INFO: 172.21.0.1:37858 - "POST /node/69409964575bc060ce161cd7 HTTP/1.0" 202 Accepted 2025-12-17 23:27:48,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940963d575bc060ce16154e 2025-12-17 23:27:48,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963d575bc060ce16154e HTTP/1.1" 502 150 2025-12-17 23:27:48,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940963d575bc060ce16154e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:49,488 - lava_callback - INFO - Received callback for node 69409966575bc060ce161cdc INFO: 172.21.0.1:37872 - "POST /node/69409966575bc060ce161cdc HTTP/1.0" 202 Accepted 2025-12-17 23:27:49,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940962c575bc060ce161524 2025-12-17 23:27:49,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962c575bc060ce161524 HTTP/1.1" 502 150 2025-12-17 23:27:49,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940962c575bc060ce161524'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:50,110 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409624575bc060ce161506 2025-12-17 23:27:50,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409624575bc060ce161506 HTTP/1.1" 502 150 2025-12-17 23:27:50,112 - lava_callback - ERROR - Error processing callback for node 69409624575bc060ce161506: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409624575bc060ce161506 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409624575bc060ce161506 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409624575bc060ce161506 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:50,113 - lava_callback - INFO - Starting async job submit for node 6940963a575bc060ce161546 2025-12-17 23:27:50,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:50,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963a575bc060ce161546 HTTP/1.1" 502 150 2025-12-17 23:27:50,140 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940963a575bc060ce161546'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:50,140 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940963a575bc060ce161546 2025-12-17 23:27:50,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963a575bc060ce161546 HTTP/1.1" 502 150 2025-12-17 23:27:50,141 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940963a575bc060ce161546'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:50,163 - lava_callback - INFO - Received callback for node 69409969575bc060ce161ce4 INFO: 172.21.0.1:47296 - "POST /node/69409969575bc060ce161ce4 HTTP/1.0" 202 Accepted 2025-12-17 23:27:50,838 - lava_callback - INFO - Received callback for node 6940996b575bc060ce161ce7 INFO: 172.21.0.1:47298 - "POST /node/6940996b575bc060ce161ce7 HTTP/1.0" 202 Accepted 2025-12-17 23:27:51,528 - lava_callback - INFO - Received callback for node 69409970575bc060ce161ced INFO: 172.21.0.1:47314 - "POST /node/69409970575bc060ce161ced HTTP/1.0" 202 Accepted 2025-12-17 23:27:51,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940963b575bc060ce161549 2025-12-17 23:27:51,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963b575bc060ce161549 HTTP/1.1" 502 150 2025-12-17 23:27:51,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940963b575bc060ce161549'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:52,141 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940963a575bc060ce161546 2025-12-17 23:27:52,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963a575bc060ce161546 HTTP/1.1" 502 150 2025-12-17 23:27:52,143 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940963a575bc060ce161546'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:52,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940963d575bc060ce16154e 2025-12-17 23:27:52,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963d575bc060ce16154e HTTP/1.1" 502 150 2025-12-17 23:27:52,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940963d575bc060ce16154e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:52,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940960e575bc060ce1614c1 2025-12-17 23:27:52,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940960e575bc060ce1614c1 HTTP/1.1" 502 150 2025-12-17 23:27:52,881 - lava_callback - ERROR - Error processing callback for node 6940960e575bc060ce1614c1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940960e575bc060ce1614c1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940960e575bc060ce1614c1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940960e575bc060ce1614c1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:52,883 - lava_callback - INFO - Starting async job submit for node 69409633575bc060ce16153a 2025-12-17 23:27:52,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:52,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409633575bc060ce16153a HTTP/1.1" 502 150 2025-12-17 23:27:52,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409633575bc060ce16153a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:52,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409633575bc060ce16153a 2025-12-17 23:27:52,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409633575bc060ce16153a HTTP/1.1" 502 150 2025-12-17 23:27:52,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409633575bc060ce16153a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:53,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097fc575bc060ce161985 2025-12-17 23:27:53,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fc575bc060ce161985 HTTP/1.1" 502 150 2025-12-17 23:27:53,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097fc575bc060ce161985'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:54,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409625575bc060ce161509 2025-12-17 23:27:54,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409625575bc060ce161509 HTTP/1.1" 502 150 2025-12-17 23:27:54,831 - lava_callback - ERROR - Error processing callback for node 69409625575bc060ce161509: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409625575bc060ce161509 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409625575bc060ce161509 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409625575bc060ce161509 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:54,833 - lava_callback - INFO - Starting async job submit for node 69409635575bc060ce16153d 2025-12-17 23:27:54,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:54,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409635575bc060ce16153d HTTP/1.1" 502 150 2025-12-17 23:27:54,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409635575bc060ce16153d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:54,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409635575bc060ce16153d 2025-12-17 23:27:54,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409635575bc060ce16153d HTTP/1.1" 502 150 2025-12-17 23:27:54,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409635575bc060ce16153d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:54,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409633575bc060ce16153a 2025-12-17 23:27:54,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409633575bc060ce16153a HTTP/1.1" 502 150 2025-12-17 23:27:54,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409633575bc060ce16153a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:55,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409626575bc060ce16150c 2025-12-17 23:27:55,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409626575bc060ce16150c HTTP/1.1" 502 150 2025-12-17 23:27:55,612 - lava_callback - ERROR - Error processing callback for node 69409626575bc060ce16150c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409626575bc060ce16150c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409626575bc060ce16150c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409626575bc060ce16150c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:55,614 - lava_callback - INFO - Starting async job submit for node 69409636575bc060ce161540 2025-12-17 23:27:55,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:55,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409636575bc060ce161540 HTTP/1.1" 502 150 2025-12-17 23:27:55,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409636575bc060ce161540'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:55,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409636575bc060ce161540 2025-12-17 23:27:55,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409636575bc060ce161540 HTTP/1.1" 502 150 2025-12-17 23:27:55,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409636575bc060ce161540'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:56,143 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940963a575bc060ce161546 2025-12-17 23:27:56,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963a575bc060ce161546 HTTP/1.1" 502 150 2025-12-17 23:27:56,144 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940963a575bc060ce161546'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47318 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:27:56,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409635575bc060ce16153d 2025-12-17 23:27:56,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409635575bc060ce16153d HTTP/1.1" 502 150 2025-12-17 23:27:56,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409635575bc060ce16153d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:56,930 - lava_callback - INFO - Received callback for node 6940996d575bc060ce161cea INFO: 172.21.0.1:47328 - "POST /node/6940996d575bc060ce161cea HTTP/1.0" 202 Accepted 2025-12-17 23:27:57,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940962c575bc060ce161524 2025-12-17 23:27:57,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962c575bc060ce161524 HTTP/1.1" 502 150 2025-12-17 23:27:57,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940962c575bc060ce161524'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:57,610 - lava_callback - INFO - Received callback for node 69409975575bc060ce161cfd INFO: 172.21.0.1:47332 - "POST /node/69409975575bc060ce161cfd HTTP/1.0" 202 Accepted 2025-12-17 23:27:57,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409636575bc060ce161540 2025-12-17 23:27:57,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409636575bc060ce161540 HTTP/1.1" 502 150 2025-12-17 23:27:57,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409636575bc060ce161540'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:57,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409627575bc060ce16150f 2025-12-17 23:27:57,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409627575bc060ce16150f HTTP/1.1" 502 150 2025-12-17 23:27:57,778 - lava_callback - ERROR - Error processing callback for node 69409627575bc060ce16150f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409627575bc060ce16150f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409627575bc060ce16150f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409627575bc060ce16150f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:57,780 - lava_callback - INFO - Starting async job submit for node 69409640575bc060ce161559 2025-12-17 23:27:57,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:57,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409640575bc060ce161559 HTTP/1.1" 502 150 2025-12-17 23:27:57,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409640575bc060ce161559'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:57,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409640575bc060ce161559 2025-12-17 23:27:57,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409640575bc060ce161559 HTTP/1.1" 502 150 2025-12-17 23:27:57,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409640575bc060ce161559'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:57,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940962e575bc060ce161527 2025-12-17 23:27:57,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962e575bc060ce161527 HTTP/1.1" 502 150 2025-12-17 23:27:57,799 - lava_callback - ERROR - Error processing callback for node 6940962e575bc060ce161527: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940962e575bc060ce161527 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940962e575bc060ce161527 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940962e575bc060ce161527 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:57,800 - lava_callback - INFO - Starting async job submit for node 69409644575bc060ce161563 2025-12-17 23:27:57,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:57,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409644575bc060ce161563 HTTP/1.1" 502 150 2025-12-17 23:27:57,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409644575bc060ce161563'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:57,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409644575bc060ce161563 2025-12-17 23:27:57,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409644575bc060ce161563 HTTP/1.1" 502 150 2025-12-17 23:27:57,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409644575bc060ce161563'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:57,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409630575bc060ce16152f 2025-12-17 23:27:57,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409630575bc060ce16152f HTTP/1.1" 502 150 2025-12-17 23:27:57,976 - lava_callback - ERROR - Error processing callback for node 69409630575bc060ce16152f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409630575bc060ce16152f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409630575bc060ce16152f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409630575bc060ce16152f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:57,978 - lava_callback - INFO - Starting async job submit for node 69409642575bc060ce161560 2025-12-17 23:27:57,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:57,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409642575bc060ce161560 HTTP/1.1" 502 150 2025-12-17 23:27:57,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409642575bc060ce161560'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:57,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409642575bc060ce161560 2025-12-17 23:27:57,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409642575bc060ce161560 HTTP/1.1" 502 150 2025-12-17 23:27:57,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409642575bc060ce161560'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:58,273 - lava_callback - INFO - Received callback for node 6940997b575bc060ce161d0c INFO: 172.21.0.1:47336 - "POST /node/6940997b575bc060ce161d0c HTTP/1.0" 202 Accepted 2025-12-17 23:27:58,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409628575bc060ce161512 HTTP/1.1" 502 150 2025-12-17 23:27:58,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409628575bc060ce161512'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:58,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409628575bc060ce161512 2025-12-17 23:27:58,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409633575bc060ce16153a 2025-12-17 23:27:58,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409633575bc060ce16153a HTTP/1.1" 502 150 2025-12-17 23:27:58,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409633575bc060ce16153a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:58,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409631575bc060ce161532 2025-12-17 23:27:58,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409631575bc060ce161532 HTTP/1.1" 502 150 2025-12-17 23:27:58,937 - lava_callback - ERROR - Error processing callback for node 69409631575bc060ce161532: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409631575bc060ce161532 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409631575bc060ce161532 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409631575bc060ce161532 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:27:58,939 - lava_callback - INFO - Starting async job submit for node 69409646575bc060ce161566 2025-12-17 23:27:58,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:27:58,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409646575bc060ce161566 HTTP/1.1" 502 150 2025-12-17 23:27:58,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409646575bc060ce161566'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:58,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409646575bc060ce161566 2025-12-17 23:27:58,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409646575bc060ce161566 HTTP/1.1" 502 150 2025-12-17 23:27:58,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409646575bc060ce161566'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:59,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409640575bc060ce161559 2025-12-17 23:27:59,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409640575bc060ce161559 HTTP/1.1" 502 150 2025-12-17 23:27:59,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409640575bc060ce161559'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:59,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409644575bc060ce161563 2025-12-17 23:27:59,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409644575bc060ce161563 HTTP/1.1" 502 150 2025-12-17 23:27:59,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409644575bc060ce161563'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:59,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940963b575bc060ce161549 2025-12-17 23:27:59,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963b575bc060ce161549 HTTP/1.1" 502 150 2025-12-17 23:27:59,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940963b575bc060ce161549'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:27:59,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409642575bc060ce161560 2025-12-17 23:27:59,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409642575bc060ce161560 HTTP/1.1" 502 150 2025-12-17 23:27:59,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409642575bc060ce161560'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:00,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962a575bc060ce161519 HTTP/1.1" 502 150 2025-12-17 23:28:00,176 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940962a575bc060ce161519'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:00,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409635575bc060ce16153d 2025-12-17 23:28:00,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409635575bc060ce16153d HTTP/1.1" 502 150 2025-12-17 23:28:00,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409635575bc060ce16153d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:00,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940963d575bc060ce16154e 2025-12-17 23:28:00,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963d575bc060ce16154e HTTP/1.1" 502 150 2025-12-17 23:28:00,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940963d575bc060ce16154e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:00,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409646575bc060ce161566 2025-12-17 23:28:00,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409646575bc060ce161566 HTTP/1.1" 502 150 2025-12-17 23:28:00,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409646575bc060ce161566'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:00,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409632575bc060ce161535 2025-12-17 23:28:00,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409632575bc060ce161535 HTTP/1.1" 502 150 2025-12-17 23:28:00,991 - lava_callback - ERROR - Error processing callback for node 69409632575bc060ce161535: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409632575bc060ce161535 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409632575bc060ce161535 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409632575bc060ce161535 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:00,993 - lava_callback - INFO - Starting async job submit for node 69411ffa459ff5fc64d2b337 2025-12-17 23:28:01,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:01,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411ffa459ff5fc64d2b337 HTTP/1.1" 502 150 2025-12-17 23:28:01,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69411ffa459ff5fc64d2b337'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:01,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69411ffa459ff5fc64d2b337 2025-12-17 23:28:01,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411ffa459ff5fc64d2b337 HTTP/1.1" 502 150 2025-12-17 23:28:01,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69411ffa459ff5fc64d2b337'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:01,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a0575bc060ce161630 2025-12-17 23:28:01,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a0575bc060ce161630 HTTP/1.1" 502 150 2025-12-17 23:28:01,582 - lava_callback - ERROR - Error processing callback for node 694096a0575bc060ce161630: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a0575bc060ce161630 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a0575bc060ce161630 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a0575bc060ce161630 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:01,584 - lava_callback - INFO - Starting async job submit for node 69409648575bc060ce161569 2025-12-17 23:28:01,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:01,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409648575bc060ce161569 HTTP/1.1" 502 150 2025-12-17 23:28:01,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409648575bc060ce161569'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:01,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409648575bc060ce161569 2025-12-17 23:28:01,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409648575bc060ce161569 HTTP/1.1" 502 150 2025-12-17 23:28:01,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409648575bc060ce161569'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:01,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409636575bc060ce161540 2025-12-17 23:28:01,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409636575bc060ce161540 HTTP/1.1" 502 150 2025-12-17 23:28:01,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409636575bc060ce161540'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:03,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69411ffa459ff5fc64d2b337 2025-12-17 23:28:03,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411ffa459ff5fc64d2b337 HTTP/1.1" 502 150 2025-12-17 23:28:03,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69411ffa459ff5fc64d2b337'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57332 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:28:03,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409648575bc060ce161569 2025-12-17 23:28:03,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409648575bc060ce161569 HTTP/1.1" 502 150 2025-12-17 23:28:03,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409648575bc060ce161569'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:03,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409640575bc060ce161559 2025-12-17 23:28:03,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409640575bc060ce161559 HTTP/1.1" 502 150 2025-12-17 23:28:03,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409640575bc060ce161559'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:03,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409644575bc060ce161563 2025-12-17 23:28:03,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409644575bc060ce161563 HTTP/1.1" 502 150 2025-12-17 23:28:03,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409644575bc060ce161563'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:03,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409642575bc060ce161560 2025-12-17 23:28:03,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409642575bc060ce161560 HTTP/1.1" 502 150 2025-12-17 23:28:03,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409642575bc060ce161560'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:04,144 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940963a575bc060ce161546 2025-12-17 23:28:04,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963a575bc060ce161546 HTTP/1.1" 502 150 2025-12-17 23:28:04,146 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940963a575bc060ce161546'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:04,177 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940962a575bc060ce161519 2025-12-17 23:28:04,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962a575bc060ce161519 HTTP/1.1" 502 150 2025-12-17 23:28:04,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940962a575bc060ce161519'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:04,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409646575bc060ce161566 2025-12-17 23:28:04,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409646575bc060ce161566 HTTP/1.1" 502 150 2025-12-17 23:28:04,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409646575bc060ce161566'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57336 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:28:06,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409633575bc060ce16153a 2025-12-17 23:28:06,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409633575bc060ce16153a HTTP/1.1" 502 150 2025-12-17 23:28:06,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409633575bc060ce16153a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:07,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69411ffa459ff5fc64d2b337 2025-12-17 23:28:07,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411ffa459ff5fc64d2b337 HTTP/1.1" 502 150 2025-12-17 23:28:07,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69411ffa459ff5fc64d2b337'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:07,520 - lava_callback - INFO - Received callback for node 69409973575bc060ce161cf4 INFO: 172.21.0.1:57342 - "POST /node/69409973575bc060ce161cf4 HTTP/1.0" 202 Accepted 2025-12-17 23:28:07,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409648575bc060ce161569 2025-12-17 23:28:07,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409648575bc060ce161569 HTTP/1.1" 502 150 2025-12-17 23:28:07,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409648575bc060ce161569'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:08,189 - lava_callback - INFO - Received callback for node 69409977575bc060ce161d02 INFO: 172.21.0.1:57358 - "POST /node/69409977575bc060ce161d02 HTTP/1.0" 202 Accepted 2025-12-17 23:28:08,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409635575bc060ce16153d 2025-12-17 23:28:08,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409635575bc060ce16153d HTTP/1.1" 502 150 2025-12-17 23:28:08,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409635575bc060ce16153d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:08,874 - lava_callback - INFO - Received callback for node 69409979575bc060ce161d09 INFO: 172.21.0.1:57366 - "POST /node/69409979575bc060ce161d09 HTTP/1.0" 202 Accepted 2025-12-17 23:28:09,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097fc575bc060ce161985 2025-12-17 23:28:09,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fc575bc060ce161985 HTTP/1.1" 502 150 2025-12-17 23:28:09,401 - lava_callback - ERROR - Error processing callback for node 694097fc575bc060ce161985: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097fc575bc060ce161985 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097fc575bc060ce161985 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097fc575bc060ce161985 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:09,403 - lava_callback - INFO - Starting async job submit for node 6940964a575bc060ce16156c 2025-12-17 23:28:09,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:09,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964a575bc060ce16156c HTTP/1.1" 502 150 2025-12-17 23:28:09,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940964a575bc060ce16156c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:09,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940964a575bc060ce16156c 2025-12-17 23:28:09,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964a575bc060ce16156c HTTP/1.1" 502 150 2025-12-17 23:28:09,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940964a575bc060ce16156c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:09,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409636575bc060ce161540 2025-12-17 23:28:09,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409636575bc060ce161540 HTTP/1.1" 502 150 2025-12-17 23:28:09,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409636575bc060ce161540'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:11,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940964a575bc060ce16156c 2025-12-17 23:28:11,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409640575bc060ce161559 2025-12-17 23:28:11,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409640575bc060ce161559 HTTP/1.1" 502 150 2025-12-17 23:28:11,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409640575bc060ce161559'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:11,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409644575bc060ce161563 2025-12-17 23:28:11,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409644575bc060ce161563 HTTP/1.1" 502 150 2025-12-17 23:28:11,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409644575bc060ce161563'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:11,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409642575bc060ce161560 2025-12-17 23:28:11,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409642575bc060ce161560 HTTP/1.1" 502 150 2025-12-17 23:28:11,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409642575bc060ce161560'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:12,178 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940962a575bc060ce161519 2025-12-17 23:28:12,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962a575bc060ce161519 HTTP/1.1" 502 150 2025-12-17 23:28:12,179 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940962a575bc060ce161519'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:12,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409646575bc060ce161566 2025-12-17 23:28:12,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409646575bc060ce161566 HTTP/1.1" 502 150 2025-12-17 23:28:12,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409646575bc060ce161566'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:13,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940962c575bc060ce161524 2025-12-17 23:28:13,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962c575bc060ce161524 HTTP/1.1" 502 150 2025-12-17 23:28:13,542 - lava_callback - ERROR - Error processing callback for node 6940962c575bc060ce161524: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940962c575bc060ce161524 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940962c575bc060ce161524 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940962c575bc060ce161524 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:13,543 - lava_callback - INFO - Starting async job submit for node 6940964e575bc060ce161572 2025-12-17 23:28:13,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:13,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964e575bc060ce161572 HTTP/1.1" 502 150 2025-12-17 23:28:13,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940964e575bc060ce161572'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:13,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940964e575bc060ce161572 2025-12-17 23:28:13,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964e575bc060ce161572 HTTP/1.1" 502 150 2025-12-17 23:28:13,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940964e575bc060ce161572'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:15,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69411ffa459ff5fc64d2b337 2025-12-17 23:28:15,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411ffa459ff5fc64d2b337 HTTP/1.1" 502 150 2025-12-17 23:28:15,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69411ffa459ff5fc64d2b337'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49824 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:28:15,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940964e575bc060ce161572 2025-12-17 23:28:15,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964e575bc060ce161572 HTTP/1.1" 502 150 2025-12-17 23:28:15,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940964e575bc060ce161572'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:15,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409648575bc060ce161569 2025-12-17 23:28:15,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409648575bc060ce161569 HTTP/1.1" 502 150 2025-12-17 23:28:15,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409648575bc060ce161569'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:15,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940963b575bc060ce161549 2025-12-17 23:28:15,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963b575bc060ce161549 HTTP/1.1" 502 150 2025-12-17 23:28:15,911 - lava_callback - ERROR - Error processing callback for node 6940963b575bc060ce161549: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940963b575bc060ce161549 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940963b575bc060ce161549 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940963b575bc060ce161549 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:15,912 - lava_callback - INFO - Starting async job submit for node 69409650575bc060ce161575 2025-12-17 23:28:15,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:15,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409650575bc060ce161575 HTTP/1.1" 502 150 2025-12-17 23:28:15,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409650575bc060ce161575'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:15,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409650575bc060ce161575 2025-12-17 23:28:15,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409650575bc060ce161575 HTTP/1.1" 502 150 2025-12-17 23:28:15,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409650575bc060ce161575'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49836 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:28:16,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940963d575bc060ce16154e 2025-12-17 23:28:16,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963d575bc060ce16154e HTTP/1.1" 502 150 2025-12-17 23:28:16,872 - lava_callback - ERROR - Error processing callback for node 6940963d575bc060ce16154e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940963d575bc060ce16154e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940963d575bc060ce16154e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940963d575bc060ce16154e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:16,874 - lava_callback - INFO - Starting async job submit for node 6940964c575bc060ce16156f 2025-12-17 23:28:16,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:16,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964c575bc060ce16156f HTTP/1.1" 502 150 2025-12-17 23:28:16,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940964c575bc060ce16156f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:16,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940964c575bc060ce16156f 2025-12-17 23:28:16,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964c575bc060ce16156f HTTP/1.1" 502 150 2025-12-17 23:28:16,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940964c575bc060ce16156f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:16,927 - lava_callback - INFO - Received callback for node 6940997d575bc060ce161d17 INFO: 172.21.0.1:49850 - "POST /node/6940997d575bc060ce161d17 HTTP/1.0" 202 Accepted 2025-12-17 23:28:17,577 - lava_callback - INFO - Received callback for node 6940997f575bc060ce161d1c INFO: 172.21.0.1:49852 - "POST /node/6940997f575bc060ce161d1c HTTP/1.0" 202 Accepted 2025-12-17 23:28:17,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409650575bc060ce161575 2025-12-17 23:28:17,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409650575bc060ce161575 HTTP/1.1" 502 150 2025-12-17 23:28:17,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409650575bc060ce161575'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:18,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940964c575bc060ce16156f 2025-12-17 23:28:18,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964c575bc060ce16156f HTTP/1.1" 502 150 2025-12-17 23:28:18,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940964c575bc060ce16156f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:19,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940964e575bc060ce161572 2025-12-17 23:28:19,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964e575bc060ce161572 HTTP/1.1" 502 150 2025-12-17 23:28:19,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940964e575bc060ce161572'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:20,146 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940963a575bc060ce161546 2025-12-17 23:28:20,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940963a575bc060ce161546 HTTP/1.1" 502 150 2025-12-17 23:28:20,156 - lava_callback - ERROR - Error processing callback for node 6940963a575bc060ce161546: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940963a575bc060ce161546 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940963a575bc060ce161546 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940963a575bc060ce161546 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:20,159 - lava_callback - INFO - Starting async job submit for node 69409658575bc060ce16158d 2025-12-17 23:28:20,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:20,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409658575bc060ce16158d HTTP/1.1" 502 150 2025-12-17 23:28:20,190 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409658575bc060ce16158d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:20,190 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409658575bc060ce16158d 2025-12-17 23:28:20,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409658575bc060ce16158d HTTP/1.1" 502 150 2025-12-17 23:28:20,191 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409658575bc060ce16158d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:21,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409650575bc060ce161575 2025-12-17 23:28:21,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409650575bc060ce161575 HTTP/1.1" 502 150 2025-12-17 23:28:21,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409650575bc060ce161575'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:22,192 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409658575bc060ce16158d 2025-12-17 23:28:22,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940964c575bc060ce16156f 2025-12-17 23:28:22,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964c575bc060ce16156f HTTP/1.1" 502 150 2025-12-17 23:28:22,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940964c575bc060ce16156f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:22,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409633575bc060ce16153a 2025-12-17 23:28:22,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409633575bc060ce16153a HTTP/1.1" 502 150 2025-12-17 23:28:22,905 - lava_callback - ERROR - Error processing callback for node 69409633575bc060ce16153a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409633575bc060ce16153a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409633575bc060ce16153a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409633575bc060ce16153a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:22,907 - lava_callback - INFO - Starting async job submit for node 6940965a575bc060ce161593 2025-12-17 23:28:22,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:22,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965a575bc060ce161593 HTTP/1.1" 502 150 2025-12-17 23:28:22,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940965a575bc060ce161593'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:22,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940965a575bc060ce161593 2025-12-17 23:28:22,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965a575bc060ce161593 HTTP/1.1" 502 150 2025-12-17 23:28:22,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940965a575bc060ce161593'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:24,786 - lava_callback - INFO - Received callback for node 69409c3f575bc060ce16254a INFO: 172.21.0.1:42156 - "POST /node/69409c3f575bc060ce16254a HTTP/1.0" 202 Accepted 2025-12-17 23:28:24,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409635575bc060ce16153d 2025-12-17 23:28:24,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409635575bc060ce16153d HTTP/1.1" 502 150 2025-12-17 23:28:24,845 - lava_callback - ERROR - Error processing callback for node 69409635575bc060ce16153d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409635575bc060ce16153d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409635575bc060ce16153d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409635575bc060ce16153d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:24,847 - lava_callback - INFO - Starting async job submit for node 69409652575bc060ce16157c 2025-12-17 23:28:24,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:24,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409652575bc060ce16157c HTTP/1.1" 502 150 2025-12-17 23:28:24,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409652575bc060ce16157c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:24,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409652575bc060ce16157c 2025-12-17 23:28:24,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409652575bc060ce16157c HTTP/1.1" 502 150 2025-12-17 23:28:24,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409652575bc060ce16157c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:24,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940965a575bc060ce161593 2025-12-17 23:28:24,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965a575bc060ce161593 HTTP/1.1" 502 150 2025-12-17 23:28:24,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940965a575bc060ce161593'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:25,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409636575bc060ce161540 2025-12-17 23:28:25,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409636575bc060ce161540 HTTP/1.1" 502 150 2025-12-17 23:28:25,637 - lava_callback - ERROR - Error processing callback for node 69409636575bc060ce161540: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409636575bc060ce161540 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409636575bc060ce161540 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409636575bc060ce161540 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:25,639 - lava_callback - INFO - Starting async job submit for node 69409654575bc060ce161585 2025-12-17 23:28:25,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:25,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409654575bc060ce161585 HTTP/1.1" 502 150 2025-12-17 23:28:25,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409654575bc060ce161585'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:25,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409654575bc060ce161585 2025-12-17 23:28:25,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409654575bc060ce161585 HTTP/1.1" 502 150 2025-12-17 23:28:25,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409654575bc060ce161585'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42158 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:28:26,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964a575bc060ce16156c HTTP/1.1" 502 150 2025-12-17 23:28:26,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940964a575bc060ce16156c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:26,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409652575bc060ce16157c 2025-12-17 23:28:26,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409652575bc060ce16157c HTTP/1.1" 502 150 2025-12-17 23:28:26,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409652575bc060ce16157c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:27,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940964e575bc060ce161572 2025-12-17 23:28:27,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964e575bc060ce161572 HTTP/1.1" 502 150 2025-12-17 23:28:27,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940964e575bc060ce161572'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:27,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409654575bc060ce161585 2025-12-17 23:28:27,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409654575bc060ce161585 HTTP/1.1" 502 150 2025-12-17 23:28:27,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409654575bc060ce161585'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:27,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409640575bc060ce161559 2025-12-17 23:28:27,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409640575bc060ce161559 HTTP/1.1" 502 150 2025-12-17 23:28:27,794 - lava_callback - ERROR - Error processing callback for node 69409640575bc060ce161559: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409640575bc060ce161559 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409640575bc060ce161559 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409640575bc060ce161559 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:27,796 - lava_callback - INFO - Starting async job submit for node 69409656575bc060ce16158a 2025-12-17 23:28:27,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:27,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409644575bc060ce161563 2025-12-17 23:28:27,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409644575bc060ce161563 HTTP/1.1" 502 150 2025-12-17 23:28:27,813 - lava_callback - ERROR - Error processing callback for node 69409644575bc060ce161563: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409644575bc060ce161563 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409644575bc060ce161563 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409644575bc060ce161563 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:27,815 - lava_callback - INFO - Starting async job submit for node 6940965c575bc060ce161596 2025-12-17 23:28:27,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:27,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965c575bc060ce161596 HTTP/1.1" 502 150 2025-12-17 23:28:27,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940965c575bc060ce161596'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:27,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940965c575bc060ce161596 2025-12-17 23:28:27,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965c575bc060ce161596 HTTP/1.1" 502 150 2025-12-17 23:28:27,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940965c575bc060ce161596'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:27,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409656575bc060ce16158a HTTP/1.1" 502 150 2025-12-17 23:28:27,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409656575bc060ce16158a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:27,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409656575bc060ce16158a 2025-12-17 23:28:27,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409656575bc060ce16158a HTTP/1.1" 502 150 2025-12-17 23:28:27,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409656575bc060ce16158a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:27,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409642575bc060ce161560 2025-12-17 23:28:27,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409642575bc060ce161560 HTTP/1.1" 502 150 2025-12-17 23:28:27,992 - lava_callback - ERROR - Error processing callback for node 69409642575bc060ce161560: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409642575bc060ce161560 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409642575bc060ce161560 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409642575bc060ce161560 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:27,995 - lava_callback - INFO - Starting async job submit for node 6940965f575bc060ce161599 2025-12-17 23:28:27,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:28,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965f575bc060ce161599 HTTP/1.1" 502 150 2025-12-17 23:28:28,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940965f575bc060ce161599'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:28,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940965f575bc060ce161599 2025-12-17 23:28:28,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965f575bc060ce161599 HTTP/1.1" 502 150 2025-12-17 23:28:28,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940965f575bc060ce161599'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:28,180 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940962a575bc060ce161519 2025-12-17 23:28:28,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940962a575bc060ce161519 HTTP/1.1" 502 150 2025-12-17 23:28:28,181 - lava_callback - ERROR - Error processing callback for node 6940962a575bc060ce161519: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940962a575bc060ce161519 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940962a575bc060ce161519 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940962a575bc060ce161519 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:28,183 - lava_callback - INFO - Starting async job submit for node 69409661575bc060ce16159c 2025-12-17 23:28:28,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:28,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409661575bc060ce16159c HTTP/1.1" 502 150 2025-12-17 23:28:28,190 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409661575bc060ce16159c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:28,190 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409661575bc060ce16159c 2025-12-17 23:28:28,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409661575bc060ce16159c HTTP/1.1" 502 150 2025-12-17 23:28:28,191 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409661575bc060ce16159c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:28,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940965a575bc060ce161593 2025-12-17 23:28:28,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965a575bc060ce161593 HTTP/1.1" 502 150 2025-12-17 23:28:28,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940965a575bc060ce161593'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:28,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409646575bc060ce161566 2025-12-17 23:28:28,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409646575bc060ce161566 HTTP/1.1" 502 150 2025-12-17 23:28:28,953 - lava_callback - ERROR - Error processing callback for node 69409646575bc060ce161566: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409646575bc060ce161566 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409646575bc060ce161566 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409646575bc060ce161566 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:28,955 - lava_callback - INFO - Starting async job submit for node 69409663575bc060ce16159f 2025-12-17 23:28:28,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:28,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409663575bc060ce16159f HTTP/1.1" 502 150 2025-12-17 23:28:28,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409663575bc060ce16159f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:28,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409663575bc060ce16159f 2025-12-17 23:28:28,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409663575bc060ce16159f HTTP/1.1" 502 150 2025-12-17 23:28:28,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409663575bc060ce16159f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:29,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409628575bc060ce161512 HTTP/1.1" 502 150 2025-12-17 23:28:29,104 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409628575bc060ce161512'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:29,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940965c575bc060ce161596 2025-12-17 23:28:29,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965c575bc060ce161596 HTTP/1.1" 502 150 2025-12-17 23:28:29,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940965c575bc060ce161596'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:29,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409656575bc060ce16158a 2025-12-17 23:28:29,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409656575bc060ce16158a HTTP/1.1" 502 150 2025-12-17 23:28:29,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409656575bc060ce16158a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:29,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409650575bc060ce161575 2025-12-17 23:28:29,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409650575bc060ce161575 HTTP/1.1" 502 150 2025-12-17 23:28:29,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409650575bc060ce161575'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:30,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940965f575bc060ce161599 2025-12-17 23:28:30,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965f575bc060ce161599 HTTP/1.1" 502 150 2025-12-17 23:28:30,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940965f575bc060ce161599'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:30,191 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409661575bc060ce16159c 2025-12-17 23:28:30,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409661575bc060ce16159c HTTP/1.1" 502 150 2025-12-17 23:28:30,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409661575bc060ce16159c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:30,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940964a575bc060ce16156c 2025-12-17 23:28:30,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964a575bc060ce16156c HTTP/1.1" 502 150 2025-12-17 23:28:30,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940964a575bc060ce16156c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:30,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409652575bc060ce16157c 2025-12-17 23:28:30,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409652575bc060ce16157c HTTP/1.1" 502 150 2025-12-17 23:28:30,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409652575bc060ce16157c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:30,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940964c575bc060ce16156f 2025-12-17 23:28:30,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964c575bc060ce16156f HTTP/1.1" 502 150 2025-12-17 23:28:30,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940964c575bc060ce16156f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:30,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409663575bc060ce16159f 2025-12-17 23:28:30,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409663575bc060ce16159f HTTP/1.1" 502 150 2025-12-17 23:28:30,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409663575bc060ce16159f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:31,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69411ffa459ff5fc64d2b337 2025-12-17 23:28:31,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69411ffa459ff5fc64d2b337 HTTP/1.1" 502 150 2025-12-17 23:28:31,026 - lava_callback - ERROR - Error processing callback for node 69411ffa459ff5fc64d2b337: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69411ffa459ff5fc64d2b337 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69411ffa459ff5fc64d2b337 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69411ffa459ff5fc64d2b337 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:31,028 - lava_callback - INFO - Starting async job submit for node 69409665575bc060ce1615a4 2025-12-17 23:28:31,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:31,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409665575bc060ce1615a4 HTTP/1.1" 502 150 2025-12-17 23:28:31,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409665575bc060ce1615a4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:31,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409665575bc060ce1615a4 2025-12-17 23:28:31,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409665575bc060ce1615a4 HTTP/1.1" 502 150 2025-12-17 23:28:31,039 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409665575bc060ce1615a4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:31,105 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409628575bc060ce161512 2025-12-17 23:28:31,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409648575bc060ce161569 2025-12-17 23:28:31,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409648575bc060ce161569 HTTP/1.1" 502 150 2025-12-17 23:28:31,597 - lava_callback - ERROR - Error processing callback for node 69409648575bc060ce161569: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409648575bc060ce161569 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409648575bc060ce161569 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409648575bc060ce161569 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:31,599 - lava_callback - INFO - Starting async job submit for node 6940966e575bc060ce1615c1 2025-12-17 23:28:31,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:31,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940966e575bc060ce1615c1 HTTP/1.1" 502 150 2025-12-17 23:28:31,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940966e575bc060ce1615c1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:31,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940966e575bc060ce1615c1 2025-12-17 23:28:31,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940966e575bc060ce1615c1 HTTP/1.1" 502 150 2025-12-17 23:28:31,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940966e575bc060ce1615c1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:31,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409654575bc060ce161585 2025-12-17 23:28:31,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409654575bc060ce161585 HTTP/1.1" 502 150 2025-12-17 23:28:31,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409654575bc060ce161585'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:33,039 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409665575bc060ce1615a4 2025-12-17 23:28:33,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409665575bc060ce1615a4 HTTP/1.1" 502 150 2025-12-17 23:28:33,040 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409665575bc060ce1615a4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:33,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940966e575bc060ce1615c1 2025-12-17 23:28:33,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940966e575bc060ce1615c1 HTTP/1.1" 502 150 2025-12-17 23:28:33,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940966e575bc060ce1615c1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:33,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940965c575bc060ce161596 2025-12-17 23:28:33,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965c575bc060ce161596 HTTP/1.1" 502 150 2025-12-17 23:28:33,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940965c575bc060ce161596'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:33,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409656575bc060ce16158a 2025-12-17 23:28:33,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409656575bc060ce16158a HTTP/1.1" 502 150 2025-12-17 23:28:33,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409656575bc060ce16158a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:34,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940965f575bc060ce161599 2025-12-17 23:28:34,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965f575bc060ce161599 HTTP/1.1" 502 150 2025-12-17 23:28:34,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940965f575bc060ce161599'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:34,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409661575bc060ce16159c 2025-12-17 23:28:34,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409661575bc060ce16159c HTTP/1.1" 502 150 2025-12-17 23:28:34,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409661575bc060ce16159c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:34,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409663575bc060ce16159f 2025-12-17 23:28:34,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409663575bc060ce16159f HTTP/1.1" 502 150 2025-12-17 23:28:34,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409663575bc060ce16159f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47200 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:28:36,563 - lava_callback - INFO - Received callback for node 69409ca3575bc060ce162641 INFO: 172.21.0.1:47206 - "POST /node/69409ca3575bc060ce162641 HTTP/1.0" 202 Accepted 2025-12-17 23:28:36,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940965a575bc060ce161593 2025-12-17 23:28:36,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965a575bc060ce161593 HTTP/1.1" 502 150 2025-12-17 23:28:36,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940965a575bc060ce161593'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:36,925 - lava_callback - INFO - Received callback for node 69409994575bc060ce161d45 INFO: 172.21.0.1:47220 - "POST /node/69409994575bc060ce161d45 HTTP/1.0" 202 Accepted 2025-12-17 23:28:37,040 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409665575bc060ce1615a4 2025-12-17 23:28:37,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409658575bc060ce16158d HTTP/1.1" 502 150 2025-12-17 23:28:37,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409658575bc060ce16158d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:37,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940966e575bc060ce1615c1 2025-12-17 23:28:37,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940966e575bc060ce1615c1 HTTP/1.1" 502 150 2025-12-17 23:28:37,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940966e575bc060ce1615c1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:37,617 - lava_callback - INFO - Received callback for node 69409996575bc060ce161d48 INFO: 172.21.0.1:47224 - "POST /node/69409996575bc060ce161d48 HTTP/1.0" 202 Accepted 2025-12-17 23:28:38,300 - lava_callback - INFO - Received callback for node 6940998f575bc060ce161d3f INFO: 172.21.0.1:47230 - "POST /node/6940998f575bc060ce161d3f HTTP/1.0" 202 Accepted 2025-12-17 23:28:38,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940964a575bc060ce16156c 2025-12-17 23:28:38,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964a575bc060ce16156c HTTP/1.1" 502 150 2025-12-17 23:28:38,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940964a575bc060ce16156c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:38,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409652575bc060ce16157c 2025-12-17 23:28:38,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409652575bc060ce16157c HTTP/1.1" 502 150 2025-12-17 23:28:38,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409652575bc060ce16157c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:38,949 - lava_callback - INFO - Received callback for node 69409992575bc060ce161d42 INFO: 172.21.0.1:47234 - "POST /node/69409992575bc060ce161d42 HTTP/1.0" 202 Accepted 2025-12-17 23:28:39,617 - lava_callback - INFO - Received callback for node 69409998575bc060ce161d50 INFO: 172.21.0.1:47244 - "POST /node/69409998575bc060ce161d50 HTTP/1.0" 202 Accepted 2025-12-17 23:28:39,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409654575bc060ce161585 2025-12-17 23:28:40,277 - lava_callback - INFO - Received callback for node 6940999a575bc060ce161d57 INFO: 172.21.0.1:35670 - "POST /node/6940999a575bc060ce161d57 HTTP/1.0" 202 Accepted 2025-12-17 23:28:40,932 - lava_callback - INFO - Received callback for node 6940999d575bc060ce161d60 INFO: 172.21.0.1:35672 - "POST /node/6940999d575bc060ce161d60 HTTP/1.0" 202 Accepted 2025-12-17 23:28:41,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409658575bc060ce16158d 2025-12-17 23:28:41,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409658575bc060ce16158d HTTP/1.1" 502 150 2025-12-17 23:28:41,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409658575bc060ce16158d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:41,585 - lava_callback - INFO - Received callback for node 694099a3575bc060ce161d6d INFO: 172.21.0.1:35678 - "POST /node/694099a3575bc060ce161d6d HTTP/1.0" 202 Accepted 2025-12-17 23:28:41,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940965c575bc060ce161596 2025-12-17 23:28:41,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965c575bc060ce161596 HTTP/1.1" 502 150 2025-12-17 23:28:41,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940965c575bc060ce161596'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:41,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409656575bc060ce16158a 2025-12-17 23:28:41,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409656575bc060ce16158a HTTP/1.1" 502 150 2025-12-17 23:28:41,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409656575bc060ce16158a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:42,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940965f575bc060ce161599 2025-12-17 23:28:42,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965f575bc060ce161599 HTTP/1.1" 502 150 2025-12-17 23:28:42,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940965f575bc060ce161599'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:42,194 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409661575bc060ce16159c 2025-12-17 23:28:42,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409661575bc060ce16159c HTTP/1.1" 502 150 2025-12-17 23:28:42,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409661575bc060ce16159c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:42,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409663575bc060ce16159f 2025-12-17 23:28:42,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409663575bc060ce16159f HTTP/1.1" 502 150 2025-12-17 23:28:42,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409663575bc060ce16159f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:43,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940964e575bc060ce161572 2025-12-17 23:28:43,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964e575bc060ce161572 HTTP/1.1" 502 150 2025-12-17 23:28:43,558 - lava_callback - ERROR - Error processing callback for node 6940964e575bc060ce161572: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940964e575bc060ce161572 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940964e575bc060ce161572 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940964e575bc060ce161572 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:43,560 - lava_callback - INFO - Starting async job submit for node 69409667575bc060ce1615ad 2025-12-17 23:28:43,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:43,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409667575bc060ce1615ad HTTP/1.1" 502 150 2025-12-17 23:28:43,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409667575bc060ce1615ad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:43,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409667575bc060ce1615ad 2025-12-17 23:28:43,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409667575bc060ce1615ad HTTP/1.1" 502 150 2025-12-17 23:28:43,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409667575bc060ce1615ad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:44,669 - lava_callback - INFO - Received callback for node 694230b9bb27571da24b7224 INFO: 172.21.0.1:35692 - "POST /node/694230b9bb27571da24b7224 HTTP/1.0" 202 Accepted 2025-12-17 23:28:45,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409667575bc060ce1615ad 2025-12-17 23:28:45,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409667575bc060ce1615ad HTTP/1.1" 502 150 2025-12-17 23:28:45,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409667575bc060ce1615ad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:45,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940966e575bc060ce1615c1 2025-12-17 23:28:45,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940966e575bc060ce1615c1 HTTP/1.1" 502 150 2025-12-17 23:28:45,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940966e575bc060ce1615c1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:45,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409650575bc060ce161575 2025-12-17 23:28:45,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409650575bc060ce161575 HTTP/1.1" 502 150 2025-12-17 23:28:45,927 - lava_callback - ERROR - Error processing callback for node 69409650575bc060ce161575: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409650575bc060ce161575 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409650575bc060ce161575 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409650575bc060ce161575 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:45,929 - lava_callback - INFO - Starting async job submit for node 69409669575bc060ce1615b6 2025-12-17 23:28:45,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:45,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409669575bc060ce1615b6 HTTP/1.1" 502 150 2025-12-17 23:28:45,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409669575bc060ce1615b6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:45,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409669575bc060ce1615b6 2025-12-17 23:28:45,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409669575bc060ce1615b6 HTTP/1.1" 502 150 2025-12-17 23:28:45,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409669575bc060ce1615b6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35702 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:28:46,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940964c575bc060ce16156f 2025-12-17 23:28:46,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964c575bc060ce16156f HTTP/1.1" 502 150 2025-12-17 23:28:46,889 - lava_callback - ERROR - Error processing callback for node 6940964c575bc060ce16156f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940964c575bc060ce16156f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940964c575bc060ce16156f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940964c575bc060ce16156f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:46,891 - lava_callback - INFO - Starting async job submit for node 6940966c575bc060ce1615be 2025-12-17 23:28:46,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:46,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940966c575bc060ce1615be HTTP/1.1" 502 150 2025-12-17 23:28:46,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940966c575bc060ce1615be'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:46,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940966c575bc060ce1615be 2025-12-17 23:28:46,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940966c575bc060ce1615be HTTP/1.1" 502 150 2025-12-17 23:28:46,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940966c575bc060ce1615be'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:47,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409669575bc060ce1615b6 2025-12-17 23:28:47,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409669575bc060ce1615b6 HTTP/1.1" 502 150 2025-12-17 23:28:47,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409669575bc060ce1615b6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:48,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940966c575bc060ce1615be 2025-12-17 23:28:48,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940966c575bc060ce1615be HTTP/1.1" 502 150 2025-12-17 23:28:48,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940966c575bc060ce1615be'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:49,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409658575bc060ce16158d 2025-12-17 23:28:49,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409658575bc060ce16158d HTTP/1.1" 502 150 2025-12-17 23:28:49,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409658575bc060ce16158d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:49,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409667575bc060ce1615ad 2025-12-17 23:28:49,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409667575bc060ce1615ad HTTP/1.1" 502 150 2025-12-17 23:28:49,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409667575bc060ce1615ad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:51,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409669575bc060ce1615b6 2025-12-17 23:28:52,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409665575bc060ce1615a4 HTTP/1.1" 502 150 2025-12-17 23:28:52,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409665575bc060ce1615a4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:52,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940966c575bc060ce1615be 2025-12-17 23:28:52,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940966c575bc060ce1615be HTTP/1.1" 502 150 2025-12-17 23:28:52,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940966c575bc060ce1615be'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:52,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940965a575bc060ce161593 2025-12-17 23:28:52,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965a575bc060ce161593 HTTP/1.1" 502 150 2025-12-17 23:28:52,924 - lava_callback - ERROR - Error processing callback for node 6940965a575bc060ce161593: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940965a575bc060ce161593 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940965a575bc060ce161593 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940965a575bc060ce161593 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:52,926 - lava_callback - INFO - Starting async job submit for node 69409b50575bc060ce16219b 2025-12-17 23:28:52,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:52,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b50575bc060ce16219b HTTP/1.1" 502 150 2025-12-17 23:28:52,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b50575bc060ce16219b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:52,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b50575bc060ce16219b 2025-12-17 23:28:52,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b50575bc060ce16219b HTTP/1.1" 502 150 2025-12-17 23:28:52,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b50575bc060ce16219b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:54,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940964a575bc060ce16156c 2025-12-17 23:28:54,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940964a575bc060ce16156c HTTP/1.1" 502 150 2025-12-17 23:28:54,805 - lava_callback - ERROR - Error processing callback for node 6940964a575bc060ce16156c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940964a575bc060ce16156c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940964a575bc060ce16156c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940964a575bc060ce16156c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:54,809 - lava_callback - INFO - Starting async job submit for node 69409c69575bc060ce1625af 2025-12-17 23:28:54,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409652575bc060ce16157c 2025-12-17 23:28:54,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409652575bc060ce16157c HTTP/1.1" 502 150 2025-12-17 23:28:54,882 - lava_callback - ERROR - Error processing callback for node 69409652575bc060ce16157c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409652575bc060ce16157c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409652575bc060ce16157c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409652575bc060ce16157c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:54,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:54,899 - lava_callback - INFO - Starting async job submit for node 6940967f575bc060ce1615e9 2025-12-17 23:28:54,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:54,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c69575bc060ce1625af HTTP/1.1" 502 150 2025-12-17 23:28:54,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c69575bc060ce1625af'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:54,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c69575bc060ce1625af 2025-12-17 23:28:54,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c69575bc060ce1625af HTTP/1.1" 502 150 2025-12-17 23:28:54,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c69575bc060ce1625af'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:54,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967f575bc060ce1615e9 HTTP/1.1" 502 150 2025-12-17 23:28:54,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940967f575bc060ce1615e9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:54,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940967f575bc060ce1615e9 2025-12-17 23:28:54,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967f575bc060ce1615e9 HTTP/1.1" 502 150 2025-12-17 23:28:54,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940967f575bc060ce1615e9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:54,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409654575bc060ce161585 HTTP/1.1" 502 150 2025-12-17 23:28:54,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409654575bc060ce161585'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:54,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b50575bc060ce16219b 2025-12-17 23:28:54,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b50575bc060ce16219b HTTP/1.1" 502 150 2025-12-17 23:28:54,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b50575bc060ce16219b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33492 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:28:56,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c69575bc060ce1625af 2025-12-17 23:28:56,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c69575bc060ce1625af HTTP/1.1" 502 150 2025-12-17 23:28:56,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c69575bc060ce1625af'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:56,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940967f575bc060ce1615e9 2025-12-17 23:28:56,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967f575bc060ce1615e9 HTTP/1.1" 502 150 2025-12-17 23:28:56,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940967f575bc060ce1615e9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:56,980 - lava_callback - INFO - Received callback for node 694099a5575bc060ce161d74 INFO: 172.21.0.1:33498 - "POST /node/694099a5575bc060ce161d74 HTTP/1.0" 202 Accepted 2025-12-17 23:28:57,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409667575bc060ce1615ad 2025-12-17 23:28:57,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409667575bc060ce1615ad HTTP/1.1" 502 150 2025-12-17 23:28:57,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409667575bc060ce1615ad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:57,643 - lava_callback - INFO - Received callback for node 694099a7575bc060ce161d77 INFO: 172.21.0.1:33504 - "POST /node/694099a7575bc060ce161d77 HTTP/1.0" 202 Accepted 2025-12-17 23:28:57,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940965c575bc060ce161596 2025-12-17 23:28:57,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409656575bc060ce16158a 2025-12-17 23:28:57,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965c575bc060ce161596 HTTP/1.1" 502 150 2025-12-17 23:28:57,829 - lava_callback - ERROR - Error processing callback for node 6940965c575bc060ce161596: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940965c575bc060ce161596 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940965c575bc060ce161596 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940965c575bc060ce161596 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:57,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409656575bc060ce16158a HTTP/1.1" 502 150 2025-12-17 23:28:57,831 - lava_callback - INFO - Starting async job submit for node 69409681575bc060ce1615ec 2025-12-17 23:28:57,831 - lava_callback - ERROR - Error processing callback for node 69409656575bc060ce16158a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409656575bc060ce16158a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409656575bc060ce16158a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409656575bc060ce16158a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:57,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:57,833 - lava_callback - INFO - Starting async job submit for node 69409671575bc060ce1615cb 2025-12-17 23:28:57,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:57,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409671575bc060ce1615cb HTTP/1.1" 502 150 2025-12-17 23:28:57,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409671575bc060ce1615cb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:57,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409671575bc060ce1615cb 2025-12-17 23:28:57,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409671575bc060ce1615cb HTTP/1.1" 502 150 2025-12-17 23:28:57,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409671575bc060ce1615cb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:57,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409681575bc060ce1615ec HTTP/1.1" 502 150 2025-12-17 23:28:57,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409681575bc060ce1615ec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:57,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409681575bc060ce1615ec 2025-12-17 23:28:57,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409681575bc060ce1615ec HTTP/1.1" 502 150 2025-12-17 23:28:57,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409681575bc060ce1615ec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:58,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940965f575bc060ce161599 2025-12-17 23:28:58,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940965f575bc060ce161599 HTTP/1.1" 502 150 2025-12-17 23:28:58,009 - lava_callback - ERROR - Error processing callback for node 6940965f575bc060ce161599: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940965f575bc060ce161599 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940965f575bc060ce161599 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940965f575bc060ce161599 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:58,011 - lava_callback - INFO - Starting async job submit for node 69409673575bc060ce1615ce 2025-12-17 23:28:58,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:58,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409673575bc060ce1615ce HTTP/1.1" 502 150 2025-12-17 23:28:58,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409673575bc060ce1615ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:58,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409673575bc060ce1615ce 2025-12-17 23:28:58,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409673575bc060ce1615ce HTTP/1.1" 502 150 2025-12-17 23:28:58,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409673575bc060ce1615ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:58,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409661575bc060ce16159c 2025-12-17 23:28:58,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409661575bc060ce16159c HTTP/1.1" 502 150 2025-12-17 23:28:58,197 - lava_callback - ERROR - Error processing callback for node 69409661575bc060ce16159c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409661575bc060ce16159c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409661575bc060ce16159c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409661575bc060ce16159c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:58,200 - lava_callback - INFO - Starting async job submit for node 69409675575bc060ce1615d1 2025-12-17 23:28:58,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:58,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409675575bc060ce1615d1 HTTP/1.1" 502 150 2025-12-17 23:28:58,208 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409675575bc060ce1615d1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:58,208 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409675575bc060ce1615d1 2025-12-17 23:28:58,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409675575bc060ce1615d1 HTTP/1.1" 502 150 2025-12-17 23:28:58,209 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409675575bc060ce1615d1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:58,320 - lava_callback - INFO - Received callback for node 694099a9575bc060ce161d7a INFO: 172.21.0.1:33512 - "POST /node/694099a9575bc060ce161d7a HTTP/1.0" 202 Accepted 2025-12-17 23:28:58,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409663575bc060ce16159f 2025-12-17 23:28:58,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409663575bc060ce16159f HTTP/1.1" 502 150 2025-12-17 23:28:58,973 - lava_callback - ERROR - Error processing callback for node 69409663575bc060ce16159f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409663575bc060ce16159f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409663575bc060ce16159f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409663575bc060ce16159f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:28:58,976 - lava_callback - INFO - Starting async job submit for node 69409677575bc060ce1615d4 2025-12-17 23:28:58,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:28:58,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409677575bc060ce1615d4 HTTP/1.1" 502 150 2025-12-17 23:28:58,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409677575bc060ce1615d4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:58,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409677575bc060ce1615d4 2025-12-17 23:28:58,985 - lava_callback - INFO - Received callback for node 694099ab575bc060ce161d7d 2025-12-17 23:28:58,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409677575bc060ce1615d4 HTTP/1.1" 502 150 2025-12-17 23:28:58,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b50575bc060ce16219b INFO: 172.21.0.1:33524 - "POST /node/694099ab575bc060ce161d7d HTTP/1.0" 202 Accepted 2025-12-17 23:28:58,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409677575bc060ce1615d4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:58,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b50575bc060ce16219b HTTP/1.1" 502 150 2025-12-17 23:28:58,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b50575bc060ce16219b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:59,669 - lava_callback - INFO - Received callback for node 694099ad575bc060ce161d80 INFO: 172.21.0.1:33536 - "POST /node/694099ad575bc060ce161d80 HTTP/1.0" 202 Accepted 2025-12-17 23:28:59,851 - lava_callback - INFO - Received callback for node 69423ad6bb27571da24b7ab3 INFO: 172.21.0.1:56504 - "POST /node/69423ad6bb27571da24b7ab3 HTTP/1.0" 202 Accepted 2025-12-17 23:28:59,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409671575bc060ce1615cb 2025-12-17 23:28:59,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409671575bc060ce1615cb HTTP/1.1" 502 150 2025-12-17 23:28:59,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409671575bc060ce1615cb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:28:59,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409681575bc060ce1615ec 2025-12-17 23:28:59,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409681575bc060ce1615ec HTTP/1.1" 502 150 2025-12-17 23:28:59,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409681575bc060ce1615ec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:00,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409673575bc060ce1615ce 2025-12-17 23:29:00,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409673575bc060ce1615ce HTTP/1.1" 502 150 2025-12-17 23:29:00,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409673575bc060ce1615ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:00,209 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409675575bc060ce1615d1 2025-12-17 23:29:00,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409675575bc060ce1615d1 HTTP/1.1" 502 150 2025-12-17 23:29:00,211 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409675575bc060ce1615d1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:00,338 - lava_callback - INFO - Received callback for node 694099af575bc060ce161d8d INFO: 172.21.0.1:56512 - "POST /node/694099af575bc060ce161d8d HTTP/1.0" 202 Accepted 2025-12-17 23:29:00,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409665575bc060ce1615a4 2025-12-17 23:29:00,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409665575bc060ce1615a4 HTTP/1.1" 502 150 2025-12-17 23:29:00,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409665575bc060ce1615a4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:00,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c69575bc060ce1625af 2025-12-17 23:29:00,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c69575bc060ce1625af HTTP/1.1" 502 150 2025-12-17 23:29:00,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c69575bc060ce1625af'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:00,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940966c575bc060ce1615be 2025-12-17 23:29:00,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940966c575bc060ce1615be HTTP/1.1" 502 150 2025-12-17 23:29:00,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940966c575bc060ce1615be'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:00,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940967f575bc060ce1615e9 2025-12-17 23:29:00,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967f575bc060ce1615e9 HTTP/1.1" 502 150 2025-12-17 23:29:00,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940967f575bc060ce1615e9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:00,996 - lava_callback - INFO - Received callback for node 694099b4575bc060ce161d9d 2025-12-17 23:29:01,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409677575bc060ce1615d4 2025-12-17 23:29:01,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409677575bc060ce1615d4 HTTP/1.1" 502 150 2025-12-17 23:29:01,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409677575bc060ce1615d4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56518 - "POST /node/694099b4575bc060ce161d9d HTTP/1.0" 202 Accepted 2025-12-17 23:29:01,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940966e575bc060ce1615c1 2025-12-17 23:29:01,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940966e575bc060ce1615c1 HTTP/1.1" 502 150 2025-12-17 23:29:01,611 - lava_callback - ERROR - Error processing callback for node 6940966e575bc060ce1615c1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940966e575bc060ce1615c1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940966e575bc060ce1615c1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940966e575bc060ce1615c1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:01,613 - lava_callback - INFO - Starting async job submit for node 69409679575bc060ce1615d7 2025-12-17 23:29:01,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:01,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409679575bc060ce1615d7 HTTP/1.1" 502 150 2025-12-17 23:29:01,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409679575bc060ce1615d7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:01,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409679575bc060ce1615d7 2025-12-17 23:29:01,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409679575bc060ce1615d7 HTTP/1.1" 502 150 2025-12-17 23:29:01,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409679575bc060ce1615d7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:01,650 - lava_callback - INFO - Received callback for node 694099b6575bc060ce161da0 INFO: 172.21.0.1:56526 - "POST /node/694099b6575bc060ce161da0 HTTP/1.0" 202 Accepted 2025-12-17 23:29:01,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409628575bc060ce161512 HTTP/1.1" 502 150 2025-12-17 23:29:01,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409628575bc060ce161512'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:02,298 - lava_callback - INFO - Received callback for node 694099b8575bc060ce161da3 INFO: 172.21.0.1:56542 - "POST /node/694099b8575bc060ce161da3 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:56544 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:29:03,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409679575bc060ce1615d7 2025-12-17 23:29:03,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409679575bc060ce1615d7 HTTP/1.1" 502 150 2025-12-17 23:29:03,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409679575bc060ce1615d7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:03,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409671575bc060ce1615cb 2025-12-17 23:29:03,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409671575bc060ce1615cb HTTP/1.1" 502 150 2025-12-17 23:29:03,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409671575bc060ce1615cb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:03,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409681575bc060ce1615ec 2025-12-17 23:29:03,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409681575bc060ce1615ec HTTP/1.1" 502 150 2025-12-17 23:29:03,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409681575bc060ce1615ec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:04,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409673575bc060ce1615ce 2025-12-17 23:29:04,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409673575bc060ce1615ce HTTP/1.1" 502 150 2025-12-17 23:29:04,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409673575bc060ce1615ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:04,212 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409675575bc060ce1615d1 2025-12-17 23:29:04,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409675575bc060ce1615d1 HTTP/1.1" 502 150 2025-12-17 23:29:04,213 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409675575bc060ce1615d1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:05,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409677575bc060ce1615d4 2025-12-17 23:29:05,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409658575bc060ce16158d 2025-12-17 23:29:05,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409658575bc060ce16158d HTTP/1.1" 502 150 2025-12-17 23:29:05,556 - lava_callback - ERROR - Error processing callback for node 69409658575bc060ce16158d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409658575bc060ce16158d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409658575bc060ce16158d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409658575bc060ce16158d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:05,558 - lava_callback - INFO - Starting async job submit for node 6940967b575bc060ce1615de 2025-12-17 23:29:05,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:05,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967b575bc060ce1615de HTTP/1.1" 502 150 2025-12-17 23:29:05,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940967b575bc060ce1615de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:05,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940967b575bc060ce1615de 2025-12-17 23:29:05,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967b575bc060ce1615de HTTP/1.1" 502 150 2025-12-17 23:29:05,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940967b575bc060ce1615de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:05,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409628575bc060ce161512 INFO: 172.21.0.1:56546 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:29:06,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b50575bc060ce16219b 2025-12-17 23:29:06,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b50575bc060ce16219b HTTP/1.1" 502 150 2025-12-17 23:29:06,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b50575bc060ce16219b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:07,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409669575bc060ce1615b6 HTTP/1.1" 502 150 2025-12-17 23:29:07,248 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409669575bc060ce1615b6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:07,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940967b575bc060ce1615de 2025-12-17 23:29:07,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967b575bc060ce1615de HTTP/1.1" 502 150 2025-12-17 23:29:07,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940967b575bc060ce1615de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:07,580 - lava_callback - INFO - Received callback for node 694099c2575bc060ce161db6 INFO: 172.21.0.1:56558 - "POST /node/694099c2575bc060ce161db6 HTTP/1.0" 202 Accepted 2025-12-17 23:29:07,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409679575bc060ce1615d7 2025-12-17 23:29:07,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409679575bc060ce1615d7 HTTP/1.1" 502 150 2025-12-17 23:29:07,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409679575bc060ce1615d7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:08,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c69575bc060ce1625af 2025-12-17 23:29:08,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c69575bc060ce1625af HTTP/1.1" 502 150 2025-12-17 23:29:08,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c69575bc060ce1625af'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:08,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940967f575bc060ce1615e9 2025-12-17 23:29:08,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967f575bc060ce1615e9 HTTP/1.1" 502 150 2025-12-17 23:29:08,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940967f575bc060ce1615e9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:10,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409654575bc060ce161585 2025-12-17 23:29:10,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409654575bc060ce161585 HTTP/1.1" 502 150 2025-12-17 23:29:10,962 - lava_callback - ERROR - Error processing callback for node 69409654575bc060ce161585: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409654575bc060ce161585 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409654575bc060ce161585 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409654575bc060ce161585 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:10,964 - lava_callback - INFO - Starting async job submit for node 6940967d575bc060ce1615e1 2025-12-17 23:29:10,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:10,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967d575bc060ce1615e1 HTTP/1.1" 502 150 2025-12-17 23:29:10,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940967d575bc060ce1615e1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:10,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940967d575bc060ce1615e1 2025-12-17 23:29:10,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967d575bc060ce1615e1 HTTP/1.1" 502 150 2025-12-17 23:29:10,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940967d575bc060ce1615e1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:11,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940967b575bc060ce1615de 2025-12-17 23:29:11,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967b575bc060ce1615de HTTP/1.1" 502 150 2025-12-17 23:29:11,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940967b575bc060ce1615de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:11,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409671575bc060ce1615cb 2025-12-17 23:29:11,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409671575bc060ce1615cb HTTP/1.1" 502 150 2025-12-17 23:29:11,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409671575bc060ce1615cb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:11,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409681575bc060ce1615ec 2025-12-17 23:29:11,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409681575bc060ce1615ec HTTP/1.1" 502 150 2025-12-17 23:29:11,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409681575bc060ce1615ec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:12,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409673575bc060ce1615ce 2025-12-17 23:29:12,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409673575bc060ce1615ce HTTP/1.1" 502 150 2025-12-17 23:29:12,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409673575bc060ce1615ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:12,213 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409675575bc060ce1615d1 2025-12-17 23:29:12,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409675575bc060ce1615d1 HTTP/1.1" 502 150 2025-12-17 23:29:12,214 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409675575bc060ce1615d1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:12,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940967d575bc060ce1615e1 2025-12-17 23:29:12,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967d575bc060ce1615e1 HTTP/1.1" 502 150 2025-12-17 23:29:12,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940967d575bc060ce1615e1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:13,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409667575bc060ce1615ad 2025-12-17 23:29:13,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409667575bc060ce1615ad HTTP/1.1" 502 150 2025-12-17 23:29:13,584 - lava_callback - ERROR - Error processing callback for node 69409667575bc060ce1615ad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409667575bc060ce1615ad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409667575bc060ce1615ad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409667575bc060ce1615ad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:13,586 - lava_callback - INFO - Starting async job submit for node 69409688575bc060ce1615f8 2025-12-17 23:29:13,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:13,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409688575bc060ce1615f8 HTTP/1.1" 502 150 2025-12-17 23:29:13,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409688575bc060ce1615f8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:13,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409688575bc060ce1615f8 2025-12-17 23:29:13,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409688575bc060ce1615f8 HTTP/1.1" 502 150 2025-12-17 23:29:13,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409688575bc060ce1615f8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58654 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:29:15,248 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409669575bc060ce1615b6 2025-12-17 23:29:15,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409669575bc060ce1615b6 HTTP/1.1" 502 150 2025-12-17 23:29:15,250 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409669575bc060ce1615b6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:15,347 - lava_callback - INFO - Received callback for node 69409c7b575bc060ce1625de INFO: 172.21.0.1:58662 - "POST /node/69409c7b575bc060ce1625de HTTP/1.0" 202 Accepted 2025-12-17 23:29:15,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409688575bc060ce1615f8 2025-12-17 23:29:15,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409688575bc060ce1615f8 HTTP/1.1" 502 150 2025-12-17 23:29:15,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409688575bc060ce1615f8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:15,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409679575bc060ce1615d7 2025-12-17 23:29:15,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409679575bc060ce1615d7 HTTP/1.1" 502 150 2025-12-17 23:29:15,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409679575bc060ce1615d7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:16,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409665575bc060ce1615a4 2025-12-17 23:29:16,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409665575bc060ce1615a4 HTTP/1.1" 502 150 2025-12-17 23:29:16,404 - lava_callback - ERROR - Error processing callback for node 69409665575bc060ce1615a4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409665575bc060ce1615a4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409665575bc060ce1615a4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409665575bc060ce1615a4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:16,405 - lava_callback - INFO - Starting async job submit for node 6940968b575bc060ce1615fe 2025-12-17 23:29:16,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:16,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968b575bc060ce1615fe HTTP/1.1" 502 150 2025-12-17 23:29:16,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968b575bc060ce1615fe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:16,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968b575bc060ce1615fe 2025-12-17 23:29:16,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968b575bc060ce1615fe HTTP/1.1" 502 150 2025-12-17 23:29:16,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968b575bc060ce1615fe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58672 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:29:16,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940966c575bc060ce1615be 2025-12-17 23:29:16,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940966c575bc060ce1615be HTTP/1.1" 502 150 2025-12-17 23:29:16,914 - lava_callback - ERROR - Error processing callback for node 6940966c575bc060ce1615be: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940966c575bc060ce1615be (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940966c575bc060ce1615be (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940966c575bc060ce1615be (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:16,916 - lava_callback - INFO - Starting async job submit for node 6940968a575bc060ce1615fb 2025-12-17 23:29:16,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:16,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968a575bc060ce1615fb HTTP/1.1" 502 150 2025-12-17 23:29:16,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968a575bc060ce1615fb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:16,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968a575bc060ce1615fb 2025-12-17 23:29:16,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968a575bc060ce1615fb HTTP/1.1" 502 150 2025-12-17 23:29:16,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968a575bc060ce1615fb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:16,937 - lava_callback - INFO - Received callback for node 694099ba575bc060ce161da7 INFO: 172.21.0.1:58680 - "POST /node/694099ba575bc060ce161da7 HTTP/1.0" 202 Accepted 2025-12-17 23:29:16,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940967d575bc060ce1615e1 2025-12-17 23:29:16,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967d575bc060ce1615e1 HTTP/1.1" 502 150 2025-12-17 23:29:16,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940967d575bc060ce1615e1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:17,051 - lava_callback - INFO - Received callback for node 69409c40575bc060ce16254d INFO: 172.21.0.1:58696 - "POST /node/69409c40575bc060ce16254d HTTP/1.0" 202 Accepted 2025-12-17 23:29:17,580 - lava_callback - INFO - Received callback for node 694099bc575bc060ce161daa INFO: 172.21.0.1:58712 - "POST /node/694099bc575bc060ce161daa HTTP/1.0" 202 Accepted 2025-12-17 23:29:18,239 - lava_callback - INFO - Received callback for node 694099bd575bc060ce161dad INFO: 172.21.0.1:58718 - "POST /node/694099bd575bc060ce161dad HTTP/1.0" 202 Accepted 2025-12-17 23:29:18,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968b575bc060ce1615fe 2025-12-17 23:29:18,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968b575bc060ce1615fe HTTP/1.1" 502 150 2025-12-17 23:29:18,437 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968b575bc060ce1615fe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:18,914 - lava_callback - INFO - Received callback for node 694099c0575bc060ce161db0 INFO: 172.21.0.1:58726 - "POST /node/694099c0575bc060ce161db0 HTTP/1.0" 202 Accepted 2025-12-17 23:29:18,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968a575bc060ce1615fb 2025-12-17 23:29:18,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968a575bc060ce1615fb HTTP/1.1" 502 150 2025-12-17 23:29:18,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968a575bc060ce1615fb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:19,554 - lava_callback - INFO - Received callback for node 694099c4575bc060ce161dc1 INFO: 172.21.0.1:58732 - "POST /node/694099c4575bc060ce161dc1 HTTP/1.0" 202 Accepted 2025-12-17 23:29:19,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940967b575bc060ce1615de 2025-12-17 23:29:19,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967b575bc060ce1615de HTTP/1.1" 502 150 2025-12-17 23:29:19,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940967b575bc060ce1615de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:19,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409688575bc060ce1615f8 2025-12-17 23:29:19,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409688575bc060ce1615f8 HTTP/1.1" 502 150 2025-12-17 23:29:19,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409688575bc060ce1615f8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:20,220 - lava_callback - INFO - Received callback for node 694099c7575bc060ce161dc6 INFO: 172.21.0.1:46220 - "POST /node/694099c7575bc060ce161dc6 HTTP/1.0" 202 Accepted 2025-12-17 23:29:20,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409677575bc060ce1615d4 HTTP/1.1" 502 150 2025-12-17 23:29:20,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409677575bc060ce1615d4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:20,885 - lava_callback - INFO - Received callback for node 694099c9575bc060ce161dc9 INFO: 172.21.0.1:46232 - "POST /node/694099c9575bc060ce161dc9 HTTP/1.0" 202 Accepted 2025-12-17 23:29:22,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968b575bc060ce1615fe 2025-12-17 23:29:22,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968b575bc060ce1615fe HTTP/1.1" 502 150 2025-12-17 23:29:22,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968b575bc060ce1615fe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:22,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968a575bc060ce1615fb 2025-12-17 23:29:22,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968a575bc060ce1615fb HTTP/1.1" 502 150 2025-12-17 23:29:22,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968a575bc060ce1615fb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:22,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b50575bc060ce16219b 2025-12-17 23:29:22,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b50575bc060ce16219b HTTP/1.1" 502 150 2025-12-17 23:29:23,000 - lava_callback - ERROR - Error processing callback for node 69409b50575bc060ce16219b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b50575bc060ce16219b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b50575bc060ce16219b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b50575bc060ce16219b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:23,002 - lava_callback - INFO - Starting async job submit for node 6940968c575bc060ce161601 2025-12-17 23:29:23,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:23,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968c575bc060ce161601 HTTP/1.1" 502 150 2025-12-17 23:29:23,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968c575bc060ce161601'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:23,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968c575bc060ce161601 2025-12-17 23:29:23,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968c575bc060ce161601 HTTP/1.1" 502 150 2025-12-17 23:29:23,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968c575bc060ce161601'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:24,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c69575bc060ce1625af 2025-12-17 23:29:24,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c69575bc060ce1625af HTTP/1.1" 502 150 2025-12-17 23:29:24,914 - lava_callback - ERROR - Error processing callback for node 69409c69575bc060ce1625af: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c69575bc060ce1625af (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c69575bc060ce1625af (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c69575bc060ce1625af (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:24,916 - lava_callback - INFO - Starting async job submit for node 6940968e575bc060ce161606 2025-12-17 23:29:24,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:24,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940967f575bc060ce1615e9 2025-12-17 23:29:24,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967f575bc060ce1615e9 HTTP/1.1" 502 150 2025-12-17 23:29:24,918 - lava_callback - ERROR - Error processing callback for node 6940967f575bc060ce1615e9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940967f575bc060ce1615e9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940967f575bc060ce1615e9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940967f575bc060ce1615e9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:24,920 - lava_callback - INFO - Starting async job submit for node 69409690575bc060ce16160d 2025-12-17 23:29:24,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:24,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968e575bc060ce161606 HTTP/1.1" 502 150 2025-12-17 23:29:24,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968e575bc060ce161606'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:24,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968e575bc060ce161606 2025-12-17 23:29:24,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968e575bc060ce161606 HTTP/1.1" 502 150 2025-12-17 23:29:24,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968e575bc060ce161606'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:24,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409690575bc060ce16160d HTTP/1.1" 502 150 2025-12-17 23:29:24,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409690575bc060ce16160d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:24,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409690575bc060ce16160d 2025-12-17 23:29:24,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409690575bc060ce16160d HTTP/1.1" 502 150 2025-12-17 23:29:24,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409690575bc060ce16160d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:24,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940967d575bc060ce1615e1 2025-12-17 23:29:24,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967d575bc060ce1615e1 HTTP/1.1" 502 150 2025-12-17 23:29:24,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940967d575bc060ce1615e1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:25,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968c575bc060ce161601 2025-12-17 23:29:25,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968c575bc060ce161601 HTTP/1.1" 502 150 2025-12-17 23:29:25,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968c575bc060ce161601'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46242 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:29:26,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968e575bc060ce161606 2025-12-17 23:29:26,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968e575bc060ce161606 HTTP/1.1" 502 150 2025-12-17 23:29:26,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968e575bc060ce161606'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:26,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409690575bc060ce16160d 2025-12-17 23:29:26,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409690575bc060ce16160d HTTP/1.1" 502 150 2025-12-17 23:29:26,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409690575bc060ce16160d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:27,581 - lava_callback - INFO - Received callback for node 694099d2575bc060ce161e2f INFO: 172.21.0.1:46252 - "POST /node/694099d2575bc060ce161e2f HTTP/1.0" 202 Accepted 2025-12-17 23:29:27,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409688575bc060ce1615f8 2025-12-17 23:29:27,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409688575bc060ce1615f8 HTTP/1.1" 502 150 2025-12-17 23:29:27,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409688575bc060ce1615f8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:27,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409671575bc060ce1615cb 2025-12-17 23:29:27,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409671575bc060ce1615cb HTTP/1.1" 502 150 2025-12-17 23:29:27,868 - lava_callback - ERROR - Error processing callback for node 69409671575bc060ce1615cb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409671575bc060ce1615cb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409671575bc060ce1615cb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409671575bc060ce1615cb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:27,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409681575bc060ce1615ec 2025-12-17 23:29:27,870 - lava_callback - INFO - Starting async job submit for node 69409683575bc060ce1615ef 2025-12-17 23:29:27,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:27,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409681575bc060ce1615ec HTTP/1.1" 502 150 2025-12-17 23:29:27,872 - lava_callback - ERROR - Error processing callback for node 69409681575bc060ce1615ec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409681575bc060ce1615ec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409681575bc060ce1615ec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409681575bc060ce1615ec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:27,874 - lava_callback - INFO - Starting async job submit for node 69409685575bc060ce1615f2 2025-12-17 23:29:27,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:27,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409685575bc060ce1615f2 HTTP/1.1" 502 150 2025-12-17 23:29:27,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409685575bc060ce1615f2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:27,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409685575bc060ce1615f2 2025-12-17 23:29:27,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409683575bc060ce1615ef HTTP/1.1" 502 150 2025-12-17 23:29:27,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409685575bc060ce1615f2 HTTP/1.1" 502 150 2025-12-17 23:29:27,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409683575bc060ce1615ef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:27,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409685575bc060ce1615f2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:27,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409683575bc060ce1615ef 2025-12-17 23:29:27,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409683575bc060ce1615ef HTTP/1.1" 502 150 2025-12-17 23:29:27,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409683575bc060ce1615ef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:28,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409673575bc060ce1615ce 2025-12-17 23:29:28,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409673575bc060ce1615ce HTTP/1.1" 502 150 2025-12-17 23:29:28,029 - lava_callback - ERROR - Error processing callback for node 69409673575bc060ce1615ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409673575bc060ce1615ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409673575bc060ce1615ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409673575bc060ce1615ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:28,031 - lava_callback - INFO - Starting async job submit for node 6940955f575bc060ce1612eb 2025-12-17 23:29:28,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:28,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955f575bc060ce1612eb HTTP/1.1" 502 150 2025-12-17 23:29:28,070 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955f575bc060ce1612eb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:28,071 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955f575bc060ce1612eb 2025-12-17 23:29:28,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955f575bc060ce1612eb HTTP/1.1" 502 150 2025-12-17 23:29:28,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955f575bc060ce1612eb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:28,214 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409675575bc060ce1615d1 2025-12-17 23:29:28,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409675575bc060ce1615d1 HTTP/1.1" 502 150 2025-12-17 23:29:28,216 - lava_callback - ERROR - Error processing callback for node 69409675575bc060ce1615d1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409675575bc060ce1615d1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409675575bc060ce1615d1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409675575bc060ce1615d1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:28,219 - lava_callback - INFO - Starting async job submit for node 69409697575bc060ce161624 2025-12-17 23:29:28,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:28,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409697575bc060ce161624 HTTP/1.1" 502 150 2025-12-17 23:29:28,226 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409697575bc060ce161624'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:28,226 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409697575bc060ce161624 2025-12-17 23:29:28,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409697575bc060ce161624 HTTP/1.1" 502 150 2025-12-17 23:29:28,227 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409697575bc060ce161624'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:28,244 - lava_callback - INFO - Received callback for node 694099d5575bc060ce161e32 INFO: 172.21.0.1:46256 - "POST /node/694099d5575bc060ce161e32 HTTP/1.0" 202 Accepted 2025-12-17 23:29:28,304 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409677575bc060ce1615d4 2025-12-17 23:29:28,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409677575bc060ce1615d4 HTTP/1.1" 502 150 2025-12-17 23:29:28,306 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409677575bc060ce1615d4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:29,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968c575bc060ce161601 2025-12-17 23:29:29,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409685575bc060ce1615f2 2025-12-17 23:29:29,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409683575bc060ce1615ef 2025-12-17 23:29:29,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409685575bc060ce1615f2 HTTP/1.1" 502 150 2025-12-17 23:29:29,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409685575bc060ce1615f2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:29,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409683575bc060ce1615ef HTTP/1.1" 502 150 2025-12-17 23:29:29,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409683575bc060ce1615ef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:30,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955f575bc060ce1612eb 2025-12-17 23:29:30,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955f575bc060ce1612eb HTTP/1.1" 502 150 2025-12-17 23:29:30,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955f575bc060ce1612eb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:30,227 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409697575bc060ce161624 2025-12-17 23:29:30,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409697575bc060ce161624 HTTP/1.1" 502 150 2025-12-17 23:29:30,228 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409697575bc060ce161624'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:30,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968b575bc060ce1615fe 2025-12-17 23:29:30,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968b575bc060ce1615fe HTTP/1.1" 502 150 2025-12-17 23:29:30,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968b575bc060ce1615fe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:30,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968a575bc060ce1615fb 2025-12-17 23:29:30,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968a575bc060ce1615fb HTTP/1.1" 502 150 2025-12-17 23:29:30,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968a575bc060ce1615fb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:30,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968e575bc060ce161606 2025-12-17 23:29:30,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968e575bc060ce161606 HTTP/1.1" 502 150 2025-12-17 23:29:30,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968e575bc060ce161606'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:30,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409690575bc060ce16160d 2025-12-17 23:29:30,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409690575bc060ce16160d HTTP/1.1" 502 150 2025-12-17 23:29:30,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409690575bc060ce16160d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:31,250 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409669575bc060ce1615b6 2025-12-17 23:29:31,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409679575bc060ce1615d7 2025-12-17 23:29:31,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409679575bc060ce1615d7 HTTP/1.1" 502 150 2025-12-17 23:29:31,626 - lava_callback - ERROR - Error processing callback for node 69409679575bc060ce1615d7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409679575bc060ce1615d7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409679575bc060ce1615d7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409679575bc060ce1615d7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:31,627 - lava_callback - INFO - Starting async job submit for node 69409699575bc060ce161627 2025-12-17 23:29:31,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:31,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409699575bc060ce161627 HTTP/1.1" 502 150 2025-12-17 23:29:31,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409699575bc060ce161627'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:31,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409699575bc060ce161627 2025-12-17 23:29:31,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409699575bc060ce161627 HTTP/1.1" 502 150 2025-12-17 23:29:31,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409699575bc060ce161627'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:33,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409699575bc060ce161627 2025-12-17 23:29:33,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409699575bc060ce161627 HTTP/1.1" 502 150 2025-12-17 23:29:33,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409699575bc060ce161627'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:33,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409685575bc060ce1615f2 2025-12-17 23:29:33,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409685575bc060ce1615f2 HTTP/1.1" 502 150 2025-12-17 23:29:33,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409685575bc060ce1615f2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:33,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409683575bc060ce1615ef 2025-12-17 23:29:33,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409683575bc060ce1615ef HTTP/1.1" 502 150 2025-12-17 23:29:33,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409683575bc060ce1615ef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:34,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955f575bc060ce1612eb 2025-12-17 23:29:34,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955f575bc060ce1612eb HTTP/1.1" 502 150 2025-12-17 23:29:34,075 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955f575bc060ce1612eb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:34,229 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409697575bc060ce161624 2025-12-17 23:29:34,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409697575bc060ce161624 HTTP/1.1" 502 150 2025-12-17 23:29:34,230 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409697575bc060ce161624'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:35,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940967b575bc060ce1615de 2025-12-17 23:29:35,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967b575bc060ce1615de HTTP/1.1" 502 150 2025-12-17 23:29:35,572 - lava_callback - ERROR - Error processing callback for node 6940967b575bc060ce1615de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940967b575bc060ce1615de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940967b575bc060ce1615de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940967b575bc060ce1615de (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:35,574 - lava_callback - INFO - Starting async job submit for node 69409691575bc060ce161612 2025-12-17 23:29:35,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:35,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409691575bc060ce161612 HTTP/1.1" 502 150 2025-12-17 23:29:35,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409691575bc060ce161612'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:35,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409691575bc060ce161612 2025-12-17 23:29:35,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409691575bc060ce161612 HTTP/1.1" 502 150 2025-12-17 23:29:35,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409691575bc060ce161612'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54148 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:29:36,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409628575bc060ce161512 HTTP/1.1" 502 150 2025-12-17 23:29:36,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409628575bc060ce161512'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:36,965 - lava_callback - INFO - Received callback for node 694099cf575bc060ce161e25 INFO: 172.21.0.1:54152 - "POST /node/694099cf575bc060ce161e25 HTTP/1.0" 202 Accepted 2025-12-17 23:29:37,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409691575bc060ce161612 2025-12-17 23:29:37,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409691575bc060ce161612 HTTP/1.1" 502 150 2025-12-17 23:29:37,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409691575bc060ce161612'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:37,621 - lava_callback - INFO - Received callback for node 694099d1575bc060ce161e2c INFO: 172.21.0.1:54166 - "POST /node/694099d1575bc060ce161e2c HTTP/1.0" 202 Accepted 2025-12-17 23:29:37,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409699575bc060ce161627 2025-12-17 23:29:37,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409699575bc060ce161627 HTTP/1.1" 502 150 2025-12-17 23:29:37,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409699575bc060ce161627'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:38,324 - lava_callback - INFO - Received callback for node 694099d9575bc060ce161e48 INFO: 172.21.0.1:54170 - "POST /node/694099d9575bc060ce161e48 HTTP/1.0" 202 Accepted 2025-12-17 23:29:38,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968e575bc060ce161606 2025-12-17 23:29:38,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968e575bc060ce161606 HTTP/1.1" 502 150 2025-12-17 23:29:38,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968e575bc060ce161606'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:38,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409690575bc060ce16160d 2025-12-17 23:29:38,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409690575bc060ce16160d HTTP/1.1" 502 150 2025-12-17 23:29:38,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409690575bc060ce16160d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:39,009 - lava_callback - INFO - Received callback for node 694099dd575bc060ce161e4e INFO: 172.21.0.1:54178 - "POST /node/694099dd575bc060ce161e4e HTTP/1.0" 202 Accepted 2025-12-17 23:29:40,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940967d575bc060ce1615e1 2025-12-17 23:29:40,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940967d575bc060ce1615e1 HTTP/1.1" 502 150 2025-12-17 23:29:40,984 - lava_callback - ERROR - Error processing callback for node 6940967d575bc060ce1615e1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940967d575bc060ce1615e1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940967d575bc060ce1615e1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940967d575bc060ce1615e1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:40,986 - lava_callback - INFO - Starting async job submit for node 694096a0575bc060ce161633 2025-12-17 23:29:40,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:41,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409691575bc060ce161612 2025-12-17 23:29:41,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409691575bc060ce161612 HTTP/1.1" 502 150 2025-12-17 23:29:41,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409691575bc060ce161612'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:41,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409685575bc060ce1615f2 2025-12-17 23:29:41,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409685575bc060ce1615f2 HTTP/1.1" 502 150 2025-12-17 23:29:41,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409685575bc060ce1615f2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:41,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409683575bc060ce1615ef 2025-12-17 23:29:41,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409683575bc060ce1615ef HTTP/1.1" 502 150 2025-12-17 23:29:41,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409683575bc060ce1615ef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:42,075 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955f575bc060ce1612eb 2025-12-17 23:29:42,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955f575bc060ce1612eb HTTP/1.1" 502 150 2025-12-17 23:29:42,076 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940955f575bc060ce1612eb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:42,230 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409697575bc060ce161624 2025-12-17 23:29:42,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409697575bc060ce161624 HTTP/1.1" 502 150 2025-12-17 23:29:42,232 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409697575bc060ce161624'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:43,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409688575bc060ce1615f8 2025-12-17 23:29:43,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409688575bc060ce1615f8 HTTP/1.1" 502 150 2025-12-17 23:29:43,602 - lava_callback - ERROR - Error processing callback for node 69409688575bc060ce1615f8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409688575bc060ce1615f8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409688575bc060ce1615f8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409688575bc060ce1615f8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:43,604 - lava_callback - INFO - Starting async job submit for node 69409693575bc060ce161618 2025-12-17 23:29:43,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:43,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409693575bc060ce161618 HTTP/1.1" 502 150 2025-12-17 23:29:43,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409693575bc060ce161618'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:43,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409693575bc060ce161618 2025-12-17 23:29:43,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409693575bc060ce161618 HTTP/1.1" 502 150 2025-12-17 23:29:43,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409693575bc060ce161618'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:44,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409677575bc060ce1615d4 2025-12-17 23:29:44,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409677575bc060ce1615d4 HTTP/1.1" 502 150 2025-12-17 23:29:44,307 - lava_callback - ERROR - Error processing callback for node 69409677575bc060ce1615d4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409677575bc060ce1615d4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409677575bc060ce1615d4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409677575bc060ce1615d4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:44,309 - lava_callback - INFO - Starting async job submit for node 69409693575bc060ce16161b 2025-12-17 23:29:44,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:44,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409693575bc060ce16161b HTTP/1.1" 502 150 2025-12-17 23:29:44,315 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409693575bc060ce16161b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:44,315 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409693575bc060ce16161b 2025-12-17 23:29:44,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409693575bc060ce16161b HTTP/1.1" 502 150 2025-12-17 23:29:44,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409693575bc060ce16161b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:44,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968c575bc060ce161601 HTTP/1.1" 502 150 2025-12-17 23:29:44,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968c575bc060ce161601'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:44,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409628575bc060ce161512 2025-12-17 23:29:45,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409693575bc060ce161618 2025-12-17 23:29:45,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409693575bc060ce161618 HTTP/1.1" 502 150 2025-12-17 23:29:45,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409693575bc060ce161618'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:45,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409699575bc060ce161627 2025-12-17 23:29:45,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409699575bc060ce161627 HTTP/1.1" 502 150 2025-12-17 23:29:45,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409699575bc060ce161627'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:46,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409693575bc060ce16161b 2025-12-17 23:29:46,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409693575bc060ce16161b HTTP/1.1" 502 150 2025-12-17 23:29:46,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409693575bc060ce16161b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:46,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968b575bc060ce1615fe 2025-12-17 23:29:46,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968b575bc060ce1615fe HTTP/1.1" 502 150 2025-12-17 23:29:46,441 - lava_callback - ERROR - Error processing callback for node 6940968b575bc060ce1615fe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940968b575bc060ce1615fe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940968b575bc060ce1615fe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940968b575bc060ce1615fe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:46,443 - lava_callback - INFO - Starting async job submit for node 69409694575bc060ce16161e 2025-12-17 23:29:46,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:46,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409694575bc060ce16161e HTTP/1.1" 502 150 2025-12-17 23:29:46,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409694575bc060ce16161e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:46,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409694575bc060ce16161e 2025-12-17 23:29:46,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409694575bc060ce16161e HTTP/1.1" 502 150 2025-12-17 23:29:46,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409694575bc060ce16161e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50246 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:29:46,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409669575bc060ce1615b6 HTTP/1.1" 502 150 2025-12-17 23:29:46,672 - lava_callback - ERROR - Error processing callback for node 69409669575bc060ce1615b6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409669575bc060ce1615b6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409669575bc060ce1615b6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409669575bc060ce1615b6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:46,674 - lava_callback - INFO - Starting async job submit for node 69409695575bc060ce161621 2025-12-17 23:29:46,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:46,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409695575bc060ce161621 HTTP/1.1" 502 150 2025-12-17 23:29:46,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409695575bc060ce161621'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:46,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409695575bc060ce161621 2025-12-17 23:29:46,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409695575bc060ce161621 HTTP/1.1" 502 150 2025-12-17 23:29:46,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409695575bc060ce161621'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:46,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968a575bc060ce1615fb 2025-12-17 23:29:46,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968a575bc060ce1615fb HTTP/1.1" 502 150 2025-12-17 23:29:46,930 - lava_callback - ERROR - Error processing callback for node 6940968a575bc060ce1615fb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940968a575bc060ce1615fb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940968a575bc060ce1615fb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940968a575bc060ce1615fb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:46,933 - lava_callback - INFO - Starting async job submit for node 6940969a575bc060ce16162a 2025-12-17 23:29:46,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:46,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940969a575bc060ce16162a HTTP/1.1" 502 150 2025-12-17 23:29:46,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940969a575bc060ce16162a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:46,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940969a575bc060ce16162a 2025-12-17 23:29:46,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940969a575bc060ce16162a HTTP/1.1" 502 150 2025-12-17 23:29:46,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940969a575bc060ce16162a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:47,585 - lava_callback - INFO - Received callback for node 694099db575bc060ce161e4b INFO: 172.21.0.1:50262 - "POST /node/694099db575bc060ce161e4b HTTP/1.0" 202 Accepted 2025-12-17 23:29:48,256 - lava_callback - INFO - Received callback for node 694099df575bc060ce161e51 INFO: 172.21.0.1:50274 - "POST /node/694099df575bc060ce161e51 HTTP/1.0" 202 Accepted 2025-12-17 23:29:48,420 - lava_callback - INFO - Received callback for node 694099d8575bc060ce161e3f INFO: 172.21.0.1:50280 - "POST /node/694099d8575bc060ce161e3f HTTP/1.0" 202 Accepted 2025-12-17 23:29:48,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409694575bc060ce16161e 2025-12-17 23:29:48,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409694575bc060ce16161e HTTP/1.1" 502 150 2025-12-17 23:29:48,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409694575bc060ce16161e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:48,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409695575bc060ce161621 2025-12-17 23:29:48,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409695575bc060ce161621 HTTP/1.1" 502 150 2025-12-17 23:29:48,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409695575bc060ce161621'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:48,921 - lava_callback - INFO - Received callback for node 694099e5575bc060ce161e5d INFO: 172.21.0.1:50282 - "POST /node/694099e5575bc060ce161e5d HTTP/1.0" 202 Accepted 2025-12-17 23:29:48,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940969a575bc060ce16162a 2025-12-17 23:29:48,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940969a575bc060ce16162a HTTP/1.1" 502 150 2025-12-17 23:29:48,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940969a575bc060ce16162a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:49,235 - lava_callback - INFO - Received callback for node 694099e2575bc060ce161e57 INFO: 172.21.0.1:50286 - "POST /node/694099e2575bc060ce161e57 HTTP/1.0" 202 Accepted 2025-12-17 23:29:49,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409691575bc060ce161612 2025-12-17 23:29:49,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409691575bc060ce161612 HTTP/1.1" 502 150 2025-12-17 23:29:49,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409691575bc060ce161612'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:49,598 - lava_callback - INFO - Received callback for node 694099e7575bc060ce161e60 INFO: 172.21.0.1:50298 - "POST /node/694099e7575bc060ce161e60 HTTP/1.0" 202 Accepted 2025-12-17 23:29:49,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409693575bc060ce161618 2025-12-17 23:29:49,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409693575bc060ce161618 HTTP/1.1" 502 150 2025-12-17 23:29:49,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409693575bc060ce161618'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:49,991 - lava_callback - INFO - Received callback for node 694099e3575bc060ce161e5a INFO: 172.21.0.1:57088 - "POST /node/694099e3575bc060ce161e5a HTTP/1.0" 202 Accepted 2025-12-17 23:29:50,277 - lava_callback - INFO - Received callback for node 694099e9575bc060ce161e67 INFO: 172.21.0.1:57096 - "POST /node/694099e9575bc060ce161e67 HTTP/1.0" 202 Accepted 2025-12-17 23:29:50,318 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409693575bc060ce16161b 2025-12-17 23:29:50,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409693575bc060ce16161b HTTP/1.1" 502 150 2025-12-17 23:29:50,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409693575bc060ce16161b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:51,786 - lava_callback - INFO - Received callback for node 694099e1575bc060ce161e54 INFO: 172.21.0.1:57098 - "POST /node/694099e1575bc060ce161e54 HTTP/1.0" 202 Accepted 2025-12-17 23:29:52,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968c575bc060ce161601 2025-12-17 23:29:52,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968c575bc060ce161601 HTTP/1.1" 502 150 2025-12-17 23:29:52,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940968c575bc060ce161601'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:52,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409694575bc060ce16161e 2025-12-17 23:29:52,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409694575bc060ce16161e HTTP/1.1" 502 150 2025-12-17 23:29:52,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409694575bc060ce16161e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:52,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409695575bc060ce161621 2025-12-17 23:29:52,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409695575bc060ce161621 HTTP/1.1" 502 150 2025-12-17 23:29:52,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409695575bc060ce161621'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:52,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940969a575bc060ce16162a 2025-12-17 23:29:52,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940969a575bc060ce16162a HTTP/1.1" 502 150 2025-12-17 23:29:52,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940969a575bc060ce16162a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:54,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968e575bc060ce161606 2025-12-17 23:29:54,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968e575bc060ce161606 HTTP/1.1" 502 150 2025-12-17 23:29:54,932 - lava_callback - ERROR - Error processing callback for node 6940968e575bc060ce161606: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940968e575bc060ce161606 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940968e575bc060ce161606 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940968e575bc060ce161606 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:54,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409690575bc060ce16160d 2025-12-17 23:29:54,934 - lava_callback - INFO - Starting async job submit for node 6940969d575bc060ce16162d 2025-12-17 23:29:54,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:54,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409690575bc060ce16160d HTTP/1.1" 502 150 2025-12-17 23:29:54,936 - lava_callback - ERROR - Error processing callback for node 69409690575bc060ce16160d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409690575bc060ce16160d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409690575bc060ce16160d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409690575bc060ce16160d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:54,937 - lava_callback - INFO - Starting async job submit for node 69409c6c575bc060ce1625bf 2025-12-17 23:29:55,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:55,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6c575bc060ce1625bf HTTP/1.1" 502 150 2025-12-17 23:29:55,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6c575bc060ce1625bf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:55,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6c575bc060ce1625bf 2025-12-17 23:29:55,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6c575bc060ce1625bf HTTP/1.1" 502 150 2025-12-17 23:29:55,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6c575bc060ce1625bf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:56,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a0575bc060ce161633 HTTP/1.1" 502 150 2025-12-17 23:29:56,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a0575bc060ce161633'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:56,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a0575bc060ce161633 2025-12-17 23:29:56,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a0575bc060ce161633 HTTP/1.1" 502 150 2025-12-17 23:29:56,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a0575bc060ce161633'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57114 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:29:57,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6c575bc060ce1625bf 2025-12-17 23:29:57,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409693575bc060ce161618 2025-12-17 23:29:57,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409693575bc060ce161618 HTTP/1.1" 502 150 2025-12-17 23:29:57,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409693575bc060ce161618'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:57,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409685575bc060ce1615f2 2025-12-17 23:29:57,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409685575bc060ce1615f2 HTTP/1.1" 502 150 2025-12-17 23:29:57,888 - lava_callback - ERROR - Error processing callback for node 69409685575bc060ce1615f2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409685575bc060ce1615f2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409685575bc060ce1615f2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409685575bc060ce1615f2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:57,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409683575bc060ce1615ef 2025-12-17 23:29:57,890 - lava_callback - INFO - Starting async job submit for node 694096a3575bc060ce161640 2025-12-17 23:29:57,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:57,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409683575bc060ce1615ef HTTP/1.1" 502 150 2025-12-17 23:29:57,891 - lava_callback - ERROR - Error processing callback for node 69409683575bc060ce1615ef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409683575bc060ce1615ef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409683575bc060ce1615ef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409683575bc060ce1615ef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:57,893 - lava_callback - INFO - Starting async job submit for node 694096a5575bc060ce161643 2025-12-17 23:29:57,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:57,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a3575bc060ce161640 HTTP/1.1" 502 150 2025-12-17 23:29:57,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a3575bc060ce161640'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:57,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a3575bc060ce161640 2025-12-17 23:29:57,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a3575bc060ce161640 HTTP/1.1" 502 150 2025-12-17 23:29:57,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a3575bc060ce161640'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:57,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a5575bc060ce161643 HTTP/1.1" 502 150 2025-12-17 23:29:57,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a5575bc060ce161643'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:57,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a5575bc060ce161643 2025-12-17 23:29:57,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a5575bc060ce161643 HTTP/1.1" 502 150 2025-12-17 23:29:57,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a5575bc060ce161643'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:58,076 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940955f575bc060ce1612eb 2025-12-17 23:29:58,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940955f575bc060ce1612eb HTTP/1.1" 502 150 2025-12-17 23:29:58,077 - lava_callback - ERROR - Error processing callback for node 6940955f575bc060ce1612eb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940955f575bc060ce1612eb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940955f575bc060ce1612eb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940955f575bc060ce1612eb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:58,079 - lava_callback - INFO - Starting async job submit for node 694096af575bc060ce16165a 2025-12-17 23:29:58,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:58,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096af575bc060ce16165a HTTP/1.1" 502 150 2025-12-17 23:29:58,086 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096af575bc060ce16165a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:58,086 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096af575bc060ce16165a 2025-12-17 23:29:58,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096af575bc060ce16165a HTTP/1.1" 502 150 2025-12-17 23:29:58,086 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096af575bc060ce16165a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:58,232 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409697575bc060ce161624 2025-12-17 23:29:58,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409697575bc060ce161624 HTTP/1.1" 502 150 2025-12-17 23:29:58,233 - lava_callback - ERROR - Error processing callback for node 69409697575bc060ce161624: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409697575bc060ce161624 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409697575bc060ce161624 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409697575bc060ce161624 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:29:58,235 - lava_callback - INFO - Starting async job submit for node 694096a7575bc060ce161646 2025-12-17 23:29:58,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:29:58,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a7575bc060ce161646 HTTP/1.1" 502 150 2025-12-17 23:29:58,241 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a7575bc060ce161646'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:58,241 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a7575bc060ce161646 2025-12-17 23:29:58,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a7575bc060ce161646 HTTP/1.1" 502 150 2025-12-17 23:29:58,242 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a7575bc060ce161646'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:58,319 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409693575bc060ce16161b 2025-12-17 23:29:58,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409693575bc060ce16161b HTTP/1.1" 502 150 2025-12-17 23:29:58,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409693575bc060ce16161b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:58,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a0575bc060ce161633 2025-12-17 23:29:58,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a0575bc060ce161633 HTTP/1.1" 502 150 2025-12-17 23:29:58,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a0575bc060ce161633'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:59,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a3575bc060ce161640 2025-12-17 23:29:59,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a3575bc060ce161640 HTTP/1.1" 502 150 2025-12-17 23:29:59,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a3575bc060ce161640'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:29:59,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a5575bc060ce161643 2025-12-17 23:29:59,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a5575bc060ce161643 HTTP/1.1" 502 150 2025-12-17 23:29:59,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a5575bc060ce161643'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:00,087 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096af575bc060ce16165a 2025-12-17 23:30:00,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096af575bc060ce16165a HTTP/1.1" 502 150 2025-12-17 23:30:00,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096af575bc060ce16165a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:00,242 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a7575bc060ce161646 2025-12-17 23:30:00,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a7575bc060ce161646 HTTP/1.1" 502 150 2025-12-17 23:30:00,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a7575bc060ce161646'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:00,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409694575bc060ce16161e 2025-12-17 23:30:00,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409694575bc060ce16161e HTTP/1.1" 502 150 2025-12-17 23:30:00,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409694575bc060ce16161e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:00,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409695575bc060ce161621 2025-12-17 23:30:00,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409695575bc060ce161621 HTTP/1.1" 502 150 2025-12-17 23:30:00,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409695575bc060ce161621'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:00,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940969a575bc060ce16162a 2025-12-17 23:30:00,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940969a575bc060ce16162a HTTP/1.1" 502 150 2025-12-17 23:30:00,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940969a575bc060ce16162a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:01,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409699575bc060ce161627 2025-12-17 23:30:01,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409699575bc060ce161627 HTTP/1.1" 502 150 2025-12-17 23:30:01,672 - lava_callback - ERROR - Error processing callback for node 69409699575bc060ce161627: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409699575bc060ce161627 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409699575bc060ce161627 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409699575bc060ce161627 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:01,674 - lava_callback - INFO - Starting async job submit for node 694096a9575bc060ce161649 2025-12-17 23:30:01,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:01,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a9575bc060ce161649 HTTP/1.1" 502 150 2025-12-17 23:30:01,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a9575bc060ce161649'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:01,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a9575bc060ce161649 2025-12-17 23:30:01,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a9575bc060ce161649 HTTP/1.1" 502 150 2025-12-17 23:30:01,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a9575bc060ce161649'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:02,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a0575bc060ce161633 2025-12-17 23:30:02,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a0575bc060ce161633 HTTP/1.1" 502 150 2025-12-17 23:30:02,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a0575bc060ce161633'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33226 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:30:03,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a9575bc060ce161649 2025-12-17 23:30:03,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a9575bc060ce161649 HTTP/1.1" 502 150 2025-12-17 23:30:03,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a9575bc060ce161649'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:03,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a3575bc060ce161640 2025-12-17 23:30:03,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a3575bc060ce161640 HTTP/1.1" 502 150 2025-12-17 23:30:03,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a3575bc060ce161640'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:03,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a5575bc060ce161643 2025-12-17 23:30:03,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a5575bc060ce161643 HTTP/1.1" 502 150 2025-12-17 23:30:03,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a5575bc060ce161643'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:04,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096af575bc060ce16165a 2025-12-17 23:30:04,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096af575bc060ce16165a HTTP/1.1" 502 150 2025-12-17 23:30:04,089 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096af575bc060ce16165a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:04,244 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a7575bc060ce161646 2025-12-17 23:30:04,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a7575bc060ce161646 HTTP/1.1" 502 150 2025-12-17 23:30:04,245 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a7575bc060ce161646'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:05,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409691575bc060ce161612 2025-12-17 23:30:05,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409691575bc060ce161612 HTTP/1.1" 502 150 2025-12-17 23:30:05,587 - lava_callback - ERROR - Error processing callback for node 69409691575bc060ce161612: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409691575bc060ce161612 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409691575bc060ce161612 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409691575bc060ce161612 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:05,589 - lava_callback - INFO - Starting async job submit for node 694096ab575bc060ce16164c 2025-12-17 23:30:05,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:05,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ab575bc060ce16164c HTTP/1.1" 502 150 2025-12-17 23:30:05,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ab575bc060ce16164c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:05,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ab575bc060ce16164c 2025-12-17 23:30:05,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ab575bc060ce16164c HTTP/1.1" 502 150 2025-12-17 23:30:05,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ab575bc060ce16164c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33234 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:30:07,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ab575bc060ce16164c 2025-12-17 23:30:07,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ab575bc060ce16164c HTTP/1.1" 502 150 2025-12-17 23:30:07,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ab575bc060ce16164c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:07,626 - lava_callback - INFO - Received callback for node 694099ef575bc060ce161e7a INFO: 172.21.0.1:33240 - "POST /node/694099ef575bc060ce161e7a HTTP/1.0" 202 Accepted 2025-12-17 23:30:07,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a9575bc060ce161649 2025-12-17 23:30:07,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a9575bc060ce161649 HTTP/1.1" 502 150 2025-12-17 23:30:07,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a9575bc060ce161649'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:08,286 - lava_callback - INFO - Received callback for node 694099f1575bc060ce161e7d INFO: 172.21.0.1:33248 - "POST /node/694099f1575bc060ce161e7d HTTP/1.0" 202 Accepted 2025-12-17 23:30:08,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940968c575bc060ce161601 2025-12-17 23:30:08,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940968c575bc060ce161601 HTTP/1.1" 502 150 2025-12-17 23:30:08,371 - lava_callback - ERROR - Error processing callback for node 6940968c575bc060ce161601: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940968c575bc060ce161601 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940968c575bc060ce161601 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940968c575bc060ce161601 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:08,373 - lava_callback - INFO - Starting async job submit for node 694096ad575bc060ce161657 2025-12-17 23:30:08,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:08,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ad575bc060ce161657 HTTP/1.1" 502 150 2025-12-17 23:30:08,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ad575bc060ce161657'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:08,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ad575bc060ce161657 2025-12-17 23:30:08,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ad575bc060ce161657 HTTP/1.1" 502 150 2025-12-17 23:30:08,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ad575bc060ce161657'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:08,945 - lava_callback - INFO - Received callback for node 694099f3575bc060ce161e80 INFO: 172.21.0.1:33252 - "POST /node/694099f3575bc060ce161e80 HTTP/1.0" 202 Accepted 2025-12-17 23:30:09,590 - lava_callback - INFO - Received callback for node 694099f5575bc060ce161e83 INFO: 172.21.0.1:33258 - "POST /node/694099f5575bc060ce161e83 HTTP/1.0" 202 Accepted 2025-12-17 23:30:10,248 - lava_callback - INFO - Received callback for node 694099f7575bc060ce161e89 INFO: 172.21.0.1:42554 - "POST /node/694099f7575bc060ce161e89 HTTP/1.0" 202 Accepted 2025-12-17 23:30:10,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ad575bc060ce161657 2025-12-17 23:30:10,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a0575bc060ce161633 2025-12-17 23:30:10,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a0575bc060ce161633 HTTP/1.1" 502 150 2025-12-17 23:30:10,405 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a0575bc060ce161633'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:10,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940969d575bc060ce16162d HTTP/1.1" 502 150 2025-12-17 23:30:10,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940969d575bc060ce16162d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:10,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940969d575bc060ce16162d 2025-12-17 23:30:10,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940969d575bc060ce16162d HTTP/1.1" 502 150 2025-12-17 23:30:10,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940969d575bc060ce16162d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:10,940 - lava_callback - INFO - Received callback for node 694099f9575bc060ce161e8c INFO: 172.21.0.1:42562 - "POST /node/694099f9575bc060ce161e8c HTTP/1.0" 202 Accepted 2025-12-17 23:30:11,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ab575bc060ce16164c 2025-12-17 23:30:11,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ab575bc060ce16164c HTTP/1.1" 502 150 2025-12-17 23:30:11,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ab575bc060ce16164c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:11,646 - lava_callback - INFO - Received callback for node 694099fb575bc060ce161e8f INFO: 172.21.0.1:42576 - "POST /node/694099fb575bc060ce161e8f HTTP/1.0" 202 Accepted 2025-12-17 23:30:11,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a3575bc060ce161640 2025-12-17 23:30:11,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a5575bc060ce161643 2025-12-17 23:30:11,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a3575bc060ce161640 HTTP/1.1" 502 150 2025-12-17 23:30:11,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a3575bc060ce161640'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:11,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a5575bc060ce161643 HTTP/1.1" 502 150 2025-12-17 23:30:11,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a5575bc060ce161643'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:12,090 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096af575bc060ce16165a 2025-12-17 23:30:12,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096af575bc060ce16165a HTTP/1.1" 502 150 2025-12-17 23:30:12,091 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096af575bc060ce16165a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:12,245 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a7575bc060ce161646 2025-12-17 23:30:12,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a7575bc060ce161646 HTTP/1.1" 502 150 2025-12-17 23:30:12,246 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a7575bc060ce161646'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:12,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6c575bc060ce1625bf HTTP/1.1" 502 150 2025-12-17 23:30:12,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6c575bc060ce1625bf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:12,308 - lava_callback - INFO - Received callback for node 694099fd575bc060ce161e92 INFO: 172.21.0.1:42580 - "POST /node/694099fd575bc060ce161e92 HTTP/1.0" 202 Accepted 2025-12-17 23:30:12,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940969d575bc060ce16162d 2025-12-17 23:30:12,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940969d575bc060ce16162d HTTP/1.1" 502 150 2025-12-17 23:30:12,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940969d575bc060ce16162d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:13,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409693575bc060ce161618 2025-12-17 23:30:13,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409693575bc060ce161618 HTTP/1.1" 502 150 2025-12-17 23:30:13,619 - lava_callback - ERROR - Error processing callback for node 69409693575bc060ce161618: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409693575bc060ce161618 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409693575bc060ce161618 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409693575bc060ce161618 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:13,621 - lava_callback - INFO - Starting async job submit for node 694096b0575bc060ce16165d 2025-12-17 23:30:13,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:13,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b0575bc060ce16165d HTTP/1.1" 502 150 2025-12-17 23:30:13,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b0575bc060ce16165d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:13,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b0575bc060ce16165d 2025-12-17 23:30:13,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b0575bc060ce16165d HTTP/1.1" 502 150 2025-12-17 23:30:13,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b0575bc060ce16165d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:14,320 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409693575bc060ce16161b 2025-12-17 23:30:14,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409693575bc060ce16161b HTTP/1.1" 502 150 2025-12-17 23:30:14,322 - lava_callback - ERROR - Error processing callback for node 69409693575bc060ce16161b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409693575bc060ce16161b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409693575bc060ce16161b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409693575bc060ce16161b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:14,323 - lava_callback - INFO - Starting async job submit for node 694096b2575bc060ce161660 2025-12-17 23:30:14,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:14,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b2575bc060ce161660 HTTP/1.1" 502 150 2025-12-17 23:30:14,333 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b2575bc060ce161660'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:14,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b2575bc060ce161660 2025-12-17 23:30:14,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b2575bc060ce161660 HTTP/1.1" 502 150 2025-12-17 23:30:14,334 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b2575bc060ce161660'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42592 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:30:15,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409628575bc060ce161512 HTTP/1.1" 502 150 2025-12-17 23:30:15,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409628575bc060ce161512'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:15,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b0575bc060ce16165d 2025-12-17 23:30:15,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b0575bc060ce16165d HTTP/1.1" 502 150 2025-12-17 23:30:15,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b0575bc060ce16165d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:15,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a9575bc060ce161649 2025-12-17 23:30:15,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a9575bc060ce161649 HTTP/1.1" 502 150 2025-12-17 23:30:15,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096a9575bc060ce161649'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:16,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6c575bc060ce1625bf 2025-12-17 23:30:16,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6c575bc060ce1625bf HTTP/1.1" 502 150 2025-12-17 23:30:16,275 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6c575bc060ce1625bf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:16,334 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b2575bc060ce161660 2025-12-17 23:30:16,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b2575bc060ce161660 HTTP/1.1" 502 150 2025-12-17 23:30:16,335 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b2575bc060ce161660'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:16,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409694575bc060ce16161e 2025-12-17 23:30:16,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409694575bc060ce16161e HTTP/1.1" 502 150 2025-12-17 23:30:16,457 - lava_callback - ERROR - Error processing callback for node 69409694575bc060ce16161e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409694575bc060ce16161e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409694575bc060ce16161e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409694575bc060ce16161e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:16,459 - lava_callback - INFO - Starting async job submit for node 694096b3575bc060ce161663 2025-12-17 23:30:16,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:16,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b3575bc060ce161663 HTTP/1.1" 502 150 2025-12-17 23:30:16,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b3575bc060ce161663'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:16,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b3575bc060ce161663 2025-12-17 23:30:16,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b3575bc060ce161663 HTTP/1.1" 502 150 2025-12-17 23:30:16,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b3575bc060ce161663'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:16,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940969d575bc060ce16162d 2025-12-17 23:30:16,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940969d575bc060ce16162d HTTP/1.1" 502 150 2025-12-17 23:30:16,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940969d575bc060ce16162d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42598 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:30:16,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409695575bc060ce161621 2025-12-17 23:30:16,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409695575bc060ce161621 HTTP/1.1" 502 150 2025-12-17 23:30:16,689 - lava_callback - ERROR - Error processing callback for node 69409695575bc060ce161621: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409695575bc060ce161621 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409695575bc060ce161621 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409695575bc060ce161621 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:16,691 - lava_callback - INFO - Starting async job submit for node 694096b5575bc060ce161667 2025-12-17 23:30:16,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:16,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b5575bc060ce161667 HTTP/1.1" 502 150 2025-12-17 23:30:16,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b5575bc060ce161667'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:16,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b5575bc060ce161667 2025-12-17 23:30:16,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b5575bc060ce161667 HTTP/1.1" 502 150 2025-12-17 23:30:16,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b5575bc060ce161667'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:16,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940969a575bc060ce16162a 2025-12-17 23:30:16,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940969a575bc060ce16162a HTTP/1.1" 502 150 2025-12-17 23:30:16,948 - lava_callback - ERROR - Error processing callback for node 6940969a575bc060ce16162a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940969a575bc060ce16162a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940969a575bc060ce16162a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940969a575bc060ce16162a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:16,950 - lava_callback - INFO - Starting async job submit for node 694096bb575bc060ce16167d 2025-12-17 23:30:16,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:16,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096bb575bc060ce16167d HTTP/1.1" 502 150 2025-12-17 23:30:16,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096bb575bc060ce16167d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:16,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096bb575bc060ce16167d 2025-12-17 23:30:16,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096bb575bc060ce16167d HTTP/1.1" 502 150 2025-12-17 23:30:16,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096bb575bc060ce16167d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:18,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b3575bc060ce161663 2025-12-17 23:30:18,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b3575bc060ce161663 HTTP/1.1" 502 150 2025-12-17 23:30:18,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b3575bc060ce161663'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:18,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b5575bc060ce161667 2025-12-17 23:30:18,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b5575bc060ce161667 HTTP/1.1" 502 150 2025-12-17 23:30:18,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b5575bc060ce161667'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:18,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096bb575bc060ce16167d 2025-12-17 23:30:18,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096bb575bc060ce16167d HTTP/1.1" 502 150 2025-12-17 23:30:18,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096bb575bc060ce16167d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:19,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ab575bc060ce16164c 2025-12-17 23:30:19,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ab575bc060ce16164c HTTP/1.1" 502 150 2025-12-17 23:30:19,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ab575bc060ce16164c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:19,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b0575bc060ce16165d 2025-12-17 23:30:19,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b0575bc060ce16165d HTTP/1.1" 502 150 2025-12-17 23:30:19,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b0575bc060ce16165d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:20,335 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b2575bc060ce161660 2025-12-17 23:30:20,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b2575bc060ce161660 HTTP/1.1" 502 150 2025-12-17 23:30:20,336 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b2575bc060ce161660'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:22,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b3575bc060ce161663 2025-12-17 23:30:22,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b3575bc060ce161663 HTTP/1.1" 502 150 2025-12-17 23:30:22,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b3575bc060ce161663'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:22,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b5575bc060ce161667 2025-12-17 23:30:22,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b5575bc060ce161667 HTTP/1.1" 502 150 2025-12-17 23:30:22,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b5575bc060ce161667'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:22,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096bb575bc060ce16167d 2025-12-17 23:30:22,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096bb575bc060ce16167d HTTP/1.1" 502 150 2025-12-17 23:30:22,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096bb575bc060ce16167d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:24,275 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6c575bc060ce1625bf 2025-12-17 23:30:24,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6c575bc060ce1625bf HTTP/1.1" 502 150 2025-12-17 23:30:24,276 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6c575bc060ce1625bf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:24,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940969d575bc060ce16162d 2025-12-17 23:30:24,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940969d575bc060ce16162d HTTP/1.1" 502 150 2025-12-17 23:30:24,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940969d575bc060ce16162d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:25,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ad575bc060ce161657 HTTP/1.1" 502 150 2025-12-17 23:30:25,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ad575bc060ce161657'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:26,406 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a0575bc060ce161633 2025-12-17 23:30:26,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a0575bc060ce161633 HTTP/1.1" 502 150 2025-12-17 23:30:26,407 - lava_callback - ERROR - Error processing callback for node 694096a0575bc060ce161633: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a0575bc060ce161633 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a0575bc060ce161633 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a0575bc060ce161633 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:26,409 - lava_callback - INFO - Starting async job submit for node 694096b7575bc060ce161671 2025-12-17 23:30:26,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:26,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b7575bc060ce161671 HTTP/1.1" 502 150 2025-12-17 23:30:26,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b7575bc060ce161671'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:26,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b7575bc060ce161671 2025-12-17 23:30:26,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b7575bc060ce161671 HTTP/1.1" 502 150 2025-12-17 23:30:26,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b7575bc060ce161671'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43392 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:30:27,609 - lava_callback - INFO - Received callback for node 694099ff575bc060ce161e9d INFO: 172.21.0.1:43396 - "POST /node/694099ff575bc060ce161e9d HTTP/1.0" 202 Accepted 2025-12-17 23:30:27,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b0575bc060ce16165d 2025-12-17 23:30:27,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b0575bc060ce16165d HTTP/1.1" 502 150 2025-12-17 23:30:27,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b0575bc060ce16165d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:27,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a3575bc060ce161640 2025-12-17 23:30:27,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a5575bc060ce161643 2025-12-17 23:30:27,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a3575bc060ce161640 HTTP/1.1" 502 150 2025-12-17 23:30:27,906 - lava_callback - ERROR - Error processing callback for node 694096a3575bc060ce161640: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a3575bc060ce161640 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a3575bc060ce161640 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a3575bc060ce161640 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:27,908 - lava_callback - INFO - Starting async job submit for node 694096c0575bc060ce161687 2025-12-17 23:30:27,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:27,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a5575bc060ce161643 HTTP/1.1" 502 150 2025-12-17 23:30:27,909 - lava_callback - ERROR - Error processing callback for node 694096a5575bc060ce161643: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a5575bc060ce161643 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a5575bc060ce161643 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a5575bc060ce161643 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:27,912 - lava_callback - INFO - Starting async job submit for node 694096c1575bc060ce16168a 2025-12-17 23:30:27,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:27,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c0575bc060ce161687 HTTP/1.1" 502 150 2025-12-17 23:30:27,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c0575bc060ce161687'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:27,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c0575bc060ce161687 2025-12-17 23:30:27,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c0575bc060ce161687 HTTP/1.1" 502 150 2025-12-17 23:30:27,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c0575bc060ce161687'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:28,091 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096af575bc060ce16165a 2025-12-17 23:30:28,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096af575bc060ce16165a HTTP/1.1" 502 150 2025-12-17 23:30:28,092 - lava_callback - ERROR - Error processing callback for node 694096af575bc060ce16165a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096af575bc060ce16165a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096af575bc060ce16165a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096af575bc060ce16165a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:28,094 - lava_callback - INFO - Starting async job submit for node 694096c2575bc060ce16168d 2025-12-17 23:30:28,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:28,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c2575bc060ce16168d HTTP/1.1" 502 150 2025-12-17 23:30:28,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c2575bc060ce16168d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:28,101 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c2575bc060ce16168d 2025-12-17 23:30:28,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c2575bc060ce16168d HTTP/1.1" 502 150 2025-12-17 23:30:28,102 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c2575bc060ce16168d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:28,246 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a7575bc060ce161646 2025-12-17 23:30:28,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a7575bc060ce161646 HTTP/1.1" 502 150 2025-12-17 23:30:28,248 - lava_callback - ERROR - Error processing callback for node 694096a7575bc060ce161646: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a7575bc060ce161646 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a7575bc060ce161646 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a7575bc060ce161646 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:28,249 - lava_callback - INFO - Starting async job submit for node 694096c4575bc060ce161690 2025-12-17 23:30:28,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:28,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c4575bc060ce161690 HTTP/1.1" 502 150 2025-12-17 23:30:28,256 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c4575bc060ce161690'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:28,256 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c4575bc060ce161690 2025-12-17 23:30:28,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c4575bc060ce161690 HTTP/1.1" 502 150 2025-12-17 23:30:28,257 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c4575bc060ce161690'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:28,274 - lava_callback - INFO - Received callback for node 69409a01575bc060ce161eac INFO: 172.21.0.1:43410 - "POST /node/69409a01575bc060ce161eac HTTP/1.0" 202 Accepted 2025-12-17 23:30:28,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b2575bc060ce161660 2025-12-17 23:30:28,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b2575bc060ce161660 HTTP/1.1" 502 150 2025-12-17 23:30:28,338 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b2575bc060ce161660'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:28,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b7575bc060ce161671 2025-12-17 23:30:28,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b7575bc060ce161671 HTTP/1.1" 502 150 2025-12-17 23:30:28,437 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b7575bc060ce161671'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:28,951 - lava_callback - INFO - Received callback for node 69409a06575bc060ce161eb2 INFO: 172.21.0.1:43424 - "POST /node/69409a06575bc060ce161eb2 HTTP/1.0" 202 Accepted 2025-12-17 23:30:29,626 - lava_callback - INFO - Received callback for node 69409a08575bc060ce161eb9 INFO: 172.21.0.1:43426 - "POST /node/69409a08575bc060ce161eb9 HTTP/1.0" 202 Accepted 2025-12-17 23:30:29,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ad575bc060ce161657 2025-12-17 23:30:29,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ad575bc060ce161657 HTTP/1.1" 502 150 2025-12-17 23:30:29,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ad575bc060ce161657'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:29,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c0575bc060ce161687 2025-12-17 23:30:29,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c0575bc060ce161687 HTTP/1.1" 502 150 2025-12-17 23:30:29,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c0575bc060ce161687'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:30,102 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c2575bc060ce16168d 2025-12-17 23:30:30,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c2575bc060ce16168d HTTP/1.1" 502 150 2025-12-17 23:30:30,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c2575bc060ce16168d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:30,257 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c4575bc060ce161690 2025-12-17 23:30:30,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c4575bc060ce161690 HTTP/1.1" 502 150 2025-12-17 23:30:30,258 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c4575bc060ce161690'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:30,318 - lava_callback - INFO - Received callback for node 69409a0a575bc060ce161ebc INFO: 172.21.0.1:39528 - "POST /node/69409a0a575bc060ce161ebc HTTP/1.0" 202 Accepted 2025-12-17 23:30:30,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b3575bc060ce161663 2025-12-17 23:30:30,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b3575bc060ce161663 HTTP/1.1" 502 150 2025-12-17 23:30:30,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b3575bc060ce161663'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:30,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b5575bc060ce161667 2025-12-17 23:30:30,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b5575bc060ce161667 HTTP/1.1" 502 150 2025-12-17 23:30:30,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b5575bc060ce161667'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:30,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096bb575bc060ce16167d 2025-12-17 23:30:30,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096bb575bc060ce16167d HTTP/1.1" 502 150 2025-12-17 23:30:30,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096bb575bc060ce16167d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:31,000 - lava_callback - INFO - Received callback for node 69409a0c575bc060ce161ebf INFO: 172.21.0.1:39530 - "POST /node/69409a0c575bc060ce161ebf HTTP/1.0" 202 Accepted 2025-12-17 23:30:31,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409628575bc060ce161512 2025-12-17 23:30:31,669 - lava_callback - INFO - Received callback for node 69409a0e575bc060ce161ec2 INFO: 172.21.0.1:39536 - "POST /node/69409a0e575bc060ce161ec2 HTTP/1.0" 202 Accepted 2025-12-17 23:30:31,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096a9575bc060ce161649 2025-12-17 23:30:31,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096a9575bc060ce161649 HTTP/1.1" 502 150 2025-12-17 23:30:31,705 - lava_callback - ERROR - Error processing callback for node 694096a9575bc060ce161649: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a9575bc060ce161649 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a9575bc060ce161649 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096a9575bc060ce161649 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:31,707 - lava_callback - INFO - Starting async job submit for node 694096c5575bc060ce161693 2025-12-17 23:30:31,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:31,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c5575bc060ce161693 HTTP/1.1" 502 150 2025-12-17 23:30:31,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c5575bc060ce161693'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:31,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c5575bc060ce161693 2025-12-17 23:30:31,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c5575bc060ce161693 HTTP/1.1" 502 150 2025-12-17 23:30:31,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c5575bc060ce161693'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:32,333 - lava_callback - INFO - Received callback for node 69409a10575bc060ce161ec5 INFO: 172.21.0.1:39550 - "POST /node/69409a10575bc060ce161ec5 HTTP/1.0" 202 Accepted 2025-12-17 23:30:32,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b7575bc060ce161671 2025-12-17 23:30:32,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b7575bc060ce161671 HTTP/1.1" 502 150 2025-12-17 23:30:32,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b7575bc060ce161671'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:33,008 - lava_callback - INFO - Received callback for node 69409a12575bc060ce161ec8 INFO: 172.21.0.1:39560 - "POST /node/69409a12575bc060ce161ec8 HTTP/1.0" 202 Accepted 2025-12-17 23:30:33,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c5575bc060ce161693 2025-12-17 23:30:33,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c5575bc060ce161693 HTTP/1.1" 502 150 2025-12-17 23:30:33,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c5575bc060ce161693'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:33,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c0575bc060ce161687 2025-12-17 23:30:33,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c0575bc060ce161687 HTTP/1.1" 502 150 2025-12-17 23:30:33,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c0575bc060ce161687'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:34,103 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c2575bc060ce16168d 2025-12-17 23:30:34,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c2575bc060ce16168d HTTP/1.1" 502 150 2025-12-17 23:30:34,105 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c2575bc060ce16168d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:34,258 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c4575bc060ce161690 2025-12-17 23:30:34,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c4575bc060ce161690 HTTP/1.1" 502 150 2025-12-17 23:30:34,259 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c4575bc060ce161690'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:35,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ab575bc060ce16164c 2025-12-17 23:30:35,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ab575bc060ce16164c HTTP/1.1" 502 150 2025-12-17 23:30:35,609 - lava_callback - ERROR - Error processing callback for node 694096ab575bc060ce16164c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096ab575bc060ce16164c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096ab575bc060ce16164c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096ab575bc060ce16164c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:35,610 - lava_callback - INFO - Starting async job submit for node 694096c6575bc060ce161696 2025-12-17 23:30:35,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:35,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c6575bc060ce161696 HTTP/1.1" 502 150 2025-12-17 23:30:35,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c6575bc060ce161696'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:35,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c6575bc060ce161696 2025-12-17 23:30:35,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c6575bc060ce161696 HTTP/1.1" 502 150 2025-12-17 23:30:35,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c6575bc060ce161696'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39576 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:30:37,039 - lava_callback - INFO - Received callback for node 69409a15575bc060ce161eda INFO: 172.21.0.1:39592 - "POST /node/69409a15575bc060ce161eda HTTP/1.0" 202 Accepted 2025-12-17 23:30:37,128 - lava_callback - INFO - Received callback for node 69409d1d575bc060ce16274e INFO: 172.21.0.1:39602 - "POST /node/69409d1d575bc060ce16274e HTTP/1.0" 202 Accepted 2025-12-17 23:30:37,138 - lava_callback - INFO - Received callback for node 69409ea7575bc060ce162a5b INFO: 172.21.0.1:39608 - "POST /node/69409ea7575bc060ce162a5b HTTP/1.0" 202 Accepted 2025-12-17 23:30:37,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c6575bc060ce161696 2025-12-17 23:30:37,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c6575bc060ce161696 HTTP/1.1" 502 150 2025-12-17 23:30:37,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c6575bc060ce161696'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:37,706 - lava_callback - INFO - Received callback for node 69409a18575bc060ce161edf INFO: 172.21.0.1:39618 - "POST /node/69409a18575bc060ce161edf HTTP/1.0" 202 Accepted 2025-12-17 23:30:37,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c5575bc060ce161693 2025-12-17 23:30:37,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c5575bc060ce161693 HTTP/1.1" 502 150 2025-12-17 23:30:37,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c5575bc060ce161693'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:37,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ad575bc060ce161657 2025-12-17 23:30:37,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ad575bc060ce161657 HTTP/1.1" 502 150 2025-12-17 23:30:37,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ad575bc060ce161657'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:40,276 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6c575bc060ce1625bf 2025-12-17 23:30:40,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6c575bc060ce1625bf HTTP/1.1" 502 150 2025-12-17 23:30:40,278 - lava_callback - ERROR - Error processing callback for node 69409c6c575bc060ce1625bf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c6c575bc060ce1625bf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c6c575bc060ce1625bf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c6c575bc060ce1625bf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:40,280 - lava_callback - INFO - Starting async job submit for node 694096c7575bc060ce161699 2025-12-17 23:30:40,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:40,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c7575bc060ce161699 HTTP/1.1" 502 150 2025-12-17 23:30:40,286 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c7575bc060ce161699'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:40,286 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c7575bc060ce161699 2025-12-17 23:30:40,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c7575bc060ce161699 HTTP/1.1" 502 150 2025-12-17 23:30:40,287 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c7575bc060ce161699'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:40,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b7575bc060ce161671 2025-12-17 23:30:40,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b7575bc060ce161671 HTTP/1.1" 502 150 2025-12-17 23:30:40,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096b7575bc060ce161671'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:40,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940969d575bc060ce16162d 2025-12-17 23:30:40,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940969d575bc060ce16162d HTTP/1.1" 502 150 2025-12-17 23:30:40,491 - lava_callback - ERROR - Error processing callback for node 6940969d575bc060ce16162d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940969d575bc060ce16162d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940969d575bc060ce16162d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940969d575bc060ce16162d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:40,492 - lava_callback - INFO - Starting async job submit for node 694096be575bc060ce161684 2025-12-17 23:30:40,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:40,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096be575bc060ce161684 HTTP/1.1" 502 150 2025-12-17 23:30:40,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096be575bc060ce161684'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:40,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096be575bc060ce161684 2025-12-17 23:30:40,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096be575bc060ce161684 HTTP/1.1" 502 150 2025-12-17 23:30:40,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096be575bc060ce161684'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:41,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c6575bc060ce161696 2025-12-17 23:30:41,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c6575bc060ce161696 HTTP/1.1" 502 150 2025-12-17 23:30:41,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c6575bc060ce161696'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:41,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c0575bc060ce161687 2025-12-17 23:30:41,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c0575bc060ce161687 HTTP/1.1" 502 150 2025-12-17 23:30:41,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c0575bc060ce161687'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:42,105 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c2575bc060ce16168d 2025-12-17 23:30:42,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c2575bc060ce16168d HTTP/1.1" 502 150 2025-12-17 23:30:42,106 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c2575bc060ce16168d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:42,260 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c4575bc060ce161690 2025-12-17 23:30:42,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c4575bc060ce161690 HTTP/1.1" 502 150 2025-12-17 23:30:42,261 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c4575bc060ce161690'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:42,287 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c7575bc060ce161699 2025-12-17 23:30:42,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c7575bc060ce161699 HTTP/1.1" 502 150 2025-12-17 23:30:42,288 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c7575bc060ce161699'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:42,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096be575bc060ce161684 2025-12-17 23:30:42,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096be575bc060ce161684 HTTP/1.1" 502 150 2025-12-17 23:30:42,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096be575bc060ce161684'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:43,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b0575bc060ce16165d 2025-12-17 23:30:43,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b0575bc060ce16165d HTTP/1.1" 502 150 2025-12-17 23:30:43,643 - lava_callback - ERROR - Error processing callback for node 694096b0575bc060ce16165d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b0575bc060ce16165d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b0575bc060ce16165d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b0575bc060ce16165d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:43,645 - lava_callback - INFO - Starting async job submit for node 694096c8575bc060ce16169c 2025-12-17 23:30:43,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:43,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c8575bc060ce16169c HTTP/1.1" 502 150 2025-12-17 23:30:43,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c8575bc060ce16169c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:43,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c8575bc060ce16169c 2025-12-17 23:30:43,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c8575bc060ce16169c HTTP/1.1" 502 150 2025-12-17 23:30:43,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c8575bc060ce16169c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:44,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b2575bc060ce161660 2025-12-17 23:30:44,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b2575bc060ce161660 HTTP/1.1" 502 150 2025-12-17 23:30:44,339 - lava_callback - ERROR - Error processing callback for node 694096b2575bc060ce161660: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b2575bc060ce161660 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b2575bc060ce161660 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b2575bc060ce161660 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:44,341 - lava_callback - INFO - Starting async job submit for node 694096cd575bc060ce1616b4 2025-12-17 23:30:44,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:44,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cd575bc060ce1616b4 HTTP/1.1" 502 150 2025-12-17 23:30:44,361 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096cd575bc060ce1616b4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:44,361 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096cd575bc060ce1616b4 2025-12-17 23:30:44,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cd575bc060ce1616b4 HTTP/1.1" 502 150 2025-12-17 23:30:44,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096cd575bc060ce1616b4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:45,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c8575bc060ce16169c 2025-12-17 23:30:45,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c8575bc060ce16169c HTTP/1.1" 502 150 2025-12-17 23:30:45,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c8575bc060ce16169c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:45,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c5575bc060ce161693 2025-12-17 23:30:45,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c5575bc060ce161693 HTTP/1.1" 502 150 2025-12-17 23:30:45,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c5575bc060ce161693'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:46,288 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c7575bc060ce161699 2025-12-17 23:30:46,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c7575bc060ce161699 HTTP/1.1" 502 150 2025-12-17 23:30:46,289 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c7575bc060ce161699'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:46,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096cd575bc060ce1616b4 2025-12-17 23:30:46,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cd575bc060ce1616b4 HTTP/1.1" 502 150 2025-12-17 23:30:46,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096cd575bc060ce1616b4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:46,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b3575bc060ce161663 2025-12-17 23:30:46,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b3575bc060ce161663 HTTP/1.1" 502 150 2025-12-17 23:30:46,473 - lava_callback - ERROR - Error processing callback for node 694096b3575bc060ce161663: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b3575bc060ce161663 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b3575bc060ce161663 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b3575bc060ce161663 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:46,476 - lava_callback - INFO - Starting async job submit for node 6940990f575bc060ce161c08 2025-12-17 23:30:46,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096be575bc060ce161684 2025-12-17 23:30:46,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:46,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096be575bc060ce161684 HTTP/1.1" 502 150 2025-12-17 23:30:46,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096be575bc060ce161684'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:46,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990f575bc060ce161c08 HTTP/1.1" 502 150 2025-12-17 23:30:46,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940990f575bc060ce161c08'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:46,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940990f575bc060ce161c08 2025-12-17 23:30:46,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990f575bc060ce161c08 HTTP/1.1" 502 150 2025-12-17 23:30:46,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940990f575bc060ce161c08'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40552 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:30:46,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b5575bc060ce161667 2025-12-17 23:30:46,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b5575bc060ce161667 HTTP/1.1" 502 150 2025-12-17 23:30:46,706 - lava_callback - ERROR - Error processing callback for node 694096b5575bc060ce161667: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b5575bc060ce161667 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b5575bc060ce161667 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b5575bc060ce161667 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:46,708 - lava_callback - INFO - Starting async job submit for node 694096ca575bc060ce1616a1 2025-12-17 23:30:46,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:46,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ca575bc060ce1616a1 HTTP/1.1" 502 150 2025-12-17 23:30:46,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ca575bc060ce1616a1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:46,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ca575bc060ce1616a1 2025-12-17 23:30:46,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ca575bc060ce1616a1 HTTP/1.1" 502 150 2025-12-17 23:30:46,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ca575bc060ce1616a1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:46,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409628575bc060ce161512 HTTP/1.1" 502 150 2025-12-17 23:30:46,832 - lava_callback - ERROR - Error processing callback for node 69409628575bc060ce161512: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409628575bc060ce161512 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409628575bc060ce161512 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409628575bc060ce161512 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:46,835 - lava_callback - INFO - Starting async job submit for node 694096cb575bc060ce1616a8 2025-12-17 23:30:46,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:46,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cb575bc060ce1616a8 HTTP/1.1" 502 150 2025-12-17 23:30:46,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096cb575bc060ce1616a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:46,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096cb575bc060ce1616a8 2025-12-17 23:30:46,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cb575bc060ce1616a8 HTTP/1.1" 502 150 2025-12-17 23:30:46,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096cb575bc060ce1616a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:46,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096bb575bc060ce16167d 2025-12-17 23:30:46,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096bb575bc060ce16167d HTTP/1.1" 502 150 2025-12-17 23:30:46,971 - lava_callback - ERROR - Error processing callback for node 694096bb575bc060ce16167d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096bb575bc060ce16167d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096bb575bc060ce16167d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096bb575bc060ce16167d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:46,973 - lava_callback - INFO - Starting async job submit for node 694096dc575bc060ce1616d5 2025-12-17 23:30:46,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:46,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096dc575bc060ce1616d5 HTTP/1.1" 502 150 2025-12-17 23:30:46,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096dc575bc060ce1616d5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:46,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096dc575bc060ce1616d5 2025-12-17 23:30:46,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096dc575bc060ce1616d5 HTTP/1.1" 502 150 2025-12-17 23:30:46,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096dc575bc060ce1616d5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:47,399 - lava_callback - INFO - Received callback for node 69424dacbb27571da24b8f1b INFO: 172.21.0.1:40562 - "POST /node/69424dacbb27571da24b8f1b HTTP/1.0" 202 Accepted 2025-12-17 23:30:47,661 - lava_callback - INFO - Received callback for node 69409a1c575bc060ce161ee5 INFO: 172.21.0.1:40570 - "POST /node/69409a1c575bc060ce161ee5 HTTP/1.0" 202 Accepted 2025-12-17 23:30:48,312 - lava_callback - INFO - Received callback for node 69409a1e575bc060ce161ee8 INFO: 172.21.0.1:40574 - "POST /node/69409a1e575bc060ce161ee8 HTTP/1.0" 202 Accepted 2025-12-17 23:30:48,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940990f575bc060ce161c08 2025-12-17 23:30:48,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990f575bc060ce161c08 HTTP/1.1" 502 150 2025-12-17 23:30:48,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940990f575bc060ce161c08'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:48,532 - lava_callback - INFO - Received callback for node 69409a14575bc060ce161ed3 INFO: 172.21.0.1:40584 - "POST /node/69409a14575bc060ce161ed3 HTTP/1.0" 202 Accepted 2025-12-17 23:30:48,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ca575bc060ce1616a1 2025-12-17 23:30:48,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ca575bc060ce1616a1 HTTP/1.1" 502 150 2025-12-17 23:30:48,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ca575bc060ce1616a1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:48,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096cb575bc060ce1616a8 2025-12-17 23:30:48,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cb575bc060ce1616a8 HTTP/1.1" 502 150 2025-12-17 23:30:48,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096cb575bc060ce1616a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:48,984 - lava_callback - INFO - Received callback for node 69409a24575bc060ce161ef1 2025-12-17 23:30:48,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096dc575bc060ce1616d5 INFO: 172.21.0.1:40586 - "POST /node/69409a24575bc060ce161ef1 HTTP/1.0" 202 Accepted 2025-12-17 23:30:48,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096dc575bc060ce1616d5 HTTP/1.1" 502 150 2025-12-17 23:30:48,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096dc575bc060ce1616d5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:49,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c6575bc060ce161696 2025-12-17 23:30:49,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c6575bc060ce161696 HTTP/1.1" 502 150 2025-12-17 23:30:49,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c6575bc060ce161696'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:49,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c8575bc060ce16169c 2025-12-17 23:30:49,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c8575bc060ce16169c HTTP/1.1" 502 150 2025-12-17 23:30:49,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c8575bc060ce16169c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:50,363 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096cd575bc060ce1616b4 2025-12-17 23:30:50,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cd575bc060ce1616b4 HTTP/1.1" 502 150 2025-12-17 23:30:50,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096cd575bc060ce1616b4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:52,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940990f575bc060ce161c08 2025-12-17 23:30:52,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990f575bc060ce161c08 HTTP/1.1" 502 150 2025-12-17 23:30:52,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940990f575bc060ce161c08'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:52,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ca575bc060ce1616a1 2025-12-17 23:30:52,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ca575bc060ce1616a1 HTTP/1.1" 502 150 2025-12-17 23:30:52,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ca575bc060ce1616a1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:52,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096cb575bc060ce1616a8 2025-12-17 23:30:52,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cb575bc060ce1616a8 HTTP/1.1" 502 150 2025-12-17 23:30:52,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096cb575bc060ce1616a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:52,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096dc575bc060ce1616d5 2025-12-17 23:30:52,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096dc575bc060ce1616d5 HTTP/1.1" 502 150 2025-12-17 23:30:52,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096dc575bc060ce1616d5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:53,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ad575bc060ce161657 2025-12-17 23:30:53,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ad575bc060ce161657 HTTP/1.1" 502 150 2025-12-17 23:30:53,844 - lava_callback - ERROR - Error processing callback for node 694096ad575bc060ce161657: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096ad575bc060ce161657 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096ad575bc060ce161657 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096ad575bc060ce161657 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:53,846 - lava_callback - INFO - Starting async job submit for node 694096cc575bc060ce1616af 2025-12-17 23:30:53,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:53,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cc575bc060ce1616af HTTP/1.1" 502 150 2025-12-17 23:30:53,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096cc575bc060ce1616af'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:53,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096cc575bc060ce1616af 2025-12-17 23:30:53,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cc575bc060ce1616af HTTP/1.1" 502 150 2025-12-17 23:30:53,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096cc575bc060ce1616af'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:54,290 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c7575bc060ce161699 2025-12-17 23:30:54,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c7575bc060ce161699 HTTP/1.1" 502 150 2025-12-17 23:30:54,291 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c7575bc060ce161699'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:54,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096be575bc060ce161684 2025-12-17 23:30:54,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096be575bc060ce161684 HTTP/1.1" 502 150 2025-12-17 23:30:54,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096be575bc060ce161684'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:55,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096cc575bc060ce1616af 2025-12-17 23:30:55,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cc575bc060ce1616af HTTP/1.1" 502 150 2025-12-17 23:30:55,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096cc575bc060ce1616af'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:56,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096b7575bc060ce161671 2025-12-17 23:30:56,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096b7575bc060ce161671 HTTP/1.1" 502 150 2025-12-17 23:30:56,442 - lava_callback - ERROR - Error processing callback for node 694096b7575bc060ce161671: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b7575bc060ce161671 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b7575bc060ce161671 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096b7575bc060ce161671 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:56,443 - lava_callback - INFO - Starting async job submit for node 694096ce575bc060ce1616b7 2025-12-17 23:30:56,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:56,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ce575bc060ce1616b7 HTTP/1.1" 502 150 2025-12-17 23:30:56,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ce575bc060ce1616b7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:56,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ce575bc060ce1616b7 2025-12-17 23:30:56,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ce575bc060ce1616b7 HTTP/1.1" 502 150 2025-12-17 23:30:56,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ce575bc060ce1616b7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38884 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:30:57,053 - lava_callback - INFO - Received callback for node 69409a21575bc060ce161eeb INFO: 172.21.0.1:38890 - "POST /node/69409a21575bc060ce161eeb HTTP/1.0" 202 Accepted 2025-12-17 23:30:57,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c8575bc060ce16169c 2025-12-17 23:30:57,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c8575bc060ce16169c HTTP/1.1" 502 150 2025-12-17 23:30:57,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c8575bc060ce16169c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:57,709 - lava_callback - INFO - Received callback for node 69409a29575bc060ce161f05 INFO: 172.21.0.1:38900 - "POST /node/69409a29575bc060ce161f05 HTTP/1.0" 202 Accepted 2025-12-17 23:30:57,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c0575bc060ce161687 2025-12-17 23:30:57,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c0575bc060ce161687 HTTP/1.1" 502 150 2025-12-17 23:30:57,926 - lava_callback - ERROR - Error processing callback for node 694096c0575bc060ce161687: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c0575bc060ce161687 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c0575bc060ce161687 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c0575bc060ce161687 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:57,928 - lava_callback - INFO - Starting async job submit for node 694096d0575bc060ce1616ba 2025-12-17 23:30:57,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:57,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d0575bc060ce1616ba HTTP/1.1" 502 150 2025-12-17 23:30:57,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d0575bc060ce1616ba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:57,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d0575bc060ce1616ba 2025-12-17 23:30:57,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d0575bc060ce1616ba HTTP/1.1" 502 150 2025-12-17 23:30:57,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d0575bc060ce1616ba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:58,106 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c2575bc060ce16168d 2025-12-17 23:30:58,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c2575bc060ce16168d HTTP/1.1" 502 150 2025-12-17 23:30:58,108 - lava_callback - ERROR - Error processing callback for node 694096c2575bc060ce16168d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c2575bc060ce16168d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c2575bc060ce16168d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c2575bc060ce16168d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:58,109 - lava_callback - INFO - Starting async job submit for node 694096d1575bc060ce1616bd 2025-12-17 23:30:58,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:58,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d1575bc060ce1616bd HTTP/1.1" 502 150 2025-12-17 23:30:58,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d1575bc060ce1616bd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:58,116 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d1575bc060ce1616bd 2025-12-17 23:30:58,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d1575bc060ce1616bd HTTP/1.1" 502 150 2025-12-17 23:30:58,116 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d1575bc060ce1616bd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:58,261 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c4575bc060ce161690 2025-12-17 23:30:58,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c4575bc060ce161690 HTTP/1.1" 502 150 2025-12-17 23:30:58,262 - lava_callback - ERROR - Error processing callback for node 694096c4575bc060ce161690: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c4575bc060ce161690 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c4575bc060ce161690 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c4575bc060ce161690 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:30:58,264 - lava_callback - INFO - Starting async job submit for node 694096d2575bc060ce1616c0 2025-12-17 23:30:58,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:30:58,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d2575bc060ce1616c0 HTTP/1.1" 502 150 2025-12-17 23:30:58,271 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d2575bc060ce1616c0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:58,271 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d2575bc060ce1616c0 2025-12-17 23:30:58,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d2575bc060ce1616c0 HTTP/1.1" 502 150 2025-12-17 23:30:58,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d2575bc060ce1616c0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:58,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096cd575bc060ce1616b4 2025-12-17 23:30:58,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cd575bc060ce1616b4 HTTP/1.1" 502 150 2025-12-17 23:30:58,367 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096cd575bc060ce1616b4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:58,394 - lava_callback - INFO - Received callback for node 69409a2c575bc060ce161f08 INFO: 172.21.0.1:38904 - "POST /node/69409a2c575bc060ce161f08 HTTP/1.0" 202 Accepted 2025-12-17 23:30:58,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ce575bc060ce1616b7 2025-12-17 23:30:58,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ce575bc060ce1616b7 HTTP/1.1" 502 150 2025-12-17 23:30:58,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ce575bc060ce1616b7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:58,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c1575bc060ce16168a HTTP/1.1" 502 150 2025-12-17 23:30:58,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c1575bc060ce16168a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:58,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c1575bc060ce16168a 2025-12-17 23:30:59,079 - lava_callback - INFO - Received callback for node 69409a2e575bc060ce161f0b INFO: 172.21.0.1:38914 - "POST /node/69409a2e575bc060ce161f0b HTTP/1.0" 202 Accepted 2025-12-17 23:30:59,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096cc575bc060ce1616af 2025-12-17 23:30:59,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cc575bc060ce1616af HTTP/1.1" 502 150 2025-12-17 23:30:59,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096cc575bc060ce1616af'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:30:59,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d0575bc060ce1616ba 2025-12-17 23:30:59,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d0575bc060ce1616ba HTTP/1.1" 502 150 2025-12-17 23:30:59,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d0575bc060ce1616ba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:00,116 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d1575bc060ce1616bd 2025-12-17 23:31:00,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d1575bc060ce1616bd HTTP/1.1" 502 150 2025-12-17 23:31:00,118 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d1575bc060ce1616bd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:00,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d2575bc060ce1616c0 2025-12-17 23:31:00,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d2575bc060ce1616c0 HTTP/1.1" 502 150 2025-12-17 23:31:00,273 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d2575bc060ce1616c0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:00,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940990f575bc060ce161c08 2025-12-17 23:31:00,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990f575bc060ce161c08 HTTP/1.1" 502 150 2025-12-17 23:31:00,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940990f575bc060ce161c08'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:00,611 - lava_callback - INFO - Received callback for node 6942681e1b61f1c6005f5088 INFO: 172.21.0.1:43884 - "POST /node/6942681e1b61f1c6005f5088 HTTP/1.0" 202 Accepted 2025-12-17 23:31:00,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ca575bc060ce1616a1 2025-12-17 23:31:00,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ca575bc060ce1616a1 HTTP/1.1" 502 150 2025-12-17 23:31:00,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ca575bc060ce1616a1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:00,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096cb575bc060ce1616a8 2025-12-17 23:31:00,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cb575bc060ce1616a8 HTTP/1.1" 502 150 2025-12-17 23:31:00,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096cb575bc060ce1616a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:00,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096dc575bc060ce1616d5 2025-12-17 23:31:00,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096dc575bc060ce1616d5 HTTP/1.1" 502 150 2025-12-17 23:31:00,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096dc575bc060ce1616d5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:01,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c5575bc060ce161693 2025-12-17 23:31:01,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c5575bc060ce161693 HTTP/1.1" 502 150 2025-12-17 23:31:01,733 - lava_callback - ERROR - Error processing callback for node 694096c5575bc060ce161693: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c5575bc060ce161693 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c5575bc060ce161693 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c5575bc060ce161693 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:01,734 - lava_callback - INFO - Starting async job submit for node 694096d3575bc060ce1616c3 2025-12-17 23:31:01,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:01,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d3575bc060ce1616c3 HTTP/1.1" 502 150 2025-12-17 23:31:01,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d3575bc060ce1616c3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:01,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d3575bc060ce1616c3 2025-12-17 23:31:01,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d3575bc060ce1616c3 HTTP/1.1" 502 150 2025-12-17 23:31:01,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d3575bc060ce1616c3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:02,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ce575bc060ce1616b7 2025-12-17 23:31:02,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ce575bc060ce1616b7 HTTP/1.1" 502 150 2025-12-17 23:31:02,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ce575bc060ce1616b7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43890 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:31:03,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d3575bc060ce1616c3 2025-12-17 23:31:03,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d3575bc060ce1616c3 HTTP/1.1" 502 150 2025-12-17 23:31:03,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d3575bc060ce1616c3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:03,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d0575bc060ce1616ba 2025-12-17 23:31:03,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d0575bc060ce1616ba HTTP/1.1" 502 150 2025-12-17 23:31:03,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d0575bc060ce1616ba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:04,118 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d1575bc060ce1616bd 2025-12-17 23:31:04,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d1575bc060ce1616bd HTTP/1.1" 502 150 2025-12-17 23:31:04,119 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d1575bc060ce1616bd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:04,184 - lava_callback - INFO - Received callback for node 69425180bb27571da24b95b2 INFO: 172.21.0.1:43894 - "POST /node/69425180bb27571da24b95b2 HTTP/1.0" 202 Accepted 2025-12-17 23:31:04,273 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d2575bc060ce1616c0 2025-12-17 23:31:04,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d2575bc060ce1616c0 HTTP/1.1" 502 150 2025-12-17 23:31:04,274 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d2575bc060ce1616c0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:04,939 - lava_callback - INFO - Received callback for node 69409c3c575bc060ce162544 INFO: 172.21.0.1:43910 - "POST /node/69409c3c575bc060ce162544 HTTP/1.0" 202 Accepted 2025-12-17 23:31:05,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c6575bc060ce161696 2025-12-17 23:31:05,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c6575bc060ce161696 HTTP/1.1" 502 150 2025-12-17 23:31:05,628 - lava_callback - ERROR - Error processing callback for node 694096c6575bc060ce161696: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c6575bc060ce161696 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c6575bc060ce161696 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c6575bc060ce161696 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:05,630 - lava_callback - INFO - Starting async job submit for node 694096d5575bc060ce1616c6 2025-12-17 23:31:05,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:05,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d5575bc060ce1616c6 HTTP/1.1" 502 150 2025-12-17 23:31:05,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d5575bc060ce1616c6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:05,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d5575bc060ce1616c6 2025-12-17 23:31:05,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d5575bc060ce1616c6 HTTP/1.1" 502 150 2025-12-17 23:31:05,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d5575bc060ce1616c6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43922 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:31:07,622 - lava_callback - INFO - Received callback for node 69409a27575bc060ce161efa INFO: 172.21.0.1:43928 - "POST /node/69409a27575bc060ce161efa HTTP/1.0" 202 Accepted 2025-12-17 23:31:07,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d5575bc060ce1616c6 2025-12-17 23:31:07,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d5575bc060ce1616c6 HTTP/1.1" 502 150 2025-12-17 23:31:07,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d5575bc060ce1616c6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:07,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d3575bc060ce1616c3 2025-12-17 23:31:07,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d3575bc060ce1616c3 HTTP/1.1" 502 150 2025-12-17 23:31:07,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d3575bc060ce1616c3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:07,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096cc575bc060ce1616af 2025-12-17 23:31:07,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cc575bc060ce1616af HTTP/1.1" 502 150 2025-12-17 23:31:07,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096cc575bc060ce1616af'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:08,288 - lava_callback - INFO - Received callback for node 69409a31575bc060ce161f11 INFO: 172.21.0.1:43932 - "POST /node/69409a31575bc060ce161f11 HTTP/1.0" 202 Accepted 2025-12-17 23:31:08,958 - lava_callback - INFO - Received callback for node 69409a30575bc060ce161f0e INFO: 172.21.0.1:43936 - "POST /node/69409a30575bc060ce161f0e HTTP/1.0" 202 Accepted 2025-12-17 23:31:10,291 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c7575bc060ce161699 2025-12-17 23:31:10,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c7575bc060ce161699 HTTP/1.1" 502 150 2025-12-17 23:31:10,292 - lava_callback - ERROR - Error processing callback for node 694096c7575bc060ce161699: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c7575bc060ce161699 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c7575bc060ce161699 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c7575bc060ce161699 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:10,294 - lava_callback - INFO - Starting async job submit for node 694096d7575bc060ce1616c9 2025-12-17 23:31:10,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:10,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d7575bc060ce1616c9 HTTP/1.1" 502 150 2025-12-17 23:31:10,323 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d7575bc060ce1616c9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:10,323 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d7575bc060ce1616c9 2025-12-17 23:31:10,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d7575bc060ce1616c9 HTTP/1.1" 502 150 2025-12-17 23:31:10,323 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d7575bc060ce1616c9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:10,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ce575bc060ce1616b7 2025-12-17 23:31:10,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ce575bc060ce1616b7 HTTP/1.1" 502 150 2025-12-17 23:31:10,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ce575bc060ce1616b7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:10,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096be575bc060ce161684 2025-12-17 23:31:10,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096be575bc060ce161684 HTTP/1.1" 502 150 2025-12-17 23:31:10,512 - lava_callback - ERROR - Error processing callback for node 694096be575bc060ce161684: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096be575bc060ce161684 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096be575bc060ce161684 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096be575bc060ce161684 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:10,514 - lava_callback - INFO - Starting async job submit for node 694096d8575bc060ce1616cc 2025-12-17 23:31:10,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:10,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d8575bc060ce1616cc HTTP/1.1" 502 150 2025-12-17 23:31:10,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d8575bc060ce1616cc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:10,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d8575bc060ce1616cc 2025-12-17 23:31:10,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d8575bc060ce1616cc HTTP/1.1" 502 150 2025-12-17 23:31:10,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d8575bc060ce1616cc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:11,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d5575bc060ce1616c6 2025-12-17 23:31:11,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d5575bc060ce1616c6 HTTP/1.1" 502 150 2025-12-17 23:31:11,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d5575bc060ce1616c6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:11,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d0575bc060ce1616ba 2025-12-17 23:31:11,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d0575bc060ce1616ba HTTP/1.1" 502 150 2025-12-17 23:31:11,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d0575bc060ce1616ba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:12,119 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d1575bc060ce1616bd 2025-12-17 23:31:12,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d1575bc060ce1616bd HTTP/1.1" 502 150 2025-12-17 23:31:12,121 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d1575bc060ce1616bd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:12,275 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d2575bc060ce1616c0 2025-12-17 23:31:12,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d2575bc060ce1616c0 HTTP/1.1" 502 150 2025-12-17 23:31:12,276 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d2575bc060ce1616c0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:12,324 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d7575bc060ce1616c9 2025-12-17 23:31:12,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d7575bc060ce1616c9 HTTP/1.1" 502 150 2025-12-17 23:31:12,325 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d7575bc060ce1616c9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:12,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d8575bc060ce1616cc 2025-12-17 23:31:12,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d8575bc060ce1616cc HTTP/1.1" 502 150 2025-12-17 23:31:12,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d8575bc060ce1616cc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:13,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c8575bc060ce16169c 2025-12-17 23:31:14,367 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096cd575bc060ce1616b4 2025-12-17 23:31:14,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cd575bc060ce1616b4 HTTP/1.1" 502 150 2025-12-17 23:31:14,368 - lava_callback - ERROR - Error processing callback for node 694096cd575bc060ce1616b4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096cd575bc060ce1616b4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096cd575bc060ce1616b4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096cd575bc060ce1616b4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:14,370 - lava_callback - INFO - Starting async job submit for node 694096da575bc060ce1616cf 2025-12-17 23:31:14,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:14,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096da575bc060ce1616cf HTTP/1.1" 502 150 2025-12-17 23:31:14,389 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096da575bc060ce1616cf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:14,389 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096da575bc060ce1616cf 2025-12-17 23:31:14,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096da575bc060ce1616cf HTTP/1.1" 502 150 2025-12-17 23:31:14,390 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096da575bc060ce1616cf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52116 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:31:15,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d3575bc060ce1616c3 2025-12-17 23:31:15,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d3575bc060ce1616c3 HTTP/1.1" 502 150 2025-12-17 23:31:15,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d3575bc060ce1616c3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:16,325 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d7575bc060ce1616c9 2025-12-17 23:31:16,390 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096da575bc060ce1616cf 2025-12-17 23:31:16,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096da575bc060ce1616cf HTTP/1.1" 502 150 2025-12-17 23:31:16,391 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096da575bc060ce1616cf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:16,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940990f575bc060ce161c08 2025-12-17 23:31:16,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990f575bc060ce161c08 HTTP/1.1" 502 150 2025-12-17 23:31:16,522 - lava_callback - ERROR - Error processing callback for node 6940990f575bc060ce161c08: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940990f575bc060ce161c08 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940990f575bc060ce161c08 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940990f575bc060ce161c08 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:16,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d8575bc060ce1616cc 2025-12-17 23:31:16,524 - lava_callback - INFO - Starting async job submit for node 694096db575bc060ce1616d2 2025-12-17 23:31:16,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:16,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d8575bc060ce1616cc HTTP/1.1" 502 150 2025-12-17 23:31:16,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d8575bc060ce1616cc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:16,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096db575bc060ce1616d2 HTTP/1.1" 502 150 2025-12-17 23:31:16,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096db575bc060ce1616d2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:16,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096db575bc060ce1616d2 2025-12-17 23:31:16,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096db575bc060ce1616d2 HTTP/1.1" 502 150 2025-12-17 23:31:16,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096db575bc060ce1616d2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52124 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:31:16,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ca575bc060ce1616a1 2025-12-17 23:31:16,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ca575bc060ce1616a1 HTTP/1.1" 502 150 2025-12-17 23:31:16,726 - lava_callback - ERROR - Error processing callback for node 694096ca575bc060ce1616a1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096ca575bc060ce1616a1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096ca575bc060ce1616a1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096ca575bc060ce1616a1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:16,728 - lava_callback - INFO - Starting async job submit for node 694096de575bc060ce1616da 2025-12-17 23:31:16,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:16,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096de575bc060ce1616da HTTP/1.1" 502 150 2025-12-17 23:31:16,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096de575bc060ce1616da'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:16,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096de575bc060ce1616da 2025-12-17 23:31:16,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096de575bc060ce1616da HTTP/1.1" 502 150 2025-12-17 23:31:16,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096de575bc060ce1616da'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:16,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096cb575bc060ce1616a8 2025-12-17 23:31:16,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cb575bc060ce1616a8 HTTP/1.1" 502 150 2025-12-17 23:31:16,852 - lava_callback - ERROR - Error processing callback for node 694096cb575bc060ce1616a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096cb575bc060ce1616a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096cb575bc060ce1616a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096cb575bc060ce1616a8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:16,854 - lava_callback - INFO - Starting async job submit for node 694096df575bc060ce1616e1 2025-12-17 23:31:16,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:16,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096df575bc060ce1616e1 HTTP/1.1" 502 150 2025-12-17 23:31:16,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096df575bc060ce1616e1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:16,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096df575bc060ce1616e1 2025-12-17 23:31:16,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096df575bc060ce1616e1 HTTP/1.1" 502 150 2025-12-17 23:31:16,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096df575bc060ce1616e1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:16,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096dc575bc060ce1616d5 2025-12-17 23:31:16,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096dc575bc060ce1616d5 HTTP/1.1" 502 150 2025-12-17 23:31:16,996 - lava_callback - ERROR - Error processing callback for node 694096dc575bc060ce1616d5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096dc575bc060ce1616d5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096dc575bc060ce1616d5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096dc575bc060ce1616d5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:16,998 - lava_callback - INFO - Starting async job submit for node 694096e2575bc060ce1616eb 2025-12-17 23:31:16,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:17,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e2575bc060ce1616eb HTTP/1.1" 502 150 2025-12-17 23:31:17,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e2575bc060ce1616eb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:17,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e2575bc060ce1616eb 2025-12-17 23:31:17,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e2575bc060ce1616eb HTTP/1.1" 502 150 2025-12-17 23:31:17,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e2575bc060ce1616eb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:17,024 - lava_callback - INFO - Received callback for node 69409a33575bc060ce161f16 INFO: 172.21.0.1:52138 - "POST /node/69409a33575bc060ce161f16 HTTP/1.0" 202 Accepted 2025-12-17 23:31:17,688 - lava_callback - INFO - Received callback for node 69409a35575bc060ce161f19 INFO: 172.21.0.1:52150 - "POST /node/69409a35575bc060ce161f19 HTTP/1.0" 202 Accepted 2025-12-17 23:31:18,390 - lava_callback - INFO - Received callback for node 69409a37575bc060ce161f1c INFO: 172.21.0.1:52154 - "POST /node/69409a37575bc060ce161f1c HTTP/1.0" 202 Accepted 2025-12-17 23:31:18,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096db575bc060ce1616d2 2025-12-17 23:31:18,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096db575bc060ce1616d2 HTTP/1.1" 502 150 2025-12-17 23:31:18,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096db575bc060ce1616d2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:18,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096de575bc060ce1616da 2025-12-17 23:31:18,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096de575bc060ce1616da HTTP/1.1" 502 150 2025-12-17 23:31:18,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096de575bc060ce1616da'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:18,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096df575bc060ce1616e1 2025-12-17 23:31:18,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096df575bc060ce1616e1 HTTP/1.1" 502 150 2025-12-17 23:31:18,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096df575bc060ce1616e1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:19,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e2575bc060ce1616eb 2025-12-17 23:31:19,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e2575bc060ce1616eb HTTP/1.1" 502 150 2025-12-17 23:31:19,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e2575bc060ce1616eb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:19,053 - lava_callback - INFO - Received callback for node 69409a39575bc060ce161f1f INFO: 172.21.0.1:52170 - "POST /node/69409a39575bc060ce161f1f HTTP/1.0" 202 Accepted 2025-12-17 23:31:19,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d5575bc060ce1616c6 2025-12-17 23:31:19,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d5575bc060ce1616c6 HTTP/1.1" 502 150 2025-12-17 23:31:19,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d5575bc060ce1616c6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:19,724 - lava_callback - INFO - Received callback for node 69409a3e575bc060ce161f31 INFO: 172.21.0.1:50536 - "POST /node/69409a3e575bc060ce161f31 HTTP/1.0" 202 Accepted 2025-12-17 23:31:20,391 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096da575bc060ce1616cf 2025-12-17 23:31:20,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096da575bc060ce1616cf HTTP/1.1" 502 150 2025-12-17 23:31:20,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096da575bc060ce1616cf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:20,393 - lava_callback - INFO - Received callback for node 69409a42575bc060ce161f3b INFO: 172.21.0.1:50546 - "POST /node/69409a42575bc060ce161f3b HTTP/1.0" 202 Accepted 2025-12-17 23:31:20,490 - lava_callback - INFO - Received callback for node 69426c3e1b61f1c6005f5f75 INFO: 172.21.0.1:50554 - "POST /node/69426c3e1b61f1c6005f5f75 HTTP/1.0" 202 Accepted 2025-12-17 23:31:22,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096db575bc060ce1616d2 2025-12-17 23:31:22,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096db575bc060ce1616d2 HTTP/1.1" 502 150 2025-12-17 23:31:22,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096db575bc060ce1616d2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:22,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096de575bc060ce1616da 2025-12-17 23:31:22,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096de575bc060ce1616da HTTP/1.1" 502 150 2025-12-17 23:31:22,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096de575bc060ce1616da'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:22,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096df575bc060ce1616e1 2025-12-17 23:31:22,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096df575bc060ce1616e1 HTTP/1.1" 502 150 2025-12-17 23:31:22,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096df575bc060ce1616e1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:23,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e2575bc060ce1616eb 2025-12-17 23:31:23,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e2575bc060ce1616eb HTTP/1.1" 502 150 2025-12-17 23:31:23,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e2575bc060ce1616eb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:23,314 - lava_callback - INFO - Received callback for node 6940a073575bc060ce162d34 INFO: 172.21.0.1:50556 - "POST /node/6940a073575bc060ce162d34 HTTP/1.0" 202 Accepted 2025-12-17 23:31:23,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096cc575bc060ce1616af 2025-12-17 23:31:23,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096cc575bc060ce1616af HTTP/1.1" 502 150 2025-12-17 23:31:23,859 - lava_callback - ERROR - Error processing callback for node 694096cc575bc060ce1616af: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096cc575bc060ce1616af (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096cc575bc060ce1616af (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096cc575bc060ce1616af (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:23,861 - lava_callback - INFO - Starting async job submit for node 694096e0575bc060ce1616e8 2025-12-17 23:31:23,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:23,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e0575bc060ce1616e8 HTTP/1.1" 502 150 2025-12-17 23:31:23,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e0575bc060ce1616e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:23,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e0575bc060ce1616e8 2025-12-17 23:31:23,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e0575bc060ce1616e8 HTTP/1.1" 502 150 2025-12-17 23:31:23,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e0575bc060ce1616e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:24,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d8575bc060ce1616cc 2025-12-17 23:31:24,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d8575bc060ce1616cc HTTP/1.1" 502 150 2025-12-17 23:31:24,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d8575bc060ce1616cc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:25,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e0575bc060ce1616e8 2025-12-17 23:31:26,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ce575bc060ce1616b7 2025-12-17 23:31:26,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ce575bc060ce1616b7 HTTP/1.1" 502 150 2025-12-17 23:31:26,462 - lava_callback - ERROR - Error processing callback for node 694096ce575bc060ce1616b7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096ce575bc060ce1616b7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096ce575bc060ce1616b7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096ce575bc060ce1616b7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:26,465 - lava_callback - INFO - Starting async job submit for node 694096e4575bc060ce1616f3 2025-12-17 23:31:26,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:26,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e4575bc060ce1616f3 HTTP/1.1" 502 150 2025-12-17 23:31:26,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e4575bc060ce1616f3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:26,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e4575bc060ce1616f3 2025-12-17 23:31:26,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e4575bc060ce1616f3 HTTP/1.1" 502 150 2025-12-17 23:31:26,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e4575bc060ce1616f3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50572 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:31:27,675 - lava_callback - INFO - Received callback for node 69409a3b575bc060ce161f24 INFO: 172.21.0.1:50580 - "POST /node/69409a3b575bc060ce161f24 HTTP/1.0" 202 Accepted 2025-12-17 23:31:27,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d0575bc060ce1616ba 2025-12-17 23:31:27,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d0575bc060ce1616ba HTTP/1.1" 502 150 2025-12-17 23:31:27,943 - lava_callback - ERROR - Error processing callback for node 694096d0575bc060ce1616ba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d0575bc060ce1616ba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d0575bc060ce1616ba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d0575bc060ce1616ba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:27,945 - lava_callback - INFO - Starting async job submit for node 694121ed459ff5fc64d2b7f1 2025-12-17 23:31:27,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:27,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694121ed459ff5fc64d2b7f1 HTTP/1.1" 502 150 2025-12-17 23:31:27,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694121ed459ff5fc64d2b7f1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:27,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694121ed459ff5fc64d2b7f1 2025-12-17 23:31:27,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694121ed459ff5fc64d2b7f1 HTTP/1.1" 502 150 2025-12-17 23:31:27,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694121ed459ff5fc64d2b7f1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:28,121 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d1575bc060ce1616bd 2025-12-17 23:31:28,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d1575bc060ce1616bd HTTP/1.1" 502 150 2025-12-17 23:31:28,122 - lava_callback - ERROR - Error processing callback for node 694096d1575bc060ce1616bd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d1575bc060ce1616bd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d1575bc060ce1616bd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d1575bc060ce1616bd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:28,124 - lava_callback - INFO - Starting async job submit for node 694096e5575bc060ce1616f6 2025-12-17 23:31:28,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:28,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e5575bc060ce1616f6 HTTP/1.1" 502 150 2025-12-17 23:31:28,153 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e5575bc060ce1616f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:28,153 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e5575bc060ce1616f6 2025-12-17 23:31:28,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e5575bc060ce1616f6 HTTP/1.1" 502 150 2025-12-17 23:31:28,154 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e5575bc060ce1616f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:28,276 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d2575bc060ce1616c0 2025-12-17 23:31:28,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d2575bc060ce1616c0 HTTP/1.1" 502 150 2025-12-17 23:31:28,281 - lava_callback - ERROR - Error processing callback for node 694096d2575bc060ce1616c0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d2575bc060ce1616c0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d2575bc060ce1616c0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d2575bc060ce1616c0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:28,283 - lava_callback - INFO - Starting async job submit for node 694096e7575bc060ce1616f9 2025-12-17 23:31:28,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:28,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e7575bc060ce1616f9 HTTP/1.1" 502 150 2025-12-17 23:31:28,290 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e7575bc060ce1616f9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:28,290 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e7575bc060ce1616f9 2025-12-17 23:31:28,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e7575bc060ce1616f9 HTTP/1.1" 502 150 2025-12-17 23:31:28,290 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e7575bc060ce1616f9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:28,337 - lava_callback - INFO - Received callback for node 69409a40575bc060ce161f38 INFO: 172.21.0.1:50596 - "POST /node/69409a40575bc060ce161f38 HTTP/1.0" 202 Accepted 2025-12-17 23:31:28,393 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096da575bc060ce1616cf 2025-12-17 23:31:28,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096da575bc060ce1616cf HTTP/1.1" 502 150 2025-12-17 23:31:28,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096da575bc060ce1616cf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:28,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e4575bc060ce1616f3 2025-12-17 23:31:28,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e4575bc060ce1616f3 HTTP/1.1" 502 150 2025-12-17 23:31:28,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e4575bc060ce1616f3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:28,990 - lava_callback - INFO - Received callback for node 69409a46575bc060ce161f41 INFO: 172.21.0.1:50604 - "POST /node/69409a46575bc060ce161f41 HTTP/1.0" 202 Accepted 2025-12-17 23:31:29,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c8575bc060ce16169c HTTP/1.1" 502 150 2025-12-17 23:31:29,072 - lava_callback - ERROR - Error processing callback for node 694096c8575bc060ce16169c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c8575bc060ce16169c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c8575bc060ce16169c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c8575bc060ce16169c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:29,074 - lava_callback - INFO - Starting async job submit for node 694096ed575bc060ce161704 2025-12-17 23:31:29,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:29,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ed575bc060ce161704 HTTP/1.1" 502 150 2025-12-17 23:31:29,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ed575bc060ce161704'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:29,101 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ed575bc060ce161704 2025-12-17 23:31:29,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ed575bc060ce161704 HTTP/1.1" 502 150 2025-12-17 23:31:29,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ed575bc060ce161704'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:29,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c1575bc060ce16168a HTTP/1.1" 502 150 2025-12-17 23:31:29,328 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c1575bc060ce16168a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:29,654 - lava_callback - INFO - Received callback for node 69409a48575bc060ce161f44 INFO: 172.21.0.1:50610 - "POST /node/69409a48575bc060ce161f44 HTTP/1.0" 202 Accepted 2025-12-17 23:31:29,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694121ed459ff5fc64d2b7f1 2025-12-17 23:31:30,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694121ed459ff5fc64d2b7f1 HTTP/1.1" 502 150 2025-12-17 23:31:30,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694121ed459ff5fc64d2b7f1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:30,154 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e5575bc060ce1616f6 2025-12-17 23:31:30,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e5575bc060ce1616f6 HTTP/1.1" 502 150 2025-12-17 23:31:30,155 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e5575bc060ce1616f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:30,291 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e7575bc060ce1616f9 2025-12-17 23:31:30,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e7575bc060ce1616f9 HTTP/1.1" 502 150 2025-12-17 23:31:30,294 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e7575bc060ce1616f9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:30,344 - lava_callback - INFO - Received callback for node 69409a4b575bc060ce161f47 INFO: 172.21.0.1:34618 - "POST /node/69409a4b575bc060ce161f47 HTTP/1.0" 202 Accepted 2025-12-17 23:31:30,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096db575bc060ce1616d2 2025-12-17 23:31:30,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096db575bc060ce1616d2 HTTP/1.1" 502 150 2025-12-17 23:31:30,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096db575bc060ce1616d2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:30,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096de575bc060ce1616da 2025-12-17 23:31:30,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096de575bc060ce1616da HTTP/1.1" 502 150 2025-12-17 23:31:30,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096de575bc060ce1616da'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:30,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096df575bc060ce1616e1 2025-12-17 23:31:30,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096df575bc060ce1616e1 HTTP/1.1" 502 150 2025-12-17 23:31:30,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096df575bc060ce1616e1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:31,000 - lava_callback - INFO - Received callback for node 69409a4d575bc060ce161f4a INFO: 172.21.0.1:34628 - "POST /node/69409a4d575bc060ce161f4a HTTP/1.0" 202 Accepted 2025-12-17 23:31:31,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e2575bc060ce1616eb 2025-12-17 23:31:31,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e2575bc060ce1616eb HTTP/1.1" 502 150 2025-12-17 23:31:31,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e2575bc060ce1616eb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:31,102 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ed575bc060ce161704 2025-12-17 23:31:31,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ed575bc060ce161704 HTTP/1.1" 502 150 2025-12-17 23:31:31,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ed575bc060ce161704'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:31,328 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c1575bc060ce16168a 2025-12-17 23:31:31,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d7575bc060ce1616c9 HTTP/1.1" 502 150 2025-12-17 23:31:31,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d7575bc060ce1616c9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:31,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d3575bc060ce1616c3 2025-12-17 23:31:31,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d3575bc060ce1616c3 HTTP/1.1" 502 150 2025-12-17 23:31:31,754 - lava_callback - ERROR - Error processing callback for node 694096d3575bc060ce1616c3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d3575bc060ce1616c3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d3575bc060ce1616c3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d3575bc060ce1616c3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:31,756 - lava_callback - INFO - Starting async job submit for node 694096e8575bc060ce1616fe 2025-12-17 23:31:31,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:31,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e8575bc060ce1616fe HTTP/1.1" 502 150 2025-12-17 23:31:31,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e8575bc060ce1616fe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:31,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e8575bc060ce1616fe 2025-12-17 23:31:31,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e8575bc060ce1616fe HTTP/1.1" 502 150 2025-12-17 23:31:31,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e8575bc060ce1616fe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:32,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e4575bc060ce1616f3 2025-12-17 23:31:32,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e4575bc060ce1616f3 HTTP/1.1" 502 150 2025-12-17 23:31:32,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e4575bc060ce1616f3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:33,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e8575bc060ce1616fe 2025-12-17 23:31:33,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e8575bc060ce1616fe HTTP/1.1" 502 150 2025-12-17 23:31:33,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e8575bc060ce1616fe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:34,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694121ed459ff5fc64d2b7f1 2025-12-17 23:31:34,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694121ed459ff5fc64d2b7f1 HTTP/1.1" 502 150 2025-12-17 23:31:34,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694121ed459ff5fc64d2b7f1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:34,155 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e5575bc060ce1616f6 2025-12-17 23:31:34,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e5575bc060ce1616f6 HTTP/1.1" 502 150 2025-12-17 23:31:34,157 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e5575bc060ce1616f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:34,294 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e7575bc060ce1616f9 2025-12-17 23:31:34,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e7575bc060ce1616f9 HTTP/1.1" 502 150 2025-12-17 23:31:34,295 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e7575bc060ce1616f9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:35,103 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ed575bc060ce161704 2025-12-17 23:31:35,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ed575bc060ce161704 HTTP/1.1" 502 150 2025-12-17 23:31:35,104 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ed575bc060ce161704'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:35,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d5575bc060ce1616c6 2025-12-17 23:31:35,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d5575bc060ce1616c6 HTTP/1.1" 502 150 2025-12-17 23:31:35,668 - lava_callback - ERROR - Error processing callback for node 694096d5575bc060ce1616c6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d5575bc060ce1616c6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d5575bc060ce1616c6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d5575bc060ce1616c6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:35,670 - lava_callback - INFO - Starting async job submit for node 694096eb575bc060ce161701 2025-12-17 23:31:35,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:35,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096eb575bc060ce161701 HTTP/1.1" 502 150 2025-12-17 23:31:35,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096eb575bc060ce161701'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:35,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096eb575bc060ce161701 2025-12-17 23:31:35,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096eb575bc060ce161701 HTTP/1.1" 502 150 2025-12-17 23:31:35,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096eb575bc060ce161701'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34644 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:31:37,081 - lava_callback - INFO - Received callback for node 69409a44575bc060ce161f3e INFO: 172.21.0.1:34648 - "POST /node/69409a44575bc060ce161f3e HTTP/1.0" 202 Accepted 2025-12-17 23:31:37,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096eb575bc060ce161701 2025-12-17 23:31:37,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096eb575bc060ce161701 HTTP/1.1" 502 150 2025-12-17 23:31:37,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096eb575bc060ce161701'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:37,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e8575bc060ce1616fe 2025-12-17 23:31:37,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e8575bc060ce1616fe HTTP/1.1" 502 150 2025-12-17 23:31:37,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e8575bc060ce1616fe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:39,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d7575bc060ce1616c9 2025-12-17 23:31:39,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d7575bc060ce1616c9 HTTP/1.1" 502 150 2025-12-17 23:31:39,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096d7575bc060ce1616c9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:40,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e4575bc060ce1616f3 2025-12-17 23:31:40,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d8575bc060ce1616cc 2025-12-17 23:31:40,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d8575bc060ce1616cc HTTP/1.1" 502 150 2025-12-17 23:31:40,529 - lava_callback - ERROR - Error processing callback for node 694096d8575bc060ce1616cc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d8575bc060ce1616cc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d8575bc060ce1616cc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d8575bc060ce1616cc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:40,532 - lava_callback - INFO - Starting async job submit for node 694096f0575bc060ce161707 2025-12-17 23:31:40,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:40,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f0575bc060ce161707 HTTP/1.1" 502 150 2025-12-17 23:31:40,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f0575bc060ce161707'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:40,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f0575bc060ce161707 2025-12-17 23:31:40,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f0575bc060ce161707 HTTP/1.1" 502 150 2025-12-17 23:31:40,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f0575bc060ce161707'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:41,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e0575bc060ce1616e8 HTTP/1.1" 502 150 2025-12-17 23:31:41,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e0575bc060ce1616e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:41,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096eb575bc060ce161701 2025-12-17 23:31:41,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096eb575bc060ce161701 HTTP/1.1" 502 150 2025-12-17 23:31:41,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096eb575bc060ce161701'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:42,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694121ed459ff5fc64d2b7f1 2025-12-17 23:31:42,157 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e5575bc060ce1616f6 2025-12-17 23:31:42,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e5575bc060ce1616f6 HTTP/1.1" 502 150 2025-12-17 23:31:42,158 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e5575bc060ce1616f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:42,295 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e7575bc060ce1616f9 2025-12-17 23:31:42,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e7575bc060ce1616f9 HTTP/1.1" 502 150 2025-12-17 23:31:42,297 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e7575bc060ce1616f9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:42,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f0575bc060ce161707 2025-12-17 23:31:42,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f0575bc060ce161707 HTTP/1.1" 502 150 2025-12-17 23:31:42,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f0575bc060ce161707'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:43,104 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ed575bc060ce161704 2025-12-17 23:31:43,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ed575bc060ce161704 HTTP/1.1" 502 150 2025-12-17 23:31:43,106 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096ed575bc060ce161704'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:44,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096da575bc060ce1616cf 2025-12-17 23:31:44,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096da575bc060ce1616cf HTTP/1.1" 502 150 2025-12-17 23:31:44,396 - lava_callback - ERROR - Error processing callback for node 694096da575bc060ce1616cf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096da575bc060ce1616cf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096da575bc060ce1616cf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096da575bc060ce1616cf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:44,398 - lava_callback - INFO - Starting async job submit for node 694096f4575bc060ce161714 2025-12-17 23:31:44,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:44,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f4575bc060ce161714 HTTP/1.1" 502 150 2025-12-17 23:31:44,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f4575bc060ce161714'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:44,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f4575bc060ce161714 2025-12-17 23:31:44,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f4575bc060ce161714 HTTP/1.1" 502 150 2025-12-17 23:31:44,405 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f4575bc060ce161714'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:45,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e0575bc060ce1616e8 2025-12-17 23:31:45,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e0575bc060ce1616e8 HTTP/1.1" 502 150 2025-12-17 23:31:45,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e0575bc060ce1616e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:45,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e8575bc060ce1616fe 2025-12-17 23:31:45,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e8575bc060ce1616fe HTTP/1.1" 502 150 2025-12-17 23:31:45,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e8575bc060ce1616fe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:46,405 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f4575bc060ce161714 2025-12-17 23:31:46,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f4575bc060ce161714 HTTP/1.1" 502 150 2025-12-17 23:31:46,407 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f4575bc060ce161714'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53366 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:31:46,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096db575bc060ce1616d2 2025-12-17 23:31:46,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096db575bc060ce1616d2 HTTP/1.1" 502 150 2025-12-17 23:31:46,546 - lava_callback - ERROR - Error processing callback for node 694096db575bc060ce1616d2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096db575bc060ce1616d2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096db575bc060ce1616d2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096db575bc060ce1616d2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:46,548 - lava_callback - INFO - Starting async job submit for node 694096f6575bc060ce161719 2025-12-17 23:31:46,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:46,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f0575bc060ce161707 2025-12-17 23:31:46,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f0575bc060ce161707 HTTP/1.1" 502 150 2025-12-17 23:31:46,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f0575bc060ce161707'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:46,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f6575bc060ce161719 HTTP/1.1" 502 150 2025-12-17 23:31:46,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f6575bc060ce161719'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:46,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f6575bc060ce161719 2025-12-17 23:31:46,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f6575bc060ce161719 HTTP/1.1" 502 150 2025-12-17 23:31:46,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f6575bc060ce161719'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:46,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096de575bc060ce1616da 2025-12-17 23:31:46,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096de575bc060ce1616da HTTP/1.1" 502 150 2025-12-17 23:31:46,752 - lava_callback - ERROR - Error processing callback for node 694096de575bc060ce1616da: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096de575bc060ce1616da (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096de575bc060ce1616da (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096de575bc060ce1616da (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:46,753 - lava_callback - INFO - Starting async job submit for node 694096f8575bc060ce16171c 2025-12-17 23:31:46,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:46,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f8575bc060ce16171c HTTP/1.1" 502 150 2025-12-17 23:31:46,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f8575bc060ce16171c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:46,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f8575bc060ce16171c 2025-12-17 23:31:46,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f8575bc060ce16171c HTTP/1.1" 502 150 2025-12-17 23:31:46,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f8575bc060ce16171c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:46,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096df575bc060ce1616e1 2025-12-17 23:31:46,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096df575bc060ce1616e1 HTTP/1.1" 502 150 2025-12-17 23:31:46,867 - lava_callback - ERROR - Error processing callback for node 694096df575bc060ce1616e1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096df575bc060ce1616e1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096df575bc060ce1616e1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096df575bc060ce1616e1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:46,870 - lava_callback - INFO - Starting async job submit for node 694096fa575bc060ce16171f 2025-12-17 23:31:46,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:46,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096fa575bc060ce16171f HTTP/1.1" 502 150 2025-12-17 23:31:46,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096fa575bc060ce16171f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:46,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096fa575bc060ce16171f 2025-12-17 23:31:46,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096fa575bc060ce16171f HTTP/1.1" 502 150 2025-12-17 23:31:46,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096fa575bc060ce16171f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:47,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e2575bc060ce1616eb 2025-12-17 23:31:47,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e2575bc060ce1616eb HTTP/1.1" 502 150 2025-12-17 23:31:47,013 - lava_callback - ERROR - Error processing callback for node 694096e2575bc060ce1616eb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e2575bc060ce1616eb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e2575bc060ce1616eb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e2575bc060ce1616eb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:47,015 - lava_callback - INFO - Starting async job submit for node 69409701575bc060ce16172d 2025-12-17 23:31:47,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:47,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409701575bc060ce16172d HTTP/1.1" 502 150 2025-12-17 23:31:47,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409701575bc060ce16172d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:47,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409701575bc060ce16172d 2025-12-17 23:31:47,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409701575bc060ce16172d HTTP/1.1" 502 150 2025-12-17 23:31:47,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409701575bc060ce16172d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:47,681 - lava_callback - INFO - Received callback for node 69409a4f575bc060ce161f51 INFO: 172.21.0.1:53378 - "POST /node/69409a4f575bc060ce161f51 HTTP/1.0" 202 Accepted 2025-12-17 23:31:48,354 - lava_callback - INFO - Received callback for node 69409a51575bc060ce161f5a INFO: 172.21.0.1:53390 - "POST /node/69409a51575bc060ce161f5a HTTP/1.0" 202 Accepted 2025-12-17 23:31:48,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f6575bc060ce161719 2025-12-17 23:31:48,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f6575bc060ce161719 HTTP/1.1" 502 150 2025-12-17 23:31:48,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f6575bc060ce161719'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:48,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f8575bc060ce16171c 2025-12-17 23:31:48,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f8575bc060ce16171c HTTP/1.1" 502 150 2025-12-17 23:31:48,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f8575bc060ce16171c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:48,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096fa575bc060ce16171f 2025-12-17 23:31:48,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096fa575bc060ce16171f HTTP/1.1" 502 150 2025-12-17 23:31:48,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096fa575bc060ce16171f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:49,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409701575bc060ce16172d 2025-12-17 23:31:49,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409701575bc060ce16172d HTTP/1.1" 502 150 2025-12-17 23:31:49,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409701575bc060ce16172d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:49,034 - lava_callback - INFO - Received callback for node 69409a53575bc060ce161f5f INFO: 172.21.0.1:53392 - "POST /node/69409a53575bc060ce161f5f HTTP/1.0" 202 Accepted 2025-12-17 23:31:49,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096eb575bc060ce161701 2025-12-17 23:31:49,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096eb575bc060ce161701 HTTP/1.1" 502 150 2025-12-17 23:31:49,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096eb575bc060ce161701'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:49,725 - lava_callback - INFO - Received callback for node 69409a55575bc060ce161f62 INFO: 172.21.0.1:42706 - "POST /node/69409a55575bc060ce161f62 HTTP/1.0" 202 Accepted 2025-12-17 23:31:50,396 - lava_callback - INFO - Received callback for node 69409a5e575bc060ce161f68 INFO: 172.21.0.1:42716 - "POST /node/69409a5e575bc060ce161f68 HTTP/1.0" 202 Accepted 2025-12-17 23:31:50,407 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f4575bc060ce161714 2025-12-17 23:31:50,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f4575bc060ce161714 HTTP/1.1" 502 150 2025-12-17 23:31:50,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f4575bc060ce161714'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:52,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f6575bc060ce161719 2025-12-17 23:31:52,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f6575bc060ce161719 HTTP/1.1" 502 150 2025-12-17 23:31:52,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f6575bc060ce161719'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:52,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f8575bc060ce16171c 2025-12-17 23:31:52,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f8575bc060ce16171c HTTP/1.1" 502 150 2025-12-17 23:31:52,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f8575bc060ce16171c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:52,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096fa575bc060ce16171f 2025-12-17 23:31:52,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096fa575bc060ce16171f HTTP/1.1" 502 150 2025-12-17 23:31:52,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096fa575bc060ce16171f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:53,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409701575bc060ce16172d 2025-12-17 23:31:53,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409701575bc060ce16172d HTTP/1.1" 502 150 2025-12-17 23:31:53,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409701575bc060ce16172d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:53,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e0575bc060ce1616e8 2025-12-17 23:31:53,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e0575bc060ce1616e8 HTTP/1.1" 502 150 2025-12-17 23:31:53,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e0575bc060ce1616e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:54,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f0575bc060ce161707 2025-12-17 23:31:55,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096d7575bc060ce1616c9 2025-12-17 23:31:55,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096d7575bc060ce1616c9 HTTP/1.1" 502 150 2025-12-17 23:31:55,636 - lava_callback - ERROR - Error processing callback for node 694096d7575bc060ce1616c9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d7575bc060ce1616c9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d7575bc060ce1616c9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096d7575bc060ce1616c9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:55,637 - lava_callback - INFO - Starting async job submit for node 69409704575bc060ce161730 2025-12-17 23:31:55,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:55,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409704575bc060ce161730 HTTP/1.1" 502 150 2025-12-17 23:31:55,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409704575bc060ce161730'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:55,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409704575bc060ce161730 2025-12-17 23:31:55,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409704575bc060ce161730 HTTP/1.1" 502 150 2025-12-17 23:31:55,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409704575bc060ce161730'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:55,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e4575bc060ce1616f3 HTTP/1.1" 502 150 2025-12-17 23:31:55,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096e4575bc060ce1616f3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42732 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:31:57,050 - lava_callback - INFO - Received callback for node 69409a5b575bc060ce161f65 INFO: 172.21.0.1:42744 - "POST /node/69409a5b575bc060ce161f65 HTTP/1.0" 202 Accepted 2025-12-17 23:31:57,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694121ed459ff5fc64d2b7f1 HTTP/1.1" 502 150 2025-12-17 23:31:57,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694121ed459ff5fc64d2b7f1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:57,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409704575bc060ce161730 2025-12-17 23:31:57,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409704575bc060ce161730 HTTP/1.1" 502 150 2025-12-17 23:31:57,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409704575bc060ce161730'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:57,716 - lava_callback - INFO - Received callback for node 69409a60575bc060ce161f6f INFO: 172.21.0.1:42750 - "POST /node/69409a60575bc060ce161f6f HTTP/1.0" 202 Accepted 2025-12-17 23:31:58,158 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e5575bc060ce1616f6 2025-12-17 23:31:58,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e5575bc060ce1616f6 HTTP/1.1" 502 150 2025-12-17 23:31:58,160 - lava_callback - ERROR - Error processing callback for node 694096e5575bc060ce1616f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e5575bc060ce1616f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e5575bc060ce1616f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e5575bc060ce1616f6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:58,163 - lava_callback - INFO - Starting async job submit for node 694096fd575bc060ce161722 2025-12-17 23:31:58,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:58,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096fd575bc060ce161722 HTTP/1.1" 502 150 2025-12-17 23:31:58,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096fd575bc060ce161722'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:58,194 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096fd575bc060ce161722 2025-12-17 23:31:58,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096fd575bc060ce161722 HTTP/1.1" 502 150 2025-12-17 23:31:58,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096fd575bc060ce161722'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:58,297 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e7575bc060ce1616f9 2025-12-17 23:31:58,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e7575bc060ce1616f9 HTTP/1.1" 502 150 2025-12-17 23:31:58,298 - lava_callback - ERROR - Error processing callback for node 694096e7575bc060ce1616f9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e7575bc060ce1616f9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e7575bc060ce1616f9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e7575bc060ce1616f9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:58,301 - lava_callback - INFO - Starting async job submit for node 69409700575bc060ce16172a 2025-12-17 23:31:58,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:58,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409700575bc060ce16172a HTTP/1.1" 502 150 2025-12-17 23:31:58,308 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409700575bc060ce16172a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:58,308 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409700575bc060ce16172a 2025-12-17 23:31:58,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409700575bc060ce16172a HTTP/1.1" 502 150 2025-12-17 23:31:58,314 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409700575bc060ce16172a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:58,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f4575bc060ce161714 2025-12-17 23:31:58,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f4575bc060ce161714 HTTP/1.1" 502 150 2025-12-17 23:31:58,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f4575bc060ce161714'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:59,106 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096ed575bc060ce161704 2025-12-17 23:31:59,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096ed575bc060ce161704 HTTP/1.1" 502 150 2025-12-17 23:31:59,107 - lava_callback - ERROR - Error processing callback for node 694096ed575bc060ce161704: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096ed575bc060ce161704 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096ed575bc060ce161704 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096ed575bc060ce161704 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:31:59,109 - lava_callback - INFO - Starting async job submit for node 69409706575bc060ce161739 2025-12-17 23:31:59,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:31:59,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409706575bc060ce161739 HTTP/1.1" 502 150 2025-12-17 23:31:59,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409706575bc060ce161739'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:31:59,115 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409706575bc060ce161739 2025-12-17 23:31:59,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409706575bc060ce161739 HTTP/1.1" 502 150 2025-12-17 23:31:59,116 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409706575bc060ce161739'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:00,195 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096fd575bc060ce161722 2025-12-17 23:32:00,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096fd575bc060ce161722 HTTP/1.1" 502 150 2025-12-17 23:32:00,196 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096fd575bc060ce161722'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:00,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409700575bc060ce16172a 2025-12-17 23:32:00,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409700575bc060ce16172a HTTP/1.1" 502 150 2025-12-17 23:32:00,315 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409700575bc060ce16172a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:00,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f6575bc060ce161719 2025-12-17 23:32:00,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f6575bc060ce161719 HTTP/1.1" 502 150 2025-12-17 23:32:00,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f6575bc060ce161719'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:00,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f8575bc060ce16171c 2025-12-17 23:32:00,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f8575bc060ce16171c HTTP/1.1" 502 150 2025-12-17 23:32:00,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f8575bc060ce16171c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:00,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096fa575bc060ce16171f 2025-12-17 23:32:00,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096fa575bc060ce16171f HTTP/1.1" 502 150 2025-12-17 23:32:00,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096fa575bc060ce16171f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:01,028 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409701575bc060ce16172d 2025-12-17 23:32:01,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409701575bc060ce16172d HTTP/1.1" 502 150 2025-12-17 23:32:01,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409701575bc060ce16172d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:01,116 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409706575bc060ce161739 2025-12-17 23:32:01,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409706575bc060ce161739 HTTP/1.1" 502 150 2025-12-17 23:32:01,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409706575bc060ce161739'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:01,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409704575bc060ce161730 2025-12-17 23:32:01,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409704575bc060ce161730 HTTP/1.1" 502 150 2025-12-17 23:32:01,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409704575bc060ce161730'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:01,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e8575bc060ce1616fe 2025-12-17 23:32:01,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e8575bc060ce1616fe HTTP/1.1" 502 150 2025-12-17 23:32:01,779 - lava_callback - ERROR - Error processing callback for node 694096e8575bc060ce1616fe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e8575bc060ce1616fe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e8575bc060ce1616fe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e8575bc060ce1616fe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:01,781 - lava_callback - INFO - Starting async job submit for node 69409708575bc060ce16173c 2025-12-17 23:32:01,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:01,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409708575bc060ce16173c HTTP/1.1" 502 150 2025-12-17 23:32:01,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409708575bc060ce16173c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:01,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409708575bc060ce16173c 2025-12-17 23:32:01,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409708575bc060ce16173c HTTP/1.1" 502 150 2025-12-17 23:32:01,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409708575bc060ce16173c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:02,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c1575bc060ce16168a HTTP/1.1" 502 150 2025-12-17 23:32:02,096 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c1575bc060ce16168a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39258 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:32:03,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409708575bc060ce16173c 2025-12-17 23:32:03,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409708575bc060ce16173c HTTP/1.1" 502 150 2025-12-17 23:32:03,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409708575bc060ce16173c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:04,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096fd575bc060ce161722 2025-12-17 23:32:04,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096fd575bc060ce161722 HTTP/1.1" 502 150 2025-12-17 23:32:04,198 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096fd575bc060ce161722'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:04,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409700575bc060ce16172a 2025-12-17 23:32:04,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409700575bc060ce16172a HTTP/1.1" 502 150 2025-12-17 23:32:04,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409700575bc060ce16172a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:04,937 - lava_callback - INFO - Received callback for node 69409c41575bc060ce162550 INFO: 172.21.0.1:39260 - "POST /node/69409c41575bc060ce162550 HTTP/1.0" 202 Accepted 2025-12-17 23:32:05,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409706575bc060ce161739 2025-12-17 23:32:05,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409706575bc060ce161739 HTTP/1.1" 502 150 2025-12-17 23:32:05,119 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409706575bc060ce161739'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:05,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096eb575bc060ce161701 2025-12-17 23:32:05,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096eb575bc060ce161701 HTTP/1.1" 502 150 2025-12-17 23:32:05,689 - lava_callback - ERROR - Error processing callback for node 694096eb575bc060ce161701: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096eb575bc060ce161701 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096eb575bc060ce161701 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096eb575bc060ce161701 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:05,690 - lava_callback - INFO - Starting async job submit for node 6940970e575bc060ce16173f 2025-12-17 23:32:05,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:05,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940970e575bc060ce16173f HTTP/1.1" 502 150 2025-12-17 23:32:05,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940970e575bc060ce16173f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:05,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940970e575bc060ce16173f 2025-12-17 23:32:05,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940970e575bc060ce16173f HTTP/1.1" 502 150 2025-12-17 23:32:05,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940970e575bc060ce16173f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:06,097 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c1575bc060ce16168a INFO: 172.21.0.1:39264 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:32:07,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940970e575bc060ce16173f 2025-12-17 23:32:07,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940970e575bc060ce16173f HTTP/1.1" 502 150 2025-12-17 23:32:07,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940970e575bc060ce16173f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:07,701 - lava_callback - INFO - Received callback for node 69409a62575bc060ce161f74 INFO: 172.21.0.1:39278 - "POST /node/69409a62575bc060ce161f74 HTTP/1.0" 202 Accepted 2025-12-17 23:32:07,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409708575bc060ce16173c 2025-12-17 23:32:07,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409708575bc060ce16173c HTTP/1.1" 502 150 2025-12-17 23:32:07,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409708575bc060ce16173c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:08,362 - lava_callback - INFO - Received callback for node 69409a66575bc060ce161f88 INFO: 172.21.0.1:39282 - "POST /node/69409a66575bc060ce161f88 HTTP/1.0" 202 Accepted 2025-12-17 23:32:08,993 - lava_callback - INFO - Received callback for node 69409a77575bc060ce161fa7 INFO: 172.21.0.1:39294 - "POST /node/69409a77575bc060ce161fa7 HTTP/1.0" 202 Accepted 2025-12-17 23:32:09,028 - lava_callback - INFO - Received callback for node 69409a6c575bc060ce161f91 INFO: 172.21.0.1:39308 - "POST /node/69409a6c575bc060ce161f91 HTTP/1.0" 202 Accepted 2025-12-17 23:32:09,363 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e0575bc060ce1616e8 2025-12-17 23:32:09,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e0575bc060ce1616e8 HTTP/1.1" 502 150 2025-12-17 23:32:09,365 - lava_callback - ERROR - Error processing callback for node 694096e0575bc060ce1616e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e0575bc060ce1616e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e0575bc060ce1616e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e0575bc060ce1616e8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:09,367 - lava_callback - INFO - Starting async job submit for node 69409710575bc060ce161742 2025-12-17 23:32:09,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:09,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409710575bc060ce161742 HTTP/1.1" 502 150 2025-12-17 23:32:09,379 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409710575bc060ce161742'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:09,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409710575bc060ce161742 2025-12-17 23:32:09,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409710575bc060ce161742 HTTP/1.1" 502 150 2025-12-17 23:32:09,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409710575bc060ce161742'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:09,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409704575bc060ce161730 2025-12-17 23:32:09,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409704575bc060ce161730 HTTP/1.1" 502 150 2025-12-17 23:32:09,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409704575bc060ce161730'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:09,709 - lava_callback - INFO - Received callback for node 69409a6e575bc060ce161f94 INFO: 172.21.0.1:60178 - "POST /node/69409a6e575bc060ce161f94 HTTP/1.0" 202 Accepted 2025-12-17 23:32:10,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f0575bc060ce161707 HTTP/1.1" 502 150 2025-12-17 23:32:10,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096f0575bc060ce161707'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:10,393 - lava_callback - INFO - Received callback for node 69409a70575bc060ce161f97 INFO: 172.21.0.1:60184 - "POST /node/69409a70575bc060ce161f97 HTTP/1.0" 202 Accepted 2025-12-17 23:32:11,049 - lava_callback - INFO - Received callback for node 69409a72575bc060ce161f9a INFO: 172.21.0.1:60196 - "POST /node/69409a72575bc060ce161f9a HTTP/1.0" 202 Accepted 2025-12-17 23:32:11,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409710575bc060ce161742 2025-12-17 23:32:11,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409710575bc060ce161742 HTTP/1.1" 502 150 2025-12-17 23:32:11,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409710575bc060ce161742'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:11,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940970e575bc060ce16173f 2025-12-17 23:32:11,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940970e575bc060ce16173f HTTP/1.1" 502 150 2025-12-17 23:32:11,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940970e575bc060ce16173f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:11,740 - lava_callback - INFO - Received callback for node 69409a74575bc060ce161f9d INFO: 172.21.0.1:60204 - "POST /node/69409a74575bc060ce161f9d HTTP/1.0" 202 Accepted 2025-12-17 23:32:11,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096e4575bc060ce1616f3 2025-12-17 23:32:11,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096e4575bc060ce1616f3 HTTP/1.1" 502 150 2025-12-17 23:32:11,954 - lava_callback - ERROR - Error processing callback for node 694096e4575bc060ce1616f3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e4575bc060ce1616f3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e4575bc060ce1616f3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096e4575bc060ce1616f3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:11,956 - lava_callback - INFO - Starting async job submit for node 69409711575bc060ce161745 2025-12-17 23:32:11,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:11,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409711575bc060ce161745 HTTP/1.1" 502 150 2025-12-17 23:32:11,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409711575bc060ce161745'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:11,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409711575bc060ce161745 2025-12-17 23:32:11,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409711575bc060ce161745 HTTP/1.1" 502 150 2025-12-17 23:32:11,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409711575bc060ce161745'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:12,020 - lava_callback - INFO - Received callback for node 69409a76575bc060ce161fa0 INFO: 172.21.0.1:60210 - "POST /node/69409a76575bc060ce161fa0 HTTP/1.0" 202 Accepted 2025-12-17 23:32:12,198 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096fd575bc060ce161722 2025-12-17 23:32:12,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096fd575bc060ce161722 HTTP/1.1" 502 150 2025-12-17 23:32:12,199 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096fd575bc060ce161722'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:12,291 - lava_callback - INFO - Received callback for node 69409d67575bc060ce1627fe INFO: 172.21.0.1:60226 - "POST /node/69409d67575bc060ce1627fe HTTP/1.0" 202 Accepted 2025-12-17 23:32:12,317 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409700575bc060ce16172a 2025-12-17 23:32:12,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409700575bc060ce16172a HTTP/1.1" 502 150 2025-12-17 23:32:12,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409700575bc060ce16172a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:13,119 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409706575bc060ce161739 2025-12-17 23:32:13,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409706575bc060ce161739 HTTP/1.1" 502 150 2025-12-17 23:32:13,120 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409706575bc060ce161739'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:13,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694121ed459ff5fc64d2b7f1 2025-12-17 23:32:13,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694121ed459ff5fc64d2b7f1 HTTP/1.1" 502 150 2025-12-17 23:32:13,494 - lava_callback - ERROR - Error processing callback for node 694121ed459ff5fc64d2b7f1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694121ed459ff5fc64d2b7f1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694121ed459ff5fc64d2b7f1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694121ed459ff5fc64d2b7f1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:13,496 - lava_callback - INFO - Starting async job submit for node 69409713575bc060ce161748 2025-12-17 23:32:13,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:13,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409713575bc060ce161748 HTTP/1.1" 502 150 2025-12-17 23:32:13,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409713575bc060ce161748'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:13,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409713575bc060ce161748 2025-12-17 23:32:13,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409713575bc060ce161748 HTTP/1.1" 502 150 2025-12-17 23:32:13,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409713575bc060ce161748'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:13,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409711575bc060ce161745 2025-12-17 23:32:13,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409711575bc060ce161745 HTTP/1.1" 502 150 2025-12-17 23:32:13,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409711575bc060ce161745'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:14,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f4575bc060ce161714 2025-12-17 23:32:14,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f4575bc060ce161714 HTTP/1.1" 502 150 2025-12-17 23:32:14,411 - lava_callback - ERROR - Error processing callback for node 694096f4575bc060ce161714: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096f4575bc060ce161714 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096f4575bc060ce161714 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096f4575bc060ce161714 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:14,413 - lava_callback - INFO - Starting async job submit for node 69409714575bc060ce16174b 2025-12-17 23:32:14,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:14,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409714575bc060ce16174b HTTP/1.1" 502 150 2025-12-17 23:32:14,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409714575bc060ce16174b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:14,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409714575bc060ce16174b 2025-12-17 23:32:14,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409714575bc060ce16174b HTTP/1.1" 502 150 2025-12-17 23:32:14,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409714575bc060ce16174b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60228 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:32:15,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409710575bc060ce161742 2025-12-17 23:32:15,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409710575bc060ce161742 HTTP/1.1" 502 150 2025-12-17 23:32:15,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409710575bc060ce161742'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:15,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409713575bc060ce161748 2025-12-17 23:32:15,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409713575bc060ce161748 HTTP/1.1" 502 150 2025-12-17 23:32:15,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409713575bc060ce161748'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:15,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409708575bc060ce16173c 2025-12-17 23:32:15,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409708575bc060ce16173c HTTP/1.1" 502 150 2025-12-17 23:32:15,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409708575bc060ce16173c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:16,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409714575bc060ce16174b 2025-12-17 23:32:16,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409714575bc060ce16174b HTTP/1.1" 502 150 2025-12-17 23:32:16,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409714575bc060ce16174b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60244 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:32:16,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f6575bc060ce161719 2025-12-17 23:32:16,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f6575bc060ce161719 HTTP/1.1" 502 150 2025-12-17 23:32:16,561 - lava_callback - ERROR - Error processing callback for node 694096f6575bc060ce161719: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096f6575bc060ce161719 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096f6575bc060ce161719 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096f6575bc060ce161719 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:16,563 - lava_callback - INFO - Starting async job submit for node 69409715575bc060ce16174e 2025-12-17 23:32:16,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:16,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409715575bc060ce16174e HTTP/1.1" 502 150 2025-12-17 23:32:16,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409715575bc060ce16174e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:16,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409715575bc060ce16174e 2025-12-17 23:32:16,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409715575bc060ce16174e HTTP/1.1" 502 150 2025-12-17 23:32:16,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409715575bc060ce16174e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:16,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f8575bc060ce16171c 2025-12-17 23:32:16,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f8575bc060ce16171c HTTP/1.1" 502 150 2025-12-17 23:32:16,801 - lava_callback - ERROR - Error processing callback for node 694096f8575bc060ce16171c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096f8575bc060ce16171c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096f8575bc060ce16171c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096f8575bc060ce16171c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:16,803 - lava_callback - INFO - Starting async job submit for node 69409717575bc060ce161751 2025-12-17 23:32:16,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:16,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409717575bc060ce161751 HTTP/1.1" 502 150 2025-12-17 23:32:16,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409717575bc060ce161751'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:16,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409717575bc060ce161751 2025-12-17 23:32:16,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409717575bc060ce161751 HTTP/1.1" 502 150 2025-12-17 23:32:16,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409717575bc060ce161751'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:16,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096fa575bc060ce16171f 2025-12-17 23:32:16,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096fa575bc060ce16171f HTTP/1.1" 502 150 2025-12-17 23:32:16,893 - lava_callback - ERROR - Error processing callback for node 694096fa575bc060ce16171f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096fa575bc060ce16171f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096fa575bc060ce16171f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096fa575bc060ce16171f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:16,895 - lava_callback - INFO - Starting async job submit for node 69409718575bc060ce161754 2025-12-17 23:32:16,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:16,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409718575bc060ce161754 HTTP/1.1" 502 150 2025-12-17 23:32:16,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409718575bc060ce161754'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:16,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409718575bc060ce161754 2025-12-17 23:32:16,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409718575bc060ce161754 HTTP/1.1" 502 150 2025-12-17 23:32:16,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409718575bc060ce161754'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:17,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409701575bc060ce16172d 2025-12-17 23:32:17,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409701575bc060ce16172d HTTP/1.1" 502 150 2025-12-17 23:32:17,030 - lava_callback - ERROR - Error processing callback for node 69409701575bc060ce16172d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409701575bc060ce16172d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409701575bc060ce16172d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409701575bc060ce16172d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:17,032 - lava_callback - INFO - Starting async job submit for node 69409c7a575bc060ce1625db 2025-12-17 23:32:17,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:17,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7a575bc060ce1625db HTTP/1.1" 502 150 2025-12-17 23:32:17,109 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7a575bc060ce1625db'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:17,110 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7a575bc060ce1625db 2025-12-17 23:32:17,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7a575bc060ce1625db HTTP/1.1" 502 150 2025-12-17 23:32:17,110 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7a575bc060ce1625db'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:17,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409711575bc060ce161745 2025-12-17 23:32:17,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409711575bc060ce161745 HTTP/1.1" 502 150 2025-12-17 23:32:17,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409711575bc060ce161745'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:18,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409715575bc060ce16174e 2025-12-17 23:32:18,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409715575bc060ce16174e HTTP/1.1" 502 150 2025-12-17 23:32:18,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409715575bc060ce16174e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:18,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409717575bc060ce161751 2025-12-17 23:32:18,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409717575bc060ce161751 HTTP/1.1" 502 150 2025-12-17 23:32:18,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409717575bc060ce161751'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:18,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409718575bc060ce161754 2025-12-17 23:32:18,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409718575bc060ce161754 HTTP/1.1" 502 150 2025-12-17 23:32:18,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409718575bc060ce161754'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:19,111 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7a575bc060ce1625db 2025-12-17 23:32:19,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7a575bc060ce1625db HTTP/1.1" 502 150 2025-12-17 23:32:19,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7a575bc060ce1625db'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:19,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409713575bc060ce161748 2025-12-17 23:32:19,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409713575bc060ce161748 HTTP/1.1" 502 150 2025-12-17 23:32:19,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409713575bc060ce161748'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:19,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940970e575bc060ce16173f 2025-12-17 23:32:19,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940970e575bc060ce16173f HTTP/1.1" 502 150 2025-12-17 23:32:19,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940970e575bc060ce16173f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:20,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409714575bc060ce16174b 2025-12-17 23:32:20,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409714575bc060ce16174b HTTP/1.1" 502 150 2025-12-17 23:32:20,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409714575bc060ce16174b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:22,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409715575bc060ce16174e 2025-12-17 23:32:22,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409715575bc060ce16174e HTTP/1.1" 502 150 2025-12-17 23:32:22,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409715575bc060ce16174e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:22,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409717575bc060ce161751 2025-12-17 23:32:22,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409717575bc060ce161751 HTTP/1.1" 502 150 2025-12-17 23:32:22,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409717575bc060ce161751'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:22,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409718575bc060ce161754 2025-12-17 23:32:22,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409718575bc060ce161754 HTTP/1.1" 502 150 2025-12-17 23:32:22,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409718575bc060ce161754'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:23,112 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7a575bc060ce1625db 2025-12-17 23:32:23,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7a575bc060ce1625db HTTP/1.1" 502 150 2025-12-17 23:32:23,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7a575bc060ce1625db'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:23,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409710575bc060ce161742 2025-12-17 23:32:23,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409710575bc060ce161742 HTTP/1.1" 502 150 2025-12-17 23:32:23,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409710575bc060ce161742'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:25,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409704575bc060ce161730 2025-12-17 23:32:25,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409704575bc060ce161730 HTTP/1.1" 502 150 2025-12-17 23:32:25,651 - lava_callback - ERROR - Error processing callback for node 69409704575bc060ce161730: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409704575bc060ce161730 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409704575bc060ce161730 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409704575bc060ce161730 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:25,654 - lava_callback - INFO - Starting async job submit for node 6940971b575bc060ce16175f 2025-12-17 23:32:25,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:25,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971b575bc060ce16175f HTTP/1.1" 502 150 2025-12-17 23:32:25,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940971b575bc060ce16175f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:25,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940971b575bc060ce16175f 2025-12-17 23:32:25,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971b575bc060ce16175f HTTP/1.1" 502 150 2025-12-17 23:32:25,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940971b575bc060ce16175f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:25,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409711575bc060ce161745 2025-12-17 23:32:25,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409711575bc060ce161745 HTTP/1.1" 502 150 2025-12-17 23:32:25,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409711575bc060ce161745'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:26,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096f0575bc060ce161707 2025-12-17 23:32:26,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096f0575bc060ce161707 HTTP/1.1" 502 150 2025-12-17 23:32:26,034 - lava_callback - ERROR - Error processing callback for node 694096f0575bc060ce161707: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096f0575bc060ce161707 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096f0575bc060ce161707 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096f0575bc060ce161707 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:26,035 - lava_callback - INFO - Starting async job submit for node 6940971d575bc060ce161764 2025-12-17 23:32:26,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:26,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971d575bc060ce161764 HTTP/1.1" 502 150 2025-12-17 23:32:26,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940971d575bc060ce161764'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:26,042 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940971d575bc060ce161764 2025-12-17 23:32:26,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971d575bc060ce161764 HTTP/1.1" 502 150 2025-12-17 23:32:26,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940971d575bc060ce161764'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44958 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:32:27,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409713575bc060ce161748 2025-12-17 23:32:27,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409713575bc060ce161748 HTTP/1.1" 502 150 2025-12-17 23:32:27,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409713575bc060ce161748'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:27,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940971b575bc060ce16175f 2025-12-17 23:32:27,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971b575bc060ce16175f HTTP/1.1" 502 150 2025-12-17 23:32:27,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940971b575bc060ce16175f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:27,716 - lava_callback - INFO - Received callback for node 69409a78575bc060ce161fae INFO: 172.21.0.1:44974 - "POST /node/69409a78575bc060ce161fae HTTP/1.0" 202 Accepted 2025-12-17 23:32:28,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940971d575bc060ce161764 2025-12-17 23:32:28,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971d575bc060ce161764 HTTP/1.1" 502 150 2025-12-17 23:32:28,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940971d575bc060ce161764'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:28,199 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096fd575bc060ce161722 2025-12-17 23:32:28,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096fd575bc060ce161722 HTTP/1.1" 502 150 2025-12-17 23:32:28,202 - lava_callback - ERROR - Error processing callback for node 694096fd575bc060ce161722: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096fd575bc060ce161722 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096fd575bc060ce161722 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096fd575bc060ce161722 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:28,203 - lava_callback - INFO - Starting async job submit for node 6940971f575bc060ce16176b 2025-12-17 23:32:28,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:28,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971f575bc060ce16176b HTTP/1.1" 502 150 2025-12-17 23:32:28,221 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940971f575bc060ce16176b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:28,221 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940971f575bc060ce16176b 2025-12-17 23:32:28,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971f575bc060ce16176b HTTP/1.1" 502 150 2025-12-17 23:32:28,222 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940971f575bc060ce16176b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:28,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409700575bc060ce16172a 2025-12-17 23:32:28,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409700575bc060ce16172a HTTP/1.1" 502 150 2025-12-17 23:32:28,321 - lava_callback - ERROR - Error processing callback for node 69409700575bc060ce16172a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409700575bc060ce16172a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409700575bc060ce16172a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409700575bc060ce16172a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:28,323 - lava_callback - INFO - Starting async job submit for node 69409721575bc060ce16176e 2025-12-17 23:32:28,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:28,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409721575bc060ce16176e HTTP/1.1" 502 150 2025-12-17 23:32:28,329 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409721575bc060ce16176e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:28,329 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409721575bc060ce16176e 2025-12-17 23:32:28,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409721575bc060ce16176e HTTP/1.1" 502 150 2025-12-17 23:32:28,330 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409721575bc060ce16176e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:28,399 - lava_callback - INFO - Received callback for node 69409a7a575bc060ce161fb7 INFO: 172.21.0.1:44988 - "POST /node/69409a7a575bc060ce161fb7 HTTP/1.0" 202 Accepted 2025-12-17 23:32:28,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409714575bc060ce16174b 2025-12-17 23:32:28,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409714575bc060ce16174b HTTP/1.1" 502 150 2025-12-17 23:32:28,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409714575bc060ce16174b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:29,050 - lava_callback - INFO - Received callback for node 69409a7c575bc060ce161fbc INFO: 172.21.0.1:45002 - "POST /node/69409a7c575bc060ce161fbc HTTP/1.0" 202 Accepted 2025-12-17 23:32:29,120 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409706575bc060ce161739 2025-12-17 23:32:29,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409706575bc060ce161739 HTTP/1.1" 502 150 2025-12-17 23:32:29,122 - lava_callback - ERROR - Error processing callback for node 69409706575bc060ce161739: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409706575bc060ce161739 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409706575bc060ce161739 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409706575bc060ce161739 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:29,124 - lava_callback - INFO - Starting async job submit for node 69409723575bc060ce161771 2025-12-17 23:32:29,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:29,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409723575bc060ce161771 HTTP/1.1" 502 150 2025-12-17 23:32:29,130 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409723575bc060ce161771'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:29,130 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409723575bc060ce161771 2025-12-17 23:32:29,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409723575bc060ce161771 HTTP/1.1" 502 150 2025-12-17 23:32:29,131 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409723575bc060ce161771'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:29,732 - lava_callback - INFO - Received callback for node 69409a7e575bc060ce161fbf INFO: 172.21.0.1:44744 - "POST /node/69409a7e575bc060ce161fbf HTTP/1.0" 202 Accepted 2025-12-17 23:32:30,222 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940971f575bc060ce16176b 2025-12-17 23:32:30,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971f575bc060ce16176b HTTP/1.1" 502 150 2025-12-17 23:32:30,223 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940971f575bc060ce16176b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:30,330 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409721575bc060ce16176e 2025-12-17 23:32:30,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409721575bc060ce16176e HTTP/1.1" 502 150 2025-12-17 23:32:30,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409721575bc060ce16176e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:30,405 - lava_callback - INFO - Received callback for node 69409a84575bc060ce161fc2 INFO: 172.21.0.1:44754 - "POST /node/69409a84575bc060ce161fc2 HTTP/1.0" 202 Accepted 2025-12-17 23:32:30,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409715575bc060ce16174e 2025-12-17 23:32:30,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409715575bc060ce16174e HTTP/1.1" 502 150 2025-12-17 23:32:30,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409715575bc060ce16174e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:30,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409717575bc060ce161751 2025-12-17 23:32:30,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409717575bc060ce161751 HTTP/1.1" 502 150 2025-12-17 23:32:30,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409717575bc060ce161751'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:30,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409718575bc060ce161754 2025-12-17 23:32:30,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409718575bc060ce161754 HTTP/1.1" 502 150 2025-12-17 23:32:30,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409718575bc060ce161754'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:31,077 - lava_callback - INFO - Received callback for node 69409a87575bc060ce161fc8 INFO: 172.21.0.1:44768 - "POST /node/69409a87575bc060ce161fc8 HTTP/1.0" 202 Accepted 2025-12-17 23:32:31,113 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7a575bc060ce1625db 2025-12-17 23:32:31,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7a575bc060ce1625db HTTP/1.1" 502 150 2025-12-17 23:32:31,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7a575bc060ce1625db'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:31,131 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409723575bc060ce161771 2025-12-17 23:32:31,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409723575bc060ce161771 HTTP/1.1" 502 150 2025-12-17 23:32:31,132 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409723575bc060ce161771'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:31,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940971b575bc060ce16175f 2025-12-17 23:32:31,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971b575bc060ce16175f HTTP/1.1" 502 150 2025-12-17 23:32:31,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940971b575bc060ce16175f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:31,745 - lava_callback - INFO - Received callback for node 69409a89575bc060ce161fcb INFO: 172.21.0.1:44778 - "POST /node/69409a89575bc060ce161fcb HTTP/1.0" 202 Accepted 2025-12-17 23:32:31,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409708575bc060ce16173c 2025-12-17 23:32:31,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409708575bc060ce16173c HTTP/1.1" 502 150 2025-12-17 23:32:31,799 - lava_callback - ERROR - Error processing callback for node 69409708575bc060ce16173c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409708575bc060ce16173c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409708575bc060ce16173c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409708575bc060ce16173c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:31,800 - lava_callback - INFO - Starting async job submit for node 69409724575bc060ce161774 2025-12-17 23:32:31,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:31,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409724575bc060ce161774 HTTP/1.1" 502 150 2025-12-17 23:32:31,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409724575bc060ce161774'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:31,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409724575bc060ce161774 2025-12-17 23:32:31,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409724575bc060ce161774 HTTP/1.1" 502 150 2025-12-17 23:32:31,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409724575bc060ce161774'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:32,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940971d575bc060ce161764 2025-12-17 23:32:33,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409724575bc060ce161774 2025-12-17 23:32:33,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409724575bc060ce161774 HTTP/1.1" 502 150 2025-12-17 23:32:33,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409724575bc060ce161774'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:34,223 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940971f575bc060ce16176b 2025-12-17 23:32:34,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971f575bc060ce16176b HTTP/1.1" 502 150 2025-12-17 23:32:34,225 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940971f575bc060ce16176b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:34,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409721575bc060ce16176e 2025-12-17 23:32:34,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409721575bc060ce16176e HTTP/1.1" 502 150 2025-12-17 23:32:34,333 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409721575bc060ce16176e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:35,132 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409723575bc060ce161771 2025-12-17 23:32:35,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409723575bc060ce161771 HTTP/1.1" 502 150 2025-12-17 23:32:35,135 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409723575bc060ce161771'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:35,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940970e575bc060ce16173f 2025-12-17 23:32:35,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940970e575bc060ce16173f HTTP/1.1" 502 150 2025-12-17 23:32:35,705 - lava_callback - ERROR - Error processing callback for node 6940970e575bc060ce16173f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940970e575bc060ce16173f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940970e575bc060ce16173f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940970e575bc060ce16173f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:35,706 - lava_callback - INFO - Starting async job submit for node 69409726575bc060ce161777 2025-12-17 23:32:35,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:35,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409726575bc060ce161777 HTTP/1.1" 502 150 2025-12-17 23:32:35,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409726575bc060ce161777'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:35,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409726575bc060ce161777 2025-12-17 23:32:35,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409726575bc060ce161777 HTTP/1.1" 502 150 2025-12-17 23:32:35,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409726575bc060ce161777'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44792 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:32:36,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c1575bc060ce16168a HTTP/1.1" 502 150 2025-12-17 23:32:36,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c1575bc060ce16168a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:37,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409726575bc060ce161777 2025-12-17 23:32:37,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409726575bc060ce161777 HTTP/1.1" 502 150 2025-12-17 23:32:37,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409726575bc060ce161777'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:37,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409724575bc060ce161774 2025-12-17 23:32:37,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409724575bc060ce161774 HTTP/1.1" 502 150 2025-12-17 23:32:37,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409724575bc060ce161774'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:39,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409710575bc060ce161742 2025-12-17 23:32:39,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409710575bc060ce161742 HTTP/1.1" 502 150 2025-12-17 23:32:39,386 - lava_callback - ERROR - Error processing callback for node 69409710575bc060ce161742: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409710575bc060ce161742 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409710575bc060ce161742 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409710575bc060ce161742 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:39,388 - lava_callback - INFO - Starting async job submit for node 69409729575bc060ce16177e 2025-12-17 23:32:39,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:39,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409729575bc060ce16177e HTTP/1.1" 502 150 2025-12-17 23:32:39,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409729575bc060ce16177e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:39,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409729575bc060ce16177e 2025-12-17 23:32:39,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409729575bc060ce16177e HTTP/1.1" 502 150 2025-12-17 23:32:39,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409729575bc060ce16177e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:39,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940971b575bc060ce16175f 2025-12-17 23:32:39,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971b575bc060ce16175f HTTP/1.1" 502 150 2025-12-17 23:32:39,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940971b575bc060ce16175f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:41,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409729575bc060ce16177e 2025-12-17 23:32:41,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409729575bc060ce16177e HTTP/1.1" 502 150 2025-12-17 23:32:41,405 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409729575bc060ce16177e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:41,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409726575bc060ce161777 2025-12-17 23:32:41,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409726575bc060ce161777 HTTP/1.1" 502 150 2025-12-17 23:32:41,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409726575bc060ce161777'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:41,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409711575bc060ce161745 2025-12-17 23:32:41,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409711575bc060ce161745 HTTP/1.1" 502 150 2025-12-17 23:32:41,971 - lava_callback - ERROR - Error processing callback for node 69409711575bc060ce161745: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409711575bc060ce161745 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409711575bc060ce161745 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409711575bc060ce161745 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:41,973 - lava_callback - INFO - Starting async job submit for node 6940972b575bc060ce161781 2025-12-17 23:32:41,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:41,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972b575bc060ce161781 HTTP/1.1" 502 150 2025-12-17 23:32:41,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940972b575bc060ce161781'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:41,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940972b575bc060ce161781 2025-12-17 23:32:41,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972b575bc060ce161781 HTTP/1.1" 502 150 2025-12-17 23:32:41,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940972b575bc060ce161781'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:42,225 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940971f575bc060ce16176b 2025-12-17 23:32:42,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971f575bc060ce16176b HTTP/1.1" 502 150 2025-12-17 23:32:42,226 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940971f575bc060ce16176b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:42,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409721575bc060ce16176e 2025-12-17 23:32:42,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409721575bc060ce16176e HTTP/1.1" 502 150 2025-12-17 23:32:42,334 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409721575bc060ce16176e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:43,135 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409723575bc060ce161771 2025-12-17 23:32:43,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409723575bc060ce161771 HTTP/1.1" 502 150 2025-12-17 23:32:43,136 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409723575bc060ce161771'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:43,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409713575bc060ce161748 2025-12-17 23:32:43,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409713575bc060ce161748 HTTP/1.1" 502 150 2025-12-17 23:32:43,508 - lava_callback - ERROR - Error processing callback for node 69409713575bc060ce161748: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409713575bc060ce161748 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409713575bc060ce161748 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409713575bc060ce161748 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:43,511 - lava_callback - INFO - Starting async job submit for node 6940972d575bc060ce161784 2025-12-17 23:32:43,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:43,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972d575bc060ce161784 HTTP/1.1" 502 150 2025-12-17 23:32:43,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940972d575bc060ce161784'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:43,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940972d575bc060ce161784 2025-12-17 23:32:43,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972d575bc060ce161784 HTTP/1.1" 502 150 2025-12-17 23:32:43,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940972d575bc060ce161784'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:43,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940972b575bc060ce161781 2025-12-17 23:32:43,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972b575bc060ce161781 HTTP/1.1" 502 150 2025-12-17 23:32:43,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940972b575bc060ce161781'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:44,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409714575bc060ce16174b 2025-12-17 23:32:44,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409714575bc060ce16174b HTTP/1.1" 502 150 2025-12-17 23:32:44,426 - lava_callback - ERROR - Error processing callback for node 69409714575bc060ce16174b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409714575bc060ce16174b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409714575bc060ce16174b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409714575bc060ce16174b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:44,428 - lava_callback - INFO - Starting async job submit for node 6940972f575bc060ce161787 2025-12-17 23:32:44,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:44,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972f575bc060ce161787 HTTP/1.1" 502 150 2025-12-17 23:32:44,437 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940972f575bc060ce161787'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:44,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940972f575bc060ce161787 2025-12-17 23:32:44,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972f575bc060ce161787 HTTP/1.1" 502 150 2025-12-17 23:32:44,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940972f575bc060ce161787'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:44,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c1575bc060ce16168a 2025-12-17 23:32:45,405 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409729575bc060ce16177e 2025-12-17 23:32:45,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409729575bc060ce16177e HTTP/1.1" 502 150 2025-12-17 23:32:45,406 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409729575bc060ce16177e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:45,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940972d575bc060ce161784 2025-12-17 23:32:45,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972d575bc060ce161784 HTTP/1.1" 502 150 2025-12-17 23:32:45,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940972d575bc060ce161784'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:45,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409724575bc060ce161774 2025-12-17 23:32:45,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409724575bc060ce161774 HTTP/1.1" 502 150 2025-12-17 23:32:45,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409724575bc060ce161774'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:46,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940972f575bc060ce161787 2025-12-17 23:32:46,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972f575bc060ce161787 HTTP/1.1" 502 150 2025-12-17 23:32:46,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940972f575bc060ce161787'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38040 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:32:46,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409715575bc060ce16174e 2025-12-17 23:32:46,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409715575bc060ce16174e HTTP/1.1" 502 150 2025-12-17 23:32:46,599 - lava_callback - ERROR - Error processing callback for node 69409715575bc060ce16174e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409715575bc060ce16174e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409715575bc060ce16174e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409715575bc060ce16174e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:46,600 - lava_callback - INFO - Starting async job submit for node 69409731575bc060ce16178a 2025-12-17 23:32:46,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:46,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409731575bc060ce16178a HTTP/1.1" 502 150 2025-12-17 23:32:46,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409731575bc060ce16178a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:46,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409731575bc060ce16178a 2025-12-17 23:32:46,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409731575bc060ce16178a HTTP/1.1" 502 150 2025-12-17 23:32:46,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409731575bc060ce16178a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:46,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409717575bc060ce161751 2025-12-17 23:32:46,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409717575bc060ce161751 HTTP/1.1" 502 150 2025-12-17 23:32:46,851 - lava_callback - ERROR - Error processing callback for node 69409717575bc060ce161751: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409717575bc060ce161751 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409717575bc060ce161751 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409717575bc060ce161751 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:46,853 - lava_callback - INFO - Starting async job submit for node 694097ff575bc060ce16198b 2025-12-17 23:32:46,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:46,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ff575bc060ce16198b HTTP/1.1" 502 150 2025-12-17 23:32:46,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ff575bc060ce16198b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:46,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ff575bc060ce16198b 2025-12-17 23:32:46,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ff575bc060ce16198b HTTP/1.1" 502 150 2025-12-17 23:32:46,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ff575bc060ce16198b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:46,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409718575bc060ce161754 2025-12-17 23:32:46,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409718575bc060ce161754 HTTP/1.1" 502 150 2025-12-17 23:32:46,911 - lava_callback - ERROR - Error processing callback for node 69409718575bc060ce161754: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409718575bc060ce161754 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409718575bc060ce161754 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409718575bc060ce161754 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:46,913 - lava_callback - INFO - Starting async job submit for node 69409733575bc060ce16178d 2025-12-17 23:32:46,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:46,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409733575bc060ce16178d HTTP/1.1" 502 150 2025-12-17 23:32:46,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409733575bc060ce16178d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:46,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409733575bc060ce16178d 2025-12-17 23:32:46,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409733575bc060ce16178d HTTP/1.1" 502 150 2025-12-17 23:32:46,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409733575bc060ce16178d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:47,115 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7a575bc060ce1625db 2025-12-17 23:32:47,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971d575bc060ce161764 HTTP/1.1" 502 150 2025-12-17 23:32:47,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940971d575bc060ce161764'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:47,742 - lava_callback - INFO - Received callback for node 69409a8b575bc060ce161fd4 INFO: 172.21.0.1:38056 - "POST /node/69409a8b575bc060ce161fd4 HTTP/1.0" 202 Accepted 2025-12-17 23:32:47,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940972b575bc060ce161781 2025-12-17 23:32:47,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972b575bc060ce161781 HTTP/1.1" 502 150 2025-12-17 23:32:47,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940972b575bc060ce161781'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:48,420 - lava_callback - INFO - Received callback for node 69409a8e575bc060ce161fd9 INFO: 172.21.0.1:38060 - "POST /node/69409a8e575bc060ce161fd9 HTTP/1.0" 202 Accepted 2025-12-17 23:32:48,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409731575bc060ce16178a 2025-12-17 23:32:48,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409731575bc060ce16178a HTTP/1.1" 502 150 2025-12-17 23:32:48,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409731575bc060ce16178a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:48,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ff575bc060ce16198b 2025-12-17 23:32:48,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ff575bc060ce16198b HTTP/1.1" 502 150 2025-12-17 23:32:48,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ff575bc060ce16198b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:48,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409733575bc060ce16178d 2025-12-17 23:32:48,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409733575bc060ce16178d HTTP/1.1" 502 150 2025-12-17 23:32:48,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409733575bc060ce16178d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:49,064 - lava_callback - INFO - Received callback for node 69409a90575bc060ce161fdc INFO: 172.21.0.1:38068 - "POST /node/69409a90575bc060ce161fdc HTTP/1.0" 202 Accepted 2025-12-17 23:32:49,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940972d575bc060ce161784 2025-12-17 23:32:49,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972d575bc060ce161784 HTTP/1.1" 502 150 2025-12-17 23:32:49,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940972d575bc060ce161784'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:49,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409726575bc060ce161777 2025-12-17 23:32:49,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409726575bc060ce161777 HTTP/1.1" 502 150 2025-12-17 23:32:49,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409726575bc060ce161777'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:49,728 - lava_callback - INFO - Received callback for node 69409a92575bc060ce161fdf INFO: 172.21.0.1:60862 - "POST /node/69409a92575bc060ce161fdf HTTP/1.0" 202 Accepted 2025-12-17 23:32:50,397 - lava_callback - INFO - Received callback for node 69409a94575bc060ce161fe2 INFO: 172.21.0.1:60868 - "POST /node/69409a94575bc060ce161fe2 HTTP/1.0" 202 Accepted 2025-12-17 23:32:50,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940972f575bc060ce161787 2025-12-17 23:32:50,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972f575bc060ce161787 HTTP/1.1" 502 150 2025-12-17 23:32:50,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940972f575bc060ce161787'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:51,061 - lava_callback - INFO - Received callback for node 69409a96575bc060ce161fe5 INFO: 172.21.0.1:60884 - "POST /node/69409a96575bc060ce161fe5 HTTP/1.0" 202 Accepted 2025-12-17 23:32:51,735 - lava_callback - INFO - Received callback for node 69409a9a575bc060ce161feb INFO: 172.21.0.1:60892 - "POST /node/69409a9a575bc060ce161feb HTTP/1.0" 202 Accepted 2025-12-17 23:32:52,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409731575bc060ce16178a 2025-12-17 23:32:52,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409731575bc060ce16178a HTTP/1.1" 502 150 2025-12-17 23:32:52,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409731575bc060ce16178a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:52,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ff575bc060ce16198b 2025-12-17 23:32:52,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ff575bc060ce16198b HTTP/1.1" 502 150 2025-12-17 23:32:52,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ff575bc060ce16198b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:52,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409733575bc060ce16178d 2025-12-17 23:32:52,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409733575bc060ce16178d HTTP/1.1" 502 150 2025-12-17 23:32:52,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409733575bc060ce16178d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:53,406 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409729575bc060ce16177e 2025-12-17 23:32:53,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409729575bc060ce16177e HTTP/1.1" 502 150 2025-12-17 23:32:53,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409729575bc060ce16177e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:55,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940971d575bc060ce161764 2025-12-17 23:32:55,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971d575bc060ce161764 HTTP/1.1" 502 150 2025-12-17 23:32:55,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940971d575bc060ce161764'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:55,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940971b575bc060ce16175f 2025-12-17 23:32:55,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971b575bc060ce16175f HTTP/1.1" 502 150 2025-12-17 23:32:55,680 - lava_callback - ERROR - Error processing callback for node 6940971b575bc060ce16175f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940971b575bc060ce16175f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940971b575bc060ce16175f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940971b575bc060ce16175f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:55,682 - lava_callback - INFO - Starting async job submit for node 69409737575bc060ce161793 2025-12-17 23:32:55,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:55,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409737575bc060ce161793 HTTP/1.1" 502 150 2025-12-17 23:32:55,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409737575bc060ce161793'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:55,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409737575bc060ce161793 2025-12-17 23:32:55,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409737575bc060ce161793 HTTP/1.1" 502 150 2025-12-17 23:32:55,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409737575bc060ce161793'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:55,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940972b575bc060ce161781 2025-12-17 23:32:55,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972b575bc060ce161781 HTTP/1.1" 502 150 2025-12-17 23:32:55,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940972b575bc060ce161781'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60902 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:32:57,092 - lava_callback - INFO - Received callback for node 69409a98575bc060ce161fe8 INFO: 172.21.0.1:60918 - "POST /node/69409a98575bc060ce161fe8 HTTP/1.0" 202 Accepted 2025-12-17 23:32:57,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940972d575bc060ce161784 2025-12-17 23:32:57,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972d575bc060ce161784 HTTP/1.1" 502 150 2025-12-17 23:32:57,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940972d575bc060ce161784'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:57,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409737575bc060ce161793 2025-12-17 23:32:57,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409737575bc060ce161793 HTTP/1.1" 502 150 2025-12-17 23:32:57,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409737575bc060ce161793'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:57,765 - lava_callback - INFO - Received callback for node 69409a9d575bc060ce161fee INFO: 172.21.0.1:60920 - "POST /node/69409a9d575bc060ce161fee HTTP/1.0" 202 Accepted 2025-12-17 23:32:58,226 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940971f575bc060ce16176b 2025-12-17 23:32:58,334 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409721575bc060ce16176e 2025-12-17 23:32:58,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409721575bc060ce16176e HTTP/1.1" 502 150 2025-12-17 23:32:58,337 - lava_callback - ERROR - Error processing callback for node 69409721575bc060ce16176e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409721575bc060ce16176e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409721575bc060ce16176e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409721575bc060ce16176e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:32:58,339 - lava_callback - INFO - Starting async job submit for node 69409735575bc060ce161790 2025-12-17 23:32:58,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:32:58,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409735575bc060ce161790 HTTP/1.1" 502 150 2025-12-17 23:32:58,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409735575bc060ce161790'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:58,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409735575bc060ce161790 2025-12-17 23:32:58,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409735575bc060ce161790 HTTP/1.1" 502 150 2025-12-17 23:32:58,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409735575bc060ce161790'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:58,432 - lava_callback - INFO - Received callback for node 69409aa1575bc060ce162000 INFO: 172.21.0.1:60936 - "POST /node/69409aa1575bc060ce162000 HTTP/1.0" 202 Accepted 2025-12-17 23:32:58,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940972f575bc060ce161787 2025-12-17 23:32:58,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972f575bc060ce161787 HTTP/1.1" 502 150 2025-12-17 23:32:58,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940972f575bc060ce161787'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:32:59,131 - lava_callback - INFO - Received callback for node 69409aa3575bc060ce162005 INFO: 172.21.0.1:60938 - "POST /node/69409aa3575bc060ce162005 HTTP/1.0" 202 Accepted 2025-12-17 23:32:59,137 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409723575bc060ce161771 2025-12-17 23:33:00,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409735575bc060ce161790 2025-12-17 23:33:00,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409735575bc060ce161790 HTTP/1.1" 502 150 2025-12-17 23:33:00,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409735575bc060ce161790'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:00,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409731575bc060ce16178a 2025-12-17 23:33:00,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409731575bc060ce16178a HTTP/1.1" 502 150 2025-12-17 23:33:00,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409731575bc060ce16178a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:00,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ff575bc060ce16198b 2025-12-17 23:33:00,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ff575bc060ce16198b HTTP/1.1" 502 150 2025-12-17 23:33:00,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ff575bc060ce16198b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:00,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409733575bc060ce16178d 2025-12-17 23:33:00,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409733575bc060ce16178d HTTP/1.1" 502 150 2025-12-17 23:33:00,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409733575bc060ce16178d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:01,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409737575bc060ce161793 2025-12-17 23:33:01,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409737575bc060ce161793 HTTP/1.1" 502 150 2025-12-17 23:33:01,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409737575bc060ce161793'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:01,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409724575bc060ce161774 2025-12-17 23:33:01,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409724575bc060ce161774 HTTP/1.1" 502 150 2025-12-17 23:33:01,815 - lava_callback - ERROR - Error processing callback for node 69409724575bc060ce161774: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409724575bc060ce161774 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409724575bc060ce161774 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409724575bc060ce161774 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:01,817 - lava_callback - INFO - Starting async job submit for node 6940973b575bc060ce161799 2025-12-17 23:33:01,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:01,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973b575bc060ce161799 HTTP/1.1" 502 150 2025-12-17 23:33:01,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940973b575bc060ce161799'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:01,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940973b575bc060ce161799 2025-12-17 23:33:01,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973b575bc060ce161799 HTTP/1.1" 502 150 2025-12-17 23:33:01,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940973b575bc060ce161799'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:02,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7a575bc060ce1625db HTTP/1.1" 502 150 2025-12-17 23:33:02,513 - lava_callback - ERROR - Error processing callback for node 69409c7a575bc060ce1625db: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c7a575bc060ce1625db (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c7a575bc060ce1625db (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c7a575bc060ce1625db (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:02,514 - lava_callback - INFO - Starting async job submit for node 6940973d575bc060ce16179c 2025-12-17 23:33:02,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:02,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973d575bc060ce16179c HTTP/1.1" 502 150 2025-12-17 23:33:02,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940973d575bc060ce16179c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:02,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940973d575bc060ce16179c 2025-12-17 23:33:02,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973d575bc060ce16179c HTTP/1.1" 502 150 2025-12-17 23:33:02,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940973d575bc060ce16179c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55074 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:33:03,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940973b575bc060ce161799 2025-12-17 23:33:03,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973b575bc060ce161799 HTTP/1.1" 502 150 2025-12-17 23:33:03,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940973b575bc060ce161799'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:04,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409735575bc060ce161790 2025-12-17 23:33:04,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409735575bc060ce161790 HTTP/1.1" 502 150 2025-12-17 23:33:04,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409735575bc060ce161790'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:04,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940973d575bc060ce16179c 2025-12-17 23:33:04,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973d575bc060ce16179c HTTP/1.1" 502 150 2025-12-17 23:33:04,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940973d575bc060ce16179c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:05,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409726575bc060ce161777 2025-12-17 23:33:05,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409726575bc060ce161777 HTTP/1.1" 502 150 2025-12-17 23:33:05,721 - lava_callback - ERROR - Error processing callback for node 69409726575bc060ce161777: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409726575bc060ce161777 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409726575bc060ce161777 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409726575bc060ce161777 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:05,723 - lava_callback - INFO - Starting async job submit for node 6940973e575bc060ce16179f 2025-12-17 23:33:05,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:05,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973e575bc060ce16179f HTTP/1.1" 502 150 2025-12-17 23:33:05,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940973e575bc060ce16179f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:05,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940973e575bc060ce16179f 2025-12-17 23:33:05,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973e575bc060ce16179f HTTP/1.1" 502 150 2025-12-17 23:33:05,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940973e575bc060ce16179f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55090 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:33:07,744 - lava_callback - INFO - Received callback for node 69409a9f575bc060ce161ff6 INFO: 172.21.0.1:55098 - "POST /node/69409a9f575bc060ce161ff6 HTTP/1.0" 202 Accepted 2025-12-17 23:33:07,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940973e575bc060ce16179f 2025-12-17 23:33:07,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973e575bc060ce16179f HTTP/1.1" 502 150 2025-12-17 23:33:07,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940973e575bc060ce16179f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:07,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940973b575bc060ce161799 2025-12-17 23:33:07,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973b575bc060ce161799 HTTP/1.1" 502 150 2025-12-17 23:33:07,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940973b575bc060ce161799'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:08,420 - lava_callback - INFO - Received callback for node 69409aa7575bc060ce16200e INFO: 172.21.0.1:55114 - "POST /node/69409aa7575bc060ce16200e HTTP/1.0" 202 Accepted 2025-12-17 23:33:08,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940973d575bc060ce16179c 2025-12-17 23:33:08,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973d575bc060ce16179c HTTP/1.1" 502 150 2025-12-17 23:33:08,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940973d575bc060ce16179c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:08,607 - lava_callback - INFO - Received callback for node 69409aa6575bc060ce16200b INFO: 172.21.0.1:55118 - "POST /node/69409aa6575bc060ce16200b HTTP/1.0" 202 Accepted 2025-12-17 23:33:09,114 - lava_callback - INFO - Received callback for node 69409aac575bc060ce162017 INFO: 172.21.0.1:55130 - "POST /node/69409aac575bc060ce162017 HTTP/1.0" 202 Accepted 2025-12-17 23:33:09,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409729575bc060ce16177e 2025-12-17 23:33:09,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409729575bc060ce16177e HTTP/1.1" 502 150 2025-12-17 23:33:09,409 - lava_callback - ERROR - Error processing callback for node 69409729575bc060ce16177e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409729575bc060ce16177e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409729575bc060ce16177e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409729575bc060ce16177e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:09,411 - lava_callback - INFO - Starting async job submit for node 69409740575bc060ce1617a2 2025-12-17 23:33:09,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:09,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409740575bc060ce1617a2 HTTP/1.1" 502 150 2025-12-17 23:33:09,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409740575bc060ce1617a2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:09,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409740575bc060ce1617a2 2025-12-17 23:33:09,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409740575bc060ce1617a2 HTTP/1.1" 502 150 2025-12-17 23:33:09,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409740575bc060ce1617a2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:09,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409737575bc060ce161793 2025-12-17 23:33:09,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409737575bc060ce161793 HTTP/1.1" 502 150 2025-12-17 23:33:09,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409737575bc060ce161793'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:09,791 - lava_callback - INFO - Received callback for node 69409ab0575bc060ce16201d INFO: 172.21.0.1:46350 - "POST /node/69409ab0575bc060ce16201d HTTP/1.0" 202 Accepted 2025-12-17 23:33:11,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940971d575bc060ce161764 2025-12-17 23:33:11,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971d575bc060ce161764 HTTP/1.1" 502 150 2025-12-17 23:33:11,415 - lava_callback - ERROR - Error processing callback for node 6940971d575bc060ce161764: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940971d575bc060ce161764 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940971d575bc060ce161764 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940971d575bc060ce161764 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:11,417 - lava_callback - INFO - Starting async job submit for node 69409739575bc060ce161796 2025-12-17 23:33:11,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:11,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409740575bc060ce1617a2 2025-12-17 23:33:11,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409740575bc060ce1617a2 HTTP/1.1" 502 150 2025-12-17 23:33:11,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409740575bc060ce1617a2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:11,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409739575bc060ce161796 HTTP/1.1" 502 150 2025-12-17 23:33:11,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409739575bc060ce161796'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:11,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409739575bc060ce161796 2025-12-17 23:33:11,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409739575bc060ce161796 HTTP/1.1" 502 150 2025-12-17 23:33:11,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409739575bc060ce161796'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:11,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940973e575bc060ce16179f 2025-12-17 23:33:11,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973e575bc060ce16179f HTTP/1.1" 502 150 2025-12-17 23:33:11,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940973e575bc060ce16179f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:11,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940972b575bc060ce161781 2025-12-17 23:33:11,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972b575bc060ce161781 HTTP/1.1" 502 150 2025-12-17 23:33:11,990 - lava_callback - ERROR - Error processing callback for node 6940972b575bc060ce161781: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940972b575bc060ce161781 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940972b575bc060ce161781 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940972b575bc060ce161781 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:11,991 - lava_callback - INFO - Starting async job submit for node 69409800575bc060ce16198e 2025-12-17 23:33:12,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:12,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409800575bc060ce16198e HTTP/1.1" 502 150 2025-12-17 23:33:12,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409800575bc060ce16198e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:12,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409800575bc060ce16198e 2025-12-17 23:33:12,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409800575bc060ce16198e HTTP/1.1" 502 150 2025-12-17 23:33:12,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409800575bc060ce16198e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:12,068 - lava_callback - INFO - Received callback for node 69409aa5575bc060ce162008 INFO: 172.21.0.1:46354 - "POST /node/69409aa5575bc060ce162008 HTTP/1.0" 202 Accepted 2025-12-17 23:33:12,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409735575bc060ce161790 2025-12-17 23:33:12,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409735575bc060ce161790 HTTP/1.1" 502 150 2025-12-17 23:33:12,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409735575bc060ce161790'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:13,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409739575bc060ce161796 2025-12-17 23:33:13,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940971f575bc060ce16176b HTTP/1.1" 502 150 2025-12-17 23:33:13,524 - lava_callback - ERROR - Error processing callback for node 6940971f575bc060ce16176b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940971f575bc060ce16176b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940971f575bc060ce16176b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940971f575bc060ce16176b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:13,527 - lava_callback - INFO - Starting async job submit for node 69409742575bc060ce1617a7 2025-12-17 23:33:13,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:13,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940972d575bc060ce161784 2025-12-17 23:33:13,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972d575bc060ce161784 HTTP/1.1" 502 150 2025-12-17 23:33:13,535 - lava_callback - ERROR - Error processing callback for node 6940972d575bc060ce161784: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940972d575bc060ce161784 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940972d575bc060ce161784 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940972d575bc060ce161784 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:13,538 - lava_callback - INFO - Starting async job submit for node 69409746575bc060ce1617ad 2025-12-17 23:33:13,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:13,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409742575bc060ce1617a7 HTTP/1.1" 502 150 2025-12-17 23:33:13,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409742575bc060ce1617a7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:13,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409742575bc060ce1617a7 2025-12-17 23:33:13,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409742575bc060ce1617a7 HTTP/1.1" 502 150 2025-12-17 23:33:13,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409742575bc060ce1617a7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:13,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409746575bc060ce1617ad HTTP/1.1" 502 150 2025-12-17 23:33:13,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409746575bc060ce1617ad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:13,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409746575bc060ce1617ad 2025-12-17 23:33:13,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409746575bc060ce1617ad HTTP/1.1" 502 150 2025-12-17 23:33:13,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409746575bc060ce1617ad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:14,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409800575bc060ce16198e 2025-12-17 23:33:14,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409800575bc060ce16198e HTTP/1.1" 502 150 2025-12-17 23:33:14,037 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409800575bc060ce16198e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:14,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940972f575bc060ce161787 2025-12-17 23:33:14,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940972f575bc060ce161787 HTTP/1.1" 502 150 2025-12-17 23:33:14,444 - lava_callback - ERROR - Error processing callback for node 6940972f575bc060ce161787: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940972f575bc060ce161787 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940972f575bc060ce161787 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940972f575bc060ce161787 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:14,447 - lava_callback - INFO - Starting async job submit for node 69409745575bc060ce1617aa 2025-12-17 23:33:14,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:14,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409745575bc060ce1617aa HTTP/1.1" 502 150 2025-12-17 23:33:14,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409745575bc060ce1617aa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:14,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409745575bc060ce1617aa 2025-12-17 23:33:14,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409745575bc060ce1617aa HTTP/1.1" 502 150 2025-12-17 23:33:14,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409745575bc060ce1617aa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:14,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409723575bc060ce161771 HTTP/1.1" 502 150 2025-12-17 23:33:14,544 - lava_callback - ERROR - Error processing callback for node 69409723575bc060ce161771: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409723575bc060ce161771 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409723575bc060ce161771 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409723575bc060ce161771 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:14,547 - lava_callback - INFO - Starting async job submit for node 69409747575bc060ce1617b0 2025-12-17 23:33:14,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:14,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409747575bc060ce1617b0 HTTP/1.1" 502 150 2025-12-17 23:33:14,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409747575bc060ce1617b0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:14,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409747575bc060ce1617b0 2025-12-17 23:33:14,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409747575bc060ce1617b0 HTTP/1.1" 502 150 2025-12-17 23:33:14,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409747575bc060ce1617b0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46366 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:33:15,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409740575bc060ce1617a2 2025-12-17 23:33:15,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409740575bc060ce1617a2 HTTP/1.1" 502 150 2025-12-17 23:33:15,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409740575bc060ce1617a2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:15,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409742575bc060ce1617a7 2025-12-17 23:33:15,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409742575bc060ce1617a7 HTTP/1.1" 502 150 2025-12-17 23:33:15,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409742575bc060ce1617a7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:15,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409746575bc060ce1617ad 2025-12-17 23:33:15,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409746575bc060ce1617ad HTTP/1.1" 502 150 2025-12-17 23:33:15,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409746575bc060ce1617ad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:15,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c1575bc060ce16168a HTTP/1.1" 502 150 2025-12-17 23:33:15,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694096c1575bc060ce16168a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:15,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940973b575bc060ce161799 2025-12-17 23:33:15,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973b575bc060ce161799 HTTP/1.1" 502 150 2025-12-17 23:33:15,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940973b575bc060ce161799'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:16,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409745575bc060ce1617aa 2025-12-17 23:33:16,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409745575bc060ce1617aa HTTP/1.1" 502 150 2025-12-17 23:33:16,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409745575bc060ce1617aa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46368 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:33:16,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940973d575bc060ce16179c 2025-12-17 23:33:16,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973d575bc060ce16179c HTTP/1.1" 502 150 2025-12-17 23:33:16,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940973d575bc060ce16179c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:16,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409747575bc060ce1617b0 2025-12-17 23:33:16,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409747575bc060ce1617b0 HTTP/1.1" 502 150 2025-12-17 23:33:16,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409747575bc060ce1617b0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:16,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409731575bc060ce16178a 2025-12-17 23:33:16,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409731575bc060ce16178a HTTP/1.1" 502 150 2025-12-17 23:33:16,613 - lava_callback - ERROR - Error processing callback for node 69409731575bc060ce16178a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409731575bc060ce16178a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409731575bc060ce16178a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409731575bc060ce16178a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:16,615 - lava_callback - INFO - Starting async job submit for node 69409749575bc060ce1617b3 2025-12-17 23:33:16,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:16,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409749575bc060ce1617b3 HTTP/1.1" 502 150 2025-12-17 23:33:16,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409749575bc060ce1617b3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:16,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409749575bc060ce1617b3 2025-12-17 23:33:16,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409749575bc060ce1617b3 HTTP/1.1" 502 150 2025-12-17 23:33:16,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409749575bc060ce1617b3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:16,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ff575bc060ce16198b 2025-12-17 23:33:16,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ff575bc060ce16198b HTTP/1.1" 502 150 2025-12-17 23:33:16,914 - lava_callback - ERROR - Error processing callback for node 694097ff575bc060ce16198b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ff575bc060ce16198b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ff575bc060ce16198b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ff575bc060ce16198b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:16,916 - lava_callback - INFO - Starting async job submit for node 69409751575bc060ce1617bf 2025-12-17 23:33:16,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:16,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409751575bc060ce1617bf HTTP/1.1" 502 150 2025-12-17 23:33:16,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409751575bc060ce1617bf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:16,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409751575bc060ce1617bf 2025-12-17 23:33:16,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409751575bc060ce1617bf HTTP/1.1" 502 150 2025-12-17 23:33:16,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409751575bc060ce1617bf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:16,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409733575bc060ce16178d 2025-12-17 23:33:16,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409733575bc060ce16178d HTTP/1.1" 502 150 2025-12-17 23:33:16,927 - lava_callback - ERROR - Error processing callback for node 69409733575bc060ce16178d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409733575bc060ce16178d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409733575bc060ce16178d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409733575bc060ce16178d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:16,928 - lava_callback - INFO - Starting async job submit for node 6940974b575bc060ce1617b6 2025-12-17 23:33:16,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:16,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974b575bc060ce1617b6 HTTP/1.1" 502 150 2025-12-17 23:33:16,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940974b575bc060ce1617b6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:16,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940974b575bc060ce1617b6 2025-12-17 23:33:16,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974b575bc060ce1617b6 HTTP/1.1" 502 150 2025-12-17 23:33:16,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940974b575bc060ce1617b6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:17,099 - lava_callback - INFO - Received callback for node 69409aaa575bc060ce162014 INFO: 172.21.0.1:46372 - "POST /node/69409aaa575bc060ce162014 HTTP/1.0" 202 Accepted 2025-12-17 23:33:17,772 - lava_callback - INFO - Received callback for node 69409aae575bc060ce16201a INFO: 172.21.0.1:46382 - "POST /node/69409aae575bc060ce16201a HTTP/1.0" 202 Accepted 2025-12-17 23:33:18,037 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409800575bc060ce16198e 2025-12-17 23:33:18,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409800575bc060ce16198e HTTP/1.1" 502 150 2025-12-17 23:33:18,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409800575bc060ce16198e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:18,437 - lava_callback - INFO - Received callback for node 69409ab2575bc060ce162022 INFO: 172.21.0.1:46386 - "POST /node/69409ab2575bc060ce162022 HTTP/1.0" 202 Accepted 2025-12-17 23:33:18,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409749575bc060ce1617b3 2025-12-17 23:33:18,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409749575bc060ce1617b3 HTTP/1.1" 502 150 2025-12-17 23:33:18,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409749575bc060ce1617b3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:18,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409751575bc060ce1617bf 2025-12-17 23:33:18,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409751575bc060ce1617bf HTTP/1.1" 502 150 2025-12-17 23:33:18,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409751575bc060ce1617bf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:18,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940974b575bc060ce1617b6 2025-12-17 23:33:18,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974b575bc060ce1617b6 HTTP/1.1" 502 150 2025-12-17 23:33:18,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940974b575bc060ce1617b6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:19,108 - lava_callback - INFO - Received callback for node 69409ab6575bc060ce162032 INFO: 172.21.0.1:46392 - "POST /node/69409ab6575bc060ce162032 HTTP/1.0" 202 Accepted 2025-12-17 23:33:19,228 - lava_callback - INFO - Received callback for node 6940a105575bc060ce162dee INFO: 172.21.0.1:46406 - "POST /node/6940a105575bc060ce162dee HTTP/1.0" 202 Accepted 2025-12-17 23:33:19,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409742575bc060ce1617a7 2025-12-17 23:33:19,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409742575bc060ce1617a7 HTTP/1.1" 502 150 2025-12-17 23:33:19,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409742575bc060ce1617a7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:19,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409746575bc060ce1617ad 2025-12-17 23:33:19,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409746575bc060ce1617ad HTTP/1.1" 502 150 2025-12-17 23:33:19,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409746575bc060ce1617ad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:19,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940973e575bc060ce16179f 2025-12-17 23:33:19,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973e575bc060ce16179f HTTP/1.1" 502 150 2025-12-17 23:33:19,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940973e575bc060ce16179f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:19,779 - lava_callback - INFO - Received callback for node 69409ab8575bc060ce162035 INFO: 172.21.0.1:59272 - "POST /node/69409ab8575bc060ce162035 HTTP/1.0" 202 Accepted 2025-12-17 23:33:20,452 - lava_callback - INFO - Received callback for node 69409aba575bc060ce162038 INFO: 172.21.0.1:59286 - "POST /node/69409aba575bc060ce162038 HTTP/1.0" 202 Accepted 2025-12-17 23:33:20,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409745575bc060ce1617aa 2025-12-17 23:33:20,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409745575bc060ce1617aa HTTP/1.1" 502 150 2025-12-17 23:33:20,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409745575bc060ce1617aa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:20,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409747575bc060ce1617b0 2025-12-17 23:33:20,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409747575bc060ce1617b0 HTTP/1.1" 502 150 2025-12-17 23:33:20,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409747575bc060ce1617b0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:22,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409749575bc060ce1617b3 2025-12-17 23:33:22,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409749575bc060ce1617b3 HTTP/1.1" 502 150 2025-12-17 23:33:22,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409749575bc060ce1617b3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:22,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409751575bc060ce1617bf 2025-12-17 23:33:22,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409751575bc060ce1617bf HTTP/1.1" 502 150 2025-12-17 23:33:22,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409751575bc060ce1617bf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:22,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940974b575bc060ce1617b6 2025-12-17 23:33:22,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974b575bc060ce1617b6 HTTP/1.1" 502 150 2025-12-17 23:33:22,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940974b575bc060ce1617b6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:23,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409740575bc060ce1617a2 2025-12-17 23:33:23,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409740575bc060ce1617a2 HTTP/1.1" 502 150 2025-12-17 23:33:23,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409740575bc060ce1617a2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:25,201 - lava_callback - INFO - Received callback for node 6942853c1b61f1c6005fab91 INFO: 172.21.0.1:59294 - "POST /node/6942853c1b61f1c6005fab91 HTTP/1.0" 202 Accepted 2025-12-17 23:33:25,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409737575bc060ce161793 2025-12-17 23:33:25,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409737575bc060ce161793 HTTP/1.1" 502 150 2025-12-17 23:33:25,698 - lava_callback - ERROR - Error processing callback for node 69409737575bc060ce161793: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409737575bc060ce161793 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409737575bc060ce161793 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409737575bc060ce161793 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:25,700 - lava_callback - INFO - Starting async job submit for node 69409756575bc060ce1617c8 2025-12-17 23:33:25,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:25,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409756575bc060ce1617c8 HTTP/1.1" 502 150 2025-12-17 23:33:25,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409756575bc060ce1617c8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:25,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409756575bc060ce1617c8 2025-12-17 23:33:25,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409756575bc060ce1617c8 HTTP/1.1" 502 150 2025-12-17 23:33:25,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409756575bc060ce1617c8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:26,039 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409800575bc060ce16198e 2025-12-17 23:33:26,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409800575bc060ce16198e HTTP/1.1" 502 150 2025-12-17 23:33:26,040 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409800575bc060ce16198e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59298 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:33:27,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409742575bc060ce1617a7 2025-12-17 23:33:27,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409742575bc060ce1617a7 HTTP/1.1" 502 150 2025-12-17 23:33:27,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409742575bc060ce1617a7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:27,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409746575bc060ce1617ad 2025-12-17 23:33:27,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409746575bc060ce1617ad HTTP/1.1" 502 150 2025-12-17 23:33:27,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409746575bc060ce1617ad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:27,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409756575bc060ce1617c8 2025-12-17 23:33:27,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409756575bc060ce1617c8 HTTP/1.1" 502 150 2025-12-17 23:33:27,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409756575bc060ce1617c8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:27,749 - lava_callback - INFO - Received callback for node 69409ab4575bc060ce16202b INFO: 172.21.0.1:59310 - "POST /node/69409ab4575bc060ce16202b HTTP/1.0" 202 Accepted 2025-12-17 23:33:28,351 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409735575bc060ce161790 2025-12-17 23:33:28,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409735575bc060ce161790 HTTP/1.1" 502 150 2025-12-17 23:33:28,352 - lava_callback - ERROR - Error processing callback for node 69409735575bc060ce161790: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409735575bc060ce161790 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409735575bc060ce161790 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409735575bc060ce161790 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:28,354 - lava_callback - INFO - Starting async job submit for node 6940974d575bc060ce1617b9 2025-12-17 23:33:28,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:28,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974d575bc060ce1617b9 HTTP/1.1" 502 150 2025-12-17 23:33:28,361 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940974d575bc060ce1617b9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:28,361 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940974d575bc060ce1617b9 2025-12-17 23:33:28,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974d575bc060ce1617b9 HTTP/1.1" 502 150 2025-12-17 23:33:28,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940974d575bc060ce1617b9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:28,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409745575bc060ce1617aa 2025-12-17 23:33:28,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409745575bc060ce1617aa HTTP/1.1" 502 150 2025-12-17 23:33:28,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409745575bc060ce1617aa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:28,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409747575bc060ce1617b0 2025-12-17 23:33:28,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409747575bc060ce1617b0 HTTP/1.1" 502 150 2025-12-17 23:33:28,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409747575bc060ce1617b0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:28,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409739575bc060ce161796 HTTP/1.1" 502 150 2025-12-17 23:33:28,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409739575bc060ce161796'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:30,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940974d575bc060ce1617b9 2025-12-17 23:33:30,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974d575bc060ce1617b9 HTTP/1.1" 502 150 2025-12-17 23:33:30,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940974d575bc060ce1617b9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:30,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409749575bc060ce1617b3 2025-12-17 23:33:30,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409749575bc060ce1617b3 HTTP/1.1" 502 150 2025-12-17 23:33:30,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409749575bc060ce1617b3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:30,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409751575bc060ce1617bf 2025-12-17 23:33:30,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409751575bc060ce1617bf HTTP/1.1" 502 150 2025-12-17 23:33:30,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409751575bc060ce1617bf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:30,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940974b575bc060ce1617b6 2025-12-17 23:33:30,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974b575bc060ce1617b6 HTTP/1.1" 502 150 2025-12-17 23:33:30,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940974b575bc060ce1617b6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:31,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694096c1575bc060ce16168a 2025-12-17 23:33:31,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409756575bc060ce1617c8 2025-12-17 23:33:31,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409756575bc060ce1617c8 HTTP/1.1" 502 150 2025-12-17 23:33:31,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409756575bc060ce1617c8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:31,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940973b575bc060ce161799 2025-12-17 23:33:31,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973b575bc060ce161799 HTTP/1.1" 502 150 2025-12-17 23:33:31,836 - lava_callback - ERROR - Error processing callback for node 6940973b575bc060ce161799: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940973b575bc060ce161799 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940973b575bc060ce161799 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940973b575bc060ce161799 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:31,838 - lava_callback - INFO - Starting async job submit for node 6940974f575bc060ce1617bc 2025-12-17 23:33:31,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:31,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974f575bc060ce1617bc HTTP/1.1" 502 150 2025-12-17 23:33:31,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940974f575bc060ce1617bc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:31,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940974f575bc060ce1617bc 2025-12-17 23:33:31,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974f575bc060ce1617bc HTTP/1.1" 502 150 2025-12-17 23:33:31,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940974f575bc060ce1617bc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:32,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940973d575bc060ce16179c 2025-12-17 23:33:32,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973d575bc060ce16179c HTTP/1.1" 502 150 2025-12-17 23:33:32,550 - lava_callback - ERROR - Error processing callback for node 6940973d575bc060ce16179c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940973d575bc060ce16179c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940973d575bc060ce16179c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940973d575bc060ce16179c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:32,552 - lava_callback - INFO - Starting async job submit for node 69409752575bc060ce1617c2 2025-12-17 23:33:32,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:32,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409752575bc060ce1617c2 HTTP/1.1" 502 150 2025-12-17 23:33:32,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409752575bc060ce1617c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:32,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409752575bc060ce1617c2 2025-12-17 23:33:32,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409752575bc060ce1617c2 HTTP/1.1" 502 150 2025-12-17 23:33:32,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409752575bc060ce1617c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:32,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409739575bc060ce161796 2025-12-17 23:33:32,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409739575bc060ce161796 HTTP/1.1" 502 150 2025-12-17 23:33:32,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409739575bc060ce161796'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:33,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940974f575bc060ce1617bc 2025-12-17 23:33:33,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974f575bc060ce1617bc HTTP/1.1" 502 150 2025-12-17 23:33:33,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940974f575bc060ce1617bc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:34,363 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940974d575bc060ce1617b9 2025-12-17 23:33:34,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974d575bc060ce1617b9 HTTP/1.1" 502 150 2025-12-17 23:33:34,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940974d575bc060ce1617b9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:34,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409752575bc060ce1617c2 2025-12-17 23:33:34,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409752575bc060ce1617c2 HTTP/1.1" 502 150 2025-12-17 23:33:34,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409752575bc060ce1617c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:35,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940973e575bc060ce16179f 2025-12-17 23:33:35,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940973e575bc060ce16179f HTTP/1.1" 502 150 2025-12-17 23:33:35,761 - lava_callback - ERROR - Error processing callback for node 6940973e575bc060ce16179f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940973e575bc060ce16179f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940973e575bc060ce16179f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940973e575bc060ce16179f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:35,762 - lava_callback - INFO - Starting async job submit for node 69409754575bc060ce1617c5 2025-12-17 23:33:35,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:35,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409754575bc060ce1617c5 HTTP/1.1" 502 150 2025-12-17 23:33:35,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409754575bc060ce1617c5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:35,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409754575bc060ce1617c5 2025-12-17 23:33:35,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409754575bc060ce1617c5 HTTP/1.1" 502 150 2025-12-17 23:33:35,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409754575bc060ce1617c5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49818 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:33:36,991 - lava_callback - INFO - Received callback for node 69409dba575bc060ce1628a8 INFO: 172.21.0.1:49830 - "POST /node/69409dba575bc060ce1628a8 HTTP/1.0" 202 Accepted 2025-12-17 23:33:37,070 - lava_callback - INFO - Received callback for node 69409abc575bc060ce16203b INFO: 172.21.0.1:49844 - "POST /node/69409abc575bc060ce16203b HTTP/1.0" 202 Accepted 2025-12-17 23:33:37,746 - lava_callback - INFO - Received callback for node 69409abe575bc060ce162041 INFO: 172.21.0.1:49854 - "POST /node/69409abe575bc060ce162041 HTTP/1.0" 202 Accepted 2025-12-17 23:33:37,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409754575bc060ce1617c5 2025-12-17 23:33:37,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940974f575bc060ce1617bc 2025-12-17 23:33:37,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974f575bc060ce1617bc HTTP/1.1" 502 150 2025-12-17 23:33:37,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940974f575bc060ce1617bc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:38,409 - lava_callback - INFO - Received callback for node 69409ac0575bc060ce162044 INFO: 172.21.0.1:49870 - "POST /node/69409ac0575bc060ce162044 HTTP/1.0" 202 Accepted 2025-12-17 23:33:38,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409752575bc060ce1617c2 2025-12-17 23:33:38,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409752575bc060ce1617c2 HTTP/1.1" 502 150 2025-12-17 23:33:38,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409752575bc060ce1617c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:39,085 - lava_callback - INFO - Received callback for node 69409ac2575bc060ce162047 INFO: 172.21.0.1:49872 - "POST /node/69409ac2575bc060ce162047 HTTP/1.0" 202 Accepted 2025-12-17 23:33:39,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409740575bc060ce1617a2 2025-12-17 23:33:39,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409740575bc060ce1617a2 HTTP/1.1" 502 150 2025-12-17 23:33:39,429 - lava_callback - ERROR - Error processing callback for node 69409740575bc060ce1617a2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409740575bc060ce1617a2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409740575bc060ce1617a2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409740575bc060ce1617a2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:39,431 - lava_callback - INFO - Starting async job submit for node 6940975c575bc060ce1617d5 2025-12-17 23:33:39,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:39,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940975c575bc060ce1617d5 HTTP/1.1" 502 150 2025-12-17 23:33:39,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940975c575bc060ce1617d5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:39,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940975c575bc060ce1617d5 2025-12-17 23:33:39,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940975c575bc060ce1617d5 HTTP/1.1" 502 150 2025-12-17 23:33:39,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940975c575bc060ce1617d5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:39,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409756575bc060ce1617c8 2025-12-17 23:33:39,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409756575bc060ce1617c8 HTTP/1.1" 502 150 2025-12-17 23:33:39,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409756575bc060ce1617c8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:39,763 - lava_callback - INFO - Received callback for node 69409ac9575bc060ce16205d INFO: 172.21.0.1:41904 - "POST /node/69409ac9575bc060ce16205d HTTP/1.0" 202 Accepted 2025-12-17 23:33:39,785 - lava_callback - INFO - Received callback for node 69428f571b61f1c6005fbd14 INFO: 172.21.0.1:41908 - "POST /node/69428f571b61f1c6005fbd14 HTTP/1.0" 202 Accepted 2025-12-17 23:33:40,423 - lava_callback - INFO - Received callback for node 69409acd575bc060ce162064 INFO: 172.21.0.1:41920 - "POST /node/69409acd575bc060ce162064 HTTP/1.0" 202 Accepted 2025-12-17 23:33:40,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409739575bc060ce161796 2025-12-17 23:33:40,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409739575bc060ce161796 HTTP/1.1" 502 150 2025-12-17 23:33:40,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409739575bc060ce161796'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:41,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940975c575bc060ce1617d5 2025-12-17 23:33:41,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940975c575bc060ce1617d5 HTTP/1.1" 502 150 2025-12-17 23:33:41,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940975c575bc060ce1617d5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:42,040 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409800575bc060ce16198e 2025-12-17 23:33:42,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409800575bc060ce16198e HTTP/1.1" 502 150 2025-12-17 23:33:42,042 - lava_callback - ERROR - Error processing callback for node 69409800575bc060ce16198e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409800575bc060ce16198e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409800575bc060ce16198e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409800575bc060ce16198e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:42,045 - lava_callback - INFO - Starting async job submit for node 6940975e575bc060ce1617d8 2025-12-17 23:33:42,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:42,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940975e575bc060ce1617d8 HTTP/1.1" 502 150 2025-12-17 23:33:42,055 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940975e575bc060ce1617d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:42,056 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940975e575bc060ce1617d8 2025-12-17 23:33:42,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940975e575bc060ce1617d8 HTTP/1.1" 502 150 2025-12-17 23:33:42,057 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940975e575bc060ce1617d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:42,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940974d575bc060ce1617b9 2025-12-17 23:33:42,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974d575bc060ce1617b9 HTTP/1.1" 502 150 2025-12-17 23:33:42,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940974d575bc060ce1617b9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:43,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409742575bc060ce1617a7 2025-12-17 23:33:43,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409742575bc060ce1617a7 HTTP/1.1" 502 150 2025-12-17 23:33:43,548 - lava_callback - ERROR - Error processing callback for node 69409742575bc060ce1617a7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409742575bc060ce1617a7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409742575bc060ce1617a7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409742575bc060ce1617a7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:43,550 - lava_callback - INFO - Starting async job submit for node 69409757575bc060ce1617cf 2025-12-17 23:33:43,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:43,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409746575bc060ce1617ad 2025-12-17 23:33:43,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409746575bc060ce1617ad HTTP/1.1" 502 150 2025-12-17 23:33:43,552 - lava_callback - ERROR - Error processing callback for node 69409746575bc060ce1617ad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409746575bc060ce1617ad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409746575bc060ce1617ad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409746575bc060ce1617ad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:43,553 - lava_callback - INFO - Starting async job submit for node 69409759575bc060ce1617d2 2025-12-17 23:33:43,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:43,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409759575bc060ce1617d2 HTTP/1.1" 502 150 2025-12-17 23:33:43,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409759575bc060ce1617d2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:43,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409759575bc060ce1617d2 2025-12-17 23:33:43,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409759575bc060ce1617d2 HTTP/1.1" 502 150 2025-12-17 23:33:43,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409759575bc060ce1617d2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:43,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409757575bc060ce1617cf HTTP/1.1" 502 150 2025-12-17 23:33:43,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409757575bc060ce1617cf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:43,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409757575bc060ce1617cf 2025-12-17 23:33:43,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409757575bc060ce1617cf HTTP/1.1" 502 150 2025-12-17 23:33:43,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409757575bc060ce1617cf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:44,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940975e575bc060ce1617d8 2025-12-17 23:33:44,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940975e575bc060ce1617d8 HTTP/1.1" 502 150 2025-12-17 23:33:44,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940975e575bc060ce1617d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:44,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409745575bc060ce1617aa 2025-12-17 23:33:44,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409745575bc060ce1617aa HTTP/1.1" 502 150 2025-12-17 23:33:44,462 - lava_callback - ERROR - Error processing callback for node 69409745575bc060ce1617aa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409745575bc060ce1617aa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409745575bc060ce1617aa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409745575bc060ce1617aa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:44,464 - lava_callback - INFO - Starting async job submit for node 69409763575bc060ce1617f3 2025-12-17 23:33:44,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:44,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409763575bc060ce1617f3 HTTP/1.1" 502 150 2025-12-17 23:33:44,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409763575bc060ce1617f3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:44,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409763575bc060ce1617f3 2025-12-17 23:33:44,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409763575bc060ce1617f3 HTTP/1.1" 502 150 2025-12-17 23:33:44,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409763575bc060ce1617f3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:44,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409747575bc060ce1617b0 2025-12-17 23:33:44,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409747575bc060ce1617b0 HTTP/1.1" 502 150 2025-12-17 23:33:44,562 - lava_callback - ERROR - Error processing callback for node 69409747575bc060ce1617b0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409747575bc060ce1617b0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409747575bc060ce1617b0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409747575bc060ce1617b0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:44,564 - lava_callback - INFO - Starting async job submit for node 69409766575bc060ce1617f9 2025-12-17 23:33:44,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:44,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409766575bc060ce1617f9 HTTP/1.1" 502 150 2025-12-17 23:33:44,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409766575bc060ce1617f9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:44,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409766575bc060ce1617f9 2025-12-17 23:33:44,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409766575bc060ce1617f9 HTTP/1.1" 502 150 2025-12-17 23:33:44,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409766575bc060ce1617f9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:45,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940975c575bc060ce1617d5 2025-12-17 23:33:45,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940975c575bc060ce1617d5 HTTP/1.1" 502 150 2025-12-17 23:33:45,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940975c575bc060ce1617d5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:45,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409759575bc060ce1617d2 2025-12-17 23:33:45,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409759575bc060ce1617d2 HTTP/1.1" 502 150 2025-12-17 23:33:45,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409759575bc060ce1617d2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:45,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409757575bc060ce1617cf 2025-12-17 23:33:45,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409757575bc060ce1617cf HTTP/1.1" 502 150 2025-12-17 23:33:45,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409757575bc060ce1617cf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:45,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940974f575bc060ce1617bc 2025-12-17 23:33:45,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974f575bc060ce1617bc HTTP/1.1" 502 150 2025-12-17 23:33:45,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940974f575bc060ce1617bc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:46,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409763575bc060ce1617f3 2025-12-17 23:33:46,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409763575bc060ce1617f3 HTTP/1.1" 502 150 2025-12-17 23:33:46,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409763575bc060ce1617f3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41936 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:33:46,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409752575bc060ce1617c2 2025-12-17 23:33:46,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409752575bc060ce1617c2 HTTP/1.1" 502 150 2025-12-17 23:33:46,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409752575bc060ce1617c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:46,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409766575bc060ce1617f9 2025-12-17 23:33:46,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409766575bc060ce1617f9 HTTP/1.1" 502 150 2025-12-17 23:33:46,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409766575bc060ce1617f9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:46,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409749575bc060ce1617b3 2025-12-17 23:33:46,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409749575bc060ce1617b3 HTTP/1.1" 502 150 2025-12-17 23:33:46,636 - lava_callback - ERROR - Error processing callback for node 69409749575bc060ce1617b3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409749575bc060ce1617b3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409749575bc060ce1617b3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409749575bc060ce1617b3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:46,638 - lava_callback - INFO - Starting async job submit for node 69409761575bc060ce1617f0 2025-12-17 23:33:46,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:46,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409761575bc060ce1617f0 HTTP/1.1" 502 150 2025-12-17 23:33:46,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409761575bc060ce1617f0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:46,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409761575bc060ce1617f0 2025-12-17 23:33:46,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409761575bc060ce1617f0 HTTP/1.1" 502 150 2025-12-17 23:33:46,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409761575bc060ce1617f0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:46,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409751575bc060ce1617bf 2025-12-17 23:33:46,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409751575bc060ce1617bf HTTP/1.1" 502 150 2025-12-17 23:33:46,932 - lava_callback - ERROR - Error processing callback for node 69409751575bc060ce1617bf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409751575bc060ce1617bf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409751575bc060ce1617bf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409751575bc060ce1617bf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:46,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940974b575bc060ce1617b6 2025-12-17 23:33:46,940 - lava_callback - INFO - Starting async job submit for node 69409768575bc060ce1617fc 2025-12-17 23:33:46,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974b575bc060ce1617b6 HTTP/1.1" 502 150 2025-12-17 23:33:46,941 - lava_callback - ERROR - Error processing callback for node 6940974b575bc060ce1617b6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940974b575bc060ce1617b6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940974b575bc060ce1617b6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940974b575bc060ce1617b6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:46,944 - lava_callback - INFO - Starting async job submit for node 6940976a575bc060ce161802 2025-12-17 23:33:46,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:46,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:46,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976a575bc060ce161802 HTTP/1.1" 502 150 2025-12-17 23:33:46,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976a575bc060ce161802'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:46,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976a575bc060ce161802 2025-12-17 23:33:46,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976a575bc060ce161802 HTTP/1.1" 502 150 2025-12-17 23:33:46,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976a575bc060ce161802'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:47,782 - lava_callback - INFO - Received callback for node 69409ac6575bc060ce16204f INFO: 172.21.0.1:41940 - "POST /node/69409ac6575bc060ce16204f HTTP/1.0" 202 Accepted 2025-12-17 23:33:48,064 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940975e575bc060ce1617d8 2025-12-17 23:33:48,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940975e575bc060ce1617d8 HTTP/1.1" 502 150 2025-12-17 23:33:48,068 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940975e575bc060ce1617d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:48,464 - lava_callback - INFO - Received callback for node 69409acb575bc060ce162061 INFO: 172.21.0.1:41956 - "POST /node/69409acb575bc060ce162061 HTTP/1.0" 202 Accepted 2025-12-17 23:33:48,564 - lava_callback - INFO - Received callback for node 69409ad4575bc060ce16206d INFO: 172.21.0.1:41968 - "POST /node/69409ad4575bc060ce16206d HTTP/1.0" 202 Accepted 2025-12-17 23:33:48,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409761575bc060ce1617f0 2025-12-17 23:33:48,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409761575bc060ce1617f0 HTTP/1.1" 502 150 2025-12-17 23:33:48,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409761575bc060ce1617f0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:48,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976a575bc060ce161802 2025-12-17 23:33:48,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976a575bc060ce161802 HTTP/1.1" 502 150 2025-12-17 23:33:48,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976a575bc060ce161802'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:49,140 - lava_callback - INFO - Received callback for node 69409acf575bc060ce162067 INFO: 172.21.0.1:41982 - "POST /node/69409acf575bc060ce162067 HTTP/1.0" 202 Accepted 2025-12-17 23:33:49,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409759575bc060ce1617d2 2025-12-17 23:33:49,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409759575bc060ce1617d2 HTTP/1.1" 502 150 2025-12-17 23:33:49,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409759575bc060ce1617d2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:49,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409757575bc060ce1617cf 2025-12-17 23:33:49,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409757575bc060ce1617cf HTTP/1.1" 502 150 2025-12-17 23:33:49,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409757575bc060ce1617cf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:49,798 - lava_callback - INFO - Received callback for node 69409ad1575bc060ce16206a INFO: 172.21.0.1:41754 - "POST /node/69409ad1575bc060ce16206a HTTP/1.0" 202 Accepted 2025-12-17 23:33:50,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409763575bc060ce1617f3 2025-12-17 23:33:50,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409763575bc060ce1617f3 HTTP/1.1" 502 150 2025-12-17 23:33:50,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409763575bc060ce1617f3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:50,476 - lava_callback - INFO - Received callback for node 69409ad4575bc060ce162070 INFO: 172.21.0.1:41758 - "POST /node/69409ad4575bc060ce162070 HTTP/1.0" 202 Accepted 2025-12-17 23:33:50,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409766575bc060ce1617f9 2025-12-17 23:33:50,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409766575bc060ce1617f9 HTTP/1.1" 502 150 2025-12-17 23:33:50,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409766575bc060ce1617f9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:51,147 - lava_callback - INFO - Received callback for node 69409ad8575bc060ce162076 INFO: 172.21.0.1:41770 - "POST /node/69409ad8575bc060ce162076 HTTP/1.0" 202 Accepted 2025-12-17 23:33:52,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409761575bc060ce1617f0 2025-12-17 23:33:52,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409761575bc060ce1617f0 HTTP/1.1" 502 150 2025-12-17 23:33:52,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409761575bc060ce1617f0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:52,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976a575bc060ce161802 2025-12-17 23:33:52,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976a575bc060ce161802 HTTP/1.1" 502 150 2025-12-17 23:33:52,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976a575bc060ce161802'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:53,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409754575bc060ce1617c5 HTTP/1.1" 502 150 2025-12-17 23:33:53,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409754575bc060ce1617c5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:53,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940975c575bc060ce1617d5 2025-12-17 23:33:53,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940975c575bc060ce1617d5 HTTP/1.1" 502 150 2025-12-17 23:33:53,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940975c575bc060ce1617d5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:55,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409756575bc060ce1617c8 2025-12-17 23:33:55,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409756575bc060ce1617c8 HTTP/1.1" 502 150 2025-12-17 23:33:55,713 - lava_callback - ERROR - Error processing callback for node 69409756575bc060ce1617c8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409756575bc060ce1617c8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409756575bc060ce1617c8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409756575bc060ce1617c8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:55,716 - lava_callback - INFO - Starting async job submit for node 6940976b575bc060ce16180a 2025-12-17 23:33:55,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:55,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976b575bc060ce16180a HTTP/1.1" 502 150 2025-12-17 23:33:55,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976b575bc060ce16180a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:55,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976b575bc060ce16180a 2025-12-17 23:33:55,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976b575bc060ce16180a HTTP/1.1" 502 150 2025-12-17 23:33:55,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976b575bc060ce16180a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:56,068 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940975e575bc060ce1617d8 2025-12-17 23:33:56,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940975e575bc060ce1617d8 HTTP/1.1" 502 150 2025-12-17 23:33:56,069 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940975e575bc060ce1617d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41784 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:33:56,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409739575bc060ce161796 2025-12-17 23:33:56,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409739575bc060ce161796 HTTP/1.1" 502 150 2025-12-17 23:33:56,885 - lava_callback - ERROR - Error processing callback for node 69409739575bc060ce161796: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409739575bc060ce161796 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409739575bc060ce161796 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409739575bc060ce161796 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:56,887 - lava_callback - INFO - Starting async job submit for node 6940976c575bc060ce161811 2025-12-17 23:33:56,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:56,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976c575bc060ce161811 HTTP/1.1" 502 150 2025-12-17 23:33:56,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976c575bc060ce161811'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:56,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976c575bc060ce161811 2025-12-17 23:33:56,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976c575bc060ce161811 HTTP/1.1" 502 150 2025-12-17 23:33:56,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976c575bc060ce161811'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:57,111 - lava_callback - INFO - Received callback for node 69409ad6575bc060ce162073 INFO: 172.21.0.1:41790 - "POST /node/69409ad6575bc060ce162073 HTTP/1.0" 202 Accepted 2025-12-17 23:33:57,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409754575bc060ce1617c5 2025-12-17 23:33:57,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409754575bc060ce1617c5 HTTP/1.1" 502 150 2025-12-17 23:33:57,202 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409754575bc060ce1617c5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:57,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409759575bc060ce1617d2 2025-12-17 23:33:57,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409759575bc060ce1617d2 HTTP/1.1" 502 150 2025-12-17 23:33:57,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409759575bc060ce1617d2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:57,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409757575bc060ce1617cf 2025-12-17 23:33:57,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409757575bc060ce1617cf HTTP/1.1" 502 150 2025-12-17 23:33:57,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409757575bc060ce1617cf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:57,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976b575bc060ce16180a 2025-12-17 23:33:57,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976b575bc060ce16180a HTTP/1.1" 502 150 2025-12-17 23:33:57,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976b575bc060ce16180a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:57,769 - lava_callback - INFO - Received callback for node 69409adb575bc060ce16207b INFO: 172.21.0.1:41796 - "POST /node/69409adb575bc060ce16207b HTTP/1.0" 202 Accepted 2025-12-17 23:33:58,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940974d575bc060ce1617b9 2025-12-17 23:33:58,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974d575bc060ce1617b9 HTTP/1.1" 502 150 2025-12-17 23:33:58,368 - lava_callback - ERROR - Error processing callback for node 6940974d575bc060ce1617b9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940974d575bc060ce1617b9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940974d575bc060ce1617b9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940974d575bc060ce1617b9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:33:58,371 - lava_callback - INFO - Starting async job submit for node 6940976e575bc060ce161818 2025-12-17 23:33:58,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:33:58,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976e575bc060ce161818 HTTP/1.1" 502 150 2025-12-17 23:33:58,378 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976e575bc060ce161818'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:58,378 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976e575bc060ce161818 2025-12-17 23:33:58,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976e575bc060ce161818 HTTP/1.1" 502 150 2025-12-17 23:33:58,379 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976e575bc060ce161818'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:58,408 - lava_callback - INFO - Received callback for node 69409ae1575bc060ce162098 INFO: 172.21.0.1:41798 - "POST /node/69409ae1575bc060ce162098 HTTP/1.0" 202 Accepted 2025-12-17 23:33:58,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409763575bc060ce1617f3 2025-12-17 23:33:58,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409763575bc060ce1617f3 HTTP/1.1" 502 150 2025-12-17 23:33:58,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409763575bc060ce1617f3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:58,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409766575bc060ce1617f9 2025-12-17 23:33:58,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409766575bc060ce1617f9 HTTP/1.1" 502 150 2025-12-17 23:33:58,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409766575bc060ce1617f9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:33:58,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976c575bc060ce161811 2025-12-17 23:33:58,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976c575bc060ce161811 HTTP/1.1" 502 150 2025-12-17 23:33:58,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976c575bc060ce161811'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:00,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976e575bc060ce161818 2025-12-17 23:34:00,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976e575bc060ce161818 HTTP/1.1" 502 150 2025-12-17 23:34:00,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976e575bc060ce161818'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:00,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409761575bc060ce1617f0 2025-12-17 23:34:00,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409761575bc060ce1617f0 HTTP/1.1" 502 150 2025-12-17 23:34:00,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409761575bc060ce1617f0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:00,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976a575bc060ce161802 2025-12-17 23:34:01,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976a575bc060ce161802 HTTP/1.1" 502 150 2025-12-17 23:34:01,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976a575bc060ce161802'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:01,336 - lava_callback - INFO - Received callback for node 6942a2251b61f1c6005fd048 INFO: 172.21.0.1:50564 - "POST /node/6942a2251b61f1c6005fd048 HTTP/1.0" 202 Accepted 2025-12-17 23:34:01,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976b575bc060ce16180a 2025-12-17 23:34:01,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976b575bc060ce16180a HTTP/1.1" 502 150 2025-12-17 23:34:01,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976b575bc060ce16180a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:01,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940974f575bc060ce1617bc 2025-12-17 23:34:01,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940974f575bc060ce1617bc HTTP/1.1" 502 150 2025-12-17 23:34:01,875 - lava_callback - ERROR - Error processing callback for node 6940974f575bc060ce1617bc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940974f575bc060ce1617bc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940974f575bc060ce1617bc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940974f575bc060ce1617bc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:01,877 - lava_callback - INFO - Starting async job submit for node 6940976f575bc060ce16181d 2025-12-17 23:34:01,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:01,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976f575bc060ce16181d HTTP/1.1" 502 150 2025-12-17 23:34:01,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976f575bc060ce16181d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:01,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976f575bc060ce16181d 2025-12-17 23:34:01,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976f575bc060ce16181d HTTP/1.1" 502 150 2025-12-17 23:34:01,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976f575bc060ce16181d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:02,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694096c1575bc060ce16168a HTTP/1.1" 502 150 2025-12-17 23:34:02,416 - lava_callback - ERROR - Error processing callback for node 694096c1575bc060ce16168a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c1575bc060ce16168a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c1575bc060ce16168a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694096c1575bc060ce16168a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:02,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409768575bc060ce1617fc HTTP/1.1" 502 150 2025-12-17 23:34:02,418 - lava_callback - INFO - Starting async job submit for node 69409770575bc060ce161820 2025-12-17 23:34:02,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409768575bc060ce1617fc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:02,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:02,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409768575bc060ce1617fc 2025-12-17 23:34:02,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409768575bc060ce1617fc HTTP/1.1" 502 150 2025-12-17 23:34:02,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409768575bc060ce1617fc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:02,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409770575bc060ce161820 HTTP/1.1" 502 150 2025-12-17 23:34:02,425 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409770575bc060ce161820'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:02,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409770575bc060ce161820 2025-12-17 23:34:02,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409770575bc060ce161820 HTTP/1.1" 502 150 2025-12-17 23:34:02,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409770575bc060ce161820'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:02,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409752575bc060ce1617c2 2025-12-17 23:34:02,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409752575bc060ce1617c2 HTTP/1.1" 502 150 2025-12-17 23:34:02,570 - lava_callback - ERROR - Error processing callback for node 69409752575bc060ce1617c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409752575bc060ce1617c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409752575bc060ce1617c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409752575bc060ce1617c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:02,572 - lava_callback - INFO - Starting async job submit for node 69409771575bc060ce161823 2025-12-17 23:34:02,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:02,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409771575bc060ce161823 HTTP/1.1" 502 150 2025-12-17 23:34:02,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409771575bc060ce161823'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:02,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409771575bc060ce161823 2025-12-17 23:34:02,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409771575bc060ce161823 HTTP/1.1" 502 150 2025-12-17 23:34:02,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409771575bc060ce161823'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:02,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976c575bc060ce161811 2025-12-17 23:34:02,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976c575bc060ce161811 HTTP/1.1" 502 150 2025-12-17 23:34:02,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976c575bc060ce161811'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50576 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:34:03,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976f575bc060ce16181d 2025-12-17 23:34:03,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976f575bc060ce16181d HTTP/1.1" 502 150 2025-12-17 23:34:03,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976f575bc060ce16181d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:04,381 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976e575bc060ce161818 2025-12-17 23:34:04,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976e575bc060ce161818 HTTP/1.1" 502 150 2025-12-17 23:34:04,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976e575bc060ce161818'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:04,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409768575bc060ce1617fc 2025-12-17 23:34:04,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409768575bc060ce1617fc HTTP/1.1" 502 150 2025-12-17 23:34:04,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409768575bc060ce1617fc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:04,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409770575bc060ce161820 2025-12-17 23:34:04,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409770575bc060ce161820 HTTP/1.1" 502 150 2025-12-17 23:34:04,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409770575bc060ce161820'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:04,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409771575bc060ce161823 2025-12-17 23:34:04,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409771575bc060ce161823 HTTP/1.1" 502 150 2025-12-17 23:34:04,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409771575bc060ce161823'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:05,202 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409754575bc060ce1617c5 INFO: 172.21.0.1:50590 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:34:07,784 - lava_callback - INFO - Received callback for node 69409add575bc060ce162086 INFO: 172.21.0.1:50594 - "POST /node/69409add575bc060ce162086 HTTP/1.0" 202 Accepted 2025-12-17 23:34:07,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976f575bc060ce16181d 2025-12-17 23:34:07,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976f575bc060ce16181d HTTP/1.1" 502 150 2025-12-17 23:34:07,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976f575bc060ce16181d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:08,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409768575bc060ce1617fc 2025-12-17 23:34:08,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409768575bc060ce1617fc HTTP/1.1" 502 150 2025-12-17 23:34:08,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409768575bc060ce1617fc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:08,427 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409770575bc060ce161820 2025-12-17 23:34:08,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409770575bc060ce161820 HTTP/1.1" 502 150 2025-12-17 23:34:08,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409770575bc060ce161820'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:08,454 - lava_callback - INFO - Received callback for node 69409adf575bc060ce162091 INFO: 172.21.0.1:50598 - "POST /node/69409adf575bc060ce162091 HTTP/1.0" 202 Accepted 2025-12-17 23:34:08,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409771575bc060ce161823 2025-12-17 23:34:08,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409771575bc060ce161823 HTTP/1.1" 502 150 2025-12-17 23:34:08,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409771575bc060ce161823'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:09,102 - lava_callback - INFO - Received callback for node 69409ae3575bc060ce16209b INFO: 172.21.0.1:50602 - "POST /node/69409ae3575bc060ce16209b HTTP/1.0" 202 Accepted 2025-12-17 23:34:09,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940975c575bc060ce1617d5 2025-12-17 23:34:09,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940975c575bc060ce1617d5 HTTP/1.1" 502 150 2025-12-17 23:34:09,446 - lava_callback - ERROR - Error processing callback for node 6940975c575bc060ce1617d5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940975c575bc060ce1617d5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940975c575bc060ce1617d5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940975c575bc060ce1617d5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:09,448 - lava_callback - INFO - Starting async job submit for node 69409773575bc060ce161826 2025-12-17 23:34:09,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:09,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409773575bc060ce161826 HTTP/1.1" 502 150 2025-12-17 23:34:09,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409773575bc060ce161826'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:09,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409773575bc060ce161826 2025-12-17 23:34:09,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409773575bc060ce161826 HTTP/1.1" 502 150 2025-12-17 23:34:09,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409773575bc060ce161826'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:09,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976b575bc060ce16180a 2025-12-17 23:34:09,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976b575bc060ce16180a HTTP/1.1" 502 150 2025-12-17 23:34:09,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976b575bc060ce16180a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:09,755 - lava_callback - INFO - Received callback for node 69409ae5575bc060ce16209e INFO: 172.21.0.1:45500 - "POST /node/69409ae5575bc060ce16209e HTTP/1.0" 202 Accepted 2025-12-17 23:34:10,430 - lava_callback - INFO - Received callback for node 69409ae7575bc060ce1620a2 INFO: 172.21.0.1:45506 - "POST /node/69409ae7575bc060ce1620a2 HTTP/1.0" 202 Accepted 2025-12-17 23:34:10,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976c575bc060ce161811 2025-12-17 23:34:10,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976c575bc060ce161811 HTTP/1.1" 502 150 2025-12-17 23:34:10,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976c575bc060ce161811'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:11,101 - lava_callback - INFO - Received callback for node 69409ae9575bc060ce1620a5 INFO: 172.21.0.1:45512 - "POST /node/69409ae9575bc060ce1620a5 HTTP/1.0" 202 Accepted 2025-12-17 23:34:11,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409773575bc060ce161826 2025-12-17 23:34:11,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409773575bc060ce161826 HTTP/1.1" 502 150 2025-12-17 23:34:11,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409773575bc060ce161826'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:11,786 - lava_callback - INFO - Received callback for node 69409aed575bc060ce1620ae INFO: 172.21.0.1:45526 - "POST /node/69409aed575bc060ce1620ae HTTP/1.0" 202 Accepted 2025-12-17 23:34:12,069 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940975e575bc060ce1617d8 2025-12-17 23:34:12,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940975e575bc060ce1617d8 HTTP/1.1" 502 150 2025-12-17 23:34:12,072 - lava_callback - ERROR - Error processing callback for node 6940975e575bc060ce1617d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940975e575bc060ce1617d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940975e575bc060ce1617d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940975e575bc060ce1617d8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:12,074 - lava_callback - INFO - Starting async job submit for node 69409775575bc060ce16182c 2025-12-17 23:34:12,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:12,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409775575bc060ce16182c HTTP/1.1" 502 150 2025-12-17 23:34:12,082 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409775575bc060ce16182c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:12,082 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409775575bc060ce16182c 2025-12-17 23:34:12,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409775575bc060ce16182c HTTP/1.1" 502 150 2025-12-17 23:34:12,083 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409775575bc060ce16182c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:12,245 - lava_callback - INFO - Received callback for node 69409aeb575bc060ce1620a8 INFO: 172.21.0.1:45534 - "POST /node/69409aeb575bc060ce1620a8 HTTP/1.0" 202 Accepted 2025-12-17 23:34:12,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976e575bc060ce161818 2025-12-17 23:34:12,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976e575bc060ce161818 HTTP/1.1" 502 150 2025-12-17 23:34:12,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976e575bc060ce161818'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:12,935 - lava_callback - INFO - Received callback for node 69409aec575bc060ce1620ab INFO: 172.21.0.1:45550 - "POST /node/69409aec575bc060ce1620ab HTTP/1.0" 202 Accepted 2025-12-17 23:34:13,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409759575bc060ce1617d2 2025-12-17 23:34:13,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409759575bc060ce1617d2 HTTP/1.1" 502 150 2025-12-17 23:34:13,566 - lava_callback - ERROR - Error processing callback for node 69409759575bc060ce1617d2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409759575bc060ce1617d2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409759575bc060ce1617d2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409759575bc060ce1617d2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:13,568 - lava_callback - INFO - Starting async job submit for node 69409777575bc060ce16182f 2025-12-17 23:34:13,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:13,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409777575bc060ce16182f HTTP/1.1" 502 150 2025-12-17 23:34:13,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409777575bc060ce16182f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:13,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409777575bc060ce16182f 2025-12-17 23:34:13,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409777575bc060ce16182f HTTP/1.1" 502 150 2025-12-17 23:34:13,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409777575bc060ce16182f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:13,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409757575bc060ce1617cf 2025-12-17 23:34:13,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409757575bc060ce1617cf HTTP/1.1" 502 150 2025-12-17 23:34:13,587 - lava_callback - ERROR - Error processing callback for node 69409757575bc060ce1617cf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409757575bc060ce1617cf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409757575bc060ce1617cf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409757575bc060ce1617cf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:13,589 - lava_callback - INFO - Starting async job submit for node 6940977b575bc060ce161835 2025-12-17 23:34:13,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:13,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977b575bc060ce161835 HTTP/1.1" 502 150 2025-12-17 23:34:13,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977b575bc060ce161835'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:13,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977b575bc060ce161835 2025-12-17 23:34:13,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977b575bc060ce161835 HTTP/1.1" 502 150 2025-12-17 23:34:13,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977b575bc060ce161835'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:14,083 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409775575bc060ce16182c 2025-12-17 23:34:14,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409775575bc060ce16182c HTTP/1.1" 502 150 2025-12-17 23:34:14,084 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409775575bc060ce16182c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:14,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409763575bc060ce1617f3 2025-12-17 23:34:14,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409763575bc060ce1617f3 HTTP/1.1" 502 150 2025-12-17 23:34:14,477 - lava_callback - ERROR - Error processing callback for node 69409763575bc060ce1617f3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409763575bc060ce1617f3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409763575bc060ce1617f3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409763575bc060ce1617f3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:14,479 - lava_callback - INFO - Starting async job submit for node 6940992d575bc060ce161c5a 2025-12-17 23:34:14,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:14,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992d575bc060ce161c5a HTTP/1.1" 502 150 2025-12-17 23:34:14,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992d575bc060ce161c5a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:14,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992d575bc060ce161c5a 2025-12-17 23:34:14,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992d575bc060ce161c5a HTTP/1.1" 502 150 2025-12-17 23:34:14,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992d575bc060ce161c5a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:14,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409766575bc060ce1617f9 2025-12-17 23:34:14,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409766575bc060ce1617f9 HTTP/1.1" 502 150 2025-12-17 23:34:14,577 - lava_callback - ERROR - Error processing callback for node 69409766575bc060ce1617f9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409766575bc060ce1617f9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409766575bc060ce1617f9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409766575bc060ce1617f9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:14,579 - lava_callback - INFO - Starting async job submit for node 69409774575bc060ce161829 2025-12-17 23:34:14,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:14,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409774575bc060ce161829 HTTP/1.1" 502 150 2025-12-17 23:34:14,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409774575bc060ce161829'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:14,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409774575bc060ce161829 2025-12-17 23:34:14,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409774575bc060ce161829 HTTP/1.1" 502 150 2025-12-17 23:34:14,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409774575bc060ce161829'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45560 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:34:15,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409773575bc060ce161826 2025-12-17 23:34:15,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409773575bc060ce161826 HTTP/1.1" 502 150 2025-12-17 23:34:15,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409773575bc060ce161826'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:15,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409777575bc060ce16182f 2025-12-17 23:34:15,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409777575bc060ce16182f HTTP/1.1" 502 150 2025-12-17 23:34:15,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409777575bc060ce16182f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:15,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977b575bc060ce161835 2025-12-17 23:34:15,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977b575bc060ce161835 HTTP/1.1" 502 150 2025-12-17 23:34:15,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977b575bc060ce161835'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:15,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976f575bc060ce16181d 2025-12-17 23:34:15,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976f575bc060ce16181d HTTP/1.1" 502 150 2025-12-17 23:34:15,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940976f575bc060ce16181d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:16,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409768575bc060ce1617fc 2025-12-17 23:34:16,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409770575bc060ce161820 2025-12-17 23:34:16,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409770575bc060ce161820 HTTP/1.1" 502 150 2025-12-17 23:34:16,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409770575bc060ce161820'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:16,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992d575bc060ce161c5a 2025-12-17 23:34:16,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992d575bc060ce161c5a HTTP/1.1" 502 150 2025-12-17 23:34:16,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992d575bc060ce161c5a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45576 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:34:16,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409774575bc060ce161829 2025-12-17 23:34:16,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409774575bc060ce161829 HTTP/1.1" 502 150 2025-12-17 23:34:16,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409774575bc060ce161829'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:16,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409771575bc060ce161823 2025-12-17 23:34:16,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409771575bc060ce161823 HTTP/1.1" 502 150 2025-12-17 23:34:16,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409771575bc060ce161823'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:16,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409761575bc060ce1617f0 2025-12-17 23:34:16,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409761575bc060ce1617f0 HTTP/1.1" 502 150 2025-12-17 23:34:16,661 - lava_callback - ERROR - Error processing callback for node 69409761575bc060ce1617f0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409761575bc060ce1617f0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409761575bc060ce1617f0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409761575bc060ce1617f0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:16,663 - lava_callback - INFO - Starting async job submit for node 6940977a575bc060ce161832 2025-12-17 23:34:16,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:16,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977a575bc060ce161832 HTTP/1.1" 502 150 2025-12-17 23:34:16,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977a575bc060ce161832'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:16,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977a575bc060ce161832 2025-12-17 23:34:16,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977a575bc060ce161832 HTTP/1.1" 502 150 2025-12-17 23:34:16,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977a575bc060ce161832'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:17,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976a575bc060ce161802 2025-12-17 23:34:18,085 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409775575bc060ce16182c 2025-12-17 23:34:18,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409775575bc060ce16182c HTTP/1.1" 502 150 2025-12-17 23:34:18,086 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409775575bc060ce16182c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:18,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977a575bc060ce161832 2025-12-17 23:34:18,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977a575bc060ce161832 HTTP/1.1" 502 150 2025-12-17 23:34:18,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977a575bc060ce161832'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:19,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409777575bc060ce16182f 2025-12-17 23:34:19,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409777575bc060ce16182f HTTP/1.1" 502 150 2025-12-17 23:34:19,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409777575bc060ce16182f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:19,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977b575bc060ce161835 2025-12-17 23:34:19,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977b575bc060ce161835 HTTP/1.1" 502 150 2025-12-17 23:34:19,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977b575bc060ce161835'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:20,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992d575bc060ce161c5a 2025-12-17 23:34:20,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992d575bc060ce161c5a HTTP/1.1" 502 150 2025-12-17 23:34:20,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992d575bc060ce161c5a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:20,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409774575bc060ce161829 2025-12-17 23:34:20,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409774575bc060ce161829 HTTP/1.1" 502 150 2025-12-17 23:34:20,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409774575bc060ce161829'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:20,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409754575bc060ce1617c5 HTTP/1.1" 502 150 2025-12-17 23:34:20,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409754575bc060ce1617c5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:22,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977a575bc060ce161832 2025-12-17 23:34:22,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977a575bc060ce161832 HTTP/1.1" 502 150 2025-12-17 23:34:22,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977a575bc060ce161832'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:23,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409773575bc060ce161826 2025-12-17 23:34:23,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409773575bc060ce161826 HTTP/1.1" 502 150 2025-12-17 23:34:23,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409773575bc060ce161826'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:25,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976b575bc060ce16180a 2025-12-17 23:34:25,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976b575bc060ce16180a HTTP/1.1" 502 150 2025-12-17 23:34:25,730 - lava_callback - ERROR - Error processing callback for node 6940976b575bc060ce16180a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940976b575bc060ce16180a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940976b575bc060ce16180a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940976b575bc060ce16180a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:25,732 - lava_callback - INFO - Starting async job submit for node 6940977d575bc060ce161838 2025-12-17 23:34:25,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:25,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977d575bc060ce161838 HTTP/1.1" 502 150 2025-12-17 23:34:25,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977d575bc060ce161838'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:25,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977d575bc060ce161838 2025-12-17 23:34:25,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977d575bc060ce161838 HTTP/1.1" 502 150 2025-12-17 23:34:25,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977d575bc060ce161838'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:26,086 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409775575bc060ce16182c 2025-12-17 23:34:26,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409775575bc060ce16182c HTTP/1.1" 502 150 2025-12-17 23:34:26,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409775575bc060ce16182c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54658 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:34:26,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976c575bc060ce161811 2025-12-17 23:34:26,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976c575bc060ce161811 HTTP/1.1" 502 150 2025-12-17 23:34:26,903 - lava_callback - ERROR - Error processing callback for node 6940976c575bc060ce161811: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940976c575bc060ce161811 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940976c575bc060ce161811 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940976c575bc060ce161811 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:26,905 - lava_callback - INFO - Starting async job submit for node 6940977e575bc060ce16183d 2025-12-17 23:34:26,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:26,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977e575bc060ce16183d HTTP/1.1" 502 150 2025-12-17 23:34:26,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977e575bc060ce16183d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:26,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977e575bc060ce16183d 2025-12-17 23:34:26,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977e575bc060ce16183d HTTP/1.1" 502 150 2025-12-17 23:34:26,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977e575bc060ce16183d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:27,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409777575bc060ce16182f 2025-12-17 23:34:27,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409777575bc060ce16182f HTTP/1.1" 502 150 2025-12-17 23:34:27,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409777575bc060ce16182f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:27,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977b575bc060ce161835 2025-12-17 23:34:27,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977b575bc060ce161835 HTTP/1.1" 502 150 2025-12-17 23:34:27,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977b575bc060ce161835'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:27,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977d575bc060ce161838 2025-12-17 23:34:27,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977d575bc060ce161838 HTTP/1.1" 502 150 2025-12-17 23:34:27,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977d575bc060ce161838'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:27,829 - lava_callback - INFO - Received callback for node 69409af0575bc060ce1620bd INFO: 172.21.0.1:54668 - "POST /node/69409af0575bc060ce1620bd HTTP/1.0" 202 Accepted 2025-12-17 23:34:28,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976e575bc060ce161818 2025-12-17 23:34:28,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976e575bc060ce161818 HTTP/1.1" 502 150 2025-12-17 23:34:28,385 - lava_callback - ERROR - Error processing callback for node 6940976e575bc060ce161818: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940976e575bc060ce161818 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940976e575bc060ce161818 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940976e575bc060ce161818 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:28,386 - lava_callback - INFO - Starting async job submit for node 69409780575bc060ce161844 2025-12-17 23:34:28,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:28,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409780575bc060ce161844 HTTP/1.1" 502 150 2025-12-17 23:34:28,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409780575bc060ce161844'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:28,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409780575bc060ce161844 2025-12-17 23:34:28,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409780575bc060ce161844 HTTP/1.1" 502 150 2025-12-17 23:34:28,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409780575bc060ce161844'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:28,512 - lava_callback - INFO - Received callback for node 69409af2575bc060ce1620c2 2025-12-17 23:34:28,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992d575bc060ce161c5a INFO: 172.21.0.1:54682 - "POST /node/69409af2575bc060ce1620c2 HTTP/1.0" 202 Accepted 2025-12-17 23:34:28,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992d575bc060ce161c5a HTTP/1.1" 502 150 2025-12-17 23:34:28,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992d575bc060ce161c5a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:28,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409774575bc060ce161829 2025-12-17 23:34:28,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409774575bc060ce161829 HTTP/1.1" 502 150 2025-12-17 23:34:28,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409774575bc060ce161829'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:28,773 - lava_callback - INFO - Received callback for node 69409af9575bc060ce1620ce INFO: 172.21.0.1:54688 - "POST /node/69409af9575bc060ce1620ce HTTP/1.0" 202 Accepted 2025-12-17 23:34:28,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977e575bc060ce16183d 2025-12-17 23:34:28,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977e575bc060ce16183d HTTP/1.1" 502 150 2025-12-17 23:34:28,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977e575bc060ce16183d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:29,208 - lava_callback - INFO - Received callback for node 69409af3575bc060ce1620c5 INFO: 172.21.0.1:54692 - "POST /node/69409af3575bc060ce1620c5 HTTP/1.0" 202 Accepted 2025-12-17 23:34:29,883 - lava_callback - INFO - Received callback for node 69409af5575bc060ce1620c8 INFO: 172.21.0.1:50256 - "POST /node/69409af5575bc060ce1620c8 HTTP/1.0" 202 Accepted 2025-12-17 23:34:30,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409780575bc060ce161844 2025-12-17 23:34:30,548 - lava_callback - INFO - Received callback for node 69409af7575bc060ce1620cb INFO: 172.21.0.1:50270 - "POST /node/69409af7575bc060ce1620cb HTTP/1.0" 202 Accepted 2025-12-17 23:34:30,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977a575bc060ce161832 2025-12-17 23:34:30,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977a575bc060ce161832 HTTP/1.1" 502 150 2025-12-17 23:34:30,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977a575bc060ce161832'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:31,239 - lava_callback - INFO - Received callback for node 69409afa575bc060ce1620d1 INFO: 172.21.0.1:50272 - "POST /node/69409afa575bc060ce1620d1 HTTP/1.0" 202 Accepted 2025-12-17 23:34:31,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977d575bc060ce161838 2025-12-17 23:34:31,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940976f575bc060ce16181d 2025-12-17 23:34:31,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976f575bc060ce16181d HTTP/1.1" 502 150 2025-12-17 23:34:31,896 - lava_callback - ERROR - Error processing callback for node 6940976f575bc060ce16181d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940976f575bc060ce16181d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940976f575bc060ce16181d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940976f575bc060ce16181d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:31,898 - lava_callback - INFO - Starting async job submit for node 69409782575bc060ce16184d 2025-12-17 23:34:31,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:31,904 - lava_callback - INFO - Received callback for node 69409afb575bc060ce1620d4 INFO: 172.21.0.1:50280 - "POST /node/69409afb575bc060ce1620d4 HTTP/1.0" 202 Accepted 2025-12-17 23:34:31,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409782575bc060ce16184d HTTP/1.1" 502 150 2025-12-17 23:34:31,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409782575bc060ce16184d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:31,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409782575bc060ce16184d 2025-12-17 23:34:31,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409782575bc060ce16184d HTTP/1.1" 502 150 2025-12-17 23:34:31,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409782575bc060ce16184d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:32,098 - lava_callback - INFO - Received callback for node 69409b03575bc060ce1620eb INFO: 172.21.0.1:50294 - "POST /node/69409b03575bc060ce1620eb HTTP/1.0" 202 Accepted 2025-12-17 23:34:32,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940976a575bc060ce161802 HTTP/1.1" 502 150 2025-12-17 23:34:32,368 - lava_callback - ERROR - Error processing callback for node 6940976a575bc060ce161802: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940976a575bc060ce161802 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940976a575bc060ce161802 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940976a575bc060ce161802 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:32,370 - lava_callback - INFO - Starting async job submit for node 69409784575bc060ce161852 2025-12-17 23:34:32,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:32,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409784575bc060ce161852 HTTP/1.1" 502 150 2025-12-17 23:34:32,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409784575bc060ce161852'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:32,381 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409784575bc060ce161852 2025-12-17 23:34:32,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409784575bc060ce161852 HTTP/1.1" 502 150 2025-12-17 23:34:32,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409784575bc060ce161852'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:32,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409770575bc060ce161820 2025-12-17 23:34:32,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409770575bc060ce161820 HTTP/1.1" 502 150 2025-12-17 23:34:32,430 - lava_callback - ERROR - Error processing callback for node 69409770575bc060ce161820: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409770575bc060ce161820 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409770575bc060ce161820 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409770575bc060ce161820 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:32,432 - lava_callback - INFO - Starting async job submit for node 69409786575bc060ce161859 2025-12-17 23:34:32,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:32,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409786575bc060ce161859 HTTP/1.1" 502 150 2025-12-17 23:34:32,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409786575bc060ce161859'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:32,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409786575bc060ce161859 2025-12-17 23:34:32,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409786575bc060ce161859 HTTP/1.1" 502 150 2025-12-17 23:34:32,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409786575bc060ce161859'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:32,591 - lava_callback - INFO - Received callback for node 69409afd575bc060ce1620d7 INFO: 172.21.0.1:50300 - "POST /node/69409afd575bc060ce1620d7 HTTP/1.0" 202 Accepted 2025-12-17 23:34:32,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409771575bc060ce161823 2025-12-17 23:34:32,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409771575bc060ce161823 HTTP/1.1" 502 150 2025-12-17 23:34:32,611 - lava_callback - ERROR - Error processing callback for node 69409771575bc060ce161823: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409771575bc060ce161823 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409771575bc060ce161823 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409771575bc060ce161823 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:32,613 - lava_callback - INFO - Starting async job submit for node 694122fc459ff5fc64d2bae1 2025-12-17 23:34:32,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:32,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694122fc459ff5fc64d2bae1 HTTP/1.1" 502 150 2025-12-17 23:34:32,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694122fc459ff5fc64d2bae1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:32,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694122fc459ff5fc64d2bae1 2025-12-17 23:34:32,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694122fc459ff5fc64d2bae1 HTTP/1.1" 502 150 2025-12-17 23:34:32,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694122fc459ff5fc64d2bae1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:32,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977e575bc060ce16183d 2025-12-17 23:34:32,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977e575bc060ce16183d HTTP/1.1" 502 150 2025-12-17 23:34:32,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977e575bc060ce16183d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:33,271 - lava_callback - INFO - Received callback for node 69409aff575bc060ce1620da INFO: 172.21.0.1:50306 - "POST /node/69409aff575bc060ce1620da HTTP/1.0" 202 Accepted 2025-12-17 23:34:33,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409782575bc060ce16184d 2025-12-17 23:34:33,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409782575bc060ce16184d HTTP/1.1" 502 150 2025-12-17 23:34:33,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409782575bc060ce16184d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:33,936 - lava_callback - INFO - Received callback for node 69409b01575bc060ce1620dd INFO: 172.21.0.1:50320 - "POST /node/69409b01575bc060ce1620dd HTTP/1.0" 202 Accepted 2025-12-17 23:34:34,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409784575bc060ce161852 2025-12-17 23:34:34,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409784575bc060ce161852 HTTP/1.1" 502 150 2025-12-17 23:34:34,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409784575bc060ce161852'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:34,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409786575bc060ce161859 2025-12-17 23:34:34,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409786575bc060ce161859 HTTP/1.1" 502 150 2025-12-17 23:34:34,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409786575bc060ce161859'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:34,595 - lava_callback - INFO - Received callback for node 69409b02575bc060ce1620e0 INFO: 172.21.0.1:50328 - "POST /node/69409b02575bc060ce1620e0 HTTP/1.0" 202 Accepted 2025-12-17 23:34:34,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694122fc459ff5fc64d2bae1 2025-12-17 23:34:34,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694122fc459ff5fc64d2bae1 HTTP/1.1" 502 150 2025-12-17 23:34:34,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694122fc459ff5fc64d2bae1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50336 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:34:36,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409754575bc060ce1617c5 2025-12-17 23:34:36,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409754575bc060ce1617c5 HTTP/1.1" 502 150 2025-12-17 23:34:36,594 - lava_callback - ERROR - Error processing callback for node 69409754575bc060ce1617c5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409754575bc060ce1617c5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409754575bc060ce1617c5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409754575bc060ce1617c5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:36,595 - lava_callback - INFO - Starting async job submit for node 6940978a575bc060ce161864 2025-12-17 23:34:36,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:36,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978a575bc060ce161864 HTTP/1.1" 502 150 2025-12-17 23:34:36,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940978a575bc060ce161864'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:36,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940978a575bc060ce161864 2025-12-17 23:34:36,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978a575bc060ce161864 HTTP/1.1" 502 150 2025-12-17 23:34:36,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940978a575bc060ce161864'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:37,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409782575bc060ce16184d 2025-12-17 23:34:37,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409782575bc060ce16184d HTTP/1.1" 502 150 2025-12-17 23:34:37,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409782575bc060ce16184d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:38,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409784575bc060ce161852 2025-12-17 23:34:38,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409784575bc060ce161852 HTTP/1.1" 502 150 2025-12-17 23:34:38,385 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409784575bc060ce161852'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:38,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409786575bc060ce161859 2025-12-17 23:34:38,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409786575bc060ce161859 HTTP/1.1" 502 150 2025-12-17 23:34:38,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409786575bc060ce161859'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:38,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940978a575bc060ce161864 2025-12-17 23:34:38,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978a575bc060ce161864 HTTP/1.1" 502 150 2025-12-17 23:34:38,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940978a575bc060ce161864'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:38,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694122fc459ff5fc64d2bae1 2025-12-17 23:34:38,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694122fc459ff5fc64d2bae1 HTTP/1.1" 502 150 2025-12-17 23:34:38,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694122fc459ff5fc64d2bae1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:39,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409773575bc060ce161826 2025-12-17 23:34:39,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409773575bc060ce161826 HTTP/1.1" 502 150 2025-12-17 23:34:39,461 - lava_callback - ERROR - Error processing callback for node 69409773575bc060ce161826: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409773575bc060ce161826 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409773575bc060ce161826 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409773575bc060ce161826 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:39,463 - lava_callback - INFO - Starting async job submit for node 69409789575bc060ce16185c 2025-12-17 23:34:39,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:39,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409789575bc060ce16185c HTTP/1.1" 502 150 2025-12-17 23:34:39,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409789575bc060ce16185c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:39,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409789575bc060ce16185c 2025-12-17 23:34:39,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409789575bc060ce16185c HTTP/1.1" 502 150 2025-12-17 23:34:39,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409789575bc060ce16185c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:40,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977e575bc060ce16183d 2025-12-17 23:34:40,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977e575bc060ce16183d HTTP/1.1" 502 150 2025-12-17 23:34:40,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977e575bc060ce16183d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:41,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409789575bc060ce16185c 2025-12-17 23:34:41,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409789575bc060ce16185c HTTP/1.1" 502 150 2025-12-17 23:34:41,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409789575bc060ce16185c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:42,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409775575bc060ce16182c 2025-12-17 23:34:42,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409775575bc060ce16182c HTTP/1.1" 502 150 2025-12-17 23:34:42,089 - lava_callback - ERROR - Error processing callback for node 69409775575bc060ce16182c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409775575bc060ce16182c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409775575bc060ce16182c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409775575bc060ce16182c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:42,091 - lava_callback - INFO - Starting async job submit for node 69409838575bc060ce161a1b 2025-12-17 23:34:42,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:42,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409838575bc060ce161a1b HTTP/1.1" 502 150 2025-12-17 23:34:42,162 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409838575bc060ce161a1b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:42,162 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409838575bc060ce161a1b 2025-12-17 23:34:42,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409838575bc060ce161a1b HTTP/1.1" 502 150 2025-12-17 23:34:42,163 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409838575bc060ce161a1b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:42,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940978a575bc060ce161864 2025-12-17 23:34:42,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978a575bc060ce161864 HTTP/1.1" 502 150 2025-12-17 23:34:42,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940978a575bc060ce161864'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:43,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409777575bc060ce16182f 2025-12-17 23:34:43,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409777575bc060ce16182f HTTP/1.1" 502 150 2025-12-17 23:34:43,581 - lava_callback - ERROR - Error processing callback for node 69409777575bc060ce16182f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409777575bc060ce16182f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409777575bc060ce16182f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409777575bc060ce16182f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:43,583 - lava_callback - INFO - Starting async job submit for node 6940978d575bc060ce161867 2025-12-17 23:34:43,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:43,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978d575bc060ce161867 HTTP/1.1" 502 150 2025-12-17 23:34:43,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940978d575bc060ce161867'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:43,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940978d575bc060ce161867 2025-12-17 23:34:43,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978d575bc060ce161867 HTTP/1.1" 502 150 2025-12-17 23:34:43,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940978d575bc060ce161867'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:43,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977b575bc060ce161835 2025-12-17 23:34:43,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977b575bc060ce161835 HTTP/1.1" 502 150 2025-12-17 23:34:43,605 - lava_callback - ERROR - Error processing callback for node 6940977b575bc060ce161835: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940977b575bc060ce161835 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940977b575bc060ce161835 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940977b575bc060ce161835 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:43,606 - lava_callback - INFO - Starting async job submit for node 6940978f575bc060ce16186a 2025-12-17 23:34:43,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:43,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978f575bc060ce16186a HTTP/1.1" 502 150 2025-12-17 23:34:43,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940978f575bc060ce16186a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:43,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940978f575bc060ce16186a 2025-12-17 23:34:43,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978f575bc060ce16186a HTTP/1.1" 502 150 2025-12-17 23:34:43,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940978f575bc060ce16186a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:44,163 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409838575bc060ce161a1b 2025-12-17 23:34:44,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409838575bc060ce161a1b HTTP/1.1" 502 150 2025-12-17 23:34:44,165 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409838575bc060ce161a1b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:44,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992d575bc060ce161c5a 2025-12-17 23:34:44,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992d575bc060ce161c5a HTTP/1.1" 502 150 2025-12-17 23:34:44,521 - lava_callback - ERROR - Error processing callback for node 6940992d575bc060ce161c5a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940992d575bc060ce161c5a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940992d575bc060ce161c5a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940992d575bc060ce161c5a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:44,524 - lava_callback - INFO - Starting async job submit for node 69409793575bc060ce161872 2025-12-17 23:34:44,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:44,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409793575bc060ce161872 HTTP/1.1" 502 150 2025-12-17 23:34:44,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409793575bc060ce161872'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:44,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409793575bc060ce161872 2025-12-17 23:34:44,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409793575bc060ce161872 HTTP/1.1" 502 150 2025-12-17 23:34:44,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409793575bc060ce161872'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:44,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409774575bc060ce161829 2025-12-17 23:34:44,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409774575bc060ce161829 HTTP/1.1" 502 150 2025-12-17 23:34:44,591 - lava_callback - ERROR - Error processing callback for node 69409774575bc060ce161829: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409774575bc060ce161829 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409774575bc060ce161829 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409774575bc060ce161829 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:44,594 - lava_callback - INFO - Starting async job submit for node 69409795575bc060ce16187d 2025-12-17 23:34:44,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:44,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409795575bc060ce16187d HTTP/1.1" 502 150 2025-12-17 23:34:44,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409795575bc060ce16187d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:44,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409795575bc060ce16187d 2025-12-17 23:34:44,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409795575bc060ce16187d HTTP/1.1" 502 150 2025-12-17 23:34:44,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409795575bc060ce16187d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:45,225 - lava_callback - INFO - Received callback for node 69409803575bc060ce161994 INFO: 172.21.0.1:40838 - "POST /node/69409803575bc060ce161994 HTTP/1.0" 202 Accepted 2025-12-17 23:34:45,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409789575bc060ce16185c 2025-12-17 23:34:45,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409789575bc060ce16185c HTTP/1.1" 502 150 2025-12-17 23:34:45,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409789575bc060ce16185c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:45,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940978d575bc060ce161867 2025-12-17 23:34:45,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978d575bc060ce161867 HTTP/1.1" 502 150 2025-12-17 23:34:45,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940978d575bc060ce161867'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:45,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940978f575bc060ce16186a 2025-12-17 23:34:45,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978f575bc060ce16186a HTTP/1.1" 502 150 2025-12-17 23:34:45,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940978f575bc060ce16186a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:45,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409780575bc060ce161844 HTTP/1.1" 502 150 2025-12-17 23:34:45,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409780575bc060ce161844'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:45,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409782575bc060ce16184d 2025-12-17 23:34:45,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409782575bc060ce16184d HTTP/1.1" 502 150 2025-12-17 23:34:45,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409782575bc060ce16184d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:46,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409784575bc060ce161852 2025-12-17 23:34:46,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409784575bc060ce161852 HTTP/1.1" 502 150 2025-12-17 23:34:46,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409784575bc060ce161852'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:46,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409786575bc060ce161859 2025-12-17 23:34:46,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409786575bc060ce161859 HTTP/1.1" 502 150 2025-12-17 23:34:46,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409786575bc060ce161859'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:46,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409793575bc060ce161872 2025-12-17 23:34:46,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409793575bc060ce161872 HTTP/1.1" 502 150 2025-12-17 23:34:46,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409793575bc060ce161872'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:46,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409795575bc060ce16187d 2025-12-17 23:34:46,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409795575bc060ce16187d HTTP/1.1" 502 150 2025-12-17 23:34:46,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409795575bc060ce16187d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:46,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694122fc459ff5fc64d2bae1 2025-12-17 23:34:46,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694122fc459ff5fc64d2bae1 HTTP/1.1" 502 150 2025-12-17 23:34:46,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694122fc459ff5fc64d2bae1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:46,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977a575bc060ce161832 2025-12-17 23:34:46,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977a575bc060ce161832 HTTP/1.1" 502 150 2025-12-17 23:34:46,677 - lava_callback - ERROR - Error processing callback for node 6940977a575bc060ce161832: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940977a575bc060ce161832 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940977a575bc060ce161832 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940977a575bc060ce161832 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:46,679 - lava_callback - INFO - Starting async job submit for node 69409797575bc060ce161888 2025-12-17 23:34:46,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:46,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409797575bc060ce161888 HTTP/1.1" 502 150 2025-12-17 23:34:46,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409797575bc060ce161888'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:46,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409797575bc060ce161888 2025-12-17 23:34:46,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409797575bc060ce161888 HTTP/1.1" 502 150 2025-12-17 23:34:46,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409797575bc060ce161888'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:47,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977d575bc060ce161838 HTTP/1.1" 502 150 2025-12-17 23:34:47,216 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977d575bc060ce161838'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:47,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409768575bc060ce1617fc HTTP/1.1" 502 150 2025-12-17 23:34:47,217 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409768575bc060ce1617fc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:47,845 - lava_callback - INFO - Received callback for node 69409b06575bc060ce1620f7 INFO: 172.21.0.1:40846 - "POST /node/69409b06575bc060ce1620f7 HTTP/1.0" 202 Accepted 2025-12-17 23:34:48,165 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409838575bc060ce161a1b 2025-12-17 23:34:48,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409838575bc060ce161a1b HTTP/1.1" 502 150 2025-12-17 23:34:48,167 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409838575bc060ce161a1b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:48,534 - lava_callback - INFO - Received callback for node 69409b08575bc060ce1620fa INFO: 172.21.0.1:40854 - "POST /node/69409b08575bc060ce1620fa HTTP/1.0" 202 Accepted 2025-12-17 23:34:48,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409797575bc060ce161888 2025-12-17 23:34:48,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409797575bc060ce161888 HTTP/1.1" 502 150 2025-12-17 23:34:48,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409797575bc060ce161888'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:49,187 - lava_callback - INFO - Received callback for node 69409b0a575bc060ce1620ff INFO: 172.21.0.1:40862 - "POST /node/69409b0a575bc060ce1620ff HTTP/1.0" 202 Accepted 2025-12-17 23:34:49,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940978d575bc060ce161867 2025-12-17 23:34:49,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978d575bc060ce161867 HTTP/1.1" 502 150 2025-12-17 23:34:49,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940978d575bc060ce161867'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:49,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940978f575bc060ce16186a 2025-12-17 23:34:49,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978f575bc060ce16186a HTTP/1.1" 502 150 2025-12-17 23:34:49,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940978f575bc060ce16186a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:49,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409780575bc060ce161844 2025-12-17 23:34:49,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409780575bc060ce161844 HTTP/1.1" 502 150 2025-12-17 23:34:49,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409780575bc060ce161844'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:49,850 - lava_callback - INFO - Received callback for node 69409b0b575bc060ce162102 INFO: 172.21.0.1:33130 - "POST /node/69409b0b575bc060ce162102 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:33142 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:34:50,511 - lava_callback - INFO - Received callback for node 69409b0d575bc060ce162105 INFO: 172.21.0.1:33146 - "POST /node/69409b0d575bc060ce162105 HTTP/1.0" 202 Accepted 2025-12-17 23:34:50,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409793575bc060ce161872 2025-12-17 23:34:50,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409793575bc060ce161872 HTTP/1.1" 502 150 2025-12-17 23:34:50,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409793575bc060ce161872'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:50,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409795575bc060ce16187d 2025-12-17 23:34:50,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409795575bc060ce16187d HTTP/1.1" 502 150 2025-12-17 23:34:50,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409795575bc060ce16187d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:50,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940978a575bc060ce161864 2025-12-17 23:34:50,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978a575bc060ce161864 HTTP/1.1" 502 150 2025-12-17 23:34:50,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940978a575bc060ce161864'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:51,169 - lava_callback - INFO - Received callback for node 69409b0f575bc060ce162108 INFO: 172.21.0.1:33156 - "POST /node/69409b0f575bc060ce162108 HTTP/1.0" 202 Accepted 2025-12-17 23:34:51,835 - lava_callback - INFO - Received callback for node 69409b10575bc060ce16210b INFO: 172.21.0.1:33162 - "POST /node/69409b10575bc060ce16210b HTTP/1.0" 202 Accepted 2025-12-17 23:34:52,510 - lava_callback - INFO - Received callback for node 69409b12575bc060ce16210e INFO: 172.21.0.1:33170 - "POST /node/69409b12575bc060ce16210e HTTP/1.0" 202 Accepted 2025-12-17 23:34:52,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409797575bc060ce161888 2025-12-17 23:34:52,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409797575bc060ce161888 HTTP/1.1" 502 150 2025-12-17 23:34:52,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409797575bc060ce161888'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:53,172 - lava_callback - INFO - Received callback for node 69409b13575bc060ce162111 INFO: 172.21.0.1:33176 - "POST /node/69409b13575bc060ce162111 HTTP/1.0" 202 Accepted 2025-12-17 23:34:53,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409789575bc060ce16185c 2025-12-17 23:34:53,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409789575bc060ce16185c HTTP/1.1" 502 150 2025-12-17 23:34:53,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409789575bc060ce16185c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:55,217 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977d575bc060ce161838 2025-12-17 23:34:55,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977d575bc060ce161838 HTTP/1.1" 502 150 2025-12-17 23:34:55,219 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940977d575bc060ce161838'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:56,167 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409838575bc060ce161a1b INFO: 172.21.0.1:33192 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:34:56,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977e575bc060ce16183d 2025-12-17 23:34:56,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977e575bc060ce16183d HTTP/1.1" 502 150 2025-12-17 23:34:56,925 - lava_callback - ERROR - Error processing callback for node 6940977e575bc060ce16183d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940977e575bc060ce16183d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940977e575bc060ce16183d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940977e575bc060ce16183d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:34:56,927 - lava_callback - INFO - Starting async job submit for node 6940979a575bc060ce161894 2025-12-17 23:34:56,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:34:56,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979a575bc060ce161894 HTTP/1.1" 502 150 2025-12-17 23:34:56,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940979a575bc060ce161894'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:56,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940979a575bc060ce161894 2025-12-17 23:34:56,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979a575bc060ce161894 HTTP/1.1" 502 150 2025-12-17 23:34:56,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940979a575bc060ce161894'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:57,153 - lava_callback - INFO - Received callback for node 69409b15575bc060ce162114 INFO: 172.21.0.1:33202 - "POST /node/69409b15575bc060ce162114 HTTP/1.0" 202 Accepted 2025-12-17 23:34:57,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940978d575bc060ce161867 2025-12-17 23:34:57,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978d575bc060ce161867 HTTP/1.1" 502 150 2025-12-17 23:34:57,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940978d575bc060ce161867'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:57,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940978f575bc060ce16186a 2025-12-17 23:34:57,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978f575bc060ce16186a HTTP/1.1" 502 150 2025-12-17 23:34:57,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940978f575bc060ce16186a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:57,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409780575bc060ce161844 2025-12-17 23:34:57,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409780575bc060ce161844 HTTP/1.1" 502 150 2025-12-17 23:34:57,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409780575bc060ce161844'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:57,838 - lava_callback - INFO - Received callback for node 69409b20575bc060ce162132 INFO: 172.21.0.1:33214 - "POST /node/69409b20575bc060ce162132 HTTP/1.0" 202 Accepted 2025-12-17 23:34:58,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409793575bc060ce161872 2025-12-17 23:34:58,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409793575bc060ce161872 HTTP/1.1" 502 150 2025-12-17 23:34:58,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409793575bc060ce161872'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:58,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409795575bc060ce16187d 2025-12-17 23:34:58,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409795575bc060ce16187d HTTP/1.1" 502 150 2025-12-17 23:34:58,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409795575bc060ce16187d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:34:58,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940979a575bc060ce161894 2025-12-17 23:34:58,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979a575bc060ce161894 HTTP/1.1" 502 150 2025-12-17 23:34:58,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940979a575bc060ce161894'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:00,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409797575bc060ce161888 2025-12-17 23:35:00,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409797575bc060ce161888 HTTP/1.1" 502 150 2025-12-17 23:35:00,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409797575bc060ce161888'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:01,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409782575bc060ce16184d 2025-12-17 23:35:01,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409782575bc060ce16184d HTTP/1.1" 502 150 2025-12-17 23:35:01,919 - lava_callback - ERROR - Error processing callback for node 69409782575bc060ce16184d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409782575bc060ce16184d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409782575bc060ce16184d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409782575bc060ce16184d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:01,921 - lava_callback - INFO - Starting async job submit for node 6940979c575bc060ce16189b 2025-12-17 23:35:01,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:01,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979c575bc060ce16189b HTTP/1.1" 502 150 2025-12-17 23:35:01,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940979c575bc060ce16189b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:01,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940979c575bc060ce16189b 2025-12-17 23:35:01,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979c575bc060ce16189b HTTP/1.1" 502 150 2025-12-17 23:35:01,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940979c575bc060ce16189b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:02,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409784575bc060ce161852 2025-12-17 23:35:02,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409784575bc060ce161852 HTTP/1.1" 502 150 2025-12-17 23:35:02,388 - lava_callback - ERROR - Error processing callback for node 69409784575bc060ce161852: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409784575bc060ce161852 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409784575bc060ce161852 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409784575bc060ce161852 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:02,390 - lava_callback - INFO - Starting async job submit for node 6940979e575bc060ce1618a7 2025-12-17 23:35:02,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:02,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979e575bc060ce1618a7 HTTP/1.1" 502 150 2025-12-17 23:35:02,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940979e575bc060ce1618a7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:02,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940979e575bc060ce1618a7 2025-12-17 23:35:02,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979e575bc060ce1618a7 HTTP/1.1" 502 150 2025-12-17 23:35:02,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940979e575bc060ce1618a7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:02,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409786575bc060ce161859 2025-12-17 23:35:02,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409786575bc060ce161859 HTTP/1.1" 502 150 2025-12-17 23:35:02,444 - lava_callback - ERROR - Error processing callback for node 69409786575bc060ce161859: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409786575bc060ce161859 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409786575bc060ce161859 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409786575bc060ce161859 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:02,446 - lava_callback - INFO - Starting async job submit for node 694097a0575bc060ce1618aa 2025-12-17 23:35:02,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:02,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a0575bc060ce1618aa HTTP/1.1" 502 150 2025-12-17 23:35:02,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a0575bc060ce1618aa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:02,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a0575bc060ce1618aa 2025-12-17 23:35:02,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a0575bc060ce1618aa HTTP/1.1" 502 150 2025-12-17 23:35:02,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a0575bc060ce1618aa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:02,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694122fc459ff5fc64d2bae1 2025-12-17 23:35:02,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694122fc459ff5fc64d2bae1 HTTP/1.1" 502 150 2025-12-17 23:35:02,646 - lava_callback - ERROR - Error processing callback for node 694122fc459ff5fc64d2bae1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694122fc459ff5fc64d2bae1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694122fc459ff5fc64d2bae1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694122fc459ff5fc64d2bae1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:02,647 - lava_callback - INFO - Starting async job submit for node 694097a4575bc060ce1618b0 2025-12-17 23:35:02,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:02,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a4575bc060ce1618b0 HTTP/1.1" 502 150 2025-12-17 23:35:02,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a4575bc060ce1618b0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:02,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a4575bc060ce1618b0 2025-12-17 23:35:02,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a4575bc060ce1618b0 HTTP/1.1" 502 150 2025-12-17 23:35:02,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a4575bc060ce1618b0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:02,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940979a575bc060ce161894 2025-12-17 23:35:02,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979a575bc060ce161894 HTTP/1.1" 502 150 2025-12-17 23:35:02,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940979a575bc060ce161894'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:03,217 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409768575bc060ce1617fc INFO: 172.21.0.1:56420 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:35:03,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940979c575bc060ce16189b 2025-12-17 23:35:03,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979c575bc060ce16189b HTTP/1.1" 502 150 2025-12-17 23:35:03,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940979c575bc060ce16189b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:04,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940979e575bc060ce1618a7 2025-12-17 23:35:04,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979e575bc060ce1618a7 HTTP/1.1" 502 150 2025-12-17 23:35:04,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940979e575bc060ce1618a7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:04,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a0575bc060ce1618aa 2025-12-17 23:35:04,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a0575bc060ce1618aa HTTP/1.1" 502 150 2025-12-17 23:35:04,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a0575bc060ce1618aa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:04,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a4575bc060ce1618b0 2025-12-17 23:35:04,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a4575bc060ce1618b0 HTTP/1.1" 502 150 2025-12-17 23:35:04,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a4575bc060ce1618b0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56424 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:35:06,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940978a575bc060ce161864 2025-12-17 23:35:06,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978a575bc060ce161864 HTTP/1.1" 502 150 2025-12-17 23:35:06,634 - lava_callback - ERROR - Error processing callback for node 6940978a575bc060ce161864: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940978a575bc060ce161864 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940978a575bc060ce161864 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940978a575bc060ce161864 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:06,637 - lava_callback - INFO - Starting async job submit for node 694097a8575bc060ce1618be 2025-12-17 23:35:06,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:06,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a8575bc060ce1618be HTTP/1.1" 502 150 2025-12-17 23:35:06,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a8575bc060ce1618be'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:06,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a8575bc060ce1618be 2025-12-17 23:35:06,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a8575bc060ce1618be HTTP/1.1" 502 150 2025-12-17 23:35:06,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a8575bc060ce1618be'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:07,842 - lava_callback - INFO - Received callback for node 69409b1d575bc060ce16212f INFO: 172.21.0.1:56430 - "POST /node/69409b1d575bc060ce16212f HTTP/1.0" 202 Accepted 2025-12-17 23:35:07,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940979c575bc060ce16189b 2025-12-17 23:35:07,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979c575bc060ce16189b HTTP/1.1" 502 150 2025-12-17 23:35:07,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940979c575bc060ce16189b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:08,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940979e575bc060ce1618a7 2025-12-17 23:35:08,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979e575bc060ce1618a7 HTTP/1.1" 502 150 2025-12-17 23:35:08,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940979e575bc060ce1618a7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:08,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a0575bc060ce1618aa 2025-12-17 23:35:08,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a0575bc060ce1618aa HTTP/1.1" 502 150 2025-12-17 23:35:08,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a0575bc060ce1618aa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:08,530 - lava_callback - INFO - Received callback for node 69409b21575bc060ce162135 INFO: 172.21.0.1:56432 - "POST /node/69409b21575bc060ce162135 HTTP/1.0" 202 Accepted 2025-12-17 23:35:08,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a8575bc060ce1618be 2025-12-17 23:35:08,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a8575bc060ce1618be HTTP/1.1" 502 150 2025-12-17 23:35:08,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a8575bc060ce1618be'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:08,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a4575bc060ce1618b0 2025-12-17 23:35:08,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a4575bc060ce1618b0 HTTP/1.1" 502 150 2025-12-17 23:35:08,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a4575bc060ce1618b0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:09,217 - lava_callback - INFO - Received callback for node 69409b23575bc060ce16213c INFO: 172.21.0.1:56442 - "POST /node/69409b23575bc060ce16213c HTTP/1.0" 202 Accepted 2025-12-17 23:35:09,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409789575bc060ce16185c 2025-12-17 23:35:09,898 - lava_callback - INFO - Received callback for node 69409b25575bc060ce16213f INFO: 172.21.0.1:46250 - "POST /node/69409b25575bc060ce16213f HTTP/1.0" 202 Accepted 2025-12-17 23:35:10,560 - lava_callback - INFO - Received callback for node 69409b27575bc060ce162142 INFO: 172.21.0.1:46254 - "POST /node/69409b27575bc060ce162142 HTTP/1.0" 202 Accepted 2025-12-17 23:35:10,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940979a575bc060ce161894 2025-12-17 23:35:10,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979a575bc060ce161894 HTTP/1.1" 502 150 2025-12-17 23:35:10,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940979a575bc060ce161894'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:11,219 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940977d575bc060ce161838 2025-12-17 23:35:11,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940977d575bc060ce161838 HTTP/1.1" 502 150 2025-12-17 23:35:11,220 - lava_callback - ERROR - Error processing callback for node 6940977d575bc060ce161838: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940977d575bc060ce161838 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940977d575bc060ce161838 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940977d575bc060ce161838 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:11,222 - lava_callback - INFO - Starting async job submit for node 694097ab575bc060ce1618c1 2025-12-17 23:35:11,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:11,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ab575bc060ce1618c1 HTTP/1.1" 502 150 2025-12-17 23:35:11,251 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ab575bc060ce1618c1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:11,251 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ab575bc060ce1618c1 2025-12-17 23:35:11,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ab575bc060ce1618c1 HTTP/1.1" 502 150 2025-12-17 23:35:11,251 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ab575bc060ce1618c1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:11,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409838575bc060ce161a1b HTTP/1.1" 502 150 2025-12-17 23:35:11,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409838575bc060ce161a1b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:12,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a8575bc060ce1618be 2025-12-17 23:35:12,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a8575bc060ce1618be HTTP/1.1" 502 150 2025-12-17 23:35:12,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a8575bc060ce1618be'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:13,252 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ab575bc060ce1618c1 2025-12-17 23:35:13,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ab575bc060ce1618c1 HTTP/1.1" 502 150 2025-12-17 23:35:13,253 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ab575bc060ce1618c1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:13,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940978d575bc060ce161867 2025-12-17 23:35:13,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978d575bc060ce161867 HTTP/1.1" 502 150 2025-12-17 23:35:13,597 - lava_callback - ERROR - Error processing callback for node 6940978d575bc060ce161867: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940978d575bc060ce161867 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940978d575bc060ce161867 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940978d575bc060ce161867 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:13,599 - lava_callback - INFO - Starting async job submit for node 694097ad575bc060ce1618c4 2025-12-17 23:35:13,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:13,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940978f575bc060ce16186a 2025-12-17 23:35:13,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940978f575bc060ce16186a HTTP/1.1" 502 150 2025-12-17 23:35:13,618 - lava_callback - ERROR - Error processing callback for node 6940978f575bc060ce16186a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940978f575bc060ce16186a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940978f575bc060ce16186a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940978f575bc060ce16186a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:13,620 - lava_callback - INFO - Starting async job submit for node 694097af575bc060ce1618c7 2025-12-17 23:35:13,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:13,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ad575bc060ce1618c4 HTTP/1.1" 502 150 2025-12-17 23:35:13,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ad575bc060ce1618c4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:13,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ad575bc060ce1618c4 2025-12-17 23:35:13,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ad575bc060ce1618c4 HTTP/1.1" 502 150 2025-12-17 23:35:13,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ad575bc060ce1618c4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:13,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097af575bc060ce1618c7 HTTP/1.1" 502 150 2025-12-17 23:35:13,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097af575bc060ce1618c7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:13,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097af575bc060ce1618c7 2025-12-17 23:35:13,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097af575bc060ce1618c7 HTTP/1.1" 502 150 2025-12-17 23:35:13,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097af575bc060ce1618c7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:13,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409780575bc060ce161844 2025-12-17 23:35:13,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409780575bc060ce161844 HTTP/1.1" 502 150 2025-12-17 23:35:13,684 - lava_callback - ERROR - Error processing callback for node 69409780575bc060ce161844: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409780575bc060ce161844 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409780575bc060ce161844 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409780575bc060ce161844 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:13,685 - lava_callback - INFO - Starting async job submit for node 694097a6575bc060ce1618b7 2025-12-17 23:35:13,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:13,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a6575bc060ce1618b7 HTTP/1.1" 502 150 2025-12-17 23:35:13,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a6575bc060ce1618b7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:13,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a6575bc060ce1618b7 2025-12-17 23:35:13,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a6575bc060ce1618b7 HTTP/1.1" 502 150 2025-12-17 23:35:13,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a6575bc060ce1618b7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:14,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409793575bc060ce161872 2025-12-17 23:35:14,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409793575bc060ce161872 HTTP/1.1" 502 150 2025-12-17 23:35:14,538 - lava_callback - ERROR - Error processing callback for node 69409793575bc060ce161872: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409793575bc060ce161872 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409793575bc060ce161872 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409793575bc060ce161872 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:14,540 - lava_callback - INFO - Starting async job submit for node 694097b3575bc060ce1618cd 2025-12-17 23:35:14,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:14,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b3575bc060ce1618cd HTTP/1.1" 502 150 2025-12-17 23:35:14,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b3575bc060ce1618cd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:14,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b3575bc060ce1618cd 2025-12-17 23:35:14,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b3575bc060ce1618cd HTTP/1.1" 502 150 2025-12-17 23:35:14,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b3575bc060ce1618cd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:14,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409795575bc060ce16187d 2025-12-17 23:35:14,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409795575bc060ce16187d HTTP/1.1" 502 150 2025-12-17 23:35:14,610 - lava_callback - ERROR - Error processing callback for node 69409795575bc060ce16187d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409795575bc060ce16187d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409795575bc060ce16187d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409795575bc060ce16187d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:14,611 - lava_callback - INFO - Starting async job submit for node 694097b5575bc060ce1618d0 2025-12-17 23:35:14,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:14,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b5575bc060ce1618d0 HTTP/1.1" 502 150 2025-12-17 23:35:14,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b5575bc060ce1618d0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:14,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b5575bc060ce1618d0 2025-12-17 23:35:14,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b5575bc060ce1618d0 HTTP/1.1" 502 150 2025-12-17 23:35:14,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b5575bc060ce1618d0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46262 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:35:15,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ad575bc060ce1618c4 2025-12-17 23:35:15,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ad575bc060ce1618c4 HTTP/1.1" 502 150 2025-12-17 23:35:15,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ad575bc060ce1618c4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:15,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097af575bc060ce1618c7 2025-12-17 23:35:15,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097af575bc060ce1618c7 HTTP/1.1" 502 150 2025-12-17 23:35:15,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097af575bc060ce1618c7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:15,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a6575bc060ce1618b7 2025-12-17 23:35:15,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a6575bc060ce1618b7 HTTP/1.1" 502 150 2025-12-17 23:35:15,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a6575bc060ce1618b7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:15,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940979c575bc060ce16189b 2025-12-17 23:35:15,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979c575bc060ce16189b HTTP/1.1" 502 150 2025-12-17 23:35:15,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940979c575bc060ce16189b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:16,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940979e575bc060ce1618a7 2025-12-17 23:35:16,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979e575bc060ce1618a7 HTTP/1.1" 502 150 2025-12-17 23:35:16,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940979e575bc060ce1618a7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:16,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a0575bc060ce1618aa 2025-12-17 23:35:16,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a0575bc060ce1618aa HTTP/1.1" 502 150 2025-12-17 23:35:16,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a0575bc060ce1618aa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46270 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:35:16,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b3575bc060ce1618cd 2025-12-17 23:35:16,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b3575bc060ce1618cd HTTP/1.1" 502 150 2025-12-17 23:35:16,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b3575bc060ce1618cd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:16,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b5575bc060ce1618d0 2025-12-17 23:35:16,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b5575bc060ce1618d0 HTTP/1.1" 502 150 2025-12-17 23:35:16,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b5575bc060ce1618d0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:16,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a4575bc060ce1618b0 2025-12-17 23:35:16,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a4575bc060ce1618b0 HTTP/1.1" 502 150 2025-12-17 23:35:16,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a4575bc060ce1618b0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:16,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409797575bc060ce161888 2025-12-17 23:35:16,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409797575bc060ce161888 HTTP/1.1" 502 150 2025-12-17 23:35:16,696 - lava_callback - ERROR - Error processing callback for node 69409797575bc060ce161888: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409797575bc060ce161888 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409797575bc060ce161888 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409797575bc060ce161888 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:16,699 - lava_callback - INFO - Starting async job submit for node 694097b9575bc060ce1618d6 2025-12-17 23:35:16,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:16,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b9575bc060ce1618d6 HTTP/1.1" 502 150 2025-12-17 23:35:16,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b9575bc060ce1618d6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:16,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b9575bc060ce1618d6 2025-12-17 23:35:16,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b9575bc060ce1618d6 HTTP/1.1" 502 150 2025-12-17 23:35:16,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b9575bc060ce1618d6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:17,121 - lava_callback - INFO - Received callback for node 69409b28575bc060ce162145 INFO: 172.21.0.1:46274 - "POST /node/69409b28575bc060ce162145 HTTP/1.0" 202 Accepted 2025-12-17 23:35:17,253 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ab575bc060ce1618c1 2025-12-17 23:35:17,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ab575bc060ce1618c1 HTTP/1.1" 502 150 2025-12-17 23:35:17,254 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ab575bc060ce1618c1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:17,806 - lava_callback - INFO - Received callback for node 69409b2a575bc060ce162148 INFO: 172.21.0.1:46288 - "POST /node/69409b2a575bc060ce162148 HTTP/1.0" 202 Accepted 2025-12-17 23:35:18,481 - lava_callback - INFO - Received callback for node 69409b2d575bc060ce162157 INFO: 172.21.0.1:46292 - "POST /node/69409b2d575bc060ce162157 HTTP/1.0" 202 Accepted 2025-12-17 23:35:18,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b9575bc060ce1618d6 2025-12-17 23:35:18,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b9575bc060ce1618d6 HTTP/1.1" 502 150 2025-12-17 23:35:18,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b9575bc060ce1618d6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:19,159 - lava_callback - INFO - Received callback for node 69409b2f575bc060ce16215e INFO: 172.21.0.1:46308 - "POST /node/69409b2f575bc060ce16215e HTTP/1.0" 202 Accepted 2025-12-17 23:35:19,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ad575bc060ce1618c4 2025-12-17 23:35:19,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097af575bc060ce1618c7 2025-12-17 23:35:19,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a6575bc060ce1618b7 2025-12-17 23:35:19,833 - lava_callback - INFO - Received callback for node 69409b31575bc060ce162161 INFO: 172.21.0.1:58196 - "POST /node/69409b31575bc060ce162161 HTTP/1.0" 202 Accepted 2025-12-17 23:35:20,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b3575bc060ce1618cd 2025-12-17 23:35:20,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b5575bc060ce1618d0 2025-12-17 23:35:20,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a8575bc060ce1618be 2025-12-17 23:35:22,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b9575bc060ce1618d6 2025-12-17 23:35:25,254 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ab575bc060ce1618c1 2025-12-17 23:35:25,343 - lava_callback - INFO - Received callback for node 6942a5e81b61f1c6005fd3d5 INFO: 172.21.0.1:58204 - "POST /node/6942a5e81b61f1c6005fd3d5 HTTP/1.0" 202 Accepted 2025-12-17 23:35:26,164 - lava_callback - INFO - Received callback for node 69409c45575bc060ce162563 INFO: 172.21.0.1:58210 - "POST /node/69409c45575bc060ce162563 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:58222 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:35:26,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940979a575bc060ce161894 2025-12-17 23:35:27,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409838575bc060ce161a1b 2025-12-17 23:35:27,851 - lava_callback - INFO - Received callback for node 69409b32575bc060ce162164 INFO: 172.21.0.1:58232 - "POST /node/69409b32575bc060ce162164 HTTP/1.0" 202 Accepted 2025-12-17 23:35:28,495 - lava_callback - INFO - Received callback for node 69409b36575bc060ce16216e INFO: 172.21.0.1:58238 - "POST /node/69409b36575bc060ce16216e HTTP/1.0" 202 Accepted 2025-12-17 23:35:29,162 - lava_callback - INFO - Received callback for node 69409b3a575bc060ce162174 INFO: 172.21.0.1:58252 - "POST /node/69409b3a575bc060ce162174 HTTP/1.0" 202 Accepted 2025-12-17 23:35:29,861 - lava_callback - INFO - Received callback for node 69409b3e575bc060ce16217a INFO: 172.21.0.1:44346 - "POST /node/69409b3e575bc060ce16217a HTTP/1.0" 202 Accepted 2025-12-17 23:35:31,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940979c575bc060ce16189b 2025-12-17 23:35:32,128 - lava_callback - INFO - Received callback for node 69409b41575bc060ce162180 INFO: 172.21.0.1:44362 - "POST /node/69409b41575bc060ce162180 HTTP/1.0" 202 Accepted 2025-12-17 23:35:32,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940979e575bc060ce1618a7 2025-12-17 23:35:32,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a0575bc060ce1618aa 2025-12-17 23:35:32,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a4575bc060ce1618b0 2025-12-17 23:35:34,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409768575bc060ce1617fc HTTP/1.1" 502 150 2025-12-17 23:35:34,064 - lava_callback - ERROR - Error processing callback for node 69409768575bc060ce1617fc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409768575bc060ce1617fc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409768575bc060ce1617fc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409768575bc060ce1617fc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:34,066 - lava_callback - INFO - Starting async job submit for node 694097b1575bc060ce1618ca 2025-12-17 23:35:34,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:35,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097af575bc060ce1618c7 HTTP/1.1" 502 150 2025-12-17 23:35:35,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ad575bc060ce1618c4 HTTP/1.1" 502 150 2025-12-17 23:35:35,089 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097af575bc060ce1618c7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:35,089 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ad575bc060ce1618c4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:35,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b3575bc060ce1618cd HTTP/1.1" 502 150 2025-12-17 23:35:35,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b3575bc060ce1618cd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:36,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a8575bc060ce1618be HTTP/1.1" 502 150 2025-12-17 23:35:36,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a8575bc060ce1618be'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:36,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b5575bc060ce1618d0 HTTP/1.1" 502 150 2025-12-17 23:35:36,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b5575bc060ce1618d0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44364 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:35:37,160 - lava_callback - INFO - Received callback for node 69409b34575bc060ce162167 INFO: 172.21.0.1:44380 - "POST /node/69409b34575bc060ce162167 HTTP/1.0" 202 Accepted 2025-12-17 23:35:37,832 - lava_callback - INFO - Received callback for node 69409b38575bc060ce162171 INFO: 172.21.0.1:44388 - "POST /node/69409b38575bc060ce162171 HTTP/1.0" 202 Accepted 2025-12-17 23:35:38,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b9575bc060ce1618d6 HTTP/1.1" 502 150 2025-12-17 23:35:38,160 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b9575bc060ce1618d6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:38,503 - lava_callback - INFO - Received callback for node 69409b3c575bc060ce162177 INFO: 172.21.0.1:44404 - "POST /node/69409b3c575bc060ce162177 HTTP/1.0" 202 Accepted 2025-12-17 23:35:39,172 - lava_callback - INFO - Received callback for node 69409b3f575bc060ce16217d INFO: 172.21.0.1:44406 - "POST /node/69409b3f575bc060ce16217d HTTP/1.0" 202 Accepted 2025-12-17 23:35:40,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409789575bc060ce16185c HTTP/1.1" 502 150 2025-12-17 23:35:40,208 - lava_callback - ERROR - Error processing callback for node 69409789575bc060ce16185c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409789575bc060ce16185c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409789575bc060ce16185c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409789575bc060ce16185c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:40,210 - lava_callback - INFO - Starting async job submit for node 694097b7575bc060ce1618d3 2025-12-17 23:35:40,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:40,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b7575bc060ce1618d3 HTTP/1.1" 502 150 2025-12-17 23:35:40,217 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b7575bc060ce1618d3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:40,217 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b7575bc060ce1618d3 2025-12-17 23:35:40,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b7575bc060ce1618d3 HTTP/1.1" 502 150 2025-12-17 23:35:40,218 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b7575bc060ce1618d3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:40,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ab575bc060ce1618c1 HTTP/1.1" 502 150 2025-12-17 23:35:40,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ab575bc060ce1618c1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:42,218 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b7575bc060ce1618d3 2025-12-17 23:35:42,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b7575bc060ce1618d3 HTTP/1.1" 502 150 2025-12-17 23:35:42,220 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b7575bc060ce1618d3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:42,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979a575bc060ce161894 HTTP/1.1" 502 150 2025-12-17 23:35:42,256 - lava_callback - ERROR - Error processing callback for node 6940979a575bc060ce161894: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940979a575bc060ce161894 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940979a575bc060ce161894 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940979a575bc060ce161894 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:42,258 - lava_callback - INFO - Starting async job submit for node 694097bb575bc060ce1618dd 2025-12-17 23:35:42,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:42,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bb575bc060ce1618dd HTTP/1.1" 502 150 2025-12-17 23:35:42,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097bb575bc060ce1618dd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:42,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097bb575bc060ce1618dd 2025-12-17 23:35:42,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bb575bc060ce1618dd HTTP/1.1" 502 150 2025-12-17 23:35:42,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097bb575bc060ce1618dd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:43,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409838575bc060ce161a1b HTTP/1.1" 502 150 2025-12-17 23:35:43,024 - lava_callback - ERROR - Error processing callback for node 69409838575bc060ce161a1b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409838575bc060ce161a1b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409838575bc060ce161a1b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409838575bc060ce161a1b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:43,026 - lava_callback - INFO - Starting async job submit for node 694097bc575bc060ce1618e0 2025-12-17 23:35:43,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:43,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bc575bc060ce1618e0 HTTP/1.1" 502 150 2025-12-17 23:35:43,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097bc575bc060ce1618e0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:43,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097bc575bc060ce1618e0 2025-12-17 23:35:43,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bc575bc060ce1618e0 HTTP/1.1" 502 150 2025-12-17 23:35:43,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097bc575bc060ce1618e0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:43,089 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097af575bc060ce1618c7 2025-12-17 23:35:43,089 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ad575bc060ce1618c4 2025-12-17 23:35:43,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097af575bc060ce1618c7 HTTP/1.1" 502 150 2025-12-17 23:35:43,090 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097af575bc060ce1618c7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:43,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ad575bc060ce1618c4 HTTP/1.1" 502 150 2025-12-17 23:35:43,090 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ad575bc060ce1618c4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:43,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b3575bc060ce1618cd 2025-12-17 23:35:43,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b3575bc060ce1618cd HTTP/1.1" 502 150 2025-12-17 23:35:43,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b3575bc060ce1618cd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:44,113 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b5575bc060ce1618d0 2025-12-17 23:35:44,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b5575bc060ce1618d0 HTTP/1.1" 502 150 2025-12-17 23:35:44,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b5575bc060ce1618d0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:44,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097bb575bc060ce1618dd 2025-12-17 23:35:44,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bb575bc060ce1618dd HTTP/1.1" 502 150 2025-12-17 23:35:44,268 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097bb575bc060ce1618dd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:45,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097bc575bc060ce1618e0 2025-12-17 23:35:45,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bc575bc060ce1618e0 HTTP/1.1" 502 150 2025-12-17 23:35:45,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097bc575bc060ce1618e0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:46,161 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b9575bc060ce1618d6 2025-12-17 23:35:46,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b9575bc060ce1618d6 HTTP/1.1" 502 150 2025-12-17 23:35:46,162 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b9575bc060ce1618d6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:46,220 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b7575bc060ce1618d3 2025-12-17 23:35:46,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b7575bc060ce1618d3 HTTP/1.1" 502 150 2025-12-17 23:35:46,221 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b7575bc060ce1618d3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56766 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:35:47,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979c575bc060ce16189b HTTP/1.1" 502 150 2025-12-17 23:35:47,376 - lava_callback - ERROR - Error processing callback for node 6940979c575bc060ce16189b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940979c575bc060ce16189b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940979c575bc060ce16189b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940979c575bc060ce16189b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:47,379 - lava_callback - INFO - Starting async job submit for node 694097be575bc060ce1618e3 2025-12-17 23:35:47,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:47,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097be575bc060ce1618e3 HTTP/1.1" 502 150 2025-12-17 23:35:47,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097be575bc060ce1618e3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:47,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097be575bc060ce1618e3 2025-12-17 23:35:47,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097be575bc060ce1618e3 HTTP/1.1" 502 150 2025-12-17 23:35:47,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097be575bc060ce1618e3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:47,851 - lava_callback - INFO - Received callback for node 69409b42575bc060ce162183 INFO: 172.21.0.1:56772 - "POST /node/69409b42575bc060ce162183 HTTP/1.0" 202 Accepted 2025-12-17 23:35:47,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a0575bc060ce1618aa HTTP/1.1" 502 150 2025-12-17 23:35:47,888 - lava_callback - ERROR - Error processing callback for node 694097a0575bc060ce1618aa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097a0575bc060ce1618aa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097a0575bc060ce1618aa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097a0575bc060ce1618aa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:47,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940979e575bc060ce1618a7 HTTP/1.1" 502 150 2025-12-17 23:35:47,891 - lava_callback - ERROR - Error processing callback for node 6940979e575bc060ce1618a7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940979e575bc060ce1618a7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940979e575bc060ce1618a7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940979e575bc060ce1618a7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:47,891 - lava_callback - INFO - Starting async job submit for node 694097bf575bc060ce1618e6 2025-12-17 23:35:47,894 - lava_callback - INFO - Starting async job submit for node 694097c1575bc060ce1618ec 2025-12-17 23:35:47,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:47,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:47,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c1575bc060ce1618ec HTTP/1.1" 502 150 2025-12-17 23:35:47,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c1575bc060ce1618ec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:47,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c1575bc060ce1618ec 2025-12-17 23:35:47,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bf575bc060ce1618e6 HTTP/1.1" 502 150 2025-12-17 23:35:47,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097bf575bc060ce1618e6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:47,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097bf575bc060ce1618e6 2025-12-17 23:35:47,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c1575bc060ce1618ec HTTP/1.1" 502 150 2025-12-17 23:35:47,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c1575bc060ce1618ec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:47,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bf575bc060ce1618e6 HTTP/1.1" 502 150 2025-12-17 23:35:47,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097bf575bc060ce1618e6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:48,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a4575bc060ce1618b0 HTTP/1.1" 502 150 2025-12-17 23:35:48,144 - lava_callback - ERROR - Error processing callback for node 694097a4575bc060ce1618b0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097a4575bc060ce1618b0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097a4575bc060ce1618b0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097a4575bc060ce1618b0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:48,146 - lava_callback - INFO - Starting async job submit for node 694097c3575bc060ce1618ef 2025-12-17 23:35:48,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:48,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c3575bc060ce1618ef HTTP/1.1" 502 150 2025-12-17 23:35:48,155 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c3575bc060ce1618ef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:48,155 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c3575bc060ce1618ef 2025-12-17 23:35:48,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c3575bc060ce1618ef HTTP/1.1" 502 150 2025-12-17 23:35:48,161 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c3575bc060ce1618ef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:48,268 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097bb575bc060ce1618dd 2025-12-17 23:35:48,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bb575bc060ce1618dd HTTP/1.1" 502 150 2025-12-17 23:35:48,270 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097bb575bc060ce1618dd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:48,531 - lava_callback - INFO - Received callback for node 69409b44575bc060ce162186 INFO: 172.21.0.1:56784 - "POST /node/69409b44575bc060ce162186 HTTP/1.0" 202 Accepted 2025-12-17 23:35:49,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097bc575bc060ce1618e0 2025-12-17 23:35:49,216 - lava_callback - INFO - Received callback for node 69409b46575bc060ce162189 INFO: 172.21.0.1:56796 - "POST /node/69409b46575bc060ce162189 HTTP/1.0" 202 Accepted 2025-12-17 23:35:49,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097be575bc060ce1618e3 2025-12-17 23:35:49,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097be575bc060ce1618e3 HTTP/1.1" 502 150 2025-12-17 23:35:49,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097be575bc060ce1618e3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:49,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b1575bc060ce1618ca HTTP/1.1" 502 150 2025-12-17 23:35:49,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b1575bc060ce1618ca'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:49,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b1575bc060ce1618ca 2025-12-17 23:35:49,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b1575bc060ce1618ca HTTP/1.1" 502 150 2025-12-17 23:35:49,425 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b1575bc060ce1618ca'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:49,897 - lava_callback - INFO - Received callback for node 69409b47575bc060ce16218c INFO: 172.21.0.1:55596 - "POST /node/69409b47575bc060ce16218c HTTP/1.0" 202 Accepted 2025-12-17 23:35:49,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c1575bc060ce1618ec 2025-12-17 23:35:49,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097bf575bc060ce1618e6 2025-12-17 23:35:49,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c1575bc060ce1618ec HTTP/1.1" 502 150 2025-12-17 23:35:49,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c1575bc060ce1618ec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:49,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bf575bc060ce1618e6 HTTP/1.1" 502 150 2025-12-17 23:35:49,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097bf575bc060ce1618e6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:50,161 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c3575bc060ce1618ef 2025-12-17 23:35:50,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c3575bc060ce1618ef HTTP/1.1" 502 150 2025-12-17 23:35:50,163 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c3575bc060ce1618ef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:50,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a6575bc060ce1618b7 HTTP/1.1" 502 150 2025-12-17 23:35:50,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a6575bc060ce1618b7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:50,548 - lava_callback - INFO - Received callback for node 69409b49575bc060ce16218f INFO: 172.21.0.1:55602 - "POST /node/69409b49575bc060ce16218f HTTP/1.0" 202 Accepted 2025-12-17 23:35:51,212 - lava_callback - INFO - Received callback for node 69409b4b575bc060ce162192 INFO: 172.21.0.1:55618 - "POST /node/69409b4b575bc060ce162192 HTTP/1.0" 202 Accepted 2025-12-17 23:35:51,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b1575bc060ce1618ca 2025-12-17 23:35:51,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b1575bc060ce1618ca HTTP/1.1" 502 150 2025-12-17 23:35:51,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b1575bc060ce1618ca'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:51,873 - lava_callback - INFO - Received callback for node 69409b4d575bc060ce162195 INFO: 172.21.0.1:55628 - "POST /node/69409b4d575bc060ce162195 HTTP/1.0" 202 Accepted 2025-12-17 23:35:52,113 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a8575bc060ce1618be 2025-12-17 23:35:52,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a8575bc060ce1618be HTTP/1.1" 502 150 2025-12-17 23:35:52,114 - lava_callback - ERROR - Error processing callback for node 694097a8575bc060ce1618be: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097a8575bc060ce1618be (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097a8575bc060ce1618be (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097a8575bc060ce1618be (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:52,116 - lava_callback - INFO - Starting async job submit for node 694097c0575bc060ce1618e9 2025-12-17 23:35:52,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:52,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c0575bc060ce1618e9 HTTP/1.1" 502 150 2025-12-17 23:35:52,122 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c0575bc060ce1618e9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:52,122 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c0575bc060ce1618e9 2025-12-17 23:35:52,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c0575bc060ce1618e9 HTTP/1.1" 502 150 2025-12-17 23:35:52,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c0575bc060ce1618e9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:52,532 - lava_callback - INFO - Received callback for node 69409b4e575bc060ce162198 INFO: 172.21.0.1:55634 - "POST /node/69409b4e575bc060ce162198 HTTP/1.0" 202 Accepted 2025-12-17 23:35:53,187 - lava_callback - INFO - Received callback for node 69409b52575bc060ce16219e INFO: 172.21.0.1:55638 - "POST /node/69409b52575bc060ce16219e HTTP/1.0" 202 Accepted 2025-12-17 23:35:53,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097be575bc060ce1618e3 2025-12-17 23:35:53,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097be575bc060ce1618e3 HTTP/1.1" 502 150 2025-12-17 23:35:53,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097be575bc060ce1618e3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:53,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c1575bc060ce1618ec 2025-12-17 23:35:53,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097bf575bc060ce1618e6 2025-12-17 23:35:53,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c1575bc060ce1618ec HTTP/1.1" 502 150 2025-12-17 23:35:53,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c1575bc060ce1618ec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:53,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bf575bc060ce1618e6 HTTP/1.1" 502 150 2025-12-17 23:35:53,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097bf575bc060ce1618e6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:54,123 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c0575bc060ce1618e9 2025-12-17 23:35:54,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c0575bc060ce1618e9 HTTP/1.1" 502 150 2025-12-17 23:35:54,125 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c0575bc060ce1618e9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:54,163 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c3575bc060ce1618ef 2025-12-17 23:35:54,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c3575bc060ce1618ef HTTP/1.1" 502 150 2025-12-17 23:35:54,164 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c3575bc060ce1618ef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:54,221 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b7575bc060ce1618d3 2025-12-17 23:35:54,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b7575bc060ce1618d3 HTTP/1.1" 502 150 2025-12-17 23:35:54,222 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b7575bc060ce1618d3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:55,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b1575bc060ce1618ca 2025-12-17 23:35:55,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b1575bc060ce1618ca HTTP/1.1" 502 150 2025-12-17 23:35:55,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b1575bc060ce1618ca'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:56,270 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097bb575bc060ce1618dd 2025-12-17 23:35:56,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bb575bc060ce1618dd HTTP/1.1" 502 150 2025-12-17 23:35:56,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097bb575bc060ce1618dd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55642 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:35:56,710 - lava_callback - INFO - Received callback for node 694098e0575bc060ce161b98 INFO: 172.21.0.1:55650 - "POST /node/694098e0575bc060ce161b98 HTTP/1.0" 202 Accepted 2025-12-17 23:35:56,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ab575bc060ce1618c1 2025-12-17 23:35:56,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ab575bc060ce1618c1 HTTP/1.1" 502 150 2025-12-17 23:35:56,721 - lava_callback - ERROR - Error processing callback for node 694097ab575bc060ce1618c1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ab575bc060ce1618c1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ab575bc060ce1618c1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ab575bc060ce1618c1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:56,723 - lava_callback - INFO - Starting async job submit for node 694097c5575bc060ce1618f6 2025-12-17 23:35:56,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:56,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c5575bc060ce1618f6 HTTP/1.1" 502 150 2025-12-17 23:35:56,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c5575bc060ce1618f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:56,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c5575bc060ce1618f6 2025-12-17 23:35:56,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c5575bc060ce1618f6 HTTP/1.1" 502 150 2025-12-17 23:35:56,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c5575bc060ce1618f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:57,216 - lava_callback - INFO - Received callback for node 69409b54575bc060ce1621a5 INFO: 172.21.0.1:55666 - "POST /node/69409b54575bc060ce1621a5 HTTP/1.0" 202 Accepted 2025-12-17 23:35:57,906 - lava_callback - INFO - Received callback for node 69409b5a575bc060ce1621b0 INFO: 172.21.0.1:55670 - "POST /node/69409b5a575bc060ce1621b0 HTTP/1.0" 202 Accepted 2025-12-17 23:35:58,125 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c0575bc060ce1618e9 2025-12-17 23:35:58,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c0575bc060ce1618e9 HTTP/1.1" 502 150 2025-12-17 23:35:58,126 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c0575bc060ce1618e9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:58,357 - lava_callback - INFO - Received callback for node 69409c44575bc060ce16255c INFO: 172.21.0.1:55684 - "POST /node/69409c44575bc060ce16255c HTTP/1.0" 202 Accepted 2025-12-17 23:35:58,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a6575bc060ce1618b7 2025-12-17 23:35:58,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a6575bc060ce1618b7 HTTP/1.1" 502 150 2025-12-17 23:35:58,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097a6575bc060ce1618b7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:58,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c5575bc060ce1618f6 2025-12-17 23:35:58,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c5575bc060ce1618f6 HTTP/1.1" 502 150 2025-12-17 23:35:58,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c5575bc060ce1618f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:59,090 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097af575bc060ce1618c7 2025-12-17 23:35:59,091 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ad575bc060ce1618c4 2025-12-17 23:35:59,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097af575bc060ce1618c7 HTTP/1.1" 502 150 2025-12-17 23:35:59,092 - lava_callback - ERROR - Error processing callback for node 694097af575bc060ce1618c7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097af575bc060ce1618c7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097af575bc060ce1618c7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097af575bc060ce1618c7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:59,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ad575bc060ce1618c4 HTTP/1.1" 502 150 2025-12-17 23:35:59,094 - lava_callback - INFO - Starting async job submit for node 694097c7575bc060ce1618f9 2025-12-17 23:35:59,094 - lava_callback - ERROR - Error processing callback for node 694097ad575bc060ce1618c4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ad575bc060ce1618c4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ad575bc060ce1618c4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ad575bc060ce1618c4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:59,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:59,096 - lava_callback - INFO - Starting async job submit for node 694097c8575bc060ce1618fc 2025-12-17 23:35:59,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:59,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c8575bc060ce1618fc HTTP/1.1" 502 150 2025-12-17 23:35:59,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c8575bc060ce1618fc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:59,103 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c8575bc060ce1618fc 2025-12-17 23:35:59,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c8575bc060ce1618fc HTTP/1.1" 502 150 2025-12-17 23:35:59,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c8575bc060ce1618fc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:59,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c7575bc060ce1618f9 HTTP/1.1" 502 150 2025-12-17 23:35:59,108 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c7575bc060ce1618f9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:59,108 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c7575bc060ce1618f9 2025-12-17 23:35:59,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c7575bc060ce1618f9 HTTP/1.1" 502 150 2025-12-17 23:35:59,109 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c7575bc060ce1618f9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:59,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b3575bc060ce1618cd 2025-12-17 23:35:59,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b3575bc060ce1618cd HTTP/1.1" 502 150 2025-12-17 23:35:59,859 - lava_callback - ERROR - Error processing callback for node 694097b3575bc060ce1618cd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097b3575bc060ce1618cd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097b3575bc060ce1618cd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097b3575bc060ce1618cd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:35:59,861 - lava_callback - INFO - Starting async job submit for node 694097c9575bc060ce1618ff 2025-12-17 23:35:59,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:35:59,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c9575bc060ce1618ff HTTP/1.1" 502 150 2025-12-17 23:35:59,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c9575bc060ce1618ff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:35:59,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c9575bc060ce1618ff 2025-12-17 23:35:59,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c9575bc060ce1618ff HTTP/1.1" 502 150 2025-12-17 23:35:59,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c9575bc060ce1618ff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:00,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b5575bc060ce1618d0 2025-12-17 23:36:01,104 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c8575bc060ce1618fc 2025-12-17 23:36:01,109 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c7575bc060ce1618f9 2025-12-17 23:36:01,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c7575bc060ce1618f9 HTTP/1.1" 502 150 2025-12-17 23:36:01,110 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c7575bc060ce1618f9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:01,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097be575bc060ce1618e3 2025-12-17 23:36:01,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097be575bc060ce1618e3 HTTP/1.1" 502 150 2025-12-17 23:36:01,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097be575bc060ce1618e3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:01,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c9575bc060ce1618ff 2025-12-17 23:36:01,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c9575bc060ce1618ff HTTP/1.1" 502 150 2025-12-17 23:36:01,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c9575bc060ce1618ff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:01,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c1575bc060ce1618ec 2025-12-17 23:36:01,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097bf575bc060ce1618e6 2025-12-17 23:36:01,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c1575bc060ce1618ec HTTP/1.1" 502 150 2025-12-17 23:36:01,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c1575bc060ce1618ec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:01,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bf575bc060ce1618e6 HTTP/1.1" 502 150 2025-12-17 23:36:01,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097bf575bc060ce1618e6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:02,030 - lava_callback - INFO - Received callback for node 6942bc911b61f1c6005fe49c INFO: 172.21.0.1:43394 - "POST /node/6942bc911b61f1c6005fe49c HTTP/1.0" 202 Accepted 2025-12-17 23:36:02,162 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b9575bc060ce1618d6 2025-12-17 23:36:02,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b9575bc060ce1618d6 HTTP/1.1" 502 150 2025-12-17 23:36:02,165 - lava_callback - ERROR - Error processing callback for node 694097b9575bc060ce1618d6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097b9575bc060ce1618d6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097b9575bc060ce1618d6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097b9575bc060ce1618d6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:02,166 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c3575bc060ce1618ef 2025-12-17 23:36:02,168 - lava_callback - INFO - Starting async job submit for node 694097ca575bc060ce161902 2025-12-17 23:36:02,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c3575bc060ce1618ef HTTP/1.1" 502 150 2025-12-17 23:36:02,168 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c3575bc060ce1618ef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:02,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:02,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ca575bc060ce161902 HTTP/1.1" 502 150 2025-12-17 23:36:02,177 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ca575bc060ce161902'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:02,177 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ca575bc060ce161902 2025-12-17 23:36:02,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ca575bc060ce161902 HTTP/1.1" 502 150 2025-12-17 23:36:02,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ca575bc060ce161902'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:02,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c5575bc060ce1618f6 2025-12-17 23:36:02,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c5575bc060ce1618f6 HTTP/1.1" 502 150 2025-12-17 23:36:02,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c5575bc060ce1618f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43402 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:36:03,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b1575bc060ce1618ca 2025-12-17 23:36:03,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b1575bc060ce1618ca HTTP/1.1" 502 150 2025-12-17 23:36:03,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097b1575bc060ce1618ca'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:04,178 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ca575bc060ce161902 2025-12-17 23:36:04,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ca575bc060ce161902 HTTP/1.1" 502 150 2025-12-17 23:36:04,179 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ca575bc060ce161902'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:04,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bc575bc060ce1618e0 HTTP/1.1" 502 150 2025-12-17 23:36:04,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097bc575bc060ce1618e0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:05,110 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c7575bc060ce1618f9 2025-12-17 23:36:05,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c7575bc060ce1618f9 HTTP/1.1" 502 150 2025-12-17 23:36:05,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c7575bc060ce1618f9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:05,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c9575bc060ce1618ff 2025-12-17 23:36:05,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c9575bc060ce1618ff HTTP/1.1" 502 150 2025-12-17 23:36:05,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c9575bc060ce1618ff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:06,126 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c0575bc060ce1618e9 2025-12-17 23:36:06,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c0575bc060ce1618e9 HTTP/1.1" 502 150 2025-12-17 23:36:06,127 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c0575bc060ce1618e9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43416 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:36:07,869 - lava_callback - INFO - Received callback for node 69409b58575bc060ce1621ad INFO: 172.21.0.1:43424 - "POST /node/69409b58575bc060ce1621ad HTTP/1.0" 202 Accepted 2025-12-17 23:36:08,180 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ca575bc060ce161902 2025-12-17 23:36:08,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ca575bc060ce161902 HTTP/1.1" 502 150 2025-12-17 23:36:08,181 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ca575bc060ce161902'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:08,524 - lava_callback - INFO - Received callback for node 69409b5b575bc060ce1621b3 INFO: 172.21.0.1:43432 - "POST /node/69409b5b575bc060ce1621b3 HTTP/1.0" 202 Accepted 2025-12-17 23:36:08,667 - lava_callback - INFO - Received callback for node 69409b56575bc060ce1621aa INFO: 172.21.0.1:43448 - "POST /node/69409b56575bc060ce1621aa HTTP/1.0" 202 Accepted 2025-12-17 23:36:09,194 - lava_callback - INFO - Received callback for node 69409b5f575bc060ce1621c0 INFO: 172.21.0.1:43462 - "POST /node/69409b5f575bc060ce1621c0 HTTP/1.0" 202 Accepted 2025-12-17 23:36:09,487 - lava_callback - INFO - Received callback for node 69409b5d575bc060ce1621b6 INFO: 172.21.0.1:43466 - "POST /node/69409b5d575bc060ce1621b6 HTTP/1.0" 202 Accepted 2025-12-17 23:36:09,871 - lava_callback - INFO - Received callback for node 69409b62575bc060ce1621c6 INFO: 172.21.0.1:60260 - "POST /node/69409b62575bc060ce1621c6 HTTP/1.0" 202 Accepted 2025-12-17 23:36:10,222 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b7575bc060ce1618d3 2025-12-17 23:36:10,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b7575bc060ce1618d3 HTTP/1.1" 502 150 2025-12-17 23:36:10,224 - lava_callback - ERROR - Error processing callback for node 694097b7575bc060ce1618d3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097b7575bc060ce1618d3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097b7575bc060ce1618d3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097b7575bc060ce1618d3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:10,226 - lava_callback - INFO - Starting async job submit for node 694097cc575bc060ce161905 2025-12-17 23:36:10,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:10,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097cc575bc060ce161905 HTTP/1.1" 502 150 2025-12-17 23:36:10,255 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097cc575bc060ce161905'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:10,255 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097cc575bc060ce161905 2025-12-17 23:36:10,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097cc575bc060ce161905 HTTP/1.1" 502 150 2025-12-17 23:36:10,256 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097cc575bc060ce161905'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:10,265 - lava_callback - INFO - Received callback for node 69409b61575bc060ce1621c3 INFO: 172.21.0.1:60264 - "POST /node/69409b61575bc060ce1621c3 HTTP/1.0" 202 Accepted 2025-12-17 23:36:10,551 - lava_callback - INFO - Received callback for node 69409b64575bc060ce1621c9 INFO: 172.21.0.1:60270 - "POST /node/69409b64575bc060ce1621c9 HTTP/1.0" 202 Accepted 2025-12-17 23:36:10,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c5575bc060ce1618f6 2025-12-17 23:36:10,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c5575bc060ce1618f6 HTTP/1.1" 502 150 2025-12-17 23:36:10,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c5575bc060ce1618f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:11,207 - lava_callback - INFO - Received callback for node 69409b66575bc060ce1621cc INFO: 172.21.0.1:60272 - "POST /node/69409b66575bc060ce1621cc HTTP/1.0" 202 Accepted 2025-12-17 23:36:12,256 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097cc575bc060ce161905 2025-12-17 23:36:12,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097bb575bc060ce1618dd 2025-12-17 23:36:12,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bb575bc060ce1618dd HTTP/1.1" 502 150 2025-12-17 23:36:12,273 - lava_callback - ERROR - Error processing callback for node 694097bb575bc060ce1618dd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097bb575bc060ce1618dd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097bb575bc060ce1618dd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097bb575bc060ce1618dd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:12,275 - lava_callback - INFO - Starting async job submit for node 694097cd575bc060ce161908 2025-12-17 23:36:12,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:12,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097cd575bc060ce161908 HTTP/1.1" 502 150 2025-12-17 23:36:12,306 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097cd575bc060ce161908'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:12,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097cd575bc060ce161908 2025-12-17 23:36:12,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097cd575bc060ce161908 HTTP/1.1" 502 150 2025-12-17 23:36:12,307 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097cd575bc060ce161908'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:12,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097bc575bc060ce1618e0 2025-12-17 23:36:12,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bc575bc060ce1618e0 HTTP/1.1" 502 150 2025-12-17 23:36:12,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097bc575bc060ce1618e0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:13,112 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c7575bc060ce1618f9 2025-12-17 23:36:13,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c7575bc060ce1618f9 HTTP/1.1" 502 150 2025-12-17 23:36:13,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c7575bc060ce1618f9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:13,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c9575bc060ce1618ff 2025-12-17 23:36:13,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c9575bc060ce1618ff HTTP/1.1" 502 150 2025-12-17 23:36:13,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c9575bc060ce1618ff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:14,307 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097cd575bc060ce161908 2025-12-17 23:36:14,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097cd575bc060ce161908 HTTP/1.1" 502 150 2025-12-17 23:36:14,309 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097cd575bc060ce161908'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:14,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097a6575bc060ce1618b7 2025-12-17 23:36:14,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097a6575bc060ce1618b7 HTTP/1.1" 502 150 2025-12-17 23:36:14,458 - lava_callback - ERROR - Error processing callback for node 694097a6575bc060ce1618b7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097a6575bc060ce1618b7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097a6575bc060ce1618b7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097a6575bc060ce1618b7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:14,461 - lava_callback - INFO - Starting async job submit for node 694097ce575bc060ce161913 2025-12-17 23:36:14,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:14,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ce575bc060ce161913 HTTP/1.1" 502 150 2025-12-17 23:36:14,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ce575bc060ce161913'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:14,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ce575bc060ce161913 2025-12-17 23:36:14,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ce575bc060ce161913 HTTP/1.1" 502 150 2025-12-17 23:36:14,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ce575bc060ce161913'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60282 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:36:15,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b5575bc060ce1618d0 HTTP/1.1" 502 150 2025-12-17 23:36:15,536 - lava_callback - ERROR - Error processing callback for node 694097b5575bc060ce1618d0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097b5575bc060ce1618d0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097b5575bc060ce1618d0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097b5575bc060ce1618d0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:15,538 - lava_callback - INFO - Starting async job submit for node 694097d0575bc060ce16191c 2025-12-17 23:36:15,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:15,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d0575bc060ce16191c HTTP/1.1" 502 150 2025-12-17 23:36:15,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d0575bc060ce16191c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:15,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d0575bc060ce16191c 2025-12-17 23:36:15,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d0575bc060ce16191c HTTP/1.1" 502 150 2025-12-17 23:36:15,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d0575bc060ce16191c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:16,181 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ca575bc060ce161902 2025-12-17 23:36:16,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ca575bc060ce161902 HTTP/1.1" 502 150 2025-12-17 23:36:16,182 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ca575bc060ce161902'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:16,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ce575bc060ce161913 2025-12-17 23:36:16,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ce575bc060ce161913 HTTP/1.1" 502 150 2025-12-17 23:36:16,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ce575bc060ce161913'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60298 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:36:16,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c8575bc060ce1618fc HTTP/1.1" 502 150 2025-12-17 23:36:16,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c8575bc060ce1618fc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:17,191 - lava_callback - INFO - Received callback for node 69409b5e575bc060ce1621b9 INFO: 172.21.0.1:60308 - "POST /node/69409b5e575bc060ce1621b9 HTTP/1.0" 202 Accepted 2025-12-17 23:36:17,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097be575bc060ce1618e3 2025-12-17 23:36:17,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097be575bc060ce1618e3 HTTP/1.1" 502 150 2025-12-17 23:36:17,403 - lava_callback - ERROR - Error processing callback for node 694097be575bc060ce1618e3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097be575bc060ce1618e3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097be575bc060ce1618e3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097be575bc060ce1618e3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:17,405 - lava_callback - INFO - Starting async job submit for node 694097d2575bc060ce16191f 2025-12-17 23:36:17,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:17,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d2575bc060ce16191f HTTP/1.1" 502 150 2025-12-17 23:36:17,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d2575bc060ce16191f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:17,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d2575bc060ce16191f 2025-12-17 23:36:17,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d2575bc060ce16191f HTTP/1.1" 502 150 2025-12-17 23:36:17,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d2575bc060ce16191f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:17,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d0575bc060ce16191c 2025-12-17 23:36:17,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d0575bc060ce16191c HTTP/1.1" 502 150 2025-12-17 23:36:17,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d0575bc060ce16191c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:17,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c1575bc060ce1618ec 2025-12-17 23:36:17,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097bf575bc060ce1618e6 2025-12-17 23:36:17,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bf575bc060ce1618e6 HTTP/1.1" 502 150 2025-12-17 23:36:17,910 - lava_callback - ERROR - Error processing callback for node 694097bf575bc060ce1618e6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097bf575bc060ce1618e6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097bf575bc060ce1618e6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097bf575bc060ce1618e6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:17,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c1575bc060ce1618ec HTTP/1.1" 502 150 2025-12-17 23:36:17,913 - lava_callback - ERROR - Error processing callback for node 694097c1575bc060ce1618ec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c1575bc060ce1618ec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c1575bc060ce1618ec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c1575bc060ce1618ec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:17,913 - lava_callback - INFO - Starting async job submit for node 694097d3575bc060ce161922 2025-12-17 23:36:17,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:17,917 - lava_callback - INFO - Starting async job submit for node 694097d5575bc060ce161925 2025-12-17 23:36:17,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:17,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d5575bc060ce161925 HTTP/1.1" 502 150 2025-12-17 23:36:17,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d5575bc060ce161925'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:17,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d5575bc060ce161925 2025-12-17 23:36:17,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d5575bc060ce161925 HTTP/1.1" 502 150 2025-12-17 23:36:17,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d5575bc060ce161925'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:17,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d3575bc060ce161922 HTTP/1.1" 502 150 2025-12-17 23:36:17,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d3575bc060ce161922'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:17,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d3575bc060ce161922 2025-12-17 23:36:17,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d3575bc060ce161922 HTTP/1.1" 502 150 2025-12-17 23:36:17,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d3575bc060ce161922'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:18,169 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c3575bc060ce1618ef 2025-12-17 23:36:18,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c3575bc060ce1618ef HTTP/1.1" 502 150 2025-12-17 23:36:18,170 - lava_callback - ERROR - Error processing callback for node 694097c3575bc060ce1618ef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c3575bc060ce1618ef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c3575bc060ce1618ef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c3575bc060ce1618ef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:18,172 - lava_callback - INFO - Starting async job submit for node 694097d6575bc060ce161928 2025-12-17 23:36:18,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:18,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d6575bc060ce161928 HTTP/1.1" 502 150 2025-12-17 23:36:18,179 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d6575bc060ce161928'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:18,179 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d6575bc060ce161928 2025-12-17 23:36:18,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d6575bc060ce161928 HTTP/1.1" 502 150 2025-12-17 23:36:18,180 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d6575bc060ce161928'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:18,309 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097cd575bc060ce161908 2025-12-17 23:36:18,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097cd575bc060ce161908 HTTP/1.1" 502 150 2025-12-17 23:36:18,310 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097cd575bc060ce161908'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:19,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097b1575bc060ce1618ca 2025-12-17 23:36:19,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097b1575bc060ce1618ca HTTP/1.1" 502 150 2025-12-17 23:36:19,430 - lava_callback - ERROR - Error processing callback for node 694097b1575bc060ce1618ca: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097b1575bc060ce1618ca (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097b1575bc060ce1618ca (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097b1575bc060ce1618ca (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:19,433 - lava_callback - INFO - Starting async job submit for node 694097d9575bc060ce161930 2025-12-17 23:36:19,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:19,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d2575bc060ce16191f 2025-12-17 23:36:19,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d2575bc060ce16191f HTTP/1.1" 502 150 2025-12-17 23:36:19,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d2575bc060ce16191f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:19,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d9575bc060ce161930 HTTP/1.1" 502 150 2025-12-17 23:36:19,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d9575bc060ce161930'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:19,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d9575bc060ce161930 2025-12-17 23:36:19,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d9575bc060ce161930 HTTP/1.1" 502 150 2025-12-17 23:36:19,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d9575bc060ce161930'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:19,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d5575bc060ce161925 2025-12-17 23:36:19,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d5575bc060ce161925 HTTP/1.1" 502 150 2025-12-17 23:36:19,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d5575bc060ce161925'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:19,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d3575bc060ce161922 2025-12-17 23:36:19,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d3575bc060ce161922 HTTP/1.1" 502 150 2025-12-17 23:36:19,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d3575bc060ce161922'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:20,180 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d6575bc060ce161928 2025-12-17 23:36:20,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d6575bc060ce161928 HTTP/1.1" 502 150 2025-12-17 23:36:20,182 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d6575bc060ce161928'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:20,405 - lava_callback - INFO - Received callback for node 6942c0d71b61f1c6005fea14 INFO: 172.21.0.1:47474 - "POST /node/6942c0d71b61f1c6005fea14 HTTP/1.0" 202 Accepted 2025-12-17 23:36:20,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ce575bc060ce161913 2025-12-17 23:36:20,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ce575bc060ce161913 HTTP/1.1" 502 150 2025-12-17 23:36:20,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ce575bc060ce161913'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:20,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c8575bc060ce1618fc 2025-12-17 23:36:20,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c8575bc060ce1618fc HTTP/1.1" 502 150 2025-12-17 23:36:20,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c8575bc060ce1618fc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:21,447 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d9575bc060ce161930 2025-12-17 23:36:21,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d9575bc060ce161930 HTTP/1.1" 502 150 2025-12-17 23:36:21,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d9575bc060ce161930'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:21,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d0575bc060ce16191c 2025-12-17 23:36:21,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d0575bc060ce16191c HTTP/1.1" 502 150 2025-12-17 23:36:21,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d0575bc060ce16191c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:22,128 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c0575bc060ce1618e9 2025-12-17 23:36:22,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c0575bc060ce1618e9 HTTP/1.1" 502 150 2025-12-17 23:36:22,129 - lava_callback - ERROR - Error processing callback for node 694097c0575bc060ce1618e9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c0575bc060ce1618e9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c0575bc060ce1618e9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c0575bc060ce1618e9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:22,131 - lava_callback - INFO - Starting async job submit for node 694097db575bc060ce161934 2025-12-17 23:36:22,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:22,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097db575bc060ce161934 HTTP/1.1" 502 150 2025-12-17 23:36:22,159 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097db575bc060ce161934'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:22,160 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097db575bc060ce161934 2025-12-17 23:36:22,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097db575bc060ce161934 HTTP/1.1" 502 150 2025-12-17 23:36:22,160 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097db575bc060ce161934'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:23,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d2575bc060ce16191f 2025-12-17 23:36:23,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d2575bc060ce16191f HTTP/1.1" 502 150 2025-12-17 23:36:23,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d2575bc060ce16191f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:23,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d5575bc060ce161925 2025-12-17 23:36:23,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d5575bc060ce161925 HTTP/1.1" 502 150 2025-12-17 23:36:23,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d5575bc060ce161925'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:23,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d3575bc060ce161922 2025-12-17 23:36:23,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d3575bc060ce161922 HTTP/1.1" 502 150 2025-12-17 23:36:23,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d3575bc060ce161922'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:24,161 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097db575bc060ce161934 2025-12-17 23:36:24,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097db575bc060ce161934 HTTP/1.1" 502 150 2025-12-17 23:36:24,162 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097db575bc060ce161934'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:24,182 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d6575bc060ce161928 2025-12-17 23:36:24,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d6575bc060ce161928 HTTP/1.1" 502 150 2025-12-17 23:36:24,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d6575bc060ce161928'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:25,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d9575bc060ce161930 2025-12-17 23:36:25,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d9575bc060ce161930 HTTP/1.1" 502 150 2025-12-17 23:36:25,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d9575bc060ce161930'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:26,310 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097cd575bc060ce161908 INFO: 172.21.0.1:47480 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:36:26,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c5575bc060ce1618f6 2025-12-17 23:36:26,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c5575bc060ce1618f6 HTTP/1.1" 502 150 2025-12-17 23:36:26,741 - lava_callback - ERROR - Error processing callback for node 694097c5575bc060ce1618f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c5575bc060ce1618f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c5575bc060ce1618f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c5575bc060ce1618f6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:26,742 - lava_callback - INFO - Starting async job submit for node 694097dd575bc060ce161937 2025-12-17 23:36:26,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:26,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097dd575bc060ce161937 HTTP/1.1" 502 150 2025-12-17 23:36:26,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097dd575bc060ce161937'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:26,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097dd575bc060ce161937 2025-12-17 23:36:26,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097dd575bc060ce161937 HTTP/1.1" 502 150 2025-12-17 23:36:26,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097dd575bc060ce161937'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:27,871 - lava_callback - INFO - Received callback for node 69409b69575bc060ce1621d7 INFO: 172.21.0.1:47492 - "POST /node/69409b69575bc060ce1621d7 HTTP/1.0" 202 Accepted 2025-12-17 23:36:28,162 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097db575bc060ce161934 2025-12-17 23:36:28,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097db575bc060ce161934 HTTP/1.1" 502 150 2025-12-17 23:36:28,163 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097db575bc060ce161934'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:28,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ce575bc060ce161913 2025-12-17 23:36:28,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ce575bc060ce161913 HTTP/1.1" 502 150 2025-12-17 23:36:28,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ce575bc060ce161913'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:28,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097bc575bc060ce1618e0 2025-12-17 23:36:28,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097bc575bc060ce1618e0 HTTP/1.1" 502 150 2025-12-17 23:36:28,531 - lava_callback - ERROR - Error processing callback for node 694097bc575bc060ce1618e0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097bc575bc060ce1618e0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097bc575bc060ce1618e0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097bc575bc060ce1618e0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:28,533 - lava_callback - INFO - Starting async job submit for node 694097df575bc060ce16193a 2025-12-17 23:36:28,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:28,536 - lava_callback - INFO - Received callback for node 69409b6b575bc060ce1621e4 INFO: 172.21.0.1:47502 - "POST /node/69409b6b575bc060ce1621e4 HTTP/1.0" 202 Accepted 2025-12-17 23:36:28,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c8575bc060ce1618fc 2025-12-17 23:36:28,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c8575bc060ce1618fc HTTP/1.1" 502 150 2025-12-17 23:36:28,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097c8575bc060ce1618fc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:28,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097dd575bc060ce161937 2025-12-17 23:36:28,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097dd575bc060ce161937 HTTP/1.1" 502 150 2025-12-17 23:36:28,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097dd575bc060ce161937'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:29,113 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c7575bc060ce1618f9 2025-12-17 23:36:29,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c7575bc060ce1618f9 HTTP/1.1" 502 150 2025-12-17 23:36:29,115 - lava_callback - ERROR - Error processing callback for node 694097c7575bc060ce1618f9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c7575bc060ce1618f9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c7575bc060ce1618f9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c7575bc060ce1618f9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:29,117 - lava_callback - INFO - Starting async job submit for node 694097e1575bc060ce16193d 2025-12-17 23:36:29,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:29,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e1575bc060ce16193d HTTP/1.1" 502 150 2025-12-17 23:36:29,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e1575bc060ce16193d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:29,123 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e1575bc060ce16193d 2025-12-17 23:36:29,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e1575bc060ce16193d HTTP/1.1" 502 150 2025-12-17 23:36:29,124 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e1575bc060ce16193d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:29,188 - lava_callback - INFO - Received callback for node 69409b6d575bc060ce1621e7 INFO: 172.21.0.1:47506 - "POST /node/69409b6d575bc060ce1621e7 HTTP/1.0" 202 Accepted 2025-12-17 23:36:29,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d0575bc060ce16191c 2025-12-17 23:36:29,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d0575bc060ce16191c HTTP/1.1" 502 150 2025-12-17 23:36:29,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d0575bc060ce16191c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:29,875 - lava_callback - INFO - Received callback for node 69409b6f575bc060ce1621ea 2025-12-17 23:36:29,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c9575bc060ce1618ff INFO: 172.21.0.1:32880 - "POST /node/69409b6f575bc060ce1621ea HTTP/1.0" 202 Accepted 2025-12-17 23:36:29,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c9575bc060ce1618ff HTTP/1.1" 502 150 2025-12-17 23:36:29,882 - lava_callback - ERROR - Error processing callback for node 694097c9575bc060ce1618ff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c9575bc060ce1618ff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c9575bc060ce1618ff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c9575bc060ce1618ff (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:29,884 - lava_callback - INFO - Starting async job submit for node 694097e2575bc060ce161944 2025-12-17 23:36:29,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:29,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e2575bc060ce161944 HTTP/1.1" 502 150 2025-12-17 23:36:29,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e2575bc060ce161944'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:29,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e2575bc060ce161944 2025-12-17 23:36:29,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e2575bc060ce161944 HTTP/1.1" 502 150 2025-12-17 23:36:29,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e2575bc060ce161944'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:30,545 - lava_callback - INFO - Received callback for node 69409b71575bc060ce1621f1 INFO: 172.21.0.1:32884 - "POST /node/69409b71575bc060ce1621f1 HTTP/1.0" 202 Accepted 2025-12-17 23:36:31,124 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e1575bc060ce16193d 2025-12-17 23:36:31,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e1575bc060ce16193d HTTP/1.1" 502 150 2025-12-17 23:36:31,125 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e1575bc060ce16193d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:31,213 - lava_callback - INFO - Received callback for node 69409b76575bc060ce1621f7 INFO: 172.21.0.1:32888 - "POST /node/69409b76575bc060ce1621f7 HTTP/1.0" 202 Accepted 2025-12-17 23:36:31,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d2575bc060ce16191f 2025-12-17 23:36:31,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d2575bc060ce16191f HTTP/1.1" 502 150 2025-12-17 23:36:31,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d2575bc060ce16191f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:31,892 - lava_callback - INFO - Received callback for node 69409b78575bc060ce1621fa INFO: 172.21.0.1:32896 - "POST /node/69409b78575bc060ce1621fa HTTP/1.0" 202 Accepted 2025-12-17 23:36:31,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e2575bc060ce161944 2025-12-17 23:36:31,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e2575bc060ce161944 HTTP/1.1" 502 150 2025-12-17 23:36:31,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e2575bc060ce161944'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:31,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d5575bc060ce161925 2025-12-17 23:36:31,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d5575bc060ce161925 HTTP/1.1" 502 150 2025-12-17 23:36:31,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d5575bc060ce161925'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:31,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d3575bc060ce161922 2025-12-17 23:36:31,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d3575bc060ce161922 HTTP/1.1" 502 150 2025-12-17 23:36:31,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d3575bc060ce161922'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:32,182 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ca575bc060ce161902 2025-12-17 23:36:32,183 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d6575bc060ce161928 2025-12-17 23:36:32,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ca575bc060ce161902 HTTP/1.1" 502 150 2025-12-17 23:36:32,184 - lava_callback - ERROR - Error processing callback for node 694097ca575bc060ce161902: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ca575bc060ce161902 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ca575bc060ce161902 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ca575bc060ce161902 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:32,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d6575bc060ce161928 HTTP/1.1" 502 150 2025-12-17 23:36:32,186 - lava_callback - INFO - Starting async job submit for node 694097e4575bc060ce16194c 2025-12-17 23:36:32,186 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d6575bc060ce161928'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:32,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:32,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e4575bc060ce16194c HTTP/1.1" 502 150 2025-12-17 23:36:32,223 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e4575bc060ce16194c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:32,223 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e4575bc060ce16194c 2025-12-17 23:36:32,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e4575bc060ce16194c HTTP/1.1" 502 150 2025-12-17 23:36:32,224 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e4575bc060ce16194c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:32,572 - lava_callback - INFO - Received callback for node 69409b7a575bc060ce1621fd INFO: 172.21.0.1:32910 - "POST /node/69409b7a575bc060ce1621fd HTTP/1.0" 202 Accepted 2025-12-17 23:36:32,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097dd575bc060ce161937 2025-12-17 23:36:32,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097dd575bc060ce161937 HTTP/1.1" 502 150 2025-12-17 23:36:32,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097dd575bc060ce161937'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:33,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d9575bc060ce161930 2025-12-17 23:36:33,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d9575bc060ce161930 HTTP/1.1" 502 150 2025-12-17 23:36:33,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097d9575bc060ce161930'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:34,224 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e4575bc060ce16194c 2025-12-17 23:36:34,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e4575bc060ce16194c HTTP/1.1" 502 150 2025-12-17 23:36:34,225 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e4575bc060ce16194c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:35,125 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e1575bc060ce16193d 2025-12-17 23:36:35,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e1575bc060ce16193d HTTP/1.1" 502 150 2025-12-17 23:36:35,128 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e1575bc060ce16193d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:35,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e2575bc060ce161944 2025-12-17 23:36:35,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e2575bc060ce161944 HTTP/1.1" 502 150 2025-12-17 23:36:35,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e2575bc060ce161944'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:36,164 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097db575bc060ce161934 2025-12-17 23:36:36,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097db575bc060ce161934 HTTP/1.1" 502 150 2025-12-17 23:36:36,165 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097db575bc060ce161934'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:32918 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:36:36,750 - lava_callback - INFO - Received callback for node 69409e6b575bc060ce16297f INFO: 172.21.0.1:32924 - "POST /node/69409e6b575bc060ce16297f HTTP/1.0" 202 Accepted 2025-12-17 23:36:37,209 - lava_callback - INFO - Received callback for node 69409b73575bc060ce1621f4 INFO: 172.21.0.1:32938 - "POST /node/69409b73575bc060ce1621f4 HTTP/1.0" 202 Accepted 2025-12-17 23:36:37,880 - lava_callback - INFO - Received callback for node 69409b82575bc060ce162217 INFO: 172.21.0.1:32940 - "POST /node/69409b82575bc060ce162217 HTTP/1.0" 202 Accepted 2025-12-17 23:36:38,226 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e4575bc060ce16194c 2025-12-17 23:36:38,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e4575bc060ce16194c HTTP/1.1" 502 150 2025-12-17 23:36:38,227 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e4575bc060ce16194c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:38,560 - lava_callback - INFO - Received callback for node 69409b84575bc060ce16221a INFO: 172.21.0.1:32956 - "POST /node/69409b84575bc060ce16221a HTTP/1.0" 202 Accepted 2025-12-17 23:36:40,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097dd575bc060ce161937 2025-12-17 23:36:40,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097dd575bc060ce161937 HTTP/1.1" 502 150 2025-12-17 23:36:40,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097dd575bc060ce161937'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:41,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097cd575bc060ce161908 HTTP/1.1" 502 150 2025-12-17 23:36:41,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097cd575bc060ce161908'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:42,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097cc575bc060ce161905 HTTP/1.1" 502 150 2025-12-17 23:36:42,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097cc575bc060ce161905'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:43,128 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e1575bc060ce16193d 2025-12-17 23:36:43,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e1575bc060ce16193d HTTP/1.1" 502 150 2025-12-17 23:36:43,129 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e1575bc060ce16193d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:43,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e2575bc060ce161944 2025-12-17 23:36:43,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e2575bc060ce161944 HTTP/1.1" 502 150 2025-12-17 23:36:43,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e2575bc060ce161944'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:44,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ce575bc060ce161913 2025-12-17 23:36:44,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ce575bc060ce161913 HTTP/1.1" 502 150 2025-12-17 23:36:44,477 - lava_callback - ERROR - Error processing callback for node 694097ce575bc060ce161913: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ce575bc060ce161913 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ce575bc060ce161913 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ce575bc060ce161913 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:44,479 - lava_callback - INFO - Starting async job submit for node 694097e6575bc060ce161952 2025-12-17 23:36:44,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:44,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e6575bc060ce161952 HTTP/1.1" 502 150 2025-12-17 23:36:44,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e6575bc060ce161952'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:44,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e6575bc060ce161952 2025-12-17 23:36:44,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e6575bc060ce161952 HTTP/1.1" 502 150 2025-12-17 23:36:44,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e6575bc060ce161952'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:44,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097c8575bc060ce1618fc 2025-12-17 23:36:44,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097c8575bc060ce1618fc HTTP/1.1" 502 150 2025-12-17 23:36:44,564 - lava_callback - ERROR - Error processing callback for node 694097c8575bc060ce1618fc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c8575bc060ce1618fc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c8575bc060ce1618fc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097c8575bc060ce1618fc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:44,565 - lava_callback - INFO - Starting async job submit for node 6941dc3becc59770562efc10 2025-12-17 23:36:44,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:44,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941dc3becc59770562efc10 HTTP/1.1" 502 150 2025-12-17 23:36:44,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941dc3becc59770562efc10'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:44,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941dc3becc59770562efc10 2025-12-17 23:36:44,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941dc3becc59770562efc10 HTTP/1.1" 502 150 2025-12-17 23:36:44,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941dc3becc59770562efc10'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:45,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d0575bc060ce16191c 2025-12-17 23:36:45,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d0575bc060ce16191c HTTP/1.1" 502 150 2025-12-17 23:36:45,559 - lava_callback - ERROR - Error processing callback for node 694097d0575bc060ce16191c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d0575bc060ce16191c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d0575bc060ce16191c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d0575bc060ce16191c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:45,561 - lava_callback - INFO - Starting async job submit for node 694097f1575bc060ce161964 2025-12-17 23:36:45,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:45,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f1575bc060ce161964 HTTP/1.1" 502 150 2025-12-17 23:36:45,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f1575bc060ce161964'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:45,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f1575bc060ce161964 2025-12-17 23:36:45,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f1575bc060ce161964 HTTP/1.1" 502 150 2025-12-17 23:36:45,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f1575bc060ce161964'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:46,227 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e4575bc060ce16194c 2025-12-17 23:36:46,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e4575bc060ce16194c HTTP/1.1" 502 150 2025-12-17 23:36:46,228 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e4575bc060ce16194c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:46,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e6575bc060ce161952 2025-12-17 23:36:46,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e6575bc060ce161952 HTTP/1.1" 502 150 2025-12-17 23:36:46,495 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e6575bc060ce161952'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54728 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:36:46,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941dc3becc59770562efc10 2025-12-17 23:36:46,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941dc3becc59770562efc10 HTTP/1.1" 502 150 2025-12-17 23:36:46,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941dc3becc59770562efc10'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:46,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097cc575bc060ce161905 2025-12-17 23:36:46,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097cc575bc060ce161905 HTTP/1.1" 502 150 2025-12-17 23:36:46,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097cc575bc060ce161905'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:47,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d2575bc060ce16191f 2025-12-17 23:36:47,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d2575bc060ce16191f HTTP/1.1" 502 150 2025-12-17 23:36:47,447 - lava_callback - ERROR - Error processing callback for node 694097d2575bc060ce16191f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d2575bc060ce16191f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d2575bc060ce16191f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d2575bc060ce16191f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:47,449 - lava_callback - INFO - Starting async job submit for node 694097f4575bc060ce16196b 2025-12-17 23:36:47,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:47,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f4575bc060ce16196b HTTP/1.1" 502 150 2025-12-17 23:36:47,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f4575bc060ce16196b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:47,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f4575bc060ce16196b 2025-12-17 23:36:47,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f4575bc060ce16196b HTTP/1.1" 502 150 2025-12-17 23:36:47,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f4575bc060ce16196b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:47,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f1575bc060ce161964 2025-12-17 23:36:47,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f1575bc060ce161964 HTTP/1.1" 502 150 2025-12-17 23:36:47,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f1575bc060ce161964'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:47,913 - lava_callback - INFO - Received callback for node 69409b7c575bc060ce162204 INFO: 172.21.0.1:54734 - "POST /node/69409b7c575bc060ce162204 HTTP/1.0" 202 Accepted 2025-12-17 23:36:47,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d5575bc060ce161925 2025-12-17 23:36:47,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d5575bc060ce161925 HTTP/1.1" 502 150 2025-12-17 23:36:47,930 - lava_callback - ERROR - Error processing callback for node 694097d5575bc060ce161925: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d5575bc060ce161925 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d5575bc060ce161925 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d5575bc060ce161925 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:47,931 - lava_callback - INFO - Starting async job submit for node 694097ea575bc060ce161958 2025-12-17 23:36:47,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:47,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d3575bc060ce161922 2025-12-17 23:36:47,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d3575bc060ce161922 HTTP/1.1" 502 150 2025-12-17 23:36:47,938 - lava_callback - ERROR - Error processing callback for node 694097d3575bc060ce161922: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d3575bc060ce161922 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d3575bc060ce161922 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d3575bc060ce161922 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:47,939 - lava_callback - INFO - Starting async job submit for node 694097f6575bc060ce161970 2025-12-17 23:36:47,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:47,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f6575bc060ce161970 HTTP/1.1" 502 150 2025-12-17 23:36:47,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f6575bc060ce161970'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:47,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f6575bc060ce161970 2025-12-17 23:36:47,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f6575bc060ce161970 HTTP/1.1" 502 150 2025-12-17 23:36:47,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f6575bc060ce161970'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:47,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ea575bc060ce161958 HTTP/1.1" 502 150 2025-12-17 23:36:47,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ea575bc060ce161958'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:47,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ea575bc060ce161958 2025-12-17 23:36:47,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ea575bc060ce161958 HTTP/1.1" 502 150 2025-12-17 23:36:47,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ea575bc060ce161958'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:48,187 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d6575bc060ce161928 2025-12-17 23:36:48,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d6575bc060ce161928 HTTP/1.1" 502 150 2025-12-17 23:36:48,189 - lava_callback - ERROR - Error processing callback for node 694097d6575bc060ce161928: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d6575bc060ce161928 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d6575bc060ce161928 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d6575bc060ce161928 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:48,191 - lava_callback - INFO - Starting async job submit for node 694097eb575bc060ce16195b 2025-12-17 23:36:48,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:48,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097eb575bc060ce16195b HTTP/1.1" 502 150 2025-12-17 23:36:48,204 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097eb575bc060ce16195b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:48,204 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097eb575bc060ce16195b 2025-12-17 23:36:48,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097eb575bc060ce16195b HTTP/1.1" 502 150 2025-12-17 23:36:48,205 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097eb575bc060ce16195b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:48,574 - lava_callback - INFO - Received callback for node 69409b7e575bc060ce16220b INFO: 172.21.0.1:54750 - "POST /node/69409b7e575bc060ce16220b HTTP/1.0" 202 Accepted 2025-12-17 23:36:49,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097d9575bc060ce161930 2025-12-17 23:36:49,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097d9575bc060ce161930 HTTP/1.1" 502 150 2025-12-17 23:36:49,453 - lava_callback - ERROR - Error processing callback for node 694097d9575bc060ce161930: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d9575bc060ce161930 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d9575bc060ce161930 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097d9575bc060ce161930 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:49,455 - lava_callback - INFO - Starting async job submit for node 694097ed575bc060ce16195e 2025-12-17 23:36:49,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:49,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ed575bc060ce16195e HTTP/1.1" 502 150 2025-12-17 23:36:49,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ed575bc060ce16195e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:49,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ed575bc060ce16195e 2025-12-17 23:36:49,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ed575bc060ce16195e HTTP/1.1" 502 150 2025-12-17 23:36:49,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ed575bc060ce16195e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:49,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f4575bc060ce16196b 2025-12-17 23:36:49,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f4575bc060ce16196b HTTP/1.1" 502 150 2025-12-17 23:36:49,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f4575bc060ce16196b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:49,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f6575bc060ce161970 2025-12-17 23:36:49,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f6575bc060ce161970 HTTP/1.1" 502 150 2025-12-17 23:36:49,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f6575bc060ce161970'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:49,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ea575bc060ce161958 2025-12-17 23:36:49,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ea575bc060ce161958 HTTP/1.1" 502 150 2025-12-17 23:36:49,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ea575bc060ce161958'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:50,205 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097eb575bc060ce16195b 2025-12-17 23:36:50,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097eb575bc060ce16195b HTTP/1.1" 502 150 2025-12-17 23:36:50,206 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097eb575bc060ce16195b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:50,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e6575bc060ce161952 2025-12-17 23:36:50,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e6575bc060ce161952 HTTP/1.1" 502 150 2025-12-17 23:36:50,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e6575bc060ce161952'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:50,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941dc3becc59770562efc10 2025-12-17 23:36:50,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941dc3becc59770562efc10 HTTP/1.1" 502 150 2025-12-17 23:36:50,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941dc3becc59770562efc10'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:51,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ed575bc060ce16195e 2025-12-17 23:36:51,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ed575bc060ce16195e HTTP/1.1" 502 150 2025-12-17 23:36:51,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ed575bc060ce16195e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:51,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f1575bc060ce161964 2025-12-17 23:36:51,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f1575bc060ce161964 HTTP/1.1" 502 150 2025-12-17 23:36:51,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f1575bc060ce161964'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:52,165 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097db575bc060ce161934 2025-12-17 23:36:52,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097db575bc060ce161934 HTTP/1.1" 502 150 2025-12-17 23:36:52,166 - lava_callback - ERROR - Error processing callback for node 694097db575bc060ce161934: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097db575bc060ce161934 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097db575bc060ce161934 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097db575bc060ce161934 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:52,169 - lava_callback - INFO - Starting async job submit for node 694097ef575bc060ce161961 2025-12-17 23:36:52,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:52,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ef575bc060ce161961 HTTP/1.1" 502 150 2025-12-17 23:36:52,177 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ef575bc060ce161961'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:52,177 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ef575bc060ce161961 2025-12-17 23:36:52,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ef575bc060ce161961 HTTP/1.1" 502 150 2025-12-17 23:36:52,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ef575bc060ce161961'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:53,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f4575bc060ce16196b 2025-12-17 23:36:53,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f4575bc060ce16196b HTTP/1.1" 502 150 2025-12-17 23:36:53,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f4575bc060ce16196b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:53,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f6575bc060ce161970 2025-12-17 23:36:53,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f6575bc060ce161970 HTTP/1.1" 502 150 2025-12-17 23:36:53,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f6575bc060ce161970'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:53,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ea575bc060ce161958 2025-12-17 23:36:53,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ea575bc060ce161958 HTTP/1.1" 502 150 2025-12-17 23:36:53,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ea575bc060ce161958'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:54,178 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ef575bc060ce161961 2025-12-17 23:36:54,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ef575bc060ce161961 HTTP/1.1" 502 150 2025-12-17 23:36:54,179 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ef575bc060ce161961'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:54,206 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097eb575bc060ce16195b 2025-12-17 23:36:54,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097eb575bc060ce16195b HTTP/1.1" 502 150 2025-12-17 23:36:54,207 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097eb575bc060ce16195b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:54,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097cc575bc060ce161905 2025-12-17 23:36:54,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097cc575bc060ce161905 HTTP/1.1" 502 150 2025-12-17 23:36:54,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097cc575bc060ce161905'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:55,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ed575bc060ce16195e 2025-12-17 23:36:55,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ed575bc060ce16195e HTTP/1.1" 502 150 2025-12-17 23:36:55,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ed575bc060ce16195e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56062 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:36:56,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097dd575bc060ce161937 2025-12-17 23:36:56,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097dd575bc060ce161937 HTTP/1.1" 502 150 2025-12-17 23:36:56,757 - lava_callback - ERROR - Error processing callback for node 694097dd575bc060ce161937: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097dd575bc060ce161937 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097dd575bc060ce161937 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097dd575bc060ce161937 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:56,759 - lava_callback - INFO - Starting async job submit for node 694097fd575bc060ce161988 2025-12-17 23:36:56,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:56,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fd575bc060ce161988 HTTP/1.1" 502 150 2025-12-17 23:36:56,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097fd575bc060ce161988'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:56,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097fd575bc060ce161988 2025-12-17 23:36:56,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fd575bc060ce161988 HTTP/1.1" 502 150 2025-12-17 23:36:56,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097fd575bc060ce161988'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:57,269 - lava_callback - INFO - Received callback for node 69409b85575bc060ce16221d INFO: 172.21.0.1:56074 - "POST /node/69409b85575bc060ce16221d HTTP/1.0" 202 Accepted 2025-12-17 23:36:57,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097cd575bc060ce161908 2025-12-17 23:36:57,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097cd575bc060ce161908 HTTP/1.1" 502 150 2025-12-17 23:36:57,650 - lava_callback - ERROR - Error processing callback for node 694097cd575bc060ce161908: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097cd575bc060ce161908 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097cd575bc060ce161908 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097cd575bc060ce161908 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:57,652 - lava_callback - INFO - Starting async job submit for node 694097fa575bc060ce161982 2025-12-17 23:36:57,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:57,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fa575bc060ce161982 HTTP/1.1" 502 150 2025-12-17 23:36:57,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097fa575bc060ce161982'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:57,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097fa575bc060ce161982 2025-12-17 23:36:57,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fa575bc060ce161982 HTTP/1.1" 502 150 2025-12-17 23:36:57,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097fa575bc060ce161982'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:57,936 - lava_callback - INFO - Received callback for node 69409b87575bc060ce162220 INFO: 172.21.0.1:56086 - "POST /node/69409b87575bc060ce162220 HTTP/1.0" 202 Accepted 2025-12-17 23:36:58,179 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ef575bc060ce161961 2025-12-17 23:36:58,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ef575bc060ce161961 HTTP/1.1" 502 150 2025-12-17 23:36:58,180 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ef575bc060ce161961'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:58,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e6575bc060ce161952 2025-12-17 23:36:58,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e6575bc060ce161952 HTTP/1.1" 502 150 2025-12-17 23:36:58,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097e6575bc060ce161952'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:58,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941dc3becc59770562efc10 2025-12-17 23:36:58,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941dc3becc59770562efc10 HTTP/1.1" 502 150 2025-12-17 23:36:58,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941dc3becc59770562efc10'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:58,601 - lava_callback - INFO - Received callback for node 69409b89575bc060ce162223 INFO: 172.21.0.1:56088 - "POST /node/69409b89575bc060ce162223 HTTP/1.0" 202 Accepted 2025-12-17 23:36:58,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097fd575bc060ce161988 2025-12-17 23:36:58,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fd575bc060ce161988 HTTP/1.1" 502 150 2025-12-17 23:36:58,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097fd575bc060ce161988'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:59,129 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e1575bc060ce16193d 2025-12-17 23:36:59,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e1575bc060ce16193d HTTP/1.1" 502 150 2025-12-17 23:36:59,130 - lava_callback - ERROR - Error processing callback for node 694097e1575bc060ce16193d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097e1575bc060ce16193d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097e1575bc060ce16193d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097e1575bc060ce16193d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:59,132 - lava_callback - INFO - Starting async job submit for node 69409805575bc060ce16199a 2025-12-17 23:36:59,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:59,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409805575bc060ce16199a HTTP/1.1" 502 150 2025-12-17 23:36:59,147 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409805575bc060ce16199a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:59,148 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409805575bc060ce16199a 2025-12-17 23:36:59,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409805575bc060ce16199a HTTP/1.1" 502 150 2025-12-17 23:36:59,148 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409805575bc060ce16199a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:59,260 - lava_callback - INFO - Received callback for node 69409b8b575bc060ce162226 INFO: 172.21.0.1:56094 - "POST /node/69409b8b575bc060ce162226 HTTP/1.0" 202 Accepted 2025-12-17 23:36:59,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097df575bc060ce16193a HTTP/1.1" 502 150 2025-12-17 23:36:59,312 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097df575bc060ce16193a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:59,312 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097df575bc060ce16193a 2025-12-17 23:36:59,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f1575bc060ce161964 2025-12-17 23:36:59,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f1575bc060ce161964 HTTP/1.1" 502 150 2025-12-17 23:36:59,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f1575bc060ce161964'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:59,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097fa575bc060ce161982 2025-12-17 23:36:59,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fa575bc060ce161982 HTTP/1.1" 502 150 2025-12-17 23:36:59,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097fa575bc060ce161982'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:59,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e2575bc060ce161944 2025-12-17 23:36:59,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e2575bc060ce161944 HTTP/1.1" 502 150 2025-12-17 23:36:59,917 - lava_callback - ERROR - Error processing callback for node 694097e2575bc060ce161944: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097e2575bc060ce161944 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097e2575bc060ce161944 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097e2575bc060ce161944 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:36:59,918 - lava_callback - INFO - Starting async job submit for node 69409808575bc060ce1619a5 2025-12-17 23:36:59,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:36:59,929 - lava_callback - INFO - Received callback for node 69409b8c575bc060ce162229 INFO: 172.21.0.1:54142 - "POST /node/69409b8c575bc060ce162229 HTTP/1.0" 202 Accepted 2025-12-17 23:36:59,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409808575bc060ce1619a5 HTTP/1.1" 502 150 2025-12-17 23:36:59,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409808575bc060ce1619a5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:36:59,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409808575bc060ce1619a5 2025-12-17 23:36:59,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409808575bc060ce1619a5 HTTP/1.1" 502 150 2025-12-17 23:36:59,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409808575bc060ce1619a5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:00,585 - lava_callback - INFO - Received callback for node 69409b8e575bc060ce16222c INFO: 172.21.0.1:54152 - "POST /node/69409b8e575bc060ce16222c HTTP/1.0" 202 Accepted 2025-12-17 23:37:01,149 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409805575bc060ce16199a 2025-12-17 23:37:01,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409805575bc060ce16199a HTTP/1.1" 502 150 2025-12-17 23:37:01,152 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409805575bc060ce16199a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:01,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f4575bc060ce16196b 2025-12-17 23:37:01,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f4575bc060ce16196b HTTP/1.1" 502 150 2025-12-17 23:37:01,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f4575bc060ce16196b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:01,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409808575bc060ce1619a5 2025-12-17 23:37:01,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409808575bc060ce1619a5 HTTP/1.1" 502 150 2025-12-17 23:37:01,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409808575bc060ce1619a5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:01,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f6575bc060ce161970 2025-12-17 23:37:01,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f6575bc060ce161970 HTTP/1.1" 502 150 2025-12-17 23:37:01,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097f6575bc060ce161970'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:01,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ea575bc060ce161958 2025-12-17 23:37:01,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ea575bc060ce161958 HTTP/1.1" 502 150 2025-12-17 23:37:01,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ea575bc060ce161958'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:02,208 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097eb575bc060ce16195b 2025-12-17 23:37:02,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097eb575bc060ce16195b HTTP/1.1" 502 150 2025-12-17 23:37:02,209 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097eb575bc060ce16195b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:02,228 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e4575bc060ce16194c 2025-12-17 23:37:02,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e4575bc060ce16194c HTTP/1.1" 502 150 2025-12-17 23:37:02,230 - lava_callback - ERROR - Error processing callback for node 694097e4575bc060ce16194c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097e4575bc060ce16194c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097e4575bc060ce16194c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097e4575bc060ce16194c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:02,232 - lava_callback - INFO - Starting async job submit for node 69409809575bc060ce1619a8 2025-12-17 23:37:02,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:02,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409809575bc060ce1619a8 HTTP/1.1" 502 150 2025-12-17 23:37:02,238 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409809575bc060ce1619a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:02,238 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409809575bc060ce1619a8 2025-12-17 23:37:02,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409809575bc060ce1619a8 HTTP/1.1" 502 150 2025-12-17 23:37:02,239 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409809575bc060ce1619a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:02,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097fd575bc060ce161988 2025-12-17 23:37:02,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fd575bc060ce161988 HTTP/1.1" 502 150 2025-12-17 23:37:02,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097fd575bc060ce161988'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54156 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:37:03,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ed575bc060ce16195e 2025-12-17 23:37:03,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ed575bc060ce16195e HTTP/1.1" 502 150 2025-12-17 23:37:03,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ed575bc060ce16195e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:03,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097fa575bc060ce161982 2025-12-17 23:37:03,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fa575bc060ce161982 HTTP/1.1" 502 150 2025-12-17 23:37:03,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097fa575bc060ce161982'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:04,239 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409809575bc060ce1619a8 2025-12-17 23:37:05,153 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409805575bc060ce16199a 2025-12-17 23:37:05,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409805575bc060ce16199a HTTP/1.1" 502 150 2025-12-17 23:37:05,154 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409805575bc060ce16199a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:05,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409808575bc060ce1619a5 2025-12-17 23:37:05,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409808575bc060ce1619a5 HTTP/1.1" 502 150 2025-12-17 23:37:05,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409808575bc060ce1619a5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:06,181 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ef575bc060ce161961 2025-12-17 23:37:06,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ef575bc060ce161961 HTTP/1.1" 502 150 2025-12-17 23:37:06,182 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097ef575bc060ce161961'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54160 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:37:07,927 - lava_callback - INFO - Received callback for node 69409b90575bc060ce162231 INFO: 172.21.0.1:54174 - "POST /node/69409b90575bc060ce162231 HTTP/1.0" 202 Accepted 2025-12-17 23:37:08,592 - lava_callback - INFO - Received callback for node 69409b92575bc060ce162234 INFO: 172.21.0.1:54182 - "POST /node/69409b92575bc060ce162234 HTTP/1.0" 202 Accepted 2025-12-17 23:37:09,266 - lava_callback - INFO - Received callback for node 69409b94575bc060ce16223b INFO: 172.21.0.1:54196 - "POST /node/69409b94575bc060ce16223b HTTP/1.0" 202 Accepted 2025-12-17 23:37:09,945 - lava_callback - INFO - Received callback for node 69409b96575bc060ce16223e INFO: 172.21.0.1:47058 - "POST /node/69409b96575bc060ce16223e HTTP/1.0" 202 Accepted 2025-12-17 23:37:10,610 - lava_callback - INFO - Received callback for node 69409b98575bc060ce162241 INFO: 172.21.0.1:47062 - "POST /node/69409b98575bc060ce162241 HTTP/1.0" 202 Accepted 2025-12-17 23:37:10,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097fd575bc060ce161988 2025-12-17 23:37:10,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fd575bc060ce161988 HTTP/1.1" 502 150 2025-12-17 23:37:10,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097fd575bc060ce161988'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:10,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097cc575bc060ce161905 2025-12-17 23:37:10,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097cc575bc060ce161905 HTTP/1.1" 502 150 2025-12-17 23:37:10,933 - lava_callback - ERROR - Error processing callback for node 694097cc575bc060ce161905: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097cc575bc060ce161905 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097cc575bc060ce161905 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097cc575bc060ce161905 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:10,935 - lava_callback - INFO - Starting async job submit for node 6940980a575bc060ce1619ad 2025-12-17 23:37:10,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:10,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980a575bc060ce1619ad HTTP/1.1" 502 150 2025-12-17 23:37:10,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940980a575bc060ce1619ad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:10,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940980a575bc060ce1619ad 2025-12-17 23:37:10,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980a575bc060ce1619ad HTTP/1.1" 502 150 2025-12-17 23:37:10,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940980a575bc060ce1619ad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:11,256 - lava_callback - INFO - Received callback for node 69409b9a575bc060ce162244 INFO: 172.21.0.1:47072 - "POST /node/69409b9a575bc060ce162244 HTTP/1.0" 202 Accepted 2025-12-17 23:37:11,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097fa575bc060ce161982 2025-12-17 23:37:11,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fa575bc060ce161982 HTTP/1.1" 502 150 2025-12-17 23:37:11,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097fa575bc060ce161982'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:11,943 - lava_callback - INFO - Received callback for node 69409b9c575bc060ce162247 INFO: 172.21.0.1:47074 - "POST /node/69409b9c575bc060ce162247 HTTP/1.0" 202 Accepted 2025-12-17 23:37:12,606 - lava_callback - INFO - Received callback for node 69409ba0575bc060ce16224d INFO: 172.21.0.1:47076 - "POST /node/69409ba0575bc060ce16224d HTTP/1.0" 202 Accepted 2025-12-17 23:37:12,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940980a575bc060ce1619ad 2025-12-17 23:37:12,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980a575bc060ce1619ad HTTP/1.1" 502 150 2025-12-17 23:37:12,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940980a575bc060ce1619ad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:13,154 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409805575bc060ce16199a 2025-12-17 23:37:13,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409805575bc060ce16199a HTTP/1.1" 502 150 2025-12-17 23:37:13,155 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409805575bc060ce16199a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:13,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409808575bc060ce1619a5 2025-12-17 23:37:13,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409808575bc060ce1619a5 HTTP/1.1" 502 150 2025-12-17 23:37:13,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409808575bc060ce1619a5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:14,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097e6575bc060ce161952 2025-12-17 23:37:14,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097e6575bc060ce161952 HTTP/1.1" 502 150 2025-12-17 23:37:14,499 - lava_callback - ERROR - Error processing callback for node 694097e6575bc060ce161952: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097e6575bc060ce161952 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097e6575bc060ce161952 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097e6575bc060ce161952 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:14,501 - lava_callback - INFO - Starting async job submit for node 6940980c575bc060ce1619b8 2025-12-17 23:37:14,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:14,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980c575bc060ce1619b8 HTTP/1.1" 502 150 2025-12-17 23:37:14,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940980c575bc060ce1619b8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:14,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940980c575bc060ce1619b8 2025-12-17 23:37:14,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980c575bc060ce1619b8 HTTP/1.1" 502 150 2025-12-17 23:37:14,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940980c575bc060ce1619b8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:14,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941dc3becc59770562efc10 2025-12-17 23:37:14,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941dc3becc59770562efc10 HTTP/1.1" 502 150 2025-12-17 23:37:14,580 - lava_callback - ERROR - Error processing callback for node 6941dc3becc59770562efc10: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941dc3becc59770562efc10 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941dc3becc59770562efc10 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941dc3becc59770562efc10 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:14,582 - lava_callback - INFO - Starting async job submit for node 6940980f575bc060ce1619bf 2025-12-17 23:37:14,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:14,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980f575bc060ce1619bf HTTP/1.1" 502 150 2025-12-17 23:37:14,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940980f575bc060ce1619bf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:14,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940980f575bc060ce1619bf 2025-12-17 23:37:14,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980f575bc060ce1619bf HTTP/1.1" 502 150 2025-12-17 23:37:14,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940980f575bc060ce1619bf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47092 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:37:15,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f1575bc060ce161964 2025-12-17 23:37:15,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f1575bc060ce161964 HTTP/1.1" 502 150 2025-12-17 23:37:15,617 - lava_callback - ERROR - Error processing callback for node 694097f1575bc060ce161964: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097f1575bc060ce161964 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097f1575bc060ce161964 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097f1575bc060ce161964 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:15,619 - lava_callback - INFO - Starting async job submit for node 69409811575bc060ce1619c2 2025-12-17 23:37:15,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:15,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409811575bc060ce1619c2 HTTP/1.1" 502 150 2025-12-17 23:37:15,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409811575bc060ce1619c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:15,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409811575bc060ce1619c2 2025-12-17 23:37:15,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409811575bc060ce1619c2 HTTP/1.1" 502 150 2025-12-17 23:37:15,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409811575bc060ce1619c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47098 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:37:16,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940980c575bc060ce1619b8 2025-12-17 23:37:16,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980c575bc060ce1619b8 HTTP/1.1" 502 150 2025-12-17 23:37:16,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940980c575bc060ce1619b8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:16,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940980f575bc060ce1619bf 2025-12-17 23:37:16,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980f575bc060ce1619bf HTTP/1.1" 502 150 2025-12-17 23:37:16,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940980f575bc060ce1619bf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:16,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940980a575bc060ce1619ad 2025-12-17 23:37:16,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980a575bc060ce1619ad HTTP/1.1" 502 150 2025-12-17 23:37:16,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940980a575bc060ce1619ad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:17,312 - lava_callback - INFO - Received callback for node 69409b9e575bc060ce16224a INFO: 172.21.0.1:47106 - "POST /node/69409b9e575bc060ce16224a HTTP/1.0" 202 Accepted 2025-12-17 23:37:17,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f4575bc060ce16196b 2025-12-17 23:37:17,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f4575bc060ce16196b HTTP/1.1" 502 150 2025-12-17 23:37:17,486 - lava_callback - ERROR - Error processing callback for node 694097f4575bc060ce16196b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097f4575bc060ce16196b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097f4575bc060ce16196b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097f4575bc060ce16196b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:17,488 - lava_callback - INFO - Starting async job submit for node 69409812575bc060ce1619c5 2025-12-17 23:37:17,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:17,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409812575bc060ce1619c5 HTTP/1.1" 502 150 2025-12-17 23:37:17,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409812575bc060ce1619c5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:17,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409812575bc060ce1619c5 2025-12-17 23:37:17,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409812575bc060ce1619c5 HTTP/1.1" 502 150 2025-12-17 23:37:17,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409812575bc060ce1619c5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:17,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409811575bc060ce1619c2 2025-12-17 23:37:17,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409811575bc060ce1619c2 HTTP/1.1" 502 150 2025-12-17 23:37:17,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409811575bc060ce1619c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:17,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097f6575bc060ce161970 2025-12-17 23:37:17,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097f6575bc060ce161970 HTTP/1.1" 502 150 2025-12-17 23:37:17,954 - lava_callback - ERROR - Error processing callback for node 694097f6575bc060ce161970: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097f6575bc060ce161970 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097f6575bc060ce161970 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097f6575bc060ce161970 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:17,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ea575bc060ce161958 2025-12-17 23:37:17,956 - lava_callback - INFO - Starting async job submit for node 69409816575bc060ce1619cd 2025-12-17 23:37:17,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:17,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ea575bc060ce161958 HTTP/1.1" 502 150 2025-12-17 23:37:17,957 - lava_callback - ERROR - Error processing callback for node 694097ea575bc060ce161958: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ea575bc060ce161958 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ea575bc060ce161958 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ea575bc060ce161958 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:17,959 - lava_callback - INFO - Starting async job submit for node 69409815575bc060ce1619ca 2025-12-17 23:37:17,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:17,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409815575bc060ce1619ca HTTP/1.1" 502 150 2025-12-17 23:37:17,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409815575bc060ce1619ca'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:17,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409815575bc060ce1619ca 2025-12-17 23:37:17,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409815575bc060ce1619ca HTTP/1.1" 502 150 2025-12-17 23:37:17,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409815575bc060ce1619ca'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:17,983 - lava_callback - INFO - Received callback for node 69409ba3575bc060ce162252 INFO: 172.21.0.1:47118 - "POST /node/69409ba3575bc060ce162252 HTTP/1.0" 202 Accepted 2025-12-17 23:37:17,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409816575bc060ce1619cd HTTP/1.1" 502 150 2025-12-17 23:37:17,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409816575bc060ce1619cd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:17,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409816575bc060ce1619cd 2025-12-17 23:37:17,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409816575bc060ce1619cd HTTP/1.1" 502 150 2025-12-17 23:37:17,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409816575bc060ce1619cd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:18,209 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097eb575bc060ce16195b 2025-12-17 23:37:18,663 - lava_callback - INFO - Received callback for node 69409ba7575bc060ce16225e INFO: 172.21.0.1:47124 - "POST /node/69409ba7575bc060ce16225e HTTP/1.0" 202 Accepted 2025-12-17 23:37:19,326 - lava_callback - INFO - Received callback for node 69409ba8575bc060ce162261 INFO: 172.21.0.1:47136 - "POST /node/69409ba8575bc060ce162261 HTTP/1.0" 202 Accepted 2025-12-17 23:37:19,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ed575bc060ce16195e 2025-12-17 23:37:19,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ed575bc060ce16195e HTTP/1.1" 502 150 2025-12-17 23:37:19,473 - lava_callback - ERROR - Error processing callback for node 694097ed575bc060ce16195e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ed575bc060ce16195e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ed575bc060ce16195e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ed575bc060ce16195e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:19,475 - lava_callback - INFO - Starting async job submit for node 6940981a575bc060ce1619d3 2025-12-17 23:37:19,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:19,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981a575bc060ce1619d3 HTTP/1.1" 502 150 2025-12-17 23:37:19,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981a575bc060ce1619d3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:19,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981a575bc060ce1619d3 2025-12-17 23:37:19,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981a575bc060ce1619d3 HTTP/1.1" 502 150 2025-12-17 23:37:19,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981a575bc060ce1619d3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:19,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409812575bc060ce1619c5 2025-12-17 23:37:19,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409812575bc060ce1619c5 HTTP/1.1" 502 150 2025-12-17 23:37:19,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409812575bc060ce1619c5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:19,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409809575bc060ce1619a8 HTTP/1.1" 502 150 2025-12-17 23:37:19,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409809575bc060ce1619a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:19,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409815575bc060ce1619ca 2025-12-17 23:37:19,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409815575bc060ce1619ca HTTP/1.1" 502 150 2025-12-17 23:37:19,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409815575bc060ce1619ca'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:19,973 - lava_callback - INFO - Received callback for node 69409baa575bc060ce162264 INFO: 172.21.0.1:38390 - "POST /node/69409baa575bc060ce162264 HTTP/1.0" 202 Accepted 2025-12-17 23:37:19,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409816575bc060ce1619cd 2025-12-17 23:37:19,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409816575bc060ce1619cd HTTP/1.1" 502 150 2025-12-17 23:37:19,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409816575bc060ce1619cd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:20,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940980c575bc060ce1619b8 2025-12-17 23:37:20,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980c575bc060ce1619b8 HTTP/1.1" 502 150 2025-12-17 23:37:20,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940980c575bc060ce1619b8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:20,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940980f575bc060ce1619bf 2025-12-17 23:37:20,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980f575bc060ce1619bf HTTP/1.1" 502 150 2025-12-17 23:37:20,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940980f575bc060ce1619bf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:20,613 - lava_callback - INFO - Received callback for node 69409bab575bc060ce162267 INFO: 172.21.0.1:38404 - "POST /node/69409bab575bc060ce162267 HTTP/1.0" 202 Accepted 2025-12-17 23:37:21,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981a575bc060ce1619d3 2025-12-17 23:37:21,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981a575bc060ce1619d3 HTTP/1.1" 502 150 2025-12-17 23:37:21,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981a575bc060ce1619d3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:21,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409811575bc060ce1619c2 2025-12-17 23:37:21,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409811575bc060ce1619c2 HTTP/1.1" 502 150 2025-12-17 23:37:21,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409811575bc060ce1619c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:22,182 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097ef575bc060ce161961 2025-12-17 23:37:22,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097ef575bc060ce161961 HTTP/1.1" 502 150 2025-12-17 23:37:22,184 - lava_callback - ERROR - Error processing callback for node 694097ef575bc060ce161961: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ef575bc060ce161961 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ef575bc060ce161961 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097ef575bc060ce161961 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:22,186 - lava_callback - INFO - Starting async job submit for node 6940981c575bc060ce1619d9 2025-12-17 23:37:22,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:22,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981c575bc060ce1619d9 HTTP/1.1" 502 150 2025-12-17 23:37:22,193 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981c575bc060ce1619d9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:22,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981c575bc060ce1619d9 2025-12-17 23:37:22,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981c575bc060ce1619d9 HTTP/1.1" 502 150 2025-12-17 23:37:22,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981c575bc060ce1619d9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:23,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409812575bc060ce1619c5 2025-12-17 23:37:23,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409812575bc060ce1619c5 HTTP/1.1" 502 150 2025-12-17 23:37:23,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409812575bc060ce1619c5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:23,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409809575bc060ce1619a8 2025-12-17 23:37:23,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409809575bc060ce1619a8 HTTP/1.1" 502 150 2025-12-17 23:37:23,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409809575bc060ce1619a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:23,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409815575bc060ce1619ca 2025-12-17 23:37:23,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409815575bc060ce1619ca HTTP/1.1" 502 150 2025-12-17 23:37:23,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409815575bc060ce1619ca'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:23,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409816575bc060ce1619cd 2025-12-17 23:37:23,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409816575bc060ce1619cd HTTP/1.1" 502 150 2025-12-17 23:37:23,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409816575bc060ce1619cd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:24,194 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981c575bc060ce1619d9 2025-12-17 23:37:24,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981c575bc060ce1619d9 HTTP/1.1" 502 150 2025-12-17 23:37:24,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981c575bc060ce1619d9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:24,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940980a575bc060ce1619ad 2025-12-17 23:37:24,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980a575bc060ce1619ad HTTP/1.1" 502 150 2025-12-17 23:37:24,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940980a575bc060ce1619ad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:25,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981a575bc060ce1619d3 2025-12-17 23:37:25,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981a575bc060ce1619d3 HTTP/1.1" 502 150 2025-12-17 23:37:25,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981a575bc060ce1619d3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38408 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:37:26,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097fd575bc060ce161988 2025-12-17 23:37:26,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fd575bc060ce161988 HTTP/1.1" 502 150 2025-12-17 23:37:26,773 - lava_callback - ERROR - Error processing callback for node 694097fd575bc060ce161988: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097fd575bc060ce161988 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097fd575bc060ce161988 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097fd575bc060ce161988 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:26,775 - lava_callback - INFO - Starting async job submit for node 6940981b575bc060ce1619d6 2025-12-17 23:37:26,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:26,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981b575bc060ce1619d6 HTTP/1.1" 502 150 2025-12-17 23:37:26,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981b575bc060ce1619d6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:26,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981b575bc060ce1619d6 2025-12-17 23:37:26,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981b575bc060ce1619d6 HTTP/1.1" 502 150 2025-12-17 23:37:26,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981b575bc060ce1619d6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:27,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097fa575bc060ce161982 2025-12-17 23:37:27,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097fa575bc060ce161982 HTTP/1.1" 502 150 2025-12-17 23:37:27,665 - lava_callback - ERROR - Error processing callback for node 694097fa575bc060ce161982: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097fa575bc060ce161982 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097fa575bc060ce161982 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097fa575bc060ce161982 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:27,667 - lava_callback - INFO - Starting async job submit for node 69409818575bc060ce1619d0 2025-12-17 23:37:27,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:27,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409818575bc060ce1619d0 HTTP/1.1" 502 150 2025-12-17 23:37:27,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409818575bc060ce1619d0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:27,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409818575bc060ce1619d0 2025-12-17 23:37:27,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409818575bc060ce1619d0 HTTP/1.1" 502 150 2025-12-17 23:37:27,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409818575bc060ce1619d0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:27,919 - lava_callback - INFO - Received callback for node 69409ba5575bc060ce16225b INFO: 172.21.0.1:38424 - "POST /node/69409ba5575bc060ce16225b HTTP/1.0" 202 Accepted 2025-12-17 23:37:28,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981c575bc060ce1619d9 2025-12-17 23:37:28,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981c575bc060ce1619d9 HTTP/1.1" 502 150 2025-12-17 23:37:28,197 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981c575bc060ce1619d9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:28,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940980c575bc060ce1619b8 2025-12-17 23:37:28,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980c575bc060ce1619b8 HTTP/1.1" 502 150 2025-12-17 23:37:28,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940980c575bc060ce1619b8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:28,607 - lava_callback - INFO - Received callback for node 69409bad575bc060ce16226e INFO: 172.21.0.1:38428 - "POST /node/69409bad575bc060ce16226e HTTP/1.0" 202 Accepted 2025-12-17 23:37:28,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940980f575bc060ce1619bf 2025-12-17 23:37:28,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980f575bc060ce1619bf HTTP/1.1" 502 150 2025-12-17 23:37:28,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940980f575bc060ce1619bf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:28,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981b575bc060ce1619d6 2025-12-17 23:37:28,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981b575bc060ce1619d6 HTTP/1.1" 502 150 2025-12-17 23:37:28,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981b575bc060ce1619d6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:29,156 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409805575bc060ce16199a 2025-12-17 23:37:29,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409805575bc060ce16199a HTTP/1.1" 502 150 2025-12-17 23:37:29,157 - lava_callback - ERROR - Error processing callback for node 69409805575bc060ce16199a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409805575bc060ce16199a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409805575bc060ce16199a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409805575bc060ce16199a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:29,159 - lava_callback - INFO - Starting async job submit for node 6940981e575bc060ce1619dc 2025-12-17 23:37:29,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:29,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981e575bc060ce1619dc HTTP/1.1" 502 150 2025-12-17 23:37:29,166 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981e575bc060ce1619dc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:29,166 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981e575bc060ce1619dc 2025-12-17 23:37:29,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981e575bc060ce1619dc HTTP/1.1" 502 150 2025-12-17 23:37:29,167 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981e575bc060ce1619dc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:29,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409811575bc060ce1619c2 2025-12-17 23:37:29,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409811575bc060ce1619c2 HTTP/1.1" 502 150 2025-12-17 23:37:29,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409811575bc060ce1619c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:29,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409818575bc060ce1619d0 2025-12-17 23:37:29,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409818575bc060ce1619d0 HTTP/1.1" 502 150 2025-12-17 23:37:29,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409818575bc060ce1619d0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:29,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409808575bc060ce1619a5 2025-12-17 23:37:29,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409808575bc060ce1619a5 HTTP/1.1" 502 150 2025-12-17 23:37:29,945 - lava_callback - ERROR - Error processing callback for node 69409808575bc060ce1619a5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409808575bc060ce1619a5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409808575bc060ce1619a5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409808575bc060ce1619a5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:29,946 - lava_callback - INFO - Starting async job submit for node 69409823575bc060ce1619e6 2025-12-17 23:37:29,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:29,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409823575bc060ce1619e6 HTTP/1.1" 502 150 2025-12-17 23:37:29,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409823575bc060ce1619e6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:29,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409823575bc060ce1619e6 2025-12-17 23:37:29,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409823575bc060ce1619e6 HTTP/1.1" 502 150 2025-12-17 23:37:29,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409823575bc060ce1619e6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:30,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097df575bc060ce16193a HTTP/1.1" 502 150 2025-12-17 23:37:30,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097df575bc060ce16193a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:31,167 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981e575bc060ce1619dc 2025-12-17 23:37:31,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409812575bc060ce1619c5 2025-12-17 23:37:31,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409812575bc060ce1619c5 HTTP/1.1" 502 150 2025-12-17 23:37:31,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409812575bc060ce1619c5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:31,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409809575bc060ce1619a8 2025-12-17 23:37:31,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409809575bc060ce1619a8 HTTP/1.1" 502 150 2025-12-17 23:37:31,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409809575bc060ce1619a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:31,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409823575bc060ce1619e6 2025-12-17 23:37:31,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409823575bc060ce1619e6 HTTP/1.1" 502 150 2025-12-17 23:37:31,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409823575bc060ce1619e6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:31,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409815575bc060ce1619ca 2025-12-17 23:37:31,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409815575bc060ce1619ca HTTP/1.1" 502 150 2025-12-17 23:37:31,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409815575bc060ce1619ca'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:31,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409816575bc060ce1619cd 2025-12-17 23:37:31,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409816575bc060ce1619cd HTTP/1.1" 502 150 2025-12-17 23:37:31,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409816575bc060ce1619cd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:32,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097df575bc060ce16193a 2025-12-17 23:37:32,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981b575bc060ce1619d6 2025-12-17 23:37:32,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981b575bc060ce1619d6 HTTP/1.1" 502 150 2025-12-17 23:37:32,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981b575bc060ce1619d6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:33,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981a575bc060ce1619d3 2025-12-17 23:37:33,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981a575bc060ce1619d3 HTTP/1.1" 502 150 2025-12-17 23:37:33,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981a575bc060ce1619d3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:33,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097eb575bc060ce16195b HTTP/1.1" 502 150 2025-12-17 23:37:33,616 - lava_callback - ERROR - Error processing callback for node 694097eb575bc060ce16195b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097eb575bc060ce16195b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097eb575bc060ce16195b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097eb575bc060ce16195b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:33,619 - lava_callback - INFO - Starting async job submit for node 694124cd459ff5fc64d2bd7b 2025-12-17 23:37:33,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:33,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409818575bc060ce1619d0 2025-12-17 23:37:33,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409818575bc060ce1619d0 HTTP/1.1" 502 150 2025-12-17 23:37:33,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409818575bc060ce1619d0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:33,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694124cd459ff5fc64d2bd7b HTTP/1.1" 502 150 2025-12-17 23:37:33,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694124cd459ff5fc64d2bd7b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:33,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694124cd459ff5fc64d2bd7b 2025-12-17 23:37:33,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694124cd459ff5fc64d2bd7b HTTP/1.1" 502 150 2025-12-17 23:37:33,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694124cd459ff5fc64d2bd7b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:35,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694124cd459ff5fc64d2bd7b 2025-12-17 23:37:35,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694124cd459ff5fc64d2bd7b HTTP/1.1" 502 150 2025-12-17 23:37:35,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694124cd459ff5fc64d2bd7b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:35,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409823575bc060ce1619e6 2025-12-17 23:37:35,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409823575bc060ce1619e6 HTTP/1.1" 502 150 2025-12-17 23:37:35,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409823575bc060ce1619e6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:36,197 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981c575bc060ce1619d9 2025-12-17 23:37:36,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981c575bc060ce1619d9 HTTP/1.1" 502 150 2025-12-17 23:37:36,198 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981c575bc060ce1619d9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33820 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:37:37,215 - lava_callback - INFO - Received callback for node 69409bb0575bc060ce162271 INFO: 172.21.0.1:33824 - "POST /node/69409bb0575bc060ce162271 HTTP/1.0" 202 Accepted 2025-12-17 23:37:37,894 - lava_callback - INFO - Received callback for node 69409bb4575bc060ce1623d8 INFO: 172.21.0.1:33830 - "POST /node/69409bb4575bc060ce1623d8 HTTP/1.0" 202 Accepted 2025-12-17 23:37:38,585 - lava_callback - INFO - Received callback for node 69409bb6575bc060ce1623dd INFO: 172.21.0.1:33844 - "POST /node/69409bb6575bc060ce1623dd HTTP/1.0" 202 Accepted 2025-12-17 23:37:39,265 - lava_callback - INFO - Received callback for node 69409bb8575bc060ce1623e4 INFO: 172.21.0.1:33846 - "POST /node/69409bb8575bc060ce1623e4 HTTP/1.0" 202 Accepted 2025-12-17 23:37:39,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694124cd459ff5fc64d2bd7b 2025-12-17 23:37:39,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694124cd459ff5fc64d2bd7b HTTP/1.1" 502 150 2025-12-17 23:37:39,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694124cd459ff5fc64d2bd7b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:39,941 - lava_callback - INFO - Received callback for node 69409bba575bc060ce1623e9 INFO: 172.21.0.1:38356 - "POST /node/69409bba575bc060ce1623e9 HTTP/1.0" 202 Accepted 2025-12-17 23:37:40,617 - lava_callback - INFO - Received callback for node 69409bbb575bc060ce1623ec INFO: 172.21.0.1:38366 - "POST /node/69409bbb575bc060ce1623ec HTTP/1.0" 202 Accepted 2025-12-17 23:37:40,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981b575bc060ce1619d6 2025-12-17 23:37:40,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981b575bc060ce1619d6 HTTP/1.1" 502 150 2025-12-17 23:37:40,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981b575bc060ce1619d6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:40,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940980a575bc060ce1619ad 2025-12-17 23:37:40,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980a575bc060ce1619ad HTTP/1.1" 502 150 2025-12-17 23:37:40,948 - lava_callback - ERROR - Error processing callback for node 6940980a575bc060ce1619ad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940980a575bc060ce1619ad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940980a575bc060ce1619ad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940980a575bc060ce1619ad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:40,950 - lava_callback - INFO - Starting async job submit for node 69409821575bc060ce1619e3 2025-12-17 23:37:40,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:40,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409821575bc060ce1619e3 HTTP/1.1" 502 150 2025-12-17 23:37:40,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409821575bc060ce1619e3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:40,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409821575bc060ce1619e3 2025-12-17 23:37:40,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409821575bc060ce1619e3 HTTP/1.1" 502 150 2025-12-17 23:37:40,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409821575bc060ce1619e3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:41,291 - lava_callback - INFO - Received callback for node 69409bbe575bc060ce1623f4 INFO: 172.21.0.1:38378 - "POST /node/69409bbe575bc060ce1623f4 HTTP/1.0" 202 Accepted 2025-12-17 23:37:41,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409818575bc060ce1619d0 2025-12-17 23:37:41,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409818575bc060ce1619d0 HTTP/1.1" 502 150 2025-12-17 23:37:41,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409818575bc060ce1619d0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:42,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409821575bc060ce1619e3 2025-12-17 23:37:42,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409821575bc060ce1619e3 HTTP/1.1" 502 150 2025-12-17 23:37:42,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409821575bc060ce1619e3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:43,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409823575bc060ce1619e6 2025-12-17 23:37:43,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409823575bc060ce1619e6 HTTP/1.1" 502 150 2025-12-17 23:37:43,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409823575bc060ce1619e6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:44,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940980c575bc060ce1619b8 2025-12-17 23:37:44,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940980f575bc060ce1619bf 2025-12-17 23:37:44,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980f575bc060ce1619bf HTTP/1.1" 502 150 2025-12-17 23:37:44,615 - lava_callback - ERROR - Error processing callback for node 6940980f575bc060ce1619bf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940980f575bc060ce1619bf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940980f575bc060ce1619bf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940980f575bc060ce1619bf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:44,617 - lava_callback - INFO - Starting async job submit for node 69409826575bc060ce1619ec 2025-12-17 23:37:44,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:44,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409826575bc060ce1619ec HTTP/1.1" 502 150 2025-12-17 23:37:44,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409826575bc060ce1619ec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:44,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409826575bc060ce1619ec 2025-12-17 23:37:44,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409826575bc060ce1619ec HTTP/1.1" 502 150 2025-12-17 23:37:44,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409826575bc060ce1619ec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:45,150 - lava_callback - INFO - Received callback for node 69409c43575bc060ce162555 INFO: 172.21.0.1:38392 - "POST /node/69409c43575bc060ce162555 HTTP/1.0" 202 Accepted 2025-12-17 23:37:45,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409811575bc060ce1619c2 2025-12-17 23:37:46,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981e575bc060ce1619dc HTTP/1.1" 502 150 2025-12-17 23:37:46,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981e575bc060ce1619dc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38406 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:37:46,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409826575bc060ce1619ec 2025-12-17 23:37:46,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409826575bc060ce1619ec HTTP/1.1" 502 150 2025-12-17 23:37:46,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409826575bc060ce1619ec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:46,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409821575bc060ce1619e3 2025-12-17 23:37:46,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409821575bc060ce1619e3 HTTP/1.1" 502 150 2025-12-17 23:37:46,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409821575bc060ce1619e3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:47,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409812575bc060ce1619c5 2025-12-17 23:37:47,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409812575bc060ce1619c5 HTTP/1.1" 502 150 2025-12-17 23:37:47,510 - lava_callback - ERROR - Error processing callback for node 69409812575bc060ce1619c5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409812575bc060ce1619c5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409812575bc060ce1619c5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409812575bc060ce1619c5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:47,512 - lava_callback - INFO - Starting async job submit for node 69409825575bc060ce1619e9 2025-12-17 23:37:47,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:47,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409825575bc060ce1619e9 HTTP/1.1" 502 150 2025-12-17 23:37:47,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409825575bc060ce1619e9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:47,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409825575bc060ce1619e9 2025-12-17 23:37:47,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409825575bc060ce1619e9 HTTP/1.1" 502 150 2025-12-17 23:37:47,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409825575bc060ce1619e9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:47,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409809575bc060ce1619a8 2025-12-17 23:37:47,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409809575bc060ce1619a8 HTTP/1.1" 502 150 2025-12-17 23:37:47,541 - lava_callback - ERROR - Error processing callback for node 69409809575bc060ce1619a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409809575bc060ce1619a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409809575bc060ce1619a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409809575bc060ce1619a8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:47,543 - lava_callback - INFO - Starting async job submit for node 69409828575bc060ce1619f4 2025-12-17 23:37:47,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:47,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409828575bc060ce1619f4 HTTP/1.1" 502 150 2025-12-17 23:37:47,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409828575bc060ce1619f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:47,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409828575bc060ce1619f4 2025-12-17 23:37:47,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409828575bc060ce1619f4 HTTP/1.1" 502 150 2025-12-17 23:37:47,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409828575bc060ce1619f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:47,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694124cd459ff5fc64d2bd7b 2025-12-17 23:37:47,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694124cd459ff5fc64d2bd7b HTTP/1.1" 502 150 2025-12-17 23:37:47,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694124cd459ff5fc64d2bd7b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:47,965 - lava_callback - INFO - Received callback for node 69409bc0575bc060ce1623fe INFO: 172.21.0.1:38408 - "POST /node/69409bc0575bc060ce1623fe HTTP/1.0" 202 Accepted 2025-12-17 23:37:47,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409815575bc060ce1619ca 2025-12-17 23:37:47,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409815575bc060ce1619ca HTTP/1.1" 502 150 2025-12-17 23:37:47,973 - lava_callback - ERROR - Error processing callback for node 69409815575bc060ce1619ca: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409815575bc060ce1619ca (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409815575bc060ce1619ca (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409815575bc060ce1619ca (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:47,974 - lava_callback - INFO - Starting async job submit for node 6940982a575bc060ce1619f7 2025-12-17 23:37:47,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:47,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409816575bc060ce1619cd 2025-12-17 23:37:48,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409816575bc060ce1619cd HTTP/1.1" 502 150 2025-12-17 23:37:48,000 - lava_callback - ERROR - Error processing callback for node 69409816575bc060ce1619cd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409816575bc060ce1619cd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409816575bc060ce1619cd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409816575bc060ce1619cd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:48,002 - lava_callback - INFO - Starting async job submit for node 6940982c575bc060ce1619fe 2025-12-17 23:37:48,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:48,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982a575bc060ce1619f7 HTTP/1.1" 502 150 2025-12-17 23:37:48,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940982a575bc060ce1619f7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:48,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940982a575bc060ce1619f7 2025-12-17 23:37:48,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982a575bc060ce1619f7 HTTP/1.1" 502 150 2025-12-17 23:37:48,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940982a575bc060ce1619f7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:48,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982c575bc060ce1619fe HTTP/1.1" 502 150 2025-12-17 23:37:48,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940982c575bc060ce1619fe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:48,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940982c575bc060ce1619fe 2025-12-17 23:37:48,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982c575bc060ce1619fe HTTP/1.1" 502 150 2025-12-17 23:37:48,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940982c575bc060ce1619fe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:48,639 - lava_callback - INFO - Received callback for node 69409bc8575bc060ce16240a INFO: 172.21.0.1:38414 - "POST /node/69409bc8575bc060ce16240a HTTP/1.0" 202 Accepted 2025-12-17 23:37:49,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981a575bc060ce1619d3 2025-12-17 23:37:49,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981a575bc060ce1619d3 HTTP/1.1" 502 150 2025-12-17 23:37:49,510 - lava_callback - ERROR - Error processing callback for node 6940981a575bc060ce1619d3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940981a575bc060ce1619d3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940981a575bc060ce1619d3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940981a575bc060ce1619d3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:49,512 - lava_callback - INFO - Starting async job submit for node 69409831575bc060ce161a04 2025-12-17 23:37:49,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:49,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409831575bc060ce161a04 HTTP/1.1" 502 150 2025-12-17 23:37:49,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409831575bc060ce161a04'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:49,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409831575bc060ce161a04 2025-12-17 23:37:49,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409831575bc060ce161a04 HTTP/1.1" 502 150 2025-12-17 23:37:49,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409831575bc060ce161a04'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:49,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409825575bc060ce1619e9 2025-12-17 23:37:49,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409825575bc060ce1619e9 HTTP/1.1" 502 150 2025-12-17 23:37:49,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409825575bc060ce1619e9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:49,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409828575bc060ce1619f4 2025-12-17 23:37:49,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409828575bc060ce1619f4 HTTP/1.1" 502 150 2025-12-17 23:37:49,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409828575bc060ce1619f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:50,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940982a575bc060ce1619f7 2025-12-17 23:37:50,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982a575bc060ce1619f7 HTTP/1.1" 502 150 2025-12-17 23:37:50,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940982a575bc060ce1619f7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:50,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940982c575bc060ce1619fe 2025-12-17 23:37:50,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982c575bc060ce1619fe HTTP/1.1" 502 150 2025-12-17 23:37:50,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940982c575bc060ce1619fe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:50,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981e575bc060ce1619dc 2025-12-17 23:37:50,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981e575bc060ce1619dc HTTP/1.1" 502 150 2025-12-17 23:37:50,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981e575bc060ce1619dc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:50,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409826575bc060ce1619ec 2025-12-17 23:37:50,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409826575bc060ce1619ec HTTP/1.1" 502 150 2025-12-17 23:37:50,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409826575bc060ce1619ec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:51,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409831575bc060ce161a04 2025-12-17 23:37:51,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409831575bc060ce161a04 HTTP/1.1" 502 150 2025-12-17 23:37:51,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409831575bc060ce161a04'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:52,198 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981c575bc060ce1619d9 2025-12-17 23:37:52,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981c575bc060ce1619d9 HTTP/1.1" 502 150 2025-12-17 23:37:52,200 - lava_callback - ERROR - Error processing callback for node 6940981c575bc060ce1619d9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940981c575bc060ce1619d9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940981c575bc060ce1619d9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940981c575bc060ce1619d9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:52,202 - lava_callback - INFO - Starting async job submit for node 69409958575bc060ce161cb9 2025-12-17 23:37:52,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:52,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409958575bc060ce161cb9 HTTP/1.1" 502 150 2025-12-17 23:37:52,230 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409958575bc060ce161cb9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:52,230 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409958575bc060ce161cb9 2025-12-17 23:37:52,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409958575bc060ce161cb9 HTTP/1.1" 502 150 2025-12-17 23:37:52,238 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409958575bc060ce161cb9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:53,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409825575bc060ce1619e9 2025-12-17 23:37:53,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409825575bc060ce1619e9 HTTP/1.1" 502 150 2025-12-17 23:37:53,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409825575bc060ce1619e9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:53,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409828575bc060ce1619f4 2025-12-17 23:37:53,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409828575bc060ce1619f4 HTTP/1.1" 502 150 2025-12-17 23:37:53,551 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409828575bc060ce1619f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:54,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940982a575bc060ce1619f7 2025-12-17 23:37:54,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982a575bc060ce1619f7 HTTP/1.1" 502 150 2025-12-17 23:37:54,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940982a575bc060ce1619f7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:54,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940982c575bc060ce1619fe 2025-12-17 23:37:54,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982c575bc060ce1619fe HTTP/1.1" 502 150 2025-12-17 23:37:54,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940982c575bc060ce1619fe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:54,238 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409958575bc060ce161cb9 2025-12-17 23:37:54,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409958575bc060ce161cb9 HTTP/1.1" 502 150 2025-12-17 23:37:54,239 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409958575bc060ce161cb9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:54,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409821575bc060ce1619e3 2025-12-17 23:37:54,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409821575bc060ce1619e3 HTTP/1.1" 502 150 2025-12-17 23:37:54,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409821575bc060ce1619e3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:55,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409831575bc060ce161a04 2025-12-17 23:37:55,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409831575bc060ce161a04 HTTP/1.1" 502 150 2025-12-17 23:37:55,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409831575bc060ce161a04'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59550 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:37:56,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981b575bc060ce1619d6 2025-12-17 23:37:56,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981b575bc060ce1619d6 HTTP/1.1" 502 150 2025-12-17 23:37:56,818 - lava_callback - ERROR - Error processing callback for node 6940981b575bc060ce1619d6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940981b575bc060ce1619d6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940981b575bc060ce1619d6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940981b575bc060ce1619d6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:56,819 - lava_callback - INFO - Starting async job submit for node 6940982e575bc060ce161a01 2025-12-17 23:37:56,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:56,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982e575bc060ce161a01 HTTP/1.1" 502 150 2025-12-17 23:37:56,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940982e575bc060ce161a01'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:56,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940982e575bc060ce161a01 2025-12-17 23:37:56,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982e575bc060ce161a01 HTTP/1.1" 502 150 2025-12-17 23:37:56,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940982e575bc060ce161a01'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:57,276 - lava_callback - INFO - Received callback for node 69409bc2575bc060ce162401 INFO: 172.21.0.1:59552 - "POST /node/69409bc2575bc060ce162401 HTTP/1.0" 202 Accepted 2025-12-17 23:37:57,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409818575bc060ce1619d0 2025-12-17 23:37:57,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409818575bc060ce1619d0 HTTP/1.1" 502 150 2025-12-17 23:37:57,685 - lava_callback - ERROR - Error processing callback for node 69409818575bc060ce1619d0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409818575bc060ce1619d0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409818575bc060ce1619d0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409818575bc060ce1619d0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:57,687 - lava_callback - INFO - Starting async job submit for node 69409833575bc060ce161a0d 2025-12-17 23:37:57,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:57,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409833575bc060ce161a0d HTTP/1.1" 502 150 2025-12-17 23:37:57,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409833575bc060ce161a0d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:57,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409833575bc060ce161a0d 2025-12-17 23:37:57,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409833575bc060ce161a0d HTTP/1.1" 502 150 2025-12-17 23:37:57,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409833575bc060ce161a0d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:57,955 - lava_callback - INFO - Received callback for node 69409bc5575bc060ce162404 INFO: 172.21.0.1:59554 - "POST /node/69409bc5575bc060ce162404 HTTP/1.0" 202 Accepted 2025-12-17 23:37:58,239 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409958575bc060ce161cb9 2025-12-17 23:37:58,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409958575bc060ce161cb9 HTTP/1.1" 502 150 2025-12-17 23:37:58,241 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409958575bc060ce161cb9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:58,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981e575bc060ce1619dc 2025-12-17 23:37:58,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981e575bc060ce1619dc HTTP/1.1" 502 150 2025-12-17 23:37:58,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940981e575bc060ce1619dc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:58,621 - lava_callback - INFO - Received callback for node 69409bc7575bc060ce162407 INFO: 172.21.0.1:59556 - "POST /node/69409bc7575bc060ce162407 HTTP/1.0" 202 Accepted 2025-12-17 23:37:58,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409826575bc060ce1619ec 2025-12-17 23:37:58,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409826575bc060ce1619ec HTTP/1.1" 502 150 2025-12-17 23:37:58,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409826575bc060ce1619ec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:58,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940982e575bc060ce161a01 2025-12-17 23:37:58,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982e575bc060ce161a01 HTTP/1.1" 502 150 2025-12-17 23:37:58,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940982e575bc060ce161a01'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:59,299 - lava_callback - INFO - Received callback for node 69409bcb575bc060ce162411 INFO: 172.21.0.1:59562 - "POST /node/69409bcb575bc060ce162411 HTTP/1.0" 202 Accepted 2025-12-17 23:37:59,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409833575bc060ce161a0d 2025-12-17 23:37:59,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409833575bc060ce161a0d HTTP/1.1" 502 150 2025-12-17 23:37:59,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409833575bc060ce161a0d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:59,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409823575bc060ce1619e6 2025-12-17 23:37:59,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409823575bc060ce1619e6 HTTP/1.1" 502 150 2025-12-17 23:37:59,960 - lava_callback - ERROR - Error processing callback for node 69409823575bc060ce1619e6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409823575bc060ce1619e6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409823575bc060ce1619e6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409823575bc060ce1619e6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:59,961 - lava_callback - INFO - Starting async job submit for node 69409836575bc060ce161a18 2025-12-17 23:37:59,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:59,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409836575bc060ce161a18 HTTP/1.1" 502 150 2025-12-17 23:37:59,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409836575bc060ce161a18'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:59,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409836575bc060ce161a18 2025-12-17 23:37:59,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409836575bc060ce161a18 HTTP/1.1" 502 150 2025-12-17 23:37:59,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409836575bc060ce161a18'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:59,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940980c575bc060ce1619b8 HTTP/1.1" 502 150 2025-12-17 23:37:59,984 - lava_callback - ERROR - Error processing callback for node 6940980c575bc060ce1619b8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940980c575bc060ce1619b8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940980c575bc060ce1619b8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940980c575bc060ce1619b8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:37:59,986 - lava_callback - INFO - Starting async job submit for node 69409839575bc060ce161a1e 2025-12-17 23:37:59,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:37:59,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409839575bc060ce161a1e HTTP/1.1" 502 150 2025-12-17 23:37:59,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409839575bc060ce161a1e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:37:59,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409839575bc060ce161a1e 2025-12-17 23:37:59,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409839575bc060ce161a1e HTTP/1.1" 502 150 2025-12-17 23:37:59,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409839575bc060ce161a1e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:01,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409811575bc060ce1619c2 HTTP/1.1" 502 150 2025-12-17 23:38:01,008 - lava_callback - ERROR - Error processing callback for node 69409811575bc060ce1619c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409811575bc060ce1619c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409811575bc060ce1619c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409811575bc060ce1619c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:01,010 - lava_callback - INFO - Starting async job submit for node 6940983b575bc060ce161a21 2025-12-17 23:38:01,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:01,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983b575bc060ce161a21 HTTP/1.1" 502 150 2025-12-17 23:38:01,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983b575bc060ce161a21'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:01,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983b575bc060ce161a21 2025-12-17 23:38:01,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983b575bc060ce161a21 HTTP/1.1" 502 150 2025-12-17 23:38:01,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983b575bc060ce161a21'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:01,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409825575bc060ce1619e9 2025-12-17 23:38:01,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409825575bc060ce1619e9 HTTP/1.1" 502 150 2025-12-17 23:38:01,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409825575bc060ce1619e9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:01,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409828575bc060ce1619f4 2025-12-17 23:38:01,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409828575bc060ce1619f4 HTTP/1.1" 502 150 2025-12-17 23:38:01,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409828575bc060ce1619f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:01,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409836575bc060ce161a18 2025-12-17 23:38:01,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409836575bc060ce161a18 HTTP/1.1" 502 150 2025-12-17 23:38:01,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409836575bc060ce161a18'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:01,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409839575bc060ce161a1e 2025-12-17 23:38:01,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409839575bc060ce161a1e HTTP/1.1" 502 150 2025-12-17 23:38:01,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409839575bc060ce161a1e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:02,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940982a575bc060ce1619f7 2025-12-17 23:38:02,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982a575bc060ce1619f7 HTTP/1.1" 502 150 2025-12-17 23:38:02,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940982a575bc060ce1619f7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:02,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940982c575bc060ce1619fe 2025-12-17 23:38:02,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982c575bc060ce1619fe HTTP/1.1" 502 150 2025-12-17 23:38:02,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940982c575bc060ce1619fe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:02,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097df575bc060ce16193a HTTP/1.1" 502 150 2025-12-17 23:38:02,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097df575bc060ce16193a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:02,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940982e575bc060ce161a01 2025-12-17 23:38:02,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982e575bc060ce161a01 HTTP/1.1" 502 150 2025-12-17 23:38:02,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940982e575bc060ce161a01'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:03,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983b575bc060ce161a21 2025-12-17 23:38:03,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983b575bc060ce161a21 HTTP/1.1" 502 150 2025-12-17 23:38:03,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983b575bc060ce161a21'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33664 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:38:03,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409831575bc060ce161a04 2025-12-17 23:38:03,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409831575bc060ce161a04 HTTP/1.1" 502 150 2025-12-17 23:38:03,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409831575bc060ce161a04'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:03,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409833575bc060ce161a0d 2025-12-17 23:38:03,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409833575bc060ce161a0d HTTP/1.1" 502 150 2025-12-17 23:38:03,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409833575bc060ce161a0d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:03,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694124cd459ff5fc64d2bd7b 2025-12-17 23:38:03,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694124cd459ff5fc64d2bd7b HTTP/1.1" 502 150 2025-12-17 23:38:03,716 - lava_callback - ERROR - Error processing callback for node 694124cd459ff5fc64d2bd7b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694124cd459ff5fc64d2bd7b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694124cd459ff5fc64d2bd7b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694124cd459ff5fc64d2bd7b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:03,718 - lava_callback - INFO - Starting async job submit for node 6940c5431e3c24e95681d6b6 2025-12-17 23:38:03,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:03,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5431e3c24e95681d6b6 HTTP/1.1" 502 150 2025-12-17 23:38:03,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5431e3c24e95681d6b6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:03,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5431e3c24e95681d6b6 2025-12-17 23:38:03,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5431e3c24e95681d6b6 HTTP/1.1" 502 150 2025-12-17 23:38:03,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5431e3c24e95681d6b6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:05,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5431e3c24e95681d6b6 2025-12-17 23:38:05,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5431e3c24e95681d6b6 HTTP/1.1" 502 150 2025-12-17 23:38:05,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5431e3c24e95681d6b6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:05,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409836575bc060ce161a18 2025-12-17 23:38:05,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409836575bc060ce161a18 HTTP/1.1" 502 150 2025-12-17 23:38:05,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409836575bc060ce161a18'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:05,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409839575bc060ce161a1e 2025-12-17 23:38:05,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409839575bc060ce161a1e HTTP/1.1" 502 150 2025-12-17 23:38:05,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409839575bc060ce161a1e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:06,241 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409958575bc060ce161cb9 2025-12-17 23:38:06,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409958575bc060ce161cb9 HTTP/1.1" 502 150 2025-12-17 23:38:06,242 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409958575bc060ce161cb9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33672 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:38:06,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097df575bc060ce16193a 2025-12-17 23:38:07,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983b575bc060ce161a21 2025-12-17 23:38:07,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983b575bc060ce161a21 HTTP/1.1" 502 150 2025-12-17 23:38:07,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983b575bc060ce161a21'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:07,980 - lava_callback - INFO - Received callback for node 69409bcd575bc060ce16241a INFO: 172.21.0.1:33674 - "POST /node/69409bcd575bc060ce16241a HTTP/1.0" 202 Accepted 2025-12-17 23:38:08,640 - lava_callback - INFO - Received callback for node 69409bcf575bc060ce16241f INFO: 172.21.0.1:33690 - "POST /node/69409bcf575bc060ce16241f HTTP/1.0" 202 Accepted 2025-12-17 23:38:09,333 - lava_callback - INFO - Received callback for node 69409bd1575bc060ce162427 INFO: 172.21.0.1:33704 - "POST /node/69409bd1575bc060ce162427 HTTP/1.0" 202 Accepted 2025-12-17 23:38:09,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5431e3c24e95681d6b6 2025-12-17 23:38:09,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5431e3c24e95681d6b6 HTTP/1.1" 502 150 2025-12-17 23:38:09,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5431e3c24e95681d6b6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:09,996 - lava_callback - INFO - Received callback for node 69409bd3575bc060ce16242a INFO: 172.21.0.1:54338 - "POST /node/69409bd3575bc060ce16242a HTTP/1.0" 202 Accepted 2025-12-17 23:38:10,652 - lava_callback - INFO - Received callback for node 69409bd5575bc060ce16242d INFO: 172.21.0.1:54352 - "POST /node/69409bd5575bc060ce16242d HTTP/1.0" 202 Accepted 2025-12-17 23:38:10,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940982e575bc060ce161a01 2025-12-17 23:38:10,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982e575bc060ce161a01 HTTP/1.1" 502 150 2025-12-17 23:38:10,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940982e575bc060ce161a01'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:10,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409821575bc060ce1619e3 2025-12-17 23:38:10,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409821575bc060ce1619e3 HTTP/1.1" 502 150 2025-12-17 23:38:10,964 - lava_callback - ERROR - Error processing callback for node 69409821575bc060ce1619e3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409821575bc060ce1619e3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409821575bc060ce1619e3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409821575bc060ce1619e3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:10,965 - lava_callback - INFO - Starting async job submit for node 69409804575bc060ce161997 2025-12-17 23:38:11,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:11,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409804575bc060ce161997 HTTP/1.1" 502 150 2025-12-17 23:38:11,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409804575bc060ce161997'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:11,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409804575bc060ce161997 2025-12-17 23:38:11,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409804575bc060ce161997 HTTP/1.1" 502 150 2025-12-17 23:38:11,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409804575bc060ce161997'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:11,341 - lava_callback - INFO - Received callback for node 69409bd7575bc060ce162434 INFO: 172.21.0.1:54354 - "POST /node/69409bd7575bc060ce162434 HTTP/1.0" 202 Accepted 2025-12-17 23:38:11,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409833575bc060ce161a0d 2025-12-17 23:38:11,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409833575bc060ce161a0d HTTP/1.1" 502 150 2025-12-17 23:38:11,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409833575bc060ce161a0d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:13,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409804575bc060ce161997 2025-12-17 23:38:13,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409804575bc060ce161997 HTTP/1.1" 502 150 2025-12-17 23:38:13,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409804575bc060ce161997'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:13,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409836575bc060ce161a18 2025-12-17 23:38:13,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409836575bc060ce161a18 HTTP/1.1" 502 150 2025-12-17 23:38:13,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409836575bc060ce161a18'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:13,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409839575bc060ce161a1e 2025-12-17 23:38:13,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409839575bc060ce161a1e HTTP/1.1" 502 150 2025-12-17 23:38:13,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409839575bc060ce161a1e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:14,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940981e575bc060ce1619dc 2025-12-17 23:38:14,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940981e575bc060ce1619dc HTTP/1.1" 502 150 2025-12-17 23:38:14,423 - lava_callback - ERROR - Error processing callback for node 6940981e575bc060ce1619dc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940981e575bc060ce1619dc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940981e575bc060ce1619dc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940981e575bc060ce1619dc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:14,425 - lava_callback - INFO - Starting async job submit for node 6940983d575bc060ce161a24 2025-12-17 23:38:14,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:14,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983d575bc060ce161a24 HTTP/1.1" 502 150 2025-12-17 23:38:14,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983d575bc060ce161a24'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:14,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983d575bc060ce161a24 2025-12-17 23:38:14,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983d575bc060ce161a24 HTTP/1.1" 502 150 2025-12-17 23:38:14,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983d575bc060ce161a24'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:14,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409826575bc060ce1619ec 2025-12-17 23:38:14,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409826575bc060ce1619ec HTTP/1.1" 502 150 2025-12-17 23:38:14,630 - lava_callback - ERROR - Error processing callback for node 69409826575bc060ce1619ec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409826575bc060ce1619ec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409826575bc060ce1619ec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409826575bc060ce1619ec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:14,632 - lava_callback - INFO - Starting async job submit for node 6940983e575bc060ce161a27 2025-12-17 23:38:14,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:14,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983e575bc060ce161a27 HTTP/1.1" 502 150 2025-12-17 23:38:14,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983e575bc060ce161a27'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:14,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983e575bc060ce161a27 2025-12-17 23:38:14,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983e575bc060ce161a27 HTTP/1.1" 502 150 2025-12-17 23:38:14,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983e575bc060ce161a27'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:15,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983b575bc060ce161a21 2025-12-17 23:38:15,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983b575bc060ce161a21 HTTP/1.1" 502 150 2025-12-17 23:38:15,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983b575bc060ce161a21'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54366 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:38:16,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983d575bc060ce161a24 2025-12-17 23:38:16,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983d575bc060ce161a24 HTTP/1.1" 502 150 2025-12-17 23:38:16,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983d575bc060ce161a24'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54372 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:38:16,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983e575bc060ce161a27 2025-12-17 23:38:16,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983e575bc060ce161a27 HTTP/1.1" 502 150 2025-12-17 23:38:16,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983e575bc060ce161a27'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:17,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409804575bc060ce161997 2025-12-17 23:38:17,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409804575bc060ce161997 HTTP/1.1" 502 150 2025-12-17 23:38:17,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409804575bc060ce161997'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:17,255 - lava_callback - INFO - Received callback for node 69409bd9575bc060ce162437 INFO: 172.21.0.1:54380 - "POST /node/69409bd9575bc060ce162437 HTTP/1.0" 202 Accepted 2025-12-17 23:38:17,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409825575bc060ce1619e9 2025-12-17 23:38:17,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409825575bc060ce1619e9 HTTP/1.1" 502 150 2025-12-17 23:38:17,541 - lava_callback - ERROR - Error processing callback for node 69409825575bc060ce1619e9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409825575bc060ce1619e9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409825575bc060ce1619e9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409825575bc060ce1619e9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:17,543 - lava_callback - INFO - Starting async job submit for node 69409843575bc060ce161a30 2025-12-17 23:38:17,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:17,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409843575bc060ce161a30 HTTP/1.1" 502 150 2025-12-17 23:38:17,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409843575bc060ce161a30'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:17,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409843575bc060ce161a30 2025-12-17 23:38:17,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409843575bc060ce161a30 HTTP/1.1" 502 150 2025-12-17 23:38:17,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409843575bc060ce161a30'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:17,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409828575bc060ce1619f4 2025-12-17 23:38:17,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409828575bc060ce1619f4 HTTP/1.1" 502 150 2025-12-17 23:38:17,553 - lava_callback - ERROR - Error processing callback for node 69409828575bc060ce1619f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409828575bc060ce1619f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409828575bc060ce1619f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409828575bc060ce1619f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:17,554 - lava_callback - INFO - Starting async job submit for node 69409841575bc060ce161a2d 2025-12-17 23:38:17,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:17,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409841575bc060ce161a2d HTTP/1.1" 502 150 2025-12-17 23:38:17,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409841575bc060ce161a2d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:17,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409841575bc060ce161a2d 2025-12-17 23:38:17,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409841575bc060ce161a2d HTTP/1.1" 502 150 2025-12-17 23:38:17,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409841575bc060ce161a2d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:17,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5431e3c24e95681d6b6 2025-12-17 23:38:17,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5431e3c24e95681d6b6 HTTP/1.1" 502 150 2025-12-17 23:38:17,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5431e3c24e95681d6b6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:17,928 - lava_callback - INFO - Received callback for node 69409bda575bc060ce16243a INFO: 172.21.0.1:54382 - "POST /node/69409bda575bc060ce16243a HTTP/1.0" 202 Accepted 2025-12-17 23:38:18,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940982a575bc060ce1619f7 2025-12-17 23:38:18,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982a575bc060ce1619f7 HTTP/1.1" 502 150 2025-12-17 23:38:18,015 - lava_callback - ERROR - Error processing callback for node 6940982a575bc060ce1619f7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940982a575bc060ce1619f7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940982a575bc060ce1619f7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940982a575bc060ce1619f7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:18,016 - lava_callback - INFO - Starting async job submit for node 69409845575bc060ce161a36 2025-12-17 23:38:18,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:18,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940982c575bc060ce1619fe 2025-12-17 23:38:18,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982c575bc060ce1619fe HTTP/1.1" 502 150 2025-12-17 23:38:18,035 - lava_callback - ERROR - Error processing callback for node 6940982c575bc060ce1619fe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940982c575bc060ce1619fe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940982c575bc060ce1619fe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940982c575bc060ce1619fe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:18,037 - lava_callback - INFO - Starting async job submit for node 6940983f575bc060ce161a2a 2025-12-17 23:38:18,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:18,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409845575bc060ce161a36 HTTP/1.1" 502 150 2025-12-17 23:38:18,050 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409845575bc060ce161a36'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:18,050 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409845575bc060ce161a36 2025-12-17 23:38:18,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409845575bc060ce161a36 HTTP/1.1" 502 150 2025-12-17 23:38:18,051 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409845575bc060ce161a36'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:18,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983f575bc060ce161a2a HTTP/1.1" 502 150 2025-12-17 23:38:18,064 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983f575bc060ce161a2a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:18,064 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983f575bc060ce161a2a 2025-12-17 23:38:18,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983f575bc060ce161a2a HTTP/1.1" 502 150 2025-12-17 23:38:18,065 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983f575bc060ce161a2a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:18,625 - lava_callback - INFO - Received callback for node 69409bdc575bc060ce16243d INFO: 172.21.0.1:54384 - "POST /node/69409bdc575bc060ce16243d HTTP/1.0" 202 Accepted 2025-12-17 23:38:19,090 - lava_callback - INFO - Received callback for node 6940c5461e3c24e95681d6b9 INFO: 172.21.0.1:54386 - "POST /node/6940c5461e3c24e95681d6b9 HTTP/1.0" 202 Accepted 2025-12-17 23:38:19,285 - lava_callback - INFO - Received callback for node 69409bdd575bc060ce162440 INFO: 172.21.0.1:54398 - "POST /node/69409bdd575bc060ce162440 HTTP/1.0" 202 Accepted 2025-12-17 23:38:19,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409831575bc060ce161a04 2025-12-17 23:38:19,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409831575bc060ce161a04 HTTP/1.1" 502 150 2025-12-17 23:38:19,526 - lava_callback - ERROR - Error processing callback for node 69409831575bc060ce161a04: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409831575bc060ce161a04 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409831575bc060ce161a04 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409831575bc060ce161a04 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:19,528 - lava_callback - INFO - Starting async job submit for node 69408d07575bc060ce15f74f 2025-12-17 23:38:19,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409843575bc060ce161a30 2025-12-17 23:38:19,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409843575bc060ce161a30 HTTP/1.1" 502 150 2025-12-17 23:38:19,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409843575bc060ce161a30'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:19,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409841575bc060ce161a2d 2025-12-17 23:38:19,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409841575bc060ce161a2d HTTP/1.1" 502 150 2025-12-17 23:38:19,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409841575bc060ce161a2d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:19,942 - lava_callback - INFO - Received callback for node 69409bdf575bc060ce162447 INFO: 172.21.0.1:46220 - "POST /node/69409bdf575bc060ce162447 HTTP/1.0" 202 Accepted 2025-12-17 23:38:20,056 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409845575bc060ce161a36 2025-12-17 23:38:20,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409845575bc060ce161a36 HTTP/1.1" 502 150 2025-12-17 23:38:20,067 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409845575bc060ce161a36'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:20,067 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983f575bc060ce161a2a 2025-12-17 23:38:20,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983f575bc060ce161a2a HTTP/1.1" 502 150 2025-12-17 23:38:20,078 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983f575bc060ce161a2a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:20,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983d575bc060ce161a24 2025-12-17 23:38:20,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983d575bc060ce161a24 HTTP/1.1" 502 150 2025-12-17 23:38:20,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983d575bc060ce161a24'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:20,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983e575bc060ce161a27 2025-12-17 23:38:20,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983e575bc060ce161a27 HTTP/1.1" 502 150 2025-12-17 23:38:20,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983e575bc060ce161a27'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:21,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:21,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d07575bc060ce15f74f HTTP/1.1" 502 150 2025-12-17 23:38:21,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d07575bc060ce15f74f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:21,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d07575bc060ce15f74f 2025-12-17 23:38:21,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d07575bc060ce15f74f HTTP/1.1" 502 150 2025-12-17 23:38:21,074 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d07575bc060ce15f74f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:22,242 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409958575bc060ce161cb9 2025-12-17 23:38:23,074 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d07575bc060ce15f74f 2025-12-17 23:38:23,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d07575bc060ce15f74f HTTP/1.1" 502 150 2025-12-17 23:38:23,075 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d07575bc060ce15f74f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:23,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409843575bc060ce161a30 2025-12-17 23:38:23,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409843575bc060ce161a30 HTTP/1.1" 502 150 2025-12-17 23:38:23,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409843575bc060ce161a30'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:23,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409841575bc060ce161a2d 2025-12-17 23:38:23,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409841575bc060ce161a2d HTTP/1.1" 502 150 2025-12-17 23:38:23,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409841575bc060ce161a2d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:24,068 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409845575bc060ce161a36 2025-12-17 23:38:24,084 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983f575bc060ce161a2a 2025-12-17 23:38:24,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983f575bc060ce161a2a HTTP/1.1" 502 150 2025-12-17 23:38:24,085 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983f575bc060ce161a2a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:25,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409804575bc060ce161997 2025-12-17 23:38:25,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409804575bc060ce161997 HTTP/1.1" 502 150 2025-12-17 23:38:25,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409804575bc060ce161997'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46234 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:38:26,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940982e575bc060ce161a01 2025-12-17 23:38:26,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940982e575bc060ce161a01 HTTP/1.1" 502 150 2025-12-17 23:38:26,838 - lava_callback - ERROR - Error processing callback for node 6940982e575bc060ce161a01: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940982e575bc060ce161a01 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940982e575bc060ce161a01 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940982e575bc060ce161a01 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:26,840 - lava_callback - INFO - Starting async job submit for node 6942e2f1e43d2b76e9023918 2025-12-17 23:38:26,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:26,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f1e43d2b76e9023918 HTTP/1.1" 502 150 2025-12-17 23:38:26,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e2f1e43d2b76e9023918'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:26,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e2f1e43d2b76e9023918 2025-12-17 23:38:26,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f1e43d2b76e9023918 HTTP/1.1" 502 150 2025-12-17 23:38:26,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e2f1e43d2b76e9023918'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:27,075 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d07575bc060ce15f74f 2025-12-17 23:38:27,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d07575bc060ce15f74f HTTP/1.1" 502 150 2025-12-17 23:38:27,077 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d07575bc060ce15f74f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:27,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409833575bc060ce161a0d 2025-12-17 23:38:27,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409833575bc060ce161a0d HTTP/1.1" 502 150 2025-12-17 23:38:27,701 - lava_callback - ERROR - Error processing callback for node 69409833575bc060ce161a0d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409833575bc060ce161a0d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409833575bc060ce161a0d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409833575bc060ce161a0d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:27,704 - lava_callback - INFO - Starting async job submit for node 69409846575bc060ce161a3f 2025-12-17 23:38:27,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:27,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409846575bc060ce161a3f HTTP/1.1" 502 150 2025-12-17 23:38:27,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409846575bc060ce161a3f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:27,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409846575bc060ce161a3f 2025-12-17 23:38:27,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409846575bc060ce161a3f HTTP/1.1" 502 150 2025-12-17 23:38:27,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409846575bc060ce161a3f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:27,956 - lava_callback - INFO - Received callback for node 69409be1575bc060ce162450 INFO: 172.21.0.1:46246 - "POST /node/69409be1575bc060ce162450 HTTP/1.0" 202 Accepted 2025-12-17 23:38:28,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983d575bc060ce161a24 2025-12-17 23:38:28,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983d575bc060ce161a24 HTTP/1.1" 502 150 2025-12-17 23:38:28,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983d575bc060ce161a24'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:28,633 - lava_callback - INFO - Received callback for node 69409be4575bc060ce162457 INFO: 172.21.0.1:46258 - "POST /node/69409be4575bc060ce162457 HTTP/1.0" 202 Accepted 2025-12-17 23:38:28,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983e575bc060ce161a27 2025-12-17 23:38:28,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983e575bc060ce161a27 HTTP/1.1" 502 150 2025-12-17 23:38:28,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983e575bc060ce161a27'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:28,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e2f1e43d2b76e9023918 2025-12-17 23:38:28,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f1e43d2b76e9023918 HTTP/1.1" 502 150 2025-12-17 23:38:28,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e2f1e43d2b76e9023918'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:29,296 - lava_callback - INFO - Received callback for node 69409be7575bc060ce162461 INFO: 172.21.0.1:46272 - "POST /node/69409be7575bc060ce162461 HTTP/1.0" 202 Accepted 2025-12-17 23:38:29,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409846575bc060ce161a3f 2025-12-17 23:38:29,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409846575bc060ce161a3f HTTP/1.1" 502 150 2025-12-17 23:38:29,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409846575bc060ce161a3f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:29,971 - lava_callback - INFO - Received callback for node 69409be9575bc060ce162468 2025-12-17 23:38:29,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409836575bc060ce161a18 INFO: 172.21.0.1:59342 - "POST /node/69409be9575bc060ce162468 HTTP/1.0" 202 Accepted 2025-12-17 23:38:29,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409836575bc060ce161a18 HTTP/1.1" 502 150 2025-12-17 23:38:29,981 - lava_callback - ERROR - Error processing callback for node 69409836575bc060ce161a18: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409836575bc060ce161a18 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409836575bc060ce161a18 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409836575bc060ce161a18 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:29,983 - lava_callback - INFO - Starting async job submit for node 69409849575bc060ce161a46 2025-12-17 23:38:29,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:29,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409849575bc060ce161a46 HTTP/1.1" 502 150 2025-12-17 23:38:29,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409849575bc060ce161a46'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:29,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409849575bc060ce161a46 2025-12-17 23:38:29,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409849575bc060ce161a46 HTTP/1.1" 502 150 2025-12-17 23:38:29,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409849575bc060ce161a46'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:29,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409839575bc060ce161a1e 2025-12-17 23:38:29,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409839575bc060ce161a1e HTTP/1.1" 502 150 2025-12-17 23:38:29,997 - lava_callback - ERROR - Error processing callback for node 69409839575bc060ce161a1e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409839575bc060ce161a1e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409839575bc060ce161a1e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409839575bc060ce161a1e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:30,000 - lava_callback - INFO - Starting async job submit for node 6940984b575bc060ce161a49 2025-12-17 23:38:30,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:30,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984b575bc060ce161a49 HTTP/1.1" 502 150 2025-12-17 23:38:30,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940984b575bc060ce161a49'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:30,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940984b575bc060ce161a49 2025-12-17 23:38:30,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984b575bc060ce161a49 HTTP/1.1" 502 150 2025-12-17 23:38:30,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940984b575bc060ce161a49'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:30,636 - lava_callback - INFO - Received callback for node 69409beb575bc060ce16246b INFO: 172.21.0.1:59350 - "POST /node/69409beb575bc060ce16246b HTTP/1.0" 202 Accepted 2025-12-17 23:38:31,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983b575bc060ce161a21 2025-12-17 23:38:31,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983b575bc060ce161a21 HTTP/1.1" 502 150 2025-12-17 23:38:31,024 - lava_callback - ERROR - Error processing callback for node 6940983b575bc060ce161a21: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940983b575bc060ce161a21 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940983b575bc060ce161a21 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940983b575bc060ce161a21 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:31,026 - lava_callback - INFO - Starting async job submit for node 6940984d575bc060ce161a54 2025-12-17 23:38:31,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:31,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984d575bc060ce161a54 HTTP/1.1" 502 150 2025-12-17 23:38:31,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940984d575bc060ce161a54'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:31,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940984d575bc060ce161a54 2025-12-17 23:38:31,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984d575bc060ce161a54 HTTP/1.1" 502 150 2025-12-17 23:38:31,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940984d575bc060ce161a54'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:31,314 - lava_callback - INFO - Received callback for node 69409bed575bc060ce16246e INFO: 172.21.0.1:59354 - "POST /node/69409bed575bc060ce16246e HTTP/1.0" 202 Accepted 2025-12-17 23:38:31,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409843575bc060ce161a30 2025-12-17 23:38:31,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409843575bc060ce161a30 HTTP/1.1" 502 150 2025-12-17 23:38:31,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409843575bc060ce161a30'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:31,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409841575bc060ce161a2d 2025-12-17 23:38:31,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409841575bc060ce161a2d HTTP/1.1" 502 150 2025-12-17 23:38:31,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409841575bc060ce161a2d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:31,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409849575bc060ce161a46 2025-12-17 23:38:31,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409849575bc060ce161a46 HTTP/1.1" 502 150 2025-12-17 23:38:31,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409849575bc060ce161a46'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:31,993 - lava_callback - INFO - Received callback for node 69409bef575bc060ce162471 INFO: 172.21.0.1:59370 - "POST /node/69409bef575bc060ce162471 HTTP/1.0" 202 Accepted 2025-12-17 23:38:32,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940984b575bc060ce161a49 2025-12-17 23:38:32,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984b575bc060ce161a49 HTTP/1.1" 502 150 2025-12-17 23:38:32,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940984b575bc060ce161a49'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:32,085 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983f575bc060ce161a2a 2025-12-17 23:38:32,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983f575bc060ce161a2a HTTP/1.1" 502 150 2025-12-17 23:38:32,086 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940983f575bc060ce161a2a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:32,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e2f1e43d2b76e9023918 2025-12-17 23:38:32,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f1e43d2b76e9023918 HTTP/1.1" 502 150 2025-12-17 23:38:32,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e2f1e43d2b76e9023918'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:33,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940984d575bc060ce161a54 2025-12-17 23:38:33,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409846575bc060ce161a3f 2025-12-17 23:38:33,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409846575bc060ce161a3f HTTP/1.1" 502 150 2025-12-17 23:38:33,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409846575bc060ce161a3f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:33,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5431e3c24e95681d6b6 2025-12-17 23:38:33,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5431e3c24e95681d6b6 HTTP/1.1" 502 150 2025-12-17 23:38:33,776 - lava_callback - ERROR - Error processing callback for node 6940c5431e3c24e95681d6b6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5431e3c24e95681d6b6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5431e3c24e95681d6b6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5431e3c24e95681d6b6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:33,777 - lava_callback - INFO - Starting async job submit for node 6940984f575bc060ce161a57 2025-12-17 23:38:33,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:33,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984f575bc060ce161a57 HTTP/1.1" 502 150 2025-12-17 23:38:33,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940984f575bc060ce161a57'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:33,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940984f575bc060ce161a57 2025-12-17 23:38:33,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984f575bc060ce161a57 HTTP/1.1" 502 150 2025-12-17 23:38:33,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940984f575bc060ce161a57'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:35,077 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d07575bc060ce15f74f 2025-12-17 23:38:35,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d07575bc060ce15f74f HTTP/1.1" 502 150 2025-12-17 23:38:35,078 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69408d07575bc060ce15f74f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:35,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940984f575bc060ce161a57 2025-12-17 23:38:35,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984f575bc060ce161a57 HTTP/1.1" 502 150 2025-12-17 23:38:35,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940984f575bc060ce161a57'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:35,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409849575bc060ce161a46 2025-12-17 23:38:35,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409849575bc060ce161a46 HTTP/1.1" 502 150 2025-12-17 23:38:35,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409849575bc060ce161a46'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:36,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940984b575bc060ce161a49 2025-12-17 23:38:36,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984b575bc060ce161a49 HTTP/1.1" 502 150 2025-12-17 23:38:36,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940984b575bc060ce161a49'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59380 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:38:37,278 - lava_callback - INFO - Received callback for node 69409bf1575bc060ce162474 INFO: 172.21.0.1:59386 - "POST /node/69409bf1575bc060ce162474 HTTP/1.0" 202 Accepted 2025-12-17 23:38:37,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409958575bc060ce161cb9 HTTP/1.1" 502 150 2025-12-17 23:38:37,616 - lava_callback - ERROR - Error processing callback for node 69409958575bc060ce161cb9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409958575bc060ce161cb9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409958575bc060ce161cb9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409958575bc060ce161cb9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:37,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097df575bc060ce16193a HTTP/1.1" 502 150 2025-12-17 23:38:37,619 - lava_callback - INFO - Starting async job submit for node 69409851575bc060ce161a60 2025-12-17 23:38:37,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097df575bc060ce16193a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:37,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:37,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409851575bc060ce161a60 HTTP/1.1" 502 150 2025-12-17 23:38:37,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409851575bc060ce161a60'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:37,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409851575bc060ce161a60 2025-12-17 23:38:37,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409851575bc060ce161a60 HTTP/1.1" 502 150 2025-12-17 23:38:37,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409851575bc060ce161a60'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:37,955 - lava_callback - INFO - Received callback for node 69409bf5575bc060ce162484 INFO: 172.21.0.1:59390 - "POST /node/69409bf5575bc060ce162484 HTTP/1.0" 202 Accepted 2025-12-17 23:38:38,636 - lava_callback - INFO - Received callback for node 69409bf7575bc060ce16248b INFO: 172.21.0.1:59398 - "POST /node/69409bf7575bc060ce16248b HTTP/1.0" 202 Accepted 2025-12-17 23:38:39,310 - lava_callback - INFO - Received callback for node 69409bf9575bc060ce16248e INFO: 172.21.0.1:59406 - "POST /node/69409bf9575bc060ce16248e HTTP/1.0" 202 Accepted 2025-12-17 23:38:39,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409845575bc060ce161a36 HTTP/1.1" 502 150 2025-12-17 23:38:39,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409845575bc060ce161a36'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:39,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409851575bc060ce161a60 2025-12-17 23:38:39,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409851575bc060ce161a60 HTTP/1.1" 502 150 2025-12-17 23:38:39,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409851575bc060ce161a60'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:39,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940984f575bc060ce161a57 2025-12-17 23:38:39,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984f575bc060ce161a57 HTTP/1.1" 502 150 2025-12-17 23:38:39,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940984f575bc060ce161a57'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:39,972 - lava_callback - INFO - Received callback for node 69409bfb575bc060ce162491 INFO: 172.21.0.1:55506 - "POST /node/69409bfb575bc060ce162491 HTTP/1.0" 202 Accepted 2025-12-17 23:38:40,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e2f1e43d2b76e9023918 2025-12-17 23:38:40,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f1e43d2b76e9023918 HTTP/1.1" 502 150 2025-12-17 23:38:40,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e2f1e43d2b76e9023918'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:41,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409804575bc060ce161997 2025-12-17 23:38:41,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409804575bc060ce161997 HTTP/1.1" 502 150 2025-12-17 23:38:41,020 - lava_callback - ERROR - Error processing callback for node 69409804575bc060ce161997: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409804575bc060ce161997 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409804575bc060ce161997 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409804575bc060ce161997 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:41,022 - lava_callback - INFO - Starting async job submit for node 69409853575bc060ce161a63 2025-12-17 23:38:41,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:41,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409853575bc060ce161a63 HTTP/1.1" 502 150 2025-12-17 23:38:41,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409853575bc060ce161a63'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:41,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409853575bc060ce161a63 2025-12-17 23:38:41,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409853575bc060ce161a63 HTTP/1.1" 502 150 2025-12-17 23:38:41,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409853575bc060ce161a63'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:41,620 - lava_callback - INFO - Received callback for node 6942d8e1e43d2b76e9021f27 INFO: 172.21.0.1:55522 - "POST /node/6942d8e1e43d2b76e9021f27 HTTP/1.0" 202 Accepted 2025-12-17 23:38:41,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409846575bc060ce161a3f 2025-12-17 23:38:41,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409846575bc060ce161a3f HTTP/1.1" 502 150 2025-12-17 23:38:41,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409846575bc060ce161a3f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:43,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409853575bc060ce161a63 2025-12-17 23:38:43,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409853575bc060ce161a63 HTTP/1.1" 502 150 2025-12-17 23:38:43,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409853575bc060ce161a63'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:43,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409851575bc060ce161a60 2025-12-17 23:38:43,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409851575bc060ce161a60 HTTP/1.1" 502 150 2025-12-17 23:38:43,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409851575bc060ce161a60'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:43,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409849575bc060ce161a46 2025-12-17 23:38:43,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409849575bc060ce161a46 HTTP/1.1" 502 150 2025-12-17 23:38:43,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409849575bc060ce161a46'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:44,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940984b575bc060ce161a49 2025-12-17 23:38:44,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984b575bc060ce161a49 HTTP/1.1" 502 150 2025-12-17 23:38:44,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940984b575bc060ce161a49'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:44,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983d575bc060ce161a24 2025-12-17 23:38:44,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983d575bc060ce161a24 HTTP/1.1" 502 150 2025-12-17 23:38:44,468 - lava_callback - ERROR - Error processing callback for node 6940983d575bc060ce161a24: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940983d575bc060ce161a24 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940983d575bc060ce161a24 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940983d575bc060ce161a24 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:44,470 - lava_callback - INFO - Starting async job submit for node 69409855575bc060ce161a66 2025-12-17 23:38:44,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:44,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409855575bc060ce161a66 HTTP/1.1" 502 150 2025-12-17 23:38:44,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409855575bc060ce161a66'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:44,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409855575bc060ce161a66 2025-12-17 23:38:44,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409855575bc060ce161a66 HTTP/1.1" 502 150 2025-12-17 23:38:44,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409855575bc060ce161a66'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:44,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983e575bc060ce161a27 2025-12-17 23:38:44,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983e575bc060ce161a27 HTTP/1.1" 502 150 2025-12-17 23:38:44,672 - lava_callback - ERROR - Error processing callback for node 6940983e575bc060ce161a27: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940983e575bc060ce161a27 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940983e575bc060ce161a27 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940983e575bc060ce161a27 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:44,674 - lava_callback - INFO - Starting async job submit for node 69409857575bc060ce161a69 2025-12-17 23:38:44,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:44,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409857575bc060ce161a69 HTTP/1.1" 502 150 2025-12-17 23:38:44,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409857575bc060ce161a69'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:44,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409857575bc060ce161a69 2025-12-17 23:38:44,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409857575bc060ce161a69 HTTP/1.1" 502 150 2025-12-17 23:38:44,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409857575bc060ce161a69'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:45,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097df575bc060ce16193a 2025-12-17 23:38:46,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409855575bc060ce161a66 2025-12-17 23:38:46,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409855575bc060ce161a66 HTTP/1.1" 502 150 2025-12-17 23:38:46,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409855575bc060ce161a66'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55524 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:38:46,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409857575bc060ce161a69 2025-12-17 23:38:46,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409857575bc060ce161a69 HTTP/1.1" 502 150 2025-12-17 23:38:46,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409857575bc060ce161a69'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:47,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409853575bc060ce161a63 2025-12-17 23:38:47,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409853575bc060ce161a63 HTTP/1.1" 502 150 2025-12-17 23:38:47,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409853575bc060ce161a63'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:47,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409845575bc060ce161a36 2025-12-17 23:38:47,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409845575bc060ce161a36 HTTP/1.1" 502 150 2025-12-17 23:38:47,410 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409845575bc060ce161a36'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:47,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409843575bc060ce161a30 2025-12-17 23:38:47,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409843575bc060ce161a30 HTTP/1.1" 502 150 2025-12-17 23:38:47,572 - lava_callback - ERROR - Error processing callback for node 69409843575bc060ce161a30: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409843575bc060ce161a30 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409843575bc060ce161a30 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409843575bc060ce161a30 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:47,574 - lava_callback - INFO - Starting async job submit for node 69409859575bc060ce161a6c 2025-12-17 23:38:47,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:47,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409859575bc060ce161a6c HTTP/1.1" 502 150 2025-12-17 23:38:47,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409859575bc060ce161a6c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:47,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409859575bc060ce161a6c 2025-12-17 23:38:47,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409859575bc060ce161a6c HTTP/1.1" 502 150 2025-12-17 23:38:47,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409859575bc060ce161a6c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:47,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409841575bc060ce161a2d 2025-12-17 23:38:47,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409841575bc060ce161a2d HTTP/1.1" 502 150 2025-12-17 23:38:47,592 - lava_callback - ERROR - Error processing callback for node 69409841575bc060ce161a2d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409841575bc060ce161a2d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409841575bc060ce161a2d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409841575bc060ce161a2d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:47,594 - lava_callback - INFO - Starting async job submit for node 6940985b575bc060ce161a75 2025-12-17 23:38:47,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:47,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985b575bc060ce161a75 HTTP/1.1" 502 150 2025-12-17 23:38:47,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940985b575bc060ce161a75'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:47,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940985b575bc060ce161a75 2025-12-17 23:38:47,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985b575bc060ce161a75 HTTP/1.1" 502 150 2025-12-17 23:38:47,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940985b575bc060ce161a75'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:47,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940984f575bc060ce161a57 2025-12-17 23:38:47,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984f575bc060ce161a57 HTTP/1.1" 502 150 2025-12-17 23:38:47,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940984f575bc060ce161a57'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:47,993 - lava_callback - INFO - Received callback for node 69409bf3575bc060ce16247b INFO: 172.21.0.1:55536 - "POST /node/69409bf3575bc060ce16247b HTTP/1.0" 202 Accepted 2025-12-17 23:38:48,086 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940983f575bc060ce161a2a 2025-12-17 23:38:48,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984d575bc060ce161a54 HTTP/1.1" 502 150 2025-12-17 23:38:48,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940984d575bc060ce161a54'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:48,640 - lava_callback - INFO - Received callback for node 69409bfd575bc060ce162498 INFO: 172.21.0.1:55550 - "POST /node/69409bfd575bc060ce162498 HTTP/1.0" 202 Accepted 2025-12-17 23:38:48,756 - lava_callback - INFO - Received callback for node 69409c04575bc060ce1624a1 INFO: 172.21.0.1:55562 - "POST /node/69409c04575bc060ce1624a1 HTTP/1.0" 202 Accepted 2025-12-17 23:38:49,279 - lava_callback - INFO - Received callback for node 69409c02575bc060ce16249e INFO: 172.21.0.1:55564 - "POST /node/69409c02575bc060ce16249e HTTP/1.0" 202 Accepted 2025-12-17 23:38:49,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409859575bc060ce161a6c 2025-12-17 23:38:49,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409859575bc060ce161a6c HTTP/1.1" 502 150 2025-12-17 23:38:49,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409859575bc060ce161a6c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:49,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940985b575bc060ce161a75 2025-12-17 23:38:49,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985b575bc060ce161a75 HTTP/1.1" 502 150 2025-12-17 23:38:49,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940985b575bc060ce161a75'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:50,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409855575bc060ce161a66 2025-12-17 23:38:50,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409855575bc060ce161a66 HTTP/1.1" 502 150 2025-12-17 23:38:50,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409855575bc060ce161a66'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:50,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409857575bc060ce161a69 2025-12-17 23:38:50,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409857575bc060ce161a69 HTTP/1.1" 502 150 2025-12-17 23:38:50,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409857575bc060ce161a69'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:51,078 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69408d07575bc060ce15f74f 2025-12-17 23:38:51,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69408d07575bc060ce15f74f HTTP/1.1" 502 150 2025-12-17 23:38:51,080 - lava_callback - ERROR - Error processing callback for node 69408d07575bc060ce15f74f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d07575bc060ce15f74f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d07575bc060ce15f74f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69408d07575bc060ce15f74f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:51,082 - lava_callback - INFO - Starting async job submit for node 6940985d575bc060ce161a7c 2025-12-17 23:38:51,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:51,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985d575bc060ce161a7c HTTP/1.1" 502 150 2025-12-17 23:38:51,089 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940985d575bc060ce161a7c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:51,089 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940985d575bc060ce161a7c 2025-12-17 23:38:51,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985d575bc060ce161a7c HTTP/1.1" 502 150 2025-12-17 23:38:51,090 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940985d575bc060ce161a7c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:51,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409851575bc060ce161a60 2025-12-17 23:38:51,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409851575bc060ce161a60 HTTP/1.1" 502 150 2025-12-17 23:38:51,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409851575bc060ce161a60'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:52,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940984d575bc060ce161a54 2025-12-17 23:38:52,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984d575bc060ce161a54 HTTP/1.1" 502 150 2025-12-17 23:38:52,374 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940984d575bc060ce161a54'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:53,090 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940985d575bc060ce161a7c 2025-12-17 23:38:53,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985d575bc060ce161a7c HTTP/1.1" 502 150 2025-12-17 23:38:53,091 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940985d575bc060ce161a7c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:53,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409859575bc060ce161a6c 2025-12-17 23:38:53,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409859575bc060ce161a6c HTTP/1.1" 502 150 2025-12-17 23:38:53,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409859575bc060ce161a6c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:53,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940985b575bc060ce161a75 2025-12-17 23:38:53,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985b575bc060ce161a75 HTTP/1.1" 502 150 2025-12-17 23:38:53,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940985b575bc060ce161a75'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:55,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409853575bc060ce161a63 2025-12-17 23:38:55,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409853575bc060ce161a63 HTTP/1.1" 502 150 2025-12-17 23:38:55,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409853575bc060ce161a63'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49014 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:38:56,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e2f1e43d2b76e9023918 2025-12-17 23:38:56,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e2f1e43d2b76e9023918 HTTP/1.1" 502 150 2025-12-17 23:38:56,856 - lava_callback - ERROR - Error processing callback for node 6942e2f1e43d2b76e9023918: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e2f1e43d2b76e9023918 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e2f1e43d2b76e9023918 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e2f1e43d2b76e9023918 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:56,858 - lava_callback - INFO - Starting async job submit for node 6940985f575bc060ce161a7f 2025-12-17 23:38:56,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:56,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985f575bc060ce161a7f HTTP/1.1" 502 150 2025-12-17 23:38:56,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940985f575bc060ce161a7f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:56,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940985f575bc060ce161a7f 2025-12-17 23:38:56,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985f575bc060ce161a7f HTTP/1.1" 502 150 2025-12-17 23:38:56,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940985f575bc060ce161a7f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:57,092 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940985d575bc060ce161a7c 2025-12-17 23:38:57,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985d575bc060ce161a7c HTTP/1.1" 502 150 2025-12-17 23:38:57,093 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940985d575bc060ce161a7c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:57,317 - lava_callback - INFO - Received callback for node 69409c00575bc060ce16249b INFO: 172.21.0.1:49026 - "POST /node/69409c00575bc060ce16249b HTTP/1.0" 202 Accepted 2025-12-17 23:38:57,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409846575bc060ce161a3f 2025-12-17 23:38:57,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409846575bc060ce161a3f HTTP/1.1" 502 150 2025-12-17 23:38:57,729 - lava_callback - ERROR - Error processing callback for node 69409846575bc060ce161a3f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409846575bc060ce161a3f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409846575bc060ce161a3f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409846575bc060ce161a3f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:38:57,731 - lava_callback - INFO - Starting async job submit for node 69409861575bc060ce161a82 2025-12-17 23:38:57,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:38:57,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409861575bc060ce161a82 HTTP/1.1" 502 150 2025-12-17 23:38:57,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409861575bc060ce161a82'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:57,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409861575bc060ce161a82 2025-12-17 23:38:57,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409861575bc060ce161a82 HTTP/1.1" 502 150 2025-12-17 23:38:57,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409861575bc060ce161a82'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:57,974 - lava_callback - INFO - Received callback for node 69409c05575bc060ce1624a4 INFO: 172.21.0.1:49034 - "POST /node/69409c05575bc060ce1624a4 HTTP/1.0" 202 Accepted 2025-12-17 23:38:58,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409855575bc060ce161a66 2025-12-17 23:38:58,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409855575bc060ce161a66 HTTP/1.1" 502 150 2025-12-17 23:38:58,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409855575bc060ce161a66'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:58,627 - lava_callback - INFO - Received callback for node 69409c0a575bc060ce1624ba INFO: 172.21.0.1:49046 - "POST /node/69409c0a575bc060ce1624ba HTTP/1.0" 202 Accepted 2025-12-17 23:38:58,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409857575bc060ce161a69 2025-12-17 23:38:58,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409857575bc060ce161a69 HTTP/1.1" 502 150 2025-12-17 23:38:58,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409857575bc060ce161a69'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:58,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940985f575bc060ce161a7f 2025-12-17 23:38:58,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985f575bc060ce161a7f HTTP/1.1" 502 150 2025-12-17 23:38:58,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940985f575bc060ce161a7f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:38:59,308 - lava_callback - INFO - Received callback for node 69409c0d575bc060ce1624c2 INFO: 172.21.0.1:49052 - "POST /node/69409c0d575bc060ce1624c2 HTTP/1.0" 202 Accepted 2025-12-17 23:38:59,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409861575bc060ce161a82 2025-12-17 23:38:59,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409849575bc060ce161a46 2025-12-17 23:38:59,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409849575bc060ce161a46 HTTP/1.1" 502 150 2025-12-17 23:38:59,997 - lava_callback - ERROR - Error processing callback for node 69409849575bc060ce161a46: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409849575bc060ce161a46 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409849575bc060ce161a46 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409849575bc060ce161a46 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:00,000 - lava_callback - INFO - Starting async job submit for node 69409866575bc060ce161a88 2025-12-17 23:39:00,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:00,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940984b575bc060ce161a49 2025-12-17 23:39:00,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984b575bc060ce161a49 HTTP/1.1" 502 150 2025-12-17 23:39:00,014 - lava_callback - ERROR - Error processing callback for node 6940984b575bc060ce161a49: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940984b575bc060ce161a49 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940984b575bc060ce161a49 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940984b575bc060ce161a49 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:00,017 - lava_callback - INFO - Starting async job submit for node 6940986a575bc060ce161a8e 2025-12-17 23:39:00,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:00,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409866575bc060ce161a88 HTTP/1.1" 502 150 2025-12-17 23:39:00,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409866575bc060ce161a88'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:00,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409866575bc060ce161a88 2025-12-17 23:39:00,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409866575bc060ce161a88 HTTP/1.1" 502 150 2025-12-17 23:39:00,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409866575bc060ce161a88'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:00,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986a575bc060ce161a8e HTTP/1.1" 502 150 2025-12-17 23:39:00,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940986a575bc060ce161a8e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:00,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940986a575bc060ce161a8e 2025-12-17 23:39:00,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986a575bc060ce161a8e HTTP/1.1" 502 150 2025-12-17 23:39:00,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940986a575bc060ce161a8e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:00,374 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940984d575bc060ce161a54 2025-12-17 23:39:00,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984d575bc060ce161a54 HTTP/1.1" 502 150 2025-12-17 23:39:00,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940984d575bc060ce161a54'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:01,476 - lava_callback - INFO - Received callback for node 6942ddc6e43d2b76e9022d35 INFO: 172.21.0.1:35886 - "POST /node/6942ddc6e43d2b76e9022d35 HTTP/1.0" 202 Accepted 2025-12-17 23:39:01,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409859575bc060ce161a6c 2025-12-17 23:39:01,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409859575bc060ce161a6c HTTP/1.1" 502 150 2025-12-17 23:39:01,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409859575bc060ce161a6c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:01,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940985b575bc060ce161a75 2025-12-17 23:39:01,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985b575bc060ce161a75 HTTP/1.1" 502 150 2025-12-17 23:39:01,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940985b575bc060ce161a75'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:02,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409866575bc060ce161a88 2025-12-17 23:39:02,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409866575bc060ce161a88 HTTP/1.1" 502 150 2025-12-17 23:39:02,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409866575bc060ce161a88'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:02,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940986a575bc060ce161a8e 2025-12-17 23:39:02,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986a575bc060ce161a8e HTTP/1.1" 502 150 2025-12-17 23:39:02,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940986a575bc060ce161a8e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:02,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940985f575bc060ce161a7f 2025-12-17 23:39:02,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985f575bc060ce161a7f HTTP/1.1" 502 150 2025-12-17 23:39:02,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940985f575bc060ce161a7f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:03,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409845575bc060ce161a36 INFO: 172.21.0.1:35892 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:39:03,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940983f575bc060ce161a2a HTTP/1.1" 502 150 2025-12-17 23:39:03,473 - lava_callback - ERROR - Error processing callback for node 6940983f575bc060ce161a2a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940983f575bc060ce161a2a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940983f575bc060ce161a2a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940983f575bc060ce161a2a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:03,475 - lava_callback - INFO - Starting async job submit for node 6940986c575bc060ce161a91 2025-12-17 23:39:03,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:03,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986c575bc060ce161a91 HTTP/1.1" 502 150 2025-12-17 23:39:03,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940986c575bc060ce161a91'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:03,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940986c575bc060ce161a91 2025-12-17 23:39:03,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986c575bc060ce161a91 HTTP/1.1" 502 150 2025-12-17 23:39:03,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940986c575bc060ce161a91'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:03,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940984f575bc060ce161a57 2025-12-17 23:39:03,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984f575bc060ce161a57 HTTP/1.1" 502 150 2025-12-17 23:39:03,790 - lava_callback - ERROR - Error processing callback for node 6940984f575bc060ce161a57: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940984f575bc060ce161a57 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940984f575bc060ce161a57 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940984f575bc060ce161a57 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:03,792 - lava_callback - INFO - Starting async job submit for node 6940986e575bc060ce161a96 2025-12-17 23:39:03,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:03,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986e575bc060ce161a96 HTTP/1.1" 502 150 2025-12-17 23:39:03,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940986e575bc060ce161a96'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:03,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940986e575bc060ce161a96 2025-12-17 23:39:03,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986e575bc060ce161a96 HTTP/1.1" 502 150 2025-12-17 23:39:03,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940986e575bc060ce161a96'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:05,093 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940985d575bc060ce161a7c 2025-12-17 23:39:05,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985d575bc060ce161a7c HTTP/1.1" 502 150 2025-12-17 23:39:05,094 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940985d575bc060ce161a7c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:05,460 - lava_callback - INFO - Received callback for node 69409c49575bc060ce162574 INFO: 172.21.0.1:35894 - "POST /node/69409c49575bc060ce162574 HTTP/1.0" 202 Accepted 2025-12-17 23:39:05,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940986c575bc060ce161a91 2025-12-17 23:39:05,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986c575bc060ce161a91 HTTP/1.1" 502 150 2025-12-17 23:39:05,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940986c575bc060ce161a91'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:05,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940986e575bc060ce161a96 2025-12-17 23:39:05,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986e575bc060ce161a96 HTTP/1.1" 502 150 2025-12-17 23:39:05,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940986e575bc060ce161a96'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:06,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409866575bc060ce161a88 2025-12-17 23:39:06,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409866575bc060ce161a88 HTTP/1.1" 502 150 2025-12-17 23:39:06,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409866575bc060ce161a88'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:06,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940986a575bc060ce161a8e 2025-12-17 23:39:06,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986a575bc060ce161a8e HTTP/1.1" 502 150 2025-12-17 23:39:06,037 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940986a575bc060ce161a8e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35908 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:39:07,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409851575bc060ce161a60 2025-12-17 23:39:07,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409851575bc060ce161a60 HTTP/1.1" 502 150 2025-12-17 23:39:07,636 - lava_callback - ERROR - Error processing callback for node 69409851575bc060ce161a60: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409851575bc060ce161a60 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409851575bc060ce161a60 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409851575bc060ce161a60 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:07,638 - lava_callback - INFO - Starting async job submit for node 69409863575bc060ce161a85 2025-12-17 23:39:07,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:07,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409863575bc060ce161a85 HTTP/1.1" 502 150 2025-12-17 23:39:07,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409863575bc060ce161a85'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:07,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409863575bc060ce161a85 2025-12-17 23:39:07,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409863575bc060ce161a85 HTTP/1.1" 502 150 2025-12-17 23:39:07,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409863575bc060ce161a85'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:07,998 - lava_callback - INFO - Received callback for node 69409c08575bc060ce1624b0 INFO: 172.21.0.1:35916 - "POST /node/69409c08575bc060ce1624b0 HTTP/1.0" 202 Accepted 2025-12-17 23:39:08,659 - lava_callback - INFO - Received callback for node 69409c0c575bc060ce1624bd INFO: 172.21.0.1:35930 - "POST /node/69409c0c575bc060ce1624bd HTTP/1.0" 202 Accepted 2025-12-17 23:39:09,313 - lava_callback - INFO - Received callback for node 69409c0f575bc060ce1624c5 INFO: 172.21.0.1:35946 - "POST /node/69409c0f575bc060ce1624c5 HTTP/1.0" 202 Accepted 2025-12-17 23:39:09,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940986c575bc060ce161a91 2025-12-17 23:39:09,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986c575bc060ce161a91 HTTP/1.1" 502 150 2025-12-17 23:39:09,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940986c575bc060ce161a91'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:09,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409863575bc060ce161a85 2025-12-17 23:39:09,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409863575bc060ce161a85 HTTP/1.1" 502 150 2025-12-17 23:39:09,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409863575bc060ce161a85'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:09,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940986e575bc060ce161a96 2025-12-17 23:39:09,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986e575bc060ce161a96 HTTP/1.1" 502 150 2025-12-17 23:39:09,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940986e575bc060ce161a96'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:09,981 - lava_callback - INFO - Received callback for node 69409c1a575bc060ce1624da INFO: 172.21.0.1:34404 - "POST /node/69409c1a575bc060ce1624da HTTP/1.0" 202 Accepted 2025-12-17 23:39:10,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940985f575bc060ce161a7f 2025-12-17 23:39:10,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985f575bc060ce161a7f HTTP/1.1" 502 150 2025-12-17 23:39:10,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940985f575bc060ce161a7f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:11,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409853575bc060ce161a63 2025-12-17 23:39:11,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409853575bc060ce161a63 HTTP/1.1" 502 150 2025-12-17 23:39:11,037 - lava_callback - ERROR - Error processing callback for node 69409853575bc060ce161a63: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409853575bc060ce161a63 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409853575bc060ce161a63 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409853575bc060ce161a63 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:11,038 - lava_callback - INFO - Starting async job submit for node 69409876575bc060ce161aaa 2025-12-17 23:39:11,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:11,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409876575bc060ce161aaa HTTP/1.1" 502 150 2025-12-17 23:39:11,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409876575bc060ce161aaa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:11,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409876575bc060ce161aaa 2025-12-17 23:39:11,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409876575bc060ce161aaa HTTP/1.1" 502 150 2025-12-17 23:39:11,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409876575bc060ce161aaa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:12,428 - lava_callback - INFO - Received callback for node 69409c18575bc060ce1624d7 INFO: 172.21.0.1:34406 - "POST /node/69409c18575bc060ce1624d7 HTTP/1.0" 202 Accepted 2025-12-17 23:39:13,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409876575bc060ce161aaa 2025-12-17 23:39:13,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409876575bc060ce161aaa HTTP/1.1" 502 150 2025-12-17 23:39:13,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409876575bc060ce161aaa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:13,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409863575bc060ce161a85 2025-12-17 23:39:13,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409863575bc060ce161a85 HTTP/1.1" 502 150 2025-12-17 23:39:13,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409863575bc060ce161a85'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:14,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409866575bc060ce161a88 2025-12-17 23:39:14,037 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940986a575bc060ce161a8e 2025-12-17 23:39:14,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986a575bc060ce161a8e HTTP/1.1" 502 150 2025-12-17 23:39:14,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940986a575bc060ce161a8e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:14,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409855575bc060ce161a66 2025-12-17 23:39:14,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409855575bc060ce161a66 HTTP/1.1" 502 150 2025-12-17 23:39:14,485 - lava_callback - ERROR - Error processing callback for node 69409855575bc060ce161a66: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409855575bc060ce161a66 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409855575bc060ce161a66 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409855575bc060ce161a66 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:14,487 - lava_callback - INFO - Starting async job submit for node 69409545575bc060ce1612a2 2025-12-17 23:39:14,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:14,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409545575bc060ce1612a2 HTTP/1.1" 502 150 2025-12-17 23:39:14,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409545575bc060ce1612a2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:14,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409545575bc060ce1612a2 2025-12-17 23:39:14,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409545575bc060ce1612a2 HTTP/1.1" 502 150 2025-12-17 23:39:14,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409545575bc060ce1612a2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:14,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409857575bc060ce161a69 2025-12-17 23:39:14,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409857575bc060ce161a69 HTTP/1.1" 502 150 2025-12-17 23:39:14,710 - lava_callback - ERROR - Error processing callback for node 69409857575bc060ce161a69: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409857575bc060ce161a69 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409857575bc060ce161a69 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409857575bc060ce161a69 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:14,712 - lava_callback - INFO - Starting async job submit for node 69409871575bc060ce161aa4 2025-12-17 23:39:14,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:14,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409871575bc060ce161aa4 HTTP/1.1" 502 150 2025-12-17 23:39:14,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409871575bc060ce161aa4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:14,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409871575bc060ce161aa4 2025-12-17 23:39:14,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409871575bc060ce161aa4 HTTP/1.1" 502 150 2025-12-17 23:39:14,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409871575bc060ce161aa4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:14,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409861575bc060ce161a82 HTTP/1.1" 502 150 2025-12-17 23:39:14,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409861575bc060ce161a82'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34408 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:39:16,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097df575bc060ce16193a HTTP/1.1" 502 150 2025-12-17 23:39:16,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694097df575bc060ce16193a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:16,376 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940984d575bc060ce161a54 2025-12-17 23:39:16,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940984d575bc060ce161a54 HTTP/1.1" 502 150 2025-12-17 23:39:16,377 - lava_callback - ERROR - Error processing callback for node 6940984d575bc060ce161a54: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940984d575bc060ce161a54 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940984d575bc060ce161a54 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940984d575bc060ce161a54 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:16,379 - lava_callback - INFO - Starting async job submit for node 69409873575bc060ce161aa7 2025-12-17 23:39:16,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:16,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409873575bc060ce161aa7 HTTP/1.1" 502 150 2025-12-17 23:39:16,385 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409873575bc060ce161aa7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:16,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409873575bc060ce161aa7 2025-12-17 23:39:16,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409873575bc060ce161aa7 HTTP/1.1" 502 150 2025-12-17 23:39:16,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409873575bc060ce161aa7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:16,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409545575bc060ce1612a2 2025-12-17 23:39:16,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409545575bc060ce1612a2 HTTP/1.1" 502 150 2025-12-17 23:39:16,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409545575bc060ce1612a2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34422 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:39:16,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409871575bc060ce161aa4 2025-12-17 23:39:16,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409871575bc060ce161aa4 HTTP/1.1" 502 150 2025-12-17 23:39:16,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409871575bc060ce161aa4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:17,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409876575bc060ce161aaa 2025-12-17 23:39:17,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409876575bc060ce161aaa HTTP/1.1" 502 150 2025-12-17 23:39:17,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409876575bc060ce161aaa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:17,296 - lava_callback - INFO - Received callback for node 69409c10575bc060ce1624c8 INFO: 172.21.0.1:34430 - "POST /node/69409c10575bc060ce1624c8 HTTP/1.0" 202 Accepted 2025-12-17 23:39:17,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940986c575bc060ce161a91 2025-12-17 23:39:17,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986c575bc060ce161a91 HTTP/1.1" 502 150 2025-12-17 23:39:17,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940986c575bc060ce161a91'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:17,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409859575bc060ce161a6c 2025-12-17 23:39:17,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409859575bc060ce161a6c HTTP/1.1" 502 150 2025-12-17 23:39:17,588 - lava_callback - ERROR - Error processing callback for node 69409859575bc060ce161a6c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409859575bc060ce161a6c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409859575bc060ce161a6c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409859575bc060ce161a6c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:17,589 - lava_callback - INFO - Starting async job submit for node 69409878575bc060ce161aad 2025-12-17 23:39:17,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:17,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409878575bc060ce161aad HTTP/1.1" 502 150 2025-12-17 23:39:17,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409878575bc060ce161aad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:17,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409878575bc060ce161aad 2025-12-17 23:39:17,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409878575bc060ce161aad HTTP/1.1" 502 150 2025-12-17 23:39:17,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409878575bc060ce161aad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:17,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940985b575bc060ce161a75 2025-12-17 23:39:17,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985b575bc060ce161a75 HTTP/1.1" 502 150 2025-12-17 23:39:17,606 - lava_callback - ERROR - Error processing callback for node 6940985b575bc060ce161a75: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940985b575bc060ce161a75 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940985b575bc060ce161a75 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940985b575bc060ce161a75 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:17,607 - lava_callback - INFO - Starting async job submit for node 6940987a575bc060ce161ab0 2025-12-17 23:39:17,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:17,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987a575bc060ce161ab0 HTTP/1.1" 502 150 2025-12-17 23:39:17,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940987a575bc060ce161ab0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:17,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940987a575bc060ce161ab0 2025-12-17 23:39:17,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987a575bc060ce161ab0 HTTP/1.1" 502 150 2025-12-17 23:39:17,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940987a575bc060ce161ab0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:17,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940986e575bc060ce161a96 2025-12-17 23:39:17,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986e575bc060ce161a96 HTTP/1.1" 502 150 2025-12-17 23:39:17,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940986e575bc060ce161a96'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:17,955 - lava_callback - INFO - Received callback for node 69409c12575bc060ce1624cb INFO: 172.21.0.1:34434 - "POST /node/69409c12575bc060ce1624cb HTTP/1.0" 202 Accepted 2025-12-17 23:39:18,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409873575bc060ce161aa7 2025-12-17 23:39:18,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409873575bc060ce161aa7 HTTP/1.1" 502 150 2025-12-17 23:39:18,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409873575bc060ce161aa7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:18,631 - lava_callback - INFO - Received callback for node 69409c14575bc060ce1624d1 INFO: 172.21.0.1:34440 - "POST /node/69409c14575bc060ce1624d1 HTTP/1.0" 202 Accepted 2025-12-17 23:39:18,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409845575bc060ce161a36 HTTP/1.1" 502 150 2025-12-17 23:39:18,833 - lava_callback - ERROR - Error processing callback for node 69409845575bc060ce161a36: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409845575bc060ce161a36 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409845575bc060ce161a36 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409845575bc060ce161a36 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:18,835 - lava_callback - INFO - Starting async job submit for node 6940987c575bc060ce161ab3 2025-12-17 23:39:18,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:18,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987c575bc060ce161ab3 HTTP/1.1" 502 150 2025-12-17 23:39:18,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940987c575bc060ce161ab3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:18,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940987c575bc060ce161ab3 2025-12-17 23:39:18,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987c575bc060ce161ab3 HTTP/1.1" 502 150 2025-12-17 23:39:18,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940987c575bc060ce161ab3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:18,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409861575bc060ce161a82 2025-12-17 23:39:18,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409861575bc060ce161a82 HTTP/1.1" 502 150 2025-12-17 23:39:18,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409861575bc060ce161a82'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:19,286 - lava_callback - INFO - Received callback for node 69409c16575bc060ce1624d4 INFO: 172.21.0.1:34454 - "POST /node/69409c16575bc060ce1624d4 HTTP/1.0" 202 Accepted 2025-12-17 23:39:19,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409878575bc060ce161aad 2025-12-17 23:39:19,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409878575bc060ce161aad HTTP/1.1" 502 150 2025-12-17 23:39:19,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409878575bc060ce161aad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:19,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940987a575bc060ce161ab0 2025-12-17 23:39:19,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987a575bc060ce161ab0 HTTP/1.1" 502 150 2025-12-17 23:39:19,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940987a575bc060ce161ab0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:19,968 - lava_callback - INFO - Received callback for node 69409c1c575bc060ce1624e3 INFO: 172.21.0.1:35550 - "POST /node/69409c1c575bc060ce1624e3 HTTP/1.0" 202 Accepted 2025-12-17 23:39:20,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409545575bc060ce1612a2 2025-12-17 23:39:20,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409545575bc060ce1612a2 HTTP/1.1" 502 150 2025-12-17 23:39:20,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409545575bc060ce1612a2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:20,632 - lava_callback - INFO - Received callback for node 69409c20575bc060ce1624f4 INFO: 172.21.0.1:35562 - "POST /node/69409c20575bc060ce1624f4 HTTP/1.0" 202 Accepted 2025-12-17 23:39:20,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409871575bc060ce161aa4 2025-12-17 23:39:20,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409871575bc060ce161aa4 HTTP/1.1" 502 150 2025-12-17 23:39:20,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409871575bc060ce161aa4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:20,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940987c575bc060ce161ab3 2025-12-17 23:39:20,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987c575bc060ce161ab3 HTTP/1.1" 502 150 2025-12-17 23:39:20,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940987c575bc060ce161ab3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:21,094 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940985d575bc060ce161a7c 2025-12-17 23:39:21,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985d575bc060ce161a7c HTTP/1.1" 502 150 2025-12-17 23:39:21,096 - lava_callback - ERROR - Error processing callback for node 6940985d575bc060ce161a7c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940985d575bc060ce161a7c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940985d575bc060ce161a7c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940985d575bc060ce161a7c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:21,097 - lava_callback - INFO - Starting async job submit for node 6940987e575bc060ce161ab6 2025-12-17 23:39:21,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:21,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987e575bc060ce161ab6 HTTP/1.1" 502 150 2025-12-17 23:39:21,124 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940987e575bc060ce161ab6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:21,124 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940987e575bc060ce161ab6 2025-12-17 23:39:21,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987e575bc060ce161ab6 HTTP/1.1" 502 150 2025-12-17 23:39:21,125 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940987e575bc060ce161ab6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:21,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409863575bc060ce161a85 2025-12-17 23:39:21,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409863575bc060ce161a85 HTTP/1.1" 502 150 2025-12-17 23:39:21,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409863575bc060ce161a85'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:22,388 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409873575bc060ce161aa7 2025-12-17 23:39:22,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409873575bc060ce161aa7 HTTP/1.1" 502 150 2025-12-17 23:39:22,389 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409873575bc060ce161aa7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:23,125 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940987e575bc060ce161ab6 2025-12-17 23:39:23,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987e575bc060ce161ab6 HTTP/1.1" 502 150 2025-12-17 23:39:23,126 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940987e575bc060ce161ab6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:23,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409878575bc060ce161aad 2025-12-17 23:39:23,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409878575bc060ce161aad HTTP/1.1" 502 150 2025-12-17 23:39:23,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409878575bc060ce161aad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:23,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940987a575bc060ce161ab0 2025-12-17 23:39:23,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987a575bc060ce161ab0 HTTP/1.1" 502 150 2025-12-17 23:39:23,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940987a575bc060ce161ab0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:24,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940987c575bc060ce161ab3 2025-12-17 23:39:24,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987c575bc060ce161ab3 HTTP/1.1" 502 150 2025-12-17 23:39:24,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940987c575bc060ce161ab3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:25,050 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409876575bc060ce161aaa INFO: 172.21.0.1:35566 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:39:26,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940985f575bc060ce161a7f 2025-12-17 23:39:26,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940985f575bc060ce161a7f HTTP/1.1" 502 150 2025-12-17 23:39:26,876 - lava_callback - ERROR - Error processing callback for node 6940985f575bc060ce161a7f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940985f575bc060ce161a7f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940985f575bc060ce161a7f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940985f575bc060ce161a7f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:26,878 - lava_callback - INFO - Starting async job submit for node 69409880575bc060ce161ab9 2025-12-17 23:39:26,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:26,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409880575bc060ce161ab9 HTTP/1.1" 502 150 2025-12-17 23:39:26,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409880575bc060ce161ab9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:26,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409880575bc060ce161ab9 2025-12-17 23:39:26,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409880575bc060ce161ab9 HTTP/1.1" 502 150 2025-12-17 23:39:26,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409880575bc060ce161ab9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:26,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409861575bc060ce161a82 2025-12-17 23:39:26,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409861575bc060ce161a82 HTTP/1.1" 502 150 2025-12-17 23:39:26,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409861575bc060ce161a82'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:27,127 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940987e575bc060ce161ab6 2025-12-17 23:39:27,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987e575bc060ce161ab6 HTTP/1.1" 502 150 2025-12-17 23:39:27,128 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940987e575bc060ce161ab6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:27,997 - lava_callback - INFO - Received callback for node 69409c1d575bc060ce1624ea INFO: 172.21.0.1:35582 - "POST /node/69409c1d575bc060ce1624ea HTTP/1.0" 202 Accepted 2025-12-17 23:39:28,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409545575bc060ce1612a2 2025-12-17 23:39:28,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409545575bc060ce1612a2 HTTP/1.1" 502 150 2025-12-17 23:39:28,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409545575bc060ce1612a2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:28,652 - lava_callback - INFO - Received callback for node 69409c22575bc060ce1624fb INFO: 172.21.0.1:35586 - "POST /node/69409c22575bc060ce1624fb HTTP/1.0" 202 Accepted 2025-12-17 23:39:28,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409871575bc060ce161aa4 2025-12-17 23:39:28,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409871575bc060ce161aa4 HTTP/1.1" 502 150 2025-12-17 23:39:28,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409871575bc060ce161aa4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:28,796 - lava_callback - INFO - Received callback for node 69409c1f575bc060ce1624f1 INFO: 172.21.0.1:35594 - "POST /node/69409c1f575bc060ce1624f1 HTTP/1.0" 202 Accepted 2025-12-17 23:39:28,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409880575bc060ce161ab9 2025-12-17 23:39:28,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409880575bc060ce161ab9 HTTP/1.1" 502 150 2025-12-17 23:39:28,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409880575bc060ce161ab9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:29,301 - lava_callback - INFO - Received callback for node 69409c26575bc060ce162505 INFO: 172.21.0.1:35600 - "POST /node/69409c26575bc060ce162505 HTTP/1.0" 202 Accepted 2025-12-17 23:39:29,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409866575bc060ce161a88 HTTP/1.1" 502 150 2025-12-17 23:39:29,328 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409866575bc060ce161a88'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:29,538 - lava_callback - INFO - Received callback for node 69409c2a575bc060ce16250e INFO: 172.21.0.1:35612 - "POST /node/69409c2a575bc060ce16250e HTTP/1.0" 202 Accepted 2025-12-17 23:39:30,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940986a575bc060ce161a8e 2025-12-17 23:39:30,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986a575bc060ce161a8e HTTP/1.1" 502 150 2025-12-17 23:39:30,040 - lava_callback - ERROR - Error processing callback for node 6940986a575bc060ce161a8e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940986a575bc060ce161a8e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940986a575bc060ce161a8e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940986a575bc060ce161a8e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:30,042 - lava_callback - INFO - Starting async job submit for node 69409802575bc060ce161991 2025-12-17 23:39:30,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:30,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409802575bc060ce161991 HTTP/1.1" 502 150 2025-12-17 23:39:30,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409802575bc060ce161991'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:30,101 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409802575bc060ce161991 2025-12-17 23:39:30,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409802575bc060ce161991 HTTP/1.1" 502 150 2025-12-17 23:39:30,102 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409802575bc060ce161991'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:30,305 - lava_callback - INFO - Received callback for node 69409c2d575bc060ce162514 INFO: 172.21.0.1:58680 - "POST /node/69409c2d575bc060ce162514 HTTP/1.0" 202 Accepted 2025-12-17 23:39:30,389 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409873575bc060ce161aa7 2025-12-17 23:39:30,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409873575bc060ce161aa7 HTTP/1.1" 502 150 2025-12-17 23:39:30,390 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409873575bc060ce161aa7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:31,045 - lava_callback - INFO - Received callback for node 69409c31575bc060ce162522 INFO: 172.21.0.1:58686 - "POST /node/69409c31575bc060ce162522 HTTP/1.0" 202 Accepted 2025-12-17 23:39:31,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409878575bc060ce161aad 2025-12-17 23:39:31,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409878575bc060ce161aad HTTP/1.1" 502 150 2025-12-17 23:39:31,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409878575bc060ce161aad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:31,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940987a575bc060ce161ab0 2025-12-17 23:39:31,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987a575bc060ce161ab0 HTTP/1.1" 502 150 2025-12-17 23:39:31,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940987a575bc060ce161ab0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:32,102 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409802575bc060ce161991 2025-12-17 23:39:32,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409802575bc060ce161991 HTTP/1.1" 502 150 2025-12-17 23:39:32,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409802575bc060ce161991'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:32,273 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694097df575bc060ce16193a 2025-12-17 23:39:32,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940987c575bc060ce161ab3 2025-12-17 23:39:32,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987c575bc060ce161ab3 HTTP/1.1" 502 150 2025-12-17 23:39:32,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940987c575bc060ce161ab3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:32,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409880575bc060ce161ab9 2025-12-17 23:39:32,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409880575bc060ce161ab9 HTTP/1.1" 502 150 2025-12-17 23:39:32,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409880575bc060ce161ab9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:33,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940986c575bc060ce161a91 2025-12-17 23:39:33,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986c575bc060ce161a91 HTTP/1.1" 502 150 2025-12-17 23:39:33,489 - lava_callback - ERROR - Error processing callback for node 6940986c575bc060ce161a91: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940986c575bc060ce161a91 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940986c575bc060ce161a91 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940986c575bc060ce161a91 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:33,490 - lava_callback - INFO - Starting async job submit for node 6942e37fe43d2b76e9023a5a 2025-12-17 23:39:33,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:33,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e37fe43d2b76e9023a5a HTTP/1.1" 502 150 2025-12-17 23:39:33,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e37fe43d2b76e9023a5a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:33,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e37fe43d2b76e9023a5a 2025-12-17 23:39:33,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e37fe43d2b76e9023a5a HTTP/1.1" 502 150 2025-12-17 23:39:33,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e37fe43d2b76e9023a5a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:33,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940986e575bc060ce161a96 2025-12-17 23:39:33,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940986e575bc060ce161a96 HTTP/1.1" 502 150 2025-12-17 23:39:33,805 - lava_callback - ERROR - Error processing callback for node 6940986e575bc060ce161a96: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940986e575bc060ce161a96 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940986e575bc060ce161a96 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940986e575bc060ce161a96 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:33,807 - lava_callback - INFO - Starting async job submit for node 69409882575bc060ce161ac0 2025-12-17 23:39:33,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:33,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409882575bc060ce161ac0 HTTP/1.1" 502 150 2025-12-17 23:39:33,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409882575bc060ce161ac0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:33,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409882575bc060ce161ac0 2025-12-17 23:39:33,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409882575bc060ce161ac0 HTTP/1.1" 502 150 2025-12-17 23:39:33,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409882575bc060ce161ac0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:35,128 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940987e575bc060ce161ab6 2025-12-17 23:39:35,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987e575bc060ce161ab6 HTTP/1.1" 502 150 2025-12-17 23:39:35,129 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940987e575bc060ce161ab6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:35,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e37fe43d2b76e9023a5a 2025-12-17 23:39:35,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e37fe43d2b76e9023a5a HTTP/1.1" 502 150 2025-12-17 23:39:35,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e37fe43d2b76e9023a5a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:35,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409882575bc060ce161ac0 2025-12-17 23:39:35,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409882575bc060ce161ac0 HTTP/1.1" 502 150 2025-12-17 23:39:35,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409882575bc060ce161ac0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:36,103 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409802575bc060ce161991 2025-12-17 23:39:36,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409802575bc060ce161991 HTTP/1.1" 502 150 2025-12-17 23:39:36,105 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409802575bc060ce161991'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58690 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:39:36,958 - lava_callback - INFO - Received callback for node 69409be5575bc060ce16245e INFO: 172.21.0.1:58698 - "POST /node/69409be5575bc060ce16245e HTTP/1.0" 202 Accepted 2025-12-17 23:39:37,317 - lava_callback - INFO - Received callback for node 69409c24575bc060ce1624fe INFO: 172.21.0.1:58704 - "POST /node/69409c24575bc060ce1624fe HTTP/1.0" 202 Accepted 2025-12-17 23:39:37,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409863575bc060ce161a85 2025-12-17 23:39:37,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409863575bc060ce161a85 HTTP/1.1" 502 150 2025-12-17 23:39:37,651 - lava_callback - ERROR - Error processing callback for node 69409863575bc060ce161a85: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409863575bc060ce161a85 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409863575bc060ce161a85 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409863575bc060ce161a85 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:37,653 - lava_callback - INFO - Starting async job submit for node 69409884575bc060ce161ac9 2025-12-17 23:39:37,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:37,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409884575bc060ce161ac9 HTTP/1.1" 502 150 2025-12-17 23:39:37,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409884575bc060ce161ac9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:37,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409884575bc060ce161ac9 2025-12-17 23:39:37,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409884575bc060ce161ac9 HTTP/1.1" 502 150 2025-12-17 23:39:37,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409884575bc060ce161ac9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:37,975 - lava_callback - INFO - Received callback for node 69409c28575bc060ce16250b INFO: 172.21.0.1:58718 - "POST /node/69409c28575bc060ce16250b HTTP/1.0" 202 Accepted 2025-12-17 23:39:38,569 - lava_callback - INFO - Received callback for node 6940a2d9575bc060ce163100 INFO: 172.21.0.1:58728 - "POST /node/6940a2d9575bc060ce163100 HTTP/1.0" 202 Accepted 2025-12-17 23:39:38,634 - lava_callback - INFO - Received callback for node 69409c2b575bc060ce162511 INFO: 172.21.0.1:58738 - "POST /node/69409c2b575bc060ce162511 HTTP/1.0" 202 Accepted 2025-12-17 23:39:38,734 - lava_callback - INFO - Received callback for node 6940a36d575bc060ce1631b0 INFO: 172.21.0.1:58754 - "POST /node/6940a36d575bc060ce1631b0 HTTP/1.0" 202 Accepted 2025-12-17 23:39:39,306 - lava_callback - INFO - Received callback for node 69409c2e575bc060ce162517 INFO: 172.21.0.1:58766 - "POST /node/69409c2e575bc060ce162517 HTTP/1.0" 202 Accepted 2025-12-17 23:39:39,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e37fe43d2b76e9023a5a 2025-12-17 23:39:39,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e37fe43d2b76e9023a5a HTTP/1.1" 502 150 2025-12-17 23:39:39,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e37fe43d2b76e9023a5a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:39,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409884575bc060ce161ac9 2025-12-17 23:39:39,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409884575bc060ce161ac9 HTTP/1.1" 502 150 2025-12-17 23:39:39,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409884575bc060ce161ac9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:39,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409882575bc060ce161ac0 2025-12-17 23:39:39,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409882575bc060ce161ac0 HTTP/1.1" 502 150 2025-12-17 23:39:39,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409882575bc060ce161ac0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:39,957 - lava_callback - INFO - Received callback for node 69409c32575bc060ce162527 INFO: 172.21.0.1:46852 - "POST /node/69409c32575bc060ce162527 HTTP/1.0" 202 Accepted 2025-12-17 23:39:40,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409876575bc060ce161aaa HTTP/1.1" 502 150 2025-12-17 23:39:40,336 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409876575bc060ce161aaa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:40,629 - lava_callback - INFO - Received callback for node 69409c34575bc060ce16252e INFO: 172.21.0.1:46860 - "POST /node/69409c34575bc060ce16252e HTTP/1.0" 202 Accepted 2025-12-17 23:39:40,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409880575bc060ce161ab9 2025-12-17 23:39:40,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409880575bc060ce161ab9 HTTP/1.1" 502 150 2025-12-17 23:39:40,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409880575bc060ce161ab9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:41,287 - lava_callback - INFO - Received callback for node 69409c35575bc060ce162531 INFO: 172.21.0.1:46876 - "POST /node/69409c35575bc060ce162531 HTTP/1.0" 202 Accepted 2025-12-17 23:39:41,940 - lava_callback - INFO - Received callback for node 69409c37575bc060ce162534 INFO: 172.21.0.1:46884 - "POST /node/69409c37575bc060ce162534 HTTP/1.0" 202 Accepted 2025-12-17 23:39:42,625 - lava_callback - INFO - Received callback for node 69409c38575bc060ce162537 INFO: 172.21.0.1:46900 - "POST /node/69409c38575bc060ce162537 HTTP/1.0" 202 Accepted 2025-12-17 23:39:42,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409861575bc060ce161a82 2025-12-17 23:39:43,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409861575bc060ce161a82 HTTP/1.1" 502 150 2025-12-17 23:39:43,000 - lava_callback - ERROR - Error processing callback for node 69409861575bc060ce161a82: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409861575bc060ce161a82 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409861575bc060ce161a82 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409861575bc060ce161a82 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:43,003 - lava_callback - INFO - Starting async job submit for node 6941e655ecc59770562f0bb7 2025-12-17 23:39:43,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:43,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941e655ecc59770562f0bb7 HTTP/1.1" 502 150 2025-12-17 23:39:43,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941e655ecc59770562f0bb7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:43,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941e655ecc59770562f0bb7 2025-12-17 23:39:43,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941e655ecc59770562f0bb7 HTTP/1.1" 502 150 2025-12-17 23:39:43,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941e655ecc59770562f0bb7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:43,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409884575bc060ce161ac9 2025-12-17 23:39:43,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409884575bc060ce161ac9 HTTP/1.1" 502 150 2025-12-17 23:39:43,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409884575bc060ce161ac9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:44,105 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409802575bc060ce161991 2025-12-17 23:39:44,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409802575bc060ce161991 HTTP/1.1" 502 150 2025-12-17 23:39:44,106 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409802575bc060ce161991'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:44,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409545575bc060ce1612a2 2025-12-17 23:39:44,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409545575bc060ce1612a2 HTTP/1.1" 502 150 2025-12-17 23:39:44,525 - lava_callback - ERROR - Error processing callback for node 69409545575bc060ce1612a2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409545575bc060ce1612a2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409545575bc060ce1612a2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409545575bc060ce1612a2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:44,527 - lava_callback - INFO - Starting async job submit for node 69409887575bc060ce161ad0 2025-12-17 23:39:44,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:44,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409887575bc060ce161ad0 HTTP/1.1" 502 150 2025-12-17 23:39:44,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409887575bc060ce161ad0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:44,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409887575bc060ce161ad0 2025-12-17 23:39:44,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409887575bc060ce161ad0 HTTP/1.1" 502 150 2025-12-17 23:39:44,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409887575bc060ce161ad0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:44,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409871575bc060ce161aa4 2025-12-17 23:39:44,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409871575bc060ce161aa4 HTTP/1.1" 502 150 2025-12-17 23:39:44,725 - lava_callback - ERROR - Error processing callback for node 69409871575bc060ce161aa4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409871575bc060ce161aa4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409871575bc060ce161aa4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409871575bc060ce161aa4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:44,727 - lava_callback - INFO - Starting async job submit for node 69409888575bc060ce161ad3 2025-12-17 23:39:44,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:44,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409888575bc060ce161ad3 HTTP/1.1" 502 150 2025-12-17 23:39:44,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409888575bc060ce161ad3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:44,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409888575bc060ce161ad3 2025-12-17 23:39:44,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409888575bc060ce161ad3 HTTP/1.1" 502 150 2025-12-17 23:39:44,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409888575bc060ce161ad3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:45,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941e655ecc59770562f0bb7 2025-12-17 23:39:45,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941e655ecc59770562f0bb7 HTTP/1.1" 502 150 2025-12-17 23:39:45,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941e655ecc59770562f0bb7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:45,328 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409866575bc060ce161a88 2025-12-17 23:39:45,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409866575bc060ce161a88 HTTP/1.1" 502 150 2025-12-17 23:39:45,330 - lava_callback - ERROR - Error processing callback for node 69409866575bc060ce161a88: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409866575bc060ce161a88 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409866575bc060ce161a88 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409866575bc060ce161a88 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:45,333 - lava_callback - INFO - Starting async job submit for node 6940988a575bc060ce161ad6 2025-12-17 23:39:45,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:45,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988a575bc060ce161ad6 HTTP/1.1" 502 150 2025-12-17 23:39:45,341 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988a575bc060ce161ad6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:45,341 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988a575bc060ce161ad6 2025-12-17 23:39:45,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988a575bc060ce161ad6 HTTP/1.1" 502 150 2025-12-17 23:39:45,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988a575bc060ce161ad6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:46,390 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409873575bc060ce161aa7 2025-12-17 23:39:46,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409873575bc060ce161aa7 HTTP/1.1" 502 150 2025-12-17 23:39:46,392 - lava_callback - ERROR - Error processing callback for node 69409873575bc060ce161aa7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409873575bc060ce161aa7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409873575bc060ce161aa7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409873575bc060ce161aa7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:46,394 - lava_callback - INFO - Starting async job submit for node 6940988c575bc060ce161add 2025-12-17 23:39:46,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:46,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988c575bc060ce161add HTTP/1.1" 502 150 2025-12-17 23:39:46,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988c575bc060ce161add'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:46,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988c575bc060ce161add 2025-12-17 23:39:46,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988c575bc060ce161add HTTP/1.1" 502 150 2025-12-17 23:39:46,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988c575bc060ce161add'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46908 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:39:46,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409887575bc060ce161ad0 2025-12-17 23:39:46,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409887575bc060ce161ad0 HTTP/1.1" 502 150 2025-12-17 23:39:46,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409887575bc060ce161ad0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:46,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409888575bc060ce161ad3 2025-12-17 23:39:46,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409888575bc060ce161ad3 HTTP/1.1" 502 150 2025-12-17 23:39:46,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409888575bc060ce161ad3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:47,343 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988a575bc060ce161ad6 2025-12-17 23:39:47,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988a575bc060ce161ad6 HTTP/1.1" 502 150 2025-12-17 23:39:47,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988a575bc060ce161ad6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:47,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e37fe43d2b76e9023a5a 2025-12-17 23:39:47,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e37fe43d2b76e9023a5a HTTP/1.1" 502 150 2025-12-17 23:39:47,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942e37fe43d2b76e9023a5a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:47,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409878575bc060ce161aad 2025-12-17 23:39:47,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409878575bc060ce161aad HTTP/1.1" 502 150 2025-12-17 23:39:47,603 - lava_callback - ERROR - Error processing callback for node 69409878575bc060ce161aad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409878575bc060ce161aad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409878575bc060ce161aad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409878575bc060ce161aad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:47,604 - lava_callback - INFO - Starting async job submit for node 6940988d575bc060ce161ae0 2025-12-17 23:39:47,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:47,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988d575bc060ce161ae0 HTTP/1.1" 502 150 2025-12-17 23:39:47,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988d575bc060ce161ae0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:47,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988d575bc060ce161ae0 2025-12-17 23:39:47,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988d575bc060ce161ae0 HTTP/1.1" 502 150 2025-12-17 23:39:47,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988d575bc060ce161ae0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:47,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940987a575bc060ce161ab0 2025-12-17 23:39:47,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987a575bc060ce161ab0 HTTP/1.1" 502 150 2025-12-17 23:39:47,620 - lava_callback - ERROR - Error processing callback for node 6940987a575bc060ce161ab0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940987a575bc060ce161ab0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940987a575bc060ce161ab0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940987a575bc060ce161ab0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:47,621 - lava_callback - INFO - Starting async job submit for node 69409890575bc060ce161ae6 2025-12-17 23:39:47,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:47,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409890575bc060ce161ae6 HTTP/1.1" 502 150 2025-12-17 23:39:47,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409890575bc060ce161ae6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:47,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409890575bc060ce161ae6 2025-12-17 23:39:47,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409890575bc060ce161ae6 HTTP/1.1" 502 150 2025-12-17 23:39:47,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409890575bc060ce161ae6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:47,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409882575bc060ce161ac0 2025-12-17 23:39:47,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409882575bc060ce161ac0 HTTP/1.1" 502 150 2025-12-17 23:39:47,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409882575bc060ce161ac0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:48,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988c575bc060ce161add 2025-12-17 23:39:48,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988c575bc060ce161add HTTP/1.1" 502 150 2025-12-17 23:39:48,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988c575bc060ce161add'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:48,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940987c575bc060ce161ab3 2025-12-17 23:39:48,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987c575bc060ce161ab3 HTTP/1.1" 502 150 2025-12-17 23:39:48,874 - lava_callback - ERROR - Error processing callback for node 6940987c575bc060ce161ab3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940987c575bc060ce161ab3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940987c575bc060ce161ab3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940987c575bc060ce161ab3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:48,876 - lava_callback - INFO - Starting async job submit for node 69409891575bc060ce161ae9 2025-12-17 23:39:48,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:48,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409891575bc060ce161ae9 HTTP/1.1" 502 150 2025-12-17 23:39:48,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409891575bc060ce161ae9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:48,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409891575bc060ce161ae9 2025-12-17 23:39:48,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409891575bc060ce161ae9 HTTP/1.1" 502 150 2025-12-17 23:39:48,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409891575bc060ce161ae9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:49,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941e655ecc59770562f0bb7 2025-12-17 23:39:49,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941e655ecc59770562f0bb7 HTTP/1.1" 502 150 2025-12-17 23:39:49,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941e655ecc59770562f0bb7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:49,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988d575bc060ce161ae0 2025-12-17 23:39:49,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988d575bc060ce161ae0 HTTP/1.1" 502 150 2025-12-17 23:39:49,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988d575bc060ce161ae0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:49,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409890575bc060ce161ae6 2025-12-17 23:39:49,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409890575bc060ce161ae6 HTTP/1.1" 502 150 2025-12-17 23:39:49,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409890575bc060ce161ae6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:50,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409887575bc060ce161ad0 2025-12-17 23:39:50,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409887575bc060ce161ad0 HTTP/1.1" 502 150 2025-12-17 23:39:50,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409887575bc060ce161ad0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:50,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409888575bc060ce161ad3 2025-12-17 23:39:50,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409888575bc060ce161ad3 HTTP/1.1" 502 150 2025-12-17 23:39:50,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409888575bc060ce161ad3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:50,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409891575bc060ce161ae9 2025-12-17 23:39:50,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409891575bc060ce161ae9 HTTP/1.1" 502 150 2025-12-17 23:39:50,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409891575bc060ce161ae9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:51,129 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940987e575bc060ce161ab6 2025-12-17 23:39:51,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988a575bc060ce161ad6 2025-12-17 23:39:51,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988a575bc060ce161ad6 HTTP/1.1" 502 150 2025-12-17 23:39:51,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988a575bc060ce161ad6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:51,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409884575bc060ce161ac9 2025-12-17 23:39:51,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409884575bc060ce161ac9 HTTP/1.1" 502 150 2025-12-17 23:39:51,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409884575bc060ce161ac9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:52,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988c575bc060ce161add 2025-12-17 23:39:52,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988c575bc060ce161add HTTP/1.1" 502 150 2025-12-17 23:39:52,405 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988c575bc060ce161add'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:53,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988d575bc060ce161ae0 2025-12-17 23:39:53,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988d575bc060ce161ae0 HTTP/1.1" 502 150 2025-12-17 23:39:53,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988d575bc060ce161ae0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:53,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409890575bc060ce161ae6 2025-12-17 23:39:53,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409890575bc060ce161ae6 HTTP/1.1" 502 150 2025-12-17 23:39:53,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409890575bc060ce161ae6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:54,549 - lava_callback - INFO - Received callback for node 69409c39575bc060ce16253e INFO: 172.21.0.1:43394 - "POST /node/69409c39575bc060ce16253e HTTP/1.0" 202 Accepted 2025-12-17 23:39:54,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409891575bc060ce161ae9 2025-12-17 23:39:54,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409891575bc060ce161ae9 HTTP/1.1" 502 150 2025-12-17 23:39:54,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409891575bc060ce161ae9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:56,336 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409876575bc060ce161aaa 2025-12-17 23:39:56,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409876575bc060ce161aaa HTTP/1.1" 502 150 2025-12-17 23:39:56,338 - lava_callback - ERROR - Error processing callback for node 69409876575bc060ce161aaa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409876575bc060ce161aaa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409876575bc060ce161aaa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409876575bc060ce161aaa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:56,340 - lava_callback - INFO - Starting async job submit for node 69409893575bc060ce161aec 2025-12-17 23:39:56,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:56,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409893575bc060ce161aec HTTP/1.1" 502 150 2025-12-17 23:39:56,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409893575bc060ce161aec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:56,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409893575bc060ce161aec 2025-12-17 23:39:56,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409893575bc060ce161aec HTTP/1.1" 502 150 2025-12-17 23:39:56,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409893575bc060ce161aec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43400 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:39:56,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409880575bc060ce161ab9 2025-12-17 23:39:56,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409880575bc060ce161ab9 HTTP/1.1" 502 150 2025-12-17 23:39:56,897 - lava_callback - ERROR - Error processing callback for node 69409880575bc060ce161ab9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409880575bc060ce161ab9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409880575bc060ce161ab9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409880575bc060ce161ab9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:39:56,899 - lava_callback - INFO - Starting async job submit for node 69409895575bc060ce161aef 2025-12-17 23:39:56,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:39:56,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409895575bc060ce161aef HTTP/1.1" 502 150 2025-12-17 23:39:56,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409895575bc060ce161aef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:56,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409895575bc060ce161aef 2025-12-17 23:39:56,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409895575bc060ce161aef HTTP/1.1" 502 150 2025-12-17 23:39:56,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409895575bc060ce161aef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:57,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941e655ecc59770562f0bb7 2025-12-17 23:39:57,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941e655ecc59770562f0bb7 HTTP/1.1" 502 150 2025-12-17 23:39:57,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941e655ecc59770562f0bb7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:57,365 - lava_callback - INFO - Received callback for node 69409c3a575bc060ce162541 INFO: 172.21.0.1:43414 - "POST /node/69409c3a575bc060ce162541 HTTP/1.0" 202 Accepted 2025-12-17 23:39:58,048 - lava_callback - INFO - Received callback for node 69409c3d575bc060ce162547 INFO: 172.21.0.1:43418 - "POST /node/69409c3d575bc060ce162547 HTTP/1.0" 202 Accepted 2025-12-17 23:39:58,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409893575bc060ce161aec 2025-12-17 23:39:58,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409893575bc060ce161aec HTTP/1.1" 502 150 2025-12-17 23:39:58,350 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409893575bc060ce161aec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:58,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409887575bc060ce161ad0 2025-12-17 23:39:58,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409887575bc060ce161ad0 HTTP/1.1" 502 150 2025-12-17 23:39:58,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409887575bc060ce161ad0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:58,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409888575bc060ce161ad3 2025-12-17 23:39:58,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409888575bc060ce161ad3 HTTP/1.1" 502 150 2025-12-17 23:39:58,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409888575bc060ce161ad3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:58,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409895575bc060ce161aef 2025-12-17 23:39:58,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409895575bc060ce161aef HTTP/1.1" 502 150 2025-12-17 23:39:58,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409895575bc060ce161aef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:39:59,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988a575bc060ce161ad6 2025-12-17 23:39:59,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988a575bc060ce161ad6 HTTP/1.1" 502 150 2025-12-17 23:39:59,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988a575bc060ce161ad6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:00,106 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409802575bc060ce161991 2025-12-17 23:40:00,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409802575bc060ce161991 HTTP/1.1" 502 150 2025-12-17 23:40:00,108 - lava_callback - ERROR - Error processing callback for node 69409802575bc060ce161991: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409802575bc060ce161991 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409802575bc060ce161991 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409802575bc060ce161991 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:00,110 - lava_callback - INFO - Starting async job submit for node 6940989e575bc060ce161b08 2025-12-17 23:40:00,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:00,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989e575bc060ce161b08 HTTP/1.1" 502 150 2025-12-17 23:40:00,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940989e575bc060ce161b08'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:00,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940989e575bc060ce161b08 2025-12-17 23:40:00,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989e575bc060ce161b08 HTTP/1.1" 502 150 2025-12-17 23:40:00,118 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940989e575bc060ce161b08'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:00,405 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988c575bc060ce161add 2025-12-17 23:40:00,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988c575bc060ce161add HTTP/1.1" 502 150 2025-12-17 23:40:00,406 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988c575bc060ce161add'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:01,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988d575bc060ce161ae0 2025-12-17 23:40:01,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988d575bc060ce161ae0 HTTP/1.1" 502 150 2025-12-17 23:40:01,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940988d575bc060ce161ae0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:01,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409890575bc060ce161ae6 2025-12-17 23:40:01,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409890575bc060ce161ae6 HTTP/1.1" 502 150 2025-12-17 23:40:01,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409890575bc060ce161ae6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:02,118 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940989e575bc060ce161b08 2025-12-17 23:40:02,350 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409893575bc060ce161aec 2025-12-17 23:40:02,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409893575bc060ce161aec HTTP/1.1" 502 150 2025-12-17 23:40:02,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409893575bc060ce161aec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:02,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409891575bc060ce161ae9 2025-12-17 23:40:02,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409891575bc060ce161ae9 HTTP/1.1" 502 150 2025-12-17 23:40:02,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409891575bc060ce161ae9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:02,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409895575bc060ce161aef 2025-12-17 23:40:02,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409895575bc060ce161aef HTTP/1.1" 502 150 2025-12-17 23:40:02,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409895575bc060ce161aef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:03,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694097df575bc060ce16193a HTTP/1.1" 502 150 2025-12-17 23:40:03,121 - lava_callback - ERROR - Error processing callback for node 694097df575bc060ce16193a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097df575bc060ce16193a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097df575bc060ce16193a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694097df575bc060ce16193a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:03,122 - lava_callback - INFO - Starting async job submit for node 6940945f575bc060ce16100e 2025-12-17 23:40:03,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:03,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940945f575bc060ce16100e HTTP/1.1" 502 150 2025-12-17 23:40:03,173 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940945f575bc060ce16100e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:03,173 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940945f575bc060ce16100e 2025-12-17 23:40:03,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940945f575bc060ce16100e HTTP/1.1" 502 150 2025-12-17 23:40:03,174 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940945f575bc060ce16100e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47610 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:40:03,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942e37fe43d2b76e9023a5a 2025-12-17 23:40:03,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942e37fe43d2b76e9023a5a HTTP/1.1" 502 150 2025-12-17 23:40:03,512 - lava_callback - ERROR - Error processing callback for node 6942e37fe43d2b76e9023a5a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e37fe43d2b76e9023a5a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e37fe43d2b76e9023a5a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942e37fe43d2b76e9023a5a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:03,514 - lava_callback - INFO - Starting async job submit for node 69409897575bc060ce161af8 2025-12-17 23:40:03,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:03,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409897575bc060ce161af8 HTTP/1.1" 502 150 2025-12-17 23:40:03,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409897575bc060ce161af8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:03,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409897575bc060ce161af8 2025-12-17 23:40:03,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409897575bc060ce161af8 HTTP/1.1" 502 150 2025-12-17 23:40:03,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409897575bc060ce161af8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:03,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409882575bc060ce161ac0 2025-12-17 23:40:03,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409882575bc060ce161ac0 HTTP/1.1" 502 150 2025-12-17 23:40:03,833 - lava_callback - ERROR - Error processing callback for node 69409882575bc060ce161ac0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409882575bc060ce161ac0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409882575bc060ce161ac0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409882575bc060ce161ac0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:03,836 - lava_callback - INFO - Starting async job submit for node 69409898575bc060ce161aff 2025-12-17 23:40:03,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:03,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409898575bc060ce161aff HTTP/1.1" 502 150 2025-12-17 23:40:03,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409898575bc060ce161aff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:03,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409898575bc060ce161aff 2025-12-17 23:40:03,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409898575bc060ce161aff HTTP/1.1" 502 150 2025-12-17 23:40:03,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409898575bc060ce161aff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:05,174 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940945f575bc060ce16100e 2025-12-17 23:40:05,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940945f575bc060ce16100e HTTP/1.1" 502 150 2025-12-17 23:40:05,176 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940945f575bc060ce16100e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:05,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409897575bc060ce161af8 2025-12-17 23:40:05,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409897575bc060ce161af8 HTTP/1.1" 502 150 2025-12-17 23:40:05,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409897575bc060ce161af8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:05,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409898575bc060ce161aff 2025-12-17 23:40:05,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409898575bc060ce161aff HTTP/1.1" 502 150 2025-12-17 23:40:05,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409898575bc060ce161aff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:06,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940987e575bc060ce161ab6 HTTP/1.1" 502 150 2025-12-17 23:40:06,448 - lava_callback - ERROR - Error processing callback for node 6940987e575bc060ce161ab6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940987e575bc060ce161ab6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940987e575bc060ce161ab6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940987e575bc060ce161ab6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:06,450 - lava_callback - INFO - Starting async job submit for node 6940989c575bc060ce161b05 2025-12-17 23:40:06,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:06,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989c575bc060ce161b05 HTTP/1.1" 502 150 2025-12-17 23:40:06,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940989c575bc060ce161b05'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:06,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940989c575bc060ce161b05 2025-12-17 23:40:06,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989c575bc060ce161b05 HTTP/1.1" 502 150 2025-12-17 23:40:06,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940989c575bc060ce161b05'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47618 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:40:07,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409884575bc060ce161ac9 2025-12-17 23:40:07,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409884575bc060ce161ac9 HTTP/1.1" 502 150 2025-12-17 23:40:07,671 - lava_callback - ERROR - Error processing callback for node 69409884575bc060ce161ac9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409884575bc060ce161ac9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409884575bc060ce161ac9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409884575bc060ce161ac9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:07,673 - lava_callback - INFO - Starting async job submit for node 694098a1575bc060ce161b0f 2025-12-17 23:40:07,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:07,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a1575bc060ce161b0f HTTP/1.1" 502 150 2025-12-17 23:40:07,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a1575bc060ce161b0f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:07,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a1575bc060ce161b0f 2025-12-17 23:40:07,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a1575bc060ce161b0f HTTP/1.1" 502 150 2025-12-17 23:40:07,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a1575bc060ce161b0f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:08,041 - lava_callback - INFO - Received callback for node 69409c4c575bc060ce16257d INFO: 172.21.0.1:47630 - "POST /node/69409c4c575bc060ce16257d HTTP/1.0" 202 Accepted 2025-12-17 23:40:08,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940989c575bc060ce161b05 2025-12-17 23:40:08,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989c575bc060ce161b05 HTTP/1.1" 502 150 2025-12-17 23:40:08,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940989c575bc060ce161b05'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:08,685 - lava_callback - INFO - Received callback for node 69409c4f575bc060ce162580 INFO: 172.21.0.1:47642 - "POST /node/69409c4f575bc060ce162580 HTTP/1.0" 202 Accepted 2025-12-17 23:40:08,839 - lava_callback - INFO - Received callback for node 69409c47575bc060ce16256a INFO: 172.21.0.1:47646 - "POST /node/69409c47575bc060ce16256a HTTP/1.0" 202 Accepted 2025-12-17 23:40:09,176 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940945f575bc060ce16100e 2025-12-17 23:40:09,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940945f575bc060ce16100e HTTP/1.1" 502 150 2025-12-17 23:40:09,177 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940945f575bc060ce16100e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:09,346 - lava_callback - INFO - Received callback for node 69409c51575bc060ce162585 INFO: 172.21.0.1:47648 - "POST /node/69409c51575bc060ce162585 HTTP/1.0" 202 Accepted 2025-12-17 23:40:09,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409897575bc060ce161af8 2025-12-17 23:40:09,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409897575bc060ce161af8 HTTP/1.1" 502 150 2025-12-17 23:40:09,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409897575bc060ce161af8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:09,654 - lava_callback - INFO - Received callback for node 69409c4b575bc060ce16257a INFO: 172.21.0.1:47660 - "POST /node/69409c4b575bc060ce16257a HTTP/1.0" 202 Accepted 2025-12-17 23:40:09,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a1575bc060ce161b0f 2025-12-17 23:40:09,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a1575bc060ce161b0f HTTP/1.1" 502 150 2025-12-17 23:40:09,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a1575bc060ce161b0f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:09,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409898575bc060ce161aff 2025-12-17 23:40:09,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409898575bc060ce161aff HTTP/1.1" 502 150 2025-12-17 23:40:09,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409898575bc060ce161aff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:09,996 - lava_callback - INFO - Received callback for node 69409c52575bc060ce162588 INFO: 172.21.0.1:38282 - "POST /node/69409c52575bc060ce162588 HTTP/1.0" 202 Accepted 2025-12-17 23:40:10,352 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409893575bc060ce161aec 2025-12-17 23:40:10,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409893575bc060ce161aec HTTP/1.1" 502 150 2025-12-17 23:40:10,353 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409893575bc060ce161aec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:10,643 - lava_callback - INFO - Received callback for node 69409c54575bc060ce16258b INFO: 172.21.0.1:38284 - "POST /node/69409c54575bc060ce16258b HTTP/1.0" 202 Accepted 2025-12-17 23:40:10,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409895575bc060ce161aef 2025-12-17 23:40:10,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409895575bc060ce161aef HTTP/1.1" 502 150 2025-12-17 23:40:10,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409895575bc060ce161aef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:12,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940989c575bc060ce161b05 2025-12-17 23:40:12,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989c575bc060ce161b05 HTTP/1.1" 502 150 2025-12-17 23:40:12,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940989c575bc060ce161b05'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:13,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941e655ecc59770562f0bb7 2025-12-17 23:40:13,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941e655ecc59770562f0bb7 HTTP/1.1" 502 150 2025-12-17 23:40:13,019 - lava_callback - ERROR - Error processing callback for node 6941e655ecc59770562f0bb7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941e655ecc59770562f0bb7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941e655ecc59770562f0bb7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941e655ecc59770562f0bb7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:13,021 - lava_callback - INFO - Starting async job submit for node 694098a3575bc060ce161b12 2025-12-17 23:40:13,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:13,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a3575bc060ce161b12 HTTP/1.1" 502 150 2025-12-17 23:40:13,067 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a3575bc060ce161b12'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:13,067 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a3575bc060ce161b12 2025-12-17 23:40:13,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a3575bc060ce161b12 HTTP/1.1" 502 150 2025-12-17 23:40:13,068 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a3575bc060ce161b12'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:13,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a1575bc060ce161b0f 2025-12-17 23:40:13,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a1575bc060ce161b0f HTTP/1.1" 502 150 2025-12-17 23:40:13,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a1575bc060ce161b0f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:14,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409887575bc060ce161ad0 2025-12-17 23:40:14,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409887575bc060ce161ad0 HTTP/1.1" 502 150 2025-12-17 23:40:14,544 - lava_callback - ERROR - Error processing callback for node 69409887575bc060ce161ad0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409887575bc060ce161ad0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409887575bc060ce161ad0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409887575bc060ce161ad0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:14,546 - lava_callback - INFO - Starting async job submit for node 694098a5575bc060ce161b1a 2025-12-17 23:40:14,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:14,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a5575bc060ce161b1a HTTP/1.1" 502 150 2025-12-17 23:40:14,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a5575bc060ce161b1a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:14,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a5575bc060ce161b1a 2025-12-17 23:40:14,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a5575bc060ce161b1a HTTP/1.1" 502 150 2025-12-17 23:40:14,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a5575bc060ce161b1a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:14,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409888575bc060ce161ad3 2025-12-17 23:40:14,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409888575bc060ce161ad3 HTTP/1.1" 502 150 2025-12-17 23:40:14,749 - lava_callback - ERROR - Error processing callback for node 69409888575bc060ce161ad3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409888575bc060ce161ad3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409888575bc060ce161ad3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409888575bc060ce161ad3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:14,751 - lava_callback - INFO - Starting async job submit for node 694098a7575bc060ce161b1d 2025-12-17 23:40:14,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:14,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a7575bc060ce161b1d HTTP/1.1" 502 150 2025-12-17 23:40:14,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a7575bc060ce161b1d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:14,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a7575bc060ce161b1d 2025-12-17 23:40:14,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a7575bc060ce161b1d HTTP/1.1" 502 150 2025-12-17 23:40:14,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a7575bc060ce161b1d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:15,068 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a3575bc060ce161b12 2025-12-17 23:40:15,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a3575bc060ce161b12 HTTP/1.1" 502 150 2025-12-17 23:40:15,069 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a3575bc060ce161b12'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38292 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:40:15,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988a575bc060ce161ad6 2025-12-17 23:40:15,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988a575bc060ce161ad6 HTTP/1.1" 502 150 2025-12-17 23:40:15,349 - lava_callback - ERROR - Error processing callback for node 6940988a575bc060ce161ad6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940988a575bc060ce161ad6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940988a575bc060ce161ad6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940988a575bc060ce161ad6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:15,351 - lava_callback - INFO - Starting async job submit for node 694098a9575bc060ce161b20 2025-12-17 23:40:15,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:15,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a9575bc060ce161b20 HTTP/1.1" 502 150 2025-12-17 23:40:15,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a9575bc060ce161b20'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:15,357 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a9575bc060ce161b20 2025-12-17 23:40:15,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a9575bc060ce161b20 HTTP/1.1" 502 150 2025-12-17 23:40:15,358 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a9575bc060ce161b20'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:16,406 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988c575bc060ce161add 2025-12-17 23:40:16,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988c575bc060ce161add HTTP/1.1" 502 150 2025-12-17 23:40:16,409 - lava_callback - ERROR - Error processing callback for node 6940988c575bc060ce161add: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940988c575bc060ce161add (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940988c575bc060ce161add (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940988c575bc060ce161add (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:16,411 - lava_callback - INFO - Starting async job submit for node 694098af575bc060ce161b36 2025-12-17 23:40:16,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:16,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098af575bc060ce161b36 HTTP/1.1" 502 150 2025-12-17 23:40:16,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098af575bc060ce161b36'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:16,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098af575bc060ce161b36 2025-12-17 23:40:16,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098af575bc060ce161b36 HTTP/1.1" 502 150 2025-12-17 23:40:16,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098af575bc060ce161b36'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38308 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:40:16,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a5575bc060ce161b1a 2025-12-17 23:40:16,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a5575bc060ce161b1a HTTP/1.1" 502 150 2025-12-17 23:40:16,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a5575bc060ce161b1a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:16,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a7575bc060ce161b1d 2025-12-17 23:40:16,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a7575bc060ce161b1d HTTP/1.1" 502 150 2025-12-17 23:40:16,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a7575bc060ce161b1d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:17,177 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940945f575bc060ce16100e 2025-12-17 23:40:17,358 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a9575bc060ce161b20 2025-12-17 23:40:17,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a9575bc060ce161b20 HTTP/1.1" 502 150 2025-12-17 23:40:17,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a9575bc060ce161b20'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:17,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989e575bc060ce161b08 HTTP/1.1" 502 150 2025-12-17 23:40:17,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940989e575bc060ce161b08'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:17,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409897575bc060ce161af8 2025-12-17 23:40:17,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409897575bc060ce161af8 HTTP/1.1" 502 150 2025-12-17 23:40:17,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409897575bc060ce161af8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:17,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940988d575bc060ce161ae0 2025-12-17 23:40:17,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940988d575bc060ce161ae0 HTTP/1.1" 502 150 2025-12-17 23:40:17,618 - lava_callback - ERROR - Error processing callback for node 6940988d575bc060ce161ae0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940988d575bc060ce161ae0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940988d575bc060ce161ae0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940988d575bc060ce161ae0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:17,620 - lava_callback - INFO - Starting async job submit for node 694098b1575bc060ce161b39 2025-12-17 23:40:17,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:17,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b1575bc060ce161b39 HTTP/1.1" 502 150 2025-12-17 23:40:17,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b1575bc060ce161b39'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:17,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b1575bc060ce161b39 2025-12-17 23:40:17,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b1575bc060ce161b39 HTTP/1.1" 502 150 2025-12-17 23:40:17,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b1575bc060ce161b39'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:17,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409890575bc060ce161ae6 2025-12-17 23:40:17,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409890575bc060ce161ae6 HTTP/1.1" 502 150 2025-12-17 23:40:17,634 - lava_callback - ERROR - Error processing callback for node 69409890575bc060ce161ae6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409890575bc060ce161ae6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409890575bc060ce161ae6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409890575bc060ce161ae6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:17,635 - lava_callback - INFO - Starting async job submit for node 694098ab575bc060ce161b28 2025-12-17 23:40:17,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:17,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ab575bc060ce161b28 HTTP/1.1" 502 150 2025-12-17 23:40:17,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ab575bc060ce161b28'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:17,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ab575bc060ce161b28 2025-12-17 23:40:17,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ab575bc060ce161b28 HTTP/1.1" 502 150 2025-12-17 23:40:17,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ab575bc060ce161b28'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:17,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409898575bc060ce161aff 2025-12-17 23:40:17,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409898575bc060ce161aff HTTP/1.1" 502 150 2025-12-17 23:40:17,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409898575bc060ce161aff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:18,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098af575bc060ce161b36 2025-12-17 23:40:18,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098af575bc060ce161b36 HTTP/1.1" 502 150 2025-12-17 23:40:18,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098af575bc060ce161b36'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:18,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409891575bc060ce161ae9 2025-12-17 23:40:18,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409891575bc060ce161ae9 HTTP/1.1" 502 150 2025-12-17 23:40:18,889 - lava_callback - ERROR - Error processing callback for node 69409891575bc060ce161ae9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409891575bc060ce161ae9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409891575bc060ce161ae9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409891575bc060ce161ae9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:18,891 - lava_callback - INFO - Starting async job submit for node 694098ad575bc060ce161b31 2025-12-17 23:40:18,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:18,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ad575bc060ce161b31 HTTP/1.1" 502 150 2025-12-17 23:40:18,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ad575bc060ce161b31'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:18,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ad575bc060ce161b31 2025-12-17 23:40:18,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ad575bc060ce161b31 HTTP/1.1" 502 150 2025-12-17 23:40:18,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ad575bc060ce161b31'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:19,070 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a3575bc060ce161b12 2025-12-17 23:40:19,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a3575bc060ce161b12 HTTP/1.1" 502 150 2025-12-17 23:40:19,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a3575bc060ce161b12'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:19,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b1575bc060ce161b39 2025-12-17 23:40:19,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b1575bc060ce161b39 HTTP/1.1" 502 150 2025-12-17 23:40:19,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b1575bc060ce161b39'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:19,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ab575bc060ce161b28 2025-12-17 23:40:19,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ab575bc060ce161b28 HTTP/1.1" 502 150 2025-12-17 23:40:19,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ab575bc060ce161b28'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:20,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940989c575bc060ce161b05 2025-12-17 23:40:20,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989c575bc060ce161b05 HTTP/1.1" 502 150 2025-12-17 23:40:20,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940989c575bc060ce161b05'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:20,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a5575bc060ce161b1a 2025-12-17 23:40:20,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a5575bc060ce161b1a HTTP/1.1" 502 150 2025-12-17 23:40:20,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a5575bc060ce161b1a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:20,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a7575bc060ce161b1d 2025-12-17 23:40:20,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a7575bc060ce161b1d HTTP/1.1" 502 150 2025-12-17 23:40:20,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a7575bc060ce161b1d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:20,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ad575bc060ce161b31 2025-12-17 23:40:20,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ad575bc060ce161b31 HTTP/1.1" 502 150 2025-12-17 23:40:20,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ad575bc060ce161b31'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:21,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a9575bc060ce161b20 2025-12-17 23:40:21,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a9575bc060ce161b20 HTTP/1.1" 502 150 2025-12-17 23:40:21,361 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a9575bc060ce161b20'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:21,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940989e575bc060ce161b08 2025-12-17 23:40:21,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989e575bc060ce161b08 HTTP/1.1" 502 150 2025-12-17 23:40:21,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940989e575bc060ce161b08'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:21,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a1575bc060ce161b0f 2025-12-17 23:40:21,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a1575bc060ce161b0f HTTP/1.1" 502 150 2025-12-17 23:40:21,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a1575bc060ce161b0f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:22,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098af575bc060ce161b36 2025-12-17 23:40:23,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b1575bc060ce161b39 2025-12-17 23:40:23,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b1575bc060ce161b39 HTTP/1.1" 502 150 2025-12-17 23:40:23,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b1575bc060ce161b39'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:23,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ab575bc060ce161b28 2025-12-17 23:40:23,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ab575bc060ce161b28 HTTP/1.1" 502 150 2025-12-17 23:40:23,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ab575bc060ce161b28'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:24,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ad575bc060ce161b31 2025-12-17 23:40:24,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ad575bc060ce161b31 HTTP/1.1" 502 150 2025-12-17 23:40:24,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ad575bc060ce161b31'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:26,353 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409893575bc060ce161aec 2025-12-17 23:40:26,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409893575bc060ce161aec HTTP/1.1" 502 150 2025-12-17 23:40:26,355 - lava_callback - ERROR - Error processing callback for node 69409893575bc060ce161aec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409893575bc060ce161aec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409893575bc060ce161aec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409893575bc060ce161aec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:26,356 - lava_callback - INFO - Starting async job submit for node 69409569575bc060ce16130c 2025-12-17 23:40:26,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:26,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409569575bc060ce16130c HTTP/1.1" 502 150 2025-12-17 23:40:26,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409569575bc060ce16130c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:26,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409569575bc060ce16130c 2025-12-17 23:40:26,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409569575bc060ce16130c HTTP/1.1" 502 150 2025-12-17 23:40:26,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409569575bc060ce16130c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51890 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:40:26,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409895575bc060ce161aef 2025-12-17 23:40:26,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409895575bc060ce161aef HTTP/1.1" 502 150 2025-12-17 23:40:26,913 - lava_callback - ERROR - Error processing callback for node 69409895575bc060ce161aef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409895575bc060ce161aef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409895575bc060ce161aef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409895575bc060ce161aef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:26,914 - lava_callback - INFO - Starting async job submit for node 694098b4575bc060ce161b3c 2025-12-17 23:40:26,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:26,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b4575bc060ce161b3c HTTP/1.1" 502 150 2025-12-17 23:40:26,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b4575bc060ce161b3c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:26,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b4575bc060ce161b3c 2025-12-17 23:40:26,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b4575bc060ce161b3c HTTP/1.1" 502 150 2025-12-17 23:40:26,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b4575bc060ce161b3c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:27,071 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a3575bc060ce161b12 2025-12-17 23:40:27,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a3575bc060ce161b12 HTTP/1.1" 502 150 2025-12-17 23:40:27,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a3575bc060ce161b12'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:28,071 - lava_callback - INFO - Received callback for node 69409c59575bc060ce16259b INFO: 172.21.0.1:51902 - "POST /node/69409c59575bc060ce16259b HTTP/1.0" 202 Accepted 2025-12-17 23:40:28,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409569575bc060ce16130c 2025-12-17 23:40:28,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409569575bc060ce16130c HTTP/1.1" 502 150 2025-12-17 23:40:28,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409569575bc060ce16130c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:28,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a5575bc060ce161b1a 2025-12-17 23:40:28,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a5575bc060ce161b1a HTTP/1.1" 502 150 2025-12-17 23:40:28,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a5575bc060ce161b1a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:28,734 - lava_callback - INFO - Received callback for node 69409c5b575bc060ce1625a2 INFO: 172.21.0.1:51912 - "POST /node/69409c5b575bc060ce1625a2 HTTP/1.0" 202 Accepted 2025-12-17 23:40:28,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a7575bc060ce161b1d 2025-12-17 23:40:28,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a7575bc060ce161b1d HTTP/1.1" 502 150 2025-12-17 23:40:28,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a7575bc060ce161b1d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:28,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b4575bc060ce161b3c 2025-12-17 23:40:28,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b4575bc060ce161b3c HTTP/1.1" 502 150 2025-12-17 23:40:28,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b4575bc060ce161b3c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:29,361 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a9575bc060ce161b20 2025-12-17 23:40:29,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a9575bc060ce161b20 HTTP/1.1" 502 150 2025-12-17 23:40:29,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098a9575bc060ce161b20'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:29,398 - lava_callback - INFO - Received callback for node 69409c5d575bc060ce1625a5 INFO: 172.21.0.1:51918 - "POST /node/69409c5d575bc060ce1625a5 HTTP/1.0" 202 Accepted 2025-12-17 23:40:29,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940989e575bc060ce161b08 2025-12-17 23:40:29,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989e575bc060ce161b08 HTTP/1.1" 502 150 2025-12-17 23:40:29,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940989e575bc060ce161b08'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:30,064 - lava_callback - INFO - Received callback for node 69409c63575bc060ce1625ac INFO: 172.21.0.1:36300 - "POST /node/69409c63575bc060ce1625ac HTTP/1.0" 202 Accepted 2025-12-17 23:40:31,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b1575bc060ce161b39 2025-12-17 23:40:31,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b1575bc060ce161b39 HTTP/1.1" 502 150 2025-12-17 23:40:31,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b1575bc060ce161b39'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:31,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ab575bc060ce161b28 2025-12-17 23:40:31,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ab575bc060ce161b28 HTTP/1.1" 502 150 2025-12-17 23:40:31,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ab575bc060ce161b28'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:32,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409569575bc060ce16130c 2025-12-17 23:40:32,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409569575bc060ce16130c HTTP/1.1" 502 150 2025-12-17 23:40:32,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409569575bc060ce16130c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:32,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940945f575bc060ce16100e HTTP/1.1" 502 150 2025-12-17 23:40:32,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940945f575bc060ce16100e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:32,850 - lava_callback - INFO - Received callback for node 69409c6a575bc060ce1625b4 INFO: 172.21.0.1:36308 - "POST /node/69409c6a575bc060ce1625b4 HTTP/1.0" 202 Accepted 2025-12-17 23:40:32,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ad575bc060ce161b31 2025-12-17 23:40:32,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ad575bc060ce161b31 HTTP/1.1" 502 150 2025-12-17 23:40:32,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ad575bc060ce161b31'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:32,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b4575bc060ce161b3c 2025-12-17 23:40:32,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b4575bc060ce161b3c HTTP/1.1" 502 150 2025-12-17 23:40:32,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b4575bc060ce161b3c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:33,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409897575bc060ce161af8 2025-12-17 23:40:33,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409897575bc060ce161af8 HTTP/1.1" 502 150 2025-12-17 23:40:33,532 - lava_callback - ERROR - Error processing callback for node 69409897575bc060ce161af8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409897575bc060ce161af8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409897575bc060ce161af8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409897575bc060ce161af8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:33,534 - lava_callback - INFO - Starting async job submit for node 694098b6575bc060ce161b3f 2025-12-17 23:40:33,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:33,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b6575bc060ce161b3f HTTP/1.1" 502 150 2025-12-17 23:40:33,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b6575bc060ce161b3f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:33,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b6575bc060ce161b3f 2025-12-17 23:40:33,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b6575bc060ce161b3f HTTP/1.1" 502 150 2025-12-17 23:40:33,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b6575bc060ce161b3f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:33,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409898575bc060ce161aff 2025-12-17 23:40:33,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409898575bc060ce161aff HTTP/1.1" 502 150 2025-12-17 23:40:33,867 - lava_callback - ERROR - Error processing callback for node 69409898575bc060ce161aff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409898575bc060ce161aff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409898575bc060ce161aff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409898575bc060ce161aff (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:33,869 - lava_callback - INFO - Starting async job submit for node 694098b8575bc060ce161b46 2025-12-17 23:40:33,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:33,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b8575bc060ce161b46 HTTP/1.1" 502 150 2025-12-17 23:40:33,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b8575bc060ce161b46'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:33,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b8575bc060ce161b46 2025-12-17 23:40:33,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b8575bc060ce161b46 HTTP/1.1" 502 150 2025-12-17 23:40:33,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b8575bc060ce161b46'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:35,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b6575bc060ce161b3f 2025-12-17 23:40:35,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b6575bc060ce161b3f HTTP/1.1" 502 150 2025-12-17 23:40:35,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b6575bc060ce161b3f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:35,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b8575bc060ce161b46 2025-12-17 23:40:35,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b8575bc060ce161b46 HTTP/1.1" 502 150 2025-12-17 23:40:35,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b8575bc060ce161b46'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:36,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940989c575bc060ce161b05 2025-12-17 23:40:36,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989c575bc060ce161b05 HTTP/1.1" 502 150 2025-12-17 23:40:36,465 - lava_callback - ERROR - Error processing callback for node 6940989c575bc060ce161b05: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940989c575bc060ce161b05 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940989c575bc060ce161b05 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940989c575bc060ce161b05 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:36,466 - lava_callback - INFO - Starting async job submit for node 694098ba575bc060ce161b49 2025-12-17 23:40:36,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:36,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ba575bc060ce161b49 HTTP/1.1" 502 150 2025-12-17 23:40:36,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ba575bc060ce161b49'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:36,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ba575bc060ce161b49 2025-12-17 23:40:36,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ba575bc060ce161b49 HTTP/1.1" 502 150 2025-12-17 23:40:36,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ba575bc060ce161b49'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36316 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:40:37,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098af575bc060ce161b36 HTTP/1.1" 502 150 2025-12-17 23:40:37,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098af575bc060ce161b36'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:37,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a1575bc060ce161b0f 2025-12-17 23:40:37,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a1575bc060ce161b0f HTTP/1.1" 502 150 2025-12-17 23:40:37,685 - lava_callback - ERROR - Error processing callback for node 694098a1575bc060ce161b0f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098a1575bc060ce161b0f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098a1575bc060ce161b0f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098a1575bc060ce161b0f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:37,687 - lava_callback - INFO - Starting async job submit for node 694098bc575bc060ce161b4c 2025-12-17 23:40:37,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:37,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098bc575bc060ce161b4c HTTP/1.1" 502 150 2025-12-17 23:40:37,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098bc575bc060ce161b4c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:37,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098bc575bc060ce161b4c 2025-12-17 23:40:37,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098bc575bc060ce161b4c HTTP/1.1" 502 150 2025-12-17 23:40:37,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098bc575bc060ce161b4c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:38,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ba575bc060ce161b49 2025-12-17 23:40:38,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ba575bc060ce161b49 HTTP/1.1" 502 150 2025-12-17 23:40:38,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ba575bc060ce161b49'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:39,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b6575bc060ce161b3f 2025-12-17 23:40:39,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b6575bc060ce161b3f HTTP/1.1" 502 150 2025-12-17 23:40:39,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b6575bc060ce161b3f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:39,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098bc575bc060ce161b4c 2025-12-17 23:40:39,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098bc575bc060ce161b4c HTTP/1.1" 502 150 2025-12-17 23:40:39,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098bc575bc060ce161b4c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:39,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b8575bc060ce161b46 2025-12-17 23:40:39,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b8575bc060ce161b46 HTTP/1.1" 502 150 2025-12-17 23:40:39,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b8575bc060ce161b46'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:40,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409569575bc060ce16130c 2025-12-17 23:40:40,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409569575bc060ce16130c HTTP/1.1" 502 150 2025-12-17 23:40:40,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409569575bc060ce16130c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:40,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b4575bc060ce161b3c 2025-12-17 23:40:40,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b4575bc060ce161b3c HTTP/1.1" 502 150 2025-12-17 23:40:40,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b4575bc060ce161b3c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:42,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ba575bc060ce161b49 2025-12-17 23:40:42,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ba575bc060ce161b49 HTTP/1.1" 502 150 2025-12-17 23:40:42,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ba575bc060ce161b49'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:43,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a3575bc060ce161b12 2025-12-17 23:40:43,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098bc575bc060ce161b4c 2025-12-17 23:40:43,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098bc575bc060ce161b4c HTTP/1.1" 502 150 2025-12-17 23:40:43,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098bc575bc060ce161b4c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:44,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a5575bc060ce161b1a 2025-12-17 23:40:44,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a5575bc060ce161b1a HTTP/1.1" 502 150 2025-12-17 23:40:44,582 - lava_callback - ERROR - Error processing callback for node 694098a5575bc060ce161b1a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098a5575bc060ce161b1a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098a5575bc060ce161b1a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098a5575bc060ce161b1a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:44,584 - lava_callback - INFO - Starting async job submit for node 694098c6575bc060ce161b5f 2025-12-17 23:40:44,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:44,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c6575bc060ce161b5f HTTP/1.1" 502 150 2025-12-17 23:40:44,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c6575bc060ce161b5f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:44,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c6575bc060ce161b5f 2025-12-17 23:40:44,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c6575bc060ce161b5f HTTP/1.1" 502 150 2025-12-17 23:40:44,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c6575bc060ce161b5f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:44,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a7575bc060ce161b1d 2025-12-17 23:40:44,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a7575bc060ce161b1d HTTP/1.1" 502 150 2025-12-17 23:40:44,763 - lava_callback - ERROR - Error processing callback for node 694098a7575bc060ce161b1d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098a7575bc060ce161b1d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098a7575bc060ce161b1d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098a7575bc060ce161b1d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:44,765 - lava_callback - INFO - Starting async job submit for node 694098c2575bc060ce161b55 2025-12-17 23:40:44,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:44,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c2575bc060ce161b55 HTTP/1.1" 502 150 2025-12-17 23:40:44,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c2575bc060ce161b55'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:44,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c2575bc060ce161b55 2025-12-17 23:40:44,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c2575bc060ce161b55 HTTP/1.1" 502 150 2025-12-17 23:40:44,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c2575bc060ce161b55'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:45,226 - lava_callback - INFO - Received callback for node 69409c48575bc060ce162571 INFO: 172.21.0.1:46810 - "POST /node/69409c48575bc060ce162571 HTTP/1.0" 202 Accepted 2025-12-17 23:40:45,363 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098a9575bc060ce161b20 2025-12-17 23:40:45,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a9575bc060ce161b20 HTTP/1.1" 502 150 2025-12-17 23:40:45,364 - lava_callback - ERROR - Error processing callback for node 694098a9575bc060ce161b20: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098a9575bc060ce161b20 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098a9575bc060ce161b20 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098a9575bc060ce161b20 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:45,366 - lava_callback - INFO - Starting async job submit for node 694098c8575bc060ce161b62 2025-12-17 23:40:45,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:45,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c8575bc060ce161b62 HTTP/1.1" 502 150 2025-12-17 23:40:45,373 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c8575bc060ce161b62'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:45,373 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c8575bc060ce161b62 2025-12-17 23:40:45,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c8575bc060ce161b62 HTTP/1.1" 502 150 2025-12-17 23:40:45,373 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c8575bc060ce161b62'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:45,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940989e575bc060ce161b08 2025-12-17 23:40:45,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940989e575bc060ce161b08 HTTP/1.1" 502 150 2025-12-17 23:40:45,460 - lava_callback - ERROR - Error processing callback for node 6940989e575bc060ce161b08: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940989e575bc060ce161b08 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940989e575bc060ce161b08 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940989e575bc060ce161b08 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:45,462 - lava_callback - INFO - Starting async job submit for node 694098cb575bc060ce161b69 2025-12-17 23:40:45,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:45,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098cb575bc060ce161b69 HTTP/1.1" 502 150 2025-12-17 23:40:45,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098cb575bc060ce161b69'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:45,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098cb575bc060ce161b69 2025-12-17 23:40:45,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098cb575bc060ce161b69 HTTP/1.1" 502 150 2025-12-17 23:40:45,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098cb575bc060ce161b69'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:45,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098af575bc060ce161b36 2025-12-17 23:40:45,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098af575bc060ce161b36 HTTP/1.1" 502 150 2025-12-17 23:40:45,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098af575bc060ce161b36'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46816 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:40:46,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c6575bc060ce161b5f 2025-12-17 23:40:46,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c6575bc060ce161b5f HTTP/1.1" 502 150 2025-12-17 23:40:46,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c6575bc060ce161b5f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:46,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c2575bc060ce161b55 2025-12-17 23:40:46,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c2575bc060ce161b55 HTTP/1.1" 502 150 2025-12-17 23:40:46,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c2575bc060ce161b55'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:47,374 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c8575bc060ce161b62 2025-12-17 23:40:47,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c8575bc060ce161b62 HTTP/1.1" 502 150 2025-12-17 23:40:47,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c8575bc060ce161b62'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:47,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098cb575bc060ce161b69 2025-12-17 23:40:47,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098cb575bc060ce161b69 HTTP/1.1" 502 150 2025-12-17 23:40:47,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098cb575bc060ce161b69'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:47,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b6575bc060ce161b3f 2025-12-17 23:40:47,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b6575bc060ce161b3f HTTP/1.1" 502 150 2025-12-17 23:40:47,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b6575bc060ce161b3f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:47,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b1575bc060ce161b39 2025-12-17 23:40:47,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b1575bc060ce161b39 HTTP/1.1" 502 150 2025-12-17 23:40:47,634 - lava_callback - ERROR - Error processing callback for node 694098b1575bc060ce161b39: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098b1575bc060ce161b39 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098b1575bc060ce161b39 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098b1575bc060ce161b39 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:47,636 - lava_callback - INFO - Starting async job submit for node 694098cc575bc060ce161b6c 2025-12-17 23:40:47,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:47,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098cc575bc060ce161b6c HTTP/1.1" 502 150 2025-12-17 23:40:47,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098cc575bc060ce161b6c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:47,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098cc575bc060ce161b6c 2025-12-17 23:40:47,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098cc575bc060ce161b6c HTTP/1.1" 502 150 2025-12-17 23:40:47,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098cc575bc060ce161b6c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:47,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ab575bc060ce161b28 2025-12-17 23:40:47,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ab575bc060ce161b28 HTTP/1.1" 502 150 2025-12-17 23:40:47,651 - lava_callback - ERROR - Error processing callback for node 694098ab575bc060ce161b28: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ab575bc060ce161b28 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ab575bc060ce161b28 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ab575bc060ce161b28 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:47,654 - lava_callback - INFO - Starting async job submit for node 694098ce575bc060ce161b6f 2025-12-17 23:40:47,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:47,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ce575bc060ce161b6f HTTP/1.1" 502 150 2025-12-17 23:40:47,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ce575bc060ce161b6f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:47,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ce575bc060ce161b6f 2025-12-17 23:40:47,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ce575bc060ce161b6f HTTP/1.1" 502 150 2025-12-17 23:40:47,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ce575bc060ce161b6f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:47,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b8575bc060ce161b46 2025-12-17 23:40:47,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b8575bc060ce161b46 HTTP/1.1" 502 150 2025-12-17 23:40:47,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098b8575bc060ce161b46'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:48,080 - lava_callback - INFO - Received callback for node 69409c6e575bc060ce1625c6 INFO: 172.21.0.1:46818 - "POST /node/69409c6e575bc060ce1625c6 HTTP/1.0" 202 Accepted 2025-12-17 23:40:48,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940945f575bc060ce16100e 2025-12-17 23:40:48,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940945f575bc060ce16100e HTTP/1.1" 502 150 2025-12-17 23:40:48,565 - lava_callback - ERROR - Error processing callback for node 6940945f575bc060ce16100e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940945f575bc060ce16100e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940945f575bc060ce16100e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940945f575bc060ce16100e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:48,567 - lava_callback - INFO - Starting async job submit for node 694098d0575bc060ce161b72 2025-12-17 23:40:48,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:48,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d0575bc060ce161b72 HTTP/1.1" 502 150 2025-12-17 23:40:48,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d0575bc060ce161b72'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:48,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d0575bc060ce161b72 2025-12-17 23:40:48,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d0575bc060ce161b72 HTTP/1.1" 502 150 2025-12-17 23:40:48,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d0575bc060ce161b72'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:48,743 - lava_callback - INFO - Received callback for node 69409c6f575bc060ce1625c9 INFO: 172.21.0.1:46834 - "POST /node/69409c6f575bc060ce1625c9 HTTP/1.0" 202 Accepted 2025-12-17 23:40:48,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ad575bc060ce161b31 2025-12-17 23:40:48,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ad575bc060ce161b31 HTTP/1.1" 502 150 2025-12-17 23:40:48,904 - lava_callback - ERROR - Error processing callback for node 694098ad575bc060ce161b31: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ad575bc060ce161b31 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ad575bc060ce161b31 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ad575bc060ce161b31 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:48,906 - lava_callback - INFO - Starting async job submit for node 694098d2575bc060ce161b75 2025-12-17 23:40:48,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:48,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d2575bc060ce161b75 HTTP/1.1" 502 150 2025-12-17 23:40:48,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d2575bc060ce161b75'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:48,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d2575bc060ce161b75 2025-12-17 23:40:48,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d2575bc060ce161b75 HTTP/1.1" 502 150 2025-12-17 23:40:48,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d2575bc060ce161b75'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:49,386 - lava_callback - INFO - Received callback for node 69409c71575bc060ce1625cc INFO: 172.21.0.1:46848 - "POST /node/69409c71575bc060ce1625cc HTTP/1.0" 202 Accepted 2025-12-17 23:40:49,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098cc575bc060ce161b6c 2025-12-17 23:40:49,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098cc575bc060ce161b6c HTTP/1.1" 502 150 2025-12-17 23:40:49,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098cc575bc060ce161b6c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:49,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ce575bc060ce161b6f 2025-12-17 23:40:49,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ce575bc060ce161b6f HTTP/1.1" 502 150 2025-12-17 23:40:49,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ce575bc060ce161b6f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:50,027 - lava_callback - INFO - Received callback for node 69409c72575bc060ce1625cf INFO: 172.21.0.1:34942 - "POST /node/69409c72575bc060ce1625cf HTTP/1.0" 202 Accepted 2025-12-17 23:40:50,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ba575bc060ce161b49 2025-12-17 23:40:50,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ba575bc060ce161b49 HTTP/1.1" 502 150 2025-12-17 23:40:50,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ba575bc060ce161b49'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:50,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d0575bc060ce161b72 2025-12-17 23:40:50,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d0575bc060ce161b72 HTTP/1.1" 502 150 2025-12-17 23:40:50,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d0575bc060ce161b72'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:50,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c6575bc060ce161b5f 2025-12-17 23:40:50,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c6575bc060ce161b5f HTTP/1.1" 502 150 2025-12-17 23:40:50,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c6575bc060ce161b5f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:50,674 - lava_callback - INFO - Received callback for node 69409c74575bc060ce1625d2 INFO: 172.21.0.1:34952 - "POST /node/69409c74575bc060ce1625d2 HTTP/1.0" 202 Accepted 2025-12-17 23:40:50,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c2575bc060ce161b55 2025-12-17 23:40:50,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c2575bc060ce161b55 HTTP/1.1" 502 150 2025-12-17 23:40:50,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c2575bc060ce161b55'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:50,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d2575bc060ce161b75 2025-12-17 23:40:50,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d2575bc060ce161b75 HTTP/1.1" 502 150 2025-12-17 23:40:50,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d2575bc060ce161b75'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:51,348 - lava_callback - INFO - Received callback for node 69409c76575bc060ce1625d5 INFO: 172.21.0.1:34956 - "POST /node/69409c76575bc060ce1625d5 HTTP/1.0" 202 Accepted 2025-12-17 23:40:51,375 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c8575bc060ce161b62 2025-12-17 23:40:51,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c8575bc060ce161b62 HTTP/1.1" 502 150 2025-12-17 23:40:51,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c8575bc060ce161b62'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:51,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098cb575bc060ce161b69 2025-12-17 23:40:51,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098cb575bc060ce161b69 HTTP/1.1" 502 150 2025-12-17 23:40:51,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098cb575bc060ce161b69'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:51,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098bc575bc060ce161b4c 2025-12-17 23:40:51,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098bc575bc060ce161b4c HTTP/1.1" 502 150 2025-12-17 23:40:51,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098bc575bc060ce161b4c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:52,002 - lava_callback - INFO - Received callback for node 69409c7c575bc060ce1625e1 INFO: 172.21.0.1:34968 - "POST /node/69409c7c575bc060ce1625e1 HTTP/1.0" 202 Accepted 2025-12-17 23:40:52,652 - lava_callback - INFO - Received callback for node 69409c7e575bc060ce1625e6 INFO: 172.21.0.1:34978 - "POST /node/69409c7e575bc060ce1625e6 HTTP/1.0" 202 Accepted 2025-12-17 23:40:53,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098cc575bc060ce161b6c 2025-12-17 23:40:53,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098cc575bc060ce161b6c HTTP/1.1" 502 150 2025-12-17 23:40:53,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098cc575bc060ce161b6c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:53,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ce575bc060ce161b6f 2025-12-17 23:40:53,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ce575bc060ce161b6f HTTP/1.1" 502 150 2025-12-17 23:40:53,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ce575bc060ce161b6f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:54,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d0575bc060ce161b72 2025-12-17 23:40:54,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d0575bc060ce161b72 HTTP/1.1" 502 150 2025-12-17 23:40:54,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d0575bc060ce161b72'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:54,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d2575bc060ce161b75 2025-12-17 23:40:54,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d2575bc060ce161b75 HTTP/1.1" 502 150 2025-12-17 23:40:54,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d2575bc060ce161b75'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:56,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409569575bc060ce16130c 2025-12-17 23:40:56,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409569575bc060ce16130c HTTP/1.1" 502 150 2025-12-17 23:40:56,418 - lava_callback - ERROR - Error processing callback for node 69409569575bc060ce16130c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409569575bc060ce16130c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409569575bc060ce16130c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409569575bc060ce16130c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:56,420 - lava_callback - INFO - Starting async job submit for node 694098d3575bc060ce161b7e 2025-12-17 23:40:56,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:56,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d3575bc060ce161b7e HTTP/1.1" 502 150 2025-12-17 23:40:56,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d3575bc060ce161b7e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:56,427 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d3575bc060ce161b7e 2025-12-17 23:40:56,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d3575bc060ce161b7e HTTP/1.1" 502 150 2025-12-17 23:40:56,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d3575bc060ce161b7e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34982 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:40:56,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b4575bc060ce161b3c 2025-12-17 23:40:56,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b4575bc060ce161b3c HTTP/1.1" 502 150 2025-12-17 23:40:56,928 - lava_callback - ERROR - Error processing callback for node 694098b4575bc060ce161b3c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098b4575bc060ce161b3c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098b4575bc060ce161b3c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098b4575bc060ce161b3c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:56,929 - lava_callback - INFO - Starting async job submit for node 694098d6575bc060ce161b85 2025-12-17 23:40:56,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:56,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d6575bc060ce161b85 HTTP/1.1" 502 150 2025-12-17 23:40:56,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d6575bc060ce161b85'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:56,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d6575bc060ce161b85 2025-12-17 23:40:56,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d6575bc060ce161b85 HTTP/1.1" 502 150 2025-12-17 23:40:56,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d6575bc060ce161b85'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:58,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098a3575bc060ce161b12 HTTP/1.1" 502 150 2025-12-17 23:40:58,417 - lava_callback - ERROR - Error processing callback for node 694098a3575bc060ce161b12: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098a3575bc060ce161b12 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098a3575bc060ce161b12 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098a3575bc060ce161b12 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:40:58,419 - lava_callback - INFO - Starting async job submit for node 694098d8575bc060ce161b88 2025-12-17 23:40:58,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:40:58,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d8575bc060ce161b88 HTTP/1.1" 502 150 2025-12-17 23:40:58,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d8575bc060ce161b88'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:58,427 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d8575bc060ce161b88 2025-12-17 23:40:58,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d8575bc060ce161b88 HTTP/1.1" 502 150 2025-12-17 23:40:58,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d8575bc060ce161b88'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:58,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d3575bc060ce161b7e 2025-12-17 23:40:58,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d3575bc060ce161b7e HTTP/1.1" 502 150 2025-12-17 23:40:58,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d3575bc060ce161b7e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:58,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c6575bc060ce161b5f 2025-12-17 23:40:58,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c6575bc060ce161b5f HTTP/1.1" 502 150 2025-12-17 23:40:58,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c6575bc060ce161b5f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:58,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c2575bc060ce161b55 2025-12-17 23:40:58,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c2575bc060ce161b55 HTTP/1.1" 502 150 2025-12-17 23:40:58,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c2575bc060ce161b55'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:58,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d6575bc060ce161b85 2025-12-17 23:40:58,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d6575bc060ce161b85 HTTP/1.1" 502 150 2025-12-17 23:40:58,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d6575bc060ce161b85'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:59,377 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c8575bc060ce161b62 2025-12-17 23:40:59,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c8575bc060ce161b62 HTTP/1.1" 502 150 2025-12-17 23:40:59,378 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c8575bc060ce161b62'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:40:59,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098cb575bc060ce161b69 2025-12-17 23:40:59,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098cb575bc060ce161b69 HTTP/1.1" 502 150 2025-12-17 23:40:59,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098cb575bc060ce161b69'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:00,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d8575bc060ce161b88 2025-12-17 23:41:00,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d8575bc060ce161b88 HTTP/1.1" 502 150 2025-12-17 23:41:00,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d8575bc060ce161b88'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:01,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098cc575bc060ce161b6c 2025-12-17 23:41:01,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ce575bc060ce161b6f 2025-12-17 23:41:01,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ce575bc060ce161b6f HTTP/1.1" 502 150 2025-12-17 23:41:01,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ce575bc060ce161b6f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:01,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098af575bc060ce161b36 2025-12-17 23:41:01,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098af575bc060ce161b36 HTTP/1.1" 502 150 2025-12-17 23:41:01,683 - lava_callback - ERROR - Error processing callback for node 694098af575bc060ce161b36: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098af575bc060ce161b36 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098af575bc060ce161b36 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098af575bc060ce161b36 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:01,685 - lava_callback - INFO - Starting async job submit for node 694098da575bc060ce161b8f 2025-12-17 23:41:01,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:01,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098da575bc060ce161b8f HTTP/1.1" 502 150 2025-12-17 23:41:01,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098da575bc060ce161b8f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:01,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098da575bc060ce161b8f 2025-12-17 23:41:01,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098da575bc060ce161b8f HTTP/1.1" 502 150 2025-12-17 23:41:01,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098da575bc060ce161b8f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:02,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d3575bc060ce161b7e 2025-12-17 23:41:02,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d3575bc060ce161b7e HTTP/1.1" 502 150 2025-12-17 23:41:02,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d3575bc060ce161b7e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:02,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d0575bc060ce161b72 2025-12-17 23:41:02,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d0575bc060ce161b72 HTTP/1.1" 502 150 2025-12-17 23:41:02,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d0575bc060ce161b72'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:02,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d2575bc060ce161b75 2025-12-17 23:41:02,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d2575bc060ce161b75 HTTP/1.1" 502 150 2025-12-17 23:41:02,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d2575bc060ce161b75'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:02,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d6575bc060ce161b85 2025-12-17 23:41:02,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d6575bc060ce161b85 HTTP/1.1" 502 150 2025-12-17 23:41:02,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d6575bc060ce161b85'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36688 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:41:03,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b6575bc060ce161b3f 2025-12-17 23:41:03,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b6575bc060ce161b3f HTTP/1.1" 502 150 2025-12-17 23:41:03,559 - lava_callback - ERROR - Error processing callback for node 694098b6575bc060ce161b3f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098b6575bc060ce161b3f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098b6575bc060ce161b3f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098b6575bc060ce161b3f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:03,561 - lava_callback - INFO - Starting async job submit for node 694098dc575bc060ce161b92 2025-12-17 23:41:03,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:03,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098dc575bc060ce161b92 HTTP/1.1" 502 150 2025-12-17 23:41:03,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098dc575bc060ce161b92'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:03,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098dc575bc060ce161b92 2025-12-17 23:41:03,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098dc575bc060ce161b92 HTTP/1.1" 502 150 2025-12-17 23:41:03,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098dc575bc060ce161b92'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:03,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098da575bc060ce161b8f 2025-12-17 23:41:03,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098da575bc060ce161b8f HTTP/1.1" 502 150 2025-12-17 23:41:03,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098da575bc060ce161b8f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:03,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098b8575bc060ce161b46 2025-12-17 23:41:03,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098b8575bc060ce161b46 HTTP/1.1" 502 150 2025-12-17 23:41:03,882 - lava_callback - ERROR - Error processing callback for node 694098b8575bc060ce161b46: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098b8575bc060ce161b46 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098b8575bc060ce161b46 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098b8575bc060ce161b46 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:03,884 - lava_callback - INFO - Starting async job submit for node 694098de575bc060ce161b95 2025-12-17 23:41:03,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:03,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098de575bc060ce161b95 HTTP/1.1" 502 150 2025-12-17 23:41:03,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098de575bc060ce161b95'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:03,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098de575bc060ce161b95 2025-12-17 23:41:03,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098de575bc060ce161b95 HTTP/1.1" 502 150 2025-12-17 23:41:03,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098de575bc060ce161b95'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:04,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d8575bc060ce161b88 2025-12-17 23:41:04,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d8575bc060ce161b88 HTTP/1.1" 502 150 2025-12-17 23:41:04,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d8575bc060ce161b88'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:05,466 - lava_callback - INFO - Received callback for node 69409c4a575bc060ce162577 INFO: 172.21.0.1:36702 - "POST /node/69409c4a575bc060ce162577 HTTP/1.0" 202 Accepted 2025-12-17 23:41:05,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098dc575bc060ce161b92 2025-12-17 23:41:05,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098dc575bc060ce161b92 HTTP/1.1" 502 150 2025-12-17 23:41:05,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098dc575bc060ce161b92'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:05,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098de575bc060ce161b95 2025-12-17 23:41:05,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098de575bc060ce161b95 HTTP/1.1" 502 150 2025-12-17 23:41:05,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098de575bc060ce161b95'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:06,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ba575bc060ce161b49 2025-12-17 23:41:06,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ba575bc060ce161b49 HTTP/1.1" 502 150 2025-12-17 23:41:06,484 - lava_callback - ERROR - Error processing callback for node 694098ba575bc060ce161b49: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ba575bc060ce161b49 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ba575bc060ce161b49 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ba575bc060ce161b49 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:06,486 - lava_callback - INFO - Starting async job submit for node 69409a86575bc060ce161fc5 INFO: 172.21.0.1:36718 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:41:06,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:06,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a86575bc060ce161fc5 HTTP/1.1" 502 150 2025-12-17 23:41:06,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a86575bc060ce161fc5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:06,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a86575bc060ce161fc5 2025-12-17 23:41:06,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a86575bc060ce161fc5 HTTP/1.1" 502 150 2025-12-17 23:41:06,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a86575bc060ce161fc5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:07,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098da575bc060ce161b8f 2025-12-17 23:41:07,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098da575bc060ce161b8f HTTP/1.1" 502 150 2025-12-17 23:41:07,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098da575bc060ce161b8f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:07,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098bc575bc060ce161b4c 2025-12-17 23:41:07,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098bc575bc060ce161b4c HTTP/1.1" 502 150 2025-12-17 23:41:07,708 - lava_callback - ERROR - Error processing callback for node 694098bc575bc060ce161b4c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098bc575bc060ce161b4c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098bc575bc060ce161b4c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098bc575bc060ce161b4c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:07,710 - lava_callback - INFO - Starting async job submit for node 69409c78575bc060ce1625d8 2025-12-17 23:41:07,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:07,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c78575bc060ce1625d8 HTTP/1.1" 502 150 2025-12-17 23:41:07,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c78575bc060ce1625d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:07,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c78575bc060ce1625d8 2025-12-17 23:41:07,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c78575bc060ce1625d8 HTTP/1.1" 502 150 2025-12-17 23:41:07,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c78575bc060ce1625d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:08,110 - lava_callback - INFO - Received callback for node 69409c81575bc060ce1625ed INFO: 172.21.0.1:36726 - "POST /node/69409c81575bc060ce1625ed HTTP/1.0" 202 Accepted 2025-12-17 23:41:08,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a86575bc060ce161fc5 2025-12-17 23:41:08,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a86575bc060ce161fc5 HTTP/1.1" 502 150 2025-12-17 23:41:08,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a86575bc060ce161fc5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:08,760 - lava_callback - INFO - Received callback for node 69409c83575bc060ce1625f0 INFO: 172.21.0.1:36734 - "POST /node/69409c83575bc060ce1625f0 HTTP/1.0" 202 Accepted 2025-12-17 23:41:09,414 - lava_callback - INFO - Received callback for node 69409c85575bc060ce1625f7 INFO: 172.21.0.1:36748 - "POST /node/69409c85575bc060ce1625f7 HTTP/1.0" 202 Accepted 2025-12-17 23:41:09,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098dc575bc060ce161b92 2025-12-17 23:41:09,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098dc575bc060ce161b92 HTTP/1.1" 502 150 2025-12-17 23:41:09,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098dc575bc060ce161b92'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:09,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c78575bc060ce1625d8 2025-12-17 23:41:09,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c78575bc060ce1625d8 HTTP/1.1" 502 150 2025-12-17 23:41:09,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c78575bc060ce1625d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:09,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098de575bc060ce161b95 2025-12-17 23:41:09,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098de575bc060ce161b95 HTTP/1.1" 502 150 2025-12-17 23:41:09,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098de575bc060ce161b95'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:10,065 - lava_callback - INFO - Received callback for node 69409c87575bc060ce1625fa INFO: 172.21.0.1:59990 - "POST /node/69409c87575bc060ce1625fa HTTP/1.0" 202 Accepted 2025-12-17 23:41:10,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d3575bc060ce161b7e 2025-12-17 23:41:10,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d3575bc060ce161b7e HTTP/1.1" 502 150 2025-12-17 23:41:10,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d3575bc060ce161b7e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:10,713 - lava_callback - INFO - Received callback for node 69409c89575bc060ce1625fd INFO: 172.21.0.1:60002 - "POST /node/69409c89575bc060ce1625fd HTTP/1.0" 202 Accepted 2025-12-17 23:41:10,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d6575bc060ce161b85 2025-12-17 23:41:10,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d6575bc060ce161b85 HTTP/1.1" 502 150 2025-12-17 23:41:10,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d6575bc060ce161b85'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:11,359 - lava_callback - INFO - Received callback for node 69409c8b575bc060ce162600 INFO: 172.21.0.1:60012 - "POST /node/69409c8b575bc060ce162600 HTTP/1.0" 202 Accepted 2025-12-17 23:41:12,002 - lava_callback - INFO - Received callback for node 69409c8d575bc060ce162606 INFO: 172.21.0.1:60024 - "POST /node/69409c8d575bc060ce162606 HTTP/1.0" 202 Accepted 2025-12-17 23:41:12,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d8575bc060ce161b88 2025-12-17 23:41:12,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a86575bc060ce161fc5 2025-12-17 23:41:12,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a86575bc060ce161fc5 HTTP/1.1" 502 150 2025-12-17 23:41:12,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a86575bc060ce161fc5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:12,670 - lava_callback - INFO - Received callback for node 69409c8e575bc060ce162609 INFO: 172.21.0.1:60036 - "POST /node/69409c8e575bc060ce162609 HTTP/1.0" 202 Accepted 2025-12-17 23:41:13,314 - lava_callback - INFO - Received callback for node 69409c90575bc060ce16260c INFO: 172.21.0.1:60044 - "POST /node/69409c90575bc060ce16260c HTTP/1.0" 202 Accepted 2025-12-17 23:41:13,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c78575bc060ce1625d8 2025-12-17 23:41:13,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c78575bc060ce1625d8 HTTP/1.1" 502 150 2025-12-17 23:41:13,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c78575bc060ce1625d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:13,973 - lava_callback - INFO - Received callback for node 69409c92575bc060ce16260f INFO: 172.21.0.1:60048 - "POST /node/69409c92575bc060ce16260f HTTP/1.0" 202 Accepted 2025-12-17 23:41:14,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c6575bc060ce161b5f 2025-12-17 23:41:14,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c6575bc060ce161b5f HTTP/1.1" 502 150 2025-12-17 23:41:14,616 - lava_callback - ERROR - Error processing callback for node 694098c6575bc060ce161b5f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c6575bc060ce161b5f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c6575bc060ce161b5f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c6575bc060ce161b5f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:14,619 - lava_callback - INFO - Starting async job submit for node 694098e6575bc060ce161ba0 2025-12-17 23:41:14,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:14,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e6575bc060ce161ba0 HTTP/1.1" 502 150 2025-12-17 23:41:14,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e6575bc060ce161ba0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:14,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e6575bc060ce161ba0 2025-12-17 23:41:14,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e6575bc060ce161ba0 HTTP/1.1" 502 150 2025-12-17 23:41:14,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e6575bc060ce161ba0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:14,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c2575bc060ce161b55 2025-12-17 23:41:14,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c2575bc060ce161b55 HTTP/1.1" 502 150 2025-12-17 23:41:14,784 - lava_callback - ERROR - Error processing callback for node 694098c2575bc060ce161b55: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c2575bc060ce161b55 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c2575bc060ce161b55 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c2575bc060ce161b55 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:14,787 - lava_callback - INFO - Starting async job submit for node 694098e9575bc060ce161bab 2025-12-17 23:41:14,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:14,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e9575bc060ce161bab HTTP/1.1" 502 150 2025-12-17 23:41:14,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e9575bc060ce161bab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:14,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e9575bc060ce161bab 2025-12-17 23:41:14,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e9575bc060ce161bab HTTP/1.1" 502 150 2025-12-17 23:41:14,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e9575bc060ce161bab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60052 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:41:15,378 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c8575bc060ce161b62 2025-12-17 23:41:15,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c8575bc060ce161b62 HTTP/1.1" 502 150 2025-12-17 23:41:15,379 - lava_callback - ERROR - Error processing callback for node 694098c8575bc060ce161b62: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c8575bc060ce161b62 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c8575bc060ce161b62 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c8575bc060ce161b62 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:15,381 - lava_callback - INFO - Starting async job submit for node 694098eb575bc060ce161bb6 2025-12-17 23:41:15,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:15,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098eb575bc060ce161bb6 HTTP/1.1" 502 150 2025-12-17 23:41:15,389 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098eb575bc060ce161bb6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:15,389 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098eb575bc060ce161bb6 2025-12-17 23:41:15,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098eb575bc060ce161bb6 HTTP/1.1" 502 150 2025-12-17 23:41:15,390 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098eb575bc060ce161bb6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:15,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098cb575bc060ce161b69 2025-12-17 23:41:15,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098cb575bc060ce161b69 HTTP/1.1" 502 150 2025-12-17 23:41:15,476 - lava_callback - ERROR - Error processing callback for node 694098cb575bc060ce161b69: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098cb575bc060ce161b69 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098cb575bc060ce161b69 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098cb575bc060ce161b69 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:15,477 - lava_callback - INFO - Starting async job submit for node 694098ed575bc060ce161bb9 2025-12-17 23:41:15,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:15,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ed575bc060ce161bb9 HTTP/1.1" 502 150 2025-12-17 23:41:15,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ed575bc060ce161bb9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:15,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ed575bc060ce161bb9 2025-12-17 23:41:15,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ed575bc060ce161bb9 HTTP/1.1" 502 150 2025-12-17 23:41:15,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ed575bc060ce161bb9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:15,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098da575bc060ce161b8f 2025-12-17 23:41:15,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098da575bc060ce161b8f HTTP/1.1" 502 150 2025-12-17 23:41:15,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098da575bc060ce161b8f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60066 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:41:16,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e6575bc060ce161ba0 2025-12-17 23:41:16,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e6575bc060ce161ba0 HTTP/1.1" 502 150 2025-12-17 23:41:16,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e6575bc060ce161ba0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:16,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e9575bc060ce161bab 2025-12-17 23:41:16,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e9575bc060ce161bab HTTP/1.1" 502 150 2025-12-17 23:41:16,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e9575bc060ce161bab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:17,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098cc575bc060ce161b6c HTTP/1.1" 502 150 2025-12-17 23:41:17,104 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098cc575bc060ce161b6c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:17,390 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098eb575bc060ce161bb6 2025-12-17 23:41:17,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098eb575bc060ce161bb6 HTTP/1.1" 502 150 2025-12-17 23:41:17,391 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098eb575bc060ce161bb6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:17,454 - lava_callback - INFO - Received callback for node 69409c98575bc060ce162629 INFO: 172.21.0.1:60068 - "POST /node/69409c98575bc060ce162629 HTTP/1.0" 202 Accepted 2025-12-17 23:41:17,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ed575bc060ce161bb9 2025-12-17 23:41:17,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ed575bc060ce161bb9 HTTP/1.1" 502 150 2025-12-17 23:41:17,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ed575bc060ce161bb9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:17,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098dc575bc060ce161b92 2025-12-17 23:41:17,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098dc575bc060ce161b92 HTTP/1.1" 502 150 2025-12-17 23:41:17,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098dc575bc060ce161b92'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:17,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ce575bc060ce161b6f 2025-12-17 23:41:17,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ce575bc060ce161b6f HTTP/1.1" 502 150 2025-12-17 23:41:17,669 - lava_callback - ERROR - Error processing callback for node 694098ce575bc060ce161b6f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ce575bc060ce161b6f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ce575bc060ce161b6f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ce575bc060ce161b6f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:17,671 - lava_callback - INFO - Starting async job submit for node 694098ef575bc060ce161bbc 2025-12-17 23:41:17,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:17,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ef575bc060ce161bbc HTTP/1.1" 502 150 2025-12-17 23:41:17,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ef575bc060ce161bbc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:17,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ef575bc060ce161bbc 2025-12-17 23:41:17,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ef575bc060ce161bbc HTTP/1.1" 502 150 2025-12-17 23:41:17,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ef575bc060ce161bbc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:17,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098de575bc060ce161b95 2025-12-17 23:41:17,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098de575bc060ce161b95 HTTP/1.1" 502 150 2025-12-17 23:41:17,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098de575bc060ce161b95'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:18,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d0575bc060ce161b72 2025-12-17 23:41:18,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d0575bc060ce161b72 HTTP/1.1" 502 150 2025-12-17 23:41:18,584 - lava_callback - ERROR - Error processing callback for node 694098d0575bc060ce161b72: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098d0575bc060ce161b72 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098d0575bc060ce161b72 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098d0575bc060ce161b72 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:18,586 - lava_callback - INFO - Starting async job submit for node 694098f1575bc060ce161bbf 2025-12-17 23:41:18,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:18,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f1575bc060ce161bbf HTTP/1.1" 502 150 2025-12-17 23:41:18,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f1575bc060ce161bbf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:18,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f1575bc060ce161bbf 2025-12-17 23:41:18,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f1575bc060ce161bbf HTTP/1.1" 502 150 2025-12-17 23:41:18,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f1575bc060ce161bbf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:18,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d2575bc060ce161b75 2025-12-17 23:41:18,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d2575bc060ce161b75 HTTP/1.1" 502 150 2025-12-17 23:41:18,922 - lava_callback - ERROR - Error processing callback for node 694098d2575bc060ce161b75: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098d2575bc060ce161b75 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098d2575bc060ce161b75 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098d2575bc060ce161b75 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:18,925 - lava_callback - INFO - Starting async job submit for node 694098f3575bc060ce161bc2 2025-12-17 23:41:18,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:18,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f3575bc060ce161bc2 HTTP/1.1" 502 150 2025-12-17 23:41:18,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f3575bc060ce161bc2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:18,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f3575bc060ce161bc2 2025-12-17 23:41:18,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f3575bc060ce161bc2 HTTP/1.1" 502 150 2025-12-17 23:41:18,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f3575bc060ce161bc2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:19,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ef575bc060ce161bbc 2025-12-17 23:41:19,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ef575bc060ce161bbc HTTP/1.1" 502 150 2025-12-17 23:41:19,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ef575bc060ce161bbc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:20,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a86575bc060ce161fc5 2025-12-17 23:41:20,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a86575bc060ce161fc5 HTTP/1.1" 502 150 2025-12-17 23:41:20,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a86575bc060ce161fc5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:20,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f1575bc060ce161bbf 2025-12-17 23:41:20,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f1575bc060ce161bbf HTTP/1.1" 502 150 2025-12-17 23:41:20,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f1575bc060ce161bbf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:20,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e6575bc060ce161ba0 2025-12-17 23:41:20,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e6575bc060ce161ba0 HTTP/1.1" 502 150 2025-12-17 23:41:20,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e6575bc060ce161ba0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:20,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e9575bc060ce161bab 2025-12-17 23:41:20,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e9575bc060ce161bab HTTP/1.1" 502 150 2025-12-17 23:41:20,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e9575bc060ce161bab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:20,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f3575bc060ce161bc2 2025-12-17 23:41:20,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f3575bc060ce161bc2 HTTP/1.1" 502 150 2025-12-17 23:41:20,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f3575bc060ce161bc2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:21,391 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098eb575bc060ce161bb6 2025-12-17 23:41:21,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098eb575bc060ce161bb6 HTTP/1.1" 502 150 2025-12-17 23:41:21,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098eb575bc060ce161bb6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:21,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ed575bc060ce161bb9 2025-12-17 23:41:21,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ed575bc060ce161bb9 HTTP/1.1" 502 150 2025-12-17 23:41:21,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ed575bc060ce161bb9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:21,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c78575bc060ce1625d8 2025-12-17 23:41:21,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c78575bc060ce1625d8 HTTP/1.1" 502 150 2025-12-17 23:41:21,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c78575bc060ce1625d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:23,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ef575bc060ce161bbc 2025-12-17 23:41:23,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ef575bc060ce161bbc HTTP/1.1" 502 150 2025-12-17 23:41:23,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ef575bc060ce161bbc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:24,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f1575bc060ce161bbf 2025-12-17 23:41:24,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f1575bc060ce161bbf HTTP/1.1" 502 150 2025-12-17 23:41:24,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f1575bc060ce161bbf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:24,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f3575bc060ce161bc2 2025-12-17 23:41:24,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f3575bc060ce161bc2 HTTP/1.1" 502 150 2025-12-17 23:41:24,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f3575bc060ce161bc2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:26,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d3575bc060ce161b7e 2025-12-17 23:41:26,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d3575bc060ce161b7e HTTP/1.1" 502 150 2025-12-17 23:41:26,433 - lava_callback - ERROR - Error processing callback for node 694098d3575bc060ce161b7e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098d3575bc060ce161b7e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098d3575bc060ce161b7e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098d3575bc060ce161b7e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:26,435 - lava_callback - INFO - Starting async job submit for node 694098f5575bc060ce161bc5 2025-12-17 23:41:26,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:26,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f5575bc060ce161bc5 HTTP/1.1" 502 150 2025-12-17 23:41:26,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f5575bc060ce161bc5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:26,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f5575bc060ce161bc5 2025-12-17 23:41:26,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f5575bc060ce161bc5 HTTP/1.1" 502 150 2025-12-17 23:41:26,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f5575bc060ce161bc5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48100 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:41:26,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d6575bc060ce161b85 2025-12-17 23:41:26,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d6575bc060ce161b85 HTTP/1.1" 502 150 2025-12-17 23:41:26,941 - lava_callback - ERROR - Error processing callback for node 694098d6575bc060ce161b85: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098d6575bc060ce161b85 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098d6575bc060ce161b85 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098d6575bc060ce161b85 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:26,943 - lava_callback - INFO - Starting async job submit for node 694098f7575bc060ce161bc8 2025-12-17 23:41:26,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:26,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f7575bc060ce161bc8 HTTP/1.1" 502 150 2025-12-17 23:41:26,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f7575bc060ce161bc8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:26,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f7575bc060ce161bc8 2025-12-17 23:41:26,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f7575bc060ce161bc8 HTTP/1.1" 502 150 2025-12-17 23:41:26,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f7575bc060ce161bc8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:27,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d8575bc060ce161b88 HTTP/1.1" 502 150 2025-12-17 23:41:27,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098d8575bc060ce161b88'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:28,097 - lava_callback - INFO - Received callback for node 69409c94575bc060ce16261c INFO: 172.21.0.1:48104 - "POST /node/69409c94575bc060ce16261c HTTP/1.0" 202 Accepted 2025-12-17 23:41:28,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f5575bc060ce161bc5 2025-12-17 23:41:28,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f5575bc060ce161bc5 HTTP/1.1" 502 150 2025-12-17 23:41:28,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f5575bc060ce161bc5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:28,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e6575bc060ce161ba0 2025-12-17 23:41:28,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e6575bc060ce161ba0 HTTP/1.1" 502 150 2025-12-17 23:41:28,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e6575bc060ce161ba0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:28,752 - lava_callback - INFO - Received callback for node 69409c96575bc060ce162623 INFO: 172.21.0.1:48106 - "POST /node/69409c96575bc060ce162623 HTTP/1.0" 202 Accepted 2025-12-17 23:41:28,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e9575bc060ce161bab 2025-12-17 23:41:28,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e9575bc060ce161bab HTTP/1.1" 502 150 2025-12-17 23:41:28,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e9575bc060ce161bab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:28,924 - lava_callback - INFO - Received callback for node 69409c9b575bc060ce16262f INFO: 172.21.0.1:48120 - "POST /node/69409c9b575bc060ce16262f HTTP/1.0" 202 Accepted 2025-12-17 23:41:28,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f7575bc060ce161bc8 2025-12-17 23:41:28,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f7575bc060ce161bc8 HTTP/1.1" 502 150 2025-12-17 23:41:28,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f7575bc060ce161bc8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:29,393 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098eb575bc060ce161bb6 2025-12-17 23:41:29,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098eb575bc060ce161bb6 HTTP/1.1" 502 150 2025-12-17 23:41:29,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098eb575bc060ce161bb6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:29,419 - lava_callback - INFO - Received callback for node 69409c97575bc060ce162626 INFO: 172.21.0.1:48134 - "POST /node/69409c97575bc060ce162626 HTTP/1.0" 202 Accepted 2025-12-17 23:41:29,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ed575bc060ce161bb9 2025-12-17 23:41:29,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ed575bc060ce161bb9 HTTP/1.1" 502 150 2025-12-17 23:41:29,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ed575bc060ce161bb9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:30,075 - lava_callback - INFO - Received callback for node 69409c9a575bc060ce16262c INFO: 172.21.0.1:35136 - "POST /node/69409c9a575bc060ce16262c HTTP/1.0" 202 Accepted 2025-12-17 23:41:30,730 - lava_callback - INFO - Received callback for node 69409c9e575bc060ce162635 INFO: 172.21.0.1:35150 - "POST /node/69409c9e575bc060ce162635 HTTP/1.0" 202 Accepted 2025-12-17 23:41:31,374 - lava_callback - INFO - Received callback for node 69409ca1575bc060ce16263b INFO: 172.21.0.1:35166 - "POST /node/69409ca1575bc060ce16263b HTTP/1.0" 202 Accepted 2025-12-17 23:41:31,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ef575bc060ce161bbc 2025-12-17 23:41:31,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098da575bc060ce161b8f 2025-12-17 23:41:31,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098da575bc060ce161b8f HTTP/1.1" 502 150 2025-12-17 23:41:31,698 - lava_callback - ERROR - Error processing callback for node 694098da575bc060ce161b8f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098da575bc060ce161b8f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098da575bc060ce161b8f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098da575bc060ce161b8f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:31,700 - lava_callback - INFO - Starting async job submit for node 694098f9575bc060ce161bcb 2025-12-17 23:41:31,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:31,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f9575bc060ce161bcb HTTP/1.1" 502 150 2025-12-17 23:41:31,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f9575bc060ce161bcb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:31,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f9575bc060ce161bcb 2025-12-17 23:41:31,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f9575bc060ce161bcb HTTP/1.1" 502 150 2025-12-17 23:41:31,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f9575bc060ce161bcb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:31,901 - lava_callback - INFO - Received callback for node 69409ca2575bc060ce16263e INFO: 172.21.0.1:35182 - "POST /node/69409ca2575bc060ce16263e HTTP/1.0" 202 Accepted 2025-12-17 23:41:32,037 - lava_callback - INFO - Received callback for node 69409ca5575bc060ce162644 INFO: 172.21.0.1:35186 - "POST /node/69409ca5575bc060ce162644 HTTP/1.0" 202 Accepted 2025-12-17 23:41:32,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f5575bc060ce161bc5 2025-12-17 23:41:32,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f5575bc060ce161bc5 HTTP/1.1" 502 150 2025-12-17 23:41:32,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f5575bc060ce161bc5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:32,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f1575bc060ce161bbf 2025-12-17 23:41:32,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f1575bc060ce161bbf HTTP/1.1" 502 150 2025-12-17 23:41:32,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f1575bc060ce161bbf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:32,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f3575bc060ce161bc2 2025-12-17 23:41:32,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f3575bc060ce161bc2 HTTP/1.1" 502 150 2025-12-17 23:41:32,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f3575bc060ce161bc2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:32,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f7575bc060ce161bc8 2025-12-17 23:41:32,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f7575bc060ce161bc8 HTTP/1.1" 502 150 2025-12-17 23:41:32,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f7575bc060ce161bc8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:33,105 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098cc575bc060ce161b6c 2025-12-17 23:41:33,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098cc575bc060ce161b6c HTTP/1.1" 502 150 2025-12-17 23:41:33,106 - lava_callback - ERROR - Error processing callback for node 694098cc575bc060ce161b6c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098cc575bc060ce161b6c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098cc575bc060ce161b6c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098cc575bc060ce161b6c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:33,107 - lava_callback - INFO - Starting async job submit for node 694098fb575bc060ce161bd4 2025-12-17 23:41:33,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:33,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098fb575bc060ce161bd4 HTTP/1.1" 502 150 2025-12-17 23:41:33,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098fb575bc060ce161bd4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:33,115 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098fb575bc060ce161bd4 2025-12-17 23:41:33,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098fb575bc060ce161bd4 HTTP/1.1" 502 150 2025-12-17 23:41:33,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098fb575bc060ce161bd4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:33,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098dc575bc060ce161b92 2025-12-17 23:41:33,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098dc575bc060ce161b92 HTTP/1.1" 502 150 2025-12-17 23:41:33,573 - lava_callback - ERROR - Error processing callback for node 694098dc575bc060ce161b92: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098dc575bc060ce161b92 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098dc575bc060ce161b92 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098dc575bc060ce161b92 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:33,575 - lava_callback - INFO - Starting async job submit for node 694098fd575bc060ce161bdf 2025-12-17 23:41:33,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:33,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098fd575bc060ce161bdf HTTP/1.1" 502 150 2025-12-17 23:41:33,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098fd575bc060ce161bdf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:33,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098fd575bc060ce161bdf 2025-12-17 23:41:33,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098fd575bc060ce161bdf HTTP/1.1" 502 150 2025-12-17 23:41:33,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098fd575bc060ce161bdf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:33,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f9575bc060ce161bcb 2025-12-17 23:41:33,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f9575bc060ce161bcb HTTP/1.1" 502 150 2025-12-17 23:41:33,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f9575bc060ce161bcb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:33,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098de575bc060ce161b95 2025-12-17 23:41:33,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098de575bc060ce161b95 HTTP/1.1" 502 150 2025-12-17 23:41:33,900 - lava_callback - ERROR - Error processing callback for node 694098de575bc060ce161b95: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098de575bc060ce161b95 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098de575bc060ce161b95 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098de575bc060ce161b95 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:33,902 - lava_callback - INFO - Starting async job submit for node 694098ff575bc060ce161be2 2025-12-17 23:41:33,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:33,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ff575bc060ce161be2 HTTP/1.1" 502 150 2025-12-17 23:41:33,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ff575bc060ce161be2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:33,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ff575bc060ce161be2 2025-12-17 23:41:33,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ff575bc060ce161be2 HTTP/1.1" 502 150 2025-12-17 23:41:33,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ff575bc060ce161be2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:35,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098fb575bc060ce161bd4 2025-12-17 23:41:35,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098fb575bc060ce161bd4 HTTP/1.1" 502 150 2025-12-17 23:41:35,119 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098fb575bc060ce161bd4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:35,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098fd575bc060ce161bdf 2025-12-17 23:41:35,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098fd575bc060ce161bdf HTTP/1.1" 502 150 2025-12-17 23:41:35,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098fd575bc060ce161bdf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:35,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ff575bc060ce161be2 2025-12-17 23:41:35,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ff575bc060ce161be2 HTTP/1.1" 502 150 2025-12-17 23:41:35,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ff575bc060ce161be2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35188 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:41:36,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a86575bc060ce161fc5 2025-12-17 23:41:36,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a86575bc060ce161fc5 HTTP/1.1" 502 150 2025-12-17 23:41:36,534 - lava_callback - ERROR - Error processing callback for node 69409a86575bc060ce161fc5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a86575bc060ce161fc5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a86575bc060ce161fc5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a86575bc060ce161fc5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:36,535 - lava_callback - INFO - Starting async job submit for node 69409901575bc060ce161be9 2025-12-17 23:41:36,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:36,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409901575bc060ce161be9 HTTP/1.1" 502 150 2025-12-17 23:41:36,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409901575bc060ce161be9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:36,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409901575bc060ce161be9 2025-12-17 23:41:36,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409901575bc060ce161be9 HTTP/1.1" 502 150 2025-12-17 23:41:36,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409901575bc060ce161be9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:37,421 - lava_callback - INFO - Received callback for node 69409c9d575bc060ce162632 INFO: 172.21.0.1:35194 - "POST /node/69409c9d575bc060ce162632 HTTP/1.0" 202 Accepted 2025-12-17 23:41:37,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f9575bc060ce161bcb 2025-12-17 23:41:37,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f9575bc060ce161bcb HTTP/1.1" 502 150 2025-12-17 23:41:37,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f9575bc060ce161bcb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:37,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c78575bc060ce1625d8 2025-12-17 23:41:37,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c78575bc060ce1625d8 HTTP/1.1" 502 150 2025-12-17 23:41:37,752 - lava_callback - ERROR - Error processing callback for node 69409c78575bc060ce1625d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c78575bc060ce1625d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c78575bc060ce1625d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c78575bc060ce1625d8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:37,754 - lava_callback - INFO - Starting async job submit for node 69409903575bc060ce161bec 2025-12-17 23:41:37,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:37,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409903575bc060ce161bec HTTP/1.1" 502 150 2025-12-17 23:41:37,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409903575bc060ce161bec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:37,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409903575bc060ce161bec 2025-12-17 23:41:37,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409903575bc060ce161bec HTTP/1.1" 502 150 2025-12-17 23:41:37,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409903575bc060ce161bec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:38,087 - lava_callback - INFO - Received callback for node 69409c9f575bc060ce162638 INFO: 172.21.0.1:35206 - "POST /node/69409c9f575bc060ce162638 HTTP/1.0" 202 Accepted 2025-12-17 23:41:38,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409901575bc060ce161be9 2025-12-17 23:41:38,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409901575bc060ce161be9 HTTP/1.1" 502 150 2025-12-17 23:41:38,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409901575bc060ce161be9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:39,119 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098fb575bc060ce161bd4 2025-12-17 23:41:39,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098fb575bc060ce161bd4 HTTP/1.1" 502 150 2025-12-17 23:41:39,120 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098fb575bc060ce161bd4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:39,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098fd575bc060ce161bdf 2025-12-17 23:41:39,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098fd575bc060ce161bdf HTTP/1.1" 502 150 2025-12-17 23:41:39,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098fd575bc060ce161bdf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:39,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409903575bc060ce161bec 2025-12-17 23:41:39,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409903575bc060ce161bec HTTP/1.1" 502 150 2025-12-17 23:41:39,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409903575bc060ce161bec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:39,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ff575bc060ce161be2 2025-12-17 23:41:39,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ff575bc060ce161be2 HTTP/1.1" 502 150 2025-12-17 23:41:39,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ff575bc060ce161be2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:40,447 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f5575bc060ce161bc5 2025-12-17 23:41:40,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f5575bc060ce161bc5 HTTP/1.1" 502 150 2025-12-17 23:41:40,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f5575bc060ce161bc5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:40,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f7575bc060ce161bc8 2025-12-17 23:41:40,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f7575bc060ce161bc8 HTTP/1.1" 502 150 2025-12-17 23:41:40,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f7575bc060ce161bc8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:42,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409901575bc060ce161be9 2025-12-17 23:41:42,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409901575bc060ce161be9 HTTP/1.1" 502 150 2025-12-17 23:41:42,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409901575bc060ce161be9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:43,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409903575bc060ce161bec 2025-12-17 23:41:43,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098d8575bc060ce161b88 2025-12-17 23:41:44,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e6575bc060ce161ba0 2025-12-17 23:41:44,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e9575bc060ce161bab 2025-12-17 23:41:45,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098eb575bc060ce161bb6 2025-12-17 23:41:45,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ed575bc060ce161bb9 2025-12-17 23:41:45,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f9575bc060ce161bcb INFO: 172.21.0.1:37412 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:41:47,121 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098fb575bc060ce161bd4 2025-12-17 23:41:47,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098fd575bc060ce161bdf 2025-12-17 23:41:47,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ff575bc060ce161be2 2025-12-17 23:41:48,124 - lava_callback - INFO - Received callback for node 69409ca7575bc060ce16264b INFO: 172.21.0.1:37416 - "POST /node/69409ca7575bc060ce16264b HTTP/1.0" 202 Accepted 2025-12-17 23:41:48,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f1575bc060ce161bbf 2025-12-17 23:41:48,772 - lava_callback - INFO - Received callback for node 69409ca9575bc060ce162654 INFO: 172.21.0.1:37422 - "POST /node/69409ca9575bc060ce162654 HTTP/1.0" 202 Accepted 2025-12-17 23:41:48,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f3575bc060ce161bc2 2025-12-17 23:41:49,419 - lava_callback - INFO - Received callback for node 69409cab575bc060ce162657 INFO: 172.21.0.1:37428 - "POST /node/69409cab575bc060ce162657 HTTP/1.0" 202 Accepted 2025-12-17 23:41:50,081 - lava_callback - INFO - Received callback for node 69409cad575bc060ce16265a INFO: 172.21.0.1:52364 - "POST /node/69409cad575bc060ce16265a HTTP/1.0" 202 Accepted 2025-12-17 23:41:50,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409901575bc060ce161be9 2025-12-17 23:41:50,742 - lava_callback - INFO - Received callback for node 69409caf575bc060ce162661 INFO: 172.21.0.1:52380 - "POST /node/69409caf575bc060ce162661 HTTP/1.0" 202 Accepted 2025-12-17 23:41:51,384 - lava_callback - INFO - Received callback for node 69409cb1575bc060ce162664 INFO: 172.21.0.1:52394 - "POST /node/69409cb1575bc060ce162664 HTTP/1.0" 202 Accepted 2025-12-17 23:41:52,032 - lava_callback - INFO - Received callback for node 69409cb3575bc060ce162667 INFO: 172.21.0.1:52410 - "POST /node/69409cb3575bc060ce162667 HTTP/1.0" 202 Accepted 2025-12-17 23:41:52,679 - lava_callback - INFO - Received callback for node 69409cb5575bc060ce16266a INFO: 172.21.0.1:52418 - "POST /node/69409cb5575bc060ce16266a HTTP/1.0" 202 Accepted 2025-12-17 23:41:53,330 - lava_callback - INFO - Received callback for node 69409cb7575bc060ce16266d INFO: 172.21.0.1:52422 - "POST /node/69409cb7575bc060ce16266d HTTP/1.0" 202 Accepted 2025-12-17 23:41:53,991 - lava_callback - INFO - Received callback for node 69409cb9575bc060ce162670 INFO: 172.21.0.1:52436 - "POST /node/69409cb9575bc060ce162670 HTTP/1.0" 202 Accepted 2025-12-17 23:41:56,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f5575bc060ce161bc5 INFO: 172.21.0.1:52440 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:41:56,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f7575bc060ce161bc8 2025-12-17 23:41:59,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409903575bc060ce161bec HTTP/1.1" 502 150 2025-12-17 23:41:59,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409903575bc060ce161bec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:59,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098d8575bc060ce161b88 HTTP/1.1" 502 150 2025-12-17 23:41:59,345 - lava_callback - ERROR - Error processing callback for node 694098d8575bc060ce161b88: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098d8575bc060ce161b88 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098d8575bc060ce161b88 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098d8575bc060ce161b88 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:41:59,346 - lava_callback - INFO - Starting async job submit for node 69409905575bc060ce161bef 2025-12-17 23:41:59,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:41:59,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409905575bc060ce161bef HTTP/1.1" 502 150 2025-12-17 23:41:59,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409905575bc060ce161bef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:41:59,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409905575bc060ce161bef 2025-12-17 23:41:59,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409905575bc060ce161bef HTTP/1.1" 502 150 2025-12-17 23:41:59,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409905575bc060ce161bef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:00,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e6575bc060ce161ba0 HTTP/1.1" 502 150 2025-12-17 23:42:00,112 - lava_callback - ERROR - Error processing callback for node 694098e6575bc060ce161ba0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098e6575bc060ce161ba0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098e6575bc060ce161ba0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098e6575bc060ce161ba0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:00,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e9575bc060ce161bab HTTP/1.1" 502 150 2025-12-17 23:42:00,115 - lava_callback - INFO - Starting async job submit for node 69409907575bc060ce161bf6 2025-12-17 23:42:00,115 - lava_callback - ERROR - Error processing callback for node 694098e9575bc060ce161bab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098e9575bc060ce161bab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098e9575bc060ce161bab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098e9575bc060ce161bab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:00,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:00,117 - lava_callback - INFO - Starting async job submit for node 69409909575bc060ce161bf9 2025-12-17 23:42:00,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:00,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409907575bc060ce161bf6 HTTP/1.1" 502 150 2025-12-17 23:42:00,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409907575bc060ce161bf6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:00,123 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409907575bc060ce161bf6 2025-12-17 23:42:00,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409907575bc060ce161bf6 HTTP/1.1" 502 150 2025-12-17 23:42:00,124 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409907575bc060ce161bf6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:00,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409909575bc060ce161bf9 HTTP/1.1" 502 150 2025-12-17 23:42:00,148 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409909575bc060ce161bf9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:00,148 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409909575bc060ce161bf9 2025-12-17 23:42:00,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409909575bc060ce161bf9 HTTP/1.1" 502 150 2025-12-17 23:42:00,149 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409909575bc060ce161bf9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:00,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098eb575bc060ce161bb6 HTTP/1.1" 502 150 2025-12-17 23:42:00,881 - lava_callback - ERROR - Error processing callback for node 694098eb575bc060ce161bb6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098eb575bc060ce161bb6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098eb575bc060ce161bb6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098eb575bc060ce161bb6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:00,883 - lava_callback - INFO - Starting async job submit for node 6940990b575bc060ce161bfc 2025-12-17 23:42:00,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:00,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ed575bc060ce161bb9 HTTP/1.1" 502 150 2025-12-17 23:42:00,885 - lava_callback - ERROR - Error processing callback for node 694098ed575bc060ce161bb9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ed575bc060ce161bb9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ed575bc060ce161bb9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ed575bc060ce161bb9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:00,888 - lava_callback - INFO - Starting async job submit for node 6940990d575bc060ce161bff 2025-12-17 23:42:00,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:00,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990d575bc060ce161bff HTTP/1.1" 502 150 2025-12-17 23:42:00,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940990d575bc060ce161bff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:00,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940990d575bc060ce161bff 2025-12-17 23:42:00,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990d575bc060ce161bff HTTP/1.1" 502 150 2025-12-17 23:42:00,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940990d575bc060ce161bff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:00,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990b575bc060ce161bfc HTTP/1.1" 502 150 2025-12-17 23:42:00,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940990b575bc060ce161bfc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:00,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940990b575bc060ce161bfc 2025-12-17 23:42:00,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990b575bc060ce161bfc HTTP/1.1" 502 150 2025-12-17 23:42:00,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940990b575bc060ce161bfc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:01,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f9575bc060ce161bcb HTTP/1.1" 502 150 2025-12-17 23:42:01,136 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098f9575bc060ce161bcb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:01,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409905575bc060ce161bef 2025-12-17 23:42:01,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409905575bc060ce161bef HTTP/1.1" 502 150 2025-12-17 23:42:01,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409905575bc060ce161bef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:02,124 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409907575bc060ce161bf6 2025-12-17 23:42:02,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409907575bc060ce161bf6 HTTP/1.1" 502 150 2025-12-17 23:42:02,126 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409907575bc060ce161bf6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:02,149 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409909575bc060ce161bf9 2025-12-17 23:42:02,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409909575bc060ce161bf9 HTTP/1.1" 502 150 2025-12-17 23:42:02,150 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409909575bc060ce161bf9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:02,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ef575bc060ce161bbc HTTP/1.1" 502 150 2025-12-17 23:42:02,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ef575bc060ce161bbc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:02,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098fb575bc060ce161bd4 HTTP/1.1" 502 150 2025-12-17 23:42:02,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098fb575bc060ce161bd4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:02,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940990d575bc060ce161bff 2025-12-17 23:42:02,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990d575bc060ce161bff HTTP/1.1" 502 150 2025-12-17 23:42:02,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940990d575bc060ce161bff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:02,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940990b575bc060ce161bfc 2025-12-17 23:42:02,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990b575bc060ce161bfc HTTP/1.1" 502 150 2025-12-17 23:42:02,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940990b575bc060ce161bfc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:02,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098fd575bc060ce161bdf HTTP/1.1" 502 150 2025-12-17 23:42:02,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098fd575bc060ce161bdf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:03,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ff575bc060ce161be2 HTTP/1.1" 502 150 2025-12-17 23:42:03,184 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098ff575bc060ce161be2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37256 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:42:05,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409905575bc060ce161bef 2025-12-17 23:42:06,126 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409907575bc060ce161bf6 2025-12-17 23:42:06,150 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409909575bc060ce161bf9 INFO: 172.21.0.1:37272 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:42:06,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940990d575bc060ce161bff 2025-12-17 23:42:06,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940990b575bc060ce161bfc 2025-12-17 23:42:07,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409903575bc060ce161bec 2025-12-17 23:42:08,114 - lava_callback - INFO - Received callback for node 69409cbb575bc060ce162677 INFO: 172.21.0.1:37288 - "POST /node/69409cbb575bc060ce162677 HTTP/1.0" 202 Accepted 2025-12-17 23:42:08,755 - lava_callback - INFO - Received callback for node 69409cbe575bc060ce162680 INFO: 172.21.0.1:37290 - "POST /node/69409cbe575bc060ce162680 HTTP/1.0" 202 Accepted 2025-12-17 23:42:09,420 - lava_callback - INFO - Received callback for node 69409cc0575bc060ce162685 INFO: 172.21.0.1:37292 - "POST /node/69409cc0575bc060ce162685 HTTP/1.0" 202 Accepted 2025-12-17 23:42:10,079 - lava_callback - INFO - Received callback for node 69409cc2575bc060ce162688 INFO: 172.21.0.1:58324 - "POST /node/69409cc2575bc060ce162688 HTTP/1.0" 202 Accepted 2025-12-17 23:42:10,724 - lava_callback - INFO - Received callback for node 69409cc4575bc060ce16268b INFO: 172.21.0.1:58340 - "POST /node/69409cc4575bc060ce16268b HTTP/1.0" 202 Accepted 2025-12-17 23:42:11,371 - lava_callback - INFO - Received callback for node 69409cc5575bc060ce16268e INFO: 172.21.0.1:58350 - "POST /node/69409cc5575bc060ce16268e HTTP/1.0" 202 Accepted 2025-12-17 23:42:12,016 - lava_callback - INFO - Received callback for node 69409cc8575bc060ce162695 INFO: 172.21.0.1:58356 - "POST /node/69409cc8575bc060ce162695 HTTP/1.0" 202 Accepted 2025-12-17 23:42:12,672 - lava_callback - INFO - Received callback for node 69409cc9575bc060ce162698 INFO: 172.21.0.1:58358 - "POST /node/69409cc9575bc060ce162698 HTTP/1.0" 202 Accepted 2025-12-17 23:42:13,310 - lava_callback - INFO - Received callback for node 69409ccb575bc060ce16269b INFO: 172.21.0.1:58360 - "POST /node/69409ccb575bc060ce16269b HTTP/1.0" 202 Accepted 2025-12-17 23:42:13,944 - lava_callback - INFO - Received callback for node 69409ccd575bc060ce16269e INFO: 172.21.0.1:58370 - "POST /node/69409ccd575bc060ce16269e HTTP/1.0" 202 Accepted INFO: 172.21.0.1:58384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58400 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:42:17,137 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098f9575bc060ce161bcb 2025-12-17 23:42:18,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ef575bc060ce161bbc 2025-12-17 23:42:18,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098fb575bc060ce161bd4 2025-12-17 23:42:18,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098fd575bc060ce161bdf 2025-12-17 23:42:19,184 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098ff575bc060ce161be2 2025-12-17 23:42:19,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ff575bc060ce161be2 HTTP/1.1" 502 150 2025-12-17 23:42:19,186 - lava_callback - ERROR - Error processing callback for node 694098ff575bc060ce161be2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ff575bc060ce161be2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ff575bc060ce161be2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ff575bc060ce161be2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:19,188 - lava_callback - INFO - Starting async job submit for node 69409915575bc060ce161c1c 2025-12-17 23:42:19,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:19,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409915575bc060ce161c1c HTTP/1.1" 502 150 2025-12-17 23:42:19,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409915575bc060ce161c1c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:19,194 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409915575bc060ce161c1c 2025-12-17 23:42:19,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409915575bc060ce161c1c HTTP/1.1" 502 150 2025-12-17 23:42:19,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409915575bc060ce161c1c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:19,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f1575bc060ce161bbf HTTP/1.1" 502 150 2025-12-17 23:42:19,312 - lava_callback - ERROR - Error processing callback for node 694098f1575bc060ce161bbf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098f1575bc060ce161bbf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098f1575bc060ce161bbf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098f1575bc060ce161bbf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:19,314 - lava_callback - INFO - Starting async job submit for node 69409917575bc060ce161c1f 2025-12-17 23:42:19,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:19,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409917575bc060ce161c1f HTTP/1.1" 502 150 2025-12-17 23:42:19,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409917575bc060ce161c1f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:19,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409917575bc060ce161c1f 2025-12-17 23:42:19,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409917575bc060ce161c1f HTTP/1.1" 502 150 2025-12-17 23:42:19,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409917575bc060ce161c1f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:19,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f3575bc060ce161bc2 HTTP/1.1" 502 150 2025-12-17 23:42:19,568 - lava_callback - ERROR - Error processing callback for node 694098f3575bc060ce161bc2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098f3575bc060ce161bc2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098f3575bc060ce161bc2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098f3575bc060ce161bc2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:19,570 - lava_callback - INFO - Starting async job submit for node 69409912575bc060ce161c16 2025-12-17 23:42:19,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:19,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409912575bc060ce161c16 HTTP/1.1" 502 150 2025-12-17 23:42:19,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409912575bc060ce161c16'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:19,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409912575bc060ce161c16 2025-12-17 23:42:19,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409912575bc060ce161c16 HTTP/1.1" 502 150 2025-12-17 23:42:19,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409912575bc060ce161c16'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:20,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409905575bc060ce161bef HTTP/1.1" 502 150 2025-12-17 23:42:20,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409905575bc060ce161bef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:21,195 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409915575bc060ce161c1c 2025-12-17 23:42:21,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409915575bc060ce161c1c HTTP/1.1" 502 150 2025-12-17 23:42:21,197 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409915575bc060ce161c1c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:21,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409917575bc060ce161c1f 2025-12-17 23:42:21,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409917575bc060ce161c1f HTTP/1.1" 502 150 2025-12-17 23:42:21,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409917575bc060ce161c1f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:21,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409901575bc060ce161be9 HTTP/1.1" 502 150 2025-12-17 23:42:21,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409901575bc060ce161be9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:21,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409912575bc060ce161c16 2025-12-17 23:42:21,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409912575bc060ce161c16 HTTP/1.1" 502 150 2025-12-17 23:42:21,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409912575bc060ce161c16'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:21,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409909575bc060ce161bf9 HTTP/1.1" 502 150 2025-12-17 23:42:21,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409909575bc060ce161bf9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:21,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409907575bc060ce161bf6 HTTP/1.1" 502 150 2025-12-17 23:42:21,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409907575bc060ce161bf6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:22,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990d575bc060ce161bff HTTP/1.1" 502 150 2025-12-17 23:42:22,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940990d575bc060ce161bff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:22,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409903575bc060ce161bec HTTP/1.1" 502 150 2025-12-17 23:42:22,385 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409903575bc060ce161bec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:22,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990b575bc060ce161bfc HTTP/1.1" 502 150 2025-12-17 23:42:22,385 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940990b575bc060ce161bfc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:25,197 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409915575bc060ce161c1c 2025-12-17 23:42:25,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409915575bc060ce161c1c HTTP/1.1" 502 150 2025-12-17 23:42:25,198 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409915575bc060ce161c1c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:25,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409917575bc060ce161c1f 2025-12-17 23:42:25,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409917575bc060ce161c1f HTTP/1.1" 502 150 2025-12-17 23:42:25,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409917575bc060ce161c1f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:25,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409912575bc060ce161c16 2025-12-17 23:42:25,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409912575bc060ce161c16 HTTP/1.1" 502 150 2025-12-17 23:42:25,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409912575bc060ce161c16'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58292 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:42:27,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f5575bc060ce161bc5 HTTP/1.1" 502 150 2025-12-17 23:42:27,248 - lava_callback - ERROR - Error processing callback for node 694098f5575bc060ce161bc5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098f5575bc060ce161bc5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098f5575bc060ce161bc5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098f5575bc060ce161bc5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:27,250 - lava_callback - INFO - Starting async job submit for node 6941f90cbb27571da24af054 2025-12-17 23:42:27,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:27,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941f90cbb27571da24af054 HTTP/1.1" 502 150 2025-12-17 23:42:27,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941f90cbb27571da24af054'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:27,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941f90cbb27571da24af054 2025-12-17 23:42:27,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941f90cbb27571da24af054 HTTP/1.1" 502 150 2025-12-17 23:42:27,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941f90cbb27571da24af054'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:27,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f7575bc060ce161bc8 HTTP/1.1" 502 150 2025-12-17 23:42:27,760 - lava_callback - ERROR - Error processing callback for node 694098f7575bc060ce161bc8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098f7575bc060ce161bc8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098f7575bc060ce161bc8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098f7575bc060ce161bc8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:27,762 - lava_callback - INFO - Starting async job submit for node 69409913575bc060ce161c19 2025-12-17 23:42:27,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:27,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409913575bc060ce161c19 HTTP/1.1" 502 150 2025-12-17 23:42:27,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409913575bc060ce161c19'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:27,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409913575bc060ce161c19 2025-12-17 23:42:27,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409913575bc060ce161c19 HTTP/1.1" 502 150 2025-12-17 23:42:27,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409913575bc060ce161c19'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:28,142 - lava_callback - INFO - Received callback for node 69409ccf575bc060ce1626a7 INFO: 172.21.0.1:58296 - "POST /node/69409ccf575bc060ce1626a7 HTTP/1.0" 202 Accepted 2025-12-17 23:42:28,789 - lava_callback - INFO - Received callback for node 69409cd1575bc060ce1626b2 INFO: 172.21.0.1:58302 - "POST /node/69409cd1575bc060ce1626b2 HTTP/1.0" 202 Accepted 2025-12-17 23:42:28,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409905575bc060ce161bef 2025-12-17 23:42:28,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409905575bc060ce161bef HTTP/1.1" 502 150 2025-12-17 23:42:28,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409905575bc060ce161bef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:29,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941f90cbb27571da24af054 2025-12-17 23:42:29,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941f90cbb27571da24af054 HTTP/1.1" 502 150 2025-12-17 23:42:29,267 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941f90cbb27571da24af054'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:29,445 - lava_callback - INFO - Received callback for node 69409cd3575bc060ce1626b5 INFO: 172.21.0.1:58318 - "POST /node/69409cd3575bc060ce1626b5 HTTP/1.0" 202 Accepted 2025-12-17 23:42:29,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409909575bc060ce161bf9 2025-12-17 23:42:29,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409907575bc060ce161bf6 2025-12-17 23:42:29,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409909575bc060ce161bf9 HTTP/1.1" 502 150 2025-12-17 23:42:29,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409909575bc060ce161bf9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:29,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409907575bc060ce161bf6 HTTP/1.1" 502 150 2025-12-17 23:42:29,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409907575bc060ce161bf6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:29,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409913575bc060ce161c19 2025-12-17 23:42:29,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409913575bc060ce161c19 HTTP/1.1" 502 150 2025-12-17 23:42:29,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409913575bc060ce161c19'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:30,109 - lava_callback - INFO - Received callback for node 69409cd5575bc060ce1626b8 INFO: 172.21.0.1:44568 - "POST /node/69409cd5575bc060ce1626b8 HTTP/1.0" 202 Accepted 2025-12-17 23:42:30,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940990d575bc060ce161bff 2025-12-17 23:42:30,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940990b575bc060ce161bfc 2025-12-17 23:42:30,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990d575bc060ce161bff HTTP/1.1" 502 150 2025-12-17 23:42:30,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940990d575bc060ce161bff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:30,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990b575bc060ce161bfc HTTP/1.1" 502 150 2025-12-17 23:42:30,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940990b575bc060ce161bfc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:30,756 - lava_callback - INFO - Received callback for node 69409cd7575bc060ce1626bb INFO: 172.21.0.1:44578 - "POST /node/69409cd7575bc060ce1626bb HTTP/1.0" 202 Accepted 2025-12-17 23:42:31,406 - lava_callback - INFO - Received callback for node 69409cd9575bc060ce1626c2 INFO: 172.21.0.1:44586 - "POST /node/69409cd9575bc060ce1626c2 HTTP/1.0" 202 Accepted 2025-12-17 23:42:32,055 - lava_callback - INFO - Received callback for node 69409cdb575bc060ce1626c5 INFO: 172.21.0.1:44602 - "POST /node/69409cdb575bc060ce1626c5 HTTP/1.0" 202 Accepted 2025-12-17 23:42:32,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098f9575bc060ce161bcb HTTP/1.1" 502 150 2025-12-17 23:42:32,624 - lava_callback - ERROR - Error processing callback for node 694098f9575bc060ce161bcb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098f9575bc060ce161bcb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098f9575bc060ce161bcb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098f9575bc060ce161bcb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:32,626 - lava_callback - INFO - Starting async job submit for node 6940991a575bc060ce161c29 2025-12-17 23:42:32,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:32,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991a575bc060ce161c29 HTTP/1.1" 502 150 2025-12-17 23:42:32,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940991a575bc060ce161c29'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:32,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940991a575bc060ce161c29 2025-12-17 23:42:32,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991a575bc060ce161c29 HTTP/1.1" 502 150 2025-12-17 23:42:32,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940991a575bc060ce161c29'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:32,722 - lava_callback - INFO - Received callback for node 69409cde575bc060ce1626c8 INFO: 172.21.0.1:44614 - "POST /node/69409cde575bc060ce1626c8 HTTP/1.0" 202 Accepted 2025-12-17 23:42:33,198 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409915575bc060ce161c1c 2025-12-17 23:42:33,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409915575bc060ce161c1c HTTP/1.1" 502 150 2025-12-17 23:42:33,201 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409915575bc060ce161c1c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:33,267 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941f90cbb27571da24af054 2025-12-17 23:42:33,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941f90cbb27571da24af054 HTTP/1.1" 502 150 2025-12-17 23:42:33,268 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941f90cbb27571da24af054'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:33,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409917575bc060ce161c1f 2025-12-17 23:42:33,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409917575bc060ce161c1f HTTP/1.1" 502 150 2025-12-17 23:42:33,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409917575bc060ce161c1f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:33,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409912575bc060ce161c16 2025-12-17 23:42:33,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409912575bc060ce161c16 HTTP/1.1" 502 150 2025-12-17 23:42:33,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409912575bc060ce161c16'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:33,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409913575bc060ce161c19 2025-12-17 23:42:33,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409913575bc060ce161c19 HTTP/1.1" 502 150 2025-12-17 23:42:33,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409913575bc060ce161c19'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:33,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098ef575bc060ce161bbc HTTP/1.1" 502 150 2025-12-17 23:42:33,904 - lava_callback - ERROR - Error processing callback for node 694098ef575bc060ce161bbc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ef575bc060ce161bbc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ef575bc060ce161bbc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098ef575bc060ce161bbc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:33,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098fb575bc060ce161bd4 HTTP/1.1" 502 150 2025-12-17 23:42:33,907 - lava_callback - ERROR - Error processing callback for node 694098fb575bc060ce161bd4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098fb575bc060ce161bd4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098fb575bc060ce161bd4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098fb575bc060ce161bd4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:33,907 - lava_callback - INFO - Starting async job submit for node 6940991d575bc060ce161c2c 2025-12-17 23:42:33,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:33,911 - lava_callback - INFO - Starting async job submit for node 6940991f575bc060ce161c2f 2025-12-17 23:42:33,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:33,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991f575bc060ce161c2f HTTP/1.1" 502 150 2025-12-17 23:42:33,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940991f575bc060ce161c2f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:33,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940991f575bc060ce161c2f 2025-12-17 23:42:33,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991f575bc060ce161c2f HTTP/1.1" 502 150 2025-12-17 23:42:33,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940991f575bc060ce161c2f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:33,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991d575bc060ce161c2c HTTP/1.1" 502 150 2025-12-17 23:42:33,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940991d575bc060ce161c2c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:33,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940991d575bc060ce161c2c 2025-12-17 23:42:33,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991d575bc060ce161c2c HTTP/1.1" 502 150 2025-12-17 23:42:33,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940991d575bc060ce161c2c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:34,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940991a575bc060ce161c29 2025-12-17 23:42:34,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991a575bc060ce161c29 HTTP/1.1" 502 150 2025-12-17 23:42:34,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940991a575bc060ce161c29'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:35,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940991f575bc060ce161c2f 2025-12-17 23:42:35,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991f575bc060ce161c2f HTTP/1.1" 502 150 2025-12-17 23:42:35,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940991f575bc060ce161c2f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:35,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940991d575bc060ce161c2c 2025-12-17 23:42:35,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991d575bc060ce161c2c HTTP/1.1" 502 150 2025-12-17 23:42:35,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940991d575bc060ce161c2c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44628 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:42:37,200 - lava_callback - INFO - Received callback for node 6940a38d575bc060ce1631f6 INFO: 172.21.0.1:44642 - "POST /node/6940a38d575bc060ce1631f6 HTTP/1.0" 202 Accepted 2025-12-17 23:42:37,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409901575bc060ce161be9 2025-12-17 23:42:37,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409901575bc060ce161be9 HTTP/1.1" 502 150 2025-12-17 23:42:37,362 - lava_callback - ERROR - Error processing callback for node 69409901575bc060ce161be9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409901575bc060ce161be9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409901575bc060ce161be9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409901575bc060ce161be9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:37,364 - lava_callback - INFO - Starting async job submit for node 69409920575bc060ce161c32 2025-12-17 23:42:37,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:37,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409920575bc060ce161c32 HTTP/1.1" 502 150 2025-12-17 23:42:37,374 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409920575bc060ce161c32'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:37,374 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409920575bc060ce161c32 2025-12-17 23:42:37,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409920575bc060ce161c32 HTTP/1.1" 502 150 2025-12-17 23:42:37,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409920575bc060ce161c32'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:37,494 - lava_callback - INFO - Received callback for node 69409ceb575bc060ce1626e7 INFO: 172.21.0.1:44646 - "POST /node/69409ceb575bc060ce1626e7 HTTP/1.0" 202 Accepted 2025-12-17 23:42:38,154 - lava_callback - INFO - Received callback for node 69409ce0575bc060ce1626cb INFO: 172.21.0.1:44656 - "POST /node/69409ce0575bc060ce1626cb HTTP/1.0" 202 Accepted 2025-12-17 23:42:38,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409903575bc060ce161bec 2025-12-17 23:42:38,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409903575bc060ce161bec HTTP/1.1" 502 150 2025-12-17 23:42:38,386 - lava_callback - ERROR - Error processing callback for node 69409903575bc060ce161bec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409903575bc060ce161bec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409903575bc060ce161bec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409903575bc060ce161bec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:38,388 - lava_callback - INFO - Starting async job submit for node 69409927575bc060ce161c48 2025-12-17 23:42:38,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:38,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409927575bc060ce161c48 HTTP/1.1" 502 150 2025-12-17 23:42:38,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409927575bc060ce161c48'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:38,395 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409927575bc060ce161c48 2025-12-17 23:42:38,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409927575bc060ce161c48 HTTP/1.1" 502 150 2025-12-17 23:42:38,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409927575bc060ce161c48'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:38,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940991a575bc060ce161c29 2025-12-17 23:42:38,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991a575bc060ce161c29 HTTP/1.1" 502 150 2025-12-17 23:42:38,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940991a575bc060ce161c29'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:38,827 - lava_callback - INFO - Received callback for node 69409ce7575bc060ce1626dd INFO: 172.21.0.1:44662 - "POST /node/69409ce7575bc060ce1626dd HTTP/1.0" 202 Accepted 2025-12-17 23:42:39,375 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409920575bc060ce161c32 2025-12-17 23:42:39,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409920575bc060ce161c32 HTTP/1.1" 502 150 2025-12-17 23:42:39,377 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409920575bc060ce161c32'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:39,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940991f575bc060ce161c2f 2025-12-17 23:42:39,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991f575bc060ce161c2f HTTP/1.1" 502 150 2025-12-17 23:42:39,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940991f575bc060ce161c2f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:39,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940991d575bc060ce161c2c 2025-12-17 23:42:39,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991d575bc060ce161c2c HTTP/1.1" 502 150 2025-12-17 23:42:39,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940991d575bc060ce161c2c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:40,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409927575bc060ce161c48 2025-12-17 23:42:40,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409927575bc060ce161c48 HTTP/1.1" 502 150 2025-12-17 23:42:40,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409927575bc060ce161c48'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:41,268 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941f90cbb27571da24af054 2025-12-17 23:42:41,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941f90cbb27571da24af054 HTTP/1.1" 502 150 2025-12-17 23:42:41,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941f90cbb27571da24af054'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:41,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409913575bc060ce161c19 2025-12-17 23:42:41,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409913575bc060ce161c19 HTTP/1.1" 502 150 2025-12-17 23:42:41,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409913575bc060ce161c19'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:43,377 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409920575bc060ce161c32 2025-12-17 23:42:43,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409920575bc060ce161c32 HTTP/1.1" 502 150 2025-12-17 23:42:43,378 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409920575bc060ce161c32'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:44,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409927575bc060ce161c48 2025-12-17 23:42:44,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409927575bc060ce161c48 HTTP/1.1" 502 150 2025-12-17 23:42:44,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409927575bc060ce161c48'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:44,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409905575bc060ce161bef 2025-12-17 23:42:44,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409905575bc060ce161bef HTTP/1.1" 502 150 2025-12-17 23:42:44,860 - lava_callback - ERROR - Error processing callback for node 69409905575bc060ce161bef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409905575bc060ce161bef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409905575bc060ce161bef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409905575bc060ce161bef (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:44,862 - lava_callback - INFO - Starting async job submit for node 6940992b575bc060ce161c54 2025-12-17 23:42:44,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:44,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992b575bc060ce161c54 HTTP/1.1" 502 150 2025-12-17 23:42:44,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992b575bc060ce161c54'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:44,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992b575bc060ce161c54 2025-12-17 23:42:44,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992b575bc060ce161c54 HTTP/1.1" 502 150 2025-12-17 23:42:44,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992b575bc060ce161c54'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:45,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409909575bc060ce161bf9 2025-12-17 23:42:45,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409907575bc060ce161bf6 2025-12-17 23:42:45,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409909575bc060ce161bf9 HTTP/1.1" 502 150 2025-12-17 23:42:45,619 - lava_callback - ERROR - Error processing callback for node 69409909575bc060ce161bf9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409909575bc060ce161bf9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409909575bc060ce161bf9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409909575bc060ce161bf9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:45,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409907575bc060ce161bf6 HTTP/1.1" 502 150 2025-12-17 23:42:45,622 - lava_callback - ERROR - Error processing callback for node 69409907575bc060ce161bf6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409907575bc060ce161bf6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409907575bc060ce161bf6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409907575bc060ce161bf6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:45,621 - lava_callback - INFO - Starting async job submit for node 6941fcdcbb27571da24afbdb 2025-12-17 23:42:45,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:45,627 - lava_callback - INFO - Starting async job submit for node 69409922575bc060ce161c35 2025-12-17 23:42:45,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:45,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941fcdcbb27571da24afbdb HTTP/1.1" 502 150 2025-12-17 23:42:45,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941fcdcbb27571da24afbdb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:45,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941fcdcbb27571da24afbdb 2025-12-17 23:42:45,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941fcdcbb27571da24afbdb HTTP/1.1" 502 150 2025-12-17 23:42:45,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941fcdcbb27571da24afbdb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:45,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409922575bc060ce161c35 HTTP/1.1" 502 150 2025-12-17 23:42:45,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409922575bc060ce161c35'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:45,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409922575bc060ce161c35 2025-12-17 23:42:45,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409922575bc060ce161c35 HTTP/1.1" 502 150 2025-12-17 23:42:45,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409922575bc060ce161c35'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:46,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940990d575bc060ce161bff 2025-12-17 23:42:46,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940990b575bc060ce161bfc 2025-12-17 23:42:46,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990d575bc060ce161bff HTTP/1.1" 502 150 2025-12-17 23:42:46,388 - lava_callback - ERROR - Error processing callback for node 6940990d575bc060ce161bff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940990d575bc060ce161bff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940990d575bc060ce161bff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940990d575bc060ce161bff (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:46,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940990b575bc060ce161bfc HTTP/1.1" 502 150 2025-12-17 23:42:46,390 - lava_callback - ERROR - Error processing callback for node 6940990b575bc060ce161bfc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940990b575bc060ce161bfc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940990b575bc060ce161bfc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940990b575bc060ce161bfc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:46,390 - lava_callback - INFO - Starting async job submit for node 69409924575bc060ce161c3e 2025-12-17 23:42:46,392 - lava_callback - INFO - Starting async job submit for node 69409929575bc060ce161c51 2025-12-17 23:42:46,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:46,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:46,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409924575bc060ce161c3e HTTP/1.1" 502 150 2025-12-17 23:42:46,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409924575bc060ce161c3e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:46,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409924575bc060ce161c3e 2025-12-17 23:42:46,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409924575bc060ce161c3e HTTP/1.1" 502 150 2025-12-17 23:42:46,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409924575bc060ce161c3e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:46,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409929575bc060ce161c51 HTTP/1.1" 502 150 2025-12-17 23:42:46,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409929575bc060ce161c51'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:46,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409929575bc060ce161c51 2025-12-17 23:42:46,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409929575bc060ce161c51 HTTP/1.1" 502 150 2025-12-17 23:42:46,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409929575bc060ce161c51'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48866 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:42:46,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940991a575bc060ce161c29 2025-12-17 23:42:46,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991a575bc060ce161c29 HTTP/1.1" 502 150 2025-12-17 23:42:46,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940991a575bc060ce161c29'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:46,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992b575bc060ce161c54 2025-12-17 23:42:46,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992b575bc060ce161c54 HTTP/1.1" 502 150 2025-12-17 23:42:46,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992b575bc060ce161c54'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:47,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941fcdcbb27571da24afbdb 2025-12-17 23:42:47,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941fcdcbb27571da24afbdb HTTP/1.1" 502 150 2025-12-17 23:42:47,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941fcdcbb27571da24afbdb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:47,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409922575bc060ce161c35 2025-12-17 23:42:47,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409922575bc060ce161c35 HTTP/1.1" 502 150 2025-12-17 23:42:47,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409922575bc060ce161c35'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:47,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940991f575bc060ce161c2f 2025-12-17 23:42:47,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991f575bc060ce161c2f HTTP/1.1" 502 150 2025-12-17 23:42:47,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940991f575bc060ce161c2f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:47,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940991d575bc060ce161c2c 2025-12-17 23:42:47,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991d575bc060ce161c2c HTTP/1.1" 502 150 2025-12-17 23:42:47,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940991d575bc060ce161c2c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:48,144 - lava_callback - INFO - Received callback for node 69409ce5575bc060ce1626da INFO: 172.21.0.1:48874 - "POST /node/69409ce5575bc060ce1626da HTTP/1.0" 202 Accepted 2025-12-17 23:42:48,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409924575bc060ce161c3e 2025-12-17 23:42:48,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409924575bc060ce161c3e HTTP/1.1" 502 150 2025-12-17 23:42:48,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409924575bc060ce161c3e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:48,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409929575bc060ce161c51 2025-12-17 23:42:48,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409929575bc060ce161c51 HTTP/1.1" 502 150 2025-12-17 23:42:48,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409929575bc060ce161c51'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:48,794 - lava_callback - INFO - Received callback for node 69409ce9575bc060ce1626e4 INFO: 172.21.0.1:48888 - "POST /node/69409ce9575bc060ce1626e4 HTTP/1.0" 202 Accepted 2025-12-17 23:42:49,201 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409915575bc060ce161c1c 2025-12-17 23:42:49,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409915575bc060ce161c1c HTTP/1.1" 502 150 2025-12-17 23:42:49,203 - lava_callback - ERROR - Error processing callback for node 69409915575bc060ce161c1c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409915575bc060ce161c1c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409915575bc060ce161c1c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409915575bc060ce161c1c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:49,205 - lava_callback - INFO - Starting async job submit for node 69409930575bc060ce161c60 2025-12-17 23:42:49,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:49,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409930575bc060ce161c60 HTTP/1.1" 502 150 2025-12-17 23:42:49,213 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409930575bc060ce161c60'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:49,214 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409930575bc060ce161c60 2025-12-17 23:42:49,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409930575bc060ce161c60 HTTP/1.1" 502 150 2025-12-17 23:42:49,214 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409930575bc060ce161c60'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:49,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409917575bc060ce161c1f 2025-12-17 23:42:49,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409917575bc060ce161c1f HTTP/1.1" 502 150 2025-12-17 23:42:49,350 - lava_callback - ERROR - Error processing callback for node 69409917575bc060ce161c1f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409917575bc060ce161c1f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409917575bc060ce161c1f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409917575bc060ce161c1f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:49,353 - lava_callback - INFO - Starting async job submit for node 69409933575bc060ce161c63 2025-12-17 23:42:49,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:49,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409933575bc060ce161c63 HTTP/1.1" 502 150 2025-12-17 23:42:49,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409933575bc060ce161c63'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:49,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409933575bc060ce161c63 2025-12-17 23:42:49,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409933575bc060ce161c63 HTTP/1.1" 502 150 2025-12-17 23:42:49,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409933575bc060ce161c63'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:49,456 - lava_callback - INFO - Received callback for node 69409ced575bc060ce1626ea INFO: 172.21.0.1:48890 - "POST /node/69409ced575bc060ce1626ea HTTP/1.0" 202 Accepted 2025-12-17 23:42:49,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409912575bc060ce161c16 2025-12-17 23:42:49,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409912575bc060ce161c16 HTTP/1.1" 502 150 2025-12-17 23:42:49,602 - lava_callback - ERROR - Error processing callback for node 69409912575bc060ce161c16: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409912575bc060ce161c16 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409912575bc060ce161c16 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409912575bc060ce161c16 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:49,604 - lava_callback - INFO - Starting async job submit for node 69409935575bc060ce161c66 2025-12-17 23:42:49,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:49,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409935575bc060ce161c66 HTTP/1.1" 502 150 2025-12-17 23:42:49,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409935575bc060ce161c66'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:49,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409935575bc060ce161c66 2025-12-17 23:42:49,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409935575bc060ce161c66 HTTP/1.1" 502 150 2025-12-17 23:42:49,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409935575bc060ce161c66'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:49,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098fd575bc060ce161bdf HTTP/1.1" 502 150 2025-12-17 23:42:49,776 - lava_callback - ERROR - Error processing callback for node 694098fd575bc060ce161bdf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098fd575bc060ce161bdf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098fd575bc060ce161bdf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098fd575bc060ce161bdf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:49,778 - lava_callback - INFO - Starting async job submit for node 6940992f575bc060ce161c5d 2025-12-17 23:42:49,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:49,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992f575bc060ce161c5d HTTP/1.1" 502 150 2025-12-17 23:42:49,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992f575bc060ce161c5d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:49,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992f575bc060ce161c5d 2025-12-17 23:42:49,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992f575bc060ce161c5d HTTP/1.1" 502 150 2025-12-17 23:42:49,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992f575bc060ce161c5d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:50,099 - lava_callback - INFO - Received callback for node 69409cef575bc060ce1626ed INFO: 172.21.0.1:34288 - "POST /node/69409cef575bc060ce1626ed HTTP/1.0" 202 Accepted 2025-12-17 23:42:50,744 - lava_callback - INFO - Received callback for node 69409cf1575bc060ce1626f0 INFO: 172.21.0.1:34304 - "POST /node/69409cf1575bc060ce1626f0 HTTP/1.0" 202 Accepted 2025-12-17 23:42:50,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992b575bc060ce161c54 2025-12-17 23:42:50,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992b575bc060ce161c54 HTTP/1.1" 502 150 2025-12-17 23:42:50,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992b575bc060ce161c54'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:51,215 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409930575bc060ce161c60 2025-12-17 23:42:51,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409930575bc060ce161c60 HTTP/1.1" 502 150 2025-12-17 23:42:51,216 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409930575bc060ce161c60'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:51,364 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409933575bc060ce161c63 2025-12-17 23:42:51,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409933575bc060ce161c63 HTTP/1.1" 502 150 2025-12-17 23:42:51,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409933575bc060ce161c63'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:51,378 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409920575bc060ce161c32 2025-12-17 23:42:51,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409920575bc060ce161c32 HTTP/1.1" 502 150 2025-12-17 23:42:51,379 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409920575bc060ce161c32'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:51,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409935575bc060ce161c66 2025-12-17 23:42:51,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409935575bc060ce161c66 HTTP/1.1" 502 150 2025-12-17 23:42:51,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409935575bc060ce161c66'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:51,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941fcdcbb27571da24afbdb 2025-12-17 23:42:51,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941fcdcbb27571da24afbdb HTTP/1.1" 502 150 2025-12-17 23:42:51,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941fcdcbb27571da24afbdb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:51,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409922575bc060ce161c35 2025-12-17 23:42:51,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409922575bc060ce161c35 HTTP/1.1" 502 150 2025-12-17 23:42:51,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409922575bc060ce161c35'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:51,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992f575bc060ce161c5d 2025-12-17 23:42:51,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992f575bc060ce161c5d HTTP/1.1" 502 150 2025-12-17 23:42:51,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992f575bc060ce161c5d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:52,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409924575bc060ce161c3e 2025-12-17 23:42:52,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409924575bc060ce161c3e HTTP/1.1" 502 150 2025-12-17 23:42:52,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409924575bc060ce161c3e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:52,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409927575bc060ce161c48 2025-12-17 23:42:52,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409927575bc060ce161c48 HTTP/1.1" 502 150 2025-12-17 23:42:52,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409927575bc060ce161c48'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:52,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409929575bc060ce161c51 2025-12-17 23:42:52,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409929575bc060ce161c51 HTTP/1.1" 502 150 2025-12-17 23:42:52,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409929575bc060ce161c51'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:55,216 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409930575bc060ce161c60 2025-12-17 23:42:55,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409930575bc060ce161c60 HTTP/1.1" 502 150 2025-12-17 23:42:55,218 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409930575bc060ce161c60'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:55,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409933575bc060ce161c63 2025-12-17 23:42:55,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409933575bc060ce161c63 HTTP/1.1" 502 150 2025-12-17 23:42:55,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409933575bc060ce161c63'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:55,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409935575bc060ce161c66 2025-12-17 23:42:55,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409935575bc060ce161c66 HTTP/1.1" 502 150 2025-12-17 23:42:55,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409935575bc060ce161c66'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:55,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992f575bc060ce161c5d 2025-12-17 23:42:55,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992f575bc060ce161c5d HTTP/1.1" 502 150 2025-12-17 23:42:55,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992f575bc060ce161c5d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34308 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:42:57,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941f90cbb27571da24af054 2025-12-17 23:42:57,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941f90cbb27571da24af054 HTTP/1.1" 502 150 2025-12-17 23:42:57,273 - lava_callback - ERROR - Error processing callback for node 6941f90cbb27571da24af054: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941f90cbb27571da24af054 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941f90cbb27571da24af054 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941f90cbb27571da24af054 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:57,275 - lava_callback - INFO - Starting async job submit for node 69409937575bc060ce161c6f 2025-12-17 23:42:57,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:57,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409937575bc060ce161c6f HTTP/1.1" 502 150 2025-12-17 23:42:57,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409937575bc060ce161c6f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:57,284 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409937575bc060ce161c6f 2025-12-17 23:42:57,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409937575bc060ce161c6f HTTP/1.1" 502 150 2025-12-17 23:42:57,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409937575bc060ce161c6f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:57,482 - lava_callback - INFO - Received callback for node 69409cfb575bc060ce162709 INFO: 172.21.0.1:34324 - "POST /node/69409cfb575bc060ce162709 HTTP/1.0" 202 Accepted 2025-12-17 23:42:57,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409913575bc060ce161c19 2025-12-17 23:42:57,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409913575bc060ce161c19 HTTP/1.1" 502 150 2025-12-17 23:42:57,785 - lava_callback - ERROR - Error processing callback for node 69409913575bc060ce161c19: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409913575bc060ce161c19 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409913575bc060ce161c19 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409913575bc060ce161c19 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:42:57,786 - lava_callback - INFO - Starting async job submit for node 69409939575bc060ce161c78 2025-12-17 23:42:57,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:42:57,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409939575bc060ce161c78 HTTP/1.1" 502 150 2025-12-17 23:42:57,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409939575bc060ce161c78'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:57,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409939575bc060ce161c78 2025-12-17 23:42:57,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409939575bc060ce161c78 HTTP/1.1" 502 150 2025-12-17 23:42:57,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409939575bc060ce161c78'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:58,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992b575bc060ce161c54 2025-12-17 23:42:58,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992b575bc060ce161c54 HTTP/1.1" 502 150 2025-12-17 23:42:58,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992b575bc060ce161c54'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:59,285 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409937575bc060ce161c6f 2025-12-17 23:42:59,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409937575bc060ce161c6f HTTP/1.1" 502 150 2025-12-17 23:42:59,286 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409937575bc060ce161c6f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:59,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941fcdcbb27571da24afbdb 2025-12-17 23:42:59,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941fcdcbb27571da24afbdb HTTP/1.1" 502 150 2025-12-17 23:42:59,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941fcdcbb27571da24afbdb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:59,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409922575bc060ce161c35 2025-12-17 23:42:59,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409922575bc060ce161c35 HTTP/1.1" 502 150 2025-12-17 23:42:59,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409922575bc060ce161c35'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:42:59,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409939575bc060ce161c78 2025-12-17 23:42:59,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409939575bc060ce161c78 HTTP/1.1" 502 150 2025-12-17 23:42:59,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409939575bc060ce161c78'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:00,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409924575bc060ce161c3e 2025-12-17 23:43:00,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409929575bc060ce161c51 2025-12-17 23:43:00,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409929575bc060ce161c51 HTTP/1.1" 502 150 2025-12-17 23:43:00,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409929575bc060ce161c51'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:01,336 - lava_callback - INFO - Received callback for node 694098c3575bc060ce161b5c INFO: 172.21.0.1:37766 - "POST /node/694098c3575bc060ce161b5c HTTP/1.0" 202 Accepted 2025-12-17 23:43:02,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940991a575bc060ce161c29 2025-12-17 23:43:02,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991a575bc060ce161c29 HTTP/1.1" 502 150 2025-12-17 23:43:02,664 - lava_callback - ERROR - Error processing callback for node 6940991a575bc060ce161c29: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940991a575bc060ce161c29 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940991a575bc060ce161c29 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940991a575bc060ce161c29 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:02,666 - lava_callback - INFO - Starting async job submit for node 6940993b575bc060ce161c7b 2025-12-17 23:43:02,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:02,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993b575bc060ce161c7b HTTP/1.1" 502 150 2025-12-17 23:43:02,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940993b575bc060ce161c7b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:02,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940993b575bc060ce161c7b 2025-12-17 23:43:02,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993b575bc060ce161c7b HTTP/1.1" 502 150 2025-12-17 23:43:02,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940993b575bc060ce161c7b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:03,218 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409930575bc060ce161c60 2025-12-17 23:43:03,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409930575bc060ce161c60 HTTP/1.1" 502 150 2025-12-17 23:43:03,219 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409930575bc060ce161c60'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:03,286 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409937575bc060ce161c6f 2025-12-17 23:43:03,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409937575bc060ce161c6f HTTP/1.1" 502 150 2025-12-17 23:43:03,287 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409937575bc060ce161c6f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:03,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409933575bc060ce161c63 2025-12-17 23:43:03,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409933575bc060ce161c63 HTTP/1.1" 502 150 2025-12-17 23:43:03,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409933575bc060ce161c63'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37774 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:43:03,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409935575bc060ce161c66 2025-12-17 23:43:03,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409935575bc060ce161c66 HTTP/1.1" 502 150 2025-12-17 23:43:03,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409935575bc060ce161c66'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:03,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992f575bc060ce161c5d 2025-12-17 23:43:03,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992f575bc060ce161c5d HTTP/1.1" 502 150 2025-12-17 23:43:03,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940992f575bc060ce161c5d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:03,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409939575bc060ce161c78 2025-12-17 23:43:03,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409939575bc060ce161c78 HTTP/1.1" 502 150 2025-12-17 23:43:03,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409939575bc060ce161c78'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:03,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940991f575bc060ce161c2f 2025-12-17 23:43:03,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991f575bc060ce161c2f HTTP/1.1" 502 150 2025-12-17 23:43:03,925 - lava_callback - ERROR - Error processing callback for node 6940991f575bc060ce161c2f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940991f575bc060ce161c2f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940991f575bc060ce161c2f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940991f575bc060ce161c2f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:03,926 - lava_callback - INFO - Starting async job submit for node 6940993d575bc060ce161c82 2025-12-17 23:43:03,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:03,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940991d575bc060ce161c2c 2025-12-17 23:43:03,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940991d575bc060ce161c2c HTTP/1.1" 502 150 2025-12-17 23:43:03,932 - lava_callback - ERROR - Error processing callback for node 6940991d575bc060ce161c2c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940991d575bc060ce161c2c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940991d575bc060ce161c2c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940991d575bc060ce161c2c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:03,934 - lava_callback - INFO - Starting async job submit for node 69409940575bc060ce161c88 2025-12-17 23:43:03,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:03,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993d575bc060ce161c82 HTTP/1.1" 502 150 2025-12-17 23:43:03,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940993d575bc060ce161c82'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:03,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940993d575bc060ce161c82 2025-12-17 23:43:03,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993d575bc060ce161c82 HTTP/1.1" 502 150 2025-12-17 23:43:03,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940993d575bc060ce161c82'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:03,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409940575bc060ce161c88 HTTP/1.1" 502 150 2025-12-17 23:43:03,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409940575bc060ce161c88'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:03,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409940575bc060ce161c88 2025-12-17 23:43:03,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409940575bc060ce161c88 HTTP/1.1" 502 150 2025-12-17 23:43:03,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409940575bc060ce161c88'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:04,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940993b575bc060ce161c7b 2025-12-17 23:43:04,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993b575bc060ce161c7b HTTP/1.1" 502 150 2025-12-17 23:43:04,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940993b575bc060ce161c7b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:05,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940993d575bc060ce161c82 2025-12-17 23:43:05,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993d575bc060ce161c82 HTTP/1.1" 502 150 2025-12-17 23:43:05,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940993d575bc060ce161c82'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:05,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409940575bc060ce161c88 2025-12-17 23:43:05,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409940575bc060ce161c88 HTTP/1.1" 502 150 2025-12-17 23:43:05,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409940575bc060ce161c88'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37786 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:43:07,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409920575bc060ce161c32 2025-12-17 23:43:07,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409920575bc060ce161c32 HTTP/1.1" 502 150 2025-12-17 23:43:07,381 - lava_callback - ERROR - Error processing callback for node 69409920575bc060ce161c32: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409920575bc060ce161c32 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409920575bc060ce161c32 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409920575bc060ce161c32 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:07,383 - lava_callback - INFO - Starting async job submit for node 69409942575bc060ce161c91 2025-12-17 23:43:07,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:07,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409942575bc060ce161c91 HTTP/1.1" 502 150 2025-12-17 23:43:07,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409942575bc060ce161c91'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:07,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409942575bc060ce161c91 2025-12-17 23:43:07,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409942575bc060ce161c91 HTTP/1.1" 502 150 2025-12-17 23:43:07,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409942575bc060ce161c91'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:08,158 - lava_callback - INFO - Received callback for node 69409cf7575bc060ce1626fa INFO: 172.21.0.1:37794 - "POST /node/69409cf7575bc060ce1626fa HTTP/1.0" 202 Accepted 2025-12-17 23:43:08,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409927575bc060ce161c48 2025-12-17 23:43:08,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409927575bc060ce161c48 HTTP/1.1" 502 150 2025-12-17 23:43:08,405 - lava_callback - ERROR - Error processing callback for node 69409927575bc060ce161c48: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409927575bc060ce161c48 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409927575bc060ce161c48 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409927575bc060ce161c48 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:08,408 - lava_callback - INFO - Starting async job submit for node 69409944575bc060ce161c94 2025-12-17 23:43:08,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:08,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409944575bc060ce161c94 HTTP/1.1" 502 150 2025-12-17 23:43:08,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409944575bc060ce161c94'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:08,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409944575bc060ce161c94 2025-12-17 23:43:08,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409944575bc060ce161c94 HTTP/1.1" 502 150 2025-12-17 23:43:08,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409944575bc060ce161c94'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:08,795 - lava_callback - INFO - Received callback for node 69409cfa575bc060ce162704 INFO: 172.21.0.1:37800 - "POST /node/69409cfa575bc060ce162704 HTTP/1.0" 202 Accepted 2025-12-17 23:43:08,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940993b575bc060ce161c7b 2025-12-17 23:43:08,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993b575bc060ce161c7b HTTP/1.1" 502 150 2025-12-17 23:43:08,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940993b575bc060ce161c7b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:09,073 - lava_callback - INFO - Received callback for node 69409cfe575bc060ce162715 INFO: 172.21.0.1:37814 - "POST /node/69409cfe575bc060ce162715 HTTP/1.0" 202 Accepted 2025-12-17 23:43:09,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409942575bc060ce161c91 2025-12-17 23:43:09,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409942575bc060ce161c91 HTTP/1.1" 502 150 2025-12-17 23:43:09,425 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409942575bc060ce161c91'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:09,436 - lava_callback - INFO - Received callback for node 69409cfd575bc060ce162710 INFO: 172.21.0.1:37818 - "POST /node/69409cfd575bc060ce162710 HTTP/1.0" 202 Accepted 2025-12-17 23:43:09,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940993d575bc060ce161c82 2025-12-17 23:43:09,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993d575bc060ce161c82 HTTP/1.1" 502 150 2025-12-17 23:43:09,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940993d575bc060ce161c82'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:09,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409940575bc060ce161c88 2025-12-17 23:43:09,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409940575bc060ce161c88 HTTP/1.1" 502 150 2025-12-17 23:43:09,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409940575bc060ce161c88'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:10,089 - lava_callback - INFO - Received callback for node 69409d04575bc060ce162721 INFO: 172.21.0.1:46060 - "POST /node/69409d04575bc060ce162721 HTTP/1.0" 202 Accepted 2025-12-17 23:43:10,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409944575bc060ce161c94 2025-12-17 23:43:10,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409944575bc060ce161c94 HTTP/1.1" 502 150 2025-12-17 23:43:10,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409944575bc060ce161c94'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:11,287 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409937575bc060ce161c6f 2025-12-17 23:43:11,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409937575bc060ce161c6f HTTP/1.1" 502 150 2025-12-17 23:43:11,289 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409937575bc060ce161c6f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:11,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409939575bc060ce161c78 2025-12-17 23:43:11,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409939575bc060ce161c78 HTTP/1.1" 502 150 2025-12-17 23:43:11,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409939575bc060ce161c78'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:13,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409942575bc060ce161c91 2025-12-17 23:43:13,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409942575bc060ce161c91 HTTP/1.1" 502 150 2025-12-17 23:43:13,426 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409942575bc060ce161c91'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:14,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409944575bc060ce161c94 2025-12-17 23:43:14,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409944575bc060ce161c94 HTTP/1.1" 502 150 2025-12-17 23:43:14,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409944575bc060ce161c94'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:14,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992b575bc060ce161c54 2025-12-17 23:43:14,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992b575bc060ce161c54 HTTP/1.1" 502 150 2025-12-17 23:43:14,907 - lava_callback - ERROR - Error processing callback for node 6940992b575bc060ce161c54: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940992b575bc060ce161c54 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940992b575bc060ce161c54 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940992b575bc060ce161c54 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:14,909 - lava_callback - INFO - Starting async job submit for node 69409946575bc060ce161c97 2025-12-17 23:43:14,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:14,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409946575bc060ce161c97 HTTP/1.1" 502 150 2025-12-17 23:43:14,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409946575bc060ce161c97'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:14,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409946575bc060ce161c97 2025-12-17 23:43:14,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409946575bc060ce161c97 HTTP/1.1" 502 150 2025-12-17 23:43:14,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409946575bc060ce161c97'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46064 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:43:15,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941fcdcbb27571da24afbdb 2025-12-17 23:43:15,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409922575bc060ce161c35 2025-12-17 23:43:15,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409922575bc060ce161c35 HTTP/1.1" 502 150 2025-12-17 23:43:15,652 - lava_callback - ERROR - Error processing callback for node 69409922575bc060ce161c35: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409922575bc060ce161c35 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409922575bc060ce161c35 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409922575bc060ce161c35 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:15,654 - lava_callback - INFO - Starting async job submit for node 69409947575bc060ce161c9a 2025-12-17 23:43:15,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:15,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409947575bc060ce161c9a HTTP/1.1" 502 150 2025-12-17 23:43:15,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409947575bc060ce161c9a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:15,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409947575bc060ce161c9a 2025-12-17 23:43:15,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409947575bc060ce161c9a HTTP/1.1" 502 150 2025-12-17 23:43:15,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409947575bc060ce161c9a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:15,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409924575bc060ce161c3e HTTP/1.1" 502 150 2025-12-17 23:43:15,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409924575bc060ce161c3e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:16,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409929575bc060ce161c51 2025-12-17 23:43:16,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409929575bc060ce161c51 HTTP/1.1" 502 150 2025-12-17 23:43:16,419 - lava_callback - ERROR - Error processing callback for node 69409929575bc060ce161c51: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409929575bc060ce161c51 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409929575bc060ce161c51 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409929575bc060ce161c51 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:16,421 - lava_callback - INFO - Starting async job submit for node 6940994a575bc060ce161c9d 2025-12-17 23:43:16,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:16,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994a575bc060ce161c9d HTTP/1.1" 502 150 2025-12-17 23:43:16,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940994a575bc060ce161c9d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:16,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940994a575bc060ce161c9d 2025-12-17 23:43:16,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994a575bc060ce161c9d HTTP/1.1" 502 150 2025-12-17 23:43:16,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940994a575bc060ce161c9d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46076 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:43:16,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940993b575bc060ce161c7b 2025-12-17 23:43:16,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993b575bc060ce161c7b HTTP/1.1" 502 150 2025-12-17 23:43:16,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940993b575bc060ce161c7b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:16,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409946575bc060ce161c97 2025-12-17 23:43:16,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409946575bc060ce161c97 HTTP/1.1" 502 150 2025-12-17 23:43:16,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409946575bc060ce161c97'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:17,511 - lava_callback - INFO - Received callback for node 69409d00575bc060ce162718 INFO: 172.21.0.1:46088 - "POST /node/69409d00575bc060ce162718 HTTP/1.0" 202 Accepted 2025-12-17 23:43:17,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409947575bc060ce161c9a 2025-12-17 23:43:17,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409947575bc060ce161c9a HTTP/1.1" 502 150 2025-12-17 23:43:17,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409947575bc060ce161c9a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:17,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940993d575bc060ce161c82 2025-12-17 23:43:17,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993d575bc060ce161c82 HTTP/1.1" 502 150 2025-12-17 23:43:17,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940993d575bc060ce161c82'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:17,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409940575bc060ce161c88 2025-12-17 23:43:17,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409940575bc060ce161c88 HTTP/1.1" 502 150 2025-12-17 23:43:17,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409940575bc060ce161c88'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:18,184 - lava_callback - INFO - Received callback for node 69409d01575bc060ce16271b INFO: 172.21.0.1:46090 - "POST /node/69409d01575bc060ce16271b HTTP/1.0" 202 Accepted 2025-12-17 23:43:18,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940994a575bc060ce161c9d 2025-12-17 23:43:18,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994a575bc060ce161c9d HTTP/1.1" 502 150 2025-12-17 23:43:18,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940994a575bc060ce161c9d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:18,873 - lava_callback - INFO - Received callback for node 69409d03575bc060ce16271e INFO: 172.21.0.1:46096 - "POST /node/69409d03575bc060ce16271e HTTP/1.0" 202 Accepted 2025-12-17 23:43:19,220 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409930575bc060ce161c60 2025-12-17 23:43:19,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409930575bc060ce161c60 HTTP/1.1" 502 150 2025-12-17 23:43:19,221 - lava_callback - ERROR - Error processing callback for node 69409930575bc060ce161c60: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409930575bc060ce161c60 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409930575bc060ce161c60 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409930575bc060ce161c60 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:19,223 - lava_callback - INFO - Starting async job submit for node 69409807575bc060ce1619a2 2025-12-17 23:43:19,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:19,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409807575bc060ce1619a2 HTTP/1.1" 502 150 2025-12-17 23:43:19,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409807575bc060ce1619a2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:19,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409807575bc060ce1619a2 2025-12-17 23:43:19,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409807575bc060ce1619a2 HTTP/1.1" 502 150 2025-12-17 23:43:19,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409807575bc060ce1619a2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:19,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409933575bc060ce161c63 2025-12-17 23:43:19,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409933575bc060ce161c63 HTTP/1.1" 502 150 2025-12-17 23:43:19,369 - lava_callback - ERROR - Error processing callback for node 69409933575bc060ce161c63: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409933575bc060ce161c63 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409933575bc060ce161c63 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409933575bc060ce161c63 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:19,371 - lava_callback - INFO - Starting async job submit for node 6940994c575bc060ce161caa 2025-12-17 23:43:19,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:19,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994c575bc060ce161caa HTTP/1.1" 502 150 2025-12-17 23:43:19,378 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940994c575bc060ce161caa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:19,378 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940994c575bc060ce161caa 2025-12-17 23:43:19,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994c575bc060ce161caa HTTP/1.1" 502 150 2025-12-17 23:43:19,379 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940994c575bc060ce161caa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:19,554 - lava_callback - INFO - Received callback for node 69409d07575bc060ce162724 INFO: 172.21.0.1:46098 - "POST /node/69409d07575bc060ce162724 HTTP/1.0" 202 Accepted 2025-12-17 23:43:19,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409935575bc060ce161c66 2025-12-17 23:43:19,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409935575bc060ce161c66 HTTP/1.1" 502 150 2025-12-17 23:43:19,617 - lava_callback - ERROR - Error processing callback for node 69409935575bc060ce161c66: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409935575bc060ce161c66 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409935575bc060ce161c66 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409935575bc060ce161c66 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:19,619 - lava_callback - INFO - Starting async job submit for node 6940994e575bc060ce161cad 2025-12-17 23:43:19,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:19,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994e575bc060ce161cad HTTP/1.1" 502 150 2025-12-17 23:43:19,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940994e575bc060ce161cad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:19,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940994e575bc060ce161cad 2025-12-17 23:43:19,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994e575bc060ce161cad HTTP/1.1" 502 150 2025-12-17 23:43:19,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940994e575bc060ce161cad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:19,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940992f575bc060ce161c5d 2025-12-17 23:43:19,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940992f575bc060ce161c5d HTTP/1.1" 502 150 2025-12-17 23:43:19,791 - lava_callback - ERROR - Error processing callback for node 6940992f575bc060ce161c5d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940992f575bc060ce161c5d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940992f575bc060ce161c5d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940992f575bc060ce161c5d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:19,792 - lava_callback - INFO - Starting async job submit for node 6940995a575bc060ce161cbc 2025-12-17 23:43:19,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:19,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940995a575bc060ce161cbc HTTP/1.1" 502 150 2025-12-17 23:43:19,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940995a575bc060ce161cbc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:19,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940995a575bc060ce161cbc 2025-12-17 23:43:19,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940995a575bc060ce161cbc HTTP/1.1" 502 150 2025-12-17 23:43:19,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940995a575bc060ce161cbc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:20,250 - lava_callback - INFO - Received callback for node 69409d09575bc060ce162727 INFO: 172.21.0.1:47616 - "POST /node/69409d09575bc060ce162727 HTTP/1.0" 202 Accepted 2025-12-17 23:43:20,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409946575bc060ce161c97 2025-12-17 23:43:20,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409946575bc060ce161c97 HTTP/1.1" 502 150 2025-12-17 23:43:20,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409946575bc060ce161c97'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:20,932 - lava_callback - INFO - Received callback for node 69409d0a575bc060ce16272a INFO: 172.21.0.1:47620 - "POST /node/69409d0a575bc060ce16272a HTTP/1.0" 202 Accepted 2025-12-17 23:43:21,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409807575bc060ce1619a2 2025-12-17 23:43:21,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409807575bc060ce1619a2 HTTP/1.1" 502 150 2025-12-17 23:43:21,333 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409807575bc060ce1619a2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:21,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940994c575bc060ce161caa 2025-12-17 23:43:21,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994c575bc060ce161caa HTTP/1.1" 502 150 2025-12-17 23:43:21,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940994c575bc060ce161caa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:21,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409942575bc060ce161c91 2025-12-17 23:43:21,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409942575bc060ce161c91 HTTP/1.1" 502 150 2025-12-17 23:43:21,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409942575bc060ce161c91'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:21,595 - lava_callback - INFO - Received callback for node 69409d0d575bc060ce162734 INFO: 172.21.0.1:47624 - "POST /node/69409d0d575bc060ce162734 HTTP/1.0" 202 Accepted 2025-12-17 23:43:21,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940994e575bc060ce161cad 2025-12-17 23:43:21,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994e575bc060ce161cad HTTP/1.1" 502 150 2025-12-17 23:43:21,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940994e575bc060ce161cad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:21,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409947575bc060ce161c9a 2025-12-17 23:43:21,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409947575bc060ce161c9a HTTP/1.1" 502 150 2025-12-17 23:43:21,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409947575bc060ce161c9a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:21,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940995a575bc060ce161cbc 2025-12-17 23:43:21,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940995a575bc060ce161cbc HTTP/1.1" 502 150 2025-12-17 23:43:21,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940995a575bc060ce161cbc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:22,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409944575bc060ce161c94 2025-12-17 23:43:22,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940994a575bc060ce161c9d 2025-12-17 23:43:22,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994a575bc060ce161c9d HTTP/1.1" 502 150 2025-12-17 23:43:22,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940994a575bc060ce161c9d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:25,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409807575bc060ce1619a2 2025-12-17 23:43:25,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409807575bc060ce1619a2 HTTP/1.1" 502 150 2025-12-17 23:43:25,335 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409807575bc060ce1619a2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:25,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940994c575bc060ce161caa 2025-12-17 23:43:25,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994c575bc060ce161caa HTTP/1.1" 502 150 2025-12-17 23:43:25,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940994c575bc060ce161caa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:25,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940994e575bc060ce161cad 2025-12-17 23:43:25,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994e575bc060ce161cad HTTP/1.1" 502 150 2025-12-17 23:43:25,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940994e575bc060ce161cad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:25,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940995a575bc060ce161cbc 2025-12-17 23:43:25,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940995a575bc060ce161cbc HTTP/1.1" 502 150 2025-12-17 23:43:25,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940995a575bc060ce161cbc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47638 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:43:27,289 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409937575bc060ce161c6f 2025-12-17 23:43:27,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409937575bc060ce161c6f HTTP/1.1" 502 150 2025-12-17 23:43:27,290 - lava_callback - ERROR - Error processing callback for node 69409937575bc060ce161c6f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409937575bc060ce161c6f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409937575bc060ce161c6f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409937575bc060ce161c6f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:27,292 - lava_callback - INFO - Starting async job submit for node 69409950575bc060ce161cb0 2025-12-17 23:43:27,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:27,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409950575bc060ce161cb0 HTTP/1.1" 502 150 2025-12-17 23:43:27,315 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409950575bc060ce161cb0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:27,315 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409950575bc060ce161cb0 2025-12-17 23:43:27,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409950575bc060ce161cb0 HTTP/1.1" 502 150 2025-12-17 23:43:27,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409950575bc060ce161cb0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:27,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409939575bc060ce161c78 2025-12-17 23:43:27,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409939575bc060ce161c78 HTTP/1.1" 502 150 2025-12-17 23:43:27,802 - lava_callback - ERROR - Error processing callback for node 69409939575bc060ce161c78: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409939575bc060ce161c78 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409939575bc060ce161c78 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409939575bc060ce161c78 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:27,804 - lava_callback - INFO - Starting async job submit for node 69409954575bc060ce161cb3 2025-12-17 23:43:27,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:27,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409954575bc060ce161cb3 HTTP/1.1" 502 150 2025-12-17 23:43:27,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409954575bc060ce161cb3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:27,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409954575bc060ce161cb3 2025-12-17 23:43:27,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409954575bc060ce161cb3 HTTP/1.1" 502 150 2025-12-17 23:43:27,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409954575bc060ce161cb3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:28,202 - lava_callback - INFO - Received callback for node 69409d0e575bc060ce162739 INFO: 172.21.0.1:47646 - "POST /node/69409d0e575bc060ce162739 HTTP/1.0" 202 Accepted 2025-12-17 23:43:28,847 - lava_callback - INFO - Received callback for node 69409d11575bc060ce16273c INFO: 172.21.0.1:47662 - "POST /node/69409d11575bc060ce16273c HTTP/1.0" 202 Accepted 2025-12-17 23:43:28,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409946575bc060ce161c97 2025-12-17 23:43:28,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409946575bc060ce161c97 HTTP/1.1" 502 150 2025-12-17 23:43:28,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409946575bc060ce161c97'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:29,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409950575bc060ce161cb0 2025-12-17 23:43:29,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409950575bc060ce161cb0 HTTP/1.1" 502 150 2025-12-17 23:43:29,318 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409950575bc060ce161cb0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:29,499 - lava_callback - INFO - Received callback for node 69409d13575bc060ce16273f INFO: 172.21.0.1:47676 - "POST /node/69409d13575bc060ce16273f HTTP/1.0" 202 Accepted 2025-12-17 23:43:29,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409947575bc060ce161c9a 2025-12-17 23:43:29,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409947575bc060ce161c9a HTTP/1.1" 502 150 2025-12-17 23:43:29,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409947575bc060ce161c9a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:29,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409954575bc060ce161cb3 2025-12-17 23:43:29,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409954575bc060ce161cb3 HTTP/1.1" 502 150 2025-12-17 23:43:29,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409954575bc060ce161cb3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:30,163 - lava_callback - INFO - Received callback for node 69409d15575bc060ce162742 INFO: 172.21.0.1:44554 - "POST /node/69409d15575bc060ce162742 HTTP/1.0" 202 Accepted 2025-12-17 23:43:30,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940994a575bc060ce161c9d 2025-12-17 23:43:30,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994a575bc060ce161c9d HTTP/1.1" 502 150 2025-12-17 23:43:30,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940994a575bc060ce161c9d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:30,816 - lava_callback - INFO - Received callback for node 69409d17575bc060ce162745 INFO: 172.21.0.1:44566 - "POST /node/69409d17575bc060ce162745 HTTP/1.0" 202 Accepted 2025-12-17 23:43:30,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941fcdcbb27571da24afbdb HTTP/1.1" 502 150 2025-12-17 23:43:30,992 - lava_callback - ERROR - Error processing callback for node 6941fcdcbb27571da24afbdb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941fcdcbb27571da24afbdb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941fcdcbb27571da24afbdb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941fcdcbb27571da24afbdb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:30,994 - lava_callback - INFO - Starting async job submit for node 69409956575bc060ce161cb6 2025-12-17 23:43:30,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:31,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409956575bc060ce161cb6 HTTP/1.1" 502 150 2025-12-17 23:43:31,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409956575bc060ce161cb6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:31,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409956575bc060ce161cb6 2025-12-17 23:43:31,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409956575bc060ce161cb6 HTTP/1.1" 502 150 2025-12-17 23:43:31,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409956575bc060ce161cb6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:31,476 - lava_callback - INFO - Received callback for node 69409d19575bc060ce162748 INFO: 172.21.0.1:44582 - "POST /node/69409d19575bc060ce162748 HTTP/1.0" 202 Accepted 2025-12-17 23:43:31,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409924575bc060ce161c3e 2025-12-17 23:43:31,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409924575bc060ce161c3e HTTP/1.1" 502 150 2025-12-17 23:43:31,890 - lava_callback - ERROR - Error processing callback for node 69409924575bc060ce161c3e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409924575bc060ce161c3e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409924575bc060ce161c3e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409924575bc060ce161c3e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:31,891 - lava_callback - INFO - Starting async job submit for node 6940995d575bc060ce161cc2 2025-12-17 23:43:31,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:31,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940995d575bc060ce161cc2 HTTP/1.1" 502 150 2025-12-17 23:43:31,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940995d575bc060ce161cc2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:31,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940995d575bc060ce161cc2 2025-12-17 23:43:31,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940995d575bc060ce161cc2 HTTP/1.1" 502 150 2025-12-17 23:43:31,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940995d575bc060ce161cc2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:32,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940993b575bc060ce161c7b 2025-12-17 23:43:32,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993b575bc060ce161c7b HTTP/1.1" 502 150 2025-12-17 23:43:32,854 - lava_callback - ERROR - Error processing callback for node 6940993b575bc060ce161c7b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940993b575bc060ce161c7b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940993b575bc060ce161c7b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940993b575bc060ce161c7b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:32,856 - lava_callback - INFO - Starting async job submit for node 69409d1b575bc060ce16274b 2025-12-17 23:43:32,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:32,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1b575bc060ce16274b HTTP/1.1" 502 150 2025-12-17 23:43:32,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d1b575bc060ce16274b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:32,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d1b575bc060ce16274b 2025-12-17 23:43:32,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1b575bc060ce16274b HTTP/1.1" 502 150 2025-12-17 23:43:32,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d1b575bc060ce16274b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:33,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409956575bc060ce161cb6 2025-12-17 23:43:33,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409956575bc060ce161cb6 HTTP/1.1" 502 150 2025-12-17 23:43:33,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409956575bc060ce161cb6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:33,318 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409950575bc060ce161cb0 2025-12-17 23:43:33,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409950575bc060ce161cb0 HTTP/1.1" 502 150 2025-12-17 23:43:33,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409950575bc060ce161cb0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:33,335 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409807575bc060ce1619a2 2025-12-17 23:43:33,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409807575bc060ce1619a2 HTTP/1.1" 502 150 2025-12-17 23:43:33,336 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409807575bc060ce1619a2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:33,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940994c575bc060ce161caa 2025-12-17 23:43:33,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994c575bc060ce161caa HTTP/1.1" 502 150 2025-12-17 23:43:33,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940994c575bc060ce161caa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:33,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940994e575bc060ce161cad 2025-12-17 23:43:33,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994e575bc060ce161cad HTTP/1.1" 502 150 2025-12-17 23:43:33,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940994e575bc060ce161cad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:33,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940995a575bc060ce161cbc 2025-12-17 23:43:33,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940995a575bc060ce161cbc HTTP/1.1" 502 150 2025-12-17 23:43:33,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940995a575bc060ce161cbc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:33,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409954575bc060ce161cb3 2025-12-17 23:43:33,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409954575bc060ce161cb3 HTTP/1.1" 502 150 2025-12-17 23:43:33,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409954575bc060ce161cb3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:33,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940995d575bc060ce161cc2 2025-12-17 23:43:33,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940995d575bc060ce161cc2 HTTP/1.1" 502 150 2025-12-17 23:43:33,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940995d575bc060ce161cc2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:33,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940993d575bc060ce161c82 2025-12-17 23:43:33,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940993d575bc060ce161c82 HTTP/1.1" 502 150 2025-12-17 23:43:33,942 - lava_callback - ERROR - Error processing callback for node 6940993d575bc060ce161c82: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940993d575bc060ce161c82 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940993d575bc060ce161c82 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940993d575bc060ce161c82 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:33,944 - lava_callback - INFO - Starting async job submit for node 69409c27575bc060ce162508 2025-12-17 23:43:33,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409940575bc060ce161c88 2025-12-17 23:43:33,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409940575bc060ce161c88 HTTP/1.1" 502 150 2025-12-17 23:43:33,960 - lava_callback - ERROR - Error processing callback for node 69409940575bc060ce161c88: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409940575bc060ce161c88 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409940575bc060ce161c88 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409940575bc060ce161c88 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:33,982 - lava_callback - INFO - Starting async job submit for node 69409960575bc060ce161ccd 2025-12-17 23:43:33,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:33,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:33,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409960575bc060ce161ccd HTTP/1.1" 502 150 2025-12-17 23:43:33,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409960575bc060ce161ccd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:33,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409960575bc060ce161ccd 2025-12-17 23:43:33,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409960575bc060ce161ccd HTTP/1.1" 502 150 2025-12-17 23:43:33,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409960575bc060ce161ccd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:34,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c27575bc060ce162508 HTTP/1.1" 502 150 2025-12-17 23:43:34,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c27575bc060ce162508'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:34,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c27575bc060ce162508 2025-12-17 23:43:34,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c27575bc060ce162508 HTTP/1.1" 502 150 2025-12-17 23:43:34,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c27575bc060ce162508'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:34,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d1b575bc060ce16274b 2025-12-17 23:43:34,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1b575bc060ce16274b HTTP/1.1" 502 150 2025-12-17 23:43:34,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d1b575bc060ce16274b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:35,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409960575bc060ce161ccd 2025-12-17 23:43:35,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409960575bc060ce161ccd HTTP/1.1" 502 150 2025-12-17 23:43:35,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409960575bc060ce161ccd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:36,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c27575bc060ce162508 2025-12-17 23:43:36,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c27575bc060ce162508 HTTP/1.1" 502 150 2025-12-17 23:43:36,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c27575bc060ce162508'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44594 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:43:37,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409956575bc060ce161cb6 2025-12-17 23:43:37,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409942575bc060ce161c91 2025-12-17 23:43:37,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409942575bc060ce161c91 HTTP/1.1" 502 150 2025-12-17 23:43:37,430 - lava_callback - ERROR - Error processing callback for node 69409942575bc060ce161c91: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409942575bc060ce161c91 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409942575bc060ce161c91 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409942575bc060ce161c91 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:37,432 - lava_callback - INFO - Starting async job submit for node 69409962575bc060ce161cd4 2025-12-17 23:43:37,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:37,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409962575bc060ce161cd4 HTTP/1.1" 502 150 2025-12-17 23:43:37,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409962575bc060ce161cd4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:37,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409962575bc060ce161cd4 2025-12-17 23:43:37,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409962575bc060ce161cd4 HTTP/1.1" 502 150 2025-12-17 23:43:37,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409962575bc060ce161cd4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:37,546 - lava_callback - INFO - Received callback for node 69409d22575bc060ce16275e INFO: 172.21.0.1:44606 - "POST /node/69409d22575bc060ce16275e HTTP/1.0" 202 Accepted 2025-12-17 23:43:37,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940995d575bc060ce161cc2 2025-12-17 23:43:37,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940995d575bc060ce161cc2 HTTP/1.1" 502 150 2025-12-17 23:43:37,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940995d575bc060ce161cc2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:37,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409944575bc060ce161c94 HTTP/1.1" 502 150 2025-12-17 23:43:37,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409944575bc060ce161c94'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:38,204 - lava_callback - INFO - Received callback for node 69409d27575bc060ce162768 INFO: 172.21.0.1:44614 - "POST /node/69409d27575bc060ce162768 HTTP/1.0" 202 Accepted 2025-12-17 23:43:38,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d1b575bc060ce16274b 2025-12-17 23:43:38,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1b575bc060ce16274b HTTP/1.1" 502 150 2025-12-17 23:43:38,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d1b575bc060ce16274b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:39,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409962575bc060ce161cd4 2025-12-17 23:43:39,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409962575bc060ce161cd4 HTTP/1.1" 502 150 2025-12-17 23:43:39,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409962575bc060ce161cd4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:39,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409960575bc060ce161ccd 2025-12-17 23:43:39,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409960575bc060ce161ccd HTTP/1.1" 502 150 2025-12-17 23:43:39,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409960575bc060ce161ccd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:40,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c27575bc060ce162508 2025-12-17 23:43:40,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c27575bc060ce162508 HTTP/1.1" 502 150 2025-12-17 23:43:40,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c27575bc060ce162508'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:41,320 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409950575bc060ce161cb0 2025-12-17 23:43:41,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409950575bc060ce161cb0 HTTP/1.1" 502 150 2025-12-17 23:43:41,321 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409950575bc060ce161cb0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:41,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409954575bc060ce161cb3 2025-12-17 23:43:41,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409954575bc060ce161cb3 HTTP/1.1" 502 150 2025-12-17 23:43:41,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409954575bc060ce161cb3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:43,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409962575bc060ce161cd4 2025-12-17 23:43:43,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409962575bc060ce161cd4 HTTP/1.1" 502 150 2025-12-17 23:43:43,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409962575bc060ce161cd4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:44,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409946575bc060ce161c97 2025-12-17 23:43:44,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409946575bc060ce161c97 HTTP/1.1" 502 150 2025-12-17 23:43:44,928 - lava_callback - ERROR - Error processing callback for node 69409946575bc060ce161c97: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409946575bc060ce161c97 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409946575bc060ce161c97 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409946575bc060ce161c97 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:44,930 - lava_callback - INFO - Starting async job submit for node 69409964575bc060ce161cd7 2025-12-17 23:43:44,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:44,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409964575bc060ce161cd7 HTTP/1.1" 502 150 2025-12-17 23:43:44,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409964575bc060ce161cd7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:44,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409964575bc060ce161cd7 2025-12-17 23:43:44,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409964575bc060ce161cd7 HTTP/1.1" 502 150 2025-12-17 23:43:44,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409964575bc060ce161cd7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:45,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409947575bc060ce161c9a 2025-12-17 23:43:45,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409947575bc060ce161c9a HTTP/1.1" 502 150 2025-12-17 23:43:45,667 - lava_callback - ERROR - Error processing callback for node 69409947575bc060ce161c9a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409947575bc060ce161c9a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409947575bc060ce161c9a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409947575bc060ce161c9a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:45,669 - lava_callback - INFO - Starting async job submit for node 69409966575bc060ce161cdc 2025-12-17 23:43:45,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:45,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409966575bc060ce161cdc HTTP/1.1" 502 150 2025-12-17 23:43:45,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409966575bc060ce161cdc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:45,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409966575bc060ce161cdc 2025-12-17 23:43:45,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409966575bc060ce161cdc HTTP/1.1" 502 150 2025-12-17 23:43:45,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409966575bc060ce161cdc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:45,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940995d575bc060ce161cc2 2025-12-17 23:43:45,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940995d575bc060ce161cc2 HTTP/1.1" 502 150 2025-12-17 23:43:45,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940995d575bc060ce161cc2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:46,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940994a575bc060ce161c9d 2025-12-17 23:43:46,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994a575bc060ce161c9d HTTP/1.1" 502 150 2025-12-17 23:43:46,437 - lava_callback - ERROR - Error processing callback for node 6940994a575bc060ce161c9d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940994a575bc060ce161c9d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940994a575bc060ce161c9d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940994a575bc060ce161c9d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:46,439 - lava_callback - INFO - Starting async job submit for node 69409969575bc060ce161ce4 2025-12-17 23:43:46,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:46,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409969575bc060ce161ce4 HTTP/1.1" 502 150 2025-12-17 23:43:46,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409969575bc060ce161ce4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:46,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409969575bc060ce161ce4 2025-12-17 23:43:46,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409969575bc060ce161ce4 HTTP/1.1" 502 150 2025-12-17 23:43:46,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409969575bc060ce161ce4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36298 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:43:46,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d1b575bc060ce16274b 2025-12-17 23:43:46,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1b575bc060ce16274b HTTP/1.1" 502 150 2025-12-17 23:43:46,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d1b575bc060ce16274b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:46,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409964575bc060ce161cd7 2025-12-17 23:43:46,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409964575bc060ce161cd7 HTTP/1.1" 502 150 2025-12-17 23:43:46,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409964575bc060ce161cd7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:47,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409966575bc060ce161cdc 2025-12-17 23:43:47,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409966575bc060ce161cdc HTTP/1.1" 502 150 2025-12-17 23:43:47,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409966575bc060ce161cdc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:47,723 - lava_callback - INFO - Received callback for node 69409910575bc060ce161c13 INFO: 172.21.0.1:36302 - "POST /node/69409910575bc060ce161c13 HTTP/1.0" 202 Accepted 2025-12-17 23:43:47,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409960575bc060ce161ccd 2025-12-17 23:43:48,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409960575bc060ce161ccd HTTP/1.1" 502 150 2025-12-17 23:43:48,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409960575bc060ce161ccd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:48,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c27575bc060ce162508 2025-12-17 23:43:48,207 - lava_callback - INFO - Received callback for node 69409d1f575bc060ce162753 INFO: 172.21.0.1:36312 - "POST /node/69409d1f575bc060ce162753 HTTP/1.0" 202 Accepted 2025-12-17 23:43:48,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409969575bc060ce161ce4 2025-12-17 23:43:48,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409969575bc060ce161ce4 HTTP/1.1" 502 150 2025-12-17 23:43:48,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409969575bc060ce161ce4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:48,854 - lava_callback - INFO - Received callback for node 69409d25575bc060ce162765 INFO: 172.21.0.1:36320 - "POST /node/69409d25575bc060ce162765 HTTP/1.0" 202 Accepted 2025-12-17 23:43:49,336 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409807575bc060ce1619a2 2025-12-17 23:43:49,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409807575bc060ce1619a2 HTTP/1.1" 502 150 2025-12-17 23:43:49,338 - lava_callback - ERROR - Error processing callback for node 69409807575bc060ce1619a2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409807575bc060ce1619a2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409807575bc060ce1619a2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409807575bc060ce1619a2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:49,340 - lava_callback - INFO - Starting async job submit for node 6940996b575bc060ce161ce7 2025-12-17 23:43:49,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:49,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940996b575bc060ce161ce7 HTTP/1.1" 502 150 2025-12-17 23:43:49,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940996b575bc060ce161ce7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:49,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940996b575bc060ce161ce7 2025-12-17 23:43:49,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940996b575bc060ce161ce7 HTTP/1.1" 502 150 2025-12-17 23:43:49,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940996b575bc060ce161ce7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:49,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940994c575bc060ce161caa 2025-12-17 23:43:49,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994c575bc060ce161caa HTTP/1.1" 502 150 2025-12-17 23:43:49,384 - lava_callback - ERROR - Error processing callback for node 6940994c575bc060ce161caa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940994c575bc060ce161caa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940994c575bc060ce161caa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940994c575bc060ce161caa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:49,386 - lava_callback - INFO - Starting async job submit for node 69409970575bc060ce161ced 2025-12-17 23:43:49,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:49,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409970575bc060ce161ced HTTP/1.1" 502 150 2025-12-17 23:43:49,391 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409970575bc060ce161ced'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:49,391 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409970575bc060ce161ced 2025-12-17 23:43:49,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409970575bc060ce161ced HTTP/1.1" 502 150 2025-12-17 23:43:49,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409970575bc060ce161ced'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:49,509 - lava_callback - INFO - Received callback for node 69409d29575bc060ce16276b INFO: 172.21.0.1:36326 - "POST /node/69409d29575bc060ce16276b HTTP/1.0" 202 Accepted 2025-12-17 23:43:49,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940994e575bc060ce161cad 2025-12-17 23:43:49,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940994e575bc060ce161cad HTTP/1.1" 502 150 2025-12-17 23:43:49,632 - lava_callback - ERROR - Error processing callback for node 6940994e575bc060ce161cad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940994e575bc060ce161cad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940994e575bc060ce161cad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940994e575bc060ce161cad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:49,634 - lava_callback - INFO - Starting async job submit for node 6940996d575bc060ce161cea 2025-12-17 23:43:49,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:49,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940996d575bc060ce161cea HTTP/1.1" 502 150 2025-12-17 23:43:49,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940996d575bc060ce161cea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:49,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940996d575bc060ce161cea 2025-12-17 23:43:49,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940996d575bc060ce161cea HTTP/1.1" 502 150 2025-12-17 23:43:49,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940996d575bc060ce161cea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:49,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940995a575bc060ce161cbc 2025-12-17 23:43:49,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940995a575bc060ce161cbc HTTP/1.1" 502 150 2025-12-17 23:43:49,808 - lava_callback - ERROR - Error processing callback for node 6940995a575bc060ce161cbc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940995a575bc060ce161cbc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940995a575bc060ce161cbc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940995a575bc060ce161cbc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:49,810 - lava_callback - INFO - Starting async job submit for node 69409975575bc060ce161cfd 2025-12-17 23:43:49,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:49,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409975575bc060ce161cfd HTTP/1.1" 502 150 2025-12-17 23:43:49,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409975575bc060ce161cfd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:49,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409975575bc060ce161cfd 2025-12-17 23:43:49,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409975575bc060ce161cfd HTTP/1.1" 502 150 2025-12-17 23:43:49,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409975575bc060ce161cfd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:50,172 - lava_callback - INFO - Received callback for node 69409d2b575bc060ce16276e INFO: 172.21.0.1:59848 - "POST /node/69409d2b575bc060ce16276e HTTP/1.0" 202 Accepted 2025-12-17 23:43:50,835 - lava_callback - INFO - Received callback for node 69409d2d575bc060ce162771 INFO: 172.21.0.1:59850 - "POST /node/69409d2d575bc060ce162771 HTTP/1.0" 202 Accepted 2025-12-17 23:43:50,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409964575bc060ce161cd7 2025-12-17 23:43:50,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409964575bc060ce161cd7 HTTP/1.1" 502 150 2025-12-17 23:43:50,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409964575bc060ce161cd7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:51,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940996b575bc060ce161ce7 2025-12-17 23:43:51,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940996b575bc060ce161ce7 HTTP/1.1" 502 150 2025-12-17 23:43:51,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940996b575bc060ce161ce7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:51,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409970575bc060ce161ced 2025-12-17 23:43:51,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409970575bc060ce161ced HTTP/1.1" 502 150 2025-12-17 23:43:51,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409970575bc060ce161ced'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:51,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409962575bc060ce161cd4 2025-12-17 23:43:51,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409962575bc060ce161cd4 HTTP/1.1" 502 150 2025-12-17 23:43:51,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409962575bc060ce161cd4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:51,502 - lava_callback - INFO - Received callback for node 69409d2f575bc060ce162774 INFO: 172.21.0.1:59862 - "POST /node/69409d2f575bc060ce162774 HTTP/1.0" 202 Accepted 2025-12-17 23:43:51,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940996d575bc060ce161cea 2025-12-17 23:43:51,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940996d575bc060ce161cea HTTP/1.1" 502 150 2025-12-17 23:43:51,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940996d575bc060ce161cea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:51,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409966575bc060ce161cdc 2025-12-17 23:43:51,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409966575bc060ce161cdc HTTP/1.1" 502 150 2025-12-17 23:43:51,679 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409966575bc060ce161cdc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:51,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409975575bc060ce161cfd 2025-12-17 23:43:51,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409975575bc060ce161cfd HTTP/1.1" 502 150 2025-12-17 23:43:51,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409975575bc060ce161cfd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:52,151 - lava_callback - INFO - Received callback for node 69409d33575bc060ce16277a INFO: 172.21.0.1:59864 - "POST /node/69409d33575bc060ce16277a HTTP/1.0" 202 Accepted 2025-12-17 23:43:52,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409969575bc060ce161ce4 2025-12-17 23:43:52,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409969575bc060ce161ce4 HTTP/1.1" 502 150 2025-12-17 23:43:52,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409969575bc060ce161ce4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:52,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409956575bc060ce161cb6 HTTP/1.1" 502 150 2025-12-17 23:43:52,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409956575bc060ce161cb6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:53,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409944575bc060ce161c94 2025-12-17 23:43:53,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409944575bc060ce161c94 HTTP/1.1" 502 150 2025-12-17 23:43:53,909 - lava_callback - ERROR - Error processing callback for node 69409944575bc060ce161c94: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409944575bc060ce161c94 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409944575bc060ce161c94 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409944575bc060ce161c94 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:53,911 - lava_callback - INFO - Starting async job submit for node 6940997b575bc060ce161d0c 2025-12-17 23:43:53,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:53,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997b575bc060ce161d0c HTTP/1.1" 502 150 2025-12-17 23:43:53,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940997b575bc060ce161d0c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:53,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940997b575bc060ce161d0c 2025-12-17 23:43:53,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997b575bc060ce161d0c HTTP/1.1" 502 150 2025-12-17 23:43:53,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940997b575bc060ce161d0c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:55,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940996b575bc060ce161ce7 2025-12-17 23:43:55,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940996b575bc060ce161ce7 HTTP/1.1" 502 150 2025-12-17 23:43:55,350 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940996b575bc060ce161ce7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:55,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409970575bc060ce161ced 2025-12-17 23:43:55,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409970575bc060ce161ced HTTP/1.1" 502 150 2025-12-17 23:43:55,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409970575bc060ce161ced'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:55,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940996d575bc060ce161cea 2025-12-17 23:43:55,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940996d575bc060ce161cea HTTP/1.1" 502 150 2025-12-17 23:43:55,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940996d575bc060ce161cea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:55,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409975575bc060ce161cfd 2025-12-17 23:43:55,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409975575bc060ce161cfd HTTP/1.1" 502 150 2025-12-17 23:43:55,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409975575bc060ce161cfd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:55,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940997b575bc060ce161d0c 2025-12-17 23:43:55,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997b575bc060ce161d0c HTTP/1.1" 502 150 2025-12-17 23:43:55,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940997b575bc060ce161d0c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59872 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:43:57,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409950575bc060ce161cb0 2025-12-17 23:43:57,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409950575bc060ce161cb0 HTTP/1.1" 502 150 2025-12-17 23:43:57,323 - lava_callback - ERROR - Error processing callback for node 69409950575bc060ce161cb0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409950575bc060ce161cb0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409950575bc060ce161cb0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409950575bc060ce161cb0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:57,325 - lava_callback - INFO - Starting async job submit for node 69409973575bc060ce161cf4 2025-12-17 23:43:57,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:57,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409973575bc060ce161cf4 HTTP/1.1" 502 150 2025-12-17 23:43:57,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409973575bc060ce161cf4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:57,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409973575bc060ce161cf4 2025-12-17 23:43:57,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409973575bc060ce161cf4 HTTP/1.1" 502 150 2025-12-17 23:43:57,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409973575bc060ce161cf4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:57,530 - lava_callback - INFO - Received callback for node 69409d31575bc060ce162777 INFO: 172.21.0.1:59888 - "POST /node/69409d31575bc060ce162777 HTTP/1.0" 202 Accepted 2025-12-17 23:43:57,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409954575bc060ce161cb3 2025-12-17 23:43:57,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409954575bc060ce161cb3 HTTP/1.1" 502 150 2025-12-17 23:43:57,853 - lava_callback - ERROR - Error processing callback for node 69409954575bc060ce161cb3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409954575bc060ce161cb3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409954575bc060ce161cb3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409954575bc060ce161cb3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:43:57,855 - lava_callback - INFO - Starting async job submit for node 69409977575bc060ce161d02 2025-12-17 23:43:57,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:43:57,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409977575bc060ce161d02 HTTP/1.1" 502 150 2025-12-17 23:43:57,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409977575bc060ce161d02'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:57,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409977575bc060ce161d02 2025-12-17 23:43:57,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409977575bc060ce161d02 HTTP/1.1" 502 150 2025-12-17 23:43:57,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409977575bc060ce161d02'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:58,207 - lava_callback - INFO - Received callback for node 69409d35575bc060ce162784 INFO: 172.21.0.1:59892 - "POST /node/69409d35575bc060ce162784 HTTP/1.0" 202 Accepted 2025-12-17 23:43:58,908 - lava_callback - INFO - Received callback for node 69409d3b575bc060ce162797 INFO: 172.21.0.1:59900 - "POST /node/69409d3b575bc060ce162797 HTTP/1.0" 202 Accepted 2025-12-17 23:43:58,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409964575bc060ce161cd7 2025-12-17 23:43:58,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409964575bc060ce161cd7 HTTP/1.1" 502 150 2025-12-17 23:43:58,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409964575bc060ce161cd7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:59,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409973575bc060ce161cf4 2025-12-17 23:43:59,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409973575bc060ce161cf4 HTTP/1.1" 502 150 2025-12-17 23:43:59,334 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409973575bc060ce161cf4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:59,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409966575bc060ce161cdc 2025-12-17 23:43:59,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409966575bc060ce161cdc HTTP/1.1" 502 150 2025-12-17 23:43:59,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409966575bc060ce161cdc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:59,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409977575bc060ce161d02 2025-12-17 23:43:59,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409977575bc060ce161d02 HTTP/1.1" 502 150 2025-12-17 23:43:59,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409977575bc060ce161d02'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:43:59,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940997b575bc060ce161d0c 2025-12-17 23:43:59,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997b575bc060ce161d0c HTTP/1.1" 502 150 2025-12-17 23:43:59,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940997b575bc060ce161d0c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:00,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409969575bc060ce161ce4 2025-12-17 23:44:00,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409969575bc060ce161ce4 HTTP/1.1" 502 150 2025-12-17 23:44:00,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409969575bc060ce161ce4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:00,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409956575bc060ce161cb6 2025-12-17 23:44:00,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409956575bc060ce161cb6 HTTP/1.1" 502 150 2025-12-17 23:44:00,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409956575bc060ce161cb6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:01,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940995d575bc060ce161cc2 2025-12-17 23:44:01,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940995d575bc060ce161cc2 HTTP/1.1" 502 150 2025-12-17 23:44:01,905 - lava_callback - ERROR - Error processing callback for node 6940995d575bc060ce161cc2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940995d575bc060ce161cc2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940995d575bc060ce161cc2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940995d575bc060ce161cc2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:01,908 - lava_callback - INFO - Starting async job submit for node 69409979575bc060ce161d09 2025-12-17 23:44:01,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:01,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409979575bc060ce161d09 HTTP/1.1" 502 150 2025-12-17 23:44:01,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409979575bc060ce161d09'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:01,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409979575bc060ce161d09 2025-12-17 23:44:01,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409979575bc060ce161d09 HTTP/1.1" 502 150 2025-12-17 23:44:01,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409979575bc060ce161d09'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:02,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d1b575bc060ce16274b 2025-12-17 23:44:02,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1b575bc060ce16274b HTTP/1.1" 502 150 2025-12-17 23:44:02,942 - lava_callback - ERROR - Error processing callback for node 69409d1b575bc060ce16274b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d1b575bc060ce16274b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d1b575bc060ce16274b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d1b575bc060ce16274b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:02,945 - lava_callback - INFO - Starting async job submit for node 6940997d575bc060ce161d17 2025-12-17 23:44:02,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:02,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997d575bc060ce161d17 HTTP/1.1" 502 150 2025-12-17 23:44:02,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940997d575bc060ce161d17'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:02,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940997d575bc060ce161d17 2025-12-17 23:44:02,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997d575bc060ce161d17 HTTP/1.1" 502 150 2025-12-17 23:44:02,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940997d575bc060ce161d17'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:03,334 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409973575bc060ce161cf4 2025-12-17 23:44:03,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409973575bc060ce161cf4 HTTP/1.1" 502 150 2025-12-17 23:44:03,335 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409973575bc060ce161cf4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:03,351 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940996b575bc060ce161ce7 2025-12-17 23:44:03,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940996b575bc060ce161ce7 HTTP/1.1" 502 150 2025-12-17 23:44:03,351 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940996b575bc060ce161ce7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:03,395 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409970575bc060ce161ced 2025-12-17 23:44:03,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409970575bc060ce161ced HTTP/1.1" 502 150 2025-12-17 23:44:03,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409970575bc060ce161ced'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:03,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c27575bc060ce162508 HTTP/1.1" 502 150 2025-12-17 23:44:03,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c27575bc060ce162508'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41398 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:44:03,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940996d575bc060ce161cea 2025-12-17 23:44:03,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940996d575bc060ce161cea HTTP/1.1" 502 150 2025-12-17 23:44:03,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940996d575bc060ce161cea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:03,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409975575bc060ce161cfd 2025-12-17 23:44:03,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409975575bc060ce161cfd HTTP/1.1" 502 150 2025-12-17 23:44:03,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409975575bc060ce161cfd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:03,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409977575bc060ce161d02 2025-12-17 23:44:03,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409977575bc060ce161d02 HTTP/1.1" 502 150 2025-12-17 23:44:03,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409977575bc060ce161d02'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:03,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409979575bc060ce161d09 2025-12-17 23:44:03,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409979575bc060ce161d09 HTTP/1.1" 502 150 2025-12-17 23:44:03,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409979575bc060ce161d09'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:04,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409960575bc060ce161ccd 2025-12-17 23:44:04,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409960575bc060ce161ccd HTTP/1.1" 502 150 2025-12-17 23:44:04,001 - lava_callback - ERROR - Error processing callback for node 69409960575bc060ce161ccd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409960575bc060ce161ccd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409960575bc060ce161ccd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409960575bc060ce161ccd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:04,003 - lava_callback - INFO - Starting async job submit for node 6940997f575bc060ce161d1c 2025-12-17 23:44:04,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:04,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997f575bc060ce161d1c HTTP/1.1" 502 150 2025-12-17 23:44:04,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940997f575bc060ce161d1c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:04,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940997f575bc060ce161d1c 2025-12-17 23:44:04,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997f575bc060ce161d1c HTTP/1.1" 502 150 2025-12-17 23:44:04,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940997f575bc060ce161d1c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:04,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940997d575bc060ce161d17 2025-12-17 23:44:04,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997d575bc060ce161d17 HTTP/1.1" 502 150 2025-12-17 23:44:04,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940997d575bc060ce161d17'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:06,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940997f575bc060ce161d1c 2025-12-17 23:44:06,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997f575bc060ce161d1c HTTP/1.1" 502 150 2025-12-17 23:44:06,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940997f575bc060ce161d1c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41402 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:44:07,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409962575bc060ce161cd4 2025-12-17 23:44:07,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409962575bc060ce161cd4 HTTP/1.1" 502 150 2025-12-17 23:44:07,447 - lava_callback - ERROR - Error processing callback for node 69409962575bc060ce161cd4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409962575bc060ce161cd4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409962575bc060ce161cd4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409962575bc060ce161cd4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:07,449 - lava_callback - INFO - Starting async job submit for node 69409c3f575bc060ce16254a 2025-12-17 23:44:07,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:07,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3f575bc060ce16254a HTTP/1.1" 502 150 2025-12-17 23:44:07,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3f575bc060ce16254a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:07,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3f575bc060ce16254a 2025-12-17 23:44:07,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3f575bc060ce16254a HTTP/1.1" 502 150 2025-12-17 23:44:07,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3f575bc060ce16254a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:07,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409979575bc060ce161d09 2025-12-17 23:44:07,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409979575bc060ce161d09 HTTP/1.1" 502 150 2025-12-17 23:44:07,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409979575bc060ce161d09'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:07,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940997b575bc060ce161d0c 2025-12-17 23:44:07,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997b575bc060ce161d0c HTTP/1.1" 502 150 2025-12-17 23:44:07,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940997b575bc060ce161d0c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:08,221 - lava_callback - INFO - Received callback for node 69409d37575bc060ce16278d INFO: 172.21.0.1:41416 - "POST /node/69409d37575bc060ce16278d HTTP/1.0" 202 Accepted 2025-12-17 23:44:08,874 - lava_callback - INFO - Received callback for node 69409d39575bc060ce162794 INFO: 172.21.0.1:41428 - "POST /node/69409d39575bc060ce162794 HTTP/1.0" 202 Accepted 2025-12-17 23:44:08,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940997d575bc060ce161d17 2025-12-17 23:44:08,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997d575bc060ce161d17 HTTP/1.1" 502 150 2025-12-17 23:44:08,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940997d575bc060ce161d17'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:09,148 - lava_callback - INFO - Received callback for node 69409d40575bc060ce1627a0 INFO: 172.21.0.1:41430 - "POST /node/69409d40575bc060ce1627a0 HTTP/1.0" 202 Accepted 2025-12-17 23:44:09,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3f575bc060ce16254a 2025-12-17 23:44:09,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3f575bc060ce16254a HTTP/1.1" 502 150 2025-12-17 23:44:09,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3f575bc060ce16254a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:09,512 - lava_callback - INFO - Received callback for node 69409d3d575bc060ce16279a INFO: 172.21.0.1:41438 - "POST /node/69409d3d575bc060ce16279a HTTP/1.0" 202 Accepted 2025-12-17 23:44:10,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940997f575bc060ce161d1c 2025-12-17 23:44:10,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997f575bc060ce161d1c HTTP/1.1" 502 150 2025-12-17 23:44:10,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940997f575bc060ce161d1c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:11,335 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409973575bc060ce161cf4 2025-12-17 23:44:11,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409973575bc060ce161cf4 HTTP/1.1" 502 150 2025-12-17 23:44:11,336 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409973575bc060ce161cf4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:11,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409977575bc060ce161d02 2025-12-17 23:44:11,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409977575bc060ce161d02 HTTP/1.1" 502 150 2025-12-17 23:44:11,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409977575bc060ce161d02'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:13,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3f575bc060ce16254a 2025-12-17 23:44:13,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3f575bc060ce16254a HTTP/1.1" 502 150 2025-12-17 23:44:13,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3f575bc060ce16254a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:14,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409964575bc060ce161cd7 2025-12-17 23:44:14,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409964575bc060ce161cd7 HTTP/1.1" 502 150 2025-12-17 23:44:14,957 - lava_callback - ERROR - Error processing callback for node 69409964575bc060ce161cd7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409964575bc060ce161cd7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409964575bc060ce161cd7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409964575bc060ce161cd7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:14,960 - lava_callback - INFO - Starting async job submit for node 69409ca3575bc060ce162641 2025-12-17 23:44:14,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:14,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca3575bc060ce162641 HTTP/1.1" 502 150 2025-12-17 23:44:14,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca3575bc060ce162641'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:14,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca3575bc060ce162641 2025-12-17 23:44:14,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca3575bc060ce162641 HTTP/1.1" 502 150 2025-12-17 23:44:14,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca3575bc060ce162641'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47198 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:44:15,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409966575bc060ce161cdc 2025-12-17 23:44:15,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409966575bc060ce161cdc HTTP/1.1" 502 150 2025-12-17 23:44:15,682 - lava_callback - ERROR - Error processing callback for node 69409966575bc060ce161cdc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409966575bc060ce161cdc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409966575bc060ce161cdc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409966575bc060ce161cdc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:15,684 - lava_callback - INFO - Starting async job submit for node 69409994575bc060ce161d45 2025-12-17 23:44:15,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:15,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409994575bc060ce161d45 HTTP/1.1" 502 150 2025-12-17 23:44:15,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409994575bc060ce161d45'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:15,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409994575bc060ce161d45 2025-12-17 23:44:15,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409994575bc060ce161d45 HTTP/1.1" 502 150 2025-12-17 23:44:15,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409994575bc060ce161d45'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:15,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409979575bc060ce161d09 2025-12-17 23:44:15,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409979575bc060ce161d09 HTTP/1.1" 502 150 2025-12-17 23:44:15,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409979575bc060ce161d09'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:16,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409969575bc060ce161ce4 2025-12-17 23:44:16,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409969575bc060ce161ce4 HTTP/1.1" 502 150 2025-12-17 23:44:16,458 - lava_callback - ERROR - Error processing callback for node 69409969575bc060ce161ce4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409969575bc060ce161ce4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409969575bc060ce161ce4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409969575bc060ce161ce4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:16,460 - lava_callback - INFO - Starting async job submit for node 69409996575bc060ce161d48 2025-12-17 23:44:16,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:16,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409996575bc060ce161d48 HTTP/1.1" 502 150 2025-12-17 23:44:16,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409996575bc060ce161d48'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:16,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409996575bc060ce161d48 2025-12-17 23:44:16,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409996575bc060ce161d48 HTTP/1.1" 502 150 2025-12-17 23:44:16,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409996575bc060ce161d48'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:16,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409956575bc060ce161cb6 2025-12-17 23:44:16,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409956575bc060ce161cb6 HTTP/1.1" 502 150 2025-12-17 23:44:16,499 - lava_callback - ERROR - Error processing callback for node 69409956575bc060ce161cb6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409956575bc060ce161cb6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409956575bc060ce161cb6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409956575bc060ce161cb6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:16,500 - lava_callback - INFO - Starting async job submit for node 6940998f575bc060ce161d3f 2025-12-17 23:44:16,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:16,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998f575bc060ce161d3f HTTP/1.1" 502 150 2025-12-17 23:44:16,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940998f575bc060ce161d3f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:16,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940998f575bc060ce161d3f 2025-12-17 23:44:16,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998f575bc060ce161d3f HTTP/1.1" 502 150 2025-12-17 23:44:16,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940998f575bc060ce161d3f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47200 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:44:16,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940997d575bc060ce161d17 2025-12-17 23:44:16,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997d575bc060ce161d17 HTTP/1.1" 502 150 2025-12-17 23:44:16,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940997d575bc060ce161d17'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:16,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca3575bc060ce162641 2025-12-17 23:44:16,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca3575bc060ce162641 HTTP/1.1" 502 150 2025-12-17 23:44:16,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca3575bc060ce162641'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:17,533 - lava_callback - INFO - Received callback for node 69409d4f575bc060ce1627cc INFO: 172.21.0.1:47216 - "POST /node/69409d4f575bc060ce1627cc HTTP/1.0" 202 Accepted 2025-12-17 23:44:17,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409994575bc060ce161d45 2025-12-17 23:44:17,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409994575bc060ce161d45 HTTP/1.1" 502 150 2025-12-17 23:44:17,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409994575bc060ce161d45'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:18,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940997f575bc060ce161d1c 2025-12-17 23:44:18,174 - lava_callback - INFO - Received callback for node 69409d40575bc060ce1627a3 INFO: 172.21.0.1:47232 - "POST /node/69409d40575bc060ce1627a3 HTTP/1.0" 202 Accepted 2025-12-17 23:44:18,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409996575bc060ce161d48 2025-12-17 23:44:18,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409996575bc060ce161d48 HTTP/1.1" 502 150 2025-12-17 23:44:18,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409996575bc060ce161d48'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:18,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940998f575bc060ce161d3f 2025-12-17 23:44:18,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998f575bc060ce161d3f HTTP/1.1" 502 150 2025-12-17 23:44:18,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940998f575bc060ce161d3f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:18,831 - lava_callback - INFO - Received callback for node 69409d42575bc060ce1627a6 INFO: 172.21.0.1:47236 - "POST /node/69409d42575bc060ce1627a6 HTTP/1.0" 202 Accepted 2025-12-17 23:44:19,352 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940996b575bc060ce161ce7 2025-12-17 23:44:19,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940996b575bc060ce161ce7 HTTP/1.1" 502 150 2025-12-17 23:44:19,353 - lava_callback - ERROR - Error processing callback for node 6940996b575bc060ce161ce7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940996b575bc060ce161ce7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940996b575bc060ce161ce7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940996b575bc060ce161ce7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:19,355 - lava_callback - INFO - Starting async job submit for node 69409992575bc060ce161d42 2025-12-17 23:44:19,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:19,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409992575bc060ce161d42 HTTP/1.1" 502 150 2025-12-17 23:44:19,361 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409992575bc060ce161d42'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:19,361 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409992575bc060ce161d42 2025-12-17 23:44:19,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409992575bc060ce161d42 HTTP/1.1" 502 150 2025-12-17 23:44:19,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409992575bc060ce161d42'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:19,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409970575bc060ce161ced 2025-12-17 23:44:19,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409970575bc060ce161ced HTTP/1.1" 502 150 2025-12-17 23:44:19,397 - lava_callback - ERROR - Error processing callback for node 69409970575bc060ce161ced: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409970575bc060ce161ced (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409970575bc060ce161ced (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409970575bc060ce161ced (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:19,399 - lava_callback - INFO - Starting async job submit for node 69409998575bc060ce161d50 2025-12-17 23:44:19,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:19,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409998575bc060ce161d50 HTTP/1.1" 502 150 2025-12-17 23:44:19,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409998575bc060ce161d50'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:19,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409998575bc060ce161d50 2025-12-17 23:44:19,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409998575bc060ce161d50 HTTP/1.1" 502 150 2025-12-17 23:44:19,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409998575bc060ce161d50'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:19,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c27575bc060ce162508 2025-12-17 23:44:19,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c27575bc060ce162508 HTTP/1.1" 502 150 2025-12-17 23:44:19,506 - lava_callback - ERROR - Error processing callback for node 69409c27575bc060ce162508: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c27575bc060ce162508 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c27575bc060ce162508 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c27575bc060ce162508 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:19,508 - lava_callback - INFO - Starting async job submit for node 6940999a575bc060ce161d57 2025-12-17 23:44:19,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:19,509 - lava_callback - INFO - Received callback for node 69409d43575bc060ce1627a9 INFO: 172.21.0.1:47246 - "POST /node/69409d43575bc060ce1627a9 HTTP/1.0" 202 Accepted 2025-12-17 23:44:19,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940999a575bc060ce161d57 HTTP/1.1" 502 150 2025-12-17 23:44:19,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940999a575bc060ce161d57'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:19,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940999a575bc060ce161d57 2025-12-17 23:44:19,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940999a575bc060ce161d57 HTTP/1.1" 502 150 2025-12-17 23:44:19,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940999a575bc060ce161d57'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:19,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940996d575bc060ce161cea 2025-12-17 23:44:19,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940996d575bc060ce161cea HTTP/1.1" 502 150 2025-12-17 23:44:19,647 - lava_callback - ERROR - Error processing callback for node 6940996d575bc060ce161cea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940996d575bc060ce161cea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940996d575bc060ce161cea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940996d575bc060ce161cea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:19,649 - lava_callback - INFO - Starting async job submit for node 6940999d575bc060ce161d60 2025-12-17 23:44:19,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:19,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940999d575bc060ce161d60 HTTP/1.1" 502 150 2025-12-17 23:44:19,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940999d575bc060ce161d60'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:19,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940999d575bc060ce161d60 2025-12-17 23:44:19,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940999d575bc060ce161d60 HTTP/1.1" 502 150 2025-12-17 23:44:19,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940999d575bc060ce161d60'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:19,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409975575bc060ce161cfd 2025-12-17 23:44:19,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409975575bc060ce161cfd HTTP/1.1" 502 150 2025-12-17 23:44:19,825 - lava_callback - ERROR - Error processing callback for node 69409975575bc060ce161cfd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409975575bc060ce161cfd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409975575bc060ce161cfd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409975575bc060ce161cfd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:19,827 - lava_callback - INFO - Starting async job submit for node 694099a3575bc060ce161d6d 2025-12-17 23:44:19,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:19,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a3575bc060ce161d6d HTTP/1.1" 502 150 2025-12-17 23:44:19,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a3575bc060ce161d6d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:19,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a3575bc060ce161d6d 2025-12-17 23:44:19,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a3575bc060ce161d6d HTTP/1.1" 502 150 2025-12-17 23:44:19,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a3575bc060ce161d6d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:20,184 - lava_callback - INFO - Received callback for node 69409d45575bc060ce1627ac INFO: 172.21.0.1:39652 - "POST /node/69409d45575bc060ce1627ac HTTP/1.0" 202 Accepted 2025-12-17 23:44:20,881 - lava_callback - INFO - Received callback for node 69409d47575bc060ce1627b4 INFO: 172.21.0.1:39658 - "POST /node/69409d47575bc060ce1627b4 HTTP/1.0" 202 Accepted 2025-12-17 23:44:20,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca3575bc060ce162641 2025-12-17 23:44:20,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca3575bc060ce162641 HTTP/1.1" 502 150 2025-12-17 23:44:20,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca3575bc060ce162641'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:21,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409992575bc060ce161d42 2025-12-17 23:44:21,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409992575bc060ce161d42 HTTP/1.1" 502 150 2025-12-17 23:44:21,364 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409992575bc060ce161d42'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:21,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409998575bc060ce161d50 2025-12-17 23:44:21,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409998575bc060ce161d50 HTTP/1.1" 502 150 2025-12-17 23:44:21,425 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409998575bc060ce161d50'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:21,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3f575bc060ce16254a 2025-12-17 23:44:21,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3f575bc060ce16254a HTTP/1.1" 502 150 2025-12-17 23:44:21,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3f575bc060ce16254a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:21,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940999a575bc060ce161d57 2025-12-17 23:44:21,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940999a575bc060ce161d57 HTTP/1.1" 502 150 2025-12-17 23:44:21,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940999a575bc060ce161d57'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:21,540 - lava_callback - INFO - Received callback for node 69409d4b575bc060ce1627c4 INFO: 172.21.0.1:39672 - "POST /node/69409d4b575bc060ce1627c4 HTTP/1.0" 202 Accepted 2025-12-17 23:44:21,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940999d575bc060ce161d60 2025-12-17 23:44:21,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940999d575bc060ce161d60 HTTP/1.1" 502 150 2025-12-17 23:44:21,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940999d575bc060ce161d60'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:21,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409994575bc060ce161d45 2025-12-17 23:44:21,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409994575bc060ce161d45 HTTP/1.1" 502 150 2025-12-17 23:44:21,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409994575bc060ce161d45'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:21,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a3575bc060ce161d6d 2025-12-17 23:44:21,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a3575bc060ce161d6d HTTP/1.1" 502 150 2025-12-17 23:44:21,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a3575bc060ce161d6d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:22,195 - lava_callback - INFO - Received callback for node 69409d4d575bc060ce1627c9 INFO: 172.21.0.1:39674 - "POST /node/69409d4d575bc060ce1627c9 HTTP/1.0" 202 Accepted 2025-12-17 23:44:22,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409996575bc060ce161d48 2025-12-17 23:44:22,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409996575bc060ce161d48 HTTP/1.1" 502 150 2025-12-17 23:44:22,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409996575bc060ce161d48'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:22,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940998f575bc060ce161d3f 2025-12-17 23:44:22,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998f575bc060ce161d3f HTTP/1.1" 502 150 2025-12-17 23:44:22,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940998f575bc060ce161d3f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:23,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940997b575bc060ce161d0c 2025-12-17 23:44:23,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997b575bc060ce161d0c HTTP/1.1" 502 150 2025-12-17 23:44:23,928 - lava_callback - ERROR - Error processing callback for node 6940997b575bc060ce161d0c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940997b575bc060ce161d0c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940997b575bc060ce161d0c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940997b575bc060ce161d0c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:23,929 - lava_callback - INFO - Starting async job submit for node 694230b9bb27571da24b7224 2025-12-17 23:44:23,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:23,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694230b9bb27571da24b7224 HTTP/1.1" 502 150 2025-12-17 23:44:23,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694230b9bb27571da24b7224'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:23,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694230b9bb27571da24b7224 2025-12-17 23:44:23,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694230b9bb27571da24b7224 HTTP/1.1" 502 150 2025-12-17 23:44:23,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694230b9bb27571da24b7224'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:25,364 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409992575bc060ce161d42 2025-12-17 23:44:25,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409992575bc060ce161d42 HTTP/1.1" 502 150 2025-12-17 23:44:25,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409992575bc060ce161d42'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:25,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409998575bc060ce161d50 2025-12-17 23:44:25,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409998575bc060ce161d50 HTTP/1.1" 502 150 2025-12-17 23:44:25,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409998575bc060ce161d50'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:25,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940999a575bc060ce161d57 2025-12-17 23:44:25,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940999a575bc060ce161d57 HTTP/1.1" 502 150 2025-12-17 23:44:25,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940999a575bc060ce161d57'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:25,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940999d575bc060ce161d60 2025-12-17 23:44:25,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940999d575bc060ce161d60 HTTP/1.1" 502 150 2025-12-17 23:44:25,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940999d575bc060ce161d60'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:25,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a3575bc060ce161d6d 2025-12-17 23:44:25,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a3575bc060ce161d6d HTTP/1.1" 502 150 2025-12-17 23:44:25,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a3575bc060ce161d6d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:25,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694230b9bb27571da24b7224 2025-12-17 23:44:25,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694230b9bb27571da24b7224 HTTP/1.1" 502 150 2025-12-17 23:44:25,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694230b9bb27571da24b7224'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39678 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:44:27,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409973575bc060ce161cf4 2025-12-17 23:44:27,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409973575bc060ce161cf4 HTTP/1.1" 502 150 2025-12-17 23:44:27,338 - lava_callback - ERROR - Error processing callback for node 69409973575bc060ce161cf4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409973575bc060ce161cf4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409973575bc060ce161cf4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409973575bc060ce161cf4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:27,340 - lava_callback - INFO - Starting async job submit for node 694099a5575bc060ce161d74 2025-12-17 23:44:27,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:27,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a5575bc060ce161d74 HTTP/1.1" 502 150 2025-12-17 23:44:27,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a5575bc060ce161d74'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:27,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a5575bc060ce161d74 2025-12-17 23:44:27,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a5575bc060ce161d74 HTTP/1.1" 502 150 2025-12-17 23:44:27,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a5575bc060ce161d74'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:27,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409977575bc060ce161d02 2025-12-17 23:44:27,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409977575bc060ce161d02 HTTP/1.1" 502 150 2025-12-17 23:44:27,873 - lava_callback - ERROR - Error processing callback for node 69409977575bc060ce161d02: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409977575bc060ce161d02 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409977575bc060ce161d02 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409977575bc060ce161d02 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:27,875 - lava_callback - INFO - Starting async job submit for node 694099a7575bc060ce161d77 2025-12-17 23:44:27,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:27,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a7575bc060ce161d77 HTTP/1.1" 502 150 2025-12-17 23:44:27,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a7575bc060ce161d77'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:27,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a7575bc060ce161d77 2025-12-17 23:44:27,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a7575bc060ce161d77 HTTP/1.1" 502 150 2025-12-17 23:44:27,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a7575bc060ce161d77'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:28,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca3575bc060ce162641 2025-12-17 23:44:28,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca3575bc060ce162641 HTTP/1.1" 502 150 2025-12-17 23:44:28,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca3575bc060ce162641'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:29,167 - lava_callback - INFO - Received callback for node 69409d4a575bc060ce1627bf INFO: 172.21.0.1:39680 - "POST /node/69409d4a575bc060ce1627bf HTTP/1.0" 202 Accepted 2025-12-17 23:44:29,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a5575bc060ce161d74 2025-12-17 23:44:29,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409994575bc060ce161d45 2025-12-17 23:44:29,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409994575bc060ce161d45 HTTP/1.1" 502 150 2025-12-17 23:44:29,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409994575bc060ce161d45'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:29,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a7575bc060ce161d77 2025-12-17 23:44:29,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a7575bc060ce161d77 HTTP/1.1" 502 150 2025-12-17 23:44:29,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a7575bc060ce161d77'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:29,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694230b9bb27571da24b7224 2025-12-17 23:44:29,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694230b9bb27571da24b7224 HTTP/1.1" 502 150 2025-12-17 23:44:29,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694230b9bb27571da24b7224'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:30,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409996575bc060ce161d48 2025-12-17 23:44:30,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409996575bc060ce161d48 HTTP/1.1" 502 150 2025-12-17 23:44:30,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409996575bc060ce161d48'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:30,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940998f575bc060ce161d3f 2025-12-17 23:44:30,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998f575bc060ce161d3f HTTP/1.1" 502 150 2025-12-17 23:44:30,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940998f575bc060ce161d3f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:31,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409979575bc060ce161d09 2025-12-17 23:44:31,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409979575bc060ce161d09 HTTP/1.1" 502 150 2025-12-17 23:44:31,925 - lava_callback - ERROR - Error processing callback for node 69409979575bc060ce161d09: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409979575bc060ce161d09 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409979575bc060ce161d09 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409979575bc060ce161d09 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:31,927 - lava_callback - INFO - Starting async job submit for node 694099a9575bc060ce161d7a 2025-12-17 23:44:31,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:31,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a9575bc060ce161d7a HTTP/1.1" 502 150 2025-12-17 23:44:31,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a9575bc060ce161d7a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:31,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a9575bc060ce161d7a 2025-12-17 23:44:31,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a9575bc060ce161d7a HTTP/1.1" 502 150 2025-12-17 23:44:31,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a9575bc060ce161d7a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:32,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940997d575bc060ce161d17 2025-12-17 23:44:32,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997d575bc060ce161d17 HTTP/1.1" 502 150 2025-12-17 23:44:32,959 - lava_callback - ERROR - Error processing callback for node 6940997d575bc060ce161d17: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940997d575bc060ce161d17 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940997d575bc060ce161d17 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940997d575bc060ce161d17 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:32,961 - lava_callback - INFO - Starting async job submit for node 694099ab575bc060ce161d7d 2025-12-17 23:44:32,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:32,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ab575bc060ce161d7d HTTP/1.1" 502 150 2025-12-17 23:44:32,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ab575bc060ce161d7d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:32,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ab575bc060ce161d7d 2025-12-17 23:44:32,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ab575bc060ce161d7d HTTP/1.1" 502 150 2025-12-17 23:44:32,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ab575bc060ce161d7d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:33,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409992575bc060ce161d42 2025-12-17 23:44:33,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409998575bc060ce161d50 2025-12-17 23:44:33,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409998575bc060ce161d50 HTTP/1.1" 502 150 2025-12-17 23:44:33,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409998575bc060ce161d50'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:33,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997f575bc060ce161d1c HTTP/1.1" 502 150 2025-12-17 23:44:33,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940997f575bc060ce161d1c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:33,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940999a575bc060ce161d57 2025-12-17 23:44:33,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940999a575bc060ce161d57 HTTP/1.1" 502 150 2025-12-17 23:44:33,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940999a575bc060ce161d57'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:33,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940999d575bc060ce161d60 2025-12-17 23:44:33,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940999d575bc060ce161d60 HTTP/1.1" 502 150 2025-12-17 23:44:33,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940999d575bc060ce161d60'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:33,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a3575bc060ce161d6d 2025-12-17 23:44:33,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a3575bc060ce161d6d HTTP/1.1" 502 150 2025-12-17 23:44:33,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a3575bc060ce161d6d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:33,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a7575bc060ce161d77 2025-12-17 23:44:33,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a7575bc060ce161d77 HTTP/1.1" 502 150 2025-12-17 23:44:33,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a7575bc060ce161d77'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:33,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a9575bc060ce161d7a 2025-12-17 23:44:33,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a9575bc060ce161d7a HTTP/1.1" 502 150 2025-12-17 23:44:33,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a9575bc060ce161d7a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:34,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ab575bc060ce161d7d 2025-12-17 23:44:34,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ab575bc060ce161d7d HTTP/1.1" 502 150 2025-12-17 23:44:34,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ab575bc060ce161d7d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48144 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:44:37,326 - lava_callback - INFO - Received callback for node 6940a31c575bc060ce16314c INFO: 172.21.0.1:48156 - "POST /node/6940a31c575bc060ce16314c HTTP/1.0" 202 Accepted 2025-12-17 23:44:37,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3f575bc060ce16254a 2025-12-17 23:44:37,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3f575bc060ce16254a HTTP/1.1" 502 150 2025-12-17 23:44:37,521 - lava_callback - ERROR - Error processing callback for node 69409c3f575bc060ce16254a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c3f575bc060ce16254a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c3f575bc060ce16254a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c3f575bc060ce16254a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:37,523 - lava_callback - INFO - Starting async job submit for node 694099ad575bc060ce161d80 2025-12-17 23:44:37,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:37,549 - lava_callback - INFO - Received callback for node 69409d64575bc060ce1627f8 INFO: 172.21.0.1:48168 - "POST /node/69409d64575bc060ce1627f8 HTTP/1.0" 202 Accepted 2025-12-17 23:44:37,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ad575bc060ce161d80 HTTP/1.1" 502 150 2025-12-17 23:44:37,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ad575bc060ce161d80'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:37,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ad575bc060ce161d80 2025-12-17 23:44:37,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ad575bc060ce161d80 HTTP/1.1" 502 150 2025-12-17 23:44:37,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ad575bc060ce161d80'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:37,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694230b9bb27571da24b7224 2025-12-17 23:44:37,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694230b9bb27571da24b7224 HTTP/1.1" 502 150 2025-12-17 23:44:37,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694230b9bb27571da24b7224'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:37,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a9575bc060ce161d7a 2025-12-17 23:44:37,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a9575bc060ce161d7a HTTP/1.1" 502 150 2025-12-17 23:44:37,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a9575bc060ce161d7a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:38,186 - lava_callback - INFO - Received callback for node 69409d51575bc060ce1627cf INFO: 172.21.0.1:48174 - "POST /node/69409d51575bc060ce1627cf HTTP/1.0" 202 Accepted 2025-12-17 23:44:38,834 - lava_callback - INFO - Received callback for node 69409d53575bc060ce1627d2 INFO: 172.21.0.1:48178 - "POST /node/69409d53575bc060ce1627d2 HTTP/1.0" 202 Accepted 2025-12-17 23:44:38,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ab575bc060ce161d7d 2025-12-17 23:44:38,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ab575bc060ce161d7d HTTP/1.1" 502 150 2025-12-17 23:44:38,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ab575bc060ce161d7d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:39,503 - lava_callback - INFO - Received callback for node 69409d55575bc060ce1627d5 INFO: 172.21.0.1:48186 - "POST /node/69409d55575bc060ce1627d5 HTTP/1.0" 202 Accepted 2025-12-17 23:44:39,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ad575bc060ce161d80 2025-12-17 23:44:39,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ad575bc060ce161d80 HTTP/1.1" 502 150 2025-12-17 23:44:39,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ad575bc060ce161d80'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:40,146 - lava_callback - INFO - Received callback for node 69409d57575bc060ce1627d8 INFO: 172.21.0.1:45270 - "POST /node/69409d57575bc060ce1627d8 HTTP/1.0" 202 Accepted 2025-12-17 23:44:40,813 - lava_callback - INFO - Received callback for node 69409d59575bc060ce1627db INFO: 172.21.0.1:45278 - "POST /node/69409d59575bc060ce1627db HTTP/1.0" 202 Accepted 2025-12-17 23:44:41,459 - lava_callback - INFO - Received callback for node 69409d5b575bc060ce1627e0 INFO: 172.21.0.1:45286 - "POST /node/69409d5b575bc060ce1627e0 HTTP/1.0" 202 Accepted 2025-12-17 23:44:41,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a7575bc060ce161d77 2025-12-17 23:44:41,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a7575bc060ce161d77 HTTP/1.1" 502 150 2025-12-17 23:44:41,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a7575bc060ce161d77'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:42,107 - lava_callback - INFO - Received callback for node 69409d5d575bc060ce1627e9 INFO: 172.21.0.1:45288 - "POST /node/69409d5d575bc060ce1627e9 HTTP/1.0" 202 Accepted 2025-12-17 23:44:42,748 - lava_callback - INFO - Received callback for node 69409d60575bc060ce1627f2 INFO: 172.21.0.1:45298 - "POST /node/69409d60575bc060ce1627f2 HTTP/1.0" 202 Accepted 2025-12-17 23:44:43,406 - lava_callback - INFO - Received callback for node 69409d62575bc060ce1627f5 INFO: 172.21.0.1:45306 - "POST /node/69409d62575bc060ce1627f5 HTTP/1.0" 202 Accepted 2025-12-17 23:44:43,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ad575bc060ce161d80 2025-12-17 23:44:43,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ad575bc060ce161d80 HTTP/1.1" 502 150 2025-12-17 23:44:43,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ad575bc060ce161d80'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:44,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a5575bc060ce161d74 HTTP/1.1" 502 150 2025-12-17 23:44:44,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a5575bc060ce161d74'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:44,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca3575bc060ce162641 2025-12-17 23:44:44,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca3575bc060ce162641 HTTP/1.1" 502 150 2025-12-17 23:44:44,997 - lava_callback - ERROR - Error processing callback for node 69409ca3575bc060ce162641: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca3575bc060ce162641 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca3575bc060ce162641 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca3575bc060ce162641 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:44,999 - lava_callback - INFO - Starting async job submit for node 69423ad6bb27571da24b7ab3 2025-12-17 23:44:45,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:45,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69423ad6bb27571da24b7ab3 HTTP/1.1" 502 150 2025-12-17 23:44:45,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69423ad6bb27571da24b7ab3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:45,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69423ad6bb27571da24b7ab3 2025-12-17 23:44:45,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69423ad6bb27571da24b7ab3 HTTP/1.1" 502 150 2025-12-17 23:44:45,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69423ad6bb27571da24b7ab3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:45,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409994575bc060ce161d45 2025-12-17 23:44:45,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409994575bc060ce161d45 HTTP/1.1" 502 150 2025-12-17 23:44:45,699 - lava_callback - ERROR - Error processing callback for node 69409994575bc060ce161d45: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409994575bc060ce161d45 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409994575bc060ce161d45 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409994575bc060ce161d45 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:45,700 - lava_callback - INFO - Starting async job submit for node 694099af575bc060ce161d8d 2025-12-17 23:44:45,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:45,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099af575bc060ce161d8d HTTP/1.1" 502 150 2025-12-17 23:44:45,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099af575bc060ce161d8d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:45,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099af575bc060ce161d8d 2025-12-17 23:44:45,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099af575bc060ce161d8d HTTP/1.1" 502 150 2025-12-17 23:44:45,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099af575bc060ce161d8d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:45,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a9575bc060ce161d7a 2025-12-17 23:44:45,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a9575bc060ce161d7a HTTP/1.1" 502 150 2025-12-17 23:44:45,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a9575bc060ce161d7a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:46,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409996575bc060ce161d48 2025-12-17 23:44:46,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409996575bc060ce161d48 HTTP/1.1" 502 150 2025-12-17 23:44:46,494 - lava_callback - ERROR - Error processing callback for node 69409996575bc060ce161d48: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409996575bc060ce161d48 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409996575bc060ce161d48 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409996575bc060ce161d48 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:46,496 - lava_callback - INFO - Starting async job submit for node 694099b4575bc060ce161d9d 2025-12-17 23:44:46,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:46,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b4575bc060ce161d9d HTTP/1.1" 502 150 2025-12-17 23:44:46,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b4575bc060ce161d9d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:46,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b4575bc060ce161d9d INFO: 172.21.0.1:45312 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:44:46,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b4575bc060ce161d9d HTTP/1.1" 502 150 2025-12-17 23:44:46,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b4575bc060ce161d9d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:46,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940998f575bc060ce161d3f 2025-12-17 23:44:46,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940998f575bc060ce161d3f HTTP/1.1" 502 150 2025-12-17 23:44:46,512 - lava_callback - ERROR - Error processing callback for node 6940998f575bc060ce161d3f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940998f575bc060ce161d3f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940998f575bc060ce161d3f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940998f575bc060ce161d3f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:46,513 - lava_callback - INFO - Starting async job submit for node 694099b6575bc060ce161da0 2025-12-17 23:44:46,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:46,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b6575bc060ce161da0 HTTP/1.1" 502 150 2025-12-17 23:44:46,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b6575bc060ce161da0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:46,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b6575bc060ce161da0 2025-12-17 23:44:46,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b6575bc060ce161da0 HTTP/1.1" 502 150 2025-12-17 23:44:46,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b6575bc060ce161da0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:46,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ab575bc060ce161d7d 2025-12-17 23:44:46,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ab575bc060ce161d7d HTTP/1.1" 502 150 2025-12-17 23:44:46,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ab575bc060ce161d7d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:47,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69423ad6bb27571da24b7ab3 2025-12-17 23:44:47,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69423ad6bb27571da24b7ab3 HTTP/1.1" 502 150 2025-12-17 23:44:47,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69423ad6bb27571da24b7ab3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:47,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099af575bc060ce161d8d 2025-12-17 23:44:47,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099af575bc060ce161d8d HTTP/1.1" 502 150 2025-12-17 23:44:47,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099af575bc060ce161d8d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:48,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b4575bc060ce161d9d 2025-12-17 23:44:48,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b4575bc060ce161d9d HTTP/1.1" 502 150 2025-12-17 23:44:48,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b4575bc060ce161d9d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:48,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b6575bc060ce161da0 2025-12-17 23:44:48,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b6575bc060ce161da0 HTTP/1.1" 502 150 2025-12-17 23:44:48,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b6575bc060ce161da0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:48,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a5575bc060ce161d74 2025-12-17 23:44:48,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a5575bc060ce161d74 HTTP/1.1" 502 150 2025-12-17 23:44:48,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a5575bc060ce161d74'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:48,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409992575bc060ce161d42 HTTP/1.1" 502 150 2025-12-17 23:44:48,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409992575bc060ce161d42'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:49,431 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409998575bc060ce161d50 2025-12-17 23:44:49,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409998575bc060ce161d50 HTTP/1.1" 502 150 2025-12-17 23:44:49,432 - lava_callback - ERROR - Error processing callback for node 69409998575bc060ce161d50: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409998575bc060ce161d50 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409998575bc060ce161d50 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409998575bc060ce161d50 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:49,434 - lava_callback - INFO - Starting async job submit for node 694099b8575bc060ce161da3 2025-12-17 23:44:49,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:49,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b8575bc060ce161da3 HTTP/1.1" 502 150 2025-12-17 23:44:49,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b8575bc060ce161da3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:49,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b8575bc060ce161da3 2025-12-17 23:44:49,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b8575bc060ce161da3 HTTP/1.1" 502 150 2025-12-17 23:44:49,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b8575bc060ce161da3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:49,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940997f575bc060ce161d1c 2025-12-17 23:44:49,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940997f575bc060ce161d1c HTTP/1.1" 502 150 2025-12-17 23:44:49,457 - lava_callback - ERROR - Error processing callback for node 6940997f575bc060ce161d1c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940997f575bc060ce161d1c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940997f575bc060ce161d1c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940997f575bc060ce161d1c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:49,459 - lava_callback - INFO - Starting async job submit for node 694099c2575bc060ce161db6 2025-12-17 23:44:49,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:49,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c2575bc060ce161db6 HTTP/1.1" 502 150 2025-12-17 23:44:49,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c2575bc060ce161db6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:49,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c2575bc060ce161db6 2025-12-17 23:44:49,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c2575bc060ce161db6 HTTP/1.1" 502 150 2025-12-17 23:44:49,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c2575bc060ce161db6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:49,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940999a575bc060ce161d57 2025-12-17 23:44:49,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940999a575bc060ce161d57 HTTP/1.1" 502 150 2025-12-17 23:44:49,526 - lava_callback - ERROR - Error processing callback for node 6940999a575bc060ce161d57: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940999a575bc060ce161d57 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940999a575bc060ce161d57 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940999a575bc060ce161d57 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:49,528 - lava_callback - INFO - Starting async job submit for node 69409c7b575bc060ce1625de 2025-12-17 23:44:49,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:49,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7b575bc060ce1625de HTTP/1.1" 502 150 2025-12-17 23:44:49,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7b575bc060ce1625de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:49,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7b575bc060ce1625de 2025-12-17 23:44:49,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7b575bc060ce1625de HTTP/1.1" 502 150 2025-12-17 23:44:49,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7b575bc060ce1625de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:49,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940999d575bc060ce161d60 2025-12-17 23:44:49,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940999d575bc060ce161d60 HTTP/1.1" 502 150 2025-12-17 23:44:49,662 - lava_callback - ERROR - Error processing callback for node 6940999d575bc060ce161d60: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940999d575bc060ce161d60 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940999d575bc060ce161d60 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940999d575bc060ce161d60 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:49,664 - lava_callback - INFO - Starting async job submit for node 694099ba575bc060ce161da7 2025-12-17 23:44:49,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:49,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ba575bc060ce161da7 HTTP/1.1" 502 150 2025-12-17 23:44:49,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ba575bc060ce161da7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:49,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ba575bc060ce161da7 2025-12-17 23:44:49,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ba575bc060ce161da7 HTTP/1.1" 502 150 2025-12-17 23:44:49,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ba575bc060ce161da7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:49,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a3575bc060ce161d6d 2025-12-17 23:44:49,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a3575bc060ce161d6d HTTP/1.1" 502 150 2025-12-17 23:44:49,843 - lava_callback - ERROR - Error processing callback for node 694099a3575bc060ce161d6d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099a3575bc060ce161d6d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099a3575bc060ce161d6d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099a3575bc060ce161d6d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:49,845 - lava_callback - INFO - Starting async job submit for node 69409c40575bc060ce16254d 2025-12-17 23:44:49,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:49,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c40575bc060ce16254d HTTP/1.1" 502 150 2025-12-17 23:44:49,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c40575bc060ce16254d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:49,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c40575bc060ce16254d 2025-12-17 23:44:49,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c40575bc060ce16254d HTTP/1.1" 502 150 2025-12-17 23:44:49,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c40575bc060ce16254d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:51,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69423ad6bb27571da24b7ab3 2025-12-17 23:44:51,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69423ad6bb27571da24b7ab3 HTTP/1.1" 502 150 2025-12-17 23:44:51,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69423ad6bb27571da24b7ab3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:51,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b8575bc060ce161da3 2025-12-17 23:44:51,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b8575bc060ce161da3 HTTP/1.1" 502 150 2025-12-17 23:44:51,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b8575bc060ce161da3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:51,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c2575bc060ce161db6 2025-12-17 23:44:51,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c2575bc060ce161db6 HTTP/1.1" 502 150 2025-12-17 23:44:51,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c2575bc060ce161db6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:51,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ad575bc060ce161d80 2025-12-17 23:44:51,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7b575bc060ce1625de 2025-12-17 23:44:51,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ad575bc060ce161d80 HTTP/1.1" 502 150 2025-12-17 23:44:51,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ad575bc060ce161d80'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:51,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7b575bc060ce1625de HTTP/1.1" 502 150 2025-12-17 23:44:51,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7b575bc060ce1625de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:51,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ba575bc060ce161da7 2025-12-17 23:44:51,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ba575bc060ce161da7 HTTP/1.1" 502 150 2025-12-17 23:44:51,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ba575bc060ce161da7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:51,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099af575bc060ce161d8d 2025-12-17 23:44:51,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099af575bc060ce161d8d HTTP/1.1" 502 150 2025-12-17 23:44:51,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099af575bc060ce161d8d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:51,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c40575bc060ce16254d 2025-12-17 23:44:51,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c40575bc060ce16254d HTTP/1.1" 502 150 2025-12-17 23:44:51,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c40575bc060ce16254d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:52,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b4575bc060ce161d9d 2025-12-17 23:44:52,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b4575bc060ce161d9d HTTP/1.1" 502 150 2025-12-17 23:44:52,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b4575bc060ce161d9d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:52,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b6575bc060ce161da0 2025-12-17 23:44:52,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b6575bc060ce161da0 HTTP/1.1" 502 150 2025-12-17 23:44:52,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b6575bc060ce161da0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:53,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694230b9bb27571da24b7224 2025-12-17 23:44:53,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694230b9bb27571da24b7224 HTTP/1.1" 502 150 2025-12-17 23:44:53,945 - lava_callback - ERROR - Error processing callback for node 694230b9bb27571da24b7224: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694230b9bb27571da24b7224 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694230b9bb27571da24b7224 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694230b9bb27571da24b7224 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:53,946 - lava_callback - INFO - Starting async job submit for node 694099bc575bc060ce161daa 2025-12-17 23:44:53,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:53,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099bc575bc060ce161daa HTTP/1.1" 502 150 2025-12-17 23:44:53,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099bc575bc060ce161daa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:53,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099bc575bc060ce161daa 2025-12-17 23:44:53,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099bc575bc060ce161daa HTTP/1.1" 502 150 2025-12-17 23:44:53,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099bc575bc060ce161daa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:55,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b8575bc060ce161da3 2025-12-17 23:44:55,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b8575bc060ce161da3 HTTP/1.1" 502 150 2025-12-17 23:44:55,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b8575bc060ce161da3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:55,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c2575bc060ce161db6 2025-12-17 23:44:55,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c2575bc060ce161db6 HTTP/1.1" 502 150 2025-12-17 23:44:55,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c2575bc060ce161db6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:55,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7b575bc060ce1625de 2025-12-17 23:44:55,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7b575bc060ce1625de HTTP/1.1" 502 150 2025-12-17 23:44:55,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7b575bc060ce1625de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:55,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ba575bc060ce161da7 2025-12-17 23:44:55,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ba575bc060ce161da7 HTTP/1.1" 502 150 2025-12-17 23:44:55,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ba575bc060ce161da7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:55,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c40575bc060ce16254d 2025-12-17 23:44:55,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c40575bc060ce16254d HTTP/1.1" 502 150 2025-12-17 23:44:55,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c40575bc060ce16254d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:55,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099bc575bc060ce161daa 2025-12-17 23:44:55,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099bc575bc060ce161daa HTTP/1.1" 502 150 2025-12-17 23:44:55,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099bc575bc060ce161daa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40210 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:44:56,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a5575bc060ce161d74 2025-12-17 23:44:56,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a5575bc060ce161d74 HTTP/1.1" 502 150 2025-12-17 23:44:56,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099a5575bc060ce161d74'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:57,565 - lava_callback - INFO - Received callback for node 69409d66575bc060ce1627fb INFO: 172.21.0.1:40224 - "POST /node/69409d66575bc060ce1627fb HTTP/1.0" 202 Accepted 2025-12-17 23:44:57,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a7575bc060ce161d77 2025-12-17 23:44:57,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a7575bc060ce161d77 HTTP/1.1" 502 150 2025-12-17 23:44:57,893 - lava_callback - ERROR - Error processing callback for node 694099a7575bc060ce161d77: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099a7575bc060ce161d77 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099a7575bc060ce161d77 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099a7575bc060ce161d77 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:44:57,895 - lava_callback - INFO - Starting async job submit for node 694099bd575bc060ce161dad 2025-12-17 23:44:57,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:44:57,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099bd575bc060ce161dad HTTP/1.1" 502 150 2025-12-17 23:44:57,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099bd575bc060ce161dad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:57,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099bd575bc060ce161dad 2025-12-17 23:44:57,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099bd575bc060ce161dad HTTP/1.1" 502 150 2025-12-17 23:44:57,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099bd575bc060ce161dad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:58,255 - lava_callback - INFO - Received callback for node 69409d6a575bc060ce162801 INFO: 172.21.0.1:40234 - "POST /node/69409d6a575bc060ce162801 HTTP/1.0" 202 Accepted 2025-12-17 23:44:58,944 - lava_callback - INFO - Received callback for node 69409d6b575bc060ce162804 INFO: 172.21.0.1:40244 - "POST /node/69409d6b575bc060ce162804 HTTP/1.0" 202 Accepted 2025-12-17 23:44:59,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69423ad6bb27571da24b7ab3 2025-12-17 23:44:59,618 - lava_callback - INFO - Received callback for node 69409d6d575bc060ce162807 INFO: 172.21.0.1:40246 - "POST /node/69409d6d575bc060ce162807 HTTP/1.0" 202 Accepted 2025-12-17 23:44:59,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099af575bc060ce161d8d 2025-12-17 23:44:59,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099af575bc060ce161d8d HTTP/1.1" 502 150 2025-12-17 23:44:59,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099af575bc060ce161d8d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:59,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099bd575bc060ce161dad 2025-12-17 23:44:59,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099bd575bc060ce161dad HTTP/1.1" 502 150 2025-12-17 23:44:59,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099bd575bc060ce161dad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:44:59,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099bc575bc060ce161daa 2025-12-17 23:44:59,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099bc575bc060ce161daa HTTP/1.1" 502 150 2025-12-17 23:44:59,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099bc575bc060ce161daa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:00,283 - lava_callback - INFO - Received callback for node 69409d6f575bc060ce16280c INFO: 172.21.0.1:55546 - "POST /node/69409d6f575bc060ce16280c HTTP/1.0" 202 Accepted 2025-12-17 23:45:00,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b4575bc060ce161d9d 2025-12-17 23:45:00,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b4575bc060ce161d9d HTTP/1.1" 502 150 2025-12-17 23:45:00,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b4575bc060ce161d9d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:00,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b6575bc060ce161da0 2025-12-17 23:45:00,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b6575bc060ce161da0 HTTP/1.1" 502 150 2025-12-17 23:45:00,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b6575bc060ce161da0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:00,933 - lava_callback - INFO - Received callback for node 69409d71575bc060ce162811 INFO: 172.21.0.1:55562 - "POST /node/69409d71575bc060ce162811 HTTP/1.0" 202 Accepted 2025-12-17 23:45:01,605 - lava_callback - INFO - Received callback for node 69409d73575bc060ce162814 INFO: 172.21.0.1:55578 - "POST /node/69409d73575bc060ce162814 HTTP/1.0" 202 Accepted 2025-12-17 23:45:01,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a9575bc060ce161d7a 2025-12-17 23:45:01,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a9575bc060ce161d7a HTTP/1.1" 502 150 2025-12-17 23:45:01,951 - lava_callback - ERROR - Error processing callback for node 694099a9575bc060ce161d7a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099a9575bc060ce161d7a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099a9575bc060ce161d7a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099a9575bc060ce161d7a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:01,953 - lava_callback - INFO - Starting async job submit for node 694099c0575bc060ce161db0 2025-12-17 23:45:01,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:01,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c0575bc060ce161db0 HTTP/1.1" 502 150 2025-12-17 23:45:01,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c0575bc060ce161db0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:01,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c0575bc060ce161db0 2025-12-17 23:45:01,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c0575bc060ce161db0 HTTP/1.1" 502 150 2025-12-17 23:45:01,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c0575bc060ce161db0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:02,279 - lava_callback - INFO - Received callback for node 69409d75575bc060ce162817 INFO: 172.21.0.1:55584 - "POST /node/69409d75575bc060ce162817 HTTP/1.0" 202 Accepted 2025-12-17 23:45:02,929 - lava_callback - INFO - Received callback for node 69409d77575bc060ce16281a INFO: 172.21.0.1:55594 - "POST /node/69409d77575bc060ce16281a HTTP/1.0" 202 Accepted 2025-12-17 23:45:02,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ab575bc060ce161d7d 2025-12-17 23:45:02,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ab575bc060ce161d7d HTTP/1.1" 502 150 2025-12-17 23:45:02,974 - lava_callback - ERROR - Error processing callback for node 694099ab575bc060ce161d7d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099ab575bc060ce161d7d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099ab575bc060ce161d7d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099ab575bc060ce161d7d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:02,976 - lava_callback - INFO - Starting async job submit for node 694099c4575bc060ce161dc1 2025-12-17 23:45:02,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:02,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c4575bc060ce161dc1 HTTP/1.1" 502 150 2025-12-17 23:45:02,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c4575bc060ce161dc1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:02,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c4575bc060ce161dc1 2025-12-17 23:45:02,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c4575bc060ce161dc1 HTTP/1.1" 502 150 2025-12-17 23:45:02,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c4575bc060ce161dc1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:03,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b8575bc060ce161da3 2025-12-17 23:45:03,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b8575bc060ce161da3 HTTP/1.1" 502 150 2025-12-17 23:45:03,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099b8575bc060ce161da3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:03,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c2575bc060ce161db6 2025-12-17 23:45:03,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c2575bc060ce161db6 HTTP/1.1" 502 150 2025-12-17 23:45:03,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c2575bc060ce161db6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55602 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:45:03,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7b575bc060ce1625de 2025-12-17 23:45:03,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7b575bc060ce1625de HTTP/1.1" 502 150 2025-12-17 23:45:03,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7b575bc060ce1625de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:03,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ba575bc060ce161da7 2025-12-17 23:45:03,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ba575bc060ce161da7 HTTP/1.1" 502 150 2025-12-17 23:45:03,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ba575bc060ce161da7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:03,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c40575bc060ce16254d 2025-12-17 23:45:03,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c40575bc060ce16254d HTTP/1.1" 502 150 2025-12-17 23:45:03,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c40575bc060ce16254d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:03,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099bd575bc060ce161dad 2025-12-17 23:45:03,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099bd575bc060ce161dad HTTP/1.1" 502 150 2025-12-17 23:45:03,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099bd575bc060ce161dad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:03,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c0575bc060ce161db0 2025-12-17 23:45:03,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c0575bc060ce161db0 HTTP/1.1" 502 150 2025-12-17 23:45:03,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c0575bc060ce161db0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:04,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409992575bc060ce161d42 2025-12-17 23:45:04,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409992575bc060ce161d42 HTTP/1.1" 502 150 2025-12-17 23:45:04,818 - lava_callback - ERROR - Error processing callback for node 69409992575bc060ce161d42: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409992575bc060ce161d42 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409992575bc060ce161d42 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409992575bc060ce161d42 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:04,819 - lava_callback - INFO - Starting async job submit for node 694099c7575bc060ce161dc6 2025-12-17 23:45:04,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:04,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c7575bc060ce161dc6 HTTP/1.1" 502 150 2025-12-17 23:45:04,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c7575bc060ce161dc6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:04,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c7575bc060ce161dc6 2025-12-17 23:45:04,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c7575bc060ce161dc6 HTTP/1.1" 502 150 2025-12-17 23:45:04,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c7575bc060ce161dc6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:04,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c4575bc060ce161dc1 2025-12-17 23:45:04,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c4575bc060ce161dc1 HTTP/1.1" 502 150 2025-12-17 23:45:04,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c4575bc060ce161dc1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55612 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:45:06,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c7575bc060ce161dc6 2025-12-17 23:45:06,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c7575bc060ce161dc6 HTTP/1.1" 502 150 2025-12-17 23:45:06,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c7575bc060ce161dc6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:07,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ad575bc060ce161d80 2025-12-17 23:45:07,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ad575bc060ce161d80 HTTP/1.1" 502 150 2025-12-17 23:45:07,570 - lava_callback - ERROR - Error processing callback for node 694099ad575bc060ce161d80: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099ad575bc060ce161d80 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099ad575bc060ce161d80 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099ad575bc060ce161d80 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:07,573 - lava_callback - INFO - Starting async job submit for node 694099c9575bc060ce161dc9 2025-12-17 23:45:07,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:07,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c9575bc060ce161dc9 HTTP/1.1" 502 150 2025-12-17 23:45:07,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c9575bc060ce161dc9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:07,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c9575bc060ce161dc9 2025-12-17 23:45:07,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c9575bc060ce161dc9 HTTP/1.1" 502 150 2025-12-17 23:45:07,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c9575bc060ce161dc9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:07,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099bc575bc060ce161daa 2025-12-17 23:45:07,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099bc575bc060ce161daa HTTP/1.1" 502 150 2025-12-17 23:45:07,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099bc575bc060ce161daa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:07,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c0575bc060ce161db0 2025-12-17 23:45:07,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c0575bc060ce161db0 HTTP/1.1" 502 150 2025-12-17 23:45:07,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c0575bc060ce161db0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:08,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c4575bc060ce161dc1 2025-12-17 23:45:08,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c4575bc060ce161dc1 HTTP/1.1" 502 150 2025-12-17 23:45:08,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c4575bc060ce161dc1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:09,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c9575bc060ce161dc9 2025-12-17 23:45:09,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c9575bc060ce161dc9 HTTP/1.1" 502 150 2025-12-17 23:45:09,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c9575bc060ce161dc9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:10,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c7575bc060ce161dc6 2025-12-17 23:45:11,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099bd575bc060ce161dad 2025-12-17 23:45:11,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099bd575bc060ce161dad HTTP/1.1" 502 150 2025-12-17 23:45:11,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099bd575bc060ce161dad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:12,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099a5575bc060ce161d74 2025-12-17 23:45:12,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099a5575bc060ce161d74 HTTP/1.1" 502 150 2025-12-17 23:45:12,724 - lava_callback - ERROR - Error processing callback for node 694099a5575bc060ce161d74: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099a5575bc060ce161d74 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099a5575bc060ce161d74 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099a5575bc060ce161d74 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:12,726 - lava_callback - INFO - Starting async job submit for node 694099d2575bc060ce161e2f 2025-12-17 23:45:12,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:12,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d2575bc060ce161e2f HTTP/1.1" 502 150 2025-12-17 23:45:12,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d2575bc060ce161e2f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:12,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d2575bc060ce161e2f 2025-12-17 23:45:12,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d2575bc060ce161e2f HTTP/1.1" 502 150 2025-12-17 23:45:12,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d2575bc060ce161e2f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:13,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c9575bc060ce161dc9 2025-12-17 23:45:13,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c9575bc060ce161dc9 HTTP/1.1" 502 150 2025-12-17 23:45:13,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c9575bc060ce161dc9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:14,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69423ad6bb27571da24b7ab3 HTTP/1.1" 502 150 2025-12-17 23:45:14,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69423ad6bb27571da24b7ab3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:14,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d2575bc060ce161e2f 2025-12-17 23:45:14,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d2575bc060ce161e2f HTTP/1.1" 502 150 2025-12-17 23:45:14,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d2575bc060ce161e2f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39452 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:45:15,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099af575bc060ce161d8d 2025-12-17 23:45:15,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099af575bc060ce161d8d HTTP/1.1" 502 150 2025-12-17 23:45:15,713 - lava_callback - ERROR - Error processing callback for node 694099af575bc060ce161d8d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099af575bc060ce161d8d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099af575bc060ce161d8d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099af575bc060ce161d8d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:15,715 - lava_callback - INFO - Starting async job submit for node 694099d5575bc060ce161e32 2025-12-17 23:45:15,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:15,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d5575bc060ce161e32 HTTP/1.1" 502 150 2025-12-17 23:45:15,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d5575bc060ce161e32'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:15,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d5575bc060ce161e32 2025-12-17 23:45:15,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d5575bc060ce161e32 HTTP/1.1" 502 150 2025-12-17 23:45:15,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d5575bc060ce161e32'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:15,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c0575bc060ce161db0 2025-12-17 23:45:15,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c0575bc060ce161db0 HTTP/1.1" 502 150 2025-12-17 23:45:15,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c0575bc060ce161db0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:16,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b4575bc060ce161d9d 2025-12-17 23:45:16,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b4575bc060ce161d9d HTTP/1.1" 502 150 2025-12-17 23:45:16,515 - lava_callback - ERROR - Error processing callback for node 694099b4575bc060ce161d9d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099b4575bc060ce161d9d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099b4575bc060ce161d9d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099b4575bc060ce161d9d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:16,517 - lava_callback - INFO - Starting async job submit for node 694099cf575bc060ce161e25 2025-12-17 23:45:16,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:16,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b6575bc060ce161da0 2025-12-17 23:45:16,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b6575bc060ce161da0 HTTP/1.1" 502 150 2025-12-17 23:45:16,526 - lava_callback - ERROR - Error processing callback for node 694099b6575bc060ce161da0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099b6575bc060ce161da0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099b6575bc060ce161da0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099b6575bc060ce161da0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:16,527 - lava_callback - INFO - Starting async job submit for node 694099d1575bc060ce161e2c 2025-12-17 23:45:16,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:39458 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:45:16,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d1575bc060ce161e2c HTTP/1.1" 502 150 2025-12-17 23:45:16,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d1575bc060ce161e2c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:16,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d1575bc060ce161e2c 2025-12-17 23:45:16,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d1575bc060ce161e2c HTTP/1.1" 502 150 2025-12-17 23:45:16,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d1575bc060ce161e2c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:16,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099cf575bc060ce161e25 HTTP/1.1" 502 150 2025-12-17 23:45:16,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099cf575bc060ce161e25'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:16,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099cf575bc060ce161e25 2025-12-17 23:45:16,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099cf575bc060ce161e25 HTTP/1.1" 502 150 2025-12-17 23:45:16,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099cf575bc060ce161e25'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:16,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c4575bc060ce161dc1 2025-12-17 23:45:16,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c4575bc060ce161dc1 HTTP/1.1" 502 150 2025-12-17 23:45:16,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c4575bc060ce161dc1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:17,553 - lava_callback - INFO - Received callback for node 69409d82575bc060ce16282a INFO: 172.21.0.1:39466 - "POST /node/69409d82575bc060ce16282a HTTP/1.0" 202 Accepted 2025-12-17 23:45:17,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d5575bc060ce161e32 2025-12-17 23:45:17,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d5575bc060ce161e32 HTTP/1.1" 502 150 2025-12-17 23:45:17,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d5575bc060ce161e32'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:18,247 - lava_callback - INFO - Received callback for node 69409d79575bc060ce16281d INFO: 172.21.0.1:39470 - "POST /node/69409d79575bc060ce16281d HTTP/1.0" 202 Accepted 2025-12-17 23:45:18,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d1575bc060ce161e2c 2025-12-17 23:45:18,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d1575bc060ce161e2c HTTP/1.1" 502 150 2025-12-17 23:45:18,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d1575bc060ce161e2c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:18,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099cf575bc060ce161e25 2025-12-17 23:45:18,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099cf575bc060ce161e25 HTTP/1.1" 502 150 2025-12-17 23:45:18,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099cf575bc060ce161e25'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:18,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d2575bc060ce161e2f 2025-12-17 23:45:18,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d2575bc060ce161e2f HTTP/1.1" 502 150 2025-12-17 23:45:18,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d2575bc060ce161e2f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:18,909 - lava_callback - INFO - Received callback for node 69409d7c575bc060ce162821 INFO: 172.21.0.1:39484 - "POST /node/69409d7c575bc060ce162821 HTTP/1.0" 202 Accepted 2025-12-17 23:45:19,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099b8575bc060ce161da3 2025-12-17 23:45:19,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099b8575bc060ce161da3 HTTP/1.1" 502 150 2025-12-17 23:45:19,447 - lava_callback - ERROR - Error processing callback for node 694099b8575bc060ce161da3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099b8575bc060ce161da3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099b8575bc060ce161da3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099b8575bc060ce161da3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:19,449 - lava_callback - INFO - Starting async job submit for node 694099d9575bc060ce161e48 2025-12-17 23:45:19,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:19,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d9575bc060ce161e48 HTTP/1.1" 502 150 2025-12-17 23:45:19,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d9575bc060ce161e48'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:19,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d9575bc060ce161e48 2025-12-17 23:45:19,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d9575bc060ce161e48 HTTP/1.1" 502 150 2025-12-17 23:45:19,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d9575bc060ce161e48'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:19,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c2575bc060ce161db6 2025-12-17 23:45:19,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c2575bc060ce161db6 HTTP/1.1" 502 150 2025-12-17 23:45:19,495 - lava_callback - ERROR - Error processing callback for node 694099c2575bc060ce161db6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099c2575bc060ce161db6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099c2575bc060ce161db6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099c2575bc060ce161db6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:19,498 - lava_callback - INFO - Starting async job submit for node 694099dd575bc060ce161e4e 2025-12-17 23:45:19,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:19,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099dd575bc060ce161e4e HTTP/1.1" 502 150 2025-12-17 23:45:19,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099dd575bc060ce161e4e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:19,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099dd575bc060ce161e4e 2025-12-17 23:45:19,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099dd575bc060ce161e4e HTTP/1.1" 502 150 2025-12-17 23:45:19,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099dd575bc060ce161e4e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:19,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7b575bc060ce1625de 2025-12-17 23:45:19,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7b575bc060ce1625de HTTP/1.1" 502 150 2025-12-17 23:45:19,576 - lava_callback - ERROR - Error processing callback for node 69409c7b575bc060ce1625de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c7b575bc060ce1625de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c7b575bc060ce1625de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c7b575bc060ce1625de (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:19,579 - lava_callback - INFO - Starting async job submit for node 694099db575bc060ce161e4b 2025-12-17 23:45:19,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:19,578 - lava_callback - INFO - Received callback for node 69409d7e575bc060ce162824 INFO: 172.21.0.1:39490 - "POST /node/69409d7e575bc060ce162824 HTTP/1.0" 202 Accepted 2025-12-17 23:45:19,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099db575bc060ce161e4b HTTP/1.1" 502 150 2025-12-17 23:45:19,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099db575bc060ce161e4b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:19,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099db575bc060ce161e4b 2025-12-17 23:45:19,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099db575bc060ce161e4b HTTP/1.1" 502 150 2025-12-17 23:45:19,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099db575bc060ce161e4b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:19,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ba575bc060ce161da7 2025-12-17 23:45:19,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ba575bc060ce161da7 HTTP/1.1" 502 150 2025-12-17 23:45:19,687 - lava_callback - ERROR - Error processing callback for node 694099ba575bc060ce161da7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099ba575bc060ce161da7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099ba575bc060ce161da7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099ba575bc060ce161da7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:19,690 - lava_callback - INFO - Starting async job submit for node 694099df575bc060ce161e51 2025-12-17 23:45:19,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:19,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099df575bc060ce161e51 HTTP/1.1" 502 150 2025-12-17 23:45:19,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099df575bc060ce161e51'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:19,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099df575bc060ce161e51 2025-12-17 23:45:19,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099df575bc060ce161e51 HTTP/1.1" 502 150 2025-12-17 23:45:19,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099df575bc060ce161e51'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:19,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c40575bc060ce16254d 2025-12-17 23:45:19,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c40575bc060ce16254d HTTP/1.1" 502 150 2025-12-17 23:45:19,894 - lava_callback - ERROR - Error processing callback for node 69409c40575bc060ce16254d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c40575bc060ce16254d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c40575bc060ce16254d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c40575bc060ce16254d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:19,896 - lava_callback - INFO - Starting async job submit for node 694099d8575bc060ce161e3f 2025-12-17 23:45:19,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:19,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d8575bc060ce161e3f HTTP/1.1" 502 150 2025-12-17 23:45:19,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d8575bc060ce161e3f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:19,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d8575bc060ce161e3f 2025-12-17 23:45:19,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d8575bc060ce161e3f HTTP/1.1" 502 150 2025-12-17 23:45:19,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d8575bc060ce161e3f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:20,242 - lava_callback - INFO - Received callback for node 69409d80575bc060ce162827 INFO: 172.21.0.1:60730 - "POST /node/69409d80575bc060ce162827 HTTP/1.0" 202 Accepted 2025-12-17 23:45:20,915 - lava_callback - INFO - Received callback for node 69409d84575bc060ce162835 INFO: 172.21.0.1:60732 - "POST /node/69409d84575bc060ce162835 HTTP/1.0" 202 Accepted 2025-12-17 23:45:21,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d9575bc060ce161e48 2025-12-17 23:45:21,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d9575bc060ce161e48 HTTP/1.1" 502 150 2025-12-17 23:45:21,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d9575bc060ce161e48'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:21,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099dd575bc060ce161e4e 2025-12-17 23:45:21,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099dd575bc060ce161e4e HTTP/1.1" 502 150 2025-12-17 23:45:21,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099dd575bc060ce161e4e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:21,591 - lava_callback - INFO - Received callback for node 69409d86575bc060ce16283c INFO: 172.21.0.1:60740 - "POST /node/69409d86575bc060ce16283c HTTP/1.0" 202 Accepted 2025-12-17 23:45:21,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c9575bc060ce161dc9 2025-12-17 23:45:21,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c9575bc060ce161dc9 HTTP/1.1" 502 150 2025-12-17 23:45:21,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c9575bc060ce161dc9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:21,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099db575bc060ce161e4b 2025-12-17 23:45:21,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099db575bc060ce161e4b HTTP/1.1" 502 150 2025-12-17 23:45:21,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099db575bc060ce161e4b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:21,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099df575bc060ce161e51 2025-12-17 23:45:21,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099df575bc060ce161e51 HTTP/1.1" 502 150 2025-12-17 23:45:21,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099df575bc060ce161e51'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:21,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d5575bc060ce161e32 2025-12-17 23:45:21,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d5575bc060ce161e32 HTTP/1.1" 502 150 2025-12-17 23:45:21,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d5575bc060ce161e32'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:21,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d8575bc060ce161e3f 2025-12-17 23:45:21,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d8575bc060ce161e3f HTTP/1.1" 502 150 2025-12-17 23:45:21,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d8575bc060ce161e3f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:22,271 - lava_callback - INFO - Received callback for node 69409d88575bc060ce16283f INFO: 172.21.0.1:60754 - "POST /node/69409d88575bc060ce16283f HTTP/1.0" 202 Accepted 2025-12-17 23:45:22,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d1575bc060ce161e2c 2025-12-17 23:45:22,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d1575bc060ce161e2c HTTP/1.1" 502 150 2025-12-17 23:45:22,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d1575bc060ce161e2c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:22,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099cf575bc060ce161e25 2025-12-17 23:45:22,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099cf575bc060ce161e25 HTTP/1.1" 502 150 2025-12-17 23:45:22,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099cf575bc060ce161e25'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:23,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099bc575bc060ce161daa 2025-12-17 23:45:23,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099bc575bc060ce161daa HTTP/1.1" 502 150 2025-12-17 23:45:23,962 - lava_callback - ERROR - Error processing callback for node 694099bc575bc060ce161daa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099bc575bc060ce161daa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099bc575bc060ce161daa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099bc575bc060ce161daa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:23,963 - lava_callback - INFO - Starting async job submit for node 694099e5575bc060ce161e5d 2025-12-17 23:45:23,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:23,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e5575bc060ce161e5d HTTP/1.1" 502 150 2025-12-17 23:45:23,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e5575bc060ce161e5d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:23,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e5575bc060ce161e5d 2025-12-17 23:45:23,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e5575bc060ce161e5d HTTP/1.1" 502 150 2025-12-17 23:45:23,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e5575bc060ce161e5d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:25,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d9575bc060ce161e48 2025-12-17 23:45:25,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d9575bc060ce161e48 HTTP/1.1" 502 150 2025-12-17 23:45:25,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d9575bc060ce161e48'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:25,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099dd575bc060ce161e4e 2025-12-17 23:45:25,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099dd575bc060ce161e4e HTTP/1.1" 502 150 2025-12-17 23:45:25,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099dd575bc060ce161e4e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:25,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099db575bc060ce161e4b 2025-12-17 23:45:25,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099db575bc060ce161e4b HTTP/1.1" 502 150 2025-12-17 23:45:25,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099db575bc060ce161e4b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:25,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099df575bc060ce161e51 2025-12-17 23:45:25,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099df575bc060ce161e51 HTTP/1.1" 502 150 2025-12-17 23:45:25,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099df575bc060ce161e51'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:25,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d8575bc060ce161e3f 2025-12-17 23:45:25,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d8575bc060ce161e3f HTTP/1.1" 502 150 2025-12-17 23:45:25,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d8575bc060ce161e3f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:25,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e5575bc060ce161e5d 2025-12-17 23:45:25,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e5575bc060ce161e5d HTTP/1.1" 502 150 2025-12-17 23:45:25,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e5575bc060ce161e5d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:26,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c7575bc060ce161dc6 HTTP/1.1" 502 150 2025-12-17 23:45:26,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c7575bc060ce161dc6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60758 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:45:26,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d2575bc060ce161e2f 2025-12-17 23:45:26,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d2575bc060ce161e2f HTTP/1.1" 502 150 2025-12-17 23:45:26,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d2575bc060ce161e2f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:27,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099bd575bc060ce161dad 2025-12-17 23:45:27,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099bd575bc060ce161dad HTTP/1.1" 502 150 2025-12-17 23:45:27,907 - lava_callback - ERROR - Error processing callback for node 694099bd575bc060ce161dad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099bd575bc060ce161dad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099bd575bc060ce161dad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099bd575bc060ce161dad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:27,910 - lava_callback - INFO - Starting async job submit for node 694099e2575bc060ce161e57 2025-12-17 23:45:27,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:27,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e2575bc060ce161e57 HTTP/1.1" 502 150 2025-12-17 23:45:27,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e2575bc060ce161e57'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:27,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e2575bc060ce161e57 2025-12-17 23:45:27,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e2575bc060ce161e57 HTTP/1.1" 502 150 2025-12-17 23:45:27,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e2575bc060ce161e57'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:28,277 - lava_callback - INFO - Received callback for node 69409d8a575bc060ce162842 INFO: 172.21.0.1:60764 - "POST /node/69409d8a575bc060ce162842 HTTP/1.0" 202 Accepted 2025-12-17 23:45:28,934 - lava_callback - INFO - Received callback for node 69409d8e575bc060ce162848 INFO: 172.21.0.1:60776 - "POST /node/69409d8e575bc060ce162848 HTTP/1.0" 202 Accepted 2025-12-17 23:45:29,599 - lava_callback - INFO - Received callback for node 69409d90575bc060ce16284b INFO: 172.21.0.1:60784 - "POST /node/69409d90575bc060ce16284b HTTP/1.0" 202 Accepted 2025-12-17 23:45:29,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d5575bc060ce161e32 2025-12-17 23:45:29,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d5575bc060ce161e32 HTTP/1.1" 502 150 2025-12-17 23:45:29,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d5575bc060ce161e32'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:29,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e2575bc060ce161e57 2025-12-17 23:45:29,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e2575bc060ce161e57 HTTP/1.1" 502 150 2025-12-17 23:45:29,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e2575bc060ce161e57'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:29,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e5575bc060ce161e5d 2025-12-17 23:45:29,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e5575bc060ce161e5d HTTP/1.1" 502 150 2025-12-17 23:45:29,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e5575bc060ce161e5d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:30,242 - lava_callback - INFO - Received callback for node 69409d92575bc060ce16284e INFO: 172.21.0.1:34886 - "POST /node/69409d92575bc060ce16284e HTTP/1.0" 202 Accepted 2025-12-17 23:45:30,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69423ad6bb27571da24b7ab3 2025-12-17 23:45:30,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69423ad6bb27571da24b7ab3 HTTP/1.1" 502 150 2025-12-17 23:45:30,424 - lava_callback - ERROR - Error processing callback for node 69423ad6bb27571da24b7ab3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69423ad6bb27571da24b7ab3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69423ad6bb27571da24b7ab3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69423ad6bb27571da24b7ab3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:30,426 - lava_callback - INFO - Starting async job submit for node 694099e7575bc060ce161e60 2025-12-17 23:45:30,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:30,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e7575bc060ce161e60 HTTP/1.1" 502 150 2025-12-17 23:45:30,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e7575bc060ce161e60'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:30,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e7575bc060ce161e60 2025-12-17 23:45:30,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e7575bc060ce161e60 HTTP/1.1" 502 150 2025-12-17 23:45:30,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e7575bc060ce161e60'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:30,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d1575bc060ce161e2c 2025-12-17 23:45:30,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d1575bc060ce161e2c HTTP/1.1" 502 150 2025-12-17 23:45:30,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d1575bc060ce161e2c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:30,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099cf575bc060ce161e25 2025-12-17 23:45:30,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099cf575bc060ce161e25 HTTP/1.1" 502 150 2025-12-17 23:45:30,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099cf575bc060ce161e25'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:30,909 - lava_callback - INFO - Received callback for node 69409d94575bc060ce162851 INFO: 172.21.0.1:34900 - "POST /node/69409d94575bc060ce162851 HTTP/1.0" 202 Accepted 2025-12-17 23:45:31,569 - lava_callback - INFO - Received callback for node 69409d97575bc060ce162856 INFO: 172.21.0.1:34904 - "POST /node/69409d97575bc060ce162856 HTTP/1.0" 202 Accepted 2025-12-17 23:45:31,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c0575bc060ce161db0 2025-12-17 23:45:31,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c0575bc060ce161db0 HTTP/1.1" 502 150 2025-12-17 23:45:31,966 - lava_callback - ERROR - Error processing callback for node 694099c0575bc060ce161db0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099c0575bc060ce161db0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099c0575bc060ce161db0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099c0575bc060ce161db0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:31,968 - lava_callback - INFO - Starting async job submit for node 694099e3575bc060ce161e5a 2025-12-17 23:45:31,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:31,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e3575bc060ce161e5a HTTP/1.1" 502 150 2025-12-17 23:45:31,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e3575bc060ce161e5a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:31,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e3575bc060ce161e5a 2025-12-17 23:45:31,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e3575bc060ce161e5a HTTP/1.1" 502 150 2025-12-17 23:45:31,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e3575bc060ce161e5a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:32,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e7575bc060ce161e60 2025-12-17 23:45:32,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e7575bc060ce161e60 HTTP/1.1" 502 150 2025-12-17 23:45:32,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e7575bc060ce161e60'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:32,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c4575bc060ce161dc1 2025-12-17 23:45:32,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c4575bc060ce161dc1 HTTP/1.1" 502 150 2025-12-17 23:45:32,990 - lava_callback - ERROR - Error processing callback for node 694099c4575bc060ce161dc1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099c4575bc060ce161dc1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099c4575bc060ce161dc1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099c4575bc060ce161dc1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:32,992 - lava_callback - INFO - Starting async job submit for node 694099e9575bc060ce161e67 2025-12-17 23:45:32,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:33,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d9575bc060ce161e48 2025-12-17 23:45:33,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d9575bc060ce161e48 HTTP/1.1" 502 150 2025-12-17 23:45:33,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d9575bc060ce161e48'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:33,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099dd575bc060ce161e4e 2025-12-17 23:45:33,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099dd575bc060ce161e4e HTTP/1.1" 502 150 2025-12-17 23:45:33,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099dd575bc060ce161e4e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:33,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099db575bc060ce161e4b 2025-12-17 23:45:33,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099db575bc060ce161e4b HTTP/1.1" 502 150 2025-12-17 23:45:33,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099db575bc060ce161e4b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:33,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099df575bc060ce161e51 2025-12-17 23:45:33,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099df575bc060ce161e51 HTTP/1.1" 502 150 2025-12-17 23:45:33,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099df575bc060ce161e51'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:33,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d8575bc060ce161e3f 2025-12-17 23:45:33,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d8575bc060ce161e3f HTTP/1.1" 502 150 2025-12-17 23:45:33,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099d8575bc060ce161e3f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:33,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e2575bc060ce161e57 2025-12-17 23:45:33,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e2575bc060ce161e57 HTTP/1.1" 502 150 2025-12-17 23:45:33,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e2575bc060ce161e57'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:33,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e3575bc060ce161e5a 2025-12-17 23:45:33,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e3575bc060ce161e5a HTTP/1.1" 502 150 2025-12-17 23:45:33,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e3575bc060ce161e5a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:34,192 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c7575bc060ce161dc6 2025-12-17 23:45:34,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c7575bc060ce161dc6 HTTP/1.1" 502 150 2025-12-17 23:45:34,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099c7575bc060ce161dc6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:36,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e7575bc060ce161e60 2025-12-17 23:45:36,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e7575bc060ce161e60 HTTP/1.1" 502 150 2025-12-17 23:45:36,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e7575bc060ce161e60'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34908 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:45:37,269 - lava_callback - INFO - Received callback for node 6940a4411e3c24e956819c49 INFO: 172.21.0.1:34922 - "POST /node/6940a4411e3c24e956819c49 HTTP/1.0" 202 Accepted 2025-12-17 23:45:37,593 - lava_callback - INFO - Received callback for node 69409d8d575bc060ce162845 INFO: 172.21.0.1:34934 - "POST /node/69409d8d575bc060ce162845 HTTP/1.0" 202 Accepted 2025-12-17 23:45:37,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c9575bc060ce161dc9 2025-12-17 23:45:37,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e5575bc060ce161e5d 2025-12-17 23:45:37,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e5575bc060ce161e5d HTTP/1.1" 502 150 2025-12-17 23:45:37,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e5575bc060ce161e5d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:37,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e3575bc060ce161e5a 2025-12-17 23:45:37,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e3575bc060ce161e5a HTTP/1.1" 502 150 2025-12-17 23:45:37,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e3575bc060ce161e5a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:41,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e2575bc060ce161e57 2025-12-17 23:45:41,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e2575bc060ce161e57 HTTP/1.1" 502 150 2025-12-17 23:45:41,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e2575bc060ce161e57'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:42,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d2575bc060ce161e2f 2025-12-17 23:45:42,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d2575bc060ce161e2f HTTP/1.1" 502 150 2025-12-17 23:45:42,743 - lava_callback - ERROR - Error processing callback for node 694099d2575bc060ce161e2f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099d2575bc060ce161e2f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099d2575bc060ce161e2f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099d2575bc060ce161e2f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:42,745 - lava_callback - INFO - Starting async job submit for node 694099e1575bc060ce161e54 2025-12-17 23:45:42,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:42,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e1575bc060ce161e54 HTTP/1.1" 502 150 2025-12-17 23:45:42,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e1575bc060ce161e54'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:42,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e1575bc060ce161e54 2025-12-17 23:45:42,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e1575bc060ce161e54 HTTP/1.1" 502 150 2025-12-17 23:45:42,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e1575bc060ce161e54'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:44,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e7575bc060ce161e60 2025-12-17 23:45:44,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e7575bc060ce161e60 HTTP/1.1" 502 150 2025-12-17 23:45:44,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e7575bc060ce161e60'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:44,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e1575bc060ce161e54 2025-12-17 23:45:44,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e1575bc060ce161e54 HTTP/1.1" 502 150 2025-12-17 23:45:44,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e1575bc060ce161e54'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:45,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d5575bc060ce161e32 2025-12-17 23:45:45,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d5575bc060ce161e32 HTTP/1.1" 502 150 2025-12-17 23:45:45,728 - lava_callback - ERROR - Error processing callback for node 694099d5575bc060ce161e32: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099d5575bc060ce161e32 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099d5575bc060ce161e32 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099d5575bc060ce161e32 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:45,729 - lava_callback - INFO - Starting async job submit for node 694099ef575bc060ce161e7a 2025-12-17 23:45:45,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:45,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ef575bc060ce161e7a HTTP/1.1" 502 150 2025-12-17 23:45:45,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ef575bc060ce161e7a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:45,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ef575bc060ce161e7a 2025-12-17 23:45:45,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ef575bc060ce161e7a HTTP/1.1" 502 150 2025-12-17 23:45:45,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ef575bc060ce161e7a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:45,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e3575bc060ce161e5a 2025-12-17 23:45:45,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e3575bc060ce161e5a HTTP/1.1" 502 150 2025-12-17 23:45:45,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e3575bc060ce161e5a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33110 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:45:46,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d1575bc060ce161e2c 2025-12-17 23:45:46,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d1575bc060ce161e2c HTTP/1.1" 502 150 2025-12-17 23:45:46,541 - lava_callback - ERROR - Error processing callback for node 694099d1575bc060ce161e2c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099d1575bc060ce161e2c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099d1575bc060ce161e2c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099d1575bc060ce161e2c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:46,543 - lava_callback - INFO - Starting async job submit for node 694099f1575bc060ce161e7d 2025-12-17 23:45:46,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:46,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099cf575bc060ce161e25 2025-12-17 23:45:46,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099cf575bc060ce161e25 HTTP/1.1" 502 150 2025-12-17 23:45:46,557 - lava_callback - ERROR - Error processing callback for node 694099cf575bc060ce161e25: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099cf575bc060ce161e25 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099cf575bc060ce161e25 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099cf575bc060ce161e25 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:46,559 - lava_callback - INFO - Starting async job submit for node 694099f3575bc060ce161e80 2025-12-17 23:45:46,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:46,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f1575bc060ce161e7d HTTP/1.1" 502 150 2025-12-17 23:45:46,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f1575bc060ce161e7d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:46,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f1575bc060ce161e7d 2025-12-17 23:45:46,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f1575bc060ce161e7d HTTP/1.1" 502 150 2025-12-17 23:45:46,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f1575bc060ce161e7d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:46,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f3575bc060ce161e80 HTTP/1.1" 502 150 2025-12-17 23:45:46,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f3575bc060ce161e80'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:46,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f3575bc060ce161e80 2025-12-17 23:45:46,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f3575bc060ce161e80 HTTP/1.1" 502 150 2025-12-17 23:45:46,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f3575bc060ce161e80'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:47,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ef575bc060ce161e7a 2025-12-17 23:45:48,309 - lava_callback - INFO - Received callback for node 69409d99575bc060ce16285f INFO: 172.21.0.1:33124 - "POST /node/69409d99575bc060ce16285f HTTP/1.0" 202 Accepted 2025-12-17 23:45:48,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e9575bc060ce161e67 HTTP/1.1" 502 150 2025-12-17 23:45:48,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e9575bc060ce161e67'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:48,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e9575bc060ce161e67 2025-12-17 23:45:48,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e9575bc060ce161e67 HTTP/1.1" 502 150 2025-12-17 23:45:48,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e9575bc060ce161e67'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:48,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f1575bc060ce161e7d 2025-12-17 23:45:48,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f1575bc060ce161e7d HTTP/1.1" 502 150 2025-12-17 23:45:48,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f1575bc060ce161e7d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:48,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f3575bc060ce161e80 2025-12-17 23:45:48,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f3575bc060ce161e80 HTTP/1.1" 502 150 2025-12-17 23:45:48,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f3575bc060ce161e80'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:48,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e1575bc060ce161e54 2025-12-17 23:45:48,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e1575bc060ce161e54 HTTP/1.1" 502 150 2025-12-17 23:45:48,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e1575bc060ce161e54'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:48,954 - lava_callback - INFO - Received callback for node 69409d9b575bc060ce162868 INFO: 172.21.0.1:33132 - "POST /node/69409d9b575bc060ce162868 HTTP/1.0" 202 Accepted 2025-12-17 23:45:49,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d9575bc060ce161e48 2025-12-17 23:45:49,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d9575bc060ce161e48 HTTP/1.1" 502 150 2025-12-17 23:45:49,470 - lava_callback - ERROR - Error processing callback for node 694099d9575bc060ce161e48: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099d9575bc060ce161e48 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099d9575bc060ce161e48 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099d9575bc060ce161e48 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:49,472 - lava_callback - INFO - Starting async job submit for node 694099f5575bc060ce161e83 2025-12-17 23:45:49,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:49,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f5575bc060ce161e83 HTTP/1.1" 502 150 2025-12-17 23:45:49,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f5575bc060ce161e83'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:49,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f5575bc060ce161e83 2025-12-17 23:45:49,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f5575bc060ce161e83 HTTP/1.1" 502 150 2025-12-17 23:45:49,480 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f5575bc060ce161e83'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:49,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099dd575bc060ce161e4e 2025-12-17 23:45:49,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099dd575bc060ce161e4e HTTP/1.1" 502 150 2025-12-17 23:45:49,531 - lava_callback - ERROR - Error processing callback for node 694099dd575bc060ce161e4e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099dd575bc060ce161e4e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099dd575bc060ce161e4e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099dd575bc060ce161e4e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:49,533 - lava_callback - INFO - Starting async job submit for node 694099f7575bc060ce161e89 2025-12-17 23:45:49,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:49,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f7575bc060ce161e89 HTTP/1.1" 502 150 2025-12-17 23:45:49,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f7575bc060ce161e89'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:49,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f7575bc060ce161e89 2025-12-17 23:45:49,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f7575bc060ce161e89 HTTP/1.1" 502 150 2025-12-17 23:45:49,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f7575bc060ce161e89'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:49,608 - lava_callback - INFO - Received callback for node 69409d9d575bc060ce16286b INFO: 172.21.0.1:33138 - "POST /node/69409d9d575bc060ce16286b HTTP/1.0" 202 Accepted 2025-12-17 23:45:49,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099db575bc060ce161e4b 2025-12-17 23:45:49,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099db575bc060ce161e4b HTTP/1.1" 502 150 2025-12-17 23:45:49,617 - lava_callback - ERROR - Error processing callback for node 694099db575bc060ce161e4b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099db575bc060ce161e4b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099db575bc060ce161e4b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099db575bc060ce161e4b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:49,619 - lava_callback - INFO - Starting async job submit for node 694099f9575bc060ce161e8c 2025-12-17 23:45:49,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:49,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f9575bc060ce161e8c HTTP/1.1" 502 150 2025-12-17 23:45:49,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f9575bc060ce161e8c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:49,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f9575bc060ce161e8c 2025-12-17 23:45:49,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f9575bc060ce161e8c HTTP/1.1" 502 150 2025-12-17 23:45:49,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f9575bc060ce161e8c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:49,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099df575bc060ce161e51 2025-12-17 23:45:49,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099df575bc060ce161e51 HTTP/1.1" 502 150 2025-12-17 23:45:49,705 - lava_callback - ERROR - Error processing callback for node 694099df575bc060ce161e51: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099df575bc060ce161e51 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099df575bc060ce161e51 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099df575bc060ce161e51 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:49,707 - lava_callback - INFO - Starting async job submit for node 694099fb575bc060ce161e8f 2025-12-17 23:45:49,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:49,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099fb575bc060ce161e8f HTTP/1.1" 502 150 2025-12-17 23:45:49,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099fb575bc060ce161e8f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:49,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099fb575bc060ce161e8f 2025-12-17 23:45:49,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099fb575bc060ce161e8f HTTP/1.1" 502 150 2025-12-17 23:45:49,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099fb575bc060ce161e8f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:49,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099d8575bc060ce161e3f 2025-12-17 23:45:49,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099d8575bc060ce161e3f HTTP/1.1" 502 150 2025-12-17 23:45:49,909 - lava_callback - ERROR - Error processing callback for node 694099d8575bc060ce161e3f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099d8575bc060ce161e3f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099d8575bc060ce161e3f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099d8575bc060ce161e3f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:49,911 - lava_callback - INFO - Starting async job submit for node 694099fd575bc060ce161e92 2025-12-17 23:45:49,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:49,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099fd575bc060ce161e92 HTTP/1.1" 502 150 2025-12-17 23:45:49,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099fd575bc060ce161e92'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:49,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099fd575bc060ce161e92 2025-12-17 23:45:49,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099fd575bc060ce161e92 HTTP/1.1" 502 150 2025-12-17 23:45:49,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099fd575bc060ce161e92'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:50,194 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099c7575bc060ce161dc6 2025-12-17 23:45:50,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c7575bc060ce161dc6 HTTP/1.1" 502 150 2025-12-17 23:45:50,195 - lava_callback - ERROR - Error processing callback for node 694099c7575bc060ce161dc6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099c7575bc060ce161dc6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099c7575bc060ce161dc6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099c7575bc060ce161dc6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:50,197 - lava_callback - INFO - Starting async job submit for node 694099ff575bc060ce161e9d 2025-12-17 23:45:50,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:50,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ff575bc060ce161e9d HTTP/1.1" 502 150 2025-12-17 23:45:50,204 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ff575bc060ce161e9d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:50,204 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ff575bc060ce161e9d 2025-12-17 23:45:50,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ff575bc060ce161e9d HTTP/1.1" 502 150 2025-12-17 23:45:50,204 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ff575bc060ce161e9d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:50,250 - lava_callback - INFO - Received callback for node 69409d9f575bc060ce16286e INFO: 172.21.0.1:54468 - "POST /node/69409d9f575bc060ce16286e HTTP/1.0" 202 Accepted 2025-12-17 23:45:50,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e9575bc060ce161e67 2025-12-17 23:45:50,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e9575bc060ce161e67 HTTP/1.1" 502 150 2025-12-17 23:45:50,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e9575bc060ce161e67'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:50,884 - lava_callback - INFO - Received callback for node 69409da1575bc060ce162871 INFO: 172.21.0.1:54478 - "POST /node/69409da1575bc060ce162871 HTTP/1.0" 202 Accepted 2025-12-17 23:45:51,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f5575bc060ce161e83 2025-12-17 23:45:51,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f7575bc060ce161e89 2025-12-17 23:45:51,540 - lava_callback - INFO - Received callback for node 69409da3575bc060ce162874 2025-12-17 23:45:51,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f7575bc060ce161e89 HTTP/1.1" 502 150 2025-12-17 23:45:51,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f7575bc060ce161e89'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54482 - "POST /node/69409da3575bc060ce162874 HTTP/1.0" 202 Accepted 2025-12-17 23:45:51,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f9575bc060ce161e8c 2025-12-17 23:45:51,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f9575bc060ce161e8c HTTP/1.1" 502 150 2025-12-17 23:45:51,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f9575bc060ce161e8c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:51,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099fb575bc060ce161e8f 2025-12-17 23:45:51,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099fb575bc060ce161e8f HTTP/1.1" 502 150 2025-12-17 23:45:51,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099fb575bc060ce161e8f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:51,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099fd575bc060ce161e92 2025-12-17 23:45:51,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099fd575bc060ce161e92 HTTP/1.1" 502 150 2025-12-17 23:45:51,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099fd575bc060ce161e92'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:52,205 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ff575bc060ce161e9d 2025-12-17 23:45:52,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ff575bc060ce161e9d HTTP/1.1" 502 150 2025-12-17 23:45:52,206 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ff575bc060ce161e9d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:52,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f1575bc060ce161e7d 2025-12-17 23:45:52,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f1575bc060ce161e7d HTTP/1.1" 502 150 2025-12-17 23:45:52,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f1575bc060ce161e7d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:52,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f3575bc060ce161e80 2025-12-17 23:45:52,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f3575bc060ce161e80 HTTP/1.1" 502 150 2025-12-17 23:45:52,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f3575bc060ce161e80'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:53,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099c9575bc060ce161dc9 HTTP/1.1" 502 150 2025-12-17 23:45:53,072 - lava_callback - ERROR - Error processing callback for node 694099c9575bc060ce161dc9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099c9575bc060ce161dc9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099c9575bc060ce161dc9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099c9575bc060ce161dc9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:53,074 - lava_callback - INFO - Starting async job submit for node 69409a01575bc060ce161eac 2025-12-17 23:45:53,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:53,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a01575bc060ce161eac HTTP/1.1" 502 150 2025-12-17 23:45:53,082 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a01575bc060ce161eac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:53,082 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a01575bc060ce161eac 2025-12-17 23:45:53,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a01575bc060ce161eac HTTP/1.1" 502 150 2025-12-17 23:45:53,083 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a01575bc060ce161eac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:53,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e5575bc060ce161e5d 2025-12-17 23:45:53,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e5575bc060ce161e5d HTTP/1.1" 502 150 2025-12-17 23:45:53,977 - lava_callback - ERROR - Error processing callback for node 694099e5575bc060ce161e5d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e5575bc060ce161e5d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e5575bc060ce161e5d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e5575bc060ce161e5d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:53,979 - lava_callback - INFO - Starting async job submit for node 69409a06575bc060ce161eb2 2025-12-17 23:45:53,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:53,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a06575bc060ce161eb2 HTTP/1.1" 502 150 2025-12-17 23:45:53,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a06575bc060ce161eb2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:53,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a06575bc060ce161eb2 2025-12-17 23:45:53,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a06575bc060ce161eb2 HTTP/1.1" 502 150 2025-12-17 23:45:53,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a06575bc060ce161eb2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:54,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e9575bc060ce161e67 2025-12-17 23:45:54,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e9575bc060ce161e67 HTTP/1.1" 502 150 2025-12-17 23:45:54,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e9575bc060ce161e67'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:55,083 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a01575bc060ce161eac 2025-12-17 23:45:55,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a01575bc060ce161eac HTTP/1.1" 502 150 2025-12-17 23:45:55,084 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a01575bc060ce161eac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:55,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f7575bc060ce161e89 2025-12-17 23:45:55,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f7575bc060ce161e89 HTTP/1.1" 502 150 2025-12-17 23:45:55,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f7575bc060ce161e89'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:55,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f9575bc060ce161e8c 2025-12-17 23:45:55,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f9575bc060ce161e8c HTTP/1.1" 502 150 2025-12-17 23:45:55,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f9575bc060ce161e8c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:55,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099fb575bc060ce161e8f 2025-12-17 23:45:55,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099fb575bc060ce161e8f HTTP/1.1" 502 150 2025-12-17 23:45:55,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099fb575bc060ce161e8f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:55,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099fd575bc060ce161e92 2025-12-17 23:45:55,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099fd575bc060ce161e92 HTTP/1.1" 502 150 2025-12-17 23:45:55,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099fd575bc060ce161e92'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:55,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a06575bc060ce161eb2 2025-12-17 23:45:55,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a06575bc060ce161eb2 HTTP/1.1" 502 150 2025-12-17 23:45:55,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a06575bc060ce161eb2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:56,206 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ff575bc060ce161e9d 2025-12-17 23:45:56,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ff575bc060ce161e9d HTTP/1.1" 502 150 2025-12-17 23:45:56,207 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ff575bc060ce161e9d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54498 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:45:56,546 - lava_callback - INFO - Received callback for node 6940a381575bc060ce1631ee INFO: 172.21.0.1:54514 - "POST /node/6940a381575bc060ce1631ee HTTP/1.0" 202 Accepted 2025-12-17 23:45:56,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e1575bc060ce161e54 2025-12-17 23:45:56,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e1575bc060ce161e54 HTTP/1.1" 502 150 2025-12-17 23:45:56,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e1575bc060ce161e54'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:57,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e2575bc060ce161e57 2025-12-17 23:45:57,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e2575bc060ce161e57 HTTP/1.1" 502 150 2025-12-17 23:45:57,924 - lava_callback - ERROR - Error processing callback for node 694099e2575bc060ce161e57: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e2575bc060ce161e57 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e2575bc060ce161e57 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e2575bc060ce161e57 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:45:57,925 - lava_callback - INFO - Starting async job submit for node 69409a08575bc060ce161eb9 2025-12-17 23:45:57,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:45:57,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a08575bc060ce161eb9 HTTP/1.1" 502 150 2025-12-17 23:45:57,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a08575bc060ce161eb9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:57,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a08575bc060ce161eb9 2025-12-17 23:45:57,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a08575bc060ce161eb9 HTTP/1.1" 502 150 2025-12-17 23:45:57,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a08575bc060ce161eb9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:59,084 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a01575bc060ce161eac 2025-12-17 23:45:59,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a08575bc060ce161eb9 2025-12-17 23:45:59,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a08575bc060ce161eb9 HTTP/1.1" 502 150 2025-12-17 23:45:59,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a08575bc060ce161eb9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:45:59,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a06575bc060ce161eb2 2025-12-17 23:45:59,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a06575bc060ce161eb2 HTTP/1.1" 502 150 2025-12-17 23:45:59,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a06575bc060ce161eb2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:00,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e7575bc060ce161e60 2025-12-17 23:46:00,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e7575bc060ce161e60 HTTP/1.1" 502 150 2025-12-17 23:46:00,447 - lava_callback - ERROR - Error processing callback for node 694099e7575bc060ce161e60: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e7575bc060ce161e60 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e7575bc060ce161e60 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e7575bc060ce161e60 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:00,449 - lava_callback - INFO - Starting async job submit for node 69409a0a575bc060ce161ebc 2025-12-17 23:46:00,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:00,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0a575bc060ce161ebc HTTP/1.1" 502 150 2025-12-17 23:46:00,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a0a575bc060ce161ebc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:00,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a0a575bc060ce161ebc 2025-12-17 23:46:00,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0a575bc060ce161ebc HTTP/1.1" 502 150 2025-12-17 23:46:00,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a0a575bc060ce161ebc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:00,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f1575bc060ce161e7d 2025-12-17 23:46:00,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f3575bc060ce161e80 2025-12-17 23:46:00,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f1575bc060ce161e7d HTTP/1.1" 502 150 2025-12-17 23:46:00,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f1575bc060ce161e7d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:00,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f3575bc060ce161e80 HTTP/1.1" 502 150 2025-12-17 23:46:00,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f3575bc060ce161e80'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:01,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e3575bc060ce161e5a 2025-12-17 23:46:01,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e3575bc060ce161e5a HTTP/1.1" 502 150 2025-12-17 23:46:01,980 - lava_callback - ERROR - Error processing callback for node 694099e3575bc060ce161e5a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e3575bc060ce161e5a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e3575bc060ce161e5a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e3575bc060ce161e5a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:01,982 - lava_callback - INFO - Starting async job submit for node 69409a0c575bc060ce161ebf 2025-12-17 23:46:01,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:01,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0c575bc060ce161ebf HTTP/1.1" 502 150 2025-12-17 23:46:01,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a0c575bc060ce161ebf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:01,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a0c575bc060ce161ebf 2025-12-17 23:46:01,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0c575bc060ce161ebf HTTP/1.1" 502 150 2025-12-17 23:46:01,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a0c575bc060ce161ebf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:02,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a0a575bc060ce161ebc 2025-12-17 23:46:02,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0a575bc060ce161ebc HTTP/1.1" 502 150 2025-12-17 23:46:02,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a0a575bc060ce161ebc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:02,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e9575bc060ce161e67 2025-12-17 23:46:02,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e9575bc060ce161e67 HTTP/1.1" 502 150 2025-12-17 23:46:02,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099e9575bc060ce161e67'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:03,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ef575bc060ce161e7a HTTP/1.1" 502 150 2025-12-17 23:46:03,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ef575bc060ce161e7a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:03,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f7575bc060ce161e89 2025-12-17 23:46:03,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f7575bc060ce161e89 HTTP/1.1" 502 150 2025-12-17 23:46:03,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f7575bc060ce161e89'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52310 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:46:03,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f9575bc060ce161e8c 2025-12-17 23:46:03,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f9575bc060ce161e8c HTTP/1.1" 502 150 2025-12-17 23:46:03,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f9575bc060ce161e8c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:03,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099fb575bc060ce161e8f 2025-12-17 23:46:03,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099fb575bc060ce161e8f HTTP/1.1" 502 150 2025-12-17 23:46:03,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099fb575bc060ce161e8f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:03,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099fd575bc060ce161e92 2025-12-17 23:46:03,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099fd575bc060ce161e92 HTTP/1.1" 502 150 2025-12-17 23:46:03,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099fd575bc060ce161e92'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:03,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a08575bc060ce161eb9 2025-12-17 23:46:03,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a08575bc060ce161eb9 HTTP/1.1" 502 150 2025-12-17 23:46:03,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a08575bc060ce161eb9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:03,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a0c575bc060ce161ebf 2025-12-17 23:46:03,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0c575bc060ce161ebf HTTP/1.1" 502 150 2025-12-17 23:46:03,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a0c575bc060ce161ebf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:04,207 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ff575bc060ce161e9d 2025-12-17 23:46:06,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a0a575bc060ce161ebc 2025-12-17 23:46:06,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0a575bc060ce161ebc HTTP/1.1" 502 150 2025-12-17 23:46:06,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a0a575bc060ce161ebc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52324 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:46:06,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f5575bc060ce161e83 HTTP/1.1" 502 150 2025-12-17 23:46:06,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f5575bc060ce161e83'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:07,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ef575bc060ce161e7a 2025-12-17 23:46:07,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ef575bc060ce161e7a HTTP/1.1" 502 150 2025-12-17 23:46:07,060 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ef575bc060ce161e7a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:07,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a06575bc060ce161eb2 2025-12-17 23:46:07,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a0c575bc060ce161ebf 2025-12-17 23:46:07,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a06575bc060ce161eb2 HTTP/1.1" 502 150 2025-12-17 23:46:07,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a06575bc060ce161eb2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:07,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0c575bc060ce161ebf HTTP/1.1" 502 150 2025-12-17 23:46:07,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a0c575bc060ce161ebf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:08,324 - lava_callback - INFO - Received callback for node 69409dab575bc060ce162879 INFO: 172.21.0.1:52332 - "POST /node/69409dab575bc060ce162879 HTTP/1.0" 202 Accepted 2025-12-17 23:46:08,982 - lava_callback - INFO - Received callback for node 69409dad575bc060ce162884 INFO: 172.21.0.1:52342 - "POST /node/69409dad575bc060ce162884 HTTP/1.0" 202 Accepted 2025-12-17 23:46:09,632 - lava_callback - INFO - Received callback for node 69409db0575bc060ce16288b INFO: 172.21.0.1:52346 - "POST /node/69409db0575bc060ce16288b HTTP/1.0" 202 Accepted 2025-12-17 23:46:10,285 - lava_callback - INFO - Received callback for node 69409db2575bc060ce162897 INFO: 172.21.0.1:44676 - "POST /node/69409db2575bc060ce162897 HTTP/1.0" 202 Accepted 2025-12-17 23:46:10,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f5575bc060ce161e83 2025-12-17 23:46:10,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f5575bc060ce161e83 HTTP/1.1" 502 150 2025-12-17 23:46:10,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f5575bc060ce161e83'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:10,930 - lava_callback - INFO - Received callback for node 69409db4575bc060ce16289a INFO: 172.21.0.1:44680 - "POST /node/69409db4575bc060ce16289a HTTP/1.0" 202 Accepted 2025-12-17 23:46:11,598 - lava_callback - INFO - Received callback for node 69409db6575bc060ce16289d INFO: 172.21.0.1:44686 - "POST /node/69409db6575bc060ce16289d HTTP/1.0" 202 Accepted 2025-12-17 23:46:11,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a08575bc060ce161eb9 2025-12-17 23:46:11,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a08575bc060ce161eb9 HTTP/1.1" 502 150 2025-12-17 23:46:11,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a08575bc060ce161eb9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:12,250 - lava_callback - INFO - Received callback for node 69409db8575bc060ce1628a5 INFO: 172.21.0.1:44694 - "POST /node/69409db8575bc060ce1628a5 HTTP/1.0" 202 Accepted 2025-12-17 23:46:12,362 - lava_callback - INFO - Received callback for node 69409dbc575bc060ce1628ab INFO: 172.21.0.1:44696 - "POST /node/69409dbc575bc060ce1628ab HTTP/1.0" 202 Accepted 2025-12-17 23:46:12,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e1575bc060ce161e54 2025-12-17 23:46:12,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e1575bc060ce161e54 HTTP/1.1" 502 150 2025-12-17 23:46:12,762 - lava_callback - ERROR - Error processing callback for node 694099e1575bc060ce161e54: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e1575bc060ce161e54 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e1575bc060ce161e54 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e1575bc060ce161e54 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:12,764 - lava_callback - INFO - Starting async job submit for node 69409a0e575bc060ce161ec2 2025-12-17 23:46:12,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:12,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0e575bc060ce161ec2 HTTP/1.1" 502 150 2025-12-17 23:46:12,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a0e575bc060ce161ec2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:12,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a0e575bc060ce161ec2 2025-12-17 23:46:12,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0e575bc060ce161ec2 HTTP/1.1" 502 150 2025-12-17 23:46:12,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a0e575bc060ce161ec2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:14,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a0a575bc060ce161ebc 2025-12-17 23:46:14,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0a575bc060ce161ebc HTTP/1.1" 502 150 2025-12-17 23:46:14,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a0a575bc060ce161ebc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:14,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a01575bc060ce161eac HTTP/1.1" 502 150 2025-12-17 23:46:14,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a01575bc060ce161eac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:14,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a0e575bc060ce161ec2 2025-12-17 23:46:14,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0e575bc060ce161ec2 HTTP/1.1" 502 150 2025-12-17 23:46:14,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a0e575bc060ce161ec2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:15,061 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ef575bc060ce161e7a 2025-12-17 23:46:15,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ef575bc060ce161e7a HTTP/1.1" 502 150 2025-12-17 23:46:15,062 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ef575bc060ce161e7a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44704 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:46:15,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a0c575bc060ce161ebf 2025-12-17 23:46:15,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0c575bc060ce161ebf HTTP/1.1" 502 150 2025-12-17 23:46:15,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a0c575bc060ce161ebf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44718 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:46:16,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f1575bc060ce161e7d 2025-12-17 23:46:16,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f1575bc060ce161e7d HTTP/1.1" 502 150 2025-12-17 23:46:16,571 - lava_callback - ERROR - Error processing callback for node 694099f1575bc060ce161e7d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099f1575bc060ce161e7d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099f1575bc060ce161e7d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099f1575bc060ce161e7d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:16,573 - lava_callback - INFO - Starting async job submit for node 69409a10575bc060ce161ec5 2025-12-17 23:46:16,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:16,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f3575bc060ce161e80 2025-12-17 23:46:16,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f3575bc060ce161e80 HTTP/1.1" 502 150 2025-12-17 23:46:16,575 - lava_callback - ERROR - Error processing callback for node 694099f3575bc060ce161e80: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099f3575bc060ce161e80 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099f3575bc060ce161e80 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099f3575bc060ce161e80 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:16,577 - lava_callback - INFO - Starting async job submit for node 69409a12575bc060ce161ec8 2025-12-17 23:46:16,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:16,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a10575bc060ce161ec5 HTTP/1.1" 502 150 2025-12-17 23:46:16,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a10575bc060ce161ec5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:16,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a10575bc060ce161ec5 2025-12-17 23:46:16,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a10575bc060ce161ec5 HTTP/1.1" 502 150 2025-12-17 23:46:16,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a10575bc060ce161ec5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:16,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a12575bc060ce161ec8 HTTP/1.1" 502 150 2025-12-17 23:46:16,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a12575bc060ce161ec8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:16,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a12575bc060ce161ec8 2025-12-17 23:46:16,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a12575bc060ce161ec8 HTTP/1.1" 502 150 2025-12-17 23:46:16,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a12575bc060ce161ec8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:18,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099e9575bc060ce161e67 2025-12-17 23:46:18,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099e9575bc060ce161e67 HTTP/1.1" 502 150 2025-12-17 23:46:18,472 - lava_callback - ERROR - Error processing callback for node 694099e9575bc060ce161e67: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e9575bc060ce161e67 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e9575bc060ce161e67 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099e9575bc060ce161e67 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:18,474 - lava_callback - INFO - Starting async job submit for node 69409a15575bc060ce161eda 2025-12-17 23:46:18,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:18,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a15575bc060ce161eda HTTP/1.1" 502 150 2025-12-17 23:46:18,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a15575bc060ce161eda'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:18,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a15575bc060ce161eda 2025-12-17 23:46:18,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a15575bc060ce161eda HTTP/1.1" 502 150 2025-12-17 23:46:18,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a15575bc060ce161eda'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:18,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a10575bc060ce161ec5 2025-12-17 23:46:18,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a10575bc060ce161ec5 HTTP/1.1" 502 150 2025-12-17 23:46:18,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a10575bc060ce161ec5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:18,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a12575bc060ce161ec8 2025-12-17 23:46:18,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a12575bc060ce161ec8 HTTP/1.1" 502 150 2025-12-17 23:46:18,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a12575bc060ce161ec8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:18,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a0e575bc060ce161ec2 2025-12-17 23:46:18,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0e575bc060ce161ec2 HTTP/1.1" 502 150 2025-12-17 23:46:18,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a0e575bc060ce161ec2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:18,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f5575bc060ce161e83 2025-12-17 23:46:18,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f5575bc060ce161e83 HTTP/1.1" 502 150 2025-12-17 23:46:18,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099f5575bc060ce161e83'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:19,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f7575bc060ce161e89 2025-12-17 23:46:19,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f7575bc060ce161e89 HTTP/1.1" 502 150 2025-12-17 23:46:19,550 - lava_callback - ERROR - Error processing callback for node 694099f7575bc060ce161e89: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099f7575bc060ce161e89 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099f7575bc060ce161e89 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099f7575bc060ce161e89 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:19,552 - lava_callback - INFO - Starting async job submit for node 69409d1d575bc060ce16274e 2025-12-17 23:46:19,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:19,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1d575bc060ce16274e HTTP/1.1" 502 150 2025-12-17 23:46:19,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d1d575bc060ce16274e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:19,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d1d575bc060ce16274e 2025-12-17 23:46:19,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1d575bc060ce16274e HTTP/1.1" 502 150 2025-12-17 23:46:19,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d1d575bc060ce16274e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:19,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f9575bc060ce161e8c 2025-12-17 23:46:19,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f9575bc060ce161e8c HTTP/1.1" 502 150 2025-12-17 23:46:19,633 - lava_callback - ERROR - Error processing callback for node 694099f9575bc060ce161e8c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099f9575bc060ce161e8c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099f9575bc060ce161e8c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099f9575bc060ce161e8c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:19,635 - lava_callback - INFO - Starting async job submit for node 69409ea7575bc060ce162a5b 2025-12-17 23:46:19,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:19,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ff575bc060ce161e9d HTTP/1.1" 502 150 2025-12-17 23:46:19,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694099ff575bc060ce161e9d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:19,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea7575bc060ce162a5b HTTP/1.1" 502 150 2025-12-17 23:46:19,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea7575bc060ce162a5b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:19,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea7575bc060ce162a5b 2025-12-17 23:46:19,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea7575bc060ce162a5b HTTP/1.1" 502 150 2025-12-17 23:46:19,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea7575bc060ce162a5b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:19,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099fb575bc060ce161e8f 2025-12-17 23:46:19,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099fb575bc060ce161e8f HTTP/1.1" 502 150 2025-12-17 23:46:19,722 - lava_callback - ERROR - Error processing callback for node 694099fb575bc060ce161e8f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099fb575bc060ce161e8f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099fb575bc060ce161e8f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099fb575bc060ce161e8f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:19,724 - lava_callback - INFO - Starting async job submit for node 69409a18575bc060ce161edf 2025-12-17 23:46:19,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:19,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a18575bc060ce161edf HTTP/1.1" 502 150 2025-12-17 23:46:19,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a18575bc060ce161edf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:19,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a18575bc060ce161edf 2025-12-17 23:46:19,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a18575bc060ce161edf HTTP/1.1" 502 150 2025-12-17 23:46:19,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a18575bc060ce161edf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:19,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099fd575bc060ce161e92 2025-12-17 23:46:19,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099fd575bc060ce161e92 HTTP/1.1" 502 150 2025-12-17 23:46:19,945 - lava_callback - ERROR - Error processing callback for node 694099fd575bc060ce161e92: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099fd575bc060ce161e92 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099fd575bc060ce161e92 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099fd575bc060ce161e92 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:19,947 - lava_callback - INFO - Starting async job submit for node 69424dacbb27571da24b8f1b 2025-12-17 23:46:19,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:19,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69424dacbb27571da24b8f1b HTTP/1.1" 502 150 2025-12-17 23:46:19,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69424dacbb27571da24b8f1b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:19,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69424dacbb27571da24b8f1b 2025-12-17 23:46:19,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69424dacbb27571da24b8f1b HTTP/1.1" 502 150 2025-12-17 23:46:19,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69424dacbb27571da24b8f1b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:20,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a15575bc060ce161eda 2025-12-17 23:46:20,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a15575bc060ce161eda HTTP/1.1" 502 150 2025-12-17 23:46:20,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a15575bc060ce161eda'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:21,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d1d575bc060ce16274e 2025-12-17 23:46:21,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1d575bc060ce16274e HTTP/1.1" 502 150 2025-12-17 23:46:21,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d1d575bc060ce16274e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:21,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea7575bc060ce162a5b 2025-12-17 23:46:21,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea7575bc060ce162a5b HTTP/1.1" 502 150 2025-12-17 23:46:21,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea7575bc060ce162a5b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:21,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a18575bc060ce161edf 2025-12-17 23:46:21,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a18575bc060ce161edf HTTP/1.1" 502 150 2025-12-17 23:46:21,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a18575bc060ce161edf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:21,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69424dacbb27571da24b8f1b 2025-12-17 23:46:21,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69424dacbb27571da24b8f1b HTTP/1.1" 502 150 2025-12-17 23:46:21,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69424dacbb27571da24b8f1b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:22,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a01575bc060ce161eac 2025-12-17 23:46:22,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a01575bc060ce161eac HTTP/1.1" 502 150 2025-12-17 23:46:22,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a01575bc060ce161eac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:22,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a10575bc060ce161ec5 2025-12-17 23:46:22,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a10575bc060ce161ec5 HTTP/1.1" 502 150 2025-12-17 23:46:22,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a10575bc060ce161ec5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:22,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a12575bc060ce161ec8 2025-12-17 23:46:22,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a12575bc060ce161ec8 HTTP/1.1" 502 150 2025-12-17 23:46:22,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a12575bc060ce161ec8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:23,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a06575bc060ce161eb2 2025-12-17 23:46:23,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a06575bc060ce161eb2 HTTP/1.1" 502 150 2025-12-17 23:46:23,993 - lava_callback - ERROR - Error processing callback for node 69409a06575bc060ce161eb2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a06575bc060ce161eb2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a06575bc060ce161eb2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a06575bc060ce161eb2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:23,995 - lava_callback - INFO - Starting async job submit for node 69409a1c575bc060ce161ee5 2025-12-17 23:46:23,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:24,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1c575bc060ce161ee5 HTTP/1.1" 502 150 2025-12-17 23:46:24,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a1c575bc060ce161ee5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:24,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a1c575bc060ce161ee5 2025-12-17 23:46:24,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1c575bc060ce161ee5 HTTP/1.1" 502 150 2025-12-17 23:46:24,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a1c575bc060ce161ee5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:24,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a15575bc060ce161eda 2025-12-17 23:46:24,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a15575bc060ce161eda HTTP/1.1" 502 150 2025-12-17 23:46:24,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a15575bc060ce161eda'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:25,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d1d575bc060ce16274e 2025-12-17 23:46:25,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1d575bc060ce16274e HTTP/1.1" 502 150 2025-12-17 23:46:25,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d1d575bc060ce16274e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:25,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea7575bc060ce162a5b 2025-12-17 23:46:25,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea7575bc060ce162a5b HTTP/1.1" 502 150 2025-12-17 23:46:25,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea7575bc060ce162a5b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:25,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a18575bc060ce161edf 2025-12-17 23:46:25,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a18575bc060ce161edf HTTP/1.1" 502 150 2025-12-17 23:46:25,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a18575bc060ce161edf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:25,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69424dacbb27571da24b8f1b 2025-12-17 23:46:25,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69424dacbb27571da24b8f1b HTTP/1.1" 502 150 2025-12-17 23:46:25,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69424dacbb27571da24b8f1b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:26,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a1c575bc060ce161ee5 2025-12-17 23:46:26,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1c575bc060ce161ee5 HTTP/1.1" 502 150 2025-12-17 23:46:26,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a1c575bc060ce161ee5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59428 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:46:26,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a0e575bc060ce161ec2 2025-12-17 23:46:26,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0e575bc060ce161ec2 HTTP/1.1" 502 150 2025-12-17 23:46:26,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a0e575bc060ce161ec2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:27,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a08575bc060ce161eb9 2025-12-17 23:46:27,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a08575bc060ce161eb9 HTTP/1.1" 502 150 2025-12-17 23:46:27,960 - lava_callback - ERROR - Error processing callback for node 69409a08575bc060ce161eb9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a08575bc060ce161eb9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a08575bc060ce161eb9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a08575bc060ce161eb9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:27,961 - lava_callback - INFO - Starting async job submit for node 69409a1e575bc060ce161ee8 2025-12-17 23:46:27,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:27,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1e575bc060ce161ee8 HTTP/1.1" 502 150 2025-12-17 23:46:27,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a1e575bc060ce161ee8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:27,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a1e575bc060ce161ee8 2025-12-17 23:46:27,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1e575bc060ce161ee8 HTTP/1.1" 502 150 2025-12-17 23:46:27,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a1e575bc060ce161ee8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:28,316 - lava_callback - INFO - Received callback for node 69409dbf575bc060ce1628b4 INFO: 172.21.0.1:59432 - "POST /node/69409dbf575bc060ce1628b4 HTTP/1.0" 202 Accepted 2025-12-17 23:46:28,968 - lava_callback - INFO - Received callback for node 69409dc1575bc060ce1628bd INFO: 172.21.0.1:59446 - "POST /node/69409dc1575bc060ce1628bd HTTP/1.0" 202 Accepted 2025-12-17 23:46:29,632 - lava_callback - INFO - Received callback for node 69409dc3575bc060ce1628c2 INFO: 172.21.0.1:59450 - "POST /node/69409dc3575bc060ce1628c2 HTTP/1.0" 202 Accepted 2025-12-17 23:46:29,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a1e575bc060ce161ee8 2025-12-17 23:46:29,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1e575bc060ce161ee8 HTTP/1.1" 502 150 2025-12-17 23:46:29,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a1e575bc060ce161ee8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:30,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a1c575bc060ce161ee5 2025-12-17 23:46:30,274 - lava_callback - INFO - Received callback for node 69409dc5575bc060ce1628c5 INFO: 172.21.0.1:47730 - "POST /node/69409dc5575bc060ce1628c5 HTTP/1.0" 202 Accepted 2025-12-17 23:46:30,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a0a575bc060ce161ebc 2025-12-17 23:46:30,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0a575bc060ce161ebc HTTP/1.1" 502 150 2025-12-17 23:46:30,465 - lava_callback - ERROR - Error processing callback for node 69409a0a575bc060ce161ebc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a0a575bc060ce161ebc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a0a575bc060ce161ebc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a0a575bc060ce161ebc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:30,467 - lava_callback - INFO - Starting async job submit for node 69409a14575bc060ce161ed3 2025-12-17 23:46:30,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:30,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a14575bc060ce161ed3 HTTP/1.1" 502 150 2025-12-17 23:46:30,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a14575bc060ce161ed3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:30,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a14575bc060ce161ed3 2025-12-17 23:46:30,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a14575bc060ce161ed3 HTTP/1.1" 502 150 2025-12-17 23:46:30,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a14575bc060ce161ed3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:30,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a10575bc060ce161ec5 2025-12-17 23:46:30,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a12575bc060ce161ec8 2025-12-17 23:46:30,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a10575bc060ce161ec5 HTTP/1.1" 502 150 2025-12-17 23:46:30,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a10575bc060ce161ec5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:30,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a12575bc060ce161ec8 HTTP/1.1" 502 150 2025-12-17 23:46:30,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a12575bc060ce161ec8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:31,062 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ef575bc060ce161e7a 2025-12-17 23:46:31,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ef575bc060ce161e7a HTTP/1.1" 502 150 2025-12-17 23:46:31,064 - lava_callback - ERROR - Error processing callback for node 694099ef575bc060ce161e7a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099ef575bc060ce161e7a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099ef575bc060ce161e7a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099ef575bc060ce161e7a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:31,066 - lava_callback - INFO - Starting async job submit for node 69409a24575bc060ce161ef1 2025-12-17 23:46:31,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:31,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a24575bc060ce161ef1 HTTP/1.1" 502 150 2025-12-17 23:46:31,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a24575bc060ce161ef1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:31,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a24575bc060ce161ef1 2025-12-17 23:46:31,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a24575bc060ce161ef1 HTTP/1.1" 502 150 2025-12-17 23:46:31,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a24575bc060ce161ef1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:31,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a0c575bc060ce161ebf 2025-12-17 23:46:31,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0c575bc060ce161ebf HTTP/1.1" 502 150 2025-12-17 23:46:31,995 - lava_callback - ERROR - Error processing callback for node 69409a0c575bc060ce161ebf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a0c575bc060ce161ebf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a0c575bc060ce161ebf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a0c575bc060ce161ebf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:31,997 - lava_callback - INFO - Starting async job submit for node 69409a21575bc060ce161eeb 2025-12-17 23:46:31,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:32,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a21575bc060ce161eeb HTTP/1.1" 502 150 2025-12-17 23:46:32,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a21575bc060ce161eeb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:32,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a21575bc060ce161eeb 2025-12-17 23:46:32,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a21575bc060ce161eeb HTTP/1.1" 502 150 2025-12-17 23:46:32,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a21575bc060ce161eeb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:32,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a14575bc060ce161ed3 2025-12-17 23:46:32,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a14575bc060ce161ed3 HTTP/1.1" 502 150 2025-12-17 23:46:32,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a14575bc060ce161ed3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:32,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a15575bc060ce161eda 2025-12-17 23:46:32,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a15575bc060ce161eda HTTP/1.1" 502 150 2025-12-17 23:46:32,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a15575bc060ce161eda'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:33,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a24575bc060ce161ef1 2025-12-17 23:46:33,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a24575bc060ce161ef1 HTTP/1.1" 502 150 2025-12-17 23:46:33,074 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a24575bc060ce161ef1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:33,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d1d575bc060ce16274e 2025-12-17 23:46:33,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1d575bc060ce16274e HTTP/1.1" 502 150 2025-12-17 23:46:33,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d1d575bc060ce16274e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:33,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea7575bc060ce162a5b 2025-12-17 23:46:33,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea7575bc060ce162a5b HTTP/1.1" 502 150 2025-12-17 23:46:33,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea7575bc060ce162a5b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:33,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a18575bc060ce161edf 2025-12-17 23:46:33,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a18575bc060ce161edf HTTP/1.1" 502 150 2025-12-17 23:46:33,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a18575bc060ce161edf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:33,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a1e575bc060ce161ee8 2025-12-17 23:46:33,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1e575bc060ce161ee8 HTTP/1.1" 502 150 2025-12-17 23:46:33,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a1e575bc060ce161ee8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:33,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69424dacbb27571da24b8f1b 2025-12-17 23:46:33,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69424dacbb27571da24b8f1b HTTP/1.1" 502 150 2025-12-17 23:46:33,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69424dacbb27571da24b8f1b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:34,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a21575bc060ce161eeb 2025-12-17 23:46:34,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a21575bc060ce161eeb HTTP/1.1" 502 150 2025-12-17 23:46:34,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a21575bc060ce161eeb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:34,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099f5575bc060ce161e83 2025-12-17 23:46:34,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099f5575bc060ce161e83 HTTP/1.1" 502 150 2025-12-17 23:46:34,900 - lava_callback - ERROR - Error processing callback for node 694099f5575bc060ce161e83: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099f5575bc060ce161e83 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099f5575bc060ce161e83 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099f5575bc060ce161e83 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:34,902 - lava_callback - INFO - Starting async job submit for node 69409a29575bc060ce161f05 2025-12-17 23:46:34,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:34,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a29575bc060ce161f05 HTTP/1.1" 502 150 2025-12-17 23:46:34,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a29575bc060ce161f05'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:34,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a29575bc060ce161f05 2025-12-17 23:46:34,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a29575bc060ce161f05 HTTP/1.1" 502 150 2025-12-17 23:46:34,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a29575bc060ce161f05'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:35,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694099ff575bc060ce161e9d 2025-12-17 23:46:35,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694099ff575bc060ce161e9d HTTP/1.1" 502 150 2025-12-17 23:46:35,702 - lava_callback - ERROR - Error processing callback for node 694099ff575bc060ce161e9d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099ff575bc060ce161e9d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099ff575bc060ce161e9d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694099ff575bc060ce161e9d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:35,704 - lava_callback - INFO - Starting async job submit for node 69409a2c575bc060ce161f08 2025-12-17 23:46:35,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:35,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a2c575bc060ce161f08 HTTP/1.1" 502 150 2025-12-17 23:46:35,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a2c575bc060ce161f08'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:35,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a2c575bc060ce161f08 2025-12-17 23:46:35,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a2c575bc060ce161f08 HTTP/1.1" 502 150 2025-12-17 23:46:35,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a2c575bc060ce161f08'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:36,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a14575bc060ce161ed3 2025-12-17 23:46:36,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a14575bc060ce161ed3 HTTP/1.1" 502 150 2025-12-17 23:46:36,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a14575bc060ce161ed3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47732 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:46:36,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a29575bc060ce161f05 2025-12-17 23:46:36,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a29575bc060ce161f05 HTTP/1.1" 502 150 2025-12-17 23:46:36,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a29575bc060ce161f05'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:37,075 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a24575bc060ce161ef1 2025-12-17 23:46:37,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a24575bc060ce161ef1 HTTP/1.1" 502 150 2025-12-17 23:46:37,076 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a24575bc060ce161ef1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:37,636 - lava_callback - INFO - Received callback for node 69409dd6575bc060ce1628d8 INFO: 172.21.0.1:47734 - "POST /node/69409dd6575bc060ce1628d8 HTTP/1.0" 202 Accepted 2025-12-17 23:46:37,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a2c575bc060ce161f08 2025-12-17 23:46:37,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a2c575bc060ce161f08 HTTP/1.1" 502 150 2025-12-17 23:46:37,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a2c575bc060ce161f08'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:38,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a21575bc060ce161eeb 2025-12-17 23:46:38,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a21575bc060ce161eeb HTTP/1.1" 502 150 2025-12-17 23:46:38,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a21575bc060ce161eeb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:38,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a01575bc060ce161eac 2025-12-17 23:46:38,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a01575bc060ce161eac HTTP/1.1" 502 150 2025-12-17 23:46:38,579 - lava_callback - ERROR - Error processing callback for node 69409a01575bc060ce161eac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a01575bc060ce161eac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a01575bc060ce161eac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a01575bc060ce161eac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:38,581 - lava_callback - INFO - Starting async job submit for node 69409a2e575bc060ce161f0b 2025-12-17 23:46:38,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:38,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a2e575bc060ce161f0b HTTP/1.1" 502 150 2025-12-17 23:46:38,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a2e575bc060ce161f0b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:38,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a2e575bc060ce161f0b 2025-12-17 23:46:38,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a2e575bc060ce161f0b HTTP/1.1" 502 150 2025-12-17 23:46:38,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a2e575bc060ce161f0b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:40,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a2e575bc060ce161f0b 2025-12-17 23:46:40,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a29575bc060ce161f05 2025-12-17 23:46:40,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a29575bc060ce161f05 HTTP/1.1" 502 150 2025-12-17 23:46:40,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a29575bc060ce161f05'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:41,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a2c575bc060ce161f08 2025-12-17 23:46:41,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a2c575bc060ce161f08 HTTP/1.1" 502 150 2025-12-17 23:46:41,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a2c575bc060ce161f08'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:41,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a1e575bc060ce161ee8 2025-12-17 23:46:41,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1e575bc060ce161ee8 HTTP/1.1" 502 150 2025-12-17 23:46:41,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a1e575bc060ce161ee8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:42,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a0e575bc060ce161ec2 2025-12-17 23:46:42,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a0e575bc060ce161ec2 HTTP/1.1" 502 150 2025-12-17 23:46:42,786 - lava_callback - ERROR - Error processing callback for node 69409a0e575bc060ce161ec2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a0e575bc060ce161ec2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a0e575bc060ce161ec2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a0e575bc060ce161ec2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:42,789 - lava_callback - INFO - Starting async job submit for node 6942681e1b61f1c6005f5088 2025-12-17 23:46:42,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:42,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942681e1b61f1c6005f5088 HTTP/1.1" 502 150 2025-12-17 23:46:42,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942681e1b61f1c6005f5088'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:42,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942681e1b61f1c6005f5088 2025-12-17 23:46:42,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942681e1b61f1c6005f5088 HTTP/1.1" 502 150 2025-12-17 23:46:42,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942681e1b61f1c6005f5088'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:44,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a14575bc060ce161ed3 2025-12-17 23:46:44,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a14575bc060ce161ed3 HTTP/1.1" 502 150 2025-12-17 23:46:44,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a14575bc060ce161ed3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:44,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942681e1b61f1c6005f5088 2025-12-17 23:46:44,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942681e1b61f1c6005f5088 HTTP/1.1" 502 150 2025-12-17 23:46:44,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942681e1b61f1c6005f5088'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:45,076 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a24575bc060ce161ef1 2025-12-17 23:46:45,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a24575bc060ce161ef1 HTTP/1.1" 502 150 2025-12-17 23:46:45,077 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a24575bc060ce161ef1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:45,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1c575bc060ce161ee5 HTTP/1.1" 502 150 2025-12-17 23:46:45,296 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a1c575bc060ce161ee5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:46,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a21575bc060ce161eeb 2025-12-17 23:46:46,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a21575bc060ce161eeb HTTP/1.1" 502 150 2025-12-17 23:46:46,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a21575bc060ce161eeb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60844 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:46:46,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a10575bc060ce161ec5 2025-12-17 23:46:46,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a12575bc060ce161ec8 2025-12-17 23:46:46,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a10575bc060ce161ec5 HTTP/1.1" 502 150 2025-12-17 23:46:46,593 - lava_callback - ERROR - Error processing callback for node 69409a10575bc060ce161ec5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a10575bc060ce161ec5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a10575bc060ce161ec5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a10575bc060ce161ec5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:46,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a12575bc060ce161ec8 HTTP/1.1" 502 150 2025-12-17 23:46:46,595 - lava_callback - INFO - Starting async job submit for node 69425180bb27571da24b95b2 2025-12-17 23:46:46,595 - lava_callback - ERROR - Error processing callback for node 69409a12575bc060ce161ec8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a12575bc060ce161ec8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a12575bc060ce161ec8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a12575bc060ce161ec8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:46,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:46,600 - lava_callback - INFO - Starting async job submit for node 69409c3c575bc060ce162544 2025-12-17 23:46:46,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69425180bb27571da24b95b2 HTTP/1.1" 502 150 2025-12-17 23:46:46,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69425180bb27571da24b95b2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:46,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69425180bb27571da24b95b2 2025-12-17 23:46:46,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69425180bb27571da24b95b2 HTTP/1.1" 502 150 2025-12-17 23:46:46,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69425180bb27571da24b95b2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:46,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:46,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3c575bc060ce162544 HTTP/1.1" 502 150 2025-12-17 23:46:46,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3c575bc060ce162544'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:46,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3c575bc060ce162544 2025-12-17 23:46:46,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3c575bc060ce162544 HTTP/1.1" 502 150 2025-12-17 23:46:46,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3c575bc060ce162544'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:48,348 - lava_callback - INFO - Received callback for node 69409dd3575bc060ce1628cd INFO: 172.21.0.1:60846 - "POST /node/69409dd3575bc060ce1628cd HTTP/1.0" 202 Accepted 2025-12-17 23:46:48,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a15575bc060ce161eda 2025-12-17 23:46:48,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a15575bc060ce161eda HTTP/1.1" 502 150 2025-12-17 23:46:48,517 - lava_callback - ERROR - Error processing callback for node 69409a15575bc060ce161eda: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a15575bc060ce161eda (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a15575bc060ce161eda (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a15575bc060ce161eda (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:48,519 - lava_callback - INFO - Starting async job submit for node 69409a27575bc060ce161efa 2025-12-17 23:46:48,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:48,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a27575bc060ce161efa HTTP/1.1" 502 150 2025-12-17 23:46:48,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a27575bc060ce161efa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:48,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a27575bc060ce161efa 2025-12-17 23:46:48,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a27575bc060ce161efa HTTP/1.1" 502 150 2025-12-17 23:46:48,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a27575bc060ce161efa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:48,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69425180bb27571da24b95b2 2025-12-17 23:46:48,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69425180bb27571da24b95b2 HTTP/1.1" 502 150 2025-12-17 23:46:48,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69425180bb27571da24b95b2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:48,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3c575bc060ce162544 2025-12-17 23:46:48,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3c575bc060ce162544 HTTP/1.1" 502 150 2025-12-17 23:46:48,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3c575bc060ce162544'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:48,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942681e1b61f1c6005f5088 2025-12-17 23:46:48,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942681e1b61f1c6005f5088 HTTP/1.1" 502 150 2025-12-17 23:46:48,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942681e1b61f1c6005f5088'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:48,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a29575bc060ce161f05 2025-12-17 23:46:48,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a29575bc060ce161f05 HTTP/1.1" 502 150 2025-12-17 23:46:48,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a29575bc060ce161f05'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:49,007 - lava_callback - INFO - Received callback for node 69409dd9575bc060ce1628db INFO: 172.21.0.1:60862 - "POST /node/69409dd9575bc060ce1628db HTTP/1.0" 202 Accepted 2025-12-17 23:46:49,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d1d575bc060ce16274e 2025-12-17 23:46:49,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1d575bc060ce16274e HTTP/1.1" 502 150 2025-12-17 23:46:49,584 - lava_callback - ERROR - Error processing callback for node 69409d1d575bc060ce16274e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d1d575bc060ce16274e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d1d575bc060ce16274e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d1d575bc060ce16274e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:49,586 - lava_callback - INFO - Starting async job submit for node 69409a31575bc060ce161f11 2025-12-17 23:46:49,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:49,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a31575bc060ce161f11 HTTP/1.1" 502 150 2025-12-17 23:46:49,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a31575bc060ce161f11'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:49,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a31575bc060ce161f11 2025-12-17 23:46:49,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a31575bc060ce161f11 HTTP/1.1" 502 150 2025-12-17 23:46:49,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a31575bc060ce161f11'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:49,651 - lava_callback - INFO - Received callback for node 69409ddb575bc060ce1628de INFO: 172.21.0.1:60868 - "POST /node/69409ddb575bc060ce1628de HTTP/1.0" 202 Accepted 2025-12-17 23:46:49,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea7575bc060ce162a5b 2025-12-17 23:46:49,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea7575bc060ce162a5b HTTP/1.1" 502 150 2025-12-17 23:46:49,714 - lava_callback - ERROR - Error processing callback for node 69409ea7575bc060ce162a5b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ea7575bc060ce162a5b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ea7575bc060ce162a5b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ea7575bc060ce162a5b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:49,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a2c575bc060ce161f08 2025-12-17 23:46:49,717 - lava_callback - INFO - Starting async job submit for node 69409a30575bc060ce161f0e 2025-12-17 23:46:49,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:49,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a2c575bc060ce161f08 HTTP/1.1" 502 150 2025-12-17 23:46:49,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a2c575bc060ce161f08'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:49,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a30575bc060ce161f0e HTTP/1.1" 502 150 2025-12-17 23:46:49,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a30575bc060ce161f0e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:49,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a30575bc060ce161f0e 2025-12-17 23:46:49,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a30575bc060ce161f0e HTTP/1.1" 502 150 2025-12-17 23:46:49,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a30575bc060ce161f0e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:49,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a18575bc060ce161edf 2025-12-17 23:46:49,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a18575bc060ce161edf HTTP/1.1" 502 150 2025-12-17 23:46:49,736 - lava_callback - ERROR - Error processing callback for node 69409a18575bc060ce161edf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a18575bc060ce161edf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a18575bc060ce161edf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a18575bc060ce161edf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:49,737 - lava_callback - INFO - Starting async job submit for node 69409a33575bc060ce161f16 2025-12-17 23:46:49,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:49,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a33575bc060ce161f16 HTTP/1.1" 502 150 2025-12-17 23:46:49,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a33575bc060ce161f16'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:49,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a33575bc060ce161f16 2025-12-17 23:46:49,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a33575bc060ce161f16 HTTP/1.1" 502 150 2025-12-17 23:46:49,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a33575bc060ce161f16'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:49,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69424dacbb27571da24b8f1b 2025-12-17 23:46:49,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69424dacbb27571da24b8f1b HTTP/1.1" 502 150 2025-12-17 23:46:49,980 - lava_callback - ERROR - Error processing callback for node 69424dacbb27571da24b8f1b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69424dacbb27571da24b8f1b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69424dacbb27571da24b8f1b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69424dacbb27571da24b8f1b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:49,982 - lava_callback - INFO - Starting async job submit for node 69409a35575bc060ce161f19 2025-12-17 23:46:49,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:50,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a35575bc060ce161f19 HTTP/1.1" 502 150 2025-12-17 23:46:50,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a35575bc060ce161f19'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:50,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a35575bc060ce161f19 2025-12-17 23:46:50,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a35575bc060ce161f19 HTTP/1.1" 502 150 2025-12-17 23:46:50,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a35575bc060ce161f19'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:50,310 - lava_callback - INFO - Received callback for node 69409ddd575bc060ce1628e1 INFO: 172.21.0.1:38016 - "POST /node/69409ddd575bc060ce1628e1 HTTP/1.0" 202 Accepted 2025-12-17 23:46:50,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a27575bc060ce161efa 2025-12-17 23:46:50,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a27575bc060ce161efa HTTP/1.1" 502 150 2025-12-17 23:46:50,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a27575bc060ce161efa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:51,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a31575bc060ce161f11 2025-12-17 23:46:51,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a30575bc060ce161f0e 2025-12-17 23:46:51,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a30575bc060ce161f0e HTTP/1.1" 502 150 2025-12-17 23:46:51,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a30575bc060ce161f0e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:51,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a33575bc060ce161f16 2025-12-17 23:46:51,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a33575bc060ce161f16 HTTP/1.1" 502 150 2025-12-17 23:46:51,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a33575bc060ce161f16'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:52,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a35575bc060ce161f19 2025-12-17 23:46:52,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a35575bc060ce161f19 HTTP/1.1" 502 150 2025-12-17 23:46:52,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a35575bc060ce161f19'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:52,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69425180bb27571da24b95b2 2025-12-17 23:46:52,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69425180bb27571da24b95b2 HTTP/1.1" 502 150 2025-12-17 23:46:52,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69425180bb27571da24b95b2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:52,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3c575bc060ce162544 2025-12-17 23:46:52,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3c575bc060ce162544 HTTP/1.1" 502 150 2025-12-17 23:46:52,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3c575bc060ce162544'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:53,296 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a1c575bc060ce161ee5 2025-12-17 23:46:53,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1c575bc060ce161ee5 HTTP/1.1" 502 150 2025-12-17 23:46:53,298 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a1c575bc060ce161ee5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:54,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a27575bc060ce161efa 2025-12-17 23:46:54,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a27575bc060ce161efa HTTP/1.1" 502 150 2025-12-17 23:46:54,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a27575bc060ce161efa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:55,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a30575bc060ce161f0e 2025-12-17 23:46:55,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a30575bc060ce161f0e HTTP/1.1" 502 150 2025-12-17 23:46:55,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a30575bc060ce161f0e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:55,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a33575bc060ce161f16 2025-12-17 23:46:55,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a33575bc060ce161f16 HTTP/1.1" 502 150 2025-12-17 23:46:55,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a33575bc060ce161f16'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:56,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a35575bc060ce161f19 2025-12-17 23:46:56,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a2e575bc060ce161f0b HTTP/1.1" 502 150 2025-12-17 23:46:56,052 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a2e575bc060ce161f0b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38020 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:46:56,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942681e1b61f1c6005f5088 2025-12-17 23:46:56,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942681e1b61f1c6005f5088 HTTP/1.1" 502 150 2025-12-17 23:46:56,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942681e1b61f1c6005f5088'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:57,622 - lava_callback - INFO - Received callback for node 69409dea575bc060ce1628f5 INFO: 172.21.0.1:38026 - "POST /node/69409dea575bc060ce1628f5 HTTP/1.0" 202 Accepted 2025-12-17 23:46:57,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a1e575bc060ce161ee8 2025-12-17 23:46:57,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1e575bc060ce161ee8 HTTP/1.1" 502 150 2025-12-17 23:46:57,977 - lava_callback - ERROR - Error processing callback for node 69409a1e575bc060ce161ee8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a1e575bc060ce161ee8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a1e575bc060ce161ee8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a1e575bc060ce161ee8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:46:57,979 - lava_callback - INFO - Starting async job submit for node 69409a37575bc060ce161f1c 2025-12-17 23:46:57,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:46:58,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a37575bc060ce161f1c HTTP/1.1" 502 150 2025-12-17 23:46:58,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a37575bc060ce161f1c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:58,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a37575bc060ce161f1c 2025-12-17 23:46:58,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a37575bc060ce161f1c HTTP/1.1" 502 150 2025-12-17 23:46:58,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a37575bc060ce161f1c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:46:58,265 - lava_callback - INFO - Received callback for node 69409dee575bc060ce1628fb INFO: 172.21.0.1:38028 - "POST /node/69409dee575bc060ce1628fb HTTP/1.0" 202 Accepted 2025-12-17 23:47:00,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a37575bc060ce161f1c 2025-12-17 23:47:00,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a37575bc060ce161f1c HTTP/1.1" 502 150 2025-12-17 23:47:00,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a37575bc060ce161f1c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:00,052 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a2e575bc060ce161f0b 2025-12-17 23:47:00,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a2e575bc060ce161f0b HTTP/1.1" 502 150 2025-12-17 23:47:00,054 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a2e575bc060ce161f0b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:00,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a14575bc060ce161ed3 2025-12-17 23:47:00,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a14575bc060ce161ed3 HTTP/1.1" 502 150 2025-12-17 23:47:00,484 - lava_callback - ERROR - Error processing callback for node 69409a14575bc060ce161ed3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a14575bc060ce161ed3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a14575bc060ce161ed3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a14575bc060ce161ed3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:00,487 - lava_callback - INFO - Starting async job submit for node 69409a39575bc060ce161f1f 2025-12-17 23:47:00,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:00,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a39575bc060ce161f1f HTTP/1.1" 502 150 2025-12-17 23:47:00,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a39575bc060ce161f1f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:00,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a39575bc060ce161f1f 2025-12-17 23:47:00,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a39575bc060ce161f1f HTTP/1.1" 502 150 2025-12-17 23:47:00,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a39575bc060ce161f1f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:00,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69425180bb27571da24b95b2 2025-12-17 23:47:00,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69425180bb27571da24b95b2 HTTP/1.1" 502 150 2025-12-17 23:47:00,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69425180bb27571da24b95b2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:00,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3c575bc060ce162544 2025-12-17 23:47:00,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3c575bc060ce162544 HTTP/1.1" 502 150 2025-12-17 23:47:00,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3c575bc060ce162544'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:01,077 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a24575bc060ce161ef1 2025-12-17 23:47:01,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a24575bc060ce161ef1 HTTP/1.1" 502 150 2025-12-17 23:47:01,079 - lava_callback - ERROR - Error processing callback for node 69409a24575bc060ce161ef1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a24575bc060ce161ef1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a24575bc060ce161ef1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a24575bc060ce161ef1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:01,082 - lava_callback - INFO - Starting async job submit for node 69409a3e575bc060ce161f31 2025-12-17 23:47:01,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:01,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a3e575bc060ce161f31 HTTP/1.1" 502 150 2025-12-17 23:47:01,090 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a3e575bc060ce161f31'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:01,090 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a3e575bc060ce161f31 2025-12-17 23:47:01,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a3e575bc060ce161f31 HTTP/1.1" 502 150 2025-12-17 23:47:01,091 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a3e575bc060ce161f31'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:02,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a21575bc060ce161eeb 2025-12-17 23:47:02,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a21575bc060ce161eeb HTTP/1.1" 502 150 2025-12-17 23:47:02,011 - lava_callback - ERROR - Error processing callback for node 69409a21575bc060ce161eeb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a21575bc060ce161eeb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a21575bc060ce161eeb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a21575bc060ce161eeb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:02,014 - lava_callback - INFO - Starting async job submit for node 69409a42575bc060ce161f3b 2025-12-17 23:47:02,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:02,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a42575bc060ce161f3b HTTP/1.1" 502 150 2025-12-17 23:47:02,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a42575bc060ce161f3b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:02,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a42575bc060ce161f3b 2025-12-17 23:47:02,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a42575bc060ce161f3b HTTP/1.1" 502 150 2025-12-17 23:47:02,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a42575bc060ce161f3b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:02,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a39575bc060ce161f1f 2025-12-17 23:47:02,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a39575bc060ce161f1f HTTP/1.1" 502 150 2025-12-17 23:47:02,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a39575bc060ce161f1f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:02,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a27575bc060ce161efa 2025-12-17 23:47:02,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a27575bc060ce161efa HTTP/1.1" 502 150 2025-12-17 23:47:02,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a27575bc060ce161efa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:03,091 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a3e575bc060ce161f31 2025-12-17 23:47:03,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a3e575bc060ce161f31 HTTP/1.1" 502 150 2025-12-17 23:47:03,093 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a3e575bc060ce161f31'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:32812 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:47:03,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a30575bc060ce161f0e 2025-12-17 23:47:03,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a30575bc060ce161f0e HTTP/1.1" 502 150 2025-12-17 23:47:03,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a30575bc060ce161f0e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:03,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a33575bc060ce161f16 2025-12-17 23:47:03,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a33575bc060ce161f16 HTTP/1.1" 502 150 2025-12-17 23:47:03,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a33575bc060ce161f16'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:04,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a37575bc060ce161f1c 2025-12-17 23:47:04,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a37575bc060ce161f1c HTTP/1.1" 502 150 2025-12-17 23:47:04,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a37575bc060ce161f1c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:04,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a42575bc060ce161f3b 2025-12-17 23:47:04,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a42575bc060ce161f3b HTTP/1.1" 502 150 2025-12-17 23:47:04,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a42575bc060ce161f3b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:04,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a29575bc060ce161f05 2025-12-17 23:47:04,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a29575bc060ce161f05 HTTP/1.1" 502 150 2025-12-17 23:47:04,915 - lava_callback - ERROR - Error processing callback for node 69409a29575bc060ce161f05: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a29575bc060ce161f05 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a29575bc060ce161f05 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a29575bc060ce161f05 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:04,918 - lava_callback - INFO - Starting async job submit for node 69426c3e1b61f1c6005f5f75 2025-12-17 23:47:04,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:04,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69426c3e1b61f1c6005f5f75 HTTP/1.1" 502 150 2025-12-17 23:47:04,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69426c3e1b61f1c6005f5f75'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:04,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69426c3e1b61f1c6005f5f75 2025-12-17 23:47:04,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69426c3e1b61f1c6005f5f75 HTTP/1.1" 502 150 2025-12-17 23:47:04,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69426c3e1b61f1c6005f5f75'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:05,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a2c575bc060ce161f08 2025-12-17 23:47:05,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a2c575bc060ce161f08 HTTP/1.1" 502 150 2025-12-17 23:47:05,720 - lava_callback - ERROR - Error processing callback for node 69409a2c575bc060ce161f08: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a2c575bc060ce161f08 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a2c575bc060ce161f08 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a2c575bc060ce161f08 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:05,722 - lava_callback - INFO - Starting async job submit for node 6940a073575bc060ce162d34 2025-12-17 23:47:05,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:05,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a073575bc060ce162d34 HTTP/1.1" 502 150 2025-12-17 23:47:05,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a073575bc060ce162d34'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:05,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a073575bc060ce162d34 2025-12-17 23:47:05,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a073575bc060ce162d34 HTTP/1.1" 502 150 2025-12-17 23:47:05,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a073575bc060ce162d34'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:32818 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:47:06,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a39575bc060ce161f1f 2025-12-17 23:47:06,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a39575bc060ce161f1f HTTP/1.1" 502 150 2025-12-17 23:47:06,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a39575bc060ce161f1f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:06,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69426c3e1b61f1c6005f5f75 2025-12-17 23:47:06,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69426c3e1b61f1c6005f5f75 HTTP/1.1" 502 150 2025-12-17 23:47:06,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69426c3e1b61f1c6005f5f75'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:07,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a31575bc060ce161f11 HTTP/1.1" 502 150 2025-12-17 23:47:07,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a31575bc060ce161f11'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:07,093 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a3e575bc060ce161f31 2025-12-17 23:47:07,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a3e575bc060ce161f31 HTTP/1.1" 502 150 2025-12-17 23:47:07,094 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a3e575bc060ce161f31'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:07,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a073575bc060ce162d34 2025-12-17 23:47:07,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a073575bc060ce162d34 HTTP/1.1" 502 150 2025-12-17 23:47:07,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a073575bc060ce162d34'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:08,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a42575bc060ce161f3b 2025-12-17 23:47:08,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a42575bc060ce161f3b HTTP/1.1" 502 150 2025-12-17 23:47:08,037 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a42575bc060ce161f3b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:08,054 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a2e575bc060ce161f0b 2025-12-17 23:47:08,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a2e575bc060ce161f0b HTTP/1.1" 502 150 2025-12-17 23:47:08,055 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a2e575bc060ce161f0b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:08,345 - lava_callback - INFO - Received callback for node 69409de7575bc060ce1628e6 INFO: 172.21.0.1:32828 - "POST /node/69409de7575bc060ce1628e6 HTTP/1.0" 202 Accepted 2025-12-17 23:47:09,007 - lava_callback - INFO - Received callback for node 69409df0575bc060ce1628fe INFO: 172.21.0.1:32830 - "POST /node/69409df0575bc060ce1628fe HTTP/1.0" 202 Accepted 2025-12-17 23:47:09,298 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a1c575bc060ce161ee5 2025-12-17 23:47:09,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a1c575bc060ce161ee5 HTTP/1.1" 502 150 2025-12-17 23:47:09,299 - lava_callback - ERROR - Error processing callback for node 69409a1c575bc060ce161ee5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a1c575bc060ce161ee5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a1c575bc060ce161ee5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a1c575bc060ce161ee5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:09,301 - lava_callback - INFO - Starting async job submit for node 69409a3b575bc060ce161f24 2025-12-17 23:47:09,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:09,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a3b575bc060ce161f24 HTTP/1.1" 502 150 2025-12-17 23:47:09,311 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a3b575bc060ce161f24'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:09,311 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a3b575bc060ce161f24 2025-12-17 23:47:09,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a3b575bc060ce161f24 HTTP/1.1" 502 150 2025-12-17 23:47:09,312 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a3b575bc060ce161f24'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:09,658 - lava_callback - INFO - Received callback for node 69409df3575bc060ce162901 INFO: 172.21.0.1:32840 - "POST /node/69409df3575bc060ce162901 HTTP/1.0" 202 Accepted 2025-12-17 23:47:10,294 - lava_callback - INFO - Received callback for node 69409df5575bc060ce162904 INFO: 172.21.0.1:37676 - "POST /node/69409df5575bc060ce162904 HTTP/1.0" 202 Accepted 2025-12-17 23:47:10,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69426c3e1b61f1c6005f5f75 2025-12-17 23:47:10,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69426c3e1b61f1c6005f5f75 HTTP/1.1" 502 150 2025-12-17 23:47:10,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69426c3e1b61f1c6005f5f75'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:11,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a31575bc060ce161f11 2025-12-17 23:47:11,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a31575bc060ce161f11 HTTP/1.1" 502 150 2025-12-17 23:47:11,057 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a31575bc060ce161f11'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:11,312 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a3b575bc060ce161f24 2025-12-17 23:47:11,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a3b575bc060ce161f24 HTTP/1.1" 502 150 2025-12-17 23:47:11,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a3b575bc060ce161f24'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:11,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a35575bc060ce161f19 HTTP/1.1" 502 150 2025-12-17 23:47:11,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a35575bc060ce161f19'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:11,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a073575bc060ce162d34 2025-12-17 23:47:11,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a073575bc060ce162d34 HTTP/1.1" 502 150 2025-12-17 23:47:11,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a073575bc060ce162d34'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:12,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a37575bc060ce161f1c 2025-12-17 23:47:12,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a37575bc060ce161f1c HTTP/1.1" 502 150 2025-12-17 23:47:12,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a37575bc060ce161f1c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:12,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942681e1b61f1c6005f5088 2025-12-17 23:47:12,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942681e1b61f1c6005f5088 HTTP/1.1" 502 150 2025-12-17 23:47:12,809 - lava_callback - ERROR - Error processing callback for node 6942681e1b61f1c6005f5088: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942681e1b61f1c6005f5088 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942681e1b61f1c6005f5088 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942681e1b61f1c6005f5088 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:12,811 - lava_callback - INFO - Starting async job submit for node 69409a40575bc060ce161f38 2025-12-17 23:47:12,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:12,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a40575bc060ce161f38 HTTP/1.1" 502 150 2025-12-17 23:47:12,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a40575bc060ce161f38'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:12,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a40575bc060ce161f38 2025-12-17 23:47:12,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a40575bc060ce161f38 HTTP/1.1" 502 150 2025-12-17 23:47:12,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a40575bc060ce161f38'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:14,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a39575bc060ce161f1f 2025-12-17 23:47:14,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a39575bc060ce161f1f HTTP/1.1" 502 150 2025-12-17 23:47:14,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a39575bc060ce161f1f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:14,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a40575bc060ce161f38 2025-12-17 23:47:14,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a40575bc060ce161f38 HTTP/1.1" 502 150 2025-12-17 23:47:14,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a40575bc060ce161f38'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:15,094 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a3e575bc060ce161f31 2025-12-17 23:47:15,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a3e575bc060ce161f31 HTTP/1.1" 502 150 2025-12-17 23:47:15,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a3e575bc060ce161f31'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37680 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:47:15,313 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a3b575bc060ce161f24 2025-12-17 23:47:15,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a3b575bc060ce161f24 HTTP/1.1" 502 150 2025-12-17 23:47:15,320 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a3b575bc060ce161f24'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:16,037 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a42575bc060ce161f3b 2025-12-17 23:47:16,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a42575bc060ce161f3b HTTP/1.1" 502 150 2025-12-17 23:47:16,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a42575bc060ce161f3b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37684 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:47:16,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69425180bb27571da24b95b2 2025-12-17 23:47:16,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69425180bb27571da24b95b2 HTTP/1.1" 502 150 2025-12-17 23:47:16,694 - lava_callback - ERROR - Error processing callback for node 69425180bb27571da24b95b2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69425180bb27571da24b95b2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69425180bb27571da24b95b2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69425180bb27571da24b95b2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:16,696 - lava_callback - INFO - Starting async job submit for node 69409a46575bc060ce161f41 2025-12-17 23:47:16,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:16,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a46575bc060ce161f41 HTTP/1.1" 502 150 2025-12-17 23:47:16,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a46575bc060ce161f41'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:16,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a46575bc060ce161f41 2025-12-17 23:47:16,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a46575bc060ce161f41 HTTP/1.1" 502 150 2025-12-17 23:47:16,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a46575bc060ce161f41'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:16,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3c575bc060ce162544 2025-12-17 23:47:16,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3c575bc060ce162544 HTTP/1.1" 502 150 2025-12-17 23:47:16,737 - lava_callback - ERROR - Error processing callback for node 69409c3c575bc060ce162544: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c3c575bc060ce162544 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c3c575bc060ce162544 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c3c575bc060ce162544 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:16,739 - lava_callback - INFO - Starting async job submit for node 69409a48575bc060ce161f44 2025-12-17 23:47:16,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:16,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a48575bc060ce161f44 HTTP/1.1" 502 150 2025-12-17 23:47:16,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a48575bc060ce161f44'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:16,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a48575bc060ce161f44 2025-12-17 23:47:16,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a48575bc060ce161f44 HTTP/1.1" 502 150 2025-12-17 23:47:16,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a48575bc060ce161f44'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:17,659 - lava_callback - INFO - Received callback for node 69409dfe575bc060ce162916 INFO: 172.21.0.1:37686 - "POST /node/69409dfe575bc060ce162916 HTTP/1.0" 202 Accepted 2025-12-17 23:47:18,317 - lava_callback - INFO - Received callback for node 69409e03575bc060ce16291e INFO: 172.21.0.1:37694 - "POST /node/69409e03575bc060ce16291e HTTP/1.0" 202 Accepted 2025-12-17 23:47:18,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a27575bc060ce161efa 2025-12-17 23:47:18,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a27575bc060ce161efa HTTP/1.1" 502 150 2025-12-17 23:47:18,532 - lava_callback - ERROR - Error processing callback for node 69409a27575bc060ce161efa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a27575bc060ce161efa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a27575bc060ce161efa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a27575bc060ce161efa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:18,534 - lava_callback - INFO - Starting async job submit for node 69409a4b575bc060ce161f47 2025-12-17 23:47:18,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:18,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4b575bc060ce161f47 HTTP/1.1" 502 150 2025-12-17 23:47:18,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a4b575bc060ce161f47'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:18,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a4b575bc060ce161f47 2025-12-17 23:47:18,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4b575bc060ce161f47 HTTP/1.1" 502 150 2025-12-17 23:47:18,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a4b575bc060ce161f47'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:18,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a46575bc060ce161f41 2025-12-17 23:47:18,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a46575bc060ce161f41 HTTP/1.1" 502 150 2025-12-17 23:47:18,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a46575bc060ce161f41'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:18,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a48575bc060ce161f44 2025-12-17 23:47:18,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a48575bc060ce161f44 HTTP/1.1" 502 150 2025-12-17 23:47:18,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a48575bc060ce161f44'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:18,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a40575bc060ce161f38 2025-12-17 23:47:18,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a40575bc060ce161f38 HTTP/1.1" 502 150 2025-12-17 23:47:18,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a40575bc060ce161f38'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:18,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69426c3e1b61f1c6005f5f75 2025-12-17 23:47:18,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69426c3e1b61f1c6005f5f75 HTTP/1.1" 502 150 2025-12-17 23:47:18,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69426c3e1b61f1c6005f5f75'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:19,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a31575bc060ce161f11 2025-12-17 23:47:19,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a31575bc060ce161f11 HTTP/1.1" 502 150 2025-12-17 23:47:19,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a31575bc060ce161f11'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:19,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a35575bc060ce161f19 2025-12-17 23:47:19,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a35575bc060ce161f19 HTTP/1.1" 502 150 2025-12-17 23:47:19,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a35575bc060ce161f19'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:19,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a30575bc060ce161f0e 2025-12-17 23:47:19,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a30575bc060ce161f0e HTTP/1.1" 502 150 2025-12-17 23:47:19,730 - lava_callback - ERROR - Error processing callback for node 69409a30575bc060ce161f0e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a30575bc060ce161f0e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a30575bc060ce161f0e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a30575bc060ce161f0e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:19,733 - lava_callback - INFO - Starting async job submit for node 69409a4d575bc060ce161f4a 2025-12-17 23:47:19,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:19,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a33575bc060ce161f16 2025-12-17 23:47:19,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a33575bc060ce161f16 HTTP/1.1" 502 150 2025-12-17 23:47:19,768 - lava_callback - ERROR - Error processing callback for node 69409a33575bc060ce161f16: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a33575bc060ce161f16 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a33575bc060ce161f16 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a33575bc060ce161f16 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:19,769 - lava_callback - INFO - Starting async job submit for node 69409a44575bc060ce161f3e 2025-12-17 23:47:19,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:19,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4d575bc060ce161f4a HTTP/1.1" 502 150 2025-12-17 23:47:19,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a4d575bc060ce161f4a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:19,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a4d575bc060ce161f4a 2025-12-17 23:47:19,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4d575bc060ce161f4a HTTP/1.1" 502 150 2025-12-17 23:47:19,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a4d575bc060ce161f4a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:19,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a44575bc060ce161f3e HTTP/1.1" 502 150 2025-12-17 23:47:19,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a44575bc060ce161f3e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:19,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a44575bc060ce161f3e 2025-12-17 23:47:19,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a44575bc060ce161f3e HTTP/1.1" 502 150 2025-12-17 23:47:19,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a44575bc060ce161f3e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:19,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a073575bc060ce162d34 2025-12-17 23:47:19,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a073575bc060ce162d34 HTTP/1.1" 502 150 2025-12-17 23:47:19,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a073575bc060ce162d34'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:20,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a4b575bc060ce161f47 2025-12-17 23:47:20,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4b575bc060ce161f47 HTTP/1.1" 502 150 2025-12-17 23:47:20,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a4b575bc060ce161f47'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:21,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a4d575bc060ce161f4a 2025-12-17 23:47:21,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4d575bc060ce161f4a HTTP/1.1" 502 150 2025-12-17 23:47:21,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a4d575bc060ce161f4a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:21,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a44575bc060ce161f3e 2025-12-17 23:47:21,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a44575bc060ce161f3e HTTP/1.1" 502 150 2025-12-17 23:47:21,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a44575bc060ce161f3e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:22,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a46575bc060ce161f41 2025-12-17 23:47:22,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a48575bc060ce161f44 2025-12-17 23:47:22,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a48575bc060ce161f44 HTTP/1.1" 502 150 2025-12-17 23:47:22,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a48575bc060ce161f44'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:23,320 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a3b575bc060ce161f24 2025-12-17 23:47:23,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a3b575bc060ce161f24 HTTP/1.1" 502 150 2025-12-17 23:47:23,321 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a3b575bc060ce161f24'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:24,055 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a2e575bc060ce161f0b 2025-12-17 23:47:24,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a4b575bc060ce161f47 2025-12-17 23:47:25,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a4d575bc060ce161f4a 2025-12-17 23:47:25,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a44575bc060ce161f3e INFO: 172.21.0.1:45326 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:47:26,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a40575bc060ce161f38 2025-12-17 23:47:28,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a37575bc060ce161f1c 2025-12-17 23:47:28,364 - lava_callback - INFO - Received callback for node 69409dfb575bc060ce162909 INFO: 172.21.0.1:45338 - "POST /node/69409dfb575bc060ce162909 HTTP/1.0" 202 Accepted 2025-12-17 23:47:29,003 - lava_callback - INFO - Received callback for node 69409e00575bc060ce16291b INFO: 172.21.0.1:45342 - "POST /node/69409e00575bc060ce16291b HTTP/1.0" 202 Accepted 2025-12-17 23:47:30,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a39575bc060ce161f1f 2025-12-17 23:47:30,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a48575bc060ce161f44 2025-12-17 23:47:31,096 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a3e575bc060ce161f31 2025-12-17 23:47:32,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a42575bc060ce161f3b 2025-12-17 23:47:34,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69426c3e1b61f1c6005f5f75 2025-12-17 23:47:35,059 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a31575bc060ce161f11 2025-12-17 23:47:35,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a35575bc060ce161f19 2025-12-17 23:47:35,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a073575bc060ce162d34 INFO: 172.21.0.1:51400 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:47:37,160 - lava_callback - INFO - Received callback for node 6940a4f61e3c24e956819ca5 INFO: 172.21.0.1:51404 - "POST /node/6940a4f61e3c24e956819ca5 HTTP/1.0" 202 Accepted 2025-12-17 23:47:37,690 - lava_callback - INFO - Received callback for node 69409e05575bc060ce162921 INFO: 172.21.0.1:51414 - "POST /node/69409e05575bc060ce162921 HTTP/1.0" 202 Accepted 2025-12-17 23:47:38,345 - lava_callback - INFO - Received callback for node 69409e07575bc060ce162924 INFO: 172.21.0.1:51418 - "POST /node/69409e07575bc060ce162924 HTTP/1.0" 202 Accepted 2025-12-17 23:47:39,017 - lava_callback - INFO - Received callback for node 69409e09575bc060ce162927 INFO: 172.21.0.1:51422 - "POST /node/69409e09575bc060ce162927 HTTP/1.0" 202 Accepted 2025-12-17 23:47:39,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a2e575bc060ce161f0b HTTP/1.1" 502 150 2025-12-17 23:47:39,312 - lava_callback - ERROR - Error processing callback for node 69409a2e575bc060ce161f0b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a2e575bc060ce161f0b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a2e575bc060ce161f0b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a2e575bc060ce161f0b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:39,314 - lava_callback - INFO - Starting async job submit for node 69409a4f575bc060ce161f51 2025-12-17 23:47:39,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:39,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a3b575bc060ce161f24 2025-12-17 23:47:39,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a3b575bc060ce161f24 HTTP/1.1" 502 150 2025-12-17 23:47:39,322 - lava_callback - ERROR - Error processing callback for node 69409a3b575bc060ce161f24: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a3b575bc060ce161f24 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a3b575bc060ce161f24 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a3b575bc060ce161f24 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:39,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4f575bc060ce161f51 HTTP/1.1" 502 150 2025-12-17 23:47:39,325 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a4f575bc060ce161f51'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:39,325 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a4f575bc060ce161f51 2025-12-17 23:47:39,325 - lava_callback - INFO - Starting async job submit for node 69409a51575bc060ce161f5a 2025-12-17 23:47:39,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:39,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4f575bc060ce161f51 HTTP/1.1" 502 150 2025-12-17 23:47:39,326 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a4f575bc060ce161f51'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:39,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a51575bc060ce161f5a HTTP/1.1" 502 150 2025-12-17 23:47:39,353 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a51575bc060ce161f5a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:39,353 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a51575bc060ce161f5a 2025-12-17 23:47:39,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a51575bc060ce161f5a HTTP/1.1" 502 150 2025-12-17 23:47:39,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a51575bc060ce161f5a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:39,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4b575bc060ce161f47 HTTP/1.1" 502 150 2025-12-17 23:47:39,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a4b575bc060ce161f47'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:41,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a44575bc060ce161f3e HTTP/1.1" 502 150 2025-12-17 23:47:41,104 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a44575bc060ce161f3e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:41,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4d575bc060ce161f4a HTTP/1.1" 502 150 2025-12-17 23:47:41,105 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a4d575bc060ce161f4a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:41,326 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a4f575bc060ce161f51 2025-12-17 23:47:41,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4f575bc060ce161f51 HTTP/1.1" 502 150 2025-12-17 23:47:41,328 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a4f575bc060ce161f51'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:41,354 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a51575bc060ce161f5a 2025-12-17 23:47:41,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a51575bc060ce161f5a HTTP/1.1" 502 150 2025-12-17 23:47:41,355 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a51575bc060ce161f5a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:42,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a40575bc060ce161f38 HTTP/1.1" 502 150 2025-12-17 23:47:42,128 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a40575bc060ce161f38'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:43,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a37575bc060ce161f1c HTTP/1.1" 502 150 2025-12-17 23:47:43,408 - lava_callback - ERROR - Error processing callback for node 69409a37575bc060ce161f1c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a37575bc060ce161f1c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a37575bc060ce161f1c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a37575bc060ce161f1c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:43,410 - lava_callback - INFO - Starting async job submit for node 69409a53575bc060ce161f5f 2025-12-17 23:47:43,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:43,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a53575bc060ce161f5f HTTP/1.1" 502 150 2025-12-17 23:47:43,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a53575bc060ce161f5f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:43,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a53575bc060ce161f5f 2025-12-17 23:47:43,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a53575bc060ce161f5f HTTP/1.1" 502 150 2025-12-17 23:47:43,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a53575bc060ce161f5f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:45,328 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a4f575bc060ce161f51 2025-12-17 23:47:45,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4f575bc060ce161f51 HTTP/1.1" 502 150 2025-12-17 23:47:45,329 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a4f575bc060ce161f51'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:45,356 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a51575bc060ce161f5a 2025-12-17 23:47:45,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a51575bc060ce161f5a HTTP/1.1" 502 150 2025-12-17 23:47:45,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a51575bc060ce161f5a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:45,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a53575bc060ce161f5f 2025-12-17 23:47:45,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a53575bc060ce161f5f HTTP/1.1" 502 150 2025-12-17 23:47:45,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a53575bc060ce161f5f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:45,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a39575bc060ce161f1f HTTP/1.1" 502 150 2025-12-17 23:47:45,968 - lava_callback - ERROR - Error processing callback for node 69409a39575bc060ce161f1f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a39575bc060ce161f1f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a39575bc060ce161f1f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a39575bc060ce161f1f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:45,971 - lava_callback - INFO - Starting async job submit for node 69409a55575bc060ce161f62 2025-12-17 23:47:45,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:46,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a55575bc060ce161f62 HTTP/1.1" 502 150 2025-12-17 23:47:46,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a55575bc060ce161f62'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:46,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a55575bc060ce161f62 2025-12-17 23:47:46,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a55575bc060ce161f62 HTTP/1.1" 502 150 2025-12-17 23:47:46,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a55575bc060ce161f62'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:46,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a48575bc060ce161f44 HTTP/1.1" 502 150 2025-12-17 23:47:46,228 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a48575bc060ce161f44'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:46,265 - lava_callback - INFO - Received callback for node 6940a6c21e3c24e956819dd8 INFO: 172.21.0.1:38748 - "POST /node/6940a6c21e3c24e956819dd8 HTTP/1.0" 202 Accepted 2025-12-17 23:47:46,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a3e575bc060ce161f31 HTTP/1.1" 502 150 2025-12-17 23:47:46,480 - lava_callback - ERROR - Error processing callback for node 69409a3e575bc060ce161f31: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a3e575bc060ce161f31 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a3e575bc060ce161f31 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a3e575bc060ce161f31 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:46,482 - lava_callback - INFO - Starting async job submit for node 69409a5e575bc060ce161f68 2025-12-17 23:47:46,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:46,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a5e575bc060ce161f68 HTTP/1.1" 502 150 2025-12-17 23:47:46,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a5e575bc060ce161f68'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:46,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a5e575bc060ce161f68 2025-12-17 23:47:46,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a5e575bc060ce161f68 HTTP/1.1" 502 150 2025-12-17 23:47:46,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a5e575bc060ce161f68'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38750 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:47:47,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a42575bc060ce161f3b HTTP/1.1" 502 150 2025-12-17 23:47:47,505 - lava_callback - ERROR - Error processing callback for node 69409a42575bc060ce161f3b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a42575bc060ce161f3b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a42575bc060ce161f3b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a42575bc060ce161f3b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:47,507 - lava_callback - INFO - Starting async job submit for node 69409a5b575bc060ce161f65 2025-12-17 23:47:47,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:47,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a5b575bc060ce161f65 HTTP/1.1" 502 150 2025-12-17 23:47:47,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a5b575bc060ce161f65'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:47,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a5b575bc060ce161f65 2025-12-17 23:47:47,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a5b575bc060ce161f65 HTTP/1.1" 502 150 2025-12-17 23:47:47,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a5b575bc060ce161f65'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:47,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a4b575bc060ce161f47 2025-12-17 23:47:47,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4b575bc060ce161f47 HTTP/1.1" 502 150 2025-12-17 23:47:47,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a4b575bc060ce161f47'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:48,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a55575bc060ce161f62 2025-12-17 23:47:48,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a55575bc060ce161f62 HTTP/1.1" 502 150 2025-12-17 23:47:48,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a55575bc060ce161f62'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:48,378 - lava_callback - INFO - Received callback for node 69409e0f575bc060ce16292c INFO: 172.21.0.1:38762 - "POST /node/69409e0f575bc060ce16292c HTTP/1.0" 202 Accepted 2025-12-17 23:47:48,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a5e575bc060ce161f68 2025-12-17 23:47:48,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a5e575bc060ce161f68 HTTP/1.1" 502 150 2025-12-17 23:47:48,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a5e575bc060ce161f68'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:49,105 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a44575bc060ce161f3e 2025-12-17 23:47:49,105 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a4d575bc060ce161f4a 2025-12-17 23:47:49,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4d575bc060ce161f4a HTTP/1.1" 502 150 2025-12-17 23:47:49,106 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a4d575bc060ce161f4a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:49,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a53575bc060ce161f5f 2025-12-17 23:47:49,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a53575bc060ce161f5f HTTP/1.1" 502 150 2025-12-17 23:47:49,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a53575bc060ce161f5f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:49,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a5b575bc060ce161f65 2025-12-17 23:47:49,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a5b575bc060ce161f65 HTTP/1.1" 502 150 2025-12-17 23:47:49,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a5b575bc060ce161f65'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:50,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a31575bc060ce161f11 HTTP/1.1" 502 150 2025-12-17 23:47:50,321 - lava_callback - ERROR - Error processing callback for node 69409a31575bc060ce161f11: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a31575bc060ce161f11 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a31575bc060ce161f11 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a31575bc060ce161f11 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:50,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69426c3e1b61f1c6005f5f75 HTTP/1.1" 502 150 2025-12-17 23:47:50,324 - lava_callback - ERROR - Error processing callback for node 69426c3e1b61f1c6005f5f75: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69426c3e1b61f1c6005f5f75 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69426c3e1b61f1c6005f5f75 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69426c3e1b61f1c6005f5f75 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:50,323 - lava_callback - INFO - Starting async job submit for node 69409a60575bc060ce161f6f 2025-12-17 23:47:50,326 - lava_callback - INFO - Starting async job submit for node 69409c41575bc060ce162550 2025-12-17 23:47:50,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:50,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:50,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a60575bc060ce161f6f HTTP/1.1" 502 150 2025-12-17 23:47:50,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a60575bc060ce161f6f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:50,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a60575bc060ce161f6f 2025-12-17 23:47:50,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a60575bc060ce161f6f HTTP/1.1" 502 150 2025-12-17 23:47:50,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a60575bc060ce161f6f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:50,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c41575bc060ce162550 HTTP/1.1" 502 150 2025-12-17 23:47:50,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c41575bc060ce162550'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:50,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c41575bc060ce162550 2025-12-17 23:47:50,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c41575bc060ce162550 HTTP/1.1" 502 150 2025-12-17 23:47:50,388 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c41575bc060ce162550'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:50,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a35575bc060ce161f19 HTTP/1.1" 502 150 2025-12-17 23:47:50,832 - lava_callback - ERROR - Error processing callback for node 69409a35575bc060ce161f19: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a35575bc060ce161f19 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a35575bc060ce161f19 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a35575bc060ce161f19 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:50,834 - lava_callback - INFO - Starting async job submit for node 69409a62575bc060ce161f74 2025-12-17 23:47:50,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:50,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a62575bc060ce161f74 HTTP/1.1" 502 150 2025-12-17 23:47:50,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a62575bc060ce161f74'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:50,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a62575bc060ce161f74 2025-12-17 23:47:50,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a62575bc060ce161f74 HTTP/1.1" 502 150 2025-12-17 23:47:50,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a62575bc060ce161f74'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:51,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a073575bc060ce162d34 HTTP/1.1" 502 150 2025-12-17 23:47:51,089 - lava_callback - ERROR - Error processing callback for node 6940a073575bc060ce162d34: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a073575bc060ce162d34 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a073575bc060ce162d34 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a073575bc060ce162d34 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:51,091 - lava_callback - INFO - Starting async job submit for node 69409a66575bc060ce161f88 2025-12-17 23:47:51,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:51,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a66575bc060ce161f88 HTTP/1.1" 502 150 2025-12-17 23:47:51,122 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a66575bc060ce161f88'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:51,122 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a66575bc060ce161f88 2025-12-17 23:47:51,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a66575bc060ce161f88 HTTP/1.1" 502 150 2025-12-17 23:47:51,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a66575bc060ce161f88'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:52,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a55575bc060ce161f62 2025-12-17 23:47:52,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a55575bc060ce161f62 HTTP/1.1" 502 150 2025-12-17 23:47:52,005 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a55575bc060ce161f62'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:52,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a60575bc060ce161f6f 2025-12-17 23:47:52,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a60575bc060ce161f6f HTTP/1.1" 502 150 2025-12-17 23:47:52,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a60575bc060ce161f6f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:52,388 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c41575bc060ce162550 2025-12-17 23:47:52,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c41575bc060ce162550 HTTP/1.1" 502 150 2025-12-17 23:47:52,389 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c41575bc060ce162550'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:52,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a5e575bc060ce161f68 2025-12-17 23:47:52,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a5e575bc060ce161f68 HTTP/1.1" 502 150 2025-12-17 23:47:52,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a5e575bc060ce161f68'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:52,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a62575bc060ce161f74 2025-12-17 23:47:52,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a62575bc060ce161f74 HTTP/1.1" 502 150 2025-12-17 23:47:52,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a62575bc060ce161f74'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:53,123 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a66575bc060ce161f88 2025-12-17 23:47:53,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a66575bc060ce161f88 HTTP/1.1" 502 150 2025-12-17 23:47:53,124 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a66575bc060ce161f88'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:53,329 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a4f575bc060ce161f51 2025-12-17 23:47:53,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4f575bc060ce161f51 HTTP/1.1" 502 150 2025-12-17 23:47:53,330 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a4f575bc060ce161f51'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:53,357 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a51575bc060ce161f5a 2025-12-17 23:47:53,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a51575bc060ce161f5a HTTP/1.1" 502 150 2025-12-17 23:47:53,358 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a51575bc060ce161f5a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:53,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a46575bc060ce161f41 HTTP/1.1" 502 150 2025-12-17 23:47:53,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a46575bc060ce161f41'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:53,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a5b575bc060ce161f65 2025-12-17 23:47:53,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a5b575bc060ce161f65 HTTP/1.1" 502 150 2025-12-17 23:47:53,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a5b575bc060ce161f65'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:56,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a60575bc060ce161f6f 2025-12-17 23:47:56,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a60575bc060ce161f6f HTTP/1.1" 502 150 2025-12-17 23:47:56,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a60575bc060ce161f6f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:56,390 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c41575bc060ce162550 2025-12-17 23:47:56,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c41575bc060ce162550 HTTP/1.1" 502 150 2025-12-17 23:47:56,390 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c41575bc060ce162550'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48556 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:47:56,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a62575bc060ce161f74 2025-12-17 23:47:56,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a62575bc060ce161f74 HTTP/1.1" 502 150 2025-12-17 23:47:56,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a62575bc060ce161f74'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:57,124 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a66575bc060ce161f88 2025-12-17 23:47:57,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a66575bc060ce161f88 HTTP/1.1" 502 150 2025-12-17 23:47:57,125 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a66575bc060ce161f88'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:57,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a53575bc060ce161f5f 2025-12-17 23:47:57,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a53575bc060ce161f5f HTTP/1.1" 502 150 2025-12-17 23:47:57,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a53575bc060ce161f5f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:58,129 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a40575bc060ce161f38 2025-12-17 23:47:58,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a40575bc060ce161f38 HTTP/1.1" 502 150 2025-12-17 23:47:58,130 - lava_callback - ERROR - Error processing callback for node 69409a40575bc060ce161f38: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a40575bc060ce161f38 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a40575bc060ce161f38 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a40575bc060ce161f38 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:47:58,132 - lava_callback - INFO - Starting async job submit for node 69409a77575bc060ce161fa7 2025-12-17 23:47:58,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:47:58,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a77575bc060ce161fa7 HTTP/1.1" 502 150 2025-12-17 23:47:58,138 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a77575bc060ce161fa7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:47:58,138 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a77575bc060ce161fa7 2025-12-17 23:47:58,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a77575bc060ce161fa7 HTTP/1.1" 502 150 2025-12-17 23:47:58,139 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a77575bc060ce161fa7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:00,005 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a55575bc060ce161f62 2025-12-17 23:48:00,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a55575bc060ce161f62 HTTP/1.1" 502 150 2025-12-17 23:48:00,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a55575bc060ce161f62'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:00,139 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a77575bc060ce161fa7 2025-12-17 23:48:00,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a77575bc060ce161fa7 HTTP/1.1" 502 150 2025-12-17 23:48:00,141 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a77575bc060ce161fa7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:00,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a5e575bc060ce161f68 2025-12-17 23:48:00,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a5e575bc060ce161f68 HTTP/1.1" 502 150 2025-12-17 23:48:00,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a5e575bc060ce161f68'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:01,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a46575bc060ce161f41 2025-12-17 23:48:01,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a46575bc060ce161f41 HTTP/1.1" 502 150 2025-12-17 23:48:01,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a46575bc060ce161f41'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:01,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a5b575bc060ce161f65 2025-12-17 23:48:01,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a5b575bc060ce161f65 HTTP/1.1" 502 150 2025-12-17 23:48:01,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a5b575bc060ce161f65'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:02,228 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a48575bc060ce161f44 2025-12-17 23:48:02,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a48575bc060ce161f44 HTTP/1.1" 502 150 2025-12-17 23:48:02,230 - lava_callback - ERROR - Error processing callback for node 69409a48575bc060ce161f44: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a48575bc060ce161f44 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a48575bc060ce161f44 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a48575bc060ce161f44 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:02,232 - lava_callback - INFO - Starting async job submit for node 69409a6c575bc060ce161f91 2025-12-17 23:48:02,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:02,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a6c575bc060ce161f91 HTTP/1.1" 502 150 2025-12-17 23:48:02,261 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a6c575bc060ce161f91'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:02,261 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a6c575bc060ce161f91 2025-12-17 23:48:02,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a6c575bc060ce161f91 HTTP/1.1" 502 150 2025-12-17 23:48:02,262 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a6c575bc060ce161f91'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38788 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:48:03,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a4b575bc060ce161f47 2025-12-17 23:48:03,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4b575bc060ce161f47 HTTP/1.1" 502 150 2025-12-17 23:48:03,827 - lava_callback - ERROR - Error processing callback for node 69409a4b575bc060ce161f47: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a4b575bc060ce161f47 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a4b575bc060ce161f47 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a4b575bc060ce161f47 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:03,829 - lava_callback - INFO - Starting async job submit for node 69409a6e575bc060ce161f94 2025-12-17 23:48:03,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:03,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a6e575bc060ce161f94 HTTP/1.1" 502 150 2025-12-17 23:48:03,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a6e575bc060ce161f94'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:03,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a6e575bc060ce161f94 2025-12-17 23:48:03,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a6e575bc060ce161f94 HTTP/1.1" 502 150 2025-12-17 23:48:03,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a6e575bc060ce161f94'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:04,141 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a77575bc060ce161fa7 2025-12-17 23:48:04,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a77575bc060ce161fa7 HTTP/1.1" 502 150 2025-12-17 23:48:04,142 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a77575bc060ce161fa7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:04,262 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a6c575bc060ce161f91 2025-12-17 23:48:04,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a6c575bc060ce161f91 HTTP/1.1" 502 150 2025-12-17 23:48:04,263 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a6c575bc060ce161f91'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:04,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a60575bc060ce161f6f 2025-12-17 23:48:04,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a60575bc060ce161f6f HTTP/1.1" 502 150 2025-12-17 23:48:04,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a60575bc060ce161f6f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:04,391 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c41575bc060ce162550 2025-12-17 23:48:04,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c41575bc060ce162550 HTTP/1.1" 502 150 2025-12-17 23:48:04,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c41575bc060ce162550'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:04,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a44575bc060ce161f3e HTTP/1.1" 502 150 2025-12-17 23:48:04,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a44575bc060ce161f3e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:04,542 - lava_callback - INFO - Received callback for node 6941273d459ff5fc64d2bf54 INFO: 172.21.0.1:38802 - "POST /node/6941273d459ff5fc64d2bf54 HTTP/1.0" 202 Accepted 2025-12-17 23:48:04,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a62575bc060ce161f74 2025-12-17 23:48:04,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a62575bc060ce161f74 HTTP/1.1" 502 150 2025-12-17 23:48:04,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a62575bc060ce161f74'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:05,107 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a4d575bc060ce161f4a 2025-12-17 23:48:05,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4d575bc060ce161f4a HTTP/1.1" 502 150 2025-12-17 23:48:05,108 - lava_callback - ERROR - Error processing callback for node 69409a4d575bc060ce161f4a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a4d575bc060ce161f4a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a4d575bc060ce161f4a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a4d575bc060ce161f4a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:05,110 - lava_callback - INFO - Starting async job submit for node 69409a70575bc060ce161f97 2025-12-17 23:48:05,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:05,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a70575bc060ce161f97 HTTP/1.1" 502 150 2025-12-17 23:48:05,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a70575bc060ce161f97'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:05,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a70575bc060ce161f97 2025-12-17 23:48:05,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a70575bc060ce161f97 HTTP/1.1" 502 150 2025-12-17 23:48:05,118 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a70575bc060ce161f97'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:05,126 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a66575bc060ce161f88 2025-12-17 23:48:05,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a66575bc060ce161f88 HTTP/1.1" 502 150 2025-12-17 23:48:05,127 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a66575bc060ce161f88'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:05,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a6e575bc060ce161f94 2025-12-17 23:48:05,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a6e575bc060ce161f94 HTTP/1.1" 502 150 2025-12-17 23:48:05,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a6e575bc060ce161f94'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38814 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:48:07,118 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a70575bc060ce161f97 2025-12-17 23:48:07,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a70575bc060ce161f97 HTTP/1.1" 502 150 2025-12-17 23:48:07,119 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a70575bc060ce161f97'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:08,264 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a6c575bc060ce161f91 2025-12-17 23:48:08,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a6c575bc060ce161f91 HTTP/1.1" 502 150 2025-12-17 23:48:08,265 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a6c575bc060ce161f91'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:08,677 - lava_callback - INFO - Received callback for node 6940ab631e3c24e95681a57a INFO: 172.21.0.1:38820 - "POST /node/6940ab631e3c24e95681a57a HTTP/1.0" 202 Accepted 2025-12-17 23:48:09,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a4f575bc060ce161f51 2025-12-17 23:48:09,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a4f575bc060ce161f51 HTTP/1.1" 502 150 2025-12-17 23:48:09,332 - lava_callback - ERROR - Error processing callback for node 69409a4f575bc060ce161f51: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a4f575bc060ce161f51 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a4f575bc060ce161f51 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a4f575bc060ce161f51 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:09,334 - lava_callback - INFO - Starting async job submit for node 69409a72575bc060ce161f9a 2025-12-17 23:48:09,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:09,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a72575bc060ce161f9a HTTP/1.1" 502 150 2025-12-17 23:48:09,342 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a72575bc060ce161f9a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:09,342 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a72575bc060ce161f9a 2025-12-17 23:48:09,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a72575bc060ce161f9a HTTP/1.1" 502 150 2025-12-17 23:48:09,343 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a72575bc060ce161f9a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:09,359 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a51575bc060ce161f5a 2025-12-17 23:48:09,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a51575bc060ce161f5a HTTP/1.1" 502 150 2025-12-17 23:48:09,360 - lava_callback - ERROR - Error processing callback for node 69409a51575bc060ce161f5a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a51575bc060ce161f5a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a51575bc060ce161f5a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a51575bc060ce161f5a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:09,361 - lava_callback - INFO - Starting async job submit for node 69409a74575bc060ce161f9d 2025-12-17 23:48:09,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:09,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a74575bc060ce161f9d HTTP/1.1" 502 150 2025-12-17 23:48:09,391 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a74575bc060ce161f9d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:09,391 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a74575bc060ce161f9d 2025-12-17 23:48:09,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a74575bc060ce161f9d HTTP/1.1" 502 150 2025-12-17 23:48:09,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a74575bc060ce161f9d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:09,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a6e575bc060ce161f94 2025-12-17 23:48:09,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a6e575bc060ce161f94 HTTP/1.1" 502 150 2025-12-17 23:48:09,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a6e575bc060ce161f94'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:10,384 - lava_callback - INFO - Received callback for node 6940ab891e3c24e95681a5bc INFO: 172.21.0.1:51958 - "POST /node/6940ab891e3c24e95681a5bc HTTP/1.0" 202 Accepted 2025-12-17 23:48:11,120 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a70575bc060ce161f97 2025-12-17 23:48:11,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a70575bc060ce161f97 HTTP/1.1" 502 150 2025-12-17 23:48:11,122 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a70575bc060ce161f97'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:11,343 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a72575bc060ce161f9a 2025-12-17 23:48:11,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a72575bc060ce161f9a HTTP/1.1" 502 150 2025-12-17 23:48:11,346 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a72575bc060ce161f9a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:11,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a74575bc060ce161f9d 2025-12-17 23:48:11,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a74575bc060ce161f9d HTTP/1.1" 502 150 2025-12-17 23:48:11,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a74575bc060ce161f9d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:12,142 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a77575bc060ce161fa7 2025-12-17 23:48:12,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a77575bc060ce161fa7 HTTP/1.1" 502 150 2025-12-17 23:48:12,144 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a77575bc060ce161fa7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:13,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a53575bc060ce161f5f 2025-12-17 23:48:13,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a53575bc060ce161f5f HTTP/1.1" 502 150 2025-12-17 23:48:13,451 - lava_callback - ERROR - Error processing callback for node 69409a53575bc060ce161f5f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a53575bc060ce161f5f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a53575bc060ce161f5f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a53575bc060ce161f5f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:13,453 - lava_callback - INFO - Starting async job submit for node 69409a76575bc060ce161fa0 2025-12-17 23:48:13,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:13,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a76575bc060ce161fa0 HTTP/1.1" 502 150 2025-12-17 23:48:13,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a76575bc060ce161fa0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:13,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a76575bc060ce161fa0 2025-12-17 23:48:13,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a76575bc060ce161fa0 HTTP/1.1" 502 150 2025-12-17 23:48:13,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a76575bc060ce161fa0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51974 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:48:15,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a72575bc060ce161f9a 2025-12-17 23:48:15,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a74575bc060ce161f9d 2025-12-17 23:48:15,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a76575bc060ce161fa0 2025-12-17 23:48:15,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a76575bc060ce161fa0 HTTP/1.1" 502 150 2025-12-17 23:48:15,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a76575bc060ce161fa0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:16,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a55575bc060ce161f62 2025-12-17 23:48:16,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a55575bc060ce161f62 HTTP/1.1" 502 150 2025-12-17 23:48:16,008 - lava_callback - ERROR - Error processing callback for node 69409a55575bc060ce161f62: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a55575bc060ce161f62 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a55575bc060ce161f62 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a55575bc060ce161f62 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:16,010 - lava_callback - INFO - Starting async job submit for node 69409d67575bc060ce1627fe 2025-12-17 23:48:16,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:16,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d67575bc060ce1627fe HTTP/1.1" 502 150 2025-12-17 23:48:16,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d67575bc060ce1627fe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:16,080 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d67575bc060ce1627fe 2025-12-17 23:48:16,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d67575bc060ce1627fe HTTP/1.1" 502 150 2025-12-17 23:48:16,081 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d67575bc060ce1627fe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:16,265 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a6c575bc060ce161f91 2025-12-17 23:48:16,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a6c575bc060ce161f91 HTTP/1.1" 502 150 2025-12-17 23:48:16,266 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a6c575bc060ce161f91'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:16,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a5e575bc060ce161f68 2025-12-17 23:48:16,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a5e575bc060ce161f68 HTTP/1.1" 502 150 2025-12-17 23:48:16,497 - lava_callback - ERROR - Error processing callback for node 69409a5e575bc060ce161f68: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a5e575bc060ce161f68 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a5e575bc060ce161f68 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a5e575bc060ce161f68 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:16,499 - lava_callback - INFO - Starting async job submit for node 69409a78575bc060ce161fae 2025-12-17 23:48:16,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:16,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a78575bc060ce161fae HTTP/1.1" 502 150 2025-12-17 23:48:16,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a78575bc060ce161fae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:16,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a78575bc060ce161fae 2025-12-17 23:48:16,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a78575bc060ce161fae HTTP/1.1" 502 150 2025-12-17 23:48:16,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a78575bc060ce161fae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51980 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:48:17,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a46575bc060ce161f41 2025-12-17 23:48:17,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a46575bc060ce161f41 HTTP/1.1" 502 150 2025-12-17 23:48:17,395 - lava_callback - ERROR - Error processing callback for node 69409a46575bc060ce161f41: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a46575bc060ce161f41 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a46575bc060ce161f41 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a46575bc060ce161f41 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:17,398 - lava_callback - INFO - Starting async job submit for node 69409a7a575bc060ce161fb7 2025-12-17 23:48:17,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:17,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7a575bc060ce161fb7 HTTP/1.1" 502 150 2025-12-17 23:48:17,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a7a575bc060ce161fb7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:17,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a7a575bc060ce161fb7 2025-12-17 23:48:17,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7a575bc060ce161fb7 HTTP/1.1" 502 150 2025-12-17 23:48:17,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a7a575bc060ce161fb7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:17,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a5b575bc060ce161f65 2025-12-17 23:48:17,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a5b575bc060ce161f65 HTTP/1.1" 502 150 2025-12-17 23:48:17,524 - lava_callback - ERROR - Error processing callback for node 69409a5b575bc060ce161f65: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a5b575bc060ce161f65 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a5b575bc060ce161f65 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a5b575bc060ce161f65 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:17,527 - lava_callback - INFO - Starting async job submit for node 69409a7c575bc060ce161fbc 2025-12-17 23:48:17,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:17,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7c575bc060ce161fbc HTTP/1.1" 502 150 2025-12-17 23:48:17,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a7c575bc060ce161fbc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:17,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a7c575bc060ce161fbc 2025-12-17 23:48:17,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7c575bc060ce161fbc HTTP/1.1" 502 150 2025-12-17 23:48:17,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a7c575bc060ce161fbc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:17,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a6e575bc060ce161f94 2025-12-17 23:48:17,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a6e575bc060ce161f94 HTTP/1.1" 502 150 2025-12-17 23:48:17,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a6e575bc060ce161f94'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:18,081 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d67575bc060ce1627fe 2025-12-17 23:48:18,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d67575bc060ce1627fe HTTP/1.1" 502 150 2025-12-17 23:48:18,082 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d67575bc060ce1627fe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:18,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a78575bc060ce161fae 2025-12-17 23:48:18,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a78575bc060ce161fae HTTP/1.1" 502 150 2025-12-17 23:48:18,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a78575bc060ce161fae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:19,123 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a70575bc060ce161f97 2025-12-17 23:48:19,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a70575bc060ce161f97 HTTP/1.1" 502 150 2025-12-17 23:48:19,124 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a70575bc060ce161f97'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:19,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a7a575bc060ce161fb7 2025-12-17 23:48:19,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7a575bc060ce161fb7 HTTP/1.1" 502 150 2025-12-17 23:48:19,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a7a575bc060ce161fb7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:19,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a76575bc060ce161fa0 2025-12-17 23:48:19,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a76575bc060ce161fa0 HTTP/1.1" 502 150 2025-12-17 23:48:19,481 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a76575bc060ce161fa0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:19,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a7c575bc060ce161fbc 2025-12-17 23:48:19,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7c575bc060ce161fbc HTTP/1.1" 502 150 2025-12-17 23:48:19,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a7c575bc060ce161fbc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:20,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a60575bc060ce161f6f 2025-12-17 23:48:20,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a60575bc060ce161f6f HTTP/1.1" 502 150 2025-12-17 23:48:20,389 - lava_callback - ERROR - Error processing callback for node 69409a60575bc060ce161f6f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a60575bc060ce161f6f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a60575bc060ce161f6f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a60575bc060ce161f6f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:20,391 - lava_callback - INFO - Starting async job submit for node 69409a7e575bc060ce161fbf 2025-12-17 23:48:20,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:20,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c41575bc060ce162550 2025-12-17 23:48:20,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c41575bc060ce162550 HTTP/1.1" 502 150 2025-12-17 23:48:20,395 - lava_callback - ERROR - Error processing callback for node 69409c41575bc060ce162550: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c41575bc060ce162550 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c41575bc060ce162550 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c41575bc060ce162550 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:20,398 - lava_callback - INFO - Starting async job submit for node 69409a84575bc060ce161fc2 2025-12-17 23:48:20,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:20,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a44575bc060ce161f3e 2025-12-17 23:48:20,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a44575bc060ce161f3e HTTP/1.1" 502 150 2025-12-17 23:48:20,402 - lava_callback - ERROR - Error processing callback for node 69409a44575bc060ce161f3e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a44575bc060ce161f3e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a44575bc060ce161f3e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a44575bc060ce161f3e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:20,406 - lava_callback - INFO - Starting async job submit for node 69409a87575bc060ce161fc8 2025-12-17 23:48:20,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:20,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7e575bc060ce161fbf HTTP/1.1" 502 150 2025-12-17 23:48:20,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a7e575bc060ce161fbf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:20,409 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a7e575bc060ce161fbf 2025-12-17 23:48:20,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7e575bc060ce161fbf HTTP/1.1" 502 150 2025-12-17 23:48:20,409 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a7e575bc060ce161fbf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:20,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a84575bc060ce161fc2 HTTP/1.1" 502 150 2025-12-17 23:48:20,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a84575bc060ce161fc2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:20,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a84575bc060ce161fc2 2025-12-17 23:48:20,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a84575bc060ce161fc2 HTTP/1.1" 502 150 2025-12-17 23:48:20,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a84575bc060ce161fc2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:20,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a87575bc060ce161fc8 HTTP/1.1" 502 150 2025-12-17 23:48:20,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a87575bc060ce161fc8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:20,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a87575bc060ce161fc8 2025-12-17 23:48:20,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a87575bc060ce161fc8 HTTP/1.1" 502 150 2025-12-17 23:48:20,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a87575bc060ce161fc8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:20,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a62575bc060ce161f74 2025-12-17 23:48:20,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a62575bc060ce161f74 HTTP/1.1" 502 150 2025-12-17 23:48:20,852 - lava_callback - ERROR - Error processing callback for node 69409a62575bc060ce161f74: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a62575bc060ce161f74 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a62575bc060ce161f74 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a62575bc060ce161f74 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:20,855 - lava_callback - INFO - Starting async job submit for node 69409a89575bc060ce161fcb 2025-12-17 23:48:20,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:20,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a89575bc060ce161fcb HTTP/1.1" 502 150 2025-12-17 23:48:20,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a89575bc060ce161fcb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:20,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a89575bc060ce161fcb 2025-12-17 23:48:20,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a89575bc060ce161fcb HTTP/1.1" 502 150 2025-12-17 23:48:20,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a89575bc060ce161fcb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:21,127 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a66575bc060ce161f88 2025-12-17 23:48:21,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a66575bc060ce161f88 HTTP/1.1" 502 150 2025-12-17 23:48:21,128 - lava_callback - ERROR - Error processing callback for node 69409a66575bc060ce161f88: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a66575bc060ce161f88 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a66575bc060ce161f88 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a66575bc060ce161f88 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:21,130 - lava_callback - INFO - Starting async job submit for node 69409a8b575bc060ce161fd4 2025-12-17 23:48:21,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:21,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a8b575bc060ce161fd4 HTTP/1.1" 502 150 2025-12-17 23:48:21,137 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a8b575bc060ce161fd4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:21,137 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a8b575bc060ce161fd4 2025-12-17 23:48:21,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a8b575bc060ce161fd4 HTTP/1.1" 502 150 2025-12-17 23:48:21,138 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a8b575bc060ce161fd4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:22,082 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d67575bc060ce1627fe 2025-12-17 23:48:22,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d67575bc060ce1627fe HTTP/1.1" 502 150 2025-12-17 23:48:22,084 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d67575bc060ce1627fe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:22,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a7e575bc060ce161fbf 2025-12-17 23:48:22,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7e575bc060ce161fbf HTTP/1.1" 502 150 2025-12-17 23:48:22,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a7e575bc060ce161fbf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:22,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a84575bc060ce161fc2 2025-12-17 23:48:22,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a84575bc060ce161fc2 HTTP/1.1" 502 150 2025-12-17 23:48:22,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a84575bc060ce161fc2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:22,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a87575bc060ce161fc8 2025-12-17 23:48:22,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a87575bc060ce161fc8 HTTP/1.1" 502 150 2025-12-17 23:48:22,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a87575bc060ce161fc8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:22,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a78575bc060ce161fae 2025-12-17 23:48:22,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a78575bc060ce161fae HTTP/1.1" 502 150 2025-12-17 23:48:22,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a78575bc060ce161fae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:22,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a89575bc060ce161fcb 2025-12-17 23:48:22,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a89575bc060ce161fcb HTTP/1.1" 502 150 2025-12-17 23:48:22,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a89575bc060ce161fcb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:23,138 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a8b575bc060ce161fd4 2025-12-17 23:48:23,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a8b575bc060ce161fd4 HTTP/1.1" 502 150 2025-12-17 23:48:23,139 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a8b575bc060ce161fd4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:23,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a7a575bc060ce161fb7 2025-12-17 23:48:23,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7a575bc060ce161fb7 HTTP/1.1" 502 150 2025-12-17 23:48:23,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a7a575bc060ce161fb7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:23,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a7c575bc060ce161fbc 2025-12-17 23:48:23,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7c575bc060ce161fbc HTTP/1.1" 502 150 2025-12-17 23:48:23,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a7c575bc060ce161fbc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:26,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a7e575bc060ce161fbf 2025-12-17 23:48:26,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7e575bc060ce161fbf HTTP/1.1" 502 150 2025-12-17 23:48:26,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a7e575bc060ce161fbf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:26,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a84575bc060ce161fc2 2025-12-17 23:48:26,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a84575bc060ce161fc2 HTTP/1.1" 502 150 2025-12-17 23:48:26,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a84575bc060ce161fc2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:26,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a87575bc060ce161fc8 2025-12-17 23:48:26,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a87575bc060ce161fc8 HTTP/1.1" 502 150 2025-12-17 23:48:26,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a87575bc060ce161fc8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58794 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:48:26,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a89575bc060ce161fcb 2025-12-17 23:48:26,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a89575bc060ce161fcb HTTP/1.1" 502 150 2025-12-17 23:48:26,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a89575bc060ce161fcb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:27,140 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a8b575bc060ce161fd4 2025-12-17 23:48:27,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a76575bc060ce161fa0 2025-12-17 23:48:27,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a76575bc060ce161fa0 HTTP/1.1" 502 150 2025-12-17 23:48:27,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a76575bc060ce161fa0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:28,144 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a77575bc060ce161fa7 2025-12-17 23:48:28,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a77575bc060ce161fa7 HTTP/1.1" 502 150 2025-12-17 23:48:28,145 - lava_callback - ERROR - Error processing callback for node 69409a77575bc060ce161fa7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a77575bc060ce161fa7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a77575bc060ce161fa7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a77575bc060ce161fa7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:28,147 - lava_callback - INFO - Starting async job submit for node 69409a8e575bc060ce161fd9 2025-12-17 23:48:28,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:28,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a8e575bc060ce161fd9 HTTP/1.1" 502 150 2025-12-17 23:48:28,154 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a8e575bc060ce161fd9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:28,154 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a8e575bc060ce161fd9 2025-12-17 23:48:28,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a8e575bc060ce161fd9 HTTP/1.1" 502 150 2025-12-17 23:48:28,154 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a8e575bc060ce161fd9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:28,422 - lava_callback - INFO - Received callback for node 69409e37575bc060ce162931 INFO: 172.21.0.1:58802 - "POST /node/69409e37575bc060ce162931 HTTP/1.0" 202 Accepted 2025-12-17 23:48:28,641 - lava_callback - INFO - Received callback for node 6940ae341e3c24e95681aa94 INFO: 172.21.0.1:58804 - "POST /node/6940ae341e3c24e95681aa94 HTTP/1.0" 202 Accepted 2025-12-17 23:48:29,085 - lava_callback - INFO - Received callback for node 69409e3a575bc060ce16293c INFO: 172.21.0.1:58810 - "POST /node/69409e3a575bc060ce16293c HTTP/1.0" 202 Accepted 2025-12-17 23:48:29,737 - lava_callback - INFO - Received callback for node 69409e3c575bc060ce16293f INFO: 172.21.0.1:37338 - "POST /node/69409e3c575bc060ce16293f HTTP/1.0" 202 Accepted 2025-12-17 23:48:30,084 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d67575bc060ce1627fe 2025-12-17 23:48:30,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d67575bc060ce1627fe HTTP/1.1" 502 150 2025-12-17 23:48:30,085 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d67575bc060ce1627fe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:30,155 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a8e575bc060ce161fd9 2025-12-17 23:48:30,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a8e575bc060ce161fd9 HTTP/1.1" 502 150 2025-12-17 23:48:30,160 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a8e575bc060ce161fd9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:30,379 - lava_callback - INFO - Received callback for node 69409e3e575bc060ce162946 INFO: 172.21.0.1:37348 - "POST /node/69409e3e575bc060ce162946 HTTP/1.0" 202 Accepted 2025-12-17 23:48:30,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a78575bc060ce161fae 2025-12-17 23:48:30,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a78575bc060ce161fae HTTP/1.1" 502 150 2025-12-17 23:48:30,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a78575bc060ce161fae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:30,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a74575bc060ce161f9d HTTP/1.1" 502 150 2025-12-17 23:48:30,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a74575bc060ce161f9d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:30,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a72575bc060ce161f9a HTTP/1.1" 502 150 2025-12-17 23:48:30,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a72575bc060ce161f9a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:31,038 - lava_callback - INFO - Received callback for node 69409e40575bc060ce162949 INFO: 172.21.0.1:37360 - "POST /node/69409e40575bc060ce162949 HTTP/1.0" 202 Accepted 2025-12-17 23:48:31,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a7a575bc060ce161fb7 2025-12-17 23:48:31,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7a575bc060ce161fb7 HTTP/1.1" 502 150 2025-12-17 23:48:31,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a7a575bc060ce161fb7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:31,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a7c575bc060ce161fbc 2025-12-17 23:48:31,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7c575bc060ce161fbc HTTP/1.1" 502 150 2025-12-17 23:48:31,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a7c575bc060ce161fbc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:32,266 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a6c575bc060ce161f91 2025-12-17 23:48:32,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a6c575bc060ce161f91 HTTP/1.1" 502 150 2025-12-17 23:48:32,268 - lava_callback - ERROR - Error processing callback for node 69409a6c575bc060ce161f91: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a6c575bc060ce161f91 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a6c575bc060ce161f91 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a6c575bc060ce161f91 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:32,269 - lava_callback - INFO - Starting async job submit for node 69409a90575bc060ce161fdc 2025-12-17 23:48:32,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:32,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a90575bc060ce161fdc HTTP/1.1" 502 150 2025-12-17 23:48:32,279 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a90575bc060ce161fdc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:32,279 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a90575bc060ce161fdc 2025-12-17 23:48:32,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a90575bc060ce161fdc HTTP/1.1" 502 150 2025-12-17 23:48:32,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a90575bc060ce161fdc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:33,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a6e575bc060ce161f94 2025-12-17 23:48:33,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a6e575bc060ce161f94 HTTP/1.1" 502 150 2025-12-17 23:48:33,852 - lava_callback - ERROR - Error processing callback for node 69409a6e575bc060ce161f94: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a6e575bc060ce161f94 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a6e575bc060ce161f94 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a6e575bc060ce161f94 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:33,854 - lava_callback - INFO - Starting async job submit for node 69409a92575bc060ce161fdf 2025-12-17 23:48:33,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:33,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a92575bc060ce161fdf HTTP/1.1" 502 150 2025-12-17 23:48:33,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a92575bc060ce161fdf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:33,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a92575bc060ce161fdf 2025-12-17 23:48:33,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a92575bc060ce161fdf HTTP/1.1" 502 150 2025-12-17 23:48:33,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a92575bc060ce161fdf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:34,161 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a8e575bc060ce161fd9 2025-12-17 23:48:34,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a8e575bc060ce161fd9 HTTP/1.1" 502 150 2025-12-17 23:48:34,162 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a8e575bc060ce161fd9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:34,280 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a90575bc060ce161fdc 2025-12-17 23:48:34,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a90575bc060ce161fdc HTTP/1.1" 502 150 2025-12-17 23:48:34,282 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a90575bc060ce161fdc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:34,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a7e575bc060ce161fbf 2025-12-17 23:48:34,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a84575bc060ce161fc2 2025-12-17 23:48:34,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7e575bc060ce161fbf HTTP/1.1" 502 150 2025-12-17 23:48:34,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a7e575bc060ce161fbf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:34,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a84575bc060ce161fc2 HTTP/1.1" 502 150 2025-12-17 23:48:34,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a84575bc060ce161fc2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:34,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a87575bc060ce161fc8 2025-12-17 23:48:34,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a87575bc060ce161fc8 HTTP/1.1" 502 150 2025-12-17 23:48:34,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a87575bc060ce161fc8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:34,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a89575bc060ce161fcb 2025-12-17 23:48:34,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a89575bc060ce161fcb HTTP/1.1" 502 150 2025-12-17 23:48:34,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a89575bc060ce161fcb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:35,124 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a70575bc060ce161f97 2025-12-17 23:48:35,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a70575bc060ce161f97 HTTP/1.1" 502 150 2025-12-17 23:48:35,125 - lava_callback - ERROR - Error processing callback for node 69409a70575bc060ce161f97: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a70575bc060ce161f97 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a70575bc060ce161f97 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a70575bc060ce161f97 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:35,127 - lava_callback - INFO - Starting async job submit for node 69409a94575bc060ce161fe2 2025-12-17 23:48:35,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:35,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a94575bc060ce161fe2 HTTP/1.1" 502 150 2025-12-17 23:48:35,134 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a94575bc060ce161fe2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:35,134 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a94575bc060ce161fe2 2025-12-17 23:48:35,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a94575bc060ce161fe2 HTTP/1.1" 502 150 2025-12-17 23:48:35,134 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a94575bc060ce161fe2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:35,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a92575bc060ce161fdf 2025-12-17 23:48:35,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a92575bc060ce161fdf HTTP/1.1" 502 150 2025-12-17 23:48:35,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a92575bc060ce161fdf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37370 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:48:37,135 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a94575bc060ce161fe2 2025-12-17 23:48:37,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a94575bc060ce161fe2 HTTP/1.1" 502 150 2025-12-17 23:48:37,136 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a94575bc060ce161fe2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:37,736 - lava_callback - INFO - Received callback for node 69409e4e575bc060ce162953 INFO: 172.21.0.1:37386 - "POST /node/69409e4e575bc060ce162953 HTTP/1.0" 202 Accepted 2025-12-17 23:48:38,282 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a90575bc060ce161fdc 2025-12-17 23:48:38,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a90575bc060ce161fdc HTTP/1.1" 502 150 2025-12-17 23:48:38,283 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a90575bc060ce161fdc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:38,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a74575bc060ce161f9d 2025-12-17 23:48:38,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a72575bc060ce161f9a 2025-12-17 23:48:38,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a74575bc060ce161f9d HTTP/1.1" 502 150 2025-12-17 23:48:38,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a74575bc060ce161f9d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:38,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a72575bc060ce161f9a HTTP/1.1" 502 150 2025-12-17 23:48:38,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a72575bc060ce161f9a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:39,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a92575bc060ce161fdf 2025-12-17 23:48:39,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a92575bc060ce161fdf HTTP/1.1" 502 150 2025-12-17 23:48:39,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a92575bc060ce161fdf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:41,136 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a94575bc060ce161fe2 2025-12-17 23:48:42,162 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a8e575bc060ce161fd9 2025-12-17 23:48:42,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a8e575bc060ce161fd9 HTTP/1.1" 502 150 2025-12-17 23:48:42,163 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a8e575bc060ce161fd9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:42,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a8b575bc060ce161fd4 HTTP/1.1" 502 150 2025-12-17 23:48:42,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a8b575bc060ce161fd4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:43,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a76575bc060ce161fa0 2025-12-17 23:48:43,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a76575bc060ce161fa0 HTTP/1.1" 502 150 2025-12-17 23:48:43,484 - lava_callback - ERROR - Error processing callback for node 69409a76575bc060ce161fa0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a76575bc060ce161fa0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a76575bc060ce161fa0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a76575bc060ce161fa0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:43,486 - lava_callback - INFO - Starting async job submit for node 69409a96575bc060ce161fe5 2025-12-17 23:48:43,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:43,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a96575bc060ce161fe5 HTTP/1.1" 502 150 2025-12-17 23:48:43,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a96575bc060ce161fe5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:43,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a96575bc060ce161fe5 2025-12-17 23:48:43,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a96575bc060ce161fe5 HTTP/1.1" 502 150 2025-12-17 23:48:43,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a96575bc060ce161fe5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:45,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a96575bc060ce161fe5 2025-12-17 23:48:45,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a96575bc060ce161fe5 HTTP/1.1" 502 150 2025-12-17 23:48:45,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a96575bc060ce161fe5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:46,085 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d67575bc060ce1627fe 2025-12-17 23:48:46,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d67575bc060ce1627fe HTTP/1.1" 502 150 2025-12-17 23:48:46,086 - lava_callback - ERROR - Error processing callback for node 69409d67575bc060ce1627fe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d67575bc060ce1627fe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d67575bc060ce1627fe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d67575bc060ce1627fe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:46,088 - lava_callback - INFO - Starting async job submit for node 69409a9a575bc060ce161feb 2025-12-17 23:48:46,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:46,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9a575bc060ce161feb HTTP/1.1" 502 150 2025-12-17 23:48:46,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a9a575bc060ce161feb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:46,095 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a9a575bc060ce161feb 2025-12-17 23:48:46,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9a575bc060ce161feb HTTP/1.1" 502 150 2025-12-17 23:48:46,096 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a9a575bc060ce161feb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:46,283 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a90575bc060ce161fdc 2025-12-17 23:48:46,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a90575bc060ce161fdc HTTP/1.1" 502 150 2025-12-17 23:48:46,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a90575bc060ce161fdc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:46,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a78575bc060ce161fae 2025-12-17 23:48:46,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a78575bc060ce161fae HTTP/1.1" 502 150 2025-12-17 23:48:46,514 - lava_callback - ERROR - Error processing callback for node 69409a78575bc060ce161fae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a78575bc060ce161fae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a78575bc060ce161fae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a78575bc060ce161fae (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:46,517 - lava_callback - INFO - Starting async job submit for node 69409a98575bc060ce161fe8 2025-12-17 23:48:46,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:46,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a98575bc060ce161fe8 HTTP/1.1" 502 150 2025-12-17 23:48:46,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a98575bc060ce161fe8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:46,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a98575bc060ce161fe8 2025-12-17 23:48:46,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a98575bc060ce161fe8 HTTP/1.1" 502 150 2025-12-17 23:48:46,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a98575bc060ce161fe8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41704 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:48:47,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a7a575bc060ce161fb7 2025-12-17 23:48:47,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7a575bc060ce161fb7 HTTP/1.1" 502 150 2025-12-17 23:48:47,419 - lava_callback - ERROR - Error processing callback for node 69409a7a575bc060ce161fb7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a7a575bc060ce161fb7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a7a575bc060ce161fb7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a7a575bc060ce161fb7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:47,421 - lava_callback - INFO - Starting async job submit for node 69409a9d575bc060ce161fee 2025-12-17 23:48:47,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:47,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9d575bc060ce161fee HTTP/1.1" 502 150 2025-12-17 23:48:47,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a9d575bc060ce161fee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:47,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a9d575bc060ce161fee 2025-12-17 23:48:47,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9d575bc060ce161fee HTTP/1.1" 502 150 2025-12-17 23:48:47,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a9d575bc060ce161fee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:47,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a7c575bc060ce161fbc 2025-12-17 23:48:47,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7c575bc060ce161fbc HTTP/1.1" 502 150 2025-12-17 23:48:47,555 - lava_callback - ERROR - Error processing callback for node 69409a7c575bc060ce161fbc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a7c575bc060ce161fbc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a7c575bc060ce161fbc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a7c575bc060ce161fbc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:47,557 - lava_callback - INFO - Starting async job submit for node 69409aa1575bc060ce162000 2025-12-17 23:48:47,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:47,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa1575bc060ce162000 HTTP/1.1" 502 150 2025-12-17 23:48:47,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa1575bc060ce162000'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:47,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa1575bc060ce162000 2025-12-17 23:48:47,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa1575bc060ce162000 HTTP/1.1" 502 150 2025-12-17 23:48:47,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa1575bc060ce162000'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:47,658 - lava_callback - INFO - Received callback for node 6940a7181e3c24e956819e2c INFO: 172.21.0.1:41714 - "POST /node/6940a7181e3c24e956819e2c HTTP/1.0" 202 Accepted 2025-12-17 23:48:47,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a92575bc060ce161fdf 2025-12-17 23:48:47,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a92575bc060ce161fdf HTTP/1.1" 502 150 2025-12-17 23:48:47,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a92575bc060ce161fdf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:48,096 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a9a575bc060ce161feb 2025-12-17 23:48:48,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9a575bc060ce161feb HTTP/1.1" 502 150 2025-12-17 23:48:48,097 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a9a575bc060ce161feb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:48,463 - lava_callback - INFO - Received callback for node 69409e4b575bc060ce16294e INFO: 172.21.0.1:41730 - "POST /node/69409e4b575bc060ce16294e HTTP/1.0" 202 Accepted 2025-12-17 23:48:48,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a98575bc060ce161fe8 2025-12-17 23:48:48,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a98575bc060ce161fe8 HTTP/1.1" 502 150 2025-12-17 23:48:48,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a98575bc060ce161fe8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:49,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a9d575bc060ce161fee 2025-12-17 23:48:49,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9d575bc060ce161fee HTTP/1.1" 502 150 2025-12-17 23:48:49,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a9d575bc060ce161fee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:49,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a96575bc060ce161fe5 2025-12-17 23:48:49,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a96575bc060ce161fe5 HTTP/1.1" 502 150 2025-12-17 23:48:49,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a96575bc060ce161fe5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:49,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa1575bc060ce162000 2025-12-17 23:48:49,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa1575bc060ce162000 HTTP/1.1" 502 150 2025-12-17 23:48:49,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa1575bc060ce162000'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:49,664 - lava_callback - INFO - Received callback for node 6940ae391e3c24e95681aac4 INFO: 172.21.0.1:41744 - "POST /node/6940ae391e3c24e95681aac4 HTTP/1.0" 202 Accepted 2025-12-17 23:48:50,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a7e575bc060ce161fbf 2025-12-17 23:48:50,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a84575bc060ce161fc2 2025-12-17 23:48:50,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a87575bc060ce161fc8 2025-12-17 23:48:50,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a84575bc060ce161fc2 HTTP/1.1" 502 150 2025-12-17 23:48:50,419 - lava_callback - ERROR - Error processing callback for node 69409a84575bc060ce161fc2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a84575bc060ce161fc2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a84575bc060ce161fc2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a84575bc060ce161fc2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:50,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a7e575bc060ce161fbf HTTP/1.1" 502 150 2025-12-17 23:48:50,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a87575bc060ce161fc8 HTTP/1.1" 502 150 2025-12-17 23:48:50,422 - lava_callback - ERROR - Error processing callback for node 69409a7e575bc060ce161fbf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a7e575bc060ce161fbf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a7e575bc060ce161fbf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a7e575bc060ce161fbf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:50,422 - lava_callback - INFO - Starting async job submit for node 69409aa3575bc060ce162005 2025-12-17 23:48:50,423 - lava_callback - ERROR - Error processing callback for node 69409a87575bc060ce161fc8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a87575bc060ce161fc8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a87575bc060ce161fc8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a87575bc060ce161fc8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:50,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:50,425 - lava_callback - INFO - Starting async job submit for node 69409a9f575bc060ce161ff6 2025-12-17 23:48:50,427 - lava_callback - INFO - Starting async job submit for node 69409aa7575bc060ce16200e 2025-12-17 23:48:50,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:50,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:50,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa3575bc060ce162005 HTTP/1.1" 502 150 2025-12-17 23:48:50,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa3575bc060ce162005'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:50,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa3575bc060ce162005 2025-12-17 23:48:50,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9f575bc060ce161ff6 HTTP/1.1" 502 150 2025-12-17 23:48:50,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a9f575bc060ce161ff6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:50,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a9f575bc060ce161ff6 2025-12-17 23:48:50,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa3575bc060ce162005 HTTP/1.1" 502 150 2025-12-17 23:48:50,436 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa3575bc060ce162005'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:50,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9f575bc060ce161ff6 HTTP/1.1" 502 150 2025-12-17 23:48:50,437 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a9f575bc060ce161ff6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:50,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a8b575bc060ce161fd4 2025-12-17 23:48:50,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a8b575bc060ce161fd4 HTTP/1.1" 502 150 2025-12-17 23:48:50,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a8b575bc060ce161fd4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:50,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a89575bc060ce161fcb 2025-12-17 23:48:50,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a89575bc060ce161fcb HTTP/1.1" 502 150 2025-12-17 23:48:50,869 - lava_callback - ERROR - Error processing callback for node 69409a89575bc060ce161fcb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a89575bc060ce161fcb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a89575bc060ce161fcb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a89575bc060ce161fcb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:50,871 - lava_callback - INFO - Starting async job submit for node 69409aa6575bc060ce16200b 2025-12-17 23:48:50,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:50,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa6575bc060ce16200b HTTP/1.1" 502 150 2025-12-17 23:48:50,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa6575bc060ce16200b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:50,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa6575bc060ce16200b 2025-12-17 23:48:50,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa6575bc060ce16200b HTTP/1.1" 502 150 2025-12-17 23:48:50,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa6575bc060ce16200b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:52,097 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a9a575bc060ce161feb 2025-12-17 23:48:52,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9a575bc060ce161feb HTTP/1.1" 502 150 2025-12-17 23:48:52,099 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a9a575bc060ce161feb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:52,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa3575bc060ce162005 2025-12-17 23:48:52,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a9f575bc060ce161ff6 2025-12-17 23:48:52,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa3575bc060ce162005 HTTP/1.1" 502 150 2025-12-17 23:48:52,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa3575bc060ce162005'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:52,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9f575bc060ce161ff6 HTTP/1.1" 502 150 2025-12-17 23:48:52,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a9f575bc060ce161ff6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:52,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a98575bc060ce161fe8 2025-12-17 23:48:52,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a98575bc060ce161fe8 HTTP/1.1" 502 150 2025-12-17 23:48:52,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a98575bc060ce161fe8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:52,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa6575bc060ce16200b 2025-12-17 23:48:52,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa6575bc060ce16200b HTTP/1.1" 502 150 2025-12-17 23:48:52,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa6575bc060ce16200b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:53,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a9d575bc060ce161fee 2025-12-17 23:48:53,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9d575bc060ce161fee HTTP/1.1" 502 150 2025-12-17 23:48:53,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a9d575bc060ce161fee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:53,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa1575bc060ce162000 2025-12-17 23:48:53,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa1575bc060ce162000 HTTP/1.1" 502 150 2025-12-17 23:48:53,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa1575bc060ce162000'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:54,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a74575bc060ce161f9d 2025-12-17 23:48:54,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a72575bc060ce161f9a 2025-12-17 23:48:54,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a74575bc060ce161f9d HTTP/1.1" 502 150 2025-12-17 23:48:54,771 - lava_callback - ERROR - Error processing callback for node 69409a74575bc060ce161f9d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a74575bc060ce161f9d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a74575bc060ce161f9d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a74575bc060ce161f9d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:54,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a72575bc060ce161f9a HTTP/1.1" 502 150 2025-12-17 23:48:54,773 - lava_callback - ERROR - Error processing callback for node 69409a72575bc060ce161f9a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a72575bc060ce161f9a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a72575bc060ce161f9a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a72575bc060ce161f9a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:54,773 - lava_callback - INFO - Starting async job submit for node 69409aac575bc060ce162017 2025-12-17 23:48:54,776 - lava_callback - INFO - Starting async job submit for node 69409ab0575bc060ce16201d 2025-12-17 23:48:54,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:54,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:54,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aac575bc060ce162017 HTTP/1.1" 502 150 2025-12-17 23:48:54,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aac575bc060ce162017'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:54,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aac575bc060ce162017 2025-12-17 23:48:54,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aac575bc060ce162017 HTTP/1.1" 502 150 2025-12-17 23:48:54,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aac575bc060ce162017'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:54,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab0575bc060ce16201d HTTP/1.1" 502 150 2025-12-17 23:48:54,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab0575bc060ce16201d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:54,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab0575bc060ce16201d 2025-12-17 23:48:54,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab0575bc060ce16201d HTTP/1.1" 502 150 2025-12-17 23:48:54,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab0575bc060ce16201d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:56,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa3575bc060ce162005 2025-12-17 23:48:56,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a9f575bc060ce161ff6 2025-12-17 23:48:56,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa3575bc060ce162005 HTTP/1.1" 502 150 2025-12-17 23:48:56,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa3575bc060ce162005'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:56,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9f575bc060ce161ff6 HTTP/1.1" 502 150 2025-12-17 23:48:56,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a9f575bc060ce161ff6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50742 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:48:56,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a94575bc060ce161fe2 HTTP/1.1" 502 150 2025-12-17 23:48:56,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a94575bc060ce161fe2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:56,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aac575bc060ce162017 2025-12-17 23:48:56,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aac575bc060ce162017 HTTP/1.1" 502 150 2025-12-17 23:48:56,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aac575bc060ce162017'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:56,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab0575bc060ce16201d 2025-12-17 23:48:56,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab0575bc060ce16201d HTTP/1.1" 502 150 2025-12-17 23:48:56,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab0575bc060ce16201d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:56,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa6575bc060ce16200b 2025-12-17 23:48:56,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa6575bc060ce16200b HTTP/1.1" 502 150 2025-12-17 23:48:56,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa6575bc060ce16200b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:57,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a96575bc060ce161fe5 2025-12-17 23:48:57,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a96575bc060ce161fe5 HTTP/1.1" 502 150 2025-12-17 23:48:57,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a96575bc060ce161fe5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:57,714 - lava_callback - INFO - Received callback for node 69409e62575bc060ce16296a INFO: 172.21.0.1:50752 - "POST /node/69409e62575bc060ce16296a HTTP/1.0" 202 Accepted 2025-12-17 23:48:58,164 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a8e575bc060ce161fd9 2025-12-17 23:48:58,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a8e575bc060ce161fd9 HTTP/1.1" 502 150 2025-12-17 23:48:58,165 - lava_callback - ERROR - Error processing callback for node 69409a8e575bc060ce161fd9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a8e575bc060ce161fd9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a8e575bc060ce161fd9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a8e575bc060ce161fd9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:48:58,167 - lava_callback - INFO - Starting async job submit for node 69409aa5575bc060ce162008 2025-12-17 23:48:58,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:48:58,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa5575bc060ce162008 HTTP/1.1" 502 150 2025-12-17 23:48:58,173 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa5575bc060ce162008'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:58,174 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa5575bc060ce162008 2025-12-17 23:48:58,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa5575bc060ce162008 HTTP/1.1" 502 150 2025-12-17 23:48:58,174 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa5575bc060ce162008'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:48:58,361 - lava_callback - INFO - Received callback for node 69409e67575bc060ce162979 INFO: 172.21.0.1:50766 - "POST /node/69409e67575bc060ce162979 HTTP/1.0" 202 Accepted 2025-12-17 23:49:00,099 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a9a575bc060ce161feb 2025-12-17 23:49:00,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9a575bc060ce161feb HTTP/1.1" 502 150 2025-12-17 23:49:00,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a9a575bc060ce161feb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:00,175 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa5575bc060ce162008 2025-12-17 23:49:00,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa5575bc060ce162008 HTTP/1.1" 502 150 2025-12-17 23:49:00,176 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa5575bc060ce162008'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:00,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a98575bc060ce161fe8 2025-12-17 23:49:00,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a98575bc060ce161fe8 HTTP/1.1" 502 150 2025-12-17 23:49:00,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a98575bc060ce161fe8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:00,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aac575bc060ce162017 2025-12-17 23:49:00,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aac575bc060ce162017 HTTP/1.1" 502 150 2025-12-17 23:49:00,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aac575bc060ce162017'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:00,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab0575bc060ce16201d 2025-12-17 23:49:00,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab0575bc060ce16201d HTTP/1.1" 502 150 2025-12-17 23:49:00,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab0575bc060ce16201d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:01,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a9d575bc060ce161fee 2025-12-17 23:49:01,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9d575bc060ce161fee HTTP/1.1" 502 150 2025-12-17 23:49:01,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a9d575bc060ce161fee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:01,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa1575bc060ce162000 2025-12-17 23:49:01,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa1575bc060ce162000 HTTP/1.1" 502 150 2025-12-17 23:49:01,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa1575bc060ce162000'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:02,284 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a90575bc060ce161fdc 2025-12-17 23:49:02,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a90575bc060ce161fdc HTTP/1.1" 502 150 2025-12-17 23:49:02,286 - lava_callback - ERROR - Error processing callback for node 69409a90575bc060ce161fdc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a90575bc060ce161fdc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a90575bc060ce161fdc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a90575bc060ce161fdc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:02,288 - lava_callback - INFO - Starting async job submit for node 69409aaa575bc060ce162014 2025-12-17 23:49:02,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:02,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aaa575bc060ce162014 HTTP/1.1" 502 150 2025-12-17 23:49:02,294 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aaa575bc060ce162014'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:02,294 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aaa575bc060ce162014 2025-12-17 23:49:02,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aaa575bc060ce162014 HTTP/1.1" 502 150 2025-12-17 23:49:02,295 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aaa575bc060ce162014'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42500 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:49:03,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a92575bc060ce161fdf 2025-12-17 23:49:03,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a92575bc060ce161fdf HTTP/1.1" 502 150 2025-12-17 23:49:03,870 - lava_callback - ERROR - Error processing callback for node 69409a92575bc060ce161fdf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a92575bc060ce161fdf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a92575bc060ce161fdf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a92575bc060ce161fdf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:03,871 - lava_callback - INFO - Starting async job submit for node 69409aae575bc060ce16201a 2025-12-17 23:49:03,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:03,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aae575bc060ce16201a HTTP/1.1" 502 150 2025-12-17 23:49:03,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aae575bc060ce16201a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:03,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aae575bc060ce16201a 2025-12-17 23:49:03,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aae575bc060ce16201a HTTP/1.1" 502 150 2025-12-17 23:49:03,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aae575bc060ce16201a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:04,176 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa5575bc060ce162008 2025-12-17 23:49:04,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa5575bc060ce162008 HTTP/1.1" 502 150 2025-12-17 23:49:04,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa5575bc060ce162008'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:04,295 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aaa575bc060ce162014 2025-12-17 23:49:04,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aaa575bc060ce162014 HTTP/1.1" 502 150 2025-12-17 23:49:04,296 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aaa575bc060ce162014'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:04,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa3575bc060ce162005 2025-12-17 23:49:04,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a9f575bc060ce161ff6 2025-12-17 23:49:04,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9f575bc060ce161ff6 HTTP/1.1" 502 150 2025-12-17 23:49:04,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a9f575bc060ce161ff6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:04,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa3575bc060ce162005 HTTP/1.1" 502 150 2025-12-17 23:49:04,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa3575bc060ce162005'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:04,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a94575bc060ce161fe2 2025-12-17 23:49:04,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a94575bc060ce161fe2 HTTP/1.1" 502 150 2025-12-17 23:49:04,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409a94575bc060ce161fe2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:04,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa6575bc060ce16200b 2025-12-17 23:49:04,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa6575bc060ce16200b HTTP/1.1" 502 150 2025-12-17 23:49:04,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa6575bc060ce16200b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:05,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aae575bc060ce16201a 2025-12-17 23:49:05,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aae575bc060ce16201a HTTP/1.1" 502 150 2025-12-17 23:49:05,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aae575bc060ce16201a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42512 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:49:06,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a8b575bc060ce161fd4 2025-12-17 23:49:06,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a8b575bc060ce161fd4 HTTP/1.1" 502 150 2025-12-17 23:49:06,547 - lava_callback - ERROR - Error processing callback for node 69409a8b575bc060ce161fd4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a8b575bc060ce161fd4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a8b575bc060ce161fd4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a8b575bc060ce161fd4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:06,549 - lava_callback - INFO - Starting async job submit for node 69409ab2575bc060ce162022 2025-12-17 23:49:06,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:06,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab2575bc060ce162022 HTTP/1.1" 502 150 2025-12-17 23:49:06,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab2575bc060ce162022'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:06,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab2575bc060ce162022 2025-12-17 23:49:06,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab2575bc060ce162022 HTTP/1.1" 502 150 2025-12-17 23:49:06,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab2575bc060ce162022'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:08,297 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aaa575bc060ce162014 2025-12-17 23:49:08,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aaa575bc060ce162014 HTTP/1.1" 502 150 2025-12-17 23:49:08,303 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aaa575bc060ce162014'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:08,442 - lava_callback - INFO - Received callback for node 69409e5f575bc060ce16295f INFO: 172.21.0.1:42526 - "POST /node/69409e5f575bc060ce16295f HTTP/1.0" 202 Accepted 2025-12-17 23:49:08,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab2575bc060ce162022 2025-12-17 23:49:08,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab2575bc060ce162022 HTTP/1.1" 502 150 2025-12-17 23:49:08,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab2575bc060ce162022'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:08,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aac575bc060ce162017 2025-12-17 23:49:08,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aac575bc060ce162017 HTTP/1.1" 502 150 2025-12-17 23:49:08,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aac575bc060ce162017'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:08,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab0575bc060ce16201d 2025-12-17 23:49:08,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab0575bc060ce16201d HTTP/1.1" 502 150 2025-12-17 23:49:08,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab0575bc060ce16201d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:09,100 - lava_callback - INFO - Received callback for node 69409e65575bc060ce162971 INFO: 172.21.0.1:42540 - "POST /node/69409e65575bc060ce162971 HTTP/1.0" 202 Accepted 2025-12-17 23:49:09,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aae575bc060ce16201a 2025-12-17 23:49:09,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aae575bc060ce16201a HTTP/1.1" 502 150 2025-12-17 23:49:09,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aae575bc060ce16201a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:12,178 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa5575bc060ce162008 2025-12-17 23:49:12,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa5575bc060ce162008 HTTP/1.1" 502 150 2025-12-17 23:49:12,179 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa5575bc060ce162008'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:12,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab2575bc060ce162022 2025-12-17 23:49:12,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab2575bc060ce162022 HTTP/1.1" 502 150 2025-12-17 23:49:12,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab2575bc060ce162022'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:13,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a96575bc060ce161fe5 2025-12-17 23:49:13,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a96575bc060ce161fe5 HTTP/1.1" 502 150 2025-12-17 23:49:13,499 - lava_callback - ERROR - Error processing callback for node 69409a96575bc060ce161fe5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a96575bc060ce161fe5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a96575bc060ce161fe5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a96575bc060ce161fe5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:13,501 - lava_callback - INFO - Starting async job submit for node 69409ab6575bc060ce162032 2025-12-17 23:49:13,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:13,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab6575bc060ce162032 HTTP/1.1" 502 150 2025-12-17 23:49:13,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab6575bc060ce162032'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:13,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab6575bc060ce162032 2025-12-17 23:49:13,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab6575bc060ce162032 HTTP/1.1" 502 150 2025-12-17 23:49:13,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab6575bc060ce162032'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49798 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:49:15,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab6575bc060ce162032 2025-12-17 23:49:15,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab6575bc060ce162032 HTTP/1.1" 502 150 2025-12-17 23:49:15,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab6575bc060ce162032'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:16,101 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a9a575bc060ce161feb 2025-12-17 23:49:16,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9a575bc060ce161feb HTTP/1.1" 502 150 2025-12-17 23:49:16,104 - lava_callback - ERROR - Error processing callback for node 69409a9a575bc060ce161feb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a9a575bc060ce161feb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a9a575bc060ce161feb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a9a575bc060ce161feb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:16,106 - lava_callback - INFO - Starting async job submit for node 6940a105575bc060ce162dee 2025-12-17 23:49:16,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:16,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a105575bc060ce162dee HTTP/1.1" 502 150 2025-12-17 23:49:16,197 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a105575bc060ce162dee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:16,197 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a105575bc060ce162dee 2025-12-17 23:49:16,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a105575bc060ce162dee HTTP/1.1" 502 150 2025-12-17 23:49:16,198 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a105575bc060ce162dee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:16,303 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aaa575bc060ce162014 2025-12-17 23:49:16,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aaa575bc060ce162014 HTTP/1.1" 502 150 2025-12-17 23:49:16,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aaa575bc060ce162014'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49800 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:49:16,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a98575bc060ce161fe8 2025-12-17 23:49:16,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a98575bc060ce161fe8 HTTP/1.1" 502 150 2025-12-17 23:49:16,532 - lava_callback - ERROR - Error processing callback for node 69409a98575bc060ce161fe8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a98575bc060ce161fe8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a98575bc060ce161fe8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a98575bc060ce161fe8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:16,534 - lava_callback - INFO - Starting async job submit for node 69409ab8575bc060ce162035 2025-12-17 23:49:16,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:16,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab8575bc060ce162035 HTTP/1.1" 502 150 2025-12-17 23:49:16,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab8575bc060ce162035'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:16,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab8575bc060ce162035 2025-12-17 23:49:16,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab8575bc060ce162035 HTTP/1.1" 502 150 2025-12-17 23:49:16,566 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab8575bc060ce162035'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:17,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a9d575bc060ce161fee 2025-12-17 23:49:17,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9d575bc060ce161fee HTTP/1.1" 502 150 2025-12-17 23:49:17,458 - lava_callback - ERROR - Error processing callback for node 69409a9d575bc060ce161fee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a9d575bc060ce161fee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a9d575bc060ce161fee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a9d575bc060ce161fee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:17,460 - lava_callback - INFO - Starting async job submit for node 69409aba575bc060ce162038 2025-12-17 23:49:17,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:17,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aba575bc060ce162038 HTTP/1.1" 502 150 2025-12-17 23:49:17,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aba575bc060ce162038'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:17,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aba575bc060ce162038 2025-12-17 23:49:17,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aba575bc060ce162038 HTTP/1.1" 502 150 2025-12-17 23:49:17,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aba575bc060ce162038'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:17,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa1575bc060ce162000 2025-12-17 23:49:17,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa1575bc060ce162000 HTTP/1.1" 502 150 2025-12-17 23:49:17,596 - lava_callback - ERROR - Error processing callback for node 69409aa1575bc060ce162000: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aa1575bc060ce162000 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aa1575bc060ce162000 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aa1575bc060ce162000 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:17,598 - lava_callback - INFO - Starting async job submit for node 6942853c1b61f1c6005fab91 2025-12-17 23:49:17,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:17,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942853c1b61f1c6005fab91 HTTP/1.1" 502 150 2025-12-17 23:49:17,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942853c1b61f1c6005fab91'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:17,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942853c1b61f1c6005fab91 2025-12-17 23:49:17,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942853c1b61f1c6005fab91 HTTP/1.1" 502 150 2025-12-17 23:49:17,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942853c1b61f1c6005fab91'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:17,716 - lava_callback - INFO - Received callback for node 69409e69575bc060ce16297c INFO: 172.21.0.1:49804 - "POST /node/69409e69575bc060ce16297c HTTP/1.0" 202 Accepted 2025-12-17 23:49:17,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aae575bc060ce16201a 2025-12-17 23:49:17,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aae575bc060ce16201a HTTP/1.1" 502 150 2025-12-17 23:49:17,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aae575bc060ce16201a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:18,198 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a105575bc060ce162dee 2025-12-17 23:49:18,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a105575bc060ce162dee HTTP/1.1" 502 150 2025-12-17 23:49:18,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a105575bc060ce162dee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:18,376 - lava_callback - INFO - Received callback for node 69409e79575bc060ce162997 INFO: 172.21.0.1:49806 - "POST /node/69409e79575bc060ce162997 HTTP/1.0" 202 Accepted 2025-12-17 23:49:18,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab8575bc060ce162035 2025-12-17 23:49:18,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab8575bc060ce162035 HTTP/1.1" 502 150 2025-12-17 23:49:18,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab8575bc060ce162035'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:19,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aba575bc060ce162038 2025-12-17 23:49:19,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aba575bc060ce162038 HTTP/1.1" 502 150 2025-12-17 23:49:19,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aba575bc060ce162038'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:19,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab6575bc060ce162032 2025-12-17 23:49:19,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab6575bc060ce162032 HTTP/1.1" 502 150 2025-12-17 23:49:19,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab6575bc060ce162032'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:19,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942853c1b61f1c6005fab91 2025-12-17 23:49:19,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942853c1b61f1c6005fab91 HTTP/1.1" 502 150 2025-12-17 23:49:19,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942853c1b61f1c6005fab91'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:20,442 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a9f575bc060ce161ff6 2025-12-17 23:49:20,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa3575bc060ce162005 2025-12-17 23:49:20,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a9f575bc060ce161ff6 HTTP/1.1" 502 150 2025-12-17 23:49:20,444 - lava_callback - ERROR - Error processing callback for node 69409a9f575bc060ce161ff6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a9f575bc060ce161ff6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a9f575bc060ce161ff6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a9f575bc060ce161ff6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:20,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa3575bc060ce162005 HTTP/1.1" 502 150 2025-12-17 23:49:20,446 - lava_callback - ERROR - Error processing callback for node 69409aa3575bc060ce162005: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aa3575bc060ce162005 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aa3575bc060ce162005 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aa3575bc060ce162005 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:20,446 - lava_callback - INFO - Starting async job submit for node 69409ab4575bc060ce16202b 2025-12-17 23:49:20,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:20,449 - lava_callback - INFO - Starting async job submit for node 69409dba575bc060ce1628a8 2025-12-17 23:49:20,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:20,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab4575bc060ce16202b HTTP/1.1" 502 150 2025-12-17 23:49:20,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab4575bc060ce16202b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:20,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab4575bc060ce16202b 2025-12-17 23:49:20,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab4575bc060ce16202b HTTP/1.1" 502 150 2025-12-17 23:49:20,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab4575bc060ce16202b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:20,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dba575bc060ce1628a8 HTTP/1.1" 502 150 2025-12-17 23:49:20,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dba575bc060ce1628a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:20,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dba575bc060ce1628a8 2025-12-17 23:49:20,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dba575bc060ce1628a8 HTTP/1.1" 502 150 2025-12-17 23:49:20,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dba575bc060ce1628a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:20,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab2575bc060ce162022 2025-12-17 23:49:20,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab2575bc060ce162022 HTTP/1.1" 502 150 2025-12-17 23:49:20,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab2575bc060ce162022'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:20,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409a94575bc060ce161fe2 2025-12-17 23:49:20,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409a94575bc060ce161fe2 HTTP/1.1" 502 150 2025-12-17 23:49:20,628 - lava_callback - ERROR - Error processing callback for node 69409a94575bc060ce161fe2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a94575bc060ce161fe2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a94575bc060ce161fe2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409a94575bc060ce161fe2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:20,630 - lava_callback - INFO - Starting async job submit for node 69409abc575bc060ce16203b 2025-12-17 23:49:20,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:20,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409abc575bc060ce16203b HTTP/1.1" 502 150 2025-12-17 23:49:20,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409abc575bc060ce16203b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:20,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409abc575bc060ce16203b 2025-12-17 23:49:20,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409abc575bc060ce16203b HTTP/1.1" 502 150 2025-12-17 23:49:20,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409abc575bc060ce16203b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:20,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa6575bc060ce16200b 2025-12-17 23:49:20,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa6575bc060ce16200b HTTP/1.1" 502 150 2025-12-17 23:49:20,894 - lava_callback - ERROR - Error processing callback for node 69409aa6575bc060ce16200b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aa6575bc060ce16200b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aa6575bc060ce16200b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aa6575bc060ce16200b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:20,896 - lava_callback - INFO - Starting async job submit for node 69409abe575bc060ce162041 2025-12-17 23:49:20,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:20,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409abe575bc060ce162041 HTTP/1.1" 502 150 2025-12-17 23:49:20,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409abe575bc060ce162041'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:20,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409abe575bc060ce162041 2025-12-17 23:49:20,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409abe575bc060ce162041 HTTP/1.1" 502 150 2025-12-17 23:49:20,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409abe575bc060ce162041'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:21,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa7575bc060ce16200e HTTP/1.1" 502 150 2025-12-17 23:49:21,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa7575bc060ce16200e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:21,201 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa7575bc060ce16200e 2025-12-17 23:49:22,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a105575bc060ce162dee 2025-12-17 23:49:22,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab4575bc060ce16202b 2025-12-17 23:49:22,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab4575bc060ce16202b HTTP/1.1" 502 150 2025-12-17 23:49:22,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab4575bc060ce16202b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:22,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dba575bc060ce1628a8 2025-12-17 23:49:22,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dba575bc060ce1628a8 HTTP/1.1" 502 150 2025-12-17 23:49:22,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dba575bc060ce1628a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:22,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab8575bc060ce162035 2025-12-17 23:49:22,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab8575bc060ce162035 HTTP/1.1" 502 150 2025-12-17 23:49:22,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab8575bc060ce162035'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:22,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409abc575bc060ce16203b 2025-12-17 23:49:22,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409abc575bc060ce16203b HTTP/1.1" 502 150 2025-12-17 23:49:22,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409abc575bc060ce16203b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:22,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409abe575bc060ce162041 2025-12-17 23:49:22,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409abe575bc060ce162041 HTTP/1.1" 502 150 2025-12-17 23:49:22,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409abe575bc060ce162041'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:23,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aba575bc060ce162038 2025-12-17 23:49:23,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aba575bc060ce162038 HTTP/1.1" 502 150 2025-12-17 23:49:23,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aba575bc060ce162038'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:23,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942853c1b61f1c6005fab91 2025-12-17 23:49:23,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942853c1b61f1c6005fab91 HTTP/1.1" 502 150 2025-12-17 23:49:23,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942853c1b61f1c6005fab91'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:24,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aac575bc060ce162017 2025-12-17 23:49:24,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aac575bc060ce162017 HTTP/1.1" 502 150 2025-12-17 23:49:24,790 - lava_callback - ERROR - Error processing callback for node 69409aac575bc060ce162017: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aac575bc060ce162017 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aac575bc060ce162017 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aac575bc060ce162017 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:24,792 - lava_callback - INFO - Starting async job submit for node 69409ac0575bc060ce162044 2025-12-17 23:49:24,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab0575bc060ce16201d 2025-12-17 23:49:24,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:24,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab0575bc060ce16201d HTTP/1.1" 502 150 2025-12-17 23:49:24,795 - lava_callback - ERROR - Error processing callback for node 69409ab0575bc060ce16201d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ab0575bc060ce16201d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ab0575bc060ce16201d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ab0575bc060ce16201d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:24,797 - lava_callback - INFO - Starting async job submit for node 69409ac2575bc060ce162047 2025-12-17 23:49:24,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:24,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac0575bc060ce162044 HTTP/1.1" 502 150 2025-12-17 23:49:24,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac0575bc060ce162044'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:24,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac0575bc060ce162044 2025-12-17 23:49:24,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac0575bc060ce162044 HTTP/1.1" 502 150 2025-12-17 23:49:24,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac0575bc060ce162044'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:24,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac2575bc060ce162047 HTTP/1.1" 502 150 2025-12-17 23:49:24,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac2575bc060ce162047'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:24,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac2575bc060ce162047 2025-12-17 23:49:24,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac2575bc060ce162047 HTTP/1.1" 502 150 2025-12-17 23:49:24,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac2575bc060ce162047'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:26,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab4575bc060ce16202b 2025-12-17 23:49:26,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab4575bc060ce16202b HTTP/1.1" 502 150 2025-12-17 23:49:26,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab4575bc060ce16202b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:26,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dba575bc060ce1628a8 2025-12-17 23:49:26,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dba575bc060ce1628a8 HTTP/1.1" 502 150 2025-12-17 23:49:26,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dba575bc060ce1628a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33330 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:49:26,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409abc575bc060ce16203b 2025-12-17 23:49:26,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409abc575bc060ce16203b HTTP/1.1" 502 150 2025-12-17 23:49:26,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409abc575bc060ce16203b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:26,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac0575bc060ce162044 2025-12-17 23:49:26,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac0575bc060ce162044 HTTP/1.1" 502 150 2025-12-17 23:49:26,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac0575bc060ce162044'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:26,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac2575bc060ce162047 2025-12-17 23:49:26,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac2575bc060ce162047 HTTP/1.1" 502 150 2025-12-17 23:49:26,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac2575bc060ce162047'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:26,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409abe575bc060ce162041 2025-12-17 23:49:26,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409abe575bc060ce162041 HTTP/1.1" 502 150 2025-12-17 23:49:26,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409abe575bc060ce162041'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:27,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab6575bc060ce162032 2025-12-17 23:49:27,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab6575bc060ce162032 HTTP/1.1" 502 150 2025-12-17 23:49:27,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab6575bc060ce162032'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:28,179 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa5575bc060ce162008 2025-12-17 23:49:28,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa5575bc060ce162008 HTTP/1.1" 502 150 2025-12-17 23:49:28,181 - lava_callback - ERROR - Error processing callback for node 69409aa5575bc060ce162008: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aa5575bc060ce162008 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aa5575bc060ce162008 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aa5575bc060ce162008 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:28,183 - lava_callback - INFO - Starting async job submit for node 69409ac9575bc060ce16205d 2025-12-17 23:49:28,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:28,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac9575bc060ce16205d HTTP/1.1" 502 150 2025-12-17 23:49:28,191 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac9575bc060ce16205d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:28,191 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac9575bc060ce16205d 2025-12-17 23:49:28,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac9575bc060ce16205d HTTP/1.1" 502 150 2025-12-17 23:49:28,191 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac9575bc060ce16205d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:28,479 - lava_callback - INFO - Received callback for node 69409e7c575bc060ce16299a INFO: 172.21.0.1:33342 - "POST /node/69409e7c575bc060ce16299a HTTP/1.0" 202 Accepted 2025-12-17 23:49:29,145 - lava_callback - INFO - Received callback for node 69409e84575bc060ce1629a0 INFO: 172.21.0.1:33350 - "POST /node/69409e84575bc060ce1629a0 HTTP/1.0" 202 Accepted 2025-12-17 23:49:29,811 - lava_callback - INFO - Received callback for node 69409e86575bc060ce1629a7 INFO: 172.21.0.1:40612 - "POST /node/69409e86575bc060ce1629a7 HTTP/1.0" 202 Accepted 2025-12-17 23:49:30,192 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac9575bc060ce16205d 2025-12-17 23:49:30,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac9575bc060ce16205d HTTP/1.1" 502 150 2025-12-17 23:49:30,193 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac9575bc060ce16205d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:30,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab8575bc060ce162035 2025-12-17 23:49:30,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab8575bc060ce162035 HTTP/1.1" 502 150 2025-12-17 23:49:30,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab8575bc060ce162035'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:30,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac0575bc060ce162044 2025-12-17 23:49:30,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac0575bc060ce162044 HTTP/1.1" 502 150 2025-12-17 23:49:30,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac0575bc060ce162044'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:30,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac2575bc060ce162047 2025-12-17 23:49:30,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac2575bc060ce162047 HTTP/1.1" 502 150 2025-12-17 23:49:30,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac2575bc060ce162047'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:31,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aba575bc060ce162038 2025-12-17 23:49:31,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aba575bc060ce162038 HTTP/1.1" 502 150 2025-12-17 23:49:31,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aba575bc060ce162038'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:31,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942853c1b61f1c6005fab91 2025-12-17 23:49:31,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942853c1b61f1c6005fab91 HTTP/1.1" 502 150 2025-12-17 23:49:31,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942853c1b61f1c6005fab91'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:32,304 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aaa575bc060ce162014 2025-12-17 23:49:32,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aaa575bc060ce162014 HTTP/1.1" 502 150 2025-12-17 23:49:32,305 - lava_callback - ERROR - Error processing callback for node 69409aaa575bc060ce162014: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aaa575bc060ce162014 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aaa575bc060ce162014 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aaa575bc060ce162014 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:32,307 - lava_callback - INFO - Starting async job submit for node 69428f571b61f1c6005fbd14 2025-12-17 23:49:32,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:32,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69428f571b61f1c6005fbd14 HTTP/1.1" 502 150 2025-12-17 23:49:32,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69428f571b61f1c6005fbd14'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:32,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69428f571b61f1c6005fbd14 2025-12-17 23:49:32,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69428f571b61f1c6005fbd14 HTTP/1.1" 502 150 2025-12-17 23:49:32,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69428f571b61f1c6005fbd14'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:32,539 - lava_callback - INFO - Received callback for node 69409e8b575bc060ce1629bc INFO: 172.21.0.1:40628 - "POST /node/69409e8b575bc060ce1629bc HTTP/1.0" 202 Accepted 2025-12-17 23:49:33,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aae575bc060ce16201a 2025-12-17 23:49:34,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac9575bc060ce16205d 2025-12-17 23:49:34,317 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69428f571b61f1c6005fbd14 2025-12-17 23:49:34,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69428f571b61f1c6005fbd14 HTTP/1.1" 502 150 2025-12-17 23:49:34,318 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69428f571b61f1c6005fbd14'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:34,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab4575bc060ce16202b 2025-12-17 23:49:34,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab4575bc060ce16202b HTTP/1.1" 502 150 2025-12-17 23:49:34,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ab4575bc060ce16202b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:34,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dba575bc060ce1628a8 2025-12-17 23:49:34,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dba575bc060ce1628a8 HTTP/1.1" 502 150 2025-12-17 23:49:34,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dba575bc060ce1628a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:34,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409abc575bc060ce16203b 2025-12-17 23:49:34,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409abc575bc060ce16203b HTTP/1.1" 502 150 2025-12-17 23:49:34,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409abc575bc060ce16203b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:34,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409abe575bc060ce162041 2025-12-17 23:49:34,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409abe575bc060ce162041 HTTP/1.1" 502 150 2025-12-17 23:49:34,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409abe575bc060ce162041'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40642 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:49:36,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab2575bc060ce162022 2025-12-17 23:49:36,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab2575bc060ce162022 HTTP/1.1" 502 150 2025-12-17 23:49:36,564 - lava_callback - ERROR - Error processing callback for node 69409ab2575bc060ce162022: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ab2575bc060ce162022 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ab2575bc060ce162022 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ab2575bc060ce162022 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:36,566 - lava_callback - INFO - Starting async job submit for node 69409acd575bc060ce162064 2025-12-17 23:49:36,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:36,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acd575bc060ce162064 HTTP/1.1" 502 150 2025-12-17 23:49:36,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409acd575bc060ce162064'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:36,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409acd575bc060ce162064 2025-12-17 23:49:36,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acd575bc060ce162064 HTTP/1.1" 502 150 2025-12-17 23:49:36,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409acd575bc060ce162064'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:37,140 - lava_callback - INFO - Received callback for node 6940a5311e3c24e956819cc0 INFO: 172.21.0.1:40648 - "POST /node/6940a5311e3c24e956819cc0 HTTP/1.0" 202 Accepted 2025-12-17 23:49:37,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a105575bc060ce162dee HTTP/1.1" 502 150 2025-12-17 23:49:37,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a105575bc060ce162dee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:37,774 - lava_callback - INFO - Received callback for node 69409e82575bc060ce16299d INFO: 172.21.0.1:40660 - "POST /node/69409e82575bc060ce16299d HTTP/1.0" 202 Accepted 2025-12-17 23:49:38,318 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69428f571b61f1c6005fbd14 2025-12-17 23:49:38,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69428f571b61f1c6005fbd14 HTTP/1.1" 502 150 2025-12-17 23:49:38,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69428f571b61f1c6005fbd14'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:38,428 - lava_callback - INFO - Received callback for node 69409e8e575bc060ce1629c2 INFO: 172.21.0.1:40668 - "POST /node/69409e8e575bc060ce1629c2 HTTP/1.0" 202 Accepted 2025-12-17 23:49:38,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409acd575bc060ce162064 2025-12-17 23:49:38,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acd575bc060ce162064 HTTP/1.1" 502 150 2025-12-17 23:49:38,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409acd575bc060ce162064'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:38,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac0575bc060ce162044 2025-12-17 23:49:38,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac0575bc060ce162044 HTTP/1.1" 502 150 2025-12-17 23:49:38,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac0575bc060ce162044'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:38,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac2575bc060ce162047 2025-12-17 23:49:38,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac2575bc060ce162047 HTTP/1.1" 502 150 2025-12-17 23:49:38,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac2575bc060ce162047'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:42,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409acd575bc060ce162064 2025-12-17 23:49:42,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acd575bc060ce162064 HTTP/1.1" 502 150 2025-12-17 23:49:42,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409acd575bc060ce162064'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:43,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab6575bc060ce162032 2025-12-17 23:49:43,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab6575bc060ce162032 HTTP/1.1" 502 150 2025-12-17 23:49:43,515 - lava_callback - ERROR - Error processing callback for node 69409ab6575bc060ce162032: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ab6575bc060ce162032 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ab6575bc060ce162032 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ab6575bc060ce162032 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:43,517 - lava_callback - INFO - Starting async job submit for node 69409ac6575bc060ce16204f 2025-12-17 23:49:43,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:43,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac6575bc060ce16204f HTTP/1.1" 502 150 2025-12-17 23:49:43,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac6575bc060ce16204f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:43,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac6575bc060ce16204f 2025-12-17 23:49:43,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac6575bc060ce16204f HTTP/1.1" 502 150 2025-12-17 23:49:43,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac6575bc060ce16204f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:45,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac6575bc060ce16204f 2025-12-17 23:49:45,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a105575bc060ce162dee 2025-12-17 23:49:45,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a105575bc060ce162dee HTTP/1.1" 502 150 2025-12-17 23:49:45,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a105575bc060ce162dee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:46,319 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69428f571b61f1c6005fbd14 2025-12-17 23:49:46,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69428f571b61f1c6005fbd14 HTTP/1.1" 502 150 2025-12-17 23:49:46,321 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69428f571b61f1c6005fbd14'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44760 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:49:46,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab8575bc060ce162035 2025-12-17 23:49:46,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab8575bc060ce162035 HTTP/1.1" 502 150 2025-12-17 23:49:46,571 - lava_callback - ERROR - Error processing callback for node 69409ab8575bc060ce162035: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ab8575bc060ce162035 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ab8575bc060ce162035 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ab8575bc060ce162035 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:46,573 - lava_callback - INFO - Starting async job submit for node 69409acb575bc060ce162061 2025-12-17 23:49:46,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:46,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acb575bc060ce162061 HTTP/1.1" 502 150 2025-12-17 23:49:46,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409acb575bc060ce162061'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:46,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409acb575bc060ce162061 2025-12-17 23:49:46,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acb575bc060ce162061 HTTP/1.1" 502 150 2025-12-17 23:49:46,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409acb575bc060ce162061'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:47,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aba575bc060ce162038 2025-12-17 23:49:47,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aba575bc060ce162038 HTTP/1.1" 502 150 2025-12-17 23:49:47,474 - lava_callback - ERROR - Error processing callback for node 69409aba575bc060ce162038: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aba575bc060ce162038 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aba575bc060ce162038 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aba575bc060ce162038 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:47,476 - lava_callback - INFO - Starting async job submit for node 69409ad4575bc060ce16206d 2025-12-17 23:49:47,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:47,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad4575bc060ce16206d HTTP/1.1" 502 150 2025-12-17 23:49:47,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad4575bc060ce16206d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:47,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad4575bc060ce16206d 2025-12-17 23:49:47,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad4575bc060ce16206d HTTP/1.1" 502 150 2025-12-17 23:49:47,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad4575bc060ce16206d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:47,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942853c1b61f1c6005fab91 2025-12-17 23:49:47,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942853c1b61f1c6005fab91 HTTP/1.1" 502 150 2025-12-17 23:49:47,619 - lava_callback - ERROR - Error processing callback for node 6942853c1b61f1c6005fab91: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942853c1b61f1c6005fab91 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942853c1b61f1c6005fab91 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942853c1b61f1c6005fab91 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:47,621 - lava_callback - INFO - Starting async job submit for node 69409acf575bc060ce162067 2025-12-17 23:49:47,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:47,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acf575bc060ce162067 HTTP/1.1" 502 150 2025-12-17 23:49:47,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409acf575bc060ce162067'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:47,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409acf575bc060ce162067 2025-12-17 23:49:47,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acf575bc060ce162067 HTTP/1.1" 502 150 2025-12-17 23:49:47,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409acf575bc060ce162067'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:48,502 - lava_callback - INFO - Received callback for node 69409e89575bc060ce1629b2 INFO: 172.21.0.1:44774 - "POST /node/69409e89575bc060ce1629b2 HTTP/1.0" 202 Accepted 2025-12-17 23:49:48,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409acb575bc060ce162061 2025-12-17 23:49:48,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acb575bc060ce162061 HTTP/1.1" 502 150 2025-12-17 23:49:48,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409acb575bc060ce162061'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:49,148 - lava_callback - INFO - Received callback for node 69409e8c575bc060ce1629bf INFO: 172.21.0.1:44788 - "POST /node/69409e8c575bc060ce1629bf HTTP/1.0" 202 Accepted 2025-12-17 23:49:49,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aae575bc060ce16201a HTTP/1.1" 502 150 2025-12-17 23:49:49,360 - lava_callback - ERROR - Error processing callback for node 69409aae575bc060ce16201a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aae575bc060ce16201a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aae575bc060ce16201a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aae575bc060ce16201a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:49,362 - lava_callback - INFO - Starting async job submit for node 69409ad1575bc060ce16206a 2025-12-17 23:49:49,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:49,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad1575bc060ce16206a HTTP/1.1" 502 150 2025-12-17 23:49:49,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad1575bc060ce16206a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:49,369 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad1575bc060ce16206a 2025-12-17 23:49:49,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad1575bc060ce16206a HTTP/1.1" 502 150 2025-12-17 23:49:49,370 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad1575bc060ce16206a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:49,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad4575bc060ce16206d 2025-12-17 23:49:49,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad4575bc060ce16206d HTTP/1.1" 502 150 2025-12-17 23:49:49,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad4575bc060ce16206d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:49,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac9575bc060ce16205d HTTP/1.1" 502 150 2025-12-17 23:49:49,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac9575bc060ce16205d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:49,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409acf575bc060ce162067 2025-12-17 23:49:49,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acf575bc060ce162067 HTTP/1.1" 502 150 2025-12-17 23:49:49,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409acf575bc060ce162067'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:49,812 - lava_callback - INFO - Received callback for node 69409e90575bc060ce1629c7 INFO: 172.21.0.1:37834 - "POST /node/69409e90575bc060ce1629c7 HTTP/1.0" 202 Accepted 2025-12-17 23:49:50,447 - lava_callback - INFO - Received callback for node 69409e92575bc060ce1629ca INFO: 172.21.0.1:37842 - "POST /node/69409e92575bc060ce1629ca HTTP/1.0" 202 Accepted 2025-12-17 23:49:50,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ab4575bc060ce16202b 2025-12-17 23:49:50,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ab4575bc060ce16202b HTTP/1.1" 502 150 2025-12-17 23:49:50,480 - lava_callback - ERROR - Error processing callback for node 69409ab4575bc060ce16202b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ab4575bc060ce16202b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ab4575bc060ce16202b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ab4575bc060ce16202b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:50,480 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dba575bc060ce1628a8 2025-12-17 23:49:50,482 - lava_callback - INFO - Starting async job submit for node 69409ad4575bc060ce162070 2025-12-17 23:49:50,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:50,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dba575bc060ce1628a8 HTTP/1.1" 502 150 2025-12-17 23:49:50,484 - lava_callback - ERROR - Error processing callback for node 69409dba575bc060ce1628a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dba575bc060ce1628a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dba575bc060ce1628a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dba575bc060ce1628a8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:50,486 - lava_callback - INFO - Starting async job submit for node 69409ad8575bc060ce162076 2025-12-17 23:49:50,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:50,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad4575bc060ce162070 HTTP/1.1" 502 150 2025-12-17 23:49:50,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad4575bc060ce162070'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:50,492 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad4575bc060ce162070 2025-12-17 23:49:50,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad4575bc060ce162070 HTTP/1.1" 502 150 2025-12-17 23:49:50,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad4575bc060ce162070'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:50,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad8575bc060ce162076 HTTP/1.1" 502 150 2025-12-17 23:49:50,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad8575bc060ce162076'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:50,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad8575bc060ce162076 2025-12-17 23:49:50,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad8575bc060ce162076 HTTP/1.1" 502 150 2025-12-17 23:49:50,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad8575bc060ce162076'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:50,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409acd575bc060ce162064 2025-12-17 23:49:50,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acd575bc060ce162064 HTTP/1.1" 502 150 2025-12-17 23:49:50,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409acd575bc060ce162064'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:50,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409abc575bc060ce16203b 2025-12-17 23:49:50,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409abc575bc060ce16203b HTTP/1.1" 502 150 2025-12-17 23:49:50,650 - lava_callback - ERROR - Error processing callback for node 69409abc575bc060ce16203b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409abc575bc060ce16203b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409abc575bc060ce16203b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409abc575bc060ce16203b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:50,652 - lava_callback - INFO - Starting async job submit for node 69409ad6575bc060ce162073 2025-12-17 23:49:50,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:50,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad6575bc060ce162073 HTTP/1.1" 502 150 2025-12-17 23:49:50,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad6575bc060ce162073'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:50,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad6575bc060ce162073 2025-12-17 23:49:50,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad6575bc060ce162073 HTTP/1.1" 502 150 2025-12-17 23:49:50,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad6575bc060ce162073'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:50,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409abe575bc060ce162041 2025-12-17 23:49:50,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409abe575bc060ce162041 HTTP/1.1" 502 150 2025-12-17 23:49:50,926 - lava_callback - ERROR - Error processing callback for node 69409abe575bc060ce162041: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409abe575bc060ce162041 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409abe575bc060ce162041 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409abe575bc060ce162041 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:50,929 - lava_callback - INFO - Starting async job submit for node 69409adb575bc060ce16207b 2025-12-17 23:49:50,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:50,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409adb575bc060ce16207b HTTP/1.1" 502 150 2025-12-17 23:49:50,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409adb575bc060ce16207b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:50,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409adb575bc060ce16207b 2025-12-17 23:49:50,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409adb575bc060ce16207b HTTP/1.1" 502 150 2025-12-17 23:49:50,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409adb575bc060ce16207b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:51,090 - lava_callback - INFO - Received callback for node 69409e94575bc060ce1629cd INFO: 172.21.0.1:37848 - "POST /node/69409e94575bc060ce1629cd HTTP/1.0" 202 Accepted 2025-12-17 23:49:51,371 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad1575bc060ce16206a 2025-12-17 23:49:51,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad1575bc060ce16206a HTTP/1.1" 502 150 2025-12-17 23:49:51,372 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad1575bc060ce16206a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:51,740 - lava_callback - INFO - Received callback for node 69409e96575bc060ce1629d0 INFO: 172.21.0.1:37850 - "POST /node/69409e96575bc060ce1629d0 HTTP/1.0" 202 Accepted 2025-12-17 23:49:51,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa7575bc060ce16200e HTTP/1.1" 502 150 2025-12-17 23:49:51,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa7575bc060ce16200e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:52,244 - lava_callback - INFO - Received callback for node 69409e98575bc060ce1629d3 INFO: 172.21.0.1:37856 - "POST /node/69409e98575bc060ce1629d3 HTTP/1.0" 202 Accepted 2025-12-17 23:49:52,393 - lava_callback - INFO - Received callback for node 69409e9b575bc060ce1629d8 INFO: 172.21.0.1:37870 - "POST /node/69409e9b575bc060ce1629d8 HTTP/1.0" 202 Accepted 2025-12-17 23:49:52,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad4575bc060ce162070 2025-12-17 23:49:52,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad4575bc060ce162070 HTTP/1.1" 502 150 2025-12-17 23:49:52,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad4575bc060ce162070'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:52,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad8575bc060ce162076 2025-12-17 23:49:52,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad8575bc060ce162076 HTTP/1.1" 502 150 2025-12-17 23:49:52,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad8575bc060ce162076'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:52,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409acb575bc060ce162061 2025-12-17 23:49:52,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acb575bc060ce162061 HTTP/1.1" 502 150 2025-12-17 23:49:52,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409acb575bc060ce162061'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:52,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad6575bc060ce162073 2025-12-17 23:49:52,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad6575bc060ce162073 HTTP/1.1" 502 150 2025-12-17 23:49:52,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad6575bc060ce162073'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:52,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409adb575bc060ce16207b 2025-12-17 23:49:52,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409adb575bc060ce16207b HTTP/1.1" 502 150 2025-12-17 23:49:52,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409adb575bc060ce16207b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:53,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad4575bc060ce16206d 2025-12-17 23:49:53,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad4575bc060ce16206d HTTP/1.1" 502 150 2025-12-17 23:49:53,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad4575bc060ce16206d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:53,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409acf575bc060ce162067 2025-12-17 23:49:53,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acf575bc060ce162067 HTTP/1.1" 502 150 2025-12-17 23:49:53,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409acf575bc060ce162067'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:53,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa7575bc060ce16200e 2025-12-17 23:49:54,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac0575bc060ce162044 2025-12-17 23:49:54,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac2575bc060ce162047 2025-12-17 23:49:54,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac0575bc060ce162044 HTTP/1.1" 502 150 2025-12-17 23:49:54,811 - lava_callback - ERROR - Error processing callback for node 69409ac0575bc060ce162044: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ac0575bc060ce162044 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ac0575bc060ce162044 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ac0575bc060ce162044 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:54,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac2575bc060ce162047 HTTP/1.1" 502 150 2025-12-17 23:49:54,814 - lava_callback - ERROR - Error processing callback for node 69409ac2575bc060ce162047: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ac2575bc060ce162047 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ac2575bc060ce162047 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ac2575bc060ce162047 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:49:54,816 - lava_callback - INFO - Starting async job submit for node 69409ae1575bc060ce162098 2025-12-17 23:49:54,817 - lava_callback - INFO - Starting async job submit for node 6942a2251b61f1c6005fd048 2025-12-17 23:49:54,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:54,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:49:54,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae1575bc060ce162098 HTTP/1.1" 502 150 2025-12-17 23:49:54,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae1575bc060ce162098'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:54,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae1575bc060ce162098 2025-12-17 23:49:54,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae1575bc060ce162098 HTTP/1.1" 502 150 2025-12-17 23:49:54,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae1575bc060ce162098'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:54,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942a2251b61f1c6005fd048 HTTP/1.1" 502 150 2025-12-17 23:49:54,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942a2251b61f1c6005fd048'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:54,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942a2251b61f1c6005fd048 2025-12-17 23:49:54,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942a2251b61f1c6005fd048 HTTP/1.1" 502 150 2025-12-17 23:49:54,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942a2251b61f1c6005fd048'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:55,372 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad1575bc060ce16206a 2025-12-17 23:49:55,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad1575bc060ce16206a HTTP/1.1" 502 150 2025-12-17 23:49:55,374 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad1575bc060ce16206a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:56,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad4575bc060ce162070 2025-12-17 23:49:56,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad4575bc060ce162070 HTTP/1.1" 502 150 2025-12-17 23:49:56,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad4575bc060ce162070'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:56,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad8575bc060ce162076 2025-12-17 23:49:56,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad8575bc060ce162076 HTTP/1.1" 502 150 2025-12-17 23:49:56,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad8575bc060ce162076'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37872 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:49:56,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad6575bc060ce162073 2025-12-17 23:49:56,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad6575bc060ce162073 HTTP/1.1" 502 150 2025-12-17 23:49:56,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad6575bc060ce162073'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:56,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae1575bc060ce162098 2025-12-17 23:49:56,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae1575bc060ce162098 HTTP/1.1" 502 150 2025-12-17 23:49:56,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae1575bc060ce162098'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:56,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942a2251b61f1c6005fd048 2025-12-17 23:49:56,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942a2251b61f1c6005fd048 HTTP/1.1" 502 150 2025-12-17 23:49:56,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942a2251b61f1c6005fd048'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:56,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409adb575bc060ce16207b 2025-12-17 23:49:56,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409adb575bc060ce16207b HTTP/1.1" 502 150 2025-12-17 23:49:56,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409adb575bc060ce16207b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:49:57,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac9575bc060ce16205d 2025-12-17 23:49:57,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac9575bc060ce16205d HTTP/1.1" 502 150 2025-12-17 23:49:57,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac9575bc060ce16205d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:00,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409acb575bc060ce162061 2025-12-17 23:50:00,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae1575bc060ce162098 2025-12-17 23:50:00,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae1575bc060ce162098 HTTP/1.1" 502 150 2025-12-17 23:50:00,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae1575bc060ce162098'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:00,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942a2251b61f1c6005fd048 2025-12-17 23:50:00,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942a2251b61f1c6005fd048 HTTP/1.1" 502 150 2025-12-17 23:50:00,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942a2251b61f1c6005fd048'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:00,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac6575bc060ce16204f HTTP/1.1" 502 150 2025-12-17 23:50:00,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac6575bc060ce16204f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:01,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad4575bc060ce16206d 2025-12-17 23:50:01,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad4575bc060ce16206d HTTP/1.1" 502 150 2025-12-17 23:50:01,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad4575bc060ce16206d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:01,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a105575bc060ce162dee 2025-12-17 23:50:01,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a105575bc060ce162dee HTTP/1.1" 502 150 2025-12-17 23:50:01,587 - lava_callback - ERROR - Error processing callback for node 6940a105575bc060ce162dee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a105575bc060ce162dee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a105575bc060ce162dee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a105575bc060ce162dee (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:01,589 - lava_callback - INFO - Starting async job submit for node 69409add575bc060ce162086 2025-12-17 23:50:01,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:01,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409add575bc060ce162086 HTTP/1.1" 502 150 2025-12-17 23:50:01,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409add575bc060ce162086'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:01,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409add575bc060ce162086 2025-12-17 23:50:01,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409add575bc060ce162086 HTTP/1.1" 502 150 2025-12-17 23:50:01,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409add575bc060ce162086'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:01,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409acf575bc060ce162067 2025-12-17 23:50:01,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acf575bc060ce162067 HTTP/1.1" 502 150 2025-12-17 23:50:01,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409acf575bc060ce162067'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:02,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69428f571b61f1c6005fbd14 2025-12-17 23:50:02,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69428f571b61f1c6005fbd14 HTTP/1.1" 502 150 2025-12-17 23:50:02,322 - lava_callback - ERROR - Error processing callback for node 69428f571b61f1c6005fbd14: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69428f571b61f1c6005fbd14 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69428f571b61f1c6005fbd14 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69428f571b61f1c6005fbd14 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:02,324 - lava_callback - INFO - Starting async job submit for node 69409adf575bc060ce162091 2025-12-17 23:50:02,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:02,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409adf575bc060ce162091 HTTP/1.1" 502 150 2025-12-17 23:50:02,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409adf575bc060ce162091'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:02,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409adf575bc060ce162091 2025-12-17 23:50:02,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409adf575bc060ce162091 HTTP/1.1" 502 150 2025-12-17 23:50:02,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409adf575bc060ce162091'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:03,374 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad1575bc060ce16206a 2025-12-17 23:50:03,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad1575bc060ce16206a HTTP/1.1" 502 150 2025-12-17 23:50:03,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad1575bc060ce16206a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:03,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409add575bc060ce162086 2025-12-17 23:50:03,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409add575bc060ce162086 HTTP/1.1" 502 150 2025-12-17 23:50:03,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409add575bc060ce162086'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39128 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:50:04,332 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409adf575bc060ce162091 2025-12-17 23:50:04,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409adf575bc060ce162091 HTTP/1.1" 502 150 2025-12-17 23:50:04,333 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409adf575bc060ce162091'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:04,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad4575bc060ce162070 2025-12-17 23:50:04,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad4575bc060ce162070 HTTP/1.1" 502 150 2025-12-17 23:50:04,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad4575bc060ce162070'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:04,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad8575bc060ce162076 2025-12-17 23:50:04,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad8575bc060ce162076 HTTP/1.1" 502 150 2025-12-17 23:50:04,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad8575bc060ce162076'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:04,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad6575bc060ce162073 2025-12-17 23:50:04,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad6575bc060ce162073 HTTP/1.1" 502 150 2025-12-17 23:50:04,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ad6575bc060ce162073'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:04,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac6575bc060ce16204f 2025-12-17 23:50:04,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac6575bc060ce16204f HTTP/1.1" 502 150 2025-12-17 23:50:04,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac6575bc060ce16204f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:04,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409adb575bc060ce16207b 2025-12-17 23:50:04,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409adb575bc060ce16207b HTTP/1.1" 502 150 2025-12-17 23:50:04,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409adb575bc060ce16207b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39142 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:50:06,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409acd575bc060ce162064 2025-12-17 23:50:06,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acd575bc060ce162064 HTTP/1.1" 502 150 2025-12-17 23:50:06,579 - lava_callback - ERROR - Error processing callback for node 69409acd575bc060ce162064: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409acd575bc060ce162064 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409acd575bc060ce162064 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409acd575bc060ce162064 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:06,581 - lava_callback - INFO - Starting async job submit for node 69409ae3575bc060ce16209b 2025-12-17 23:50:06,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:06,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae3575bc060ce16209b HTTP/1.1" 502 150 2025-12-17 23:50:06,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae3575bc060ce16209b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:06,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae3575bc060ce16209b 2025-12-17 23:50:06,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae3575bc060ce16209b HTTP/1.1" 502 150 2025-12-17 23:50:06,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae3575bc060ce16209b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:07,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409add575bc060ce162086 2025-12-17 23:50:07,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409add575bc060ce162086 HTTP/1.1" 502 150 2025-12-17 23:50:07,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409add575bc060ce162086'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:08,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409adf575bc060ce162091 2025-12-17 23:50:08,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409adf575bc060ce162091 HTTP/1.1" 502 150 2025-12-17 23:50:08,334 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409adf575bc060ce162091'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:08,517 - lava_callback - INFO - Received callback for node 69409e9d575bc060ce1629e0 INFO: 172.21.0.1:39152 - "POST /node/69409e9d575bc060ce1629e0 HTTP/1.0" 202 Accepted 2025-12-17 23:50:08,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae3575bc060ce16209b 2025-12-17 23:50:08,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae3575bc060ce16209b HTTP/1.1" 502 150 2025-12-17 23:50:08,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae3575bc060ce16209b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:08,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae1575bc060ce162098 2025-12-17 23:50:08,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae1575bc060ce162098 HTTP/1.1" 502 150 2025-12-17 23:50:08,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae1575bc060ce162098'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:08,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942a2251b61f1c6005fd048 2025-12-17 23:50:08,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942a2251b61f1c6005fd048 HTTP/1.1" 502 150 2025-12-17 23:50:08,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942a2251b61f1c6005fd048'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:09,164 - lava_callback - INFO - Received callback for node 69409e9f575bc060ce1629e9 INFO: 172.21.0.1:39168 - "POST /node/69409e9f575bc060ce1629e9 HTTP/1.0" 202 Accepted 2025-12-17 23:50:09,836 - lava_callback - INFO - Received callback for node 69409ea1575bc060ce1629ec INFO: 172.21.0.1:48284 - "POST /node/69409ea1575bc060ce1629ec HTTP/1.0" 202 Accepted 2025-12-17 23:50:10,500 - lava_callback - INFO - Received callback for node 69409ea3575bc060ce1629ef INFO: 172.21.0.1:48292 - "POST /node/69409ea3575bc060ce1629ef HTTP/1.0" 202 Accepted 2025-12-17 23:50:11,131 - lava_callback - INFO - Received callback for node 69409ea5575bc060ce1629f2 INFO: 172.21.0.1:48296 - "POST /node/69409ea5575bc060ce1629f2 HTTP/1.0" 202 Accepted 2025-12-17 23:50:12,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae3575bc060ce16209b 2025-12-17 23:50:12,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae3575bc060ce16209b HTTP/1.1" 502 150 2025-12-17 23:50:12,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae3575bc060ce16209b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:12,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac6575bc060ce16204f 2025-12-17 23:50:12,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac6575bc060ce16204f HTTP/1.1" 502 150 2025-12-17 23:50:12,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ac6575bc060ce16204f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:13,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac9575bc060ce16205d 2025-12-17 23:50:13,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac9575bc060ce16205d HTTP/1.1" 502 150 2025-12-17 23:50:13,619 - lava_callback - ERROR - Error processing callback for node 69409ac9575bc060ce16205d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ac9575bc060ce16205d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ac9575bc060ce16205d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ac9575bc060ce16205d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:13,621 - lava_callback - INFO - Starting async job submit for node 69409ae5575bc060ce16209e 2025-12-17 23:50:13,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:13,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae5575bc060ce16209e HTTP/1.1" 502 150 2025-12-17 23:50:13,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae5575bc060ce16209e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:13,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae5575bc060ce16209e 2025-12-17 23:50:13,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae5575bc060ce16209e HTTP/1.1" 502 150 2025-12-17 23:50:13,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae5575bc060ce16209e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48298 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:50:15,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409add575bc060ce162086 2025-12-17 23:50:15,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409add575bc060ce162086 HTTP/1.1" 502 150 2025-12-17 23:50:15,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409add575bc060ce162086'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:15,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae5575bc060ce16209e 2025-12-17 23:50:15,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae5575bc060ce16209e HTTP/1.1" 502 150 2025-12-17 23:50:15,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae5575bc060ce16209e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:15,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acb575bc060ce162061 HTTP/1.1" 502 150 2025-12-17 23:50:15,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409acb575bc060ce162061'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:16,335 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409adf575bc060ce162091 2025-12-17 23:50:16,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409adf575bc060ce162091 HTTP/1.1" 502 150 2025-12-17 23:50:16,336 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409adf575bc060ce162091'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48302 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:50:17,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad4575bc060ce16206d 2025-12-17 23:50:17,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad4575bc060ce16206d HTTP/1.1" 502 150 2025-12-17 23:50:17,525 - lava_callback - ERROR - Error processing callback for node 69409ad4575bc060ce16206d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ad4575bc060ce16206d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ad4575bc060ce16206d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ad4575bc060ce16206d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:17,527 - lava_callback - INFO - Starting async job submit for node 69409ae7575bc060ce1620a2 2025-12-17 23:50:17,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:17,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae7575bc060ce1620a2 HTTP/1.1" 502 150 2025-12-17 23:50:17,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae7575bc060ce1620a2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:17,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae7575bc060ce1620a2 2025-12-17 23:50:17,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae7575bc060ce1620a2 HTTP/1.1" 502 150 2025-12-17 23:50:17,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae7575bc060ce1620a2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:17,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409acf575bc060ce162067 2025-12-17 23:50:17,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acf575bc060ce162067 HTTP/1.1" 502 150 2025-12-17 23:50:17,634 - lava_callback - ERROR - Error processing callback for node 69409acf575bc060ce162067: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409acf575bc060ce162067 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409acf575bc060ce162067 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409acf575bc060ce162067 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:17,636 - lava_callback - INFO - Starting async job submit for node 69409ae9575bc060ce1620a5 2025-12-17 23:50:17,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:17,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae9575bc060ce1620a5 HTTP/1.1" 502 150 2025-12-17 23:50:17,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae9575bc060ce1620a5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:17,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae9575bc060ce1620a5 2025-12-17 23:50:17,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae9575bc060ce1620a5 HTTP/1.1" 502 150 2025-12-17 23:50:17,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae9575bc060ce1620a5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:19,375 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad1575bc060ce16206a 2025-12-17 23:50:19,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad1575bc060ce16206a HTTP/1.1" 502 150 2025-12-17 23:50:19,377 - lava_callback - ERROR - Error processing callback for node 69409ad1575bc060ce16206a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ad1575bc060ce16206a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ad1575bc060ce16206a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ad1575bc060ce16206a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:19,379 - lava_callback - INFO - Starting async job submit for node 69409aed575bc060ce1620ae 2025-12-17 23:50:19,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:19,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aed575bc060ce1620ae HTTP/1.1" 502 150 2025-12-17 23:50:19,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aed575bc060ce1620ae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:19,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aed575bc060ce1620ae 2025-12-17 23:50:19,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aed575bc060ce1620ae HTTP/1.1" 502 150 2025-12-17 23:50:19,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aed575bc060ce1620ae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:19,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae7575bc060ce1620a2 2025-12-17 23:50:19,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae7575bc060ce1620a2 HTTP/1.1" 502 150 2025-12-17 23:50:19,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae7575bc060ce1620a2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:19,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae5575bc060ce16209e 2025-12-17 23:50:19,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae5575bc060ce16209e HTTP/1.1" 502 150 2025-12-17 23:50:19,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae5575bc060ce16209e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:19,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae9575bc060ce1620a5 2025-12-17 23:50:19,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae9575bc060ce1620a5 HTTP/1.1" 502 150 2025-12-17 23:50:19,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae9575bc060ce1620a5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:20,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad4575bc060ce162070 2025-12-17 23:50:20,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad4575bc060ce162070 HTTP/1.1" 502 150 2025-12-17 23:50:20,499 - lava_callback - ERROR - Error processing callback for node 69409ad4575bc060ce162070: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ad4575bc060ce162070 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ad4575bc060ce162070 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ad4575bc060ce162070 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:20,501 - lava_callback - INFO - Starting async job submit for node 69409aeb575bc060ce1620a8 2025-12-17 23:50:20,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:20,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aeb575bc060ce1620a8 HTTP/1.1" 502 150 2025-12-17 23:50:20,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aeb575bc060ce1620a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:20,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aeb575bc060ce1620a8 2025-12-17 23:50:20,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aeb575bc060ce1620a8 HTTP/1.1" 502 150 2025-12-17 23:50:20,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aeb575bc060ce1620a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:20,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad8575bc060ce162076 2025-12-17 23:50:20,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad8575bc060ce162076 HTTP/1.1" 502 150 2025-12-17 23:50:20,521 - lava_callback - ERROR - Error processing callback for node 69409ad8575bc060ce162076: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ad8575bc060ce162076 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ad8575bc060ce162076 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ad8575bc060ce162076 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:20,523 - lava_callback - INFO - Starting async job submit for node 69409aec575bc060ce1620ab 2025-12-17 23:50:20,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:20,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aec575bc060ce1620ab HTTP/1.1" 502 150 2025-12-17 23:50:20,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aec575bc060ce1620ab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:20,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aec575bc060ce1620ab 2025-12-17 23:50:20,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aec575bc060ce1620ab HTTP/1.1" 502 150 2025-12-17 23:50:20,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aec575bc060ce1620ab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:20,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae3575bc060ce16209b 2025-12-17 23:50:20,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae3575bc060ce16209b HTTP/1.1" 502 150 2025-12-17 23:50:20,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae3575bc060ce16209b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:20,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ad6575bc060ce162073 2025-12-17 23:50:20,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ad6575bc060ce162073 HTTP/1.1" 502 150 2025-12-17 23:50:20,689 - lava_callback - ERROR - Error processing callback for node 69409ad6575bc060ce162073: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ad6575bc060ce162073 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ad6575bc060ce162073 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ad6575bc060ce162073 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:20,691 - lava_callback - INFO - Starting async job submit for node 69409af0575bc060ce1620bd 2025-12-17 23:50:20,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:20,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af0575bc060ce1620bd HTTP/1.1" 502 150 2025-12-17 23:50:20,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af0575bc060ce1620bd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:20,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af0575bc060ce1620bd 2025-12-17 23:50:20,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af0575bc060ce1620bd HTTP/1.1" 502 150 2025-12-17 23:50:20,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af0575bc060ce1620bd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:20,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409adb575bc060ce16207b 2025-12-17 23:50:20,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409adb575bc060ce16207b HTTP/1.1" 502 150 2025-12-17 23:50:20,950 - lava_callback - ERROR - Error processing callback for node 69409adb575bc060ce16207b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409adb575bc060ce16207b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409adb575bc060ce16207b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409adb575bc060ce16207b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:20,953 - lava_callback - INFO - Starting async job submit for node 69409af2575bc060ce1620c2 2025-12-17 23:50:20,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:20,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af2575bc060ce1620c2 HTTP/1.1" 502 150 2025-12-17 23:50:20,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af2575bc060ce1620c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:20,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af2575bc060ce1620c2 2025-12-17 23:50:20,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af2575bc060ce1620c2 HTTP/1.1" 502 150 2025-12-17 23:50:20,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af2575bc060ce1620c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:21,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aed575bc060ce1620ae 2025-12-17 23:50:21,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aed575bc060ce1620ae HTTP/1.1" 502 150 2025-12-17 23:50:21,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aed575bc060ce1620ae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:22,509 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aeb575bc060ce1620a8 2025-12-17 23:50:22,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aeb575bc060ce1620a8 HTTP/1.1" 502 150 2025-12-17 23:50:22,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aeb575bc060ce1620a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:22,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aec575bc060ce1620ab 2025-12-17 23:50:22,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aec575bc060ce1620ab HTTP/1.1" 502 150 2025-12-17 23:50:22,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aec575bc060ce1620ab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:22,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af0575bc060ce1620bd 2025-12-17 23:50:22,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af0575bc060ce1620bd HTTP/1.1" 502 150 2025-12-17 23:50:22,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af0575bc060ce1620bd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:22,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af2575bc060ce1620c2 2025-12-17 23:50:22,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af2575bc060ce1620c2 HTTP/1.1" 502 150 2025-12-17 23:50:22,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af2575bc060ce1620c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:23,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae7575bc060ce1620a2 2025-12-17 23:50:23,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae7575bc060ce1620a2 HTTP/1.1" 502 150 2025-12-17 23:50:23,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae7575bc060ce1620a2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:23,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae9575bc060ce1620a5 2025-12-17 23:50:23,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae9575bc060ce1620a5 HTTP/1.1" 502 150 2025-12-17 23:50:23,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae9575bc060ce1620a5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:24,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa7575bc060ce16200e HTTP/1.1" 502 150 2025-12-17 23:50:24,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa7575bc060ce16200e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:24,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae1575bc060ce162098 2025-12-17 23:50:24,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942a2251b61f1c6005fd048 2025-12-17 23:50:24,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae1575bc060ce162098 HTTP/1.1" 502 150 2025-12-17 23:50:24,834 - lava_callback - ERROR - Error processing callback for node 69409ae1575bc060ce162098: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ae1575bc060ce162098 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ae1575bc060ce162098 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ae1575bc060ce162098 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:24,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942a2251b61f1c6005fd048 HTTP/1.1" 502 150 2025-12-17 23:50:24,837 - lava_callback - ERROR - Error processing callback for node 6942a2251b61f1c6005fd048: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942a2251b61f1c6005fd048 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942a2251b61f1c6005fd048 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942a2251b61f1c6005fd048 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:24,837 - lava_callback - INFO - Starting async job submit for node 69409af9575bc060ce1620ce 2025-12-17 23:50:24,838 - lava_callback - INFO - Starting async job submit for node 69409af3575bc060ce1620c5 2025-12-17 23:50:24,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:24,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:24,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af9575bc060ce1620ce HTTP/1.1" 502 150 2025-12-17 23:50:24,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af9575bc060ce1620ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:24,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af9575bc060ce1620ce 2025-12-17 23:50:24,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af3575bc060ce1620c5 HTTP/1.1" 502 150 2025-12-17 23:50:24,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af3575bc060ce1620c5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:24,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af3575bc060ce1620c5 2025-12-17 23:50:24,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af9575bc060ce1620ce HTTP/1.1" 502 150 2025-12-17 23:50:24,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af9575bc060ce1620ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:24,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af3575bc060ce1620c5 HTTP/1.1" 502 150 2025-12-17 23:50:24,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af3575bc060ce1620c5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:25,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aed575bc060ce1620ae 2025-12-17 23:50:25,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aed575bc060ce1620ae HTTP/1.1" 502 150 2025-12-17 23:50:25,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aed575bc060ce1620ae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39084 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:50:26,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aeb575bc060ce1620a8 2025-12-17 23:50:26,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aec575bc060ce1620ab 2025-12-17 23:50:26,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af0575bc060ce1620bd 2025-12-17 23:50:26,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af0575bc060ce1620bd HTTP/1.1" 502 150 2025-12-17 23:50:26,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af0575bc060ce1620bd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:26,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af9575bc060ce1620ce 2025-12-17 23:50:26,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af3575bc060ce1620c5 2025-12-17 23:50:26,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af3575bc060ce1620c5 HTTP/1.1" 502 150 2025-12-17 23:50:26,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af3575bc060ce1620c5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:26,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af9575bc060ce1620ce HTTP/1.1" 502 150 2025-12-17 23:50:26,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af9575bc060ce1620ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:26,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af2575bc060ce1620c2 2025-12-17 23:50:26,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af2575bc060ce1620c2 HTTP/1.1" 502 150 2025-12-17 23:50:26,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af2575bc060ce1620c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:27,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae5575bc060ce16209e 2025-12-17 23:50:27,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae5575bc060ce16209e HTTP/1.1" 502 150 2025-12-17 23:50:27,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae5575bc060ce16209e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:28,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa7575bc060ce16200e 2025-12-17 23:50:28,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ac6575bc060ce16204f 2025-12-17 23:50:28,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ac6575bc060ce16204f HTTP/1.1" 502 150 2025-12-17 23:50:28,885 - lava_callback - ERROR - Error processing callback for node 69409ac6575bc060ce16204f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ac6575bc060ce16204f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ac6575bc060ce16204f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ac6575bc060ce16204f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:28,888 - lava_callback - INFO - Starting async job submit for node 69409af5575bc060ce1620c8 2025-12-17 23:50:28,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:28,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af5575bc060ce1620c8 HTTP/1.1" 502 150 2025-12-17 23:50:28,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af5575bc060ce1620c8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:28,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af5575bc060ce1620c8 2025-12-17 23:50:28,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af5575bc060ce1620c8 HTTP/1.1" 502 150 2025-12-17 23:50:28,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af5575bc060ce1620c8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:29,717 - lava_callback - INFO - Received callback for node 69409ec4575bc060ce162a75 INFO: 172.21.0.1:42048 - "POST /node/69409ec4575bc060ce162a75 HTTP/1.0" 202 Accepted 2025-12-17 23:50:30,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af3575bc060ce1620c5 2025-12-17 23:50:30,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af9575bc060ce1620ce 2025-12-17 23:50:30,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af3575bc060ce1620c5 HTTP/1.1" 502 150 2025-12-17 23:50:30,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af3575bc060ce1620c5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:30,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af9575bc060ce1620ce HTTP/1.1" 502 150 2025-12-17 23:50:30,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af9575bc060ce1620ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:30,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af5575bc060ce1620c8 2025-12-17 23:50:30,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af5575bc060ce1620c8 HTTP/1.1" 502 150 2025-12-17 23:50:30,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af5575bc060ce1620c8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:31,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae7575bc060ce1620a2 2025-12-17 23:50:31,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae7575bc060ce1620a2 HTTP/1.1" 502 150 2025-12-17 23:50:31,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae7575bc060ce1620a2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:31,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409add575bc060ce162086 2025-12-17 23:50:31,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409add575bc060ce162086 HTTP/1.1" 502 150 2025-12-17 23:50:31,605 - lava_callback - ERROR - Error processing callback for node 69409add575bc060ce162086: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409add575bc060ce162086 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409add575bc060ce162086 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409add575bc060ce162086 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:31,607 - lava_callback - INFO - Starting async job submit for node 69409af7575bc060ce1620cb 2025-12-17 23:50:31,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:31,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af7575bc060ce1620cb HTTP/1.1" 502 150 2025-12-17 23:50:31,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af7575bc060ce1620cb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:31,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af7575bc060ce1620cb 2025-12-17 23:50:31,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af7575bc060ce1620cb HTTP/1.1" 502 150 2025-12-17 23:50:31,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af7575bc060ce1620cb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:31,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae9575bc060ce1620a5 2025-12-17 23:50:31,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae9575bc060ce1620a5 HTTP/1.1" 502 150 2025-12-17 23:50:31,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ae9575bc060ce1620a5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:31,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409acb575bc060ce162061 2025-12-17 23:50:31,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409acb575bc060ce162061 HTTP/1.1" 502 150 2025-12-17 23:50:31,986 - lava_callback - ERROR - Error processing callback for node 69409acb575bc060ce162061: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409acb575bc060ce162061 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409acb575bc060ce162061 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409acb575bc060ce162061 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:31,988 - lava_callback - INFO - Starting async job submit for node 69409afa575bc060ce1620d1 2025-12-17 23:50:31,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:31,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afa575bc060ce1620d1 HTTP/1.1" 502 150 2025-12-17 23:50:31,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409afa575bc060ce1620d1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:31,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409afa575bc060ce1620d1 2025-12-17 23:50:31,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afa575bc060ce1620d1 HTTP/1.1" 502 150 2025-12-17 23:50:31,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409afa575bc060ce1620d1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:32,336 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409adf575bc060ce162091 2025-12-17 23:50:32,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409adf575bc060ce162091 HTTP/1.1" 502 150 2025-12-17 23:50:32,338 - lava_callback - ERROR - Error processing callback for node 69409adf575bc060ce162091: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409adf575bc060ce162091 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409adf575bc060ce162091 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409adf575bc060ce162091 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:32,340 - lava_callback - INFO - Starting async job submit for node 69409afb575bc060ce1620d4 2025-12-17 23:50:32,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:32,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afb575bc060ce1620d4 HTTP/1.1" 502 150 2025-12-17 23:50:32,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409afb575bc060ce1620d4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:32,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409afb575bc060ce1620d4 2025-12-17 23:50:32,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afb575bc060ce1620d4 HTTP/1.1" 502 150 2025-12-17 23:50:32,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409afb575bc060ce1620d4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:33,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aed575bc060ce1620ae 2025-12-17 23:50:33,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aed575bc060ce1620ae HTTP/1.1" 502 150 2025-12-17 23:50:33,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aed575bc060ce1620ae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:33,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af7575bc060ce1620cb 2025-12-17 23:50:33,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af7575bc060ce1620cb HTTP/1.1" 502 150 2025-12-17 23:50:33,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af7575bc060ce1620cb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:33,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409afa575bc060ce1620d1 2025-12-17 23:50:33,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afa575bc060ce1620d1 HTTP/1.1" 502 150 2025-12-17 23:50:33,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409afa575bc060ce1620d1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:34,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409afb575bc060ce1620d4 2025-12-17 23:50:34,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afb575bc060ce1620d4 HTTP/1.1" 502 150 2025-12-17 23:50:34,350 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409afb575bc060ce1620d4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:34,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af0575bc060ce1620bd 2025-12-17 23:50:34,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af0575bc060ce1620bd HTTP/1.1" 502 150 2025-12-17 23:50:34,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af0575bc060ce1620bd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:34,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af5575bc060ce1620c8 2025-12-17 23:50:34,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af5575bc060ce1620c8 HTTP/1.1" 502 150 2025-12-17 23:50:34,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af5575bc060ce1620c8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:34,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af2575bc060ce1620c2 2025-12-17 23:50:34,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af2575bc060ce1620c2 HTTP/1.1" 502 150 2025-12-17 23:50:34,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af2575bc060ce1620c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42060 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:50:36,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae3575bc060ce16209b 2025-12-17 23:50:36,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae3575bc060ce16209b HTTP/1.1" 502 150 2025-12-17 23:50:36,595 - lava_callback - ERROR - Error processing callback for node 69409ae3575bc060ce16209b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ae3575bc060ce16209b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ae3575bc060ce16209b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ae3575bc060ce16209b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:36,597 - lava_callback - INFO - Starting async job submit for node 69409b03575bc060ce1620eb 2025-12-17 23:50:36,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:36,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b03575bc060ce1620eb HTTP/1.1" 502 150 2025-12-17 23:50:36,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b03575bc060ce1620eb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:36,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b03575bc060ce1620eb 2025-12-17 23:50:36,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b03575bc060ce1620eb HTTP/1.1" 502 150 2025-12-17 23:50:36,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b03575bc060ce1620eb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:37,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af7575bc060ce1620cb 2025-12-17 23:50:37,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af7575bc060ce1620cb HTTP/1.1" 502 150 2025-12-17 23:50:37,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af7575bc060ce1620cb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:37,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409afa575bc060ce1620d1 2025-12-17 23:50:38,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afa575bc060ce1620d1 HTTP/1.1" 502 150 2025-12-17 23:50:38,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409afa575bc060ce1620d1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:38,350 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409afb575bc060ce1620d4 2025-12-17 23:50:38,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b03575bc060ce1620eb 2025-12-17 23:50:38,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b03575bc060ce1620eb HTTP/1.1" 502 150 2025-12-17 23:50:38,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b03575bc060ce1620eb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:38,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af3575bc060ce1620c5 2025-12-17 23:50:38,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af9575bc060ce1620ce 2025-12-17 23:50:38,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af3575bc060ce1620c5 HTTP/1.1" 502 150 2025-12-17 23:50:38,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af3575bc060ce1620c5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:38,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af9575bc060ce1620ce HTTP/1.1" 502 150 2025-12-17 23:50:38,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af9575bc060ce1620ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:41,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aec575bc060ce1620ab HTTP/1.1" 502 150 2025-12-17 23:50:41,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aec575bc060ce1620ab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:41,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aeb575bc060ce1620a8 HTTP/1.1" 502 150 2025-12-17 23:50:41,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aeb575bc060ce1620a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:42,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b03575bc060ce1620eb 2025-12-17 23:50:42,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b03575bc060ce1620eb HTTP/1.1" 502 150 2025-12-17 23:50:42,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b03575bc060ce1620eb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:42,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af5575bc060ce1620c8 2025-12-17 23:50:42,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af5575bc060ce1620c8 HTTP/1.1" 502 150 2025-12-17 23:50:42,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af5575bc060ce1620c8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:43,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae5575bc060ce16209e 2025-12-17 23:50:43,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae5575bc060ce16209e HTTP/1.1" 502 150 2025-12-17 23:50:43,637 - lava_callback - ERROR - Error processing callback for node 69409ae5575bc060ce16209e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ae5575bc060ce16209e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ae5575bc060ce16209e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ae5575bc060ce16209e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:43,640 - lava_callback - INFO - Starting async job submit for node 69409afd575bc060ce1620d7 2025-12-17 23:50:43,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:43,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afd575bc060ce1620d7 HTTP/1.1" 502 150 2025-12-17 23:50:43,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409afd575bc060ce1620d7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:43,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409afd575bc060ce1620d7 2025-12-17 23:50:43,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afd575bc060ce1620d7 HTTP/1.1" 502 150 2025-12-17 23:50:43,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409afd575bc060ce1620d7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:45,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af7575bc060ce1620cb 2025-12-17 23:50:45,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af7575bc060ce1620cb HTTP/1.1" 502 150 2025-12-17 23:50:45,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409af7575bc060ce1620cb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:45,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409afd575bc060ce1620d7 2025-12-17 23:50:45,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afd575bc060ce1620d7 HTTP/1.1" 502 150 2025-12-17 23:50:45,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409afd575bc060ce1620d7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:46,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409afa575bc060ce1620d1 2025-12-17 23:50:46,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afa575bc060ce1620d1 HTTP/1.1" 502 150 2025-12-17 23:50:46,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409afa575bc060ce1620d1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41118 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:50:47,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae7575bc060ce1620a2 2025-12-17 23:50:47,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae7575bc060ce1620a2 HTTP/1.1" 502 150 2025-12-17 23:50:47,548 - lava_callback - ERROR - Error processing callback for node 69409ae7575bc060ce1620a2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ae7575bc060ce1620a2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ae7575bc060ce1620a2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ae7575bc060ce1620a2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:47,550 - lava_callback - INFO - Starting async job submit for node 69409aff575bc060ce1620da 2025-12-17 23:50:47,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:47,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aff575bc060ce1620da HTTP/1.1" 502 150 2025-12-17 23:50:47,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aff575bc060ce1620da'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:47,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aff575bc060ce1620da 2025-12-17 23:50:47,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aff575bc060ce1620da HTTP/1.1" 502 150 2025-12-17 23:50:47,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aff575bc060ce1620da'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:47,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ae9575bc060ce1620a5 2025-12-17 23:50:47,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ae9575bc060ce1620a5 HTTP/1.1" 502 150 2025-12-17 23:50:47,650 - lava_callback - ERROR - Error processing callback for node 69409ae9575bc060ce1620a5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ae9575bc060ce1620a5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ae9575bc060ce1620a5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ae9575bc060ce1620a5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:47,652 - lava_callback - INFO - Starting async job submit for node 69409b01575bc060ce1620dd 2025-12-17 23:50:47,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:47,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b01575bc060ce1620dd HTTP/1.1" 502 150 2025-12-17 23:50:47,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b01575bc060ce1620dd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:47,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b01575bc060ce1620dd 2025-12-17 23:50:47,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b01575bc060ce1620dd HTTP/1.1" 502 150 2025-12-17 23:50:47,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b01575bc060ce1620dd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:49,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aed575bc060ce1620ae 2025-12-17 23:50:49,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aed575bc060ce1620ae HTTP/1.1" 502 150 2025-12-17 23:50:49,403 - lava_callback - ERROR - Error processing callback for node 69409aed575bc060ce1620ae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aed575bc060ce1620ae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aed575bc060ce1620ae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aed575bc060ce1620ae (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:49,405 - lava_callback - INFO - Starting async job submit for node 69409b02575bc060ce1620e0 2025-12-17 23:50:49,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:49,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b02575bc060ce1620e0 HTTP/1.1" 502 150 2025-12-17 23:50:49,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b02575bc060ce1620e0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:49,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b02575bc060ce1620e0 2025-12-17 23:50:49,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b02575bc060ce1620e0 HTTP/1.1" 502 150 2025-12-17 23:50:49,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b02575bc060ce1620e0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:49,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aff575bc060ce1620da 2025-12-17 23:50:49,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aff575bc060ce1620da HTTP/1.1" 502 150 2025-12-17 23:50:49,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aff575bc060ce1620da'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:49,642 - lava_callback - INFO - Received callback for node 69412836459ff5fc64d2bf6a INFO: 172.21.0.1:41120 - "POST /node/69412836459ff5fc64d2bf6a HTTP/1.0" 202 Accepted 2025-12-17 23:50:49,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b01575bc060ce1620dd 2025-12-17 23:50:49,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b01575bc060ce1620dd HTTP/1.1" 502 150 2025-12-17 23:50:49,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b01575bc060ce1620dd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:49,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409afd575bc060ce1620d7 2025-12-17 23:50:49,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afd575bc060ce1620d7 HTTP/1.1" 502 150 2025-12-17 23:50:49,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409afd575bc060ce1620d7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:49,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aec575bc060ce1620ab 2025-12-17 23:50:49,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aeb575bc060ce1620a8 2025-12-17 23:50:49,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aec575bc060ce1620ab HTTP/1.1" 502 150 2025-12-17 23:50:49,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aec575bc060ce1620ab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:49,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aeb575bc060ce1620a8 HTTP/1.1" 502 150 2025-12-17 23:50:49,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aeb575bc060ce1620a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:49,891 - lava_callback - INFO - Received callback for node 69409ec5575bc060ce162a7b INFO: 172.21.0.1:42226 - "POST /node/69409ec5575bc060ce162a7b HTTP/1.0" 202 Accepted 2025-12-17 23:50:50,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b03575bc060ce1620eb 2025-12-17 23:50:50,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b03575bc060ce1620eb HTTP/1.1" 502 150 2025-12-17 23:50:50,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b03575bc060ce1620eb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:50,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af0575bc060ce1620bd 2025-12-17 23:50:50,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af0575bc060ce1620bd HTTP/1.1" 502 150 2025-12-17 23:50:50,706 - lava_callback - ERROR - Error processing callback for node 69409af0575bc060ce1620bd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af0575bc060ce1620bd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af0575bc060ce1620bd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af0575bc060ce1620bd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:50,708 - lava_callback - INFO - Starting async job submit for node 69409803575bc060ce161994 2025-12-17 23:50:50,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:50,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409803575bc060ce161994 HTTP/1.1" 502 150 2025-12-17 23:50:50,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409803575bc060ce161994'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:50,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409803575bc060ce161994 2025-12-17 23:50:50,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409803575bc060ce161994 HTTP/1.1" 502 150 2025-12-17 23:50:50,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409803575bc060ce161994'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:50,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af2575bc060ce1620c2 2025-12-17 23:50:50,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af2575bc060ce1620c2 HTTP/1.1" 502 150 2025-12-17 23:50:50,975 - lava_callback - ERROR - Error processing callback for node 69409af2575bc060ce1620c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af2575bc060ce1620c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af2575bc060ce1620c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af2575bc060ce1620c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:50,977 - lava_callback - INFO - Starting async job submit for node 69409b06575bc060ce1620f7 2025-12-17 23:50:50,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:51,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b06575bc060ce1620f7 HTTP/1.1" 502 150 2025-12-17 23:50:51,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b06575bc060ce1620f7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:51,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b06575bc060ce1620f7 2025-12-17 23:50:51,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b06575bc060ce1620f7 HTTP/1.1" 502 150 2025-12-17 23:50:51,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b06575bc060ce1620f7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:51,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b02575bc060ce1620e0 2025-12-17 23:50:51,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b02575bc060ce1620e0 HTTP/1.1" 502 150 2025-12-17 23:50:51,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b02575bc060ce1620e0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:52,718 - lava_callback - INFO - Received callback for node 69409ecb575bc060ce162a80 INFO: 172.21.0.1:42238 - "POST /node/69409ecb575bc060ce162a80 HTTP/1.0" 202 Accepted 2025-12-17 23:50:52,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409803575bc060ce161994 2025-12-17 23:50:52,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409803575bc060ce161994 HTTP/1.1" 502 150 2025-12-17 23:50:52,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409803575bc060ce161994'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:53,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b06575bc060ce1620f7 2025-12-17 23:50:53,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b06575bc060ce1620f7 HTTP/1.1" 502 150 2025-12-17 23:50:53,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b06575bc060ce1620f7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:53,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aff575bc060ce1620da 2025-12-17 23:50:53,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aff575bc060ce1620da HTTP/1.1" 502 150 2025-12-17 23:50:53,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aff575bc060ce1620da'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:53,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afb575bc060ce1620d4 HTTP/1.1" 502 150 2025-12-17 23:50:53,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409afb575bc060ce1620d4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:53,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b01575bc060ce1620dd 2025-12-17 23:50:53,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b01575bc060ce1620dd HTTP/1.1" 502 150 2025-12-17 23:50:53,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b01575bc060ce1620dd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:54,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af3575bc060ce1620c5 2025-12-17 23:50:54,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af9575bc060ce1620ce 2025-12-17 23:50:54,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af9575bc060ce1620ce HTTP/1.1" 502 150 2025-12-17 23:50:54,854 - lava_callback - ERROR - Error processing callback for node 69409af9575bc060ce1620ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af9575bc060ce1620ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af9575bc060ce1620ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af9575bc060ce1620ce (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:54,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af3575bc060ce1620c5 HTTP/1.1" 502 150 2025-12-17 23:50:54,857 - lava_callback - ERROR - Error processing callback for node 69409af3575bc060ce1620c5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af3575bc060ce1620c5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af3575bc060ce1620c5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af3575bc060ce1620c5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:54,857 - lava_callback - INFO - Starting async job submit for node 69409b08575bc060ce1620fa 2025-12-17 23:50:54,858 - lava_callback - INFO - Starting async job submit for node 69409b0a575bc060ce1620ff 2025-12-17 23:50:54,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:54,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:54,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b08575bc060ce1620fa HTTP/1.1" 502 150 2025-12-17 23:50:54,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b08575bc060ce1620fa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:54,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b08575bc060ce1620fa 2025-12-17 23:50:54,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b08575bc060ce1620fa HTTP/1.1" 502 150 2025-12-17 23:50:54,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b08575bc060ce1620fa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:54,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0a575bc060ce1620ff HTTP/1.1" 502 150 2025-12-17 23:50:54,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0a575bc060ce1620ff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:54,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0a575bc060ce1620ff 2025-12-17 23:50:54,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0a575bc060ce1620ff HTTP/1.1" 502 150 2025-12-17 23:50:54,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0a575bc060ce1620ff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:55,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b02575bc060ce1620e0 2025-12-17 23:50:55,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b02575bc060ce1620e0 HTTP/1.1" 502 150 2025-12-17 23:50:55,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b02575bc060ce1620e0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42252 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:50:56,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409803575bc060ce161994 2025-12-17 23:50:56,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409803575bc060ce161994 HTTP/1.1" 502 150 2025-12-17 23:50:56,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409803575bc060ce161994'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:56,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b08575bc060ce1620fa 2025-12-17 23:50:56,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b08575bc060ce1620fa HTTP/1.1" 502 150 2025-12-17 23:50:56,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b08575bc060ce1620fa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:56,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0a575bc060ce1620ff 2025-12-17 23:50:56,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0a575bc060ce1620ff HTTP/1.1" 502 150 2025-12-17 23:50:56,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0a575bc060ce1620ff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:57,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b06575bc060ce1620f7 2025-12-17 23:50:57,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b06575bc060ce1620f7 HTTP/1.1" 502 150 2025-12-17 23:50:57,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b06575bc060ce1620f7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:57,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409afd575bc060ce1620d7 2025-12-17 23:50:57,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afd575bc060ce1620d7 HTTP/1.1" 502 150 2025-12-17 23:50:57,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409afd575bc060ce1620d7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:57,811 - lava_callback - INFO - Received callback for node 69409ed8575bc060ce162a8d INFO: 172.21.0.1:42254 - "POST /node/69409ed8575bc060ce162a8d HTTP/1.0" 202 Accepted 2025-12-17 23:50:58,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af5575bc060ce1620c8 2025-12-17 23:50:58,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af5575bc060ce1620c8 HTTP/1.1" 502 150 2025-12-17 23:50:58,924 - lava_callback - ERROR - Error processing callback for node 69409af5575bc060ce1620c8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af5575bc060ce1620c8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af5575bc060ce1620c8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af5575bc060ce1620c8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:50:58,925 - lava_callback - INFO - Starting async job submit for node 69409b0b575bc060ce162102 2025-12-17 23:50:58,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:50:58,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0b575bc060ce162102 HTTP/1.1" 502 150 2025-12-17 23:50:58,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0b575bc060ce162102'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:58,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0b575bc060ce162102 2025-12-17 23:50:58,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0b575bc060ce162102 HTTP/1.1" 502 150 2025-12-17 23:50:58,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0b575bc060ce162102'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:50:59,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa7575bc060ce16200e HTTP/1.1" 502 150 2025-12-17 23:50:59,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa7575bc060ce16200e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:00,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b08575bc060ce1620fa 2025-12-17 23:51:00,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b08575bc060ce1620fa HTTP/1.1" 502 150 2025-12-17 23:51:00,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b08575bc060ce1620fa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:00,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0a575bc060ce1620ff 2025-12-17 23:51:00,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0a575bc060ce1620ff HTTP/1.1" 502 150 2025-12-17 23:51:00,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0a575bc060ce1620ff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:00,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0b575bc060ce162102 2025-12-17 23:51:00,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0b575bc060ce162102 HTTP/1.1" 502 150 2025-12-17 23:51:00,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0b575bc060ce162102'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:01,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aff575bc060ce1620da 2025-12-17 23:51:01,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aff575bc060ce1620da HTTP/1.1" 502 150 2025-12-17 23:51:01,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aff575bc060ce1620da'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:01,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409afb575bc060ce1620d4 2025-12-17 23:51:01,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afb575bc060ce1620d4 HTTP/1.1" 502 150 2025-12-17 23:51:01,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409afb575bc060ce1620d4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:01,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409af7575bc060ce1620cb 2025-12-17 23:51:01,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409af7575bc060ce1620cb HTTP/1.1" 502 150 2025-12-17 23:51:01,619 - lava_callback - ERROR - Error processing callback for node 69409af7575bc060ce1620cb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af7575bc060ce1620cb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af7575bc060ce1620cb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409af7575bc060ce1620cb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:01,621 - lava_callback - INFO - Starting async job submit for node 69409b0d575bc060ce162105 2025-12-17 23:51:01,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:01,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0d575bc060ce162105 HTTP/1.1" 502 150 2025-12-17 23:51:01,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0d575bc060ce162105'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:01,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0d575bc060ce162105 2025-12-17 23:51:01,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0d575bc060ce162105 HTTP/1.1" 502 150 2025-12-17 23:51:01,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0d575bc060ce162105'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:01,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b01575bc060ce1620dd 2025-12-17 23:51:01,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b01575bc060ce1620dd HTTP/1.1" 502 150 2025-12-17 23:51:01,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b01575bc060ce1620dd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:02,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409afa575bc060ce1620d1 2025-12-17 23:51:02,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afa575bc060ce1620d1 HTTP/1.1" 502 150 2025-12-17 23:51:02,003 - lava_callback - ERROR - Error processing callback for node 69409afa575bc060ce1620d1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409afa575bc060ce1620d1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409afa575bc060ce1620d1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409afa575bc060ce1620d1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:02,005 - lava_callback - INFO - Starting async job submit for node 69409b0f575bc060ce162108 2025-12-17 23:51:02,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:02,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0f575bc060ce162108 HTTP/1.1" 502 150 2025-12-17 23:51:02,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0f575bc060ce162108'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:02,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0f575bc060ce162108 2025-12-17 23:51:02,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0f575bc060ce162108 HTTP/1.1" 502 150 2025-12-17 23:51:02,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0f575bc060ce162108'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:03,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b02575bc060ce1620e0 2025-12-17 23:51:03,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0d575bc060ce162105 2025-12-17 23:51:03,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0d575bc060ce162105 HTTP/1.1" 502 150 2025-12-17 23:51:03,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0d575bc060ce162105'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46588 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:51:04,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0f575bc060ce162108 2025-12-17 23:51:04,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409803575bc060ce161994 2025-12-17 23:51:04,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409803575bc060ce161994 HTTP/1.1" 502 150 2025-12-17 23:51:04,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409803575bc060ce161994'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:04,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0b575bc060ce162102 2025-12-17 23:51:04,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0b575bc060ce162102 HTTP/1.1" 502 150 2025-12-17 23:51:04,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0b575bc060ce162102'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:05,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b06575bc060ce1620f7 2025-12-17 23:51:05,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b06575bc060ce1620f7 HTTP/1.1" 502 150 2025-12-17 23:51:05,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b06575bc060ce1620f7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:05,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aec575bc060ce1620ab 2025-12-17 23:51:05,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aeb575bc060ce1620a8 2025-12-17 23:51:05,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aec575bc060ce1620ab HTTP/1.1" 502 150 2025-12-17 23:51:05,844 - lava_callback - ERROR - Error processing callback for node 69409aec575bc060ce1620ab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aec575bc060ce1620ab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aec575bc060ce1620ab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aec575bc060ce1620ab (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:05,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aeb575bc060ce1620a8 HTTP/1.1" 502 150 2025-12-17 23:51:05,846 - lava_callback - ERROR - Error processing callback for node 69409aeb575bc060ce1620a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aeb575bc060ce1620a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aeb575bc060ce1620a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aeb575bc060ce1620a8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:05,846 - lava_callback - INFO - Starting async job submit for node 69409b10575bc060ce16210b 2025-12-17 23:51:05,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:05,848 - lava_callback - INFO - Starting async job submit for node 69409b12575bc060ce16210e 2025-12-17 23:51:05,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:05,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b12575bc060ce16210e HTTP/1.1" 502 150 2025-12-17 23:51:05,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b12575bc060ce16210e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:05,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b10575bc060ce16210b HTTP/1.1" 502 150 2025-12-17 23:51:05,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b12575bc060ce16210e 2025-12-17 23:51:05,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b10575bc060ce16210b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:05,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b10575bc060ce16210b 2025-12-17 23:51:05,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b12575bc060ce16210e HTTP/1.1" 502 150 2025-12-17 23:51:05,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b12575bc060ce16210e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:05,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b10575bc060ce16210b HTTP/1.1" 502 150 2025-12-17 23:51:05,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b10575bc060ce16210b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46590 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:51:06,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b03575bc060ce1620eb 2025-12-17 23:51:06,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b03575bc060ce1620eb HTTP/1.1" 502 150 2025-12-17 23:51:06,610 - lava_callback - ERROR - Error processing callback for node 69409b03575bc060ce1620eb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b03575bc060ce1620eb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b03575bc060ce1620eb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b03575bc060ce1620eb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:06,612 - lava_callback - INFO - Starting async job submit for node 69409b13575bc060ce162111 2025-12-17 23:51:06,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:06,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b13575bc060ce162111 HTTP/1.1" 502 150 2025-12-17 23:51:06,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b13575bc060ce162111'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:06,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b13575bc060ce162111 2025-12-17 23:51:06,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b13575bc060ce162111 HTTP/1.1" 502 150 2025-12-17 23:51:06,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b13575bc060ce162111'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:07,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa7575bc060ce16200e 2025-12-17 23:51:07,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0d575bc060ce162105 2025-12-17 23:51:07,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0d575bc060ce162105 HTTP/1.1" 502 150 2025-12-17 23:51:07,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0d575bc060ce162105'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:07,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b12575bc060ce16210e 2025-12-17 23:51:07,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b10575bc060ce16210b 2025-12-17 23:51:07,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b12575bc060ce16210e HTTP/1.1" 502 150 2025-12-17 23:51:07,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b12575bc060ce16210e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:07,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b10575bc060ce16210b HTTP/1.1" 502 150 2025-12-17 23:51:07,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b10575bc060ce16210b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:08,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b13575bc060ce162111 2025-12-17 23:51:08,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b13575bc060ce162111 HTTP/1.1" 502 150 2025-12-17 23:51:08,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b13575bc060ce162111'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:08,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b08575bc060ce1620fa 2025-12-17 23:51:08,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b08575bc060ce1620fa HTTP/1.1" 502 150 2025-12-17 23:51:08,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b08575bc060ce1620fa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:08,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0a575bc060ce1620ff 2025-12-17 23:51:08,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0a575bc060ce1620ff HTTP/1.1" 502 150 2025-12-17 23:51:08,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0a575bc060ce1620ff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:09,493 - lava_callback - INFO - Received callback for node 69409eda575bc060ce162a90 INFO: 172.21.0.1:46598 - "POST /node/69409eda575bc060ce162a90 HTTP/1.0" 202 Accepted 2025-12-17 23:51:11,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b12575bc060ce16210e 2025-12-17 23:51:11,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b10575bc060ce16210b 2025-12-17 23:51:11,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b10575bc060ce16210b HTTP/1.1" 502 150 2025-12-17 23:51:11,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b10575bc060ce16210b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:11,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b12575bc060ce16210e HTTP/1.1" 502 150 2025-12-17 23:51:11,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b12575bc060ce16210e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:12,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b13575bc060ce162111 2025-12-17 23:51:12,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b13575bc060ce162111 HTTP/1.1" 502 150 2025-12-17 23:51:12,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b13575bc060ce162111'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:12,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0b575bc060ce162102 2025-12-17 23:51:12,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0b575bc060ce162102 HTTP/1.1" 502 150 2025-12-17 23:51:12,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0b575bc060ce162102'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:13,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409afd575bc060ce1620d7 2025-12-17 23:51:13,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afd575bc060ce1620d7 HTTP/1.1" 502 150 2025-12-17 23:51:13,677 - lava_callback - ERROR - Error processing callback for node 69409afd575bc060ce1620d7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409afd575bc060ce1620d7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409afd575bc060ce1620d7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409afd575bc060ce1620d7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:13,679 - lava_callback - INFO - Starting async job submit for node 69409b15575bc060ce162114 2025-12-17 23:51:13,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:13,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b15575bc060ce162114 HTTP/1.1" 502 150 2025-12-17 23:51:13,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b15575bc060ce162114'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:13,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b15575bc060ce162114 2025-12-17 23:51:13,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b15575bc060ce162114 HTTP/1.1" 502 150 2025-12-17 23:51:13,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b15575bc060ce162114'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48448 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:51:15,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0d575bc060ce162105 2025-12-17 23:51:15,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0d575bc060ce162105 HTTP/1.1" 502 150 2025-12-17 23:51:15,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0d575bc060ce162105'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:15,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b15575bc060ce162114 2025-12-17 23:51:15,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b15575bc060ce162114 HTTP/1.1" 502 150 2025-12-17 23:51:15,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b15575bc060ce162114'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48464 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:51:17,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aff575bc060ce1620da 2025-12-17 23:51:17,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aff575bc060ce1620da HTTP/1.1" 502 150 2025-12-17 23:51:17,563 - lava_callback - ERROR - Error processing callback for node 69409aff575bc060ce1620da: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aff575bc060ce1620da (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aff575bc060ce1620da (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aff575bc060ce1620da (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:17,565 - lava_callback - INFO - Starting async job submit for node 69409b20575bc060ce162132 2025-12-17 23:51:17,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:17,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b20575bc060ce162132 HTTP/1.1" 502 150 2025-12-17 23:51:17,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b20575bc060ce162132'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:17,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b20575bc060ce162132 2025-12-17 23:51:17,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b20575bc060ce162132 HTTP/1.1" 502 150 2025-12-17 23:51:17,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b20575bc060ce162132'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:17,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409afb575bc060ce1620d4 2025-12-17 23:51:17,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409afb575bc060ce1620d4 HTTP/1.1" 502 150 2025-12-17 23:51:17,619 - lava_callback - ERROR - Error processing callback for node 69409afb575bc060ce1620d4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409afb575bc060ce1620d4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409afb575bc060ce1620d4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409afb575bc060ce1620d4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:17,620 - lava_callback - INFO - Starting async job submit for node 69409b1d575bc060ce16212f 2025-12-17 23:51:17,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:17,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b1d575bc060ce16212f HTTP/1.1" 502 150 2025-12-17 23:51:17,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b1d575bc060ce16212f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:17,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b1d575bc060ce16212f 2025-12-17 23:51:17,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b1d575bc060ce16212f HTTP/1.1" 502 150 2025-12-17 23:51:17,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b1d575bc060ce16212f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:17,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b01575bc060ce1620dd 2025-12-17 23:51:17,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b01575bc060ce1620dd HTTP/1.1" 502 150 2025-12-17 23:51:17,664 - lava_callback - ERROR - Error processing callback for node 69409b01575bc060ce1620dd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b01575bc060ce1620dd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b01575bc060ce1620dd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b01575bc060ce1620dd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:17,666 - lava_callback - INFO - Starting async job submit for node 69409b21575bc060ce162135 2025-12-17 23:51:17,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:17,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b21575bc060ce162135 HTTP/1.1" 502 150 2025-12-17 23:51:17,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b21575bc060ce162135'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:17,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b21575bc060ce162135 2025-12-17 23:51:17,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b21575bc060ce162135 HTTP/1.1" 502 150 2025-12-17 23:51:17,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b21575bc060ce162135'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:17,851 - lava_callback - INFO - Received callback for node 69409eec575bc060ce162aa3 INFO: 172.21.0.1:48476 - "POST /node/69409eec575bc060ce162aa3 HTTP/1.0" 202 Accepted 2025-12-17 23:51:18,511 - lava_callback - INFO - Received callback for node 69409eef575bc060ce162aae INFO: 172.21.0.1:48482 - "POST /node/69409eef575bc060ce162aae HTTP/1.0" 202 Accepted 2025-12-17 23:51:18,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b02575bc060ce1620e0 HTTP/1.1" 502 150 2025-12-17 23:51:18,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b02575bc060ce1620e0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:19,207 - lava_callback - INFO - Received callback for node 69409ef1575bc060ce162ab1 INFO: 172.21.0.1:48486 - "POST /node/69409ef1575bc060ce162ab1 HTTP/1.0" 202 Accepted 2025-12-17 23:51:19,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0f575bc060ce162108 HTTP/1.1" 502 150 2025-12-17 23:51:19,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0f575bc060ce162108'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:19,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b20575bc060ce162132 2025-12-17 23:51:19,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b20575bc060ce162132 HTTP/1.1" 502 150 2025-12-17 23:51:19,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b20575bc060ce162132'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:19,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b1d575bc060ce16212f 2025-12-17 23:51:19,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b1d575bc060ce16212f HTTP/1.1" 502 150 2025-12-17 23:51:19,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b1d575bc060ce16212f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:19,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b21575bc060ce162135 2025-12-17 23:51:19,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b21575bc060ce162135 HTTP/1.1" 502 150 2025-12-17 23:51:19,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b21575bc060ce162135'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:19,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b15575bc060ce162114 2025-12-17 23:51:19,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b15575bc060ce162114 HTTP/1.1" 502 150 2025-12-17 23:51:19,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b15575bc060ce162114'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:19,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b10575bc060ce16210b 2025-12-17 23:51:19,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b12575bc060ce16210e 2025-12-17 23:51:19,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b10575bc060ce16210b HTTP/1.1" 502 150 2025-12-17 23:51:19,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b10575bc060ce16210b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:19,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b12575bc060ce16210e HTTP/1.1" 502 150 2025-12-17 23:51:19,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b12575bc060ce16210e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:19,867 - lava_callback - INFO - Received callback for node 69409ef3575bc060ce162ab4 INFO: 172.21.0.1:39406 - "POST /node/69409ef3575bc060ce162ab4 HTTP/1.0" 202 Accepted 2025-12-17 23:51:20,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b13575bc060ce162111 2025-12-17 23:51:20,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b13575bc060ce162111 HTTP/1.1" 502 150 2025-12-17 23:51:20,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b13575bc060ce162111'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:20,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409803575bc060ce161994 2025-12-17 23:51:20,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409803575bc060ce161994 HTTP/1.1" 502 150 2025-12-17 23:51:20,750 - lava_callback - ERROR - Error processing callback for node 69409803575bc060ce161994: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409803575bc060ce161994 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409803575bc060ce161994 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409803575bc060ce161994 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:20,752 - lava_callback - INFO - Starting async job submit for node 69409b23575bc060ce16213c 2025-12-17 23:51:20,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:20,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b23575bc060ce16213c HTTP/1.1" 502 150 2025-12-17 23:51:20,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b23575bc060ce16213c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:20,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b23575bc060ce16213c 2025-12-17 23:51:20,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b23575bc060ce16213c HTTP/1.1" 502 150 2025-12-17 23:51:20,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b23575bc060ce16213c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:21,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b06575bc060ce1620f7 2025-12-17 23:51:21,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b06575bc060ce1620f7 HTTP/1.1" 502 150 2025-12-17 23:51:21,013 - lava_callback - ERROR - Error processing callback for node 69409b06575bc060ce1620f7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b06575bc060ce1620f7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b06575bc060ce1620f7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b06575bc060ce1620f7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:21,015 - lava_callback - INFO - Starting async job submit for node 69409b25575bc060ce16213f 2025-12-17 23:51:21,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:21,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b25575bc060ce16213f HTTP/1.1" 502 150 2025-12-17 23:51:21,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b25575bc060ce16213f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:21,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b25575bc060ce16213f 2025-12-17 23:51:21,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b25575bc060ce16213f HTTP/1.1" 502 150 2025-12-17 23:51:21,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b25575bc060ce16213f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:22,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b23575bc060ce16213c 2025-12-17 23:51:22,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b23575bc060ce16213c HTTP/1.1" 502 150 2025-12-17 23:51:22,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b23575bc060ce16213c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:23,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b25575bc060ce16213f 2025-12-17 23:51:23,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b25575bc060ce16213f HTTP/1.1" 502 150 2025-12-17 23:51:23,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b25575bc060ce16213f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:23,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0f575bc060ce162108 2025-12-17 23:51:23,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0f575bc060ce162108 HTTP/1.1" 502 150 2025-12-17 23:51:23,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0f575bc060ce162108'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:23,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b20575bc060ce162132 2025-12-17 23:51:23,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b20575bc060ce162132 HTTP/1.1" 502 150 2025-12-17 23:51:23,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b20575bc060ce162132'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:23,630 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b1d575bc060ce16212f 2025-12-17 23:51:23,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b1d575bc060ce16212f HTTP/1.1" 502 150 2025-12-17 23:51:23,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b1d575bc060ce16212f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:23,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b21575bc060ce162135 2025-12-17 23:51:23,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b21575bc060ce162135 HTTP/1.1" 502 150 2025-12-17 23:51:23,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b21575bc060ce162135'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:24,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b08575bc060ce1620fa 2025-12-17 23:51:24,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b08575bc060ce1620fa HTTP/1.1" 502 150 2025-12-17 23:51:24,878 - lava_callback - ERROR - Error processing callback for node 69409b08575bc060ce1620fa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b08575bc060ce1620fa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b08575bc060ce1620fa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b08575bc060ce1620fa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:24,880 - lava_callback - INFO - Starting async job submit for node 69409b27575bc060ce162142 2025-12-17 23:51:24,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:24,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b27575bc060ce162142 HTTP/1.1" 502 150 2025-12-17 23:51:24,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b27575bc060ce162142'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:24,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b27575bc060ce162142 2025-12-17 23:51:24,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b27575bc060ce162142 HTTP/1.1" 502 150 2025-12-17 23:51:24,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b27575bc060ce162142'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:24,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0a575bc060ce1620ff 2025-12-17 23:51:24,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0a575bc060ce1620ff HTTP/1.1" 502 150 2025-12-17 23:51:24,893 - lava_callback - ERROR - Error processing callback for node 69409b0a575bc060ce1620ff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b0a575bc060ce1620ff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b0a575bc060ce1620ff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b0a575bc060ce1620ff (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:24,894 - lava_callback - INFO - Starting async job submit for node 69409b28575bc060ce162145 2025-12-17 23:51:24,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:24,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b28575bc060ce162145 HTTP/1.1" 502 150 2025-12-17 23:51:24,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b28575bc060ce162145'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:24,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b28575bc060ce162145 2025-12-17 23:51:24,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b28575bc060ce162145 HTTP/1.1" 502 150 2025-12-17 23:51:24,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b28575bc060ce162145'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39418 - "GET /node/68bf743b2f2cebd04fd7570c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:39428 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:51:26,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b23575bc060ce16213c 2025-12-17 23:51:26,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b23575bc060ce16213c HTTP/1.1" 502 150 2025-12-17 23:51:26,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b23575bc060ce16213c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:26,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b27575bc060ce162142 2025-12-17 23:51:26,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b27575bc060ce162142 HTTP/1.1" 502 150 2025-12-17 23:51:26,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b27575bc060ce162142'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:26,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b28575bc060ce162145 2025-12-17 23:51:26,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b28575bc060ce162145 HTTP/1.1" 502 150 2025-12-17 23:51:26,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b28575bc060ce162145'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:27,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b25575bc060ce16213f 2025-12-17 23:51:27,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b25575bc060ce16213f HTTP/1.1" 502 150 2025-12-17 23:51:27,047 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b25575bc060ce16213f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:27,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b15575bc060ce162114 2025-12-17 23:51:27,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b15575bc060ce162114 HTTP/1.1" 502 150 2025-12-17 23:51:27,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b15575bc060ce162114'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:28,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0b575bc060ce162102 2025-12-17 23:51:28,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0b575bc060ce162102 HTTP/1.1" 502 150 2025-12-17 23:51:28,938 - lava_callback - ERROR - Error processing callback for node 69409b0b575bc060ce162102: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b0b575bc060ce162102 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b0b575bc060ce162102 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b0b575bc060ce162102 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:28,940 - lava_callback - INFO - Starting async job submit for node 69409b2a575bc060ce162148 2025-12-17 23:51:28,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:28,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2a575bc060ce162148 HTTP/1.1" 502 150 2025-12-17 23:51:28,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b2a575bc060ce162148'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:28,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b2a575bc060ce162148 2025-12-17 23:51:28,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2a575bc060ce162148 HTTP/1.1" 502 150 2025-12-17 23:51:28,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b2a575bc060ce162148'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:30,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b27575bc060ce162142 2025-12-17 23:51:30,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b28575bc060ce162145 2025-12-17 23:51:30,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b28575bc060ce162145 HTTP/1.1" 502 150 2025-12-17 23:51:30,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b28575bc060ce162145'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:30,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b2a575bc060ce162148 2025-12-17 23:51:30,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2a575bc060ce162148 HTTP/1.1" 502 150 2025-12-17 23:51:30,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b2a575bc060ce162148'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:31,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0f575bc060ce162108 2025-12-17 23:51:31,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0f575bc060ce162108 HTTP/1.1" 502 150 2025-12-17 23:51:31,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b0f575bc060ce162108'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:31,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b20575bc060ce162132 2025-12-17 23:51:31,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b20575bc060ce162132 HTTP/1.1" 502 150 2025-12-17 23:51:31,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b20575bc060ce162132'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:31,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b1d575bc060ce16212f 2025-12-17 23:51:31,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b1d575bc060ce16212f HTTP/1.1" 502 150 2025-12-17 23:51:31,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b1d575bc060ce16212f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:31,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0d575bc060ce162105 2025-12-17 23:51:31,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0d575bc060ce162105 HTTP/1.1" 502 150 2025-12-17 23:51:31,642 - lava_callback - ERROR - Error processing callback for node 69409b0d575bc060ce162105: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b0d575bc060ce162105 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b0d575bc060ce162105 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b0d575bc060ce162105 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:31,644 - lava_callback - INFO - Starting async job submit for node 69409b2d575bc060ce162157 2025-12-17 23:51:31,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:31,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2d575bc060ce162157 HTTP/1.1" 502 150 2025-12-17 23:51:31,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b2d575bc060ce162157'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:31,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b2d575bc060ce162157 2025-12-17 23:51:31,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2d575bc060ce162157 HTTP/1.1" 502 150 2025-12-17 23:51:31,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b2d575bc060ce162157'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:31,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b21575bc060ce162135 2025-12-17 23:51:31,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b21575bc060ce162135 HTTP/1.1" 502 150 2025-12-17 23:51:31,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b21575bc060ce162135'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:33,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b2d575bc060ce162157 2025-12-17 23:51:33,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2d575bc060ce162157 HTTP/1.1" 502 150 2025-12-17 23:51:33,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b2d575bc060ce162157'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:34,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b02575bc060ce1620e0 2025-12-17 23:51:34,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b02575bc060ce1620e0 HTTP/1.1" 502 150 2025-12-17 23:51:34,705 - lava_callback - ERROR - Error processing callback for node 69409b02575bc060ce1620e0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b02575bc060ce1620e0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b02575bc060ce1620e0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b02575bc060ce1620e0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:34,707 - lava_callback - INFO - Starting async job submit for node 69409b2f575bc060ce16215e 2025-12-17 23:51:34,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:34,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2f575bc060ce16215e HTTP/1.1" 502 150 2025-12-17 23:51:34,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b2f575bc060ce16215e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:34,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b2f575bc060ce16215e 2025-12-17 23:51:34,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2f575bc060ce16215e HTTP/1.1" 502 150 2025-12-17 23:51:34,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b2f575bc060ce16215e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:34,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b23575bc060ce16213c 2025-12-17 23:51:34,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b23575bc060ce16213c HTTP/1.1" 502 150 2025-12-17 23:51:34,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b23575bc060ce16213c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:34,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b2a575bc060ce162148 2025-12-17 23:51:34,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2a575bc060ce162148 HTTP/1.1" 502 150 2025-12-17 23:51:34,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b2a575bc060ce162148'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:35,048 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b25575bc060ce16213f 2025-12-17 23:51:35,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b25575bc060ce16213f HTTP/1.1" 502 150 2025-12-17 23:51:35,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b25575bc060ce16213f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:35,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b10575bc060ce16210b 2025-12-17 23:51:35,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b12575bc060ce16210e 2025-12-17 23:51:35,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b10575bc060ce16210b HTTP/1.1" 502 150 2025-12-17 23:51:35,866 - lava_callback - ERROR - Error processing callback for node 69409b10575bc060ce16210b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b10575bc060ce16210b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b10575bc060ce16210b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b10575bc060ce16210b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:35,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b12575bc060ce16210e HTTP/1.1" 502 150 2025-12-17 23:51:35,868 - lava_callback - INFO - Starting async job submit for node 69409b31575bc060ce162161 2025-12-17 23:51:35,868 - lava_callback - ERROR - Error processing callback for node 69409b12575bc060ce16210e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b12575bc060ce16210e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b12575bc060ce16210e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b12575bc060ce16210e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:35,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:35,871 - lava_callback - INFO - Starting async job submit for node 6942a5e81b61f1c6005fd3d5 2025-12-17 23:51:35,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:35,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b31575bc060ce162161 HTTP/1.1" 502 150 2025-12-17 23:51:35,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b31575bc060ce162161'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:35,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b31575bc060ce162161 2025-12-17 23:51:35,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942a5e81b61f1c6005fd3d5 HTTP/1.1" 502 150 2025-12-17 23:51:35,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942a5e81b61f1c6005fd3d5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:35,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942a5e81b61f1c6005fd3d5 2025-12-17 23:51:35,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b31575bc060ce162161 HTTP/1.1" 502 150 2025-12-17 23:51:35,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b31575bc060ce162161'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:35,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942a5e81b61f1c6005fd3d5 HTTP/1.1" 502 150 2025-12-17 23:51:35,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942a5e81b61f1c6005fd3d5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43098 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:51:36,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b13575bc060ce162111 2025-12-17 23:51:36,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b13575bc060ce162111 HTTP/1.1" 502 150 2025-12-17 23:51:36,640 - lava_callback - ERROR - Error processing callback for node 69409b13575bc060ce162111: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b13575bc060ce162111 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b13575bc060ce162111 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b13575bc060ce162111 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:36,643 - lava_callback - INFO - Starting async job submit for node 69409c45575bc060ce162563 2025-12-17 23:51:36,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:36,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c45575bc060ce162563 HTTP/1.1" 502 150 2025-12-17 23:51:36,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c45575bc060ce162563'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:36,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c45575bc060ce162563 2025-12-17 23:51:36,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c45575bc060ce162563 HTTP/1.1" 502 150 2025-12-17 23:51:36,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c45575bc060ce162563'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:36,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b2f575bc060ce16215e 2025-12-17 23:51:36,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2f575bc060ce16215e HTTP/1.1" 502 150 2025-12-17 23:51:36,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b2f575bc060ce16215e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:37,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b2d575bc060ce162157 2025-12-17 23:51:37,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2d575bc060ce162157 HTTP/1.1" 502 150 2025-12-17 23:51:37,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b2d575bc060ce162157'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:37,798 - lava_callback - INFO - Received callback for node 69409ef5575bc060ce162ab7 INFO: 172.21.0.1:43106 - "POST /node/69409ef5575bc060ce162ab7 HTTP/1.0" 202 Accepted 2025-12-17 23:51:37,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b31575bc060ce162161 2025-12-17 23:51:37,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942a5e81b61f1c6005fd3d5 2025-12-17 23:51:37,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b31575bc060ce162161 HTTP/1.1" 502 150 2025-12-17 23:51:37,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b31575bc060ce162161'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:37,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942a5e81b61f1c6005fd3d5 HTTP/1.1" 502 150 2025-12-17 23:51:37,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942a5e81b61f1c6005fd3d5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:38,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa7575bc060ce16200e HTTP/1.1" 502 150 2025-12-17 23:51:38,160 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409aa7575bc060ce16200e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:38,467 - lava_callback - INFO - Received callback for node 69409f00575bc060ce162abc INFO: 172.21.0.1:43116 - "POST /node/69409f00575bc060ce162abc HTTP/1.0" 202 Accepted 2025-12-17 23:51:38,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c45575bc060ce162563 2025-12-17 23:51:38,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c45575bc060ce162563 HTTP/1.1" 502 150 2025-12-17 23:51:38,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c45575bc060ce162563'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:38,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b28575bc060ce162145 2025-12-17 23:51:38,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b28575bc060ce162145 HTTP/1.1" 502 150 2025-12-17 23:51:38,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b28575bc060ce162145'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:39,144 - lava_callback - INFO - Received callback for node 69409f02575bc060ce162ac3 INFO: 172.21.0.1:43124 - "POST /node/69409f02575bc060ce162ac3 HTTP/1.0" 202 Accepted 2025-12-17 23:51:39,807 - lava_callback - INFO - Received callback for node 69409f04575bc060ce162ac6 INFO: 172.21.0.1:59488 - "POST /node/69409f04575bc060ce162ac6 HTTP/1.0" 202 Accepted 2025-12-17 23:51:40,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b2f575bc060ce16215e 2025-12-17 23:51:40,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2f575bc060ce16215e HTTP/1.1" 502 150 2025-12-17 23:51:40,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b2f575bc060ce16215e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:41,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b31575bc060ce162161 2025-12-17 23:51:41,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942a5e81b61f1c6005fd3d5 2025-12-17 23:51:41,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b31575bc060ce162161 HTTP/1.1" 502 150 2025-12-17 23:51:41,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b31575bc060ce162161'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:41,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942a5e81b61f1c6005fd3d5 HTTP/1.1" 502 150 2025-12-17 23:51:41,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942a5e81b61f1c6005fd3d5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:42,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c45575bc060ce162563 2025-12-17 23:51:42,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c45575bc060ce162563 HTTP/1.1" 502 150 2025-12-17 23:51:42,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c45575bc060ce162563'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:42,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b2a575bc060ce162148 2025-12-17 23:51:42,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2a575bc060ce162148 HTTP/1.1" 502 150 2025-12-17 23:51:42,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b2a575bc060ce162148'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:43,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b15575bc060ce162114 2025-12-17 23:51:43,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b15575bc060ce162114 HTTP/1.1" 502 150 2025-12-17 23:51:43,692 - lava_callback - ERROR - Error processing callback for node 69409b15575bc060ce162114: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b15575bc060ce162114 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b15575bc060ce162114 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b15575bc060ce162114 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:43,694 - lava_callback - INFO - Starting async job submit for node 69409b32575bc060ce162164 2025-12-17 23:51:43,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:43,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b32575bc060ce162164 HTTP/1.1" 502 150 2025-12-17 23:51:43,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b32575bc060ce162164'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:43,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b32575bc060ce162164 2025-12-17 23:51:43,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b32575bc060ce162164 HTTP/1.1" 502 150 2025-12-17 23:51:43,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b32575bc060ce162164'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:45,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b2d575bc060ce162157 2025-12-17 23:51:45,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2d575bc060ce162157 HTTP/1.1" 502 150 2025-12-17 23:51:45,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b2d575bc060ce162157'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:45,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b32575bc060ce162164 2025-12-17 23:51:45,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b32575bc060ce162164 HTTP/1.1" 502 150 2025-12-17 23:51:45,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b32575bc060ce162164'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:46,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b27575bc060ce162142 HTTP/1.1" 502 150 2025-12-17 23:51:46,352 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b27575bc060ce162142'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59504 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:51:47,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b0f575bc060ce162108 2025-12-17 23:51:47,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b0f575bc060ce162108 HTTP/1.1" 502 150 2025-12-17 23:51:47,482 - lava_callback - ERROR - Error processing callback for node 69409b0f575bc060ce162108: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b0f575bc060ce162108 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b0f575bc060ce162108 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b0f575bc060ce162108 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:47,484 - lava_callback - INFO - Starting async job submit for node 69409b36575bc060ce16216e 2025-12-17 23:51:47,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:47,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b36575bc060ce16216e HTTP/1.1" 502 150 2025-12-17 23:51:47,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b36575bc060ce16216e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:47,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b36575bc060ce16216e 2025-12-17 23:51:47,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b36575bc060ce16216e HTTP/1.1" 502 150 2025-12-17 23:51:47,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b36575bc060ce16216e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:47,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b20575bc060ce162132 2025-12-17 23:51:47,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b20575bc060ce162132 HTTP/1.1" 502 150 2025-12-17 23:51:47,579 - lava_callback - ERROR - Error processing callback for node 69409b20575bc060ce162132: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b20575bc060ce162132 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b20575bc060ce162132 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b20575bc060ce162132 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:47,580 - lava_callback - INFO - Starting async job submit for node 69409b3a575bc060ce162174 2025-12-17 23:51:47,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:47,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3a575bc060ce162174 HTTP/1.1" 502 150 2025-12-17 23:51:47,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3a575bc060ce162174'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:47,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3a575bc060ce162174 2025-12-17 23:51:47,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3a575bc060ce162174 HTTP/1.1" 502 150 2025-12-17 23:51:47,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3a575bc060ce162174'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:47,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b1d575bc060ce16212f 2025-12-17 23:51:47,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b1d575bc060ce16212f HTTP/1.1" 502 150 2025-12-17 23:51:47,634 - lava_callback - ERROR - Error processing callback for node 69409b1d575bc060ce16212f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b1d575bc060ce16212f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b1d575bc060ce16212f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b1d575bc060ce16212f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:47,636 - lava_callback - INFO - Starting async job submit for node 69409b3e575bc060ce16217a 2025-12-17 23:51:47,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:47,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3e575bc060ce16217a HTTP/1.1" 502 150 2025-12-17 23:51:47,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3e575bc060ce16217a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:47,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3e575bc060ce16217a 2025-12-17 23:51:47,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3e575bc060ce16217a HTTP/1.1" 502 150 2025-12-17 23:51:47,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3e575bc060ce16217a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:47,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b21575bc060ce162135 2025-12-17 23:51:47,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b21575bc060ce162135 HTTP/1.1" 502 150 2025-12-17 23:51:47,678 - lava_callback - ERROR - Error processing callback for node 69409b21575bc060ce162135: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b21575bc060ce162135 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b21575bc060ce162135 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b21575bc060ce162135 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:47,680 - lava_callback - INFO - Starting async job submit for node 69409b41575bc060ce162180 2025-12-17 23:51:47,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:47,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b41575bc060ce162180 HTTP/1.1" 502 150 2025-12-17 23:51:47,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b41575bc060ce162180'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:47,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b41575bc060ce162180 2025-12-17 23:51:47,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b41575bc060ce162180 HTTP/1.1" 502 150 2025-12-17 23:51:47,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b41575bc060ce162180'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:48,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b2f575bc060ce16215e 2025-12-17 23:51:48,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2f575bc060ce16215e HTTP/1.1" 502 150 2025-12-17 23:51:48,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b2f575bc060ce16215e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:49,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b36575bc060ce16216e 2025-12-17 23:51:49,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b36575bc060ce16216e HTTP/1.1" 502 150 2025-12-17 23:51:49,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b36575bc060ce16216e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:49,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3a575bc060ce162174 2025-12-17 23:51:49,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3a575bc060ce162174 HTTP/1.1" 502 150 2025-12-17 23:51:49,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3a575bc060ce162174'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:49,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3e575bc060ce16217a 2025-12-17 23:51:49,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3e575bc060ce16217a HTTP/1.1" 502 150 2025-12-17 23:51:49,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3e575bc060ce16217a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:49,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b41575bc060ce162180 2025-12-17 23:51:49,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b41575bc060ce162180 HTTP/1.1" 502 150 2025-12-17 23:51:49,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b41575bc060ce162180'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:49,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b32575bc060ce162164 2025-12-17 23:51:49,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b32575bc060ce162164 HTTP/1.1" 502 150 2025-12-17 23:51:49,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b32575bc060ce162164'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:49,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b31575bc060ce162161 2025-12-17 23:51:49,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942a5e81b61f1c6005fd3d5 2025-12-17 23:51:49,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b31575bc060ce162161 HTTP/1.1" 502 150 2025-12-17 23:51:49,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b31575bc060ce162161'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:49,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942a5e81b61f1c6005fd3d5 HTTP/1.1" 502 150 2025-12-17 23:51:49,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942a5e81b61f1c6005fd3d5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:50,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c45575bc060ce162563 2025-12-17 23:51:50,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c45575bc060ce162563 HTTP/1.1" 502 150 2025-12-17 23:51:50,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c45575bc060ce162563'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:50,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b23575bc060ce16213c 2025-12-17 23:51:50,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b23575bc060ce16213c HTTP/1.1" 502 150 2025-12-17 23:51:50,772 - lava_callback - ERROR - Error processing callback for node 69409b23575bc060ce16213c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b23575bc060ce16213c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b23575bc060ce16213c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b23575bc060ce16213c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:50,774 - lava_callback - INFO - Starting async job submit for node 69409b34575bc060ce162167 2025-12-17 23:51:50,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:51,049 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b25575bc060ce16213f 2025-12-17 23:51:51,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b25575bc060ce16213f HTTP/1.1" 502 150 2025-12-17 23:51:51,050 - lava_callback - ERROR - Error processing callback for node 69409b25575bc060ce16213f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b25575bc060ce16213f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b25575bc060ce16213f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b25575bc060ce16213f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:51,052 - lava_callback - INFO - Starting async job submit for node 69409b38575bc060ce162171 2025-12-17 23:51:51,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:51,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b38575bc060ce162171 HTTP/1.1" 502 150 2025-12-17 23:51:51,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b38575bc060ce162171'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:51,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b38575bc060ce162171 2025-12-17 23:51:51,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b38575bc060ce162171 HTTP/1.1" 502 150 2025-12-17 23:51:51,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b38575bc060ce162171'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:53,059 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b38575bc060ce162171 2025-12-17 23:51:53,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b38575bc060ce162171 HTTP/1.1" 502 150 2025-12-17 23:51:53,060 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b38575bc060ce162171'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:53,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b36575bc060ce16216e 2025-12-17 23:51:53,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b36575bc060ce16216e HTTP/1.1" 502 150 2025-12-17 23:51:53,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b36575bc060ce16216e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:53,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3a575bc060ce162174 2025-12-17 23:51:53,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3a575bc060ce162174 HTTP/1.1" 502 150 2025-12-17 23:51:53,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3a575bc060ce162174'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:53,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3e575bc060ce16217a 2025-12-17 23:51:53,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3e575bc060ce16217a HTTP/1.1" 502 150 2025-12-17 23:51:53,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3e575bc060ce16217a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:53,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b41575bc060ce162180 2025-12-17 23:51:53,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b41575bc060ce162180 HTTP/1.1" 502 150 2025-12-17 23:51:53,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b41575bc060ce162180'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:54,161 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409aa7575bc060ce16200e 2025-12-17 23:51:54,353 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b27575bc060ce162142 2025-12-17 23:51:54,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b27575bc060ce162142 HTTP/1.1" 502 150 2025-12-17 23:51:54,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b27575bc060ce162142'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:54,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b34575bc060ce162167 HTTP/1.1" 502 150 2025-12-17 23:51:54,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b34575bc060ce162167'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:54,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b34575bc060ce162167 2025-12-17 23:51:54,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b34575bc060ce162167 HTTP/1.1" 502 150 2025-12-17 23:51:54,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b34575bc060ce162167'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:54,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b28575bc060ce162145 2025-12-17 23:51:54,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b28575bc060ce162145 HTTP/1.1" 502 150 2025-12-17 23:51:54,936 - lava_callback - ERROR - Error processing callback for node 69409b28575bc060ce162145: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b28575bc060ce162145 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b28575bc060ce162145 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b28575bc060ce162145 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:54,938 - lava_callback - INFO - Starting async job submit for node 69409b3c575bc060ce162177 2025-12-17 23:51:54,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:54,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3c575bc060ce162177 HTTP/1.1" 502 150 2025-12-17 23:51:54,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3c575bc060ce162177'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:54,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3c575bc060ce162177 2025-12-17 23:51:54,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3c575bc060ce162177 HTTP/1.1" 502 150 2025-12-17 23:51:54,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3c575bc060ce162177'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42010 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:51:56,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b34575bc060ce162167 2025-12-17 23:51:56,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b34575bc060ce162167 HTTP/1.1" 502 150 2025-12-17 23:51:56,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b34575bc060ce162167'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:56,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3c575bc060ce162177 2025-12-17 23:51:56,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3c575bc060ce162177 HTTP/1.1" 502 150 2025-12-17 23:51:56,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3c575bc060ce162177'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:57,061 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b38575bc060ce162171 2025-12-17 23:51:57,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b38575bc060ce162171 HTTP/1.1" 502 150 2025-12-17 23:51:57,062 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b38575bc060ce162171'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:57,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b32575bc060ce162164 2025-12-17 23:51:57,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b32575bc060ce162164 HTTP/1.1" 502 150 2025-12-17 23:51:57,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b32575bc060ce162164'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:58,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b2a575bc060ce162148 2025-12-17 23:51:58,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2a575bc060ce162148 HTTP/1.1" 502 150 2025-12-17 23:51:58,953 - lava_callback - ERROR - Error processing callback for node 69409b2a575bc060ce162148: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b2a575bc060ce162148 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b2a575bc060ce162148 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b2a575bc060ce162148 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:51:58,955 - lava_callback - INFO - Starting async job submit for node 69409b3f575bc060ce16217d 2025-12-17 23:51:58,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:51:58,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3f575bc060ce16217d HTTP/1.1" 502 150 2025-12-17 23:51:58,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3f575bc060ce16217d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:51:58,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3f575bc060ce16217d 2025-12-17 23:51:58,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3f575bc060ce16217d HTTP/1.1" 502 150 2025-12-17 23:51:58,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3f575bc060ce16217d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:00,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b34575bc060ce162167 2025-12-17 23:52:00,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b34575bc060ce162167 HTTP/1.1" 502 150 2025-12-17 23:52:00,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b34575bc060ce162167'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:00,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3c575bc060ce162177 2025-12-17 23:52:00,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3c575bc060ce162177 HTTP/1.1" 502 150 2025-12-17 23:52:00,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3c575bc060ce162177'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:00,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3f575bc060ce16217d 2025-12-17 23:52:00,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3f575bc060ce16217d HTTP/1.1" 502 150 2025-12-17 23:52:00,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3f575bc060ce16217d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:01,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b36575bc060ce16216e 2025-12-17 23:52:01,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b36575bc060ce16216e HTTP/1.1" 502 150 2025-12-17 23:52:01,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b36575bc060ce16216e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:01,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3a575bc060ce162174 2025-12-17 23:52:01,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3a575bc060ce162174 HTTP/1.1" 502 150 2025-12-17 23:52:01,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3a575bc060ce162174'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:01,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3e575bc060ce16217a 2025-12-17 23:52:01,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3e575bc060ce16217a HTTP/1.1" 502 150 2025-12-17 23:52:01,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3e575bc060ce16217a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:01,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b2d575bc060ce162157 2025-12-17 23:52:01,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2d575bc060ce162157 HTTP/1.1" 502 150 2025-12-17 23:52:01,660 - lava_callback - ERROR - Error processing callback for node 69409b2d575bc060ce162157: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b2d575bc060ce162157 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b2d575bc060ce162157 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b2d575bc060ce162157 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:01,662 - lava_callback - INFO - Starting async job submit for node 69409b42575bc060ce162183 2025-12-17 23:52:01,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:01,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b42575bc060ce162183 HTTP/1.1" 502 150 2025-12-17 23:52:01,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b42575bc060ce162183'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:01,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b42575bc060ce162183 2025-12-17 23:52:01,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b42575bc060ce162183 HTTP/1.1" 502 150 2025-12-17 23:52:01,669 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b42575bc060ce162183'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:01,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b41575bc060ce162180 2025-12-17 23:52:01,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b41575bc060ce162180 HTTP/1.1" 502 150 2025-12-17 23:52:01,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b41575bc060ce162180'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47970 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:52:03,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b42575bc060ce162183 2025-12-17 23:52:03,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b42575bc060ce162183 HTTP/1.1" 502 150 2025-12-17 23:52:03,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b42575bc060ce162183'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:04,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b2f575bc060ce16215e 2025-12-17 23:52:04,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b2f575bc060ce16215e HTTP/1.1" 502 150 2025-12-17 23:52:04,721 - lava_callback - ERROR - Error processing callback for node 69409b2f575bc060ce16215e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b2f575bc060ce16215e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b2f575bc060ce16215e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b2f575bc060ce16215e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:04,723 - lava_callback - INFO - Starting async job submit for node 69409b44575bc060ce162186 2025-12-17 23:52:04,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:04,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b44575bc060ce162186 HTTP/1.1" 502 150 2025-12-17 23:52:04,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b44575bc060ce162186'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:04,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b44575bc060ce162186 2025-12-17 23:52:04,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b44575bc060ce162186 HTTP/1.1" 502 150 2025-12-17 23:52:04,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b44575bc060ce162186'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:04,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3f575bc060ce16217d 2025-12-17 23:52:04,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3f575bc060ce16217d HTTP/1.1" 502 150 2025-12-17 23:52:04,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3f575bc060ce16217d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:05,062 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b38575bc060ce162171 2025-12-17 23:52:05,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b38575bc060ce162171 HTTP/1.1" 502 150 2025-12-17 23:52:05,063 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b38575bc060ce162171'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:05,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b31575bc060ce162161 2025-12-17 23:52:05,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942a5e81b61f1c6005fd3d5 2025-12-17 23:52:05,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942a5e81b61f1c6005fd3d5 HTTP/1.1" 502 150 2025-12-17 23:52:05,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b31575bc060ce162161 HTTP/1.1" 502 150 2025-12-17 23:52:05,888 - lava_callback - ERROR - Error processing callback for node 69409b31575bc060ce162161: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b31575bc060ce162161 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b31575bc060ce162161 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b31575bc060ce162161 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:05,891 - lava_callback - INFO - Starting async job submit for node 69409b46575bc060ce162189 2025-12-17 23:52:05,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:05,887 - lava_callback - ERROR - Error processing callback for node 6942a5e81b61f1c6005fd3d5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942a5e81b61f1c6005fd3d5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942a5e81b61f1c6005fd3d5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942a5e81b61f1c6005fd3d5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:05,895 - lava_callback - INFO - Starting async job submit for node 69409b47575bc060ce16218c 2025-12-17 23:52:05,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:05,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b46575bc060ce162189 HTTP/1.1" 502 150 2025-12-17 23:52:05,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b46575bc060ce162189'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:05,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b46575bc060ce162189 2025-12-17 23:52:05,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b46575bc060ce162189 HTTP/1.1" 502 150 2025-12-17 23:52:05,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b46575bc060ce162189'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:05,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b47575bc060ce16218c HTTP/1.1" 502 150 2025-12-17 23:52:05,936 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b47575bc060ce16218c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:05,936 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b47575bc060ce16218c 2025-12-17 23:52:05,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b47575bc060ce16218c HTTP/1.1" 502 150 2025-12-17 23:52:05,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b47575bc060ce16218c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47982 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:52:06,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c45575bc060ce162563 2025-12-17 23:52:06,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c45575bc060ce162563 HTTP/1.1" 502 150 2025-12-17 23:52:06,704 - lava_callback - ERROR - Error processing callback for node 69409c45575bc060ce162563: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c45575bc060ce162563 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c45575bc060ce162563 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c45575bc060ce162563 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:06,707 - lava_callback - INFO - Starting async job submit for node 69409b49575bc060ce16218f 2025-12-17 23:52:06,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:06,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b49575bc060ce16218f HTTP/1.1" 502 150 2025-12-17 23:52:06,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b49575bc060ce16218f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:06,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b49575bc060ce16218f 2025-12-17 23:52:06,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b49575bc060ce16218f HTTP/1.1" 502 150 2025-12-17 23:52:06,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b49575bc060ce16218f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:06,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b44575bc060ce162186 2025-12-17 23:52:06,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b44575bc060ce162186 HTTP/1.1" 502 150 2025-12-17 23:52:06,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b44575bc060ce162186'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:07,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b42575bc060ce162183 2025-12-17 23:52:07,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b42575bc060ce162183 HTTP/1.1" 502 150 2025-12-17 23:52:07,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b42575bc060ce162183'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:07,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b46575bc060ce162189 2025-12-17 23:52:07,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b46575bc060ce162189 HTTP/1.1" 502 150 2025-12-17 23:52:07,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b46575bc060ce162189'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:07,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b47575bc060ce16218c 2025-12-17 23:52:07,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b47575bc060ce16218c HTTP/1.1" 502 150 2025-12-17 23:52:07,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b47575bc060ce16218c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:08,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b49575bc060ce16218f 2025-12-17 23:52:08,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b49575bc060ce16218f HTTP/1.1" 502 150 2025-12-17 23:52:08,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b49575bc060ce16218f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:08,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b34575bc060ce162167 2025-12-17 23:52:08,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b34575bc060ce162167 HTTP/1.1" 502 150 2025-12-17 23:52:08,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b34575bc060ce162167'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:08,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3c575bc060ce162177 2025-12-17 23:52:08,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3c575bc060ce162177 HTTP/1.1" 502 150 2025-12-17 23:52:08,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3c575bc060ce162177'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:10,354 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b27575bc060ce162142 2025-12-17 23:52:10,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b27575bc060ce162142 HTTP/1.1" 502 150 2025-12-17 23:52:10,355 - lava_callback - ERROR - Error processing callback for node 69409b27575bc060ce162142: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b27575bc060ce162142 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b27575bc060ce162142 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b27575bc060ce162142 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:10,357 - lava_callback - INFO - Starting async job submit for node 69409b4b575bc060ce162192 2025-12-17 23:52:10,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:10,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4b575bc060ce162192 HTTP/1.1" 502 150 2025-12-17 23:52:10,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b4b575bc060ce162192'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:10,365 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b4b575bc060ce162192 2025-12-17 23:52:10,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4b575bc060ce162192 HTTP/1.1" 502 150 2025-12-17 23:52:10,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b4b575bc060ce162192'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:10,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b44575bc060ce162186 2025-12-17 23:52:10,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b44575bc060ce162186 HTTP/1.1" 502 150 2025-12-17 23:52:10,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b44575bc060ce162186'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:11,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b46575bc060ce162189 2025-12-17 23:52:11,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b46575bc060ce162189 HTTP/1.1" 502 150 2025-12-17 23:52:11,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b46575bc060ce162189'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:11,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b47575bc060ce16218c 2025-12-17 23:52:11,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b47575bc060ce16218c HTTP/1.1" 502 150 2025-12-17 23:52:11,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b47575bc060ce16218c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:12,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b4b575bc060ce162192 2025-12-17 23:52:12,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4b575bc060ce162192 HTTP/1.1" 502 150 2025-12-17 23:52:12,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b4b575bc060ce162192'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:12,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b49575bc060ce16218f 2025-12-17 23:52:12,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b49575bc060ce16218f HTTP/1.1" 502 150 2025-12-17 23:52:12,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b49575bc060ce16218f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:12,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3f575bc060ce16217d 2025-12-17 23:52:12,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3f575bc060ce16217d HTTP/1.1" 502 150 2025-12-17 23:52:12,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b3f575bc060ce16217d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:13,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b32575bc060ce162164 2025-12-17 23:52:13,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b32575bc060ce162164 HTTP/1.1" 502 150 2025-12-17 23:52:13,706 - lava_callback - ERROR - Error processing callback for node 69409b32575bc060ce162164: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b32575bc060ce162164 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b32575bc060ce162164 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b32575bc060ce162164 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:13,708 - lava_callback - INFO - Starting async job submit for node 69409b4d575bc060ce162195 2025-12-17 23:52:13,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:13,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4d575bc060ce162195 HTTP/1.1" 502 150 2025-12-17 23:52:13,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b4d575bc060ce162195'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:13,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b4d575bc060ce162195 2025-12-17 23:52:13,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4d575bc060ce162195 HTTP/1.1" 502 150 2025-12-17 23:52:13,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b4d575bc060ce162195'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54858 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:52:15,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b42575bc060ce162183 2025-12-17 23:52:15,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b42575bc060ce162183 HTTP/1.1" 502 150 2025-12-17 23:52:15,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b42575bc060ce162183'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:15,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b4d575bc060ce162195 2025-12-17 23:52:15,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4d575bc060ce162195 HTTP/1.1" 502 150 2025-12-17 23:52:15,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b4d575bc060ce162195'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:16,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b4b575bc060ce162192 2025-12-17 23:52:16,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4b575bc060ce162192 HTTP/1.1" 502 150 2025-12-17 23:52:16,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b4b575bc060ce162192'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54866 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:52:17,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b36575bc060ce16216e 2025-12-17 23:52:17,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b36575bc060ce16216e HTTP/1.1" 502 150 2025-12-17 23:52:17,499 - lava_callback - ERROR - Error processing callback for node 69409b36575bc060ce16216e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b36575bc060ce16216e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b36575bc060ce16216e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b36575bc060ce16216e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:17,500 - lava_callback - INFO - Starting async job submit for node 69409b4e575bc060ce162198 2025-12-17 23:52:17,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:17,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4e575bc060ce162198 HTTP/1.1" 502 150 2025-12-17 23:52:17,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b4e575bc060ce162198'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:17,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b4e575bc060ce162198 2025-12-17 23:52:17,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4e575bc060ce162198 HTTP/1.1" 502 150 2025-12-17 23:52:17,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b4e575bc060ce162198'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:17,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3a575bc060ce162174 2025-12-17 23:52:17,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3a575bc060ce162174 HTTP/1.1" 502 150 2025-12-17 23:52:17,601 - lava_callback - ERROR - Error processing callback for node 69409b3a575bc060ce162174: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b3a575bc060ce162174 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b3a575bc060ce162174 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b3a575bc060ce162174 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:17,602 - lava_callback - INFO - Starting async job submit for node 69409b52575bc060ce16219e 2025-12-17 23:52:17,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:17,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b52575bc060ce16219e HTTP/1.1" 502 150 2025-12-17 23:52:17,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b52575bc060ce16219e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:17,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b52575bc060ce16219e 2025-12-17 23:52:17,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b52575bc060ce16219e HTTP/1.1" 502 150 2025-12-17 23:52:17,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b52575bc060ce16219e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:17,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3e575bc060ce16217a 2025-12-17 23:52:17,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3e575bc060ce16217a HTTP/1.1" 502 150 2025-12-17 23:52:17,648 - lava_callback - ERROR - Error processing callback for node 69409b3e575bc060ce16217a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b3e575bc060ce16217a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b3e575bc060ce16217a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b3e575bc060ce16217a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:17,649 - lava_callback - INFO - Starting async job submit for node 694098e0575bc060ce161b98 2025-12-17 23:52:17,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:17,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e0575bc060ce161b98 HTTP/1.1" 502 150 2025-12-17 23:52:17,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e0575bc060ce161b98'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:17,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e0575bc060ce161b98 2025-12-17 23:52:17,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e0575bc060ce161b98 HTTP/1.1" 502 150 2025-12-17 23:52:17,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e0575bc060ce161b98'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:17,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b41575bc060ce162180 2025-12-17 23:52:17,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b41575bc060ce162180 HTTP/1.1" 502 150 2025-12-17 23:52:17,691 - lava_callback - ERROR - Error processing callback for node 69409b41575bc060ce162180: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b41575bc060ce162180 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b41575bc060ce162180 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b41575bc060ce162180 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:17,693 - lava_callback - INFO - Starting async job submit for node 69409b54575bc060ce1621a5 2025-12-17 23:52:17,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:17,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b54575bc060ce1621a5 HTTP/1.1" 502 150 2025-12-17 23:52:17,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b54575bc060ce1621a5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:17,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b54575bc060ce1621a5 2025-12-17 23:52:17,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b54575bc060ce1621a5 HTTP/1.1" 502 150 2025-12-17 23:52:17,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b54575bc060ce1621a5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:17,866 - lava_callback - INFO - Received callback for node 69409f28575bc060ce162ad6 INFO: 172.21.0.1:54872 - "POST /node/69409f28575bc060ce162ad6 HTTP/1.0" 202 Accepted 2025-12-17 23:52:18,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b44575bc060ce162186 2025-12-17 23:52:18,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b44575bc060ce162186 HTTP/1.1" 502 150 2025-12-17 23:52:18,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b44575bc060ce162186'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:19,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b4e575bc060ce162198 2025-12-17 23:52:19,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4e575bc060ce162198 HTTP/1.1" 502 150 2025-12-17 23:52:19,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b4e575bc060ce162198'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:19,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b52575bc060ce16219e 2025-12-17 23:52:19,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b52575bc060ce16219e HTTP/1.1" 502 150 2025-12-17 23:52:19,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b52575bc060ce16219e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:19,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e0575bc060ce161b98 2025-12-17 23:52:19,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e0575bc060ce161b98 HTTP/1.1" 502 150 2025-12-17 23:52:19,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e0575bc060ce161b98'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:19,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b54575bc060ce1621a5 2025-12-17 23:52:19,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b54575bc060ce1621a5 HTTP/1.1" 502 150 2025-12-17 23:52:19,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b54575bc060ce1621a5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:19,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b4d575bc060ce162195 2025-12-17 23:52:19,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4d575bc060ce162195 HTTP/1.1" 502 150 2025-12-17 23:52:19,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b4d575bc060ce162195'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:19,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b46575bc060ce162189 2025-12-17 23:52:19,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b46575bc060ce162189 HTTP/1.1" 502 150 2025-12-17 23:52:19,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b46575bc060ce162189'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:19,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b47575bc060ce16218c 2025-12-17 23:52:19,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b47575bc060ce16218c HTTP/1.1" 502 150 2025-12-17 23:52:19,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b47575bc060ce16218c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:20,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b49575bc060ce16218f 2025-12-17 23:52:20,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b49575bc060ce16218f HTTP/1.1" 502 150 2025-12-17 23:52:20,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b49575bc060ce16218f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:21,064 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b38575bc060ce162171 2025-12-17 23:52:21,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b38575bc060ce162171 HTTP/1.1" 502 150 2025-12-17 23:52:21,065 - lava_callback - ERROR - Error processing callback for node 69409b38575bc060ce162171: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b38575bc060ce162171 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b38575bc060ce162171 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b38575bc060ce162171 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:21,067 - lava_callback - INFO - Starting async job submit for node 69409b5a575bc060ce1621b0 2025-12-17 23:52:21,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:21,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5a575bc060ce1621b0 HTTP/1.1" 502 150 2025-12-17 23:52:21,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5a575bc060ce1621b0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:21,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5a575bc060ce1621b0 2025-12-17 23:52:21,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5a575bc060ce1621b0 HTTP/1.1" 502 150 2025-12-17 23:52:21,074 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5a575bc060ce1621b0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:23,074 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5a575bc060ce1621b0 2025-12-17 23:52:23,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5a575bc060ce1621b0 HTTP/1.1" 502 150 2025-12-17 23:52:23,076 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5a575bc060ce1621b0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:23,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b4e575bc060ce162198 2025-12-17 23:52:23,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4e575bc060ce162198 HTTP/1.1" 502 150 2025-12-17 23:52:23,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b4e575bc060ce162198'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:23,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b52575bc060ce16219e 2025-12-17 23:52:23,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b52575bc060ce16219e HTTP/1.1" 502 150 2025-12-17 23:52:23,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b52575bc060ce16219e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:23,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e0575bc060ce161b98 2025-12-17 23:52:23,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e0575bc060ce161b98 HTTP/1.1" 502 150 2025-12-17 23:52:23,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e0575bc060ce161b98'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:23,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b54575bc060ce1621a5 2025-12-17 23:52:23,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b54575bc060ce1621a5 HTTP/1.1" 502 150 2025-12-17 23:52:23,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b54575bc060ce1621a5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:24,369 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b4b575bc060ce162192 2025-12-17 23:52:24,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4b575bc060ce162192 HTTP/1.1" 502 150 2025-12-17 23:52:24,371 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b4b575bc060ce162192'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:24,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b34575bc060ce162167 2025-12-17 23:52:24,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b34575bc060ce162167 HTTP/1.1" 502 150 2025-12-17 23:52:24,788 - lava_callback - ERROR - Error processing callback for node 69409b34575bc060ce162167: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b34575bc060ce162167 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b34575bc060ce162167 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b34575bc060ce162167 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:24,790 - lava_callback - INFO - Starting async job submit for node 69409c44575bc060ce16255c 2025-12-17 23:52:24,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:24,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c44575bc060ce16255c HTTP/1.1" 502 150 2025-12-17 23:52:24,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c44575bc060ce16255c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:24,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c44575bc060ce16255c 2025-12-17 23:52:24,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c44575bc060ce16255c HTTP/1.1" 502 150 2025-12-17 23:52:24,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c44575bc060ce16255c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:24,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3c575bc060ce162177 2025-12-17 23:52:24,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3c575bc060ce162177 HTTP/1.1" 502 150 2025-12-17 23:52:24,952 - lava_callback - ERROR - Error processing callback for node 69409b3c575bc060ce162177: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b3c575bc060ce162177 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b3c575bc060ce162177 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b3c575bc060ce162177 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:24,953 - lava_callback - INFO - Starting async job submit for node 6942bc911b61f1c6005fe49c 2025-12-17 23:52:24,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:24,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942bc911b61f1c6005fe49c HTTP/1.1" 502 150 2025-12-17 23:52:24,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942bc911b61f1c6005fe49c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:24,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942bc911b61f1c6005fe49c 2025-12-17 23:52:24,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942bc911b61f1c6005fe49c HTTP/1.1" 502 150 2025-12-17 23:52:24,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942bc911b61f1c6005fe49c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:25,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409aa7575bc060ce16200e HTTP/1.1" 502 150 2025-12-17 23:52:25,008 - lava_callback - ERROR - Error processing callback for node 69409aa7575bc060ce16200e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aa7575bc060ce16200e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aa7575bc060ce16200e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409aa7575bc060ce16200e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:25,011 - lava_callback - INFO - Starting async job submit for node 69409b58575bc060ce1621ad 2025-12-17 23:52:25,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:25,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b58575bc060ce1621ad HTTP/1.1" 502 150 2025-12-17 23:52:25,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b58575bc060ce1621ad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:25,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b58575bc060ce1621ad 2025-12-17 23:52:25,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b58575bc060ce1621ad HTTP/1.1" 502 150 2025-12-17 23:52:25,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b58575bc060ce1621ad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48676 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:52:26,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c44575bc060ce16255c 2025-12-17 23:52:26,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c44575bc060ce16255c HTTP/1.1" 502 150 2025-12-17 23:52:26,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c44575bc060ce16255c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:26,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942bc911b61f1c6005fe49c 2025-12-17 23:52:26,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942bc911b61f1c6005fe49c HTTP/1.1" 502 150 2025-12-17 23:52:26,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942bc911b61f1c6005fe49c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:27,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b58575bc060ce1621ad 2025-12-17 23:52:27,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b58575bc060ce1621ad HTTP/1.1" 502 150 2025-12-17 23:52:27,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b58575bc060ce1621ad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:27,076 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5a575bc060ce1621b0 2025-12-17 23:52:27,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5a575bc060ce1621b0 HTTP/1.1" 502 150 2025-12-17 23:52:27,077 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5a575bc060ce1621b0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:27,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b4d575bc060ce162195 2025-12-17 23:52:27,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4d575bc060ce162195 HTTP/1.1" 502 150 2025-12-17 23:52:27,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b4d575bc060ce162195'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:28,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b3f575bc060ce16217d 2025-12-17 23:52:28,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b3f575bc060ce16217d HTTP/1.1" 502 150 2025-12-17 23:52:28,996 - lava_callback - ERROR - Error processing callback for node 69409b3f575bc060ce16217d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b3f575bc060ce16217d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b3f575bc060ce16217d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b3f575bc060ce16217d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:28,998 - lava_callback - INFO - Starting async job submit for node 69409b5b575bc060ce1621b3 2025-12-17 23:52:28,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:29,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5b575bc060ce1621b3 HTTP/1.1" 502 150 2025-12-17 23:52:29,042 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5b575bc060ce1621b3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:29,042 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5b575bc060ce1621b3 2025-12-17 23:52:29,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5b575bc060ce1621b3 HTTP/1.1" 502 150 2025-12-17 23:52:29,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5b575bc060ce1621b3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:30,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c44575bc060ce16255c 2025-12-17 23:52:30,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c44575bc060ce16255c HTTP/1.1" 502 150 2025-12-17 23:52:30,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c44575bc060ce16255c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:30,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942bc911b61f1c6005fe49c 2025-12-17 23:52:30,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942bc911b61f1c6005fe49c HTTP/1.1" 502 150 2025-12-17 23:52:30,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942bc911b61f1c6005fe49c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:31,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b58575bc060ce1621ad 2025-12-17 23:52:31,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b58575bc060ce1621ad HTTP/1.1" 502 150 2025-12-17 23:52:31,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b58575bc060ce1621ad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:31,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5b575bc060ce1621b3 2025-12-17 23:52:31,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5b575bc060ce1621b3 HTTP/1.1" 502 150 2025-12-17 23:52:31,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5b575bc060ce1621b3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:31,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b4e575bc060ce162198 2025-12-17 23:52:31,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4e575bc060ce162198 HTTP/1.1" 502 150 2025-12-17 23:52:31,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b4e575bc060ce162198'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:31,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b52575bc060ce16219e 2025-12-17 23:52:31,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b52575bc060ce16219e HTTP/1.1" 502 150 2025-12-17 23:52:31,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b52575bc060ce16219e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:31,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b42575bc060ce162183 2025-12-17 23:52:31,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b42575bc060ce162183 HTTP/1.1" 502 150 2025-12-17 23:52:31,675 - lava_callback - ERROR - Error processing callback for node 69409b42575bc060ce162183: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b42575bc060ce162183 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b42575bc060ce162183 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b42575bc060ce162183 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:31,676 - lava_callback - INFO - Starting async job submit for node 69409b56575bc060ce1621aa 2025-12-17 23:52:31,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:31,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b56575bc060ce1621aa HTTP/1.1" 502 150 2025-12-17 23:52:31,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b56575bc060ce1621aa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:31,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b56575bc060ce1621aa 2025-12-17 23:52:31,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b56575bc060ce1621aa HTTP/1.1" 502 150 2025-12-17 23:52:31,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b56575bc060ce1621aa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:31,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e0575bc060ce161b98 2025-12-17 23:52:31,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e0575bc060ce161b98 HTTP/1.1" 502 150 2025-12-17 23:52:31,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098e0575bc060ce161b98'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:31,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b54575bc060ce1621a5 2025-12-17 23:52:31,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b54575bc060ce1621a5 HTTP/1.1" 502 150 2025-12-17 23:52:31,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b54575bc060ce1621a5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:33,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b56575bc060ce1621aa 2025-12-17 23:52:34,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b44575bc060ce162186 2025-12-17 23:52:34,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b44575bc060ce162186 HTTP/1.1" 502 150 2025-12-17 23:52:34,741 - lava_callback - ERROR - Error processing callback for node 69409b44575bc060ce162186: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b44575bc060ce162186 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b44575bc060ce162186 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b44575bc060ce162186 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:34,743 - lava_callback - INFO - Starting async job submit for node 69409b5f575bc060ce1621c0 2025-12-17 23:52:34,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:34,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5f575bc060ce1621c0 HTTP/1.1" 502 150 2025-12-17 23:52:34,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5f575bc060ce1621c0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:34,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5f575bc060ce1621c0 2025-12-17 23:52:34,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5f575bc060ce1621c0 HTTP/1.1" 502 150 2025-12-17 23:52:34,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5f575bc060ce1621c0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:35,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5b575bc060ce1621b3 2025-12-17 23:52:35,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5b575bc060ce1621b3 HTTP/1.1" 502 150 2025-12-17 23:52:35,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5b575bc060ce1621b3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:35,077 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5a575bc060ce1621b0 2025-12-17 23:52:35,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5a575bc060ce1621b0 HTTP/1.1" 502 150 2025-12-17 23:52:35,078 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5a575bc060ce1621b0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:35,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b46575bc060ce162189 2025-12-17 23:52:35,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b46575bc060ce162189 HTTP/1.1" 502 150 2025-12-17 23:52:35,931 - lava_callback - ERROR - Error processing callback for node 69409b46575bc060ce162189: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b46575bc060ce162189 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b46575bc060ce162189 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b46575bc060ce162189 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:35,933 - lava_callback - INFO - Starting async job submit for node 69409b5d575bc060ce1621b6 2025-12-17 23:52:35,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:35,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5d575bc060ce1621b6 HTTP/1.1" 502 150 2025-12-17 23:52:35,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5d575bc060ce1621b6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:35,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5d575bc060ce1621b6 2025-12-17 23:52:35,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b47575bc060ce16218c 2025-12-17 23:52:35,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5d575bc060ce1621b6 HTTP/1.1" 502 150 2025-12-17 23:52:35,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5d575bc060ce1621b6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:35,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b47575bc060ce16218c HTTP/1.1" 502 150 2025-12-17 23:52:35,942 - lava_callback - ERROR - Error processing callback for node 69409b47575bc060ce16218c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b47575bc060ce16218c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b47575bc060ce16218c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b47575bc060ce16218c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:35,944 - lava_callback - INFO - Starting async job submit for node 69409b62575bc060ce1621c6 2025-12-17 23:52:35,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:35,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b62575bc060ce1621c6 HTTP/1.1" 502 150 2025-12-17 23:52:35,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b62575bc060ce1621c6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:35,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b62575bc060ce1621c6 2025-12-17 23:52:35,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b62575bc060ce1621c6 HTTP/1.1" 502 150 2025-12-17 23:52:35,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b62575bc060ce1621c6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38574 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:52:36,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b49575bc060ce16218f 2025-12-17 23:52:36,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b49575bc060ce16218f HTTP/1.1" 502 150 2025-12-17 23:52:36,721 - lava_callback - ERROR - Error processing callback for node 69409b49575bc060ce16218f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b49575bc060ce16218f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b49575bc060ce16218f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b49575bc060ce16218f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:36,723 - lava_callback - INFO - Starting async job submit for node 69409b61575bc060ce1621c3 2025-12-17 23:52:36,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:36,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b61575bc060ce1621c3 HTTP/1.1" 502 150 2025-12-17 23:52:36,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b61575bc060ce1621c3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:36,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b61575bc060ce1621c3 2025-12-17 23:52:36,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b61575bc060ce1621c3 HTTP/1.1" 502 150 2025-12-17 23:52:36,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b61575bc060ce1621c3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:36,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5f575bc060ce1621c0 2025-12-17 23:52:36,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5f575bc060ce1621c0 HTTP/1.1" 502 150 2025-12-17 23:52:36,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5f575bc060ce1621c0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:37,843 - lava_callback - INFO - Received callback for node 69409f3c575bc060ce162ae5 INFO: 172.21.0.1:38584 - "POST /node/69409f3c575bc060ce162ae5 HTTP/1.0" 202 Accepted 2025-12-17 23:52:37,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5d575bc060ce1621b6 2025-12-17 23:52:37,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5d575bc060ce1621b6 HTTP/1.1" 502 150 2025-12-17 23:52:37,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5d575bc060ce1621b6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:37,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b62575bc060ce1621c6 2025-12-17 23:52:37,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b62575bc060ce1621c6 HTTP/1.1" 502 150 2025-12-17 23:52:37,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b62575bc060ce1621c6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:38,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b61575bc060ce1621c3 2025-12-17 23:52:38,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b61575bc060ce1621c3 HTTP/1.1" 502 150 2025-12-17 23:52:38,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b61575bc060ce1621c3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:38,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c44575bc060ce16255c 2025-12-17 23:52:38,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c44575bc060ce16255c HTTP/1.1" 502 150 2025-12-17 23:52:38,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c44575bc060ce16255c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:38,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942bc911b61f1c6005fe49c 2025-12-17 23:52:38,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942bc911b61f1c6005fe49c HTTP/1.1" 502 150 2025-12-17 23:52:38,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942bc911b61f1c6005fe49c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:39,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b58575bc060ce1621ad 2025-12-17 23:52:39,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b58575bc060ce1621ad HTTP/1.1" 502 150 2025-12-17 23:52:39,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b58575bc060ce1621ad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:40,371 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b4b575bc060ce162192 2025-12-17 23:52:40,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4b575bc060ce162192 HTTP/1.1" 502 150 2025-12-17 23:52:40,373 - lava_callback - ERROR - Error processing callback for node 69409b4b575bc060ce162192: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b4b575bc060ce162192 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b4b575bc060ce162192 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b4b575bc060ce162192 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:40,375 - lava_callback - INFO - Starting async job submit for node 69409b64575bc060ce1621c9 2025-12-17 23:52:40,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:40,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b64575bc060ce1621c9 HTTP/1.1" 502 150 2025-12-17 23:52:40,382 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b64575bc060ce1621c9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:40,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b64575bc060ce1621c9 2025-12-17 23:52:40,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b64575bc060ce1621c9 HTTP/1.1" 502 150 2025-12-17 23:52:40,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b64575bc060ce1621c9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:40,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5f575bc060ce1621c0 2025-12-17 23:52:40,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5f575bc060ce1621c0 HTTP/1.1" 502 150 2025-12-17 23:52:40,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5f575bc060ce1621c0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:41,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5d575bc060ce1621b6 2025-12-17 23:52:41,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5d575bc060ce1621b6 HTTP/1.1" 502 150 2025-12-17 23:52:41,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5d575bc060ce1621b6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:41,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b62575bc060ce1621c6 2025-12-17 23:52:41,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b62575bc060ce1621c6 HTTP/1.1" 502 150 2025-12-17 23:52:41,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b62575bc060ce1621c6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:42,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b64575bc060ce1621c9 2025-12-17 23:52:42,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b64575bc060ce1621c9 HTTP/1.1" 502 150 2025-12-17 23:52:42,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b64575bc060ce1621c9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:42,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b61575bc060ce1621c3 2025-12-17 23:52:42,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b61575bc060ce1621c3 HTTP/1.1" 502 150 2025-12-17 23:52:42,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b61575bc060ce1621c3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:43,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5b575bc060ce1621b3 2025-12-17 23:52:43,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5b575bc060ce1621b3 HTTP/1.1" 502 150 2025-12-17 23:52:43,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5b575bc060ce1621b3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:43,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b4d575bc060ce162195 2025-12-17 23:52:43,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4d575bc060ce162195 HTTP/1.1" 502 150 2025-12-17 23:52:43,725 - lava_callback - ERROR - Error processing callback for node 69409b4d575bc060ce162195: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b4d575bc060ce162195 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b4d575bc060ce162195 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b4d575bc060ce162195 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:43,727 - lava_callback - INFO - Starting async job submit for node 69409b66575bc060ce1621cc 2025-12-17 23:52:43,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:43,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b66575bc060ce1621cc HTTP/1.1" 502 150 2025-12-17 23:52:43,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b66575bc060ce1621cc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:43,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b66575bc060ce1621cc 2025-12-17 23:52:43,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b66575bc060ce1621cc HTTP/1.1" 502 150 2025-12-17 23:52:43,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b66575bc060ce1621cc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:45,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b66575bc060ce1621cc 2025-12-17 23:52:45,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b66575bc060ce1621cc HTTP/1.1" 502 150 2025-12-17 23:52:45,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b66575bc060ce1621cc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:46,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b64575bc060ce1621c9 2025-12-17 23:52:46,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b64575bc060ce1621c9 HTTP/1.1" 502 150 2025-12-17 23:52:46,385 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b64575bc060ce1621c9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53534 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:52:47,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b4e575bc060ce162198 2025-12-17 23:52:47,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b52575bc060ce16219e 2025-12-17 23:52:47,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b52575bc060ce16219e HTTP/1.1" 502 150 2025-12-17 23:52:47,615 - lava_callback - ERROR - Error processing callback for node 69409b52575bc060ce16219e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b52575bc060ce16219e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b52575bc060ce16219e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b52575bc060ce16219e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:47,617 - lava_callback - INFO - Starting async job submit for node 69409b5e575bc060ce1621b9 2025-12-17 23:52:47,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:47,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5e575bc060ce1621b9 HTTP/1.1" 502 150 2025-12-17 23:52:47,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5e575bc060ce1621b9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:47,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5e575bc060ce1621b9 2025-12-17 23:52:47,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5e575bc060ce1621b9 HTTP/1.1" 502 150 2025-12-17 23:52:47,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5e575bc060ce1621b9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:47,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098e0575bc060ce161b98 2025-12-17 23:52:47,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098e0575bc060ce161b98 HTTP/1.1" 502 150 2025-12-17 23:52:47,689 - lava_callback - ERROR - Error processing callback for node 694098e0575bc060ce161b98: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098e0575bc060ce161b98 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098e0575bc060ce161b98 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098e0575bc060ce161b98 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:47,692 - lava_callback - INFO - Starting async job submit for node 6942c0d71b61f1c6005fea14 2025-12-17 23:52:47,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:47,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b54575bc060ce1621a5 2025-12-17 23:52:47,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b54575bc060ce1621a5 HTTP/1.1" 502 150 2025-12-17 23:52:47,706 - lava_callback - ERROR - Error processing callback for node 69409b54575bc060ce1621a5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b54575bc060ce1621a5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b54575bc060ce1621a5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b54575bc060ce1621a5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:47,708 - lava_callback - INFO - Starting async job submit for node 69409b69575bc060ce1621d7 2025-12-17 23:52:47,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:47,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b69575bc060ce1621d7 HTTP/1.1" 502 150 2025-12-17 23:52:47,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b69575bc060ce1621d7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:47,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b69575bc060ce1621d7 2025-12-17 23:52:47,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b69575bc060ce1621d7 HTTP/1.1" 502 150 2025-12-17 23:52:47,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b69575bc060ce1621d7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:47,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942c0d71b61f1c6005fea14 HTTP/1.1" 502 150 2025-12-17 23:52:47,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942c0d71b61f1c6005fea14'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:47,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942c0d71b61f1c6005fea14 2025-12-17 23:52:47,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942c0d71b61f1c6005fea14 HTTP/1.1" 502 150 2025-12-17 23:52:47,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942c0d71b61f1c6005fea14'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:48,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5f575bc060ce1621c0 2025-12-17 23:52:48,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5f575bc060ce1621c0 HTTP/1.1" 502 150 2025-12-17 23:52:48,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5f575bc060ce1621c0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:49,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b56575bc060ce1621aa HTTP/1.1" 502 150 2025-12-17 23:52:49,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b56575bc060ce1621aa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:49,640 - lava_callback - INFO - Received callback for node 69409f40575bc060ce162af2 INFO: 172.21.0.1:53538 - "POST /node/69409f40575bc060ce162af2 HTTP/1.0" 202 Accepted 2025-12-17 23:52:49,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5e575bc060ce1621b9 2025-12-17 23:52:49,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5e575bc060ce1621b9 HTTP/1.1" 502 150 2025-12-17 23:52:49,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5e575bc060ce1621b9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:49,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b69575bc060ce1621d7 2025-12-17 23:52:49,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b69575bc060ce1621d7 HTTP/1.1" 502 150 2025-12-17 23:52:49,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b69575bc060ce1621d7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:49,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942c0d71b61f1c6005fea14 2025-12-17 23:52:49,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942c0d71b61f1c6005fea14 HTTP/1.1" 502 150 2025-12-17 23:52:49,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942c0d71b61f1c6005fea14'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:49,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b66575bc060ce1621cc 2025-12-17 23:52:49,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b66575bc060ce1621cc HTTP/1.1" 502 150 2025-12-17 23:52:49,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b66575bc060ce1621cc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:49,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5d575bc060ce1621b6 2025-12-17 23:52:49,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5d575bc060ce1621b6 HTTP/1.1" 502 150 2025-12-17 23:52:49,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5d575bc060ce1621b6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:49,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b62575bc060ce1621c6 2025-12-17 23:52:49,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b62575bc060ce1621c6 HTTP/1.1" 502 150 2025-12-17 23:52:49,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b62575bc060ce1621c6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:50,436 - lava_callback - INFO - Received callback for node 69409f49575bc060ce162b06 INFO: 172.21.0.1:57354 - "POST /node/69409f49575bc060ce162b06 HTTP/1.0" 202 Accepted 2025-12-17 23:52:50,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b61575bc060ce1621c3 2025-12-17 23:52:50,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b61575bc060ce1621c3 HTTP/1.1" 502 150 2025-12-17 23:52:50,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b61575bc060ce1621c3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:51,079 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5a575bc060ce1621b0 2025-12-17 23:52:51,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5a575bc060ce1621b0 HTTP/1.1" 502 150 2025-12-17 23:52:51,080 - lava_callback - ERROR - Error processing callback for node 69409b5a575bc060ce1621b0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b5a575bc060ce1621b0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b5a575bc060ce1621b0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b5a575bc060ce1621b0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:51,082 - lava_callback - INFO - Starting async job submit for node 69409b6b575bc060ce1621e4 2025-12-17 23:52:51,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:51,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6b575bc060ce1621e4 HTTP/1.1" 502 150 2025-12-17 23:52:51,089 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b6b575bc060ce1621e4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:51,089 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b6b575bc060ce1621e4 2025-12-17 23:52:51,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6b575bc060ce1621e4 HTTP/1.1" 502 150 2025-12-17 23:52:51,090 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b6b575bc060ce1621e4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57368 - "GET /node/6940258d575bc060ce1540f3%5Cn HTTP/1.0" 405 Method Not Allowed 2025-12-17 23:52:52,552 - lava_callback - INFO - Received callback for node 69409f50575bc060ce162b14 INFO: 172.21.0.1:57374 - "POST /node/69409f50575bc060ce162b14 HTTP/1.0" 202 Accepted 2025-12-17 23:52:53,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b56575bc060ce1621aa 2025-12-17 23:52:53,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b56575bc060ce1621aa HTTP/1.1" 502 150 2025-12-17 23:52:53,085 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b56575bc060ce1621aa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:53,090 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b6b575bc060ce1621e4 2025-12-17 23:52:53,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6b575bc060ce1621e4 HTTP/1.1" 502 150 2025-12-17 23:52:53,091 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b6b575bc060ce1621e4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:53,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5e575bc060ce1621b9 2025-12-17 23:52:53,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5e575bc060ce1621b9 HTTP/1.1" 502 150 2025-12-17 23:52:53,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5e575bc060ce1621b9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:53,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b69575bc060ce1621d7 2025-12-17 23:52:53,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b69575bc060ce1621d7 HTTP/1.1" 502 150 2025-12-17 23:52:53,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b69575bc060ce1621d7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:53,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942c0d71b61f1c6005fea14 2025-12-17 23:52:53,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942c0d71b61f1c6005fea14 HTTP/1.1" 502 150 2025-12-17 23:52:53,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942c0d71b61f1c6005fea14'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:54,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b64575bc060ce1621c9 2025-12-17 23:52:54,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b64575bc060ce1621c9 HTTP/1.1" 502 150 2025-12-17 23:52:54,387 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b64575bc060ce1621c9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:54,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c44575bc060ce16255c 2025-12-17 23:52:54,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c44575bc060ce16255c HTTP/1.1" 502 150 2025-12-17 23:52:54,850 - lava_callback - ERROR - Error processing callback for node 69409c44575bc060ce16255c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c44575bc060ce16255c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c44575bc060ce16255c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c44575bc060ce16255c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:54,852 - lava_callback - INFO - Starting async job submit for node 69409b6d575bc060ce1621e7 2025-12-17 23:52:54,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:54,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6d575bc060ce1621e7 HTTP/1.1" 502 150 2025-12-17 23:52:54,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b6d575bc060ce1621e7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:54,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b6d575bc060ce1621e7 2025-12-17 23:52:54,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6d575bc060ce1621e7 HTTP/1.1" 502 150 2025-12-17 23:52:54,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b6d575bc060ce1621e7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:54,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942bc911b61f1c6005fe49c 2025-12-17 23:52:54,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942bc911b61f1c6005fe49c HTTP/1.1" 502 150 2025-12-17 23:52:54,981 - lava_callback - ERROR - Error processing callback for node 6942bc911b61f1c6005fe49c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942bc911b61f1c6005fe49c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942bc911b61f1c6005fe49c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942bc911b61f1c6005fe49c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:54,984 - lava_callback - INFO - Starting async job submit for node 69409b6f575bc060ce1621ea 2025-12-17 23:52:54,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:54,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6f575bc060ce1621ea HTTP/1.1" 502 150 2025-12-17 23:52:54,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b6f575bc060ce1621ea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:54,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b6f575bc060ce1621ea 2025-12-17 23:52:54,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6f575bc060ce1621ea HTTP/1.1" 502 150 2025-12-17 23:52:55,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b6f575bc060ce1621ea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:55,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b58575bc060ce1621ad 2025-12-17 23:52:55,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b58575bc060ce1621ad HTTP/1.1" 502 150 2025-12-17 23:52:55,028 - lava_callback - ERROR - Error processing callback for node 69409b58575bc060ce1621ad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b58575bc060ce1621ad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b58575bc060ce1621ad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b58575bc060ce1621ad (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:55,030 - lava_callback - INFO - Starting async job submit for node 69409b71575bc060ce1621f1 2025-12-17 23:52:55,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:55,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b71575bc060ce1621f1 HTTP/1.1" 502 150 2025-12-17 23:52:55,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b71575bc060ce1621f1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:55,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b71575bc060ce1621f1 2025-12-17 23:52:55,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b71575bc060ce1621f1 HTTP/1.1" 502 150 2025-12-17 23:52:55,039 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b71575bc060ce1621f1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57382 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:52:56,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b6d575bc060ce1621e7 2025-12-17 23:52:56,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6d575bc060ce1621e7 HTTP/1.1" 502 150 2025-12-17 23:52:56,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b6d575bc060ce1621e7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:57,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b6f575bc060ce1621ea 2025-12-17 23:52:57,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6f575bc060ce1621ea HTTP/1.1" 502 150 2025-12-17 23:52:57,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b6f575bc060ce1621ea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:57,039 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b71575bc060ce1621f1 2025-12-17 23:52:57,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b71575bc060ce1621f1 HTTP/1.1" 502 150 2025-12-17 23:52:57,040 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b71575bc060ce1621f1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:57,091 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b6b575bc060ce1621e4 2025-12-17 23:52:57,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6b575bc060ce1621e4 HTTP/1.1" 502 150 2025-12-17 23:52:57,092 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b6b575bc060ce1621e4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:57,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b66575bc060ce1621cc 2025-12-17 23:52:57,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b66575bc060ce1621cc HTTP/1.1" 502 150 2025-12-17 23:52:57,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b66575bc060ce1621cc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:57,856 - lava_callback - INFO - Received callback for node 69409f47575bc060ce162aff INFO: 172.21.0.1:57394 - "POST /node/69409f47575bc060ce162aff HTTP/1.0" 202 Accepted 2025-12-17 23:52:58,255 - lava_callback - INFO - Received callback for node 69409f52575bc060ce162b17 INFO: 172.21.0.1:57406 - "POST /node/69409f52575bc060ce162b17 HTTP/1.0" 202 Accepted 2025-12-17 23:52:58,524 - lava_callback - INFO - Received callback for node 69409f4a575bc060ce162b09 INFO: 172.21.0.1:57412 - "POST /node/69409f4a575bc060ce162b09 HTTP/1.0" 202 Accepted 2025-12-17 23:52:59,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5b575bc060ce1621b3 2025-12-17 23:52:59,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5b575bc060ce1621b3 HTTP/1.1" 502 150 2025-12-17 23:52:59,048 - lava_callback - ERROR - Error processing callback for node 69409b5b575bc060ce1621b3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b5b575bc060ce1621b3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b5b575bc060ce1621b3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b5b575bc060ce1621b3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:52:59,050 - lava_callback - INFO - Starting async job submit for node 69409b76575bc060ce1621f7 2025-12-17 23:52:59,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:52:59,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b76575bc060ce1621f7 HTTP/1.1" 502 150 2025-12-17 23:52:59,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b76575bc060ce1621f7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:59,059 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b76575bc060ce1621f7 2025-12-17 23:52:59,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b76575bc060ce1621f7 HTTP/1.1" 502 150 2025-12-17 23:52:59,060 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b76575bc060ce1621f7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:52:59,190 - lava_callback - INFO - Received callback for node 69409f4c575bc060ce162b0c INFO: 172.21.0.1:57418 - "POST /node/69409f4c575bc060ce162b0c HTTP/1.0" 202 Accepted 2025-12-17 23:52:59,848 - lava_callback - INFO - Received callback for node 69409f4e575bc060ce162b0f INFO: 172.21.0.1:35032 - "POST /node/69409f4e575bc060ce162b0f HTTP/1.0" 202 Accepted 2025-12-17 23:53:00,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b6d575bc060ce1621e7 2025-12-17 23:53:00,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6d575bc060ce1621e7 HTTP/1.1" 502 150 2025-12-17 23:53:00,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b6d575bc060ce1621e7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:01,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b6f575bc060ce1621ea 2025-12-17 23:53:01,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6f575bc060ce1621ea HTTP/1.1" 502 150 2025-12-17 23:53:01,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b6f575bc060ce1621ea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:01,040 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b71575bc060ce1621f1 2025-12-17 23:53:01,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b71575bc060ce1621f1 HTTP/1.1" 502 150 2025-12-17 23:53:01,043 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b71575bc060ce1621f1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:01,060 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b76575bc060ce1621f7 2025-12-17 23:53:01,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b76575bc060ce1621f7 HTTP/1.1" 502 150 2025-12-17 23:53:01,061 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b76575bc060ce1621f7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:01,085 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b56575bc060ce1621aa 2025-12-17 23:53:01,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b56575bc060ce1621aa HTTP/1.1" 502 150 2025-12-17 23:53:01,086 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b56575bc060ce1621aa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:01,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5e575bc060ce1621b9 2025-12-17 23:53:01,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5e575bc060ce1621b9 HTTP/1.1" 502 150 2025-12-17 23:53:01,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b5e575bc060ce1621b9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:01,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b69575bc060ce1621d7 2025-12-17 23:53:01,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b69575bc060ce1621d7 HTTP/1.1" 502 150 2025-12-17 23:53:01,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b69575bc060ce1621d7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:01,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942c0d71b61f1c6005fea14 2025-12-17 23:53:01,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942c0d71b61f1c6005fea14 HTTP/1.1" 502 150 2025-12-17 23:53:01,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942c0d71b61f1c6005fea14'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:02,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b4e575bc060ce162198 HTTP/1.1" 502 150 2025-12-17 23:53:02,896 - lava_callback - ERROR - Error processing callback for node 69409b4e575bc060ce162198: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b4e575bc060ce162198 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b4e575bc060ce162198 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b4e575bc060ce162198 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:02,898 - lava_callback - INFO - Starting async job submit for node 69409b78575bc060ce1621fa 2025-12-17 23:53:02,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:02,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b78575bc060ce1621fa HTTP/1.1" 502 150 2025-12-17 23:53:02,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b78575bc060ce1621fa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:02,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b78575bc060ce1621fa 2025-12-17 23:53:02,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b78575bc060ce1621fa HTTP/1.1" 502 150 2025-12-17 23:53:02,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b78575bc060ce1621fa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35048 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:53:04,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5f575bc060ce1621c0 2025-12-17 23:53:04,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5f575bc060ce1621c0 HTTP/1.1" 502 150 2025-12-17 23:53:04,761 - lava_callback - ERROR - Error processing callback for node 69409b5f575bc060ce1621c0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b5f575bc060ce1621c0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b5f575bc060ce1621c0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b5f575bc060ce1621c0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:04,763 - lava_callback - INFO - Starting async job submit for node 69409b7a575bc060ce1621fd 2025-12-17 23:53:04,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:04,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7a575bc060ce1621fd HTTP/1.1" 502 150 2025-12-17 23:53:04,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b7a575bc060ce1621fd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:04,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b7a575bc060ce1621fd 2025-12-17 23:53:04,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7a575bc060ce1621fd HTTP/1.1" 502 150 2025-12-17 23:53:04,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b7a575bc060ce1621fd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:04,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b78575bc060ce1621fa 2025-12-17 23:53:04,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b78575bc060ce1621fa HTTP/1.1" 502 150 2025-12-17 23:53:04,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b78575bc060ce1621fa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:05,062 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b76575bc060ce1621f7 2025-12-17 23:53:05,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b76575bc060ce1621f7 HTTP/1.1" 502 150 2025-12-17 23:53:05,063 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b76575bc060ce1621f7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:05,093 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b6b575bc060ce1621e4 2025-12-17 23:53:05,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6b575bc060ce1621e4 HTTP/1.1" 502 150 2025-12-17 23:53:05,094 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b6b575bc060ce1621e4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:05,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5d575bc060ce1621b6 2025-12-17 23:53:05,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5d575bc060ce1621b6 HTTP/1.1" 502 150 2025-12-17 23:53:05,951 - lava_callback - ERROR - Error processing callback for node 69409b5d575bc060ce1621b6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b5d575bc060ce1621b6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b5d575bc060ce1621b6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b5d575bc060ce1621b6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:05,952 - lava_callback - INFO - Starting async job submit for node 69409e6b575bc060ce16297f 2025-12-17 23:53:05,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:05,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b62575bc060ce1621c6 2025-12-17 23:53:05,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b62575bc060ce1621c6 HTTP/1.1" 502 150 2025-12-17 23:53:05,973 - lava_callback - ERROR - Error processing callback for node 69409b62575bc060ce1621c6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b62575bc060ce1621c6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b62575bc060ce1621c6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b62575bc060ce1621c6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:05,974 - lava_callback - INFO - Starting async job submit for node 69409b73575bc060ce1621f4 2025-12-17 23:53:05,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:05,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e6b575bc060ce16297f HTTP/1.1" 502 150 2025-12-17 23:53:05,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e6b575bc060ce16297f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:05,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e6b575bc060ce16297f 2025-12-17 23:53:05,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e6b575bc060ce16297f HTTP/1.1" 502 150 2025-12-17 23:53:05,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e6b575bc060ce16297f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:05,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b73575bc060ce1621f4 HTTP/1.1" 502 150 2025-12-17 23:53:05,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b73575bc060ce1621f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:05,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b73575bc060ce1621f4 2025-12-17 23:53:05,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b73575bc060ce1621f4 HTTP/1.1" 502 150 2025-12-17 23:53:05,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b73575bc060ce1621f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35054 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:53:06,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b61575bc060ce1621c3 2025-12-17 23:53:06,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b61575bc060ce1621c3 HTTP/1.1" 502 150 2025-12-17 23:53:06,745 - lava_callback - ERROR - Error processing callback for node 69409b61575bc060ce1621c3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b61575bc060ce1621c3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b61575bc060ce1621c3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b61575bc060ce1621c3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:06,747 - lava_callback - INFO - Starting async job submit for node 69409b82575bc060ce162217 2025-12-17 23:53:06,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:06,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b82575bc060ce162217 HTTP/1.1" 502 150 2025-12-17 23:53:06,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b82575bc060ce162217'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:06,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b82575bc060ce162217 2025-12-17 23:53:06,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b82575bc060ce162217 HTTP/1.1" 502 150 2025-12-17 23:53:06,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b82575bc060ce162217'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:06,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b7a575bc060ce1621fd 2025-12-17 23:53:06,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7a575bc060ce1621fd HTTP/1.1" 502 150 2025-12-17 23:53:06,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b7a575bc060ce1621fd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:07,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e6b575bc060ce16297f 2025-12-17 23:53:07,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e6b575bc060ce16297f HTTP/1.1" 502 150 2025-12-17 23:53:07,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e6b575bc060ce16297f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:07,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b73575bc060ce1621f4 2025-12-17 23:53:07,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b73575bc060ce1621f4 HTTP/1.1" 502 150 2025-12-17 23:53:07,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b73575bc060ce1621f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:08,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b82575bc060ce162217 2025-12-17 23:53:08,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b82575bc060ce162217 HTTP/1.1" 502 150 2025-12-17 23:53:08,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b82575bc060ce162217'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:08,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b6d575bc060ce1621e7 2025-12-17 23:53:08,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6d575bc060ce1621e7 HTTP/1.1" 502 150 2025-12-17 23:53:08,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b6d575bc060ce1621e7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:08,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b78575bc060ce1621fa 2025-12-17 23:53:08,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b78575bc060ce1621fa HTTP/1.1" 502 150 2025-12-17 23:53:08,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b78575bc060ce1621fa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:09,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b6f575bc060ce1621ea 2025-12-17 23:53:09,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6f575bc060ce1621ea HTTP/1.1" 502 150 2025-12-17 23:53:09,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b6f575bc060ce1621ea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:09,043 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b71575bc060ce1621f1 2025-12-17 23:53:09,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b71575bc060ce1621f1 HTTP/1.1" 502 150 2025-12-17 23:53:09,044 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b71575bc060ce1621f1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:09,440 - lava_callback - INFO - Received callback for node 69409f54575bc060ce162b1a INFO: 172.21.0.1:35068 - "POST /node/69409f54575bc060ce162b1a HTTP/1.0" 202 Accepted 2025-12-17 23:53:10,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b64575bc060ce1621c9 2025-12-17 23:53:10,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b7a575bc060ce1621fd 2025-12-17 23:53:10,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7a575bc060ce1621fd HTTP/1.1" 502 150 2025-12-17 23:53:10,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b7a575bc060ce1621fd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:11,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e6b575bc060ce16297f 2025-12-17 23:53:11,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b73575bc060ce1621f4 2025-12-17 23:53:11,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e6b575bc060ce16297f HTTP/1.1" 502 150 2025-12-17 23:53:11,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e6b575bc060ce16297f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:11,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b73575bc060ce1621f4 HTTP/1.1" 502 150 2025-12-17 23:53:11,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b73575bc060ce1621f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:12,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b82575bc060ce162217 2025-12-17 23:53:12,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b82575bc060ce162217 HTTP/1.1" 502 150 2025-12-17 23:53:12,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b82575bc060ce162217'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:13,063 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b76575bc060ce1621f7 2025-12-17 23:53:13,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b66575bc060ce1621cc 2025-12-17 23:53:13,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b66575bc060ce1621cc HTTP/1.1" 502 150 2025-12-17 23:53:13,760 - lava_callback - ERROR - Error processing callback for node 69409b66575bc060ce1621cc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b66575bc060ce1621cc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b66575bc060ce1621cc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b66575bc060ce1621cc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:13,762 - lava_callback - INFO - Starting async job submit for node 69409b84575bc060ce16221a 2025-12-17 23:53:13,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:13,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b84575bc060ce16221a HTTP/1.1" 502 150 2025-12-17 23:53:13,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b84575bc060ce16221a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:13,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b84575bc060ce16221a 2025-12-17 23:53:13,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b84575bc060ce16221a HTTP/1.1" 502 150 2025-12-17 23:53:13,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b84575bc060ce16221a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52322 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:53:15,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b84575bc060ce16221a 2025-12-17 23:53:15,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b84575bc060ce16221a HTTP/1.1" 502 150 2025-12-17 23:53:15,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b84575bc060ce16221a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52332 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:53:16,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b78575bc060ce1621fa 2025-12-17 23:53:16,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b78575bc060ce1621fa HTTP/1.1" 502 150 2025-12-17 23:53:16,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b78575bc060ce1621fa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:17,086 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b56575bc060ce1621aa 2025-12-17 23:53:17,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b56575bc060ce1621aa HTTP/1.1" 502 150 2025-12-17 23:53:17,087 - lava_callback - ERROR - Error processing callback for node 69409b56575bc060ce1621aa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b56575bc060ce1621aa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b56575bc060ce1621aa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b56575bc060ce1621aa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:17,089 - lava_callback - INFO - Starting async job submit for node 69409b7c575bc060ce162204 2025-12-17 23:53:17,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:17,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7c575bc060ce162204 HTTP/1.1" 502 150 2025-12-17 23:53:17,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b7c575bc060ce162204'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:17,095 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b7c575bc060ce162204 2025-12-17 23:53:17,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7c575bc060ce162204 HTTP/1.1" 502 150 2025-12-17 23:53:17,096 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b7c575bc060ce162204'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:17,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b5e575bc060ce1621b9 2025-12-17 23:53:17,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b5e575bc060ce1621b9 HTTP/1.1" 502 150 2025-12-17 23:53:17,655 - lava_callback - ERROR - Error processing callback for node 69409b5e575bc060ce1621b9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b5e575bc060ce1621b9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b5e575bc060ce1621b9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b5e575bc060ce1621b9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:17,657 - lava_callback - INFO - Starting async job submit for node 69409b7e575bc060ce16220b 2025-12-17 23:53:17,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:17,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7e575bc060ce16220b HTTP/1.1" 502 150 2025-12-17 23:53:17,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b7e575bc060ce16220b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:17,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b7e575bc060ce16220b 2025-12-17 23:53:17,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7e575bc060ce16220b HTTP/1.1" 502 150 2025-12-17 23:53:17,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b7e575bc060ce16220b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:17,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b69575bc060ce1621d7 2025-12-17 23:53:17,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b69575bc060ce1621d7 HTTP/1.1" 502 150 2025-12-17 23:53:17,719 - lava_callback - ERROR - Error processing callback for node 69409b69575bc060ce1621d7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b69575bc060ce1621d7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b69575bc060ce1621d7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b69575bc060ce1621d7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:17,721 - lava_callback - INFO - Starting async job submit for node 69409b85575bc060ce16221d 2025-12-17 23:53:17,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:17,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942c0d71b61f1c6005fea14 2025-12-17 23:53:17,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942c0d71b61f1c6005fea14 HTTP/1.1" 502 150 2025-12-17 23:53:17,729 - lava_callback - ERROR - Error processing callback for node 6942c0d71b61f1c6005fea14: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942c0d71b61f1c6005fea14 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942c0d71b61f1c6005fea14 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942c0d71b61f1c6005fea14 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:17,731 - lava_callback - INFO - Starting async job submit for node 69409b87575bc060ce162220 2025-12-17 23:53:17,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:17,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b85575bc060ce16221d HTTP/1.1" 502 150 2025-12-17 23:53:17,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b85575bc060ce16221d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:17,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b85575bc060ce16221d 2025-12-17 23:53:17,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b85575bc060ce16221d HTTP/1.1" 502 150 2025-12-17 23:53:17,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b85575bc060ce16221d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:17,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b87575bc060ce162220 HTTP/1.1" 502 150 2025-12-17 23:53:17,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b87575bc060ce162220'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:17,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b87575bc060ce162220 2025-12-17 23:53:17,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b87575bc060ce162220 HTTP/1.1" 502 150 2025-12-17 23:53:17,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b87575bc060ce162220'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:17,884 - lava_callback - INFO - Received callback for node 69409f64575bc060ce162b2b INFO: 172.21.0.1:52344 - "POST /node/69409f64575bc060ce162b2b HTTP/1.0" 202 Accepted 2025-12-17 23:53:18,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b7a575bc060ce1621fd 2025-12-17 23:53:18,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7a575bc060ce1621fd HTTP/1.1" 502 150 2025-12-17 23:53:18,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b7a575bc060ce1621fd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:19,096 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b7c575bc060ce162204 2025-12-17 23:53:19,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7c575bc060ce162204 HTTP/1.1" 502 150 2025-12-17 23:53:19,098 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b7c575bc060ce162204'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:19,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b7e575bc060ce16220b 2025-12-17 23:53:19,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7e575bc060ce16220b HTTP/1.1" 502 150 2025-12-17 23:53:19,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b7e575bc060ce16220b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:19,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b85575bc060ce16221d 2025-12-17 23:53:19,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b85575bc060ce16221d HTTP/1.1" 502 150 2025-12-17 23:53:19,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b85575bc060ce16221d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:19,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b87575bc060ce162220 2025-12-17 23:53:19,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b87575bc060ce162220 HTTP/1.1" 502 150 2025-12-17 23:53:19,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b87575bc060ce162220'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:19,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b84575bc060ce16221a 2025-12-17 23:53:19,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b84575bc060ce16221a HTTP/1.1" 502 150 2025-12-17 23:53:19,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b84575bc060ce16221a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:19,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e6b575bc060ce16297f 2025-12-17 23:53:19,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b73575bc060ce1621f4 2025-12-17 23:53:19,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b73575bc060ce1621f4 HTTP/1.1" 502 150 2025-12-17 23:53:19,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b73575bc060ce1621f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:19,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e6b575bc060ce16297f HTTP/1.1" 502 150 2025-12-17 23:53:19,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e6b575bc060ce16297f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:20,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b82575bc060ce162217 2025-12-17 23:53:20,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b82575bc060ce162217 HTTP/1.1" 502 150 2025-12-17 23:53:20,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b82575bc060ce162217'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:21,094 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b6b575bc060ce1621e4 2025-12-17 23:53:23,098 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b7c575bc060ce162204 2025-12-17 23:53:23,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7c575bc060ce162204 HTTP/1.1" 502 150 2025-12-17 23:53:23,099 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b7c575bc060ce162204'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:23,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b7e575bc060ce16220b 2025-12-17 23:53:23,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7e575bc060ce16220b HTTP/1.1" 502 150 2025-12-17 23:53:23,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b7e575bc060ce16220b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:23,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b85575bc060ce16221d 2025-12-17 23:53:23,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b85575bc060ce16221d HTTP/1.1" 502 150 2025-12-17 23:53:23,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b85575bc060ce16221d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:23,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b87575bc060ce162220 2025-12-17 23:53:23,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b87575bc060ce162220 HTTP/1.1" 502 150 2025-12-17 23:53:23,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b87575bc060ce162220'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:24,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b6d575bc060ce1621e7 2025-12-17 23:53:24,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6d575bc060ce1621e7 HTTP/1.1" 502 150 2025-12-17 23:53:24,866 - lava_callback - ERROR - Error processing callback for node 69409b6d575bc060ce1621e7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b6d575bc060ce1621e7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b6d575bc060ce1621e7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b6d575bc060ce1621e7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:24,867 - lava_callback - INFO - Starting async job submit for node 69409b89575bc060ce162223 2025-12-17 23:53:24,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:24,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b89575bc060ce162223 HTTP/1.1" 502 150 2025-12-17 23:53:24,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b89575bc060ce162223'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:24,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b89575bc060ce162223 2025-12-17 23:53:24,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b89575bc060ce162223 HTTP/1.1" 502 150 2025-12-17 23:53:24,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b89575bc060ce162223'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:25,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b6f575bc060ce1621ea 2025-12-17 23:53:25,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6f575bc060ce1621ea HTTP/1.1" 502 150 2025-12-17 23:53:25,005 - lava_callback - ERROR - Error processing callback for node 69409b6f575bc060ce1621ea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b6f575bc060ce1621ea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b6f575bc060ce1621ea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b6f575bc060ce1621ea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:25,007 - lava_callback - INFO - Starting async job submit for node 69409b8b575bc060ce162226 2025-12-17 23:53:25,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:25,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8b575bc060ce162226 HTTP/1.1" 502 150 2025-12-17 23:53:25,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b8b575bc060ce162226'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:25,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b8b575bc060ce162226 2025-12-17 23:53:25,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8b575bc060ce162226 HTTP/1.1" 502 150 2025-12-17 23:53:25,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b8b575bc060ce162226'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:25,044 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b71575bc060ce1621f1 2025-12-17 23:53:25,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b71575bc060ce1621f1 HTTP/1.1" 502 150 2025-12-17 23:53:25,046 - lava_callback - ERROR - Error processing callback for node 69409b71575bc060ce1621f1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b71575bc060ce1621f1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b71575bc060ce1621f1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b71575bc060ce1621f1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:25,047 - lava_callback - INFO - Starting async job submit for node 69409b8c575bc060ce162229 2025-12-17 23:53:25,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:25,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8c575bc060ce162229 HTTP/1.1" 502 150 2025-12-17 23:53:25,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b8c575bc060ce162229'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:25,060 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b8c575bc060ce162229 2025-12-17 23:53:25,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8c575bc060ce162229 HTTP/1.1" 502 150 2025-12-17 23:53:25,060 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b8c575bc060ce162229'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:25,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b64575bc060ce1621c9 HTTP/1.1" 502 150 2025-12-17 23:53:25,680 - lava_callback - ERROR - Error processing callback for node 69409b64575bc060ce1621c9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b64575bc060ce1621c9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b64575bc060ce1621c9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b64575bc060ce1621c9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:25,683 - lava_callback - INFO - Starting async job submit for node 69409b8e575bc060ce16222c 2025-12-17 23:53:25,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:25,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8e575bc060ce16222c HTTP/1.1" 502 150 2025-12-17 23:53:25,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b8e575bc060ce16222c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:25,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b8e575bc060ce16222c 2025-12-17 23:53:25,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8e575bc060ce16222c HTTP/1.1" 502 150 2025-12-17 23:53:25,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b8e575bc060ce16222c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36316 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:53:26,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b89575bc060ce162223 2025-12-17 23:53:26,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b89575bc060ce162223 HTTP/1.1" 502 150 2025-12-17 23:53:26,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b89575bc060ce162223'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:27,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b8b575bc060ce162226 2025-12-17 23:53:27,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8b575bc060ce162226 HTTP/1.1" 502 150 2025-12-17 23:53:27,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b8b575bc060ce162226'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:27,061 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b8c575bc060ce162229 2025-12-17 23:53:27,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8c575bc060ce162229 HTTP/1.1" 502 150 2025-12-17 23:53:27,062 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b8c575bc060ce162229'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:27,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b8e575bc060ce16222c 2025-12-17 23:53:27,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8e575bc060ce16222c HTTP/1.1" 502 150 2025-12-17 23:53:27,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b8e575bc060ce16222c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:27,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b84575bc060ce16221a 2025-12-17 23:53:27,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b84575bc060ce16221a HTTP/1.1" 502 150 2025-12-17 23:53:27,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b84575bc060ce16221a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:28,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b76575bc060ce1621f7 HTTP/1.1" 502 150 2025-12-17 23:53:28,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b76575bc060ce1621f7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:29,500 - lava_callback - INFO - Received callback for node 69413242459ff5fc64d2c2cf INFO: 172.21.0.1:36332 - "POST /node/69413242459ff5fc64d2c2cf HTTP/1.0" 202 Accepted 2025-12-17 23:53:30,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b89575bc060ce162223 2025-12-17 23:53:30,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b89575bc060ce162223 HTTP/1.1" 502 150 2025-12-17 23:53:30,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b89575bc060ce162223'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:31,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b8b575bc060ce162226 2025-12-17 23:53:31,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8b575bc060ce162226 HTTP/1.1" 502 150 2025-12-17 23:53:31,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b8b575bc060ce162226'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:31,062 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b8c575bc060ce162229 2025-12-17 23:53:31,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8c575bc060ce162229 HTTP/1.1" 502 150 2025-12-17 23:53:31,063 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b8c575bc060ce162229'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:31,100 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b7c575bc060ce162204 2025-12-17 23:53:31,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7c575bc060ce162204 HTTP/1.1" 502 150 2025-12-17 23:53:31,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b7c575bc060ce162204'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:31,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b7e575bc060ce16220b 2025-12-17 23:53:31,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7e575bc060ce16220b HTTP/1.1" 502 150 2025-12-17 23:53:31,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b7e575bc060ce16220b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:31,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b8e575bc060ce16222c 2025-12-17 23:53:31,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8e575bc060ce16222c HTTP/1.1" 502 150 2025-12-17 23:53:31,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b8e575bc060ce16222c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:31,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b85575bc060ce16221d 2025-12-17 23:53:31,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b85575bc060ce16221d HTTP/1.1" 502 150 2025-12-17 23:53:31,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b85575bc060ce16221d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:31,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b87575bc060ce162220 2025-12-17 23:53:31,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b87575bc060ce162220 HTTP/1.1" 502 150 2025-12-17 23:53:31,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b87575bc060ce162220'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:32,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b78575bc060ce1621fa 2025-12-17 23:53:32,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b78575bc060ce1621fa HTTP/1.1" 502 150 2025-12-17 23:53:32,916 - lava_callback - ERROR - Error processing callback for node 69409b78575bc060ce1621fa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b78575bc060ce1621fa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b78575bc060ce1621fa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b78575bc060ce1621fa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:32,918 - lava_callback - INFO - Starting async job submit for node 69409b90575bc060ce162231 2025-12-17 23:53:32,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:32,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b90575bc060ce162231 HTTP/1.1" 502 150 2025-12-17 23:53:32,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b90575bc060ce162231'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:32,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b90575bc060ce162231 2025-12-17 23:53:32,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b90575bc060ce162231 HTTP/1.1" 502 150 2025-12-17 23:53:32,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b90575bc060ce162231'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:34,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b7a575bc060ce1621fd 2025-12-17 23:53:34,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7a575bc060ce1621fd HTTP/1.1" 502 150 2025-12-17 23:53:34,775 - lava_callback - ERROR - Error processing callback for node 69409b7a575bc060ce1621fd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b7a575bc060ce1621fd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b7a575bc060ce1621fd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b7a575bc060ce1621fd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:34,777 - lava_callback - INFO - Starting async job submit for node 69409b92575bc060ce162234 2025-12-17 23:53:34,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:34,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b92575bc060ce162234 HTTP/1.1" 502 150 2025-12-17 23:53:34,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b92575bc060ce162234'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:34,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b92575bc060ce162234 2025-12-17 23:53:34,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b92575bc060ce162234 HTTP/1.1" 502 150 2025-12-17 23:53:34,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b92575bc060ce162234'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:34,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b90575bc060ce162231 2025-12-17 23:53:34,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b90575bc060ce162231 HTTP/1.1" 502 150 2025-12-17 23:53:34,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b90575bc060ce162231'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:35,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b73575bc060ce1621f4 2025-12-17 23:53:35,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e6b575bc060ce16297f 2025-12-17 23:53:35,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b73575bc060ce1621f4 HTTP/1.1" 502 150 2025-12-17 23:53:35,992 - lava_callback - ERROR - Error processing callback for node 69409b73575bc060ce1621f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b73575bc060ce1621f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b73575bc060ce1621f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b73575bc060ce1621f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:35,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e6b575bc060ce16297f HTTP/1.1" 502 150 2025-12-17 23:53:35,994 - lava_callback - ERROR - Error processing callback for node 69409e6b575bc060ce16297f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e6b575bc060ce16297f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e6b575bc060ce16297f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e6b575bc060ce16297f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:35,994 - lava_callback - INFO - Starting async job submit for node 69409b94575bc060ce16223b 2025-12-17 23:53:35,996 - lava_callback - INFO - Starting async job submit for node 69409b96575bc060ce16223e 2025-12-17 23:53:35,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:35,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:36,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b96575bc060ce16223e HTTP/1.1" 502 150 2025-12-17 23:53:36,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b96575bc060ce16223e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:36,006 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b96575bc060ce16223e 2025-12-17 23:53:36,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b96575bc060ce16223e HTTP/1.1" 502 150 2025-12-17 23:53:36,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b96575bc060ce16223e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:36,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b6b575bc060ce1621e4 HTTP/1.1" 502 150 2025-12-17 23:53:36,432 - lava_callback - ERROR - Error processing callback for node 69409b6b575bc060ce1621e4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b6b575bc060ce1621e4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b6b575bc060ce1621e4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b6b575bc060ce1621e4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:36,434 - lava_callback - INFO - Starting async job submit for node 69409b98575bc060ce162241 2025-12-17 23:53:36,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:36,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b98575bc060ce162241 HTTP/1.1" 502 150 2025-12-17 23:53:36,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b98575bc060ce162241'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:36,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b98575bc060ce162241 2025-12-17 23:53:36,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b98575bc060ce162241 HTTP/1.1" 502 150 2025-12-17 23:53:36,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b98575bc060ce162241'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56876 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:53:36,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b82575bc060ce162217 2025-12-17 23:53:36,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b82575bc060ce162217 HTTP/1.1" 502 150 2025-12-17 23:53:36,760 - lava_callback - ERROR - Error processing callback for node 69409b82575bc060ce162217: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b82575bc060ce162217 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b82575bc060ce162217 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b82575bc060ce162217 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:36,762 - lava_callback - INFO - Starting async job submit for node 69409b9a575bc060ce162244 2025-12-17 23:53:36,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:36,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9a575bc060ce162244 HTTP/1.1" 502 150 2025-12-17 23:53:36,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b9a575bc060ce162244'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:36,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b9a575bc060ce162244 2025-12-17 23:53:36,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9a575bc060ce162244 HTTP/1.1" 502 150 2025-12-17 23:53:36,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b9a575bc060ce162244'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:36,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b92575bc060ce162234 2025-12-17 23:53:36,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b92575bc060ce162234 HTTP/1.1" 502 150 2025-12-17 23:53:36,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b92575bc060ce162234'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:37,895 - lava_callback - INFO - Received callback for node 69409f7d575bc060ce162b3d INFO: 172.21.0.1:56880 - "POST /node/69409f7d575bc060ce162b3d HTTP/1.0" 202 Accepted 2025-12-17 23:53:38,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b96575bc060ce16223e 2025-12-17 23:53:38,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b96575bc060ce16223e HTTP/1.1" 502 150 2025-12-17 23:53:38,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b96575bc060ce16223e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:38,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b98575bc060ce162241 2025-12-17 23:53:38,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b98575bc060ce162241 HTTP/1.1" 502 150 2025-12-17 23:53:38,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b98575bc060ce162241'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:38,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b9a575bc060ce162244 2025-12-17 23:53:38,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9a575bc060ce162244 HTTP/1.1" 502 150 2025-12-17 23:53:38,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b9a575bc060ce162244'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:38,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b89575bc060ce162223 2025-12-17 23:53:38,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b89575bc060ce162223 HTTP/1.1" 502 150 2025-12-17 23:53:38,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b89575bc060ce162223'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:38,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b90575bc060ce162231 2025-12-17 23:53:38,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b90575bc060ce162231 HTTP/1.1" 502 150 2025-12-17 23:53:38,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b90575bc060ce162231'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:39,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b8b575bc060ce162226 2025-12-17 23:53:39,063 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b8c575bc060ce162229 2025-12-17 23:53:39,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8c575bc060ce162229 HTTP/1.1" 502 150 2025-12-17 23:53:39,064 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b8c575bc060ce162229'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:39,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b8e575bc060ce16222c 2025-12-17 23:53:39,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8e575bc060ce16222c HTTP/1.1" 502 150 2025-12-17 23:53:39,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b8e575bc060ce16222c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:40,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b92575bc060ce162234 2025-12-17 23:53:40,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b92575bc060ce162234 HTTP/1.1" 502 150 2025-12-17 23:53:40,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b92575bc060ce162234'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:42,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b96575bc060ce16223e 2025-12-17 23:53:42,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b96575bc060ce16223e HTTP/1.1" 502 150 2025-12-17 23:53:42,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b96575bc060ce16223e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:42,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b98575bc060ce162241 2025-12-17 23:53:42,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b98575bc060ce162241 HTTP/1.1" 502 150 2025-12-17 23:53:42,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b98575bc060ce162241'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:42,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b9a575bc060ce162244 2025-12-17 23:53:42,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9a575bc060ce162244 HTTP/1.1" 502 150 2025-12-17 23:53:42,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b9a575bc060ce162244'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:43,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b84575bc060ce16221a 2025-12-17 23:53:43,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b84575bc060ce16221a HTTP/1.1" 502 150 2025-12-17 23:53:43,781 - lava_callback - ERROR - Error processing callback for node 69409b84575bc060ce16221a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b84575bc060ce16221a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b84575bc060ce16221a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b84575bc060ce16221a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:43,783 - lava_callback - INFO - Starting async job submit for node 69409b9c575bc060ce162247 2025-12-17 23:53:43,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:43,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9c575bc060ce162247 HTTP/1.1" 502 150 2025-12-17 23:53:43,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b9c575bc060ce162247'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:43,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b9c575bc060ce162247 2025-12-17 23:53:43,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9c575bc060ce162247 HTTP/1.1" 502 150 2025-12-17 23:53:43,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b9c575bc060ce162247'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:44,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b76575bc060ce1621f7 2025-12-17 23:53:44,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b76575bc060ce1621f7 HTTP/1.1" 502 150 2025-12-17 23:53:44,498 - lava_callback - ERROR - Error processing callback for node 69409b76575bc060ce1621f7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b76575bc060ce1621f7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b76575bc060ce1621f7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b76575bc060ce1621f7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:44,500 - lava_callback - INFO - Starting async job submit for node 69409ba0575bc060ce16224d 2025-12-17 23:53:44,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:44,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba0575bc060ce16224d HTTP/1.1" 502 150 2025-12-17 23:53:44,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba0575bc060ce16224d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:44,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba0575bc060ce16224d 2025-12-17 23:53:44,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba0575bc060ce16224d HTTP/1.1" 502 150 2025-12-17 23:53:44,507 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba0575bc060ce16224d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:45,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b9c575bc060ce162247 2025-12-17 23:53:45,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9c575bc060ce162247 HTTP/1.1" 502 150 2025-12-17 23:53:45,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b9c575bc060ce162247'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:46,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba0575bc060ce16224d 2025-12-17 23:53:46,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba0575bc060ce16224d HTTP/1.1" 502 150 2025-12-17 23:53:46,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba0575bc060ce16224d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55460 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:53:46,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b90575bc060ce162231 2025-12-17 23:53:46,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b90575bc060ce162231 HTTP/1.1" 502 150 2025-12-17 23:53:46,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b90575bc060ce162231'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:47,101 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b7c575bc060ce162204 2025-12-17 23:53:47,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7c575bc060ce162204 HTTP/1.1" 502 150 2025-12-17 23:53:47,102 - lava_callback - ERROR - Error processing callback for node 69409b7c575bc060ce162204: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b7c575bc060ce162204 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b7c575bc060ce162204 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b7c575bc060ce162204 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:47,104 - lava_callback - INFO - Starting async job submit for node 69409b9e575bc060ce16224a 2025-12-17 23:53:47,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:47,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9e575bc060ce16224a HTTP/1.1" 502 150 2025-12-17 23:53:47,135 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b9e575bc060ce16224a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:47,136 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b9e575bc060ce16224a 2025-12-17 23:53:47,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9e575bc060ce16224a HTTP/1.1" 502 150 2025-12-17 23:53:47,136 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b9e575bc060ce16224a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:47,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b7e575bc060ce16220b 2025-12-17 23:53:47,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b7e575bc060ce16220b HTTP/1.1" 502 150 2025-12-17 23:53:47,670 - lava_callback - ERROR - Error processing callback for node 69409b7e575bc060ce16220b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b7e575bc060ce16220b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b7e575bc060ce16220b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b7e575bc060ce16220b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:47,672 - lava_callback - INFO - Starting async job submit for node 69409ba3575bc060ce162252 2025-12-17 23:53:47,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:47,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba3575bc060ce162252 HTTP/1.1" 502 150 2025-12-17 23:53:47,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba3575bc060ce162252'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:47,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba3575bc060ce162252 2025-12-17 23:53:47,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba3575bc060ce162252 HTTP/1.1" 502 150 2025-12-17 23:53:47,681 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba3575bc060ce162252'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:47,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b85575bc060ce16221d 2025-12-17 23:53:47,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b85575bc060ce16221d HTTP/1.1" 502 150 2025-12-17 23:53:47,741 - lava_callback - ERROR - Error processing callback for node 69409b85575bc060ce16221d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b85575bc060ce16221d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b85575bc060ce16221d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b85575bc060ce16221d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:47,742 - lava_callback - INFO - Starting async job submit for node 69409ba7575bc060ce16225e 2025-12-17 23:53:47,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:47,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b87575bc060ce162220 2025-12-17 23:53:47,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b87575bc060ce162220 HTTP/1.1" 502 150 2025-12-17 23:53:47,744 - lava_callback - ERROR - Error processing callback for node 69409b87575bc060ce162220: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b87575bc060ce162220 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b87575bc060ce162220 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b87575bc060ce162220 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:47,746 - lava_callback - INFO - Starting async job submit for node 69409ba8575bc060ce162261 2025-12-17 23:53:47,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:47,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba7575bc060ce16225e HTTP/1.1" 502 150 2025-12-17 23:53:47,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba7575bc060ce16225e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:47,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba7575bc060ce16225e 2025-12-17 23:53:47,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba7575bc060ce16225e HTTP/1.1" 502 150 2025-12-17 23:53:47,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba7575bc060ce16225e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:47,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba8575bc060ce162261 HTTP/1.1" 502 150 2025-12-17 23:53:47,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba8575bc060ce162261'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:47,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba8575bc060ce162261 2025-12-17 23:53:47,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba8575bc060ce162261 HTTP/1.1" 502 150 2025-12-17 23:53:47,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba8575bc060ce162261'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:48,750 - lava_callback - INFO - Received callback for node 69409f77575bc060ce162b30 INFO: 172.21.0.1:55464 - "POST /node/69409f77575bc060ce162b30 HTTP/1.0" 202 Accepted 2025-12-17 23:53:48,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b92575bc060ce162234 2025-12-17 23:53:48,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b92575bc060ce162234 HTTP/1.1" 502 150 2025-12-17 23:53:48,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b92575bc060ce162234'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:49,136 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b9e575bc060ce16224a 2025-12-17 23:53:49,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9e575bc060ce16224a HTTP/1.1" 502 150 2025-12-17 23:53:49,138 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b9e575bc060ce16224a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:49,392 - lava_callback - INFO - Received callback for node 69409f80575bc060ce162b44 INFO: 172.21.0.1:55478 - "POST /node/69409f80575bc060ce162b44 HTTP/1.0" 202 Accepted 2025-12-17 23:53:49,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba3575bc060ce162252 2025-12-17 23:53:49,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba3575bc060ce162252 HTTP/1.1" 502 150 2025-12-17 23:53:49,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba3575bc060ce162252'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:49,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba7575bc060ce16225e 2025-12-17 23:53:49,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba7575bc060ce16225e HTTP/1.1" 502 150 2025-12-17 23:53:49,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba7575bc060ce16225e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:49,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba8575bc060ce162261 2025-12-17 23:53:49,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba8575bc060ce162261 HTTP/1.1" 502 150 2025-12-17 23:53:49,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba8575bc060ce162261'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:49,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b9c575bc060ce162247 2025-12-17 23:53:49,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9c575bc060ce162247 HTTP/1.1" 502 150 2025-12-17 23:53:49,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b9c575bc060ce162247'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:50,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b96575bc060ce16223e 2025-12-17 23:53:50,042 - lava_callback - INFO - Received callback for node 69409f84575bc060ce162b4a INFO: 172.21.0.1:49948 - "POST /node/69409f84575bc060ce162b4a HTTP/1.0" 202 Accepted 2025-12-17 23:53:50,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b98575bc060ce162241 2025-12-17 23:53:50,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b98575bc060ce162241 HTTP/1.1" 502 150 2025-12-17 23:53:50,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b98575bc060ce162241'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:50,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba0575bc060ce16224d 2025-12-17 23:53:50,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba0575bc060ce16224d HTTP/1.1" 502 150 2025-12-17 23:53:50,509 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba0575bc060ce16224d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:50,681 - lava_callback - INFO - Received callback for node 69409f86575bc060ce162b4d INFO: 172.21.0.1:49950 - "POST /node/69409f86575bc060ce162b4d HTTP/1.0" 202 Accepted 2025-12-17 23:53:50,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b9a575bc060ce162244 2025-12-17 23:53:50,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9a575bc060ce162244 HTTP/1.1" 502 150 2025-12-17 23:53:50,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b9a575bc060ce162244'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:51,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b94575bc060ce16223b HTTP/1.1" 502 150 2025-12-17 23:53:51,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b94575bc060ce16223b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:51,280 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b94575bc060ce16223b 2025-12-17 23:53:51,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b94575bc060ce16223b HTTP/1.1" 502 150 2025-12-17 23:53:51,281 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b94575bc060ce16223b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:52,829 - lava_callback - INFO - Received callback for node 69409f87575bc060ce162b50 INFO: 172.21.0.1:49966 - "POST /node/69409f87575bc060ce162b50 HTTP/1.0" 202 Accepted 2025-12-17 23:53:53,138 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b9e575bc060ce16224a 2025-12-17 23:53:53,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9e575bc060ce16224a HTTP/1.1" 502 150 2025-12-17 23:53:53,139 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b9e575bc060ce16224a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:53,282 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b94575bc060ce16223b 2025-12-17 23:53:53,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b94575bc060ce16223b HTTP/1.1" 502 150 2025-12-17 23:53:53,283 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b94575bc060ce16223b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:53,474 - lava_callback - INFO - Received callback for node 69409f89575bc060ce162b53 INFO: 172.21.0.1:49968 - "POST /node/69409f89575bc060ce162b53 HTTP/1.0" 202 Accepted 2025-12-17 23:53:53,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba3575bc060ce162252 2025-12-17 23:53:53,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba3575bc060ce162252 HTTP/1.1" 502 150 2025-12-17 23:53:53,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba3575bc060ce162252'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:53,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba7575bc060ce16225e 2025-12-17 23:53:53,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba7575bc060ce16225e HTTP/1.1" 502 150 2025-12-17 23:53:53,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba7575bc060ce16225e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:53,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba8575bc060ce162261 2025-12-17 23:53:53,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba8575bc060ce162261 HTTP/1.1" 502 150 2025-12-17 23:53:53,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba8575bc060ce162261'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:54,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8b575bc060ce162226 HTTP/1.1" 502 150 2025-12-17 23:53:54,352 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b8b575bc060ce162226'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:54,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b89575bc060ce162223 2025-12-17 23:53:54,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b89575bc060ce162223 HTTP/1.1" 502 150 2025-12-17 23:53:54,881 - lava_callback - ERROR - Error processing callback for node 69409b89575bc060ce162223: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b89575bc060ce162223 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b89575bc060ce162223 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b89575bc060ce162223 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:54,883 - lava_callback - INFO - Starting async job submit for node 69409baa575bc060ce162264 2025-12-17 23:53:54,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:54,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409baa575bc060ce162264 HTTP/1.1" 502 150 2025-12-17 23:53:54,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409baa575bc060ce162264'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:54,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409baa575bc060ce162264 2025-12-17 23:53:54,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409baa575bc060ce162264 HTTP/1.1" 502 150 2025-12-17 23:53:54,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409baa575bc060ce162264'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:55,064 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b8c575bc060ce162229 2025-12-17 23:53:55,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8c575bc060ce162229 HTTP/1.1" 502 150 2025-12-17 23:53:55,066 - lava_callback - ERROR - Error processing callback for node 69409b8c575bc060ce162229: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b8c575bc060ce162229 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b8c575bc060ce162229 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b8c575bc060ce162229 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:55,068 - lava_callback - INFO - Starting async job submit for node 69409bab575bc060ce162267 2025-12-17 23:53:55,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:55,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bab575bc060ce162267 HTTP/1.1" 502 150 2025-12-17 23:53:55,097 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bab575bc060ce162267'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:55,097 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bab575bc060ce162267 2025-12-17 23:53:55,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bab575bc060ce162267 HTTP/1.1" 502 150 2025-12-17 23:53:55,098 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bab575bc060ce162267'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:55,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b8e575bc060ce16222c 2025-12-17 23:53:55,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8e575bc060ce16222c HTTP/1.1" 502 150 2025-12-17 23:53:55,697 - lava_callback - ERROR - Error processing callback for node 69409b8e575bc060ce16222c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b8e575bc060ce16222c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b8e575bc060ce16222c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b8e575bc060ce16222c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:53:55,699 - lava_callback - INFO - Starting async job submit for node 69409ba5575bc060ce16225b 2025-12-17 23:53:55,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:53:55,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba5575bc060ce16225b HTTP/1.1" 502 150 2025-12-17 23:53:55,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba5575bc060ce16225b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:55,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba5575bc060ce16225b 2025-12-17 23:53:55,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba5575bc060ce16225b HTTP/1.1" 502 150 2025-12-17 23:53:55,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba5575bc060ce16225b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49970 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:53:56,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409baa575bc060ce162264 2025-12-17 23:53:56,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409baa575bc060ce162264 HTTP/1.1" 502 150 2025-12-17 23:53:56,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409baa575bc060ce162264'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:57,098 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bab575bc060ce162267 2025-12-17 23:53:57,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bab575bc060ce162267 HTTP/1.1" 502 150 2025-12-17 23:53:57,100 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bab575bc060ce162267'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:57,283 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b94575bc060ce16223b 2025-12-17 23:53:57,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b94575bc060ce16223b HTTP/1.1" 502 150 2025-12-17 23:53:57,284 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b94575bc060ce16223b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:57,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba5575bc060ce16225b 2025-12-17 23:53:57,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba5575bc060ce16225b HTTP/1.1" 502 150 2025-12-17 23:53:57,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba5575bc060ce16225b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:57,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b9c575bc060ce162247 2025-12-17 23:53:57,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9c575bc060ce162247 HTTP/1.1" 502 150 2025-12-17 23:53:57,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b9c575bc060ce162247'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:57,893 - lava_callback - INFO - Received callback for node 69409f82575bc060ce162b47 INFO: 172.21.0.1:49974 - "POST /node/69409f82575bc060ce162b47 HTTP/1.0" 202 Accepted 2025-12-17 23:53:58,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba0575bc060ce16224d 2025-12-17 23:53:58,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba0575bc060ce16224d HTTP/1.1" 502 150 2025-12-17 23:53:58,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba0575bc060ce16224d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:53:59,042 - lava_callback - INFO - Received callback for node 6940c5481e3c24e95681d6c1 INFO: 172.21.0.1:49976 - "POST /node/6940c5481e3c24e95681d6c1 HTTP/1.0" 202 Accepted 2025-12-17 23:54:00,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409baa575bc060ce162264 2025-12-17 23:54:00,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409baa575bc060ce162264 HTTP/1.1" 502 150 2025-12-17 23:54:00,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409baa575bc060ce162264'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:01,100 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bab575bc060ce162267 2025-12-17 23:54:01,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bab575bc060ce162267 HTTP/1.1" 502 150 2025-12-17 23:54:01,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bab575bc060ce162267'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:01,139 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b9e575bc060ce16224a 2025-12-17 23:54:01,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9e575bc060ce16224a HTTP/1.1" 502 150 2025-12-17 23:54:01,140 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b9e575bc060ce16224a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:01,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba3575bc060ce162252 2025-12-17 23:54:01,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba3575bc060ce162252 HTTP/1.1" 502 150 2025-12-17 23:54:01,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba3575bc060ce162252'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:01,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba5575bc060ce16225b 2025-12-17 23:54:01,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba5575bc060ce16225b HTTP/1.1" 502 150 2025-12-17 23:54:01,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba5575bc060ce16225b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:01,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba7575bc060ce16225e 2025-12-17 23:54:01,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba7575bc060ce16225e HTTP/1.1" 502 150 2025-12-17 23:54:01,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba7575bc060ce16225e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:01,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba8575bc060ce162261 2025-12-17 23:54:01,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba8575bc060ce162261 HTTP/1.1" 502 150 2025-12-17 23:54:01,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba8575bc060ce162261'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:02,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b90575bc060ce162231 2025-12-17 23:54:02,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b90575bc060ce162231 HTTP/1.1" 502 150 2025-12-17 23:54:02,931 - lava_callback - ERROR - Error processing callback for node 69409b90575bc060ce162231: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b90575bc060ce162231 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b90575bc060ce162231 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b90575bc060ce162231 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:02,934 - lava_callback - INFO - Starting async job submit for node 69409bad575bc060ce16226e 2025-12-17 23:54:02,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:02,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bad575bc060ce16226e HTTP/1.1" 502 150 2025-12-17 23:54:02,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bad575bc060ce16226e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:02,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bad575bc060ce16226e 2025-12-17 23:54:02,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bad575bc060ce16226e HTTP/1.1" 502 150 2025-12-17 23:54:02,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bad575bc060ce16226e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46434 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:54:04,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b92575bc060ce162234 2025-12-17 23:54:04,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b92575bc060ce162234 HTTP/1.1" 502 150 2025-12-17 23:54:04,790 - lava_callback - ERROR - Error processing callback for node 69409b92575bc060ce162234: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b92575bc060ce162234 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b92575bc060ce162234 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b92575bc060ce162234 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:04,792 - lava_callback - INFO - Starting async job submit for node 69409bb0575bc060ce162271 2025-12-17 23:54:04,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:04,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb0575bc060ce162271 HTTP/1.1" 502 150 2025-12-17 23:54:04,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb0575bc060ce162271'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:04,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb0575bc060ce162271 2025-12-17 23:54:04,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb0575bc060ce162271 HTTP/1.1" 502 150 2025-12-17 23:54:04,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb0575bc060ce162271'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:04,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bad575bc060ce16226e 2025-12-17 23:54:04,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bad575bc060ce16226e HTTP/1.1" 502 150 2025-12-17 23:54:04,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bad575bc060ce16226e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:05,284 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b94575bc060ce16223b 2025-12-17 23:54:05,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b96575bc060ce16223e HTTP/1.1" 502 150 2025-12-17 23:54:05,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b96575bc060ce16223e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:06,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b98575bc060ce162241 2025-12-17 23:54:06,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b98575bc060ce162241 HTTP/1.1" 502 150 2025-12-17 23:54:06,469 - lava_callback - ERROR - Error processing callback for node 69409b98575bc060ce162241: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b98575bc060ce162241 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b98575bc060ce162241 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b98575bc060ce162241 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:06,471 - lava_callback - INFO - Starting async job submit for node 69409bb4575bc060ce1623d8 2025-12-17 23:54:06,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:06,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb4575bc060ce1623d8 HTTP/1.1" 502 150 2025-12-17 23:54:06,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb4575bc060ce1623d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:06,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb4575bc060ce1623d8 2025-12-17 23:54:06,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb4575bc060ce1623d8 HTTP/1.1" 502 150 2025-12-17 23:54:06,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb4575bc060ce1623d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46448 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:54:06,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b9a575bc060ce162244 2025-12-17 23:54:06,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9a575bc060ce162244 HTTP/1.1" 502 150 2025-12-17 23:54:06,778 - lava_callback - ERROR - Error processing callback for node 69409b9a575bc060ce162244: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b9a575bc060ce162244 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b9a575bc060ce162244 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b9a575bc060ce162244 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:06,780 - lava_callback - INFO - Starting async job submit for node 69409bb6575bc060ce1623dd 2025-12-17 23:54:06,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:06,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb6575bc060ce1623dd HTTP/1.1" 502 150 2025-12-17 23:54:06,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb6575bc060ce1623dd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:06,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb6575bc060ce1623dd 2025-12-17 23:54:06,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb6575bc060ce1623dd HTTP/1.1" 502 150 2025-12-17 23:54:06,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb6575bc060ce1623dd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:06,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb0575bc060ce162271 2025-12-17 23:54:06,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb0575bc060ce162271 HTTP/1.1" 502 150 2025-12-17 23:54:06,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb0575bc060ce162271'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:08,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb4575bc060ce1623d8 2025-12-17 23:54:08,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb4575bc060ce1623d8 HTTP/1.1" 502 150 2025-12-17 23:54:08,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb4575bc060ce1623d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:08,738 - lava_callback - INFO - Received callback for node 69409f8b575bc060ce162b58 INFO: 172.21.0.1:46454 - "POST /node/69409f8b575bc060ce162b58 HTTP/1.0" 202 Accepted 2025-12-17 23:54:08,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb6575bc060ce1623dd 2025-12-17 23:54:08,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb6575bc060ce1623dd HTTP/1.1" 502 150 2025-12-17 23:54:08,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb6575bc060ce1623dd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:08,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409baa575bc060ce162264 2025-12-17 23:54:08,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409baa575bc060ce162264 HTTP/1.1" 502 150 2025-12-17 23:54:08,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409baa575bc060ce162264'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:08,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bad575bc060ce16226e 2025-12-17 23:54:08,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bad575bc060ce16226e HTTP/1.1" 502 150 2025-12-17 23:54:08,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bad575bc060ce16226e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:09,101 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bab575bc060ce162267 2025-12-17 23:54:09,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bab575bc060ce162267 HTTP/1.1" 502 150 2025-12-17 23:54:09,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bab575bc060ce162267'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:09,411 - lava_callback - INFO - Received callback for node 69409f8d575bc060ce162b5f INFO: 172.21.0.1:46464 - "POST /node/69409f8d575bc060ce162b5f HTTP/1.0" 202 Accepted 2025-12-17 23:54:09,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba5575bc060ce16225b 2025-12-17 23:54:09,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba5575bc060ce16225b HTTP/1.1" 502 150 2025-12-17 23:54:09,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ba5575bc060ce16225b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:10,050 - lava_callback - INFO - Received callback for node 69409f8f575bc060ce162b62 INFO: 172.21.0.1:57916 - "POST /node/69409f8f575bc060ce162b62 HTTP/1.0" 202 Accepted 2025-12-17 23:54:10,353 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b8b575bc060ce162226 2025-12-17 23:54:10,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b8b575bc060ce162226 HTTP/1.1" 502 150 2025-12-17 23:54:10,354 - lava_callback - ERROR - Error processing callback for node 69409b8b575bc060ce162226: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b8b575bc060ce162226 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b8b575bc060ce162226 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b8b575bc060ce162226 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:10,356 - lava_callback - INFO - Starting async job submit for node 69409bb8575bc060ce1623e4 2025-12-17 23:54:10,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:10,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb8575bc060ce1623e4 HTTP/1.1" 502 150 2025-12-17 23:54:10,365 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb8575bc060ce1623e4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:10,366 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb8575bc060ce1623e4 2025-12-17 23:54:10,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb8575bc060ce1623e4 HTTP/1.1" 502 150 2025-12-17 23:54:10,366 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb8575bc060ce1623e4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:10,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb0575bc060ce162271 2025-12-17 23:54:10,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb0575bc060ce162271 HTTP/1.1" 502 150 2025-12-17 23:54:10,815 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb0575bc060ce162271'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:12,367 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb8575bc060ce1623e4 2025-12-17 23:54:12,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb8575bc060ce1623e4 HTTP/1.1" 502 150 2025-12-17 23:54:12,368 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb8575bc060ce1623e4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:12,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb4575bc060ce1623d8 2025-12-17 23:54:12,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb4575bc060ce1623d8 HTTP/1.1" 502 150 2025-12-17 23:54:12,501 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb4575bc060ce1623d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:12,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb6575bc060ce1623dd 2025-12-17 23:54:12,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb6575bc060ce1623dd HTTP/1.1" 502 150 2025-12-17 23:54:12,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb6575bc060ce1623dd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:13,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b9c575bc060ce162247 2025-12-17 23:54:14,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba0575bc060ce16224d 2025-12-17 23:54:14,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba0575bc060ce16224d HTTP/1.1" 502 150 2025-12-17 23:54:14,513 - lava_callback - ERROR - Error processing callback for node 69409ba0575bc060ce16224d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ba0575bc060ce16224d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ba0575bc060ce16224d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ba0575bc060ce16224d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:14,514 - lava_callback - INFO - Starting async job submit for node 69409bba575bc060ce1623e9 2025-12-17 23:54:14,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:14,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bba575bc060ce1623e9 HTTP/1.1" 502 150 2025-12-17 23:54:14,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bba575bc060ce1623e9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:14,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bba575bc060ce1623e9 2025-12-17 23:54:14,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bba575bc060ce1623e9 HTTP/1.1" 502 150 2025-12-17 23:54:14,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bba575bc060ce1623e9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57924 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:54:16,368 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb8575bc060ce1623e4 2025-12-17 23:54:16,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb8575bc060ce1623e4 HTTP/1.1" 502 150 2025-12-17 23:54:16,370 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb8575bc060ce1623e4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:16,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bba575bc060ce1623e9 2025-12-17 23:54:16,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bba575bc060ce1623e9 HTTP/1.1" 502 150 2025-12-17 23:54:16,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bba575bc060ce1623e9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57926 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:54:16,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bad575bc060ce16226e 2025-12-17 23:54:16,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bad575bc060ce16226e HTTP/1.1" 502 150 2025-12-17 23:54:16,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bad575bc060ce16226e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:17,140 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b9e575bc060ce16224a 2025-12-17 23:54:17,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9e575bc060ce16224a HTTP/1.1" 502 150 2025-12-17 23:54:17,142 - lava_callback - ERROR - Error processing callback for node 69409b9e575bc060ce16224a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b9e575bc060ce16224a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b9e575bc060ce16224a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b9e575bc060ce16224a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:17,144 - lava_callback - INFO - Starting async job submit for node 69409bbb575bc060ce1623ec 2025-12-17 23:54:17,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:17,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bbb575bc060ce1623ec HTTP/1.1" 502 150 2025-12-17 23:54:17,152 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bbb575bc060ce1623ec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:17,152 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bbb575bc060ce1623ec 2025-12-17 23:54:17,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bbb575bc060ce1623ec HTTP/1.1" 502 150 2025-12-17 23:54:17,153 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bbb575bc060ce1623ec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:17,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba3575bc060ce162252 2025-12-17 23:54:17,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba3575bc060ce162252 HTTP/1.1" 502 150 2025-12-17 23:54:17,687 - lava_callback - ERROR - Error processing callback for node 69409ba3575bc060ce162252: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ba3575bc060ce162252 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ba3575bc060ce162252 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ba3575bc060ce162252 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:17,689 - lava_callback - INFO - Starting async job submit for node 69409bbe575bc060ce1623f4 2025-12-17 23:54:17,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:17,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bbe575bc060ce1623f4 HTTP/1.1" 502 150 2025-12-17 23:54:17,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bbe575bc060ce1623f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:17,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bbe575bc060ce1623f4 2025-12-17 23:54:17,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bbe575bc060ce1623f4 HTTP/1.1" 502 150 2025-12-17 23:54:17,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bbe575bc060ce1623f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:17,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba7575bc060ce16225e 2025-12-17 23:54:17,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba7575bc060ce16225e HTTP/1.1" 502 150 2025-12-17 23:54:17,756 - lava_callback - ERROR - Error processing callback for node 69409ba7575bc060ce16225e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ba7575bc060ce16225e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ba7575bc060ce16225e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ba7575bc060ce16225e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:17,758 - lava_callback - INFO - Starting async job submit for node 69409c43575bc060ce162555 2025-12-17 23:54:17,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba8575bc060ce162261 2025-12-17 23:54:17,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba8575bc060ce162261 HTTP/1.1" 502 150 2025-12-17 23:54:17,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:17,800 - lava_callback - ERROR - Error processing callback for node 69409ba8575bc060ce162261: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ba8575bc060ce162261 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ba8575bc060ce162261 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ba8575bc060ce162261 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:17,802 - lava_callback - INFO - Starting async job submit for node 69409bc0575bc060ce1623fe 2025-12-17 23:54:17,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:17,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc0575bc060ce1623fe HTTP/1.1" 502 150 2025-12-17 23:54:17,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc0575bc060ce1623fe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:17,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc0575bc060ce1623fe 2025-12-17 23:54:17,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c43575bc060ce162555 HTTP/1.1" 502 150 2025-12-17 23:54:17,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c43575bc060ce162555'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:17,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c43575bc060ce162555 2025-12-17 23:54:17,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc0575bc060ce1623fe HTTP/1.1" 502 150 2025-12-17 23:54:17,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc0575bc060ce1623fe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:17,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c43575bc060ce162555 HTTP/1.1" 502 150 2025-12-17 23:54:17,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c43575bc060ce162555'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:18,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb0575bc060ce162271 2025-12-17 23:54:18,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb0575bc060ce162271 HTTP/1.1" 502 150 2025-12-17 23:54:18,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb0575bc060ce162271'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:19,153 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bbb575bc060ce1623ec 2025-12-17 23:54:19,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bbb575bc060ce1623ec HTTP/1.1" 502 150 2025-12-17 23:54:19,154 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bbb575bc060ce1623ec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:19,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bbe575bc060ce1623f4 2025-12-17 23:54:19,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bbe575bc060ce1623f4 HTTP/1.1" 502 150 2025-12-17 23:54:19,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bbe575bc060ce1623f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:19,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc0575bc060ce1623fe 2025-12-17 23:54:19,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c43575bc060ce162555 2025-12-17 23:54:19,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c43575bc060ce162555 HTTP/1.1" 502 150 2025-12-17 23:54:19,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c43575bc060ce162555'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:19,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc0575bc060ce1623fe HTTP/1.1" 502 150 2025-12-17 23:54:19,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc0575bc060ce1623fe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:20,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb4575bc060ce1623d8 2025-12-17 23:54:20,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb4575bc060ce1623d8 HTTP/1.1" 502 150 2025-12-17 23:54:20,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb4575bc060ce1623d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:20,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bba575bc060ce1623e9 2025-12-17 23:54:20,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bba575bc060ce1623e9 HTTP/1.1" 502 150 2025-12-17 23:54:20,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bba575bc060ce1623e9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:20,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b94575bc060ce16223b HTTP/1.1" 502 150 2025-12-17 23:54:20,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409b94575bc060ce16223b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:20,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb6575bc060ce1623dd 2025-12-17 23:54:20,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb6575bc060ce1623dd HTTP/1.1" 502 150 2025-12-17 23:54:20,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb6575bc060ce1623dd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:21,361 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b96575bc060ce16223e 2025-12-17 23:54:21,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b96575bc060ce16223e HTTP/1.1" 502 150 2025-12-17 23:54:21,362 - lava_callback - ERROR - Error processing callback for node 69409b96575bc060ce16223e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b96575bc060ce16223e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b96575bc060ce16223e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b96575bc060ce16223e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:21,364 - lava_callback - INFO - Starting async job submit for node 69409bc8575bc060ce16240a 2025-12-17 23:54:21,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:21,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc8575bc060ce16240a HTTP/1.1" 502 150 2025-12-17 23:54:21,370 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc8575bc060ce16240a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:21,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc8575bc060ce16240a 2025-12-17 23:54:21,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc8575bc060ce16240a HTTP/1.1" 502 150 2025-12-17 23:54:21,371 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc8575bc060ce16240a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:23,154 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bbb575bc060ce1623ec 2025-12-17 23:54:23,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bbb575bc060ce1623ec HTTP/1.1" 502 150 2025-12-17 23:54:23,156 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bbb575bc060ce1623ec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:23,371 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc8575bc060ce16240a 2025-12-17 23:54:23,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc8575bc060ce16240a HTTP/1.1" 502 150 2025-12-17 23:54:23,373 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc8575bc060ce16240a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:23,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bbe575bc060ce1623f4 2025-12-17 23:54:23,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bbe575bc060ce1623f4 HTTP/1.1" 502 150 2025-12-17 23:54:23,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bbe575bc060ce1623f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:23,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c43575bc060ce162555 2025-12-17 23:54:23,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c43575bc060ce162555 HTTP/1.1" 502 150 2025-12-17 23:54:23,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c43575bc060ce162555'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:23,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc0575bc060ce1623fe 2025-12-17 23:54:23,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc0575bc060ce1623fe HTTP/1.1" 502 150 2025-12-17 23:54:23,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc0575bc060ce1623fe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:24,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb8575bc060ce1623e4 2025-12-17 23:54:24,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb8575bc060ce1623e4 HTTP/1.1" 502 150 2025-12-17 23:54:24,371 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bb8575bc060ce1623e4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:24,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409baa575bc060ce162264 2025-12-17 23:54:24,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409baa575bc060ce162264 HTTP/1.1" 502 150 2025-12-17 23:54:24,896 - lava_callback - ERROR - Error processing callback for node 69409baa575bc060ce162264: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409baa575bc060ce162264 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409baa575bc060ce162264 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409baa575bc060ce162264 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:24,898 - lava_callback - INFO - Starting async job submit for node 69409bc2575bc060ce162401 2025-12-17 23:54:24,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:24,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc2575bc060ce162401 HTTP/1.1" 502 150 2025-12-17 23:54:24,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc2575bc060ce162401'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:24,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc2575bc060ce162401 2025-12-17 23:54:24,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc2575bc060ce162401 HTTP/1.1" 502 150 2025-12-17 23:54:24,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc2575bc060ce162401'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:25,103 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bab575bc060ce162267 2025-12-17 23:54:25,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bab575bc060ce162267 HTTP/1.1" 502 150 2025-12-17 23:54:25,104 - lava_callback - ERROR - Error processing callback for node 69409bab575bc060ce162267: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bab575bc060ce162267 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bab575bc060ce162267 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bab575bc060ce162267 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:25,106 - lava_callback - INFO - Starting async job submit for node 69409bc5575bc060ce162404 2025-12-17 23:54:25,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:25,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc5575bc060ce162404 HTTP/1.1" 502 150 2025-12-17 23:54:25,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc5575bc060ce162404'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:25,112 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc5575bc060ce162404 2025-12-17 23:54:25,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc5575bc060ce162404 HTTP/1.1" 502 150 2025-12-17 23:54:25,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc5575bc060ce162404'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:25,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ba5575bc060ce16225b 2025-12-17 23:54:25,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ba5575bc060ce16225b HTTP/1.1" 502 150 2025-12-17 23:54:25,712 - lava_callback - ERROR - Error processing callback for node 69409ba5575bc060ce16225b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ba5575bc060ce16225b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ba5575bc060ce16225b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ba5575bc060ce16225b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:25,714 - lava_callback - INFO - Starting async job submit for node 69409bc7575bc060ce162407 2025-12-17 23:54:25,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:25,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc7575bc060ce162407 HTTP/1.1" 502 150 2025-12-17 23:54:25,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc7575bc060ce162407'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:25,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc7575bc060ce162407 2025-12-17 23:54:25,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc7575bc060ce162407 HTTP/1.1" 502 150 2025-12-17 23:54:25,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc7575bc060ce162407'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54390 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:54:26,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc2575bc060ce162401 2025-12-17 23:54:26,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc2575bc060ce162401 HTTP/1.1" 502 150 2025-12-17 23:54:26,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc2575bc060ce162401'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:27,113 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc5575bc060ce162404 2025-12-17 23:54:27,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc5575bc060ce162404 HTTP/1.1" 502 150 2025-12-17 23:54:27,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc5575bc060ce162404'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:27,373 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc8575bc060ce16240a 2025-12-17 23:54:27,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc8575bc060ce16240a HTTP/1.1" 502 150 2025-12-17 23:54:27,374 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc8575bc060ce16240a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:27,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc7575bc060ce162407 2025-12-17 23:54:27,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc7575bc060ce162407 HTTP/1.1" 502 150 2025-12-17 23:54:27,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc7575bc060ce162407'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:28,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bba575bc060ce1623e9 2025-12-17 23:54:28,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bba575bc060ce1623e9 HTTP/1.1" 502 150 2025-12-17 23:54:28,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bba575bc060ce1623e9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:28,754 - lava_callback - INFO - Received callback for node 69409f9f575bc060ce162b6c INFO: 172.21.0.1:54392 - "POST /node/69409f9f575bc060ce162b6c HTTP/1.0" 202 Accepted 2025-12-17 23:54:29,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b9c575bc060ce162247 HTTP/1.1" 502 150 2025-12-17 23:54:29,168 - lava_callback - ERROR - Error processing callback for node 69409b9c575bc060ce162247: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b9c575bc060ce162247 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b9c575bc060ce162247 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b9c575bc060ce162247 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:29,170 - lava_callback - INFO - Starting async job submit for node 69409bcb575bc060ce162411 2025-12-17 23:54:29,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:29,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcb575bc060ce162411 HTTP/1.1" 502 150 2025-12-17 23:54:29,177 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bcb575bc060ce162411'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:29,177 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bcb575bc060ce162411 2025-12-17 23:54:29,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcb575bc060ce162411 HTTP/1.1" 502 150 2025-12-17 23:54:29,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bcb575bc060ce162411'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:29,398 - lava_callback - INFO - Received callback for node 69409fa1575bc060ce162b73 INFO: 172.21.0.1:54404 - "POST /node/69409fa1575bc060ce162b73 HTTP/1.0" 202 Accepted 2025-12-17 23:54:30,064 - lava_callback - INFO - Received callback for node 69409fa2575bc060ce162b7a INFO: 172.21.0.1:54004 - "POST /node/69409fa2575bc060ce162b7a HTTP/1.0" 202 Accepted 2025-12-17 23:54:30,730 - lava_callback - INFO - Received callback for node 69409fa3575bc060ce162b7f INFO: 172.21.0.1:54008 - "POST /node/69409fa3575bc060ce162b7f HTTP/1.0" 202 Accepted 2025-12-17 23:54:30,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc2575bc060ce162401 2025-12-17 23:54:30,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc2575bc060ce162401 HTTP/1.1" 502 150 2025-12-17 23:54:30,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc2575bc060ce162401'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:31,115 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc5575bc060ce162404 2025-12-17 23:54:31,156 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bbb575bc060ce1623ec 2025-12-17 23:54:31,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bbb575bc060ce1623ec HTTP/1.1" 502 150 2025-12-17 23:54:31,157 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bbb575bc060ce1623ec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:31,178 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bcb575bc060ce162411 2025-12-17 23:54:31,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcb575bc060ce162411 HTTP/1.1" 502 150 2025-12-17 23:54:31,179 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bcb575bc060ce162411'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:31,386 - lava_callback - INFO - Received callback for node 69409fa5575bc060ce162b82 INFO: 172.21.0.1:54016 - "POST /node/69409fa5575bc060ce162b82 HTTP/1.0" 202 Accepted 2025-12-17 23:54:31,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bbe575bc060ce1623f4 2025-12-17 23:54:31,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bbe575bc060ce1623f4 HTTP/1.1" 502 150 2025-12-17 23:54:31,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bbe575bc060ce1623f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:31,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc7575bc060ce162407 2025-12-17 23:54:31,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc7575bc060ce162407 HTTP/1.1" 502 150 2025-12-17 23:54:31,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc7575bc060ce162407'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:31,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c43575bc060ce162555 2025-12-17 23:54:31,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c43575bc060ce162555 HTTP/1.1" 502 150 2025-12-17 23:54:31,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c43575bc060ce162555'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:31,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc0575bc060ce1623fe 2025-12-17 23:54:31,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc0575bc060ce1623fe HTTP/1.1" 502 150 2025-12-17 23:54:31,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc0575bc060ce1623fe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:32,039 - lava_callback - INFO - Received callback for node 69409fa6575bc060ce162b85 INFO: 172.21.0.1:54032 - "POST /node/69409fa6575bc060ce162b85 HTTP/1.0" 202 Accepted 2025-12-17 23:54:32,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bad575bc060ce16226e 2025-12-17 23:54:32,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bad575bc060ce16226e HTTP/1.1" 502 150 2025-12-17 23:54:32,955 - lava_callback - ERROR - Error processing callback for node 69409bad575bc060ce16226e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bad575bc060ce16226e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bad575bc060ce16226e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bad575bc060ce16226e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:32,958 - lava_callback - INFO - Starting async job submit for node 69409bcd575bc060ce16241a 2025-12-17 23:54:32,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:32,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcd575bc060ce16241a HTTP/1.1" 502 150 2025-12-17 23:54:32,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bcd575bc060ce16241a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:32,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bcd575bc060ce16241a 2025-12-17 23:54:32,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcd575bc060ce16241a HTTP/1.1" 502 150 2025-12-17 23:54:32,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bcd575bc060ce16241a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:34,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb0575bc060ce162271 2025-12-17 23:54:34,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb0575bc060ce162271 HTTP/1.1" 502 150 2025-12-17 23:54:34,818 - lava_callback - ERROR - Error processing callback for node 69409bb0575bc060ce162271: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bb0575bc060ce162271 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bb0575bc060ce162271 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bb0575bc060ce162271 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:34,820 - lava_callback - INFO - Starting async job submit for node 69409bcf575bc060ce16241f 2025-12-17 23:54:34,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:34,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcf575bc060ce16241f HTTP/1.1" 502 150 2025-12-17 23:54:34,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bcf575bc060ce16241f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:34,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bcf575bc060ce16241f 2025-12-17 23:54:34,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcf575bc060ce16241f HTTP/1.1" 502 150 2025-12-17 23:54:34,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bcf575bc060ce16241f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:34,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bcd575bc060ce16241a 2025-12-17 23:54:34,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcd575bc060ce16241a HTTP/1.1" 502 150 2025-12-17 23:54:34,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bcd575bc060ce16241a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:35,179 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bcb575bc060ce162411 2025-12-17 23:54:35,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcb575bc060ce162411 HTTP/1.1" 502 150 2025-12-17 23:54:35,182 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bcb575bc060ce162411'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:35,374 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc8575bc060ce16240a 2025-12-17 23:54:35,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc8575bc060ce16240a HTTP/1.1" 502 150 2025-12-17 23:54:35,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc8575bc060ce16240a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:36,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb4575bc060ce1623d8 2025-12-17 23:54:36,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb4575bc060ce1623d8 HTTP/1.1" 502 150 2025-12-17 23:54:36,505 - lava_callback - ERROR - Error processing callback for node 69409bb4575bc060ce1623d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bb4575bc060ce1623d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bb4575bc060ce1623d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bb4575bc060ce1623d8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:36,506 - lava_callback - INFO - Starting async job submit for node 69409bd1575bc060ce162427 2025-12-17 23:54:36,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:54046 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:54:36,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd1575bc060ce162427 HTTP/1.1" 502 150 2025-12-17 23:54:36,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd1575bc060ce162427'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:36,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd1575bc060ce162427 2025-12-17 23:54:36,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd1575bc060ce162427 HTTP/1.1" 502 150 2025-12-17 23:54:36,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd1575bc060ce162427'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:36,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409b94575bc060ce16223b 2025-12-17 23:54:36,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409b94575bc060ce16223b HTTP/1.1" 502 150 2025-12-17 23:54:36,721 - lava_callback - ERROR - Error processing callback for node 69409b94575bc060ce16223b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b94575bc060ce16223b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b94575bc060ce16223b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409b94575bc060ce16223b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:36,723 - lava_callback - INFO - Starting async job submit for node 69409bd3575bc060ce16242a 2025-12-17 23:54:36,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:36,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd3575bc060ce16242a HTTP/1.1" 502 150 2025-12-17 23:54:36,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd3575bc060ce16242a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:36,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd3575bc060ce16242a 2025-12-17 23:54:36,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd3575bc060ce16242a HTTP/1.1" 502 150 2025-12-17 23:54:36,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd3575bc060ce16242a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:36,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb6575bc060ce1623dd 2025-12-17 23:54:36,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb6575bc060ce1623dd HTTP/1.1" 502 150 2025-12-17 23:54:36,793 - lava_callback - ERROR - Error processing callback for node 69409bb6575bc060ce1623dd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bb6575bc060ce1623dd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bb6575bc060ce1623dd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bb6575bc060ce1623dd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:36,795 - lava_callback - INFO - Starting async job submit for node 69409bd5575bc060ce16242d 2025-12-17 23:54:36,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:36,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd5575bc060ce16242d HTTP/1.1" 502 150 2025-12-17 23:54:36,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd5575bc060ce16242d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:36,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd5575bc060ce16242d 2025-12-17 23:54:36,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd5575bc060ce16242d HTTP/1.1" 502 150 2025-12-17 23:54:36,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd5575bc060ce16242d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:36,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bcf575bc060ce16241f 2025-12-17 23:54:36,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcf575bc060ce16241f HTTP/1.1" 502 150 2025-12-17 23:54:36,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bcf575bc060ce16241f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:38,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd1575bc060ce162427 2025-12-17 23:54:38,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd1575bc060ce162427 HTTP/1.1" 502 150 2025-12-17 23:54:38,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd1575bc060ce162427'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:38,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd3575bc060ce16242a 2025-12-17 23:54:38,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd3575bc060ce16242a HTTP/1.1" 502 150 2025-12-17 23:54:38,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd3575bc060ce16242a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:38,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd5575bc060ce16242d 2025-12-17 23:54:38,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd5575bc060ce16242d HTTP/1.1" 502 150 2025-12-17 23:54:38,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd5575bc060ce16242d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:38,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc2575bc060ce162401 2025-12-17 23:54:38,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc2575bc060ce162401 HTTP/1.1" 502 150 2025-12-17 23:54:38,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc2575bc060ce162401'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:38,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bcd575bc060ce16241a 2025-12-17 23:54:38,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcd575bc060ce16241a HTTP/1.1" 502 150 2025-12-17 23:54:38,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bcd575bc060ce16241a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:39,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc7575bc060ce162407 2025-12-17 23:54:39,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc7575bc060ce162407 HTTP/1.1" 502 150 2025-12-17 23:54:39,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc7575bc060ce162407'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:40,371 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bb8575bc060ce1623e4 2025-12-17 23:54:40,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bb8575bc060ce1623e4 HTTP/1.1" 502 150 2025-12-17 23:54:40,374 - lava_callback - ERROR - Error processing callback for node 69409bb8575bc060ce1623e4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bb8575bc060ce1623e4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bb8575bc060ce1623e4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bb8575bc060ce1623e4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:40,376 - lava_callback - INFO - Starting async job submit for node 69409bd7575bc060ce162434 2025-12-17 23:54:40,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:40,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd7575bc060ce162434 HTTP/1.1" 502 150 2025-12-17 23:54:40,384 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd7575bc060ce162434'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:40,384 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd7575bc060ce162434 2025-12-17 23:54:40,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd7575bc060ce162434 HTTP/1.1" 502 150 2025-12-17 23:54:40,385 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd7575bc060ce162434'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:40,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bcf575bc060ce16241f 2025-12-17 23:54:40,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcf575bc060ce16241f HTTP/1.1" 502 150 2025-12-17 23:54:40,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bcf575bc060ce16241f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:42,385 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd7575bc060ce162434 2025-12-17 23:54:42,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd7575bc060ce162434 HTTP/1.1" 502 150 2025-12-17 23:54:42,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd7575bc060ce162434'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:42,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd1575bc060ce162427 2025-12-17 23:54:42,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd1575bc060ce162427 HTTP/1.1" 502 150 2025-12-17 23:54:42,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd1575bc060ce162427'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:42,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd3575bc060ce16242a 2025-12-17 23:54:42,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd3575bc060ce16242a HTTP/1.1" 502 150 2025-12-17 23:54:42,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd3575bc060ce16242a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:42,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd5575bc060ce16242d 2025-12-17 23:54:42,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd5575bc060ce16242d HTTP/1.1" 502 150 2025-12-17 23:54:42,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd5575bc060ce16242d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:43,182 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bcb575bc060ce162411 2025-12-17 23:54:43,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcb575bc060ce162411 HTTP/1.1" 502 150 2025-12-17 23:54:43,184 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bcb575bc060ce162411'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:44,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bba575bc060ce1623e9 2025-12-17 23:54:44,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bba575bc060ce1623e9 HTTP/1.1" 502 150 2025-12-17 23:54:44,529 - lava_callback - ERROR - Error processing callback for node 69409bba575bc060ce1623e9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bba575bc060ce1623e9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bba575bc060ce1623e9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bba575bc060ce1623e9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:44,531 - lava_callback - INFO - Starting async job submit for node 69409bd9575bc060ce162437 2025-12-17 23:54:44,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:44,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd9575bc060ce162437 HTTP/1.1" 502 150 2025-12-17 23:54:44,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd9575bc060ce162437'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:44,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd9575bc060ce162437 2025-12-17 23:54:44,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd9575bc060ce162437 HTTP/1.1" 502 150 2025-12-17 23:54:44,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd9575bc060ce162437'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:46,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd7575bc060ce162434 2025-12-17 23:54:46,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd7575bc060ce162434 HTTP/1.1" 502 150 2025-12-17 23:54:46,388 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd7575bc060ce162434'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37492 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:54:46,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd9575bc060ce162437 2025-12-17 23:54:46,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd9575bc060ce162437 HTTP/1.1" 502 150 2025-12-17 23:54:46,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd9575bc060ce162437'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:46,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc5575bc060ce162404 HTTP/1.1" 502 150 2025-12-17 23:54:46,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc5575bc060ce162404'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:46,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bcd575bc060ce16241a 2025-12-17 23:54:46,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcd575bc060ce16241a HTTP/1.1" 502 150 2025-12-17 23:54:46,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bcd575bc060ce16241a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:47,157 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bbb575bc060ce1623ec 2025-12-17 23:54:47,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bbb575bc060ce1623ec HTTP/1.1" 502 150 2025-12-17 23:54:47,158 - lava_callback - ERROR - Error processing callback for node 69409bbb575bc060ce1623ec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bbb575bc060ce1623ec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bbb575bc060ce1623ec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bbb575bc060ce1623ec (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:47,160 - lava_callback - INFO - Starting async job submit for node 69409bda575bc060ce16243a 2025-12-17 23:54:47,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:47,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bda575bc060ce16243a HTTP/1.1" 502 150 2025-12-17 23:54:47,196 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bda575bc060ce16243a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:47,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bda575bc060ce16243a 2025-12-17 23:54:47,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bda575bc060ce16243a HTTP/1.1" 502 150 2025-12-17 23:54:47,197 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bda575bc060ce16243a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:47,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bbe575bc060ce1623f4 2025-12-17 23:54:47,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bbe575bc060ce1623f4 HTTP/1.1" 502 150 2025-12-17 23:54:47,710 - lava_callback - ERROR - Error processing callback for node 69409bbe575bc060ce1623f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bbe575bc060ce1623f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bbe575bc060ce1623f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bbe575bc060ce1623f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:47,712 - lava_callback - INFO - Starting async job submit for node 69409bdc575bc060ce16243d 2025-12-17 23:54:47,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:47,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdc575bc060ce16243d HTTP/1.1" 502 150 2025-12-17 23:54:47,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bdc575bc060ce16243d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:47,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bdc575bc060ce16243d 2025-12-17 23:54:47,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdc575bc060ce16243d HTTP/1.1" 502 150 2025-12-17 23:54:47,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bdc575bc060ce16243d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:47,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c43575bc060ce162555 2025-12-17 23:54:47,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c43575bc060ce162555 HTTP/1.1" 502 150 2025-12-17 23:54:47,820 - lava_callback - ERROR - Error processing callback for node 69409c43575bc060ce162555: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c43575bc060ce162555 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c43575bc060ce162555 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c43575bc060ce162555 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:47,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc0575bc060ce1623fe 2025-12-17 23:54:47,822 - lava_callback - INFO - Starting async job submit for node 6940c5461e3c24e95681d6b9 2025-12-17 23:54:47,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc0575bc060ce1623fe HTTP/1.1" 502 150 2025-12-17 23:54:47,834 - lava_callback - ERROR - Error processing callback for node 69409bc0575bc060ce1623fe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bc0575bc060ce1623fe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bc0575bc060ce1623fe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bc0575bc060ce1623fe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:47,857 - lava_callback - INFO - Starting async job submit for node 69409bdd575bc060ce162440 2025-12-17 23:54:47,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:47,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:47,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdd575bc060ce162440 HTTP/1.1" 502 150 2025-12-17 23:54:47,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bdd575bc060ce162440'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:47,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bdd575bc060ce162440 2025-12-17 23:54:47,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdd575bc060ce162440 HTTP/1.1" 502 150 2025-12-17 23:54:47,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bdd575bc060ce162440'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:47,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5461e3c24e95681d6b9 HTTP/1.1" 502 150 2025-12-17 23:54:47,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5461e3c24e95681d6b9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:47,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5461e3c24e95681d6b9 2025-12-17 23:54:47,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5461e3c24e95681d6b9 HTTP/1.1" 502 150 2025-12-17 23:54:47,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5461e3c24e95681d6b9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:48,774 - lava_callback - INFO - Received callback for node 69409fb3575bc060ce162b8a INFO: 172.21.0.1:37500 - "POST /node/69409fb3575bc060ce162b8a HTTP/1.0" 202 Accepted 2025-12-17 23:54:48,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bcf575bc060ce16241f 2025-12-17 23:54:48,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcf575bc060ce16241f HTTP/1.1" 502 150 2025-12-17 23:54:48,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bcf575bc060ce16241f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:49,197 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bda575bc060ce16243a 2025-12-17 23:54:49,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bda575bc060ce16243a HTTP/1.1" 502 150 2025-12-17 23:54:49,198 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bda575bc060ce16243a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:49,417 - lava_callback - INFO - Received callback for node 69409fb5575bc060ce162b95 INFO: 172.21.0.1:37504 - "POST /node/69409fb5575bc060ce162b95 HTTP/1.0" 202 Accepted 2025-12-17 23:54:49,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bdc575bc060ce16243d 2025-12-17 23:54:49,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdc575bc060ce16243d HTTP/1.1" 502 150 2025-12-17 23:54:49,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bdc575bc060ce16243d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:49,733 - lava_callback - INFO - Received callback for node 69409fb6575bc060ce162b98 INFO: 172.21.0.1:47064 - "POST /node/69409fb6575bc060ce162b98 HTTP/1.0" 202 Accepted 2025-12-17 23:54:49,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bdd575bc060ce162440 2025-12-17 23:54:49,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdd575bc060ce162440 HTTP/1.1" 502 150 2025-12-17 23:54:49,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bdd575bc060ce162440'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:49,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5461e3c24e95681d6b9 2025-12-17 23:54:49,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5461e3c24e95681d6b9 HTTP/1.1" 502 150 2025-12-17 23:54:49,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5461e3c24e95681d6b9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:50,062 - lava_callback - INFO - Received callback for node 69409fb8575bc060ce162b9b INFO: 172.21.0.1:47066 - "POST /node/69409fb8575bc060ce162b9b HTTP/1.0" 202 Accepted 2025-12-17 23:54:50,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd1575bc060ce162427 2025-12-17 23:54:50,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd1575bc060ce162427 HTTP/1.1" 502 150 2025-12-17 23:54:50,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd1575bc060ce162427'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:50,540 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd9575bc060ce162437 2025-12-17 23:54:50,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd9575bc060ce162437 HTTP/1.1" 502 150 2025-12-17 23:54:50,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd9575bc060ce162437'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:50,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd3575bc060ce16242a 2025-12-17 23:54:50,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd3575bc060ce16242a HTTP/1.1" 502 150 2025-12-17 23:54:50,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd3575bc060ce16242a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:50,759 - lava_callback - INFO - Received callback for node 69409fb9575bc060ce162ba2 INFO: 172.21.0.1:47068 - "POST /node/69409fb9575bc060ce162ba2 HTTP/1.0" 202 Accepted 2025-12-17 23:54:50,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd5575bc060ce16242d 2025-12-17 23:54:50,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd5575bc060ce16242d HTTP/1.1" 502 150 2025-12-17 23:54:50,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd5575bc060ce16242d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:51,376 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc8575bc060ce16240a 2025-12-17 23:54:51,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc8575bc060ce16240a HTTP/1.1" 502 150 2025-12-17 23:54:51,377 - lava_callback - ERROR - Error processing callback for node 69409bc8575bc060ce16240a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bc8575bc060ce16240a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bc8575bc060ce16240a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bc8575bc060ce16240a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:51,379 - lava_callback - INFO - Starting async job submit for node 69409bdf575bc060ce162447 2025-12-17 23:54:51,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:51,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdf575bc060ce162447 HTTP/1.1" 502 150 2025-12-17 23:54:51,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bdf575bc060ce162447'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:51,386 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bdf575bc060ce162447 2025-12-17 23:54:51,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdf575bc060ce162447 HTTP/1.1" 502 150 2025-12-17 23:54:51,386 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bdf575bc060ce162447'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:53,199 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bda575bc060ce16243a 2025-12-17 23:54:53,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bda575bc060ce16243a HTTP/1.1" 502 150 2025-12-17 23:54:53,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bda575bc060ce16243a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:53,387 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bdf575bc060ce162447 2025-12-17 23:54:53,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdf575bc060ce162447 HTTP/1.1" 502 150 2025-12-17 23:54:53,388 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bdf575bc060ce162447'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:53,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bdc575bc060ce16243d 2025-12-17 23:54:53,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdc575bc060ce16243d HTTP/1.1" 502 150 2025-12-17 23:54:53,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bdc575bc060ce16243d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:53,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bdd575bc060ce162440 2025-12-17 23:54:53,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdd575bc060ce162440 HTTP/1.1" 502 150 2025-12-17 23:54:53,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bdd575bc060ce162440'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:53,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5461e3c24e95681d6b9 2025-12-17 23:54:53,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5461e3c24e95681d6b9 HTTP/1.1" 502 150 2025-12-17 23:54:53,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5461e3c24e95681d6b9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:54,388 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd7575bc060ce162434 2025-12-17 23:54:54,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd7575bc060ce162434 HTTP/1.1" 502 150 2025-12-17 23:54:54,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd7575bc060ce162434'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:54,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc5575bc060ce162404 2025-12-17 23:54:54,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc5575bc060ce162404 HTTP/1.1" 502 150 2025-12-17 23:54:54,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bc5575bc060ce162404'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:54,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc2575bc060ce162401 2025-12-17 23:54:54,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc2575bc060ce162401 HTTP/1.1" 502 150 2025-12-17 23:54:54,921 - lava_callback - ERROR - Error processing callback for node 69409bc2575bc060ce162401: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bc2575bc060ce162401 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bc2575bc060ce162401 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bc2575bc060ce162401 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:54,923 - lava_callback - INFO - Starting async job submit for node 69409be1575bc060ce162450 2025-12-17 23:54:54,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:54,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be1575bc060ce162450 HTTP/1.1" 502 150 2025-12-17 23:54:54,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be1575bc060ce162450'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:54,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be1575bc060ce162450 2025-12-17 23:54:54,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be1575bc060ce162450 HTTP/1.1" 502 150 2025-12-17 23:54:54,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be1575bc060ce162450'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:55,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc7575bc060ce162407 2025-12-17 23:54:55,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc7575bc060ce162407 HTTP/1.1" 502 150 2025-12-17 23:54:55,731 - lava_callback - ERROR - Error processing callback for node 69409bc7575bc060ce162407: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bc7575bc060ce162407 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bc7575bc060ce162407 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bc7575bc060ce162407 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:55,732 - lava_callback - INFO - Starting async job submit for node 69409be4575bc060ce162457 2025-12-17 23:54:55,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:55,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be4575bc060ce162457 HTTP/1.1" 502 150 2025-12-17 23:54:55,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be4575bc060ce162457'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:55,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be4575bc060ce162457 2025-12-17 23:54:55,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be4575bc060ce162457 HTTP/1.1" 502 150 2025-12-17 23:54:55,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be4575bc060ce162457'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47076 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:54:56,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be1575bc060ce162450 2025-12-17 23:54:56,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be1575bc060ce162450 HTTP/1.1" 502 150 2025-12-17 23:54:56,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be1575bc060ce162450'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:57,388 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bdf575bc060ce162447 2025-12-17 23:54:57,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be4575bc060ce162457 2025-12-17 23:54:57,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be4575bc060ce162457 HTTP/1.1" 502 150 2025-12-17 23:54:57,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be4575bc060ce162457'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:58,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd9575bc060ce162437 2025-12-17 23:54:58,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd9575bc060ce162437 HTTP/1.1" 502 150 2025-12-17 23:54:58,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bd9575bc060ce162437'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:59,184 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bcb575bc060ce162411 2025-12-17 23:54:59,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcb575bc060ce162411 HTTP/1.1" 502 150 2025-12-17 23:54:59,185 - lava_callback - ERROR - Error processing callback for node 69409bcb575bc060ce162411: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bcb575bc060ce162411 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bcb575bc060ce162411 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bcb575bc060ce162411 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:54:59,187 - lava_callback - INFO - Starting async job submit for node 69409be7575bc060ce162461 2025-12-17 23:54:59,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:54:59,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be7575bc060ce162461 HTTP/1.1" 502 150 2025-12-17 23:54:59,199 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be7575bc060ce162461'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:54:59,199 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be7575bc060ce162461 2025-12-17 23:54:59,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be7575bc060ce162461 HTTP/1.1" 502 150 2025-12-17 23:54:59,199 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be7575bc060ce162461'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:00,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be1575bc060ce162450 2025-12-17 23:55:00,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be1575bc060ce162450 HTTP/1.1" 502 150 2025-12-17 23:55:00,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be1575bc060ce162450'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:01,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be7575bc060ce162461 2025-12-17 23:55:01,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bda575bc060ce16243a 2025-12-17 23:55:01,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be7575bc060ce162461 HTTP/1.1" 502 150 2025-12-17 23:55:01,201 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be7575bc060ce162461'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:01,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bda575bc060ce16243a HTTP/1.1" 502 150 2025-12-17 23:55:01,201 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bda575bc060ce16243a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:01,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bdc575bc060ce16243d 2025-12-17 23:55:01,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdc575bc060ce16243d HTTP/1.1" 502 150 2025-12-17 23:55:01,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bdc575bc060ce16243d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:01,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bdd575bc060ce162440 2025-12-17 23:55:01,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdd575bc060ce162440 HTTP/1.1" 502 150 2025-12-17 23:55:01,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bdd575bc060ce162440'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:01,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5461e3c24e95681d6b9 2025-12-17 23:55:01,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5461e3c24e95681d6b9 HTTP/1.1" 502 150 2025-12-17 23:55:01,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5461e3c24e95681d6b9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:01,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be4575bc060ce162457 2025-12-17 23:55:01,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be4575bc060ce162457 HTTP/1.1" 502 150 2025-12-17 23:55:01,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be4575bc060ce162457'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:02,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bcd575bc060ce16241a 2025-12-17 23:55:02,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcd575bc060ce16241a HTTP/1.1" 502 150 2025-12-17 23:55:02,972 - lava_callback - ERROR - Error processing callback for node 69409bcd575bc060ce16241a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bcd575bc060ce16241a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bcd575bc060ce16241a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bcd575bc060ce16241a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:02,974 - lava_callback - INFO - Starting async job submit for node 69409be9575bc060ce162468 2025-12-17 23:55:02,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:03,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be9575bc060ce162468 HTTP/1.1" 502 150 2025-12-17 23:55:03,148 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be9575bc060ce162468'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:03,148 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be9575bc060ce162468 2025-12-17 23:55:03,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be9575bc060ce162468 HTTP/1.1" 502 150 2025-12-17 23:55:03,149 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be9575bc060ce162468'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59940 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:55:04,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bcf575bc060ce16241f 2025-12-17 23:55:04,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bcf575bc060ce16241f HTTP/1.1" 502 150 2025-12-17 23:55:04,835 - lava_callback - ERROR - Error processing callback for node 69409bcf575bc060ce16241f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bcf575bc060ce16241f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bcf575bc060ce16241f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bcf575bc060ce16241f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:04,838 - lava_callback - INFO - Starting async job submit for node 69409beb575bc060ce16246b 2025-12-17 23:55:04,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:04,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409beb575bc060ce16246b HTTP/1.1" 502 150 2025-12-17 23:55:04,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409beb575bc060ce16246b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:04,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409beb575bc060ce16246b 2025-12-17 23:55:04,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409beb575bc060ce16246b HTTP/1.1" 502 150 2025-12-17 23:55:04,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409beb575bc060ce16246b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:05,149 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be9575bc060ce162468 2025-12-17 23:55:05,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be9575bc060ce162468 HTTP/1.1" 502 150 2025-12-17 23:55:05,150 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be9575bc060ce162468'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:05,201 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be7575bc060ce162461 2025-12-17 23:55:05,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be7575bc060ce162461 HTTP/1.1" 502 150 2025-12-17 23:55:05,202 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be7575bc060ce162461'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59944 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:55:06,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd1575bc060ce162427 2025-12-17 23:55:06,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd3575bc060ce16242a 2025-12-17 23:55:06,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd3575bc060ce16242a HTTP/1.1" 502 150 2025-12-17 23:55:06,737 - lava_callback - ERROR - Error processing callback for node 69409bd3575bc060ce16242a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bd3575bc060ce16242a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bd3575bc060ce16242a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bd3575bc060ce16242a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:06,739 - lava_callback - INFO - Starting async job submit for node 69409bed575bc060ce16246e 2025-12-17 23:55:06,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:06,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bed575bc060ce16246e HTTP/1.1" 502 150 2025-12-17 23:55:06,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bed575bc060ce16246e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:06,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bed575bc060ce16246e 2025-12-17 23:55:06,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bed575bc060ce16246e HTTP/1.1" 502 150 2025-12-17 23:55:06,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bed575bc060ce16246e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:06,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd5575bc060ce16242d 2025-12-17 23:55:06,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd5575bc060ce16242d HTTP/1.1" 502 150 2025-12-17 23:55:06,812 - lava_callback - ERROR - Error processing callback for node 69409bd5575bc060ce16242d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bd5575bc060ce16242d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bd5575bc060ce16242d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bd5575bc060ce16242d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:06,814 - lava_callback - INFO - Starting async job submit for node 69409bef575bc060ce162471 2025-12-17 23:55:06,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:06,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bef575bc060ce162471 HTTP/1.1" 502 150 2025-12-17 23:55:06,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bef575bc060ce162471'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:06,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bef575bc060ce162471 2025-12-17 23:55:06,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bef575bc060ce162471 HTTP/1.1" 502 150 2025-12-17 23:55:06,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bef575bc060ce162471'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:06,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409beb575bc060ce16246b 2025-12-17 23:55:06,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409beb575bc060ce16246b HTTP/1.1" 502 150 2025-12-17 23:55:06,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409beb575bc060ce16246b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:08,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bed575bc060ce16246e 2025-12-17 23:55:08,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bed575bc060ce16246e HTTP/1.1" 502 150 2025-12-17 23:55:08,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bed575bc060ce16246e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:08,798 - lava_callback - INFO - Received callback for node 69409fc7575bc060ce162ba7 INFO: 172.21.0.1:59950 - "POST /node/69409fc7575bc060ce162ba7 HTTP/1.0" 202 Accepted 2025-12-17 23:55:08,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bef575bc060ce162471 2025-12-17 23:55:08,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bef575bc060ce162471 HTTP/1.1" 502 150 2025-12-17 23:55:08,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bef575bc060ce162471'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:08,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be1575bc060ce162450 2025-12-17 23:55:08,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be1575bc060ce162450 HTTP/1.1" 502 150 2025-12-17 23:55:08,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be1575bc060ce162450'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:09,150 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be9575bc060ce162468 2025-12-17 23:55:09,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be9575bc060ce162468 HTTP/1.1" 502 150 2025-12-17 23:55:09,152 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be9575bc060ce162468'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:09,437 - lava_callback - INFO - Received callback for node 69409fc9575bc060ce162bb0 INFO: 172.21.0.1:59956 - "POST /node/69409fc9575bc060ce162bb0 HTTP/1.0" 202 Accepted 2025-12-17 23:55:09,686 - lava_callback - INFO - Received callback for node 69409fca575bc060ce162bb7 INFO: 172.21.0.1:59958 - "POST /node/69409fca575bc060ce162bb7 HTTP/1.0" 202 Accepted 2025-12-17 23:55:09,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be4575bc060ce162457 2025-12-17 23:55:09,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be4575bc060ce162457 HTTP/1.1" 502 150 2025-12-17 23:55:09,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be4575bc060ce162457'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:10,083 - lava_callback - INFO - Received callback for node 69409fcb575bc060ce162bbc INFO: 172.21.0.1:54120 - "POST /node/69409fcb575bc060ce162bbc HTTP/1.0" 202 Accepted 2025-12-17 23:55:10,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd7575bc060ce162434 2025-12-17 23:55:10,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd7575bc060ce162434 HTTP/1.1" 502 150 2025-12-17 23:55:10,393 - lava_callback - ERROR - Error processing callback for node 69409bd7575bc060ce162434: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bd7575bc060ce162434 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bd7575bc060ce162434 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bd7575bc060ce162434 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:10,395 - lava_callback - INFO - Starting async job submit for node 69409bf1575bc060ce162474 2025-12-17 23:55:10,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:10,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf1575bc060ce162474 HTTP/1.1" 502 150 2025-12-17 23:55:10,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf1575bc060ce162474'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:10,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf1575bc060ce162474 2025-12-17 23:55:10,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf1575bc060ce162474 HTTP/1.1" 502 150 2025-12-17 23:55:10,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf1575bc060ce162474'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:10,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bc5575bc060ce162404 2025-12-17 23:55:10,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bc5575bc060ce162404 HTTP/1.1" 502 150 2025-12-17 23:55:10,579 - lava_callback - ERROR - Error processing callback for node 69409bc5575bc060ce162404: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bc5575bc060ce162404 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bc5575bc060ce162404 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bc5575bc060ce162404 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:10,580 - lava_callback - INFO - Starting async job submit for node 69409bf5575bc060ce162484 2025-12-17 23:55:10,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:10,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf5575bc060ce162484 HTTP/1.1" 502 150 2025-12-17 23:55:10,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf5575bc060ce162484'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:10,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf5575bc060ce162484 2025-12-17 23:55:10,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf5575bc060ce162484 HTTP/1.1" 502 150 2025-12-17 23:55:10,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf5575bc060ce162484'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:10,715 - lava_callback - INFO - Received callback for node 69409fcd575bc060ce162bc3 INFO: 172.21.0.1:54122 - "POST /node/69409fcd575bc060ce162bc3 HTTP/1.0" 202 Accepted 2025-12-17 23:55:10,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409beb575bc060ce16246b 2025-12-17 23:55:10,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409beb575bc060ce16246b HTTP/1.1" 502 150 2025-12-17 23:55:10,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409beb575bc060ce16246b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:11,357 - lava_callback - INFO - Received callback for node 69409fce575bc060ce162bc6 INFO: 172.21.0.1:54130 - "POST /node/69409fce575bc060ce162bc6 HTTP/1.0" 202 Accepted 2025-12-17 23:55:12,011 - lava_callback - INFO - Received callback for node 69409fd0575bc060ce162bcd INFO: 172.21.0.1:54138 - "POST /node/69409fd0575bc060ce162bcd HTTP/1.0" 202 Accepted 2025-12-17 23:55:12,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf1575bc060ce162474 2025-12-17 23:55:12,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf1575bc060ce162474 HTTP/1.1" 502 150 2025-12-17 23:55:12,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf1575bc060ce162474'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:12,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf5575bc060ce162484 2025-12-17 23:55:12,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf5575bc060ce162484 HTTP/1.1" 502 150 2025-12-17 23:55:12,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf5575bc060ce162484'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:12,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdf575bc060ce162447 HTTP/1.1" 502 150 2025-12-17 23:55:12,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bdf575bc060ce162447'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:12,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bed575bc060ce16246e 2025-12-17 23:55:12,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bed575bc060ce16246e HTTP/1.1" 502 150 2025-12-17 23:55:12,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bed575bc060ce16246e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:12,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bef575bc060ce162471 2025-12-17 23:55:12,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bef575bc060ce162471 HTTP/1.1" 502 150 2025-12-17 23:55:12,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bef575bc060ce162471'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:13,202 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be7575bc060ce162461 2025-12-17 23:55:13,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be7575bc060ce162461 HTTP/1.1" 502 150 2025-12-17 23:55:13,204 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be7575bc060ce162461'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:13,257 - lava_callback - INFO - Received callback for node 69409fd5575bc060ce162bdd INFO: 172.21.0.1:54148 - "POST /node/69409fd5575bc060ce162bdd HTTP/1.0" 202 Accepted 2025-12-17 23:55:13,947 - lava_callback - INFO - Received callback for node 69409fd2575bc060ce162bd0 INFO: 172.21.0.1:54156 - "POST /node/69409fd2575bc060ce162bd0 HTTP/1.0" 202 Accepted 2025-12-17 23:55:14,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bd9575bc060ce162437 2025-12-17 23:55:14,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd9575bc060ce162437 HTTP/1.1" 502 150 2025-12-17 23:55:14,546 - lava_callback - ERROR - Error processing callback for node 69409bd9575bc060ce162437: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bd9575bc060ce162437 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bd9575bc060ce162437 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bd9575bc060ce162437 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:14,548 - lava_callback - INFO - Starting async job submit for node 69409bf7575bc060ce16248b 2025-12-17 23:55:14,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:14,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf7575bc060ce16248b HTTP/1.1" 502 150 2025-12-17 23:55:14,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf7575bc060ce16248b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:14,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf7575bc060ce16248b 2025-12-17 23:55:14,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf7575bc060ce16248b HTTP/1.1" 502 150 2025-12-17 23:55:14,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf7575bc060ce16248b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:14,639 - lava_callback - INFO - Received callback for node 69409fd3575bc060ce162bd7 INFO: 172.21.0.1:54166 - "POST /node/69409fd3575bc060ce162bd7 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:54168 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:55:16,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf1575bc060ce162474 2025-12-17 23:55:16,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf1575bc060ce162474 HTTP/1.1" 502 150 2025-12-17 23:55:16,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf1575bc060ce162474'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54176 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:55:16,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf5575bc060ce162484 2025-12-17 23:55:16,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf7575bc060ce16248b 2025-12-17 23:55:16,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf5575bc060ce162484 HTTP/1.1" 502 150 2025-12-17 23:55:16,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf5575bc060ce162484'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:16,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf7575bc060ce16248b HTTP/1.1" 502 150 2025-12-17 23:55:16,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf7575bc060ce16248b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:17,152 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be9575bc060ce162468 2025-12-17 23:55:17,202 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bda575bc060ce16243a 2025-12-17 23:55:17,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bda575bc060ce16243a HTTP/1.1" 502 150 2025-12-17 23:55:17,203 - lava_callback - ERROR - Error processing callback for node 69409bda575bc060ce16243a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bda575bc060ce16243a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bda575bc060ce16243a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bda575bc060ce16243a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:17,205 - lava_callback - INFO - Starting async job submit for node 69409bf9575bc060ce16248e 2025-12-17 23:55:17,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:17,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf9575bc060ce16248e HTTP/1.1" 502 150 2025-12-17 23:55:17,239 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf9575bc060ce16248e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:17,239 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf9575bc060ce16248e 2025-12-17 23:55:17,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf9575bc060ce16248e HTTP/1.1" 502 150 2025-12-17 23:55:17,240 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf9575bc060ce16248e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:17,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bdc575bc060ce16243d 2025-12-17 23:55:17,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdc575bc060ce16243d HTTP/1.1" 502 150 2025-12-17 23:55:17,726 - lava_callback - ERROR - Error processing callback for node 69409bdc575bc060ce16243d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bdc575bc060ce16243d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bdc575bc060ce16243d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bdc575bc060ce16243d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:17,728 - lava_callback - INFO - Starting async job submit for node 69409bfb575bc060ce162491 2025-12-17 23:55:17,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:17,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bfb575bc060ce162491 HTTP/1.1" 502 150 2025-12-17 23:55:17,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bfb575bc060ce162491'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:17,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bfb575bc060ce162491 2025-12-17 23:55:17,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bfb575bc060ce162491 HTTP/1.1" 502 150 2025-12-17 23:55:17,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bfb575bc060ce162491'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:17,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bdd575bc060ce162440 2025-12-17 23:55:17,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdd575bc060ce162440 HTTP/1.1" 502 150 2025-12-17 23:55:17,904 - lava_callback - ERROR - Error processing callback for node 69409bdd575bc060ce162440: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bdd575bc060ce162440 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bdd575bc060ce162440 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bdd575bc060ce162440 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:17,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5461e3c24e95681d6b9 2025-12-17 23:55:17,906 - lava_callback - INFO - Starting async job submit for node 6942d8e1e43d2b76e9021f27 2025-12-17 23:55:17,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:17,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5461e3c24e95681d6b9 HTTP/1.1" 502 150 2025-12-17 23:55:17,910 - lava_callback - ERROR - Error processing callback for node 6940c5461e3c24e95681d6b9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5461e3c24e95681d6b9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5461e3c24e95681d6b9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5461e3c24e95681d6b9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:17,913 - lava_callback - INFO - Starting async job submit for node 69409bf3575bc060ce16247b 2025-12-17 23:55:17,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:17,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d8e1e43d2b76e9021f27 HTTP/1.1" 502 150 2025-12-17 23:55:17,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942d8e1e43d2b76e9021f27'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:17,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942d8e1e43d2b76e9021f27 2025-12-17 23:55:17,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d8e1e43d2b76e9021f27 HTTP/1.1" 502 150 2025-12-17 23:55:17,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942d8e1e43d2b76e9021f27'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:17,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf3575bc060ce16247b HTTP/1.1" 502 150 2025-12-17 23:55:17,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf3575bc060ce16247b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:17,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf3575bc060ce16247b 2025-12-17 23:55:17,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf3575bc060ce16247b HTTP/1.1" 502 150 2025-12-17 23:55:17,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf3575bc060ce16247b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:18,020 - lava_callback - INFO - Received callback for node 69409fde575bc060ce162bef INFO: 172.21.0.1:54186 - "POST /node/69409fde575bc060ce162bef HTTP/1.0" 202 Accepted 2025-12-17 23:55:18,671 - lava_callback - INFO - Received callback for node 69409fe3575bc060ce162bf5 INFO: 172.21.0.1:54190 - "POST /node/69409fe3575bc060ce162bf5 HTTP/1.0" 202 Accepted 2025-12-17 23:55:18,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409beb575bc060ce16246b 2025-12-17 23:55:18,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409beb575bc060ce16246b HTTP/1.1" 502 150 2025-12-17 23:55:18,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409beb575bc060ce16246b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:19,240 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf9575bc060ce16248e 2025-12-17 23:55:19,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf9575bc060ce16248e HTTP/1.1" 502 150 2025-12-17 23:55:19,242 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf9575bc060ce16248e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:19,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bfb575bc060ce162491 2025-12-17 23:55:19,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bfb575bc060ce162491 HTTP/1.1" 502 150 2025-12-17 23:55:19,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bfb575bc060ce162491'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:19,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942d8e1e43d2b76e9021f27 2025-12-17 23:55:19,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d8e1e43d2b76e9021f27 HTTP/1.1" 502 150 2025-12-17 23:55:19,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942d8e1e43d2b76e9021f27'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:19,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf3575bc060ce16247b 2025-12-17 23:55:19,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf3575bc060ce16247b HTTP/1.1" 502 150 2025-12-17 23:55:19,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf3575bc060ce16247b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:20,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf7575bc060ce16248b 2025-12-17 23:55:20,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf7575bc060ce16248b HTTP/1.1" 502 150 2025-12-17 23:55:20,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf7575bc060ce16248b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:20,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bdf575bc060ce162447 2025-12-17 23:55:20,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdf575bc060ce162447 HTTP/1.1" 502 150 2025-12-17 23:55:20,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bdf575bc060ce162447'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:20,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bed575bc060ce16246e 2025-12-17 23:55:20,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bed575bc060ce16246e HTTP/1.1" 502 150 2025-12-17 23:55:20,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bed575bc060ce16246e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:20,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bef575bc060ce162471 2025-12-17 23:55:20,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bef575bc060ce162471 HTTP/1.1" 502 150 2025-12-17 23:55:20,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bef575bc060ce162471'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:21,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bd1575bc060ce162427 HTTP/1.1" 502 150 2025-12-17 23:55:21,904 - lava_callback - ERROR - Error processing callback for node 69409bd1575bc060ce162427: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bd1575bc060ce162427 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bd1575bc060ce162427 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bd1575bc060ce162427 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:21,907 - lava_callback - INFO - Starting async job submit for node 69409bfd575bc060ce162498 2025-12-17 23:55:21,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:21,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bfd575bc060ce162498 HTTP/1.1" 502 150 2025-12-17 23:55:21,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bfd575bc060ce162498'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:21,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bfd575bc060ce162498 2025-12-17 23:55:21,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bfd575bc060ce162498 HTTP/1.1" 502 150 2025-12-17 23:55:21,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bfd575bc060ce162498'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:23,242 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf9575bc060ce16248e 2025-12-17 23:55:23,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bfb575bc060ce162491 2025-12-17 23:55:23,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bfb575bc060ce162491 HTTP/1.1" 502 150 2025-12-17 23:55:23,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bfb575bc060ce162491'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:23,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942d8e1e43d2b76e9021f27 2025-12-17 23:55:23,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d8e1e43d2b76e9021f27 HTTP/1.1" 502 150 2025-12-17 23:55:23,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942d8e1e43d2b76e9021f27'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:23,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bfd575bc060ce162498 2025-12-17 23:55:23,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bfd575bc060ce162498 HTTP/1.1" 502 150 2025-12-17 23:55:23,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bfd575bc060ce162498'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:23,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf3575bc060ce16247b 2025-12-17 23:55:23,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf3575bc060ce16247b HTTP/1.1" 502 150 2025-12-17 23:55:23,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf3575bc060ce16247b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:24,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf1575bc060ce162474 2025-12-17 23:55:24,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf1575bc060ce162474 HTTP/1.1" 502 150 2025-12-17 23:55:24,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf1575bc060ce162474'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:24,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf5575bc060ce162484 2025-12-17 23:55:24,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf5575bc060ce162484 HTTP/1.1" 502 150 2025-12-17 23:55:24,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf5575bc060ce162484'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:24,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be1575bc060ce162450 2025-12-17 23:55:24,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be1575bc060ce162450 HTTP/1.1" 502 150 2025-12-17 23:55:24,936 - lava_callback - ERROR - Error processing callback for node 69409be1575bc060ce162450: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409be1575bc060ce162450 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409be1575bc060ce162450 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409be1575bc060ce162450 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:24,939 - lava_callback - INFO - Starting async job submit for node 69409c04575bc060ce1624a1 2025-12-17 23:55:24,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:24,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c04575bc060ce1624a1 HTTP/1.1" 502 150 2025-12-17 23:55:24,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c04575bc060ce1624a1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:24,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c04575bc060ce1624a1 2025-12-17 23:55:24,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c04575bc060ce1624a1 HTTP/1.1" 502 150 2025-12-17 23:55:24,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c04575bc060ce1624a1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:25,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be4575bc060ce162457 2025-12-17 23:55:25,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be4575bc060ce162457 HTTP/1.1" 502 150 2025-12-17 23:55:25,933 - lava_callback - ERROR - Error processing callback for node 69409be4575bc060ce162457: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409be4575bc060ce162457 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409be4575bc060ce162457 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409be4575bc060ce162457 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:25,935 - lava_callback - INFO - Starting async job submit for node 69409c02575bc060ce16249e 2025-12-17 23:55:25,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:25,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c02575bc060ce16249e HTTP/1.1" 502 150 2025-12-17 23:55:25,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c02575bc060ce16249e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:25,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c02575bc060ce16249e 2025-12-17 23:55:25,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c02575bc060ce16249e HTTP/1.1" 502 150 2025-12-17 23:55:25,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c02575bc060ce16249e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49684 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:55:26,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c04575bc060ce1624a1 2025-12-17 23:55:26,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c04575bc060ce1624a1 HTTP/1.1" 502 150 2025-12-17 23:55:26,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c04575bc060ce1624a1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:27,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bfd575bc060ce162498 2025-12-17 23:55:27,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bfd575bc060ce162498 HTTP/1.1" 502 150 2025-12-17 23:55:27,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bfd575bc060ce162498'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:27,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c02575bc060ce16249e 2025-12-17 23:55:27,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c02575bc060ce16249e HTTP/1.1" 502 150 2025-12-17 23:55:27,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c02575bc060ce16249e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:28,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf7575bc060ce16248b 2025-12-17 23:55:28,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf7575bc060ce16248b HTTP/1.1" 502 150 2025-12-17 23:55:28,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf7575bc060ce16248b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:28,805 - lava_callback - INFO - Received callback for node 69409fdb575bc060ce162be2 INFO: 172.21.0.1:49698 - "POST /node/69409fdb575bc060ce162be2 HTTP/1.0" 202 Accepted 2025-12-17 23:55:29,204 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be7575bc060ce162461 2025-12-17 23:55:29,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be7575bc060ce162461 HTTP/1.1" 502 150 2025-12-17 23:55:29,205 - lava_callback - ERROR - Error processing callback for node 69409be7575bc060ce162461: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409be7575bc060ce162461 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409be7575bc060ce162461 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409be7575bc060ce162461 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:29,207 - lava_callback - INFO - Starting async job submit for node 69409c00575bc060ce16249b 2025-12-17 23:55:29,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:29,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c00575bc060ce16249b HTTP/1.1" 502 150 2025-12-17 23:55:29,214 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c00575bc060ce16249b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:29,214 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c00575bc060ce16249b 2025-12-17 23:55:29,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c00575bc060ce16249b HTTP/1.1" 502 150 2025-12-17 23:55:29,214 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c00575bc060ce16249b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:29,454 - lava_callback - INFO - Received callback for node 69409fe1575bc060ce162bf2 INFO: 172.21.0.1:49712 - "POST /node/69409fe1575bc060ce162bf2 HTTP/1.0" 202 Accepted 2025-12-17 23:55:30,128 - lava_callback - INFO - Received callback for node 69409fe5575bc060ce162bf8 INFO: 172.21.0.1:51408 - "POST /node/69409fe5575bc060ce162bf8 HTTP/1.0" 202 Accepted 2025-12-17 23:55:30,787 - lava_callback - INFO - Received callback for node 69409fe7575bc060ce162bfb INFO: 172.21.0.1:51416 - "POST /node/69409fe7575bc060ce162bfb HTTP/1.0" 202 Accepted 2025-12-17 23:55:30,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c04575bc060ce1624a1 2025-12-17 23:55:30,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c04575bc060ce1624a1 HTTP/1.1" 502 150 2025-12-17 23:55:30,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c04575bc060ce1624a1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:31,215 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c00575bc060ce16249b 2025-12-17 23:55:31,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c00575bc060ce16249b HTTP/1.1" 502 150 2025-12-17 23:55:31,216 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c00575bc060ce16249b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:31,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bfb575bc060ce162491 2025-12-17 23:55:31,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bfb575bc060ce162491 HTTP/1.1" 502 150 2025-12-17 23:55:31,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bfb575bc060ce162491'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:31,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942d8e1e43d2b76e9021f27 2025-12-17 23:55:31,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d8e1e43d2b76e9021f27 HTTP/1.1" 502 150 2025-12-17 23:55:31,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942d8e1e43d2b76e9021f27'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:31,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf3575bc060ce16247b 2025-12-17 23:55:31,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf3575bc060ce16247b HTTP/1.1" 502 150 2025-12-17 23:55:31,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf3575bc060ce16247b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:31,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c02575bc060ce16249e 2025-12-17 23:55:31,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c02575bc060ce16249e HTTP/1.1" 502 150 2025-12-17 23:55:31,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c02575bc060ce16249e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:32,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be9575bc060ce162468 HTTP/1.1" 502 150 2025-12-17 23:55:32,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be9575bc060ce162468'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:34,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409beb575bc060ce16246b 2025-12-17 23:55:34,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409beb575bc060ce16246b HTTP/1.1" 502 150 2025-12-17 23:55:34,852 - lava_callback - ERROR - Error processing callback for node 69409beb575bc060ce16246b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409beb575bc060ce16246b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409beb575bc060ce16246b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409beb575bc060ce16246b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:34,854 - lava_callback - INFO - Starting async job submit for node 69409c05575bc060ce1624a4 2025-12-17 23:55:34,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:34,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c05575bc060ce1624a4 HTTP/1.1" 502 150 2025-12-17 23:55:34,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c05575bc060ce1624a4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:34,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c05575bc060ce1624a4 2025-12-17 23:55:34,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c05575bc060ce1624a4 HTTP/1.1" 502 150 2025-12-17 23:55:34,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c05575bc060ce1624a4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:35,216 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c00575bc060ce16249b 2025-12-17 23:55:35,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c00575bc060ce16249b HTTP/1.1" 502 150 2025-12-17 23:55:35,217 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c00575bc060ce16249b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:35,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bfd575bc060ce162498 2025-12-17 23:55:35,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bfd575bc060ce162498 HTTP/1.1" 502 150 2025-12-17 23:55:35,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bfd575bc060ce162498'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51428 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:55:36,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bdf575bc060ce162447 2025-12-17 23:55:36,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bdf575bc060ce162447 HTTP/1.1" 502 150 2025-12-17 23:55:36,691 - lava_callback - ERROR - Error processing callback for node 69409bdf575bc060ce162447: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bdf575bc060ce162447 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bdf575bc060ce162447 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bdf575bc060ce162447 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:36,693 - lava_callback - INFO - Starting async job submit for node 69409c0a575bc060ce1624ba 2025-12-17 23:55:36,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:36,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bed575bc060ce16246e 2025-12-17 23:55:36,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bed575bc060ce16246e HTTP/1.1" 502 150 2025-12-17 23:55:36,751 - lava_callback - ERROR - Error processing callback for node 69409bed575bc060ce16246e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bed575bc060ce16246e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bed575bc060ce16246e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bed575bc060ce16246e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:36,753 - lava_callback - INFO - Starting async job submit for node 69409c0d575bc060ce1624c2 2025-12-17 23:55:36,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:36,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0d575bc060ce1624c2 HTTP/1.1" 502 150 2025-12-17 23:55:36,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0d575bc060ce1624c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:36,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0d575bc060ce1624c2 2025-12-17 23:55:36,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0d575bc060ce1624c2 HTTP/1.1" 502 150 2025-12-17 23:55:36,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0d575bc060ce1624c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:36,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bef575bc060ce162471 2025-12-17 23:55:36,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bef575bc060ce162471 HTTP/1.1" 502 150 2025-12-17 23:55:36,850 - lava_callback - ERROR - Error processing callback for node 69409bef575bc060ce162471: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bef575bc060ce162471 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bef575bc060ce162471 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bef575bc060ce162471 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:36,853 - lava_callback - INFO - Starting async job submit for node 6942ddc6e43d2b76e9022d35 2025-12-17 23:55:36,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:36,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942ddc6e43d2b76e9022d35 HTTP/1.1" 502 150 2025-12-17 23:55:36,868 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942ddc6e43d2b76e9022d35'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:36,868 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942ddc6e43d2b76e9022d35 2025-12-17 23:55:36,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942ddc6e43d2b76e9022d35 HTTP/1.1" 502 150 2025-12-17 23:55:36,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942ddc6e43d2b76e9022d35'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:36,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c05575bc060ce1624a4 2025-12-17 23:55:36,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c05575bc060ce1624a4 HTTP/1.1" 502 150 2025-12-17 23:55:36,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c05575bc060ce1624a4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:36,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0a575bc060ce1624ba HTTP/1.1" 502 150 2025-12-17 23:55:36,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0a575bc060ce1624ba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:36,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0a575bc060ce1624ba 2025-12-17 23:55:36,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0a575bc060ce1624ba HTTP/1.1" 502 150 2025-12-17 23:55:36,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0a575bc060ce1624ba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:37,984 - lava_callback - INFO - Received callback for node 69409ff2575bc060ce162c07 INFO: 172.21.0.1:51436 - "POST /node/69409ff2575bc060ce162c07 HTTP/1.0" 202 Accepted 2025-12-17 23:55:38,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf9575bc060ce16248e HTTP/1.1" 502 150 2025-12-17 23:55:38,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf9575bc060ce16248e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:38,638 - lava_callback - INFO - Received callback for node 69409ff3575bc060ce162c0e INFO: 172.21.0.1:51446 - "POST /node/69409ff3575bc060ce162c0e HTTP/1.0" 202 Accepted 2025-12-17 23:55:38,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0d575bc060ce1624c2 2025-12-17 23:55:38,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0d575bc060ce1624c2 HTTP/1.1" 502 150 2025-12-17 23:55:38,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0d575bc060ce1624c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:38,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942ddc6e43d2b76e9022d35 2025-12-17 23:55:38,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942ddc6e43d2b76e9022d35 HTTP/1.1" 502 150 2025-12-17 23:55:38,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942ddc6e43d2b76e9022d35'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:38,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0a575bc060ce1624ba 2025-12-17 23:55:38,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0a575bc060ce1624ba HTTP/1.1" 502 150 2025-12-17 23:55:38,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0a575bc060ce1624ba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:38,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c04575bc060ce1624a1 2025-12-17 23:55:38,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c04575bc060ce1624a1 HTTP/1.1" 502 150 2025-12-17 23:55:38,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c04575bc060ce1624a1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:39,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c02575bc060ce16249e 2025-12-17 23:55:39,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c02575bc060ce16249e HTTP/1.1" 502 150 2025-12-17 23:55:39,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c02575bc060ce16249e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:40,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf1575bc060ce162474 2025-12-17 23:55:40,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf1575bc060ce162474 HTTP/1.1" 502 150 2025-12-17 23:55:40,416 - lava_callback - ERROR - Error processing callback for node 69409bf1575bc060ce162474: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bf1575bc060ce162474 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bf1575bc060ce162474 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bf1575bc060ce162474 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:40,418 - lava_callback - INFO - Starting async job submit for node 69409c49575bc060ce162574 2025-12-17 23:55:40,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:40,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c49575bc060ce162574 HTTP/1.1" 502 150 2025-12-17 23:55:40,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c49575bc060ce162574'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:40,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c49575bc060ce162574 2025-12-17 23:55:40,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c49575bc060ce162574 HTTP/1.1" 502 150 2025-12-17 23:55:40,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c49575bc060ce162574'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:40,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf5575bc060ce162484 2025-12-17 23:55:40,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf5575bc060ce162484 HTTP/1.1" 502 150 2025-12-17 23:55:40,593 - lava_callback - ERROR - Error processing callback for node 69409bf5575bc060ce162484: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bf5575bc060ce162484 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bf5575bc060ce162484 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bf5575bc060ce162484 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:40,596 - lava_callback - INFO - Starting async job submit for node 69409c08575bc060ce1624b0 2025-12-17 23:55:40,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:40,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c08575bc060ce1624b0 HTTP/1.1" 502 150 2025-12-17 23:55:40,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c08575bc060ce1624b0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:40,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c08575bc060ce1624b0 2025-12-17 23:55:40,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c08575bc060ce1624b0 HTTP/1.1" 502 150 2025-12-17 23:55:40,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c08575bc060ce1624b0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:40,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c05575bc060ce1624a4 2025-12-17 23:55:40,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c05575bc060ce1624a4 HTTP/1.1" 502 150 2025-12-17 23:55:40,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c05575bc060ce1624a4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:42,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c49575bc060ce162574 2025-12-17 23:55:42,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c49575bc060ce162574 HTTP/1.1" 502 150 2025-12-17 23:55:42,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c49575bc060ce162574'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:42,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c08575bc060ce1624b0 2025-12-17 23:55:42,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c08575bc060ce1624b0 HTTP/1.1" 502 150 2025-12-17 23:55:42,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c08575bc060ce1624b0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:42,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0d575bc060ce1624c2 2025-12-17 23:55:42,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0d575bc060ce1624c2 HTTP/1.1" 502 150 2025-12-17 23:55:42,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0d575bc060ce1624c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:42,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942ddc6e43d2b76e9022d35 2025-12-17 23:55:42,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942ddc6e43d2b76e9022d35 HTTP/1.1" 502 150 2025-12-17 23:55:42,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942ddc6e43d2b76e9022d35'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:42,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0a575bc060ce1624ba 2025-12-17 23:55:42,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0a575bc060ce1624ba HTTP/1.1" 502 150 2025-12-17 23:55:42,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0a575bc060ce1624ba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:43,217 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c00575bc060ce16249b 2025-12-17 23:55:44,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf7575bc060ce16248b 2025-12-17 23:55:44,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf7575bc060ce16248b HTTP/1.1" 502 150 2025-12-17 23:55:44,595 - lava_callback - ERROR - Error processing callback for node 69409bf7575bc060ce16248b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bf7575bc060ce16248b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bf7575bc060ce16248b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bf7575bc060ce16248b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:44,597 - lava_callback - INFO - Starting async job submit for node 69409c0c575bc060ce1624bd 2025-12-17 23:55:44,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:44,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0c575bc060ce1624bd HTTP/1.1" 502 150 2025-12-17 23:55:44,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0c575bc060ce1624bd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:44,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0c575bc060ce1624bd 2025-12-17 23:55:44,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0c575bc060ce1624bd HTTP/1.1" 502 150 2025-12-17 23:55:44,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0c575bc060ce1624bd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:46,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c49575bc060ce162574 2025-12-17 23:55:46,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c49575bc060ce162574 HTTP/1.1" 502 150 2025-12-17 23:55:46,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c49575bc060ce162574'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37564 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:55:46,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf9575bc060ce16248e 2025-12-17 23:55:46,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf9575bc060ce16248e HTTP/1.1" 502 150 2025-12-17 23:55:46,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409bf9575bc060ce16248e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:46,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0c575bc060ce1624bd 2025-12-17 23:55:46,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0c575bc060ce1624bd HTTP/1.1" 502 150 2025-12-17 23:55:46,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0c575bc060ce1624bd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:46,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c08575bc060ce1624b0 2025-12-17 23:55:46,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c08575bc060ce1624b0 HTTP/1.1" 502 150 2025-12-17 23:55:46,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c08575bc060ce1624b0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:47,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bfb575bc060ce162491 2025-12-17 23:55:47,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bfb575bc060ce162491 HTTP/1.1" 502 150 2025-12-17 23:55:47,742 - lava_callback - ERROR - Error processing callback for node 69409bfb575bc060ce162491: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bfb575bc060ce162491 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bfb575bc060ce162491 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bfb575bc060ce162491 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:47,744 - lava_callback - INFO - Starting async job submit for node 69409c0f575bc060ce1624c5 2025-12-17 23:55:47,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:47,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0f575bc060ce1624c5 HTTP/1.1" 502 150 2025-12-17 23:55:47,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0f575bc060ce1624c5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:47,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0f575bc060ce1624c5 2025-12-17 23:55:47,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0f575bc060ce1624c5 HTTP/1.1" 502 150 2025-12-17 23:55:47,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0f575bc060ce1624c5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:47,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942d8e1e43d2b76e9021f27 2025-12-17 23:55:47,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942d8e1e43d2b76e9021f27 HTTP/1.1" 502 150 2025-12-17 23:55:47,928 - lava_callback - ERROR - Error processing callback for node 6942d8e1e43d2b76e9021f27: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942d8e1e43d2b76e9021f27 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942d8e1e43d2b76e9021f27 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942d8e1e43d2b76e9021f27 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:47,930 - lava_callback - INFO - Starting async job submit for node 69409c1a575bc060ce1624da 2025-12-17 23:55:47,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:47,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1a575bc060ce1624da HTTP/1.1" 502 150 2025-12-17 23:55:47,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1a575bc060ce1624da'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:47,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1a575bc060ce1624da 2025-12-17 23:55:47,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1a575bc060ce1624da HTTP/1.1" 502 150 2025-12-17 23:55:47,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1a575bc060ce1624da'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:47,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf3575bc060ce16247b 2025-12-17 23:55:47,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf3575bc060ce16247b HTTP/1.1" 502 150 2025-12-17 23:55:47,942 - lava_callback - ERROR - Error processing callback for node 69409bf3575bc060ce16247b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bf3575bc060ce16247b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bf3575bc060ce16247b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bf3575bc060ce16247b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:47,944 - lava_callback - INFO - Starting async job submit for node 69409c18575bc060ce1624d7 2025-12-17 23:55:47,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:47,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c18575bc060ce1624d7 HTTP/1.1" 502 150 2025-12-17 23:55:47,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c18575bc060ce1624d7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:47,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c18575bc060ce1624d7 2025-12-17 23:55:47,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c18575bc060ce1624d7 HTTP/1.1" 502 150 2025-12-17 23:55:47,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c18575bc060ce1624d7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:48,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be9575bc060ce162468 2025-12-17 23:55:48,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be9575bc060ce162468 HTTP/1.1" 502 150 2025-12-17 23:55:48,402 - lava_callback - ERROR - Error processing callback for node 69409be9575bc060ce162468: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409be9575bc060ce162468 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409be9575bc060ce162468 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409be9575bc060ce162468 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:48,404 - lava_callback - INFO - Starting async job submit for node 69409c10575bc060ce1624c8 2025-12-17 23:55:48,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:48,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c10575bc060ce1624c8 HTTP/1.1" 502 150 2025-12-17 23:55:48,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c10575bc060ce1624c8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:48,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c10575bc060ce1624c8 2025-12-17 23:55:48,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c10575bc060ce1624c8 HTTP/1.1" 502 150 2025-12-17 23:55:48,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c10575bc060ce1624c8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:48,809 - lava_callback - INFO - Received callback for node 69409fef575bc060ce162c00 INFO: 172.21.0.1:37578 - "POST /node/69409fef575bc060ce162c00 HTTP/1.0" 202 Accepted 2025-12-17 23:55:48,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c05575bc060ce1624a4 2025-12-17 23:55:48,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c05575bc060ce1624a4 HTTP/1.1" 502 150 2025-12-17 23:55:48,881 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c05575bc060ce1624a4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:49,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0f575bc060ce1624c5 2025-12-17 23:55:49,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0f575bc060ce1624c5 HTTP/1.1" 502 150 2025-12-17 23:55:49,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0f575bc060ce1624c5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:49,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1a575bc060ce1624da 2025-12-17 23:55:49,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1a575bc060ce1624da HTTP/1.1" 502 150 2025-12-17 23:55:49,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1a575bc060ce1624da'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:49,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c18575bc060ce1624d7 2025-12-17 23:55:49,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c18575bc060ce1624d7 HTTP/1.1" 502 150 2025-12-17 23:55:49,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c18575bc060ce1624d7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:50,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c10575bc060ce1624c8 2025-12-17 23:55:50,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c10575bc060ce1624c8 HTTP/1.1" 502 150 2025-12-17 23:55:50,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c10575bc060ce1624c8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:50,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0c575bc060ce1624bd 2025-12-17 23:55:50,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0c575bc060ce1624bd HTTP/1.1" 502 150 2025-12-17 23:55:50,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0c575bc060ce1624bd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:50,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0d575bc060ce1624c2 2025-12-17 23:55:50,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0d575bc060ce1624c2 HTTP/1.1" 502 150 2025-12-17 23:55:50,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0d575bc060ce1624c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:50,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942ddc6e43d2b76e9022d35 2025-12-17 23:55:50,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942ddc6e43d2b76e9022d35 HTTP/1.1" 502 150 2025-12-17 23:55:50,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6942ddc6e43d2b76e9022d35'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:50,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0a575bc060ce1624ba 2025-12-17 23:55:50,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0a575bc060ce1624ba HTTP/1.1" 502 150 2025-12-17 23:55:50,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0a575bc060ce1624ba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:51,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bfd575bc060ce162498 2025-12-17 23:55:51,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bfd575bc060ce162498 HTTP/1.1" 502 150 2025-12-17 23:55:51,934 - lava_callback - ERROR - Error processing callback for node 69409bfd575bc060ce162498: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bfd575bc060ce162498 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bfd575bc060ce162498 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bfd575bc060ce162498 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:51,935 - lava_callback - INFO - Starting async job submit for node 69409c12575bc060ce1624cb 2025-12-17 23:55:51,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:51,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c12575bc060ce1624cb HTTP/1.1" 502 150 2025-12-17 23:55:51,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c12575bc060ce1624cb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:51,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c12575bc060ce1624cb 2025-12-17 23:55:51,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c12575bc060ce1624cb HTTP/1.1" 502 150 2025-12-17 23:55:51,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c12575bc060ce1624cb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:52,994 - lava_callback - INFO - Received callback for node 69409ff7575bc060ce162c13 INFO: 172.21.0.1:42688 - "POST /node/69409ff7575bc060ce162c13 HTTP/1.0" 202 Accepted 2025-12-17 23:55:53,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0f575bc060ce1624c5 2025-12-17 23:55:53,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0f575bc060ce1624c5 HTTP/1.1" 502 150 2025-12-17 23:55:53,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0f575bc060ce1624c5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:53,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c12575bc060ce1624cb 2025-12-17 23:55:53,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c12575bc060ce1624cb HTTP/1.1" 502 150 2025-12-17 23:55:53,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c12575bc060ce1624cb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:53,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1a575bc060ce1624da 2025-12-17 23:55:53,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1a575bc060ce1624da HTTP/1.1" 502 150 2025-12-17 23:55:53,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1a575bc060ce1624da'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:53,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c18575bc060ce1624d7 2025-12-17 23:55:53,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c18575bc060ce1624d7 HTTP/1.1" 502 150 2025-12-17 23:55:53,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c18575bc060ce1624d7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:54,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c10575bc060ce1624c8 2025-12-17 23:55:54,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c10575bc060ce1624c8 HTTP/1.1" 502 150 2025-12-17 23:55:54,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c10575bc060ce1624c8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:54,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c49575bc060ce162574 2025-12-17 23:55:54,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c49575bc060ce162574 HTTP/1.1" 502 150 2025-12-17 23:55:54,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c49575bc060ce162574'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:54,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c08575bc060ce1624b0 2025-12-17 23:55:54,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c08575bc060ce1624b0 HTTP/1.1" 502 150 2025-12-17 23:55:54,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c08575bc060ce1624b0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:54,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c04575bc060ce1624a1 2025-12-17 23:55:54,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c04575bc060ce1624a1 HTTP/1.1" 502 150 2025-12-17 23:55:54,952 - lava_callback - ERROR - Error processing callback for node 69409c04575bc060ce1624a1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c04575bc060ce1624a1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c04575bc060ce1624a1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c04575bc060ce1624a1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:54,953 - lava_callback - INFO - Starting async job submit for node 69409c14575bc060ce1624d1 2025-12-17 23:55:54,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:54,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c14575bc060ce1624d1 HTTP/1.1" 502 150 2025-12-17 23:55:54,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c14575bc060ce1624d1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:54,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c14575bc060ce1624d1 2025-12-17 23:55:54,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c14575bc060ce1624d1 HTTP/1.1" 502 150 2025-12-17 23:55:54,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c14575bc060ce1624d1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:55,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c02575bc060ce16249e 2025-12-17 23:55:55,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c02575bc060ce16249e HTTP/1.1" 502 150 2025-12-17 23:55:55,948 - lava_callback - ERROR - Error processing callback for node 69409c02575bc060ce16249e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c02575bc060ce16249e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c02575bc060ce16249e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c02575bc060ce16249e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:55:55,950 - lava_callback - INFO - Starting async job submit for node 69409c16575bc060ce1624d4 2025-12-17 23:55:55,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:55:55,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c16575bc060ce1624d4 HTTP/1.1" 502 150 2025-12-17 23:55:55,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c16575bc060ce1624d4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:55,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c16575bc060ce1624d4 2025-12-17 23:55:55,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c16575bc060ce1624d4 HTTP/1.1" 502 150 2025-12-17 23:55:55,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c16575bc060ce1624d4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42694 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:55:56,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c14575bc060ce1624d1 2025-12-17 23:55:56,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c14575bc060ce1624d1 HTTP/1.1" 502 150 2025-12-17 23:55:56,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c14575bc060ce1624d1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:57,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c12575bc060ce1624cb 2025-12-17 23:55:57,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c12575bc060ce1624cb HTTP/1.1" 502 150 2025-12-17 23:55:57,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c12575bc060ce1624cb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:57,972 - lava_callback - INFO - Received callback for node 6940a005575bc060ce162c23 INFO: 172.21.0.1:42702 - "POST /node/6940a005575bc060ce162c23 HTTP/1.0" 202 Accepted 2025-12-17 23:55:57,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c16575bc060ce1624d4 2025-12-17 23:55:57,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c16575bc060ce1624d4 HTTP/1.1" 502 150 2025-12-17 23:55:57,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c16575bc060ce1624d4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:58,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c00575bc060ce16249b HTTP/1.1" 502 150 2025-12-17 23:55:58,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c00575bc060ce16249b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:58,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0c575bc060ce1624bd 2025-12-17 23:55:58,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0c575bc060ce1624bd HTTP/1.1" 502 150 2025-12-17 23:55:58,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0c575bc060ce1624bd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:55:58,618 - lava_callback - INFO - Received callback for node 6940a009575bc060ce162c39 INFO: 172.21.0.1:42710 - "POST /node/6940a009575bc060ce162c39 HTTP/1.0" 202 Accepted 2025-12-17 23:55:59,870 - lava_callback - INFO - Received callback for node 6940a7751e3c24e956819ea5 INFO: 172.21.0.1:56932 - "POST /node/6940a7751e3c24e956819ea5 HTTP/1.0" 202 Accepted 2025-12-17 23:56:00,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c14575bc060ce1624d1 2025-12-17 23:56:01,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0f575bc060ce1624c5 2025-12-17 23:56:01,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0f575bc060ce1624c5 HTTP/1.1" 502 150 2025-12-17 23:56:01,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c0f575bc060ce1624c5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:01,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1a575bc060ce1624da 2025-12-17 23:56:01,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1a575bc060ce1624da HTTP/1.1" 502 150 2025-12-17 23:56:01,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1a575bc060ce1624da'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:01,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c18575bc060ce1624d7 2025-12-17 23:56:01,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c18575bc060ce1624d7 HTTP/1.1" 502 150 2025-12-17 23:56:01,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c18575bc060ce1624d7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:01,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c16575bc060ce1624d4 2025-12-17 23:56:01,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c16575bc060ce1624d4 HTTP/1.1" 502 150 2025-12-17 23:56:01,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c16575bc060ce1624d4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:02,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c10575bc060ce1624c8 2025-12-17 23:56:02,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c10575bc060ce1624c8 HTTP/1.1" 502 150 2025-12-17 23:56:02,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c10575bc060ce1624c8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:02,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409bf9575bc060ce16248e 2025-12-17 23:56:02,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409bf9575bc060ce16248e HTTP/1.1" 502 150 2025-12-17 23:56:02,547 - lava_callback - ERROR - Error processing callback for node 69409bf9575bc060ce16248e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bf9575bc060ce16248e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bf9575bc060ce16248e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409bf9575bc060ce16248e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:02,549 - lava_callback - INFO - Starting async job submit for node 69409c1c575bc060ce1624e3 2025-12-17 23:56:02,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:02,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1c575bc060ce1624e3 HTTP/1.1" 502 150 2025-12-17 23:56:02,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1c575bc060ce1624e3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:02,555 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1c575bc060ce1624e3 2025-12-17 23:56:02,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1c575bc060ce1624e3 HTTP/1.1" 502 150 2025-12-17 23:56:02,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1c575bc060ce1624e3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56946 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:56:04,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1c575bc060ce1624e3 2025-12-17 23:56:04,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1c575bc060ce1624e3 HTTP/1.1" 502 150 2025-12-17 23:56:04,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1c575bc060ce1624e3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:04,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c05575bc060ce1624a4 2025-12-17 23:56:04,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c05575bc060ce1624a4 HTTP/1.1" 502 150 2025-12-17 23:56:04,883 - lava_callback - ERROR - Error processing callback for node 69409c05575bc060ce1624a4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c05575bc060ce1624a4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c05575bc060ce1624a4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c05575bc060ce1624a4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:04,885 - lava_callback - INFO - Starting async job submit for node 69409c20575bc060ce1624f4 2025-12-17 23:56:04,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:04,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c20575bc060ce1624f4 HTTP/1.1" 502 150 2025-12-17 23:56:04,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c20575bc060ce1624f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:04,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c20575bc060ce1624f4 2025-12-17 23:56:04,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c20575bc060ce1624f4 HTTP/1.1" 502 150 2025-12-17 23:56:04,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c20575bc060ce1624f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:05,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c12575bc060ce1624cb 2025-12-17 23:56:05,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c12575bc060ce1624cb HTTP/1.1" 502 150 2025-12-17 23:56:05,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c12575bc060ce1624cb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56958 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:56:06,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0d575bc060ce1624c2 2025-12-17 23:56:06,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0d575bc060ce1624c2 HTTP/1.1" 502 150 2025-12-17 23:56:06,768 - lava_callback - ERROR - Error processing callback for node 69409c0d575bc060ce1624c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c0d575bc060ce1624c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c0d575bc060ce1624c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c0d575bc060ce1624c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:06,770 - lava_callback - INFO - Starting async job submit for node 69409c1d575bc060ce1624ea 2025-12-17 23:56:06,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:06,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1d575bc060ce1624ea HTTP/1.1" 502 150 2025-12-17 23:56:06,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1d575bc060ce1624ea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:06,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1d575bc060ce1624ea 2025-12-17 23:56:06,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1d575bc060ce1624ea HTTP/1.1" 502 150 2025-12-17 23:56:06,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1d575bc060ce1624ea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:06,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6942ddc6e43d2b76e9022d35 2025-12-17 23:56:06,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6942ddc6e43d2b76e9022d35 HTTP/1.1" 502 150 2025-12-17 23:56:06,875 - lava_callback - ERROR - Error processing callback for node 6942ddc6e43d2b76e9022d35: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942ddc6e43d2b76e9022d35 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942ddc6e43d2b76e9022d35 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6942ddc6e43d2b76e9022d35 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:06,877 - lava_callback - INFO - Starting async job submit for node 69409c22575bc060ce1624fb 2025-12-17 23:56:06,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:06,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c22575bc060ce1624fb HTTP/1.1" 502 150 2025-12-17 23:56:06,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c22575bc060ce1624fb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:06,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c22575bc060ce1624fb 2025-12-17 23:56:06,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c22575bc060ce1624fb HTTP/1.1" 502 150 2025-12-17 23:56:06,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c22575bc060ce1624fb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:06,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0a575bc060ce1624ba 2025-12-17 23:56:06,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0a575bc060ce1624ba HTTP/1.1" 502 150 2025-12-17 23:56:06,888 - lava_callback - ERROR - Error processing callback for node 69409c0a575bc060ce1624ba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c0a575bc060ce1624ba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c0a575bc060ce1624ba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c0a575bc060ce1624ba (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:06,890 - lava_callback - INFO - Starting async job submit for node 69409c1f575bc060ce1624f1 2025-12-17 23:56:06,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:06,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c20575bc060ce1624f4 2025-12-17 23:56:06,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c20575bc060ce1624f4 HTTP/1.1" 502 150 2025-12-17 23:56:06,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c20575bc060ce1624f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:06,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1f575bc060ce1624f1 HTTP/1.1" 502 150 2025-12-17 23:56:06,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1f575bc060ce1624f1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:06,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1f575bc060ce1624f1 2025-12-17 23:56:06,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1f575bc060ce1624f1 HTTP/1.1" 502 150 2025-12-17 23:56:06,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1f575bc060ce1624f1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:08,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1c575bc060ce1624e3 2025-12-17 23:56:08,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1c575bc060ce1624e3 HTTP/1.1" 502 150 2025-12-17 23:56:08,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1c575bc060ce1624e3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:08,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1d575bc060ce1624ea 2025-12-17 23:56:08,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1d575bc060ce1624ea HTTP/1.1" 502 150 2025-12-17 23:56:08,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1d575bc060ce1624ea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:08,830 - lava_callback - INFO - Received callback for node 6940a003575bc060ce162c18 INFO: 172.21.0.1:56972 - "POST /node/6940a003575bc060ce162c18 HTTP/1.0" 202 Accepted 2025-12-17 23:56:08,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c22575bc060ce1624fb 2025-12-17 23:56:08,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c22575bc060ce1624fb HTTP/1.1" 502 150 2025-12-17 23:56:08,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c22575bc060ce1624fb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:08,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1f575bc060ce1624f1 2025-12-17 23:56:08,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1f575bc060ce1624f1 HTTP/1.1" 502 150 2025-12-17 23:56:08,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1f575bc060ce1624f1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:09,475 - lava_callback - INFO - Received callback for node 6940a007575bc060ce162c2e INFO: 172.21.0.1:56976 - "POST /node/6940a007575bc060ce162c2e HTTP/1.0" 202 Accepted 2025-12-17 23:56:09,988 - lava_callback - INFO - Received callback for node 6941359e459ff5fc64d2c36f 2025-12-17 23:56:09,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c16575bc060ce1624d4 INFO: 172.21.0.1:41518 - "POST /node/6941359e459ff5fc64d2c36f HTTP/1.0" 202 Accepted 2025-12-17 23:56:09,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c16575bc060ce1624d4 HTTP/1.1" 502 150 2025-12-17 23:56:09,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c16575bc060ce1624d4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:10,144 - lava_callback - INFO - Received callback for node 6940a00a575bc060ce162c3c INFO: 172.21.0.1:41530 - "POST /node/6940a00a575bc060ce162c3c HTTP/1.0" 202 Accepted 2025-12-17 23:56:10,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c49575bc060ce162574 2025-12-17 23:56:10,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c49575bc060ce162574 HTTP/1.1" 502 150 2025-12-17 23:56:10,535 - lava_callback - ERROR - Error processing callback for node 69409c49575bc060ce162574: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c49575bc060ce162574 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c49575bc060ce162574 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c49575bc060ce162574 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:10,538 - lava_callback - INFO - Starting async job submit for node 69409c26575bc060ce162505 2025-12-17 23:56:10,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:10,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c26575bc060ce162505 HTTP/1.1" 502 150 2025-12-17 23:56:10,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c26575bc060ce162505'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:10,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c26575bc060ce162505 2025-12-17 23:56:10,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c26575bc060ce162505 HTTP/1.1" 502 150 2025-12-17 23:56:10,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c26575bc060ce162505'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:10,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c08575bc060ce1624b0 2025-12-17 23:56:10,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c08575bc060ce1624b0 HTTP/1.1" 502 150 2025-12-17 23:56:10,611 - lava_callback - ERROR - Error processing callback for node 69409c08575bc060ce1624b0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c08575bc060ce1624b0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c08575bc060ce1624b0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c08575bc060ce1624b0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:10,613 - lava_callback - INFO - Starting async job submit for node 69409c2a575bc060ce16250e 2025-12-17 23:56:10,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:10,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2a575bc060ce16250e HTTP/1.1" 502 150 2025-12-17 23:56:10,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2a575bc060ce16250e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:10,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2a575bc060ce16250e 2025-12-17 23:56:10,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2a575bc060ce16250e HTTP/1.1" 502 150 2025-12-17 23:56:10,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2a575bc060ce16250e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:10,779 - lava_callback - INFO - Received callback for node 6940a00b575bc060ce162c3f INFO: 172.21.0.1:41540 - "POST /node/6940a00b575bc060ce162c3f HTTP/1.0" 202 Accepted 2025-12-17 23:56:10,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c20575bc060ce1624f4 2025-12-17 23:56:10,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c20575bc060ce1624f4 HTTP/1.1" 502 150 2025-12-17 23:56:10,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c20575bc060ce1624f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:11,423 - lava_callback - INFO - Received callback for node 6940a00c575bc060ce162c42 INFO: 172.21.0.1:41544 - "POST /node/6940a00c575bc060ce162c42 HTTP/1.0" 202 Accepted 2025-12-17 23:56:12,066 - lava_callback - INFO - Received callback for node 6940a00e575bc060ce162c45 INFO: 172.21.0.1:41558 - "POST /node/6940a00e575bc060ce162c45 HTTP/1.0" 202 Accepted 2025-12-17 23:56:12,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c26575bc060ce162505 2025-12-17 23:56:12,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c26575bc060ce162505 HTTP/1.1" 502 150 2025-12-17 23:56:12,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c26575bc060ce162505'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:12,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2a575bc060ce16250e 2025-12-17 23:56:12,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2a575bc060ce16250e HTTP/1.1" 502 150 2025-12-17 23:56:12,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2a575bc060ce16250e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:12,714 - lava_callback - INFO - Received callback for node 6940a00f575bc060ce162c48 INFO: 172.21.0.1:41564 - "POST /node/6940a00f575bc060ce162c48 HTTP/1.0" 202 Accepted 2025-12-17 23:56:12,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1d575bc060ce1624ea 2025-12-17 23:56:12,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1d575bc060ce1624ea HTTP/1.1" 502 150 2025-12-17 23:56:12,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1d575bc060ce1624ea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:12,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c22575bc060ce1624fb 2025-12-17 23:56:12,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c22575bc060ce1624fb HTTP/1.1" 502 150 2025-12-17 23:56:12,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c22575bc060ce1624fb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:12,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1f575bc060ce1624f1 2025-12-17 23:56:12,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1f575bc060ce1624f1 HTTP/1.1" 502 150 2025-12-17 23:56:12,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1f575bc060ce1624f1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:13,362 - lava_callback - INFO - Received callback for node 6940a010575bc060ce162c4b INFO: 172.21.0.1:41570 - "POST /node/6940a010575bc060ce162c4b HTTP/1.0" 202 Accepted 2025-12-17 23:56:14,012 - lava_callback - INFO - Received callback for node 6940a011575bc060ce162c4f INFO: 172.21.0.1:41582 - "POST /node/6940a011575bc060ce162c4f HTTP/1.0" 202 Accepted 2025-12-17 23:56:14,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c00575bc060ce16249b 2025-12-17 23:56:14,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c00575bc060ce16249b HTTP/1.1" 502 150 2025-12-17 23:56:14,514 - lava_callback - ERROR - Error processing callback for node 69409c00575bc060ce16249b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c00575bc060ce16249b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c00575bc060ce16249b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c00575bc060ce16249b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:14,515 - lava_callback - INFO - Starting async job submit for node 69409c2d575bc060ce162514 2025-12-17 23:56:14,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:14,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2d575bc060ce162514 HTTP/1.1" 502 150 2025-12-17 23:56:14,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2d575bc060ce162514'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:14,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2d575bc060ce162514 2025-12-17 23:56:14,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2d575bc060ce162514 HTTP/1.1" 502 150 2025-12-17 23:56:14,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2d575bc060ce162514'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:14,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0c575bc060ce1624bd 2025-12-17 23:56:14,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0c575bc060ce1624bd HTTP/1.1" 502 150 2025-12-17 23:56:14,612 - lava_callback - ERROR - Error processing callback for node 69409c0c575bc060ce1624bd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c0c575bc060ce1624bd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c0c575bc060ce1624bd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c0c575bc060ce1624bd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:14,615 - lava_callback - INFO - Starting async job submit for node 69409c31575bc060ce162522 2025-12-17 23:56:14,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:14,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c31575bc060ce162522 HTTP/1.1" 502 150 2025-12-17 23:56:14,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c31575bc060ce162522'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:14,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c31575bc060ce162522 2025-12-17 23:56:14,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c31575bc060ce162522 HTTP/1.1" 502 150 2025-12-17 23:56:14,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c31575bc060ce162522'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41598 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-17 23:56:16,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c14575bc060ce1624d1 HTTP/1.1" 502 150 2025-12-17 23:56:16,432 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c14575bc060ce1624d1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41606 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:56:16,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2d575bc060ce162514 2025-12-17 23:56:16,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2d575bc060ce162514 HTTP/1.1" 502 150 2025-12-17 23:56:16,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2d575bc060ce162514'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:16,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c26575bc060ce162505 2025-12-17 23:56:16,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c26575bc060ce162505 HTTP/1.1" 502 150 2025-12-17 23:56:16,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c26575bc060ce162505'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:16,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1c575bc060ce1624e3 2025-12-17 23:56:16,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1c575bc060ce1624e3 HTTP/1.1" 502 150 2025-12-17 23:56:16,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1c575bc060ce1624e3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:16,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c31575bc060ce162522 2025-12-17 23:56:16,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2a575bc060ce16250e 2025-12-17 23:56:16,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c31575bc060ce162522 HTTP/1.1" 502 150 2025-12-17 23:56:16,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c31575bc060ce162522'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:16,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2a575bc060ce16250e HTTP/1.1" 502 150 2025-12-17 23:56:16,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2a575bc060ce16250e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:17,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c0f575bc060ce1624c5 2025-12-17 23:56:17,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c0f575bc060ce1624c5 HTTP/1.1" 502 150 2025-12-17 23:56:17,758 - lava_callback - ERROR - Error processing callback for node 69409c0f575bc060ce1624c5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c0f575bc060ce1624c5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c0f575bc060ce1624c5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c0f575bc060ce1624c5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:17,760 - lava_callback - INFO - Starting async job submit for node 69409be5575bc060ce16245e 2025-12-17 23:56:17,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:17,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be5575bc060ce16245e HTTP/1.1" 502 150 2025-12-17 23:56:17,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be5575bc060ce16245e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:17,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be5575bc060ce16245e 2025-12-17 23:56:17,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be5575bc060ce16245e HTTP/1.1" 502 150 2025-12-17 23:56:17,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be5575bc060ce16245e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:17,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1a575bc060ce1624da 2025-12-17 23:56:17,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1a575bc060ce1624da HTTP/1.1" 502 150 2025-12-17 23:56:17,953 - lava_callback - ERROR - Error processing callback for node 69409c1a575bc060ce1624da: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c1a575bc060ce1624da (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c1a575bc060ce1624da (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c1a575bc060ce1624da (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:17,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c18575bc060ce1624d7 2025-12-17 23:56:17,955 - lava_callback - INFO - Starting async job submit for node 69409c24575bc060ce1624fe 2025-12-17 23:56:17,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:17,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c18575bc060ce1624d7 HTTP/1.1" 502 150 2025-12-17 23:56:17,956 - lava_callback - ERROR - Error processing callback for node 69409c18575bc060ce1624d7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c18575bc060ce1624d7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c18575bc060ce1624d7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c18575bc060ce1624d7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:17,958 - lava_callback - INFO - Starting async job submit for node 69409c28575bc060ce16250b 2025-12-17 23:56:17,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:17,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c24575bc060ce1624fe HTTP/1.1" 502 150 2025-12-17 23:56:17,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c24575bc060ce1624fe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:17,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c24575bc060ce1624fe 2025-12-17 23:56:17,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c24575bc060ce1624fe HTTP/1.1" 502 150 2025-12-17 23:56:17,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c24575bc060ce1624fe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:18,076 - lava_callback - INFO - Received callback for node 6940a019575bc060ce162c63 INFO: 172.21.0.1:41622 - "POST /node/6940a019575bc060ce162c63 HTTP/1.0" 202 Accepted 2025-12-17 23:56:18,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c10575bc060ce1624c8 2025-12-17 23:56:18,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c10575bc060ce1624c8 HTTP/1.1" 502 150 2025-12-17 23:56:18,417 - lava_callback - ERROR - Error processing callback for node 69409c10575bc060ce1624c8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c10575bc060ce1624c8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c10575bc060ce1624c8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c10575bc060ce1624c8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:18,419 - lava_callback - INFO - Starting async job submit for node 6940a2d9575bc060ce163100 2025-12-17 23:56:18,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:18,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a2d9575bc060ce163100 HTTP/1.1" 502 150 2025-12-17 23:56:18,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a2d9575bc060ce163100'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:18,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a2d9575bc060ce163100 2025-12-17 23:56:18,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a2d9575bc060ce163100 HTTP/1.1" 502 150 2025-12-17 23:56:18,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a2d9575bc060ce163100'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:18,748 - lava_callback - INFO - Received callback for node 6940a01d575bc060ce162c71 INFO: 172.21.0.1:41636 - "POST /node/6940a01d575bc060ce162c71 HTTP/1.0" 202 Accepted 2025-12-17 23:56:18,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c20575bc060ce1624f4 2025-12-17 23:56:18,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c20575bc060ce1624f4 HTTP/1.1" 502 150 2025-12-17 23:56:18,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c20575bc060ce1624f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:19,402 - lava_callback - INFO - Received callback for node 6940a01f575bc060ce162c77 INFO: 172.21.0.1:41638 - "POST /node/6940a01f575bc060ce162c77 HTTP/1.0" 202 Accepted 2025-12-17 23:56:19,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be5575bc060ce16245e 2025-12-17 23:56:19,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be5575bc060ce16245e HTTP/1.1" 502 150 2025-12-17 23:56:19,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be5575bc060ce16245e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:19,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c24575bc060ce1624fe 2025-12-17 23:56:19,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c24575bc060ce1624fe HTTP/1.1" 502 150 2025-12-17 23:56:19,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c24575bc060ce1624fe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:20,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a2d9575bc060ce163100 2025-12-17 23:56:20,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2d575bc060ce162514 2025-12-17 23:56:20,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2d575bc060ce162514 HTTP/1.1" 502 150 2025-12-17 23:56:20,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2d575bc060ce162514'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:20,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c31575bc060ce162522 2025-12-17 23:56:20,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c31575bc060ce162522 HTTP/1.1" 502 150 2025-12-17 23:56:20,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c31575bc060ce162522'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:20,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1d575bc060ce1624ea 2025-12-17 23:56:20,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1d575bc060ce1624ea HTTP/1.1" 502 150 2025-12-17 23:56:20,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1d575bc060ce1624ea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:20,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c22575bc060ce1624fb 2025-12-17 23:56:20,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c22575bc060ce1624fb HTTP/1.1" 502 150 2025-12-17 23:56:20,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c22575bc060ce1624fb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:20,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1f575bc060ce1624f1 2025-12-17 23:56:20,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1f575bc060ce1624f1 HTTP/1.1" 502 150 2025-12-17 23:56:20,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c1f575bc060ce1624f1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:21,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c12575bc060ce1624cb 2025-12-17 23:56:21,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c12575bc060ce1624cb HTTP/1.1" 502 150 2025-12-17 23:56:21,949 - lava_callback - ERROR - Error processing callback for node 69409c12575bc060ce1624cb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c12575bc060ce1624cb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c12575bc060ce1624cb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c12575bc060ce1624cb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:21,952 - lava_callback - INFO - Starting async job submit for node 69409c2b575bc060ce162511 2025-12-17 23:56:21,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:21,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2b575bc060ce162511 HTTP/1.1" 502 150 2025-12-17 23:56:21,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2b575bc060ce162511'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:21,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2b575bc060ce162511 2025-12-17 23:56:21,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2b575bc060ce162511 HTTP/1.1" 502 150 2025-12-17 23:56:21,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2b575bc060ce162511'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:23,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be5575bc060ce16245e 2025-12-17 23:56:23,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be5575bc060ce16245e HTTP/1.1" 502 150 2025-12-17 23:56:23,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be5575bc060ce16245e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:23,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c24575bc060ce1624fe 2025-12-17 23:56:23,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2b575bc060ce162511 2025-12-17 23:56:23,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2b575bc060ce162511 HTTP/1.1" 502 150 2025-12-17 23:56:23,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2b575bc060ce162511'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:23,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c24575bc060ce1624fe HTTP/1.1" 502 150 2025-12-17 23:56:23,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c24575bc060ce1624fe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:24,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c14575bc060ce1624d1 2025-12-17 23:56:24,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c14575bc060ce1624d1 HTTP/1.1" 502 150 2025-12-17 23:56:24,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c14575bc060ce1624d1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:24,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c26575bc060ce162505 2025-12-17 23:56:24,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c26575bc060ce162505 HTTP/1.1" 502 150 2025-12-17 23:56:24,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c26575bc060ce162505'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:24,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2a575bc060ce16250e 2025-12-17 23:56:24,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2a575bc060ce16250e HTTP/1.1" 502 150 2025-12-17 23:56:24,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2a575bc060ce16250e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:25,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c16575bc060ce1624d4 2025-12-17 23:56:25,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c16575bc060ce1624d4 HTTP/1.1" 502 150 2025-12-17 23:56:25,999 - lava_callback - ERROR - Error processing callback for node 69409c16575bc060ce1624d4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c16575bc060ce1624d4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c16575bc060ce1624d4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c16575bc060ce1624d4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:26,001 - lava_callback - INFO - Starting async job submit for node 6940a36d575bc060ce1631b0 2025-12-17 23:56:26,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:26,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a36d575bc060ce1631b0 HTTP/1.1" 502 150 2025-12-17 23:56:26,066 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a36d575bc060ce1631b0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:26,066 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a36d575bc060ce1631b0 2025-12-17 23:56:26,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a36d575bc060ce1631b0 HTTP/1.1" 502 150 2025-12-17 23:56:26,066 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a36d575bc060ce1631b0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59970 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:56:27,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2b575bc060ce162511 2025-12-17 23:56:27,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2b575bc060ce162511 HTTP/1.1" 502 150 2025-12-17 23:56:27,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2b575bc060ce162511'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:28,067 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a36d575bc060ce1631b0 2025-12-17 23:56:28,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a36d575bc060ce1631b0 HTTP/1.1" 502 150 2025-12-17 23:56:28,068 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a36d575bc060ce1631b0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:28,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2d575bc060ce162514 2025-12-17 23:56:28,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2d575bc060ce162514 HTTP/1.1" 502 150 2025-12-17 23:56:28,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2d575bc060ce162514'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:28,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c31575bc060ce162522 2025-12-17 23:56:28,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c31575bc060ce162522 HTTP/1.1" 502 150 2025-12-17 23:56:28,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c31575bc060ce162522'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:28,838 - lava_callback - INFO - Received callback for node 6940a017575bc060ce162c54 INFO: 172.21.0.1:59980 - "POST /node/6940a017575bc060ce162c54 HTTP/1.0" 202 Accepted 2025-12-17 23:56:29,482 - lava_callback - INFO - Received callback for node 6940a01b575bc060ce162c6e INFO: 172.21.0.1:59996 - "POST /node/6940a01b575bc060ce162c6e HTTP/1.0" 202 Accepted 2025-12-17 23:56:30,126 - lava_callback - INFO - Received callback for node 6940a01e575bc060ce162c74 INFO: 172.21.0.1:36248 - "POST /node/6940a01e575bc060ce162c74 HTTP/1.0" 202 Accepted 2025-12-17 23:56:30,776 - lava_callback - INFO - Received callback for node 6940a020575bc060ce162c7a INFO: 172.21.0.1:36262 - "POST /node/6940a020575bc060ce162c7a HTTP/1.0" 202 Accepted 2025-12-17 23:56:31,438 - lava_callback - INFO - Received callback for node 6940a022575bc060ce162c80 INFO: 172.21.0.1:36274 - "POST /node/6940a022575bc060ce162c80 HTTP/1.0" 202 Accepted 2025-12-17 23:56:31,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be5575bc060ce16245e 2025-12-17 23:56:31,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c24575bc060ce1624fe 2025-12-17 23:56:31,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c24575bc060ce1624fe HTTP/1.1" 502 150 2025-12-17 23:56:31,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c24575bc060ce1624fe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:32,068 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a36d575bc060ce1631b0 2025-12-17 23:56:32,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a36d575bc060ce1631b0 HTTP/1.1" 502 150 2025-12-17 23:56:32,069 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a36d575bc060ce1631b0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:32,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1c575bc060ce1624e3 2025-12-17 23:56:32,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1c575bc060ce1624e3 HTTP/1.1" 502 150 2025-12-17 23:56:32,562 - lava_callback - ERROR - Error processing callback for node 69409c1c575bc060ce1624e3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c1c575bc060ce1624e3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c1c575bc060ce1624e3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c1c575bc060ce1624e3 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:32,563 - lava_callback - INFO - Starting async job submit for node 69409c2e575bc060ce162517 2025-12-17 23:56:32,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:32,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2e575bc060ce162517 HTTP/1.1" 502 150 2025-12-17 23:56:32,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2e575bc060ce162517'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:32,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2e575bc060ce162517 2025-12-17 23:56:32,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2e575bc060ce162517 HTTP/1.1" 502 150 2025-12-17 23:56:32,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2e575bc060ce162517'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:34,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2e575bc060ce162517 2025-12-17 23:56:34,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2e575bc060ce162517 HTTP/1.1" 502 150 2025-12-17 23:56:34,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2e575bc060ce162517'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:34,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c20575bc060ce1624f4 2025-12-17 23:56:34,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c20575bc060ce1624f4 HTTP/1.1" 502 150 2025-12-17 23:56:34,899 - lava_callback - ERROR - Error processing callback for node 69409c20575bc060ce1624f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c20575bc060ce1624f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c20575bc060ce1624f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c20575bc060ce1624f4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:34,900 - lava_callback - INFO - Starting async job submit for node 69409c32575bc060ce162527 2025-12-17 23:56:34,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:34,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c32575bc060ce162527 HTTP/1.1" 502 150 2025-12-17 23:56:34,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c32575bc060ce162527'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:34,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c32575bc060ce162527 2025-12-17 23:56:34,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c32575bc060ce162527 HTTP/1.1" 502 150 2025-12-17 23:56:34,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c32575bc060ce162527'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:35,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a2d9575bc060ce163100 HTTP/1.1" 502 150 2025-12-17 23:56:35,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a2d9575bc060ce163100'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:35,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2b575bc060ce162511 2025-12-17 23:56:35,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2b575bc060ce162511 HTTP/1.1" 502 150 2025-12-17 23:56:35,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2b575bc060ce162511'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36288 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:56:36,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1d575bc060ce1624ea 2025-12-17 23:56:36,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1d575bc060ce1624ea HTTP/1.1" 502 150 2025-12-17 23:56:36,784 - lava_callback - ERROR - Error processing callback for node 69409c1d575bc060ce1624ea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c1d575bc060ce1624ea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c1d575bc060ce1624ea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c1d575bc060ce1624ea (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:36,786 - lava_callback - INFO - Starting async job submit for node 69409c34575bc060ce16252e 2025-12-17 23:56:36,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:36,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c34575bc060ce16252e HTTP/1.1" 502 150 2025-12-17 23:56:36,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c34575bc060ce16252e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:36,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c34575bc060ce16252e 2025-12-17 23:56:36,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c34575bc060ce16252e HTTP/1.1" 502 150 2025-12-17 23:56:36,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c34575bc060ce16252e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:36,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c22575bc060ce1624fb 2025-12-17 23:56:36,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c22575bc060ce1624fb HTTP/1.1" 502 150 2025-12-17 23:56:36,893 - lava_callback - ERROR - Error processing callback for node 69409c22575bc060ce1624fb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c22575bc060ce1624fb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c22575bc060ce1624fb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c22575bc060ce1624fb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:36,895 - lava_callback - INFO - Starting async job submit for node 69409c35575bc060ce162531 2025-12-17 23:56:36,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:36,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c35575bc060ce162531 HTTP/1.1" 502 150 2025-12-17 23:56:36,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c35575bc060ce162531'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:36,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c35575bc060ce162531 2025-12-17 23:56:36,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c35575bc060ce162531 HTTP/1.1" 502 150 2025-12-17 23:56:36,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c35575bc060ce162531'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:36,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c32575bc060ce162527 2025-12-17 23:56:36,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c32575bc060ce162527 HTTP/1.1" 502 150 2025-12-17 23:56:36,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c32575bc060ce162527'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:36,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c1f575bc060ce1624f1 2025-12-17 23:56:36,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c1f575bc060ce1624f1 HTTP/1.1" 502 150 2025-12-17 23:56:36,912 - lava_callback - ERROR - Error processing callback for node 69409c1f575bc060ce1624f1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c1f575bc060ce1624f1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c1f575bc060ce1624f1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c1f575bc060ce1624f1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:36,914 - lava_callback - INFO - Starting async job submit for node 69409c37575bc060ce162534 2025-12-17 23:56:36,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:38,022 - lava_callback - INFO - Received callback for node 6940a021575bc060ce162c7d INFO: 172.21.0.1:36292 - "POST /node/6940a021575bc060ce162c7d HTTP/1.0" 202 Accepted 2025-12-17 23:56:38,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2e575bc060ce162517 2025-12-17 23:56:38,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2e575bc060ce162517 HTTP/1.1" 502 150 2025-12-17 23:56:38,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2e575bc060ce162517'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:38,675 - lava_callback - INFO - Received callback for node 6940a024575bc060ce162c83 INFO: 172.21.0.1:36302 - "POST /node/6940a024575bc060ce162c83 HTTP/1.0" 202 Accepted 2025-12-17 23:56:38,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c34575bc060ce16252e 2025-12-17 23:56:38,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c34575bc060ce16252e HTTP/1.1" 502 150 2025-12-17 23:56:38,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c34575bc060ce16252e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:38,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c35575bc060ce162531 2025-12-17 23:56:38,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c35575bc060ce162531 HTTP/1.1" 502 150 2025-12-17 23:56:38,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c35575bc060ce162531'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:39,363 - lava_callback - INFO - Received callback for node 6940a025575bc060ce162c86 INFO: 172.21.0.1:36310 - "POST /node/6940a025575bc060ce162c86 HTTP/1.0" 202 Accepted 2025-12-17 23:56:39,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a2d9575bc060ce163100 2025-12-17 23:56:39,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a2d9575bc060ce163100 HTTP/1.1" 502 150 2025-12-17 23:56:39,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a2d9575bc060ce163100'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:40,016 - lava_callback - INFO - Received callback for node 6940a02d575bc060ce162c9c INFO: 172.21.0.1:56728 - "POST /node/6940a02d575bc060ce162c9c HTTP/1.0" 202 Accepted 2025-12-17 23:56:40,070 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a36d575bc060ce1631b0 2025-12-17 23:56:40,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a36d575bc060ce1631b0 HTTP/1.1" 502 150 2025-12-17 23:56:40,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a36d575bc060ce1631b0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:40,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c14575bc060ce1624d1 2025-12-17 23:56:40,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c14575bc060ce1624d1 HTTP/1.1" 502 150 2025-12-17 23:56:40,436 - lava_callback - ERROR - Error processing callback for node 69409c14575bc060ce1624d1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c14575bc060ce1624d1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c14575bc060ce1624d1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c14575bc060ce1624d1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:40,438 - lava_callback - INFO - Starting async job submit for node 69409c38575bc060ce162537 2025-12-17 23:56:40,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:40,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c38575bc060ce162537 HTTP/1.1" 502 150 2025-12-17 23:56:40,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c38575bc060ce162537'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:40,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c38575bc060ce162537 2025-12-17 23:56:40,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c38575bc060ce162537 HTTP/1.1" 502 150 2025-12-17 23:56:40,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c38575bc060ce162537'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:40,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c26575bc060ce162505 2025-12-17 23:56:40,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c26575bc060ce162505 HTTP/1.1" 502 150 2025-12-17 23:56:40,553 - lava_callback - ERROR - Error processing callback for node 69409c26575bc060ce162505: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c26575bc060ce162505 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c26575bc060ce162505 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c26575bc060ce162505 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:40,556 - lava_callback - INFO - Starting async job submit for node 69409c39575bc060ce16253e 2025-12-17 23:56:40,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:40,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2a575bc060ce16250e 2025-12-17 23:56:40,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2a575bc060ce16250e HTTP/1.1" 502 150 2025-12-17 23:56:40,628 - lava_callback - ERROR - Error processing callback for node 69409c2a575bc060ce16250e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c2a575bc060ce16250e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c2a575bc060ce16250e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c2a575bc060ce16250e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:40,629 - lava_callback - INFO - Starting async job submit for node 69409c3a575bc060ce162541 2025-12-17 23:56:40,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:40,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3a575bc060ce162541 HTTP/1.1" 502 150 2025-12-17 23:56:40,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3a575bc060ce162541'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:40,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3a575bc060ce162541 2025-12-17 23:56:40,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3a575bc060ce162541 HTTP/1.1" 502 150 2025-12-17 23:56:40,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3a575bc060ce162541'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:40,683 - lava_callback - INFO - Received callback for node 6940a032575bc060ce162ca5 INFO: 172.21.0.1:56740 - "POST /node/6940a032575bc060ce162ca5 HTTP/1.0" 202 Accepted 2025-12-17 23:56:40,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c39575bc060ce16253e HTTP/1.1" 502 150 2025-12-17 23:56:40,736 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c39575bc060ce16253e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:40,736 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c39575bc060ce16253e 2025-12-17 23:56:40,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c39575bc060ce16253e HTTP/1.1" 502 150 2025-12-17 23:56:40,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c39575bc060ce16253e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:40,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c32575bc060ce162527 2025-12-17 23:56:40,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c32575bc060ce162527 HTTP/1.1" 502 150 2025-12-17 23:56:40,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c32575bc060ce162527'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:42,447 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c38575bc060ce162537 2025-12-17 23:56:42,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c38575bc060ce162537 HTTP/1.1" 502 150 2025-12-17 23:56:42,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c38575bc060ce162537'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:42,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3a575bc060ce162541 2025-12-17 23:56:42,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3a575bc060ce162541 HTTP/1.1" 502 150 2025-12-17 23:56:42,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3a575bc060ce162541'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:42,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c39575bc060ce16253e 2025-12-17 23:56:42,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c39575bc060ce16253e HTTP/1.1" 502 150 2025-12-17 23:56:42,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c39575bc060ce16253e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:42,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c34575bc060ce16252e 2025-12-17 23:56:42,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c34575bc060ce16252e HTTP/1.1" 502 150 2025-12-17 23:56:42,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c34575bc060ce16252e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:42,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c35575bc060ce162531 2025-12-17 23:56:42,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c35575bc060ce162531 HTTP/1.1" 502 150 2025-12-17 23:56:42,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c35575bc060ce162531'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:44,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2d575bc060ce162514 2025-12-17 23:56:44,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2d575bc060ce162514 HTTP/1.1" 502 150 2025-12-17 23:56:44,536 - lava_callback - ERROR - Error processing callback for node 69409c2d575bc060ce162514: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c2d575bc060ce162514 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c2d575bc060ce162514 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c2d575bc060ce162514 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:44,538 - lava_callback - INFO - Starting async job submit for node 69409c3d575bc060ce162547 2025-12-17 23:56:44,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:44,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3d575bc060ce162547 HTTP/1.1" 502 150 2025-12-17 23:56:44,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3d575bc060ce162547'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:44,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3d575bc060ce162547 2025-12-17 23:56:44,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3d575bc060ce162547 HTTP/1.1" 502 150 2025-12-17 23:56:44,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3d575bc060ce162547'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:44,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c31575bc060ce162522 2025-12-17 23:56:44,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c31575bc060ce162522 HTTP/1.1" 502 150 2025-12-17 23:56:44,629 - lava_callback - ERROR - Error processing callback for node 69409c31575bc060ce162522: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c31575bc060ce162522 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c31575bc060ce162522 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c31575bc060ce162522 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:44,631 - lava_callback - INFO - Starting async job submit for node 69409c4c575bc060ce16257d 2025-12-17 23:56:44,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:44,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4c575bc060ce16257d HTTP/1.1" 502 150 2025-12-17 23:56:44,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4c575bc060ce16257d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:44,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4c575bc060ce16257d 2025-12-17 23:56:44,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4c575bc060ce16257d HTTP/1.1" 502 150 2025-12-17 23:56:44,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4c575bc060ce16257d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:46,214 - lava_callback - INFO - Received callback for node 6940a037575bc060ce162cb6 INFO: 172.21.0.1:56750 - "POST /node/6940a037575bc060ce162cb6 HTTP/1.0" 202 Accepted 2025-12-17 23:56:46,449 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c38575bc060ce162537 INFO: 172.21.0.1:56752 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:56:46,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3d575bc060ce162547 2025-12-17 23:56:46,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3d575bc060ce162547 HTTP/1.1" 502 150 2025-12-17 23:56:46,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3d575bc060ce162547'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:46,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2e575bc060ce162517 2025-12-17 23:56:46,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2e575bc060ce162517 HTTP/1.1" 502 150 2025-12-17 23:56:46,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c2e575bc060ce162517'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:46,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4c575bc060ce16257d 2025-12-17 23:56:46,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4c575bc060ce16257d HTTP/1.1" 502 150 2025-12-17 23:56:46,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4c575bc060ce16257d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:46,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3a575bc060ce162541 2025-12-17 23:56:46,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3a575bc060ce162541 HTTP/1.1" 502 150 2025-12-17 23:56:46,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3a575bc060ce162541'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:46,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c39575bc060ce16253e 2025-12-17 23:56:46,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c39575bc060ce16253e HTTP/1.1" 502 150 2025-12-17 23:56:46,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c39575bc060ce16253e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:47,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be5575bc060ce16245e HTTP/1.1" 502 150 2025-12-17 23:56:47,152 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409be5575bc060ce16245e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:47,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a2d9575bc060ce163100 2025-12-17 23:56:47,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a2d9575bc060ce163100 HTTP/1.1" 502 150 2025-12-17 23:56:47,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a2d9575bc060ce163100'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:47,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c24575bc060ce1624fe 2025-12-17 23:56:47,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c24575bc060ce1624fe HTTP/1.1" 502 150 2025-12-17 23:56:47,970 - lava_callback - ERROR - Error processing callback for node 69409c24575bc060ce1624fe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c24575bc060ce1624fe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c24575bc060ce1624fe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c24575bc060ce1624fe (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:47,972 - lava_callback - INFO - Starting async job submit for node 69409c4f575bc060ce162580 2025-12-17 23:56:47,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:47,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4f575bc060ce162580 HTTP/1.1" 502 150 2025-12-17 23:56:47,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4f575bc060ce162580'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:47,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4f575bc060ce162580 2025-12-17 23:56:47,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4f575bc060ce162580 HTTP/1.1" 502 150 2025-12-17 23:56:47,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4f575bc060ce162580'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:48,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c28575bc060ce16250b HTTP/1.1" 502 150 2025-12-17 23:56:48,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c28575bc060ce16250b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:48,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c28575bc060ce16250b 2025-12-17 23:56:48,872 - lava_callback - INFO - Received callback for node 6940a02b575bc060ce162c8b INFO: 172.21.0.1:56760 - "POST /node/6940a02b575bc060ce162c8b HTTP/1.0" 202 Accepted 2025-12-17 23:56:48,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c32575bc060ce162527 2025-12-17 23:56:48,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c32575bc060ce162527 HTTP/1.1" 502 150 2025-12-17 23:56:48,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c32575bc060ce162527'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:49,523 - lava_callback - INFO - Received callback for node 6940a02e575bc060ce162c9f INFO: 172.21.0.1:56764 - "POST /node/6940a02e575bc060ce162c9f HTTP/1.0" 202 Accepted 2025-12-17 23:56:49,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4f575bc060ce162580 2025-12-17 23:56:49,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4f575bc060ce162580 HTTP/1.1" 502 150 2025-12-17 23:56:49,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4f575bc060ce162580'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:50,169 - lava_callback - INFO - Received callback for node 6940a033575bc060ce162ca8 INFO: 172.21.0.1:35446 - "POST /node/6940a033575bc060ce162ca8 HTTP/1.0" 202 Accepted 2025-12-17 23:56:50,214 - lava_callback - INFO - Received callback for node 6940a030575bc060ce162ca2 INFO: 172.21.0.1:35456 - "POST /node/6940a030575bc060ce162ca2 HTTP/1.0" 202 Accepted 2025-12-17 23:56:50,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3d575bc060ce162547 2025-12-17 23:56:50,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3d575bc060ce162547 HTTP/1.1" 502 150 2025-12-17 23:56:50,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3d575bc060ce162547'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:50,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4c575bc060ce16257d 2025-12-17 23:56:50,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4c575bc060ce16257d HTTP/1.1" 502 150 2025-12-17 23:56:50,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4c575bc060ce16257d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:50,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c34575bc060ce16252e 2025-12-17 23:56:50,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c34575bc060ce16252e HTTP/1.1" 502 150 2025-12-17 23:56:50,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c34575bc060ce16252e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:50,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c35575bc060ce162531 2025-12-17 23:56:50,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c35575bc060ce162531 HTTP/1.1" 502 150 2025-12-17 23:56:50,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c35575bc060ce162531'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:51,007 - lava_callback - INFO - Received callback for node 6940a034575bc060ce162cab INFO: 172.21.0.1:35462 - "POST /node/6940a034575bc060ce162cab HTTP/1.0" 202 Accepted 2025-12-17 23:56:51,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2b575bc060ce162511 2025-12-17 23:56:51,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2b575bc060ce162511 HTTP/1.1" 502 150 2025-12-17 23:56:51,971 - lava_callback - ERROR - Error processing callback for node 69409c2b575bc060ce162511: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c2b575bc060ce162511 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c2b575bc060ce162511 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c2b575bc060ce162511 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:51,973 - lava_callback - INFO - Starting async job submit for node 69409c47575bc060ce16256a 2025-12-17 23:56:51,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:51,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c47575bc060ce16256a HTTP/1.1" 502 150 2025-12-17 23:56:51,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c47575bc060ce16256a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:51,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c47575bc060ce16256a 2025-12-17 23:56:51,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c47575bc060ce16256a HTTP/1.1" 502 150 2025-12-17 23:56:51,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c47575bc060ce16256a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:52,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c37575bc060ce162534 HTTP/1.1" 502 150 2025-12-17 23:56:52,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c37575bc060ce162534'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:52,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c37575bc060ce162534 2025-12-17 23:56:52,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c37575bc060ce162534 HTTP/1.1" 502 150 2025-12-17 23:56:52,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c37575bc060ce162534'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:53,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c47575bc060ce16256a 2025-12-17 23:56:53,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4f575bc060ce162580 2025-12-17 23:56:53,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4f575bc060ce162580 HTTP/1.1" 502 150 2025-12-17 23:56:53,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4f575bc060ce162580'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:53,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c47575bc060ce16256a HTTP/1.1" 502 150 2025-12-17 23:56:53,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c47575bc060ce16256a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:54,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c37575bc060ce162534 2025-12-17 23:56:54,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c37575bc060ce162534 HTTP/1.1" 502 150 2025-12-17 23:56:54,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c37575bc060ce162534'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:54,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3a575bc060ce162541 2025-12-17 23:56:54,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3a575bc060ce162541 HTTP/1.1" 502 150 2025-12-17 23:56:54,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3a575bc060ce162541'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:54,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c39575bc060ce16253e 2025-12-17 23:56:54,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c39575bc060ce16253e HTTP/1.1" 502 150 2025-12-17 23:56:54,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c39575bc060ce16253e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:56,071 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a36d575bc060ce1631b0 2025-12-17 23:56:56,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a36d575bc060ce1631b0 HTTP/1.1" 502 150 2025-12-17 23:56:56,072 - lava_callback - ERROR - Error processing callback for node 6940a36d575bc060ce1631b0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a36d575bc060ce1631b0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a36d575bc060ce1631b0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a36d575bc060ce1631b0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:56:56,074 - lava_callback - INFO - Starting async job submit for node 69409c51575bc060ce162585 2025-12-17 23:56:56,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:56:56,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c51575bc060ce162585 HTTP/1.1" 502 150 2025-12-17 23:56:56,081 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c51575bc060ce162585'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:56,081 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c51575bc060ce162585 2025-12-17 23:56:56,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c51575bc060ce162585 HTTP/1.1" 502 150 2025-12-17 23:56:56,082 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c51575bc060ce162585'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35468 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:56:57,657 - lava_callback - INFO - Received callback for node 6940a2c1575bc060ce1630f8 INFO: 172.21.0.1:35478 - "POST /node/6940a2c1575bc060ce1630f8 HTTP/1.0" 202 Accepted 2025-12-17 23:56:57,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c47575bc060ce16256a 2025-12-17 23:56:57,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c47575bc060ce16256a HTTP/1.1" 502 150 2025-12-17 23:56:57,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c47575bc060ce16256a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:58,082 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c51575bc060ce162585 2025-12-17 23:56:58,145 - lava_callback - INFO - Received callback for node 6940a046575bc060ce162cd4 INFO: 172.21.0.1:35480 - "POST /node/6940a046575bc060ce162cd4 HTTP/1.0" 202 Accepted 2025-12-17 23:56:58,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c37575bc060ce162534 2025-12-17 23:56:58,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c37575bc060ce162534 HTTP/1.1" 502 150 2025-12-17 23:56:58,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c37575bc060ce162534'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:58,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3d575bc060ce162547 2025-12-17 23:56:58,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3d575bc060ce162547 HTTP/1.1" 502 150 2025-12-17 23:56:58,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c3d575bc060ce162547'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:58,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4c575bc060ce16257d 2025-12-17 23:56:58,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4c575bc060ce16257d HTTP/1.1" 502 150 2025-12-17 23:56:58,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4c575bc060ce16257d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:56:58,830 - lava_callback - INFO - Received callback for node 6940a036575bc060ce162cb3 INFO: 172.21.0.1:35488 - "POST /node/6940a036575bc060ce162cb3 HTTP/1.0" 202 Accepted 2025-12-17 23:56:59,518 - lava_callback - INFO - Received callback for node 6940a041575bc060ce162cc7 INFO: 172.21.0.1:35496 - "POST /node/6940a041575bc060ce162cc7 HTTP/1.0" 202 Accepted 2025-12-17 23:57:00,196 - lava_callback - INFO - Received callback for node 6940a043575bc060ce162cca INFO: 172.21.0.1:39130 - "POST /node/6940a043575bc060ce162cca HTTP/1.0" 202 Accepted 2025-12-17 23:57:01,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c38575bc060ce162537 HTTP/1.1" 502 150 2025-12-17 23:57:01,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c38575bc060ce162537'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:01,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4f575bc060ce162580 2025-12-17 23:57:01,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4f575bc060ce162580 HTTP/1.1" 502 150 2025-12-17 23:57:01,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4f575bc060ce162580'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:02,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c2e575bc060ce162517 2025-12-17 23:57:02,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c2e575bc060ce162517 HTTP/1.1" 502 150 2025-12-17 23:57:02,591 - lava_callback - ERROR - Error processing callback for node 69409c2e575bc060ce162517: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c2e575bc060ce162517 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c2e575bc060ce162517 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c2e575bc060ce162517 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:02,593 - lava_callback - INFO - Starting async job submit for node 69409c4b575bc060ce16257a 2025-12-17 23:57:02,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:02,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4b575bc060ce16257a HTTP/1.1" 502 150 2025-12-17 23:57:02,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4b575bc060ce16257a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:02,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4b575bc060ce16257a 2025-12-17 23:57:02,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4b575bc060ce16257a HTTP/1.1" 502 150 2025-12-17 23:57:02,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4b575bc060ce16257a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:03,152 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409be5575bc060ce16245e INFO: 172.21.0.1:39140 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:57:03,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a2d9575bc060ce163100 2025-12-17 23:57:03,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a2d9575bc060ce163100 HTTP/1.1" 502 150 2025-12-17 23:57:03,893 - lava_callback - ERROR - Error processing callback for node 6940a2d9575bc060ce163100: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a2d9575bc060ce163100 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a2d9575bc060ce163100 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a2d9575bc060ce163100 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:03,895 - lava_callback - INFO - Starting async job submit for node 69409c52575bc060ce162588 2025-12-17 23:57:03,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:03,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c52575bc060ce162588 HTTP/1.1" 502 150 2025-12-17 23:57:03,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c52575bc060ce162588'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:03,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c52575bc060ce162588 2025-12-17 23:57:03,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c52575bc060ce162588 HTTP/1.1" 502 150 2025-12-17 23:57:03,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c52575bc060ce162588'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:04,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4b575bc060ce16257a 2025-12-17 23:57:04,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4b575bc060ce16257a HTTP/1.1" 502 150 2025-12-17 23:57:04,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4b575bc060ce16257a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:04,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c32575bc060ce162527 2025-12-17 23:57:04,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c32575bc060ce162527 HTTP/1.1" 502 150 2025-12-17 23:57:04,914 - lava_callback - ERROR - Error processing callback for node 69409c32575bc060ce162527: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c32575bc060ce162527 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c32575bc060ce162527 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c32575bc060ce162527 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:04,916 - lava_callback - INFO - Starting async job submit for node 69409c54575bc060ce16258b 2025-12-17 23:57:04,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:04,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c54575bc060ce16258b HTTP/1.1" 502 150 2025-12-17 23:57:04,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c54575bc060ce16258b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:04,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c54575bc060ce16258b 2025-12-17 23:57:04,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c54575bc060ce16258b HTTP/1.1" 502 150 2025-12-17 23:57:04,924 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c54575bc060ce16258b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:05,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c52575bc060ce162588 2025-12-17 23:57:05,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c52575bc060ce162588 HTTP/1.1" 502 150 2025-12-17 23:57:05,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c52575bc060ce162588'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:05,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c47575bc060ce16256a 2025-12-17 23:57:05,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c47575bc060ce16256a HTTP/1.1" 502 150 2025-12-17 23:57:05,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c47575bc060ce16256a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39150 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:57:06,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c37575bc060ce162534 2025-12-17 23:57:06,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c37575bc060ce162534 HTTP/1.1" 502 150 2025-12-17 23:57:06,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c37575bc060ce162534'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:06,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c34575bc060ce16252e 2025-12-17 23:57:06,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c34575bc060ce16252e HTTP/1.1" 502 150 2025-12-17 23:57:06,798 - lava_callback - ERROR - Error processing callback for node 69409c34575bc060ce16252e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c34575bc060ce16252e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c34575bc060ce16252e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c34575bc060ce16252e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:06,800 - lava_callback - INFO - Starting async job submit for node 69409c59575bc060ce16259b 2025-12-17 23:57:06,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:06,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c59575bc060ce16259b HTTP/1.1" 502 150 2025-12-17 23:57:06,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c59575bc060ce16259b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:06,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c59575bc060ce16259b 2025-12-17 23:57:06,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c59575bc060ce16259b HTTP/1.1" 502 150 2025-12-17 23:57:06,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c59575bc060ce16259b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:06,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c35575bc060ce162531 2025-12-17 23:57:06,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c35575bc060ce162531 HTTP/1.1" 502 150 2025-12-17 23:57:06,907 - lava_callback - ERROR - Error processing callback for node 69409c35575bc060ce162531: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c35575bc060ce162531 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c35575bc060ce162531 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c35575bc060ce162531 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:06,909 - lava_callback - INFO - Starting async job submit for node 69409c5b575bc060ce1625a2 2025-12-17 23:57:06,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:06,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c5b575bc060ce1625a2 HTTP/1.1" 502 150 2025-12-17 23:57:06,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c5b575bc060ce1625a2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:06,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c5b575bc060ce1625a2 2025-12-17 23:57:06,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c5b575bc060ce1625a2 HTTP/1.1" 502 150 2025-12-17 23:57:06,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c5b575bc060ce1625a2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:06,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c54575bc060ce16258b 2025-12-17 23:57:06,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c54575bc060ce16258b HTTP/1.1" 502 150 2025-12-17 23:57:06,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c54575bc060ce16258b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:08,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4b575bc060ce16257a 2025-12-17 23:57:08,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4b575bc060ce16257a HTTP/1.1" 502 150 2025-12-17 23:57:08,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4b575bc060ce16257a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:08,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c59575bc060ce16259b 2025-12-17 23:57:08,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c59575bc060ce16259b HTTP/1.1" 502 150 2025-12-17 23:57:08,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c59575bc060ce16259b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:08,868 - lava_callback - INFO - Received callback for node 6940a03f575bc060ce162cbb INFO: 172.21.0.1:39154 - "POST /node/6940a03f575bc060ce162cbb HTTP/1.0" 202 Accepted 2025-12-17 23:57:08,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c5b575bc060ce1625a2 2025-12-17 23:57:08,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c5b575bc060ce1625a2 HTTP/1.1" 502 150 2025-12-17 23:57:08,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c5b575bc060ce1625a2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:09,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c38575bc060ce162537 2025-12-17 23:57:09,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c38575bc060ce162537 HTTP/1.1" 502 150 2025-12-17 23:57:09,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c38575bc060ce162537'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:09,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c52575bc060ce162588 2025-12-17 23:57:09,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c52575bc060ce162588 HTTP/1.1" 502 150 2025-12-17 23:57:09,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c52575bc060ce162588'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:10,200 - lava_callback - INFO - Received callback for node 6940a045575bc060ce162cd1 INFO: 172.21.0.1:55688 - "POST /node/6940a045575bc060ce162cd1 HTTP/1.0" 202 Accepted 2025-12-17 23:57:10,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3a575bc060ce162541 2025-12-17 23:57:10,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3a575bc060ce162541 HTTP/1.1" 502 150 2025-12-17 23:57:10,665 - lava_callback - ERROR - Error processing callback for node 69409c3a575bc060ce162541: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c3a575bc060ce162541 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c3a575bc060ce162541 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c3a575bc060ce162541 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:10,667 - lava_callback - INFO - Starting async job submit for node 69409c5d575bc060ce1625a5 2025-12-17 23:57:10,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:10,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c5d575bc060ce1625a5 HTTP/1.1" 502 150 2025-12-17 23:57:10,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c5d575bc060ce1625a5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:10,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c5d575bc060ce1625a5 2025-12-17 23:57:10,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c5d575bc060ce1625a5 HTTP/1.1" 502 150 2025-12-17 23:57:10,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c5d575bc060ce1625a5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:10,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c39575bc060ce16253e 2025-12-17 23:57:10,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c39575bc060ce16253e HTTP/1.1" 502 150 2025-12-17 23:57:10,742 - lava_callback - ERROR - Error processing callback for node 69409c39575bc060ce16253e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c39575bc060ce16253e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c39575bc060ce16253e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c39575bc060ce16253e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:10,745 - lava_callback - INFO - Starting async job submit for node 69409c63575bc060ce1625ac 2025-12-17 23:57:10,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:10,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c63575bc060ce1625ac HTTP/1.1" 502 150 2025-12-17 23:57:10,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c63575bc060ce1625ac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:10,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c63575bc060ce1625ac 2025-12-17 23:57:10,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c63575bc060ce1625ac HTTP/1.1" 502 150 2025-12-17 23:57:10,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c63575bc060ce1625ac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:10,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c54575bc060ce16258b 2025-12-17 23:57:10,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c54575bc060ce16258b HTTP/1.1" 502 150 2025-12-17 23:57:10,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c54575bc060ce16258b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:12,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c5d575bc060ce1625a5 2025-12-17 23:57:12,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c5d575bc060ce1625a5 HTTP/1.1" 502 150 2025-12-17 23:57:12,677 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c5d575bc060ce1625a5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:12,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c63575bc060ce1625ac 2025-12-17 23:57:12,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c63575bc060ce1625ac HTTP/1.1" 502 150 2025-12-17 23:57:12,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c63575bc060ce1625ac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:12,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c59575bc060ce16259b 2025-12-17 23:57:12,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c59575bc060ce16259b HTTP/1.1" 502 150 2025-12-17 23:57:12,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c59575bc060ce16259b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:12,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c5b575bc060ce1625a2 2025-12-17 23:57:12,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c5b575bc060ce1625a2 HTTP/1.1" 502 150 2025-12-17 23:57:12,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c5b575bc060ce1625a2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:13,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c51575bc060ce162585 HTTP/1.1" 502 150 2025-12-17 23:57:13,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c51575bc060ce162585'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:14,550 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c3d575bc060ce162547 2025-12-17 23:57:14,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4c575bc060ce16257d 2025-12-17 23:57:14,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4c575bc060ce16257d HTTP/1.1" 502 150 2025-12-17 23:57:14,643 - lava_callback - ERROR - Error processing callback for node 69409c4c575bc060ce16257d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c4c575bc060ce16257d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c4c575bc060ce16257d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c4c575bc060ce16257d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:14,644 - lava_callback - INFO - Starting async job submit for node 69409c6a575bc060ce1625b4 2025-12-17 23:57:14,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:14,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6a575bc060ce1625b4 HTTP/1.1" 502 150 2025-12-17 23:57:14,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6a575bc060ce1625b4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:14,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6a575bc060ce1625b4 2025-12-17 23:57:14,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6a575bc060ce1625b4 HTTP/1.1" 502 150 2025-12-17 23:57:14,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6a575bc060ce1625b4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55704 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:57:16,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4b575bc060ce16257a 2025-12-17 23:57:16,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4b575bc060ce16257a HTTP/1.1" 502 150 2025-12-17 23:57:16,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4b575bc060ce16257a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:16,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6a575bc060ce1625b4 2025-12-17 23:57:16,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6a575bc060ce1625b4 HTTP/1.1" 502 150 2025-12-17 23:57:16,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6a575bc060ce1625b4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:16,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c5d575bc060ce1625a5 2025-12-17 23:57:16,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c5d575bc060ce1625a5 HTTP/1.1" 502 150 2025-12-17 23:57:16,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c5d575bc060ce1625a5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:16,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c63575bc060ce1625ac 2025-12-17 23:57:16,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c63575bc060ce1625ac HTTP/1.1" 502 150 2025-12-17 23:57:16,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c63575bc060ce1625ac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:17,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c51575bc060ce162585 2025-12-17 23:57:17,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c51575bc060ce162585 HTTP/1.1" 502 150 2025-12-17 23:57:17,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c51575bc060ce162585'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:17,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c52575bc060ce162588 2025-12-17 23:57:17,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c52575bc060ce162588 HTTP/1.1" 502 150 2025-12-17 23:57:17,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c52575bc060ce162588'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:17,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4f575bc060ce162580 2025-12-17 23:57:17,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4f575bc060ce162580 HTTP/1.1" 502 150 2025-12-17 23:57:17,985 - lava_callback - ERROR - Error processing callback for node 69409c4f575bc060ce162580: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c4f575bc060ce162580 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c4f575bc060ce162580 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c4f575bc060ce162580 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:17,987 - lava_callback - INFO - Starting async job submit for node 69409c48575bc060ce162571 2025-12-17 23:57:18,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:18,076 - lava_callback - INFO - Received callback for node 6940a054575bc060ce162cda INFO: 172.21.0.1:55716 - "POST /node/6940a054575bc060ce162cda HTTP/1.0" 202 Accepted 2025-12-17 23:57:18,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409be5575bc060ce16245e HTTP/1.1" 502 150 2025-12-17 23:57:18,640 - lava_callback - ERROR - Error processing callback for node 69409be5575bc060ce16245e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409be5575bc060ce16245e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409be5575bc060ce16245e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409be5575bc060ce16245e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:18,642 - lava_callback - INFO - Starting async job submit for node 69409c6e575bc060ce1625c6 2025-12-17 23:57:18,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:18,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6e575bc060ce1625c6 HTTP/1.1" 502 150 2025-12-17 23:57:18,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6e575bc060ce1625c6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:18,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6e575bc060ce1625c6 2025-12-17 23:57:18,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6e575bc060ce1625c6 HTTP/1.1" 502 150 2025-12-17 23:57:18,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6e575bc060ce1625c6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:18,752 - lava_callback - INFO - Received callback for node 6940a056575bc060ce162ce4 INFO: 172.21.0.1:55724 - "POST /node/6940a056575bc060ce162ce4 HTTP/1.0" 202 Accepted 2025-12-17 23:57:18,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c54575bc060ce16258b 2025-12-17 23:57:18,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c54575bc060ce16258b HTTP/1.1" 502 150 2025-12-17 23:57:18,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c54575bc060ce16258b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:19,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c28575bc060ce16250b HTTP/1.1" 502 150 2025-12-17 23:57:19,408 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c28575bc060ce16250b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:19,457 - lava_callback - INFO - Received callback for node 6940a058575bc060ce162cee INFO: 172.21.0.1:55726 - "POST /node/6940a058575bc060ce162cee HTTP/1.0" 202 Accepted 2025-12-17 23:57:20,136 - lava_callback - INFO - Received callback for node 6940a05a575bc060ce162cf5 INFO: 172.21.0.1:50314 - "POST /node/6940a05a575bc060ce162cf5 HTTP/1.0" 202 Accepted 2025-12-17 23:57:20,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6e575bc060ce1625c6 2025-12-17 23:57:20,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6e575bc060ce1625c6 HTTP/1.1" 502 150 2025-12-17 23:57:20,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6e575bc060ce1625c6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:20,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6a575bc060ce1625b4 2025-12-17 23:57:20,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6a575bc060ce1625b4 HTTP/1.1" 502 150 2025-12-17 23:57:20,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6a575bc060ce1625b4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:20,814 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c59575bc060ce16259b 2025-12-17 23:57:20,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c59575bc060ce16259b HTTP/1.1" 502 150 2025-12-17 23:57:20,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c59575bc060ce16259b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:20,821 - lava_callback - INFO - Received callback for node 6940a05c575bc060ce162cf8 INFO: 172.21.0.1:50320 - "POST /node/6940a05c575bc060ce162cf8 HTTP/1.0" 202 Accepted 2025-12-17 23:57:20,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c5b575bc060ce1625a2 2025-12-17 23:57:20,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c5b575bc060ce1625a2 HTTP/1.1" 502 150 2025-12-17 23:57:20,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c5b575bc060ce1625a2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:21,408 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c28575bc060ce16250b 2025-12-17 23:57:21,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c47575bc060ce16256a 2025-12-17 23:57:21,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c47575bc060ce16256a HTTP/1.1" 502 150 2025-12-17 23:57:21,987 - lava_callback - ERROR - Error processing callback for node 69409c47575bc060ce16256a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c47575bc060ce16256a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c47575bc060ce16256a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c47575bc060ce16256a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:21,989 - lava_callback - INFO - Starting async job submit for node 69409c6f575bc060ce1625c9 2025-12-17 23:57:21,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:21,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6f575bc060ce1625c9 HTTP/1.1" 502 150 2025-12-17 23:57:21,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6f575bc060ce1625c9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:21,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6f575bc060ce1625c9 2025-12-17 23:57:21,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6f575bc060ce1625c9 HTTP/1.1" 502 150 2025-12-17 23:57:21,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6f575bc060ce1625c9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:22,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c37575bc060ce162534 2025-12-17 23:57:22,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c37575bc060ce162534 HTTP/1.1" 502 150 2025-12-17 23:57:22,535 - lava_callback - ERROR - Error processing callback for node 69409c37575bc060ce162534: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c37575bc060ce162534 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c37575bc060ce162534 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c37575bc060ce162534 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:22,537 - lava_callback - INFO - Starting async job submit for node 69409c71575bc060ce1625cc 2025-12-17 23:57:22,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:22,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c71575bc060ce1625cc HTTP/1.1" 502 150 2025-12-17 23:57:22,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c71575bc060ce1625cc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:22,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c71575bc060ce1625cc 2025-12-17 23:57:22,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c71575bc060ce1625cc HTTP/1.1" 502 150 2025-12-17 23:57:22,553 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c71575bc060ce1625cc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:23,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6f575bc060ce1625c9 2025-12-17 23:57:23,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6f575bc060ce1625c9 HTTP/1.1" 502 150 2025-12-17 23:57:23,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6f575bc060ce1625c9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:24,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c71575bc060ce1625cc 2025-12-17 23:57:24,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c71575bc060ce1625cc HTTP/1.1" 502 150 2025-12-17 23:57:24,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c71575bc060ce1625cc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:24,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6e575bc060ce1625c6 2025-12-17 23:57:24,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6e575bc060ce1625c6 HTTP/1.1" 502 150 2025-12-17 23:57:24,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6e575bc060ce1625c6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:24,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c5d575bc060ce1625a5 2025-12-17 23:57:24,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c5d575bc060ce1625a5 HTTP/1.1" 502 150 2025-12-17 23:57:24,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c5d575bc060ce1625a5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:24,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c63575bc060ce1625ac 2025-12-17 23:57:24,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c63575bc060ce1625ac HTTP/1.1" 502 150 2025-12-17 23:57:24,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c63575bc060ce1625ac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:25,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c51575bc060ce162585 2025-12-17 23:57:25,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c51575bc060ce162585 HTTP/1.1" 502 150 2025-12-17 23:57:25,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c51575bc060ce162585'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:25,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c38575bc060ce162537 2025-12-17 23:57:25,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c38575bc060ce162537 HTTP/1.1" 502 150 2025-12-17 23:57:25,755 - lava_callback - ERROR - Error processing callback for node 69409c38575bc060ce162537: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c38575bc060ce162537 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c38575bc060ce162537 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c38575bc060ce162537 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:25,757 - lava_callback - INFO - Starting async job submit for node 69409c72575bc060ce1625cf 2025-12-17 23:57:25,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:25,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c72575bc060ce1625cf HTTP/1.1" 502 150 2025-12-17 23:57:25,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c72575bc060ce1625cf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:25,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c72575bc060ce1625cf 2025-12-17 23:57:25,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c72575bc060ce1625cf HTTP/1.1" 502 150 2025-12-17 23:57:25,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c72575bc060ce1625cf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50326 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:57:27,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c72575bc060ce1625cf 2025-12-17 23:57:27,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c72575bc060ce1625cf HTTP/1.1" 502 150 2025-12-17 23:57:27,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c72575bc060ce1625cf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:27,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6f575bc060ce1625c9 2025-12-17 23:57:27,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6f575bc060ce1625c9 HTTP/1.1" 502 150 2025-12-17 23:57:27,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6f575bc060ce1625c9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:28,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c71575bc060ce1625cc 2025-12-17 23:57:28,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c71575bc060ce1625cc HTTP/1.1" 502 150 2025-12-17 23:57:28,556 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c71575bc060ce1625cc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:28,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6a575bc060ce1625b4 2025-12-17 23:57:28,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6a575bc060ce1625b4 HTTP/1.1" 502 150 2025-12-17 23:57:28,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6a575bc060ce1625b4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:29,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c3d575bc060ce162547 HTTP/1.1" 502 150 2025-12-17 23:57:29,904 - lava_callback - ERROR - Error processing callback for node 69409c3d575bc060ce162547: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c3d575bc060ce162547 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c3d575bc060ce162547 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c3d575bc060ce162547 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:29,907 - lava_callback - INFO - Starting async job submit for node 69409c74575bc060ce1625d2 2025-12-17 23:57:29,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:29,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c74575bc060ce1625d2 HTTP/1.1" 502 150 2025-12-17 23:57:29,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c74575bc060ce1625d2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:29,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c74575bc060ce1625d2 2025-12-17 23:57:29,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c74575bc060ce1625d2 HTTP/1.1" 502 150 2025-12-17 23:57:29,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c74575bc060ce1625d2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:31,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c72575bc060ce1625cf 2025-12-17 23:57:31,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c72575bc060ce1625cf HTTP/1.1" 502 150 2025-12-17 23:57:31,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c72575bc060ce1625cf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:31,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c74575bc060ce1625d2 2025-12-17 23:57:31,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c74575bc060ce1625d2 HTTP/1.1" 502 150 2025-12-17 23:57:31,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c74575bc060ce1625d2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:32,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4b575bc060ce16257a 2025-12-17 23:57:32,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4b575bc060ce16257a HTTP/1.1" 502 150 2025-12-17 23:57:32,605 - lava_callback - ERROR - Error processing callback for node 69409c4b575bc060ce16257a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c4b575bc060ce16257a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c4b575bc060ce16257a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c4b575bc060ce16257a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:32,607 - lava_callback - INFO - Starting async job submit for node 69409c76575bc060ce1625d5 2025-12-17 23:57:32,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:32,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c76575bc060ce1625d5 HTTP/1.1" 502 150 2025-12-17 23:57:32,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c76575bc060ce1625d5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:32,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c76575bc060ce1625d5 2025-12-17 23:57:32,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c76575bc060ce1625d5 HTTP/1.1" 502 150 2025-12-17 23:57:32,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c76575bc060ce1625d5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:32,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6e575bc060ce1625c6 2025-12-17 23:57:32,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6e575bc060ce1625c6 HTTP/1.1" 502 150 2025-12-17 23:57:32,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6e575bc060ce1625c6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:33,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c52575bc060ce162588 2025-12-17 23:57:33,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c52575bc060ce162588 HTTP/1.1" 502 150 2025-12-17 23:57:33,923 - lava_callback - ERROR - Error processing callback for node 69409c52575bc060ce162588: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c52575bc060ce162588 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c52575bc060ce162588 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c52575bc060ce162588 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:33,925 - lava_callback - INFO - Starting async job submit for node 69409c7c575bc060ce1625e1 2025-12-17 23:57:33,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:33,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7c575bc060ce1625e1 HTTP/1.1" 502 150 2025-12-17 23:57:33,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7c575bc060ce1625e1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:33,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7c575bc060ce1625e1 2025-12-17 23:57:33,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7c575bc060ce1625e1 HTTP/1.1" 502 150 2025-12-17 23:57:33,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7c575bc060ce1625e1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:34,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c76575bc060ce1625d5 2025-12-17 23:57:34,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c76575bc060ce1625d5 HTTP/1.1" 502 150 2025-12-17 23:57:34,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c76575bc060ce1625d5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:34,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c54575bc060ce16258b 2025-12-17 23:57:34,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c54575bc060ce16258b HTTP/1.1" 502 150 2025-12-17 23:57:34,930 - lava_callback - ERROR - Error processing callback for node 69409c54575bc060ce16258b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c54575bc060ce16258b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c54575bc060ce16258b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c54575bc060ce16258b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:34,932 - lava_callback - INFO - Starting async job submit for node 69409c7e575bc060ce1625e6 2025-12-17 23:57:34,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:34,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7e575bc060ce1625e6 HTTP/1.1" 502 150 2025-12-17 23:57:34,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7e575bc060ce1625e6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:34,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7e575bc060ce1625e6 2025-12-17 23:57:34,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7e575bc060ce1625e6 HTTP/1.1" 502 150 2025-12-17 23:57:34,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7e575bc060ce1625e6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:35,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c74575bc060ce1625d2 2025-12-17 23:57:35,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c74575bc060ce1625d2 HTTP/1.1" 502 150 2025-12-17 23:57:35,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c74575bc060ce1625d2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:35,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7c575bc060ce1625e1 2025-12-17 23:57:35,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7c575bc060ce1625e1 HTTP/1.1" 502 150 2025-12-17 23:57:35,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7c575bc060ce1625e1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:36,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6f575bc060ce1625c9 2025-12-17 23:57:36,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6f575bc060ce1625c9 HTTP/1.1" 502 150 2025-12-17 23:57:36,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c6f575bc060ce1625c9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57556 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:57:36,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c71575bc060ce1625cc 2025-12-17 23:57:36,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c71575bc060ce1625cc HTTP/1.1" 502 150 2025-12-17 23:57:36,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c71575bc060ce1625cc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:36,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c59575bc060ce16259b 2025-12-17 23:57:36,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c28575bc060ce16250b HTTP/1.1" 502 150 2025-12-17 23:57:36,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c28575bc060ce16250b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:36,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c59575bc060ce16259b HTTP/1.1" 502 150 2025-12-17 23:57:36,817 - lava_callback - ERROR - Error processing callback for node 69409c59575bc060ce16259b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c59575bc060ce16259b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c59575bc060ce16259b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c59575bc060ce16259b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:36,819 - lava_callback - INFO - Starting async job submit for node 69409c4a575bc060ce162577 2025-12-17 23:57:36,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:36,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4a575bc060ce162577 HTTP/1.1" 502 150 2025-12-17 23:57:36,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4a575bc060ce162577'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:36,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4a575bc060ce162577 2025-12-17 23:57:36,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4a575bc060ce162577 HTTP/1.1" 502 150 2025-12-17 23:57:36,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4a575bc060ce162577'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:36,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c5b575bc060ce1625a2 2025-12-17 23:57:36,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c5b575bc060ce1625a2 HTTP/1.1" 502 150 2025-12-17 23:57:36,921 - lava_callback - ERROR - Error processing callback for node 69409c5b575bc060ce1625a2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c5b575bc060ce1625a2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c5b575bc060ce1625a2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c5b575bc060ce1625a2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:36,922 - lava_callback - INFO - Starting async job submit for node 69409c81575bc060ce1625ed 2025-12-17 23:57:36,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:36,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c81575bc060ce1625ed HTTP/1.1" 502 150 2025-12-17 23:57:36,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c81575bc060ce1625ed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:36,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c81575bc060ce1625ed 2025-12-17 23:57:36,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c81575bc060ce1625ed HTTP/1.1" 502 150 2025-12-17 23:57:36,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c81575bc060ce1625ed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:36,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7e575bc060ce1625e6 2025-12-17 23:57:36,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7e575bc060ce1625e6 HTTP/1.1" 502 150 2025-12-17 23:57:36,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7e575bc060ce1625e6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:38,061 - lava_callback - INFO - Received callback for node 6940a05d575bc060ce162cfb INFO: 172.21.0.1:57572 - "POST /node/6940a05d575bc060ce162cfb HTTP/1.0" 202 Accepted 2025-12-17 23:57:38,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c76575bc060ce1625d5 2025-12-17 23:57:38,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c76575bc060ce1625d5 HTTP/1.1" 502 150 2025-12-17 23:57:38,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c76575bc060ce1625d5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:38,720 - lava_callback - INFO - Received callback for node 6940a05e575bc060ce162cfe INFO: 172.21.0.1:57578 - "POST /node/6940a05e575bc060ce162cfe HTTP/1.0" 202 Accepted 2025-12-17 23:57:38,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4a575bc060ce162577 2025-12-17 23:57:38,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4a575bc060ce162577 HTTP/1.1" 502 150 2025-12-17 23:57:38,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4a575bc060ce162577'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:38,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c81575bc060ce1625ed 2025-12-17 23:57:38,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c81575bc060ce1625ed HTTP/1.1" 502 150 2025-12-17 23:57:38,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c81575bc060ce1625ed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:39,385 - lava_callback - INFO - Received callback for node 6940a05f575bc060ce162d01 INFO: 172.21.0.1:57594 - "POST /node/6940a05f575bc060ce162d01 HTTP/1.0" 202 Accepted 2025-12-17 23:57:39,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c72575bc060ce1625cf 2025-12-17 23:57:39,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c72575bc060ce1625cf HTTP/1.1" 502 150 2025-12-17 23:57:39,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c72575bc060ce1625cf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:39,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7c575bc060ce1625e1 2025-12-17 23:57:39,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7c575bc060ce1625e1 HTTP/1.1" 502 150 2025-12-17 23:57:39,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7c575bc060ce1625e1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:40,068 - lava_callback - INFO - Received callback for node 6940a060575bc060ce162d04 INFO: 172.21.0.1:59532 - "POST /node/6940a060575bc060ce162d04 HTTP/1.0" 202 Accepted 2025-12-17 23:57:40,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c5d575bc060ce1625a5 2025-12-17 23:57:40,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c5d575bc060ce1625a5 HTTP/1.1" 502 150 2025-12-17 23:57:40,681 - lava_callback - ERROR - Error processing callback for node 69409c5d575bc060ce1625a5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c5d575bc060ce1625a5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c5d575bc060ce1625a5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c5d575bc060ce1625a5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:40,683 - lava_callback - INFO - Starting async job submit for node 69409c83575bc060ce1625f0 2025-12-17 23:57:40,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:40,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c83575bc060ce1625f0 HTTP/1.1" 502 150 2025-12-17 23:57:40,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c83575bc060ce1625f0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:40,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c83575bc060ce1625f0 2025-12-17 23:57:40,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c83575bc060ce1625f0 HTTP/1.1" 502 150 2025-12-17 23:57:40,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c83575bc060ce1625f0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:40,743 - lava_callback - INFO - Received callback for node 6940a062575bc060ce162d07 INFO: 172.21.0.1:59542 - "POST /node/6940a062575bc060ce162d07 HTTP/1.0" 202 Accepted 2025-12-17 23:57:40,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c63575bc060ce1625ac 2025-12-17 23:57:40,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c63575bc060ce1625ac HTTP/1.1" 502 150 2025-12-17 23:57:40,760 - lava_callback - ERROR - Error processing callback for node 69409c63575bc060ce1625ac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c63575bc060ce1625ac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c63575bc060ce1625ac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c63575bc060ce1625ac (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:40,763 - lava_callback - INFO - Starting async job submit for node 69409c85575bc060ce1625f7 2025-12-17 23:57:40,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:40,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c85575bc060ce1625f7 HTTP/1.1" 502 150 2025-12-17 23:57:40,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c85575bc060ce1625f7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:40,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c85575bc060ce1625f7 2025-12-17 23:57:40,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c85575bc060ce1625f7 HTTP/1.1" 502 150 2025-12-17 23:57:40,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c85575bc060ce1625f7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:40,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c28575bc060ce16250b 2025-12-17 23:57:40,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c28575bc060ce16250b HTTP/1.1" 502 150 2025-12-17 23:57:40,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c28575bc060ce16250b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:40,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7e575bc060ce1625e6 2025-12-17 23:57:40,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7e575bc060ce1625e6 HTTP/1.1" 502 150 2025-12-17 23:57:40,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7e575bc060ce1625e6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:41,415 - lava_callback - INFO - Received callback for node 6940a068575bc060ce162d0e INFO: 172.21.0.1:59554 - "POST /node/6940a068575bc060ce162d0e HTTP/1.0" 202 Accepted 2025-12-17 23:57:41,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c51575bc060ce162585 2025-12-17 23:57:41,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c51575bc060ce162585 HTTP/1.1" 502 150 2025-12-17 23:57:41,525 - lava_callback - ERROR - Error processing callback for node 69409c51575bc060ce162585: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c51575bc060ce162585 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c51575bc060ce162585 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c51575bc060ce162585 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:41,527 - lava_callback - INFO - Starting async job submit for node 69409c87575bc060ce1625fa 2025-12-17 23:57:41,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:41,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c87575bc060ce1625fa HTTP/1.1" 502 150 2025-12-17 23:57:41,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c87575bc060ce1625fa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:41,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c87575bc060ce1625fa 2025-12-17 23:57:41,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c87575bc060ce1625fa HTTP/1.1" 502 150 2025-12-17 23:57:41,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c87575bc060ce1625fa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:42,088 - lava_callback - INFO - Received callback for node 6940a06c575bc060ce162d25 INFO: 172.21.0.1:59568 - "POST /node/6940a06c575bc060ce162d25 HTTP/1.0" 202 Accepted 2025-12-17 23:57:42,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c83575bc060ce1625f0 2025-12-17 23:57:42,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c83575bc060ce1625f0 HTTP/1.1" 502 150 2025-12-17 23:57:42,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c83575bc060ce1625f0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:42,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c85575bc060ce1625f7 2025-12-17 23:57:42,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c85575bc060ce1625f7 HTTP/1.1" 502 150 2025-12-17 23:57:42,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c85575bc060ce1625f7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:42,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4a575bc060ce162577 2025-12-17 23:57:42,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4a575bc060ce162577 HTTP/1.1" 502 150 2025-12-17 23:57:42,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4a575bc060ce162577'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:42,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c81575bc060ce1625ed 2025-12-17 23:57:42,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c81575bc060ce1625ed HTTP/1.1" 502 150 2025-12-17 23:57:42,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c81575bc060ce1625ed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:43,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c87575bc060ce1625fa 2025-12-17 23:57:43,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c87575bc060ce1625fa HTTP/1.1" 502 150 2025-12-17 23:57:43,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c87575bc060ce1625fa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:43,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c74575bc060ce1625d2 2025-12-17 23:57:43,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c74575bc060ce1625d2 HTTP/1.1" 502 150 2025-12-17 23:57:43,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c74575bc060ce1625d2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:44,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6a575bc060ce1625b4 2025-12-17 23:57:44,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6a575bc060ce1625b4 HTTP/1.1" 502 150 2025-12-17 23:57:44,659 - lava_callback - ERROR - Error processing callback for node 69409c6a575bc060ce1625b4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c6a575bc060ce1625b4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c6a575bc060ce1625b4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c6a575bc060ce1625b4 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:44,660 - lava_callback - INFO - Starting async job submit for node 69409c89575bc060ce1625fd 2025-12-17 23:57:44,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:44,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c89575bc060ce1625fd HTTP/1.1" 502 150 2025-12-17 23:57:44,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c89575bc060ce1625fd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:44,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c89575bc060ce1625fd 2025-12-17 23:57:44,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c89575bc060ce1625fd HTTP/1.1" 502 150 2025-12-17 23:57:44,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c89575bc060ce1625fd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59572 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:57:46,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c76575bc060ce1625d5 2025-12-17 23:57:46,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c76575bc060ce1625d5 HTTP/1.1" 502 150 2025-12-17 23:57:46,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c76575bc060ce1625d5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:46,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c89575bc060ce1625fd 2025-12-17 23:57:46,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c89575bc060ce1625fd HTTP/1.1" 502 150 2025-12-17 23:57:46,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c89575bc060ce1625fd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:46,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c83575bc060ce1625f0 2025-12-17 23:57:46,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c83575bc060ce1625f0 HTTP/1.1" 502 150 2025-12-17 23:57:46,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c83575bc060ce1625f0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:46,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c85575bc060ce1625f7 2025-12-17 23:57:46,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c85575bc060ce1625f7 HTTP/1.1" 502 150 2025-12-17 23:57:46,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c85575bc060ce1625f7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:47,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c87575bc060ce1625fa 2025-12-17 23:57:47,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c87575bc060ce1625fa HTTP/1.1" 502 150 2025-12-17 23:57:47,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c87575bc060ce1625fa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:47,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7c575bc060ce1625e1 2025-12-17 23:57:47,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7c575bc060ce1625e1 HTTP/1.1" 502 150 2025-12-17 23:57:47,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7c575bc060ce1625e1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:48,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6e575bc060ce1625c6 2025-12-17 23:57:48,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6e575bc060ce1625c6 HTTP/1.1" 502 150 2025-12-17 23:57:48,655 - lava_callback - ERROR - Error processing callback for node 69409c6e575bc060ce1625c6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c6e575bc060ce1625c6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c6e575bc060ce1625c6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c6e575bc060ce1625c6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:48,657 - lava_callback - INFO - Starting async job submit for node 69409c8b575bc060ce162600 2025-12-17 23:57:48,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:48,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8b575bc060ce162600 HTTP/1.1" 502 150 2025-12-17 23:57:48,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c8b575bc060ce162600'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:48,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c8b575bc060ce162600 2025-12-17 23:57:48,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8b575bc060ce162600 HTTP/1.1" 502 150 2025-12-17 23:57:48,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c8b575bc060ce162600'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:48,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c28575bc060ce16250b 2025-12-17 23:57:48,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c48575bc060ce162571 HTTP/1.1" 502 150 2025-12-17 23:57:48,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c48575bc060ce162571'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:48,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c48575bc060ce162571 2025-12-17 23:57:48,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7e575bc060ce1625e6 2025-12-17 23:57:48,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7e575bc060ce1625e6 HTTP/1.1" 502 150 2025-12-17 23:57:48,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c7e575bc060ce1625e6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:49,801 - lava_callback - INFO - Received callback for node 6940a06b575bc060ce162d20 INFO: 172.21.0.1:55608 - "POST /node/6940a06b575bc060ce162d20 HTTP/1.0" 202 Accepted 2025-12-17 23:57:50,593 - lava_callback - INFO - Received callback for node 6940a06d575bc060ce162d2c INFO: 172.21.0.1:55622 - "POST /node/6940a06d575bc060ce162d2c HTTP/1.0" 202 Accepted 2025-12-17 23:57:50,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c8b575bc060ce162600 2025-12-17 23:57:50,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8b575bc060ce162600 HTTP/1.1" 502 150 2025-12-17 23:57:50,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c8b575bc060ce162600'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:50,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c89575bc060ce1625fd 2025-12-17 23:57:50,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c89575bc060ce1625fd HTTP/1.1" 502 150 2025-12-17 23:57:50,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c89575bc060ce1625fd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:50,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4a575bc060ce162577 2025-12-17 23:57:50,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4a575bc060ce162577 HTTP/1.1" 502 150 2025-12-17 23:57:50,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c4a575bc060ce162577'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:50,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c81575bc060ce1625ed 2025-12-17 23:57:50,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c81575bc060ce1625ed HTTP/1.1" 502 150 2025-12-17 23:57:50,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c81575bc060ce1625ed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:52,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c6f575bc060ce1625c9 2025-12-17 23:57:52,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c6f575bc060ce1625c9 HTTP/1.1" 502 150 2025-12-17 23:57:52,002 - lava_callback - ERROR - Error processing callback for node 69409c6f575bc060ce1625c9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c6f575bc060ce1625c9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c6f575bc060ce1625c9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c6f575bc060ce1625c9 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:52,004 - lava_callback - INFO - Starting async job submit for node 69409c8d575bc060ce162606 2025-12-17 23:57:52,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:52,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8d575bc060ce162606 HTTP/1.1" 502 150 2025-12-17 23:57:52,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c8d575bc060ce162606'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:52,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c8d575bc060ce162606 2025-12-17 23:57:52,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8d575bc060ce162606 HTTP/1.1" 502 150 2025-12-17 23:57:52,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c8d575bc060ce162606'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:52,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c71575bc060ce1625cc 2025-12-17 23:57:52,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c71575bc060ce1625cc HTTP/1.1" 502 150 2025-12-17 23:57:52,559 - lava_callback - ERROR - Error processing callback for node 69409c71575bc060ce1625cc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c71575bc060ce1625cc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c71575bc060ce1625cc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c71575bc060ce1625cc (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:52,561 - lava_callback - INFO - Starting async job submit for node 69409c8e575bc060ce162609 2025-12-17 23:57:52,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:52,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8e575bc060ce162609 HTTP/1.1" 502 150 2025-12-17 23:57:52,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c8e575bc060ce162609'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:52,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c8e575bc060ce162609 2025-12-17 23:57:52,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8e575bc060ce162609 HTTP/1.1" 502 150 2025-12-17 23:57:52,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c8e575bc060ce162609'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:54,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c8d575bc060ce162606 2025-12-17 23:57:54,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8d575bc060ce162606 HTTP/1.1" 502 150 2025-12-17 23:57:54,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c8d575bc060ce162606'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:54,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c8e575bc060ce162609 2025-12-17 23:57:54,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8e575bc060ce162609 HTTP/1.1" 502 150 2025-12-17 23:57:54,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c8e575bc060ce162609'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:54,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c8b575bc060ce162600 2025-12-17 23:57:54,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8b575bc060ce162600 HTTP/1.1" 502 150 2025-12-17 23:57:54,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c8b575bc060ce162600'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:54,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c83575bc060ce1625f0 2025-12-17 23:57:54,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c83575bc060ce1625f0 HTTP/1.1" 502 150 2025-12-17 23:57:54,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c83575bc060ce1625f0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:54,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c85575bc060ce1625f7 2025-12-17 23:57:54,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c85575bc060ce1625f7 HTTP/1.1" 502 150 2025-12-17 23:57:54,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c85575bc060ce1625f7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:55,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c87575bc060ce1625fa 2025-12-17 23:57:55,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c87575bc060ce1625fa HTTP/1.1" 502 150 2025-12-17 23:57:55,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c87575bc060ce1625fa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:55,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c72575bc060ce1625cf 2025-12-17 23:57:55,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c72575bc060ce1625cf HTTP/1.1" 502 150 2025-12-17 23:57:55,786 - lava_callback - ERROR - Error processing callback for node 69409c72575bc060ce1625cf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c72575bc060ce1625cf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c72575bc060ce1625cf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c72575bc060ce1625cf (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:55,787 - lava_callback - INFO - Starting async job submit for node 69409c90575bc060ce16260c 2025-12-17 23:57:55,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:55,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c90575bc060ce16260c HTTP/1.1" 502 150 2025-12-17 23:57:55,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c90575bc060ce16260c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:55,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c90575bc060ce16260c 2025-12-17 23:57:55,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c90575bc060ce16260c HTTP/1.1" 502 150 2025-12-17 23:57:55,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c90575bc060ce16260c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55630 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:57:57,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c90575bc060ce16260c 2025-12-17 23:57:57,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c90575bc060ce16260c HTTP/1.1" 502 150 2025-12-17 23:57:57,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c90575bc060ce16260c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:58,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c8d575bc060ce162606 2025-12-17 23:57:58,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8d575bc060ce162606 HTTP/1.1" 502 150 2025-12-17 23:57:58,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c8d575bc060ce162606'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:58,050 - lava_callback - INFO - Received callback for node 6940a06e575bc060ce162d31 INFO: 172.21.0.1:55642 - "POST /node/6940a06e575bc060ce162d31 HTTP/1.0" 202 Accepted 2025-12-17 23:57:58,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c8e575bc060ce162609 2025-12-17 23:57:58,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8e575bc060ce162609 HTTP/1.1" 502 150 2025-12-17 23:57:58,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c8e575bc060ce162609'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:58,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c89575bc060ce1625fd 2025-12-17 23:57:58,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c89575bc060ce1625fd HTTP/1.1" 502 150 2025-12-17 23:57:58,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c89575bc060ce1625fd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:58,714 - lava_callback - INFO - Received callback for node 6940a075575bc060ce162d37 INFO: 172.21.0.1:55654 - "POST /node/6940a075575bc060ce162d37 HTTP/1.0" 202 Accepted 2025-12-17 23:57:59,381 - lava_callback - INFO - Received callback for node 6940a076575bc060ce162d3a INFO: 172.21.0.1:55662 - "POST /node/6940a076575bc060ce162d3a HTTP/1.0" 202 Accepted 2025-12-17 23:57:59,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c74575bc060ce1625d2 2025-12-17 23:57:59,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c74575bc060ce1625d2 HTTP/1.1" 502 150 2025-12-17 23:57:59,920 - lava_callback - ERROR - Error processing callback for node 69409c74575bc060ce1625d2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c74575bc060ce1625d2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c74575bc060ce1625d2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c74575bc060ce1625d2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:57:59,922 - lava_callback - INFO - Starting async job submit for node 69409c92575bc060ce16260f 2025-12-17 23:57:59,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:57:59,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c92575bc060ce16260f HTTP/1.1" 502 150 2025-12-17 23:57:59,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c92575bc060ce16260f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:57:59,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c92575bc060ce16260f 2025-12-17 23:57:59,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c92575bc060ce16260f HTTP/1.1" 502 150 2025-12-17 23:57:59,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c92575bc060ce16260f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:00,044 - lava_callback - INFO - Received callback for node 6940a077575bc060ce162d3d INFO: 172.21.0.1:54374 - "POST /node/6940a077575bc060ce162d3d HTTP/1.0" 202 Accepted 2025-12-17 23:58:00,717 - lava_callback - INFO - Received callback for node 6940a078575bc060ce162d40 INFO: 172.21.0.1:54380 - "POST /node/6940a078575bc060ce162d40 HTTP/1.0" 202 Accepted 2025-12-17 23:58:01,391 - lava_callback - INFO - Received callback for node 6940a07a575bc060ce162d43 INFO: 172.21.0.1:54390 - "POST /node/6940a07a575bc060ce162d43 HTTP/1.0" 202 Accepted 2025-12-17 23:58:01,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c90575bc060ce16260c 2025-12-17 23:58:01,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c90575bc060ce16260c HTTP/1.1" 502 150 2025-12-17 23:58:01,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c90575bc060ce16260c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:01,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c92575bc060ce16260f 2025-12-17 23:58:01,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c92575bc060ce16260f HTTP/1.1" 502 150 2025-12-17 23:58:01,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c92575bc060ce16260f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:02,058 - lava_callback - INFO - Received callback for node 6940a07b575bc060ce162d46 INFO: 172.21.0.1:54404 - "POST /node/6940a07b575bc060ce162d46 HTTP/1.0" 202 Accepted 2025-12-17 23:58:02,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c76575bc060ce1625d5 2025-12-17 23:58:02,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c76575bc060ce1625d5 HTTP/1.1" 502 150 2025-12-17 23:58:02,623 - lava_callback - ERROR - Error processing callback for node 69409c76575bc060ce1625d5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c76575bc060ce1625d5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c76575bc060ce1625d5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c76575bc060ce1625d5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:02,626 - lava_callback - INFO - Starting async job submit for node 69409c98575bc060ce162629 2025-12-17 23:58:02,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:02,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c98575bc060ce162629 HTTP/1.1" 502 150 2025-12-17 23:58:02,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c98575bc060ce162629'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:02,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c98575bc060ce162629 2025-12-17 23:58:02,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c98575bc060ce162629 HTTP/1.1" 502 150 2025-12-17 23:58:02,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c98575bc060ce162629'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:02,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c8b575bc060ce162600 2025-12-17 23:58:02,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8b575bc060ce162600 HTTP/1.1" 502 150 2025-12-17 23:58:02,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c8b575bc060ce162600'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54420 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:58:03,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7c575bc060ce1625e1 2025-12-17 23:58:03,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7c575bc060ce1625e1 HTTP/1.1" 502 150 2025-12-17 23:58:03,939 - lava_callback - ERROR - Error processing callback for node 69409c7c575bc060ce1625e1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c7c575bc060ce1625e1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c7c575bc060ce1625e1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c7c575bc060ce1625e1 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:03,940 - lava_callback - INFO - Starting async job submit for node 69409c94575bc060ce16261c 2025-12-17 23:58:03,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:03,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c94575bc060ce16261c HTTP/1.1" 502 150 2025-12-17 23:58:03,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c94575bc060ce16261c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:03,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c94575bc060ce16261c 2025-12-17 23:58:03,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c94575bc060ce16261c HTTP/1.1" 502 150 2025-12-17 23:58:03,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c94575bc060ce16261c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:04,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c48575bc060ce162571 HTTP/1.1" 502 150 2025-12-17 23:58:04,208 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c48575bc060ce162571'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:04,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c98575bc060ce162629 2025-12-17 23:58:04,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c98575bc060ce162629 HTTP/1.1" 502 150 2025-12-17 23:58:04,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c98575bc060ce162629'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:04,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c7e575bc060ce1625e6 2025-12-17 23:58:04,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c7e575bc060ce1625e6 HTTP/1.1" 502 150 2025-12-17 23:58:04,952 - lava_callback - ERROR - Error processing callback for node 69409c7e575bc060ce1625e6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c7e575bc060ce1625e6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c7e575bc060ce1625e6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c7e575bc060ce1625e6 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:04,954 - lava_callback - INFO - Starting async job submit for node 69409c96575bc060ce162623 2025-12-17 23:58:04,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:04,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c96575bc060ce162623 HTTP/1.1" 502 150 2025-12-17 23:58:04,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c96575bc060ce162623'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:04,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c96575bc060ce162623 2025-12-17 23:58:04,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c96575bc060ce162623 HTTP/1.1" 502 150 2025-12-17 23:58:04,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c96575bc060ce162623'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:05,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c92575bc060ce16260f 2025-12-17 23:58:05,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c92575bc060ce16260f HTTP/1.1" 502 150 2025-12-17 23:58:05,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c92575bc060ce16260f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:05,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c94575bc060ce16261c 2025-12-17 23:58:05,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c94575bc060ce16261c HTTP/1.1" 502 150 2025-12-17 23:58:05,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c94575bc060ce16261c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:06,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c8d575bc060ce162606 2025-12-17 23:58:06,209 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c48575bc060ce162571 2025-12-17 23:58:06,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c48575bc060ce162571 HTTP/1.1" 502 150 2025-12-17 23:58:06,210 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c48575bc060ce162571'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54424 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:58:06,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c8e575bc060ce162609 2025-12-17 23:58:06,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8e575bc060ce162609 HTTP/1.1" 502 150 2025-12-17 23:58:06,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c8e575bc060ce162609'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:06,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c4a575bc060ce162577 2025-12-17 23:58:06,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c4a575bc060ce162577 HTTP/1.1" 502 150 2025-12-17 23:58:06,866 - lava_callback - ERROR - Error processing callback for node 69409c4a575bc060ce162577: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c4a575bc060ce162577 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c4a575bc060ce162577 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c4a575bc060ce162577 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:06,868 - lava_callback - INFO - Starting async job submit for node 69409c9b575bc060ce16262f 2025-12-17 23:58:06,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:06,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9b575bc060ce16262f HTTP/1.1" 502 150 2025-12-17 23:58:06,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9b575bc060ce16262f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:06,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9b575bc060ce16262f 2025-12-17 23:58:06,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9b575bc060ce16262f HTTP/1.1" 502 150 2025-12-17 23:58:06,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9b575bc060ce16262f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:06,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c81575bc060ce1625ed 2025-12-17 23:58:06,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c81575bc060ce1625ed HTTP/1.1" 502 150 2025-12-17 23:58:06,934 - lava_callback - ERROR - Error processing callback for node 69409c81575bc060ce1625ed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c81575bc060ce1625ed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c81575bc060ce1625ed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c81575bc060ce1625ed (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:06,937 - lava_callback - INFO - Starting async job submit for node 69409c97575bc060ce162626 2025-12-17 23:58:06,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:06,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c96575bc060ce162623 2025-12-17 23:58:06,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c96575bc060ce162623 HTTP/1.1" 502 150 2025-12-17 23:58:06,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c96575bc060ce162623'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:06,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c97575bc060ce162626 HTTP/1.1" 502 150 2025-12-17 23:58:06,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c97575bc060ce162626'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:06,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c97575bc060ce162626 2025-12-17 23:58:06,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c97575bc060ce162626 HTTP/1.1" 502 150 2025-12-17 23:58:06,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c97575bc060ce162626'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:08,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c98575bc060ce162629 2025-12-17 23:58:08,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c98575bc060ce162629 HTTP/1.1" 502 150 2025-12-17 23:58:08,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c98575bc060ce162629'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:08,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9b575bc060ce16262f 2025-12-17 23:58:08,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9b575bc060ce16262f HTTP/1.1" 502 150 2025-12-17 23:58:08,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9b575bc060ce16262f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:08,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c97575bc060ce162626 2025-12-17 23:58:08,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c97575bc060ce162626 HTTP/1.1" 502 150 2025-12-17 23:58:08,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c97575bc060ce162626'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:09,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c90575bc060ce16260c 2025-12-17 23:58:09,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c90575bc060ce16260c HTTP/1.1" 502 150 2025-12-17 23:58:09,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c90575bc060ce16260c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:09,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c94575bc060ce16261c 2025-12-17 23:58:09,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c94575bc060ce16261c HTTP/1.1" 502 150 2025-12-17 23:58:09,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c94575bc060ce16261c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:10,210 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c48575bc060ce162571 2025-12-17 23:58:10,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c48575bc060ce162571 HTTP/1.1" 502 150 2025-12-17 23:58:10,211 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c48575bc060ce162571'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:10,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c83575bc060ce1625f0 2025-12-17 23:58:10,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c83575bc060ce1625f0 HTTP/1.1" 502 150 2025-12-17 23:58:10,719 - lava_callback - ERROR - Error processing callback for node 69409c83575bc060ce1625f0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c83575bc060ce1625f0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c83575bc060ce1625f0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c83575bc060ce1625f0 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:10,721 - lava_callback - INFO - Starting async job submit for node 69409c9a575bc060ce16262c 2025-12-17 23:58:10,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:10,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9a575bc060ce16262c HTTP/1.1" 502 150 2025-12-17 23:58:10,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9a575bc060ce16262c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:10,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9a575bc060ce16262c 2025-12-17 23:58:10,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9a575bc060ce16262c HTTP/1.1" 502 150 2025-12-17 23:58:10,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9a575bc060ce16262c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:10,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c85575bc060ce1625f7 2025-12-17 23:58:10,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c85575bc060ce1625f7 HTTP/1.1" 502 150 2025-12-17 23:58:10,799 - lava_callback - ERROR - Error processing callback for node 69409c85575bc060ce1625f7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c85575bc060ce1625f7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c85575bc060ce1625f7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c85575bc060ce1625f7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:10,800 - lava_callback - INFO - Starting async job submit for node 69409c9e575bc060ce162635 2025-12-17 23:58:10,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:10,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9e575bc060ce162635 HTTP/1.1" 502 150 2025-12-17 23:58:10,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9e575bc060ce162635'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:10,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9e575bc060ce162635 2025-12-17 23:58:10,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9e575bc060ce162635 HTTP/1.1" 502 150 2025-12-17 23:58:10,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9e575bc060ce162635'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:10,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c96575bc060ce162623 2025-12-17 23:58:10,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c96575bc060ce162623 HTTP/1.1" 502 150 2025-12-17 23:58:10,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c96575bc060ce162623'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:11,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c87575bc060ce1625fa 2025-12-17 23:58:11,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c87575bc060ce1625fa HTTP/1.1" 502 150 2025-12-17 23:58:11,548 - lava_callback - ERROR - Error processing callback for node 69409c87575bc060ce1625fa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c87575bc060ce1625fa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c87575bc060ce1625fa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c87575bc060ce1625fa (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:11,549 - lava_callback - INFO - Starting async job submit for node 69409ca1575bc060ce16263b 2025-12-17 23:58:11,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:11,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca1575bc060ce16263b HTTP/1.1" 502 150 2025-12-17 23:58:11,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca1575bc060ce16263b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:11,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca1575bc060ce16263b 2025-12-17 23:58:11,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca1575bc060ce16263b HTTP/1.1" 502 150 2025-12-17 23:58:11,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca1575bc060ce16263b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:12,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9a575bc060ce16262c 2025-12-17 23:58:12,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9a575bc060ce16262c HTTP/1.1" 502 150 2025-12-17 23:58:12,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9a575bc060ce16262c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:12,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9e575bc060ce162635 2025-12-17 23:58:12,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9e575bc060ce162635 HTTP/1.1" 502 150 2025-12-17 23:58:12,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9e575bc060ce162635'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:12,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9b575bc060ce16262f 2025-12-17 23:58:12,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9b575bc060ce16262f HTTP/1.1" 502 150 2025-12-17 23:58:12,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9b575bc060ce16262f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:12,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c97575bc060ce162626 2025-12-17 23:58:12,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c97575bc060ce162626 HTTP/1.1" 502 150 2025-12-17 23:58:12,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c97575bc060ce162626'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:13,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca1575bc060ce16263b 2025-12-17 23:58:13,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca1575bc060ce16263b HTTP/1.1" 502 150 2025-12-17 23:58:13,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca1575bc060ce16263b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:13,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c92575bc060ce16260f 2025-12-17 23:58:13,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c92575bc060ce16260f HTTP/1.1" 502 150 2025-12-17 23:58:13,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c92575bc060ce16260f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:14,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c89575bc060ce1625fd 2025-12-17 23:58:14,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c89575bc060ce1625fd HTTP/1.1" 502 150 2025-12-17 23:58:14,699 - lava_callback - ERROR - Error processing callback for node 69409c89575bc060ce1625fd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c89575bc060ce1625fd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c89575bc060ce1625fd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c89575bc060ce1625fd (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:14,701 - lava_callback - INFO - Starting async job submit for node 69409ca2575bc060ce16263e 2025-12-17 23:58:14,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:14,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca2575bc060ce16263e HTTP/1.1" 502 150 2025-12-17 23:58:14,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca2575bc060ce16263e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:14,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca2575bc060ce16263e 2025-12-17 23:58:14,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca2575bc060ce16263e HTTP/1.1" 502 150 2025-12-17 23:58:14,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca2575bc060ce16263e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47612 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:58:16,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c98575bc060ce162629 2025-12-17 23:58:16,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c98575bc060ce162629 HTTP/1.1" 502 150 2025-12-17 23:58:16,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c98575bc060ce162629'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:16,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca2575bc060ce16263e 2025-12-17 23:58:16,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca2575bc060ce16263e HTTP/1.1" 502 150 2025-12-17 23:58:16,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca2575bc060ce16263e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:16,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9a575bc060ce16262c 2025-12-17 23:58:16,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9a575bc060ce16262c HTTP/1.1" 502 150 2025-12-17 23:58:16,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9a575bc060ce16262c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:16,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9e575bc060ce162635 2025-12-17 23:58:16,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9e575bc060ce162635 HTTP/1.1" 502 150 2025-12-17 23:58:16,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9e575bc060ce162635'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:17,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca1575bc060ce16263b 2025-12-17 23:58:17,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca1575bc060ce16263b HTTP/1.1" 502 150 2025-12-17 23:58:17,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca1575bc060ce16263b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:17,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c94575bc060ce16261c 2025-12-17 23:58:17,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c94575bc060ce16261c HTTP/1.1" 502 150 2025-12-17 23:58:17,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c94575bc060ce16261c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:18,212 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c48575bc060ce162571 2025-12-17 23:58:18,669 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c8b575bc060ce162600 2025-12-17 23:58:18,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8b575bc060ce162600 HTTP/1.1" 502 150 2025-12-17 23:58:18,674 - lava_callback - ERROR - Error processing callback for node 69409c8b575bc060ce162600: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c8b575bc060ce162600 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c8b575bc060ce162600 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c8b575bc060ce162600 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:18,676 - lava_callback - INFO - Starting async job submit for node 69409ca5575bc060ce162644 2025-12-17 23:58:18,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:18,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca5575bc060ce162644 HTTP/1.1" 502 150 2025-12-17 23:58:18,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca5575bc060ce162644'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:18,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca5575bc060ce162644 2025-12-17 23:58:18,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca5575bc060ce162644 HTTP/1.1" 502 150 2025-12-17 23:58:18,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca5575bc060ce162644'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:18,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c96575bc060ce162623 2025-12-17 23:58:18,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c96575bc060ce162623 HTTP/1.1" 502 150 2025-12-17 23:58:18,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c96575bc060ce162623'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:19,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c28575bc060ce16250b HTTP/1.1" 502 150 2025-12-17 23:58:19,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c28575bc060ce16250b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:20,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca5575bc060ce162644 2025-12-17 23:58:20,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca5575bc060ce162644 HTTP/1.1" 502 150 2025-12-17 23:58:20,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca5575bc060ce162644'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:20,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca2575bc060ce16263e 2025-12-17 23:58:20,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca2575bc060ce16263e HTTP/1.1" 502 150 2025-12-17 23:58:20,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca2575bc060ce16263e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:20,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9b575bc060ce16262f 2025-12-17 23:58:20,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9b575bc060ce16262f HTTP/1.1" 502 150 2025-12-17 23:58:20,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9b575bc060ce16262f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:20,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c97575bc060ce162626 2025-12-17 23:58:20,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c97575bc060ce162626 HTTP/1.1" 502 150 2025-12-17 23:58:20,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c97575bc060ce162626'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:21,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8d575bc060ce162606 HTTP/1.1" 502 150 2025-12-17 23:58:21,364 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c8d575bc060ce162606'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:22,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c8e575bc060ce162609 2025-12-17 23:58:22,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8e575bc060ce162609 HTTP/1.1" 502 150 2025-12-17 23:58:22,599 - lava_callback - ERROR - Error processing callback for node 69409c8e575bc060ce162609: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c8e575bc060ce162609 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c8e575bc060ce162609 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c8e575bc060ce162609 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:22,601 - lava_callback - INFO - Starting async job submit for node 69409c9d575bc060ce162632 2025-12-17 23:58:22,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:22,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9d575bc060ce162632 HTTP/1.1" 502 150 2025-12-17 23:58:22,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9d575bc060ce162632'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:22,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9d575bc060ce162632 2025-12-17 23:58:22,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9d575bc060ce162632 HTTP/1.1" 502 150 2025-12-17 23:58:22,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9d575bc060ce162632'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:24,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9d575bc060ce162632 2025-12-17 23:58:24,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9d575bc060ce162632 HTTP/1.1" 502 150 2025-12-17 23:58:24,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9d575bc060ce162632'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:24,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca5575bc060ce162644 2025-12-17 23:58:24,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca5575bc060ce162644 HTTP/1.1" 502 150 2025-12-17 23:58:24,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca5575bc060ce162644'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:24,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9a575bc060ce16262c 2025-12-17 23:58:24,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9a575bc060ce16262c HTTP/1.1" 502 150 2025-12-17 23:58:24,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9a575bc060ce16262c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:24,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9e575bc060ce162635 2025-12-17 23:58:24,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9e575bc060ce162635 HTTP/1.1" 502 150 2025-12-17 23:58:24,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9e575bc060ce162635'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:25,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca1575bc060ce16263b 2025-12-17 23:58:25,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca1575bc060ce16263b HTTP/1.1" 502 150 2025-12-17 23:58:25,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca1575bc060ce16263b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:25,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c90575bc060ce16260c 2025-12-17 23:58:25,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c90575bc060ce16260c HTTP/1.1" 502 150 2025-12-17 23:58:25,805 - lava_callback - ERROR - Error processing callback for node 69409c90575bc060ce16260c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c90575bc060ce16260c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c90575bc060ce16260c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c90575bc060ce16260c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:25,807 - lava_callback - INFO - Starting async job submit for node 69409c9f575bc060ce162638 2025-12-17 23:58:25,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:25,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9f575bc060ce162638 HTTP/1.1" 502 150 2025-12-17 23:58:25,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9f575bc060ce162638'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:25,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9f575bc060ce162638 2025-12-17 23:58:25,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9f575bc060ce162638 HTTP/1.1" 502 150 2025-12-17 23:58:25,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9f575bc060ce162638'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51592 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:58:27,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9f575bc060ce162638 2025-12-17 23:58:27,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9f575bc060ce162638 HTTP/1.1" 502 150 2025-12-17 23:58:27,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9f575bc060ce162638'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:28,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9d575bc060ce162632 2025-12-17 23:58:28,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9d575bc060ce162632 HTTP/1.1" 502 150 2025-12-17 23:58:28,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9d575bc060ce162632'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:28,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca2575bc060ce16263e 2025-12-17 23:58:28,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca2575bc060ce16263e HTTP/1.1" 502 150 2025-12-17 23:58:28,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca2575bc060ce16263e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:28,958 - lava_callback - INFO - Received callback for node 6940a090575bc060ce162d4e INFO: 172.21.0.1:51596 - "POST /node/6940a090575bc060ce162d4e HTTP/1.0" 202 Accepted 2025-12-17 23:58:29,602 - lava_callback - INFO - Received callback for node 6940a092575bc060ce162d59 INFO: 172.21.0.1:51600 - "POST /node/6940a092575bc060ce162d59 HTTP/1.0" 202 Accepted 2025-12-17 23:58:29,718 - lava_callback - INFO - Received callback for node 6940a095575bc060ce162d5c INFO: 172.21.0.1:33606 - "POST /node/6940a095575bc060ce162d5c HTTP/1.0" 202 Accepted 2025-12-17 23:58:29,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c92575bc060ce16260f 2025-12-17 23:58:29,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c92575bc060ce16260f HTTP/1.1" 502 150 2025-12-17 23:58:29,936 - lava_callback - ERROR - Error processing callback for node 69409c92575bc060ce16260f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c92575bc060ce16260f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c92575bc060ce16260f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c92575bc060ce16260f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:29,938 - lava_callback - INFO - Starting async job submit for node 69409ca7575bc060ce16264b 2025-12-17 23:58:29,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:29,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca7575bc060ce16264b HTTP/1.1" 502 150 2025-12-17 23:58:29,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca7575bc060ce16264b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:29,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca7575bc060ce16264b 2025-12-17 23:58:29,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca7575bc060ce16264b HTTP/1.1" 502 150 2025-12-17 23:58:29,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca7575bc060ce16264b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:30,253 - lava_callback - INFO - Received callback for node 6940a095575bc060ce162d5f INFO: 172.21.0.1:33620 - "POST /node/6940a095575bc060ce162d5f HTTP/1.0" 202 Accepted 2025-12-17 23:58:30,917 - lava_callback - INFO - Received callback for node 6940a097575bc060ce162d64 INFO: 172.21.0.1:33632 - "POST /node/6940a097575bc060ce162d64 HTTP/1.0" 202 Accepted 2025-12-17 23:58:31,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9f575bc060ce162638 2025-12-17 23:58:31,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9f575bc060ce162638 HTTP/1.1" 502 150 2025-12-17 23:58:31,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9f575bc060ce162638'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:31,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca7575bc060ce16264b 2025-12-17 23:58:31,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca7575bc060ce16264b HTTP/1.1" 502 150 2025-12-17 23:58:31,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca7575bc060ce16264b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:32,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c98575bc060ce162629 2025-12-17 23:58:32,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca5575bc060ce162644 2025-12-17 23:58:32,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca5575bc060ce162644 HTTP/1.1" 502 150 2025-12-17 23:58:32,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca5575bc060ce162644'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:33,123 - lava_callback - INFO - Received callback for node 6940a099575bc060ce162d67 INFO: 172.21.0.1:33634 - "POST /node/6940a099575bc060ce162d67 HTTP/1.0" 202 Accepted 2025-12-17 23:58:33,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c94575bc060ce16261c 2025-12-17 23:58:33,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c94575bc060ce16261c HTTP/1.1" 502 150 2025-12-17 23:58:33,953 - lava_callback - ERROR - Error processing callback for node 69409c94575bc060ce16261c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c94575bc060ce16261c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c94575bc060ce16261c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c94575bc060ce16261c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:33,955 - lava_callback - INFO - Starting async job submit for node 69409ca9575bc060ce162654 2025-12-17 23:58:33,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:33,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca9575bc060ce162654 HTTP/1.1" 502 150 2025-12-17 23:58:33,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca9575bc060ce162654'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:33,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca9575bc060ce162654 2025-12-17 23:58:33,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca9575bc060ce162654 HTTP/1.1" 502 150 2025-12-17 23:58:33,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca9575bc060ce162654'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:34,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c96575bc060ce162623 2025-12-17 23:58:34,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c96575bc060ce162623 HTTP/1.1" 502 150 2025-12-17 23:58:34,966 - lava_callback - ERROR - Error processing callback for node 69409c96575bc060ce162623: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c96575bc060ce162623 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c96575bc060ce162623 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c96575bc060ce162623 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:34,968 - lava_callback - INFO - Starting async job submit for node 69409cab575bc060ce162657 2025-12-17 23:58:34,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:34,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cab575bc060ce162657 HTTP/1.1" 502 150 2025-12-17 23:58:34,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cab575bc060ce162657'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:34,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cab575bc060ce162657 2025-12-17 23:58:34,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cab575bc060ce162657 HTTP/1.1" 502 150 2025-12-17 23:58:34,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cab575bc060ce162657'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:35,569 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c28575bc060ce16250b 2025-12-17 23:58:35,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c28575bc060ce16250b HTTP/1.1" 502 150 2025-12-17 23:58:35,570 - lava_callback - ERROR - Error processing callback for node 69409c28575bc060ce16250b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c28575bc060ce16250b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c28575bc060ce16250b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c28575bc060ce16250b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:35,573 - lava_callback - INFO - Starting async job submit for node 69409cad575bc060ce16265a 2025-12-17 23:58:35,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:35,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cad575bc060ce16265a HTTP/1.1" 502 150 2025-12-17 23:58:35,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cad575bc060ce16265a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:35,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cad575bc060ce16265a 2025-12-17 23:58:35,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cad575bc060ce16265a HTTP/1.1" 502 150 2025-12-17 23:58:35,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cad575bc060ce16265a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:35,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca7575bc060ce16264b 2025-12-17 23:58:35,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca7575bc060ce16264b HTTP/1.1" 502 150 2025-12-17 23:58:35,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca7575bc060ce16264b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:35,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca9575bc060ce162654 2025-12-17 23:58:35,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca9575bc060ce162654 HTTP/1.1" 502 150 2025-12-17 23:58:35,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca9575bc060ce162654'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33646 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:58:36,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9d575bc060ce162632 2025-12-17 23:58:36,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9d575bc060ce162632 HTTP/1.1" 502 150 2025-12-17 23:58:36,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9d575bc060ce162632'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:36,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9b575bc060ce16262f 2025-12-17 23:58:36,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9b575bc060ce16262f HTTP/1.1" 502 150 2025-12-17 23:58:36,902 - lava_callback - ERROR - Error processing callback for node 69409c9b575bc060ce16262f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c9b575bc060ce16262f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c9b575bc060ce16262f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c9b575bc060ce16262f (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:36,904 - lava_callback - INFO - Starting async job submit for node 69409caf575bc060ce162661 2025-12-17 23:58:36,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:36,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409caf575bc060ce162661 HTTP/1.1" 502 150 2025-12-17 23:58:36,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409caf575bc060ce162661'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:36,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409caf575bc060ce162661 2025-12-17 23:58:36,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409caf575bc060ce162661 HTTP/1.1" 502 150 2025-12-17 23:58:36,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409caf575bc060ce162661'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:36,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c97575bc060ce162626 2025-12-17 23:58:36,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c97575bc060ce162626 HTTP/1.1" 502 150 2025-12-17 23:58:36,968 - lava_callback - ERROR - Error processing callback for node 69409c97575bc060ce162626: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c97575bc060ce162626 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c97575bc060ce162626 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c97575bc060ce162626 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:36,970 - lava_callback - INFO - Starting async job submit for node 69409cb1575bc060ce162664 2025-12-17 23:58:36,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:36,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb1575bc060ce162664 HTTP/1.1" 502 150 2025-12-17 23:58:36,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb1575bc060ce162664'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:36,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb1575bc060ce162664 2025-12-17 23:58:36,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb1575bc060ce162664 HTTP/1.1" 502 150 2025-12-17 23:58:36,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb1575bc060ce162664'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:36,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cab575bc060ce162657 2025-12-17 23:58:36,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cab575bc060ce162657 HTTP/1.1" 502 150 2025-12-17 23:58:36,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cab575bc060ce162657'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:37,364 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c8d575bc060ce162606 2025-12-17 23:58:37,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c8d575bc060ce162606 HTTP/1.1" 502 150 2025-12-17 23:58:37,366 - lava_callback - ERROR - Error processing callback for node 69409c8d575bc060ce162606: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c8d575bc060ce162606 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c8d575bc060ce162606 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c8d575bc060ce162606 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:37,367 - lava_callback - INFO - Starting async job submit for node 69409cb3575bc060ce162667 2025-12-17 23:58:37,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:37,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb3575bc060ce162667 HTTP/1.1" 502 150 2025-12-17 23:58:37,374 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb3575bc060ce162667'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:37,374 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb3575bc060ce162667 2025-12-17 23:58:37,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb3575bc060ce162667 HTTP/1.1" 502 150 2025-12-17 23:58:37,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb3575bc060ce162667'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:37,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cad575bc060ce16265a 2025-12-17 23:58:37,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cad575bc060ce16265a HTTP/1.1" 502 150 2025-12-17 23:58:37,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cad575bc060ce16265a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:38,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409caf575bc060ce162661 2025-12-17 23:58:38,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409caf575bc060ce162661 HTTP/1.1" 502 150 2025-12-17 23:58:38,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409caf575bc060ce162661'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:38,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb1575bc060ce162664 2025-12-17 23:58:38,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb1575bc060ce162664 HTTP/1.1" 502 150 2025-12-17 23:58:38,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb1575bc060ce162664'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:39,375 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb3575bc060ce162667 2025-12-17 23:58:39,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb3575bc060ce162667 HTTP/1.1" 502 150 2025-12-17 23:58:39,376 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb3575bc060ce162667'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:39,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9f575bc060ce162638 2025-12-17 23:58:39,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9f575bc060ce162638 HTTP/1.1" 502 150 2025-12-17 23:58:39,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c9f575bc060ce162638'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:39,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca9575bc060ce162654 2025-12-17 23:58:39,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca9575bc060ce162654 HTTP/1.1" 502 150 2025-12-17 23:58:39,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca9575bc060ce162654'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:40,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9a575bc060ce16262c 2025-12-17 23:58:40,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9a575bc060ce16262c HTTP/1.1" 502 150 2025-12-17 23:58:40,735 - lava_callback - ERROR - Error processing callback for node 69409c9a575bc060ce16262c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c9a575bc060ce16262c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c9a575bc060ce16262c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c9a575bc060ce16262c (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:40,737 - lava_callback - INFO - Starting async job submit for node 69409cb5575bc060ce16266a 2025-12-17 23:58:40,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:40,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb5575bc060ce16266a HTTP/1.1" 502 150 2025-12-17 23:58:40,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb5575bc060ce16266a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:40,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb5575bc060ce16266a 2025-12-17 23:58:40,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb5575bc060ce16266a HTTP/1.1" 502 150 2025-12-17 23:58:40,765 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb5575bc060ce16266a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:40,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9e575bc060ce162635 2025-12-17 23:58:40,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9e575bc060ce162635 HTTP/1.1" 502 150 2025-12-17 23:58:40,815 - lava_callback - ERROR - Error processing callback for node 69409c9e575bc060ce162635: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c9e575bc060ce162635 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c9e575bc060ce162635 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c9e575bc060ce162635 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:40,817 - lava_callback - INFO - Starting async job submit for node 69409cb7575bc060ce16266d 2025-12-17 23:58:40,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:40,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb7575bc060ce16266d HTTP/1.1" 502 150 2025-12-17 23:58:40,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb7575bc060ce16266d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:40,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb7575bc060ce16266d 2025-12-17 23:58:40,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb7575bc060ce16266d HTTP/1.1" 502 150 2025-12-17 23:58:40,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb7575bc060ce16266d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:40,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cab575bc060ce162657 2025-12-17 23:58:40,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cab575bc060ce162657 HTTP/1.1" 502 150 2025-12-17 23:58:40,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cab575bc060ce162657'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:41,565 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca1575bc060ce16263b 2025-12-17 23:58:41,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca1575bc060ce16263b HTTP/1.1" 502 150 2025-12-17 23:58:41,566 - lava_callback - ERROR - Error processing callback for node 69409ca1575bc060ce16263b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca1575bc060ce16263b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca1575bc060ce16263b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca1575bc060ce16263b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:41,569 - lava_callback - INFO - Starting async job submit for node 69409cb9575bc060ce162670 2025-12-17 23:58:41,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:41,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb9575bc060ce162670 HTTP/1.1" 502 150 2025-12-17 23:58:41,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb9575bc060ce162670'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:41,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb9575bc060ce162670 2025-12-17 23:58:41,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb9575bc060ce162670 HTTP/1.1" 502 150 2025-12-17 23:58:41,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb9575bc060ce162670'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:41,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cad575bc060ce16265a 2025-12-17 23:58:41,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cad575bc060ce16265a HTTP/1.1" 502 150 2025-12-17 23:58:41,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cad575bc060ce16265a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:42,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb5575bc060ce16266a 2025-12-17 23:58:42,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb5575bc060ce16266a HTTP/1.1" 502 150 2025-12-17 23:58:42,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb5575bc060ce16266a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:42,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb7575bc060ce16266d 2025-12-17 23:58:42,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb7575bc060ce16266d HTTP/1.1" 502 150 2025-12-17 23:58:42,827 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb7575bc060ce16266d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:42,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409caf575bc060ce162661 2025-12-17 23:58:42,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409caf575bc060ce162661 HTTP/1.1" 502 150 2025-12-17 23:58:42,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409caf575bc060ce162661'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:42,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb1575bc060ce162664 2025-12-17 23:58:42,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb1575bc060ce162664 HTTP/1.1" 502 150 2025-12-17 23:58:42,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb1575bc060ce162664'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:43,376 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb3575bc060ce162667 2025-12-17 23:58:43,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb3575bc060ce162667 HTTP/1.1" 502 150 2025-12-17 23:58:43,377 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb3575bc060ce162667'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:43,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb9575bc060ce162670 2025-12-17 23:58:43,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb9575bc060ce162670 HTTP/1.1" 502 150 2025-12-17 23:58:43,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb9575bc060ce162670'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:43,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca7575bc060ce16264b 2025-12-17 23:58:43,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca7575bc060ce16264b HTTP/1.1" 502 150 2025-12-17 23:58:43,959 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca7575bc060ce16264b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:44,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca2575bc060ce16263e 2025-12-17 23:58:44,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca2575bc060ce16263e HTTP/1.1" 502 150 2025-12-17 23:58:44,715 - lava_callback - ERROR - Error processing callback for node 69409ca2575bc060ce16263e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca2575bc060ce16263e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca2575bc060ce16263e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca2575bc060ce16263e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:44,717 - lava_callback - INFO - Starting async job submit for node 69409cbb575bc060ce162677 2025-12-17 23:58:44,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:44,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cbb575bc060ce162677 HTTP/1.1" 502 150 2025-12-17 23:58:44,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cbb575bc060ce162677'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:44,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cbb575bc060ce162677 2025-12-17 23:58:44,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cbb575bc060ce162677 HTTP/1.1" 502 150 2025-12-17 23:58:44,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cbb575bc060ce162677'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48432 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:58:46,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cbb575bc060ce162677 2025-12-17 23:58:46,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cbb575bc060ce162677 HTTP/1.1" 502 150 2025-12-17 23:58:46,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cbb575bc060ce162677'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:46,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb5575bc060ce16266a 2025-12-17 23:58:46,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb5575bc060ce16266a HTTP/1.1" 502 150 2025-12-17 23:58:46,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb5575bc060ce16266a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:46,827 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb7575bc060ce16266d 2025-12-17 23:58:46,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb7575bc060ce16266d HTTP/1.1" 502 150 2025-12-17 23:58:46,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb7575bc060ce16266d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:47,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb9575bc060ce162670 2025-12-17 23:58:47,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb9575bc060ce162670 HTTP/1.1" 502 150 2025-12-17 23:58:47,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb9575bc060ce162670'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:47,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca9575bc060ce162654 2025-12-17 23:58:47,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca9575bc060ce162654 HTTP/1.1" 502 150 2025-12-17 23:58:47,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ca9575bc060ce162654'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:47,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c98575bc060ce162629 HTTP/1.1" 502 150 2025-12-17 23:58:47,984 - lava_callback - ERROR - Error processing callback for node 69409c98575bc060ce162629: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c98575bc060ce162629 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c98575bc060ce162629 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c98575bc060ce162629 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:47,986 - lava_callback - INFO - Starting async job submit for node 69409cbe575bc060ce162680 2025-12-17 23:58:47,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:48,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cbe575bc060ce162680 HTTP/1.1" 502 150 2025-12-17 23:58:48,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cbe575bc060ce162680'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:48,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cbe575bc060ce162680 2025-12-17 23:58:48,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cbe575bc060ce162680 HTTP/1.1" 502 150 2025-12-17 23:58:48,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cbe575bc060ce162680'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:48,694 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca5575bc060ce162644 2025-12-17 23:58:48,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca5575bc060ce162644 HTTP/1.1" 502 150 2025-12-17 23:58:48,695 - lava_callback - ERROR - Error processing callback for node 69409ca5575bc060ce162644: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca5575bc060ce162644 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca5575bc060ce162644 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca5575bc060ce162644 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:48,697 - lava_callback - INFO - Starting async job submit for node 69409cc0575bc060ce162685 2025-12-17 23:58:48,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:48,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc0575bc060ce162685 HTTP/1.1" 502 150 2025-12-17 23:58:48,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc0575bc060ce162685'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:48,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc0575bc060ce162685 2025-12-17 23:58:48,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc0575bc060ce162685 HTTP/1.1" 502 150 2025-12-17 23:58:48,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc0575bc060ce162685'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:48,980 - lava_callback - INFO - Received callback for node 6940a0a4575bc060ce162d6c INFO: 172.21.0.1:48442 - "POST /node/6940a0a4575bc060ce162d6c HTTP/1.0" 202 Accepted 2025-12-17 23:58:48,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cab575bc060ce162657 2025-12-17 23:58:48,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cab575bc060ce162657 HTTP/1.1" 502 150 2025-12-17 23:58:48,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cab575bc060ce162657'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:49,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c48575bc060ce162571 HTTP/1.1" 502 150 2025-12-17 23:58:49,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409c48575bc060ce162571'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:49,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cad575bc060ce16265a 2025-12-17 23:58:49,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cad575bc060ce16265a HTTP/1.1" 502 150 2025-12-17 23:58:49,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cad575bc060ce16265a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:49,631 - lava_callback - INFO - Received callback for node 6940a0a6575bc060ce162d79 INFO: 172.21.0.1:48450 - "POST /node/6940a0a6575bc060ce162d79 HTTP/1.0" 202 Accepted 2025-12-17 23:58:49,907 - lava_callback - INFO - Received callback for node 6940a0a9575bc060ce162d7c INFO: 172.21.0.1:38940 - "POST /node/6940a0a9575bc060ce162d7c HTTP/1.0" 202 Accepted 2025-12-17 23:58:50,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cbe575bc060ce162680 2025-12-17 23:58:50,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cbe575bc060ce162680 HTTP/1.1" 502 150 2025-12-17 23:58:50,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cbe575bc060ce162680'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:50,279 - lava_callback - INFO - Received callback for node 6940a0aa575bc060ce162d7f INFO: 172.21.0.1:38946 - "POST /node/6940a0aa575bc060ce162d7f HTTP/1.0" 202 Accepted 2025-12-17 23:58:50,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc0575bc060ce162685 2025-12-17 23:58:50,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc0575bc060ce162685 HTTP/1.1" 502 150 2025-12-17 23:58:50,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc0575bc060ce162685'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:50,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cbb575bc060ce162677 2025-12-17 23:58:50,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cbb575bc060ce162677 HTTP/1.1" 502 150 2025-12-17 23:58:50,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cbb575bc060ce162677'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:50,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409caf575bc060ce162661 2025-12-17 23:58:50,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409caf575bc060ce162661 HTTP/1.1" 502 150 2025-12-17 23:58:50,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409caf575bc060ce162661'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:50,948 - lava_callback - INFO - Received callback for node 6940a0ac575bc060ce162d82 INFO: 172.21.0.1:38954 - "POST /node/6940a0ac575bc060ce162d82 HTTP/1.0" 202 Accepted 2025-12-17 23:58:50,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb1575bc060ce162664 2025-12-17 23:58:50,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb1575bc060ce162664 HTTP/1.1" 502 150 2025-12-17 23:58:50,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb1575bc060ce162664'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:51,378 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb3575bc060ce162667 2025-12-17 23:58:51,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb3575bc060ce162667 HTTP/1.1" 502 150 2025-12-17 23:58:51,379 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb3575bc060ce162667'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:51,584 - lava_callback - INFO - Received callback for node 6940a0ae575bc060ce162d85 INFO: 172.21.0.1:38962 - "POST /node/6940a0ae575bc060ce162d85 HTTP/1.0" 202 Accepted 2025-12-17 23:58:52,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9d575bc060ce162632 2025-12-17 23:58:52,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9d575bc060ce162632 HTTP/1.1" 502 150 2025-12-17 23:58:52,625 - lava_callback - ERROR - Error processing callback for node 69409c9d575bc060ce162632: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c9d575bc060ce162632 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c9d575bc060ce162632 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c9d575bc060ce162632 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:52,627 - lava_callback - INFO - Starting async job submit for node 69409cc2575bc060ce162688 2025-12-17 23:58:52,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:52,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc2575bc060ce162688 HTTP/1.1" 502 150 2025-12-17 23:58:52,633 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc2575bc060ce162688'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:52,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc2575bc060ce162688 2025-12-17 23:58:52,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc2575bc060ce162688 HTTP/1.1" 502 150 2025-12-17 23:58:52,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc2575bc060ce162688'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:54,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cbe575bc060ce162680 2025-12-17 23:58:54,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cbe575bc060ce162680 HTTP/1.1" 502 150 2025-12-17 23:58:54,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cbe575bc060ce162680'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:54,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc2575bc060ce162688 2025-12-17 23:58:54,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc2575bc060ce162688 HTTP/1.1" 502 150 2025-12-17 23:58:54,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc2575bc060ce162688'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:54,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc0575bc060ce162685 2025-12-17 23:58:54,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc0575bc060ce162685 HTTP/1.1" 502 150 2025-12-17 23:58:54,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc0575bc060ce162685'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:54,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb5575bc060ce16266a 2025-12-17 23:58:54,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb5575bc060ce16266a HTTP/1.1" 502 150 2025-12-17 23:58:54,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb5575bc060ce16266a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:54,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb7575bc060ce16266d 2025-12-17 23:58:54,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb7575bc060ce16266d HTTP/1.1" 502 150 2025-12-17 23:58:54,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb7575bc060ce16266d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:55,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb9575bc060ce162670 2025-12-17 23:58:55,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb9575bc060ce162670 HTTP/1.1" 502 150 2025-12-17 23:58:55,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cb9575bc060ce162670'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:55,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c9f575bc060ce162638 2025-12-17 23:58:55,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c9f575bc060ce162638 HTTP/1.1" 502 150 2025-12-17 23:58:55,836 - lava_callback - ERROR - Error processing callback for node 69409c9f575bc060ce162638: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c9f575bc060ce162638 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c9f575bc060ce162638 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c9f575bc060ce162638 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:55,838 - lava_callback - INFO - Starting async job submit for node 69409cc4575bc060ce16268b 2025-12-17 23:58:55,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:55,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc4575bc060ce16268b HTTP/1.1" 502 150 2025-12-17 23:58:55,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc4575bc060ce16268b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:55,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc4575bc060ce16268b 2025-12-17 23:58:55,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc4575bc060ce16268b HTTP/1.1" 502 150 2025-12-17 23:58:55,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc4575bc060ce16268b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38970 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:58:57,561 - lava_callback - INFO - Received callback for node 6940a0c1575bc060ce162da1 INFO: 172.21.0.1:38986 - "POST /node/6940a0c1575bc060ce162da1 HTTP/1.0" 202 Accepted 2025-12-17 23:58:57,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc4575bc060ce16268b 2025-12-17 23:58:57,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc4575bc060ce16268b HTTP/1.1" 502 150 2025-12-17 23:58:57,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc4575bc060ce16268b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:58,169 - lava_callback - INFO - Received callback for node 6940a0b8575bc060ce162d8a INFO: 172.21.0.1:38992 - "POST /node/6940a0b8575bc060ce162d8a HTTP/1.0" 202 Accepted 2025-12-17 23:58:58,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc2575bc060ce162688 2025-12-17 23:58:58,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc2575bc060ce162688 HTTP/1.1" 502 150 2025-12-17 23:58:58,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc2575bc060ce162688'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:58,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cbb575bc060ce162677 2025-12-17 23:58:58,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cbb575bc060ce162677 HTTP/1.1" 502 150 2025-12-17 23:58:58,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cbb575bc060ce162677'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:58,840 - lava_callback - INFO - Received callback for node 6940a0bb575bc060ce162d93 INFO: 172.21.0.1:39004 - "POST /node/6940a0bb575bc060ce162d93 HTTP/1.0" 202 Accepted 2025-12-17 23:58:59,518 - lava_callback - INFO - Received callback for node 6940a0bd575bc060ce162d9b INFO: 172.21.0.1:39010 - "POST /node/6940a0bd575bc060ce162d9b HTTP/1.0" 202 Accepted 2025-12-17 23:58:59,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca7575bc060ce16264b 2025-12-17 23:58:59,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca7575bc060ce16264b HTTP/1.1" 502 150 2025-12-17 23:58:59,960 - lava_callback - ERROR - Error processing callback for node 69409ca7575bc060ce16264b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca7575bc060ce16264b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca7575bc060ce16264b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca7575bc060ce16264b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:58:59,962 - lava_callback - INFO - Starting async job submit for node 69409cc5575bc060ce16268e 2025-12-17 23:58:59,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:58:59,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc5575bc060ce16268e HTTP/1.1" 502 150 2025-12-17 23:58:59,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc5575bc060ce16268e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:58:59,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc5575bc060ce16268e 2025-12-17 23:58:59,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc5575bc060ce16268e HTTP/1.1" 502 150 2025-12-17 23:58:59,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc5575bc060ce16268e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:00,193 - lava_callback - INFO - Received callback for node 6940a0bf575bc060ce162d9e INFO: 172.21.0.1:35332 - "POST /node/6940a0bf575bc060ce162d9e HTTP/1.0" 202 Accepted 2025-12-17 23:59:01,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc4575bc060ce16268b 2025-12-17 23:59:01,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc4575bc060ce16268b HTTP/1.1" 502 150 2025-12-17 23:59:01,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc4575bc060ce16268b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:01,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc5575bc060ce16268e 2025-12-17 23:59:01,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc5575bc060ce16268e HTTP/1.1" 502 150 2025-12-17 23:59:01,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc5575bc060ce16268e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:02,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cbe575bc060ce162680 2025-12-17 23:59:02,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cbe575bc060ce162680 HTTP/1.1" 502 150 2025-12-17 23:59:02,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cbe575bc060ce162680'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:02,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc0575bc060ce162685 2025-12-17 23:59:02,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc0575bc060ce162685 HTTP/1.1" 502 150 2025-12-17 23:59:02,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc0575bc060ce162685'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35346 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:59:03,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ca9575bc060ce162654 2025-12-17 23:59:03,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ca9575bc060ce162654 HTTP/1.1" 502 150 2025-12-17 23:59:03,968 - lava_callback - ERROR - Error processing callback for node 69409ca9575bc060ce162654: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca9575bc060ce162654 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca9575bc060ce162654 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ca9575bc060ce162654 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:03,970 - lava_callback - INFO - Starting async job submit for node 69409cc8575bc060ce162695 2025-12-17 23:59:03,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:03,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc8575bc060ce162695 HTTP/1.1" 502 150 2025-12-17 23:59:03,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc8575bc060ce162695'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:03,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc8575bc060ce162695 2025-12-17 23:59:03,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc8575bc060ce162695 HTTP/1.1" 502 150 2025-12-17 23:59:03,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc8575bc060ce162695'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:04,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cab575bc060ce162657 2025-12-17 23:59:04,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cab575bc060ce162657 HTTP/1.1" 502 150 2025-12-17 23:59:04,992 - lava_callback - ERROR - Error processing callback for node 69409cab575bc060ce162657: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cab575bc060ce162657 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cab575bc060ce162657 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cab575bc060ce162657 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:04,994 - lava_callback - INFO - Starting async job submit for node 69409cc9575bc060ce162698 2025-12-17 23:59:04,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:05,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc9575bc060ce162698 HTTP/1.1" 502 150 2025-12-17 23:59:05,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc9575bc060ce162698'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:05,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc9575bc060ce162698 2025-12-17 23:59:05,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc9575bc060ce162698 HTTP/1.1" 502 150 2025-12-17 23:59:05,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc9575bc060ce162698'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:05,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409c48575bc060ce162571 2025-12-17 23:59:05,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cad575bc060ce16265a 2025-12-17 23:59:05,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cad575bc060ce16265a HTTP/1.1" 502 150 2025-12-17 23:59:05,601 - lava_callback - ERROR - Error processing callback for node 69409cad575bc060ce16265a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cad575bc060ce16265a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cad575bc060ce16265a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cad575bc060ce16265a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:05,603 - lava_callback - INFO - Starting async job submit for node 69409ccb575bc060ce16269b 2025-12-17 23:59:05,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:05,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccb575bc060ce16269b HTTP/1.1" 502 150 2025-12-17 23:59:05,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ccb575bc060ce16269b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:05,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ccb575bc060ce16269b 2025-12-17 23:59:05,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccb575bc060ce16269b HTTP/1.1" 502 150 2025-12-17 23:59:05,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ccb575bc060ce16269b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:05,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc5575bc060ce16268e 2025-12-17 23:59:05,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc5575bc060ce16268e HTTP/1.1" 502 150 2025-12-17 23:59:05,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc5575bc060ce16268e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:05,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc8575bc060ce162695 2025-12-17 23:59:05,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc8575bc060ce162695 HTTP/1.1" 502 150 2025-12-17 23:59:05,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc8575bc060ce162695'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35354 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:59:06,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc2575bc060ce162688 2025-12-17 23:59:06,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc2575bc060ce162688 HTTP/1.1" 502 150 2025-12-17 23:59:06,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc2575bc060ce162688'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:06,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409caf575bc060ce162661 2025-12-17 23:59:06,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409caf575bc060ce162661 HTTP/1.1" 502 150 2025-12-17 23:59:06,917 - lava_callback - ERROR - Error processing callback for node 69409caf575bc060ce162661: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409caf575bc060ce162661 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409caf575bc060ce162661 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409caf575bc060ce162661 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:06,919 - lava_callback - INFO - Starting async job submit for node 69409ccd575bc060ce16269e 2025-12-17 23:59:06,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:06,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccd575bc060ce16269e HTTP/1.1" 502 150 2025-12-17 23:59:06,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ccd575bc060ce16269e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:06,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ccd575bc060ce16269e 2025-12-17 23:59:06,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccd575bc060ce16269e HTTP/1.1" 502 150 2025-12-17 23:59:06,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ccd575bc060ce16269e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:06,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb1575bc060ce162664 2025-12-17 23:59:06,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb1575bc060ce162664 HTTP/1.1" 502 150 2025-12-17 23:59:06,983 - lava_callback - ERROR - Error processing callback for node 69409cb1575bc060ce162664: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cb1575bc060ce162664 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cb1575bc060ce162664 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cb1575bc060ce162664 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:06,986 - lava_callback - INFO - Starting async job submit for node 69409ccf575bc060ce1626a7 2025-12-17 23:59:06,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:06,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccf575bc060ce1626a7 HTTP/1.1" 502 150 2025-12-17 23:59:06,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ccf575bc060ce1626a7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:06,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ccf575bc060ce1626a7 2025-12-17 23:59:06,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccf575bc060ce1626a7 HTTP/1.1" 502 150 2025-12-17 23:59:06,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ccf575bc060ce1626a7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:07,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc9575bc060ce162698 2025-12-17 23:59:07,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc9575bc060ce162698 HTTP/1.1" 502 150 2025-12-17 23:59:07,002 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc9575bc060ce162698'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:07,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb3575bc060ce162667 2025-12-17 23:59:07,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb3575bc060ce162667 HTTP/1.1" 502 150 2025-12-17 23:59:07,381 - lava_callback - ERROR - Error processing callback for node 69409cb3575bc060ce162667: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cb3575bc060ce162667 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cb3575bc060ce162667 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cb3575bc060ce162667 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:07,383 - lava_callback - INFO - Starting async job submit for node 69409cd1575bc060ce1626b2 2025-12-17 23:59:07,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:07,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd1575bc060ce1626b2 HTTP/1.1" 502 150 2025-12-17 23:59:07,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd1575bc060ce1626b2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:07,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd1575bc060ce1626b2 2025-12-17 23:59:07,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd1575bc060ce1626b2 HTTP/1.1" 502 150 2025-12-17 23:59:07,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd1575bc060ce1626b2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:07,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ccb575bc060ce16269b 2025-12-17 23:59:07,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccb575bc060ce16269b HTTP/1.1" 502 150 2025-12-17 23:59:07,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ccb575bc060ce16269b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:08,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ccd575bc060ce16269e 2025-12-17 23:59:08,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccd575bc060ce16269e HTTP/1.1" 502 150 2025-12-17 23:59:08,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ccd575bc060ce16269e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:08,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ccf575bc060ce1626a7 2025-12-17 23:59:08,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccf575bc060ce1626a7 HTTP/1.1" 502 150 2025-12-17 23:59:08,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ccf575bc060ce1626a7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:09,393 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd1575bc060ce1626b2 2025-12-17 23:59:09,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd1575bc060ce1626b2 HTTP/1.1" 502 150 2025-12-17 23:59:09,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd1575bc060ce1626b2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:09,852 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc4575bc060ce16268b 2025-12-17 23:59:09,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc4575bc060ce16268b HTTP/1.1" 502 150 2025-12-17 23:59:09,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc4575bc060ce16268b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:09,859 - lava_callback - INFO - Received callback for node 69414248459ff5fc64d2ca17 INFO: 172.21.0.1:33722 - "POST /node/69414248459ff5fc64d2ca17 HTTP/1.0" 202 Accepted 2025-12-17 23:59:09,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc8575bc060ce162695 2025-12-17 23:59:09,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc8575bc060ce162695 HTTP/1.1" 502 150 2025-12-17 23:59:09,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc8575bc060ce162695'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:10,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb5575bc060ce16266a 2025-12-17 23:59:10,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb5575bc060ce16266a HTTP/1.1" 502 150 2025-12-17 23:59:10,771 - lava_callback - ERROR - Error processing callback for node 69409cb5575bc060ce16266a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cb5575bc060ce16266a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cb5575bc060ce16266a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cb5575bc060ce16266a (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:10,773 - lava_callback - INFO - Starting async job submit for node 69409cd3575bc060ce1626b5 2025-12-17 23:59:10,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:10,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd3575bc060ce1626b5 HTTP/1.1" 502 150 2025-12-17 23:59:10,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd3575bc060ce1626b5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:10,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd3575bc060ce1626b5 2025-12-17 23:59:10,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd3575bc060ce1626b5 HTTP/1.1" 502 150 2025-12-17 23:59:10,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd3575bc060ce1626b5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:10,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb7575bc060ce16266d 2025-12-17 23:59:10,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb7575bc060ce16266d HTTP/1.1" 502 150 2025-12-17 23:59:10,831 - lava_callback - ERROR - Error processing callback for node 69409cb7575bc060ce16266d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cb7575bc060ce16266d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cb7575bc060ce16266d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cb7575bc060ce16266d (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:10,833 - lava_callback - INFO - Starting async job submit for node 69409cd5575bc060ce1626b8 2025-12-17 23:59:10,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:10,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd5575bc060ce1626b8 HTTP/1.1" 502 150 2025-12-17 23:59:10,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd5575bc060ce1626b8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:10,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd5575bc060ce1626b8 2025-12-17 23:59:10,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd5575bc060ce1626b8 HTTP/1.1" 502 150 2025-12-17 23:59:10,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd5575bc060ce1626b8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:11,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc9575bc060ce162698 2025-12-17 23:59:11,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc9575bc060ce162698 HTTP/1.1" 502 150 2025-12-17 23:59:11,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc9575bc060ce162698'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:11,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cb9575bc060ce162670 2025-12-17 23:59:11,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cb9575bc060ce162670 HTTP/1.1" 502 150 2025-12-17 23:59:11,588 - lava_callback - ERROR - Error processing callback for node 69409cb9575bc060ce162670: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cb9575bc060ce162670 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cb9575bc060ce162670 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cb9575bc060ce162670 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:11,591 - lava_callback - INFO - Starting async job submit for node 69409cd7575bc060ce1626bb 2025-12-17 23:59:11,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:11,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd7575bc060ce1626bb HTTP/1.1" 502 150 2025-12-17 23:59:11,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd7575bc060ce1626bb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:11,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd7575bc060ce1626bb 2025-12-17 23:59:11,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd7575bc060ce1626bb HTTP/1.1" 502 150 2025-12-17 23:59:11,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd7575bc060ce1626bb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:11,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ccb575bc060ce16269b 2025-12-17 23:59:11,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccb575bc060ce16269b HTTP/1.1" 502 150 2025-12-17 23:59:11,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ccb575bc060ce16269b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:12,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd3575bc060ce1626b5 2025-12-17 23:59:12,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd3575bc060ce1626b5 HTTP/1.1" 502 150 2025-12-17 23:59:12,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd3575bc060ce1626b5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:12,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd5575bc060ce1626b8 2025-12-17 23:59:12,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd5575bc060ce1626b8 HTTP/1.1" 502 150 2025-12-17 23:59:12,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd5575bc060ce1626b8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:12,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ccd575bc060ce16269e 2025-12-17 23:59:12,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccd575bc060ce16269e HTTP/1.1" 502 150 2025-12-17 23:59:12,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ccd575bc060ce16269e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:12,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ccf575bc060ce1626a7 2025-12-17 23:59:12,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccf575bc060ce1626a7 HTTP/1.1" 502 150 2025-12-17 23:59:12,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ccf575bc060ce1626a7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:13,167 - lava_callback - INFO - Received callback for node 6940a0c5575bc060ce162da6 INFO: 172.21.0.1:33732 - "POST /node/6940a0c5575bc060ce162da6 HTTP/1.0" 202 Accepted 2025-12-17 23:59:13,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd1575bc060ce1626b2 2025-12-17 23:59:13,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd1575bc060ce1626b2 HTTP/1.1" 502 150 2025-12-17 23:59:13,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd1575bc060ce1626b2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:13,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd7575bc060ce1626bb 2025-12-17 23:59:13,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd7575bc060ce1626bb HTTP/1.1" 502 150 2025-12-17 23:59:13,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd7575bc060ce1626bb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:13,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc5575bc060ce16268e 2025-12-17 23:59:13,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc5575bc060ce16268e HTTP/1.1" 502 150 2025-12-17 23:59:13,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc5575bc060ce16268e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:14,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cbb575bc060ce162677 2025-12-17 23:59:14,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cbb575bc060ce162677 HTTP/1.1" 502 150 2025-12-17 23:59:14,734 - lava_callback - ERROR - Error processing callback for node 69409cbb575bc060ce162677: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cbb575bc060ce162677 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cbb575bc060ce162677 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cbb575bc060ce162677 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:14,737 - lava_callback - INFO - Starting async job submit for node 69409cd9575bc060ce1626c2 2025-12-17 23:59:14,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:14,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd9575bc060ce1626c2 HTTP/1.1" 502 150 2025-12-17 23:59:14,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd9575bc060ce1626c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:14,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd9575bc060ce1626c2 2025-12-17 23:59:14,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd9575bc060ce1626c2 HTTP/1.1" 502 150 2025-12-17 23:59:14,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd9575bc060ce1626c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33756 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:59:16,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd9575bc060ce1626c2 2025-12-17 23:59:16,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd9575bc060ce1626c2 HTTP/1.1" 502 150 2025-12-17 23:59:16,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd9575bc060ce1626c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:16,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd3575bc060ce1626b5 2025-12-17 23:59:16,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd3575bc060ce1626b5 HTTP/1.1" 502 150 2025-12-17 23:59:16,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd3575bc060ce1626b5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:16,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd5575bc060ce1626b8 2025-12-17 23:59:16,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd5575bc060ce1626b8 HTTP/1.1" 502 150 2025-12-17 23:59:16,870 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd5575bc060ce1626b8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:17,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd7575bc060ce1626bb 2025-12-17 23:59:17,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd7575bc060ce1626bb HTTP/1.1" 502 150 2025-12-17 23:59:17,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd7575bc060ce1626bb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:17,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc8575bc060ce162695 2025-12-17 23:59:17,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc8575bc060ce162695 HTTP/1.1" 502 150 2025-12-17 23:59:17,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc8575bc060ce162695'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:18,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cbe575bc060ce162680 2025-12-17 23:59:18,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cbe575bc060ce162680 HTTP/1.1" 502 150 2025-12-17 23:59:18,027 - lava_callback - ERROR - Error processing callback for node 69409cbe575bc060ce162680: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cbe575bc060ce162680 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cbe575bc060ce162680 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cbe575bc060ce162680 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:18,029 - lava_callback - INFO - Starting async job submit for node 69409cdb575bc060ce1626c5 2025-12-17 23:59:18,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:18,060 - lava_callback - INFO - Received callback for node 6940a0cc575bc060ce162dad INFO: 172.21.0.1:33764 - "POST /node/6940a0cc575bc060ce162dad HTTP/1.0" 202 Accepted 2025-12-17 23:59:18,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cdb575bc060ce1626c5 HTTP/1.1" 502 150 2025-12-17 23:59:18,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cdb575bc060ce1626c5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:18,071 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cdb575bc060ce1626c5 2025-12-17 23:59:18,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cdb575bc060ce1626c5 HTTP/1.1" 502 150 2025-12-17 23:59:18,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cdb575bc060ce1626c5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:18,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc0575bc060ce162685 2025-12-17 23:59:18,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc0575bc060ce162685 HTTP/1.1" 502 150 2025-12-17 23:59:18,722 - lava_callback - ERROR - Error processing callback for node 69409cc0575bc060ce162685: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc0575bc060ce162685 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc0575bc060ce162685 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc0575bc060ce162685 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:18,724 - lava_callback - INFO - Starting async job submit for node 69409cde575bc060ce1626c8 2025-12-17 23:59:18,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:18,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cde575bc060ce1626c8 HTTP/1.1" 502 150 2025-12-17 23:59:18,732 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cde575bc060ce1626c8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:18,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cde575bc060ce1626c8 2025-12-17 23:59:18,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cde575bc060ce1626c8 HTTP/1.1" 502 150 2025-12-17 23:59:18,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cde575bc060ce1626c8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:19,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc9575bc060ce162698 2025-12-17 23:59:19,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ccb575bc060ce16269b 2025-12-17 23:59:20,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cdb575bc060ce1626c5 2025-12-17 23:59:20,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409c48575bc060ce162571 HTTP/1.1" 502 150 2025-12-17 23:59:20,496 - lava_callback - ERROR - Error processing callback for node 69409c48575bc060ce162571: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c48575bc060ce162571 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c48575bc060ce162571 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409c48575bc060ce162571 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:20,498 - lava_callback - INFO - Starting async job submit for node 6940a38d575bc060ce1631f6 2025-12-17 23:59:20,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:20,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cde575bc060ce1626c8 2025-12-17 23:59:20,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd9575bc060ce1626c2 2025-12-17 23:59:20,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ccd575bc060ce16269e 2025-12-17 23:59:20,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ccf575bc060ce1626a7 2025-12-17 23:59:21,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd1575bc060ce1626b2 2025-12-17 23:59:22,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc2575bc060ce162688 2025-12-17 23:59:24,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd3575bc060ce1626b5 2025-12-17 23:59:24,870 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd5575bc060ce1626b8 2025-12-17 23:59:25,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd7575bc060ce1626bb 2025-12-17 23:59:25,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc4575bc060ce16268b INFO: 172.21.0.1:51804 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:59:29,796 - lava_callback - INFO - Received callback for node 6940a0cf575bc060ce162db4 INFO: 172.21.0.1:58660 - "POST /node/6940a0cf575bc060ce162db4 HTTP/1.0" 202 Accepted 2025-12-17 23:59:29,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc5575bc060ce16268e 2025-12-17 23:59:30,537 - lava_callback - INFO - Received callback for node 6940a0d0575bc060ce162db7 INFO: 172.21.0.1:58676 - "POST /node/6940a0d0575bc060ce162db7 HTTP/1.0" 202 Accepted 2025-12-17 23:59:31,271 - lava_callback - INFO - Received callback for node 6940a0d1575bc060ce162dba INFO: 172.21.0.1:58684 - "POST /node/6940a0d1575bc060ce162dba HTTP/1.0" 202 Accepted 2025-12-17 23:59:33,167 - lava_callback - INFO - Received callback for node 6940a0d3575bc060ce162dbf INFO: 172.21.0.1:58692 - "POST /node/6940a0d3575bc060ce162dbf HTTP/1.0" 202 Accepted 2025-12-17 23:59:33,854 - lava_callback - INFO - Received callback for node 6940a0d4575bc060ce162dc4 INFO: 172.21.0.1:58694 - "POST /node/6940a0d4575bc060ce162dc4 HTTP/1.0" 202 Accepted 2025-12-17 23:59:33,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc8575bc060ce162695 2025-12-17 23:59:34,522 - lava_callback - INFO - Received callback for node 6940a0d4575bc060ce162dc9 INFO: 172.21.0.1:58706 - "POST /node/6940a0d4575bc060ce162dc9 HTTP/1.0" 202 Accepted 2025-12-17 23:59:35,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccb575bc060ce16269b HTTP/1.1" 502 150 2025-12-17 23:59:35,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ccb575bc060ce16269b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:35,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cdb575bc060ce1626c5 HTTP/1.1" 502 150 2025-12-17 23:59:35,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cdb575bc060ce1626c5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:35,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a38d575bc060ce1631f6 HTTP/1.1" 502 150 2025-12-17 23:59:35,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a38d575bc060ce1631f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:35,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a38d575bc060ce1631f6 2025-12-17 23:59:35,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a38d575bc060ce1631f6 HTTP/1.1" 502 150 2025-12-17 23:59:35,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a38d575bc060ce1631f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:36,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd9575bc060ce1626c2 HTTP/1.1" 502 150 2025-12-17 23:59:36,112 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd9575bc060ce1626c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:36,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cde575bc060ce1626c8 HTTP/1.1" 502 150 2025-12-17 23:59:36,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cde575bc060ce1626c8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:36,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccf575bc060ce1626a7 HTTP/1.1" 502 150 2025-12-17 23:59:36,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccd575bc060ce16269e HTTP/1.1" 502 150 2025-12-17 23:59:36,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ccf575bc060ce1626a7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:36,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ccd575bc060ce16269e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58722 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:59:36,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd1575bc060ce1626b2 HTTP/1.1" 502 150 2025-12-17 23:59:36,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd1575bc060ce1626b2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:37,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a38d575bc060ce1631f6 2025-12-17 23:59:37,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a38d575bc060ce1631f6 HTTP/1.1" 502 150 2025-12-17 23:59:37,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a38d575bc060ce1631f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:37,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc2575bc060ce162688 HTTP/1.1" 502 150 2025-12-17 23:59:37,904 - lava_callback - ERROR - Error processing callback for node 69409cc2575bc060ce162688: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc2575bc060ce162688 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc2575bc060ce162688 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc2575bc060ce162688 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:37,906 - lava_callback - INFO - Starting async job submit for node 69409ceb575bc060ce1626e7 2025-12-17 23:59:37,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:37,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ceb575bc060ce1626e7 HTTP/1.1" 502 150 2025-12-17 23:59:37,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ceb575bc060ce1626e7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:37,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ceb575bc060ce1626e7 2025-12-17 23:59:37,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ceb575bc060ce1626e7 HTTP/1.1" 502 150 2025-12-17 23:59:37,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ceb575bc060ce1626e7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:39,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cdb575bc060ce1626c5 2025-12-17 23:59:39,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cdb575bc060ce1626c5 HTTP/1.1" 502 150 2025-12-17 23:59:39,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cdb575bc060ce1626c5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:39,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ceb575bc060ce1626e7 2025-12-17 23:59:39,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ceb575bc060ce1626e7 HTTP/1.1" 502 150 2025-12-17 23:59:39,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ceb575bc060ce1626e7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:40,113 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cde575bc060ce1626c8 2025-12-17 23:59:40,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cde575bc060ce1626c8 HTTP/1.1" 502 150 2025-12-17 23:59:40,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cde575bc060ce1626c8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:40,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd5575bc060ce1626b8 HTTP/1.1" 502 150 2025-12-17 23:59:40,208 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd5575bc060ce1626b8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:40,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd3575bc060ce1626b5 HTTP/1.1" 502 150 2025-12-17 23:59:40,209 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd3575bc060ce1626b5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:40,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd7575bc060ce1626bb HTTP/1.1" 502 150 2025-12-17 23:59:40,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd7575bc060ce1626bb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:41,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc4575bc060ce16268b HTTP/1.1" 502 150 2025-12-17 23:59:41,232 - lava_callback - ERROR - Error processing callback for node 69409cc4575bc060ce16268b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc4575bc060ce16268b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc4575bc060ce16268b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc4575bc060ce16268b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:41,234 - lava_callback - INFO - Starting async job submit for node 69409ce0575bc060ce1626cb 2025-12-17 23:59:41,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:41,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce0575bc060ce1626cb HTTP/1.1" 502 150 2025-12-17 23:59:41,240 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce0575bc060ce1626cb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:41,241 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce0575bc060ce1626cb 2025-12-17 23:59:41,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce0575bc060ce1626cb HTTP/1.1" 502 150 2025-12-17 23:59:41,241 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce0575bc060ce1626cb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:41,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a38d575bc060ce1631f6 2025-12-17 23:59:41,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a38d575bc060ce1631f6 HTTP/1.1" 502 150 2025-12-17 23:59:41,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a38d575bc060ce1631f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:43,241 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce0575bc060ce1626cb 2025-12-17 23:59:43,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce0575bc060ce1626cb HTTP/1.1" 502 150 2025-12-17 23:59:43,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce0575bc060ce1626cb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:43,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ceb575bc060ce1626e7 2025-12-17 23:59:43,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ceb575bc060ce1626e7 HTTP/1.1" 502 150 2025-12-17 23:59:43,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ceb575bc060ce1626e7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:44,112 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd9575bc060ce1626c2 2025-12-17 23:59:44,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd9575bc060ce1626c2 HTTP/1.1" 502 150 2025-12-17 23:59:44,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cd9575bc060ce1626c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52942 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:59:47,243 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce0575bc060ce1626cb 2025-12-17 23:59:47,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce0575bc060ce1626cb HTTP/1.1" 502 150 2025-12-17 23:59:47,244 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce0575bc060ce1626cb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:47,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cdb575bc060ce1626c5 2025-12-17 23:59:47,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cdb575bc060ce1626c5 HTTP/1.1" 502 150 2025-12-17 23:59:47,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cdb575bc060ce1626c5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:48,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cde575bc060ce1626c8 2025-12-17 23:59:48,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cde575bc060ce1626c8 HTTP/1.1" 502 150 2025-12-17 23:59:48,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cde575bc060ce1626c8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:49,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc8575bc060ce162695 HTTP/1.1" 502 150 2025-12-17 23:59:49,425 - lava_callback - ERROR - Error processing callback for node 69409cc8575bc060ce162695: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc8575bc060ce162695 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc8575bc060ce162695 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc8575bc060ce162695 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:49,427 - lava_callback - INFO - Starting async job submit for node 69409ce7575bc060ce1626dd 2025-12-17 23:59:49,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:49,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce7575bc060ce1626dd HTTP/1.1" 502 150 2025-12-17 23:59:49,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce7575bc060ce1626dd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:49,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce7575bc060ce1626dd 2025-12-17 23:59:49,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce7575bc060ce1626dd HTTP/1.1" 502 150 2025-12-17 23:59:49,439 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce7575bc060ce1626dd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:49,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc9575bc060ce162698 HTTP/1.1" 502 150 2025-12-17 23:59:49,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cc9575bc060ce162698'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:49,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a38d575bc060ce1631f6 2025-12-17 23:59:49,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a38d575bc060ce1631f6 HTTP/1.1" 502 150 2025-12-17 23:59:49,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a38d575bc060ce1631f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:51,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ccb575bc060ce16269b 2025-12-17 23:59:51,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccb575bc060ce16269b HTTP/1.1" 502 150 2025-12-17 23:59:51,090 - lava_callback - ERROR - Error processing callback for node 69409ccb575bc060ce16269b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ccb575bc060ce16269b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ccb575bc060ce16269b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ccb575bc060ce16269b (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:51,091 - lava_callback - INFO - Starting async job submit for node 69409ce5575bc060ce1626da 2025-12-17 23:59:51,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:51,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce5575bc060ce1626da HTTP/1.1" 502 150 2025-12-17 23:59:51,100 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce5575bc060ce1626da'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:51,100 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce5575bc060ce1626da 2025-12-17 23:59:51,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce5575bc060ce1626da HTTP/1.1" 502 150 2025-12-17 23:59:51,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce5575bc060ce1626da'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:51,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce7575bc060ce1626dd 2025-12-17 23:59:51,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce7575bc060ce1626dd HTTP/1.1" 502 150 2025-12-17 23:59:51,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce7575bc060ce1626dd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:51,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ceb575bc060ce1626e7 2025-12-17 23:59:51,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ceb575bc060ce1626e7 HTTP/1.1" 502 150 2025-12-17 23:59:51,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ceb575bc060ce1626e7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:52,369 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ccf575bc060ce1626a7 2025-12-17 23:59:52,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ccd575bc060ce16269e 2025-12-17 23:59:52,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccf575bc060ce1626a7 HTTP/1.1" 502 150 2025-12-17 23:59:52,376 - lava_callback - ERROR - Error processing callback for node 69409ccf575bc060ce1626a7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ccf575bc060ce1626a7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ccf575bc060ce1626a7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ccf575bc060ce1626a7 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:52,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ccd575bc060ce16269e HTTP/1.1" 502 150 2025-12-17 23:59:52,378 - lava_callback - INFO - Starting async job submit for node 69409ce9575bc060ce1626e4 2025-12-17 23:59:52,378 - lava_callback - ERROR - Error processing callback for node 69409ccd575bc060ce16269e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ccd575bc060ce16269e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ccd575bc060ce16269e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ccd575bc060ce16269e (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:52,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:52,380 - lava_callback - INFO - Starting async job submit for node 69409ced575bc060ce1626ea 2025-12-17 23:59:52,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:52,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ced575bc060ce1626ea HTTP/1.1" 502 150 2025-12-17 23:59:52,388 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ced575bc060ce1626ea'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:52,388 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ced575bc060ce1626ea 2025-12-17 23:59:52,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ced575bc060ce1626ea HTTP/1.1" 502 150 2025-12-17 23:59:52,388 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ced575bc060ce1626ea'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:52,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce9575bc060ce1626e4 HTTP/1.1" 502 150 2025-12-17 23:59:52,389 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce9575bc060ce1626e4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:52,389 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce9575bc060ce1626e4 2025-12-17 23:59:52,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce9575bc060ce1626e4 HTTP/1.1" 502 150 2025-12-17 23:59:52,390 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce9575bc060ce1626e4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:52,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd1575bc060ce1626b2 2025-12-17 23:59:52,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd1575bc060ce1626b2 HTTP/1.1" 502 150 2025-12-17 23:59:52,882 - lava_callback - ERROR - Error processing callback for node 69409cd1575bc060ce1626b2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cd1575bc060ce1626b2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cd1575bc060ce1626b2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cd1575bc060ce1626b2 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:52,883 - lava_callback - INFO - Starting async job submit for node 69409cef575bc060ce1626ed 2025-12-17 23:59:52,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:52,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cef575bc060ce1626ed HTTP/1.1" 502 150 2025-12-17 23:59:52,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cef575bc060ce1626ed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:52,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cef575bc060ce1626ed 2025-12-17 23:59:52,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cef575bc060ce1626ed HTTP/1.1" 502 150 2025-12-17 23:59:52,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cef575bc060ce1626ed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:53,101 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce5575bc060ce1626da 2025-12-17 23:59:53,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce5575bc060ce1626da HTTP/1.1" 502 150 2025-12-17 23:59:53,102 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce5575bc060ce1626da'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:54,389 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ced575bc060ce1626ea 2025-12-17 23:59:54,390 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce9575bc060ce1626e4 2025-12-17 23:59:54,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ced575bc060ce1626ea HTTP/1.1" 502 150 2025-12-17 23:59:54,391 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ced575bc060ce1626ea'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:54,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce9575bc060ce1626e4 HTTP/1.1" 502 150 2025-12-17 23:59:54,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce9575bc060ce1626e4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:54,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cef575bc060ce1626ed 2025-12-17 23:59:54,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cef575bc060ce1626ed HTTP/1.1" 502 150 2025-12-17 23:59:54,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cef575bc060ce1626ed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:55,244 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce0575bc060ce1626cb 2025-12-17 23:59:55,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce0575bc060ce1626cb HTTP/1.1" 502 150 2025-12-17 23:59:55,246 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce0575bc060ce1626cb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:55,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce7575bc060ce1626dd 2025-12-17 23:59:55,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce7575bc060ce1626dd HTTP/1.1" 502 150 2025-12-17 23:59:55,442 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce7575bc060ce1626dd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:56,209 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd5575bc060ce1626b8 2025-12-17 23:59:56,209 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd3575bc060ce1626b5 2025-12-17 23:59:56,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd5575bc060ce1626b8 HTTP/1.1" 502 150 2025-12-17 23:59:56,210 - lava_callback - ERROR - Error processing callback for node 69409cd5575bc060ce1626b8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cd5575bc060ce1626b8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cd5575bc060ce1626b8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cd5575bc060ce1626b8 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:56,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd3575bc060ce1626b5 HTTP/1.1" 502 150 2025-12-17 23:59:56,212 - lava_callback - ERROR - Error processing callback for node 69409cd3575bc060ce1626b5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cd3575bc060ce1626b5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cd3575bc060ce1626b5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cd3575bc060ce1626b5 (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:56,213 - lava_callback - INFO - Starting async job submit for node 69409cf1575bc060ce1626f0 2025-12-17 23:59:56,214 - lava_callback - INFO - Starting async job submit for node 69409cfb575bc060ce162709 2025-12-17 23:59:56,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:56,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:56,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cf1575bc060ce1626f0 HTTP/1.1" 502 150 2025-12-17 23:59:56,223 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cf1575bc060ce1626f0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:56,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfb575bc060ce162709 HTTP/1.1" 502 150 2025-12-17 23:59:56,223 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cf1575bc060ce1626f0 2025-12-17 23:59:56,223 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfb575bc060ce162709'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:56,224 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfb575bc060ce162709 2025-12-17 23:59:56,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cf1575bc060ce1626f0 HTTP/1.1" 502 150 2025-12-17 23:59:56,224 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cf1575bc060ce1626f0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:56,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfb575bc060ce162709 HTTP/1.1" 502 150 2025-12-17 23:59:56,225 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfb575bc060ce162709'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52344 - "GET / HTTP/1.0" 200 OK 2025-12-17 23:59:56,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd7575bc060ce1626bb 2025-12-17 23:59:56,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd7575bc060ce1626bb HTTP/1.1" 502 150 2025-12-17 23:59:56,979 - lava_callback - ERROR - Error processing callback for node 69409cd7575bc060ce1626bb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cd7575bc060ce1626bb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cd7575bc060ce1626bb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cd7575bc060ce1626bb (Caused by ResponseError('too many 502 error responses')) 2025-12-17 23:59:56,981 - lava_callback - INFO - Starting async job submit for node 694098c3575bc060ce161b5c 2025-12-17 23:59:57,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-17 23:59:57,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c3575bc060ce161b5c HTTP/1.1" 502 150 2025-12-17 23:59:57,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c3575bc060ce161b5c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:57,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c3575bc060ce161b5c 2025-12-17 23:59:57,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c3575bc060ce161b5c HTTP/1.1" 502 150 2025-12-17 23:59:57,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c3575bc060ce161b5c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:57,102 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce5575bc060ce1626da 2025-12-17 23:59:57,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce5575bc060ce1626da HTTP/1.1" 502 150 2025-12-17 23:59:57,104 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce5575bc060ce1626da'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:58,224 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cf1575bc060ce1626f0 2025-12-17 23:59:58,225 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfb575bc060ce162709 2025-12-17 23:59:58,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cf1575bc060ce1626f0 HTTP/1.1" 502 150 2025-12-17 23:59:58,226 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cf1575bc060ce1626f0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:58,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfb575bc060ce162709 HTTP/1.1" 502 150 2025-12-17 23:59:58,226 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfb575bc060ce162709'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:58,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ced575bc060ce1626ea 2025-12-17 23:59:58,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce9575bc060ce1626e4 2025-12-17 23:59:58,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce9575bc060ce1626e4 HTTP/1.1" 502 150 2025-12-17 23:59:58,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce9575bc060ce1626e4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:58,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ced575bc060ce1626ea HTTP/1.1" 502 150 2025-12-17 23:59:58,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ced575bc060ce1626ea'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:58,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cef575bc060ce1626ed 2025-12-17 23:59:58,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cef575bc060ce1626ed HTTP/1.1" 502 150 2025-12-17 23:59:58,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cef575bc060ce1626ed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-17 23:59:59,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c3575bc060ce161b5c 2025-12-17 23:59:59,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c3575bc060ce161b5c HTTP/1.1" 502 150 2025-12-17 23:59:59,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c3575bc060ce161b5c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:00,115 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cd9575bc060ce1626c2 2025-12-18 00:00:00,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc5575bc060ce16268e HTTP/1.1" 502 150 2025-12-18 00:00:00,689 - lava_callback - ERROR - Error processing callback for node 69409cc5575bc060ce16268e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc5575bc060ce16268e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc5575bc060ce16268e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc5575bc060ce16268e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:00,691 - lava_callback - INFO - Starting async job submit for node 69409cf7575bc060ce1626fa 2025-12-18 00:00:00,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:00,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cf7575bc060ce1626fa HTTP/1.1" 502 150 2025-12-18 00:00:00,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cf7575bc060ce1626fa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:00,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cf7575bc060ce1626fa 2025-12-18 00:00:00,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cf7575bc060ce1626fa HTTP/1.1" 502 150 2025-12-18 00:00:00,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cf7575bc060ce1626fa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:02,226 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cf1575bc060ce1626f0 2025-12-18 00:00:02,227 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfb575bc060ce162709 2025-12-18 00:00:02,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cf1575bc060ce1626f0 HTTP/1.1" 502 150 2025-12-18 00:00:02,228 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cf1575bc060ce1626f0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:02,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfb575bc060ce162709 HTTP/1.1" 502 150 2025-12-18 00:00:02,228 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfb575bc060ce162709'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:02,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cf7575bc060ce1626fa 2025-12-18 00:00:02,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cf7575bc060ce1626fa HTTP/1.1" 502 150 2025-12-18 00:00:02,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cf7575bc060ce1626fa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:03,017 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c3575bc060ce161b5c 2025-12-18 00:00:03,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c3575bc060ce161b5c HTTP/1.1" 502 150 2025-12-18 00:00:03,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c3575bc060ce161b5c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:03,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cdb575bc060ce1626c5 2025-12-18 00:00:03,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cdb575bc060ce1626c5 HTTP/1.1" 502 150 2025-12-18 00:00:03,349 - lava_callback - ERROR - Error processing callback for node 69409cdb575bc060ce1626c5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cdb575bc060ce1626c5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cdb575bc060ce1626c5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cdb575bc060ce1626c5 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:03,352 - lava_callback - INFO - Starting async job submit for node 69409cfa575bc060ce162704 2025-12-18 00:00:03,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:03,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfa575bc060ce162704 HTTP/1.1" 502 150 2025-12-18 00:00:03,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfa575bc060ce162704'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:03,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfa575bc060ce162704 2025-12-18 00:00:03,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfa575bc060ce162704 HTTP/1.1" 502 150 2025-12-18 00:00:03,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfa575bc060ce162704'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:03,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce7575bc060ce1626dd 2025-12-18 00:00:03,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce7575bc060ce1626dd HTTP/1.1" 502 150 2025-12-18 00:00:03,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce7575bc060ce1626dd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42840 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:00:04,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cde575bc060ce1626c8 2025-12-18 00:00:04,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cde575bc060ce1626c8 HTTP/1.1" 502 150 2025-12-18 00:00:04,118 - lava_callback - ERROR - Error processing callback for node 69409cde575bc060ce1626c8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cde575bc060ce1626c8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cde575bc060ce1626c8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cde575bc060ce1626c8 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:04,124 - lava_callback - INFO - Starting async job submit for node 69409cfe575bc060ce162715 2025-12-18 00:00:04,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:04,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfe575bc060ce162715 HTTP/1.1" 502 150 2025-12-18 00:00:04,157 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfe575bc060ce162715'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:04,157 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfe575bc060ce162715 2025-12-18 00:00:04,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfe575bc060ce162715 HTTP/1.1" 502 150 2025-12-18 00:00:04,158 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfe575bc060ce162715'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:05,104 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce5575bc060ce1626da 2025-12-18 00:00:05,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce5575bc060ce1626da HTTP/1.1" 502 150 2025-12-18 00:00:05,105 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce5575bc060ce1626da'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:05,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfa575bc060ce162704 2025-12-18 00:00:05,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfa575bc060ce162704 HTTP/1.1" 502 150 2025-12-18 00:00:05,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfa575bc060ce162704'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:05,680 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cc9575bc060ce162698 2025-12-18 00:00:05,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cc9575bc060ce162698 HTTP/1.1" 502 150 2025-12-18 00:00:05,682 - lava_callback - ERROR - Error processing callback for node 69409cc9575bc060ce162698: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc9575bc060ce162698 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc9575bc060ce162698 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cc9575bc060ce162698 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:05,686 - lava_callback - INFO - Starting async job submit for node 69409cfd575bc060ce162710 2025-12-18 00:00:05,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:05,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfd575bc060ce162710 HTTP/1.1" 502 150 2025-12-18 00:00:05,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfd575bc060ce162710'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:05,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfd575bc060ce162710 2025-12-18 00:00:05,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfd575bc060ce162710 HTTP/1.1" 502 150 2025-12-18 00:00:05,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfd575bc060ce162710'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:05,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a38d575bc060ce1631f6 2025-12-18 00:00:05,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a38d575bc060ce1631f6 HTTP/1.1" 502 150 2025-12-18 00:00:05,866 - lava_callback - ERROR - Error processing callback for node 6940a38d575bc060ce1631f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a38d575bc060ce1631f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a38d575bc060ce1631f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a38d575bc060ce1631f6 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:05,869 - lava_callback - INFO - Starting async job submit for node 69409d04575bc060ce162721 2025-12-18 00:00:05,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:05,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d04575bc060ce162721 HTTP/1.1" 502 150 2025-12-18 00:00:05,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d04575bc060ce162721'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:05,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d04575bc060ce162721 2025-12-18 00:00:05,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d04575bc060ce162721 HTTP/1.1" 502 150 2025-12-18 00:00:05,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d04575bc060ce162721'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:06,158 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfe575bc060ce162715 2025-12-18 00:00:06,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfe575bc060ce162715 HTTP/1.1" 502 150 2025-12-18 00:00:06,160 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfe575bc060ce162715'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:06,393 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce9575bc060ce1626e4 2025-12-18 00:00:06,394 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ced575bc060ce1626ea 2025-12-18 00:00:06,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce9575bc060ce1626e4 HTTP/1.1" 502 150 2025-12-18 00:00:06,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ce9575bc060ce1626e4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:06,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ced575bc060ce1626ea HTTP/1.1" 502 150 2025-12-18 00:00:06,395 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ced575bc060ce1626ea'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:42846 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:00:06,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cf7575bc060ce1626fa 2025-12-18 00:00:06,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cf7575bc060ce1626fa HTTP/1.1" 502 150 2025-12-18 00:00:06,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cf7575bc060ce1626fa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:06,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cef575bc060ce1626ed 2025-12-18 00:00:06,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cef575bc060ce1626ed HTTP/1.1" 502 150 2025-12-18 00:00:06,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cef575bc060ce1626ed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:07,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfd575bc060ce162710 2025-12-18 00:00:07,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfd575bc060ce162710 HTTP/1.1" 502 150 2025-12-18 00:00:07,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfd575bc060ce162710'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:07,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d04575bc060ce162721 2025-12-18 00:00:07,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d04575bc060ce162721 HTTP/1.1" 502 150 2025-12-18 00:00:07,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d04575bc060ce162721'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:07,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ceb575bc060ce1626e7 2025-12-18 00:00:07,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ceb575bc060ce1626e7 HTTP/1.1" 502 150 2025-12-18 00:00:07,929 - lava_callback - ERROR - Error processing callback for node 69409ceb575bc060ce1626e7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ceb575bc060ce1626e7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ceb575bc060ce1626e7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ceb575bc060ce1626e7 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:07,931 - lava_callback - INFO - Starting async job submit for node 69409d00575bc060ce162718 2025-12-18 00:00:07,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:07,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d00575bc060ce162718 HTTP/1.1" 502 150 2025-12-18 00:00:07,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d00575bc060ce162718'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:07,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d00575bc060ce162718 2025-12-18 00:00:07,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d00575bc060ce162718 HTTP/1.1" 502 150 2025-12-18 00:00:07,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d00575bc060ce162718'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:09,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfa575bc060ce162704 2025-12-18 00:00:09,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfa575bc060ce162704 HTTP/1.1" 502 150 2025-12-18 00:00:09,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfa575bc060ce162704'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:09,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d00575bc060ce162718 2025-12-18 00:00:09,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d00575bc060ce162718 HTTP/1.1" 502 150 2025-12-18 00:00:09,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d00575bc060ce162718'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:10,160 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfe575bc060ce162715 2025-12-18 00:00:10,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfe575bc060ce162715 HTTP/1.1" 502 150 2025-12-18 00:00:10,161 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfe575bc060ce162715'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:10,228 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cf1575bc060ce1626f0 2025-12-18 00:00:10,229 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfb575bc060ce162709 2025-12-18 00:00:10,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cf1575bc060ce1626f0 HTTP/1.1" 502 150 2025-12-18 00:00:10,229 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cf1575bc060ce1626f0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:10,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfb575bc060ce162709 HTTP/1.1" 502 150 2025-12-18 00:00:10,230 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfb575bc060ce162709'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:11,020 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c3575bc060ce161b5c 2025-12-18 00:00:11,246 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce0575bc060ce1626cb 2025-12-18 00:00:11,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfd575bc060ce162710 2025-12-18 00:00:11,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfd575bc060ce162710 HTTP/1.1" 502 150 2025-12-18 00:00:11,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfd575bc060ce162710'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:11,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d04575bc060ce162721 2025-12-18 00:00:11,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d04575bc060ce162721 HTTP/1.1" 502 150 2025-12-18 00:00:11,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d04575bc060ce162721'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:13,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d00575bc060ce162718 2025-12-18 00:00:13,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d00575bc060ce162718 HTTP/1.1" 502 150 2025-12-18 00:00:13,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d00575bc060ce162718'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:14,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cf7575bc060ce1626fa 2025-12-18 00:00:14,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cf7575bc060ce1626fa HTTP/1.1" 502 150 2025-12-18 00:00:14,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cf7575bc060ce1626fa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49668 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-18 00:00:15,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cd9575bc060ce1626c2 HTTP/1.1" 502 150 2025-12-18 00:00:15,536 - lava_callback - ERROR - Error processing callback for node 69409cd9575bc060ce1626c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cd9575bc060ce1626c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cd9575bc060ce1626c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cd9575bc060ce1626c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:15,539 - lava_callback - INFO - Starting async job submit for node 69409d01575bc060ce16271b 2025-12-18 00:00:15,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:15,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d01575bc060ce16271b HTTP/1.1" 502 150 2025-12-18 00:00:15,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d01575bc060ce16271b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:15,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d01575bc060ce16271b 2025-12-18 00:00:15,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d01575bc060ce16271b HTTP/1.1" 502 150 2025-12-18 00:00:15,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d01575bc060ce16271b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:49684 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:00:17,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfa575bc060ce162704 2025-12-18 00:00:17,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfa575bc060ce162704 HTTP/1.1" 502 150 2025-12-18 00:00:17,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfa575bc060ce162704'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:17,508 - lava_callback - INFO - Received callback for node 6940a0fd575bc060ce162ddb INFO: 172.21.0.1:49694 - "POST /node/6940a0fd575bc060ce162ddb HTTP/1.0" 202 Accepted 2025-12-18 00:00:17,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d01575bc060ce16271b 2025-12-18 00:00:17,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d01575bc060ce16271b HTTP/1.1" 502 150 2025-12-18 00:00:17,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d01575bc060ce16271b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:18,162 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfe575bc060ce162715 2025-12-18 00:00:18,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfe575bc060ce162715 HTTP/1.1" 502 150 2025-12-18 00:00:18,163 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfe575bc060ce162715'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:18,350 - lava_callback - INFO - Received callback for node 6940a100575bc060ce162de0 INFO: 172.21.0.1:49708 - "POST /node/6940a100575bc060ce162de0 HTTP/1.0" 202 Accepted 2025-12-18 00:00:19,376 - lava_callback - INFO - Received callback for node 6940a103575bc060ce162de5 INFO: 172.21.0.1:49722 - "POST /node/6940a103575bc060ce162de5 HTTP/1.0" 202 Accepted 2025-12-18 00:00:19,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce7575bc060ce1626dd 2025-12-18 00:00:19,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce7575bc060ce1626dd HTTP/1.1" 502 150 2025-12-18 00:00:19,446 - lava_callback - ERROR - Error processing callback for node 69409ce7575bc060ce1626dd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ce7575bc060ce1626dd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ce7575bc060ce1626dd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ce7575bc060ce1626dd (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:19,447 - lava_callback - INFO - Starting async job submit for node 69409d03575bc060ce16271e 2025-12-18 00:00:19,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:19,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d03575bc060ce16271e HTTP/1.1" 502 150 2025-12-18 00:00:19,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d03575bc060ce16271e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:19,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d03575bc060ce16271e 2025-12-18 00:00:19,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d03575bc060ce16271e HTTP/1.1" 502 150 2025-12-18 00:00:19,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d03575bc060ce16271e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:19,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfd575bc060ce162710 2025-12-18 00:00:19,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfd575bc060ce162710 HTTP/1.1" 502 150 2025-12-18 00:00:19,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409cfd575bc060ce162710'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:19,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d04575bc060ce162721 2025-12-18 00:00:19,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d04575bc060ce162721 HTTP/1.1" 502 150 2025-12-18 00:00:19,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d04575bc060ce162721'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:20,112 - lava_callback - INFO - Received callback for node 6940a104575bc060ce162de8 INFO: 172.21.0.1:38728 - "POST /node/6940a104575bc060ce162de8 HTTP/1.0" 202 Accepted 2025-12-18 00:00:20,882 - lava_callback - INFO - Received callback for node 6940a105575bc060ce162deb INFO: 172.21.0.1:38730 - "POST /node/6940a105575bc060ce162deb HTTP/1.0" 202 Accepted 2025-12-18 00:00:21,106 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce5575bc060ce1626da 2025-12-18 00:00:21,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce5575bc060ce1626da HTTP/1.1" 502 150 2025-12-18 00:00:21,107 - lava_callback - ERROR - Error processing callback for node 69409ce5575bc060ce1626da: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ce5575bc060ce1626da (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ce5575bc060ce1626da (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ce5575bc060ce1626da (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:21,109 - lava_callback - INFO - Starting async job submit for node 69409d07575bc060ce162724 2025-12-18 00:00:21,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:21,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d07575bc060ce162724 HTTP/1.1" 502 150 2025-12-18 00:00:21,139 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d07575bc060ce162724'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:21,139 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d07575bc060ce162724 2025-12-18 00:00:21,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d07575bc060ce162724 HTTP/1.1" 502 150 2025-12-18 00:00:21,140 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d07575bc060ce162724'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:21,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d03575bc060ce16271e 2025-12-18 00:00:21,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d03575bc060ce16271e HTTP/1.1" 502 150 2025-12-18 00:00:21,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d03575bc060ce16271e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:21,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d01575bc060ce16271b 2025-12-18 00:00:21,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d01575bc060ce16271b HTTP/1.1" 502 150 2025-12-18 00:00:21,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d01575bc060ce16271b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:21,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d00575bc060ce162718 2025-12-18 00:00:21,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d00575bc060ce162718 HTTP/1.1" 502 150 2025-12-18 00:00:21,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d00575bc060ce162718'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:22,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ce9575bc060ce1626e4 2025-12-18 00:00:22,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ced575bc060ce1626ea 2025-12-18 00:00:22,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ced575bc060ce1626ea HTTP/1.1" 502 150 2025-12-18 00:00:22,400 - lava_callback - ERROR - Error processing callback for node 69409ced575bc060ce1626ea: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ced575bc060ce1626ea (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ced575bc060ce1626ea (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ced575bc060ce1626ea (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:22,407 - lava_callback - INFO - Starting async job submit for node 69409d09575bc060ce162727 2025-12-18 00:00:22,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce9575bc060ce1626e4 HTTP/1.1" 502 150 2025-12-18 00:00:22,409 - lava_callback - ERROR - Error processing callback for node 69409ce9575bc060ce1626e4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ce9575bc060ce1626e4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ce9575bc060ce1626e4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ce9575bc060ce1626e4 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:22,413 - lava_callback - INFO - Starting async job submit for node 69409d0a575bc060ce16272a 2025-12-18 00:00:22,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:22,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:22,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0a575bc060ce16272a HTTP/1.1" 502 150 2025-12-18 00:00:22,427 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d0a575bc060ce16272a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:22,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d0a575bc060ce16272a 2025-12-18 00:00:22,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0a575bc060ce16272a HTTP/1.1" 502 150 2025-12-18 00:00:22,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d0a575bc060ce16272a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:22,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d09575bc060ce162727 HTTP/1.1" 502 150 2025-12-18 00:00:22,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d09575bc060ce162727'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:22,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d09575bc060ce162727 2025-12-18 00:00:22,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d09575bc060ce162727 HTTP/1.1" 502 150 2025-12-18 00:00:22,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d09575bc060ce162727'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:22,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cef575bc060ce1626ed 2025-12-18 00:00:22,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cef575bc060ce1626ed HTTP/1.1" 502 150 2025-12-18 00:00:22,917 - lava_callback - ERROR - Error processing callback for node 69409cef575bc060ce1626ed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cef575bc060ce1626ed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cef575bc060ce1626ed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cef575bc060ce1626ed (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:22,920 - lava_callback - INFO - Starting async job submit for node 69409d0d575bc060ce162734 2025-12-18 00:00:22,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:22,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0d575bc060ce162734 HTTP/1.1" 502 150 2025-12-18 00:00:22,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d0d575bc060ce162734'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:22,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d0d575bc060ce162734 2025-12-18 00:00:22,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0d575bc060ce162734 HTTP/1.1" 502 150 2025-12-18 00:00:22,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d0d575bc060ce162734'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:23,141 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d07575bc060ce162724 2025-12-18 00:00:23,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d07575bc060ce162724 HTTP/1.1" 502 150 2025-12-18 00:00:23,142 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d07575bc060ce162724'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:24,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d0a575bc060ce16272a 2025-12-18 00:00:24,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d09575bc060ce162727 2025-12-18 00:00:24,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d09575bc060ce162727 HTTP/1.1" 502 150 2025-12-18 00:00:24,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d09575bc060ce162727'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:24,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d0d575bc060ce162734 2025-12-18 00:00:24,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0d575bc060ce162734 HTTP/1.1" 502 150 2025-12-18 00:00:24,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d0d575bc060ce162734'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:25,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d03575bc060ce16271e 2025-12-18 00:00:25,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d03575bc060ce16271e HTTP/1.1" 502 150 2025-12-18 00:00:25,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d03575bc060ce16271e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:26,229 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cf1575bc060ce1626f0 2025-12-18 00:00:26,230 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfb575bc060ce162709 2025-12-18 00:00:26,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c3575bc060ce161b5c HTTP/1.1" 502 150 2025-12-18 00:00:26,288 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/694098c3575bc060ce161b5c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38736 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:00:26,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ce0575bc060ce1626cb HTTP/1.1" 502 150 2025-12-18 00:00:26,548 - lava_callback - ERROR - Error processing callback for node 69409ce0575bc060ce1626cb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ce0575bc060ce1626cb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ce0575bc060ce1626cb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ce0575bc060ce1626cb (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:26,551 - lava_callback - INFO - Starting async job submit for node 69409d0e575bc060ce162739 2025-12-18 00:00:26,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:26,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0e575bc060ce162739 HTTP/1.1" 502 150 2025-12-18 00:00:26,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d0e575bc060ce162739'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:26,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d0e575bc060ce162739 2025-12-18 00:00:26,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0e575bc060ce162739 HTTP/1.1" 502 150 2025-12-18 00:00:26,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d0e575bc060ce162739'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:27,143 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d07575bc060ce162724 2025-12-18 00:00:27,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d07575bc060ce162724 HTTP/1.1" 502 150 2025-12-18 00:00:27,144 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d07575bc060ce162724'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:28,436 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d09575bc060ce162727 2025-12-18 00:00:28,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d09575bc060ce162727 HTTP/1.1" 502 150 2025-12-18 00:00:28,446 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d09575bc060ce162727'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:28,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d0e575bc060ce162739 2025-12-18 00:00:28,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0e575bc060ce162739 HTTP/1.1" 502 150 2025-12-18 00:00:28,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d0e575bc060ce162739'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:28,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d0d575bc060ce162734 2025-12-18 00:00:28,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0d575bc060ce162734 HTTP/1.1" 502 150 2025-12-18 00:00:28,932 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d0d575bc060ce162734'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:29,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d01575bc060ce16271b 2025-12-18 00:00:29,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d01575bc060ce16271b HTTP/1.1" 502 150 2025-12-18 00:00:29,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d01575bc060ce16271b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:30,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cf7575bc060ce1626fa 2025-12-18 00:00:30,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cf7575bc060ce1626fa HTTP/1.1" 502 150 2025-12-18 00:00:30,713 - lava_callback - ERROR - Error processing callback for node 69409cf7575bc060ce1626fa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cf7575bc060ce1626fa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cf7575bc060ce1626fa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cf7575bc060ce1626fa (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:30,716 - lava_callback - INFO - Starting async job submit for node 69409d11575bc060ce16273c 2025-12-18 00:00:30,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:30,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d11575bc060ce16273c HTTP/1.1" 502 150 2025-12-18 00:00:30,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d11575bc060ce16273c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:30,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d11575bc060ce16273c 2025-12-18 00:00:30,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d11575bc060ce16273c HTTP/1.1" 502 150 2025-12-18 00:00:30,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d11575bc060ce16273c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:32,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d0e575bc060ce162739 2025-12-18 00:00:32,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0e575bc060ce162739 HTTP/1.1" 502 150 2025-12-18 00:00:32,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d0e575bc060ce162739'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:32,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d11575bc060ce16273c 2025-12-18 00:00:32,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d11575bc060ce16273c HTTP/1.1" 502 150 2025-12-18 00:00:32,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d11575bc060ce16273c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:33,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfa575bc060ce162704 2025-12-18 00:00:33,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfa575bc060ce162704 HTTP/1.1" 502 150 2025-12-18 00:00:33,407 - lava_callback - ERROR - Error processing callback for node 69409cfa575bc060ce162704: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cfa575bc060ce162704 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cfa575bc060ce162704 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cfa575bc060ce162704 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:33,410 - lava_callback - INFO - Starting async job submit for node 69409d13575bc060ce16273f 2025-12-18 00:00:33,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:33,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d13575bc060ce16273f HTTP/1.1" 502 150 2025-12-18 00:00:33,437 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d13575bc060ce16273f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:33,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d13575bc060ce16273f 2025-12-18 00:00:33,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d13575bc060ce16273f HTTP/1.1" 502 150 2025-12-18 00:00:33,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d13575bc060ce16273f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:33,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d03575bc060ce16271e 2025-12-18 00:00:33,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d03575bc060ce16271e HTTP/1.1" 502 150 2025-12-18 00:00:33,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d03575bc060ce16271e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:34,163 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfe575bc060ce162715 2025-12-18 00:00:34,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfe575bc060ce162715 HTTP/1.1" 502 150 2025-12-18 00:00:34,165 - lava_callback - ERROR - Error processing callback for node 69409cfe575bc060ce162715: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cfe575bc060ce162715 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cfe575bc060ce162715 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cfe575bc060ce162715 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:34,168 - lava_callback - INFO - Starting async job submit for node 69409d15575bc060ce162742 2025-12-18 00:00:34,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:34,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d15575bc060ce162742 HTTP/1.1" 502 150 2025-12-18 00:00:34,176 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d15575bc060ce162742'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:34,176 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d15575bc060ce162742 2025-12-18 00:00:34,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d15575bc060ce162742 HTTP/1.1" 502 150 2025-12-18 00:00:34,177 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d15575bc060ce162742'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:34,265 - lava_callback - INFO - Received callback for node 6940a112575bc060ce162df3 INFO: 172.21.0.1:57324 - "POST /node/6940a112575bc060ce162df3 HTTP/1.0" 202 Accepted 2025-12-18 00:00:35,144 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d07575bc060ce162724 2025-12-18 00:00:35,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d07575bc060ce162724 HTTP/1.1" 502 150 2025-12-18 00:00:35,147 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d07575bc060ce162724'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:35,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d13575bc060ce16273f 2025-12-18 00:00:35,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d13575bc060ce16273f HTTP/1.1" 502 150 2025-12-18 00:00:35,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d13575bc060ce16273f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:35,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409cfd575bc060ce162710 2025-12-18 00:00:35,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfd575bc060ce162710 HTTP/1.1" 502 150 2025-12-18 00:00:35,712 - lava_callback - ERROR - Error processing callback for node 69409cfd575bc060ce162710: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cfd575bc060ce162710 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cfd575bc060ce162710 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cfd575bc060ce162710 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:35,714 - lava_callback - INFO - Starting async job submit for node 69409d17575bc060ce162745 2025-12-18 00:00:35,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:35,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d17575bc060ce162745 HTTP/1.1" 502 150 2025-12-18 00:00:35,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d17575bc060ce162745'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:35,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d17575bc060ce162745 2025-12-18 00:00:35,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d17575bc060ce162745 HTTP/1.1" 502 150 2025-12-18 00:00:35,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d17575bc060ce162745'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:35,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d04575bc060ce162721 2025-12-18 00:00:35,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d04575bc060ce162721 HTTP/1.1" 502 150 2025-12-18 00:00:35,902 - lava_callback - ERROR - Error processing callback for node 69409d04575bc060ce162721: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d04575bc060ce162721 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d04575bc060ce162721 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d04575bc060ce162721 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:35,905 - lava_callback - INFO - Starting async job submit for node 69409d19575bc060ce162748 2025-12-18 00:00:35,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:35,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d19575bc060ce162748 HTTP/1.1" 502 150 2025-12-18 00:00:35,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d19575bc060ce162748'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:35,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d19575bc060ce162748 2025-12-18 00:00:35,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d19575bc060ce162748 HTTP/1.1" 502 150 2025-12-18 00:00:35,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d19575bc060ce162748'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:36,177 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d15575bc060ce162742 2025-12-18 00:00:36,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d15575bc060ce162742 HTTP/1.1" 502 150 2025-12-18 00:00:36,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d15575bc060ce162742'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:36,446 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d09575bc060ce162727 2025-12-18 00:00:36,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d09575bc060ce162727 HTTP/1.1" 502 150 2025-12-18 00:00:36,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d09575bc060ce162727'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57332 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:00:36,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d11575bc060ce16273c 2025-12-18 00:00:36,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d11575bc060ce16273c HTTP/1.1" 502 150 2025-12-18 00:00:36,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d11575bc060ce16273c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:36,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d0d575bc060ce162734 2025-12-18 00:00:37,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d17575bc060ce162745 2025-12-18 00:00:37,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d19575bc060ce162748 2025-12-18 00:00:37,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d19575bc060ce162748 HTTP/1.1" 502 150 2025-12-18 00:00:37,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d19575bc060ce162748'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:37,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d00575bc060ce162718 2025-12-18 00:00:37,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d00575bc060ce162718 HTTP/1.1" 502 150 2025-12-18 00:00:37,947 - lava_callback - ERROR - Error processing callback for node 69409d00575bc060ce162718: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d00575bc060ce162718 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d00575bc060ce162718 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d00575bc060ce162718 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:37,949 - lava_callback - INFO - Starting async job submit for node 69409d22575bc060ce16275e 2025-12-18 00:00:37,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:37,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d22575bc060ce16275e HTTP/1.1" 502 150 2025-12-18 00:00:37,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d22575bc060ce16275e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:37,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d22575bc060ce16275e 2025-12-18 00:00:37,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d22575bc060ce16275e HTTP/1.1" 502 150 2025-12-18 00:00:37,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d22575bc060ce16275e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:39,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d13575bc060ce16273f 2025-12-18 00:00:39,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d13575bc060ce16273f HTTP/1.1" 502 150 2025-12-18 00:00:39,441 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d13575bc060ce16273f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:39,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0a575bc060ce16272a HTTP/1.1" 502 150 2025-12-18 00:00:39,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d0a575bc060ce16272a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:39,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d22575bc060ce16275e 2025-12-18 00:00:39,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d22575bc060ce16275e HTTP/1.1" 502 150 2025-12-18 00:00:39,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d22575bc060ce16275e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:40,179 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d15575bc060ce162742 2025-12-18 00:00:40,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d15575bc060ce162742 HTTP/1.1" 502 150 2025-12-18 00:00:40,185 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d15575bc060ce162742'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:40,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d0e575bc060ce162739 2025-12-18 00:00:40,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0e575bc060ce162739 HTTP/1.1" 502 150 2025-12-18 00:00:40,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d0e575bc060ce162739'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:41,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cf1575bc060ce1626f0 HTTP/1.1" 502 150 2025-12-18 00:00:41,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409cfb575bc060ce162709 HTTP/1.1" 502 150 2025-12-18 00:00:41,649 - lava_callback - ERROR - Error processing callback for node 69409cf1575bc060ce1626f0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cf1575bc060ce1626f0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cf1575bc060ce1626f0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cf1575bc060ce1626f0 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:41,649 - lava_callback - ERROR - Error processing callback for node 69409cfb575bc060ce162709: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cfb575bc060ce162709 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cfb575bc060ce162709 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409cfb575bc060ce162709 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:41,652 - lava_callback - INFO - Starting async job submit for node 69409d27575bc060ce162768 2025-12-18 00:00:41,654 - lava_callback - INFO - Starting async job submit for node 69409910575bc060ce161c13 2025-12-18 00:00:41,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:41,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d27575bc060ce162768 HTTP/1.1" 502 150 2025-12-18 00:00:41,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d27575bc060ce162768'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:41,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d27575bc060ce162768 2025-12-18 00:00:41,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:41,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d27575bc060ce162768 HTTP/1.1" 502 150 2025-12-18 00:00:41,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d27575bc060ce162768'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:41,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409910575bc060ce161c13 HTTP/1.1" 502 150 2025-12-18 00:00:41,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409910575bc060ce161c13'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:41,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409910575bc060ce161c13 2025-12-18 00:00:41,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409910575bc060ce161c13 HTTP/1.1" 502 150 2025-12-18 00:00:41,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409910575bc060ce161c13'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:41,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d19575bc060ce162748 2025-12-18 00:00:41,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d19575bc060ce162748 HTTP/1.1" 502 150 2025-12-18 00:00:41,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d19575bc060ce162748'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:42,288 - urllib3.connectionpool - DEBUG - Retry: /latest/node/694098c3575bc060ce161b5c 2025-12-18 00:00:42,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/694098c3575bc060ce161b5c HTTP/1.1" 502 150 2025-12-18 00:00:42,290 - lava_callback - ERROR - Error processing callback for node 694098c3575bc060ce161b5c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c3575bc060ce161b5c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c3575bc060ce161b5c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/694098c3575bc060ce161b5c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:42,292 - lava_callback - INFO - Starting async job submit for node 69409d1f575bc060ce162753 2025-12-18 00:00:42,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:42,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1f575bc060ce162753 HTTP/1.1" 502 150 2025-12-18 00:00:42,300 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d1f575bc060ce162753'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:42,300 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d1f575bc060ce162753 2025-12-18 00:00:42,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1f575bc060ce162753 HTTP/1.1" 502 150 2025-12-18 00:00:42,301 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d1f575bc060ce162753'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:43,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d27575bc060ce162768 2025-12-18 00:00:43,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d27575bc060ce162768 HTTP/1.1" 502 150 2025-12-18 00:00:43,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d27575bc060ce162768'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:43,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409910575bc060ce161c13 2025-12-18 00:00:43,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409910575bc060ce161c13 HTTP/1.1" 502 150 2025-12-18 00:00:43,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409910575bc060ce161c13'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:43,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d0a575bc060ce16272a 2025-12-18 00:00:43,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0a575bc060ce16272a HTTP/1.1" 502 150 2025-12-18 00:00:43,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d0a575bc060ce16272a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:43,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d22575bc060ce16275e 2025-12-18 00:00:43,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d22575bc060ce16275e HTTP/1.1" 502 150 2025-12-18 00:00:43,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d22575bc060ce16275e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:44,301 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d1f575bc060ce162753 2025-12-18 00:00:44,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1f575bc060ce162753 HTTP/1.1" 502 150 2025-12-18 00:00:44,302 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d1f575bc060ce162753'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:44,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d11575bc060ce16273c 2025-12-18 00:00:44,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d11575bc060ce16273c HTTP/1.1" 502 150 2025-12-18 00:00:44,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d11575bc060ce16273c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:45,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d01575bc060ce16271b 2025-12-18 00:00:45,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d01575bc060ce16271b HTTP/1.1" 502 150 2025-12-18 00:00:45,554 - lava_callback - ERROR - Error processing callback for node 69409d01575bc060ce16271b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d01575bc060ce16271b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d01575bc060ce16271b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d01575bc060ce16271b (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:45,557 - lava_callback - INFO - Starting async job submit for node 69409d25575bc060ce162765 2025-12-18 00:00:45,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:45,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d25575bc060ce162765 HTTP/1.1" 502 150 2025-12-18 00:00:45,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d25575bc060ce162765'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:45,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d25575bc060ce162765 2025-12-18 00:00:45,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d25575bc060ce162765 HTTP/1.1" 502 150 2025-12-18 00:00:45,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d25575bc060ce162765'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56174 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:00:47,441 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d13575bc060ce16273f 2025-12-18 00:00:47,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d13575bc060ce16273f HTTP/1.1" 502 150 2025-12-18 00:00:47,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d13575bc060ce16273f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:47,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d25575bc060ce162765 2025-12-18 00:00:47,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d25575bc060ce162765 HTTP/1.1" 502 150 2025-12-18 00:00:47,574 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d25575bc060ce162765'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:47,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d27575bc060ce162768 2025-12-18 00:00:47,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d27575bc060ce162768 HTTP/1.1" 502 150 2025-12-18 00:00:47,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d27575bc060ce162768'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:47,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409910575bc060ce161c13 2025-12-18 00:00:47,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409910575bc060ce161c13 HTTP/1.1" 502 150 2025-12-18 00:00:47,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409910575bc060ce161c13'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:48,185 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d15575bc060ce162742 2025-12-18 00:00:48,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d15575bc060ce162742 HTTP/1.1" 502 150 2025-12-18 00:00:48,187 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d15575bc060ce162742'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:48,302 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d1f575bc060ce162753 2025-12-18 00:00:48,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1f575bc060ce162753 HTTP/1.1" 502 150 2025-12-18 00:00:48,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d1f575bc060ce162753'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:49,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d03575bc060ce16271e 2025-12-18 00:00:49,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d03575bc060ce16271e HTTP/1.1" 502 150 2025-12-18 00:00:49,463 - lava_callback - ERROR - Error processing callback for node 69409d03575bc060ce16271e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d03575bc060ce16271e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d03575bc060ce16271e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d03575bc060ce16271e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:49,466 - lava_callback - INFO - Starting async job submit for node 69409d29575bc060ce16276b 2025-12-18 00:00:49,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:49,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d29575bc060ce16276b HTTP/1.1" 502 150 2025-12-18 00:00:49,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d29575bc060ce16276b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:49,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d29575bc060ce16276b 2025-12-18 00:00:49,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d29575bc060ce16276b HTTP/1.1" 502 150 2025-12-18 00:00:49,475 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d29575bc060ce16276b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:49,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d19575bc060ce162748 2025-12-18 00:00:49,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d19575bc060ce162748 HTTP/1.1" 502 150 2025-12-18 00:00:49,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d19575bc060ce162748'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:51,147 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d07575bc060ce162724 2025-12-18 00:00:51,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d07575bc060ce162724 HTTP/1.1" 502 150 2025-12-18 00:00:51,149 - lava_callback - ERROR - Error processing callback for node 69409d07575bc060ce162724: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d07575bc060ce162724 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d07575bc060ce162724 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d07575bc060ce162724 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:51,151 - lava_callback - INFO - Starting async job submit for node 69409d2b575bc060ce16276e 2025-12-18 00:00:51,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:51,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2b575bc060ce16276e HTTP/1.1" 502 150 2025-12-18 00:00:51,182 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d2b575bc060ce16276e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:51,182 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d2b575bc060ce16276e 2025-12-18 00:00:51,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2b575bc060ce16276e HTTP/1.1" 502 150 2025-12-18 00:00:51,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d2b575bc060ce16276e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:51,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d29575bc060ce16276b 2025-12-18 00:00:51,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d29575bc060ce16276b HTTP/1.1" 502 150 2025-12-18 00:00:51,477 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d29575bc060ce16276b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:51,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d25575bc060ce162765 2025-12-18 00:00:51,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d25575bc060ce162765 HTTP/1.1" 502 150 2025-12-18 00:00:51,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d25575bc060ce162765'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:51,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d0a575bc060ce16272a 2025-12-18 00:00:51,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d22575bc060ce16275e 2025-12-18 00:00:51,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d22575bc060ce16275e HTTP/1.1" 502 150 2025-12-18 00:00:51,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d22575bc060ce16275e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:52,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0d575bc060ce162734 HTTP/1.1" 502 150 2025-12-18 00:00:52,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d0d575bc060ce162734'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:52,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d09575bc060ce162727 2025-12-18 00:00:52,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d09575bc060ce162727 HTTP/1.1" 502 150 2025-12-18 00:00:52,449 - lava_callback - ERROR - Error processing callback for node 69409d09575bc060ce162727: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d09575bc060ce162727 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d09575bc060ce162727 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d09575bc060ce162727 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:52,451 - lava_callback - INFO - Starting async job submit for node 69409d2d575bc060ce162771 2025-12-18 00:00:52,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:52,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2d575bc060ce162771 HTTP/1.1" 502 150 2025-12-18 00:00:52,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d2d575bc060ce162771'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:52,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d2d575bc060ce162771 2025-12-18 00:00:52,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2d575bc060ce162771 HTTP/1.1" 502 150 2025-12-18 00:00:52,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d2d575bc060ce162771'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:53,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d17575bc060ce162745 HTTP/1.1" 502 150 2025-12-18 00:00:53,169 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d17575bc060ce162745'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:53,183 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d2b575bc060ce16276e 2025-12-18 00:00:53,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2b575bc060ce16276e HTTP/1.1" 502 150 2025-12-18 00:00:53,186 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d2b575bc060ce16276e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:53,356 - lava_callback - INFO - Received callback for node 6940a129575bc060ce162dfa INFO: 172.21.0.1:46238 - "POST /node/6940a129575bc060ce162dfa HTTP/1.0" 202 Accepted 2025-12-18 00:00:54,198 - lava_callback - INFO - Received callback for node 6940a12d575bc060ce162dfd INFO: 172.21.0.1:46240 - "POST /node/6940a12d575bc060ce162dfd HTTP/1.0" 202 Accepted 2025-12-18 00:00:54,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d2d575bc060ce162771 2025-12-18 00:00:54,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2d575bc060ce162771 HTTP/1.1" 502 150 2025-12-18 00:00:54,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d2d575bc060ce162771'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:55,477 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d29575bc060ce16276b 2025-12-18 00:00:55,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d29575bc060ce16276b HTTP/1.1" 502 150 2025-12-18 00:00:55,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d29575bc060ce16276b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:55,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d27575bc060ce162768 2025-12-18 00:00:55,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d27575bc060ce162768 HTTP/1.1" 502 150 2025-12-18 00:00:55,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d27575bc060ce162768'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:55,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409910575bc060ce161c13 2025-12-18 00:00:55,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409910575bc060ce161c13 HTTP/1.1" 502 150 2025-12-18 00:00:55,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409910575bc060ce161c13'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:56,304 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d1f575bc060ce162753 2025-12-18 00:00:56,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1f575bc060ce162753 HTTP/1.1" 502 150 2025-12-18 00:00:56,305 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d1f575bc060ce162753'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46256 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:00:56,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d0e575bc060ce162739 2025-12-18 00:00:56,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0e575bc060ce162739 HTTP/1.1" 502 150 2025-12-18 00:00:56,574 - lava_callback - ERROR - Error processing callback for node 69409d0e575bc060ce162739: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d0e575bc060ce162739 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d0e575bc060ce162739 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d0e575bc060ce162739 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:00:56,577 - lava_callback - INFO - Starting async job submit for node 69409d2f575bc060ce162774 2025-12-18 00:00:56,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:00:56,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2f575bc060ce162774 HTTP/1.1" 502 150 2025-12-18 00:00:56,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d2f575bc060ce162774'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:56,585 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d2f575bc060ce162774 2025-12-18 00:00:56,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2f575bc060ce162774 HTTP/1.1" 502 150 2025-12-18 00:00:56,586 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d2f575bc060ce162774'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:57,169 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d17575bc060ce162745 2025-12-18 00:00:57,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d17575bc060ce162745 HTTP/1.1" 502 150 2025-12-18 00:00:57,170 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d17575bc060ce162745'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:57,186 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d2b575bc060ce16276e 2025-12-18 00:00:57,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2b575bc060ce16276e HTTP/1.1" 502 150 2025-12-18 00:00:57,187 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d2b575bc060ce16276e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:58,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d2d575bc060ce162771 2025-12-18 00:00:58,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2d575bc060ce162771 HTTP/1.1" 502 150 2025-12-18 00:00:58,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d2d575bc060ce162771'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:58,586 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d2f575bc060ce162774 2025-12-18 00:00:58,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2f575bc060ce162774 HTTP/1.1" 502 150 2025-12-18 00:00:58,587 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d2f575bc060ce162774'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:00:59,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d25575bc060ce162765 2025-12-18 00:00:59,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d25575bc060ce162765 HTTP/1.1" 502 150 2025-12-18 00:00:59,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d25575bc060ce162765'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:00,731 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d11575bc060ce16273c 2025-12-18 00:01:00,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d11575bc060ce16273c HTTP/1.1" 502 150 2025-12-18 00:01:00,732 - lava_callback - ERROR - Error processing callback for node 69409d11575bc060ce16273c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d11575bc060ce16273c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d11575bc060ce16273c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d11575bc060ce16273c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:00,735 - lava_callback - INFO - Starting async job submit for node 69409d33575bc060ce16277a 2025-12-18 00:01:00,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:00,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d33575bc060ce16277a HTTP/1.1" 502 150 2025-12-18 00:01:00,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d33575bc060ce16277a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:00,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d33575bc060ce16277a 2025-12-18 00:01:00,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d33575bc060ce16277a HTTP/1.1" 502 150 2025-12-18 00:01:00,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d33575bc060ce16277a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:02,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d2f575bc060ce162774 2025-12-18 00:01:02,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2f575bc060ce162774 HTTP/1.1" 502 150 2025-12-18 00:01:02,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d2f575bc060ce162774'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:02,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d33575bc060ce16277a 2025-12-18 00:01:02,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d33575bc060ce16277a HTTP/1.1" 502 150 2025-12-18 00:01:02,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d33575bc060ce16277a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:03,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d13575bc060ce16273f 2025-12-18 00:01:03,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d13575bc060ce16273f HTTP/1.1" 502 150 2025-12-18 00:01:03,444 - lava_callback - ERROR - Error processing callback for node 69409d13575bc060ce16273f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d13575bc060ce16273f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d13575bc060ce16273f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d13575bc060ce16273f (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:03,446 - lava_callback - INFO - Starting async job submit for node 69409d31575bc060ce162777 2025-12-18 00:01:03,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:03,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d31575bc060ce162777 HTTP/1.1" 502 150 2025-12-18 00:01:03,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d31575bc060ce162777'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:03,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d31575bc060ce162777 2025-12-18 00:01:03,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d31575bc060ce162777 HTTP/1.1" 502 150 2025-12-18 00:01:03,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d31575bc060ce162777'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:03,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d29575bc060ce16276b 2025-12-18 00:01:03,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d29575bc060ce16276b HTTP/1.1" 502 150 2025-12-18 00:01:03,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d29575bc060ce16276b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44042 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:01:04,187 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d15575bc060ce162742 2025-12-18 00:01:05,171 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d17575bc060ce162745 2025-12-18 00:01:05,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d17575bc060ce162745 HTTP/1.1" 502 150 2025-12-18 00:01:05,172 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d17575bc060ce162745'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:05,188 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d2b575bc060ce16276e 2025-12-18 00:01:05,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2b575bc060ce16276e HTTP/1.1" 502 150 2025-12-18 00:01:05,189 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d2b575bc060ce16276e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:05,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d31575bc060ce162777 2025-12-18 00:01:05,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d31575bc060ce162777 HTTP/1.1" 502 150 2025-12-18 00:01:05,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d31575bc060ce162777'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:05,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d19575bc060ce162748 2025-12-18 00:01:05,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d19575bc060ce162748 HTTP/1.1" 502 150 2025-12-18 00:01:05,924 - lava_callback - ERROR - Error processing callback for node 69409d19575bc060ce162748: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d19575bc060ce162748 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d19575bc060ce162748 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d19575bc060ce162748 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:05,927 - lava_callback - INFO - Starting async job submit for node 69409d35575bc060ce162784 2025-12-18 00:01:05,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:05,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d35575bc060ce162784 HTTP/1.1" 502 150 2025-12-18 00:01:05,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d35575bc060ce162784'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:05,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d35575bc060ce162784 2025-12-18 00:01:05,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d35575bc060ce162784 HTTP/1.1" 502 150 2025-12-18 00:01:05,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d35575bc060ce162784'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:06,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d2d575bc060ce162771 2025-12-18 00:01:06,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2d575bc060ce162771 HTTP/1.1" 502 150 2025-12-18 00:01:06,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d2d575bc060ce162771'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:44056 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:01:06,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d33575bc060ce16277a 2025-12-18 00:01:06,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d33575bc060ce16277a HTTP/1.1" 502 150 2025-12-18 00:01:06,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d33575bc060ce16277a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:07,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d35575bc060ce162784 2025-12-18 00:01:07,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d35575bc060ce162784 HTTP/1.1" 502 150 2025-12-18 00:01:07,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d35575bc060ce162784'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:07,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d22575bc060ce16275e 2025-12-18 00:01:07,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d22575bc060ce16275e HTTP/1.1" 502 150 2025-12-18 00:01:07,967 - lava_callback - ERROR - Error processing callback for node 69409d22575bc060ce16275e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d22575bc060ce16275e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d22575bc060ce16275e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d22575bc060ce16275e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:07,969 - lava_callback - INFO - Starting async job submit for node 69409d3b575bc060ce162797 2025-12-18 00:01:07,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:07,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d3b575bc060ce162797 HTTP/1.1" 502 150 2025-12-18 00:01:07,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d3b575bc060ce162797'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:07,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d3b575bc060ce162797 2025-12-18 00:01:07,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d3b575bc060ce162797 HTTP/1.1" 502 150 2025-12-18 00:01:07,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d3b575bc060ce162797'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:08,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d0d575bc060ce162734 2025-12-18 00:01:08,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0d575bc060ce162734 HTTP/1.1" 502 150 2025-12-18 00:01:08,406 - lava_callback - ERROR - Error processing callback for node 69409d0d575bc060ce162734: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d0d575bc060ce162734 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d0d575bc060ce162734 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d0d575bc060ce162734 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:08,407 - lava_callback - INFO - Starting async job submit for node 69409d37575bc060ce16278d 2025-12-18 00:01:08,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:08,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d37575bc060ce16278d HTTP/1.1" 502 150 2025-12-18 00:01:08,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d37575bc060ce16278d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:08,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d37575bc060ce16278d 2025-12-18 00:01:08,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d37575bc060ce16278d HTTP/1.1" 502 150 2025-12-18 00:01:08,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d37575bc060ce16278d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:09,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d31575bc060ce162777 2025-12-18 00:01:09,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d31575bc060ce162777 HTTP/1.1" 502 150 2025-12-18 00:01:09,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d31575bc060ce162777'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:09,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d3b575bc060ce162797 2025-12-18 00:01:09,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d3b575bc060ce162797 HTTP/1.1" 502 150 2025-12-18 00:01:09,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d3b575bc060ce162797'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:10,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d37575bc060ce16278d 2025-12-18 00:01:10,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d37575bc060ce16278d HTTP/1.1" 502 150 2025-12-18 00:01:10,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d37575bc060ce16278d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:10,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d2f575bc060ce162774 2025-12-18 00:01:10,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2f575bc060ce162774 HTTP/1.1" 502 150 2025-12-18 00:01:10,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d2f575bc060ce162774'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:11,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d27575bc060ce162768 2025-12-18 00:01:11,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d27575bc060ce162768 HTTP/1.1" 502 150 2025-12-18 00:01:11,722 - lava_callback - ERROR - Error processing callback for node 69409d27575bc060ce162768: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d27575bc060ce162768 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d27575bc060ce162768 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d27575bc060ce162768 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:11,724 - lava_callback - INFO - Starting async job submit for node 69409d39575bc060ce162794 2025-12-18 00:01:11,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:11,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409910575bc060ce161c13 2025-12-18 00:01:11,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409910575bc060ce161c13 HTTP/1.1" 502 150 2025-12-18 00:01:11,728 - lava_callback - ERROR - Error processing callback for node 69409910575bc060ce161c13: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409910575bc060ce161c13 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409910575bc060ce161c13 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409910575bc060ce161c13 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:11,730 - lava_callback - INFO - Starting async job submit for node 69409d40575bc060ce1627a0 2025-12-18 00:01:11,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:11,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d39575bc060ce162794 HTTP/1.1" 502 150 2025-12-18 00:01:11,734 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d39575bc060ce162794'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:11,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d39575bc060ce162794 2025-12-18 00:01:11,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d39575bc060ce162794 HTTP/1.1" 502 150 2025-12-18 00:01:11,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d39575bc060ce162794'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:11,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d40575bc060ce1627a0 HTTP/1.1" 502 150 2025-12-18 00:01:11,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d40575bc060ce1627a0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:11,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d40575bc060ce1627a0 2025-12-18 00:01:11,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d40575bc060ce1627a0 HTTP/1.1" 502 150 2025-12-18 00:01:11,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d40575bc060ce1627a0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:11,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d35575bc060ce162784 2025-12-18 00:01:11,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d35575bc060ce162784 HTTP/1.1" 502 150 2025-12-18 00:01:11,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d35575bc060ce162784'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:12,305 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d1f575bc060ce162753 2025-12-18 00:01:12,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d1f575bc060ce162753 HTTP/1.1" 502 150 2025-12-18 00:01:12,307 - lava_callback - ERROR - Error processing callback for node 69409d1f575bc060ce162753: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d1f575bc060ce162753 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d1f575bc060ce162753 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d1f575bc060ce162753 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:12,309 - lava_callback - INFO - Starting async job submit for node 69409d3d575bc060ce16279a 2025-12-18 00:01:12,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:12,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d3d575bc060ce16279a HTTP/1.1" 502 150 2025-12-18 00:01:12,315 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d3d575bc060ce16279a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:12,315 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d3d575bc060ce16279a 2025-12-18 00:01:12,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d3d575bc060ce16279a HTTP/1.1" 502 150 2025-12-18 00:01:12,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d3d575bc060ce16279a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:13,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d39575bc060ce162794 2025-12-18 00:01:13,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d39575bc060ce162794 HTTP/1.1" 502 150 2025-12-18 00:01:13,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d39575bc060ce162794'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:13,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d40575bc060ce1627a0 2025-12-18 00:01:13,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d40575bc060ce1627a0 HTTP/1.1" 502 150 2025-12-18 00:01:13,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d40575bc060ce1627a0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:13,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d3b575bc060ce162797 2025-12-18 00:01:13,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d3b575bc060ce162797 HTTP/1.1" 502 150 2025-12-18 00:01:13,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d3b575bc060ce162797'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:14,022 - lava_callback - INFO - Received callback for node 6940a139575bc060ce162e02 INFO: 172.21.0.1:57836 - "POST /node/6940a139575bc060ce162e02 HTTP/1.0" 202 Accepted 2025-12-18 00:01:14,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d3d575bc060ce16279a 2025-12-18 00:01:14,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d3d575bc060ce16279a HTTP/1.1" 502 150 2025-12-18 00:01:14,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d3d575bc060ce16279a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:14,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d37575bc060ce16278d 2025-12-18 00:01:14,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d37575bc060ce16278d HTTP/1.1" 502 150 2025-12-18 00:01:14,423 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d37575bc060ce16278d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:14,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d33575bc060ce16277a 2025-12-18 00:01:14,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d33575bc060ce16277a HTTP/1.1" 502 150 2025-12-18 00:01:14,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d33575bc060ce16277a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57842 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-18 00:01:15,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d25575bc060ce162765 2025-12-18 00:01:15,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d25575bc060ce162765 HTTP/1.1" 502 150 2025-12-18 00:01:15,581 - lava_callback - ERROR - Error processing callback for node 69409d25575bc060ce162765: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d25575bc060ce162765 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d25575bc060ce162765 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d25575bc060ce162765 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:15,583 - lava_callback - INFO - Starting async job submit for node 69409d4f575bc060ce1627cc 2025-12-18 00:01:15,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:15,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4f575bc060ce1627cc HTTP/1.1" 502 150 2025-12-18 00:01:15,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4f575bc060ce1627cc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:15,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4f575bc060ce1627cc 2025-12-18 00:01:15,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4f575bc060ce1627cc HTTP/1.1" 502 150 2025-12-18 00:01:15,615 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4f575bc060ce1627cc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57854 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:01:17,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d31575bc060ce162777 2025-12-18 00:01:17,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d31575bc060ce162777 HTTP/1.1" 502 150 2025-12-18 00:01:17,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d31575bc060ce162777'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:17,615 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4f575bc060ce1627cc 2025-12-18 00:01:17,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4f575bc060ce1627cc HTTP/1.1" 502 150 2025-12-18 00:01:17,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4f575bc060ce1627cc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:17,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d39575bc060ce162794 2025-12-18 00:01:17,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d39575bc060ce162794 HTTP/1.1" 502 150 2025-12-18 00:01:17,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d39575bc060ce162794'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:17,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d40575bc060ce1627a0 2025-12-18 00:01:17,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d40575bc060ce1627a0 HTTP/1.1" 502 150 2025-12-18 00:01:17,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d40575bc060ce1627a0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:18,127 - lava_callback - INFO - Received callback for node 6940a95b1e3c24e95681a22b INFO: 172.21.0.1:57856 - "POST /node/6940a95b1e3c24e95681a22b HTTP/1.0" 202 Accepted 2025-12-18 00:01:18,318 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d3d575bc060ce16279a 2025-12-18 00:01:19,479 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d29575bc060ce16276b 2025-12-18 00:01:19,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d29575bc060ce16276b HTTP/1.1" 502 150 2025-12-18 00:01:19,481 - lava_callback - ERROR - Error processing callback for node 69409d29575bc060ce16276b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d29575bc060ce16276b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d29575bc060ce16276b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d29575bc060ce16276b (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:19,483 - lava_callback - INFO - Starting async job submit for node 69409d40575bc060ce1627a3 2025-12-18 00:01:19,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:19,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d40575bc060ce1627a3 HTTP/1.1" 502 150 2025-12-18 00:01:19,504 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d40575bc060ce1627a3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:19,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d40575bc060ce1627a3 2025-12-18 00:01:19,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d40575bc060ce1627a3 HTTP/1.1" 502 150 2025-12-18 00:01:19,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d40575bc060ce1627a3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:19,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d15575bc060ce162742 HTTP/1.1" 502 150 2025-12-18 00:01:19,536 - lava_callback - ERROR - Error processing callback for node 69409d15575bc060ce162742: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d15575bc060ce162742 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d15575bc060ce162742 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d15575bc060ce162742 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:19,538 - lava_callback - INFO - Starting async job submit for node 69409d42575bc060ce1627a6 2025-12-18 00:01:19,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:19,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d42575bc060ce1627a6 HTTP/1.1" 502 150 2025-12-18 00:01:19,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d42575bc060ce1627a6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:19,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d42575bc060ce1627a6 2025-12-18 00:01:19,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d42575bc060ce1627a6 HTTP/1.1" 502 150 2025-12-18 00:01:19,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d42575bc060ce1627a6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:19,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d35575bc060ce162784 2025-12-18 00:01:19,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d35575bc060ce162784 HTTP/1.1" 502 150 2025-12-18 00:01:19,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d35575bc060ce162784'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:21,172 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d17575bc060ce162745 2025-12-18 00:01:21,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d17575bc060ce162745 HTTP/1.1" 502 150 2025-12-18 00:01:21,174 - lava_callback - ERROR - Error processing callback for node 69409d17575bc060ce162745: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d17575bc060ce162745 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d17575bc060ce162745 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d17575bc060ce162745 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:21,175 - lava_callback - INFO - Starting async job submit for node 69409d43575bc060ce1627a9 2025-12-18 00:01:21,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:21,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d43575bc060ce1627a9 HTTP/1.1" 502 150 2025-12-18 00:01:21,182 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d43575bc060ce1627a9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:21,182 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d43575bc060ce1627a9 2025-12-18 00:01:21,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d43575bc060ce1627a9 HTTP/1.1" 502 150 2025-12-18 00:01:21,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d43575bc060ce1627a9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:21,189 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d2b575bc060ce16276e 2025-12-18 00:01:21,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2b575bc060ce16276e HTTP/1.1" 502 150 2025-12-18 00:01:21,190 - lava_callback - ERROR - Error processing callback for node 69409d2b575bc060ce16276e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d2b575bc060ce16276e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d2b575bc060ce16276e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d2b575bc060ce16276e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:21,192 - lava_callback - INFO - Starting async job submit for node 69409d45575bc060ce1627ac 2025-12-18 00:01:21,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:21,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d45575bc060ce1627ac HTTP/1.1" 502 150 2025-12-18 00:01:21,197 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d45575bc060ce1627ac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:21,197 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d45575bc060ce1627ac 2025-12-18 00:01:21,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d45575bc060ce1627ac HTTP/1.1" 502 150 2025-12-18 00:01:21,198 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d45575bc060ce1627ac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:21,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d40575bc060ce1627a3 2025-12-18 00:01:21,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d40575bc060ce1627a3 HTTP/1.1" 502 150 2025-12-18 00:01:21,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d40575bc060ce1627a3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:21,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d42575bc060ce1627a6 2025-12-18 00:01:21,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d42575bc060ce1627a6 HTTP/1.1" 502 150 2025-12-18 00:01:21,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d42575bc060ce1627a6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:21,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4f575bc060ce1627cc 2025-12-18 00:01:21,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4f575bc060ce1627cc HTTP/1.1" 502 150 2025-12-18 00:01:21,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4f575bc060ce1627cc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:21,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d3b575bc060ce162797 2025-12-18 00:01:21,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d3b575bc060ce162797 HTTP/1.1" 502 150 2025-12-18 00:01:21,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d3b575bc060ce162797'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:22,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d37575bc060ce16278d 2025-12-18 00:01:22,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d37575bc060ce16278d HTTP/1.1" 502 150 2025-12-18 00:01:22,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d37575bc060ce16278d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:22,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d2d575bc060ce162771 2025-12-18 00:01:22,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2d575bc060ce162771 HTTP/1.1" 502 150 2025-12-18 00:01:22,496 - lava_callback - ERROR - Error processing callback for node 69409d2d575bc060ce162771: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d2d575bc060ce162771 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d2d575bc060ce162771 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d2d575bc060ce162771 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:22,498 - lava_callback - INFO - Starting async job submit for node 69409d47575bc060ce1627b4 2025-12-18 00:01:22,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:22,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d47575bc060ce1627b4 HTTP/1.1" 502 150 2025-12-18 00:01:22,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d47575bc060ce1627b4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:22,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d47575bc060ce1627b4 2025-12-18 00:01:22,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d47575bc060ce1627b4 HTTP/1.1" 502 150 2025-12-18 00:01:22,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d47575bc060ce1627b4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:22,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0a575bc060ce16272a HTTP/1.1" 502 150 2025-12-18 00:01:22,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d0a575bc060ce16272a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:23,183 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d43575bc060ce1627a9 2025-12-18 00:01:23,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d43575bc060ce1627a9 HTTP/1.1" 502 150 2025-12-18 00:01:23,189 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d43575bc060ce1627a9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:23,198 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d45575bc060ce1627ac 2025-12-18 00:01:23,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d45575bc060ce1627ac HTTP/1.1" 502 150 2025-12-18 00:01:23,199 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d45575bc060ce1627ac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:24,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d47575bc060ce1627b4 2025-12-18 00:01:24,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d47575bc060ce1627b4 HTTP/1.1" 502 150 2025-12-18 00:01:24,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d47575bc060ce1627b4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:25,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d40575bc060ce1627a3 2025-12-18 00:01:25,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d40575bc060ce1627a3 HTTP/1.1" 502 150 2025-12-18 00:01:25,510 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d40575bc060ce1627a3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:25,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d42575bc060ce1627a6 2025-12-18 00:01:25,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d42575bc060ce1627a6 HTTP/1.1" 502 150 2025-12-18 00:01:25,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d42575bc060ce1627a6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:25,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d39575bc060ce162794 2025-12-18 00:01:25,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d39575bc060ce162794 HTTP/1.1" 502 150 2025-12-18 00:01:25,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d39575bc060ce162794'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:25,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d40575bc060ce1627a0 2025-12-18 00:01:25,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d40575bc060ce1627a0 HTTP/1.1" 502 150 2025-12-18 00:01:25,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d40575bc060ce1627a0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38994 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:01:26,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d2f575bc060ce162774 2025-12-18 00:01:26,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d2f575bc060ce162774 HTTP/1.1" 502 150 2025-12-18 00:01:26,592 - lava_callback - ERROR - Error processing callback for node 69409d2f575bc060ce162774: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d2f575bc060ce162774 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d2f575bc060ce162774 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d2f575bc060ce162774 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:26,593 - lava_callback - INFO - Starting async job submit for node 69409d4b575bc060ce1627c4 2025-12-18 00:01:26,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:26,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4b575bc060ce1627c4 HTTP/1.1" 502 150 2025-12-18 00:01:26,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4b575bc060ce1627c4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:26,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4b575bc060ce1627c4 2025-12-18 00:01:26,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4b575bc060ce1627c4 HTTP/1.1" 502 150 2025-12-18 00:01:26,601 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4b575bc060ce1627c4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:27,189 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d43575bc060ce1627a9 2025-12-18 00:01:27,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d43575bc060ce1627a9 HTTP/1.1" 502 150 2025-12-18 00:01:27,191 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d43575bc060ce1627a9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:27,199 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d45575bc060ce1627ac 2025-12-18 00:01:27,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d45575bc060ce1627ac HTTP/1.1" 502 150 2025-12-18 00:01:27,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d45575bc060ce1627ac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:28,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d47575bc060ce1627b4 2025-12-18 00:01:28,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d47575bc060ce1627b4 HTTP/1.1" 502 150 2025-12-18 00:01:28,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d47575bc060ce1627b4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:28,601 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4b575bc060ce1627c4 2025-12-18 00:01:28,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4b575bc060ce1627c4 HTTP/1.1" 502 150 2025-12-18 00:01:28,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4b575bc060ce1627c4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:29,159 - lava_callback - INFO - Received callback for node 6940a144575bc060ce162e12 INFO: 172.21.0.1:39010 - "POST /node/6940a144575bc060ce162e12 HTTP/1.0" 202 Accepted 2025-12-18 00:01:29,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4f575bc060ce1627cc 2025-12-18 00:01:29,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4f575bc060ce1627cc HTTP/1.1" 502 150 2025-12-18 00:01:29,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4f575bc060ce1627cc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:29,801 - lava_callback - INFO - Received callback for node 6940a147575bc060ce162e19 INFO: 172.21.0.1:52102 - "POST /node/6940a147575bc060ce162e19 HTTP/1.0" 202 Accepted 2025-12-18 00:01:30,554 - lava_callback - INFO - Received callback for node 6940a149575bc060ce162e1c INFO: 172.21.0.1:52106 - "POST /node/6940a149575bc060ce162e1c HTTP/1.0" 202 Accepted 2025-12-18 00:01:30,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d33575bc060ce16277a 2025-12-18 00:01:32,602 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4b575bc060ce1627c4 2025-12-18 00:01:32,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4b575bc060ce1627c4 HTTP/1.1" 502 150 2025-12-18 00:01:32,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4b575bc060ce1627c4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:33,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d31575bc060ce162777 2025-12-18 00:01:33,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d31575bc060ce162777 HTTP/1.1" 502 150 2025-12-18 00:01:33,468 - lava_callback - ERROR - Error processing callback for node 69409d31575bc060ce162777: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d31575bc060ce162777 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d31575bc060ce162777 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d31575bc060ce162777 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:33,470 - lava_callback - INFO - Starting async job submit for node 69409d4d575bc060ce1627c9 2025-12-18 00:01:33,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:33,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4d575bc060ce1627c9 HTTP/1.1" 502 150 2025-12-18 00:01:33,501 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4d575bc060ce1627c9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:33,501 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4d575bc060ce1627c9 2025-12-18 00:01:33,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4d575bc060ce1627c9 HTTP/1.1" 502 150 2025-12-18 00:01:33,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4d575bc060ce1627c9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:33,510 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d40575bc060ce1627a3 2025-12-18 00:01:33,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d40575bc060ce1627a3 HTTP/1.1" 502 150 2025-12-18 00:01:33,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d40575bc060ce1627a3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:33,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d42575bc060ce1627a6 2025-12-18 00:01:33,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d42575bc060ce1627a6 HTTP/1.1" 502 150 2025-12-18 00:01:33,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d42575bc060ce1627a6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:33,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d3d575bc060ce16279a HTTP/1.1" 502 150 2025-12-18 00:01:33,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d3d575bc060ce16279a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:33,747 - lava_callback - INFO - Received callback for node 6940a14c575bc060ce162e21 INFO: 172.21.0.1:52120 - "POST /node/6940a14c575bc060ce162e21 HTTP/1.0" 202 Accepted 2025-12-18 00:01:34,439 - lava_callback - INFO - Received callback for node 6940a150575bc060ce162e36 INFO: 172.21.0.1:52134 - "POST /node/6940a150575bc060ce162e36 HTTP/1.0" 202 Accepted 2025-12-18 00:01:35,191 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d43575bc060ce1627a9 2025-12-18 00:01:35,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d43575bc060ce1627a9 HTTP/1.1" 502 150 2025-12-18 00:01:35,196 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d43575bc060ce1627a9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:35,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d45575bc060ce1627ac 2025-12-18 00:01:35,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d45575bc060ce1627ac HTTP/1.1" 502 150 2025-12-18 00:01:35,201 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d45575bc060ce1627ac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:35,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4d575bc060ce1627c9 2025-12-18 00:01:35,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4d575bc060ce1627c9 HTTP/1.1" 502 150 2025-12-18 00:01:35,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4d575bc060ce1627c9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:35,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d35575bc060ce162784 2025-12-18 00:01:35,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d35575bc060ce162784 HTTP/1.1" 502 150 2025-12-18 00:01:35,953 - lava_callback - ERROR - Error processing callback for node 69409d35575bc060ce162784: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d35575bc060ce162784 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d35575bc060ce162784 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d35575bc060ce162784 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:35,957 - lava_callback - INFO - Starting async job submit for node 69409d4a575bc060ce1627bf 2025-12-18 00:01:35,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:35,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4a575bc060ce1627bf HTTP/1.1" 502 150 2025-12-18 00:01:35,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4a575bc060ce1627bf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:35,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4a575bc060ce1627bf 2025-12-18 00:01:35,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4a575bc060ce1627bf HTTP/1.1" 502 150 2025-12-18 00:01:35,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4a575bc060ce1627bf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52146 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:01:36,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d47575bc060ce1627b4 2025-12-18 00:01:36,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d47575bc060ce1627b4 HTTP/1.1" 502 150 2025-12-18 00:01:36,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d47575bc060ce1627b4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:37,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4a575bc060ce1627bf 2025-12-18 00:01:37,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4a575bc060ce1627bf HTTP/1.1" 502 150 2025-12-18 00:01:37,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4a575bc060ce1627bf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:37,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d3b575bc060ce162797 2025-12-18 00:01:37,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d3b575bc060ce162797 HTTP/1.1" 502 150 2025-12-18 00:01:37,996 - lava_callback - ERROR - Error processing callback for node 69409d3b575bc060ce162797: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d3b575bc060ce162797 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d3b575bc060ce162797 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d3b575bc060ce162797 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:37,999 - lava_callback - INFO - Starting async job submit for node 6940a31c575bc060ce16314c 2025-12-18 00:01:38,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:38,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a31c575bc060ce16314c HTTP/1.1" 502 150 2025-12-18 00:01:38,065 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a31c575bc060ce16314c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:38,066 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a31c575bc060ce16314c 2025-12-18 00:01:38,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a31c575bc060ce16314c HTTP/1.1" 502 150 2025-12-18 00:01:38,067 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a31c575bc060ce16314c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:38,425 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d37575bc060ce16278d 2025-12-18 00:01:38,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d37575bc060ce16278d HTTP/1.1" 502 150 2025-12-18 00:01:38,430 - lava_callback - ERROR - Error processing callback for node 69409d37575bc060ce16278d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d37575bc060ce16278d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d37575bc060ce16278d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d37575bc060ce16278d (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:38,433 - lava_callback - INFO - Starting async job submit for node 69409d64575bc060ce1627f8 2025-12-18 00:01:38,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:38,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d64575bc060ce1627f8 HTTP/1.1" 502 150 2025-12-18 00:01:38,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d64575bc060ce1627f8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:38,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d64575bc060ce1627f8 2025-12-18 00:01:38,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d64575bc060ce1627f8 HTTP/1.1" 502 150 2025-12-18 00:01:38,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d64575bc060ce1627f8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:38,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d0a575bc060ce16272a 2025-12-18 00:01:39,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4d575bc060ce1627c9 2025-12-18 00:01:39,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4d575bc060ce1627c9 HTTP/1.1" 502 150 2025-12-18 00:01:39,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4d575bc060ce1627c9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:40,067 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a31c575bc060ce16314c 2025-12-18 00:01:40,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a31c575bc060ce16314c HTTP/1.1" 502 150 2025-12-18 00:01:40,068 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a31c575bc060ce16314c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:40,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d64575bc060ce1627f8 2025-12-18 00:01:40,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d64575bc060ce1627f8 HTTP/1.1" 502 150 2025-12-18 00:01:40,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d64575bc060ce1627f8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:40,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4b575bc060ce1627c4 2025-12-18 00:01:40,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4b575bc060ce1627c4 HTTP/1.1" 502 150 2025-12-18 00:01:40,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4b575bc060ce1627c4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:41,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d3d575bc060ce16279a 2025-12-18 00:01:41,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d3d575bc060ce16279a HTTP/1.1" 502 150 2025-12-18 00:01:41,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d3d575bc060ce16279a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:41,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d39575bc060ce162794 2025-12-18 00:01:41,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d39575bc060ce162794 HTTP/1.1" 502 150 2025-12-18 00:01:41,741 - lava_callback - ERROR - Error processing callback for node 69409d39575bc060ce162794: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d39575bc060ce162794 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d39575bc060ce162794 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d39575bc060ce162794 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:41,743 - lava_callback - INFO - Starting async job submit for node 69409d51575bc060ce1627cf 2025-12-18 00:01:41,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:41,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d40575bc060ce1627a0 2025-12-18 00:01:41,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d40575bc060ce1627a0 HTTP/1.1" 502 150 2025-12-18 00:01:41,745 - lava_callback - ERROR - Error processing callback for node 69409d40575bc060ce1627a0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d40575bc060ce1627a0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d40575bc060ce1627a0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d40575bc060ce1627a0 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:41,747 - lava_callback - INFO - Starting async job submit for node 69409d53575bc060ce1627d2 2025-12-18 00:01:41,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:41,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d51575bc060ce1627cf HTTP/1.1" 502 150 2025-12-18 00:01:41,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d51575bc060ce1627cf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:41,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d51575bc060ce1627cf 2025-12-18 00:01:41,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d51575bc060ce1627cf HTTP/1.1" 502 150 2025-12-18 00:01:41,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d51575bc060ce1627cf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:41,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d53575bc060ce1627d2 HTTP/1.1" 502 150 2025-12-18 00:01:41,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d53575bc060ce1627d2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:41,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d53575bc060ce1627d2 2025-12-18 00:01:41,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d53575bc060ce1627d2 HTTP/1.1" 502 150 2025-12-18 00:01:41,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d53575bc060ce1627d2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:41,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4a575bc060ce1627bf 2025-12-18 00:01:41,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4a575bc060ce1627bf HTTP/1.1" 502 150 2025-12-18 00:01:41,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4a575bc060ce1627bf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:43,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d51575bc060ce1627cf 2025-12-18 00:01:43,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d51575bc060ce1627cf HTTP/1.1" 502 150 2025-12-18 00:01:43,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d51575bc060ce1627cf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:43,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d53575bc060ce1627d2 2025-12-18 00:01:43,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d53575bc060ce1627d2 HTTP/1.1" 502 150 2025-12-18 00:01:43,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d53575bc060ce1627d2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:44,068 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a31c575bc060ce16314c 2025-12-18 00:01:44,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d64575bc060ce1627f8 2025-12-18 00:01:45,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4f575bc060ce1627cc 2025-12-18 00:01:45,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4f575bc060ce1627cc HTTP/1.1" 502 150 2025-12-18 00:01:45,621 - lava_callback - ERROR - Error processing callback for node 69409d4f575bc060ce1627cc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d4f575bc060ce1627cc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d4f575bc060ce1627cc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d4f575bc060ce1627cc (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:45,623 - lava_callback - INFO - Starting async job submit for node 69409d55575bc060ce1627d5 2025-12-18 00:01:45,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:45,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d55575bc060ce1627d5 HTTP/1.1" 502 150 2025-12-18 00:01:45,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d55575bc060ce1627d5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:45,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d55575bc060ce1627d5 2025-12-18 00:01:45,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d55575bc060ce1627d5 HTTP/1.1" 502 150 2025-12-18 00:01:45,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d55575bc060ce1627d5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:46,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d33575bc060ce16277a HTTP/1.1" 502 150 2025-12-18 00:01:46,160 - lava_callback - ERROR - Error processing callback for node 69409d33575bc060ce16277a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d33575bc060ce16277a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d33575bc060ce16277a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d33575bc060ce16277a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:46,162 - lava_callback - INFO - Starting async job submit for node 69409d57575bc060ce1627d8 2025-12-18 00:01:46,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:46,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d57575bc060ce1627d8 HTTP/1.1" 502 150 2025-12-18 00:01:46,170 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d57575bc060ce1627d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:46,170 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d57575bc060ce1627d8 2025-12-18 00:01:46,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d57575bc060ce1627d8 HTTP/1.1" 502 150 2025-12-18 00:01:46,171 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d57575bc060ce1627d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59744 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:01:47,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4d575bc060ce1627c9 2025-12-18 00:01:47,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4d575bc060ce1627c9 HTTP/1.1" 502 150 2025-12-18 00:01:47,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4d575bc060ce1627c9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:47,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d55575bc060ce1627d5 2025-12-18 00:01:47,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d55575bc060ce1627d5 HTTP/1.1" 502 150 2025-12-18 00:01:47,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d55575bc060ce1627d5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:47,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d51575bc060ce1627cf 2025-12-18 00:01:47,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d53575bc060ce1627d2 2025-12-18 00:01:47,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d51575bc060ce1627cf HTTP/1.1" 502 150 2025-12-18 00:01:47,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d51575bc060ce1627cf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:47,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d53575bc060ce1627d2 HTTP/1.1" 502 150 2025-12-18 00:01:47,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d53575bc060ce1627d2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:48,171 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d57575bc060ce1627d8 2025-12-18 00:01:48,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d57575bc060ce1627d8 HTTP/1.1" 502 150 2025-12-18 00:01:48,172 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d57575bc060ce1627d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:49,148 - lava_callback - INFO - Received callback for node 6940a158575bc060ce162e3b INFO: 172.21.0.1:59750 - "POST /node/6940a158575bc060ce162e3b HTTP/1.0" 202 Accepted 2025-12-18 00:01:49,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d40575bc060ce1627a3 2025-12-18 00:01:49,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d40575bc060ce1627a3 HTTP/1.1" 502 150 2025-12-18 00:01:49,513 - lava_callback - ERROR - Error processing callback for node 69409d40575bc060ce1627a3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d40575bc060ce1627a3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d40575bc060ce1627a3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d40575bc060ce1627a3 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:49,515 - lava_callback - INFO - Starting async job submit for node 69409d59575bc060ce1627db 2025-12-18 00:01:49,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:49,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d59575bc060ce1627db HTTP/1.1" 502 150 2025-12-18 00:01:49,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d59575bc060ce1627db'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:49,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d59575bc060ce1627db 2025-12-18 00:01:49,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d59575bc060ce1627db HTTP/1.1" 502 150 2025-12-18 00:01:49,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d59575bc060ce1627db'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:49,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d42575bc060ce1627a6 2025-12-18 00:01:49,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d42575bc060ce1627a6 HTTP/1.1" 502 150 2025-12-18 00:01:49,549 - lava_callback - ERROR - Error processing callback for node 69409d42575bc060ce1627a6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d42575bc060ce1627a6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d42575bc060ce1627a6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d42575bc060ce1627a6 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:49,551 - lava_callback - INFO - Starting async job submit for node 69409d5b575bc060ce1627e0 2025-12-18 00:01:49,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:49,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d5b575bc060ce1627e0 HTTP/1.1" 502 150 2025-12-18 00:01:49,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d5b575bc060ce1627e0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:49,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d5b575bc060ce1627e0 2025-12-18 00:01:49,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d5b575bc060ce1627e0 HTTP/1.1" 502 150 2025-12-18 00:01:49,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d5b575bc060ce1627e0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:49,798 - lava_callback - INFO - Received callback for node 6940a15a575bc060ce162e45 INFO: 172.21.0.1:34668 - "POST /node/6940a15a575bc060ce162e45 HTTP/1.0" 202 Accepted 2025-12-18 00:01:49,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4a575bc060ce1627bf 2025-12-18 00:01:49,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4a575bc060ce1627bf HTTP/1.1" 502 150 2025-12-18 00:01:49,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d4a575bc060ce1627bf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:50,244 - lava_callback - INFO - Received callback for node 69414297459ff5fc64d2ca36 INFO: 172.21.0.1:34672 - "POST /node/69414297459ff5fc64d2ca36 HTTP/1.0" 202 Accepted 2025-12-18 00:01:50,461 - lava_callback - INFO - Received callback for node 6940a15c575bc060ce162e4d INFO: 172.21.0.1:34674 - "POST /node/6940a15c575bc060ce162e4d HTTP/1.0" 202 Accepted 2025-12-18 00:01:51,124 - lava_callback - INFO - Received callback for node 6940a15e575bc060ce162e50 INFO: 172.21.0.1:34686 - "POST /node/6940a15e575bc060ce162e50 HTTP/1.0" 202 Accepted 2025-12-18 00:01:51,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d43575bc060ce1627a9 2025-12-18 00:01:51,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d43575bc060ce1627a9 HTTP/1.1" 502 150 2025-12-18 00:01:51,197 - lava_callback - ERROR - Error processing callback for node 69409d43575bc060ce1627a9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d43575bc060ce1627a9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d43575bc060ce1627a9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d43575bc060ce1627a9 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:51,199 - lava_callback - INFO - Starting async job submit for node 69409d5d575bc060ce1627e9 2025-12-18 00:01:51,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:51,201 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d45575bc060ce1627ac 2025-12-18 00:01:51,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d45575bc060ce1627ac HTTP/1.1" 502 150 2025-12-18 00:01:51,202 - lava_callback - ERROR - Error processing callback for node 69409d45575bc060ce1627ac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d45575bc060ce1627ac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d45575bc060ce1627ac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d45575bc060ce1627ac (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:51,204 - lava_callback - INFO - Starting async job submit for node 69409d60575bc060ce1627f2 2025-12-18 00:01:51,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:51,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d60575bc060ce1627f2 HTTP/1.1" 502 150 2025-12-18 00:01:51,213 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d60575bc060ce1627f2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:51,213 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d60575bc060ce1627f2 2025-12-18 00:01:51,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d60575bc060ce1627f2 HTTP/1.1" 502 150 2025-12-18 00:01:51,214 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d60575bc060ce1627f2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:51,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d5d575bc060ce1627e9 HTTP/1.1" 502 150 2025-12-18 00:01:51,229 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d5d575bc060ce1627e9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:51,229 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d5d575bc060ce1627e9 2025-12-18 00:01:51,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d5d575bc060ce1627e9 HTTP/1.1" 502 150 2025-12-18 00:01:51,230 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d5d575bc060ce1627e9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:51,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d59575bc060ce1627db 2025-12-18 00:01:51,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d59575bc060ce1627db HTTP/1.1" 502 150 2025-12-18 00:01:51,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d59575bc060ce1627db'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:51,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d5b575bc060ce1627e0 2025-12-18 00:01:51,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d5b575bc060ce1627e0 HTTP/1.1" 502 150 2025-12-18 00:01:51,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d5b575bc060ce1627e0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:51,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d55575bc060ce1627d5 2025-12-18 00:01:51,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d55575bc060ce1627d5 HTTP/1.1" 502 150 2025-12-18 00:01:51,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d55575bc060ce1627d5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:51,758 - lava_callback - INFO - Received callback for node 6940a160575bc060ce162e53 INFO: 172.21.0.1:34696 - "POST /node/6940a160575bc060ce162e53 HTTP/1.0" 202 Accepted 2025-12-18 00:01:52,173 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d57575bc060ce1627d8 2025-12-18 00:01:52,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d57575bc060ce1627d8 HTTP/1.1" 502 150 2025-12-18 00:01:52,174 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d57575bc060ce1627d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:52,402 - lava_callback - INFO - Received callback for node 6940a162575bc060ce162e56 INFO: 172.21.0.1:34708 - "POST /node/6940a162575bc060ce162e56 HTTP/1.0" 202 Accepted 2025-12-18 00:01:52,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d47575bc060ce1627b4 2025-12-18 00:01:52,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d47575bc060ce1627b4 HTTP/1.1" 502 150 2025-12-18 00:01:52,538 - lava_callback - ERROR - Error processing callback for node 69409d47575bc060ce1627b4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d47575bc060ce1627b4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d47575bc060ce1627b4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d47575bc060ce1627b4 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:52,540 - lava_callback - INFO - Starting async job submit for node 69409d62575bc060ce1627f5 2025-12-18 00:01:52,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:52,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d62575bc060ce1627f5 HTTP/1.1" 502 150 2025-12-18 00:01:52,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d62575bc060ce1627f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:52,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d62575bc060ce1627f5 2025-12-18 00:01:52,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d62575bc060ce1627f5 HTTP/1.1" 502 150 2025-12-18 00:01:52,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d62575bc060ce1627f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:53,047 - lava_callback - INFO - Received callback for node 6940a164575bc060ce162e59 INFO: 172.21.0.1:34714 - "POST /node/6940a164575bc060ce162e59 HTTP/1.0" 202 Accepted 2025-12-18 00:01:53,214 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d60575bc060ce1627f2 2025-12-18 00:01:53,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d60575bc060ce1627f2 HTTP/1.1" 502 150 2025-12-18 00:01:53,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d60575bc060ce1627f2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:53,230 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d5d575bc060ce1627e9 2025-12-18 00:01:53,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d5d575bc060ce1627e9 HTTP/1.1" 502 150 2025-12-18 00:01:53,231 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d5d575bc060ce1627e9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:54,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d62575bc060ce1627f5 2025-12-18 00:01:54,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d62575bc060ce1627f5 HTTP/1.1" 502 150 2025-12-18 00:01:54,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d62575bc060ce1627f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:55,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d59575bc060ce1627db 2025-12-18 00:01:55,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d5b575bc060ce1627e0 2025-12-18 00:01:55,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d5b575bc060ce1627e0 HTTP/1.1" 502 150 2025-12-18 00:01:55,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d5b575bc060ce1627e0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:55,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d51575bc060ce1627cf 2025-12-18 00:01:55,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d53575bc060ce1627d2 2025-12-18 00:01:55,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d51575bc060ce1627cf HTTP/1.1" 502 150 2025-12-18 00:01:55,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d51575bc060ce1627cf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:55,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d53575bc060ce1627d2 HTTP/1.1" 502 150 2025-12-18 00:01:55,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d53575bc060ce1627d2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34720 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:01:56,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4b575bc060ce1627c4 2025-12-18 00:01:56,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4b575bc060ce1627c4 HTTP/1.1" 502 150 2025-12-18 00:01:56,607 - lava_callback - ERROR - Error processing callback for node 69409d4b575bc060ce1627c4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d4b575bc060ce1627c4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d4b575bc060ce1627c4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d4b575bc060ce1627c4 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:56,608 - lava_callback - INFO - Starting async job submit for node 69409d66575bc060ce1627fb 2025-12-18 00:01:56,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:56,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d66575bc060ce1627fb HTTP/1.1" 502 150 2025-12-18 00:01:56,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d66575bc060ce1627fb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:56,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d66575bc060ce1627fb 2025-12-18 00:01:56,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d66575bc060ce1627fb HTTP/1.1" 502 150 2025-12-18 00:01:56,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d66575bc060ce1627fb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:57,216 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d60575bc060ce1627f2 2025-12-18 00:01:57,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d60575bc060ce1627f2 HTTP/1.1" 502 150 2025-12-18 00:01:57,217 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d60575bc060ce1627f2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:57,231 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d5d575bc060ce1627e9 2025-12-18 00:01:57,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d5d575bc060ce1627e9 HTTP/1.1" 502 150 2025-12-18 00:01:57,232 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d5d575bc060ce1627e9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:57,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d3d575bc060ce16279a 2025-12-18 00:01:57,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d3d575bc060ce16279a HTTP/1.1" 502 150 2025-12-18 00:01:57,619 - lava_callback - ERROR - Error processing callback for node 69409d3d575bc060ce16279a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d3d575bc060ce16279a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d3d575bc060ce16279a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d3d575bc060ce16279a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:01:57,621 - lava_callback - INFO - Starting async job submit for node 69409d6a575bc060ce162801 2025-12-18 00:01:57,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:01:57,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6a575bc060ce162801 HTTP/1.1" 502 150 2025-12-18 00:01:57,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6a575bc060ce162801'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:57,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6a575bc060ce162801 2025-12-18 00:01:57,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6a575bc060ce162801 HTTP/1.1" 502 150 2025-12-18 00:01:57,629 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6a575bc060ce162801'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:58,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d62575bc060ce1627f5 2025-12-18 00:01:58,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d62575bc060ce1627f5 HTTP/1.1" 502 150 2025-12-18 00:01:58,550 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d62575bc060ce1627f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:58,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d66575bc060ce1627fb 2025-12-18 00:01:58,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d66575bc060ce1627fb HTTP/1.1" 502 150 2025-12-18 00:01:58,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d66575bc060ce1627fb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:59,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a31c575bc060ce16314c HTTP/1.1" 502 150 2025-12-18 00:01:59,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a31c575bc060ce16314c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:59,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6a575bc060ce162801 2025-12-18 00:01:59,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6a575bc060ce162801 HTTP/1.1" 502 150 2025-12-18 00:01:59,630 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6a575bc060ce162801'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:59,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d55575bc060ce1627d5 2025-12-18 00:01:59,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d55575bc060ce1627d5 HTTP/1.1" 502 150 2025-12-18 00:01:59,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d55575bc060ce1627d5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:01:59,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d64575bc060ce1627f8 HTTP/1.1" 502 150 2025-12-18 00:01:59,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d64575bc060ce1627f8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:00,174 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d57575bc060ce1627d8 2025-12-18 00:02:00,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d57575bc060ce1627d8 HTTP/1.1" 502 150 2025-12-18 00:02:00,176 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d57575bc060ce1627d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:02,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d66575bc060ce1627fb 2025-12-18 00:02:02,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d66575bc060ce1627fb HTTP/1.1" 502 150 2025-12-18 00:02:02,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d66575bc060ce1627fb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:03,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4d575bc060ce1627c9 2025-12-18 00:02:03,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4d575bc060ce1627c9 HTTP/1.1" 502 150 2025-12-18 00:02:03,508 - lava_callback - ERROR - Error processing callback for node 69409d4d575bc060ce1627c9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d4d575bc060ce1627c9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d4d575bc060ce1627c9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d4d575bc060ce1627c9 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:03,509 - lava_callback - INFO - Starting async job submit for node 69409d6b575bc060ce162804 2025-12-18 00:02:03,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:03,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6b575bc060ce162804 HTTP/1.1" 502 150 2025-12-18 00:02:03,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6b575bc060ce162804'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:03,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6b575bc060ce162804 2025-12-18 00:02:03,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6b575bc060ce162804 HTTP/1.1" 502 150 2025-12-18 00:02:03,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6b575bc060ce162804'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:03,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d5b575bc060ce1627e0 2025-12-18 00:02:03,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d5b575bc060ce1627e0 HTTP/1.1" 502 150 2025-12-18 00:02:03,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d5b575bc060ce1627e0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:03,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6a575bc060ce162801 2025-12-18 00:02:03,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6a575bc060ce162801 HTTP/1.1" 502 150 2025-12-18 00:02:03,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6a575bc060ce162801'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54206 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:02:05,217 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d60575bc060ce1627f2 2025-12-18 00:02:05,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d60575bc060ce1627f2 HTTP/1.1" 502 150 2025-12-18 00:02:05,218 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d60575bc060ce1627f2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:05,232 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d5d575bc060ce1627e9 2025-12-18 00:02:05,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d5d575bc060ce1627e9 HTTP/1.1" 502 150 2025-12-18 00:02:05,233 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d5d575bc060ce1627e9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:05,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6b575bc060ce162804 2025-12-18 00:02:05,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6b575bc060ce162804 HTTP/1.1" 502 150 2025-12-18 00:02:05,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6b575bc060ce162804'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:05,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d4a575bc060ce1627bf 2025-12-18 00:02:05,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d4a575bc060ce1627bf HTTP/1.1" 502 150 2025-12-18 00:02:05,978 - lava_callback - ERROR - Error processing callback for node 69409d4a575bc060ce1627bf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d4a575bc060ce1627bf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d4a575bc060ce1627bf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d4a575bc060ce1627bf (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:05,980 - lava_callback - INFO - Starting async job submit for node 69409d6d575bc060ce162807 2025-12-18 00:02:05,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:05,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6d575bc060ce162807 HTTP/1.1" 502 150 2025-12-18 00:02:05,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6d575bc060ce162807'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:05,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6d575bc060ce162807 2025-12-18 00:02:05,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6d575bc060ce162807 HTTP/1.1" 502 150 2025-12-18 00:02:05,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6d575bc060ce162807'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54208 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:02:06,551 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d62575bc060ce1627f5 2025-12-18 00:02:06,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d62575bc060ce1627f5 HTTP/1.1" 502 150 2025-12-18 00:02:06,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d62575bc060ce1627f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:07,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a31c575bc060ce16314c 2025-12-18 00:02:07,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a31c575bc060ce16314c HTTP/1.1" 502 150 2025-12-18 00:02:07,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a31c575bc060ce16314c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:07,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d64575bc060ce1627f8 2025-12-18 00:02:07,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d64575bc060ce1627f8 HTTP/1.1" 502 150 2025-12-18 00:02:07,730 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d64575bc060ce1627f8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:07,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6d575bc060ce162807 2025-12-18 00:02:07,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6d575bc060ce162807 HTTP/1.1" 502 150 2025-12-18 00:02:07,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6d575bc060ce162807'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:09,161 - lava_callback - INFO - Received callback for node 6940a16c575bc060ce162e5e INFO: 172.21.0.1:54210 - "POST /node/6940a16c575bc060ce162e5e HTTP/1.0" 202 Accepted 2025-12-18 00:02:09,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d0a575bc060ce16272a HTTP/1.1" 502 150 2025-12-18 00:02:09,456 - lava_callback - ERROR - Error processing callback for node 69409d0a575bc060ce16272a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d0a575bc060ce16272a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d0a575bc060ce16272a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d0a575bc060ce16272a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:09,458 - lava_callback - INFO - Starting async job submit for node 69409d6f575bc060ce16280c 2025-12-18 00:02:09,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:09,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6f575bc060ce16280c HTTP/1.1" 502 150 2025-12-18 00:02:09,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6f575bc060ce16280c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:09,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6f575bc060ce16280c 2025-12-18 00:02:09,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6f575bc060ce16280c HTTP/1.1" 502 150 2025-12-18 00:02:09,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6f575bc060ce16280c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:09,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6b575bc060ce162804 2025-12-18 00:02:09,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6b575bc060ce162804 HTTP/1.1" 502 150 2025-12-18 00:02:09,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6b575bc060ce162804'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:09,808 - lava_callback - INFO - Received callback for node 6940a16e575bc060ce162e69 INFO: 172.21.0.1:35986 - "POST /node/6940a16e575bc060ce162e69 HTTP/1.0" 202 Accepted 2025-12-18 00:02:09,855 - lava_callback - INFO - Received callback for node 6940a17a575bc060ce162e90 INFO: 172.21.0.1:35990 - "POST /node/6940a17a575bc060ce162e90 HTTP/1.0" 202 Accepted 2025-12-18 00:02:10,465 - lava_callback - INFO - Received callback for node 6940a170575bc060ce162e70 INFO: 172.21.0.1:35996 - "POST /node/6940a170575bc060ce162e70 HTTP/1.0" 202 Accepted 2025-12-18 00:02:10,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d66575bc060ce1627fb 2025-12-18 00:02:10,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d66575bc060ce1627fb HTTP/1.1" 502 150 2025-12-18 00:02:10,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d66575bc060ce1627fb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:10,654 - lava_callback - INFO - Received callback for node 6940a17c575bc060ce162e93 INFO: 172.21.0.1:36002 - "POST /node/6940a17c575bc060ce162e93 HTTP/1.0" 202 Accepted 2025-12-18 00:02:10,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d59575bc060ce1627db HTTP/1.1" 502 150 2025-12-18 00:02:10,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d59575bc060ce1627db'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:11,110 - lava_callback - INFO - Received callback for node 6940a172575bc060ce162e77 INFO: 172.21.0.1:36018 - "POST /node/6940a172575bc060ce162e77 HTTP/1.0" 202 Accepted 2025-12-18 00:02:11,462 - lava_callback - INFO - Received callback for node 6940a17d575bc060ce162e96 INFO: 172.21.0.1:36022 - "POST /node/6940a17d575bc060ce162e96 HTTP/1.0" 202 Accepted 2025-12-18 00:02:11,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6f575bc060ce16280c 2025-12-18 00:02:11,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6f575bc060ce16280c HTTP/1.1" 502 150 2025-12-18 00:02:11,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6f575bc060ce16280c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:11,633 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6a575bc060ce162801 2025-12-18 00:02:11,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6a575bc060ce162801 HTTP/1.1" 502 150 2025-12-18 00:02:11,634 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6a575bc060ce162801'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:11,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d51575bc060ce1627cf 2025-12-18 00:02:11,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d53575bc060ce1627d2 2025-12-18 00:02:11,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d53575bc060ce1627d2 HTTP/1.1" 502 150 2025-12-18 00:02:11,767 - lava_callback - ERROR - Error processing callback for node 69409d53575bc060ce1627d2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d53575bc060ce1627d2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d53575bc060ce1627d2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d53575bc060ce1627d2 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:11,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d51575bc060ce1627cf HTTP/1.1" 502 150 2025-12-18 00:02:11,769 - lava_callback - ERROR - Error processing callback for node 69409d51575bc060ce1627cf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d51575bc060ce1627cf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d51575bc060ce1627cf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d51575bc060ce1627cf (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:11,769 - lava_callback - INFO - Starting async job submit for node 69409d71575bc060ce162811 2025-12-18 00:02:11,772 - lava_callback - INFO - Starting async job submit for node 69409d73575bc060ce162814 2025-12-18 00:02:11,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:11,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:11,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d71575bc060ce162811 HTTP/1.1" 502 150 2025-12-18 00:02:11,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d71575bc060ce162811'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:11,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d71575bc060ce162811 2025-12-18 00:02:11,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d73575bc060ce162814 HTTP/1.1" 502 150 2025-12-18 00:02:11,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d73575bc060ce162814'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:11,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d73575bc060ce162814 2025-12-18 00:02:11,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d71575bc060ce162811 HTTP/1.1" 502 150 2025-12-18 00:02:11,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d71575bc060ce162811'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:11,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d73575bc060ce162814 HTTP/1.1" 502 150 2025-12-18 00:02:11,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d73575bc060ce162814'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:11,786 - lava_callback - INFO - Received callback for node 6940a174575bc060ce162e7e INFO: 172.21.0.1:36024 - "POST /node/6940a174575bc060ce162e7e HTTP/1.0" 202 Accepted 2025-12-18 00:02:11,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6d575bc060ce162807 2025-12-18 00:02:11,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6d575bc060ce162807 HTTP/1.1" 502 150 2025-12-18 00:02:11,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6d575bc060ce162807'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:12,420 - lava_callback - INFO - Received callback for node 6940a176575bc060ce162e81 INFO: 172.21.0.1:36028 - "POST /node/6940a176575bc060ce162e81 HTTP/1.0" 202 Accepted 2025-12-18 00:02:13,066 - lava_callback - INFO - Received callback for node 6940a178575bc060ce162e84 INFO: 172.21.0.1:36032 - "POST /node/6940a178575bc060ce162e84 HTTP/1.0" 202 Accepted 2025-12-18 00:02:13,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d71575bc060ce162811 2025-12-18 00:02:13,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d73575bc060ce162814 2025-12-18 00:02:13,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d71575bc060ce162811 HTTP/1.1" 502 150 2025-12-18 00:02:13,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d71575bc060ce162811'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:13,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d73575bc060ce162814 HTTP/1.1" 502 150 2025-12-18 00:02:13,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d73575bc060ce162814'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:14,485 - lava_callback - INFO - Received callback for node 6940a17e575bc060ce162e99 INFO: 172.21.0.1:36040 - "POST /node/6940a17e575bc060ce162e99 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:36054 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-18 00:02:15,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6f575bc060ce16280c 2025-12-18 00:02:15,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6f575bc060ce16280c HTTP/1.1" 502 150 2025-12-18 00:02:15,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6f575bc060ce16280c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:15,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d55575bc060ce1627d5 2025-12-18 00:02:15,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d55575bc060ce1627d5 HTTP/1.1" 502 150 2025-12-18 00:02:15,645 - lava_callback - ERROR - Error processing callback for node 69409d55575bc060ce1627d5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d55575bc060ce1627d5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d55575bc060ce1627d5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d55575bc060ce1627d5 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:15,647 - lava_callback - INFO - Starting async job submit for node 69409d75575bc060ce162817 2025-12-18 00:02:15,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:15,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d75575bc060ce162817 HTTP/1.1" 502 150 2025-12-18 00:02:15,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d75575bc060ce162817'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:15,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d75575bc060ce162817 2025-12-18 00:02:15,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d75575bc060ce162817 HTTP/1.1" 502 150 2025-12-18 00:02:15,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d75575bc060ce162817'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:16,177 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d57575bc060ce1627d8 2025-12-18 00:02:16,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d57575bc060ce1627d8 HTTP/1.1" 502 150 2025-12-18 00:02:16,178 - lava_callback - ERROR - Error processing callback for node 69409d57575bc060ce1627d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d57575bc060ce1627d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d57575bc060ce1627d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d57575bc060ce1627d8 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:16,181 - lava_callback - INFO - Starting async job submit for node 69409d77575bc060ce16281a 2025-12-18 00:02:16,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:16,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d77575bc060ce16281a HTTP/1.1" 502 150 2025-12-18 00:02:16,190 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d77575bc060ce16281a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:16,190 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d77575bc060ce16281a 2025-12-18 00:02:16,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d77575bc060ce16281a HTTP/1.1" 502 150 2025-12-18 00:02:16,191 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d77575bc060ce16281a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36056 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:02:17,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6b575bc060ce162804 2025-12-18 00:02:17,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6b575bc060ce162804 HTTP/1.1" 502 150 2025-12-18 00:02:17,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6b575bc060ce162804'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:17,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d75575bc060ce162817 2025-12-18 00:02:17,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d75575bc060ce162817 HTTP/1.1" 502 150 2025-12-18 00:02:17,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d75575bc060ce162817'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:17,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d71575bc060ce162811 2025-12-18 00:02:17,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d73575bc060ce162814 2025-12-18 00:02:17,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d71575bc060ce162811 HTTP/1.1" 502 150 2025-12-18 00:02:17,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d71575bc060ce162811'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:17,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d73575bc060ce162814 HTTP/1.1" 502 150 2025-12-18 00:02:17,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d73575bc060ce162814'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:18,191 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d77575bc060ce16281a 2025-12-18 00:02:18,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d77575bc060ce16281a HTTP/1.1" 502 150 2025-12-18 00:02:18,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d77575bc060ce16281a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:18,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d59575bc060ce1627db 2025-12-18 00:02:18,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d59575bc060ce1627db HTTP/1.1" 502 150 2025-12-18 00:02:18,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d59575bc060ce1627db'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:19,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d5b575bc060ce1627e0 2025-12-18 00:02:19,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d5b575bc060ce1627e0 HTTP/1.1" 502 150 2025-12-18 00:02:19,567 - lava_callback - ERROR - Error processing callback for node 69409d5b575bc060ce1627e0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d5b575bc060ce1627e0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d5b575bc060ce1627e0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d5b575bc060ce1627e0 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:19,569 - lava_callback - INFO - Starting async job submit for node 69409d82575bc060ce16282a 2025-12-18 00:02:19,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:19,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d82575bc060ce16282a HTTP/1.1" 502 150 2025-12-18 00:02:19,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d82575bc060ce16282a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:19,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d82575bc060ce16282a 2025-12-18 00:02:19,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d82575bc060ce16282a HTTP/1.1" 502 150 2025-12-18 00:02:19,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d82575bc060ce16282a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:19,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6d575bc060ce162807 2025-12-18 00:02:19,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6d575bc060ce162807 HTTP/1.1" 502 150 2025-12-18 00:02:19,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6d575bc060ce162807'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:21,219 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d60575bc060ce1627f2 2025-12-18 00:02:21,233 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d5d575bc060ce1627e9 2025-12-18 00:02:21,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d5d575bc060ce1627e9 HTTP/1.1" 502 150 2025-12-18 00:02:21,234 - lava_callback - ERROR - Error processing callback for node 69409d5d575bc060ce1627e9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d5d575bc060ce1627e9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d5d575bc060ce1627e9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d5d575bc060ce1627e9 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:21,236 - lava_callback - INFO - Starting async job submit for node 69409d79575bc060ce16281d 2025-12-18 00:02:21,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:21,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d79575bc060ce16281d HTTP/1.1" 502 150 2025-12-18 00:02:21,245 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d79575bc060ce16281d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:21,245 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d79575bc060ce16281d 2025-12-18 00:02:21,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d79575bc060ce16281d HTTP/1.1" 502 150 2025-12-18 00:02:21,246 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d79575bc060ce16281d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:21,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d82575bc060ce16282a 2025-12-18 00:02:21,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d82575bc060ce16282a HTTP/1.1" 502 150 2025-12-18 00:02:21,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d82575bc060ce16282a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:21,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d75575bc060ce162817 2025-12-18 00:02:21,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d75575bc060ce162817 HTTP/1.1" 502 150 2025-12-18 00:02:21,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d75575bc060ce162817'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:22,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d77575bc060ce16281a 2025-12-18 00:02:22,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d77575bc060ce16281a HTTP/1.1" 502 150 2025-12-18 00:02:22,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d77575bc060ce16281a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:22,343 - lava_callback - INFO - Received callback for node 69409dc7575bc060ce1628c8 INFO: 172.21.0.1:35698 - "POST /node/69409dc7575bc060ce1628c8 HTTP/1.0" 202 Accepted 2025-12-18 00:02:22,552 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d62575bc060ce1627f5 2025-12-18 00:02:22,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d62575bc060ce1627f5 HTTP/1.1" 502 150 2025-12-18 00:02:22,553 - lava_callback - ERROR - Error processing callback for node 69409d62575bc060ce1627f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d62575bc060ce1627f5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d62575bc060ce1627f5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d62575bc060ce1627f5 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:22,555 - lava_callback - INFO - Starting async job submit for node 69409d7c575bc060ce162821 2025-12-18 00:02:22,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:22,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d7c575bc060ce162821 HTTP/1.1" 502 150 2025-12-18 00:02:22,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d7c575bc060ce162821'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:22,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d7c575bc060ce162821 2025-12-18 00:02:22,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d7c575bc060ce162821 HTTP/1.1" 502 150 2025-12-18 00:02:22,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d7c575bc060ce162821'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:23,246 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d79575bc060ce16281d 2025-12-18 00:02:23,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6f575bc060ce16280c 2025-12-18 00:02:23,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6f575bc060ce16280c HTTP/1.1" 502 150 2025-12-18 00:02:23,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d6f575bc060ce16280c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:23,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a31c575bc060ce16314c 2025-12-18 00:02:23,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a31c575bc060ce16314c HTTP/1.1" 502 150 2025-12-18 00:02:23,475 - lava_callback - ERROR - Error processing callback for node 6940a31c575bc060ce16314c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a31c575bc060ce16314c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a31c575bc060ce16314c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a31c575bc060ce16314c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:23,477 - lava_callback - INFO - Starting async job submit for node 69409d7e575bc060ce162824 2025-12-18 00:02:23,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:23,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d7e575bc060ce162824 HTTP/1.1" 502 150 2025-12-18 00:02:23,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d7e575bc060ce162824'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:23,503 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d7e575bc060ce162824 2025-12-18 00:02:23,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d7e575bc060ce162824 HTTP/1.1" 502 150 2025-12-18 00:02:23,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d7e575bc060ce162824'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:23,730 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d64575bc060ce1627f8 2025-12-18 00:02:23,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d64575bc060ce1627f8 HTTP/1.1" 502 150 2025-12-18 00:02:23,731 - lava_callback - ERROR - Error processing callback for node 69409d64575bc060ce1627f8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d64575bc060ce1627f8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d64575bc060ce1627f8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d64575bc060ce1627f8 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:23,733 - lava_callback - INFO - Starting async job submit for node 69409d80575bc060ce162827 2025-12-18 00:02:23,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:23,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d80575bc060ce162827 HTTP/1.1" 502 150 2025-12-18 00:02:23,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d80575bc060ce162827'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:23,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d80575bc060ce162827 2025-12-18 00:02:23,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d80575bc060ce162827 HTTP/1.1" 502 150 2025-12-18 00:02:23,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d80575bc060ce162827'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:24,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d7c575bc060ce162821 2025-12-18 00:02:24,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d7c575bc060ce162821 HTTP/1.1" 502 150 2025-12-18 00:02:24,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d7c575bc060ce162821'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:25,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d7e575bc060ce162824 2025-12-18 00:02:25,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d7e575bc060ce162824 HTTP/1.1" 502 150 2025-12-18 00:02:25,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d7e575bc060ce162824'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:25,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d82575bc060ce16282a 2025-12-18 00:02:25,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d82575bc060ce16282a HTTP/1.1" 502 150 2025-12-18 00:02:25,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d82575bc060ce16282a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:25,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d80575bc060ce162827 2025-12-18 00:02:25,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d80575bc060ce162827 HTTP/1.1" 502 150 2025-12-18 00:02:25,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d80575bc060ce162827'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:25,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d71575bc060ce162811 2025-12-18 00:02:25,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d73575bc060ce162814 2025-12-18 00:02:25,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d71575bc060ce162811 HTTP/1.1" 502 150 2025-12-18 00:02:25,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d71575bc060ce162811'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:25,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d73575bc060ce162814 HTTP/1.1" 502 150 2025-12-18 00:02:25,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d73575bc060ce162814'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35700 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:02:26,629 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d66575bc060ce1627fb 2025-12-18 00:02:26,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d66575bc060ce1627fb HTTP/1.1" 502 150 2025-12-18 00:02:26,630 - lava_callback - ERROR - Error processing callback for node 69409d66575bc060ce1627fb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d66575bc060ce1627fb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d66575bc060ce1627fb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d66575bc060ce1627fb (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:26,632 - lava_callback - INFO - Starting async job submit for node 69409d84575bc060ce162835 2025-12-18 00:02:26,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:26,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d84575bc060ce162835 HTTP/1.1" 502 150 2025-12-18 00:02:26,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d84575bc060ce162835'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:26,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d84575bc060ce162835 2025-12-18 00:02:26,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d84575bc060ce162835 HTTP/1.1" 502 150 2025-12-18 00:02:26,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d84575bc060ce162835'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:27,634 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6a575bc060ce162801 2025-12-18 00:02:27,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6a575bc060ce162801 HTTP/1.1" 502 150 2025-12-18 00:02:27,636 - lava_callback - ERROR - Error processing callback for node 69409d6a575bc060ce162801: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d6a575bc060ce162801 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d6a575bc060ce162801 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d6a575bc060ce162801 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:27,637 - lava_callback - INFO - Starting async job submit for node 69409d86575bc060ce16283c 2025-12-18 00:02:27,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:27,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d86575bc060ce16283c HTTP/1.1" 502 150 2025-12-18 00:02:27,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d86575bc060ce16283c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:27,648 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d86575bc060ce16283c 2025-12-18 00:02:27,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d86575bc060ce16283c HTTP/1.1" 502 150 2025-12-18 00:02:27,649 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d86575bc060ce16283c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:28,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d7c575bc060ce162821 2025-12-18 00:02:28,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d7c575bc060ce162821 HTTP/1.1" 502 150 2025-12-18 00:02:28,565 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d7c575bc060ce162821'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:28,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d84575bc060ce162835 2025-12-18 00:02:28,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d84575bc060ce162835 HTTP/1.1" 502 150 2025-12-18 00:02:28,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d84575bc060ce162835'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:29,167 - lava_callback - INFO - Received callback for node 6940a180575bc060ce162e9e INFO: 172.21.0.1:35714 - "POST /node/6940a180575bc060ce162e9e HTTP/1.0" 202 Accepted 2025-12-18 00:02:29,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d7e575bc060ce162824 2025-12-18 00:02:29,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d7e575bc060ce162824 HTTP/1.1" 502 150 2025-12-18 00:02:29,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d7e575bc060ce162824'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:29,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d86575bc060ce16283c 2025-12-18 00:02:29,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d86575bc060ce16283c HTTP/1.1" 502 150 2025-12-18 00:02:29,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d86575bc060ce16283c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:29,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d75575bc060ce162817 2025-12-18 00:02:29,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d75575bc060ce162817 HTTP/1.1" 502 150 2025-12-18 00:02:29,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d75575bc060ce162817'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:29,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d80575bc060ce162827 2025-12-18 00:02:29,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d80575bc060ce162827 HTTP/1.1" 502 150 2025-12-18 00:02:29,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d80575bc060ce162827'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:29,812 - lava_callback - INFO - Received callback for node 6940a182575bc060ce162ea9 INFO: 172.21.0.1:42138 - "POST /node/6940a182575bc060ce162ea9 HTTP/1.0" 202 Accepted 2025-12-18 00:02:30,194 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d77575bc060ce16281a 2025-12-18 00:02:30,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d77575bc060ce16281a HTTP/1.1" 502 150 2025-12-18 00:02:30,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d77575bc060ce16281a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:30,460 - lava_callback - INFO - Received callback for node 6940a184575bc060ce162eae INFO: 172.21.0.1:42140 - "POST /node/6940a184575bc060ce162eae HTTP/1.0" 202 Accepted 2025-12-18 00:02:31,109 - lava_callback - INFO - Received callback for node 6940a186575bc060ce162eb1 INFO: 172.21.0.1:42150 - "POST /node/6940a186575bc060ce162eb1 HTTP/1.0" 202 Accepted 2025-12-18 00:02:31,743 - lava_callback - INFO - Received callback for node 6940a188575bc060ce162eb4 INFO: 172.21.0.1:42164 - "POST /node/6940a188575bc060ce162eb4 HTTP/1.0" 202 Accepted 2025-12-18 00:02:32,402 - lava_callback - INFO - Received callback for node 6940a18a575bc060ce162eb7 INFO: 172.21.0.1:42170 - "POST /node/6940a18a575bc060ce162eb7 HTTP/1.0" 202 Accepted 2025-12-18 00:02:32,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d84575bc060ce162835 2025-12-18 00:02:32,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d84575bc060ce162835 HTTP/1.1" 502 150 2025-12-18 00:02:32,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d84575bc060ce162835'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:33,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6b575bc060ce162804 2025-12-18 00:02:33,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6b575bc060ce162804 HTTP/1.1" 502 150 2025-12-18 00:02:33,523 - lava_callback - ERROR - Error processing callback for node 69409d6b575bc060ce162804: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d6b575bc060ce162804 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d6b575bc060ce162804 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d6b575bc060ce162804 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:33,526 - lava_callback - INFO - Starting async job submit for node 69409d88575bc060ce16283f 2025-12-18 00:02:33,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:33,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d88575bc060ce16283f HTTP/1.1" 502 150 2025-12-18 00:02:33,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d88575bc060ce16283f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:33,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d88575bc060ce16283f 2025-12-18 00:02:33,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d88575bc060ce16283f HTTP/1.1" 502 150 2025-12-18 00:02:33,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d88575bc060ce16283f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:33,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d82575bc060ce16282a 2025-12-18 00:02:33,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d82575bc060ce16282a HTTP/1.1" 502 150 2025-12-18 00:02:33,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d82575bc060ce16282a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:33,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d86575bc060ce16283c 2025-12-18 00:02:33,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d86575bc060ce16283c HTTP/1.1" 502 150 2025-12-18 00:02:33,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d86575bc060ce16283c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:34,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d59575bc060ce1627db 2025-12-18 00:02:34,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d59575bc060ce1627db HTTP/1.1" 502 150 2025-12-18 00:02:34,995 - lava_callback - ERROR - Error processing callback for node 69409d59575bc060ce1627db: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d59575bc060ce1627db (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d59575bc060ce1627db (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d59575bc060ce1627db (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:34,997 - lava_callback - INFO - Starting async job submit for node 69409d8a575bc060ce162842 2025-12-18 00:02:34,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:35,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8a575bc060ce162842 HTTP/1.1" 502 150 2025-12-18 00:02:35,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d8a575bc060ce162842'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:35,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d8a575bc060ce162842 2025-12-18 00:02:35,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8a575bc060ce162842 HTTP/1.1" 502 150 2025-12-18 00:02:35,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d8a575bc060ce162842'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:35,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d88575bc060ce16283f 2025-12-18 00:02:35,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d88575bc060ce16283f HTTP/1.1" 502 150 2025-12-18 00:02:35,535 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d88575bc060ce16283f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:35,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6d575bc060ce162807 2025-12-18 00:02:35,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6d575bc060ce162807 HTTP/1.1" 502 150 2025-12-18 00:02:35,994 - lava_callback - ERROR - Error processing callback for node 69409d6d575bc060ce162807: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d6d575bc060ce162807 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d6d575bc060ce162807 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d6d575bc060ce162807 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:35,996 - lava_callback - INFO - Starting async job submit for node 69409d8e575bc060ce162848 2025-12-18 00:02:35,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:42182 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:02:36,566 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d7c575bc060ce162821 2025-12-18 00:02:36,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d7c575bc060ce162821 HTTP/1.1" 502 150 2025-12-18 00:02:36,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d7c575bc060ce162821'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:36,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d60575bc060ce1627f2 HTTP/1.1" 502 150 2025-12-18 00:02:36,592 - lava_callback - ERROR - Error processing callback for node 69409d60575bc060ce1627f2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d60575bc060ce1627f2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d60575bc060ce1627f2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d60575bc060ce1627f2 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:36,594 - lava_callback - INFO - Starting async job submit for node 69409d90575bc060ce16284b 2025-12-18 00:02:36,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:36,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d90575bc060ce16284b HTTP/1.1" 502 150 2025-12-18 00:02:36,612 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d90575bc060ce16284b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:36,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d90575bc060ce16284b 2025-12-18 00:02:36,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d90575bc060ce16284b HTTP/1.1" 502 150 2025-12-18 00:02:36,613 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d90575bc060ce16284b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:37,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d8a575bc060ce162842 2025-12-18 00:02:37,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d7e575bc060ce162824 2025-12-18 00:02:37,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d7e575bc060ce162824 HTTP/1.1" 502 150 2025-12-18 00:02:37,508 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d7e575bc060ce162824'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:37,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d80575bc060ce162827 2025-12-18 00:02:37,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d80575bc060ce162827 HTTP/1.1" 502 150 2025-12-18 00:02:37,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d80575bc060ce162827'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:38,613 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d90575bc060ce16284b 2025-12-18 00:02:38,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d90575bc060ce16284b HTTP/1.1" 502 150 2025-12-18 00:02:38,614 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d90575bc060ce16284b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:38,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d79575bc060ce16281d HTTP/1.1" 502 150 2025-12-18 00:02:38,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d79575bc060ce16281d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:39,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d6f575bc060ce16280c 2025-12-18 00:02:39,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d6f575bc060ce16280c HTTP/1.1" 502 150 2025-12-18 00:02:39,475 - lava_callback - ERROR - Error processing callback for node 69409d6f575bc060ce16280c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d6f575bc060ce16280c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d6f575bc060ce16280c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d6f575bc060ce16280c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:39,478 - lava_callback - INFO - Starting async job submit for node 69409d92575bc060ce16284e 2025-12-18 00:02:39,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:39,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d92575bc060ce16284e HTTP/1.1" 502 150 2025-12-18 00:02:39,493 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d92575bc060ce16284e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:39,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d92575bc060ce16284e 2025-12-18 00:02:39,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d92575bc060ce16284e HTTP/1.1" 502 150 2025-12-18 00:02:39,494 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d92575bc060ce16284e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:39,535 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d88575bc060ce16283f 2025-12-18 00:02:39,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d88575bc060ce16283f HTTP/1.1" 502 150 2025-12-18 00:02:39,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d88575bc060ce16283f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:40,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d84575bc060ce162835 2025-12-18 00:02:40,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d84575bc060ce162835 HTTP/1.1" 502 150 2025-12-18 00:02:40,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d84575bc060ce162835'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:41,494 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d92575bc060ce16284e 2025-12-18 00:02:41,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d92575bc060ce16284e HTTP/1.1" 502 150 2025-12-18 00:02:41,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d92575bc060ce16284e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:41,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d86575bc060ce16283c 2025-12-18 00:02:41,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d86575bc060ce16283c HTTP/1.1" 502 150 2025-12-18 00:02:41,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d86575bc060ce16283c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:41,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d71575bc060ce162811 2025-12-18 00:02:41,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d73575bc060ce162814 2025-12-18 00:02:41,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d71575bc060ce162811 HTTP/1.1" 502 150 2025-12-18 00:02:41,789 - lava_callback - ERROR - Error processing callback for node 69409d71575bc060ce162811: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d71575bc060ce162811 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d71575bc060ce162811 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d71575bc060ce162811 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:41,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d73575bc060ce162814 HTTP/1.1" 502 150 2025-12-18 00:02:41,791 - lava_callback - INFO - Starting async job submit for node 69409d94575bc060ce162851 2025-12-18 00:02:41,791 - lava_callback - ERROR - Error processing callback for node 69409d73575bc060ce162814: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d73575bc060ce162814 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d73575bc060ce162814 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d73575bc060ce162814 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:41,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:41,794 - lava_callback - INFO - Starting async job submit for node 69409d97575bc060ce162856 2025-12-18 00:02:41,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:41,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d94575bc060ce162851 HTTP/1.1" 502 150 2025-12-18 00:02:41,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d94575bc060ce162851'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:41,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d94575bc060ce162851 2025-12-18 00:02:41,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d94575bc060ce162851 HTTP/1.1" 502 150 2025-12-18 00:02:41,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d94575bc060ce162851'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:41,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d97575bc060ce162856 HTTP/1.1" 502 150 2025-12-18 00:02:41,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d97575bc060ce162856'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:41,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d97575bc060ce162856 2025-12-18 00:02:41,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d97575bc060ce162856 HTTP/1.1" 502 150 2025-12-18 00:02:41,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d97575bc060ce162856'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:42,614 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d90575bc060ce16284b 2025-12-18 00:02:42,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d90575bc060ce16284b HTTP/1.1" 502 150 2025-12-18 00:02:42,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d90575bc060ce16284b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:42,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d79575bc060ce16281d 2025-12-18 00:02:42,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d79575bc060ce16281d HTTP/1.1" 502 150 2025-12-18 00:02:42,641 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d79575bc060ce16281d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:43,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d94575bc060ce162851 2025-12-18 00:02:43,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d94575bc060ce162851 HTTP/1.1" 502 150 2025-12-18 00:02:43,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d94575bc060ce162851'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:43,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d97575bc060ce162856 2025-12-18 00:02:43,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d97575bc060ce162856 HTTP/1.1" 502 150 2025-12-18 00:02:43,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d97575bc060ce162856'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:45,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d92575bc060ce16284e 2025-12-18 00:02:45,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d92575bc060ce16284e HTTP/1.1" 502 150 2025-12-18 00:02:45,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d92575bc060ce16284e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:45,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d75575bc060ce162817 2025-12-18 00:02:45,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d75575bc060ce162817 HTTP/1.1" 502 150 2025-12-18 00:02:45,662 - lava_callback - ERROR - Error processing callback for node 69409d75575bc060ce162817: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d75575bc060ce162817 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d75575bc060ce162817 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d75575bc060ce162817 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:45,663 - lava_callback - INFO - Starting async job submit for node 6940a4411e3c24e956819c49 2025-12-18 00:02:45,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:45,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a4411e3c24e956819c49 HTTP/1.1" 502 150 2025-12-18 00:02:45,709 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a4411e3c24e956819c49'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:45,709 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a4411e3c24e956819c49 2025-12-18 00:02:45,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a4411e3c24e956819c49 HTTP/1.1" 502 150 2025-12-18 00:02:45,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a4411e3c24e956819c49'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:46,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d77575bc060ce16281a 2025-12-18 00:02:46,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d77575bc060ce16281a HTTP/1.1" 502 150 2025-12-18 00:02:46,197 - lava_callback - ERROR - Error processing callback for node 69409d77575bc060ce16281a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d77575bc060ce16281a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d77575bc060ce16281a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d77575bc060ce16281a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:46,199 - lava_callback - INFO - Starting async job submit for node 69409d8d575bc060ce162845 2025-12-18 00:02:46,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:46,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8d575bc060ce162845 HTTP/1.1" 502 150 2025-12-18 00:02:46,207 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d8d575bc060ce162845'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:46,207 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d8d575bc060ce162845 2025-12-18 00:02:46,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8d575bc060ce162845 HTTP/1.1" 502 150 2025-12-18 00:02:46,208 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d8d575bc060ce162845'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38678 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:02:47,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d88575bc060ce16283f 2025-12-18 00:02:47,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d88575bc060ce16283f HTTP/1.1" 502 150 2025-12-18 00:02:47,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d88575bc060ce16283f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:47,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a4411e3c24e956819c49 2025-12-18 00:02:47,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a4411e3c24e956819c49 HTTP/1.1" 502 150 2025-12-18 00:02:47,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a4411e3c24e956819c49'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:47,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d94575bc060ce162851 2025-12-18 00:02:47,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d94575bc060ce162851 HTTP/1.1" 502 150 2025-12-18 00:02:47,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d94575bc060ce162851'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:47,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d97575bc060ce162856 2025-12-18 00:02:47,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d97575bc060ce162856 HTTP/1.1" 502 150 2025-12-18 00:02:47,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d97575bc060ce162856'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:48,208 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d8d575bc060ce162845 2025-12-18 00:02:48,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8d575bc060ce162845 HTTP/1.1" 502 150 2025-12-18 00:02:48,210 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d8d575bc060ce162845'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:49,196 - lava_callback - INFO - Received callback for node 6940a194575bc060ce162ebc INFO: 172.21.0.1:38680 - "POST /node/6940a194575bc060ce162ebc HTTP/1.0" 202 Accepted 2025-12-18 00:02:49,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d82575bc060ce16282a 2025-12-18 00:02:49,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d82575bc060ce16282a HTTP/1.1" 502 150 2025-12-18 00:02:49,583 - lava_callback - ERROR - Error processing callback for node 69409d82575bc060ce16282a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d82575bc060ce16282a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d82575bc060ce16282a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d82575bc060ce16282a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:49,584 - lava_callback - INFO - Starting async job submit for node 69409d99575bc060ce16285f 2025-12-18 00:02:49,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:49,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d99575bc060ce16285f HTTP/1.1" 502 150 2025-12-18 00:02:49,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d99575bc060ce16285f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:49,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d99575bc060ce16285f 2025-12-18 00:02:49,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d99575bc060ce16285f HTTP/1.1" 502 150 2025-12-18 00:02:49,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d99575bc060ce16285f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:49,851 - lava_callback - INFO - Received callback for node 6940a196575bc060ce162ecb INFO: 172.21.0.1:45638 - "POST /node/6940a196575bc060ce162ecb HTTP/1.0" 202 Accepted 2025-12-18 00:02:50,013 - lava_callback - INFO - Received callback for node 6940a198575bc060ce162ece INFO: 172.21.0.1:45648 - "POST /node/6940a198575bc060ce162ece HTTP/1.0" 202 Accepted 2025-12-18 00:02:50,512 - lava_callback - INFO - Received callback for node 6940a19a575bc060ce162ed1 INFO: 172.21.0.1:45660 - "POST /node/6940a19a575bc060ce162ed1 HTTP/1.0" 202 Accepted 2025-12-18 00:02:50,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d90575bc060ce16284b 2025-12-18 00:02:50,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d90575bc060ce16284b HTTP/1.1" 502 150 2025-12-18 00:02:50,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d90575bc060ce16284b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:50,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d79575bc060ce16281d 2025-12-18 00:02:50,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d79575bc060ce16281d HTTP/1.1" 502 150 2025-12-18 00:02:50,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d79575bc060ce16281d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:51,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8e575bc060ce162848 HTTP/1.1" 502 150 2025-12-18 00:02:51,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d8e575bc060ce162848'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:51,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d8e575bc060ce162848 2025-12-18 00:02:51,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8e575bc060ce162848 HTTP/1.1" 502 150 2025-12-18 00:02:51,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d8e575bc060ce162848'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:51,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d99575bc060ce16285f 2025-12-18 00:02:51,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d99575bc060ce16285f HTTP/1.1" 502 150 2025-12-18 00:02:51,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d99575bc060ce16285f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:51,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a4411e3c24e956819c49 2025-12-18 00:02:51,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a4411e3c24e956819c49 HTTP/1.1" 502 150 2025-12-18 00:02:51,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a4411e3c24e956819c49'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:52,210 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d8d575bc060ce162845 2025-12-18 00:02:52,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8d575bc060ce162845 HTTP/1.1" 502 150 2025-12-18 00:02:52,211 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d8d575bc060ce162845'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:52,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8a575bc060ce162842 HTTP/1.1" 502 150 2025-12-18 00:02:52,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d8a575bc060ce162842'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:52,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d7c575bc060ce162821 2025-12-18 00:02:52,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d7c575bc060ce162821 HTTP/1.1" 502 150 2025-12-18 00:02:52,569 - lava_callback - ERROR - Error processing callback for node 69409d7c575bc060ce162821: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d7c575bc060ce162821 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d7c575bc060ce162821 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d7c575bc060ce162821 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:52,571 - lava_callback - INFO - Starting async job submit for node 69409d9b575bc060ce162868 2025-12-18 00:02:52,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:52,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9b575bc060ce162868 HTTP/1.1" 502 150 2025-12-18 00:02:52,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d9b575bc060ce162868'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:52,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d9b575bc060ce162868 2025-12-18 00:02:52,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9b575bc060ce162868 HTTP/1.1" 502 150 2025-12-18 00:02:52,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d9b575bc060ce162868'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:53,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d8e575bc060ce162848 2025-12-18 00:02:53,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8e575bc060ce162848 HTTP/1.1" 502 150 2025-12-18 00:02:53,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d8e575bc060ce162848'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:53,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d92575bc060ce16284e 2025-12-18 00:02:53,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d92575bc060ce16284e HTTP/1.1" 502 150 2025-12-18 00:02:53,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d92575bc060ce16284e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:53,508 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d7e575bc060ce162824 2025-12-18 00:02:53,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d7e575bc060ce162824 HTTP/1.1" 502 150 2025-12-18 00:02:53,509 - lava_callback - ERROR - Error processing callback for node 69409d7e575bc060ce162824: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d7e575bc060ce162824 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d7e575bc060ce162824 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d7e575bc060ce162824 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:53,511 - lava_callback - INFO - Starting async job submit for node 69409d9d575bc060ce16286b 2025-12-18 00:02:53,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:53,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9d575bc060ce16286b HTTP/1.1" 502 150 2025-12-18 00:02:53,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d9d575bc060ce16286b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:53,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d9d575bc060ce16286b 2025-12-18 00:02:53,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9d575bc060ce16286b HTTP/1.1" 502 150 2025-12-18 00:02:53,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d9d575bc060ce16286b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:53,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d80575bc060ce162827 2025-12-18 00:02:53,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d80575bc060ce162827 HTTP/1.1" 502 150 2025-12-18 00:02:53,749 - lava_callback - ERROR - Error processing callback for node 69409d80575bc060ce162827: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d80575bc060ce162827 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d80575bc060ce162827 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d80575bc060ce162827 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:53,751 - lava_callback - INFO - Starting async job submit for node 69409d9f575bc060ce16286e 2025-12-18 00:02:53,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:53,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9f575bc060ce16286e HTTP/1.1" 502 150 2025-12-18 00:02:53,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d9f575bc060ce16286e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:53,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d9f575bc060ce16286e 2025-12-18 00:02:53,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9f575bc060ce16286e HTTP/1.1" 502 150 2025-12-18 00:02:53,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d9f575bc060ce16286e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:54,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d9b575bc060ce162868 2025-12-18 00:02:54,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9b575bc060ce162868 HTTP/1.1" 502 150 2025-12-18 00:02:54,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d9b575bc060ce162868'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:55,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d9d575bc060ce16286b 2025-12-18 00:02:55,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9d575bc060ce16286b HTTP/1.1" 502 150 2025-12-18 00:02:55,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d9d575bc060ce16286b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:55,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d99575bc060ce16285f 2025-12-18 00:02:55,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d99575bc060ce16285f HTTP/1.1" 502 150 2025-12-18 00:02:55,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d99575bc060ce16285f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:55,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d9f575bc060ce16286e 2025-12-18 00:02:55,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9f575bc060ce16286e HTTP/1.1" 502 150 2025-12-18 00:02:55,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d9f575bc060ce16286e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:55,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d94575bc060ce162851 2025-12-18 00:02:55,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d94575bc060ce162851 HTTP/1.1" 502 150 2025-12-18 00:02:55,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d94575bc060ce162851'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:55,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d97575bc060ce162856 2025-12-18 00:02:55,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d97575bc060ce162856 HTTP/1.1" 502 150 2025-12-18 00:02:55,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d97575bc060ce162856'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:56,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d8a575bc060ce162842 2025-12-18 00:02:56,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8a575bc060ce162842 HTTP/1.1" 502 150 2025-12-18 00:02:56,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d8a575bc060ce162842'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45672 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:02:56,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d84575bc060ce162835 2025-12-18 00:02:56,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d84575bc060ce162835 HTTP/1.1" 502 150 2025-12-18 00:02:56,667 - lava_callback - ERROR - Error processing callback for node 69409d84575bc060ce162835: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d84575bc060ce162835 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d84575bc060ce162835 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d84575bc060ce162835 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:56,670 - lava_callback - INFO - Starting async job submit for node 69409da1575bc060ce162871 2025-12-18 00:02:56,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:56,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409da1575bc060ce162871 HTTP/1.1" 502 150 2025-12-18 00:02:56,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409da1575bc060ce162871'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:56,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409da1575bc060ce162871 2025-12-18 00:02:56,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409da1575bc060ce162871 HTTP/1.1" 502 150 2025-12-18 00:02:56,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409da1575bc060ce162871'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:57,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d8e575bc060ce162848 2025-12-18 00:02:57,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8e575bc060ce162848 HTTP/1.1" 502 150 2025-12-18 00:02:57,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d8e575bc060ce162848'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:57,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d86575bc060ce16283c 2025-12-18 00:02:57,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d86575bc060ce16283c HTTP/1.1" 502 150 2025-12-18 00:02:57,654 - lava_callback - ERROR - Error processing callback for node 69409d86575bc060ce16283c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d86575bc060ce16283c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d86575bc060ce16283c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d86575bc060ce16283c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:02:57,656 - lava_callback - INFO - Starting async job submit for node 69409da3575bc060ce162874 2025-12-18 00:02:57,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:02:57,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409da3575bc060ce162874 HTTP/1.1" 502 150 2025-12-18 00:02:57,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409da3575bc060ce162874'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:57,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409da3575bc060ce162874 2025-12-18 00:02:57,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409da3575bc060ce162874 HTTP/1.1" 502 150 2025-12-18 00:02:57,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409da3575bc060ce162874'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:58,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d9b575bc060ce162868 2025-12-18 00:02:58,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9b575bc060ce162868 HTTP/1.1" 502 150 2025-12-18 00:02:58,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d9b575bc060ce162868'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:58,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409da1575bc060ce162871 2025-12-18 00:02:58,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409da1575bc060ce162871 HTTP/1.1" 502 150 2025-12-18 00:02:58,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409da1575bc060ce162871'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:59,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d9d575bc060ce16286b 2025-12-18 00:02:59,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9d575bc060ce16286b HTTP/1.1" 502 150 2025-12-18 00:02:59,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d9d575bc060ce16286b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:59,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409da3575bc060ce162874 2025-12-18 00:02:59,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409da3575bc060ce162874 HTTP/1.1" 502 150 2025-12-18 00:02:59,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409da3575bc060ce162874'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:59,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a4411e3c24e956819c49 2025-12-18 00:02:59,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a4411e3c24e956819c49 HTTP/1.1" 502 150 2025-12-18 00:02:59,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a4411e3c24e956819c49'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:02:59,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d9f575bc060ce16286e 2025-12-18 00:02:59,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9f575bc060ce16286e HTTP/1.1" 502 150 2025-12-18 00:02:59,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d9f575bc060ce16286e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:00,212 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d8d575bc060ce162845 2025-12-18 00:03:00,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8d575bc060ce162845 HTTP/1.1" 502 150 2025-12-18 00:03:00,213 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d8d575bc060ce162845'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:02,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409da1575bc060ce162871 2025-12-18 00:03:02,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409da1575bc060ce162871 HTTP/1.1" 502 150 2025-12-18 00:03:02,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409da1575bc060ce162871'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:03,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d88575bc060ce16283f 2025-12-18 00:03:03,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d99575bc060ce16285f 2025-12-18 00:03:03,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d99575bc060ce16285f HTTP/1.1" 502 150 2025-12-18 00:03:03,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d99575bc060ce16285f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:03,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409da3575bc060ce162874 2025-12-18 00:03:03,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409da3575bc060ce162874 HTTP/1.1" 502 150 2025-12-18 00:03:03,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409da3575bc060ce162874'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38188 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:03:04,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d8a575bc060ce162842 2025-12-18 00:03:04,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8a575bc060ce162842 HTTP/1.1" 502 150 2025-12-18 00:03:04,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d8a575bc060ce162842'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:05,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d8e575bc060ce162848 2025-12-18 00:03:05,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8e575bc060ce162848 HTTP/1.1" 502 150 2025-12-18 00:03:05,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d8e575bc060ce162848'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38190 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:03:06,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d9b575bc060ce162868 2025-12-18 00:03:06,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9b575bc060ce162868 HTTP/1.1" 502 150 2025-12-18 00:03:06,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d9b575bc060ce162868'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:06,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d90575bc060ce16284b 2025-12-18 00:03:06,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d90575bc060ce16284b HTTP/1.1" 502 150 2025-12-18 00:03:06,620 - lava_callback - ERROR - Error processing callback for node 69409d90575bc060ce16284b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d90575bc060ce16284b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d90575bc060ce16284b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d90575bc060ce16284b (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:06,622 - lava_callback - INFO - Starting async job submit for node 6940a381575bc060ce1631ee 2025-12-18 00:03:06,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d79575bc060ce16281d 2025-12-18 00:03:06,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d79575bc060ce16281d HTTP/1.1" 502 150 2025-12-18 00:03:06,660 - lava_callback - ERROR - Error processing callback for node 69409d79575bc060ce16281d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d79575bc060ce16281d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d79575bc060ce16281d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d79575bc060ce16281d (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:06,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:06,689 - lava_callback - INFO - Starting async job submit for node 69409dab575bc060ce162879 2025-12-18 00:03:06,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:06,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a381575bc060ce1631ee HTTP/1.1" 502 150 2025-12-18 00:03:06,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a381575bc060ce1631ee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:06,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a381575bc060ce1631ee 2025-12-18 00:03:06,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dab575bc060ce162879 HTTP/1.1" 502 150 2025-12-18 00:03:06,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dab575bc060ce162879'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:06,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dab575bc060ce162879 2025-12-18 00:03:06,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a381575bc060ce1631ee HTTP/1.1" 502 150 2025-12-18 00:03:06,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a381575bc060ce1631ee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:06,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dab575bc060ce162879 HTTP/1.1" 502 150 2025-12-18 00:03:06,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dab575bc060ce162879'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:07,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d9d575bc060ce16286b 2025-12-18 00:03:07,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9d575bc060ce16286b HTTP/1.1" 502 150 2025-12-18 00:03:07,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d9d575bc060ce16286b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:07,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d9f575bc060ce16286e 2025-12-18 00:03:07,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9f575bc060ce16286e HTTP/1.1" 502 150 2025-12-18 00:03:07,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d9f575bc060ce16286e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:08,268 - lava_callback - INFO - Received callback for node 69409d46575bc060ce1627af INFO: 172.21.0.1:38196 - "POST /node/69409d46575bc060ce1627af HTTP/1.0" 202 Accepted 2025-12-18 00:03:08,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a381575bc060ce1631ee 2025-12-18 00:03:08,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dab575bc060ce162879 2025-12-18 00:03:08,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a381575bc060ce1631ee HTTP/1.1" 502 150 2025-12-18 00:03:08,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a381575bc060ce1631ee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:08,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dab575bc060ce162879 HTTP/1.1" 502 150 2025-12-18 00:03:08,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dab575bc060ce162879'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:09,200 - lava_callback - INFO - Received callback for node 6940a1a0575bc060ce162ed6 INFO: 172.21.0.1:38210 - "POST /node/6940a1a0575bc060ce162ed6 HTTP/1.0" 202 Accepted 2025-12-18 00:03:09,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d92575bc060ce16284e 2025-12-18 00:03:09,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d92575bc060ce16284e HTTP/1.1" 502 150 2025-12-18 00:03:09,500 - lava_callback - ERROR - Error processing callback for node 69409d92575bc060ce16284e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d92575bc060ce16284e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d92575bc060ce16284e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d92575bc060ce16284e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:09,503 - lava_callback - INFO - Starting async job submit for node 69409dad575bc060ce162884 2025-12-18 00:03:09,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:09,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dad575bc060ce162884 HTTP/1.1" 502 150 2025-12-18 00:03:09,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dad575bc060ce162884'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:09,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dad575bc060ce162884 2025-12-18 00:03:09,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dad575bc060ce162884 HTTP/1.1" 502 150 2025-12-18 00:03:09,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dad575bc060ce162884'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:09,845 - lava_callback - INFO - Received callback for node 6940a1b9575bc060ce162eec INFO: 172.21.0.1:39968 - "POST /node/6940a1b9575bc060ce162eec HTTP/1.0" 202 Accepted 2025-12-18 00:03:10,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409da1575bc060ce162871 2025-12-18 00:03:10,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409da1575bc060ce162871 HTTP/1.1" 502 150 2025-12-18 00:03:10,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409da1575bc060ce162871'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:11,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dad575bc060ce162884 2025-12-18 00:03:11,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dad575bc060ce162884 HTTP/1.1" 502 150 2025-12-18 00:03:11,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dad575bc060ce162884'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:11,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409da3575bc060ce162874 2025-12-18 00:03:11,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409da3575bc060ce162874 HTTP/1.1" 502 150 2025-12-18 00:03:11,668 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409da3575bc060ce162874'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:11,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d94575bc060ce162851 2025-12-18 00:03:11,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d94575bc060ce162851 HTTP/1.1" 502 150 2025-12-18 00:03:11,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d97575bc060ce162856 2025-12-18 00:03:11,808 - lava_callback - ERROR - Error processing callback for node 69409d94575bc060ce162851: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d94575bc060ce162851 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d94575bc060ce162851 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d94575bc060ce162851 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:11,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d97575bc060ce162856 HTTP/1.1" 502 150 2025-12-18 00:03:11,811 - lava_callback - ERROR - Error processing callback for node 69409d97575bc060ce162856: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d97575bc060ce162856 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d97575bc060ce162856 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d97575bc060ce162856 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:11,814 - lava_callback - INFO - Starting async job submit for node 69409db0575bc060ce16288b 2025-12-18 00:03:11,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:11,815 - lava_callback - INFO - Starting async job submit for node 69409db2575bc060ce162897 2025-12-18 00:03:11,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:11,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db2575bc060ce162897 HTTP/1.1" 502 150 2025-12-18 00:03:11,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db2575bc060ce162897'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:11,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db0575bc060ce16288b HTTP/1.1" 502 150 2025-12-18 00:03:11,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db0575bc060ce16288b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:11,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db2575bc060ce162897 2025-12-18 00:03:11,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db0575bc060ce16288b 2025-12-18 00:03:11,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db2575bc060ce162897 HTTP/1.1" 502 150 2025-12-18 00:03:11,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db2575bc060ce162897'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:11,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db0575bc060ce16288b HTTP/1.1" 502 150 2025-12-18 00:03:11,830 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db0575bc060ce16288b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:12,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a381575bc060ce1631ee 2025-12-18 00:03:12,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dab575bc060ce162879 2025-12-18 00:03:12,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a381575bc060ce1631ee HTTP/1.1" 502 150 2025-12-18 00:03:12,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a381575bc060ce1631ee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:12,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dab575bc060ce162879 HTTP/1.1" 502 150 2025-12-18 00:03:12,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dab575bc060ce162879'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:13,830 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db2575bc060ce162897 2025-12-18 00:03:13,831 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db0575bc060ce16288b 2025-12-18 00:03:13,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db2575bc060ce162897 HTTP/1.1" 502 150 2025-12-18 00:03:13,831 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db2575bc060ce162897'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:13,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db0575bc060ce16288b HTTP/1.1" 502 150 2025-12-18 00:03:13,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db0575bc060ce16288b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39970 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-18 00:03:15,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dad575bc060ce162884 2025-12-18 00:03:15,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dad575bc060ce162884 HTTP/1.1" 502 150 2025-12-18 00:03:15,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dad575bc060ce162884'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:15,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a4411e3c24e956819c49 2025-12-18 00:03:15,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a4411e3c24e956819c49 HTTP/1.1" 502 150 2025-12-18 00:03:15,716 - lava_callback - ERROR - Error processing callback for node 6940a4411e3c24e956819c49: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a4411e3c24e956819c49 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a4411e3c24e956819c49 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a4411e3c24e956819c49 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:15,718 - lava_callback - INFO - Starting async job submit for node 69409db4575bc060ce16289a 2025-12-18 00:03:15,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:15,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db4575bc060ce16289a HTTP/1.1" 502 150 2025-12-18 00:03:15,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db4575bc060ce16289a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:15,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db4575bc060ce16289a 2025-12-18 00:03:15,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db4575bc060ce16289a HTTP/1.1" 502 150 2025-12-18 00:03:15,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db4575bc060ce16289a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:16,213 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d8d575bc060ce162845 2025-12-18 00:03:16,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8d575bc060ce162845 HTTP/1.1" 502 150 2025-12-18 00:03:16,215 - lava_callback - ERROR - Error processing callback for node 69409d8d575bc060ce162845: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d8d575bc060ce162845 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d8d575bc060ce162845 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d8d575bc060ce162845 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:16,217 - lava_callback - INFO - Starting async job submit for node 69409db6575bc060ce16289d 2025-12-18 00:03:16,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:16,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db6575bc060ce16289d HTTP/1.1" 502 150 2025-12-18 00:03:16,231 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db6575bc060ce16289d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:16,231 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db6575bc060ce16289d 2025-12-18 00:03:16,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db6575bc060ce16289d HTTP/1.1" 502 150 2025-12-18 00:03:16,232 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db6575bc060ce16289d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39978 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:03:17,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db4575bc060ce16289a 2025-12-18 00:03:17,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db4575bc060ce16289a HTTP/1.1" 502 150 2025-12-18 00:03:17,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db4575bc060ce16289a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:17,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db2575bc060ce162897 2025-12-18 00:03:17,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db0575bc060ce16288b 2025-12-18 00:03:17,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db2575bc060ce162897 HTTP/1.1" 502 150 2025-12-18 00:03:17,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db2575bc060ce162897'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:17,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db0575bc060ce16288b HTTP/1.1" 502 150 2025-12-18 00:03:17,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db0575bc060ce16288b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:18,232 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db6575bc060ce16289d 2025-12-18 00:03:18,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db6575bc060ce16289d HTTP/1.1" 502 150 2025-12-18 00:03:18,233 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db6575bc060ce16289d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:18,340 - lava_callback - INFO - Received callback for node 6940a1bd575bc060ce162efa INFO: 172.21.0.1:39980 - "POST /node/6940a1bd575bc060ce162efa HTTP/1.0" 202 Accepted 2025-12-18 00:03:18,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d88575bc060ce16283f HTTP/1.1" 502 150 2025-12-18 00:03:18,832 - lava_callback - ERROR - Error processing callback for node 69409d88575bc060ce16283f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d88575bc060ce16283f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d88575bc060ce16283f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d88575bc060ce16283f (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:18,834 - lava_callback - INFO - Starting async job submit for node 69409db8575bc060ce1628a5 2025-12-18 00:03:18,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:18,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db8575bc060ce1628a5 HTTP/1.1" 502 150 2025-12-18 00:03:18,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db8575bc060ce1628a5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:18,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db8575bc060ce1628a5 2025-12-18 00:03:18,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db8575bc060ce1628a5 HTTP/1.1" 502 150 2025-12-18 00:03:18,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db8575bc060ce1628a5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:19,016 - lava_callback - INFO - Received callback for node 6940a1c0575bc060ce162f03 INFO: 172.21.0.1:39990 - "POST /node/6940a1c0575bc060ce162f03 HTTP/1.0" 202 Accepted 2025-12-18 00:03:19,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d99575bc060ce16285f 2025-12-18 00:03:19,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d99575bc060ce16285f HTTP/1.1" 502 150 2025-12-18 00:03:19,597 - lava_callback - ERROR - Error processing callback for node 69409d99575bc060ce16285f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d99575bc060ce16285f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d99575bc060ce16285f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d99575bc060ce16285f (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:19,598 - lava_callback - INFO - Starting async job submit for node 69409dbc575bc060ce1628ab 2025-12-18 00:03:19,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:19,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dbc575bc060ce1628ab HTTP/1.1" 502 150 2025-12-18 00:03:19,605 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dbc575bc060ce1628ab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:19,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dbc575bc060ce1628ab 2025-12-18 00:03:19,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dbc575bc060ce1628ab HTTP/1.1" 502 150 2025-12-18 00:03:19,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dbc575bc060ce1628ab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:19,684 - lava_callback - INFO - Received callback for node 6940a1c2575bc060ce162f06 INFO: 172.21.0.1:40002 - "POST /node/6940a1c2575bc060ce162f06 HTTP/1.0" 202 Accepted 2025-12-18 00:03:20,291 - lava_callback - INFO - Received callback for node 6940adcd1e3c24e95681aa13 INFO: 172.21.0.1:37004 - "POST /node/6940adcd1e3c24e95681aa13 HTTP/1.0" 202 Accepted 2025-12-18 00:03:20,337 - lava_callback - INFO - Received callback for node 6940a1c4575bc060ce162f09 INFO: 172.21.0.1:37008 - "POST /node/6940a1c4575bc060ce162f09 HTTP/1.0" 202 Accepted 2025-12-18 00:03:20,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d8a575bc060ce162842 2025-12-18 00:03:20,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8a575bc060ce162842 HTTP/1.1" 502 150 2025-12-18 00:03:20,469 - lava_callback - ERROR - Error processing callback for node 69409d8a575bc060ce162842: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d8a575bc060ce162842 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d8a575bc060ce162842 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d8a575bc060ce162842 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:20,470 - lava_callback - INFO - Starting async job submit for node 69409dbf575bc060ce1628b4 2025-12-18 00:03:20,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:20,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dbf575bc060ce1628b4 HTTP/1.1" 502 150 2025-12-18 00:03:20,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dbf575bc060ce1628b4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:20,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dbf575bc060ce1628b4 2025-12-18 00:03:20,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dbf575bc060ce1628b4 HTTP/1.1" 502 150 2025-12-18 00:03:20,501 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dbf575bc060ce1628b4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:20,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a381575bc060ce1631ee 2025-12-18 00:03:20,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dab575bc060ce162879 2025-12-18 00:03:20,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a381575bc060ce1631ee HTTP/1.1" 502 150 2025-12-18 00:03:20,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a381575bc060ce1631ee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:20,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dab575bc060ce162879 HTTP/1.1" 502 150 2025-12-18 00:03:20,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dab575bc060ce162879'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:20,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db8575bc060ce1628a5 2025-12-18 00:03:20,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db8575bc060ce1628a5 HTTP/1.1" 502 150 2025-12-18 00:03:20,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db8575bc060ce1628a5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:21,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d8e575bc060ce162848 2025-12-18 00:03:21,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d8e575bc060ce162848 HTTP/1.1" 502 150 2025-12-18 00:03:21,461 - lava_callback - ERROR - Error processing callback for node 69409d8e575bc060ce162848: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d8e575bc060ce162848 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d8e575bc060ce162848 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d8e575bc060ce162848 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:21,463 - lava_callback - INFO - Starting async job submit for node 69409dc1575bc060ce1628bd 2025-12-18 00:03:21,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:21,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc1575bc060ce1628bd HTTP/1.1" 502 150 2025-12-18 00:03:21,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc1575bc060ce1628bd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:21,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc1575bc060ce1628bd 2025-12-18 00:03:21,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc1575bc060ce1628bd HTTP/1.1" 502 150 2025-12-18 00:03:21,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc1575bc060ce1628bd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:21,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dbc575bc060ce1628ab 2025-12-18 00:03:21,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dbc575bc060ce1628ab HTTP/1.1" 502 150 2025-12-18 00:03:21,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dbc575bc060ce1628ab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:21,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db4575bc060ce16289a 2025-12-18 00:03:21,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db4575bc060ce16289a HTTP/1.1" 502 150 2025-12-18 00:03:21,761 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db4575bc060ce16289a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:22,233 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db6575bc060ce16289d 2025-12-18 00:03:22,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db6575bc060ce16289d HTTP/1.1" 502 150 2025-12-18 00:03:22,235 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db6575bc060ce16289d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:22,501 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dbf575bc060ce1628b4 2025-12-18 00:03:22,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dbf575bc060ce1628b4 HTTP/1.1" 502 150 2025-12-18 00:03:22,502 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dbf575bc060ce1628b4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:22,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d9b575bc060ce162868 2025-12-18 00:03:22,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9b575bc060ce162868 HTTP/1.1" 502 150 2025-12-18 00:03:22,585 - lava_callback - ERROR - Error processing callback for node 69409d9b575bc060ce162868: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d9b575bc060ce162868 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d9b575bc060ce162868 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d9b575bc060ce162868 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:22,587 - lava_callback - INFO - Starting async job submit for node 69409dc3575bc060ce1628c2 2025-12-18 00:03:22,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:22,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc3575bc060ce1628c2 HTTP/1.1" 502 150 2025-12-18 00:03:22,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc3575bc060ce1628c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:22,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc3575bc060ce1628c2 2025-12-18 00:03:22,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc3575bc060ce1628c2 HTTP/1.1" 502 150 2025-12-18 00:03:22,628 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc3575bc060ce1628c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:23,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc1575bc060ce1628bd 2025-12-18 00:03:23,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc1575bc060ce1628bd HTTP/1.1" 502 150 2025-12-18 00:03:23,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc1575bc060ce1628bd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:23,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dad575bc060ce162884 2025-12-18 00:03:23,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dad575bc060ce162884 HTTP/1.1" 502 150 2025-12-18 00:03:23,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dad575bc060ce162884'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:23,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d9d575bc060ce16286b 2025-12-18 00:03:23,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9d575bc060ce16286b HTTP/1.1" 502 150 2025-12-18 00:03:23,525 - lava_callback - ERROR - Error processing callback for node 69409d9d575bc060ce16286b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d9d575bc060ce16286b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d9d575bc060ce16286b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d9d575bc060ce16286b (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:23,527 - lava_callback - INFO - Starting async job submit for node 69409dc5575bc060ce1628c5 2025-12-18 00:03:23,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:23,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc5575bc060ce1628c5 HTTP/1.1" 502 150 2025-12-18 00:03:23,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc5575bc060ce1628c5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:23,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc5575bc060ce1628c5 2025-12-18 00:03:23,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc5575bc060ce1628c5 HTTP/1.1" 502 150 2025-12-18 00:03:23,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc5575bc060ce1628c5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:23,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d9f575bc060ce16286e 2025-12-18 00:03:23,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d9f575bc060ce16286e HTTP/1.1" 502 150 2025-12-18 00:03:23,772 - lava_callback - ERROR - Error processing callback for node 69409d9f575bc060ce16286e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d9f575bc060ce16286e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d9f575bc060ce16286e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d9f575bc060ce16286e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:23,774 - lava_callback - INFO - Starting async job submit for node 69409dd6575bc060ce1628d8 2025-12-18 00:03:23,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:23,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd6575bc060ce1628d8 HTTP/1.1" 502 150 2025-12-18 00:03:23,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dd6575bc060ce1628d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:23,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dd6575bc060ce1628d8 2025-12-18 00:03:23,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd6575bc060ce1628d8 HTTP/1.1" 502 150 2025-12-18 00:03:23,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dd6575bc060ce1628d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:24,628 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc3575bc060ce1628c2 2025-12-18 00:03:24,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db8575bc060ce1628a5 2025-12-18 00:03:24,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db8575bc060ce1628a5 HTTP/1.1" 502 150 2025-12-18 00:03:24,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db8575bc060ce1628a5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:25,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc5575bc060ce1628c5 2025-12-18 00:03:25,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc5575bc060ce1628c5 HTTP/1.1" 502 150 2025-12-18 00:03:25,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc5575bc060ce1628c5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:25,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dbc575bc060ce1628ab 2025-12-18 00:03:25,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dbc575bc060ce1628ab HTTP/1.1" 502 150 2025-12-18 00:03:25,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dbc575bc060ce1628ab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:25,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dd6575bc060ce1628d8 2025-12-18 00:03:25,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd6575bc060ce1628d8 HTTP/1.1" 502 150 2025-12-18 00:03:25,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dd6575bc060ce1628d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:25,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db2575bc060ce162897 2025-12-18 00:03:25,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db0575bc060ce16288b 2025-12-18 00:03:25,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db2575bc060ce162897 HTTP/1.1" 502 150 2025-12-18 00:03:25,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db2575bc060ce162897'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:25,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db0575bc060ce16288b HTTP/1.1" 502 150 2025-12-18 00:03:25,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db0575bc060ce16288b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:26,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dbf575bc060ce1628b4 2025-12-18 00:03:26,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dbf575bc060ce1628b4 HTTP/1.1" 502 150 2025-12-18 00:03:26,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dbf575bc060ce1628b4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37020 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:03:26,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409da1575bc060ce162871 2025-12-18 00:03:26,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409da1575bc060ce162871 HTTP/1.1" 502 150 2025-12-18 00:03:26,694 - lava_callback - ERROR - Error processing callback for node 69409da1575bc060ce162871: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409da1575bc060ce162871 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409da1575bc060ce162871 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409da1575bc060ce162871 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:26,695 - lava_callback - INFO - Starting async job submit for node 69409dd3575bc060ce1628cd 2025-12-18 00:03:26,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:26,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd3575bc060ce1628cd HTTP/1.1" 502 150 2025-12-18 00:03:26,702 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dd3575bc060ce1628cd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:26,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dd3575bc060ce1628cd 2025-12-18 00:03:26,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd3575bc060ce1628cd HTTP/1.1" 502 150 2025-12-18 00:03:26,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dd3575bc060ce1628cd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:27,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc1575bc060ce1628bd 2025-12-18 00:03:27,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc1575bc060ce1628bd HTTP/1.1" 502 150 2025-12-18 00:03:27,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc1575bc060ce1628bd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:27,668 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409da3575bc060ce162874 2025-12-18 00:03:27,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409da3575bc060ce162874 HTTP/1.1" 502 150 2025-12-18 00:03:27,670 - lava_callback - ERROR - Error processing callback for node 69409da3575bc060ce162874: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409da3575bc060ce162874 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409da3575bc060ce162874 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409da3575bc060ce162874 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:27,671 - lava_callback - INFO - Starting async job submit for node 69409dd9575bc060ce1628db 2025-12-18 00:03:27,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:27,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd9575bc060ce1628db HTTP/1.1" 502 150 2025-12-18 00:03:27,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dd9575bc060ce1628db'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:27,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dd9575bc060ce1628db 2025-12-18 00:03:27,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd9575bc060ce1628db HTTP/1.1" 502 150 2025-12-18 00:03:27,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dd9575bc060ce1628db'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:28,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dd3575bc060ce1628cd 2025-12-18 00:03:28,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd3575bc060ce1628cd HTTP/1.1" 502 150 2025-12-18 00:03:28,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dd3575bc060ce1628cd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:29,219 - lava_callback - INFO - Received callback for node 6940a1ce575bc060ce162f1b INFO: 172.21.0.1:37022 - "POST /node/6940a1ce575bc060ce162f1b HTTP/1.0" 202 Accepted 2025-12-18 00:03:29,536 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc5575bc060ce1628c5 2025-12-18 00:03:29,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dd9575bc060ce1628db 2025-12-18 00:03:29,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd9575bc060ce1628db HTTP/1.1" 502 150 2025-12-18 00:03:29,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dd9575bc060ce1628db'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:29,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db4575bc060ce16289a 2025-12-18 00:03:29,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db4575bc060ce16289a HTTP/1.1" 502 150 2025-12-18 00:03:29,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db4575bc060ce16289a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:29,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dd6575bc060ce1628d8 2025-12-18 00:03:29,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd6575bc060ce1628d8 HTTP/1.1" 502 150 2025-12-18 00:03:29,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dd6575bc060ce1628d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:30,077 - lava_callback - INFO - Received callback for node 6940a1ca575bc060ce162f12 INFO: 172.21.0.1:57310 - "POST /node/6940a1ca575bc060ce162f12 HTTP/1.0" 202 Accepted 2025-12-18 00:03:30,235 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db6575bc060ce16289d 2025-12-18 00:03:30,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db6575bc060ce16289d HTTP/1.1" 502 150 2025-12-18 00:03:30,236 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db6575bc060ce16289d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:30,967 - lava_callback - INFO - Received callback for node 6940a1cc575bc060ce162f15 INFO: 172.21.0.1:57318 - "POST /node/6940a1cc575bc060ce162f15 HTTP/1.0" 202 Accepted 2025-12-18 00:03:31,789 - lava_callback - INFO - Received callback for node 6940a1cd575bc060ce162f18 INFO: 172.21.0.1:57334 - "POST /node/6940a1cd575bc060ce162f18 HTTP/1.0" 202 Accepted 2025-12-18 00:03:32,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dd3575bc060ce1628cd 2025-12-18 00:03:32,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd3575bc060ce1628cd HTTP/1.1" 502 150 2025-12-18 00:03:32,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dd3575bc060ce1628cd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:32,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db8575bc060ce1628a5 2025-12-18 00:03:32,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db8575bc060ce1628a5 HTTP/1.1" 502 150 2025-12-18 00:03:32,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409db8575bc060ce1628a5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:33,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dbc575bc060ce1628ab 2025-12-18 00:03:33,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dbc575bc060ce1628ab HTTP/1.1" 502 150 2025-12-18 00:03:33,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dbc575bc060ce1628ab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:33,619 - lava_callback - INFO - Received callback for node 6940a1c8575bc060ce162f0f INFO: 172.21.0.1:57342 - "POST /node/6940a1c8575bc060ce162f0f HTTP/1.0" 202 Accepted 2025-12-18 00:03:33,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dd9575bc060ce1628db 2025-12-18 00:03:33,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd9575bc060ce1628db HTTP/1.1" 502 150 2025-12-18 00:03:33,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dd9575bc060ce1628db'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:34,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dbf575bc060ce1628b4 2025-12-18 00:03:34,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dbf575bc060ce1628b4 HTTP/1.1" 502 150 2025-12-18 00:03:34,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dbf575bc060ce1628b4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:35,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc1575bc060ce1628bd 2025-12-18 00:03:35,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc1575bc060ce1628bd HTTP/1.1" 502 150 2025-12-18 00:03:35,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc1575bc060ce1628bd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57352 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:03:36,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a381575bc060ce1631ee 2025-12-18 00:03:36,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dab575bc060ce162879 2025-12-18 00:03:36,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a381575bc060ce1631ee HTTP/1.1" 502 150 2025-12-18 00:03:36,705 - lava_callback - ERROR - Error processing callback for node 6940a381575bc060ce1631ee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a381575bc060ce1631ee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a381575bc060ce1631ee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a381575bc060ce1631ee (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:36,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dab575bc060ce162879 HTTP/1.1" 502 150 2025-12-18 00:03:36,707 - lava_callback - ERROR - Error processing callback for node 69409dab575bc060ce162879: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dab575bc060ce162879 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dab575bc060ce162879 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dab575bc060ce162879 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:36,707 - lava_callback - INFO - Starting async job submit for node 69409ddb575bc060ce1628de 2025-12-18 00:03:36,709 - lava_callback - INFO - Starting async job submit for node 69409ddd575bc060ce1628e1 2025-12-18 00:03:36,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:36,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:36,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ddd575bc060ce1628e1 HTTP/1.1" 502 150 2025-12-18 00:03:36,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ddd575bc060ce1628e1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:36,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ddd575bc060ce1628e1 2025-12-18 00:03:36,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ddd575bc060ce1628e1 HTTP/1.1" 502 150 2025-12-18 00:03:36,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ddd575bc060ce1628e1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:36,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ddb575bc060ce1628de HTTP/1.1" 502 150 2025-12-18 00:03:36,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ddb575bc060ce1628de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:36,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ddb575bc060ce1628de 2025-12-18 00:03:36,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ddb575bc060ce1628de HTTP/1.1" 502 150 2025-12-18 00:03:36,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ddb575bc060ce1628de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:37,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dd6575bc060ce1628d8 2025-12-18 00:03:37,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd6575bc060ce1628d8 HTTP/1.1" 502 150 2025-12-18 00:03:37,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dd6575bc060ce1628d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:38,324 - lava_callback - INFO - Received callback for node 6940a1c7575bc060ce162f0c INFO: 172.21.0.1:57358 - "POST /node/6940a1c7575bc060ce162f0c HTTP/1.0" 202 Accepted 2025-12-18 00:03:38,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ddd575bc060ce1628e1 2025-12-18 00:03:38,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ddd575bc060ce1628e1 HTTP/1.1" 502 150 2025-12-18 00:03:38,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ddd575bc060ce1628e1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:38,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ddb575bc060ce1628de 2025-12-18 00:03:38,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ddb575bc060ce1628de HTTP/1.1" 502 150 2025-12-18 00:03:38,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ddb575bc060ce1628de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:39,015 - lava_callback - INFO - Received callback for node 6940a1d1575bc060ce162f22 INFO: 172.21.0.1:57362 - "POST /node/6940a1d1575bc060ce162f22 HTTP/1.0" 202 Accepted 2025-12-18 00:03:39,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dad575bc060ce162884 2025-12-18 00:03:39,680 - lava_callback - INFO - Received callback for node 6940a1d7575bc060ce162f28 INFO: 172.21.0.1:57374 - "POST /node/6940a1d7575bc060ce162f28 HTTP/1.0" 202 Accepted 2025-12-18 00:03:40,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc3575bc060ce1628c2 HTTP/1.1" 502 150 2025-12-18 00:03:40,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc3575bc060ce1628c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:40,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dd3575bc060ce1628cd 2025-12-18 00:03:40,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd3575bc060ce1628cd HTTP/1.1" 502 150 2025-12-18 00:03:40,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dd3575bc060ce1628cd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:41,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dd9575bc060ce1628db 2025-12-18 00:03:41,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd9575bc060ce1628db HTTP/1.1" 502 150 2025-12-18 00:03:41,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dd9575bc060ce1628db'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:41,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db2575bc060ce162897 2025-12-18 00:03:41,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db0575bc060ce16288b 2025-12-18 00:03:41,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db2575bc060ce162897 HTTP/1.1" 502 150 2025-12-18 00:03:41,836 - lava_callback - ERROR - Error processing callback for node 69409db2575bc060ce162897: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409db2575bc060ce162897 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409db2575bc060ce162897 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409db2575bc060ce162897 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:41,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db0575bc060ce16288b HTTP/1.1" 502 150 2025-12-18 00:03:41,838 - lava_callback - ERROR - Error processing callback for node 69409db0575bc060ce16288b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409db0575bc060ce16288b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409db0575bc060ce16288b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409db0575bc060ce16288b (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:41,838 - lava_callback - INFO - Starting async job submit for node 69409dea575bc060ce1628f5 2025-12-18 00:03:41,841 - lava_callback - INFO - Starting async job submit for node 69409dee575bc060ce1628fb 2025-12-18 00:03:41,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:41,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:41,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dee575bc060ce1628fb HTTP/1.1" 502 150 2025-12-18 00:03:41,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dee575bc060ce1628fb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:41,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dee575bc060ce1628fb 2025-12-18 00:03:41,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dee575bc060ce1628fb HTTP/1.1" 502 150 2025-12-18 00:03:41,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dee575bc060ce1628fb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:42,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ddd575bc060ce1628e1 2025-12-18 00:03:42,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ddd575bc060ce1628e1 HTTP/1.1" 502 150 2025-12-18 00:03:42,726 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ddd575bc060ce1628e1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:42,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ddb575bc060ce1628de 2025-12-18 00:03:42,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ddb575bc060ce1628de HTTP/1.1" 502 150 2025-12-18 00:03:42,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ddb575bc060ce1628de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:43,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dee575bc060ce1628fb 2025-12-18 00:03:43,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dee575bc060ce1628fb HTTP/1.1" 502 150 2025-12-18 00:03:43,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dee575bc060ce1628fb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:44,081 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc3575bc060ce1628c2 2025-12-18 00:03:44,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc3575bc060ce1628c2 HTTP/1.1" 502 150 2025-12-18 00:03:44,082 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc3575bc060ce1628c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:44,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc5575bc060ce1628c5 HTTP/1.1" 502 150 2025-12-18 00:03:44,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc5575bc060ce1628c5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:45,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db4575bc060ce16289a 2025-12-18 00:03:45,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db4575bc060ce16289a HTTP/1.1" 502 150 2025-12-18 00:03:45,765 - lava_callback - ERROR - Error processing callback for node 69409db4575bc060ce16289a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409db4575bc060ce16289a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409db4575bc060ce16289a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409db4575bc060ce16289a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:45,767 - lava_callback - INFO - Starting async job submit for node 69409de7575bc060ce1628e6 2025-12-18 00:03:45,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:45,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409de7575bc060ce1628e6 HTTP/1.1" 502 150 2025-12-18 00:03:45,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409de7575bc060ce1628e6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:45,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409de7575bc060ce1628e6 2025-12-18 00:03:45,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409de7575bc060ce1628e6 HTTP/1.1" 502 150 2025-12-18 00:03:45,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409de7575bc060ce1628e6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:46,236 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db6575bc060ce16289d 2025-12-18 00:03:46,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db6575bc060ce16289d HTTP/1.1" 502 150 2025-12-18 00:03:46,237 - lava_callback - ERROR - Error processing callback for node 69409db6575bc060ce16289d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409db6575bc060ce16289d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409db6575bc060ce16289d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409db6575bc060ce16289d (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:46,239 - lava_callback - INFO - Starting async job submit for node 69409df0575bc060ce1628fe 2025-12-18 00:03:46,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:46,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df0575bc060ce1628fe HTTP/1.1" 502 150 2025-12-18 00:03:46,276 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409df0575bc060ce1628fe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:46,276 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409df0575bc060ce1628fe 2025-12-18 00:03:46,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df0575bc060ce1628fe HTTP/1.1" 502 150 2025-12-18 00:03:46,276 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409df0575bc060ce1628fe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55460 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:03:47,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409de7575bc060ce1628e6 2025-12-18 00:03:47,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409de7575bc060ce1628e6 HTTP/1.1" 502 150 2025-12-18 00:03:47,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409de7575bc060ce1628e6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:47,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dee575bc060ce1628fb 2025-12-18 00:03:47,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dee575bc060ce1628fb HTTP/1.1" 502 150 2025-12-18 00:03:47,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dee575bc060ce1628fb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:48,277 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409df0575bc060ce1628fe 2025-12-18 00:03:48,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df0575bc060ce1628fe HTTP/1.1" 502 150 2025-12-18 00:03:48,278 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409df0575bc060ce1628fe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:48,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409db8575bc060ce1628a5 2025-12-18 00:03:48,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409db8575bc060ce1628a5 HTTP/1.1" 502 150 2025-12-18 00:03:48,852 - lava_callback - ERROR - Error processing callback for node 69409db8575bc060ce1628a5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409db8575bc060ce1628a5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409db8575bc060ce1628a5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409db8575bc060ce1628a5 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:48,853 - lava_callback - INFO - Starting async job submit for node 69409df3575bc060ce162901 2025-12-18 00:03:48,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:48,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df3575bc060ce162901 HTTP/1.1" 502 150 2025-12-18 00:03:48,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409df3575bc060ce162901'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:48,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409df3575bc060ce162901 2025-12-18 00:03:48,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df3575bc060ce162901 HTTP/1.1" 502 150 2025-12-18 00:03:48,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409df3575bc060ce162901'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:49,250 - lava_callback - INFO - Received callback for node 6940a1d5575bc060ce162f25 INFO: 172.21.0.1:55464 - "POST /node/6940a1d5575bc060ce162f25 HTTP/1.0" 202 Accepted 2025-12-18 00:03:49,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dbc575bc060ce1628ab 2025-12-18 00:03:49,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dbc575bc060ce1628ab HTTP/1.1" 502 150 2025-12-18 00:03:49,612 - lava_callback - ERROR - Error processing callback for node 69409dbc575bc060ce1628ab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dbc575bc060ce1628ab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dbc575bc060ce1628ab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dbc575bc060ce1628ab (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:49,614 - lava_callback - INFO - Starting async job submit for node 69409df5575bc060ce162904 2025-12-18 00:03:49,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:49,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df5575bc060ce162904 HTTP/1.1" 502 150 2025-12-18 00:03:49,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409df5575bc060ce162904'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:49,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409df5575bc060ce162904 2025-12-18 00:03:49,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df5575bc060ce162904 HTTP/1.1" 502 150 2025-12-18 00:03:49,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409df5575bc060ce162904'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:49,917 - lava_callback - INFO - Received callback for node 6940a1d9575bc060ce162f2b INFO: 172.21.0.1:52970 - "POST /node/6940a1d9575bc060ce162f2b HTTP/1.0" 202 Accepted 2025-12-18 00:03:50,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dbf575bc060ce1628b4 2025-12-18 00:03:50,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dbf575bc060ce1628b4 HTTP/1.1" 502 150 2025-12-18 00:03:50,506 - lava_callback - ERROR - Error processing callback for node 69409dbf575bc060ce1628b4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dbf575bc060ce1628b4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dbf575bc060ce1628b4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dbf575bc060ce1628b4 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:50,508 - lava_callback - INFO - Starting async job submit for node 69409dfe575bc060ce162916 2025-12-18 00:03:50,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:50,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dfe575bc060ce162916 HTTP/1.1" 502 150 2025-12-18 00:03:50,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dfe575bc060ce162916'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:50,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dfe575bc060ce162916 2025-12-18 00:03:50,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dfe575bc060ce162916 HTTP/1.1" 502 150 2025-12-18 00:03:50,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dfe575bc060ce162916'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:50,576 - lava_callback - INFO - Received callback for node 6940a1db575bc060ce162f2e INFO: 172.21.0.1:52984 - "POST /node/6940a1db575bc060ce162f2e HTTP/1.0" 202 Accepted 2025-12-18 00:03:50,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ddd575bc060ce1628e1 2025-12-18 00:03:50,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ddd575bc060ce1628e1 HTTP/1.1" 502 150 2025-12-18 00:03:50,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ddd575bc060ce1628e1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:50,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ddb575bc060ce1628de 2025-12-18 00:03:50,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ddb575bc060ce1628de HTTP/1.1" 502 150 2025-12-18 00:03:50,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ddb575bc060ce1628de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:50,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409df3575bc060ce162901 2025-12-18 00:03:50,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df3575bc060ce162901 HTTP/1.1" 502 150 2025-12-18 00:03:50,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409df3575bc060ce162901'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:51,220 - lava_callback - INFO - Received callback for node 6940a1dd575bc060ce162f31 INFO: 172.21.0.1:52998 - "POST /node/6940a1dd575bc060ce162f31 HTTP/1.0" 202 Accepted 2025-12-18 00:03:51,475 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc1575bc060ce1628bd 2025-12-18 00:03:51,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc1575bc060ce1628bd HTTP/1.1" 502 150 2025-12-18 00:03:51,476 - lava_callback - ERROR - Error processing callback for node 69409dc1575bc060ce1628bd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dc1575bc060ce1628bd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dc1575bc060ce1628bd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dc1575bc060ce1628bd (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:51,478 - lava_callback - INFO - Starting async job submit for node 69409e03575bc060ce16291e 2025-12-18 00:03:51,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:51,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e03575bc060ce16291e HTTP/1.1" 502 150 2025-12-18 00:03:51,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e03575bc060ce16291e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:51,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e03575bc060ce16291e 2025-12-18 00:03:51,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e03575bc060ce16291e HTTP/1.1" 502 150 2025-12-18 00:03:51,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e03575bc060ce16291e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:51,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409df5575bc060ce162904 2025-12-18 00:03:51,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df5575bc060ce162904 HTTP/1.1" 502 150 2025-12-18 00:03:51,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409df5575bc060ce162904'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:51,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409de7575bc060ce1628e6 2025-12-18 00:03:51,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409de7575bc060ce1628e6 HTTP/1.1" 502 150 2025-12-18 00:03:51,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409de7575bc060ce1628e6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:52,082 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc3575bc060ce1628c2 2025-12-18 00:03:52,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc3575bc060ce1628c2 HTTP/1.1" 502 150 2025-12-18 00:03:52,083 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc3575bc060ce1628c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:52,278 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409df0575bc060ce1628fe 2025-12-18 00:03:52,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df0575bc060ce1628fe HTTP/1.1" 502 150 2025-12-18 00:03:52,280 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409df0575bc060ce1628fe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:52,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dfe575bc060ce162916 2025-12-18 00:03:52,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dfe575bc060ce162916 HTTP/1.1" 502 150 2025-12-18 00:03:52,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dfe575bc060ce162916'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:52,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc5575bc060ce1628c5 2025-12-18 00:03:52,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc5575bc060ce1628c5 HTTP/1.1" 502 150 2025-12-18 00:03:52,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc5575bc060ce1628c5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:53,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e03575bc060ce16291e 2025-12-18 00:03:53,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e03575bc060ce16291e HTTP/1.1" 502 150 2025-12-18 00:03:53,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e03575bc060ce16291e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:53,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dd6575bc060ce1628d8 2025-12-18 00:03:53,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd6575bc060ce1628d8 HTTP/1.1" 502 150 2025-12-18 00:03:53,787 - lava_callback - ERROR - Error processing callback for node 69409dd6575bc060ce1628d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dd6575bc060ce1628d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dd6575bc060ce1628d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dd6575bc060ce1628d8 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:53,790 - lava_callback - INFO - Starting async job submit for node 69409dfb575bc060ce162909 2025-12-18 00:03:53,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:53,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dfb575bc060ce162909 HTTP/1.1" 502 150 2025-12-18 00:03:53,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dfb575bc060ce162909'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:53,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dfb575bc060ce162909 2025-12-18 00:03:53,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dfb575bc060ce162909 HTTP/1.1" 502 150 2025-12-18 00:03:53,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dfb575bc060ce162909'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:54,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409df3575bc060ce162901 2025-12-18 00:03:54,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df3575bc060ce162901 HTTP/1.1" 502 150 2025-12-18 00:03:54,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409df3575bc060ce162901'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:54,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dad575bc060ce162884 HTTP/1.1" 502 150 2025-12-18 00:03:54,928 - lava_callback - ERROR - Error processing callback for node 69409dad575bc060ce162884: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dad575bc060ce162884 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dad575bc060ce162884 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dad575bc060ce162884 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:54,930 - lava_callback - INFO - Starting async job submit for node 69409e00575bc060ce16291b 2025-12-18 00:03:54,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:54,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e00575bc060ce16291b HTTP/1.1" 502 150 2025-12-18 00:03:54,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e00575bc060ce16291b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:54,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e00575bc060ce16291b 2025-12-18 00:03:54,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e00575bc060ce16291b HTTP/1.1" 502 150 2025-12-18 00:03:54,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e00575bc060ce16291b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:55,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409df5575bc060ce162904 2025-12-18 00:03:55,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dfb575bc060ce162909 2025-12-18 00:03:55,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dfb575bc060ce162909 HTTP/1.1" 502 150 2025-12-18 00:03:55,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dfb575bc060ce162909'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:55,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dee575bc060ce1628fb 2025-12-18 00:03:55,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dee575bc060ce1628fb HTTP/1.1" 502 150 2025-12-18 00:03:55,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dee575bc060ce1628fb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53008 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:03:56,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dfe575bc060ce162916 2025-12-18 00:03:56,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dfe575bc060ce162916 HTTP/1.1" 502 150 2025-12-18 00:03:56,519 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dfe575bc060ce162916'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:56,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dd3575bc060ce1628cd 2025-12-18 00:03:56,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd3575bc060ce1628cd HTTP/1.1" 502 150 2025-12-18 00:03:56,709 - lava_callback - ERROR - Error processing callback for node 69409dd3575bc060ce1628cd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dd3575bc060ce1628cd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dd3575bc060ce1628cd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dd3575bc060ce1628cd (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:56,711 - lava_callback - INFO - Starting async job submit for node 6940a4f61e3c24e956819ca5 2025-12-18 00:03:56,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:56,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a4f61e3c24e956819ca5 HTTP/1.1" 502 150 2025-12-18 00:03:56,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a4f61e3c24e956819ca5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:56,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a4f61e3c24e956819ca5 2025-12-18 00:03:56,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a4f61e3c24e956819ca5 HTTP/1.1" 502 150 2025-12-18 00:03:56,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a4f61e3c24e956819ca5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:56,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e00575bc060ce16291b 2025-12-18 00:03:56,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e00575bc060ce16291b HTTP/1.1" 502 150 2025-12-18 00:03:56,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e00575bc060ce16291b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:57,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e03575bc060ce16291e 2025-12-18 00:03:57,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e03575bc060ce16291e HTTP/1.1" 502 150 2025-12-18 00:03:57,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e03575bc060ce16291e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:57,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dd9575bc060ce1628db 2025-12-18 00:03:57,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dd9575bc060ce1628db HTTP/1.1" 502 150 2025-12-18 00:03:57,710 - lava_callback - ERROR - Error processing callback for node 69409dd9575bc060ce1628db: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dd9575bc060ce1628db (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dd9575bc060ce1628db (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dd9575bc060ce1628db (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:03:57,712 - lava_callback - INFO - Starting async job submit for node 69409e05575bc060ce162921 2025-12-18 00:03:57,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:03:57,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e05575bc060ce162921 HTTP/1.1" 502 150 2025-12-18 00:03:57,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e05575bc060ce162921'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:57,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e05575bc060ce162921 2025-12-18 00:03:57,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e05575bc060ce162921 HTTP/1.1" 502 150 2025-12-18 00:03:57,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e05575bc060ce162921'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:58,357 - lava_callback - INFO - Received callback for node 6940a1df575bc060ce162f34 INFO: 172.21.0.1:53022 - "POST /node/6940a1df575bc060ce162f34 HTTP/1.0" 202 Accepted 2025-12-18 00:03:58,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a4f61e3c24e956819ca5 2025-12-18 00:03:58,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a4f61e3c24e956819ca5 HTTP/1.1" 502 150 2025-12-18 00:03:58,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a4f61e3c24e956819ca5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:59,035 - lava_callback - INFO - Received callback for node 6940a1e2575bc060ce162f37 INFO: 172.21.0.1:53032 - "POST /node/6940a1e2575bc060ce162f37 HTTP/1.0" 202 Accepted 2025-12-18 00:03:59,702 - lava_callback - INFO - Received callback for node 6940a1e4575bc060ce162f3c INFO: 172.21.0.1:37648 - "POST /node/6940a1e4575bc060ce162f3c HTTP/1.0" 202 Accepted 2025-12-18 00:03:59,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e05575bc060ce162921 2025-12-18 00:03:59,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e05575bc060ce162921 HTTP/1.1" 502 150 2025-12-18 00:03:59,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e05575bc060ce162921'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:59,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409de7575bc060ce1628e6 2025-12-18 00:03:59,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409de7575bc060ce1628e6 HTTP/1.1" 502 150 2025-12-18 00:03:59,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409de7575bc060ce1628e6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:03:59,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dfb575bc060ce162909 2025-12-18 00:03:59,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dfb575bc060ce162909 HTTP/1.1" 502 150 2025-12-18 00:03:59,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dfb575bc060ce162909'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:00,280 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409df0575bc060ce1628fe 2025-12-18 00:04:00,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df0575bc060ce1628fe HTTP/1.1" 502 150 2025-12-18 00:04:00,281 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409df0575bc060ce1628fe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:00,385 - lava_callback - INFO - Received callback for node 6940a1e7575bc060ce162f49 INFO: 172.21.0.1:37658 - "POST /node/6940a1e7575bc060ce162f49 HTTP/1.0" 202 Accepted 2025-12-18 00:04:00,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e00575bc060ce16291b 2025-12-18 00:04:00,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e00575bc060ce16291b HTTP/1.1" 502 150 2025-12-18 00:04:00,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e00575bc060ce16291b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:01,094 - lava_callback - INFO - Received callback for node 6940a1e9575bc060ce162f4c INFO: 172.21.0.1:37670 - "POST /node/6940a1e9575bc060ce162f4c HTTP/1.0" 202 Accepted 2025-12-18 00:04:02,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a4f61e3c24e956819ca5 2025-12-18 00:04:02,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a4f61e3c24e956819ca5 HTTP/1.1" 502 150 2025-12-18 00:04:02,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a4f61e3c24e956819ca5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:02,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409df3575bc060ce162901 2025-12-18 00:04:02,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df3575bc060ce162901 HTTP/1.1" 502 150 2025-12-18 00:04:02,867 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409df3575bc060ce162901'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:03,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e05575bc060ce162921 2025-12-18 00:04:03,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e05575bc060ce162921 HTTP/1.1" 502 150 2025-12-18 00:04:03,721 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e05575bc060ce162921'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37678 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:04:04,519 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dfe575bc060ce162916 2025-12-18 00:04:04,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dfe575bc060ce162916 HTTP/1.1" 502 150 2025-12-18 00:04:04,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dfe575bc060ce162916'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:05,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e03575bc060ce16291e 2025-12-18 00:04:05,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e03575bc060ce16291e HTTP/1.1" 502 150 2025-12-18 00:04:05,490 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e03575bc060ce16291e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:05,710 - lava_callback - INFO - Received callback for node 6940b2261e3c24e95681ad30 INFO: 172.21.0.1:37690 - "POST /node/6940b2261e3c24e95681ad30 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:37698 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:04:06,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ddd575bc060ce1628e1 2025-12-18 00:04:06,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ddd575bc060ce1628e1 HTTP/1.1" 502 150 2025-12-18 00:04:06,729 - lava_callback - ERROR - Error processing callback for node 69409ddd575bc060ce1628e1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ddd575bc060ce1628e1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ddd575bc060ce1628e1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ddd575bc060ce1628e1 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:06,731 - lava_callback - INFO - Starting async job submit for node 69409e07575bc060ce162924 2025-12-18 00:04:06,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:06,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e07575bc060ce162924 HTTP/1.1" 502 150 2025-12-18 00:04:06,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e07575bc060ce162924'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:06,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e07575bc060ce162924 2025-12-18 00:04:06,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e07575bc060ce162924 HTTP/1.1" 502 150 2025-12-18 00:04:06,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e07575bc060ce162924'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:06,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ddb575bc060ce1628de 2025-12-18 00:04:06,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ddb575bc060ce1628de HTTP/1.1" 502 150 2025-12-18 00:04:06,748 - lava_callback - ERROR - Error processing callback for node 69409ddb575bc060ce1628de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ddb575bc060ce1628de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ddb575bc060ce1628de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ddb575bc060ce1628de (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:06,750 - lava_callback - INFO - Starting async job submit for node 69409e09575bc060ce162927 2025-12-18 00:04:06,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:06,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e09575bc060ce162927 HTTP/1.1" 502 150 2025-12-18 00:04:06,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e09575bc060ce162927'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:06,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e09575bc060ce162927 2025-12-18 00:04:06,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e09575bc060ce162927 HTTP/1.1" 502 150 2025-12-18 00:04:06,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e09575bc060ce162927'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:07,234 - lava_callback - INFO - Received callback for node 6940b21c1e3c24e95681ad28 INFO: 172.21.0.1:37714 - "POST /node/6940b21c1e3c24e95681ad28 HTTP/1.0" 202 Accepted 2025-12-18 00:04:07,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dfb575bc060ce162909 2025-12-18 00:04:08,084 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc3575bc060ce1628c2 2025-12-18 00:04:08,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc3575bc060ce1628c2 HTTP/1.1" 502 150 2025-12-18 00:04:08,085 - lava_callback - ERROR - Error processing callback for node 69409dc3575bc060ce1628c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dc3575bc060ce1628c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dc3575bc060ce1628c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dc3575bc060ce1628c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:08,088 - lava_callback - INFO - Starting async job submit for node 6940a6c21e3c24e956819dd8 2025-12-18 00:04:08,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:08,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a6c21e3c24e956819dd8 HTTP/1.1" 502 150 2025-12-18 00:04:08,178 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a6c21e3c24e956819dd8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:08,178 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a6c21e3c24e956819dd8 2025-12-18 00:04:08,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a6c21e3c24e956819dd8 HTTP/1.1" 502 150 2025-12-18 00:04:08,179 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a6c21e3c24e956819dd8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:08,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e07575bc060ce162924 2025-12-18 00:04:08,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e07575bc060ce162924 HTTP/1.1" 502 150 2025-12-18 00:04:08,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e07575bc060ce162924'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:08,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e09575bc060ce162927 2025-12-18 00:04:08,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e09575bc060ce162927 HTTP/1.1" 502 150 2025-12-18 00:04:08,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e09575bc060ce162927'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:08,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e00575bc060ce16291b 2025-12-18 00:04:08,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e00575bc060ce16291b HTTP/1.1" 502 150 2025-12-18 00:04:08,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e00575bc060ce16291b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:08,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc5575bc060ce1628c5 2025-12-18 00:04:08,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc5575bc060ce1628c5 HTTP/1.1" 502 150 2025-12-18 00:04:08,948 - lava_callback - ERROR - Error processing callback for node 69409dc5575bc060ce1628c5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dc5575bc060ce1628c5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dc5575bc060ce1628c5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dc5575bc060ce1628c5 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:08,949 - lava_callback - INFO - Starting async job submit for node 69409e0f575bc060ce16292c 2025-12-18 00:04:08,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:08,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e0f575bc060ce16292c HTTP/1.1" 502 150 2025-12-18 00:04:08,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e0f575bc060ce16292c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:08,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e0f575bc060ce16292c 2025-12-18 00:04:08,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e0f575bc060ce16292c HTTP/1.1" 502 150 2025-12-18 00:04:08,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e0f575bc060ce16292c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:10,179 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a6c21e3c24e956819dd8 2025-12-18 00:04:10,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a6c21e3c24e956819dd8 HTTP/1.1" 502 150 2025-12-18 00:04:10,181 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a6c21e3c24e956819dd8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:10,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a4f61e3c24e956819ca5 2025-12-18 00:04:10,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a4f61e3c24e956819ca5 HTTP/1.1" 502 150 2025-12-18 00:04:10,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a4f61e3c24e956819ca5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:10,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e0f575bc060ce16292c 2025-12-18 00:04:10,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e0f575bc060ce16292c HTTP/1.1" 502 150 2025-12-18 00:04:10,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e0f575bc060ce16292c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:11,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df5575bc060ce162904 HTTP/1.1" 502 150 2025-12-18 00:04:11,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409df5575bc060ce162904'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:11,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e05575bc060ce162921 2025-12-18 00:04:11,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e05575bc060ce162921 HTTP/1.1" 502 150 2025-12-18 00:04:11,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e05575bc060ce162921'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:11,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dee575bc060ce1628fb 2025-12-18 00:04:11,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dee575bc060ce1628fb HTTP/1.1" 502 150 2025-12-18 00:04:11,863 - lava_callback - ERROR - Error processing callback for node 69409dee575bc060ce1628fb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dee575bc060ce1628fb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dee575bc060ce1628fb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dee575bc060ce1628fb (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:11,864 - lava_callback - INFO - Starting async job submit for node 6941273d459ff5fc64d2bf54 2025-12-18 00:04:11,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:11,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941273d459ff5fc64d2bf54 HTTP/1.1" 502 150 2025-12-18 00:04:11,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941273d459ff5fc64d2bf54'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:11,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941273d459ff5fc64d2bf54 2025-12-18 00:04:11,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941273d459ff5fc64d2bf54 HTTP/1.1" 502 150 2025-12-18 00:04:11,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941273d459ff5fc64d2bf54'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:12,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dea575bc060ce1628f5 HTTP/1.1" 502 150 2025-12-18 00:04:12,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dea575bc060ce1628f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:12,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dea575bc060ce1628f5 2025-12-18 00:04:12,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e07575bc060ce162924 2025-12-18 00:04:12,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e07575bc060ce162924 HTTP/1.1" 502 150 2025-12-18 00:04:12,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e07575bc060ce162924'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:12,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e09575bc060ce162927 2025-12-18 00:04:12,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e09575bc060ce162927 HTTP/1.1" 502 150 2025-12-18 00:04:12,759 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e09575bc060ce162927'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:13,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941273d459ff5fc64d2bf54 2025-12-18 00:04:13,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941273d459ff5fc64d2bf54 HTTP/1.1" 502 150 2025-12-18 00:04:13,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941273d459ff5fc64d2bf54'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:13,987 - lava_callback - INFO - Received callback for node 6940a1f3575bc060ce162f64 INFO: 172.21.0.1:38720 - "POST /node/6940a1f3575bc060ce162f64 HTTP/1.0" 202 Accepted 2025-12-18 00:04:14,181 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a6c21e3c24e956819dd8 2025-12-18 00:04:14,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a6c21e3c24e956819dd8 HTTP/1.1" 502 150 2025-12-18 00:04:14,182 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a6c21e3c24e956819dd8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:14,959 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e0f575bc060ce16292c 2025-12-18 00:04:14,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e0f575bc060ce16292c HTTP/1.1" 502 150 2025-12-18 00:04:14,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e0f575bc060ce16292c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38730 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-18 00:04:15,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409de7575bc060ce1628e6 2025-12-18 00:04:15,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409de7575bc060ce1628e6 HTTP/1.1" 502 150 2025-12-18 00:04:15,785 - lava_callback - ERROR - Error processing callback for node 69409de7575bc060ce1628e6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409de7575bc060ce1628e6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409de7575bc060ce1628e6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409de7575bc060ce1628e6 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:15,787 - lava_callback - INFO - Starting async job submit for node 6940ab631e3c24e95681a57a 2025-12-18 00:04:15,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:15,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ab631e3c24e95681a57a HTTP/1.1" 502 150 2025-12-18 00:04:15,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ab631e3c24e95681a57a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:15,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ab631e3c24e95681a57a 2025-12-18 00:04:15,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ab631e3c24e95681a57a HTTP/1.1" 502 150 2025-12-18 00:04:15,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ab631e3c24e95681a57a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:16,281 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409df0575bc060ce1628fe 2025-12-18 00:04:16,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df0575bc060ce1628fe HTTP/1.1" 502 150 2025-12-18 00:04:16,283 - lava_callback - ERROR - Error processing callback for node 69409df0575bc060ce1628fe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409df0575bc060ce1628fe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409df0575bc060ce1628fe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409df0575bc060ce1628fe (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:16,285 - lava_callback - INFO - Starting async job submit for node 6940ab891e3c24e95681a5bc 2025-12-18 00:04:16,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:16,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ab891e3c24e95681a5bc HTTP/1.1" 502 150 2025-12-18 00:04:16,296 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ab891e3c24e95681a5bc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:16,296 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ab891e3c24e95681a5bc 2025-12-18 00:04:16,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ab891e3c24e95681a5bc HTTP/1.1" 502 150 2025-12-18 00:04:16,297 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ab891e3c24e95681a5bc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38742 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:04:17,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ab631e3c24e95681a57a 2025-12-18 00:04:17,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ab631e3c24e95681a57a HTTP/1.1" 502 150 2025-12-18 00:04:17,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ab631e3c24e95681a57a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:17,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941273d459ff5fc64d2bf54 2025-12-18 00:04:17,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941273d459ff5fc64d2bf54 HTTP/1.1" 502 150 2025-12-18 00:04:17,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941273d459ff5fc64d2bf54'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:18,297 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ab891e3c24e95681a5bc 2025-12-18 00:04:18,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ab891e3c24e95681a5bc HTTP/1.1" 502 150 2025-12-18 00:04:18,298 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ab891e3c24e95681a5bc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:18,867 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409df3575bc060ce162901 2025-12-18 00:04:18,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df3575bc060ce162901 HTTP/1.1" 502 150 2025-12-18 00:04:18,868 - lava_callback - ERROR - Error processing callback for node 69409df3575bc060ce162901: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409df3575bc060ce162901 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409df3575bc060ce162901 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409df3575bc060ce162901 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:18,870 - lava_callback - INFO - Starting async job submit for node 69409e37575bc060ce162931 2025-12-18 00:04:18,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:18,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e37575bc060ce162931 HTTP/1.1" 502 150 2025-12-18 00:04:18,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e37575bc060ce162931'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:18,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e37575bc060ce162931 2025-12-18 00:04:18,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e37575bc060ce162931 HTTP/1.1" 502 150 2025-12-18 00:04:18,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e37575bc060ce162931'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:19,056 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409df5575bc060ce162904 2025-12-18 00:04:19,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df5575bc060ce162904 HTTP/1.1" 502 150 2025-12-18 00:04:19,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409df5575bc060ce162904'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:20,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dfe575bc060ce162916 2025-12-18 00:04:20,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dfe575bc060ce162916 HTTP/1.1" 502 150 2025-12-18 00:04:20,522 - lava_callback - ERROR - Error processing callback for node 69409dfe575bc060ce162916: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dfe575bc060ce162916 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dfe575bc060ce162916 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dfe575bc060ce162916 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:20,524 - lava_callback - INFO - Starting async job submit for node 6940ae341e3c24e95681aa94 2025-12-18 00:04:20,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:20,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ae341e3c24e95681aa94 HTTP/1.1" 502 150 2025-12-18 00:04:20,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ae341e3c24e95681aa94'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:20,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ae341e3c24e95681aa94 2025-12-18 00:04:20,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ae341e3c24e95681aa94 HTTP/1.1" 502 150 2025-12-18 00:04:20,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ae341e3c24e95681aa94'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:20,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e07575bc060ce162924 2025-12-18 00:04:20,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e07575bc060ce162924 HTTP/1.1" 502 150 2025-12-18 00:04:20,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e07575bc060ce162924'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:20,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e09575bc060ce162927 2025-12-18 00:04:20,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e09575bc060ce162927 HTTP/1.1" 502 150 2025-12-18 00:04:20,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e09575bc060ce162927'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:20,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e37575bc060ce162931 2025-12-18 00:04:20,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e37575bc060ce162931 HTTP/1.1" 502 150 2025-12-18 00:04:20,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e37575bc060ce162931'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:21,490 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e03575bc060ce16291e 2025-12-18 00:04:21,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e03575bc060ce16291e HTTP/1.1" 502 150 2025-12-18 00:04:21,492 - lava_callback - ERROR - Error processing callback for node 69409e03575bc060ce16291e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e03575bc060ce16291e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e03575bc060ce16291e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e03575bc060ce16291e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:21,494 - lava_callback - INFO - Starting async job submit for node 69409e3a575bc060ce16293c 2025-12-18 00:04:21,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:21,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3a575bc060ce16293c HTTP/1.1" 502 150 2025-12-18 00:04:21,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e3a575bc060ce16293c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:21,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e3a575bc060ce16293c 2025-12-18 00:04:21,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3a575bc060ce16293c HTTP/1.1" 502 150 2025-12-18 00:04:21,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e3a575bc060ce16293c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:21,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ab631e3c24e95681a57a 2025-12-18 00:04:21,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ab631e3c24e95681a57a HTTP/1.1" 502 150 2025-12-18 00:04:21,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ab631e3c24e95681a57a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:22,182 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a6c21e3c24e956819dd8 2025-12-18 00:04:22,298 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ab891e3c24e95681a5bc 2025-12-18 00:04:22,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ab891e3c24e95681a5bc HTTP/1.1" 502 150 2025-12-18 00:04:22,299 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ab891e3c24e95681a5bc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:22,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ae341e3c24e95681aa94 2025-12-18 00:04:22,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ae341e3c24e95681aa94 HTTP/1.1" 502 150 2025-12-18 00:04:22,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ae341e3c24e95681aa94'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:22,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e0f575bc060ce16292c 2025-12-18 00:04:22,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e0f575bc060ce16292c HTTP/1.1" 502 150 2025-12-18 00:04:22,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e0f575bc060ce16292c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:23,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dfb575bc060ce162909 HTTP/1.1" 502 150 2025-12-18 00:04:23,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dfb575bc060ce162909'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:23,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e3a575bc060ce16293c 2025-12-18 00:04:23,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3a575bc060ce16293c HTTP/1.1" 502 150 2025-12-18 00:04:23,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e3a575bc060ce16293c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:24,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e37575bc060ce162931 2025-12-18 00:04:24,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e37575bc060ce162931 HTTP/1.1" 502 150 2025-12-18 00:04:24,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e37575bc060ce162931'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:24,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e00575bc060ce16291b 2025-12-18 00:04:24,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e00575bc060ce16291b HTTP/1.1" 502 150 2025-12-18 00:04:24,944 - lava_callback - ERROR - Error processing callback for node 69409e00575bc060ce16291b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e00575bc060ce16291b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e00575bc060ce16291b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e00575bc060ce16291b (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:24,946 - lava_callback - INFO - Starting async job submit for node 69409e3c575bc060ce16293f 2025-12-18 00:04:24,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:24,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3c575bc060ce16293f HTTP/1.1" 502 150 2025-12-18 00:04:24,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e3c575bc060ce16293f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:24,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e3c575bc060ce16293f 2025-12-18 00:04:24,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3c575bc060ce16293f HTTP/1.1" 502 150 2025-12-18 00:04:24,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e3c575bc060ce16293f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:25,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941273d459ff5fc64d2bf54 2025-12-18 00:04:25,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941273d459ff5fc64d2bf54 HTTP/1.1" 502 150 2025-12-18 00:04:25,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941273d459ff5fc64d2bf54'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36658 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:04:26,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ae341e3c24e95681aa94 2025-12-18 00:04:26,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ae341e3c24e95681aa94 HTTP/1.1" 502 150 2025-12-18 00:04:26,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ae341e3c24e95681aa94'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:26,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a4f61e3c24e956819ca5 2025-12-18 00:04:26,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a4f61e3c24e956819ca5 HTTP/1.1" 502 150 2025-12-18 00:04:26,765 - lava_callback - ERROR - Error processing callback for node 6940a4f61e3c24e956819ca5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a4f61e3c24e956819ca5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a4f61e3c24e956819ca5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a4f61e3c24e956819ca5 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:26,767 - lava_callback - INFO - Starting async job submit for node 69409e3e575bc060ce162946 2025-12-18 00:04:26,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:26,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3e575bc060ce162946 HTTP/1.1" 502 150 2025-12-18 00:04:26,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e3e575bc060ce162946'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:26,773 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e3e575bc060ce162946 2025-12-18 00:04:26,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3e575bc060ce162946 HTTP/1.1" 502 150 2025-12-18 00:04:26,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e3e575bc060ce162946'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:26,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e3c575bc060ce16293f 2025-12-18 00:04:26,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3c575bc060ce16293f HTTP/1.1" 502 150 2025-12-18 00:04:26,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e3c575bc060ce16293f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:27,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e3a575bc060ce16293c 2025-12-18 00:04:27,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3a575bc060ce16293c HTTP/1.1" 502 150 2025-12-18 00:04:27,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e3a575bc060ce16293c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:27,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e05575bc060ce162921 2025-12-18 00:04:27,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e05575bc060ce162921 HTTP/1.1" 502 150 2025-12-18 00:04:27,725 - lava_callback - ERROR - Error processing callback for node 69409e05575bc060ce162921: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e05575bc060ce162921 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e05575bc060ce162921 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e05575bc060ce162921 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:27,727 - lava_callback - INFO - Starting async job submit for node 69409e40575bc060ce162949 2025-12-18 00:04:27,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:27,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e40575bc060ce162949 HTTP/1.1" 502 150 2025-12-18 00:04:27,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e40575bc060ce162949'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:27,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e40575bc060ce162949 2025-12-18 00:04:27,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e40575bc060ce162949 HTTP/1.1" 502 150 2025-12-18 00:04:27,769 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e40575bc060ce162949'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:28,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e3e575bc060ce162946 2025-12-18 00:04:28,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3e575bc060ce162946 HTTP/1.1" 502 150 2025-12-18 00:04:28,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e3e575bc060ce162946'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:29,285 - lava_callback - INFO - Received callback for node 6940a1f8575bc060ce162f6c INFO: 172.21.0.1:36664 - "POST /node/6940a1f8575bc060ce162f6c HTTP/1.0" 202 Accepted 2025-12-18 00:04:29,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e40575bc060ce162949 2025-12-18 00:04:29,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e40575bc060ce162949 HTTP/1.1" 502 150 2025-12-18 00:04:29,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e40575bc060ce162949'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:29,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ab631e3c24e95681a57a 2025-12-18 00:04:29,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ab631e3c24e95681a57a HTTP/1.1" 502 150 2025-12-18 00:04:29,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ab631e3c24e95681a57a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:29,951 - lava_callback - INFO - Received callback for node 6940a1fa575bc060ce162f7a INFO: 172.21.0.1:37398 - "POST /node/6940a1fa575bc060ce162f7a HTTP/1.0" 202 Accepted 2025-12-18 00:04:30,300 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ab891e3c24e95681a5bc 2025-12-18 00:04:30,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ab891e3c24e95681a5bc HTTP/1.1" 502 150 2025-12-18 00:04:30,301 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ab891e3c24e95681a5bc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:30,597 - lava_callback - INFO - Received callback for node 6940a1fd575bc060ce162f82 INFO: 172.21.0.1:37404 - "POST /node/6940a1fd575bc060ce162f82 HTTP/1.0" 202 Accepted 2025-12-18 00:04:30,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e3c575bc060ce16293f 2025-12-18 00:04:30,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3c575bc060ce16293f HTTP/1.1" 502 150 2025-12-18 00:04:30,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e3c575bc060ce16293f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:31,244 - lava_callback - INFO - Received callback for node 6940a1ff575bc060ce162f85 INFO: 172.21.0.1:37414 - "POST /node/6940a1ff575bc060ce162f85 HTTP/1.0" 202 Accepted 2025-12-18 00:04:31,902 - lava_callback - INFO - Received callback for node 6940a201575bc060ce162f88 INFO: 172.21.0.1:37418 - "POST /node/6940a201575bc060ce162f88 HTTP/1.0" 202 Accepted 2025-12-18 00:04:32,553 - lava_callback - INFO - Received callback for node 6940a203575bc060ce162f8e INFO: 172.21.0.1:37424 - "POST /node/6940a203575bc060ce162f8e HTTP/1.0" 202 Accepted 2025-12-18 00:04:32,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e3e575bc060ce162946 2025-12-18 00:04:32,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3e575bc060ce162946 HTTP/1.1" 502 150 2025-12-18 00:04:32,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e3e575bc060ce162946'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:32,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e37575bc060ce162931 2025-12-18 00:04:32,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e37575bc060ce162931 HTTP/1.1" 502 150 2025-12-18 00:04:32,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e37575bc060ce162931'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:33,633 - lava_callback - INFO - Received callback for node 69414b43459ff5fc64d2cf85 INFO: 172.21.0.1:37428 - "POST /node/69414b43459ff5fc64d2cf85 HTTP/1.0" 202 Accepted 2025-12-18 00:04:33,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e40575bc060ce162949 2025-12-18 00:04:33,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e40575bc060ce162949 HTTP/1.1" 502 150 2025-12-18 00:04:33,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e40575bc060ce162949'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:34,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ae341e3c24e95681aa94 2025-12-18 00:04:35,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409df5575bc060ce162904 2025-12-18 00:04:35,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409df5575bc060ce162904 HTTP/1.1" 502 150 2025-12-18 00:04:35,059 - lava_callback - ERROR - Error processing callback for node 69409df5575bc060ce162904: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409df5575bc060ce162904 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409df5575bc060ce162904 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409df5575bc060ce162904 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:35,061 - lava_callback - INFO - Starting async job submit for node 69409e4e575bc060ce162953 2025-12-18 00:04:35,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:35,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e4e575bc060ce162953 HTTP/1.1" 502 150 2025-12-18 00:04:35,068 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e4e575bc060ce162953'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:35,068 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e4e575bc060ce162953 2025-12-18 00:04:35,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e4e575bc060ce162953 HTTP/1.1" 502 150 2025-12-18 00:04:35,069 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e4e575bc060ce162953'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:35,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e3a575bc060ce16293c 2025-12-18 00:04:35,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3a575bc060ce16293c HTTP/1.1" 502 150 2025-12-18 00:04:35,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e3a575bc060ce16293c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37442 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:04:36,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e07575bc060ce162924 2025-12-18 00:04:36,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e07575bc060ce162924 HTTP/1.1" 502 150 2025-12-18 00:04:36,745 - lava_callback - ERROR - Error processing callback for node 69409e07575bc060ce162924: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e07575bc060ce162924 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e07575bc060ce162924 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e07575bc060ce162924 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:36,747 - lava_callback - INFO - Starting async job submit for node 6940a7181e3c24e956819e2c 2025-12-18 00:04:36,765 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e09575bc060ce162927 2025-12-18 00:04:36,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e09575bc060ce162927 HTTP/1.1" 502 150 2025-12-18 00:04:36,776 - lava_callback - ERROR - Error processing callback for node 69409e09575bc060ce162927: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e09575bc060ce162927 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e09575bc060ce162927 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e09575bc060ce162927 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:36,809 - lava_callback - INFO - Starting async job submit for node 69409e4b575bc060ce16294e 2025-12-18 00:04:36,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:36,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:36,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e4b575bc060ce16294e HTTP/1.1" 502 150 2025-12-18 00:04:36,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e4b575bc060ce16294e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:36,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e4b575bc060ce16294e 2025-12-18 00:04:36,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e4b575bc060ce16294e HTTP/1.1" 502 150 2025-12-18 00:04:36,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e4b575bc060ce16294e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:36,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a7181e3c24e956819e2c HTTP/1.1" 502 150 2025-12-18 00:04:36,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a7181e3c24e956819e2c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:36,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a7181e3c24e956819e2c 2025-12-18 00:04:36,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a7181e3c24e956819e2c HTTP/1.1" 502 150 2025-12-18 00:04:36,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a7181e3c24e956819e2c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:37,069 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e4e575bc060ce162953 2025-12-18 00:04:37,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e4e575bc060ce162953 HTTP/1.1" 502 150 2025-12-18 00:04:37,070 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e4e575bc060ce162953'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:37,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a6c21e3c24e956819dd8 HTTP/1.1" 502 150 2025-12-18 00:04:37,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a6c21e3c24e956819dd8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:38,394 - lava_callback - INFO - Received callback for node 6940a20f575bc060ce162fad INFO: 172.21.0.1:37452 - "POST /node/6940a20f575bc060ce162fad HTTP/1.0" 202 Accepted 2025-12-18 00:04:38,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e4b575bc060ce16294e 2025-12-18 00:04:38,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e4b575bc060ce16294e HTTP/1.1" 502 150 2025-12-18 00:04:38,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e4b575bc060ce16294e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:38,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a7181e3c24e956819e2c 2025-12-18 00:04:38,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a7181e3c24e956819e2c HTTP/1.1" 502 150 2025-12-18 00:04:38,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a7181e3c24e956819e2c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:38,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e0f575bc060ce16292c 2025-12-18 00:04:38,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e0f575bc060ce16292c HTTP/1.1" 502 150 2025-12-18 00:04:38,965 - lava_callback - ERROR - Error processing callback for node 69409e0f575bc060ce16292c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e0f575bc060ce16292c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e0f575bc060ce16292c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e0f575bc060ce16292c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:38,966 - lava_callback - INFO - Starting async job submit for node 6940ae391e3c24e95681aac4 2025-12-18 00:04:38,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:38,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ae391e3c24e95681aac4 HTTP/1.1" 502 150 2025-12-18 00:04:38,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ae391e3c24e95681aac4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:38,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ae391e3c24e95681aac4 2025-12-18 00:04:38,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ae391e3c24e95681aac4 HTTP/1.1" 502 150 2025-12-18 00:04:38,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ae391e3c24e95681aac4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:38,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e3c575bc060ce16293f 2025-12-18 00:04:38,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3c575bc060ce16293f HTTP/1.1" 502 150 2025-12-18 00:04:38,986 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e3c575bc060ce16293f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:39,048 - lava_callback - INFO - Received callback for node 6940a214575bc060ce162fb6 INFO: 172.21.0.1:37456 - "POST /node/6940a214575bc060ce162fb6 HTTP/1.0" 202 Accepted 2025-12-18 00:04:39,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dfb575bc060ce162909 2025-12-18 00:04:39,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dfb575bc060ce162909 HTTP/1.1" 502 150 2025-12-18 00:04:39,090 - lava_callback - ERROR - Error processing callback for node 69409dfb575bc060ce162909: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dfb575bc060ce162909 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dfb575bc060ce162909 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dfb575bc060ce162909 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:39,092 - lava_callback - INFO - Starting async job submit for node 69409e62575bc060ce16296a 2025-12-18 00:04:39,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:39,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e62575bc060ce16296a HTTP/1.1" 502 150 2025-12-18 00:04:39,098 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e62575bc060ce16296a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:39,099 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e62575bc060ce16296a 2025-12-18 00:04:39,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e62575bc060ce16296a HTTP/1.1" 502 150 2025-12-18 00:04:39,099 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e62575bc060ce16296a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:40,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e3e575bc060ce162946 2025-12-18 00:04:40,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3e575bc060ce162946 HTTP/1.1" 502 150 2025-12-18 00:04:40,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e3e575bc060ce162946'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:40,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ae391e3c24e95681aac4 2025-12-18 00:04:40,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ae391e3c24e95681aac4 HTTP/1.1" 502 150 2025-12-18 00:04:40,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ae391e3c24e95681aac4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:41,070 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e4e575bc060ce162953 2025-12-18 00:04:41,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e4e575bc060ce162953 HTTP/1.1" 502 150 2025-12-18 00:04:41,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e4e575bc060ce162953'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:41,100 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e62575bc060ce16296a 2025-12-18 00:04:41,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e62575bc060ce16296a HTTP/1.1" 502 150 2025-12-18 00:04:41,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e62575bc060ce16296a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:41,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e40575bc060ce162949 2025-12-18 00:04:41,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e40575bc060ce162949 HTTP/1.1" 502 150 2025-12-18 00:04:41,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e40575bc060ce162949'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:41,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941273d459ff5fc64d2bf54 2025-12-18 00:04:41,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941273d459ff5fc64d2bf54 HTTP/1.1" 502 150 2025-12-18 00:04:41,898 - lava_callback - ERROR - Error processing callback for node 6941273d459ff5fc64d2bf54: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941273d459ff5fc64d2bf54 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941273d459ff5fc64d2bf54 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941273d459ff5fc64d2bf54 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:41,900 - lava_callback - INFO - Starting async job submit for node 69409e67575bc060ce162979 2025-12-18 00:04:41,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:41,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e67575bc060ce162979 HTTP/1.1" 502 150 2025-12-18 00:04:41,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e67575bc060ce162979'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:41,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e67575bc060ce162979 2025-12-18 00:04:41,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e67575bc060ce162979 HTTP/1.1" 502 150 2025-12-18 00:04:41,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e67575bc060ce162979'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:42,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e4b575bc060ce16294e 2025-12-18 00:04:42,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e4b575bc060ce16294e HTTP/1.1" 502 150 2025-12-18 00:04:42,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e4b575bc060ce16294e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:42,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a7181e3c24e956819e2c 2025-12-18 00:04:42,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a7181e3c24e956819e2c HTTP/1.1" 502 150 2025-12-18 00:04:42,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a7181e3c24e956819e2c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:43,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dea575bc060ce1628f5 HTTP/1.1" 502 150 2025-12-18 00:04:43,312 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dea575bc060ce1628f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:43,908 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e67575bc060ce162979 2025-12-18 00:04:43,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e67575bc060ce162979 HTTP/1.1" 502 150 2025-12-18 00:04:43,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e67575bc060ce162979'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:44,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ae391e3c24e95681aac4 2025-12-18 00:04:44,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ae391e3c24e95681aac4 HTTP/1.1" 502 150 2025-12-18 00:04:44,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ae391e3c24e95681aac4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:45,101 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e62575bc060ce16296a 2025-12-18 00:04:45,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e62575bc060ce16296a HTTP/1.1" 502 150 2025-12-18 00:04:45,102 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e62575bc060ce16296a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:45,312 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dea575bc060ce1628f5 2025-12-18 00:04:45,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ab631e3c24e95681a57a 2025-12-18 00:04:45,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ab631e3c24e95681a57a HTTP/1.1" 502 150 2025-12-18 00:04:45,803 - lava_callback - ERROR - Error processing callback for node 6940ab631e3c24e95681a57a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ab631e3c24e95681a57a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ab631e3c24e95681a57a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ab631e3c24e95681a57a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:45,804 - lava_callback - INFO - Starting async job submit for node 69409e5f575bc060ce16295f 2025-12-18 00:04:45,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:45,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e5f575bc060ce16295f HTTP/1.1" 502 150 2025-12-18 00:04:45,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e5f575bc060ce16295f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:45,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e5f575bc060ce16295f 2025-12-18 00:04:45,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e5f575bc060ce16295f HTTP/1.1" 502 150 2025-12-18 00:04:45,835 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e5f575bc060ce16295f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:46,301 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ab891e3c24e95681a5bc 2025-12-18 00:04:46,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ab891e3c24e95681a5bc HTTP/1.1" 502 150 2025-12-18 00:04:46,303 - lava_callback - ERROR - Error processing callback for node 6940ab891e3c24e95681a5bc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ab891e3c24e95681a5bc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ab891e3c24e95681a5bc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ab891e3c24e95681a5bc (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:46,306 - lava_callback - INFO - Starting async job submit for node 69409e65575bc060ce162971 2025-12-18 00:04:46,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:46,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e65575bc060ce162971 HTTP/1.1" 502 150 2025-12-18 00:04:46,325 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e65575bc060ce162971'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:46,325 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e65575bc060ce162971 2025-12-18 00:04:46,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e65575bc060ce162971 HTTP/1.1" 502 150 2025-12-18 00:04:46,326 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e65575bc060ce162971'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58104 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:04:47,835 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e5f575bc060ce16295f 2025-12-18 00:04:47,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e5f575bc060ce16295f HTTP/1.1" 502 150 2025-12-18 00:04:47,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e5f575bc060ce16295f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:47,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e67575bc060ce162979 2025-12-18 00:04:47,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e67575bc060ce162979 HTTP/1.1" 502 150 2025-12-18 00:04:47,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e67575bc060ce162979'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:48,326 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e65575bc060ce162971 2025-12-18 00:04:48,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e37575bc060ce162931 2025-12-18 00:04:48,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e37575bc060ce162931 HTTP/1.1" 502 150 2025-12-18 00:04:48,891 - lava_callback - ERROR - Error processing callback for node 69409e37575bc060ce162931: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e37575bc060ce162931 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e37575bc060ce162931 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e37575bc060ce162931 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:48,893 - lava_callback - INFO - Starting async job submit for node 69409e69575bc060ce16297c 2025-12-18 00:04:48,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:48,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e69575bc060ce16297c HTTP/1.1" 502 150 2025-12-18 00:04:48,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e69575bc060ce16297c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:48,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e69575bc060ce16297c 2025-12-18 00:04:48,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e69575bc060ce16297c HTTP/1.1" 502 150 2025-12-18 00:04:48,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e69575bc060ce16297c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:49,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e4e575bc060ce162953 2025-12-18 00:04:49,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e4e575bc060ce162953 HTTP/1.1" 502 150 2025-12-18 00:04:49,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e4e575bc060ce162953'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:49,292 - lava_callback - INFO - Received callback for node 6940a20c575bc060ce162fa1 INFO: 172.21.0.1:58108 - "POST /node/6940a20c575bc060ce162fa1 HTTP/1.0" 202 Accepted 2025-12-18 00:04:49,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ae341e3c24e95681aa94 HTTP/1.1" 502 150 2025-12-18 00:04:49,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ae341e3c24e95681aa94'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:50,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e4b575bc060ce16294e 2025-12-18 00:04:50,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e4b575bc060ce16294e HTTP/1.1" 502 150 2025-12-18 00:04:50,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e4b575bc060ce16294e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:50,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a7181e3c24e956819e2c 2025-12-18 00:04:50,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a7181e3c24e956819e2c HTTP/1.1" 502 150 2025-12-18 00:04:50,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a7181e3c24e956819e2c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:50,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e69575bc060ce16297c 2025-12-18 00:04:50,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e69575bc060ce16297c HTTP/1.1" 502 150 2025-12-18 00:04:50,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e69575bc060ce16297c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:51,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e3a575bc060ce16293c 2025-12-18 00:04:51,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3a575bc060ce16293c HTTP/1.1" 502 150 2025-12-18 00:04:51,527 - lava_callback - ERROR - Error processing callback for node 69409e3a575bc060ce16293c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e3a575bc060ce16293c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e3a575bc060ce16293c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e3a575bc060ce16293c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:51,529 - lava_callback - INFO - Starting async job submit for node 69409e79575bc060ce162997 2025-12-18 00:04:51,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:51,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e79575bc060ce162997 HTTP/1.1" 502 150 2025-12-18 00:04:51,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e79575bc060ce162997'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:51,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e79575bc060ce162997 2025-12-18 00:04:51,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e79575bc060ce162997 HTTP/1.1" 502 150 2025-12-18 00:04:51,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e79575bc060ce162997'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:51,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e5f575bc060ce16295f 2025-12-18 00:04:51,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e5f575bc060ce16295f HTTP/1.1" 502 150 2025-12-18 00:04:51,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e5f575bc060ce16295f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:52,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ae391e3c24e95681aac4 2025-12-18 00:04:52,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ae391e3c24e95681aac4 HTTP/1.1" 502 150 2025-12-18 00:04:52,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940ae391e3c24e95681aac4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:53,102 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e62575bc060ce16296a 2025-12-18 00:04:53,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e62575bc060ce16296a HTTP/1.1" 502 150 2025-12-18 00:04:53,104 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e62575bc060ce16296a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:53,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e79575bc060ce162997 2025-12-18 00:04:53,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e79575bc060ce162997 HTTP/1.1" 502 150 2025-12-18 00:04:53,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e79575bc060ce162997'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:53,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a6c21e3c24e956819dd8 2025-12-18 00:04:53,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a6c21e3c24e956819dd8 HTTP/1.1" 502 150 2025-12-18 00:04:53,682 - lava_callback - ERROR - Error processing callback for node 6940a6c21e3c24e956819dd8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a6c21e3c24e956819dd8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a6c21e3c24e956819dd8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a6c21e3c24e956819dd8 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:53,683 - lava_callback - INFO - Starting async job submit for node 69409e7c575bc060ce16299a 2025-12-18 00:04:53,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:53,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e7c575bc060ce16299a HTTP/1.1" 502 150 2025-12-18 00:04:53,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e7c575bc060ce16299a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:53,691 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e7c575bc060ce16299a 2025-12-18 00:04:53,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e7c575bc060ce16299a HTTP/1.1" 502 150 2025-12-18 00:04:53,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e7c575bc060ce16299a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:54,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e69575bc060ce16297c 2025-12-18 00:04:54,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e69575bc060ce16297c HTTP/1.1" 502 150 2025-12-18 00:04:54,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e69575bc060ce16297c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:54,986 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e3c575bc060ce16293f 2025-12-18 00:04:54,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3c575bc060ce16293f HTTP/1.1" 502 150 2025-12-18 00:04:54,987 - lava_callback - ERROR - Error processing callback for node 69409e3c575bc060ce16293f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e3c575bc060ce16293f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e3c575bc060ce16293f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e3c575bc060ce16293f (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:54,989 - lava_callback - INFO - Starting async job submit for node 69409e84575bc060ce1629a0 2025-12-18 00:04:54,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:54,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e84575bc060ce1629a0 HTTP/1.1" 502 150 2025-12-18 00:04:54,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e84575bc060ce1629a0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:54,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e84575bc060ce1629a0 2025-12-18 00:04:54,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e84575bc060ce1629a0 HTTP/1.1" 502 150 2025-12-18 00:04:54,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e84575bc060ce1629a0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:55,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e7c575bc060ce16299a 2025-12-18 00:04:55,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e7c575bc060ce16299a HTTP/1.1" 502 150 2025-12-18 00:04:55,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e7c575bc060ce16299a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:55,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e67575bc060ce162979 2025-12-18 00:04:55,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e67575bc060ce162979 HTTP/1.1" 502 150 2025-12-18 00:04:55,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e67575bc060ce162979'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43676 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:04:56,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e3e575bc060ce162946 2025-12-18 00:04:56,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e3e575bc060ce162946 HTTP/1.1" 502 150 2025-12-18 00:04:56,780 - lava_callback - ERROR - Error processing callback for node 69409e3e575bc060ce162946: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e3e575bc060ce162946 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e3e575bc060ce162946 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e3e575bc060ce162946 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:56,782 - lava_callback - INFO - Starting async job submit for node 69409e86575bc060ce1629a7 2025-12-18 00:04:56,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:56,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e86575bc060ce1629a7 HTTP/1.1" 502 150 2025-12-18 00:04:56,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e86575bc060ce1629a7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:56,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e86575bc060ce1629a7 2025-12-18 00:04:56,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e86575bc060ce1629a7 HTTP/1.1" 502 150 2025-12-18 00:04:56,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e86575bc060ce1629a7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:56,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e84575bc060ce1629a0 2025-12-18 00:04:56,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e84575bc060ce1629a0 HTTP/1.1" 502 150 2025-12-18 00:04:56,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e84575bc060ce1629a0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:57,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e79575bc060ce162997 2025-12-18 00:04:57,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e40575bc060ce162949 2025-12-18 00:04:57,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e40575bc060ce162949 HTTP/1.1" 502 150 2025-12-18 00:04:57,775 - lava_callback - ERROR - Error processing callback for node 69409e40575bc060ce162949: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e40575bc060ce162949 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e40575bc060ce162949 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e40575bc060ce162949 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:04:57,777 - lava_callback - INFO - Starting async job submit for node 69409e8b575bc060ce1629bc 2025-12-18 00:04:57,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:04:57,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8b575bc060ce1629bc HTTP/1.1" 502 150 2025-12-18 00:04:57,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e8b575bc060ce1629bc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:57,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e8b575bc060ce1629bc 2025-12-18 00:04:57,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8b575bc060ce1629bc HTTP/1.1" 502 150 2025-12-18 00:04:57,785 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e8b575bc060ce1629bc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:58,400 - lava_callback - INFO - Received callback for node 6940a224575bc060ce162fd2 INFO: 172.21.0.1:43682 - "POST /node/6940a224575bc060ce162fd2 HTTP/1.0" 202 Accepted 2025-12-18 00:04:58,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e86575bc060ce1629a7 2025-12-18 00:04:58,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e86575bc060ce1629a7 HTTP/1.1" 502 150 2025-12-18 00:04:58,826 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e86575bc060ce1629a7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:59,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e7c575bc060ce16299a 2025-12-18 00:04:59,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e7c575bc060ce16299a HTTP/1.1" 502 150 2025-12-18 00:04:59,694 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e7c575bc060ce16299a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:59,785 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e8b575bc060ce1629bc 2025-12-18 00:04:59,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8b575bc060ce1629bc HTTP/1.1" 502 150 2025-12-18 00:04:59,786 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e8b575bc060ce1629bc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:04:59,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e5f575bc060ce16295f 2025-12-18 00:04:59,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e5f575bc060ce16295f HTTP/1.1" 502 150 2025-12-18 00:04:59,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e5f575bc060ce16295f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:00,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e84575bc060ce1629a0 2025-12-18 00:05:01,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e84575bc060ce1629a0 HTTP/1.1" 502 150 2025-12-18 00:05:01,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e84575bc060ce1629a0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:02,826 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e86575bc060ce1629a7 2025-12-18 00:05:02,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e86575bc060ce1629a7 HTTP/1.1" 502 150 2025-12-18 00:05:02,828 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e86575bc060ce1629a7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:02,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e69575bc060ce16297c 2025-12-18 00:05:02,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e69575bc060ce16297c HTTP/1.1" 502 150 2025-12-18 00:05:02,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e69575bc060ce16297c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:03,786 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e8b575bc060ce1629bc 2025-12-18 00:05:03,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8b575bc060ce1629bc HTTP/1.1" 502 150 2025-12-18 00:05:03,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e8b575bc060ce1629bc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:03,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e65575bc060ce162971 HTTP/1.1" 502 150 2025-12-18 00:05:03,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e65575bc060ce162971'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37444 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:05:05,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e4e575bc060ce162953 2025-12-18 00:05:05,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e4e575bc060ce162953 HTTP/1.1" 502 150 2025-12-18 00:05:05,075 - lava_callback - ERROR - Error processing callback for node 69409e4e575bc060ce162953: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e4e575bc060ce162953 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e4e575bc060ce162953 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e4e575bc060ce162953 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:05,076 - lava_callback - INFO - Starting async job submit for node 6940a5311e3c24e956819cc0 2025-12-18 00:05:05,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:05,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a5311e3c24e956819cc0 HTTP/1.1" 502 150 2025-12-18 00:05:05,102 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a5311e3c24e956819cc0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:05,102 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a5311e3c24e956819cc0 2025-12-18 00:05:05,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a5311e3c24e956819cc0 HTTP/1.1" 502 150 2025-12-18 00:05:05,103 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a5311e3c24e956819cc0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:05,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ae341e3c24e95681aa94 2025-12-18 00:05:05,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ae341e3c24e95681aa94 HTTP/1.1" 502 150 2025-12-18 00:05:05,970 - lava_callback - ERROR - Error processing callback for node 6940ae341e3c24e95681aa94: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ae341e3c24e95681aa94 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ae341e3c24e95681aa94 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ae341e3c24e95681aa94 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:05,972 - lava_callback - INFO - Starting async job submit for node 69409e82575bc060ce16299d 2025-12-18 00:05:05,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:05,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e82575bc060ce16299d HTTP/1.1" 502 150 2025-12-18 00:05:05,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e82575bc060ce16299d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:05,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e82575bc060ce16299d 2025-12-18 00:05:05,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e82575bc060ce16299d HTTP/1.1" 502 150 2025-12-18 00:05:05,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e82575bc060ce16299d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:37452 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:05:06,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e4b575bc060ce16294e 2025-12-18 00:05:06,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e4b575bc060ce16294e HTTP/1.1" 502 150 2025-12-18 00:05:06,829 - lava_callback - ERROR - Error processing callback for node 69409e4b575bc060ce16294e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e4b575bc060ce16294e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e4b575bc060ce16294e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e4b575bc060ce16294e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:06,831 - lava_callback - INFO - Starting async job submit for node 69409e8e575bc060ce1629c2 2025-12-18 00:05:06,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:06,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8e575bc060ce1629c2 HTTP/1.1" 502 150 2025-12-18 00:05:06,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e8e575bc060ce1629c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:06,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e8e575bc060ce1629c2 2025-12-18 00:05:06,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8e575bc060ce1629c2 HTTP/1.1" 502 150 2025-12-18 00:05:06,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e8e575bc060ce1629c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:06,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a7181e3c24e956819e2c 2025-12-18 00:05:06,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a7181e3c24e956819e2c HTTP/1.1" 502 150 2025-12-18 00:05:06,848 - lava_callback - ERROR - Error processing callback for node 6940a7181e3c24e956819e2c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a7181e3c24e956819e2c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a7181e3c24e956819e2c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a7181e3c24e956819e2c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:06,850 - lava_callback - INFO - Starting async job submit for node 69409e89575bc060ce1629b2 2025-12-18 00:05:06,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:06,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e89575bc060ce1629b2 HTTP/1.1" 502 150 2025-12-18 00:05:06,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e89575bc060ce1629b2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:06,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e89575bc060ce1629b2 2025-12-18 00:05:06,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e89575bc060ce1629b2 HTTP/1.1" 502 150 2025-12-18 00:05:06,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e89575bc060ce1629b2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:07,103 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a5311e3c24e956819cc0 2025-12-18 00:05:07,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a5311e3c24e956819cc0 HTTP/1.1" 502 150 2025-12-18 00:05:07,104 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a5311e3c24e956819cc0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:07,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e7c575bc060ce16299a 2025-12-18 00:05:07,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e7c575bc060ce16299a HTTP/1.1" 502 150 2025-12-18 00:05:07,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e7c575bc060ce16299a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:07,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e65575bc060ce162971 2025-12-18 00:05:07,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e65575bc060ce162971 HTTP/1.1" 502 150 2025-12-18 00:05:07,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e65575bc060ce162971'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:07,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e82575bc060ce16299d 2025-12-18 00:05:07,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e82575bc060ce16299d HTTP/1.1" 502 150 2025-12-18 00:05:07,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e82575bc060ce16299d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:08,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e8e575bc060ce1629c2 2025-12-18 00:05:08,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8e575bc060ce1629c2 HTTP/1.1" 502 150 2025-12-18 00:05:08,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e8e575bc060ce1629c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:08,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e89575bc060ce1629b2 2025-12-18 00:05:08,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e89575bc060ce1629b2 HTTP/1.1" 502 150 2025-12-18 00:05:08,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e89575bc060ce1629b2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:08,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940ae391e3c24e95681aac4 2025-12-18 00:05:08,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940ae391e3c24e95681aac4 HTTP/1.1" 502 150 2025-12-18 00:05:08,982 - lava_callback - ERROR - Error processing callback for node 6940ae391e3c24e95681aac4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ae391e3c24e95681aac4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ae391e3c24e95681aac4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940ae391e3c24e95681aac4 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:08,984 - lava_callback - INFO - Starting async job submit for node 69409e8c575bc060ce1629bf 2025-12-18 00:05:08,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:08,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8c575bc060ce1629bf HTTP/1.1" 502 150 2025-12-18 00:05:08,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e8c575bc060ce1629bf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:08,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e8c575bc060ce1629bf 2025-12-18 00:05:08,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8c575bc060ce1629bf HTTP/1.1" 502 150 2025-12-18 00:05:08,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e8c575bc060ce1629bf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:09,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e84575bc060ce1629a0 2025-12-18 00:05:09,104 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e62575bc060ce16296a 2025-12-18 00:05:09,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e62575bc060ce16296a HTTP/1.1" 502 150 2025-12-18 00:05:09,105 - lava_callback - ERROR - Error processing callback for node 69409e62575bc060ce16296a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e62575bc060ce16296a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e62575bc060ce16296a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e62575bc060ce16296a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:09,107 - lava_callback - INFO - Starting async job submit for node 69409e90575bc060ce1629c7 2025-12-18 00:05:09,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:09,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e90575bc060ce1629c7 HTTP/1.1" 502 150 2025-12-18 00:05:09,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e90575bc060ce1629c7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:09,113 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e90575bc060ce1629c7 2025-12-18 00:05:09,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e90575bc060ce1629c7 HTTP/1.1" 502 150 2025-12-18 00:05:09,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e90575bc060ce1629c7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:09,325 - lava_callback - INFO - Received callback for node 6940a220575bc060ce162fc6 INFO: 172.21.0.1:37458 - "POST /node/6940a220575bc060ce162fc6 HTTP/1.0" 202 Accepted 2025-12-18 00:05:09,958 - lava_callback - INFO - Received callback for node 6940a227575bc060ce162fe0 INFO: 172.21.0.1:54182 - "POST /node/6940a227575bc060ce162fe0 HTTP/1.0" 202 Accepted 2025-12-18 00:05:10,597 - lava_callback - INFO - Received callback for node 6940a229575bc060ce162fe4 INFO: 172.21.0.1:54196 - "POST /node/6940a229575bc060ce162fe4 HTTP/1.0" 202 Accepted 2025-12-18 00:05:10,828 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e86575bc060ce1629a7 2025-12-18 00:05:10,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e86575bc060ce1629a7 HTTP/1.1" 502 150 2025-12-18 00:05:10,829 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e86575bc060ce1629a7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:10,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e8c575bc060ce1629bf 2025-12-18 00:05:10,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8c575bc060ce1629bf HTTP/1.1" 502 150 2025-12-18 00:05:10,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e8c575bc060ce1629bf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:11,105 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a5311e3c24e956819cc0 2025-12-18 00:05:11,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a5311e3c24e956819cc0 HTTP/1.1" 502 150 2025-12-18 00:05:11,106 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a5311e3c24e956819cc0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:11,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e90575bc060ce1629c7 2025-12-18 00:05:11,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e90575bc060ce1629c7 HTTP/1.1" 502 150 2025-12-18 00:05:11,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e90575bc060ce1629c7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:11,258 - lava_callback - INFO - Received callback for node 6940a22b575bc060ce162fe7 INFO: 172.21.0.1:54210 - "POST /node/6940a22b575bc060ce162fe7 HTTP/1.0" 202 Accepted 2025-12-18 00:05:11,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e8b575bc060ce1629bc 2025-12-18 00:05:11,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8b575bc060ce1629bc HTTP/1.1" 502 150 2025-12-18 00:05:11,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e8b575bc060ce1629bc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:11,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e67575bc060ce162979 2025-12-18 00:05:11,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e67575bc060ce162979 HTTP/1.1" 502 150 2025-12-18 00:05:11,914 - lava_callback - ERROR - Error processing callback for node 69409e67575bc060ce162979: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e67575bc060ce162979 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e67575bc060ce162979 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e67575bc060ce162979 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:11,916 - lava_callback - INFO - Starting async job submit for node 69409e92575bc060ce1629ca 2025-12-18 00:05:11,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:11,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e92575bc060ce1629ca HTTP/1.1" 502 150 2025-12-18 00:05:11,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e92575bc060ce1629ca'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:11,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e92575bc060ce1629ca 2025-12-18 00:05:11,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e92575bc060ce1629ca HTTP/1.1" 502 150 2025-12-18 00:05:11,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e92575bc060ce1629ca'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:11,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e82575bc060ce16299d 2025-12-18 00:05:11,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e82575bc060ce16299d HTTP/1.1" 502 150 2025-12-18 00:05:11,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e82575bc060ce16299d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:12,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e8e575bc060ce1629c2 2025-12-18 00:05:12,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8e575bc060ce1629c2 HTTP/1.1" 502 150 2025-12-18 00:05:12,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e8e575bc060ce1629c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:12,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e89575bc060ce1629b2 2025-12-18 00:05:12,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e89575bc060ce1629b2 HTTP/1.1" 502 150 2025-12-18 00:05:12,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e89575bc060ce1629b2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:13,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e79575bc060ce162997 HTTP/1.1" 502 150 2025-12-18 00:05:13,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e79575bc060ce162997'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:13,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e92575bc060ce1629ca 2025-12-18 00:05:13,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e92575bc060ce1629ca HTTP/1.1" 502 150 2025-12-18 00:05:13,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e92575bc060ce1629ca'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:14,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e8c575bc060ce1629bf 2025-12-18 00:05:15,116 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e90575bc060ce1629c7 2025-12-18 00:05:15,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e90575bc060ce1629c7 HTTP/1.1" 502 150 2025-12-18 00:05:15,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e90575bc060ce1629c7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54224 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-18 00:05:15,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e65575bc060ce162971 2025-12-18 00:05:15,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e65575bc060ce162971 HTTP/1.1" 502 150 2025-12-18 00:05:15,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e65575bc060ce162971'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:15,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e5f575bc060ce16295f 2025-12-18 00:05:15,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e5f575bc060ce16295f HTTP/1.1" 502 150 2025-12-18 00:05:15,841 - lava_callback - ERROR - Error processing callback for node 69409e5f575bc060ce16295f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e5f575bc060ce16295f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e5f575bc060ce16295f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e5f575bc060ce16295f (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:15,843 - lava_callback - INFO - Starting async job submit for node 69409e94575bc060ce1629cd 2025-12-18 00:05:15,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:15,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e94575bc060ce1629cd HTTP/1.1" 502 150 2025-12-18 00:05:15,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e94575bc060ce1629cd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:15,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e94575bc060ce1629cd 2025-12-18 00:05:15,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e94575bc060ce1629cd HTTP/1.1" 502 150 2025-12-18 00:05:15,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e94575bc060ce1629cd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:16,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dea575bc060ce1628f5 HTTP/1.1" 502 150 2025-12-18 00:05:16,084 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dea575bc060ce1628f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:54240 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:05:17,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e94575bc060ce1629cd 2025-12-18 00:05:17,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e94575bc060ce1629cd HTTP/1.1" 502 150 2025-12-18 00:05:17,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e94575bc060ce1629cd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:17,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e92575bc060ce1629ca 2025-12-18 00:05:17,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e92575bc060ce1629ca HTTP/1.1" 502 150 2025-12-18 00:05:17,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e92575bc060ce1629ca'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:18,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e69575bc060ce16297c 2025-12-18 00:05:18,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e69575bc060ce16297c HTTP/1.1" 502 150 2025-12-18 00:05:18,907 - lava_callback - ERROR - Error processing callback for node 69409e69575bc060ce16297c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e69575bc060ce16297c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e69575bc060ce16297c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e69575bc060ce16297c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:18,910 - lava_callback - INFO - Starting async job submit for node 69409e96575bc060ce1629d0 2025-12-18 00:05:18,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:18,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e96575bc060ce1629d0 HTTP/1.1" 502 150 2025-12-18 00:05:18,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e96575bc060ce1629d0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:18,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e96575bc060ce1629d0 2025-12-18 00:05:18,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e96575bc060ce1629d0 HTTP/1.1" 502 150 2025-12-18 00:05:18,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e96575bc060ce1629d0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:19,106 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a5311e3c24e956819cc0 2025-12-18 00:05:19,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a5311e3c24e956819cc0 HTTP/1.1" 502 150 2025-12-18 00:05:19,107 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a5311e3c24e956819cc0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:19,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e82575bc060ce16299d 2025-12-18 00:05:19,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e82575bc060ce16299d HTTP/1.1" 502 150 2025-12-18 00:05:19,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e82575bc060ce16299d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:20,084 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dea575bc060ce1628f5 2025-12-18 00:05:20,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e8e575bc060ce1629c2 2025-12-18 00:05:20,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8e575bc060ce1629c2 HTTP/1.1" 502 150 2025-12-18 00:05:20,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e8e575bc060ce1629c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:20,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e89575bc060ce1629b2 2025-12-18 00:05:20,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e89575bc060ce1629b2 HTTP/1.1" 502 150 2025-12-18 00:05:20,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e89575bc060ce1629b2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:20,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e96575bc060ce1629d0 2025-12-18 00:05:20,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e96575bc060ce1629d0 HTTP/1.1" 502 150 2025-12-18 00:05:20,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e96575bc060ce1629d0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:21,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e79575bc060ce162997 2025-12-18 00:05:21,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e79575bc060ce162997 HTTP/1.1" 502 150 2025-12-18 00:05:21,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e79575bc060ce162997'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:21,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e94575bc060ce1629cd 2025-12-18 00:05:21,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e94575bc060ce1629cd HTTP/1.1" 502 150 2025-12-18 00:05:21,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e94575bc060ce1629cd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:23,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e90575bc060ce1629c7 2025-12-18 00:05:23,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e90575bc060ce1629c7 HTTP/1.1" 502 150 2025-12-18 00:05:23,118 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e90575bc060ce1629c7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:23,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e7c575bc060ce16299a 2025-12-18 00:05:23,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e7c575bc060ce16299a HTTP/1.1" 502 150 2025-12-18 00:05:23,700 - lava_callback - ERROR - Error processing callback for node 69409e7c575bc060ce16299a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e7c575bc060ce16299a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e7c575bc060ce16299a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e7c575bc060ce16299a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:23,703 - lava_callback - INFO - Starting async job submit for node 69409e98575bc060ce1629d3 2025-12-18 00:05:23,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:23,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e98575bc060ce1629d3 HTTP/1.1" 502 150 2025-12-18 00:05:23,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e98575bc060ce1629d3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:23,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e98575bc060ce1629d3 2025-12-18 00:05:23,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e98575bc060ce1629d3 HTTP/1.1" 502 150 2025-12-18 00:05:23,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e98575bc060ce1629d3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:24,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e84575bc060ce1629a0 HTTP/1.1" 502 150 2025-12-18 00:05:24,272 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e84575bc060ce1629a0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:24,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e96575bc060ce1629d0 2025-12-18 00:05:24,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e96575bc060ce1629d0 HTTP/1.1" 502 150 2025-12-18 00:05:24,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e96575bc060ce1629d0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:25,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e98575bc060ce1629d3 2025-12-18 00:05:25,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e98575bc060ce1629d3 HTTP/1.1" 502 150 2025-12-18 00:05:25,715 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e98575bc060ce1629d3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:25,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e92575bc060ce1629ca 2025-12-18 00:05:25,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e92575bc060ce1629ca HTTP/1.1" 502 150 2025-12-18 00:05:25,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e92575bc060ce1629ca'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38300 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:05:26,829 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e86575bc060ce1629a7 2025-12-18 00:05:26,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e86575bc060ce1629a7 HTTP/1.1" 502 150 2025-12-18 00:05:26,831 - lava_callback - ERROR - Error processing callback for node 69409e86575bc060ce1629a7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e86575bc060ce1629a7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e86575bc060ce1629a7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e86575bc060ce1629a7 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:26,833 - lava_callback - INFO - Starting async job submit for node 69409e9b575bc060ce1629d8 2025-12-18 00:05:26,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:26,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9b575bc060ce1629d8 HTTP/1.1" 502 150 2025-12-18 00:05:26,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e9b575bc060ce1629d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:26,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e9b575bc060ce1629d8 2025-12-18 00:05:26,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9b575bc060ce1629d8 HTTP/1.1" 502 150 2025-12-18 00:05:26,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e9b575bc060ce1629d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:27,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e8b575bc060ce1629bc 2025-12-18 00:05:27,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8b575bc060ce1629bc HTTP/1.1" 502 150 2025-12-18 00:05:27,791 - lava_callback - ERROR - Error processing callback for node 69409e8b575bc060ce1629bc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e8b575bc060ce1629bc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e8b575bc060ce1629bc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e8b575bc060ce1629bc (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:27,793 - lava_callback - INFO - Starting async job submit for node 69409e9d575bc060ce1629e0 2025-12-18 00:05:27,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:27,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9d575bc060ce1629e0 HTTP/1.1" 502 150 2025-12-18 00:05:27,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e9d575bc060ce1629e0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:27,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e9d575bc060ce1629e0 2025-12-18 00:05:27,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9d575bc060ce1629e0 HTTP/1.1" 502 150 2025-12-18 00:05:27,801 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e9d575bc060ce1629e0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:28,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e9b575bc060ce1629d8 2025-12-18 00:05:28,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9b575bc060ce1629d8 HTTP/1.1" 502 150 2025-12-18 00:05:28,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e9b575bc060ce1629d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:29,295 - lava_callback - INFO - Received callback for node 6940a234575bc060ce162fef INFO: 172.21.0.1:38308 - "POST /node/6940a234575bc060ce162fef HTTP/1.0" 202 Accepted 2025-12-18 00:05:29,716 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e98575bc060ce1629d3 2025-12-18 00:05:29,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e98575bc060ce1629d3 HTTP/1.1" 502 150 2025-12-18 00:05:29,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e98575bc060ce1629d3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:29,801 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e9d575bc060ce1629e0 2025-12-18 00:05:29,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9d575bc060ce1629e0 HTTP/1.1" 502 150 2025-12-18 00:05:29,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e9d575bc060ce1629e0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:29,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e94575bc060ce1629cd 2025-12-18 00:05:29,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e94575bc060ce1629cd HTTP/1.1" 502 150 2025-12-18 00:05:29,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e94575bc060ce1629cd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:29,951 - lava_callback - INFO - Received callback for node 6940a237575bc060ce162ffd INFO: 172.21.0.1:43586 - "POST /node/6940a237575bc060ce162ffd HTTP/1.0" 202 Accepted 2025-12-18 00:05:30,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8c575bc060ce1629bf HTTP/1.1" 502 150 2025-12-18 00:05:30,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e8c575bc060ce1629bf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:30,609 - lava_callback - INFO - Received callback for node 6940a239575bc060ce163001 INFO: 172.21.0.1:43596 - "POST /node/6940a239575bc060ce163001 HTTP/1.0" 202 Accepted 2025-12-18 00:05:31,254 - lava_callback - INFO - Received callback for node 6940a23c575bc060ce163005 INFO: 172.21.0.1:43612 - "POST /node/6940a23c575bc060ce163005 HTTP/1.0" 202 Accepted 2025-12-18 00:05:31,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e65575bc060ce162971 2025-12-18 00:05:31,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e65575bc060ce162971 HTTP/1.1" 502 150 2025-12-18 00:05:31,796 - lava_callback - ERROR - Error processing callback for node 69409e65575bc060ce162971: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e65575bc060ce162971 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e65575bc060ce162971 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e65575bc060ce162971 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:31,798 - lava_callback - INFO - Starting async job submit for node 69409e9f575bc060ce1629e9 2025-12-18 00:05:31,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:31,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9f575bc060ce1629e9 HTTP/1.1" 502 150 2025-12-18 00:05:31,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e9f575bc060ce1629e9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:31,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e9f575bc060ce1629e9 2025-12-18 00:05:31,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9f575bc060ce1629e9 HTTP/1.1" 502 150 2025-12-18 00:05:31,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e9f575bc060ce1629e9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:31,909 - lava_callback - INFO - Received callback for node 6940a23e575bc060ce163008 INFO: 172.21.0.1:43618 - "POST /node/6940a23e575bc060ce163008 HTTP/1.0" 202 Accepted 2025-12-18 00:05:32,564 - lava_callback - INFO - Received callback for node 6940a240575bc060ce16300f INFO: 172.21.0.1:43622 - "POST /node/6940a240575bc060ce16300f HTTP/1.0" 202 Accepted 2025-12-18 00:05:32,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e9b575bc060ce1629d8 2025-12-18 00:05:32,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9b575bc060ce1629d8 HTTP/1.1" 502 150 2025-12-18 00:05:32,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e9b575bc060ce1629d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:32,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e96575bc060ce1629d0 2025-12-18 00:05:32,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e96575bc060ce1629d0 HTTP/1.1" 502 150 2025-12-18 00:05:32,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e96575bc060ce1629d0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:33,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e9d575bc060ce1629e0 2025-12-18 00:05:33,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9d575bc060ce1629e0 HTTP/1.1" 502 150 2025-12-18 00:05:33,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e9d575bc060ce1629e0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:33,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e9f575bc060ce1629e9 2025-12-18 00:05:33,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9f575bc060ce1629e9 HTTP/1.1" 502 150 2025-12-18 00:05:33,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e9f575bc060ce1629e9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:35,108 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a5311e3c24e956819cc0 2025-12-18 00:05:35,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e82575bc060ce16299d 2025-12-18 00:05:35,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e82575bc060ce16299d HTTP/1.1" 502 150 2025-12-18 00:05:35,984 - lava_callback - ERROR - Error processing callback for node 69409e82575bc060ce16299d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e82575bc060ce16299d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e82575bc060ce16299d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e82575bc060ce16299d (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:35,986 - lava_callback - INFO - Starting async job submit for node 69409ea1575bc060ce1629ec 2025-12-18 00:05:35,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:35,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea1575bc060ce1629ec HTTP/1.1" 502 150 2025-12-18 00:05:35,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea1575bc060ce1629ec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:35,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea1575bc060ce1629ec 2025-12-18 00:05:35,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea1575bc060ce1629ec HTTP/1.1" 502 150 2025-12-18 00:05:35,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea1575bc060ce1629ec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43628 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:05:36,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e8e575bc060ce1629c2 2025-12-18 00:05:36,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8e575bc060ce1629c2 HTTP/1.1" 502 150 2025-12-18 00:05:36,848 - lava_callback - ERROR - Error processing callback for node 69409e8e575bc060ce1629c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e8e575bc060ce1629c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e8e575bc060ce1629c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e8e575bc060ce1629c2 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:36,851 - lava_callback - INFO - Starting async job submit for node 69409ea3575bc060ce1629ef 2025-12-18 00:05:36,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:36,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e89575bc060ce1629b2 2025-12-18 00:05:36,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e89575bc060ce1629b2 HTTP/1.1" 502 150 2025-12-18 00:05:36,865 - lava_callback - ERROR - Error processing callback for node 69409e89575bc060ce1629b2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e89575bc060ce1629b2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e89575bc060ce1629b2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e89575bc060ce1629b2 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:36,866 - lava_callback - INFO - Starting async job submit for node 69409ea5575bc060ce1629f2 2025-12-18 00:05:36,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:36,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea3575bc060ce1629ef HTTP/1.1" 502 150 2025-12-18 00:05:36,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea3575bc060ce1629ef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:36,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea3575bc060ce1629ef 2025-12-18 00:05:36,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea3575bc060ce1629ef HTTP/1.1" 502 150 2025-12-18 00:05:36,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea3575bc060ce1629ef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:36,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea5575bc060ce1629f2 HTTP/1.1" 502 150 2025-12-18 00:05:36,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea5575bc060ce1629f2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:36,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea5575bc060ce1629f2 2025-12-18 00:05:36,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea5575bc060ce1629f2 HTTP/1.1" 502 150 2025-12-18 00:05:36,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea5575bc060ce1629f2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:37,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e79575bc060ce162997 2025-12-18 00:05:37,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e79575bc060ce162997 HTTP/1.1" 502 150 2025-12-18 00:05:37,015 - lava_callback - ERROR - Error processing callback for node 69409e79575bc060ce162997: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e79575bc060ce162997 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e79575bc060ce162997 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e79575bc060ce162997 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:37,018 - lava_callback - INFO - Starting async job submit for node 69409ec4575bc060ce162a75 2025-12-18 00:05:37,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:37,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ec4575bc060ce162a75 HTTP/1.1" 502 150 2025-12-18 00:05:37,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ec4575bc060ce162a75'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:37,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ec4575bc060ce162a75 2025-12-18 00:05:37,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ec4575bc060ce162a75 HTTP/1.1" 502 150 2025-12-18 00:05:37,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ec4575bc060ce162a75'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:37,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e98575bc060ce1629d3 2025-12-18 00:05:37,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e98575bc060ce1629d3 HTTP/1.1" 502 150 2025-12-18 00:05:37,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e98575bc060ce1629d3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:37,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e9f575bc060ce1629e9 2025-12-18 00:05:37,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9f575bc060ce1629e9 HTTP/1.1" 502 150 2025-12-18 00:05:37,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e9f575bc060ce1629e9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:37,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea1575bc060ce1629ec 2025-12-18 00:05:37,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea1575bc060ce1629ec HTTP/1.1" 502 150 2025-12-18 00:05:37,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea1575bc060ce1629ec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:38,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e8c575bc060ce1629bf 2025-12-18 00:05:38,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8c575bc060ce1629bf HTTP/1.1" 502 150 2025-12-18 00:05:38,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e8c575bc060ce1629bf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:38,874 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea3575bc060ce1629ef 2025-12-18 00:05:38,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea3575bc060ce1629ef HTTP/1.1" 502 150 2025-12-18 00:05:38,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea5575bc060ce1629f2 2025-12-18 00:05:38,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea3575bc060ce1629ef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:38,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea5575bc060ce1629f2 HTTP/1.1" 502 150 2025-12-18 00:05:38,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea5575bc060ce1629f2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:39,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ec4575bc060ce162a75 2025-12-18 00:05:39,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ec4575bc060ce162a75 HTTP/1.1" 502 150 2025-12-18 00:05:39,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ec4575bc060ce162a75'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:39,119 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e90575bc060ce1629c7 2025-12-18 00:05:39,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e90575bc060ce1629c7 HTTP/1.1" 502 150 2025-12-18 00:05:39,120 - lava_callback - ERROR - Error processing callback for node 69409e90575bc060ce1629c7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e90575bc060ce1629c7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e90575bc060ce1629c7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e90575bc060ce1629c7 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:39,122 - lava_callback - INFO - Starting async job submit for node 69412836459ff5fc64d2bf6a 2025-12-18 00:05:39,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:39,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69412836459ff5fc64d2bf6a HTTP/1.1" 502 150 2025-12-18 00:05:39,171 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69412836459ff5fc64d2bf6a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:39,171 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69412836459ff5fc64d2bf6a 2025-12-18 00:05:39,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69412836459ff5fc64d2bf6a HTTP/1.1" 502 150 2025-12-18 00:05:39,172 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69412836459ff5fc64d2bf6a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:40,272 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e84575bc060ce1629a0 2025-12-18 00:05:40,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e84575bc060ce1629a0 HTTP/1.1" 502 150 2025-12-18 00:05:40,274 - lava_callback - ERROR - Error processing callback for node 69409e84575bc060ce1629a0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e84575bc060ce1629a0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e84575bc060ce1629a0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e84575bc060ce1629a0 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:40,276 - lava_callback - INFO - Starting async job submit for node 69409ec5575bc060ce162a7b 2025-12-18 00:05:40,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:40,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ec5575bc060ce162a7b HTTP/1.1" 502 150 2025-12-18 00:05:40,292 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ec5575bc060ce162a7b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:40,292 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ec5575bc060ce162a7b 2025-12-18 00:05:40,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ec5575bc060ce162a7b HTTP/1.1" 502 150 2025-12-18 00:05:40,293 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ec5575bc060ce162a7b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:40,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e9b575bc060ce1629d8 2025-12-18 00:05:40,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9b575bc060ce1629d8 HTTP/1.1" 502 150 2025-12-18 00:05:40,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e9b575bc060ce1629d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:41,172 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69412836459ff5fc64d2bf6a 2025-12-18 00:05:41,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e9d575bc060ce1629e0 2025-12-18 00:05:41,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9d575bc060ce1629e0 HTTP/1.1" 502 150 2025-12-18 00:05:41,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e9d575bc060ce1629e0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:41,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e92575bc060ce1629ca 2025-12-18 00:05:41,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e92575bc060ce1629ca HTTP/1.1" 502 150 2025-12-18 00:05:41,933 - lava_callback - ERROR - Error processing callback for node 69409e92575bc060ce1629ca: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e92575bc060ce1629ca (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e92575bc060ce1629ca (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e92575bc060ce1629ca (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:41,934 - lava_callback - INFO - Starting async job submit for node 69409ecb575bc060ce162a80 2025-12-18 00:05:41,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:41,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ecb575bc060ce162a80 HTTP/1.1" 502 150 2025-12-18 00:05:41,963 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ecb575bc060ce162a80'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:41,963 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ecb575bc060ce162a80 2025-12-18 00:05:41,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ecb575bc060ce162a80 HTTP/1.1" 502 150 2025-12-18 00:05:41,964 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ecb575bc060ce162a80'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:41,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea1575bc060ce1629ec 2025-12-18 00:05:41,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea1575bc060ce1629ec HTTP/1.1" 502 150 2025-12-18 00:05:41,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea1575bc060ce1629ec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:42,293 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ec5575bc060ce162a7b 2025-12-18 00:05:42,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ec5575bc060ce162a7b HTTP/1.1" 502 150 2025-12-18 00:05:42,294 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ec5575bc060ce162a7b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:42,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea3575bc060ce1629ef 2025-12-18 00:05:42,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea5575bc060ce1629f2 2025-12-18 00:05:42,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea5575bc060ce1629f2 HTTP/1.1" 502 150 2025-12-18 00:05:42,883 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea5575bc060ce1629f2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:42,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea3575bc060ce1629ef HTTP/1.1" 502 150 2025-12-18 00:05:42,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea3575bc060ce1629ef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:43,028 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ec4575bc060ce162a75 2025-12-18 00:05:43,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ec4575bc060ce162a75 HTTP/1.1" 502 150 2025-12-18 00:05:43,029 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ec4575bc060ce162a75'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:43,964 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ecb575bc060ce162a80 2025-12-18 00:05:43,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ecb575bc060ce162a80 HTTP/1.1" 502 150 2025-12-18 00:05:43,965 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ecb575bc060ce162a80'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:45,549 - lava_callback - INFO - Received callback for node 6942e2f3e43d2b76e9023928 INFO: 172.21.0.1:58282 - "POST /node/6942e2f3e43d2b76e9023928 HTTP/1.0" 202 Accepted 2025-12-18 00:05:45,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e9f575bc060ce1629e9 2025-12-18 00:05:45,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9f575bc060ce1629e9 HTTP/1.1" 502 150 2025-12-18 00:05:45,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409e9f575bc060ce1629e9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:45,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e94575bc060ce1629cd 2025-12-18 00:05:45,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e94575bc060ce1629cd HTTP/1.1" 502 150 2025-12-18 00:05:45,881 - lava_callback - ERROR - Error processing callback for node 69409e94575bc060ce1629cd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e94575bc060ce1629cd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e94575bc060ce1629cd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e94575bc060ce1629cd (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:45,883 - lava_callback - INFO - Starting async job submit for node 69409ed8575bc060ce162a8d 2025-12-18 00:05:45,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:45,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ed8575bc060ce162a8d HTTP/1.1" 502 150 2025-12-18 00:05:45,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ed8575bc060ce162a8d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:45,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ed8575bc060ce162a8d 2025-12-18 00:05:45,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ed8575bc060ce162a8d HTTP/1.1" 502 150 2025-12-18 00:05:45,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ed8575bc060ce162a8d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:46,295 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ec5575bc060ce162a7b 2025-12-18 00:05:46,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ec5575bc060ce162a7b HTTP/1.1" 502 150 2025-12-18 00:05:46,296 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ec5575bc060ce162a7b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58290 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:05:47,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ed8575bc060ce162a8d 2025-12-18 00:05:47,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ed8575bc060ce162a8d HTTP/1.1" 502 150 2025-12-18 00:05:47,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ed8575bc060ce162a8d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:47,965 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ecb575bc060ce162a80 2025-12-18 00:05:47,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ecb575bc060ce162a80 HTTP/1.1" 502 150 2025-12-18 00:05:47,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ecb575bc060ce162a80'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:48,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e96575bc060ce1629d0 2025-12-18 00:05:48,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e96575bc060ce1629d0 HTTP/1.1" 502 150 2025-12-18 00:05:48,925 - lava_callback - ERROR - Error processing callback for node 69409e96575bc060ce1629d0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e96575bc060ce1629d0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e96575bc060ce1629d0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e96575bc060ce1629d0 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:48,927 - lava_callback - INFO - Starting async job submit for node 69409eda575bc060ce162a90 2025-12-18 00:05:48,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:48,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eda575bc060ce162a90 HTTP/1.1" 502 150 2025-12-18 00:05:48,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409eda575bc060ce162a90'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:48,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409eda575bc060ce162a90 2025-12-18 00:05:48,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eda575bc060ce162a90 HTTP/1.1" 502 150 2025-12-18 00:05:48,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409eda575bc060ce162a90'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:49,346 - lava_callback - INFO - Received callback for node 6940a248575bc060ce16301b INFO: 172.21.0.1:58300 - "POST /node/6940a248575bc060ce16301b HTTP/1.0" 202 Accepted 2025-12-18 00:05:49,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea1575bc060ce1629ec 2025-12-18 00:05:49,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea1575bc060ce1629ec HTTP/1.1" 502 150 2025-12-18 00:05:49,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea1575bc060ce1629ec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:50,015 - lava_callback - INFO - Received callback for node 6940a24b575bc060ce163026 INFO: 172.21.0.1:48890 - "POST /node/6940a24b575bc060ce163026 HTTP/1.0" 202 Accepted 2025-12-18 00:05:50,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a5311e3c24e956819cc0 HTTP/1.1" 502 150 2025-12-18 00:05:50,388 - lava_callback - ERROR - Error processing callback for node 6940a5311e3c24e956819cc0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a5311e3c24e956819cc0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a5311e3c24e956819cc0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a5311e3c24e956819cc0 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:50,390 - lava_callback - INFO - Starting async job submit for node 69409eec575bc060ce162aa3 2025-12-18 00:05:50,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:50,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eec575bc060ce162aa3 HTTP/1.1" 502 150 2025-12-18 00:05:50,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409eec575bc060ce162aa3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:50,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409eec575bc060ce162aa3 2025-12-18 00:05:50,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eec575bc060ce162aa3 HTTP/1.1" 502 150 2025-12-18 00:05:50,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409eec575bc060ce162aa3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:50,680 - lava_callback - INFO - Received callback for node 6940a24d575bc060ce16302a INFO: 172.21.0.1:48892 - "POST /node/6940a24d575bc060ce16302a HTTP/1.0" 202 Accepted 2025-12-18 00:05:50,883 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea5575bc060ce1629f2 2025-12-18 00:05:50,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea5575bc060ce1629f2 HTTP/1.1" 502 150 2025-12-18 00:05:50,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea5575bc060ce1629f2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:50,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea3575bc060ce1629ef 2025-12-18 00:05:50,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea3575bc060ce1629ef HTTP/1.1" 502 150 2025-12-18 00:05:50,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ea3575bc060ce1629ef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:50,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dea575bc060ce1628f5 HTTP/1.1" 502 150 2025-12-18 00:05:50,896 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dea575bc060ce1628f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:50,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409eda575bc060ce162a90 2025-12-18 00:05:50,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eda575bc060ce162a90 HTTP/1.1" 502 150 2025-12-18 00:05:50,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409eda575bc060ce162a90'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:51,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ec4575bc060ce162a75 2025-12-18 00:05:51,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ec4575bc060ce162a75 HTTP/1.1" 502 150 2025-12-18 00:05:51,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ec4575bc060ce162a75'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:51,348 - lava_callback - INFO - Received callback for node 6940a24f575bc060ce16302d INFO: 172.21.0.1:48904 - "POST /node/6940a24f575bc060ce16302d HTTP/1.0" 202 Accepted 2025-12-18 00:05:51,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ed8575bc060ce162a8d 2025-12-18 00:05:51,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ed8575bc060ce162a8d HTTP/1.1" 502 150 2025-12-18 00:05:51,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ed8575bc060ce162a8d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:52,399 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409eec575bc060ce162aa3 2025-12-18 00:05:52,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eec575bc060ce162aa3 HTTP/1.1" 502 150 2025-12-18 00:05:52,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409eec575bc060ce162aa3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:53,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e98575bc060ce1629d3 2025-12-18 00:05:53,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e98575bc060ce1629d3 HTTP/1.1" 502 150 2025-12-18 00:05:53,720 - lava_callback - ERROR - Error processing callback for node 69409e98575bc060ce1629d3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e98575bc060ce1629d3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e98575bc060ce1629d3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e98575bc060ce1629d3 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:53,722 - lava_callback - INFO - Starting async job submit for node 69409eef575bc060ce162aae 2025-12-18 00:05:53,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:53,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eef575bc060ce162aae HTTP/1.1" 502 150 2025-12-18 00:05:53,729 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409eef575bc060ce162aae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:53,729 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409eef575bc060ce162aae 2025-12-18 00:05:53,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eef575bc060ce162aae HTTP/1.1" 502 150 2025-12-18 00:05:53,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409eef575bc060ce162aae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:54,296 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ec5575bc060ce162a7b 2025-12-18 00:05:54,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ec5575bc060ce162a7b HTTP/1.1" 502 150 2025-12-18 00:05:54,300 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ec5575bc060ce162a7b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:54,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e8c575bc060ce1629bf 2025-12-18 00:05:54,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e8c575bc060ce1629bf HTTP/1.1" 502 150 2025-12-18 00:05:54,419 - lava_callback - ERROR - Error processing callback for node 69409e8c575bc060ce1629bf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e8c575bc060ce1629bf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e8c575bc060ce1629bf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e8c575bc060ce1629bf (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:54,421 - lava_callback - INFO - Starting async job submit for node 69409ef1575bc060ce162ab1 2025-12-18 00:05:54,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:54,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef1575bc060ce162ab1 HTTP/1.1" 502 150 2025-12-18 00:05:54,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ef1575bc060ce162ab1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:54,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ef1575bc060ce162ab1 2025-12-18 00:05:54,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef1575bc060ce162ab1 HTTP/1.1" 502 150 2025-12-18 00:05:54,429 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ef1575bc060ce162ab1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:54,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409eda575bc060ce162a90 2025-12-18 00:05:54,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eda575bc060ce162a90 HTTP/1.1" 502 150 2025-12-18 00:05:54,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409eda575bc060ce162a90'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:55,734 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409eef575bc060ce162aae 2025-12-18 00:05:55,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eef575bc060ce162aae HTTP/1.1" 502 150 2025-12-18 00:05:55,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409eef575bc060ce162aae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:55,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ecb575bc060ce162a80 2025-12-18 00:05:55,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ecb575bc060ce162a80 HTTP/1.1" 502 150 2025-12-18 00:05:55,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ecb575bc060ce162a80'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:56,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409eec575bc060ce162aa3 2025-12-18 00:05:56,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eec575bc060ce162aa3 HTTP/1.1" 502 150 2025-12-18 00:05:56,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409eec575bc060ce162aa3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:56,429 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ef1575bc060ce162ab1 2025-12-18 00:05:56,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef1575bc060ce162ab1 HTTP/1.1" 502 150 2025-12-18 00:05:56,430 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ef1575bc060ce162ab1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:48912 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:05:56,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69412836459ff5fc64d2bf6a HTTP/1.1" 502 150 2025-12-18 00:05:56,528 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69412836459ff5fc64d2bf6a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:56,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e9b575bc060ce1629d8 2025-12-18 00:05:56,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9b575bc060ce1629d8 HTTP/1.1" 502 150 2025-12-18 00:05:56,853 - lava_callback - ERROR - Error processing callback for node 69409e9b575bc060ce1629d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e9b575bc060ce1629d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e9b575bc060ce1629d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e9b575bc060ce1629d8 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:56,856 - lava_callback - INFO - Starting async job submit for node 69409ef3575bc060ce162ab4 2025-12-18 00:05:56,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:56,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef3575bc060ce162ab4 HTTP/1.1" 502 150 2025-12-18 00:05:56,886 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ef3575bc060ce162ab4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:56,886 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ef3575bc060ce162ab4 2025-12-18 00:05:56,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef3575bc060ce162ab4 HTTP/1.1" 502 150 2025-12-18 00:05:56,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ef3575bc060ce162ab4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:57,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e9d575bc060ce1629e0 2025-12-18 00:05:57,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9d575bc060ce1629e0 HTTP/1.1" 502 150 2025-12-18 00:05:57,807 - lava_callback - ERROR - Error processing callback for node 69409e9d575bc060ce1629e0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e9d575bc060ce1629e0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e9d575bc060ce1629e0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e9d575bc060ce1629e0 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:05:57,809 - lava_callback - INFO - Starting async job submit for node 69409ef5575bc060ce162ab7 2025-12-18 00:05:57,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:05:57,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef5575bc060ce162ab7 HTTP/1.1" 502 150 2025-12-18 00:05:57,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ef5575bc060ce162ab7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:57,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ef5575bc060ce162ab7 2025-12-18 00:05:57,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef5575bc060ce162ab7 HTTP/1.1" 502 150 2025-12-18 00:05:57,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ef5575bc060ce162ab7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:58,459 - lava_callback - INFO - Received callback for node 6940a25c575bc060ce163037 INFO: 172.21.0.1:48916 - "POST /node/6940a25c575bc060ce163037 HTTP/1.0" 202 Accepted 2025-12-18 00:05:58,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ef3575bc060ce162ab4 2025-12-18 00:05:58,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef3575bc060ce162ab4 HTTP/1.1" 502 150 2025-12-18 00:05:58,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ef3575bc060ce162ab4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:58,896 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dea575bc060ce1628f5 2025-12-18 00:05:59,116 - lava_callback - INFO - Received callback for node 6940a261575bc060ce163042 INFO: 172.21.0.1:48930 - "POST /node/6940a261575bc060ce163042 HTTP/1.0" 202 Accepted 2025-12-18 00:05:59,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409eef575bc060ce162aae 2025-12-18 00:05:59,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eef575bc060ce162aae HTTP/1.1" 502 150 2025-12-18 00:05:59,737 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409eef575bc060ce162aae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:59,764 - lava_callback - INFO - Received callback for node 6940a263575bc060ce163045 INFO: 172.21.0.1:36718 - "POST /node/6940a263575bc060ce163045 HTTP/1.0" 202 Accepted 2025-12-18 00:05:59,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ef5575bc060ce162ab7 2025-12-18 00:05:59,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef5575bc060ce162ab7 HTTP/1.1" 502 150 2025-12-18 00:05:59,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ef5575bc060ce162ab7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:05:59,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ed8575bc060ce162a8d 2025-12-18 00:05:59,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ed8575bc060ce162a8d HTTP/1.1" 502 150 2025-12-18 00:05:59,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ed8575bc060ce162a8d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:00,430 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ef1575bc060ce162ab1 2025-12-18 00:06:00,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef1575bc060ce162ab1 HTTP/1.1" 502 150 2025-12-18 00:06:00,431 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ef1575bc060ce162ab1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:00,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69412836459ff5fc64d2bf6a 2025-12-18 00:06:00,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69412836459ff5fc64d2bf6a HTTP/1.1" 502 150 2025-12-18 00:06:00,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69412836459ff5fc64d2bf6a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:01,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409e9f575bc060ce1629e9 2025-12-18 00:06:01,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409e9f575bc060ce1629e9 HTTP/1.1" 502 150 2025-12-18 00:06:01,812 - lava_callback - ERROR - Error processing callback for node 69409e9f575bc060ce1629e9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e9f575bc060ce1629e9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e9f575bc060ce1629e9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409e9f575bc060ce1629e9 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:01,814 - lava_callback - INFO - Starting async job submit for node 69409f00575bc060ce162abc 2025-12-18 00:06:01,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:01,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f00575bc060ce162abc HTTP/1.1" 502 150 2025-12-18 00:06:01,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f00575bc060ce162abc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:01,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f00575bc060ce162abc 2025-12-18 00:06:01,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f00575bc060ce162abc HTTP/1.1" 502 150 2025-12-18 00:06:01,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f00575bc060ce162abc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:02,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ef3575bc060ce162ab4 2025-12-18 00:06:02,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef3575bc060ce162ab4 HTTP/1.1" 502 150 2025-12-18 00:06:02,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ef3575bc060ce162ab4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:02,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409eda575bc060ce162a90 2025-12-18 00:06:02,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eda575bc060ce162a90 HTTP/1.1" 502 150 2025-12-18 00:06:02,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409eda575bc060ce162a90'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:03,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f00575bc060ce162abc 2025-12-18 00:06:03,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f00575bc060ce162abc HTTP/1.1" 502 150 2025-12-18 00:06:03,823 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f00575bc060ce162abc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:03,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ef5575bc060ce162ab7 2025-12-18 00:06:03,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef5575bc060ce162ab7 HTTP/1.1" 502 150 2025-12-18 00:06:03,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ef5575bc060ce162ab7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36722 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:06:04,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409eec575bc060ce162aa3 2025-12-18 00:06:04,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eec575bc060ce162aa3 HTTP/1.1" 502 150 2025-12-18 00:06:04,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409eec575bc060ce162aa3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:05,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea1575bc060ce1629ec 2025-12-18 00:06:05,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea1575bc060ce1629ec HTTP/1.1" 502 150 2025-12-18 00:06:05,999 - lava_callback - ERROR - Error processing callback for node 69409ea1575bc060ce1629ec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ea1575bc060ce1629ec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ea1575bc060ce1629ec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ea1575bc060ce1629ec (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:06,001 - lava_callback - INFO - Starting async job submit for node 69409f02575bc060ce162ac3 2025-12-18 00:06:06,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:06,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f02575bc060ce162ac3 HTTP/1.1" 502 150 2025-12-18 00:06:06,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f02575bc060ce162ac3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:06,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f02575bc060ce162ac3 2025-12-18 00:06:06,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f02575bc060ce162ac3 HTTP/1.1" 502 150 2025-12-18 00:06:06,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f02575bc060ce162ac3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:36730 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:06:06,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea5575bc060ce1629f2 2025-12-18 00:06:06,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea5575bc060ce1629f2 HTTP/1.1" 502 150 2025-12-18 00:06:06,886 - lava_callback - ERROR - Error processing callback for node 69409ea5575bc060ce1629f2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ea5575bc060ce1629f2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ea5575bc060ce1629f2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ea5575bc060ce1629f2 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:06,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ea3575bc060ce1629ef 2025-12-18 00:06:06,888 - lava_callback - INFO - Starting async job submit for node 69409f04575bc060ce162ac6 2025-12-18 00:06:06,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:06,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ea3575bc060ce1629ef HTTP/1.1" 502 150 2025-12-18 00:06:06,890 - lava_callback - ERROR - Error processing callback for node 69409ea3575bc060ce1629ef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ea3575bc060ce1629ef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ea3575bc060ce1629ef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ea3575bc060ce1629ef (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:06,891 - lava_callback - INFO - Starting async job submit for node 69409f28575bc060ce162ad6 2025-12-18 00:06:06,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:06,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f28575bc060ce162ad6 HTTP/1.1" 502 150 2025-12-18 00:06:06,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f28575bc060ce162ad6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:06,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f28575bc060ce162ad6 2025-12-18 00:06:06,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f28575bc060ce162ad6 HTTP/1.1" 502 150 2025-12-18 00:06:06,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f28575bc060ce162ad6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:06,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f04575bc060ce162ac6 HTTP/1.1" 502 150 2025-12-18 00:06:06,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f04575bc060ce162ac6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:06,903 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f04575bc060ce162ac6 2025-12-18 00:06:06,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f04575bc060ce162ac6 HTTP/1.1" 502 150 2025-12-18 00:06:06,904 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f04575bc060ce162ac6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:07,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ec4575bc060ce162a75 2025-12-18 00:06:07,737 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409eef575bc060ce162aae 2025-12-18 00:06:07,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eef575bc060ce162aae HTTP/1.1" 502 150 2025-12-18 00:06:07,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409eef575bc060ce162aae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:07,823 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f00575bc060ce162abc 2025-12-18 00:06:07,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f00575bc060ce162abc HTTP/1.1" 502 150 2025-12-18 00:06:07,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f00575bc060ce162abc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:08,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f02575bc060ce162ac3 2025-12-18 00:06:08,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f02575bc060ce162ac3 HTTP/1.1" 502 150 2025-12-18 00:06:08,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f02575bc060ce162ac3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:08,432 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ef1575bc060ce162ab1 2025-12-18 00:06:08,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef1575bc060ce162ab1 HTTP/1.1" 502 150 2025-12-18 00:06:08,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ef1575bc060ce162ab1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:08,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69412836459ff5fc64d2bf6a 2025-12-18 00:06:08,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69412836459ff5fc64d2bf6a HTTP/1.1" 502 150 2025-12-18 00:06:08,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69412836459ff5fc64d2bf6a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:08,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f28575bc060ce162ad6 2025-12-18 00:06:08,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f28575bc060ce162ad6 HTTP/1.1" 502 150 2025-12-18 00:06:08,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f28575bc060ce162ad6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:08,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f04575bc060ce162ac6 2025-12-18 00:06:08,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f04575bc060ce162ac6 HTTP/1.1" 502 150 2025-12-18 00:06:08,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f04575bc060ce162ac6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:10,300 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ec5575bc060ce162a7b 2025-12-18 00:06:10,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ec5575bc060ce162a7b HTTP/1.1" 502 150 2025-12-18 00:06:10,301 - lava_callback - ERROR - Error processing callback for node 69409ec5575bc060ce162a7b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ec5575bc060ce162a7b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ec5575bc060ce162a7b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ec5575bc060ce162a7b (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:10,303 - lava_callback - INFO - Starting async job submit for node 69409f3c575bc060ce162ae5 2025-12-18 00:06:10,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:10,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f3c575bc060ce162ae5 HTTP/1.1" 502 150 2025-12-18 00:06:10,310 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f3c575bc060ce162ae5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:10,310 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f3c575bc060ce162ae5 2025-12-18 00:06:10,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f3c575bc060ce162ae5 HTTP/1.1" 502 150 2025-12-18 00:06:10,311 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f3c575bc060ce162ae5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:10,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ef3575bc060ce162ab4 2025-12-18 00:06:10,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef3575bc060ce162ab4 HTTP/1.1" 502 150 2025-12-18 00:06:10,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ef3575bc060ce162ab4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:11,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ef5575bc060ce162ab7 2025-12-18 00:06:11,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef5575bc060ce162ab7 HTTP/1.1" 502 150 2025-12-18 00:06:11,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ef5575bc060ce162ab7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:11,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ecb575bc060ce162a80 2025-12-18 00:06:11,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ecb575bc060ce162a80 HTTP/1.1" 502 150 2025-12-18 00:06:11,969 - lava_callback - ERROR - Error processing callback for node 69409ecb575bc060ce162a80: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ecb575bc060ce162a80 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ecb575bc060ce162a80 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ecb575bc060ce162a80 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:11,971 - lava_callback - INFO - Starting async job submit for node 69409f40575bc060ce162af2 2025-12-18 00:06:11,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:11,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f40575bc060ce162af2 HTTP/1.1" 502 150 2025-12-18 00:06:11,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f40575bc060ce162af2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:11,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f40575bc060ce162af2 2025-12-18 00:06:11,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f40575bc060ce162af2 HTTP/1.1" 502 150 2025-12-18 00:06:11,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f40575bc060ce162af2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:12,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f02575bc060ce162ac3 2025-12-18 00:06:12,311 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f3c575bc060ce162ae5 2025-12-18 00:06:12,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f3c575bc060ce162ae5 HTTP/1.1" 502 150 2025-12-18 00:06:12,312 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f3c575bc060ce162ae5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:12,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f28575bc060ce162ad6 2025-12-18 00:06:12,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f28575bc060ce162ad6 HTTP/1.1" 502 150 2025-12-18 00:06:12,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f28575bc060ce162ad6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:12,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f04575bc060ce162ac6 2025-12-18 00:06:12,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f04575bc060ce162ac6 HTTP/1.1" 502 150 2025-12-18 00:06:12,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f04575bc060ce162ac6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:13,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f40575bc060ce162af2 2025-12-18 00:06:13,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f40575bc060ce162af2 HTTP/1.1" 502 150 2025-12-18 00:06:13,980 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f40575bc060ce162af2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34654 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-18 00:06:15,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f00575bc060ce162abc 2025-12-18 00:06:15,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f00575bc060ce162abc HTTP/1.1" 502 150 2025-12-18 00:06:15,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f00575bc060ce162abc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:15,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ed8575bc060ce162a8d 2025-12-18 00:06:15,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ed8575bc060ce162a8d HTTP/1.1" 502 150 2025-12-18 00:06:15,901 - lava_callback - ERROR - Error processing callback for node 69409ed8575bc060ce162a8d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ed8575bc060ce162a8d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ed8575bc060ce162a8d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ed8575bc060ce162a8d (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:15,903 - lava_callback - INFO - Starting async job submit for node 69409f49575bc060ce162b06 2025-12-18 00:06:15,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:15,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f49575bc060ce162b06 HTTP/1.1" 502 150 2025-12-18 00:06:15,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f49575bc060ce162b06'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:15,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f49575bc060ce162b06 2025-12-18 00:06:15,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f49575bc060ce162b06 HTTP/1.1" 502 150 2025-12-18 00:06:15,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f49575bc060ce162b06'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:16,313 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f3c575bc060ce162ae5 2025-12-18 00:06:16,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f3c575bc060ce162ae5 HTTP/1.1" 502 150 2025-12-18 00:06:16,314 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f3c575bc060ce162ae5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34658 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:06:17,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f49575bc060ce162b06 2025-12-18 00:06:17,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f49575bc060ce162b06 HTTP/1.1" 502 150 2025-12-18 00:06:17,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f49575bc060ce162b06'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:17,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f40575bc060ce162af2 2025-12-18 00:06:17,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f40575bc060ce162af2 HTTP/1.1" 502 150 2025-12-18 00:06:17,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f40575bc060ce162af2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:18,448 - lava_callback - INFO - Received callback for node 6940a266575bc060ce163048 INFO: 172.21.0.1:34670 - "POST /node/6940a266575bc060ce163048 HTTP/1.0" 202 Accepted 2025-12-18 00:06:18,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409eda575bc060ce162a90 2025-12-18 00:06:18,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eda575bc060ce162a90 HTTP/1.1" 502 150 2025-12-18 00:06:18,941 - lava_callback - ERROR - Error processing callback for node 69409eda575bc060ce162a90: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409eda575bc060ce162a90 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409eda575bc060ce162a90 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409eda575bc060ce162a90 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:18,943 - lava_callback - INFO - Starting async job submit for node 69409f50575bc060ce162b14 2025-12-18 00:06:18,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:18,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f50575bc060ce162b14 HTTP/1.1" 502 150 2025-12-18 00:06:18,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f50575bc060ce162b14'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:18,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f50575bc060ce162b14 2025-12-18 00:06:18,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f50575bc060ce162b14 HTTP/1.1" 502 150 2025-12-18 00:06:18,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f50575bc060ce162b14'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:19,109 - lava_callback - INFO - Received callback for node 6940a268575bc060ce16304d INFO: 172.21.0.1:34684 - "POST /node/6940a268575bc060ce16304d HTTP/1.0" 202 Accepted 2025-12-18 00:06:19,776 - lava_callback - INFO - Received callback for node 6940a271575bc060ce16305c INFO: 172.21.0.1:60980 - "POST /node/6940a271575bc060ce16305c HTTP/1.0" 202 Accepted 2025-12-18 00:06:20,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409eec575bc060ce162aa3 2025-12-18 00:06:20,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eec575bc060ce162aa3 HTTP/1.1" 502 150 2025-12-18 00:06:20,405 - lava_callback - ERROR - Error processing callback for node 69409eec575bc060ce162aa3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409eec575bc060ce162aa3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409eec575bc060ce162aa3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409eec575bc060ce162aa3 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:20,407 - lava_callback - INFO - Starting async job submit for node 69409f47575bc060ce162aff 2025-12-18 00:06:20,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:20,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f47575bc060ce162aff HTTP/1.1" 502 150 2025-12-18 00:06:20,414 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f47575bc060ce162aff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:20,414 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f47575bc060ce162aff 2025-12-18 00:06:20,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f47575bc060ce162aff HTTP/1.1" 502 150 2025-12-18 00:06:20,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f47575bc060ce162aff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:20,424 - lava_callback - INFO - Received callback for node 6940a273575bc060ce163066 INFO: 172.21.0.1:60992 - "POST /node/6940a273575bc060ce163066 HTTP/1.0" 202 Accepted 2025-12-18 00:06:20,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f28575bc060ce162ad6 2025-12-18 00:06:20,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f28575bc060ce162ad6 HTTP/1.1" 502 150 2025-12-18 00:06:20,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f28575bc060ce162ad6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:20,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f04575bc060ce162ac6 2025-12-18 00:06:20,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f04575bc060ce162ac6 HTTP/1.1" 502 150 2025-12-18 00:06:20,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f04575bc060ce162ac6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:20,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f50575bc060ce162b14 2025-12-18 00:06:20,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f50575bc060ce162b14 HTTP/1.1" 502 150 2025-12-18 00:06:20,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f50575bc060ce162b14'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:21,079 - lava_callback - INFO - Received callback for node 6940a275575bc060ce163069 INFO: 172.21.0.1:32776 - "POST /node/6940a275575bc060ce163069 HTTP/1.0" 202 Accepted 2025-12-18 00:06:21,745 - lava_callback - INFO - Received callback for node 6940a277575bc060ce16306c INFO: 172.21.0.1:32788 - "POST /node/6940a277575bc060ce16306c HTTP/1.0" 202 Accepted 2025-12-18 00:06:21,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f49575bc060ce162b06 2025-12-18 00:06:21,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f49575bc060ce162b06 HTTP/1.1" 502 150 2025-12-18 00:06:21,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f49575bc060ce162b06'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:22,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ec4575bc060ce162a75 HTTP/1.1" 502 150 2025-12-18 00:06:22,384 - lava_callback - ERROR - Error processing callback for node 69409ec4575bc060ce162a75: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ec4575bc060ce162a75 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ec4575bc060ce162a75 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ec4575bc060ce162a75 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:22,387 - lava_callback - INFO - Starting async job submit for node 69409f52575bc060ce162b17 2025-12-18 00:06:22,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:22,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f52575bc060ce162b17 HTTP/1.1" 502 150 2025-12-18 00:06:22,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f52575bc060ce162b17'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:22,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f52575bc060ce162b17 2025-12-18 00:06:22,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f52575bc060ce162b17 HTTP/1.1" 502 150 2025-12-18 00:06:22,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f52575bc060ce162b17'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:22,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f47575bc060ce162aff 2025-12-18 00:06:22,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f47575bc060ce162aff HTTP/1.1" 502 150 2025-12-18 00:06:22,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f47575bc060ce162aff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:23,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409eef575bc060ce162aae 2025-12-18 00:06:23,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409eef575bc060ce162aae HTTP/1.1" 502 150 2025-12-18 00:06:23,740 - lava_callback - ERROR - Error processing callback for node 69409eef575bc060ce162aae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409eef575bc060ce162aae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409eef575bc060ce162aae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409eef575bc060ce162aae (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:23,742 - lava_callback - INFO - Starting async job submit for node 69409f4a575bc060ce162b09 2025-12-18 00:06:23,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:23,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4a575bc060ce162b09 HTTP/1.1" 502 150 2025-12-18 00:06:23,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f4a575bc060ce162b09'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:23,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f4a575bc060ce162b09 2025-12-18 00:06:23,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4a575bc060ce162b09 HTTP/1.1" 502 150 2025-12-18 00:06:23,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f4a575bc060ce162b09'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:24,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f3c575bc060ce162ae5 2025-12-18 00:06:24,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f3c575bc060ce162ae5 HTTP/1.1" 502 150 2025-12-18 00:06:24,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f3c575bc060ce162ae5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:24,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f52575bc060ce162b17 2025-12-18 00:06:24,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f52575bc060ce162b17 HTTP/1.1" 502 150 2025-12-18 00:06:24,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f52575bc060ce162b17'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:24,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ef1575bc060ce162ab1 2025-12-18 00:06:24,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef1575bc060ce162ab1 HTTP/1.1" 502 150 2025-12-18 00:06:24,434 - lava_callback - ERROR - Error processing callback for node 69409ef1575bc060ce162ab1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ef1575bc060ce162ab1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ef1575bc060ce162ab1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ef1575bc060ce162ab1 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:24,436 - lava_callback - INFO - Starting async job submit for node 69409f4c575bc060ce162b0c 2025-12-18 00:06:24,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:24,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4c575bc060ce162b0c HTTP/1.1" 502 150 2025-12-18 00:06:24,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f4c575bc060ce162b0c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:24,443 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f4c575bc060ce162b0c 2025-12-18 00:06:24,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4c575bc060ce162b0c HTTP/1.1" 502 150 2025-12-18 00:06:24,444 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f4c575bc060ce162b0c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:24,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69412836459ff5fc64d2bf6a 2025-12-18 00:06:24,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69412836459ff5fc64d2bf6a HTTP/1.1" 502 150 2025-12-18 00:06:24,532 - lava_callback - ERROR - Error processing callback for node 69412836459ff5fc64d2bf6a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69412836459ff5fc64d2bf6a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69412836459ff5fc64d2bf6a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69412836459ff5fc64d2bf6a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:24,535 - lava_callback - INFO - Starting async job submit for node 69409f4e575bc060ce162b0f 2025-12-18 00:06:24,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:24,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4e575bc060ce162b0f HTTP/1.1" 502 150 2025-12-18 00:06:24,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f4e575bc060ce162b0f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:24,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f4e575bc060ce162b0f 2025-12-18 00:06:24,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4e575bc060ce162b0f HTTP/1.1" 502 150 2025-12-18 00:06:24,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f4e575bc060ce162b0f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:24,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f50575bc060ce162b14 2025-12-18 00:06:24,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f50575bc060ce162b14 HTTP/1.1" 502 150 2025-12-18 00:06:24,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f50575bc060ce162b14'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:25,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f4a575bc060ce162b09 2025-12-18 00:06:25,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4a575bc060ce162b09 HTTP/1.1" 502 150 2025-12-18 00:06:25,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f4a575bc060ce162b09'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:25,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f40575bc060ce162af2 2025-12-18 00:06:25,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f40575bc060ce162af2 HTTP/1.1" 502 150 2025-12-18 00:06:25,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f40575bc060ce162af2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:26,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f47575bc060ce162aff 2025-12-18 00:06:26,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f47575bc060ce162aff HTTP/1.1" 502 150 2025-12-18 00:06:26,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f47575bc060ce162aff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:26,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f4c575bc060ce162b0c 2025-12-18 00:06:26,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4c575bc060ce162b0c HTTP/1.1" 502 150 2025-12-18 00:06:26,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f4c575bc060ce162b0c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:32800 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:06:26,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f4e575bc060ce162b0f 2025-12-18 00:06:26,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4e575bc060ce162b0f HTTP/1.1" 502 150 2025-12-18 00:06:26,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f4e575bc060ce162b0f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:26,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ef3575bc060ce162ab4 2025-12-18 00:06:26,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef3575bc060ce162ab4 HTTP/1.1" 502 150 2025-12-18 00:06:26,893 - lava_callback - ERROR - Error processing callback for node 69409ef3575bc060ce162ab4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ef3575bc060ce162ab4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ef3575bc060ce162ab4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ef3575bc060ce162ab4 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:26,895 - lava_callback - INFO - Starting async job submit for node 69409f54575bc060ce162b1a 2025-12-18 00:06:26,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:26,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f54575bc060ce162b1a HTTP/1.1" 502 150 2025-12-18 00:06:26,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f54575bc060ce162b1a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:26,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f54575bc060ce162b1a 2025-12-18 00:06:26,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f54575bc060ce162b1a HTTP/1.1" 502 150 2025-12-18 00:06:26,902 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f54575bc060ce162b1a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:27,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f02575bc060ce162ac3 HTTP/1.1" 502 150 2025-12-18 00:06:27,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f02575bc060ce162ac3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:27,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ef5575bc060ce162ab7 2025-12-18 00:06:27,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ef5575bc060ce162ab7 HTTP/1.1" 502 150 2025-12-18 00:06:27,827 - lava_callback - ERROR - Error processing callback for node 69409ef5575bc060ce162ab7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ef5575bc060ce162ab7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ef5575bc060ce162ab7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ef5575bc060ce162ab7 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:27,828 - lava_callback - INFO - Starting async job submit for node 69409f64575bc060ce162b2b 2025-12-18 00:06:27,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:27,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f64575bc060ce162b2b HTTP/1.1" 502 150 2025-12-18 00:06:27,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f64575bc060ce162b2b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:27,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f64575bc060ce162b2b 2025-12-18 00:06:27,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f64575bc060ce162b2b HTTP/1.1" 502 150 2025-12-18 00:06:27,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f64575bc060ce162b2b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:28,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f52575bc060ce162b17 2025-12-18 00:06:28,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f52575bc060ce162b17 HTTP/1.1" 502 150 2025-12-18 00:06:28,399 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f52575bc060ce162b17'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:28,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f54575bc060ce162b1a 2025-12-18 00:06:28,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f54575bc060ce162b1a HTTP/1.1" 502 150 2025-12-18 00:06:28,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f54575bc060ce162b1a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:29,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dea575bc060ce1628f5 HTTP/1.1" 502 150 2025-12-18 00:06:29,552 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dea575bc060ce1628f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:29,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f4a575bc060ce162b09 2025-12-18 00:06:29,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4a575bc060ce162b09 HTTP/1.1" 502 150 2025-12-18 00:06:29,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f4a575bc060ce162b09'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:29,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f64575bc060ce162b2b 2025-12-18 00:06:29,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f64575bc060ce162b2b HTTP/1.1" 502 150 2025-12-18 00:06:29,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f64575bc060ce162b2b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:29,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f49575bc060ce162b06 2025-12-18 00:06:29,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f49575bc060ce162b06 HTTP/1.1" 502 150 2025-12-18 00:06:29,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f49575bc060ce162b06'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:30,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f4c575bc060ce162b0c 2025-12-18 00:06:30,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4c575bc060ce162b0c HTTP/1.1" 502 150 2025-12-18 00:06:30,447 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f4c575bc060ce162b0c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:30,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f4e575bc060ce162b0f 2025-12-18 00:06:30,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4e575bc060ce162b0f HTTP/1.1" 502 150 2025-12-18 00:06:30,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f4e575bc060ce162b0f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:31,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f00575bc060ce162abc 2025-12-18 00:06:31,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f00575bc060ce162abc HTTP/1.1" 502 150 2025-12-18 00:06:31,827 - lava_callback - ERROR - Error processing callback for node 69409f00575bc060ce162abc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f00575bc060ce162abc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f00575bc060ce162abc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f00575bc060ce162abc (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:31,830 - lava_callback - INFO - Starting async job submit for node 69413242459ff5fc64d2c2cf 2025-12-18 00:06:31,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:31,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69413242459ff5fc64d2c2cf HTTP/1.1" 502 150 2025-12-18 00:06:31,871 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69413242459ff5fc64d2c2cf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:31,871 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69413242459ff5fc64d2c2cf 2025-12-18 00:06:31,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69413242459ff5fc64d2c2cf HTTP/1.1" 502 150 2025-12-18 00:06:31,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69413242459ff5fc64d2c2cf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:32,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f54575bc060ce162b1a 2025-12-18 00:06:32,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f54575bc060ce162b1a HTTP/1.1" 502 150 2025-12-18 00:06:32,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f54575bc060ce162b1a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:32,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f50575bc060ce162b14 2025-12-18 00:06:32,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f50575bc060ce162b14 HTTP/1.1" 502 150 2025-12-18 00:06:32,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f50575bc060ce162b14'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:33,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f64575bc060ce162b2b 2025-12-18 00:06:33,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f64575bc060ce162b2b HTTP/1.1" 502 150 2025-12-18 00:06:33,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f64575bc060ce162b2b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:33,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69413242459ff5fc64d2c2cf 2025-12-18 00:06:33,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69413242459ff5fc64d2c2cf HTTP/1.1" 502 150 2025-12-18 00:06:33,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69413242459ff5fc64d2c2cf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:34,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f47575bc060ce162aff 2025-12-18 00:06:34,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f47575bc060ce162aff HTTP/1.1" 502 150 2025-12-18 00:06:34,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f47575bc060ce162aff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:35,505 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f02575bc060ce162ac3 2025-12-18 00:06:35,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f02575bc060ce162ac3 HTTP/1.1" 502 150 2025-12-18 00:06:35,506 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f02575bc060ce162ac3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:36,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f52575bc060ce162b17 2025-12-18 00:06:36,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f52575bc060ce162b17 HTTP/1.1" 502 150 2025-12-18 00:06:36,401 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f52575bc060ce162b17'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34658 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:06:36,902 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f28575bc060ce162ad6 2025-12-18 00:06:36,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f28575bc060ce162ad6 HTTP/1.1" 502 150 2025-12-18 00:06:36,904 - lava_callback - ERROR - Error processing callback for node 69409f28575bc060ce162ad6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f28575bc060ce162ad6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f28575bc060ce162ad6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f28575bc060ce162ad6 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:36,906 - lava_callback - INFO - Starting async job submit for node 69409f7d575bc060ce162b3d 2025-12-18 00:06:36,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:36,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f04575bc060ce162ac6 2025-12-18 00:06:36,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f04575bc060ce162ac6 HTTP/1.1" 502 150 2025-12-18 00:06:36,908 - lava_callback - ERROR - Error processing callback for node 69409f04575bc060ce162ac6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f04575bc060ce162ac6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f04575bc060ce162ac6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f04575bc060ce162ac6 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:36,910 - lava_callback - INFO - Starting async job submit for node 69409f77575bc060ce162b30 2025-12-18 00:06:36,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:36,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f77575bc060ce162b30 HTTP/1.1" 502 150 2025-12-18 00:06:36,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f77575bc060ce162b30'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:36,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f77575bc060ce162b30 2025-12-18 00:06:36,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f77575bc060ce162b30 HTTP/1.1" 502 150 2025-12-18 00:06:36,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f77575bc060ce162b30'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:37,227 - lava_callback - INFO - Received callback for node 69409e53575bc060ce16295a INFO: 172.21.0.1:34672 - "POST /node/69409e53575bc060ce16295a HTTP/1.0" 202 Accepted 2025-12-18 00:06:37,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f4a575bc060ce162b09 2025-12-18 00:06:37,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4a575bc060ce162b09 HTTP/1.1" 502 150 2025-12-18 00:06:37,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f4a575bc060ce162b09'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:37,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69413242459ff5fc64d2c2cf 2025-12-18 00:06:37,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69413242459ff5fc64d2c2cf HTTP/1.1" 502 150 2025-12-18 00:06:37,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69413242459ff5fc64d2c2cf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:38,447 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f4c575bc060ce162b0c 2025-12-18 00:06:38,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4c575bc060ce162b0c HTTP/1.1" 502 150 2025-12-18 00:06:38,448 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f4c575bc060ce162b0c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:38,459 - lava_callback - INFO - Received callback for node 6940a279575bc060ce16306f INFO: 172.21.0.1:34674 - "POST /node/6940a279575bc060ce16306f HTTP/1.0" 202 Accepted 2025-12-18 00:06:38,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f4e575bc060ce162b0f 2025-12-18 00:06:38,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4e575bc060ce162b0f HTTP/1.1" 502 150 2025-12-18 00:06:38,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f4e575bc060ce162b0f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:38,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f77575bc060ce162b30 2025-12-18 00:06:38,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f77575bc060ce162b30 HTTP/1.1" 502 150 2025-12-18 00:06:38,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f77575bc060ce162b30'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:39,100 - lava_callback - INFO - Received callback for node 6940a285575bc060ce163083 INFO: 172.21.0.1:34680 - "POST /node/6940a285575bc060ce163083 HTTP/1.0" 202 Accepted 2025-12-18 00:06:39,768 - lava_callback - INFO - Received callback for node 6940a289575bc060ce163094 INFO: 172.21.0.1:47186 - "POST /node/6940a289575bc060ce163094 HTTP/1.0" 202 Accepted 2025-12-18 00:06:40,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f3c575bc060ce162ae5 2025-12-18 00:06:40,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f3c575bc060ce162ae5 HTTP/1.1" 502 150 2025-12-18 00:06:40,317 - lava_callback - ERROR - Error processing callback for node 69409f3c575bc060ce162ae5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f3c575bc060ce162ae5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f3c575bc060ce162ae5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f3c575bc060ce162ae5 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:40,319 - lava_callback - INFO - Starting async job submit for node 69409f80575bc060ce162b44 2025-12-18 00:06:40,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:40,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f80575bc060ce162b44 HTTP/1.1" 502 150 2025-12-18 00:06:40,327 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f80575bc060ce162b44'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:40,327 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f80575bc060ce162b44 2025-12-18 00:06:40,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f80575bc060ce162b44 HTTP/1.1" 502 150 2025-12-18 00:06:40,328 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f80575bc060ce162b44'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:40,422 - lava_callback - INFO - Received callback for node 6940a28b575bc060ce163098 INFO: 172.21.0.1:47202 - "POST /node/6940a28b575bc060ce163098 HTTP/1.0" 202 Accepted 2025-12-18 00:06:40,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f54575bc060ce162b1a 2025-12-18 00:06:40,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f54575bc060ce162b1a HTTP/1.1" 502 150 2025-12-18 00:06:40,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f54575bc060ce162b1a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:41,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f64575bc060ce162b2b 2025-12-18 00:06:41,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f64575bc060ce162b2b HTTP/1.1" 502 150 2025-12-18 00:06:41,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f64575bc060ce162b2b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:41,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f40575bc060ce162af2 2025-12-18 00:06:41,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f40575bc060ce162af2 HTTP/1.1" 502 150 2025-12-18 00:06:41,984 - lava_callback - ERROR - Error processing callback for node 69409f40575bc060ce162af2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f40575bc060ce162af2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f40575bc060ce162af2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f40575bc060ce162af2 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:41,986 - lava_callback - INFO - Starting async job submit for node 69409f84575bc060ce162b4a 2025-12-18 00:06:41,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:41,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f84575bc060ce162b4a HTTP/1.1" 502 150 2025-12-18 00:06:41,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f84575bc060ce162b4a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:41,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f84575bc060ce162b4a 2025-12-18 00:06:41,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f84575bc060ce162b4a HTTP/1.1" 502 150 2025-12-18 00:06:41,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f84575bc060ce162b4a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:42,328 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f80575bc060ce162b44 2025-12-18 00:06:42,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f80575bc060ce162b44 HTTP/1.1" 502 150 2025-12-18 00:06:42,329 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f80575bc060ce162b44'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:42,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f77575bc060ce162b30 2025-12-18 00:06:42,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f77575bc060ce162b30 HTTP/1.1" 502 150 2025-12-18 00:06:42,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f77575bc060ce162b30'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:43,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f84575bc060ce162b4a 2025-12-18 00:06:43,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f84575bc060ce162b4a HTTP/1.1" 502 150 2025-12-18 00:06:43,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f84575bc060ce162b4a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:45,553 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dea575bc060ce1628f5 2025-12-18 00:06:45,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69413242459ff5fc64d2c2cf 2025-12-18 00:06:45,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69413242459ff5fc64d2c2cf HTTP/1.1" 502 150 2025-12-18 00:06:45,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69413242459ff5fc64d2c2cf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:45,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f49575bc060ce162b06 2025-12-18 00:06:45,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f49575bc060ce162b06 HTTP/1.1" 502 150 2025-12-18 00:06:45,934 - lava_callback - ERROR - Error processing callback for node 69409f49575bc060ce162b06: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f49575bc060ce162b06 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f49575bc060ce162b06 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f49575bc060ce162b06 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:45,936 - lava_callback - INFO - Starting async job submit for node 69409f86575bc060ce162b4d 2025-12-18 00:06:45,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:45,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f86575bc060ce162b4d HTTP/1.1" 502 150 2025-12-18 00:06:45,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f86575bc060ce162b4d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:45,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f86575bc060ce162b4d 2025-12-18 00:06:45,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f86575bc060ce162b4d HTTP/1.1" 502 150 2025-12-18 00:06:45,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f86575bc060ce162b4d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:46,330 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f80575bc060ce162b44 2025-12-18 00:06:46,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f80575bc060ce162b44 HTTP/1.1" 502 150 2025-12-18 00:06:46,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f80575bc060ce162b44'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47204 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:06:47,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f86575bc060ce162b4d 2025-12-18 00:06:47,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f86575bc060ce162b4d HTTP/1.1" 502 150 2025-12-18 00:06:47,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f86575bc060ce162b4d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:47,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f84575bc060ce162b4a 2025-12-18 00:06:47,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f84575bc060ce162b4a HTTP/1.1" 502 150 2025-12-18 00:06:47,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f84575bc060ce162b4a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:48,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f50575bc060ce162b14 2025-12-18 00:06:48,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f50575bc060ce162b14 HTTP/1.1" 502 150 2025-12-18 00:06:48,960 - lava_callback - ERROR - Error processing callback for node 69409f50575bc060ce162b14: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f50575bc060ce162b14 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f50575bc060ce162b14 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f50575bc060ce162b14 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:48,962 - lava_callback - INFO - Starting async job submit for node 69409f87575bc060ce162b50 2025-12-18 00:06:48,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:48,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f87575bc060ce162b50 HTTP/1.1" 502 150 2025-12-18 00:06:48,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f87575bc060ce162b50'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:48,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f87575bc060ce162b50 2025-12-18 00:06:48,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f87575bc060ce162b50 HTTP/1.1" 502 150 2025-12-18 00:06:48,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f87575bc060ce162b50'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:50,132 - lava_callback - INFO - Received callback for node 6940a287575bc060ce163091 INFO: 172.21.0.1:57182 - "POST /node/6940a287575bc060ce163091 HTTP/1.0" 202 Accepted 2025-12-18 00:06:50,420 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f47575bc060ce162aff 2025-12-18 00:06:50,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f47575bc060ce162aff HTTP/1.1" 502 150 2025-12-18 00:06:50,421 - lava_callback - ERROR - Error processing callback for node 69409f47575bc060ce162aff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f47575bc060ce162aff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f47575bc060ce162aff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f47575bc060ce162aff (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:50,423 - lava_callback - INFO - Starting async job submit for node 69409f89575bc060ce162b53 2025-12-18 00:06:50,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:50,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f89575bc060ce162b53 HTTP/1.1" 502 150 2025-12-18 00:06:50,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f89575bc060ce162b53'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:50,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f89575bc060ce162b53 2025-12-18 00:06:50,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f89575bc060ce162b53 HTTP/1.1" 502 150 2025-12-18 00:06:50,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f89575bc060ce162b53'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:50,924 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f77575bc060ce162b30 2025-12-18 00:06:50,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f77575bc060ce162b30 HTTP/1.1" 502 150 2025-12-18 00:06:50,925 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f77575bc060ce162b30'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:50,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f87575bc060ce162b50 2025-12-18 00:06:50,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f87575bc060ce162b50 HTTP/1.1" 502 150 2025-12-18 00:06:50,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f87575bc060ce162b50'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:50,992 - lava_callback - INFO - Received callback for node 6940a28d575bc060ce16309b INFO: 172.21.0.1:57186 - "POST /node/6940a28d575bc060ce16309b HTTP/1.0" 202 Accepted 2025-12-18 00:06:51,507 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f02575bc060ce162ac3 2025-12-18 00:06:51,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f02575bc060ce162ac3 HTTP/1.1" 502 150 2025-12-18 00:06:51,508 - lava_callback - ERROR - Error processing callback for node 69409f02575bc060ce162ac3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f02575bc060ce162ac3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f02575bc060ce162ac3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f02575bc060ce162ac3 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:51,510 - lava_callback - INFO - Starting async job submit for node 69409f82575bc060ce162b47 2025-12-18 00:06:51,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:51,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f82575bc060ce162b47 HTTP/1.1" 502 150 2025-12-18 00:06:51,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f82575bc060ce162b47'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:51,527 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f82575bc060ce162b47 2025-12-18 00:06:51,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f82575bc060ce162b47 HTTP/1.1" 502 150 2025-12-18 00:06:51,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f82575bc060ce162b47'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:51,814 - lava_callback - INFO - Received callback for node 6940a290575bc060ce1630a1 INFO: 172.21.0.1:57198 - "POST /node/6940a290575bc060ce1630a1 HTTP/1.0" 202 Accepted 2025-12-18 00:06:51,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f86575bc060ce162b4d 2025-12-18 00:06:51,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f86575bc060ce162b4d HTTP/1.1" 502 150 2025-12-18 00:06:51,955 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f86575bc060ce162b4d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:52,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f7d575bc060ce162b3d HTTP/1.1" 502 150 2025-12-18 00:06:52,336 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f7d575bc060ce162b3d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:52,336 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f7d575bc060ce162b3d 2025-12-18 00:06:52,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f7d575bc060ce162b3d HTTP/1.1" 502 150 2025-12-18 00:06:52,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f7d575bc060ce162b3d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:52,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f52575bc060ce162b17 2025-12-18 00:06:52,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f52575bc060ce162b17 HTTP/1.1" 502 150 2025-12-18 00:06:52,402 - lava_callback - ERROR - Error processing callback for node 69409f52575bc060ce162b17: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f52575bc060ce162b17 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f52575bc060ce162b17 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f52575bc060ce162b17 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:52,404 - lava_callback - INFO - Starting async job submit for node 6940c5481e3c24e95681d6c1 2025-12-18 00:06:52,439 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f89575bc060ce162b53 2025-12-18 00:06:52,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:52,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f89575bc060ce162b53 HTTP/1.1" 502 150 2025-12-18 00:06:52,451 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f89575bc060ce162b53'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:52,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5481e3c24e95681d6c1 HTTP/1.1" 502 150 2025-12-18 00:06:52,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5481e3c24e95681d6c1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:52,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5481e3c24e95681d6c1 2025-12-18 00:06:52,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5481e3c24e95681d6c1 HTTP/1.1" 502 150 2025-12-18 00:06:52,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5481e3c24e95681d6c1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:52,697 - lava_callback - INFO - Received callback for node 6940a29b575bc060ce1630b1 INFO: 172.21.0.1:57202 - "POST /node/6940a29b575bc060ce1630b1 HTTP/1.0" 202 Accepted 2025-12-18 00:06:53,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f82575bc060ce162b47 2025-12-18 00:06:53,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f82575bc060ce162b47 HTTP/1.1" 502 150 2025-12-18 00:06:53,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f82575bc060ce162b47'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:53,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f4a575bc060ce162b09 2025-12-18 00:06:53,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4a575bc060ce162b09 HTTP/1.1" 502 150 2025-12-18 00:06:53,782 - lava_callback - ERROR - Error processing callback for node 69409f4a575bc060ce162b09: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f4a575bc060ce162b09 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f4a575bc060ce162b09 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f4a575bc060ce162b09 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:53,785 - lava_callback - INFO - Starting async job submit for node 69409f8b575bc060ce162b58 2025-12-18 00:06:53,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:53,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8b575bc060ce162b58 HTTP/1.1" 502 150 2025-12-18 00:06:53,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f8b575bc060ce162b58'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:53,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f8b575bc060ce162b58 2025-12-18 00:06:53,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8b575bc060ce162b58 HTTP/1.1" 502 150 2025-12-18 00:06:53,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f8b575bc060ce162b58'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:54,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f80575bc060ce162b44 2025-12-18 00:06:54,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f80575bc060ce162b44 HTTP/1.1" 502 150 2025-12-18 00:06:54,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f80575bc060ce162b44'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:54,338 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f7d575bc060ce162b3d 2025-12-18 00:06:54,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f7d575bc060ce162b3d HTTP/1.1" 502 150 2025-12-18 00:06:54,339 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f7d575bc060ce162b3d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:54,448 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f4c575bc060ce162b0c 2025-12-18 00:06:54,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4c575bc060ce162b0c HTTP/1.1" 502 150 2025-12-18 00:06:54,450 - lava_callback - ERROR - Error processing callback for node 69409f4c575bc060ce162b0c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f4c575bc060ce162b0c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f4c575bc060ce162b0c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f4c575bc060ce162b0c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:54,451 - lava_callback - INFO - Starting async job submit for node 69409f8d575bc060ce162b5f 2025-12-18 00:06:54,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:54,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5481e3c24e95681d6c1 2025-12-18 00:06:54,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5481e3c24e95681d6c1 HTTP/1.1" 502 150 2025-12-18 00:06:54,462 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5481e3c24e95681d6c1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:54,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8d575bc060ce162b5f HTTP/1.1" 502 150 2025-12-18 00:06:54,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f8d575bc060ce162b5f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:54,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f8d575bc060ce162b5f 2025-12-18 00:06:54,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8d575bc060ce162b5f HTTP/1.1" 502 150 2025-12-18 00:06:54,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f8d575bc060ce162b5f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:54,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f4e575bc060ce162b0f 2025-12-18 00:06:54,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f4e575bc060ce162b0f HTTP/1.1" 502 150 2025-12-18 00:06:54,549 - lava_callback - ERROR - Error processing callback for node 69409f4e575bc060ce162b0f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f4e575bc060ce162b0f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f4e575bc060ce162b0f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f4e575bc060ce162b0f (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:54,550 - lava_callback - INFO - Starting async job submit for node 69409f8f575bc060ce162b62 2025-12-18 00:06:54,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:54,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8f575bc060ce162b62 HTTP/1.1" 502 150 2025-12-18 00:06:54,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f8f575bc060ce162b62'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:54,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f8f575bc060ce162b62 2025-12-18 00:06:54,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8f575bc060ce162b62 HTTP/1.1" 502 150 2025-12-18 00:06:54,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f8f575bc060ce162b62'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:54,971 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f87575bc060ce162b50 2025-12-18 00:06:54,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f87575bc060ce162b50 HTTP/1.1" 502 150 2025-12-18 00:06:54,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f87575bc060ce162b50'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:55,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f8b575bc060ce162b58 2025-12-18 00:06:55,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8b575bc060ce162b58 HTTP/1.1" 502 150 2025-12-18 00:06:55,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f8b575bc060ce162b58'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:55,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f84575bc060ce162b4a 2025-12-18 00:06:55,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f84575bc060ce162b4a HTTP/1.1" 502 150 2025-12-18 00:06:55,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f84575bc060ce162b4a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:56,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f89575bc060ce162b53 2025-12-18 00:06:56,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f89575bc060ce162b53 HTTP/1.1" 502 150 2025-12-18 00:06:56,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f89575bc060ce162b53'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:56,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f8d575bc060ce162b5f 2025-12-18 00:06:56,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8d575bc060ce162b5f HTTP/1.1" 502 150 2025-12-18 00:06:56,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f8d575bc060ce162b5f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:57214 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:06:56,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f8f575bc060ce162b62 2025-12-18 00:06:56,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8f575bc060ce162b62 HTTP/1.1" 502 150 2025-12-18 00:06:56,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f8f575bc060ce162b62'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:56,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f54575bc060ce162b1a 2025-12-18 00:06:56,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f54575bc060ce162b1a HTTP/1.1" 502 150 2025-12-18 00:06:56,908 - lava_callback - ERROR - Error processing callback for node 69409f54575bc060ce162b1a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f54575bc060ce162b1a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f54575bc060ce162b1a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f54575bc060ce162b1a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:56,910 - lava_callback - INFO - Starting async job submit for node 69409f9f575bc060ce162b6c 2025-12-18 00:06:56,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:56,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f9f575bc060ce162b6c HTTP/1.1" 502 150 2025-12-18 00:06:56,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f9f575bc060ce162b6c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:56,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f9f575bc060ce162b6c 2025-12-18 00:06:56,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f9f575bc060ce162b6c HTTP/1.1" 502 150 2025-12-18 00:06:56,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f9f575bc060ce162b6c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:57,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f82575bc060ce162b47 2025-12-18 00:06:57,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f82575bc060ce162b47 HTTP/1.1" 502 150 2025-12-18 00:06:57,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f82575bc060ce162b47'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:57,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f64575bc060ce162b2b 2025-12-18 00:06:57,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f64575bc060ce162b2b HTTP/1.1" 502 150 2025-12-18 00:06:57,845 - lava_callback - ERROR - Error processing callback for node 69409f64575bc060ce162b2b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f64575bc060ce162b2b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f64575bc060ce162b2b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f64575bc060ce162b2b (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:06:57,848 - lava_callback - INFO - Starting async job submit for node 69409fa1575bc060ce162b73 2025-12-18 00:06:57,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:06:57,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa1575bc060ce162b73 HTTP/1.1" 502 150 2025-12-18 00:06:57,864 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa1575bc060ce162b73'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:57,864 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa1575bc060ce162b73 2025-12-18 00:06:57,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa1575bc060ce162b73 HTTP/1.1" 502 150 2025-12-18 00:06:57,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa1575bc060ce162b73'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:58,339 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f7d575bc060ce162b3d 2025-12-18 00:06:58,462 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5481e3c24e95681d6c1 2025-12-18 00:06:58,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5481e3c24e95681d6c1 HTTP/1.1" 502 150 2025-12-18 00:06:58,464 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5481e3c24e95681d6c1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:58,507 - lava_callback - INFO - Received callback for node 6940a28e575bc060ce16309e INFO: 172.21.0.1:57228 - "POST /node/6940a28e575bc060ce16309e HTTP/1.0" 202 Accepted 2025-12-18 00:06:58,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f9f575bc060ce162b6c 2025-12-18 00:06:58,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f9f575bc060ce162b6c HTTP/1.1" 502 150 2025-12-18 00:06:58,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f9f575bc060ce162b6c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:59,169 - lava_callback - INFO - Received callback for node 6940a299575bc060ce1630ac INFO: 172.21.0.1:57242 - "POST /node/6940a299575bc060ce1630ac HTTP/1.0" 202 Accepted 2025-12-18 00:06:59,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f8b575bc060ce162b58 2025-12-18 00:06:59,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa1575bc060ce162b73 2025-12-18 00:06:59,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa1575bc060ce162b73 HTTP/1.1" 502 150 2025-12-18 00:06:59,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa1575bc060ce162b73'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:06:59,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f86575bc060ce162b4d 2025-12-18 00:06:59,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f86575bc060ce162b4d HTTP/1.1" 502 150 2025-12-18 00:06:59,962 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f86575bc060ce162b4d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:00,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f8d575bc060ce162b5f 2025-12-18 00:07:00,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8d575bc060ce162b5f HTTP/1.1" 502 150 2025-12-18 00:07:00,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f8d575bc060ce162b5f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:00,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f8f575bc060ce162b62 2025-12-18 00:07:00,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8f575bc060ce162b62 HTTP/1.1" 502 150 2025-12-18 00:07:00,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f8f575bc060ce162b62'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:01,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dea575bc060ce1628f5 HTTP/1.1" 502 150 2025-12-18 00:07:01,040 - lava_callback - ERROR - Error processing callback for node 69409dea575bc060ce1628f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dea575bc060ce1628f5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dea575bc060ce1628f5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dea575bc060ce1628f5 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:01,042 - lava_callback - INFO - Starting async job submit for node 69409fa2575bc060ce162b7a 2025-12-18 00:07:01,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:01,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa2575bc060ce162b7a HTTP/1.1" 502 150 2025-12-18 00:07:01,085 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa2575bc060ce162b7a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:01,085 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa2575bc060ce162b7a 2025-12-18 00:07:01,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa2575bc060ce162b7a HTTP/1.1" 502 150 2025-12-18 00:07:01,086 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa2575bc060ce162b7a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:01,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69413242459ff5fc64d2c2cf 2025-12-18 00:07:01,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69413242459ff5fc64d2c2cf HTTP/1.1" 502 150 2025-12-18 00:07:01,878 - lava_callback - ERROR - Error processing callback for node 69413242459ff5fc64d2c2cf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69413242459ff5fc64d2c2cf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69413242459ff5fc64d2c2cf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69413242459ff5fc64d2c2cf (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:01,880 - lava_callback - INFO - Starting async job submit for node 69409fa3575bc060ce162b7f 2025-12-18 00:07:01,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:01,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa3575bc060ce162b7f HTTP/1.1" 502 150 2025-12-18 00:07:01,889 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa3575bc060ce162b7f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:01,889 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa3575bc060ce162b7f 2025-12-18 00:07:01,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa3575bc060ce162b7f HTTP/1.1" 502 150 2025-12-18 00:07:01,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa3575bc060ce162b7f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:02,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f9f575bc060ce162b6c 2025-12-18 00:07:02,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f9f575bc060ce162b6c HTTP/1.1" 502 150 2025-12-18 00:07:02,921 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f9f575bc060ce162b6c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:02,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f87575bc060ce162b50 2025-12-18 00:07:02,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f87575bc060ce162b50 HTTP/1.1" 502 150 2025-12-18 00:07:02,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f87575bc060ce162b50'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:03,086 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa2575bc060ce162b7a 2025-12-18 00:07:03,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa2575bc060ce162b7a HTTP/1.1" 502 150 2025-12-18 00:07:03,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa2575bc060ce162b7a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:03,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa1575bc060ce162b73 2025-12-18 00:07:03,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa1575bc060ce162b73 HTTP/1.1" 502 150 2025-12-18 00:07:03,869 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa1575bc060ce162b73'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34240 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:07:03,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa3575bc060ce162b7f 2025-12-18 00:07:03,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa3575bc060ce162b7f HTTP/1.1" 502 150 2025-12-18 00:07:03,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa3575bc060ce162b7f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:04,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f89575bc060ce162b53 2025-12-18 00:07:05,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f82575bc060ce162b47 2025-12-18 00:07:05,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f82575bc060ce162b47 HTTP/1.1" 502 150 2025-12-18 00:07:05,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f82575bc060ce162b47'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:06,464 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5481e3c24e95681d6c1 2025-12-18 00:07:06,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5481e3c24e95681d6c1 HTTP/1.1" 502 150 2025-12-18 00:07:06,465 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940c5481e3c24e95681d6c1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34256 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:07:06,925 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f77575bc060ce162b30 2025-12-18 00:07:06,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f77575bc060ce162b30 HTTP/1.1" 502 150 2025-12-18 00:07:06,929 - lava_callback - ERROR - Error processing callback for node 69409f77575bc060ce162b30: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f77575bc060ce162b30 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f77575bc060ce162b30 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f77575bc060ce162b30 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:06,931 - lava_callback - INFO - Starting async job submit for node 69409fa5575bc060ce162b82 2025-12-18 00:07:06,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:06,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa5575bc060ce162b82 HTTP/1.1" 502 150 2025-12-18 00:07:06,940 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa5575bc060ce162b82'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:06,940 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa5575bc060ce162b82 2025-12-18 00:07:06,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa5575bc060ce162b82 HTTP/1.1" 502 150 2025-12-18 00:07:06,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa5575bc060ce162b82'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:07,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa2575bc060ce162b7a 2025-12-18 00:07:07,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa2575bc060ce162b7a HTTP/1.1" 502 150 2025-12-18 00:07:07,089 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa2575bc060ce162b7a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:07,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa3575bc060ce162b7f 2025-12-18 00:07:07,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa3575bc060ce162b7f HTTP/1.1" 502 150 2025-12-18 00:07:07,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa3575bc060ce162b7f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:08,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f8d575bc060ce162b5f 2025-12-18 00:07:08,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8d575bc060ce162b5f HTTP/1.1" 502 150 2025-12-18 00:07:08,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f8d575bc060ce162b5f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:08,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f8f575bc060ce162b62 2025-12-18 00:07:08,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8f575bc060ce162b62 HTTP/1.1" 502 150 2025-12-18 00:07:08,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f8f575bc060ce162b62'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:08,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa5575bc060ce162b82 2025-12-18 00:07:08,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa5575bc060ce162b82 HTTP/1.1" 502 150 2025-12-18 00:07:08,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa5575bc060ce162b82'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:10,085 - lava_callback - INFO - Received callback for node 6940a29c575bc060ce1630b4 INFO: 172.21.0.1:55358 - "POST /node/6940a29c575bc060ce1630b4 HTTP/1.0" 202 Accepted 2025-12-18 00:07:10,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f80575bc060ce162b44 2025-12-18 00:07:10,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f80575bc060ce162b44 HTTP/1.1" 502 150 2025-12-18 00:07:10,334 - lava_callback - ERROR - Error processing callback for node 69409f80575bc060ce162b44: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f80575bc060ce162b44 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f80575bc060ce162b44 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f80575bc060ce162b44 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:10,336 - lava_callback - INFO - Starting async job submit for node 69409fa6575bc060ce162b85 2025-12-18 00:07:10,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:10,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa6575bc060ce162b85 HTTP/1.1" 502 150 2025-12-18 00:07:10,343 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa6575bc060ce162b85'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:10,343 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa6575bc060ce162b85 2025-12-18 00:07:10,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa6575bc060ce162b85 HTTP/1.1" 502 150 2025-12-18 00:07:10,343 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa6575bc060ce162b85'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:10,921 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f9f575bc060ce162b6c 2025-12-18 00:07:10,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f9f575bc060ce162b6c HTTP/1.1" 502 150 2025-12-18 00:07:10,923 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f9f575bc060ce162b6c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:11,869 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa1575bc060ce162b73 2025-12-18 00:07:11,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f84575bc060ce162b4a 2025-12-18 00:07:12,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f84575bc060ce162b4a HTTP/1.1" 502 150 2025-12-18 00:07:12,000 - lava_callback - ERROR - Error processing callback for node 69409f84575bc060ce162b4a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f84575bc060ce162b4a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f84575bc060ce162b4a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f84575bc060ce162b4a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:12,002 - lava_callback - INFO - Starting async job submit for node 69409fb3575bc060ce162b8a 2025-12-18 00:07:12,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:12,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb3575bc060ce162b8a HTTP/1.1" 502 150 2025-12-18 00:07:12,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb3575bc060ce162b8a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:12,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb3575bc060ce162b8a 2025-12-18 00:07:12,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb3575bc060ce162b8a HTTP/1.1" 502 150 2025-12-18 00:07:12,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb3575bc060ce162b8a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:12,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa6575bc060ce162b85 2025-12-18 00:07:12,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa6575bc060ce162b85 HTTP/1.1" 502 150 2025-12-18 00:07:12,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa6575bc060ce162b85'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:12,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa5575bc060ce162b82 2025-12-18 00:07:12,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa5575bc060ce162b82 HTTP/1.1" 502 150 2025-12-18 00:07:12,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa5575bc060ce162b82'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:13,899 - lava_callback - INFO - Received callback for node 6940a2a7575bc060ce1630da INFO: 172.21.0.1:55360 - "POST /node/6940a2a7575bc060ce1630da HTTP/1.0" 202 Accepted 2025-12-18 00:07:14,010 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb3575bc060ce162b8a 2025-12-18 00:07:14,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb3575bc060ce162b8a HTTP/1.1" 502 150 2025-12-18 00:07:14,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb3575bc060ce162b8a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:14,608 - lava_callback - INFO - Received callback for node 6940a2a1575bc060ce1630cc INFO: 172.21.0.1:55362 - "POST /node/6940a2a1575bc060ce1630cc HTTP/1.0" 202 Accepted 2025-12-18 00:07:15,089 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa2575bc060ce162b7a 2025-12-18 00:07:15,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa2575bc060ce162b7a HTTP/1.1" 502 150 2025-12-18 00:07:15,090 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa2575bc060ce162b7a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:15,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8b575bc060ce162b58 HTTP/1.1" 502 150 2025-12-18 00:07:15,120 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f8b575bc060ce162b58'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55372 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-18 00:07:15,326 - lava_callback - INFO - Received callback for node 6940a2a3575bc060ce1630cf INFO: 172.21.0.1:55384 - "POST /node/6940a2a3575bc060ce1630cf HTTP/1.0" 202 Accepted 2025-12-18 00:07:15,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa3575bc060ce162b7f 2025-12-18 00:07:15,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa3575bc060ce162b7f HTTP/1.1" 502 150 2025-12-18 00:07:15,894 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa3575bc060ce162b7f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:15,962 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f86575bc060ce162b4d 2025-12-18 00:07:15,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f86575bc060ce162b4d HTTP/1.1" 502 150 2025-12-18 00:07:15,963 - lava_callback - ERROR - Error processing callback for node 69409f86575bc060ce162b4d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f86575bc060ce162b4d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f86575bc060ce162b4d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f86575bc060ce162b4d (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:15,965 - lava_callback - INFO - Starting async job submit for node 69409fb5575bc060ce162b95 2025-12-18 00:07:15,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:15,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb5575bc060ce162b95 HTTP/1.1" 502 150 2025-12-18 00:07:15,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb5575bc060ce162b95'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:15,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb5575bc060ce162b95 2025-12-18 00:07:15,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb5575bc060ce162b95 HTTP/1.1" 502 150 2025-12-18 00:07:15,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb5575bc060ce162b95'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:16,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa6575bc060ce162b85 INFO: 172.21.0.1:55394 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:07:17,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb5575bc060ce162b95 2025-12-18 00:07:17,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb5575bc060ce162b95 HTTP/1.1" 502 150 2025-12-18 00:07:17,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb5575bc060ce162b95'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:18,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb3575bc060ce162b8a 2025-12-18 00:07:18,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb3575bc060ce162b8a HTTP/1.1" 502 150 2025-12-18 00:07:18,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb3575bc060ce162b8a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:18,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f87575bc060ce162b50 2025-12-18 00:07:18,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f87575bc060ce162b50 HTTP/1.1" 502 150 2025-12-18 00:07:18,975 - lava_callback - ERROR - Error processing callback for node 69409f87575bc060ce162b50: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f87575bc060ce162b50 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f87575bc060ce162b50 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f87575bc060ce162b50 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:18,977 - lava_callback - INFO - Starting async job submit for node 69409fb6575bc060ce162b98 2025-12-18 00:07:18,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:18,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb6575bc060ce162b98 HTTP/1.1" 502 150 2025-12-18 00:07:18,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb6575bc060ce162b98'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:18,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb6575bc060ce162b98 2025-12-18 00:07:18,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb6575bc060ce162b98 HTTP/1.1" 502 150 2025-12-18 00:07:18,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb6575bc060ce162b98'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:19,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f89575bc060ce162b53 HTTP/1.1" 502 150 2025-12-18 00:07:19,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f89575bc060ce162b53'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:20,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa5575bc060ce162b82 2025-12-18 00:07:20,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa5575bc060ce162b82 HTTP/1.1" 502 150 2025-12-18 00:07:20,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa5575bc060ce162b82'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:20,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb6575bc060ce162b98 2025-12-18 00:07:20,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb6575bc060ce162b98 HTTP/1.1" 502 150 2025-12-18 00:07:20,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb6575bc060ce162b98'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:21,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f82575bc060ce162b47 2025-12-18 00:07:21,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f82575bc060ce162b47 HTTP/1.1" 502 150 2025-12-18 00:07:21,534 - lava_callback - ERROR - Error processing callback for node 69409f82575bc060ce162b47: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f82575bc060ce162b47 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f82575bc060ce162b47 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f82575bc060ce162b47 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:21,536 - lava_callback - INFO - Starting async job submit for node 69409fb8575bc060ce162b9b 2025-12-18 00:07:21,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:21,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb8575bc060ce162b9b HTTP/1.1" 502 150 2025-12-18 00:07:21,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb8575bc060ce162b9b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:21,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb8575bc060ce162b9b 2025-12-18 00:07:21,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb8575bc060ce162b9b HTTP/1.1" 502 150 2025-12-18 00:07:21,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb8575bc060ce162b9b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:21,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb5575bc060ce162b95 2025-12-18 00:07:21,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb5575bc060ce162b95 HTTP/1.1" 502 150 2025-12-18 00:07:21,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb5575bc060ce162b95'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:22,465 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940c5481e3c24e95681d6c1 2025-12-18 00:07:22,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940c5481e3c24e95681d6c1 HTTP/1.1" 502 150 2025-12-18 00:07:22,467 - lava_callback - ERROR - Error processing callback for node 6940c5481e3c24e95681d6c1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5481e3c24e95681d6c1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5481e3c24e95681d6c1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940c5481e3c24e95681d6c1 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:22,469 - lava_callback - INFO - Starting async job submit for node 69409fb9575bc060ce162ba2 2025-12-18 00:07:22,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:22,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb9575bc060ce162ba2 HTTP/1.1" 502 150 2025-12-18 00:07:22,478 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb9575bc060ce162ba2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:22,478 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb9575bc060ce162ba2 2025-12-18 00:07:22,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb9575bc060ce162ba2 HTTP/1.1" 502 150 2025-12-18 00:07:22,479 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb9575bc060ce162ba2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:23,120 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f8b575bc060ce162b58 2025-12-18 00:07:23,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8b575bc060ce162b58 HTTP/1.1" 502 150 2025-12-18 00:07:23,121 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f8b575bc060ce162b58'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:23,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb8575bc060ce162b9b 2025-12-18 00:07:23,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb8575bc060ce162b9b HTTP/1.1" 502 150 2025-12-18 00:07:23,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb8575bc060ce162b9b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:24,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f8d575bc060ce162b5f 2025-12-18 00:07:24,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8d575bc060ce162b5f HTTP/1.1" 502 150 2025-12-18 00:07:24,470 - lava_callback - ERROR - Error processing callback for node 69409f8d575bc060ce162b5f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f8d575bc060ce162b5f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f8d575bc060ce162b5f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f8d575bc060ce162b5f (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:24,473 - lava_callback - INFO - Starting async job submit for node 69409fc7575bc060ce162ba7 2025-12-18 00:07:24,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:24,481 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb9575bc060ce162ba2 2025-12-18 00:07:24,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb9575bc060ce162ba2 HTTP/1.1" 502 150 2025-12-18 00:07:24,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb9575bc060ce162ba2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:24,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fc7575bc060ce162ba7 HTTP/1.1" 502 150 2025-12-18 00:07:24,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fc7575bc060ce162ba7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:24,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fc7575bc060ce162ba7 2025-12-18 00:07:24,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fc7575bc060ce162ba7 HTTP/1.1" 502 150 2025-12-18 00:07:24,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fc7575bc060ce162ba7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:24,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f8f575bc060ce162b62 2025-12-18 00:07:24,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8f575bc060ce162b62 HTTP/1.1" 502 150 2025-12-18 00:07:24,594 - lava_callback - ERROR - Error processing callback for node 69409f8f575bc060ce162b62: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f8f575bc060ce162b62 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f8f575bc060ce162b62 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f8f575bc060ce162b62 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:24,596 - lava_callback - INFO - Starting async job submit for node 69409fc9575bc060ce162bb0 2025-12-18 00:07:24,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:24,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fc9575bc060ce162bb0 HTTP/1.1" 502 150 2025-12-18 00:07:24,602 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fc9575bc060ce162bb0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:24,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fc9575bc060ce162bb0 2025-12-18 00:07:24,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fc9575bc060ce162bb0 HTTP/1.1" 502 150 2025-12-18 00:07:24,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fc9575bc060ce162bb0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:24,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb6575bc060ce162b98 2025-12-18 00:07:24,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb6575bc060ce162b98 HTTP/1.1" 502 150 2025-12-18 00:07:24,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb6575bc060ce162b98'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:26,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb3575bc060ce162b8a 2025-12-18 00:07:26,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fc7575bc060ce162ba7 2025-12-18 00:07:26,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fc7575bc060ce162ba7 HTTP/1.1" 502 150 2025-12-18 00:07:26,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fc7575bc060ce162ba7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:51680 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:07:26,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fc9575bc060ce162bb0 2025-12-18 00:07:26,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fc9575bc060ce162bb0 HTTP/1.1" 502 150 2025-12-18 00:07:26,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fc9575bc060ce162bb0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:26,923 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f9f575bc060ce162b6c 2025-12-18 00:07:26,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f9f575bc060ce162b6c HTTP/1.1" 502 150 2025-12-18 00:07:26,924 - lava_callback - ERROR - Error processing callback for node 69409f9f575bc060ce162b6c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f9f575bc060ce162b6c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f9f575bc060ce162b6c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f9f575bc060ce162b6c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:26,926 - lava_callback - INFO - Starting async job submit for node 69409fca575bc060ce162bb7 2025-12-18 00:07:26,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:26,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fca575bc060ce162bb7 HTTP/1.1" 502 150 2025-12-18 00:07:26,934 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fca575bc060ce162bb7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:26,934 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fca575bc060ce162bb7 2025-12-18 00:07:26,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fca575bc060ce162bb7 HTTP/1.1" 502 150 2025-12-18 00:07:26,935 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fca575bc060ce162bb7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:27,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa1575bc060ce162b73 HTTP/1.1" 502 150 2025-12-18 00:07:27,156 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa1575bc060ce162b73'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:27,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb8575bc060ce162b9b 2025-12-18 00:07:27,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb8575bc060ce162b9b HTTP/1.1" 502 150 2025-12-18 00:07:27,547 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb8575bc060ce162b9b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:28,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb9575bc060ce162ba2 2025-12-18 00:07:28,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb9575bc060ce162ba2 HTTP/1.1" 502 150 2025-12-18 00:07:28,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb9575bc060ce162ba2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:28,935 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fca575bc060ce162bb7 2025-12-18 00:07:28,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fca575bc060ce162bb7 HTTP/1.1" 502 150 2025-12-18 00:07:28,942 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fca575bc060ce162bb7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:28,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f7d575bc060ce162b3d HTTP/1.1" 502 150 2025-12-18 00:07:28,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f7d575bc060ce162b3d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:29,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb5575bc060ce162b95 2025-12-18 00:07:29,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb5575bc060ce162b95 HTTP/1.1" 502 150 2025-12-18 00:07:29,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb5575bc060ce162b95'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:30,449 - lava_callback - INFO - Received callback for node 6940a2ad575bc060ce1630e5 INFO: 172.21.0.1:58994 - "POST /node/6940a2ad575bc060ce1630e5 HTTP/1.0" 202 Accepted 2025-12-18 00:07:30,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fc7575bc060ce162ba7 2025-12-18 00:07:30,605 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fc9575bc060ce162bb0 2025-12-18 00:07:30,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fc9575bc060ce162bb0 HTTP/1.1" 502 150 2025-12-18 00:07:30,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fc9575bc060ce162bb0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:31,091 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa2575bc060ce162b7a 2025-12-18 00:07:31,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa2575bc060ce162b7a HTTP/1.1" 502 150 2025-12-18 00:07:31,092 - lava_callback - ERROR - Error processing callback for node 69409fa2575bc060ce162b7a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fa2575bc060ce162b7a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fa2575bc060ce162b7a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fa2575bc060ce162b7a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:31,093 - lava_callback - INFO - Starting async job submit for node 69409fcb575bc060ce162bbc 2025-12-18 00:07:31,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:31,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fcb575bc060ce162bbc HTTP/1.1" 502 150 2025-12-18 00:07:31,121 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fcb575bc060ce162bbc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:31,121 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fcb575bc060ce162bbc 2025-12-18 00:07:31,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fcb575bc060ce162bbc HTTP/1.1" 502 150 2025-12-18 00:07:31,122 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fcb575bc060ce162bbc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:31,214 - lava_callback - INFO - Received callback for node 6940a2af575bc060ce1630eb INFO: 172.21.0.1:59006 - "POST /node/6940a2af575bc060ce1630eb HTTP/1.0" 202 Accepted 2025-12-18 00:07:31,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa6575bc060ce162b85 HTTP/1.1" 502 150 2025-12-18 00:07:31,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa6575bc060ce162b85'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:31,894 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa3575bc060ce162b7f 2025-12-18 00:07:31,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa3575bc060ce162b7f HTTP/1.1" 502 150 2025-12-18 00:07:31,896 - lava_callback - ERROR - Error processing callback for node 69409fa3575bc060ce162b7f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fa3575bc060ce162b7f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fa3575bc060ce162b7f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fa3575bc060ce162b7f (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:31,897 - lava_callback - INFO - Starting async job submit for node 69409fcd575bc060ce162bc3 2025-12-18 00:07:31,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:31,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fcd575bc060ce162bc3 HTTP/1.1" 502 150 2025-12-18 00:07:31,906 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fcd575bc060ce162bc3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:31,906 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fcd575bc060ce162bc3 2025-12-18 00:07:31,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fcd575bc060ce162bc3 HTTP/1.1" 502 150 2025-12-18 00:07:31,907 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fcd575bc060ce162bc3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:32,030 - lava_callback - INFO - Received callback for node 6940a2b0575bc060ce1630ee INFO: 172.21.0.1:59010 - "POST /node/6940a2b0575bc060ce1630ee HTTP/1.0" 202 Accepted 2025-12-18 00:07:32,942 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fca575bc060ce162bb7 2025-12-18 00:07:32,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fca575bc060ce162bb7 HTTP/1.1" 502 150 2025-12-18 00:07:32,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fca575bc060ce162bb7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:32,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb6575bc060ce162b98 2025-12-18 00:07:32,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb6575bc060ce162b98 HTTP/1.1" 502 150 2025-12-18 00:07:32,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb6575bc060ce162b98'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:33,122 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fcb575bc060ce162bbc 2025-12-18 00:07:33,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fcb575bc060ce162bbc HTTP/1.1" 502 150 2025-12-18 00:07:33,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fcb575bc060ce162bbc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:33,907 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fcd575bc060ce162bc3 2025-12-18 00:07:33,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fcd575bc060ce162bc3 HTTP/1.1" 502 150 2025-12-18 00:07:33,908 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fcd575bc060ce162bc3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:35,547 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb8575bc060ce162b9b 2025-12-18 00:07:35,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb8575bc060ce162b9b HTTP/1.1" 502 150 2025-12-18 00:07:35,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb8575bc060ce162b9b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:35,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f89575bc060ce162b53 2025-12-18 00:07:35,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f89575bc060ce162b53 HTTP/1.1" 502 150 2025-12-18 00:07:35,730 - lava_callback - ERROR - Error processing callback for node 69409f89575bc060ce162b53: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f89575bc060ce162b53 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f89575bc060ce162b53 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f89575bc060ce162b53 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:35,732 - lava_callback - INFO - Starting async job submit for node 69409fce575bc060ce162bc6 2025-12-18 00:07:35,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:35,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fce575bc060ce162bc6 HTTP/1.1" 502 150 2025-12-18 00:07:35,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fce575bc060ce162bc6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:35,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fce575bc060ce162bc6 2025-12-18 00:07:35,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fce575bc060ce162bc6 HTTP/1.1" 502 150 2025-12-18 00:07:35,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fce575bc060ce162bc6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:36,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb9575bc060ce162ba2 2025-12-18 00:07:36,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb9575bc060ce162ba2 HTTP/1.1" 502 150 2025-12-18 00:07:36,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb9575bc060ce162ba2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59024 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:07:36,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f7d575bc060ce162b3d 2025-12-18 00:07:36,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa5575bc060ce162b82 2025-12-18 00:07:36,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa5575bc060ce162b82 HTTP/1.1" 502 150 2025-12-18 00:07:36,948 - lava_callback - ERROR - Error processing callback for node 69409fa5575bc060ce162b82: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fa5575bc060ce162b82 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fa5575bc060ce162b82 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fa5575bc060ce162b82 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:36,951 - lava_callback - INFO - Starting async job submit for node 69409fd0575bc060ce162bcd 2025-12-18 00:07:36,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:36,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd0575bc060ce162bcd HTTP/1.1" 502 150 2025-12-18 00:07:36,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd0575bc060ce162bcd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:36,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd0575bc060ce162bcd 2025-12-18 00:07:36,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd0575bc060ce162bcd HTTP/1.1" 502 150 2025-12-18 00:07:36,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd0575bc060ce162bcd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:37,124 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fcb575bc060ce162bbc 2025-12-18 00:07:37,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fcb575bc060ce162bbc HTTP/1.1" 502 150 2025-12-18 00:07:37,126 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fcb575bc060ce162bbc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:37,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fce575bc060ce162bc6 2025-12-18 00:07:37,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fce575bc060ce162bc6 HTTP/1.1" 502 150 2025-12-18 00:07:37,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fce575bc060ce162bc6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:37,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fcd575bc060ce162bc3 2025-12-18 00:07:37,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fcd575bc060ce162bc3 HTTP/1.1" 502 150 2025-12-18 00:07:37,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fcd575bc060ce162bc3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:38,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fc9575bc060ce162bb0 2025-12-18 00:07:38,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fc9575bc060ce162bb0 HTTP/1.1" 502 150 2025-12-18 00:07:38,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fc9575bc060ce162bb0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:38,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd0575bc060ce162bcd 2025-12-18 00:07:38,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd0575bc060ce162bcd HTTP/1.1" 502 150 2025-12-18 00:07:38,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd0575bc060ce162bcd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:39,121 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f8b575bc060ce162b58 2025-12-18 00:07:39,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f8b575bc060ce162b58 HTTP/1.1" 502 150 2025-12-18 00:07:39,123 - lava_callback - ERROR - Error processing callback for node 69409f8b575bc060ce162b58: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f8b575bc060ce162b58 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f8b575bc060ce162b58 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f8b575bc060ce162b58 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:39,126 - lava_callback - INFO - Starting async job submit for node 69409fd5575bc060ce162bdd 2025-12-18 00:07:39,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:39,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd5575bc060ce162bdd HTTP/1.1" 502 150 2025-12-18 00:07:39,134 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd5575bc060ce162bdd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:39,134 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd5575bc060ce162bdd 2025-12-18 00:07:39,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd5575bc060ce162bdd HTTP/1.1" 502 150 2025-12-18 00:07:39,135 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd5575bc060ce162bdd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:39,761 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa6575bc060ce162b85 2025-12-18 00:07:39,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa6575bc060ce162b85 HTTP/1.1" 502 150 2025-12-18 00:07:39,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fa6575bc060ce162b85'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:40,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fca575bc060ce162bb7 2025-12-18 00:07:40,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fca575bc060ce162bb7 HTTP/1.1" 502 150 2025-12-18 00:07:40,945 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fca575bc060ce162bb7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:41,135 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd5575bc060ce162bdd 2025-12-18 00:07:41,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd5575bc060ce162bdd HTTP/1.1" 502 150 2025-12-18 00:07:41,137 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd5575bc060ce162bdd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:41,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb3575bc060ce162b8a HTTP/1.1" 502 150 2025-12-18 00:07:41,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fb3575bc060ce162b8a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:41,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fce575bc060ce162bc6 2025-12-18 00:07:41,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fce575bc060ce162bc6 HTTP/1.1" 502 150 2025-12-18 00:07:41,744 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fce575bc060ce162bc6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:42,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd0575bc060ce162bcd 2025-12-18 00:07:43,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd0575bc060ce162bcd HTTP/1.1" 502 150 2025-12-18 00:07:43,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd0575bc060ce162bcd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:43,156 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa1575bc060ce162b73 2025-12-18 00:07:43,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa1575bc060ce162b73 HTTP/1.1" 502 150 2025-12-18 00:07:43,158 - lava_callback - ERROR - Error processing callback for node 69409fa1575bc060ce162b73: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fa1575bc060ce162b73 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fa1575bc060ce162b73 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fa1575bc060ce162b73 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:43,160 - lava_callback - INFO - Starting async job submit for node 69409fd2575bc060ce162bd0 2025-12-18 00:07:43,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:43,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd2575bc060ce162bd0 HTTP/1.1" 502 150 2025-12-18 00:07:43,184 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd2575bc060ce162bd0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:43,184 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd2575bc060ce162bd0 2025-12-18 00:07:43,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd2575bc060ce162bd0 HTTP/1.1" 502 150 2025-12-18 00:07:43,185 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd2575bc060ce162bd0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:45,126 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fcb575bc060ce162bbc 2025-12-18 00:07:45,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fcb575bc060ce162bbc HTTP/1.1" 502 150 2025-12-18 00:07:45,127 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fcb575bc060ce162bbc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:45,137 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd5575bc060ce162bdd 2025-12-18 00:07:45,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd5575bc060ce162bdd HTTP/1.1" 502 150 2025-12-18 00:07:45,138 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd5575bc060ce162bdd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:45,185 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd2575bc060ce162bd0 2025-12-18 00:07:45,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd2575bc060ce162bd0 HTTP/1.1" 502 150 2025-12-18 00:07:45,185 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd2575bc060ce162bd0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:45,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fc7575bc060ce162ba7 HTTP/1.1" 502 150 2025-12-18 00:07:45,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fc7575bc060ce162ba7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:45,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fcd575bc060ce162bc3 2025-12-18 00:07:45,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fcd575bc060ce162bc3 HTTP/1.1" 502 150 2025-12-18 00:07:45,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fcd575bc060ce162bc3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:45,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb5575bc060ce162b95 2025-12-18 00:07:45,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb5575bc060ce162b95 HTTP/1.1" 502 150 2025-12-18 00:07:45,979 - lava_callback - ERROR - Error processing callback for node 69409fb5575bc060ce162b95: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fb5575bc060ce162b95 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fb5575bc060ce162b95 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fb5575bc060ce162b95 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:45,980 - lava_callback - INFO - Starting async job submit for node 69409fd3575bc060ce162bd7 2025-12-18 00:07:45,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:45,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd3575bc060ce162bd7 HTTP/1.1" 502 150 2025-12-18 00:07:45,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd3575bc060ce162bd7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:45,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd3575bc060ce162bd7 2025-12-18 00:07:45,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd3575bc060ce162bd7 HTTP/1.1" 502 150 2025-12-18 00:07:45,988 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd3575bc060ce162bd7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60542 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:07:47,988 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd3575bc060ce162bd7 2025-12-18 00:07:47,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd3575bc060ce162bd7 HTTP/1.1" 502 150 2025-12-18 00:07:47,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd3575bc060ce162bd7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:48,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb6575bc060ce162b98 2025-12-18 00:07:48,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb6575bc060ce162b98 HTTP/1.1" 502 150 2025-12-18 00:07:48,997 - lava_callback - ERROR - Error processing callback for node 69409fb6575bc060ce162b98: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fb6575bc060ce162b98 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fb6575bc060ce162b98 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fb6575bc060ce162b98 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:48,999 - lava_callback - INFO - Starting async job submit for node 69409fde575bc060ce162bef 2025-12-18 00:07:49,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:49,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fde575bc060ce162bef HTTP/1.1" 502 150 2025-12-18 00:07:49,006 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fde575bc060ce162bef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:49,007 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fde575bc060ce162bef 2025-12-18 00:07:49,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fde575bc060ce162bef HTTP/1.1" 502 150 2025-12-18 00:07:49,007 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fde575bc060ce162bef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:49,186 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd2575bc060ce162bd0 2025-12-18 00:07:49,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd2575bc060ce162bd0 HTTP/1.1" 502 150 2025-12-18 00:07:49,187 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd2575bc060ce162bd0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:49,744 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fce575bc060ce162bc6 2025-12-18 00:07:49,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fce575bc060ce162bc6 HTTP/1.1" 502 150 2025-12-18 00:07:49,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fce575bc060ce162bc6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:50,196 - lava_callback - INFO - Received callback for node 69414ece459ff5fc64d2d0d1 INFO: 172.21.0.1:52304 - "POST /node/69414ece459ff5fc64d2d0d1 HTTP/1.0" 202 Accepted 2025-12-18 00:07:51,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd0575bc060ce162bcd 2025-12-18 00:07:51,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd0575bc060ce162bcd HTTP/1.1" 502 150 2025-12-18 00:07:51,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd0575bc060ce162bcd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:51,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fde575bc060ce162bef 2025-12-18 00:07:51,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fde575bc060ce162bef HTTP/1.1" 502 150 2025-12-18 00:07:51,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fde575bc060ce162bef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:51,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb8575bc060ce162b9b 2025-12-18 00:07:51,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb8575bc060ce162b9b HTTP/1.1" 502 150 2025-12-18 00:07:51,550 - lava_callback - ERROR - Error processing callback for node 69409fb8575bc060ce162b9b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fb8575bc060ce162b9b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fb8575bc060ce162b9b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fb8575bc060ce162b9b (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:51,552 - lava_callback - INFO - Starting async job submit for node 69409fe3575bc060ce162bf5 2025-12-18 00:07:51,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:51,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe3575bc060ce162bf5 HTTP/1.1" 502 150 2025-12-18 00:07:51,558 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe3575bc060ce162bf5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:51,558 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe3575bc060ce162bf5 2025-12-18 00:07:51,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe3575bc060ce162bf5 HTTP/1.1" 502 150 2025-12-18 00:07:51,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe3575bc060ce162bf5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:51,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd3575bc060ce162bd7 2025-12-18 00:07:51,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd3575bc060ce162bd7 HTTP/1.1" 502 150 2025-12-18 00:07:51,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd3575bc060ce162bd7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:52,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb9575bc060ce162ba2 2025-12-18 00:07:52,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb9575bc060ce162ba2 HTTP/1.1" 502 150 2025-12-18 00:07:52,487 - lava_callback - ERROR - Error processing callback for node 69409fb9575bc060ce162ba2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fb9575bc060ce162ba2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fb9575bc060ce162ba2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fb9575bc060ce162ba2 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:52,489 - lava_callback - INFO - Starting async job submit for node 69409fdb575bc060ce162be2 2025-12-18 00:07:52,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:52,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fdb575bc060ce162be2 HTTP/1.1" 502 150 2025-12-18 00:07:52,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fdb575bc060ce162be2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:52,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fdb575bc060ce162be2 2025-12-18 00:07:52,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fdb575bc060ce162be2 HTTP/1.1" 502 150 2025-12-18 00:07:52,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fdb575bc060ce162be2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:53,138 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd5575bc060ce162bdd 2025-12-18 00:07:53,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd5575bc060ce162bdd HTTP/1.1" 502 150 2025-12-18 00:07:53,140 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd5575bc060ce162bdd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:53,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe3575bc060ce162bf5 2025-12-18 00:07:53,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe3575bc060ce162bf5 HTTP/1.1" 502 150 2025-12-18 00:07:53,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe3575bc060ce162bf5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:53,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fc7575bc060ce162ba7 2025-12-18 00:07:53,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fc7575bc060ce162ba7 HTTP/1.1" 502 150 2025-12-18 00:07:53,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fc7575bc060ce162ba7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:54,497 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fdb575bc060ce162be2 2025-12-18 00:07:54,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fdb575bc060ce162be2 HTTP/1.1" 502 150 2025-12-18 00:07:54,498 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fdb575bc060ce162be2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:54,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fc9575bc060ce162bb0 2025-12-18 00:07:54,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fc9575bc060ce162bb0 HTTP/1.1" 502 150 2025-12-18 00:07:54,609 - lava_callback - ERROR - Error processing callback for node 69409fc9575bc060ce162bb0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fc9575bc060ce162bb0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fc9575bc060ce162bb0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fc9575bc060ce162bb0 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:54,611 - lava_callback - INFO - Starting async job submit for node 69409fe1575bc060ce162bf2 2025-12-18 00:07:54,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:54,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe1575bc060ce162bf2 HTTP/1.1" 502 150 2025-12-18 00:07:54,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe1575bc060ce162bf2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:54,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe1575bc060ce162bf2 2025-12-18 00:07:54,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe1575bc060ce162bf2 HTTP/1.1" 502 150 2025-12-18 00:07:54,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe1575bc060ce162bf2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:55,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fde575bc060ce162bef 2025-12-18 00:07:55,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fde575bc060ce162bef HTTP/1.1" 502 150 2025-12-18 00:07:55,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fde575bc060ce162bef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:55,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fa6575bc060ce162b85 2025-12-18 00:07:55,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fa6575bc060ce162b85 HTTP/1.1" 502 150 2025-12-18 00:07:55,764 - lava_callback - ERROR - Error processing callback for node 69409fa6575bc060ce162b85: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fa6575bc060ce162b85 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fa6575bc060ce162b85 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fa6575bc060ce162b85 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:55,766 - lava_callback - INFO - Starting async job submit for node 69409fe5575bc060ce162bf8 2025-12-18 00:07:55,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:55,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe5575bc060ce162bf8 HTTP/1.1" 502 150 2025-12-18 00:07:55,773 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe5575bc060ce162bf8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:55,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe5575bc060ce162bf8 2025-12-18 00:07:55,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe5575bc060ce162bf8 HTTP/1.1" 502 150 2025-12-18 00:07:55,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe5575bc060ce162bf8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52320 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:07:56,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe1575bc060ce162bf2 2025-12-18 00:07:56,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe1575bc060ce162bf2 HTTP/1.1" 502 150 2025-12-18 00:07:56,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe1575bc060ce162bf2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:56,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fca575bc060ce162bb7 2025-12-18 00:07:56,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fca575bc060ce162bb7 HTTP/1.1" 502 150 2025-12-18 00:07:56,947 - lava_callback - ERROR - Error processing callback for node 69409fca575bc060ce162bb7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fca575bc060ce162bb7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fca575bc060ce162bb7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fca575bc060ce162bb7 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:56,949 - lava_callback - INFO - Starting async job submit for node 69409fe7575bc060ce162bfb 2025-12-18 00:07:56,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:56,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe7575bc060ce162bfb HTTP/1.1" 502 150 2025-12-18 00:07:56,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe7575bc060ce162bfb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:56,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe7575bc060ce162bfb 2025-12-18 00:07:56,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe7575bc060ce162bfb HTTP/1.1" 502 150 2025-12-18 00:07:56,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe7575bc060ce162bfb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:57,187 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd2575bc060ce162bd0 2025-12-18 00:07:57,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd2575bc060ce162bd0 HTTP/1.1" 502 150 2025-12-18 00:07:57,189 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd2575bc060ce162bd0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:57,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fb3575bc060ce162b8a 2025-12-18 00:07:57,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fb3575bc060ce162b8a HTTP/1.1" 502 150 2025-12-18 00:07:57,490 - lava_callback - ERROR - Error processing callback for node 69409fb3575bc060ce162b8a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fb3575bc060ce162b8a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fb3575bc060ce162b8a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fb3575bc060ce162b8a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:07:57,492 - lava_callback - INFO - Starting async job submit for node 69409ff2575bc060ce162c07 2025-12-18 00:07:57,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:07:57,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff2575bc060ce162c07 HTTP/1.1" 502 150 2025-12-18 00:07:57,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ff2575bc060ce162c07'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:57,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ff2575bc060ce162c07 2025-12-18 00:07:57,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff2575bc060ce162c07 HTTP/1.1" 502 150 2025-12-18 00:07:57,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ff2575bc060ce162c07'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:57,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe3575bc060ce162bf5 2025-12-18 00:07:57,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe3575bc060ce162bf5 HTTP/1.1" 502 150 2025-12-18 00:07:57,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe3575bc060ce162bf5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:57,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe5575bc060ce162bf8 2025-12-18 00:07:57,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe5575bc060ce162bf8 HTTP/1.1" 502 150 2025-12-18 00:07:57,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe5575bc060ce162bf8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:58,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fdb575bc060ce162be2 2025-12-18 00:07:58,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fdb575bc060ce162be2 HTTP/1.1" 502 150 2025-12-18 00:07:58,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fdb575bc060ce162be2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:58,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe7575bc060ce162bfb 2025-12-18 00:07:58,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe7575bc060ce162bfb HTTP/1.1" 502 150 2025-12-18 00:07:58,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe7575bc060ce162bfb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:59,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ff2575bc060ce162c07 2025-12-18 00:07:59,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff2575bc060ce162c07 HTTP/1.1" 502 150 2025-12-18 00:07:59,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ff2575bc060ce162c07'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:07:59,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd3575bc060ce162bd7 2025-12-18 00:07:59,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd3575bc060ce162bd7 HTTP/1.1" 502 150 2025-12-18 00:07:59,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fd3575bc060ce162bd7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:00,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe1575bc060ce162bf2 2025-12-18 00:08:00,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe1575bc060ce162bf2 HTTP/1.1" 502 150 2025-12-18 00:08:00,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe1575bc060ce162bf2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:01,127 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fcb575bc060ce162bbc 2025-12-18 00:08:01,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fcb575bc060ce162bbc HTTP/1.1" 502 150 2025-12-18 00:08:01,129 - lava_callback - ERROR - Error processing callback for node 69409fcb575bc060ce162bbc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fcb575bc060ce162bbc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fcb575bc060ce162bbc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fcb575bc060ce162bbc (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:01,131 - lava_callback - INFO - Starting async job submit for node 69409ff3575bc060ce162c0e 2025-12-18 00:08:01,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:01,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff3575bc060ce162c0e HTTP/1.1" 502 150 2025-12-18 00:08:01,139 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ff3575bc060ce162c0e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:01,139 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ff3575bc060ce162c0e 2025-12-18 00:08:01,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff3575bc060ce162c0e HTTP/1.1" 502 150 2025-12-18 00:08:01,140 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ff3575bc060ce162c0e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:01,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe5575bc060ce162bf8 2025-12-18 00:08:01,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe5575bc060ce162bf8 HTTP/1.1" 502 150 2025-12-18 00:08:01,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe5575bc060ce162bf8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:01,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fcd575bc060ce162bc3 2025-12-18 00:08:01,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fcd575bc060ce162bc3 HTTP/1.1" 502 150 2025-12-18 00:08:01,913 - lava_callback - ERROR - Error processing callback for node 69409fcd575bc060ce162bc3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fcd575bc060ce162bc3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fcd575bc060ce162bc3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fcd575bc060ce162bc3 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:01,915 - lava_callback - INFO - Starting async job submit for node 69409fef575bc060ce162c00 2025-12-18 00:08:01,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:01,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fef575bc060ce162c00 HTTP/1.1" 502 150 2025-12-18 00:08:01,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fef575bc060ce162c00'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:01,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fef575bc060ce162c00 2025-12-18 00:08:01,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fef575bc060ce162c00 HTTP/1.1" 502 150 2025-12-18 00:08:01,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fef575bc060ce162c00'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:02,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe7575bc060ce162bfb 2025-12-18 00:08:02,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe7575bc060ce162bfb HTTP/1.1" 502 150 2025-12-18 00:08:02,960 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe7575bc060ce162bfb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:03,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fde575bc060ce162bef 2025-12-18 00:08:03,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fde575bc060ce162bef HTTP/1.1" 502 150 2025-12-18 00:08:03,014 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fde575bc060ce162bef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:03,140 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ff3575bc060ce162c0e 2025-12-18 00:08:03,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff3575bc060ce162c0e HTTP/1.1" 502 150 2025-12-18 00:08:03,142 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ff3575bc060ce162c0e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:03,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ff2575bc060ce162c07 2025-12-18 00:08:03,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff2575bc060ce162c07 HTTP/1.1" 502 150 2025-12-18 00:08:03,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ff2575bc060ce162c07'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53436 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:08:03,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fef575bc060ce162c00 2025-12-18 00:08:03,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fef575bc060ce162c00 HTTP/1.1" 502 150 2025-12-18 00:08:03,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fef575bc060ce162c00'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:05,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe3575bc060ce162bf5 2025-12-18 00:08:05,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe3575bc060ce162bf5 HTTP/1.1" 502 150 2025-12-18 00:08:05,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe3575bc060ce162bf5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:05,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fce575bc060ce162bc6 2025-12-18 00:08:05,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fce575bc060ce162bc6 HTTP/1.1" 502 150 2025-12-18 00:08:05,755 - lava_callback - ERROR - Error processing callback for node 69409fce575bc060ce162bc6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fce575bc060ce162bc6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fce575bc060ce162bc6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fce575bc060ce162bc6 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:05,756 - lava_callback - INFO - Starting async job submit for node 69409ff7575bc060ce162c13 2025-12-18 00:08:05,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:05,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff7575bc060ce162c13 HTTP/1.1" 502 150 2025-12-18 00:08:05,763 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ff7575bc060ce162c13'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:05,763 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ff7575bc060ce162c13 2025-12-18 00:08:05,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff7575bc060ce162c13 HTTP/1.1" 502 150 2025-12-18 00:08:05,764 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ff7575bc060ce162c13'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:06,112 - lava_callback - INFO - Received callback for node 6942e381e43d2b76e9023a6e INFO: 172.21.0.1:53444 - "POST /node/6942e381e43d2b76e9023a6e HTTP/1.0" 202 Accepted 2025-12-18 00:08:06,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fdb575bc060ce162be2 2025-12-18 00:08:06,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fdb575bc060ce162be2 HTTP/1.1" 502 150 2025-12-18 00:08:06,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fdb575bc060ce162be2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53454 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:08:07,001 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd0575bc060ce162bcd 2025-12-18 00:08:07,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd0575bc060ce162bcd HTTP/1.1" 502 150 2025-12-18 00:08:07,003 - lava_callback - ERROR - Error processing callback for node 69409fd0575bc060ce162bcd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fd0575bc060ce162bcd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fd0575bc060ce162bcd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fd0575bc060ce162bcd (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:07,005 - lava_callback - INFO - Starting async job submit for node 6940a005575bc060ce162c23 2025-12-18 00:08:07,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:07,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a005575bc060ce162c23 HTTP/1.1" 502 150 2025-12-18 00:08:07,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a005575bc060ce162c23'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:07,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a005575bc060ce162c23 2025-12-18 00:08:07,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a005575bc060ce162c23 HTTP/1.1" 502 150 2025-12-18 00:08:07,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a005575bc060ce162c23'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:07,142 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ff3575bc060ce162c0e 2025-12-18 00:08:07,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff3575bc060ce162c0e HTTP/1.1" 502 150 2025-12-18 00:08:07,143 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ff3575bc060ce162c0e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:07,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f7d575bc060ce162b3d HTTP/1.1" 502 150 2025-12-18 00:08:07,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409f7d575bc060ce162b3d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:07,764 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ff7575bc060ce162c13 2025-12-18 00:08:07,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff7575bc060ce162c13 HTTP/1.1" 502 150 2025-12-18 00:08:07,766 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ff7575bc060ce162c13'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:07,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fef575bc060ce162c00 2025-12-18 00:08:07,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fef575bc060ce162c00 HTTP/1.1" 502 150 2025-12-18 00:08:07,953 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fef575bc060ce162c00'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:08,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe1575bc060ce162bf2 2025-12-18 00:08:08,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe1575bc060ce162bf2 HTTP/1.1" 502 150 2025-12-18 00:08:08,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe1575bc060ce162bf2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:09,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a005575bc060ce162c23 2025-12-18 00:08:09,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a005575bc060ce162c23 HTTP/1.1" 502 150 2025-12-18 00:08:09,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a005575bc060ce162c23'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:09,140 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd5575bc060ce162bdd 2025-12-18 00:08:09,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd5575bc060ce162bdd HTTP/1.1" 502 150 2025-12-18 00:08:09,141 - lava_callback - ERROR - Error processing callback for node 69409fd5575bc060ce162bdd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fd5575bc060ce162bdd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fd5575bc060ce162bdd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fd5575bc060ce162bdd (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:09,143 - lava_callback - INFO - Starting async job submit for node 6940a009575bc060ce162c39 2025-12-18 00:08:09,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:09,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a009575bc060ce162c39 HTTP/1.1" 502 150 2025-12-18 00:08:09,151 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a009575bc060ce162c39'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:09,151 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a009575bc060ce162c39 2025-12-18 00:08:09,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a009575bc060ce162c39 HTTP/1.1" 502 150 2025-12-18 00:08:09,151 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a009575bc060ce162c39'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:09,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe5575bc060ce162bf8 2025-12-18 00:08:09,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe5575bc060ce162bf8 HTTP/1.1" 502 150 2025-12-18 00:08:09,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe5575bc060ce162bf8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:09,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fc7575bc060ce162ba7 2025-12-18 00:08:09,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fc7575bc060ce162ba7 HTTP/1.1" 502 150 2025-12-18 00:08:09,844 - lava_callback - ERROR - Error processing callback for node 69409fc7575bc060ce162ba7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fc7575bc060ce162ba7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fc7575bc060ce162ba7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fc7575bc060ce162ba7 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:09,845 - lava_callback - INFO - Starting async job submit for node 6940a7751e3c24e956819ea5 2025-12-18 00:08:09,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:09,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a7751e3c24e956819ea5 HTTP/1.1" 502 150 2025-12-18 00:08:09,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a7751e3c24e956819ea5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:09,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a7751e3c24e956819ea5 2025-12-18 00:08:09,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a7751e3c24e956819ea5 HTTP/1.1" 502 150 2025-12-18 00:08:09,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a7751e3c24e956819ea5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:10,960 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe7575bc060ce162bfb 2025-12-18 00:08:10,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe7575bc060ce162bfb HTTP/1.1" 502 150 2025-12-18 00:08:10,961 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fe7575bc060ce162bfb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:11,152 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a009575bc060ce162c39 2025-12-18 00:08:11,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a009575bc060ce162c39 HTTP/1.1" 502 150 2025-12-18 00:08:11,153 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a009575bc060ce162c39'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:11,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ff2575bc060ce162c07 2025-12-18 00:08:11,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff2575bc060ce162c07 HTTP/1.1" 502 150 2025-12-18 00:08:11,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ff2575bc060ce162c07'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:11,766 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ff7575bc060ce162c13 2025-12-18 00:08:11,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff7575bc060ce162c13 HTTP/1.1" 502 150 2025-12-18 00:08:11,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ff7575bc060ce162c13'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:11,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a7751e3c24e956819ea5 2025-12-18 00:08:11,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a7751e3c24e956819ea5 HTTP/1.1" 502 150 2025-12-18 00:08:11,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a7751e3c24e956819ea5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:13,014 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a005575bc060ce162c23 2025-12-18 00:08:13,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a005575bc060ce162c23 HTTP/1.1" 502 150 2025-12-18 00:08:13,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a005575bc060ce162c23'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:13,189 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd2575bc060ce162bd0 2025-12-18 00:08:13,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd2575bc060ce162bd0 HTTP/1.1" 502 150 2025-12-18 00:08:13,190 - lava_callback - ERROR - Error processing callback for node 69409fd2575bc060ce162bd0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fd2575bc060ce162bd0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fd2575bc060ce162bd0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fd2575bc060ce162bd0 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:13,192 - lava_callback - INFO - Starting async job submit for node 6940a003575bc060ce162c18 2025-12-18 00:08:13,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:13,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a003575bc060ce162c18 HTTP/1.1" 502 150 2025-12-18 00:08:13,222 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a003575bc060ce162c18'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:13,222 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a003575bc060ce162c18 2025-12-18 00:08:13,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a003575bc060ce162c18 HTTP/1.1" 502 150 2025-12-18 00:08:13,223 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a003575bc060ce162c18'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:14,050 - lava_callback - INFO - Received callback for node 6940a2de575bc060ce163105 INFO: 172.21.0.1:53818 - "POST /node/6940a2de575bc060ce163105 HTTP/1.0" 202 Accepted 2025-12-18 00:08:15,144 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ff3575bc060ce162c0e 2025-12-18 00:08:15,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff3575bc060ce162c0e HTTP/1.1" 502 150 2025-12-18 00:08:15,150 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ff3575bc060ce162c0e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:15,153 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a009575bc060ce162c39 2025-12-18 00:08:15,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a009575bc060ce162c39 HTTP/1.1" 502 150 2025-12-18 00:08:15,154 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a009575bc060ce162c39'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53826 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-18 00:08:15,223 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a003575bc060ce162c18 2025-12-18 00:08:15,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a003575bc060ce162c18 HTTP/1.1" 502 150 2025-12-18 00:08:15,224 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a003575bc060ce162c18'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:15,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fef575bc060ce162c00 2025-12-18 00:08:15,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fef575bc060ce162c00 HTTP/1.1" 502 150 2025-12-18 00:08:15,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409fef575bc060ce162c00'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:15,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fd3575bc060ce162bd7 2025-12-18 00:08:15,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fd3575bc060ce162bd7 HTTP/1.1" 502 150 2025-12-18 00:08:15,994 - lava_callback - ERROR - Error processing callback for node 69409fd3575bc060ce162bd7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fd3575bc060ce162bd7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fd3575bc060ce162bd7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fd3575bc060ce162bd7 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:15,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a7751e3c24e956819ea5 2025-12-18 00:08:15,996 - lava_callback - INFO - Starting async job submit for node 6940a007575bc060ce162c2e 2025-12-18 00:08:15,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:15,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a7751e3c24e956819ea5 HTTP/1.1" 502 150 2025-12-18 00:08:15,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a7751e3c24e956819ea5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:16,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a007575bc060ce162c2e HTTP/1.1" 502 150 2025-12-18 00:08:16,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a007575bc060ce162c2e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:16,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a007575bc060ce162c2e 2025-12-18 00:08:16,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a007575bc060ce162c2e HTTP/1.1" 502 150 2025-12-18 00:08:16,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a007575bc060ce162c2e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:53838 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:08:18,025 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a007575bc060ce162c2e 2025-12-18 00:08:18,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a007575bc060ce162c2e HTTP/1.1" 502 150 2025-12-18 00:08:18,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a007575bc060ce162c2e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:19,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fde575bc060ce162bef 2025-12-18 00:08:19,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fde575bc060ce162bef HTTP/1.1" 502 150 2025-12-18 00:08:19,016 - lava_callback - ERROR - Error processing callback for node 69409fde575bc060ce162bef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fde575bc060ce162bef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fde575bc060ce162bef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fde575bc060ce162bef (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:19,018 - lava_callback - INFO - Starting async job submit for node 6941359e459ff5fc64d2c36f 2025-12-18 00:08:19,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:19,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941359e459ff5fc64d2c36f HTTP/1.1" 502 150 2025-12-18 00:08:19,071 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941359e459ff5fc64d2c36f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:19,071 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941359e459ff5fc64d2c36f 2025-12-18 00:08:19,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941359e459ff5fc64d2c36f HTTP/1.1" 502 150 2025-12-18 00:08:19,072 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941359e459ff5fc64d2c36f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:19,224 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a003575bc060ce162c18 2025-12-18 00:08:19,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a003575bc060ce162c18 HTTP/1.1" 502 150 2025-12-18 00:08:19,225 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a003575bc060ce162c18'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:19,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ff7575bc060ce162c13 2025-12-18 00:08:19,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff7575bc060ce162c13 HTTP/1.1" 502 150 2025-12-18 00:08:19,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409ff7575bc060ce162c13'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:21,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a005575bc060ce162c23 2025-12-18 00:08:21,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a005575bc060ce162c23 HTTP/1.1" 502 150 2025-12-18 00:08:21,017 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a005575bc060ce162c23'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:21,072 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941359e459ff5fc64d2c36f 2025-12-18 00:08:21,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941359e459ff5fc64d2c36f HTTP/1.1" 502 150 2025-12-18 00:08:21,073 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941359e459ff5fc64d2c36f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:21,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe3575bc060ce162bf5 2025-12-18 00:08:21,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe3575bc060ce162bf5 HTTP/1.1" 502 150 2025-12-18 00:08:21,564 - lava_callback - ERROR - Error processing callback for node 69409fe3575bc060ce162bf5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fe3575bc060ce162bf5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fe3575bc060ce162bf5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fe3575bc060ce162bf5 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:21,566 - lava_callback - INFO - Starting async job submit for node 6940a00a575bc060ce162c3c 2025-12-18 00:08:21,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:21,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00a575bc060ce162c3c HTTP/1.1" 502 150 2025-12-18 00:08:21,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00a575bc060ce162c3c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:21,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00a575bc060ce162c3c 2025-12-18 00:08:21,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00a575bc060ce162c3c HTTP/1.1" 502 150 2025-12-18 00:08:21,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00a575bc060ce162c3c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:22,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a007575bc060ce162c2e 2025-12-18 00:08:22,501 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fdb575bc060ce162be2 2025-12-18 00:08:22,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fdb575bc060ce162be2 HTTP/1.1" 502 150 2025-12-18 00:08:22,502 - lava_callback - ERROR - Error processing callback for node 69409fdb575bc060ce162be2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fdb575bc060ce162be2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fdb575bc060ce162be2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fdb575bc060ce162be2 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:22,504 - lava_callback - INFO - Starting async job submit for node 6940a00b575bc060ce162c3f 2025-12-18 00:08:22,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:22,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00b575bc060ce162c3f HTTP/1.1" 502 150 2025-12-18 00:08:22,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00b575bc060ce162c3f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:22,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00b575bc060ce162c3f 2025-12-18 00:08:22,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00b575bc060ce162c3f HTTP/1.1" 502 150 2025-12-18 00:08:22,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00b575bc060ce162c3f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:23,154 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a009575bc060ce162c39 2025-12-18 00:08:23,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a009575bc060ce162c39 HTTP/1.1" 502 150 2025-12-18 00:08:23,156 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a009575bc060ce162c39'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:23,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00a575bc060ce162c3c 2025-12-18 00:08:23,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00a575bc060ce162c3c HTTP/1.1" 502 150 2025-12-18 00:08:23,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00a575bc060ce162c3c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:23,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409f7d575bc060ce162b3d 2025-12-18 00:08:23,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a7751e3c24e956819ea5 2025-12-18 00:08:23,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a7751e3c24e956819ea5 HTTP/1.1" 502 150 2025-12-18 00:08:23,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a7751e3c24e956819ea5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:24,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00b575bc060ce162c3f 2025-12-18 00:08:24,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00b575bc060ce162c3f HTTP/1.1" 502 150 2025-12-18 00:08:24,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00b575bc060ce162c3f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:24,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe1575bc060ce162bf2 2025-12-18 00:08:24,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe1575bc060ce162bf2 HTTP/1.1" 502 150 2025-12-18 00:08:24,626 - lava_callback - ERROR - Error processing callback for node 69409fe1575bc060ce162bf2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fe1575bc060ce162bf2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fe1575bc060ce162bf2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fe1575bc060ce162bf2 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:24,629 - lava_callback - INFO - Starting async job submit for node 6940a00c575bc060ce162c42 2025-12-18 00:08:24,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:24,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00c575bc060ce162c42 HTTP/1.1" 502 150 2025-12-18 00:08:24,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00c575bc060ce162c42'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:24,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00c575bc060ce162c42 2025-12-18 00:08:24,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00c575bc060ce162c42 HTTP/1.1" 502 150 2025-12-18 00:08:24,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00c575bc060ce162c42'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:25,073 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941359e459ff5fc64d2c36f 2025-12-18 00:08:25,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941359e459ff5fc64d2c36f HTTP/1.1" 502 150 2025-12-18 00:08:25,075 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941359e459ff5fc64d2c36f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:25,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe5575bc060ce162bf8 2025-12-18 00:08:25,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe5575bc060ce162bf8 HTTP/1.1" 502 150 2025-12-18 00:08:25,780 - lava_callback - ERROR - Error processing callback for node 69409fe5575bc060ce162bf8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fe5575bc060ce162bf8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fe5575bc060ce162bf8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fe5575bc060ce162bf8 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:25,782 - lava_callback - INFO - Starting async job submit for node 6940a00e575bc060ce162c45 2025-12-18 00:08:25,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:25,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00e575bc060ce162c45 HTTP/1.1" 502 150 2025-12-18 00:08:25,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00e575bc060ce162c45'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:25,789 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00e575bc060ce162c45 2025-12-18 00:08:25,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00e575bc060ce162c45 HTTP/1.1" 502 150 2025-12-18 00:08:25,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00e575bc060ce162c45'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55366 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:08:26,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00c575bc060ce162c42 2025-12-18 00:08:26,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00c575bc060ce162c42 HTTP/1.1" 502 150 2025-12-18 00:08:26,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00c575bc060ce162c42'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:26,961 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fe7575bc060ce162bfb 2025-12-18 00:08:26,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fe7575bc060ce162bfb HTTP/1.1" 502 150 2025-12-18 00:08:26,962 - lava_callback - ERROR - Error processing callback for node 69409fe7575bc060ce162bfb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fe7575bc060ce162bfb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fe7575bc060ce162bfb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fe7575bc060ce162bfb (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:26,964 - lava_callback - INFO - Starting async job submit for node 6940a00f575bc060ce162c48 2025-12-18 00:08:26,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:26,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00f575bc060ce162c48 HTTP/1.1" 502 150 2025-12-18 00:08:26,972 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00f575bc060ce162c48'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:26,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00f575bc060ce162c48 2025-12-18 00:08:26,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00f575bc060ce162c48 HTTP/1.1" 502 150 2025-12-18 00:08:26,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00f575bc060ce162c48'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:27,226 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a003575bc060ce162c18 2025-12-18 00:08:27,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a003575bc060ce162c18 HTTP/1.1" 502 150 2025-12-18 00:08:27,227 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a003575bc060ce162c18'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:27,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ff2575bc060ce162c07 2025-12-18 00:08:27,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff2575bc060ce162c07 HTTP/1.1" 502 150 2025-12-18 00:08:27,529 - lava_callback - ERROR - Error processing callback for node 69409ff2575bc060ce162c07: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ff2575bc060ce162c07 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ff2575bc060ce162c07 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ff2575bc060ce162c07 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:27,531 - lava_callback - INFO - Starting async job submit for node 6940a010575bc060ce162c4b 2025-12-18 00:08:27,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:27,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a010575bc060ce162c4b HTTP/1.1" 502 150 2025-12-18 00:08:27,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a010575bc060ce162c4b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:27,538 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a010575bc060ce162c4b 2025-12-18 00:08:27,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a010575bc060ce162c4b HTTP/1.1" 502 150 2025-12-18 00:08:27,539 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a010575bc060ce162c4b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:27,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00a575bc060ce162c3c 2025-12-18 00:08:27,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00a575bc060ce162c3c HTTP/1.1" 502 150 2025-12-18 00:08:27,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00a575bc060ce162c3c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:27,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00e575bc060ce162c45 2025-12-18 00:08:27,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00e575bc060ce162c45 HTTP/1.1" 502 150 2025-12-18 00:08:27,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00e575bc060ce162c45'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:28,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00b575bc060ce162c3f 2025-12-18 00:08:28,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00b575bc060ce162c3f HTTP/1.1" 502 150 2025-12-18 00:08:28,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00b575bc060ce162c3f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:28,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00f575bc060ce162c48 2025-12-18 00:08:28,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00f575bc060ce162c48 HTTP/1.1" 502 150 2025-12-18 00:08:28,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00f575bc060ce162c48'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:29,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a010575bc060ce162c4b 2025-12-18 00:08:29,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a010575bc060ce162c4b HTTP/1.1" 502 150 2025-12-18 00:08:29,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a010575bc060ce162c4b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:30,646 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00c575bc060ce162c42 2025-12-18 00:08:30,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00c575bc060ce162c42 HTTP/1.1" 502 150 2025-12-18 00:08:30,647 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00c575bc060ce162c42'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:31,150 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ff3575bc060ce162c0e 2025-12-18 00:08:31,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00e575bc060ce162c45 2025-12-18 00:08:31,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00e575bc060ce162c45 HTTP/1.1" 502 150 2025-12-18 00:08:31,793 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00e575bc060ce162c45'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:31,954 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409fef575bc060ce162c00 2025-12-18 00:08:31,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409fef575bc060ce162c00 HTTP/1.1" 502 150 2025-12-18 00:08:31,956 - lava_callback - ERROR - Error processing callback for node 69409fef575bc060ce162c00: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fef575bc060ce162c00 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fef575bc060ce162c00 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409fef575bc060ce162c00 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:31,958 - lava_callback - INFO - Starting async job submit for node 6940a011575bc060ce162c4f 2025-12-18 00:08:31,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:31,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a011575bc060ce162c4f HTTP/1.1" 502 150 2025-12-18 00:08:31,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a011575bc060ce162c4f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:31,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a011575bc060ce162c4f 2025-12-18 00:08:31,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a011575bc060ce162c4f HTTP/1.1" 502 150 2025-12-18 00:08:31,967 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a011575bc060ce162c4f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:32,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00f575bc060ce162c48 2025-12-18 00:08:32,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00f575bc060ce162c48 HTTP/1.1" 502 150 2025-12-18 00:08:32,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00f575bc060ce162c48'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:33,075 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941359e459ff5fc64d2c36f 2025-12-18 00:08:33,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a010575bc060ce162c4b 2025-12-18 00:08:33,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a010575bc060ce162c4b HTTP/1.1" 502 150 2025-12-18 00:08:33,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a010575bc060ce162c4b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:33,701 - lava_callback - INFO - Received callback for node 6940a2f2575bc060ce16310d INFO: 172.21.0.1:56520 - "POST /node/6940a2f2575bc060ce16310d HTTP/1.0" 202 Accepted 2025-12-18 00:08:33,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a011575bc060ce162c4f 2025-12-18 00:08:33,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a011575bc060ce162c4f HTTP/1.1" 502 150 2025-12-18 00:08:33,969 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a011575bc060ce162c4f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:34,397 - lava_callback - INFO - Received callback for node 6940a2f5575bc060ce163110 INFO: 172.21.0.1:56530 - "POST /node/6940a2f5575bc060ce163110 HTTP/1.0" 202 Accepted 2025-12-18 00:08:35,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00a575bc060ce162c3c 2025-12-18 00:08:35,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00a575bc060ce162c3c HTTP/1.1" 502 150 2025-12-18 00:08:35,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00a575bc060ce162c3c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:35,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409ff7575bc060ce162c13 2025-12-18 00:08:35,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff7575bc060ce162c13 HTTP/1.1" 502 150 2025-12-18 00:08:35,770 - lava_callback - ERROR - Error processing callback for node 69409ff7575bc060ce162c13: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ff7575bc060ce162c13 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ff7575bc060ce162c13 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ff7575bc060ce162c13 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:35,772 - lava_callback - INFO - Starting async job submit for node 6940a019575bc060ce162c63 2025-12-18 00:08:35,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:35,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a019575bc060ce162c63 HTTP/1.1" 502 150 2025-12-18 00:08:35,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a019575bc060ce162c63'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:35,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a019575bc060ce162c63 2025-12-18 00:08:35,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a019575bc060ce162c63 HTTP/1.1" 502 150 2025-12-18 00:08:35,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a019575bc060ce162c63'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56536 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:08:36,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00b575bc060ce162c3f 2025-12-18 00:08:36,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00b575bc060ce162c3f HTTP/1.1" 502 150 2025-12-18 00:08:36,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00b575bc060ce162c3f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:37,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a005575bc060ce162c23 2025-12-18 00:08:37,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a005575bc060ce162c23 HTTP/1.1" 502 150 2025-12-18 00:08:37,019 - lava_callback - ERROR - Error processing callback for node 6940a005575bc060ce162c23: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a005575bc060ce162c23 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a005575bc060ce162c23 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a005575bc060ce162c23 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:37,021 - lava_callback - INFO - Starting async job submit for node 6940a01d575bc060ce162c71 2025-12-18 00:08:37,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:37,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01d575bc060ce162c71 HTTP/1.1" 502 150 2025-12-18 00:08:37,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01d575bc060ce162c71'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:37,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01d575bc060ce162c71 2025-12-18 00:08:37,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01d575bc060ce162c71 HTTP/1.1" 502 150 2025-12-18 00:08:37,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01d575bc060ce162c71'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:37,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a007575bc060ce162c2e HTTP/1.1" 502 150 2025-12-18 00:08:37,296 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a007575bc060ce162c2e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:37,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a019575bc060ce162c63 2025-12-18 00:08:37,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a019575bc060ce162c63 HTTP/1.1" 502 150 2025-12-18 00:08:37,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a019575bc060ce162c63'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:37,969 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a011575bc060ce162c4f 2025-12-18 00:08:37,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a011575bc060ce162c4f HTTP/1.1" 502 150 2025-12-18 00:08:37,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a011575bc060ce162c4f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:38,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00c575bc060ce162c42 2025-12-18 00:08:38,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00c575bc060ce162c42 HTTP/1.1" 502 150 2025-12-18 00:08:38,648 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00c575bc060ce162c42'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:39,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01d575bc060ce162c71 2025-12-18 00:08:39,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01d575bc060ce162c71 HTTP/1.1" 502 150 2025-12-18 00:08:39,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01d575bc060ce162c71'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:39,156 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a009575bc060ce162c39 2025-12-18 00:08:39,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a009575bc060ce162c39 HTTP/1.1" 502 150 2025-12-18 00:08:39,158 - lava_callback - ERROR - Error processing callback for node 6940a009575bc060ce162c39: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a009575bc060ce162c39 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a009575bc060ce162c39 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a009575bc060ce162c39 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:39,160 - lava_callback - INFO - Starting async job submit for node 6940a01f575bc060ce162c77 2025-12-18 00:08:39,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:39,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01f575bc060ce162c77 HTTP/1.1" 502 150 2025-12-18 00:08:39,167 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01f575bc060ce162c77'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:39,167 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01f575bc060ce162c77 2025-12-18 00:08:39,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01f575bc060ce162c77 HTTP/1.1" 502 150 2025-12-18 00:08:39,168 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01f575bc060ce162c77'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:39,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00e575bc060ce162c45 2025-12-18 00:08:39,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00e575bc060ce162c45 HTTP/1.1" 502 150 2025-12-18 00:08:39,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00e575bc060ce162c45'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:39,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a7751e3c24e956819ea5 2025-12-18 00:08:40,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a7751e3c24e956819ea5 HTTP/1.1" 502 150 2025-12-18 00:08:40,001 - lava_callback - ERROR - Error processing callback for node 6940a7751e3c24e956819ea5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a7751e3c24e956819ea5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a7751e3c24e956819ea5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a7751e3c24e956819ea5 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:40,004 - lava_callback - INFO - Starting async job submit for node 6940a017575bc060ce162c54 2025-12-18 00:08:40,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:40,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a017575bc060ce162c54 HTTP/1.1" 502 150 2025-12-18 00:08:40,032 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a017575bc060ce162c54'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:40,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a017575bc060ce162c54 2025-12-18 00:08:40,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a017575bc060ce162c54 HTTP/1.1" 502 150 2025-12-18 00:08:40,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a017575bc060ce162c54'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:40,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00f575bc060ce162c48 2025-12-18 00:08:40,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00f575bc060ce162c48 HTTP/1.1" 502 150 2025-12-18 00:08:40,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a00f575bc060ce162c48'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:41,168 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01f575bc060ce162c77 2025-12-18 00:08:41,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01f575bc060ce162c77 HTTP/1.1" 502 150 2025-12-18 00:08:41,173 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01f575bc060ce162c77'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:41,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a010575bc060ce162c4b 2025-12-18 00:08:41,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a010575bc060ce162c4b HTTP/1.1" 502 150 2025-12-18 00:08:41,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a010575bc060ce162c4b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:41,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a019575bc060ce162c63 2025-12-18 00:08:41,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a019575bc060ce162c63 HTTP/1.1" 502 150 2025-12-18 00:08:41,783 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a019575bc060ce162c63'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:42,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a017575bc060ce162c54 2025-12-18 00:08:42,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a017575bc060ce162c54 HTTP/1.1" 502 150 2025-12-18 00:08:42,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a017575bc060ce162c54'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:43,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01d575bc060ce162c71 2025-12-18 00:08:43,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01d575bc060ce162c71 HTTP/1.1" 502 150 2025-12-18 00:08:43,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01d575bc060ce162c71'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:43,227 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a003575bc060ce162c18 2025-12-18 00:08:43,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a003575bc060ce162c18 HTTP/1.1" 502 150 2025-12-18 00:08:43,229 - lava_callback - ERROR - Error processing callback for node 6940a003575bc060ce162c18: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a003575bc060ce162c18 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a003575bc060ce162c18 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a003575bc060ce162c18 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:43,231 - lava_callback - INFO - Starting async job submit for node 6940a01b575bc060ce162c6e 2025-12-18 00:08:43,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:43,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01b575bc060ce162c6e HTTP/1.1" 502 150 2025-12-18 00:08:43,237 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01b575bc060ce162c6e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:43,237 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01b575bc060ce162c6e 2025-12-18 00:08:43,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01b575bc060ce162c6e HTTP/1.1" 502 150 2025-12-18 00:08:43,238 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01b575bc060ce162c6e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:45,173 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01f575bc060ce162c77 2025-12-18 00:08:45,238 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01b575bc060ce162c6e 2025-12-18 00:08:45,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01b575bc060ce162c6e HTTP/1.1" 502 150 2025-12-18 00:08:45,239 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01b575bc060ce162c6e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:45,297 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a007575bc060ce162c2e 2025-12-18 00:08:45,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a007575bc060ce162c2e HTTP/1.1" 502 150 2025-12-18 00:08:45,298 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a007575bc060ce162c2e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:45,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a011575bc060ce162c4f 2025-12-18 00:08:45,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a011575bc060ce162c4f HTTP/1.1" 502 150 2025-12-18 00:08:45,971 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a011575bc060ce162c4f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:46,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a017575bc060ce162c54 2025-12-18 00:08:46,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a017575bc060ce162c54 HTTP/1.1" 502 150 2025-12-18 00:08:46,035 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a017575bc060ce162c54'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:46,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409ff3575bc060ce162c0e HTTP/1.1" 502 150 2025-12-18 00:08:46,512 - lava_callback - ERROR - Error processing callback for node 69409ff3575bc060ce162c0e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ff3575bc060ce162c0e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ff3575bc060ce162c0e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409ff3575bc060ce162c0e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:46,514 - lava_callback - INFO - Starting async job submit for node 6940a01e575bc060ce162c74 2025-12-18 00:08:46,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:46,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01e575bc060ce162c74 HTTP/1.1" 502 150 2025-12-18 00:08:46,522 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01e575bc060ce162c74'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:46,522 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01e575bc060ce162c74 2025-12-18 00:08:46,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01e575bc060ce162c74 HTTP/1.1" 502 150 2025-12-18 00:08:46,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01e575bc060ce162c74'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:59444 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:08:48,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01e575bc060ce162c74 2025-12-18 00:08:48,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01e575bc060ce162c74 HTTP/1.1" 502 150 2025-12-18 00:08:48,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01e575bc060ce162c74'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:48,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941359e459ff5fc64d2c36f HTTP/1.1" 502 150 2025-12-18 00:08:48,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6941359e459ff5fc64d2c36f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:49,240 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01b575bc060ce162c6e 2025-12-18 00:08:49,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01b575bc060ce162c6e HTTP/1.1" 502 150 2025-12-18 00:08:49,247 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01b575bc060ce162c6e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:49,783 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a019575bc060ce162c63 2025-12-18 00:08:49,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a019575bc060ce162c63 HTTP/1.1" 502 150 2025-12-18 00:08:49,784 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a019575bc060ce162c63'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:50,909 - lava_callback - INFO - Received callback for node 6940a311575bc060ce16311f INFO: 172.21.0.1:39502 - "POST /node/6940a311575bc060ce16311f HTTP/1.0" 202 Accepted 2025-12-18 00:08:51,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01d575bc060ce162c71 2025-12-18 00:08:51,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01d575bc060ce162c71 HTTP/1.1" 502 150 2025-12-18 00:08:51,040 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01d575bc060ce162c71'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:51,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00a575bc060ce162c3c 2025-12-18 00:08:51,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00a575bc060ce162c3c HTTP/1.1" 502 150 2025-12-18 00:08:51,579 - lava_callback - ERROR - Error processing callback for node 6940a00a575bc060ce162c3c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a00a575bc060ce162c3c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a00a575bc060ce162c3c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a00a575bc060ce162c3c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:51,581 - lava_callback - INFO - Starting async job submit for node 6940a020575bc060ce162c7a 2025-12-18 00:08:51,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:51,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a020575bc060ce162c7a HTTP/1.1" 502 150 2025-12-18 00:08:51,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a020575bc060ce162c7a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:51,588 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a020575bc060ce162c7a 2025-12-18 00:08:51,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a020575bc060ce162c7a HTTP/1.1" 502 150 2025-12-18 00:08:51,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a020575bc060ce162c7a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:51,715 - lava_callback - INFO - Received callback for node 6940a312575bc060ce163124 INFO: 172.21.0.1:39516 - "POST /node/6940a312575bc060ce163124 HTTP/1.0" 202 Accepted 2025-12-18 00:08:52,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01e575bc060ce162c74 2025-12-18 00:08:52,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01e575bc060ce162c74 HTTP/1.1" 502 150 2025-12-18 00:08:52,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01e575bc060ce162c74'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:52,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00b575bc060ce162c3f 2025-12-18 00:08:52,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00b575bc060ce162c3f HTTP/1.1" 502 150 2025-12-18 00:08:52,530 - lava_callback - ERROR - Error processing callback for node 6940a00b575bc060ce162c3f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a00b575bc060ce162c3f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a00b575bc060ce162c3f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a00b575bc060ce162c3f (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:52,532 - lava_callback - INFO - Starting async job submit for node 6940a022575bc060ce162c80 2025-12-18 00:08:52,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:52,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a022575bc060ce162c80 HTTP/1.1" 502 150 2025-12-18 00:08:52,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a022575bc060ce162c80'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:52,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a022575bc060ce162c80 2025-12-18 00:08:52,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a022575bc060ce162c80 HTTP/1.1" 502 150 2025-12-18 00:08:52,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a022575bc060ce162c80'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:53,543 - lava_callback - INFO - Received callback for node 6940a306575bc060ce163117 INFO: 172.21.0.1:39522 - "POST /node/6940a306575bc060ce163117 HTTP/1.0" 202 Accepted 2025-12-18 00:08:53,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a020575bc060ce162c7a 2025-12-18 00:08:53,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a020575bc060ce162c7a HTTP/1.1" 502 150 2025-12-18 00:08:53,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a020575bc060ce162c7a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:54,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a017575bc060ce162c54 2025-12-18 00:08:54,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a017575bc060ce162c54 HTTP/1.1" 502 150 2025-12-18 00:08:54,037 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a017575bc060ce162c54'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:54,241 - lava_callback - INFO - Received callback for node 6940a306575bc060ce16311a INFO: 172.21.0.1:39532 - "POST /node/6940a306575bc060ce16311a HTTP/1.0" 202 Accepted 2025-12-18 00:08:54,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409f7d575bc060ce162b3d HTTP/1.1" 502 150 2025-12-18 00:08:54,448 - lava_callback - ERROR - Error processing callback for node 69409f7d575bc060ce162b3d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f7d575bc060ce162b3d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f7d575bc060ce162b3d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409f7d575bc060ce162b3d (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:54,451 - lava_callback - INFO - Starting async job submit for node 6940a021575bc060ce162c7d 2025-12-18 00:08:54,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:54,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a021575bc060ce162c7d HTTP/1.1" 502 150 2025-12-18 00:08:54,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a021575bc060ce162c7d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:54,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a021575bc060ce162c7d 2025-12-18 00:08:54,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a021575bc060ce162c7d HTTP/1.1" 502 150 2025-12-18 00:08:54,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a021575bc060ce162c7d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:54,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a022575bc060ce162c80 2025-12-18 00:08:54,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a022575bc060ce162c80 HTTP/1.1" 502 150 2025-12-18 00:08:54,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a022575bc060ce162c80'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:54,649 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00c575bc060ce162c42 2025-12-18 00:08:54,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00c575bc060ce162c42 HTTP/1.1" 502 150 2025-12-18 00:08:54,650 - lava_callback - ERROR - Error processing callback for node 6940a00c575bc060ce162c42: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a00c575bc060ce162c42 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a00c575bc060ce162c42 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a00c575bc060ce162c42 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:54,652 - lava_callback - INFO - Starting async job submit for node 6940a024575bc060ce162c83 2025-12-18 00:08:54,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:54,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a024575bc060ce162c83 HTTP/1.1" 502 150 2025-12-18 00:08:54,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a024575bc060ce162c83'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:54,678 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a024575bc060ce162c83 2025-12-18 00:08:54,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a024575bc060ce162c83 HTTP/1.1" 502 150 2025-12-18 00:08:54,678 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a024575bc060ce162c83'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:55,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00e575bc060ce162c45 2025-12-18 00:08:55,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00e575bc060ce162c45 HTTP/1.1" 502 150 2025-12-18 00:08:55,796 - lava_callback - ERROR - Error processing callback for node 6940a00e575bc060ce162c45: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a00e575bc060ce162c45 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a00e575bc060ce162c45 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a00e575bc060ce162c45 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:55,798 - lava_callback - INFO - Starting async job submit for node 6940a025575bc060ce162c86 2025-12-18 00:08:55,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:55,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a025575bc060ce162c86 HTTP/1.1" 502 150 2025-12-18 00:08:55,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a025575bc060ce162c86'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:55,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a025575bc060ce162c86 2025-12-18 00:08:55,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a025575bc060ce162c86 HTTP/1.1" 502 150 2025-12-18 00:08:55,806 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a025575bc060ce162c86'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:56,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a021575bc060ce162c7d 2025-12-18 00:08:56,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a021575bc060ce162c7d HTTP/1.1" 502 150 2025-12-18 00:08:56,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a021575bc060ce162c7d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:39548 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:08:56,679 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a024575bc060ce162c83 2025-12-18 00:08:56,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a024575bc060ce162c83 HTTP/1.1" 502 150 2025-12-18 00:08:56,680 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a024575bc060ce162c83'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:56,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a00f575bc060ce162c48 2025-12-18 00:08:56,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a00f575bc060ce162c48 HTTP/1.1" 502 150 2025-12-18 00:08:56,980 - lava_callback - ERROR - Error processing callback for node 6940a00f575bc060ce162c48: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a00f575bc060ce162c48 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a00f575bc060ce162c48 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a00f575bc060ce162c48 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:56,982 - lava_callback - INFO - Starting async job submit for node 6940a02d575bc060ce162c9c 2025-12-18 00:08:56,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:56,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02d575bc060ce162c9c HTTP/1.1" 502 150 2025-12-18 00:08:56,989 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a02d575bc060ce162c9c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:56,989 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a02d575bc060ce162c9c 2025-12-18 00:08:56,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02d575bc060ce162c9c HTTP/1.1" 502 150 2025-12-18 00:08:56,990 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a02d575bc060ce162c9c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:57,247 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01b575bc060ce162c6e 2025-12-18 00:08:57,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a010575bc060ce162c4b 2025-12-18 00:08:57,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a010575bc060ce162c4b HTTP/1.1" 502 150 2025-12-18 00:08:57,545 - lava_callback - ERROR - Error processing callback for node 6940a010575bc060ce162c4b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a010575bc060ce162c4b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a010575bc060ce162c4b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a010575bc060ce162c4b (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:08:57,548 - lava_callback - INFO - Starting async job submit for node 6940a032575bc060ce162ca5 2025-12-18 00:08:57,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:08:57,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a032575bc060ce162ca5 HTTP/1.1" 502 150 2025-12-18 00:08:57,559 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a032575bc060ce162ca5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:57,559 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a032575bc060ce162ca5 2025-12-18 00:08:57,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a032575bc060ce162ca5 HTTP/1.1" 502 150 2025-12-18 00:08:57,560 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a032575bc060ce162ca5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:57,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a020575bc060ce162c7a 2025-12-18 00:08:57,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a020575bc060ce162c7a HTTP/1.1" 502 150 2025-12-18 00:08:57,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a020575bc060ce162c7a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:57,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a025575bc060ce162c86 2025-12-18 00:08:57,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a025575bc060ce162c86 HTTP/1.1" 502 150 2025-12-18 00:08:57,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a025575bc060ce162c86'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:58,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a022575bc060ce162c80 2025-12-18 00:08:58,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a022575bc060ce162c80 HTTP/1.1" 502 150 2025-12-18 00:08:58,563 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a022575bc060ce162c80'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:58,990 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a02d575bc060ce162c9c 2025-12-18 00:08:58,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02d575bc060ce162c9c HTTP/1.1" 502 150 2025-12-18 00:08:58,991 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a02d575bc060ce162c9c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:08:59,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a032575bc060ce162ca5 2025-12-18 00:08:59,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a032575bc060ce162ca5 HTTP/1.1" 502 150 2025-12-18 00:08:59,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a032575bc060ce162ca5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:00,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a021575bc060ce162c7d 2025-12-18 00:09:00,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a021575bc060ce162c7d HTTP/1.1" 502 150 2025-12-18 00:09:00,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a021575bc060ce162c7d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:00,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01e575bc060ce162c74 2025-12-18 00:09:00,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01e575bc060ce162c74 HTTP/1.1" 502 150 2025-12-18 00:09:00,527 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01e575bc060ce162c74'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:00,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01f575bc060ce162c77 HTTP/1.1" 502 150 2025-12-18 00:09:00,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01f575bc060ce162c77'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:00,681 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a024575bc060ce162c83 2025-12-18 00:09:00,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a024575bc060ce162c83 HTTP/1.1" 502 150 2025-12-18 00:09:00,682 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a024575bc060ce162c83'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:01,298 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a007575bc060ce162c2e 2025-12-18 00:09:01,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a007575bc060ce162c2e HTTP/1.1" 502 150 2025-12-18 00:09:01,299 - lava_callback - ERROR - Error processing callback for node 6940a007575bc060ce162c2e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a007575bc060ce162c2e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a007575bc060ce162c2e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a007575bc060ce162c2e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:01,301 - lava_callback - INFO - Starting async job submit for node 6940a037575bc060ce162cb6 2025-12-18 00:09:01,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:01,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a037575bc060ce162cb6 HTTP/1.1" 502 150 2025-12-18 00:09:01,343 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a037575bc060ce162cb6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:01,343 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a037575bc060ce162cb6 2025-12-18 00:09:01,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a037575bc060ce162cb6 HTTP/1.1" 502 150 2025-12-18 00:09:01,344 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a037575bc060ce162cb6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:01,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a025575bc060ce162c86 2025-12-18 00:09:01,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a025575bc060ce162c86 HTTP/1.1" 502 150 2025-12-18 00:09:01,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a025575bc060ce162c86'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:01,972 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a011575bc060ce162c4f 2025-12-18 00:09:01,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a011575bc060ce162c4f HTTP/1.1" 502 150 2025-12-18 00:09:01,973 - lava_callback - ERROR - Error processing callback for node 6940a011575bc060ce162c4f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a011575bc060ce162c4f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a011575bc060ce162c4f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a011575bc060ce162c4f (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:01,975 - lava_callback - INFO - Starting async job submit for node 6940a02b575bc060ce162c8b 2025-12-18 00:09:01,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:01,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02b575bc060ce162c8b HTTP/1.1" 502 150 2025-12-18 00:09:01,982 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a02b575bc060ce162c8b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:01,982 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a02b575bc060ce162c8b 2025-12-18 00:09:01,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02b575bc060ce162c8b HTTP/1.1" 502 150 2025-12-18 00:09:01,983 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a02b575bc060ce162c8b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:02,991 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a02d575bc060ce162c9c 2025-12-18 00:09:02,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02d575bc060ce162c9c HTTP/1.1" 502 150 2025-12-18 00:09:02,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a02d575bc060ce162c9c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:03,344 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a037575bc060ce162cb6 2025-12-18 00:09:03,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a037575bc060ce162cb6 HTTP/1.1" 502 150 2025-12-18 00:09:03,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a037575bc060ce162cb6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:03,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a032575bc060ce162ca5 2025-12-18 00:09:03,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a032575bc060ce162ca5 HTTP/1.1" 502 150 2025-12-18 00:09:03,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a032575bc060ce162ca5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47014 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:09:03,983 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a02b575bc060ce162c8b 2025-12-18 00:09:03,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02b575bc060ce162c8b HTTP/1.1" 502 150 2025-12-18 00:09:03,984 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a02b575bc060ce162c8b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:04,560 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6941359e459ff5fc64d2c36f 2025-12-18 00:09:04,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6941359e459ff5fc64d2c36f HTTP/1.1" 502 150 2025-12-18 00:09:04,561 - lava_callback - ERROR - Error processing callback for node 6941359e459ff5fc64d2c36f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941359e459ff5fc64d2c36f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941359e459ff5fc64d2c36f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6941359e459ff5fc64d2c36f (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:04,563 - lava_callback - INFO - Starting async job submit for node 6940a02e575bc060ce162c9f 2025-12-18 00:09:04,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:04,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02e575bc060ce162c9f HTTP/1.1" 502 150 2025-12-18 00:09:04,570 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a02e575bc060ce162c9f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:04,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a02e575bc060ce162c9f 2025-12-18 00:09:04,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02e575bc060ce162c9f HTTP/1.1" 502 150 2025-12-18 00:09:04,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a02e575bc060ce162c9f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:05,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a020575bc060ce162c7a 2025-12-18 00:09:05,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a020575bc060ce162c7a HTTP/1.1" 502 150 2025-12-18 00:09:05,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a020575bc060ce162c7a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:05,784 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a019575bc060ce162c63 2025-12-18 00:09:05,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a019575bc060ce162c63 HTTP/1.1" 502 150 2025-12-18 00:09:05,786 - lava_callback - ERROR - Error processing callback for node 6940a019575bc060ce162c63: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a019575bc060ce162c63 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a019575bc060ce162c63 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a019575bc060ce162c63 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:05,788 - lava_callback - INFO - Starting async job submit for node 6940a033575bc060ce162ca8 2025-12-18 00:09:05,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:05,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a033575bc060ce162ca8 HTTP/1.1" 502 150 2025-12-18 00:09:05,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a033575bc060ce162ca8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:05,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a033575bc060ce162ca8 2025-12-18 00:09:05,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a033575bc060ce162ca8 HTTP/1.1" 502 150 2025-12-18 00:09:05,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a033575bc060ce162ca8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47030 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:09:06,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a022575bc060ce162c80 2025-12-18 00:09:06,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a022575bc060ce162c80 HTTP/1.1" 502 150 2025-12-18 00:09:06,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a022575bc060ce162c80'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:06,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a02e575bc060ce162c9f 2025-12-18 00:09:06,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02e575bc060ce162c9f HTTP/1.1" 502 150 2025-12-18 00:09:06,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a02e575bc060ce162c9f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:07,041 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01d575bc060ce162c71 2025-12-18 00:09:07,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01d575bc060ce162c71 HTTP/1.1" 502 150 2025-12-18 00:09:07,042 - lava_callback - ERROR - Error processing callback for node 6940a01d575bc060ce162c71: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a01d575bc060ce162c71 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a01d575bc060ce162c71 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a01d575bc060ce162c71 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:07,044 - lava_callback - INFO - Starting async job submit for node 6940a030575bc060ce162ca2 2025-12-18 00:09:07,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:07,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a030575bc060ce162ca2 HTTP/1.1" 502 150 2025-12-18 00:09:07,054 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a030575bc060ce162ca2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:07,054 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a030575bc060ce162ca2 2025-12-18 00:09:07,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a030575bc060ce162ca2 HTTP/1.1" 502 150 2025-12-18 00:09:07,055 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a030575bc060ce162ca2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:07,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a037575bc060ce162cb6 2025-12-18 00:09:07,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a037575bc060ce162cb6 HTTP/1.1" 502 150 2025-12-18 00:09:07,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a037575bc060ce162cb6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:07,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a033575bc060ce162ca8 2025-12-18 00:09:07,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a033575bc060ce162ca8 HTTP/1.1" 502 150 2025-12-18 00:09:07,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a033575bc060ce162ca8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:07,984 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a02b575bc060ce162c8b 2025-12-18 00:09:07,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02b575bc060ce162c8b HTTP/1.1" 502 150 2025-12-18 00:09:07,985 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a02b575bc060ce162c8b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:08,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a021575bc060ce162c7d 2025-12-18 00:09:08,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a021575bc060ce162c7d HTTP/1.1" 502 150 2025-12-18 00:09:08,463 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a021575bc060ce162c7d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:08,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01f575bc060ce162c77 2025-12-18 00:09:08,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01f575bc060ce162c77 HTTP/1.1" 502 150 2025-12-18 00:09:08,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01f575bc060ce162c77'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:08,682 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a024575bc060ce162c83 2025-12-18 00:09:08,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a024575bc060ce162c83 HTTP/1.1" 502 150 2025-12-18 00:09:08,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a024575bc060ce162c83'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:09,055 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a030575bc060ce162ca2 2025-12-18 00:09:09,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a030575bc060ce162ca2 HTTP/1.1" 502 150 2025-12-18 00:09:09,056 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a030575bc060ce162ca2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:09,809 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a025575bc060ce162c86 2025-12-18 00:09:09,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a025575bc060ce162c86 HTTP/1.1" 502 150 2025-12-18 00:09:09,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a025575bc060ce162c86'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:10,037 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a017575bc060ce162c54 2025-12-18 00:09:10,291 - lava_callback - INFO - Received callback for node 6940a313575bc060ce163129 INFO: 172.21.0.1:43836 - "POST /node/6940a313575bc060ce163129 HTTP/1.0" 202 Accepted 2025-12-18 00:09:10,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a02e575bc060ce162c9f 2025-12-18 00:09:10,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02e575bc060ce162c9f HTTP/1.1" 502 150 2025-12-18 00:09:10,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a02e575bc060ce162c9f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:10,993 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a02d575bc060ce162c9c 2025-12-18 00:09:10,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02d575bc060ce162c9c HTTP/1.1" 502 150 2025-12-18 00:09:10,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a02d575bc060ce162c9c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:11,563 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a032575bc060ce162ca5 2025-12-18 00:09:11,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a032575bc060ce162ca5 HTTP/1.1" 502 150 2025-12-18 00:09:11,564 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a032575bc060ce162ca5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:11,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a033575bc060ce162ca8 2025-12-18 00:09:11,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a033575bc060ce162ca8 HTTP/1.1" 502 150 2025-12-18 00:09:11,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a033575bc060ce162ca8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:12,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01b575bc060ce162c6e HTTP/1.1" 502 150 2025-12-18 00:09:12,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a01b575bc060ce162c6e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:13,057 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a030575bc060ce162ca2 2025-12-18 00:09:13,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a030575bc060ce162ca2 HTTP/1.1" 502 150 2025-12-18 00:09:13,058 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a030575bc060ce162ca2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:13,882 - lava_callback - INFO - Received callback for node 6940a317575bc060ce163136 INFO: 172.21.0.1:43842 - "POST /node/6940a317575bc060ce163136 HTTP/1.0" 202 Accepted 2025-12-18 00:09:14,566 - lava_callback - INFO - Received callback for node 6940a319575bc060ce16313b INFO: 172.21.0.1:43852 - "POST /node/6940a319575bc060ce16313b HTTP/1.0" 202 Accepted INFO: 172.21.0.1:43854 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-18 00:09:15,290 - lava_callback - INFO - Received callback for node 6940a31a575bc060ce163146 INFO: 172.21.0.1:43868 - "POST /node/6940a31a575bc060ce163146 HTTP/1.0" 202 Accepted 2025-12-18 00:09:15,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a037575bc060ce162cb6 2025-12-18 00:09:15,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a037575bc060ce162cb6 HTTP/1.1" 502 150 2025-12-18 00:09:15,348 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a037575bc060ce162cb6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:15,985 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a02b575bc060ce162c8b 2025-12-18 00:09:15,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02b575bc060ce162c8b HTTP/1.1" 502 150 2025-12-18 00:09:15,987 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a02b575bc060ce162c8b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:15,990 - lava_callback - INFO - Received callback for node 6940a31c575bc060ce16314f INFO: 172.21.0.1:43878 - "POST /node/6940a31c575bc060ce16314f HTTP/1.0" 202 Accepted INFO: 172.21.0.1:43880 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:09:16,528 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01e575bc060ce162c74 2025-12-18 00:09:16,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01e575bc060ce162c74 HTTP/1.1" 502 150 2025-12-18 00:09:16,529 - lava_callback - ERROR - Error processing callback for node 6940a01e575bc060ce162c74: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a01e575bc060ce162c74 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a01e575bc060ce162c74 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a01e575bc060ce162c74 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:16,530 - lava_callback - INFO - Starting async job submit for node 6940a034575bc060ce162cab 2025-12-18 00:09:16,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:16,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a034575bc060ce162cab HTTP/1.1" 502 150 2025-12-18 00:09:16,541 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a034575bc060ce162cab'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:16,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a034575bc060ce162cab 2025-12-18 00:09:16,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a034575bc060ce162cab HTTP/1.1" 502 150 2025-12-18 00:09:16,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a034575bc060ce162cab'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:16,657 - lava_callback - INFO - Received callback for node 6940a323575bc060ce16315c INFO: 172.21.0.1:43886 - "POST /node/6940a323575bc060ce16315c HTTP/1.0" 202 Accepted 2025-12-18 00:09:18,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a034575bc060ce162cab 2025-12-18 00:09:18,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a034575bc060ce162cab HTTP/1.1" 502 150 2025-12-18 00:09:18,543 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a034575bc060ce162cab'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:18,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a02e575bc060ce162c9f 2025-12-18 00:09:18,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02e575bc060ce162c9f HTTP/1.1" 502 150 2025-12-18 00:09:18,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a02e575bc060ce162c9f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:19,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a033575bc060ce162ca8 2025-12-18 00:09:19,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a033575bc060ce162ca8 HTTP/1.1" 502 150 2025-12-18 00:09:19,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a033575bc060ce162ca8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:21,058 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a030575bc060ce162ca2 2025-12-18 00:09:21,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a030575bc060ce162ca2 HTTP/1.1" 502 150 2025-12-18 00:09:21,059 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a030575bc060ce162ca2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:21,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a020575bc060ce162c7a 2025-12-18 00:09:21,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a020575bc060ce162c7a HTTP/1.1" 502 150 2025-12-18 00:09:21,598 - lava_callback - ERROR - Error processing callback for node 6940a020575bc060ce162c7a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a020575bc060ce162c7a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a020575bc060ce162c7a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a020575bc060ce162c7a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:21,600 - lava_callback - INFO - Starting async job submit for node 6940a2c1575bc060ce1630f8 2025-12-18 00:09:21,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:21,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a2c1575bc060ce1630f8 HTTP/1.1" 502 150 2025-12-18 00:09:21,666 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a2c1575bc060ce1630f8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:21,666 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a2c1575bc060ce1630f8 2025-12-18 00:09:21,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a2c1575bc060ce1630f8 HTTP/1.1" 502 150 2025-12-18 00:09:21,667 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a2c1575bc060ce1630f8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:22,543 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a034575bc060ce162cab 2025-12-18 00:09:22,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a034575bc060ce162cab HTTP/1.1" 502 150 2025-12-18 00:09:22,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a034575bc060ce162cab'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:22,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a022575bc060ce162c80 2025-12-18 00:09:22,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a022575bc060ce162c80 HTTP/1.1" 502 150 2025-12-18 00:09:22,565 - lava_callback - ERROR - Error processing callback for node 6940a022575bc060ce162c80: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a022575bc060ce162c80 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a022575bc060ce162c80 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a022575bc060ce162c80 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:22,567 - lava_callback - INFO - Starting async job submit for node 6940a046575bc060ce162cd4 2025-12-18 00:09:22,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:22,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a046575bc060ce162cd4 HTTP/1.1" 502 150 2025-12-18 00:09:22,576 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a046575bc060ce162cd4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:22,576 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a046575bc060ce162cd4 2025-12-18 00:09:22,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a046575bc060ce162cd4 HTTP/1.1" 502 150 2025-12-18 00:09:22,577 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a046575bc060ce162cd4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:23,667 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a2c1575bc060ce1630f8 2025-12-18 00:09:24,463 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a021575bc060ce162c7d 2025-12-18 00:09:24,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a021575bc060ce162c7d HTTP/1.1" 502 150 2025-12-18 00:09:24,464 - lava_callback - ERROR - Error processing callback for node 6940a021575bc060ce162c7d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a021575bc060ce162c7d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a021575bc060ce162c7d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a021575bc060ce162c7d (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:24,466 - lava_callback - INFO - Starting async job submit for node 6940a036575bc060ce162cb3 2025-12-18 00:09:24,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:24,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a036575bc060ce162cb3 HTTP/1.1" 502 150 2025-12-18 00:09:24,495 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a036575bc060ce162cb3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:24,495 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a036575bc060ce162cb3 2025-12-18 00:09:24,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a036575bc060ce162cb3 HTTP/1.1" 502 150 2025-12-18 00:09:24,496 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a036575bc060ce162cb3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:24,577 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a046575bc060ce162cd4 2025-12-18 00:09:24,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a046575bc060ce162cd4 HTTP/1.1" 502 150 2025-12-18 00:09:24,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a046575bc060ce162cd4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:24,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01f575bc060ce162c77 2025-12-18 00:09:24,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01f575bc060ce162c77 HTTP/1.1" 502 150 2025-12-18 00:09:24,595 - lava_callback - ERROR - Error processing callback for node 6940a01f575bc060ce162c77: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a01f575bc060ce162c77 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a01f575bc060ce162c77 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a01f575bc060ce162c77 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:24,597 - lava_callback - INFO - Starting async job submit for node 6940a041575bc060ce162cc7 2025-12-18 00:09:24,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:24,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a041575bc060ce162cc7 HTTP/1.1" 502 150 2025-12-18 00:09:24,603 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a041575bc060ce162cc7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:24,603 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a041575bc060ce162cc7 2025-12-18 00:09:24,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a041575bc060ce162cc7 HTTP/1.1" 502 150 2025-12-18 00:09:24,604 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a041575bc060ce162cc7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:24,683 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a024575bc060ce162c83 2025-12-18 00:09:24,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a024575bc060ce162c83 HTTP/1.1" 502 150 2025-12-18 00:09:24,685 - lava_callback - ERROR - Error processing callback for node 6940a024575bc060ce162c83: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a024575bc060ce162c83 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a024575bc060ce162c83 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a024575bc060ce162c83 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:24,687 - lava_callback - INFO - Starting async job submit for node 6940a043575bc060ce162cca 2025-12-18 00:09:24,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:24,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a043575bc060ce162cca HTTP/1.1" 502 150 2025-12-18 00:09:24,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a043575bc060ce162cca'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:24,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a043575bc060ce162cca 2025-12-18 00:09:24,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a043575bc060ce162cca HTTP/1.1" 502 150 2025-12-18 00:09:24,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a043575bc060ce162cca'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:25,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a017575bc060ce162c54 HTTP/1.1" 502 150 2025-12-18 00:09:25,424 - lava_callback - ERROR - Error processing callback for node 6940a017575bc060ce162c54: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a017575bc060ce162c54 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a017575bc060ce162c54 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a017575bc060ce162c54 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:25,426 - lava_callback - INFO - Starting async job submit for node 6940a03f575bc060ce162cbb 2025-12-18 00:09:25,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:25,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a03f575bc060ce162cbb HTTP/1.1" 502 150 2025-12-18 00:09:25,433 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a03f575bc060ce162cbb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:25,433 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a03f575bc060ce162cbb 2025-12-18 00:09:25,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a03f575bc060ce162cbb HTTP/1.1" 502 150 2025-12-18 00:09:25,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a03f575bc060ce162cbb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:25,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a025575bc060ce162c86 2025-12-18 00:09:25,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a025575bc060ce162c86 HTTP/1.1" 502 150 2025-12-18 00:09:25,811 - lava_callback - ERROR - Error processing callback for node 6940a025575bc060ce162c86: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a025575bc060ce162c86 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a025575bc060ce162c86 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a025575bc060ce162c86 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:25,813 - lava_callback - INFO - Starting async job submit for node 6940a045575bc060ce162cd1 2025-12-18 00:09:25,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:25,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a045575bc060ce162cd1 HTTP/1.1" 502 150 2025-12-18 00:09:25,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a045575bc060ce162cd1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:25,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a045575bc060ce162cd1 2025-12-18 00:09:25,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a045575bc060ce162cd1 HTTP/1.1" 502 150 2025-12-18 00:09:25,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a045575bc060ce162cd1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:26,496 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a036575bc060ce162cb3 2025-12-18 00:09:26,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a036575bc060ce162cb3 HTTP/1.1" 502 150 2025-12-18 00:09:26,497 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a036575bc060ce162cb3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33938 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:09:26,604 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a041575bc060ce162cc7 2025-12-18 00:09:26,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a041575bc060ce162cc7 HTTP/1.1" 502 150 2025-12-18 00:09:26,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a041575bc060ce162cc7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:26,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a043575bc060ce162cca 2025-12-18 00:09:26,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a043575bc060ce162cca HTTP/1.1" 502 150 2025-12-18 00:09:26,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a043575bc060ce162cca'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:26,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a02d575bc060ce162c9c 2025-12-18 00:09:26,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02d575bc060ce162c9c HTTP/1.1" 502 150 2025-12-18 00:09:26,996 - lava_callback - ERROR - Error processing callback for node 6940a02d575bc060ce162c9c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a02d575bc060ce162c9c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a02d575bc060ce162c9c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a02d575bc060ce162c9c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:26,999 - lava_callback - INFO - Starting async job submit for node 6940a054575bc060ce162cda 2025-12-18 00:09:26,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:27,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a054575bc060ce162cda HTTP/1.1" 502 150 2025-12-18 00:09:27,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a054575bc060ce162cda'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:27,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a054575bc060ce162cda 2025-12-18 00:09:27,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a054575bc060ce162cda HTTP/1.1" 502 150 2025-12-18 00:09:27,012 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a054575bc060ce162cda'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:27,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a03f575bc060ce162cbb 2025-12-18 00:09:27,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a03f575bc060ce162cbb HTTP/1.1" 502 150 2025-12-18 00:09:27,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a03f575bc060ce162cbb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:27,564 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a032575bc060ce162ca5 2025-12-18 00:09:27,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a032575bc060ce162ca5 HTTP/1.1" 502 150 2025-12-18 00:09:27,566 - lava_callback - ERROR - Error processing callback for node 6940a032575bc060ce162ca5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a032575bc060ce162ca5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a032575bc060ce162ca5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a032575bc060ce162ca5 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:27,569 - lava_callback - INFO - Starting async job submit for node 6940a056575bc060ce162ce4 2025-12-18 00:09:27,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:27,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a056575bc060ce162ce4 HTTP/1.1" 502 150 2025-12-18 00:09:27,578 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a056575bc060ce162ce4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:27,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a056575bc060ce162ce4 2025-12-18 00:09:27,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a056575bc060ce162ce4 HTTP/1.1" 502 150 2025-12-18 00:09:27,579 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a056575bc060ce162ce4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:27,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a045575bc060ce162cd1 2025-12-18 00:09:27,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a045575bc060ce162cd1 HTTP/1.1" 502 150 2025-12-18 00:09:27,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a045575bc060ce162cd1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:28,578 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a046575bc060ce162cd4 2025-12-18 00:09:28,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a046575bc060ce162cd4 HTTP/1.1" 502 150 2025-12-18 00:09:28,580 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a046575bc060ce162cd4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:28,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a01b575bc060ce162c6e 2025-12-18 00:09:28,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a01b575bc060ce162c6e HTTP/1.1" 502 150 2025-12-18 00:09:28,626 - lava_callback - ERROR - Error processing callback for node 6940a01b575bc060ce162c6e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a01b575bc060ce162c6e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a01b575bc060ce162c6e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a01b575bc060ce162c6e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:28,628 - lava_callback - INFO - Starting async job submit for node 6940a058575bc060ce162cee 2025-12-18 00:09:28,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:28,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a058575bc060ce162cee HTTP/1.1" 502 150 2025-12-18 00:09:28,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a058575bc060ce162cee'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:28,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a058575bc060ce162cee 2025-12-18 00:09:28,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a058575bc060ce162cee HTTP/1.1" 502 150 2025-12-18 00:09:28,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a058575bc060ce162cee'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:29,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a054575bc060ce162cda 2025-12-18 00:09:29,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a054575bc060ce162cda HTTP/1.1" 502 150 2025-12-18 00:09:29,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a054575bc060ce162cda'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:29,579 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a056575bc060ce162ce4 2025-12-18 00:09:29,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a056575bc060ce162ce4 HTTP/1.1" 502 150 2025-12-18 00:09:29,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a056575bc060ce162ce4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:30,351 - lava_callback - INFO - Received callback for node 6940a339575bc060ce163174 INFO: 172.21.0.1:35198 - "POST /node/6940a339575bc060ce163174 HTTP/1.0" 202 Accepted 2025-12-18 00:09:30,498 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a036575bc060ce162cb3 2025-12-18 00:09:30,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a036575bc060ce162cb3 HTTP/1.1" 502 150 2025-12-18 00:09:30,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a036575bc060ce162cb3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:30,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a034575bc060ce162cab 2025-12-18 00:09:30,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a034575bc060ce162cab HTTP/1.1" 502 150 2025-12-18 00:09:30,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a034575bc060ce162cab'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:30,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a041575bc060ce162cc7 2025-12-18 00:09:30,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a041575bc060ce162cc7 HTTP/1.1" 502 150 2025-12-18 00:09:30,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a041575bc060ce162cc7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:30,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a058575bc060ce162cee 2025-12-18 00:09:30,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a058575bc060ce162cee HTTP/1.1" 502 150 2025-12-18 00:09:30,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a058575bc060ce162cee'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:30,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a043575bc060ce162cca 2025-12-18 00:09:30,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a043575bc060ce162cca HTTP/1.1" 502 150 2025-12-18 00:09:30,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a043575bc060ce162cca'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:31,348 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a037575bc060ce162cb6 2025-12-18 00:09:31,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a037575bc060ce162cb6 HTTP/1.1" 502 150 2025-12-18 00:09:31,349 - lava_callback - ERROR - Error processing callback for node 6940a037575bc060ce162cb6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a037575bc060ce162cb6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a037575bc060ce162cb6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a037575bc060ce162cb6 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:31,351 - lava_callback - INFO - Starting async job submit for node 6940a05a575bc060ce162cf5 2025-12-18 00:09:31,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:31,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05a575bc060ce162cf5 HTTP/1.1" 502 150 2025-12-18 00:09:31,358 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05a575bc060ce162cf5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:31,358 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05a575bc060ce162cf5 2025-12-18 00:09:31,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05a575bc060ce162cf5 HTTP/1.1" 502 150 2025-12-18 00:09:31,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05a575bc060ce162cf5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:31,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a03f575bc060ce162cbb 2025-12-18 00:09:31,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a03f575bc060ce162cbb HTTP/1.1" 502 150 2025-12-18 00:09:31,443 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a03f575bc060ce162cbb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:31,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a045575bc060ce162cd1 2025-12-18 00:09:31,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a045575bc060ce162cd1 HTTP/1.1" 502 150 2025-12-18 00:09:31,824 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a045575bc060ce162cd1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:31,987 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a02b575bc060ce162c8b 2025-12-18 00:09:31,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02b575bc060ce162c8b HTTP/1.1" 502 150 2025-12-18 00:09:31,988 - lava_callback - ERROR - Error processing callback for node 6940a02b575bc060ce162c8b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a02b575bc060ce162c8b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a02b575bc060ce162c8b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a02b575bc060ce162c8b (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:31,990 - lava_callback - INFO - Starting async job submit for node 6940a05c575bc060ce162cf8 2025-12-18 00:09:31,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:31,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05c575bc060ce162cf8 HTTP/1.1" 502 150 2025-12-18 00:09:31,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05c575bc060ce162cf8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:31,999 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05c575bc060ce162cf8 2025-12-18 00:09:32,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05c575bc060ce162cf8 HTTP/1.1" 502 150 2025-12-18 00:09:32,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05c575bc060ce162cf8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:33,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a054575bc060ce162cda 2025-12-18 00:09:33,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a054575bc060ce162cda HTTP/1.1" 502 150 2025-12-18 00:09:33,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a054575bc060ce162cda'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:33,359 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05a575bc060ce162cf5 2025-12-18 00:09:33,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05a575bc060ce162cf5 HTTP/1.1" 502 150 2025-12-18 00:09:33,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05a575bc060ce162cf5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:33,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a056575bc060ce162ce4 2025-12-18 00:09:33,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a056575bc060ce162ce4 HTTP/1.1" 502 150 2025-12-18 00:09:33,582 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a056575bc060ce162ce4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:34,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05c575bc060ce162cf8 2025-12-18 00:09:34,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05c575bc060ce162cf8 HTTP/1.1" 502 150 2025-12-18 00:09:34,001 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05c575bc060ce162cf8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:34,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a02e575bc060ce162c9f 2025-12-18 00:09:34,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a02e575bc060ce162c9f HTTP/1.1" 502 150 2025-12-18 00:09:34,589 - lava_callback - ERROR - Error processing callback for node 6940a02e575bc060ce162c9f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a02e575bc060ce162c9f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a02e575bc060ce162c9f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a02e575bc060ce162c9f (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:34,591 - lava_callback - INFO - Starting async job submit for node 6940a05d575bc060ce162cfb 2025-12-18 00:09:34,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:34,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05d575bc060ce162cfb HTTP/1.1" 502 150 2025-12-18 00:09:34,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05d575bc060ce162cfb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:34,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05d575bc060ce162cfb 2025-12-18 00:09:34,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05d575bc060ce162cfb HTTP/1.1" 502 150 2025-12-18 00:09:34,599 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05d575bc060ce162cfb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:34,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a058575bc060ce162cee 2025-12-18 00:09:34,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a058575bc060ce162cee HTTP/1.1" 502 150 2025-12-18 00:09:34,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a058575bc060ce162cee'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:35,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a033575bc060ce162ca8 2025-12-18 00:09:35,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a033575bc060ce162ca8 HTTP/1.1" 502 150 2025-12-18 00:09:35,801 - lava_callback - ERROR - Error processing callback for node 6940a033575bc060ce162ca8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a033575bc060ce162ca8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a033575bc060ce162ca8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a033575bc060ce162ca8 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:35,803 - lava_callback - INFO - Starting async job submit for node 6940a05e575bc060ce162cfe 2025-12-18 00:09:35,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:35,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05e575bc060ce162cfe HTTP/1.1" 502 150 2025-12-18 00:09:35,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05e575bc060ce162cfe'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:35,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05e575bc060ce162cfe 2025-12-18 00:09:35,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05e575bc060ce162cfe HTTP/1.1" 502 150 2025-12-18 00:09:35,810 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05e575bc060ce162cfe'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:35210 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:09:36,580 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a046575bc060ce162cd4 2025-12-18 00:09:36,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a046575bc060ce162cd4 HTTP/1.1" 502 150 2025-12-18 00:09:36,581 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a046575bc060ce162cd4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:36,599 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05d575bc060ce162cfb 2025-12-18 00:09:36,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05d575bc060ce162cfb HTTP/1.1" 502 150 2025-12-18 00:09:36,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05d575bc060ce162cfb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:37,060 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a030575bc060ce162ca2 2025-12-18 00:09:37,361 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05a575bc060ce162cf5 2025-12-18 00:09:37,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05a575bc060ce162cf5 HTTP/1.1" 502 150 2025-12-18 00:09:37,362 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05a575bc060ce162cf5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:37,439 - lava_callback - INFO - Received callback for node 69409e8a575bc060ce1629b9 INFO: 172.21.0.1:35216 - "POST /node/69409e8a575bc060ce1629b9 HTTP/1.0" 202 Accepted 2025-12-18 00:09:37,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05e575bc060ce162cfe 2025-12-18 00:09:37,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05e575bc060ce162cfe HTTP/1.1" 502 150 2025-12-18 00:09:37,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05e575bc060ce162cfe'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:38,002 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05c575bc060ce162cf8 2025-12-18 00:09:38,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05c575bc060ce162cf8 HTTP/1.1" 502 150 2025-12-18 00:09:38,003 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05c575bc060ce162cf8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:38,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a036575bc060ce162cb3 2025-12-18 00:09:38,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a036575bc060ce162cb3 HTTP/1.1" 502 150 2025-12-18 00:09:38,501 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a036575bc060ce162cb3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:38,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a041575bc060ce162cc7 2025-12-18 00:09:38,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a041575bc060ce162cc7 HTTP/1.1" 502 150 2025-12-18 00:09:38,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a041575bc060ce162cc7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:38,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a043575bc060ce162cca 2025-12-18 00:09:38,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a043575bc060ce162cca HTTP/1.1" 502 150 2025-12-18 00:09:38,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a043575bc060ce162cca'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:38,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a2c1575bc060ce1630f8 HTTP/1.1" 502 150 2025-12-18 00:09:38,992 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a2c1575bc060ce1630f8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:39,444 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a03f575bc060ce162cbb 2025-12-18 00:09:39,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a03f575bc060ce162cbb HTTP/1.1" 502 150 2025-12-18 00:09:39,445 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a03f575bc060ce162cbb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:39,824 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a045575bc060ce162cd1 2025-12-18 00:09:39,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a045575bc060ce162cd1 HTTP/1.1" 502 150 2025-12-18 00:09:39,825 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a045575bc060ce162cd1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:40,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05d575bc060ce162cfb 2025-12-18 00:09:40,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05d575bc060ce162cfb HTTP/1.1" 502 150 2025-12-18 00:09:40,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05d575bc060ce162cfb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:41,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a054575bc060ce162cda 2025-12-18 00:09:41,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a054575bc060ce162cda HTTP/1.1" 502 150 2025-12-18 00:09:41,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a054575bc060ce162cda'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:41,582 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a056575bc060ce162ce4 2025-12-18 00:09:41,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a056575bc060ce162ce4 HTTP/1.1" 502 150 2025-12-18 00:09:41,584 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a056575bc060ce162ce4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:41,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05e575bc060ce162cfe 2025-12-18 00:09:41,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05e575bc060ce162cfe HTTP/1.1" 502 150 2025-12-18 00:09:41,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05e575bc060ce162cfe'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:42,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a058575bc060ce162cee 2025-12-18 00:09:42,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a058575bc060ce162cee HTTP/1.1" 502 150 2025-12-18 00:09:42,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a058575bc060ce162cee'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:42,992 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a2c1575bc060ce1630f8 2025-12-18 00:09:42,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a2c1575bc060ce1630f8 HTTP/1.1" 502 150 2025-12-18 00:09:42,993 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a2c1575bc060ce1630f8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:45,362 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05a575bc060ce162cf5 2025-12-18 00:09:45,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05a575bc060ce162cf5 HTTP/1.1" 502 150 2025-12-18 00:09:45,363 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05a575bc060ce162cf5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:46,003 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05c575bc060ce162cf8 2025-12-18 00:09:46,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05c575bc060ce162cf8 HTTP/1.1" 502 150 2025-12-18 00:09:46,004 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05c575bc060ce162cf8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:56116 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:09:46,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a034575bc060ce162cab 2025-12-18 00:09:46,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a034575bc060ce162cab HTTP/1.1" 502 150 2025-12-18 00:09:46,551 - lava_callback - ERROR - Error processing callback for node 6940a034575bc060ce162cab: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a034575bc060ce162cab (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a034575bc060ce162cab (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a034575bc060ce162cab (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:46,553 - lava_callback - INFO - Starting async job submit for node 6940a05f575bc060ce162d01 2025-12-18 00:09:46,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:46,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05f575bc060ce162d01 HTTP/1.1" 502 150 2025-12-18 00:09:46,561 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05f575bc060ce162d01'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:46,561 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05f575bc060ce162d01 2025-12-18 00:09:46,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05f575bc060ce162d01 HTTP/1.1" 502 150 2025-12-18 00:09:46,562 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05f575bc060ce162d01'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:48,562 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05f575bc060ce162d01 2025-12-18 00:09:48,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05f575bc060ce162d01 HTTP/1.1" 502 150 2025-12-18 00:09:48,571 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05f575bc060ce162d01'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:48,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05d575bc060ce162cfb 2025-12-18 00:09:48,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05d575bc060ce162cfb HTTP/1.1" 502 150 2025-12-18 00:09:48,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05d575bc060ce162cfb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:49,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05e575bc060ce162cfe 2025-12-18 00:09:50,105 - lava_callback - INFO - Received callback for node 6940c54b1e3c24e95681d6c4 INFO: 172.21.0.1:52964 - "POST /node/6940c54b1e3c24e95681d6c4 HTTP/1.0" 202 Accepted 2025-12-18 00:09:50,270 - lava_callback - INFO - Received callback for node 6940a33a575bc060ce16317b INFO: 172.21.0.1:52970 - "POST /node/6940a33a575bc060ce16317b HTTP/1.0" 202 Accepted 2025-12-18 00:09:50,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a2c1575bc060ce1630f8 2025-12-18 00:09:50,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a2c1575bc060ce1630f8 HTTP/1.1" 502 150 2025-12-18 00:09:50,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a2c1575bc060ce1630f8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:51,020 - lava_callback - INFO - Received callback for node 6940a33c575bc060ce163186 INFO: 172.21.0.1:52972 - "POST /node/6940a33c575bc060ce163186 HTTP/1.0" 202 Accepted 2025-12-18 00:09:51,805 - lava_callback - INFO - Received callback for node 6940a33d575bc060ce163189 INFO: 172.21.0.1:52980 - "POST /node/6940a33d575bc060ce163189 HTTP/1.0" 202 Accepted 2025-12-18 00:09:52,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a030575bc060ce162ca2 HTTP/1.1" 502 150 2025-12-18 00:09:52,304 - lava_callback - ERROR - Error processing callback for node 6940a030575bc060ce162ca2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a030575bc060ce162ca2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a030575bc060ce162ca2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a030575bc060ce162ca2 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:52,306 - lava_callback - INFO - Starting async job submit for node 6940a060575bc060ce162d04 2025-12-18 00:09:52,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:52,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a060575bc060ce162d04 HTTP/1.1" 502 150 2025-12-18 00:09:52,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a060575bc060ce162d04'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:52,313 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a060575bc060ce162d04 2025-12-18 00:09:52,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a060575bc060ce162d04 HTTP/1.1" 502 150 2025-12-18 00:09:52,314 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a060575bc060ce162d04'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:52,571 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05f575bc060ce162d01 2025-12-18 00:09:52,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05f575bc060ce162d01 HTTP/1.1" 502 150 2025-12-18 00:09:52,572 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05f575bc060ce162d01'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:52,581 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a046575bc060ce162cd4 2025-12-18 00:09:52,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a046575bc060ce162cd4 HTTP/1.1" 502 150 2025-12-18 00:09:52,582 - lava_callback - ERROR - Error processing callback for node 6940a046575bc060ce162cd4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a046575bc060ce162cd4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a046575bc060ce162cd4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a046575bc060ce162cd4 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:52,584 - lava_callback - INFO - Starting async job submit for node 6940a062575bc060ce162d07 2025-12-18 00:09:52,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:52,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a062575bc060ce162d07 HTTP/1.1" 502 150 2025-12-18 00:09:52,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a062575bc060ce162d07'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:52,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a062575bc060ce162d07 2025-12-18 00:09:52,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a062575bc060ce162d07 HTTP/1.1" 502 150 2025-12-18 00:09:52,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a062575bc060ce162d07'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:54,314 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a060575bc060ce162d04 2025-12-18 00:09:54,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a060575bc060ce162d04 HTTP/1.1" 502 150 2025-12-18 00:09:54,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a060575bc060ce162d04'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:54,501 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a036575bc060ce162cb3 2025-12-18 00:09:54,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a036575bc060ce162cb3 HTTP/1.1" 502 150 2025-12-18 00:09:54,502 - lava_callback - ERROR - Error processing callback for node 6940a036575bc060ce162cb3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a036575bc060ce162cb3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a036575bc060ce162cb3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a036575bc060ce162cb3 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:54,504 - lava_callback - INFO - Starting async job submit for node 6940a068575bc060ce162d0e 2025-12-18 00:09:54,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:54,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a068575bc060ce162d0e HTTP/1.1" 502 150 2025-12-18 00:09:54,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a068575bc060ce162d0e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:54,511 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a068575bc060ce162d0e 2025-12-18 00:09:54,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a068575bc060ce162d0e HTTP/1.1" 502 150 2025-12-18 00:09:54,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a068575bc060ce162d0e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:54,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a062575bc060ce162d07 2025-12-18 00:09:54,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a062575bc060ce162d07 HTTP/1.1" 502 150 2025-12-18 00:09:54,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a062575bc060ce162d07'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:54,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a041575bc060ce162cc7 2025-12-18 00:09:54,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a041575bc060ce162cc7 HTTP/1.1" 502 150 2025-12-18 00:09:54,610 - lava_callback - ERROR - Error processing callback for node 6940a041575bc060ce162cc7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a041575bc060ce162cc7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a041575bc060ce162cc7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a041575bc060ce162cc7 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:54,611 - lava_callback - INFO - Starting async job submit for node 6940a06c575bc060ce162d25 2025-12-18 00:09:54,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:54,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06c575bc060ce162d25 HTTP/1.1" 502 150 2025-12-18 00:09:54,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06c575bc060ce162d25'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:54,618 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06c575bc060ce162d25 2025-12-18 00:09:54,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06c575bc060ce162d25 HTTP/1.1" 502 150 2025-12-18 00:09:54,618 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06c575bc060ce162d25'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:54,701 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a043575bc060ce162cca 2025-12-18 00:09:54,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a043575bc060ce162cca HTTP/1.1" 502 150 2025-12-18 00:09:54,702 - lava_callback - ERROR - Error processing callback for node 6940a043575bc060ce162cca: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a043575bc060ce162cca (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a043575bc060ce162cca (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a043575bc060ce162cca (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:54,704 - lava_callback - INFO - Starting async job submit for node 6940a06b575bc060ce162d20 2025-12-18 00:09:54,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:54,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06b575bc060ce162d20 HTTP/1.1" 502 150 2025-12-18 00:09:54,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06b575bc060ce162d20'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:54,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06b575bc060ce162d20 2025-12-18 00:09:54,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06b575bc060ce162d20 HTTP/1.1" 502 150 2025-12-18 00:09:54,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06b575bc060ce162d20'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:55,445 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a03f575bc060ce162cbb 2025-12-18 00:09:55,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a03f575bc060ce162cbb HTTP/1.1" 502 150 2025-12-18 00:09:55,446 - lava_callback - ERROR - Error processing callback for node 6940a03f575bc060ce162cbb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a03f575bc060ce162cbb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a03f575bc060ce162cbb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a03f575bc060ce162cbb (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:55,448 - lava_callback - INFO - Starting async job submit for node 6940a06d575bc060ce162d2c 2025-12-18 00:09:55,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:55,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06d575bc060ce162d2c HTTP/1.1" 502 150 2025-12-18 00:09:55,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06d575bc060ce162d2c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:55,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06d575bc060ce162d2c 2025-12-18 00:09:55,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06d575bc060ce162d2c HTTP/1.1" 502 150 2025-12-18 00:09:55,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06d575bc060ce162d2c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:55,825 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a045575bc060ce162cd1 2025-12-18 00:09:55,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a045575bc060ce162cd1 HTTP/1.1" 502 150 2025-12-18 00:09:55,827 - lava_callback - ERROR - Error processing callback for node 6940a045575bc060ce162cd1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a045575bc060ce162cd1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a045575bc060ce162cd1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a045575bc060ce162cd1 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:55,829 - lava_callback - INFO - Starting async job submit for node 6940a06e575bc060ce162d31 2025-12-18 00:09:55,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:55,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06e575bc060ce162d31 HTTP/1.1" 502 150 2025-12-18 00:09:55,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06e575bc060ce162d31'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:55,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06e575bc060ce162d31 2025-12-18 00:09:55,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06e575bc060ce162d31 HTTP/1.1" 502 150 2025-12-18 00:09:55,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06e575bc060ce162d31'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:52986 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:09:56,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a068575bc060ce162d0e 2025-12-18 00:09:56,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a068575bc060ce162d0e HTTP/1.1" 502 150 2025-12-18 00:09:56,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a068575bc060ce162d0e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:56,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06c575bc060ce162d25 2025-12-18 00:09:56,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06c575bc060ce162d25 HTTP/1.1" 502 150 2025-12-18 00:09:56,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06c575bc060ce162d25'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:56,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06b575bc060ce162d20 2025-12-18 00:09:56,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06b575bc060ce162d20 HTTP/1.1" 502 150 2025-12-18 00:09:56,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06b575bc060ce162d20'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:57,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a054575bc060ce162cda 2025-12-18 00:09:57,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a054575bc060ce162cda HTTP/1.1" 502 150 2025-12-18 00:09:57,018 - lava_callback - ERROR - Error processing callback for node 6940a054575bc060ce162cda: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a054575bc060ce162cda (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a054575bc060ce162cda (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a054575bc060ce162cda (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:57,019 - lava_callback - INFO - Starting async job submit for node 6940a075575bc060ce162d37 2025-12-18 00:09:57,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:57,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a075575bc060ce162d37 HTTP/1.1" 502 150 2025-12-18 00:09:57,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a075575bc060ce162d37'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:57,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a075575bc060ce162d37 2025-12-18 00:09:57,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a075575bc060ce162d37 HTTP/1.1" 502 150 2025-12-18 00:09:57,028 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a075575bc060ce162d37'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:57,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06d575bc060ce162d2c 2025-12-18 00:09:57,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06d575bc060ce162d2c HTTP/1.1" 502 150 2025-12-18 00:09:57,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06d575bc060ce162d2c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:57,584 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a056575bc060ce162ce4 2025-12-18 00:09:57,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a056575bc060ce162ce4 HTTP/1.1" 502 150 2025-12-18 00:09:57,585 - lava_callback - ERROR - Error processing callback for node 6940a056575bc060ce162ce4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a056575bc060ce162ce4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a056575bc060ce162ce4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a056575bc060ce162ce4 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:57,587 - lava_callback - INFO - Starting async job submit for node 6940a076575bc060ce162d3a 2025-12-18 00:09:57,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:57,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a076575bc060ce162d3a HTTP/1.1" 502 150 2025-12-18 00:09:57,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a076575bc060ce162d3a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:57,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a076575bc060ce162d3a 2025-12-18 00:09:57,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a076575bc060ce162d3a HTTP/1.1" 502 150 2025-12-18 00:09:57,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a076575bc060ce162d3a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:57,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06e575bc060ce162d31 2025-12-18 00:09:57,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06e575bc060ce162d31 HTTP/1.1" 502 150 2025-12-18 00:09:57,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06e575bc060ce162d31'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:58,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a060575bc060ce162d04 2025-12-18 00:09:58,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a060575bc060ce162d04 HTTP/1.1" 502 150 2025-12-18 00:09:58,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a060575bc060ce162d04'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:58,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a062575bc060ce162d07 2025-12-18 00:09:58,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a062575bc060ce162d07 HTTP/1.1" 502 150 2025-12-18 00:09:58,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a062575bc060ce162d07'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:58,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a058575bc060ce162cee 2025-12-18 00:09:58,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a058575bc060ce162cee HTTP/1.1" 502 150 2025-12-18 00:09:58,644 - lava_callback - ERROR - Error processing callback for node 6940a058575bc060ce162cee: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a058575bc060ce162cee (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a058575bc060ce162cee (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a058575bc060ce162cee (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:09:58,646 - lava_callback - INFO - Starting async job submit for node 6940a077575bc060ce162d3d 2025-12-18 00:09:58,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:09:58,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a077575bc060ce162d3d HTTP/1.1" 502 150 2025-12-18 00:09:58,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a077575bc060ce162d3d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:58,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a077575bc060ce162d3d 2025-12-18 00:09:58,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a077575bc060ce162d3d HTTP/1.1" 502 150 2025-12-18 00:09:58,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a077575bc060ce162d3d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:59,029 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a075575bc060ce162d37 2025-12-18 00:09:59,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a075575bc060ce162d37 HTTP/1.1" 502 150 2025-12-18 00:09:59,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a075575bc060ce162d37'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:09:59,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a076575bc060ce162d3a 2025-12-18 00:09:59,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a076575bc060ce162d3a HTTP/1.1" 502 150 2025-12-18 00:09:59,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a076575bc060ce162d3a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:00,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a068575bc060ce162d0e 2025-12-18 00:10:00,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a068575bc060ce162d0e HTTP/1.1" 502 150 2025-12-18 00:10:00,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a068575bc060ce162d0e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:00,572 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05f575bc060ce162d01 2025-12-18 00:10:00,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05f575bc060ce162d01 HTTP/1.1" 502 150 2025-12-18 00:10:00,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05f575bc060ce162d01'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:00,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06c575bc060ce162d25 2025-12-18 00:10:00,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06c575bc060ce162d25 HTTP/1.1" 502 150 2025-12-18 00:10:00,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06c575bc060ce162d25'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:00,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a077575bc060ce162d3d 2025-12-18 00:10:00,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a077575bc060ce162d3d HTTP/1.1" 502 150 2025-12-18 00:10:00,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a077575bc060ce162d3d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:00,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06b575bc060ce162d20 2025-12-18 00:10:00,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06b575bc060ce162d20 HTTP/1.1" 502 150 2025-12-18 00:10:00,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06b575bc060ce162d20'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:01,364 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05a575bc060ce162cf5 2025-12-18 00:10:01,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05a575bc060ce162cf5 HTTP/1.1" 502 150 2025-12-18 00:10:01,365 - lava_callback - ERROR - Error processing callback for node 6940a05a575bc060ce162cf5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a05a575bc060ce162cf5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a05a575bc060ce162cf5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a05a575bc060ce162cf5 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:01,368 - lava_callback - INFO - Starting async job submit for node 6940a078575bc060ce162d40 2025-12-18 00:10:01,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:01,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a078575bc060ce162d40 HTTP/1.1" 502 150 2025-12-18 00:10:01,377 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a078575bc060ce162d40'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:01,377 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a078575bc060ce162d40 2025-12-18 00:10:01,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a078575bc060ce162d40 HTTP/1.1" 502 150 2025-12-18 00:10:01,378 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a078575bc060ce162d40'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:01,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06d575bc060ce162d2c 2025-12-18 00:10:01,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06d575bc060ce162d2c HTTP/1.1" 502 150 2025-12-18 00:10:01,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06d575bc060ce162d2c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:01,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06e575bc060ce162d31 2025-12-18 00:10:01,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06e575bc060ce162d31 HTTP/1.1" 502 150 2025-12-18 00:10:01,865 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06e575bc060ce162d31'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:02,004 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05c575bc060ce162cf8 2025-12-18 00:10:03,030 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a075575bc060ce162d37 2025-12-18 00:10:03,379 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a078575bc060ce162d40 2025-12-18 00:10:03,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a078575bc060ce162d40 HTTP/1.1" 502 150 2025-12-18 00:10:03,380 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a078575bc060ce162d40'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:03,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a076575bc060ce162d3a 2025-12-18 00:10:03,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a076575bc060ce162d3a HTTP/1.1" 502 150 2025-12-18 00:10:03,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a076575bc060ce162d3a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55356 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:10:04,612 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05d575bc060ce162cfb 2025-12-18 00:10:04,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05d575bc060ce162cfb HTTP/1.1" 502 150 2025-12-18 00:10:04,613 - lava_callback - ERROR - Error processing callback for node 6940a05d575bc060ce162cfb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a05d575bc060ce162cfb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a05d575bc060ce162cfb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a05d575bc060ce162cfb (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:04,615 - lava_callback - INFO - Starting async job submit for node 6940a07a575bc060ce162d43 2025-12-18 00:10:04,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:04,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a07a575bc060ce162d43 HTTP/1.1" 502 150 2025-12-18 00:10:04,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a07a575bc060ce162d43'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:04,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a07a575bc060ce162d43 2025-12-18 00:10:04,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a07a575bc060ce162d43 HTTP/1.1" 502 150 2025-12-18 00:10:04,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a07a575bc060ce162d43'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:04,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a077575bc060ce162d3d 2025-12-18 00:10:04,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a077575bc060ce162d3d HTTP/1.1" 502 150 2025-12-18 00:10:04,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a077575bc060ce162d3d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:05,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05e575bc060ce162cfe HTTP/1.1" 502 150 2025-12-18 00:10:05,105 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a05e575bc060ce162cfe'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:06,317 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a060575bc060ce162d04 2025-12-18 00:10:06,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a060575bc060ce162d04 HTTP/1.1" 502 150 2025-12-18 00:10:06,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a060575bc060ce162d04'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55366 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:10:06,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a062575bc060ce162d07 2025-12-18 00:10:06,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a062575bc060ce162d07 HTTP/1.1" 502 150 2025-12-18 00:10:06,597 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a062575bc060ce162d07'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:06,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a07a575bc060ce162d43 2025-12-18 00:10:06,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a07a575bc060ce162d43 HTTP/1.1" 502 150 2025-12-18 00:10:06,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a07a575bc060ce162d43'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:06,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a2c1575bc060ce1630f8 2025-12-18 00:10:06,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a2c1575bc060ce1630f8 HTTP/1.1" 502 150 2025-12-18 00:10:06,996 - lava_callback - ERROR - Error processing callback for node 6940a2c1575bc060ce1630f8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a2c1575bc060ce1630f8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a2c1575bc060ce1630f8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a2c1575bc060ce1630f8 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:06,998 - lava_callback - INFO - Starting async job submit for node 6940a07b575bc060ce162d46 2025-12-18 00:10:06,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:07,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a07b575bc060ce162d46 HTTP/1.1" 502 150 2025-12-18 00:10:07,010 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a07b575bc060ce162d46'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:07,011 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a07b575bc060ce162d46 2025-12-18 00:10:07,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a07b575bc060ce162d46 HTTP/1.1" 502 150 2025-12-18 00:10:07,011 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a07b575bc060ce162d46'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:07,380 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a078575bc060ce162d40 2025-12-18 00:10:07,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a078575bc060ce162d40 HTTP/1.1" 502 150 2025-12-18 00:10:07,381 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a078575bc060ce162d40'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:08,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a068575bc060ce162d0e 2025-12-18 00:10:08,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a068575bc060ce162d0e HTTP/1.1" 502 150 2025-12-18 00:10:08,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a068575bc060ce162d0e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:08,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06c575bc060ce162d25 2025-12-18 00:10:08,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06c575bc060ce162d25 HTTP/1.1" 502 150 2025-12-18 00:10:08,626 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06c575bc060ce162d25'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:08,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06b575bc060ce162d20 2025-12-18 00:10:08,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06b575bc060ce162d20 HTTP/1.1" 502 150 2025-12-18 00:10:08,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06b575bc060ce162d20'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:09,012 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a07b575bc060ce162d46 2025-12-18 00:10:09,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a07b575bc060ce162d46 HTTP/1.1" 502 150 2025-12-18 00:10:09,013 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a07b575bc060ce162d46'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:09,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06d575bc060ce162d2c 2025-12-18 00:10:09,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06d575bc060ce162d2c HTTP/1.1" 502 150 2025-12-18 00:10:09,461 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06d575bc060ce162d2c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:09,865 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06e575bc060ce162d31 2025-12-18 00:10:09,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06e575bc060ce162d31 HTTP/1.1" 502 150 2025-12-18 00:10:09,866 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a06e575bc060ce162d31'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:10,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a07a575bc060ce162d43 2025-12-18 00:10:10,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a07a575bc060ce162d43 HTTP/1.1" 502 150 2025-12-18 00:10:10,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a07a575bc060ce162d43'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:11,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a076575bc060ce162d3a 2025-12-18 00:10:11,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a076575bc060ce162d3a HTTP/1.1" 502 150 2025-12-18 00:10:11,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a076575bc060ce162d3a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:12,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a077575bc060ce162d3d 2025-12-18 00:10:12,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a077575bc060ce162d3d HTTP/1.1" 502 150 2025-12-18 00:10:12,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a077575bc060ce162d3d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:13,013 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a07b575bc060ce162d46 2025-12-18 00:10:14,498 - lava_callback - INFO - Received callback for node 6940a354575bc060ce163192 INFO: 172.21.0.1:60358 - "POST /node/6940a354575bc060ce163192 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:60370 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-18 00:10:15,183 - lava_callback - INFO - Received callback for node 6940a355575bc060ce163197 INFO: 172.21.0.1:60376 - "POST /node/6940a355575bc060ce163197 HTTP/1.0" 202 Accepted 2025-12-18 00:10:15,382 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a078575bc060ce162d40 2025-12-18 00:10:15,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a078575bc060ce162d40 HTTP/1.1" 502 150 2025-12-18 00:10:15,383 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a078575bc060ce162d40'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60384 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:10:16,574 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05f575bc060ce162d01 2025-12-18 00:10:16,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05f575bc060ce162d01 HTTP/1.1" 502 150 2025-12-18 00:10:16,575 - lava_callback - ERROR - Error processing callback for node 6940a05f575bc060ce162d01: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a05f575bc060ce162d01 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a05f575bc060ce162d01 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a05f575bc060ce162d01 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:16,578 - lava_callback - INFO - Starting async job submit for node 6940a090575bc060ce162d4e 2025-12-18 00:10:16,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:16,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a090575bc060ce162d4e HTTP/1.1" 502 150 2025-12-18 00:10:16,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a090575bc060ce162d4e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:16,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a090575bc060ce162d4e 2025-12-18 00:10:16,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a090575bc060ce162d4e HTTP/1.1" 502 150 2025-12-18 00:10:16,589 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a090575bc060ce162d4e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:17,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05c575bc060ce162cf8 HTTP/1.1" 502 150 2025-12-18 00:10:17,392 - lava_callback - ERROR - Error processing callback for node 6940a05c575bc060ce162cf8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a05c575bc060ce162cf8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a05c575bc060ce162cf8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a05c575bc060ce162cf8 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:17,395 - lava_callback - INFO - Starting async job submit for node 6940a092575bc060ce162d59 2025-12-18 00:10:17,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:17,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a092575bc060ce162d59 HTTP/1.1" 502 150 2025-12-18 00:10:17,403 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a092575bc060ce162d59'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:17,403 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a092575bc060ce162d59 2025-12-18 00:10:17,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a092575bc060ce162d59 HTTP/1.1" 502 150 2025-12-18 00:10:17,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a092575bc060ce162d59'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:18,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a075575bc060ce162d37 HTTP/1.1" 502 150 2025-12-18 00:10:18,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a075575bc060ce162d37'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:18,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a090575bc060ce162d4e 2025-12-18 00:10:18,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a090575bc060ce162d4e HTTP/1.1" 502 150 2025-12-18 00:10:18,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a090575bc060ce162d4e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:18,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a07a575bc060ce162d43 2025-12-18 00:10:18,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a07a575bc060ce162d43 HTTP/1.1" 502 150 2025-12-18 00:10:18,627 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a07a575bc060ce162d43'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:19,404 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a092575bc060ce162d59 2025-12-18 00:10:19,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a092575bc060ce162d59 HTTP/1.1" 502 150 2025-12-18 00:10:19,405 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a092575bc060ce162d59'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:21,105 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a05e575bc060ce162cfe 2025-12-18 00:10:21,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a05e575bc060ce162cfe HTTP/1.1" 502 150 2025-12-18 00:10:21,106 - lava_callback - ERROR - Error processing callback for node 6940a05e575bc060ce162cfe: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a05e575bc060ce162cfe (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a05e575bc060ce162cfe (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a05e575bc060ce162cfe (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:21,108 - lava_callback - INFO - Starting async job submit for node 6940a095575bc060ce162d5c 2025-12-18 00:10:21,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:21,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a095575bc060ce162d5c HTTP/1.1" 502 150 2025-12-18 00:10:21,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a095575bc060ce162d5c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:21,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a095575bc060ce162d5c 2025-12-18 00:10:21,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a095575bc060ce162d5c HTTP/1.1" 502 150 2025-12-18 00:10:21,115 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a095575bc060ce162d5c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:22,319 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a060575bc060ce162d04 2025-12-18 00:10:22,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a060575bc060ce162d04 HTTP/1.1" 502 150 2025-12-18 00:10:22,320 - lava_callback - ERROR - Error processing callback for node 6940a060575bc060ce162d04: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a060575bc060ce162d04 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a060575bc060ce162d04 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a060575bc060ce162d04 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:22,322 - lava_callback - INFO - Starting async job submit for node 6940a095575bc060ce162d5f 2025-12-18 00:10:22,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:22,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a095575bc060ce162d5f HTTP/1.1" 502 150 2025-12-18 00:10:22,330 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a095575bc060ce162d5f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:22,330 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a095575bc060ce162d5f 2025-12-18 00:10:22,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a095575bc060ce162d5f HTTP/1.1" 502 150 2025-12-18 00:10:22,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a095575bc060ce162d5f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:22,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a090575bc060ce162d4e 2025-12-18 00:10:22,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a090575bc060ce162d4e HTTP/1.1" 502 150 2025-12-18 00:10:22,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a090575bc060ce162d4e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:22,597 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a062575bc060ce162d07 2025-12-18 00:10:22,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a062575bc060ce162d07 HTTP/1.1" 502 150 2025-12-18 00:10:22,598 - lava_callback - ERROR - Error processing callback for node 6940a062575bc060ce162d07: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a062575bc060ce162d07 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a062575bc060ce162d07 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a062575bc060ce162d07 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:22,600 - lava_callback - INFO - Starting async job submit for node 6940a097575bc060ce162d64 2025-12-18 00:10:22,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:22,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a097575bc060ce162d64 HTTP/1.1" 502 150 2025-12-18 00:10:22,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a097575bc060ce162d64'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:22,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a097575bc060ce162d64 2025-12-18 00:10:22,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a097575bc060ce162d64 HTTP/1.1" 502 150 2025-12-18 00:10:22,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a097575bc060ce162d64'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:23,115 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a095575bc060ce162d5c 2025-12-18 00:10:23,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a095575bc060ce162d5c HTTP/1.1" 502 150 2025-12-18 00:10:23,116 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a095575bc060ce162d5c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:23,405 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a092575bc060ce162d59 2025-12-18 00:10:23,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a092575bc060ce162d59 HTTP/1.1" 502 150 2025-12-18 00:10:23,406 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a092575bc060ce162d59'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:24,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a095575bc060ce162d5f 2025-12-18 00:10:24,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a095575bc060ce162d5f HTTP/1.1" 502 150 2025-12-18 00:10:24,332 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a095575bc060ce162d5f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:24,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a068575bc060ce162d0e 2025-12-18 00:10:24,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a068575bc060ce162d0e HTTP/1.1" 502 150 2025-12-18 00:10:24,518 - lava_callback - ERROR - Error processing callback for node 6940a068575bc060ce162d0e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a068575bc060ce162d0e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a068575bc060ce162d0e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a068575bc060ce162d0e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:24,519 - lava_callback - INFO - Starting async job submit for node 6940a099575bc060ce162d67 2025-12-18 00:10:24,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:24,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a099575bc060ce162d67 HTTP/1.1" 502 150 2025-12-18 00:10:24,529 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a099575bc060ce162d67'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:24,529 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a099575bc060ce162d67 2025-12-18 00:10:24,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a099575bc060ce162d67 HTTP/1.1" 502 150 2025-12-18 00:10:24,530 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a099575bc060ce162d67'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:24,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a097575bc060ce162d64 2025-12-18 00:10:24,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a097575bc060ce162d64 HTTP/1.1" 502 150 2025-12-18 00:10:24,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a097575bc060ce162d64'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:24,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06c575bc060ce162d25 2025-12-18 00:10:24,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06c575bc060ce162d25 HTTP/1.1" 502 150 2025-12-18 00:10:24,628 - lava_callback - ERROR - Error processing callback for node 6940a06c575bc060ce162d25: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a06c575bc060ce162d25 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a06c575bc060ce162d25 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a06c575bc060ce162d25 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:24,629 - lava_callback - INFO - Starting async job submit for node 6940a0a4575bc060ce162d6c 2025-12-18 00:10:24,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:24,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a4575bc060ce162d6c HTTP/1.1" 502 150 2025-12-18 00:10:24,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0a4575bc060ce162d6c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:24,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0a4575bc060ce162d6c 2025-12-18 00:10:24,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a4575bc060ce162d6c HTTP/1.1" 502 150 2025-12-18 00:10:24,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0a4575bc060ce162d6c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:24,714 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06b575bc060ce162d20 2025-12-18 00:10:24,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06b575bc060ce162d20 HTTP/1.1" 502 150 2025-12-18 00:10:24,715 - lava_callback - ERROR - Error processing callback for node 6940a06b575bc060ce162d20: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a06b575bc060ce162d20 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a06b575bc060ce162d20 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a06b575bc060ce162d20 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:24,717 - lava_callback - INFO - Starting async job submit for node 6940a0a6575bc060ce162d79 2025-12-18 00:10:24,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:24,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a6575bc060ce162d79 HTTP/1.1" 502 150 2025-12-18 00:10:24,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0a6575bc060ce162d79'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:24,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0a6575bc060ce162d79 2025-12-18 00:10:24,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a6575bc060ce162d79 HTTP/1.1" 502 150 2025-12-18 00:10:24,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0a6575bc060ce162d79'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:25,461 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06d575bc060ce162d2c 2025-12-18 00:10:25,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06d575bc060ce162d2c HTTP/1.1" 502 150 2025-12-18 00:10:25,462 - lava_callback - ERROR - Error processing callback for node 6940a06d575bc060ce162d2c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a06d575bc060ce162d2c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a06d575bc060ce162d2c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a06d575bc060ce162d2c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:25,464 - lava_callback - INFO - Starting async job submit for node 6940a0a9575bc060ce162d7c 2025-12-18 00:10:25,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:25,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a9575bc060ce162d7c HTTP/1.1" 502 150 2025-12-18 00:10:25,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0a9575bc060ce162d7c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:25,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0a9575bc060ce162d7c 2025-12-18 00:10:25,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a9575bc060ce162d7c HTTP/1.1" 502 150 2025-12-18 00:10:25,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0a9575bc060ce162d7c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:25,866 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a06e575bc060ce162d31 2025-12-18 00:10:25,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a06e575bc060ce162d31 HTTP/1.1" 502 150 2025-12-18 00:10:25,868 - lava_callback - ERROR - Error processing callback for node 6940a06e575bc060ce162d31: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a06e575bc060ce162d31 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a06e575bc060ce162d31 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a06e575bc060ce162d31 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:25,870 - lava_callback - INFO - Starting async job submit for node 6940a0aa575bc060ce162d7f 2025-12-18 00:10:25,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:25,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0aa575bc060ce162d7f HTTP/1.1" 502 150 2025-12-18 00:10:25,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0aa575bc060ce162d7f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:25,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0aa575bc060ce162d7f 2025-12-18 00:10:25,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0aa575bc060ce162d7f HTTP/1.1" 502 150 2025-12-18 00:10:25,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0aa575bc060ce162d7f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:26,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a075575bc060ce162d37 2025-12-18 00:10:26,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a075575bc060ce162d37 HTTP/1.1" 502 150 2025-12-18 00:10:26,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a075575bc060ce162d37'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:26,530 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a099575bc060ce162d67 2025-12-18 00:10:26,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a099575bc060ce162d67 HTTP/1.1" 502 150 2025-12-18 00:10:26,531 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a099575bc060ce162d67'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:60654 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:10:26,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0a4575bc060ce162d6c 2025-12-18 00:10:26,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a4575bc060ce162d6c HTTP/1.1" 502 150 2025-12-18 00:10:26,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0a4575bc060ce162d6c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:26,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0a6575bc060ce162d79 2025-12-18 00:10:26,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a6575bc060ce162d79 HTTP/1.1" 502 150 2025-12-18 00:10:26,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0a6575bc060ce162d79'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:27,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a095575bc060ce162d5c 2025-12-18 00:10:27,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a095575bc060ce162d5c HTTP/1.1" 502 150 2025-12-18 00:10:27,118 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a095575bc060ce162d5c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:27,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0a9575bc060ce162d7c 2025-12-18 00:10:27,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a9575bc060ce162d7c HTTP/1.1" 502 150 2025-12-18 00:10:27,473 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0a9575bc060ce162d7c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:27,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a076575bc060ce162d3a 2025-12-18 00:10:27,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a076575bc060ce162d3a HTTP/1.1" 502 150 2025-12-18 00:10:27,601 - lava_callback - ERROR - Error processing callback for node 6940a076575bc060ce162d3a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a076575bc060ce162d3a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a076575bc060ce162d3a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a076575bc060ce162d3a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:27,603 - lava_callback - INFO - Starting async job submit for node 6940a0ac575bc060ce162d82 2025-12-18 00:10:27,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:27,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0ac575bc060ce162d82 HTTP/1.1" 502 150 2025-12-18 00:10:27,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0ac575bc060ce162d82'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:27,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0ac575bc060ce162d82 2025-12-18 00:10:27,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0ac575bc060ce162d82 HTTP/1.1" 502 150 2025-12-18 00:10:27,616 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0ac575bc060ce162d82'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:27,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0aa575bc060ce162d7f 2025-12-18 00:10:27,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0aa575bc060ce162d7f HTTP/1.1" 502 150 2025-12-18 00:10:27,879 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0aa575bc060ce162d7f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:28,333 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a095575bc060ce162d5f 2025-12-18 00:10:28,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a07b575bc060ce162d46 HTTP/1.1" 502 150 2025-12-18 00:10:28,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a07b575bc060ce162d46'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:28,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a097575bc060ce162d64 2025-12-18 00:10:28,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a097575bc060ce162d64 HTTP/1.1" 502 150 2025-12-18 00:10:28,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a097575bc060ce162d64'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:28,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a077575bc060ce162d3d 2025-12-18 00:10:28,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a077575bc060ce162d3d HTTP/1.1" 502 150 2025-12-18 00:10:28,659 - lava_callback - ERROR - Error processing callback for node 6940a077575bc060ce162d3d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a077575bc060ce162d3d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a077575bc060ce162d3d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a077575bc060ce162d3d (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:28,661 - lava_callback - INFO - Starting async job submit for node 6940a0ae575bc060ce162d85 2025-12-18 00:10:28,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:28,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0ae575bc060ce162d85 HTTP/1.1" 502 150 2025-12-18 00:10:28,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0ae575bc060ce162d85'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:28,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0ae575bc060ce162d85 2025-12-18 00:10:28,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0ae575bc060ce162d85 HTTP/1.1" 502 150 2025-12-18 00:10:28,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0ae575bc060ce162d85'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:29,616 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0ac575bc060ce162d82 2025-12-18 00:10:29,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0ac575bc060ce162d82 HTTP/1.1" 502 150 2025-12-18 00:10:29,617 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0ac575bc060ce162d82'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:30,156 - lava_callback - INFO - Received callback for node 69415346459ff5fc64d2d32a INFO: 172.21.0.1:38918 - "POST /node/69415346459ff5fc64d2d32a HTTP/1.0" 202 Accepted 2025-12-18 00:10:30,531 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a099575bc060ce162d67 2025-12-18 00:10:30,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a099575bc060ce162d67 HTTP/1.1" 502 150 2025-12-18 00:10:30,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a099575bc060ce162d67'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:30,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a090575bc060ce162d4e 2025-12-18 00:10:30,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a090575bc060ce162d4e HTTP/1.1" 502 150 2025-12-18 00:10:30,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a090575bc060ce162d4e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:30,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0a4575bc060ce162d6c 2025-12-18 00:10:30,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a4575bc060ce162d6c HTTP/1.1" 502 150 2025-12-18 00:10:30,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0a4575bc060ce162d6c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:30,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0ae575bc060ce162d85 2025-12-18 00:10:30,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0ae575bc060ce162d85 HTTP/1.1" 502 150 2025-12-18 00:10:30,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0ae575bc060ce162d85'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:30,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0a6575bc060ce162d79 2025-12-18 00:10:30,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a6575bc060ce162d79 HTTP/1.1" 502 150 2025-12-18 00:10:30,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0a6575bc060ce162d79'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:31,383 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a078575bc060ce162d40 2025-12-18 00:10:31,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a078575bc060ce162d40 HTTP/1.1" 502 150 2025-12-18 00:10:31,385 - lava_callback - ERROR - Error processing callback for node 6940a078575bc060ce162d40: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a078575bc060ce162d40 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a078575bc060ce162d40 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a078575bc060ce162d40 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:31,386 - lava_callback - INFO - Starting async job submit for node 6940a0c1575bc060ce162da1 2025-12-18 00:10:31,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:31,410 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a092575bc060ce162d59 2025-12-18 00:10:31,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a092575bc060ce162d59 HTTP/1.1" 502 150 2025-12-18 00:10:31,411 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a092575bc060ce162d59'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:31,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0c1575bc060ce162da1 HTTP/1.1" 502 150 2025-12-18 00:10:31,415 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0c1575bc060ce162da1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:31,415 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0c1575bc060ce162da1 2025-12-18 00:10:31,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0c1575bc060ce162da1 HTTP/1.1" 502 150 2025-12-18 00:10:31,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0c1575bc060ce162da1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:31,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0a9575bc060ce162d7c 2025-12-18 00:10:31,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a9575bc060ce162d7c HTTP/1.1" 502 150 2025-12-18 00:10:31,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0a9575bc060ce162d7c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:31,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0aa575bc060ce162d7f 2025-12-18 00:10:31,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0aa575bc060ce162d7f HTTP/1.1" 502 150 2025-12-18 00:10:31,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0aa575bc060ce162d7f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:33,416 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0c1575bc060ce162da1 2025-12-18 00:10:33,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0c1575bc060ce162da1 HTTP/1.1" 502 150 2025-12-18 00:10:33,418 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0c1575bc060ce162da1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:33,617 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0ac575bc060ce162d82 2025-12-18 00:10:33,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0ac575bc060ce162d82 HTTP/1.1" 502 150 2025-12-18 00:10:33,619 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0ac575bc060ce162d82'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:34,627 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a07a575bc060ce162d43 2025-12-18 00:10:34,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a07a575bc060ce162d43 HTTP/1.1" 502 150 2025-12-18 00:10:34,628 - lava_callback - ERROR - Error processing callback for node 6940a07a575bc060ce162d43: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a07a575bc060ce162d43 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a07a575bc060ce162d43 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a07a575bc060ce162d43 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:34,631 - lava_callback - INFO - Starting async job submit for node 6940a0b8575bc060ce162d8a 2025-12-18 00:10:34,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:34,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0b8575bc060ce162d8a HTTP/1.1" 502 150 2025-12-18 00:10:34,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0b8575bc060ce162d8a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:34,650 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0b8575bc060ce162d8a 2025-12-18 00:10:34,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0b8575bc060ce162d8a HTTP/1.1" 502 150 2025-12-18 00:10:34,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0b8575bc060ce162d8a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:34,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0ae575bc060ce162d85 2025-12-18 00:10:34,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0ae575bc060ce162d85 HTTP/1.1" 502 150 2025-12-18 00:10:34,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0ae575bc060ce162d85'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:35,118 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a095575bc060ce162d5c 2025-12-18 00:10:35,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a095575bc060ce162d5c HTTP/1.1" 502 150 2025-12-18 00:10:35,122 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a095575bc060ce162d5c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:36,400 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a07b575bc060ce162d46 2025-12-18 00:10:36,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a07b575bc060ce162d46 HTTP/1.1" 502 150 2025-12-18 00:10:36,402 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a07b575bc060ce162d46'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38924 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:10:36,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a097575bc060ce162d64 2025-12-18 00:10:36,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a097575bc060ce162d64 HTTP/1.1" 502 150 2025-12-18 00:10:36,610 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a097575bc060ce162d64'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:36,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0b8575bc060ce162d8a 2025-12-18 00:10:36,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0b8575bc060ce162d8a HTTP/1.1" 502 150 2025-12-18 00:10:36,652 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0b8575bc060ce162d8a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:37,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0c1575bc060ce162da1 2025-12-18 00:10:37,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0c1575bc060ce162da1 HTTP/1.1" 502 150 2025-12-18 00:10:37,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0c1575bc060ce162da1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:38,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a099575bc060ce162d67 2025-12-18 00:10:38,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a099575bc060ce162d67 HTTP/1.1" 502 150 2025-12-18 00:10:38,534 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a099575bc060ce162d67'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:38,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0a4575bc060ce162d6c 2025-12-18 00:10:38,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a4575bc060ce162d6c HTTP/1.1" 502 150 2025-12-18 00:10:38,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0a4575bc060ce162d6c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:38,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0a6575bc060ce162d79 2025-12-18 00:10:38,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a6575bc060ce162d79 HTTP/1.1" 502 150 2025-12-18 00:10:38,728 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0a6575bc060ce162d79'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:39,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0a9575bc060ce162d7c 2025-12-18 00:10:39,881 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0aa575bc060ce162d7f 2025-12-18 00:10:39,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0aa575bc060ce162d7f HTTP/1.1" 502 150 2025-12-18 00:10:39,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0aa575bc060ce162d7f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:40,652 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0b8575bc060ce162d8a 2025-12-18 00:10:40,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0b8575bc060ce162d8a HTTP/1.1" 502 150 2025-12-18 00:10:40,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0b8575bc060ce162d8a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:41,619 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0ac575bc060ce162d82 2025-12-18 00:10:41,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0ac575bc060ce162d82 HTTP/1.1" 502 150 2025-12-18 00:10:41,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0ac575bc060ce162d82'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:42,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a075575bc060ce162d37 2025-12-18 00:10:42,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a075575bc060ce162d37 HTTP/1.1" 502 150 2025-12-18 00:10:42,419 - lava_callback - ERROR - Error processing callback for node 6940a075575bc060ce162d37: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a075575bc060ce162d37 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a075575bc060ce162d37 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a075575bc060ce162d37 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:42,421 - lava_callback - INFO - Starting async job submit for node 6940a0bb575bc060ce162d93 2025-12-18 00:10:42,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:42,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bb575bc060ce162d93 HTTP/1.1" 502 150 2025-12-18 00:10:42,449 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0bb575bc060ce162d93'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:42,450 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0bb575bc060ce162d93 2025-12-18 00:10:42,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bb575bc060ce162d93 HTTP/1.1" 502 150 2025-12-18 00:10:42,450 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0bb575bc060ce162d93'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:42,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0ae575bc060ce162d85 2025-12-18 00:10:42,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0ae575bc060ce162d85 HTTP/1.1" 502 150 2025-12-18 00:10:42,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0ae575bc060ce162d85'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:43,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a095575bc060ce162d5f HTTP/1.1" 502 150 2025-12-18 00:10:43,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a095575bc060ce162d5f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:44,451 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0bb575bc060ce162d93 2025-12-18 00:10:44,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bb575bc060ce162d93 HTTP/1.1" 502 150 2025-12-18 00:10:44,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0bb575bc060ce162d93'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:45,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0c1575bc060ce162da1 2025-12-18 00:10:45,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0c1575bc060ce162da1 HTTP/1.1" 502 150 2025-12-18 00:10:45,421 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0c1575bc060ce162da1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:41912 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:10:46,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a090575bc060ce162d4e 2025-12-18 00:10:46,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a090575bc060ce162d4e HTTP/1.1" 502 150 2025-12-18 00:10:46,597 - lava_callback - ERROR - Error processing callback for node 6940a090575bc060ce162d4e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a090575bc060ce162d4e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a090575bc060ce162d4e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a090575bc060ce162d4e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:46,599 - lava_callback - INFO - Starting async job submit for node 6940a0bd575bc060ce162d9b 2025-12-18 00:10:46,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:46,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bd575bc060ce162d9b HTTP/1.1" 502 150 2025-12-18 00:10:46,606 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0bd575bc060ce162d9b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:46,606 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0bd575bc060ce162d9b 2025-12-18 00:10:46,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bd575bc060ce162d9b HTTP/1.1" 502 150 2025-12-18 00:10:46,607 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0bd575bc060ce162d9b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:47,411 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a092575bc060ce162d59 2025-12-18 00:10:47,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a092575bc060ce162d59 HTTP/1.1" 502 150 2025-12-18 00:10:47,413 - lava_callback - ERROR - Error processing callback for node 6940a092575bc060ce162d59: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a092575bc060ce162d59 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a092575bc060ce162d59 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a092575bc060ce162d59 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:47,415 - lava_callback - INFO - Starting async job submit for node 6940a0bf575bc060ce162d9e 2025-12-18 00:10:47,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:47,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bf575bc060ce162d9e HTTP/1.1" 502 150 2025-12-18 00:10:47,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0bf575bc060ce162d9e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:47,422 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0bf575bc060ce162d9e 2025-12-18 00:10:47,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bf575bc060ce162d9e HTTP/1.1" 502 150 2025-12-18 00:10:47,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0bf575bc060ce162d9e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:48,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0bb575bc060ce162d93 2025-12-18 00:10:48,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bb575bc060ce162d93 HTTP/1.1" 502 150 2025-12-18 00:10:48,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0bb575bc060ce162d93'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:48,607 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0bd575bc060ce162d9b 2025-12-18 00:10:48,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bd575bc060ce162d9b HTTP/1.1" 502 150 2025-12-18 00:10:48,608 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0bd575bc060ce162d9b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:48,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0b8575bc060ce162d8a 2025-12-18 00:10:48,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0b8575bc060ce162d8a HTTP/1.1" 502 150 2025-12-18 00:10:48,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0b8575bc060ce162d8a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:49,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0bf575bc060ce162d9e 2025-12-18 00:10:49,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bf575bc060ce162d9e HTTP/1.1" 502 150 2025-12-18 00:10:49,424 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0bf575bc060ce162d9e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:50,548 - lava_callback - INFO - Received callback for node 6940a376575bc060ce1631bb INFO: 172.21.0.1:58730 - "POST /node/6940a376575bc060ce1631bb HTTP/1.0" 202 Accepted 2025-12-18 00:10:51,122 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a095575bc060ce162d5c 2025-12-18 00:10:51,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a095575bc060ce162d5c HTTP/1.1" 502 150 2025-12-18 00:10:51,123 - lava_callback - ERROR - Error processing callback for node 6940a095575bc060ce162d5c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a095575bc060ce162d5c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a095575bc060ce162d5c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a095575bc060ce162d5c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:51,125 - lava_callback - INFO - Starting async job submit for node 69414248459ff5fc64d2ca17 2025-12-18 00:10:51,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:51,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69414248459ff5fc64d2ca17 HTTP/1.1" 502 150 2025-12-18 00:10:51,154 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69414248459ff5fc64d2ca17'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:51,154 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69414248459ff5fc64d2ca17 2025-12-18 00:10:51,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69414248459ff5fc64d2ca17 HTTP/1.1" 502 150 2025-12-18 00:10:51,155 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69414248459ff5fc64d2ca17'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:51,344 - lava_callback - INFO - Received callback for node 6940a377575bc060ce1631c0 INFO: 172.21.0.1:58744 - "POST /node/6940a377575bc060ce1631c0 HTTP/1.0" 202 Accepted 2025-12-18 00:10:51,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a095575bc060ce162d5f 2025-12-18 00:10:51,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a095575bc060ce162d5f HTTP/1.1" 502 150 2025-12-18 00:10:51,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a095575bc060ce162d5f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:52,154 - lava_callback - INFO - Received callback for node 6940a378575bc060ce1631c9 INFO: 172.21.0.1:58756 - "POST /node/6940a378575bc060ce1631c9 HTTP/1.0" 202 Accepted 2025-12-18 00:10:52,402 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a07b575bc060ce162d46 2025-12-18 00:10:52,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a07b575bc060ce162d46 HTTP/1.1" 502 150 2025-12-18 00:10:52,403 - lava_callback - ERROR - Error processing callback for node 6940a07b575bc060ce162d46: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a07b575bc060ce162d46 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a07b575bc060ce162d46 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a07b575bc060ce162d46 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:52,405 - lava_callback - INFO - Starting async job submit for node 6940a0c5575bc060ce162da6 2025-12-18 00:10:52,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:52,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0c5575bc060ce162da6 HTTP/1.1" 502 150 2025-12-18 00:10:52,412 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0c5575bc060ce162da6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:52,412 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0c5575bc060ce162da6 2025-12-18 00:10:52,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0c5575bc060ce162da6 HTTP/1.1" 502 150 2025-12-18 00:10:52,413 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0c5575bc060ce162da6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:52,608 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0bd575bc060ce162d9b 2025-12-18 00:10:52,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bd575bc060ce162d9b HTTP/1.1" 502 150 2025-12-18 00:10:52,609 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0bd575bc060ce162d9b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:52,610 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a097575bc060ce162d64 2025-12-18 00:10:52,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a097575bc060ce162d64 HTTP/1.1" 502 150 2025-12-18 00:10:52,611 - lava_callback - ERROR - Error processing callback for node 6940a097575bc060ce162d64: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a097575bc060ce162d64 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a097575bc060ce162d64 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a097575bc060ce162d64 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:52,613 - lava_callback - INFO - Starting async job submit for node 6940a0cc575bc060ce162dad 2025-12-18 00:10:52,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:52,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0cc575bc060ce162dad HTTP/1.1" 502 150 2025-12-18 00:10:52,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0cc575bc060ce162dad'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:52,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0cc575bc060ce162dad 2025-12-18 00:10:52,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0cc575bc060ce162dad HTTP/1.1" 502 150 2025-12-18 00:10:52,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0cc575bc060ce162dad'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:52,932 - lava_callback - INFO - Received callback for node 6940a379575bc060ce1631ce INFO: 172.21.0.1:58766 - "POST /node/6940a379575bc060ce1631ce HTTP/1.0" 202 Accepted 2025-12-18 00:10:53,155 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69414248459ff5fc64d2ca17 2025-12-18 00:10:53,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69414248459ff5fc64d2ca17 HTTP/1.1" 502 150 2025-12-18 00:10:53,156 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69414248459ff5fc64d2ca17'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:53,424 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0bf575bc060ce162d9e 2025-12-18 00:10:53,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bf575bc060ce162d9e HTTP/1.1" 502 150 2025-12-18 00:10:53,425 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0bf575bc060ce162d9e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:53,750 - lava_callback - INFO - Received callback for node 6940a37d575bc060ce1631d8 INFO: 172.21.0.1:58774 - "POST /node/6940a37d575bc060ce1631d8 HTTP/1.0" 202 Accepted 2025-12-18 00:10:53,913 - lava_callback - INFO - Received callback for node 6940a380575bc060ce1631e8 INFO: 172.21.0.1:58776 - "POST /node/6940a380575bc060ce1631e8 HTTP/1.0" 202 Accepted 2025-12-18 00:10:54,413 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0c5575bc060ce162da6 2025-12-18 00:10:54,534 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a099575bc060ce162d67 2025-12-18 00:10:54,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a099575bc060ce162d67 HTTP/1.1" 502 150 2025-12-18 00:10:54,536 - lava_callback - ERROR - Error processing callback for node 6940a099575bc060ce162d67: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a099575bc060ce162d67 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a099575bc060ce162d67 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a099575bc060ce162d67 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:54,537 - lava_callback - INFO - Starting async job submit for node 6940a0cf575bc060ce162db4 2025-12-18 00:10:54,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:54,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0cf575bc060ce162db4 HTTP/1.1" 502 150 2025-12-18 00:10:54,544 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0cf575bc060ce162db4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:54,544 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0cf575bc060ce162db4 2025-12-18 00:10:54,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0cf575bc060ce162db4 HTTP/1.1" 502 150 2025-12-18 00:10:54,545 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0cf575bc060ce162db4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:54,564 - lava_callback - INFO - Received callback for node 6940a37e575bc060ce1631dd INFO: 172.21.0.1:58792 - "POST /node/6940a37e575bc060ce1631dd HTTP/1.0" 202 Accepted 2025-12-18 00:10:54,609 - lava_callback - INFO - Received callback for node 6940a380575bc060ce1631eb INFO: 172.21.0.1:58806 - "POST /node/6940a380575bc060ce1631eb HTTP/1.0" 202 Accepted 2025-12-18 00:10:54,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0cc575bc060ce162dad 2025-12-18 00:10:54,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0cc575bc060ce162dad HTTP/1.1" 502 150 2025-12-18 00:10:54,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0cc575bc060ce162dad'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:54,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0a4575bc060ce162d6c 2025-12-18 00:10:54,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a4575bc060ce162d6c HTTP/1.1" 502 150 2025-12-18 00:10:54,646 - lava_callback - ERROR - Error processing callback for node 6940a0a4575bc060ce162d6c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0a4575bc060ce162d6c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0a4575bc060ce162d6c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0a4575bc060ce162d6c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:54,648 - lava_callback - INFO - Starting async job submit for node 6940a0d0575bc060ce162db7 2025-12-18 00:10:54,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:54,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d0575bc060ce162db7 HTTP/1.1" 502 150 2025-12-18 00:10:54,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d0575bc060ce162db7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:54,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d0575bc060ce162db7 2025-12-18 00:10:54,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d0575bc060ce162db7 HTTP/1.1" 502 150 2025-12-18 00:10:54,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d0575bc060ce162db7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:54,728 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0a6575bc060ce162d79 2025-12-18 00:10:54,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a6575bc060ce162d79 HTTP/1.1" 502 150 2025-12-18 00:10:54,730 - lava_callback - ERROR - Error processing callback for node 6940a0a6575bc060ce162d79: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0a6575bc060ce162d79 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0a6575bc060ce162d79 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0a6575bc060ce162d79 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:54,732 - lava_callback - INFO - Starting async job submit for node 6940a0d1575bc060ce162dba 2025-12-18 00:10:54,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:54,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d1575bc060ce162dba HTTP/1.1" 502 150 2025-12-18 00:10:54,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d1575bc060ce162dba'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:54,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d1575bc060ce162dba 2025-12-18 00:10:54,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d1575bc060ce162dba HTTP/1.1" 502 150 2025-12-18 00:10:54,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d1575bc060ce162dba'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:54,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a9575bc060ce162d7c HTTP/1.1" 502 150 2025-12-18 00:10:54,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0a9575bc060ce162d7c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:55,882 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0aa575bc060ce162d7f 2025-12-18 00:10:55,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0aa575bc060ce162d7f HTTP/1.1" 502 150 2025-12-18 00:10:55,884 - lava_callback - ERROR - Error processing callback for node 6940a0aa575bc060ce162d7f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0aa575bc060ce162d7f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0aa575bc060ce162d7f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0aa575bc060ce162d7f (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:55,885 - lava_callback - INFO - Starting async job submit for node 6940a0d3575bc060ce162dbf 2025-12-18 00:10:55,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:55,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d3575bc060ce162dbf HTTP/1.1" 502 150 2025-12-18 00:10:55,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d3575bc060ce162dbf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:55,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d3575bc060ce162dbf 2025-12-18 00:10:55,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d3575bc060ce162dbf HTTP/1.1" 502 150 2025-12-18 00:10:55,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d3575bc060ce162dbf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:56,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0bb575bc060ce162d93 2025-12-18 00:10:56,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bb575bc060ce162d93 HTTP/1.1" 502 150 2025-12-18 00:10:56,455 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0bb575bc060ce162d93'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:58820 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:10:56,545 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0cf575bc060ce162db4 2025-12-18 00:10:56,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0cf575bc060ce162db4 HTTP/1.1" 502 150 2025-12-18 00:10:56,546 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0cf575bc060ce162db4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:56,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d0575bc060ce162db7 2025-12-18 00:10:56,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d0575bc060ce162db7 HTTP/1.1" 502 150 2025-12-18 00:10:56,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d0575bc060ce162db7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:56,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d1575bc060ce162dba 2025-12-18 00:10:56,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d1575bc060ce162dba HTTP/1.1" 502 150 2025-12-18 00:10:56,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d1575bc060ce162dba'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:57,156 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69414248459ff5fc64d2ca17 2025-12-18 00:10:57,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69414248459ff5fc64d2ca17 HTTP/1.1" 502 150 2025-12-18 00:10:57,158 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69414248459ff5fc64d2ca17'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:57,386 - lava_callback - INFO - Received callback for node 69409edf575bc060ce162a9b INFO: 172.21.0.1:58828 - "POST /node/69409edf575bc060ce162a9b HTTP/1.0" 202 Accepted 2025-12-18 00:10:57,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0ac575bc060ce162d82 2025-12-18 00:10:57,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0ac575bc060ce162d82 HTTP/1.1" 502 150 2025-12-18 00:10:57,622 - lava_callback - ERROR - Error processing callback for node 6940a0ac575bc060ce162d82: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0ac575bc060ce162d82 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0ac575bc060ce162d82 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0ac575bc060ce162d82 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:57,624 - lava_callback - INFO - Starting async job submit for node 6940a0d4575bc060ce162dc4 2025-12-18 00:10:57,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:57,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d4575bc060ce162dc4 HTTP/1.1" 502 150 2025-12-18 00:10:57,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d4575bc060ce162dc4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:57,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d4575bc060ce162dc4 2025-12-18 00:10:57,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d4575bc060ce162dc4 HTTP/1.1" 502 150 2025-12-18 00:10:57,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d4575bc060ce162dc4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:57,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d3575bc060ce162dbf 2025-12-18 00:10:57,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d3575bc060ce162dbf HTTP/1.1" 502 150 2025-12-18 00:10:57,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d3575bc060ce162dbf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:58,622 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0cc575bc060ce162dad 2025-12-18 00:10:58,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0cc575bc060ce162dad HTTP/1.1" 502 150 2025-12-18 00:10:58,623 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0cc575bc060ce162dad'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:58,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0ae575bc060ce162d85 2025-12-18 00:10:58,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0ae575bc060ce162d85 HTTP/1.1" 502 150 2025-12-18 00:10:58,678 - lava_callback - ERROR - Error processing callback for node 6940a0ae575bc060ce162d85: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0ae575bc060ce162d85 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0ae575bc060ce162d85 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0ae575bc060ce162d85 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:10:58,680 - lava_callback - INFO - Starting async job submit for node 6940a0d4575bc060ce162dc9 2025-12-18 00:10:58,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:10:58,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d4575bc060ce162dc9 HTTP/1.1" 502 150 2025-12-18 00:10:58,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d4575bc060ce162dc9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:58,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d4575bc060ce162dc9 2025-12-18 00:10:58,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d4575bc060ce162dc9 HTTP/1.1" 502 150 2025-12-18 00:10:58,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d4575bc060ce162dc9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:10:59,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d4575bc060ce162dc4 2025-12-18 00:10:59,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d4575bc060ce162dc4 HTTP/1.1" 502 150 2025-12-18 00:10:59,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d4575bc060ce162dc4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:00,546 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0cf575bc060ce162db4 2025-12-18 00:11:00,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0cf575bc060ce162db4 HTTP/1.1" 502 150 2025-12-18 00:11:00,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0cf575bc060ce162db4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:00,609 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0bd575bc060ce162d9b 2025-12-18 00:11:00,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bd575bc060ce162d9b HTTP/1.1" 502 150 2025-12-18 00:11:00,611 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0bd575bc060ce162d9b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:00,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d0575bc060ce162db7 2025-12-18 00:11:00,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d0575bc060ce162db7 HTTP/1.1" 502 150 2025-12-18 00:11:00,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d0575bc060ce162db7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:00,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d4575bc060ce162dc9 2025-12-18 00:11:00,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d4575bc060ce162dc9 HTTP/1.1" 502 150 2025-12-18 00:11:00,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d4575bc060ce162dc9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:00,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d1575bc060ce162dba 2025-12-18 00:11:00,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d1575bc060ce162dba HTTP/1.1" 502 150 2025-12-18 00:11:00,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d1575bc060ce162dba'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:01,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0c1575bc060ce162da1 2025-12-18 00:11:01,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0c1575bc060ce162da1 HTTP/1.1" 502 150 2025-12-18 00:11:01,423 - lava_callback - ERROR - Error processing callback for node 6940a0c1575bc060ce162da1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0c1575bc060ce162da1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0c1575bc060ce162da1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0c1575bc060ce162da1 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:01,426 - lava_callback - INFO - Starting async job submit for node 6940a0fd575bc060ce162ddb 2025-12-18 00:11:01,426 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0bf575bc060ce162d9e 2025-12-18 00:11:01,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:01,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bf575bc060ce162d9e HTTP/1.1" 502 150 2025-12-18 00:11:01,428 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0bf575bc060ce162d9e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:01,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0fd575bc060ce162ddb HTTP/1.1" 502 150 2025-12-18 00:11:01,434 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0fd575bc060ce162ddb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:01,434 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0fd575bc060ce162ddb 2025-12-18 00:11:01,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0fd575bc060ce162ddb HTTP/1.1" 502 150 2025-12-18 00:11:01,435 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0fd575bc060ce162ddb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:01,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d3575bc060ce162dbf 2025-12-18 00:11:01,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d3575bc060ce162dbf HTTP/1.1" 502 150 2025-12-18 00:11:01,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d3575bc060ce162dbf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:03,435 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0fd575bc060ce162ddb 2025-12-18 00:11:03,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0fd575bc060ce162ddb HTTP/1.1" 502 150 2025-12-18 00:11:03,437 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0fd575bc060ce162ddb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:03,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d4575bc060ce162dc4 2025-12-18 00:11:03,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d4575bc060ce162dc4 HTTP/1.1" 502 150 2025-12-18 00:11:03,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d4575bc060ce162dc4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:33296 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:11:04,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0b8575bc060ce162d8a 2025-12-18 00:11:04,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0b8575bc060ce162d8a HTTP/1.1" 502 150 2025-12-18 00:11:04,658 - lava_callback - ERROR - Error processing callback for node 6940a0b8575bc060ce162d8a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0b8575bc060ce162d8a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0b8575bc060ce162d8a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0b8575bc060ce162d8a (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:04,660 - lava_callback - INFO - Starting async job submit for node 6940a100575bc060ce162de0 2025-12-18 00:11:04,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:04,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a100575bc060ce162de0 HTTP/1.1" 502 150 2025-12-18 00:11:04,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a100575bc060ce162de0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:04,670 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a100575bc060ce162de0 2025-12-18 00:11:04,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a100575bc060ce162de0 HTTP/1.1" 502 150 2025-12-18 00:11:04,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a100575bc060ce162de0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:04,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d4575bc060ce162dc9 2025-12-18 00:11:04,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d4575bc060ce162dc9 HTTP/1.1" 502 150 2025-12-18 00:11:04,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d4575bc060ce162dc9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:05,158 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69414248459ff5fc64d2ca17 INFO: 172.21.0.1:33312 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:11:06,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0cc575bc060ce162dad 2025-12-18 00:11:06,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0cc575bc060ce162dad HTTP/1.1" 502 150 2025-12-18 00:11:06,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0cc575bc060ce162dad'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:06,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a100575bc060ce162de0 2025-12-18 00:11:06,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a100575bc060ce162de0 HTTP/1.1" 502 150 2025-12-18 00:11:06,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a100575bc060ce162de0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:07,437 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0fd575bc060ce162ddb 2025-12-18 00:11:07,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0fd575bc060ce162ddb HTTP/1.1" 502 150 2025-12-18 00:11:07,438 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0fd575bc060ce162ddb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:07,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a095575bc060ce162d5f 2025-12-18 00:11:07,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a095575bc060ce162d5f HTTP/1.1" 502 150 2025-12-18 00:11:07,763 - lava_callback - ERROR - Error processing callback for node 6940a095575bc060ce162d5f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a095575bc060ce162d5f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a095575bc060ce162d5f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a095575bc060ce162d5f (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:07,765 - lava_callback - INFO - Starting async job submit for node 6940a103575bc060ce162de5 2025-12-18 00:11:07,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:07,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a103575bc060ce162de5 HTTP/1.1" 502 150 2025-12-18 00:11:07,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a103575bc060ce162de5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:07,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a103575bc060ce162de5 2025-12-18 00:11:07,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a103575bc060ce162de5 HTTP/1.1" 502 150 2025-12-18 00:11:07,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a103575bc060ce162de5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:08,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0cf575bc060ce162db4 2025-12-18 00:11:08,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0cf575bc060ce162db4 HTTP/1.1" 502 150 2025-12-18 00:11:08,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0cf575bc060ce162db4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:08,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d0575bc060ce162db7 2025-12-18 00:11:08,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d0575bc060ce162db7 HTTP/1.1" 502 150 2025-12-18 00:11:08,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d0575bc060ce162db7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:08,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d1575bc060ce162dba 2025-12-18 00:11:08,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d1575bc060ce162dba HTTP/1.1" 502 150 2025-12-18 00:11:08,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d1575bc060ce162dba'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:09,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a103575bc060ce162de5 2025-12-18 00:11:09,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a103575bc060ce162de5 HTTP/1.1" 502 150 2025-12-18 00:11:09,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a103575bc060ce162de5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:09,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0c5575bc060ce162da6 HTTP/1.1" 502 150 2025-12-18 00:11:09,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0c5575bc060ce162da6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:09,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d3575bc060ce162dbf 2025-12-18 00:11:09,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d3575bc060ce162dbf HTTP/1.1" 502 150 2025-12-18 00:11:09,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d3575bc060ce162dbf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:10,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a100575bc060ce162de0 2025-12-18 00:11:10,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a100575bc060ce162de0 HTTP/1.1" 502 150 2025-12-18 00:11:10,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a100575bc060ce162de0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:10,768 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0a9575bc060ce162d7c 2025-12-18 00:11:10,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0a9575bc060ce162d7c HTTP/1.1" 502 150 2025-12-18 00:11:10,769 - lava_callback - ERROR - Error processing callback for node 6940a0a9575bc060ce162d7c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0a9575bc060ce162d7c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0a9575bc060ce162d7c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0a9575bc060ce162d7c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:10,771 - lava_callback - INFO - Starting async job submit for node 6940a104575bc060ce162de8 2025-12-18 00:11:10,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:10,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a104575bc060ce162de8 HTTP/1.1" 502 150 2025-12-18 00:11:10,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a104575bc060ce162de8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:10,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a104575bc060ce162de8 2025-12-18 00:11:10,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a104575bc060ce162de8 HTTP/1.1" 502 150 2025-12-18 00:11:10,804 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a104575bc060ce162de8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:11,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d4575bc060ce162dc4 2025-12-18 00:11:11,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d4575bc060ce162dc4 HTTP/1.1" 502 150 2025-12-18 00:11:11,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d4575bc060ce162dc4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:12,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0bb575bc060ce162d93 2025-12-18 00:11:12,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bb575bc060ce162d93 HTTP/1.1" 502 150 2025-12-18 00:11:12,456 - lava_callback - ERROR - Error processing callback for node 6940a0bb575bc060ce162d93: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0bb575bc060ce162d93 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0bb575bc060ce162d93 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0bb575bc060ce162d93 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:12,458 - lava_callback - INFO - Starting async job submit for node 6940a105575bc060ce162deb 2025-12-18 00:11:12,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:12,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a105575bc060ce162deb HTTP/1.1" 502 150 2025-12-18 00:11:12,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a105575bc060ce162deb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:12,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a105575bc060ce162deb 2025-12-18 00:11:12,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a105575bc060ce162deb HTTP/1.1" 502 150 2025-12-18 00:11:12,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a105575bc060ce162deb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:12,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d4575bc060ce162dc9 2025-12-18 00:11:12,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d4575bc060ce162dc9 HTTP/1.1" 502 150 2025-12-18 00:11:12,693 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0d4575bc060ce162dc9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:12,804 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a104575bc060ce162de8 2025-12-18 00:11:12,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a104575bc060ce162de8 HTTP/1.1" 502 150 2025-12-18 00:11:12,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a104575bc060ce162de8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:13,797 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a103575bc060ce162de5 2025-12-18 00:11:13,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a103575bc060ce162de5 HTTP/1.1" 502 150 2025-12-18 00:11:13,798 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a103575bc060ce162de5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:13,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0c5575bc060ce162da6 2025-12-18 00:11:13,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0c5575bc060ce162da6 HTTP/1.1" 502 150 2025-12-18 00:11:13,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0c5575bc060ce162da6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:14,254 - lava_callback - INFO - Received callback for node 6940a391575bc060ce163200 INFO: 172.21.0.1:47194 - "POST /node/6940a391575bc060ce163200 HTTP/1.0" 202 Accepted 2025-12-18 00:11:14,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a105575bc060ce162deb 2025-12-18 00:11:14,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a105575bc060ce162deb HTTP/1.1" 502 150 2025-12-18 00:11:14,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a105575bc060ce162deb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:14,945 - lava_callback - INFO - Received callback for node 6940a391575bc060ce163205 INFO: 172.21.0.1:47210 - "POST /node/6940a391575bc060ce163205 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:47222 - "GET /api/metrics HTTP/1.0" 200 OK 2025-12-18 00:11:15,438 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0fd575bc060ce162ddb 2025-12-18 00:11:15,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0fd575bc060ce162ddb HTTP/1.1" 502 150 2025-12-18 00:11:15,440 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0fd575bc060ce162ddb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:15,631 - lava_callback - INFO - Received callback for node 6940a392575bc060ce16320a INFO: 172.21.0.1:47236 - "POST /node/6940a392575bc060ce16320a HTTP/1.0" 202 Accepted INFO: 172.21.0.1:47240 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:11:16,611 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0bd575bc060ce162d9b 2025-12-18 00:11:16,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bd575bc060ce162d9b HTTP/1.1" 502 150 2025-12-18 00:11:16,612 - lava_callback - ERROR - Error processing callback for node 6940a0bd575bc060ce162d9b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0bd575bc060ce162d9b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0bd575bc060ce162d9b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0bd575bc060ce162d9b (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:16,616 - lava_callback - INFO - Starting async job submit for node 6940a112575bc060ce162df3 2025-12-18 00:11:16,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:16,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a112575bc060ce162df3 HTTP/1.1" 502 150 2025-12-18 00:11:16,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a112575bc060ce162df3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:16,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a112575bc060ce162df3 2025-12-18 00:11:16,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a112575bc060ce162df3 HTTP/1.1" 502 150 2025-12-18 00:11:16,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a112575bc060ce162df3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:16,806 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a104575bc060ce162de8 2025-12-18 00:11:16,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a104575bc060ce162de8 HTTP/1.1" 502 150 2025-12-18 00:11:16,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a104575bc060ce162de8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:17,428 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0bf575bc060ce162d9e 2025-12-18 00:11:17,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0bf575bc060ce162d9e HTTP/1.1" 502 150 2025-12-18 00:11:17,430 - lava_callback - ERROR - Error processing callback for node 6940a0bf575bc060ce162d9e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0bf575bc060ce162d9e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0bf575bc060ce162d9e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0bf575bc060ce162d9e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:17,433 - lava_callback - INFO - Starting async job submit for node 6940a129575bc060ce162dfa 2025-12-18 00:11:17,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:17,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a129575bc060ce162dfa HTTP/1.1" 502 150 2025-12-18 00:11:17,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a129575bc060ce162dfa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:17,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a129575bc060ce162dfa 2025-12-18 00:11:17,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a129575bc060ce162dfa HTTP/1.1" 502 150 2025-12-18 00:11:17,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a129575bc060ce162dfa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:18,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a105575bc060ce162deb 2025-12-18 00:11:18,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a105575bc060ce162deb HTTP/1.1" 502 150 2025-12-18 00:11:18,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a105575bc060ce162deb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:18,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a112575bc060ce162df3 2025-12-18 00:11:18,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a112575bc060ce162df3 HTTP/1.1" 502 150 2025-12-18 00:11:18,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a112575bc060ce162df3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:18,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a100575bc060ce162de0 2025-12-18 00:11:18,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a100575bc060ce162de0 HTTP/1.1" 502 150 2025-12-18 00:11:18,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a100575bc060ce162de0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:19,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a129575bc060ce162dfa 2025-12-18 00:11:19,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a129575bc060ce162dfa HTTP/1.1" 502 150 2025-12-18 00:11:19,471 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a129575bc060ce162dfa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:20,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69414248459ff5fc64d2ca17 HTTP/1.1" 502 150 2025-12-18 00:11:20,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69414248459ff5fc64d2ca17'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:21,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a103575bc060ce162de5 2025-12-18 00:11:21,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a103575bc060ce162de5 HTTP/1.1" 502 150 2025-12-18 00:11:21,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a103575bc060ce162de5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:21,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0c5575bc060ce162da6 2025-12-18 00:11:21,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0c5575bc060ce162da6 HTTP/1.1" 502 150 2025-12-18 00:11:21,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a0c5575bc060ce162da6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:22,625 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0cc575bc060ce162dad 2025-12-18 00:11:22,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0cc575bc060ce162dad HTTP/1.1" 502 150 2025-12-18 00:11:22,627 - lava_callback - ERROR - Error processing callback for node 6940a0cc575bc060ce162dad: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0cc575bc060ce162dad (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0cc575bc060ce162dad (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0cc575bc060ce162dad (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:22,628 - lava_callback - INFO - Starting async job submit for node 6940a12d575bc060ce162dfd 2025-12-18 00:11:22,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:22,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a12d575bc060ce162dfd HTTP/1.1" 502 150 2025-12-18 00:11:22,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a12d575bc060ce162dfd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:22,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a12d575bc060ce162dfd 2025-12-18 00:11:22,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a12d575bc060ce162dfd HTTP/1.1" 502 150 2025-12-18 00:11:22,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a12d575bc060ce162dfd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:22,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a112575bc060ce162df3 2025-12-18 00:11:22,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a112575bc060ce162df3 HTTP/1.1" 502 150 2025-12-18 00:11:22,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a112575bc060ce162df3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:23,471 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a129575bc060ce162dfa 2025-12-18 00:11:23,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a129575bc060ce162dfa HTTP/1.1" 502 150 2025-12-18 00:11:23,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a129575bc060ce162dfa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:24,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0cf575bc060ce162db4 2025-12-18 00:11:24,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0cf575bc060ce162db4 HTTP/1.1" 502 150 2025-12-18 00:11:24,551 - lava_callback - ERROR - Error processing callback for node 6940a0cf575bc060ce162db4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0cf575bc060ce162db4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0cf575bc060ce162db4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0cf575bc060ce162db4 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:24,552 - lava_callback - INFO - Starting async job submit for node 6940a139575bc060ce162e02 2025-12-18 00:11:24,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:24,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a139575bc060ce162e02 HTTP/1.1" 502 150 2025-12-18 00:11:24,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a139575bc060ce162e02'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:24,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a139575bc060ce162e02 2025-12-18 00:11:24,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a139575bc060ce162e02 HTTP/1.1" 502 150 2025-12-18 00:11:24,592 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a139575bc060ce162e02'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:24,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a12d575bc060ce162dfd 2025-12-18 00:11:24,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a12d575bc060ce162dfd HTTP/1.1" 502 150 2025-12-18 00:11:24,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a12d575bc060ce162dfd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:24,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d0575bc060ce162db7 2025-12-18 00:11:24,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d0575bc060ce162db7 HTTP/1.1" 502 150 2025-12-18 00:11:24,661 - lava_callback - ERROR - Error processing callback for node 6940a0d0575bc060ce162db7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0d0575bc060ce162db7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0d0575bc060ce162db7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0d0575bc060ce162db7 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:24,663 - lava_callback - INFO - Starting async job submit for node 6940a95b1e3c24e95681a22b 2025-12-18 00:11:24,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d1575bc060ce162dba 2025-12-18 00:11:24,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:24,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d1575bc060ce162dba HTTP/1.1" 502 150 2025-12-18 00:11:24,748 - lava_callback - ERROR - Error processing callback for node 6940a0d1575bc060ce162dba: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0d1575bc060ce162dba (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0d1575bc060ce162dba (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0d1575bc060ce162dba (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:24,750 - lava_callback - INFO - Starting async job submit for node 6940a144575bc060ce162e12 2025-12-18 00:11:24,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:24,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a95b1e3c24e95681a22b HTTP/1.1" 502 150 2025-12-18 00:11:24,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a95b1e3c24e95681a22b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:24,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a95b1e3c24e95681a22b 2025-12-18 00:11:24,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a95b1e3c24e95681a22b HTTP/1.1" 502 150 2025-12-18 00:11:24,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a95b1e3c24e95681a22b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:24,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a144575bc060ce162e12 HTTP/1.1" 502 150 2025-12-18 00:11:24,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a144575bc060ce162e12'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:24,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a144575bc060ce162e12 2025-12-18 00:11:24,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a144575bc060ce162e12 HTTP/1.1" 502 150 2025-12-18 00:11:24,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a144575bc060ce162e12'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:24,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a104575bc060ce162de8 2025-12-18 00:11:24,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a104575bc060ce162de8 HTTP/1.1" 502 150 2025-12-18 00:11:24,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a104575bc060ce162de8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:25,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d3575bc060ce162dbf 2025-12-18 00:11:25,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d3575bc060ce162dbf HTTP/1.1" 502 150 2025-12-18 00:11:25,900 - lava_callback - ERROR - Error processing callback for node 6940a0d3575bc060ce162dbf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0d3575bc060ce162dbf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0d3575bc060ce162dbf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0d3575bc060ce162dbf (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:25,901 - lava_callback - INFO - Starting async job submit for node 6940a147575bc060ce162e19 2025-12-18 00:11:25,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:25,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a147575bc060ce162e19 HTTP/1.1" 502 150 2025-12-18 00:11:25,911 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a147575bc060ce162e19'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:25,911 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a147575bc060ce162e19 2025-12-18 00:11:25,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a147575bc060ce162e19 HTTP/1.1" 502 150 2025-12-18 00:11:25,912 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a147575bc060ce162e19'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:26,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a105575bc060ce162deb 2025-12-18 00:11:26,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a105575bc060ce162deb HTTP/1.1" 502 150 2025-12-18 00:11:26,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a105575bc060ce162deb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:50014 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:11:26,592 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a139575bc060ce162e02 2025-12-18 00:11:26,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a139575bc060ce162e02 HTTP/1.1" 502 150 2025-12-18 00:11:26,593 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a139575bc060ce162e02'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:26,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a95b1e3c24e95681a22b 2025-12-18 00:11:26,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a95b1e3c24e95681a22b HTTP/1.1" 502 150 2025-12-18 00:11:26,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a95b1e3c24e95681a22b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:26,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a144575bc060ce162e12 2025-12-18 00:11:26,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a144575bc060ce162e12 HTTP/1.1" 502 150 2025-12-18 00:11:26,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a144575bc060ce162e12'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:27,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d4575bc060ce162dc4 2025-12-18 00:11:27,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d4575bc060ce162dc4 HTTP/1.1" 502 150 2025-12-18 00:11:27,662 - lava_callback - ERROR - Error processing callback for node 6940a0d4575bc060ce162dc4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0d4575bc060ce162dc4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0d4575bc060ce162dc4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0d4575bc060ce162dc4 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:27,664 - lava_callback - INFO - Starting async job submit for node 6940a149575bc060ce162e1c 2025-12-18 00:11:27,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:27,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a149575bc060ce162e1c HTTP/1.1" 502 150 2025-12-18 00:11:27,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a149575bc060ce162e1c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:27,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a149575bc060ce162e1c 2025-12-18 00:11:27,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a149575bc060ce162e1c HTTP/1.1" 502 150 2025-12-18 00:11:27,672 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a149575bc060ce162e1c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:27,912 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a147575bc060ce162e19 2025-12-18 00:11:27,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a147575bc060ce162e19 HTTP/1.1" 502 150 2025-12-18 00:11:27,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a147575bc060ce162e19'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:28,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a12d575bc060ce162dfd 2025-12-18 00:11:28,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a12d575bc060ce162dfd HTTP/1.1" 502 150 2025-12-18 00:11:28,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a12d575bc060ce162dfd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:28,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0d4575bc060ce162dc9 2025-12-18 00:11:28,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0d4575bc060ce162dc9 HTTP/1.1" 502 150 2025-12-18 00:11:28,694 - lava_callback - ERROR - Error processing callback for node 6940a0d4575bc060ce162dc9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0d4575bc060ce162dc9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0d4575bc060ce162dc9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0d4575bc060ce162dc9 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:28,696 - lava_callback - INFO - Starting async job submit for node 6940a14c575bc060ce162e21 2025-12-18 00:11:28,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:28,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a14c575bc060ce162e21 HTTP/1.1" 502 150 2025-12-18 00:11:28,703 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a14c575bc060ce162e21'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:28,703 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a14c575bc060ce162e21 2025-12-18 00:11:28,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a14c575bc060ce162e21 HTTP/1.1" 502 150 2025-12-18 00:11:28,704 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a14c575bc060ce162e21'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:29,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a149575bc060ce162e1c 2025-12-18 00:11:29,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a149575bc060ce162e1c HTTP/1.1" 502 150 2025-12-18 00:11:29,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a149575bc060ce162e1c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:30,593 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a139575bc060ce162e02 2025-12-18 00:11:30,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a139575bc060ce162e02 HTTP/1.1" 502 150 2025-12-18 00:11:30,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a139575bc060ce162e02'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:30,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a112575bc060ce162df3 2025-12-18 00:11:30,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a112575bc060ce162df3 HTTP/1.1" 502 150 2025-12-18 00:11:30,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a112575bc060ce162df3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:30,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a14c575bc060ce162e21 2025-12-18 00:11:30,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a14c575bc060ce162e21 HTTP/1.1" 502 150 2025-12-18 00:11:30,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a14c575bc060ce162e21'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:30,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a95b1e3c24e95681a22b 2025-12-18 00:11:30,759 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a144575bc060ce162e12 2025-12-18 00:11:30,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a95b1e3c24e95681a22b HTTP/1.1" 502 150 2025-12-18 00:11:30,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a95b1e3c24e95681a22b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:30,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a144575bc060ce162e12 HTTP/1.1" 502 150 2025-12-18 00:11:30,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a144575bc060ce162e12'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:31,440 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0fd575bc060ce162ddb 2025-12-18 00:11:31,473 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a129575bc060ce162dfa 2025-12-18 00:11:31,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a129575bc060ce162dfa HTTP/1.1" 502 150 2025-12-18 00:11:31,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a129575bc060ce162dfa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:31,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a147575bc060ce162e19 2025-12-18 00:11:31,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a147575bc060ce162e19 HTTP/1.1" 502 150 2025-12-18 00:11:31,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a147575bc060ce162e19'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:33,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a149575bc060ce162e1c 2025-12-18 00:11:33,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a149575bc060ce162e1c HTTP/1.1" 502 150 2025-12-18 00:11:33,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a149575bc060ce162e1c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:33,942 - lava_callback - INFO - Received callback for node 6940a3a7575bc060ce163213 INFO: 172.21.0.1:39900 - "POST /node/6940a3a7575bc060ce163213 HTTP/1.0" 202 Accepted 2025-12-18 00:11:34,630 - lava_callback - INFO - Received callback for node 6940a3aa575bc060ce163216 INFO: 172.21.0.1:39916 - "POST /node/6940a3aa575bc060ce163216 HTTP/1.0" 202 Accepted 2025-12-18 00:11:34,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a100575bc060ce162de0 2025-12-18 00:11:34,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a100575bc060ce162de0 HTTP/1.1" 502 150 2025-12-18 00:11:34,676 - lava_callback - ERROR - Error processing callback for node 6940a100575bc060ce162de0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a100575bc060ce162de0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a100575bc060ce162de0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a100575bc060ce162de0 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:34,678 - lava_callback - INFO - Starting async job submit for node 6940a150575bc060ce162e36 2025-12-18 00:11:34,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:34,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a150575bc060ce162e36 HTTP/1.1" 502 150 2025-12-18 00:11:34,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a150575bc060ce162e36'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:34,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a150575bc060ce162e36 2025-12-18 00:11:34,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a150575bc060ce162e36 HTTP/1.1" 502 150 2025-12-18 00:11:34,685 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a150575bc060ce162e36'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:34,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a14c575bc060ce162e21 2025-12-18 00:11:34,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a14c575bc060ce162e21 HTTP/1.1" 502 150 2025-12-18 00:11:34,706 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a14c575bc060ce162e21'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:35,308 - lava_callback - INFO - Received callback for node 6940a3ae575bc060ce163219 INFO: 172.21.0.1:39926 - "POST /node/6940a3ae575bc060ce163219 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:39936 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:11:36,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69414248459ff5fc64d2ca17 2025-12-18 00:11:36,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69414248459ff5fc64d2ca17 HTTP/1.1" 502 150 2025-12-18 00:11:36,625 - lava_callback - ERROR - Error processing callback for node 69414248459ff5fc64d2ca17: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69414248459ff5fc64d2ca17 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69414248459ff5fc64d2ca17 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69414248459ff5fc64d2ca17 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:36,627 - lava_callback - INFO - Starting async job submit for node 6940a158575bc060ce162e3b 2025-12-18 00:11:36,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:36,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a158575bc060ce162e3b HTTP/1.1" 502 150 2025-12-18 00:11:36,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a158575bc060ce162e3b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:36,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a158575bc060ce162e3b 2025-12-18 00:11:36,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a158575bc060ce162e3b HTTP/1.1" 502 150 2025-12-18 00:11:36,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a158575bc060ce162e3b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:36,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a12d575bc060ce162dfd 2025-12-18 00:11:36,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a12d575bc060ce162dfd HTTP/1.1" 502 150 2025-12-18 00:11:36,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a12d575bc060ce162dfd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:36,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a150575bc060ce162e36 2025-12-18 00:11:36,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a150575bc060ce162e36 HTTP/1.1" 502 150 2025-12-18 00:11:36,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a150575bc060ce162e36'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:37,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a103575bc060ce162de5 2025-12-18 00:11:37,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a103575bc060ce162de5 HTTP/1.1" 502 150 2025-12-18 00:11:37,802 - lava_callback - ERROR - Error processing callback for node 6940a103575bc060ce162de5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a103575bc060ce162de5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a103575bc060ce162de5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a103575bc060ce162de5 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:37,805 - lava_callback - INFO - Starting async job submit for node 6940a15a575bc060ce162e45 2025-12-18 00:11:37,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:37,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15a575bc060ce162e45 HTTP/1.1" 502 150 2025-12-18 00:11:37,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a15a575bc060ce162e45'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:37,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a15a575bc060ce162e45 2025-12-18 00:11:37,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15a575bc060ce162e45 HTTP/1.1" 502 150 2025-12-18 00:11:37,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a15a575bc060ce162e45'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:37,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a0c5575bc060ce162da6 2025-12-18 00:11:37,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0c5575bc060ce162da6 HTTP/1.1" 502 150 2025-12-18 00:11:37,876 - lava_callback - ERROR - Error processing callback for node 6940a0c5575bc060ce162da6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0c5575bc060ce162da6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0c5575bc060ce162da6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0c5575bc060ce162da6 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:37,878 - lava_callback - INFO - Starting async job submit for node 69414297459ff5fc64d2ca36 2025-12-18 00:11:37,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:37,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69414297459ff5fc64d2ca36 HTTP/1.1" 502 150 2025-12-18 00:11:37,913 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69414297459ff5fc64d2ca36'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:37,913 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69414297459ff5fc64d2ca36 2025-12-18 00:11:37,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69414297459ff5fc64d2ca36 HTTP/1.1" 502 150 2025-12-18 00:11:37,914 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69414297459ff5fc64d2ca36'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:38,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a139575bc060ce162e02 2025-12-18 00:11:38,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a139575bc060ce162e02 HTTP/1.1" 502 150 2025-12-18 00:11:38,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a139575bc060ce162e02'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:38,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a158575bc060ce162e3b 2025-12-18 00:11:38,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a158575bc060ce162e3b HTTP/1.1" 502 150 2025-12-18 00:11:38,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a158575bc060ce162e3b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:38,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a95b1e3c24e95681a22b 2025-12-18 00:11:38,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a144575bc060ce162e12 2025-12-18 00:11:38,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a144575bc060ce162e12 HTTP/1.1" 502 150 2025-12-18 00:11:38,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a144575bc060ce162e12'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:38,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a95b1e3c24e95681a22b HTTP/1.1" 502 150 2025-12-18 00:11:38,762 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a95b1e3c24e95681a22b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:39,838 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a15a575bc060ce162e45 2025-12-18 00:11:39,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15a575bc060ce162e45 HTTP/1.1" 502 150 2025-12-18 00:11:39,839 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a15a575bc060ce162e45'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:39,914 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69414297459ff5fc64d2ca36 2025-12-18 00:11:39,915 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a147575bc060ce162e19 2025-12-18 00:11:39,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69414297459ff5fc64d2ca36 HTTP/1.1" 502 150 2025-12-18 00:11:39,915 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69414297459ff5fc64d2ca36'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:39,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a147575bc060ce162e19 HTTP/1.1" 502 150 2025-12-18 00:11:39,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a147575bc060ce162e19'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:40,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a150575bc060ce162e36 2025-12-18 00:11:40,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a150575bc060ce162e36 HTTP/1.1" 502 150 2025-12-18 00:11:40,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a150575bc060ce162e36'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:40,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a104575bc060ce162de8 2025-12-18 00:11:40,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a104575bc060ce162de8 HTTP/1.1" 502 150 2025-12-18 00:11:40,809 - lava_callback - ERROR - Error processing callback for node 6940a104575bc060ce162de8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a104575bc060ce162de8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a104575bc060ce162de8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a104575bc060ce162de8 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:40,811 - lava_callback - INFO - Starting async job submit for node 6940a15c575bc060ce162e4d 2025-12-18 00:11:40,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:40,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15c575bc060ce162e4d HTTP/1.1" 502 150 2025-12-18 00:11:40,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a15c575bc060ce162e4d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:40,818 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a15c575bc060ce162e4d 2025-12-18 00:11:40,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15c575bc060ce162e4d HTTP/1.1" 502 150 2025-12-18 00:11:40,819 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a15c575bc060ce162e4d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:41,675 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a149575bc060ce162e1c 2025-12-18 00:11:41,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a149575bc060ce162e1c HTTP/1.1" 502 150 2025-12-18 00:11:41,676 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a149575bc060ce162e1c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:42,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a105575bc060ce162deb 2025-12-18 00:11:42,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a105575bc060ce162deb HTTP/1.1" 502 150 2025-12-18 00:11:42,473 - lava_callback - ERROR - Error processing callback for node 6940a105575bc060ce162deb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a105575bc060ce162deb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a105575bc060ce162deb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a105575bc060ce162deb (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:42,475 - lava_callback - INFO - Starting async job submit for node 6940a15e575bc060ce162e50 2025-12-18 00:11:42,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:42,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15e575bc060ce162e50 HTTP/1.1" 502 150 2025-12-18 00:11:42,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a15e575bc060ce162e50'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:42,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a15e575bc060ce162e50 2025-12-18 00:11:42,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15e575bc060ce162e50 HTTP/1.1" 502 150 2025-12-18 00:11:42,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a15e575bc060ce162e50'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:42,637 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a158575bc060ce162e3b 2025-12-18 00:11:42,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a158575bc060ce162e3b HTTP/1.1" 502 150 2025-12-18 00:11:42,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a158575bc060ce162e3b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:42,706 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a14c575bc060ce162e21 2025-12-18 00:11:42,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a14c575bc060ce162e21 HTTP/1.1" 502 150 2025-12-18 00:11:42,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a14c575bc060ce162e21'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:42,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a15c575bc060ce162e4d 2025-12-18 00:11:42,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15c575bc060ce162e4d HTTP/1.1" 502 150 2025-12-18 00:11:42,820 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a15c575bc060ce162e4d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:43,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a15a575bc060ce162e45 2025-12-18 00:11:43,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15a575bc060ce162e45 HTTP/1.1" 502 150 2025-12-18 00:11:43,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a15a575bc060ce162e45'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:43,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69414297459ff5fc64d2ca36 2025-12-18 00:11:43,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69414297459ff5fc64d2ca36 HTTP/1.1" 502 150 2025-12-18 00:11:43,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69414297459ff5fc64d2ca36'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:44,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a15e575bc060ce162e50 2025-12-18 00:11:44,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15e575bc060ce162e50 HTTP/1.1" 502 150 2025-12-18 00:11:44,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a15e575bc060ce162e50'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:43886 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:11:46,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a112575bc060ce162df3 2025-12-18 00:11:46,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a112575bc060ce162df3 HTTP/1.1" 502 150 2025-12-18 00:11:46,661 - lava_callback - ERROR - Error processing callback for node 6940a112575bc060ce162df3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a112575bc060ce162df3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a112575bc060ce162df3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a112575bc060ce162df3 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:46,663 - lava_callback - INFO - Starting async job submit for node 6940a160575bc060ce162e53 2025-12-18 00:11:46,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:46,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a160575bc060ce162e53 HTTP/1.1" 502 150 2025-12-18 00:11:46,670 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a160575bc060ce162e53'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:46,671 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a160575bc060ce162e53 2025-12-18 00:11:46,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a160575bc060ce162e53 HTTP/1.1" 502 150 2025-12-18 00:11:46,671 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a160575bc060ce162e53'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:46,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a0fd575bc060ce162ddb HTTP/1.1" 502 150 2025-12-18 00:11:46,736 - lava_callback - ERROR - Error processing callback for node 6940a0fd575bc060ce162ddb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0fd575bc060ce162ddb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0fd575bc060ce162ddb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a0fd575bc060ce162ddb (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:46,738 - lava_callback - INFO - Starting async job submit for node 6940a162575bc060ce162e56 2025-12-18 00:11:46,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:46,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a162575bc060ce162e56 HTTP/1.1" 502 150 2025-12-18 00:11:46,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a162575bc060ce162e56'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:46,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a162575bc060ce162e56 2025-12-18 00:11:46,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a162575bc060ce162e56 HTTP/1.1" 502 150 2025-12-18 00:11:46,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a162575bc060ce162e56'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:46,820 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a15c575bc060ce162e4d 2025-12-18 00:11:46,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15c575bc060ce162e4d HTTP/1.1" 502 150 2025-12-18 00:11:46,821 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a15c575bc060ce162e4d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:47,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a129575bc060ce162dfa 2025-12-18 00:11:47,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a129575bc060ce162dfa HTTP/1.1" 502 150 2025-12-18 00:11:47,475 - lava_callback - ERROR - Error processing callback for node 6940a129575bc060ce162dfa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a129575bc060ce162dfa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a129575bc060ce162dfa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a129575bc060ce162dfa (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:47,477 - lava_callback - INFO - Starting async job submit for node 6940a164575bc060ce162e59 2025-12-18 00:11:47,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:47,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a164575bc060ce162e59 HTTP/1.1" 502 150 2025-12-18 00:11:47,484 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a164575bc060ce162e59'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:47,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a164575bc060ce162e59 2025-12-18 00:11:47,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a164575bc060ce162e59 HTTP/1.1" 502 150 2025-12-18 00:11:47,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a164575bc060ce162e59'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:47,707 - lava_callback - INFO - Received callback for node 69409870575bc060ce161aa1 INFO: 172.21.0.1:43892 - "POST /node/69409870575bc060ce161aa1 HTTP/1.0" 202 Accepted 2025-12-18 00:11:48,484 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a15e575bc060ce162e50 2025-12-18 00:11:48,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15e575bc060ce162e50 HTTP/1.1" 502 150 2025-12-18 00:11:48,485 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a15e575bc060ce162e50'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:48,672 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a160575bc060ce162e53 2025-12-18 00:11:48,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a160575bc060ce162e53 HTTP/1.1" 502 150 2025-12-18 00:11:48,673 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a160575bc060ce162e53'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:48,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a150575bc060ce162e36 2025-12-18 00:11:48,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a150575bc060ce162e36 HTTP/1.1" 502 150 2025-12-18 00:11:48,691 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a150575bc060ce162e36'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:48,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a162575bc060ce162e56 2025-12-18 00:11:48,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a162575bc060ce162e56 HTTP/1.1" 502 150 2025-12-18 00:11:48,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a162575bc060ce162e56'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:49,485 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a164575bc060ce162e59 2025-12-18 00:11:49,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a164575bc060ce162e59 HTTP/1.1" 502 150 2025-12-18 00:11:49,486 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a164575bc060ce162e59'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:50,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a158575bc060ce162e3b 2025-12-18 00:11:50,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a158575bc060ce162e3b HTTP/1.1" 502 150 2025-12-18 00:11:50,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a158575bc060ce162e3b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:51,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a15a575bc060ce162e45 2025-12-18 00:11:51,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15a575bc060ce162e45 HTTP/1.1" 502 150 2025-12-18 00:11:51,843 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a15a575bc060ce162e45'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:51,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69414297459ff5fc64d2ca36 2025-12-18 00:11:51,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69414297459ff5fc64d2ca36 HTTP/1.1" 502 150 2025-12-18 00:11:51,918 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69414297459ff5fc64d2ca36'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:52,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a12d575bc060ce162dfd 2025-12-18 00:11:52,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a12d575bc060ce162dfd HTTP/1.1" 502 150 2025-12-18 00:11:52,641 - lava_callback - ERROR - Error processing callback for node 6940a12d575bc060ce162dfd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a12d575bc060ce162dfd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a12d575bc060ce162dfd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a12d575bc060ce162dfd (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:52,643 - lava_callback - INFO - Starting async job submit for node 6940a16c575bc060ce162e5e 2025-12-18 00:11:52,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:52,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a16c575bc060ce162e5e HTTP/1.1" 502 150 2025-12-18 00:11:52,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a16c575bc060ce162e5e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:52,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a16c575bc060ce162e5e 2025-12-18 00:11:52,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a16c575bc060ce162e5e HTTP/1.1" 502 150 2025-12-18 00:11:52,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a16c575bc060ce162e5e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:52,673 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a160575bc060ce162e53 2025-12-18 00:11:52,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a160575bc060ce162e53 HTTP/1.1" 502 150 2025-12-18 00:11:52,674 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a160575bc060ce162e53'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:52,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a162575bc060ce162e56 2025-12-18 00:11:52,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a162575bc060ce162e56 HTTP/1.1" 502 150 2025-12-18 00:11:52,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a162575bc060ce162e56'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:53,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a164575bc060ce162e59 2025-12-18 00:11:53,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a164575bc060ce162e59 HTTP/1.1" 502 150 2025-12-18 00:11:53,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a164575bc060ce162e59'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:54,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a139575bc060ce162e02 2025-12-18 00:11:54,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a139575bc060ce162e02 HTTP/1.1" 502 150 2025-12-18 00:11:54,598 - lava_callback - ERROR - Error processing callback for node 6940a139575bc060ce162e02: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a139575bc060ce162e02 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a139575bc060ce162e02 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a139575bc060ce162e02 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:54,600 - lava_callback - INFO - Starting async job submit for node 6940a16e575bc060ce162e69 2025-12-18 00:11:54,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:54,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a16e575bc060ce162e69 HTTP/1.1" 502 150 2025-12-18 00:11:54,635 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a16e575bc060ce162e69'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:54,635 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a16e575bc060ce162e69 2025-12-18 00:11:54,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a16e575bc060ce162e69 HTTP/1.1" 502 150 2025-12-18 00:11:54,636 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a16e575bc060ce162e69'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:54,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a16c575bc060ce162e5e 2025-12-18 00:11:54,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a16c575bc060ce162e5e HTTP/1.1" 502 150 2025-12-18 00:11:54,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a16c575bc060ce162e5e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:54,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a144575bc060ce162e12 2025-12-18 00:11:54,762 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a95b1e3c24e95681a22b 2025-12-18 00:11:54,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a144575bc060ce162e12 HTTP/1.1" 502 150 2025-12-18 00:11:54,764 - lava_callback - ERROR - Error processing callback for node 6940a144575bc060ce162e12: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a144575bc060ce162e12 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a144575bc060ce162e12 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a144575bc060ce162e12 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:54,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a95b1e3c24e95681a22b HTTP/1.1" 502 150 2025-12-18 00:11:54,766 - lava_callback - ERROR - Error processing callback for node 6940a95b1e3c24e95681a22b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a95b1e3c24e95681a22b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a95b1e3c24e95681a22b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a95b1e3c24e95681a22b (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:54,766 - lava_callback - INFO - Starting async job submit for node 6940a17a575bc060ce162e90 2025-12-18 00:11:54,768 - lava_callback - INFO - Starting async job submit for node 6940a170575bc060ce162e70 2025-12-18 00:11:54,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:54,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:54,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17a575bc060ce162e90 HTTP/1.1" 502 150 2025-12-18 00:11:54,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17a575bc060ce162e90'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:54,775 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17a575bc060ce162e90 2025-12-18 00:11:54,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a170575bc060ce162e70 HTTP/1.1" 502 150 2025-12-18 00:11:54,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a170575bc060ce162e70'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:54,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a170575bc060ce162e70 2025-12-18 00:11:54,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17a575bc060ce162e90 HTTP/1.1" 502 150 2025-12-18 00:11:54,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17a575bc060ce162e90'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:54,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a170575bc060ce162e70 HTTP/1.1" 502 150 2025-12-18 00:11:54,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a170575bc060ce162e70'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:54,821 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a15c575bc060ce162e4d 2025-12-18 00:11:54,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15c575bc060ce162e4d HTTP/1.1" 502 150 2025-12-18 00:11:54,822 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a15c575bc060ce162e4d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:55,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a147575bc060ce162e19 2025-12-18 00:11:55,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a147575bc060ce162e19 HTTP/1.1" 502 150 2025-12-18 00:11:55,917 - lava_callback - ERROR - Error processing callback for node 6940a147575bc060ce162e19: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a147575bc060ce162e19 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a147575bc060ce162e19 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a147575bc060ce162e19 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:55,919 - lava_callback - INFO - Starting async job submit for node 6940a17c575bc060ce162e93 2025-12-18 00:11:55,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:55,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17c575bc060ce162e93 HTTP/1.1" 502 150 2025-12-18 00:11:55,926 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17c575bc060ce162e93'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:55,926 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17c575bc060ce162e93 2025-12-18 00:11:55,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17c575bc060ce162e93 HTTP/1.1" 502 150 2025-12-18 00:11:55,927 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17c575bc060ce162e93'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:56,486 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a15e575bc060ce162e50 2025-12-18 00:11:56,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15e575bc060ce162e50 HTTP/1.1" 502 150 2025-12-18 00:11:56,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a15e575bc060ce162e50'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:38896 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:11:56,636 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a16e575bc060ce162e69 2025-12-18 00:11:56,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a16e575bc060ce162e69 HTTP/1.1" 502 150 2025-12-18 00:11:56,643 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a16e575bc060ce162e69'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:56,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17a575bc060ce162e90 2025-12-18 00:11:56,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a170575bc060ce162e70 2025-12-18 00:11:56,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17a575bc060ce162e90 HTTP/1.1" 502 150 2025-12-18 00:11:56,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17a575bc060ce162e90'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:56,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a170575bc060ce162e70 HTTP/1.1" 502 150 2025-12-18 00:11:56,778 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a170575bc060ce162e70'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:57,677 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a149575bc060ce162e1c 2025-12-18 00:11:57,927 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17c575bc060ce162e93 2025-12-18 00:11:57,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17c575bc060ce162e93 HTTP/1.1" 502 150 2025-12-18 00:11:57,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17c575bc060ce162e93'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:58,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a16c575bc060ce162e5e 2025-12-18 00:11:58,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a16c575bc060ce162e5e HTTP/1.1" 502 150 2025-12-18 00:11:58,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a16c575bc060ce162e5e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:58,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a14c575bc060ce162e21 2025-12-18 00:11:58,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a14c575bc060ce162e21 HTTP/1.1" 502 150 2025-12-18 00:11:58,713 - lava_callback - ERROR - Error processing callback for node 6940a14c575bc060ce162e21: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a14c575bc060ce162e21 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a14c575bc060ce162e21 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a14c575bc060ce162e21 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:11:58,715 - lava_callback - INFO - Starting async job submit for node 6940a172575bc060ce162e77 2025-12-18 00:11:58,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:11:58,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a172575bc060ce162e77 HTTP/1.1" 502 150 2025-12-18 00:11:58,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a172575bc060ce162e77'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:11:58,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a172575bc060ce162e77 2025-12-18 00:11:58,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a172575bc060ce162e77 HTTP/1.1" 502 150 2025-12-18 00:11:58,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a172575bc060ce162e77'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:00,643 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a16e575bc060ce162e69 2025-12-18 00:12:00,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a16e575bc060ce162e69 HTTP/1.1" 502 150 2025-12-18 00:12:00,645 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a16e575bc060ce162e69'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:00,674 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a160575bc060ce162e53 2025-12-18 00:12:00,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a160575bc060ce162e53 HTTP/1.1" 502 150 2025-12-18 00:12:00,675 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a160575bc060ce162e53'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:00,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a172575bc060ce162e77 2025-12-18 00:12:00,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a172575bc060ce162e77 HTTP/1.1" 502 150 2025-12-18 00:12:00,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a172575bc060ce162e77'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:00,778 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17a575bc060ce162e90 2025-12-18 00:12:00,779 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a170575bc060ce162e70 2025-12-18 00:12:00,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17a575bc060ce162e90 HTTP/1.1" 502 150 2025-12-18 00:12:00,779 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17a575bc060ce162e90'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:00,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a170575bc060ce162e70 HTTP/1.1" 502 150 2025-12-18 00:12:00,780 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a170575bc060ce162e70'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:00,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a162575bc060ce162e56 2025-12-18 00:12:00,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a162575bc060ce162e56 HTTP/1.1" 502 150 2025-12-18 00:12:00,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a162575bc060ce162e56'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:01,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a164575bc060ce162e59 2025-12-18 00:12:01,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a164575bc060ce162e59 HTTP/1.1" 502 150 2025-12-18 00:12:01,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a164575bc060ce162e59'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:01,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17c575bc060ce162e93 2025-12-18 00:12:01,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17c575bc060ce162e93 HTTP/1.1" 502 150 2025-12-18 00:12:01,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17c575bc060ce162e93'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47474 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:12:04,692 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a150575bc060ce162e36 2025-12-18 00:12:04,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a150575bc060ce162e36 HTTP/1.1" 502 150 2025-12-18 00:12:04,693 - lava_callback - ERROR - Error processing callback for node 6940a150575bc060ce162e36: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a150575bc060ce162e36 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a150575bc060ce162e36 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a150575bc060ce162e36 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:04,696 - lava_callback - INFO - Starting async job submit for node 6940a17d575bc060ce162e96 2025-12-18 00:12:04,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:04,724 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a172575bc060ce162e77 2025-12-18 00:12:04,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a172575bc060ce162e77 HTTP/1.1" 502 150 2025-12-18 00:12:04,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a172575bc060ce162e77'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:04,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17d575bc060ce162e96 HTTP/1.1" 502 150 2025-12-18 00:12:04,738 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17d575bc060ce162e96'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:04,738 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17d575bc060ce162e96 2025-12-18 00:12:04,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17d575bc060ce162e96 HTTP/1.1" 502 150 2025-12-18 00:12:04,739 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17d575bc060ce162e96'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:47488 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:12:06,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a158575bc060ce162e3b 2025-12-18 00:12:06,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a158575bc060ce162e3b HTTP/1.1" 502 150 2025-12-18 00:12:06,642 - lava_callback - ERROR - Error processing callback for node 6940a158575bc060ce162e3b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a158575bc060ce162e3b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a158575bc060ce162e3b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a158575bc060ce162e3b (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:06,645 - lava_callback - INFO - Starting async job submit for node 6940a174575bc060ce162e7e 2025-12-18 00:12:06,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:06,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a174575bc060ce162e7e HTTP/1.1" 502 150 2025-12-18 00:12:06,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a174575bc060ce162e7e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:06,659 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a174575bc060ce162e7e 2025-12-18 00:12:06,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a174575bc060ce162e7e HTTP/1.1" 502 150 2025-12-18 00:12:06,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a174575bc060ce162e7e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:06,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a16c575bc060ce162e5e 2025-12-18 00:12:06,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a16c575bc060ce162e5e HTTP/1.1" 502 150 2025-12-18 00:12:06,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a16c575bc060ce162e5e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:06,739 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17d575bc060ce162e96 2025-12-18 00:12:06,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17d575bc060ce162e96 HTTP/1.1" 502 150 2025-12-18 00:12:06,740 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17d575bc060ce162e96'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:07,843 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a15a575bc060ce162e45 2025-12-18 00:12:07,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15a575bc060ce162e45 HTTP/1.1" 502 150 2025-12-18 00:12:07,844 - lava_callback - ERROR - Error processing callback for node 6940a15a575bc060ce162e45: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a15a575bc060ce162e45 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a15a575bc060ce162e45 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a15a575bc060ce162e45 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:07,846 - lava_callback - INFO - Starting async job submit for node 6940a176575bc060ce162e81 2025-12-18 00:12:07,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:07,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a176575bc060ce162e81 HTTP/1.1" 502 150 2025-12-18 00:12:07,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a176575bc060ce162e81'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:07,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a176575bc060ce162e81 2025-12-18 00:12:07,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a176575bc060ce162e81 HTTP/1.1" 502 150 2025-12-18 00:12:07,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a176575bc060ce162e81'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:07,918 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69414297459ff5fc64d2ca36 2025-12-18 00:12:07,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69414297459ff5fc64d2ca36 HTTP/1.1" 502 150 2025-12-18 00:12:07,919 - lava_callback - ERROR - Error processing callback for node 69414297459ff5fc64d2ca36: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69414297459ff5fc64d2ca36 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69414297459ff5fc64d2ca36 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69414297459ff5fc64d2ca36 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:07,921 - lava_callback - INFO - Starting async job submit for node 6940a178575bc060ce162e84 2025-12-18 00:12:07,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:07,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a178575bc060ce162e84 HTTP/1.1" 502 150 2025-12-18 00:12:07,928 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a178575bc060ce162e84'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:07,928 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a178575bc060ce162e84 2025-12-18 00:12:07,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a178575bc060ce162e84 HTTP/1.1" 502 150 2025-12-18 00:12:07,929 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a178575bc060ce162e84'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:08,645 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a16e575bc060ce162e69 2025-12-18 00:12:08,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a16e575bc060ce162e69 HTTP/1.1" 502 150 2025-12-18 00:12:08,646 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a16e575bc060ce162e69'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:08,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a174575bc060ce162e7e 2025-12-18 00:12:08,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a174575bc060ce162e7e HTTP/1.1" 502 150 2025-12-18 00:12:08,661 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a174575bc060ce162e7e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:08,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17a575bc060ce162e90 2025-12-18 00:12:08,780 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a170575bc060ce162e70 2025-12-18 00:12:08,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17a575bc060ce162e90 HTTP/1.1" 502 150 2025-12-18 00:12:08,781 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17a575bc060ce162e90'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:08,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a170575bc060ce162e70 HTTP/1.1" 502 150 2025-12-18 00:12:08,782 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a170575bc060ce162e70'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:09,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a176575bc060ce162e81 2025-12-18 00:12:09,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a176575bc060ce162e81 HTTP/1.1" 502 150 2025-12-18 00:12:09,855 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a176575bc060ce162e81'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:09,929 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a178575bc060ce162e84 2025-12-18 00:12:09,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a178575bc060ce162e84 HTTP/1.1" 502 150 2025-12-18 00:12:09,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17c575bc060ce162e93 2025-12-18 00:12:09,930 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a178575bc060ce162e84'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:09,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17c575bc060ce162e93 HTTP/1.1" 502 150 2025-12-18 00:12:09,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17c575bc060ce162e93'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:10,740 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17d575bc060ce162e96 2025-12-18 00:12:10,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17d575bc060ce162e96 HTTP/1.1" 502 150 2025-12-18 00:12:10,741 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17d575bc060ce162e96'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:10,822 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a15c575bc060ce162e4d 2025-12-18 00:12:10,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15c575bc060ce162e4d HTTP/1.1" 502 150 2025-12-18 00:12:10,824 - lava_callback - ERROR - Error processing callback for node 6940a15c575bc060ce162e4d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a15c575bc060ce162e4d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a15c575bc060ce162e4d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a15c575bc060ce162e4d (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:10,825 - lava_callback - INFO - Starting async job submit for node 6940a17e575bc060ce162e99 2025-12-18 00:12:10,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:10,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17e575bc060ce162e99 HTTP/1.1" 502 150 2025-12-18 00:12:10,832 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17e575bc060ce162e99'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:10,832 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17e575bc060ce162e99 2025-12-18 00:12:10,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17e575bc060ce162e99 HTTP/1.1" 502 150 2025-12-18 00:12:10,833 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17e575bc060ce162e99'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:12,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a15e575bc060ce162e50 2025-12-18 00:12:12,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a15e575bc060ce162e50 HTTP/1.1" 502 150 2025-12-18 00:12:12,490 - lava_callback - ERROR - Error processing callback for node 6940a15e575bc060ce162e50: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a15e575bc060ce162e50 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a15e575bc060ce162e50 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a15e575bc060ce162e50 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:12,492 - lava_callback - INFO - Starting async job submit for node 69409dc7575bc060ce1628c8 2025-12-18 00:12:12,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:12,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc7575bc060ce1628c8 HTTP/1.1" 502 150 2025-12-18 00:12:12,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc7575bc060ce1628c8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:12,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc7575bc060ce1628c8 2025-12-18 00:12:12,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc7575bc060ce1628c8 HTTP/1.1" 502 150 2025-12-18 00:12:12,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc7575bc060ce1628c8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:12,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a174575bc060ce162e7e 2025-12-18 00:12:12,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a174575bc060ce162e7e HTTP/1.1" 502 150 2025-12-18 00:12:12,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a174575bc060ce162e7e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:12,726 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a172575bc060ce162e77 2025-12-18 00:12:12,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a172575bc060ce162e77 HTTP/1.1" 502 150 2025-12-18 00:12:12,727 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a172575bc060ce162e77'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:12,833 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17e575bc060ce162e99 2025-12-18 00:12:12,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17e575bc060ce162e99 HTTP/1.1" 502 150 2025-12-18 00:12:12,834 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17e575bc060ce162e99'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:13,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a149575bc060ce162e1c HTTP/1.1" 502 150 2025-12-18 00:12:13,104 - lava_callback - ERROR - Error processing callback for node 6940a149575bc060ce162e1c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a149575bc060ce162e1c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a149575bc060ce162e1c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a149575bc060ce162e1c (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:13,106 - lava_callback - INFO - Starting async job submit for node 6940a180575bc060ce162e9e 2025-12-18 00:12:13,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:13,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a180575bc060ce162e9e HTTP/1.1" 502 150 2025-12-18 00:12:13,113 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a180575bc060ce162e9e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:13,113 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a180575bc060ce162e9e 2025-12-18 00:12:13,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a180575bc060ce162e9e HTTP/1.1" 502 150 2025-12-18 00:12:13,114 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a180575bc060ce162e9e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:13,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a176575bc060ce162e81 2025-12-18 00:12:13,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a176575bc060ce162e81 HTTP/1.1" 502 150 2025-12-18 00:12:13,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a176575bc060ce162e81'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:13,930 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a178575bc060ce162e84 2025-12-18 00:12:13,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a178575bc060ce162e84 HTTP/1.1" 502 150 2025-12-18 00:12:13,931 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a178575bc060ce162e84'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:14,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc7575bc060ce1628c8 2025-12-18 00:12:14,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc7575bc060ce1628c8 HTTP/1.1" 502 150 2025-12-18 00:12:14,526 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc7575bc060ce1628c8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:15,114 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a180575bc060ce162e9e 2025-12-18 00:12:15,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a180575bc060ce162e9e HTTP/1.1" 502 150 2025-12-18 00:12:15,116 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a180575bc060ce162e9e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:45000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45012 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:12:16,676 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a160575bc060ce162e53 2025-12-18 00:12:16,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a160575bc060ce162e53 HTTP/1.1" 502 150 2025-12-18 00:12:16,677 - lava_callback - ERROR - Error processing callback for node 6940a160575bc060ce162e53: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a160575bc060ce162e53 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a160575bc060ce162e53 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a160575bc060ce162e53 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:16,679 - lava_callback - INFO - Starting async job submit for node 6940a182575bc060ce162ea9 2025-12-18 00:12:16,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:16,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a182575bc060ce162ea9 HTTP/1.1" 502 150 2025-12-18 00:12:16,687 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a182575bc060ce162ea9'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:16,687 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a182575bc060ce162ea9 2025-12-18 00:12:16,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a182575bc060ce162ea9 HTTP/1.1" 502 150 2025-12-18 00:12:16,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a182575bc060ce162ea9'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:16,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a162575bc060ce162e56 2025-12-18 00:12:16,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a162575bc060ce162e56 HTTP/1.1" 502 150 2025-12-18 00:12:16,782 - lava_callback - ERROR - Error processing callback for node 6940a162575bc060ce162e56: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a162575bc060ce162e56 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a162575bc060ce162e56 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a162575bc060ce162e56 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:16,784 - lava_callback - INFO - Starting async job submit for node 6940a184575bc060ce162eae 2025-12-18 00:12:16,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:16,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a184575bc060ce162eae HTTP/1.1" 502 150 2025-12-18 00:12:16,790 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a184575bc060ce162eae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:16,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a184575bc060ce162eae 2025-12-18 00:12:16,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a184575bc060ce162eae HTTP/1.1" 502 150 2025-12-18 00:12:16,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a184575bc060ce162eae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:16,834 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17e575bc060ce162e99 2025-12-18 00:12:16,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17e575bc060ce162e99 HTTP/1.1" 502 150 2025-12-18 00:12:16,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17e575bc060ce162e99'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:17,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a164575bc060ce162e59 2025-12-18 00:12:17,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a164575bc060ce162e59 HTTP/1.1" 502 150 2025-12-18 00:12:17,490 - lava_callback - ERROR - Error processing callback for node 6940a164575bc060ce162e59: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a164575bc060ce162e59 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a164575bc060ce162e59 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a164575bc060ce162e59 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:17,492 - lava_callback - INFO - Starting async job submit for node 6940a186575bc060ce162eb1 2025-12-18 00:12:17,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:17,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a186575bc060ce162eb1 HTTP/1.1" 502 150 2025-12-18 00:12:17,499 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a186575bc060ce162eb1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:17,499 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a186575bc060ce162eb1 2025-12-18 00:12:17,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a186575bc060ce162eb1 HTTP/1.1" 502 150 2025-12-18 00:12:17,500 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a186575bc060ce162eb1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:18,526 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc7575bc060ce1628c8 2025-12-18 00:12:18,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc7575bc060ce1628c8 HTTP/1.1" 502 150 2025-12-18 00:12:18,532 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc7575bc060ce1628c8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:18,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a182575bc060ce162ea9 2025-12-18 00:12:18,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a182575bc060ce162ea9 HTTP/1.1" 502 150 2025-12-18 00:12:18,690 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a182575bc060ce162ea9'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:18,741 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17d575bc060ce162e96 2025-12-18 00:12:18,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17d575bc060ce162e96 HTTP/1.1" 502 150 2025-12-18 00:12:18,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17d575bc060ce162e96'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:18,792 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a184575bc060ce162eae 2025-12-18 00:12:18,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a184575bc060ce162eae HTTP/1.1" 502 150 2025-12-18 00:12:18,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a184575bc060ce162eae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:19,116 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a180575bc060ce162e9e 2025-12-18 00:12:19,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a180575bc060ce162e9e HTTP/1.1" 502 150 2025-12-18 00:12:19,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a180575bc060ce162e9e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:19,500 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a186575bc060ce162eb1 2025-12-18 00:12:19,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a186575bc060ce162eb1 HTTP/1.1" 502 150 2025-12-18 00:12:19,501 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a186575bc060ce162eb1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:20,663 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a174575bc060ce162e7e 2025-12-18 00:12:20,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a174575bc060ce162e7e HTTP/1.1" 502 150 2025-12-18 00:12:20,664 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a174575bc060ce162e7e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:21,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a176575bc060ce162e81 2025-12-18 00:12:21,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a176575bc060ce162e81 HTTP/1.1" 502 150 2025-12-18 00:12:21,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a176575bc060ce162e81'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:21,932 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a178575bc060ce162e84 2025-12-18 00:12:21,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a178575bc060ce162e84 HTTP/1.1" 502 150 2025-12-18 00:12:21,933 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a178575bc060ce162e84'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:22,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a16c575bc060ce162e5e 2025-12-18 00:12:22,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a16c575bc060ce162e5e HTTP/1.1" 502 150 2025-12-18 00:12:22,663 - lava_callback - ERROR - Error processing callback for node 6940a16c575bc060ce162e5e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a16c575bc060ce162e5e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a16c575bc060ce162e5e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a16c575bc060ce162e5e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:22,666 - lava_callback - INFO - Starting async job submit for node 6940a188575bc060ce162eb4 2025-12-18 00:12:22,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:22,690 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a182575bc060ce162ea9 2025-12-18 00:12:22,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a182575bc060ce162ea9 HTTP/1.1" 502 150 2025-12-18 00:12:22,692 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a182575bc060ce162ea9'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:22,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a188575bc060ce162eb4 HTTP/1.1" 502 150 2025-12-18 00:12:22,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a188575bc060ce162eb4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:22,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a188575bc060ce162eb4 2025-12-18 00:12:22,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a188575bc060ce162eb4 HTTP/1.1" 502 150 2025-12-18 00:12:22,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a188575bc060ce162eb4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:22,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a184575bc060ce162eae 2025-12-18 00:12:22,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a184575bc060ce162eae HTTP/1.1" 502 150 2025-12-18 00:12:22,794 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a184575bc060ce162eae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:23,502 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a186575bc060ce162eb1 2025-12-18 00:12:23,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a186575bc060ce162eb1 HTTP/1.1" 502 150 2025-12-18 00:12:23,503 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a186575bc060ce162eb1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:24,647 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a16e575bc060ce162e69 2025-12-18 00:12:24,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a188575bc060ce162eb4 2025-12-18 00:12:24,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a188575bc060ce162eb4 HTTP/1.1" 502 150 2025-12-18 00:12:24,699 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a188575bc060ce162eb4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:24,781 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17a575bc060ce162e90 2025-12-18 00:12:24,782 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a170575bc060ce162e70 2025-12-18 00:12:24,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17a575bc060ce162e90 HTTP/1.1" 502 150 2025-12-18 00:12:24,783 - lava_callback - ERROR - Error processing callback for node 6940a17a575bc060ce162e90: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a17a575bc060ce162e90 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a17a575bc060ce162e90 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a17a575bc060ce162e90 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:24,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a170575bc060ce162e70 HTTP/1.1" 502 150 2025-12-18 00:12:24,786 - lava_callback - ERROR - Error processing callback for node 6940a170575bc060ce162e70: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a170575bc060ce162e70 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a170575bc060ce162e70 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a170575bc060ce162e70 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:24,789 - lava_callback - INFO - Starting async job submit for node 6940a18a575bc060ce162eb7 2025-12-18 00:12:24,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:24,790 - lava_callback - INFO - Starting async job submit for node 6940a194575bc060ce162ebc 2025-12-18 00:12:24,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:24,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a194575bc060ce162ebc HTTP/1.1" 502 150 2025-12-18 00:12:24,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a194575bc060ce162ebc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:24,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a194575bc060ce162ebc 2025-12-18 00:12:24,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a194575bc060ce162ebc HTTP/1.1" 502 150 2025-12-18 00:12:24,800 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a194575bc060ce162ebc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:24,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a18a575bc060ce162eb7 HTTP/1.1" 502 150 2025-12-18 00:12:24,813 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a18a575bc060ce162eb7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:24,813 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a18a575bc060ce162eb7 2025-12-18 00:12:24,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a18a575bc060ce162eb7 HTTP/1.1" 502 150 2025-12-18 00:12:24,814 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a18a575bc060ce162eb7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:24,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17e575bc060ce162e99 2025-12-18 00:12:24,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17e575bc060ce162e99 HTTP/1.1" 502 150 2025-12-18 00:12:24,842 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a17e575bc060ce162e99'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:25,931 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17c575bc060ce162e93 2025-12-18 00:12:25,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17c575bc060ce162e93 HTTP/1.1" 502 150 2025-12-18 00:12:25,932 - lava_callback - ERROR - Error processing callback for node 6940a17c575bc060ce162e93: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a17c575bc060ce162e93 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a17c575bc060ce162e93 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a17c575bc060ce162e93 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:25,935 - lava_callback - INFO - Starting async job submit for node 6940a196575bc060ce162ecb 2025-12-18 00:12:25,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:25,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a196575bc060ce162ecb HTTP/1.1" 502 150 2025-12-18 00:12:25,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a196575bc060ce162ecb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:25,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a196575bc060ce162ecb 2025-12-18 00:12:25,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a196575bc060ce162ecb HTTP/1.1" 502 150 2025-12-18 00:12:25,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a196575bc060ce162ecb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:40442 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:12:26,532 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc7575bc060ce1628c8 2025-12-18 00:12:26,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc7575bc060ce1628c8 HTTP/1.1" 502 150 2025-12-18 00:12:26,533 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409dc7575bc060ce1628c8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:26,800 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a194575bc060ce162ebc 2025-12-18 00:12:26,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a194575bc060ce162ebc HTTP/1.1" 502 150 2025-12-18 00:12:26,802 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a194575bc060ce162ebc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:26,815 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a18a575bc060ce162eb7 2025-12-18 00:12:26,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a18a575bc060ce162eb7 HTTP/1.1" 502 150 2025-12-18 00:12:26,816 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a18a575bc060ce162eb7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:27,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a180575bc060ce162e9e 2025-12-18 00:12:27,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a180575bc060ce162e9e HTTP/1.1" 502 150 2025-12-18 00:12:27,119 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a180575bc060ce162e9e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:27,945 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a196575bc060ce162ecb 2025-12-18 00:12:27,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a196575bc060ce162ecb HTTP/1.1" 502 150 2025-12-18 00:12:27,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a196575bc060ce162ecb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:28,699 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a188575bc060ce162eb4 2025-12-18 00:12:28,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a188575bc060ce162eb4 HTTP/1.1" 502 150 2025-12-18 00:12:28,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a188575bc060ce162eb4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:28,727 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a172575bc060ce162e77 2025-12-18 00:12:28,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a172575bc060ce162e77 HTTP/1.1" 502 150 2025-12-18 00:12:28,735 - lava_callback - ERROR - Error processing callback for node 6940a172575bc060ce162e77: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a172575bc060ce162e77 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a172575bc060ce162e77 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a172575bc060ce162e77 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:28,737 - lava_callback - INFO - Starting async job submit for node 6940a198575bc060ce162ece 2025-12-18 00:12:28,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:28,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a198575bc060ce162ece HTTP/1.1" 502 150 2025-12-18 00:12:28,767 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a198575bc060ce162ece'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:28,767 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a198575bc060ce162ece 2025-12-18 00:12:28,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a198575bc060ce162ece HTTP/1.1" 502 150 2025-12-18 00:12:28,768 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a198575bc060ce162ece'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:30,693 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a182575bc060ce162ea9 2025-12-18 00:12:30,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a182575bc060ce162ea9 HTTP/1.1" 502 150 2025-12-18 00:12:30,695 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a182575bc060ce162ea9'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:30,769 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a198575bc060ce162ece 2025-12-18 00:12:30,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a198575bc060ce162ece HTTP/1.1" 502 150 2025-12-18 00:12:30,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a198575bc060ce162ece'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:30,794 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a184575bc060ce162eae 2025-12-18 00:12:30,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a184575bc060ce162eae HTTP/1.1" 502 150 2025-12-18 00:12:30,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a184575bc060ce162eae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:30,802 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a194575bc060ce162ebc 2025-12-18 00:12:30,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a194575bc060ce162ebc HTTP/1.1" 502 150 2025-12-18 00:12:30,803 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a194575bc060ce162ebc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:30,816 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a18a575bc060ce162eb7 2025-12-18 00:12:30,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a18a575bc060ce162eb7 HTTP/1.1" 502 150 2025-12-18 00:12:30,817 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a18a575bc060ce162eb7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:31,504 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a186575bc060ce162eb1 2025-12-18 00:12:31,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a186575bc060ce162eb1 HTTP/1.1" 502 150 2025-12-18 00:12:31,505 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a186575bc060ce162eb1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:31,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a196575bc060ce162ecb 2025-12-18 00:12:31,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a196575bc060ce162ecb HTTP/1.1" 502 150 2025-12-18 00:12:31,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a196575bc060ce162ecb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:33,863 - lava_callback - INFO - Received callback for node 6940a3ea1e3c24e956819be7 INFO: 172.21.0.1:48878 - "POST /node/6940a3ea1e3c24e956819be7 HTTP/1.0" 202 Accepted 2025-12-18 00:12:34,551 - lava_callback - INFO - Received callback for node 6940a3ec1e3c24e956819bf4 INFO: 172.21.0.1:48890 - "POST /node/6940a3ec1e3c24e956819bf4 HTTP/1.0" 202 Accepted 2025-12-18 00:12:34,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17d575bc060ce162e96 2025-12-18 00:12:34,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17d575bc060ce162e96 HTTP/1.1" 502 150 2025-12-18 00:12:34,744 - lava_callback - ERROR - Error processing callback for node 6940a17d575bc060ce162e96: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a17d575bc060ce162e96 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a17d575bc060ce162e96 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a17d575bc060ce162e96 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:34,746 - lava_callback - INFO - Starting async job submit for node 6940a19a575bc060ce162ed1 2025-12-18 00:12:34,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:34,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a19a575bc060ce162ed1 HTTP/1.1" 502 150 2025-12-18 00:12:34,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a19a575bc060ce162ed1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:34,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a19a575bc060ce162ed1 2025-12-18 00:12:34,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a19a575bc060ce162ed1 HTTP/1.1" 502 150 2025-12-18 00:12:34,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a19a575bc060ce162ed1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:34,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a198575bc060ce162ece 2025-12-18 00:12:34,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a198575bc060ce162ece HTTP/1.1" 502 150 2025-12-18 00:12:34,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a198575bc060ce162ece'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:35,228 - lava_callback - INFO - Received callback for node 6940a3ee1e3c24e956819bf7 INFO: 172.21.0.1:48906 - "POST /node/6940a3ee1e3c24e956819bf7 HTTP/1.0" 202 Accepted INFO: 172.21.0.1:48908 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:12:36,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a174575bc060ce162e7e 2025-12-18 00:12:36,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a174575bc060ce162e7e HTTP/1.1" 502 150 2025-12-18 00:12:36,666 - lava_callback - ERROR - Error processing callback for node 6940a174575bc060ce162e7e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a174575bc060ce162e7e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a174575bc060ce162e7e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a174575bc060ce162e7e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:36,668 - lava_callback - INFO - Starting async job submit for node 69409d46575bc060ce1627af 2025-12-18 00:12:36,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:36,704 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a188575bc060ce162eb4 2025-12-18 00:12:36,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a188575bc060ce162eb4 HTTP/1.1" 502 150 2025-12-18 00:12:36,705 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a188575bc060ce162eb4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:36,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d46575bc060ce1627af HTTP/1.1" 502 150 2025-12-18 00:12:36,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d46575bc060ce1627af'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:36,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d46575bc060ce1627af 2025-12-18 00:12:36,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d46575bc060ce1627af HTTP/1.1" 502 150 2025-12-18 00:12:36,711 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d46575bc060ce1627af'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:36,756 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a19a575bc060ce162ed1 2025-12-18 00:12:36,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a19a575bc060ce162ed1 HTTP/1.1" 502 150 2025-12-18 00:12:36,756 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a19a575bc060ce162ed1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:37,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a176575bc060ce162e81 2025-12-18 00:12:37,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a176575bc060ce162e81 HTTP/1.1" 502 150 2025-12-18 00:12:37,861 - lava_callback - ERROR - Error processing callback for node 6940a176575bc060ce162e81: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a176575bc060ce162e81 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a176575bc060ce162e81 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a176575bc060ce162e81 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:37,864 - lava_callback - INFO - Starting async job submit for node 6940a1a0575bc060ce162ed6 2025-12-18 00:12:37,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:37,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1a0575bc060ce162ed6 HTTP/1.1" 502 150 2025-12-18 00:12:37,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1a0575bc060ce162ed6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:37,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1a0575bc060ce162ed6 2025-12-18 00:12:37,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1a0575bc060ce162ed6 HTTP/1.1" 502 150 2025-12-18 00:12:37,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1a0575bc060ce162ed6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:37,933 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a178575bc060ce162e84 2025-12-18 00:12:37,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a178575bc060ce162e84 HTTP/1.1" 502 150 2025-12-18 00:12:37,934 - lava_callback - ERROR - Error processing callback for node 6940a178575bc060ce162e84: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a178575bc060ce162e84 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a178575bc060ce162e84 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a178575bc060ce162e84 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:37,936 - lava_callback - INFO - Starting async job submit for node 6940a1b9575bc060ce162eec 2025-12-18 00:12:37,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:37,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1b9575bc060ce162eec HTTP/1.1" 502 150 2025-12-18 00:12:37,973 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1b9575bc060ce162eec'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:37,973 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1b9575bc060ce162eec 2025-12-18 00:12:37,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1b9575bc060ce162eec HTTP/1.1" 502 150 2025-12-18 00:12:37,974 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1b9575bc060ce162eec'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:38,711 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d46575bc060ce1627af 2025-12-18 00:12:38,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d46575bc060ce1627af HTTP/1.1" 502 150 2025-12-18 00:12:38,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d46575bc060ce1627af'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:38,803 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a194575bc060ce162ebc 2025-12-18 00:12:38,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a194575bc060ce162ebc HTTP/1.1" 502 150 2025-12-18 00:12:38,805 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a194575bc060ce162ebc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:38,817 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a18a575bc060ce162eb7 2025-12-18 00:12:38,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a18a575bc060ce162eb7 HTTP/1.1" 502 150 2025-12-18 00:12:38,818 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a18a575bc060ce162eb7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:39,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1a0575bc060ce162ed6 2025-12-18 00:12:39,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1a0575bc060ce162ed6 HTTP/1.1" 502 150 2025-12-18 00:12:39,874 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1a0575bc060ce162ed6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:39,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a196575bc060ce162ecb 2025-12-18 00:12:39,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a196575bc060ce162ecb HTTP/1.1" 502 150 2025-12-18 00:12:39,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a196575bc060ce162ecb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:39,974 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1b9575bc060ce162eec 2025-12-18 00:12:39,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1b9575bc060ce162eec HTTP/1.1" 502 150 2025-12-18 00:12:39,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1b9575bc060ce162eec'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:39,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a16e575bc060ce162e69 HTTP/1.1" 502 150 2025-12-18 00:12:39,984 - lava_callback - ERROR - Error processing callback for node 6940a16e575bc060ce162e69: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a16e575bc060ce162e69 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a16e575bc060ce162e69 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a16e575bc060ce162e69 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:39,986 - lava_callback - INFO - Starting async job submit for node 6940a1bd575bc060ce162efa 2025-12-18 00:12:39,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:39,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1bd575bc060ce162efa HTTP/1.1" 502 150 2025-12-18 00:12:39,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1bd575bc060ce162efa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:39,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1bd575bc060ce162efa 2025-12-18 00:12:39,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1bd575bc060ce162efa HTTP/1.1" 502 150 2025-12-18 00:12:39,995 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1bd575bc060ce162efa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:40,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a19a575bc060ce162ed1 2025-12-18 00:12:40,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a19a575bc060ce162ed1 HTTP/1.1" 502 150 2025-12-18 00:12:40,758 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a19a575bc060ce162ed1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:40,842 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a17e575bc060ce162e99 2025-12-18 00:12:40,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a17e575bc060ce162e99 HTTP/1.1" 502 150 2025-12-18 00:12:40,843 - lava_callback - ERROR - Error processing callback for node 6940a17e575bc060ce162e99: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a17e575bc060ce162e99 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a17e575bc060ce162e99 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a17e575bc060ce162e99 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:40,846 - lava_callback - INFO - Starting async job submit for node 6940a1c0575bc060ce162f03 2025-12-18 00:12:40,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:40,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c0575bc060ce162f03 HTTP/1.1" 502 150 2025-12-18 00:12:40,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c0575bc060ce162f03'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:40,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c0575bc060ce162f03 2025-12-18 00:12:40,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c0575bc060ce162f03 HTTP/1.1" 502 150 2025-12-18 00:12:40,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c0575bc060ce162f03'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:41,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1bd575bc060ce162efa 2025-12-18 00:12:41,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1bd575bc060ce162efa HTTP/1.1" 502 150 2025-12-18 00:12:41,997 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1bd575bc060ce162efa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:42,533 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409dc7575bc060ce1628c8 2025-12-18 00:12:42,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409dc7575bc060ce1628c8 HTTP/1.1" 502 150 2025-12-18 00:12:42,535 - lava_callback - ERROR - Error processing callback for node 69409dc7575bc060ce1628c8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dc7575bc060ce1628c8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dc7575bc060ce1628c8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409dc7575bc060ce1628c8 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:42,538 - lava_callback - INFO - Starting async job submit for node 6940a1c2575bc060ce162f06 2025-12-18 00:12:42,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:42,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c2575bc060ce162f06 HTTP/1.1" 502 150 2025-12-18 00:12:42,548 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c2575bc060ce162f06'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:42,548 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c2575bc060ce162f06 2025-12-18 00:12:42,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c2575bc060ce162f06 HTTP/1.1" 502 150 2025-12-18 00:12:42,549 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c2575bc060ce162f06'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:42,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d46575bc060ce1627af 2025-12-18 00:12:42,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d46575bc060ce1627af HTTP/1.1" 502 150 2025-12-18 00:12:42,714 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d46575bc060ce1627af'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:42,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a198575bc060ce162ece 2025-12-18 00:12:42,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a198575bc060ce162ece HTTP/1.1" 502 150 2025-12-18 00:12:42,775 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a198575bc060ce162ece'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:42,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c0575bc060ce162f03 2025-12-18 00:12:42,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c0575bc060ce162f03 HTTP/1.1" 502 150 2025-12-18 00:12:42,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c0575bc060ce162f03'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:43,119 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a180575bc060ce162e9e 2025-12-18 00:12:43,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a180575bc060ce162e9e HTTP/1.1" 502 150 2025-12-18 00:12:43,121 - lava_callback - ERROR - Error processing callback for node 6940a180575bc060ce162e9e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a180575bc060ce162e9e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a180575bc060ce162e9e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a180575bc060ce162e9e (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:43,123 - lava_callback - INFO - Starting async job submit for node 6940adcd1e3c24e95681aa13 2025-12-18 00:12:43,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:43,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940adcd1e3c24e95681aa13 HTTP/1.1" 502 150 2025-12-18 00:12:43,259 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940adcd1e3c24e95681aa13'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:43,259 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940adcd1e3c24e95681aa13 2025-12-18 00:12:43,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940adcd1e3c24e95681aa13 HTTP/1.1" 502 150 2025-12-18 00:12:43,260 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940adcd1e3c24e95681aa13'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:43,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1a0575bc060ce162ed6 2025-12-18 00:12:43,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1a0575bc060ce162ed6 HTTP/1.1" 502 150 2025-12-18 00:12:43,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1a0575bc060ce162ed6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:43,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1b9575bc060ce162eec 2025-12-18 00:12:43,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1b9575bc060ce162eec HTTP/1.1" 502 150 2025-12-18 00:12:43,977 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1b9575bc060ce162eec'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:44,549 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c2575bc060ce162f06 2025-12-18 00:12:44,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c2575bc060ce162f06 HTTP/1.1" 502 150 2025-12-18 00:12:44,554 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c2575bc060ce162f06'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:45,260 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940adcd1e3c24e95681aa13 2025-12-18 00:12:45,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940adcd1e3c24e95681aa13 HTTP/1.1" 502 150 2025-12-18 00:12:45,262 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940adcd1e3c24e95681aa13'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:45,997 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1bd575bc060ce162efa 2025-12-18 00:12:45,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1bd575bc060ce162efa HTTP/1.1" 502 150 2025-12-18 00:12:45,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1bd575bc060ce162efa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:55014 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:12:46,695 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a182575bc060ce162ea9 2025-12-18 00:12:46,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a182575bc060ce162ea9 HTTP/1.1" 502 150 2025-12-18 00:12:46,696 - lava_callback - ERROR - Error processing callback for node 6940a182575bc060ce162ea9: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a182575bc060ce162ea9 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a182575bc060ce162ea9 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a182575bc060ce162ea9 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:46,699 - lava_callback - INFO - Starting async job submit for node 6940a1c4575bc060ce162f09 2025-12-18 00:12:46,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:46,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c4575bc060ce162f09 HTTP/1.1" 502 150 2025-12-18 00:12:46,707 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c4575bc060ce162f09'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:46,707 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c4575bc060ce162f09 2025-12-18 00:12:46,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c4575bc060ce162f09 HTTP/1.1" 502 150 2025-12-18 00:12:46,708 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c4575bc060ce162f09'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:46,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a184575bc060ce162eae 2025-12-18 00:12:46,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a184575bc060ce162eae HTTP/1.1" 502 150 2025-12-18 00:12:46,797 - lava_callback - ERROR - Error processing callback for node 6940a184575bc060ce162eae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a184575bc060ce162eae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a184575bc060ce162eae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a184575bc060ce162eae (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:46,799 - lava_callback - INFO - Starting async job submit for node 6940a1ce575bc060ce162f1b 2025-12-18 00:12:46,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:46,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1ce575bc060ce162f1b HTTP/1.1" 502 150 2025-12-18 00:12:46,807 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1ce575bc060ce162f1b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:46,807 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1ce575bc060ce162f1b 2025-12-18 00:12:46,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1ce575bc060ce162f1b HTTP/1.1" 502 150 2025-12-18 00:12:46,808 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1ce575bc060ce162f1b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:46,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c0575bc060ce162f03 2025-12-18 00:12:46,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c0575bc060ce162f03 HTTP/1.1" 502 150 2025-12-18 00:12:46,862 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c0575bc060ce162f03'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:47,506 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a186575bc060ce162eb1 2025-12-18 00:12:47,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a186575bc060ce162eb1 HTTP/1.1" 502 150 2025-12-18 00:12:47,507 - lava_callback - ERROR - Error processing callback for node 6940a186575bc060ce162eb1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a186575bc060ce162eb1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a186575bc060ce162eb1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a186575bc060ce162eb1 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:47,509 - lava_callback - INFO - Starting async job submit for node 6940a1ca575bc060ce162f12 2025-12-18 00:12:47,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:47,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1ca575bc060ce162f12 HTTP/1.1" 502 150 2025-12-18 00:12:47,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1ca575bc060ce162f12'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:47,517 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1ca575bc060ce162f12 2025-12-18 00:12:47,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1ca575bc060ce162f12 HTTP/1.1" 502 150 2025-12-18 00:12:47,518 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1ca575bc060ce162f12'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:48,554 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c2575bc060ce162f06 2025-12-18 00:12:48,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c2575bc060ce162f06 HTTP/1.1" 502 150 2025-12-18 00:12:48,555 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c2575bc060ce162f06'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:48,708 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c4575bc060ce162f09 2025-12-18 00:12:48,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c4575bc060ce162f09 HTTP/1.1" 502 150 2025-12-18 00:12:48,710 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c4575bc060ce162f09'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:48,758 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a19a575bc060ce162ed1 2025-12-18 00:12:48,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a19a575bc060ce162ed1 HTTP/1.1" 502 150 2025-12-18 00:12:48,760 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a19a575bc060ce162ed1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:48,808 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1ce575bc060ce162f1b 2025-12-18 00:12:48,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1ce575bc060ce162f1b HTTP/1.1" 502 150 2025-12-18 00:12:48,809 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1ce575bc060ce162f1b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:49,262 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940adcd1e3c24e95681aa13 2025-12-18 00:12:49,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940adcd1e3c24e95681aa13 HTTP/1.1" 502 150 2025-12-18 00:12:49,263 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940adcd1e3c24e95681aa13'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:49,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1ca575bc060ce162f12 2025-12-18 00:12:49,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1ca575bc060ce162f12 HTTP/1.1" 502 150 2025-12-18 00:12:49,520 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1ca575bc060ce162f12'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:50,715 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d46575bc060ce1627af 2025-12-18 00:12:50,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d46575bc060ce1627af HTTP/1.1" 502 150 2025-12-18 00:12:50,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69409d46575bc060ce1627af'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:50,880 - lava_callback - INFO - Received callback for node 6940a3f11e3c24e956819c01 INFO: 172.21.0.1:46604 - "POST /node/6940a3f11e3c24e956819c01 HTTP/1.0" 202 Accepted 2025-12-18 00:12:51,673 - lava_callback - INFO - Received callback for node 6940a3f21e3c24e956819c04 INFO: 172.21.0.1:46620 - "POST /node/6940a3f21e3c24e956819c04 HTTP/1.0" 202 Accepted 2025-12-18 00:12:51,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1a0575bc060ce162ed6 2025-12-18 00:12:51,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1a0575bc060ce162ed6 HTTP/1.1" 502 150 2025-12-18 00:12:51,882 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1a0575bc060ce162ed6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:51,977 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1b9575bc060ce162eec 2025-12-18 00:12:51,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1b9575bc060ce162eec HTTP/1.1" 502 150 2025-12-18 00:12:51,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1b9575bc060ce162eec'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:52,427 - lava_callback - INFO - Received callback for node 6940a3f41e3c24e956819c09 INFO: 172.21.0.1:46634 - "POST /node/6940a3f41e3c24e956819c09 HTTP/1.0" 202 Accepted 2025-12-18 00:12:52,705 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a188575bc060ce162eb4 2025-12-18 00:12:52,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a188575bc060ce162eb4 HTTP/1.1" 502 150 2025-12-18 00:12:52,707 - lava_callback - ERROR - Error processing callback for node 6940a188575bc060ce162eb4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a188575bc060ce162eb4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a188575bc060ce162eb4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a188575bc060ce162eb4 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:52,709 - lava_callback - INFO - Starting async job submit for node 6940a1cc575bc060ce162f15 2025-12-18 00:12:52,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:52,710 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c4575bc060ce162f09 2025-12-18 00:12:52,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c4575bc060ce162f09 HTTP/1.1" 502 150 2025-12-18 00:12:52,712 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c4575bc060ce162f09'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:52,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1cc575bc060ce162f15 HTTP/1.1" 502 150 2025-12-18 00:12:52,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1cc575bc060ce162f15'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:52,718 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1cc575bc060ce162f15 2025-12-18 00:12:52,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1cc575bc060ce162f15 HTTP/1.1" 502 150 2025-12-18 00:12:52,718 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1cc575bc060ce162f15'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:52,810 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1ce575bc060ce162f1b 2025-12-18 00:12:52,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1ce575bc060ce162f1b HTTP/1.1" 502 150 2025-12-18 00:12:52,811 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1ce575bc060ce162f1b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:53,203 - lava_callback - INFO - Received callback for node 6940a3f61e3c24e956819c0e INFO: 172.21.0.1:46642 - "POST /node/6940a3f61e3c24e956819c0e HTTP/1.0" 202 Accepted 2025-12-18 00:12:53,520 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1ca575bc060ce162f12 2025-12-18 00:12:53,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1ca575bc060ce162f12 HTTP/1.1" 502 150 2025-12-18 00:12:53,521 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1ca575bc060ce162f12'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:53,850 - lava_callback - INFO - Received callback for node 6940a3fb1e3c24e956819c18 INFO: 172.21.0.1:46654 - "POST /node/6940a3fb1e3c24e956819c18 HTTP/1.0" 202 Accepted 2025-12-18 00:12:53,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1bd575bc060ce162efa 2025-12-18 00:12:54,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1bd575bc060ce162efa HTTP/1.1" 502 150 2025-12-18 00:12:54,000 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1bd575bc060ce162efa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:54,001 - lava_callback - INFO - Received callback for node 6940a3f71e3c24e956819c15 INFO: 172.21.0.1:46660 - "POST /node/6940a3f71e3c24e956819c15 HTTP/1.0" 202 Accepted 2025-12-18 00:12:54,541 - lava_callback - INFO - Received callback for node 6940a3fd1e3c24e956819c1b INFO: 172.21.0.1:46674 - "POST /node/6940a3fd1e3c24e956819c1b HTTP/1.0" 202 Accepted 2025-12-18 00:12:54,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1cc575bc060ce162f15 2025-12-18 00:12:54,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1cc575bc060ce162f15 HTTP/1.1" 502 150 2025-12-18 00:12:54,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1cc575bc060ce162f15'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:54,805 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a194575bc060ce162ebc 2025-12-18 00:12:54,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a194575bc060ce162ebc HTTP/1.1" 502 150 2025-12-18 00:12:54,807 - lava_callback - ERROR - Error processing callback for node 6940a194575bc060ce162ebc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a194575bc060ce162ebc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a194575bc060ce162ebc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a194575bc060ce162ebc (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:54,810 - lava_callback - INFO - Starting async job submit for node 6940a1cd575bc060ce162f18 2025-12-18 00:12:54,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:54,819 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a18a575bc060ce162eb7 2025-12-18 00:12:54,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a18a575bc060ce162eb7 HTTP/1.1" 502 150 2025-12-18 00:12:54,832 - lava_callback - ERROR - Error processing callback for node 6940a18a575bc060ce162eb7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a18a575bc060ce162eb7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a18a575bc060ce162eb7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a18a575bc060ce162eb7 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:54,834 - lava_callback - INFO - Starting async job submit for node 6940a1c8575bc060ce162f0f 2025-12-18 00:12:54,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:54,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c8575bc060ce162f0f HTTP/1.1" 502 150 2025-12-18 00:12:54,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c8575bc060ce162f0f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:54,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c8575bc060ce162f0f 2025-12-18 00:12:54,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c8575bc060ce162f0f HTTP/1.1" 502 150 2025-12-18 00:12:54,847 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c8575bc060ce162f0f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:54,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1cd575bc060ce162f18 HTTP/1.1" 502 150 2025-12-18 00:12:54,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1cd575bc060ce162f18'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:54,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1cd575bc060ce162f18 2025-12-18 00:12:54,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1cd575bc060ce162f18 HTTP/1.1" 502 150 2025-12-18 00:12:54,858 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1cd575bc060ce162f18'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:54,862 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c0575bc060ce162f03 2025-12-18 00:12:54,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c0575bc060ce162f03 HTTP/1.1" 502 150 2025-12-18 00:12:54,863 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c0575bc060ce162f03'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:55,204 - lava_callback - INFO - Received callback for node 6940a3fe1e3c24e956819c1e INFO: 172.21.0.1:46684 - "POST /node/6940a3fe1e3c24e956819c1e HTTP/1.0" 202 Accepted 2025-12-18 00:12:55,892 - lava_callback - INFO - Received callback for node 6940a4001e3c24e956819c25 INFO: 172.21.0.1:46698 - "POST /node/6940a4001e3c24e956819c25 HTTP/1.0" 202 Accepted 2025-12-18 00:12:55,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a196575bc060ce162ecb 2025-12-18 00:12:55,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a196575bc060ce162ecb HTTP/1.1" 502 150 2025-12-18 00:12:55,952 - lava_callback - ERROR - Error processing callback for node 6940a196575bc060ce162ecb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a196575bc060ce162ecb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a196575bc060ce162ecb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a196575bc060ce162ecb (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:55,955 - lava_callback - INFO - Starting async job submit for node 6940a1c7575bc060ce162f0c 2025-12-18 00:12:55,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:55,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c7575bc060ce162f0c HTTP/1.1" 502 150 2025-12-18 00:12:55,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c7575bc060ce162f0c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:55,966 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c7575bc060ce162f0c 2025-12-18 00:12:55,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c7575bc060ce162f0c HTTP/1.1" 502 150 2025-12-18 00:12:55,966 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c7575bc060ce162f0c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:46708 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:12:56,556 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c2575bc060ce162f06 2025-12-18 00:12:56,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c2575bc060ce162f06 HTTP/1.1" 502 150 2025-12-18 00:12:56,557 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c2575bc060ce162f06'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:56,847 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c8575bc060ce162f0f 2025-12-18 00:12:56,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c8575bc060ce162f0f HTTP/1.1" 502 150 2025-12-18 00:12:56,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c8575bc060ce162f0f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:56,858 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1cd575bc060ce162f18 2025-12-18 00:12:56,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1cd575bc060ce162f18 HTTP/1.1" 502 150 2025-12-18 00:12:56,860 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1cd575bc060ce162f18'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:57,267 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940adcd1e3c24e95681aa13 2025-12-18 00:12:57,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940adcd1e3c24e95681aa13 HTTP/1.1" 502 150 2025-12-18 00:12:57,268 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940adcd1e3c24e95681aa13'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:57,967 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c7575bc060ce162f0c 2025-12-18 00:12:57,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c7575bc060ce162f0c HTTP/1.1" 502 150 2025-12-18 00:12:57,968 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c7575bc060ce162f0c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:58,720 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1cc575bc060ce162f15 2025-12-18 00:12:58,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1cc575bc060ce162f15 HTTP/1.1" 502 150 2025-12-18 00:12:58,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1cc575bc060ce162f15'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:58,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a198575bc060ce162ece 2025-12-18 00:12:58,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a198575bc060ce162ece HTTP/1.1" 502 150 2025-12-18 00:12:58,777 - lava_callback - ERROR - Error processing callback for node 6940a198575bc060ce162ece: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a198575bc060ce162ece (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a198575bc060ce162ece (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a198575bc060ce162ece (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:12:58,779 - lava_callback - INFO - Starting async job submit for node 6940a1d1575bc060ce162f22 2025-12-18 00:12:58,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:12:58,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1d1575bc060ce162f22 HTTP/1.1" 502 150 2025-12-18 00:12:58,795 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1d1575bc060ce162f22'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:12:58,795 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1d1575bc060ce162f22 2025-12-18 00:12:58,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1d1575bc060ce162f22 HTTP/1.1" 502 150 2025-12-18 00:12:58,796 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1d1575bc060ce162f22'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:00,712 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c4575bc060ce162f09 2025-12-18 00:13:00,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c4575bc060ce162f09 HTTP/1.1" 502 150 2025-12-18 00:13:00,713 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c4575bc060ce162f09'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:00,796 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1d1575bc060ce162f22 2025-12-18 00:13:00,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1d1575bc060ce162f22 HTTP/1.1" 502 150 2025-12-18 00:13:00,797 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1d1575bc060ce162f22'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:00,811 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1ce575bc060ce162f1b 2025-12-18 00:13:00,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1ce575bc060ce162f1b HTTP/1.1" 502 150 2025-12-18 00:13:00,812 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1ce575bc060ce162f1b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:00,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c8575bc060ce162f0f 2025-12-18 00:13:00,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c8575bc060ce162f0f HTTP/1.1" 502 150 2025-12-18 00:13:00,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c8575bc060ce162f0f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:00,860 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1cd575bc060ce162f18 2025-12-18 00:13:00,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1cd575bc060ce162f18 HTTP/1.1" 502 150 2025-12-18 00:13:00,861 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1cd575bc060ce162f18'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:01,521 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1ca575bc060ce162f12 2025-12-18 00:13:01,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1ca575bc060ce162f12 HTTP/1.1" 502 150 2025-12-18 00:13:01,523 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1ca575bc060ce162f12'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:01,968 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c7575bc060ce162f0c 2025-12-18 00:13:01,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1c7575bc060ce162f0c HTTP/1.1" 502 150 2025-12-18 00:13:01,970 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1c7575bc060ce162f0c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.21.0.1:34852 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:13:04,760 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a19a575bc060ce162ed1 2025-12-18 00:13:04,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a19a575bc060ce162ed1 HTTP/1.1" 502 150 2025-12-18 00:13:04,761 - lava_callback - ERROR - Error processing callback for node 6940a19a575bc060ce162ed1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a19a575bc060ce162ed1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a19a575bc060ce162ed1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a19a575bc060ce162ed1 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:13:04,764 - lava_callback - INFO - Starting async job submit for node 6940a1d7575bc060ce162f28 2025-12-18 00:13:04,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:13:04,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1d7575bc060ce162f28 HTTP/1.1" 502 150 2025-12-18 00:13:04,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1d7575bc060ce162f28'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:04,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1d7575bc060ce162f28 2025-12-18 00:13:04,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1d7575bc060ce162f28 HTTP/1.1" 502 150 2025-12-18 00:13:04,772 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1d7575bc060ce162f28'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:04,798 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1d1575bc060ce162f22 2025-12-18 00:13:04,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1d1575bc060ce162f22 HTTP/1.1" 502 150 2025-12-18 00:13:04,799 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1d1575bc060ce162f22'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:06,530 - lava_callback - INFO - Received callback for node 6940a3ef1e3c24e956819bfe INFO: 172.21.0.1:34858 - "POST /node/6940a3ef1e3c24e956819bfe HTTP/1.0" 202 Accepted INFO: 172.21.0.1:34870 - "GET / HTTP/1.0" 200 OK 2025-12-18 00:13:06,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69409d46575bc060ce1627af 2025-12-18 00:13:06,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69409d46575bc060ce1627af HTTP/1.1" 502 150 2025-12-18 00:13:06,719 - lava_callback - ERROR - Error processing callback for node 69409d46575bc060ce1627af: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d46575bc060ce1627af (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d46575bc060ce1627af (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69409d46575bc060ce1627af (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:13:06,721 - lava_callback - INFO - Starting async job submit for node 6940a1d5575bc060ce162f25 2025-12-18 00:13:06,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1cc575bc060ce162f15 2025-12-18 00:13:06,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:13:06,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1cc575bc060ce162f15 HTTP/1.1" 502 150 2025-12-18 00:13:06,724 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1cc575bc060ce162f15'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:06,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1d5575bc060ce162f25 HTTP/1.1" 502 150 2025-12-18 00:13:06,731 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1d5575bc060ce162f25'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:06,732 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1d5575bc060ce162f25 2025-12-18 00:13:06,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1d5575bc060ce162f25 HTTP/1.1" 502 150 2025-12-18 00:13:06,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1d5575bc060ce162f25'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:06,772 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1d7575bc060ce162f28 2025-12-18 00:13:06,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1d7575bc060ce162f28 HTTP/1.1" 502 150 2025-12-18 00:13:06,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1d7575bc060ce162f28'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:07,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1a0575bc060ce162ed6 2025-12-18 00:13:07,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1a0575bc060ce162ed6 HTTP/1.1" 502 150 2025-12-18 00:13:07,916 - lava_callback - ERROR - Error processing callback for node 6940a1a0575bc060ce162ed6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a1a0575bc060ce162ed6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a1a0575bc060ce162ed6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a1a0575bc060ce162ed6 (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:13:07,918 - lava_callback - INFO - Starting async job submit for node 6940a1d9575bc060ce162f2b 2025-12-18 00:13:07,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:13:07,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1d9575bc060ce162f2b HTTP/1.1" 502 150 2025-12-18 00:13:07,957 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1d9575bc060ce162f2b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:07,957 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1d9575bc060ce162f2b 2025-12-18 00:13:07,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1d9575bc060ce162f2b HTTP/1.1" 502 150 2025-12-18 00:13:07,958 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1d9575bc060ce162f2b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:07,979 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1b9575bc060ce162eec 2025-12-18 00:13:07,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1b9575bc060ce162eec HTTP/1.1" 502 150 2025-12-18 00:13:07,980 - lava_callback - ERROR - Error processing callback for node 6940a1b9575bc060ce162eec: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a1b9575bc060ce162eec (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a1b9575bc060ce162eec (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 691, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6940a1b9575bc060ce162eec (Caused by ResponseError('too many 502 error responses')) 2025-12-18 00:13:07,983 - lava_callback - INFO - Starting async job submit for node 6940a1db575bc060ce162f2e 2025-12-18 00:13:07,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-12-18 00:13:08,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1db575bc060ce162f2e HTTP/1.1" 502 150 2025-12-18 00:13:08,008 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1db575bc060ce162f2e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2025-12-18 00:13:08,008 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1db575bc060ce162f2e 2025-12-18 00:13:08,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6940a1db575bc060ce162f2e HTTP/1.1" 502 150 2025-12-18 00:13:08,009 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6940a1db575bc060ce162f2e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1] 2025-12-18 00:13:08,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1d5575bc060ce162f25 2025-12-18 00:13:08,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c8575bc060ce162f0f 2025-12-18 00:13:08,861 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1cd575bc060ce162f18 2025-12-18 00:13:09,958 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1d9575bc060ce162f2b 2025-12-18 00:13:09,970 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c7575bc060ce162f0c 2025-12-18 00:13:10,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1bd575bc060ce162efa 2025-12-18 00:13:10,009 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1db575bc060ce162f2e 2025-12-18 00:13:10,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1d7575bc060ce162f28 2025-12-18 00:13:10,863 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c0575bc060ce162f03 2025-12-18 00:13:12,557 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c2575bc060ce162f06 2025-12-18 00:13:12,799 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1d1575bc060ce162f22 2025-12-18 00:13:13,269 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940adcd1e3c24e95681aa13 2025-12-18 00:13:16,713 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1c4575bc060ce162f09 2025-12-18 00:13:16,812 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1ce575bc060ce162f1b 2025-12-18 00:13:17,523 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6940a1ca575bc060ce162f12